2023-11-15T02:39:33.193 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-15T02:39:33.194 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7458153 --owner scheduled_yuriw@teuthology --verbose --description fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage} --name yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi -- /home/teuthworker/archive/yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7458153/orig.config.yaml 2023-11-15T02:39:33.257 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T02:39:33.339 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7458153 branch: wip-yuri4-testing-2023-11-13-0820-pacific description: fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage} email: yweinste@redhat.com first_in_suite: false job_id: '7458153' kernel: client: branch: testing kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-yuri4-testing-2023-11-13-0820-pacific ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - Error loading MDS rank - missing journal object - Error recovering journal - error decoding table object - failed to read JournalPointer - Corrupt directory entry - Corrupt fnode header - corrupt sessionmap header - Corrupt dentry - Scrub error on inode - Metadata damage detected - MDS_READ_ONLY - force file system read-only - with standby daemon mds sha1: 620f25c1794f62207424fc9319427943fb8d1fae ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 620f25c1794f62207424fc9319427943fb8d1fae selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri4-testing-2023-11-13-0820-pacific sha1: 620f25c1794f62207424fc9319427943fb8d1fae owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5345 sha1: 620f25c1794f62207424fc9319427943fb8d1fae sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri4-testing-2023-11-13-0820-pacific suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_620f25c1794f62207424fc9319427943fb8d1fae/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 620f25c1794f62207424fc9319427943fb8d1fae targets: smithi121.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_damage teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 6899cd26fceddb2fec83dc1a1349394b28c8998e timestamp: 2023-11-14_20:31:57 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3590714 2023-11-15T02:39:33.340 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_620f25c1794f62207424fc9319427943fb8d1fae/qa; will attempt to use it 2023-11-15T02:39:33.341 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_620f25c1794f62207424fc9319427943fb8d1fae/qa/tasks 2023-11-15T02:39:33.341 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-15T02:39:33.343 INFO:teuthology.task.internal:Checking packages... 2023-11-15T02:39:33.368 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '620f25c1794f62207424fc9319427943fb8d1fae' 2023-11-15T02:39:33.368 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-15T02:39:33.369 INFO:teuthology.packaging:ref: None 2023-11-15T02:39:33.369 INFO:teuthology.packaging:tag: None 2023-11-15T02:39:33.369 INFO:teuthology.packaging:branch: wip-yuri4-testing-2023-11-13-0820-pacific 2023-11-15T02:39:33.369 INFO:teuthology.packaging:sha1: 620f25c1794f62207424fc9319427943fb8d1fae 2023-11-15T02:39:33.370 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-yuri4-testing-2023-11-13-0820-pacific 2023-11-15T02:39:33.514 INFO:teuthology.task.internal:Found packages for ceph version 16.2.14-170-g620f25c1-1focal 2023-11-15T02:39:33.515 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-15T02:39:33.526 INFO:teuthology.task.internal:no buildpackages task found 2023-11-15T02:39:33.527 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-15T02:39:33.597 INFO:teuthology.task.internal:Saving configuration 2023-11-15T02:39:33.615 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-15T02:39:33.634 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-15T02:39:33.659 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7458153', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-15 02:33:42.828157', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-11-15T02:39:33.683 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7458153', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-15 02:33:42.829761', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-11-15T02:39:33.683 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-15T02:39:33.693 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-15T02:39:33.694 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-15T02:39:33.694 INFO:teuthology.run_tasks:Running task console_log... 2023-11-15T02:39:33.792 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8590abbb80>, signals=[15]) 2023-11-15T02:39:33.794 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-15T02:39:33.801 INFO:teuthology.task.internal:Opening connections... 2023-11-15T02:39:33.801 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T02:39:33.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T02:39:33.856 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2023-11-15T02:39:33.857 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T02:39:33.904 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-15T02:39:33.912 DEBUG:teuthology.orchestra.run.smithi121:> uname -m 2023-11-15T02:39:33.940 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2023-11-15T02:39:33.940 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2023-11-15T02:39:33.945 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="Ubuntu" 2023-11-15T02:39:33.946 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-11-15T02:39:33.946 INFO:teuthology.orchestra.run.smithi121.stdout:ID=ubuntu 2023-11-15T02:39:33.946 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE=debian 2023-11-15T02:39:33.946 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-11-15T02:39:33.946 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="20.04" 2023-11-15T02:39:33.947 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-15T02:39:33.947 INFO:teuthology.orchestra.run.smithi121.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-15T02:39:33.947 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-15T02:39:33.947 INFO:teuthology.orchestra.run.smithi121.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-15T02:39:33.947 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_CODENAME=focal 2023-11-15T02:39:33.948 INFO:teuthology.orchestra.run.smithi121.stdout:UBUNTU_CODENAME=focal 2023-11-15T02:39:33.948 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2023-11-15T02:39:33.971 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2023-11-15T02:39:33.986 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2023-11-15T02:39:33.987 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2023-11-15T02:39:33.993 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="Ubuntu" 2023-11-15T02:39:33.993 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-11-15T02:39:33.993 INFO:teuthology.orchestra.run.smithi190.stdout:ID=ubuntu 2023-11-15T02:39:33.994 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE=debian 2023-11-15T02:39:33.994 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-11-15T02:39:33.994 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="20.04" 2023-11-15T02:39:33.994 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-15T02:39:33.995 INFO:teuthology.orchestra.run.smithi190.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-15T02:39:33.995 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-15T02:39:33.995 INFO:teuthology.orchestra.run.smithi190.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-15T02:39:33.995 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_CODENAME=focal 2023-11-15T02:39:33.995 INFO:teuthology.orchestra.run.smithi190.stdout:UBUNTU_CODENAME=focal 2023-11-15T02:39:33.996 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2023-11-15T02:39:34.020 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-15T02:39:34.032 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-15T02:39:34.073 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-15T02:39:34.073 DEBUG:teuthology.orchestra.run.smithi121:> test '!' -e /home/ubuntu/cephtest 2023-11-15T02:39:34.076 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2023-11-15T02:39:34.082 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-15T02:39:34.106 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-15T02:39:34.106 DEBUG:teuthology.orchestra.run.smithi121:> test -z $(ls -A /var/lib/ceph) 2023-11-15T02:39:34.109 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2023-11-15T02:39:34.125 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-15T02:39:34.192 INFO:teuthology.run_tasks:Running task kernel... 2023-11-15T02:39:34.208 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-11-15T02:39:34.209 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-11-15T02:39:34.210 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-11-15T02:39:34.210 DEBUG:teuthology.orchestra.run.smithi121:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-15T02:39:34.211 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-15T02:39:34.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T02:39:34.217 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=testing 2023-11-15T02:39:34.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T02:39:34.221 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=testing 2023-11-15T02:39:34.427 DEBUG:teuthology.task.kernel:sha1 for client.1 is 3b796cfe2969fb2f2d017e8146dccc1ef582e837 2023-11-15T02:39:34.427 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "3b796cfe2969fb2f2d017e8146dccc1ef582e837"... 2023-11-15T02:39:34.428 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2023-11-15T02:39:34.435 INFO:teuthology.orchestra.run.smithi190.stdout:5.4.0-124-generic 2023-11-15T02:39:34.435 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-124-generic vs 3b796cfe2969fb2f2d017e8146dccc1ef582e837 2023-11-15T02:39:34.436 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-15T02:39:34.436 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2023-11-15T02:39:34.437 DEBUG:teuthology.orchestra.run.smithi190:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-11-15T02:39:34.452 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 0: cd: /lib/firmware/updates: No such file or directory 2023-11-15T02:39:34.559 INFO:teuthology.orchestra.run.smithi190.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-11-15T02:39:34.561 DEBUG:teuthology.orchestra.run.smithi190:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-11-15T02:39:34.566 DEBUG:teuthology.task.kernel:sha1 for client.0 is 3b796cfe2969fb2f2d017e8146dccc1ef582e837 2023-11-15T02:39:34.566 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "3b796cfe2969fb2f2d017e8146dccc1ef582e837"... 2023-11-15T02:39:34.567 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2023-11-15T02:39:34.573 INFO:teuthology.orchestra.run.smithi121.stdout:5.4.0-124-generic 2023-11-15T02:39:34.574 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-124-generic vs 3b796cfe2969fb2f2d017e8146dccc1ef582e837 2023-11-15T02:39:34.574 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-15T02:39:34.574 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2023-11-15T02:39:34.575 DEBUG:teuthology.orchestra.run.smithi121:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-11-15T02:39:34.599 INFO:teuthology.orchestra.run.smithi121.stderr:bash: line 0: cd: /lib/firmware/updates: No such file or directory 2023-11-15T02:39:34.645 DEBUG:teuthology.orchestra.run.smithi190:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-11-15T02:39:34.663 DEBUG:teuthology.orchestra.run.smithi190:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-11-15T02:39:34.717 INFO:teuthology.orchestra.run.smithi121.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-11-15T02:39:34.720 DEBUG:teuthology.orchestra.run.smithi121:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-11-15T02:39:34.750 DEBUG:teuthology.orchestra.run.smithi121:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-11-15T02:39:34.811 DEBUG:teuthology.orchestra.run.smithi121:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-11-15T02:41:38.870 INFO:teuthology.orchestra.run.smithi121.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-11-15T02:41:38.870 INFO:teuthology.orchestra.run.smithi121.stderr: * [new branch] main -> origin/main 2023-11-15T02:41:38.928 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20190312 -> 20190312 2023-11-15T02:41:38.928 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20190416 -> 20190416 2023-11-15T02:41:38.929 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20190514 -> 20190514 2023-11-15T02:41:38.929 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20190618 -> 20190618 2023-11-15T02:41:38.929 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20190717 -> 20190717 2023-11-15T02:41:38.929 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20190815 -> 20190815 2023-11-15T02:41:38.930 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20190923 -> 20190923 2023-11-15T02:41:38.930 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20191022 -> 20191022 2023-11-15T02:41:38.930 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20191215 -> 20191215 2023-11-15T02:41:38.930 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20200122 -> 20200122 2023-11-15T02:41:38.930 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20200316 -> 20200316 2023-11-15T02:41:38.931 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20200421 -> 20200421 2023-11-15T02:41:38.931 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20200519 -> 20200519 2023-11-15T02:41:38.932 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20200619 -> 20200619 2023-11-15T02:41:38.933 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20200721 -> 20200721 2023-11-15T02:41:38.933 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20200817 -> 20200817 2023-11-15T02:41:38.933 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20200918 -> 20200918 2023-11-15T02:41:38.933 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20201022 -> 20201022 2023-11-15T02:41:38.933 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20201118 -> 20201118 2023-11-15T02:41:38.934 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20201218 -> 20201218 2023-11-15T02:41:38.934 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20210208 -> 20210208 2023-11-15T02:41:38.934 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20210315 -> 20210315 2023-11-15T02:41:38.934 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20210511 -> 20210511 2023-11-15T02:41:38.934 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20210716 -> 20210716 2023-11-15T02:41:38.935 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20210818 -> 20210818 2023-11-15T02:41:38.938 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20210919 -> 20210919 2023-11-15T02:41:38.938 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20211027 -> 20211027 2023-11-15T02:41:38.938 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20211216 -> 20211216 2023-11-15T02:41:38.938 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20220209 -> 20220209 2023-11-15T02:41:38.938 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20220310 -> 20220310 2023-11-15T02:41:38.939 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20220411 -> 20220411 2023-11-15T02:41:38.939 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20220509 -> 20220509 2023-11-15T02:41:38.939 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20220610 -> 20220610 2023-11-15T02:41:38.939 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20220708 -> 20220708 2023-11-15T02:41:38.940 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20220815 -> 20220815 2023-11-15T02:41:38.940 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20220913 -> 20220913 2023-11-15T02:41:38.940 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20221012 -> 20221012 2023-11-15T02:41:38.940 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20221109 -> 20221109 2023-11-15T02:41:38.940 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20221214 -> 20221214 2023-11-15T02:41:38.940 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20230117 -> 20230117 2023-11-15T02:41:38.941 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20230210 -> 20230210 2023-11-15T02:41:38.941 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20230310 -> 20230310 2023-11-15T02:41:38.941 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20230404 -> 20230404 2023-11-15T02:41:38.941 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20230515 -> 20230515 2023-11-15T02:41:38.941 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20230625 -> 20230625 2023-11-15T02:41:38.942 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20230804 -> 20230804 2023-11-15T02:41:38.942 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20230919 -> 20230919 2023-11-15T02:41:38.942 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20231030 -> 20231030 2023-11-15T02:41:38.942 INFO:teuthology.orchestra.run.smithi121.stderr: * [new tag] 20231111 -> 20231111 2023-11-15T02:41:42.801 INFO:teuthology.orchestra.run.smithi121.stderr:Updating files: 51% (1642/3198) Updating files: 52% (1663/3198) Updating files: 53% (1695/3198) Updating files: 54% (1727/3198) Updating files: 55% (1759/3198) Updating files: 56% (1791/3198) Updating files: 57% (1823/3198) Updating files: 58% (1855/3198) Updating files: 59% (1887/3198) Updating files: 60% (1919/3198) Updating files: 61% (1951/3198) Updating files: 62% (1983/3198) Updating files: 63% (2015/3198) Updating files: 64% (2047/3198) Updating files: 65% (2079/3198) Updating files: 66% (2111/3198) Updating files: 66% (2124/3198) Updating files: 67% (2143/3198) Updating files: 68% (2175/3198) Updating files: 69% (2207/3198) Updating files: 70% (2239/3198) Updating files: 71% (2271/3198) Updating files: 72% (2303/3198) Updating files: 73% (2335/3198) Updating files: 74% (2367/3198) Updating files: 75% (2399/3198) Updating files: 76% (2431/3198) Updating files: 77% (2463/3198) Updating files: 78% (2495/3198) Updating files: 79% (2527/3198) Updating files: 80% (2559/3198) Updating files: 81% (2591/3198) Updating files: 81% (2608/3198) Updating files: 82% (2623/3198) Updating files: 83% (2655/3198) Updating files: 84% (2687/3198) Updating files: 85% (2719/3198) Updating files: 86% (2751/3198) Updating files: 87% (2783/3198) Updating files: 88% (2815/3198) Updating files: 89% (2847/3198) Updating files: 90% (2879/3198) Updating files: 91% (2911/3198) Updating files: 92% (2943/3198) Updating files: 93% (2975/3198) Updating files: 94% (3007/3198) Updating files: 95% (3039/3198) Updating files: 96% (3071/3198) Updating files: 97% (3103/3198) Updating files: 98% (3135/3198) Updating files: 99% (3167/3198) Updating files: 100% (3198/3198) Updating files: 100% (3198/3198), done. 2023-11-15T02:41:42.805 INFO:teuthology.orchestra.run.smithi121.stdout:HEAD is now at b72eeb60 Merge branch 'mlimonci/robot-pr-improvements' into 'main' 2023-11-15T02:41:42.822 INFO:teuthology.task.kernel:Downloading kernel 3b796cfe2969fb2f2d017e8146dccc1ef582e837 on client.0... 2023-11-15T02:41:42.823 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=3b796cfe2969fb2f2d017e8146dccc1ef582e837 2023-11-15T02:41:43.042 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/3b796cfe2969fb2f2d017e8146dccc1ef582e837/ubuntu/focal/flavors/default/pool/main/l/linux-6.6.0-rc5-g3b796cfe2969/ 2023-11-15T02:41:43.043 DEBUG:teuthology.orchestra.run.smithi121:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-rc5-g3b796cfe2969_6.6.0-rc5-g3b796cfe2969-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://1.chacra.ceph.com/r/kernel/testing/3b796cfe2969fb2f2d017e8146dccc1ef582e837/ubuntu/focal/flavors/default/pool/main/l/linux-6.6.0-rc5-g3b796cfe2969/ --input-file=- 2023-11-15T02:41:43.046 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi121.front.sepia.ceph.com to complete... 2023-11-15T02:41:47.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15 02:41:47 URL:https://1.chacra.ceph.com/r/kernel/testing/3b796cfe2969fb2f2d017e8146dccc1ef582e837/ubuntu/focal/flavors/default/pool/main/l/linux-6.6.0-rc5-g3b796cfe2969/linux-image-6.6.0-rc5-g3b796cfe2969_6.6.0-rc5-g3b796cfe2969-1_amd64.deb [38912044/38912044] -> "/tmp/linux-image.deb" [1] 2023-11-15T02:41:47.263 INFO:teuthology.orchestra.run.smithi121.stderr:FINISHED --2023-11-15 02:41:47-- 2023-11-15T02:41:47.263 INFO:teuthology.orchestra.run.smithi121.stderr:Total wall clock time: 4.2s 2023-11-15T02:41:47.263 INFO:teuthology.orchestra.run.smithi121.stderr:Downloaded: 1 files, 37M in 4.0s (9.37 MB/s) 2023-11-15T02:41:47.264 INFO:teuthology.task.kernel:Installing kernel 3b796cfe2969fb2f2d017e8146dccc1ef582e837 on client.0... 2023-11-15T02:41:47.265 DEBUG:teuthology.orchestra.run.smithi121:> sudo dpkg -i /tmp/linux-image.deb 2023-11-15T02:41:47.564 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package linux-image-6.6.0-rc5-g3b796cfe2969. 2023-11-15T02:41:49.371 INFO:teuthology.orchestra.run.smithi121.stdout:(Reading database ... 104373 files and directories currently installed.) 2023-11-15T02:41:49.376 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-11-15T02:41:49.482 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking linux-image-6.6.0-rc5-g3b796cfe2969 (6.6.0-rc5-g3b796cfe2969-1) ... 2023-11-15T02:41:52.331 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up linux-image-6.6.0-rc5-g3b796cfe2969 (6.6.0-rc5-g3b796cfe2969-1) ... 2023-11-15T02:41:52.443 INFO:teuthology.orchestra.run.smithi121.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-rc5-g3b796cfe2969 2023-11-15T02:41:52.722 INFO:teuthology.orchestra.run.smithi190.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-11-15T02:41:52.723 INFO:teuthology.orchestra.run.smithi190.stderr: * [new branch] main -> origin/main 2023-11-15T02:41:52.783 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20190312 -> 20190312 2023-11-15T02:41:52.784 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20190416 -> 20190416 2023-11-15T02:41:52.784 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20190514 -> 20190514 2023-11-15T02:41:52.784 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20190618 -> 20190618 2023-11-15T02:41:52.784 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20190717 -> 20190717 2023-11-15T02:41:52.785 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20190815 -> 20190815 2023-11-15T02:41:52.787 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20190923 -> 20190923 2023-11-15T02:41:52.787 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20191022 -> 20191022 2023-11-15T02:41:52.787 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20191215 -> 20191215 2023-11-15T02:41:52.787 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20200122 -> 20200122 2023-11-15T02:41:52.788 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20200316 -> 20200316 2023-11-15T02:41:52.788 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20200421 -> 20200421 2023-11-15T02:41:52.788 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20200519 -> 20200519 2023-11-15T02:41:52.788 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20200619 -> 20200619 2023-11-15T02:41:52.788 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20200721 -> 20200721 2023-11-15T02:41:52.789 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20200817 -> 20200817 2023-11-15T02:41:52.789 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20200918 -> 20200918 2023-11-15T02:41:52.789 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20201022 -> 20201022 2023-11-15T02:41:52.789 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20201118 -> 20201118 2023-11-15T02:41:52.789 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20201218 -> 20201218 2023-11-15T02:41:52.790 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20210208 -> 20210208 2023-11-15T02:41:52.790 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20210315 -> 20210315 2023-11-15T02:41:52.790 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20210511 -> 20210511 2023-11-15T02:41:52.790 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20210716 -> 20210716 2023-11-15T02:41:52.791 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20210818 -> 20210818 2023-11-15T02:41:52.793 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20210919 -> 20210919 2023-11-15T02:41:52.793 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20211027 -> 20211027 2023-11-15T02:41:52.794 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20211216 -> 20211216 2023-11-15T02:41:52.794 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20220209 -> 20220209 2023-11-15T02:41:52.794 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20220310 -> 20220310 2023-11-15T02:41:52.794 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20220411 -> 20220411 2023-11-15T02:41:52.794 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20220509 -> 20220509 2023-11-15T02:41:52.795 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20220610 -> 20220610 2023-11-15T02:41:52.795 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20220708 -> 20220708 2023-11-15T02:41:52.795 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20220815 -> 20220815 2023-11-15T02:41:52.795 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20220913 -> 20220913 2023-11-15T02:41:52.795 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20221012 -> 20221012 2023-11-15T02:41:52.796 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20221109 -> 20221109 2023-11-15T02:41:52.796 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20221214 -> 20221214 2023-11-15T02:41:52.796 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20230117 -> 20230117 2023-11-15T02:41:52.796 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20230210 -> 20230210 2023-11-15T02:41:52.796 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20230310 -> 20230310 2023-11-15T02:41:52.796 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20230404 -> 20230404 2023-11-15T02:41:52.797 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20230515 -> 20230515 2023-11-15T02:41:52.797 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20230625 -> 20230625 2023-11-15T02:41:52.797 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20230804 -> 20230804 2023-11-15T02:41:52.797 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20230919 -> 20230919 2023-11-15T02:41:52.797 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20231030 -> 20231030 2023-11-15T02:41:52.798 INFO:teuthology.orchestra.run.smithi190.stderr: * [new tag] 20231111 -> 20231111 2023-11-15T02:41:56.673 INFO:teuthology.orchestra.run.smithi190.stderr:Updating files: 51% (1643/3198) Updating files: 52% (1663/3198) Updating files: 53% (1695/3198) Updating files: 54% (1727/3198) Updating files: 55% (1759/3198) Updating files: 56% (1791/3198) Updating files: 57% (1823/3198) Updating files: 58% (1855/3198) Updating files: 59% (1887/3198) Updating files: 60% (1919/3198) Updating files: 61% (1951/3198) Updating files: 62% (1983/3198) Updating files: 63% (2015/3198) Updating files: 64% (2047/3198) Updating files: 65% (2079/3198) Updating files: 66% (2111/3198) Updating files: 66% (2124/3198) Updating files: 67% (2143/3198) Updating files: 68% (2175/3198) Updating files: 69% (2207/3198) Updating files: 70% (2239/3198) Updating files: 71% (2271/3198) Updating files: 72% (2303/3198) Updating files: 73% (2335/3198) Updating files: 74% (2367/3198) Updating files: 75% (2399/3198) Updating files: 76% (2431/3198) Updating files: 77% (2463/3198) Updating files: 78% (2495/3198) Updating files: 79% (2527/3198) Updating files: 80% (2559/3198) Updating files: 81% (2591/3198) Updating files: 81% (2608/3198) Updating files: 82% (2623/3198) Updating files: 83% (2655/3198) Updating files: 84% (2687/3198) Updating files: 85% (2719/3198) Updating files: 86% (2751/3198) Updating files: 87% (2783/3198) Updating files: 88% (2815/3198) Updating files: 89% (2847/3198) Updating files: 90% (2879/3198) Updating files: 91% (2911/3198) Updating files: 92% (2943/3198) Updating files: 93% (2975/3198) Updating files: 94% (3007/3198) Updating files: 95% (3039/3198) Updating files: 96% (3071/3198) Updating files: 97% (3103/3198) Updating files: 98% (3135/3198) Updating files: 99% (3167/3198) Updating files: 100% (3198/3198) Updating files: 100% (3198/3198), done. 2023-11-15T02:41:56.677 INFO:teuthology.orchestra.run.smithi190.stdout:HEAD is now at b72eeb60 Merge branch 'mlimonci/robot-pr-improvements' into 'main' 2023-11-15T02:41:56.694 INFO:teuthology.task.kernel:Downloading kernel 3b796cfe2969fb2f2d017e8146dccc1ef582e837 on client.1... 2023-11-15T02:41:56.695 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=3b796cfe2969fb2f2d017e8146dccc1ef582e837 2023-11-15T02:41:56.921 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/3b796cfe2969fb2f2d017e8146dccc1ef582e837/ubuntu/focal/flavors/default/pool/main/l/linux-6.6.0-rc5-g3b796cfe2969/ 2023-11-15T02:41:56.921 DEBUG:teuthology.orchestra.run.smithi190:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-rc5-g3b796cfe2969_6.6.0-rc5-g3b796cfe2969-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://1.chacra.ceph.com/r/kernel/testing/3b796cfe2969fb2f2d017e8146dccc1ef582e837/ubuntu/focal/flavors/default/pool/main/l/linux-6.6.0-rc5-g3b796cfe2969/ --input-file=- 2023-11-15T02:41:56.924 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi190.front.sepia.ceph.com to complete... 2023-11-15T02:42:02.687 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15 02:42:02 URL:https://1.chacra.ceph.com/r/kernel/testing/3b796cfe2969fb2f2d017e8146dccc1ef582e837/ubuntu/focal/flavors/default/pool/main/l/linux-6.6.0-rc5-g3b796cfe2969/linux-image-6.6.0-rc5-g3b796cfe2969_6.6.0-rc5-g3b796cfe2969-1_amd64.deb [38912044/38912044] -> "/tmp/linux-image.deb" [1] 2023-11-15T02:42:02.688 INFO:teuthology.orchestra.run.smithi190.stderr:FINISHED --2023-11-15 02:42:02-- 2023-11-15T02:42:02.688 INFO:teuthology.orchestra.run.smithi190.stderr:Total wall clock time: 5.7s 2023-11-15T02:42:02.688 INFO:teuthology.orchestra.run.smithi190.stderr:Downloaded: 1 files, 37M in 5.3s (6.95 MB/s) 2023-11-15T02:42:02.690 INFO:teuthology.task.kernel:Installing kernel 3b796cfe2969fb2f2d017e8146dccc1ef582e837 on client.1... 2023-11-15T02:42:02.690 DEBUG:teuthology.orchestra.run.smithi190:> sudo dpkg -i /tmp/linux-image.deb 2023-11-15T02:42:02.971 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package linux-image-6.6.0-rc5-g3b796cfe2969. 2023-11-15T02:42:04.724 INFO:teuthology.orchestra.run.smithi190.stdout:(Reading database ... 104373 files and directories currently installed.) 2023-11-15T02:42:04.729 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-11-15T02:42:04.897 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-image-6.6.0-rc5-g3b796cfe2969 (6.6.0-rc5-g3b796cfe2969-1) ... 2023-11-15T02:42:07.912 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-image-6.6.0-rc5-g3b796cfe2969 (6.6.0-rc5-g3b796cfe2969-1) ... 2023-11-15T02:42:08.030 INFO:teuthology.orchestra.run.smithi190.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:17.857 INFO:teuthology.orchestra.run.smithi121.stderr:Sourcing file `/etc/default/grub' 2023-11-15T02:42:17.869 INFO:teuthology.orchestra.run.smithi121.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-15T02:42:17.896 INFO:teuthology.orchestra.run.smithi121.stderr:Generating grub configuration file ... 2023-11-15T02:42:18.244 INFO:teuthology.orchestra.run.smithi121.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:18.282 INFO:teuthology.orchestra.run.smithi121.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:18.659 INFO:teuthology.orchestra.run.smithi121.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-11-15T02:42:18.672 INFO:teuthology.orchestra.run.smithi121.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-11-15T02:42:20.510 INFO:teuthology.orchestra.run.smithi121.stderr:done 2023-11-15T02:42:20.680 DEBUG:teuthology.orchestra.run.smithi121:> dpkg-deb -c /tmp/linux-image.deb 2023-11-15T02:42:20.689 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./ 2023-11-15T02:42:20.690 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./boot/ 2023-11-15T02:42:20.690 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 4415097 2023-11-07 03:19 ./boot/System.map-6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:20.705 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 169233 2023-11-07 03:19 ./boot/config-6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:20.706 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11352928 2023-11-07 03:19 ./boot/vmlinuz-6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:20.726 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/ 2023-11-15T02:42:20.727 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/ 2023-11-15T02:42:20.727 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/postinst.d/ 2023-11-15T02:42:20.727 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/postrm.d/ 2023-11-15T02:42:20.728 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/preinst.d/ 2023-11-15T02:42:20.728 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/prerm.d/ 2023-11-15T02:42:20.728 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/ 2023-11-15T02:42:20.728 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/ 2023-11-15T02:42:20.728 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/ 2023-11-15T02:42:20.728 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/ 2023-11-15T02:42:20.729 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/ 2023-11-15T02:42:20.729 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/ 2023-11-15T02:42:20.729 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/ 2023-11-15T02:42:20.729 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 490233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/aesni-intel.ko 2023-11-15T02:42:20.732 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-15T02:42:20.732 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-15T02:42:20.732 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-15T02:42:20.733 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 93113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-15T02:42:20.733 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 106729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-15T02:42:20.733 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 152241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-15T02:42:20.733 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-15T02:42:20.733 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-15T02:42:20.734 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 83889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-15T02:42:20.734 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-15T02:42:20.734 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-15T02:42:20.734 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-15T02:42:20.735 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-15T02:42:20.735 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-15T02:42:20.735 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-15T02:42:20.735 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 58585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-15T02:42:20.735 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-15T02:42:20.736 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-15T02:42:20.736 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-15T02:42:20.736 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/ 2023-11-15T02:42:20.736 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/cpu/ 2023-11-15T02:42:20.736 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/cpu/mce/ 2023-11-15T02:42:20.736 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-15T02:42:20.737 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/cpuid.ko 2023-11-15T02:42:20.737 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/msr.ko 2023-11-15T02:42:20.737 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kvm/ 2023-11-15T02:42:20.737 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 353761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-15T02:42:20.737 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 881713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-15T02:42:20.738 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 2276817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kvm/kvm.ko 2023-11-15T02:42:20.802 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/ 2023-11-15T02:42:20.803 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/af_alg.ko 2023-11-15T02:42:20.803 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/algif_hash.ko 2023-11-15T02:42:20.804 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/algif_skcipher.ko 2023-11-15T02:42:20.804 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/ansi_cprng.ko 2023-11-15T02:42:20.804 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/anubis.ko 2023-11-15T02:42:20.804 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/arc4.ko 2023-11-15T02:42:20.804 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/asymmetric_keys/ 2023-11-15T02:42:20.805 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-11-15T02:42:20.805 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/ 2023-11-15T02:42:20.805 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_memcpy.ko 2023-11-15T02:42:20.805 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_pq.ko 2023-11-15T02:42:20.806 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-15T02:42:20.806 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_tx.ko 2023-11-15T02:42:20.808 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_xor.ko 2023-11-15T02:42:20.808 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/raid6test.ko 2023-11-15T02:42:20.808 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/authenc.ko 2023-11-15T02:42:20.808 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/authencesn.ko 2023-11-15T02:42:20.809 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/blake2b_generic.ko 2023-11-15T02:42:20.809 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/blowfish_common.ko 2023-11-15T02:42:20.809 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/blowfish_generic.ko 2023-11-15T02:42:20.809 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/camellia_generic.ko 2023-11-15T02:42:20.809 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cast5_generic.ko 2023-11-15T02:42:20.810 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cast6_generic.ko 2023-11-15T02:42:20.810 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cast_common.ko 2023-11-15T02:42:20.810 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/ccm.ko 2023-11-15T02:42:20.810 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cmac.ko 2023-11-15T02:42:20.811 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/crc32_generic.ko 2023-11-15T02:42:20.811 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cryptd.ko 2023-11-15T02:42:20.811 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/crypto_engine.ko 2023-11-15T02:42:20.812 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/crypto_simd.ko 2023-11-15T02:42:20.812 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/crypto_user.ko 2023-11-15T02:42:20.812 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cts.ko 2023-11-15T02:42:20.812 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/des_generic.ko 2023-11-15T02:42:20.812 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/dh_generic.ko 2023-11-15T02:42:20.813 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/echainiv.ko 2023-11-15T02:42:20.813 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/essiv.ko 2023-11-15T02:42:20.813 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/fcrypt.ko 2023-11-15T02:42:20.813 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/khazad.ko 2023-11-15T02:42:20.813 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/lrw.ko 2023-11-15T02:42:20.814 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/lz4.ko 2023-11-15T02:42:20.814 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/lz4hc.ko 2023-11-15T02:42:20.814 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/md4.ko 2023-11-15T02:42:20.814 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/michael_mic.ko 2023-11-15T02:42:20.814 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/pcbc.ko 2023-11-15T02:42:20.815 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/pcrypt.ko 2023-11-15T02:42:20.815 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/rmd160.ko 2023-11-15T02:42:20.815 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/seed.ko 2023-11-15T02:42:20.815 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/serpent_generic.ko 2023-11-15T02:42:20.816 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 149193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/tcrypt.ko 2023-11-15T02:42:20.816 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/tea.ko 2023-11-15T02:42:20.816 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/twofish_common.ko 2023-11-15T02:42:20.816 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/twofish_generic.ko 2023-11-15T02:42:20.816 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/vmac.ko 2023-11-15T02:42:20.817 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/wp512.ko 2023-11-15T02:42:20.817 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/xcbc.ko 2023-11-15T02:42:20.817 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/xor.ko 2023-11-15T02:42:20.817 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/xts.ko 2023-11-15T02:42:20.817 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/xxhash_generic.ko 2023-11-15T02:42:20.818 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ 2023-11-15T02:42:20.818 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/ 2023-11-15T02:42:20.818 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/acpi_extlog.ko 2023-11-15T02:42:20.818 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-15T02:42:20.819 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/acpi_pad.ko 2023-11-15T02:42:20.819 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/apei/ 2023-11-15T02:42:20.819 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/apei/einj.ko 2023-11-15T02:42:20.819 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/ec_sys.ko 2023-11-15T02:42:20.820 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/platform_profile.ko 2023-11-15T02:42:20.820 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/sbs.ko 2023-11-15T02:42:20.820 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/sbshc.ko 2023-11-15T02:42:20.820 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 103025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/video.ko 2023-11-15T02:42:20.820 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/ 2023-11-15T02:42:20.821 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/acard-ahci.ko 2023-11-15T02:42:20.821 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 88865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/ahci.ko 2023-11-15T02:42:20.821 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/ahci_platform.ko 2023-11-15T02:42:20.822 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 88289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/libahci.ko 2023-11-15T02:42:20.822 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/libahci_platform.ko 2023-11-15T02:42:20.822 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_acpi.ko 2023-11-15T02:42:20.822 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_ali.ko 2023-11-15T02:42:20.823 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_amd.ko 2023-11-15T02:42:20.823 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_artop.ko 2023-11-15T02:42:20.823 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_atiixp.ko 2023-11-15T02:42:20.823 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_atp867x.ko 2023-11-15T02:42:20.823 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_cmd640.ko 2023-11-15T02:42:20.824 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_cmd64x.ko 2023-11-15T02:42:20.824 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_cypress.ko 2023-11-15T02:42:20.824 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_efar.ko 2023-11-15T02:42:20.824 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_hpt366.ko 2023-11-15T02:42:20.824 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_hpt37x.ko 2023-11-15T02:42:20.825 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-15T02:42:20.825 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-15T02:42:20.825 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_it8213.ko 2023-11-15T02:42:20.825 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_it821x.ko 2023-11-15T02:42:20.825 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_jmicron.ko 2023-11-15T02:42:20.826 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_legacy.ko 2023-11-15T02:42:20.826 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_marvell.ko 2023-11-15T02:42:20.826 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_mpiix.ko 2023-11-15T02:42:20.826 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_netcell.ko 2023-11-15T02:42:20.826 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_ninja32.ko 2023-11-15T02:42:20.827 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_ns87410.ko 2023-11-15T02:42:20.827 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_ns87415.ko 2023-11-15T02:42:20.827 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_oldpiix.ko 2023-11-15T02:42:20.827 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_opti.ko 2023-11-15T02:42:20.827 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_optidma.ko 2023-11-15T02:42:20.828 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_pcmcia.ko 2023-11-15T02:42:20.828 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-15T02:42:20.828 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-15T02:42:20.828 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_piccolo.ko 2023-11-15T02:42:20.829 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_radisys.ko 2023-11-15T02:42:20.829 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_rdc.ko 2023-11-15T02:42:20.829 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_rz1000.ko 2023-11-15T02:42:20.829 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_sch.ko 2023-11-15T02:42:20.829 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_serverworks.ko 2023-11-15T02:42:20.830 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_sil680.ko 2023-11-15T02:42:20.830 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_sl82c105.ko 2023-11-15T02:42:20.830 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_triflex.ko 2023-11-15T02:42:20.831 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_via.ko 2023-11-15T02:42:20.831 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pdc_adma.ko 2023-11-15T02:42:20.831 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_inic162x.ko 2023-11-15T02:42:20.831 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 83345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_mv.ko 2023-11-15T02:42:20.832 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 70393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_nv.ko 2023-11-15T02:42:20.832 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_promise.ko 2023-11-15T02:42:20.832 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_qstor.ko 2023-11-15T02:42:20.832 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_sil.ko 2023-11-15T02:42:20.833 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_sil24.ko 2023-11-15T02:42:20.833 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_sis.ko 2023-11-15T02:42:20.833 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_svw.ko 2023-11-15T02:42:20.833 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_sx4.ko 2023-11-15T02:42:20.834 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_uli.ko 2023-11-15T02:42:20.834 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_via.ko 2023-11-15T02:42:20.834 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_vsc.ko 2023-11-15T02:42:20.834 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/auxdisplay/ 2023-11-15T02:42:20.834 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/auxdisplay/ks0108.ko 2023-11-15T02:42:20.835 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/bcma/ 2023-11-15T02:42:20.835 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 124473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/bcma/bcma.ko 2023-11-15T02:42:20.835 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/ 2023-11-15T02:42:20.835 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/aoe/ 2023-11-15T02:42:20.836 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 113665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/aoe/aoe.ko 2023-11-15T02:42:20.836 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/drbd/ 2023-11-15T02:42:20.836 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 965585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/drbd/drbd.ko 2023-11-15T02:42:20.836 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 173225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/floppy.ko 2023-11-15T02:42:20.837 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/mtip32xx/ 2023-11-15T02:42:20.837 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 94297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-15T02:42:20.837 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 115793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/nbd.ko 2023-11-15T02:42:20.837 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/null_blk/ 2023-11-15T02:42:20.838 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 110449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/null_blk/null_blk.ko 2023-11-15T02:42:20.838 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 98193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/pktcdvd.ko 2023-11-15T02:42:20.838 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 263769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/rbd.ko 2023-11-15T02:42:20.839 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/xen-blkback/ 2023-11-15T02:42:20.839 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 98145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-15T02:42:20.839 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/zram/ 2023-11-15T02:42:20.839 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 60217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/zram/zram.ko 2023-11-15T02:42:20.840 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ 2023-11-15T02:42:20.840 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/applicom.ko 2023-11-15T02:42:20.840 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hangcheck-timer.ko 2023-11-15T02:42:20.840 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/ 2023-11-15T02:42:20.840 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-15T02:42:20.841 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-15T02:42:20.841 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-15T02:42:20.841 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/via-rng.ko 2023-11-15T02:42:20.841 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-15T02:42:20.842 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ 2023-11-15T02:42:20.842 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-15T02:42:20.843 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 148601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-15T02:42:20.843 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-15T02:42:20.843 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 139313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-15T02:42:20.843 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 70921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-15T02:42:20.843 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-15T02:42:20.844 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/lp.ko 2023-11-15T02:42:20.844 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/mwave/ 2023-11-15T02:42:20.844 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 84097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/mwave/mwave.ko 2023-11-15T02:42:20.844 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/nvram.ko 2023-11-15T02:42:20.844 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ppdev.ko 2023-11-15T02:42:20.845 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tlclk.ko 2023-11-15T02:42:20.845 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/ 2023-11-15T02:42:20.845 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-15T02:42:20.845 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-11-15T02:42:20.846 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-11-15T02:42:20.846 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-11-15T02:42:20.846 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-15T02:42:20.846 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-15T02:42:20.846 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-11-15T02:42:20.847 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/xillybus/ 2023-11-15T02:42:20.847 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/xillybus/xillybus_class.ko 2023-11-15T02:42:20.847 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/xillybus/xillybus_core.ko 2023-11-15T02:42:20.847 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-11-15T02:42:20.847 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/ 2023-11-15T02:42:20.848 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/clk-palmas.ko 2023-11-15T02:42:20.848 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/clk-si5351.ko 2023-11-15T02:42:20.848 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/clk-twl6040.ko 2023-11-15T02:42:20.848 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/clk-wm831x.ko 2023-11-15T02:42:20.848 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/cpufreq/ 2023-11-15T02:42:20.849 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-15T02:42:20.849 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-15T02:42:20.849 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-15T02:42:20.849 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/ 2023-11-15T02:42:20.850 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/ccp/ 2023-11-15T02:42:20.850 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 70801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-15T02:42:20.850 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 243025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/ccp/ccp.ko 2023-11-15T02:42:20.850 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/ 2023-11-15T02:42:20.851 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/ 2023-11-15T02:42:20.851 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/qat_common/ 2023-11-15T02:42:20.851 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 438761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-15T02:42:20.851 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-11-15T02:42:20.851 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-15T02:42:20.852 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/padlock-aes.ko 2023-11-15T02:42:20.852 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/padlock-sha.ko 2023-11-15T02:42:20.852 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/virtio/ 2023-11-15T02:42:20.852 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 68857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-15T02:42:20.853 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dca/ 2023-11-15T02:42:20.853 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dca/dca.ko 2023-11-15T02:42:20.854 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/ 2023-11-15T02:42:20.854 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/dw/ 2023-11-15T02:42:20.854 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-15T02:42:20.854 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/ioat/ 2023-11-15T02:42:20.854 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 122569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-15T02:42:20.855 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/ 2023-11-15T02:42:20.855 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 79953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/amd64_edac.ko 2023-11-15T02:42:20.855 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/e752x_edac.ko 2023-11-15T02:42:20.855 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 63617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/edac_mce_amd.ko 2023-11-15T02:42:20.855 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i3000_edac.ko 2023-11-15T02:42:20.856 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i3200_edac.ko 2023-11-15T02:42:20.856 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i5100_edac.ko 2023-11-15T02:42:20.856 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i5400_edac.ko 2023-11-15T02:42:20.856 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i7300_edac.ko 2023-11-15T02:42:20.857 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 52041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i7core_edac.ko 2023-11-15T02:42:20.857 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i82975x_edac.ko 2023-11-15T02:42:20.857 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/ie31200_edac.ko 2023-11-15T02:42:20.857 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 56825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/sb_edac.ko 2023-11-15T02:42:20.857 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/x38_edac.ko 2023-11-15T02:42:20.858 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/ 2023-11-15T02:42:20.858 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-gpio.ko 2023-11-15T02:42:20.858 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-max14577.ko 2023-11-15T02:42:20.858 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-max77693.ko 2023-11-15T02:42:20.859 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-max8997.ko 2023-11-15T02:42:20.859 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-palmas.ko 2023-11-15T02:42:20.859 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-rt8973a.ko 2023-11-15T02:42:20.859 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-sm5502.ko 2023-11-15T02:42:20.859 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/ 2023-11-15T02:42:20.860 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/dmi-sysfs.ko 2023-11-15T02:42:20.860 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/efi/ 2023-11-15T02:42:20.860 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/efi/efi-pstore.ko 2023-11-15T02:42:20.860 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-15T02:42:20.860 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/ 2023-11-15T02:42:20.861 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-adp5520.ko 2023-11-15T02:42:20.861 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-amd8111.ko 2023-11-15T02:42:20.861 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-arizona.ko 2023-11-15T02:42:20.861 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-da9052.ko 2023-11-15T02:42:20.861 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-da9055.ko 2023-11-15T02:42:20.862 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-dln2.ko 2023-11-15T02:42:20.862 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 77849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-f7188x.ko 2023-11-15T02:42:20.862 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-generic.ko 2023-11-15T02:42:20.862 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-ich.ko 2023-11-15T02:42:20.862 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-11-15T02:42:20.863 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-kempld.ko 2023-11-15T02:42:20.863 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-lp3943.ko 2023-11-15T02:42:20.863 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-max7300.ko 2023-11-15T02:42:20.864 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-max730x.ko 2023-11-15T02:42:20.864 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-max732x.ko 2023-11-15T02:42:20.864 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-11-15T02:42:20.864 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-pca953x.ko 2023-11-15T02:42:20.864 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-pcf857x.ko 2023-11-15T02:42:20.865 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-rdc321x.ko 2023-11-15T02:42:20.865 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-sch.ko 2023-11-15T02:42:20.865 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-sch311x.ko 2023-11-15T02:42:20.865 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-tps65912.ko 2023-11-15T02:42:20.865 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-twl4030.ko 2023-11-15T02:42:20.866 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-twl6040.ko 2023-11-15T02:42:20.866 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-15T02:42:20.866 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-vx855.ko 2023-11-15T02:42:20.866 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-wm831x.ko 2023-11-15T02:42:20.866 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-wm8350.ko 2023-11-15T02:42:20.867 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-wm8994.ko 2023-11-15T02:42:20.867 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/ 2023-11-15T02:42:20.867 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-a4tech.ko 2023-11-15T02:42:20.867 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-apple.ko 2023-11-15T02:42:20.868 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-appleir.ko 2023-11-15T02:42:20.868 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-aureal.ko 2023-11-15T02:42:20.868 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-axff.ko 2023-11-15T02:42:20.868 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-belkin.ko 2023-11-15T02:42:20.868 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-cherry.ko 2023-11-15T02:42:20.869 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-chicony.ko 2023-11-15T02:42:20.869 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 60649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-cp2112.ko 2023-11-15T02:42:20.869 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-cypress.ko 2023-11-15T02:42:20.869 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-dr.ko 2023-11-15T02:42:20.869 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-elecom.ko 2023-11-15T02:42:20.870 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-elo.ko 2023-11-15T02:42:20.870 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-emsff.ko 2023-11-15T02:42:20.870 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-ezkey.ko 2023-11-15T02:42:20.870 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-gaff.ko 2023-11-15T02:42:20.870 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-generic.ko 2023-11-15T02:42:20.871 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-gt683r.ko 2023-11-15T02:42:20.871 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-gyration.ko 2023-11-15T02:42:20.871 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-15T02:42:20.871 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-15T02:42:20.871 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-holtekff.ko 2023-11-15T02:42:20.872 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-hyperv.ko 2023-11-15T02:42:20.872 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-icade.ko 2023-11-15T02:42:20.872 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-kensington.ko 2023-11-15T02:42:20.872 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-keytouch.ko 2023-11-15T02:42:20.872 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-kye.ko 2023-11-15T02:42:20.873 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-lcpower.ko 2023-11-15T02:42:20.873 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-led.ko 2023-11-15T02:42:20.873 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-lenovo.ko 2023-11-15T02:42:20.874 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-lg-g15.ko 2023-11-15T02:42:20.874 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 58185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-15T02:42:20.874 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 101369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-15T02:42:20.874 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 80993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-logitech.ko 2023-11-15T02:42:20.874 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-magicmouse.ko 2023-11-15T02:42:20.875 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-microsoft.ko 2023-11-15T02:42:20.875 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-monterey.ko 2023-11-15T02:42:20.875 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-multitouch.ko 2023-11-15T02:42:20.875 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-ntrig.ko 2023-11-15T02:42:20.875 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-ortek.ko 2023-11-15T02:42:20.876 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-penmount.ko 2023-11-15T02:42:20.876 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-petalynx.ko 2023-11-15T02:42:20.876 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 72937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-picolcd.ko 2023-11-15T02:42:20.876 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-pl.ko 2023-11-15T02:42:20.877 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-plantronics.ko 2023-11-15T02:42:20.877 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-primax.ko 2023-11-15T02:42:20.877 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-rmi.ko 2023-11-15T02:42:20.877 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-15T02:42:20.877 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-common.ko 2023-11-15T02:42:20.878 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-15T02:42:20.878 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-15T02:42:20.878 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-15T02:42:20.878 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-15T02:42:20.878 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-15T02:42:20.879 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-15T02:42:20.879 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-15T02:42:20.879 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-15T02:42:20.879 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-15T02:42:20.879 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat.ko 2023-11-15T02:42:20.880 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-saitek.ko 2023-11-15T02:42:20.880 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-samsung.ko 2023-11-15T02:42:20.880 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-15T02:42:20.880 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-sjoy.ko 2023-11-15T02:42:20.880 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-sony.ko 2023-11-15T02:42:20.881 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-speedlink.ko 2023-11-15T02:42:20.881 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-steelseries.ko 2023-11-15T02:42:20.881 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-sunplus.ko 2023-11-15T02:42:20.881 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-15T02:42:20.881 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-tivo.ko 2023-11-15T02:42:20.882 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-tmff.ko 2023-11-15T02:42:20.882 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-topseed.ko 2023-11-15T02:42:20.882 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-twinhan.ko 2023-11-15T02:42:20.882 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 71073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-uclogic.ko 2023-11-15T02:42:20.882 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-waltop.ko 2023-11-15T02:42:20.883 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 143473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-wiimote.ko 2023-11-15T02:42:20.883 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-xinmo.ko 2023-11-15T02:42:20.883 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-zpff.ko 2023-11-15T02:42:20.883 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-zydacron.ko 2023-11-15T02:42:20.884 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 267825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid.ko 2023-11-15T02:42:20.884 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/uhid.ko 2023-11-15T02:42:20.884 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/usbhid/ 2023-11-15T02:42:20.884 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 125721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/usbhid/usbhid.ko 2023-11-15T02:42:20.884 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/usbhid/usbkbd.ko 2023-11-15T02:42:20.885 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/usbhid/usbmouse.ko 2023-11-15T02:42:20.885 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 217873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/wacom.ko 2023-11-15T02:42:20.885 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hv/ 2023-11-15T02:42:20.885 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 64201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hv/hv_balloon.ko 2023-11-15T02:42:20.885 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 89625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hv/hv_utils.ko 2023-11-15T02:42:20.886 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 305465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hv/hv_vmbus.ko 2023-11-15T02:42:20.886 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ 2023-11-15T02:42:20.886 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 66521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/abituguru.ko 2023-11-15T02:42:20.886 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 61393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/abituguru3.ko 2023-11-15T02:42:20.887 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-15T02:42:20.887 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ad7414.ko 2023-11-15T02:42:20.887 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ad7418.ko 2023-11-15T02:42:20.887 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adc128d818.ko 2023-11-15T02:42:20.887 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm1025.ko 2023-11-15T02:42:20.888 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 97977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm1026.ko 2023-11-15T02:42:20.888 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm1029.ko 2023-11-15T02:42:20.888 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 45553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm1031.ko 2023-11-15T02:42:20.888 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm9240.ko 2023-11-15T02:42:20.888 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ads7828.ko 2023-11-15T02:42:20.889 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7410.ko 2023-11-15T02:42:20.889 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7411.ko 2023-11-15T02:42:20.889 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 74433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7462.ko 2023-11-15T02:42:20.889 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7470.ko 2023-11-15T02:42:20.889 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 67929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7475.ko 2023-11-15T02:42:20.889 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7x10.ko 2023-11-15T02:42:20.890 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/amc6821.ko 2023-11-15T02:42:20.890 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 55265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/applesmc.ko 2023-11-15T02:42:20.890 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/asb100.ko 2023-11-15T02:42:20.890 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 58617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/asc7621.ko 2023-11-15T02:42:20.891 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-15T02:42:20.891 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/atxp1.ko 2023-11-15T02:42:20.891 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/coretemp.ko 2023-11-15T02:42:20.891 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/da9052-hwmon.ko 2023-11-15T02:42:20.891 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/da9055-hwmon.ko 2023-11-15T02:42:20.891 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-15T02:42:20.892 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 99601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/dme1737.ko 2023-11-15T02:42:20.892 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ds1621.ko 2023-11-15T02:42:20.892 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ds620.ko 2023-11-15T02:42:20.892 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/emc1403.ko 2023-11-15T02:42:20.892 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/emc2103.ko 2023-11-15T02:42:20.893 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/emc6w201.ko 2023-11-15T02:42:20.893 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 66985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/f71805f.ko 2023-11-15T02:42:20.893 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 94457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/f71882fg.ko 2023-11-15T02:42:20.893 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/f75375s.ko 2023-11-15T02:42:20.893 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/fam15h_power.ko 2023-11-15T02:42:20.894 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 55449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/fschmd.ko 2023-11-15T02:42:20.894 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/g760a.ko 2023-11-15T02:42:20.894 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/g762.ko 2023-11-15T02:42:20.894 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 46801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/gl518sm.ko 2023-11-15T02:42:20.894 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 45529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/gl520sm.ko 2023-11-15T02:42:20.895 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/hih6130.ko 2023-11-15T02:42:20.895 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-15T02:42:20.895 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/i5500_temp.ko 2023-11-15T02:42:20.895 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/i5k_amb.ko 2023-11-15T02:42:20.895 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ibmaem.ko 2023-11-15T02:42:20.896 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ibmpex.ko 2023-11-15T02:42:20.896 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ina209.ko 2023-11-15T02:42:20.896 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ina2xx.ko 2023-11-15T02:42:20.896 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 130449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/it87.ko 2023-11-15T02:42:20.896 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/jc42.ko 2023-11-15T02:42:20.897 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/k10temp.ko 2023-11-15T02:42:20.897 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/k8temp.ko 2023-11-15T02:42:20.897 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lineage-pem.ko 2023-11-15T02:42:20.897 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 48601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm63.ko 2023-11-15T02:42:20.897 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm73.ko 2023-11-15T02:42:20.898 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm75.ko 2023-11-15T02:42:20.898 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm77.ko 2023-11-15T02:42:20.898 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm78.ko 2023-11-15T02:42:20.898 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm80.ko 2023-11-15T02:42:20.898 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm83.ko 2023-11-15T02:42:20.899 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 72833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm85.ko 2023-11-15T02:42:20.899 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 46153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm87.ko 2023-11-15T02:42:20.899 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm90.ko 2023-11-15T02:42:20.899 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm92.ko 2023-11-15T02:42:20.899 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 108841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm93.ko 2023-11-15T02:42:20.900 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm95234.ko 2023-11-15T02:42:20.900 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm95241.ko 2023-11-15T02:42:20.900 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm95245.ko 2023-11-15T02:42:20.900 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc2945.ko 2023-11-15T02:42:20.900 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4151.ko 2023-11-15T02:42:20.901 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4215.ko 2023-11-15T02:42:20.901 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4222.ko 2023-11-15T02:42:20.901 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4245.ko 2023-11-15T02:42:20.901 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4260.ko 2023-11-15T02:42:20.901 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4261.ko 2023-11-15T02:42:20.902 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max16065.ko 2023-11-15T02:42:20.902 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max1619.ko 2023-11-15T02:42:20.902 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max1668.ko 2023-11-15T02:42:20.902 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max197.ko 2023-11-15T02:42:20.903 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max6639.ko 2023-11-15T02:42:20.903 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max6650.ko 2023-11-15T02:42:20.903 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max6697.ko 2023-11-15T02:42:20.903 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/mc13783-adc.ko 2023-11-15T02:42:20.904 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/mcp3021.ko 2023-11-15T02:42:20.904 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-11-15T02:42:20.904 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/nct6683.ko 2023-11-15T02:42:20.904 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 119817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/nct6775-core.ko 2023-11-15T02:42:20.904 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 60705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/nct6775.ko 2023-11-15T02:42:20.905 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 59737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/nct7802.ko 2023-11-15T02:42:20.905 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 82337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pc87360.ko 2023-11-15T02:42:20.905 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pc87427.ko 2023-11-15T02:42:20.905 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pcf8591.ko 2023-11-15T02:42:20.905 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/ 2023-11-15T02:42:20.906 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-15T02:42:20.906 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-15T02:42:20.906 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-15T02:42:20.906 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-15T02:42:20.906 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-15T02:42:20.907 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-15T02:42:20.907 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-15T02:42:20.907 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 82577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-15T02:42:20.907 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-15T02:42:20.908 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-15T02:42:20.908 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-15T02:42:20.908 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-15T02:42:20.908 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/powr1220.ko 2023-11-15T02:42:20.908 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sch5627.ko 2023-11-15T02:42:20.909 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sch5636.ko 2023-11-15T02:42:20.909 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-15T02:42:20.909 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sht15.ko 2023-11-15T02:42:20.909 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sht21.ko 2023-11-15T02:42:20.909 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/shtc1.ko 2023-11-15T02:42:20.910 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sis5595.ko 2023-11-15T02:42:20.910 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/smsc47b397.ko 2023-11-15T02:42:20.910 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/smsc47m1.ko 2023-11-15T02:42:20.910 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/smsc47m192.ko 2023-11-15T02:42:20.911 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/thmc50.ko 2023-11-15T02:42:20.911 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/tmp102.ko 2023-11-15T02:42:20.911 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/tmp103.ko 2023-11-15T02:42:20.911 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/tmp401.ko 2023-11-15T02:42:20.911 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/tmp421.ko 2023-11-15T02:42:20.912 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/via-cputemp.ko 2023-11-15T02:42:20.912 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/via686a.ko 2023-11-15T02:42:20.912 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 52457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/vt1211.ko 2023-11-15T02:42:20.912 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/vt8231.ko 2023-11-15T02:42:20.913 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 57513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83627ehf.ko 2023-11-15T02:42:20.913 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 78873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83627hf.ko 2023-11-15T02:42:20.913 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 71345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83781d.ko 2023-11-15T02:42:20.913 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 63289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83791d.ko 2023-11-15T02:42:20.913 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 71321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83792d.ko 2023-11-15T02:42:20.914 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 101473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83793.ko 2023-11-15T02:42:20.914 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 119505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83795.ko 2023-11-15T02:42:20.914 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83l785ts.ko 2023-11-15T02:42:20.914 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83l786ng.ko 2023-11-15T02:42:20.915 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-11-15T02:42:20.915 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-11-15T02:42:20.915 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/ 2023-11-15T02:42:20.915 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/algos/ 2023-11-15T02:42:20.915 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-15T02:42:20.916 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-15T02:42:20.916 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/ 2023-11-15T02:42:20.916 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-11-15T02:42:20.916 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-11-15T02:42:20.916 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-11-15T02:42:20.917 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-15T02:42:20.917 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-15T02:42:20.917 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-15T02:42:20.917 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-11-15T02:42:20.918 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 6929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-15T02:42:20.918 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-11-15T02:42:20.918 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-15T02:42:20.918 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-15T02:42:20.918 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-15T02:42:20.919 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-15T02:42:20.919 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-11-15T02:42:20.919 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-11-15T02:42:20.919 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 62817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-15T02:42:20.919 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-15T02:42:20.920 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-15T02:42:20.920 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-11-15T02:42:20.920 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-15T02:42:20.920 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-15T02:42:20.920 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-11-15T02:42:20.921 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-15T02:42:20.921 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-15T02:42:20.921 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-15T02:42:20.921 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-11-15T02:42:20.921 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-15T02:42:20.922 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-15T02:42:20.922 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-11-15T02:42:20.922 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-11-15T02:42:20.922 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-15T02:42:20.923 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-11-15T02:42:20.923 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-15T02:42:20.923 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-15T02:42:20.923 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-15T02:42:20.923 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-15T02:42:20.924 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-11-15T02:42:20.924 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/i2c-mux.ko 2023-11-15T02:42:20.924 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/i2c-smbus.ko 2023-11-15T02:42:20.924 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/i2c-stub.ko 2023-11-15T02:42:20.924 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/muxes/ 2023-11-15T02:42:20.925 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-11-15T02:42:20.925 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-11-15T02:42:20.925 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-11-15T02:42:20.925 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ 2023-11-15T02:42:20.925 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ 2023-11-15T02:42:20.925 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 273609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-15T02:42:20.926 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 879969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ib_core.ko 2023-11-15T02:42:20.926 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 69337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-15T02:42:20.926 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 324217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-15T02:42:20.926 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 97985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-15T02:42:20.926 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 282273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-15T02:42:20.927 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 66201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-15T02:42:20.927 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/ 2023-11-15T02:42:20.927 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/cxgb4/ 2023-11-15T02:42:20.927 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 493073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-15T02:42:20.927 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mlx4/ 2023-11-15T02:42:20.928 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 419737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-15T02:42:20.928 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mlx5/ 2023-11-15T02:42:20.928 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 739025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-15T02:42:20.928 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mthca/ 2023-11-15T02:42:20.928 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 296625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-15T02:42:20.929 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/ocrdma/ 2023-11-15T02:42:20.929 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 153697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-11-15T02:42:20.929 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/usnic/ 2023-11-15T02:42:20.929 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 168969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-11-15T02:42:20.929 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/ 2023-11-15T02:42:20.930 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/ipoib/ 2023-11-15T02:42:20.930 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 263081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-15T02:42:20.930 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/iser/ 2023-11-15T02:42:20.930 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 100241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-15T02:42:20.930 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/isert/ 2023-11-15T02:42:20.931 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 111425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-15T02:42:20.931 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/srp/ 2023-11-15T02:42:20.931 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 128753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-15T02:42:20.931 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/srpt/ 2023-11-15T02:42:20.931 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 129529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-15T02:42:20.932 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/ 2023-11-15T02:42:20.932 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/evbug.ko 2023-11-15T02:42:20.932 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/ff-memless.ko 2023-11-15T02:42:20.932 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/gameport/ 2023-11-15T02:42:20.932 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-15T02:42:20.933 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-15T02:42:20.933 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/gameport/gameport.ko 2023-11-15T02:42:20.933 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/joydev.ko 2023-11-15T02:42:20.933 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/ 2023-11-15T02:42:20.934 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-11-15T02:42:20.934 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-11-15T02:42:20.934 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-11-15T02:42:20.934 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-11-15T02:42:20.934 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-15T02:42:20.935 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-11-15T02:42:20.935 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/lkkbd.ko 2023-11-15T02:42:20.935 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/lm8323.ko 2023-11-15T02:42:20.935 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/lm8333.ko 2023-11-15T02:42:20.935 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-11-15T02:42:20.935 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-11-15T02:42:20.936 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-11-15T02:42:20.936 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-11-15T02:42:20.936 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/newtonkbd.ko 2023-11-15T02:42:20.936 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-11-15T02:42:20.936 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/qt1070.ko 2023-11-15T02:42:20.937 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/qt2160.ko 2023-11-15T02:42:20.937 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-11-15T02:42:20.937 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/stowaway.ko 2023-11-15T02:42:20.937 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/sunkbd.ko 2023-11-15T02:42:20.937 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-11-15T02:42:20.938 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-11-15T02:42:20.938 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-11-15T02:42:20.938 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/xtkbd.ko 2023-11-15T02:42:20.938 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/matrix-keymap.ko 2023-11-15T02:42:20.939 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/ 2023-11-15T02:42:20.939 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/appletouch.ko 2023-11-15T02:42:20.939 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/bcm5974.ko 2023-11-15T02:42:20.939 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 123681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/cyapatp.ko 2023-11-15T02:42:20.939 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 89449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-15T02:42:20.940 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/gpio_mouse.ko 2023-11-15T02:42:20.940 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 335977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/psmouse.ko 2023-11-15T02:42:20.940 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/sermouse.ko 2023-11-15T02:42:20.940 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-15T02:42:20.940 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-15T02:42:20.941 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-15T02:42:20.941 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/rmi4/ 2023-11-15T02:42:20.941 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 109873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-15T02:42:20.941 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/ 2023-11-15T02:42:20.941 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/altera_ps2.ko 2023-11-15T02:42:20.942 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/arc_ps2.ko 2023-11-15T02:42:20.942 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/ct82c710.ko 2023-11-15T02:42:20.942 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-15T02:42:20.942 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/parkbd.ko 2023-11-15T02:42:20.942 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/pcips2.ko 2023-11-15T02:42:20.943 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/ps2mult.ko 2023-11-15T02:42:20.943 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/serio_raw.ko 2023-11-15T02:42:20.943 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/serport.ko 2023-11-15T02:42:20.943 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/sparse-keymap.ko 2023-11-15T02:42:20.943 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/iommu/ 2023-11-15T02:42:20.944 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/iommu/amd/ 2023-11-15T02:42:20.944 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-15T02:42:20.944 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/ 2023-11-15T02:42:20.944 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-88pm860x.ko 2023-11-15T02:42:20.944 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-adp5520.ko 2023-11-15T02:42:20.945 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 59753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-bd2802.ko 2023-11-15T02:42:20.945 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-blinkm.ko 2023-11-15T02:42:20.945 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-da903x.ko 2023-11-15T02:42:20.945 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-da9052.ko 2023-11-15T02:42:20.945 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-gpio.ko 2023-11-15T02:42:20.946 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lm3530.ko 2023-11-15T02:42:20.946 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lm3533.ko 2023-11-15T02:42:20.946 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lm355x.ko 2023-11-15T02:42:20.946 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lm3642.ko 2023-11-15T02:42:20.946 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lp3944.ko 2023-11-15T02:42:20.947 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lp8788.ko 2023-11-15T02:42:20.947 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-max8997.ko 2023-11-15T02:42:20.947 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-mc13783.ko 2023-11-15T02:42:20.947 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-menf21bmc.ko 2023-11-15T02:42:20.947 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-pca9532.ko 2023-11-15T02:42:20.948 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-pca955x.ko 2023-11-15T02:42:20.948 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-pca963x.ko 2023-11-15T02:42:20.948 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-ss4200.ko 2023-11-15T02:42:20.948 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-tca6507.ko 2023-11-15T02:42:20.948 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-wm831x-status.ko 2023-11-15T02:42:20.949 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-wm8350.ko 2023-11-15T02:42:20.949 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ 2023-11-15T02:42:20.949 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-15T02:42:20.949 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-15T02:42:20.949 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-15T02:42:20.950 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-15T02:42:20.950 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-15T02:42:20.950 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-15T02:42:20.950 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-15T02:42:20.950 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-15T02:42:20.951 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/ 2023-11-15T02:42:20.951 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/bcache/ 2023-11-15T02:42:20.951 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 654969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/bcache/bcache.ko 2023-11-15T02:42:20.951 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-bio-prison.ko 2023-11-15T02:42:20.951 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 89601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-bufio.ko 2023-11-15T02:42:20.952 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-cache-smq.ko 2023-11-15T02:42:20.952 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 156009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-cache.ko 2023-11-15T02:42:20.952 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 106897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-crypt.ko 2023-11-15T02:42:20.952 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-delay.ko 2023-11-15T02:42:20.953 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-era.ko 2023-11-15T02:42:20.953 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-flakey.ko 2023-11-15T02:42:20.953 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-log-userspace.ko 2023-11-15T02:42:20.953 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-log.ko 2023-11-15T02:42:20.953 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-mirror.ko 2023-11-15T02:42:20.954 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 78033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-multipath.ko 2023-11-15T02:42:20.954 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-queue-length.ko 2023-11-15T02:42:20.954 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 70825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-raid.ko 2023-11-15T02:42:20.954 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-region-hash.ko 2023-11-15T02:42:20.954 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-round-robin.ko 2023-11-15T02:42:20.955 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-service-time.ko 2023-11-15T02:42:20.955 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 109593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-snapshot.ko 2023-11-15T02:42:20.955 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-switch.ko 2023-11-15T02:42:20.955 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 181865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-thin-pool.ko 2023-11-15T02:42:20.955 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-verity.ko 2023-11-15T02:42:20.956 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-zero.ko 2023-11-15T02:42:20.956 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/faulty.ko 2023-11-15T02:42:20.956 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/linear.ko 2023-11-15T02:42:20.956 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/multipath.ko 2023-11-15T02:42:20.956 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/persistent-data/ 2023-11-15T02:42:20.957 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 199937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-15T02:42:20.957 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/raid0.ko 2023-11-15T02:42:20.957 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 113073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/raid1.ko 2023-11-15T02:42:20.957 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 158681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/raid10.ko 2023-11-15T02:42:20.958 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 395721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/raid456.ko 2023-11-15T02:42:20.958 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/ 2023-11-15T02:42:20.958 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/ 2023-11-15T02:42:20.958 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 178137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptbase.ko 2023-11-15T02:42:20.958 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 63385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptctl.ko 2023-11-15T02:42:20.959 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptfc.ko 2023-11-15T02:42:20.959 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptlan.ko 2023-11-15T02:42:20.959 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 123321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptsas.ko 2023-11-15T02:42:20.959 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 79041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptscsih.ko 2023-11-15T02:42:20.959 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptspi.ko 2023-11-15T02:42:20.960 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/ 2023-11-15T02:42:20.960 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/88pm800.ko 2023-11-15T02:42:20.960 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/88pm805.ko 2023-11-15T02:42:20.960 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/88pm80x.ko 2023-11-15T02:42:20.961 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/arizona-i2c.ko 2023-11-15T02:42:20.961 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 123721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/arizona.ko 2023-11-15T02:42:20.961 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/bcm590xx.ko 2023-11-15T02:42:20.961 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/cros_ec_dev.ko 2023-11-15T02:42:20.961 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/dln2.ko 2023-11-15T02:42:20.962 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/janz-cmodio.ko 2023-11-15T02:42:20.962 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/kempld-core.ko 2023-11-15T02:42:20.962 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lm3533-core.ko 2023-11-15T02:42:20.962 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-11-15T02:42:20.962 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lp3943.ko 2023-11-15T02:42:20.963 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lpc_ich.ko 2023-11-15T02:42:20.963 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lpc_sch.ko 2023-11-15T02:42:20.963 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/max8907.ko 2023-11-15T02:42:20.963 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/mc13xxx-core.ko 2023-11-15T02:42:20.963 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-11-15T02:42:20.963 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/menf21bmc.ko 2023-11-15T02:42:20.964 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/pcf50633-adc.ko 2023-11-15T02:42:20.964 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/pcf50633-gpio.ko 2023-11-15T02:42:20.964 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/pcf50633.ko 2023-11-15T02:42:20.964 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-11-15T02:42:20.964 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/retu-mfd.ko 2023-11-15T02:42:20.965 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 80425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/si476x-core.ko 2023-11-15T02:42:20.965 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 48457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/sm501.ko 2023-11-15T02:42:20.965 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-11-15T02:42:20.965 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/tps65010.ko 2023-11-15T02:42:20.965 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/tps6507x.ko 2023-11-15T02:42:20.966 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/viperboard.ko 2023-11-15T02:42:20.966 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/vx855.ko 2023-11-15T02:42:20.966 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/wl1273-core.ko 2023-11-15T02:42:20.966 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/wm8994.ko 2023-11-15T02:42:20.966 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ 2023-11-15T02:42:20.967 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-11-15T02:42:20.967 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 48489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ad525x_dpot.ko 2023-11-15T02:42:20.967 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/altera-stapl/ 2023-11-15T02:42:20.967 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 46585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-15T02:42:20.968 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/apds9802als.ko 2023-11-15T02:42:20.968 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/apds990x.ko 2023-11-15T02:42:20.968 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 45593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/bh1770glc.ko 2023-11-15T02:42:20.968 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/c2port/ 2023-11-15T02:42:20.968 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-11-15T02:42:20.969 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/c2port/core.ko 2023-11-15T02:42:20.969 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/cb710/ 2023-11-15T02:42:20.969 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/cb710/cb710.ko 2023-11-15T02:42:20.969 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ds1682.ko 2023-11-15T02:42:20.969 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/dummy-irq.ko 2023-11-15T02:42:20.970 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/echo/ 2023-11-15T02:42:20.970 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/echo/echo.ko 2023-11-15T02:42:20.970 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/ 2023-11-15T02:42:20.970 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/at24.ko 2023-11-15T02:42:20.971 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-15T02:42:20.971 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-15T02:42:20.971 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/max6875.ko 2023-11-15T02:42:20.971 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/enclosure.ko 2023-11-15T02:42:20.971 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/genwqe/ 2023-11-15T02:42:20.972 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 152201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-11-15T02:42:20.972 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/hmc6352.ko 2023-11-15T02:42:20.972 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/hpilo.ko 2023-11-15T02:42:20.972 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ibmasm/ 2023-11-15T02:42:20.972 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 72129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-11-15T02:42:20.973 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ics932s401.ko 2023-11-15T02:42:20.973 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/isl29003.ko 2023-11-15T02:42:20.973 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/isl29020.ko 2023-11-15T02:42:20.973 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/lis3lv02d/ 2023-11-15T02:42:20.973 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-15T02:42:20.974 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-15T02:42:20.974 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/mei/ 2023-11-15T02:42:20.974 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 125209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/mei/mei-me.ko 2023-11-15T02:42:20.974 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 56521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/mei/mei-txe.ko 2023-11-15T02:42:20.975 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 310113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/mei/mei.ko 2023-11-15T02:42:20.975 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/phantom.ko 2023-11-15T02:42:20.975 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ti-st/ 2023-11-15T02:42:20.975 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 74265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ti-st/st_drv.ko 2023-11-15T02:42:20.975 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/tifm_7xx1.ko 2023-11-15T02:42:20.975 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/tifm_core.ko 2023-11-15T02:42:20.976 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/tsl2550.ko 2023-11-15T02:42:20.976 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 48697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/vmw_balloon.ko 2023-11-15T02:42:20.976 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/vmw_vmci/ 2023-11-15T02:42:20.976 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 183609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-15T02:42:20.977 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ 2023-11-15T02:42:20.977 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/bonding/ 2023-11-15T02:42:20.977 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 490129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/bonding/bonding.ko 2023-11-15T02:42:20.977 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/ 2023-11-15T02:42:20.977 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/b53/ 2023-11-15T02:42:20.977 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 106033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-15T02:42:20.978 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-15T02:42:20.978 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-15T02:42:20.978 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/mv88e6xxx/ 2023-11-15T02:42:20.978 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 453737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-15T02:42:20.978 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dummy.ko 2023-11-15T02:42:20.979 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/eql.ko 2023-11-15T02:42:20.979 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ 2023-11-15T02:42:20.979 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/ 2023-11-15T02:42:20.979 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 45785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-15T02:42:20.979 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-15T02:42:20.980 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 130553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-15T02:42:20.980 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 68849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-15T02:42:20.980 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/ 2023-11-15T02:42:20.980 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-15T02:42:20.980 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 59689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-15T02:42:20.981 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-15T02:42:20.981 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 99113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-15T02:42:20.981 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/adaptec/ 2023-11-15T02:42:20.981 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-15T02:42:20.981 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/agere/ 2023-11-15T02:42:20.982 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 69201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-15T02:42:20.982 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/alteon/ 2023-11-15T02:42:20.982 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 59585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-15T02:42:20.982 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/altera/ 2023-11-15T02:42:20.983 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-15T02:42:20.983 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/amd/ 2023-11-15T02:42:20.983 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-15T02:42:20.983 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-15T02:42:20.983 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 87297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-15T02:42:20.984 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/ 2023-11-15T02:42:20.984 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/alx/ 2023-11-15T02:42:20.984 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 97337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-15T02:42:20.984 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-11-15T02:42:20.984 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 104673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-15T02:42:20.985 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-11-15T02:42:20.985 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 90257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-15T02:42:20.985 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atlx/ 2023-11-15T02:42:20.985 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 81569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-15T02:42:20.985 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 69729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-15T02:42:20.985 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/ 2023-11-15T02:42:20.986 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 78401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-15T02:42:20.986 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 176761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-15T02:42:20.986 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-11-15T02:42:20.986 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1421497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-15T02:42:21.088 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 128569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-15T02:42:21.089 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/genet/ 2023-11-15T02:42:21.089 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 132089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-15T02:42:21.089 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 312081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-15T02:42:21.089 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/brocade/ 2023-11-15T02:42:21.090 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/brocade/bna/ 2023-11-15T02:42:21.090 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 323513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-15T02:42:21.090 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/ 2023-11-15T02:42:21.090 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-11-15T02:42:21.090 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 193497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-15T02:42:21.091 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-11-15T02:42:21.091 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 347417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-15T02:42:21.091 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-11-15T02:42:21.091 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1044433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-15T02:42:21.091 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-11-15T02:42:21.092 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 166193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-15T02:42:21.092 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-11-15T02:42:21.092 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-15T02:42:21.092 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/cisco/ 2023-11-15T02:42:21.092 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/cisco/enic/ 2023-11-15T02:42:21.093 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 204345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-15T02:42:21.093 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/ 2023-11-15T02:42:21.093 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/ 2023-11-15T02:42:21.093 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 69449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-15T02:42:21.094 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 61489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-15T02:42:21.094 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 141225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-15T02:42:21.094 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-15T02:42:21.094 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 62817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-15T02:42:21.095 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-15T02:42:21.095 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dlink/ 2023-11-15T02:42:21.095 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 60873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-15T02:42:21.095 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 62833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-15T02:42:21.095 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 45721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dnet.ko 2023-11-15T02:42:21.096 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ec_bhf.ko 2023-11-15T02:42:21.096 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/emulex/ 2023-11-15T02:42:21.096 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/emulex/benet/ 2023-11-15T02:42:21.096 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 281137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-15T02:42:21.096 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ethoc.ko 2023-11-15T02:42:21.097 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/fealnx.ko 2023-11-15T02:42:21.097 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/fujitsu/ 2023-11-15T02:42:21.097 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 45345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-11-15T02:42:21.097 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ 2023-11-15T02:42:21.098 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 90465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-15T02:42:21.098 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e1000/ 2023-11-15T02:42:21.098 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 295577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-15T02:42:21.098 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e1000e/ 2023-11-15T02:42:21.098 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 584497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-15T02:42:21.099 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/fm10k/ 2023-11-15T02:42:21.099 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 246209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-15T02:42:21.099 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/i40e/ 2023-11-15T02:42:21.099 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 925697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-15T02:42:21.100 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/iavf/ 2023-11-15T02:42:21.100 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 302849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-15T02:42:21.100 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/igb/ 2023-11-15T02:42:21.100 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 514681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-15T02:42:21.100 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/igbvf/ 2023-11-15T02:42:21.101 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 93593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-15T02:42:21.101 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-11-15T02:42:21.101 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 734305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-15T02:42:21.101 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-11-15T02:42:21.101 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 151769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-15T02:42:21.102 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 86921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/jme.ko 2023-11-15T02:42:21.102 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/marvell/ 2023-11-15T02:42:21.102 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-15T02:42:21.102 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 102289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-15T02:42:21.102 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 122649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-15T02:42:21.103 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/ 2023-11-15T02:42:21.103 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-11-15T02:42:21.103 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 727601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-15T02:42:21.103 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 278937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-15T02:42:21.103 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-11-15T02:42:21.104 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-11-15T02:42:21.104 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1113993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-15T02:42:21.104 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ 2023-11-15T02:42:21.104 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-11-15T02:42:21.104 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-11-15T02:42:21.104 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-11-15T02:42:21.105 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 95665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-15T02:42:21.105 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/myricom/ 2023-11-15T02:42:21.105 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-11-15T02:42:21.105 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 108649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-15T02:42:21.105 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/natsemi/ 2023-11-15T02:42:21.106 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 73441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-15T02:42:21.106 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-15T02:42:21.106 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/neterion/ 2023-11-15T02:42:21.106 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 150905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-15T02:42:21.106 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/nvidia/ 2023-11-15T02:42:21.107 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 136721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-15T02:42:21.107 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/packetengines/ 2023-11-15T02:42:21.107 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-15T02:42:21.107 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 43145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-15T02:42:21.108 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/ 2023-11-15T02:42:21.108 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-11-15T02:42:21.108 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 227897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-15T02:42:21.108 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 87081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-15T02:42:21.108 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-11-15T02:42:21.109 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 672049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-15T02:42:21.109 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/rdc/ 2023-11-15T02:42:21.109 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 43017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-15T02:42:21.109 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/ 2023-11-15T02:42:21.109 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 67385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-15T02:42:21.110 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 83985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-15T02:42:21.110 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-15T02:42:21.110 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 189801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-15T02:42:21.110 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/samsung/ 2023-11-15T02:42:21.110 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-11-15T02:42:21.111 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 104241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-11-15T02:42:21.111 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sfc/ 2023-11-15T02:42:21.111 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1053921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-15T02:42:21.111 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/silan/ 2023-11-15T02:42:21.111 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-15T02:42:21.112 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sis/ 2023-11-15T02:42:21.112 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-15T02:42:21.112 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 63337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-15T02:42:21.112 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/ 2023-11-15T02:42:21.112 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 61601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-15T02:42:21.112 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 67193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-15T02:42:21.113 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 73409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-15T02:42:21.113 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 68177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-15T02:42:21.113 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/ 2023-11-15T02:42:21.113 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-11-15T02:42:21.113 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-11-15T02:42:21.114 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-15T02:42:21.114 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-11-15T02:42:21.114 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 486961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-15T02:42:21.114 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/ 2023-11-15T02:42:21.114 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 103337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-15T02:42:21.115 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 166161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-15T02:42:21.115 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 66329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-15T02:42:21.115 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 78801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-15T02:42:21.115 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/tehuti/ 2023-11-15T02:42:21.115 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-15T02:42:21.116 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ti/ 2023-11-15T02:42:21.116 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 86529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-15T02:42:21.116 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/via/ 2023-11-15T02:42:21.116 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 80273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-15T02:42:21.116 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 83633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-15T02:42:21.117 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/wiznet/ 2023-11-15T02:42:21.117 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 46241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-15T02:42:21.117 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-15T02:42:21.117 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/xircom/ 2023-11-15T02:42:21.117 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 60057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-15T02:42:21.118 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/fddi/ 2023-11-15T02:42:21.118 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/fddi/defxx.ko 2023-11-15T02:42:21.118 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/fddi/skfp/ 2023-11-15T02:42:21.118 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 216985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/fddi/skfp/skfp.ko 2023-11-15T02:42:21.118 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 77345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/geneve.ko 2023-11-15T02:42:21.119 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/hyperv/ 2023-11-15T02:42:21.119 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 235521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-15T02:42:21.119 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ieee802154/ 2023-11-15T02:42:21.119 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-15T02:42:21.119 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ifb.ko 2023-11-15T02:42:21.120 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ipvlan/ 2023-11-15T02:42:21.120 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 83769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-15T02:42:21.120 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 62209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/macvlan.ko 2023-11-15T02:42:21.120 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/macvtap.ko 2023-11-15T02:42:21.120 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/ 2023-11-15T02:42:21.121 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-15T02:42:21.121 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-15T02:42:21.121 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/mdio-gpio.ko 2023-11-15T02:42:21.121 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-15T02:42:21.121 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio.ko 2023-11-15T02:42:21.122 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mii.ko 2023-11-15T02:42:21.122 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 43953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/netconsole.ko 2023-11-15T02:42:21.122 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/nlmon.ko 2023-11-15T02:42:21.122 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/pcs/ 2023-11-15T02:42:21.122 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-15T02:42:21.123 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-15T02:42:21.123 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/ 2023-11-15T02:42:21.123 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/amd.ko 2023-11-15T02:42:21.123 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-15T02:42:21.123 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-15T02:42:21.124 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/bcm87xx.ko 2023-11-15T02:42:21.124 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/broadcom.ko 2023-11-15T02:42:21.124 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/cicada.ko 2023-11-15T02:42:21.124 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/davicom.ko 2023-11-15T02:42:21.124 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/et1011c.ko 2023-11-15T02:42:21.124 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/icplus.ko 2023-11-15T02:42:21.125 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/lxt.ko 2023-11-15T02:42:21.125 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 82049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/marvell.ko 2023-11-15T02:42:21.125 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 76177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/micrel.ko 2023-11-15T02:42:21.125 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/national.ko 2023-11-15T02:42:21.125 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 97857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/phylink.ko 2023-11-15T02:42:21.126 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/qsemi.ko 2023-11-15T02:42:21.126 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/realtek.ko 2023-11-15T02:42:21.126 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/smsc.ko 2023-11-15T02:42:21.126 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/ste10Xp.ko 2023-11-15T02:42:21.126 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/vitesse.ko 2023-11-15T02:42:21.127 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/plip/ 2023-11-15T02:42:21.127 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 43569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/plip/plip.ko 2023-11-15T02:42:21.127 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ 2023-11-15T02:42:21.127 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-15T02:42:21.127 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ppp_async.ko 2023-11-15T02:42:21.128 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-15T02:42:21.128 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-15T02:42:21.128 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-15T02:42:21.128 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/pppoe.ko 2023-11-15T02:42:21.128 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/pppox.ko 2023-11-15T02:42:21.129 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/pptp.ko 2023-11-15T02:42:21.129 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/sb1000.ko 2023-11-15T02:42:21.129 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/slip/ 2023-11-15T02:42:21.129 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/slip/slip.ko 2023-11-15T02:42:21.129 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/sungem_phy.ko 2023-11-15T02:42:21.130 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 64385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/tap.ko 2023-11-15T02:42:21.130 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/ 2023-11-15T02:42:21.130 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 106169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team.ko 2023-11-15T02:42:21.130 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-15T02:42:21.130 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-15T02:42:21.131 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-15T02:42:21.131 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_random.ko 2023-11-15T02:42:21.131 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-15T02:42:21.131 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 87017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/veth.ko 2023-11-15T02:42:21.131 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/vmxnet3/ 2023-11-15T02:42:21.131 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 144297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-15T02:42:21.132 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/vxlan/ 2023-11-15T02:42:21.132 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 297329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/vxlan/vxlan.ko 2023-11-15T02:42:21.132 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/xen-netback/ 2023-11-15T02:42:21.132 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 144649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-15T02:42:21.132 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ntb/ 2023-11-15T02:42:21.133 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ntb/ntb.ko 2023-11-15T02:42:21.133 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/nvme/ 2023-11-15T02:42:21.133 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/nvme/host/ 2023-11-15T02:42:21.133 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 264937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/nvme/host/nvme-core.ko 2023-11-15T02:42:21.133 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 102209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/nvme/host/nvme.ko 2023-11-15T02:42:21.134 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/ 2023-11-15T02:42:21.134 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 133841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/parport.ko 2023-11-15T02:42:21.134 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/parport_cs.ko 2023-11-15T02:42:21.134 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 84465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/parport_pc.ko 2023-11-15T02:42:21.134 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/parport_serial.ko 2023-11-15T02:42:21.135 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/ 2023-11-15T02:42:21.135 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/hotplug/ 2023-11-15T02:42:21.135 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-15T02:42:21.135 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-11-15T02:42:21.135 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-11-15T02:42:21.136 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/pci-stub.ko 2023-11-15T02:42:21.136 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 46529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/xen-pcifront.ko 2023-11-15T02:42:21.136 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/ 2023-11-15T02:42:21.136 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/i82092.ko 2023-11-15T02:42:21.136 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 145809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/pcmcia.ko 2023-11-15T02:42:21.136 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 62137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/pcmcia_core.ko 2023-11-15T02:42:21.137 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-11-15T02:42:21.137 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/pd6729.ko 2023-11-15T02:42:21.137 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 112889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-15T02:42:21.137 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/ 2023-11-15T02:42:21.137 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/broadcom/ 2023-11-15T02:42:21.138 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-11-15T02:42:21.138 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/samsung/ 2023-11-15T02:42:21.138 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-11-15T02:42:21.138 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pinctrl/ 2023-11-15T02:42:21.138 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pinctrl/intel/ 2023-11-15T02:42:21.139 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 70153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-15T02:42:21.139 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/ 2023-11-15T02:42:21.139 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/ 2023-11-15T02:42:21.139 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-11-15T02:42:21.139 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec.ko 2023-11-15T02:42:21.139 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-11-15T02:42:21.140 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-11-15T02:42:21.140 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-11-15T02:42:21.140 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-11-15T02:42:21.140 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-11-15T02:42:21.140 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/ 2023-11-15T02:42:21.141 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 67097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-15T02:42:21.141 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/acerhdf.ko 2023-11-15T02:42:21.141 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-15T02:42:21.141 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 59665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-15T02:42:21.141 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-15T02:42:21.142 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 127569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-15T02:42:21.142 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-15T02:42:21.142 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-15T02:42:21.142 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-15T02:42:21.142 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-15T02:42:21.143 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-15T02:42:21.143 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/hdaps.ko 2023-11-15T02:42:21.143 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/ibm_rtl.ko 2023-11-15T02:42:21.143 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/intel/ 2023-11-15T02:42:21.143 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-15T02:42:21.144 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-11-15T02:42:21.144 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 43497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/intel_ips.ko 2023-11-15T02:42:21.144 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-15T02:42:21.144 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-15T02:42:21.144 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-15T02:42:21.145 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-15T02:42:21.145 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-15T02:42:21.145 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-15T02:42:21.145 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-15T02:42:21.145 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-15T02:42:21.145 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-15T02:42:21.146 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 57569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/wmi.ko 2023-11-15T02:42:21.146 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/ 2023-11-15T02:42:21.146 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ 2023-11-15T02:42:21.146 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/88pm860x_battery.ko 2023-11-15T02:42:21.146 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/88pm860x_charger.ko 2023-11-15T02:42:21.147 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 55057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/bq2415x_charger.ko 2023-11-15T02:42:21.147 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/bq24190_charger.ko 2023-11-15T02:42:21.147 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/bq24735-charger.ko 2023-11-15T02:42:21.147 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-11-15T02:42:21.147 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/da9030_battery.ko 2023-11-15T02:42:21.148 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/da9052-battery.ko 2023-11-15T02:42:21.148 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ds2760_battery.ko 2023-11-15T02:42:21.148 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ds2780_battery.ko 2023-11-15T02:42:21.148 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ds2781_battery.ko 2023-11-15T02:42:21.148 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ds2782_battery.ko 2023-11-15T02:42:21.149 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/gpio-charger.ko 2023-11-15T02:42:21.149 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/isp1704_charger.ko 2023-11-15T02:42:21.149 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/lp8727_charger.ko 2023-11-15T02:42:21.149 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max14577_charger.ko 2023-11-15T02:42:21.149 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max17040_battery.ko 2023-11-15T02:42:21.149 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max17042_battery.ko 2023-11-15T02:42:21.150 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max8903_charger.ko 2023-11-15T02:42:21.150 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max8925_power.ko 2023-11-15T02:42:21.150 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/pcf50633-charger.ko 2023-11-15T02:42:21.150 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/sbs-battery.ko 2023-11-15T02:42:21.150 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/test_power.ko 2023-11-15T02:42:21.151 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/tps65090-charger.ko 2023-11-15T02:42:21.151 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/wm831x_backup.ko 2023-11-15T02:42:21.151 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/wm831x_power.ko 2023-11-15T02:42:21.151 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/wm8350_power.ko 2023-11-15T02:42:21.151 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/powercap/ 2023-11-15T02:42:21.152 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 57729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-15T02:42:21.152 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-15T02:42:21.152 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/ 2023-11-15T02:42:21.152 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/clients/ 2023-11-15T02:42:21.152 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-15T02:42:21.152 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-15T02:42:21.153 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/clients/pps_parport.ko 2023-11-15T02:42:21.153 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/pps_core.ko 2023-11-15T02:42:21.153 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ptp/ 2023-11-15T02:42:21.153 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 66385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ptp/ptp.ko 2023-11-15T02:42:21.154 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ptp/ptp_kvm.ko 2023-11-15T02:42:21.154 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/ 2023-11-15T02:42:21.154 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-88pm80x.ko 2023-11-15T02:42:21.154 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-88pm860x.ko 2023-11-15T02:42:21.154 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-15T02:42:21.155 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-da9052.ko 2023-11-15T02:42:21.155 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-da9055.ko 2023-11-15T02:42:21.155 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-da9063.ko 2023-11-15T02:42:21.155 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-15T02:42:21.155 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 68577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-15T02:42:21.155 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-15T02:42:21.156 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-15T02:42:21.156 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-15T02:42:21.156 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-15T02:42:21.156 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-15T02:42:21.156 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-15T02:42:21.157 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-15T02:42:21.157 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-em3027.ko 2023-11-15T02:42:21.157 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-15T02:42:21.157 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-15T02:42:21.157 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-15T02:42:21.158 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-lp8788.ko 2023-11-15T02:42:21.158 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-15T02:42:21.158 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-15T02:42:21.158 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-15T02:42:21.158 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-m48t86.ko 2023-11-15T02:42:21.159 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max6900.ko 2023-11-15T02:42:21.159 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max8907.ko 2023-11-15T02:42:21.159 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max8925.ko 2023-11-15T02:42:21.159 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max8997.ko 2023-11-15T02:42:21.159 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max8998.ko 2023-11-15T02:42:21.159 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-11-15T02:42:21.160 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-15T02:42:21.160 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-palmas.ko 2023-11-15T02:42:21.160 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 46633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-15T02:42:21.160 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf50633.ko 2023-11-15T02:42:21.160 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-15T02:42:21.161 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-15T02:42:21.161 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-15T02:42:21.161 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-15T02:42:21.161 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rc5t583.ko 2023-11-15T02:42:21.161 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-15T02:42:21.161 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-15T02:42:21.162 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-15T02:42:21.162 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-15T02:42:21.162 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-15T02:42:21.162 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-s35390a.ko 2023-11-15T02:42:21.162 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-15T02:42:21.163 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-tps6586x.ko 2023-11-15T02:42:21.163 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-tps65910.ko 2023-11-15T02:42:21.163 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-wm831x.ko 2023-11-15T02:42:21.163 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-wm8350.ko 2023-11-15T02:42:21.163 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-x1205.ko 2023-11-15T02:42:21.164 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ 2023-11-15T02:42:21.164 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 77617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/3w-9xxx.ko 2023-11-15T02:42:21.164 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 57665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/3w-sas.ko 2023-11-15T02:42:21.164 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 55169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/3w-xxxx.ko 2023-11-15T02:42:21.164 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 135185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/BusLogic.ko 2023-11-15T02:42:21.165 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/a100u2w.ko 2023-11-15T02:42:21.165 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aacraid/ 2023-11-15T02:42:21.165 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 272305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-15T02:42:21.165 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 100617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/advansys.ko 2023-11-15T02:42:21.165 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic7xxx/ 2023-11-15T02:42:21.166 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 356265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-15T02:42:21.166 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 295801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-15T02:42:21.166 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic94xx/ 2023-11-15T02:42:21.166 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 189185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-11-15T02:42:21.166 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/am53c974.ko 2023-11-15T02:42:21.167 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/arcmsr/ 2023-11-15T02:42:21.167 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 106881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-15T02:42:21.167 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/atp870u.ko 2023-11-15T02:42:21.167 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/be2iscsi/ 2023-11-15T02:42:21.167 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 246985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-15T02:42:21.168 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bfa/ 2023-11-15T02:42:21.168 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1171945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bfa/bfa.ko 2023-11-15T02:42:21.168 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bnx2fc/ 2023-11-15T02:42:21.168 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 237273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-15T02:42:21.168 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bnx2i/ 2023-11-15T02:42:21.169 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 115369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-15T02:42:21.169 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ch.ko 2023-11-15T02:42:21.169 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/csiostor/ 2023-11-15T02:42:21.169 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 308921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-15T02:42:21.169 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/ 2023-11-15T02:42:21.169 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-11-15T02:42:21.170 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 76145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-15T02:42:21.170 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-11-15T02:42:21.170 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 101449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-15T02:42:21.170 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 146689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-15T02:42:21.171 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 75553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/dc395x.ko 2023-11-15T02:42:21.171 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/dmx3191d.ko 2023-11-15T02:42:21.171 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/esas2r/ 2023-11-15T02:42:21.171 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 212425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-15T02:42:21.171 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 60505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/esp_scsi.ko 2023-11-15T02:42:21.172 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fcoe/ 2023-11-15T02:42:21.172 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 78209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-15T02:42:21.172 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 158505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-15T02:42:21.172 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fdomain.ko 2023-11-15T02:42:21.172 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fnic/ 2023-11-15T02:42:21.173 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 263489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fnic/fnic.ko 2023-11-15T02:42:21.173 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 201817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/hpsa.ko 2023-11-15T02:42:21.173 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/hptiop.ko 2023-11-15T02:42:21.173 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/hv_storvsc.ko 2023-11-15T02:42:21.173 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/imm.ko 2023-11-15T02:42:21.174 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/initio.ko 2023-11-15T02:42:21.174 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 266537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ipr.ko 2023-11-15T02:42:21.174 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 101881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ips.ko 2023-11-15T02:42:21.174 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/isci/ 2023-11-15T02:42:21.174 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 317153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/isci/isci.ko 2023-11-15T02:42:21.175 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-15T02:42:21.175 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 52961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-15T02:42:21.175 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libfc/ 2023-11-15T02:42:21.175 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 314865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libfc/libfc.ko 2023-11-15T02:42:21.175 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 145585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libiscsi.ko 2023-11-15T02:42:21.176 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-15T02:42:21.176 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libsas/ 2023-11-15T02:42:21.176 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 216665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libsas/libsas.ko 2023-11-15T02:42:21.176 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/lpfc/ 2023-11-15T02:42:21.176 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 2197249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-15T02:42:21.177 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid/ 2023-11-15T02:42:21.177 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 73569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-15T02:42:21.177 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-15T02:42:21.177 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 310617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-15T02:42:21.177 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 84721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid.ko 2023-11-15T02:42:21.178 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mpt3sas/ 2023-11-15T02:42:21.178 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 659777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-15T02:42:21.178 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mvsas/ 2023-11-15T02:42:21.178 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 119561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-15T02:42:21.178 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 78129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mvumi.ko 2023-11-15T02:42:21.179 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/ 2023-11-15T02:42:21.179 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 81625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-11-15T02:42:21.179 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-11-15T02:42:21.179 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-11-15T02:42:21.179 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-11-15T02:42:21.179 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pm8001/ 2023-11-15T02:42:21.180 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 364105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-15T02:42:21.180 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 123617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pmcraid.ko 2023-11-15T02:42:21.180 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ppa.ko 2023-11-15T02:42:21.180 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 55361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla1280.ko 2023-11-15T02:42:21.181 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla2xxx/ 2023-11-15T02:42:21.181 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1876625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-15T02:42:21.181 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 85289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-15T02:42:21.181 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla4xxx/ 2023-11-15T02:42:21.181 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 559033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-15T02:42:21.181 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qlogicfas408.ko 2023-11-15T02:42:21.182 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/raid_class.ko 2023-11-15T02:42:21.182 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 238825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_debug.ko 2023-11-15T02:42:21.182 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 173937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-15T02:42:21.182 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 279313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-15T02:42:21.182 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 97729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-15T02:42:21.183 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 73881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-15T02:42:21.183 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-15T02:42:21.183 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ses.ko 2023-11-15T02:42:21.183 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 115889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/st.ko 2023-11-15T02:42:21.183 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 46825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/stex.ko 2023-11-15T02:42:21.184 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/sym53c8xx_2/ 2023-11-15T02:42:21.184 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 162121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-15T02:42:21.184 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/virtio_scsi.ko 2023-11-15T02:42:21.184 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-15T02:42:21.184 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/wd719x.ko 2023-11-15T02:42:21.185 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/xen-scsifront.ko 2023-11-15T02:42:21.185 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ssb/ 2023-11-15T02:42:21.185 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 129441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ssb/ssb.ko 2023-11-15T02:42:21.185 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/ 2023-11-15T02:42:21.185 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/iscsi/ 2023-11-15T02:42:21.185 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 736601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-15T02:42:21.186 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/loopback/ 2023-11-15T02:42:21.186 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-15T02:42:21.186 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_file.ko 2023-11-15T02:42:21.186 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 43025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_iblock.ko 2023-11-15T02:42:21.186 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 930321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_mod.ko 2023-11-15T02:42:21.187 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_pscsi.ko 2023-11-15T02:42:21.187 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 122521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_user.ko 2023-11-15T02:42:21.187 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/tcm_fc/ 2023-11-15T02:42:21.187 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 75593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-15T02:42:21.187 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/ 2023-11-15T02:42:21.188 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/ 2023-11-15T02:42:21.188 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/ 2023-11-15T02:42:21.188 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-15T02:42:21.188 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-15T02:42:21.188 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-15T02:42:21.189 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-15T02:42:21.189 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-15T02:42:21.189 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-15T02:42:21.189 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-15T02:42:21.189 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-15T02:42:21.190 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-15T02:42:21.190 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-15T02:42:21.190 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-15T02:42:21.190 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-15T02:42:21.190 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-11-15T02:42:21.191 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-15T02:42:21.191 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-15T02:42:21.191 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-15T02:42:21.191 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/ 2023-11-15T02:42:21.191 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/ipwireless/ 2023-11-15T02:42:21.192 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 78425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-15T02:42:21.192 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 57865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/moxa.ko 2023-11-15T02:42:21.192 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 50953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/mxser.ko 2023-11-15T02:42:21.192 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 108601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/n_gsm.ko 2023-11-15T02:42:21.192 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/n_hdlc.ko 2023-11-15T02:42:21.193 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 67929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/nozomi.ko 2023-11-15T02:42:21.193 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/ 2023-11-15T02:42:21.193 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/8250/ 2023-11-15T02:42:21.193 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-11-15T02:42:21.193 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 59497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-15T02:42:21.193 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-11-15T02:42:21.194 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/altera_uart.ko 2023-11-15T02:42:21.194 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/arc_uart.ko 2023-11-15T02:42:21.194 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 72425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-11-15T02:42:21.194 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/jsm/ 2023-11-15T02:42:21.197 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 103073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-15T02:42:21.197 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/rp2.ko 2023-11-15T02:42:21.197 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 52817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/sc16is7xx.ko 2023-11-15T02:42:21.197 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/ 2023-11-15T02:42:21.198 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/core/ 2023-11-15T02:42:21.198 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 531265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-15T02:42:21.198 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/host/ 2023-11-15T02:42:21.198 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-15T02:42:21.198 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-11-15T02:42:21.199 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/ 2023-11-15T02:42:21.199 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio.ko 2023-11-15T02:42:21.199 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_aec.ko 2023-11-15T02:42:21.199 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_cif.ko 2023-11-15T02:42:21.199 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_dmem_genirq.ko 2023-11-15T02:42:21.200 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_mf624.ko 2023-11-15T02:42:21.200 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_netx.ko 2023-11-15T02:42:21.200 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_pci_generic.ko 2023-11-15T02:42:21.200 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-11-15T02:42:21.200 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_sercos3.ko 2023-11-15T02:42:21.200 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/ 2023-11-15T02:42:21.201 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/c67x00/ 2023-11-15T02:42:21.201 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 73297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/c67x00/c67x00.ko 2023-11-15T02:42:21.201 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ 2023-11-15T02:42:21.201 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 144377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-11-15T02:42:21.201 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-11-15T02:42:21.202 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-11-15T02:42:21.202 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-11-15T02:42:21.202 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/ 2023-11-15T02:42:21.202 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 87089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/cdc-acm.ko 2023-11-15T02:42:21.203 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-15T02:42:21.203 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/usblp.ko 2023-11-15T02:42:21.203 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 89857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/usbtmc.ko 2023-11-15T02:42:21.203 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/common/ 2023-11-15T02:42:21.203 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/common/ulpi.ko 2023-11-15T02:42:21.203 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc2/ 2023-11-15T02:42:21.204 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-11-15T02:42:21.204 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc3/ 2023-11-15T02:42:21.204 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-11-15T02:42:21.204 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-11-15T02:42:21.205 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 444553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc3/dwc3.ko 2023-11-15T02:42:21.205 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/ 2023-11-15T02:42:21.205 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/ 2023-11-15T02:42:21.205 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 60177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/u_ether.ko 2023-11-15T02:42:21.205 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/u_serial.ko 2023-11-15T02:42:21.206 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-11-15T02:42:21.206 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-11-15T02:42:21.206 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-11-15T02:42:21.206 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-11-15T02:42:21.206 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 92489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-11-15T02:42:21.207 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-11-15T02:42:21.207 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 122081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-11-15T02:42:21.207 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 57617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-11-15T02:42:21.207 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-11-15T02:42:21.207 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-11-15T02:42:21.207 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 57729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-11-15T02:42:21.208 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 85609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-11-15T02:42:21.208 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-11-15T02:42:21.208 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 58113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-11-15T02:42:21.208 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 67337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-11-15T02:42:21.208 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/ 2023-11-15T02:42:21.209 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-11-15T02:42:21.209 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-11-15T02:42:21.209 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-11-15T02:42:21.209 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-11-15T02:42:21.209 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-11-15T02:42:21.210 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-11-15T02:42:21.210 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-11-15T02:42:21.210 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-11-15T02:42:21.210 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-11-15T02:42:21.210 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-11-15T02:42:21.211 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-11-15T02:42:21.211 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-11-15T02:42:21.211 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 62185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-11-15T02:42:21.211 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-11-15T02:42:21.211 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 161225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/libcomposite.ko 2023-11-15T02:42:21.212 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/ 2023-11-15T02:42:21.212 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-11-15T02:42:21.212 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/bdc/ 2023-11-15T02:42:21.212 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 157681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-11-15T02:42:21.212 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-11-15T02:42:21.213 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 52721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-11-15T02:42:21.213 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 69705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-11-15T02:42:21.213 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 56033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-11-15T02:42:21.213 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 76057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/net2272.ko 2023-11-15T02:42:21.213 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 75073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/net2280.ko 2023-11-15T02:42:21.213 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 73169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-11-15T02:42:21.214 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 65809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-11-15T02:42:21.214 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 66097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-11-15T02:42:21.214 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 81505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-11-15T02:42:21.214 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 156073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-11-15T02:42:21.214 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/ 2023-11-15T02:42:21.215 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/bcma-hcd.ko 2023-11-15T02:42:21.215 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 80113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/isp116x-hcd.ko 2023-11-15T02:42:21.215 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 87353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-11-15T02:42:21.215 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 78873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-11-15T02:42:21.215 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-15T02:42:21.216 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/sl811_cs.ko 2023-11-15T02:42:21.216 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/ssb-hcd.ko 2023-11-15T02:42:21.216 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-11-15T02:42:21.216 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/image/ 2023-11-15T02:42:21.216 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/image/mdc800.ko 2023-11-15T02:42:21.217 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/image/microtek.ko 2023-11-15T02:42:21.217 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/ 2023-11-15T02:42:21.217 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/adutux.ko 2023-11-15T02:42:21.217 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/appledisplay.ko 2023-11-15T02:42:21.217 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-11-15T02:42:21.217 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/cytherm.ko 2023-11-15T02:42:21.218 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/ehset.ko 2023-11-15T02:42:21.218 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/emi26.ko 2023-11-15T02:42:21.218 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/emi62.ko 2023-11-15T02:42:21.218 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/ezusb.ko 2023-11-15T02:42:21.218 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/idmouse.ko 2023-11-15T02:42:21.219 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/iowarrior.ko 2023-11-15T02:42:21.219 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-15T02:42:21.219 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/ldusb.ko 2023-11-15T02:42:21.219 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/legousbtower.ko 2023-11-15T02:42:21.219 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/lvstest.ko 2023-11-15T02:42:21.220 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/sisusbvga/ 2023-11-15T02:42:21.220 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 64841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-15T02:42:21.220 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-15T02:42:21.220 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/usb3503.ko 2023-11-15T02:42:21.220 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/usblcd.ko 2023-11-15T02:42:21.221 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-15T02:42:21.221 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 70873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/usbtest.ko 2023-11-15T02:42:21.221 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/uss720.ko 2023-11-15T02:42:21.221 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/yurex.ko 2023-11-15T02:42:21.221 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/mon/ 2023-11-15T02:42:21.222 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 72953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/mon/usbmon.ko 2023-11-15T02:42:21.222 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/musb/ 2023-11-15T02:42:21.222 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 301545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/musb/musb_hdrc.ko 2023-11-15T02:42:21.222 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/ 2023-11-15T02:42:21.222 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/phy-generic.ko 2023-11-15T02:42:21.222 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-11-15T02:42:21.223 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/phy-isp1301.ko 2023-11-15T02:42:21.223 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/phy-tahvo.ko 2023-11-15T02:42:21.223 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ 2023-11-15T02:42:21.223 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/aircable.ko 2023-11-15T02:42:21.223 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ark3116.ko 2023-11-15T02:42:21.224 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-15T02:42:21.224 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ch341.ko 2023-11-15T02:42:21.224 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 70257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/cp210x.ko 2023-11-15T02:42:21.224 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/cyberjack.ko 2023-11-15T02:42:21.224 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 58217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-15T02:42:21.225 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-15T02:42:21.225 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/empeg.ko 2023-11-15T02:42:21.225 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/f81232.ko 2023-11-15T02:42:21.225 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 175137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-15T02:42:21.225 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-15T02:42:21.226 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 137081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-15T02:42:21.226 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 124833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/io_ti.ko 2023-11-15T02:42:21.226 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 67769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ipaq.ko 2023-11-15T02:42:21.226 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ipw.ko 2023-11-15T02:42:21.226 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ir-usb.ko 2023-11-15T02:42:21.227 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 58233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-15T02:42:21.227 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 107521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/keyspan.ko 2023-11-15T02:42:21.227 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-15T02:42:21.227 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-15T02:42:21.227 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-15T02:42:21.228 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/mct_u232.ko 2023-11-15T02:42:21.228 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/metro-usb.ko 2023-11-15T02:42:21.228 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 66449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/mos7720.ko 2023-11-15T02:42:21.228 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 78217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/mos7840.ko 2023-11-15T02:42:21.228 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 56337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/mxuport.ko 2023-11-15T02:42:21.229 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/navman.ko 2023-11-15T02:42:21.229 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/omninet.ko 2023-11-15T02:42:21.229 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/opticon.ko 2023-11-15T02:42:21.229 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 190633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/option.ko 2023-11-15T02:42:21.229 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/oti6858.ko 2023-11-15T02:42:21.230 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/pl2303.ko 2023-11-15T02:42:21.230 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/qcaux.ko 2023-11-15T02:42:21.230 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/qcserial.ko 2023-11-15T02:42:21.230 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/quatech2.ko 2023-11-15T02:42:21.230 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/safe_serial.ko 2023-11-15T02:42:21.231 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 51625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/sierra.ko 2023-11-15T02:42:21.231 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-15T02:42:21.231 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ssu100.ko 2023-11-15T02:42:21.231 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/symbolserial.ko 2023-11-15T02:42:21.231 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 61489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-15T02:42:21.232 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-15T02:42:21.232 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/usb_debug.ko 2023-11-15T02:42:21.232 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-15T02:42:21.232 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 125609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/usbserial.ko 2023-11-15T02:42:21.232 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/visor.ko 2023-11-15T02:42:21.232 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/whiteheat.ko 2023-11-15T02:42:21.233 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/wishbone-serial.ko 2023-11-15T02:42:21.233 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-15T02:42:21.233 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ 2023-11-15T02:42:21.233 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 52913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/uas.ko 2023-11-15T02:42:21.233 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-15T02:42:21.234 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-15T02:42:21.234 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-15T02:42:21.234 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-15T02:42:21.234 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-15T02:42:21.234 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-15T02:42:21.235 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-15T02:42:21.235 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-karma.ko 2023-11-15T02:42:21.235 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-15T02:42:21.235 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-15T02:42:21.235 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-15T02:42:21.236 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-15T02:42:21.236 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-15T02:42:21.236 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 179969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/usb-storage.ko 2023-11-15T02:42:21.236 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/usbip/ 2023-11-15T02:42:21.236 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 67481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-15T02:42:21.236 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 72177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-15T02:42:21.237 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 107001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-15T02:42:21.237 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/ 2023-11-15T02:42:21.237 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/pci/ 2023-11-15T02:42:21.237 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 154297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-15T02:42:21.237 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-15T02:42:21.238 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 108737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/vfio.ko 2023-11-15T02:42:21.238 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 77609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-15T02:42:21.238 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/ 2023-11-15T02:42:21.238 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 113873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/vhost.ko 2023-11-15T02:42:21.238 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-15T02:42:21.239 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 58089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/vhost_net.ko 2023-11-15T02:42:21.239 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 89249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/vhost_scsi.ko 2023-11-15T02:42:21.239 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/ 2023-11-15T02:42:21.239 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/ 2023-11-15T02:42:21.239 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-11-15T02:42:21.240 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/aat2870_bl.ko 2023-11-15T02:42:21.240 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/adp5520_bl.ko 2023-11-15T02:42:21.240 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/adp8860_bl.ko 2023-11-15T02:42:21.240 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/adp8870_bl.ko 2023-11-15T02:42:21.240 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/apple_bl.ko 2023-11-15T02:42:21.241 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/as3711_bl.ko 2023-11-15T02:42:21.241 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/bd6107.ko 2023-11-15T02:42:21.241 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/da903x_bl.ko 2023-11-15T02:42:21.241 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/da9052_bl.ko 2023-11-15T02:42:21.241 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/gpio_backlight.ko 2023-11-15T02:42:21.242 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/kb3886_bl.ko 2023-11-15T02:42:21.242 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/lcd.ko 2023-11-15T02:42:21.242 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/lm3533_bl.ko 2023-11-15T02:42:21.242 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/lm3639_bl.ko 2023-11-15T02:42:21.242 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/lv5207lp.ko 2023-11-15T02:42:21.243 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/max8925_bl.ko 2023-11-15T02:42:21.243 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/pandora_bl.ko 2023-11-15T02:42:21.243 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-11-15T02:42:21.243 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-15T02:42:21.243 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/wm831x_bl.ko 2023-11-15T02:42:21.244 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/ 2023-11-15T02:42:21.244 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/ 2023-11-15T02:42:21.244 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/ds2482.ko 2023-11-15T02:42:21.244 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/ds2490.ko 2023-11-15T02:42:21.244 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/matrox_w1.ko 2023-11-15T02:42:21.244 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/w1-gpio.ko 2023-11-15T02:42:21.245 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/ 2023-11-15T02:42:21.245 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-15T02:42:21.245 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-15T02:42:21.245 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-15T02:42:21.245 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-15T02:42:21.246 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-15T02:42:21.246 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-15T02:42:21.246 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-15T02:42:21.246 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-15T02:42:21.247 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-15T02:42:21.247 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-15T02:42:21.247 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 56425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-15T02:42:21.247 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 102673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/wire.ko 2023-11-15T02:42:21.247 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/ 2023-11-15T02:42:21.247 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/acquirewdt.ko 2023-11-15T02:42:21.248 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/advantechwdt.ko 2023-11-15T02:42:21.248 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-15T02:42:21.248 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-15T02:42:21.248 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/cpu5wdt.ko 2023-11-15T02:42:21.248 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/da9052_wdt.ko 2023-11-15T02:42:21.249 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/da9055_wdt.ko 2023-11-15T02:42:21.249 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/da9063_wdt.ko 2023-11-15T02:42:21.249 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/dw_wdt.ko 2023-11-15T02:42:21.249 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/eurotechwdt.ko 2023-11-15T02:42:21.249 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-15T02:42:21.250 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/hpwdt.ko 2023-11-15T02:42:21.250 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/i6300esb.ko 2023-11-15T02:42:21.250 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-15T02:42:21.250 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-15T02:42:21.250 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/ib700wdt.ko 2023-11-15T02:42:21.251 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/ibmasr.ko 2023-11-15T02:42:21.251 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-15T02:42:21.251 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-15T02:42:21.251 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/it87_wdt.ko 2023-11-15T02:42:21.251 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/kempld_wdt.ko 2023-11-15T02:42:21.252 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/machzwd.ko 2023-11-15T02:42:21.252 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/mena21_wdt.ko 2023-11-15T02:42:21.252 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-11-15T02:42:21.252 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/nv_tco.ko 2023-11-15T02:42:21.252 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-11-15T02:42:21.253 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/pc87413_wdt.ko 2023-11-15T02:42:21.253 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-15T02:42:21.253 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-15T02:42:21.253 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/retu_wdt.ko 2023-11-15T02:42:21.254 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-11-15T02:42:21.254 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-11-15T02:42:21.254 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-15T02:42:21.254 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sc1200wdt.ko 2023-11-15T02:42:21.254 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-15T02:42:21.255 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-11-15T02:42:21.255 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/softdog.ko 2023-11-15T02:42:21.255 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-15T02:42:21.255 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/twl4030_wdt.ko 2023-11-15T02:42:21.255 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/via_wdt.ko 2023-11-15T02:42:21.256 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-15T02:42:21.256 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-15T02:42:21.256 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-15T02:42:21.256 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/wafer5823wdt.ko 2023-11-15T02:42:21.256 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/wdt_pci.ko 2023-11-15T02:42:21.257 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/wm831x_wdt.ko 2023-11-15T02:42:21.257 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/wm8350_wdt.ko 2023-11-15T02:42:21.257 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/xen_wdt.ko 2023-11-15T02:42:21.257 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/ 2023-11-15T02:42:21.257 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-evtchn.ko 2023-11-15T02:42:21.258 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-gntalloc.ko 2023-11-15T02:42:21.258 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-gntdev.ko 2023-11-15T02:42:21.258 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-pciback/ 2023-11-15T02:42:21.258 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 176441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-15T02:42:21.258 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-privcmd.ko 2023-11-15T02:42:21.259 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 71249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-scsiback.ko 2023-11-15T02:42:21.259 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xenfs/ 2023-11-15T02:42:21.259 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-15T02:42:21.259 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ 2023-11-15T02:42:21.259 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/9p/ 2023-11-15T02:42:21.260 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 166425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/9p/9p.ko 2023-11-15T02:42:21.260 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/adfs/ 2023-11-15T02:42:21.260 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 61073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/adfs/adfs.ko 2023-11-15T02:42:21.260 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/affs/ 2023-11-15T02:42:21.260 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 163593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/affs/affs.ko 2023-11-15T02:42:21.261 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/afs/ 2023-11-15T02:42:21.261 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 932817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/afs/kafs.ko 2023-11-15T02:42:21.261 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/autofs/ 2023-11-15T02:42:21.261 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 101057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/autofs/autofs4.ko 2023-11-15T02:42:21.261 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/befs/ 2023-11-15T02:42:21.262 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 64545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/befs/befs.ko 2023-11-15T02:42:21.262 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/bfs/ 2023-11-15T02:42:21.262 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/bfs/bfs.ko 2023-11-15T02:42:21.262 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/binfmt_misc.ko 2023-11-15T02:42:21.262 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/btrfs/ 2023-11-15T02:42:21.263 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 3402585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/btrfs/btrfs.ko 2023-11-15T02:42:21.263 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/cachefiles/ 2023-11-15T02:42:21.263 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 306073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/cachefiles/cachefiles.ko 2023-11-15T02:42:21.263 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ceph/ 2023-11-15T02:42:21.263 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1323945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ceph/ceph.ko 2023-11-15T02:42:21.264 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/coda/ 2023-11-15T02:42:21.264 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 100801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/coda/coda.ko 2023-11-15T02:42:21.264 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/configfs/ 2023-11-15T02:42:21.264 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 108393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/configfs/configfs.ko 2023-11-15T02:42:21.264 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/cramfs/ 2023-11-15T02:42:21.265 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 29625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/cramfs/cramfs.ko 2023-11-15T02:42:21.265 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/dlm/ 2023-11-15T02:42:21.265 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 642577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/dlm/dlm.ko 2023-11-15T02:42:21.265 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/efs/ 2023-11-15T02:42:21.265 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/efs/efs.ko 2023-11-15T02:42:21.265 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/f2fs/ 2023-11-15T02:42:21.266 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1505025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/f2fs/f2fs.ko 2023-11-15T02:42:21.266 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fat/ 2023-11-15T02:42:21.266 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fat/msdos.ko 2023-11-15T02:42:21.266 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/freevxfs/ 2023-11-15T02:42:21.266 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/freevxfs/freevxfs.ko 2023-11-15T02:42:21.267 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fscache/ 2023-11-15T02:42:21.267 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 226481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fscache/fscache.ko 2023-11-15T02:42:21.267 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fuse/ 2023-11-15T02:42:21.267 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fuse/cuse.ko 2023-11-15T02:42:21.267 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/gfs2/ 2023-11-15T02:42:21.268 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1231529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/gfs2/gfs2.ko 2023-11-15T02:42:21.268 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hfs/ 2023-11-15T02:42:21.268 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 141209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hfs/hfs.ko 2023-11-15T02:42:21.268 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hfsplus/ 2023-11-15T02:42:21.268 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 222817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hfsplus/hfsplus.ko 2023-11-15T02:42:21.269 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hpfs/ 2023-11-15T02:42:21.269 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 192329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hpfs/hpfs.ko 2023-11-15T02:42:21.269 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/isofs/ 2023-11-15T02:42:21.269 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 74065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/isofs/isofs.ko 2023-11-15T02:42:21.269 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/jfs/ 2023-11-15T02:42:21.270 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 419497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/jfs/jfs.ko 2023-11-15T02:42:21.270 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/lockd/ 2023-11-15T02:42:21.270 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 280313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/lockd/lockd.ko 2023-11-15T02:42:21.270 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/minix/ 2023-11-15T02:42:21.270 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 89193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/minix/minix.ko 2023-11-15T02:42:21.271 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/netfs/ 2023-11-15T02:42:21.271 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 124649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/netfs/netfs.ko 2023-11-15T02:42:21.271 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/ 2023-11-15T02:42:21.271 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/blocklayout/ 2023-11-15T02:42:21.271 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 81449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-15T02:42:21.271 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/filelayout/ 2023-11-15T02:42:21.272 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 59449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-15T02:42:21.272 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/flexfilelayout/ 2023-11-15T02:42:21.272 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 117513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-15T02:42:21.272 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1055649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/nfs.ko 2023-11-15T02:42:21.272 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 66441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/nfsv2.ko 2023-11-15T02:42:21.273 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 120177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/nfsv3.ko 2023-11-15T02:42:21.273 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 2041689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/nfsv4.ko 2023-11-15T02:42:21.273 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs_common/ 2023-11-15T02:42:21.273 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs_common/grace.ko 2023-11-15T02:42:21.273 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs_common/nfs_acl.ko 2023-11-15T02:42:21.274 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfsd/ 2023-11-15T02:42:21.274 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1516809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfsd/nfsd.ko 2023-11-15T02:42:21.274 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nilfs2/ 2023-11-15T02:42:21.274 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 592833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nilfs2/nilfs2.ko 2023-11-15T02:42:21.274 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/ 2023-11-15T02:42:21.275 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-celtic.ko 2023-11-15T02:42:21.275 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-centeuro.ko 2023-11-15T02:42:21.275 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-croatian.ko 2023-11-15T02:42:21.275 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-cyrillic.ko 2023-11-15T02:42:21.275 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-gaelic.ko 2023-11-15T02:42:21.275 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-greek.ko 2023-11-15T02:42:21.276 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-iceland.ko 2023-11-15T02:42:21.276 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-inuit.ko 2023-11-15T02:42:21.276 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-roman.ko 2023-11-15T02:42:21.276 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-romanian.ko 2023-11-15T02:42:21.276 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-turkish.ko 2023-11-15T02:42:21.277 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_ascii.ko 2023-11-15T02:42:21.277 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp1250.ko 2023-11-15T02:42:21.277 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp1251.ko 2023-11-15T02:42:21.277 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp1255.ko 2023-11-15T02:42:21.277 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp737.ko 2023-11-15T02:42:21.278 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp775.ko 2023-11-15T02:42:21.278 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp850.ko 2023-11-15T02:42:21.278 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp852.ko 2023-11-15T02:42:21.278 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp855.ko 2023-11-15T02:42:21.278 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp857.ko 2023-11-15T02:42:21.279 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp860.ko 2023-11-15T02:42:21.279 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp861.ko 2023-11-15T02:42:21.279 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp862.ko 2023-11-15T02:42:21.279 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp863.ko 2023-11-15T02:42:21.279 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp864.ko 2023-11-15T02:42:21.279 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp865.ko 2023-11-15T02:42:21.280 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp866.ko 2023-11-15T02:42:21.280 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp869.ko 2023-11-15T02:42:21.280 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp874.ko 2023-11-15T02:42:21.280 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 94137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp932.ko 2023-11-15T02:42:21.280 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 142673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp936.ko 2023-11-15T02:42:21.281 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 165745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp949.ko 2023-11-15T02:42:21.281 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 118121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp950.ko 2023-11-15T02:42:21.281 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_euc-jp.ko 2023-11-15T02:42:21.281 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-1.ko 2023-11-15T02:42:21.281 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-13.ko 2023-11-15T02:42:21.282 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-14.ko 2023-11-15T02:42:21.282 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-15.ko 2023-11-15T02:42:21.282 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-2.ko 2023-11-15T02:42:21.282 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-3.ko 2023-11-15T02:42:21.282 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-4.ko 2023-11-15T02:42:21.283 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-5.ko 2023-11-15T02:42:21.283 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-6.ko 2023-11-15T02:42:21.283 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-7.ko 2023-11-15T02:42:21.283 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-9.ko 2023-11-15T02:42:21.283 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_koi8-r.ko 2023-11-15T02:42:21.283 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_koi8-ru.ko 2023-11-15T02:42:21.284 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_koi8-u.ko 2023-11-15T02:42:21.284 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-15T02:42:21.284 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_utf8.ko 2023-11-15T02:42:21.284 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ntfs/ 2023-11-15T02:42:21.284 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 219417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ntfs/ntfs.ko 2023-11-15T02:42:21.285 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ 2023-11-15T02:42:21.285 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/cluster/ 2023-11-15T02:42:21.285 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 244409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-15T02:42:21.285 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/dlm/ 2023-11-15T02:42:21.285 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 440737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-15T02:42:21.286 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/dlmfs/ 2023-11-15T02:42:21.286 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-15T02:42:21.286 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 2592433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ocfs2.ko 2023-11-15T02:42:21.286 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-15T02:42:21.286 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-15T02:42:21.287 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-15T02:42:21.287 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/omfs/ 2023-11-15T02:42:21.287 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/omfs/omfs.ko 2023-11-15T02:42:21.287 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/overlayfs/ 2023-11-15T02:42:21.287 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 352153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/overlayfs/overlay.ko 2023-11-15T02:42:21.288 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/pstore/ 2023-11-15T02:42:21.288 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/pstore/ramoops.ko 2023-11-15T02:42:21.288 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/qnx4/ 2023-11-15T02:42:21.288 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/qnx4/qnx4.ko 2023-11-15T02:42:21.288 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/qnx6/ 2023-11-15T02:42:21.289 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 46689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/qnx6/qnx6.ko 2023-11-15T02:42:21.289 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/quota/ 2023-11-15T02:42:21.289 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/quota/quota_tree.ko 2023-11-15T02:42:21.289 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/quota/quota_v1.ko 2023-11-15T02:42:21.289 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/quota/quota_v2.ko 2023-11-15T02:42:21.290 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/reiserfs/ 2023-11-15T02:42:21.290 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 532673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/reiserfs/reiserfs.ko 2023-11-15T02:42:21.290 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/romfs/ 2023-11-15T02:42:21.290 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/romfs/romfs.ko 2023-11-15T02:42:21.290 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/ 2023-11-15T02:42:21.291 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/client/ 2023-11-15T02:42:21.291 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 2668121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/client/cifs.ko 2023-11-15T02:42:21.291 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/common/ 2023-11-15T02:42:21.291 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/common/cifs_arc4.ko 2023-11-15T02:42:21.291 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/common/cifs_md4.ko 2023-11-15T02:42:21.292 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/squashfs/ 2023-11-15T02:42:21.292 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 134641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/squashfs/squashfs.ko 2023-11-15T02:42:21.292 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/sysv/ 2023-11-15T02:42:21.292 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 90721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/sysv/sysv.ko 2023-11-15T02:42:21.292 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/udf/ 2023-11-15T02:42:21.293 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 247737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/udf/udf.ko 2023-11-15T02:42:21.293 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ufs/ 2023-11-15T02:42:21.293 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 162497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ufs/ufs.ko 2023-11-15T02:42:21.293 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/xfs/ 2023-11-15T02:42:21.293 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 4010729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/xfs/xfs.ko 2023-11-15T02:42:21.294 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/ 2023-11-15T02:42:21.294 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/locking/ 2023-11-15T02:42:21.294 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 75489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/locking/locktorture.ko 2023-11-15T02:42:21.294 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/time/ 2023-11-15T02:42:21.294 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/time/test_udelay.ko 2023-11-15T02:42:21.294 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 60553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/torture.ko 2023-11-15T02:42:21.295 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/ 2023-11-15T02:42:21.295 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 6833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crc-itu-t.ko 2023-11-15T02:42:21.295 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 6529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crc7.ko 2023-11-15T02:42:21.295 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crc8.ko 2023-11-15T02:42:21.295 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crypto/ 2023-11-15T02:42:21.296 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crypto/libarc4.ko 2023-11-15T02:42:21.296 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crypto/libdes.ko 2023-11-15T02:42:21.296 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/interval_tree_test.ko 2023-11-15T02:42:21.296 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/libcrc32c.ko 2023-11-15T02:42:21.296 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/lru_cache.ko 2023-11-15T02:42:21.297 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/lz4/ 2023-11-15T02:42:21.297 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/lz4/lz4_compress.ko 2023-11-15T02:42:21.297 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/lz4/lz4hc_compress.ko 2023-11-15T02:42:21.297 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/math/ 2023-11-15T02:42:21.297 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 6617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/math/cordic.ko 2023-11-15T02:42:21.298 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/memory-notifier-error-inject.ko 2023-11-15T02:42:21.298 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/notifier-error-inject.ko 2023-11-15T02:42:21.298 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/percpu_test.ko 2023-11-15T02:42:21.298 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/pm-notifier-error-inject.ko 2023-11-15T02:42:21.298 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/raid6/ 2023-11-15T02:42:21.299 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 143433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/raid6/raid6_pq.ko 2023-11-15T02:42:21.299 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/rbtree_test.ko 2023-11-15T02:42:21.299 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/reed_solomon/ 2023-11-15T02:42:21.299 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-15T02:42:21.299 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test-kstrtox.ko 2023-11-15T02:42:21.300 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test-string_helpers.ko 2023-11-15T02:42:21.300 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 4760425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test_bpf.ko 2023-11-15T02:42:21.300 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test_firmware.ko 2023-11-15T02:42:21.300 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 6369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test_module.ko 2023-11-15T02:42:21.300 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test_user_copy.ko 2023-11-15T02:42:21.301 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/ts_bm.ko 2023-11-15T02:42:21.301 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/ts_fsm.ko 2023-11-15T02:42:21.301 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/ts_kmp.ko 2023-11-15T02:42:21.301 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/xz/ 2023-11-15T02:42:21.301 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/xz/xz_dec_test.ko 2023-11-15T02:42:21.301 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/mm/ 2023-11-15T02:42:21.302 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/mm/hwpoison-inject.ko 2023-11-15T02:42:21.302 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ 2023-11-15T02:42:21.302 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/ 2023-11-15T02:42:21.302 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 55737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/6lowpan.ko 2023-11-15T02:42:21.302 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_dest.ko 2023-11-15T02:42:21.303 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_fragment.ko 2023-11-15T02:42:21.303 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_hop.ko 2023-11-15T02:42:21.303 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 6985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-15T02:42:21.303 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_mobility.ko 2023-11-15T02:42:21.303 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_routing.ko 2023-11-15T02:42:21.304 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_udp.ko 2023-11-15T02:42:21.304 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/ 2023-11-15T02:42:21.304 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/garp.ko 2023-11-15T02:42:21.304 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/mrp.ko 2023-11-15T02:42:21.305 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/p8022.ko 2023-11-15T02:42:21.305 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/psnap.ko 2023-11-15T02:42:21.305 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/stp.ko 2023-11-15T02:42:21.305 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/8021q/ 2023-11-15T02:42:21.305 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 81737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/8021q/8021q.ko 2023-11-15T02:42:21.306 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/ 2023-11-15T02:42:21.306 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 198337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/9pnet.ko 2023-11-15T02:42:21.306 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/9pnet_fd.ko 2023-11-15T02:42:21.306 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/9pnet_rdma.ko 2023-11-15T02:42:21.306 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/9pnet_virtio.ko 2023-11-15T02:42:21.307 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/batman-adv/ 2023-11-15T02:42:21.307 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 649297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/batman-adv/batman-adv.ko 2023-11-15T02:42:21.307 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/ 2023-11-15T02:42:21.307 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 63809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/br_netfilter.ko 2023-11-15T02:42:21.307 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 777689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/bridge.ko 2023-11-15T02:42:21.308 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ 2023-11-15T02:42:21.308 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-15T02:42:21.308 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-15T02:42:21.308 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-15T02:42:21.308 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-15T02:42:21.309 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-15T02:42:21.309 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-15T02:42:21.309 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-15T02:42:21.309 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-15T02:42:21.309 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-15T02:42:21.309 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-15T02:42:21.310 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-15T02:42:21.310 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-15T02:42:21.310 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-15T02:42:21.310 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-15T02:42:21.310 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-15T02:42:21.311 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-15T02:42:21.311 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-15T02:42:21.311 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-15T02:42:21.311 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-15T02:42:21.311 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-15T02:42:21.312 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 55929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebtables.ko 2023-11-15T02:42:21.312 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-15T02:42:21.312 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-15T02:42:21.312 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ceph/ 2023-11-15T02:42:21.312 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1046857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ceph/libceph.ko 2023-11-15T02:42:21.313 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/core/ 2023-11-15T02:42:21.313 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 125697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/core/pktgen.ko 2023-11-15T02:42:21.313 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/ 2023-11-15T02:42:21.313 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 194209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/dccp.ko 2023-11-15T02:42:21.313 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/dccp_diag.ko 2023-11-15T02:42:21.313 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 57881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/dccp_ipv4.ko 2023-11-15T02:42:21.314 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 60337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/dccp_ipv6.ko 2023-11-15T02:42:21.314 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/ 2023-11-15T02:42:21.314 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 390473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/dsa_core.ko 2023-11-15T02:42:21.314 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/tag_brcm.ko 2023-11-15T02:42:21.314 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/tag_dsa.ko 2023-11-15T02:42:21.315 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/tag_none.ko 2023-11-15T02:42:21.315 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/tag_trailer.ko 2023-11-15T02:42:21.315 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/hsr/ 2023-11-15T02:42:21.315 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 99729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/hsr/hsr.ko 2023-11-15T02:42:21.315 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/ 2023-11-15T02:42:21.316 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/6lowpan/ 2023-11-15T02:42:21.316 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 45233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-15T02:42:21.316 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 254625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/ieee802154.ko 2023-11-15T02:42:21.316 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-15T02:42:21.316 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ 2023-11-15T02:42:21.317 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ah4.ko 2023-11-15T02:42:21.317 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/esp4.ko 2023-11-15T02:42:21.317 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 48401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/fou.ko 2023-11-15T02:42:21.317 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/gre.ko 2023-11-15T02:42:21.317 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 48657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/inet_diag.ko 2023-11-15T02:42:21.318 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 60633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ip_gre.ko 2023-11-15T02:42:21.318 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 61225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ip_tunnel.ko 2023-11-15T02:42:21.318 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ip_vti.ko 2023-11-15T02:42:21.318 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ipcomp.ko 2023-11-15T02:42:21.318 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ipip.ko 2023-11-15T02:42:21.319 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ 2023-11-15T02:42:21.319 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-15T02:42:21.319 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-15T02:42:21.319 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-15T02:42:21.319 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ip_tables.ko 2023-11-15T02:42:21.320 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-15T02:42:21.320 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-11-15T02:42:21.320 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-15T02:42:21.320 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-15T02:42:21.321 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-15T02:42:21.321 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-11-15T02:42:21.321 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-15T02:42:21.321 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-15T02:42:21.321 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-15T02:42:21.322 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-15T02:42:21.322 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-15T02:42:21.322 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-15T02:42:21.322 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-15T02:42:21.322 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-15T02:42:21.323 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-15T02:42:21.323 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-11-15T02:42:21.323 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-15T02:42:21.323 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-15T02:42:21.323 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-15T02:42:21.323 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-15T02:42:21.324 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-15T02:42:21.324 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_bic.ko 2023-11-15T02:42:21.324 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_dctcp.ko 2023-11-15T02:42:21.324 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_diag.ko 2023-11-15T02:42:21.324 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_highspeed.ko 2023-11-15T02:42:21.325 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_htcp.ko 2023-11-15T02:42:21.325 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_hybla.ko 2023-11-15T02:42:21.325 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_illinois.ko 2023-11-15T02:42:21.325 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_lp.ko 2023-11-15T02:42:21.325 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_scalable.ko 2023-11-15T02:42:21.326 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_vegas.ko 2023-11-15T02:42:21.326 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_veno.ko 2023-11-15T02:42:21.326 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_westwood.ko 2023-11-15T02:42:21.326 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_yeah.ko 2023-11-15T02:42:21.326 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tunnel4.ko 2023-11-15T02:42:21.327 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/udp_diag.ko 2023-11-15T02:42:21.327 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 41665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/udp_tunnel.ko 2023-11-15T02:42:21.327 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-15T02:42:21.327 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ 2023-11-15T02:42:21.327 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ah6.ko 2023-11-15T02:42:21.328 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/esp6.ko 2023-11-15T02:42:21.328 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/fou6.ko 2023-11-15T02:42:21.328 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 80897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ip6_gre.ko 2023-11-15T02:42:21.328 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 93577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ip6_tunnel.ko 2023-11-15T02:42:21.328 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-15T02:42:21.329 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ip6_vti.ko 2023-11-15T02:42:21.329 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ipcomp6.ko 2023-11-15T02:42:21.329 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/mip6.ko 2023-11-15T02:42:21.329 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ 2023-11-15T02:42:21.329 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-15T02:42:21.330 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-11-15T02:42:21.330 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-15T02:42:21.330 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-15T02:42:21.330 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-15T02:42:21.331 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-15T02:42:21.331 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-15T02:42:21.331 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-15T02:42:21.331 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-15T02:42:21.331 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-15T02:42:21.332 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-15T02:42:21.332 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-15T02:42:21.332 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-15T02:42:21.332 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-15T02:42:21.332 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-15T02:42:21.332 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-15T02:42:21.333 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-15T02:42:21.333 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-15T02:42:21.333 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-15T02:42:21.333 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-15T02:42:21.333 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-15T02:42:21.334 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-15T02:42:21.334 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-15T02:42:21.334 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-15T02:42:21.334 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-15T02:42:21.334 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 74073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/sit.ko 2023-11-15T02:42:21.335 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/tunnel6.ko 2023-11-15T02:42:21.335 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-15T02:42:21.335 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/key/ 2023-11-15T02:42:21.335 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 82833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/key/af_key.ko 2023-11-15T02:42:21.335 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/ 2023-11-15T02:42:21.335 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 125281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_core.ko 2023-11-15T02:42:21.336 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-15T02:42:21.336 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 25953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_eth.ko 2023-11-15T02:42:21.336 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_ip.ko 2023-11-15T02:42:21.336 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_ip6.ko 2023-11-15T02:42:21.336 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_netlink.ko 2023-11-15T02:42:21.337 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 53241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_ppp.ko 2023-11-15T02:42:21.337 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/lapb/ 2023-11-15T02:42:21.337 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 47977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/lapb/lapb.ko 2023-11-15T02:42:21.337 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/llc/ 2023-11-15T02:42:21.337 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/llc/llc.ko 2023-11-15T02:42:21.338 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 267777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/llc/llc2.ko 2023-11-15T02:42:21.338 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/mac802154/ 2023-11-15T02:42:21.338 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 290417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/mac802154/mac802154.ko 2023-11-15T02:42:21.338 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/mpls/ 2023-11-15T02:42:21.338 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/mpls/mpls_gso.ko 2023-11-15T02:42:21.339 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ 2023-11-15T02:42:21.339 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ 2023-11-15T02:42:21.339 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 119585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set.ko 2023-11-15T02:42:21.339 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-15T02:42:21.339 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-15T02:42:21.339 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-15T02:42:21.340 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 100297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-15T02:42:21.340 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 101441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-15T02:42:21.340 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 102817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-15T02:42:21.340 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 103481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-15T02:42:21.340 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 112849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-15T02:42:21.341 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 60329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-15T02:42:21.341 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 109593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-15T02:42:21.341 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 113129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-15T02:42:21.341 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 112585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-15T02:42:21.341 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 111585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-15T02:42:21.342 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 114465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-15T02:42:21.342 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-15T02:42:21.342 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ 2023-11-15T02:42:21.342 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 411729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-15T02:42:21.342 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-15T02:42:21.343 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-15T02:42:21.343 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-15T02:42:21.343 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-15T02:42:21.343 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-15T02:42:21.343 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-15T02:42:21.344 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-15T02:42:21.344 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-15T02:42:21.344 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-15T02:42:21.344 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-15T02:42:21.344 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-15T02:42:21.345 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-15T02:42:21.345 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-15T02:42:21.345 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conncount.ko 2023-11-15T02:42:21.345 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 408257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack.ko 2023-11-15T02:42:21.345 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-15T02:42:21.346 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-15T02:42:21.346 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-15T02:42:21.346 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 118145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-15T02:42:21.346 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-15T02:42:21.346 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-15T02:42:21.347 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 107297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-15T02:42:21.347 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-15T02:42:21.347 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-15T02:42:21.347 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 66641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-15T02:42:21.347 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-15T02:42:21.348 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-15T02:42:21.348 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-15T02:42:21.348 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 89473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_flow_table.ko 2023-11-15T02:42:21.348 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-15T02:42:21.348 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 37129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_log_syslog.ko 2023-11-15T02:42:21.348 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 95417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat.ko 2023-11-15T02:42:21.349 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-15T02:42:21.349 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-15T02:42:21.349 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_irc.ko 2023-11-15T02:42:21.349 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_sip.ko 2023-11-15T02:42:21.349 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-15T02:42:21.350 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 44065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-15T02:42:21.350 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 615313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_tables.ko 2023-11-15T02:42:21.350 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 42561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink.ko 2023-11-15T02:42:21.350 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-15T02:42:21.350 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-11-15T02:42:21.351 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 39785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_log.ko 2023-11-15T02:42:21.351 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-15T02:42:21.351 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-15T02:42:21.351 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_chain_nat.ko 2023-11-15T02:42:21.351 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_compat.ko 2023-11-15T02:42:21.352 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 40721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_ct.ko 2023-11-15T02:42:21.352 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-15T02:42:21.352 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_fib.ko 2023-11-15T02:42:21.352 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-15T02:42:21.352 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_hash.ko 2023-11-15T02:42:21.353 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_limit.ko 2023-11-15T02:42:21.353 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_log.ko 2023-11-15T02:42:21.353 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_masq.ko 2023-11-15T02:42:21.353 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_nat.ko 2023-11-15T02:42:21.353 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_numgen.ko 2023-11-15T02:42:21.353 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_osf.ko 2023-11-15T02:42:21.354 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_queue.ko 2023-11-15T02:42:21.354 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_redir.ko 2023-11-15T02:42:21.354 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_reject.ko 2023-11-15T02:42:21.354 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_reject_inet.ko 2023-11-15T02:42:21.354 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_socket.ko 2023-11-15T02:42:21.355 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_synproxy.ko 2023-11-15T02:42:21.355 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_tproxy.ko 2023-11-15T02:42:21.355 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 89313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/x_tables.ko 2023-11-15T02:42:21.355 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_AUDIT.ko 2023-11-15T02:42:21.355 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-15T02:42:21.356 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-15T02:42:21.356 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-15T02:42:21.356 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 18169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_CT.ko 2023-11-15T02:42:21.356 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_DSCP.ko 2023-11-15T02:42:21.356 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_HL.ko 2023-11-15T02:42:21.357 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_HMARK.ko 2023-11-15T02:42:21.357 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-15T02:42:21.357 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_LED.ko 2023-11-15T02:42:21.357 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_LOG.ko 2023-11-15T02:42:21.357 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-15T02:42:21.358 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_NETMAP.ko 2023-11-15T02:42:21.358 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_NFLOG.ko 2023-11-15T02:42:21.358 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-15T02:42:21.358 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_RATEEST.ko 2023-11-15T02:42:21.358 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-15T02:42:21.359 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 14337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_SECMARK.ko 2023-11-15T02:42:21.359 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-15T02:42:21.359 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-15T02:42:21.359 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TEE.ko 2023-11-15T02:42:21.359 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TPROXY.ko 2023-11-15T02:42:21.359 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TRACE.ko 2023-11-15T02:42:21.360 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_addrtype.ko 2023-11-15T02:42:21.360 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_bpf.ko 2023-11-15T02:42:21.360 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_cgroup.ko 2023-11-15T02:42:21.360 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_cluster.ko 2023-11-15T02:42:21.360 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_comment.ko 2023-11-15T02:42:21.361 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_connbytes.ko 2023-11-15T02:42:21.361 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_connlabel.ko 2023-11-15T02:42:21.361 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_connlimit.ko 2023-11-15T02:42:21.361 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_connmark.ko 2023-11-15T02:42:21.361 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_conntrack.ko 2023-11-15T02:42:21.362 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_cpu.ko 2023-11-15T02:42:21.362 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_dccp.ko 2023-11-15T02:42:21.362 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_devgroup.ko 2023-11-15T02:42:21.362 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_dscp.ko 2023-11-15T02:42:21.362 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_ecn.ko 2023-11-15T02:42:21.363 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_esp.ko 2023-11-15T02:42:21.363 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 43289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_hashlimit.ko 2023-11-15T02:42:21.363 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_helper.ko 2023-11-15T02:42:21.363 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_hl.ko 2023-11-15T02:42:21.363 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_ipcomp.ko 2023-11-15T02:42:21.363 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_iprange.ko 2023-11-15T02:42:21.364 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_ipvs.ko 2023-11-15T02:42:21.364 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_l2tp.ko 2023-11-15T02:42:21.364 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_length.ko 2023-11-15T02:42:21.364 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_limit.ko 2023-11-15T02:42:21.364 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_mac.ko 2023-11-15T02:42:21.365 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_mark.ko 2023-11-15T02:42:21.365 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_multiport.ko 2023-11-15T02:42:21.365 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_nat.ko 2023-11-15T02:42:21.365 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_nfacct.ko 2023-11-15T02:42:21.365 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_osf.ko 2023-11-15T02:42:21.366 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_owner.ko 2023-11-15T02:42:21.366 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_physdev.ko 2023-11-15T02:42:21.366 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_pkttype.ko 2023-11-15T02:42:21.366 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_policy.ko 2023-11-15T02:42:21.366 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_quota.ko 2023-11-15T02:42:21.367 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_rateest.ko 2023-11-15T02:42:21.367 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_realm.ko 2023-11-15T02:42:21.367 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_recent.ko 2023-11-15T02:42:21.367 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_sctp.ko 2023-11-15T02:42:21.368 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_set.ko 2023-11-15T02:42:21.368 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 16089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_socket.ko 2023-11-15T02:42:21.368 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_state.ko 2023-11-15T02:42:21.368 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_statistic.ko 2023-11-15T02:42:21.368 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_string.ko 2023-11-15T02:42:21.369 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 8553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_tcpmss.ko 2023-11-15T02:42:21.369 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_tcpudp.ko 2023-11-15T02:42:21.369 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_time.ko 2023-11-15T02:42:21.369 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_u32.ko 2023-11-15T02:42:21.370 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netlink/ 2023-11-15T02:42:21.370 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netlink/netlink_diag.ko 2023-11-15T02:42:21.370 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/nsh/ 2023-11-15T02:42:21.370 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 12425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/nsh/nsh.ko 2023-11-15T02:42:21.370 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/ 2023-11-15T02:42:21.371 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 380033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/openvswitch.ko 2023-11-15T02:42:21.371 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/vport-geneve.ko 2023-11-15T02:42:21.371 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/vport-gre.ko 2023-11-15T02:42:21.371 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 11793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/vport-vxlan.ko 2023-11-15T02:42:21.371 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/packet/ 2023-11-15T02:42:21.372 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/packet/af_packet_diag.ko 2023-11-15T02:42:21.372 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/phonet/ 2023-11-15T02:42:21.372 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 110201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/phonet/phonet.ko 2023-11-15T02:42:21.372 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/phonet/pn_pep.ko 2023-11-15T02:42:21.372 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rds/ 2023-11-15T02:42:21.373 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 251481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rds/rds.ko 2023-11-15T02:42:21.373 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 164937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rds/rds_rdma.ko 2023-11-15T02:42:21.373 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 67377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rds/rds_tcp.ko 2023-11-15T02:42:21.373 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rxrpc/ 2023-11-15T02:42:21.373 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 819657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rxrpc/rxrpc.ko 2023-11-15T02:42:21.373 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/ 2023-11-15T02:42:21.374 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_csum.ko 2023-11-15T02:42:21.374 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_gact.ko 2023-11-15T02:42:21.374 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_ipt.ko 2023-11-15T02:42:21.374 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_mirred.ko 2023-11-15T02:42:21.374 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_nat.ko 2023-11-15T02:42:21.375 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 43449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_pedit.ko 2023-11-15T02:42:21.375 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_police.ko 2023-11-15T02:42:21.375 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_simple.ko 2023-11-15T02:42:21.375 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_skbedit.ko 2023-11-15T02:42:21.375 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_vlan.ko 2023-11-15T02:42:21.376 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_basic.ko 2023-11-15T02:42:21.376 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_bpf.ko 2023-11-15T02:42:21.376 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_cgroup.ko 2023-11-15T02:42:21.376 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_flow.ko 2023-11-15T02:42:21.376 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_fw.ko 2023-11-15T02:42:21.376 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 38489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_route.ko 2023-11-15T02:42:21.377 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 61409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_u32.ko 2023-11-15T02:42:21.377 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_cmp.ko 2023-11-15T02:42:21.377 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 13409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_ipset.ko 2023-11-15T02:42:21.377 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_meta.ko 2023-11-15T02:42:21.377 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_nbyte.ko 2023-11-15T02:42:21.378 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_text.ko 2023-11-15T02:42:21.378 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_u32.ko 2023-11-15T02:42:21.378 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 21649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_choke.ko 2023-11-15T02:42:21.378 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_codel.ko 2023-11-15T02:42:21.378 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 34633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_drr.ko 2023-11-15T02:42:21.379 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 33609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_fq.ko 2023-11-15T02:42:21.379 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_fq_codel.ko 2023-11-15T02:42:21.379 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 35433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_gred.ko 2023-11-15T02:42:21.379 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 49601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_hfsc.ko 2023-11-15T02:42:21.379 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_hhf.ko 2023-11-15T02:42:21.379 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 66393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_htb.ko 2023-11-15T02:42:21.380 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_ingress.ko 2023-11-15T02:42:21.380 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 30273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_mqprio.ko 2023-11-15T02:42:21.380 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_mqprio_lib.ko 2023-11-15T02:42:21.380 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_multiq.ko 2023-11-15T02:42:21.380 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 36137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_netem.ko 2023-11-15T02:42:21.381 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 27497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_pie.ko 2023-11-15T02:42:21.381 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_plug.ko 2023-11-15T02:42:21.381 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 24729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_prio.ko 2023-11-15T02:42:21.381 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 48793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_qfq.ko 2023-11-15T02:42:21.381 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 32905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_red.ko 2023-11-15T02:42:21.382 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 26881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_sfb.ko 2023-11-15T02:42:21.382 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 31353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_sfq.ko 2023-11-15T02:42:21.382 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_tbf.ko 2023-11-15T02:42:21.382 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 28105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_teql.ko 2023-11-15T02:42:21.382 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sctp/ 2023-11-15T02:42:21.383 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 929745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sctp/sctp.ko 2023-11-15T02:42:21.383 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 19033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sctp/sctp_diag.ko 2023-11-15T02:42:21.383 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/ 2023-11-15T02:42:21.383 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/auth_gss/ 2023-11-15T02:42:21.383 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 336529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-15T02:42:21.384 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 54737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-15T02:42:21.384 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 1520161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/sunrpc.ko 2023-11-15T02:42:21.384 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/xprtrdma/ 2023-11-15T02:42:21.384 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 707721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-15T02:42:21.384 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/tipc/ 2023-11-15T02:42:21.385 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/tipc/diag.ko 2023-11-15T02:42:21.385 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 862641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/tipc/tipc.ko 2023-11-15T02:42:21.385 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/unix/ 2023-11-15T02:42:21.385 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/unix/unix_diag.ko 2023-11-15T02:42:21.385 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/ 2023-11-15T02:42:21.385 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 85337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-15T02:42:21.386 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 68049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-15T02:42:21.386 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 96193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vsock.ko 2023-11-15T02:42:21.386 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 10705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-15T02:42:21.386 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 17521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-15T02:42:21.386 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/x25/ 2023-11-15T02:42:21.387 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 125177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/x25/x25.ko 2023-11-15T02:42:21.387 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/xfrm/ 2023-11-15T02:42:21.387 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 22657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/xfrm/xfrm_algo.ko 2023-11-15T02:42:21.387 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 20649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-15T02:42:21.387 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 102257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/xfrm/xfrm_user.ko 2023-11-15T02:42:21.388 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/virt/ 2023-11-15T02:42:21.388 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/virt/lib/ 2023-11-15T02:42:21.388 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 15537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/virt/lib/irqbypass.ko 2023-11-15T02:42:21.388 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 639863 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.alias 2023-11-15T02:42:21.389 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 623919 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.alias.bin 2023-11-15T02:42:21.389 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 7663 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.builtin 2023-11-15T02:42:21.389 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 23428 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.builtin.alias.bin 2023-11-15T02:42:21.389 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 9821 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.builtin.bin 2023-11-15T02:42:21.389 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 70004 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.builtin.modinfo 2023-11-15T02:42:21.389 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 146313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.dep 2023-11-15T02:42:21.390 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 232548 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.dep.bin 2023-11-15T02:42:21.390 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 210 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.devname 2023-11-15T02:42:21.390 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 72606 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.order 2023-11-15T02:42:21.390 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 879 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.softdep 2023-11-15T02:42:21.391 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 254438 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.symbols 2023-11-15T02:42:21.391 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 309363 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.symbols.bin 2023-11-15T02:42:21.391 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./usr/ 2023-11-15T02:42:21.391 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./usr/share/ 2023-11-15T02:42:21.391 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./usr/share/doc/ 2023-11-15T02:42:21.392 INFO:teuthology.orchestra.run.smithi121.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./usr/share/doc/linux-image-6.6.0-rc5-g3b796cfe2969/ 2023-11-15T02:42:21.392 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 173 2023-11-07 03:19 ./usr/share/doc/linux-image-6.6.0-rc5-g3b796cfe2969/changelog.Debian.gz 2023-11-15T02:42:21.392 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r-- root/root 691 2023-11-07 03:19 ./usr/share/doc/linux-image-6.6.0-rc5-g3b796cfe2969/copyright 2023-11-15T02:42:21.393 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:21.393 INFO:teuthology.task.kernel:searching for kernel 6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:21.393 DEBUG:teuthology.orchestra.run.smithi121:> egrep '(submenu|menuentry.*6.6.0-rc5-g3b796cfe2969).*{' /boot/grub/grub.cfg 2023-11-15T02:42:21.402 INFO:teuthology.orchestra.run.smithi121.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-11-15T02:42:21.403 INFO:teuthology.orchestra.run.smithi121.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g3b796cfe2969' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g3b796cfe2969-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-11-15T02:42:21.403 INFO:teuthology.orchestra.run.smithi121.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g3b796cfe2969 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g3b796cfe2969-recovery-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-11-15T02:42:21.403 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-11-15T02:42:21.404 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:21.404 DEBUG:teuthology.orchestra.run.smithi121:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-11-15T02:42:21.406 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi121.front.sepia.ceph.com to complete... 2023-11-15T02:42:21.946 INFO:teuthology.orchestra.run.smithi121.stderr:Sourcing file `/etc/default/grub' 2023-11-15T02:42:21.948 INFO:teuthology.orchestra.run.smithi121.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-15T02:42:21.961 INFO:teuthology.orchestra.run.smithi121.stderr:Generating grub configuration file ... 2023-11-15T02:42:22.223 INFO:teuthology.orchestra.run.smithi121.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:22.235 INFO:teuthology.orchestra.run.smithi121.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:22.591 INFO:teuthology.orchestra.run.smithi121.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-11-15T02:42:22.602 INFO:teuthology.orchestra.run.smithi121.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-11-15T02:42:23.360 INFO:teuthology.orchestra.run.smithi121.stderr:done 2023-11-15T02:42:33.492 INFO:teuthology.orchestra.run.smithi190.stderr:Sourcing file `/etc/default/grub' 2023-11-15T02:42:33.503 INFO:teuthology.orchestra.run.smithi190.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-15T02:42:33.536 INFO:teuthology.orchestra.run.smithi190.stderr:Generating grub configuration file ... 2023-11-15T02:42:33.894 INFO:teuthology.orchestra.run.smithi190.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:33.920 INFO:teuthology.orchestra.run.smithi190.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:34.319 INFO:teuthology.orchestra.run.smithi190.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-11-15T02:42:34.331 INFO:teuthology.orchestra.run.smithi190.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-11-15T02:42:36.104 INFO:teuthology.orchestra.run.smithi190.stderr:done 2023-11-15T02:42:36.281 DEBUG:teuthology.orchestra.run.smithi190:> dpkg-deb -c /tmp/linux-image.deb 2023-11-15T02:42:36.291 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./ 2023-11-15T02:42:36.291 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./boot/ 2023-11-15T02:42:36.292 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 4415097 2023-11-07 03:19 ./boot/System.map-6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:36.309 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 169233 2023-11-07 03:19 ./boot/config-6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:36.309 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11352928 2023-11-07 03:19 ./boot/vmlinuz-6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:36.329 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/ 2023-11-15T02:42:36.330 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/ 2023-11-15T02:42:36.330 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/postinst.d/ 2023-11-15T02:42:36.330 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/postrm.d/ 2023-11-15T02:42:36.330 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/preinst.d/ 2023-11-15T02:42:36.330 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/prerm.d/ 2023-11-15T02:42:36.331 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/ 2023-11-15T02:42:36.331 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/ 2023-11-15T02:42:36.331 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/ 2023-11-15T02:42:36.331 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/ 2023-11-15T02:42:36.331 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/ 2023-11-15T02:42:36.332 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/ 2023-11-15T02:42:36.332 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/ 2023-11-15T02:42:36.332 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 490233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/aesni-intel.ko 2023-11-15T02:42:36.335 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-15T02:42:36.335 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-15T02:42:36.335 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-15T02:42:36.335 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 93113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-15T02:42:36.336 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 106729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-15T02:42:36.336 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 152241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-15T02:42:36.336 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-15T02:42:36.336 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-15T02:42:36.336 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 83889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-15T02:42:36.337 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-15T02:42:36.337 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 47225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-15T02:42:36.337 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 47209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-15T02:42:36.337 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 51801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-15T02:42:36.337 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 38937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-15T02:42:36.338 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 40529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-15T02:42:36.338 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 58585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-15T02:42:36.338 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 50497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-15T02:42:36.338 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 32065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-15T02:42:36.338 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-15T02:42:36.339 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/ 2023-11-15T02:42:36.339 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/cpu/ 2023-11-15T02:42:36.339 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/cpu/mce/ 2023-11-15T02:42:36.339 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 40113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-15T02:42:36.339 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/cpuid.ko 2023-11-15T02:42:36.340 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/msr.ko 2023-11-15T02:42:36.340 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kvm/ 2023-11-15T02:42:36.340 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 353761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-15T02:42:36.340 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 881713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-15T02:42:36.340 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 2276817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kvm/kvm.ko 2023-11-15T02:42:36.400 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/ 2023-11-15T02:42:36.401 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 53761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/af_alg.ko 2023-11-15T02:42:36.401 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/algif_hash.ko 2023-11-15T02:42:36.401 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/algif_skcipher.ko 2023-11-15T02:42:36.401 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/ansi_cprng.ko 2023-11-15T02:42:36.402 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/anubis.ko 2023-11-15T02:42:36.402 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/arc4.ko 2023-11-15T02:42:36.402 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/asymmetric_keys/ 2023-11-15T02:42:36.402 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-11-15T02:42:36.403 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/ 2023-11-15T02:42:36.403 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_memcpy.ko 2023-11-15T02:42:36.403 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_pq.ko 2023-11-15T02:42:36.403 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-15T02:42:36.403 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_tx.ko 2023-11-15T02:42:36.403 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_xor.ko 2023-11-15T02:42:36.404 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/raid6test.ko 2023-11-15T02:42:36.404 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/authenc.ko 2023-11-15T02:42:36.404 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/authencesn.ko 2023-11-15T02:42:36.404 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/blake2b_generic.ko 2023-11-15T02:42:36.405 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/blowfish_common.ko 2023-11-15T02:42:36.405 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/blowfish_generic.ko 2023-11-15T02:42:36.405 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 38873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/camellia_generic.ko 2023-11-15T02:42:36.405 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cast5_generic.ko 2023-11-15T02:42:36.405 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cast6_generic.ko 2023-11-15T02:42:36.406 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cast_common.ko 2023-11-15T02:42:36.406 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 35097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/ccm.ko 2023-11-15T02:42:36.406 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cmac.ko 2023-11-15T02:42:36.406 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/crc32_generic.ko 2023-11-15T02:42:36.406 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 47505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cryptd.ko 2023-11-15T02:42:36.407 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 39505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/crypto_engine.ko 2023-11-15T02:42:36.407 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/crypto_simd.ko 2023-11-15T02:42:36.407 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/crypto_user.ko 2023-11-15T02:42:36.407 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cts.ko 2023-11-15T02:42:36.407 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/des_generic.ko 2023-11-15T02:42:36.407 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/dh_generic.ko 2023-11-15T02:42:36.408 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/echainiv.ko 2023-11-15T02:42:36.408 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/essiv.ko 2023-11-15T02:42:36.408 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/fcrypt.ko 2023-11-15T02:42:36.408 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/khazad.ko 2023-11-15T02:42:36.408 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/lrw.ko 2023-11-15T02:42:36.409 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/lz4.ko 2023-11-15T02:42:36.409 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/lz4hc.ko 2023-11-15T02:42:36.409 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/md4.ko 2023-11-15T02:42:36.409 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/michael_mic.ko 2023-11-15T02:42:36.409 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/pcbc.ko 2023-11-15T02:42:36.409 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/pcrypt.ko 2023-11-15T02:42:36.410 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/rmd160.ko 2023-11-15T02:42:36.410 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/seed.ko 2023-11-15T02:42:36.410 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/serpent_generic.ko 2023-11-15T02:42:36.410 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 149193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/tcrypt.ko 2023-11-15T02:42:36.411 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/tea.ko 2023-11-15T02:42:36.411 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/twofish_common.ko 2023-11-15T02:42:36.411 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/twofish_generic.ko 2023-11-15T02:42:36.411 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/vmac.ko 2023-11-15T02:42:36.411 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/wp512.ko 2023-11-15T02:42:36.411 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/xcbc.ko 2023-11-15T02:42:36.412 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/xor.ko 2023-11-15T02:42:36.412 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/xts.ko 2023-11-15T02:42:36.412 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/xxhash_generic.ko 2023-11-15T02:42:36.412 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ 2023-11-15T02:42:36.412 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/ 2023-11-15T02:42:36.413 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/acpi_extlog.ko 2023-11-15T02:42:36.413 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-15T02:42:36.413 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/acpi_pad.ko 2023-11-15T02:42:36.413 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/apei/ 2023-11-15T02:42:36.413 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 31241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/apei/einj.ko 2023-11-15T02:42:36.414 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/ec_sys.ko 2023-11-15T02:42:36.414 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/platform_profile.ko 2023-11-15T02:42:36.414 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/sbs.ko 2023-11-15T02:42:36.414 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/sbshc.ko 2023-11-15T02:42:36.414 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 103025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/video.ko 2023-11-15T02:42:36.415 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/ 2023-11-15T02:42:36.415 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/acard-ahci.ko 2023-11-15T02:42:36.415 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 88865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/ahci.ko 2023-11-15T02:42:36.415 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/ahci_platform.ko 2023-11-15T02:42:36.415 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 88289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/libahci.ko 2023-11-15T02:42:36.416 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/libahci_platform.ko 2023-11-15T02:42:36.416 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_acpi.ko 2023-11-15T02:42:36.416 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_ali.ko 2023-11-15T02:42:36.416 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_amd.ko 2023-11-15T02:42:36.416 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_artop.ko 2023-11-15T02:42:36.417 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_atiixp.ko 2023-11-15T02:42:36.417 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_atp867x.ko 2023-11-15T02:42:36.417 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_cmd640.ko 2023-11-15T02:42:36.417 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_cmd64x.ko 2023-11-15T02:42:36.417 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_cypress.ko 2023-11-15T02:42:36.418 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_efar.ko 2023-11-15T02:42:36.418 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_hpt366.ko 2023-11-15T02:42:36.418 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_hpt37x.ko 2023-11-15T02:42:36.418 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-15T02:42:36.418 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-15T02:42:36.419 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_it8213.ko 2023-11-15T02:42:36.419 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 35273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_it821x.ko 2023-11-15T02:42:36.419 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_jmicron.ko 2023-11-15T02:42:36.419 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 51481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_legacy.ko 2023-11-15T02:42:36.419 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_marvell.ko 2023-11-15T02:42:36.420 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_mpiix.ko 2023-11-15T02:42:36.420 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_netcell.ko 2023-11-15T02:42:36.420 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_ninja32.ko 2023-11-15T02:42:36.420 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_ns87410.ko 2023-11-15T02:42:36.420 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_ns87415.ko 2023-11-15T02:42:36.421 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_oldpiix.ko 2023-11-15T02:42:36.421 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_opti.ko 2023-11-15T02:42:36.421 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_optidma.ko 2023-11-15T02:42:36.421 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 32585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_pcmcia.ko 2023-11-15T02:42:36.421 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 37793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-15T02:42:36.421 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-15T02:42:36.422 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_piccolo.ko 2023-11-15T02:42:36.422 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_radisys.ko 2023-11-15T02:42:36.422 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_rdc.ko 2023-11-15T02:42:36.422 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_rz1000.ko 2023-11-15T02:42:36.423 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_sch.ko 2023-11-15T02:42:36.423 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_serverworks.ko 2023-11-15T02:42:36.423 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_sil680.ko 2023-11-15T02:42:36.423 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_sl82c105.ko 2023-11-15T02:42:36.423 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_triflex.ko 2023-11-15T02:42:36.423 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_via.ko 2023-11-15T02:42:36.424 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pdc_adma.ko 2023-11-15T02:42:36.424 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_inic162x.ko 2023-11-15T02:42:36.424 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 83345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_mv.ko 2023-11-15T02:42:36.424 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 70393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_nv.ko 2023-11-15T02:42:36.424 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 37377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_promise.ko 2023-11-15T02:42:36.425 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_qstor.ko 2023-11-15T02:42:36.425 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_sil.ko 2023-11-15T02:42:36.425 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 38337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_sil24.ko 2023-11-15T02:42:36.425 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_sis.ko 2023-11-15T02:42:36.425 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_svw.ko 2023-11-15T02:42:36.426 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 44793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_sx4.ko 2023-11-15T02:42:36.426 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_uli.ko 2023-11-15T02:42:36.426 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 35065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_via.ko 2023-11-15T02:42:36.426 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_vsc.ko 2023-11-15T02:42:36.426 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/auxdisplay/ 2023-11-15T02:42:36.427 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/auxdisplay/ks0108.ko 2023-11-15T02:42:36.427 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/bcma/ 2023-11-15T02:42:36.427 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 124473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/bcma/bcma.ko 2023-11-15T02:42:36.427 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/ 2023-11-15T02:42:36.427 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/aoe/ 2023-11-15T02:42:36.428 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 113665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/aoe/aoe.ko 2023-11-15T02:42:36.428 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/drbd/ 2023-11-15T02:42:36.428 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 965585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/drbd/drbd.ko 2023-11-15T02:42:36.428 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 173225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/floppy.ko 2023-11-15T02:42:36.428 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/mtip32xx/ 2023-11-15T02:42:36.428 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 94297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-15T02:42:36.429 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 115793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/nbd.ko 2023-11-15T02:42:36.429 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/null_blk/ 2023-11-15T02:42:36.429 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 110449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/null_blk/null_blk.ko 2023-11-15T02:42:36.429 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 98193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/pktcdvd.ko 2023-11-15T02:42:36.429 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 263769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/rbd.ko 2023-11-15T02:42:36.430 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/xen-blkback/ 2023-11-15T02:42:36.430 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 98145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-15T02:42:36.430 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/zram/ 2023-11-15T02:42:36.430 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 60217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/zram/zram.ko 2023-11-15T02:42:36.430 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ 2023-11-15T02:42:36.431 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/applicom.ko 2023-11-15T02:42:36.431 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hangcheck-timer.ko 2023-11-15T02:42:36.431 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/ 2023-11-15T02:42:36.431 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-15T02:42:36.431 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-15T02:42:36.432 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-15T02:42:36.432 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/via-rng.ko 2023-11-15T02:42:36.432 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-15T02:42:36.432 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ 2023-11-15T02:42:36.432 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-15T02:42:36.433 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 148601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-15T02:42:36.433 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-15T02:42:36.433 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 139313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-15T02:42:36.433 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 70921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-15T02:42:36.433 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 53513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-15T02:42:36.433 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/lp.ko 2023-11-15T02:42:36.434 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/mwave/ 2023-11-15T02:42:36.434 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 84097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/mwave/mwave.ko 2023-11-15T02:42:36.434 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/nvram.ko 2023-11-15T02:42:36.434 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 32169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ppdev.ko 2023-11-15T02:42:36.434 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 51113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tlclk.ko 2023-11-15T02:42:36.435 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/ 2023-11-15T02:42:36.435 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-15T02:42:36.435 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-11-15T02:42:36.435 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-11-15T02:42:36.436 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 32985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-11-15T02:42:36.436 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-15T02:42:36.436 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-15T02:42:36.436 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-11-15T02:42:36.436 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/xillybus/ 2023-11-15T02:42:36.436 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/xillybus/xillybus_class.ko 2023-11-15T02:42:36.437 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 50945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/xillybus/xillybus_core.ko 2023-11-15T02:42:36.437 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-11-15T02:42:36.437 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/ 2023-11-15T02:42:36.437 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/clk-palmas.ko 2023-11-15T02:42:36.437 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 47849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/clk-si5351.ko 2023-11-15T02:42:36.438 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/clk-twl6040.ko 2023-11-15T02:42:36.438 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/clk-wm831x.ko 2023-11-15T02:42:36.438 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/cpufreq/ 2023-11-15T02:42:36.438 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-15T02:42:36.438 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-15T02:42:36.439 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-15T02:42:36.439 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/ 2023-11-15T02:42:36.439 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/ccp/ 2023-11-15T02:42:36.439 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 70801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-15T02:42:36.439 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 243025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/ccp/ccp.ko 2023-11-15T02:42:36.440 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/ 2023-11-15T02:42:36.440 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/ 2023-11-15T02:42:36.440 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/qat_common/ 2023-11-15T02:42:36.440 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 438761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-15T02:42:36.440 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-11-15T02:42:36.440 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-15T02:42:36.441 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/padlock-aes.ko 2023-11-15T02:42:36.441 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/padlock-sha.ko 2023-11-15T02:42:36.441 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/virtio/ 2023-11-15T02:42:36.441 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 68857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-15T02:42:36.442 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dca/ 2023-11-15T02:42:36.442 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dca/dca.ko 2023-11-15T02:42:36.442 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/ 2023-11-15T02:42:36.442 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/dw/ 2023-11-15T02:42:36.442 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-15T02:42:36.443 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/ioat/ 2023-11-15T02:42:36.443 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 122569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-15T02:42:36.443 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/ 2023-11-15T02:42:36.443 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 79953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/amd64_edac.ko 2023-11-15T02:42:36.443 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 35769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/e752x_edac.ko 2023-11-15T02:42:36.444 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 63617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/edac_mce_amd.ko 2023-11-15T02:42:36.444 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i3000_edac.ko 2023-11-15T02:42:36.444 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i3200_edac.ko 2023-11-15T02:42:36.444 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i5100_edac.ko 2023-11-15T02:42:36.444 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i5400_edac.ko 2023-11-15T02:42:36.444 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i7300_edac.ko 2023-11-15T02:42:36.445 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 52041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i7core_edac.ko 2023-11-15T02:42:36.445 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i82975x_edac.ko 2023-11-15T02:42:36.445 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/ie31200_edac.ko 2023-11-15T02:42:36.445 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 56825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/sb_edac.ko 2023-11-15T02:42:36.445 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/x38_edac.ko 2023-11-15T02:42:36.446 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/ 2023-11-15T02:42:36.446 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-gpio.ko 2023-11-15T02:42:36.446 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-max14577.ko 2023-11-15T02:42:36.446 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-max77693.ko 2023-11-15T02:42:36.446 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-max8997.ko 2023-11-15T02:42:36.447 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-palmas.ko 2023-11-15T02:42:36.447 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-rt8973a.ko 2023-11-15T02:42:36.447 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-sm5502.ko 2023-11-15T02:42:36.447 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/ 2023-11-15T02:42:36.447 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/dmi-sysfs.ko 2023-11-15T02:42:36.448 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/efi/ 2023-11-15T02:42:36.448 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/efi/efi-pstore.ko 2023-11-15T02:42:36.448 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-15T02:42:36.448 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/ 2023-11-15T02:42:36.448 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-adp5520.ko 2023-11-15T02:42:36.449 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-amd8111.ko 2023-11-15T02:42:36.449 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-arizona.ko 2023-11-15T02:42:36.449 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-da9052.ko 2023-11-15T02:42:36.449 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-da9055.ko 2023-11-15T02:42:36.449 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-dln2.ko 2023-11-15T02:42:36.450 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 77849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-f7188x.ko 2023-11-15T02:42:36.450 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-generic.ko 2023-11-15T02:42:36.450 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-ich.ko 2023-11-15T02:42:36.450 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-11-15T02:42:36.450 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-kempld.ko 2023-11-15T02:42:36.451 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-lp3943.ko 2023-11-15T02:42:36.451 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-max7300.ko 2023-11-15T02:42:36.451 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-max730x.ko 2023-11-15T02:42:36.451 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-max732x.ko 2023-11-15T02:42:36.451 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-11-15T02:42:36.452 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 47073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-pca953x.ko 2023-11-15T02:42:36.452 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-pcf857x.ko 2023-11-15T02:42:36.452 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-rdc321x.ko 2023-11-15T02:42:36.452 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-sch.ko 2023-11-15T02:42:36.452 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-sch311x.ko 2023-11-15T02:42:36.452 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-tps65912.ko 2023-11-15T02:42:36.453 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-twl4030.ko 2023-11-15T02:42:36.453 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-twl6040.ko 2023-11-15T02:42:36.453 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-15T02:42:36.453 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-vx855.ko 2023-11-15T02:42:36.453 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-wm831x.ko 2023-11-15T02:42:36.454 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-wm8350.ko 2023-11-15T02:42:36.454 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-wm8994.ko 2023-11-15T02:42:36.454 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/ 2023-11-15T02:42:36.454 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-a4tech.ko 2023-11-15T02:42:36.454 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 39169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-apple.ko 2023-11-15T02:42:36.455 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-appleir.ko 2023-11-15T02:42:36.455 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-aureal.ko 2023-11-15T02:42:36.455 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-axff.ko 2023-11-15T02:42:36.455 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-belkin.ko 2023-11-15T02:42:36.455 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-cherry.ko 2023-11-15T02:42:36.456 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-chicony.ko 2023-11-15T02:42:36.456 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 60649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-cp2112.ko 2023-11-15T02:42:36.456 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-cypress.ko 2023-11-15T02:42:36.456 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-dr.ko 2023-11-15T02:42:36.456 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-elecom.ko 2023-11-15T02:42:36.457 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-elo.ko 2023-11-15T02:42:36.457 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-emsff.ko 2023-11-15T02:42:36.457 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-ezkey.ko 2023-11-15T02:42:36.457 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-gaff.ko 2023-11-15T02:42:36.457 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-generic.ko 2023-11-15T02:42:36.458 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-gt683r.ko 2023-11-15T02:42:36.458 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-gyration.ko 2023-11-15T02:42:36.458 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-15T02:42:36.458 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-15T02:42:36.458 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-holtekff.ko 2023-11-15T02:42:36.459 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-hyperv.ko 2023-11-15T02:42:36.459 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-icade.ko 2023-11-15T02:42:36.459 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-kensington.ko 2023-11-15T02:42:36.459 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-keytouch.ko 2023-11-15T02:42:36.459 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-kye.ko 2023-11-15T02:42:36.459 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-lcpower.ko 2023-11-15T02:42:36.460 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-led.ko 2023-11-15T02:42:36.460 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 42361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-lenovo.ko 2023-11-15T02:42:36.460 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-lg-g15.ko 2023-11-15T02:42:36.460 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 58185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-15T02:42:36.460 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 101369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-15T02:42:36.461 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 80993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-logitech.ko 2023-11-15T02:42:36.461 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-magicmouse.ko 2023-11-15T02:42:36.461 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-microsoft.ko 2023-11-15T02:42:36.461 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-monterey.ko 2023-11-15T02:42:36.461 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 50161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-multitouch.ko 2023-11-15T02:42:36.462 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 32593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-ntrig.ko 2023-11-15T02:42:36.462 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-ortek.ko 2023-11-15T02:42:36.462 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-penmount.ko 2023-11-15T02:42:36.462 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-petalynx.ko 2023-11-15T02:42:36.462 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 72937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-picolcd.ko 2023-11-15T02:42:36.463 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-pl.ko 2023-11-15T02:42:36.463 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-plantronics.ko 2023-11-15T02:42:36.463 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-primax.ko 2023-11-15T02:42:36.463 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 35457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-rmi.ko 2023-11-15T02:42:36.463 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-15T02:42:36.464 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-common.ko 2023-11-15T02:42:36.464 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 31425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-15T02:42:36.464 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-15T02:42:36.464 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 31129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-15T02:42:36.464 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-15T02:42:36.465 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-15T02:42:36.465 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-15T02:42:36.465 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-15T02:42:36.465 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-15T02:42:36.465 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-15T02:42:36.465 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat.ko 2023-11-15T02:42:36.466 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-saitek.ko 2023-11-15T02:42:36.466 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-samsung.ko 2023-11-15T02:42:36.466 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-15T02:42:36.466 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-sjoy.ko 2023-11-15T02:42:36.466 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 51249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-sony.ko 2023-11-15T02:42:36.467 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-speedlink.ko 2023-11-15T02:42:36.467 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-steelseries.ko 2023-11-15T02:42:36.467 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-sunplus.ko 2023-11-15T02:42:36.467 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-15T02:42:36.467 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-tivo.ko 2023-11-15T02:42:36.468 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-tmff.ko 2023-11-15T02:42:36.468 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-topseed.ko 2023-11-15T02:42:36.468 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-twinhan.ko 2023-11-15T02:42:36.468 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 71073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-uclogic.ko 2023-11-15T02:42:36.468 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-waltop.ko 2023-11-15T02:42:36.469 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 143473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-wiimote.ko 2023-11-15T02:42:36.469 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-xinmo.ko 2023-11-15T02:42:36.469 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-zpff.ko 2023-11-15T02:42:36.469 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-zydacron.ko 2023-11-15T02:42:36.469 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 267825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid.ko 2023-11-15T02:42:36.469 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 32321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/uhid.ko 2023-11-15T02:42:36.470 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/usbhid/ 2023-11-15T02:42:36.470 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 125721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/usbhid/usbhid.ko 2023-11-15T02:42:36.470 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/usbhid/usbkbd.ko 2023-11-15T02:42:36.470 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/usbhid/usbmouse.ko 2023-11-15T02:42:36.470 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 217873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/wacom.ko 2023-11-15T02:42:36.471 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hv/ 2023-11-15T02:42:36.471 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 64201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hv/hv_balloon.ko 2023-11-15T02:42:36.471 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 89625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hv/hv_utils.ko 2023-11-15T02:42:36.471 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 305465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hv/hv_vmbus.ko 2023-11-15T02:42:36.471 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ 2023-11-15T02:42:36.472 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 66521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/abituguru.ko 2023-11-15T02:42:36.472 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 61393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/abituguru3.ko 2023-11-15T02:42:36.472 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-15T02:42:36.472 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ad7414.ko 2023-11-15T02:42:36.472 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ad7418.ko 2023-11-15T02:42:36.473 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adc128d818.ko 2023-11-15T02:42:36.473 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 31729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm1025.ko 2023-11-15T02:42:36.473 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 97977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm1026.ko 2023-11-15T02:42:36.473 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm1029.ko 2023-11-15T02:42:36.473 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 45553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm1031.ko 2023-11-15T02:42:36.474 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm9240.ko 2023-11-15T02:42:36.474 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ads7828.ko 2023-11-15T02:42:36.474 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7410.ko 2023-11-15T02:42:36.474 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7411.ko 2023-11-15T02:42:36.474 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 74433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7462.ko 2023-11-15T02:42:36.474 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 39585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7470.ko 2023-11-15T02:42:36.475 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 67929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7475.ko 2023-11-15T02:42:36.475 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7x10.ko 2023-11-15T02:42:36.475 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 39073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/amc6821.ko 2023-11-15T02:42:36.475 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 55265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/applesmc.ko 2023-11-15T02:42:36.475 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 49913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/asb100.ko 2023-11-15T02:42:36.476 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 58617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/asc7621.ko 2023-11-15T02:42:36.476 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 40889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-15T02:42:36.476 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/atxp1.ko 2023-11-15T02:42:36.476 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/coretemp.ko 2023-11-15T02:42:36.476 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/da9052-hwmon.ko 2023-11-15T02:42:36.477 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/da9055-hwmon.ko 2023-11-15T02:42:36.477 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 44465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-15T02:42:36.477 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 99601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/dme1737.ko 2023-11-15T02:42:36.477 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ds1621.ko 2023-11-15T02:42:36.477 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ds620.ko 2023-11-15T02:42:36.478 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/emc1403.ko 2023-11-15T02:42:36.478 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/emc2103.ko 2023-11-15T02:42:36.478 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/emc6w201.ko 2023-11-15T02:42:36.478 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 66985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/f71805f.ko 2023-11-15T02:42:36.478 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 94457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/f71882fg.ko 2023-11-15T02:42:36.479 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 41401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/f75375s.ko 2023-11-15T02:42:36.479 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/fam15h_power.ko 2023-11-15T02:42:36.479 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 55449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/fschmd.ko 2023-11-15T02:42:36.479 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/g760a.ko 2023-11-15T02:42:36.479 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/g762.ko 2023-11-15T02:42:36.479 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 46801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/gl518sm.ko 2023-11-15T02:42:36.480 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 45529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/gl520sm.ko 2023-11-15T02:42:36.480 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/hih6130.ko 2023-11-15T02:42:36.480 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-15T02:42:36.480 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/i5500_temp.ko 2023-11-15T02:42:36.480 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/i5k_amb.ko 2023-11-15T02:42:36.481 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ibmaem.ko 2023-11-15T02:42:36.481 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ibmpex.ko 2023-11-15T02:42:36.481 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ina209.ko 2023-11-15T02:42:36.481 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ina2xx.ko 2023-11-15T02:42:36.481 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 130449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/it87.ko 2023-11-15T02:42:36.482 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/jc42.ko 2023-11-15T02:42:36.482 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/k10temp.ko 2023-11-15T02:42:36.482 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/k8temp.ko 2023-11-15T02:42:36.482 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lineage-pem.ko 2023-11-15T02:42:36.482 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 48601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm63.ko 2023-11-15T02:42:36.483 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm73.ko 2023-11-15T02:42:36.483 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm75.ko 2023-11-15T02:42:36.483 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm77.ko 2023-11-15T02:42:36.483 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm78.ko 2023-11-15T02:42:36.483 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 31745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm80.ko 2023-11-15T02:42:36.484 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm83.ko 2023-11-15T02:42:36.484 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 72833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm85.ko 2023-11-15T02:42:36.484 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 46153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm87.ko 2023-11-15T02:42:36.484 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 37913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm90.ko 2023-11-15T02:42:36.484 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm92.ko 2023-11-15T02:42:36.484 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 108841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm93.ko 2023-11-15T02:42:36.485 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm95234.ko 2023-11-15T02:42:36.485 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm95241.ko 2023-11-15T02:42:36.485 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm95245.ko 2023-11-15T02:42:36.485 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc2945.ko 2023-11-15T02:42:36.485 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4151.ko 2023-11-15T02:42:36.486 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4215.ko 2023-11-15T02:42:36.486 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4222.ko 2023-11-15T02:42:36.486 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4245.ko 2023-11-15T02:42:36.486 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4260.ko 2023-11-15T02:42:36.486 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4261.ko 2023-11-15T02:42:36.487 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max16065.ko 2023-11-15T02:42:36.487 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max1619.ko 2023-11-15T02:42:36.487 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max1668.ko 2023-11-15T02:42:36.487 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max197.ko 2023-11-15T02:42:36.487 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max6639.ko 2023-11-15T02:42:36.488 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max6650.ko 2023-11-15T02:42:36.488 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 35777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max6697.ko 2023-11-15T02:42:36.488 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/mc13783-adc.ko 2023-11-15T02:42:36.488 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/mcp3021.ko 2023-11-15T02:42:36.488 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-11-15T02:42:36.488 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 44129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/nct6683.ko 2023-11-15T02:42:36.489 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 119817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/nct6775-core.ko 2023-11-15T02:42:36.489 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 60705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/nct6775.ko 2023-11-15T02:42:36.489 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 59737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/nct7802.ko 2023-11-15T02:42:36.489 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 82337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pc87360.ko 2023-11-15T02:42:36.489 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 54433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pc87427.ko 2023-11-15T02:42:36.490 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pcf8591.ko 2023-11-15T02:42:36.490 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/ 2023-11-15T02:42:36.490 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-15T02:42:36.490 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-15T02:42:36.490 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-15T02:42:36.491 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-15T02:42:36.491 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-15T02:42:36.491 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-15T02:42:36.491 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-15T02:42:36.491 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 82577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-15T02:42:36.492 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-15T02:42:36.492 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-15T02:42:36.492 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-15T02:42:36.492 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-15T02:42:36.492 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/powr1220.ko 2023-11-15T02:42:36.493 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sch5627.ko 2023-11-15T02:42:36.493 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sch5636.ko 2023-11-15T02:42:36.493 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-15T02:42:36.493 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 32137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sht15.ko 2023-11-15T02:42:36.493 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sht21.ko 2023-11-15T02:42:36.494 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/shtc1.ko 2023-11-15T02:42:36.494 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 40105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sis5595.ko 2023-11-15T02:42:36.494 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/smsc47b397.ko 2023-11-15T02:42:36.494 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 39193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/smsc47m1.ko 2023-11-15T02:42:36.494 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 37385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/smsc47m192.ko 2023-11-15T02:42:36.495 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/thmc50.ko 2023-11-15T02:42:36.495 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/tmp102.ko 2023-11-15T02:42:36.495 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/tmp103.ko 2023-11-15T02:42:36.495 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/tmp401.ko 2023-11-15T02:42:36.495 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/tmp421.ko 2023-11-15T02:42:36.495 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/via-cputemp.ko 2023-11-15T02:42:36.496 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 41945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/via686a.ko 2023-11-15T02:42:36.496 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 52457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/vt1211.ko 2023-11-15T02:42:36.496 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 51545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/vt8231.ko 2023-11-15T02:42:36.496 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 57513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83627ehf.ko 2023-11-15T02:42:36.496 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 78873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83627hf.ko 2023-11-15T02:42:36.497 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 71345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83781d.ko 2023-11-15T02:42:36.497 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 63289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83791d.ko 2023-11-15T02:42:36.497 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 71321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83792d.ko 2023-11-15T02:42:36.497 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 101473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83793.ko 2023-11-15T02:42:36.497 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 119505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83795.ko 2023-11-15T02:42:36.498 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83l785ts.ko 2023-11-15T02:42:36.498 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 35625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83l786ng.ko 2023-11-15T02:42:36.498 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-11-15T02:42:36.498 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-11-15T02:42:36.498 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/ 2023-11-15T02:42:36.499 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/algos/ 2023-11-15T02:42:36.499 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-15T02:42:36.499 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-15T02:42:36.499 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/ 2023-11-15T02:42:36.499 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-11-15T02:42:36.499 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-11-15T02:42:36.500 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-11-15T02:42:36.500 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-15T02:42:36.500 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-15T02:42:36.500 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-15T02:42:36.500 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-11-15T02:42:36.501 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 6929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-15T02:42:36.501 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-11-15T02:42:36.501 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 50201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-15T02:42:36.501 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-15T02:42:36.501 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-15T02:42:36.502 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-15T02:42:36.502 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-11-15T02:42:36.502 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-11-15T02:42:36.502 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 62817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-15T02:42:36.502 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-15T02:42:36.503 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 47889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-15T02:42:36.503 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-11-15T02:42:36.503 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-15T02:42:36.503 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-15T02:42:36.503 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-11-15T02:42:36.503 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-15T02:42:36.504 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-15T02:42:36.504 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 47241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-15T02:42:36.504 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-11-15T02:42:36.504 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-15T02:42:36.504 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-15T02:42:36.505 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-11-15T02:42:36.505 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-11-15T02:42:36.505 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-15T02:42:36.505 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-11-15T02:42:36.505 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-15T02:42:36.506 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-15T02:42:36.506 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-15T02:42:36.506 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-15T02:42:36.506 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 47657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-11-15T02:42:36.506 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/i2c-mux.ko 2023-11-15T02:42:36.507 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/i2c-smbus.ko 2023-11-15T02:42:36.507 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/i2c-stub.ko 2023-11-15T02:42:36.507 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/muxes/ 2023-11-15T02:42:36.507 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-11-15T02:42:36.507 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-11-15T02:42:36.508 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-11-15T02:42:36.508 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ 2023-11-15T02:42:36.508 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ 2023-11-15T02:42:36.508 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 273609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-15T02:42:36.508 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 879969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ib_core.ko 2023-11-15T02:42:36.509 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 69337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-15T02:42:36.509 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 324217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-15T02:42:36.509 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 97985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-15T02:42:36.509 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 282273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-15T02:42:36.509 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 66201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-15T02:42:36.510 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/ 2023-11-15T02:42:36.510 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/cxgb4/ 2023-11-15T02:42:36.510 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 493073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-15T02:42:36.510 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mlx4/ 2023-11-15T02:42:36.510 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 419737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-15T02:42:36.511 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mlx5/ 2023-11-15T02:42:36.511 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 739025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-15T02:42:36.511 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mthca/ 2023-11-15T02:42:36.511 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 296625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-15T02:42:36.511 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/ocrdma/ 2023-11-15T02:42:36.512 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 153697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-11-15T02:42:36.512 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/usnic/ 2023-11-15T02:42:36.512 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 168969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-11-15T02:42:36.512 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/ 2023-11-15T02:42:36.512 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/ipoib/ 2023-11-15T02:42:36.513 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 263081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-15T02:42:36.513 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/iser/ 2023-11-15T02:42:36.513 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 100241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-15T02:42:36.513 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/isert/ 2023-11-15T02:42:36.513 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 111425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-15T02:42:36.513 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/srp/ 2023-11-15T02:42:36.514 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 128753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-15T02:42:36.514 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/srpt/ 2023-11-15T02:42:36.514 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 129529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-15T02:42:36.514 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/ 2023-11-15T02:42:36.515 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/evbug.ko 2023-11-15T02:42:36.515 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/ff-memless.ko 2023-11-15T02:42:36.515 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/gameport/ 2023-11-15T02:42:36.515 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-15T02:42:36.515 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-15T02:42:36.515 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 41241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/gameport/gameport.ko 2023-11-15T02:42:36.516 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 41801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/joydev.ko 2023-11-15T02:42:36.516 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/ 2023-11-15T02:42:36.516 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-11-15T02:42:36.516 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-11-15T02:42:36.516 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-11-15T02:42:36.517 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-11-15T02:42:36.517 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-15T02:42:36.517 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-11-15T02:42:36.517 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/lkkbd.ko 2023-11-15T02:42:36.517 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/lm8323.ko 2023-11-15T02:42:36.518 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/lm8333.ko 2023-11-15T02:42:36.518 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-11-15T02:42:36.518 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-11-15T02:42:36.518 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-11-15T02:42:36.518 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-11-15T02:42:36.519 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/newtonkbd.ko 2023-11-15T02:42:36.519 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-11-15T02:42:36.519 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/qt1070.ko 2023-11-15T02:42:36.519 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/qt2160.ko 2023-11-15T02:42:36.519 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-11-15T02:42:36.520 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/stowaway.ko 2023-11-15T02:42:36.520 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/sunkbd.ko 2023-11-15T02:42:36.520 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-11-15T02:42:36.520 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-11-15T02:42:36.520 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-11-15T02:42:36.521 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/xtkbd.ko 2023-11-15T02:42:36.521 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/matrix-keymap.ko 2023-11-15T02:42:36.521 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/ 2023-11-15T02:42:36.521 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/appletouch.ko 2023-11-15T02:42:36.521 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/bcm5974.ko 2023-11-15T02:42:36.522 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 123681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/cyapatp.ko 2023-11-15T02:42:36.522 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 89449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-15T02:42:36.522 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/gpio_mouse.ko 2023-11-15T02:42:36.522 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 335977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/psmouse.ko 2023-11-15T02:42:36.522 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/sermouse.ko 2023-11-15T02:42:36.523 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-15T02:42:36.523 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-15T02:42:36.523 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-15T02:42:36.523 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/rmi4/ 2023-11-15T02:42:36.523 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 109873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-15T02:42:36.524 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/ 2023-11-15T02:42:36.524 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/altera_ps2.ko 2023-11-15T02:42:36.524 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/arc_ps2.ko 2023-11-15T02:42:36.524 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/ct82c710.ko 2023-11-15T02:42:36.524 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-15T02:42:36.525 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/parkbd.ko 2023-11-15T02:42:36.525 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/pcips2.ko 2023-11-15T02:42:36.525 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/ps2mult.ko 2023-11-15T02:42:36.525 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/serio_raw.ko 2023-11-15T02:42:36.525 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/serport.ko 2023-11-15T02:42:36.526 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/sparse-keymap.ko 2023-11-15T02:42:36.526 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/iommu/ 2023-11-15T02:42:36.526 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/iommu/amd/ 2023-11-15T02:42:36.526 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 39289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-15T02:42:36.526 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/ 2023-11-15T02:42:36.527 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-88pm860x.ko 2023-11-15T02:42:36.527 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-adp5520.ko 2023-11-15T02:42:36.527 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 59753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-bd2802.ko 2023-11-15T02:42:36.527 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-blinkm.ko 2023-11-15T02:42:36.527 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-da903x.ko 2023-11-15T02:42:36.528 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-da9052.ko 2023-11-15T02:42:36.528 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-gpio.ko 2023-11-15T02:42:36.528 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lm3530.ko 2023-11-15T02:42:36.528 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lm3533.ko 2023-11-15T02:42:36.528 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lm355x.ko 2023-11-15T02:42:36.528 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lm3642.ko 2023-11-15T02:42:36.529 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lp3944.ko 2023-11-15T02:42:36.529 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lp8788.ko 2023-11-15T02:42:36.529 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-max8997.ko 2023-11-15T02:42:36.529 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-mc13783.ko 2023-11-15T02:42:36.529 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-menf21bmc.ko 2023-11-15T02:42:36.530 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-pca9532.ko 2023-11-15T02:42:36.530 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-pca955x.ko 2023-11-15T02:42:36.530 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-pca963x.ko 2023-11-15T02:42:36.530 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-ss4200.ko 2023-11-15T02:42:36.530 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-tca6507.ko 2023-11-15T02:42:36.531 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-wm831x-status.ko 2023-11-15T02:42:36.531 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-wm8350.ko 2023-11-15T02:42:36.531 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ 2023-11-15T02:42:36.531 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-15T02:42:36.531 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-15T02:42:36.532 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-15T02:42:36.532 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-15T02:42:36.532 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-15T02:42:36.532 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-15T02:42:36.532 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-15T02:42:36.533 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-15T02:42:36.533 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/ 2023-11-15T02:42:36.533 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/bcache/ 2023-11-15T02:42:36.533 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 654969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/bcache/bcache.ko 2023-11-15T02:42:36.533 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 40577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-bio-prison.ko 2023-11-15T02:42:36.534 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 89601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-bufio.ko 2023-11-15T02:42:36.534 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 47529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-cache-smq.ko 2023-11-15T02:42:36.534 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 156009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-cache.ko 2023-11-15T02:42:36.534 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 106897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-crypt.ko 2023-11-15T02:42:36.534 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-delay.ko 2023-11-15T02:42:36.535 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 53857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-era.ko 2023-11-15T02:42:36.535 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-flakey.ko 2023-11-15T02:42:36.535 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 41593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-log-userspace.ko 2023-11-15T02:42:36.535 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 32737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-log.ko 2023-11-15T02:42:36.535 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 49545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-mirror.ko 2023-11-15T02:42:36.536 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 78033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-multipath.ko 2023-11-15T02:42:36.536 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-queue-length.ko 2023-11-15T02:42:36.536 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 70825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-raid.ko 2023-11-15T02:42:36.536 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-region-hash.ko 2023-11-15T02:42:36.536 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-round-robin.ko 2023-11-15T02:42:36.536 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-service-time.ko 2023-11-15T02:42:36.537 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 109593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-snapshot.ko 2023-11-15T02:42:36.537 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-switch.ko 2023-11-15T02:42:36.537 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 181865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-thin-pool.ko 2023-11-15T02:42:36.537 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 51737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-verity.ko 2023-11-15T02:42:36.537 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-zero.ko 2023-11-15T02:42:36.538 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/faulty.ko 2023-11-15T02:42:36.538 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/linear.ko 2023-11-15T02:42:36.538 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 32417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/multipath.ko 2023-11-15T02:42:36.538 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/persistent-data/ 2023-11-15T02:42:36.538 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 199937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-15T02:42:36.539 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 37337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/raid0.ko 2023-11-15T02:42:36.539 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 113073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/raid1.ko 2023-11-15T02:42:36.539 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 158681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/raid10.ko 2023-11-15T02:42:36.539 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 395721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/raid456.ko 2023-11-15T02:42:36.539 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/ 2023-11-15T02:42:36.540 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/ 2023-11-15T02:42:36.540 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 178137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptbase.ko 2023-11-15T02:42:36.540 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 63385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptctl.ko 2023-11-15T02:42:36.540 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 44665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptfc.ko 2023-11-15T02:42:36.540 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 37225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptlan.ko 2023-11-15T02:42:36.541 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 123321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptsas.ko 2023-11-15T02:42:36.541 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 79041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptscsih.ko 2023-11-15T02:42:36.541 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 53081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptspi.ko 2023-11-15T02:42:36.541 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/ 2023-11-15T02:42:36.541 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/88pm800.ko 2023-11-15T02:42:36.542 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/88pm805.ko 2023-11-15T02:42:36.542 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/88pm80x.ko 2023-11-15T02:42:36.542 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/arizona-i2c.ko 2023-11-15T02:42:36.542 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 123721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/arizona.ko 2023-11-15T02:42:36.542 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/bcm590xx.ko 2023-11-15T02:42:36.543 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/cros_ec_dev.ko 2023-11-15T02:42:36.543 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/dln2.ko 2023-11-15T02:42:36.543 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/janz-cmodio.ko 2023-11-15T02:42:36.543 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 53585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/kempld-core.ko 2023-11-15T02:42:36.543 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lm3533-core.ko 2023-11-15T02:42:36.543 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-11-15T02:42:36.544 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lp3943.ko 2023-11-15T02:42:36.544 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 54569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lpc_ich.ko 2023-11-15T02:42:36.544 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lpc_sch.ko 2023-11-15T02:42:36.544 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/max8907.ko 2023-11-15T02:42:36.544 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/mc13xxx-core.ko 2023-11-15T02:42:36.545 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-11-15T02:42:36.545 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/menf21bmc.ko 2023-11-15T02:42:36.545 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/pcf50633-adc.ko 2023-11-15T02:42:36.545 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/pcf50633-gpio.ko 2023-11-15T02:42:36.545 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/pcf50633.ko 2023-11-15T02:42:36.546 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-11-15T02:42:36.546 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/retu-mfd.ko 2023-11-15T02:42:36.546 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 80425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/si476x-core.ko 2023-11-15T02:42:36.546 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 48457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/sm501.ko 2023-11-15T02:42:36.547 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-11-15T02:42:36.547 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 51137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/tps65010.ko 2023-11-15T02:42:36.547 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/tps6507x.ko 2023-11-15T02:42:36.547 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/viperboard.ko 2023-11-15T02:42:36.547 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/vx855.ko 2023-11-15T02:42:36.547 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/wl1273-core.ko 2023-11-15T02:42:36.548 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 44345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/wm8994.ko 2023-11-15T02:42:36.548 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ 2023-11-15T02:42:36.548 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-11-15T02:42:36.548 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 48489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ad525x_dpot.ko 2023-11-15T02:42:36.548 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/altera-stapl/ 2023-11-15T02:42:36.549 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 46585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-15T02:42:36.549 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/apds9802als.ko 2023-11-15T02:42:36.549 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/apds990x.ko 2023-11-15T02:42:36.549 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 45593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/bh1770glc.ko 2023-11-15T02:42:36.549 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/c2port/ 2023-11-15T02:42:36.550 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-11-15T02:42:36.550 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 35281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/c2port/core.ko 2023-11-15T02:42:36.550 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/cb710/ 2023-11-15T02:42:36.550 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/cb710/cb710.ko 2023-11-15T02:42:36.550 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ds1682.ko 2023-11-15T02:42:36.551 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/dummy-irq.ko 2023-11-15T02:42:36.551 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/echo/ 2023-11-15T02:42:36.551 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/echo/echo.ko 2023-11-15T02:42:36.551 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/ 2023-11-15T02:42:36.551 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 37849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/at24.ko 2023-11-15T02:42:36.551 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-15T02:42:36.552 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-15T02:42:36.552 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/max6875.ko 2023-11-15T02:42:36.552 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/enclosure.ko 2023-11-15T02:42:36.552 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/genwqe/ 2023-11-15T02:42:36.552 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 152201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-11-15T02:42:36.553 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/hmc6352.ko 2023-11-15T02:42:36.553 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 31321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/hpilo.ko 2023-11-15T02:42:36.553 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ibmasm/ 2023-11-15T02:42:36.553 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 72129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-11-15T02:42:36.553 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ics932s401.ko 2023-11-15T02:42:36.553 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/isl29003.ko 2023-11-15T02:42:36.554 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/isl29020.ko 2023-11-15T02:42:36.554 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/lis3lv02d/ 2023-11-15T02:42:36.554 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 42513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-15T02:42:36.554 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-15T02:42:36.555 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/mei/ 2023-11-15T02:42:36.555 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 125209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/mei/mei-me.ko 2023-11-15T02:42:36.555 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 56521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/mei/mei-txe.ko 2023-11-15T02:42:36.555 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 310113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/mei/mei.ko 2023-11-15T02:42:36.555 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 31329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/phantom.ko 2023-11-15T02:42:36.555 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ti-st/ 2023-11-15T02:42:36.556 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 74265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ti-st/st_drv.ko 2023-11-15T02:42:36.556 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/tifm_7xx1.ko 2023-11-15T02:42:36.556 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/tifm_core.ko 2023-11-15T02:42:36.556 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/tsl2550.ko 2023-11-15T02:42:36.556 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 48697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/vmw_balloon.ko 2023-11-15T02:42:36.557 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/vmw_vmci/ 2023-11-15T02:42:36.557 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 183609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-15T02:42:36.557 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ 2023-11-15T02:42:36.557 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/bonding/ 2023-11-15T02:42:36.557 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 490129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/bonding/bonding.ko 2023-11-15T02:42:36.658 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/ 2023-11-15T02:42:36.658 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/b53/ 2023-11-15T02:42:36.658 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 106033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-15T02:42:36.659 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 40297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-15T02:42:36.659 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-15T02:42:36.659 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/mv88e6xxx/ 2023-11-15T02:42:36.659 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 453737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-15T02:42:36.659 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dummy.ko 2023-11-15T02:42:36.659 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/eql.ko 2023-11-15T02:42:36.660 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ 2023-11-15T02:42:36.660 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/ 2023-11-15T02:42:36.660 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 45785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-15T02:42:36.660 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 42105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-15T02:42:36.660 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 130553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-15T02:42:36.661 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 68849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-15T02:42:36.661 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/ 2023-11-15T02:42:36.661 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 37649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-15T02:42:36.661 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 59689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-15T02:42:36.661 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-15T02:42:36.662 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 99113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-15T02:42:36.662 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/adaptec/ 2023-11-15T02:42:36.662 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 54521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-15T02:42:36.662 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/agere/ 2023-11-15T02:42:36.662 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 69201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-15T02:42:36.663 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/alteon/ 2023-11-15T02:42:36.663 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 59585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-15T02:42:36.663 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/altera/ 2023-11-15T02:42:36.663 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 41409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-15T02:42:36.663 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/amd/ 2023-11-15T02:42:36.664 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 54609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-15T02:42:36.664 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 49289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-15T02:42:36.664 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 87297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-15T02:42:36.664 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/ 2023-11-15T02:42:36.664 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/alx/ 2023-11-15T02:42:36.664 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 97337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-15T02:42:36.665 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-11-15T02:42:36.665 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 104673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-15T02:42:36.665 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-11-15T02:42:36.665 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 90257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-15T02:42:36.665 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atlx/ 2023-11-15T02:42:36.666 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 81569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-15T02:42:36.666 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 69729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-15T02:42:36.666 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/ 2023-11-15T02:42:36.666 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 78401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-15T02:42:36.666 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 176761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-15T02:42:36.667 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-11-15T02:42:36.667 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 1421497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-15T02:42:36.667 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 128569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-15T02:42:36.667 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/genet/ 2023-11-15T02:42:36.667 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 132089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-15T02:42:36.668 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 312081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-15T02:42:36.668 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/brocade/ 2023-11-15T02:42:36.668 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/brocade/bna/ 2023-11-15T02:42:36.668 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 323513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-15T02:42:36.668 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/ 2023-11-15T02:42:36.669 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-11-15T02:42:36.669 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 193497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-15T02:42:36.669 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-11-15T02:42:36.669 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 347417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-15T02:42:36.669 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-11-15T02:42:36.670 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 1044433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-15T02:42:36.670 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-11-15T02:42:36.670 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 166193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-15T02:42:36.670 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-11-15T02:42:36.670 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-15T02:42:36.671 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/cisco/ 2023-11-15T02:42:36.671 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/cisco/enic/ 2023-11-15T02:42:36.671 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 204345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-15T02:42:36.671 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/ 2023-11-15T02:42:36.671 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/ 2023-11-15T02:42:36.672 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 69449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-15T02:42:36.672 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 61489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-15T02:42:36.672 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 141225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-15T02:42:36.672 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 51497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-15T02:42:36.672 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 62817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-15T02:42:36.673 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 31369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-15T02:42:36.673 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dlink/ 2023-11-15T02:42:36.673 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 60873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-15T02:42:36.673 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 62833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-15T02:42:36.673 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 45721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dnet.ko 2023-11-15T02:42:36.674 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ec_bhf.ko 2023-11-15T02:42:36.674 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/emulex/ 2023-11-15T02:42:36.674 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/emulex/benet/ 2023-11-15T02:42:36.674 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 281137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-15T02:42:36.674 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 49457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ethoc.ko 2023-11-15T02:42:36.674 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 51737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/fealnx.ko 2023-11-15T02:42:36.675 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/fujitsu/ 2023-11-15T02:42:36.675 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 45345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-11-15T02:42:36.675 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ 2023-11-15T02:42:36.675 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 90465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-15T02:42:36.675 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e1000/ 2023-11-15T02:42:36.676 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 295577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-15T02:42:36.676 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e1000e/ 2023-11-15T02:42:36.676 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 584497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-15T02:42:36.676 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/fm10k/ 2023-11-15T02:42:36.676 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 246209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-15T02:42:36.677 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/i40e/ 2023-11-15T02:42:36.677 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 925697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-15T02:42:36.677 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/iavf/ 2023-11-15T02:42:36.677 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 302849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-15T02:42:36.677 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/igb/ 2023-11-15T02:42:36.678 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 514681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-15T02:42:36.678 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/igbvf/ 2023-11-15T02:42:36.678 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 93593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-15T02:42:36.678 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-11-15T02:42:36.678 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 734305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-15T02:42:36.679 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-11-15T02:42:36.679 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 151769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-15T02:42:36.679 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 86921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/jme.ko 2023-11-15T02:42:36.679 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/marvell/ 2023-11-15T02:42:36.679 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-15T02:42:36.679 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 102289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-15T02:42:36.680 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 122649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-15T02:42:36.680 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/ 2023-11-15T02:42:36.680 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-11-15T02:42:36.680 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 727601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-15T02:42:36.680 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 278937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-15T02:42:36.681 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-11-15T02:42:36.681 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-11-15T02:42:36.681 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 1113993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-15T02:42:36.681 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ 2023-11-15T02:42:36.681 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 53105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-11-15T02:42:36.682 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 40993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-11-15T02:42:36.682 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-11-15T02:42:36.682 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 95665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-15T02:42:36.682 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/myricom/ 2023-11-15T02:42:36.682 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-11-15T02:42:36.683 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 108649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-15T02:42:36.683 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/natsemi/ 2023-11-15T02:42:36.683 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 73441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-15T02:42:36.683 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 47777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-15T02:42:36.683 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/neterion/ 2023-11-15T02:42:36.684 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 150905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-15T02:42:36.684 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/nvidia/ 2023-11-15T02:42:36.684 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 136721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-15T02:42:36.684 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/packetengines/ 2023-11-15T02:42:36.684 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 50481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-15T02:42:36.685 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 43145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-15T02:42:36.685 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/ 2023-11-15T02:42:36.685 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-11-15T02:42:36.685 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 227897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-15T02:42:36.685 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 87081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-15T02:42:36.685 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-11-15T02:42:36.686 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 672049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-15T02:42:36.686 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/rdc/ 2023-11-15T02:42:36.686 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 43017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-15T02:42:36.686 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/ 2023-11-15T02:42:36.686 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 67385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-15T02:42:36.687 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 83985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-15T02:42:36.687 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-15T02:42:36.687 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 189801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-15T02:42:36.687 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/samsung/ 2023-11-15T02:42:36.687 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-11-15T02:42:36.688 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 104241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-11-15T02:42:36.688 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sfc/ 2023-11-15T02:42:36.688 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 1053921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-15T02:42:36.688 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/silan/ 2023-11-15T02:42:36.688 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 47097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-15T02:42:36.689 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sis/ 2023-11-15T02:42:36.689 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 54817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-15T02:42:36.689 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 63337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-15T02:42:36.689 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/ 2023-11-15T02:42:36.689 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 61601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-15T02:42:36.690 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 67193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-15T02:42:36.690 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 73409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-15T02:42:36.690 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 68177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-15T02:42:36.690 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/ 2023-11-15T02:42:36.690 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-11-15T02:42:36.691 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-11-15T02:42:36.691 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 39905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-15T02:42:36.691 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-11-15T02:42:36.691 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 486961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-15T02:42:36.691 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/ 2023-11-15T02:42:36.692 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 103337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-15T02:42:36.692 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 166161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-15T02:42:36.692 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 66329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-15T02:42:36.692 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 78801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-15T02:42:36.692 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/tehuti/ 2023-11-15T02:42:36.693 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 50057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-15T02:42:36.693 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ti/ 2023-11-15T02:42:36.693 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 86529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-15T02:42:36.693 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/via/ 2023-11-15T02:42:36.693 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 80273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-15T02:42:36.693 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 83633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-15T02:42:36.694 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/wiznet/ 2023-11-15T02:42:36.694 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 46241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-15T02:42:36.694 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-15T02:42:36.694 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/xircom/ 2023-11-15T02:42:36.695 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 60057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-15T02:42:36.695 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/fddi/ 2023-11-15T02:42:36.695 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 41065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/fddi/defxx.ko 2023-11-15T02:42:36.695 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/fddi/skfp/ 2023-11-15T02:42:36.695 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 216985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/fddi/skfp/skfp.ko 2023-11-15T02:42:36.695 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 77345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/geneve.ko 2023-11-15T02:42:36.696 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/hyperv/ 2023-11-15T02:42:36.696 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 235521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-15T02:42:36.696 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ieee802154/ 2023-11-15T02:42:36.696 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-15T02:42:36.696 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ifb.ko 2023-11-15T02:42:36.697 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ipvlan/ 2023-11-15T02:42:36.697 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 83769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-15T02:42:36.697 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 62209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/macvlan.ko 2023-11-15T02:42:36.697 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/macvtap.ko 2023-11-15T02:42:36.697 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/ 2023-11-15T02:42:36.698 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-15T02:42:36.698 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-15T02:42:36.698 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/mdio-gpio.ko 2023-11-15T02:42:36.698 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-15T02:42:36.698 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio.ko 2023-11-15T02:42:36.699 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mii.ko 2023-11-15T02:42:36.699 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 43953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/netconsole.ko 2023-11-15T02:42:36.699 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/nlmon.ko 2023-11-15T02:42:36.699 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/pcs/ 2023-11-15T02:42:36.699 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-15T02:42:36.700 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 38977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-15T02:42:36.700 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/ 2023-11-15T02:42:36.700 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/amd.ko 2023-11-15T02:42:36.700 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 50097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-15T02:42:36.700 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 44505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-15T02:42:36.701 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/bcm87xx.ko 2023-11-15T02:42:36.701 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 42241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/broadcom.ko 2023-11-15T02:42:36.701 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/cicada.ko 2023-11-15T02:42:36.701 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/davicom.ko 2023-11-15T02:42:36.701 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/et1011c.ko 2023-11-15T02:42:36.702 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/icplus.ko 2023-11-15T02:42:36.702 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/lxt.ko 2023-11-15T02:42:36.702 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 82049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/marvell.ko 2023-11-15T02:42:36.702 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 76177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/micrel.ko 2023-11-15T02:42:36.702 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/national.ko 2023-11-15T02:42:36.703 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 97857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/phylink.ko 2023-11-15T02:42:36.703 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/qsemi.ko 2023-11-15T02:42:36.703 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 50065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/realtek.ko 2023-11-15T02:42:36.703 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/smsc.ko 2023-11-15T02:42:36.703 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/ste10Xp.ko 2023-11-15T02:42:36.704 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/vitesse.ko 2023-11-15T02:42:36.704 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/plip/ 2023-11-15T02:42:36.704 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 43569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/plip/plip.ko 2023-11-15T02:42:36.704 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ 2023-11-15T02:42:36.704 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-15T02:42:36.705 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ppp_async.ko 2023-11-15T02:42:36.705 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-15T02:42:36.705 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-15T02:42:36.705 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-15T02:42:36.705 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 42545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/pppoe.ko 2023-11-15T02:42:36.705 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/pppox.ko 2023-11-15T02:42:36.706 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 32689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/pptp.ko 2023-11-15T02:42:36.706 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/sb1000.ko 2023-11-15T02:42:36.706 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/slip/ 2023-11-15T02:42:36.706 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 42465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/slip/slip.ko 2023-11-15T02:42:36.706 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/sungem_phy.ko 2023-11-15T02:42:36.707 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 64385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/tap.ko 2023-11-15T02:42:36.707 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/ 2023-11-15T02:42:36.707 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 106169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team.ko 2023-11-15T02:42:36.707 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-15T02:42:36.707 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-15T02:42:36.708 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-15T02:42:36.708 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_random.ko 2023-11-15T02:42:36.708 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-15T02:42:36.708 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 87017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/veth.ko 2023-11-15T02:42:36.708 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/vmxnet3/ 2023-11-15T02:42:36.709 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 144297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-15T02:42:36.709 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/vxlan/ 2023-11-15T02:42:36.709 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 297329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/vxlan/vxlan.ko 2023-11-15T02:42:36.709 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/xen-netback/ 2023-11-15T02:42:36.709 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 144649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-15T02:42:36.710 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ntb/ 2023-11-15T02:42:36.710 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ntb/ntb.ko 2023-11-15T02:42:36.710 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/nvme/ 2023-11-15T02:42:36.710 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/nvme/host/ 2023-11-15T02:42:36.710 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 264937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/nvme/host/nvme-core.ko 2023-11-15T02:42:36.711 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 102209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/nvme/host/nvme.ko 2023-11-15T02:42:36.711 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/ 2023-11-15T02:42:36.711 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 133841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/parport.ko 2023-11-15T02:42:36.711 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/parport_cs.ko 2023-11-15T02:42:36.711 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 84465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/parport_pc.ko 2023-11-15T02:42:36.712 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/parport_serial.ko 2023-11-15T02:42:36.712 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/ 2023-11-15T02:42:36.712 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/hotplug/ 2023-11-15T02:42:36.712 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-15T02:42:36.713 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-11-15T02:42:36.713 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-11-15T02:42:36.713 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/pci-stub.ko 2023-11-15T02:42:36.713 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 46529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/xen-pcifront.ko 2023-11-15T02:42:36.713 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/ 2023-11-15T02:42:36.713 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/i82092.ko 2023-11-15T02:42:36.714 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 145809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/pcmcia.ko 2023-11-15T02:42:36.714 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 62137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/pcmcia_core.ko 2023-11-15T02:42:36.714 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-11-15T02:42:36.714 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/pd6729.ko 2023-11-15T02:42:36.714 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 112889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-15T02:42:36.715 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/ 2023-11-15T02:42:36.715 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/broadcom/ 2023-11-15T02:42:36.715 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-11-15T02:42:36.715 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/samsung/ 2023-11-15T02:42:36.715 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-11-15T02:42:36.716 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pinctrl/ 2023-11-15T02:42:36.716 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pinctrl/intel/ 2023-11-15T02:42:36.716 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 70153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-15T02:42:36.716 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/ 2023-11-15T02:42:36.716 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/ 2023-11-15T02:42:36.717 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 50889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-11-15T02:42:36.717 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec.ko 2023-11-15T02:42:36.717 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-11-15T02:42:36.717 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-11-15T02:42:36.717 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-11-15T02:42:36.717 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-11-15T02:42:36.718 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-11-15T02:42:36.718 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/ 2023-11-15T02:42:36.718 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 67097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-15T02:42:36.718 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 39145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/acerhdf.ko 2023-11-15T02:42:36.718 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-15T02:42:36.719 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 59665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-15T02:42:36.719 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 31097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-15T02:42:36.719 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 127569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-15T02:42:36.719 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-15T02:42:36.719 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 49121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-15T02:42:36.720 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-15T02:42:36.720 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 40265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-15T02:42:36.720 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-15T02:42:36.720 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/hdaps.ko 2023-11-15T02:42:36.720 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/ibm_rtl.ko 2023-11-15T02:42:36.721 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/intel/ 2023-11-15T02:42:36.721 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-15T02:42:36.721 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-11-15T02:42:36.721 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 43497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/intel_ips.ko 2023-11-15T02:42:36.721 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-15T02:42:36.722 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-15T02:42:36.722 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-15T02:42:36.722 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 42329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-15T02:42:36.722 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-15T02:42:36.722 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-15T02:42:36.723 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-15T02:42:36.723 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-15T02:42:36.723 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-15T02:42:36.723 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 57569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/wmi.ko 2023-11-15T02:42:36.723 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/ 2023-11-15T02:42:36.724 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ 2023-11-15T02:42:36.724 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/88pm860x_battery.ko 2023-11-15T02:42:36.724 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/88pm860x_charger.ko 2023-11-15T02:42:36.724 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 55057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/bq2415x_charger.ko 2023-11-15T02:42:36.724 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 54081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/bq24190_charger.ko 2023-11-15T02:42:36.724 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/bq24735-charger.ko 2023-11-15T02:42:36.725 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-11-15T02:42:36.725 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/da9030_battery.ko 2023-11-15T02:42:36.725 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/da9052-battery.ko 2023-11-15T02:42:36.725 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ds2760_battery.ko 2023-11-15T02:42:36.725 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ds2780_battery.ko 2023-11-15T02:42:36.726 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ds2781_battery.ko 2023-11-15T02:42:36.726 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ds2782_battery.ko 2023-11-15T02:42:36.726 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/gpio-charger.ko 2023-11-15T02:42:36.726 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/isp1704_charger.ko 2023-11-15T02:42:36.726 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/lp8727_charger.ko 2023-11-15T02:42:36.727 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max14577_charger.ko 2023-11-15T02:42:36.727 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max17040_battery.ko 2023-11-15T02:42:36.727 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max17042_battery.ko 2023-11-15T02:42:36.727 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max8903_charger.ko 2023-11-15T02:42:36.727 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max8925_power.ko 2023-11-15T02:42:36.728 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/pcf50633-charger.ko 2023-11-15T02:42:36.728 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 38009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/sbs-battery.ko 2023-11-15T02:42:36.728 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/test_power.ko 2023-11-15T02:42:36.728 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/tps65090-charger.ko 2023-11-15T02:42:36.728 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/wm831x_backup.ko 2023-11-15T02:42:36.728 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/wm831x_power.ko 2023-11-15T02:42:36.729 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/wm8350_power.ko 2023-11-15T02:42:36.729 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/powercap/ 2023-11-15T02:42:36.729 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 57729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-15T02:42:36.729 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-15T02:42:36.729 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/ 2023-11-15T02:42:36.730 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/clients/ 2023-11-15T02:42:36.730 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-15T02:42:36.730 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-15T02:42:36.730 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/clients/pps_parport.ko 2023-11-15T02:42:36.730 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 41873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/pps_core.ko 2023-11-15T02:42:36.731 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ptp/ 2023-11-15T02:42:36.731 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 66385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ptp/ptp.ko 2023-11-15T02:42:36.731 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ptp/ptp_kvm.ko 2023-11-15T02:42:36.731 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/ 2023-11-15T02:42:36.731 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-88pm80x.ko 2023-11-15T02:42:36.731 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-88pm860x.ko 2023-11-15T02:42:36.732 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-15T02:42:36.732 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-da9052.ko 2023-11-15T02:42:36.732 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-da9055.ko 2023-11-15T02:42:36.732 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-da9063.ko 2023-11-15T02:42:36.732 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-15T02:42:36.733 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 68577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-15T02:42:36.733 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-15T02:42:36.733 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-15T02:42:36.733 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-15T02:42:36.733 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-15T02:42:36.733 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-15T02:42:36.734 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-15T02:42:36.734 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-15T02:42:36.734 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-em3027.ko 2023-11-15T02:42:36.734 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-15T02:42:36.735 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-15T02:42:36.735 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 35969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-15T02:42:36.735 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-lp8788.ko 2023-11-15T02:42:36.735 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 38097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-15T02:42:36.735 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-15T02:42:36.735 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-15T02:42:36.736 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-m48t86.ko 2023-11-15T02:42:36.736 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max6900.ko 2023-11-15T02:42:36.736 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max8907.ko 2023-11-15T02:42:36.736 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max8925.ko 2023-11-15T02:42:36.736 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max8997.ko 2023-11-15T02:42:36.737 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max8998.ko 2023-11-15T02:42:36.737 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-11-15T02:42:36.737 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-15T02:42:36.737 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-palmas.ko 2023-11-15T02:42:36.737 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 46633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-15T02:42:36.738 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf50633.ko 2023-11-15T02:42:36.738 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-15T02:42:36.738 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-15T02:42:36.738 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-15T02:42:36.738 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-15T02:42:36.739 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rc5t583.ko 2023-11-15T02:42:36.739 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-15T02:42:36.739 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-15T02:42:36.739 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-15T02:42:36.739 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-15T02:42:36.739 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-15T02:42:36.740 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-s35390a.ko 2023-11-15T02:42:36.740 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-15T02:42:36.740 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-tps6586x.ko 2023-11-15T02:42:36.740 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-tps65910.ko 2023-11-15T02:42:36.740 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-wm831x.ko 2023-11-15T02:42:36.741 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-wm8350.ko 2023-11-15T02:42:36.741 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-x1205.ko 2023-11-15T02:42:36.741 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ 2023-11-15T02:42:36.741 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 77617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/3w-9xxx.ko 2023-11-15T02:42:36.741 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 57665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/3w-sas.ko 2023-11-15T02:42:36.742 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 55169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/3w-xxxx.ko 2023-11-15T02:42:36.742 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 135185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/BusLogic.ko 2023-11-15T02:42:36.742 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/a100u2w.ko 2023-11-15T02:42:36.742 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aacraid/ 2023-11-15T02:42:36.742 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 272305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-15T02:42:36.742 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 100617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/advansys.ko 2023-11-15T02:42:36.743 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic7xxx/ 2023-11-15T02:42:36.743 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 356265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-15T02:42:36.743 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 295801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-15T02:42:36.743 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic94xx/ 2023-11-15T02:42:36.743 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 189185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-11-15T02:42:36.744 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/am53c974.ko 2023-11-15T02:42:36.744 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/arcmsr/ 2023-11-15T02:42:36.744 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 106881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-15T02:42:36.744 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 38185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/atp870u.ko 2023-11-15T02:42:36.744 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/be2iscsi/ 2023-11-15T02:42:36.745 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 246985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-15T02:42:36.745 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bfa/ 2023-11-15T02:42:36.745 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 1171945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bfa/bfa.ko 2023-11-15T02:42:36.745 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bnx2fc/ 2023-11-15T02:42:36.745 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 237273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-15T02:42:36.745 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bnx2i/ 2023-11-15T02:42:36.746 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 115369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-15T02:42:36.746 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 40585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ch.ko 2023-11-15T02:42:36.746 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/csiostor/ 2023-11-15T02:42:36.746 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 308921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-15T02:42:36.746 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/ 2023-11-15T02:42:36.747 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-11-15T02:42:36.747 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 76145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-15T02:42:36.747 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-11-15T02:42:36.747 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 101449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-15T02:42:36.747 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 146689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-15T02:42:36.748 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 75553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/dc395x.ko 2023-11-15T02:42:36.748 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 38137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/dmx3191d.ko 2023-11-15T02:42:36.748 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/esas2r/ 2023-11-15T02:42:36.748 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 212425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-15T02:42:36.748 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 60505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/esp_scsi.ko 2023-11-15T02:42:36.749 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fcoe/ 2023-11-15T02:42:36.749 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 78209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-15T02:42:36.749 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 158505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-15T02:42:36.749 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fdomain.ko 2023-11-15T02:42:36.749 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fnic/ 2023-11-15T02:42:36.750 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 263489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fnic/fnic.ko 2023-11-15T02:42:36.750 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 201817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/hpsa.ko 2023-11-15T02:42:36.750 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 51209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/hptiop.ko 2023-11-15T02:42:36.750 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 49041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/hv_storvsc.ko 2023-11-15T02:42:36.750 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 35337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/imm.ko 2023-11-15T02:42:36.750 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 44057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/initio.ko 2023-11-15T02:42:36.751 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 266537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ipr.ko 2023-11-15T02:42:36.751 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 101881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ips.ko 2023-11-15T02:42:36.751 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/isci/ 2023-11-15T02:42:36.751 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 317153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/isci/isci.ko 2023-11-15T02:42:36.751 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-15T02:42:36.752 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 52961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-15T02:42:36.752 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libfc/ 2023-11-15T02:42:36.752 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 314865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libfc/libfc.ko 2023-11-15T02:42:36.752 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 145585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libiscsi.ko 2023-11-15T02:42:36.752 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 53521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-15T02:42:36.753 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libsas/ 2023-11-15T02:42:36.753 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 216665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libsas/libsas.ko 2023-11-15T02:42:36.753 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/lpfc/ 2023-11-15T02:42:36.753 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 2197249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-15T02:42:36.753 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid/ 2023-11-15T02:42:36.754 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 73569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-15T02:42:36.754 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-15T02:42:36.754 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 310617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-15T02:42:36.754 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 84721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid.ko 2023-11-15T02:42:36.754 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mpt3sas/ 2023-11-15T02:42:36.755 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 659777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-15T02:42:36.755 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mvsas/ 2023-11-15T02:42:36.755 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 119561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-15T02:42:36.755 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 78129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mvumi.ko 2023-11-15T02:42:36.755 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/ 2023-11-15T02:42:36.756 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 81625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-11-15T02:42:36.756 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-11-15T02:42:36.756 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-11-15T02:42:36.756 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-11-15T02:42:36.756 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pm8001/ 2023-11-15T02:42:36.756 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 364105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-15T02:42:36.757 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 123617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pmcraid.ko 2023-11-15T02:42:36.757 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ppa.ko 2023-11-15T02:42:36.757 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 55361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla1280.ko 2023-11-15T02:42:36.757 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla2xxx/ 2023-11-15T02:42:36.757 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 1876625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-15T02:42:36.758 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 85289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-15T02:42:36.758 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla4xxx/ 2023-11-15T02:42:36.758 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 559033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-15T02:42:36.758 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qlogicfas408.ko 2023-11-15T02:42:36.758 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/raid_class.ko 2023-11-15T02:42:36.759 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 238825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_debug.ko 2023-11-15T02:42:36.759 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 173937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-15T02:42:36.759 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 279313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-15T02:42:36.759 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 97729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-15T02:42:36.759 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 73881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-15T02:42:36.760 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 44993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-15T02:42:36.760 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ses.ko 2023-11-15T02:42:36.760 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 115889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/st.ko 2023-11-15T02:42:36.760 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 46825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/stex.ko 2023-11-15T02:42:36.760 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/sym53c8xx_2/ 2023-11-15T02:42:36.761 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 162121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-15T02:42:36.761 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 42297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/virtio_scsi.ko 2023-11-15T02:42:36.761 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 50321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-15T02:42:36.761 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 40025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/wd719x.ko 2023-11-15T02:42:36.761 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 50025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/xen-scsifront.ko 2023-11-15T02:42:36.761 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ssb/ 2023-11-15T02:42:36.762 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 129441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ssb/ssb.ko 2023-11-15T02:42:36.762 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/ 2023-11-15T02:42:36.762 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/iscsi/ 2023-11-15T02:42:36.762 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 736601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-15T02:42:36.762 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/loopback/ 2023-11-15T02:42:36.763 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 49985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-15T02:42:36.763 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 39713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_file.ko 2023-11-15T02:42:36.763 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 43025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_iblock.ko 2023-11-15T02:42:36.763 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 930321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_mod.ko 2023-11-15T02:42:36.763 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 44305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_pscsi.ko 2023-11-15T02:42:36.764 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 122521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_user.ko 2023-11-15T02:42:36.764 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/tcm_fc/ 2023-11-15T02:42:36.764 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 75593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-15T02:42:36.764 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/ 2023-11-15T02:42:36.764 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/ 2023-11-15T02:42:36.764 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/ 2023-11-15T02:42:36.765 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-15T02:42:36.765 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-15T02:42:36.765 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-15T02:42:36.765 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-15T02:42:36.765 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-15T02:42:36.766 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-15T02:42:36.766 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-15T02:42:36.766 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-15T02:42:36.766 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-15T02:42:36.766 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-15T02:42:36.767 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-15T02:42:36.767 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 37065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-15T02:42:36.767 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-11-15T02:42:36.767 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-15T02:42:36.767 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-15T02:42:36.768 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-15T02:42:36.768 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/ 2023-11-15T02:42:36.768 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/ipwireless/ 2023-11-15T02:42:36.768 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 78425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-15T02:42:36.768 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 57865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/moxa.ko 2023-11-15T02:42:36.768 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 50953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/mxser.ko 2023-11-15T02:42:36.769 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 108601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/n_gsm.ko 2023-11-15T02:42:36.769 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 31961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/n_hdlc.ko 2023-11-15T02:42:36.769 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 67929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/nozomi.ko 2023-11-15T02:42:36.769 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/ 2023-11-15T02:42:36.769 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/8250/ 2023-11-15T02:42:36.770 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 35753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-11-15T02:42:36.770 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 59497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-15T02:42:36.770 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-11-15T02:42:36.770 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/altera_uart.ko 2023-11-15T02:42:36.770 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/arc_uart.ko 2023-11-15T02:42:36.771 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 72425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-11-15T02:42:36.771 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/jsm/ 2023-11-15T02:42:36.771 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 103073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-15T02:42:36.771 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/rp2.ko 2023-11-15T02:42:36.771 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 52817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/sc16is7xx.ko 2023-11-15T02:42:36.771 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/ 2023-11-15T02:42:36.772 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/core/ 2023-11-15T02:42:36.772 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 531265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-15T02:42:36.772 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/host/ 2023-11-15T02:42:36.772 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-15T02:42:36.772 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-11-15T02:42:36.773 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/ 2023-11-15T02:42:36.773 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 44513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio.ko 2023-11-15T02:42:36.773 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_aec.ko 2023-11-15T02:42:36.773 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_cif.ko 2023-11-15T02:42:36.773 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_dmem_genirq.ko 2023-11-15T02:42:36.774 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_mf624.ko 2023-11-15T02:42:36.774 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_netx.ko 2023-11-15T02:42:36.774 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_pci_generic.ko 2023-11-15T02:42:36.774 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-11-15T02:42:36.774 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_sercos3.ko 2023-11-15T02:42:36.775 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/ 2023-11-15T02:42:36.775 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/c67x00/ 2023-11-15T02:42:36.775 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 73297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/c67x00/c67x00.ko 2023-11-15T02:42:36.775 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ 2023-11-15T02:42:36.775 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 144377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-11-15T02:42:36.776 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-11-15T02:42:36.777 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-11-15T02:42:36.777 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-11-15T02:42:36.778 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/ 2023-11-15T02:42:36.778 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 87089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/cdc-acm.ko 2023-11-15T02:42:36.778 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 51409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-15T02:42:36.778 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 47369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/usblp.ko 2023-11-15T02:42:36.778 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 89857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/usbtmc.ko 2023-11-15T02:42:36.779 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/common/ 2023-11-15T02:42:36.779 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/common/ulpi.ko 2023-11-15T02:42:36.779 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc2/ 2023-11-15T02:42:36.779 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-11-15T02:42:36.779 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc3/ 2023-11-15T02:42:36.780 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-11-15T02:42:36.780 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-11-15T02:42:36.780 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 444553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc3/dwc3.ko 2023-11-15T02:42:36.780 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/ 2023-11-15T02:42:36.780 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/ 2023-11-15T02:42:36.781 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 60177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/u_ether.ko 2023-11-15T02:42:36.781 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 51881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/u_serial.ko 2023-11-15T02:42:36.781 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-11-15T02:42:36.781 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 42449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-11-15T02:42:36.781 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-11-15T02:42:36.782 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-11-15T02:42:36.782 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 92489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-11-15T02:42:36.782 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 51049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-11-15T02:42:36.782 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 122081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-11-15T02:42:36.782 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 57617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-11-15T02:42:36.783 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-11-15T02:42:36.783 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 31265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-11-15T02:42:36.783 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 57729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-11-15T02:42:36.783 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 85609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-11-15T02:42:36.783 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-11-15T02:42:36.783 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 58113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-11-15T02:42:36.784 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 67337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-11-15T02:42:36.784 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/ 2023-11-15T02:42:36.784 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-11-15T02:42:36.784 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-11-15T02:42:36.784 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-11-15T02:42:36.785 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-11-15T02:42:36.785 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-11-15T02:42:36.785 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-11-15T02:42:36.785 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-11-15T02:42:36.785 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-11-15T02:42:36.786 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 32073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-11-15T02:42:36.786 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-11-15T02:42:36.786 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-11-15T02:42:36.786 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-11-15T02:42:36.786 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 62185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-11-15T02:42:36.787 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-11-15T02:42:36.787 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 161225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/libcomposite.ko 2023-11-15T02:42:36.787 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/ 2023-11-15T02:42:36.787 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-11-15T02:42:36.787 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/bdc/ 2023-11-15T02:42:36.788 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 157681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-11-15T02:42:36.788 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-11-15T02:42:36.788 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 52721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-11-15T02:42:36.788 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 69705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-11-15T02:42:36.788 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 56033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-11-15T02:42:36.789 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 76057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/net2272.ko 2023-11-15T02:42:36.789 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 75073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/net2280.ko 2023-11-15T02:42:36.789 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 73169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-11-15T02:42:36.789 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 65809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-11-15T02:42:36.789 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 66097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-11-15T02:42:36.789 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 81505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-11-15T02:42:36.790 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 156073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-11-15T02:42:36.790 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/ 2023-11-15T02:42:36.790 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/bcma-hcd.ko 2023-11-15T02:42:36.790 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 80113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/isp116x-hcd.ko 2023-11-15T02:42:36.790 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 87353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-11-15T02:42:36.791 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 78873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-11-15T02:42:36.791 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 51329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-15T02:42:36.791 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/sl811_cs.ko 2023-11-15T02:42:36.791 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/ssb-hcd.ko 2023-11-15T02:42:36.791 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-11-15T02:42:36.792 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/image/ 2023-11-15T02:42:36.792 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/image/mdc800.ko 2023-11-15T02:42:36.792 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/image/microtek.ko 2023-11-15T02:42:36.792 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/ 2023-11-15T02:42:36.792 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 41721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/adutux.ko 2023-11-15T02:42:36.793 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/appledisplay.ko 2023-11-15T02:42:36.793 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-11-15T02:42:36.793 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/cytherm.ko 2023-11-15T02:42:36.793 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/ehset.ko 2023-11-15T02:42:36.793 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/emi26.ko 2023-11-15T02:42:36.794 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/emi62.ko 2023-11-15T02:42:36.794 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/ezusb.ko 2023-11-15T02:42:36.794 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/idmouse.ko 2023-11-15T02:42:36.794 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/iowarrior.ko 2023-11-15T02:42:36.794 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-15T02:42:36.795 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/ldusb.ko 2023-11-15T02:42:36.795 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/legousbtower.ko 2023-11-15T02:42:36.795 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/lvstest.ko 2023-11-15T02:42:36.795 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/sisusbvga/ 2023-11-15T02:42:36.795 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 64841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-15T02:42:36.796 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-15T02:42:36.796 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/usb3503.ko 2023-11-15T02:42:36.796 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/usblcd.ko 2023-11-15T02:42:36.796 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-15T02:42:36.796 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 70873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/usbtest.ko 2023-11-15T02:42:36.797 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 41969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/uss720.ko 2023-11-15T02:42:36.797 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/yurex.ko 2023-11-15T02:42:36.797 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/mon/ 2023-11-15T02:42:36.797 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 72953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/mon/usbmon.ko 2023-11-15T02:42:36.797 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/musb/ 2023-11-15T02:42:36.797 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 301545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/musb/musb_hdrc.ko 2023-11-15T02:42:36.798 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/ 2023-11-15T02:42:36.798 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/phy-generic.ko 2023-11-15T02:42:36.798 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-11-15T02:42:36.798 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/phy-isp1301.ko 2023-11-15T02:42:36.799 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/phy-tahvo.ko 2023-11-15T02:42:36.799 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ 2023-11-15T02:42:36.799 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/aircable.ko 2023-11-15T02:42:36.799 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ark3116.ko 2023-11-15T02:42:36.799 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-15T02:42:36.800 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 38105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ch341.ko 2023-11-15T02:42:36.800 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 70257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/cp210x.ko 2023-11-15T02:42:36.800 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 31057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/cyberjack.ko 2023-11-15T02:42:36.800 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 58217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-15T02:42:36.800 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 49817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-15T02:42:36.801 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/empeg.ko 2023-11-15T02:42:36.801 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 44337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/f81232.ko 2023-11-15T02:42:36.801 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 175137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-15T02:42:36.801 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 53113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-15T02:42:36.801 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 137081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-15T02:42:36.801 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 124833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/io_ti.ko 2023-11-15T02:42:36.802 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 67769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ipaq.ko 2023-11-15T02:42:36.802 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ipw.ko 2023-11-15T02:42:36.802 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ir-usb.ko 2023-11-15T02:42:36.802 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 58233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-15T02:42:36.802 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 107521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/keyspan.ko 2023-11-15T02:42:36.803 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 35617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-15T02:42:36.803 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-15T02:42:36.803 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-15T02:42:36.803 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 32953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/mct_u232.ko 2023-11-15T02:42:36.803 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/metro-usb.ko 2023-11-15T02:42:36.804 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 66449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/mos7720.ko 2023-11-15T02:42:36.804 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 78217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/mos7840.ko 2023-11-15T02:42:36.804 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 56337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/mxuport.ko 2023-11-15T02:42:36.804 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/navman.ko 2023-11-15T02:42:36.804 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/omninet.ko 2023-11-15T02:42:36.805 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/opticon.ko 2023-11-15T02:42:36.805 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 190633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/option.ko 2023-11-15T02:42:36.805 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 39073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/oti6858.ko 2023-11-15T02:42:36.805 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 54353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/pl2303.ko 2023-11-15T02:42:36.805 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/qcaux.ko 2023-11-15T02:42:36.806 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 37825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/qcserial.ko 2023-11-15T02:42:36.806 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/quatech2.ko 2023-11-15T02:42:36.806 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/safe_serial.ko 2023-11-15T02:42:36.806 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 51625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/sierra.ko 2023-11-15T02:42:36.806 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-15T02:42:36.807 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ssu100.ko 2023-11-15T02:42:36.807 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/symbolserial.ko 2023-11-15T02:42:36.807 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 61489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-15T02:42:36.807 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-15T02:42:36.807 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/usb_debug.ko 2023-11-15T02:42:36.807 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-15T02:42:36.808 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 125609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/usbserial.ko 2023-11-15T02:42:36.808 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/visor.ko 2023-11-15T02:42:36.808 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 37409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/whiteheat.ko 2023-11-15T02:42:36.808 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/wishbone-serial.ko 2023-11-15T02:42:36.808 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-15T02:42:36.809 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ 2023-11-15T02:42:36.809 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 52913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/uas.ko 2023-11-15T02:42:36.809 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-15T02:42:36.809 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-15T02:42:36.809 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-15T02:42:36.810 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-15T02:42:36.810 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-15T02:42:36.810 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-15T02:42:36.810 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-15T02:42:36.810 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-karma.ko 2023-11-15T02:42:36.811 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-15T02:42:36.811 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-15T02:42:36.811 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-15T02:42:36.811 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-15T02:42:36.811 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 35337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-15T02:42:36.812 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 179969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/usb-storage.ko 2023-11-15T02:42:36.812 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/usbip/ 2023-11-15T02:42:36.812 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 67481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-15T02:42:36.812 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 72177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-15T02:42:36.812 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 107001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-15T02:42:36.812 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/ 2023-11-15T02:42:36.813 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/pci/ 2023-11-15T02:42:36.813 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 154297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-15T02:42:36.813 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-15T02:42:36.813 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 108737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/vfio.ko 2023-11-15T02:42:36.813 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 77609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-15T02:42:36.814 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/ 2023-11-15T02:42:36.814 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 113873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/vhost.ko 2023-11-15T02:42:36.814 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-15T02:42:36.814 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 58089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/vhost_net.ko 2023-11-15T02:42:36.814 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 89249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/vhost_scsi.ko 2023-11-15T02:42:36.815 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/ 2023-11-15T02:42:36.815 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/ 2023-11-15T02:42:36.815 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-11-15T02:42:36.815 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/aat2870_bl.ko 2023-11-15T02:42:36.815 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/adp5520_bl.ko 2023-11-15T02:42:36.816 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 32609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/adp8860_bl.ko 2023-11-15T02:42:36.816 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 39993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/adp8870_bl.ko 2023-11-15T02:42:36.816 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/apple_bl.ko 2023-11-15T02:42:36.816 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/as3711_bl.ko 2023-11-15T02:42:36.816 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/bd6107.ko 2023-11-15T02:42:36.816 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/da903x_bl.ko 2023-11-15T02:42:36.817 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/da9052_bl.ko 2023-11-15T02:42:36.817 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/gpio_backlight.ko 2023-11-15T02:42:36.817 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/kb3886_bl.ko 2023-11-15T02:42:36.817 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/lcd.ko 2023-11-15T02:42:36.817 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/lm3533_bl.ko 2023-11-15T02:42:36.818 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/lm3639_bl.ko 2023-11-15T02:42:36.818 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/lv5207lp.ko 2023-11-15T02:42:36.818 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/max8925_bl.ko 2023-11-15T02:42:36.818 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/pandora_bl.ko 2023-11-15T02:42:36.818 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-11-15T02:42:36.819 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-15T02:42:36.819 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/wm831x_bl.ko 2023-11-15T02:42:36.819 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/ 2023-11-15T02:42:36.819 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/ 2023-11-15T02:42:36.819 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/ds2482.ko 2023-11-15T02:42:36.820 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 38985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/ds2490.ko 2023-11-15T02:42:36.820 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/matrox_w1.ko 2023-11-15T02:42:36.820 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/w1-gpio.ko 2023-11-15T02:42:36.820 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/ 2023-11-15T02:42:36.820 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-15T02:42:36.820 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-15T02:42:36.821 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-15T02:42:36.821 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-15T02:42:36.821 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-15T02:42:36.821 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-15T02:42:36.821 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-15T02:42:36.822 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-15T02:42:36.822 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-15T02:42:36.822 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-15T02:42:36.822 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 56425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-15T02:42:36.822 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 102673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/wire.ko 2023-11-15T02:42:36.823 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/ 2023-11-15T02:42:36.823 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/acquirewdt.ko 2023-11-15T02:42:36.823 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/advantechwdt.ko 2023-11-15T02:42:36.823 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-15T02:42:36.823 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-15T02:42:36.824 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/cpu5wdt.ko 2023-11-15T02:42:36.824 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/da9052_wdt.ko 2023-11-15T02:42:36.824 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/da9055_wdt.ko 2023-11-15T02:42:36.824 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/da9063_wdt.ko 2023-11-15T02:42:36.824 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/dw_wdt.ko 2023-11-15T02:42:36.825 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/eurotechwdt.ko 2023-11-15T02:42:36.825 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-15T02:42:36.825 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/hpwdt.ko 2023-11-15T02:42:36.825 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/i6300esb.ko 2023-11-15T02:42:36.825 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-15T02:42:36.826 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-15T02:42:36.826 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/ib700wdt.ko 2023-11-15T02:42:36.826 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/ibmasr.ko 2023-11-15T02:42:36.826 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-15T02:42:36.826 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-15T02:42:36.826 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/it87_wdt.ko 2023-11-15T02:42:36.827 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/kempld_wdt.ko 2023-11-15T02:42:36.827 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/machzwd.ko 2023-11-15T02:42:36.827 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/mena21_wdt.ko 2023-11-15T02:42:36.827 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-11-15T02:42:36.827 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/nv_tco.ko 2023-11-15T02:42:36.828 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-11-15T02:42:36.828 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/pc87413_wdt.ko 2023-11-15T02:42:36.828 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 38657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-15T02:42:36.828 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 31825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-15T02:42:36.828 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/retu_wdt.ko 2023-11-15T02:42:36.829 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-11-15T02:42:36.829 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-11-15T02:42:36.829 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-15T02:42:36.829 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sc1200wdt.ko 2023-11-15T02:42:36.829 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-15T02:42:36.829 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-11-15T02:42:36.830 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/softdog.ko 2023-11-15T02:42:36.830 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-15T02:42:36.830 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/twl4030_wdt.ko 2023-11-15T02:42:36.830 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/via_wdt.ko 2023-11-15T02:42:36.830 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-15T02:42:36.831 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-15T02:42:36.831 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-15T02:42:36.831 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/wafer5823wdt.ko 2023-11-15T02:42:36.831 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/wdt_pci.ko 2023-11-15T02:42:36.831 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/wm831x_wdt.ko 2023-11-15T02:42:36.832 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/wm8350_wdt.ko 2023-11-15T02:42:36.832 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/xen_wdt.ko 2023-11-15T02:42:36.832 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/ 2023-11-15T02:42:36.832 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-evtchn.ko 2023-11-15T02:42:36.832 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-gntalloc.ko 2023-11-15T02:42:36.833 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 47865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-gntdev.ko 2023-11-15T02:42:36.833 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-pciback/ 2023-11-15T02:42:36.833 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 176441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-15T02:42:36.833 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 40121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-privcmd.ko 2023-11-15T02:42:36.833 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 71249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-scsiback.ko 2023-11-15T02:42:36.833 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xenfs/ 2023-11-15T02:42:36.834 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-15T02:42:36.834 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ 2023-11-15T02:42:36.834 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/9p/ 2023-11-15T02:42:36.834 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 166425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/9p/9p.ko 2023-11-15T02:42:36.834 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/adfs/ 2023-11-15T02:42:36.835 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 61073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/adfs/adfs.ko 2023-11-15T02:42:36.835 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/affs/ 2023-11-15T02:42:36.835 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 163593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/affs/affs.ko 2023-11-15T02:42:36.835 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/afs/ 2023-11-15T02:42:36.835 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 932817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/afs/kafs.ko 2023-11-15T02:42:36.836 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/autofs/ 2023-11-15T02:42:36.836 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 101057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/autofs/autofs4.ko 2023-11-15T02:42:36.836 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/befs/ 2023-11-15T02:42:36.836 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 64545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/befs/befs.ko 2023-11-15T02:42:36.836 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/bfs/ 2023-11-15T02:42:36.837 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 41257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/bfs/bfs.ko 2023-11-15T02:42:36.837 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/binfmt_misc.ko 2023-11-15T02:42:36.837 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/btrfs/ 2023-11-15T02:42:36.837 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 3402585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/btrfs/btrfs.ko 2023-11-15T02:42:36.837 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/cachefiles/ 2023-11-15T02:42:36.837 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 306073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/cachefiles/cachefiles.ko 2023-11-15T02:42:36.838 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ceph/ 2023-11-15T02:42:36.838 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 1323945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ceph/ceph.ko 2023-11-15T02:42:36.838 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/coda/ 2023-11-15T02:42:36.838 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 100801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/coda/coda.ko 2023-11-15T02:42:36.838 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/configfs/ 2023-11-15T02:42:36.839 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 108393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/configfs/configfs.ko 2023-11-15T02:42:36.839 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/cramfs/ 2023-11-15T02:42:36.839 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 29625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/cramfs/cramfs.ko 2023-11-15T02:42:36.839 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/dlm/ 2023-11-15T02:42:36.839 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 642577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/dlm/dlm.ko 2023-11-15T02:42:36.840 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/efs/ 2023-11-15T02:42:36.840 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/efs/efs.ko 2023-11-15T02:42:36.840 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/f2fs/ 2023-11-15T02:42:36.840 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 1505025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/f2fs/f2fs.ko 2023-11-15T02:42:36.840 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fat/ 2023-11-15T02:42:36.840 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fat/msdos.ko 2023-11-15T02:42:36.841 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/freevxfs/ 2023-11-15T02:42:36.841 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 42545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/freevxfs/freevxfs.ko 2023-11-15T02:42:36.841 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fscache/ 2023-11-15T02:42:36.841 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 226481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fscache/fscache.ko 2023-11-15T02:42:36.841 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fuse/ 2023-11-15T02:42:36.842 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fuse/cuse.ko 2023-11-15T02:42:36.842 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/gfs2/ 2023-11-15T02:42:36.842 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 1231529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/gfs2/gfs2.ko 2023-11-15T02:42:36.842 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hfs/ 2023-11-15T02:42:36.842 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 141209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hfs/hfs.ko 2023-11-15T02:42:36.843 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hfsplus/ 2023-11-15T02:42:36.843 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 222817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hfsplus/hfsplus.ko 2023-11-15T02:42:36.843 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hpfs/ 2023-11-15T02:42:36.843 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 192329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hpfs/hpfs.ko 2023-11-15T02:42:36.843 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/isofs/ 2023-11-15T02:42:36.844 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 74065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/isofs/isofs.ko 2023-11-15T02:42:36.844 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/jfs/ 2023-11-15T02:42:36.844 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 419497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/jfs/jfs.ko 2023-11-15T02:42:36.844 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/lockd/ 2023-11-15T02:42:36.844 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 280313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/lockd/lockd.ko 2023-11-15T02:42:36.844 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/minix/ 2023-11-15T02:42:36.845 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 89193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/minix/minix.ko 2023-11-15T02:42:36.845 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/netfs/ 2023-11-15T02:42:36.845 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 124649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/netfs/netfs.ko 2023-11-15T02:42:36.845 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/ 2023-11-15T02:42:36.845 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/blocklayout/ 2023-11-15T02:42:36.846 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 81449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-15T02:42:36.846 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/filelayout/ 2023-11-15T02:42:36.846 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 59449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-15T02:42:36.846 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/flexfilelayout/ 2023-11-15T02:42:36.846 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 117513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-15T02:42:36.847 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 1055649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/nfs.ko 2023-11-15T02:42:36.847 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 66441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/nfsv2.ko 2023-11-15T02:42:36.847 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 120177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/nfsv3.ko 2023-11-15T02:42:36.847 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 2041689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/nfsv4.ko 2023-11-15T02:42:36.847 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs_common/ 2023-11-15T02:42:36.848 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs_common/grace.ko 2023-11-15T02:42:36.848 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs_common/nfs_acl.ko 2023-11-15T02:42:36.848 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfsd/ 2023-11-15T02:42:36.848 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 1516809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfsd/nfsd.ko 2023-11-15T02:42:36.848 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nilfs2/ 2023-11-15T02:42:36.848 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 592833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nilfs2/nilfs2.ko 2023-11-15T02:42:36.849 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/ 2023-11-15T02:42:36.849 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-celtic.ko 2023-11-15T02:42:36.849 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-centeuro.ko 2023-11-15T02:42:36.849 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-croatian.ko 2023-11-15T02:42:36.849 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-cyrillic.ko 2023-11-15T02:42:36.850 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-gaelic.ko 2023-11-15T02:42:36.850 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-greek.ko 2023-11-15T02:42:36.850 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-iceland.ko 2023-11-15T02:42:36.850 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-inuit.ko 2023-11-15T02:42:36.850 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-roman.ko 2023-11-15T02:42:36.851 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-romanian.ko 2023-11-15T02:42:36.851 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-turkish.ko 2023-11-15T02:42:36.851 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_ascii.ko 2023-11-15T02:42:36.851 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp1250.ko 2023-11-15T02:42:36.851 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp1251.ko 2023-11-15T02:42:36.852 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp1255.ko 2023-11-15T02:42:36.852 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp737.ko 2023-11-15T02:42:36.852 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp775.ko 2023-11-15T02:42:36.852 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp850.ko 2023-11-15T02:42:36.852 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp852.ko 2023-11-15T02:42:36.853 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp855.ko 2023-11-15T02:42:36.853 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp857.ko 2023-11-15T02:42:36.853 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp860.ko 2023-11-15T02:42:36.853 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp861.ko 2023-11-15T02:42:36.853 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp862.ko 2023-11-15T02:42:36.853 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp863.ko 2023-11-15T02:42:36.854 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp864.ko 2023-11-15T02:42:36.854 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp865.ko 2023-11-15T02:42:36.854 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp866.ko 2023-11-15T02:42:36.854 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp869.ko 2023-11-15T02:42:36.854 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp874.ko 2023-11-15T02:42:36.855 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 94137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp932.ko 2023-11-15T02:42:36.855 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 142673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp936.ko 2023-11-15T02:42:36.855 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 165745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp949.ko 2023-11-15T02:42:36.855 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 118121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp950.ko 2023-11-15T02:42:36.855 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_euc-jp.ko 2023-11-15T02:42:36.856 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-1.ko 2023-11-15T02:42:36.856 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-13.ko 2023-11-15T02:42:36.856 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-14.ko 2023-11-15T02:42:36.856 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-15.ko 2023-11-15T02:42:36.856 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-2.ko 2023-11-15T02:42:36.857 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-3.ko 2023-11-15T02:42:36.857 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-4.ko 2023-11-15T02:42:36.857 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-5.ko 2023-11-15T02:42:36.857 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-6.ko 2023-11-15T02:42:36.857 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-7.ko 2023-11-15T02:42:36.858 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-9.ko 2023-11-15T02:42:36.858 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_koi8-r.ko 2023-11-15T02:42:36.858 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_koi8-ru.ko 2023-11-15T02:42:36.858 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_koi8-u.ko 2023-11-15T02:42:36.858 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-15T02:42:36.859 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_utf8.ko 2023-11-15T02:42:36.859 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ntfs/ 2023-11-15T02:42:36.859 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 219417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ntfs/ntfs.ko 2023-11-15T02:42:36.859 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ 2023-11-15T02:42:36.859 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/cluster/ 2023-11-15T02:42:36.859 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 244409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-15T02:42:36.860 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/dlm/ 2023-11-15T02:42:36.860 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 440737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-15T02:42:36.860 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/dlmfs/ 2023-11-15T02:42:36.860 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 54049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-15T02:42:36.860 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 2592433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ocfs2.ko 2023-11-15T02:42:36.861 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-15T02:42:36.861 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 31729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-15T02:42:36.861 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 35897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-15T02:42:36.861 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/omfs/ 2023-11-15T02:42:36.861 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 53433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/omfs/omfs.ko 2023-11-15T02:42:36.862 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/overlayfs/ 2023-11-15T02:42:36.862 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 352153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/overlayfs/overlay.ko 2023-11-15T02:42:36.862 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/pstore/ 2023-11-15T02:42:36.862 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 54809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/pstore/ramoops.ko 2023-11-15T02:42:36.862 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/qnx4/ 2023-11-15T02:42:36.863 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/qnx4/qnx4.ko 2023-11-15T02:42:36.863 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/qnx6/ 2023-11-15T02:42:36.863 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 46689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/qnx6/qnx6.ko 2023-11-15T02:42:36.863 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/quota/ 2023-11-15T02:42:36.863 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/quota/quota_tree.ko 2023-11-15T02:42:36.863 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/quota/quota_v1.ko 2023-11-15T02:42:36.864 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/quota/quota_v2.ko 2023-11-15T02:42:36.864 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/reiserfs/ 2023-11-15T02:42:36.864 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 532673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/reiserfs/reiserfs.ko 2023-11-15T02:42:36.864 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/romfs/ 2023-11-15T02:42:36.864 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/romfs/romfs.ko 2023-11-15T02:42:36.865 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/ 2023-11-15T02:42:36.865 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/client/ 2023-11-15T02:42:36.865 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 2668121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/client/cifs.ko 2023-11-15T02:42:36.865 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/common/ 2023-11-15T02:42:36.865 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/common/cifs_arc4.ko 2023-11-15T02:42:36.866 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/common/cifs_md4.ko 2023-11-15T02:42:36.866 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/squashfs/ 2023-11-15T02:42:36.866 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 134641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/squashfs/squashfs.ko 2023-11-15T02:42:36.866 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/sysv/ 2023-11-15T02:42:36.866 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 90721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/sysv/sysv.ko 2023-11-15T02:42:36.867 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/udf/ 2023-11-15T02:42:36.867 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 247737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/udf/udf.ko 2023-11-15T02:42:36.867 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ufs/ 2023-11-15T02:42:36.867 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 162497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ufs/ufs.ko 2023-11-15T02:42:36.867 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/xfs/ 2023-11-15T02:42:36.868 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 4010729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/xfs/xfs.ko 2023-11-15T02:42:36.868 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/ 2023-11-15T02:42:36.868 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/locking/ 2023-11-15T02:42:36.868 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 75489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/locking/locktorture.ko 2023-11-15T02:42:36.868 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/time/ 2023-11-15T02:42:36.869 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/time/test_udelay.ko 2023-11-15T02:42:36.869 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 60553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/torture.ko 2023-11-15T02:42:36.869 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/ 2023-11-15T02:42:36.869 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 6833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crc-itu-t.ko 2023-11-15T02:42:36.869 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 6529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crc7.ko 2023-11-15T02:42:36.869 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crc8.ko 2023-11-15T02:42:36.870 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crypto/ 2023-11-15T02:42:36.870 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crypto/libarc4.ko 2023-11-15T02:42:36.870 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 37081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crypto/libdes.ko 2023-11-15T02:42:36.870 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/interval_tree_test.ko 2023-11-15T02:42:36.870 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/libcrc32c.ko 2023-11-15T02:42:36.871 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/lru_cache.ko 2023-11-15T02:42:36.871 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/lz4/ 2023-11-15T02:42:36.871 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/lz4/lz4_compress.ko 2023-11-15T02:42:36.871 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/lz4/lz4hc_compress.ko 2023-11-15T02:42:36.871 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/math/ 2023-11-15T02:42:36.872 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 6617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/math/cordic.ko 2023-11-15T02:42:36.872 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/memory-notifier-error-inject.ko 2023-11-15T02:42:36.872 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/notifier-error-inject.ko 2023-11-15T02:42:36.872 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/percpu_test.ko 2023-11-15T02:42:36.872 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/pm-notifier-error-inject.ko 2023-11-15T02:42:36.872 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/raid6/ 2023-11-15T02:42:36.873 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 143433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/raid6/raid6_pq.ko 2023-11-15T02:42:36.873 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/rbtree_test.ko 2023-11-15T02:42:36.873 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/reed_solomon/ 2023-11-15T02:42:36.873 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-15T02:42:36.873 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 38841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test-kstrtox.ko 2023-11-15T02:42:36.874 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test-string_helpers.ko 2023-11-15T02:42:36.874 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 4760425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test_bpf.ko 2023-11-15T02:42:36.874 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 49089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test_firmware.ko 2023-11-15T02:42:36.874 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 6369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test_module.ko 2023-11-15T02:42:36.874 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test_user_copy.ko 2023-11-15T02:42:36.875 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/ts_bm.ko 2023-11-15T02:42:36.875 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/ts_fsm.ko 2023-11-15T02:42:36.875 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/ts_kmp.ko 2023-11-15T02:42:36.875 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/xz/ 2023-11-15T02:42:36.875 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/xz/xz_dec_test.ko 2023-11-15T02:42:36.875 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/mm/ 2023-11-15T02:42:36.876 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/mm/hwpoison-inject.ko 2023-11-15T02:42:36.876 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ 2023-11-15T02:42:36.876 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/ 2023-11-15T02:42:36.876 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 55737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/6lowpan.ko 2023-11-15T02:42:36.876 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_dest.ko 2023-11-15T02:42:36.877 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_fragment.ko 2023-11-15T02:42:36.877 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_hop.ko 2023-11-15T02:42:36.877 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 6985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-15T02:42:36.877 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_mobility.ko 2023-11-15T02:42:36.877 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_routing.ko 2023-11-15T02:42:36.878 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_udp.ko 2023-11-15T02:42:36.878 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/ 2023-11-15T02:42:36.878 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/garp.ko 2023-11-15T02:42:36.878 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/mrp.ko 2023-11-15T02:42:36.878 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/p8022.ko 2023-11-15T02:42:36.879 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/psnap.ko 2023-11-15T02:42:36.879 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/stp.ko 2023-11-15T02:42:36.879 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/8021q/ 2023-11-15T02:42:36.879 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 81737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/8021q/8021q.ko 2023-11-15T02:42:36.879 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/ 2023-11-15T02:42:36.879 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 198337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/9pnet.ko 2023-11-15T02:42:36.880 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 35977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/9pnet_fd.ko 2023-11-15T02:42:36.880 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/9pnet_rdma.ko 2023-11-15T02:42:36.880 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 32337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/9pnet_virtio.ko 2023-11-15T02:42:36.880 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/batman-adv/ 2023-11-15T02:42:36.880 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 649297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/batman-adv/batman-adv.ko 2023-11-15T02:42:36.881 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/ 2023-11-15T02:42:36.881 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 63809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/br_netfilter.ko 2023-11-15T02:42:36.881 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 777689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/bridge.ko 2023-11-15T02:42:36.881 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ 2023-11-15T02:42:36.881 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-15T02:42:36.882 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-15T02:42:36.882 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-15T02:42:36.882 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-15T02:42:36.882 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-15T02:42:36.882 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-15T02:42:36.883 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-15T02:42:36.883 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-15T02:42:36.883 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-15T02:42:36.883 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-15T02:42:36.883 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-15T02:42:36.883 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-15T02:42:36.884 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-15T02:42:36.884 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-15T02:42:36.884 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-15T02:42:36.884 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-15T02:42:36.884 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-15T02:42:36.885 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-15T02:42:36.885 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-15T02:42:36.885 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-15T02:42:36.885 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 55929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebtables.ko 2023-11-15T02:42:36.885 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-15T02:42:36.885 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-15T02:42:36.886 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ceph/ 2023-11-15T02:42:36.886 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 1046857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ceph/libceph.ko 2023-11-15T02:42:36.886 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/core/ 2023-11-15T02:42:36.886 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 125697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/core/pktgen.ko 2023-11-15T02:42:36.886 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/ 2023-11-15T02:42:36.887 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 194209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/dccp.ko 2023-11-15T02:42:36.887 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/dccp_diag.ko 2023-11-15T02:42:36.887 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 57881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/dccp_ipv4.ko 2023-11-15T02:42:36.887 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 60337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/dccp_ipv6.ko 2023-11-15T02:42:36.887 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/ 2023-11-15T02:42:36.888 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 390473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/dsa_core.ko 2023-11-15T02:42:36.888 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/tag_brcm.ko 2023-11-15T02:42:36.888 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/tag_dsa.ko 2023-11-15T02:42:36.888 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/tag_none.ko 2023-11-15T02:42:36.888 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/tag_trailer.ko 2023-11-15T02:42:36.889 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/hsr/ 2023-11-15T02:42:36.889 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 99729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/hsr/hsr.ko 2023-11-15T02:42:36.889 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/ 2023-11-15T02:42:36.889 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/6lowpan/ 2023-11-15T02:42:36.889 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 45233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-15T02:42:36.889 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 254625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/ieee802154.ko 2023-11-15T02:42:36.890 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 47289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-15T02:42:36.890 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ 2023-11-15T02:42:36.890 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ah4.ko 2023-11-15T02:42:36.890 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 38905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/esp4.ko 2023-11-15T02:42:36.891 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 48401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/fou.ko 2023-11-15T02:42:36.891 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/gre.ko 2023-11-15T02:42:36.891 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 48657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/inet_diag.ko 2023-11-15T02:42:36.891 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 60633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ip_gre.ko 2023-11-15T02:42:36.891 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 61225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ip_tunnel.ko 2023-11-15T02:42:36.891 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ip_vti.ko 2023-11-15T02:42:36.892 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ipcomp.ko 2023-11-15T02:42:36.892 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ipip.ko 2023-11-15T02:42:36.892 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ 2023-11-15T02:42:36.892 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 37641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-15T02:42:36.892 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-15T02:42:36.893 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-15T02:42:36.893 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 41673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ip_tables.ko 2023-11-15T02:42:36.893 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-15T02:42:36.893 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-11-15T02:42:36.893 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-15T02:42:36.894 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-15T02:42:36.894 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-15T02:42:36.894 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-11-15T02:42:36.894 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-15T02:42:36.894 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-15T02:42:36.895 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-15T02:42:36.895 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-15T02:42:36.895 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-15T02:42:36.895 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-15T02:42:36.895 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-15T02:42:36.896 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-15T02:42:36.896 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-15T02:42:36.896 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-11-15T02:42:36.896 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-15T02:42:36.896 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-15T02:42:36.896 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-15T02:42:36.897 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-15T02:42:36.897 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-15T02:42:36.897 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_bic.ko 2023-11-15T02:42:36.897 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_dctcp.ko 2023-11-15T02:42:36.897 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_diag.ko 2023-11-15T02:42:36.898 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_highspeed.ko 2023-11-15T02:42:36.898 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_htcp.ko 2023-11-15T02:42:36.898 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_hybla.ko 2023-11-15T02:42:36.898 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_illinois.ko 2023-11-15T02:42:36.898 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_lp.ko 2023-11-15T02:42:36.899 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_scalable.ko 2023-11-15T02:42:36.899 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_vegas.ko 2023-11-15T02:42:36.899 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_veno.ko 2023-11-15T02:42:36.899 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_westwood.ko 2023-11-15T02:42:36.899 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_yeah.ko 2023-11-15T02:42:36.900 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tunnel4.ko 2023-11-15T02:42:36.900 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/udp_diag.ko 2023-11-15T02:42:36.900 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 41665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/udp_tunnel.ko 2023-11-15T02:42:36.900 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-15T02:42:36.900 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ 2023-11-15T02:42:36.901 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ah6.ko 2023-11-15T02:42:36.901 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 39537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/esp6.ko 2023-11-15T02:42:36.901 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/fou6.ko 2023-11-15T02:42:36.901 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 80897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ip6_gre.ko 2023-11-15T02:42:36.901 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 93577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ip6_tunnel.ko 2023-11-15T02:42:36.901 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-15T02:42:36.902 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 53273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ip6_vti.ko 2023-11-15T02:42:36.902 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ipcomp6.ko 2023-11-15T02:42:36.902 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/mip6.ko 2023-11-15T02:42:36.902 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ 2023-11-15T02:42:36.902 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 42169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-15T02:42:36.903 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-11-15T02:42:36.903 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-15T02:42:36.903 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-15T02:42:36.903 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-15T02:42:36.903 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-15T02:42:36.904 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-15T02:42:36.904 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-15T02:42:36.904 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-15T02:42:36.904 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-15T02:42:36.904 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-15T02:42:36.904 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-15T02:42:36.905 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-15T02:42:36.905 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-15T02:42:36.905 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-15T02:42:36.905 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-15T02:42:36.905 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-15T02:42:36.906 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 39113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-15T02:42:36.906 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-15T02:42:36.906 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-15T02:42:36.906 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-15T02:42:36.906 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-15T02:42:36.907 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-15T02:42:36.907 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-15T02:42:36.907 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-15T02:42:36.907 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 74073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/sit.ko 2023-11-15T02:42:36.907 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/tunnel6.ko 2023-11-15T02:42:36.907 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-15T02:42:36.908 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/key/ 2023-11-15T02:42:36.908 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 82833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/key/af_key.ko 2023-11-15T02:42:36.908 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/ 2023-11-15T02:42:36.908 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 125281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_core.ko 2023-11-15T02:42:36.908 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-15T02:42:36.909 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 25953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_eth.ko 2023-11-15T02:42:36.909 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 35569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_ip.ko 2023-11-15T02:42:36.909 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_ip6.ko 2023-11-15T02:42:36.909 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_netlink.ko 2023-11-15T02:42:36.909 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 53241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_ppp.ko 2023-11-15T02:42:36.909 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/lapb/ 2023-11-15T02:42:36.910 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 47977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/lapb/lapb.ko 2023-11-15T02:42:36.910 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/llc/ 2023-11-15T02:42:36.910 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/llc/llc.ko 2023-11-15T02:42:36.910 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 267777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/llc/llc2.ko 2023-11-15T02:42:36.910 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/mac802154/ 2023-11-15T02:42:36.911 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 290417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/mac802154/mac802154.ko 2023-11-15T02:42:36.911 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/mpls/ 2023-11-15T02:42:36.911 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/mpls/mpls_gso.ko 2023-11-15T02:42:36.911 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ 2023-11-15T02:42:36.911 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ 2023-11-15T02:42:36.912 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 119585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set.ko 2023-11-15T02:42:36.912 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-15T02:42:36.912 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-15T02:42:36.912 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-15T02:42:36.912 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 100297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-15T02:42:36.912 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 101441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-15T02:42:36.913 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 102817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-15T02:42:36.913 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 103481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-15T02:42:36.913 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 112849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-15T02:42:36.913 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 60329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-15T02:42:36.913 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 109593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-15T02:42:36.914 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 113129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-15T02:42:36.914 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 112585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-15T02:42:36.914 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 111585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-15T02:42:36.914 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 114465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-15T02:42:36.914 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-15T02:42:36.915 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ 2023-11-15T02:42:36.915 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 411729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-15T02:42:36.915 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-15T02:42:36.915 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-15T02:42:36.915 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-15T02:42:36.916 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-15T02:42:36.916 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-15T02:42:36.916 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-15T02:42:36.916 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-15T02:42:36.916 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-15T02:42:36.916 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-15T02:42:36.917 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-15T02:42:36.917 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-15T02:42:36.917 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-15T02:42:36.917 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-15T02:42:36.917 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 49257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conncount.ko 2023-11-15T02:42:36.918 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 408257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack.ko 2023-11-15T02:42:36.918 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-15T02:42:36.918 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-15T02:42:36.918 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-15T02:42:36.918 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 118145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-15T02:42:36.919 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-15T02:42:36.919 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-15T02:42:36.919 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 107297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-15T02:42:36.919 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-15T02:42:36.919 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-15T02:42:36.920 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 66641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-15T02:42:36.920 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-15T02:42:36.920 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-15T02:42:36.920 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-15T02:42:36.920 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 89473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_flow_table.ko 2023-11-15T02:42:36.920 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-15T02:42:36.921 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 37129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_log_syslog.ko 2023-11-15T02:42:36.921 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 95417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat.ko 2023-11-15T02:42:36.921 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-15T02:42:36.921 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-15T02:42:36.921 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_irc.ko 2023-11-15T02:42:36.922 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_sip.ko 2023-11-15T02:42:36.922 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-15T02:42:36.922 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 44065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-15T02:42:36.922 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 615313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_tables.ko 2023-11-15T02:42:36.922 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 42561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink.ko 2023-11-15T02:42:36.923 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-15T02:42:36.923 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-11-15T02:42:36.923 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 39785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_log.ko 2023-11-15T02:42:36.923 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-15T02:42:36.923 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 49377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-15T02:42:36.924 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_chain_nat.ko 2023-11-15T02:42:36.924 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_compat.ko 2023-11-15T02:42:36.924 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 40721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_ct.ko 2023-11-15T02:42:36.924 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-15T02:42:36.924 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_fib.ko 2023-11-15T02:42:36.925 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-15T02:42:36.925 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_hash.ko 2023-11-15T02:42:36.925 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_limit.ko 2023-11-15T02:42:36.925 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_log.ko 2023-11-15T02:42:36.925 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_masq.ko 2023-11-15T02:42:36.926 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_nat.ko 2023-11-15T02:42:36.926 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_numgen.ko 2023-11-15T02:42:36.926 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_osf.ko 2023-11-15T02:42:36.926 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_queue.ko 2023-11-15T02:42:36.926 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_redir.ko 2023-11-15T02:42:36.927 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_reject.ko 2023-11-15T02:42:36.927 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_reject_inet.ko 2023-11-15T02:42:36.927 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_socket.ko 2023-11-15T02:42:36.927 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_synproxy.ko 2023-11-15T02:42:36.927 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_tproxy.ko 2023-11-15T02:42:36.928 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 89313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/x_tables.ko 2023-11-15T02:42:36.928 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_AUDIT.ko 2023-11-15T02:42:36.928 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-15T02:42:36.928 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-15T02:42:36.928 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-15T02:42:36.929 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 18169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_CT.ko 2023-11-15T02:42:36.929 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_DSCP.ko 2023-11-15T02:42:36.929 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_HL.ko 2023-11-15T02:42:36.929 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_HMARK.ko 2023-11-15T02:42:36.929 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 38873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-15T02:42:36.930 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_LED.ko 2023-11-15T02:42:36.930 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_LOG.ko 2023-11-15T02:42:36.930 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-15T02:42:36.930 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_NETMAP.ko 2023-11-15T02:42:36.930 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_NFLOG.ko 2023-11-15T02:42:36.931 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-15T02:42:36.931 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_RATEEST.ko 2023-11-15T02:42:36.931 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-15T02:42:36.931 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 14337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_SECMARK.ko 2023-11-15T02:42:36.931 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-15T02:42:36.931 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-15T02:42:36.932 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TEE.ko 2023-11-15T02:42:36.932 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TPROXY.ko 2023-11-15T02:42:36.932 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TRACE.ko 2023-11-15T02:42:36.932 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_addrtype.ko 2023-11-15T02:42:36.932 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_bpf.ko 2023-11-15T02:42:36.933 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_cgroup.ko 2023-11-15T02:42:36.933 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_cluster.ko 2023-11-15T02:42:36.933 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_comment.ko 2023-11-15T02:42:36.933 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_connbytes.ko 2023-11-15T02:42:36.933 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_connlabel.ko 2023-11-15T02:42:36.934 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_connlimit.ko 2023-11-15T02:42:36.934 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_connmark.ko 2023-11-15T02:42:36.934 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_conntrack.ko 2023-11-15T02:42:36.934 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_cpu.ko 2023-11-15T02:42:36.934 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_dccp.ko 2023-11-15T02:42:36.935 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_devgroup.ko 2023-11-15T02:42:36.935 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_dscp.ko 2023-11-15T02:42:36.935 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_ecn.ko 2023-11-15T02:42:36.935 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_esp.ko 2023-11-15T02:42:36.935 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 43289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_hashlimit.ko 2023-11-15T02:42:36.936 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_helper.ko 2023-11-15T02:42:36.936 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_hl.ko 2023-11-15T02:42:36.936 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_ipcomp.ko 2023-11-15T02:42:36.936 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_iprange.ko 2023-11-15T02:42:36.936 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_ipvs.ko 2023-11-15T02:42:36.937 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_l2tp.ko 2023-11-15T02:42:36.937 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_length.ko 2023-11-15T02:42:36.937 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_limit.ko 2023-11-15T02:42:36.937 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_mac.ko 2023-11-15T02:42:36.937 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_mark.ko 2023-11-15T02:42:36.937 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_multiport.ko 2023-11-15T02:42:36.938 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_nat.ko 2023-11-15T02:42:36.938 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_nfacct.ko 2023-11-15T02:42:36.938 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_osf.ko 2023-11-15T02:42:36.938 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_owner.ko 2023-11-15T02:42:36.939 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_physdev.ko 2023-11-15T02:42:36.939 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_pkttype.ko 2023-11-15T02:42:36.939 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_policy.ko 2023-11-15T02:42:36.939 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_quota.ko 2023-11-15T02:42:36.939 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_rateest.ko 2023-11-15T02:42:36.940 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_realm.ko 2023-11-15T02:42:36.940 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_recent.ko 2023-11-15T02:42:36.940 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_sctp.ko 2023-11-15T02:42:36.940 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_set.ko 2023-11-15T02:42:36.940 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 16089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_socket.ko 2023-11-15T02:42:36.940 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_state.ko 2023-11-15T02:42:36.941 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_statistic.ko 2023-11-15T02:42:36.941 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_string.ko 2023-11-15T02:42:36.941 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 8553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_tcpmss.ko 2023-11-15T02:42:36.941 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_tcpudp.ko 2023-11-15T02:42:36.941 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_time.ko 2023-11-15T02:42:36.942 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_u32.ko 2023-11-15T02:42:36.942 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netlink/ 2023-11-15T02:42:36.942 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netlink/netlink_diag.ko 2023-11-15T02:42:36.942 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/nsh/ 2023-11-15T02:42:36.942 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 12425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/nsh/nsh.ko 2023-11-15T02:42:36.943 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/ 2023-11-15T02:42:36.943 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 380033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/openvswitch.ko 2023-11-15T02:42:36.943 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/vport-geneve.ko 2023-11-15T02:42:36.943 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/vport-gre.ko 2023-11-15T02:42:36.943 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 11793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/vport-vxlan.ko 2023-11-15T02:42:36.944 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/packet/ 2023-11-15T02:42:36.944 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/packet/af_packet_diag.ko 2023-11-15T02:42:36.944 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/phonet/ 2023-11-15T02:42:36.944 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 110201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/phonet/phonet.ko 2023-11-15T02:42:36.945 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 54697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/phonet/pn_pep.ko 2023-11-15T02:42:36.945 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rds/ 2023-11-15T02:42:36.945 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 251481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rds/rds.ko 2023-11-15T02:42:36.945 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 164937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rds/rds_rdma.ko 2023-11-15T02:42:36.945 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 67377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rds/rds_tcp.ko 2023-11-15T02:42:36.945 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rxrpc/ 2023-11-15T02:42:36.946 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 819657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rxrpc/rxrpc.ko 2023-11-15T02:42:36.946 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/ 2023-11-15T02:42:36.946 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 32241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_csum.ko 2023-11-15T02:42:36.946 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_gact.ko 2023-11-15T02:42:36.947 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_ipt.ko 2023-11-15T02:42:36.947 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_mirred.ko 2023-11-15T02:42:36.947 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_nat.ko 2023-11-15T02:42:36.947 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 43449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_pedit.ko 2023-11-15T02:42:36.947 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 31961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_police.ko 2023-11-15T02:42:36.947 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_simple.ko 2023-11-15T02:42:36.948 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 35297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_skbedit.ko 2023-11-15T02:42:36.948 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_vlan.ko 2023-11-15T02:42:36.948 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_basic.ko 2023-11-15T02:42:36.948 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_bpf.ko 2023-11-15T02:42:36.948 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_cgroup.ko 2023-11-15T02:42:36.949 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 31401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_flow.ko 2023-11-15T02:42:36.949 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 31769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_fw.ko 2023-11-15T02:42:36.949 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 38489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_route.ko 2023-11-15T02:42:36.949 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 61409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_u32.ko 2023-11-15T02:42:36.949 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_cmp.ko 2023-11-15T02:42:36.950 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 13409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_ipset.ko 2023-11-15T02:42:36.950 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_meta.ko 2023-11-15T02:42:36.950 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_nbyte.ko 2023-11-15T02:42:36.950 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_text.ko 2023-11-15T02:42:36.950 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_u32.ko 2023-11-15T02:42:36.951 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 21649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_choke.ko 2023-11-15T02:42:36.951 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_codel.ko 2023-11-15T02:42:36.951 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 34633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_drr.ko 2023-11-15T02:42:36.951 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 33609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_fq.ko 2023-11-15T02:42:36.951 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 31617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_fq_codel.ko 2023-11-15T02:42:36.952 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 35433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_gred.ko 2023-11-15T02:42:36.952 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 49601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_hfsc.ko 2023-11-15T02:42:36.952 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_hhf.ko 2023-11-15T02:42:36.952 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 66393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_htb.ko 2023-11-15T02:42:36.952 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_ingress.ko 2023-11-15T02:42:36.952 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 30273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_mqprio.ko 2023-11-15T02:42:36.953 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_mqprio_lib.ko 2023-11-15T02:42:36.953 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_multiq.ko 2023-11-15T02:42:36.953 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 36137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_netem.ko 2023-11-15T02:42:36.953 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 27497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_pie.ko 2023-11-15T02:42:36.953 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_plug.ko 2023-11-15T02:42:36.954 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 24729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_prio.ko 2023-11-15T02:42:36.954 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 48793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_qfq.ko 2023-11-15T02:42:36.954 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 32905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_red.ko 2023-11-15T02:42:36.954 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 26881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_sfb.ko 2023-11-15T02:42:36.954 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 31353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_sfq.ko 2023-11-15T02:42:36.955 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_tbf.ko 2023-11-15T02:42:36.955 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 28105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_teql.ko 2023-11-15T02:42:36.955 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sctp/ 2023-11-15T02:42:36.955 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 929745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sctp/sctp.ko 2023-11-15T02:42:36.955 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 19033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sctp/sctp_diag.ko 2023-11-15T02:42:36.956 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/ 2023-11-15T02:42:36.956 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/auth_gss/ 2023-11-15T02:42:36.956 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 336529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-15T02:42:36.956 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 54737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-15T02:42:36.956 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 1520161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/sunrpc.ko 2023-11-15T02:42:36.957 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/xprtrdma/ 2023-11-15T02:42:36.957 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 707721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-15T02:42:36.957 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/tipc/ 2023-11-15T02:42:36.957 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/tipc/diag.ko 2023-11-15T02:42:36.957 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 862641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/tipc/tipc.ko 2023-11-15T02:42:36.957 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/unix/ 2023-11-15T02:42:36.958 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/unix/unix_diag.ko 2023-11-15T02:42:36.958 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/ 2023-11-15T02:42:36.958 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 85337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-15T02:42:36.958 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 68049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-15T02:42:36.958 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 96193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vsock.ko 2023-11-15T02:42:36.959 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 10705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-15T02:42:36.959 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 17521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-15T02:42:36.959 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/x25/ 2023-11-15T02:42:36.960 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 125177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/x25/x25.ko 2023-11-15T02:42:36.960 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/xfrm/ 2023-11-15T02:42:36.960 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 22657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/xfrm/xfrm_algo.ko 2023-11-15T02:42:36.960 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 20649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-15T02:42:36.960 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 102257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/xfrm/xfrm_user.ko 2023-11-15T02:42:36.961 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/virt/ 2023-11-15T02:42:36.961 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/virt/lib/ 2023-11-15T02:42:36.961 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 15537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/virt/lib/irqbypass.ko 2023-11-15T02:42:36.961 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 639863 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.alias 2023-11-15T02:42:36.962 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 623919 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.alias.bin 2023-11-15T02:42:36.962 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 7663 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.builtin 2023-11-15T02:42:36.962 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 23428 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.builtin.alias.bin 2023-11-15T02:42:36.962 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 9821 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.builtin.bin 2023-11-15T02:42:36.963 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 70004 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.builtin.modinfo 2023-11-15T02:42:36.963 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 146313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.dep 2023-11-15T02:42:36.963 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 232548 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.dep.bin 2023-11-15T02:42:36.963 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 210 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.devname 2023-11-15T02:42:36.963 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 72606 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.order 2023-11-15T02:42:36.964 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 879 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.softdep 2023-11-15T02:42:36.964 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 254438 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.symbols 2023-11-15T02:42:36.964 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 309363 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.symbols.bin 2023-11-15T02:42:36.964 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./usr/ 2023-11-15T02:42:36.964 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./usr/share/ 2023-11-15T02:42:36.965 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./usr/share/doc/ 2023-11-15T02:42:36.965 INFO:teuthology.orchestra.run.smithi190.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./usr/share/doc/linux-image-6.6.0-rc5-g3b796cfe2969/ 2023-11-15T02:42:36.965 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 173 2023-11-07 03:19 ./usr/share/doc/linux-image-6.6.0-rc5-g3b796cfe2969/changelog.Debian.gz 2023-11-15T02:42:36.965 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r-- root/root 691 2023-11-07 03:19 ./usr/share/doc/linux-image-6.6.0-rc5-g3b796cfe2969/copyright 2023-11-15T02:42:36.966 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:36.966 INFO:teuthology.task.kernel:searching for kernel 6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:36.967 DEBUG:teuthology.orchestra.run.smithi190:> egrep '(submenu|menuentry.*6.6.0-rc5-g3b796cfe2969).*{' /boot/grub/grub.cfg 2023-11-15T02:42:36.975 INFO:teuthology.orchestra.run.smithi190.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-11-15T02:42:36.976 INFO:teuthology.orchestra.run.smithi190.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g3b796cfe2969' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g3b796cfe2969-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-11-15T02:42:36.976 INFO:teuthology.orchestra.run.smithi190.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g3b796cfe2969 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g3b796cfe2969-recovery-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-11-15T02:42:36.976 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-11-15T02:42:36.977 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:36.977 DEBUG:teuthology.orchestra.run.smithi190:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-11-15T02:42:37.019 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi190.front.sepia.ceph.com to complete... 2023-11-15T02:42:37.739 INFO:teuthology.orchestra.run.smithi190.stderr:Sourcing file `/etc/default/grub' 2023-11-15T02:42:37.751 INFO:teuthology.orchestra.run.smithi190.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-15T02:42:37.760 INFO:teuthology.orchestra.run.smithi190.stderr:Generating grub configuration file ... 2023-11-15T02:42:38.004 INFO:teuthology.orchestra.run.smithi190.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:38.016 INFO:teuthology.orchestra.run.smithi190.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g3b796cfe2969 2023-11-15T02:42:38.424 INFO:teuthology.orchestra.run.smithi190.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-11-15T02:42:38.436 INFO:teuthology.orchestra.run.smithi190.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-11-15T02:42:39.218 INFO:teuthology.orchestra.run.smithi190.stderr:done 2023-11-15T02:42:54.478 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-11-15T02:42:54.479 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-11-15T02:42:54.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T02:43:10.346 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-11-15T02:43:10.347 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-11-15T02:43:10.347 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T02:43:12.890 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.121 2023-11-15T02:43:21.899 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-11-15T02:43:21.900 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T02:43:24.958 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.121 2023-11-15T02:43:28.762 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-11-15T02:43:36.966 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-11-15T02:43:36.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T02:43:37.764 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-11-15T02:43:37.765 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T02:43:40.826 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-11-15T02:43:52.838 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-11-15T02:43:52.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T02:44:08.690 DEBUG:teuthology.orchestra.run.smithi121:> true 2023-11-15T02:44:09.413 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-11-15T02:44:09.414 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-rc5-g3b796cfe2969"... 2023-11-15T02:44:09.414 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2023-11-15T02:44:09.474 INFO:teuthology.orchestra.run.smithi121.stdout:6.6.0-rc5-g3b796cfe2969 2023-11-15T02:44:09.474 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g3b796cfe2969 vs 6.6.0-rc5-g3b796cfe2969 2023-11-15T02:44:09.475 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-15T02:44:09.475 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-11-15T02:44:10.476 DEBUG:teuthology.parallel:result is None 2023-11-15T02:44:24.576 DEBUG:teuthology.orchestra.run.smithi190:> true 2023-11-15T02:44:25.276 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-11-15T02:44:25.276 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.6.0-rc5-g3b796cfe2969"... 2023-11-15T02:44:25.277 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2023-11-15T02:44:25.340 INFO:teuthology.orchestra.run.smithi190.stdout:6.6.0-rc5-g3b796cfe2969 2023-11-15T02:44:25.340 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g3b796cfe2969 vs 6.6.0-rc5-g3b796cfe2969 2023-11-15T02:44:25.341 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-15T02:44:25.341 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-11-15T02:44:26.342 DEBUG:teuthology.parallel:result is None 2023-11-15T02:44:26.343 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-15T02:44:26.351 INFO:teuthology.task.internal:Creating test directory... 2023-11-15T02:44:26.351 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-15T02:44:26.358 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-15T02:44:26.377 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-15T02:44:26.391 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-15T02:44:26.426 INFO:teuthology.task.internal:Creating archive directory... 2023-11-15T02:44:26.426 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-15T02:44:26.434 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-15T02:44:26.470 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-15T02:44:26.526 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-15T02:44:26.526 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-15T02:44:26.533 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-15T02:44:26.645 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-15T02:44:26.662 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-15T02:44:26.672 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-15T02:44:26.688 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-15T02:44:26.692 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-15T02:44:26.700 INFO:teuthology.task.internal:Configuring sudo... 2023-11-15T02:44:26.700 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-15T02:44:26.707 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-15T02:44:26.757 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-15T02:44:26.766 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-15T02:44:26.767 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-15T02:44:26.790 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-15T02:44:26.822 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-15T02:44:26.871 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-15T02:44:26.932 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:44:26.933 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-15T02:44:27.039 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-15T02:44:27.058 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-15T02:44:27.123 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:44:27.124 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-15T02:44:27.207 DEBUG:teuthology.orchestra.run.smithi121:> sudo service rsyslog restart 2023-11-15T02:44:27.215 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2023-11-15T02:44:27.361 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-15T02:44:27.369 INFO:teuthology.task.internal:Starting timer... 2023-11-15T02:44:27.369 INFO:teuthology.run_tasks:Running task pcp... 2023-11-15T02:44:27.404 INFO:teuthology.run_tasks:Running task selinux... 2023-11-15T02:44:27.444 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-15T02:44:27.445 DEBUG:teuthology.task.selinux:Excluding smithi121: OS 'ubuntu' does not support SELinux 2023-11-15T02:44:27.445 DEBUG:teuthology.task.selinux:Excluding smithi190: OS 'ubuntu' does not support SELinux 2023-11-15T02:44:27.445 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-15T02:44:27.446 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-11-15T02:44:27.447 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-15T02:44:27.447 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-15T02:44:27.486 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-15T02:44:27.487 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-15T02:44:27.508 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-15T02:44:27.510 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi121.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-15T02:52:29.729 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi121.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2023-11-15T02:52:29.731 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-11-15T02:52:29.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T02:52:29.843 DEBUG:teuthology.orchestra.run.smithi121:> true 2023-11-15T02:52:30.070 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-11-15T02:52:30.070 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-11-15T02:52:30.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T02:52:30.180 DEBUG:teuthology.orchestra.run.smithi190:> true 2023-11-15T02:52:30.419 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-11-15T02:52:30.420 INFO:teuthology.run_tasks:Running task clock... 2023-11-15T02:52:30.430 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-15T02:52:30.431 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-15T02:52:30.432 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-15T02:52:30.439 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-15T02:52:30.439 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-15T02:52:30.508 INFO:teuthology.orchestra.run.smithi121.stdout:15 Nov 02:52:30 ntpd[22725]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-11-15T02:52:30.508 INFO:teuthology.orchestra.run.smithi121.stdout:15 Nov 02:52:30 ntpd[22725]: Command line: ntpd -gq 2023-11-15T02:52:30.509 INFO:teuthology.orchestra.run.smithi121.stdout:15 Nov 02:52:30 ntpd[22725]: proto: precision = 0.041 usec (-24) 2023-11-15T02:52:30.509 INFO:teuthology.orchestra.run.smithi121.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-15T02:52:30.510 INFO:teuthology.orchestra.run.smithi121.stdout:15 Nov 02:52:30 ntpd[22725]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-15T02:52:30.510 INFO:teuthology.orchestra.run.smithi121.stdout:15 Nov 02:52:30 ntpd[22725]: restrict ::: KOD does nothing without LIMITED. 2023-11-15T02:52:30.510 INFO:teuthology.orchestra.run.smithi121.stderr:restrict ::: KOD does nothing without LIMITED. 2023-11-15T02:52:30.511 INFO:teuthology.orchestra.run.smithi121.stdout:15 Nov 02:52:30 ntpd[22725]: Listen and drop on 0 v6wildcard [::]:123 2023-11-15T02:52:30.512 INFO:teuthology.orchestra.run.smithi121.stdout:15 Nov 02:52:30 ntpd[22725]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-11-15T02:52:30.512 INFO:teuthology.orchestra.run.smithi121.stdout:15 Nov 02:52:30 ntpd[22725]: Listen normally on 2 lo 127.0.0.1:123 2023-11-15T02:52:30.512 INFO:teuthology.orchestra.run.smithi121.stdout:15 Nov 02:52:30 ntpd[22725]: Listen normally on 3 enp3s0f1 172.21.15.121:123 2023-11-15T02:52:30.512 INFO:teuthology.orchestra.run.smithi121.stdout:15 Nov 02:52:30 ntpd[22725]: Listen normally on 4 lo [::1]:123 2023-11-15T02:52:30.512 INFO:teuthology.orchestra.run.smithi121.stdout:15 Nov 02:52:30 ntpd[22725]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cba3%5]:123 2023-11-15T02:52:30.513 INFO:teuthology.orchestra.run.smithi121.stdout:15 Nov 02:52:30 ntpd[22725]: Listening on routing socket on fd #22 for interface updates 2023-11-15T02:52:30.540 INFO:teuthology.orchestra.run.smithi190.stdout:15 Nov 02:52:30 ntpd[22682]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-11-15T02:52:30.540 INFO:teuthology.orchestra.run.smithi190.stdout:15 Nov 02:52:30 ntpd[22682]: Command line: ntpd -gq 2023-11-15T02:52:30.541 INFO:teuthology.orchestra.run.smithi190.stdout:15 Nov 02:52:30 ntpd[22682]: proto: precision = 0.065 usec (-24) 2023-11-15T02:52:30.542 INFO:teuthology.orchestra.run.smithi190.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-15T02:52:30.542 INFO:teuthology.orchestra.run.smithi190.stdout:15 Nov 02:52:30 ntpd[22682]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-15T02:52:30.543 INFO:teuthology.orchestra.run.smithi190.stdout:15 Nov 02:52:30 ntpd[22682]: restrict ::: KOD does nothing without LIMITED. 2023-11-15T02:52:30.543 INFO:teuthology.orchestra.run.smithi190.stderr:restrict ::: KOD does nothing without LIMITED. 2023-11-15T02:52:30.544 INFO:teuthology.orchestra.run.smithi190.stdout:15 Nov 02:52:30 ntpd[22682]: Listen and drop on 0 v6wildcard [::]:123 2023-11-15T02:52:30.544 INFO:teuthology.orchestra.run.smithi190.stdout:15 Nov 02:52:30 ntpd[22682]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-11-15T02:52:30.544 INFO:teuthology.orchestra.run.smithi190.stdout:15 Nov 02:52:30 ntpd[22682]: Listen normally on 2 lo 127.0.0.1:123 2023-11-15T02:52:30.545 INFO:teuthology.orchestra.run.smithi190.stdout:15 Nov 02:52:30 ntpd[22682]: Listen normally on 3 enp3s0f1 172.21.15.190:123 2023-11-15T02:52:30.545 INFO:teuthology.orchestra.run.smithi190.stdout:15 Nov 02:52:30 ntpd[22682]: Listen normally on 4 lo [::1]:123 2023-11-15T02:52:30.545 INFO:teuthology.orchestra.run.smithi190.stdout:15 Nov 02:52:30 ntpd[22682]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7763%5]:123 2023-11-15T02:52:30.545 INFO:teuthology.orchestra.run.smithi190.stdout:15 Nov 02:52:30 ntpd[22682]: Listening on routing socket on fd #22 for interface updates 2023-11-15T02:52:31.509 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:31 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:31.509 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:31 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:31.509 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:31 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:31.509 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:31 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:31.511 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:31 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:31.511 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:31 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:31.511 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:31 ntpd[22725]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:31.512 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:31 ntpd[22725]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:31.512 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:31 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:31.512 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:31 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:31.512 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:31 ntpd[22725]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:31.512 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:31 ntpd[22725]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:31.540 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:31 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:31.541 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:31 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:32.509 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:32 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:32.509 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:32 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:32.510 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:32 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:32.510 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:32 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:32.510 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:32 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:32.510 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:32 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:32.511 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:32 ntpd[22725]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:32.511 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:32 ntpd[22725]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:33.509 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:33 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:33.509 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:33 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:33.510 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:33 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:33.510 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:33 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:33.510 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:33 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:33.510 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:33 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:33.510 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:33 ntpd[22725]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:33.511 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:33 ntpd[22725]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:33.511 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:33 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:33.511 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:33 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:33.511 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:33 ntpd[22725]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:33.511 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:33 ntpd[22725]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:33.541 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:33 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:33.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:33 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:33.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:33 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:33.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:33 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:33.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:33 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:33.543 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:33 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:33.543 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:33 ntpd[22682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:33.543 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:33 ntpd[22682]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:33.543 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:33 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:33.544 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:33 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:33.544 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:33 ntpd[22682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:33.544 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:33 ntpd[22682]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:34.509 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:34 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:34.509 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:34 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:34.509 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:34 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:34.509 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:34 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:34.510 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:34 ntpd[22725]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:34.510 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:34 ntpd[22725]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:34.541 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:34 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:34.541 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:34 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:34.541 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:34 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:34.541 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:34 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:34.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:34 ntpd[22682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:34.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:34 ntpd[22682]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:35.511 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:35 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:35.512 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:35 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:35.512 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:35 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:35.513 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:35 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:35.513 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:35 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:35.513 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:35 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:35.514 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:35 ntpd[22725]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:35.514 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:35 ntpd[22725]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:35.515 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:35 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:35.515 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:35 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:35.515 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:35 ntpd[22725]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:35.516 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:35 ntpd[22725]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:35.541 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:35 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:35.541 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:35 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:35.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:35 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:35.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:35 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:35.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:35 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:35.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:35 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:35.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:35 ntpd[22682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:35.543 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:35 ntpd[22682]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:35.543 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:35 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:35.543 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:35 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:35.543 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:35 ntpd[22682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:35.544 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:35 ntpd[22682]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:36.509 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:36 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:36.509 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:36 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:36.509 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:36 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:36.509 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:36 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:36.510 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:36 ntpd[22725]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:36.510 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:36 ntpd[22725]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:36.541 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:36 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:36.541 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:36 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:36.541 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:36 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:36.541 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:36 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:36.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:36 ntpd[22682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:36.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:36 ntpd[22682]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:37.509 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:37 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:37.509 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:37 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:37.510 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:37 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:37.510 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:37 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:37.510 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:37 ntpd[22725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:37.510 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:37 ntpd[22725]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:37.510 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:37 ntpd[22725]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:37.511 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:37 ntpd[22725]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:37.511 INFO:teuthology.orchestra.run.smithi121.stdout:15 Nov 02:52:37 ntpd[22725]: ntpd: time slew +0.000245 s 2023-11-15T02:52:37.511 INFO:teuthology.orchestra.run.smithi121.stdout:ntpd: time slew +0.000245s 2023-11-15T02:52:37.512 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:37 ntpd[22725]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-11-15T02:52:37.512 INFO:teuthology.orchestra.run.smithi121.stderr:15 Nov 02:52:37 ntpd[22725]: can't open /var/log/ntpstats/loopstats.20231115: Permission denied 2023-11-15T02:52:37.541 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:37 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:37.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:37 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:37.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:37 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:37.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:37 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:37.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:37 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:37.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:37 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:37.543 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:37 ntpd[22682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:37.543 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:37 ntpd[22682]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:37.543 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:37 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:37.543 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:37 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:37.543 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:37 ntpd[22682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:37.544 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:37 ntpd[22682]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:37.655 INFO:teuthology.orchestra.run.smithi121.stdout: remote refid st t when poll reach delay offset jitter 2023-11-15T02:52:37.656 INFO:teuthology.orchestra.run.smithi121.stdout:============================================================================== 2023-11-15T02:52:37.656 INFO:teuthology.orchestra.run.smithi121.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-11-15T02:52:37.656 INFO:teuthology.orchestra.run.smithi121.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-11-15T02:52:37.656 INFO:teuthology.orchestra.run.smithi121.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-11-15T02:52:37.656 INFO:teuthology.orchestra.run.smithi121.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-11-15T02:52:38.541 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:38 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:38.541 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:38 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:38.541 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:38 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:38.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:38 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:38.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:38 ntpd[22682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:38.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:38 ntpd[22682]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:39.541 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:39 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:39.541 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:39 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:39.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:39 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:39.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:39 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:39.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:39 ntpd[22682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-15T02:52:39.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:39 ntpd[22682]: can't open /var/log/ntpstats/rawstats.20231115: Permission denied 2023-11-15T02:52:39.542 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:39 ntpd[22682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-15T02:52:39.543 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:39 ntpd[22682]: can't open /var/log/ntpstats/peerstats.20231115: Permission denied 2023-11-15T02:52:39.543 INFO:teuthology.orchestra.run.smithi190.stdout:15 Nov 02:52:39 ntpd[22682]: ntpd: time slew -0.001548 s 2023-11-15T02:52:39.543 INFO:teuthology.orchestra.run.smithi190.stdout:ntpd: time slew -0.001548s 2023-11-15T02:52:39.544 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:39 ntpd[22682]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-11-15T02:52:39.544 INFO:teuthology.orchestra.run.smithi190.stderr:15 Nov 02:52:39 ntpd[22682]: can't open /var/log/ntpstats/loopstats.20231115: Permission denied 2023-11-15T02:52:39.678 INFO:teuthology.orchestra.run.smithi190.stdout: remote refid st t when poll reach delay offset jitter 2023-11-15T02:52:39.678 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================== 2023-11-15T02:52:39.678 INFO:teuthology.orchestra.run.smithi190.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-11-15T02:52:39.679 INFO:teuthology.orchestra.run.smithi190.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-11-15T02:52:39.679 INFO:teuthology.orchestra.run.smithi190.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-11-15T02:52:39.679 INFO:teuthology.orchestra.run.smithi190.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-11-15T02:52:39.680 INFO:teuthology.run_tasks:Running task install... 2023-11-15T02:52:39.689 DEBUG:teuthology.task.install:project ceph 2023-11-15T02:52:39.690 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '620f25c1794f62207424fc9319427943fb8d1fae'}} 2023-11-15T02:52:39.690 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '620f25c1794f62207424fc9319427943fb8d1fae'} 2023-11-15T02:52:39.690 INFO:teuthology.task.install:Using flavor: default 2023-11-15T02:52:39.696 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-15T02:52:39.697 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-15T02:52:39.697 DEBUG:teuthology.orchestra.run.smithi121:> sudo apt-key list | grep Ceph 2023-11-15T02:52:39.698 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-key list | grep Ceph 2023-11-15T02:52:39.932 INFO:teuthology.orchestra.run.smithi190.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-11-15T02:52:39.934 INFO:teuthology.orchestra.run.smithi121.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-11-15T02:52:40.077 INFO:teuthology.orchestra.run.smithi190.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-11-15T02:52:40.078 INFO:teuthology.orchestra.run.smithi190.stdout:uid [ unknown] Ceph.com (release key) 2023-11-15T02:52:40.080 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-11-15T02:52:40.080 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2023-11-15T02:52:40.080 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=620f25c1794f62207424fc9319427943fb8d1fae 2023-11-15T02:52:40.085 INFO:teuthology.orchestra.run.smithi121.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-11-15T02:52:40.085 INFO:teuthology.orchestra.run.smithi121.stdout:uid [ unknown] Ceph.com (release key) 2023-11-15T02:52:40.085 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-11-15T02:52:40.086 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2023-11-15T02:52:40.086 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=620f25c1794f62207424fc9319427943fb8d1fae 2023-11-15T02:52:40.228 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default/ 2023-11-15T02:52:40.229 INFO:teuthology.task.install.deb:Package version is 16.2.14-170-g620f25c1-1focal 2023-11-15T02:52:40.236 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default/ 2023-11-15T02:52:40.236 INFO:teuthology.task.install.deb:Package version is 16.2.14-170-g620f25c1-1focal 2023-11-15T02:52:40.410 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:52:40.410 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-11-15T02:52:40.412 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:52:40.413 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-11-15T02:52:40.449 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-get update 2023-11-15T02:52:40.454 DEBUG:teuthology.orchestra.run.smithi121:> sudo apt-get update 2023-11-15T02:52:40.734 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-11-15T02:52:40.734 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-11-15T02:52:40.735 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:3 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-11-15T02:52:40.744 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-11-15T02:52:40.774 INFO:teuthology.orchestra.run.smithi190.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal InRelease 2023-11-15T02:52:40.797 INFO:teuthology.orchestra.run.smithi190.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal Release [7,680 B] 2023-11-15T02:52:40.822 INFO:teuthology.orchestra.run.smithi190.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal Release.gpg 2023-11-15T02:52:40.845 INFO:teuthology.orchestra.run.smithi190.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 Packages [17.2 kB] 2023-11-15T02:52:40.865 INFO:teuthology.orchestra.run.smithi121.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-11-15T02:52:40.866 INFO:teuthology.orchestra.run.smithi121.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-11-15T02:52:40.866 INFO:teuthology.orchestra.run.smithi121.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal InRelease 2023-11-15T02:52:40.889 INFO:teuthology.orchestra.run.smithi190.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main i386 Packages [3,105 B] 2023-11-15T02:52:40.903 INFO:teuthology.orchestra.run.smithi121.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal Release [7,680 B] 2023-11-15T02:52:40.922 INFO:teuthology.orchestra.run.smithi121.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-11-15T02:52:40.942 INFO:teuthology.orchestra.run.smithi121.stdout:Ign:6 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal Release.gpg 2023-11-15T02:52:40.979 INFO:teuthology.orchestra.run.smithi121.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main i386 Packages [3,105 B] 2023-11-15T02:52:41.004 INFO:teuthology.orchestra.run.smithi121.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-11-15T02:52:41.035 INFO:teuthology.orchestra.run.smithi121.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 Packages [17.2 kB] 2023-11-15T02:52:42.152 INFO:teuthology.orchestra.run.smithi190.stdout:Fetched 28.0 kB in 1s (20.4 kB/s) 2023-11-15T02:52:42.273 INFO:teuthology.orchestra.run.smithi121.stdout:Fetched 28.0 kB in 1s (18.9 kB/s) 2023-11-15T02:52:43.356 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2023-11-15T02:52:43.395 DEBUG:teuthology.orchestra.run.smithi190:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.2.14-170-g620f25c1-1focal cephadm=16.2.14-170-g620f25c1-1focal ceph-mds=16.2.14-170-g620f25c1-1focal ceph-mgr=16.2.14-170-g620f25c1-1focal ceph-common=16.2.14-170-g620f25c1-1focal ceph-fuse=16.2.14-170-g620f25c1-1focal ceph-test=16.2.14-170-g620f25c1-1focal radosgw=16.2.14-170-g620f25c1-1focal python3-rados=16.2.14-170-g620f25c1-1focal python3-rgw=16.2.14-170-g620f25c1-1focal python3-cephfs=16.2.14-170-g620f25c1-1focal python3-rbd=16.2.14-170-g620f25c1-1focal libcephfs2=16.2.14-170-g620f25c1-1focal libcephfs-dev=16.2.14-170-g620f25c1-1focal librados2=16.2.14-170-g620f25c1-1focal librbd1=16.2.14-170-g620f25c1-1focal rbd-fuse=16.2.14-170-g620f25c1-1focal python3-cephfs=16.2.14-170-g620f25c1-1focal cephfs-shell=16.2.14-170-g620f25c1-1focal cephfs-top=16.2.14-170-g620f25c1-1focal cephfs-mirror=16.2.14-170-g620f25c1-1focal 2023-11-15T02:52:43.468 INFO:teuthology.orchestra.run.smithi121.stdout:Reading package lists... 2023-11-15T02:52:43.514 DEBUG:teuthology.orchestra.run.smithi121:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.2.14-170-g620f25c1-1focal cephadm=16.2.14-170-g620f25c1-1focal ceph-mds=16.2.14-170-g620f25c1-1focal ceph-mgr=16.2.14-170-g620f25c1-1focal ceph-common=16.2.14-170-g620f25c1-1focal ceph-fuse=16.2.14-170-g620f25c1-1focal ceph-test=16.2.14-170-g620f25c1-1focal radosgw=16.2.14-170-g620f25c1-1focal python3-rados=16.2.14-170-g620f25c1-1focal python3-rgw=16.2.14-170-g620f25c1-1focal python3-cephfs=16.2.14-170-g620f25c1-1focal python3-rbd=16.2.14-170-g620f25c1-1focal libcephfs2=16.2.14-170-g620f25c1-1focal libcephfs-dev=16.2.14-170-g620f25c1-1focal librados2=16.2.14-170-g620f25c1-1focal librbd1=16.2.14-170-g620f25c1-1focal rbd-fuse=16.2.14-170-g620f25c1-1focal python3-cephfs=16.2.14-170-g620f25c1-1focal cephfs-shell=16.2.14-170-g620f25c1-1focal cephfs-top=16.2.14-170-g620f25c1-1focal cephfs-mirror=16.2.14-170-g620f25c1-1focal 2023-11-15T02:52:43.519 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2023-11-15T02:52:43.635 INFO:teuthology.orchestra.run.smithi121.stdout:Reading package lists... 2023-11-15T02:52:43.721 INFO:teuthology.orchestra.run.smithi190.stdout:Building dependency tree... 2023-11-15T02:52:43.721 INFO:teuthology.orchestra.run.smithi190.stdout:Reading state information... 2023-11-15T02:52:43.835 INFO:teuthology.orchestra.run.smithi121.stdout:Building dependency tree... 2023-11-15T02:52:43.835 INFO:teuthology.orchestra.run.smithi121.stdout:Reading state information... 2023-11-15T02:52:43.901 INFO:teuthology.orchestra.run.smithi190.stdout:The following packages were automatically installed and are no longer required: 2023-11-15T02:52:43.901 INFO:teuthology.orchestra.run.smithi190.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-11-15T02:52:43.902 INFO:teuthology.orchestra.run.smithi190.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-15T02:52:43.902 INFO:teuthology.orchestra.run.smithi190.stdout:The following additional packages will be installed: 2023-11-15T02:52:43.902 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-11-15T02:52:43.903 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq libjaeger 2023-11-15T02:52:43.903 INFO:teuthology.orchestra.run.smithi190.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2023-11-15T02:52:43.904 INFO:teuthology.orchestra.run.smithi190.stdout: liblttng-ust0 liblua5.3-0 liboath0 libonig5 librabbitmq4 libradosstriper1 2023-11-15T02:52:43.904 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph nvme-cli python-pastedeploy-tpl 2023-11-15T02:52:43.904 INFO:teuthology.orchestra.run.smithi190.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2023-11-15T02:52:43.904 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2023-11-15T02:52:43.904 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2023-11-15T02:52:43.905 INFO:teuthology.orchestra.run.smithi190.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2023-11-15T02:52:43.905 INFO:teuthology.orchestra.run.smithi190.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2023-11-15T02:52:43.905 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2023-11-15T02:52:43.906 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-11-15T02:52:43.906 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-11-15T02:52:43.906 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-11-15T02:52:43.906 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2023-11-15T02:52:43.906 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-11-15T02:52:43.907 INFO:teuthology.orchestra.run.smithi190.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-11-15T02:52:43.907 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-11-15T02:52:43.907 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2023-11-15T02:52:43.907 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet 2023-11-15T02:52:43.908 INFO:teuthology.orchestra.run.smithi190.stdout:Suggested packages: 2023-11-15T02:52:43.908 INFO:teuthology.orchestra.run.smithi190.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2023-11-15T02:52:43.908 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-11-15T02:52:43.908 INFO:teuthology.orchestra.run.smithi190.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-11-15T02:52:43.909 INFO:teuthology.orchestra.run.smithi190.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-11-15T02:52:43.909 INFO:teuthology.orchestra.run.smithi190.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-11-15T02:52:43.909 INFO:teuthology.orchestra.run.smithi190.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-11-15T02:52:43.910 INFO:teuthology.orchestra.run.smithi190.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-11-15T02:52:43.910 INFO:teuthology.orchestra.run.smithi190.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-11-15T02:52:43.910 INFO:teuthology.orchestra.run.smithi190.stdout:Recommended packages: 2023-11-15T02:52:43.910 INFO:teuthology.orchestra.run.smithi190.stdout: btrfs-tools 2023-11-15T02:52:44.004 INFO:teuthology.orchestra.run.smithi121.stdout:The following packages were automatically installed and are no longer required: 2023-11-15T02:52:44.005 INFO:teuthology.orchestra.run.smithi121.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-11-15T02:52:44.005 INFO:teuthology.orchestra.run.smithi121.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-15T02:52:44.005 INFO:teuthology.orchestra.run.smithi121.stdout:The following additional packages will be installed: 2023-11-15T02:52:44.005 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-11-15T02:52:44.006 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq libjaeger 2023-11-15T02:52:44.007 INFO:teuthology.orchestra.run.smithi121.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2023-11-15T02:52:44.007 INFO:teuthology.orchestra.run.smithi121.stdout: liblttng-ust0 liblua5.3-0 liboath0 libonig5 librabbitmq4 libradosstriper1 2023-11-15T02:52:44.007 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph nvme-cli python-pastedeploy-tpl 2023-11-15T02:52:44.007 INFO:teuthology.orchestra.run.smithi121.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2023-11-15T02:52:44.008 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2023-11-15T02:52:44.008 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2023-11-15T02:52:44.008 INFO:teuthology.orchestra.run.smithi121.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2023-11-15T02:52:44.008 INFO:teuthology.orchestra.run.smithi121.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2023-11-15T02:52:44.008 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2023-11-15T02:52:44.009 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-11-15T02:52:44.009 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-11-15T02:52:44.009 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-11-15T02:52:44.009 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2023-11-15T02:52:44.009 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-11-15T02:52:44.010 INFO:teuthology.orchestra.run.smithi121.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-11-15T02:52:44.010 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-11-15T02:52:44.010 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2023-11-15T02:52:44.010 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet 2023-11-15T02:52:44.011 INFO:teuthology.orchestra.run.smithi121.stdout:Suggested packages: 2023-11-15T02:52:44.011 INFO:teuthology.orchestra.run.smithi121.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2023-11-15T02:52:44.011 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-11-15T02:52:44.012 INFO:teuthology.orchestra.run.smithi121.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-11-15T02:52:44.012 INFO:teuthology.orchestra.run.smithi121.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-11-15T02:52:44.012 INFO:teuthology.orchestra.run.smithi121.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-11-15T02:52:44.012 INFO:teuthology.orchestra.run.smithi121.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-11-15T02:52:44.012 INFO:teuthology.orchestra.run.smithi121.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-11-15T02:52:44.013 INFO:teuthology.orchestra.run.smithi121.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-11-15T02:52:44.013 INFO:teuthology.orchestra.run.smithi121.stdout:Recommended packages: 2023-11-15T02:52:44.013 INFO:teuthology.orchestra.run.smithi121.stdout: btrfs-tools 2023-11-15T02:52:44.098 INFO:teuthology.orchestra.run.smithi190.stdout:The following NEW packages will be installed: 2023-11-15T02:52:44.098 INFO:teuthology.orchestra.run.smithi190.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-11-15T02:52:44.099 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-11-15T02:52:44.099 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test cephadm cephfs-mirror 2023-11-15T02:52:44.099 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 libjaeger libjq1 2023-11-15T02:52:44.099 INFO:teuthology.orchestra.run.smithi190.stdout: libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d liblttng-ust0 2023-11-15T02:52:44.099 INFO:teuthology.orchestra.run.smithi190.stdout: liblua5.3-0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2023-11-15T02:52:44.100 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2 libsqlite3-mod-ceph nvme-cli python-pastedeploy-tpl 2023-11-15T02:52:44.101 INFO:teuthology.orchestra.run.smithi190.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2023-11-15T02:52:44.101 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-11-15T02:52:44.101 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2023-11-15T02:52:44.101 INFO:teuthology.orchestra.run.smithi190.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2023-11-15T02:52:44.102 INFO:teuthology.orchestra.run.smithi190.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2023-11-15T02:52:44.102 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2023-11-15T02:52:44.102 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-11-15T02:52:44.102 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-11-15T02:52:44.102 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-11-15T02:52:44.103 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-11-15T02:52:44.103 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-11-15T02:52:44.103 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-11-15T02:52:44.103 INFO:teuthology.orchestra.run.smithi190.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-11-15T02:52:44.103 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-11-15T02:52:44.103 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug radosgw 2023-11-15T02:52:44.104 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse socat xmlstarlet 2023-11-15T02:52:44.104 INFO:teuthology.orchestra.run.smithi190.stdout:The following packages will be upgraded: 2023-11-15T02:52:44.104 INFO:teuthology.orchestra.run.smithi190.stdout: librados2 librbd1 2023-11-15T02:52:44.191 INFO:teuthology.orchestra.run.smithi190.stdout:2 upgraded, 104 newly installed, 0 to remove and 260 not upgraded. 2023-11-15T02:52:44.192 INFO:teuthology.orchestra.run.smithi190.stdout:Need to get 174 MB of archives. 2023-11-15T02:52:44.192 INFO:teuthology.orchestra.run.smithi190.stdout:After this operation, 806 MB of additional disk space will be used. 2023-11-15T02:52:44.192 INFO:teuthology.orchestra.run.smithi190.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-11-15T02:52:44.199 INFO:teuthology.orchestra.run.smithi121.stdout:The following NEW packages will be installed: 2023-11-15T02:52:44.199 INFO:teuthology.orchestra.run.smithi121.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-11-15T02:52:44.199 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-11-15T02:52:44.200 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test cephadm cephfs-mirror 2023-11-15T02:52:44.201 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 libjaeger libjq1 2023-11-15T02:52:44.201 INFO:teuthology.orchestra.run.smithi121.stdout: libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d liblttng-ust0 2023-11-15T02:52:44.201 INFO:teuthology.orchestra.run.smithi121.stdout: liblua5.3-0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2023-11-15T02:52:44.201 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2 libsqlite3-mod-ceph nvme-cli python-pastedeploy-tpl 2023-11-15T02:52:44.202 INFO:teuthology.orchestra.run.smithi121.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2023-11-15T02:52:44.202 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-11-15T02:52:44.202 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2023-11-15T02:52:44.202 INFO:teuthology.orchestra.run.smithi121.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2023-11-15T02:52:44.202 INFO:teuthology.orchestra.run.smithi121.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2023-11-15T02:52:44.203 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2023-11-15T02:52:44.203 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-11-15T02:52:44.203 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-11-15T02:52:44.203 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-11-15T02:52:44.203 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-11-15T02:52:44.204 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-11-15T02:52:44.204 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-11-15T02:52:44.204 INFO:teuthology.orchestra.run.smithi121.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-11-15T02:52:44.204 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-11-15T02:52:44.204 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug radosgw 2023-11-15T02:52:44.204 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse socat xmlstarlet 2023-11-15T02:52:44.205 INFO:teuthology.orchestra.run.smithi121.stdout:The following packages will be upgraded: 2023-11-15T02:52:44.205 INFO:teuthology.orchestra.run.smithi121.stdout: librados2 librbd1 2023-11-15T02:52:44.210 INFO:teuthology.orchestra.run.smithi190.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-11-15T02:52:44.219 INFO:teuthology.orchestra.run.smithi190.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-11-15T02:52:44.246 INFO:teuthology.orchestra.run.smithi190.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-11-15T02:52:44.264 INFO:teuthology.orchestra.run.smithi190.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 libjaeger amd64 16.2.14-170-g620f25c1-1focal [3,960 B] 2023-11-15T02:52:44.274 INFO:teuthology.orchestra.run.smithi190.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-11-15T02:52:44.275 INFO:teuthology.orchestra.run.smithi190.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-11-15T02:52:44.285 INFO:teuthology.orchestra.run.smithi190.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-11-15T02:52:44.287 INFO:teuthology.orchestra.run.smithi190.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-11-15T02:52:44.305 INFO:teuthology.orchestra.run.smithi190.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2023-11-15T02:52:44.309 INFO:teuthology.orchestra.run.smithi190.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-11-15T02:52:44.309 INFO:teuthology.orchestra.run.smithi190.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 16.2.14-170-g620f25c1-1focal [3,383 kB] 2023-11-15T02:52:44.310 INFO:teuthology.orchestra.run.smithi190.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-11-15T02:52:44.314 INFO:teuthology.orchestra.run.smithi190.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-11-15T02:52:44.315 INFO:teuthology.orchestra.run.smithi190.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-11-15T02:52:44.315 INFO:teuthology.orchestra.run.smithi190.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-11-15T02:52:44.319 INFO:teuthology.orchestra.run.smithi190.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-11-15T02:52:44.319 INFO:teuthology.orchestra.run.smithi190.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-11-15T02:52:44.320 INFO:teuthology.orchestra.run.smithi190.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-11-15T02:52:44.326 INFO:teuthology.orchestra.run.smithi190.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-11-15T02:52:44.337 INFO:teuthology.orchestra.run.smithi190.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-11-15T02:52:44.338 INFO:teuthology.orchestra.run.smithi190.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-11-15T02:52:44.339 INFO:teuthology.orchestra.run.smithi190.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-11-15T02:52:44.342 INFO:teuthology.orchestra.run.smithi190.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-11-15T02:52:44.345 INFO:teuthology.orchestra.run.smithi190.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-11-15T02:52:44.346 INFO:teuthology.orchestra.run.smithi190.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-11-15T02:52:44.349 INFO:teuthology.orchestra.run.smithi190.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-11-15T02:52:44.350 INFO:teuthology.orchestra.run.smithi190.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-11-15T02:52:44.354 INFO:teuthology.orchestra.run.smithi190.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-11-15T02:52:44.360 INFO:teuthology.orchestra.run.smithi190.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-11-15T02:52:44.361 INFO:teuthology.orchestra.run.smithi190.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-11-15T02:52:44.361 INFO:teuthology.orchestra.run.smithi190.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-11-15T02:52:44.362 INFO:teuthology.orchestra.run.smithi121.stdout:2 upgraded, 104 newly installed, 0 to remove and 260 not upgraded. 2023-11-15T02:52:44.363 INFO:teuthology.orchestra.run.smithi121.stdout:Need to get 174 MB of archives. 2023-11-15T02:52:44.363 INFO:teuthology.orchestra.run.smithi121.stdout:After this operation, 806 MB of additional disk space will be used. 2023-11-15T02:52:44.363 INFO:teuthology.orchestra.run.smithi121.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 libjaeger amd64 16.2.14-170-g620f25c1-1focal [3,960 B] 2023-11-15T02:52:44.377 INFO:teuthology.orchestra.run.smithi190.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-11-15T02:52:44.383 INFO:teuthology.orchestra.run.smithi190.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-11-15T02:52:44.385 INFO:teuthology.orchestra.run.smithi190.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-11-15T02:52:44.386 INFO:teuthology.orchestra.run.smithi190.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-11-15T02:52:44.387 INFO:teuthology.orchestra.run.smithi190.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-11-15T02:52:44.389 INFO:teuthology.orchestra.run.smithi190.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-11-15T02:52:44.390 INFO:teuthology.orchestra.run.smithi190.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-11-15T02:52:44.393 INFO:teuthology.orchestra.run.smithi190.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-11-15T02:52:44.394 INFO:teuthology.orchestra.run.smithi190.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-11-15T02:52:44.407 INFO:teuthology.orchestra.run.smithi121.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 16.2.14-170-g620f25c1-1focal [3,383 kB] 2023-11-15T02:52:44.420 INFO:teuthology.orchestra.run.smithi121.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-11-15T02:52:44.493 INFO:teuthology.orchestra.run.smithi190.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-11-15T02:52:44.505 INFO:teuthology.orchestra.run.smithi121.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-11-15T02:52:44.513 INFO:teuthology.orchestra.run.smithi190.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-11-15T02:52:44.515 INFO:teuthology.orchestra.run.smithi190.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-11-15T02:52:44.530 INFO:teuthology.orchestra.run.smithi190.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-11-15T02:52:44.531 INFO:teuthology.orchestra.run.smithi190.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-11-15T02:52:44.531 INFO:teuthology.orchestra.run.smithi190.stdout:Get:47 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 16.2.14-170-g620f25c1-1focal [3,388 kB] 2023-11-15T02:52:44.532 INFO:teuthology.orchestra.run.smithi190.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-11-15T02:52:44.532 INFO:teuthology.orchestra.run.smithi190.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-11-15T02:52:44.533 INFO:teuthology.orchestra.run.smithi190.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-11-15T02:52:44.533 INFO:teuthology.orchestra.run.smithi190.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-11-15T02:52:44.533 INFO:teuthology.orchestra.run.smithi190.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-11-15T02:52:44.539 INFO:teuthology.orchestra.run.smithi121.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-11-15T02:52:44.545 INFO:teuthology.orchestra.run.smithi190.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-11-15T02:52:44.545 INFO:teuthology.orchestra.run.smithi190.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-11-15T02:52:44.561 INFO:teuthology.orchestra.run.smithi190.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-11-15T02:52:44.569 INFO:teuthology.orchestra.run.smithi190.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-11-15T02:52:44.573 INFO:teuthology.orchestra.run.smithi190.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-11-15T02:52:44.576 INFO:teuthology.orchestra.run.smithi190.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-11-15T02:52:44.577 INFO:teuthology.orchestra.run.smithi190.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-11-15T02:52:44.580 INFO:teuthology.orchestra.run.smithi190.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-11-15T02:52:44.583 INFO:teuthology.orchestra.run.smithi190.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-11-15T02:52:44.583 INFO:teuthology.orchestra.run.smithi190.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-11-15T02:52:44.587 INFO:teuthology.orchestra.run.smithi190.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-11-15T02:52:44.587 INFO:teuthology.orchestra.run.smithi190.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-11-15T02:52:44.594 INFO:teuthology.orchestra.run.smithi190.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-11-15T02:52:44.595 INFO:teuthology.orchestra.run.smithi190.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-11-15T02:52:44.597 INFO:teuthology.orchestra.run.smithi190.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-11-15T02:52:44.615 INFO:teuthology.orchestra.run.smithi190.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-11-15T02:52:44.615 INFO:teuthology.orchestra.run.smithi190.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-11-15T02:52:44.616 INFO:teuthology.orchestra.run.smithi190.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-11-15T02:52:44.616 INFO:teuthology.orchestra.run.smithi190.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-11-15T02:52:44.617 INFO:teuthology.orchestra.run.smithi190.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-11-15T02:52:44.618 INFO:teuthology.orchestra.run.smithi190.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-11-15T02:52:44.619 INFO:teuthology.orchestra.run.smithi190.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-11-15T02:52:44.628 INFO:teuthology.orchestra.run.smithi190.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-11-15T02:52:44.678 INFO:teuthology.orchestra.run.smithi121.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-11-15T02:52:44.729 INFO:teuthology.orchestra.run.smithi190.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 16.2.14-170-g620f25c1-1focal [729 kB] 2023-11-15T02:52:44.740 INFO:teuthology.orchestra.run.smithi121.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 16.2.14-170-g620f25c1-1focal [3,388 kB] 2023-11-15T02:52:44.772 INFO:teuthology.orchestra.run.smithi190.stdout:Get:77 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 16.2.14-170-g620f25c1-1focal [413 kB] 2023-11-15T02:52:44.807 INFO:teuthology.orchestra.run.smithi190.stdout:Get:78 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 16.2.14-170-g620f25c1-1focal [22.2 kB] 2023-11-15T02:52:44.808 INFO:teuthology.orchestra.run.smithi190.stdout:Get:79 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 16.2.14-170-g620f25c1-1focal [212 kB] 2023-11-15T02:52:44.813 INFO:teuthology.orchestra.run.smithi121.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-11-15T02:52:44.814 INFO:teuthology.orchestra.run.smithi190.stdout:Get:80 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 16.2.14-170-g620f25c1-1focal [38.5 kB] 2023-11-15T02:52:44.815 INFO:teuthology.orchestra.run.smithi190.stdout:Get:81 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 16.2.14-170-g620f25c1-1focal [385 kB] 2023-11-15T02:52:44.824 INFO:teuthology.orchestra.run.smithi121.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-11-15T02:52:44.851 INFO:teuthology.orchestra.run.smithi190.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 16.2.14-170-g620f25c1-1focal [3,705 kB] 2023-11-15T02:52:44.868 INFO:teuthology.orchestra.run.smithi121.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-11-15T02:52:44.879 INFO:teuthology.orchestra.run.smithi121.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-11-15T02:52:44.964 INFO:teuthology.orchestra.run.smithi121.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2023-11-15T02:52:44.982 INFO:teuthology.orchestra.run.smithi121.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-11-15T02:52:44.987 INFO:teuthology.orchestra.run.smithi121.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-11-15T02:52:44.996 INFO:teuthology.orchestra.run.smithi121.stdout:Get:15 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 16.2.14-170-g620f25c1-1focal [729 kB] 2023-11-15T02:52:45.048 INFO:teuthology.orchestra.run.smithi190.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 16.2.14-170-g620f25c1-1focal [116 kB] 2023-11-15T02:52:45.051 INFO:teuthology.orchestra.run.smithi121.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 16.2.14-170-g620f25c1-1focal [413 kB] 2023-11-15T02:52:45.063 INFO:teuthology.orchestra.run.smithi121.stdout:Get:17 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 16.2.14-170-g620f25c1-1focal [22.2 kB] 2023-11-15T02:52:45.064 INFO:teuthology.orchestra.run.smithi121.stdout:Get:18 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 16.2.14-170-g620f25c1-1focal [212 kB] 2023-11-15T02:52:45.071 INFO:teuthology.orchestra.run.smithi121.stdout:Get:19 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 16.2.14-170-g620f25c1-1focal [38.5 kB] 2023-11-15T02:52:45.072 INFO:teuthology.orchestra.run.smithi121.stdout:Get:20 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 16.2.14-170-g620f25c1-1focal [385 kB] 2023-11-15T02:52:45.078 INFO:teuthology.orchestra.run.smithi190.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 16.2.14-170-g620f25c1-1focal [449 kB] 2023-11-15T02:52:45.092 INFO:teuthology.orchestra.run.smithi190.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 16.2.14-170-g620f25c1-1focal [22.7 MB] 2023-11-15T02:52:45.120 INFO:teuthology.orchestra.run.smithi121.stdout:Get:21 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 16.2.14-170-g620f25c1-1focal [3,705 kB] 2023-11-15T02:52:45.175 INFO:teuthology.orchestra.run.smithi121.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-11-15T02:52:45.402 INFO:teuthology.orchestra.run.smithi121.stdout:Get:23 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 16.2.14-170-g620f25c1-1focal [116 kB] 2023-11-15T02:52:45.404 INFO:teuthology.orchestra.run.smithi121.stdout:Get:24 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 16.2.14-170-g620f25c1-1focal [449 kB] 2023-11-15T02:52:45.418 INFO:teuthology.orchestra.run.smithi121.stdout:Get:25 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 16.2.14-170-g620f25c1-1focal [22.7 MB] 2023-11-15T02:52:45.421 INFO:teuthology.orchestra.run.smithi121.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-11-15T02:52:45.451 INFO:teuthology.orchestra.run.smithi121.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-11-15T02:52:45.643 INFO:teuthology.orchestra.run.smithi121.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-11-15T02:52:45.654 INFO:teuthology.orchestra.run.smithi121.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-11-15T02:52:45.664 INFO:teuthology.orchestra.run.smithi121.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-11-15T02:52:45.673 INFO:teuthology.orchestra.run.smithi121.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-11-15T02:52:45.694 INFO:teuthology.orchestra.run.smithi121.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-11-15T02:52:45.697 INFO:teuthology.orchestra.run.smithi121.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-11-15T02:52:45.700 INFO:teuthology.orchestra.run.smithi121.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-11-15T02:52:45.727 INFO:teuthology.orchestra.run.smithi121.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-11-15T02:52:45.899 INFO:teuthology.orchestra.run.smithi121.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-11-15T02:52:46.237 INFO:teuthology.orchestra.run.smithi121.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-11-15T02:52:46.305 INFO:teuthology.orchestra.run.smithi121.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-11-15T02:52:46.317 INFO:teuthology.orchestra.run.smithi121.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-11-15T02:52:46.425 INFO:teuthology.orchestra.run.smithi190.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 16.2.14-170-g620f25c1-1focal [5,916 kB] 2023-11-15T02:52:46.503 INFO:teuthology.orchestra.run.smithi121.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-11-15T02:52:46.503 INFO:teuthology.orchestra.run.smithi121.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-11-15T02:52:46.508 INFO:teuthology.orchestra.run.smithi121.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-11-15T02:52:46.513 INFO:teuthology.orchestra.run.smithi121.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-11-15T02:52:46.527 INFO:teuthology.orchestra.run.smithi121.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-11-15T02:52:46.551 INFO:teuthology.orchestra.run.smithi121.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-11-15T02:52:46.562 INFO:teuthology.orchestra.run.smithi121.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-11-15T02:52:46.727 INFO:teuthology.orchestra.run.smithi121.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-11-15T02:52:46.758 INFO:teuthology.orchestra.run.smithi190.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 16.2.14-170-g620f25c1-1focal [204 kB] 2023-11-15T02:52:46.761 INFO:teuthology.orchestra.run.smithi190.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 16.2.14-170-g620f25c1-1focal [126 kB] 2023-11-15T02:52:46.766 INFO:teuthology.orchestra.run.smithi190.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 16.2.14-170-g620f25c1-1focal [1,535 kB] 2023-11-15T02:52:46.877 INFO:teuthology.orchestra.run.smithi190.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 16.2.14-170-g620f25c1-1focal [6,998 kB] 2023-11-15T02:52:47.082 INFO:teuthology.orchestra.run.smithi121.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-11-15T02:52:47.140 INFO:teuthology.orchestra.run.smithi121.stdout:Get:49 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 16.2.14-170-g620f25c1-1focal [5,916 kB] 2023-11-15T02:52:47.149 INFO:teuthology.orchestra.run.smithi121.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-11-15T02:52:47.212 INFO:teuthology.orchestra.run.smithi121.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-11-15T02:52:47.218 INFO:teuthology.orchestra.run.smithi121.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-11-15T02:52:47.234 INFO:teuthology.orchestra.run.smithi121.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-11-15T02:52:47.295 INFO:teuthology.orchestra.run.smithi190.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 16.2.14-170-g620f25c1-1focal [26.2 MB] 2023-11-15T02:52:47.568 INFO:teuthology.orchestra.run.smithi121.stdout:Get:54 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 16.2.14-170-g620f25c1-1focal [204 kB] 2023-11-15T02:52:47.571 INFO:teuthology.orchestra.run.smithi121.stdout:Get:55 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 16.2.14-170-g620f25c1-1focal [126 kB] 2023-11-15T02:52:47.574 INFO:teuthology.orchestra.run.smithi121.stdout:Get:56 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 16.2.14-170-g620f25c1-1focal [1,535 kB] 2023-11-15T02:52:47.707 INFO:teuthology.orchestra.run.smithi121.stdout:Get:57 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 16.2.14-170-g620f25c1-1focal [6,998 kB] 2023-11-15T02:52:47.761 INFO:teuthology.orchestra.run.smithi121.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-11-15T02:52:47.820 INFO:teuthology.orchestra.run.smithi121.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-11-15T02:52:47.821 INFO:teuthology.orchestra.run.smithi121.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-11-15T02:52:47.889 INFO:teuthology.orchestra.run.smithi121.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-11-15T02:52:47.889 INFO:teuthology.orchestra.run.smithi121.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-11-15T02:52:48.053 INFO:teuthology.orchestra.run.smithi121.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-11-15T02:52:48.233 INFO:teuthology.orchestra.run.smithi121.stdout:Get:64 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 16.2.14-170-g620f25c1-1focal [26.2 MB] 2023-11-15T02:52:48.253 INFO:teuthology.orchestra.run.smithi121.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-11-15T02:52:48.295 INFO:teuthology.orchestra.run.smithi121.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-11-15T02:52:48.386 INFO:teuthology.orchestra.run.smithi121.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-11-15T02:52:48.426 INFO:teuthology.orchestra.run.smithi121.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-11-15T02:52:48.505 INFO:teuthology.orchestra.run.smithi121.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-11-15T02:52:48.520 INFO:teuthology.orchestra.run.smithi121.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-11-15T02:52:48.539 INFO:teuthology.orchestra.run.smithi121.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-11-15T02:52:48.637 INFO:teuthology.orchestra.run.smithi121.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-11-15T02:52:48.669 INFO:teuthology.orchestra.run.smithi121.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-11-15T02:52:48.690 INFO:teuthology.orchestra.run.smithi121.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-11-15T02:52:48.828 INFO:teuthology.orchestra.run.smithi190.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 16.2.14-170-g620f25c1-1focal [4,056 B] 2023-11-15T02:52:48.829 INFO:teuthology.orchestra.run.smithi190.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 16.2.14-170-g620f25c1-1focal [834 kB] 2023-11-15T02:52:48.861 INFO:teuthology.orchestra.run.smithi121.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-11-15T02:52:48.877 INFO:teuthology.orchestra.run.smithi190.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 16.2.14-170-g620f25c1-1focal [2,366 kB] 2023-11-15T02:52:49.025 INFO:teuthology.orchestra.run.smithi190.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 16.2.14-170-g620f25c1-1focal [77.4 kB] 2023-11-15T02:52:49.026 INFO:teuthology.orchestra.run.smithi190.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 16.2.14-170-g620f25c1-1focal [100 kB] 2023-11-15T02:52:49.030 INFO:teuthology.orchestra.run.smithi190.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 16.2.14-170-g620f25c1-1focal [2,824 kB] 2023-11-15T02:52:49.194 INFO:teuthology.orchestra.run.smithi190.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 16.2.14-170-g620f25c1-1focal [7,872 kB] 2023-11-15T02:52:49.359 INFO:teuthology.orchestra.run.smithi121.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-11-15T02:52:49.410 INFO:teuthology.orchestra.run.smithi121.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-11-15T02:52:49.414 INFO:teuthology.orchestra.run.smithi121.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-11-15T02:52:49.486 INFO:teuthology.orchestra.run.smithi121.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-11-15T02:52:49.486 INFO:teuthology.orchestra.run.smithi121.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-11-15T02:52:49.493 INFO:teuthology.orchestra.run.smithi121.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-11-15T02:52:49.494 INFO:teuthology.orchestra.run.smithi121.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-11-15T02:52:49.508 INFO:teuthology.orchestra.run.smithi121.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-11-15T02:52:49.621 INFO:teuthology.orchestra.run.smithi121.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-11-15T02:52:49.622 INFO:teuthology.orchestra.run.smithi121.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-11-15T02:52:49.702 INFO:teuthology.orchestra.run.smithi190.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 16.2.14-170-g620f25c1-1focal [4,168 B] 2023-11-15T02:52:49.703 INFO:teuthology.orchestra.run.smithi190.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 16.2.14-170-g620f25c1-1focal [49.4 MB] 2023-11-15T02:52:49.789 INFO:teuthology.orchestra.run.smithi121.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-11-15T02:52:49.956 INFO:teuthology.orchestra.run.smithi121.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-11-15T02:52:50.062 INFO:teuthology.orchestra.run.smithi121.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 16.2.14-170-g620f25c1-1focal [4,056 B] 2023-11-15T02:52:50.062 INFO:teuthology.orchestra.run.smithi121.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 16.2.14-170-g620f25c1-1focal [834 kB] 2023-11-15T02:52:50.076 INFO:teuthology.orchestra.run.smithi121.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 16.2.14-170-g620f25c1-1focal [2,366 kB] 2023-11-15T02:52:50.191 INFO:teuthology.orchestra.run.smithi121.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-11-15T02:52:50.242 INFO:teuthology.orchestra.run.smithi121.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-11-15T02:52:50.248 INFO:teuthology.orchestra.run.smithi121.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 16.2.14-170-g620f25c1-1focal [77.4 kB] 2023-11-15T02:52:50.255 INFO:teuthology.orchestra.run.smithi121.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-11-15T02:52:50.256 INFO:teuthology.orchestra.run.smithi121.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 16.2.14-170-g620f25c1-1focal [100 kB] 2023-11-15T02:52:50.260 INFO:teuthology.orchestra.run.smithi121.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 16.2.14-170-g620f25c1-1focal [2,824 kB] 2023-11-15T02:52:50.321 INFO:teuthology.orchestra.run.smithi121.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-11-15T02:52:50.421 INFO:teuthology.orchestra.run.smithi121.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 16.2.14-170-g620f25c1-1focal [7,872 kB] 2023-11-15T02:52:50.921 INFO:teuthology.orchestra.run.smithi121.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 16.2.14-170-g620f25c1-1focal [4,168 B] 2023-11-15T02:52:50.921 INFO:teuthology.orchestra.run.smithi121.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 16.2.14-170-g620f25c1-1focal [49.4 MB] 2023-11-15T02:52:53.034 INFO:teuthology.orchestra.run.smithi190.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 16.2.14-170-g620f25c1-1focal [237 kB] 2023-11-15T02:52:53.052 INFO:teuthology.orchestra.run.smithi190.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 16.2.14-170-g620f25c1-1focal [17.5 kB] 2023-11-15T02:52:53.053 INFO:teuthology.orchestra.run.smithi190.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 16.2.14-170-g620f25c1-1focal [13.1 kB] 2023-11-15T02:52:53.054 INFO:teuthology.orchestra.run.smithi190.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 16.2.14-170-g620f25c1-1focal [21.5 kB] 2023-11-15T02:52:53.055 INFO:teuthology.orchestra.run.smithi190.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 16.2.14-170-g620f25c1-1focal [11.3 MB] 2023-11-15T02:52:53.802 INFO:teuthology.orchestra.run.smithi190.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 16.2.14-170-g620f25c1-1focal [78.7 kB] 2023-11-15T02:52:54.003 INFO:teuthology.orchestra.run.smithi121.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 16.2.14-170-g620f25c1-1focal [237 kB] 2023-11-15T02:52:54.005 INFO:teuthology.orchestra.run.smithi121.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 16.2.14-170-g620f25c1-1focal [17.5 kB] 2023-11-15T02:52:54.006 INFO:teuthology.orchestra.run.smithi121.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 16.2.14-170-g620f25c1-1focal [13.1 kB] 2023-11-15T02:52:54.006 INFO:teuthology.orchestra.run.smithi121.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 16.2.14-170-g620f25c1-1focal [21.5 kB] 2023-11-15T02:52:54.007 INFO:teuthology.orchestra.run.smithi121.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 16.2.14-170-g620f25c1-1focal [11.3 MB] 2023-11-15T02:52:54.590 INFO:teuthology.orchestra.run.smithi121.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-11-13-0820-pacific/620f25c1794f62207424fc9319427943fb8d1fae/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 16.2.14-170-g620f25c1-1focal [78.7 kB] 2023-11-15T02:52:54.717 INFO:teuthology.orchestra.run.smithi190.stdout:Fetched 174 MB in 10s (18.0 MB/s) 2023-11-15T02:52:54.914 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pyperclip. 2023-11-15T02:52:54.988 INFO:teuthology.orchestra.run.smithi190.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 107758 files and directories currently installed.) 2023-11-15T02:52:54.991 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-11-15T02:52:55.035 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-11-15T02:52:55.346 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-wcwidth. 2023-11-15T02:52:55.369 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-11-15T02:52:55.404 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-11-15T02:52:55.496 INFO:teuthology.orchestra.run.smithi121.stdout:Fetched 174 MB in 10s (16.8 MB/s) 2023-11-15T02:52:55.671 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-pyperclip. 2023-11-15T02:52:55.723 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cmd2. 2023-11-15T02:52:55.744 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-11-15T02:52:55.745 INFO:teuthology.orchestra.run.smithi121.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 107758 files and directories currently installed.) 2023-11-15T02:52:55.748 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-11-15T02:52:55.784 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-11-15T02:52:55.790 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-11-15T02:52:56.109 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libjaeger. 2023-11-15T02:52:56.126 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../003-libjaeger_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:52:56.128 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-wcwidth. 2023-11-15T02:52:56.149 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-11-15T02:52:56.167 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libjaeger (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:52:56.187 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-11-15T02:52:56.532 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-cmd2. 2023-11-15T02:52:56.545 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-11-15T02:52:56.551 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-11-15T02:52:56.565 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../004-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-11-15T02:52:56.590 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-11-15T02:52:56.645 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-11-15T02:52:56.868 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libjaeger. 2023-11-15T02:52:56.888 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../003-libjaeger_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:52:56.942 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libjaeger (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:52:57.200 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../005-librbd1_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:52:57.358 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librbd1 (16.2.14-170-g620f25c1-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-11-15T02:52:57.471 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-11-15T02:52:57.492 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../004-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-11-15T02:52:57.579 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-11-15T02:52:57.995 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../005-librbd1_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:52:58.099 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../006-librados2_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:52:58.116 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking librbd1 (16.2.14-170-g620f25c1-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-11-15T02:52:58.220 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librados2 (16.2.14-170-g620f25c1-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-11-15T02:52:58.882 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../006-librados2_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:52:58.941 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libcephfs2. 2023-11-15T02:52:58.962 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../007-libcephfs2_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:52:59.004 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking librados2 (16.2.14-170-g620f25c1-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-11-15T02:52:59.016 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libcephfs2 (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:52:59.418 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rados. 2023-11-15T02:52:59.439 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../008-python3-rados_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:52:59.476 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rados (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:52:59.741 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libcephfs2. 2023-11-15T02:52:59.763 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../007-libcephfs2_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:52:59.808 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libcephfs2 (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:52:59.863 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-11-15T02:52:59.880 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../009-python3-ceph-argparse_16.2.14-170-g620f25c1-1focal_all.deb ... 2023-11-15T02:52:59.964 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-ceph-argparse (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:00.202 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-rados. 2023-11-15T02:53:00.223 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../008-python3-rados_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:00.260 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-rados (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:00.358 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cephfs. 2023-11-15T02:53:00.378 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../010-python3-cephfs_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:00.416 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cephfs (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:00.656 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-11-15T02:53:00.675 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../009-python3-ceph-argparse_16.2.14-170-g620f25c1-1focal_all.deb ... 2023-11-15T02:53:00.713 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-ceph-argparse (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:00.777 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-ceph-common. 2023-11-15T02:53:00.797 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../011-python3-ceph-common_16.2.14-170-g620f25c1-1focal_all.deb ... 2023-11-15T02:53:00.835 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-ceph-common (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:01.057 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-cephfs. 2023-11-15T02:53:01.079 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../010-python3-cephfs_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:01.123 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-cephfs (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:01.163 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-prettytable. 2023-11-15T02:53:01.184 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-5_all.deb ... 2023-11-15T02:53:01.213 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-11-15T02:53:01.499 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rbd. 2023-11-15T02:53:01.502 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-ceph-common. 2023-11-15T02:53:01.518 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../013-python3-rbd_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:01.522 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../011-python3-ceph-common_16.2.14-170-g620f25c1-1focal_all.deb ... 2023-11-15T02:53:01.557 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rbd (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:01.561 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-ceph-common (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:01.931 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-prettytable. 2023-11-15T02:53:01.948 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-5_all.deb ... 2023-11-15T02:53:01.960 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-11-15T02:53:01.973 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../014-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-11-15T02:53:01.988 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-11-15T02:53:02.018 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-11-15T02:53:02.316 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-rbd. 2023-11-15T02:53:02.337 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../013-python3-rbd_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:02.374 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-rbd (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:02.379 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-11-15T02:53:02.397 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../015-librabbitmq4_0.10.0-1_amd64.deb ... 2023-11-15T02:53:02.429 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-11-15T02:53:02.786 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-11-15T02:53:02.806 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../014-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-11-15T02:53:02.807 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-11-15T02:53:02.827 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../016-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-11-15T02:53:02.844 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-11-15T02:53:02.865 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-11-15T02:53:03.247 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-11-15T02:53:03.268 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../015-librabbitmq4_0.10.0-1_amd64.deb ... 2023-11-15T02:53:03.268 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package librgw2. 2023-11-15T02:53:03.286 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../017-librgw2_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:03.305 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-11-15T02:53:03.326 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librgw2 (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:03.708 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-11-15T02:53:03.729 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../016-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-11-15T02:53:03.766 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-11-15T02:53:03.971 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rgw. 2023-11-15T02:53:03.994 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../018-python3-rgw_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:04.037 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rgw (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:04.161 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package librgw2. 2023-11-15T02:53:04.181 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../017-librgw2_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:04.219 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking librgw2 (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:04.398 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libleveldb1d:amd64. 2023-11-15T02:53:04.421 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../019-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2023-11-15T02:53:04.465 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2023-11-15T02:53:04.871 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-rgw. 2023-11-15T02:53:04.884 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../018-python3-rgw_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:04.901 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package liboath0. 2023-11-15T02:53:04.918 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../020-liboath0_2.6.1-1.3_amd64.deb ... 2023-11-15T02:53:04.921 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-rgw (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:04.984 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-11-15T02:53:05.275 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libleveldb1d:amd64. 2023-11-15T02:53:05.292 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../019-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2023-11-15T02:53:05.320 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libradosstriper1. 2023-11-15T02:53:05.333 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2023-11-15T02:53:05.340 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../021-libradosstriper1_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:05.387 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libradosstriper1 (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:05.728 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package liboath0. 2023-11-15T02:53:05.748 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../020-liboath0_2.6.1-1.3_amd64.deb ... 2023-11-15T02:53:05.786 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-11-15T02:53:05.865 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-common. 2023-11-15T02:53:05.885 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../022-ceph-common_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:05.923 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-common (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:06.139 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libradosstriper1. 2023-11-15T02:53:06.155 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../021-libradosstriper1_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:06.196 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libradosstriper1 (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:06.708 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-common. 2023-11-15T02:53:06.728 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../022-ceph-common_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:06.799 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-common (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:08.278 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-base. 2023-11-15T02:53:08.301 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../023-ceph-base_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:08.367 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-base (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:09.164 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-base. 2023-11-15T02:53:09.185 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../023-ceph-base_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:09.245 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-base (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:09.382 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cryptography. 2023-11-15T02:53:09.405 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../024-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-11-15T02:53:09.449 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-11-15T02:53:09.896 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-openssl. 2023-11-15T02:53:09.914 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../025-python3-openssl_19.0.0-1build1_all.deb ... 2023-11-15T02:53:09.954 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-11-15T02:53:10.185 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-cryptography. 2023-11-15T02:53:10.205 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../024-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-11-15T02:53:10.234 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-11-15T02:53:10.282 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-11-15T02:53:10.300 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../026-ceph-mgr-modules-core_16.2.14-170-g620f25c1-1focal_all.deb ... 2023-11-15T02:53:10.340 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-modules-core (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:10.680 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-openssl. 2023-11-15T02:53:10.702 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../025-python3-openssl_19.0.0-1build1_all.deb ... 2023-11-15T02:53:10.746 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-11-15T02:53:10.853 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-bcrypt. 2023-11-15T02:53:10.874 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../027-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-11-15T02:53:10.912 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-11-15T02:53:11.057 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-11-15T02:53:11.078 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../026-ceph-mgr-modules-core_16.2.14-170-g620f25c1-1focal_all.deb ... 2023-11-15T02:53:11.115 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-mgr-modules-core (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:11.282 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cherrypy3. 2023-11-15T02:53:11.301 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2023-11-15T02:53:11.339 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-11-15T02:53:11.545 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-bcrypt. 2023-11-15T02:53:11.563 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../027-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-11-15T02:53:11.603 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-11-15T02:53:11.760 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-jwt. 2023-11-15T02:53:11.780 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../029-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-11-15T02:53:11.818 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-11-15T02:53:11.940 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-cherrypy3. 2023-11-15T02:53:11.959 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2023-11-15T02:53:11.997 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-11-15T02:53:12.188 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-logutils. 2023-11-15T02:53:12.210 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2023-11-15T02:53:12.255 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-11-15T02:53:12.393 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-jwt. 2023-11-15T02:53:12.411 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../029-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-11-15T02:53:12.451 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-11-15T02:53:12.641 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-markupsafe. 2023-11-15T02:53:12.661 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-11-15T02:53:12.699 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-11-15T02:53:12.879 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-logutils. 2023-11-15T02:53:12.900 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2023-11-15T02:53:12.937 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-11-15T02:53:13.069 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-mako. 2023-11-15T02:53:13.089 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-11-15T02:53:13.152 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-11-15T02:53:13.316 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-markupsafe. 2023-11-15T02:53:13.335 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-11-15T02:53:13.373 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-11-15T02:53:13.555 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-simplegeneric. 2023-11-15T02:53:13.577 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2023-11-15T02:53:13.613 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-11-15T02:53:13.743 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-mako. 2023-11-15T02:53:13.763 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-11-15T02:53:13.801 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-11-15T02:53:13.966 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-singledispatch. 2023-11-15T02:53:13.986 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-11-15T02:53:14.024 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-11-15T02:53:14.179 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-simplegeneric. 2023-11-15T02:53:14.197 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2023-11-15T02:53:14.237 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-11-15T02:53:14.402 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-webob. 2023-11-15T02:53:14.425 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2_all.deb ... 2023-11-15T02:53:14.468 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-11-15T02:53:14.590 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-singledispatch. 2023-11-15T02:53:14.609 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-11-15T02:53:14.648 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-11-15T02:53:14.863 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-soupsieve. 2023-11-15T02:53:14.885 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-11-15T02:53:14.930 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-11-15T02:53:15.026 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-webob. 2023-11-15T02:53:15.046 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2_all.deb ... 2023-11-15T02:53:15.084 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-11-15T02:53:15.299 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-bs4. 2023-11-15T02:53:15.321 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2023-11-15T02:53:15.365 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-11-15T02:53:15.462 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-soupsieve. 2023-11-15T02:53:15.482 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-11-15T02:53:15.520 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-11-15T02:53:15.803 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-waitress. 2023-11-15T02:53:15.825 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-11-15T02:53:15.882 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-bs4. 2023-11-15T02:53:15.892 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-11-15T02:53:15.903 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2023-11-15T02:53:15.948 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-11-15T02:53:16.272 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-tempita. 2023-11-15T02:53:16.292 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2023-11-15T02:53:16.330 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-11-15T02:53:16.353 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-waitress. 2023-11-15T02:53:16.377 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-11-15T02:53:16.426 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-11-15T02:53:16.700 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-paste. 2023-11-15T02:53:16.720 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-11-15T02:53:16.758 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-11-15T02:53:16.798 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-tempita. 2023-11-15T02:53:16.816 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2023-11-15T02:53:16.856 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-11-15T02:53:17.137 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-11-15T02:53:17.157 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-11-15T02:53:17.194 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-11-15T02:53:17.226 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-paste. 2023-11-15T02:53:17.247 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-11-15T02:53:17.292 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-11-15T02:53:17.679 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-11-15T02:53:17.696 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-11-15T02:53:17.737 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-11-15T02:53:18.139 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pastedeploy. 2023-11-15T02:53:18.159 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2023-11-15T02:53:18.197 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-11-15T02:53:18.542 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-webtest. 2023-11-15T02:53:18.562 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2023-11-15T02:53:18.600 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-11-15T02:53:18.774 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-pastedeploy. 2023-11-15T02:53:18.794 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2023-11-15T02:53:18.831 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-11-15T02:53:18.970 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pecan. 2023-11-15T02:53:18.993 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-11-15T02:53:19.037 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-11-15T02:53:19.210 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-webtest. 2023-11-15T02:53:19.229 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2023-11-15T02:53:19.276 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-11-15T02:53:19.458 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-werkzeug. 2023-11-15T02:53:19.477 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-11-15T02:53:19.516 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-11-15T02:53:19.646 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-pecan. 2023-11-15T02:53:19.666 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-11-15T02:53:19.704 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-11-15T02:53:19.903 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-11-15T02:53:19.921 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../046-libsqlite3-mod-ceph_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:19.986 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libsqlite3-mod-ceph (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:20.142 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-werkzeug. 2023-11-15T02:53:20.162 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-11-15T02:53:20.200 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-11-15T02:53:20.356 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr. 2023-11-15T02:53:20.373 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../047-ceph-mgr_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:20.405 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:20.604 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-11-15T02:53:20.624 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../046-libsqlite3-mod-ceph_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:20.687 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libsqlite3-mod-ceph (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:20.950 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mon. 2023-11-15T02:53:20.972 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../048-ceph-mon_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:21.016 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mon (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:21.082 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-mgr. 2023-11-15T02:53:21.103 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../047-ceph-mgr_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:21.165 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-mgr (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:21.702 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-mon. 2023-11-15T02:53:21.722 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../048-ceph-mon_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:21.768 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-mon (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:22.104 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-osd. 2023-11-15T02:53:22.122 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../049-ceph-osd_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:22.153 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-osd (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:22.872 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-osd. 2023-11-15T02:53:22.894 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../049-ceph-osd_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:22.954 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-osd (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:24.534 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph. 2023-11-15T02:53:24.554 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../050-ceph_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:24.592 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:24.970 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-fuse. 2023-11-15T02:53:24.990 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../051-ceph-fuse_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:25.028 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-fuse (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:25.302 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph. 2023-11-15T02:53:25.323 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../050-ceph_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:25.360 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:25.498 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mds. 2023-11-15T02:53:25.516 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../052-ceph-mds_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:25.556 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mds (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:25.746 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-fuse. 2023-11-15T02:53:25.769 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../051-ceph-fuse_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:25.804 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-fuse (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:26.117 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package cephadm. 2023-11-15T02:53:26.135 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../053-cephadm_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:26.175 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking cephadm (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:26.258 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-mds. 2023-11-15T02:53:26.278 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../052-ceph-mds_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:26.307 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-mds (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:26.570 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-jinja2. 2023-11-15T02:53:26.593 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../054-python3-jinja2_2.10.1-2_all.deb ... 2023-11-15T02:53:26.636 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-11-15T02:53:26.861 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package cephadm. 2023-11-15T02:53:26.879 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../053-cephadm_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:26.918 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking cephadm (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:26.998 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-11-15T02:53:27.016 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../055-ceph-mgr-cephadm_16.2.14-170-g620f25c1-1focal_all.deb ... 2023-11-15T02:53:27.081 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-cephadm (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:27.280 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-jinja2. 2023-11-15T02:53:27.299 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../054-python3-jinja2_2.10.1-2_all.deb ... 2023-11-15T02:53:27.338 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-11-15T02:53:27.484 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-repoze.lru. 2023-11-15T02:53:27.497 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../056-python3-repoze.lru_0.7-2_all.deb ... 2023-11-15T02:53:27.542 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-11-15T02:53:27.733 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-11-15T02:53:27.753 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../055-ceph-mgr-cephadm_16.2.14-170-g620f25c1-1focal_all.deb ... 2023-11-15T02:53:27.799 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-mgr-cephadm (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:27.853 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libjs-underscore. 2023-11-15T02:53:27.866 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../057-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-11-15T02:53:28.192 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-11-15T02:53:28.211 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-repoze.lru. 2023-11-15T02:53:28.230 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../056-python3-repoze.lru_0.7-2_all.deb ... 2023-11-15T02:53:28.269 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-11-15T02:53:28.572 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-11-15T02:53:28.594 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../058-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-11-15T02:53:28.655 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-11-15T02:53:28.897 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libjs-underscore. 2023-11-15T02:53:28.917 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../057-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-11-15T02:53:28.954 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-11-15T02:53:29.025 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-routes. 2023-11-15T02:53:29.045 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../059-python3-routes_2.4.1-2_all.deb ... 2023-11-15T02:53:29.083 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-11-15T02:53:29.265 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-11-15T02:53:29.286 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../058-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-11-15T02:53:29.357 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-11-15T02:53:29.503 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-11-15T02:53:29.524 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../060-ceph-mgr-dashboard_16.2.14-170-g620f25c1-1focal_all.deb ... 2023-11-15T02:53:29.569 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-dashboard (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:29.710 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-routes. 2023-11-15T02:53:29.730 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../059-python3-routes_2.4.1-2_all.deb ... 2023-11-15T02:53:29.801 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-11-15T02:53:30.163 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-11-15T02:53:30.183 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../060-ceph-mgr-dashboard_16.2.14-170-g620f25c1-1focal_all.deb ... 2023-11-15T02:53:30.221 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-mgr-dashboard (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:30.736 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-decorator. 2023-11-15T02:53:30.758 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../061-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-11-15T02:53:30.802 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-11-15T02:53:31.172 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-11-15T02:53:31.190 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../062-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-11-15T02:53:31.229 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-11-15T02:53:31.412 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-decorator. 2023-11-15T02:53:31.433 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../061-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-11-15T02:53:31.478 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-11-15T02:53:31.599 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-scipy. 2023-11-15T02:53:31.620 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../063-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-11-15T02:53:31.665 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-11-15T02:53:31.840 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-11-15T02:53:31.860 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../062-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-11-15T02:53:31.906 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-11-15T02:53:32.333 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-scipy. 2023-11-15T02:53:32.347 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../063-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-11-15T02:53:32.392 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-11-15T02:53:33.116 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-11-15T02:53:33.130 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-11-15T02:53:33.174 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-11-15T02:53:33.795 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-joblib. 2023-11-15T02:53:33.808 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../065-python3-joblib_0.14.0-3_all.deb ... 2023-11-15T02:53:33.853 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-11-15T02:53:33.876 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-11-15T02:53:33.896 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-11-15T02:53:33.959 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-11-15T02:53:34.282 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-sklearn. 2023-11-15T02:53:34.303 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../066-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-11-15T02:53:34.348 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-11-15T02:53:34.571 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-joblib. 2023-11-15T02:53:34.593 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../065-python3-joblib_0.14.0-3_all.deb ... 2023-11-15T02:53:34.637 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-11-15T02:53:35.058 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-sklearn. 2023-11-15T02:53:35.080 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../066-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-11-15T02:53:35.124 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-11-15T02:53:35.165 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-11-15T02:53:35.183 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_16.2.14-170-g620f25c1-1focal_all.deb ... 2023-11-15T02:53:35.223 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-diskprediction-local (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:35.957 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-11-15T02:53:35.978 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_16.2.14-170-g620f25c1-1focal_all.deb ... 2023-11-15T02:53:36.023 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-mgr-diskprediction-local (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:36.425 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cachetools. 2023-11-15T02:53:36.438 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../068-python3-cachetools_4.0.0-1_all.deb ... 2023-11-15T02:53:36.500 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-11-15T02:53:36.837 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pyasn1. 2023-11-15T02:53:36.858 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../069-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-11-15T02:53:36.912 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-11-15T02:53:37.260 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-cachetools. 2023-11-15T02:53:37.273 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../068-python3-cachetools_4.0.0-1_all.deb ... 2023-11-15T02:53:37.318 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-11-15T02:53:37.330 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-11-15T02:53:37.352 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../070-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-11-15T02:53:37.397 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-11-15T02:53:37.679 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-pyasn1. 2023-11-15T02:53:37.699 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../069-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-11-15T02:53:37.737 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-11-15T02:53:37.817 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rsa. 2023-11-15T02:53:37.837 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../071-python3-rsa_4.0-3_all.deb ... 2023-11-15T02:53:37.874 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rsa (4.0-3) ... 2023-11-15T02:53:38.124 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-11-15T02:53:38.145 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../070-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-11-15T02:53:38.190 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-11-15T02:53:38.261 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-google-auth. 2023-11-15T02:53:38.282 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../072-python3-google-auth_1.5.1-2_all.deb ... 2023-11-15T02:53:38.319 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-11-15T02:53:38.567 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-rsa. 2023-11-15T02:53:38.585 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../071-python3-rsa_4.0-3_all.deb ... 2023-11-15T02:53:38.625 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-rsa (4.0-3) ... 2023-11-15T02:53:38.723 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-blinker. 2023-11-15T02:53:38.741 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../073-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-11-15T02:53:38.780 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-11-15T02:53:39.012 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-google-auth. 2023-11-15T02:53:39.029 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../072-python3-google-auth_1.5.1-2_all.deb ... 2023-11-15T02:53:39.070 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-11-15T02:53:39.134 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-oauthlib. 2023-11-15T02:53:39.155 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../074-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-11-15T02:53:39.225 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-11-15T02:53:39.448 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-blinker. 2023-11-15T02:53:39.467 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../073-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-11-15T02:53:39.506 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-11-15T02:53:39.646 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-11-15T02:53:39.666 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../075-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-11-15T02:53:39.703 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-11-15T02:53:39.842 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-oauthlib. 2023-11-15T02:53:39.860 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../074-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-11-15T02:53:39.900 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-11-15T02:53:40.074 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-websocket. 2023-11-15T02:53:40.091 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../076-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-11-15T02:53:40.131 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-11-15T02:53:40.296 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-11-15T02:53:40.313 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../075-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-11-15T02:53:40.353 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-11-15T02:53:40.493 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-kubernetes. 2023-11-15T02:53:40.511 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../077-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-11-15T02:53:40.551 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-11-15T02:53:40.731 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-websocket. 2023-11-15T02:53:40.751 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../076-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-11-15T02:53:40.789 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-11-15T02:53:41.193 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-kubernetes. 2023-11-15T02:53:41.215 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../077-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-11-15T02:53:41.243 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-11-15T02:53:41.259 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-11-15T02:53:41.265 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../078-ceph-mgr-k8sevents_16.2.14-170-g620f25c1-1focal_all.deb ... 2023-11-15T02:53:41.343 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-k8sevents (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:41.713 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libonig5:amd64. 2023-11-15T02:53:41.731 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../079-libonig5_6.9.4-1_amd64.deb ... 2023-11-15T02:53:41.770 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-11-15T02:53:41.908 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-11-15T02:53:41.929 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../078-ceph-mgr-k8sevents_16.2.14-170-g620f25c1-1focal_all.deb ... 2023-11-15T02:53:41.976 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-mgr-k8sevents (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:42.173 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libjq1:amd64. 2023-11-15T02:53:42.190 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../080-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-11-15T02:53:42.230 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-11-15T02:53:42.371 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libonig5:amd64. 2023-11-15T02:53:42.393 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../079-libonig5_6.9.4-1_amd64.deb ... 2023-11-15T02:53:42.437 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-11-15T02:53:42.644 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package jq. 2023-11-15T02:53:42.664 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../081-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-11-15T02:53:42.710 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-11-15T02:53:42.857 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libjq1:amd64. 2023-11-15T02:53:42.878 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../080-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-11-15T02:53:42.923 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-11-15T02:53:43.038 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package socat. 2023-11-15T02:53:43.060 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../082-socat_1.7.3.3-2_amd64.deb ... 2023-11-15T02:53:43.104 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking socat (1.7.3.3-2) ... 2023-11-15T02:53:43.361 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package jq. 2023-11-15T02:53:43.384 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../081-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-11-15T02:53:43.427 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-11-15T02:53:43.475 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package xmlstarlet. 2023-11-15T02:53:43.494 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../083-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-11-15T02:53:43.533 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-11-15T02:53:43.755 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package socat. 2023-11-15T02:53:43.777 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../082-socat_1.7.3.3-2_amd64.deb ... 2023-11-15T02:53:43.821 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking socat (1.7.3.3-2) ... 2023-11-15T02:53:43.971 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-test. 2023-11-15T02:53:43.992 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../084-ceph-test_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:44.045 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-test (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:44.184 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package xmlstarlet. 2023-11-15T02:53:44.205 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../083-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-11-15T02:53:44.249 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-11-15T02:53:44.698 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ceph-test. 2023-11-15T02:53:44.718 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../084-ceph-test_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:44.763 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ceph-test (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:48.200 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package cephfs-mirror. 2023-11-15T02:53:48.222 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../085-cephfs-mirror_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:48.266 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking cephfs-mirror (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:48.620 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-colorama. 2023-11-15T02:53:48.641 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../086-python3-colorama_0.4.3-1build1_all.deb ... 2023-11-15T02:53:48.719 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-11-15T02:53:48.993 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package cephfs-mirror. 2023-11-15T02:53:49.016 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../085-cephfs-mirror_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:49.016 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package cephfs-shell. 2023-11-15T02:53:49.036 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../087-cephfs-shell_16.2.14-170-g620f25c1-1focal_all.deb ... 2023-11-15T02:53:49.058 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking cephfs-mirror (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:49.107 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking cephfs-shell (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:49.393 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package cephfs-top. 2023-11-15T02:53:49.411 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../088-cephfs-top_16.2.14-170-g620f25c1-1focal_all.deb ... 2023-11-15T02:53:49.446 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-colorama. 2023-11-15T02:53:49.451 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking cephfs-top (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:49.465 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../086-python3-colorama_0.4.3-1build1_all.deb ... 2023-11-15T02:53:49.503 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-11-15T02:53:49.738 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libcephfs-dev. 2023-11-15T02:53:49.758 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../089-libcephfs-dev_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:49.782 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package cephfs-shell. 2023-11-15T02:53:49.795 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libcephfs-dev (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:49.804 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../087-cephfs-shell_16.2.14-170-g620f25c1-1focal_all.deb ... 2023-11-15T02:53:49.881 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking cephfs-shell (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:50.176 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package cephfs-top. 2023-11-15T02:53:50.197 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../088-cephfs-top_16.2.14-170-g620f25c1-1focal_all.deb ... 2023-11-15T02:53:50.242 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking cephfs-top (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:50.291 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package nvme-cli. 2023-11-15T02:53:50.314 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../090-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-11-15T02:53:50.387 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-11-15T02:53:50.545 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libcephfs-dev. 2023-11-15T02:53:50.564 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../089-libcephfs-dev_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:50.603 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libcephfs-dev (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:50.821 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-atomicwrites. 2023-11-15T02:53:50.841 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../091-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-11-15T02:53:50.887 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-11-15T02:53:51.040 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package nvme-cli. 2023-11-15T02:53:51.060 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../090-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-11-15T02:53:51.138 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-11-15T02:53:51.257 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-attr. 2023-11-15T02:53:51.282 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../092-python3-attr_19.3.0-2_all.deb ... 2023-11-15T02:53:51.323 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-11-15T02:53:51.578 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-atomicwrites. 2023-11-15T02:53:51.602 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../091-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-11-15T02:53:51.646 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-11-15T02:53:51.685 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-webencodings. 2023-11-15T02:53:51.705 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../093-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-11-15T02:53:51.751 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-11-15T02:53:51.989 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-attr. 2023-11-15T02:53:52.010 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../092-python3-attr_19.3.0-2_all.deb ... 2023-11-15T02:53:52.058 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-11-15T02:53:52.087 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-html5lib. 2023-11-15T02:53:52.101 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../094-python3-html5lib_1.0.1-2_all.deb ... 2023-11-15T02:53:52.146 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-11-15T02:53:52.450 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-webencodings. 2023-11-15T02:53:52.472 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../093-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-11-15T02:53:52.516 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-11-15T02:53:52.549 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-11-15T02:53:52.563 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../095-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-11-15T02:53:52.649 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-11-15T02:53:52.879 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-html5lib. 2023-11-15T02:53:52.901 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../094-python3-html5lib_1.0.1-2_all.deb ... 2023-11-15T02:53:52.945 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-11-15T02:53:53.152 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-packaging. 2023-11-15T02:53:53.173 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../096-python3-packaging_20.3-1_all.deb ... 2023-11-15T02:53:53.210 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-packaging (20.3-1) ... 2023-11-15T02:53:53.357 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-11-15T02:53:53.379 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../095-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-11-15T02:53:53.423 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-11-15T02:53:53.572 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pastescript. 2023-11-15T02:53:53.593 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../097-python3-pastescript_2.0.2-4_all.deb ... 2023-11-15T02:53:53.638 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-11-15T02:53:53.944 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-packaging. 2023-11-15T02:53:53.962 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../096-python3-packaging_20.3-1_all.deb ... 2023-11-15T02:53:54.001 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-packaging (20.3-1) ... 2023-11-15T02:53:54.042 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pluggy. 2023-11-15T02:53:54.065 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../098-python3-pluggy_0.13.0-2_all.deb ... 2023-11-15T02:53:54.108 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-11-15T02:53:54.313 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-pastescript. 2023-11-15T02:53:54.332 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../097-python3-pastescript_2.0.2-4_all.deb ... 2023-11-15T02:53:54.395 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-11-15T02:53:54.486 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-psutil. 2023-11-15T02:53:54.509 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../099-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-11-15T02:53:54.577 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-11-15T02:53:54.941 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-pluggy. 2023-11-15T02:53:54.962 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../098-python3-pluggy_0.13.0-2_all.deb ... 2023-11-15T02:53:54.964 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-py. 2023-11-15T02:53:54.985 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../100-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-11-15T02:53:55.024 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-11-15T02:53:55.030 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-11-15T02:53:55.386 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-psutil. 2023-11-15T02:53:55.407 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../099-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-11-15T02:53:55.417 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pyinotify. 2023-11-15T02:53:55.438 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../101-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-11-15T02:53:55.451 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-11-15T02:53:55.483 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-11-15T02:53:55.836 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pytest. 2023-11-15T02:53:55.847 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-py. 2023-11-15T02:53:55.858 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../102-python3-pytest_4.6.9-1_all.deb ... 2023-11-15T02:53:55.869 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../100-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-11-15T02:53:55.902 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-11-15T02:53:55.912 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-11-15T02:53:56.298 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-simplejson. 2023-11-15T02:53:56.308 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-pyinotify. 2023-11-15T02:53:56.318 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../103-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-11-15T02:53:56.326 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../101-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-11-15T02:53:56.356 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-11-15T02:53:56.365 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-11-15T02:53:56.721 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-pytest. 2023-11-15T02:53:56.739 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../102-python3-pytest_4.6.9-1_all.deb ... 2023-11-15T02:53:56.778 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-11-15T02:53:56.901 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package radosgw. 2023-11-15T02:53:56.921 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../104-radosgw_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:56.967 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking radosgw (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:57.191 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package python3-simplejson. 2023-11-15T02:53:57.211 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../103-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-11-15T02:53:57.256 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-11-15T02:53:57.727 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package radosgw. 2023-11-15T02:53:57.748 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../104-radosgw_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:57.793 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking radosgw (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:58.179 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package rbd-fuse. 2023-11-15T02:53:58.200 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../105-rbd-fuse_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:58.245 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking rbd-fuse (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:58.637 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-attr (19.3.0-2) ... 2023-11-15T02:53:59.014 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package rbd-fuse. 2023-11-15T02:53:59.034 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../105-rbd-fuse_16.2.14-170-g620f25c1-1focal_amd64.deb ... 2023-11-15T02:53:59.079 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking rbd-fuse (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:59.150 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2023-11-15T02:53:59.301 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-11-15T02:53:59.428 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-11-15T02:53:59.671 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up cephadm (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:53:59.731 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-attr (19.3.0-2) ... 2023-11-15T02:54:00.118 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2023-11-15T02:54:00.277 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-11-15T02:54:00.412 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-11-15T02:54:00.414 INFO:teuthology.orchestra.run.smithi190.stdout:Adding system user cephadm....done 2023-11-15T02:54:00.611 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-11-15T02:54:00.630 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up cephadm (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:01.055 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-11-15T02:54:01.414 INFO:teuthology.orchestra.run.smithi121.stdout:Adding system user cephadm....done 2023-11-15T02:54:01.440 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-11-15T02:54:01.591 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-11-15T02:54:01.637 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-11-15T02:54:01.993 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-11-15T02:54:02.098 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-11-15T02:54:02.448 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-11-15T02:54:02.469 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-11-15T02:54:02.616 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-11-15T02:54:02.820 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-11-15T02:54:03.010 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-11-15T02:54:03.180 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-11-15T02:54:03.537 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-11-15T02:54:03.542 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-11-15T02:54:03.886 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-11-15T02:54:03.896 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-11-15T02:54:04.239 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-11-15T02:54:04.254 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-ceph-argparse (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:04.621 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-11-15T02:54:04.624 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-11-15T02:54:04.748 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-11-15T02:54:04.958 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-11-15T02:54:05.090 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-11-15T02:54:05.319 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-ceph-argparse (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:05.450 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-11-15T02:54:05.681 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-11-15T02:54:05.806 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-11-15T02:54:05.809 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-11-15T02:54:06.158 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-11-15T02:54:06.277 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-11-15T02:54:06.500 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-11-15T02:54:06.620 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-11-15T02:54:06.893 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-11-15T02:54:07.013 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-11-15T02:54:07.372 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libjaeger (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:07.386 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-11-15T02:54:07.498 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-11-15T02:54:07.753 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-11-15T02:54:07.917 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-11-15T02:54:08.179 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-11-15T02:54:08.276 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-packaging (20.3-1) ... 2023-11-15T02:54:08.547 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libjaeger (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:08.644 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-11-15T02:54:08.674 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-11-15T02:54:08.770 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-11-15T02:54:09.091 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-11-15T02:54:09.146 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-11-15T02:54:09.443 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-packaging (20.3-1) ... 2023-11-15T02:54:09.651 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-11-15T02:54:09.803 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-11-15T02:54:09.928 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-11-15T02:54:10.036 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-11-15T02:54:10.305 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-11-15T02:54:10.479 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-11-15T02:54:10.781 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-11-15T02:54:10.931 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up socat (1.7.3.3-2) ... 2023-11-15T02:54:11.057 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-ceph-common (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:11.157 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-11-15T02:54:11.461 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-11-15T02:54:11.629 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-11-15T02:54:11.935 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-11-15T02:54:12.063 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-11-15T02:54:12.073 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up socat (1.7.3.3-2) ... 2023-11-15T02:54:12.189 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-11-15T02:54:12.199 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-ceph-common (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:12.565 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-11-15T02:54:12.584 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-11-15T02:54:12.950 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-11-15T02:54:13.027 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-11-15T02:54:13.101 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-11-15T02:54:13.155 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-11-15T02:54:13.321 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-11-15T02:54:13.511 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-11-15T02:54:13.683 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-11-15T02:54:13.862 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-11-15T02:54:14.101 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-11-15T02:54:14.252 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-11-15T02:54:14.263 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-11-15T02:54:14.389 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-11-15T02:54:14.662 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-11-15T02:54:14.741 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-11-15T02:54:14.980 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-11-15T02:54:15.100 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-11-15T02:54:15.356 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-11-15T02:54:15.467 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-11-15T02:54:15.506 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-11-15T02:54:15.846 INFO:teuthology.orchestra.run.smithi190.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-11-15T02:54:15.857 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-11-15T02:54:16.212 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-11-15T02:54:16.234 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-11-15T02:54:16.337 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-11-15T02:54:16.593 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-11-15T02:54:16.731 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-11-15T02:54:16.961 INFO:teuthology.orchestra.run.smithi121.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-11-15T02:54:17.149 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-11-15T02:54:17.258 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-11-15T02:54:17.321 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-11-15T02:54:17.409 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-11-15T02:54:17.447 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-11-15T02:54:17.818 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-11-15T02:54:17.856 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-11-15T02:54:18.170 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librados2 (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:18.295 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-11-15T02:54:18.299 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-11-15T02:54:18.425 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-11-15T02:54:18.551 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-11-15T02:54:18.986 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-11-15T02:54:19.370 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up librados2 (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:19.496 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-11-15T02:54:20.105 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librgw2 (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:20.231 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libsqlite3-mod-ceph (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:20.340 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-11-15T02:54:20.483 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-11-15T02:54:20.943 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-11-15T02:54:21.329 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-11-15T02:54:21.369 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up librgw2 (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:21.497 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libsqlite3-mod-ceph (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:21.624 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-11-15T02:54:21.673 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-11-15T02:54:21.789 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-11-15T02:54:22.083 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libcephfs2 (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:22.209 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-11-15T02:54:22.260 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-11-15T02:54:22.611 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libradosstriper1 (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:22.619 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-11-15T02:54:22.762 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-11-15T02:54:23.012 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-11-15T02:54:23.171 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-11-15T02:54:23.446 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libcephfs2 (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:23.627 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-11-15T02:54:24.061 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libradosstriper1 (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:24.187 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-11-15T02:54:24.446 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librbd1 (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:24.555 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-11-15T02:54:24.605 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-11-15T02:54:24.715 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-11-15T02:54:24.841 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rsa (4.0-3) ... 2023-11-15T02:54:25.209 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-11-15T02:54:25.576 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-modules-core (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:25.702 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-fuse (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:25.871 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up librbd1 (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:25.994 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-15T02:54:25.995 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-15T02:54:25.997 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-11-15T02:54:26.123 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-11-15T02:54:26.265 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-rsa (4.0-3) ... 2023-11-15T02:54:26.658 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-11-15T02:54:26.862 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libcephfs-dev (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:27.021 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-11-15T02:54:27.059 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-mgr-modules-core (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:27.211 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-fuse (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:27.381 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-11-15T02:54:27.474 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-15T02:54:27.475 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-15T02:54:27.757 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-11-15T02:54:28.209 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rados (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:28.328 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libcephfs-dev (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:28.335 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up cephfs-top (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:28.454 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-11-15T02:54:28.597 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rbd (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:28.723 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up rbd-fuse (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:28.855 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-11-15T02:54:28.895 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-routes (2.4.1-2) ... 2023-11-15T02:54:29.240 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-11-15T02:54:29.292 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-11-15T02:54:29.684 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-rados (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:29.810 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up cephfs-top (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:29.903 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rgw (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:30.028 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cephfs (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:30.313 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-rbd (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:30.405 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up cephfs-shell (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:30.464 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up rbd-fuse (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:30.581 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-11-15T02:54:30.590 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-routes (2.4.1-2) ... 2023-11-15T02:54:30.950 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-11-15T02:54:31.328 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-rgw (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:31.453 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-cephfs (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:31.726 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-common (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:31.804 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up cephfs-shell (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:31.930 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-11-15T02:54:32.376 INFO:teuthology.orchestra.run.smithi190.stdout:Adding group ceph....done 2023-11-15T02:54:32.840 INFO:teuthology.orchestra.run.smithi190.stdout:Adding system user ceph....done 2023-11-15T02:54:33.069 INFO:teuthology.orchestra.run.smithi190.stdout:Setting system user ceph properties....done 2023-11-15T02:54:33.102 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-common (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:33.105 INFO:teuthology.orchestra.run.smithi190.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-11-15T02:54:33.307 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-11-15T02:54:33.706 INFO:teuthology.orchestra.run.smithi121.stdout:Adding group ceph....done 2023-11-15T02:54:33.861 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-11-15T02:54:34.181 INFO:teuthology.orchestra.run.smithi121.stdout:Adding system user ceph....done 2023-11-15T02:54:34.428 INFO:teuthology.orchestra.run.smithi121.stdout:Setting system user ceph properties....done 2023-11-15T02:54:34.463 INFO:teuthology.orchestra.run.smithi121.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-11-15T02:54:34.671 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-11-15T02:54:34.779 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up cephfs-mirror (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:35.027 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-15T02:54:35.027 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-15T02:54:35.192 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-11-15T02:54:35.872 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-test (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:35.998 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up radosgw (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:36.061 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up cephfs-mirror (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:36.320 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-15T02:54:36.321 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-15T02:54:36.343 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-15T02:54:36.344 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-15T02:54:37.187 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-test (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:37.313 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up radosgw (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:37.567 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-base (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:37.634 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-15T02:54:37.634 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-15T02:54:38.087 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-11-15T02:54:38.823 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-base (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:39.002 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mds (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:39.276 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-15T02:54:39.276 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-15T02:54:39.358 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-11-15T02:54:40.120 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:40.233 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-mds (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:40.397 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-15T02:54:40.398 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-15T02:54:40.496 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-15T02:54:40.496 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-15T02:54:41.280 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-osd (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:41.360 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-mgr (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:41.630 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-15T02:54:41.631 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-15T02:54:42.083 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-15T02:54:42.084 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-15T02:54:42.494 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-osd (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:42.923 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-k8sevents (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:43.049 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-diskprediction-local (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:43.233 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mon (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:43.357 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-15T02:54:43.357 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-15T02:54:43.498 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-15T02:54:43.499 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-15T02:54:44.204 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-mgr-k8sevents (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:44.354 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-mgr-diskprediction-local (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:44.368 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-cephadm (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:44.505 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-mon (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:44.519 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:44.645 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-dashboard (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:44.760 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-15T02:54:44.760 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-15T02:54:44.841 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-11-15T02:54:45.333 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-11-15T02:54:45.549 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-mgr-cephadm (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:45.809 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:45.935 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ceph-mgr-dashboard (16.2.14-170-g620f25c1-1focal) ... 2023-11-15T02:54:46.104 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-11-15T02:54:46.596 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-11-15T02:54:50.620 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-11-15T02:54:51.235 INFO:teuthology.orchestra.run.smithi190.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-15T02:54:51.241 DEBUG:teuthology.orchestra.run.smithi190:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2023-11-15T02:54:51.367 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2023-11-15T02:54:51.570 INFO:teuthology.orchestra.run.smithi190.stdout:Building dependency tree... 2023-11-15T02:54:51.571 INFO:teuthology.orchestra.run.smithi190.stdout:Reading state information... 2023-11-15T02:54:51.770 INFO:teuthology.orchestra.run.smithi190.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-11-15T02:54:51.771 INFO:teuthology.orchestra.run.smithi190.stdout:flex is already the newest version (2.6.4-6.2). 2023-11-15T02:54:51.771 INFO:teuthology.orchestra.run.smithi190.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-11-15T02:54:51.771 INFO:teuthology.orchestra.run.smithi190.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-11-15T02:54:51.771 INFO:teuthology.orchestra.run.smithi190.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-11-15T02:54:51.772 INFO:teuthology.orchestra.run.smithi190.stdout:The following packages were automatically installed and are no longer required: 2023-11-15T02:54:51.773 INFO:teuthology.orchestra.run.smithi190.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-11-15T02:54:51.773 INFO:teuthology.orchestra.run.smithi190.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-15T02:54:51.773 INFO:teuthology.orchestra.run.smithi190.stdout:The following additional packages will be installed: 2023-11-15T02:54:51.773 INFO:teuthology.orchestra.run.smithi190.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-11-15T02:54:51.773 INFO:teuthology.orchestra.run.smithi190.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2023-11-15T02:54:51.774 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2023-11-15T02:54:51.774 INFO:teuthology.orchestra.run.smithi190.stdout: usb-modeswitch-data wpasupplicant 2023-11-15T02:54:51.774 INFO:teuthology.orchestra.run.smithi190.stdout:Suggested packages: 2023-11-15T02:54:51.775 INFO:teuthology.orchestra.run.smithi190.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2023-11-15T02:54:51.775 INFO:teuthology.orchestra.run.smithi190.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2023-11-15T02:54:51.799 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-11-15T02:54:51.862 INFO:teuthology.orchestra.run.smithi190.stdout:The following NEW packages will be installed: 2023-11-15T02:54:51.863 INFO:teuthology.orchestra.run.smithi190.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-11-15T02:54:51.863 INFO:teuthology.orchestra.run.smithi190.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2023-11-15T02:54:51.864 INFO:teuthology.orchestra.run.smithi190.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-11-15T02:54:51.864 INFO:teuthology.orchestra.run.smithi190.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-15T02:54:51.865 INFO:teuthology.orchestra.run.smithi190.stdout:The following packages will be upgraded: 2023-11-15T02:54:51.866 INFO:teuthology.orchestra.run.smithi190.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-11-15T02:54:52.065 INFO:teuthology.orchestra.run.smithi190.stdout:6 upgraded, 23 newly installed, 0 to remove and 254 not upgraded. 2023-11-15T02:54:52.065 INFO:teuthology.orchestra.run.smithi190.stdout:Need to get 10.4 MB of archives. 2023-11-15T02:54:52.066 INFO:teuthology.orchestra.run.smithi190.stdout:After this operation, 33.5 MB of additional disk space will be used. 2023-11-15T02:54:52.066 INFO:teuthology.orchestra.run.smithi190.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-11-15T02:54:52.469 INFO:teuthology.orchestra.run.smithi121.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-15T02:54:52.475 DEBUG:teuthology.orchestra.run.smithi121:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2023-11-15T02:54:52.608 INFO:teuthology.orchestra.run.smithi121.stdout:Reading package lists... 2023-11-15T02:54:52.707 INFO:teuthology.orchestra.run.smithi190.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-11-15T02:54:52.708 INFO:teuthology.orchestra.run.smithi190.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-11-15T02:54:52.710 INFO:teuthology.orchestra.run.smithi190.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-11-15T02:54:52.722 INFO:teuthology.orchestra.run.smithi190.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-11-15T02:54:52.725 INFO:teuthology.orchestra.run.smithi190.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.20 [1,321 kB] 2023-11-15T02:54:52.816 INFO:teuthology.orchestra.run.smithi121.stdout:Building dependency tree... 2023-11-15T02:54:52.817 INFO:teuthology.orchestra.run.smithi121.stdout:Reading state information... 2023-11-15T02:54:52.825 INFO:teuthology.orchestra.run.smithi190.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-11-15T02:54:52.827 INFO:teuthology.orchestra.run.smithi190.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-11-15T02:54:52.834 INFO:teuthology.orchestra.run.smithi190.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-11-15T02:54:52.839 INFO:teuthology.orchestra.run.smithi190.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-11-15T02:54:52.840 INFO:teuthology.orchestra.run.smithi190.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-11-15T02:54:53.007 INFO:teuthology.orchestra.run.smithi190.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-11-15T02:54:53.014 INFO:teuthology.orchestra.run.smithi121.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-11-15T02:54:53.014 INFO:teuthology.orchestra.run.smithi121.stdout:flex is already the newest version (2.6.4-6.2). 2023-11-15T02:54:53.014 INFO:teuthology.orchestra.run.smithi121.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-11-15T02:54:53.014 INFO:teuthology.orchestra.run.smithi121.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-11-15T02:54:53.015 INFO:teuthology.orchestra.run.smithi121.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-11-15T02:54:53.015 INFO:teuthology.orchestra.run.smithi121.stdout:The following packages were automatically installed and are no longer required: 2023-11-15T02:54:53.016 INFO:teuthology.orchestra.run.smithi121.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-11-15T02:54:53.016 INFO:teuthology.orchestra.run.smithi121.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-15T02:54:53.016 INFO:teuthology.orchestra.run.smithi121.stdout:The following additional packages will be installed: 2023-11-15T02:54:53.017 INFO:teuthology.orchestra.run.smithi121.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-11-15T02:54:53.017 INFO:teuthology.orchestra.run.smithi121.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2023-11-15T02:54:53.017 INFO:teuthology.orchestra.run.smithi121.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2023-11-15T02:54:53.017 INFO:teuthology.orchestra.run.smithi121.stdout: usb-modeswitch-data wpasupplicant 2023-11-15T02:54:53.018 INFO:teuthology.orchestra.run.smithi121.stdout:Suggested packages: 2023-11-15T02:54:53.018 INFO:teuthology.orchestra.run.smithi121.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2023-11-15T02:54:53.018 INFO:teuthology.orchestra.run.smithi121.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2023-11-15T02:54:53.106 INFO:teuthology.orchestra.run.smithi121.stdout:The following NEW packages will be installed: 2023-11-15T02:54:53.107 INFO:teuthology.orchestra.run.smithi121.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-11-15T02:54:53.107 INFO:teuthology.orchestra.run.smithi121.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2023-11-15T02:54:53.108 INFO:teuthology.orchestra.run.smithi121.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-11-15T02:54:53.108 INFO:teuthology.orchestra.run.smithi121.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-15T02:54:53.109 INFO:teuthology.orchestra.run.smithi121.stdout:The following packages will be upgraded: 2023-11-15T02:54:53.110 INFO:teuthology.orchestra.run.smithi121.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-11-15T02:54:53.182 INFO:teuthology.orchestra.run.smithi121.stdout:6 upgraded, 23 newly installed, 0 to remove and 254 not upgraded. 2023-11-15T02:54:53.183 INFO:teuthology.orchestra.run.smithi121.stdout:Need to get 10.4 MB of archives. 2023-11-15T02:54:53.183 INFO:teuthology.orchestra.run.smithi121.stdout:After this operation, 33.5 MB of additional disk space will be used. 2023-11-15T02:54:53.183 INFO:teuthology.orchestra.run.smithi121.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-11-15T02:54:53.279 INFO:teuthology.orchestra.run.smithi190.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-11-15T02:54:53.346 INFO:teuthology.orchestra.run.smithi121.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-11-15T02:54:53.348 INFO:teuthology.orchestra.run.smithi121.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-11-15T02:54:53.350 INFO:teuthology.orchestra.run.smithi121.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-11-15T02:54:53.357 INFO:teuthology.orchestra.run.smithi121.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-11-15T02:54:53.358 INFO:teuthology.orchestra.run.smithi121.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.20 [1,321 kB] 2023-11-15T02:54:53.389 INFO:teuthology.orchestra.run.smithi121.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-11-15T02:54:53.390 INFO:teuthology.orchestra.run.smithi121.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-11-15T02:54:53.392 INFO:teuthology.orchestra.run.smithi121.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-11-15T02:54:53.393 INFO:teuthology.orchestra.run.smithi121.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-11-15T02:54:53.394 INFO:teuthology.orchestra.run.smithi121.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-11-15T02:54:53.396 INFO:teuthology.orchestra.run.smithi121.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-11-15T02:54:53.398 INFO:teuthology.orchestra.run.smithi121.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-11-15T02:54:53.401 INFO:teuthology.orchestra.run.smithi121.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-11-15T02:54:53.402 INFO:teuthology.orchestra.run.smithi121.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-11-15T02:54:53.418 INFO:teuthology.orchestra.run.smithi121.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-11-15T02:54:53.419 INFO:teuthology.orchestra.run.smithi121.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-11-15T02:54:53.420 INFO:teuthology.orchestra.run.smithi190.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-11-15T02:54:53.426 INFO:teuthology.orchestra.run.smithi190.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-11-15T02:54:53.428 INFO:teuthology.orchestra.run.smithi121.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-11-15T02:54:53.439 INFO:teuthology.orchestra.run.smithi121.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-11-15T02:54:53.440 INFO:teuthology.orchestra.run.smithi121.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.20 [1,584 kB] 2023-11-15T02:54:53.459 INFO:teuthology.orchestra.run.smithi121.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-11-15T02:54:53.459 INFO:teuthology.orchestra.run.smithi121.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-11-15T02:54:53.469 INFO:teuthology.orchestra.run.smithi121.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-11-15T02:54:53.487 INFO:teuthology.orchestra.run.smithi121.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-11-15T02:54:53.511 INFO:teuthology.orchestra.run.smithi121.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-11-15T02:54:53.514 INFO:teuthology.orchestra.run.smithi121.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-11-15T02:54:53.515 INFO:teuthology.orchestra.run.smithi121.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-11-15T02:54:53.516 INFO:teuthology.orchestra.run.smithi121.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-11-15T02:54:53.516 INFO:teuthology.orchestra.run.smithi121.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-11-15T02:54:53.529 INFO:teuthology.orchestra.run.smithi190.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-11-15T02:54:53.533 INFO:teuthology.orchestra.run.smithi190.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-11-15T02:54:53.613 INFO:teuthology.orchestra.run.smithi190.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-11-15T02:54:53.688 INFO:teuthology.orchestra.run.smithi190.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-11-15T02:54:53.689 INFO:teuthology.orchestra.run.smithi190.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.20 [1,584 kB] 2023-11-15T02:54:53.778 INFO:teuthology.orchestra.run.smithi190.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-11-15T02:54:53.778 INFO:teuthology.orchestra.run.smithi190.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-11-15T02:54:53.952 INFO:teuthology.orchestra.run.smithi121.stdout:Preconfiguring packages ... 2023-11-15T02:54:53.971 INFO:teuthology.orchestra.run.smithi190.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-11-15T02:54:54.130 INFO:teuthology.orchestra.run.smithi121.stdout:Fetched 10.4 MB in 0s (25.8 MB/s) 2023-11-15T02:54:54.442 INFO:teuthology.orchestra.run.smithi121.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 115279 files and directories currently installed.) 2023-11-15T02:54:54.446 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-11-15T02:54:54.631 INFO:teuthology.orchestra.run.smithi190.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-11-15T02:54:54.648 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-11-15T02:54:54.745 INFO:teuthology.orchestra.run.smithi190.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-11-15T02:54:54.760 INFO:teuthology.orchestra.run.smithi190.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-11-15T02:54:54.762 INFO:teuthology.orchestra.run.smithi190.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-11-15T02:54:54.763 INFO:teuthology.orchestra.run.smithi190.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-11-15T02:54:54.765 INFO:teuthology.orchestra.run.smithi190.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-11-15T02:54:55.203 INFO:teuthology.orchestra.run.smithi190.stdout:Preconfiguring packages ... 2023-11-15T02:54:55.425 INFO:teuthology.orchestra.run.smithi190.stdout:Fetched 10.4 MB in 3s (3,610 kB/s) 2023-11-15T02:54:55.785 INFO:teuthology.orchestra.run.smithi190.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 115279 files and directories currently installed.) 2023-11-15T02:54:55.788 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-11-15T02:54:56.006 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-11-15T02:54:56.292 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-11-15T02:54:57.669 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-11-15T02:54:58.313 INFO:teuthology.orchestra.run.smithi121.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 115279 files and directories currently installed.) 2023-11-15T02:54:58.317 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-11-15T02:54:58.472 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-11-15T02:54:58.958 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-11-15T02:54:59.161 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-11-15T02:54:59.545 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-11-15T02:54:59.735 INFO:teuthology.orchestra.run.smithi190.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 115279 files and directories currently installed.) 2023-11-15T02:54:59.739 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-11-15T02:54:59.917 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-11-15T02:55:00.067 INFO:teuthology.orchestra.run.smithi121.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 115279 files and directories currently installed.) 2023-11-15T02:55:00.071 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-11-15T02:55:00.212 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-11-15T02:55:00.742 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-11-15T02:55:00.758 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-11-15T02:55:00.903 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-11-15T02:55:00.965 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-11-15T02:55:01.361 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-11-15T02:55:01.883 INFO:teuthology.orchestra.run.smithi190.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 115279 files and directories currently installed.) 2023-11-15T02:55:01.887 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-11-15T02:55:01.919 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.20_amd64.deb ... 2023-11-15T02:55:02.023 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-11-15T02:55:02.059 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.20) over (1.1.1f-1ubuntu2.16) ... 2023-11-15T02:55:02.554 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-11-15T02:55:02.621 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-11-15T02:55:02.640 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-11-15T02:55:02.679 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-11-15T02:55:02.712 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-11-15T02:55:03.090 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package dump. 2023-11-15T02:55:03.109 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-11-15T02:55:03.148 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking dump (0.4b46-6) ... 2023-11-15T02:55:03.248 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.20_amd64.deb ... 2023-11-15T02:55:03.416 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.20) over (1.1.1f-1ubuntu2.16) ... 2023-11-15T02:55:03.494 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package indent. 2023-11-15T02:55:03.514 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-11-15T02:55:03.552 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-11-15T02:55:03.897 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-11-15T02:55:03.918 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-11-15T02:55:03.955 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-11-15T02:55:03.978 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-11-15T02:55:04.003 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-11-15T02:55:04.045 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-11-15T02:55:04.342 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-11-15T02:55:04.365 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-11-15T02:55:04.408 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-11-15T02:55:04.431 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package dump. 2023-11-15T02:55:04.452 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-11-15T02:55:04.498 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking dump (0.4b46-6) ... 2023-11-15T02:55:04.753 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-11-15T02:55:04.777 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-11-15T02:55:04.819 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-11-15T02:55:04.843 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package indent. 2023-11-15T02:55:04.861 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-11-15T02:55:04.901 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-11-15T02:55:05.239 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-11-15T02:55:05.260 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-11-15T02:55:05.270 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-11-15T02:55:05.292 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-11-15T02:55:05.305 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-11-15T02:55:05.337 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-11-15T02:55:05.642 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libmbim-proxy. 2023-11-15T02:55:05.660 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-11-15T02:55:05.700 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-11-15T02:55:05.892 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-11-15T02:55:05.906 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-11-15T02:55:05.950 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-11-15T02:55:06.086 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-11-15T02:55:06.107 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-11-15T02:55:06.152 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-11-15T02:55:06.262 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-11-15T02:55:06.281 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-11-15T02:55:06.320 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-11-15T02:55:06.705 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libndp0:amd64. 2023-11-15T02:55:06.726 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-11-15T02:55:06.731 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-11-15T02:55:06.754 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-11-15T02:55:06.765 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-11-15T02:55:06.798 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-11-15T02:55:07.142 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libmbim-proxy. 2023-11-15T02:55:07.160 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libnm0:amd64. 2023-11-15T02:55:07.161 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-11-15T02:55:07.181 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-11-15T02:55:07.209 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-11-15T02:55:07.259 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-11-15T02:55:07.603 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-11-15T02:55:07.627 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-11-15T02:55:07.669 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-11-15T02:55:07.712 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-11-15T02:55:07.731 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-11-15T02:55:07.770 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-11-15T02:55:08.122 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libndp0:amd64. 2023-11-15T02:55:08.144 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-11-15T02:55:08.173 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libqmi-proxy. 2023-11-15T02:55:08.195 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-11-15T02:55:08.239 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-11-15T02:55:08.240 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-11-15T02:55:08.526 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-11-15T02:55:08.547 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.20_amd64.deb ... 2023-11-15T02:55:08.625 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.20) ... 2023-11-15T02:55:08.633 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libnm0:amd64. 2023-11-15T02:55:08.657 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-11-15T02:55:08.700 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-11-15T02:55:09.153 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-11-15T02:55:09.171 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-11-15T02:55:09.171 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-11-15T02:55:09.189 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2023-11-15T02:55:09.211 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-11-15T02:55:09.229 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-11-15T02:55:09.622 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libqmi-proxy. 2023-11-15T02:55:09.641 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-11-15T02:55:09.674 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package modemmanager. 2023-11-15T02:55:09.680 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-11-15T02:55:09.696 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-11-15T02:55:09.773 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-11-15T02:55:09.999 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-11-15T02:55:10.018 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.20_amd64.deb ... 2023-11-15T02:55:10.057 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.20) ... 2023-11-15T02:55:10.471 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package wpasupplicant. 2023-11-15T02:55:10.492 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-11-15T02:55:10.538 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-11-15T02:55:10.627 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-11-15T02:55:10.649 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2023-11-15T02:55:10.694 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-11-15T02:55:11.152 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package network-manager. 2023-11-15T02:55:11.174 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-11-15T02:55:11.271 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-11-15T02:55:11.289 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package modemmanager. 2023-11-15T02:55:11.310 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-11-15T02:55:11.377 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-11-15T02:55:12.070 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package wpasupplicant. 2023-11-15T02:55:12.084 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package ppp. 2023-11-15T02:55:12.091 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-11-15T02:55:12.105 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-11-15T02:55:12.136 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-11-15T02:55:12.646 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-11-15T02:55:12.750 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package network-manager. 2023-11-15T02:55:12.766 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-11-15T02:55:12.851 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-11-15T02:55:13.038 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package pptp-linux. 2023-11-15T02:55:13.054 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-11-15T02:55:13.121 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-11-15T02:55:13.550 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package network-manager-pptp. 2023-11-15T02:55:13.570 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-11-15T02:55:13.641 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-11-15T02:55:13.723 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ppp. 2023-11-15T02:55:13.744 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-11-15T02:55:13.936 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-11-15T02:55:13.957 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2023-11-15T02:55:14.011 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-11-15T02:55:14.227 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-11-15T02:55:14.465 INFO:teuthology.orchestra.run.smithi121.stdout:Selecting previously unselected package usb-modeswitch. 2023-11-15T02:55:14.482 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-11-15T02:55:14.545 INFO:teuthology.orchestra.run.smithi121.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-11-15T02:55:14.678 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package pptp-linux. 2023-11-15T02:55:14.700 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-11-15T02:55:14.744 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-11-15T02:55:14.915 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.20) ... 2023-11-15T02:55:15.173 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package network-manager-pptp. 2023-11-15T02:55:15.191 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-11-15T02:55:15.290 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-11-15T02:55:15.328 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-11-15T02:55:15.454 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-11-15T02:55:15.567 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-11-15T02:55:15.580 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-11-15T02:55:15.589 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2023-11-15T02:55:15.658 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-11-15T02:55:16.050 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-15T02:55:16.050 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-15T02:55:16.104 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package usb-modeswitch. 2023-11-15T02:55:16.125 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-11-15T02:55:16.189 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-11-15T02:55:16.573 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.20) ... 2023-11-15T02:55:16.934 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-11-15T02:55:16.983 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-11-15T02:55:17.060 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-11-15T02:55:17.126 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-11-15T02:55:17.219 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-11-15T02:55:17.252 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.20) ... 2023-11-15T02:55:17.378 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-11-15T02:55:17.504 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-11-15T02:55:17.657 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-15T02:55:17.658 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-15T02:55:17.663 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-11-15T02:55:18.563 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-11-15T02:55:18.714 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-11-15T02:55:18.840 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.20) ... 2023-11-15T02:55:18.897 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-11-15T02:55:18.966 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-11-15T02:55:19.117 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-11-15T02:55:19.243 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-11-15T02:55:19.310 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up dump (0.4b46-6) ... 2023-11-15T02:55:19.443 INFO:teuthology.orchestra.run.smithi121.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-11-15T02:55:19.512 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-11-15T02:55:19.630 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-11-15T02:55:19.756 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-11-15T02:55:19.915 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-11-15T02:55:20.108 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-11-15T02:55:20.259 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-11-15T02:55:20.385 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-11-15T02:55:20.511 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-11-15T02:55:20.528 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-11-15T02:55:20.704 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-11-15T02:55:20.839 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-11-15T02:55:20.941 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up dump (0.4b46-6) ... 2023-11-15T02:55:21.306 INFO:teuthology.orchestra.run.smithi190.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-11-15T02:55:21.551 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-11-15T02:55:21.677 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-11-15T02:55:21.803 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-11-15T02:55:21.962 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-11-15T02:55:21.990 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:21.991 INFO:teuthology.orchestra.run.smithi121.stdout:The following network interfaces were found in /etc/network/interfaces 2023-11-15T02:55:21.991 INFO:teuthology.orchestra.run.smithi121.stdout:which means they are currently configured by ifupdown: 2023-11-15T02:55:21.991 INFO:teuthology.orchestra.run.smithi121.stdout:- enp3s0f1 2023-11-15T02:55:21.991 INFO:teuthology.orchestra.run.smithi121.stdout:If you want to manage those interfaces with NetworkManager instead 2023-11-15T02:55:21.992 INFO:teuthology.orchestra.run.smithi121.stdout:remove their configuration from /etc/network/interfaces. 2023-11-15T02:55:21.992 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:22.155 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-11-15T02:55:22.224 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-11-15T02:55:22.331 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-11-15T02:55:22.457 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-11-15T02:55:22.604 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-11-15T02:55:22.749 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-11-15T02:55:22.776 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-11-15T02:55:22.902 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-11-15T02:55:23.693 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-11-15T02:55:24.111 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:24.112 INFO:teuthology.orchestra.run.smithi190.stdout:The following network interfaces were found in /etc/network/interfaces 2023-11-15T02:55:24.112 INFO:teuthology.orchestra.run.smithi190.stdout:which means they are currently configured by ifupdown: 2023-11-15T02:55:24.112 INFO:teuthology.orchestra.run.smithi190.stdout:- enp3s0f1 2023-11-15T02:55:24.112 INFO:teuthology.orchestra.run.smithi190.stdout:If you want to manage those interfaces with NetworkManager instead 2023-11-15T02:55:24.112 INFO:teuthology.orchestra.run.smithi190.stdout:remove their configuration from /etc/network/interfaces. 2023-11-15T02:55:24.113 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:24.346 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-11-15T02:55:24.667 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-11-15T02:55:24.835 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-11-15T02:55:24.909 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-11-15T02:55:25.002 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-11-15T02:55:25.128 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-11-15T02:55:25.279 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-11-15T02:55:25.405 INFO:teuthology.orchestra.run.smithi121.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-11-15T02:55:25.724 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-11-15T02:55:25.724 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-11-15T02:55:25.835 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-11-15T02:55:26.666 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-11-15T02:55:26.765 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-11-15T02:55:26.784 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-11-15T02:55:26.918 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-11-15T02:55:26.932 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-11-15T02:55:27.091 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-11-15T02:55:27.217 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-11-15T02:55:27.377 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-11-15T02:55:27.462 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-11-15T02:55:27.502 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-11-15T02:55:27.823 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-11-15T02:55:27.823 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-11-15T02:55:27.946 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-11-15T02:55:28.780 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-11-15T02:55:28.914 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-11-15T02:55:29.023 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-11-15T02:55:29.217 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-11-15T02:55:29.693 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-11-15T02:55:34.842 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-11-15T02:55:34.965 INFO:teuthology.orchestra.run.smithi121.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-11-15T02:55:36.173 INFO:teuthology.orchestra.run.smithi121.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-15T02:55:36.179 DEBUG:teuthology.parallel:result is None 2023-11-15T02:55:36.480 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-11-15T02:55:36.629 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-11-15T02:55:37.895 INFO:teuthology.orchestra.run.smithi190.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-15T02:55:37.901 DEBUG:teuthology.parallel:result is None 2023-11-15T02:55:37.902 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=620f25c1794f62207424fc9319427943fb8d1fae 2023-11-15T02:55:38.145 DEBUG:teuthology.orchestra.run.smithi121:> dpkg-query -W -f '${Version}' ceph 2023-11-15T02:55:38.182 INFO:teuthology.orchestra.run.smithi121.stdout:16.2.14-170-g620f25c1-1focal 2023-11-15T02:55:38.182 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-170-g620f25c1-1focal 2023-11-15T02:55:38.183 INFO:teuthology.task.install:The correct ceph version 16.2.14-170-g620f25c1-1focal is installed. 2023-11-15T02:55:38.183 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=620f25c1794f62207424fc9319427943fb8d1fae 2023-11-15T02:55:38.369 DEBUG:teuthology.orchestra.run.smithi190:> dpkg-query -W -f '${Version}' ceph 2023-11-15T02:55:38.406 INFO:teuthology.orchestra.run.smithi190.stdout:16.2.14-170-g620f25c1-1focal 2023-11-15T02:55:38.406 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-170-g620f25c1-1focal 2023-11-15T02:55:38.407 INFO:teuthology.task.install:The correct ceph version 16.2.14-170-g620f25c1-1focal is installed. 2023-11-15T02:55:38.407 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-15T02:55:38.408 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:55:38.408 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-15T02:55:38.445 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:55:38.446 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-15T02:55:38.489 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-15T02:55:38.490 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:55:38.490 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/daemon-helper 2023-11-15T02:55:38.530 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-15T02:55:38.609 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:55:38.609 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2023-11-15T02:55:38.644 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-15T02:55:38.727 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-15T02:55:38.728 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:55:38.728 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-15T02:55:38.761 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-15T02:55:38.837 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:55:38.838 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-15T02:55:38.872 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-15T02:55:38.949 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-15T02:55:38.950 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:55:38.950 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/stdin-killer 2023-11-15T02:55:38.985 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-15T02:55:39.065 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:55:39.065 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2023-11-15T02:55:39.102 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-15T02:55:39.180 INFO:teuthology.run_tasks:Running task ceph... 2023-11-15T02:55:39.258 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-15T02:55:39.259 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 777 /var/log/ceph 2023-11-15T02:55:39.266 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /var/log/ceph 2023-11-15T02:55:39.305 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-15T02:55:39.306 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-15T02:55:39.342 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-15T02:55:39.384 INFO:tasks.ceph:Creating extra log directories... 2023-11-15T02:55:39.384 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-15T02:55:39.422 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-15T02:55:39.473 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-15T02:55:39.474 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-15T02:55:39.474 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-15T02:55:39.475 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:55:39.475 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-15T02:55:39.519 DEBUG:teuthology.orchestra.run.smithi121:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-15T02:55:39.643 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:55:39.643 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-15T02:55:39.664 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-15T02:55:39.792 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-15T02:55:39.792 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-15T02:55:39.792 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7458153', 'branch': 'wip-yuri4-testing-2023-11-13-0820-pacific', 'description': 'fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7458153', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri4-testing-2023-11-13-0820-pacific'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds'], 'sha1': '620f25c1794f62207424fc9319427943fb8d1fae'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '620f25c1794f62207424fc9319427943fb8d1fae'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri4-testing-2023-11-13-0820-pacific', 'sha1': '620f25c1794f62207424fc9319427943fb8d1fae'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5345, 'sha1': '620f25c1794f62207424fc9319427943fb8d1fae', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri4-testing-2023-11-13-0820-pacific', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_620f25c1794f62207424fc9319427943fb8d1fae/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '620f25c1794f62207424fc9319427943fb8d1fae', 'targets': {'smithi121.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi190.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '620f25c1794f62207424fc9319427943fb8d1fae'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_damage']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '6899cd26fceddb2fec83dc1a1349394b28c8998e', 'timestamp': '2023-11-14_20:31:57', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3590714'} 2023-11-15T02:55:39.793 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-15T02:55:39.801 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-15T02:55:39.821 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/run/ceph 2023-11-15T02:55:39.862 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/run/ceph 2023-11-15T02:55:39.905 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:55:39.905 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/scratch_devs of=/dev/stdout 2023-11-15T02:55:39.959 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-15T02:55:39.960 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_1 2023-11-15T02:55:40.025 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-15T02:55:40.026 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-15T02:55:40.026 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 495 Links: 1 2023-11-15T02:55:40.026 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-15T02:55:40.026 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-11-15 02:50:32.792104989 +0000 2023-11-15T02:55:40.026 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-11-15 02:50:32.788105082 +0000 2023-11-15T02:55:40.027 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-11-15 02:50:32.788105082 +0000 2023-11-15T02:55:40.027 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-11-15T02:55:40.027 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-15T02:55:40.104 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-11-15T02:55:40.104 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-11-15T02:55:40.105 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000695401 s, 736 kB/s 2023-11-15T02:55:40.107 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-15T02:55:40.171 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_2 2023-11-15T02:55:40.241 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-15T02:55:40.242 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-15T02:55:40.242 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 500 Links: 1 2023-11-15T02:55:40.242 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-15T02:55:40.243 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-11-15 02:50:34.036076301 +0000 2023-11-15T02:55:40.243 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-11-15 02:50:33.584086725 +0000 2023-11-15T02:55:40.243 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-11-15 02:50:33.584086725 +0000 2023-11-15T02:55:40.243 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-11-15T02:55:40.244 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-15T02:55:40.322 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-11-15T02:55:40.322 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-11-15T02:55:40.322 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000555137 s, 922 kB/s 2023-11-15T02:55:40.324 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-15T02:55:40.394 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_3 2023-11-15T02:55:40.457 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-15T02:55:40.457 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-15T02:55:40.458 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 508 Links: 1 2023-11-15T02:55:40.458 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-15T02:55:40.458 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-11-15 02:50:34.820058222 +0000 2023-11-15T02:55:40.458 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-11-15 02:50:34.340069291 +0000 2023-11-15T02:55:40.458 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-11-15 02:50:34.340069291 +0000 2023-11-15T02:55:40.459 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-11-15T02:55:40.459 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-15T02:55:40.540 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-11-15T02:55:40.540 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-11-15T02:55:40.540 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000872347 s, 587 kB/s 2023-11-15T02:55:40.542 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-15T02:55:40.608 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_4 2023-11-15T02:55:40.670 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-15T02:55:40.670 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-15T02:55:40.670 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 511 Links: 1 2023-11-15T02:55:40.670 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-15T02:55:40.671 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-11-15 02:50:35.112051488 +0000 2023-11-15T02:55:40.671 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-11-15 02:50:35.108051579 +0000 2023-11-15T02:55:40.671 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-11-15 02:50:35.108051579 +0000 2023-11-15T02:55:40.671 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-11-15T02:55:40.672 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-15T02:55:40.747 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-11-15T02:55:40.747 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-11-15T02:55:40.748 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000665857 s, 769 kB/s 2023-11-15T02:55:40.749 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-15T02:55:40.817 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-15T02:55:40.817 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:55:40.817 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2023-11-15T02:55:40.835 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-15T02:55:40.836 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2023-11-15T02:55:40.899 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-15T02:55:40.899 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-15T02:55:40.899 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 493 Links: 1 2023-11-15T02:55:40.900 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-15T02:55:40.900 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-11-15 02:50:33.475966410 +0000 2023-11-15T02:55:40.900 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-11-15 02:50:33.471966503 +0000 2023-11-15T02:55:40.900 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-11-15 02:50:33.471966503 +0000 2023-11-15T02:55:40.901 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2023-11-15T02:55:40.901 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-15T02:55:40.977 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-11-15T02:55:40.977 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-11-15T02:55:40.978 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000694658 s, 737 kB/s 2023-11-15T02:55:40.981 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-15T02:55:41.052 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2023-11-15T02:55:41.118 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-15T02:55:41.119 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-15T02:55:41.119 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 500 Links: 1 2023-11-15T02:55:41.119 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-15T02:55:41.120 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-11-15 02:50:34.959932188 +0000 2023-11-15T02:55:41.120 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-11-15 02:50:34.311947131 +0000 2023-11-15T02:55:41.120 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-11-15 02:50:34.311947131 +0000 2023-11-15T02:55:41.120 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2023-11-15T02:55:41.120 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-15T02:55:41.190 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-11-15T02:55:41.191 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-11-15T02:55:41.191 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000627177 s, 816 kB/s 2023-11-15T02:55:41.192 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-15T02:55:41.258 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2023-11-15T02:55:41.321 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-15T02:55:41.321 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-15T02:55:41.322 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 507 Links: 1 2023-11-15T02:55:41.322 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-15T02:55:41.322 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-11-15 02:50:35.743914108 +0000 2023-11-15T02:55:41.322 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-11-15 02:50:35.275924901 +0000 2023-11-15T02:55:41.323 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-11-15 02:50:35.275924901 +0000 2023-11-15T02:55:41.323 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2023-11-15T02:55:41.323 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-15T02:55:41.398 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-11-15T02:55:41.399 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-11-15T02:55:41.399 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000700615 s, 731 kB/s 2023-11-15T02:55:41.401 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-15T02:55:41.466 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2023-11-15T02:55:41.530 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-15T02:55:41.531 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-15T02:55:41.531 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 513 Links: 1 2023-11-15T02:55:41.531 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-15T02:55:41.531 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-11-15 02:50:36.491896859 +0000 2023-11-15T02:55:41.531 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-11-15 02:50:36.047907098 +0000 2023-11-15T02:55:41.532 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-11-15 02:50:36.047907098 +0000 2023-11-15T02:55:41.532 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2023-11-15T02:55:41.532 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-15T02:55:41.605 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-11-15T02:55:41.605 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-11-15T02:55:41.605 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000485022 s, 1.1 MB/s 2023-11-15T02:55:41.607 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-15T02:55:41.670 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-15T02:55:41.670 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-15T02:55:41.671 INFO:tasks.ceph:Generating config... 2023-11-15T02:55:41.672 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-15T02:55:41.672 INFO:tasks.ceph:[client] debug client = 20 2023-11-15T02:55:41.672 INFO:tasks.ceph:[client] debug ms = 1 2023-11-15T02:55:41.673 INFO:tasks.ceph:[client] pid file = 2023-11-15T02:55:41.673 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-15T02:55:41.673 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-15T02:55:41.673 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-15T02:55:41.674 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-15T02:55:41.674 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-15T02:55:41.674 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-15T02:55:41.675 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-15T02:55:41.675 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-15T02:55:41.675 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-15T02:55:41.675 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-15T02:55:41.676 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-15T02:55:41.676 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-15T02:55:41.676 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-15T02:55:41.676 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-15T02:55:41.677 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-15T02:55:41.677 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-15T02:55:41.677 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-15T02:55:41.677 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-15T02:55:41.678 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-15T02:55:41.678 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-15T02:55:41.678 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-15T02:55:41.678 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-15T02:55:41.679 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-15T02:55:41.679 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-15T02:55:41.679 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-15T02:55:41.680 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-15T02:55:41.680 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-15T02:55:41.680 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-15T02:55:41.680 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-15T02:55:41.681 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-15T02:55:41.681 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-15T02:55:41.681 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-15T02:55:41.681 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-15T02:55:41.682 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-15T02:55:41.682 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-15T02:55:41.682 INFO:tasks.ceph:Setting up mon.a... 2023-11-15T02:55:41.683 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-15T02:55:41.757 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/ceph.keyring 2023-11-15T02:55:41.762 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-15T02:55:41.827 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-15T02:55:41.857 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.121'), ('mon.b', '172.21.15.190'), ('mon.c', '[v2:172.21.15.190:3301,v1:172.21.15.190:6790]')] 2023-11-15T02:55:41.858 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.121,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-15T02:55:41.859 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:55:41.859 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-15T02:55:41.916 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.121 --add b 172.21.15.190 --addv c '[v2:172.21.15.190:3301,v1:172.21.15.190:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-15T02:55:42.021 INFO:teuthology.orchestra.run.smithi121.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-15T02:55:42.022 INFO:teuthology.orchestra.run.smithi121.stdout:monmaptool: generated fsid be3e38c1-527f-4644-9b96-70c42632cdc5 2023-11-15T02:55:42.022 INFO:teuthology.orchestra.run.smithi121.stdout:epoch 0 2023-11-15T02:55:42.022 INFO:teuthology.orchestra.run.smithi121.stdout:fsid be3e38c1-527f-4644-9b96-70c42632cdc5 2023-11-15T02:55:42.022 INFO:teuthology.orchestra.run.smithi121.stdout:last_changed 2023-11-15T02:55:42.019221+0000 2023-11-15T02:55:42.022 INFO:teuthology.orchestra.run.smithi121.stdout:created 2023-11-15T02:55:42.019221+0000 2023-11-15T02:55:42.023 INFO:teuthology.orchestra.run.smithi121.stdout:min_mon_release 0 (unknown) 2023-11-15T02:55:42.023 INFO:teuthology.orchestra.run.smithi121.stdout:election_strategy: 1 2023-11-15T02:55:42.023 INFO:teuthology.orchestra.run.smithi121.stdout:0: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.a 2023-11-15T02:55:42.023 INFO:teuthology.orchestra.run.smithi121.stdout:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.b 2023-11-15T02:55:42.023 INFO:teuthology.orchestra.run.smithi121.stdout:2: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.c 2023-11-15T02:55:42.024 INFO:teuthology.orchestra.run.smithi121.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-15T02:55:42.025 DEBUG:teuthology.orchestra.run.smithi121:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-15T02:55:42.043 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID be3e38c1-527f-4644-9b96-70c42632cdc5... 2023-11-15T02:55:42.044 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-15T02:55:42.090 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-15T02:55:42.169 INFO:teuthology.orchestra.run.smithi121.stdout:[global] 2023-11-15T02:55:42.169 INFO:teuthology.orchestra.run.smithi121.stdout: chdir = "" 2023-11-15T02:55:42.170 INFO:teuthology.orchestra.run.smithi121.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-15T02:55:42.170 INFO:teuthology.orchestra.run.smithi121.stdout: auth supported = cephx 2023-11-15T02:55:42.170 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.170 INFO:teuthology.orchestra.run.smithi121.stdout: filestore xattr use omap = true 2023-11-15T02:55:42.170 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.171 INFO:teuthology.orchestra.run.smithi121.stdout: mon clock drift allowed = 1.000 2023-11-15T02:55:42.171 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.171 INFO:teuthology.orchestra.run.smithi121.stdout: osd crush chooseleaf type = 0 2023-11-15T02:55:42.171 INFO:teuthology.orchestra.run.smithi121.stdout: auth debug = true 2023-11-15T02:55:42.171 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.172 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on old message = true 2023-11-15T02:55:42.172 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on bug = true 2023-11-15T02:55:42.172 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.172 INFO:teuthology.orchestra.run.smithi121.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-15T02:55:42.172 INFO:teuthology.orchestra.run.smithi121.stdout: mon pg warn max object skew = 0 2023-11-15T02:55:42.173 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.173 INFO:teuthology.orchestra.run.smithi121.stdout: # disable pg_autoscaler by default for new pools 2023-11-15T02:55:42.173 INFO:teuthology.orchestra.run.smithi121.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-15T02:55:42.173 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.173 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default size = 2 2023-11-15T02:55:42.174 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.174 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow primary affinity = true 2023-11-15T02:55:42.174 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow pg remap = true 2023-11-15T02:55:42.174 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on legacy crush tunables = false 2023-11-15T02:55:42.174 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on crush straw calc version zero = false 2023-11-15T02:55:42.175 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on no sortbitwise = false 2023-11-15T02:55:42.175 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on osd down out interval zero = false 2023-11-15T02:55:42.175 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on too few osds = false 2023-11-15T02:55:42.175 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-15T02:55:42.176 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-15T02:55:42.176 INFO:teuthology.orchestra.run.smithi121.stdout: mon_allow_pool_size_one = true 2023-11-15T02:55:42.176 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.176 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2023-11-15T02:55:42.176 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.177 INFO:teuthology.orchestra.run.smithi121.stdout: osd default data pool replay window = 5 2023-11-15T02:55:42.177 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.177 INFO:teuthology.orchestra.run.smithi121.stdout: mon allow pool delete = true 2023-11-15T02:55:42.177 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.177 INFO:teuthology.orchestra.run.smithi121.stdout: mon cluster log file level = debug 2023-11-15T02:55:42.178 INFO:teuthology.orchestra.run.smithi121.stdout: debug asserts on shutdown = true 2023-11-15T02:55:42.178 INFO:teuthology.orchestra.run.smithi121.stdout: mon health detail to clog = false 2023-11-15T02:55:42.178 INFO:teuthology.orchestra.run.smithi121.stdout: mon host = "172.21.15.121,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-11-15T02:55:42.178 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on skipped message = False 2023-11-15T02:55:42.178 INFO:teuthology.orchestra.run.smithi121.stdout: fsid = be3e38c1-527f-4644-9b96-70c42632cdc5 2023-11-15T02:55:42.179 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.179 INFO:teuthology.orchestra.run.smithi121.stdout:[osd] 2023-11-15T02:55:42.179 INFO:teuthology.orchestra.run.smithi121.stdout: osd journal size = 100 2023-11-15T02:55:42.179 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.179 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub load threshold = 5.0 2023-11-15T02:55:42.180 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub max interval = 600 2023-11-15T02:55:42.180 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.180 INFO:teuthology.orchestra.run.smithi121.stdout: osd recover clone overlap = true 2023-11-15T02:55:42.180 INFO:teuthology.orchestra.run.smithi121.stdout: osd recovery max chunk = 1048576 2023-11-15T02:55:42.180 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.180 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug shutdown = true 2023-11-15T02:55:42.181 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug op order = true 2023-11-15T02:55:42.181 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify stray on activate = true 2023-11-15T02:55:42.181 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.181 INFO:teuthology.orchestra.run.smithi121.stdout: osd open classes on start = true 2023-11-15T02:55:42.181 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug pg log writeout = true 2023-11-15T02:55:42.182 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.182 INFO:teuthology.orchestra.run.smithi121.stdout: osd deep scrub update digest min age = 30 2023-11-15T02:55:42.182 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.182 INFO:teuthology.orchestra.run.smithi121.stdout: osd map max advance = 10 2023-11-15T02:55:42.183 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.183 INFO:teuthology.orchestra.run.smithi121.stdout: journal zero on create = true 2023-11-15T02:55:42.183 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.183 INFO:teuthology.orchestra.run.smithi121.stdout: filestore ondisk finisher threads = 3 2023-11-15T02:55:42.183 INFO:teuthology.orchestra.run.smithi121.stdout: filestore apply finisher threads = 3 2023-11-15T02:55:42.184 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.184 INFO:teuthology.orchestra.run.smithi121.stdout: bdev debug aio = true 2023-11-15T02:55:42.184 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug misdirected ops = true 2023-11-15T02:55:42.184 INFO:teuthology.orchestra.run.smithi121.stdout: bdev async discard = True 2023-11-15T02:55:42.184 INFO:teuthology.orchestra.run.smithi121.stdout: bdev enable discard = True 2023-11-15T02:55:42.184 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore allocator = bitmap 2023-11-15T02:55:42.185 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore block size = 96636764160 2023-11-15T02:55:42.185 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore fsck on mount = True 2023-11-15T02:55:42.185 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluefs = 1/20 2023-11-15T02:55:42.185 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluestore = 1/20 2023-11-15T02:55:42.185 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-11-15T02:55:42.186 INFO:teuthology.orchestra.run.smithi121.stdout: debug osd = 20 2023-11-15T02:55:42.186 INFO:teuthology.orchestra.run.smithi121.stdout: debug rocksdb = 4/10 2023-11-15T02:55:42.186 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd backfillfull_ratio = 0.85 2023-11-15T02:55:42.186 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd full ratio = 0.9 2023-11-15T02:55:42.186 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd nearfull ratio = 0.8 2023-11-15T02:55:42.187 INFO:teuthology.orchestra.run.smithi121.stdout: osd failsafe full ratio = 0.95 2023-11-15T02:55:42.187 INFO:teuthology.orchestra.run.smithi121.stdout: osd objectstore = bluestore 2023-11-15T02:55:42.187 INFO:teuthology.orchestra.run.smithi121.stdout: osd op complaint time = 180 2023-11-15T02:55:42.187 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.187 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr] 2023-11-15T02:55:42.188 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-11-15T02:55:42.188 INFO:teuthology.orchestra.run.smithi121.stdout: debug mgr = 20 2023-11-15T02:55:42.188 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2023-11-15T02:55:42.188 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2023-11-15T02:55:42.188 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2023-11-15T02:55:42.189 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2023-11-15T02:55:42.189 INFO:teuthology.orchestra.run.smithi121.stdout: mgr/telemetry/nag = false 2023-11-15T02:55:42.189 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.189 INFO:teuthology.orchestra.run.smithi121.stdout:[mon] 2023-11-15T02:55:42.189 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-11-15T02:55:42.190 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2023-11-15T02:55:42.190 INFO:teuthology.orchestra.run.smithi121.stdout: debug paxos = 20 2023-11-15T02:55:42.190 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2023-11-15T02:55:42.190 INFO:teuthology.orchestra.run.smithi121.stdout: mon data avail warn = 5 2023-11-15T02:55:42.190 INFO:teuthology.orchestra.run.smithi121.stdout: mon mgr mkfs grace = 240 2023-11-15T02:55:42.191 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2023-11-15T02:55:42.191 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd reporter subtree level = osd 2023-11-15T02:55:42.191 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd prime pg temp = true 2023-11-15T02:55:42.191 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2023-11-15T02:55:42.191 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.192 INFO:teuthology.orchestra.run.smithi121.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-15T02:55:42.192 INFO:teuthology.orchestra.run.smithi121.stdout: auth mon ticket ttl = 660 # 11m 2023-11-15T02:55:42.192 INFO:teuthology.orchestra.run.smithi121.stdout: auth service ticket ttl = 240 # 4m 2023-11-15T02:55:42.192 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.192 INFO:teuthology.orchestra.run.smithi121.stdout: # don't complain about insecure global_id in the test suite 2023-11-15T02:55:42.192 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-15T02:55:42.193 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-15T02:55:42.193 INFO:teuthology.orchestra.run.smithi121.stdout: mon op complaint time = 120 2023-11-15T02:55:42.193 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:55:42.193 INFO:teuthology.orchestra.run.smithi121.stdout:[client] 2023-11-15T02:55:42.193 INFO:teuthology.orchestra.run.smithi121.stdout: rgw cache enabled = true 2023-11-15T02:55:42.194 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable ops log = true 2023-11-15T02:55:42.194 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable usage log = true 2023-11-15T02:55:42.194 INFO:teuthology.orchestra.run.smithi121.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-15T02:55:42.194 INFO:teuthology.orchestra.run.smithi121.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-15T02:55:42.194 INFO:teuthology.orchestra.run.smithi121.stdout: client mount timeout = 600 2023-11-15T02:55:42.195 INFO:teuthology.orchestra.run.smithi121.stdout: debug client = 20 2023-11-15T02:55:42.195 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-11-15T02:55:42.195 INFO:teuthology.orchestra.run.smithi121.stdout: pid file = "" 2023-11-15T02:55:42.195 INFO:teuthology.orchestra.run.smithi121.stdout: rados mon op timeout = 900 2023-11-15T02:55:42.195 INFO:teuthology.orchestra.run.smithi121.stdout: rados osd op timeout = 900 2023-11-15T02:55:42.195 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.a] 2023-11-15T02:55:42.196 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.b] 2023-11-15T02:55:42.196 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.c] 2023-11-15T02:55:42.196 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.a] 2023-11-15T02:55:42.196 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.c] 2023-11-15T02:55:42.196 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.b] 2023-11-15T02:55:42.197 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.d] 2023-11-15T02:55:42.197 INFO:teuthology.orchestra.run.smithi121.stdout:[mds] 2023-11-15T02:55:42.197 INFO:teuthology.orchestra.run.smithi121.stdout: debug mds = 20 2023-11-15T02:55:42.197 INFO:teuthology.orchestra.run.smithi121.stdout: debug mds balancer = 20 2023-11-15T02:55:42.197 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-11-15T02:55:42.197 INFO:teuthology.orchestra.run.smithi121.stdout: mds debug frag = True 2023-11-15T02:55:42.198 INFO:teuthology.orchestra.run.smithi121.stdout: mds debug scatterstat = True 2023-11-15T02:55:42.198 INFO:teuthology.orchestra.run.smithi121.stdout: mds op complaint time = 180 2023-11-15T02:55:42.198 INFO:teuthology.orchestra.run.smithi121.stdout: mds verify scatter = True 2023-11-15T02:55:42.198 INFO:teuthology.orchestra.run.smithi121.stdout: osd op complaint time = 180 2023-11-15T02:55:42.198 INFO:teuthology.orchestra.run.smithi121.stdout: rados mon op timeout = 900 2023-11-15T02:55:42.199 INFO:teuthology.orchestra.run.smithi121.stdout: rados osd op timeout = 900 2023-11-15T02:55:42.200 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2023-11-15T02:55:42.200 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2023-11-15T02:55:42.201 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-15T02:55:42.201 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2023-11-15T02:55:42.201 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.201 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2023-11-15T02:55:42.201 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.202 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2023-11-15T02:55:42.202 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.202 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2023-11-15T02:55:42.202 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2023-11-15T02:55:42.202 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.203 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2023-11-15T02:55:42.203 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2023-11-15T02:55:42.203 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.203 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-15T02:55:42.203 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2023-11-15T02:55:42.204 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.204 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2023-11-15T02:55:42.204 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-15T02:55:42.204 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.204 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2023-11-15T02:55:42.205 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.205 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2023-11-15T02:55:42.205 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2023-11-15T02:55:42.205 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2023-11-15T02:55:42.205 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2023-11-15T02:55:42.206 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2023-11-15T02:55:42.206 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2023-11-15T02:55:42.206 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2023-11-15T02:55:42.206 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-15T02:55:42.206 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-15T02:55:42.207 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2023-11-15T02:55:42.207 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.207 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2023-11-15T02:55:42.207 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.208 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2023-11-15T02:55:42.208 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.208 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2023-11-15T02:55:42.208 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.209 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2023-11-15T02:55:42.209 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2023-11-15T02:55:42.209 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2023-11-15T02:55:42.209 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.121,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-11-15T02:55:42.209 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on skipped message = False 2023-11-15T02:55:42.210 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = be3e38c1-527f-4644-9b96-70c42632cdc5 2023-11-15T02:55:42.210 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.210 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2023-11-15T02:55:42.210 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2023-11-15T02:55:42.211 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.211 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2023-11-15T02:55:42.211 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2023-11-15T02:55:42.211 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.211 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2023-11-15T02:55:42.212 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2023-11-15T02:55:42.212 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.212 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2023-11-15T02:55:42.212 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2023-11-15T02:55:42.212 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2023-11-15T02:55:42.213 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.213 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2023-11-15T02:55:42.213 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2023-11-15T02:55:42.213 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.213 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2023-11-15T02:55:42.214 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.214 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2023-11-15T02:55:42.214 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.214 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2023-11-15T02:55:42.214 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.215 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2023-11-15T02:55:42.215 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2023-11-15T02:55:42.215 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.215 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2023-11-15T02:55:42.215 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2023-11-15T02:55:42.216 INFO:teuthology.orchestra.run.smithi190.stdout: bdev async discard = True 2023-11-15T02:55:42.216 INFO:teuthology.orchestra.run.smithi190.stdout: bdev enable discard = True 2023-11-15T02:55:42.216 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore allocator = bitmap 2023-11-15T02:55:42.216 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2023-11-15T02:55:42.216 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2023-11-15T02:55:42.217 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 1/20 2023-11-15T02:55:42.217 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 1/20 2023-11-15T02:55:42.217 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-11-15T02:55:42.217 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2023-11-15T02:55:42.217 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 4/10 2023-11-15T02:55:42.218 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2023-11-15T02:55:42.218 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2023-11-15T02:55:42.218 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2023-11-15T02:55:42.218 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2023-11-15T02:55:42.218 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2023-11-15T02:55:42.219 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-11-15T02:55:42.219 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.219 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2023-11-15T02:55:42.219 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-11-15T02:55:42.219 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2023-11-15T02:55:42.220 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-11-15T02:55:42.220 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-11-15T02:55:42.220 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-11-15T02:55:42.220 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-11-15T02:55:42.220 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2023-11-15T02:55:42.221 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.221 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2023-11-15T02:55:42.221 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-11-15T02:55:42.221 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-11-15T02:55:42.221 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2023-11-15T02:55:42.222 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-11-15T02:55:42.222 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2023-11-15T02:55:42.222 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2023-11-15T02:55:42.222 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-11-15T02:55:42.222 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2023-11-15T02:55:42.222 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2023-11-15T02:55:42.223 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-11-15T02:55:42.223 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.223 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-15T02:55:42.223 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2023-11-15T02:55:42.223 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2023-11-15T02:55:42.224 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.224 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2023-11-15T02:55:42.224 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-15T02:55:42.224 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-15T02:55:42.224 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2023-11-15T02:55:42.225 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-11-15T02:55:42.225 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2023-11-15T02:55:42.225 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2023-11-15T02:55:42.225 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2023-11-15T02:55:42.225 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2023-11-15T02:55:42.226 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-15T02:55:42.226 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-15T02:55:42.226 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2023-11-15T02:55:42.226 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2023-11-15T02:55:42.226 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-11-15T02:55:42.227 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = "" 2023-11-15T02:55:42.227 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-11-15T02:55:42.227 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-11-15T02:55:42.227 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2023-11-15T02:55:42.227 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2023-11-15T02:55:42.228 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2023-11-15T02:55:42.228 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2023-11-15T02:55:42.228 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2023-11-15T02:55:42.228 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2023-11-15T02:55:42.228 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2023-11-15T02:55:42.228 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2023-11-15T02:55:42.229 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2023-11-15T02:55:42.229 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds balancer = 20 2023-11-15T02:55:42.229 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-11-15T02:55:42.229 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2023-11-15T02:55:42.229 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = True 2023-11-15T02:55:42.230 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2023-11-15T02:55:42.230 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = True 2023-11-15T02:55:42.230 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-11-15T02:55:42.230 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-11-15T02:55:42.230 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-11-15T02:55:42.231 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-15T02:55:42.231 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-15T02:55:42.320 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-15T02:55:42.321 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:55:42.321 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-15T02:55:42.341 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:55:42.341 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-15T02:55:42.405 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T02:55:42.406 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:55:42.406 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-15T02:55:42.407 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-15T02:55:42.515 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:55:42.515 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-15T02:55:42.573 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi190.front.sepia.ceph.com 2023-11-15T02:55:42.573 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:55:42.574 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-15T02:55:42.574 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-15T02:55:42.629 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:55:42.630 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-15T02:55:42.692 INFO:tasks.ceph:Setting up mon nodes... 2023-11-15T02:55:42.693 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-15T02:55:42.693 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-15T02:55:42.780 INFO:teuthology.orchestra.run.smithi121.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-15T02:55:42.787 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-15T02:55:42.894 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-15T02:55:42.900 INFO:tasks.ceph:Setting up mds nodes... 2023-11-15T02:55:42.901 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-15T02:55:42.991 INFO:teuthology.orchestra.run.smithi121.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-15T02:55:42.996 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-15T02:55:43.029 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-15T02:55:43.156 INFO:teuthology.orchestra.run.smithi121.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-15T02:55:43.161 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-15T02:55:43.199 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-15T02:55:43.290 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-15T02:55:43.297 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-15T02:55:43.332 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-15T02:55:43.462 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-15T02:55:43.468 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-15T02:55:43.498 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-15T02:55:43.498 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-15T02:55:43.568 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-15T02:55:43.596 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-15T02:55:43.655 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-15T02:55:43.683 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-15T02:55:43.747 INFO:teuthology.orchestra.run.smithi190.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-15T02:55:43.775 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-15T02:55:43.840 INFO:teuthology.orchestra.run.smithi190.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-15T02:55:43.870 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-15T02:55:43.870 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-15T02:55:43.871 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-15T02:55:43.908 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-15T02:55:43.908 INFO:tasks.ceph:role: osd.0 2023-11-15T02:55:43.909 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T02:55:43.909 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-15T02:55:44.509 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-15T02:55:44.509 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-15T02:55:44.510 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-15T02:55:44.510 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 2023-11-15T02:55:44.511 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-15T02:55:44.511 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2023-11-15T02:55:44.511 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-15T02:55:44.512 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-15T02:55:44.512 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-15T02:55:44.512 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-15T02:55:44.513 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2023-11-15T02:55:44.514 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-15T02:55:44.643 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-15T02:55:44.670 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: /sbin/restorecon: command not found 2023-11-15T02:55:44.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T02:55:44.672 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-15T02:55:44.755 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-15T02:55:44.755 INFO:tasks.ceph:role: osd.1 2023-11-15T02:55:44.756 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T02:55:44.756 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-15T02:55:45.274 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-15T02:55:45.275 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-15T02:55:45.275 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-15T02:55:45.275 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 2023-11-15T02:55:45.275 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-15T02:55:45.276 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2023-11-15T02:55:45.276 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-15T02:55:45.276 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-15T02:55:45.276 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-15T02:55:45.276 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-15T02:55:45.279 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2023-11-15T02:55:45.280 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-15T02:55:45.353 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-15T02:55:45.372 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: /sbin/restorecon: command not found 2023-11-15T02:55:45.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T02:55:45.374 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-15T02:55:45.450 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-15T02:55:45.450 INFO:tasks.ceph:role: osd.2 2023-11-15T02:55:45.451 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T02:55:45.451 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-15T02:55:46.014 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-15T02:55:46.015 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-15T02:55:46.015 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-15T02:55:46.015 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 2023-11-15T02:55:46.015 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-15T02:55:46.016 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2023-11-15T02:55:46.016 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-15T02:55:46.016 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-15T02:55:46.016 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-15T02:55:46.016 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-15T02:55:46.019 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2023-11-15T02:55:46.020 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-15T02:55:46.121 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-15T02:55:46.144 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: /sbin/restorecon: command not found 2023-11-15T02:55:46.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T02:55:46.145 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-15T02:55:46.225 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-15T02:55:46.226 INFO:tasks.ceph:role: osd.3 2023-11-15T02:55:46.226 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T02:55:46.226 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-15T02:55:46.814 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-15T02:55:46.814 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-15T02:55:46.814 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-15T02:55:46.815 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 2023-11-15T02:55:46.815 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-15T02:55:46.815 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2023-11-15T02:55:46.815 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-15T02:55:46.815 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-15T02:55:46.816 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-15T02:55:46.816 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-15T02:55:46.819 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2023-11-15T02:55:46.820 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-15T02:55:46.962 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-15T02:55:46.983 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: /sbin/restorecon: command not found 2023-11-15T02:55:46.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T02:55:46.985 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-15T02:55:47.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:47.147+0000 7fefd268b200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-15T02:55:47.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:47.151+0000 7fefd268b200 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-15T02:55:47.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:47.151+0000 7fefd268b200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-15T02:55:47.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:47.151+0000 7fefd268b200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-15T02:55:47.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:47.151+0000 7fefd268b200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-15T02:55:47.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:47.155+0000 7fefd268b200 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-15T02:55:50.703 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-15T02:55:50.735 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-15T02:55:50.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:50.843+0000 7fa1a66dd200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-15T02:55:50.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:50.843+0000 7fa1a66dd200 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-15T02:55:50.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:50.843+0000 7fa1a66dd200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-15T02:55:50.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:50.843+0000 7fa1a66dd200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-15T02:55:50.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:50.843+0000 7fa1a66dd200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-15T02:55:50.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:50.843+0000 7fa1a66dd200 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-15T02:55:54.381 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-15T02:55:54.412 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-15T02:55:54.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:54.523+0000 7f7e89ad9200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-15T02:55:54.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:54.523+0000 7f7e89ad9200 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-15T02:55:54.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:54.523+0000 7f7e89ad9200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-15T02:55:54.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:54.523+0000 7f7e89ad9200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-15T02:55:54.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:54.523+0000 7f7e89ad9200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-15T02:55:54.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:54.523+0000 7f7e89ad9200 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-15T02:55:58.073 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-15T02:55:58.109 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-15T02:55:58.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:58.215+0000 7efe41cdb200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-15T02:55:58.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:58.215+0000 7efe41cdb200 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-15T02:55:58.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:58.215+0000 7efe41cdb200 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-15T02:55:58.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:58.215+0000 7efe41cdb200 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-15T02:55:58.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:58.215+0000 7efe41cdb200 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-15T02:55:58.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:55:58.219+0000 7efe41cdb200 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-15T02:56:01.752 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-15T02:56:01.782 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-15T02:56:01.824 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-15T02:56:01.824 INFO:tasks.ceph:role: osd.4 2023-11-15T02:56:01.824 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com 2023-11-15T02:56:01.825 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-15T02:56:02.431 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-15T02:56:02.431 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-15T02:56:02.431 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-15T02:56:02.431 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 2023-11-15T02:56:02.432 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-15T02:56:02.432 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2023-11-15T02:56:02.432 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-15T02:56:02.432 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-15T02:56:02.432 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-15T02:56:02.433 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-15T02:56:02.435 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2023-11-15T02:56:02.436 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-15T02:56:02.571 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-15T02:56:02.594 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: /sbin/restorecon: command not found 2023-11-15T02:56:02.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T02:56:02.596 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-15T02:56:02.673 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-15T02:56:02.674 INFO:tasks.ceph:role: osd.5 2023-11-15T02:56:02.674 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com 2023-11-15T02:56:02.674 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-15T02:56:03.191 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-15T02:56:03.191 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-15T02:56:03.191 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-15T02:56:03.192 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 2023-11-15T02:56:03.192 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-15T02:56:03.192 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2023-11-15T02:56:03.192 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-15T02:56:03.192 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-15T02:56:03.193 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-15T02:56:03.193 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-15T02:56:03.196 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2023-11-15T02:56:03.196 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-15T02:56:03.266 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-15T02:56:03.290 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: /sbin/restorecon: command not found 2023-11-15T02:56:03.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T02:56:03.291 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-15T02:56:03.368 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-15T02:56:03.368 INFO:tasks.ceph:role: osd.6 2023-11-15T02:56:03.369 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi190.front.sepia.ceph.com 2023-11-15T02:56:03.369 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-15T02:56:03.947 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-15T02:56:03.947 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-15T02:56:03.948 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-15T02:56:03.948 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 2023-11-15T02:56:03.948 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-15T02:56:03.948 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2023-11-15T02:56:03.948 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-15T02:56:03.949 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-15T02:56:03.949 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-15T02:56:03.949 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-15T02:56:03.952 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2023-11-15T02:56:03.952 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-15T02:56:04.079 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-15T02:56:04.099 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: /sbin/restorecon: command not found 2023-11-15T02:56:04.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T02:56:04.101 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-15T02:56:04.177 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-15T02:56:04.178 INFO:tasks.ceph:role: osd.7 2023-11-15T02:56:04.178 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi190.front.sepia.ceph.com 2023-11-15T02:56:04.178 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-15T02:56:04.771 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-15T02:56:04.771 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-15T02:56:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-15T02:56:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 2023-11-15T02:56:04.772 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-15T02:56:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2023-11-15T02:56:04.773 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-15T02:56:04.773 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-15T02:56:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-15T02:56:04.773 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-15T02:56:04.775 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2023-11-15T02:56:04.776 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-15T02:56:04.889 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-15T02:56:04.912 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: /sbin/restorecon: command not found 2023-11-15T02:56:04.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T02:56:04.913 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-15T02:56:05.073 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:05.070+0000 7f0db692c200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-15T02:56:05.077 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:05.074+0000 7f0db692c200 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-15T02:56:05.077 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:05.074+0000 7f0db692c200 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-15T02:56:05.077 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:05.074+0000 7f0db692c200 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-15T02:56:05.078 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:05.074+0000 7f0db692c200 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-15T02:56:05.081 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:05.078+0000 7f0db692c200 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-15T02:56:08.615 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-15T02:56:08.649 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-15T02:56:08.768 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:08.766+0000 7f35d9ed9200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-15T02:56:08.769 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:08.766+0000 7f35d9ed9200 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-15T02:56:08.769 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:08.766+0000 7f35d9ed9200 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-15T02:56:08.770 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:08.766+0000 7f35d9ed9200 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-15T02:56:08.770 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:08.766+0000 7f35d9ed9200 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-15T02:56:08.773 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:08.770+0000 7f35d9ed9200 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-15T02:56:09.794 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T02:56:09.802 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T02:56:12.064 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-15T02:56:12.096 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-15T02:56:12.220 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:12.214+0000 7f97dc8d9200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-15T02:56:12.221 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:12.218+0000 7f97dc8d9200 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-15T02:56:12.221 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:12.218+0000 7f97dc8d9200 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-15T02:56:12.221 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:12.218+0000 7f97dc8d9200 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-15T02:56:12.222 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:12.218+0000 7f97dc8d9200 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-15T02:56:12.223 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:12.218+0000 7f97dc8d9200 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-15T02:56:15.502 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-15T02:56:15.536 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-15T02:56:15.657 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:15.654+0000 7f3fee2db200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-15T02:56:15.658 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:15.654+0000 7f3fee2db200 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-15T02:56:15.659 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:15.654+0000 7f3fee2db200 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-15T02:56:15.659 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:15.654+0000 7f3fee2db200 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-15T02:56:15.659 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:15.654+0000 7f3fee2db200 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-15T02:56:15.661 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:15.658+0000 7f3fee2db200 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-15T02:56:19.191 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-15T02:56:19.226 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-15T02:56:19.227 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:56:19.227 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-15T02:56:19.262 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:56:19.262 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-15T02:56:19.337 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:56:19.337 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-15T02:56:19.412 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:56:19.413 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-15T02:56:19.487 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:56:19.488 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-15T02:56:19.564 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:56:19.564 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-15T02:56:19.637 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:56:19.638 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-15T02:56:19.714 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:56:19.714 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-15T02:56:19.748 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:56:19.748 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-15T02:56:19.828 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:56:19.829 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-15T02:56:19.903 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:56:19.904 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-15T02:56:19.985 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:56:19.985 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-15T02:56:20.066 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:56:20.067 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-15T02:56:20.141 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:56:20.141 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-15T02:56:20.216 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:56:20.217 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-15T02:56:20.235 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:56:20.236 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-15T02:56:20.297 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:56:20.298 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-15T02:56:20.317 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:56:20.317 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-15T02:56:20.380 INFO:tasks.ceph:Adding keys to all mons... 2023-11-15T02:56:20.380 DEBUG:teuthology.orchestra.run.smithi121:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-15T02:56:20.387 DEBUG:teuthology.orchestra.run.smithi190:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-15T02:56:20.431 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr.y] 2023-11-15T02:56:20.432 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAuM1RldQJlLhAAbc46liEP03yBNeVhZCb7IQ== 2023-11-15T02:56:20.432 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.a] 2023-11-15T02:56:20.432 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAuM1Rlzy3zOhAAeJ3COoWQP/Kw+P/0AwHbwA== 2023-11-15T02:56:20.432 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.c] 2023-11-15T02:56:20.433 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAvM1RldFcvCRAATIouBNyHGip5hFrH+z4v5A== 2023-11-15T02:56:20.433 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.0] 2023-11-15T02:56:20.433 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAzM1RldPH9CBAAUCF/a47aFbbNsWV1r+u3Tw== 2023-11-15T02:56:20.433 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.1] 2023-11-15T02:56:20.433 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQA2M1RltoxwMhAAZN3bgOjSaoK3ZxE5ezZ3IA== 2023-11-15T02:56:20.434 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.2] 2023-11-15T02:56:20.434 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQA6M1RlgltjHxAAx/m5gm+qs8LiEFyZL+uLzQ== 2023-11-15T02:56:20.434 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.3] 2023-11-15T02:56:20.434 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQA+M1RlhtQcDRAA+g11PmSNKazpk+VoK17EEQ== 2023-11-15T02:56:20.435 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr.x] 2023-11-15T02:56:20.435 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAuM1RlJQFPNRAAy7E5mrn8Xh98339fWSkX9Q== 2023-11-15T02:56:20.435 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.b] 2023-11-15T02:56:20.435 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAvM1RlUglaERAAC6WUS9ftCfr+KbPpiEIyqw== 2023-11-15T02:56:20.435 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.d] 2023-11-15T02:56:20.436 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAvM1Rlg5iZGxAADwdL2RrtU0kDeb9EmNJU6A== 2023-11-15T02:56:20.436 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.4] 2023-11-15T02:56:20.436 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQBFM1RlJ7hnBBAAjskKfD63N/61T4GSPqOiRg== 2023-11-15T02:56:20.436 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.5] 2023-11-15T02:56:20.436 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQBIM1RlI1HPLRAAgVbiNq1tmgK6PaWP+c2JFw== 2023-11-15T02:56:20.437 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.6] 2023-11-15T02:56:20.437 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQBMM1RlPDshDRAAVgIqxklHSQcFb5sM6GUfyQ== 2023-11-15T02:56:20.437 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.7] 2023-11-15T02:56:20.437 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQBPM1RlN8IuJxAA2rN2848kaB2eOM6zDb+eJg== 2023-11-15T02:56:20.438 INFO:teuthology.orchestra.run.smithi121.stdout:[client.0] 2023-11-15T02:56:20.438 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAvM1RlfHPEIRAAJ4YV46zzBQ7v7y0kD4P2ZA== 2023-11-15T02:56:20.438 INFO:teuthology.orchestra.run.smithi121.stdout:[client.2] 2023-11-15T02:56:20.438 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAvM1RlHHbvJhAAX5TE6UvcpBeJPSCZkfqD8w== 2023-11-15T02:56:20.438 INFO:teuthology.orchestra.run.smithi121.stdout:[client.1] 2023-11-15T02:56:20.439 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAvM1RlibaULBAA+y1EyhVOt+wXSk2uRajJ3g== 2023-11-15T02:56:20.439 INFO:teuthology.orchestra.run.smithi121.stdout:[client.3] 2023-11-15T02:56:20.439 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAvM1RlQtQXMhAAGAvtnvJLSoUKuEiZDsSxww== 2023-11-15T02:56:20.456 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr.y] 2023-11-15T02:56:20.456 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAuM1RldQJlLhAAbc46liEP03yBNeVhZCb7IQ== 2023-11-15T02:56:20.456 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2023-11-15T02:56:20.457 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAuM1Rlzy3zOhAAeJ3COoWQP/Kw+P/0AwHbwA== 2023-11-15T02:56:20.457 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2023-11-15T02:56:20.457 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAvM1RldFcvCRAATIouBNyHGip5hFrH+z4v5A== 2023-11-15T02:56:20.457 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.0] 2023-11-15T02:56:20.457 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAzM1RldPH9CBAAUCF/a47aFbbNsWV1r+u3Tw== 2023-11-15T02:56:20.458 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.1] 2023-11-15T02:56:20.458 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQA2M1RltoxwMhAAZN3bgOjSaoK3ZxE5ezZ3IA== 2023-11-15T02:56:20.458 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.2] 2023-11-15T02:56:20.458 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQA6M1RlgltjHxAAx/m5gm+qs8LiEFyZL+uLzQ== 2023-11-15T02:56:20.458 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.3] 2023-11-15T02:56:20.459 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQA+M1RlhtQcDRAA+g11PmSNKazpk+VoK17EEQ== 2023-11-15T02:56:20.459 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr.x] 2023-11-15T02:56:20.459 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAuM1RlJQFPNRAAy7E5mrn8Xh98339fWSkX9Q== 2023-11-15T02:56:20.459 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2023-11-15T02:56:20.460 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAvM1RlUglaERAAC6WUS9ftCfr+KbPpiEIyqw== 2023-11-15T02:56:20.460 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2023-11-15T02:56:20.460 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAvM1Rlg5iZGxAADwdL2RrtU0kDeb9EmNJU6A== 2023-11-15T02:56:20.460 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.4] 2023-11-15T02:56:20.461 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBFM1RlJ7hnBBAAjskKfD63N/61T4GSPqOiRg== 2023-11-15T02:56:20.461 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.5] 2023-11-15T02:56:20.461 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBIM1RlI1HPLRAAgVbiNq1tmgK6PaWP+c2JFw== 2023-11-15T02:56:20.461 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.6] 2023-11-15T02:56:20.461 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBMM1RlPDshDRAAVgIqxklHSQcFb5sM6GUfyQ== 2023-11-15T02:56:20.461 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.7] 2023-11-15T02:56:20.462 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBPM1RlN8IuJxAA2rN2848kaB2eOM6zDb+eJg== 2023-11-15T02:56:20.462 INFO:teuthology.orchestra.run.smithi190.stdout:[client.0] 2023-11-15T02:56:20.462 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAvM1RlfHPEIRAAJ4YV46zzBQ7v7y0kD4P2ZA== 2023-11-15T02:56:20.462 INFO:teuthology.orchestra.run.smithi190.stdout:[client.2] 2023-11-15T02:56:20.463 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAvM1RlHHbvJhAAX5TE6UvcpBeJPSCZkfqD8w== 2023-11-15T02:56:20.463 INFO:teuthology.orchestra.run.smithi190.stdout:[client.1] 2023-11-15T02:56:20.463 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAvM1RlibaULBAA+y1EyhVOt+wXSk2uRajJ3g== 2023-11-15T02:56:20.463 INFO:teuthology.orchestra.run.smithi190.stdout:[client.3] 2023-11-15T02:56:20.463 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAvM1RlQtQXMhAAGAvtnvJLSoUKuEiZDsSxww== 2023-11-15T02:56:20.464 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-15T02:56:20.489 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-15T02:56:20.583 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-15T02:56:20.590 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-15T02:56:20.666 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-15T02:56:20.673 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-15T02:56:20.752 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-15T02:56:20.758 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-15T02:56:20.839 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-15T02:56:20.847 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-15T02:56:20.926 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-15T02:56:20.934 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-15T02:56:21.005 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-15T02:56:21.012 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-15T02:56:21.080 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-15T02:56:21.087 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-15T02:56:21.159 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-15T02:56:21.167 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-15T02:56:21.241 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-15T02:56:21.247 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-15T02:56:21.319 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-15T02:56:21.326 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-15T02:56:21.401 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-15T02:56:21.408 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-15T02:56:21.479 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-15T02:56:21.487 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-15T02:56:21.562 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-15T02:56:21.574 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-15T02:56:21.647 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-15T02:56:21.654 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-15T02:56:21.728 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-15T02:56:21.735 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-15T02:56:21.814 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-15T02:56:21.821 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-15T02:56:21.892 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-15T02:56:21.899 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-15T02:56:21.970 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-15T02:56:21.970 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-15T02:56:22.006 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-15T02:56:22.142 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-15T02:56:22.176 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-15T02:56:22.213 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-15T02:56:22.349 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-15T02:56:22.383 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-15T02:56:22.460 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-15T02:56:22.598 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-15T02:56:22.632 DEBUG:teuthology.orchestra.run.smithi121:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-15T02:56:22.640 DEBUG:teuthology.orchestra.run.smithi190:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-15T02:56:22.692 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-15T02:56:22.692 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-15T02:56:22.693 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-15T02:56:22.701 INFO:tasks.ceph.mon.a:Started 2023-11-15T02:56:22.702 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-15T02:56:22.702 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-15T02:56:22.746 INFO:tasks.ceph.mon.b:Started 2023-11-15T02:56:22.747 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-15T02:56:22.747 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-15T02:56:22.751 INFO:tasks.ceph.mon.c:Started 2023-11-15T02:56:22.752 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-15T02:56:22.752 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-15T02:56:22.752 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-15T02:56:22.760 INFO:tasks.ceph.mgr.y:Started 2023-11-15T02:56:22.760 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-15T02:56:22.761 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-15T02:56:22.767 INFO:tasks.ceph.mgr.x:Started 2023-11-15T02:56:22.767 INFO:tasks.ceph:Setting crush tunables to default 2023-11-15T02:56:22.767 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd crush tunables default 2023-11-15T02:56:22.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:22.912+0000 7fe23a89b700 1 Processor -- start 2023-11-15T02:56:22.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:22.912+0000 7fe23a89b700 1 -- start start 2023-11-15T02:56:22.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:22.912+0000 7fe23a89b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe2340a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:22.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:22.912+0000 7fe23a89b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe234067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:22.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:22.912+0000 7fe23a89b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe234152150 con 0x7fe234067f50 2023-11-15T02:56:22.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:22.912+0000 7fe23a89b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe234148f70 con 0x7fe2340a8b80 2023-11-15T02:56:22.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:22.912+0000 7fe23a89b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2340683d0 con 0x7fe234065850 2023-11-15T02:56:22.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:22.912+0000 7fe233fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe2340a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:22.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:22.912+0000 7fe233fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe2340a8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44646/0 (socket says 172.21.15.121:44646) 2023-11-15T02:56:22.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:22.912+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 learned_addr learned my addr 172.21.15.121:0/2882868007 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:22.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:22.912+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 msgr2=0x7fe2340a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-15T02:56:22.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:22.912+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 msgr2=0x7fe2340a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:22.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:22.912+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe2340a8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:22.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:22.912+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe234067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:22.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:22.916+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe2340a8f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-11-15T02:56:22.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:22.916+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 msgr2=0x7fe234067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-15T02:56:22.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:22.916+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 msgr2=0x7fe234067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:22.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:22.916+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe234067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:22.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:22.916+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe234067a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-11-15T02:56:23.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.116+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe2340a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:23.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.116+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 msgr2=0x7fe2340a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-15T02:56:23.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.116+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 msgr2=0x7fe2340a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:23.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.116+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe2340a8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:23.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.116+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe2340a8f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2023-11-15T02:56:23.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.116+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe234067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:23.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.116+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 msgr2=0x7fe234067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-15T02:56:23.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.116+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 msgr2=0x7fe234067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:23.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.116+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe234067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:23.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.116+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe234067a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2023-11-15T02:56:23.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.516+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe2340a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:23.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.516+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 msgr2=0x7fe2340a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-15T02:56:23.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.516+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 msgr2=0x7fe2340a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:23.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.516+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe2340a8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:23.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.516+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe2340a8f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2023-11-15T02:56:23.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.516+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe234067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:23.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.516+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 msgr2=0x7fe234067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-15T02:56:23.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.520+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 msgr2=0x7fe234067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:23.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.520+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe234067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:23.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:23.520+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe234067a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2023-11-15T02:56:24.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:24.320+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe2340a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:24.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:24.320+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 msgr2=0x7fe2340a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-15T02:56:24.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:24.320+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 msgr2=0x7fe2340a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:24.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:24.320+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe2340a8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:24.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:24.320+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe2340a8f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2023-11-15T02:56:24.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:24.320+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe234067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:24.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:24.320+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 msgr2=0x7fe234067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-15T02:56:24.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:24.320+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 msgr2=0x7fe234067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:24.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:24.320+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe234067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:24.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:24.320+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe234067a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2023-11-15T02:56:25.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:25.912+0000 7fe2327fc700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 msgr2=0x7fe234067a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-15T02:56:25.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:25.912+0000 7fe2327fc700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe234067a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:25.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:25.912+0000 7fe2327fc700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 msgr2=0x7fe2340a8f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-15T02:56:25.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:25.912+0000 7fe2327fc700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe2340a8f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:25.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:25.912+0000 7fe2327fc700 1 -- 172.21.15.121:0/2882868007 >> v1:172.21.15.190:6789/0 conn(0x7fe234067f50 legacy=0x7fe2340bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:25.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:25.912+0000 7fe2327fc700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 0x7fe2140096c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:25.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:25.912+0000 7fe2327fc700 1 -- 172.21.15.121:0/2882868007 --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2341523a0 con 0x7fe214003590 2023-11-15T02:56:25.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:25.912+0000 7fe2327fc700 1 -- 172.21.15.121:0/2882868007 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe234148f70 con 0x7fe214007200 2023-11-15T02:56:25.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:25.912+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 0x7fe2140096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:25.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:25.916+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 msgr2=0x7fe2140096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-15T02:56:25.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:25.916+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 msgr2=0x7fe2140096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:25.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:25.916+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 0x7fe2140096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:25.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:25.916+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 0x7fe2140096c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-11-15T02:56:26.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:26.116+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 0x7fe2140096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:26.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:26.116+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 msgr2=0x7fe2140096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-15T02:56:26.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:26.116+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 msgr2=0x7fe2140096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:26.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:26.116+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 0x7fe2140096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:26.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:26.116+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 0x7fe2140096c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2023-11-15T02:56:26.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:26.516+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 0x7fe2140096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:26.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:26.516+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 msgr2=0x7fe2140096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-15T02:56:26.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:26.516+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 msgr2=0x7fe2140096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:26.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:26.516+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 0x7fe2140096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:26.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:26.520+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 0x7fe2140096c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2023-11-15T02:56:27.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:27.320+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 0x7fe2140096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:27.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:27.320+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 msgr2=0x7fe2140096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-15T02:56:27.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:27.320+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 msgr2=0x7fe2140096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:27.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:27.320+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 0x7fe2140096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:27.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:27.320+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 0x7fe2140096c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2023-11-15T02:56:28.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.912+0000 7fe2327fc700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 msgr2=0x7fe2140096c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-15T02:56:28.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.912+0000 7fe2327fc700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe214007200 0x7fe2140096c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:28.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.912+0000 7fe2327fc700 1 -- 172.21.15.121:0/2882868007 >> v1:172.21.15.121:6789/0 conn(0x7fe214003590 legacy=0x7fe2140059d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:28.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.912+0000 7fe2327fc700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe214009e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:28.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.912+0000 7fe238e38700 1 -- 172.21.15.121:0/2882868007 reap_dead start 2023-11-15T02:56:28.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.912+0000 7fe2327fc700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe21400a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:28.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.912+0000 7fe2327fc700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 0x7fe2140096c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:28.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.916+0000 7fe2327fc700 1 -- 172.21.15.121:0/2882868007 --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe234148f70 con 0x7fe2340a8b80 2023-11-15T02:56:28.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.916+0000 7fe2327fc700 1 -- 172.21.15.121:0/2882868007 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2340683d0 con 0x7fe21400a860 2023-11-15T02:56:28.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.916+0000 7fe2327fc700 1 -- 172.21.15.121:0/2882868007 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe214006100 con 0x7fe234065850 2023-11-15T02:56:28.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.916+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe21400a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:28.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.916+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe214009e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:28.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.916+0000 7fe238e38700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 0x7fe2140096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:28.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.916+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 msgr2=0x7fe21400a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-15T02:56:28.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.916+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 msgr2=0x7fe21400a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:28.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.916+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe21400a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:28.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.916+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe21400a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-11-15T02:56:28.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.916+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 msgr2=0x7fe214009e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-15T02:56:28.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.916+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 msgr2=0x7fe214009e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:28.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.916+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe214009e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:28.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.916+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe214009e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-11-15T02:56:28.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.916+0000 7fe238e38700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 msgr2=0x7fe2140096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-15T02:56:28.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.916+0000 7fe238e38700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 msgr2=0x7fe2140096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:28.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.916+0000 7fe238e38700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 0x7fe2140096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:28.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:28.916+0000 7fe238e38700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 0x7fe2140096c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-11-15T02:56:29.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.120+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe21400a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:29.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.120+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 msgr2=0x7fe21400a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-15T02:56:29.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.120+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 msgr2=0x7fe21400a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:29.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.120+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe21400a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:29.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.120+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe21400a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2023-11-15T02:56:29.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.120+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe214009e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:29.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.120+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 msgr2=0x7fe214009e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-15T02:56:29.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.120+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 msgr2=0x7fe214009e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:29.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.120+0000 7fe238e38700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 0x7fe2140096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:29.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.120+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe214009e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:29.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.120+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe214009e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2023-11-15T02:56:29.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.120+0000 7fe238e38700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 msgr2=0x7fe2140096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-15T02:56:29.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.120+0000 7fe238e38700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 msgr2=0x7fe2140096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:29.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.120+0000 7fe238e38700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 0x7fe2140096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:29.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.120+0000 7fe238e38700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 0x7fe2140096c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2023-11-15T02:56:29.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.524+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe21400a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:29.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.524+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe214009e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:29.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.524+0000 7fe238e38700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 0x7fe2140096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:29.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.524+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 msgr2=0x7fe21400a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-15T02:56:29.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.524+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 msgr2=0x7fe21400a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:29.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.524+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe21400a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:29.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.524+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe21400a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2023-11-15T02:56:29.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.524+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 msgr2=0x7fe214009e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-15T02:56:29.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.524+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 msgr2=0x7fe214009e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:29.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.524+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe214009e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:29.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.524+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe214009e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2023-11-15T02:56:29.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.524+0000 7fe238e38700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 msgr2=0x7fe2140096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-15T02:56:29.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.524+0000 7fe238e38700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 msgr2=0x7fe2140096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:29.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.524+0000 7fe238e38700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 0x7fe2140096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:29.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:29.524+0000 7fe238e38700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 0x7fe2140096c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2023-11-15T02:56:30.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:30.328+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe21400a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:30.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:30.328+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe214009e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:30.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:30.328+0000 7fe238e38700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 0x7fe2140096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:30.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:30.328+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 msgr2=0x7fe21400a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-15T02:56:30.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:30.328+0000 7fe22bfff700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 msgr2=0x7fe21400a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:30.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:30.328+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe21400a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:30.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:30.328+0000 7fe238e38700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 msgr2=0x7fe2140096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-15T02:56:30.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:30.328+0000 7fe238e38700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 msgr2=0x7fe2140096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:30.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:30.328+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 msgr2=0x7fe214009e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-15T02:56:30.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:30.328+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 msgr2=0x7fe214009e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T02:56:30.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:30.328+0000 7fe238e38700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 0x7fe2140096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:30.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:30.328+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe214009e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T02:56:30.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:30.328+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe21400a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2023-11-15T02:56:30.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:30.328+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe214009e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2023-11-15T02:56:30.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:30.328+0000 7fe238e38700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 0x7fe2140096c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2023-11-15T02:56:31.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:31.916+0000 7fe2327fc700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 msgr2=0x7fe214009e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-15T02:56:31.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:31.916+0000 7fe2327fc700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe234065850 0x7fe214009e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:31.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:31.916+0000 7fe2327fc700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 msgr2=0x7fe2140096c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-15T02:56:31.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:31.916+0000 7fe2327fc700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.190:3300/0 conn(0x7fe21400a860 0x7fe2140096c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:31.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:31.916+0000 7fe2327fc700 1 -- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 msgr2=0x7fe21400a350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-15T02:56:31.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:31.916+0000 7fe2327fc700 1 --2- 172.21.15.121:0/2882868007 >> v2:172.21.15.121:3300/0 conn(0x7fe2340a8b80 0x7fe21400a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:31.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:31.916+0000 7fe2327fc700 1 -- 172.21.15.121:0/2882868007 --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe21400ad80 con 0x7fe214006a10 2023-11-15T02:56:31.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:31.916+0000 7fe2327fc700 1 -- 172.21.15.121:0/2882868007 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe214006f90 con 0x7fe2140064a0 2023-11-15T02:56:33.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.088+0000 7fe2337fe700 1 -- 172.21.15.121:0/2882868007 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642204398 0 0) 0x7fe21400ad80 con 0x7fe214006a10 2023-11-15T02:56:33.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe2337fe700 1 -- 172.21.15.121:0/2882868007 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2200033f0 con 0x7fe214006a10 2023-11-15T02:56:33.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe2337fe700 1 -- 172.21.15.121:0/2882868007 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1189597711 0 0) 0x7fe2200033f0 con 0x7fe214006a10 2023-11-15T02:56:33.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe2337fe700 1 -- 172.21.15.121:0/2882868007 >> v1:172.21.15.190:6789/0 conn(0x7fe2140064a0 legacy=0x7fe2140059d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:33.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe2337fe700 1 -- 172.21.15.121:0/2882868007 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2140157f0 con 0x7fe214006a10 2023-11-15T02:56:33.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe2337fe700 1 -- 172.21.15.121:0/2882868007 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fe21c017580 con 0x7fe214006a10 2023-11-15T02:56:33.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe2337fe700 1 -- 172.21.15.121:0/2882868007 >> v1:172.21.15.121:6789/0 conn(0x7fe214006a10 legacy=0x7fe2140120b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:33.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe238e38700 1 -- 172.21.15.121:0/2882868007 reap_dead start 2023-11-15T02:56:33.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe2337fe700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe220003910 0x7fe220003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:33.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe2337fe700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe220004690 0x7fe220004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:33.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe2337fe700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe220005100 0x7fe22000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:33.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe2337fe700 1 -- 172.21.15.121:0/2882868007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22000c050 con 0x7fe220003910 2023-11-15T02:56:33.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe2337fe700 1 -- 172.21.15.121:0/2882868007 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22000c1c0 con 0x7fe220005100 2023-11-15T02:56:33.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe2337fe700 1 -- 172.21.15.121:0/2882868007 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22000c380 con 0x7fe220004690 2023-11-15T02:56:33.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe220003910 0x7fe220003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:33.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe238e38700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe220005100 0x7fe22000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:33.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe22bfff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe220004690 0x7fe220004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:33.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe220004690 msgr2=0x7fe220004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:33.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe220004690 0x7fe220004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:33.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe220005100 msgr2=0x7fe22000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:33.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe220005100 0x7fe22000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:33.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.092+0000 7fe233fff700 1 -- 172.21.15.121:0/2882868007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2200033f0 con 0x7fe220003910 2023-11-15T02:56:33.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.096+0000 7fe233fff700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe220003910 0x7fe220003cf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fe224018d20 tx=0x7fe224008040).ready entity=mon.0 client_cookie=cc278ba9bfe97d65 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:33.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.096+0000 7fe2337fe700 1 -- 172.21.15.121:0/2882868007 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22401d070 con 0x7fe220003910 2023-11-15T02:56:33.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.096+0000 7fe2337fe700 1 -- 172.21.15.121:0/2882868007 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe224018e80 con 0x7fe220003910 2023-11-15T02:56:33.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.096+0000 7fe2337fe700 1 -- 172.21.15.121:0/2882868007 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe224018760 con 0x7fe220003910 2023-11-15T02:56:33.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.096+0000 7fe23a89b700 1 -- 172.21.15.121:0/2882868007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe220003910 msgr2=0x7fe220003cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:33.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.096+0000 7fe23a89b700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe220003910 0x7fe220003cf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fe224018d20 tx=0x7fe224008040).stop 2023-11-15T02:56:33.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.096+0000 7fe23a89b700 1 -- 172.21.15.121:0/2882868007 shutdown_connections 2023-11-15T02:56:33.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.096+0000 7fe23a89b700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe220005100 0x7fe22000b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:33.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.096+0000 7fe23a89b700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe220004690 0x7fe220004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:33.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.096+0000 7fe23a89b700 1 --2- 172.21.15.121:0/2882868007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe220003910 0x7fe220003cf0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:33.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.096+0000 7fe23a89b700 1 -- 172.21.15.121:0/2882868007 >> 172.21.15.121:0/2882868007 conn(0x7fe2340591c0 msgr2=0x7fe2340c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:33.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.096+0000 7fe23a89b700 1 -- 172.21.15.121:0/2882868007 shutdown_connections 2023-11-15T02:56:33.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.096+0000 7fe23a89b700 1 -- 172.21.15.121:0/2882868007 wait complete. 2023-11-15T02:56:33.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.100+0000 7fe23a89b700 1 Processor -- start 2023-11-15T02:56:33.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.100+0000 7fe23a89b700 1 -- start start 2023-11-15T02:56:33.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.100+0000 7fe23a89b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe220003910 0x7fe234147610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:33.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.100+0000 7fe23a89b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe220004690 0x7fe23414df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:33.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.100+0000 7fe23a89b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe220005100 0x7fe234167940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:33.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.100+0000 7fe23a89b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe234154c20 con 0x7fe220005100 2023-11-15T02:56:33.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.100+0000 7fe23a89b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe234154790 con 0x7fe220004690 2023-11-15T02:56:33.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.100+0000 7fe23a89b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe234153b80 con 0x7fe220003910 2023-11-15T02:56:33.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.100+0000 7fe233fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe220003910 0x7fe234147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:33.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.100+0000 7fe238e38700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe220005100 0x7fe234167940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:33.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.100+0000 7fe233fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe220003910 0x7fe234147610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:36278/0 (socket says 172.21.15.121:36278) 2023-11-15T02:56:33.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.100+0000 7fe233fff700 1 -- 172.21.15.121:0/1640200045 learned_addr learned my addr 172.21.15.121:0/1640200045 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:33.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.100+0000 7fe22bfff700 1 --2- 172.21.15.121:0/1640200045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe220004690 0x7fe23414df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:33.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.104+0000 7fe238e38700 1 -- 172.21.15.121:0/1640200045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe220003910 msgr2=0x7fe234147610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:33.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.104+0000 7fe238e38700 1 --2- 172.21.15.121:0/1640200045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe220003910 0x7fe234147610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:33.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.104+0000 7fe238e38700 1 -- 172.21.15.121:0/1640200045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe220004690 msgr2=0x7fe23414df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:33.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.104+0000 7fe238e38700 1 --2- 172.21.15.121:0/1640200045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe220004690 0x7fe23414df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:33.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.104+0000 7fe238e38700 1 -- 172.21.15.121:0/1640200045 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe234153e60 con 0x7fe220005100 2023-11-15T02:56:33.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.104+0000 7fe238e38700 1 --2- 172.21.15.121:0/1640200045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe220005100 0x7fe234167940 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fe23414fba0 tx=0x7fe22c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:33.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.104+0000 7fe230ff9700 1 -- 172.21.15.121:0/1640200045 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c0030d0 con 0x7fe220005100 2023-11-15T02:56:33.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.104+0000 7fe230ff9700 1 -- 172.21.15.121:0/1640200045 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22c013ad0 con 0x7fe220005100 2023-11-15T02:56:33.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.104+0000 7fe230ff9700 1 -- 172.21.15.121:0/1640200045 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c013e90 con 0x7fe220005100 2023-11-15T02:56:33.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.104+0000 7fe23a89b700 1 -- 172.21.15.121:0/1640200045 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe234154140 con 0x7fe220005100 2023-11-15T02:56:33.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.104+0000 7fe23a89b700 1 -- 172.21.15.121:0/1640200045 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe234147b00 con 0x7fe220005100 2023-11-15T02:56:33.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.104+0000 7fe23a89b700 1 -- 172.21.15.121:0/1640200045 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe234154920 con 0x7fe220005100 2023-11-15T02:56:33.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.108+0000 7fe230ff9700 1 -- 172.21.15.121:0/1640200045 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fe22c0019f0 con 0x7fe220005100 2023-11-15T02:56:33.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.108+0000 7fe230ff9700 1 -- 172.21.15.121:0/1640200045 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fe22c02f510 con 0x7fe220005100 2023-11-15T02:56:33.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.112+0000 7fe230ff9700 1 -- 172.21.15.121:0/1640200045 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fe22c001d90 con 0x7fe220005100 2023-11-15T02:56:33.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.180+0000 7fe23a89b700 1 -- 172.21.15.121:0/1640200045 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fe234047060 con 0x7fe220005100 2023-11-15T02:56:33.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.188+0000 7fe230ff9700 1 -- 172.21.15.121:0/1640200045 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe22c033090 con 0x7fe220005100 2023-11-15T02:56:33.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.188+0000 7fe23a89b700 1 -- 172.21.15.121:0/1640200045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe220005100 msgr2=0x7fe234167940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:33.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.188+0000 7fe23a89b700 1 --2- 172.21.15.121:0/1640200045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe220005100 0x7fe234167940 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fe23414fba0 tx=0x7fe22c009040).stop 2023-11-15T02:56:33.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.188+0000 7fe23a89b700 1 -- 172.21.15.121:0/1640200045 shutdown_connections 2023-11-15T02:56:33.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.188+0000 7fe23a89b700 1 --2- 172.21.15.121:0/1640200045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe220005100 0x7fe234167940 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:33.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.188+0000 7fe23a89b700 1 --2- 172.21.15.121:0/1640200045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe220004690 0x7fe23414df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:33.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.188+0000 7fe23a89b700 1 --2- 172.21.15.121:0/1640200045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe220003910 0x7fe234147610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:33.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.188+0000 7fe23a89b700 1 -- 172.21.15.121:0/1640200045 >> 172.21.15.121:0/1640200045 conn(0x7fe2340591c0 msgr2=0x7fe2340c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:33.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.188+0000 7fe23a89b700 1 -- 172.21.15.121:0/1640200045 shutdown_connections 2023-11-15T02:56:33.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:33.188+0000 7fe23a89b700 1 -- 172.21.15.121:0/1640200045 wait complete. 2023-11-15T02:56:33.196 INFO:teuthology.orchestra.run.smithi121.stderr:adjusted tunables profile to default 2023-11-15T02:56:33.213 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-15T02:56:33.214 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:56:33.214 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-15T02:56:33.253 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:56:33.253 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-15T02:56:33.321 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:33.316+0000 7f4aacf5e500 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-15T02:56:33.337 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:56:33.338 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-15T02:56:33.364 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:33.357+0000 7f0f8d344500 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-15T02:56:33.406 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:33.400+0000 7f4aacf5e500 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-15T02:56:33.410 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-11-15T02:56:33.411 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-15T02:56:33.446 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:33.441+0000 7f0f8d344500 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-15T02:56:33.488 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:56:33.488 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-15T02:56:33.516 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:56:33.517 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-15T02:56:33.591 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:56:33.591 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-15T02:56:33.664 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-11-15T02:56:33.664 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-15T02:56:33.743 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47 0 2023-11-15T02:56:33.942 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.937+0000 7efc329b9700 1 Processor -- start 2023-11-15T02:56:33.942 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.937+0000 7efc329b9700 1 -- start start 2023-11-15T02:56:33.945 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.941+0000 7efc329b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc2c0acb60 0x7efc2c0acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:33.945 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.941+0000 7efc329b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc2c0c7e90 con 0x7efc2c065680 2023-11-15T02:56:33.945 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.941+0000 7efc329b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc2c0ad4d0 con 0x7efc2c14e380 2023-11-15T02:56:33.946 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.941+0000 7efc329b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c0c8320 con 0x7efc2c0acb60 2023-11-15T02:56:33.946 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.941+0000 7efc319b7700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7efc2c14e380 0x7efc2c0c5b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:49306/0 (socket says 172.21.15.190:49306) 2023-11-15T02:56:33.946 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.941+0000 7efc319b7700 1 -- 172.21.15.190:0/956420393 learned_addr learned my addr 172.21.15.190:0/956420393 (peer_addr_for_me v1:172.21.15.190:0/0) 2023-11-15T02:56:33.947 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.941+0000 7efc311b6700 1 --2- 172.21.15.190:0/956420393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc2c0acb60 0x7efc2c0acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:33.948 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.941+0000 7efc309b5700 1 -- 172.21.15.190:0/956420393 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3396711152 0 0) 0x7efc2c0ad4d0 con 0x7efc2c14e380 2023-11-15T02:56:33.948 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.941+0000 7efc309b5700 1 -- 172.21.15.190:0/956420393 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc180033f0 con 0x7efc2c14e380 2023-11-15T02:56:33.949 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.941+0000 7efc309b5700 1 -- 172.21.15.190:0/956420393 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2324450276 0 0) 0x7efc2c0c7e90 con 0x7efc2c065680 2023-11-15T02:56:33.949 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.941+0000 7efc309b5700 1 -- 172.21.15.190:0/956420393 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc2c0ad4d0 con 0x7efc2c065680 2023-11-15T02:56:33.949 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.941+0000 7efc309b5700 1 -- 172.21.15.190:0/956420393 <== mon.1 v1:172.21.15.190:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3679815403 0 0) 0x7efc180033f0 con 0x7efc2c14e380 2023-11-15T02:56:33.949 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.941+0000 7efc311b6700 1 -- 172.21.15.190:0/956420393 >> v1:172.21.15.190:6789/0 conn(0x7efc2c14e380 legacy=0x7efc2c0c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:33.949 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.941+0000 7efc311b6700 1 -- 172.21.15.190:0/956420393 >> v1:172.21.15.121:6789/0 conn(0x7efc2c065680 legacy=0x7efc2c065b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:33.950 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.941+0000 7efc311b6700 1 -- 172.21.15.190:0/956420393 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2c14e760 con 0x7efc2c0acb60 2023-11-15T02:56:33.950 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.941+0000 7efc311b6700 1 --2- 172.21.15.190:0/956420393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc2c0acb60 0x7efc2c0acfe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7efc1c009710 tx=0x7efc1c009060).ready entity=mon.2 client_cookie=431a04b1b5ed7b3c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:33.950 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.941+0000 7efc309b5700 1 -- 172.21.15.190:0/956420393 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc280017f0 con 0x7efc2c0acb60 2023-11-15T02:56:33.951 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.941+0000 7efc309b5700 1 -- 172.21.15.190:0/956420393 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c011400 con 0x7efc2c0acb60 2023-11-15T02:56:33.951 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.941+0000 7efc309b5700 1 -- 172.21.15.190:0/956420393 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c011780 con 0x7efc2c0acb60 2023-11-15T02:56:33.952 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.945+0000 7efc329b9700 1 -- 172.21.15.190:0/956420393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc2c0acb60 msgr2=0x7efc2c0acfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:33.953 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.945+0000 7efc329b9700 1 --2- 172.21.15.190:0/956420393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc2c0acb60 0x7efc2c0acfe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7efc1c009710 tx=0x7efc1c009060).stop 2023-11-15T02:56:33.953 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.945+0000 7efc329b9700 1 -- 172.21.15.190:0/956420393 shutdown_connections 2023-11-15T02:56:33.953 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.945+0000 7efc329b9700 1 --2- 172.21.15.190:0/956420393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc2c0acb60 0x7efc2c0acfe0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:33.953 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.945+0000 7efc329b9700 1 -- 172.21.15.190:0/956420393 >> 172.21.15.190:0/956420393 conn(0x7efc2c067a60 msgr2=0x7efc2c067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:33.954 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.945+0000 7efc329b9700 1 -- 172.21.15.190:0/956420393 shutdown_connections 2023-11-15T02:56:33.954 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.945+0000 7efc329b9700 1 -- 172.21.15.190:0/956420393 wait complete. 2023-11-15T02:56:33.957 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc329b9700 1 Processor -- start 2023-11-15T02:56:33.957 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc329b9700 1 -- start start 2023-11-15T02:56:33.958 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc329b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc2c065680 0x7efc2c167620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:33.958 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc329b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc2c0acb60 0x7efc2c167b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:33.958 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc329b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc2c14e380 0x7efc2c14a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:33.958 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc329b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c168000 con 0x7efc2c14e380 2023-11-15T02:56:33.958 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc329b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c1544e0 con 0x7efc2c0acb60 2023-11-15T02:56:33.959 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc329b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c14ae20 con 0x7efc2c065680 2023-11-15T02:56:33.959 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc311b6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc2c0acb60 0x7efc2c167b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:33.959 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc319b7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc2c065680 0x7efc2c167620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:33.959 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc311b6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc2c0acb60 0x7efc2c167b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:60328/0 (socket says 172.21.15.190:60328) 2023-11-15T02:56:33.959 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc311b6700 1 -- 172.21.15.190:0/3496513109 learned_addr learned my addr 172.21.15.190:0/3496513109 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-11-15T02:56:33.960 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc321b8700 1 --2- 172.21.15.190:0/3496513109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc2c14e380 0x7efc2c14a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:33.960 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc319b7700 1 -- 172.21.15.190:0/3496513109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc2c0acb60 msgr2=0x7efc2c167b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:33.960 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc319b7700 1 --2- 172.21.15.190:0/3496513109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc2c0acb60 0x7efc2c167b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:33.960 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc319b7700 1 -- 172.21.15.190:0/3496513109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc2c14e380 msgr2=0x7efc2c14a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:33.960 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc319b7700 1 --2- 172.21.15.190:0/3496513109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc2c14e380 0x7efc2c14a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:33.961 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc319b7700 1 -- 172.21.15.190:0/3496513109 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c0092a0 con 0x7efc2c065680 2023-11-15T02:56:33.961 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc311b6700 1 --2- 172.21.15.190:0/3496513109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc2c0acb60 0x7efc2c167b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:56:33.961 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc319b7700 1 --2- 172.21.15.190:0/3496513109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc2c065680 0x7efc2c167620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7efc2800cb60 tx=0x7efc28001e50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:33.961 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.949+0000 7efc21ffb700 1 -- 172.21.15.190:0/3496513109 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc28004c40 con 0x7efc2c065680 2023-11-15T02:56:33.961 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.953+0000 7efc21ffb700 1 -- 172.21.15.190:0/3496513109 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc28004dc0 con 0x7efc2c065680 2023-11-15T02:56:33.962 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.953+0000 7efc21ffb700 1 -- 172.21.15.190:0/3496513109 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc280073c0 con 0x7efc2c065680 2023-11-15T02:56:33.962 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.953+0000 7efc329b9700 1 -- 172.21.15.190:0/3496513109 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2c14b100 con 0x7efc2c065680 2023-11-15T02:56:33.962 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.953+0000 7efc21ffb700 1 -- 172.21.15.190:0/3496513109 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7efc2800a550 con 0x7efc2c065680 2023-11-15T02:56:33.962 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.953+0000 7efc329b9700 1 -- 172.21.15.190:0/3496513109 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2c14b6c0 con 0x7efc2c065680 2023-11-15T02:56:33.962 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.953+0000 7efc21ffb700 1 -- 172.21.15.190:0/3496513109 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7efc28014460 con 0x7efc2c065680 2023-11-15T02:56:33.963 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.953+0000 7efc329b9700 1 -- 172.21.15.190:0/3496513109 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc2c06bcf0 con 0x7efc2c065680 2023-11-15T02:56:33.963 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:33.957+0000 7efc21ffb700 1 -- 172.21.15.190:0/3496513109 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7efc28019020 con 0x7efc2c065680 2023-11-15T02:56:34.045 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.041+0000 7efc329b9700 1 -- 172.21.15.190:0/3496513109 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47", "id": 0} v 0) v1 -- 0x7efc2c068a70 con 0x7efc2c065680 2023-11-15T02:56:34.053 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.049+0000 7efc21ffb700 1 -- 172.21.15.190:0/3496513109 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7efc280146a0 con 0x7efc2c065680 2023-11-15T02:56:34.053 INFO:teuthology.orchestra.run.smithi190.stdout:0 2023-11-15T02:56:34.054 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.049+0000 7efc329b9700 1 -- 172.21.15.190:0/3496513109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc2c065680 msgr2=0x7efc2c167620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:34.055 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.049+0000 7efc329b9700 1 --2- 172.21.15.190:0/3496513109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc2c065680 0x7efc2c167620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7efc2800cb60 tx=0x7efc28001e50).stop 2023-11-15T02:56:34.055 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.049+0000 7efc329b9700 1 -- 172.21.15.190:0/3496513109 shutdown_connections 2023-11-15T02:56:34.055 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.049+0000 7efc329b9700 1 --2- 172.21.15.190:0/3496513109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc2c14e380 0x7efc2c14a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.055 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.049+0000 7efc329b9700 1 --2- 172.21.15.190:0/3496513109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc2c0acb60 0x7efc2c167b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.056 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.049+0000 7efc329b9700 1 --2- 172.21.15.190:0/3496513109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc2c065680 0x7efc2c167620 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.056 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.049+0000 7efc329b9700 1 -- 172.21.15.190:0/3496513109 >> 172.21.15.190:0/3496513109 conn(0x7efc2c067a60 msgr2=0x7efc2c14c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:34.056 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.049+0000 7efc329b9700 1 -- 172.21.15.190:0/3496513109 shutdown_connections 2023-11-15T02:56:34.056 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.049+0000 7efc329b9700 1 -- 172.21.15.190:0/3496513109 wait complete. 2023-11-15T02:56:34.071 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new ae8fa944-2f30-418b-99f8-a24a5566d5e8 1 2023-11-15T02:56:34.104 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:34.100+0000 7f4aacf5e500 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-15T02:56:34.218 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.213+0000 7f111ee9b700 1 Processor -- start 2023-11-15T02:56:34.218 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.213+0000 7f111ee9b700 1 -- start start 2023-11-15T02:56:34.221 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.213+0000 7f111ee9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1118067ec0 0x7f11180682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:34.221 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.213+0000 7f111ee9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11180c10b0 0x7f1118062c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:34.221 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f111ee9b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180c0c20 con 0x7f1118063120 2023-11-15T02:56:34.221 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f111ee9b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1118158730 con 0x7f1118067ec0 2023-11-15T02:56:34.222 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f111ee9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11180c02f0 con 0x7f11180c10b0 2023-11-15T02:56:34.222 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f1117fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11180c10b0 0x7f1118062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:34.222 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f1117fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11180c10b0 0x7f1118062c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:38168/0 (socket says 172.21.15.190:38168) 2023-11-15T02:56:34.222 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f1117fff700 1 -- 172.21.15.190:0/4048147004 learned_addr learned my addr 172.21.15.190:0/4048147004 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-11-15T02:56:34.222 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f111cc37700 1 --2- 172.21.15.190:0/4048147004 >> v2:172.21.15.190:3300/0 conn(0x7f1118067ec0 0x7f11180682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:34.223 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f1117fff700 1 -- 172.21.15.190:0/4048147004 >> v2:172.21.15.190:3300/0 conn(0x7f1118067ec0 msgr2=0x7f11180682a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:34.223 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f1117fff700 1 --2- 172.21.15.190:0/4048147004 >> v2:172.21.15.190:3300/0 conn(0x7f1118067ec0 0x7f11180682a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.223 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f1117fff700 1 -- 172.21.15.190:0/4048147004 >> v1:172.21.15.121:6789/0 conn(0x7f1118063120 legacy=0x7f1118153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:34.223 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f1117fff700 1 -- 172.21.15.190:0/4048147004 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11180c05d0 con 0x7f11180c10b0 2023-11-15T02:56:34.223 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f111cc37700 1 --2- 172.21.15.190:0/4048147004 >> v2:172.21.15.190:3300/0 conn(0x7f1118067ec0 0x7f11180682a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:56:34.225 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f1117fff700 1 --2- 172.21.15.190:0/4048147004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11180c10b0 0x7f1118062c30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f1108006fd0 tx=0x7f11080090f0).ready entity=mon.2 client_cookie=75093cf98ee10cea server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:34.225 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f11177fe700 1 -- 172.21.15.190:0/4048147004 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110800b600 con 0x7f11180c10b0 2023-11-15T02:56:34.225 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f11177fe700 1 -- 172.21.15.190:0/4048147004 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110800b760 con 0x7f11180c10b0 2023-11-15T02:56:34.226 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f11177fe700 1 -- 172.21.15.190:0/4048147004 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110800bae0 con 0x7f11180c10b0 2023-11-15T02:56:34.226 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f111ee9b700 1 -- 172.21.15.190:0/4048147004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11180c10b0 msgr2=0x7f1118062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:34.226 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f111ee9b700 1 --2- 172.21.15.190:0/4048147004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11180c10b0 0x7f1118062c30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f1108006fd0 tx=0x7f11080090f0).stop 2023-11-15T02:56:34.226 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f111ee9b700 1 -- 172.21.15.190:0/4048147004 shutdown_connections 2023-11-15T02:56:34.226 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f111ee9b700 1 --2- 172.21.15.190:0/4048147004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11180c10b0 0x7f1118062c30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.227 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f111ee9b700 1 --2- 172.21.15.190:0/4048147004 >> v2:172.21.15.190:3300/0 conn(0x7f1118067ec0 0x7f11180682a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.227 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f111ee9b700 1 -- 172.21.15.190:0/4048147004 >> 172.21.15.190:0/4048147004 conn(0x7f11180cad40 msgr2=0x7f11180cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:34.227 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f111ee9b700 1 -- 172.21.15.190:0/4048147004 shutdown_connections 2023-11-15T02:56:34.227 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.217+0000 7f111ee9b700 1 -- 172.21.15.190:0/4048147004 wait complete. 2023-11-15T02:56:34.228 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111ee9b700 1 Processor -- start 2023-11-15T02:56:34.228 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111ee9b700 1 -- start start 2023-11-15T02:56:34.230 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111ee9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1118063120 0x7f1118141580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:34.231 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111ee9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1118067ec0 0x7f1118141e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:34.231 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111ee9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11180c10b0 0x7f1118157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:34.231 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111ee9b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11180c3930 con 0x7f11180c10b0 2023-11-15T02:56:34.231 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111ee9b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11180c2d20 con 0x7f1118067ec0 2023-11-15T02:56:34.232 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111ee9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11180c3000 con 0x7f1118063120 2023-11-15T02:56:34.232 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111cc37700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1118063120 0x7f1118141580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:34.232 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111cc37700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1118063120 0x7f1118141580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:38184/0 (socket says 172.21.15.190:38184) 2023-11-15T02:56:34.232 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111cc37700 1 -- 172.21.15.190:0/2922999774 learned_addr learned my addr 172.21.15.190:0/2922999774 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-11-15T02:56:34.232 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f1117fff700 1 --2- 172.21.15.190:0/2922999774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1118067ec0 0x7f1118141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:34.233 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111d438700 1 --2- 172.21.15.190:0/2922999774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11180c10b0 0x7f1118157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:34.233 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111cc37700 1 -- 172.21.15.190:0/2922999774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1118067ec0 msgr2=0x7f1118141e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:34.233 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111cc37700 1 --2- 172.21.15.190:0/2922999774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1118067ec0 0x7f1118141e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.233 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111cc37700 1 -- 172.21.15.190:0/2922999774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11180c10b0 msgr2=0x7f1118157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:34.233 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111cc37700 1 --2- 172.21.15.190:0/2922999774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11180c10b0 0x7f1118157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.234 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111cc37700 1 -- 172.21.15.190:0/2922999774 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1108009370 con 0x7f1118063120 2023-11-15T02:56:34.234 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111d438700 1 --2- 172.21.15.190:0/2922999774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11180c10b0 0x7f1118157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:56:34.234 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111cc37700 1 --2- 172.21.15.190:0/2922999774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1118063120 0x7f1118141580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f110c00c880 tx=0x7f110c008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:34.234 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f1114ff9700 1 -- 172.21.15.190:0/2922999774 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c010580 con 0x7f1118063120 2023-11-15T02:56:34.235 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f1114ff9700 1 -- 172.21.15.190:0/2922999774 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110c0106e0 con 0x7f1118063120 2023-11-15T02:56:34.235 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111ee9b700 1 -- 172.21.15.190:0/2922999774 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11180c32e0 con 0x7f1118063120 2023-11-15T02:56:34.235 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f1114ff9700 1 -- 172.21.15.190:0/2922999774 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c010a60 con 0x7f1118063120 2023-11-15T02:56:34.235 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f111ee9b700 1 -- 172.21.15.190:0/2922999774 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1118141a70 con 0x7f1118063120 2023-11-15T02:56:34.235 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f1114ff9700 1 -- 172.21.15.190:0/2922999774 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f110c01b460 con 0x7f1118063120 2023-11-15T02:56:34.236 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.221+0000 7f1114ff9700 1 -- 172.21.15.190:0/2922999774 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7f110c0117f0 con 0x7f1118063120 2023-11-15T02:56:34.236 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.225+0000 7f111ee9b700 1 -- 172.21.15.190:0/2922999774 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1118047060 con 0x7f1118063120 2023-11-15T02:56:34.236 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.225+0000 7f1114ff9700 1 -- 172.21.15.190:0/2922999774 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f110c01b8f0 con 0x7f1118063120 2023-11-15T02:56:34.243 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:34.237+0000 7f0f8d344500 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-15T02:56:34.322 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.317+0000 7f111ee9b700 1 -- 172.21.15.190:0/2922999774 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ae8fa944-2f30-418b-99f8-a24a5566d5e8", "id": 1} v 0) v1 -- 0x7f11180bffc0 con 0x7f1118063120 2023-11-15T02:56:34.330 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.325+0000 7f1114ff9700 1 -- 172.21.15.190:0/2922999774 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ae8fa944-2f30-418b-99f8-a24a5566d5e8", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f110c01ba90 con 0x7f1118063120 2023-11-15T02:56:34.330 INFO:teuthology.orchestra.run.smithi190.stdout:1 2023-11-15T02:56:34.332 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.325+0000 7f111ee9b700 1 -- 172.21.15.190:0/2922999774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1118063120 msgr2=0x7f1118141580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:34.332 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.325+0000 7f111ee9b700 1 --2- 172.21.15.190:0/2922999774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1118063120 0x7f1118141580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f110c00c880 tx=0x7f110c008040).stop 2023-11-15T02:56:34.332 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.325+0000 7f111ee9b700 1 -- 172.21.15.190:0/2922999774 shutdown_connections 2023-11-15T02:56:34.332 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.325+0000 7f111ee9b700 1 --2- 172.21.15.190:0/2922999774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11180c10b0 0x7f1118157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.332 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.325+0000 7f111ee9b700 1 --2- 172.21.15.190:0/2922999774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1118067ec0 0x7f1118141e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.333 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.325+0000 7f111ee9b700 1 --2- 172.21.15.190:0/2922999774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1118063120 0x7f1118141580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.333 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.325+0000 7f111ee9b700 1 -- 172.21.15.190:0/2922999774 >> 172.21.15.190:0/2922999774 conn(0x7f11180cad40 msgr2=0x7f11180adb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:34.333 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.325+0000 7f111ee9b700 1 -- 172.21.15.190:0/2922999774 shutdown_connections 2023-11-15T02:56:34.333 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.325+0000 7f111ee9b700 1 -- 172.21.15.190:0/2922999774 wait complete. 2023-11-15T02:56:34.349 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 4895f0ef-152e-40b2-99c5-5e12c33a05a8 2 2023-11-15T02:56:34.504 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.501+0000 7f10204db700 1 Processor -- start 2023-11-15T02:56:34.505 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.501+0000 7f10204db700 1 -- start start 2023-11-15T02:56:34.508 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.501+0000 7f10204db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1018067ec0 0x7f10180682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:34.509 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.501+0000 7f10204db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f10180c10b0 0x7f1018062c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:34.509 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.501+0000 7f10204db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1018063120 0x7f1018153700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:34.509 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.501+0000 7f10204db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1018063590 con 0x7f1018067ec0 2023-11-15T02:56:34.509 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.501+0000 7f10204db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10180c0c20 con 0x7f10180c10b0 2023-11-15T02:56:34.509 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.501+0000 7f10204db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1018158730 con 0x7f1018063120 2023-11-15T02:56:34.510 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.501+0000 7f101da76700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f10180c10b0 0x7f1018062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:34.510 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.501+0000 7f101da76700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f10180c10b0 0x7f1018062c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:60354/0 (socket says 172.21.15.190:60354) 2023-11-15T02:56:34.510 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.501+0000 7f101da76700 1 -- 172.21.15.190:0/2968089020 learned_addr learned my addr 172.21.15.190:0/2968089020 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-11-15T02:56:34.510 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.501+0000 7f101ea78700 1 --2- 172.21.15.190:0/2968089020 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1018063120 0x7f1018153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:34.510 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.501+0000 7f101e277700 1 --2- 172.21.15.190:0/2968089020 >> v2:172.21.15.121:3300/0 conn(0x7f1018067ec0 0x7f10180682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:34.511 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.505+0000 7f101da76700 1 -- 172.21.15.190:0/2968089020 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1018063120 msgr2=0x7f1018153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:34.511 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.505+0000 7f101da76700 1 --2- 172.21.15.190:0/2968089020 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1018063120 0x7f1018153700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.511 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.505+0000 7f101da76700 1 -- 172.21.15.190:0/2968089020 >> v2:172.21.15.121:3300/0 conn(0x7f1018067ec0 msgr2=0x7f10180682a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:34.511 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.505+0000 7f101da76700 1 --2- 172.21.15.190:0/2968089020 >> v2:172.21.15.121:3300/0 conn(0x7f1018067ec0 0x7f10180682a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.512 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.505+0000 7f101da76700 1 -- 172.21.15.190:0/2968089020 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10180c02f0 con 0x7f10180c10b0 2023-11-15T02:56:34.513 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.505+0000 7f101da76700 1 --2- 172.21.15.190:0/2968089020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10180c10b0 0x7f1018062c30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f1018068a10 tx=0x7f10080090f0).ready entity=mon.1 client_cookie=dbdc40a642c355a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:34.513 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.505+0000 7f101d275700 1 -- 172.21.15.190:0/2968089020 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100800f510 con 0x7f10180c10b0 2023-11-15T02:56:34.513 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.505+0000 7f101d275700 1 -- 172.21.15.190:0/2968089020 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100800f670 con 0x7f10180c10b0 2023-11-15T02:56:34.513 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.505+0000 7f101d275700 1 -- 172.21.15.190:0/2968089020 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100800fa10 con 0x7f10180c10b0 2023-11-15T02:56:34.514 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.505+0000 7f10204db700 1 -- 172.21.15.190:0/2968089020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10180c10b0 msgr2=0x7f1018062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:34.514 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.505+0000 7f10204db700 1 --2- 172.21.15.190:0/2968089020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10180c10b0 0x7f1018062c30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f1018068a10 tx=0x7f10080090f0).stop 2023-11-15T02:56:34.514 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.505+0000 7f10204db700 1 -- 172.21.15.190:0/2968089020 shutdown_connections 2023-11-15T02:56:34.514 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.505+0000 7f10204db700 1 --2- 172.21.15.190:0/2968089020 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1018063120 0x7f1018153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.515 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.505+0000 7f10204db700 1 --2- 172.21.15.190:0/2968089020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10180c10b0 0x7f1018062c30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.515 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.505+0000 7f10204db700 1 --2- 172.21.15.190:0/2968089020 >> v2:172.21.15.121:3300/0 conn(0x7f1018067ec0 0x7f10180682a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.515 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.505+0000 7f10204db700 1 -- 172.21.15.190:0/2968089020 >> 172.21.15.190:0/2968089020 conn(0x7f10180cad40 msgr2=0x7f10180cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:34.515 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.505+0000 7f10204db700 1 -- 172.21.15.190:0/2968089020 shutdown_connections 2023-11-15T02:56:34.515 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.505+0000 7f10204db700 1 -- 172.21.15.190:0/2968089020 wait complete. 2023-11-15T02:56:34.517 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.509+0000 7f10204db700 1 Processor -- start 2023-11-15T02:56:34.517 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.509+0000 7f10204db700 1 -- start start 2023-11-15T02:56:34.518 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.509+0000 7f10204db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1018063120 0x7f1018141580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:34.518 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.509+0000 7f10204db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1018067ec0 0x7f1018156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:34.518 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.509+0000 7f10204db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10180c10b0 0x7f1018156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:34.518 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f10204db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1018156a40 con 0x7f1018063120 2023-11-15T02:56:34.518 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f10204db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10181545b0 con 0x7f1018067ec0 2023-11-15T02:56:34.519 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f10204db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10180c43c0 con 0x7f10180c10b0 2023-11-15T02:56:34.519 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f101da76700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1018067ec0 0x7f1018156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:34.519 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f101ea78700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10180c10b0 0x7f1018156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:34.519 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f101da76700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1018067ec0 0x7f1018156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:60368/0 (socket says 172.21.15.190:60368) 2023-11-15T02:56:34.519 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f101da76700 1 -- 172.21.15.190:0/2610905988 learned_addr learned my addr 172.21.15.190:0/2610905988 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-11-15T02:56:34.520 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f101e277700 1 --2- 172.21.15.190:0/2610905988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1018063120 0x7f1018141580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:34.520 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f101da76700 1 -- 172.21.15.190:0/2610905988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10180c10b0 msgr2=0x7f1018156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:34.520 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f101da76700 1 --2- 172.21.15.190:0/2610905988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10180c10b0 0x7f1018156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.520 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f101da76700 1 -- 172.21.15.190:0/2610905988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1018063120 msgr2=0x7f1018141580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:34.520 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f101da76700 1 --2- 172.21.15.190:0/2610905988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1018063120 0x7f1018141580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.521 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f101da76700 1 -- 172.21.15.190:0/2610905988 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1008009370 con 0x7f1018067ec0 2023-11-15T02:56:34.521 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f101da76700 1 --2- 172.21.15.190:0/2610905988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1018067ec0 0x7f1018156060 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f1008006fd0 tx=0x7f1008017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:34.522 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f100e7fc700 1 -- 172.21.15.190:0/2610905988 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1008000bc0 con 0x7f1018067ec0 2023-11-15T02:56:34.522 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f100e7fc700 1 -- 172.21.15.190:0/2610905988 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100800f7e0 con 0x7f1018067ec0 2023-11-15T02:56:34.522 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f100e7fc700 1 -- 172.21.15.190:0/2610905988 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1008000d20 con 0x7f1018067ec0 2023-11-15T02:56:34.522 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f10204db700 1 -- 172.21.15.190:0/2610905988 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10180c46a0 con 0x7f1018067ec0 2023-11-15T02:56:34.523 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f10204db700 1 -- 172.21.15.190:0/2610905988 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1018153f40 con 0x7f1018067ec0 2023-11-15T02:56:34.523 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f100e7fc700 1 -- 172.21.15.190:0/2610905988 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f1008022020 con 0x7f1018067ec0 2023-11-15T02:56:34.523 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f100e7fc700 1 -- 172.21.15.190:0/2610905988 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7f100800f970 con 0x7f1018067ec0 2023-11-15T02:56:34.523 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.513+0000 7f10204db700 1 -- 172.21.15.190:0/2610905988 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1018047060 con 0x7f1018067ec0 2023-11-15T02:56:34.524 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.517+0000 7f100e7fc700 1 -- 172.21.15.190:0/2610905988 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f1008044e50 con 0x7f1018067ec0 2023-11-15T02:56:34.617 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.613+0000 7f10204db700 1 -- 172.21.15.190:0/2610905988 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4895f0ef-152e-40b2-99c5-5e12c33a05a8", "id": 2} v 0) v1 -- 0x7f1018062f00 con 0x7f1018067ec0 2023-11-15T02:56:34.625 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.621+0000 7f100e7fc700 1 -- 172.21.15.190:0/2610905988 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4895f0ef-152e-40b2-99c5-5e12c33a05a8", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1008033030 con 0x7f1018067ec0 2023-11-15T02:56:34.626 INFO:teuthology.orchestra.run.smithi190.stdout:2 2023-11-15T02:56:34.628 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.621+0000 7f10204db700 1 -- 172.21.15.190:0/2610905988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1018067ec0 msgr2=0x7f1018156060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:34.629 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.621+0000 7f10204db700 1 --2- 172.21.15.190:0/2610905988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1018067ec0 0x7f1018156060 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f1008006fd0 tx=0x7f1008017000).stop 2023-11-15T02:56:34.629 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.621+0000 7f10204db700 1 -- 172.21.15.190:0/2610905988 shutdown_connections 2023-11-15T02:56:34.629 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.621+0000 7f10204db700 1 --2- 172.21.15.190:0/2610905988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10180c10b0 0x7f1018156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.630 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.621+0000 7f10204db700 1 --2- 172.21.15.190:0/2610905988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1018067ec0 0x7f1018156060 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.630 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.621+0000 7f10204db700 1 --2- 172.21.15.190:0/2610905988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1018063120 0x7f1018141580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.630 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.621+0000 7f10204db700 1 -- 172.21.15.190:0/2610905988 >> 172.21.15.190:0/2610905988 conn(0x7f10180cad40 msgr2=0x7f10180adb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:34.631 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.621+0000 7f10204db700 1 -- 172.21.15.190:0/2610905988 shutdown_connections 2023-11-15T02:56:34.631 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.621+0000 7f10204db700 1 -- 172.21.15.190:0/2610905988 wait complete. 2023-11-15T02:56:34.646 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 378df070-0c7f-4a3b-8321-6f7a463b3aa7 3 2023-11-15T02:56:34.779 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:34.776+0000 7f4aacf5e500 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-15T02:56:34.794 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.789+0000 7fa911579700 1 Processor -- start 2023-11-15T02:56:34.794 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.789+0000 7fa911579700 1 -- start start 2023-11-15T02:56:34.797 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa911579700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa90c151af0 0x7fa90c0a7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:34.797 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa911579700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90c1528b0 con 0x7fa90c0a5720 2023-11-15T02:56:34.797 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa911579700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90c151ff0 con 0x7fa90c0c3650 2023-11-15T02:56:34.797 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa911579700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa90c152260 con 0x7fa90c151af0 2023-11-15T02:56:34.799 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa910d78700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fa90c0c3650 0x7fa90c065d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:49314/0 (socket says 172.21.15.190:49314) 2023-11-15T02:56:34.799 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa910d78700 1 -- 172.21.15.190:0/4089426979 learned_addr learned my addr 172.21.15.190:0/4089426979 (peer_addr_for_me v1:172.21.15.190:0/0) 2023-11-15T02:56:34.800 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa90bfff700 1 --2- 172.21.15.190:0/4089426979 >> v2:172.21.15.121:3300/0 conn(0x7fa90c151af0 0x7fa90c0a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:34.800 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa90b7fe700 1 -- 172.21.15.190:0/4089426979 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 948460344 0 0) 0x7fa90c151ff0 con 0x7fa90c0c3650 2023-11-15T02:56:34.800 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa90b7fe700 1 -- 172.21.15.190:0/4089426979 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8f80033f0 con 0x7fa90c0c3650 2023-11-15T02:56:34.800 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa90b7fe700 1 -- 172.21.15.190:0/4089426979 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449689759 0 0) 0x7fa90c1528b0 con 0x7fa90c0a5720 2023-11-15T02:56:34.800 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa90b7fe700 1 -- 172.21.15.190:0/4089426979 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa90c151ff0 con 0x7fa90c0a5720 2023-11-15T02:56:34.801 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa90b7fe700 1 -- 172.21.15.190:0/4089426979 <== mon.1 v1:172.21.15.190:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3538704755 0 0) 0x7fa8f80033f0 con 0x7fa90c0c3650 2023-11-15T02:56:34.801 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa90b7fe700 1 -- 172.21.15.190:0/4089426979 >> v2:172.21.15.121:3300/0 conn(0x7fa90c151af0 msgr2=0x7fa90c0a7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:34.801 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa90b7fe700 1 --2- 172.21.15.190:0/4089426979 >> v2:172.21.15.121:3300/0 conn(0x7fa90c151af0 0x7fa90c0a7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.801 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa90b7fe700 1 -- 172.21.15.190:0/4089426979 >> v1:172.21.15.121:6789/0 conn(0x7fa90c0a5720 legacy=0x7fa90c0c2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:34.801 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa90b7fe700 1 -- 172.21.15.190:0/4089426979 --> v1:172.21.15.190:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c152540 con 0x7fa90c0c3650 2023-11-15T02:56:34.802 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa90b7fe700 1 -- 172.21.15.190:0/4089426979 <== mon.1 v1:172.21.15.190:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fa9040017f0 con 0x7fa90c0c3650 2023-11-15T02:56:34.802 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa90b7fe700 1 -- 172.21.15.190:0/4089426979 >> v1:172.21.15.190:6789/0 conn(0x7fa90c0c3650 legacy=0x7fa90c065d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:34.802 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa90b7fe700 1 --2- 172.21.15.190:0/4089426979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f8004ad0 0x7fa8f8004ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:34.802 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa90b7fe700 1 --2- 172.21.15.190:0/4089426979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8f8003630 0x7fa8f8003ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:34.802 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa90b7fe700 1 --2- 172.21.15.190:0/4089426979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8f8003fa0 0x7fa8f800fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:34.803 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa90b7fe700 1 -- 172.21.15.190:0/4089426979 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c152260 con 0x7fa8f8003630 2023-11-15T02:56:34.803 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa90b7fe700 1 -- 172.21.15.190:0/4089426979 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f8010290 con 0x7fa8f8004ad0 2023-11-15T02:56:34.803 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa90b7fe700 1 -- 172.21.15.190:0/4089426979 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f8010450 con 0x7fa8f8003fa0 2023-11-15T02:56:34.803 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa910d78700 1 --2- 172.21.15.190:0/4089426979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8f8003fa0 0x7fa8f800fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:34.803 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa90bfff700 1 --2- 172.21.15.190:0/4089426979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f8004ad0 0x7fa8f8004ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:34.804 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.793+0000 7fa903fff700 1 --2- 172.21.15.190:0/4089426979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8f8003630 0x7fa8f8003ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:34.807 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.797+0000 7fa910d78700 1 -- 172.21.15.190:0/4089426979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f8004ad0 msgr2=0x7fa8f8004ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:34.807 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.797+0000 7fa910d78700 1 --2- 172.21.15.190:0/4089426979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f8004ad0 0x7fa8f8004ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.807 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.797+0000 7fa910d78700 1 -- 172.21.15.190:0/4089426979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8f8003630 msgr2=0x7fa8f8003ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:34.807 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.797+0000 7fa910d78700 1 --2- 172.21.15.190:0/4089426979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8f8003630 0x7fa8f8003ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.808 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.797+0000 7fa910d78700 1 -- 172.21.15.190:0/4089426979 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f80033f0 con 0x7fa8f8003fa0 2023-11-15T02:56:34.808 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.797+0000 7fa910d78700 1 -- 172.21.15.190:0/4089426979 reap_dead start 2023-11-15T02:56:34.808 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.797+0000 7fa910d78700 1 --2- 172.21.15.190:0/4089426979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8f8003fa0 0x7fa8f800fcf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fa90c0c3a50 tx=0x7fa904007f40).ready entity=mon.2 client_cookie=2cba0e7fa3342088 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:34.808 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.797+0000 7fa90b7fe700 1 -- 172.21.15.190:0/4089426979 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90400cd50 con 0x7fa8f8003fa0 2023-11-15T02:56:34.808 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.797+0000 7fa90b7fe700 1 -- 172.21.15.190:0/4089426979 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa904005b20 con 0x7fa8f8003fa0 2023-11-15T02:56:34.809 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.797+0000 7fa90b7fe700 1 -- 172.21.15.190:0/4089426979 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa904005ea0 con 0x7fa8f8003fa0 2023-11-15T02:56:34.809 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.797+0000 7fa911579700 1 -- 172.21.15.190:0/4089426979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8f8003fa0 msgr2=0x7fa8f800fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:34.809 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.797+0000 7fa911579700 1 --2- 172.21.15.190:0/4089426979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8f8003fa0 0x7fa8f800fcf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fa90c0c3a50 tx=0x7fa904007f40).stop 2023-11-15T02:56:34.809 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.797+0000 7fa911579700 1 -- 172.21.15.190:0/4089426979 shutdown_connections 2023-11-15T02:56:34.809 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.797+0000 7fa911579700 1 --2- 172.21.15.190:0/4089426979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8f8003fa0 0x7fa8f800fcf0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.809 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.797+0000 7fa911579700 1 -- 172.21.15.190:0/4089426979 >> 172.21.15.190:0/4089426979 conn(0x7fa90c067a60 msgr2=0x7fa90c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:34.810 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.797+0000 7fa911579700 1 -- 172.21.15.190:0/4089426979 shutdown_connections 2023-11-15T02:56:34.810 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.797+0000 7fa911579700 1 -- 172.21.15.190:0/4089426979 wait complete. 2023-11-15T02:56:34.810 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.797+0000 7fa911579700 1 Processor -- start 2023-11-15T02:56:34.810 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa911579700 1 -- start start 2023-11-15T02:56:34.810 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa911579700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f8003fa0 0x7fa90c14b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:34.811 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa911579700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa90c0c4310 0x7fa90c0a9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:34.811 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa911579700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa90c0c4790 0x7fa90c0664d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:34.811 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa911579700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c0ab670 con 0x7fa90c0c4790 2023-11-15T02:56:34.811 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa911579700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c0a9a30 con 0x7fa8f8003fa0 2023-11-15T02:56:34.812 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa911579700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c14be40 con 0x7fa90c0c4310 2023-11-15T02:56:34.812 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa903fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa90c0c4310 0x7fa90c0a9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:34.812 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa903fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa90c0c4310 0x7fa90c0a9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:38214/0 (socket says 172.21.15.190:38214) 2023-11-15T02:56:34.812 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa903fff700 1 -- 172.21.15.190:0/3803449362 learned_addr learned my addr 172.21.15.190:0/3803449362 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-11-15T02:56:34.812 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa90bfff700 1 --2- 172.21.15.190:0/3803449362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f8003fa0 0x7fa90c14b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:34.812 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa910d78700 1 --2- 172.21.15.190:0/3803449362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa90c0c4790 0x7fa90c0664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:34.813 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa90bfff700 1 -- 172.21.15.190:0/3803449362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa90c0c4310 msgr2=0x7fa90c0a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:34.813 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa90bfff700 1 --2- 172.21.15.190:0/3803449362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa90c0c4310 0x7fa90c0a9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.813 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa90bfff700 1 -- 172.21.15.190:0/3803449362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa90c0c4790 msgr2=0x7fa90c0664d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:34.813 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa90bfff700 1 --2- 172.21.15.190:0/3803449362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa90c0c4790 0x7fa90c0664d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.814 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa90bfff700 1 -- 172.21.15.190:0/3803449362 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c0c4cf0 con 0x7fa8f8003fa0 2023-11-15T02:56:34.814 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa903fff700 1 --2- 172.21.15.190:0/3803449362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa90c0c4310 0x7fa90c0a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:56:34.814 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa910d78700 1 --2- 172.21.15.190:0/3803449362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa90c0c4790 0x7fa90c0664d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:56:34.814 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa90bfff700 1 --2- 172.21.15.190:0/3803449362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f8003fa0 0x7fa90c14b950 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fa8fc0119f0 tx=0x7fa8fc009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:34.814 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa908ff9700 1 -- 172.21.15.190:0/3803449362 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc011e60 con 0x7fa8f8003fa0 2023-11-15T02:56:34.815 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa908ff9700 1 -- 172.21.15.190:0/3803449362 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc00f940 con 0x7fa8f8003fa0 2023-11-15T02:56:34.815 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa908ff9700 1 -- 172.21.15.190:0/3803449362 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc00fcc0 con 0x7fa8f8003fa0 2023-11-15T02:56:34.815 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa911579700 1 -- 172.21.15.190:0/3803449362 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa90c0c4fd0 con 0x7fa8f8003fa0 2023-11-15T02:56:34.815 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa911579700 1 -- 172.21.15.190:0/3803449362 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa90c0a96c0 con 0x7fa8f8003fa0 2023-11-15T02:56:34.816 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa908ff9700 1 -- 172.21.15.190:0/3803449362 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fa8fc0148b0 con 0x7fa8f8003fa0 2023-11-15T02:56:34.816 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.801+0000 7fa908ff9700 1 -- 172.21.15.190:0/3803449362 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7fa8fc014b60 con 0x7fa8f8003fa0 2023-11-15T02:56:34.816 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.805+0000 7fa911579700 1 -- 172.21.15.190:0/3803449362 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa90c06bcf0 con 0x7fa8f8003fa0 2023-11-15T02:56:34.817 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.805+0000 7fa908ff9700 1 -- 172.21.15.190:0/3803449362 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fa8fc03be50 con 0x7fa8f8003fa0 2023-11-15T02:56:34.904 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.901+0000 7fa911579700 1 -- 172.21.15.190:0/3803449362 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "378df070-0c7f-4a3b-8321-6f7a463b3aa7", "id": 3} v 0) v1 -- 0x7fa90c0a5f70 con 0x7fa8f8003fa0 2023-11-15T02:56:34.914 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.909+0000 7fa908ff9700 1 -- 172.21.15.190:0/3803449362 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "378df070-0c7f-4a3b-8321-6f7a463b3aa7", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa8fc02a030 con 0x7fa8f8003fa0 2023-11-15T02:56:34.914 INFO:teuthology.orchestra.run.smithi190.stdout:3 2023-11-15T02:56:34.915 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.909+0000 7fa911579700 1 -- 172.21.15.190:0/3803449362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f8003fa0 msgr2=0x7fa90c14b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:34.916 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.909+0000 7fa911579700 1 --2- 172.21.15.190:0/3803449362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f8003fa0 0x7fa90c14b950 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fa8fc0119f0 tx=0x7fa8fc009060).stop 2023-11-15T02:56:34.916 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.909+0000 7fa911579700 1 -- 172.21.15.190:0/3803449362 shutdown_connections 2023-11-15T02:56:34.916 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.909+0000 7fa911579700 1 --2- 172.21.15.190:0/3803449362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa90c0c4790 0x7fa90c0664d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.916 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.909+0000 7fa911579700 1 --2- 172.21.15.190:0/3803449362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa90c0c4310 0x7fa90c0a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.916 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.909+0000 7fa911579700 1 --2- 172.21.15.190:0/3803449362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f8003fa0 0x7fa90c14b950 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:34.917 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.909+0000 7fa911579700 1 -- 172.21.15.190:0/3803449362 >> 172.21.15.190:0/3803449362 conn(0x7fa90c067a60 msgr2=0x7fa90c14d680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:34.917 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.909+0000 7fa911579700 1 -- 172.21.15.190:0/3803449362 shutdown_connections 2023-11-15T02:56:34.917 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:34.909+0000 7fa911579700 1 -- 172.21.15.190:0/3803449362 wait complete. 2023-11-15T02:56:34.938 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 624feb7b-67da-4b57-b067-673923606388 4 2023-11-15T02:56:34.986 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:34.980+0000 7f4aacf5e500 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-15T02:56:35.027 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:35.021+0000 7f0f8d344500 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-15T02:56:35.083 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.077+0000 7f47d6e9b700 1 Processor -- start 2023-11-15T02:56:35.083 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.077+0000 7f47d6e9b700 1 -- start start 2023-11-15T02:56:35.085 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d6e9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d0157ed0 0x7f47d01582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:35.085 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d6e9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f47d0067a60 0x7f47d0067e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:35.085 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d6e9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f47d00bc0d0 0x7f47d00bc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:35.086 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d6e9b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d00a93a0 con 0x7f47d0067a60 2023-11-15T02:56:35.086 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d6e9b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d0062da0 con 0x7f47d00bc0d0 2023-11-15T02:56:35.086 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d6e9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d00bca70 con 0x7f47d0157ed0 2023-11-15T02:56:35.086 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d5438700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f47d00bc0d0 0x7f47d00bc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:35.087 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d4c37700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d0157ed0 0x7f47d01582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:35.087 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d5438700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f47d00bc0d0 0x7f47d00bc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:60394/0 (socket says 172.21.15.190:60394) 2023-11-15T02:56:35.087 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d5438700 1 -- 172.21.15.190:0/2442689975 learned_addr learned my addr 172.21.15.190:0/2442689975 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-11-15T02:56:35.087 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47c7fff700 1 --2- 172.21.15.190:0/2442689975 >> v2:172.21.15.121:3300/0 conn(0x7f47d0067a60 0x7f47d0067e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:35.089 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d5438700 1 -- 172.21.15.190:0/2442689975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d0157ed0 msgr2=0x7f47d01582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:35.089 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d5438700 1 --2- 172.21.15.190:0/2442689975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d0157ed0 0x7f47d01582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.090 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d5438700 1 -- 172.21.15.190:0/2442689975 >> v2:172.21.15.121:3300/0 conn(0x7f47d0067a60 msgr2=0x7f47d0067e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:35.090 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d5438700 1 --2- 172.21.15.190:0/2442689975 >> v2:172.21.15.121:3300/0 conn(0x7f47d0067a60 0x7f47d0067e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.090 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d5438700 1 -- 172.21.15.190:0/2442689975 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d0068370 con 0x7f47d00bc0d0 2023-11-15T02:56:35.090 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47c7fff700 1 --2- 172.21.15.190:0/2442689975 >> v2:172.21.15.121:3300/0 conn(0x7f47d0067a60 0x7f47d0067e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:56:35.091 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d5438700 1 --2- 172.21.15.190:0/2442689975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47d00bc0d0 0x7f47d00bc580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f47d00636c0 tx=0x7f47c8009840).ready entity=mon.1 client_cookie=233c32edfd772a3d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:35.091 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47c77fe700 1 -- 172.21.15.190:0/2442689975 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c800e070 con 0x7f47d00bc0d0 2023-11-15T02:56:35.091 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47c77fe700 1 -- 172.21.15.190:0/2442689975 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c800ba20 con 0x7f47d00bc0d0 2023-11-15T02:56:35.091 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47c77fe700 1 -- 172.21.15.190:0/2442689975 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c800bda0 con 0x7f47d00bc0d0 2023-11-15T02:56:35.091 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d6e9b700 1 -- 172.21.15.190:0/2442689975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47d00bc0d0 msgr2=0x7f47d00bc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:35.092 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d6e9b700 1 --2- 172.21.15.190:0/2442689975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47d00bc0d0 0x7f47d00bc580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f47d00636c0 tx=0x7f47c8009840).stop 2023-11-15T02:56:35.092 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d6e9b700 1 -- 172.21.15.190:0/2442689975 shutdown_connections 2023-11-15T02:56:35.092 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d6e9b700 1 --2- 172.21.15.190:0/2442689975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47d00bc0d0 0x7f47d00bc580 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.092 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d6e9b700 1 --2- 172.21.15.190:0/2442689975 >> v2:172.21.15.121:3300/0 conn(0x7f47d0067a60 0x7f47d0067e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.093 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d6e9b700 1 --2- 172.21.15.190:0/2442689975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d0157ed0 0x7f47d01582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.093 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d6e9b700 1 -- 172.21.15.190:0/2442689975 >> 172.21.15.190:0/2442689975 conn(0x7f47d00591c0 msgr2=0x7f47d00c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:35.093 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d6e9b700 1 -- 172.21.15.190:0/2442689975 shutdown_connections 2023-11-15T02:56:35.093 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.081+0000 7f47d6e9b700 1 -- 172.21.15.190:0/2442689975 wait complete. 2023-11-15T02:56:35.094 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47d6e9b700 1 Processor -- start 2023-11-15T02:56:35.094 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47d6e9b700 1 -- start start 2023-11-15T02:56:35.096 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47d6e9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47d0067a60 0x7f47d0149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:35.097 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47d6e9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47d00bc0d0 0x7f47d015a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:35.097 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47d6e9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d0157ed0 0x7f47d00bf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:35.097 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47d6e9b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d015bee0 con 0x7f47d0067a60 2023-11-15T02:56:35.097 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47d6e9b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d015b2d0 con 0x7f47d00bc0d0 2023-11-15T02:56:35.097 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47d6e9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d015b5b0 con 0x7f47d0157ed0 2023-11-15T02:56:35.098 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47c7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47d00bc0d0 0x7f47d015a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:35.098 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47c7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47d00bc0d0 0x7f47d015a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:60406/0 (socket says 172.21.15.190:60406) 2023-11-15T02:56:35.098 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47d5438700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d0157ed0 0x7f47d00bf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:35.098 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47c7fff700 1 -- 172.21.15.190:0/2156026658 learned_addr learned my addr 172.21.15.190:0/2156026658 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-11-15T02:56:35.099 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47d4c37700 1 --2- 172.21.15.190:0/2156026658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47d0067a60 0x7f47d0149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:35.099 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47c7fff700 1 -- 172.21.15.190:0/2156026658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d0157ed0 msgr2=0x7f47d00bf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:35.099 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47c7fff700 1 --2- 172.21.15.190:0/2156026658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d0157ed0 0x7f47d00bf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.099 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47c7fff700 1 -- 172.21.15.190:0/2156026658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47d0067a60 msgr2=0x7f47d0149a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:35.100 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47c7fff700 1 --2- 172.21.15.190:0/2156026658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47d0067a60 0x7f47d0149a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.100 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47c7fff700 1 -- 172.21.15.190:0/2156026658 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c8009ae0 con 0x7f47d00bc0d0 2023-11-15T02:56:35.100 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47c7fff700 1 --2- 172.21.15.190:0/2156026658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47d00bc0d0 0x7f47d015a580 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f47c000d860 tx=0x7f47c0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:35.100 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47c4ff9700 1 -- 172.21.15.190:0/2156026658 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c000f520 con 0x7f47d00bc0d0 2023-11-15T02:56:35.101 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47c4ff9700 1 -- 172.21.15.190:0/2156026658 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c000f680 con 0x7f47d00bc0d0 2023-11-15T02:56:35.101 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47c4ff9700 1 -- 172.21.15.190:0/2156026658 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c000fa00 con 0x7f47d00bc0d0 2023-11-15T02:56:35.101 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.085+0000 7f47d6e9b700 1 -- 172.21.15.190:0/2156026658 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d015b890 con 0x7f47d00bc0d0 2023-11-15T02:56:35.101 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.089+0000 7f47c4ff9700 1 -- 172.21.15.190:0/2156026658 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f47c00136e0 con 0x7f47d00bc0d0 2023-11-15T02:56:35.102 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.089+0000 7f47d6e9b700 1 -- 172.21.15.190:0/2156026658 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d0149f10 con 0x7f47d00bc0d0 2023-11-15T02:56:35.102 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.089+0000 7f47c4ff9700 1 -- 172.21.15.190:0/2156026658 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7f47c0013bc0 con 0x7f47d00bc0d0 2023-11-15T02:56:35.102 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.089+0000 7f47d6e9b700 1 -- 172.21.15.190:0/2156026658 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d0047060 con 0x7f47d00bc0d0 2023-11-15T02:56:35.102 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.093+0000 7f47c4ff9700 1 -- 172.21.15.190:0/2156026658 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f47c0018070 con 0x7f47d00bc0d0 2023-11-15T02:56:35.203 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.197+0000 7f47d6e9b700 1 -- 172.21.15.190:0/2156026658 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "624feb7b-67da-4b57-b067-673923606388", "id": 4} v 0) v1 -- 0x7f47d0068b50 con 0x7f47d00bc0d0 2023-11-15T02:56:35.216 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.209+0000 7f47c4ff9700 1 -- 172.21.15.190:0/2156026658 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "624feb7b-67da-4b57-b067-673923606388", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f47c0005880 con 0x7f47d00bc0d0 2023-11-15T02:56:35.216 INFO:teuthology.orchestra.run.smithi190.stdout:4 2023-11-15T02:56:35.217 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.213+0000 7f47d6e9b700 1 -- 172.21.15.190:0/2156026658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47d00bc0d0 msgr2=0x7f47d015a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:35.218 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.213+0000 7f47d6e9b700 1 --2- 172.21.15.190:0/2156026658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47d00bc0d0 0x7f47d015a580 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f47c000d860 tx=0x7f47c0008040).stop 2023-11-15T02:56:35.218 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.213+0000 7f47d6e9b700 1 -- 172.21.15.190:0/2156026658 shutdown_connections 2023-11-15T02:56:35.218 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.213+0000 7f47d6e9b700 1 --2- 172.21.15.190:0/2156026658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d0157ed0 0x7f47d00bf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.218 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.213+0000 7f47d6e9b700 1 --2- 172.21.15.190:0/2156026658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47d00bc0d0 0x7f47d015a580 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.218 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.213+0000 7f47d6e9b700 1 --2- 172.21.15.190:0/2156026658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47d0067a60 0x7f47d0149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.219 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.213+0000 7f47d6e9b700 1 -- 172.21.15.190:0/2156026658 >> 172.21.15.190:0/2156026658 conn(0x7f47d00591c0 msgr2=0x7f47d00abbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:35.219 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.213+0000 7f47d6e9b700 1 -- 172.21.15.190:0/2156026658 shutdown_connections 2023-11-15T02:56:35.219 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.213+0000 7f47d6e9b700 1 -- 172.21.15.190:0/2156026658 wait complete. 2023-11-15T02:56:35.234 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new cfe09666-0516-4e88-8983-4a9048d28a0c 5 2023-11-15T02:56:35.250 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:35.245+0000 7f0f8d344500 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-15T02:56:35.292 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:35.288+0000 7f4aacf5e500 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-15T02:56:35.382 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.377+0000 7f731b59e700 1 Processor -- start 2023-11-15T02:56:35.383 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.377+0000 7f731b59e700 1 -- start start 2023-11-15T02:56:35.385 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.381+0000 7f731b59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f731c068170 0x7f731c0bdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:35.385 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.381+0000 7f731b59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f731c0a4490 0x7f731c0a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:35.386 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.381+0000 7f731b59e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f731c0a57f0 con 0x7f731c157090 2023-11-15T02:56:35.386 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.381+0000 7f731b59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f731c068930 con 0x7f731c068170 2023-11-15T02:56:35.386 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.381+0000 7f731b59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731c0bd550 con 0x7f731c0a4490 2023-11-15T02:56:35.387 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.381+0000 7f731ad9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f731c0a4490 0x7f731c0a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:35.387 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.381+0000 7f731ad9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f731c0a4490 0x7f731c0a5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:38232/0 (socket says 172.21.15.190:38232) 2023-11-15T02:56:35.388 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.381+0000 7f731ad9d700 1 -- 172.21.15.190:0/829085975 learned_addr learned my addr 172.21.15.190:0/829085975 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-11-15T02:56:35.388 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.381+0000 7f7313fff700 1 --2- 172.21.15.190:0/829085975 >> v2:172.21.15.121:3300/0 conn(0x7f731c068170 0x7f731c0bdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:35.388 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.381+0000 7f731ad9d700 1 -- 172.21.15.190:0/829085975 >> v2:172.21.15.121:3300/0 conn(0x7f731c068170 msgr2=0x7f731c0bdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:35.388 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.381+0000 7f731ad9d700 1 --2- 172.21.15.190:0/829085975 >> v2:172.21.15.121:3300/0 conn(0x7f731c068170 0x7f731c0bdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.389 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.381+0000 7f731ad9d700 1 -- 172.21.15.190:0/829085975 >> v1:172.21.15.190:6789/0 conn(0x7f731c157090 legacy=0x7f731c067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:35.390 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.381+0000 7f7319d9b700 1 -- 172.21.15.190:0/829085975 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002424399 0 0) 0x7f731c0a57f0 con 0x7f731c157090 2023-11-15T02:56:35.390 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.381+0000 7f731ad9d700 1 -- 172.21.15.190:0/829085975 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c0a4be0 con 0x7f731c0a4490 2023-11-15T02:56:35.390 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.381+0000 7f731ad9d700 1 --2- 172.21.15.190:0/829085975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f731c0a4490 0x7f731c0a5c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f7314009f20 tx=0x7f7314009840).ready entity=mon.2 client_cookie=33510cf24f0e8d89 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:35.390 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.381+0000 7f7319d9b700 1 -- 172.21.15.190:0/829085975 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731400e070 con 0x7f731c0a4490 2023-11-15T02:56:35.390 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.381+0000 7f7319d9b700 1 -- 172.21.15.190:0/829085975 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731400b9d0 con 0x7f731c0a4490 2023-11-15T02:56:35.391 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f7319d9b700 1 -- 172.21.15.190:0/829085975 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731400bd50 con 0x7f731c0a4490 2023-11-15T02:56:35.391 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f731b59e700 1 -- 172.21.15.190:0/829085975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f731c0a4490 msgr2=0x7f731c0a5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:35.391 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f731b59e700 1 --2- 172.21.15.190:0/829085975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f731c0a4490 0x7f731c0a5c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f7314009f20 tx=0x7f7314009840).stop 2023-11-15T02:56:35.391 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f731b59e700 1 -- 172.21.15.190:0/829085975 shutdown_connections 2023-11-15T02:56:35.392 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f731b59e700 1 --2- 172.21.15.190:0/829085975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f731c0a4490 0x7f731c0a5c80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.392 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f731b59e700 1 --2- 172.21.15.190:0/829085975 >> v2:172.21.15.121:3300/0 conn(0x7f731c068170 0x7f731c0bdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.392 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f731b59e700 1 -- 172.21.15.190:0/829085975 >> 172.21.15.190:0/829085975 conn(0x7f731c066e00 msgr2=0x7f731c090dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:35.392 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f731b59e700 1 -- 172.21.15.190:0/829085975 shutdown_connections 2023-11-15T02:56:35.392 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f731b59e700 1 -- 172.21.15.190:0/829085975 wait complete. 2023-11-15T02:56:35.395 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f731b59e700 1 Processor -- start 2023-11-15T02:56:35.395 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f731b59e700 1 -- start start 2023-11-15T02:56:35.396 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f731b59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f731c068170 0x7f731c0c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:35.396 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f731b59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f731c0a4490 0x7f731c092f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:35.396 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f731b59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f731c157090 0x7f731c093400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:35.396 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f731b59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c0938f0 con 0x7f731c0a4490 2023-11-15T02:56:35.396 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f731b59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c0a6c50 con 0x7f731c157090 2023-11-15T02:56:35.397 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f731b59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731c0a6640 con 0x7f731c068170 2023-11-15T02:56:35.397 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f731a59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f731c068170 0x7f731c0c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:35.397 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f731a59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f731c068170 0x7f731c0c3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:38234/0 (socket says 172.21.15.190:38234) 2023-11-15T02:56:35.397 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.385+0000 7f731a59c700 1 -- 172.21.15.190:0/3771083556 learned_addr learned my addr 172.21.15.190:0/3771083556 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-11-15T02:56:35.398 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.389+0000 7f731ad9d700 1 --2- 172.21.15.190:0/3771083556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f731c157090 0x7f731c093400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:35.398 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.389+0000 7f7313fff700 1 --2- 172.21.15.190:0/3771083556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f731c0a4490 0x7f731c092f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:35.398 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.389+0000 7f731a59c700 1 -- 172.21.15.190:0/3771083556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f731c157090 msgr2=0x7f731c093400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:35.398 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.389+0000 7f731a59c700 1 --2- 172.21.15.190:0/3771083556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f731c157090 0x7f731c093400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.398 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.389+0000 7f731a59c700 1 -- 172.21.15.190:0/3771083556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f731c0a4490 msgr2=0x7f731c092f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:35.399 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.389+0000 7f731a59c700 1 --2- 172.21.15.190:0/3771083556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f731c0a4490 0x7f731c092f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.399 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.389+0000 7f731a59c700 1 -- 172.21.15.190:0/3771083556 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7314009ae0 con 0x7f731c068170 2023-11-15T02:56:35.399 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.389+0000 7f731ad9d700 1 --2- 172.21.15.190:0/3771083556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f731c157090 0x7f731c093400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:56:35.399 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.389+0000 7f731a59c700 1 --2- 172.21.15.190:0/3771083556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f731c068170 0x7f731c0c3d70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f730c00c490 tx=0x7f730c007de0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:35.399 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.389+0000 7f73127fc700 1 -- 172.21.15.190:0/3771083556 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c013450 con 0x7f731c068170 2023-11-15T02:56:35.400 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.389+0000 7f73127fc700 1 -- 172.21.15.190:0/3771083556 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730c0135d0 con 0x7f731c068170 2023-11-15T02:56:35.400 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.389+0000 7f731b59e700 1 -- 172.21.15.190:0/3771083556 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f731c0a9760 con 0x7f731c068170 2023-11-15T02:56:35.400 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.389+0000 7f73127fc700 1 -- 172.21.15.190:0/3771083556 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c013990 con 0x7f731c068170 2023-11-15T02:56:35.400 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.389+0000 7f731b59e700 1 -- 172.21.15.190:0/3771083556 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731c0a9450 con 0x7f731c068170 2023-11-15T02:56:35.400 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.389+0000 7f73127fc700 1 -- 172.21.15.190:0/3771083556 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f730c01d3f0 con 0x7f731c068170 2023-11-15T02:56:35.401 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.389+0000 7f73127fc700 1 -- 172.21.15.190:0/3771083556 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7f730c01ddd0 con 0x7f731c068170 2023-11-15T02:56:35.401 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.389+0000 7f731b59e700 1 -- 172.21.15.190:0/3771083556 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f731c06bcf0 con 0x7f731c068170 2023-11-15T02:56:35.401 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.393+0000 7f73127fc700 1 -- 172.21.15.190:0/3771083556 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f730c01d6a0 con 0x7f731c068170 2023-11-15T02:56:35.479 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:35.472+0000 7f4aacf5e500 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-15T02:56:35.486 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.481+0000 7f731b59e700 1 -- 172.21.15.190:0/3771083556 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cfe09666-0516-4e88-8983-4a9048d28a0c", "id": 5} v 0) v1 -- 0x7f731c0bcab0 con 0x7f731c068170 2023-11-15T02:56:35.495 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.489+0000 7f73127fc700 1 -- 172.21.15.190:0/3771083556 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cfe09666-0516-4e88-8983-4a9048d28a0c", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f730c01d840 con 0x7f731c068170 2023-11-15T02:56:35.495 INFO:teuthology.orchestra.run.smithi190.stdout:5 2023-11-15T02:56:35.496 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.493+0000 7f731b59e700 1 -- 172.21.15.190:0/3771083556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f731c068170 msgr2=0x7f731c0c3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:35.496 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.493+0000 7f731b59e700 1 --2- 172.21.15.190:0/3771083556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f731c068170 0x7f731c0c3d70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f730c00c490 tx=0x7f730c007de0).stop 2023-11-15T02:56:35.497 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.493+0000 7f731b59e700 1 -- 172.21.15.190:0/3771083556 shutdown_connections 2023-11-15T02:56:35.497 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.493+0000 7f731b59e700 1 --2- 172.21.15.190:0/3771083556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f731c157090 0x7f731c093400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.498 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.493+0000 7f731b59e700 1 --2- 172.21.15.190:0/3771083556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f731c0a4490 0x7f731c092f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.498 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.493+0000 7f731b59e700 1 --2- 172.21.15.190:0/3771083556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f731c068170 0x7f731c0c3d70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.498 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.493+0000 7f731b59e700 1 -- 172.21.15.190:0/3771083556 >> 172.21.15.190:0/3771083556 conn(0x7f731c066e00 msgr2=0x7f731c153a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:35.498 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.493+0000 7f731b59e700 1 -- 172.21.15.190:0/3771083556 shutdown_connections 2023-11-15T02:56:35.498 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.493+0000 7f731b59e700 1 -- 172.21.15.190:0/3771083556 wait complete. 2023-11-15T02:56:35.520 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new b18066ff-00a8-4de7-8395-8abc77382949 6 2023-11-15T02:56:35.544 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:35.540+0000 7f4aacf5e500 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-15T02:56:35.600 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:35.593+0000 7f0f8d344500 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-15T02:56:35.629 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:35.624+0000 7f4aacf5e500 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-15T02:56:35.708 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:35.704+0000 7f4aacf5e500 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-15T02:56:35.709 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.705+0000 7ff4bad07700 1 Processor -- start 2023-11-15T02:56:35.709 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.705+0000 7ff4bad07700 1 -- start start 2023-11-15T02:56:35.711 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.705+0000 7ff4bad07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff4b414e380 0x7ff4b40c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:35.712 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.705+0000 7ff4bad07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b40ad270 0x7ff4b4065680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:35.712 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.705+0000 7ff4bad07700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b40c7e90 con 0x7ff4b40c6040 2023-11-15T02:56:35.712 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.705+0000 7ff4bad07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b40c8320 con 0x7ff4b414e380 2023-11-15T02:56:35.712 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.705+0000 7ff4bad07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40c7080 con 0x7ff4b40ad270 2023-11-15T02:56:35.713 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.705+0000 7ff4ba506700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b40ad270 0x7ff4b4065680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:35.713 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.705+0000 7ff4b9d05700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff4b414e380 0x7ff4b40c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:35.713 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.705+0000 7ff4ba506700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b40ad270 0x7ff4b4065680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:38238/0 (socket says 172.21.15.190:38238) 2023-11-15T02:56:35.713 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.705+0000 7ff4ba506700 1 -- 172.21.15.190:0/1857974237 learned_addr learned my addr 172.21.15.190:0/1857974237 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-11-15T02:56:35.715 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.709+0000 7ff4ba506700 1 -- 172.21.15.190:0/1857974237 >> v2:172.21.15.190:3300/0 conn(0x7ff4b414e380 msgr2=0x7ff4b40c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:35.715 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.709+0000 7ff4ba506700 1 --2- 172.21.15.190:0/1857974237 >> v2:172.21.15.190:3300/0 conn(0x7ff4b414e380 0x7ff4b40c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.715 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.709+0000 7ff4ba506700 1 -- 172.21.15.190:0/1857974237 >> v1:172.21.15.121:6789/0 conn(0x7ff4b40c6040 legacy=0x7ff4b40acb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:35.715 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.709+0000 7ff4ba506700 1 -- 172.21.15.190:0/1857974237 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b414e760 con 0x7ff4b40ad270 2023-11-15T02:56:35.716 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.709+0000 7ff4ba506700 1 --2- 172.21.15.190:0/1857974237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b40ad270 0x7ff4b4065680 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7ff4ac009f20 tx=0x7ff4ac009840).ready entity=mon.2 client_cookie=c51be484c0033679 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:35.716 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.709+0000 7ff4b8d03700 1 -- 172.21.15.190:0/1857974237 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac00e070 con 0x7ff4b40ad270 2023-11-15T02:56:35.716 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.709+0000 7ff4b8d03700 1 -- 172.21.15.190:0/1857974237 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac00b9d0 con 0x7ff4b40ad270 2023-11-15T02:56:35.716 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.709+0000 7ff4b8d03700 1 -- 172.21.15.190:0/1857974237 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac00bd50 con 0x7ff4b40ad270 2023-11-15T02:56:35.716 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.709+0000 7ff4bad07700 1 -- 172.21.15.190:0/1857974237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b40ad270 msgr2=0x7ff4b4065680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:35.717 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.709+0000 7ff4bad07700 1 --2- 172.21.15.190:0/1857974237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b40ad270 0x7ff4b4065680 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7ff4ac009f20 tx=0x7ff4ac009840).stop 2023-11-15T02:56:35.717 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.709+0000 7ff4bad07700 1 -- 172.21.15.190:0/1857974237 shutdown_connections 2023-11-15T02:56:35.717 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.709+0000 7ff4bad07700 1 --2- 172.21.15.190:0/1857974237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b40ad270 0x7ff4b4065680 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.717 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.709+0000 7ff4bad07700 1 --2- 172.21.15.190:0/1857974237 >> v2:172.21.15.190:3300/0 conn(0x7ff4b414e380 0x7ff4b40c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.717 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.709+0000 7ff4bad07700 1 -- 172.21.15.190:0/1857974237 >> 172.21.15.190:0/1857974237 conn(0x7ff4b4067a60 msgr2=0x7ff4b4067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:35.718 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.709+0000 7ff4bad07700 1 -- 172.21.15.190:0/1857974237 shutdown_connections 2023-11-15T02:56:35.718 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.709+0000 7ff4bad07700 1 -- 172.21.15.190:0/1857974237 wait complete. 2023-11-15T02:56:35.719 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.713+0000 7ff4bad07700 1 Processor -- start 2023-11-15T02:56:35.719 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.713+0000 7ff4bad07700 1 -- start start 2023-11-15T02:56:35.719 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.713+0000 7ff4bad07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b40ad270 0x7ff4b4155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:35.720 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.713+0000 7ff4bad07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4b40c6040 0x7ff4b41561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:35.720 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.713+0000 7ff4bad07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b414e380 0x7ff4b40adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:35.720 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.713+0000 7ff4bad07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b4156cf0 con 0x7ff4b40c6040 2023-11-15T02:56:35.720 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.713+0000 7ff4bad07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b4156a10 con 0x7ff4b40ad270 2023-11-15T02:56:35.720 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.713+0000 7ff4bad07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40c2cc0 con 0x7ff4b414e380 2023-11-15T02:56:35.722 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.713+0000 7ff4b9d05700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b40ad270 0x7ff4b4155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:35.723 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.713+0000 7ff4ba506700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b414e380 0x7ff4b40adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:35.723 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.713+0000 7ff4b9d05700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b40ad270 0x7ff4b4155ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:60422/0 (socket says 172.21.15.190:60422) 2023-11-15T02:56:35.723 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.713+0000 7ff4b9d05700 1 -- 172.21.15.190:0/3102736113 learned_addr learned my addr 172.21.15.190:0/3102736113 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-11-15T02:56:35.723 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.713+0000 7ff4b9504700 1 --2- 172.21.15.190:0/3102736113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4b40c6040 0x7ff4b41561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:35.724 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.717+0000 7ff4b9d05700 1 -- 172.21.15.190:0/3102736113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b414e380 msgr2=0x7ff4b40adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:35.724 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.717+0000 7ff4b9d05700 1 --2- 172.21.15.190:0/3102736113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b414e380 0x7ff4b40adba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.724 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.717+0000 7ff4b9d05700 1 -- 172.21.15.190:0/3102736113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4b40c6040 msgr2=0x7ff4b41561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:35.724 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.717+0000 7ff4b9d05700 1 --2- 172.21.15.190:0/3102736113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4b40c6040 0x7ff4b41561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.724 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.717+0000 7ff4b9d05700 1 -- 172.21.15.190:0/3102736113 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ac009ae0 con 0x7ff4b40ad270 2023-11-15T02:56:35.725 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.717+0000 7ff4ba506700 1 --2- 172.21.15.190:0/3102736113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b414e380 0x7ff4b40adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:56:35.725 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.717+0000 7ff4b9d05700 1 --2- 172.21.15.190:0/3102736113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b40ad270 0x7ff4b4155ce0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7ff4b000e900 tx=0x7ff4b0009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:35.725 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.717+0000 7ff4a9ffb700 1 -- 172.21.15.190:0/3102736113 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0012600 con 0x7ff4b40ad270 2023-11-15T02:56:35.725 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.717+0000 7ff4a9ffb700 1 -- 172.21.15.190:0/3102736113 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b0012760 con 0x7ff4b40ad270 2023-11-15T02:56:35.725 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.717+0000 7ff4a9ffb700 1 -- 172.21.15.190:0/3102736113 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0012ae0 con 0x7ff4b40ad270 2023-11-15T02:56:35.726 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.717+0000 7ff4bad07700 1 -- 172.21.15.190:0/3102736113 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b40c2fa0 con 0x7ff4b40ad270 2023-11-15T02:56:35.726 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.717+0000 7ff4a9ffb700 1 -- 172.21.15.190:0/3102736113 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7ff4b001b660 con 0x7ff4b40ad270 2023-11-15T02:56:35.726 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.717+0000 7ff4bad07700 1 -- 172.21.15.190:0/3102736113 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b41566c0 con 0x7ff4b40ad270 2023-11-15T02:56:35.726 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.717+0000 7ff4a9ffb700 1 -- 172.21.15.190:0/3102736113 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7ff4b001baf0 con 0x7ff4b40ad270 2023-11-15T02:56:35.726 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.717+0000 7ff4bad07700 1 -- 172.21.15.190:0/3102736113 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b40c3420 con 0x7ff4b40ad270 2023-11-15T02:56:35.727 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.721+0000 7ff4a9ffb700 1 -- 172.21.15.190:0/3102736113 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7ff4b0016020 con 0x7ff4b40ad270 2023-11-15T02:56:35.811 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:35.805+0000 7f0f8d344500 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-15T02:56:35.815 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.809+0000 7ff4bad07700 1 -- 172.21.15.190:0/3102736113 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b18066ff-00a8-4de7-8395-8abc77382949", "id": 6} v 0) v1 -- 0x7ff4b406bcf0 con 0x7ff4b40ad270 2023-11-15T02:56:35.828 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.825+0000 7ff4a9ffb700 1 -- 172.21.15.190:0/3102736113 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b18066ff-00a8-4de7-8395-8abc77382949", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff4b0025480 con 0x7ff4b40ad270 2023-11-15T02:56:35.829 INFO:teuthology.orchestra.run.smithi190.stdout:6 2023-11-15T02:56:35.831 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.825+0000 7ff4bad07700 1 -- 172.21.15.190:0/3102736113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b40ad270 msgr2=0x7ff4b4155ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:35.831 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.825+0000 7ff4bad07700 1 --2- 172.21.15.190:0/3102736113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b40ad270 0x7ff4b4155ce0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7ff4b000e900 tx=0x7ff4b0009060).stop 2023-11-15T02:56:35.831 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.825+0000 7ff4bad07700 1 -- 172.21.15.190:0/3102736113 shutdown_connections 2023-11-15T02:56:35.832 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.825+0000 7ff4bad07700 1 --2- 172.21.15.190:0/3102736113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b414e380 0x7ff4b40adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.832 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.825+0000 7ff4bad07700 1 --2- 172.21.15.190:0/3102736113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4b40c6040 0x7ff4b41561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.832 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.825+0000 7ff4bad07700 1 --2- 172.21.15.190:0/3102736113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b40ad270 0x7ff4b4155ce0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:35.832 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.825+0000 7ff4bad07700 1 -- 172.21.15.190:0/3102736113 >> 172.21.15.190:0/3102736113 conn(0x7ff4b4067a60 msgr2=0x7ff4b40c0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:35.832 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.825+0000 7ff4bad07700 1 -- 172.21.15.190:0/3102736113 shutdown_connections 2023-11-15T02:56:35.833 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:35.825+0000 7ff4bad07700 1 -- 172.21.15.190:0/3102736113 wait complete. 2023-11-15T02:56:35.846 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 70d5381f-fb43-4ea6-aed2-2184162a1cce 7 2023-11-15T02:56:35.895 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:35.885+0000 7f0f8d344500 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-15T02:56:35.997 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:35.993+0000 7f0f8d344500 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-15T02:56:36.045 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.041+0000 7f9bbdcf1700 1 Processor -- start 2023-11-15T02:56:36.046 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.041+0000 7f9bbdcf1700 1 -- start start 2023-11-15T02:56:36.049 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bbdcf1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb8068170 0x7f9bb81582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:36.049 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bbdcf1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb8062da0 con 0x7f9bb8067020 2023-11-15T02:56:36.049 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb77fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f9bb8157ed0 0x7f9bb8067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:49338/0 (socket says 172.21.15.190:49338) 2023-11-15T02:56:36.049 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bbdcf1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb8062190 con 0x7f9bb8157ed0 2023-11-15T02:56:36.050 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb77fe700 1 -- 172.21.15.190:0/3166066259 learned_addr learned my addr 172.21.15.190:0/3166066259 (peer_addr_for_me v1:172.21.15.190:0/0) 2023-11-15T02:56:36.050 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bbdcf1700 1 -- 172.21.15.190:0/3166066259 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb80a93a0 con 0x7f9bb8068170 2023-11-15T02:56:36.053 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb67fc700 1 -- 172.21.15.190:0/3166066259 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198597507 0 0) 0x7f9bb8062190 con 0x7f9bb8157ed0 2023-11-15T02:56:36.053 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb67fc700 1 -- 172.21.15.190:0/3166066259 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ba00033f0 con 0x7f9bb8157ed0 2023-11-15T02:56:36.054 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb7fff700 1 --2- 172.21.15.190:0/3166066259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb8068170 0x7f9bb81582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:36.054 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb67fc700 1 -- 172.21.15.190:0/3166066259 <== mon.1 v1:172.21.15.190:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1569376289 0 0) 0x7f9ba00033f0 con 0x7f9bb8157ed0 2023-11-15T02:56:36.054 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb67fc700 1 -- 172.21.15.190:0/3166066259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb8068170 msgr2=0x7f9bb81582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:36.054 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb67fc700 1 --2- 172.21.15.190:0/3166066259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb8068170 0x7f9bb81582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:36.055 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb67fc700 1 -- 172.21.15.190:0/3166066259 >> v1:172.21.15.121:6789/0 conn(0x7f9bb8067020 legacy=0x7f9bb80bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:36.055 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb67fc700 1 -- 172.21.15.190:0/3166066259 --> v1:172.21.15.190:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb8062470 con 0x7f9bb8157ed0 2023-11-15T02:56:36.055 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb67fc700 1 -- 172.21.15.190:0/3166066259 <== mon.1 v1:172.21.15.190:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f9ba8001470 con 0x7f9bb8157ed0 2023-11-15T02:56:36.055 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb67fc700 1 -- 172.21.15.190:0/3166066259 >> v1:172.21.15.190:6789/0 conn(0x7f9bb8157ed0 legacy=0x7f9bb8067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:36.056 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb67fc700 1 --2- 172.21.15.190:0/3166066259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ba00038e0 0x7f9ba0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:36.056 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb67fc700 1 --2- 172.21.15.190:0/3166066259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ba0004690 0x7f9ba0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:36.056 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb67fc700 1 --2- 172.21.15.190:0/3166066259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb8068170 0x7f9ba0005020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:36.056 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb67fc700 1 -- 172.21.15.190:0/3166066259 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba000b990 con 0x7f9ba00038e0 2023-11-15T02:56:36.056 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb67fc700 1 -- 172.21.15.190:0/3166066259 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba000bad0 con 0x7f9ba0004690 2023-11-15T02:56:36.057 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb67fc700 1 -- 172.21.15.190:0/3166066259 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba000bc60 con 0x7f9bb8068170 2023-11-15T02:56:36.057 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb6ffd700 1 --2- 172.21.15.190:0/3166066259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ba0004690 0x7f9ba0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:36.057 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb77fe700 1 --2- 172.21.15.190:0/3166066259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ba00038e0 0x7f9ba0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:36.057 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb7fff700 1 --2- 172.21.15.190:0/3166066259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb8068170 0x7f9ba0005020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:36.057 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb6ffd700 1 -- 172.21.15.190:0/3166066259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb8068170 msgr2=0x7f9ba0005020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:36.058 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb6ffd700 1 --2- 172.21.15.190:0/3166066259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb8068170 0x7f9ba0005020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:36.058 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb6ffd700 1 -- 172.21.15.190:0/3166066259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ba00038e0 msgr2=0x7f9ba0003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:36.058 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb6ffd700 1 --2- 172.21.15.190:0/3166066259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ba00038e0 0x7f9ba0003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:36.058 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb6ffd700 1 -- 172.21.15.190:0/3166066259 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb8062470 con 0x7f9ba0004690 2023-11-15T02:56:36.059 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb7fff700 1 --2- 172.21.15.190:0/3166066259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb8068170 0x7f9ba0005020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:56:36.059 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb6ffd700 1 --2- 172.21.15.190:0/3166066259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ba0004690 0x7f9ba0004b10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f9bac001730 tx=0x7f9bac007bb0).ready entity=mon.1 client_cookie=2471fca8dd3deae3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:36.059 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb67fc700 1 -- 172.21.15.190:0/3166066259 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac007020 con 0x7f9ba0004690 2023-11-15T02:56:36.059 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb67fc700 1 -- 172.21.15.190:0/3166066259 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bac007180 con 0x7f9ba0004690 2023-11-15T02:56:36.059 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.045+0000 7f9bb67fc700 1 -- 172.21.15.190:0/3166066259 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac007500 con 0x7f9ba0004690 2023-11-15T02:56:36.060 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.049+0000 7f9bbdcf1700 1 -- 172.21.15.190:0/3166066259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ba0004690 msgr2=0x7f9ba0004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:36.060 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.049+0000 7f9bbdcf1700 1 --2- 172.21.15.190:0/3166066259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ba0004690 0x7f9ba0004b10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f9bac001730 tx=0x7f9bac007bb0).stop 2023-11-15T02:56:36.060 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.049+0000 7f9bb7fff700 1 -- 172.21.15.190:0/3166066259 reap_dead start 2023-11-15T02:56:36.060 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.049+0000 7f9bbdcf1700 1 -- 172.21.15.190:0/3166066259 shutdown_connections 2023-11-15T02:56:36.060 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.049+0000 7f9bbdcf1700 1 -- 172.21.15.190:0/3166066259 >> 172.21.15.190:0/3166066259 conn(0x7f9bb80591c0 msgr2=0x7f9bb80c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:36.061 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.049+0000 7f9bbdcf1700 1 -- 172.21.15.190:0/3166066259 shutdown_connections 2023-11-15T02:56:36.061 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.049+0000 7f9bbdcf1700 1 -- 172.21.15.190:0/3166066259 wait complete. 2023-11-15T02:56:36.063 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.049+0000 7f9bbdcf1700 1 Processor -- start 2023-11-15T02:56:36.064 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.049+0000 7f9bbdcf1700 1 -- start start 2023-11-15T02:56:36.064 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bbdcf1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ba0004690 0x7f9bb8156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:36.064 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bbdcf1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bb815b4f0 0x7f9bb815c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:36.064 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bbdcf1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bb815b970 0x7f9bb815cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:36.065 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bbdcf1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb815d000 con 0x7f9bb815b970 2023-11-15T02:56:36.065 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bbdcf1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb815a450 con 0x7f9bb815b4f0 2023-11-15T02:56:36.065 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bbdcf1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb815d200 con 0x7f9ba0004690 2023-11-15T02:56:36.065 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bb77fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ba0004690 0x7f9bb8156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:36.066 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bb77fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ba0004690 0x7f9bb8156e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:38278/0 (socket says 172.21.15.190:38278) 2023-11-15T02:56:36.066 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bb77fe700 1 -- 172.21.15.190:0/3126097439 learned_addr learned my addr 172.21.15.190:0/3126097439 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-11-15T02:56:36.066 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bb6ffd700 1 --2- 172.21.15.190:0/3126097439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bb815b4f0 0x7f9bb815c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:36.066 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bb77fe700 1 -- 172.21.15.190:0/3126097439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bb815b4f0 msgr2=0x7f9bb815c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:36.066 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bb77fe700 1 --2- 172.21.15.190:0/3126097439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bb815b4f0 0x7f9bb815c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:36.067 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bb77fe700 1 -- 172.21.15.190:0/3126097439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bb815b970 msgr2=0x7f9bb815cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:36.067 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bb7fff700 1 --2- 172.21.15.190:0/3126097439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bb815b970 0x7f9bb815cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:36.067 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bb77fe700 1 --2- 172.21.15.190:0/3126097439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bb815b970 0x7f9bb815cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:36.067 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bb77fe700 1 -- 172.21.15.190:0/3126097439 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb815aa10 con 0x7f9ba0004690 2023-11-15T02:56:36.068 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bb7fff700 1 --2- 172.21.15.190:0/3126097439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bb815b970 0x7f9bb815cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:56:36.068 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bb77fe700 1 --2- 172.21.15.190:0/3126097439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ba0004690 0x7f9bb8156e20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f9ba8000c00 tx=0x7f9ba8001440).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:36.068 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9b9f7fe700 1 -- 172.21.15.190:0/3126097439 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba800c7b0 con 0x7f9ba0004690 2023-11-15T02:56:36.068 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9b9f7fe700 1 -- 172.21.15.190:0/3126097439 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba800c910 con 0x7f9ba0004690 2023-11-15T02:56:36.069 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bbdcf1700 1 -- 172.21.15.190:0/3126097439 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb815acf0 con 0x7f9ba0004690 2023-11-15T02:56:36.069 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9b9f7fe700 1 -- 172.21.15.190:0/3126097439 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba800ccb0 con 0x7f9ba0004690 2023-11-15T02:56:36.069 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bbdcf1700 1 -- 172.21.15.190:0/3126097439 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb815d4e0 con 0x7f9ba0004690 2023-11-15T02:56:36.069 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9b9f7fe700 1 -- 172.21.15.190:0/3126097439 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f9ba80148a0 con 0x7f9ba0004690 2023-11-15T02:56:36.069 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9b9f7fe700 1 -- 172.21.15.190:0/3126097439 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1559+0+0 (secure 0 0 0) 0x7f9ba8014b50 con 0x7f9ba0004690 2023-11-15T02:56:36.070 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.053+0000 7f9bbdcf1700 1 -- 172.21.15.190:0/3126097439 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb815b1f0 con 0x7f9ba0004690 2023-11-15T02:56:36.070 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.057+0000 7f9b9f7fe700 1 -- 172.21.15.190:0/3126097439 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f9ba8022060 con 0x7f9ba0004690 2023-11-15T02:56:36.070 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:36.056+0000 7f4aacf5e500 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-15T02:56:36.077 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:36.073+0000 7f0f8d344500 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-15T02:56:36.158 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.153+0000 7f9bbdcf1700 1 -- 172.21.15.190:0/3126097439 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "70d5381f-fb43-4ea6-aed2-2184162a1cce", "id": 7} v 0) v1 -- 0x7f9bb8047060 con 0x7f9ba0004690 2023-11-15T02:56:36.169 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.165+0000 7f9b9f7fe700 1 -- 172.21.15.190:0/3126097439 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "70d5381f-fb43-4ea6-aed2-2184162a1cce", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9ba8026440 con 0x7f9ba0004690 2023-11-15T02:56:36.169 INFO:teuthology.orchestra.run.smithi190.stdout:7 2023-11-15T02:56:36.171 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.165+0000 7f9bbdcf1700 1 -- 172.21.15.190:0/3126097439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ba0004690 msgr2=0x7f9bb8156e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:36.171 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.165+0000 7f9bbdcf1700 1 --2- 172.21.15.190:0/3126097439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ba0004690 0x7f9bb8156e20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f9ba8000c00 tx=0x7f9ba8001440).stop 2023-11-15T02:56:36.172 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.165+0000 7f9bbdcf1700 1 -- 172.21.15.190:0/3126097439 shutdown_connections 2023-11-15T02:56:36.172 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.165+0000 7f9bbdcf1700 1 --2- 172.21.15.190:0/3126097439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bb815b970 0x7f9bb815cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:36.172 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.165+0000 7f9bbdcf1700 1 --2- 172.21.15.190:0/3126097439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bb815b4f0 0x7f9bb815c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:36.172 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.165+0000 7f9bbdcf1700 1 --2- 172.21.15.190:0/3126097439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ba0004690 0x7f9bb8156e20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:36.172 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.165+0000 7f9bbdcf1700 1 -- 172.21.15.190:0/3126097439 >> 172.21.15.190:0/3126097439 conn(0x7f9bb80591c0 msgr2=0x7f9bb80abb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:36.173 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.165+0000 7f9bbdcf1700 1 -- 172.21.15.190:0/3126097439 shutdown_connections 2023-11-15T02:56:36.173 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:56:36.165+0000 7f9bbdcf1700 1 -- 172.21.15.190:0/3126097439 wait complete. 2023-11-15T02:56:36.191 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-15T02:56:36.191 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-15T02:56:36.199 INFO:tasks.ceph.osd.0:Started 2023-11-15T02:56:36.199 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-15T02:56:36.200 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-15T02:56:36.204 INFO:tasks.ceph.osd.1:Started 2023-11-15T02:56:36.204 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-15T02:56:36.205 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-15T02:56:36.208 INFO:tasks.ceph.osd.2:Started 2023-11-15T02:56:36.209 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-15T02:56:36.209 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-15T02:56:36.213 INFO:tasks.ceph.osd.3:Started 2023-11-15T02:56:36.214 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-15T02:56:36.214 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-15T02:56:36.221 INFO:tasks.ceph.osd.4:Started 2023-11-15T02:56:36.221 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-15T02:56:36.222 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-15T02:56:36.229 INFO:tasks.ceph.osd.5:Started 2023-11-15T02:56:36.229 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-15T02:56:36.230 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-15T02:56:36.233 INFO:tasks.ceph.osd.6:Started 2023-11-15T02:56:36.234 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-15T02:56:36.234 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-15T02:56:36.237 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:36.232+0000 7f4aacf5e500 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-15T02:56:36.239 INFO:tasks.ceph.osd.7:Started 2023-11-15T02:56:36.240 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:56:36.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.412+0000 7fa419546700 1 Processor -- start 2023-11-15T02:56:36.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.412+0000 7fa419546700 1 -- start start 2023-11-15T02:56:36.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.412+0000 7fa419546700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa414139c70 0x7fa4141547e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:36.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.412+0000 7fa419546700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4140c37c0 con 0x7fa414067a60 2023-11-15T02:56:36.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.412+0000 7fa419546700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4140c2c40 con 0x7fa414139890 2023-11-15T02:56:36.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.412+0000 7fa419546700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa414154cd0 con 0x7fa414139c70 2023-11-15T02:56:36.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.412+0000 7fa4137fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa414067a60 0x7fa4140c4f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58846/0 (socket says 172.21.15.121:58846) 2023-11-15T02:56:36.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.416+0000 7fa4137fe700 1 -- 172.21.15.121:0/2634502137 learned_addr learned my addr 172.21.15.121:0/2634502137 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:36.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.416+0000 7fa4127fc700 1 --2- 172.21.15.121:0/2634502137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa414139c70 0x7fa4141547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:36.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.416+0000 7fa411ffb700 1 -- 172.21.15.121:0/2634502137 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12781256 0 0) 0x7fa4140c37c0 con 0x7fa414067a60 2023-11-15T02:56:36.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.416+0000 7fa411ffb700 1 -- 172.21.15.121:0/2634502137 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3fc0033f0 con 0x7fa414067a60 2023-11-15T02:56:36.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.416+0000 7fa4127fc700 1 -- 172.21.15.121:0/2634502137 >> v1:172.21.15.190:6789/0 conn(0x7fa414139890 legacy=0x7fa4140c2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:36.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.416+0000 7fa4127fc700 1 -- 172.21.15.121:0/2634502137 >> v1:172.21.15.121:6789/0 conn(0x7fa414067a60 legacy=0x7fa4140c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:36.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.416+0000 7fa4127fc700 1 -- 172.21.15.121:0/2634502137 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4140c2f20 con 0x7fa414139c70 2023-11-15T02:56:36.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.416+0000 7fa4127fc700 1 --2- 172.21.15.121:0/2634502137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa414139c70 0x7fa4141547e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fa408009790 tx=0x7fa408009060).ready entity=mon.2 client_cookie=f619d0fa596a667c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:36.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.416+0000 7fa411ffb700 1 -- 172.21.15.121:0/2634502137 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40800f510 con 0x7fa414139c70 2023-11-15T02:56:36.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.416+0000 7fa411ffb700 1 -- 172.21.15.121:0/2634502137 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40800f670 con 0x7fa414139c70 2023-11-15T02:56:36.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.416+0000 7fa411ffb700 1 -- 172.21.15.121:0/2634502137 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40800f9f0 con 0x7fa414139c70 2023-11-15T02:56:36.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.416+0000 7fa419546700 1 -- 172.21.15.121:0/2634502137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa414139c70 msgr2=0x7fa4141547e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:36.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.416+0000 7fa419546700 1 --2- 172.21.15.121:0/2634502137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa414139c70 0x7fa4141547e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fa408009790 tx=0x7fa408009060).stop 2023-11-15T02:56:36.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.416+0000 7fa419546700 1 -- 172.21.15.121:0/2634502137 shutdown_connections 2023-11-15T02:56:36.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.416+0000 7fa419546700 1 --2- 172.21.15.121:0/2634502137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa414139c70 0x7fa4141547e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:36.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.416+0000 7fa419546700 1 -- 172.21.15.121:0/2634502137 >> 172.21.15.121:0/2634502137 conn(0x7fa4140c7ac0 msgr2=0x7fa4140c3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:36.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.416+0000 7fa419546700 1 -- 172.21.15.121:0/2634502137 shutdown_connections 2023-11-15T02:56:36.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.416+0000 7fa419546700 1 -- 172.21.15.121:0/2634502137 wait complete. 2023-11-15T02:56:36.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.420+0000 7fa419546700 1 Processor -- start 2023-11-15T02:56:36.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.420+0000 7fa419546700 1 -- start start 2023-11-15T02:56:36.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.420+0000 7fa419546700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa414067a60 0x7fa414156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:36.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.420+0000 7fa419546700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa414139890 0x7fa4140a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:36.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.420+0000 7fa419546700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa414139c70 0x7fa4140a4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:36.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.420+0000 7fa419546700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140ab3c0 con 0x7fa414139890 2023-11-15T02:56:36.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.420+0000 7fa419546700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140aa7b0 con 0x7fa414067a60 2023-11-15T02:56:36.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.420+0000 7fa419546700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140aaa90 con 0x7fa414139c70 2023-11-15T02:56:36.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.420+0000 7fa4127fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa414139890 0x7fa4140a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:36.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.420+0000 7fa4127fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa414139890 0x7fa4140a9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34050/0 (socket says 172.21.15.121:34050) 2023-11-15T02:56:36.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.420+0000 7fa4127fc700 1 -- 172.21.15.121:0/1111873908 learned_addr learned my addr 172.21.15.121:0/1111873908 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:36.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.420+0000 7fa4137fe700 1 --2- 172.21.15.121:0/1111873908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa414139c70 0x7fa4140a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:36.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.420+0000 7fa412ffd700 1 --2- 172.21.15.121:0/1111873908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa414067a60 0x7fa414156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:36.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.424+0000 7fa4127fc700 1 -- 172.21.15.121:0/1111873908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa414139c70 msgr2=0x7fa4140a4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:36.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.424+0000 7fa4127fc700 1 --2- 172.21.15.121:0/1111873908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa414139c70 0x7fa4140a4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:36.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.424+0000 7fa4127fc700 1 -- 172.21.15.121:0/1111873908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa414067a60 msgr2=0x7fa414156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:36.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.424+0000 7fa4127fc700 1 --2- 172.21.15.121:0/1111873908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa414067a60 0x7fa414156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:36.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.424+0000 7fa4127fc700 1 -- 172.21.15.121:0/1111873908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4080092a0 con 0x7fa414139890 2023-11-15T02:56:36.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.424+0000 7fa4127fc700 1 --2- 172.21.15.121:0/1111873908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa414139890 0x7fa4140a9f80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fa408009e30 tx=0x7fa408017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:36.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.424+0000 7fa3faffd700 1 -- 172.21.15.121:0/1111873908 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa408000bc0 con 0x7fa414139890 2023-11-15T02:56:36.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.424+0000 7fa3faffd700 1 -- 172.21.15.121:0/1111873908 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40800f7e0 con 0x7fa414139890 2023-11-15T02:56:36.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.424+0000 7fa3faffd700 1 -- 172.21.15.121:0/1111873908 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa408000d20 con 0x7fa414139890 2023-11-15T02:56:36.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.424+0000 7fa419546700 1 -- 172.21.15.121:0/1111873908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4140aad70 con 0x7fa414139890 2023-11-15T02:56:36.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.424+0000 7fa419546700 1 -- 172.21.15.121:0/1111873908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4140aa470 con 0x7fa414139890 2023-11-15T02:56:36.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.424+0000 7fa3faffd700 1 -- 172.21.15.121:0/1111873908 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fa408021070 con 0x7fa414139890 2023-11-15T02:56:36.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.424+0000 7fa3faffd700 1 -- 172.21.15.121:0/1111873908 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7fa408024b50 con 0x7fa414139890 2023-11-15T02:56:36.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.424+0000 7fa419546700 1 -- 172.21.15.121:0/1111873908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4140c4c80 con 0x7fa414139890 2023-11-15T02:56:36.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.428+0000 7fa3faffd700 1 -- 172.21.15.121:0/1111873908 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fa4140c4c80 con 0x7fa414139890 2023-11-15T02:56:36.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.496+0000 7fa419546700 1 -- 172.21.15.121:0/1111873908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4140ab530 con 0x7fa414139890 2023-11-15T02:56:36.501 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:36.497+0000 7f0f8d344500 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-15T02:56:36.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.496+0000 7fa3faffd700 1 -- 172.21.15.121:0/1111873908 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7045 (secure 0 0 0) 0x7fa408024e20 con 0x7fa414139890 2023-11-15T02:56:36.502 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:56:36.502 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":10,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:36.158536+0000","last_up_change":"0.000000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:56:36.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.500+0000 7fa419546700 1 -- 172.21.15.121:0/1111873908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa414139890 msgr2=0x7fa4140a9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:36.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.500+0000 7fa419546700 1 --2- 172.21.15.121:0/1111873908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa414139890 0x7fa4140a9f80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fa408009e30 tx=0x7fa408017000).stop 2023-11-15T02:56:36.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.500+0000 7fa419546700 1 -- 172.21.15.121:0/1111873908 shutdown_connections 2023-11-15T02:56:36.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.500+0000 7fa419546700 1 --2- 172.21.15.121:0/1111873908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa414139c70 0x7fa4140a4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:36.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.500+0000 7fa419546700 1 --2- 172.21.15.121:0/1111873908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa414139890 0x7fa4140a9f80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:36.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.500+0000 7fa419546700 1 --2- 172.21.15.121:0/1111873908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa414067a60 0x7fa414156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:36.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.500+0000 7fa419546700 1 -- 172.21.15.121:0/1111873908 >> 172.21.15.121:0/1111873908 conn(0x7fa4140c7ac0 msgr2=0x7fa4140c2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:36.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.500+0000 7fa419546700 1 -- 172.21.15.121:0/1111873908 shutdown_connections 2023-11-15T02:56:36.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:36.500+0000 7fa419546700 1 -- 172.21.15.121:0/1111873908 wait complete. 2023-11-15T02:56:36.521 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-15T02:56:36.522 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-15T02:56:36.612 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:36.608+0000 7f4aacf5e500 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-15T02:56:36.688 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:36.685+0000 7f0f8d344500 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-15T02:56:36.710 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:36.704+0000 7f4aacf5e500 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-15T02:56:36.849 INFO:tasks.ceph.osd.1.smithi121.stderr:2023-11-15T02:56:36.844+0000 7fdc53dc0200 -1 Falling back to public interface 2023-11-15T02:56:36.877 INFO:tasks.ceph.osd.2.smithi121.stderr:2023-11-15T02:56:36.872+0000 7fe67f92e200 -1 Falling back to public interface 2023-11-15T02:56:36.884 INFO:tasks.ceph.osd.0.smithi121.stderr:2023-11-15T02:56:36.880+0000 7f00afd08200 -1 Falling back to public interface 2023-11-15T02:56:36.890 INFO:tasks.ceph.osd.3.smithi121.stderr:2023-11-15T02:56:36.884+0000 7f9c330db200 -1 Falling back to public interface 2023-11-15T02:56:36.894 INFO:tasks.ceph.osd.5.smithi190.stderr:2023-11-15T02:56:36.889+0000 7fb44cae0200 -1 Falling back to public interface 2023-11-15T02:56:36.906 INFO:tasks.ceph.osd.7.smithi190.stderr:2023-11-15T02:56:36.901+0000 7f7dcace8200 -1 Falling back to public interface 2023-11-15T02:56:36.906 INFO:tasks.ceph.osd.6.smithi190.stderr:2023-11-15T02:56:36.901+0000 7f9b7cede200 -1 Falling back to public interface 2023-11-15T02:56:36.909 INFO:tasks.ceph.osd.4.smithi190.stderr:2023-11-15T02:56:36.905+0000 7f87040f3200 -1 Falling back to public interface 2023-11-15T02:56:37.028 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:37.024+0000 7f4aacf5e500 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-15T02:56:37.032 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:37.025+0000 7f0f8d344500 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-15T02:56:37.115 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:37.108+0000 7f4aacf5e500 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-15T02:56:37.128 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:37.121+0000 7f0f8d344500 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-15T02:56:37.305 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:37.300+0000 7f4aacf5e500 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-15T02:56:37.327 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-15T02:56:37.434 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:37.428+0000 7f4aacf5e500 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-15T02:56:37.444 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:37.441+0000 7f0f8d344500 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-15T02:56:37.477 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.472+0000 7faaf2e5b700 1 Processor -- start 2023-11-15T02:56:37.477 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.472+0000 7faaf2e5b700 1 -- start start 2023-11-15T02:56:37.480 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.476+0000 7faaf2e5b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7faaec0a8c80 0x7faaec068d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:37.480 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.476+0000 7faaf2e5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faaec0a9060 0x7faaec0bd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:37.480 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.476+0000 7faaf2e5b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaec155080 con 0x7faaec0a5720 2023-11-15T02:56:37.480 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.476+0000 7faaf2e5b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaec069390 con 0x7faaec0a8c80 2023-11-15T02:56:37.481 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.476+0000 7faaf2e5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaec155510 con 0x7faaec0a9060 2023-11-15T02:56:37.481 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.476+0000 7faaf13f8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7faaec0a5720 0x7faaec150f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58852/0 (socket says 172.21.15.121:58852) 2023-11-15T02:56:37.481 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.476+0000 7faaf13f8700 1 -- 172.21.15.121:0/1257949182 learned_addr learned my addr 172.21.15.121:0/1257949182 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:37.481 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.476+0000 7faaf0bf7700 1 --2- 172.21.15.121:0/1257949182 >> v2:172.21.15.121:3300/0 conn(0x7faaec0a8c80 0x7faaec068d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:37.481 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.476+0000 7faae3fff700 1 --2- 172.21.15.121:0/1257949182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faaec0a9060 0x7faaec0bd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:37.483 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.476+0000 7faae37fe700 1 -- 172.21.15.121:0/1257949182 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 794839835 0 0) 0x7faaec155080 con 0x7faaec0a5720 2023-11-15T02:56:37.483 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.476+0000 7faae37fe700 1 -- 172.21.15.121:0/1257949182 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faad40033f0 con 0x7faaec0a5720 2023-11-15T02:56:37.484 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.476+0000 7faae3fff700 1 -- 172.21.15.121:0/1257949182 >> v2:172.21.15.121:3300/0 conn(0x7faaec0a8c80 msgr2=0x7faaec068d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:37.484 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.476+0000 7faae3fff700 1 --2- 172.21.15.121:0/1257949182 >> v2:172.21.15.121:3300/0 conn(0x7faaec0a8c80 0x7faaec068d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:37.484 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.476+0000 7faae3fff700 1 -- 172.21.15.121:0/1257949182 >> v1:172.21.15.121:6789/0 conn(0x7faaec0a5720 legacy=0x7faaec150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:37.484 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.476+0000 7faae3fff700 1 -- 172.21.15.121:0/1257949182 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaec0bd750 con 0x7faaec0a9060 2023-11-15T02:56:37.484 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.476+0000 7faae3fff700 1 --2- 172.21.15.121:0/1257949182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faaec0a9060 0x7faaec0bd260 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7faadc009760 tx=0x7faadc009060).ready entity=mon.2 client_cookie=6f28cec6039cd180 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:37.485 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.476+0000 7faae37fe700 1 -- 172.21.15.121:0/1257949182 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc00d4a0 con 0x7faaec0a9060 2023-11-15T02:56:37.485 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.476+0000 7faae37fe700 1 -- 172.21.15.121:0/1257949182 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faadc00d600 con 0x7faaec0a9060 2023-11-15T02:56:37.485 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.476+0000 7faae37fe700 1 -- 172.21.15.121:0/1257949182 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc00d980 con 0x7faaec0a9060 2023-11-15T02:56:37.485 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.480+0000 7faaf2e5b700 1 -- 172.21.15.121:0/1257949182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faaec0a9060 msgr2=0x7faaec0bd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:37.485 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.480+0000 7faaf2e5b700 1 --2- 172.21.15.121:0/1257949182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faaec0a9060 0x7faaec0bd260 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7faadc009760 tx=0x7faadc009060).stop 2023-11-15T02:56:37.486 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.480+0000 7faaf2e5b700 1 -- 172.21.15.121:0/1257949182 shutdown_connections 2023-11-15T02:56:37.486 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.480+0000 7faaf2e5b700 1 --2- 172.21.15.121:0/1257949182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faaec0a9060 0x7faaec0bd260 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:37.486 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.480+0000 7faaf2e5b700 1 --2- 172.21.15.121:0/1257949182 >> v2:172.21.15.121:3300/0 conn(0x7faaec0a8c80 0x7faaec068d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:37.486 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.480+0000 7faaf2e5b700 1 -- 172.21.15.121:0/1257949182 >> 172.21.15.121:0/1257949182 conn(0x7faaec062d00 msgr2=0x7faaec0c6850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:37.486 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.480+0000 7faaf2e5b700 1 -- 172.21.15.121:0/1257949182 shutdown_connections 2023-11-15T02:56:37.487 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.480+0000 7faaf2e5b700 1 -- 172.21.15.121:0/1257949182 wait complete. 2023-11-15T02:56:37.488 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.480+0000 7faaf2e5b700 1 Processor -- start 2023-11-15T02:56:37.489 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.480+0000 7faaf2e5b700 1 -- start start 2023-11-15T02:56:37.489 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faaf2e5b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faaec0a5720 0x7faaec0c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:37.489 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faaf2e5b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faaec0a8c80 0x7faaec0bfde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:37.489 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faaf2e5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faaec0a9060 0x7faaec0c02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:37.489 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faaf2e5b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec0bf320 con 0x7faaec0a8c80 2023-11-15T02:56:37.490 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faaf2e5b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec0c07c0 con 0x7faaec0a5720 2023-11-15T02:56:37.490 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faaf2e5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaec0aa870 con 0x7faaec0a9060 2023-11-15T02:56:37.490 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faae3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faaec0a8c80 0x7faaec0bfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:37.490 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faae3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faaec0a8c80 0x7faaec0bfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34070/0 (socket says 172.21.15.121:34070) 2023-11-15T02:56:37.490 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faae3fff700 1 -- 172.21.15.121:0/1510266255 learned_addr learned my addr 172.21.15.121:0/1510266255 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:37.491 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faaf0bf7700 1 --2- 172.21.15.121:0/1510266255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faaec0a5720 0x7faaec0c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:37.491 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faaf13f8700 1 --2- 172.21.15.121:0/1510266255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faaec0a9060 0x7faaec0c02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:37.491 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faae3fff700 1 -- 172.21.15.121:0/1510266255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faaec0a9060 msgr2=0x7faaec0c02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:37.491 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faae3fff700 1 --2- 172.21.15.121:0/1510266255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faaec0a9060 0x7faaec0c02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:37.491 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faae3fff700 1 -- 172.21.15.121:0/1510266255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faaec0a5720 msgr2=0x7faaec0c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:37.492 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faae3fff700 1 --2- 172.21.15.121:0/1510266255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faaec0a5720 0x7faaec0c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:37.492 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faae3fff700 1 -- 172.21.15.121:0/1510266255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faadc0092a0 con 0x7faaec0a8c80 2023-11-15T02:56:37.492 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faae3fff700 1 --2- 172.21.15.121:0/1510266255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faaec0a8c80 0x7faaec0bfde0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7faadc011fb0 tx=0x7faadc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:37.493 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faae0ff9700 1 -- 172.21.15.121:0/1510266255 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc00dcf0 con 0x7faaec0a8c80 2023-11-15T02:56:37.493 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faae0ff9700 1 -- 172.21.15.121:0/1510266255 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faadc00de50 con 0x7faaec0a8c80 2023-11-15T02:56:37.493 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faae0ff9700 1 -- 172.21.15.121:0/1510266255 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc00d980 con 0x7faaec0a8c80 2023-11-15T02:56:37.494 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faaf2e5b700 1 -- 172.21.15.121:0/1510266255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaec1632c0 con 0x7faaec0a8c80 2023-11-15T02:56:37.494 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faaf2e5b700 1 -- 172.21.15.121:0/1510266255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaec0aae90 con 0x7faaec0a8c80 2023-11-15T02:56:37.494 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faae0ff9700 1 -- 172.21.15.121:0/1510266255 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7faadc01d640 con 0x7faaec0a8c80 2023-11-15T02:56:37.494 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.484+0000 7faae0ff9700 1 -- 172.21.15.121:0/1510266255 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7faadc01ca90 con 0x7faaec0a8c80 2023-11-15T02:56:37.494 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.488+0000 7faaf2e5b700 1 -- 172.21.15.121:0/1510266255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaec0c3120 con 0x7faaec0a8c80 2023-11-15T02:56:37.495 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.488+0000 7faae0ff9700 1 -- 172.21.15.121:0/1510266255 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7faaec0c3120 con 0x7faaec0a8c80 2023-11-15T02:56:37.532 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:37.525+0000 7f0f8d344500 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-15T02:56:37.567 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.564+0000 7faaf2e5b700 1 -- 172.21.15.121:0/1510266255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faaec06bcd0 con 0x7faaec0a8c80 2023-11-15T02:56:37.568 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.564+0000 7faae0ff9700 1 -- 172.21.15.121:0/1510266255 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7045 (secure 0 0 0) 0x7faadc016070 con 0x7faaec0a8c80 2023-11-15T02:56:37.569 INFO:teuthology.misc.health.smithi121.stdout: 2023-11-15T02:56:37.569 INFO:teuthology.misc.health.smithi121.stdout:{"epoch":10,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:36.158536+0000","last_up_change":"0.000000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:56:37.570 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.564+0000 7faaf2e5b700 1 -- 172.21.15.121:0/1510266255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faaec0a8c80 msgr2=0x7faaec0bfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:37.571 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.564+0000 7faaf2e5b700 1 --2- 172.21.15.121:0/1510266255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faaec0a8c80 0x7faaec0bfde0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7faadc011fb0 tx=0x7faadc016000).stop 2023-11-15T02:56:37.571 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.564+0000 7faaf2e5b700 1 -- 172.21.15.121:0/1510266255 shutdown_connections 2023-11-15T02:56:37.571 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.564+0000 7faaf2e5b700 1 --2- 172.21.15.121:0/1510266255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faaec0a9060 0x7faaec0c02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:37.571 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.564+0000 7faaf2e5b700 1 --2- 172.21.15.121:0/1510266255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faaec0a8c80 0x7faaec0bfde0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:37.572 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.564+0000 7faaf2e5b700 1 --2- 172.21.15.121:0/1510266255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faaec0a5720 0x7faaec0c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:37.572 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.564+0000 7faaf2e5b700 1 -- 172.21.15.121:0/1510266255 >> 172.21.15.121:0/1510266255 conn(0x7faaec062d00 msgr2=0x7faaec066050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:37.572 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.568+0000 7faaf2e5b700 1 -- 172.21.15.121:0/1510266255 shutdown_connections 2023-11-15T02:56:37.573 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:37.568+0000 7faaf2e5b700 1 -- 172.21.15.121:0/1510266255 wait complete. 2023-11-15T02:56:37.586 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-15T02:56:37.631 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:37.628+0000 7f4aacf5e500 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-15T02:56:37.724 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:37.721+0000 7f0f8d344500 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-15T02:56:37.852 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:37.849+0000 7f0f8d344500 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-15T02:56:37.888 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:37.884+0000 7f4aacf5e500 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-15T02:56:38.040 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:38.033+0000 7f0f8d344500 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-15T02:56:38.295 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-11-15T02:56:38.289+0000 7f0f8d344500 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-15T02:56:38.926 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:38.920+0000 7f4a8cd95700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-15T02:56:38.926 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T02:56:38.920+0000 7f4a8cd95700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-15T02:56:39.893 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T02:56:39.901 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T02:56:40.980 INFO:tasks.ceph.osd.5.smithi190.stderr:2023-11-15T02:56:40.977+0000 7fb44cae0200 -1 osd.5 0 log_to_monitors {default=true} 2023-11-15T02:56:41.018 INFO:tasks.ceph.osd.1.smithi121.stderr:2023-11-15T02:56:41.012+0000 7fdc53dc0200 -1 osd.1 0 log_to_monitors {default=true} 2023-11-15T02:56:41.144 INFO:tasks.ceph.osd.7.smithi190.stderr:2023-11-15T02:56:41.141+0000 7f7dcace8200 -1 osd.7 0 log_to_monitors {default=true} 2023-11-15T02:56:41.145 INFO:tasks.ceph.osd.6.smithi190.stderr:2023-11-15T02:56:41.141+0000 7f9b7cede200 -1 osd.6 0 log_to_monitors {default=true} 2023-11-15T02:56:41.148 INFO:tasks.ceph.osd.0.smithi121.stderr:2023-11-15T02:56:41.144+0000 7f00afd08200 -1 osd.0 0 log_to_monitors {default=true} 2023-11-15T02:56:41.175 INFO:tasks.ceph.osd.4.smithi190.stderr:2023-11-15T02:56:41.169+0000 7f87040f3200 -1 osd.4 0 log_to_monitors {default=true} 2023-11-15T02:56:41.366 INFO:tasks.ceph.osd.3.smithi121.stderr:2023-11-15T02:56:41.360+0000 7f9c330db200 -1 osd.3 0 log_to_monitors {default=true} 2023-11-15T02:56:41.401 INFO:tasks.ceph.osd.2.smithi121.stderr:2023-11-15T02:56:41.396+0000 7fe67f92e200 -1 osd.2 0 log_to_monitors {default=true} 2023-11-15T02:56:42.143 INFO:tasks.ceph.osd.5.smithi190.stderr:2023-11-15T02:56:42.137+0000 7fb447a5c700 -1 osd.5 0 waiting for initial osdmap 2023-11-15T02:56:42.146 INFO:tasks.ceph.osd.1.smithi121.stderr:2023-11-15T02:56:42.140+0000 7fdc4f562700 -1 osd.1 0 waiting for initial osdmap 2023-11-15T02:56:42.154 INFO:tasks.ceph.osd.5.smithi190.stderr:2023-11-15T02:56:42.149+0000 7fb4447d0700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-15T02:56:42.156 INFO:tasks.ceph.osd.1.smithi121.stderr:2023-11-15T02:56:42.152+0000 7fdc4c2d6700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-15T02:56:43.150 INFO:tasks.ceph.osd.7.smithi190.stderr:2023-11-15T02:56:43.145+0000 7f7dc5afb700 -1 osd.7 0 waiting for initial osdmap 2023-11-15T02:56:43.150 INFO:tasks.ceph.osd.6.smithi190.stderr:2023-11-15T02:56:43.145+0000 7f9b77da4700 -1 osd.6 0 waiting for initial osdmap 2023-11-15T02:56:43.151 INFO:tasks.ceph.osd.0.smithi121.stderr:2023-11-15T02:56:43.148+0000 7f00aacb5700 -1 osd.0 0 waiting for initial osdmap 2023-11-15T02:56:43.153 INFO:tasks.ceph.osd.2.smithi121.stderr:2023-11-15T02:56:43.148+0000 7fe67b0d0700 -1 osd.2 0 waiting for initial osdmap 2023-11-15T02:56:43.153 INFO:tasks.ceph.osd.3.smithi121.stderr:2023-11-15T02:56:43.148+0000 7f9c2e87b700 -1 osd.3 0 waiting for initial osdmap 2023-11-15T02:56:43.156 INFO:tasks.ceph.osd.4.smithi190.stderr:2023-11-15T02:56:43.153+0000 7f86ff003700 -1 osd.4 0 waiting for initial osdmap 2023-11-15T02:56:43.161 INFO:tasks.ceph.osd.7.smithi190.stderr:2023-11-15T02:56:43.157+0000 7f7dc286f700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-15T02:56:43.164 INFO:tasks.ceph.osd.4.smithi190.stderr:2023-11-15T02:56:43.161+0000 7f86fbd77700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-15T02:56:43.165 INFO:tasks.ceph.osd.0.smithi121.stderr:2023-11-15T02:56:43.160+0000 7f00a7a29700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-15T02:56:43.166 INFO:tasks.ceph.osd.6.smithi190.stderr:2023-11-15T02:56:43.161+0000 7f9b74b18700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-15T02:56:43.238 INFO:tasks.ceph.osd.3.smithi121.stderr:2023-11-15T02:56:43.232+0000 7f9c2b5ef700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-15T02:56:43.239 INFO:tasks.ceph.osd.2.smithi121.stderr:2023-11-15T02:56:43.236+0000 7fe677e44700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-15T02:56:44.392 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-15T02:56:44.542 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.536+0000 7f28672db700 1 Processor -- start 2023-11-15T02:56:44.543 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.540+0000 7f28672db700 1 -- start start 2023-11-15T02:56:44.545 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.540+0000 7f28672db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f28600c2b80 0x7f28600bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:44.545 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.540+0000 7f28672db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28600bc5c0 0x7f2860065fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:44.545 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.540+0000 7f28672db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f28600664b0 0x7f28600a5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:44.546 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.540+0000 7f28672db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f286014f6a0 con 0x7f28600664b0 2023-11-15T02:56:44.546 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.540+0000 7f28672db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2860067590 con 0x7f28600c2b80 2023-11-15T02:56:44.546 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.540+0000 7f28672db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28600bca20 con 0x7f28600bc5c0 2023-11-15T02:56:44.547 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.540+0000 7f2865878700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f28600664b0 0x7f28600a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:44.547 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.540+0000 7f2865878700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f28600664b0 0x7f28600a5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34134/0 (socket says 172.21.15.121:34134) 2023-11-15T02:56:44.548 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.540+0000 7f2865878700 1 -- 172.21.15.121:0/2601113812 learned_addr learned my addr 172.21.15.121:0/2601113812 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:44.548 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.540+0000 7f2864876700 1 --2- 172.21.15.121:0/2601113812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28600bc5c0 0x7f2860065fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:44.548 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.540+0000 7f2865077700 1 --2- 172.21.15.121:0/2601113812 >> v2:172.21.15.190:3300/0 conn(0x7f28600c2b80 0x7f28600bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:44.548 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.540+0000 7f2865878700 1 -- 172.21.15.121:0/2601113812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28600bc5c0 msgr2=0x7f2860065fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:44.549 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.540+0000 7f2865878700 1 --2- 172.21.15.121:0/2601113812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28600bc5c0 0x7f2860065fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:44.549 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.544+0000 7f2865878700 1 -- 172.21.15.121:0/2601113812 >> v2:172.21.15.190:3300/0 conn(0x7f28600c2b80 msgr2=0x7f28600bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:44.549 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.544+0000 7f2865878700 1 --2- 172.21.15.121:0/2601113812 >> v2:172.21.15.190:3300/0 conn(0x7f28600c2b80 0x7f28600bc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:44.549 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.544+0000 7f2865878700 1 -- 172.21.15.121:0/2601113812 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2860066cf0 con 0x7f28600664b0 2023-11-15T02:56:44.549 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.544+0000 7f2865878700 1 --2- 172.21.15.121:0/2601113812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28600664b0 0x7f28600a5510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f2860064780 tx=0x7f2858009840).ready entity=mon.0 client_cookie=35701a51cfe04bd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:44.551 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.544+0000 7f2857fff700 1 -- 172.21.15.121:0/2601113812 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285800e070 con 0x7f28600664b0 2023-11-15T02:56:44.551 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.544+0000 7f2857fff700 1 -- 172.21.15.121:0/2601113812 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285800b970 con 0x7f28600664b0 2023-11-15T02:56:44.551 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.544+0000 7f2857fff700 1 -- 172.21.15.121:0/2601113812 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285800bcf0 con 0x7f28600664b0 2023-11-15T02:56:44.552 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.544+0000 7f28672db700 1 -- 172.21.15.121:0/2601113812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28600664b0 msgr2=0x7f28600a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:44.552 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.544+0000 7f28672db700 1 --2- 172.21.15.121:0/2601113812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28600664b0 0x7f28600a5510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f2860064780 tx=0x7f2858009840).stop 2023-11-15T02:56:44.552 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.544+0000 7f28672db700 1 -- 172.21.15.121:0/2601113812 shutdown_connections 2023-11-15T02:56:44.552 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.544+0000 7f28672db700 1 --2- 172.21.15.121:0/2601113812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28600664b0 0x7f28600a5510 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:44.552 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.544+0000 7f28672db700 1 --2- 172.21.15.121:0/2601113812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28600bc5c0 0x7f2860065fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:44.553 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.544+0000 7f28672db700 1 --2- 172.21.15.121:0/2601113812 >> v2:172.21.15.190:3300/0 conn(0x7f28600c2b80 0x7f28600bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:44.553 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.544+0000 7f28672db700 1 -- 172.21.15.121:0/2601113812 >> 172.21.15.121:0/2601113812 conn(0x7f2860062d00 msgr2=0x7f28600c6850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:44.553 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.544+0000 7f28672db700 1 -- 172.21.15.121:0/2601113812 shutdown_connections 2023-11-15T02:56:44.553 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.544+0000 7f28672db700 1 -- 172.21.15.121:0/2601113812 wait complete. 2023-11-15T02:56:44.555 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f28672db700 1 Processor -- start 2023-11-15T02:56:44.555 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f28672db700 1 -- start start 2023-11-15T02:56:44.556 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f28672db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28600664b0 0x7f28600ab520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:44.556 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f28672db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28600bc5c0 0x7f28600c03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:44.556 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f28672db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28600c2b80 0x7f28600a6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:44.556 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f28672db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28600a79b0 con 0x7f28600c2b80 2023-11-15T02:56:44.557 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f28672db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28600bd970 con 0x7f28600664b0 2023-11-15T02:56:44.557 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f28672db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28600a6da0 con 0x7f28600bc5c0 2023-11-15T02:56:44.557 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f2865878700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28600c2b80 0x7f28600a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:44.557 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f2865878700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28600c2b80 0x7f28600a6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34144/0 (socket says 172.21.15.121:34144) 2023-11-15T02:56:44.557 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f2865878700 1 -- 172.21.15.121:0/3683782973 learned_addr learned my addr 172.21.15.121:0/3683782973 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:44.558 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f2865077700 1 --2- 172.21.15.121:0/3683782973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28600664b0 0x7f28600ab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:44.558 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f2864876700 1 --2- 172.21.15.121:0/3683782973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28600bc5c0 0x7f28600c03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:44.558 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f2865878700 1 -- 172.21.15.121:0/3683782973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28600bc5c0 msgr2=0x7f28600c03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:44.558 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f2865878700 1 --2- 172.21.15.121:0/3683782973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28600bc5c0 0x7f28600c03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:44.558 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f2865878700 1 -- 172.21.15.121:0/3683782973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28600664b0 msgr2=0x7f28600ab520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:44.559 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f2865878700 1 --2- 172.21.15.121:0/3683782973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28600664b0 0x7f28600ab520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:44.559 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f2865878700 1 -- 172.21.15.121:0/3683782973 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2858009ae0 con 0x7f28600c2b80 2023-11-15T02:56:44.559 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f2865077700 1 --2- 172.21.15.121:0/3683782973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28600664b0 0x7f28600ab520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:56:44.559 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.548+0000 7f2865878700 1 --2- 172.21.15.121:0/3683782973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28600c2b80 0x7f28600a6500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f285800b610 tx=0x7f2858009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:44.559 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.552+0000 7f28557fa700 1 -- 172.21.15.121:0/3683782973 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285800bcf0 con 0x7f28600c2b80 2023-11-15T02:56:44.560 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.552+0000 7f28557fa700 1 -- 172.21.15.121:0/3683782973 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28580092e0 con 0x7f28600c2b80 2023-11-15T02:56:44.560 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.552+0000 7f28672db700 1 -- 172.21.15.121:0/3683782973 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28600a7080 con 0x7f28600c2b80 2023-11-15T02:56:44.561 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.552+0000 7f28557fa700 1 -- 172.21.15.121:0/3683782973 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285801d6b0 con 0x7f28600c2b80 2023-11-15T02:56:44.561 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.552+0000 7f28672db700 1 -- 172.21.15.121:0/3683782973 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28600a7640 con 0x7f28600c2b80 2023-11-15T02:56:44.562 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.552+0000 7f28672db700 1 -- 172.21.15.121:0/3683782973 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28600a7b60 con 0x7f28600c2b80 2023-11-15T02:56:44.563 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.552+0000 7f28557fa700 1 -- 172.21.15.121:0/3683782973 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2858003040 con 0x7f28600c2b80 2023-11-15T02:56:44.563 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.556+0000 7f28557fa700 1 --2- 172.21.15.121:0/3683782973 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f284405a510 0x7f284405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:44.563 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.556+0000 7f28557fa700 1 -- 172.21.15.121:0/3683782973 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4892+0+0 (secure 0 0 0) 0x7f2858075580 con 0x7f28600c2b80 2023-11-15T02:56:44.563 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.556+0000 7f2865077700 1 --2- 172.21.15.121:0/3683782973 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f284405a510 0x7f284405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:44.563 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.556+0000 7f2865077700 1 --2- 172.21.15.121:0/3683782973 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f284405a510 0x7f284405c9d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f285c009c00 tx=0x7f285c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:44.567 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.564+0000 7f28557fa700 1 -- 172.21.15.121:0/3683782973 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f28600a7b60 con 0x7f28600c2b80 2023-11-15T02:56:44.737 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.732+0000 7f28672db700 1 -- 172.21.15.121:0/3683782973 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28600a7b60 con 0x7f28600c2b80 2023-11-15T02:56:44.739 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.736+0000 7f28557fa700 1 -- 172.21.15.121:0/3683782973 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+12090 (secure 0 0 0) 0x7f28600a7b60 con 0x7f28600c2b80 2023-11-15T02:56:44.740 INFO:teuthology.misc.health.smithi121.stdout: 2023-11-15T02:56:44.740 INFO:teuthology.misc.health.smithi121.stdout:{"epoch":14,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:44.150053+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:56:44.742 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.736+0000 7f28672db700 1 -- 172.21.15.121:0/3683782973 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f284405a510 msgr2=0x7f284405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:44.743 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.736+0000 7f28672db700 1 --2- 172.21.15.121:0/3683782973 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f284405a510 0x7f284405c9d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f285c009c00 tx=0x7f285c008040).stop 2023-11-15T02:56:44.743 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.736+0000 7f28672db700 1 -- 172.21.15.121:0/3683782973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28600c2b80 msgr2=0x7f28600a6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:44.743 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.736+0000 7f28672db700 1 --2- 172.21.15.121:0/3683782973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28600c2b80 0x7f28600a6500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f285800b610 tx=0x7f2858009840).stop 2023-11-15T02:56:44.743 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.736+0000 7f28672db700 1 -- 172.21.15.121:0/3683782973 shutdown_connections 2023-11-15T02:56:44.744 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.736+0000 7f28672db700 1 --2- 172.21.15.121:0/3683782973 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f284405a510 0x7f284405c9d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:44.744 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.736+0000 7f28672db700 1 --2- 172.21.15.121:0/3683782973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28600c2b80 0x7f28600a6500 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:44.744 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.736+0000 7f28672db700 1 --2- 172.21.15.121:0/3683782973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28600bc5c0 0x7f28600c03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:44.744 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.736+0000 7f28672db700 1 --2- 172.21.15.121:0/3683782973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28600664b0 0x7f28600ab520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:44.744 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.736+0000 7f28672db700 1 -- 172.21.15.121:0/3683782973 >> 172.21.15.121:0/3683782973 conn(0x7f2860062d00 msgr2=0x7f286014ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:44.745 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.736+0000 7f28672db700 1 -- 172.21.15.121:0/3683782973 shutdown_connections 2023-11-15T02:56:44.745 INFO:teuthology.misc.health.smithi121.stderr:2023-11-15T02:56:44.736+0000 7f28672db700 1 -- 172.21.15.121:0/3683782973 wait complete. 2023-11-15T02:56:44.757 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-15T02:56:44.757 INFO:tasks.ceph:Creating RBD pool 2023-11-15T02:56:44.758 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd pool create rbd 8 2023-11-15T02:56:44.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.908+0000 7f6629f07700 1 Processor -- start 2023-11-15T02:56:44.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.908+0000 7f6629f07700 1 -- start start 2023-11-15T02:56:44.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.912+0000 7f6629f07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f662414e380 0x7f66240c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:44.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.912+0000 7f6629f07700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f66240c6040 0x7f66240acb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:44.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.912+0000 7f6629f07700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66240ad500 con 0x7f66240ad050 2023-11-15T02:56:44.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.912+0000 7f6629f07700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66240c8320 con 0x7f66240c6040 2023-11-15T02:56:44.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.912+0000 7f6629f07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66240c7080 con 0x7f662414e380 2023-11-15T02:56:44.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.912+0000 7f661bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f66240c6040 0x7f66240acb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:44.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.912+0000 7f6628f05700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f662414e380 0x7f66240c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:44.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.912+0000 7f6629706700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f66240ad050 0x7f6624062190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:42332/0 (socket says 172.21.15.121:42332) 2023-11-15T02:56:44.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.912+0000 7f6629706700 1 -- 172.21.15.121:0/559321873 learned_addr learned my addr 172.21.15.121:0/559321873 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:44.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.912+0000 7f661bfff700 1 -- 172.21.15.121:0/559321873 >> v2:172.21.15.190:3300/0 conn(0x7f662414e380 msgr2=0x7f66240c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:44.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.912+0000 7f661bfff700 1 --2- 172.21.15.121:0/559321873 >> v2:172.21.15.190:3300/0 conn(0x7f662414e380 0x7f66240c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:44.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.912+0000 7f661bfff700 1 -- 172.21.15.121:0/559321873 >> v1:172.21.15.121:6789/0 conn(0x7f66240ad050 legacy=0x7f6624062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:44.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.912+0000 7f661b7fe700 1 -- 172.21.15.121:0/559321873 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507896655 0 0) 0x7f661c001490 con 0x7f66240ad050 2023-11-15T02:56:44.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.912+0000 7f661bfff700 1 -- 172.21.15.121:0/559321873 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66240c7e90 con 0x7f66240c6040 2023-11-15T02:56:44.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.916+0000 7f661bfff700 1 --2- 172.21.15.121:0/559321873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66240c6040 0x7f66240acb60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f6624065190 tx=0x7f6614009060).ready entity=mon.0 client_cookie=25feade04e988d05 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:44.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.916+0000 7f661b7fe700 1 -- 172.21.15.121:0/559321873 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661400f510 con 0x7f66240c6040 2023-11-15T02:56:44.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.916+0000 7f661b7fe700 1 -- 172.21.15.121:0/559321873 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661400f670 con 0x7f66240c6040 2023-11-15T02:56:44.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.916+0000 7f661b7fe700 1 -- 172.21.15.121:0/559321873 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661400f9f0 con 0x7f66240c6040 2023-11-15T02:56:44.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.916+0000 7f6629f07700 1 -- 172.21.15.121:0/559321873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66240c6040 msgr2=0x7f66240acb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:44.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.916+0000 7f6629f07700 1 --2- 172.21.15.121:0/559321873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66240c6040 0x7f66240acb60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f6624065190 tx=0x7f6614009060).stop 2023-11-15T02:56:44.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.916+0000 7f6629f07700 1 -- 172.21.15.121:0/559321873 shutdown_connections 2023-11-15T02:56:44.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.916+0000 7f6629f07700 1 --2- 172.21.15.121:0/559321873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66240c6040 0x7f66240acb60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:44.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.916+0000 7f6629f07700 1 --2- 172.21.15.121:0/559321873 >> v2:172.21.15.190:3300/0 conn(0x7f662414e380 0x7f66240c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:44.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.916+0000 7f6629f07700 1 -- 172.21.15.121:0/559321873 >> 172.21.15.121:0/559321873 conn(0x7f6624067a60 msgr2=0x7f6624067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:44.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.916+0000 7f6629f07700 1 -- 172.21.15.121:0/559321873 shutdown_connections 2023-11-15T02:56:44.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.916+0000 7f6629f07700 1 -- 172.21.15.121:0/559321873 wait complete. 2023-11-15T02:56:44.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.916+0000 7f6629f07700 1 Processor -- start 2023-11-15T02:56:44.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f6629f07700 1 -- start start 2023-11-15T02:56:44.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f6629f07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66240ad050 0x7f66240c3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:44.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f6629f07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66240c6040 0x7f66240b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:44.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f6629f07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f662414e380 0x7f66240c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:44.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f6629f07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66241430a0 con 0x7f66240c6040 2023-11-15T02:56:44.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f6629f07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6624155e50 con 0x7f662414e380 2023-11-15T02:56:44.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f6629f07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6624156130 con 0x7f66240ad050 2023-11-15T02:56:44.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f661bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66240c6040 0x7f66240b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:44.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f661bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66240c6040 0x7f66240b01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44618/0 (socket says 172.21.15.121:44618) 2023-11-15T02:56:44.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f661bfff700 1 -- 172.21.15.121:0/1031693986 learned_addr learned my addr 172.21.15.121:0/1031693986 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:44.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f6628f05700 1 --2- 172.21.15.121:0/1031693986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66240ad050 0x7f66240c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:44.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f6629706700 1 --2- 172.21.15.121:0/1031693986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f662414e380 0x7f66240c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:44.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f661bfff700 1 -- 172.21.15.121:0/1031693986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66240ad050 msgr2=0x7f66240c3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:44.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f661bfff700 1 --2- 172.21.15.121:0/1031693986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66240ad050 0x7f66240c3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:44.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f661bfff700 1 -- 172.21.15.121:0/1031693986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f662414e380 msgr2=0x7f66240c37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:44.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f661bfff700 1 --2- 172.21.15.121:0/1031693986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f662414e380 0x7f66240c37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:44.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f661bfff700 1 -- 172.21.15.121:0/1031693986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66140092a0 con 0x7f66240c6040 2023-11-15T02:56:44.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f661bfff700 1 --2- 172.21.15.121:0/1031693986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66240c6040 0x7f66240b01d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f6614009dd0 tx=0x7f6614017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:44.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f6618ff9700 1 -- 172.21.15.121:0/1031693986 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661400f7e0 con 0x7f66240c6040 2023-11-15T02:56:44.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f6618ff9700 1 -- 172.21.15.121:0/1031693986 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661400f940 con 0x7f66240c6040 2023-11-15T02:56:44.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f6629f07700 1 -- 172.21.15.121:0/1031693986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6624156410 con 0x7f66240c6040 2023-11-15T02:56:44.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f6618ff9700 1 -- 172.21.15.121:0/1031693986 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661401b660 con 0x7f66240c6040 2023-11-15T02:56:44.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.920+0000 7f6629f07700 1 -- 172.21.15.121:0/1031693986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66241569d0 con 0x7f66240c6040 2023-11-15T02:56:44.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.924+0000 7f6629f07700 1 -- 172.21.15.121:0/1031693986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662406bcf0 con 0x7f66240c6040 2023-11-15T02:56:44.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.924+0000 7f6618ff9700 1 -- 172.21.15.121:0/1031693986 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f661401ba10 con 0x7f66240c6040 2023-11-15T02:56:44.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.924+0000 7f6618ff9700 1 --2- 172.21.15.121:0/1031693986 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f661005a510 0x7f661005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:44.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.924+0000 7f6618ff9700 1 -- 172.21.15.121:0/1031693986 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4892+0+0 (secure 0 0 0) 0x7f6614075c70 con 0x7f66240c6040 2023-11-15T02:56:44.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.928+0000 7f6628f05700 1 --2- 172.21.15.121:0/1031693986 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f661005a510 0x7f661005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:44.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.928+0000 7f6628f05700 1 --2- 172.21.15.121:0/1031693986 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f661005a510 0x7f661005c9d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f6620009b20 tx=0x7f6620008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:44.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:44.932+0000 7f6618ff9700 1 -- 172.21.15.121:0/1031693986 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6614049860 con 0x7f66240c6040 2023-11-15T02:56:45.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.096+0000 7f6629f07700 1 -- 172.21.15.121:0/1031693986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f662414e760 con 0x7f66240c6040 2023-11-15T02:56:45.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.168+0000 7f6618ff9700 1 -- 172.21.15.121:0/1031693986 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v15) v1 ==== 109+0+0 (secure 0 0 0) 0x7f661404ce80 con 0x7f66240c6040 2023-11-15T02:56:45.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.168+0000 7f6629f07700 1 -- 172.21.15.121:0/1031693986 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f661005a510 msgr2=0x7f661005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:45.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.168+0000 7f6629f07700 1 --2- 172.21.15.121:0/1031693986 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f661005a510 0x7f661005c9d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f6620009b20 tx=0x7f6620008040).stop 2023-11-15T02:56:45.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.168+0000 7f6629f07700 1 -- 172.21.15.121:0/1031693986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66240c6040 msgr2=0x7f66240b01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:45.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.168+0000 7f6629f07700 1 --2- 172.21.15.121:0/1031693986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66240c6040 0x7f66240b01d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f6614009dd0 tx=0x7f6614017000).stop 2023-11-15T02:56:45.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.168+0000 7f6629f07700 1 -- 172.21.15.121:0/1031693986 shutdown_connections 2023-11-15T02:56:45.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.168+0000 7f6629f07700 1 --2- 172.21.15.121:0/1031693986 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f661005a510 0x7f661005c9d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:45.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.168+0000 7f6629f07700 1 --2- 172.21.15.121:0/1031693986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f662414e380 0x7f66240c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:45.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.168+0000 7f6629f07700 1 --2- 172.21.15.121:0/1031693986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66240c6040 0x7f66240b01d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:45.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.168+0000 7f6629f07700 1 --2- 172.21.15.121:0/1031693986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66240ad050 0x7f66240c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:45.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.172+0000 7f6629f07700 1 -- 172.21.15.121:0/1031693986 >> 172.21.15.121:0/1031693986 conn(0x7f6624067a60 msgr2=0x7f662414c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:45.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.172+0000 7f6629f07700 1 -- 172.21.15.121:0/1031693986 shutdown_connections 2023-11-15T02:56:45.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.172+0000 7f6629f07700 1 -- 172.21.15.121:0/1031693986 wait complete. 2023-11-15T02:56:45.178 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'rbd' created 2023-11-15T02:56:45.195 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-11-15T02:56:45.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.340+0000 7f50f1db9700 1 Processor -- start 2023-11-15T02:56:45.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.340+0000 7f50f1db9700 1 -- start start 2023-11-15T02:56:45.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.340+0000 7f50f1db9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f50ec0a9f80 0x7f50ec158260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:45.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.340+0000 7f50f1db9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0c7a90 0x7f50ec154b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:45.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.340+0000 7f50f1db9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ec0c6170 con 0x7f50ec0c5a00 2023-11-15T02:56:45.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.340+0000 7f50f1db9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50ec154000 con 0x7f50ec0a9f80 2023-11-15T02:56:45.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.340+0000 7f50f1db9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec157dd0 con 0x7f50ec0c7a90 2023-11-15T02:56:45.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.344+0000 7f50f0db7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f50ec0a9f80 0x7f50ec158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:45.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.344+0000 7f50f15b8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0c7a90 0x7f50ec154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:45.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.344+0000 7f50f15b8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0c7a90 0x7f50ec154b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:55222/0 (socket says 172.21.15.121:55222) 2023-11-15T02:56:45.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.344+0000 7f50f15b8700 1 -- 172.21.15.121:0/2705843143 learned_addr learned my addr 172.21.15.121:0/2705843143 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:45.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.344+0000 7f50ebfff700 1 --1- 172.21.15.121:0/2705843143 >> v1:172.21.15.190:6789/0 conn(0x7f50ec0c5a00 0x7f50ec0c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:54568/0 (socket says 172.21.15.121:54568) 2023-11-15T02:56:45.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.344+0000 7f50eb7fe700 1 -- 172.21.15.121:0/2705843143 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 107821348 0 0) 0x7f50ec0c6170 con 0x7f50ec0c5a00 2023-11-15T02:56:45.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.344+0000 7f50f15b8700 1 -- 172.21.15.121:0/2705843143 >> v2:172.21.15.121:3300/0 conn(0x7f50ec0a9f80 msgr2=0x7f50ec158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:45.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.344+0000 7f50f15b8700 1 --2- 172.21.15.121:0/2705843143 >> v2:172.21.15.121:3300/0 conn(0x7f50ec0a9f80 0x7f50ec158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:45.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.344+0000 7f50f15b8700 1 -- 172.21.15.121:0/2705843143 >> v1:172.21.15.190:6789/0 conn(0x7f50ec0c5a00 legacy=0x7f50ec0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:45.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.344+0000 7f50f15b8700 1 -- 172.21.15.121:0/2705843143 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec155080 con 0x7f50ec0c7a90 2023-11-15T02:56:45.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.344+0000 7f50f15b8700 1 --2- 172.21.15.121:0/2705843143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0c7a90 0x7f50ec154b90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f50e4009f20 tx=0x7f50e4009840).ready entity=mon.2 client_cookie=cc872ec44c04afb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:45.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.344+0000 7f50eb7fe700 1 -- 172.21.15.121:0/2705843143 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e400e070 con 0x7f50ec0c7a90 2023-11-15T02:56:45.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.344+0000 7f50eb7fe700 1 -- 172.21.15.121:0/2705843143 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e400b9d0 con 0x7f50ec0c7a90 2023-11-15T02:56:45.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.344+0000 7f50eb7fe700 1 -- 172.21.15.121:0/2705843143 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e400bd50 con 0x7f50ec0c7a90 2023-11-15T02:56:45.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.344+0000 7f50f1db9700 1 -- 172.21.15.121:0/2705843143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0c7a90 msgr2=0x7f50ec154b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:45.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.344+0000 7f50f1db9700 1 --2- 172.21.15.121:0/2705843143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0c7a90 0x7f50ec154b90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f50e4009f20 tx=0x7f50e4009840).stop 2023-11-15T02:56:45.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.344+0000 7f50f1db9700 1 -- 172.21.15.121:0/2705843143 shutdown_connections 2023-11-15T02:56:45.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.344+0000 7f50f1db9700 1 --2- 172.21.15.121:0/2705843143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0c7a90 0x7f50ec154b90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:45.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.344+0000 7f50f1db9700 1 --2- 172.21.15.121:0/2705843143 >> v2:172.21.15.121:3300/0 conn(0x7f50ec0a9f80 0x7f50ec158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:45.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.348+0000 7f50f1db9700 1 -- 172.21.15.121:0/2705843143 >> 172.21.15.121:0/2705843143 conn(0x7f50ec067a60 msgr2=0x7f50ec067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:45.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.348+0000 7f50f1db9700 1 -- 172.21.15.121:0/2705843143 shutdown_connections 2023-11-15T02:56:45.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.348+0000 7f50f1db9700 1 -- 172.21.15.121:0/2705843143 wait complete. 2023-11-15T02:56:45.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.348+0000 7f50f1db9700 1 Processor -- start 2023-11-15T02:56:45.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.348+0000 7f50f1db9700 1 -- start start 2023-11-15T02:56:45.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.348+0000 7f50f1db9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0a9f80 0x7f50ec0b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:45.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.348+0000 7f50f1db9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec0c7a90 0x7f50ec0c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:45.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.348+0000 7f50f1db9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50ec14ee10 0x7f50ec0ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:45.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.348+0000 7f50f1db9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec0c9ac0 con 0x7f50ec14ee10 2023-11-15T02:56:45.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.348+0000 7f50f1db9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec0c8be0 con 0x7f50ec0c7a90 2023-11-15T02:56:45.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.348+0000 7f50f1db9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec0c8eb0 con 0x7f50ec0a9f80 2023-11-15T02:56:45.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.348+0000 7f50f15b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50ec14ee10 0x7f50ec0ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:45.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.348+0000 7f50f15b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50ec14ee10 0x7f50ec0ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44628/0 (socket says 172.21.15.121:44628) 2023-11-15T02:56:45.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.348+0000 7f50f15b8700 1 -- 172.21.15.121:0/3358831553 learned_addr learned my addr 172.21.15.121:0/3358831553 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:45.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.348+0000 7f50f0db7700 1 --2- 172.21.15.121:0/3358831553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0a9f80 0x7f50ec0b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:45.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.348+0000 7f50ebfff700 1 --2- 172.21.15.121:0/3358831553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec0c7a90 0x7f50ec0c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:45.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.352+0000 7f50f15b8700 1 -- 172.21.15.121:0/3358831553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0a9f80 msgr2=0x7f50ec0b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:45.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.352+0000 7f50f15b8700 1 --2- 172.21.15.121:0/3358831553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0a9f80 0x7f50ec0b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:45.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.352+0000 7f50f15b8700 1 -- 172.21.15.121:0/3358831553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec0c7a90 msgr2=0x7f50ec0c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:45.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.352+0000 7f50f15b8700 1 --2- 172.21.15.121:0/3358831553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec0c7a90 0x7f50ec0c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:45.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.352+0000 7f50f15b8700 1 -- 172.21.15.121:0/3358831553 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e4009ae0 con 0x7f50ec14ee10 2023-11-15T02:56:45.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.352+0000 7f50f15b8700 1 --2- 172.21.15.121:0/3358831553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50ec14ee10 0x7f50ec0ae280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f50e400b610 tx=0x7f50e4009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:45.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.352+0000 7f50e8ff9700 1 -- 172.21.15.121:0/3358831553 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e40092e0 con 0x7f50ec14ee10 2023-11-15T02:56:45.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.352+0000 7f50e8ff9700 1 -- 172.21.15.121:0/3358831553 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e4007610 con 0x7f50ec14ee10 2023-11-15T02:56:45.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.352+0000 7f50e8ff9700 1 -- 172.21.15.121:0/3358831553 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e40079d0 con 0x7f50ec14ee10 2023-11-15T02:56:45.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.352+0000 7f50f1db9700 1 -- 172.21.15.121:0/3358831553 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ec0c9190 con 0x7f50ec14ee10 2023-11-15T02:56:45.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.352+0000 7f50f1db9700 1 -- 172.21.15.121:0/3358831553 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ec0c9750 con 0x7f50ec14ee10 2023-11-15T02:56:45.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.352+0000 7f50f1db9700 1 -- 172.21.15.121:0/3358831553 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ec0c9c70 con 0x7f50ec14ee10 2023-11-15T02:56:45.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.352+0000 7f50e8ff9700 1 -- 172.21.15.121:0/3358831553 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f50e4003040 con 0x7f50ec14ee10 2023-11-15T02:56:45.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.356+0000 7f50e8ff9700 1 --2- 172.21.15.121:0/3358831553 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50d005a510 0x7f50d005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:45.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.356+0000 7f50e8ff9700 1 -- 172.21.15.121:0/3358831553 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5258+0+0 (secure 0 0 0) 0x7f50e4075ec0 con 0x7f50ec14ee10 2023-11-15T02:56:45.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.356+0000 7f50f0db7700 1 --2- 172.21.15.121:0/3358831553 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50d005a510 0x7f50d005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:45.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.356+0000 7f50f0db7700 1 --2- 172.21.15.121:0/3358831553 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50d005a510 0x7f50d005c9d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f50e0009c30 tx=0x7f50e0009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:45.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.364+0000 7f50e8ff9700 1 -- 172.21.15.121:0/3358831553 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f50ec0c9c70 con 0x7f50ec14ee10 2023-11-15T02:56:45.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:45.520+0000 7f50f1db9700 1 -- 172.21.15.121:0/3358831553 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f50ec06bcf0 con 0x7f50ec14ee10 2023-11-15T02:56:46.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.188+0000 7f50e8ff9700 1 -- 172.21.15.121:0/3358831553 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v16) v1 ==== 173+0+0 (secure 0 0 0) 0x7f50e4023030 con 0x7f50ec14ee10 2023-11-15T02:56:46.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.192+0000 7f50f1db9700 1 -- 172.21.15.121:0/3358831553 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50d005a510 msgr2=0x7f50d005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:46.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.192+0000 7f50f1db9700 1 --2- 172.21.15.121:0/3358831553 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50d005a510 0x7f50d005c9d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f50e0009c30 tx=0x7f50e0009060).stop 2023-11-15T02:56:46.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.192+0000 7f50f1db9700 1 -- 172.21.15.121:0/3358831553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50ec14ee10 msgr2=0x7f50ec0ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:46.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.192+0000 7f50f1db9700 1 --2- 172.21.15.121:0/3358831553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50ec14ee10 0x7f50ec0ae280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f50e400b610 tx=0x7f50e4009840).stop 2023-11-15T02:56:46.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.192+0000 7f50f1db9700 1 -- 172.21.15.121:0/3358831553 shutdown_connections 2023-11-15T02:56:46.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.192+0000 7f50f1db9700 1 --2- 172.21.15.121:0/3358831553 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50d005a510 0x7f50d005c9d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.192+0000 7f50f1db9700 1 --2- 172.21.15.121:0/3358831553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50ec14ee10 0x7f50ec0ae280 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.192+0000 7f50f1db9700 1 --2- 172.21.15.121:0/3358831553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec0c7a90 0x7f50ec0c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.192+0000 7f50f1db9700 1 --2- 172.21.15.121:0/3358831553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0a9f80 0x7f50ec0b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.192+0000 7f50f1db9700 1 -- 172.21.15.121:0/3358831553 >> 172.21.15.121:0/3358831553 conn(0x7f50ec067a60 msgr2=0x7f50ec150d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:46.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.192+0000 7f50f1db9700 1 -- 172.21.15.121:0/3358831553 shutdown_connections 2023-11-15T02:56:46.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.192+0000 7f50f1db9700 1 -- 172.21.15.121:0/3358831553 wait complete. 2023-11-15T02:56:46.201 INFO:teuthology.orchestra.run.smithi121.stderr:enabled application 'rbd' on pool 'rbd' 2023-11-15T02:56:46.220 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-15T02:56:46.220 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-15T02:56:46.221 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-15T02:56:46.224 INFO:tasks.ceph.mds.a:Started 2023-11-15T02:56:46.225 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-15T02:56:46.225 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-15T02:56:46.230 INFO:tasks.ceph.mds.c:Started 2023-11-15T02:56:46.230 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-15T02:56:46.231 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-15T02:56:46.238 INFO:tasks.ceph.mds.b:Started 2023-11-15T02:56:46.238 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-15T02:56:46.238 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-15T02:56:46.245 INFO:tasks.ceph.mds.d:Started 2023-11-15T02:56:46.245 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-15T02:56:46.246 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:56:46.334 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T02:56:46.335 INFO:tasks.ceph.mds.c.smithi121.stdout:starting mds.c at 2023-11-15T02:56:46.346 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-11-15T02:56:46.351 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-11-15T02:56:46.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.384+0000 7fa322ff9700 1 Processor -- start 2023-11-15T02:56:46.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.384+0000 7fa322ff9700 1 -- start start 2023-11-15T02:56:46.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa322ff9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa31c0cabd0 0x7fa31c0cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:46.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa322ff9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa31c0ad0f0 0x7fa31c0ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:46.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa322ff9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31c0ad650 con 0x7fa31c0acc90 2023-11-15T02:56:46.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa322ff9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31c065fc0 con 0x7fa31c0ad0f0 2023-11-15T02:56:46.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa322ff9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c0662a0 con 0x7fa31c0cabd0 2023-11-15T02:56:46.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa3217f6700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa31c0acc90 0x7fa31c066d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:42340/0 (socket says 172.21.15.121:42340) 2023-11-15T02:56:46.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa3217f6700 1 -- 172.21.15.121:0/2387649608 learned_addr learned my addr 172.21.15.121:0/2387649608 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:46.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa3227f8700 1 --2- 172.21.15.121:0/2387649608 >> v2:172.21.15.121:3300/0 conn(0x7fa31c0ad0f0 0x7fa31c0ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:46.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa321ff7700 1 --2- 172.21.15.121:0/2387649608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa31c0cabd0 0x7fa31c0cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:46.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa320ff5700 1 -- 172.21.15.121:0/2387649608 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244942259 0 0) 0x7fa31c0ad650 con 0x7fa31c0acc90 2023-11-15T02:56:46.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa320ff5700 1 -- 172.21.15.121:0/2387649608 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3080033f0 con 0x7fa31c0acc90 2023-11-15T02:56:46.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa3227f8700 1 -- 172.21.15.121:0/2387649608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa31c0cabd0 msgr2=0x7fa31c0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:46.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa3227f8700 1 --2- 172.21.15.121:0/2387649608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa31c0cabd0 0x7fa31c0cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa3227f8700 1 -- 172.21.15.121:0/2387649608 >> v1:172.21.15.121:6789/0 conn(0x7fa31c0acc90 legacy=0x7fa31c066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:46.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa3227f8700 1 -- 172.21.15.121:0/2387649608 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c066580 con 0x7fa31c0ad0f0 2023-11-15T02:56:46.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa3227f8700 1 --2- 172.21.15.121:0/2387649608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa31c0ad0f0 0x7fa31c0ae190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fa314009e90 tx=0x7fa314009780).ready entity=mon.0 client_cookie=87626993fe7b9732 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:46.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa320ff5700 1 -- 172.21.15.121:0/2387649608 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31400e070 con 0x7fa31c0ad0f0 2023-11-15T02:56:46.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa320ff5700 1 -- 172.21.15.121:0/2387649608 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31400bb20 con 0x7fa31c0ad0f0 2023-11-15T02:56:46.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa320ff5700 1 -- 172.21.15.121:0/2387649608 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31400bea0 con 0x7fa31c0ad0f0 2023-11-15T02:56:46.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa322ff9700 1 -- 172.21.15.121:0/2387649608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa31c0ad0f0 msgr2=0x7fa31c0ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:46.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa322ff9700 1 --2- 172.21.15.121:0/2387649608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa31c0ad0f0 0x7fa31c0ae190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fa314009e90 tx=0x7fa314009780).stop 2023-11-15T02:56:46.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa322ff9700 1 -- 172.21.15.121:0/2387649608 shutdown_connections 2023-11-15T02:56:46.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa322ff9700 1 --2- 172.21.15.121:0/2387649608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa31c0ad0f0 0x7fa31c0ae190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa322ff9700 1 --2- 172.21.15.121:0/2387649608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa31c0cabd0 0x7fa31c0cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa322ff9700 1 -- 172.21.15.121:0/2387649608 >> 172.21.15.121:0/2387649608 conn(0x7fa31c062190 msgr2=0x7fa31c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:46.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa322ff9700 1 -- 172.21.15.121:0/2387649608 shutdown_connections 2023-11-15T02:56:46.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.388+0000 7fa322ff9700 1 -- 172.21.15.121:0/2387649608 wait complete. 2023-11-15T02:56:46.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.392+0000 7fa322ff9700 1 Processor -- start 2023-11-15T02:56:46.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.392+0000 7fa322ff9700 1 -- start start 2023-11-15T02:56:46.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.392+0000 7fa322ff9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa31c0acc90 0x7fa31c13bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:46.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.392+0000 7fa322ff9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa31c0ad0f0 0x7fa31c155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:46.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.392+0000 7fa322ff9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa31c0cabd0 0x7fa31c153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:46.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.392+0000 7fa322ff9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c13c2b0 con 0x7fa31c0ad0f0 2023-11-15T02:56:46.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.392+0000 7fa322ff9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c155090 con 0x7fa31c0acc90 2023-11-15T02:56:46.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.392+0000 7fa322ff9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c154480 con 0x7fa31c0cabd0 2023-11-15T02:56:46.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.392+0000 7fa3217f6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa31c0ad0f0 0x7fa31c155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:46.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.392+0000 7fa3217f6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa31c0ad0f0 0x7fa31c155520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44724/0 (socket says 172.21.15.121:44724) 2023-11-15T02:56:46.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.392+0000 7fa3217f6700 1 -- 172.21.15.121:0/64480873 learned_addr learned my addr 172.21.15.121:0/64480873 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:46.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.392+0000 7fa321ff7700 1 --2- 172.21.15.121:0/64480873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa31c0acc90 0x7fa31c13bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:46.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.392+0000 7fa3227f8700 1 --2- 172.21.15.121:0/64480873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa31c0cabd0 0x7fa31c153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:46.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.392+0000 7fa3217f6700 1 -- 172.21.15.121:0/64480873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa31c0cabd0 msgr2=0x7fa31c153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:46.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.392+0000 7fa3217f6700 1 --2- 172.21.15.121:0/64480873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa31c0cabd0 0x7fa31c153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.392+0000 7fa3217f6700 1 -- 172.21.15.121:0/64480873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa31c0acc90 msgr2=0x7fa31c13bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:46.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.392+0000 7fa3217f6700 1 --2- 172.21.15.121:0/64480873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa31c0acc90 0x7fa31c13bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.392+0000 7fa3217f6700 1 -- 172.21.15.121:0/64480873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa314009a20 con 0x7fa31c0ad0f0 2023-11-15T02:56:46.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.396+0000 7fa3217f6700 1 --2- 172.21.15.121:0/64480873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa31c0ad0f0 0x7fa31c155520 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fa30c00c4c0 tx=0x7fa30c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:46.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.396+0000 7fa311ffb700 1 -- 172.21.15.121:0/64480873 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0072e0 con 0x7fa31c0ad0f0 2023-11-15T02:56:46.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.396+0000 7fa311ffb700 1 -- 172.21.15.121:0/64480873 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c007440 con 0x7fa31c0ad0f0 2023-11-15T02:56:46.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.396+0000 7fa322ff9700 1 -- 172.21.15.121:0/64480873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31c154760 con 0x7fa31c0ad0f0 2023-11-15T02:56:46.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.396+0000 7fa311ffb700 1 -- 172.21.15.121:0/64480873 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0057a0 con 0x7fa31c0ad0f0 2023-11-15T02:56:46.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.396+0000 7fa322ff9700 1 -- 172.21.15.121:0/64480873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31c154d20 con 0x7fa31c0ad0f0 2023-11-15T02:56:46.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.396+0000 7fa322ff9700 1 -- 172.21.15.121:0/64480873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31c06bcf0 con 0x7fa31c0ad0f0 2023-11-15T02:56:46.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.396+0000 7fa311ffb700 1 -- 172.21.15.121:0/64480873 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa30c005bd0 con 0x7fa31c0ad0f0 2023-11-15T02:56:46.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.400+0000 7fa311ffb700 1 --2- 172.21.15.121:0/64480873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa30005a4e0 0x7fa30005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:46.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.400+0000 7fa311ffb700 1 -- 172.21.15.121:0/64480873 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fa30c073ef0 con 0x7fa31c0ad0f0 2023-11-15T02:56:46.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.400+0000 7fa321ff7700 1 --2- 172.21.15.121:0/64480873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa30005a4e0 0x7fa30005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:46.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.400+0000 7fa321ff7700 1 --2- 172.21.15.121:0/64480873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa30005a4e0 0x7fa30005c9a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fa318006fd0 tx=0x7fa318008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:46.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.404+0000 7fa311ffb700 1 -- 172.21.15.121:0/64480873 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa30c047960 con 0x7fa31c0ad0f0 2023-11-15T02:56:46.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.576+0000 7fa322ff9700 1 -- 172.21.15.121:0/64480873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa31c066860 con 0x7fa31c0ad0f0 2023-11-15T02:56:46.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.576+0000 7fa311ffb700 1 -- 172.21.15.121:0/64480873 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+13770 (secure 0 0 0) 0x7fa30c018030 con 0x7fa31c0ad0f0 2023-11-15T02:56:46.582 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:56:46.583 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":16,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:46.178348+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:56:46.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.580+0000 7fa322ff9700 1 -- 172.21.15.121:0/64480873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa30005a4e0 msgr2=0x7fa30005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:46.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.580+0000 7fa322ff9700 1 --2- 172.21.15.121:0/64480873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa30005a4e0 0x7fa30005c9a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fa318006fd0 tx=0x7fa318008040).stop 2023-11-15T02:56:46.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.580+0000 7fa322ff9700 1 -- 172.21.15.121:0/64480873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa31c0ad0f0 msgr2=0x7fa31c155520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:46.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.580+0000 7fa322ff9700 1 --2- 172.21.15.121:0/64480873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa31c0ad0f0 0x7fa31c155520 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fa30c00c4c0 tx=0x7fa30c007c50).stop 2023-11-15T02:56:46.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.580+0000 7fa322ff9700 1 -- 172.21.15.121:0/64480873 shutdown_connections 2023-11-15T02:56:46.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.580+0000 7fa322ff9700 1 --2- 172.21.15.121:0/64480873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa30005a4e0 0x7fa30005c9a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.580+0000 7fa322ff9700 1 --2- 172.21.15.121:0/64480873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa31c0cabd0 0x7fa31c153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.580+0000 7fa322ff9700 1 --2- 172.21.15.121:0/64480873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa31c0ad0f0 0x7fa31c155520 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.580+0000 7fa322ff9700 1 --2- 172.21.15.121:0/64480873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa31c0acc90 0x7fa31c13bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.580+0000 7fa322ff9700 1 -- 172.21.15.121:0/64480873 >> 172.21.15.121:0/64480873 conn(0x7fa31c062190 msgr2=0x7fa31c150670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:46.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.580+0000 7fa322ff9700 1 -- 172.21.15.121:0/64480873 shutdown_connections 2023-11-15T02:56:46.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.580+0000 7fa322ff9700 1 -- 172.21.15.121:0/64480873 wait complete. 2023-11-15T02:56:46.606 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-15T02:56:46.607 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T02:56:46.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.764+0000 7fc6bd4b9700 1 Processor -- start 2023-11-15T02:56:46.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.764+0000 7fc6bd4b9700 1 -- start start 2023-11-15T02:56:46.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.768+0000 7fc6bd4b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc6b80acd00 0x7fc6b80ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:46.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.768+0000 7fc6bd4b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6b80c5c70 0x7fc6b81536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:46.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.768+0000 7fc6bd4b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b8065ea0 con 0x7fc6b8066330 2023-11-15T02:56:46.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.768+0000 7fc6bd4b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b8066bb0 con 0x7fc6b80acd00 2023-11-15T02:56:46.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.768+0000 7fc6bd4b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b8065890 con 0x7fc6b80c5c70 2023-11-15T02:56:46.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.768+0000 7fc6b7fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc6b80acd00 0x7fc6b80ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:46.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.768+0000 7fc6b77fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6b80c5c70 0x7fc6b81536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:46.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.768+0000 7fc6bccb8700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fc6b8066330 0x7fc6b8153bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:54582/0 (socket says 172.21.15.121:54582) 2023-11-15T02:56:46.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.768+0000 7fc6bccb8700 1 -- 172.21.15.121:0/1500553497 learned_addr learned my addr 172.21.15.121:0/1500553497 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:46.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.768+0000 7fc6b7fff700 1 -- 172.21.15.121:0/1500553497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6b80c5c70 msgr2=0x7fc6b81536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:46.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.768+0000 7fc6b7fff700 1 --2- 172.21.15.121:0/1500553497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6b80c5c70 0x7fc6b81536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.768+0000 7fc6b7fff700 1 -- 172.21.15.121:0/1500553497 >> v1:172.21.15.190:6789/0 conn(0x7fc6b8066330 legacy=0x7fc6b8153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:46.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.768+0000 7fc6b7fff700 1 -- 172.21.15.121:0/1500553497 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b80c64a0 con 0x7fc6b80acd00 2023-11-15T02:56:46.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.768+0000 7fc6b7fff700 1 --2- 172.21.15.121:0/1500553497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6b80acd00 0x7fc6b80ad0e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fc6a8006fd0 tx=0x7fc6a8006f40).ready entity=mon.1 client_cookie=494754b6aef93204 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:46.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.768+0000 7fc6b6ffd700 1 -- 172.21.15.121:0/1500553497 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8005880 con 0x7fc6b80acd00 2023-11-15T02:56:46.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.768+0000 7fc6b6ffd700 1 -- 172.21.15.121:0/1500553497 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a80059e0 con 0x7fc6b80acd00 2023-11-15T02:56:46.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.768+0000 7fc6b6ffd700 1 -- 172.21.15.121:0/1500553497 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8005d60 con 0x7fc6b80acd00 2023-11-15T02:56:46.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.772+0000 7fc6bd4b9700 1 -- 172.21.15.121:0/1500553497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6b80acd00 msgr2=0x7fc6b80ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:46.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.772+0000 7fc6bd4b9700 1 --2- 172.21.15.121:0/1500553497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6b80acd00 0x7fc6b80ad0e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fc6a8006fd0 tx=0x7fc6a8006f40).stop 2023-11-15T02:56:46.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.772+0000 7fc6bd4b9700 1 -- 172.21.15.121:0/1500553497 shutdown_connections 2023-11-15T02:56:46.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.772+0000 7fc6bd4b9700 1 --2- 172.21.15.121:0/1500553497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6b80c5c70 0x7fc6b81536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.772+0000 7fc6bd4b9700 1 --2- 172.21.15.121:0/1500553497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6b80acd00 0x7fc6b80ad0e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.772+0000 7fc6bd4b9700 1 -- 172.21.15.121:0/1500553497 >> 172.21.15.121:0/1500553497 conn(0x7fc6b8063fc0 msgr2=0x7fc6b8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:46.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.772+0000 7fc6bd4b9700 1 -- 172.21.15.121:0/1500553497 shutdown_connections 2023-11-15T02:56:46.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.772+0000 7fc6bd4b9700 1 -- 172.21.15.121:0/1500553497 wait complete. 2023-11-15T02:56:46.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.772+0000 7fc6bd4b9700 1 Processor -- start 2023-11-15T02:56:46.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.772+0000 7fc6bd4b9700 1 -- start start 2023-11-15T02:56:46.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc6bd4b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6b8066330 0x7fc6b81542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:46.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc6bd4b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6b80acd00 0x7fc6b8150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:46.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc6bd4b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6b80c5c70 0x7fc6b81529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:46.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc6bd4b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b8155540 con 0x7fc6b80c5c70 2023-11-15T02:56:46.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc6bd4b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b81548c0 con 0x7fc6b8066330 2023-11-15T02:56:46.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc6bd4b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b8154ba0 con 0x7fc6b80acd00 2023-11-15T02:56:46.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc6bccb8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6b80c5c70 0x7fc6b81529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:46.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc6b77fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6b80acd00 0x7fc6b8150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:46.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc6b7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6b8066330 0x7fc6b81542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:46.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc6b77fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6b80acd00 0x7fc6b8150db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:55286/0 (socket says 172.21.15.121:55286) 2023-11-15T02:56:46.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc6b77fe700 1 -- 172.21.15.121:0/2180803157 learned_addr learned my addr 172.21.15.121:0/2180803157 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:46.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc6b77fe700 1 -- 172.21.15.121:0/2180803157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6b8066330 msgr2=0x7fc6b81542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:46.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc6b77fe700 1 --2- 172.21.15.121:0/2180803157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6b8066330 0x7fc6b81542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc6b77fe700 1 -- 172.21.15.121:0/2180803157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6b80c5c70 msgr2=0x7fc6b81529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:46.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc6b77fe700 1 --2- 172.21.15.121:0/2180803157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6b80c5c70 0x7fc6b81529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc6b77fe700 1 -- 172.21.15.121:0/2180803157 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a8009250 con 0x7fc6b80acd00 2023-11-15T02:56:46.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc6b77fe700 1 --2- 172.21.15.121:0/2180803157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6b80acd00 0x7fc6b8150db0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fc6ac00e900 tx=0x7fc6ac009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:46.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc69ffff700 1 -- 172.21.15.121:0/2180803157 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac012600 con 0x7fc6b80acd00 2023-11-15T02:56:46.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc69ffff700 1 -- 172.21.15.121:0/2180803157 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac012760 con 0x7fc6b80acd00 2023-11-15T02:56:46.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc6bd4b9700 1 -- 172.21.15.121:0/2180803157 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b8154e80 con 0x7fc6b80acd00 2023-11-15T02:56:46.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc69ffff700 1 -- 172.21.15.121:0/2180803157 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac012ae0 con 0x7fc6b80acd00 2023-11-15T02:56:46.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.776+0000 7fc6bd4b9700 1 -- 172.21.15.121:0/2180803157 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b8152eb0 con 0x7fc6b80acd00 2023-11-15T02:56:46.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.780+0000 7fc6bd4b9700 1 -- 172.21.15.121:0/2180803157 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b8155710 con 0x7fc6b80acd00 2023-11-15T02:56:46.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.780+0000 7fc69ffff700 1 -- 172.21.15.121:0/2180803157 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc6ac01b400 con 0x7fc6b80acd00 2023-11-15T02:56:46.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.784+0000 7fc69ffff700 1 --2- 172.21.15.121:0/2180803157 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc6a405a510 0x7fc6a405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:46.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.784+0000 7fc69ffff700 1 -- 172.21.15.121:0/2180803157 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fc6ac0747b0 con 0x7fc6b80acd00 2023-11-15T02:56:46.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.784+0000 7fc6b7fff700 1 --2- 172.21.15.121:0/2180803157 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc6a405a510 0x7fc6a405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:46.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.784+0000 7fc6b7fff700 1 --2- 172.21.15.121:0/2180803157 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc6a405a510 0x7fc6a405c9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fc6a8006fa0 tx=0x7fc6a8016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:46.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.792+0000 7fc69ffff700 1 -- 172.21.15.121:0/2180803157 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc6b8155710 con 0x7fc6b80acd00 2023-11-15T02:56:46.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.956+0000 7fc6bd4b9700 1 -- 172.21.15.121:0/2180803157 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc6b8155710 con 0x7fc6b80acd00 2023-11-15T02:56:46.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.956+0000 7fc69ffff700 1 -- 172.21.15.121:0/2180803157 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v16) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc6b8155710 con 0x7fc6b80acd00 2023-11-15T02:56:46.962 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T02:56:46.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.964+0000 7fc6bd4b9700 1 -- 172.21.15.121:0/2180803157 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc6a405a510 msgr2=0x7fc6a405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:46.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.964+0000 7fc6bd4b9700 1 --2- 172.21.15.121:0/2180803157 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc6a405a510 0x7fc6a405c9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fc6a8006fa0 tx=0x7fc6a8016000).stop 2023-11-15T02:56:46.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.964+0000 7fc6bd4b9700 1 -- 172.21.15.121:0/2180803157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6b80acd00 msgr2=0x7fc6b8150db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:46.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.964+0000 7fc6bd4b9700 1 --2- 172.21.15.121:0/2180803157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6b80acd00 0x7fc6b8150db0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fc6ac00e900 tx=0x7fc6ac009060).stop 2023-11-15T02:56:46.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.964+0000 7fc6bd4b9700 1 -- 172.21.15.121:0/2180803157 shutdown_connections 2023-11-15T02:56:46.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.964+0000 7fc6bd4b9700 1 --2- 172.21.15.121:0/2180803157 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc6a405a510 0x7fc6a405c9d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.964+0000 7fc6bd4b9700 1 --2- 172.21.15.121:0/2180803157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6b80c5c70 0x7fc6b81529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.964+0000 7fc6bd4b9700 1 --2- 172.21.15.121:0/2180803157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6b80acd00 0x7fc6b8150db0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.964+0000 7fc6bd4b9700 1 --2- 172.21.15.121:0/2180803157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6b8066330 0x7fc6b81542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:46.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.964+0000 7fc6bd4b9700 1 -- 172.21.15.121:0/2180803157 >> 172.21.15.121:0/2180803157 conn(0x7fc6b8063fc0 msgr2=0x7fc6b814f540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:46.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.964+0000 7fc6bd4b9700 1 -- 172.21.15.121:0/2180803157 shutdown_connections 2023-11-15T02:56:46.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:46.964+0000 7fc6bd4b9700 1 -- 172.21.15.121:0/2180803157 wait complete. 2023-11-15T02:56:46.988 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T02:56:47.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.148+0000 7f711e41b700 1 Processor -- start 2023-11-15T02:56:47.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.148+0000 7f711e41b700 1 -- start start 2023-11-15T02:56:47.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.152+0000 7f711e41b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71180a6b80 0x7f71180bd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:47.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.152+0000 7f711e41b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f71180a6f60 0x7f71180bd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:47.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.152+0000 7f711e41b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71180c2730 con 0x7f71180c6130 2023-11-15T02:56:47.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.152+0000 7f711e41b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71180bdae0 con 0x7f71180a6f60 2023-11-15T02:56:47.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.152+0000 7f711e41b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7118150f00 con 0x7f71180a6b80 2023-11-15T02:56:47.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.152+0000 7f7117fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71180a6b80 0x7f71180bd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:47.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.152+0000 7f711c9b8700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f71180c6130 0x7f71181545e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:54590/0 (socket says 172.21.15.121:54590) 2023-11-15T02:56:47.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.152+0000 7f711c9b8700 1 -- 172.21.15.121:0/4056685766 learned_addr learned my addr 172.21.15.121:0/4056685766 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:47.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.152+0000 7f710ffff700 1 --2- 172.21.15.121:0/4056685766 >> v2:172.21.15.190:3300/0 conn(0x7f71180a6f60 0x7f71180bd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:47.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.156+0000 7f7117fff700 1 -- 172.21.15.121:0/4056685766 >> v2:172.21.15.190:3300/0 conn(0x7f71180a6f60 msgr2=0x7f71180bd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:47.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.156+0000 7f7117fff700 1 --2- 172.21.15.121:0/4056685766 >> v2:172.21.15.190:3300/0 conn(0x7f71180a6f60 0x7f71180bd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.156+0000 7f7117fff700 1 -- 172.21.15.121:0/4056685766 >> v1:172.21.15.190:6789/0 conn(0x7f71180c6130 legacy=0x7f71181545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:47.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.156+0000 7f71177fe700 1 -- 172.21.15.121:0/4056685766 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287687229 0 0) 0x7f71180c2730 con 0x7f71180c6130 2023-11-15T02:56:47.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.156+0000 7f7117fff700 1 -- 172.21.15.121:0/4056685766 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71180c75f0 con 0x7f71180a6b80 2023-11-15T02:56:47.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.156+0000 7f7117fff700 1 --2- 172.21.15.121:0/4056685766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71180a6b80 0x7f71180bd100 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f710800ba00 tx=0x7f710800b300).ready entity=mon.2 client_cookie=97b53ecf998abda3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:47.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.156+0000 7f71177fe700 1 -- 172.21.15.121:0/4056685766 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710800da70 con 0x7f71180a6b80 2023-11-15T02:56:47.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.156+0000 7f71177fe700 1 -- 172.21.15.121:0/4056685766 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710800dbd0 con 0x7f71180a6b80 2023-11-15T02:56:47.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.156+0000 7f71177fe700 1 -- 172.21.15.121:0/4056685766 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108010470 con 0x7f71180a6b80 2023-11-15T02:56:47.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.156+0000 7f711e41b700 1 -- 172.21.15.121:0/4056685766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71180a6b80 msgr2=0x7f71180bd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:47.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.156+0000 7f711e41b700 1 --2- 172.21.15.121:0/4056685766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71180a6b80 0x7f71180bd100 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f710800ba00 tx=0x7f710800b300).stop 2023-11-15T02:56:47.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.156+0000 7f711e41b700 1 -- 172.21.15.121:0/4056685766 shutdown_connections 2023-11-15T02:56:47.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.156+0000 7f711e41b700 1 --2- 172.21.15.121:0/4056685766 >> v2:172.21.15.190:3300/0 conn(0x7f71180a6f60 0x7f71180bd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.156+0000 7f711e41b700 1 --2- 172.21.15.121:0/4056685766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71180a6b80 0x7f71180bd100 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.156+0000 7f711e41b700 1 -- 172.21.15.121:0/4056685766 >> 172.21.15.121:0/4056685766 conn(0x7f7118066f50 msgr2=0x7f7118090f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:47.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.156+0000 7f711e41b700 1 -- 172.21.15.121:0/4056685766 shutdown_connections 2023-11-15T02:56:47.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.156+0000 7f711e41b700 1 -- 172.21.15.121:0/4056685766 wait complete. 2023-11-15T02:56:47.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.160+0000 7f711e41b700 1 Processor -- start 2023-11-15T02:56:47.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.160+0000 7f711e41b700 1 -- start start 2023-11-15T02:56:47.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f711e41b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71180a6b80 0x7f71180aca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:47.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f711e41b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71180a6f60 0x7f711814c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:47.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f711e41b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71180c6130 0x7f7118062ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:47.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f711e41b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71180acf00 con 0x7f71180a6f60 2023-11-15T02:56:47.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f711e41b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71180a9a80 con 0x7f71180c6130 2023-11-15T02:56:47.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f711e41b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71180bf070 con 0x7f71180a6b80 2023-11-15T02:56:47.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f710ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71180a6f60 0x7f711814c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:47.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f710ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71180a6f60 0x7f711814c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44750/0 (socket says 172.21.15.121:44750) 2023-11-15T02:56:47.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f710ffff700 1 -- 172.21.15.121:0/467276544 learned_addr learned my addr 172.21.15.121:0/467276544 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:47.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f7117fff700 1 --2- 172.21.15.121:0/467276544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71180a6b80 0x7f71180aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:47.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f711c9b8700 1 --2- 172.21.15.121:0/467276544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71180c6130 0x7f7118062ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:47.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f710ffff700 1 -- 172.21.15.121:0/467276544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71180a6b80 msgr2=0x7f71180aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:47.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f710ffff700 1 --2- 172.21.15.121:0/467276544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71180a6b80 0x7f71180aca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f710ffff700 1 -- 172.21.15.121:0/467276544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71180c6130 msgr2=0x7f7118062ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:47.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f710ffff700 1 --2- 172.21.15.121:0/467276544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71180c6130 0x7f7118062ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f710ffff700 1 -- 172.21.15.121:0/467276544 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710800b540 con 0x7f71180a6f60 2023-11-15T02:56:47.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f710ffff700 1 --2- 172.21.15.121:0/467276544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71180a6f60 0x7f711814c550 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f710000e900 tx=0x7f7100009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:47.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f7114ff9700 1 -- 172.21.15.121:0/467276544 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710000c670 con 0x7f71180a6f60 2023-11-15T02:56:47.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f7114ff9700 1 -- 172.21.15.121:0/467276544 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710000c7d0 con 0x7f71180a6f60 2023-11-15T02:56:47.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f711e41b700 1 -- 172.21.15.121:0/467276544 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71180a8860 con 0x7f71180a6f60 2023-11-15T02:56:47.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f7114ff9700 1 -- 172.21.15.121:0/467276544 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710000cb50 con 0x7f71180a6f60 2023-11-15T02:56:47.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.164+0000 7f711e41b700 1 -- 172.21.15.121:0/467276544 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71180a8e20 con 0x7f71180a6f60 2023-11-15T02:56:47.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.168+0000 7f711e41b700 1 -- 172.21.15.121:0/467276544 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7118047060 con 0x7f71180a6f60 2023-11-15T02:56:47.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.168+0000 7f7114ff9700 1 -- 172.21.15.121:0/467276544 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f710001b400 con 0x7f71180a6f60 2023-11-15T02:56:47.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.168+0000 7f7114ff9700 1 --2- 172.21.15.121:0/467276544 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f70f805a510 0x7f70f805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:47.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.168+0000 7f7114ff9700 1 -- 172.21.15.121:0/467276544 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f7100074930 con 0x7f71180a6f60 2023-11-15T02:56:47.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.168+0000 7f7117fff700 1 --2- 172.21.15.121:0/467276544 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f70f805a510 0x7f70f805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:47.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.172+0000 7f7117fff700 1 --2- 172.21.15.121:0/467276544 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f70f805a510 0x7f70f805c9d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f71080105a0 tx=0x7f71180c2960).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:47.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.176+0000 7f7114ff9700 1 -- 172.21.15.121:0/467276544 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7100048400 con 0x7f71180a6f60 2023-11-15T02:56:47.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.344+0000 7f711e41b700 1 -- 172.21.15.121:0/467276544 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7118069c60 con 0x7f71180a6f60 2023-11-15T02:56:47.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.344+0000 7f7114ff9700 1 -- 172.21.15.121:0/467276544 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7100016030 con 0x7f71180a6f60 2023-11-15T02:56:47.350 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T02:56:47.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.348+0000 7f711e41b700 1 -- 172.21.15.121:0/467276544 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f70f805a510 msgr2=0x7f70f805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:47.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.348+0000 7f711e41b700 1 --2- 172.21.15.121:0/467276544 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f70f805a510 0x7f70f805c9d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f71080105a0 tx=0x7f71180c2960).stop 2023-11-15T02:56:47.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.348+0000 7f711e41b700 1 -- 172.21.15.121:0/467276544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71180a6f60 msgr2=0x7f711814c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:47.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.348+0000 7f711e41b700 1 --2- 172.21.15.121:0/467276544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71180a6f60 0x7f711814c550 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f710000e900 tx=0x7f7100009060).stop 2023-11-15T02:56:47.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.348+0000 7f711e41b700 1 -- 172.21.15.121:0/467276544 shutdown_connections 2023-11-15T02:56:47.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.348+0000 7f711e41b700 1 --2- 172.21.15.121:0/467276544 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f70f805a510 0x7f70f805c9d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.348+0000 7f711e41b700 1 --2- 172.21.15.121:0/467276544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71180c6130 0x7f7118062ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.348+0000 7f711e41b700 1 --2- 172.21.15.121:0/467276544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71180a6f60 0x7f711814c550 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.348+0000 7f711e41b700 1 --2- 172.21.15.121:0/467276544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71180a6b80 0x7f71180aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.348+0000 7f711e41b700 1 -- 172.21.15.121:0/467276544 >> 172.21.15.121:0/467276544 conn(0x7f7118066f50 msgr2=0x7f71180bcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:47.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.348+0000 7f711e41b700 1 -- 172.21.15.121:0/467276544 shutdown_connections 2023-11-15T02:56:47.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.352+0000 7f711e41b700 1 -- 172.21.15.121:0/467276544 wait complete. 2023-11-15T02:56:47.377 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:56:47.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.532+0000 7fce9a7a9700 1 Processor -- start 2023-11-15T02:56:47.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.532+0000 7fce9a7a9700 1 -- start start 2023-11-15T02:56:47.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce9a7a9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fce94152780 0x7fce94159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:47.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce9a7a9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fce940c65d0 0x7fce940c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:47.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce9a7a9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce940c3fa0 con 0x7fce940a5510 2023-11-15T02:56:47.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce9a7a9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce94159630 con 0x7fce940c65d0 2023-11-15T02:56:47.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce9a7a9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce940c30b0 con 0x7fce94152780 2023-11-15T02:56:47.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce98d46700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fce940a5510 0x7fce940a59a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:42354/0 (socket says 172.21.15.121:42354) 2023-11-15T02:56:47.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce98d46700 1 -- 172.21.15.121:0/1856123338 learned_addr learned my addr 172.21.15.121:0/1856123338 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:47.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce937fe700 1 --2- 172.21.15.121:0/1856123338 >> v2:172.21.15.121:3300/0 conn(0x7fce940c65d0 0x7fce940c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:47.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce93fff700 1 --2- 172.21.15.121:0/1856123338 >> v2:172.21.15.190:3300/0 conn(0x7fce94152780 0x7fce94159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:47.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce92ffd700 1 -- 172.21.15.121:0/1856123338 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2566378807 0 0) 0x7fce940c3fa0 con 0x7fce940a5510 2023-11-15T02:56:47.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce92ffd700 1 -- 172.21.15.121:0/1856123338 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce7c0033f0 con 0x7fce940a5510 2023-11-15T02:56:47.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce937fe700 1 -- 172.21.15.121:0/1856123338 >> v2:172.21.15.190:3300/0 conn(0x7fce94152780 msgr2=0x7fce94159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:47.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce937fe700 1 --2- 172.21.15.121:0/1856123338 >> v2:172.21.15.190:3300/0 conn(0x7fce94152780 0x7fce94159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce937fe700 1 -- 172.21.15.121:0/1856123338 >> v1:172.21.15.121:6789/0 conn(0x7fce940a5510 legacy=0x7fce940a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:47.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce937fe700 1 -- 172.21.15.121:0/1856123338 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce940c3390 con 0x7fce940c65d0 2023-11-15T02:56:47.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce937fe700 1 --2- 172.21.15.121:0/1856123338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce940c65d0 0x7fce940c6a50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fce940653f0 tx=0x7fce88009060).ready entity=mon.0 client_cookie=1192d58b37b82edb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:47.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce92ffd700 1 -- 172.21.15.121:0/1856123338 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8800d4a0 con 0x7fce940c65d0 2023-11-15T02:56:47.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce92ffd700 1 -- 172.21.15.121:0/1856123338 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8800d600 con 0x7fce940c65d0 2023-11-15T02:56:47.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce92ffd700 1 -- 172.21.15.121:0/1856123338 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8800d980 con 0x7fce940c65d0 2023-11-15T02:56:47.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce9a7a9700 1 -- 172.21.15.121:0/1856123338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce940c65d0 msgr2=0x7fce940c6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:47.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce9a7a9700 1 --2- 172.21.15.121:0/1856123338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce940c65d0 0x7fce940c6a50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fce940653f0 tx=0x7fce88009060).stop 2023-11-15T02:56:47.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce9a7a9700 1 -- 172.21.15.121:0/1856123338 shutdown_connections 2023-11-15T02:56:47.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce9a7a9700 1 --2- 172.21.15.121:0/1856123338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce940c65d0 0x7fce940c6a50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce9a7a9700 1 --2- 172.21.15.121:0/1856123338 >> v2:172.21.15.190:3300/0 conn(0x7fce94152780 0x7fce94159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce9a7a9700 1 -- 172.21.15.121:0/1856123338 >> 172.21.15.121:0/1856123338 conn(0x7fce94062190 msgr2=0x7fce940625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:47.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce9a7a9700 1 -- 172.21.15.121:0/1856123338 shutdown_connections 2023-11-15T02:56:47.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.536+0000 7fce9a7a9700 1 -- 172.21.15.121:0/1856123338 wait complete. 2023-11-15T02:56:47.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.540+0000 7fce9a7a9700 1 Processor -- start 2023-11-15T02:56:47.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.540+0000 7fce9a7a9700 1 -- start start 2023-11-15T02:56:47.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.540+0000 7fce9a7a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce940a5510 0x7fce94066420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:47.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.540+0000 7fce9a7a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce940c65d0 0x7fce94154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:47.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.540+0000 7fce9a7a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce94152780 0x7fce94154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:47.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.540+0000 7fce9a7a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce94145f30 con 0x7fce940a5510 2023-11-15T02:56:47.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.540+0000 7fce9a7a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce94155370 con 0x7fce94152780 2023-11-15T02:56:47.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.540+0000 7fce9a7a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce94155650 con 0x7fce940c65d0 2023-11-15T02:56:47.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.540+0000 7fce93fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce940a5510 0x7fce94066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:47.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.540+0000 7fce93fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce940a5510 0x7fce94066420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44766/0 (socket says 172.21.15.121:44766) 2023-11-15T02:56:47.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.540+0000 7fce93fff700 1 -- 172.21.15.121:0/3058738887 learned_addr learned my addr 172.21.15.121:0/3058738887 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:47.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.544+0000 7fce937fe700 1 --2- 172.21.15.121:0/3058738887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce940c65d0 0x7fce94154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:47.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.544+0000 7fce98d46700 1 --2- 172.21.15.121:0/3058738887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce94152780 0x7fce94154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:47.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.544+0000 7fce93fff700 1 -- 172.21.15.121:0/3058738887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce940c65d0 msgr2=0x7fce94154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:47.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.544+0000 7fce93fff700 1 --2- 172.21.15.121:0/3058738887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce940c65d0 0x7fce94154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.544+0000 7fce93fff700 1 -- 172.21.15.121:0/3058738887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce94152780 msgr2=0x7fce94154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:47.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.544+0000 7fce93fff700 1 --2- 172.21.15.121:0/3058738887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce94152780 0x7fce94154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.544+0000 7fce93fff700 1 -- 172.21.15.121:0/3058738887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce880092a0 con 0x7fce940a5510 2023-11-15T02:56:47.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.544+0000 7fce937fe700 1 --2- 172.21.15.121:0/3058738887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce940c65d0 0x7fce94154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:56:47.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.544+0000 7fce98d46700 1 --2- 172.21.15.121:0/3058738887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce94152780 0x7fce94154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:56:47.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.544+0000 7fce93fff700 1 --2- 172.21.15.121:0/3058738887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce940a5510 0x7fce94066420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fce8400e900 tx=0x7fce84009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:47.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.544+0000 7fce7bfff700 1 -- 172.21.15.121:0/3058738887 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8400c640 con 0x7fce940a5510 2023-11-15T02:56:47.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.544+0000 7fce7bfff700 1 -- 172.21.15.121:0/3058738887 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8400c7a0 con 0x7fce940a5510 2023-11-15T02:56:47.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.544+0000 7fce7bfff700 1 -- 172.21.15.121:0/3058738887 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8400cb20 con 0x7fce940a5510 2023-11-15T02:56:47.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.544+0000 7fce9a7a9700 1 -- 172.21.15.121:0/3058738887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce94155930 con 0x7fce940a5510 2023-11-15T02:56:47.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.544+0000 7fce9a7a9700 1 -- 172.21.15.121:0/3058738887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce94155ef0 con 0x7fce940a5510 2023-11-15T02:56:47.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.544+0000 7fce9a7a9700 1 -- 172.21.15.121:0/3058738887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce9406c150 con 0x7fce940a5510 2023-11-15T02:56:47.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.544+0000 7fce7bfff700 1 -- 172.21.15.121:0/3058738887 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fce8401b400 con 0x7fce940a5510 2023-11-15T02:56:47.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.548+0000 7fce7bfff700 1 --2- 172.21.15.121:0/3058738887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fce8005a510 0x7fce8005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:47.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.548+0000 7fce7bfff700 1 -- 172.21.15.121:0/3058738887 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fce84074910 con 0x7fce940a5510 2023-11-15T02:56:47.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.548+0000 7fce937fe700 1 --2- 172.21.15.121:0/3058738887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fce8005a510 0x7fce8005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:47.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.548+0000 7fce937fe700 1 --2- 172.21.15.121:0/3058738887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fce8005a510 0x7fce8005c9d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fce94156530 tx=0x7fce88016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:47.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.552+0000 7fce7bfff700 1 -- 172.21.15.121:0/3058738887 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fce84048320 con 0x7fce940a5510 2023-11-15T02:56:47.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.756+0000 7fce9a7a9700 1 -- 172.21.15.121:0/3058738887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce940c3670 con 0x7fce940a5510 2023-11-15T02:56:47.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.756+0000 7fce7bfff700 1 -- 172.21.15.121:0/3058738887 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7fce84016030 con 0x7fce940a5510 2023-11-15T02:56:47.763 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:56:47.763 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/802010010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":802010010},{"type":"v1","addr":"172.21.15.190:6835","nonce":802010010}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3991611070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3991611070},{"type":"v1","addr":"172.21.15.121:6836","nonce":3991611070}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3514688785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3514688785},{"type":"v1","addr":"172.21.15.121:6837","nonce":3514688785}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/742721767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":742721767},{"type":"v1","addr":"172.21.15.190:6833","nonce":742721767}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-15T02:56:47.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.760+0000 7fce9a7a9700 1 -- 172.21.15.121:0/3058738887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fce8005a510 msgr2=0x7fce8005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:47.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.760+0000 7fce9a7a9700 1 --2- 172.21.15.121:0/3058738887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fce8005a510 0x7fce8005c9d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fce94156530 tx=0x7fce88016000).stop 2023-11-15T02:56:47.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.760+0000 7fce9a7a9700 1 -- 172.21.15.121:0/3058738887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce940a5510 msgr2=0x7fce94066420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:47.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.760+0000 7fce9a7a9700 1 --2- 172.21.15.121:0/3058738887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce940a5510 0x7fce94066420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fce8400e900 tx=0x7fce84009060).stop 2023-11-15T02:56:47.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.764+0000 7fce9a7a9700 1 -- 172.21.15.121:0/3058738887 shutdown_connections 2023-11-15T02:56:47.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.764+0000 7fce9a7a9700 1 --2- 172.21.15.121:0/3058738887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fce8005a510 0x7fce8005c9d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.764+0000 7fce9a7a9700 1 --2- 172.21.15.121:0/3058738887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce94152780 0x7fce94154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.764+0000 7fce9a7a9700 1 --2- 172.21.15.121:0/3058738887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce940c65d0 0x7fce94154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.764+0000 7fce9a7a9700 1 --2- 172.21.15.121:0/3058738887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce940a5510 0x7fce94066420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.764+0000 7fce9a7a9700 1 -- 172.21.15.121:0/3058738887 >> 172.21.15.121:0/3058738887 conn(0x7fce94062190 msgr2=0x7fce94062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:47.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.764+0000 7fce9a7a9700 1 -- 172.21.15.121:0/3058738887 shutdown_connections 2023-11-15T02:56:47.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.764+0000 7fce9a7a9700 1 -- 172.21.15.121:0/3058738887 wait complete. 2023-11-15T02:56:47.770 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 3 2023-11-15T02:56:47.788 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:56:47.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.936+0000 7fce53cff700 1 Processor -- start 2023-11-15T02:56:47.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.936+0000 7fce53cff700 1 -- start start 2023-11-15T02:56:47.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.940+0000 7fce53cff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce4c0a5ed0 0x7fce4c155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:47.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.940+0000 7fce53cff700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4c0c4c30 con 0x7fce4c154ca0 2023-11-15T02:56:47.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.940+0000 7fce53cff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4c0a6f50 con 0x7fce4c1562f0 2023-11-15T02:56:47.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.940+0000 7fce53cff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c0a8280 con 0x7fce4c0a5ed0 2023-11-15T02:56:47.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.940+0000 7fce51a9b700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fce4c154ca0 0x7fce4c0a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:42362/0 (socket says 172.21.15.121:42362) 2023-11-15T02:56:47.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.940+0000 7fce51a9b700 1 -- 172.21.15.121:0/1904853457 learned_addr learned my addr 172.21.15.121:0/1904853457 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:47.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.940+0000 7fce5229c700 1 --2- 172.21.15.121:0/1904853457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce4c0a5ed0 0x7fce4c155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:47.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.940+0000 7fce50a99700 1 -- 172.21.15.121:0/1904853457 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684347633 0 0) 0x7fce4c0c4c30 con 0x7fce4c154ca0 2023-11-15T02:56:47.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.944+0000 7fce50a99700 1 -- 172.21.15.121:0/1904853457 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce380033f0 con 0x7fce4c154ca0 2023-11-15T02:56:47.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.944+0000 7fce50a99700 1 -- 172.21.15.121:0/1904853457 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2558763476 0 0) 0x7fce4c0a6f50 con 0x7fce4c1562f0 2023-11-15T02:56:47.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.944+0000 7fce50a99700 1 -- 172.21.15.121:0/1904853457 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce4c0c4c30 con 0x7fce4c1562f0 2023-11-15T02:56:47.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.944+0000 7fce5229c700 1 -- 172.21.15.121:0/1904853457 >> v1:172.21.15.190:6789/0 conn(0x7fce4c1562f0 legacy=0x7fce4c0a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:47.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.944+0000 7fce5229c700 1 -- 172.21.15.121:0/1904853457 >> v1:172.21.15.121:6789/0 conn(0x7fce4c154ca0 legacy=0x7fce4c0a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:47.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.944+0000 7fce5229c700 1 -- 172.21.15.121:0/1904853457 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c0a7230 con 0x7fce4c0a5ed0 2023-11-15T02:56:47.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.944+0000 7fce5229c700 1 --2- 172.21.15.121:0/1904853457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce4c0a5ed0 0x7fce4c155080 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fce44009e90 tx=0x7fce44009780).ready entity=mon.2 client_cookie=b429fa4d8f3ac8e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:47.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.944+0000 7fce50a99700 1 -- 172.21.15.121:0/1904853457 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4400e070 con 0x7fce4c0a5ed0 2023-11-15T02:56:47.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.944+0000 7fce50a99700 1 -- 172.21.15.121:0/1904853457 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4400bab0 con 0x7fce4c0a5ed0 2023-11-15T02:56:47.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.944+0000 7fce50a99700 1 -- 172.21.15.121:0/1904853457 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4400be30 con 0x7fce4c0a5ed0 2023-11-15T02:56:47.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.944+0000 7fce53cff700 1 -- 172.21.15.121:0/1904853457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce4c0a5ed0 msgr2=0x7fce4c155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:47.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.944+0000 7fce53cff700 1 --2- 172.21.15.121:0/1904853457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce4c0a5ed0 0x7fce4c155080 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fce44009e90 tx=0x7fce44009780).stop 2023-11-15T02:56:47.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.944+0000 7fce53cff700 1 -- 172.21.15.121:0/1904853457 shutdown_connections 2023-11-15T02:56:47.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.944+0000 7fce53cff700 1 --2- 172.21.15.121:0/1904853457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce4c0a5ed0 0x7fce4c155080 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.944+0000 7fce53cff700 1 -- 172.21.15.121:0/1904853457 >> 172.21.15.121:0/1904853457 conn(0x7fce4c062190 msgr2=0x7fce4c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:47.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.944+0000 7fce53cff700 1 -- 172.21.15.121:0/1904853457 shutdown_connections 2023-11-15T02:56:47.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.944+0000 7fce53cff700 1 -- 172.21.15.121:0/1904853457 wait complete. 2023-11-15T02:56:47.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.948+0000 7fce53cff700 1 Processor -- start 2023-11-15T02:56:47.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.948+0000 7fce53cff700 1 -- start start 2023-11-15T02:56:47.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.948+0000 7fce53cff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce4c0a5ed0 0x7fce4c14d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:47.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.948+0000 7fce53cff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce4c154ca0 0x7fce4c0ab840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:47.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.948+0000 7fce53cff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce4c1562f0 0x7fce4c0abd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:47.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.948+0000 7fce53cff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c0ac220 con 0x7fce4c1562f0 2023-11-15T02:56:47.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.948+0000 7fce53cff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c164390 con 0x7fce4c0a5ed0 2023-11-15T02:56:47.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.948+0000 7fce53cff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c14ddd0 con 0x7fce4c154ca0 2023-11-15T02:56:47.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.952+0000 7fce5229c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce4c1562f0 0x7fce4c0abd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:47.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.952+0000 7fce5229c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce4c1562f0 0x7fce4c0abd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44780/0 (socket says 172.21.15.121:44780) 2023-11-15T02:56:47.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.952+0000 7fce5229c700 1 -- 172.21.15.121:0/1668010775 learned_addr learned my addr 172.21.15.121:0/1668010775 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:47.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.952+0000 7fce51a9b700 1 --2- 172.21.15.121:0/1668010775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce4c0a5ed0 0x7fce4c14d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:47.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.952+0000 7fce5129a700 1 --2- 172.21.15.121:0/1668010775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce4c154ca0 0x7fce4c0ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:47.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.952+0000 7fce5229c700 1 -- 172.21.15.121:0/1668010775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce4c154ca0 msgr2=0x7fce4c0ab840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:47.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.952+0000 7fce5229c700 1 --2- 172.21.15.121:0/1668010775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce4c154ca0 0x7fce4c0ab840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.952+0000 7fce5229c700 1 -- 172.21.15.121:0/1668010775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce4c0a5ed0 msgr2=0x7fce4c14d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:47.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.952+0000 7fce5229c700 1 --2- 172.21.15.121:0/1668010775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce4c0a5ed0 0x7fce4c14d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:47.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.952+0000 7fce5229c700 1 -- 172.21.15.121:0/1668010775 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce44009a20 con 0x7fce4c1562f0 2023-11-15T02:56:47.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.952+0000 7fce5229c700 1 --2- 172.21.15.121:0/1668010775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce4c1562f0 0x7fce4c0abd30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fce4c063d40 tx=0x7fce4c0c4ba0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:47.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.952+0000 7fce41ffb700 1 -- 172.21.15.121:0/1668010775 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce44013e40 con 0x7fce4c1562f0 2023-11-15T02:56:47.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.952+0000 7fce41ffb700 1 -- 172.21.15.121:0/1668010775 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4401d630 con 0x7fce4c1562f0 2023-11-15T02:56:47.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.952+0000 7fce41ffb700 1 -- 172.21.15.121:0/1668010775 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4401d9f0 con 0x7fce4c1562f0 2023-11-15T02:56:47.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.952+0000 7fce53cff700 1 -- 172.21.15.121:0/1668010775 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4c0c2320 con 0x7fce4c1562f0 2023-11-15T02:56:47.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.952+0000 7fce53cff700 1 -- 172.21.15.121:0/1668010775 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4c0c28e0 con 0x7fce4c1562f0 2023-11-15T02:56:47.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.956+0000 7fce53cff700 1 -- 172.21.15.121:0/1668010775 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce4c164520 con 0x7fce4c1562f0 2023-11-15T02:56:47.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.956+0000 7fce41ffb700 1 -- 172.21.15.121:0/1668010775 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fce44003040 con 0x7fce4c1562f0 2023-11-15T02:56:47.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.956+0000 7fce41ffb700 1 --2- 172.21.15.121:0/1668010775 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fce3005a510 0x7fce3005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:47.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.956+0000 7fce41ffb700 1 -- 172.21.15.121:0/1668010775 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fce44076200 con 0x7fce4c1562f0 2023-11-15T02:56:47.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.956+0000 7fce51a9b700 1 --2- 172.21.15.121:0/1668010775 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fce3005a510 0x7fce3005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:47.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.956+0000 7fce51a9b700 1 --2- 172.21.15.121:0/1668010775 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fce3005a510 0x7fce3005c9d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fce48007fd0 tx=0x7fce48009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:47.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:47.964+0000 7fce41ffb700 1 -- 172.21.15.121:0/1668010775 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fce44049d30 con 0x7fce4c1562f0 2023-11-15T02:56:48.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.132+0000 7fce53cff700 1 -- 172.21.15.121:0/1668010775 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce4c0c4500 con 0x7fce4c1562f0 2023-11-15T02:56:48.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.136+0000 7fce41ffb700 1 -- 172.21.15.121:0/1668010775 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+13757 (secure 0 0 0) 0x7fce4404d350 con 0x7fce4c1562f0 2023-11-15T02:56:48.140 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:56:48.141 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":17,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:47.188018+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:56:48.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.136+0000 7fce53cff700 1 -- 172.21.15.121:0/1668010775 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fce3005a510 msgr2=0x7fce3005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:48.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.140+0000 7fce53cff700 1 --2- 172.21.15.121:0/1668010775 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fce3005a510 0x7fce3005c9d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fce48007fd0 tx=0x7fce48009040).stop 2023-11-15T02:56:48.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.140+0000 7fce53cff700 1 -- 172.21.15.121:0/1668010775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce4c1562f0 msgr2=0x7fce4c0abd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:48.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.140+0000 7fce53cff700 1 --2- 172.21.15.121:0/1668010775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce4c1562f0 0x7fce4c0abd30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fce4c063d40 tx=0x7fce4c0c4ba0).stop 2023-11-15T02:56:48.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.140+0000 7fce53cff700 1 -- 172.21.15.121:0/1668010775 shutdown_connections 2023-11-15T02:56:48.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.140+0000 7fce53cff700 1 --2- 172.21.15.121:0/1668010775 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fce3005a510 0x7fce3005c9d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.140+0000 7fce53cff700 1 --2- 172.21.15.121:0/1668010775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce4c1562f0 0x7fce4c0abd30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.140+0000 7fce53cff700 1 --2- 172.21.15.121:0/1668010775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce4c154ca0 0x7fce4c0ab840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.140+0000 7fce53cff700 1 --2- 172.21.15.121:0/1668010775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce4c0a5ed0 0x7fce4c14d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.140+0000 7fce53cff700 1 -- 172.21.15.121:0/1668010775 >> 172.21.15.121:0/1668010775 conn(0x7fce4c062190 msgr2=0x7fce4c0c50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:48.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.140+0000 7fce53cff700 1 -- 172.21.15.121:0/1668010775 shutdown_connections 2023-11-15T02:56:48.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.140+0000 7fce53cff700 1 -- 172.21.15.121:0/1668010775 wait complete. 2023-11-15T02:56:48.165 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-15T02:56:48.166 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T02:56:48.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.332+0000 7fe6941a9700 1 Processor -- start 2023-11-15T02:56:48.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.332+0000 7fe6941a9700 1 -- start start 2023-11-15T02:56:48.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.332+0000 7fe6941a9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe68c0682a0 0x7fe68c0c10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:48.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe6941a9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe68c153700 0x7fe68c153b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:48.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe6941a9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68c0c15a0 con 0x7fe68c067ec0 2023-11-15T02:56:48.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe6941a9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe68c0686e0 con 0x7fe68c0682a0 2023-11-15T02:56:48.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe6941a9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe68c0c0c20 con 0x7fe68c153700 2023-11-15T02:56:48.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe691f45700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe68c067ec0 0x7fe68c062c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:42364/0 (socket says 172.21.15.121:42364) 2023-11-15T02:56:48.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe691f45700 1 -- 172.21.15.121:0/2142914965 learned_addr learned my addr 172.21.15.121:0/2142914965 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:48.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe691744700 1 --2- 172.21.15.121:0/2142914965 >> v2:172.21.15.121:3300/0 conn(0x7fe68c0682a0 0x7fe68c0c10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:48.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe692746700 1 --2- 172.21.15.121:0/2142914965 >> v2:172.21.15.190:3300/0 conn(0x7fe68c153700 0x7fe68c153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:48.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe690f43700 1 -- 172.21.15.121:0/2142914965 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204796828 0 0) 0x7fe688001110 con 0x7fe68c067ec0 2023-11-15T02:56:48.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe690f43700 1 -- 172.21.15.121:0/2142914965 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6780033f0 con 0x7fe68c067ec0 2023-11-15T02:56:48.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe691744700 1 -- 172.21.15.121:0/2142914965 >> v2:172.21.15.190:3300/0 conn(0x7fe68c153700 msgr2=0x7fe68c153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:48.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe691744700 1 --2- 172.21.15.121:0/2142914965 >> v2:172.21.15.190:3300/0 conn(0x7fe68c153700 0x7fe68c153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe691744700 1 -- 172.21.15.121:0/2142914965 >> v1:172.21.15.121:6789/0 conn(0x7fe68c067ec0 legacy=0x7fe68c062c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:48.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe691744700 1 -- 172.21.15.121:0/2142914965 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c158730 con 0x7fe68c0682a0 2023-11-15T02:56:48.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe691744700 1 --2- 172.21.15.121:0/2142914965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe68c0682a0 0x7fe68c0c10b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fe68c05fa90 tx=0x7fe67c009060).ready entity=mon.0 client_cookie=4b2974bbf0ba4cb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:48.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe690f43700 1 -- 172.21.15.121:0/2142914965 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c00d4a0 con 0x7fe68c0682a0 2023-11-15T02:56:48.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe690f43700 1 -- 172.21.15.121:0/2142914965 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c00d600 con 0x7fe68c0682a0 2023-11-15T02:56:48.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe690f43700 1 -- 172.21.15.121:0/2142914965 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c00d980 con 0x7fe68c0682a0 2023-11-15T02:56:48.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe6941a9700 1 -- 172.21.15.121:0/2142914965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe68c0682a0 msgr2=0x7fe68c0c10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:48.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe6941a9700 1 --2- 172.21.15.121:0/2142914965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe68c0682a0 0x7fe68c0c10b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fe68c05fa90 tx=0x7fe67c009060).stop 2023-11-15T02:56:48.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe6941a9700 1 -- 172.21.15.121:0/2142914965 shutdown_connections 2023-11-15T02:56:48.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe6941a9700 1 --2- 172.21.15.121:0/2142914965 >> v2:172.21.15.190:3300/0 conn(0x7fe68c153700 0x7fe68c153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe6941a9700 1 --2- 172.21.15.121:0/2142914965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe68c0682a0 0x7fe68c0c10b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe6941a9700 1 -- 172.21.15.121:0/2142914965 >> 172.21.15.121:0/2142914965 conn(0x7fe68c0cad40 msgr2=0x7fe68c0cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:48.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe6941a9700 1 -- 172.21.15.121:0/2142914965 shutdown_connections 2023-11-15T02:56:48.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.336+0000 7fe6941a9700 1 -- 172.21.15.121:0/2142914965 wait complete. 2023-11-15T02:56:48.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe6941a9700 1 Processor -- start 2023-11-15T02:56:48.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe6941a9700 1 -- start start 2023-11-15T02:56:48.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe6941a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe68c067ec0 0x7fe68c14fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:48.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe6941a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe68c0682a0 0x7fe68c0c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:48.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe6941a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe68c153700 0x7fe68c0c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:48.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe6941a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c1503e0 con 0x7fe68c067ec0 2023-11-15T02:56:48.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe6941a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c0c3950 con 0x7fe68c153700 2023-11-15T02:56:48.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe6941a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c0c2d40 con 0x7fe68c0682a0 2023-11-15T02:56:48.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe691f45700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe68c067ec0 0x7fe68c14fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:48.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe691f45700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe68c067ec0 0x7fe68c14fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44802/0 (socket says 172.21.15.121:44802) 2023-11-15T02:56:48.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe691f45700 1 -- 172.21.15.121:0/20036306 learned_addr learned my addr 172.21.15.121:0/20036306 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:48.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe691744700 1 --2- 172.21.15.121:0/20036306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe68c0682a0 0x7fe68c0c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:48.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe692746700 1 --2- 172.21.15.121:0/20036306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe68c153700 0x7fe68c0c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:48.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe691f45700 1 -- 172.21.15.121:0/20036306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe68c0682a0 msgr2=0x7fe68c0c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:48.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe691f45700 1 --2- 172.21.15.121:0/20036306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe68c0682a0 0x7fe68c0c3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe691f45700 1 -- 172.21.15.121:0/20036306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe68c153700 msgr2=0x7fe68c0c42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:48.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe691f45700 1 --2- 172.21.15.121:0/20036306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe68c153700 0x7fe68c0c42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe691f45700 1 -- 172.21.15.121:0/20036306 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c0092a0 con 0x7fe68c067ec0 2023-11-15T02:56:48.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe691744700 1 --2- 172.21.15.121:0/20036306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe68c0682a0 0x7fe68c0c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:56:48.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.340+0000 7fe692746700 1 --2- 172.21.15.121:0/20036306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe68c153700 0x7fe68c0c42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:56:48.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.344+0000 7fe691f45700 1 --2- 172.21.15.121:0/20036306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe68c067ec0 0x7fe68c14fef0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fe68800c4d0 tx=0x7fe688007af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:48.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.344+0000 7fe681ffb700 1 -- 172.21.15.121:0/20036306 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6880072e0 con 0x7fe68c067ec0 2023-11-15T02:56:48.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.344+0000 7fe681ffb700 1 -- 172.21.15.121:0/20036306 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe688007440 con 0x7fe68c067ec0 2023-11-15T02:56:48.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.344+0000 7fe681ffb700 1 -- 172.21.15.121:0/20036306 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6880057a0 con 0x7fe68c067ec0 2023-11-15T02:56:48.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.344+0000 7fe6941a9700 1 -- 172.21.15.121:0/20036306 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68c0c3020 con 0x7fe68c067ec0 2023-11-15T02:56:48.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.344+0000 7fe6941a9700 1 -- 172.21.15.121:0/20036306 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68c0c35e0 con 0x7fe68c067ec0 2023-11-15T02:56:48.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.344+0000 7fe6941a9700 1 -- 172.21.15.121:0/20036306 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe68c047060 con 0x7fe68c067ec0 2023-11-15T02:56:48.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.344+0000 7fe681ffb700 1 -- 172.21.15.121:0/20036306 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe688005bd0 con 0x7fe68c067ec0 2023-11-15T02:56:48.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.348+0000 7fe681ffb700 1 --2- 172.21.15.121:0/20036306 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe67005a540 0x7fe67005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:48.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.348+0000 7fe681ffb700 1 -- 172.21.15.121:0/20036306 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fe688073ae0 con 0x7fe68c067ec0 2023-11-15T02:56:48.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.348+0000 7fe691744700 1 --2- 172.21.15.121:0/20036306 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe67005a540 0x7fe67005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:48.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.348+0000 7fe691744700 1 --2- 172.21.15.121:0/20036306 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe67005a540 0x7fe67005ca00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe67c009fd0 tx=0x7fe67c016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:48.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.352+0000 7fe681ffb700 1 -- 172.21.15.121:0/20036306 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe688047610 con 0x7fe68c067ec0 2023-11-15T02:56:48.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.512+0000 7fe6941a9700 1 -- 172.21.15.121:0/20036306 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe68c0c15a0 con 0x7fe68c067ec0 2023-11-15T02:56:48.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.516+0000 7fe681ffb700 1 -- 172.21.15.121:0/20036306 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe68c0c15a0 con 0x7fe68c067ec0 2023-11-15T02:56:48.519 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T02:56:48.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.516+0000 7fe6941a9700 1 -- 172.21.15.121:0/20036306 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe67005a540 msgr2=0x7fe67005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:48.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.516+0000 7fe6941a9700 1 --2- 172.21.15.121:0/20036306 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe67005a540 0x7fe67005ca00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe67c009fd0 tx=0x7fe67c016000).stop 2023-11-15T02:56:48.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.516+0000 7fe6941a9700 1 -- 172.21.15.121:0/20036306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe68c067ec0 msgr2=0x7fe68c14fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:48.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.516+0000 7fe6941a9700 1 --2- 172.21.15.121:0/20036306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe68c067ec0 0x7fe68c14fef0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fe68800c4d0 tx=0x7fe688007af0).stop 2023-11-15T02:56:48.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.516+0000 7fe6941a9700 1 -- 172.21.15.121:0/20036306 shutdown_connections 2023-11-15T02:56:48.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.516+0000 7fe6941a9700 1 --2- 172.21.15.121:0/20036306 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe67005a540 0x7fe67005ca00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.516+0000 7fe6941a9700 1 --2- 172.21.15.121:0/20036306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe68c153700 0x7fe68c0c42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.516+0000 7fe6941a9700 1 --2- 172.21.15.121:0/20036306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe68c0682a0 0x7fe68c0c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.516+0000 7fe6941a9700 1 --2- 172.21.15.121:0/20036306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe68c067ec0 0x7fe68c14fef0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.516+0000 7fe6941a9700 1 -- 172.21.15.121:0/20036306 >> 172.21.15.121:0/20036306 conn(0x7fe68c0cad40 msgr2=0x7fe68c065850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:48.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.516+0000 7fe6941a9700 1 -- 172.21.15.121:0/20036306 shutdown_connections 2023-11-15T02:56:48.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.516+0000 7fe6941a9700 1 -- 172.21.15.121:0/20036306 wait complete. 2023-11-15T02:56:48.543 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T02:56:48.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.716+0000 7feeb6839700 1 Processor -- start 2023-11-15T02:56:48.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.716+0000 7feeb6839700 1 -- start start 2023-11-15T02:56:48.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.716+0000 7feeb6839700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeb014ac10 0x7feeb00c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:48.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.720+0000 7feeb6839700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7feeb0067380 0x7feeb00a78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:48.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.720+0000 7feeb6839700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb00657f0 con 0x7feeb014aff0 2023-11-15T02:56:48.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.720+0000 7feeb6839700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb0066650 con 0x7feeb0067380 2023-11-15T02:56:48.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.720+0000 7feeb6839700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb0065a40 con 0x7feeb014ac10 2023-11-15T02:56:48.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.720+0000 7feeb6038700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7feeb0067380 0x7feeb00a78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:48.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.720+0000 7feeb5837700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeb014ac10 0x7feeb00c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:48.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.720+0000 7feeb6038700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7feeb0067380 0x7feeb00a78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44816/0 (socket says 172.21.15.121:44816) 2023-11-15T02:56:48.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.720+0000 7feeb6038700 1 -- 172.21.15.121:0/859947414 learned_addr learned my addr 172.21.15.121:0/859947414 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:48.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.720+0000 7feeb4835700 1 -- 172.21.15.121:0/859947414 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4050132335 0 0) 0x7feeb00657f0 con 0x7feeb014aff0 2023-11-15T02:56:48.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.720+0000 7feeb6038700 1 -- 172.21.15.121:0/859947414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeb014ac10 msgr2=0x7feeb00c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:48.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.720+0000 7feeb6038700 1 --2- 172.21.15.121:0/859947414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeb014ac10 0x7feeb00c6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.720+0000 7feeb6038700 1 -- 172.21.15.121:0/859947414 >> v1:172.21.15.190:6789/0 conn(0x7feeb014aff0 legacy=0x7feeb0066c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:48.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.720+0000 7feeb6038700 1 -- 172.21.15.121:0/859947414 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb0065d20 con 0x7feeb0067380 2023-11-15T02:56:48.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.720+0000 7feeb6038700 1 --2- 172.21.15.121:0/859947414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb0067380 0x7feeb00a78a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7feea8009f20 tx=0x7feea8009840).ready entity=mon.0 client_cookie=5972dc64a1fcc68c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:48.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.720+0000 7feeb4835700 1 -- 172.21.15.121:0/859947414 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea800b9d0 con 0x7feeb0067380 2023-11-15T02:56:48.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.720+0000 7feeb4835700 1 -- 172.21.15.121:0/859947414 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea800bb30 con 0x7feeb0067380 2023-11-15T02:56:48.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.720+0000 7feeb4835700 1 -- 172.21.15.121:0/859947414 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8011480 con 0x7feeb0067380 2023-11-15T02:56:48.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.724+0000 7feeb6839700 1 -- 172.21.15.121:0/859947414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb0067380 msgr2=0x7feeb00a78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:48.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.724+0000 7feeb6839700 1 --2- 172.21.15.121:0/859947414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb0067380 0x7feeb00a78a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7feea8009f20 tx=0x7feea8009840).stop 2023-11-15T02:56:48.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.724+0000 7feeb6839700 1 -- 172.21.15.121:0/859947414 shutdown_connections 2023-11-15T02:56:48.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.724+0000 7feeb6839700 1 --2- 172.21.15.121:0/859947414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb0067380 0x7feeb00a78a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.724+0000 7feeb6839700 1 --2- 172.21.15.121:0/859947414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeb014ac10 0x7feeb00c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.724+0000 7feeb6839700 1 -- 172.21.15.121:0/859947414 >> 172.21.15.121:0/859947414 conn(0x7feeb0067a60 msgr2=0x7feeb0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:48.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.724+0000 7feeb6839700 1 -- 172.21.15.121:0/859947414 shutdown_connections 2023-11-15T02:56:48.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.724+0000 7feeb6839700 1 -- 172.21.15.121:0/859947414 wait complete. 2023-11-15T02:56:48.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.728+0000 7feeb6839700 1 Processor -- start 2023-11-15T02:56:48.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.728+0000 7feeb6839700 1 -- start start 2023-11-15T02:56:48.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.728+0000 7feeb6839700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeb0067380 0x7feeb0149b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:48.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.728+0000 7feeb6839700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb014ac10 0x7feeb01675f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:48.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.728+0000 7feeb6839700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeb014aff0 0x7feeb00ab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:48.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.728+0000 7feeb6839700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb0167ae0 con 0x7feeb014ac10 2023-11-15T02:56:48.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.728+0000 7feeb6839700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb0167d60 con 0x7feeb0067380 2023-11-15T02:56:48.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.728+0000 7feeb6839700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb0168040 con 0x7feeb014aff0 2023-11-15T02:56:48.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.728+0000 7feeb5036700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb014ac10 0x7feeb01675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:48.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.728+0000 7feeb5036700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb014ac10 0x7feeb01675f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44820/0 (socket says 172.21.15.121:44820) 2023-11-15T02:56:48.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.728+0000 7feeb5036700 1 -- 172.21.15.121:0/2775888247 learned_addr learned my addr 172.21.15.121:0/2775888247 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:48.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.728+0000 7feeb6038700 1 --2- 172.21.15.121:0/2775888247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeb014aff0 0x7feeb00ab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:48.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.728+0000 7feeb5837700 1 --2- 172.21.15.121:0/2775888247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeb0067380 0x7feeb0149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:48.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.728+0000 7feeb5036700 1 -- 172.21.15.121:0/2775888247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeb014aff0 msgr2=0x7feeb00ab6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:48.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.728+0000 7feeb5036700 1 --2- 172.21.15.121:0/2775888247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeb014aff0 0x7feeb00ab6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.728+0000 7feeb5036700 1 -- 172.21.15.121:0/2775888247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeb0067380 msgr2=0x7feeb0149b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:48.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.728+0000 7feeb5036700 1 --2- 172.21.15.121:0/2775888247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeb0067380 0x7feeb0149b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.728+0000 7feeb5036700 1 -- 172.21.15.121:0/2775888247 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea8009ae0 con 0x7feeb014ac10 2023-11-15T02:56:48.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.728+0000 7feeb5837700 1 --2- 172.21.15.121:0/2775888247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeb0067380 0x7feeb0149b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:56:48.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.732+0000 7feeb5036700 1 --2- 172.21.15.121:0/2775888247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb014ac10 0x7feeb01675f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7feea000c490 tx=0x7feea0007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:48.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.732+0000 7feea5ffb700 1 -- 172.21.15.121:0/2775888247 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea0001ea0 con 0x7feeb014ac10 2023-11-15T02:56:48.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.732+0000 7feea5ffb700 1 -- 172.21.15.121:0/2775888247 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea000ce90 con 0x7feeb014ac10 2023-11-15T02:56:48.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.732+0000 7feea5ffb700 1 -- 172.21.15.121:0/2775888247 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea0007250 con 0x7feeb014ac10 2023-11-15T02:56:48.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.732+0000 7feeb6839700 1 -- 172.21.15.121:0/2775888247 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb0168320 con 0x7feeb014ac10 2023-11-15T02:56:48.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.732+0000 7feeb6839700 1 -- 172.21.15.121:0/2775888247 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb0163ff0 con 0x7feeb014ac10 2023-11-15T02:56:48.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.732+0000 7feeb6839700 1 -- 172.21.15.121:0/2775888247 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb006bcf0 con 0x7feeb014ac10 2023-11-15T02:56:48.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.736+0000 7feea5ffb700 1 -- 172.21.15.121:0/2775888247 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7feea00066a0 con 0x7feeb014ac10 2023-11-15T02:56:48.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.736+0000 7feea5ffb700 1 --2- 172.21.15.121:0/2775888247 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fee9405a510 0x7fee9405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:48.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.736+0000 7feea5ffb700 1 -- 172.21.15.121:0/2775888247 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7feea0073d40 con 0x7feeb014ac10 2023-11-15T02:56:48.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.736+0000 7feeb5837700 1 --2- 172.21.15.121:0/2775888247 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fee9405a510 0x7fee9405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:48.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.736+0000 7feeb5837700 1 --2- 172.21.15.121:0/2775888247 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fee9405a510 0x7fee9405c9d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7feeac009c00 tx=0x7feeac008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:48.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.744+0000 7feea5ffb700 1 -- 172.21.15.121:0/2775888247 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7feea0047870 con 0x7feeb014ac10 2023-11-15T02:56:48.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.904+0000 7feeb6839700 1 -- 172.21.15.121:0/2775888247 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7feeb0066000 con 0x7feeb014ac10 2023-11-15T02:56:48.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.904+0000 7feea5ffb700 1 -- 172.21.15.121:0/2775888247 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7feea004ae90 con 0x7feeb014ac10 2023-11-15T02:56:48.911 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T02:56:48.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.908+0000 7feeb6839700 1 -- 172.21.15.121:0/2775888247 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fee9405a510 msgr2=0x7fee9405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:48.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.908+0000 7feeb6839700 1 --2- 172.21.15.121:0/2775888247 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fee9405a510 0x7fee9405c9d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7feeac009c00 tx=0x7feeac008040).stop 2023-11-15T02:56:48.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.908+0000 7feeb6839700 1 -- 172.21.15.121:0/2775888247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb014ac10 msgr2=0x7feeb01675f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:48.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.908+0000 7feeb6839700 1 --2- 172.21.15.121:0/2775888247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb014ac10 0x7feeb01675f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7feea000c490 tx=0x7feea0007dd0).stop 2023-11-15T02:56:48.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.908+0000 7feeb6839700 1 -- 172.21.15.121:0/2775888247 shutdown_connections 2023-11-15T02:56:48.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.908+0000 7feeb6839700 1 --2- 172.21.15.121:0/2775888247 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fee9405a510 0x7fee9405c9d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.908+0000 7feeb6839700 1 --2- 172.21.15.121:0/2775888247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeb014aff0 0x7feeb00ab6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.908+0000 7feeb6839700 1 --2- 172.21.15.121:0/2775888247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb014ac10 0x7feeb01675f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.908+0000 7feeb6839700 1 --2- 172.21.15.121:0/2775888247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeb0067380 0x7feeb0149b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:48.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.908+0000 7feeb6839700 1 -- 172.21.15.121:0/2775888247 >> 172.21.15.121:0/2775888247 conn(0x7feeb0067a60 msgr2=0x7feeb0142190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:48.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.908+0000 7feeb6839700 1 -- 172.21.15.121:0/2775888247 shutdown_connections 2023-11-15T02:56:48.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:48.908+0000 7feeb6839700 1 -- 172.21.15.121:0/2775888247 wait complete. 2023-11-15T02:56:48.933 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-15T02:56:49.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.084+0000 7f0ceadb9700 1 Processor -- start 2023-11-15T02:56:49.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.084+0000 7f0ceadb9700 1 -- start start 2023-11-15T02:56:49.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.088+0000 7f0ceadb9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0ce41398a0 0x7f0ce40c2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:49.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.088+0000 7f0ceadb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce4139c80 0x7f0ce4068430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:49.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.088+0000 7f0ceadb9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce40c3400 con 0x7f0ce40633b0 2023-11-15T02:56:49.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.088+0000 7f0ceadb9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce40c27f0 con 0x7f0ce41398a0 2023-11-15T02:56:49.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.088+0000 7f0ceadb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce40c2ad0 con 0x7f0ce4139c80 2023-11-15T02:56:49.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.088+0000 7f0ce9db7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0ce41398a0 0x7f0ce40c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:49.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.088+0000 7f0ce9db7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0ce41398a0 0x7f0ce40c2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:37798/0 (socket says 172.21.15.121:37798) 2023-11-15T02:56:49.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.088+0000 7f0ce9db7700 1 -- 172.21.15.121:0/257183841 learned_addr learned my addr 172.21.15.121:0/257183841 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:49.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.088+0000 7f0cea5b8700 1 --1- 172.21.15.121:0/257183841 >> v1:172.21.15.190:6789/0 conn(0x7f0ce40633b0 0x7f0ce4153ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:54606/0 (socket says 172.21.15.121:54606) 2023-11-15T02:56:49.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.088+0000 7f0ce95b6700 1 --2- 172.21.15.121:0/257183841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce4139c80 0x7f0ce4068430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:49.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.088+0000 7f0ce9db7700 1 -- 172.21.15.121:0/257183841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce4139c80 msgr2=0x7f0ce4068430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:49.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.088+0000 7f0ce9db7700 1 --2- 172.21.15.121:0/257183841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce4139c80 0x7f0ce4068430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:49.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.088+0000 7f0ce9db7700 1 -- 172.21.15.121:0/257183841 >> v1:172.21.15.190:6789/0 conn(0x7f0ce40633b0 legacy=0x7f0ce4153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:49.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.088+0000 7f0ce9db7700 1 -- 172.21.15.121:0/257183841 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce40c2db0 con 0x7f0ce41398a0 2023-11-15T02:56:49.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.088+0000 7f0ce9db7700 1 --2- 172.21.15.121:0/257183841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce41398a0 0x7f0ce40c2260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f0ce0006fd0 tx=0x7f0ce00090f0).ready entity=mon.1 client_cookie=57f552c87647e847 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:49.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.088+0000 7f0ce8db5700 1 -- 172.21.15.121:0/257183841 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce000f510 con 0x7f0ce41398a0 2023-11-15T02:56:49.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.088+0000 7f0ce8db5700 1 -- 172.21.15.121:0/257183841 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce000f670 con 0x7f0ce41398a0 2023-11-15T02:56:49.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.088+0000 7f0ce8db5700 1 -- 172.21.15.121:0/257183841 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce000fa10 con 0x7f0ce41398a0 2023-11-15T02:56:49.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.092+0000 7f0ceadb9700 1 -- 172.21.15.121:0/257183841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce41398a0 msgr2=0x7f0ce40c2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:49.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.092+0000 7f0ceadb9700 1 --2- 172.21.15.121:0/257183841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce41398a0 0x7f0ce40c2260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f0ce0006fd0 tx=0x7f0ce00090f0).stop 2023-11-15T02:56:49.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.092+0000 7f0ceadb9700 1 -- 172.21.15.121:0/257183841 shutdown_connections 2023-11-15T02:56:49.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.092+0000 7f0ceadb9700 1 --2- 172.21.15.121:0/257183841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce4139c80 0x7f0ce4068430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:49.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.092+0000 7f0ceadb9700 1 --2- 172.21.15.121:0/257183841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce41398a0 0x7f0ce40c2260 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:49.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.092+0000 7f0ceadb9700 1 -- 172.21.15.121:0/257183841 >> 172.21.15.121:0/257183841 conn(0x7f0ce40c39a0 msgr2=0x7f0ce40c3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:49.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.092+0000 7f0ceadb9700 1 -- 172.21.15.121:0/257183841 shutdown_connections 2023-11-15T02:56:49.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.092+0000 7f0ceadb9700 1 -- 172.21.15.121:0/257183841 wait complete. 2023-11-15T02:56:49.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0ceadb9700 1 Processor -- start 2023-11-15T02:56:49.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0ceadb9700 1 -- start start 2023-11-15T02:56:49.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0ceadb9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce40633b0 0x7f0ce41378d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:49.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0ceadb9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ce41398a0 0x7f0ce40bf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:49.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0ceadb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce4139c80 0x7f0ce40bfde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:49.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0ceadb9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce40a6a40 con 0x7f0ce41398a0 2023-11-15T02:56:49.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0ceadb9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce40a6d20 con 0x7f0ce40633b0 2023-11-15T02:56:49.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0ceadb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce40c03b0 con 0x7f0ce4139c80 2023-11-15T02:56:49.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0ce95b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ce41398a0 0x7f0ce40bf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:49.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0ce95b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ce41398a0 0x7f0ce40bf8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44834/0 (socket says 172.21.15.121:44834) 2023-11-15T02:56:49.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0ce95b6700 1 -- 172.21.15.121:0/1140207630 learned_addr learned my addr 172.21.15.121:0/1140207630 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:49.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0ce9db7700 1 --2- 172.21.15.121:0/1140207630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce40633b0 0x7f0ce41378d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:49.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0cea5b8700 1 --2- 172.21.15.121:0/1140207630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce4139c80 0x7f0ce40bfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:49.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0ce95b6700 1 -- 172.21.15.121:0/1140207630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce4139c80 msgr2=0x7f0ce40bfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:49.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0ce95b6700 1 --2- 172.21.15.121:0/1140207630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce4139c80 0x7f0ce40bfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:49.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0ce95b6700 1 -- 172.21.15.121:0/1140207630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce40633b0 msgr2=0x7f0ce41378d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:49.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0ce95b6700 1 --2- 172.21.15.121:0/1140207630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce40633b0 0x7f0ce41378d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:49.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0ce95b6700 1 -- 172.21.15.121:0/1140207630 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce0009370 con 0x7f0ce41398a0 2023-11-15T02:56:49.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0ce9db7700 1 --2- 172.21.15.121:0/1140207630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce40633b0 0x7f0ce41378d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:56:49.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.096+0000 7f0ce95b6700 1 --2- 172.21.15.121:0/1140207630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ce41398a0 0x7f0ce40bf8f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f0cd400e960 tx=0x7f0cd4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:49.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.100+0000 7f0cd9ffb700 1 -- 172.21.15.121:0/1140207630 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4009510 con 0x7f0ce41398a0 2023-11-15T02:56:49.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.100+0000 7f0cd9ffb700 1 -- 172.21.15.121:0/1140207630 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd4012600 con 0x7f0ce41398a0 2023-11-15T02:56:49.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.100+0000 7f0cd9ffb700 1 -- 172.21.15.121:0/1140207630 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4012980 con 0x7f0ce41398a0 2023-11-15T02:56:49.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.100+0000 7f0ceadb9700 1 -- 172.21.15.121:0/1140207630 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce40c0690 con 0x7f0ce41398a0 2023-11-15T02:56:49.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.100+0000 7f0ceadb9700 1 -- 172.21.15.121:0/1140207630 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce40a6490 con 0x7f0ce41398a0 2023-11-15T02:56:49.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.100+0000 7f0ceadb9700 1 -- 172.21.15.121:0/1140207630 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce406bcf0 con 0x7f0ce41398a0 2023-11-15T02:56:49.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.100+0000 7f0cd9ffb700 1 -- 172.21.15.121:0/1140207630 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0cd4012d30 con 0x7f0ce41398a0 2023-11-15T02:56:49.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.104+0000 7f0cd9ffb700 1 --2- 172.21.15.121:0/1140207630 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0cc805a510 0x7f0cc805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:49.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.104+0000 7f0cd9ffb700 1 -- 172.21.15.121:0/1140207630 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f0cd4074900 con 0x7f0ce41398a0 2023-11-15T02:56:49.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.104+0000 7f0ce9db7700 1 --2- 172.21.15.121:0/1140207630 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0cc805a510 0x7f0cc805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:49.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.104+0000 7f0ce9db7700 1 --2- 172.21.15.121:0/1140207630 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0cc805a510 0x7f0cc805c9d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f0ce000a010 tx=0x7f0ce0017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:49.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.108+0000 7f0cd9ffb700 1 -- 172.21.15.121:0/1140207630 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0cd4048370 con 0x7f0ce41398a0 2023-11-15T02:56:49.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.280+0000 7f0ceadb9700 1 -- 172.21.15.121:0/1140207630 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0ce40c3090 con 0x7f0ce41398a0 2023-11-15T02:56:49.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.280+0000 7f0cd9ffb700 1 -- 172.21.15.121:0/1140207630 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+143 (secure 0 0 0) 0x7f0cd404b990 con 0x7f0ce41398a0 2023-11-15T02:56:49.285 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:56:49.285 INFO:teuthology.orchestra.run.smithi121.stdout:[ 2023-11-15T02:56:49.286 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:56:49.286 INFO:teuthology.orchestra.run.smithi121.stdout: "poolnum": 1, 2023-11-15T02:56:49.286 INFO:teuthology.orchestra.run.smithi121.stdout: "poolname": "device_health_metrics" 2023-11-15T02:56:49.287 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:56:49.287 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:56:49.287 INFO:teuthology.orchestra.run.smithi121.stdout: "poolnum": 2, 2023-11-15T02:56:49.287 INFO:teuthology.orchestra.run.smithi121.stdout: "poolname": "rbd" 2023-11-15T02:56:49.287 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:56:49.288 INFO:teuthology.orchestra.run.smithi121.stdout:] 2023-11-15T02:56:49.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.284+0000 7f0ceadb9700 1 -- 172.21.15.121:0/1140207630 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0cc805a510 msgr2=0x7f0cc805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:49.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.284+0000 7f0ceadb9700 1 --2- 172.21.15.121:0/1140207630 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0cc805a510 0x7f0cc805c9d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f0ce000a010 tx=0x7f0ce0017000).stop 2023-11-15T02:56:49.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.284+0000 7f0ceadb9700 1 -- 172.21.15.121:0/1140207630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ce41398a0 msgr2=0x7f0ce40bf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:49.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.284+0000 7f0ceadb9700 1 --2- 172.21.15.121:0/1140207630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ce41398a0 0x7f0ce40bf8f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f0cd400e960 tx=0x7f0cd4009060).stop 2023-11-15T02:56:49.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.284+0000 7f0ceadb9700 1 -- 172.21.15.121:0/1140207630 shutdown_connections 2023-11-15T02:56:49.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.284+0000 7f0ceadb9700 1 --2- 172.21.15.121:0/1140207630 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0cc805a510 0x7f0cc805c9d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:49.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.284+0000 7f0ceadb9700 1 --2- 172.21.15.121:0/1140207630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce4139c80 0x7f0ce40bfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:49.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.284+0000 7f0ceadb9700 1 --2- 172.21.15.121:0/1140207630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ce41398a0 0x7f0ce40bf8f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:49.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.284+0000 7f0ceadb9700 1 --2- 172.21.15.121:0/1140207630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce40633b0 0x7f0ce41378d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:49.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.284+0000 7f0ceadb9700 1 -- 172.21.15.121:0/1140207630 >> 172.21.15.121:0/1140207630 conn(0x7f0ce40c39a0 msgr2=0x7f0ce40bbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:49.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.284+0000 7f0ceadb9700 1 -- 172.21.15.121:0/1140207630 shutdown_connections 2023-11-15T02:56:49.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.284+0000 7f0ceadb9700 1 -- 172.21.15.121:0/1140207630 wait complete. 2023-11-15T02:56:49.309 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-15T02:56:49.309 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-15T02:56:49.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.468+0000 7f9559379700 1 Processor -- start 2023-11-15T02:56:49.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.468+0000 7f9559379700 1 -- start start 2023-11-15T02:56:49.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.468+0000 7f9559379700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f95540ab0d0 0x7f95540ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:49.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f9559379700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f95540add40 0x7f95540ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:49.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f9559379700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9554066c90 con 0x7f95540ad1b0 2023-11-15T02:56:49.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f9559379700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9554065e20 con 0x7f95540ab0d0 2023-11-15T02:56:49.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f9559379700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9554066080 con 0x7f95540add40 2023-11-15T02:56:49.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f9553fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f95540ab0d0 0x7f95540ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:49.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f954b7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f95540ad1b0 0x7f9554065710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:42370/0 (socket says 172.21.15.121:42370) 2023-11-15T02:56:49.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f954b7fe700 1 -- 172.21.15.121:0/1115291903 learned_addr learned my addr 172.21.15.121:0/1115291903 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:49.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f9558b78700 1 --2- 172.21.15.121:0/1115291903 >> v2:172.21.15.190:3300/0 conn(0x7f95540add40 0x7f95540ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:49.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f95537fe700 1 -- 172.21.15.121:0/1115291903 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2607588619 0 0) 0x7f9554066c90 con 0x7f95540ad1b0 2023-11-15T02:56:49.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f95537fe700 1 -- 172.21.15.121:0/1115291903 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95400033f0 con 0x7f95540ad1b0 2023-11-15T02:56:49.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f9553fff700 1 -- 172.21.15.121:0/1115291903 >> v2:172.21.15.190:3300/0 conn(0x7f95540add40 msgr2=0x7f95540ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:49.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f9553fff700 1 --2- 172.21.15.121:0/1115291903 >> v2:172.21.15.190:3300/0 conn(0x7f95540add40 0x7f95540ae1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:49.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f9553fff700 1 -- 172.21.15.121:0/1115291903 >> v1:172.21.15.121:6789/0 conn(0x7f95540ad1b0 legacy=0x7f9554065710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:49.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f9553fff700 1 -- 172.21.15.121:0/1115291903 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9554066360 con 0x7f95540ab0d0 2023-11-15T02:56:49.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f9553fff700 1 --2- 172.21.15.121:0/1115291903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95540ab0d0 0x7f95540ab4b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f9554068ca0 tx=0x7f9544006f40).ready entity=mon.0 client_cookie=d0f8b6b42d6652df server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:49.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f95537fe700 1 -- 172.21.15.121:0/1115291903 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544005880 con 0x7f95540ab0d0 2023-11-15T02:56:49.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f95537fe700 1 -- 172.21.15.121:0/1115291903 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95440059e0 con 0x7f95540ab0d0 2023-11-15T02:56:49.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f95537fe700 1 -- 172.21.15.121:0/1115291903 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544005d60 con 0x7f95540ab0d0 2023-11-15T02:56:49.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f9559379700 1 -- 172.21.15.121:0/1115291903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95540ab0d0 msgr2=0x7f95540ab4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:49.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f9559379700 1 --2- 172.21.15.121:0/1115291903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95540ab0d0 0x7f95540ab4b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f9554068ca0 tx=0x7f9544006f40).stop 2023-11-15T02:56:49.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f9559379700 1 -- 172.21.15.121:0/1115291903 shutdown_connections 2023-11-15T02:56:49.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f9559379700 1 --2- 172.21.15.121:0/1115291903 >> v2:172.21.15.190:3300/0 conn(0x7f95540add40 0x7f95540ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:49.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f9559379700 1 --2- 172.21.15.121:0/1115291903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95540ab0d0 0x7f95540ab4b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:49.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f9559379700 1 -- 172.21.15.121:0/1115291903 >> 172.21.15.121:0/1115291903 conn(0x7f9554067a60 msgr2=0x7f9554067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:49.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f9559379700 1 -- 172.21.15.121:0/1115291903 shutdown_connections 2023-11-15T02:56:49.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.472+0000 7f9559379700 1 -- 172.21.15.121:0/1115291903 wait complete. 2023-11-15T02:56:49.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.476+0000 7f9559379700 1 Processor -- start 2023-11-15T02:56:49.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.476+0000 7f9559379700 1 -- start start 2023-11-15T02:56:49.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9559379700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95540ab0d0 0x7f9554151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:49.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9559379700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95540ad1b0 0x7f95540a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:49.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9559379700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95540add40 0x7f95540a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:49.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9559379700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95540a5de0 con 0x7f95540ab0d0 2023-11-15T02:56:49.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9559379700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95540a5b00 con 0x7f95540ad1b0 2023-11-15T02:56:49.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9559379700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95541524d0 con 0x7f95540add40 2023-11-15T02:56:49.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9553fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95540ab0d0 0x7f9554151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:49.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9553fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95540ab0d0 0x7f9554151fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44856/0 (socket says 172.21.15.121:44856) 2023-11-15T02:56:49.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9553fff700 1 -- 172.21.15.121:0/4192363593 learned_addr learned my addr 172.21.15.121:0/4192363593 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:49.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f954b7fe700 1 --2- 172.21.15.121:0/4192363593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95540ad1b0 0x7f95540a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:49.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9558b78700 1 --2- 172.21.15.121:0/4192363593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95540add40 0x7f95540a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:49.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9553fff700 1 -- 172.21.15.121:0/4192363593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95540add40 msgr2=0x7f95540a84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:49.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9553fff700 1 --2- 172.21.15.121:0/4192363593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95540add40 0x7f95540a84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:49.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9553fff700 1 -- 172.21.15.121:0/4192363593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95540ad1b0 msgr2=0x7f95540a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:49.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9553fff700 1 --2- 172.21.15.121:0/4192363593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95540ad1b0 0x7f95540a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:49.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9553fff700 1 -- 172.21.15.121:0/4192363593 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9544009250 con 0x7f95540ab0d0 2023-11-15T02:56:49.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9553fff700 1 --2- 172.21.15.121:0/4192363593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95540ab0d0 0x7f9554151fe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f9554065120 tx=0x7f9544016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:49.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9550ff9700 1 -- 172.21.15.121:0/4192363593 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544006770 con 0x7f95540ab0d0 2023-11-15T02:56:49.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9550ff9700 1 -- 172.21.15.121:0/4192363593 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954400f720 con 0x7f95540ab0d0 2023-11-15T02:56:49.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9559379700 1 -- 172.21.15.121:0/4192363593 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95540a4ab0 con 0x7f95540ab0d0 2023-11-15T02:56:49.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9550ff9700 1 -- 172.21.15.121:0/4192363593 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954400faa0 con 0x7f95540ab0d0 2023-11-15T02:56:49.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.480+0000 7f9559379700 1 -- 172.21.15.121:0/4192363593 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95540a5070 con 0x7f95540ab0d0 2023-11-15T02:56:49.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.484+0000 7f9559379700 1 -- 172.21.15.121:0/4192363593 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f955406bcf0 con 0x7f95540ab0d0 2023-11-15T02:56:49.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.484+0000 7f9550ff9700 1 -- 172.21.15.121:0/4192363593 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9544026460 con 0x7f95540ab0d0 2023-11-15T02:56:49.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.484+0000 7f9550ff9700 1 --2- 172.21.15.121:0/4192363593 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f953405a510 0x7f953405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:49.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.488+0000 7f9550ff9700 1 -- 172.21.15.121:0/4192363593 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f95440757e0 con 0x7f95540ab0d0 2023-11-15T02:56:49.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.488+0000 7f954b7fe700 1 --2- 172.21.15.121:0/4192363593 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f953405a510 0x7f953405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:49.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.488+0000 7f954b7fe700 1 --2- 172.21.15.121:0/4192363593 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f953405a510 0x7f953405c9d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f95540c69d0 tx=0x7f953c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:49.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.492+0000 7f9550ff9700 1 -- 172.21.15.121:0/4192363593 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f95440492b0 con 0x7f95540ab0d0 2023-11-15T02:56:49.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:49.652+0000 7f9559379700 1 -- 172.21.15.121:0/4192363593 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f9554068a70 con 0x7f95540ab0d0 2023-11-15T02:56:50.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.224+0000 7f9550ff9700 1 -- 172.21.15.121:0/4192363593 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 152+0+0 (secure 0 0 0) 0x7f954404c8d0 con 0x7f95540ab0d0 2023-11-15T02:56:50.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.232+0000 7f9559379700 1 -- 172.21.15.121:0/4192363593 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f953405a510 msgr2=0x7f953405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:50.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.232+0000 7f9559379700 1 --2- 172.21.15.121:0/4192363593 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f953405a510 0x7f953405c9d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f95540c69d0 tx=0x7f953c009040).stop 2023-11-15T02:56:50.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.232+0000 7f9559379700 1 -- 172.21.15.121:0/4192363593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95540ab0d0 msgr2=0x7f9554151fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:50.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.232+0000 7f9559379700 1 --2- 172.21.15.121:0/4192363593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95540ab0d0 0x7f9554151fe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f9554065120 tx=0x7f9544016000).stop 2023-11-15T02:56:50.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.232+0000 7f9559379700 1 -- 172.21.15.121:0/4192363593 shutdown_connections 2023-11-15T02:56:50.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.232+0000 7f9559379700 1 --2- 172.21.15.121:0/4192363593 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f953405a510 0x7f953405c9d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:50.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.232+0000 7f9559379700 1 --2- 172.21.15.121:0/4192363593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95540add40 0x7f95540a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:50.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.232+0000 7f9559379700 1 --2- 172.21.15.121:0/4192363593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95540ad1b0 0x7f95540a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:50.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.232+0000 7f9559379700 1 --2- 172.21.15.121:0/4192363593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95540ab0d0 0x7f9554151fe0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:50.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.232+0000 7f9559379700 1 -- 172.21.15.121:0/4192363593 >> 172.21.15.121:0/4192363593 conn(0x7f9554067a60 msgr2=0x7f9554153380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:50.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.232+0000 7f9559379700 1 -- 172.21.15.121:0/4192363593 shutdown_connections 2023-11-15T02:56:50.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.232+0000 7f9559379700 1 -- 172.21.15.121:0/4192363593 wait complete. 2023-11-15T02:56:50.242 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_metadata' created 2023-11-15T02:56:50.260 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-15T02:56:50.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.428+0000 7f457581b700 1 Processor -- start 2023-11-15T02:56:50.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.428+0000 7f457581b700 1 -- start start 2023-11-15T02:56:50.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.432+0000 7f457581b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f45701570b0 0x7f4570066860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:50.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.432+0000 7f457581b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4570062da0 con 0x7f45700c5c80 2023-11-15T02:56:50.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.432+0000 7f457581b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45700bfb90 con 0x7f4570157490 2023-11-15T02:56:50.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.432+0000 7f457581b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45700c6140 con 0x7f45701570b0 2023-11-15T02:56:50.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.432+0000 7f456effd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f45701570b0 0x7f4570066860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:50.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.432+0000 7f456f7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f45700c5c80 0x7f4570063230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:42374/0 (socket says 172.21.15.121:42374) 2023-11-15T02:56:50.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.432+0000 7f456f7fe700 1 -- 172.21.15.121:0/550107904 learned_addr learned my addr 172.21.15.121:0/550107904 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:50.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.436+0000 7f456effd700 1 -- 172.21.15.121:0/550107904 >> v1:172.21.15.190:6789/0 conn(0x7f4570157490 legacy=0x7f4570065950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:50.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.436+0000 7f456effd700 1 -- 172.21.15.121:0/550107904 >> v1:172.21.15.121:6789/0 conn(0x7f45700c5c80 legacy=0x7f4570063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:50.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.436+0000 7f456effd700 1 -- 172.21.15.121:0/550107904 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45700c6420 con 0x7f45701570b0 2023-11-15T02:56:50.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.436+0000 7f456dffb700 1 -- 172.21.15.121:0/550107904 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2194218738 0 0) 0x7f45700bfb90 con 0x7f4570157490 2023-11-15T02:56:50.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.436+0000 7f456effd700 1 --2- 172.21.15.121:0/550107904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45701570b0 0x7f4570066860 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f4560009760 tx=0x7f4560009060).ready entity=mon.0 client_cookie=6a9aa45d72d1111f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:50.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.436+0000 7f456dffb700 1 -- 172.21.15.121:0/550107904 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456000b580 con 0x7f45701570b0 2023-11-15T02:56:50.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.436+0000 7f456dffb700 1 -- 172.21.15.121:0/550107904 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456000b6e0 con 0x7f45701570b0 2023-11-15T02:56:50.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.436+0000 7f456dffb700 1 -- 172.21.15.121:0/550107904 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456000ba60 con 0x7f45701570b0 2023-11-15T02:56:50.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.436+0000 7f457581b700 1 -- 172.21.15.121:0/550107904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45701570b0 msgr2=0x7f4570066860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:50.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.436+0000 7f457581b700 1 --2- 172.21.15.121:0/550107904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45701570b0 0x7f4570066860 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f4560009760 tx=0x7f4560009060).stop 2023-11-15T02:56:50.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.436+0000 7f457581b700 1 -- 172.21.15.121:0/550107904 shutdown_connections 2023-11-15T02:56:50.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.436+0000 7f457581b700 1 --2- 172.21.15.121:0/550107904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45701570b0 0x7f4570066860 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:50.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.436+0000 7f457581b700 1 -- 172.21.15.121:0/550107904 >> 172.21.15.121:0/550107904 conn(0x7f45700684f0 msgr2=0x7f45700cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:50.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.436+0000 7f457581b700 1 -- 172.21.15.121:0/550107904 shutdown_connections 2023-11-15T02:56:50.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.436+0000 7f457581b700 1 -- 172.21.15.121:0/550107904 wait complete. 2023-11-15T02:56:50.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f457581b700 1 Processor -- start 2023-11-15T02:56:50.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f457581b700 1 -- start start 2023-11-15T02:56:50.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f457581b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45700c5c80 0x7f45700aa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:50.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f457581b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45701570b0 0x7f45700ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:50.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f457581b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4570158850 0x7f4570158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:50.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f457581b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4570156620 con 0x7f45701570b0 2023-11-15T02:56:50.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f457581b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45701591a0 con 0x7f45700c5c80 2023-11-15T02:56:50.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f457581b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45700c17c0 con 0x7f4570158850 2023-11-15T02:56:50.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f456e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45701570b0 0x7f45700ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:50.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f456e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45701570b0 0x7f45700ac7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44870/0 (socket says 172.21.15.121:44870) 2023-11-15T02:56:50.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f456e7fc700 1 -- 172.21.15.121:0/1144058534 learned_addr learned my addr 172.21.15.121:0/1144058534 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:50.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f456effd700 1 --2- 172.21.15.121:0/1144058534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45700c5c80 0x7f45700aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:50.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f456f7fe700 1 --2- 172.21.15.121:0/1144058534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4570158850 0x7f4570158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:50.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f456e7fc700 1 -- 172.21.15.121:0/1144058534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4570158850 msgr2=0x7f4570158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:50.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f456e7fc700 1 --2- 172.21.15.121:0/1144058534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4570158850 0x7f4570158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:50.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f456e7fc700 1 -- 172.21.15.121:0/1144058534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45700c5c80 msgr2=0x7f45700aa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:50.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f456e7fc700 1 --2- 172.21.15.121:0/1144058534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45700c5c80 0x7f45700aa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:50.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f456e7fc700 1 -- 172.21.15.121:0/1144058534 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45600092a0 con 0x7f45701570b0 2023-11-15T02:56:50.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f456e7fc700 1 --2- 172.21.15.121:0/1144058534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45701570b0 0x7f45700ac7e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f456400c410 tx=0x7f4564007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:50.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f4556ffd700 1 -- 172.21.15.121:0/1144058534 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456400ce70 con 0x7f45701570b0 2023-11-15T02:56:50.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f4556ffd700 1 -- 172.21.15.121:0/1144058534 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4564007020 con 0x7f45701570b0 2023-11-15T02:56:50.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f457581b700 1 -- 172.21.15.121:0/1144058534 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45700c1aa0 con 0x7f45701570b0 2023-11-15T02:56:50.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f4556ffd700 1 -- 172.21.15.121:0/1144058534 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45640073c0 con 0x7f45701570b0 2023-11-15T02:56:50.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.440+0000 7f457581b700 1 -- 172.21.15.121:0/1144058534 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45700c2060 con 0x7f45701570b0 2023-11-15T02:56:50.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.444+0000 7f457581b700 1 -- 172.21.15.121:0/1144058534 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4570167d80 con 0x7f45701570b0 2023-11-15T02:56:50.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.444+0000 7f4556ffd700 1 -- 172.21.15.121:0/1144058534 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f45640068b0 con 0x7f45701570b0 2023-11-15T02:56:50.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.448+0000 7f4556ffd700 1 --2- 172.21.15.121:0/1144058534 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f455c05a540 0x7f455c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:50.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.448+0000 7f4556ffd700 1 -- 172.21.15.121:0/1144058534 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f45640739e0 con 0x7f45701570b0 2023-11-15T02:56:50.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.448+0000 7f456effd700 1 --2- 172.21.15.121:0/1144058534 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f455c05a540 0x7f455c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:50.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.448+0000 7f456effd700 1 --2- 172.21.15.121:0/1144058534 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f455c05a540 0x7f455c05ca00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f456000b340 tx=0x7f4560017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:50.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.452+0000 7f4556ffd700 1 -- 172.21.15.121:0/1144058534 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4570167d80 con 0x7f45701570b0 2023-11-15T02:56:50.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:50.612+0000 7f457581b700 1 -- 172.21.15.121:0/1144058534 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f457006c150 con 0x7f45701570b0 2023-11-15T02:56:51.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.240+0000 7f4556ffd700 1 -- 172.21.15.121:0/1144058534 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f45640472e0 con 0x7f45701570b0 2023-11-15T02:56:51.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.248+0000 7f457581b700 1 -- 172.21.15.121:0/1144058534 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f455c05a540 msgr2=0x7f455c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:51.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.248+0000 7f457581b700 1 --2- 172.21.15.121:0/1144058534 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f455c05a540 0x7f455c05ca00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f456000b340 tx=0x7f4560017000).stop 2023-11-15T02:56:51.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.248+0000 7f457581b700 1 -- 172.21.15.121:0/1144058534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45701570b0 msgr2=0x7f45700ac7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:51.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.248+0000 7f457581b700 1 --2- 172.21.15.121:0/1144058534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45701570b0 0x7f45700ac7e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f456400c410 tx=0x7f4564007e00).stop 2023-11-15T02:56:51.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.252+0000 7f457581b700 1 -- 172.21.15.121:0/1144058534 shutdown_connections 2023-11-15T02:56:51.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.252+0000 7f457581b700 1 --2- 172.21.15.121:0/1144058534 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f455c05a540 0x7f455c05ca00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:51.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.252+0000 7f457581b700 1 --2- 172.21.15.121:0/1144058534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4570158850 0x7f4570158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:51.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.252+0000 7f457581b700 1 --2- 172.21.15.121:0/1144058534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45701570b0 0x7f45700ac7e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:51.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.252+0000 7f457581b700 1 --2- 172.21.15.121:0/1144058534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45700c5c80 0x7f45700aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:51.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.252+0000 7f457581b700 1 -- 172.21.15.121:0/1144058534 >> 172.21.15.121:0/1144058534 conn(0x7f45700684f0 msgr2=0x7f45700aa0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:51.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.256+0000 7f457581b700 1 -- 172.21.15.121:0/1144058534 shutdown_connections 2023-11-15T02:56:51.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.260+0000 7f457581b700 1 -- 172.21.15.121:0/1144058534 wait complete. 2023-11-15T02:56:51.272 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_data' created 2023-11-15T02:56:51.299 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-15T02:56:51.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.456+0000 7ffa3585b700 1 Processor -- start 2023-11-15T02:56:51.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.456+0000 7ffa3585b700 1 -- start start 2023-11-15T02:56:51.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.460+0000 7ffa3585b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa30151500 0x7ffa300be0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:51.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.460+0000 7ffa3585b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa30157e50 0x7ffa30155340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:51.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.460+0000 7ffa3585b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa300a9b40 con 0x7ffa300aabe0 2023-11-15T02:56:51.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.460+0000 7ffa3585b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa300a9e20 con 0x7ffa30151500 2023-11-15T02:56:51.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.460+0000 7ffa3585b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa300aa100 con 0x7ffa30157e50 2023-11-15T02:56:51.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.460+0000 7ffa2effd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa30151500 0x7ffa300be0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:51.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.460+0000 7ffa2e7fc700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ffa300aabe0 0x7ffa30157740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:42384/0 (socket says 172.21.15.121:42384) 2023-11-15T02:56:51.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.460+0000 7ffa2e7fc700 1 -- 172.21.15.121:0/985581223 learned_addr learned my addr 172.21.15.121:0/985581223 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:51.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.460+0000 7ffa2f7fe700 1 --2- 172.21.15.121:0/985581223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa30157e50 0x7ffa30155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:51.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.460+0000 7ffa2dffb700 1 -- 172.21.15.121:0/985581223 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125016055 0 0) 0x7ffa300a9b40 con 0x7ffa300aabe0 2023-11-15T02:56:51.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.460+0000 7ffa2dffb700 1 -- 172.21.15.121:0/985581223 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa180033f0 con 0x7ffa300aabe0 2023-11-15T02:56:51.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.460+0000 7ffa2f7fe700 1 -- 172.21.15.121:0/985581223 >> v2:172.21.15.121:3300/0 conn(0x7ffa30151500 msgr2=0x7ffa300be0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:51.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.460+0000 7ffa2f7fe700 1 --2- 172.21.15.121:0/985581223 >> v2:172.21.15.121:3300/0 conn(0x7ffa30151500 0x7ffa300be0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:51.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.460+0000 7ffa2f7fe700 1 -- 172.21.15.121:0/985581223 >> v1:172.21.15.121:6789/0 conn(0x7ffa300aabe0 legacy=0x7ffa30157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:51.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.460+0000 7ffa2f7fe700 1 -- 172.21.15.121:0/985581223 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa300aa3e0 con 0x7ffa30157e50 2023-11-15T02:56:51.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.460+0000 7ffa2effd700 1 --2- 172.21.15.121:0/985581223 >> v2:172.21.15.121:3300/0 conn(0x7ffa30151500 0x7ffa300be0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:56:51.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.460+0000 7ffa2f7fe700 1 --2- 172.21.15.121:0/985581223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa30157e50 0x7ffa30155340 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ffa28009e60 tx=0x7ffa28009780).ready entity=mon.2 client_cookie=64e3ef27cd4607a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:51.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.464+0000 7ffa2dffb700 1 -- 172.21.15.121:0/985581223 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2800e070 con 0x7ffa30157e50 2023-11-15T02:56:51.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.464+0000 7ffa2dffb700 1 -- 172.21.15.121:0/985581223 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2800bab0 con 0x7ffa30157e50 2023-11-15T02:56:51.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.464+0000 7ffa2dffb700 1 -- 172.21.15.121:0/985581223 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2800be30 con 0x7ffa30157e50 2023-11-15T02:56:51.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.464+0000 7ffa3585b700 1 -- 172.21.15.121:0/985581223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa30157e50 msgr2=0x7ffa30155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:51.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.464+0000 7ffa3585b700 1 --2- 172.21.15.121:0/985581223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa30157e50 0x7ffa30155340 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ffa28009e60 tx=0x7ffa28009780).stop 2023-11-15T02:56:51.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.464+0000 7ffa3585b700 1 -- 172.21.15.121:0/985581223 shutdown_connections 2023-11-15T02:56:51.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.464+0000 7ffa3585b700 1 --2- 172.21.15.121:0/985581223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa30157e50 0x7ffa30155340 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:51.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.464+0000 7ffa3585b700 1 --2- 172.21.15.121:0/985581223 >> v2:172.21.15.121:3300/0 conn(0x7ffa30151500 0x7ffa300be0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:51.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.464+0000 7ffa3585b700 1 -- 172.21.15.121:0/985581223 >> 172.21.15.121:0/985581223 conn(0x7ffa30067a60 msgr2=0x7ffa30067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:51.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.464+0000 7ffa3585b700 1 -- 172.21.15.121:0/985581223 shutdown_connections 2023-11-15T02:56:51.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.464+0000 7ffa3585b700 1 -- 172.21.15.121:0/985581223 wait complete. 2023-11-15T02:56:51.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.468+0000 7ffa3585b700 1 Processor -- start 2023-11-15T02:56:51.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.468+0000 7ffa3585b700 1 -- start start 2023-11-15T02:56:51.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.468+0000 7ffa3585b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa300aabe0 0x7ffa300c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:51.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.468+0000 7ffa3585b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa30151500 0x7ffa300c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:51.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.468+0000 7ffa3585b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa30157e50 0x7ffa300c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:51.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.468+0000 7ffa3585b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa300c16f0 con 0x7ffa30157e50 2023-11-15T02:56:51.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.468+0000 7ffa3585b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa300c0800 con 0x7ffa30151500 2023-11-15T02:56:51.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.468+0000 7ffa3585b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa300c0ae0 con 0x7ffa300aabe0 2023-11-15T02:56:51.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.468+0000 7ffa2f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa30157e50 0x7ffa300c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:51.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.468+0000 7ffa2effd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa300aabe0 0x7ffa300c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:51.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.468+0000 7ffa2f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa30157e50 0x7ffa300c4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44896/0 (socket says 172.21.15.121:44896) 2023-11-15T02:56:51.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.468+0000 7ffa2f7fe700 1 -- 172.21.15.121:0/2143696929 learned_addr learned my addr 172.21.15.121:0/2143696929 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:51.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.468+0000 7ffa2e7fc700 1 --2- 172.21.15.121:0/2143696929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa30151500 0x7ffa300c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:51.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.468+0000 7ffa2effd700 1 -- 172.21.15.121:0/2143696929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa30151500 msgr2=0x7ffa300c1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:51.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.468+0000 7ffa2effd700 1 --2- 172.21.15.121:0/2143696929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa30151500 0x7ffa300c1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:51.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.468+0000 7ffa2effd700 1 -- 172.21.15.121:0/2143696929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa30157e50 msgr2=0x7ffa300c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:51.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.468+0000 7ffa2effd700 1 --2- 172.21.15.121:0/2143696929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa30157e50 0x7ffa300c4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:51.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.468+0000 7ffa2effd700 1 -- 172.21.15.121:0/2143696929 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa200092a0 con 0x7ffa300aabe0 2023-11-15T02:56:51.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.472+0000 7ffa2effd700 1 --2- 172.21.15.121:0/2143696929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa300aabe0 0x7ffa300c8b20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7ffa2000e980 tx=0x7ffa20009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:51.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.472+0000 7ffa16ffd700 1 -- 172.21.15.121:0/2143696929 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa20012600 con 0x7ffa300aabe0 2023-11-15T02:56:51.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.472+0000 7ffa16ffd700 1 -- 172.21.15.121:0/2143696929 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa20012760 con 0x7ffa300aabe0 2023-11-15T02:56:51.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.472+0000 7ffa16ffd700 1 -- 172.21.15.121:0/2143696929 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa20012ae0 con 0x7ffa300aabe0 2023-11-15T02:56:51.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.472+0000 7ffa3585b700 1 -- 172.21.15.121:0/2143696929 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa28009a20 con 0x7ffa300aabe0 2023-11-15T02:56:51.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.472+0000 7ffa3585b700 1 -- 172.21.15.121:0/2143696929 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa300c10a0 con 0x7ffa300aabe0 2023-11-15T02:56:51.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.472+0000 7ffa3585b700 1 -- 172.21.15.121:0/2143696929 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa3006bcf0 con 0x7ffa300aabe0 2023-11-15T02:56:51.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.476+0000 7ffa16ffd700 1 -- 172.21.15.121:0/2143696929 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ffa2001b400 con 0x7ffa300aabe0 2023-11-15T02:56:51.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.476+0000 7ffa16ffd700 1 --2- 172.21.15.121:0/2143696929 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa1c05a510 0x7ffa1c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:51.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.476+0000 7ffa16ffd700 1 -- 172.21.15.121:0/2143696929 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6021+0+0 (secure 0 0 0) 0x7ffa20074b60 con 0x7ffa300aabe0 2023-11-15T02:56:51.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.476+0000 7ffa2e7fc700 1 --2- 172.21.15.121:0/2143696929 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa1c05a510 0x7ffa1c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:51.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.476+0000 7ffa2e7fc700 1 --2- 172.21.15.121:0/2143696929 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa1c05a510 0x7ffa1c05c9d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7ffa300c58d0 tx=0x7ffa24009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:51.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.484+0000 7ffa16ffd700 1 -- 172.21.15.121:0/2143696929 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ffa20048340 con 0x7ffa300aabe0 2023-11-15T02:56:51.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.688+0000 7ffa3585b700 1 -- 172.21.15.121:0/2143696929 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ffa300be840 con 0x7ffa300aabe0 2023-11-15T02:56:51.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.704+0000 7ffa16ffd700 1 -- 172.21.15.121:0/2143696929 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7ffa20016070 con 0x7ffa300aabe0 2023-11-15T02:56:51.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.708+0000 7ffa3585b700 1 -- 172.21.15.121:0/2143696929 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa1c05a510 msgr2=0x7ffa1c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:51.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.708+0000 7ffa3585b700 1 --2- 172.21.15.121:0/2143696929 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa1c05a510 0x7ffa1c05c9d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7ffa300c58d0 tx=0x7ffa24009040).stop 2023-11-15T02:56:51.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.708+0000 7ffa3585b700 1 -- 172.21.15.121:0/2143696929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa300aabe0 msgr2=0x7ffa300c8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:51.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.708+0000 7ffa3585b700 1 --2- 172.21.15.121:0/2143696929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa300aabe0 0x7ffa300c8b20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7ffa2000e980 tx=0x7ffa20009060).stop 2023-11-15T02:56:51.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.712+0000 7ffa3585b700 1 -- 172.21.15.121:0/2143696929 shutdown_connections 2023-11-15T02:56:51.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.712+0000 7ffa3585b700 1 --2- 172.21.15.121:0/2143696929 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa1c05a510 0x7ffa1c05c9d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:51.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.712+0000 7ffa3585b700 1 --2- 172.21.15.121:0/2143696929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa30157e50 0x7ffa300c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:51.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.712+0000 7ffa3585b700 1 --2- 172.21.15.121:0/2143696929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa30151500 0x7ffa300c1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:51.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.712+0000 7ffa3585b700 1 --2- 172.21.15.121:0/2143696929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa300aabe0 0x7ffa300c8b20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:51.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.712+0000 7ffa3585b700 1 -- 172.21.15.121:0/2143696929 >> 172.21.15.121:0/2143696929 conn(0x7ffa30067a60 msgr2=0x7ffa3014eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:51.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.712+0000 7ffa3585b700 1 -- 172.21.15.121:0/2143696929 shutdown_connections 2023-11-15T02:56:51.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.712+0000 7ffa3585b700 1 -- 172.21.15.121:0/2143696929 wait complete. 2023-11-15T02:56:51.730 INFO:teuthology.orchestra.run.smithi121.stderr:new fs with metadata pool 3 and data pool 4 2023-11-15T02:56:51.748 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:56:51.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.892+0000 7f1d6425b700 1 Processor -- start 2023-11-15T02:56:51.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.892+0000 7f1d6425b700 1 -- start start 2023-11-15T02:56:51.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.896+0000 7f1d6425b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d5c150870 0x7f1d5c0c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:51.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.896+0000 7f1d6425b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1d5c153d20 0x7f1d5c0c3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:51.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.896+0000 7f1d6425b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d5c0c6350 con 0x7f1d5c150c50 2023-11-15T02:56:51.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.896+0000 7f1d6425b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d5c151780 con 0x7f1d5c153d20 2023-11-15T02:56:51.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.896+0000 7f1d6425b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c0c3b70 con 0x7f1d5c150870 2023-11-15T02:56:51.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.896+0000 7f1d617f6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1d5c153d20 0x7f1d5c0c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:51.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.896+0000 7f1d617f6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1d5c153d20 0x7f1d5c0c3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44912/0 (socket says 172.21.15.121:44912) 2023-11-15T02:56:51.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.896+0000 7f1d617f6700 1 -- 172.21.15.121:0/3011913297 learned_addr learned my addr 172.21.15.121:0/3011913297 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:51.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.896+0000 7f1d61ff7700 1 --2- 172.21.15.121:0/3011913297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d5c150870 0x7f1d5c0c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:51.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.896+0000 7f1d617f6700 1 -- 172.21.15.121:0/3011913297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d5c150870 msgr2=0x7f1d5c0c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:51.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.896+0000 7f1d617f6700 1 --2- 172.21.15.121:0/3011913297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d5c150870 0x7f1d5c0c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:51.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.896+0000 7f1d617f6700 1 -- 172.21.15.121:0/3011913297 >> v1:172.21.15.190:6789/0 conn(0x7f1d5c150c50 legacy=0x7f1d5c151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:51.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.896+0000 7f1d617f6700 1 -- 172.21.15.121:0/3011913297 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5c1541b0 con 0x7f1d5c153d20 2023-11-15T02:56:51.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.896+0000 7f1d617f6700 1 --2- 172.21.15.121:0/3011913297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d5c153d20 0x7f1d5c0c3680 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f1d4c006fd0 tx=0x7f1d4c0090f0).ready entity=mon.0 client_cookie=c32321ffe290eb01 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:51.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.896+0000 7f1d60ff5700 1 -- 172.21.15.121:0/3011913297 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c00b580 con 0x7f1d5c153d20 2023-11-15T02:56:51.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.896+0000 7f1d60ff5700 1 -- 172.21.15.121:0/3011913297 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4c00b6e0 con 0x7f1d5c153d20 2023-11-15T02:56:51.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.896+0000 7f1d60ff5700 1 -- 172.21.15.121:0/3011913297 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c00ba60 con 0x7f1d5c153d20 2023-11-15T02:56:51.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.900+0000 7f1d6425b700 1 -- 172.21.15.121:0/3011913297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d5c153d20 msgr2=0x7f1d5c0c3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:51.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.900+0000 7f1d6425b700 1 --2- 172.21.15.121:0/3011913297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d5c153d20 0x7f1d5c0c3680 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f1d4c006fd0 tx=0x7f1d4c0090f0).stop 2023-11-15T02:56:51.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.900+0000 7f1d6425b700 1 -- 172.21.15.121:0/3011913297 shutdown_connections 2023-11-15T02:56:51.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.900+0000 7f1d6425b700 1 --2- 172.21.15.121:0/3011913297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d5c153d20 0x7f1d5c0c3680 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:51.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.900+0000 7f1d6425b700 1 --2- 172.21.15.121:0/3011913297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d5c150870 0x7f1d5c0c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:51.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.900+0000 7f1d6425b700 1 -- 172.21.15.121:0/3011913297 >> 172.21.15.121:0/3011913297 conn(0x7f1d5c062190 msgr2=0x7f1d5c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:51.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.900+0000 7f1d6425b700 1 -- 172.21.15.121:0/3011913297 shutdown_connections 2023-11-15T02:56:51.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.900+0000 7f1d6425b700 1 -- 172.21.15.121:0/3011913297 wait complete. 2023-11-15T02:56:51.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.900+0000 7f1d6425b700 1 Processor -- start 2023-11-15T02:56:51.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.900+0000 7f1d6425b700 1 -- start start 2023-11-15T02:56:51.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d6425b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d5c150870 0x7f1d5c168130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:51.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d6425b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d5c150c50 0x7f1d5c164000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:51.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d6425b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d5c0a7ff0 0x7f1d5c0bfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:51.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d6425b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c0a87b0 con 0x7f1d5c150c50 2023-11-15T02:56:51.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d6425b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c1644f0 con 0x7f1d5c150870 2023-11-15T02:56:51.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d6425b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c147f30 con 0x7f1d5c0a7ff0 2023-11-15T02:56:51.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d617f6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d5c150c50 0x7f1d5c164000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:51.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d617f6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d5c150c50 0x7f1d5c164000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44928/0 (socket says 172.21.15.121:44928) 2023-11-15T02:56:51.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d617f6700 1 -- 172.21.15.121:0/4158331155 learned_addr learned my addr 172.21.15.121:0/4158331155 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:51.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d61ff7700 1 --2- 172.21.15.121:0/4158331155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d5c150870 0x7f1d5c168130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:51.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d627f8700 1 --2- 172.21.15.121:0/4158331155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d5c0a7ff0 0x7f1d5c0bfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:51.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d617f6700 1 -- 172.21.15.121:0/4158331155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d5c0a7ff0 msgr2=0x7f1d5c0bfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:51.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d617f6700 1 --2- 172.21.15.121:0/4158331155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d5c0a7ff0 0x7f1d5c0bfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:51.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d617f6700 1 -- 172.21.15.121:0/4158331155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d5c150870 msgr2=0x7f1d5c168130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:51.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d617f6700 1 --2- 172.21.15.121:0/4158331155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d5c150870 0x7f1d5c168130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:51.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d617f6700 1 -- 172.21.15.121:0/4158331155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4c009370 con 0x7f1d5c150c50 2023-11-15T02:56:51.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d617f6700 1 --2- 172.21.15.121:0/4158331155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d5c150c50 0x7f1d5c164000 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f1d4c009130 tx=0x7f1d4c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:51.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d51ffb700 1 -- 172.21.15.121:0/4158331155 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c00bea0 con 0x7f1d5c150c50 2023-11-15T02:56:51.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d51ffb700 1 -- 172.21.15.121:0/4158331155 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4c00b850 con 0x7f1d5c150c50 2023-11-15T02:56:51.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d6425b700 1 -- 172.21.15.121:0/4158331155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5c148210 con 0x7f1d5c150c50 2023-11-15T02:56:51.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d51ffb700 1 -- 172.21.15.121:0/4158331155 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c01d680 con 0x7f1d5c150c50 2023-11-15T02:56:51.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.904+0000 7f1d6425b700 1 -- 172.21.15.121:0/4158331155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d5c1487d0 con 0x7f1d5c150c50 2023-11-15T02:56:51.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.908+0000 7f1d6425b700 1 -- 172.21.15.121:0/4158331155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d5c06c150 con 0x7f1d5c150c50 2023-11-15T02:56:51.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.908+0000 7f1d51ffb700 1 -- 172.21.15.121:0/4158331155 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1d4c01da30 con 0x7f1d5c150c50 2023-11-15T02:56:51.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.912+0000 7f1d51ffb700 1 --2- 172.21.15.121:0/4158331155 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d4005a510 0x7f1d4005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:51.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.912+0000 7f1d51ffb700 1 -- 172.21.15.121:0/4158331155 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1d4c0763e0 con 0x7f1d5c150c50 2023-11-15T02:56:51.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.912+0000 7f1d61ff7700 1 --2- 172.21.15.121:0/4158331155 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d4005a510 0x7f1d4005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:51.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.912+0000 7f1d61ff7700 1 --2- 172.21.15.121:0/4158331155 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d4005a510 0x7f1d4005c9d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1d58009b90 tx=0x7f1d58009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:51.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:51.916+0000 7f1d51ffb700 1 -- 172.21.15.121:0/4158331155 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1d4c049b00 con 0x7f1d5c150c50 2023-11-15T02:56:52.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.076+0000 7f1d6425b700 1 -- 172.21.15.121:0/4158331155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d5c0c7440 con 0x7f1d5c150c50 2023-11-15T02:56:52.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.080+0000 7f1d51ffb700 1 -- 172.21.15.121:0/4158331155 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16989 (secure 0 0 0) 0x7f1d4c021020 con 0x7f1d5c150c50 2023-11-15T02:56:52.085 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:56:52.086 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":20,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:51.695398+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:56:49.726353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-15T02:56:50.688098+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:56:52.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.084+0000 7f1d6425b700 1 -- 172.21.15.121:0/4158331155 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d4005a510 msgr2=0x7f1d4005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:52.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.084+0000 7f1d6425b700 1 --2- 172.21.15.121:0/4158331155 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d4005a510 0x7f1d4005c9d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1d58009b90 tx=0x7f1d58009060).stop 2023-11-15T02:56:52.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.084+0000 7f1d6425b700 1 -- 172.21.15.121:0/4158331155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d5c150c50 msgr2=0x7f1d5c164000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:52.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.084+0000 7f1d6425b700 1 --2- 172.21.15.121:0/4158331155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d5c150c50 0x7f1d5c164000 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f1d4c009130 tx=0x7f1d4c017000).stop 2023-11-15T02:56:52.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.084+0000 7f1d6425b700 1 -- 172.21.15.121:0/4158331155 shutdown_connections 2023-11-15T02:56:52.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.084+0000 7f1d6425b700 1 --2- 172.21.15.121:0/4158331155 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d4005a510 0x7f1d4005c9d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:52.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.084+0000 7f1d6425b700 1 --2- 172.21.15.121:0/4158331155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d5c0a7ff0 0x7f1d5c0bfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:52.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.084+0000 7f1d6425b700 1 --2- 172.21.15.121:0/4158331155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d5c150c50 0x7f1d5c164000 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:52.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.084+0000 7f1d6425b700 1 --2- 172.21.15.121:0/4158331155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d5c150870 0x7f1d5c168130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:52.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.084+0000 7f1d6425b700 1 -- 172.21.15.121:0/4158331155 >> 172.21.15.121:0/4158331155 conn(0x7f1d5c062190 msgr2=0x7f1d5c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:52.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.084+0000 7f1d6425b700 1 -- 172.21.15.121:0/4158331155 shutdown_connections 2023-11-15T02:56:52.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.084+0000 7f1d6425b700 1 -- 172.21.15.121:0/4158331155 wait complete. 2023-11-15T02:56:52.110 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:56:52.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.267+0000 7f62d6747700 1 Processor -- start 2023-11-15T02:56:52.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.267+0000 7f62d6747700 1 -- start start 2023-11-15T02:56:52.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.271+0000 7f62d6747700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d0151870 0x7f62d0151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:52.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.271+0000 7f62d6747700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62d0152140 0x7f62d00a9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:52.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.271+0000 7f62d6747700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d00657f0 con 0x7f62d00a9d20 2023-11-15T02:56:52.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.271+0000 7f62d6747700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d0065ad0 con 0x7f62d0152140 2023-11-15T02:56:52.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.271+0000 7f62d6747700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00aa260 con 0x7f62d0151870 2023-11-15T02:56:52.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.271+0000 7f62d5745700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d0151870 0x7f62d0151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:52.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.271+0000 7f62d4f44700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62d0152140 0x7f62d00a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:52.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.271+0000 7f62d5745700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d0151870 0x7f62d0151c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:55434/0 (socket says 172.21.15.121:55434) 2023-11-15T02:56:52.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.271+0000 7f62d5745700 1 -- 172.21.15.121:0/2081554297 learned_addr learned my addr 172.21.15.121:0/2081554297 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:52.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.271+0000 7f62d5f46700 1 --1- 172.21.15.121:0/2081554297 >> v1:172.21.15.190:6789/0 conn(0x7f62d00a9d20 0x7f62d00c30e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:54642/0 (socket says 172.21.15.121:54642) 2023-11-15T02:56:52.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.271+0000 7f62c7fff700 1 -- 172.21.15.121:0/2081554297 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119997884 0 0) 0x7f62d00657f0 con 0x7f62d00a9d20 2023-11-15T02:56:52.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.271+0000 7f62c7fff700 1 -- 172.21.15.121:0/2081554297 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62bc0033f0 con 0x7f62d00a9d20 2023-11-15T02:56:52.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.271+0000 7f62d4f44700 1 -- 172.21.15.121:0/2081554297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d0151870 msgr2=0x7f62d0151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:52.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.271+0000 7f62d4f44700 1 --2- 172.21.15.121:0/2081554297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d0151870 0x7f62d0151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:52.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.271+0000 7f62d4f44700 1 -- 172.21.15.121:0/2081554297 >> v1:172.21.15.190:6789/0 conn(0x7f62d00a9d20 legacy=0x7f62d00c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:52.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.271+0000 7f62d4f44700 1 -- 172.21.15.121:0/2081554297 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d00aa540 con 0x7f62d0152140 2023-11-15T02:56:52.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.271+0000 7f62d5745700 1 --2- 172.21.15.121:0/2081554297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d0151870 0x7f62d0151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:56:52.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.271+0000 7f62d4f44700 1 --2- 172.21.15.121:0/2081554297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62d0152140 0x7f62d00a9830 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f62c0009760 tx=0x7f62c0009060).ready entity=mon.1 client_cookie=5f232bccff95f871 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:52.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.275+0000 7f62c7fff700 1 -- 172.21.15.121:0/2081554297 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c000d4a0 con 0x7f62d0152140 2023-11-15T02:56:52.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.275+0000 7f62c7fff700 1 -- 172.21.15.121:0/2081554297 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c000d600 con 0x7f62d0152140 2023-11-15T02:56:52.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.275+0000 7f62c7fff700 1 -- 172.21.15.121:0/2081554297 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c000d980 con 0x7f62d0152140 2023-11-15T02:56:52.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.275+0000 7f62d6747700 1 -- 172.21.15.121:0/2081554297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62d0152140 msgr2=0x7f62d00a9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:52.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.275+0000 7f62d6747700 1 --2- 172.21.15.121:0/2081554297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62d0152140 0x7f62d00a9830 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f62c0009760 tx=0x7f62c0009060).stop 2023-11-15T02:56:52.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.275+0000 7f62d6747700 1 -- 172.21.15.121:0/2081554297 shutdown_connections 2023-11-15T02:56:52.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.275+0000 7f62d6747700 1 --2- 172.21.15.121:0/2081554297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62d0152140 0x7f62d00a9830 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:52.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.275+0000 7f62d6747700 1 --2- 172.21.15.121:0/2081554297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d0151870 0x7f62d0151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:52.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.275+0000 7f62d6747700 1 -- 172.21.15.121:0/2081554297 >> 172.21.15.121:0/2081554297 conn(0x7f62d0062190 msgr2=0x7f62d0062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:52.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.275+0000 7f62d6747700 1 -- 172.21.15.121:0/2081554297 shutdown_connections 2023-11-15T02:56:52.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.275+0000 7f62d6747700 1 -- 172.21.15.121:0/2081554297 wait complete. 2023-11-15T02:56:52.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.279+0000 7f62d6747700 1 Processor -- start 2023-11-15T02:56:52.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.279+0000 7f62d6747700 1 -- start start 2023-11-15T02:56:52.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.279+0000 7f62d6747700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d00a9d20 0x7f62d00c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:52.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.279+0000 7f62d6747700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62d0151870 0x7f62d0144010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:52.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.279+0000 7f62d6747700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62d0152140 0x7f62d0146450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:52.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.279+0000 7f62d6747700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d014d3f0 con 0x7f62d0152140 2023-11-15T02:56:52.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.279+0000 7f62d6747700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d0148800 con 0x7f62d0151870 2023-11-15T02:56:52.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.279+0000 7f62d6747700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00c68c0 con 0x7f62d00a9d20 2023-11-15T02:56:52.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.279+0000 7f62d5f46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62d0152140 0x7f62d0146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:52.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.279+0000 7f62d5f46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62d0152140 0x7f62d0146450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44934/0 (socket says 172.21.15.121:44934) 2023-11-15T02:56:52.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.279+0000 7f62d5f46700 1 -- 172.21.15.121:0/1444540048 learned_addr learned my addr 172.21.15.121:0/1444540048 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:52.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.279+0000 7f62d5745700 1 --2- 172.21.15.121:0/1444540048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d00a9d20 0x7f62d00c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:52.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.279+0000 7f62d4f44700 1 --2- 172.21.15.121:0/1444540048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62d0151870 0x7f62d0144010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:52.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.283+0000 7f62d5745700 1 -- 172.21.15.121:0/1444540048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62d0151870 msgr2=0x7f62d0144010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:52.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.283+0000 7f62d5745700 1 --2- 172.21.15.121:0/1444540048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62d0151870 0x7f62d0144010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:52.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.283+0000 7f62d5745700 1 -- 172.21.15.121:0/1444540048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62d0152140 msgr2=0x7f62d0146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:52.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.283+0000 7f62d5745700 1 --2- 172.21.15.121:0/1444540048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62d0152140 0x7f62d0146450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:52.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.283+0000 7f62d5745700 1 -- 172.21.15.121:0/1444540048 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc009370 con 0x7f62d00a9d20 2023-11-15T02:56:52.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.283+0000 7f62d5f46700 1 --2- 172.21.15.121:0/1444540048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62d0152140 0x7f62d0146450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:56:52.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.283+0000 7f62d5745700 1 --2- 172.21.15.121:0/1444540048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d00a9d20 0x7f62d00c63d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f62cc00ea00 tx=0x7f62cc0090f0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:52.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.283+0000 7f62c57fa700 1 -- 172.21.15.121:0/1444540048 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc00c6a0 con 0x7f62d00a9d20 2023-11-15T02:56:52.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.283+0000 7f62c57fa700 1 -- 172.21.15.121:0/1444540048 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62cc00c800 con 0x7f62d00a9d20 2023-11-15T02:56:52.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.283+0000 7f62c57fa700 1 -- 172.21.15.121:0/1444540048 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc00cb80 con 0x7f62d00a9d20 2023-11-15T02:56:52.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.283+0000 7f62d6747700 1 -- 172.21.15.121:0/1444540048 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c00092a0 con 0x7f62d00a9d20 2023-11-15T02:56:52.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.283+0000 7f62d6747700 1 -- 172.21.15.121:0/1444540048 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d00c6e00 con 0x7f62d00a9d20 2023-11-15T02:56:52.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.283+0000 7f62d6747700 1 -- 172.21.15.121:0/1444540048 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d01485d0 con 0x7f62d00a9d20 2023-11-15T02:56:52.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.287+0000 7f62c57fa700 1 -- 172.21.15.121:0/1444540048 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f62cc01b400 con 0x7f62d00a9d20 2023-11-15T02:56:52.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.287+0000 7f62c57fa700 1 --2- 172.21.15.121:0/1444540048 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f62b405a540 0x7f62b405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:52.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.287+0000 7f62c57fa700 1 -- 172.21.15.121:0/1444540048 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f62cc074d00 con 0x7f62d00a9d20 2023-11-15T02:56:52.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.287+0000 7f62d4f44700 1 --2- 172.21.15.121:0/1444540048 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f62b405a540 0x7f62b405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:52.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.287+0000 7f62d4f44700 1 --2- 172.21.15.121:0/1444540048 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f62b405a540 0x7f62b405ca00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f62c0009760 tx=0x7f62c0016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:52.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.295+0000 7f62c57fa700 1 -- 172.21.15.121:0/1444540048 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f62d01485d0 con 0x7f62d00a9d20 2023-11-15T02:56:52.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.459+0000 7f62d6747700 1 -- 172.21.15.121:0/1444540048 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62d006bcf0 con 0x7f62d00a9d20 2023-11-15T02:56:52.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.463+0000 7f62c57fa700 1 -- 172.21.15.121:0/1444540048 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16989 (secure 0 0 0) 0x7f62cc0483c0 con 0x7f62d00a9d20 2023-11-15T02:56:52.469 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:56:52.470 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":20,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:51.695398+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:56:49.726353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-15T02:56:50.688098+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:56:52.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.467+0000 7f62d6747700 1 -- 172.21.15.121:0/1444540048 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f62b405a540 msgr2=0x7f62b405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:52.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.467+0000 7f62d6747700 1 --2- 172.21.15.121:0/1444540048 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f62b405a540 0x7f62b405ca00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f62c0009760 tx=0x7f62c0016000).stop 2023-11-15T02:56:52.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.467+0000 7f62d6747700 1 -- 172.21.15.121:0/1444540048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d00a9d20 msgr2=0x7f62d00c63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:52.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.467+0000 7f62d6747700 1 --2- 172.21.15.121:0/1444540048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d00a9d20 0x7f62d00c63d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f62cc00ea00 tx=0x7f62cc0090f0).stop 2023-11-15T02:56:52.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.467+0000 7f62d6747700 1 -- 172.21.15.121:0/1444540048 shutdown_connections 2023-11-15T02:56:52.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.467+0000 7f62d6747700 1 --2- 172.21.15.121:0/1444540048 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f62b405a540 0x7f62b405ca00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:52.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.467+0000 7f62d6747700 1 --2- 172.21.15.121:0/1444540048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62d0152140 0x7f62d0146450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:52.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.467+0000 7f62d6747700 1 --2- 172.21.15.121:0/1444540048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62d0151870 0x7f62d0144010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:52.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.467+0000 7f62d6747700 1 --2- 172.21.15.121:0/1444540048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d00a9d20 0x7f62d00c63d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:52.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.467+0000 7f62d6747700 1 -- 172.21.15.121:0/1444540048 >> 172.21.15.121:0/1444540048 conn(0x7f62d0062190 msgr2=0x7f62d00a8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:52.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.467+0000 7f62d6747700 1 -- 172.21.15.121:0/1444540048 shutdown_connections 2023-11-15T02:56:52.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.467+0000 7f62d6747700 1 -- 172.21.15.121:0/1444540048 wait complete. 2023-11-15T02:56:52.492 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-15T02:56:52.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.675+0000 7f9a4f25b700 1 Processor -- start 2023-11-15T02:56:52.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.675+0000 7f9a4f25b700 1 -- start start 2023-11-15T02:56:52.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.675+0000 7f9a4f25b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9a48065850 0x7f9a480a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:52.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.675+0000 7f9a4f25b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9a48068170 0x7f9a480bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:52.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.675+0000 7f9a4f25b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a48152150 con 0x7f9a480a8b80 2023-11-15T02:56:52.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.675+0000 7f9a4f25b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a48148f70 con 0x7f9a48065850 2023-11-15T02:56:52.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.675+0000 7f9a4f25b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a481523a0 con 0x7f9a48068170 2023-11-15T02:56:52.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.675+0000 7f9a47fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9a48065850 0x7f9a480a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:52.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.675+0000 7f9a47fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9a48065850 0x7f9a480a8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44938/0 (socket says 172.21.15.121:44938) 2023-11-15T02:56:52.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.675+0000 7f9a47fff700 1 -- 172.21.15.121:0/2118812092 learned_addr learned my addr 172.21.15.121:0/2118812092 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:52.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.675+0000 7f9a4d7f8700 1 --2- 172.21.15.121:0/2118812092 >> v2:172.21.15.190:3300/0 conn(0x7f9a48068170 0x7f9a480bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:52.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.675+0000 7f9a47fff700 1 -- 172.21.15.121:0/2118812092 >> v2:172.21.15.190:3300/0 conn(0x7f9a48068170 msgr2=0x7f9a480bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:52.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.675+0000 7f9a47fff700 1 --2- 172.21.15.121:0/2118812092 >> v2:172.21.15.190:3300/0 conn(0x7f9a48068170 0x7f9a480bbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:52.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.679+0000 7f9a47fff700 1 -- 172.21.15.121:0/2118812092 >> v1:172.21.15.190:6789/0 conn(0x7f9a480a8b80 legacy=0x7f9a48067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:52.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.679+0000 7f9a47fff700 1 -- 172.21.15.121:0/2118812092 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a48152680 con 0x7f9a48065850 2023-11-15T02:56:52.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.679+0000 7f9a47fff700 1 --2- 172.21.15.121:0/2118812092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a48065850 0x7f9a480a8f60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f9a48152060 tx=0x7f9a340090f0).ready entity=mon.0 client_cookie=48bf34a85f38c36d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:52.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.679+0000 7f9a477fe700 1 -- 172.21.15.121:0/2118812092 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3400b5a0 con 0x7f9a48065850 2023-11-15T02:56:52.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.679+0000 7f9a477fe700 1 -- 172.21.15.121:0/2118812092 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3400b700 con 0x7f9a48065850 2023-11-15T02:56:52.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.679+0000 7f9a477fe700 1 -- 172.21.15.121:0/2118812092 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3400ba80 con 0x7f9a48065850 2023-11-15T02:56:52.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.679+0000 7f9a4f25b700 1 -- 172.21.15.121:0/2118812092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a48065850 msgr2=0x7f9a480a8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:52.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.679+0000 7f9a4f25b700 1 --2- 172.21.15.121:0/2118812092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a48065850 0x7f9a480a8f60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f9a48152060 tx=0x7f9a340090f0).stop 2023-11-15T02:56:52.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.679+0000 7f9a4f25b700 1 -- 172.21.15.121:0/2118812092 shutdown_connections 2023-11-15T02:56:52.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.679+0000 7f9a4f25b700 1 --2- 172.21.15.121:0/2118812092 >> v2:172.21.15.190:3300/0 conn(0x7f9a48068170 0x7f9a480bbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:52.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.679+0000 7f9a4f25b700 1 --2- 172.21.15.121:0/2118812092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a48065850 0x7f9a480a8f60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:52.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.679+0000 7f9a4f25b700 1 -- 172.21.15.121:0/2118812092 >> 172.21.15.121:0/2118812092 conn(0x7f9a480591c0 msgr2=0x7f9a480c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:52.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.679+0000 7f9a4f25b700 1 -- 172.21.15.121:0/2118812092 shutdown_connections 2023-11-15T02:56:52.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.679+0000 7f9a4f25b700 1 -- 172.21.15.121:0/2118812092 wait complete. 2023-11-15T02:56:52.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.683+0000 7f9a4f25b700 1 Processor -- start 2023-11-15T02:56:52.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.683+0000 7f9a4f25b700 1 -- start start 2023-11-15T02:56:52.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.683+0000 7f9a4f25b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a48065850 0x7f9a48146470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:52.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.683+0000 7f9a4f25b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a48068170 0x7f9a48147800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:52.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.683+0000 7f9a4f25b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a480a8b80 0x7f9a480aade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:52.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.683+0000 7f9a4f25b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a48146960 con 0x7f9a48065850 2023-11-15T02:56:52.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.683+0000 7f9a4f25b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a480c1ec0 con 0x7f9a480a8b80 2023-11-15T02:56:52.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.683+0000 7f9a4f25b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a48146d20 con 0x7f9a48068170 2023-11-15T02:56:52.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.683+0000 7f9a4cff7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a48065850 0x7f9a48146470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:52.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.687+0000 7f9a4cff7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a48065850 0x7f9a48146470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44940/0 (socket says 172.21.15.121:44940) 2023-11-15T02:56:52.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.687+0000 7f9a4cff7700 1 -- 172.21.15.121:0/2436822371 learned_addr learned my addr 172.21.15.121:0/2436822371 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:52.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.687+0000 7f9a4d7f8700 1 --2- 172.21.15.121:0/2436822371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a480a8b80 0x7f9a480aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:52.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.687+0000 7f9a47fff700 1 --2- 172.21.15.121:0/2436822371 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a48068170 0x7f9a48147800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:52.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.687+0000 7f9a4cff7700 1 -- 172.21.15.121:0/2436822371 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a48068170 msgr2=0x7f9a48147800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:52.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.687+0000 7f9a4cff7700 1 --2- 172.21.15.121:0/2436822371 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a48068170 0x7f9a48147800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:52.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.687+0000 7f9a4cff7700 1 -- 172.21.15.121:0/2436822371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a480a8b80 msgr2=0x7f9a480aade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:52.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.687+0000 7f9a4cff7700 1 --2- 172.21.15.121:0/2436822371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a480a8b80 0x7f9a480aade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:52.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.687+0000 7f9a4cff7700 1 -- 172.21.15.121:0/2436822371 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a34009370 con 0x7f9a48065850 2023-11-15T02:56:52.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.687+0000 7f9a4cff7700 1 --2- 172.21.15.121:0/2436822371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a48065850 0x7f9a48146470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f9a3c00b460 tx=0x7f9a3c006f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:52.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.687+0000 7f9a44ff9700 1 -- 172.21.15.121:0/2436822371 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c012410 con 0x7f9a48065850 2023-11-15T02:56:52.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.687+0000 7f9a44ff9700 1 -- 172.21.15.121:0/2436822371 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3c012590 con 0x7f9a48065850 2023-11-15T02:56:52.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.687+0000 7f9a4f25b700 1 -- 172.21.15.121:0/2436822371 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a48147000 con 0x7f9a48065850 2023-11-15T02:56:52.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.687+0000 7f9a44ff9700 1 -- 172.21.15.121:0/2436822371 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c012950 con 0x7f9a48065850 2023-11-15T02:56:52.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.687+0000 7f9a4f25b700 1 -- 172.21.15.121:0/2436822371 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a48147cf0 con 0x7f9a48065850 2023-11-15T02:56:52.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.687+0000 7f9a4f25b700 1 -- 172.21.15.121:0/2436822371 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a48146b80 con 0x7f9a48065850 2023-11-15T02:56:52.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.691+0000 7f9a44ff9700 1 -- 172.21.15.121:0/2436822371 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9a3c012cc0 con 0x7f9a48065850 2023-11-15T02:56:52.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.691+0000 7f9a44ff9700 1 --2- 172.21.15.121:0/2436822371 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9a2c05a540 0x7f9a2c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:52.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.691+0000 7f9a44ff9700 1 -- 172.21.15.121:0/2436822371 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9a3c0744e0 con 0x7f9a48065850 2023-11-15T02:56:52.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.691+0000 7f9a47fff700 1 --2- 172.21.15.121:0/2436822371 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9a2c05a540 0x7f9a2c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:52.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.695+0000 7f9a47fff700 1 --2- 172.21.15.121:0/2436822371 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9a2c05a540 0x7f9a2c05ca00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f9a34009130 tx=0x7f9a34015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:52.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.699+0000 7f9a44ff9700 1 -- 172.21.15.121:0/2436822371 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9a48146b80 con 0x7f9a48065850 2023-11-15T02:56:52.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:52.911+0000 7f9a4f25b700 1 -- 172.21.15.121:0/2436822371 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9a48146b80 con 0x7f9a48065850 2023-11-15T02:56:53.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:53.839+0000 7f9a44ff9700 1 -- 172.21.15.121:0/2436822371 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9a48146b80 con 0x7f9a48065850 2023-11-15T02:56:53.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:53.843+0000 7f9a4f25b700 1 -- 172.21.15.121:0/2436822371 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9a2c05a540 msgr2=0x7f9a2c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:53.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:53.843+0000 7f9a4f25b700 1 --2- 172.21.15.121:0/2436822371 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9a2c05a540 0x7f9a2c05ca00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f9a34009130 tx=0x7f9a34015000).stop 2023-11-15T02:56:53.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:53.843+0000 7f9a4f25b700 1 -- 172.21.15.121:0/2436822371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a48065850 msgr2=0x7f9a48146470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:53.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:53.843+0000 7f9a4f25b700 1 --2- 172.21.15.121:0/2436822371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a48065850 0x7f9a48146470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f9a3c00b460 tx=0x7f9a3c006f20).stop 2023-11-15T02:56:53.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:53.843+0000 7f9a4f25b700 1 -- 172.21.15.121:0/2436822371 shutdown_connections 2023-11-15T02:56:53.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:53.843+0000 7f9a4f25b700 1 --2- 172.21.15.121:0/2436822371 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9a2c05a540 0x7f9a2c05ca00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:53.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:53.843+0000 7f9a4f25b700 1 --2- 172.21.15.121:0/2436822371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a480a8b80 0x7f9a480aade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:53.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:53.843+0000 7f9a4f25b700 1 --2- 172.21.15.121:0/2436822371 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a48068170 0x7f9a48147800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:53.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:53.843+0000 7f9a4f25b700 1 --2- 172.21.15.121:0/2436822371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a48065850 0x7f9a48146470 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:53.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:53.843+0000 7f9a4f25b700 1 -- 172.21.15.121:0/2436822371 >> 172.21.15.121:0/2436822371 conn(0x7f9a480591c0 msgr2=0x7f9a480c5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:53.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:53.843+0000 7f9a4f25b700 1 -- 172.21.15.121:0/2436822371 shutdown_connections 2023-11-15T02:56:53.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:53.843+0000 7f9a4f25b700 1 -- 172.21.15.121:0/2436822371 wait complete. 2023-11-15T02:56:53.870 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-15T02:56:53.870 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-15T02:56:54.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.035+0000 7f16713b9700 1 Processor -- start 2023-11-15T02:56:54.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.035+0000 7f16713b9700 1 -- start start 2023-11-15T02:56:54.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.035+0000 7f16713b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f166c151af0 0x7f166c0a7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:54.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.035+0000 7f16713b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f166c0a5720 0x7f166c0c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:54.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.035+0000 7f16713b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f166c0c38e0 con 0x7f166c0c3430 2023-11-15T02:56:54.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.035+0000 7f16713b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166c066d80 con 0x7f166c0a5720 2023-11-15T02:56:54.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.035+0000 7f16713b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166c1528b0 con 0x7f166c151af0 2023-11-15T02:56:54.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.035+0000 7f1670bb8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f166c0c3430 0x7f166c0c64a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:42386/0 (socket says 172.21.15.121:42386) 2023-11-15T02:56:54.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.035+0000 7f1670bb8700 1 -- 172.21.15.121:0/1800105633 learned_addr learned my addr 172.21.15.121:0/1800105633 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:54.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.035+0000 7f166bfff700 1 --2- 172.21.15.121:0/1800105633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f166c151af0 0x7f166c0a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:54.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.035+0000 7f166b7fe700 1 --2- 172.21.15.121:0/1800105633 >> v2:172.21.15.190:3300/0 conn(0x7f166c0a5720 0x7f166c0c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:54.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.039+0000 7f166affd700 1 -- 172.21.15.121:0/1800105633 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 58071988 0 0) 0x7f1664001490 con 0x7f166c0c3430 2023-11-15T02:56:54.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.039+0000 7f166affd700 1 -- 172.21.15.121:0/1800105633 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16540033f0 con 0x7f166c0c3430 2023-11-15T02:56:54.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.039+0000 7f166b7fe700 1 -- 172.21.15.121:0/1800105633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f166c151af0 msgr2=0x7f166c0a7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:54.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.039+0000 7f166b7fe700 1 --2- 172.21.15.121:0/1800105633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f166c151af0 0x7f166c0a7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:54.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.039+0000 7f166b7fe700 1 -- 172.21.15.121:0/1800105633 >> v1:172.21.15.121:6789/0 conn(0x7f166c0c3430 legacy=0x7f166c0c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:54.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.039+0000 7f166b7fe700 1 -- 172.21.15.121:0/1800105633 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c151ff0 con 0x7f166c0a5720 2023-11-15T02:56:54.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.039+0000 7f166bfff700 1 --2- 172.21.15.121:0/1800105633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f166c151af0 0x7f166c0a7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:56:54.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.039+0000 7f166b7fe700 1 --2- 172.21.15.121:0/1800105633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f166c0a5720 0x7f166c0c2f40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f1660009760 tx=0x7f1660009060).ready entity=mon.1 client_cookie=d4d6ce84f1c92c41 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:54.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.039+0000 7f166affd700 1 -- 172.21.15.121:0/1800105633 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166000d4a0 con 0x7f166c0a5720 2023-11-15T02:56:54.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.039+0000 7f166affd700 1 -- 172.21.15.121:0/1800105633 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166000d600 con 0x7f166c0a5720 2023-11-15T02:56:54.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.039+0000 7f166affd700 1 -- 172.21.15.121:0/1800105633 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166000d980 con 0x7f166c0a5720 2023-11-15T02:56:54.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.039+0000 7f16713b9700 1 -- 172.21.15.121:0/1800105633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f166c0a5720 msgr2=0x7f166c0c2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:54.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.039+0000 7f16713b9700 1 --2- 172.21.15.121:0/1800105633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f166c0a5720 0x7f166c0c2f40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f1660009760 tx=0x7f1660009060).stop 2023-11-15T02:56:54.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.039+0000 7f16713b9700 1 -- 172.21.15.121:0/1800105633 shutdown_connections 2023-11-15T02:56:54.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.039+0000 7f16713b9700 1 --2- 172.21.15.121:0/1800105633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f166c0a5720 0x7f166c0c2f40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:54.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.039+0000 7f16713b9700 1 --2- 172.21.15.121:0/1800105633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f166c151af0 0x7f166c0a7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:54.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.039+0000 7f16713b9700 1 -- 172.21.15.121:0/1800105633 >> 172.21.15.121:0/1800105633 conn(0x7f166c067a60 msgr2=0x7f166c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:54.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.039+0000 7f16713b9700 1 -- 172.21.15.121:0/1800105633 shutdown_connections 2023-11-15T02:56:54.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.039+0000 7f16713b9700 1 -- 172.21.15.121:0/1800105633 wait complete. 2023-11-15T02:56:54.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.043+0000 7f16713b9700 1 Processor -- start 2023-11-15T02:56:54.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.043+0000 7f16713b9700 1 -- start start 2023-11-15T02:56:54.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.043+0000 7f16713b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f166c0a5720 0x7f166c1391f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:54.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.043+0000 7f16713b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f166c0c3430 0x7f166c0bfb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:54.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.043+0000 7f16713b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f166c0c0050 0x7f166c0bc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:54.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.043+0000 7f16713b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c0be2d0 con 0x7f166c0c3430 2023-11-15T02:56:54.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.043+0000 7f16713b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c0c04d0 con 0x7f166c0a5720 2023-11-15T02:56:54.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.043+0000 7f16713b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166c0bd9a0 con 0x7f166c0c0050 2023-11-15T02:56:54.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.047+0000 7f166b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f166c0c3430 0x7f166c0bfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:54.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.047+0000 7f166b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f166c0c3430 0x7f166c0bfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44944/0 (socket says 172.21.15.121:44944) 2023-11-15T02:56:54.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.047+0000 7f166bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f166c0a5720 0x7f166c1391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:54.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.047+0000 7f166b7fe700 1 -- 172.21.15.121:0/2640513100 learned_addr learned my addr 172.21.15.121:0/2640513100 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:54.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.047+0000 7f1670bb8700 1 --2- 172.21.15.121:0/2640513100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f166c0c0050 0x7f166c0bc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:54.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.047+0000 7f166bfff700 1 -- 172.21.15.121:0/2640513100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f166c0c0050 msgr2=0x7f166c0bc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:54.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.047+0000 7f166bfff700 1 --2- 172.21.15.121:0/2640513100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f166c0c0050 0x7f166c0bc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:54.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.047+0000 7f166bfff700 1 -- 172.21.15.121:0/2640513100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f166c0c3430 msgr2=0x7f166c0bfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:54.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.047+0000 7f166bfff700 1 --2- 172.21.15.121:0/2640513100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f166c0c3430 0x7f166c0bfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:54.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.047+0000 7f166bfff700 1 -- 172.21.15.121:0/2640513100 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165c0092a0 con 0x7f166c0a5720 2023-11-15T02:56:54.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.047+0000 7f166b7fe700 1 --2- 172.21.15.121:0/2640513100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f166c0c3430 0x7f166c0bfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:56:54.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.047+0000 7f166bfff700 1 --2- 172.21.15.121:0/2640513100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f166c0a5720 0x7f166c1391f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f165c00e980 tx=0x7f165c009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:54.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.047+0000 7f1653fff700 1 -- 172.21.15.121:0/2640513100 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c012600 con 0x7f166c0a5720 2023-11-15T02:56:54.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.047+0000 7f1653fff700 1 -- 172.21.15.121:0/2640513100 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165c012760 con 0x7f166c0a5720 2023-11-15T02:56:54.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.047+0000 7f16713b9700 1 -- 172.21.15.121:0/2640513100 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16600092a0 con 0x7f166c0a5720 2023-11-15T02:56:54.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.047+0000 7f1653fff700 1 -- 172.21.15.121:0/2640513100 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c012ae0 con 0x7f166c0a5720 2023-11-15T02:56:54.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.047+0000 7f16713b9700 1 -- 172.21.15.121:0/2640513100 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f166c0bdf60 con 0x7f166c0a5720 2023-11-15T02:56:54.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.047+0000 7f16713b9700 1 -- 172.21.15.121:0/2640513100 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f166c06bcf0 con 0x7f166c0a5720 2023-11-15T02:56:54.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.051+0000 7f1653fff700 1 -- 172.21.15.121:0/2640513100 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f165c01b400 con 0x7f166c0a5720 2023-11-15T02:56:54.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.051+0000 7f1653fff700 1 --2- 172.21.15.121:0/2640513100 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f165805a4e0 0x7f165805c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:54.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.051+0000 7f1653fff700 1 -- 172.21.15.121:0/2640513100 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f165c075140 con 0x7f166c0a5720 2023-11-15T02:56:54.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.051+0000 7f166b7fe700 1 --2- 172.21.15.121:0/2640513100 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f165805a4e0 0x7f165805c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:54.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.051+0000 7f166b7fe700 1 --2- 172.21.15.121:0/2640513100 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f165805a4e0 0x7f165805c9a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f166000d260 tx=0x7f1660016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:54.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.063+0000 7f1653fff700 1 -- 172.21.15.121:0/2640513100 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f165c0488e0 con 0x7f166c0a5720 2023-11-15T02:56:54.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.267+0000 7f16713b9700 1 -- 172.21.15.121:0/2640513100 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f166c068a70 con 0x7f166c0a5720 2023-11-15T02:56:54.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.851+0000 7f1653fff700 1 -- 172.21.15.121:0/2640513100 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f165c0160c0 con 0x7f166c0a5720 2023-11-15T02:56:54.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.851+0000 7f16713b9700 1 -- 172.21.15.121:0/2640513100 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f165805a4e0 msgr2=0x7f165805c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:54.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.851+0000 7f16713b9700 1 --2- 172.21.15.121:0/2640513100 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f165805a4e0 0x7f165805c9a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f166000d260 tx=0x7f1660016000).stop 2023-11-15T02:56:54.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.851+0000 7f16713b9700 1 -- 172.21.15.121:0/2640513100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f166c0a5720 msgr2=0x7f166c1391f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:54.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.851+0000 7f16713b9700 1 --2- 172.21.15.121:0/2640513100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f166c0a5720 0x7f166c1391f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f165c00e980 tx=0x7f165c009060).stop 2023-11-15T02:56:54.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.855+0000 7f16713b9700 1 -- 172.21.15.121:0/2640513100 shutdown_connections 2023-11-15T02:56:54.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.855+0000 7f16713b9700 1 --2- 172.21.15.121:0/2640513100 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f165805a4e0 0x7f165805c9a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:54.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.855+0000 7f16713b9700 1 --2- 172.21.15.121:0/2640513100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f166c0c0050 0x7f166c0bc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:54.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.855+0000 7f16713b9700 1 --2- 172.21.15.121:0/2640513100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f166c0c3430 0x7f166c0bfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:54.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.855+0000 7f16713b9700 1 --2- 172.21.15.121:0/2640513100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f166c0a5720 0x7f166c1391f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:54.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.855+0000 7f16713b9700 1 -- 172.21.15.121:0/2640513100 >> 172.21.15.121:0/2640513100 conn(0x7f166c067a60 msgr2=0x7f166c062190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:54.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.855+0000 7f16713b9700 1 -- 172.21.15.121:0/2640513100 shutdown_connections 2023-11-15T02:56:54.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:54.855+0000 7f16713b9700 1 -- 172.21.15.121:0/2640513100 wait complete. 2023-11-15T02:56:54.881 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-15T02:56:54.881 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:56:55.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.035+0000 7fb8b64db700 1 Processor -- start 2023-11-15T02:56:55.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.035+0000 7fb8b64db700 1 -- start start 2023-11-15T02:56:55.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.039+0000 7fb8b64db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8b014af80 0x7fb8b00c1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:55.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.039+0000 7fb8b64db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb8b00c1bf0 0x7fb8b0150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:55.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.039+0000 7fb8b64db700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b0152000 con 0x7fb8b01514e0 2023-11-15T02:56:55.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.039+0000 7fb8b64db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b00c1270 con 0x7fb8b00c1bf0 2023-11-15T02:56:55.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.039+0000 7fb8b64db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b0151a40 con 0x7fb8b014af80 2023-11-15T02:56:55.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.039+0000 7fb8a77fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb8b00c1bf0 0x7fb8b0150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:55.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.039+0000 7fb8affff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8b014af80 0x7fb8b00c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:55.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.039+0000 7fb8a77fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb8b00c1bf0 0x7fb8b0150ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:43130/0 (socket says 172.21.15.121:43130) 2023-11-15T02:56:55.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.039+0000 7fb8a77fe700 1 -- 172.21.15.121:0/1628139502 learned_addr learned my addr 172.21.15.121:0/1628139502 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:55.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.039+0000 7fb8a77fe700 1 -- 172.21.15.121:0/1628139502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8b014af80 msgr2=0x7fb8b00c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:55.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.039+0000 7fb8a77fe700 1 --2- 172.21.15.121:0/1628139502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8b014af80 0x7fb8b00c1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.039+0000 7fb8a77fe700 1 -- 172.21.15.121:0/1628139502 >> v1:172.21.15.190:6789/0 conn(0x7fb8b01514e0 legacy=0x7fb8b00c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:55.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.039+0000 7fb8a77fe700 1 -- 172.21.15.121:0/1628139502 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b0151d20 con 0x7fb8b00c1bf0 2023-11-15T02:56:55.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.039+0000 7fb8af7fe700 1 -- 172.21.15.121:0/1628139502 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487159096 0 0) 0x7fb8a8001490 con 0x7fb8b01514e0 2023-11-15T02:56:55.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.039+0000 7fb8a77fe700 1 --2- 172.21.15.121:0/1628139502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b00c1bf0 0x7fb8b0150ff0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fb898006fd0 tx=0x7fb8980090f0).ready entity=mon.1 client_cookie=481d6819e4c6b616 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:55.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.043+0000 7fb8af7fe700 1 -- 172.21.15.121:0/1628139502 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89800b580 con 0x7fb8b00c1bf0 2023-11-15T02:56:55.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.043+0000 7fb8af7fe700 1 -- 172.21.15.121:0/1628139502 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89800b6e0 con 0x7fb8b00c1bf0 2023-11-15T02:56:55.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.043+0000 7fb8af7fe700 1 -- 172.21.15.121:0/1628139502 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89800ba60 con 0x7fb8b00c1bf0 2023-11-15T02:56:55.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.043+0000 7fb8b64db700 1 -- 172.21.15.121:0/1628139502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b00c1bf0 msgr2=0x7fb8b0150ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:55.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.043+0000 7fb8b64db700 1 --2- 172.21.15.121:0/1628139502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b00c1bf0 0x7fb8b0150ff0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fb898006fd0 tx=0x7fb8980090f0).stop 2023-11-15T02:56:55.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.043+0000 7fb8b64db700 1 -- 172.21.15.121:0/1628139502 shutdown_connections 2023-11-15T02:56:55.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.043+0000 7fb8b64db700 1 --2- 172.21.15.121:0/1628139502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b00c1bf0 0x7fb8b0150ff0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.043+0000 7fb8b64db700 1 --2- 172.21.15.121:0/1628139502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8b014af80 0x7fb8b00c1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.043+0000 7fb8b64db700 1 -- 172.21.15.121:0/1628139502 >> 172.21.15.121:0/1628139502 conn(0x7fb8b0062190 msgr2=0x7fb8b00625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:55.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.043+0000 7fb8b64db700 1 -- 172.21.15.121:0/1628139502 shutdown_connections 2023-11-15T02:56:55.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.043+0000 7fb8b64db700 1 -- 172.21.15.121:0/1628139502 wait complete. 2023-11-15T02:56:55.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.047+0000 7fb8b64db700 1 Processor -- start 2023-11-15T02:56:55.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.047+0000 7fb8b64db700 1 -- start start 2023-11-15T02:56:55.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.047+0000 7fb8b64db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8b00c1bf0 0x7fb8b0155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:55.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.047+0000 7fb8b64db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b014af80 0x7fb8b014bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:55.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.047+0000 7fb8b64db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8b01514e0 0x7fb8b014a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:55.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.047+0000 7fb8b64db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b0155840 con 0x7fb8b01514e0 2023-11-15T02:56:55.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.047+0000 7fb8b64db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b014ceb0 con 0x7fb8b014af80 2023-11-15T02:56:55.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.047+0000 7fb8b64db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b014d750 con 0x7fb8b00c1bf0 2023-11-15T02:56:55.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.047+0000 7fb8b4a78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8b01514e0 0x7fb8b014a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:55.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.047+0000 7fb8b4a78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8b01514e0 0x7fb8b014a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32924/0 (socket says 172.21.15.121:32924) 2023-11-15T02:56:55.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.047+0000 7fb8b4a78700 1 -- 172.21.15.121:0/1874311492 learned_addr learned my addr 172.21.15.121:0/1874311492 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:55.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.051+0000 7fb8affff700 1 --2- 172.21.15.121:0/1874311492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8b00c1bf0 0x7fb8b0155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:55.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.051+0000 7fb8a77fe700 1 --2- 172.21.15.121:0/1874311492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b014af80 0x7fb8b014bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:55.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.051+0000 7fb8b4a78700 1 -- 172.21.15.121:0/1874311492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8b00c1bf0 msgr2=0x7fb8b0155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:55.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.051+0000 7fb8b4a78700 1 --2- 172.21.15.121:0/1874311492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8b00c1bf0 0x7fb8b0155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.051+0000 7fb8b4a78700 1 -- 172.21.15.121:0/1874311492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b014af80 msgr2=0x7fb8b014bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:55.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.051+0000 7fb8b4a78700 1 --2- 172.21.15.121:0/1874311492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b014af80 0x7fb8b014bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.051+0000 7fb8b4a78700 1 -- 172.21.15.121:0/1874311492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb898009370 con 0x7fb8b01514e0 2023-11-15T02:56:55.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.051+0000 7fb8b4a78700 1 --2- 172.21.15.121:0/1874311492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8b01514e0 0x7fb8b014a210 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fb8a800c7e0 tx=0x7fb8a80014a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:55.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.051+0000 7fb8acff9700 1 -- 172.21.15.121:0/1874311492 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8010510 con 0x7fb8b01514e0 2023-11-15T02:56:55.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.051+0000 7fb8acff9700 1 -- 172.21.15.121:0/1874311492 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a8010670 con 0x7fb8b01514e0 2023-11-15T02:56:55.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.051+0000 7fb8acff9700 1 -- 172.21.15.121:0/1874311492 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8010a30 con 0x7fb8b01514e0 2023-11-15T02:56:55.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.051+0000 7fb8b64db700 1 -- 172.21.15.121:0/1874311492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b014c290 con 0x7fb8b01514e0 2023-11-15T02:56:55.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.051+0000 7fb8b64db700 1 -- 172.21.15.121:0/1874311492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b014d100 con 0x7fb8b01514e0 2023-11-15T02:56:55.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.051+0000 7fb8b64db700 1 -- 172.21.15.121:0/1874311492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b006c150 con 0x7fb8b01514e0 2023-11-15T02:56:55.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.055+0000 7fb8acff9700 1 -- 172.21.15.121:0/1874311492 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb8a8012440 con 0x7fb8b01514e0 2023-11-15T02:56:55.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.055+0000 7fb8acff9700 1 --2- 172.21.15.121:0/1874311492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb89005a510 0x7fb89005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:55.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.055+0000 7fb8acff9700 1 -- 172.21.15.121:0/1874311492 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb8a80742a0 con 0x7fb8b01514e0 2023-11-15T02:56:55.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.055+0000 7fb8affff700 1 --2- 172.21.15.121:0/1874311492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb89005a510 0x7fb89005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:55.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.055+0000 7fb8affff700 1 --2- 172.21.15.121:0/1874311492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb89005a510 0x7fb89005c9d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fb8a0009bc0 tx=0x7fb8a0008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:55.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.059+0000 7fb8acff9700 1 -- 172.21.15.121:0/1874311492 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb8a8047960 con 0x7fb8b01514e0 2023-11-15T02:56:55.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.255+0000 7fb8b64db700 1 -- 172.21.15.121:0/1874311492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8b00c0f00 con 0x7fb8b01514e0 2023-11-15T02:56:55.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.255+0000 7fb8acff9700 1 -- 172.21.15.121:0/1874311492 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4489 (secure 0 0 0) 0x7fb8a804af80 con 0x7fb8b01514e0 2023-11-15T02:56:55.263 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:56:55.263 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/802010010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":802010010},{"type":"v1","addr":"172.21.15.190:6835","nonce":802010010}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3991611070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3991611070},{"type":"v1","addr":"172.21.15.121:6836","nonce":3991611070}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3514688785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3514688785},{"type":"v1","addr":"172.21.15.121:6837","nonce":3514688785}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:56:51.695751+0000","modified":"2023-11-15T02:56:54.838464+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6833/742721767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":742721767},{"type":"v1","addr":"172.21.15.190:6833","nonce":742721767}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-15T02:56:55.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.259+0000 7fb8b64db700 1 -- 172.21.15.121:0/1874311492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb89005a510 msgr2=0x7fb89005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:55.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.259+0000 7fb8b64db700 1 --2- 172.21.15.121:0/1874311492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb89005a510 0x7fb89005c9d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fb8a0009bc0 tx=0x7fb8a0008040).stop 2023-11-15T02:56:55.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.259+0000 7fb8b64db700 1 -- 172.21.15.121:0/1874311492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8b01514e0 msgr2=0x7fb8b014a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:55.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.259+0000 7fb8b64db700 1 --2- 172.21.15.121:0/1874311492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8b01514e0 0x7fb8b014a210 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fb8a800c7e0 tx=0x7fb8a80014a0).stop 2023-11-15T02:56:55.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.259+0000 7fb8b64db700 1 -- 172.21.15.121:0/1874311492 shutdown_connections 2023-11-15T02:56:55.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.259+0000 7fb8b64db700 1 --2- 172.21.15.121:0/1874311492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb89005a510 0x7fb89005c9d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.259+0000 7fb8b64db700 1 --2- 172.21.15.121:0/1874311492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8b01514e0 0x7fb8b014a210 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.259+0000 7fb8b64db700 1 --2- 172.21.15.121:0/1874311492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b014af80 0x7fb8b014bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.259+0000 7fb8b64db700 1 --2- 172.21.15.121:0/1874311492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8b00c1bf0 0x7fb8b0155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.259+0000 7fb8b64db700 1 -- 172.21.15.121:0/1874311492 >> 172.21.15.121:0/1874311492 conn(0x7fb8b0062190 msgr2=0x7fb8b00a8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:55.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.259+0000 7fb8b64db700 1 -- 172.21.15.121:0/1874311492 shutdown_connections 2023-11-15T02:56:55.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.259+0000 7fb8b64db700 1 -- 172.21.15.121:0/1874311492 wait complete. 2023-11-15T02:56:55.268 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 8 2023-11-15T02:56:55.284 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:56:55.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.439+0000 7fc481c79700 1 Processor -- start 2023-11-15T02:56:55.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.443+0000 7fc481c79700 1 -- start start 2023-11-15T02:56:55.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.443+0000 7fc481c79700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc47c0cabd0 0x7fc47c0cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:55.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.443+0000 7fc481c79700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc47c0ad0f0 0x7fc47c0ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:55.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.443+0000 7fc481c79700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc47c0ad650 con 0x7fc47c0acc90 2023-11-15T02:56:55.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.443+0000 7fc481c79700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc47c065fc0 con 0x7fc47c0cabd0 2023-11-15T02:56:55.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.443+0000 7fc481c79700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc47c0662a0 con 0x7fc47c0ad0f0 2023-11-15T02:56:55.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.443+0000 7fc47bfff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc47c0acc90 0x7fc47c066d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:57696/0 (socket says 172.21.15.121:57696) 2023-11-15T02:56:55.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.443+0000 7fc47bfff700 1 -- 172.21.15.121:0/1289393375 learned_addr learned my addr 172.21.15.121:0/1289393375 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:55.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.443+0000 7fc480c77700 1 --2- 172.21.15.121:0/1289393375 >> v2:172.21.15.121:3300/0 conn(0x7fc47c0cabd0 0x7fc47c0cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:55.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.447+0000 7fc481478700 1 --2- 172.21.15.121:0/1289393375 >> v2:172.21.15.190:3300/0 conn(0x7fc47c0ad0f0 0x7fc47c0ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:55.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.447+0000 7fc47b7fe700 1 -- 172.21.15.121:0/1289393375 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968948251 0 0) 0x7fc47c0ad650 con 0x7fc47c0acc90 2023-11-15T02:56:55.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.447+0000 7fc47b7fe700 1 -- 172.21.15.121:0/1289393375 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4640033f0 con 0x7fc47c0acc90 2023-11-15T02:56:55.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.447+0000 7fc480c77700 1 -- 172.21.15.121:0/1289393375 >> v2:172.21.15.190:3300/0 conn(0x7fc47c0ad0f0 msgr2=0x7fc47c0ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:55.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.447+0000 7fc480c77700 1 --2- 172.21.15.121:0/1289393375 >> v2:172.21.15.190:3300/0 conn(0x7fc47c0ad0f0 0x7fc47c0ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.447+0000 7fc480c77700 1 -- 172.21.15.121:0/1289393375 >> v1:172.21.15.121:6789/0 conn(0x7fc47c0acc90 legacy=0x7fc47c066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:55.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.447+0000 7fc480c77700 1 -- 172.21.15.121:0/1289393375 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c066580 con 0x7fc47c0cabd0 2023-11-15T02:56:55.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.447+0000 7fc480c77700 1 --2- 172.21.15.121:0/1289393375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47c0cabd0 0x7fc47c0cafb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fc47c0c7000 tx=0x7fc46c009060).ready entity=mon.0 client_cookie=68d48ee41291adae server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:55.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.447+0000 7fc47b7fe700 1 -- 172.21.15.121:0/1289393375 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c00f510 con 0x7fc47c0cabd0 2023-11-15T02:56:55.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.447+0000 7fc47b7fe700 1 -- 172.21.15.121:0/1289393375 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46c00f670 con 0x7fc47c0cabd0 2023-11-15T02:56:55.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.447+0000 7fc47b7fe700 1 -- 172.21.15.121:0/1289393375 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c00f9f0 con 0x7fc47c0cabd0 2023-11-15T02:56:55.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.447+0000 7fc481c79700 1 -- 172.21.15.121:0/1289393375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47c0cabd0 msgr2=0x7fc47c0cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:55.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.447+0000 7fc481c79700 1 --2- 172.21.15.121:0/1289393375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47c0cabd0 0x7fc47c0cafb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fc47c0c7000 tx=0x7fc46c009060).stop 2023-11-15T02:56:55.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.447+0000 7fc481c79700 1 -- 172.21.15.121:0/1289393375 shutdown_connections 2023-11-15T02:56:55.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.447+0000 7fc481c79700 1 --2- 172.21.15.121:0/1289393375 >> v2:172.21.15.190:3300/0 conn(0x7fc47c0ad0f0 0x7fc47c0ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.447+0000 7fc481c79700 1 --2- 172.21.15.121:0/1289393375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47c0cabd0 0x7fc47c0cafb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.447+0000 7fc481c79700 1 -- 172.21.15.121:0/1289393375 >> 172.21.15.121:0/1289393375 conn(0x7fc47c062190 msgr2=0x7fc47c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:55.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.447+0000 7fc481c79700 1 -- 172.21.15.121:0/1289393375 shutdown_connections 2023-11-15T02:56:55.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.447+0000 7fc481c79700 1 -- 172.21.15.121:0/1289393375 wait complete. 2023-11-15T02:56:55.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc481c79700 1 Processor -- start 2023-11-15T02:56:55.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc481c79700 1 -- start start 2023-11-15T02:56:55.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc481c79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc47c0acc90 0x7fc47c0bfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:55.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc481c79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47c0ad0f0 0x7fc47c166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:55.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc481c79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc47c0cabd0 0x7fc47c167910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:55.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc481c79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c158f40 con 0x7fc47c0ad0f0 2023-11-15T02:56:55.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc481c79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c158050 con 0x7fc47c0acc90 2023-11-15T02:56:55.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc481c79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c158330 con 0x7fc47c0cabd0 2023-11-15T02:56:55.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc47bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47c0ad0f0 0x7fc47c166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:55.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc47bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47c0ad0f0 0x7fc47c166ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32936/0 (socket says 172.21.15.121:32936) 2023-11-15T02:56:55.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc47bfff700 1 -- 172.21.15.121:0/3873063862 learned_addr learned my addr 172.21.15.121:0/3873063862 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:55.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc481478700 1 --2- 172.21.15.121:0/3873063862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc47c0cabd0 0x7fc47c167910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:55.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc480c77700 1 --2- 172.21.15.121:0/3873063862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc47c0acc90 0x7fc47c0bfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:55.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc47bfff700 1 -- 172.21.15.121:0/3873063862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc47c0cabd0 msgr2=0x7fc47c167910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:55.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc47bfff700 1 --2- 172.21.15.121:0/3873063862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc47c0cabd0 0x7fc47c167910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc47bfff700 1 -- 172.21.15.121:0/3873063862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc47c0acc90 msgr2=0x7fc47c0bfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:55.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc47bfff700 1 --2- 172.21.15.121:0/3873063862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc47c0acc90 0x7fc47c0bfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc47bfff700 1 -- 172.21.15.121:0/3873063862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46c0092a0 con 0x7fc47c0ad0f0 2023-11-15T02:56:55.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc480c77700 1 --2- 172.21.15.121:0/3873063862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc47c0acc90 0x7fc47c0bfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:56:55.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.451+0000 7fc47bfff700 1 --2- 172.21.15.121:0/3873063862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47c0ad0f0 0x7fc47c166ff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fc47000c590 tx=0x7fc470007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:55.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.455+0000 7fc478ff9700 1 -- 172.21.15.121:0/3873063862 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470007e40 con 0x7fc47c0ad0f0 2023-11-15T02:56:55.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.455+0000 7fc478ff9700 1 -- 172.21.15.121:0/3873063862 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4700072e0 con 0x7fc47c0ad0f0 2023-11-15T02:56:55.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.455+0000 7fc481c79700 1 -- 172.21.15.121:0/3873063862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47c158610 con 0x7fc47c0ad0f0 2023-11-15T02:56:55.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.455+0000 7fc478ff9700 1 -- 172.21.15.121:0/3873063862 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4700057a0 con 0x7fc47c0ad0f0 2023-11-15T02:56:55.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.455+0000 7fc481c79700 1 -- 172.21.15.121:0/3873063862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47c158bd0 con 0x7fc47c0ad0f0 2023-11-15T02:56:55.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.455+0000 7fc481c79700 1 -- 172.21.15.121:0/3873063862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc47c06bcf0 con 0x7fc47c0ad0f0 2023-11-15T02:56:55.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.455+0000 7fc478ff9700 1 -- 172.21.15.121:0/3873063862 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc470005bd0 con 0x7fc47c0ad0f0 2023-11-15T02:56:55.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.459+0000 7fc478ff9700 1 --2- 172.21.15.121:0/3873063862 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc46805a510 0x7fc46805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:55.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.459+0000 7fc478ff9700 1 -- 172.21.15.121:0/3873063862 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc470073d60 con 0x7fc47c0ad0f0 2023-11-15T02:56:55.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.459+0000 7fc480c77700 1 --2- 172.21.15.121:0/3873063862 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc46805a510 0x7fc46805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:55.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.459+0000 7fc480c77700 1 --2- 172.21.15.121:0/3873063862 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc46805a510 0x7fc46805c9d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fc46c00ffd0 tx=0x7fc46c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:55.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.463+0000 7fc478ff9700 1 -- 172.21.15.121:0/3873063862 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc470047540 con 0x7fc47c0ad0f0 2023-11-15T02:56:55.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.631+0000 7fc481c79700 1 -- 172.21.15.121:0/3873063862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc47c159080 con 0x7fc47c0ad0f0 2023-11-15T02:56:55.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.631+0000 7fc478ff9700 1 -- 172.21.15.121:0/3873063862 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fc47c159080 con 0x7fc47c0ad0f0 2023-11-15T02:56:55.639 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:56:55.639 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":21,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:52.702668+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:56:49.726353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-15T02:56:50.688098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:56:55.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.635+0000 7fc481c79700 1 -- 172.21.15.121:0/3873063862 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc46805a510 msgr2=0x7fc46805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:55.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.635+0000 7fc481c79700 1 --2- 172.21.15.121:0/3873063862 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc46805a510 0x7fc46805c9d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fc46c00ffd0 tx=0x7fc46c017000).stop 2023-11-15T02:56:55.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.635+0000 7fc481c79700 1 -- 172.21.15.121:0/3873063862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47c0ad0f0 msgr2=0x7fc47c166ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:55.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.635+0000 7fc481c79700 1 --2- 172.21.15.121:0/3873063862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47c0ad0f0 0x7fc47c166ff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fc47000c590 tx=0x7fc470007c80).stop 2023-11-15T02:56:55.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.635+0000 7fc481c79700 1 -- 172.21.15.121:0/3873063862 shutdown_connections 2023-11-15T02:56:55.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.635+0000 7fc481c79700 1 --2- 172.21.15.121:0/3873063862 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc46805a510 0x7fc46805c9d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.635+0000 7fc481c79700 1 --2- 172.21.15.121:0/3873063862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc47c0cabd0 0x7fc47c167910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.635+0000 7fc481c79700 1 --2- 172.21.15.121:0/3873063862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47c0ad0f0 0x7fc47c166ff0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.635+0000 7fc481c79700 1 --2- 172.21.15.121:0/3873063862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc47c0acc90 0x7fc47c0bfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.635+0000 7fc481c79700 1 -- 172.21.15.121:0/3873063862 >> 172.21.15.121:0/3873063862 conn(0x7fc47c062190 msgr2=0x7fc47c150590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:55.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.635+0000 7fc481c79700 1 -- 172.21.15.121:0/3873063862 shutdown_connections 2023-11-15T02:56:55.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.635+0000 7fc481c79700 1 -- 172.21.15.121:0/3873063862 wait complete. 2023-11-15T02:56:55.663 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-15T02:56:55.663 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T02:56:55.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.819+0000 7f13c745b700 1 Processor -- start 2023-11-15T02:56:55.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.819+0000 7f13c745b700 1 -- start start 2023-11-15T02:56:55.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.823+0000 7f13c745b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13c00aa240 0x7f13c0067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:55.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.823+0000 7f13c745b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c00cb5c0 con 0x7f13c0153bb0 2023-11-15T02:56:55.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.823+0000 7f13c745b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c00683e0 con 0x7f13c0067f50 2023-11-15T02:56:55.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.823+0000 7f13c745b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c00aa6f0 con 0x7f13c00aa240 2023-11-15T02:56:55.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.823+0000 7f13c51f7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f13c0153bb0 0x7f13c00a9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:57698/0 (socket says 172.21.15.121:57698) 2023-11-15T02:56:55.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.823+0000 7f13c51f7700 1 -- 172.21.15.121:0/3717889159 learned_addr learned my addr 172.21.15.121:0/3717889159 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:55.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.823+0000 7f13c49f6700 1 --2- 172.21.15.121:0/3717889159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13c00aa240 0x7f13c0067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:55.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.823+0000 7f13b7fff700 1 -- 172.21.15.121:0/3717889159 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656922496 0 0) 0x7f13c00cb5c0 con 0x7f13c0153bb0 2023-11-15T02:56:55.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.823+0000 7f13b7fff700 1 -- 172.21.15.121:0/3717889159 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13ac0033f0 con 0x7f13c0153bb0 2023-11-15T02:56:55.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.827+0000 7f13c49f6700 1 -- 172.21.15.121:0/3717889159 >> v1:172.21.15.190:6789/0 conn(0x7f13c0067f50 legacy=0x7f13c0065d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:55.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.827+0000 7f13c49f6700 1 -- 172.21.15.121:0/3717889159 >> v1:172.21.15.121:6789/0 conn(0x7f13c0153bb0 legacy=0x7f13c00a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:55.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.827+0000 7f13c49f6700 1 -- 172.21.15.121:0/3717889159 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c00c9ca0 con 0x7f13c00aa240 2023-11-15T02:56:55.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.827+0000 7f13c49f6700 1 --2- 172.21.15.121:0/3717889159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13c00aa240 0x7f13c0067a60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f13c0068570 tx=0x7f13b0009060).ready entity=mon.2 client_cookie=27b416a7a1a6eef1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:55.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.827+0000 7f13b7fff700 1 -- 172.21.15.121:0/3717889159 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b000f510 con 0x7f13c00aa240 2023-11-15T02:56:55.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.827+0000 7f13b7fff700 1 -- 172.21.15.121:0/3717889159 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b000f670 con 0x7f13c00aa240 2023-11-15T02:56:55.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.827+0000 7f13b7fff700 1 -- 172.21.15.121:0/3717889159 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b000f9f0 con 0x7f13c00aa240 2023-11-15T02:56:55.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.827+0000 7f13c745b700 1 -- 172.21.15.121:0/3717889159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13c00aa240 msgr2=0x7f13c0067a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:55.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.827+0000 7f13c745b700 1 --2- 172.21.15.121:0/3717889159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13c00aa240 0x7f13c0067a60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f13c0068570 tx=0x7f13b0009060).stop 2023-11-15T02:56:55.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.827+0000 7f13c745b700 1 -- 172.21.15.121:0/3717889159 shutdown_connections 2023-11-15T02:56:55.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.827+0000 7f13c745b700 1 --2- 172.21.15.121:0/3717889159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13c00aa240 0x7f13c0067a60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.827+0000 7f13c745b700 1 -- 172.21.15.121:0/3717889159 >> 172.21.15.121:0/3717889159 conn(0x7f13c0058c50 msgr2=0x7f13c0059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:55.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.827+0000 7f13c745b700 1 -- 172.21.15.121:0/3717889159 shutdown_connections 2023-11-15T02:56:55.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.827+0000 7f13c745b700 1 -- 172.21.15.121:0/3717889159 wait complete. 2023-11-15T02:56:55.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13c745b700 1 Processor -- start 2023-11-15T02:56:55.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13c745b700 1 -- start start 2023-11-15T02:56:55.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13c745b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13c0067f50 0x7f13c00ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:55.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13c745b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13c00aa240 0x7f13c0153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:55.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13c745b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13c0153bb0 0x7f13c0155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:55.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13c745b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c0164400 con 0x7f13c0153bb0 2023-11-15T02:56:55.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13c745b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c01552d0 con 0x7f13c00aa240 2023-11-15T02:56:55.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13c745b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c0154480 con 0x7f13c0067f50 2023-11-15T02:56:55.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13c59f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13c0153bb0 0x7f13c0155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:55.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13c59f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13c0153bb0 0x7f13c0155990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32952/0 (socket says 172.21.15.121:32952) 2023-11-15T02:56:55.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13c59f8700 1 -- 172.21.15.121:0/3301834042 learned_addr learned my addr 172.21.15.121:0/3301834042 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:55.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13c49f6700 1 --2- 172.21.15.121:0/3301834042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13c00aa240 0x7f13c0153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:55.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13c51f7700 1 --2- 172.21.15.121:0/3301834042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13c0067f50 0x7f13c00ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:55.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13c59f8700 1 -- 172.21.15.121:0/3301834042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13c0067f50 msgr2=0x7f13c00ad4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:55.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13c59f8700 1 --2- 172.21.15.121:0/3301834042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13c0067f50 0x7f13c00ad4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13c59f8700 1 -- 172.21.15.121:0/3301834042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13c00aa240 msgr2=0x7f13c0153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:55.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13c59f8700 1 --2- 172.21.15.121:0/3301834042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13c00aa240 0x7f13c0153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:55.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13c59f8700 1 -- 172.21.15.121:0/3301834042 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b00092a0 con 0x7f13c0153bb0 2023-11-15T02:56:55.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13c59f8700 1 --2- 172.21.15.121:0/3301834042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13c0153bb0 0x7f13c0155990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f13b8002230 tx=0x7f13b80015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:55.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13b57fa700 1 -- 172.21.15.121:0/3301834042 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b800e4a0 con 0x7f13c0153bb0 2023-11-15T02:56:55.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.831+0000 7f13b57fa700 1 -- 172.21.15.121:0/3301834042 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b800e600 con 0x7f13c0153bb0 2023-11-15T02:56:55.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.835+0000 7f13c745b700 1 -- 172.21.15.121:0/3301834042 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c01546c0 con 0x7f13c0153bb0 2023-11-15T02:56:55.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.835+0000 7f13b57fa700 1 -- 172.21.15.121:0/3301834042 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b800e980 con 0x7f13c0153bb0 2023-11-15T02:56:55.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.835+0000 7f13c745b700 1 -- 172.21.15.121:0/3301834042 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c0154d00 con 0x7f13c0153bb0 2023-11-15T02:56:55.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.835+0000 7f13c745b700 1 -- 172.21.15.121:0/3301834042 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c006c150 con 0x7f13c0153bb0 2023-11-15T02:56:55.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.835+0000 7f13b57fa700 1 -- 172.21.15.121:0/3301834042 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f13b801b3b0 con 0x7f13c0153bb0 2023-11-15T02:56:55.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.839+0000 7f13b57fa700 1 --2- 172.21.15.121:0/3301834042 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f13a405a540 0x7f13a405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:55.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.839+0000 7f13b57fa700 1 -- 172.21.15.121:0/3301834042 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f13b8073dd0 con 0x7f13c0153bb0 2023-11-15T02:56:55.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.839+0000 7f13c51f7700 1 --2- 172.21.15.121:0/3301834042 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f13a405a540 0x7f13a405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:55.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.839+0000 7f13c51f7700 1 --2- 172.21.15.121:0/3301834042 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f13a405a540 0x7f13a405ca00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f13bc007fd0 tx=0x7f13bc009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:55.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:55.847+0000 7f13b57fa700 1 -- 172.21.15.121:0/3301834042 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f13b80475c0 con 0x7f13c0153bb0 2023-11-15T02:56:56.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.003+0000 7f13c745b700 1 -- 172.21.15.121:0/3301834042 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f13c0155020 con 0x7f13a405a540 2023-11-15T02:56:56.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.023+0000 7f13b57fa700 1 -- 172.21.15.121:0/3301834042 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336240 (secure 0 0 0) 0x7f13c0155020 con 0x7f13a405a540 2023-11-15T02:56:56.034 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:56:56.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.031+0000 7f13c745b700 1 -- 172.21.15.121:0/3301834042 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f13a405a540 msgr2=0x7f13a405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:56.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.031+0000 7f13c745b700 1 --2- 172.21.15.121:0/3301834042 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f13a405a540 0x7f13a405ca00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f13bc007fd0 tx=0x7f13bc009040).stop 2023-11-15T02:56:56.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.031+0000 7f13c745b700 1 -- 172.21.15.121:0/3301834042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13c0153bb0 msgr2=0x7f13c0155990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:56.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.031+0000 7f13c745b700 1 --2- 172.21.15.121:0/3301834042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13c0153bb0 0x7f13c0155990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f13b8002230 tx=0x7f13b80015e0).stop 2023-11-15T02:56:56.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.031+0000 7f13c745b700 1 -- 172.21.15.121:0/3301834042 shutdown_connections 2023-11-15T02:56:56.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.031+0000 7f13c745b700 1 --2- 172.21.15.121:0/3301834042 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f13a405a540 0x7f13a405ca00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.031+0000 7f13c745b700 1 --2- 172.21.15.121:0/3301834042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13c0153bb0 0x7f13c0155990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.031+0000 7f13c745b700 1 --2- 172.21.15.121:0/3301834042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13c00aa240 0x7f13c0153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.031+0000 7f13c745b700 1 --2- 172.21.15.121:0/3301834042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13c0067f50 0x7f13c00ad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.031+0000 7f13c745b700 1 -- 172.21.15.121:0/3301834042 >> 172.21.15.121:0/3301834042 conn(0x7f13c0058c50 msgr2=0x7f13c00c0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:56.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.031+0000 7f13c745b700 1 -- 172.21.15.121:0/3301834042 shutdown_connections 2023-11-15T02:56:56.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.031+0000 7f13c745b700 1 -- 172.21.15.121:0/3301834042 wait complete. 2023-11-15T02:56:56.040 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T02:56:56.056 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-11-15T02:56:55.913138+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":256,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2378492,"kb_used_data":1724,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752596228,"statfs":{"total":773094113280,"available":770658537472,"internally_reserved":0,"allocated":1765376,"data_stored":635719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.736895+0000","last_change":"2023-11-15T02:56:51.733024+0000","last_active":"2023-11-15T02:56:51.736895+0000","last_peered":"2023-11-15T02:56:51.736895+0000","last_clean":"2023-11-15T02:56:51.736895+0000","last_became_active":"2023-11-15T02:56:51.732735+0000","last_became_peered":"2023-11-15T02:56:51.732735+0000","last_unstale":"2023-11-15T02:56:51.736895+0000","last_undegraded":"2023-11-15T02:56:51.736895+0000","last_fullsized":"2023-11-15T02:56:51.736895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.704467+0000","last_change":"2023-11-15T02:56:51.273328+0000","last_active":"2023-11-15T02:56:51.704467+0000","last_peered":"2023-11-15T02:56:51.704467+0000","last_clean":"2023-11-15T02:56:51.704467+0000","last_became_active":"2023-11-15T02:56:51.273044+0000","last_became_peered":"2023-11-15T02:56:51.273044+0000","last_unstale":"2023-11-15T02:56:51.704467+0000","last_undegraded":"2023-11-15T02:56:51.704467+0000","last_fullsized":"2023-11-15T02:56:51.704467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744601+0000","last_change":"2023-11-15T02:56:51.742242+0000","last_active":"2023-11-15T02:56:51.744601+0000","last_peered":"2023-11-15T02:56:51.744601+0000","last_clean":"2023-11-15T02:56:51.744601+0000","last_became_active":"2023-11-15T02:56:51.741604+0000","last_became_peered":"2023-11-15T02:56:51.741604+0000","last_unstale":"2023-11-15T02:56:51.744601+0000","last_undegraded":"2023-11-15T02:56:51.744601+0000","last_fullsized":"2023-11-15T02:56:51.744601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.739512+0000","last_change":"2023-11-15T02:56:51.277577+0000","last_active":"2023-11-15T02:56:51.739512+0000","last_peered":"2023-11-15T02:56:51.739512+0000","last_clean":"2023-11-15T02:56:51.739512+0000","last_became_active":"2023-11-15T02:56:51.276745+0000","last_became_peered":"2023-11-15T02:56:51.276745+0000","last_unstale":"2023-11-15T02:56:51.739512+0000","last_undegraded":"2023-11-15T02:56:51.739512+0000","last_fullsized":"2023-11-15T02:56:51.739512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.716112+0000","last_change":"2023-11-15T02:56:51.269238+0000","last_active":"2023-11-15T02:56:51.716112+0000","last_peered":"2023-11-15T02:56:51.716112+0000","last_clean":"2023-11-15T02:56:51.716112+0000","last_became_active":"2023-11-15T02:56:51.268872+0000","last_became_peered":"2023-11-15T02:56:51.268872+0000","last_unstale":"2023-11-15T02:56:51.716112+0000","last_undegraded":"2023-11-15T02:56:51.716112+0000","last_fullsized":"2023-11-15T02:56:51.716112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747275+0000","last_change":"2023-11-15T02:56:51.742674+0000","last_active":"2023-11-15T02:56:51.747275+0000","last_peered":"2023-11-15T02:56:51.747275+0000","last_clean":"2023-11-15T02:56:51.747275+0000","last_became_active":"2023-11-15T02:56:51.742412+0000","last_became_peered":"2023-11-15T02:56:51.742412+0000","last_unstale":"2023-11-15T02:56:51.747275+0000","last_undegraded":"2023-11-15T02:56:51.747275+0000","last_fullsized":"2023-11-15T02:56:51.747275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712980+0000","last_change":"2023-11-15T02:56:51.279772+0000","last_active":"2023-11-15T02:56:51.712980+0000","last_peered":"2023-11-15T02:56:51.712980+0000","last_clean":"2023-11-15T02:56:51.712980+0000","last_became_active":"2023-11-15T02:56:51.279423+0000","last_became_peered":"2023-11-15T02:56:51.279423+0000","last_unstale":"2023-11-15T02:56:51.712980+0000","last_undegraded":"2023-11-15T02:56:51.712980+0000","last_fullsized":"2023-11-15T02:56:51.712980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738160+0000","last_change":"2023-11-15T02:56:51.731913+0000","last_active":"2023-11-15T02:56:51.738160+0000","last_peered":"2023-11-15T02:56:51.738160+0000","last_clean":"2023-11-15T02:56:51.738160+0000","last_became_active":"2023-11-15T02:56:51.728632+0000","last_became_peered":"2023-11-15T02:56:51.728632+0000","last_unstale":"2023-11-15T02:56:51.738160+0000","last_undegraded":"2023-11-15T02:56:51.738160+0000","last_fullsized":"2023-11-15T02:56:51.738160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744972+0000","last_change":"2023-11-15T02:56:51.289877+0000","last_active":"2023-11-15T02:56:51.744972+0000","last_peered":"2023-11-15T02:56:51.744972+0000","last_clean":"2023-11-15T02:56:51.744972+0000","last_became_active":"2023-11-15T02:56:51.288940+0000","last_became_peered":"2023-11-15T02:56:51.288940+0000","last_unstale":"2023-11-15T02:56:51.744972+0000","last_undegraded":"2023-11-15T02:56:51.744972+0000","last_fullsized":"2023-11-15T02:56:51.744972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.743484+0000","last_change":"2023-11-15T02:56:51.738869+0000","last_active":"2023-11-15T02:56:51.743484+0000","last_peered":"2023-11-15T02:56:51.743484+0000","last_clean":"2023-11-15T02:56:51.743484+0000","last_became_active":"2023-11-15T02:56:51.738159+0000","last_became_peered":"2023-11-15T02:56:51.738159+0000","last_unstale":"2023-11-15T02:56:51.743484+0000","last_undegraded":"2023-11-15T02:56:51.743484+0000","last_fullsized":"2023-11-15T02:56:51.743484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713144+0000","last_change":"2023-11-15T02:56:51.284304+0000","last_active":"2023-11-15T02:56:51.713144+0000","last_peered":"2023-11-15T02:56:51.713144+0000","last_clean":"2023-11-15T02:56:51.713144+0000","last_became_active":"2023-11-15T02:56:51.284043+0000","last_became_peered":"2023-11-15T02:56:51.284043+0000","last_unstale":"2023-11-15T02:56:51.713144+0000","last_undegraded":"2023-11-15T02:56:51.713144+0000","last_fullsized":"2023-11-15T02:56:51.713144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747361+0000","last_change":"2023-11-15T02:56:51.744243+0000","last_active":"2023-11-15T02:56:51.747361+0000","last_peered":"2023-11-15T02:56:51.747361+0000","last_clean":"2023-11-15T02:56:51.747361+0000","last_became_active":"2023-11-15T02:56:51.743943+0000","last_became_peered":"2023-11-15T02:56:51.743943+0000","last_unstale":"2023-11-15T02:56:51.747361+0000","last_undegraded":"2023-11-15T02:56:51.747361+0000","last_fullsized":"2023-11-15T02:56:51.747361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.734769+0000","last_change":"2023-11-15T02:56:51.286898+0000","last_active":"2023-11-15T02:56:51.734769+0000","last_peered":"2023-11-15T02:56:51.734769+0000","last_clean":"2023-11-15T02:56:51.734769+0000","last_became_active":"2023-11-15T02:56:51.286628+0000","last_became_peered":"2023-11-15T02:56:51.286628+0000","last_unstale":"2023-11-15T02:56:51.734769+0000","last_undegraded":"2023-11-15T02:56:51.734769+0000","last_fullsized":"2023-11-15T02:56:51.734769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.749549+0000","last_change":"2023-11-15T02:56:51.745783+0000","last_active":"2023-11-15T02:56:51.749549+0000","last_peered":"2023-11-15T02:56:51.749549+0000","last_clean":"2023-11-15T02:56:51.749549+0000","last_became_active":"2023-11-15T02:56:51.745028+0000","last_became_peered":"2023-11-15T02:56:51.745028+0000","last_unstale":"2023-11-15T02:56:51.749549+0000","last_undegraded":"2023-11-15T02:56:51.749549+0000","last_fullsized":"2023-11-15T02:56:51.749549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.716637+0000","last_change":"2023-11-15T02:56:51.268012+0000","last_active":"2023-11-15T02:56:51.716637+0000","last_peered":"2023-11-15T02:56:51.716637+0000","last_clean":"2023-11-15T02:56:51.716637+0000","last_became_active":"2023-11-15T02:56:51.267734+0000","last_became_peered":"2023-11-15T02:56:51.267734+0000","last_unstale":"2023-11-15T02:56:51.716637+0000","last_undegraded":"2023-11-15T02:56:51.716637+0000","last_fullsized":"2023-11-15T02:56:51.716637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730642+0000","last_change":"2023-11-15T02:56:51.720753+0000","last_active":"2023-11-15T02:56:51.730642+0000","last_peered":"2023-11-15T02:56:51.730642+0000","last_clean":"2023-11-15T02:56:51.730642+0000","last_became_active":"2023-11-15T02:56:51.720500+0000","last_became_peered":"2023-11-15T02:56:51.720500+0000","last_unstale":"2023-11-15T02:56:51.730642+0000","last_undegraded":"2023-11-15T02:56:51.730642+0000","last_fullsized":"2023-11-15T02:56:51.730642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.711624+0000","last_change":"2023-11-15T02:56:51.280573+0000","last_active":"2023-11-15T02:56:51.711624+0000","last_peered":"2023-11-15T02:56:51.711624+0000","last_clean":"2023-11-15T02:56:51.711624+0000","last_became_active":"2023-11-15T02:56:51.275286+0000","last_became_peered":"2023-11-15T02:56:51.275286+0000","last_unstale":"2023-11-15T02:56:51.711624+0000","last_undegraded":"2023-11-15T02:56:51.711624+0000","last_fullsized":"2023-11-15T02:56:51.711624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.742388+0000","last_change":"2023-11-15T02:56:51.736450+0000","last_active":"2023-11-15T02:56:51.742388+0000","last_peered":"2023-11-15T02:56:51.742388+0000","last_clean":"2023-11-15T02:56:51.742388+0000","last_became_active":"2023-11-15T02:56:51.736143+0000","last_became_peered":"2023-11-15T02:56:51.736143+0000","last_unstale":"2023-11-15T02:56:51.742388+0000","last_undegraded":"2023-11-15T02:56:51.742388+0000","last_fullsized":"2023-11-15T02:56:51.742388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.714425+0000","last_change":"2023-11-15T02:56:51.284434+0000","last_active":"2023-11-15T02:56:51.714425+0000","last_peered":"2023-11-15T02:56:51.714425+0000","last_clean":"2023-11-15T02:56:51.714425+0000","last_became_active":"2023-11-15T02:56:51.284051+0000","last_became_peered":"2023-11-15T02:56:51.284051+0000","last_unstale":"2023-11-15T02:56:51.714425+0000","last_undegraded":"2023-11-15T02:56:51.714425+0000","last_fullsized":"2023-11-15T02:56:51.714425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744584+0000","last_change":"2023-11-15T02:56:51.742247+0000","last_active":"2023-11-15T02:56:51.744584+0000","last_peered":"2023-11-15T02:56:51.744584+0000","last_clean":"2023-11-15T02:56:51.744584+0000","last_became_active":"2023-11-15T02:56:51.741966+0000","last_became_peered":"2023-11-15T02:56:51.741966+0000","last_unstale":"2023-11-15T02:56:51.744584+0000","last_undegraded":"2023-11-15T02:56:51.744584+0000","last_fullsized":"2023-11-15T02:56:51.744584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.714493+0000","last_change":"2023-11-15T02:56:51.277678+0000","last_active":"2023-11-15T02:56:51.714493+0000","last_peered":"2023-11-15T02:56:51.714493+0000","last_clean":"2023-11-15T02:56:51.714493+0000","last_became_active":"2023-11-15T02:56:51.277408+0000","last_became_peered":"2023-11-15T02:56:51.277408+0000","last_unstale":"2023-11-15T02:56:51.714493+0000","last_undegraded":"2023-11-15T02:56:51.714493+0000","last_fullsized":"2023-11-15T02:56:51.714493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.731195+0000","last_change":"2023-11-15T02:56:51.717008+0000","last_active":"2023-11-15T02:56:51.731195+0000","last_peered":"2023-11-15T02:56:51.731195+0000","last_clean":"2023-11-15T02:56:51.731195+0000","last_became_active":"2023-11-15T02:56:51.716661+0000","last_became_peered":"2023-11-15T02:56:51.716661+0000","last_unstale":"2023-11-15T02:56:51.731195+0000","last_undegraded":"2023-11-15T02:56:51.731195+0000","last_fullsized":"2023-11-15T02:56:51.731195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713801+0000","last_change":"2023-11-15T02:56:51.281739+0000","last_active":"2023-11-15T02:56:51.713801+0000","last_peered":"2023-11-15T02:56:51.713801+0000","last_clean":"2023-11-15T02:56:51.713801+0000","last_became_active":"2023-11-15T02:56:51.281484+0000","last_became_peered":"2023-11-15T02:56:51.281484+0000","last_unstale":"2023-11-15T02:56:51.713801+0000","last_undegraded":"2023-11-15T02:56:51.713801+0000","last_fullsized":"2023-11-15T02:56:51.713801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738099+0000","last_change":"2023-11-15T02:56:51.726671+0000","last_active":"2023-11-15T02:56:51.738099+0000","last_peered":"2023-11-15T02:56:51.738099+0000","last_clean":"2023-11-15T02:56:51.738099+0000","last_became_active":"2023-11-15T02:56:51.725108+0000","last_became_peered":"2023-11-15T02:56:51.725108+0000","last_unstale":"2023-11-15T02:56:51.738099+0000","last_undegraded":"2023-11-15T02:56:51.738099+0000","last_fullsized":"2023-11-15T02:56:51.738099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.704634+0000","last_change":"2023-11-15T02:56:51.271748+0000","last_active":"2023-11-15T02:56:51.704634+0000","last_peered":"2023-11-15T02:56:51.704634+0000","last_clean":"2023-11-15T02:56:51.704634+0000","last_became_active":"2023-11-15T02:56:51.271310+0000","last_became_peered":"2023-11-15T02:56:51.271310+0000","last_unstale":"2023-11-15T02:56:51.704634+0000","last_undegraded":"2023-11-15T02:56:51.704634+0000","last_fullsized":"2023-11-15T02:56:51.704634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744179+0000","last_change":"2023-11-15T02:56:51.737961+0000","last_active":"2023-11-15T02:56:51.744179+0000","last_peered":"2023-11-15T02:56:51.744179+0000","last_clean":"2023-11-15T02:56:51.744179+0000","last_became_active":"2023-11-15T02:56:51.737603+0000","last_became_peered":"2023-11-15T02:56:51.737603+0000","last_unstale":"2023-11-15T02:56:51.744179+0000","last_undegraded":"2023-11-15T02:56:51.744179+0000","last_fullsized":"2023-11-15T02:56:51.744179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.711878+0000","last_change":"2023-11-15T02:56:51.286344+0000","last_active":"2023-11-15T02:56:51.711878+0000","last_peered":"2023-11-15T02:56:51.711878+0000","last_clean":"2023-11-15T02:56:51.711878+0000","last_became_active":"2023-11-15T02:56:51.286050+0000","last_became_peered":"2023-11-15T02:56:51.286050+0000","last_unstale":"2023-11-15T02:56:51.711878+0000","last_undegraded":"2023-11-15T02:56:51.711878+0000","last_fullsized":"2023-11-15T02:56:51.711878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.740796+0000","last_change":"2023-11-15T02:56:51.729804+0000","last_active":"2023-11-15T02:56:51.740796+0000","last_peered":"2023-11-15T02:56:51.740796+0000","last_clean":"2023-11-15T02:56:51.740796+0000","last_became_active":"2023-11-15T02:56:51.729569+0000","last_became_peered":"2023-11-15T02:56:51.729569+0000","last_unstale":"2023-11-15T02:56:51.740796+0000","last_undegraded":"2023-11-15T02:56:51.740796+0000","last_fullsized":"2023-11-15T02:56:51.740796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.721025+0000","last_change":"2023-11-15T02:56:51.280678+0000","last_active":"2023-11-15T02:56:51.721025+0000","last_peered":"2023-11-15T02:56:51.721025+0000","last_clean":"2023-11-15T02:56:51.721025+0000","last_became_active":"2023-11-15T02:56:51.279110+0000","last_became_peered":"2023-11-15T02:56:51.279110+0000","last_unstale":"2023-11-15T02:56:51.721025+0000","last_undegraded":"2023-11-15T02:56:51.721025+0000","last_fullsized":"2023-11-15T02:56:51.721025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.728488+0000","last_change":"2023-11-15T02:56:51.726563+0000","last_active":"2023-11-15T02:56:51.728488+0000","last_peered":"2023-11-15T02:56:51.728488+0000","last_clean":"2023-11-15T02:56:51.728488+0000","last_became_active":"2023-11-15T02:56:51.726284+0000","last_became_peered":"2023-11-15T02:56:51.726284+0000","last_unstale":"2023-11-15T02:56:51.728488+0000","last_undegraded":"2023-11-15T02:56:51.728488+0000","last_fullsized":"2023-11-15T02:56:51.728488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.716713+0000","last_change":"2023-11-15T02:56:51.275575+0000","last_active":"2023-11-15T02:56:51.716713+0000","last_peered":"2023-11-15T02:56:51.716713+0000","last_clean":"2023-11-15T02:56:51.716713+0000","last_became_active":"2023-11-15T02:56:51.275267+0000","last_became_peered":"2023-11-15T02:56:51.275267+0000","last_unstale":"2023-11-15T02:56:51.716713+0000","last_undegraded":"2023-11-15T02:56:51.716713+0000","last_fullsized":"2023-11-15T02:56:51.716713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.732898+0000","last_change":"2023-11-15T02:56:51.728149+0000","last_active":"2023-11-15T02:56:51.732898+0000","last_peered":"2023-11-15T02:56:51.732898+0000","last_clean":"2023-11-15T02:56:51.732898+0000","last_became_active":"2023-11-15T02:56:51.727553+0000","last_became_peered":"2023-11-15T02:56:51.727553+0000","last_unstale":"2023-11-15T02:56:51.732898+0000","last_undegraded":"2023-11-15T02:56:51.732898+0000","last_fullsized":"2023-11-15T02:56:51.732898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712886+0000","last_change":"2023-11-15T02:56:51.269197+0000","last_active":"2023-11-15T02:56:51.712886+0000","last_peered":"2023-11-15T02:56:51.712886+0000","last_clean":"2023-11-15T02:56:51.712886+0000","last_became_active":"2023-11-15T02:56:51.268904+0000","last_became_peered":"2023-11-15T02:56:51.268904+0000","last_unstale":"2023-11-15T02:56:51.712886+0000","last_undegraded":"2023-11-15T02:56:51.712886+0000","last_fullsized":"2023-11-15T02:56:51.712886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.724341+0000","last_change":"2023-11-15T02:56:51.715489+0000","last_active":"2023-11-15T02:56:51.724341+0000","last_peered":"2023-11-15T02:56:51.724341+0000","last_clean":"2023-11-15T02:56:51.724341+0000","last_became_active":"2023-11-15T02:56:51.715060+0000","last_became_peered":"2023-11-15T02:56:51.715060+0000","last_unstale":"2023-11-15T02:56:51.724341+0000","last_undegraded":"2023-11-15T02:56:51.724341+0000","last_fullsized":"2023-11-15T02:56:51.724341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712366+0000","last_change":"2023-11-15T02:56:51.276645+0000","last_active":"2023-11-15T02:56:51.712366+0000","last_peered":"2023-11-15T02:56:51.712366+0000","last_clean":"2023-11-15T02:56:51.712366+0000","last_became_active":"2023-11-15T02:56:51.276356+0000","last_became_peered":"2023-11-15T02:56:51.276356+0000","last_unstale":"2023-11-15T02:56:51.712366+0000","last_undegraded":"2023-11-15T02:56:51.712366+0000","last_fullsized":"2023-11-15T02:56:51.712366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.724735+0000","last_change":"2023-11-15T02:56:51.719961+0000","last_active":"2023-11-15T02:56:51.724735+0000","last_peered":"2023-11-15T02:56:51.724735+0000","last_clean":"2023-11-15T02:56:51.724735+0000","last_became_active":"2023-11-15T02:56:51.719598+0000","last_became_peered":"2023-11-15T02:56:51.719598+0000","last_unstale":"2023-11-15T02:56:51.724735+0000","last_undegraded":"2023-11-15T02:56:51.724735+0000","last_fullsized":"2023-11-15T02:56:51.724735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.734513+0000","last_change":"2023-11-15T02:56:51.287275+0000","last_active":"2023-11-15T02:56:51.734513+0000","last_peered":"2023-11-15T02:56:51.734513+0000","last_clean":"2023-11-15T02:56:51.734513+0000","last_became_active":"2023-11-15T02:56:51.286259+0000","last_became_peered":"2023-11-15T02:56:51.286259+0000","last_unstale":"2023-11-15T02:56:51.734513+0000","last_undegraded":"2023-11-15T02:56:51.734513+0000","last_fullsized":"2023-11-15T02:56:51.734513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.751632+0000","last_change":"2023-11-15T02:56:51.748332+0000","last_active":"2023-11-15T02:56:51.751632+0000","last_peered":"2023-11-15T02:56:51.751632+0000","last_clean":"2023-11-15T02:56:51.751632+0000","last_became_active":"2023-11-15T02:56:51.747865+0000","last_became_peered":"2023-11-15T02:56:51.747865+0000","last_unstale":"2023-11-15T02:56:51.751632+0000","last_undegraded":"2023-11-15T02:56:51.751632+0000","last_fullsized":"2023-11-15T02:56:51.751632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746250+0000","last_change":"2023-11-15T02:56:51.274812+0000","last_active":"2023-11-15T02:56:51.746250+0000","last_peered":"2023-11-15T02:56:51.746250+0000","last_clean":"2023-11-15T02:56:51.746250+0000","last_became_active":"2023-11-15T02:56:51.273693+0000","last_became_peered":"2023-11-15T02:56:51.273693+0000","last_unstale":"2023-11-15T02:56:51.746250+0000","last_undegraded":"2023-11-15T02:56:51.746250+0000","last_fullsized":"2023-11-15T02:56:51.746250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.734600+0000","last_change":"2023-11-15T02:56:51.718982+0000","last_active":"2023-11-15T02:56:51.734600+0000","last_peered":"2023-11-15T02:56:51.734600+0000","last_clean":"2023-11-15T02:56:51.734600+0000","last_became_active":"2023-11-15T02:56:51.718702+0000","last_became_peered":"2023-11-15T02:56:51.718702+0000","last_unstale":"2023-11-15T02:56:51.734600+0000","last_undegraded":"2023-11-15T02:56:51.734600+0000","last_fullsized":"2023-11-15T02:56:51.734600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.743614+0000","last_change":"2023-11-15T02:56:51.285434+0000","last_active":"2023-11-15T02:56:51.743614+0000","last_peered":"2023-11-15T02:56:51.743614+0000","last_clean":"2023-11-15T02:56:51.743614+0000","last_became_active":"2023-11-15T02:56:51.284984+0000","last_became_peered":"2023-11-15T02:56:51.284984+0000","last_unstale":"2023-11-15T02:56:51.743614+0000","last_undegraded":"2023-11-15T02:56:51.743614+0000","last_fullsized":"2023-11-15T02:56:51.743614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744214+0000","last_change":"2023-11-15T02:56:51.740534+0000","last_active":"2023-11-15T02:56:51.744214+0000","last_peered":"2023-11-15T02:56:51.744214+0000","last_clean":"2023-11-15T02:56:51.744214+0000","last_became_active":"2023-11-15T02:56:51.740090+0000","last_became_peered":"2023-11-15T02:56:51.740090+0000","last_unstale":"2023-11-15T02:56:51.744214+0000","last_undegraded":"2023-11-15T02:56:51.744214+0000","last_fullsized":"2023-11-15T02:56:51.744214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.714872+0000","last_change":"2023-11-15T02:56:51.279489+0000","last_active":"2023-11-15T02:56:51.714872+0000","last_peered":"2023-11-15T02:56:51.714872+0000","last_clean":"2023-11-15T02:56:51.714872+0000","last_became_active":"2023-11-15T02:56:51.279222+0000","last_became_peered":"2023-11-15T02:56:51.279222+0000","last_unstale":"2023-11-15T02:56:51.714872+0000","last_undegraded":"2023-11-15T02:56:51.714872+0000","last_fullsized":"2023-11-15T02:56:51.714872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.749518+0000","last_change":"2023-11-15T02:56:51.744824+0000","last_active":"2023-11-15T02:56:51.749518+0000","last_peered":"2023-11-15T02:56:51.749518+0000","last_clean":"2023-11-15T02:56:51.749518+0000","last_became_active":"2023-11-15T02:56:51.744533+0000","last_became_peered":"2023-11-15T02:56:51.744533+0000","last_unstale":"2023-11-15T02:56:51.749518+0000","last_undegraded":"2023-11-15T02:56:51.749518+0000","last_fullsized":"2023-11-15T02:56:51.749518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.721137+0000","last_change":"2023-11-15T02:56:51.273142+0000","last_active":"2023-11-15T02:56:51.721137+0000","last_peered":"2023-11-15T02:56:51.721137+0000","last_clean":"2023-11-15T02:56:51.721137+0000","last_became_active":"2023-11-15T02:56:51.272861+0000","last_became_peered":"2023-11-15T02:56:51.272861+0000","last_unstale":"2023-11-15T02:56:51.721137+0000","last_undegraded":"2023-11-15T02:56:51.721137+0000","last_fullsized":"2023-11-15T02:56:51.721137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.735105+0000","last_change":"2023-11-15T02:56:51.728416+0000","last_active":"2023-11-15T02:56:51.735105+0000","last_peered":"2023-11-15T02:56:51.735105+0000","last_clean":"2023-11-15T02:56:51.735105+0000","last_became_active":"2023-11-15T02:56:51.728084+0000","last_became_peered":"2023-11-15T02:56:51.728084+0000","last_unstale":"2023-11-15T02:56:51.735105+0000","last_undegraded":"2023-11-15T02:56:51.735105+0000","last_fullsized":"2023-11-15T02:56:51.735105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.724100+0000","last_change":"2023-11-15T02:56:51.274354+0000","last_active":"2023-11-15T02:56:51.724100+0000","last_peered":"2023-11-15T02:56:51.724100+0000","last_clean":"2023-11-15T02:56:51.724100+0000","last_became_active":"2023-11-15T02:56:51.274083+0000","last_became_peered":"2023-11-15T02:56:51.274083+0000","last_unstale":"2023-11-15T02:56:51.724100+0000","last_undegraded":"2023-11-15T02:56:51.724100+0000","last_fullsized":"2023-11-15T02:56:51.724100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.736974+0000","last_change":"2023-11-15T02:56:51.721714+0000","last_active":"2023-11-15T02:56:51.736974+0000","last_peered":"2023-11-15T02:56:51.736974+0000","last_clean":"2023-11-15T02:56:51.736974+0000","last_became_active":"2023-11-15T02:56:51.721415+0000","last_became_peered":"2023-11-15T02:56:51.721415+0000","last_unstale":"2023-11-15T02:56:51.736974+0000","last_undegraded":"2023-11-15T02:56:51.736974+0000","last_fullsized":"2023-11-15T02:56:51.736974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.704644+0000","last_change":"2023-11-15T02:56:51.283397+0000","last_active":"2023-11-15T02:56:51.704644+0000","last_peered":"2023-11-15T02:56:51.704644+0000","last_clean":"2023-11-15T02:56:51.704644+0000","last_became_active":"2023-11-15T02:56:51.283101+0000","last_became_peered":"2023-11-15T02:56:51.283101+0000","last_unstale":"2023-11-15T02:56:51.704644+0000","last_undegraded":"2023-11-15T02:56:51.704644+0000","last_fullsized":"2023-11-15T02:56:51.704644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.735611+0000","last_change":"2023-11-15T02:56:51.727142+0000","last_active":"2023-11-15T02:56:51.735611+0000","last_peered":"2023-11-15T02:56:51.735611+0000","last_clean":"2023-11-15T02:56:51.735611+0000","last_became_active":"2023-11-15T02:56:51.726872+0000","last_became_peered":"2023-11-15T02:56:51.726872+0000","last_unstale":"2023-11-15T02:56:51.735611+0000","last_undegraded":"2023-11-15T02:56:51.735611+0000","last_fullsized":"2023-11-15T02:56:51.735611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.710839+0000","last_change":"2023-11-15T02:56:51.281104+0000","last_active":"2023-11-15T02:56:51.710839+0000","last_peered":"2023-11-15T02:56:51.710839+0000","last_clean":"2023-11-15T02:56:51.710839+0000","last_became_active":"2023-11-15T02:56:51.280784+0000","last_became_peered":"2023-11-15T02:56:51.280784+0000","last_unstale":"2023-11-15T02:56:51.710839+0000","last_undegraded":"2023-11-15T02:56:51.710839+0000","last_fullsized":"2023-11-15T02:56:51.710839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.737357+0000","last_change":"2023-11-15T02:56:51.719121+0000","last_active":"2023-11-15T02:56:51.737357+0000","last_peered":"2023-11-15T02:56:51.737357+0000","last_clean":"2023-11-15T02:56:51.737357+0000","last_became_active":"2023-11-15T02:56:51.718812+0000","last_became_peered":"2023-11-15T02:56:51.718812+0000","last_unstale":"2023-11-15T02:56:51.737357+0000","last_undegraded":"2023-11-15T02:56:51.737357+0000","last_fullsized":"2023-11-15T02:56:51.737357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.722994+0000","last_change":"2023-11-15T02:56:51.285977+0000","last_active":"2023-11-15T02:56:51.722994+0000","last_peered":"2023-11-15T02:56:51.722994+0000","last_clean":"2023-11-15T02:56:51.722994+0000","last_became_active":"2023-11-15T02:56:51.277431+0000","last_became_peered":"2023-11-15T02:56:51.277431+0000","last_unstale":"2023-11-15T02:56:51.722994+0000","last_undegraded":"2023-11-15T02:56:51.722994+0000","last_fullsized":"2023-11-15T02:56:51.722994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.735350+0000","last_change":"2023-11-15T02:56:51.720775+0000","last_active":"2023-11-15T02:56:51.735350+0000","last_peered":"2023-11-15T02:56:51.735350+0000","last_clean":"2023-11-15T02:56:51.735350+0000","last_became_active":"2023-11-15T02:56:51.720446+0000","last_became_peered":"2023-11-15T02:56:51.720446+0000","last_unstale":"2023-11-15T02:56:51.735350+0000","last_undegraded":"2023-11-15T02:56:51.735350+0000","last_fullsized":"2023-11-15T02:56:51.735350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.715531+0000","last_change":"2023-11-15T02:56:51.268303+0000","last_active":"2023-11-15T02:56:51.715531+0000","last_peered":"2023-11-15T02:56:51.715531+0000","last_clean":"2023-11-15T02:56:51.715531+0000","last_became_active":"2023-11-15T02:56:51.267982+0000","last_became_peered":"2023-11-15T02:56:51.267982+0000","last_unstale":"2023-11-15T02:56:51.715531+0000","last_undegraded":"2023-11-15T02:56:51.715531+0000","last_fullsized":"2023-11-15T02:56:51.715531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738120+0000","last_change":"2023-11-15T02:56:51.726853+0000","last_active":"2023-11-15T02:56:51.738120+0000","last_peered":"2023-11-15T02:56:51.738120+0000","last_clean":"2023-11-15T02:56:51.738120+0000","last_became_active":"2023-11-15T02:56:51.726448+0000","last_became_peered":"2023-11-15T02:56:51.726448+0000","last_unstale":"2023-11-15T02:56:51.738120+0000","last_undegraded":"2023-11-15T02:56:51.738120+0000","last_fullsized":"2023-11-15T02:56:51.738120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745254+0000","last_change":"2023-11-15T02:56:51.291178+0000","last_active":"2023-11-15T02:56:51.745254+0000","last_peered":"2023-11-15T02:56:51.745254+0000","last_clean":"2023-11-15T02:56:51.745254+0000","last_became_active":"2023-11-15T02:56:51.289245+0000","last_became_peered":"2023-11-15T02:56:51.289245+0000","last_unstale":"2023-11-15T02:56:51.745254+0000","last_undegraded":"2023-11-15T02:56:51.745254+0000","last_fullsized":"2023-11-15T02:56:51.745254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.718417+0000","last_change":"2023-11-15T02:56:51.714302+0000","last_active":"2023-11-15T02:56:51.718417+0000","last_peered":"2023-11-15T02:56:51.718417+0000","last_clean":"2023-11-15T02:56:51.718417+0000","last_became_active":"2023-11-15T02:56:51.714023+0000","last_became_peered":"2023-11-15T02:56:51.714023+0000","last_unstale":"2023-11-15T02:56:51.718417+0000","last_undegraded":"2023-11-15T02:56:51.718417+0000","last_fullsized":"2023-11-15T02:56:51.718417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746516+0000","last_change":"2023-11-15T02:56:51.265289+0000","last_active":"2023-11-15T02:56:51.746516+0000","last_peered":"2023-11-15T02:56:51.746516+0000","last_clean":"2023-11-15T02:56:51.746516+0000","last_became_active":"2023-11-15T02:56:51.264945+0000","last_became_peered":"2023-11-15T02:56:51.264945+0000","last_unstale":"2023-11-15T02:56:51.746516+0000","last_undegraded":"2023-11-15T02:56:51.746516+0000","last_fullsized":"2023-11-15T02:56:51.746516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.751653+0000","last_change":"2023-11-15T02:56:51.748696+0000","last_active":"2023-11-15T02:56:51.751653+0000","last_peered":"2023-11-15T02:56:51.751653+0000","last_clean":"2023-11-15T02:56:51.751653+0000","last_became_active":"2023-11-15T02:56:51.748403+0000","last_became_peered":"2023-11-15T02:56:51.748403+0000","last_unstale":"2023-11-15T02:56:51.751653+0000","last_undegraded":"2023-11-15T02:56:51.751653+0000","last_fullsized":"2023-11-15T02:56:51.751653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713322+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T02:56:51.713322+0000","last_peered":"2023-11-15T02:56:51.713322+0000","last_clean":"2023-11-15T02:56:51.713322+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T02:56:51.713322+0000","last_undegraded":"2023-11-15T02:56:51.713322+0000","last_fullsized":"2023-11-15T02:56:51.713322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.722937+0000","last_change":"2023-11-15T02:56:51.270301+0000","last_active":"2023-11-15T02:56:51.722937+0000","last_peered":"2023-11-15T02:56:51.722937+0000","last_clean":"2023-11-15T02:56:51.722937+0000","last_became_active":"2023-11-15T02:56:51.269996+0000","last_became_peered":"2023-11-15T02:56:51.269996+0000","last_unstale":"2023-11-15T02:56:51.722937+0000","last_undegraded":"2023-11-15T02:56:51.722937+0000","last_fullsized":"2023-11-15T02:56:51.722937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.742366+0000","last_change":"2023-11-15T02:56:51.736621+0000","last_active":"2023-11-15T02:56:51.742366+0000","last_peered":"2023-11-15T02:56:51.742366+0000","last_clean":"2023-11-15T02:56:51.742366+0000","last_became_active":"2023-11-15T02:56:51.736319+0000","last_became_peered":"2023-11-15T02:56:51.736319+0000","last_unstale":"2023-11-15T02:56:51.742366+0000","last_undegraded":"2023-11-15T02:56:51.742366+0000","last_fullsized":"2023-11-15T02:56:51.742366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.706577+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T02:56:51.706577+0000","last_peered":"2023-11-15T02:56:51.706577+0000","last_clean":"2023-11-15T02:56:51.706577+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T02:56:51.706577+0000","last_undegraded":"2023-11-15T02:56:51.706577+0000","last_fullsized":"2023-11-15T02:56:51.706577+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745714+0000","last_change":"2023-11-15T02:56:51.273274+0000","last_active":"2023-11-15T02:56:51.745714+0000","last_peered":"2023-11-15T02:56:51.745714+0000","last_clean":"2023-11-15T02:56:51.745714+0000","last_became_active":"2023-11-15T02:56:51.270655+0000","last_became_peered":"2023-11-15T02:56:51.270655+0000","last_unstale":"2023-11-15T02:56:51.745714+0000","last_undegraded":"2023-11-15T02:56:51.745714+0000","last_fullsized":"2023-11-15T02:56:51.745714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738809+0000","last_change":"2023-11-15T02:56:51.734478+0000","last_active":"2023-11-15T02:56:51.738809+0000","last_peered":"2023-11-15T02:56:51.738809+0000","last_clean":"2023-11-15T02:56:51.738809+0000","last_became_active":"2023-11-15T02:56:51.732194+0000","last_became_peered":"2023-11-15T02:56:51.732194+0000","last_unstale":"2023-11-15T02:56:51.738809+0000","last_undegraded":"2023-11-15T02:56:51.738809+0000","last_fullsized":"2023-11-15T02:56:51.738809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730730+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T02:56:51.730730+0000","last_peered":"2023-11-15T02:56:51.730730+0000","last_clean":"2023-11-15T02:56:51.730730+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T02:56:51.730730+0000","last_undegraded":"2023-11-15T02:56:51.730730+0000","last_fullsized":"2023-11-15T02:56:51.730730+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.705716+0000","last_change":"2023-11-15T02:56:51.270161+0000","last_active":"2023-11-15T02:56:51.705716+0000","last_peered":"2023-11-15T02:56:51.705716+0000","last_clean":"2023-11-15T02:56:51.705716+0000","last_became_active":"2023-11-15T02:56:51.268151+0000","last_became_peered":"2023-11-15T02:56:51.268151+0000","last_unstale":"2023-11-15T02:56:51.705716+0000","last_undegraded":"2023-11-15T02:56:51.705716+0000","last_fullsized":"2023-11-15T02:56:51.705716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.737232+0000","last_change":"2023-11-15T02:56:51.731375+0000","last_active":"2023-11-15T02:56:51.737232+0000","last_peered":"2023-11-15T02:56:51.737232+0000","last_clean":"2023-11-15T02:56:51.737232+0000","last_became_active":"2023-11-15T02:56:51.731100+0000","last_became_peered":"2023-11-15T02:56:51.731100+0000","last_unstale":"2023-11-15T02:56:51.737232+0000","last_undegraded":"2023-11-15T02:56:51.737232+0000","last_fullsized":"2023-11-15T02:56:51.737232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.705942+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T02:56:51.705942+0000","last_peered":"2023-11-15T02:56:51.705942+0000","last_clean":"2023-11-15T02:56:51.705942+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T02:56:51.705942+0000","last_undegraded":"2023-11-15T02:56:51.705942+0000","last_fullsized":"2023-11-15T02:56:51.705942+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.715265+0000","last_change":"2023-11-15T02:56:51.275134+0000","last_active":"2023-11-15T02:56:51.715265+0000","last_peered":"2023-11-15T02:56:51.715265+0000","last_clean":"2023-11-15T02:56:51.715265+0000","last_became_active":"2023-11-15T02:56:51.274400+0000","last_became_peered":"2023-11-15T02:56:51.274400+0000","last_unstale":"2023-11-15T02:56:51.715265+0000","last_undegraded":"2023-11-15T02:56:51.715265+0000","last_fullsized":"2023-11-15T02:56:51.715265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.727321+0000","last_change":"2023-11-15T02:56:51.724465+0000","last_active":"2023-11-15T02:56:51.727321+0000","last_peered":"2023-11-15T02:56:51.727321+0000","last_clean":"2023-11-15T02:56:51.727321+0000","last_became_active":"2023-11-15T02:56:51.723590+0000","last_became_peered":"2023-11-15T02:56:51.723590+0000","last_unstale":"2023-11-15T02:56:51.727321+0000","last_undegraded":"2023-11-15T02:56:51.727321+0000","last_fullsized":"2023-11-15T02:56:51.727321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.731884+0000","last_change":"2023-11-15T02:56:51.282484+0000","last_active":"2023-11-15T02:56:51.731884+0000","last_peered":"2023-11-15T02:56:51.731884+0000","last_clean":"2023-11-15T02:56:51.731884+0000","last_became_active":"2023-11-15T02:56:51.282108+0000","last_became_peered":"2023-11-15T02:56:51.282108+0000","last_unstale":"2023-11-15T02:56:51.731884+0000","last_undegraded":"2023-11-15T02:56:51.731884+0000","last_fullsized":"2023-11-15T02:56:51.731884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.716774+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T02:56:51.716774+0000","last_peered":"2023-11-15T02:56:51.716774+0000","last_clean":"2023-11-15T02:56:51.716774+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T02:56:51.716774+0000","last_undegraded":"2023-11-15T02:56:51.716774+0000","last_fullsized":"2023-11-15T02:56:51.716774+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.719253+0000","last_change":"2023-11-15T02:56:51.717765+0000","last_active":"2023-11-15T02:56:51.719253+0000","last_peered":"2023-11-15T02:56:51.719253+0000","last_clean":"2023-11-15T02:56:51.719253+0000","last_became_active":"2023-11-15T02:56:51.717498+0000","last_became_peered":"2023-11-15T02:56:51.717498+0000","last_unstale":"2023-11-15T02:56:51.719253+0000","last_undegraded":"2023-11-15T02:56:51.719253+0000","last_fullsized":"2023-11-15T02:56:51.719253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.704337+0000","last_change":"2023-11-15T02:56:51.267133+0000","last_active":"2023-11-15T02:56:51.704337+0000","last_peered":"2023-11-15T02:56:51.704337+0000","last_clean":"2023-11-15T02:56:51.704337+0000","last_became_active":"2023-11-15T02:56:51.266864+0000","last_became_peered":"2023-11-15T02:56:51.266864+0000","last_unstale":"2023-11-15T02:56:51.704337+0000","last_undegraded":"2023-11-15T02:56:51.704337+0000","last_fullsized":"2023-11-15T02:56:51.704337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712961+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T02:56:51.712961+0000","last_peered":"2023-11-15T02:56:51.712961+0000","last_clean":"2023-11-15T02:56:51.712961+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T02:56:51.712961+0000","last_undegraded":"2023-11-15T02:56:51.712961+0000","last_fullsized":"2023-11-15T02:56:51.712961+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.719804+0000","last_change":"2023-11-15T02:56:51.717744+0000","last_active":"2023-11-15T02:56:51.719804+0000","last_peered":"2023-11-15T02:56:51.719804+0000","last_clean":"2023-11-15T02:56:51.719804+0000","last_became_active":"2023-11-15T02:56:51.717448+0000","last_became_peered":"2023-11-15T02:56:51.717448+0000","last_unstale":"2023-11-15T02:56:51.719804+0000","last_undegraded":"2023-11-15T02:56:51.719804+0000","last_fullsized":"2023-11-15T02:56:51.719804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.718982+0000","last_change":"2023-11-15T02:56:51.273152+0000","last_active":"2023-11-15T02:56:51.718982+0000","last_peered":"2023-11-15T02:56:51.718982+0000","last_clean":"2023-11-15T02:56:51.718982+0000","last_became_active":"2023-11-15T02:56:51.272868+0000","last_became_peered":"2023-11-15T02:56:51.272868+0000","last_unstale":"2023-11-15T02:56:51.718982+0000","last_undegraded":"2023-11-15T02:56:51.718982+0000","last_fullsized":"2023-11-15T02:56:51.718982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.728384+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T02:56:51.728384+0000","last_peered":"2023-11-15T02:56:51.728384+0000","last_clean":"2023-11-15T02:56:51.728384+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T02:56:51.728384+0000","last_undegraded":"2023-11-15T02:56:51.728384+0000","last_fullsized":"2023-11-15T02:56:51.728384+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744430+0000","last_change":"2023-11-15T02:56:51.739786+0000","last_active":"2023-11-15T02:56:51.744430+0000","last_peered":"2023-11-15T02:56:51.744430+0000","last_clean":"2023-11-15T02:56:51.744430+0000","last_became_active":"2023-11-15T02:56:51.737659+0000","last_became_peered":"2023-11-15T02:56:51.737659+0000","last_unstale":"2023-11-15T02:56:51.744430+0000","last_undegraded":"2023-11-15T02:56:51.744430+0000","last_fullsized":"2023-11-15T02:56:51.744430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.711641+0000","last_change":"2023-11-15T02:56:51.276817+0000","last_active":"2023-11-15T02:56:51.711641+0000","last_peered":"2023-11-15T02:56:51.711641+0000","last_clean":"2023-11-15T02:56:51.711641+0000","last_became_active":"2023-11-15T02:56:51.276490+0000","last_became_peered":"2023-11-15T02:56:51.276490+0000","last_unstale":"2023-11-15T02:56:51.711641+0000","last_undegraded":"2023-11-15T02:56:51.711641+0000","last_fullsized":"2023-11-15T02:56:51.711641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713989+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T02:56:51.713989+0000","last_peered":"2023-11-15T02:56:51.713989+0000","last_clean":"2023-11-15T02:56:51.713989+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T02:56:51.713989+0000","last_undegraded":"2023-11-15T02:56:51.713989+0000","last_fullsized":"2023-11-15T02:56:51.713989+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.728465+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T02:56:51.728465+0000","last_peered":"2023-11-15T02:56:51.728465+0000","last_clean":"2023-11-15T02:56:51.728465+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T02:56:51.728465+0000","last_undegraded":"2023-11-15T02:56:51.728465+0000","last_fullsized":"2023-11-15T02:56:51.728465+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.749349+0000","last_change":"2023-11-15T02:56:51.745538+0000","last_active":"2023-11-15T02:56:51.749349+0000","last_peered":"2023-11-15T02:56:51.749349+0000","last_clean":"2023-11-15T02:56:51.749349+0000","last_became_active":"2023-11-15T02:56:51.745369+0000","last_became_peered":"2023-11-15T02:56:51.745369+0000","last_unstale":"2023-11-15T02:56:51.749349+0000","last_undegraded":"2023-11-15T02:56:51.749349+0000","last_fullsized":"2023-11-15T02:56:51.749349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.710979+0000","last_change":"2023-11-15T02:56:51.280484+0000","last_active":"2023-11-15T02:56:51.710979+0000","last_peered":"2023-11-15T02:56:51.710979+0000","last_clean":"2023-11-15T02:56:51.710979+0000","last_became_active":"2023-11-15T02:56:51.272264+0000","last_became_peered":"2023-11-15T02:56:51.272264+0000","last_unstale":"2023-11-15T02:56:51.710979+0000","last_undegraded":"2023-11-15T02:56:51.710979+0000","last_fullsized":"2023-11-15T02:56:51.710979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.724717+0000","last_change":"2023-11-15T02:56:51.719709+0000","last_active":"2023-11-15T02:56:51.724717+0000","last_peered":"2023-11-15T02:56:51.724717+0000","last_clean":"2023-11-15T02:56:51.724717+0000","last_became_active":"2023-11-15T02:56:51.719230+0000","last_became_peered":"2023-11-15T02:56:51.719230+0000","last_unstale":"2023-11-15T02:56:51.724717+0000","last_undegraded":"2023-11-15T02:56:51.724717+0000","last_fullsized":"2023-11-15T02:56:51.724717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.735953+0000","last_change":"2023-11-15T02:56:51.272978+0000","last_active":"2023-11-15T02:56:51.735953+0000","last_peered":"2023-11-15T02:56:51.735953+0000","last_clean":"2023-11-15T02:56:51.735953+0000","last_became_active":"2023-11-15T02:56:51.272650+0000","last_became_peered":"2023-11-15T02:56:51.272650+0000","last_unstale":"2023-11-15T02:56:51.735953+0000","last_undegraded":"2023-11-15T02:56:51.735953+0000","last_fullsized":"2023-11-15T02:56:51.735953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.749492+0000","last_change":"2023-11-15T02:56:51.744603+0000","last_active":"2023-11-15T02:56:51.749492+0000","last_peered":"2023-11-15T02:56:51.749492+0000","last_clean":"2023-11-15T02:56:51.749492+0000","last_became_active":"2023-11-15T02:56:51.744212+0000","last_became_peered":"2023-11-15T02:56:51.744212+0000","last_unstale":"2023-11-15T02:56:51.749492+0000","last_undegraded":"2023-11-15T02:56:51.749492+0000","last_fullsized":"2023-11-15T02:56:51.749492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713620+0000","last_change":"2023-11-15T02:56:51.279629+0000","last_active":"2023-11-15T02:56:51.713620+0000","last_peered":"2023-11-15T02:56:51.713620+0000","last_clean":"2023-11-15T02:56:51.713620+0000","last_became_active":"2023-11-15T02:56:51.279367+0000","last_became_peered":"2023-11-15T02:56:51.279367+0000","last_unstale":"2023-11-15T02:56:51.713620+0000","last_undegraded":"2023-11-15T02:56:51.713620+0000","last_fullsized":"2023-11-15T02:56:51.713620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745035+0000","last_change":"2023-11-15T02:56:51.741271+0000","last_active":"2023-11-15T02:56:51.745035+0000","last_peered":"2023-11-15T02:56:51.745035+0000","last_clean":"2023-11-15T02:56:51.745035+0000","last_became_active":"2023-11-15T02:56:51.740569+0000","last_became_peered":"2023-11-15T02:56:51.740569+0000","last_unstale":"2023-11-15T02:56:51.745035+0000","last_undegraded":"2023-11-15T02:56:51.745035+0000","last_fullsized":"2023-11-15T02:56:51.745035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.740741+0000","last_change":"2023-11-15T02:56:51.278710+0000","last_active":"2023-11-15T02:56:51.740741+0000","last_peered":"2023-11-15T02:56:51.740741+0000","last_clean":"2023-11-15T02:56:51.740741+0000","last_became_active":"2023-11-15T02:56:51.278446+0000","last_became_peered":"2023-11-15T02:56:51.278446+0000","last_unstale":"2023-11-15T02:56:51.740741+0000","last_undegraded":"2023-11-15T02:56:51.740741+0000","last_fullsized":"2023-11-15T02:56:51.740741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745553+0000","last_change":"2023-11-15T02:56:51.740004+0000","last_active":"2023-11-15T02:56:51.745553+0000","last_peered":"2023-11-15T02:56:51.745553+0000","last_clean":"2023-11-15T02:56:51.745553+0000","last_became_active":"2023-11-15T02:56:51.739663+0000","last_became_peered":"2023-11-15T02:56:51.739663+0000","last_unstale":"2023-11-15T02:56:51.745553+0000","last_undegraded":"2023-11-15T02:56:51.745553+0000","last_fullsized":"2023-11-15T02:56:51.745553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.714982+0000","last_change":"2023-11-15T02:56:51.278072+0000","last_active":"2023-11-15T02:56:51.714982+0000","last_peered":"2023-11-15T02:56:51.714982+0000","last_clean":"2023-11-15T02:56:51.714982+0000","last_became_active":"2023-11-15T02:56:51.277779+0000","last_became_peered":"2023-11-15T02:56:51.277779+0000","last_unstale":"2023-11-15T02:56:51.714982+0000","last_undegraded":"2023-11-15T02:56:51.714982+0000","last_fullsized":"2023-11-15T02:56:51.714982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746826+0000","last_change":"2023-11-15T02:56:51.744994+0000","last_active":"2023-11-15T02:56:51.746826+0000","last_peered":"2023-11-15T02:56:51.746826+0000","last_clean":"2023-11-15T02:56:51.746826+0000","last_became_active":"2023-11-15T02:56:51.743902+0000","last_became_peered":"2023-11-15T02:56:51.743902+0000","last_unstale":"2023-11-15T02:56:51.746826+0000","last_undegraded":"2023-11-15T02:56:51.746826+0000","last_fullsized":"2023-11-15T02:56:51.746826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730502+0000","last_change":"2023-11-15T02:56:51.289728+0000","last_active":"2023-11-15T02:56:51.730502+0000","last_peered":"2023-11-15T02:56:51.730502+0000","last_clean":"2023-11-15T02:56:51.730502+0000","last_became_active":"2023-11-15T02:56:51.288566+0000","last_became_peered":"2023-11-15T02:56:51.288566+0000","last_unstale":"2023-11-15T02:56:51.730502+0000","last_undegraded":"2023-11-15T02:56:51.730502+0000","last_fullsized":"2023-11-15T02:56:51.730502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747477+0000","last_change":"2023-11-15T02:56:51.735076+0000","last_active":"2023-11-15T02:56:51.747477+0000","last_peered":"2023-11-15T02:56:51.747477+0000","last_clean":"2023-11-15T02:56:51.747477+0000","last_became_active":"2023-11-15T02:56:51.734802+0000","last_became_peered":"2023-11-15T02:56:51.734802+0000","last_unstale":"2023-11-15T02:56:51.747477+0000","last_undegraded":"2023-11-15T02:56:51.747477+0000","last_fullsized":"2023-11-15T02:56:51.747477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744754+0000","last_change":"2023-11-15T02:56:51.286563+0000","last_active":"2023-11-15T02:56:51.744754+0000","last_peered":"2023-11-15T02:56:51.744754+0000","last_clean":"2023-11-15T02:56:51.744754+0000","last_became_active":"2023-11-15T02:56:51.286245+0000","last_became_peered":"2023-11-15T02:56:51.286245+0000","last_unstale":"2023-11-15T02:56:51.744754+0000","last_undegraded":"2023-11-15T02:56:51.744754+0000","last_fullsized":"2023-11-15T02:56:51.744754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746575+0000","last_change":"2023-11-15T02:56:51.742254+0000","last_active":"2023-11-15T02:56:51.746575+0000","last_peered":"2023-11-15T02:56:51.746575+0000","last_clean":"2023-11-15T02:56:51.746575+0000","last_became_active":"2023-11-15T02:56:51.741950+0000","last_became_peered":"2023-11-15T02:56:51.741950+0000","last_unstale":"2023-11-15T02:56:51.746575+0000","last_undegraded":"2023-11-15T02:56:51.746575+0000","last_fullsized":"2023-11-15T02:56:51.746575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744885+0000","last_change":"2023-11-15T02:56:51.286640+0000","last_active":"2023-11-15T02:56:51.744885+0000","last_peered":"2023-11-15T02:56:51.744885+0000","last_clean":"2023-11-15T02:56:51.744885+0000","last_became_active":"2023-11-15T02:56:51.286290+0000","last_became_peered":"2023-11-15T02:56:51.286290+0000","last_unstale":"2023-11-15T02:56:51.744885+0000","last_undegraded":"2023-11-15T02:56:51.744885+0000","last_fullsized":"2023-11-15T02:56:51.744885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.733131+0000","last_change":"2023-11-15T02:56:51.727797+0000","last_active":"2023-11-15T02:56:51.733131+0000","last_peered":"2023-11-15T02:56:51.733131+0000","last_clean":"2023-11-15T02:56:51.733131+0000","last_became_active":"2023-11-15T02:56:51.727520+0000","last_became_peered":"2023-11-15T02:56:51.727520+0000","last_unstale":"2023-11-15T02:56:51.733131+0000","last_undegraded":"2023-11-15T02:56:51.733131+0000","last_fullsized":"2023-11-15T02:56:51.733131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738551+0000","last_change":"2023-11-15T02:56:51.284828+0000","last_active":"2023-11-15T02:56:51.738551+0000","last_peered":"2023-11-15T02:56:51.738551+0000","last_clean":"2023-11-15T02:56:51.738551+0000","last_became_active":"2023-11-15T02:56:51.276571+0000","last_became_peered":"2023-11-15T02:56:51.276571+0000","last_unstale":"2023-11-15T02:56:51.738551+0000","last_undegraded":"2023-11-15T02:56:51.738551+0000","last_fullsized":"2023-11-15T02:56:51.738551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.739504+0000","last_change":"2023-11-15T02:56:51.732129+0000","last_active":"2023-11-15T02:56:51.739504+0000","last_peered":"2023-11-15T02:56:51.739504+0000","last_clean":"2023-11-15T02:56:51.739504+0000","last_became_active":"2023-11-15T02:56:51.729770+0000","last_became_peered":"2023-11-15T02:56:51.729770+0000","last_unstale":"2023-11-15T02:56:51.739504+0000","last_undegraded":"2023-11-15T02:56:51.739504+0000","last_fullsized":"2023-11-15T02:56:51.739504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.731327+0000","last_change":"2023-11-15T02:56:51.280630+0000","last_active":"2023-11-15T02:56:51.731327+0000","last_peered":"2023-11-15T02:56:51.731327+0000","last_clean":"2023-11-15T02:56:51.731327+0000","last_became_active":"2023-11-15T02:56:51.280353+0000","last_became_peered":"2023-11-15T02:56:51.280353+0000","last_unstale":"2023-11-15T02:56:51.731327+0000","last_undegraded":"2023-11-15T02:56:51.731327+0000","last_fullsized":"2023-11-15T02:56:51.731327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.741375+0000","last_change":"2023-11-15T02:56:51.737689+0000","last_active":"2023-11-15T02:56:51.741375+0000","last_peered":"2023-11-15T02:56:51.741375+0000","last_clean":"2023-11-15T02:56:51.741375+0000","last_became_active":"2023-11-15T02:56:51.737369+0000","last_became_peered":"2023-11-15T02:56:51.737369+0000","last_unstale":"2023-11-15T02:56:51.741375+0000","last_undegraded":"2023-11-15T02:56:51.741375+0000","last_fullsized":"2023-11-15T02:56:51.741375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.719432+0000","last_change":"2023-11-15T02:56:51.285172+0000","last_active":"2023-11-15T02:56:51.719432+0000","last_peered":"2023-11-15T02:56:51.719432+0000","last_clean":"2023-11-15T02:56:51.719432+0000","last_became_active":"2023-11-15T02:56:51.284936+0000","last_became_peered":"2023-11-15T02:56:51.284936+0000","last_unstale":"2023-11-15T02:56:51.719432+0000","last_undegraded":"2023-11-15T02:56:51.719432+0000","last_fullsized":"2023-11-15T02:56:51.719432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.732918+0000","last_change":"2023-11-15T02:56:51.728432+0000","last_active":"2023-11-15T02:56:51.732918+0000","last_peered":"2023-11-15T02:56:51.732918+0000","last_clean":"2023-11-15T02:56:51.732918+0000","last_became_active":"2023-11-15T02:56:51.727880+0000","last_became_peered":"2023-11-15T02:56:51.727880+0000","last_unstale":"2023-11-15T02:56:51.732918+0000","last_undegraded":"2023-11-15T02:56:51.732918+0000","last_fullsized":"2023-11-15T02:56:51.732918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744376+0000","last_change":"2023-11-15T02:56:51.288049+0000","last_active":"2023-11-15T02:56:51.744376+0000","last_peered":"2023-11-15T02:56:51.744376+0000","last_clean":"2023-11-15T02:56:51.744376+0000","last_became_active":"2023-11-15T02:56:51.287794+0000","last_became_peered":"2023-11-15T02:56:51.287794+0000","last_unstale":"2023-11-15T02:56:51.744376+0000","last_undegraded":"2023-11-15T02:56:51.744376+0000","last_fullsized":"2023-11-15T02:56:51.744376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.734683+0000","last_change":"2023-11-15T02:56:51.720823+0000","last_active":"2023-11-15T02:56:51.734683+0000","last_peered":"2023-11-15T02:56:51.734683+0000","last_clean":"2023-11-15T02:56:51.734683+0000","last_became_active":"2023-11-15T02:56:51.720479+0000","last_became_peered":"2023-11-15T02:56:51.720479+0000","last_unstale":"2023-11-15T02:56:51.734683+0000","last_undegraded":"2023-11-15T02:56:51.734683+0000","last_fullsized":"2023-11-15T02:56:51.734683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.718875+0000","last_change":"2023-11-15T02:56:51.278717+0000","last_active":"2023-11-15T02:56:51.718875+0000","last_peered":"2023-11-15T02:56:51.718875+0000","last_clean":"2023-11-15T02:56:51.718875+0000","last_became_active":"2023-11-15T02:56:51.278404+0000","last_became_peered":"2023-11-15T02:56:51.278404+0000","last_unstale":"2023-11-15T02:56:51.718875+0000","last_undegraded":"2023-11-15T02:56:51.718875+0000","last_fullsized":"2023-11-15T02:56:51.718875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745052+0000","last_change":"2023-11-15T02:56:51.741288+0000","last_active":"2023-11-15T02:56:51.745052+0000","last_peered":"2023-11-15T02:56:51.745052+0000","last_clean":"2023-11-15T02:56:51.745052+0000","last_became_active":"2023-11-15T02:56:51.740865+0000","last_became_peered":"2023-11-15T02:56:51.740865+0000","last_unstale":"2023-11-15T02:56:51.745052+0000","last_undegraded":"2023-11-15T02:56:51.745052+0000","last_fullsized":"2023-11-15T02:56:51.745052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746626+0000","last_change":"2023-11-15T02:56:51.274179+0000","last_active":"2023-11-15T02:56:51.746626+0000","last_peered":"2023-11-15T02:56:51.746626+0000","last_clean":"2023-11-15T02:56:51.746626+0000","last_became_active":"2023-11-15T02:56:51.272313+0000","last_became_peered":"2023-11-15T02:56:51.272313+0000","last_unstale":"2023-11-15T02:56:51.746626+0000","last_undegraded":"2023-11-15T02:56:51.746626+0000","last_fullsized":"2023-11-15T02:56:51.746626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.737568+0000","last_change":"2023-11-15T02:56:51.736317+0000","last_active":"2023-11-15T02:56:51.737568+0000","last_peered":"2023-11-15T02:56:51.737568+0000","last_clean":"2023-11-15T02:56:51.737568+0000","last_became_active":"2023-11-15T02:56:51.736060+0000","last_became_peered":"2023-11-15T02:56:51.736060+0000","last_unstale":"2023-11-15T02:56:51.737568+0000","last_undegraded":"2023-11-15T02:56:51.737568+0000","last_fullsized":"2023-11-15T02:56:51.737568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.754612+0000","last_change":"2023-11-15T02:56:51.279248+0000","last_active":"2023-11-15T02:56:51.754612+0000","last_peered":"2023-11-15T02:56:51.754612+0000","last_clean":"2023-11-15T02:56:51.754612+0000","last_became_active":"2023-11-15T02:56:51.278771+0000","last_became_peered":"2023-11-15T02:56:51.278771+0000","last_unstale":"2023-11-15T02:56:51.754612+0000","last_undegraded":"2023-11-15T02:56:51.754612+0000","last_fullsized":"2023-11-15T02:56:51.754612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.727542+0000","last_change":"2023-11-15T02:56:51.716546+0000","last_active":"2023-11-15T02:56:51.727542+0000","last_peered":"2023-11-15T02:56:51.727542+0000","last_clean":"2023-11-15T02:56:51.727542+0000","last_became_active":"2023-11-15T02:56:51.716254+0000","last_became_peered":"2023-11-15T02:56:51.716254+0000","last_unstale":"2023-11-15T02:56:51.727542+0000","last_undegraded":"2023-11-15T02:56:51.727542+0000","last_fullsized":"2023-11-15T02:56:51.727542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712732+0000","last_change":"2023-11-15T02:56:51.286642+0000","last_active":"2023-11-15T02:56:51.712732+0000","last_peered":"2023-11-15T02:56:51.712732+0000","last_clean":"2023-11-15T02:56:51.712732+0000","last_became_active":"2023-11-15T02:56:51.286240+0000","last_became_peered":"2023-11-15T02:56:51.286240+0000","last_unstale":"2023-11-15T02:56:51.712732+0000","last_undegraded":"2023-11-15T02:56:51.712732+0000","last_fullsized":"2023-11-15T02:56:51.712732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.725581+0000","last_change":"2023-11-15T02:56:51.722322+0000","last_active":"2023-11-15T02:56:51.725581+0000","last_peered":"2023-11-15T02:56:51.725581+0000","last_clean":"2023-11-15T02:56:51.725581+0000","last_became_active":"2023-11-15T02:56:51.721931+0000","last_became_peered":"2023-11-15T02:56:51.721931+0000","last_unstale":"2023-11-15T02:56:51.725581+0000","last_undegraded":"2023-11-15T02:56:51.725581+0000","last_fullsized":"2023-11-15T02:56:51.725581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713667+0000","last_change":"2023-11-15T02:56:51.284656+0000","last_active":"2023-11-15T02:56:51.713667+0000","last_peered":"2023-11-15T02:56:51.713667+0000","last_clean":"2023-11-15T02:56:51.713667+0000","last_became_active":"2023-11-15T02:56:51.284378+0000","last_became_peered":"2023-11-15T02:56:51.284378+0000","last_unstale":"2023-11-15T02:56:51.713667+0000","last_undegraded":"2023-11-15T02:56:51.713667+0000","last_fullsized":"2023-11-15T02:56:51.713667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747336+0000","last_change":"2023-11-15T02:56:51.743546+0000","last_active":"2023-11-15T02:56:51.747336+0000","last_peered":"2023-11-15T02:56:51.747336+0000","last_clean":"2023-11-15T02:56:51.747336+0000","last_became_active":"2023-11-15T02:56:51.743276+0000","last_became_peered":"2023-11-15T02:56:51.743276+0000","last_unstale":"2023-11-15T02:56:51.747336+0000","last_undegraded":"2023-11-15T02:56:51.747336+0000","last_fullsized":"2023-11-15T02:56:51.747336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.729927+0000","last_change":"2023-11-15T02:56:51.285769+0000","last_active":"2023-11-15T02:56:51.729927+0000","last_peered":"2023-11-15T02:56:51.729927+0000","last_clean":"2023-11-15T02:56:51.729927+0000","last_became_active":"2023-11-15T02:56:51.285448+0000","last_became_peered":"2023-11-15T02:56:51.285448+0000","last_unstale":"2023-11-15T02:56:51.729927+0000","last_undegraded":"2023-11-15T02:56:51.729927+0000","last_fullsized":"2023-11-15T02:56:51.729927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.742411+0000","last_change":"2023-11-15T02:56:51.736745+0000","last_active":"2023-11-15T02:56:51.742411+0000","last_peered":"2023-11-15T02:56:51.742411+0000","last_clean":"2023-11-15T02:56:51.742411+0000","last_became_active":"2023-11-15T02:56:51.736555+0000","last_became_peered":"2023-11-15T02:56:51.736555+0000","last_unstale":"2023-11-15T02:56:51.742411+0000","last_undegraded":"2023-11-15T02:56:51.742411+0000","last_fullsized":"2023-11-15T02:56:51.742411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730754+0000","last_change":"2023-11-15T02:56:51.722343+0000","last_active":"2023-11-15T02:56:51.730754+0000","last_peered":"2023-11-15T02:56:51.730754+0000","last_clean":"2023-11-15T02:56:51.730754+0000","last_became_active":"2023-11-15T02:56:51.722090+0000","last_became_peered":"2023-11-15T02:56:51.722090+0000","last_unstale":"2023-11-15T02:56:51.730754+0000","last_undegraded":"2023-11-15T02:56:51.730754+0000","last_fullsized":"2023-11-15T02:56:51.730754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.710708+0000","last_change":"2023-11-15T02:56:51.281122+0000","last_active":"2023-11-15T02:56:51.710708+0000","last_peered":"2023-11-15T02:56:51.710708+0000","last_clean":"2023-11-15T02:56:51.710708+0000","last_became_active":"2023-11-15T02:56:51.280798+0000","last_became_peered":"2023-11-15T02:56:51.280798+0000","last_unstale":"2023-11-15T02:56:51.710708+0000","last_undegraded":"2023-11-15T02:56:51.710708+0000","last_fullsized":"2023-11-15T02:56:51.710708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.729313+0000","last_change":"2023-11-15T02:56:51.726756+0000","last_active":"2023-11-15T02:56:51.729313+0000","last_peered":"2023-11-15T02:56:51.729313+0000","last_clean":"2023-11-15T02:56:51.729313+0000","last_became_active":"2023-11-15T02:56:51.726446+0000","last_became_peered":"2023-11-15T02:56:51.726446+0000","last_unstale":"2023-11-15T02:56:51.729313+0000","last_undegraded":"2023-11-15T02:56:51.729313+0000","last_fullsized":"2023-11-15T02:56:51.729313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730560+0000","last_change":"2023-11-15T02:56:51.288389+0000","last_active":"2023-11-15T02:56:51.730560+0000","last_peered":"2023-11-15T02:56:51.730560+0000","last_clean":"2023-11-15T02:56:51.730560+0000","last_became_active":"2023-11-15T02:56:51.287933+0000","last_became_peered":"2023-11-15T02:56:51.287933+0000","last_unstale":"2023-11-15T02:56:51.730560+0000","last_undegraded":"2023-11-15T02:56:51.730560+0000","last_fullsized":"2023-11-15T02:56:51.730560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746552+0000","last_change":"2023-11-15T02:56:51.742907+0000","last_active":"2023-11-15T02:56:51.746552+0000","last_peered":"2023-11-15T02:56:51.746552+0000","last_clean":"2023-11-15T02:56:51.746552+0000","last_became_active":"2023-11-15T02:56:51.742633+0000","last_became_peered":"2023-11-15T02:56:51.742633+0000","last_unstale":"2023-11-15T02:56:51.746552+0000","last_undegraded":"2023-11-15T02:56:51.746552+0000","last_fullsized":"2023-11-15T02:56:51.746552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.736908+0000","last_change":"2023-11-15T02:56:51.278389+0000","last_active":"2023-11-15T02:56:51.736908+0000","last_peered":"2023-11-15T02:56:51.736908+0000","last_clean":"2023-11-15T02:56:51.736908+0000","last_became_active":"2023-11-15T02:56:51.277985+0000","last_became_peered":"2023-11-15T02:56:51.277985+0000","last_unstale":"2023-11-15T02:56:51.736908+0000","last_undegraded":"2023-11-15T02:56:51.736908+0000","last_fullsized":"2023-11-15T02:56:51.736908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.743771+0000","last_change":"2023-11-15T02:56:51.725491+0000","last_active":"2023-11-15T02:56:51.743771+0000","last_peered":"2023-11-15T02:56:51.743771+0000","last_clean":"2023-11-15T02:56:51.743771+0000","last_became_active":"2023-11-15T02:56:51.725072+0000","last_became_peered":"2023-11-15T02:56:51.725072+0000","last_unstale":"2023-11-15T02:56:51.743771+0000","last_undegraded":"2023-11-15T02:56:51.743771+0000","last_fullsized":"2023-11-15T02:56:51.743771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738751+0000","last_change":"2023-11-15T02:56:51.291272+0000","last_active":"2023-11-15T02:56:51.738751+0000","last_peered":"2023-11-15T02:56:51.738751+0000","last_clean":"2023-11-15T02:56:51.738751+0000","last_became_active":"2023-11-15T02:56:51.287508+0000","last_became_peered":"2023-11-15T02:56:51.287508+0000","last_unstale":"2023-11-15T02:56:51.738751+0000","last_undegraded":"2023-11-15T02:56:51.738751+0000","last_fullsized":"2023-11-15T02:56:51.738751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745082+0000","last_change":"2023-11-15T02:56:51.741837+0000","last_active":"2023-11-15T02:56:51.745082+0000","last_peered":"2023-11-15T02:56:51.745082+0000","last_clean":"2023-11-15T02:56:51.745082+0000","last_became_active":"2023-11-15T02:56:51.741487+0000","last_became_peered":"2023-11-15T02:56:51.741487+0000","last_unstale":"2023-11-15T02:56:51.745082+0000","last_undegraded":"2023-11-15T02:56:51.745082+0000","last_fullsized":"2023-11-15T02:56:51.745082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.741740+0000","last_change":"2023-11-15T02:56:51.283591+0000","last_active":"2023-11-15T02:56:51.741740+0000","last_peered":"2023-11-15T02:56:51.741740+0000","last_clean":"2023-11-15T02:56:51.741740+0000","last_became_active":"2023-11-15T02:56:51.283198+0000","last_became_peered":"2023-11-15T02:56:51.283198+0000","last_unstale":"2023-11-15T02:56:51.741740+0000","last_undegraded":"2023-11-15T02:56:51.741740+0000","last_fullsized":"2023-11-15T02:56:51.741740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744198+0000","last_change":"2023-11-15T02:56:51.739304+0000","last_active":"2023-11-15T02:56:51.744198+0000","last_peered":"2023-11-15T02:56:51.744198+0000","last_clean":"2023-11-15T02:56:51.744198+0000","last_became_active":"2023-11-15T02:56:51.738755+0000","last_became_peered":"2023-11-15T02:56:51.738755+0000","last_unstale":"2023-11-15T02:56:51.744198+0000","last_undegraded":"2023-11-15T02:56:51.744198+0000","last_fullsized":"2023-11-15T02:56:51.744198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.722852+0000","last_change":"2023-11-15T02:56:51.269310+0000","last_active":"2023-11-15T02:56:51.722852+0000","last_peered":"2023-11-15T02:56:51.722852+0000","last_clean":"2023-11-15T02:56:51.722852+0000","last_became_active":"2023-11-15T02:56:51.268977+0000","last_became_peered":"2023-11-15T02:56:51.268977+0000","last_unstale":"2023-11-15T02:56:51.722852+0000","last_undegraded":"2023-11-15T02:56:51.722852+0000","last_fullsized":"2023-11-15T02:56:51.722852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747235+0000","last_change":"2023-11-15T02:56:51.742002+0000","last_active":"2023-11-15T02:56:51.747235+0000","last_peered":"2023-11-15T02:56:51.747235+0000","last_clean":"2023-11-15T02:56:51.747235+0000","last_became_active":"2023-11-15T02:56:51.741691+0000","last_became_peered":"2023-11-15T02:56:51.741691+0000","last_unstale":"2023-11-15T02:56:51.747235+0000","last_undegraded":"2023-11-15T02:56:51.747235+0000","last_fullsized":"2023-11-15T02:56:51.747235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.706447+0000","last_change":"2023-11-15T02:56:51.280160+0000","last_active":"2023-11-15T02:56:51.706447+0000","last_peered":"2023-11-15T02:56:51.706447+0000","last_clean":"2023-11-15T02:56:51.706447+0000","last_became_active":"2023-11-15T02:56:51.278364+0000","last_became_peered":"2023-11-15T02:56:51.278364+0000","last_unstale":"2023-11-15T02:56:51.706447+0000","last_undegraded":"2023-11-15T02:56:51.706447+0000","last_fullsized":"2023-11-15T02:56:51.706447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747191+0000","last_change":"2023-11-15T02:56:51.742487+0000","last_active":"2023-11-15T02:56:51.747191+0000","last_peered":"2023-11-15T02:56:51.747191+0000","last_clean":"2023-11-15T02:56:51.747191+0000","last_became_active":"2023-11-15T02:56:51.742056+0000","last_became_peered":"2023-11-15T02:56:51.742056+0000","last_unstale":"2023-11-15T02:56:51.747191+0000","last_undegraded":"2023-11-15T02:56:51.747191+0000","last_fullsized":"2023-11-15T02:56:51.747191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.748052+0000","last_change":"2023-11-15T02:56:51.290964+0000","last_active":"2023-11-15T02:56:51.748052+0000","last_peered":"2023-11-15T02:56:51.748052+0000","last_clean":"2023-11-15T02:56:51.748052+0000","last_became_active":"2023-11-15T02:56:51.290728+0000","last_became_peered":"2023-11-15T02:56:51.290728+0000","last_unstale":"2023-11-15T02:56:51.748052+0000","last_undegraded":"2023-11-15T02:56:51.748052+0000","last_fullsized":"2023-11-15T02:56:51.748052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297332,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074508,"statfs":{"total":96636764160,"available":96332296192,"internally_reserved":0,"allocated":241664,"data_stored":80461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3380000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.375}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5309999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297312,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074528,"statfs":{"total":96636764160,"available":96332316672,"internally_reserved":0,"allocated":221184,"data_stored":80493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.512}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3440000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.423}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.458}]}]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297324,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074516,"statfs":{"total":96636764160,"available":96332304384,"internally_reserved":0,"allocated":233472,"data_stored":80639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.296}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5049999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297308,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074532,"statfs":{"total":96636764160,"available":96332320768,"internally_reserved":0,"allocated":217088,"data_stored":79137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.837}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7270000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.883}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":77848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":79047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":79047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5740000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.599}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.633}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.74}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":79047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3440000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2789999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T02:56:56.065 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T02:56:56.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.251+0000 7efc27edb700 1 Processor -- start 2023-11-15T02:56:56.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.251+0000 7efc27edb700 1 -- start start 2023-11-15T02:56:56.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.255+0000 7efc27edb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efc20153bb0 0x7efc200a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:56.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.255+0000 7efc27edb700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efc200aa020 0x7efc20067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:56.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.255+0000 7efc27edb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc200cb5c0 con 0x7efc20067f50 2023-11-15T02:56:56.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.255+0000 7efc27edb700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc200aa6f0 con 0x7efc200aa020 2023-11-15T02:56:56.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.255+0000 7efc27edb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc200c5900 con 0x7efc20153bb0 2023-11-15T02:56:56.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.255+0000 7efc25476700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efc200aa020 0x7efc20067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:56.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.255+0000 7efc25476700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efc200aa020 0x7efc20067a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32962/0 (socket says 172.21.15.121:32962) 2023-11-15T02:56:56.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.255+0000 7efc25476700 1 -- 172.21.15.121:0/1052996997 learned_addr learned my addr 172.21.15.121:0/1052996997 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:56.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.255+0000 7efc25c77700 1 --2- 172.21.15.121:0/1052996997 >> v2:172.21.15.190:3300/0 conn(0x7efc20153bb0 0x7efc200a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:56.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.255+0000 7efc25c77700 1 -- 172.21.15.121:0/1052996997 >> v2:172.21.15.121:3300/0 conn(0x7efc200aa020 msgr2=0x7efc20067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:56.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.255+0000 7efc25c77700 1 --2- 172.21.15.121:0/1052996997 >> v2:172.21.15.121:3300/0 conn(0x7efc200aa020 0x7efc20067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.255+0000 7efc25c77700 1 -- 172.21.15.121:0/1052996997 >> v1:172.21.15.190:6789/0 conn(0x7efc20067f50 legacy=0x7efc201578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:56.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.255+0000 7efc25c77700 1 -- 172.21.15.121:0/1052996997 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc200c9ca0 con 0x7efc20153bb0 2023-11-15T02:56:56.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.259+0000 7efc25c77700 1 --2- 172.21.15.121:0/1052996997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc20153bb0 0x7efc200a9b30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7efc20069f00 tx=0x7efc1c009060).ready entity=mon.1 client_cookie=bf2dc0fe2cfed65c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:56.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.259+0000 7efc24c75700 1 -- 172.21.15.121:0/1052996997 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c00f510 con 0x7efc20153bb0 2023-11-15T02:56:56.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.259+0000 7efc24c75700 1 -- 172.21.15.121:0/1052996997 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c00f670 con 0x7efc20153bb0 2023-11-15T02:56:56.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.259+0000 7efc24c75700 1 -- 172.21.15.121:0/1052996997 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c00f9f0 con 0x7efc20153bb0 2023-11-15T02:56:56.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.259+0000 7efc27edb700 1 -- 172.21.15.121:0/1052996997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc20153bb0 msgr2=0x7efc200a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:56.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.259+0000 7efc27edb700 1 --2- 172.21.15.121:0/1052996997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc20153bb0 0x7efc200a9b30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7efc20069f00 tx=0x7efc1c009060).stop 2023-11-15T02:56:56.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.259+0000 7efc27edb700 1 -- 172.21.15.121:0/1052996997 shutdown_connections 2023-11-15T02:56:56.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.259+0000 7efc27edb700 1 --2- 172.21.15.121:0/1052996997 >> v2:172.21.15.121:3300/0 conn(0x7efc200aa020 0x7efc20067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.259+0000 7efc27edb700 1 --2- 172.21.15.121:0/1052996997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc20153bb0 0x7efc200a9b30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.259+0000 7efc27edb700 1 -- 172.21.15.121:0/1052996997 >> 172.21.15.121:0/1052996997 conn(0x7efc20058c50 msgr2=0x7efc20059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:56.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.259+0000 7efc27edb700 1 -- 172.21.15.121:0/1052996997 shutdown_connections 2023-11-15T02:56:56.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.259+0000 7efc27edb700 1 -- 172.21.15.121:0/1052996997 wait complete. 2023-11-15T02:56:56.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.263+0000 7efc27edb700 1 Processor -- start 2023-11-15T02:56:56.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.263+0000 7efc27edb700 1 -- start start 2023-11-15T02:56:56.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.263+0000 7efc27edb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc20067f50 0x7efc20152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:56.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.263+0000 7efc27edb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc200aa020 0x7efc201647b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:56.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.263+0000 7efc27edb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc20153bb0 0x7efc20157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:56.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.263+0000 7efc27edb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc20165770 con 0x7efc200aa020 2023-11-15T02:56:56.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.263+0000 7efc27edb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc20157a90 con 0x7efc20067f50 2023-11-15T02:56:56.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.263+0000 7efc27edb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc201651b0 con 0x7efc20153bb0 2023-11-15T02:56:56.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.263+0000 7efc25476700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc200aa020 0x7efc201647b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:56.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.263+0000 7efc25476700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc200aa020 0x7efc201647b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32966/0 (socket says 172.21.15.121:32966) 2023-11-15T02:56:56.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.263+0000 7efc25476700 1 -- 172.21.15.121:0/1835041107 learned_addr learned my addr 172.21.15.121:0/1835041107 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:56.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.263+0000 7efc25c77700 1 --2- 172.21.15.121:0/1835041107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc20067f50 0x7efc20152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:56.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.263+0000 7efc26478700 1 --2- 172.21.15.121:0/1835041107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc20153bb0 0x7efc20157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:56.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.263+0000 7efc25476700 1 -- 172.21.15.121:0/1835041107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc20153bb0 msgr2=0x7efc20157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:56.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.263+0000 7efc25476700 1 --2- 172.21.15.121:0/1835041107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc20153bb0 0x7efc20157f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.263+0000 7efc25476700 1 -- 172.21.15.121:0/1835041107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc20067f50 msgr2=0x7efc20152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:56.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.263+0000 7efc25476700 1 --2- 172.21.15.121:0/1835041107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc20067f50 0x7efc20152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.263+0000 7efc25476700 1 -- 172.21.15.121:0/1835041107 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c0092a0 con 0x7efc200aa020 2023-11-15T02:56:56.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.267+0000 7efc25476700 1 --2- 172.21.15.121:0/1835041107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc200aa020 0x7efc201647b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7efc1000e900 tx=0x7efc100090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:56.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.267+0000 7efc15ffb700 1 -- 172.21.15.121:0/1835041107 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10012600 con 0x7efc200aa020 2023-11-15T02:56:56.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.267+0000 7efc15ffb700 1 -- 172.21.15.121:0/1835041107 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc10012760 con 0x7efc200aa020 2023-11-15T02:56:56.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.267+0000 7efc27edb700 1 -- 172.21.15.121:0/1835041107 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc20165490 con 0x7efc200aa020 2023-11-15T02:56:56.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.267+0000 7efc15ffb700 1 -- 172.21.15.121:0/1835041107 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10012ae0 con 0x7efc200aa020 2023-11-15T02:56:56.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.267+0000 7efc27edb700 1 -- 172.21.15.121:0/1835041107 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc20152b60 con 0x7efc200aa020 2023-11-15T02:56:56.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.267+0000 7efc27edb700 1 -- 172.21.15.121:0/1835041107 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc20157580 con 0x7efc200aa020 2023-11-15T02:56:56.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.271+0000 7efc15ffb700 1 -- 172.21.15.121:0/1835041107 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7efc1001b400 con 0x7efc200aa020 2023-11-15T02:56:56.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.271+0000 7efc15ffb700 1 --2- 172.21.15.121:0/1835041107 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efc0405a540 0x7efc0405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:56.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.271+0000 7efc15ffb700 1 -- 172.21.15.121:0/1835041107 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efc10074ab0 con 0x7efc200aa020 2023-11-15T02:56:56.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.271+0000 7efc25c77700 1 --2- 172.21.15.121:0/1835041107 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efc0405a540 0x7efc0405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:56.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.271+0000 7efc25c77700 1 --2- 172.21.15.121:0/1835041107 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efc0405a540 0x7efc0405ca00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7efc200a98f0 tx=0x7efc1c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:56.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.275+0000 7efc15ffb700 1 -- 172.21.15.121:0/1835041107 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efc20157580 con 0x7efc200aa020 2023-11-15T02:56:56.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.435+0000 7efc27edb700 1 -- 172.21.15.121:0/1835041107 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc201578c0 con 0x7efc0405a540 2023-11-15T02:56:56.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.451+0000 7efc15ffb700 1 -- 172.21.15.121:0/1835041107 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336240 (secure 0 0 0) 0x7efc201578c0 con 0x7efc0405a540 2023-11-15T02:56:56.464 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:56:56.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.463+0000 7efc27edb700 1 -- 172.21.15.121:0/1835041107 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efc0405a540 msgr2=0x7efc0405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:56.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.463+0000 7efc27edb700 1 --2- 172.21.15.121:0/1835041107 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efc0405a540 0x7efc0405ca00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7efc200a98f0 tx=0x7efc1c017000).stop 2023-11-15T02:56:56.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.463+0000 7efc27edb700 1 -- 172.21.15.121:0/1835041107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc200aa020 msgr2=0x7efc201647b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:56.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.463+0000 7efc27edb700 1 --2- 172.21.15.121:0/1835041107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc200aa020 0x7efc201647b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7efc1000e900 tx=0x7efc100090f0).stop 2023-11-15T02:56:56.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.463+0000 7efc27edb700 1 -- 172.21.15.121:0/1835041107 shutdown_connections 2023-11-15T02:56:56.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.463+0000 7efc27edb700 1 --2- 172.21.15.121:0/1835041107 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efc0405a540 0x7efc0405ca00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.463+0000 7efc27edb700 1 --2- 172.21.15.121:0/1835041107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc20153bb0 0x7efc20157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.463+0000 7efc27edb700 1 --2- 172.21.15.121:0/1835041107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc200aa020 0x7efc201647b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.463+0000 7efc27edb700 1 --2- 172.21.15.121:0/1835041107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc20067f50 0x7efc20152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.463+0000 7efc27edb700 1 -- 172.21.15.121:0/1835041107 >> 172.21.15.121:0/1835041107 conn(0x7efc20058c50 msgr2=0x7efc200bfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:56.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.463+0000 7efc27edb700 1 -- 172.21.15.121:0/1835041107 shutdown_connections 2023-11-15T02:56:56.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.463+0000 7efc27edb700 1 -- 172.21.15.121:0/1835041107 wait complete. 2023-11-15T02:56:56.472 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T02:56:56.489 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-11-15T02:56:55.913138+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":256,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2378492,"kb_used_data":1724,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752596228,"statfs":{"total":773094113280,"available":770658537472,"internally_reserved":0,"allocated":1765376,"data_stored":635719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.736895+0000","last_change":"2023-11-15T02:56:51.733024+0000","last_active":"2023-11-15T02:56:51.736895+0000","last_peered":"2023-11-15T02:56:51.736895+0000","last_clean":"2023-11-15T02:56:51.736895+0000","last_became_active":"2023-11-15T02:56:51.732735+0000","last_became_peered":"2023-11-15T02:56:51.732735+0000","last_unstale":"2023-11-15T02:56:51.736895+0000","last_undegraded":"2023-11-15T02:56:51.736895+0000","last_fullsized":"2023-11-15T02:56:51.736895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.704467+0000","last_change":"2023-11-15T02:56:51.273328+0000","last_active":"2023-11-15T02:56:51.704467+0000","last_peered":"2023-11-15T02:56:51.704467+0000","last_clean":"2023-11-15T02:56:51.704467+0000","last_became_active":"2023-11-15T02:56:51.273044+0000","last_became_peered":"2023-11-15T02:56:51.273044+0000","last_unstale":"2023-11-15T02:56:51.704467+0000","last_undegraded":"2023-11-15T02:56:51.704467+0000","last_fullsized":"2023-11-15T02:56:51.704467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744601+0000","last_change":"2023-11-15T02:56:51.742242+0000","last_active":"2023-11-15T02:56:51.744601+0000","last_peered":"2023-11-15T02:56:51.744601+0000","last_clean":"2023-11-15T02:56:51.744601+0000","last_became_active":"2023-11-15T02:56:51.741604+0000","last_became_peered":"2023-11-15T02:56:51.741604+0000","last_unstale":"2023-11-15T02:56:51.744601+0000","last_undegraded":"2023-11-15T02:56:51.744601+0000","last_fullsized":"2023-11-15T02:56:51.744601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.739512+0000","last_change":"2023-11-15T02:56:51.277577+0000","last_active":"2023-11-15T02:56:51.739512+0000","last_peered":"2023-11-15T02:56:51.739512+0000","last_clean":"2023-11-15T02:56:51.739512+0000","last_became_active":"2023-11-15T02:56:51.276745+0000","last_became_peered":"2023-11-15T02:56:51.276745+0000","last_unstale":"2023-11-15T02:56:51.739512+0000","last_undegraded":"2023-11-15T02:56:51.739512+0000","last_fullsized":"2023-11-15T02:56:51.739512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.716112+0000","last_change":"2023-11-15T02:56:51.269238+0000","last_active":"2023-11-15T02:56:51.716112+0000","last_peered":"2023-11-15T02:56:51.716112+0000","last_clean":"2023-11-15T02:56:51.716112+0000","last_became_active":"2023-11-15T02:56:51.268872+0000","last_became_peered":"2023-11-15T02:56:51.268872+0000","last_unstale":"2023-11-15T02:56:51.716112+0000","last_undegraded":"2023-11-15T02:56:51.716112+0000","last_fullsized":"2023-11-15T02:56:51.716112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747275+0000","last_change":"2023-11-15T02:56:51.742674+0000","last_active":"2023-11-15T02:56:51.747275+0000","last_peered":"2023-11-15T02:56:51.747275+0000","last_clean":"2023-11-15T02:56:51.747275+0000","last_became_active":"2023-11-15T02:56:51.742412+0000","last_became_peered":"2023-11-15T02:56:51.742412+0000","last_unstale":"2023-11-15T02:56:51.747275+0000","last_undegraded":"2023-11-15T02:56:51.747275+0000","last_fullsized":"2023-11-15T02:56:51.747275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712980+0000","last_change":"2023-11-15T02:56:51.279772+0000","last_active":"2023-11-15T02:56:51.712980+0000","last_peered":"2023-11-15T02:56:51.712980+0000","last_clean":"2023-11-15T02:56:51.712980+0000","last_became_active":"2023-11-15T02:56:51.279423+0000","last_became_peered":"2023-11-15T02:56:51.279423+0000","last_unstale":"2023-11-15T02:56:51.712980+0000","last_undegraded":"2023-11-15T02:56:51.712980+0000","last_fullsized":"2023-11-15T02:56:51.712980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738160+0000","last_change":"2023-11-15T02:56:51.731913+0000","last_active":"2023-11-15T02:56:51.738160+0000","last_peered":"2023-11-15T02:56:51.738160+0000","last_clean":"2023-11-15T02:56:51.738160+0000","last_became_active":"2023-11-15T02:56:51.728632+0000","last_became_peered":"2023-11-15T02:56:51.728632+0000","last_unstale":"2023-11-15T02:56:51.738160+0000","last_undegraded":"2023-11-15T02:56:51.738160+0000","last_fullsized":"2023-11-15T02:56:51.738160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744972+0000","last_change":"2023-11-15T02:56:51.289877+0000","last_active":"2023-11-15T02:56:51.744972+0000","last_peered":"2023-11-15T02:56:51.744972+0000","last_clean":"2023-11-15T02:56:51.744972+0000","last_became_active":"2023-11-15T02:56:51.288940+0000","last_became_peered":"2023-11-15T02:56:51.288940+0000","last_unstale":"2023-11-15T02:56:51.744972+0000","last_undegraded":"2023-11-15T02:56:51.744972+0000","last_fullsized":"2023-11-15T02:56:51.744972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.743484+0000","last_change":"2023-11-15T02:56:51.738869+0000","last_active":"2023-11-15T02:56:51.743484+0000","last_peered":"2023-11-15T02:56:51.743484+0000","last_clean":"2023-11-15T02:56:51.743484+0000","last_became_active":"2023-11-15T02:56:51.738159+0000","last_became_peered":"2023-11-15T02:56:51.738159+0000","last_unstale":"2023-11-15T02:56:51.743484+0000","last_undegraded":"2023-11-15T02:56:51.743484+0000","last_fullsized":"2023-11-15T02:56:51.743484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713144+0000","last_change":"2023-11-15T02:56:51.284304+0000","last_active":"2023-11-15T02:56:51.713144+0000","last_peered":"2023-11-15T02:56:51.713144+0000","last_clean":"2023-11-15T02:56:51.713144+0000","last_became_active":"2023-11-15T02:56:51.284043+0000","last_became_peered":"2023-11-15T02:56:51.284043+0000","last_unstale":"2023-11-15T02:56:51.713144+0000","last_undegraded":"2023-11-15T02:56:51.713144+0000","last_fullsized":"2023-11-15T02:56:51.713144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747361+0000","last_change":"2023-11-15T02:56:51.744243+0000","last_active":"2023-11-15T02:56:51.747361+0000","last_peered":"2023-11-15T02:56:51.747361+0000","last_clean":"2023-11-15T02:56:51.747361+0000","last_became_active":"2023-11-15T02:56:51.743943+0000","last_became_peered":"2023-11-15T02:56:51.743943+0000","last_unstale":"2023-11-15T02:56:51.747361+0000","last_undegraded":"2023-11-15T02:56:51.747361+0000","last_fullsized":"2023-11-15T02:56:51.747361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.734769+0000","last_change":"2023-11-15T02:56:51.286898+0000","last_active":"2023-11-15T02:56:51.734769+0000","last_peered":"2023-11-15T02:56:51.734769+0000","last_clean":"2023-11-15T02:56:51.734769+0000","last_became_active":"2023-11-15T02:56:51.286628+0000","last_became_peered":"2023-11-15T02:56:51.286628+0000","last_unstale":"2023-11-15T02:56:51.734769+0000","last_undegraded":"2023-11-15T02:56:51.734769+0000","last_fullsized":"2023-11-15T02:56:51.734769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.749549+0000","last_change":"2023-11-15T02:56:51.745783+0000","last_active":"2023-11-15T02:56:51.749549+0000","last_peered":"2023-11-15T02:56:51.749549+0000","last_clean":"2023-11-15T02:56:51.749549+0000","last_became_active":"2023-11-15T02:56:51.745028+0000","last_became_peered":"2023-11-15T02:56:51.745028+0000","last_unstale":"2023-11-15T02:56:51.749549+0000","last_undegraded":"2023-11-15T02:56:51.749549+0000","last_fullsized":"2023-11-15T02:56:51.749549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.716637+0000","last_change":"2023-11-15T02:56:51.268012+0000","last_active":"2023-11-15T02:56:51.716637+0000","last_peered":"2023-11-15T02:56:51.716637+0000","last_clean":"2023-11-15T02:56:51.716637+0000","last_became_active":"2023-11-15T02:56:51.267734+0000","last_became_peered":"2023-11-15T02:56:51.267734+0000","last_unstale":"2023-11-15T02:56:51.716637+0000","last_undegraded":"2023-11-15T02:56:51.716637+0000","last_fullsized":"2023-11-15T02:56:51.716637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730642+0000","last_change":"2023-11-15T02:56:51.720753+0000","last_active":"2023-11-15T02:56:51.730642+0000","last_peered":"2023-11-15T02:56:51.730642+0000","last_clean":"2023-11-15T02:56:51.730642+0000","last_became_active":"2023-11-15T02:56:51.720500+0000","last_became_peered":"2023-11-15T02:56:51.720500+0000","last_unstale":"2023-11-15T02:56:51.730642+0000","last_undegraded":"2023-11-15T02:56:51.730642+0000","last_fullsized":"2023-11-15T02:56:51.730642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.711624+0000","last_change":"2023-11-15T02:56:51.280573+0000","last_active":"2023-11-15T02:56:51.711624+0000","last_peered":"2023-11-15T02:56:51.711624+0000","last_clean":"2023-11-15T02:56:51.711624+0000","last_became_active":"2023-11-15T02:56:51.275286+0000","last_became_peered":"2023-11-15T02:56:51.275286+0000","last_unstale":"2023-11-15T02:56:51.711624+0000","last_undegraded":"2023-11-15T02:56:51.711624+0000","last_fullsized":"2023-11-15T02:56:51.711624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.742388+0000","last_change":"2023-11-15T02:56:51.736450+0000","last_active":"2023-11-15T02:56:51.742388+0000","last_peered":"2023-11-15T02:56:51.742388+0000","last_clean":"2023-11-15T02:56:51.742388+0000","last_became_active":"2023-11-15T02:56:51.736143+0000","last_became_peered":"2023-11-15T02:56:51.736143+0000","last_unstale":"2023-11-15T02:56:51.742388+0000","last_undegraded":"2023-11-15T02:56:51.742388+0000","last_fullsized":"2023-11-15T02:56:51.742388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.714425+0000","last_change":"2023-11-15T02:56:51.284434+0000","last_active":"2023-11-15T02:56:51.714425+0000","last_peered":"2023-11-15T02:56:51.714425+0000","last_clean":"2023-11-15T02:56:51.714425+0000","last_became_active":"2023-11-15T02:56:51.284051+0000","last_became_peered":"2023-11-15T02:56:51.284051+0000","last_unstale":"2023-11-15T02:56:51.714425+0000","last_undegraded":"2023-11-15T02:56:51.714425+0000","last_fullsized":"2023-11-15T02:56:51.714425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744584+0000","last_change":"2023-11-15T02:56:51.742247+0000","last_active":"2023-11-15T02:56:51.744584+0000","last_peered":"2023-11-15T02:56:51.744584+0000","last_clean":"2023-11-15T02:56:51.744584+0000","last_became_active":"2023-11-15T02:56:51.741966+0000","last_became_peered":"2023-11-15T02:56:51.741966+0000","last_unstale":"2023-11-15T02:56:51.744584+0000","last_undegraded":"2023-11-15T02:56:51.744584+0000","last_fullsized":"2023-11-15T02:56:51.744584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.714493+0000","last_change":"2023-11-15T02:56:51.277678+0000","last_active":"2023-11-15T02:56:51.714493+0000","last_peered":"2023-11-15T02:56:51.714493+0000","last_clean":"2023-11-15T02:56:51.714493+0000","last_became_active":"2023-11-15T02:56:51.277408+0000","last_became_peered":"2023-11-15T02:56:51.277408+0000","last_unstale":"2023-11-15T02:56:51.714493+0000","last_undegraded":"2023-11-15T02:56:51.714493+0000","last_fullsized":"2023-11-15T02:56:51.714493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.731195+0000","last_change":"2023-11-15T02:56:51.717008+0000","last_active":"2023-11-15T02:56:51.731195+0000","last_peered":"2023-11-15T02:56:51.731195+0000","last_clean":"2023-11-15T02:56:51.731195+0000","last_became_active":"2023-11-15T02:56:51.716661+0000","last_became_peered":"2023-11-15T02:56:51.716661+0000","last_unstale":"2023-11-15T02:56:51.731195+0000","last_undegraded":"2023-11-15T02:56:51.731195+0000","last_fullsized":"2023-11-15T02:56:51.731195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713801+0000","last_change":"2023-11-15T02:56:51.281739+0000","last_active":"2023-11-15T02:56:51.713801+0000","last_peered":"2023-11-15T02:56:51.713801+0000","last_clean":"2023-11-15T02:56:51.713801+0000","last_became_active":"2023-11-15T02:56:51.281484+0000","last_became_peered":"2023-11-15T02:56:51.281484+0000","last_unstale":"2023-11-15T02:56:51.713801+0000","last_undegraded":"2023-11-15T02:56:51.713801+0000","last_fullsized":"2023-11-15T02:56:51.713801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738099+0000","last_change":"2023-11-15T02:56:51.726671+0000","last_active":"2023-11-15T02:56:51.738099+0000","last_peered":"2023-11-15T02:56:51.738099+0000","last_clean":"2023-11-15T02:56:51.738099+0000","last_became_active":"2023-11-15T02:56:51.725108+0000","last_became_peered":"2023-11-15T02:56:51.725108+0000","last_unstale":"2023-11-15T02:56:51.738099+0000","last_undegraded":"2023-11-15T02:56:51.738099+0000","last_fullsized":"2023-11-15T02:56:51.738099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.704634+0000","last_change":"2023-11-15T02:56:51.271748+0000","last_active":"2023-11-15T02:56:51.704634+0000","last_peered":"2023-11-15T02:56:51.704634+0000","last_clean":"2023-11-15T02:56:51.704634+0000","last_became_active":"2023-11-15T02:56:51.271310+0000","last_became_peered":"2023-11-15T02:56:51.271310+0000","last_unstale":"2023-11-15T02:56:51.704634+0000","last_undegraded":"2023-11-15T02:56:51.704634+0000","last_fullsized":"2023-11-15T02:56:51.704634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744179+0000","last_change":"2023-11-15T02:56:51.737961+0000","last_active":"2023-11-15T02:56:51.744179+0000","last_peered":"2023-11-15T02:56:51.744179+0000","last_clean":"2023-11-15T02:56:51.744179+0000","last_became_active":"2023-11-15T02:56:51.737603+0000","last_became_peered":"2023-11-15T02:56:51.737603+0000","last_unstale":"2023-11-15T02:56:51.744179+0000","last_undegraded":"2023-11-15T02:56:51.744179+0000","last_fullsized":"2023-11-15T02:56:51.744179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.711878+0000","last_change":"2023-11-15T02:56:51.286344+0000","last_active":"2023-11-15T02:56:51.711878+0000","last_peered":"2023-11-15T02:56:51.711878+0000","last_clean":"2023-11-15T02:56:51.711878+0000","last_became_active":"2023-11-15T02:56:51.286050+0000","last_became_peered":"2023-11-15T02:56:51.286050+0000","last_unstale":"2023-11-15T02:56:51.711878+0000","last_undegraded":"2023-11-15T02:56:51.711878+0000","last_fullsized":"2023-11-15T02:56:51.711878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.740796+0000","last_change":"2023-11-15T02:56:51.729804+0000","last_active":"2023-11-15T02:56:51.740796+0000","last_peered":"2023-11-15T02:56:51.740796+0000","last_clean":"2023-11-15T02:56:51.740796+0000","last_became_active":"2023-11-15T02:56:51.729569+0000","last_became_peered":"2023-11-15T02:56:51.729569+0000","last_unstale":"2023-11-15T02:56:51.740796+0000","last_undegraded":"2023-11-15T02:56:51.740796+0000","last_fullsized":"2023-11-15T02:56:51.740796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.721025+0000","last_change":"2023-11-15T02:56:51.280678+0000","last_active":"2023-11-15T02:56:51.721025+0000","last_peered":"2023-11-15T02:56:51.721025+0000","last_clean":"2023-11-15T02:56:51.721025+0000","last_became_active":"2023-11-15T02:56:51.279110+0000","last_became_peered":"2023-11-15T02:56:51.279110+0000","last_unstale":"2023-11-15T02:56:51.721025+0000","last_undegraded":"2023-11-15T02:56:51.721025+0000","last_fullsized":"2023-11-15T02:56:51.721025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.728488+0000","last_change":"2023-11-15T02:56:51.726563+0000","last_active":"2023-11-15T02:56:51.728488+0000","last_peered":"2023-11-15T02:56:51.728488+0000","last_clean":"2023-11-15T02:56:51.728488+0000","last_became_active":"2023-11-15T02:56:51.726284+0000","last_became_peered":"2023-11-15T02:56:51.726284+0000","last_unstale":"2023-11-15T02:56:51.728488+0000","last_undegraded":"2023-11-15T02:56:51.728488+0000","last_fullsized":"2023-11-15T02:56:51.728488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.716713+0000","last_change":"2023-11-15T02:56:51.275575+0000","last_active":"2023-11-15T02:56:51.716713+0000","last_peered":"2023-11-15T02:56:51.716713+0000","last_clean":"2023-11-15T02:56:51.716713+0000","last_became_active":"2023-11-15T02:56:51.275267+0000","last_became_peered":"2023-11-15T02:56:51.275267+0000","last_unstale":"2023-11-15T02:56:51.716713+0000","last_undegraded":"2023-11-15T02:56:51.716713+0000","last_fullsized":"2023-11-15T02:56:51.716713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.732898+0000","last_change":"2023-11-15T02:56:51.728149+0000","last_active":"2023-11-15T02:56:51.732898+0000","last_peered":"2023-11-15T02:56:51.732898+0000","last_clean":"2023-11-15T02:56:51.732898+0000","last_became_active":"2023-11-15T02:56:51.727553+0000","last_became_peered":"2023-11-15T02:56:51.727553+0000","last_unstale":"2023-11-15T02:56:51.732898+0000","last_undegraded":"2023-11-15T02:56:51.732898+0000","last_fullsized":"2023-11-15T02:56:51.732898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712886+0000","last_change":"2023-11-15T02:56:51.269197+0000","last_active":"2023-11-15T02:56:51.712886+0000","last_peered":"2023-11-15T02:56:51.712886+0000","last_clean":"2023-11-15T02:56:51.712886+0000","last_became_active":"2023-11-15T02:56:51.268904+0000","last_became_peered":"2023-11-15T02:56:51.268904+0000","last_unstale":"2023-11-15T02:56:51.712886+0000","last_undegraded":"2023-11-15T02:56:51.712886+0000","last_fullsized":"2023-11-15T02:56:51.712886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.724341+0000","last_change":"2023-11-15T02:56:51.715489+0000","last_active":"2023-11-15T02:56:51.724341+0000","last_peered":"2023-11-15T02:56:51.724341+0000","last_clean":"2023-11-15T02:56:51.724341+0000","last_became_active":"2023-11-15T02:56:51.715060+0000","last_became_peered":"2023-11-15T02:56:51.715060+0000","last_unstale":"2023-11-15T02:56:51.724341+0000","last_undegraded":"2023-11-15T02:56:51.724341+0000","last_fullsized":"2023-11-15T02:56:51.724341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712366+0000","last_change":"2023-11-15T02:56:51.276645+0000","last_active":"2023-11-15T02:56:51.712366+0000","last_peered":"2023-11-15T02:56:51.712366+0000","last_clean":"2023-11-15T02:56:51.712366+0000","last_became_active":"2023-11-15T02:56:51.276356+0000","last_became_peered":"2023-11-15T02:56:51.276356+0000","last_unstale":"2023-11-15T02:56:51.712366+0000","last_undegraded":"2023-11-15T02:56:51.712366+0000","last_fullsized":"2023-11-15T02:56:51.712366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.724735+0000","last_change":"2023-11-15T02:56:51.719961+0000","last_active":"2023-11-15T02:56:51.724735+0000","last_peered":"2023-11-15T02:56:51.724735+0000","last_clean":"2023-11-15T02:56:51.724735+0000","last_became_active":"2023-11-15T02:56:51.719598+0000","last_became_peered":"2023-11-15T02:56:51.719598+0000","last_unstale":"2023-11-15T02:56:51.724735+0000","last_undegraded":"2023-11-15T02:56:51.724735+0000","last_fullsized":"2023-11-15T02:56:51.724735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.734513+0000","last_change":"2023-11-15T02:56:51.287275+0000","last_active":"2023-11-15T02:56:51.734513+0000","last_peered":"2023-11-15T02:56:51.734513+0000","last_clean":"2023-11-15T02:56:51.734513+0000","last_became_active":"2023-11-15T02:56:51.286259+0000","last_became_peered":"2023-11-15T02:56:51.286259+0000","last_unstale":"2023-11-15T02:56:51.734513+0000","last_undegraded":"2023-11-15T02:56:51.734513+0000","last_fullsized":"2023-11-15T02:56:51.734513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.751632+0000","last_change":"2023-11-15T02:56:51.748332+0000","last_active":"2023-11-15T02:56:51.751632+0000","last_peered":"2023-11-15T02:56:51.751632+0000","last_clean":"2023-11-15T02:56:51.751632+0000","last_became_active":"2023-11-15T02:56:51.747865+0000","last_became_peered":"2023-11-15T02:56:51.747865+0000","last_unstale":"2023-11-15T02:56:51.751632+0000","last_undegraded":"2023-11-15T02:56:51.751632+0000","last_fullsized":"2023-11-15T02:56:51.751632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746250+0000","last_change":"2023-11-15T02:56:51.274812+0000","last_active":"2023-11-15T02:56:51.746250+0000","last_peered":"2023-11-15T02:56:51.746250+0000","last_clean":"2023-11-15T02:56:51.746250+0000","last_became_active":"2023-11-15T02:56:51.273693+0000","last_became_peered":"2023-11-15T02:56:51.273693+0000","last_unstale":"2023-11-15T02:56:51.746250+0000","last_undegraded":"2023-11-15T02:56:51.746250+0000","last_fullsized":"2023-11-15T02:56:51.746250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.734600+0000","last_change":"2023-11-15T02:56:51.718982+0000","last_active":"2023-11-15T02:56:51.734600+0000","last_peered":"2023-11-15T02:56:51.734600+0000","last_clean":"2023-11-15T02:56:51.734600+0000","last_became_active":"2023-11-15T02:56:51.718702+0000","last_became_peered":"2023-11-15T02:56:51.718702+0000","last_unstale":"2023-11-15T02:56:51.734600+0000","last_undegraded":"2023-11-15T02:56:51.734600+0000","last_fullsized":"2023-11-15T02:56:51.734600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.743614+0000","last_change":"2023-11-15T02:56:51.285434+0000","last_active":"2023-11-15T02:56:51.743614+0000","last_peered":"2023-11-15T02:56:51.743614+0000","last_clean":"2023-11-15T02:56:51.743614+0000","last_became_active":"2023-11-15T02:56:51.284984+0000","last_became_peered":"2023-11-15T02:56:51.284984+0000","last_unstale":"2023-11-15T02:56:51.743614+0000","last_undegraded":"2023-11-15T02:56:51.743614+0000","last_fullsized":"2023-11-15T02:56:51.743614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744214+0000","last_change":"2023-11-15T02:56:51.740534+0000","last_active":"2023-11-15T02:56:51.744214+0000","last_peered":"2023-11-15T02:56:51.744214+0000","last_clean":"2023-11-15T02:56:51.744214+0000","last_became_active":"2023-11-15T02:56:51.740090+0000","last_became_peered":"2023-11-15T02:56:51.740090+0000","last_unstale":"2023-11-15T02:56:51.744214+0000","last_undegraded":"2023-11-15T02:56:51.744214+0000","last_fullsized":"2023-11-15T02:56:51.744214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.714872+0000","last_change":"2023-11-15T02:56:51.279489+0000","last_active":"2023-11-15T02:56:51.714872+0000","last_peered":"2023-11-15T02:56:51.714872+0000","last_clean":"2023-11-15T02:56:51.714872+0000","last_became_active":"2023-11-15T02:56:51.279222+0000","last_became_peered":"2023-11-15T02:56:51.279222+0000","last_unstale":"2023-11-15T02:56:51.714872+0000","last_undegraded":"2023-11-15T02:56:51.714872+0000","last_fullsized":"2023-11-15T02:56:51.714872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.749518+0000","last_change":"2023-11-15T02:56:51.744824+0000","last_active":"2023-11-15T02:56:51.749518+0000","last_peered":"2023-11-15T02:56:51.749518+0000","last_clean":"2023-11-15T02:56:51.749518+0000","last_became_active":"2023-11-15T02:56:51.744533+0000","last_became_peered":"2023-11-15T02:56:51.744533+0000","last_unstale":"2023-11-15T02:56:51.749518+0000","last_undegraded":"2023-11-15T02:56:51.749518+0000","last_fullsized":"2023-11-15T02:56:51.749518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.721137+0000","last_change":"2023-11-15T02:56:51.273142+0000","last_active":"2023-11-15T02:56:51.721137+0000","last_peered":"2023-11-15T02:56:51.721137+0000","last_clean":"2023-11-15T02:56:51.721137+0000","last_became_active":"2023-11-15T02:56:51.272861+0000","last_became_peered":"2023-11-15T02:56:51.272861+0000","last_unstale":"2023-11-15T02:56:51.721137+0000","last_undegraded":"2023-11-15T02:56:51.721137+0000","last_fullsized":"2023-11-15T02:56:51.721137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.735105+0000","last_change":"2023-11-15T02:56:51.728416+0000","last_active":"2023-11-15T02:56:51.735105+0000","last_peered":"2023-11-15T02:56:51.735105+0000","last_clean":"2023-11-15T02:56:51.735105+0000","last_became_active":"2023-11-15T02:56:51.728084+0000","last_became_peered":"2023-11-15T02:56:51.728084+0000","last_unstale":"2023-11-15T02:56:51.735105+0000","last_undegraded":"2023-11-15T02:56:51.735105+0000","last_fullsized":"2023-11-15T02:56:51.735105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.724100+0000","last_change":"2023-11-15T02:56:51.274354+0000","last_active":"2023-11-15T02:56:51.724100+0000","last_peered":"2023-11-15T02:56:51.724100+0000","last_clean":"2023-11-15T02:56:51.724100+0000","last_became_active":"2023-11-15T02:56:51.274083+0000","last_became_peered":"2023-11-15T02:56:51.274083+0000","last_unstale":"2023-11-15T02:56:51.724100+0000","last_undegraded":"2023-11-15T02:56:51.724100+0000","last_fullsized":"2023-11-15T02:56:51.724100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.736974+0000","last_change":"2023-11-15T02:56:51.721714+0000","last_active":"2023-11-15T02:56:51.736974+0000","last_peered":"2023-11-15T02:56:51.736974+0000","last_clean":"2023-11-15T02:56:51.736974+0000","last_became_active":"2023-11-15T02:56:51.721415+0000","last_became_peered":"2023-11-15T02:56:51.721415+0000","last_unstale":"2023-11-15T02:56:51.736974+0000","last_undegraded":"2023-11-15T02:56:51.736974+0000","last_fullsized":"2023-11-15T02:56:51.736974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.704644+0000","last_change":"2023-11-15T02:56:51.283397+0000","last_active":"2023-11-15T02:56:51.704644+0000","last_peered":"2023-11-15T02:56:51.704644+0000","last_clean":"2023-11-15T02:56:51.704644+0000","last_became_active":"2023-11-15T02:56:51.283101+0000","last_became_peered":"2023-11-15T02:56:51.283101+0000","last_unstale":"2023-11-15T02:56:51.704644+0000","last_undegraded":"2023-11-15T02:56:51.704644+0000","last_fullsized":"2023-11-15T02:56:51.704644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.735611+0000","last_change":"2023-11-15T02:56:51.727142+0000","last_active":"2023-11-15T02:56:51.735611+0000","last_peered":"2023-11-15T02:56:51.735611+0000","last_clean":"2023-11-15T02:56:51.735611+0000","last_became_active":"2023-11-15T02:56:51.726872+0000","last_became_peered":"2023-11-15T02:56:51.726872+0000","last_unstale":"2023-11-15T02:56:51.735611+0000","last_undegraded":"2023-11-15T02:56:51.735611+0000","last_fullsized":"2023-11-15T02:56:51.735611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.710839+0000","last_change":"2023-11-15T02:56:51.281104+0000","last_active":"2023-11-15T02:56:51.710839+0000","last_peered":"2023-11-15T02:56:51.710839+0000","last_clean":"2023-11-15T02:56:51.710839+0000","last_became_active":"2023-11-15T02:56:51.280784+0000","last_became_peered":"2023-11-15T02:56:51.280784+0000","last_unstale":"2023-11-15T02:56:51.710839+0000","last_undegraded":"2023-11-15T02:56:51.710839+0000","last_fullsized":"2023-11-15T02:56:51.710839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.737357+0000","last_change":"2023-11-15T02:56:51.719121+0000","last_active":"2023-11-15T02:56:51.737357+0000","last_peered":"2023-11-15T02:56:51.737357+0000","last_clean":"2023-11-15T02:56:51.737357+0000","last_became_active":"2023-11-15T02:56:51.718812+0000","last_became_peered":"2023-11-15T02:56:51.718812+0000","last_unstale":"2023-11-15T02:56:51.737357+0000","last_undegraded":"2023-11-15T02:56:51.737357+0000","last_fullsized":"2023-11-15T02:56:51.737357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.722994+0000","last_change":"2023-11-15T02:56:51.285977+0000","last_active":"2023-11-15T02:56:51.722994+0000","last_peered":"2023-11-15T02:56:51.722994+0000","last_clean":"2023-11-15T02:56:51.722994+0000","last_became_active":"2023-11-15T02:56:51.277431+0000","last_became_peered":"2023-11-15T02:56:51.277431+0000","last_unstale":"2023-11-15T02:56:51.722994+0000","last_undegraded":"2023-11-15T02:56:51.722994+0000","last_fullsized":"2023-11-15T02:56:51.722994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.735350+0000","last_change":"2023-11-15T02:56:51.720775+0000","last_active":"2023-11-15T02:56:51.735350+0000","last_peered":"2023-11-15T02:56:51.735350+0000","last_clean":"2023-11-15T02:56:51.735350+0000","last_became_active":"2023-11-15T02:56:51.720446+0000","last_became_peered":"2023-11-15T02:56:51.720446+0000","last_unstale":"2023-11-15T02:56:51.735350+0000","last_undegraded":"2023-11-15T02:56:51.735350+0000","last_fullsized":"2023-11-15T02:56:51.735350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.715531+0000","last_change":"2023-11-15T02:56:51.268303+0000","last_active":"2023-11-15T02:56:51.715531+0000","last_peered":"2023-11-15T02:56:51.715531+0000","last_clean":"2023-11-15T02:56:51.715531+0000","last_became_active":"2023-11-15T02:56:51.267982+0000","last_became_peered":"2023-11-15T02:56:51.267982+0000","last_unstale":"2023-11-15T02:56:51.715531+0000","last_undegraded":"2023-11-15T02:56:51.715531+0000","last_fullsized":"2023-11-15T02:56:51.715531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738120+0000","last_change":"2023-11-15T02:56:51.726853+0000","last_active":"2023-11-15T02:56:51.738120+0000","last_peered":"2023-11-15T02:56:51.738120+0000","last_clean":"2023-11-15T02:56:51.738120+0000","last_became_active":"2023-11-15T02:56:51.726448+0000","last_became_peered":"2023-11-15T02:56:51.726448+0000","last_unstale":"2023-11-15T02:56:51.738120+0000","last_undegraded":"2023-11-15T02:56:51.738120+0000","last_fullsized":"2023-11-15T02:56:51.738120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745254+0000","last_change":"2023-11-15T02:56:51.291178+0000","last_active":"2023-11-15T02:56:51.745254+0000","last_peered":"2023-11-15T02:56:51.745254+0000","last_clean":"2023-11-15T02:56:51.745254+0000","last_became_active":"2023-11-15T02:56:51.289245+0000","last_became_peered":"2023-11-15T02:56:51.289245+0000","last_unstale":"2023-11-15T02:56:51.745254+0000","last_undegraded":"2023-11-15T02:56:51.745254+0000","last_fullsized":"2023-11-15T02:56:51.745254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.718417+0000","last_change":"2023-11-15T02:56:51.714302+0000","last_active":"2023-11-15T02:56:51.718417+0000","last_peered":"2023-11-15T02:56:51.718417+0000","last_clean":"2023-11-15T02:56:51.718417+0000","last_became_active":"2023-11-15T02:56:51.714023+0000","last_became_peered":"2023-11-15T02:56:51.714023+0000","last_unstale":"2023-11-15T02:56:51.718417+0000","last_undegraded":"2023-11-15T02:56:51.718417+0000","last_fullsized":"2023-11-15T02:56:51.718417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746516+0000","last_change":"2023-11-15T02:56:51.265289+0000","last_active":"2023-11-15T02:56:51.746516+0000","last_peered":"2023-11-15T02:56:51.746516+0000","last_clean":"2023-11-15T02:56:51.746516+0000","last_became_active":"2023-11-15T02:56:51.264945+0000","last_became_peered":"2023-11-15T02:56:51.264945+0000","last_unstale":"2023-11-15T02:56:51.746516+0000","last_undegraded":"2023-11-15T02:56:51.746516+0000","last_fullsized":"2023-11-15T02:56:51.746516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.751653+0000","last_change":"2023-11-15T02:56:51.748696+0000","last_active":"2023-11-15T02:56:51.751653+0000","last_peered":"2023-11-15T02:56:51.751653+0000","last_clean":"2023-11-15T02:56:51.751653+0000","last_became_active":"2023-11-15T02:56:51.748403+0000","last_became_peered":"2023-11-15T02:56:51.748403+0000","last_unstale":"2023-11-15T02:56:51.751653+0000","last_undegraded":"2023-11-15T02:56:51.751653+0000","last_fullsized":"2023-11-15T02:56:51.751653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713322+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T02:56:51.713322+0000","last_peered":"2023-11-15T02:56:51.713322+0000","last_clean":"2023-11-15T02:56:51.713322+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T02:56:51.713322+0000","last_undegraded":"2023-11-15T02:56:51.713322+0000","last_fullsized":"2023-11-15T02:56:51.713322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.722937+0000","last_change":"2023-11-15T02:56:51.270301+0000","last_active":"2023-11-15T02:56:51.722937+0000","last_peered":"2023-11-15T02:56:51.722937+0000","last_clean":"2023-11-15T02:56:51.722937+0000","last_became_active":"2023-11-15T02:56:51.269996+0000","last_became_peered":"2023-11-15T02:56:51.269996+0000","last_unstale":"2023-11-15T02:56:51.722937+0000","last_undegraded":"2023-11-15T02:56:51.722937+0000","last_fullsized":"2023-11-15T02:56:51.722937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.742366+0000","last_change":"2023-11-15T02:56:51.736621+0000","last_active":"2023-11-15T02:56:51.742366+0000","last_peered":"2023-11-15T02:56:51.742366+0000","last_clean":"2023-11-15T02:56:51.742366+0000","last_became_active":"2023-11-15T02:56:51.736319+0000","last_became_peered":"2023-11-15T02:56:51.736319+0000","last_unstale":"2023-11-15T02:56:51.742366+0000","last_undegraded":"2023-11-15T02:56:51.742366+0000","last_fullsized":"2023-11-15T02:56:51.742366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.706577+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T02:56:51.706577+0000","last_peered":"2023-11-15T02:56:51.706577+0000","last_clean":"2023-11-15T02:56:51.706577+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T02:56:51.706577+0000","last_undegraded":"2023-11-15T02:56:51.706577+0000","last_fullsized":"2023-11-15T02:56:51.706577+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745714+0000","last_change":"2023-11-15T02:56:51.273274+0000","last_active":"2023-11-15T02:56:51.745714+0000","last_peered":"2023-11-15T02:56:51.745714+0000","last_clean":"2023-11-15T02:56:51.745714+0000","last_became_active":"2023-11-15T02:56:51.270655+0000","last_became_peered":"2023-11-15T02:56:51.270655+0000","last_unstale":"2023-11-15T02:56:51.745714+0000","last_undegraded":"2023-11-15T02:56:51.745714+0000","last_fullsized":"2023-11-15T02:56:51.745714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738809+0000","last_change":"2023-11-15T02:56:51.734478+0000","last_active":"2023-11-15T02:56:51.738809+0000","last_peered":"2023-11-15T02:56:51.738809+0000","last_clean":"2023-11-15T02:56:51.738809+0000","last_became_active":"2023-11-15T02:56:51.732194+0000","last_became_peered":"2023-11-15T02:56:51.732194+0000","last_unstale":"2023-11-15T02:56:51.738809+0000","last_undegraded":"2023-11-15T02:56:51.738809+0000","last_fullsized":"2023-11-15T02:56:51.738809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730730+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T02:56:51.730730+0000","last_peered":"2023-11-15T02:56:51.730730+0000","last_clean":"2023-11-15T02:56:51.730730+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T02:56:51.730730+0000","last_undegraded":"2023-11-15T02:56:51.730730+0000","last_fullsized":"2023-11-15T02:56:51.730730+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.705716+0000","last_change":"2023-11-15T02:56:51.270161+0000","last_active":"2023-11-15T02:56:51.705716+0000","last_peered":"2023-11-15T02:56:51.705716+0000","last_clean":"2023-11-15T02:56:51.705716+0000","last_became_active":"2023-11-15T02:56:51.268151+0000","last_became_peered":"2023-11-15T02:56:51.268151+0000","last_unstale":"2023-11-15T02:56:51.705716+0000","last_undegraded":"2023-11-15T02:56:51.705716+0000","last_fullsized":"2023-11-15T02:56:51.705716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.737232+0000","last_change":"2023-11-15T02:56:51.731375+0000","last_active":"2023-11-15T02:56:51.737232+0000","last_peered":"2023-11-15T02:56:51.737232+0000","last_clean":"2023-11-15T02:56:51.737232+0000","last_became_active":"2023-11-15T02:56:51.731100+0000","last_became_peered":"2023-11-15T02:56:51.731100+0000","last_unstale":"2023-11-15T02:56:51.737232+0000","last_undegraded":"2023-11-15T02:56:51.737232+0000","last_fullsized":"2023-11-15T02:56:51.737232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.705942+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T02:56:51.705942+0000","last_peered":"2023-11-15T02:56:51.705942+0000","last_clean":"2023-11-15T02:56:51.705942+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T02:56:51.705942+0000","last_undegraded":"2023-11-15T02:56:51.705942+0000","last_fullsized":"2023-11-15T02:56:51.705942+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.715265+0000","last_change":"2023-11-15T02:56:51.275134+0000","last_active":"2023-11-15T02:56:51.715265+0000","last_peered":"2023-11-15T02:56:51.715265+0000","last_clean":"2023-11-15T02:56:51.715265+0000","last_became_active":"2023-11-15T02:56:51.274400+0000","last_became_peered":"2023-11-15T02:56:51.274400+0000","last_unstale":"2023-11-15T02:56:51.715265+0000","last_undegraded":"2023-11-15T02:56:51.715265+0000","last_fullsized":"2023-11-15T02:56:51.715265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.727321+0000","last_change":"2023-11-15T02:56:51.724465+0000","last_active":"2023-11-15T02:56:51.727321+0000","last_peered":"2023-11-15T02:56:51.727321+0000","last_clean":"2023-11-15T02:56:51.727321+0000","last_became_active":"2023-11-15T02:56:51.723590+0000","last_became_peered":"2023-11-15T02:56:51.723590+0000","last_unstale":"2023-11-15T02:56:51.727321+0000","last_undegraded":"2023-11-15T02:56:51.727321+0000","last_fullsized":"2023-11-15T02:56:51.727321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.731884+0000","last_change":"2023-11-15T02:56:51.282484+0000","last_active":"2023-11-15T02:56:51.731884+0000","last_peered":"2023-11-15T02:56:51.731884+0000","last_clean":"2023-11-15T02:56:51.731884+0000","last_became_active":"2023-11-15T02:56:51.282108+0000","last_became_peered":"2023-11-15T02:56:51.282108+0000","last_unstale":"2023-11-15T02:56:51.731884+0000","last_undegraded":"2023-11-15T02:56:51.731884+0000","last_fullsized":"2023-11-15T02:56:51.731884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.716774+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T02:56:51.716774+0000","last_peered":"2023-11-15T02:56:51.716774+0000","last_clean":"2023-11-15T02:56:51.716774+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T02:56:51.716774+0000","last_undegraded":"2023-11-15T02:56:51.716774+0000","last_fullsized":"2023-11-15T02:56:51.716774+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.719253+0000","last_change":"2023-11-15T02:56:51.717765+0000","last_active":"2023-11-15T02:56:51.719253+0000","last_peered":"2023-11-15T02:56:51.719253+0000","last_clean":"2023-11-15T02:56:51.719253+0000","last_became_active":"2023-11-15T02:56:51.717498+0000","last_became_peered":"2023-11-15T02:56:51.717498+0000","last_unstale":"2023-11-15T02:56:51.719253+0000","last_undegraded":"2023-11-15T02:56:51.719253+0000","last_fullsized":"2023-11-15T02:56:51.719253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.704337+0000","last_change":"2023-11-15T02:56:51.267133+0000","last_active":"2023-11-15T02:56:51.704337+0000","last_peered":"2023-11-15T02:56:51.704337+0000","last_clean":"2023-11-15T02:56:51.704337+0000","last_became_active":"2023-11-15T02:56:51.266864+0000","last_became_peered":"2023-11-15T02:56:51.266864+0000","last_unstale":"2023-11-15T02:56:51.704337+0000","last_undegraded":"2023-11-15T02:56:51.704337+0000","last_fullsized":"2023-11-15T02:56:51.704337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712961+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T02:56:51.712961+0000","last_peered":"2023-11-15T02:56:51.712961+0000","last_clean":"2023-11-15T02:56:51.712961+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T02:56:51.712961+0000","last_undegraded":"2023-11-15T02:56:51.712961+0000","last_fullsized":"2023-11-15T02:56:51.712961+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.719804+0000","last_change":"2023-11-15T02:56:51.717744+0000","last_active":"2023-11-15T02:56:51.719804+0000","last_peered":"2023-11-15T02:56:51.719804+0000","last_clean":"2023-11-15T02:56:51.719804+0000","last_became_active":"2023-11-15T02:56:51.717448+0000","last_became_peered":"2023-11-15T02:56:51.717448+0000","last_unstale":"2023-11-15T02:56:51.719804+0000","last_undegraded":"2023-11-15T02:56:51.719804+0000","last_fullsized":"2023-11-15T02:56:51.719804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.718982+0000","last_change":"2023-11-15T02:56:51.273152+0000","last_active":"2023-11-15T02:56:51.718982+0000","last_peered":"2023-11-15T02:56:51.718982+0000","last_clean":"2023-11-15T02:56:51.718982+0000","last_became_active":"2023-11-15T02:56:51.272868+0000","last_became_peered":"2023-11-15T02:56:51.272868+0000","last_unstale":"2023-11-15T02:56:51.718982+0000","last_undegraded":"2023-11-15T02:56:51.718982+0000","last_fullsized":"2023-11-15T02:56:51.718982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.728384+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T02:56:51.728384+0000","last_peered":"2023-11-15T02:56:51.728384+0000","last_clean":"2023-11-15T02:56:51.728384+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T02:56:51.728384+0000","last_undegraded":"2023-11-15T02:56:51.728384+0000","last_fullsized":"2023-11-15T02:56:51.728384+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744430+0000","last_change":"2023-11-15T02:56:51.739786+0000","last_active":"2023-11-15T02:56:51.744430+0000","last_peered":"2023-11-15T02:56:51.744430+0000","last_clean":"2023-11-15T02:56:51.744430+0000","last_became_active":"2023-11-15T02:56:51.737659+0000","last_became_peered":"2023-11-15T02:56:51.737659+0000","last_unstale":"2023-11-15T02:56:51.744430+0000","last_undegraded":"2023-11-15T02:56:51.744430+0000","last_fullsized":"2023-11-15T02:56:51.744430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.711641+0000","last_change":"2023-11-15T02:56:51.276817+0000","last_active":"2023-11-15T02:56:51.711641+0000","last_peered":"2023-11-15T02:56:51.711641+0000","last_clean":"2023-11-15T02:56:51.711641+0000","last_became_active":"2023-11-15T02:56:51.276490+0000","last_became_peered":"2023-11-15T02:56:51.276490+0000","last_unstale":"2023-11-15T02:56:51.711641+0000","last_undegraded":"2023-11-15T02:56:51.711641+0000","last_fullsized":"2023-11-15T02:56:51.711641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713989+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T02:56:51.713989+0000","last_peered":"2023-11-15T02:56:51.713989+0000","last_clean":"2023-11-15T02:56:51.713989+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T02:56:51.713989+0000","last_undegraded":"2023-11-15T02:56:51.713989+0000","last_fullsized":"2023-11-15T02:56:51.713989+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.728465+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T02:56:51.728465+0000","last_peered":"2023-11-15T02:56:51.728465+0000","last_clean":"2023-11-15T02:56:51.728465+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T02:56:51.728465+0000","last_undegraded":"2023-11-15T02:56:51.728465+0000","last_fullsized":"2023-11-15T02:56:51.728465+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.749349+0000","last_change":"2023-11-15T02:56:51.745538+0000","last_active":"2023-11-15T02:56:51.749349+0000","last_peered":"2023-11-15T02:56:51.749349+0000","last_clean":"2023-11-15T02:56:51.749349+0000","last_became_active":"2023-11-15T02:56:51.745369+0000","last_became_peered":"2023-11-15T02:56:51.745369+0000","last_unstale":"2023-11-15T02:56:51.749349+0000","last_undegraded":"2023-11-15T02:56:51.749349+0000","last_fullsized":"2023-11-15T02:56:51.749349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.710979+0000","last_change":"2023-11-15T02:56:51.280484+0000","last_active":"2023-11-15T02:56:51.710979+0000","last_peered":"2023-11-15T02:56:51.710979+0000","last_clean":"2023-11-15T02:56:51.710979+0000","last_became_active":"2023-11-15T02:56:51.272264+0000","last_became_peered":"2023-11-15T02:56:51.272264+0000","last_unstale":"2023-11-15T02:56:51.710979+0000","last_undegraded":"2023-11-15T02:56:51.710979+0000","last_fullsized":"2023-11-15T02:56:51.710979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.724717+0000","last_change":"2023-11-15T02:56:51.719709+0000","last_active":"2023-11-15T02:56:51.724717+0000","last_peered":"2023-11-15T02:56:51.724717+0000","last_clean":"2023-11-15T02:56:51.724717+0000","last_became_active":"2023-11-15T02:56:51.719230+0000","last_became_peered":"2023-11-15T02:56:51.719230+0000","last_unstale":"2023-11-15T02:56:51.724717+0000","last_undegraded":"2023-11-15T02:56:51.724717+0000","last_fullsized":"2023-11-15T02:56:51.724717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.735953+0000","last_change":"2023-11-15T02:56:51.272978+0000","last_active":"2023-11-15T02:56:51.735953+0000","last_peered":"2023-11-15T02:56:51.735953+0000","last_clean":"2023-11-15T02:56:51.735953+0000","last_became_active":"2023-11-15T02:56:51.272650+0000","last_became_peered":"2023-11-15T02:56:51.272650+0000","last_unstale":"2023-11-15T02:56:51.735953+0000","last_undegraded":"2023-11-15T02:56:51.735953+0000","last_fullsized":"2023-11-15T02:56:51.735953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.749492+0000","last_change":"2023-11-15T02:56:51.744603+0000","last_active":"2023-11-15T02:56:51.749492+0000","last_peered":"2023-11-15T02:56:51.749492+0000","last_clean":"2023-11-15T02:56:51.749492+0000","last_became_active":"2023-11-15T02:56:51.744212+0000","last_became_peered":"2023-11-15T02:56:51.744212+0000","last_unstale":"2023-11-15T02:56:51.749492+0000","last_undegraded":"2023-11-15T02:56:51.749492+0000","last_fullsized":"2023-11-15T02:56:51.749492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713620+0000","last_change":"2023-11-15T02:56:51.279629+0000","last_active":"2023-11-15T02:56:51.713620+0000","last_peered":"2023-11-15T02:56:51.713620+0000","last_clean":"2023-11-15T02:56:51.713620+0000","last_became_active":"2023-11-15T02:56:51.279367+0000","last_became_peered":"2023-11-15T02:56:51.279367+0000","last_unstale":"2023-11-15T02:56:51.713620+0000","last_undegraded":"2023-11-15T02:56:51.713620+0000","last_fullsized":"2023-11-15T02:56:51.713620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745035+0000","last_change":"2023-11-15T02:56:51.741271+0000","last_active":"2023-11-15T02:56:51.745035+0000","last_peered":"2023-11-15T02:56:51.745035+0000","last_clean":"2023-11-15T02:56:51.745035+0000","last_became_active":"2023-11-15T02:56:51.740569+0000","last_became_peered":"2023-11-15T02:56:51.740569+0000","last_unstale":"2023-11-15T02:56:51.745035+0000","last_undegraded":"2023-11-15T02:56:51.745035+0000","last_fullsized":"2023-11-15T02:56:51.745035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.740741+0000","last_change":"2023-11-15T02:56:51.278710+0000","last_active":"2023-11-15T02:56:51.740741+0000","last_peered":"2023-11-15T02:56:51.740741+0000","last_clean":"2023-11-15T02:56:51.740741+0000","last_became_active":"2023-11-15T02:56:51.278446+0000","last_became_peered":"2023-11-15T02:56:51.278446+0000","last_unstale":"2023-11-15T02:56:51.740741+0000","last_undegraded":"2023-11-15T02:56:51.740741+0000","last_fullsized":"2023-11-15T02:56:51.740741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745553+0000","last_change":"2023-11-15T02:56:51.740004+0000","last_active":"2023-11-15T02:56:51.745553+0000","last_peered":"2023-11-15T02:56:51.745553+0000","last_clean":"2023-11-15T02:56:51.745553+0000","last_became_active":"2023-11-15T02:56:51.739663+0000","last_became_peered":"2023-11-15T02:56:51.739663+0000","last_unstale":"2023-11-15T02:56:51.745553+0000","last_undegraded":"2023-11-15T02:56:51.745553+0000","last_fullsized":"2023-11-15T02:56:51.745553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.714982+0000","last_change":"2023-11-15T02:56:51.278072+0000","last_active":"2023-11-15T02:56:51.714982+0000","last_peered":"2023-11-15T02:56:51.714982+0000","last_clean":"2023-11-15T02:56:51.714982+0000","last_became_active":"2023-11-15T02:56:51.277779+0000","last_became_peered":"2023-11-15T02:56:51.277779+0000","last_unstale":"2023-11-15T02:56:51.714982+0000","last_undegraded":"2023-11-15T02:56:51.714982+0000","last_fullsized":"2023-11-15T02:56:51.714982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746826+0000","last_change":"2023-11-15T02:56:51.744994+0000","last_active":"2023-11-15T02:56:51.746826+0000","last_peered":"2023-11-15T02:56:51.746826+0000","last_clean":"2023-11-15T02:56:51.746826+0000","last_became_active":"2023-11-15T02:56:51.743902+0000","last_became_peered":"2023-11-15T02:56:51.743902+0000","last_unstale":"2023-11-15T02:56:51.746826+0000","last_undegraded":"2023-11-15T02:56:51.746826+0000","last_fullsized":"2023-11-15T02:56:51.746826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730502+0000","last_change":"2023-11-15T02:56:51.289728+0000","last_active":"2023-11-15T02:56:51.730502+0000","last_peered":"2023-11-15T02:56:51.730502+0000","last_clean":"2023-11-15T02:56:51.730502+0000","last_became_active":"2023-11-15T02:56:51.288566+0000","last_became_peered":"2023-11-15T02:56:51.288566+0000","last_unstale":"2023-11-15T02:56:51.730502+0000","last_undegraded":"2023-11-15T02:56:51.730502+0000","last_fullsized":"2023-11-15T02:56:51.730502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747477+0000","last_change":"2023-11-15T02:56:51.735076+0000","last_active":"2023-11-15T02:56:51.747477+0000","last_peered":"2023-11-15T02:56:51.747477+0000","last_clean":"2023-11-15T02:56:51.747477+0000","last_became_active":"2023-11-15T02:56:51.734802+0000","last_became_peered":"2023-11-15T02:56:51.734802+0000","last_unstale":"2023-11-15T02:56:51.747477+0000","last_undegraded":"2023-11-15T02:56:51.747477+0000","last_fullsized":"2023-11-15T02:56:51.747477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744754+0000","last_change":"2023-11-15T02:56:51.286563+0000","last_active":"2023-11-15T02:56:51.744754+0000","last_peered":"2023-11-15T02:56:51.744754+0000","last_clean":"2023-11-15T02:56:51.744754+0000","last_became_active":"2023-11-15T02:56:51.286245+0000","last_became_peered":"2023-11-15T02:56:51.286245+0000","last_unstale":"2023-11-15T02:56:51.744754+0000","last_undegraded":"2023-11-15T02:56:51.744754+0000","last_fullsized":"2023-11-15T02:56:51.744754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746575+0000","last_change":"2023-11-15T02:56:51.742254+0000","last_active":"2023-11-15T02:56:51.746575+0000","last_peered":"2023-11-15T02:56:51.746575+0000","last_clean":"2023-11-15T02:56:51.746575+0000","last_became_active":"2023-11-15T02:56:51.741950+0000","last_became_peered":"2023-11-15T02:56:51.741950+0000","last_unstale":"2023-11-15T02:56:51.746575+0000","last_undegraded":"2023-11-15T02:56:51.746575+0000","last_fullsized":"2023-11-15T02:56:51.746575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744885+0000","last_change":"2023-11-15T02:56:51.286640+0000","last_active":"2023-11-15T02:56:51.744885+0000","last_peered":"2023-11-15T02:56:51.744885+0000","last_clean":"2023-11-15T02:56:51.744885+0000","last_became_active":"2023-11-15T02:56:51.286290+0000","last_became_peered":"2023-11-15T02:56:51.286290+0000","last_unstale":"2023-11-15T02:56:51.744885+0000","last_undegraded":"2023-11-15T02:56:51.744885+0000","last_fullsized":"2023-11-15T02:56:51.744885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.733131+0000","last_change":"2023-11-15T02:56:51.727797+0000","last_active":"2023-11-15T02:56:51.733131+0000","last_peered":"2023-11-15T02:56:51.733131+0000","last_clean":"2023-11-15T02:56:51.733131+0000","last_became_active":"2023-11-15T02:56:51.727520+0000","last_became_peered":"2023-11-15T02:56:51.727520+0000","last_unstale":"2023-11-15T02:56:51.733131+0000","last_undegraded":"2023-11-15T02:56:51.733131+0000","last_fullsized":"2023-11-15T02:56:51.733131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738551+0000","last_change":"2023-11-15T02:56:51.284828+0000","last_active":"2023-11-15T02:56:51.738551+0000","last_peered":"2023-11-15T02:56:51.738551+0000","last_clean":"2023-11-15T02:56:51.738551+0000","last_became_active":"2023-11-15T02:56:51.276571+0000","last_became_peered":"2023-11-15T02:56:51.276571+0000","last_unstale":"2023-11-15T02:56:51.738551+0000","last_undegraded":"2023-11-15T02:56:51.738551+0000","last_fullsized":"2023-11-15T02:56:51.738551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.739504+0000","last_change":"2023-11-15T02:56:51.732129+0000","last_active":"2023-11-15T02:56:51.739504+0000","last_peered":"2023-11-15T02:56:51.739504+0000","last_clean":"2023-11-15T02:56:51.739504+0000","last_became_active":"2023-11-15T02:56:51.729770+0000","last_became_peered":"2023-11-15T02:56:51.729770+0000","last_unstale":"2023-11-15T02:56:51.739504+0000","last_undegraded":"2023-11-15T02:56:51.739504+0000","last_fullsized":"2023-11-15T02:56:51.739504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.731327+0000","last_change":"2023-11-15T02:56:51.280630+0000","last_active":"2023-11-15T02:56:51.731327+0000","last_peered":"2023-11-15T02:56:51.731327+0000","last_clean":"2023-11-15T02:56:51.731327+0000","last_became_active":"2023-11-15T02:56:51.280353+0000","last_became_peered":"2023-11-15T02:56:51.280353+0000","last_unstale":"2023-11-15T02:56:51.731327+0000","last_undegraded":"2023-11-15T02:56:51.731327+0000","last_fullsized":"2023-11-15T02:56:51.731327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.741375+0000","last_change":"2023-11-15T02:56:51.737689+0000","last_active":"2023-11-15T02:56:51.741375+0000","last_peered":"2023-11-15T02:56:51.741375+0000","last_clean":"2023-11-15T02:56:51.741375+0000","last_became_active":"2023-11-15T02:56:51.737369+0000","last_became_peered":"2023-11-15T02:56:51.737369+0000","last_unstale":"2023-11-15T02:56:51.741375+0000","last_undegraded":"2023-11-15T02:56:51.741375+0000","last_fullsized":"2023-11-15T02:56:51.741375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.719432+0000","last_change":"2023-11-15T02:56:51.285172+0000","last_active":"2023-11-15T02:56:51.719432+0000","last_peered":"2023-11-15T02:56:51.719432+0000","last_clean":"2023-11-15T02:56:51.719432+0000","last_became_active":"2023-11-15T02:56:51.284936+0000","last_became_peered":"2023-11-15T02:56:51.284936+0000","last_unstale":"2023-11-15T02:56:51.719432+0000","last_undegraded":"2023-11-15T02:56:51.719432+0000","last_fullsized":"2023-11-15T02:56:51.719432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.732918+0000","last_change":"2023-11-15T02:56:51.728432+0000","last_active":"2023-11-15T02:56:51.732918+0000","last_peered":"2023-11-15T02:56:51.732918+0000","last_clean":"2023-11-15T02:56:51.732918+0000","last_became_active":"2023-11-15T02:56:51.727880+0000","last_became_peered":"2023-11-15T02:56:51.727880+0000","last_unstale":"2023-11-15T02:56:51.732918+0000","last_undegraded":"2023-11-15T02:56:51.732918+0000","last_fullsized":"2023-11-15T02:56:51.732918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744376+0000","last_change":"2023-11-15T02:56:51.288049+0000","last_active":"2023-11-15T02:56:51.744376+0000","last_peered":"2023-11-15T02:56:51.744376+0000","last_clean":"2023-11-15T02:56:51.744376+0000","last_became_active":"2023-11-15T02:56:51.287794+0000","last_became_peered":"2023-11-15T02:56:51.287794+0000","last_unstale":"2023-11-15T02:56:51.744376+0000","last_undegraded":"2023-11-15T02:56:51.744376+0000","last_fullsized":"2023-11-15T02:56:51.744376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.734683+0000","last_change":"2023-11-15T02:56:51.720823+0000","last_active":"2023-11-15T02:56:51.734683+0000","last_peered":"2023-11-15T02:56:51.734683+0000","last_clean":"2023-11-15T02:56:51.734683+0000","last_became_active":"2023-11-15T02:56:51.720479+0000","last_became_peered":"2023-11-15T02:56:51.720479+0000","last_unstale":"2023-11-15T02:56:51.734683+0000","last_undegraded":"2023-11-15T02:56:51.734683+0000","last_fullsized":"2023-11-15T02:56:51.734683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.718875+0000","last_change":"2023-11-15T02:56:51.278717+0000","last_active":"2023-11-15T02:56:51.718875+0000","last_peered":"2023-11-15T02:56:51.718875+0000","last_clean":"2023-11-15T02:56:51.718875+0000","last_became_active":"2023-11-15T02:56:51.278404+0000","last_became_peered":"2023-11-15T02:56:51.278404+0000","last_unstale":"2023-11-15T02:56:51.718875+0000","last_undegraded":"2023-11-15T02:56:51.718875+0000","last_fullsized":"2023-11-15T02:56:51.718875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745052+0000","last_change":"2023-11-15T02:56:51.741288+0000","last_active":"2023-11-15T02:56:51.745052+0000","last_peered":"2023-11-15T02:56:51.745052+0000","last_clean":"2023-11-15T02:56:51.745052+0000","last_became_active":"2023-11-15T02:56:51.740865+0000","last_became_peered":"2023-11-15T02:56:51.740865+0000","last_unstale":"2023-11-15T02:56:51.745052+0000","last_undegraded":"2023-11-15T02:56:51.745052+0000","last_fullsized":"2023-11-15T02:56:51.745052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746626+0000","last_change":"2023-11-15T02:56:51.274179+0000","last_active":"2023-11-15T02:56:51.746626+0000","last_peered":"2023-11-15T02:56:51.746626+0000","last_clean":"2023-11-15T02:56:51.746626+0000","last_became_active":"2023-11-15T02:56:51.272313+0000","last_became_peered":"2023-11-15T02:56:51.272313+0000","last_unstale":"2023-11-15T02:56:51.746626+0000","last_undegraded":"2023-11-15T02:56:51.746626+0000","last_fullsized":"2023-11-15T02:56:51.746626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.737568+0000","last_change":"2023-11-15T02:56:51.736317+0000","last_active":"2023-11-15T02:56:51.737568+0000","last_peered":"2023-11-15T02:56:51.737568+0000","last_clean":"2023-11-15T02:56:51.737568+0000","last_became_active":"2023-11-15T02:56:51.736060+0000","last_became_peered":"2023-11-15T02:56:51.736060+0000","last_unstale":"2023-11-15T02:56:51.737568+0000","last_undegraded":"2023-11-15T02:56:51.737568+0000","last_fullsized":"2023-11-15T02:56:51.737568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.754612+0000","last_change":"2023-11-15T02:56:51.279248+0000","last_active":"2023-11-15T02:56:51.754612+0000","last_peered":"2023-11-15T02:56:51.754612+0000","last_clean":"2023-11-15T02:56:51.754612+0000","last_became_active":"2023-11-15T02:56:51.278771+0000","last_became_peered":"2023-11-15T02:56:51.278771+0000","last_unstale":"2023-11-15T02:56:51.754612+0000","last_undegraded":"2023-11-15T02:56:51.754612+0000","last_fullsized":"2023-11-15T02:56:51.754612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.727542+0000","last_change":"2023-11-15T02:56:51.716546+0000","last_active":"2023-11-15T02:56:51.727542+0000","last_peered":"2023-11-15T02:56:51.727542+0000","last_clean":"2023-11-15T02:56:51.727542+0000","last_became_active":"2023-11-15T02:56:51.716254+0000","last_became_peered":"2023-11-15T02:56:51.716254+0000","last_unstale":"2023-11-15T02:56:51.727542+0000","last_undegraded":"2023-11-15T02:56:51.727542+0000","last_fullsized":"2023-11-15T02:56:51.727542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712732+0000","last_change":"2023-11-15T02:56:51.286642+0000","last_active":"2023-11-15T02:56:51.712732+0000","last_peered":"2023-11-15T02:56:51.712732+0000","last_clean":"2023-11-15T02:56:51.712732+0000","last_became_active":"2023-11-15T02:56:51.286240+0000","last_became_peered":"2023-11-15T02:56:51.286240+0000","last_unstale":"2023-11-15T02:56:51.712732+0000","last_undegraded":"2023-11-15T02:56:51.712732+0000","last_fullsized":"2023-11-15T02:56:51.712732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.725581+0000","last_change":"2023-11-15T02:56:51.722322+0000","last_active":"2023-11-15T02:56:51.725581+0000","last_peered":"2023-11-15T02:56:51.725581+0000","last_clean":"2023-11-15T02:56:51.725581+0000","last_became_active":"2023-11-15T02:56:51.721931+0000","last_became_peered":"2023-11-15T02:56:51.721931+0000","last_unstale":"2023-11-15T02:56:51.725581+0000","last_undegraded":"2023-11-15T02:56:51.725581+0000","last_fullsized":"2023-11-15T02:56:51.725581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713667+0000","last_change":"2023-11-15T02:56:51.284656+0000","last_active":"2023-11-15T02:56:51.713667+0000","last_peered":"2023-11-15T02:56:51.713667+0000","last_clean":"2023-11-15T02:56:51.713667+0000","last_became_active":"2023-11-15T02:56:51.284378+0000","last_became_peered":"2023-11-15T02:56:51.284378+0000","last_unstale":"2023-11-15T02:56:51.713667+0000","last_undegraded":"2023-11-15T02:56:51.713667+0000","last_fullsized":"2023-11-15T02:56:51.713667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747336+0000","last_change":"2023-11-15T02:56:51.743546+0000","last_active":"2023-11-15T02:56:51.747336+0000","last_peered":"2023-11-15T02:56:51.747336+0000","last_clean":"2023-11-15T02:56:51.747336+0000","last_became_active":"2023-11-15T02:56:51.743276+0000","last_became_peered":"2023-11-15T02:56:51.743276+0000","last_unstale":"2023-11-15T02:56:51.747336+0000","last_undegraded":"2023-11-15T02:56:51.747336+0000","last_fullsized":"2023-11-15T02:56:51.747336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.729927+0000","last_change":"2023-11-15T02:56:51.285769+0000","last_active":"2023-11-15T02:56:51.729927+0000","last_peered":"2023-11-15T02:56:51.729927+0000","last_clean":"2023-11-15T02:56:51.729927+0000","last_became_active":"2023-11-15T02:56:51.285448+0000","last_became_peered":"2023-11-15T02:56:51.285448+0000","last_unstale":"2023-11-15T02:56:51.729927+0000","last_undegraded":"2023-11-15T02:56:51.729927+0000","last_fullsized":"2023-11-15T02:56:51.729927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.742411+0000","last_change":"2023-11-15T02:56:51.736745+0000","last_active":"2023-11-15T02:56:51.742411+0000","last_peered":"2023-11-15T02:56:51.742411+0000","last_clean":"2023-11-15T02:56:51.742411+0000","last_became_active":"2023-11-15T02:56:51.736555+0000","last_became_peered":"2023-11-15T02:56:51.736555+0000","last_unstale":"2023-11-15T02:56:51.742411+0000","last_undegraded":"2023-11-15T02:56:51.742411+0000","last_fullsized":"2023-11-15T02:56:51.742411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730754+0000","last_change":"2023-11-15T02:56:51.722343+0000","last_active":"2023-11-15T02:56:51.730754+0000","last_peered":"2023-11-15T02:56:51.730754+0000","last_clean":"2023-11-15T02:56:51.730754+0000","last_became_active":"2023-11-15T02:56:51.722090+0000","last_became_peered":"2023-11-15T02:56:51.722090+0000","last_unstale":"2023-11-15T02:56:51.730754+0000","last_undegraded":"2023-11-15T02:56:51.730754+0000","last_fullsized":"2023-11-15T02:56:51.730754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.710708+0000","last_change":"2023-11-15T02:56:51.281122+0000","last_active":"2023-11-15T02:56:51.710708+0000","last_peered":"2023-11-15T02:56:51.710708+0000","last_clean":"2023-11-15T02:56:51.710708+0000","last_became_active":"2023-11-15T02:56:51.280798+0000","last_became_peered":"2023-11-15T02:56:51.280798+0000","last_unstale":"2023-11-15T02:56:51.710708+0000","last_undegraded":"2023-11-15T02:56:51.710708+0000","last_fullsized":"2023-11-15T02:56:51.710708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.729313+0000","last_change":"2023-11-15T02:56:51.726756+0000","last_active":"2023-11-15T02:56:51.729313+0000","last_peered":"2023-11-15T02:56:51.729313+0000","last_clean":"2023-11-15T02:56:51.729313+0000","last_became_active":"2023-11-15T02:56:51.726446+0000","last_became_peered":"2023-11-15T02:56:51.726446+0000","last_unstale":"2023-11-15T02:56:51.729313+0000","last_undegraded":"2023-11-15T02:56:51.729313+0000","last_fullsized":"2023-11-15T02:56:51.729313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730560+0000","last_change":"2023-11-15T02:56:51.288389+0000","last_active":"2023-11-15T02:56:51.730560+0000","last_peered":"2023-11-15T02:56:51.730560+0000","last_clean":"2023-11-15T02:56:51.730560+0000","last_became_active":"2023-11-15T02:56:51.287933+0000","last_became_peered":"2023-11-15T02:56:51.287933+0000","last_unstale":"2023-11-15T02:56:51.730560+0000","last_undegraded":"2023-11-15T02:56:51.730560+0000","last_fullsized":"2023-11-15T02:56:51.730560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746552+0000","last_change":"2023-11-15T02:56:51.742907+0000","last_active":"2023-11-15T02:56:51.746552+0000","last_peered":"2023-11-15T02:56:51.746552+0000","last_clean":"2023-11-15T02:56:51.746552+0000","last_became_active":"2023-11-15T02:56:51.742633+0000","last_became_peered":"2023-11-15T02:56:51.742633+0000","last_unstale":"2023-11-15T02:56:51.746552+0000","last_undegraded":"2023-11-15T02:56:51.746552+0000","last_fullsized":"2023-11-15T02:56:51.746552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.736908+0000","last_change":"2023-11-15T02:56:51.278389+0000","last_active":"2023-11-15T02:56:51.736908+0000","last_peered":"2023-11-15T02:56:51.736908+0000","last_clean":"2023-11-15T02:56:51.736908+0000","last_became_active":"2023-11-15T02:56:51.277985+0000","last_became_peered":"2023-11-15T02:56:51.277985+0000","last_unstale":"2023-11-15T02:56:51.736908+0000","last_undegraded":"2023-11-15T02:56:51.736908+0000","last_fullsized":"2023-11-15T02:56:51.736908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.743771+0000","last_change":"2023-11-15T02:56:51.725491+0000","last_active":"2023-11-15T02:56:51.743771+0000","last_peered":"2023-11-15T02:56:51.743771+0000","last_clean":"2023-11-15T02:56:51.743771+0000","last_became_active":"2023-11-15T02:56:51.725072+0000","last_became_peered":"2023-11-15T02:56:51.725072+0000","last_unstale":"2023-11-15T02:56:51.743771+0000","last_undegraded":"2023-11-15T02:56:51.743771+0000","last_fullsized":"2023-11-15T02:56:51.743771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738751+0000","last_change":"2023-11-15T02:56:51.291272+0000","last_active":"2023-11-15T02:56:51.738751+0000","last_peered":"2023-11-15T02:56:51.738751+0000","last_clean":"2023-11-15T02:56:51.738751+0000","last_became_active":"2023-11-15T02:56:51.287508+0000","last_became_peered":"2023-11-15T02:56:51.287508+0000","last_unstale":"2023-11-15T02:56:51.738751+0000","last_undegraded":"2023-11-15T02:56:51.738751+0000","last_fullsized":"2023-11-15T02:56:51.738751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745082+0000","last_change":"2023-11-15T02:56:51.741837+0000","last_active":"2023-11-15T02:56:51.745082+0000","last_peered":"2023-11-15T02:56:51.745082+0000","last_clean":"2023-11-15T02:56:51.745082+0000","last_became_active":"2023-11-15T02:56:51.741487+0000","last_became_peered":"2023-11-15T02:56:51.741487+0000","last_unstale":"2023-11-15T02:56:51.745082+0000","last_undegraded":"2023-11-15T02:56:51.745082+0000","last_fullsized":"2023-11-15T02:56:51.745082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.741740+0000","last_change":"2023-11-15T02:56:51.283591+0000","last_active":"2023-11-15T02:56:51.741740+0000","last_peered":"2023-11-15T02:56:51.741740+0000","last_clean":"2023-11-15T02:56:51.741740+0000","last_became_active":"2023-11-15T02:56:51.283198+0000","last_became_peered":"2023-11-15T02:56:51.283198+0000","last_unstale":"2023-11-15T02:56:51.741740+0000","last_undegraded":"2023-11-15T02:56:51.741740+0000","last_fullsized":"2023-11-15T02:56:51.741740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744198+0000","last_change":"2023-11-15T02:56:51.739304+0000","last_active":"2023-11-15T02:56:51.744198+0000","last_peered":"2023-11-15T02:56:51.744198+0000","last_clean":"2023-11-15T02:56:51.744198+0000","last_became_active":"2023-11-15T02:56:51.738755+0000","last_became_peered":"2023-11-15T02:56:51.738755+0000","last_unstale":"2023-11-15T02:56:51.744198+0000","last_undegraded":"2023-11-15T02:56:51.744198+0000","last_fullsized":"2023-11-15T02:56:51.744198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.722852+0000","last_change":"2023-11-15T02:56:51.269310+0000","last_active":"2023-11-15T02:56:51.722852+0000","last_peered":"2023-11-15T02:56:51.722852+0000","last_clean":"2023-11-15T02:56:51.722852+0000","last_became_active":"2023-11-15T02:56:51.268977+0000","last_became_peered":"2023-11-15T02:56:51.268977+0000","last_unstale":"2023-11-15T02:56:51.722852+0000","last_undegraded":"2023-11-15T02:56:51.722852+0000","last_fullsized":"2023-11-15T02:56:51.722852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747235+0000","last_change":"2023-11-15T02:56:51.742002+0000","last_active":"2023-11-15T02:56:51.747235+0000","last_peered":"2023-11-15T02:56:51.747235+0000","last_clean":"2023-11-15T02:56:51.747235+0000","last_became_active":"2023-11-15T02:56:51.741691+0000","last_became_peered":"2023-11-15T02:56:51.741691+0000","last_unstale":"2023-11-15T02:56:51.747235+0000","last_undegraded":"2023-11-15T02:56:51.747235+0000","last_fullsized":"2023-11-15T02:56:51.747235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.706447+0000","last_change":"2023-11-15T02:56:51.280160+0000","last_active":"2023-11-15T02:56:51.706447+0000","last_peered":"2023-11-15T02:56:51.706447+0000","last_clean":"2023-11-15T02:56:51.706447+0000","last_became_active":"2023-11-15T02:56:51.278364+0000","last_became_peered":"2023-11-15T02:56:51.278364+0000","last_unstale":"2023-11-15T02:56:51.706447+0000","last_undegraded":"2023-11-15T02:56:51.706447+0000","last_fullsized":"2023-11-15T02:56:51.706447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747191+0000","last_change":"2023-11-15T02:56:51.742487+0000","last_active":"2023-11-15T02:56:51.747191+0000","last_peered":"2023-11-15T02:56:51.747191+0000","last_clean":"2023-11-15T02:56:51.747191+0000","last_became_active":"2023-11-15T02:56:51.742056+0000","last_became_peered":"2023-11-15T02:56:51.742056+0000","last_unstale":"2023-11-15T02:56:51.747191+0000","last_undegraded":"2023-11-15T02:56:51.747191+0000","last_fullsized":"2023-11-15T02:56:51.747191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.748052+0000","last_change":"2023-11-15T02:56:51.290964+0000","last_active":"2023-11-15T02:56:51.748052+0000","last_peered":"2023-11-15T02:56:51.748052+0000","last_clean":"2023-11-15T02:56:51.748052+0000","last_became_active":"2023-11-15T02:56:51.290728+0000","last_became_peered":"2023-11-15T02:56:51.290728+0000","last_unstale":"2023-11-15T02:56:51.748052+0000","last_undegraded":"2023-11-15T02:56:51.748052+0000","last_fullsized":"2023-11-15T02:56:51.748052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297332,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074508,"statfs":{"total":96636764160,"available":96332296192,"internally_reserved":0,"allocated":241664,"data_stored":80461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3380000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.375}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5309999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297312,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074528,"statfs":{"total":96636764160,"available":96332316672,"internally_reserved":0,"allocated":221184,"data_stored":80493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.512}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3440000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.423}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.458}]}]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297324,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074516,"statfs":{"total":96636764160,"available":96332304384,"internally_reserved":0,"allocated":233472,"data_stored":80639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.296}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5049999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297308,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074532,"statfs":{"total":96636764160,"available":96332320768,"internally_reserved":0,"allocated":217088,"data_stored":79137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.837}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7270000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.883}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":77848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":79047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":79047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5740000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.599}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.633}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.74}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":79047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3440000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2789999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T02:56:56.496 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-15T02:56:56.496 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-15T02:56:56.496 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-15T02:56:56.497 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-15T02:56:56.498 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-15T02:56:56.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.695+0000 7f4949569700 1 Processor -- start 2023-11-15T02:56:56.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f4949569700 1 -- start start 2023-11-15T02:56:56.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f4949569700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49440a5ed0 0x7f4944155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:56.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f4949569700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49440c4c30 con 0x7f4944154ca0 2023-11-15T02:56:56.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f4949569700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49440a6f50 con 0x7f49441562f0 2023-11-15T02:56:56.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f4949569700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49440a8280 con 0x7f49440a5ed0 2023-11-15T02:56:56.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f4942ffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4944154ca0 0x7f49440a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:57700/0 (socket says 172.21.15.121:57700) 2023-11-15T02:56:56.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f4942ffd700 1 -- 172.21.15.121:0/3328750617 learned_addr learned my addr 172.21.15.121:0/3328750617 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:56.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f49437fe700 1 --2- 172.21.15.121:0/3328750617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49440a5ed0 0x7f4944155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:56.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f49427fc700 1 -- 172.21.15.121:0/3328750617 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1864544947 0 0) 0x7f49440c4c30 con 0x7f4944154ca0 2023-11-15T02:56:56.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f49427fc700 1 -- 172.21.15.121:0/3328750617 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49300033f0 con 0x7f4944154ca0 2023-11-15T02:56:56.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f49427fc700 1 -- 172.21.15.121:0/3328750617 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2706436321 0 0) 0x7f49300033f0 con 0x7f4944154ca0 2023-11-15T02:56:56.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f49427fc700 1 -- 172.21.15.121:0/3328750617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49440a5ed0 msgr2=0x7f4944155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:56.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f49427fc700 1 --2- 172.21.15.121:0/3328750617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49440a5ed0 0x7f4944155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f49427fc700 1 -- 172.21.15.121:0/3328750617 >> v1:172.21.15.190:6789/0 conn(0x7f49441562f0 legacy=0x7f49440a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:56.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f49427fc700 1 -- 172.21.15.121:0/3328750617 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440a7230 con 0x7f4944154ca0 2023-11-15T02:56:56.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f49427fc700 1 -- 172.21.15.121:0/3328750617 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f4934001470 con 0x7f4944154ca0 2023-11-15T02:56:56.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f49427fc700 1 -- 172.21.15.121:0/3328750617 >> v1:172.21.15.121:6789/0 conn(0x7f4944154ca0 legacy=0x7f49440a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:56.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f49427fc700 1 --2- 172.21.15.121:0/3328750617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4930003910 0x7f4930003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:56.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f49427fc700 1 --2- 172.21.15.121:0/3328750617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4930004690 0x7f4930004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:56.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f49427fc700 1 --2- 172.21.15.121:0/3328750617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49440a5ed0 0x7f4930005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:56.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f49427fc700 1 -- 172.21.15.121:0/3328750617 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49440a8280 con 0x7f4930003910 2023-11-15T02:56:56.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f49427fc700 1 -- 172.21.15.121:0/3328750617 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493000b9c0 con 0x7f4930004690 2023-11-15T02:56:56.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.699+0000 7f49427fc700 1 -- 172.21.15.121:0/3328750617 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493000bb80 con 0x7f49440a5ed0 2023-11-15T02:56:56.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f4942ffd700 1 --2- 172.21.15.121:0/3328750617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4930003910 0x7f4930003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:56.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f49437fe700 1 --2- 172.21.15.121:0/3328750617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49440a5ed0 0x7f4930005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:56.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f493a7fc700 1 --2- 172.21.15.121:0/3328750617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4930004690 0x7f4930004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:56.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f4942ffd700 1 -- 172.21.15.121:0/3328750617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49440a5ed0 msgr2=0x7f4930005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:56.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f4942ffd700 1 --2- 172.21.15.121:0/3328750617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49440a5ed0 0x7f4930005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f4942ffd700 1 -- 172.21.15.121:0/3328750617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4930004690 msgr2=0x7f4930004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:56.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f4942ffd700 1 --2- 172.21.15.121:0/3328750617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4930004690 0x7f4930004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f4942ffd700 1 -- 172.21.15.121:0/3328750617 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49300033f0 con 0x7f4930003910 2023-11-15T02:56:56.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f493a7fc700 1 --2- 172.21.15.121:0/3328750617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4930004690 0x7f4930004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:56:56.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f4942ffd700 1 --2- 172.21.15.121:0/3328750617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4930003910 0x7f4930003d30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f493400c160 tx=0x7f49440c4ba0).ready entity=mon.0 client_cookie=ca605683cc21ac96 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:56.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f49427fc700 1 -- 172.21.15.121:0/3328750617 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493400ca90 con 0x7f4930003910 2023-11-15T02:56:56.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f49427fc700 1 -- 172.21.15.121:0/3328750617 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493400cbf0 con 0x7f4930003910 2023-11-15T02:56:56.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f49427fc700 1 -- 172.21.15.121:0/3328750617 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49340070d0 con 0x7f4930003910 2023-11-15T02:56:56.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f4949569700 1 -- 172.21.15.121:0/3328750617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4930003910 msgr2=0x7f4930003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:56.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f4949569700 1 --2- 172.21.15.121:0/3328750617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4930003910 0x7f4930003d30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f493400c160 tx=0x7f49440c4ba0).stop 2023-11-15T02:56:56.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f49437fe700 1 -- 172.21.15.121:0/3328750617 reap_dead start 2023-11-15T02:56:56.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f4949569700 1 -- 172.21.15.121:0/3328750617 shutdown_connections 2023-11-15T02:56:56.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f4949569700 1 -- 172.21.15.121:0/3328750617 >> 172.21.15.121:0/3328750617 conn(0x7f4944062190 msgr2=0x7f49440625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:56.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f4949569700 1 -- 172.21.15.121:0/3328750617 shutdown_connections 2023-11-15T02:56:56.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.703+0000 7f4949569700 1 -- 172.21.15.121:0/3328750617 wait complete. 2023-11-15T02:56:56.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.707+0000 7f4949569700 1 Processor -- start 2023-11-15T02:56:56.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.707+0000 7f4949569700 1 -- start start 2023-11-15T02:56:56.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.707+0000 7f4949569700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4930003910 0x7f49441482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:56.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.707+0000 7f4949569700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4944148b00 0x7f49440bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:56.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.707+0000 7f4949569700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4944148f80 0x7f49440c0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:56.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.707+0000 7f4949569700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49440c0570 con 0x7f4944148b00 2023-11-15T02:56:56.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.707+0000 7f4949569700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49440ac760 con 0x7f4944148f80 2023-11-15T02:56:56.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.707+0000 7f4949569700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494414a5f0 con 0x7f4930003910 2023-11-15T02:56:56.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.707+0000 7f493a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4944148b00 0x7f49440bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:56.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.707+0000 7f493a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4944148b00 0x7f49440bfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32982/0 (socket says 172.21.15.121:32982) 2023-11-15T02:56:56.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.707+0000 7f493a7fc700 1 -- 172.21.15.121:0/377878513 learned_addr learned my addr 172.21.15.121:0/377878513 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:56.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.707+0000 7f4942ffd700 1 --2- 172.21.15.121:0/377878513 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4930003910 0x7f49441482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:56.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.707+0000 7f49437fe700 1 --2- 172.21.15.121:0/377878513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4944148f80 0x7f49440c0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:56.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.707+0000 7f493a7fc700 1 -- 172.21.15.121:0/377878513 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4930003910 msgr2=0x7f49441482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:56.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.707+0000 7f493a7fc700 1 --2- 172.21.15.121:0/377878513 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4930003910 0x7f49441482c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.707+0000 7f493a7fc700 1 -- 172.21.15.121:0/377878513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4944148f80 msgr2=0x7f49440c0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:56.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.707+0000 7f493a7fc700 1 --2- 172.21.15.121:0/377878513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4944148f80 0x7f49440c0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.707+0000 7f493a7fc700 1 -- 172.21.15.121:0/377878513 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440aaba0 con 0x7f4944148b00 2023-11-15T02:56:56.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.707+0000 7f493a7fc700 1 --2- 172.21.15.121:0/377878513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4944148b00 0x7f49440bfb90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f492c00d600 tx=0x7f492c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:56.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.711+0000 7f493b7fe700 1 -- 172.21.15.121:0/377878513 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c00f510 con 0x7f4944148b00 2023-11-15T02:56:56.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.711+0000 7f493b7fe700 1 -- 172.21.15.121:0/377878513 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c00f670 con 0x7f4944148b00 2023-11-15T02:56:56.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.711+0000 7f4949569700 1 -- 172.21.15.121:0/377878513 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49440aae80 con 0x7f4944148b00 2023-11-15T02:56:56.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.711+0000 7f493b7fe700 1 -- 172.21.15.121:0/377878513 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c00f9f0 con 0x7f4944148b00 2023-11-15T02:56:56.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.711+0000 7f4949569700 1 -- 172.21.15.121:0/377878513 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49440ab440 con 0x7f4944148b00 2023-11-15T02:56:56.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.711+0000 7f4949569700 1 -- 172.21.15.121:0/377878513 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494406c150 con 0x7f4944148b00 2023-11-15T02:56:56.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.711+0000 7f493b7fe700 1 -- 172.21.15.121:0/377878513 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f492c0123d0 con 0x7f4944148b00 2023-11-15T02:56:56.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.715+0000 7f493b7fe700 1 --2- 172.21.15.121:0/377878513 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f492405a510 0x7f492405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:56.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.715+0000 7f493b7fe700 1 -- 172.21.15.121:0/377878513 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f492c074380 con 0x7f4944148b00 2023-11-15T02:56:56.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.715+0000 7f4942ffd700 1 --2- 172.21.15.121:0/377878513 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f492405a510 0x7f492405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:56.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.715+0000 7f4942ffd700 1 --2- 172.21.15.121:0/377878513 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f492405a510 0x7f492405c9d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f4934007e70 tx=0x7f4934015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:56.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.719+0000 7f493b7fe700 1 -- 172.21.15.121:0/377878513 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f492c047b00 con 0x7f4944148b00 2023-11-15T02:56:56.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.919+0000 7f4949569700 1 -- 172.21.15.121:0/377878513 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f49440c4500 con 0x7f4944148b00 2023-11-15T02:56:56.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.927+0000 7f493b7fe700 1 -- 172.21.15.121:0/377878513 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+142150 (secure 0 0 0) 0x7f492c04b120 con 0x7f4944148b00 2023-11-15T02:56:56.933 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:56:56.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.931+0000 7f4949569700 1 -- 172.21.15.121:0/377878513 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f492405a510 msgr2=0x7f492405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:56.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.931+0000 7f4949569700 1 --2- 172.21.15.121:0/377878513 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f492405a510 0x7f492405c9d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f4934007e70 tx=0x7f4934015000).stop 2023-11-15T02:56:56.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.931+0000 7f4949569700 1 -- 172.21.15.121:0/377878513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4944148b00 msgr2=0x7f49440bfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:56.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.931+0000 7f4949569700 1 --2- 172.21.15.121:0/377878513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4944148b00 0x7f49440bfb90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f492c00d600 tx=0x7f492c009040).stop 2023-11-15T02:56:56.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.931+0000 7f4949569700 1 -- 172.21.15.121:0/377878513 shutdown_connections 2023-11-15T02:56:56.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.931+0000 7f4949569700 1 --2- 172.21.15.121:0/377878513 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f492405a510 0x7f492405c9d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.931+0000 7f4949569700 1 --2- 172.21.15.121:0/377878513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4944148f80 0x7f49440c0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.931+0000 7f4949569700 1 --2- 172.21.15.121:0/377878513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4944148b00 0x7f49440bfb90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.931+0000 7f4949569700 1 --2- 172.21.15.121:0/377878513 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4930003910 0x7f49441482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:56.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.931+0000 7f4949569700 1 -- 172.21.15.121:0/377878513 >> 172.21.15.121:0/377878513 conn(0x7f4944062190 msgr2=0x7f49440c7190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:56.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.931+0000 7f4949569700 1 -- 172.21.15.121:0/377878513 shutdown_connections 2023-11-15T02:56:56.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:56.931+0000 7f4949569700 1 -- 172.21.15.121:0/377878513 wait complete. 2023-11-15T02:56:56.956 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":4,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":33042},{"type":"v1","addr":"172.21.15.121:6833","nonce":33042}]},"active_addr":"172.21.15.121:6833/33042","active_change":"2023-11-15T02:56:37.890389+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540138314316775423,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":2122681064}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":160569010}]}]} 2023-11-15T02:56:56.960 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-15T02:56:56.960 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-15T02:56:56.960 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:56:57.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.119+0000 7ff31085b700 1 Processor -- start 2023-11-15T02:56:57.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.119+0000 7ff31085b700 1 -- start start 2023-11-15T02:56:57.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.123+0000 7ff31085b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff30c154ca0 0x7ff30c155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:57.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.123+0000 7ff31085b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff30c0a57c0 0x7ff30c0a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:57.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.123+0000 7ff31085b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff30c0c4c30 con 0x7ff30c1562f0 2023-11-15T02:56:57.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.123+0000 7ff31085b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff30c0a8280 con 0x7ff30c154ca0 2023-11-15T02:56:57.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.123+0000 7ff31085b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff30c0a6d40 con 0x7ff30c0a57c0 2023-11-15T02:56:57.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.123+0000 7ff30a59c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff30c154ca0 0x7ff30c155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:57.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.123+0000 7ff30a59c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff30c154ca0 0x7ff30c155080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32984/0 (socket says 172.21.15.121:32984) 2023-11-15T02:56:57.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.123+0000 7ff30a59c700 1 -- 172.21.15.121:0/2435769615 learned_addr learned my addr 172.21.15.121:0/2435769615 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:57.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.123+0000 7ff309d9b700 1 --1- 172.21.15.121:0/2435769615 >> v1:172.21.15.121:6789/0 conn(0x7ff30c1562f0 0x7ff30c0a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:57704/0 (socket says 172.21.15.121:57704) 2023-11-15T02:56:57.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.123+0000 7ff30ad9d700 1 --2- 172.21.15.121:0/2435769615 >> v2:172.21.15.190:3300/0 conn(0x7ff30c0a57c0 0x7ff30c0a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:57.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.123+0000 7ff30a59c700 1 -- 172.21.15.121:0/2435769615 >> v2:172.21.15.190:3300/0 conn(0x7ff30c0a57c0 msgr2=0x7ff30c0a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:57.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.123+0000 7ff30a59c700 1 --2- 172.21.15.121:0/2435769615 >> v2:172.21.15.190:3300/0 conn(0x7ff30c0a57c0 0x7ff30c0a5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.123+0000 7ff30a59c700 1 -- 172.21.15.121:0/2435769615 >> v1:172.21.15.121:6789/0 conn(0x7ff30c1562f0 legacy=0x7ff30c0a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:57.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.123+0000 7ff30a59c700 1 -- 172.21.15.121:0/2435769615 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c0a6100 con 0x7ff30c154ca0 2023-11-15T02:56:57.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.123+0000 7ff30ad9d700 1 --2- 172.21.15.121:0/2435769615 >> v2:172.21.15.190:3300/0 conn(0x7ff30c0a57c0 0x7ff30c0a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:56:57.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.123+0000 7ff30a59c700 1 --2- 172.21.15.121:0/2435769615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff30c154ca0 0x7ff30c155080 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7ff30c0a8820 tx=0x7ff2fc0090f0).ready entity=mon.0 client_cookie=64f2a077d64138d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:57.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.123+0000 7ff30959a700 1 -- 172.21.15.121:0/2435769615 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc00b600 con 0x7ff30c154ca0 2023-11-15T02:56:57.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.127+0000 7ff30959a700 1 -- 172.21.15.121:0/2435769615 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2fc00b760 con 0x7ff30c154ca0 2023-11-15T02:56:57.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.127+0000 7ff30959a700 1 -- 172.21.15.121:0/2435769615 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc00bae0 con 0x7ff30c154ca0 2023-11-15T02:56:57.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.127+0000 7ff31085b700 1 -- 172.21.15.121:0/2435769615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff30c154ca0 msgr2=0x7ff30c155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:57.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.127+0000 7ff31085b700 1 --2- 172.21.15.121:0/2435769615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff30c154ca0 0x7ff30c155080 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7ff30c0a8820 tx=0x7ff2fc0090f0).stop 2023-11-15T02:56:57.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.127+0000 7ff31085b700 1 -- 172.21.15.121:0/2435769615 shutdown_connections 2023-11-15T02:56:57.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.127+0000 7ff31085b700 1 --2- 172.21.15.121:0/2435769615 >> v2:172.21.15.190:3300/0 conn(0x7ff30c0a57c0 0x7ff30c0a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.127+0000 7ff31085b700 1 --2- 172.21.15.121:0/2435769615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff30c154ca0 0x7ff30c155080 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.127+0000 7ff31085b700 1 -- 172.21.15.121:0/2435769615 >> 172.21.15.121:0/2435769615 conn(0x7ff30c062190 msgr2=0x7ff30c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:57.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.127+0000 7ff31085b700 1 -- 172.21.15.121:0/2435769615 shutdown_connections 2023-11-15T02:56:57.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.127+0000 7ff31085b700 1 -- 172.21.15.121:0/2435769615 wait complete. 2023-11-15T02:56:57.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff31085b700 1 Processor -- start 2023-11-15T02:56:57.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff31085b700 1 -- start start 2023-11-15T02:56:57.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff31085b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff30c0a57c0 0x7ff30c0bf630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:57.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff31085b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff30c154ca0 0x7ff30c143110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:57.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff31085b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff30c1562f0 0x7ff30c0c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:57.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff31085b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c0bb780 con 0x7ff30c0a57c0 2023-11-15T02:56:57.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff31085b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c0bbd40 con 0x7ff30c1562f0 2023-11-15T02:56:57.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff31085b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c0bfed0 con 0x7ff30c154ca0 2023-11-15T02:56:57.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff30a59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff30c0a57c0 0x7ff30c0bf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:57.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff30a59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff30c0a57c0 0x7ff30c0bf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33000/0 (socket says 172.21.15.121:33000) 2023-11-15T02:56:57.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff30a59c700 1 -- 172.21.15.121:0/3104638129 learned_addr learned my addr 172.21.15.121:0/3104638129 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:57.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff30ad9d700 1 --2- 172.21.15.121:0/3104638129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff30c1562f0 0x7ff30c0c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:57.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff309d9b700 1 --2- 172.21.15.121:0/3104638129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff30c154ca0 0x7ff30c143110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:57.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff30a59c700 1 -- 172.21.15.121:0/3104638129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff30c154ca0 msgr2=0x7ff30c143110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:57.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff30a59c700 1 --2- 172.21.15.121:0/3104638129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff30c154ca0 0x7ff30c143110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff30a59c700 1 -- 172.21.15.121:0/3104638129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff30c1562f0 msgr2=0x7ff30c0c69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:57.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff30a59c700 1 --2- 172.21.15.121:0/3104638129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff30c1562f0 0x7ff30c0c69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff30a59c700 1 -- 172.21.15.121:0/3104638129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2fc009370 con 0x7ff30c0a57c0 2023-11-15T02:56:57.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff309d9b700 1 --2- 172.21.15.121:0/3104638129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff30c154ca0 0x7ff30c143110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:56:57.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff30ad9d700 1 --2- 172.21.15.121:0/3104638129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff30c1562f0 0x7ff30c0c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:56:57.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.131+0000 7ff30a59c700 1 --2- 172.21.15.121:0/3104638129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff30c0a57c0 0x7ff30c0bf630 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7ff2fc006fd0 tx=0x7ff2fc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:57.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.135+0000 7ff2fa7fc700 1 -- 172.21.15.121:0/3104638129 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc00bae0 con 0x7ff30c0a57c0 2023-11-15T02:56:57.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.135+0000 7ff2fa7fc700 1 -- 172.21.15.121:0/3104638129 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2fc000c90 con 0x7ff30c0a57c0 2023-11-15T02:56:57.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.135+0000 7ff2fa7fc700 1 -- 172.21.15.121:0/3104638129 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc01c650 con 0x7ff30c0a57c0 2023-11-15T02:56:57.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.135+0000 7ff31085b700 1 -- 172.21.15.121:0/3104638129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30c0c01b0 con 0x7ff30c0a57c0 2023-11-15T02:56:57.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.135+0000 7ff31085b700 1 -- 172.21.15.121:0/3104638129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30c0bfb20 con 0x7ff30c0a57c0 2023-11-15T02:56:57.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.135+0000 7ff31085b700 1 -- 172.21.15.121:0/3104638129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30c06c150 con 0x7ff30c0a57c0 2023-11-15T02:56:57.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.135+0000 7ff2fa7fc700 1 -- 172.21.15.121:0/3104638129 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff2fc01ca40 con 0x7ff30c0a57c0 2023-11-15T02:56:57.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.139+0000 7ff2fa7fc700 1 --2- 172.21.15.121:0/3104638129 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff2ec05a540 0x7ff2ec05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:57.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.139+0000 7ff2fa7fc700 1 -- 172.21.15.121:0/3104638129 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff2fc076670 con 0x7ff30c0a57c0 2023-11-15T02:56:57.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.139+0000 7ff309d9b700 1 --2- 172.21.15.121:0/3104638129 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff2ec05a540 0x7ff2ec05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:57.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.139+0000 7ff309d9b700 1 --2- 172.21.15.121:0/3104638129 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff2ec05a540 0x7ff2ec05ca00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7ff30c14e5b0 tx=0x7ff300009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:57.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.147+0000 7ff2fa7fc700 1 -- 172.21.15.121:0/3104638129 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff2fc049d90 con 0x7ff30c0a57c0 2023-11-15T02:56:57.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.311+0000 7ff31085b700 1 -- 172.21.15.121:0/3104638129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff30c0a6f50 con 0x7ff30c0a57c0 2023-11-15T02:56:57.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.311+0000 7ff2fa7fc700 1 -- 172.21.15.121:0/3104638129 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7ff2fc020020 con 0x7ff30c0a57c0 2023-11-15T02:56:57.317 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:56:57.318 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":21,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:52.702668+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:56:49.726353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-15T02:56:50.688098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:56:57.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.315+0000 7ff31085b700 1 -- 172.21.15.121:0/3104638129 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff2ec05a540 msgr2=0x7ff2ec05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:57.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.315+0000 7ff31085b700 1 --2- 172.21.15.121:0/3104638129 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff2ec05a540 0x7ff2ec05ca00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7ff30c14e5b0 tx=0x7ff300009040).stop 2023-11-15T02:56:57.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.315+0000 7ff31085b700 1 -- 172.21.15.121:0/3104638129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff30c0a57c0 msgr2=0x7ff30c0bf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:57.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.315+0000 7ff31085b700 1 --2- 172.21.15.121:0/3104638129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff30c0a57c0 0x7ff30c0bf630 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7ff2fc006fd0 tx=0x7ff2fc017000).stop 2023-11-15T02:56:57.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.315+0000 7ff31085b700 1 -- 172.21.15.121:0/3104638129 shutdown_connections 2023-11-15T02:56:57.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.315+0000 7ff31085b700 1 --2- 172.21.15.121:0/3104638129 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff2ec05a540 0x7ff2ec05ca00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.315+0000 7ff31085b700 1 --2- 172.21.15.121:0/3104638129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff30c1562f0 0x7ff30c0c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.315+0000 7ff31085b700 1 --2- 172.21.15.121:0/3104638129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff30c154ca0 0x7ff30c143110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.315+0000 7ff31085b700 1 --2- 172.21.15.121:0/3104638129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff30c0a57c0 0x7ff30c0bf630 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.315+0000 7ff31085b700 1 -- 172.21.15.121:0/3104638129 >> 172.21.15.121:0/3104638129 conn(0x7ff30c062190 msgr2=0x7ff30c14e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:57.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.315+0000 7ff31085b700 1 -- 172.21.15.121:0/3104638129 shutdown_connections 2023-11-15T02:56:57.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.315+0000 7ff31085b700 1 -- 172.21.15.121:0/3104638129 wait complete. 2023-11-15T02:56:57.339 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-15T02:56:57.340 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:56:57.373 INFO:tasks.ceph.mds.b.smithi190.stderr:2023-11-15T02:56:57.368+0000 7f210b499700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T02:56:57.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.495+0000 7fe43f49b700 1 Processor -- start 2023-11-15T02:56:57.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.499+0000 7fe43f49b700 1 -- start start 2023-11-15T02:56:57.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.499+0000 7fe43f49b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe438153bb0 0x7fe4380a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:57.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.499+0000 7fe43f49b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4380aa020 0x7fe438067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:57.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.499+0000 7fe43f49b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4380cb5c0 con 0x7fe438067f50 2023-11-15T02:56:57.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.499+0000 7fe43f49b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4380aa6f0 con 0x7fe438153bb0 2023-11-15T02:56:57.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.499+0000 7fe43f49b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4380c5900 con 0x7fe4380aa020 2023-11-15T02:56:57.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.499+0000 7fe43d237700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe438153bb0 0x7fe4380a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:57.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.499+0000 7fe43d237700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe438153bb0 0x7fe4380a9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33014/0 (socket says 172.21.15.121:33014) 2023-11-15T02:56:57.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.499+0000 7fe43d237700 1 -- 172.21.15.121:0/2538444447 learned_addr learned my addr 172.21.15.121:0/2538444447 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:57.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.499+0000 7fe43ca36700 1 --2- 172.21.15.121:0/2538444447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4380aa020 0x7fe438067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:57.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.503+0000 7fe43d237700 1 -- 172.21.15.121:0/2538444447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4380aa020 msgr2=0x7fe438067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:57.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.503+0000 7fe43d237700 1 --2- 172.21.15.121:0/2538444447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4380aa020 0x7fe438067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.503+0000 7fe43d237700 1 -- 172.21.15.121:0/2538444447 >> v1:172.21.15.190:6789/0 conn(0x7fe438067f50 legacy=0x7fe4381578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:57.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.503+0000 7fe43d237700 1 -- 172.21.15.121:0/2538444447 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4380683e0 con 0x7fe438153bb0 2023-11-15T02:56:57.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.503+0000 7fe43ca36700 1 --2- 172.21.15.121:0/2538444447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4380aa020 0x7fe438067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:56:57.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.503+0000 7fe43d237700 1 --2- 172.21.15.121:0/2538444447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe438153bb0 0x7fe4380a9b30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fe434009f20 tx=0x7fe434009840).ready entity=mon.0 client_cookie=7ae32cfdc5b2802 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:57.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.503+0000 7fe42ffff700 1 -- 172.21.15.121:0/2538444447 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43400e070 con 0x7fe438153bb0 2023-11-15T02:56:57.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.503+0000 7fe42ffff700 1 -- 172.21.15.121:0/2538444447 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43400b8d0 con 0x7fe438153bb0 2023-11-15T02:56:57.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.503+0000 7fe42ffff700 1 -- 172.21.15.121:0/2538444447 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43400bc50 con 0x7fe438153bb0 2023-11-15T02:56:57.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.503+0000 7fe43f49b700 1 -- 172.21.15.121:0/2538444447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe438153bb0 msgr2=0x7fe4380a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:57.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.503+0000 7fe43f49b700 1 --2- 172.21.15.121:0/2538444447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe438153bb0 0x7fe4380a9b30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fe434009f20 tx=0x7fe434009840).stop 2023-11-15T02:56:57.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.503+0000 7fe43f49b700 1 -- 172.21.15.121:0/2538444447 shutdown_connections 2023-11-15T02:56:57.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.503+0000 7fe43f49b700 1 --2- 172.21.15.121:0/2538444447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4380aa020 0x7fe438067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.503+0000 7fe43f49b700 1 --2- 172.21.15.121:0/2538444447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe438153bb0 0x7fe4380a9b30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.503+0000 7fe43f49b700 1 -- 172.21.15.121:0/2538444447 >> 172.21.15.121:0/2538444447 conn(0x7fe438058c50 msgr2=0x7fe438059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:57.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.503+0000 7fe43f49b700 1 -- 172.21.15.121:0/2538444447 shutdown_connections 2023-11-15T02:56:57.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.503+0000 7fe43f49b700 1 -- 172.21.15.121:0/2538444447 wait complete. 2023-11-15T02:56:57.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43f49b700 1 Processor -- start 2023-11-15T02:56:57.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43f49b700 1 -- start start 2023-11-15T02:56:57.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43f49b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe438067f50 0x7fe4380c1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:57.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43f49b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4380aa020 0x7fe43813f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:57.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43f49b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe438153bb0 0x7fe4380ac330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:57.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43f49b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4380c24f0 con 0x7fe438067f50 2023-11-15T02:56:57.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43f49b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4380c27d0 con 0x7fe438153bb0 2023-11-15T02:56:57.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43f49b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4380c2ab0 con 0x7fe4380aa020 2023-11-15T02:56:57.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43d237700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe438067f50 0x7fe4380c1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:57.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43d237700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe438067f50 0x7fe4380c1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33028/0 (socket says 172.21.15.121:33028) 2023-11-15T02:56:57.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43d237700 1 -- 172.21.15.121:0/3685073392 learned_addr learned my addr 172.21.15.121:0/3685073392 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:57.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43ca36700 1 --2- 172.21.15.121:0/3685073392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4380aa020 0x7fe43813f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:57.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43da38700 1 --2- 172.21.15.121:0/3685073392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe438153bb0 0x7fe4380ac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:57.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43d237700 1 -- 172.21.15.121:0/3685073392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4380aa020 msgr2=0x7fe43813f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:57.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43d237700 1 --2- 172.21.15.121:0/3685073392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4380aa020 0x7fe43813f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43d237700 1 -- 172.21.15.121:0/3685073392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe438153bb0 msgr2=0x7fe4380ac330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:57.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43d237700 1 --2- 172.21.15.121:0/3685073392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe438153bb0 0x7fe4380ac330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43d237700 1 -- 172.21.15.121:0/3685073392 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4280092a0 con 0x7fe438067f50 2023-11-15T02:56:57.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43d237700 1 --2- 172.21.15.121:0/3685073392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe438067f50 0x7fe4380c1cc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fe43400b610 tx=0x7fe434009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:57.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe42d7fa700 1 -- 172.21.15.121:0/3685073392 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe434009500 con 0x7fe438067f50 2023-11-15T02:56:57.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe42d7fa700 1 -- 172.21.15.121:0/3685073392 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43400ba40 con 0x7fe438067f50 2023-11-15T02:56:57.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43f49b700 1 -- 172.21.15.121:0/3685073392 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe434009ae0 con 0x7fe438067f50 2023-11-15T02:56:57.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe42d7fa700 1 -- 172.21.15.121:0/3685073392 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43401d460 con 0x7fe438067f50 2023-11-15T02:56:57.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.507+0000 7fe43f49b700 1 -- 172.21.15.121:0/3685073392 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4380c3070 con 0x7fe438067f50 2023-11-15T02:56:57.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.511+0000 7fe43f49b700 1 -- 172.21.15.121:0/3685073392 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe43806c150 con 0x7fe438067f50 2023-11-15T02:56:57.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.511+0000 7fe42d7fa700 1 -- 172.21.15.121:0/3685073392 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe434003040 con 0x7fe438067f50 2023-11-15T02:56:57.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.511+0000 7fe42d7fa700 1 --2- 172.21.15.121:0/3685073392 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe41c05a510 0x7fe41c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:57.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.511+0000 7fe42d7fa700 1 -- 172.21.15.121:0/3685073392 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe4340757e0 con 0x7fe438067f50 2023-11-15T02:56:57.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.515+0000 7fe43ca36700 1 --2- 172.21.15.121:0/3685073392 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe41c05a510 0x7fe41c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:57.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.515+0000 7fe43ca36700 1 --2- 172.21.15.121:0/3685073392 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe41c05a510 0x7fe41c05c9d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fe428009270 tx=0x7fe428008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:57.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.519+0000 7fe42d7fa700 1 -- 172.21.15.121:0/3685073392 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe434049070 con 0x7fe438067f50 2023-11-15T02:56:57.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.679+0000 7fe43f49b700 1 -- 172.21.15.121:0/3685073392 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4380c83c0 con 0x7fe438067f50 2023-11-15T02:56:57.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.679+0000 7fe42d7fa700 1 -- 172.21.15.121:0/3685073392 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fe434022070 con 0x7fe438067f50 2023-11-15T02:56:57.687 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:56:57.687 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":21,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:52.702668+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:56:49.726353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-15T02:56:50.688098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:56:57.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.683+0000 7fe43f49b700 1 -- 172.21.15.121:0/3685073392 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe41c05a510 msgr2=0x7fe41c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:57.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.683+0000 7fe43f49b700 1 --2- 172.21.15.121:0/3685073392 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe41c05a510 0x7fe41c05c9d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fe428009270 tx=0x7fe428008040).stop 2023-11-15T02:56:57.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.683+0000 7fe43f49b700 1 -- 172.21.15.121:0/3685073392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe438067f50 msgr2=0x7fe4380c1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:57.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.683+0000 7fe43f49b700 1 --2- 172.21.15.121:0/3685073392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe438067f50 0x7fe4380c1cc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fe43400b610 tx=0x7fe434009840).stop 2023-11-15T02:56:57.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.683+0000 7fe43f49b700 1 -- 172.21.15.121:0/3685073392 shutdown_connections 2023-11-15T02:56:57.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.683+0000 7fe43f49b700 1 --2- 172.21.15.121:0/3685073392 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe41c05a510 0x7fe41c05c9d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.683+0000 7fe43f49b700 1 --2- 172.21.15.121:0/3685073392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe438153bb0 0x7fe4380ac330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.683+0000 7fe43f49b700 1 --2- 172.21.15.121:0/3685073392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4380aa020 0x7fe43813f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.683+0000 7fe43f49b700 1 --2- 172.21.15.121:0/3685073392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe438067f50 0x7fe4380c1cc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.683+0000 7fe43f49b700 1 -- 172.21.15.121:0/3685073392 >> 172.21.15.121:0/3685073392 conn(0x7fe438058c50 msgr2=0x7fe438152ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:57.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.683+0000 7fe43f49b700 1 -- 172.21.15.121:0/3685073392 shutdown_connections 2023-11-15T02:56:57.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.683+0000 7fe43f49b700 1 -- 172.21.15.121:0/3685073392 wait complete. 2023-11-15T02:56:57.708 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-15T02:56:57.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.851+0000 7fd5c1c8f700 1 Processor -- start 2023-11-15T02:56:57.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.851+0000 7fd5c1c8f700 1 -- start start 2023-11-15T02:56:57.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.855+0000 7fd5c1c8f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd5bc0669b0 0x7fd5bc158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:57.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.855+0000 7fd5c1c8f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd5bc0a8750 0x7fd5bc0a8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:57.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.855+0000 7fd5c1c8f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5bc0bb910 con 0x7fd5bc157d90 2023-11-15T02:56:57.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.855+0000 7fd5c1c8f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5bc0c1fb0 con 0x7fd5bc0a8750 2023-11-15T02:56:57.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.855+0000 7fd5c1c8f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5bc15bbd0 con 0x7fd5bc0669b0 2023-11-15T02:56:57.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.855+0000 7fd5c0c8d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fd5bc157d90 0x7fd5bc0c1760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:57718/0 (socket says 172.21.15.121:57718) 2023-11-15T02:56:57.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.855+0000 7fd5c0c8d700 1 -- 172.21.15.121:0/897712540 learned_addr learned my addr 172.21.15.121:0/897712540 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:57.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.855+0000 7fd5c148e700 1 --2- 172.21.15.121:0/897712540 >> v2:172.21.15.121:3300/0 conn(0x7fd5bc0a8750 0x7fd5bc0a8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:57.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.855+0000 7fd5bbfff700 1 --2- 172.21.15.121:0/897712540 >> v2:172.21.15.190:3300/0 conn(0x7fd5bc0669b0 0x7fd5bc158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:57.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.855+0000 7fd5bb7fe700 1 -- 172.21.15.121:0/897712540 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691448419 0 0) 0x7fd5bc0bb910 con 0x7fd5bc157d90 2023-11-15T02:56:57.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.855+0000 7fd5bb7fe700 1 -- 172.21.15.121:0/897712540 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5ac0033f0 con 0x7fd5bc157d90 2023-11-15T02:56:57.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.855+0000 7fd5c148e700 1 -- 172.21.15.121:0/897712540 >> v2:172.21.15.190:3300/0 conn(0x7fd5bc0669b0 msgr2=0x7fd5bc158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:57.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.855+0000 7fd5c148e700 1 --2- 172.21.15.121:0/897712540 >> v2:172.21.15.190:3300/0 conn(0x7fd5bc0669b0 0x7fd5bc158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.855+0000 7fd5bb7fe700 1 -- 172.21.15.121:0/897712540 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1131789632 0 0) 0x7fd5ac0033f0 con 0x7fd5bc157d90 2023-11-15T02:56:57.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.855+0000 7fd5c148e700 1 -- 172.21.15.121:0/897712540 >> v1:172.21.15.121:6789/0 conn(0x7fd5bc157d90 legacy=0x7fd5bc0c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:57.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.855+0000 7fd5c148e700 1 -- 172.21.15.121:0/897712540 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc0a9090 con 0x7fd5bc0a8750 2023-11-15T02:56:57.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.855+0000 7fd5bbfff700 1 --2- 172.21.15.121:0/897712540 >> v2:172.21.15.190:3300/0 conn(0x7fd5bc0669b0 0x7fd5bc158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:56:57.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.859+0000 7fd5c148e700 1 --2- 172.21.15.121:0/897712540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5bc0a8750 0x7fd5bc0a8ba0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fd5bc15bd10 tx=0x7fd5b4009780).ready entity=mon.0 client_cookie=f66a25f98c217584 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:57.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.859+0000 7fd5bb7fe700 1 -- 172.21.15.121:0/897712540 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b400e070 con 0x7fd5bc0a8750 2023-11-15T02:56:57.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.859+0000 7fd5bb7fe700 1 -- 172.21.15.121:0/897712540 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b400b770 con 0x7fd5bc0a8750 2023-11-15T02:56:57.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.859+0000 7fd5bb7fe700 1 -- 172.21.15.121:0/897712540 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0001470 con 0x7fd5bc0a8750 2023-11-15T02:56:57.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.859+0000 7fd5c1c8f700 1 -- 172.21.15.121:0/897712540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5bc0a8750 msgr2=0x7fd5bc0a8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:57.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.859+0000 7fd5c1c8f700 1 --2- 172.21.15.121:0/897712540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5bc0a8750 0x7fd5bc0a8ba0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fd5bc15bd10 tx=0x7fd5b4009780).stop 2023-11-15T02:56:57.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.859+0000 7fd5c1c8f700 1 -- 172.21.15.121:0/897712540 shutdown_connections 2023-11-15T02:56:57.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.859+0000 7fd5c1c8f700 1 --2- 172.21.15.121:0/897712540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5bc0a8750 0x7fd5bc0a8ba0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.859+0000 7fd5c1c8f700 1 --2- 172.21.15.121:0/897712540 >> v2:172.21.15.190:3300/0 conn(0x7fd5bc0669b0 0x7fd5bc158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.859+0000 7fd5c1c8f700 1 -- 172.21.15.121:0/897712540 >> 172.21.15.121:0/897712540 conn(0x7fd5bc067a60 msgr2=0x7fd5bc067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:57.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.859+0000 7fd5c1c8f700 1 -- 172.21.15.121:0/897712540 shutdown_connections 2023-11-15T02:56:57.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.859+0000 7fd5c1c8f700 1 -- 172.21.15.121:0/897712540 wait complete. 2023-11-15T02:56:57.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.859+0000 7fd5c1c8f700 1 Processor -- start 2023-11-15T02:56:57.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.863+0000 7fd5c1c8f700 1 -- start start 2023-11-15T02:56:57.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.863+0000 7fd5c1c8f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5bc0669b0 0x7fd5bc155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:57.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.863+0000 7fd5c1c8f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5bc0a8750 0x7fd5bc15aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:57.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.863+0000 7fd5c1c8f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5bc157d90 0x7fd5bc14b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:57.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.863+0000 7fd5c1c8f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0c5760 con 0x7fd5bc0a8750 2023-11-15T02:56:57.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.863+0000 7fd5c1c8f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc156370 con 0x7fd5bc0669b0 2023-11-15T02:56:57.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.863+0000 7fd5c1c8f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc15afb0 con 0x7fd5bc157d90 2023-11-15T02:56:57.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.863+0000 7fd5bbfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5bc0a8750 0x7fd5bc15aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:57.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.863+0000 7fd5bbfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5bc0a8750 0x7fd5bc15aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33050/0 (socket says 172.21.15.121:33050) 2023-11-15T02:56:57.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.863+0000 7fd5bbfff700 1 -- 172.21.15.121:0/2006937065 learned_addr learned my addr 172.21.15.121:0/2006937065 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:57.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.863+0000 7fd5c0c8d700 1 --2- 172.21.15.121:0/2006937065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5bc0669b0 0x7fd5bc155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:57.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.863+0000 7fd5c148e700 1 --2- 172.21.15.121:0/2006937065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5bc157d90 0x7fd5bc14b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:57.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.863+0000 7fd5bbfff700 1 -- 172.21.15.121:0/2006937065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5bc157d90 msgr2=0x7fd5bc14b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:57.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.863+0000 7fd5bbfff700 1 --2- 172.21.15.121:0/2006937065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5bc157d90 0x7fd5bc14b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.863+0000 7fd5bbfff700 1 -- 172.21.15.121:0/2006937065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5bc0669b0 msgr2=0x7fd5bc155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:57.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.863+0000 7fd5bbfff700 1 --2- 172.21.15.121:0/2006937065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5bc0669b0 0x7fd5bc155e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.863+0000 7fd5bbfff700 1 -- 172.21.15.121:0/2006937065 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a8009250 con 0x7fd5bc0a8750 2023-11-15T02:56:57.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.863+0000 7fd5c0c8d700 1 --2- 172.21.15.121:0/2006937065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5bc0669b0 0x7fd5bc155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:56:57.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.863+0000 7fd5bbfff700 1 --2- 172.21.15.121:0/2006937065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5bc0a8750 0x7fd5bc15aac0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fd5a800ec80 tx=0x7fd5a8006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:57.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.867+0000 7fd5b8ff9700 1 -- 172.21.15.121:0/2006937065 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a800c9d0 con 0x7fd5bc0a8750 2023-11-15T02:56:57.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.867+0000 7fd5c1c8f700 1 -- 172.21.15.121:0/2006937065 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b4009a20 con 0x7fd5bc0a8750 2023-11-15T02:56:57.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.867+0000 7fd5b8ff9700 1 -- 172.21.15.121:0/2006937065 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a800cb30 con 0x7fd5bc0a8750 2023-11-15T02:56:57.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.867+0000 7fd5b8ff9700 1 -- 172.21.15.121:0/2006937065 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8005880 con 0x7fd5bc0a8750 2023-11-15T02:56:57.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.867+0000 7fd5c1c8f700 1 -- 172.21.15.121:0/2006937065 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5bc156b40 con 0x7fd5bc0a8750 2023-11-15T02:56:57.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.867+0000 7fd5c1c8f700 1 -- 172.21.15.121:0/2006937065 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd5bc06bcf0 con 0x7fd5bc0a8750 2023-11-15T02:56:57.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.871+0000 7fd5b8ff9700 1 -- 172.21.15.121:0/2006937065 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd5a8005cb0 con 0x7fd5bc0a8750 2023-11-15T02:56:57.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.871+0000 7fd5b8ff9700 1 --2- 172.21.15.121:0/2006937065 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd5a005a510 0x7fd5a005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:57.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.871+0000 7fd5b8ff9700 1 -- 172.21.15.121:0/2006937065 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd5a8074af0 con 0x7fd5bc0a8750 2023-11-15T02:56:57.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.871+0000 7fd5c0c8d700 1 --2- 172.21.15.121:0/2006937065 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd5a005a510 0x7fd5a005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:57.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.871+0000 7fd5b8ff9700 1 --2- 172.21.15.121:0/2006937065 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7fd5a0063820 0x7fd5a0065c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:57.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.871+0000 7fd5b8ff9700 1 -- 172.21.15.121:0/2006937065 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5a0066300 con 0x7fd5a0063820 2023-11-15T02:56:57.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.871+0000 7fd5b8ff9700 1 -- 172.21.15.121:0/2006937065 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5a8079220 con 0x7fd5bc0a8750 2023-11-15T02:56:57.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.871+0000 7fd5c148e700 1 --2- 172.21.15.121:0/2006937065 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7fd5a0063820 0x7fd5a0065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:57.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.871+0000 7fd5c0c8d700 1 --2- 172.21.15.121:0/2006937065 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd5a005a510 0x7fd5a005c9d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fd5b00011e0 tx=0x7fd5b00015e0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:57.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.871+0000 7fd5c148e700 1 --2- 172.21.15.121:0/2006937065 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7fd5a0063820 0x7fd5a0065c80 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:57.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.875+0000 7fd5b8ff9700 1 -- 172.21.15.121:0/2006937065 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd5a0066300 con 0x7fd5a0063820 2023-11-15T02:56:57.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.891+0000 7fd5c1c8f700 1 -- 172.21.15.121:0/2006937065 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd5bc068a70 con 0x7fd5a0063820 2023-11-15T02:56:57.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.891+0000 7fd5b8ff9700 1 -- 172.21.15.121:0/2006937065 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd5bc068a70 con 0x7fd5a0063820 2023-11-15T02:56:57.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.891+0000 7fd5c1c8f700 1 -- 172.21.15.121:0/2006937065 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7fd5a0063820 msgr2=0x7fd5a0065c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:57.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.891+0000 7fd5c1c8f700 1 --2- 172.21.15.121:0/2006937065 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7fd5a0063820 0x7fd5a0065c80 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:57.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.891+0000 7fd5c1c8f700 1 -- 172.21.15.121:0/2006937065 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd5a005a510 msgr2=0x7fd5a005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:57.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.891+0000 7fd5c1c8f700 1 --2- 172.21.15.121:0/2006937065 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd5a005a510 0x7fd5a005c9d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fd5b00011e0 tx=0x7fd5b00015e0).stop 2023-11-15T02:56:57.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.891+0000 7fd5c1c8f700 1 -- 172.21.15.121:0/2006937065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5bc0a8750 msgr2=0x7fd5bc15aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:57.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.891+0000 7fd5c1c8f700 1 --2- 172.21.15.121:0/2006937065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5bc0a8750 0x7fd5bc15aac0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fd5a800ec80 tx=0x7fd5a8006f40).stop 2023-11-15T02:56:57.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.891+0000 7fd5c148e700 1 -- 172.21.15.121:0/2006937065 reap_dead start 2023-11-15T02:56:57.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.891+0000 7fd5c1c8f700 1 -- 172.21.15.121:0/2006937065 shutdown_connections 2023-11-15T02:56:57.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.891+0000 7fd5c1c8f700 1 -- 172.21.15.121:0/2006937065 >> 172.21.15.121:0/2006937065 conn(0x7fd5bc067a60 msgr2=0x7fd5bc0c5900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:57.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.891+0000 7fd5c1c8f700 1 -- 172.21.15.121:0/2006937065 shutdown_connections 2023-11-15T02:56:57.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:57.891+0000 7fd5c1c8f700 1 -- 172.21.15.121:0/2006937065 wait complete. 2023-11-15T02:56:57.917 INFO:teuthology.orchestra.run.smithi121.stdout:60129542149 2023-11-15T02:56:57.917 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-15T02:56:58.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.075+0000 7f76f86db700 1 Processor -- start 2023-11-15T02:56:58.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.075+0000 7f76f86db700 1 -- start start 2023-11-15T02:56:58.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f86db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f76f0153aa0 0x7f76f0066680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f86db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f76f0153e80 0x7f76f00ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f86db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76f00c0490 0x7f76f00c0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f86db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f0066b70 con 0x7f76f0153e80 2023-11-15T02:56:58.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f86db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f00c5c50 con 0x7f76f0153aa0 2023-11-15T02:56:58.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f86db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00c0e10 con 0x7f76f00c0490 2023-11-15T02:56:58.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f5c76700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f76f0153e80 0x7f76f00ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f5c76700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f76f0153e80 0x7f76f00ad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33064/0 (socket says 172.21.15.121:33064) 2023-11-15T02:56:58.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f5c76700 1 -- 172.21.15.121:0/785623271 learned_addr learned my addr 172.21.15.121:0/785623271 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:58.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f6477700 1 --2- 172.21.15.121:0/785623271 >> v2:172.21.15.190:3300/0 conn(0x7f76f0153aa0 0x7f76f0066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f6c78700 1 --2- 172.21.15.121:0/785623271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76f00c0490 0x7f76f00c0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f5c76700 1 -- 172.21.15.121:0/785623271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76f00c0490 msgr2=0x7f76f00c0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:58.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f5c76700 1 --2- 172.21.15.121:0/785623271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76f00c0490 0x7f76f00c0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f5c76700 1 -- 172.21.15.121:0/785623271 >> v2:172.21.15.190:3300/0 conn(0x7f76f0153aa0 msgr2=0x7f76f0066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:58.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f5c76700 1 --2- 172.21.15.121:0/785623271 >> v2:172.21.15.190:3300/0 conn(0x7f76f0153aa0 0x7f76f0066680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f5c76700 1 -- 172.21.15.121:0/785623271 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f00c5920 con 0x7f76f0153e80 2023-11-15T02:56:58.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f6c78700 1 --2- 172.21.15.121:0/785623271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76f00c0490 0x7f76f00c0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:56:58.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f5c76700 1 --2- 172.21.15.121:0/785623271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76f0153e80 0x7f76f00ad150 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f76f00cbaf0 tx=0x7f76e00090f0).ready entity=mon.0 client_cookie=596f0b6d2cf3b87d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:58.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f5475700 1 -- 172.21.15.121:0/785623271 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e000b520 con 0x7f76f0153e80 2023-11-15T02:56:58.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f5475700 1 -- 172.21.15.121:0/785623271 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e000b680 con 0x7f76f0153e80 2023-11-15T02:56:58.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.079+0000 7f76f5475700 1 -- 172.21.15.121:0/785623271 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e000ba00 con 0x7f76f0153e80 2023-11-15T02:56:58.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.083+0000 7f76f86db700 1 -- 172.21.15.121:0/785623271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76f0153e80 msgr2=0x7f76f00ad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.083+0000 7f76f86db700 1 --2- 172.21.15.121:0/785623271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76f0153e80 0x7f76f00ad150 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f76f00cbaf0 tx=0x7f76e00090f0).stop 2023-11-15T02:56:58.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.083+0000 7f76f86db700 1 -- 172.21.15.121:0/785623271 shutdown_connections 2023-11-15T02:56:58.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.083+0000 7f76f86db700 1 --2- 172.21.15.121:0/785623271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76f00c0490 0x7f76f00c0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.083+0000 7f76f86db700 1 --2- 172.21.15.121:0/785623271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76f0153e80 0x7f76f00ad150 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.083+0000 7f76f86db700 1 --2- 172.21.15.121:0/785623271 >> v2:172.21.15.190:3300/0 conn(0x7f76f0153aa0 0x7f76f0066680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.083+0000 7f76f86db700 1 -- 172.21.15.121:0/785623271 >> 172.21.15.121:0/785623271 conn(0x7f76f0067a60 msgr2=0x7f76f0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:58.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.083+0000 7f76f86db700 1 -- 172.21.15.121:0/785623271 shutdown_connections 2023-11-15T02:56:58.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.083+0000 7f76f86db700 1 -- 172.21.15.121:0/785623271 wait complete. 2023-11-15T02:56:58.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.083+0000 7f76f86db700 1 Processor -- start 2023-11-15T02:56:58.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.083+0000 7f76f86db700 1 -- start start 2023-11-15T02:56:58.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76f86db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76f00c0490 0x7f76f0164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76f86db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76f0153aa0 0x7f76f0154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76f86db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76f0153e80 0x7f76f0156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76f86db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f0164a40 con 0x7f76f00c0490 2023-11-15T02:56:58.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76f86db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f0157280 con 0x7f76f0153e80 2023-11-15T02:56:58.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76f86db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f0156fa0 con 0x7f76f0153aa0 2023-11-15T02:56:58.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76f6477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76f00c0490 0x7f76f0164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76f6477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76f00c0490 0x7f76f0164ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33072/0 (socket says 172.21.15.121:33072) 2023-11-15T02:56:58.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76f6477700 1 -- 172.21.15.121:0/112239496 learned_addr learned my addr 172.21.15.121:0/112239496 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:58.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76f6477700 1 -- 172.21.15.121:0/112239496 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76f0153aa0 msgr2=0x7f76f0154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76f6477700 1 --2- 172.21.15.121:0/112239496 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76f0153aa0 0x7f76f0154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T02:56:58.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76f6477700 1 -- 172.21.15.121:0/112239496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76f0153e80 msgr2=0x7f76f0156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76f6477700 1 --2- 172.21.15.121:0/112239496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76f0153e80 0x7f76f0156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T02:56:58.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76f6477700 1 -- 172.21.15.121:0/112239496 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e0009370 con 0x7f76f00c0490 2023-11-15T02:56:58.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76f6477700 1 --2- 172.21.15.121:0/112239496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76f00c0490 0x7f76f0164ed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f76ec00e930 tx=0x7f76ec009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:58.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76e67fc700 1 -- 172.21.15.121:0/112239496 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec00eed0 con 0x7f76f00c0490 2023-11-15T02:56:58.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76e67fc700 1 -- 172.21.15.121:0/112239496 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ec00c850 con 0x7f76f00c0490 2023-11-15T02:56:58.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76f86db700 1 -- 172.21.15.121:0/112239496 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f01643f0 con 0x7f76f00c0490 2023-11-15T02:56:58.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76e67fc700 1 -- 172.21.15.121:0/112239496 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec00cbd0 con 0x7f76f00c0490 2023-11-15T02:56:58.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76f86db700 1 -- 172.21.15.121:0/112239496 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f00ad090 con 0x7f76f00c0490 2023-11-15T02:56:58.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.087+0000 7f76f86db700 1 -- 172.21.15.121:0/112239496 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f76f0047060 con 0x7f76f00c0490 2023-11-15T02:56:58.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.091+0000 7f76e67fc700 1 -- 172.21.15.121:0/112239496 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f76ec01b400 con 0x7f76f00c0490 2023-11-15T02:56:58.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.091+0000 7f76e67fc700 1 --2- 172.21.15.121:0/112239496 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f76d405a510 0x7f76d405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.091+0000 7f76e67fc700 1 -- 172.21.15.121:0/112239496 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f76ec074e60 con 0x7f76f00c0490 2023-11-15T02:56:58.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.091+0000 7f76f5c76700 1 --2- 172.21.15.121:0/112239496 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f76d405a510 0x7f76d405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.091+0000 7f76e67fc700 1 --2- 172.21.15.121:0/112239496 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7f76d4063740 0x7f76d4065ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.091+0000 7f76e67fc700 1 -- 172.21.15.121:0/112239496 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76d4066220 con 0x7f76d4063740 2023-11-15T02:56:58.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.091+0000 7f76e67fc700 1 -- 172.21.15.121:0/112239496 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76ec0773c0 con 0x7f76f00c0490 2023-11-15T02:56:58.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.091+0000 7f76f6c78700 1 --2- 172.21.15.121:0/112239496 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7f76d4063740 0x7f76d4065ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.091+0000 7f76f5c76700 1 --2- 172.21.15.121:0/112239496 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f76d405a510 0x7f76d405c9d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f76f00c99d0 tx=0x7f76e0017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:58.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.091+0000 7f76f6c78700 1 --2- 172.21.15.121:0/112239496 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7f76d4063740 0x7f76d4065ba0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:58.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.095+0000 7f76e67fc700 1 -- 172.21.15.121:0/112239496 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f76d4066220 con 0x7f76d4063740 2023-11-15T02:56:58.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.115+0000 7f76f86db700 1 -- 172.21.15.121:0/112239496 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f76f0164b80 con 0x7f76d4063740 2023-11-15T02:56:58.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.115+0000 7f76e67fc700 1 -- 172.21.15.121:0/112239496 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f76e800f040 con 0x7f76d4063740 2023-11-15T02:56:58.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.115+0000 7f76f86db700 1 -- 172.21.15.121:0/112239496 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7f76d4063740 msgr2=0x7f76d4065ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.115+0000 7f76f86db700 1 --2- 172.21.15.121:0/112239496 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7f76d4063740 0x7f76d4065ba0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.115+0000 7f76f86db700 1 -- 172.21.15.121:0/112239496 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f76d405a510 msgr2=0x7f76d405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.115+0000 7f76f86db700 1 --2- 172.21.15.121:0/112239496 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f76d405a510 0x7f76d405c9d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f76f00c99d0 tx=0x7f76e0017000).stop 2023-11-15T02:56:58.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.115+0000 7f76f86db700 1 -- 172.21.15.121:0/112239496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76f00c0490 msgr2=0x7f76f0164ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.115+0000 7f76f86db700 1 --2- 172.21.15.121:0/112239496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76f00c0490 0x7f76f0164ed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f76ec00e930 tx=0x7f76ec009060).stop 2023-11-15T02:56:58.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.115+0000 7f76f6c78700 1 -- 172.21.15.121:0/112239496 reap_dead start 2023-11-15T02:56:58.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.115+0000 7f76f86db700 1 -- 172.21.15.121:0/112239496 shutdown_connections 2023-11-15T02:56:58.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.115+0000 7f76f86db700 1 -- 172.21.15.121:0/112239496 >> 172.21.15.121:0/112239496 conn(0x7f76f0067a60 msgr2=0x7f76f0159030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:58.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.115+0000 7f76f86db700 1 -- 172.21.15.121:0/112239496 shutdown_connections 2023-11-15T02:56:58.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.115+0000 7f76f86db700 1 -- 172.21.15.121:0/112239496 wait complete. 2023-11-15T02:56:58.144 INFO:teuthology.orchestra.run.smithi121.stdout:55834574852 2023-11-15T02:56:58.144 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-15T02:56:58.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.303+0000 7f4fb08db700 1 Processor -- start 2023-11-15T02:56:58.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.303+0000 7f4fb08db700 1 -- start start 2023-11-15T02:56:58.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.307+0000 7f4fb08db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fac153e80 0x7f4fac066680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.307+0000 7f4fb08db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4fac0c0490 0x7f4fac0c0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.307+0000 7f4fb08db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fac0c5c50 con 0x7f4fac153aa0 2023-11-15T02:56:58.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.307+0000 7f4fb08db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fac066b70 con 0x7f4fac0c0490 2023-11-15T02:56:58.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.307+0000 7f4fb08db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0c0e10 con 0x7f4fac153e80 2023-11-15T02:56:58.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.307+0000 7f4faa59c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4fac153aa0 0x7f4fac0ad150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:57722/0 (socket says 172.21.15.121:57722) 2023-11-15T02:56:58.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.307+0000 7f4faa59c700 1 -- 172.21.15.121:0/454646147 learned_addr learned my addr 172.21.15.121:0/454646147 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:58.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.307+0000 7f4faad9d700 1 --2- 172.21.15.121:0/454646147 >> v2:172.21.15.190:3300/0 conn(0x7f4fac0c0490 0x7f4fac0c0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.307+0000 7f4fa3fff700 1 --2- 172.21.15.121:0/454646147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fac153e80 0x7f4fac066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.307+0000 7f4fa9d9b700 1 -- 172.21.15.121:0/454646147 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 465001345 0 0) 0x7f4fac0c5c50 con 0x7f4fac153aa0 2023-11-15T02:56:58.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.307+0000 7f4fa9d9b700 1 -- 172.21.15.121:0/454646147 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f980033f0 con 0x7f4fac153aa0 2023-11-15T02:56:58.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.307+0000 7f4faad9d700 1 -- 172.21.15.121:0/454646147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fac153e80 msgr2=0x7f4fac066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:58.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.307+0000 7f4faad9d700 1 --2- 172.21.15.121:0/454646147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fac153e80 0x7f4fac066680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.307+0000 7f4faad9d700 1 -- 172.21.15.121:0/454646147 >> v1:172.21.15.121:6789/0 conn(0x7f4fac153aa0 legacy=0x7f4fac0ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.307+0000 7f4faad9d700 1 -- 172.21.15.121:0/454646147 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fac0c5920 con 0x7f4fac0c0490 2023-11-15T02:56:58.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.307+0000 7f4faad9d700 1 --2- 172.21.15.121:0/454646147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fac0c0490 0x7f4fac0c0920 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f4fa4009e60 tx=0x7f4fa4009780).ready entity=mon.1 client_cookie=1082c04fae34982 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:58.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.311+0000 7f4fa9d9b700 1 -- 172.21.15.121:0/454646147 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa400e070 con 0x7f4fac0c0490 2023-11-15T02:56:58.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.311+0000 7f4fa9d9b700 1 -- 172.21.15.121:0/454646147 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa400bab0 con 0x7f4fac0c0490 2023-11-15T02:56:58.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.311+0000 7f4fa9d9b700 1 -- 172.21.15.121:0/454646147 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa400be30 con 0x7f4fac0c0490 2023-11-15T02:56:58.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.311+0000 7f4fb08db700 1 -- 172.21.15.121:0/454646147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fac0c0490 msgr2=0x7f4fac0c0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.311+0000 7f4fb08db700 1 --2- 172.21.15.121:0/454646147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fac0c0490 0x7f4fac0c0920 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f4fa4009e60 tx=0x7f4fa4009780).stop 2023-11-15T02:56:58.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.311+0000 7f4fb08db700 1 -- 172.21.15.121:0/454646147 shutdown_connections 2023-11-15T02:56:58.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.311+0000 7f4fb08db700 1 --2- 172.21.15.121:0/454646147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fac0c0490 0x7f4fac0c0920 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.311+0000 7f4fb08db700 1 --2- 172.21.15.121:0/454646147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fac153e80 0x7f4fac066680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.311+0000 7f4fb08db700 1 -- 172.21.15.121:0/454646147 >> 172.21.15.121:0/454646147 conn(0x7f4fac067a60 msgr2=0x7f4fac067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:58.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.311+0000 7f4fb08db700 1 -- 172.21.15.121:0/454646147 shutdown_connections 2023-11-15T02:56:58.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.311+0000 7f4fb08db700 1 -- 172.21.15.121:0/454646147 wait complete. 2023-11-15T02:56:58.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.315+0000 7f4fb08db700 1 Processor -- start 2023-11-15T02:56:58.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.315+0000 7f4fb08db700 1 -- start start 2023-11-15T02:56:58.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.315+0000 7f4fb08db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fac0c0490 0x7f4fac14f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.315+0000 7f4fb08db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fac153aa0 0x7f4fac1577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.315+0000 7f4fb08db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fac153e80 0x7f4fac1562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.315+0000 7f4fb08db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac155e10 con 0x7f4fac153aa0 2023-11-15T02:56:58.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.315+0000 7f4fb08db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac155200 con 0x7f4fac0c0490 2023-11-15T02:56:58.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.315+0000 7f4fb08db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac1554e0 con 0x7f4fac153e80 2023-11-15T02:56:58.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.315+0000 7f4fa3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fac153aa0 0x7f4fac1577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.315+0000 7f4fa3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fac153aa0 0x7f4fac1577d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33088/0 (socket says 172.21.15.121:33088) 2023-11-15T02:56:58.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.315+0000 7f4fa3fff700 1 -- 172.21.15.121:0/1804713518 learned_addr learned my addr 172.21.15.121:0/1804713518 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:58.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.315+0000 7f4faad9d700 1 --2- 172.21.15.121:0/1804713518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fac153e80 0x7f4fac1562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.315+0000 7f4faa59c700 1 --2- 172.21.15.121:0/1804713518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fac0c0490 0x7f4fac14f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.315+0000 7f4fa3fff700 1 -- 172.21.15.121:0/1804713518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fac153e80 msgr2=0x7f4fac1562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.315+0000 7f4fa3fff700 1 --2- 172.21.15.121:0/1804713518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fac153e80 0x7f4fac1562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.315+0000 7f4fa3fff700 1 -- 172.21.15.121:0/1804713518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fac0c0490 msgr2=0x7f4fac14f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.315+0000 7f4fa3fff700 1 --2- 172.21.15.121:0/1804713518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fac0c0490 0x7f4fac14f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.315+0000 7f4fa3fff700 1 -- 172.21.15.121:0/1804713518 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa4009a20 con 0x7f4fac153aa0 2023-11-15T02:56:58.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.319+0000 7f4fa3fff700 1 --2- 172.21.15.121:0/1804713518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fac153aa0 0x7f4fac1577d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f4f9400e900 tx=0x7f4f94009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:58.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.319+0000 7f4fa27fc700 1 -- 172.21.15.121:0/1804713518 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9400c640 con 0x7f4fac153aa0 2023-11-15T02:56:58.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.319+0000 7f4fa27fc700 1 -- 172.21.15.121:0/1804713518 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9400c7a0 con 0x7f4fac153aa0 2023-11-15T02:56:58.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.319+0000 7f4fa27fc700 1 -- 172.21.15.121:0/1804713518 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9400cb20 con 0x7f4fac153aa0 2023-11-15T02:56:58.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.319+0000 7f4fb08db700 1 -- 172.21.15.121:0/1804713518 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fac1557c0 con 0x7f4fac153aa0 2023-11-15T02:56:58.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.319+0000 7f4fb08db700 1 -- 172.21.15.121:0/1804713518 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fac14f5d0 con 0x7f4fac153aa0 2023-11-15T02:56:58.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.319+0000 7f4fb08db700 1 -- 172.21.15.121:0/1804713518 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4fac047060 con 0x7f4fac153aa0 2023-11-15T02:56:58.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.319+0000 7f4fa27fc700 1 -- 172.21.15.121:0/1804713518 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4f9401b400 con 0x7f4fac153aa0 2023-11-15T02:56:58.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.323+0000 7f4fa27fc700 1 --2- 172.21.15.121:0/1804713518 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4f8c05a510 0x7f4f8c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.323+0000 7f4fa27fc700 1 -- 172.21.15.121:0/1804713518 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4f940752c0 con 0x7f4fac153aa0 2023-11-15T02:56:58.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.323+0000 7f4faa59c700 1 --2- 172.21.15.121:0/1804713518 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4f8c05a510 0x7f4f8c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.323+0000 7f4fa27fc700 1 --2- 172.21.15.121:0/1804713518 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7f4f8c0638b0 0x7f4f8c065d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.323+0000 7f4fa27fc700 1 -- 172.21.15.121:0/1804713518 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f8c066390 con 0x7f4f8c0638b0 2023-11-15T02:56:58.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.323+0000 7f4fa27fc700 1 -- 172.21.15.121:0/1804713518 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f940755b0 con 0x7f4fac153aa0 2023-11-15T02:56:58.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.323+0000 7f4faad9d700 1 --2- 172.21.15.121:0/1804713518 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7f4f8c0638b0 0x7f4f8c065d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.323+0000 7f4faa59c700 1 --2- 172.21.15.121:0/1804713518 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4f8c05a510 0x7f4f8c05c9d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f4f9c005750 tx=0x7f4f9c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:58.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.323+0000 7f4faad9d700 1 --2- 172.21.15.121:0/1804713518 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7f4f8c0638b0 0x7f4f8c065d10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:58.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.323+0000 7f4fa27fc700 1 -- 172.21.15.121:0/1804713518 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4f8c066390 con 0x7f4f8c0638b0 2023-11-15T02:56:58.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.343+0000 7f4fb08db700 1 -- 172.21.15.121:0/1804713518 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4fac066d50 con 0x7f4f8c0638b0 2023-11-15T02:56:58.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.343+0000 7f4fa27fc700 1 -- 172.21.15.121:0/1804713518 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4fac066d50 con 0x7f4f8c0638b0 2023-11-15T02:56:58.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.343+0000 7f4fb08db700 1 -- 172.21.15.121:0/1804713518 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7f4f8c0638b0 msgr2=0x7f4f8c065d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.343+0000 7f4fb08db700 1 --2- 172.21.15.121:0/1804713518 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7f4f8c0638b0 0x7f4f8c065d10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.347+0000 7f4fb08db700 1 -- 172.21.15.121:0/1804713518 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4f8c05a510 msgr2=0x7f4f8c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.347+0000 7f4fb08db700 1 --2- 172.21.15.121:0/1804713518 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4f8c05a510 0x7f4f8c05c9d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f4f9c005750 tx=0x7f4f9c009040).stop 2023-11-15T02:56:58.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.347+0000 7f4fb08db700 1 -- 172.21.15.121:0/1804713518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fac153aa0 msgr2=0x7f4fac1577d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.347+0000 7f4fb08db700 1 --2- 172.21.15.121:0/1804713518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fac153aa0 0x7f4fac1577d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f4f9400e900 tx=0x7f4f94009060).stop 2023-11-15T02:56:58.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.347+0000 7f4faad9d700 1 -- 172.21.15.121:0/1804713518 reap_dead start 2023-11-15T02:56:58.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.347+0000 7f4fb08db700 1 -- 172.21.15.121:0/1804713518 shutdown_connections 2023-11-15T02:56:58.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.347+0000 7f4fb08db700 1 -- 172.21.15.121:0/1804713518 >> 172.21.15.121:0/1804713518 conn(0x7f4fac067a60 msgr2=0x7f4fac0c5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:58.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.347+0000 7f4fb08db700 1 -- 172.21.15.121:0/1804713518 shutdown_connections 2023-11-15T02:56:58.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.347+0000 7f4fb08db700 1 -- 172.21.15.121:0/1804713518 wait complete. 2023-11-15T02:56:58.370 INFO:teuthology.orchestra.run.smithi121.stdout:60129542149 2023-11-15T02:56:58.371 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-15T02:56:58.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.523+0000 7f7f9b41b700 1 Processor -- start 2023-11-15T02:56:58.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.523+0000 7f7f9b41b700 1 -- start start 2023-11-15T02:56:58.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.527+0000 7f7f9b41b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7f940628a0 0x7f7f94158210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.527+0000 7f7f9b41b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f940aaf80 con 0x7f7f940ac420 2023-11-15T02:56:58.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.527+0000 7f7f9b41b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f940aa970 con 0x7f7f94157e30 2023-11-15T02:56:58.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.527+0000 7f7f9b41b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f94066e10 con 0x7f7f940628a0 2023-11-15T02:56:58.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.527+0000 7f7f989b6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7f940628a0 0x7f7f94158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.527+0000 7f7f989b6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7f940628a0 0x7f7f94158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33104/0 (socket says 172.21.15.121:33104) 2023-11-15T02:56:58.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.527+0000 7f7f989b6700 1 -- 172.21.15.121:0/764572819 learned_addr learned my addr 172.21.15.121:0/764572819 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:58.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.527+0000 7f7f989b6700 1 -- 172.21.15.121:0/764572819 >> v1:172.21.15.190:6789/0 conn(0x7f7f94157e30 legacy=0x7f7f94062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:58.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.527+0000 7f7f989b6700 1 -- 172.21.15.121:0/764572819 >> v1:172.21.15.121:6789/0 conn(0x7f7f940ac420 legacy=0x7f7f940660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.527+0000 7f7f989b6700 1 -- 172.21.15.121:0/764572819 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f940670f0 con 0x7f7f940628a0 2023-11-15T02:56:58.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.527+0000 7f7f989b6700 1 --2- 172.21.15.121:0/764572819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f940628a0 0x7f7f94158210 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f7f84006fd0 tx=0x7f7f840090f0).ready entity=mon.0 client_cookie=f0ee8b39b6367464 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:58.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.527+0000 7f7f8bfff700 1 -- 172.21.15.121:0/764572819 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8400b600 con 0x7f7f940628a0 2023-11-15T02:56:58.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.527+0000 7f7f8bfff700 1 -- 172.21.15.121:0/764572819 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8400b760 con 0x7f7f940628a0 2023-11-15T02:56:58.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.527+0000 7f7f8bfff700 1 -- 172.21.15.121:0/764572819 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8400bae0 con 0x7f7f940628a0 2023-11-15T02:56:58.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.527+0000 7f7f9b41b700 1 -- 172.21.15.121:0/764572819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f940628a0 msgr2=0x7f7f94158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.527+0000 7f7f9b41b700 1 --2- 172.21.15.121:0/764572819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f940628a0 0x7f7f94158210 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f7f84006fd0 tx=0x7f7f840090f0).stop 2023-11-15T02:56:58.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.527+0000 7f7f9b41b700 1 -- 172.21.15.121:0/764572819 shutdown_connections 2023-11-15T02:56:58.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.527+0000 7f7f9b41b700 1 --2- 172.21.15.121:0/764572819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f940628a0 0x7f7f94158210 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.527+0000 7f7f9b41b700 1 -- 172.21.15.121:0/764572819 >> 172.21.15.121:0/764572819 conn(0x7f7f94067a60 msgr2=0x7f7f94067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:58.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.531+0000 7f7f9b41b700 1 -- 172.21.15.121:0/764572819 shutdown_connections 2023-11-15T02:56:58.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.531+0000 7f7f9b41b700 1 -- 172.21.15.121:0/764572819 wait complete. 2023-11-15T02:56:58.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.531+0000 7f7f9b41b700 1 Processor -- start 2023-11-15T02:56:58.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.531+0000 7f7f9b41b700 1 -- start start 2023-11-15T02:56:58.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f9b41b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f940628a0 0x7f7f94156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f9b41b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f940ac420 0x7f7f940bcf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f9b41b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f94157e30 0x7f7f940bb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f9b41b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f940bcad0 con 0x7f7f940628a0 2023-11-15T02:56:58.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f9b41b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f940bbec0 con 0x7f7f94157e30 2023-11-15T02:56:58.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f9b41b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f940bc1a0 con 0x7f7f940ac420 2023-11-15T02:56:58.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f991b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f940628a0 0x7f7f94156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f991b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f940628a0 0x7f7f94156500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33110/0 (socket says 172.21.15.121:33110) 2023-11-15T02:56:58.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f991b7700 1 -- 172.21.15.121:0/1569486673 learned_addr learned my addr 172.21.15.121:0/1569486673 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:58.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f989b6700 1 --2- 172.21.15.121:0/1569486673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f940ac420 0x7f7f940bcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f999b8700 1 --2- 172.21.15.121:0/1569486673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f94157e30 0x7f7f940bb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f991b7700 1 -- 172.21.15.121:0/1569486673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f940ac420 msgr2=0x7f7f940bcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f991b7700 1 --2- 172.21.15.121:0/1569486673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f940ac420 0x7f7f940bcf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f991b7700 1 -- 172.21.15.121:0/1569486673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f94157e30 msgr2=0x7f7f940bb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f991b7700 1 --2- 172.21.15.121:0/1569486673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f94157e30 0x7f7f940bb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f991b7700 1 -- 172.21.15.121:0/1569486673 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f84009370 con 0x7f7f940628a0 2023-11-15T02:56:58.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f991b7700 1 --2- 172.21.15.121:0/1569486673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f940628a0 0x7f7f94156500 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f7f9000b460 tx=0x7f7f90006f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:58.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f897fa700 1 -- 172.21.15.121:0/1569486673 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90012410 con 0x7f7f940628a0 2023-11-15T02:56:58.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f897fa700 1 -- 172.21.15.121:0/1569486673 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f90012590 con 0x7f7f940628a0 2023-11-15T02:56:58.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f897fa700 1 -- 172.21.15.121:0/1569486673 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90012950 con 0x7f7f940628a0 2023-11-15T02:56:58.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f9b41b700 1 -- 172.21.15.121:0/1569486673 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f940bc480 con 0x7f7f940628a0 2023-11-15T02:56:58.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.535+0000 7f7f9b41b700 1 -- 172.21.15.121:0/1569486673 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f940bbba0 con 0x7f7f940628a0 2023-11-15T02:56:58.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.539+0000 7f7f9b41b700 1 -- 172.21.15.121:0/1569486673 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7f94047060 con 0x7f7f940628a0 2023-11-15T02:56:58.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.539+0000 7f7f897fa700 1 -- 172.21.15.121:0/1569486673 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7f90012d00 con 0x7f7f940628a0 2023-11-15T02:56:58.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.539+0000 7f7f897fa700 1 --2- 172.21.15.121:0/1569486673 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7f7805a510 0x7f7f7805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.539+0000 7f7f897fa700 1 -- 172.21.15.121:0/1569486673 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7f90074590 con 0x7f7f940628a0 2023-11-15T02:56:58.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.539+0000 7f7f989b6700 1 --2- 172.21.15.121:0/1569486673 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7f7805a510 0x7f7f7805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.539+0000 7f7f897fa700 1 --2- 172.21.15.121:0/1569486673 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7f7f780638b0 0x7f7f78065d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.539+0000 7f7f897fa700 1 -- 172.21.15.121:0/1569486673 --> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f78066390 con 0x7f7f780638b0 2023-11-15T02:56:58.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.539+0000 7f7f897fa700 1 -- 172.21.15.121:0/1569486673 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f90076bb0 con 0x7f7f940628a0 2023-11-15T02:56:58.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.539+0000 7f7f999b8700 1 --2- 172.21.15.121:0/1569486673 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7f7f780638b0 0x7f7f78065d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.543+0000 7f7f989b6700 1 --2- 172.21.15.121:0/1569486673 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7f7805a510 0x7f7f7805c9d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f7f84009130 tx=0x7f7f84017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:58.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.543+0000 7f7f999b8700 1 --2- 172.21.15.121:0/1569486673 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7f7f780638b0 0x7f7f78065d10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:58.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.543+0000 7f7f897fa700 1 -- 172.21.15.121:0/1569486673 <== osd.3 v2:172.21.15.121:6820/33135 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7f78066390 con 0x7f7f780638b0 2023-11-15T02:56:58.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.563+0000 7f7f9b41b700 1 -- 172.21.15.121:0/1569486673 --> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7f940bcc10 con 0x7f7f780638b0 2023-11-15T02:56:58.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.563+0000 7f7f897fa700 1 -- 172.21.15.121:0/1569486673 <== osd.3 v2:172.21.15.121:6820/33135 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7f8c003e80 con 0x7f7f780638b0 2023-11-15T02:56:58.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.563+0000 7f7f9b41b700 1 -- 172.21.15.121:0/1569486673 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7f7f780638b0 msgr2=0x7f7f78065d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.563+0000 7f7f9b41b700 1 --2- 172.21.15.121:0/1569486673 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7f7f780638b0 0x7f7f78065d10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.567+0000 7f7f9b41b700 1 -- 172.21.15.121:0/1569486673 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7f7805a510 msgr2=0x7f7f7805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.567+0000 7f7f9b41b700 1 --2- 172.21.15.121:0/1569486673 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7f7805a510 0x7f7f7805c9d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f7f84009130 tx=0x7f7f84017000).stop 2023-11-15T02:56:58.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.567+0000 7f7f9b41b700 1 -- 172.21.15.121:0/1569486673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f940628a0 msgr2=0x7f7f94156500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.567+0000 7f7f9b41b700 1 --2- 172.21.15.121:0/1569486673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f940628a0 0x7f7f94156500 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f7f9000b460 tx=0x7f7f90006f20).stop 2023-11-15T02:56:58.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.567+0000 7f7f999b8700 1 -- 172.21.15.121:0/1569486673 reap_dead start 2023-11-15T02:56:58.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.567+0000 7f7f9b41b700 1 -- 172.21.15.121:0/1569486673 shutdown_connections 2023-11-15T02:56:58.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.567+0000 7f7f9b41b700 1 -- 172.21.15.121:0/1569486673 >> 172.21.15.121:0/1569486673 conn(0x7f7f94067a60 msgr2=0x7f7f940c43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:58.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.567+0000 7f7f9b41b700 1 -- 172.21.15.121:0/1569486673 shutdown_connections 2023-11-15T02:56:58.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.567+0000 7f7f9b41b700 1 -- 172.21.15.121:0/1569486673 wait complete. 2023-11-15T02:56:58.592 INFO:teuthology.orchestra.run.smithi121.stdout:60129542149 2023-11-15T02:56:58.592 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-15T02:56:58.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.755+0000 7f3fd0769700 1 Processor -- start 2023-11-15T02:56:58.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.755+0000 7f3fd0769700 1 -- start start 2023-11-15T02:56:58.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.759+0000 7f3fd0769700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fc8151920 0x7f3fc80bb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.759+0000 7f3fd0769700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3fc80ac150 0x7f3fc8062190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.759+0000 7f3fd0769700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc8155de0 con 0x7f3fc8067150 2023-11-15T02:56:58.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.759+0000 7f3fd0769700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc80c4060 con 0x7f3fc80ac150 2023-11-15T02:56:58.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.759+0000 7f3fd0769700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc80c3aa0 con 0x7f3fc8151920 2023-11-15T02:56:58.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.759+0000 7f3fce505700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fc8151920 0x7f3fc80bb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.759+0000 7f3fcdd04700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3fc8067150 0x7f3fc80aba40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:57736/0 (socket says 172.21.15.121:57736) 2023-11-15T02:56:58.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.759+0000 7f3fcdd04700 1 -- 172.21.15.121:0/3800990954 learned_addr learned my addr 172.21.15.121:0/3800990954 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:58.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.759+0000 7f3fced06700 1 --2- 172.21.15.121:0/3800990954 >> v2:172.21.15.190:3300/0 conn(0x7f3fc80ac150 0x7f3fc8062190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.759+0000 7f3fcd503700 1 -- 172.21.15.121:0/3800990954 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 959044072 0 0) 0x7f3fc8155de0 con 0x7f3fc8067150 2023-11-15T02:56:58.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.759+0000 7f3fce505700 1 -- 172.21.15.121:0/3800990954 >> v2:172.21.15.190:3300/0 conn(0x7f3fc80ac150 msgr2=0x7f3fc8062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:58.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.759+0000 7f3fce505700 1 --2- 172.21.15.121:0/3800990954 >> v2:172.21.15.190:3300/0 conn(0x7f3fc80ac150 0x7f3fc8062190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.759+0000 7f3fce505700 1 -- 172.21.15.121:0/3800990954 >> v1:172.21.15.121:6789/0 conn(0x7f3fc8067150 legacy=0x7f3fc80aba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.759+0000 7f3fce505700 1 -- 172.21.15.121:0/3800990954 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc80c3d80 con 0x7f3fc8151920 2023-11-15T02:56:58.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.759+0000 7f3fced06700 1 --2- 172.21.15.121:0/3800990954 >> v2:172.21.15.190:3300/0 conn(0x7f3fc80ac150 0x7f3fc8062190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:56:58.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.759+0000 7f3fce505700 1 --2- 172.21.15.121:0/3800990954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fc8151920 0x7f3fc80bb6b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f3fc4009760 tx=0x7f3fc4009060).ready entity=mon.2 client_cookie=df7126447d4d92fc server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:58.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.759+0000 7f3fcd503700 1 -- 172.21.15.121:0/3800990954 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc400b580 con 0x7f3fc8151920 2023-11-15T02:56:58.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.759+0000 7f3fcd503700 1 -- 172.21.15.121:0/3800990954 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc400b6e0 con 0x7f3fc8151920 2023-11-15T02:56:58.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.759+0000 7f3fcd503700 1 -- 172.21.15.121:0/3800990954 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc400ba60 con 0x7f3fc8151920 2023-11-15T02:56:58.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.763+0000 7f3fd0769700 1 -- 172.21.15.121:0/3800990954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fc8151920 msgr2=0x7f3fc80bb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.763+0000 7f3fd0769700 1 --2- 172.21.15.121:0/3800990954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fc8151920 0x7f3fc80bb6b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f3fc4009760 tx=0x7f3fc4009060).stop 2023-11-15T02:56:58.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.763+0000 7f3fd0769700 1 -- 172.21.15.121:0/3800990954 shutdown_connections 2023-11-15T02:56:58.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.763+0000 7f3fd0769700 1 --2- 172.21.15.121:0/3800990954 >> v2:172.21.15.190:3300/0 conn(0x7f3fc80ac150 0x7f3fc8062190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.763+0000 7f3fd0769700 1 --2- 172.21.15.121:0/3800990954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fc8151920 0x7f3fc80bb6b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.763+0000 7f3fd0769700 1 -- 172.21.15.121:0/3800990954 >> 172.21.15.121:0/3800990954 conn(0x7f3fc8067a60 msgr2=0x7f3fc8067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:58.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.763+0000 7f3fd0769700 1 -- 172.21.15.121:0/3800990954 shutdown_connections 2023-11-15T02:56:58.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.763+0000 7f3fd0769700 1 -- 172.21.15.121:0/3800990954 wait complete. 2023-11-15T02:56:58.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.763+0000 7f3fd0769700 1 Processor -- start 2023-11-15T02:56:58.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.763+0000 7f3fd0769700 1 -- start start 2023-11-15T02:56:58.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fd0769700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fc8067150 0x7f3fc814c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fd0769700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fc80ac150 0x7f3fc8164a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fd0769700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fc8151920 0x7f3fc8165440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fd0769700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc81504a0 con 0x7f3fc80ac150 2023-11-15T02:56:58.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fd0769700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc814c800 con 0x7f3fc8067150 2023-11-15T02:56:58.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fd0769700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc814f890 con 0x7f3fc8151920 2023-11-15T02:56:58.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fcdd04700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fc80ac150 0x7f3fc8164a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fcdd04700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fc80ac150 0x7f3fc8164a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33112/0 (socket says 172.21.15.121:33112) 2023-11-15T02:56:58.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fcdd04700 1 -- 172.21.15.121:0/2557139262 learned_addr learned my addr 172.21.15.121:0/2557139262 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:58.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fce505700 1 --2- 172.21.15.121:0/2557139262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fc8067150 0x7f3fc814c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fced06700 1 --2- 172.21.15.121:0/2557139262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fc8151920 0x7f3fc8165440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fcdd04700 1 -- 172.21.15.121:0/2557139262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fc8151920 msgr2=0x7f3fc8165440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fcdd04700 1 --2- 172.21.15.121:0/2557139262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fc8151920 0x7f3fc8165440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fcdd04700 1 -- 172.21.15.121:0/2557139262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fc8067150 msgr2=0x7f3fc814c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fcdd04700 1 --2- 172.21.15.121:0/2557139262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fc8067150 0x7f3fc814c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fcdd04700 1 -- 172.21.15.121:0/2557139262 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc40092a0 con 0x7f3fc80ac150 2023-11-15T02:56:58.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fcdd04700 1 --2- 172.21.15.121:0/2557139262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fc80ac150 0x7f3fc8164a60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f3fb8001eb0 tx=0x7f3fb8007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:58.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fbe7fc700 1 -- 172.21.15.121:0/2557139262 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb80072e0 con 0x7f3fc80ac150 2023-11-15T02:56:58.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fbe7fc700 1 -- 172.21.15.121:0/2557139262 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb8007440 con 0x7f3fc80ac150 2023-11-15T02:56:58.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fd0769700 1 -- 172.21.15.121:0/2557139262 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc814fb70 con 0x7f3fc80ac150 2023-11-15T02:56:58.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fbe7fc700 1 -- 172.21.15.121:0/2557139262 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb80057a0 con 0x7f3fc80ac150 2023-11-15T02:56:58.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.767+0000 7f3fd0769700 1 -- 172.21.15.121:0/2557139262 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc8150130 con 0x7f3fc80ac150 2023-11-15T02:56:58.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.771+0000 7f3fd0769700 1 -- 172.21.15.121:0/2557139262 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3fc8047060 con 0x7f3fc80ac150 2023-11-15T02:56:58.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.771+0000 7f3fbe7fc700 1 -- 172.21.15.121:0/2557139262 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3fb8005b90 con 0x7f3fc80ac150 2023-11-15T02:56:58.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.771+0000 7f3fbe7fc700 1 --2- 172.21.15.121:0/2557139262 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3fac05a510 0x7f3fac05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.771+0000 7f3fbe7fc700 1 -- 172.21.15.121:0/2557139262 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3fb8073fe0 con 0x7f3fc80ac150 2023-11-15T02:56:58.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.771+0000 7f3fbe7fc700 1 --2- 172.21.15.121:0/2557139262 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7f3fac0638b0 0x7f3fac065d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.771+0000 7f3fbe7fc700 1 -- 172.21.15.121:0/2557139262 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3fac066390 con 0x7f3fac0638b0 2023-11-15T02:56:58.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.771+0000 7f3fbe7fc700 1 -- 172.21.15.121:0/2557139262 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fb8076540 con 0x7f3fc80ac150 2023-11-15T02:56:58.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.771+0000 7f3fce505700 1 --2- 172.21.15.121:0/2557139262 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3fac05a510 0x7f3fac05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.771+0000 7f3fced06700 1 --2- 172.21.15.121:0/2557139262 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7f3fac0638b0 0x7f3fac065d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:58.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.771+0000 7f3fce505700 1 --2- 172.21.15.121:0/2557139262 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3fac05a510 0x7f3fac05c9d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f3fc400b340 tx=0x7f3fc4017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:58.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.775+0000 7f3fced06700 1 --2- 172.21.15.121:0/2557139262 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7f3fac0638b0 0x7f3fac065d10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:58.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.775+0000 7f3fbe7fc700 1 -- 172.21.15.121:0/2557139262 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f3fac066390 con 0x7f3fac0638b0 2023-11-15T02:56:58.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.795+0000 7f3fd0769700 1 -- 172.21.15.121:0/2557139262 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3fc81505e0 con 0x7f3fac0638b0 2023-11-15T02:56:58.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.795+0000 7f3fbe7fc700 1 -- 172.21.15.121:0/2557139262 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3fc81505e0 con 0x7f3fac0638b0 2023-11-15T02:56:58.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.799+0000 7f3fd0769700 1 -- 172.21.15.121:0/2557139262 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7f3fac0638b0 msgr2=0x7f3fac065d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.799+0000 7f3fd0769700 1 --2- 172.21.15.121:0/2557139262 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7f3fac0638b0 0x7f3fac065d10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:58.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.799+0000 7f3fd0769700 1 -- 172.21.15.121:0/2557139262 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3fac05a510 msgr2=0x7f3fac05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.799+0000 7f3fd0769700 1 --2- 172.21.15.121:0/2557139262 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3fac05a510 0x7f3fac05c9d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f3fc400b340 tx=0x7f3fc4017000).stop 2023-11-15T02:56:58.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.799+0000 7f3fd0769700 1 -- 172.21.15.121:0/2557139262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fc80ac150 msgr2=0x7f3fc8164a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:58.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.799+0000 7f3fd0769700 1 --2- 172.21.15.121:0/2557139262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fc80ac150 0x7f3fc8164a60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f3fb8001eb0 tx=0x7f3fb8007c10).stop 2023-11-15T02:56:58.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.799+0000 7f3fced06700 1 -- 172.21.15.121:0/2557139262 reap_dead start 2023-11-15T02:56:58.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.799+0000 7f3fd0769700 1 -- 172.21.15.121:0/2557139262 shutdown_connections 2023-11-15T02:56:58.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.799+0000 7f3fd0769700 1 -- 172.21.15.121:0/2557139262 >> 172.21.15.121:0/2557139262 conn(0x7f3fc8067a60 msgr2=0x7f3fc8065fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:58.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.799+0000 7f3fd0769700 1 -- 172.21.15.121:0/2557139262 shutdown_connections 2023-11-15T02:56:58.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.799+0000 7f3fd0769700 1 -- 172.21.15.121:0/2557139262 wait complete. 2023-11-15T02:56:58.826 INFO:teuthology.orchestra.run.smithi121.stdout:60129542149 2023-11-15T02:56:58.827 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-15T02:56:58.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.987+0000 7efe7461b700 1 Processor -- start 2023-11-15T02:56:58.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.987+0000 7efe7461b700 1 -- start start 2023-11-15T02:56:58.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.991+0000 7efe7461b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efe6c153ed0 0x7efe6c0be410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.991+0000 7efe7461b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6c0c8f40 0x7efe6c0be900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:58.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.991+0000 7efe7461b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6c0c32a0 con 0x7efe6c153af0 2023-11-15T02:56:58.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.991+0000 7efe7461b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe6c0c3800 con 0x7efe6c153ed0 2023-11-15T02:56:58.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.991+0000 7efe7461b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c0bf730 con 0x7efe6c0c8f40 2023-11-15T02:56:58.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.991+0000 7efe71bb6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efe6c153ed0 0x7efe6c0be410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.991+0000 7efe723b7700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7efe6c153af0 0x7efe6c0c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:59326/0 (socket says 172.21.15.121:59326) 2023-11-15T02:56:59.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.991+0000 7efe723b7700 1 -- 172.21.15.121:0/2289392410 learned_addr learned my addr 172.21.15.121:0/2289392410 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:59.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.991+0000 7efe72bb8700 1 --2- 172.21.15.121:0/2289392410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6c0c8f40 0x7efe6c0be900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.991+0000 7efe71bb6700 1 -- 172.21.15.121:0/2289392410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6c0c8f40 msgr2=0x7efe6c0be900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:59.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.991+0000 7efe71bb6700 1 --2- 172.21.15.121:0/2289392410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6c0c8f40 0x7efe6c0be900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.991+0000 7efe71bb6700 1 -- 172.21.15.121:0/2289392410 >> v1:172.21.15.190:6789/0 conn(0x7efe6c153af0 legacy=0x7efe6c0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.991+0000 7efe713b5700 1 -- 172.21.15.121:0/2289392410 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694176125 0 0) 0x7efe6c0c32a0 con 0x7efe6c153af0 2023-11-15T02:56:59.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.991+0000 7efe71bb6700 1 -- 172.21.15.121:0/2289392410 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c0c34d0 con 0x7efe6c153ed0 2023-11-15T02:56:59.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.991+0000 7efe72bb8700 1 --2- 172.21.15.121:0/2289392410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6c0c8f40 0x7efe6c0be900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:56:59.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.995+0000 7efe71bb6700 1 --2- 172.21.15.121:0/2289392410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe6c153ed0 0x7efe6c0be410 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7efe5c009760 tx=0x7efe5c009060).ready entity=mon.0 client_cookie=44dfaac7dd0cc169 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:59.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.995+0000 7efe713b5700 1 -- 172.21.15.121:0/2289392410 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c00b580 con 0x7efe6c153ed0 2023-11-15T02:56:59.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.995+0000 7efe713b5700 1 -- 172.21.15.121:0/2289392410 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5c00b6e0 con 0x7efe6c153ed0 2023-11-15T02:56:59.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.995+0000 7efe713b5700 1 -- 172.21.15.121:0/2289392410 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c00ba60 con 0x7efe6c153ed0 2023-11-15T02:56:59.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.995+0000 7efe7461b700 1 -- 172.21.15.121:0/2289392410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe6c153ed0 msgr2=0x7efe6c0be410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.995+0000 7efe7461b700 1 --2- 172.21.15.121:0/2289392410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe6c153ed0 0x7efe6c0be410 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7efe5c009760 tx=0x7efe5c009060).stop 2023-11-15T02:56:59.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.995+0000 7efe7461b700 1 -- 172.21.15.121:0/2289392410 shutdown_connections 2023-11-15T02:56:59.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.995+0000 7efe7461b700 1 --2- 172.21.15.121:0/2289392410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6c0c8f40 0x7efe6c0be900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.995+0000 7efe7461b700 1 --2- 172.21.15.121:0/2289392410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe6c153ed0 0x7efe6c0be410 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.995+0000 7efe7461b700 1 -- 172.21.15.121:0/2289392410 >> 172.21.15.121:0/2289392410 conn(0x7efe6c067a60 msgr2=0x7efe6c067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:59.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.995+0000 7efe7461b700 1 -- 172.21.15.121:0/2289392410 shutdown_connections 2023-11-15T02:56:59.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.995+0000 7efe7461b700 1 -- 172.21.15.121:0/2289392410 wait complete. 2023-11-15T02:56:59.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.995+0000 7efe7461b700 1 Processor -- start 2023-11-15T02:56:59.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe7461b700 1 -- start start 2023-11-15T02:56:59.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe7461b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6c0c8f40 0x7efe6c1569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe7461b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe6c153af0 0x7efe6c0ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe7461b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe6c153ed0 0x7efe6c0c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe7461b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c0ade50 con 0x7efe6c153af0 2023-11-15T02:56:59.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe7461b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c0ae7d0 con 0x7efe6c153ed0 2023-11-15T02:56:59.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe7461b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c0acfa0 con 0x7efe6c0c8f40 2023-11-15T02:56:59.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe71bb6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe6c153af0 0x7efe6c0ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe71bb6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe6c153af0 0x7efe6c0ae2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33122/0 (socket says 172.21.15.121:33122) 2023-11-15T02:56:59.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe71bb6700 1 -- 172.21.15.121:0/3355558706 learned_addr learned my addr 172.21.15.121:0/3355558706 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:59.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe72bb8700 1 --2- 172.21.15.121:0/3355558706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe6c153ed0 0x7efe6c0c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe723b7700 1 --2- 172.21.15.121:0/3355558706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6c0c8f40 0x7efe6c1569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe71bb6700 1 -- 172.21.15.121:0/3355558706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6c0c8f40 msgr2=0x7efe6c1569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe71bb6700 1 --2- 172.21.15.121:0/3355558706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6c0c8f40 0x7efe6c1569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe71bb6700 1 -- 172.21.15.121:0/3355558706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe6c153ed0 msgr2=0x7efe6c0c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe71bb6700 1 --2- 172.21.15.121:0/3355558706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe6c153ed0 0x7efe6c0c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe71bb6700 1 -- 172.21.15.121:0/3355558706 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5c0092a0 con 0x7efe6c153af0 2023-11-15T02:56:59.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe71bb6700 1 --2- 172.21.15.121:0/3355558706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe6c153af0 0x7efe6c0ae2e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7efe6c067120 tx=0x7efe5c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:59.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe627fc700 1 -- 172.21.15.121:0/3355558706 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c000bc0 con 0x7efe6c153af0 2023-11-15T02:56:59.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe627fc700 1 -- 172.21.15.121:0/3355558706 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5c00b850 con 0x7efe6c153af0 2023-11-15T02:56:59.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe7461b700 1 -- 172.21.15.121:0/3355558706 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6c0ad240 con 0x7efe6c153af0 2023-11-15T02:56:59.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe627fc700 1 -- 172.21.15.121:0/3355558706 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c000d20 con 0x7efe6c153af0 2023-11-15T02:56:59.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:58.999+0000 7efe7461b700 1 -- 172.21.15.121:0/3355558706 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe6c0ad800 con 0x7efe6c153af0 2023-11-15T02:56:59.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.003+0000 7efe7461b700 1 -- 172.21.15.121:0/3355558706 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efe6c047060 con 0x7efe6c153af0 2023-11-15T02:56:59.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.003+0000 7efe627fc700 1 -- 172.21.15.121:0/3355558706 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7efe5c01b960 con 0x7efe6c153af0 2023-11-15T02:56:59.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.003+0000 7efe627fc700 1 --2- 172.21.15.121:0/3355558706 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efe5005a540 0x7efe5005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.003+0000 7efe627fc700 1 -- 172.21.15.121:0/3355558706 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efe5c075ec0 con 0x7efe6c153af0 2023-11-15T02:56:59.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.007+0000 7efe723b7700 1 --2- 172.21.15.121:0/3355558706 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efe5005a540 0x7efe5005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.007+0000 7efe627fc700 1 --2- 172.21.15.121:0/3355558706 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7efe500638e0 0x7efe50065d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.007+0000 7efe627fc700 1 -- 172.21.15.121:0/3355558706 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efe500663c0 con 0x7efe500638e0 2023-11-15T02:56:59.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.007+0000 7efe627fc700 1 -- 172.21.15.121:0/3355558706 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe5c078630 con 0x7efe6c153af0 2023-11-15T02:56:59.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.007+0000 7efe723b7700 1 --2- 172.21.15.121:0/3355558706 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efe5005a540 0x7efe5005ca00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7efe680011e0 tx=0x7efe68007bb0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:59.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.007+0000 7efe72bb8700 1 --2- 172.21.15.121:0/3355558706 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7efe500638e0 0x7efe50065d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.007+0000 7efe72bb8700 1 --2- 172.21.15.121:0/3355558706 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7efe500638e0 0x7efe50065d40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:59.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.007+0000 7efe627fc700 1 -- 172.21.15.121:0/3355558706 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efe500663c0 con 0x7efe500638e0 2023-11-15T02:56:59.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.027+0000 7efe7461b700 1 -- 172.21.15.121:0/3355558706 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efe6c0bf170 con 0x7efe500638e0 2023-11-15T02:56:59.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.031+0000 7efe627fc700 1 -- 172.21.15.121:0/3355558706 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efe6c0bf170 con 0x7efe500638e0 2023-11-15T02:56:59.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.031+0000 7efe7461b700 1 -- 172.21.15.121:0/3355558706 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7efe500638e0 msgr2=0x7efe50065d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.031+0000 7efe7461b700 1 --2- 172.21.15.121:0/3355558706 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7efe500638e0 0x7efe50065d40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.031+0000 7efe7461b700 1 -- 172.21.15.121:0/3355558706 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efe5005a540 msgr2=0x7efe5005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.031+0000 7efe7461b700 1 --2- 172.21.15.121:0/3355558706 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efe5005a540 0x7efe5005ca00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7efe680011e0 tx=0x7efe68007bb0).stop 2023-11-15T02:56:59.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.031+0000 7efe7461b700 1 -- 172.21.15.121:0/3355558706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe6c153af0 msgr2=0x7efe6c0ae2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.031+0000 7efe7461b700 1 --2- 172.21.15.121:0/3355558706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe6c153af0 0x7efe6c0ae2e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7efe6c067120 tx=0x7efe5c017000).stop 2023-11-15T02:56:59.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.031+0000 7efe72bb8700 1 -- 172.21.15.121:0/3355558706 reap_dead start 2023-11-15T02:56:59.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.031+0000 7efe7461b700 1 -- 172.21.15.121:0/3355558706 shutdown_connections 2023-11-15T02:56:59.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.031+0000 7efe7461b700 1 -- 172.21.15.121:0/3355558706 >> 172.21.15.121:0/3355558706 conn(0x7efe6c067a60 msgr2=0x7efe6c0c7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:59.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.031+0000 7efe7461b700 1 -- 172.21.15.121:0/3355558706 shutdown_connections 2023-11-15T02:56:59.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.031+0000 7efe7461b700 1 -- 172.21.15.121:0/3355558706 wait complete. 2023-11-15T02:56:59.058 INFO:teuthology.orchestra.run.smithi121.stdout:55834574853 2023-11-15T02:56:59.059 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-15T02:56:59.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.215+0000 7f1938cf1700 1 Processor -- start 2023-11-15T02:56:59.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.215+0000 7f1938cf1700 1 -- start start 2023-11-15T02:56:59.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.219+0000 7f1938cf1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f19340a8de0 0x7f1934066420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.219+0000 7f1938cf1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19340628a0 0x7f1934151410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.219+0000 7f1938cf1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1934065f90 con 0x7f1934150f90 2023-11-15T02:56:59.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.219+0000 7f1938cf1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1934151900 con 0x7f19340a8de0 2023-11-15T02:56:59.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.219+0000 7f1938cf1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1934147ea0 con 0x7f19340628a0 2023-11-15T02:56:59.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.219+0000 7f193259c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f19340a8de0 0x7f1934066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.219+0000 7f1931d9b700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f1934150f90 0x7f1934062190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:59338/0 (socket says 172.21.15.121:59338) 2023-11-15T02:56:59.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.219+0000 7f1931d9b700 1 -- 172.21.15.121:0/3723171048 learned_addr learned my addr 172.21.15.121:0/3723171048 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:59.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.219+0000 7f1932d9d700 1 --2- 172.21.15.121:0/3723171048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19340628a0 0x7f1934151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.219+0000 7f193159a700 1 -- 172.21.15.121:0/3723171048 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294163638 0 0) 0x7f1934065f90 con 0x7f1934150f90 2023-11-15T02:56:59.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.219+0000 7f193159a700 1 -- 172.21.15.121:0/3723171048 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f191c0033f0 con 0x7f1934150f90 2023-11-15T02:56:59.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.223+0000 7f1932d9d700 1 -- 172.21.15.121:0/3723171048 >> v2:172.21.15.121:3300/0 conn(0x7f19340a8de0 msgr2=0x7f1934066420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:59.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.223+0000 7f1932d9d700 1 --2- 172.21.15.121:0/3723171048 >> v2:172.21.15.121:3300/0 conn(0x7f19340a8de0 0x7f1934066420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.223+0000 7f1932d9d700 1 -- 172.21.15.121:0/3723171048 >> v1:172.21.15.190:6789/0 conn(0x7f1934150f90 legacy=0x7f1934062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.223+0000 7f1932d9d700 1 -- 172.21.15.121:0/3723171048 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1934151e30 con 0x7f19340628a0 2023-11-15T02:56:59.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.223+0000 7f1932d9d700 1 --2- 172.21.15.121:0/3723171048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19340628a0 0x7f1934151410 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f192c009e60 tx=0x7f192c009780).ready entity=mon.2 client_cookie=aaec99537adcc927 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:59.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.223+0000 7f193159a700 1 -- 172.21.15.121:0/3723171048 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c00e070 con 0x7f19340628a0 2023-11-15T02:56:59.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.223+0000 7f193159a700 1 -- 172.21.15.121:0/3723171048 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c00bab0 con 0x7f19340628a0 2023-11-15T02:56:59.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.223+0000 7f193159a700 1 -- 172.21.15.121:0/3723171048 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c00be30 con 0x7f19340628a0 2023-11-15T02:56:59.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.223+0000 7f1938cf1700 1 -- 172.21.15.121:0/3723171048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19340628a0 msgr2=0x7f1934151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.223+0000 7f1938cf1700 1 --2- 172.21.15.121:0/3723171048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19340628a0 0x7f1934151410 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f192c009e60 tx=0x7f192c009780).stop 2023-11-15T02:56:59.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.223+0000 7f1938cf1700 1 -- 172.21.15.121:0/3723171048 shutdown_connections 2023-11-15T02:56:59.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.223+0000 7f1938cf1700 1 --2- 172.21.15.121:0/3723171048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19340628a0 0x7f1934151410 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.223+0000 7f1938cf1700 1 --2- 172.21.15.121:0/3723171048 >> v2:172.21.15.121:3300/0 conn(0x7f19340a8de0 0x7f1934066420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.223+0000 7f1938cf1700 1 -- 172.21.15.121:0/3723171048 >> 172.21.15.121:0/3723171048 conn(0x7f1934067a60 msgr2=0x7f1934067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:59.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.223+0000 7f1938cf1700 1 -- 172.21.15.121:0/3723171048 shutdown_connections 2023-11-15T02:56:59.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.223+0000 7f1938cf1700 1 -- 172.21.15.121:0/3723171048 wait complete. 2023-11-15T02:56:59.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.227+0000 7f1938cf1700 1 Processor -- start 2023-11-15T02:56:59.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.227+0000 7f1938cf1700 1 -- start start 2023-11-15T02:56:59.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f1938cf1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19340628a0 0x7f1934146920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f1938cf1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19340a8de0 0x7f1934161ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f1938cf1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1934150f90 0x7f1934153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f1938cf1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193414a4b0 con 0x7f19340628a0 2023-11-15T02:56:59.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f1938cf1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19340bd4c0 con 0x7f1934150f90 2023-11-15T02:56:59.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f1938cf1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1934148f20 con 0x7f19340a8de0 2023-11-15T02:56:59.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f193259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19340628a0 0x7f1934146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f193259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19340628a0 0x7f1934146920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33146/0 (socket says 172.21.15.121:33146) 2023-11-15T02:56:59.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f193259c700 1 -- 172.21.15.121:0/572126042 learned_addr learned my addr 172.21.15.121:0/572126042 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:59.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f1932d9d700 1 --2- 172.21.15.121:0/572126042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1934150f90 0x7f1934153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f1931d9b700 1 --2- 172.21.15.121:0/572126042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19340a8de0 0x7f1934161ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f193259c700 1 -- 172.21.15.121:0/572126042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19340a8de0 msgr2=0x7f1934161ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f193259c700 1 --2- 172.21.15.121:0/572126042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19340a8de0 0x7f1934161ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f193259c700 1 -- 172.21.15.121:0/572126042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1934150f90 msgr2=0x7f1934153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f193259c700 1 --2- 172.21.15.121:0/572126042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1934150f90 0x7f1934153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f193259c700 1 -- 172.21.15.121:0/572126042 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192c009a20 con 0x7f19340628a0 2023-11-15T02:56:59.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f1931d9b700 1 --2- 172.21.15.121:0/572126042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19340a8de0 0x7f1934161ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:56:59.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f193259c700 1 --2- 172.21.15.121:0/572126042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19340628a0 0x7f1934146920 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f192400e930 tx=0x7f1924009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:59.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f19227fc700 1 -- 172.21.15.121:0/572126042 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192400eed0 con 0x7f19340628a0 2023-11-15T02:56:59.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f19227fc700 1 -- 172.21.15.121:0/572126042 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192400c850 con 0x7f19340628a0 2023-11-15T02:56:59.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f1938cf1700 1 -- 172.21.15.121:0/572126042 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1934149b80 con 0x7f19340628a0 2023-11-15T02:56:59.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f19227fc700 1 -- 172.21.15.121:0/572126042 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192400cbd0 con 0x7f19340628a0 2023-11-15T02:56:59.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.231+0000 7f1938cf1700 1 -- 172.21.15.121:0/572126042 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193414a140 con 0x7f19340628a0 2023-11-15T02:56:59.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.235+0000 7f1938cf1700 1 -- 172.21.15.121:0/572126042 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1934047060 con 0x7f19340628a0 2023-11-15T02:56:59.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.235+0000 7f19227fc700 1 -- 172.21.15.121:0/572126042 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f192401b400 con 0x7f19340628a0 2023-11-15T02:56:59.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.235+0000 7f19227fc700 1 --2- 172.21.15.121:0/572126042 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f191405a510 0x7f191405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.235+0000 7f19227fc700 1 -- 172.21.15.121:0/572126042 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1924074e30 con 0x7f19340628a0 2023-11-15T02:56:59.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.235+0000 7f1931d9b700 1 --2- 172.21.15.121:0/572126042 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f191405a510 0x7f191405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.239+0000 7f19227fc700 1 --2- 172.21.15.121:0/572126042 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7f1914063740 0x7f1914065ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.239+0000 7f19227fc700 1 -- 172.21.15.121:0/572126042 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1914066220 con 0x7f1914063740 2023-11-15T02:56:59.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.239+0000 7f19227fc700 1 -- 172.21.15.121:0/572126042 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1924077390 con 0x7f19340628a0 2023-11-15T02:56:59.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.239+0000 7f1932d9d700 1 --2- 172.21.15.121:0/572126042 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7f1914063740 0x7f1914065ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.239+0000 7f1931d9b700 1 --2- 172.21.15.121:0/572126042 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f191405a510 0x7f191405c9d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f1928005750 tx=0x7f1928009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:59.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.239+0000 7f1932d9d700 1 --2- 172.21.15.121:0/572126042 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7f1914063740 0x7f1914065ba0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:59.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.239+0000 7f19227fc700 1 -- 172.21.15.121:0/572126042 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1914066220 con 0x7f1914063740 2023-11-15T02:56:59.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.259+0000 7f1938cf1700 1 -- 172.21.15.121:0/572126042 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f193414a5f0 con 0x7f1914063740 2023-11-15T02:56:59.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.259+0000 7f19227fc700 1 -- 172.21.15.121:0/572126042 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f192c00bc20 con 0x7f1914063740 2023-11-15T02:56:59.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.259+0000 7f1938cf1700 1 -- 172.21.15.121:0/572126042 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7f1914063740 msgr2=0x7f1914065ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.259+0000 7f1938cf1700 1 --2- 172.21.15.121:0/572126042 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7f1914063740 0x7f1914065ba0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.259+0000 7f1938cf1700 1 -- 172.21.15.121:0/572126042 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f191405a510 msgr2=0x7f191405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.259+0000 7f1938cf1700 1 --2- 172.21.15.121:0/572126042 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f191405a510 0x7f191405c9d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f1928005750 tx=0x7f1928009040).stop 2023-11-15T02:56:59.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.259+0000 7f1938cf1700 1 -- 172.21.15.121:0/572126042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19340628a0 msgr2=0x7f1934146920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.259+0000 7f1938cf1700 1 --2- 172.21.15.121:0/572126042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19340628a0 0x7f1934146920 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f192400e930 tx=0x7f1924009060).stop 2023-11-15T02:56:59.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.259+0000 7f1932d9d700 1 -- 172.21.15.121:0/572126042 reap_dead start 2023-11-15T02:56:59.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.259+0000 7f1938cf1700 1 -- 172.21.15.121:0/572126042 shutdown_connections 2023-11-15T02:56:59.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.259+0000 7f1938cf1700 1 -- 172.21.15.121:0/572126042 >> 172.21.15.121:0/572126042 conn(0x7f1934067a60 msgr2=0x7f1934152050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:59.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.259+0000 7f1938cf1700 1 -- 172.21.15.121:0/572126042 shutdown_connections 2023-11-15T02:56:59.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.259+0000 7f1938cf1700 1 -- 172.21.15.121:0/572126042 wait complete. 2023-11-15T02:56:59.286 INFO:teuthology.orchestra.run.smithi121.stdout:60129542148 2023-11-15T02:56:59.289 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-15T02:56:59.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.443+0000 7fadbd25b700 1 Processor -- start 2023-11-15T02:56:59.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.443+0000 7fadbd25b700 1 -- start start 2023-11-15T02:56:59.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadbd25b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fadb80628a0 0x7fadb8158210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadbd25b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fadb80ac420 0x7fadb80660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadbd25b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb80aaf80 con 0x7fadb8157e30 2023-11-15T02:56:59.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadbd25b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb80aa970 con 0x7fadb80ac420 2023-11-15T02:56:59.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadbd25b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb8066e10 con 0x7fadb80628a0 2023-11-15T02:56:59.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadb6d9d700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fadb8157e30 0x7fadb8062190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:59342/0 (socket says 172.21.15.121:59342) 2023-11-15T02:56:59.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadb6d9d700 1 -- 172.21.15.121:0/1909835247 learned_addr learned my addr 172.21.15.121:0/1909835247 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:59.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadb759e700 1 --2- 172.21.15.121:0/1909835247 >> v2:172.21.15.121:3300/0 conn(0x7fadb80ac420 0x7fadb80660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadae59c700 1 --2- 172.21.15.121:0/1909835247 >> v2:172.21.15.190:3300/0 conn(0x7fadb80628a0 0x7fadb8158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadb759e700 1 -- 172.21.15.121:0/1909835247 >> v2:172.21.15.190:3300/0 conn(0x7fadb80628a0 msgr2=0x7fadb8158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:59.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadb759e700 1 --2- 172.21.15.121:0/1909835247 >> v2:172.21.15.190:3300/0 conn(0x7fadb80628a0 0x7fadb8158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadb759e700 1 -- 172.21.15.121:0/1909835247 >> v1:172.21.15.190:6789/0 conn(0x7fadb8157e30 legacy=0x7fadb8062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadb759e700 1 -- 172.21.15.121:0/1909835247 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb80670f0 con 0x7fadb80ac420 2023-11-15T02:56:59.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadb759e700 1 --2- 172.21.15.121:0/1909835247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb80ac420 0x7fadb80660e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fadb80c7730 tx=0x7fadb0009780).ready entity=mon.0 client_cookie=c8f74b0922e4aa3b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:59.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadb659c700 1 -- 172.21.15.121:0/1909835247 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb000e070 con 0x7fadb80ac420 2023-11-15T02:56:59.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadb659c700 1 -- 172.21.15.121:0/1909835247 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb000bb60 con 0x7fadb80ac420 2023-11-15T02:56:59.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadb659c700 1 -- 172.21.15.121:0/1909835247 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0012410 con 0x7fadb80ac420 2023-11-15T02:56:59.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadbd25b700 1 -- 172.21.15.121:0/1909835247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb80ac420 msgr2=0x7fadb80660e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadbd25b700 1 --2- 172.21.15.121:0/1909835247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb80ac420 0x7fadb80660e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fadb80c7730 tx=0x7fadb0009780).stop 2023-11-15T02:56:59.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadbd25b700 1 -- 172.21.15.121:0/1909835247 shutdown_connections 2023-11-15T02:56:59.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadbd25b700 1 --2- 172.21.15.121:0/1909835247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb80ac420 0x7fadb80660e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadbd25b700 1 --2- 172.21.15.121:0/1909835247 >> v2:172.21.15.190:3300/0 conn(0x7fadb80628a0 0x7fadb8158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadbd25b700 1 -- 172.21.15.121:0/1909835247 >> 172.21.15.121:0/1909835247 conn(0x7fadb8067a60 msgr2=0x7fadb8067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:59.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadbd25b700 1 -- 172.21.15.121:0/1909835247 shutdown_connections 2023-11-15T02:56:59.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.447+0000 7fadbd25b700 1 -- 172.21.15.121:0/1909835247 wait complete. 2023-11-15T02:56:59.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.451+0000 7fadbd25b700 1 Processor -- start 2023-11-15T02:56:59.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.451+0000 7fadbd25b700 1 -- start start 2023-11-15T02:56:59.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.451+0000 7fadbd25b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadb80628a0 0x7fadb814e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.451+0000 7fadbd25b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadb80ac420 0x7fadb8155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.451+0000 7fadbd25b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb8157e30 0x7fadb8155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.451+0000 7fadbd25b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb8159320 con 0x7fadb8157e30 2023-11-15T02:56:59.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.451+0000 7fadbd25b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb815a180 con 0x7fadb80628a0 2023-11-15T02:56:59.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.451+0000 7fadbd25b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb8159570 con 0x7fadb80ac420 2023-11-15T02:56:59.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.455+0000 7fadb759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb8157e30 0x7fadb8155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.455+0000 7fadb759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb8157e30 0x7fadb8155860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33158/0 (socket says 172.21.15.121:33158) 2023-11-15T02:56:59.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.455+0000 7fadb759e700 1 -- 172.21.15.121:0/3516837814 learned_addr learned my addr 172.21.15.121:0/3516837814 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:59.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.455+0000 7fadb6d9d700 1 --2- 172.21.15.121:0/3516837814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadb80628a0 0x7fadb814e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.455+0000 7fadae59c700 1 --2- 172.21.15.121:0/3516837814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadb80ac420 0x7fadb8155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.455+0000 7fadb759e700 1 -- 172.21.15.121:0/3516837814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadb80ac420 msgr2=0x7fadb8155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.455+0000 7fadb759e700 1 --2- 172.21.15.121:0/3516837814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadb80ac420 0x7fadb8155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.455+0000 7fadb759e700 1 -- 172.21.15.121:0/3516837814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadb80628a0 msgr2=0x7fadb814e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.455+0000 7fadb759e700 1 --2- 172.21.15.121:0/3516837814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadb80628a0 0x7fadb814e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.455+0000 7fadb759e700 1 -- 172.21.15.121:0/3516837814 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb0009a20 con 0x7fadb8157e30 2023-11-15T02:56:59.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.455+0000 7fadae59c700 1 --2- 172.21.15.121:0/3516837814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadb80ac420 0x7fadb8155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:56:59.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.455+0000 7fadb6d9d700 1 --2- 172.21.15.121:0/3516837814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadb80628a0 0x7fadb814e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:56:59.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.455+0000 7fadb759e700 1 --2- 172.21.15.121:0/3516837814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb8157e30 0x7fadb8155860 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fadb0011000 tx=0x7fadb8158730).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:59.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.455+0000 7fadaf7fe700 1 -- 172.21.15.121:0/3516837814 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb001d630 con 0x7fadb8157e30 2023-11-15T02:56:59.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.455+0000 7fadaf7fe700 1 -- 172.21.15.121:0/3516837814 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb001d790 con 0x7fadb8157e30 2023-11-15T02:56:59.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.455+0000 7fadbd25b700 1 -- 172.21.15.121:0/3516837814 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb8159850 con 0x7fadb8157e30 2023-11-15T02:56:59.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.455+0000 7fadaf7fe700 1 -- 172.21.15.121:0/3516837814 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb001db10 con 0x7fadb8157e30 2023-11-15T02:56:59.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.455+0000 7fadbd25b700 1 -- 172.21.15.121:0/3516837814 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb8159e10 con 0x7fadb8157e30 2023-11-15T02:56:59.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.455+0000 7fadbd25b700 1 -- 172.21.15.121:0/3516837814 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fadb8047060 con 0x7fadb8157e30 2023-11-15T02:56:59.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.459+0000 7fadaf7fe700 1 -- 172.21.15.121:0/3516837814 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fadb0027460 con 0x7fadb8157e30 2023-11-15T02:56:59.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.459+0000 7fadaf7fe700 1 --2- 172.21.15.121:0/3516837814 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fad9805a510 0x7fad9805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.459+0000 7fadaf7fe700 1 -- 172.21.15.121:0/3516837814 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fadb0076700 con 0x7fadb8157e30 2023-11-15T02:56:59.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.459+0000 7fadb6d9d700 1 --2- 172.21.15.121:0/3516837814 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fad9805a510 0x7fad9805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.459+0000 7fadaf7fe700 1 --2- 172.21.15.121:0/3516837814 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x7fad980638b0 0x7fad98065d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.459+0000 7fadaf7fe700 1 -- 172.21.15.121:0/3516837814 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad98066390 con 0x7fad980638b0 2023-11-15T02:56:59.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.459+0000 7fadaf7fe700 1 -- 172.21.15.121:0/3516837814 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadb0078c60 con 0x7fadb8157e30 2023-11-15T02:56:59.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.459+0000 7fadae59c700 1 --2- 172.21.15.121:0/3516837814 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x7fad980638b0 0x7fad98065d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.459+0000 7fadb6d9d700 1 --2- 172.21.15.121:0/3516837814 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fad9805a510 0x7fad9805c9d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fada80016a0 tx=0x7fada8009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:59.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.463+0000 7fadae59c700 1 --2- 172.21.15.121:0/3516837814 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x7fad980638b0 0x7fad98065d10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:59.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.463+0000 7fadaf7fe700 1 -- 172.21.15.121:0/3516837814 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fad98066390 con 0x7fad980638b0 2023-11-15T02:56:59.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.483+0000 7fadbd25b700 1 -- 172.21.15.121:0/3516837814 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fadb815a2f0 con 0x7fad980638b0 2023-11-15T02:56:59.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.487+0000 7fadaf7fe700 1 -- 172.21.15.121:0/3516837814 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fadb815a2f0 con 0x7fad980638b0 2023-11-15T02:56:59.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.487+0000 7fadbd25b700 1 -- 172.21.15.121:0/3516837814 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x7fad980638b0 msgr2=0x7fad98065d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.487+0000 7fadbd25b700 1 --2- 172.21.15.121:0/3516837814 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x7fad980638b0 0x7fad98065d10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.487+0000 7fadbd25b700 1 -- 172.21.15.121:0/3516837814 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fad9805a510 msgr2=0x7fad9805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.487+0000 7fadbd25b700 1 --2- 172.21.15.121:0/3516837814 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fad9805a510 0x7fad9805c9d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fada80016a0 tx=0x7fada8009040).stop 2023-11-15T02:56:59.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.487+0000 7fadbd25b700 1 -- 172.21.15.121:0/3516837814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb8157e30 msgr2=0x7fadb8155860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.487+0000 7fadbd25b700 1 --2- 172.21.15.121:0/3516837814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb8157e30 0x7fadb8155860 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fadb0011000 tx=0x7fadb8158730).stop 2023-11-15T02:56:59.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.487+0000 7fadb759e700 1 -- 172.21.15.121:0/3516837814 reap_dead start 2023-11-15T02:56:59.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.487+0000 7fadbd25b700 1 -- 172.21.15.121:0/3516837814 shutdown_connections 2023-11-15T02:56:59.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.487+0000 7fadbd25b700 1 -- 172.21.15.121:0/3516837814 >> 172.21.15.121:0/3516837814 conn(0x7fadb8067a60 msgr2=0x7fadb8152890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:59.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.487+0000 7fadbd25b700 1 -- 172.21.15.121:0/3516837814 shutdown_connections 2023-11-15T02:56:59.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.487+0000 7fadbd25b700 1 -- 172.21.15.121:0/3516837814 wait complete. 2023-11-15T02:56:59.511 INFO:teuthology.orchestra.run.smithi121.stdout:60129542148 2023-11-15T02:56:59.512 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-15T02:56:59.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.671+0000 7f6289adb700 1 Processor -- start 2023-11-15T02:56:59.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.671+0000 7f6289adb700 1 -- start start 2023-11-15T02:56:59.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.675+0000 7f6289adb700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f62840aa130 0x7f6284062710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.675+0000 7f6289adb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62840ac150 0x7f628414ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.675+0000 7f6289adb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62840aac50 con 0x7f6284062330 2023-11-15T02:56:59.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.675+0000 7f6289adb700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62840aa6b0 con 0x7f62840aa130 2023-11-15T02:56:59.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.675+0000 7f6289adb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62840aa970 con 0x7f62840ac150 2023-11-15T02:56:59.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.675+0000 7f62837fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6284062330 0x7f6284154c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:57740/0 (socket says 172.21.15.121:57740) 2023-11-15T02:56:59.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.675+0000 7f62837fe700 1 -- 172.21.15.121:0/3926649874 learned_addr learned my addr 172.21.15.121:0/3926649874 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:56:59.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.675+0000 7f6282ffd700 1 --2- 172.21.15.121:0/3926649874 >> v2:172.21.15.121:3300/0 conn(0x7f62840aa130 0x7f6284062710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.675+0000 7f6283fff700 1 --2- 172.21.15.121:0/3926649874 >> v2:172.21.15.190:3300/0 conn(0x7f62840ac150 0x7f628414ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.675+0000 7f62827fc700 1 -- 172.21.15.121:0/3926649874 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144030729 0 0) 0x7f62840aac50 con 0x7f6284062330 2023-11-15T02:56:59.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.675+0000 7f62827fc700 1 -- 172.21.15.121:0/3926649874 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f626c0033f0 con 0x7f6284062330 2023-11-15T02:56:59.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.675+0000 7f6283fff700 1 -- 172.21.15.121:0/3926649874 >> v2:172.21.15.121:3300/0 conn(0x7f62840aa130 msgr2=0x7f6284062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:56:59.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.675+0000 7f6283fff700 1 --2- 172.21.15.121:0/3926649874 >> v2:172.21.15.121:3300/0 conn(0x7f62840aa130 0x7f6284062710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.675+0000 7f6283fff700 1 -- 172.21.15.121:0/3926649874 >> v1:172.21.15.121:6789/0 conn(0x7f6284062330 legacy=0x7f6284154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.675+0000 7f6283fff700 1 -- 172.21.15.121:0/3926649874 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6284154410 con 0x7f62840ac150 2023-11-15T02:56:59.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.675+0000 7f6283fff700 1 --2- 172.21.15.121:0/3926649874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62840ac150 0x7f628414ddd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f6284069990 tx=0x7f627c009780).ready entity=mon.1 client_cookie=e1a84b40592b3a1a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:59.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.675+0000 7f62827fc700 1 -- 172.21.15.121:0/3926649874 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c00e070 con 0x7f62840ac150 2023-11-15T02:56:59.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.675+0000 7f62827fc700 1 -- 172.21.15.121:0/3926649874 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627c00bab0 con 0x7f62840ac150 2023-11-15T02:56:59.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.675+0000 7f62827fc700 1 -- 172.21.15.121:0/3926649874 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c00be30 con 0x7f62840ac150 2023-11-15T02:56:59.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.679+0000 7f6289adb700 1 -- 172.21.15.121:0/3926649874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62840ac150 msgr2=0x7f628414ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.679+0000 7f6289adb700 1 --2- 172.21.15.121:0/3926649874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62840ac150 0x7f628414ddd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f6284069990 tx=0x7f627c009780).stop 2023-11-15T02:56:59.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.679+0000 7f6289adb700 1 -- 172.21.15.121:0/3926649874 shutdown_connections 2023-11-15T02:56:59.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.679+0000 7f6289adb700 1 --2- 172.21.15.121:0/3926649874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62840ac150 0x7f628414ddd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.679+0000 7f6289adb700 1 --2- 172.21.15.121:0/3926649874 >> v2:172.21.15.121:3300/0 conn(0x7f62840aa130 0x7f6284062710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.679+0000 7f6289adb700 1 -- 172.21.15.121:0/3926649874 >> 172.21.15.121:0/3926649874 conn(0x7f6284067a60 msgr2=0x7f6284067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:59.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.679+0000 7f6289adb700 1 -- 172.21.15.121:0/3926649874 shutdown_connections 2023-11-15T02:56:59.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.679+0000 7f6289adb700 1 -- 172.21.15.121:0/3926649874 wait complete. 2023-11-15T02:56:59.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.683+0000 7f6289adb700 1 Processor -- start 2023-11-15T02:56:59.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.683+0000 7f6289adb700 1 -- start start 2023-11-15T02:56:59.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.683+0000 7f6289adb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6284062330 0x7f6284163da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.683+0000 7f6289adb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62840aa130 0x7f628414cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.683+0000 7f6289adb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62840ac150 0x7f628414d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.683+0000 7f6289adb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628414d510 con 0x7f62840aa130 2023-11-15T02:56:59.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.683+0000 7f6289adb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6284132d50 con 0x7f6284062330 2023-11-15T02:56:59.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.683+0000 7f6289adb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6284164740 con 0x7f62840ac150 2023-11-15T02:56:59.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.683+0000 7f6282ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62840aa130 0x7f628414cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.683+0000 7f6282ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62840aa130 0x7f628414cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33186/0 (socket says 172.21.15.121:33186) 2023-11-15T02:56:59.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.683+0000 7f6282ffd700 1 -- 172.21.15.121:0/1984362209 learned_addr learned my addr 172.21.15.121:0/1984362209 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:56:59.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.683+0000 7f62837fe700 1 --2- 172.21.15.121:0/1984362209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6284062330 0x7f6284163da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.683+0000 7f6283fff700 1 --2- 172.21.15.121:0/1984362209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62840ac150 0x7f628414d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.683+0000 7f6282ffd700 1 -- 172.21.15.121:0/1984362209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62840ac150 msgr2=0x7f628414d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.683+0000 7f6282ffd700 1 --2- 172.21.15.121:0/1984362209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62840ac150 0x7f628414d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.683+0000 7f6282ffd700 1 -- 172.21.15.121:0/1984362209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6284062330 msgr2=0x7f6284163da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.683+0000 7f6282ffd700 1 --2- 172.21.15.121:0/1984362209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6284062330 0x7f6284163da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.683+0000 7f6282ffd700 1 -- 172.21.15.121:0/1984362209 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c009a20 con 0x7f62840aa130 2023-11-15T02:56:59.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.683+0000 7f6282ffd700 1 --2- 172.21.15.121:0/1984362209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62840aa130 0x7f628414cb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f627800e930 tx=0x7f6278009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:59.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.687+0000 7f626b7fe700 1 -- 172.21.15.121:0/1984362209 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278012600 con 0x7f62840aa130 2023-11-15T02:56:59.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.687+0000 7f626b7fe700 1 -- 172.21.15.121:0/1984362209 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6278012760 con 0x7f62840aa130 2023-11-15T02:56:59.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.687+0000 7f6289adb700 1 -- 172.21.15.121:0/1984362209 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6284164a20 con 0x7f62840aa130 2023-11-15T02:56:59.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.687+0000 7f626b7fe700 1 -- 172.21.15.121:0/1984362209 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278012ae0 con 0x7f62840aa130 2023-11-15T02:56:59.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.687+0000 7f6289adb700 1 -- 172.21.15.121:0/1984362209 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62841650e0 con 0x7f62840aa130 2023-11-15T02:56:59.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.687+0000 7f6289adb700 1 -- 172.21.15.121:0/1984362209 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6284062a70 con 0x7f62840aa130 2023-11-15T02:56:59.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.687+0000 7f626b7fe700 1 -- 172.21.15.121:0/1984362209 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f627801b400 con 0x7f62840aa130 2023-11-15T02:56:59.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.691+0000 7f626b7fe700 1 --2- 172.21.15.121:0/1984362209 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f627005a510 0x7f627005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:56:59.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.691+0000 7f626b7fe700 1 -- 172.21.15.121:0/1984362209 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6278074c70 con 0x7f62840aa130 2023-11-15T02:56:59.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.691+0000 7f62837fe700 1 --2- 172.21.15.121:0/1984362209 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f627005a510 0x7f627005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:56:59.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.691+0000 7f62837fe700 1 --2- 172.21.15.121:0/1984362209 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f627005a510 0x7f627005c9d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f6274007fd0 tx=0x7f6274009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:56:59.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.695+0000 7f626b7fe700 1 -- 172.21.15.121:0/1984362209 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6278048330 con 0x7f62840aa130 2023-11-15T02:56:59.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.871+0000 7f6289adb700 1 -- 172.21.15.121:0/1984362209 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f62841546f0 con 0x7f62840aa130 2023-11-15T02:56:59.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.875+0000 7f626b7fe700 1 -- 172.21.15.121:0/1984362209 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f627804b950 con 0x7f62840aa130 2023-11-15T02:56:59.879 INFO:teuthology.orchestra.run.smithi121.stdout:60129542149 2023-11-15T02:56:59.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.879+0000 7f6289adb700 1 -- 172.21.15.121:0/1984362209 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f627005a510 msgr2=0x7f627005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.879+0000 7f6289adb700 1 --2- 172.21.15.121:0/1984362209 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f627005a510 0x7f627005c9d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f6274007fd0 tx=0x7f6274009040).stop 2023-11-15T02:56:59.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.879+0000 7f6289adb700 1 -- 172.21.15.121:0/1984362209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62840aa130 msgr2=0x7f628414cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:56:59.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.879+0000 7f6289adb700 1 --2- 172.21.15.121:0/1984362209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62840aa130 0x7f628414cb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f627800e930 tx=0x7f6278009060).stop 2023-11-15T02:56:59.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.879+0000 7f6289adb700 1 -- 172.21.15.121:0/1984362209 shutdown_connections 2023-11-15T02:56:59.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.879+0000 7f6289adb700 1 --2- 172.21.15.121:0/1984362209 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f627005a510 0x7f627005c9d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.879+0000 7f6289adb700 1 --2- 172.21.15.121:0/1984362209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62840ac150 0x7f628414d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.879+0000 7f6289adb700 1 --2- 172.21.15.121:0/1984362209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62840aa130 0x7f628414cb30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.879+0000 7f6289adb700 1 --2- 172.21.15.121:0/1984362209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6284062330 0x7f6284163da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:56:59.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.879+0000 7f6289adb700 1 -- 172.21.15.121:0/1984362209 >> 172.21.15.121:0/1984362209 conn(0x7f6284067a60 msgr2=0x7f628414a980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:56:59.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.879+0000 7f6289adb700 1 -- 172.21.15.121:0/1984362209 shutdown_connections 2023-11-15T02:56:59.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:56:59.879+0000 7f6289adb700 1 -- 172.21.15.121:0/1984362209 wait complete. 2023-11-15T02:56:59.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.0 2023-11-15T02:56:59.905 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-15T02:57:00.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.067+0000 7f4c2e8f1700 1 Processor -- start 2023-11-15T02:57:00.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.067+0000 7f4c2e8f1700 1 -- start start 2023-11-15T02:57:00.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c2e8f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c280c4990 0x7f4c2815b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:00.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c2e8f1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c28150e80 con 0x7f4c280be410 2023-11-15T02:57:00.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c2e8f1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c28062a80 con 0x7f4c280c3b30 2023-11-15T02:57:00.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c2e8f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c28062d60 con 0x7f4c280c4990 2023-11-15T02:57:00.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c1ffff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4c280be410 0x7f4c280c4280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:57756/0 (socket says 172.21.15.121:57756) 2023-11-15T02:57:00.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c1ffff700 1 -- 172.21.15.121:0/4097022023 learned_addr learned my addr 172.21.15.121:0/4097022023 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:00.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c2ce8e700 1 --2- 172.21.15.121:0/4097022023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c280c4990 0x7f4c2815b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:00.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c277fe700 1 -- 172.21.15.121:0/4097022023 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573541801 0 0) 0x7f4c10001040 con 0x7f4c280be410 2023-11-15T02:57:00.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c277fe700 1 -- 172.21.15.121:0/4097022023 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c140033f0 con 0x7f4c280be410 2023-11-15T02:57:00.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c277fe700 1 -- 172.21.15.121:0/4097022023 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016674118 0 0) 0x7f4c28062a80 con 0x7f4c280c3b30 2023-11-15T02:57:00.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c277fe700 1 -- 172.21.15.121:0/4097022023 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c10001040 con 0x7f4c280c3b30 2023-11-15T02:57:00.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c277fe700 1 -- 172.21.15.121:0/4097022023 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1883136963 0 0) 0x7f4c140033f0 con 0x7f4c280be410 2023-11-15T02:57:00.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c277fe700 1 -- 172.21.15.121:0/4097022023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c280c4990 msgr2=0x7f4c2815b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:00.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c277fe700 1 --2- 172.21.15.121:0/4097022023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c280c4990 0x7f4c2815b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:00.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c277fe700 1 -- 172.21.15.121:0/4097022023 >> v1:172.21.15.190:6789/0 conn(0x7f4c280c3b30 legacy=0x7f4c280bdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:00.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c277fe700 1 -- 172.21.15.121:0/4097022023 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c28063040 con 0x7f4c280be410 2023-11-15T02:57:00.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c277fe700 1 -- 172.21.15.121:0/4097022023 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f4c10001530 con 0x7f4c280be410 2023-11-15T02:57:00.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c277fe700 1 -- 172.21.15.121:0/4097022023 >> v1:172.21.15.121:6789/0 conn(0x7f4c280be410 legacy=0x7f4c280c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:00.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c277fe700 1 --2- 172.21.15.121:0/4097022023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c280c4990 0x7f4c14004b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:00.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c277fe700 1 --2- 172.21.15.121:0/4097022023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c14004ff0 0x7f4c14003630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:00.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c277fe700 1 --2- 172.21.15.121:0/4097022023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c14003b20 0x7f4c14003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:00.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c277fe700 1 -- 172.21.15.121:0/4097022023 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c28062d60 con 0x7f4c14004ff0 2023-11-15T02:57:00.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c277fe700 1 -- 172.21.15.121:0/4097022023 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1400fef0 con 0x7f4c14003b20 2023-11-15T02:57:00.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c277fe700 1 -- 172.21.15.121:0/4097022023 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c14010060 con 0x7f4c280c4990 2023-11-15T02:57:00.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c1ffff700 1 --2- 172.21.15.121:0/4097022023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c14004ff0 0x7f4c14003630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:00.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c2ce8e700 1 --2- 172.21.15.121:0/4097022023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c14003b20 0x7f4c14003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:00.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.071+0000 7f4c27fff700 1 --2- 172.21.15.121:0/4097022023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c280c4990 0x7f4c14004b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:00.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.075+0000 7f4c1ffff700 1 -- 172.21.15.121:0/4097022023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c280c4990 msgr2=0x7f4c14004b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:00.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.075+0000 7f4c1ffff700 1 --2- 172.21.15.121:0/4097022023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c280c4990 0x7f4c14004b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:00.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.075+0000 7f4c1ffff700 1 -- 172.21.15.121:0/4097022023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c14003b20 msgr2=0x7f4c14003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:00.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.075+0000 7f4c1ffff700 1 --2- 172.21.15.121:0/4097022023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c14003b20 0x7f4c14003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:00.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.075+0000 7f4c1ffff700 1 -- 172.21.15.121:0/4097022023 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c140033f0 con 0x7f4c14004ff0 2023-11-15T02:57:00.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.075+0000 7f4c27fff700 1 --2- 172.21.15.121:0/4097022023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c280c4990 0x7f4c14004b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:57:00.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.075+0000 7f4c2ce8e700 1 --2- 172.21.15.121:0/4097022023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c14003b20 0x7f4c14003fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:57:00.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.075+0000 7f4c1ffff700 1 --2- 172.21.15.121:0/4097022023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c14004ff0 0x7f4c14003630 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f4c280be810 tx=0x7f4c100077c0).ready entity=mon.0 client_cookie=85c80e8f8b49f8c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:00.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.075+0000 7f4c277fe700 1 -- 172.21.15.121:0/4097022023 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1000cab0 con 0x7f4c14004ff0 2023-11-15T02:57:00.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.075+0000 7f4c277fe700 1 -- 172.21.15.121:0/4097022023 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1000cc10 con 0x7f4c14004ff0 2023-11-15T02:57:00.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.075+0000 7f4c277fe700 1 -- 172.21.15.121:0/4097022023 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c100070d0 con 0x7f4c14004ff0 2023-11-15T02:57:00.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.075+0000 7f4c2e8f1700 1 -- 172.21.15.121:0/4097022023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c14004ff0 msgr2=0x7f4c14003630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:00.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.075+0000 7f4c2e8f1700 1 --2- 172.21.15.121:0/4097022023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c14004ff0 0x7f4c14003630 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f4c280be810 tx=0x7f4c100077c0).stop 2023-11-15T02:57:00.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.075+0000 7f4c2ce8e700 1 -- 172.21.15.121:0/4097022023 reap_dead start 2023-11-15T02:57:00.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.075+0000 7f4c2e8f1700 1 -- 172.21.15.121:0/4097022023 shutdown_connections 2023-11-15T02:57:00.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.075+0000 7f4c2e8f1700 1 -- 172.21.15.121:0/4097022023 >> 172.21.15.121:0/4097022023 conn(0x7f4c28068b70 msgr2=0x7f4c28068f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:00.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.075+0000 7f4c2e8f1700 1 -- 172.21.15.121:0/4097022023 shutdown_connections 2023-11-15T02:57:00.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.075+0000 7f4c2e8f1700 1 -- 172.21.15.121:0/4097022023 wait complete. 2023-11-15T02:57:00.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c2e8f1700 1 Processor -- start 2023-11-15T02:57:00.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c2e8f1700 1 -- start start 2023-11-15T02:57:00.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c2e8f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c14004ff0 0x7f4c28163ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:00.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c2e8f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c280a9f10 0x7f4c28154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:00.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c2e8f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c28154a80 0x7f4c280ac200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:00.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c2e8f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c280c63c0 con 0x7f4c14004ff0 2023-11-15T02:57:00.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c2e8f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c280c57b0 con 0x7f4c280a9f10 2023-11-15T02:57:00.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c2e8f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c280c5a90 con 0x7f4c28154a80 2023-11-15T02:57:00.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c27fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c14004ff0 0x7f4c28163ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:00.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c2ce8e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c28154a80 0x7f4c280ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:00.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c27fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c14004ff0 0x7f4c28163ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33202/0 (socket says 172.21.15.121:33202) 2023-11-15T02:57:00.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c27fff700 1 -- 172.21.15.121:0/2321765195 learned_addr learned my addr 172.21.15.121:0/2321765195 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:00.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c1ffff700 1 --2- 172.21.15.121:0/2321765195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c280a9f10 0x7f4c28154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:00.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c27fff700 1 -- 172.21.15.121:0/2321765195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c28154a80 msgr2=0x7f4c280ac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:00.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c27fff700 1 --2- 172.21.15.121:0/2321765195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c28154a80 0x7f4c280ac200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:00.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c27fff700 1 -- 172.21.15.121:0/2321765195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c280a9f10 msgr2=0x7f4c28154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:00.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c27fff700 1 --2- 172.21.15.121:0/2321765195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c280a9f10 0x7f4c28154590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:00.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c27fff700 1 -- 172.21.15.121:0/2321765195 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c280c5d70 con 0x7f4c14004ff0 2023-11-15T02:57:00.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c27fff700 1 --2- 172.21.15.121:0/2321765195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c14004ff0 0x7f4c28163ed0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f4c1800f8b0 tx=0x7f4c18007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:00.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c24ff9700 1 -- 172.21.15.121:0/2321765195 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c18011590 con 0x7f4c14004ff0 2023-11-15T02:57:00.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c24ff9700 1 -- 172.21.15.121:0/2321765195 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c180116f0 con 0x7f4c14004ff0 2023-11-15T02:57:00.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c2e8f1700 1 -- 172.21.15.121:0/2321765195 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c280c6050 con 0x7f4c14004ff0 2023-11-15T02:57:00.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.079+0000 7f4c24ff9700 1 -- 172.21.15.121:0/2321765195 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c18011a70 con 0x7f4c14004ff0 2023-11-15T02:57:00.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.083+0000 7f4c2e8f1700 1 -- 172.21.15.121:0/2321765195 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c280ac6f0 con 0x7f4c14004ff0 2023-11-15T02:57:00.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.083+0000 7f4c2e8f1700 1 -- 172.21.15.121:0/2321765195 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c28047060 con 0x7f4c14004ff0 2023-11-15T02:57:00.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.083+0000 7f4c24ff9700 1 -- 172.21.15.121:0/2321765195 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4c18014450 con 0x7f4c14004ff0 2023-11-15T02:57:00.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.087+0000 7f4c24ff9700 1 --2- 172.21.15.121:0/2321765195 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4c0805a540 0x7f4c0805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:00.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.087+0000 7f4c24ff9700 1 -- 172.21.15.121:0/2321765195 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4c180743b0 con 0x7f4c14004ff0 2023-11-15T02:57:00.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.087+0000 7f4c1ffff700 1 --2- 172.21.15.121:0/2321765195 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4c0805a540 0x7f4c0805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:00.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.087+0000 7f4c1ffff700 1 --2- 172.21.15.121:0/2321765195 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4c0805a540 0x7f4c0805ca00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f4c280c8780 tx=0x7f4c10015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:00.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.091+0000 7f4c24ff9700 1 -- 172.21.15.121:0/2321765195 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4c18047b20 con 0x7f4c14004ff0 2023-11-15T02:57:00.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.247+0000 7f4c2e8f1700 1 -- 172.21.15.121:0/2321765195 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4c280abce0 con 0x7f4c14004ff0 2023-11-15T02:57:00.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.251+0000 7f4c24ff9700 1 -- 172.21.15.121:0/2321765195 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4c1804b140 con 0x7f4c14004ff0 2023-11-15T02:57:00.256 INFO:teuthology.orchestra.run.smithi121.stdout:55834574851 2023-11-15T02:57:00.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.251+0000 7f4c2e8f1700 1 -- 172.21.15.121:0/2321765195 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4c0805a540 msgr2=0x7f4c0805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:00.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.251+0000 7f4c2e8f1700 1 --2- 172.21.15.121:0/2321765195 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4c0805a540 0x7f4c0805ca00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f4c280c8780 tx=0x7f4c10015000).stop 2023-11-15T02:57:00.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.251+0000 7f4c2e8f1700 1 -- 172.21.15.121:0/2321765195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c14004ff0 msgr2=0x7f4c28163ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:00.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.251+0000 7f4c2e8f1700 1 --2- 172.21.15.121:0/2321765195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c14004ff0 0x7f4c28163ed0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f4c1800f8b0 tx=0x7f4c18007bb0).stop 2023-11-15T02:57:00.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.251+0000 7f4c2e8f1700 1 -- 172.21.15.121:0/2321765195 shutdown_connections 2023-11-15T02:57:00.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.251+0000 7f4c2e8f1700 1 --2- 172.21.15.121:0/2321765195 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4c0805a540 0x7f4c0805ca00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:00.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.251+0000 7f4c2e8f1700 1 --2- 172.21.15.121:0/2321765195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c28154a80 0x7f4c280ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:00.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.251+0000 7f4c2e8f1700 1 --2- 172.21.15.121:0/2321765195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c280a9f10 0x7f4c28154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:00.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.251+0000 7f4c2e8f1700 1 --2- 172.21.15.121:0/2321765195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c14004ff0 0x7f4c28163ed0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:00.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.251+0000 7f4c2e8f1700 1 -- 172.21.15.121:0/2321765195 >> 172.21.15.121:0/2321765195 conn(0x7f4c28068b70 msgr2=0x7f4c280bd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:00.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.251+0000 7f4c2e8f1700 1 -- 172.21.15.121:0/2321765195 shutdown_connections 2023-11-15T02:57:00.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:00.251+0000 7f4c2e8f1700 1 -- 172.21.15.121:0/2321765195 wait complete. 2023-11-15T02:57:00.277 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.1 2023-11-15T02:57:01.278 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-15T02:57:01.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.431+0000 7f938969b700 1 Processor -- start 2023-11-15T02:57:01.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.431+0000 7f938969b700 1 -- start start 2023-11-15T02:57:01.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.435+0000 7f938969b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93840c5000 0x7f93840c5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:01.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.435+0000 7f938969b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f93840a9370 0x7f93840a9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:01.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.435+0000 7f938969b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93840a8ee0 con 0x7f93840aafe0 2023-11-15T02:57:01.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.435+0000 7f938969b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9384152010 con 0x7f93840a9370 2023-11-15T02:57:01.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.435+0000 7f938969b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93840bf730 con 0x7f93840c5000 2023-11-15T02:57:01.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.435+0000 7f93827fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93840c5000 0x7f93840c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:01.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.435+0000 7f93837fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f93840a9370 0x7f93840a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:01.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.435+0000 7f93827fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93840c5000 0x7f93840c5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:43894/0 (socket says 172.21.15.121:43894) 2023-11-15T02:57:01.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.435+0000 7f93827fc700 1 -- 172.21.15.121:0/4118984355 learned_addr learned my addr 172.21.15.121:0/4118984355 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:01.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.435+0000 7f93837fe700 1 -- 172.21.15.121:0/4118984355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93840c5000 msgr2=0x7f93840c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:01.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.435+0000 7f93837fe700 1 --2- 172.21.15.121:0/4118984355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93840c5000 0x7f93840c5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:01.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.435+0000 7f93837fe700 1 -- 172.21.15.121:0/4118984355 >> v1:172.21.15.190:6789/0 conn(0x7f93840aafe0 legacy=0x7f93840be8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:01.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.435+0000 7f93837fe700 1 -- 172.21.15.121:0/4118984355 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93840c3b30 con 0x7f93840a9370 2023-11-15T02:57:01.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.435+0000 7f93827fc700 1 --2- 172.21.15.121:0/4118984355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93840c5000 0x7f93840c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:57:01.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.435+0000 7f93837fe700 1 --2- 172.21.15.121:0/4118984355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93840a9370 0x7f93840a9800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f937c009e60 tx=0x7f937c009780).ready entity=mon.1 client_cookie=f2d9aa1994fa031c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:01.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.439+0000 7f9381ffb700 1 -- 172.21.15.121:0/4118984355 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c00e070 con 0x7f93840a9370 2023-11-15T02:57:01.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.439+0000 7f9381ffb700 1 -- 172.21.15.121:0/4118984355 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937c00bb60 con 0x7f93840a9370 2023-11-15T02:57:01.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.439+0000 7f9381ffb700 1 -- 172.21.15.121:0/4118984355 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c012410 con 0x7f93840a9370 2023-11-15T02:57:01.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.439+0000 7f938969b700 1 -- 172.21.15.121:0/4118984355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93840a9370 msgr2=0x7f93840a9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:01.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.439+0000 7f938969b700 1 --2- 172.21.15.121:0/4118984355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93840a9370 0x7f93840a9800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f937c009e60 tx=0x7f937c009780).stop 2023-11-15T02:57:01.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.439+0000 7f938969b700 1 -- 172.21.15.121:0/4118984355 shutdown_connections 2023-11-15T02:57:01.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.439+0000 7f938969b700 1 --2- 172.21.15.121:0/4118984355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93840a9370 0x7f93840a9800 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:01.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.439+0000 7f938969b700 1 --2- 172.21.15.121:0/4118984355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93840c5000 0x7f93840c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:01.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.439+0000 7f938969b700 1 -- 172.21.15.121:0/4118984355 >> 172.21.15.121:0/4118984355 conn(0x7f9384067a60 msgr2=0x7f9384067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:01.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.439+0000 7f938969b700 1 -- 172.21.15.121:0/4118984355 shutdown_connections 2023-11-15T02:57:01.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.439+0000 7f938969b700 1 -- 172.21.15.121:0/4118984355 wait complete. 2023-11-15T02:57:01.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.439+0000 7f938969b700 1 Processor -- start 2023-11-15T02:57:01.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.439+0000 7f938969b700 1 -- start start 2023-11-15T02:57:01.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f938969b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93840a9370 0x7f9384165a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:01.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f938969b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93840aafe0 0x7f9384154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:01.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f938969b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93840c5000 0x7f9384161a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:01.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f938969b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9384154fb0 con 0x7f93840aafe0 2023-11-15T02:57:01.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f938969b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9384165600 con 0x7f93840c5000 2023-11-15T02:57:01.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f938969b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9384164ff0 con 0x7f93840a9370 2023-11-15T02:57:01.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f93827fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93840aafe0 0x7f9384154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:01.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f93827fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93840aafe0 0x7f9384154ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33210/0 (socket says 172.21.15.121:33210) 2023-11-15T02:57:01.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f93827fc700 1 -- 172.21.15.121:0/878879491 learned_addr learned my addr 172.21.15.121:0/878879491 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:01.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f93837fe700 1 --2- 172.21.15.121:0/878879491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93840c5000 0x7f9384161a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:01.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f9382ffd700 1 --2- 172.21.15.121:0/878879491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93840a9370 0x7f9384165a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:01.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f93837fe700 1 -- 172.21.15.121:0/878879491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93840a9370 msgr2=0x7f9384165a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:01.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f93837fe700 1 --2- 172.21.15.121:0/878879491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93840a9370 0x7f9384165a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:01.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f93837fe700 1 -- 172.21.15.121:0/878879491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93840aafe0 msgr2=0x7f9384154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:01.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f93837fe700 1 --2- 172.21.15.121:0/878879491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93840aafe0 0x7f9384154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:01.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f93837fe700 1 -- 172.21.15.121:0/878879491 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9378009370 con 0x7f93840c5000 2023-11-15T02:57:01.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f93837fe700 1 --2- 172.21.15.121:0/878879491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93840c5000 0x7f9384161a40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f937c014000 tx=0x7f937c009780).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:01.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f936affd700 1 -- 172.21.15.121:0/878879491 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c012410 con 0x7f93840c5000 2023-11-15T02:57:01.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f936affd700 1 -- 172.21.15.121:0/878879491 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937c01d630 con 0x7f93840c5000 2023-11-15T02:57:01.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f938969b700 1 -- 172.21.15.121:0/878879491 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f937c009a20 con 0x7f93840c5000 2023-11-15T02:57:01.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f936affd700 1 -- 172.21.15.121:0/878879491 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c01d9d0 con 0x7f93840c5000 2023-11-15T02:57:01.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.443+0000 7f938969b700 1 -- 172.21.15.121:0/878879491 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9384162810 con 0x7f93840c5000 2023-11-15T02:57:01.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.447+0000 7f938969b700 1 -- 172.21.15.121:0/878879491 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9384047060 con 0x7f93840c5000 2023-11-15T02:57:01.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.447+0000 7f936affd700 1 -- 172.21.15.121:0/878879491 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f937c003040 con 0x7f93840c5000 2023-11-15T02:57:01.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.451+0000 7f936affd700 1 --2- 172.21.15.121:0/878879491 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f937005a510 0x7f937005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:01.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.451+0000 7f936affd700 1 -- 172.21.15.121:0/878879491 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f937c076420 con 0x7f93840c5000 2023-11-15T02:57:01.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.451+0000 7f9382ffd700 1 --2- 172.21.15.121:0/878879491 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f937005a510 0x7f937005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:01.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.451+0000 7f9382ffd700 1 --2- 172.21.15.121:0/878879491 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f937005a510 0x7f937005c9d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f9374003900 tx=0x7f9374007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:01.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.459+0000 7f936affd700 1 -- 172.21.15.121:0/878879491 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f937c049c00 con 0x7f93840c5000 2023-11-15T02:57:01.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.623+0000 7f938969b700 1 -- 172.21.15.121:0/878879491 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9384165740 con 0x7f93840c5000 2023-11-15T02:57:01.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.627+0000 7f936affd700 1 -- 172.21.15.121:0/878879491 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f937c04d220 con 0x7f93840c5000 2023-11-15T02:57:01.632 INFO:teuthology.orchestra.run.smithi121.stdout:55834574853 2023-11-15T02:57:01.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.627+0000 7f938969b700 1 -- 172.21.15.121:0/878879491 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f937005a510 msgr2=0x7f937005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:01.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.627+0000 7f938969b700 1 --2- 172.21.15.121:0/878879491 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f937005a510 0x7f937005c9d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f9374003900 tx=0x7f9374007d40).stop 2023-11-15T02:57:01.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.627+0000 7f938969b700 1 -- 172.21.15.121:0/878879491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93840c5000 msgr2=0x7f9384161a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:01.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.627+0000 7f938969b700 1 --2- 172.21.15.121:0/878879491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93840c5000 0x7f9384161a40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f937c014000 tx=0x7f937c009780).stop 2023-11-15T02:57:01.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.627+0000 7f938969b700 1 -- 172.21.15.121:0/878879491 shutdown_connections 2023-11-15T02:57:01.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.627+0000 7f938969b700 1 --2- 172.21.15.121:0/878879491 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f937005a510 0x7f937005c9d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:01.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.627+0000 7f938969b700 1 --2- 172.21.15.121:0/878879491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93840c5000 0x7f9384161a40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:01.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.627+0000 7f938969b700 1 --2- 172.21.15.121:0/878879491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93840aafe0 0x7f9384154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:01.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.627+0000 7f938969b700 1 --2- 172.21.15.121:0/878879491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93840a9370 0x7f9384165a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:01.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.627+0000 7f938969b700 1 -- 172.21.15.121:0/878879491 >> 172.21.15.121:0/878879491 conn(0x7f9384067a60 msgr2=0x7f93840c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:01.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.627+0000 7f938969b700 1 -- 172.21.15.121:0/878879491 shutdown_connections 2023-11-15T02:57:01.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.631+0000 7f938969b700 1 -- 172.21.15.121:0/878879491 wait complete. 2023-11-15T02:57:01.655 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.1 2023-11-15T02:57:01.655 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-15T02:57:01.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.803+0000 7f91b5bf9700 1 Processor -- start 2023-11-15T02:57:01.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.803+0000 7f91b5bf9700 1 -- start start 2023-11-15T02:57:01.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91b5bf9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f91b00c35d0 0x7f91b00bb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:01.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91b5bf9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b00a8b20 con 0x7f91b0153990 2023-11-15T02:57:01.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91b5bf9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b00c1760 con 0x7f91b0153d70 2023-11-15T02:57:01.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91b5bf9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b00a8d50 con 0x7f91b00c35d0 2023-11-15T02:57:01.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91b4bf7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f91b0153990 0x7f91b01581e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:57760/0 (socket says 172.21.15.121:57760) 2023-11-15T02:57:01.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91b4bf7700 1 -- 172.21.15.121:0/2757392783 learned_addr learned my addr 172.21.15.121:0/2757392783 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:01.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91a7fff700 1 --1- 172.21.15.121:0/2757392783 >> v1:172.21.15.190:6789/0 conn(0x7f91b0153d70 0x7f91b00c2ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:59366/0 (socket says 172.21.15.121:59366) 2023-11-15T02:57:01.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91b53f8700 1 --2- 172.21.15.121:0/2757392783 >> v2:172.21.15.190:3300/0 conn(0x7f91b00c35d0 0x7f91b00bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:01.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91a77fe700 1 -- 172.21.15.121:0/2757392783 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4079471390 0 0) 0x7f91b00c1760 con 0x7f91b0153d70 2023-11-15T02:57:01.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91a77fe700 1 -- 172.21.15.121:0/2757392783 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91980033f0 con 0x7f91b0153d70 2023-11-15T02:57:01.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91a77fe700 1 -- 172.21.15.121:0/2757392783 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298680239 0 0) 0x7f91b00a8b20 con 0x7f91b0153990 2023-11-15T02:57:01.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91a77fe700 1 -- 172.21.15.121:0/2757392783 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b00c1760 con 0x7f91b0153990 2023-11-15T02:57:01.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91a77fe700 1 -- 172.21.15.121:0/2757392783 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 164068837 0 0) 0x7f91b00c1760 con 0x7f91b0153990 2023-11-15T02:57:01.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91a77fe700 1 -- 172.21.15.121:0/2757392783 >> v2:172.21.15.190:3300/0 conn(0x7f91b00c35d0 msgr2=0x7f91b00bb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:01.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91a77fe700 1 --2- 172.21.15.121:0/2757392783 >> v2:172.21.15.190:3300/0 conn(0x7f91b00c35d0 0x7f91b00bb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:01.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91a77fe700 1 -- 172.21.15.121:0/2757392783 >> v1:172.21.15.190:6789/0 conn(0x7f91b0153d70 legacy=0x7f91b00c2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:01.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91a77fe700 1 -- 172.21.15.121:0/2757392783 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b00a9030 con 0x7f91b0153990 2023-11-15T02:57:01.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91a77fe700 1 -- 172.21.15.121:0/2757392783 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f91ac0017f0 con 0x7f91b0153990 2023-11-15T02:57:01.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91a77fe700 1 -- 172.21.15.121:0/2757392783 >> v1:172.21.15.121:6789/0 conn(0x7f91b0153990 legacy=0x7f91b01581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:01.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91a77fe700 1 --2- 172.21.15.121:0/2757392783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9198004ad0 0x7f9198004ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:01.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91a77fe700 1 --2- 172.21.15.121:0/2757392783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9198003630 0x7f9198003ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:01.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91a77fe700 1 --2- 172.21.15.121:0/2757392783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9198003fa0 0x7f919800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:01.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91a77fe700 1 -- 172.21.15.121:0/2757392783 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00a8d50 con 0x7f9198004ad0 2023-11-15T02:57:01.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91a77fe700 1 -- 172.21.15.121:0/2757392783 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919800be80 con 0x7f9198003630 2023-11-15T02:57:01.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91a77fe700 1 -- 172.21.15.121:0/2757392783 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919800c040 con 0x7f9198003fa0 2023-11-15T02:57:01.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91b4bf7700 1 --2- 172.21.15.121:0/2757392783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9198004ad0 0x7f9198004ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:01.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91a7fff700 1 --2- 172.21.15.121:0/2757392783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9198003630 0x7f9198003ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:01.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.807+0000 7f91b53f8700 1 --2- 172.21.15.121:0/2757392783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9198003fa0 0x7f919800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:01.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.811+0000 7f91b4bf7700 1 -- 172.21.15.121:0/2757392783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9198003fa0 msgr2=0x7f919800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:01.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.811+0000 7f91b4bf7700 1 --2- 172.21.15.121:0/2757392783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9198003fa0 0x7f919800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:01.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.811+0000 7f91b4bf7700 1 -- 172.21.15.121:0/2757392783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9198003630 msgr2=0x7f9198003ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:01.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.811+0000 7f91b4bf7700 1 --2- 172.21.15.121:0/2757392783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9198003630 0x7f9198003ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:01.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.811+0000 7f91b4bf7700 1 -- 172.21.15.121:0/2757392783 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b00c1760 con 0x7f9198004ad0 2023-11-15T02:57:01.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.811+0000 7f91b53f8700 1 -- 172.21.15.121:0/2757392783 reap_dead start 2023-11-15T02:57:01.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.811+0000 7f91b4bf7700 1 --2- 172.21.15.121:0/2757392783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9198004ad0 0x7f9198004ed0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f91ac00c5c0 tx=0x7f91ac007f00).ready entity=mon.0 client_cookie=5d8a8280ead5b36 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:01.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.811+0000 7f91a77fe700 1 -- 172.21.15.121:0/2757392783 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac00cd80 con 0x7f9198004ad0 2023-11-15T02:57:01.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.811+0000 7f91a77fe700 1 -- 172.21.15.121:0/2757392783 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ac005b20 con 0x7f9198004ad0 2023-11-15T02:57:01.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.811+0000 7f91a77fe700 1 -- 172.21.15.121:0/2757392783 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac0070d0 con 0x7f9198004ad0 2023-11-15T02:57:01.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.811+0000 7f91b5bf9700 1 -- 172.21.15.121:0/2757392783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9198004ad0 msgr2=0x7f9198004ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:01.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.811+0000 7f91b5bf9700 1 --2- 172.21.15.121:0/2757392783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9198004ad0 0x7f9198004ed0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f91ac00c5c0 tx=0x7f91ac007f00).stop 2023-11-15T02:57:01.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.811+0000 7f91b5bf9700 1 -- 172.21.15.121:0/2757392783 shutdown_connections 2023-11-15T02:57:01.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.811+0000 7f91b5bf9700 1 --2- 172.21.15.121:0/2757392783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9198004ad0 0x7f9198004ed0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:01.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.811+0000 7f91b5bf9700 1 -- 172.21.15.121:0/2757392783 >> 172.21.15.121:0/2757392783 conn(0x7f91b0067a60 msgr2=0x7f91b0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:01.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.811+0000 7f91b5bf9700 1 -- 172.21.15.121:0/2757392783 shutdown_connections 2023-11-15T02:57:01.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.811+0000 7f91b5bf9700 1 -- 172.21.15.121:0/2757392783 wait complete. 2023-11-15T02:57:01.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.811+0000 7f91b5bf9700 1 Processor -- start 2023-11-15T02:57:01.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.811+0000 7f91b5bf9700 1 -- start start 2023-11-15T02:57:01.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91b5bf9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9198004ad0 0x7f91b00bfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:01.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91b5bf9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91b00bd8c0 0x7f91b013a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:01.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91b5bf9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91b0167ff0 0x7f91b0168470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:01.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91b5bf9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00c0220 con 0x7f91b0167ff0 2023-11-15T02:57:01.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91b5bf9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00a9d70 con 0x7f9198004ad0 2023-11-15T02:57:01.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91b5bf9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00aa330 con 0x7f91b00bd8c0 2023-11-15T02:57:01.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91b53f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91b0167ff0 0x7f91b0168470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:01.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91b53f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91b0167ff0 0x7f91b0168470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33226/0 (socket says 172.21.15.121:33226) 2023-11-15T02:57:01.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91b53f8700 1 -- 172.21.15.121:0/1628614366 learned_addr learned my addr 172.21.15.121:0/1628614366 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:01.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91b4bf7700 1 --2- 172.21.15.121:0/1628614366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9198004ad0 0x7f91b00bfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:01.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91a7fff700 1 --2- 172.21.15.121:0/1628614366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91b00bd8c0 0x7f91b013a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:01.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91b53f8700 1 -- 172.21.15.121:0/1628614366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91b00bd8c0 msgr2=0x7f91b013a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:01.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91b53f8700 1 --2- 172.21.15.121:0/1628614366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91b00bd8c0 0x7f91b013a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:01.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91b53f8700 1 -- 172.21.15.121:0/1628614366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9198004ad0 msgr2=0x7f91b00bfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:01.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91b53f8700 1 --2- 172.21.15.121:0/1628614366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9198004ad0 0x7f91b00bfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:01.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91b53f8700 1 -- 172.21.15.121:0/1628614366 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b0168960 con 0x7f91b0167ff0 2023-11-15T02:57:01.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91b4bf7700 1 --2- 172.21.15.121:0/1628614366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9198004ad0 0x7f91b00bfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:01.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91b53f8700 1 --2- 172.21.15.121:0/1628614366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91b0167ff0 0x7f91b0168470 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f91b005fc30 tx=0x7f91a8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:01.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91a4ff9700 1 -- 172.21.15.121:0/1628614366 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a8009e10 con 0x7f91b0167ff0 2023-11-15T02:57:01.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91a4ff9700 1 -- 172.21.15.121:0/1628614366 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a8013550 con 0x7f91b0167ff0 2023-11-15T02:57:01.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91b5bf9700 1 -- 172.21.15.121:0/1628614366 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b00c0460 con 0x7f91b0167ff0 2023-11-15T02:57:01.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91a4ff9700 1 -- 172.21.15.121:0/1628614366 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a80138d0 con 0x7f91b0167ff0 2023-11-15T02:57:01.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.815+0000 7f91b5bf9700 1 -- 172.21.15.121:0/1628614366 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b00a9fc0 con 0x7f91b0167ff0 2023-11-15T02:57:01.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.819+0000 7f91b5bf9700 1 -- 172.21.15.121:0/1628614366 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b006bcf0 con 0x7f91b0167ff0 2023-11-15T02:57:01.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.819+0000 7f91a4ff9700 1 -- 172.21.15.121:0/1628614366 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f91a8013d00 con 0x7f91b0167ff0 2023-11-15T02:57:01.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.819+0000 7f91a4ff9700 1 --2- 172.21.15.121:0/1628614366 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f919c05a540 0x7f919c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:01.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.819+0000 7f91a4ff9700 1 -- 172.21.15.121:0/1628614366 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f91a8074b00 con 0x7f91b0167ff0 2023-11-15T02:57:01.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.819+0000 7f91b4bf7700 1 --2- 172.21.15.121:0/1628614366 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f919c05a540 0x7f919c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:01.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.819+0000 7f91b4bf7700 1 --2- 172.21.15.121:0/1628614366 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f919c05a540 0x7f919c05ca00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f91ac001e10 tx=0x7f91ac015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:01.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.827+0000 7f91a4ff9700 1 -- 172.21.15.121:0/1628614366 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f91a8076e60 con 0x7f91b0167ff0 2023-11-15T02:57:01.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.991+0000 7f91b5bf9700 1 -- 172.21.15.121:0/1628614366 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f91b00aa470 con 0x7f91b0167ff0 2023-11-15T02:57:01.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.991+0000 7f91a4ff9700 1 -- 172.21.15.121:0/1628614366 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f91a8048220 con 0x7f91b0167ff0 2023-11-15T02:57:01.996 INFO:teuthology.orchestra.run.smithi121.stdout:60129542149 2023-11-15T02:57:01.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.995+0000 7f91b5bf9700 1 -- 172.21.15.121:0/1628614366 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f919c05a540 msgr2=0x7f919c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:01.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.995+0000 7f91b5bf9700 1 --2- 172.21.15.121:0/1628614366 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f919c05a540 0x7f919c05ca00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f91ac001e10 tx=0x7f91ac015000).stop 2023-11-15T02:57:02.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.995+0000 7f91b5bf9700 1 -- 172.21.15.121:0/1628614366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91b0167ff0 msgr2=0x7f91b0168470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:02.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.995+0000 7f91b5bf9700 1 --2- 172.21.15.121:0/1628614366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91b0167ff0 0x7f91b0168470 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f91b005fc30 tx=0x7f91a8008040).stop 2023-11-15T02:57:02.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.995+0000 7f91b5bf9700 1 -- 172.21.15.121:0/1628614366 shutdown_connections 2023-11-15T02:57:02.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.995+0000 7f91b5bf9700 1 --2- 172.21.15.121:0/1628614366 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f919c05a540 0x7f919c05ca00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.995+0000 7f91b5bf9700 1 --2- 172.21.15.121:0/1628614366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91b0167ff0 0x7f91b0168470 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.995+0000 7f91b5bf9700 1 --2- 172.21.15.121:0/1628614366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91b00bd8c0 0x7f91b013a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.995+0000 7f91b5bf9700 1 --2- 172.21.15.121:0/1628614366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9198004ad0 0x7f91b00bfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.995+0000 7f91b5bf9700 1 -- 172.21.15.121:0/1628614366 >> 172.21.15.121:0/1628614366 conn(0x7f91b0067a60 msgr2=0x7f91b0066290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:02.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.995+0000 7f91b5bf9700 1 -- 172.21.15.121:0/1628614366 shutdown_connections 2023-11-15T02:57:02.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:01.995+0000 7f91b5bf9700 1 -- 172.21.15.121:0/1628614366 wait complete. 2023-11-15T02:57:02.019 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.2 2023-11-15T02:57:02.019 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-15T02:57:02.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.167+0000 7f3da5d07700 1 Processor -- start 2023-11-15T02:57:02.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.167+0000 7f3da5d07700 1 -- start start 2023-11-15T02:57:02.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.171+0000 7f3da5d07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da00bb510 0x7f3da00abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:02.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.171+0000 7f3da5d07700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da00c2c00 con 0x7f3da00abaa0 2023-11-15T02:57:02.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.171+0000 7f3da5d07700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da00657b0 con 0x7f3da00a8880 2023-11-15T02:57:02.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.171+0000 7f3da5d07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da0065a90 con 0x7f3da00bb510 2023-11-15T02:57:02.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.171+0000 7f3da4d05700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3da00abaa0 0x7f3da0063570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:57774/0 (socket says 172.21.15.121:57774) 2023-11-15T02:57:02.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.171+0000 7f3da4d05700 1 -- 172.21.15.121:0/3852690565 learned_addr learned my addr 172.21.15.121:0/3852690565 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:02.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.171+0000 7f3d97fff700 1 --2- 172.21.15.121:0/3852690565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da00bb510 0x7f3da00abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:02.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.171+0000 7f3d977fe700 1 -- 172.21.15.121:0/3852690565 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076797433 0 0) 0x7f3da00c2c00 con 0x7f3da00abaa0 2023-11-15T02:57:02.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.171+0000 7f3d977fe700 1 -- 172.21.15.121:0/3852690565 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d880033f0 con 0x7f3da00abaa0 2023-11-15T02:57:02.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.171+0000 7f3d97fff700 1 -- 172.21.15.121:0/3852690565 >> v1:172.21.15.190:6789/0 conn(0x7f3da00a8880 legacy=0x7f3da0151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:02.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.171+0000 7f3d97fff700 1 -- 172.21.15.121:0/3852690565 >> v1:172.21.15.121:6789/0 conn(0x7f3da00abaa0 legacy=0x7f3da0063570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:02.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.171+0000 7f3d97fff700 1 -- 172.21.15.121:0/3852690565 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da00c2890 con 0x7f3da00bb510 2023-11-15T02:57:02.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.175+0000 7f3d97fff700 1 --2- 172.21.15.121:0/3852690565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da00bb510 0x7f3da00abe80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f3d90009760 tx=0x7f3d90006f40).ready entity=mon.2 client_cookie=b56a16a53a44039e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:02.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.175+0000 7f3d977fe700 1 -- 172.21.15.121:0/3852690565 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d90005880 con 0x7f3da00bb510 2023-11-15T02:57:02.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.175+0000 7f3d977fe700 1 -- 172.21.15.121:0/3852690565 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d900059e0 con 0x7f3da00bb510 2023-11-15T02:57:02.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.175+0000 7f3d977fe700 1 -- 172.21.15.121:0/3852690565 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d90005d60 con 0x7f3da00bb510 2023-11-15T02:57:02.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.175+0000 7f3da5d07700 1 -- 172.21.15.121:0/3852690565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da00bb510 msgr2=0x7f3da00abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:02.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.175+0000 7f3da5d07700 1 --2- 172.21.15.121:0/3852690565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da00bb510 0x7f3da00abe80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f3d90009760 tx=0x7f3d90006f40).stop 2023-11-15T02:57:02.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.175+0000 7f3da5d07700 1 -- 172.21.15.121:0/3852690565 shutdown_connections 2023-11-15T02:57:02.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.175+0000 7f3da5d07700 1 --2- 172.21.15.121:0/3852690565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da00bb510 0x7f3da00abe80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.175+0000 7f3da5d07700 1 -- 172.21.15.121:0/3852690565 >> 172.21.15.121:0/3852690565 conn(0x7f3da0067a60 msgr2=0x7f3da0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:02.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.175+0000 7f3da5d07700 1 -- 172.21.15.121:0/3852690565 shutdown_connections 2023-11-15T02:57:02.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.175+0000 7f3da5d07700 1 -- 172.21.15.121:0/3852690565 wait complete. 2023-11-15T02:57:02.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.179+0000 7f3da5d07700 1 Processor -- start 2023-11-15T02:57:02.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.179+0000 7f3da5d07700 1 -- start start 2023-11-15T02:57:02.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.179+0000 7f3da5d07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da00a8880 0x7f3da0163e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:02.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.179+0000 7f3da5d07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da00abaa0 0x7f3da0147f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:02.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.179+0000 7f3da5d07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da00bb510 0x7f3da0148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:02.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.179+0000 7f3da5d07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da0143910 con 0x7f3da00abaa0 2023-11-15T02:57:02.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.179+0000 7f3da5d07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da00bd5f0 con 0x7f3da00bb510 2023-11-15T02:57:02.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.179+0000 7f3da5d07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da00bd840 con 0x7f3da00a8880 2023-11-15T02:57:02.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.179+0000 7f3d97fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da00abaa0 0x7f3da0147f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:02.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.179+0000 7f3d97fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da00abaa0 0x7f3da0147f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33238/0 (socket says 172.21.15.121:33238) 2023-11-15T02:57:02.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.179+0000 7f3d97fff700 1 -- 172.21.15.121:0/2948347012 learned_addr learned my addr 172.21.15.121:0/2948347012 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:02.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.179+0000 7f3da4d05700 1 --2- 172.21.15.121:0/2948347012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da00a8880 0x7f3da0163e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:02.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.179+0000 7f3da5506700 1 --2- 172.21.15.121:0/2948347012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da00bb510 0x7f3da0148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:02.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.179+0000 7f3d97fff700 1 -- 172.21.15.121:0/2948347012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da00a8880 msgr2=0x7f3da0163e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:02.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.179+0000 7f3d97fff700 1 --2- 172.21.15.121:0/2948347012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da00a8880 0x7f3da0163e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.179+0000 7f3d97fff700 1 -- 172.21.15.121:0/2948347012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da00bb510 msgr2=0x7f3da0148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:02.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.179+0000 7f3d97fff700 1 --2- 172.21.15.121:0/2948347012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da00bb510 0x7f3da0148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.179+0000 7f3d97fff700 1 -- 172.21.15.121:0/2948347012 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d90009250 con 0x7f3da00abaa0 2023-11-15T02:57:02.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.179+0000 7f3da5506700 1 --2- 172.21.15.121:0/2948347012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da00bb510 0x7f3da0148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:02.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.183+0000 7f3d97fff700 1 --2- 172.21.15.121:0/2948347012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da00abaa0 0x7f3da0147f20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f3da0064eb0 tx=0x7f3d90015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:02.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.183+0000 7f3d94ff9700 1 -- 172.21.15.121:0/2948347012 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d90005cd0 con 0x7f3da00abaa0 2023-11-15T02:57:02.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.183+0000 7f3d94ff9700 1 -- 172.21.15.121:0/2948347012 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9000f720 con 0x7f3da00abaa0 2023-11-15T02:57:02.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.183+0000 7f3da5d07700 1 -- 172.21.15.121:0/2948347012 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da00bdb20 con 0x7f3da00abaa0 2023-11-15T02:57:02.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.183+0000 7f3d94ff9700 1 -- 172.21.15.121:0/2948347012 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9000fae0 con 0x7f3da00abaa0 2023-11-15T02:57:02.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.183+0000 7f3da5d07700 1 -- 172.21.15.121:0/2948347012 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da00be0e0 con 0x7f3da00abaa0 2023-11-15T02:57:02.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.183+0000 7f3da5d07700 1 -- 172.21.15.121:0/2948347012 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da00be300 con 0x7f3da00abaa0 2023-11-15T02:57:02.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.183+0000 7f3d94ff9700 1 -- 172.21.15.121:0/2948347012 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3d90011440 con 0x7f3da00abaa0 2023-11-15T02:57:02.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.187+0000 7f3d94ff9700 1 --2- 172.21.15.121:0/2948347012 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3d8c05a510 0x7f3d8c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:02.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.187+0000 7f3d94ff9700 1 -- 172.21.15.121:0/2948347012 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3d900757c0 con 0x7f3da00abaa0 2023-11-15T02:57:02.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.187+0000 7f3da4d05700 1 --2- 172.21.15.121:0/2948347012 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3d8c05a510 0x7f3d8c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:02.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.187+0000 7f3da4d05700 1 --2- 172.21.15.121:0/2948347012 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3d8c05a510 0x7f3d8c05c9d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f3d9c005750 tx=0x7f3d9c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:02.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.191+0000 7f3d94ff9700 1 -- 172.21.15.121:0/2948347012 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3d90048f30 con 0x7f3da00abaa0 2023-11-15T02:57:02.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.347+0000 7f3da5d07700 1 -- 172.21.15.121:0/2948347012 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3da006bcf0 con 0x7f3da00abaa0 2023-11-15T02:57:02.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.347+0000 7f3d94ff9700 1 -- 172.21.15.121:0/2948347012 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3d90022070 con 0x7f3da00abaa0 2023-11-15T02:57:02.356 INFO:teuthology.orchestra.run.smithi121.stdout:60129542149 2023-11-15T02:57:02.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.351+0000 7f3da5d07700 1 -- 172.21.15.121:0/2948347012 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3d8c05a510 msgr2=0x7f3d8c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:02.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.351+0000 7f3da5d07700 1 --2- 172.21.15.121:0/2948347012 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3d8c05a510 0x7f3d8c05c9d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f3d9c005750 tx=0x7f3d9c009040).stop 2023-11-15T02:57:02.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.351+0000 7f3da5d07700 1 -- 172.21.15.121:0/2948347012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da00abaa0 msgr2=0x7f3da0147f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:02.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.351+0000 7f3da5d07700 1 --2- 172.21.15.121:0/2948347012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da00abaa0 0x7f3da0147f20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f3da0064eb0 tx=0x7f3d90015000).stop 2023-11-15T02:57:02.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.351+0000 7f3da5d07700 1 -- 172.21.15.121:0/2948347012 shutdown_connections 2023-11-15T02:57:02.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.351+0000 7f3da5d07700 1 --2- 172.21.15.121:0/2948347012 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3d8c05a510 0x7f3d8c05c9d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.351+0000 7f3da5d07700 1 --2- 172.21.15.121:0/2948347012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da00bb510 0x7f3da0148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.351+0000 7f3da5d07700 1 --2- 172.21.15.121:0/2948347012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3da00abaa0 0x7f3da0147f20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.351+0000 7f3da5d07700 1 --2- 172.21.15.121:0/2948347012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da00a8880 0x7f3da0163e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.351+0000 7f3da5d07700 1 -- 172.21.15.121:0/2948347012 >> 172.21.15.121:0/2948347012 conn(0x7f3da0067a60 msgr2=0x7f3da014e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:02.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.351+0000 7f3da5d07700 1 -- 172.21.15.121:0/2948347012 shutdown_connections 2023-11-15T02:57:02.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.351+0000 7f3da5d07700 1 -- 172.21.15.121:0/2948347012 wait complete. 2023-11-15T02:57:02.375 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.3 2023-11-15T02:57:02.376 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-15T02:57:02.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.539+0000 7ff83165b700 1 Processor -- start 2023-11-15T02:57:02.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.539+0000 7ff83165b700 1 -- start start 2023-11-15T02:57:02.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff83165b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff82c0a8de0 0x7ff82c066420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:02.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff83165b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff82c150f90 0x7ff82c151410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:02.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff83165b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82c062190 0x7ff82c0625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:02.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff83165b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82c065f90 con 0x7ff82c150f90 2023-11-15T02:57:02.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff83165b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82c151900 con 0x7ff82c0a8de0 2023-11-15T02:57:02.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff83165b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c147ea0 con 0x7ff82c062190 2023-11-15T02:57:02.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff82a7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff82c150f90 0x7ff82c151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:02.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff82a7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff82c150f90 0x7ff82c151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33250/0 (socket says 172.21.15.121:33250) 2023-11-15T02:57:02.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff82a7fc700 1 -- 172.21.15.121:0/1411710612 learned_addr learned my addr 172.21.15.121:0/1411710612 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:02.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff82b7fe700 1 --2- 172.21.15.121:0/1411710612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82c062190 0x7ff82c0625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:02.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff82affd700 1 --2- 172.21.15.121:0/1411710612 >> v2:172.21.15.190:3300/0 conn(0x7ff82c0a8de0 0x7ff82c066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:02.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff82a7fc700 1 -- 172.21.15.121:0/1411710612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82c062190 msgr2=0x7ff82c0625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:02.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff82a7fc700 1 --2- 172.21.15.121:0/1411710612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82c062190 0x7ff82c0625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff82a7fc700 1 -- 172.21.15.121:0/1411710612 >> v2:172.21.15.190:3300/0 conn(0x7ff82c0a8de0 msgr2=0x7ff82c066420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:02.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff82a7fc700 1 --2- 172.21.15.121:0/1411710612 >> v2:172.21.15.190:3300/0 conn(0x7ff82c0a8de0 0x7ff82c066420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff82a7fc700 1 -- 172.21.15.121:0/1411710612 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c062ac0 con 0x7ff82c150f90 2023-11-15T02:57:02.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff82a7fc700 1 --2- 172.21.15.121:0/1411710612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff82c150f90 0x7ff82c151410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7ff82c0c44d0 tx=0x7ff82000b3b0).ready entity=mon.0 client_cookie=7b4f3c9930ebd79a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:02.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff829ffb700 1 -- 172.21.15.121:0/1411710612 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82000db10 con 0x7ff82c150f90 2023-11-15T02:57:02.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff829ffb700 1 -- 172.21.15.121:0/1411710612 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82000dc70 con 0x7ff82c150f90 2023-11-15T02:57:02.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff829ffb700 1 -- 172.21.15.121:0/1411710612 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff820010470 con 0x7ff82c150f90 2023-11-15T02:57:02.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff83165b700 1 -- 172.21.15.121:0/1411710612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff82c150f90 msgr2=0x7ff82c151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:02.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.543+0000 7ff83165b700 1 --2- 172.21.15.121:0/1411710612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff82c150f90 0x7ff82c151410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7ff82c0c44d0 tx=0x7ff82000b3b0).stop 2023-11-15T02:57:02.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.547+0000 7ff83165b700 1 -- 172.21.15.121:0/1411710612 shutdown_connections 2023-11-15T02:57:02.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.547+0000 7ff83165b700 1 --2- 172.21.15.121:0/1411710612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82c062190 0x7ff82c0625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.547+0000 7ff83165b700 1 --2- 172.21.15.121:0/1411710612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff82c150f90 0x7ff82c151410 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.547+0000 7ff83165b700 1 --2- 172.21.15.121:0/1411710612 >> v2:172.21.15.190:3300/0 conn(0x7ff82c0a8de0 0x7ff82c066420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.547+0000 7ff83165b700 1 -- 172.21.15.121:0/1411710612 >> 172.21.15.121:0/1411710612 conn(0x7ff82c067a60 msgr2=0x7ff82c067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:02.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.547+0000 7ff83165b700 1 -- 172.21.15.121:0/1411710612 shutdown_connections 2023-11-15T02:57:02.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.547+0000 7ff83165b700 1 -- 172.21.15.121:0/1411710612 wait complete. 2023-11-15T02:57:02.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.547+0000 7ff83165b700 1 Processor -- start 2023-11-15T02:57:02.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.547+0000 7ff83165b700 1 -- start start 2023-11-15T02:57:02.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff83165b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82c062190 0x7ff82c148300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:02.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff83165b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82c0a8de0 0x7ff82c0a9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:02.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff83165b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff82c150f90 0x7ff82c0aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:02.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff83165b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c0aa5c0 con 0x7ff82c150f90 2023-11-15T02:57:02.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff83165b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c0aa860 con 0x7ff82c062190 2023-11-15T02:57:02.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff83165b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c0aab40 con 0x7ff82c0a8de0 2023-11-15T02:57:02.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff82b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff82c150f90 0x7ff82c0aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:02.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff82b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff82c150f90 0x7ff82c0aa0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33262/0 (socket says 172.21.15.121:33262) 2023-11-15T02:57:02.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff82b7fe700 1 -- 172.21.15.121:0/2728065140 learned_addr learned my addr 172.21.15.121:0/2728065140 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:02.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff82affd700 1 --2- 172.21.15.121:0/2728065140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82c062190 0x7ff82c148300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:02.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff82a7fc700 1 --2- 172.21.15.121:0/2728065140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82c0a8de0 0x7ff82c0a9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:02.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff82b7fe700 1 -- 172.21.15.121:0/2728065140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82c0a8de0 msgr2=0x7ff82c0a9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:02.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff82b7fe700 1 --2- 172.21.15.121:0/2728065140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82c0a8de0 0x7ff82c0a9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff82b7fe700 1 -- 172.21.15.121:0/2728065140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82c062190 msgr2=0x7ff82c148300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:02.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff82b7fe700 1 --2- 172.21.15.121:0/2728065140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82c062190 0x7ff82c148300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff82b7fe700 1 -- 172.21.15.121:0/2728065140 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82000b610 con 0x7ff82c150f90 2023-11-15T02:57:02.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff82b7fe700 1 --2- 172.21.15.121:0/2728065140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff82c150f90 0x7ff82c0aa0d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7ff824009fa0 tx=0x7ff824009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:02.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff812ffd700 1 -- 172.21.15.121:0/2728065140 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82400ebc0 con 0x7ff82c150f90 2023-11-15T02:57:02.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff812ffd700 1 -- 172.21.15.121:0/2728065140 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82400ed20 con 0x7ff82c150f90 2023-11-15T02:57:02.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.551+0000 7ff812ffd700 1 -- 172.21.15.121:0/2728065140 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824014610 con 0x7ff82c150f90 2023-11-15T02:57:02.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.555+0000 7ff83165b700 1 -- 172.21.15.121:0/2728065140 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82c0aae20 con 0x7ff82c150f90 2023-11-15T02:57:02.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.555+0000 7ff83165b700 1 -- 172.21.15.121:0/2728065140 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82c0ab3e0 con 0x7ff82c150f90 2023-11-15T02:57:02.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.555+0000 7ff83165b700 1 -- 172.21.15.121:0/2728065140 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff82c047060 con 0x7ff82c150f90 2023-11-15T02:57:02.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.555+0000 7ff812ffd700 1 -- 172.21.15.121:0/2728065140 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff82401d3b0 con 0x7ff82c150f90 2023-11-15T02:57:02.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.555+0000 7ff812ffd700 1 --2- 172.21.15.121:0/2728065140 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff81805a510 0x7ff81805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:02.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.559+0000 7ff812ffd700 1 -- 172.21.15.121:0/2728065140 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff824074d50 con 0x7ff82c150f90 2023-11-15T02:57:02.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.559+0000 7ff82affd700 1 --2- 172.21.15.121:0/2728065140 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff81805a510 0x7ff81805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:02.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.559+0000 7ff82affd700 1 --2- 172.21.15.121:0/2728065140 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff81805a510 0x7ff81805c9d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7ff81c00bbc0 tx=0x7ff81c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:02.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.563+0000 7ff812ffd700 1 -- 172.21.15.121:0/2728065140 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff824048410 con 0x7ff82c150f90 2023-11-15T02:57:02.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.723+0000 7ff83165b700 1 -- 172.21.15.121:0/2728065140 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff82c0c2740 con 0x7ff82c150f90 2023-11-15T02:57:02.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.727+0000 7ff812ffd700 1 -- 172.21.15.121:0/2728065140 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff82404ba30 con 0x7ff82c150f90 2023-11-15T02:57:02.731 INFO:teuthology.orchestra.run.smithi121.stdout:60129542149 2023-11-15T02:57:02.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.731+0000 7ff83165b700 1 -- 172.21.15.121:0/2728065140 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff81805a510 msgr2=0x7ff81805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:02.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.731+0000 7ff83165b700 1 --2- 172.21.15.121:0/2728065140 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff81805a510 0x7ff81805c9d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7ff81c00bbc0 tx=0x7ff81c008040).stop 2023-11-15T02:57:02.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.731+0000 7ff83165b700 1 -- 172.21.15.121:0/2728065140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff82c150f90 msgr2=0x7ff82c0aa0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:02.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.731+0000 7ff83165b700 1 --2- 172.21.15.121:0/2728065140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff82c150f90 0x7ff82c0aa0d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7ff824009fa0 tx=0x7ff824009900).stop 2023-11-15T02:57:02.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.731+0000 7ff83165b700 1 -- 172.21.15.121:0/2728065140 shutdown_connections 2023-11-15T02:57:02.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.731+0000 7ff83165b700 1 --2- 172.21.15.121:0/2728065140 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff81805a510 0x7ff81805c9d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.731+0000 7ff83165b700 1 --2- 172.21.15.121:0/2728065140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff82c150f90 0x7ff82c0aa0d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.731+0000 7ff83165b700 1 --2- 172.21.15.121:0/2728065140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82c0a8de0 0x7ff82c0a9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.731+0000 7ff83165b700 1 --2- 172.21.15.121:0/2728065140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82c062190 0x7ff82c148300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.731+0000 7ff83165b700 1 -- 172.21.15.121:0/2728065140 >> 172.21.15.121:0/2728065140 conn(0x7ff82c067a60 msgr2=0x7ff82c0c5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:02.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.731+0000 7ff83165b700 1 -- 172.21.15.121:0/2728065140 shutdown_connections 2023-11-15T02:57:02.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.731+0000 7ff83165b700 1 -- 172.21.15.121:0/2728065140 wait complete. 2023-11-15T02:57:02.757 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.4 2023-11-15T02:57:02.757 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-15T02:57:02.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.915+0000 7f52e759e700 1 Processor -- start 2023-11-15T02:57:02.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.915+0000 7f52e759e700 1 -- start start 2023-11-15T02:57:02.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.915+0000 7f52e759e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f52e81511f0 0x7f52e8066570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:02.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.915+0000 7f52e759e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f52e80bcab0 0x7f52e81515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:02.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.915+0000 7f52e759e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e8066a60 con 0x7f52e8151ac0 2023-11-15T02:57:02.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.915+0000 7f52e759e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e8151ef0 con 0x7f52e81511f0 2023-11-15T02:57:02.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.915+0000 7f52e759e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e80660e0 con 0x7f52e80bcab0 2023-11-15T02:57:02.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.915+0000 7f52e5d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f52e80bcab0 0x7f52e81515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:02.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.915+0000 7f52e659c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f52e81511f0 0x7f52e8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:02.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.915+0000 7f52e5d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f52e80bcab0 0x7f52e81515d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:43428/0 (socket says 172.21.15.121:43428) 2023-11-15T02:57:02.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.915+0000 7f52e5d9b700 1 -- 172.21.15.121:0/3193420657 learned_addr learned my addr 172.21.15.121:0/3193420657 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:02.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.915+0000 7f52e6d9d700 1 --1- 172.21.15.121:0/3193420657 >> v1:172.21.15.190:6789/0 conn(0x7f52e8151ac0 0x7f52e80a8750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:59380/0 (socket says 172.21.15.121:59380) 2023-11-15T02:57:02.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.915+0000 7f52e659c700 1 -- 172.21.15.121:0/3193420657 >> v2:172.21.15.190:3300/0 conn(0x7f52e80bcab0 msgr2=0x7f52e81515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:02.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.915+0000 7f52e659c700 1 --2- 172.21.15.121:0/3193420657 >> v2:172.21.15.190:3300/0 conn(0x7f52e80bcab0 0x7f52e81515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.915+0000 7f52e659c700 1 -- 172.21.15.121:0/3193420657 >> v1:172.21.15.190:6789/0 conn(0x7f52e8151ac0 legacy=0x7f52e80a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:02.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.915+0000 7f52e659c700 1 -- 172.21.15.121:0/3193420657 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e80bd390 con 0x7f52e81511f0 2023-11-15T02:57:02.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.919+0000 7f52e659c700 1 --2- 172.21.15.121:0/3193420657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e81511f0 0x7f52e8066570 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f52e80bc8a0 tx=0x7f52d800b990).ready entity=mon.0 client_cookie=8a02256ffdc97b7b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:02.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.919+0000 7f52e559a700 1 -- 172.21.15.121:0/3193420657 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d8010070 con 0x7f52e81511f0 2023-11-15T02:57:02.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.919+0000 7f52e559a700 1 -- 172.21.15.121:0/3193420657 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d800b2e0 con 0x7f52e81511f0 2023-11-15T02:57:02.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.919+0000 7f52e559a700 1 -- 172.21.15.121:0/3193420657 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d800dd10 con 0x7f52e81511f0 2023-11-15T02:57:02.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.919+0000 7f52e759e700 1 -- 172.21.15.121:0/3193420657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e81511f0 msgr2=0x7f52e8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:02.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.919+0000 7f52e759e700 1 --2- 172.21.15.121:0/3193420657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e81511f0 0x7f52e8066570 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f52e80bc8a0 tx=0x7f52d800b990).stop 2023-11-15T02:57:02.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.919+0000 7f52e759e700 1 -- 172.21.15.121:0/3193420657 shutdown_connections 2023-11-15T02:57:02.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.919+0000 7f52e759e700 1 --2- 172.21.15.121:0/3193420657 >> v2:172.21.15.190:3300/0 conn(0x7f52e80bcab0 0x7f52e81515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.919+0000 7f52e759e700 1 --2- 172.21.15.121:0/3193420657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e81511f0 0x7f52e8066570 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.919+0000 7f52e759e700 1 -- 172.21.15.121:0/3193420657 >> 172.21.15.121:0/3193420657 conn(0x7f52e8067a60 msgr2=0x7f52e8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:02.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.919+0000 7f52e759e700 1 -- 172.21.15.121:0/3193420657 shutdown_connections 2023-11-15T02:57:02.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.919+0000 7f52e759e700 1 -- 172.21.15.121:0/3193420657 wait complete. 2023-11-15T02:57:02.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.919+0000 7f52e759e700 1 Processor -- start 2023-11-15T02:57:02.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.919+0000 7f52e759e700 1 -- start start 2023-11-15T02:57:02.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52e759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e80bcab0 0x7f52e81495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:02.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52e759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e81511f0 0x7f52e8154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:02.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52e759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52e8151ac0 0x7f52e80bf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:02.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52e759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e8163fe0 con 0x7f52e80bcab0 2023-11-15T02:57:02.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52e759e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e8164e00 con 0x7f52e8151ac0 2023-11-15T02:57:02.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52e759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e8149ae0 con 0x7f52e81511f0 2023-11-15T02:57:02.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52e659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e80bcab0 0x7f52e81495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:02.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52e659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e80bcab0 0x7f52e81495f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33280/0 (socket says 172.21.15.121:33280) 2023-11-15T02:57:02.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52e659c700 1 -- 172.21.15.121:0/2940969897 learned_addr learned my addr 172.21.15.121:0/2940969897 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:02.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52e5d9b700 1 --2- 172.21.15.121:0/2940969897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e81511f0 0x7f52e8154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:02.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52e6d9d700 1 --2- 172.21.15.121:0/2940969897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52e8151ac0 0x7f52e80bf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:02.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52e659c700 1 -- 172.21.15.121:0/2940969897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e81511f0 msgr2=0x7f52e8154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:02.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52e659c700 1 --2- 172.21.15.121:0/2940969897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e81511f0 0x7f52e8154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52e659c700 1 -- 172.21.15.121:0/2940969897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52e8151ac0 msgr2=0x7f52e80bf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:02.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52e659c700 1 --2- 172.21.15.121:0/2940969897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52e8151ac0 0x7f52e80bf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:02.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52e659c700 1 -- 172.21.15.121:0/2940969897 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d800bc30 con 0x7f52e80bcab0 2023-11-15T02:57:02.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52e659c700 1 --2- 172.21.15.121:0/2940969897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e80bcab0 0x7f52e81495f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f52d800d740 tx=0x7f52e8066050).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:02.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52de7fc700 1 -- 172.21.15.121:0/2940969897 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d8009280 con 0x7f52e80bcab0 2023-11-15T02:57:02.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52de7fc700 1 -- 172.21.15.121:0/2940969897 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d8009c90 con 0x7f52e80bcab0 2023-11-15T02:57:02.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52de7fc700 1 -- 172.21.15.121:0/2940969897 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d8015650 con 0x7f52e80bcab0 2023-11-15T02:57:02.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52e759e700 1 -- 172.21.15.121:0/2940969897 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e81641f0 con 0x7f52e80bcab0 2023-11-15T02:57:02.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.923+0000 7f52e759e700 1 -- 172.21.15.121:0/2940969897 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e81647b0 con 0x7f52e80bcab0 2023-11-15T02:57:02.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.927+0000 7f52e759e700 1 -- 172.21.15.121:0/2940969897 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e806bcf0 con 0x7f52e80bcab0 2023-11-15T02:57:02.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.927+0000 7f52de7fc700 1 -- 172.21.15.121:0/2940969897 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f52d8015a40 con 0x7f52e80bcab0 2023-11-15T02:57:02.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.927+0000 7f52de7fc700 1 --2- 172.21.15.121:0/2940969897 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f52c805a510 0x7f52c805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:02.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.927+0000 7f52de7fc700 1 -- 172.21.15.121:0/2940969897 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f52d8075770 con 0x7f52e80bcab0 2023-11-15T02:57:02.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.927+0000 7f52e5d9b700 1 --2- 172.21.15.121:0/2940969897 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f52c805a510 0x7f52c805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:02.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.931+0000 7f52e5d9b700 1 --2- 172.21.15.121:0/2940969897 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f52c805a510 0x7f52c805c9d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f52e80aac40 tx=0x7f52d0009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:02.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:02.935+0000 7f52de7fc700 1 -- 172.21.15.121:0/2940969897 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f52d8048e90 con 0x7f52e80bcab0 2023-11-15T02:57:03.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.095+0000 7f52e759e700 1 -- 172.21.15.121:0/2940969897 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f52e80c3ce0 con 0x7f52e80bcab0 2023-11-15T02:57:03.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.095+0000 7f52de7fc700 1 -- 172.21.15.121:0/2940969897 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f52d8023070 con 0x7f52e80bcab0 2023-11-15T02:57:03.101 INFO:teuthology.orchestra.run.smithi121.stdout:55834574854 2023-11-15T02:57:03.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.099+0000 7f52e759e700 1 -- 172.21.15.121:0/2940969897 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f52c805a510 msgr2=0x7f52c805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:03.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.099+0000 7f52e759e700 1 --2- 172.21.15.121:0/2940969897 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f52c805a510 0x7f52c805c9d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f52e80aac40 tx=0x7f52d0009060).stop 2023-11-15T02:57:03.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.099+0000 7f52e759e700 1 -- 172.21.15.121:0/2940969897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e80bcab0 msgr2=0x7f52e81495f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:03.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.099+0000 7f52e759e700 1 --2- 172.21.15.121:0/2940969897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e80bcab0 0x7f52e81495f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f52d800d740 tx=0x7f52e8066050).stop 2023-11-15T02:57:03.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.099+0000 7f52e759e700 1 -- 172.21.15.121:0/2940969897 shutdown_connections 2023-11-15T02:57:03.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.099+0000 7f52e759e700 1 --2- 172.21.15.121:0/2940969897 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f52c805a510 0x7f52c805c9d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.099+0000 7f52e759e700 1 --2- 172.21.15.121:0/2940969897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52e8151ac0 0x7f52e80bf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.099+0000 7f52e759e700 1 --2- 172.21.15.121:0/2940969897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e81511f0 0x7f52e8154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.099+0000 7f52e759e700 1 --2- 172.21.15.121:0/2940969897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e80bcab0 0x7f52e81495f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.099+0000 7f52e759e700 1 -- 172.21.15.121:0/2940969897 >> 172.21.15.121:0/2940969897 conn(0x7f52e8067a60 msgr2=0x7f52e814ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:03.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.099+0000 7f52e759e700 1 -- 172.21.15.121:0/2940969897 shutdown_connections 2023-11-15T02:57:03.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.099+0000 7f52e759e700 1 -- 172.21.15.121:0/2940969897 wait complete. 2023-11-15T02:57:03.126 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.5 2023-11-15T02:57:03.126 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-15T02:57:03.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.275+0000 7f8fc7107700 1 Processor -- start 2023-11-15T02:57:03.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.275+0000 7f8fc7107700 1 -- start start 2023-11-15T02:57:03.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.279+0000 7f8fc7107700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8fc00aed50 0x7f8fc015e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:03.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.279+0000 7f8fc7107700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc0066c90 con 0x7f8fc00ab0d0 2023-11-15T02:57:03.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.279+0000 7f8fc7107700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc0065e20 con 0x7f8fc00ad1b0 2023-11-15T02:57:03.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.279+0000 7f8fc7107700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc0066080 con 0x7f8fc00aed50 2023-11-15T02:57:03.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.279+0000 7f8fc6105700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f8fc00ab0d0 0x7f8fc00ab4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:57788/0 (socket says 172.21.15.121:57788) 2023-11-15T02:57:03.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.279+0000 7f8fc6105700 1 -- 172.21.15.121:0/581081511 learned_addr learned my addr 172.21.15.121:0/581081511 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:03.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.279+0000 7f8fc6906700 1 --2- 172.21.15.121:0/581081511 >> v2:172.21.15.190:3300/0 conn(0x7f8fc00aed50 0x7f8fc015e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:03.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.279+0000 7f8fc5103700 1 -- 172.21.15.121:0/581081511 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3797202337 0 0) 0x7f8fc0066c90 con 0x7f8fc00ab0d0 2023-11-15T02:57:03.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.279+0000 7f8fc5103700 1 -- 172.21.15.121:0/581081511 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fac0033f0 con 0x7f8fc00ab0d0 2023-11-15T02:57:03.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.279+0000 7f8fc6906700 1 -- 172.21.15.121:0/581081511 >> v1:172.21.15.190:6789/0 conn(0x7f8fc00ad1b0 legacy=0x7f8fc0065710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:03.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.279+0000 7f8fc6906700 1 -- 172.21.15.121:0/581081511 >> v1:172.21.15.121:6789/0 conn(0x7f8fc00ab0d0 legacy=0x7f8fc00ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:03.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.279+0000 7f8fc6906700 1 -- 172.21.15.121:0/581081511 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc0066360 con 0x7f8fc00aed50 2023-11-15T02:57:03.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.279+0000 7f8fc6906700 1 --2- 172.21.15.121:0/581081511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc00aed50 0x7f8fc015e740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f8fb8009e60 tx=0x7f8fb8009780).ready entity=mon.1 client_cookie=d2d73287111318c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:03.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.283+0000 7f8fc5103700 1 -- 172.21.15.121:0/581081511 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb800e070 con 0x7f8fc00aed50 2023-11-15T02:57:03.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.283+0000 7f8fc5103700 1 -- 172.21.15.121:0/581081511 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb800bb60 con 0x7f8fc00aed50 2023-11-15T02:57:03.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.283+0000 7f8fc5103700 1 -- 172.21.15.121:0/581081511 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8012410 con 0x7f8fc00aed50 2023-11-15T02:57:03.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.283+0000 7f8fc7107700 1 -- 172.21.15.121:0/581081511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc00aed50 msgr2=0x7f8fc015e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:03.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.283+0000 7f8fc7107700 1 --2- 172.21.15.121:0/581081511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc00aed50 0x7f8fc015e740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f8fb8009e60 tx=0x7f8fb8009780).stop 2023-11-15T02:57:03.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.283+0000 7f8fc7107700 1 -- 172.21.15.121:0/581081511 shutdown_connections 2023-11-15T02:57:03.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.283+0000 7f8fc7107700 1 --2- 172.21.15.121:0/581081511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc00aed50 0x7f8fc015e740 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.283+0000 7f8fc7107700 1 -- 172.21.15.121:0/581081511 >> 172.21.15.121:0/581081511 conn(0x7f8fc0067a60 msgr2=0x7f8fc0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:03.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.283+0000 7f8fc7107700 1 -- 172.21.15.121:0/581081511 shutdown_connections 2023-11-15T02:57:03.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.283+0000 7f8fc7107700 1 -- 172.21.15.121:0/581081511 wait complete. 2023-11-15T02:57:03.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc7107700 1 Processor -- start 2023-11-15T02:57:03.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc7107700 1 -- start start 2023-11-15T02:57:03.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc7107700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc00ab0d0 0x7f8fc00a8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:03.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc7107700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc00ad1b0 0x7f8fc0152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:03.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc7107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8fc00aed50 0x7f8fc00a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:03.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc7107700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc01640c0 con 0x7f8fc00ab0d0 2023-11-15T02:57:03.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc7107700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00c87a0 con 0x7f8fc00ad1b0 2023-11-15T02:57:03.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc7107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00c8a80 con 0x7f8fc00aed50 2023-11-15T02:57:03.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc6105700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc00ab0d0 0x7f8fc00a8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:03.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc6105700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc00ab0d0 0x7f8fc00a8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33288/0 (socket says 172.21.15.121:33288) 2023-11-15T02:57:03.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc6105700 1 -- 172.21.15.121:0/3880717297 learned_addr learned my addr 172.21.15.121:0/3880717297 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:03.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc6906700 1 --2- 172.21.15.121:0/3880717297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8fc00aed50 0x7f8fc00a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:03.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc5904700 1 --2- 172.21.15.121:0/3880717297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc00ad1b0 0x7f8fc0152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:03.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc6105700 1 -- 172.21.15.121:0/3880717297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8fc00aed50 msgr2=0x7f8fc00a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:03.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc6105700 1 --2- 172.21.15.121:0/3880717297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8fc00aed50 0x7f8fc00a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc6105700 1 -- 172.21.15.121:0/3880717297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc00ad1b0 msgr2=0x7f8fc0152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:03.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc6105700 1 --2- 172.21.15.121:0/3880717297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc00ad1b0 0x7f8fc0152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc6105700 1 -- 172.21.15.121:0/3880717297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb8009a20 con 0x7f8fc00ab0d0 2023-11-15T02:57:03.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc6105700 1 --2- 172.21.15.121:0/3880717297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc00ab0d0 0x7f8fc00a8980 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f8fbc00c4b0 tx=0x7f8fbc007b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:03.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fb67fc700 1 -- 172.21.15.121:0/3880717297 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0072e0 con 0x7f8fc00ab0d0 2023-11-15T02:57:03.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fb67fc700 1 -- 172.21.15.121:0/3880717297 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc007440 con 0x7f8fc00ab0d0 2023-11-15T02:57:03.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc7107700 1 -- 172.21.15.121:0/3880717297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc00a8e70 con 0x7f8fc00ab0d0 2023-11-15T02:57:03.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fb67fc700 1 -- 172.21.15.121:0/3880717297 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc005880 con 0x7f8fc00ab0d0 2023-11-15T02:57:03.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.287+0000 7f8fc7107700 1 -- 172.21.15.121:0/3880717297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc00a9450 con 0x7f8fc00ab0d0 2023-11-15T02:57:03.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.291+0000 7f8fc7107700 1 -- 172.21.15.121:0/3880717297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc00a97d0 con 0x7f8fc00ab0d0 2023-11-15T02:57:03.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.291+0000 7f8fb67fc700 1 -- 172.21.15.121:0/3880717297 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8fbc005c30 con 0x7f8fc00ab0d0 2023-11-15T02:57:03.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.291+0000 7f8fb67fc700 1 --2- 172.21.15.121:0/3880717297 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8fa405a510 0x7f8fa405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:03.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.291+0000 7f8fb67fc700 1 -- 172.21.15.121:0/3880717297 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8fbc074100 con 0x7f8fc00ab0d0 2023-11-15T02:57:03.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.295+0000 7f8fc5904700 1 --2- 172.21.15.121:0/3880717297 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8fa405a510 0x7f8fa405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:03.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.295+0000 7f8fc5904700 1 --2- 172.21.15.121:0/3880717297 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8fa405a510 0x7f8fa405c9d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f8fc00a5630 tx=0x7f8fb0009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:03.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.299+0000 7f8fb67fc700 1 -- 172.21.15.121:0/3880717297 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8fbc047820 con 0x7f8fc00ab0d0 2023-11-15T02:57:03.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.455+0000 7f8fc7107700 1 -- 172.21.15.121:0/3880717297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f8fc006bcf0 con 0x7f8fc00ab0d0 2023-11-15T02:57:03.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.459+0000 7f8fb67fc700 1 -- 172.21.15.121:0/3880717297 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8fbc04ae40 con 0x7f8fc00ab0d0 2023-11-15T02:57:03.464 INFO:teuthology.orchestra.run.smithi121.stdout:60129542149 2023-11-15T02:57:03.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.459+0000 7f8fc7107700 1 -- 172.21.15.121:0/3880717297 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8fa405a510 msgr2=0x7f8fa405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:03.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.459+0000 7f8fc7107700 1 --2- 172.21.15.121:0/3880717297 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8fa405a510 0x7f8fa405c9d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f8fc00a5630 tx=0x7f8fb0009040).stop 2023-11-15T02:57:03.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.463+0000 7f8fc7107700 1 -- 172.21.15.121:0/3880717297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc00ab0d0 msgr2=0x7f8fc00a8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:03.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.463+0000 7f8fc7107700 1 --2- 172.21.15.121:0/3880717297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc00ab0d0 0x7f8fc00a8980 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f8fbc00c4b0 tx=0x7f8fbc007b00).stop 2023-11-15T02:57:03.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.463+0000 7f8fc7107700 1 -- 172.21.15.121:0/3880717297 shutdown_connections 2023-11-15T02:57:03.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.463+0000 7f8fc7107700 1 --2- 172.21.15.121:0/3880717297 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8fa405a510 0x7f8fa405c9d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.463+0000 7f8fc7107700 1 --2- 172.21.15.121:0/3880717297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8fc00aed50 0x7f8fc00a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.463+0000 7f8fc7107700 1 --2- 172.21.15.121:0/3880717297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc00ad1b0 0x7f8fc0152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.463+0000 7f8fc7107700 1 --2- 172.21.15.121:0/3880717297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc00ab0d0 0x7f8fc00a8980 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.463+0000 7f8fc7107700 1 -- 172.21.15.121:0/3880717297 >> 172.21.15.121:0/3880717297 conn(0x7f8fc0067a60 msgr2=0x7f8fc0153410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:03.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.463+0000 7f8fc7107700 1 -- 172.21.15.121:0/3880717297 shutdown_connections 2023-11-15T02:57:03.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.463+0000 7f8fc7107700 1 -- 172.21.15.121:0/3880717297 wait complete. 2023-11-15T02:57:03.485 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.6 2023-11-15T02:57:03.485 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-15T02:57:03.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.651+0000 7f8275639700 1 Processor -- start 2023-11-15T02:57:03.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.651+0000 7f8275639700 1 -- start start 2023-11-15T02:57:03.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.655+0000 7f8275639700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8270066570 0x7f82701515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:03.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.655+0000 7f8275639700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8270151ac0 0x7f82700a8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:03.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.655+0000 7f8275639700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82700660e0 con 0x7f82701511f0 2023-11-15T02:57:03.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.655+0000 7f8275639700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82700669f0 con 0x7f8270151ac0 2023-11-15T02:57:03.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.655+0000 7f8275639700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82700bd390 con 0x7f8270066570 2023-11-15T02:57:03.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.655+0000 7f8274e38700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8270151ac0 0x7f82700a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:03.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.655+0000 7f826f7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8270066570 0x7f82701515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:03.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.655+0000 7f8274e38700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8270151ac0 0x7f82700a8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:43476/0 (socket says 172.21.15.121:43476) 2023-11-15T02:57:03.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.655+0000 7f8274e38700 1 -- 172.21.15.121:0/4029494057 learned_addr learned my addr 172.21.15.121:0/4029494057 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:03.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.655+0000 7f826ffff700 1 --1- 172.21.15.121:0/4029494057 >> v1:172.21.15.190:6789/0 conn(0x7f82701511f0 0x7f82700bcab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:59400/0 (socket says 172.21.15.121:59400) 2023-11-15T02:57:03.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.655+0000 7f826effd700 1 -- 172.21.15.121:0/4029494057 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260686562 0 0) 0x7f82700660e0 con 0x7f82701511f0 2023-11-15T02:57:03.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.655+0000 7f826effd700 1 -- 172.21.15.121:0/4029494057 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82600033f0 con 0x7f82701511f0 2023-11-15T02:57:03.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.655+0000 7f8274e38700 1 -- 172.21.15.121:0/4029494057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8270066570 msgr2=0x7f82701515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:03.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.655+0000 7f8274e38700 1 --2- 172.21.15.121:0/4029494057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8270066570 0x7f82701515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.655+0000 7f8274e38700 1 -- 172.21.15.121:0/4029494057 >> v1:172.21.15.190:6789/0 conn(0x7f82701511f0 legacy=0x7f82700bcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:03.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.655+0000 7f8274e38700 1 -- 172.21.15.121:0/4029494057 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82700657b0 con 0x7f8270151ac0 2023-11-15T02:57:03.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.655+0000 7f826f7fe700 1 --2- 172.21.15.121:0/4029494057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8270066570 0x7f82701515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:57:03.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.655+0000 7f8274e38700 1 --2- 172.21.15.121:0/4029494057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8270151ac0 0x7f82700a8750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f8268009f20 tx=0x7f8268009840).ready entity=mon.1 client_cookie=a6481a03f62a48e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:03.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.659+0000 7f826effd700 1 -- 172.21.15.121:0/4029494057 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826800e070 con 0x7f8270151ac0 2023-11-15T02:57:03.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.659+0000 7f826effd700 1 -- 172.21.15.121:0/4029494057 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826800b970 con 0x7f8270151ac0 2023-11-15T02:57:03.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.659+0000 7f826effd700 1 -- 172.21.15.121:0/4029494057 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826800bcf0 con 0x7f8270151ac0 2023-11-15T02:57:03.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.659+0000 7f8275639700 1 -- 172.21.15.121:0/4029494057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8270151ac0 msgr2=0x7f82700a8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:03.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.659+0000 7f8275639700 1 --2- 172.21.15.121:0/4029494057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8270151ac0 0x7f82700a8750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f8268009f20 tx=0x7f8268009840).stop 2023-11-15T02:57:03.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.659+0000 7f8275639700 1 -- 172.21.15.121:0/4029494057 shutdown_connections 2023-11-15T02:57:03.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.659+0000 7f8275639700 1 --2- 172.21.15.121:0/4029494057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8270151ac0 0x7f82700a8750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.659+0000 7f8275639700 1 --2- 172.21.15.121:0/4029494057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8270066570 0x7f82701515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.659+0000 7f8275639700 1 -- 172.21.15.121:0/4029494057 >> 172.21.15.121:0/4029494057 conn(0x7f8270067a60 msgr2=0x7f8270067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:03.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.659+0000 7f8275639700 1 -- 172.21.15.121:0/4029494057 shutdown_connections 2023-11-15T02:57:03.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.659+0000 7f8275639700 1 -- 172.21.15.121:0/4029494057 wait complete. 2023-11-15T02:57:03.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.659+0000 7f8275639700 1 Processor -- start 2023-11-15T02:57:03.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f8275639700 1 -- start start 2023-11-15T02:57:03.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f8275639700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8270066570 0x7f8270166fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:03.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f8275639700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82701511f0 0x7f827014f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:03.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f8275639700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8270151ac0 0x7f82700c1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:03.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f8275639700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8270164ea0 con 0x7f8270151ac0 2023-11-15T02:57:03.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f8275639700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8270164570 con 0x7f8270066570 2023-11-15T02:57:03.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f8275639700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8270164850 con 0x7f82701511f0 2023-11-15T02:57:03.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f8274e38700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8270151ac0 0x7f82700c1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:03.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f8274e38700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8270151ac0 0x7f82700c1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33304/0 (socket says 172.21.15.121:33304) 2023-11-15T02:57:03.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f8274e38700 1 -- 172.21.15.121:0/1479239675 learned_addr learned my addr 172.21.15.121:0/1479239675 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:03.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f826f7fe700 1 --2- 172.21.15.121:0/1479239675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82701511f0 0x7f827014f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:03.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f826ffff700 1 --2- 172.21.15.121:0/1479239675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8270066570 0x7f8270166fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:03.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f826f7fe700 1 -- 172.21.15.121:0/1479239675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8270066570 msgr2=0x7f8270166fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:03.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f826f7fe700 1 --2- 172.21.15.121:0/1479239675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8270066570 0x7f8270166fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f826f7fe700 1 -- 172.21.15.121:0/1479239675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8270151ac0 msgr2=0x7f82700c1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:03.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f826f7fe700 1 --2- 172.21.15.121:0/1479239675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8270151ac0 0x7f82700c1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f826f7fe700 1 -- 172.21.15.121:0/1479239675 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c009250 con 0x7f82701511f0 2023-11-15T02:57:03.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f826f7fe700 1 --2- 172.21.15.121:0/1479239675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82701511f0 0x7f827014f450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f825c009fd0 tx=0x7f825c006f40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:03.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f825bfff700 1 -- 172.21.15.121:0/1479239675 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c005880 con 0x7f82701511f0 2023-11-15T02:57:03.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f825bfff700 1 -- 172.21.15.121:0/1479239675 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c0059e0 con 0x7f82701511f0 2023-11-15T02:57:03.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f8275639700 1 -- 172.21.15.121:0/1479239675 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8268009ae0 con 0x7f82701511f0 2023-11-15T02:57:03.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.663+0000 7f825bfff700 1 -- 172.21.15.121:0/1479239675 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c005d60 con 0x7f82701511f0 2023-11-15T02:57:03.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.667+0000 7f8275639700 1 -- 172.21.15.121:0/1479239675 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8270164260 con 0x7f82701511f0 2023-11-15T02:57:03.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.667+0000 7f8275639700 1 -- 172.21.15.121:0/1479239675 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8270155280 con 0x7f82701511f0 2023-11-15T02:57:03.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.667+0000 7f825bfff700 1 -- 172.21.15.121:0/1479239675 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f825c006230 con 0x7f82701511f0 2023-11-15T02:57:03.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.671+0000 7f825bfff700 1 --2- 172.21.15.121:0/1479239675 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f825405a510 0x7f825405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:03.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.671+0000 7f825bfff700 1 -- 172.21.15.121:0/1479239675 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f825c074900 con 0x7f82701511f0 2023-11-15T02:57:03.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.671+0000 7f826ffff700 1 --2- 172.21.15.121:0/1479239675 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f825405a510 0x7f825405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:03.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.671+0000 7f826ffff700 1 --2- 172.21.15.121:0/1479239675 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f825405a510 0x7f825405c9d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f8264007640 tx=0x7f8264007bc0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:03.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.679+0000 7f825bfff700 1 -- 172.21.15.121:0/1479239675 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f825c048020 con 0x7f82701511f0 2023-11-15T02:57:03.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.843+0000 7f8275639700 1 -- 172.21.15.121:0/1479239675 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f827014c6b0 con 0x7f82701511f0 2023-11-15T02:57:03.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.847+0000 7f825bfff700 1 -- 172.21.15.121:0/1479239675 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f825c016020 con 0x7f82701511f0 2023-11-15T02:57:03.852 INFO:teuthology.orchestra.run.smithi121.stdout:60129542149 2023-11-15T02:57:03.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.847+0000 7f8275639700 1 -- 172.21.15.121:0/1479239675 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f825405a510 msgr2=0x7f825405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:03.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.847+0000 7f8275639700 1 --2- 172.21.15.121:0/1479239675 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f825405a510 0x7f825405c9d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f8264007640 tx=0x7f8264007bc0).stop 2023-11-15T02:57:03.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.847+0000 7f8275639700 1 -- 172.21.15.121:0/1479239675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82701511f0 msgr2=0x7f827014f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:03.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.847+0000 7f8275639700 1 --2- 172.21.15.121:0/1479239675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82701511f0 0x7f827014f450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f825c009fd0 tx=0x7f825c006f40).stop 2023-11-15T02:57:03.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.847+0000 7f8275639700 1 -- 172.21.15.121:0/1479239675 shutdown_connections 2023-11-15T02:57:03.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.847+0000 7f8275639700 1 --2- 172.21.15.121:0/1479239675 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f825405a510 0x7f825405c9d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.847+0000 7f8275639700 1 --2- 172.21.15.121:0/1479239675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8270151ac0 0x7f82700c1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.847+0000 7f8275639700 1 --2- 172.21.15.121:0/1479239675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82701511f0 0x7f827014f450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.847+0000 7f8275639700 1 --2- 172.21.15.121:0/1479239675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8270066570 0x7f8270166fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:03.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.847+0000 7f8275639700 1 -- 172.21.15.121:0/1479239675 >> 172.21.15.121:0/1479239675 conn(0x7f8270067a60 msgr2=0x7f82700c5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:03.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.851+0000 7f8275639700 1 -- 172.21.15.121:0/1479239675 shutdown_connections 2023-11-15T02:57:03.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:03.851+0000 7f8275639700 1 -- 172.21.15.121:0/1479239675 wait complete. 2023-11-15T02:57:03.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.7 2023-11-15T02:57:03.873 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-15T02:57:03.874 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T02:57:04.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.039+0000 7fabc359e700 1 Processor -- start 2023-11-15T02:57:04.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.043+0000 7fabc359e700 1 -- start start 2023-11-15T02:57:04.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.043+0000 7fabc359e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fabc40a9830 0x7fabc40a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:04.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.043+0000 7fabc359e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fabc40c30e0 0x7fabc40c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:04.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.043+0000 7fabc359e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc40657f0 con 0x7fabc4151870 2023-11-15T02:57:04.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.043+0000 7fabc359e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc4065ad0 con 0x7fabc40a9830 2023-11-15T02:57:04.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.043+0000 7fabc359e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc40c3a60 con 0x7fabc40c30e0 2023-11-15T02:57:04.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.043+0000 7fabc1d9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fabc40a9830 0x7fabc40a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:04.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.043+0000 7fabc259c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fabc4151870 0x7fabc4151c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:57798/0 (socket says 172.21.15.121:57798) 2023-11-15T02:57:04.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.043+0000 7fabc1d9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fabc40a9830 0x7fabc40a9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33314/0 (socket says 172.21.15.121:33314) 2023-11-15T02:57:04.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.043+0000 7fabc259c700 1 -- 172.21.15.121:0/1913190430 learned_addr learned my addr 172.21.15.121:0/1913190430 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:04.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.043+0000 7fabc2d9d700 1 --2- 172.21.15.121:0/1913190430 >> v2:172.21.15.190:3300/0 conn(0x7fabc40c30e0 0x7fabc40c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:04.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.047+0000 7fabc1d9b700 1 -- 172.21.15.121:0/1913190430 >> v2:172.21.15.190:3300/0 conn(0x7fabc40c30e0 msgr2=0x7fabc40c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:04.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.047+0000 7fabc1d9b700 1 --2- 172.21.15.121:0/1913190430 >> v2:172.21.15.190:3300/0 conn(0x7fabc40c30e0 0x7fabc40c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.047+0000 7fabc1d9b700 1 -- 172.21.15.121:0/1913190430 >> v1:172.21.15.121:6789/0 conn(0x7fabc4151870 legacy=0x7fabc4151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:04.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.047+0000 7fabc1d9b700 1 -- 172.21.15.121:0/1913190430 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc40aa260 con 0x7fabc40a9830 2023-11-15T02:57:04.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.047+0000 7fabc1d9b700 1 --2- 172.21.15.121:0/1913190430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc40a9830 0x7fabc40a9c70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fabc4069a90 tx=0x7fabb8009060).ready entity=mon.0 client_cookie=124084fd4ab47e27 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:04.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.047+0000 7fabc159a700 1 -- 172.21.15.121:0/1913190430 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb800b580 con 0x7fabc40a9830 2023-11-15T02:57:04.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.047+0000 7fabc159a700 1 -- 172.21.15.121:0/1913190430 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb800b6e0 con 0x7fabc40a9830 2023-11-15T02:57:04.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.047+0000 7fabc159a700 1 -- 172.21.15.121:0/1913190430 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb800ba60 con 0x7fabc40a9830 2023-11-15T02:57:04.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.047+0000 7fabc359e700 1 -- 172.21.15.121:0/1913190430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc40a9830 msgr2=0x7fabc40a9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:04.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.047+0000 7fabc359e700 1 --2- 172.21.15.121:0/1913190430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc40a9830 0x7fabc40a9c70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fabc4069a90 tx=0x7fabb8009060).stop 2023-11-15T02:57:04.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.047+0000 7fabc359e700 1 -- 172.21.15.121:0/1913190430 shutdown_connections 2023-11-15T02:57:04.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.047+0000 7fabc359e700 1 --2- 172.21.15.121:0/1913190430 >> v2:172.21.15.190:3300/0 conn(0x7fabc40c30e0 0x7fabc40c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.047+0000 7fabc359e700 1 --2- 172.21.15.121:0/1913190430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc40a9830 0x7fabc40a9c70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.047+0000 7fabc359e700 1 -- 172.21.15.121:0/1913190430 >> 172.21.15.121:0/1913190430 conn(0x7fabc4062190 msgr2=0x7fabc4062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:04.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.047+0000 7fabc359e700 1 -- 172.21.15.121:0/1913190430 shutdown_connections 2023-11-15T02:57:04.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.047+0000 7fabc359e700 1 -- 172.21.15.121:0/1913190430 wait complete. 2023-11-15T02:57:04.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.051+0000 7fabc359e700 1 Processor -- start 2023-11-15T02:57:04.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.051+0000 7fabc359e700 1 -- start start 2023-11-15T02:57:04.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.051+0000 7fabc359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc40a9830 0x7fabc4156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:04.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.051+0000 7fabc359e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc40c30e0 0x7fabc4145560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:04.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.051+0000 7fabc359e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc4151870 0x7fabc41544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:04.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.051+0000 7fabc359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc4156620 con 0x7fabc40a9830 2023-11-15T02:57:04.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.051+0000 7fabc359e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc4145a50 con 0x7fabc4151870 2023-11-15T02:57:04.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.051+0000 7fabc359e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc4156040 con 0x7fabc40c30e0 2023-11-15T02:57:04.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.051+0000 7fabc259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc40a9830 0x7fabc4156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:04.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.051+0000 7fabc259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc40a9830 0x7fabc4156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33316/0 (socket says 172.21.15.121:33316) 2023-11-15T02:57:04.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.051+0000 7fabc259c700 1 -- 172.21.15.121:0/1031127732 learned_addr learned my addr 172.21.15.121:0/1031127732 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:04.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.055+0000 7fabc2d9d700 1 --2- 172.21.15.121:0/1031127732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc4151870 0x7fabc41544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:04.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.055+0000 7fabc1d9b700 1 --2- 172.21.15.121:0/1031127732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc40c30e0 0x7fabc4145560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:04.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.055+0000 7fabc259c700 1 -- 172.21.15.121:0/1031127732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc40c30e0 msgr2=0x7fabc4145560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:04.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.055+0000 7fabc259c700 1 --2- 172.21.15.121:0/1031127732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc40c30e0 0x7fabc4145560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.055+0000 7fabc259c700 1 -- 172.21.15.121:0/1031127732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc4151870 msgr2=0x7fabc41544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:04.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.055+0000 7fabc259c700 1 --2- 172.21.15.121:0/1031127732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc4151870 0x7fabc41544b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.055+0000 7fabc259c700 1 -- 172.21.15.121:0/1031127732 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb80092a0 con 0x7fabc40a9830 2023-11-15T02:57:04.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.055+0000 7fabc259c700 1 --2- 172.21.15.121:0/1031127732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc40a9830 0x7fabc4156ab0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fabb400c410 tx=0x7fabb4007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:04.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.055+0000 7fabb27fc700 1 -- 172.21.15.121:0/1031127732 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb40070d0 con 0x7fabc40a9830 2023-11-15T02:57:04.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.055+0000 7fabb27fc700 1 -- 172.21.15.121:0/1031127732 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb4007230 con 0x7fabc40a9830 2023-11-15T02:57:04.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.055+0000 7fabc359e700 1 -- 172.21.15.121:0/1031127732 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc4147ee0 con 0x7fabc40a9830 2023-11-15T02:57:04.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.055+0000 7fabb27fc700 1 -- 172.21.15.121:0/1031127732 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb40075d0 con 0x7fabc40a9830 2023-11-15T02:57:04.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.055+0000 7fabc359e700 1 -- 172.21.15.121:0/1031127732 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc41562b0 con 0x7fabc40a9830 2023-11-15T02:57:04.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.055+0000 7fabc359e700 1 -- 172.21.15.121:0/1031127732 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc406bcf0 con 0x7fabc40a9830 2023-11-15T02:57:04.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.059+0000 7fabb27fc700 1 -- 172.21.15.121:0/1031127732 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fabb40059b0 con 0x7fabc40a9830 2023-11-15T02:57:04.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.059+0000 7fabb27fc700 1 --2- 172.21.15.121:0/1031127732 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faba405a510 0x7faba405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:04.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.059+0000 7fabb27fc700 1 -- 172.21.15.121:0/1031127732 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fabb4073e40 con 0x7fabc40a9830 2023-11-15T02:57:04.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.059+0000 7fabc1d9b700 1 --2- 172.21.15.121:0/1031127732 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faba405a510 0x7faba405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:04.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.059+0000 7fabc1d9b700 1 --2- 172.21.15.121:0/1031127732 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faba405a510 0x7faba405c9d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fabb8009fd0 tx=0x7fabb8017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:04.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.067+0000 7fabb27fc700 1 -- 172.21.15.121:0/1031127732 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fabb4047560 con 0x7fabc40a9830 2023-11-15T02:57:04.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.227+0000 7fabc359e700 1 -- 172.21.15.121:0/1031127732 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fabc4068a70 con 0x7faba405a510 2023-11-15T02:57:04.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.243+0000 7fabb27fc700 1 -- 172.21.15.121:0/1031127732 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336066 (secure 0 0 0) 0x7fabc4068a70 con 0x7faba405a510 2023-11-15T02:57:04.253 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:04.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.247+0000 7fabc359e700 1 -- 172.21.15.121:0/1031127732 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faba405a510 msgr2=0x7faba405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:04.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.247+0000 7fabc359e700 1 --2- 172.21.15.121:0/1031127732 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faba405a510 0x7faba405c9d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fabb8009fd0 tx=0x7fabb8017000).stop 2023-11-15T02:57:04.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.251+0000 7fabc359e700 1 -- 172.21.15.121:0/1031127732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc40a9830 msgr2=0x7fabc4156ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:04.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.251+0000 7fabc359e700 1 --2- 172.21.15.121:0/1031127732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc40a9830 0x7fabc4156ab0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fabb400c410 tx=0x7fabb4007e50).stop 2023-11-15T02:57:04.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.251+0000 7fabc359e700 1 -- 172.21.15.121:0/1031127732 shutdown_connections 2023-11-15T02:57:04.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.251+0000 7fabc359e700 1 --2- 172.21.15.121:0/1031127732 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faba405a510 0x7faba405c9d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.251+0000 7fabc359e700 1 --2- 172.21.15.121:0/1031127732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc4151870 0x7fabc41544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.251+0000 7fabc359e700 1 --2- 172.21.15.121:0/1031127732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc40c30e0 0x7fabc4145560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.251+0000 7fabc359e700 1 --2- 172.21.15.121:0/1031127732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc40a9830 0x7fabc4156ab0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.251+0000 7fabc359e700 1 -- 172.21.15.121:0/1031127732 >> 172.21.15.121:0/1031127732 conn(0x7fabc4062190 msgr2=0x7fabc40a8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:04.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.251+0000 7fabc359e700 1 -- 172.21.15.121:0/1031127732 shutdown_connections 2023-11-15T02:57:04.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.251+0000 7fabc359e700 1 -- 172.21.15.121:0/1031127732 wait complete. 2023-11-15T02:57:04.258 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T02:57:04.275 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-15T02:57:03.916329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2378492,"kb_used_data":1724,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752596228,"statfs":{"total":773094113280,"available":770658537472,"internally_reserved":0,"allocated":1765376,"data_stored":635719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003190"},"pg_stats":[{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.736895+0000","last_change":"2023-11-15T02:56:51.733024+0000","last_active":"2023-11-15T02:56:51.736895+0000","last_peered":"2023-11-15T02:56:51.736895+0000","last_clean":"2023-11-15T02:56:51.736895+0000","last_became_active":"2023-11-15T02:56:51.732735+0000","last_became_peered":"2023-11-15T02:56:51.732735+0000","last_unstale":"2023-11-15T02:56:51.736895+0000","last_undegraded":"2023-11-15T02:56:51.736895+0000","last_fullsized":"2023-11-15T02:56:51.736895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.704467+0000","last_change":"2023-11-15T02:56:51.273328+0000","last_active":"2023-11-15T02:56:51.704467+0000","last_peered":"2023-11-15T02:56:51.704467+0000","last_clean":"2023-11-15T02:56:51.704467+0000","last_became_active":"2023-11-15T02:56:51.273044+0000","last_became_peered":"2023-11-15T02:56:51.273044+0000","last_unstale":"2023-11-15T02:56:51.704467+0000","last_undegraded":"2023-11-15T02:56:51.704467+0000","last_fullsized":"2023-11-15T02:56:51.704467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744601+0000","last_change":"2023-11-15T02:56:51.742242+0000","last_active":"2023-11-15T02:56:51.744601+0000","last_peered":"2023-11-15T02:56:51.744601+0000","last_clean":"2023-11-15T02:56:51.744601+0000","last_became_active":"2023-11-15T02:56:51.741604+0000","last_became_peered":"2023-11-15T02:56:51.741604+0000","last_unstale":"2023-11-15T02:56:51.744601+0000","last_undegraded":"2023-11-15T02:56:51.744601+0000","last_fullsized":"2023-11-15T02:56:51.744601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.739512+0000","last_change":"2023-11-15T02:56:51.277577+0000","last_active":"2023-11-15T02:56:51.739512+0000","last_peered":"2023-11-15T02:56:51.739512+0000","last_clean":"2023-11-15T02:56:51.739512+0000","last_became_active":"2023-11-15T02:56:51.276745+0000","last_became_peered":"2023-11-15T02:56:51.276745+0000","last_unstale":"2023-11-15T02:56:51.739512+0000","last_undegraded":"2023-11-15T02:56:51.739512+0000","last_fullsized":"2023-11-15T02:56:51.739512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.716112+0000","last_change":"2023-11-15T02:56:51.269238+0000","last_active":"2023-11-15T02:56:51.716112+0000","last_peered":"2023-11-15T02:56:51.716112+0000","last_clean":"2023-11-15T02:56:51.716112+0000","last_became_active":"2023-11-15T02:56:51.268872+0000","last_became_peered":"2023-11-15T02:56:51.268872+0000","last_unstale":"2023-11-15T02:56:51.716112+0000","last_undegraded":"2023-11-15T02:56:51.716112+0000","last_fullsized":"2023-11-15T02:56:51.716112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747275+0000","last_change":"2023-11-15T02:56:51.742674+0000","last_active":"2023-11-15T02:56:51.747275+0000","last_peered":"2023-11-15T02:56:51.747275+0000","last_clean":"2023-11-15T02:56:51.747275+0000","last_became_active":"2023-11-15T02:56:51.742412+0000","last_became_peered":"2023-11-15T02:56:51.742412+0000","last_unstale":"2023-11-15T02:56:51.747275+0000","last_undegraded":"2023-11-15T02:56:51.747275+0000","last_fullsized":"2023-11-15T02:56:51.747275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712980+0000","last_change":"2023-11-15T02:56:51.279772+0000","last_active":"2023-11-15T02:56:51.712980+0000","last_peered":"2023-11-15T02:56:51.712980+0000","last_clean":"2023-11-15T02:56:51.712980+0000","last_became_active":"2023-11-15T02:56:51.279423+0000","last_became_peered":"2023-11-15T02:56:51.279423+0000","last_unstale":"2023-11-15T02:56:51.712980+0000","last_undegraded":"2023-11-15T02:56:51.712980+0000","last_fullsized":"2023-11-15T02:56:51.712980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738160+0000","last_change":"2023-11-15T02:56:51.731913+0000","last_active":"2023-11-15T02:56:51.738160+0000","last_peered":"2023-11-15T02:56:51.738160+0000","last_clean":"2023-11-15T02:56:51.738160+0000","last_became_active":"2023-11-15T02:56:51.728632+0000","last_became_peered":"2023-11-15T02:56:51.728632+0000","last_unstale":"2023-11-15T02:56:51.738160+0000","last_undegraded":"2023-11-15T02:56:51.738160+0000","last_fullsized":"2023-11-15T02:56:51.738160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744972+0000","last_change":"2023-11-15T02:56:51.289877+0000","last_active":"2023-11-15T02:56:51.744972+0000","last_peered":"2023-11-15T02:56:51.744972+0000","last_clean":"2023-11-15T02:56:51.744972+0000","last_became_active":"2023-11-15T02:56:51.288940+0000","last_became_peered":"2023-11-15T02:56:51.288940+0000","last_unstale":"2023-11-15T02:56:51.744972+0000","last_undegraded":"2023-11-15T02:56:51.744972+0000","last_fullsized":"2023-11-15T02:56:51.744972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.743484+0000","last_change":"2023-11-15T02:56:51.738869+0000","last_active":"2023-11-15T02:56:51.743484+0000","last_peered":"2023-11-15T02:56:51.743484+0000","last_clean":"2023-11-15T02:56:51.743484+0000","last_became_active":"2023-11-15T02:56:51.738159+0000","last_became_peered":"2023-11-15T02:56:51.738159+0000","last_unstale":"2023-11-15T02:56:51.743484+0000","last_undegraded":"2023-11-15T02:56:51.743484+0000","last_fullsized":"2023-11-15T02:56:51.743484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713144+0000","last_change":"2023-11-15T02:56:51.284304+0000","last_active":"2023-11-15T02:56:51.713144+0000","last_peered":"2023-11-15T02:56:51.713144+0000","last_clean":"2023-11-15T02:56:51.713144+0000","last_became_active":"2023-11-15T02:56:51.284043+0000","last_became_peered":"2023-11-15T02:56:51.284043+0000","last_unstale":"2023-11-15T02:56:51.713144+0000","last_undegraded":"2023-11-15T02:56:51.713144+0000","last_fullsized":"2023-11-15T02:56:51.713144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747361+0000","last_change":"2023-11-15T02:56:51.744243+0000","last_active":"2023-11-15T02:56:51.747361+0000","last_peered":"2023-11-15T02:56:51.747361+0000","last_clean":"2023-11-15T02:56:51.747361+0000","last_became_active":"2023-11-15T02:56:51.743943+0000","last_became_peered":"2023-11-15T02:56:51.743943+0000","last_unstale":"2023-11-15T02:56:51.747361+0000","last_undegraded":"2023-11-15T02:56:51.747361+0000","last_fullsized":"2023-11-15T02:56:51.747361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.734769+0000","last_change":"2023-11-15T02:56:51.286898+0000","last_active":"2023-11-15T02:56:51.734769+0000","last_peered":"2023-11-15T02:56:51.734769+0000","last_clean":"2023-11-15T02:56:51.734769+0000","last_became_active":"2023-11-15T02:56:51.286628+0000","last_became_peered":"2023-11-15T02:56:51.286628+0000","last_unstale":"2023-11-15T02:56:51.734769+0000","last_undegraded":"2023-11-15T02:56:51.734769+0000","last_fullsized":"2023-11-15T02:56:51.734769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.749549+0000","last_change":"2023-11-15T02:56:51.745783+0000","last_active":"2023-11-15T02:56:51.749549+0000","last_peered":"2023-11-15T02:56:51.749549+0000","last_clean":"2023-11-15T02:56:51.749549+0000","last_became_active":"2023-11-15T02:56:51.745028+0000","last_became_peered":"2023-11-15T02:56:51.745028+0000","last_unstale":"2023-11-15T02:56:51.749549+0000","last_undegraded":"2023-11-15T02:56:51.749549+0000","last_fullsized":"2023-11-15T02:56:51.749549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.716637+0000","last_change":"2023-11-15T02:56:51.268012+0000","last_active":"2023-11-15T02:56:51.716637+0000","last_peered":"2023-11-15T02:56:51.716637+0000","last_clean":"2023-11-15T02:56:51.716637+0000","last_became_active":"2023-11-15T02:56:51.267734+0000","last_became_peered":"2023-11-15T02:56:51.267734+0000","last_unstale":"2023-11-15T02:56:51.716637+0000","last_undegraded":"2023-11-15T02:56:51.716637+0000","last_fullsized":"2023-11-15T02:56:51.716637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730642+0000","last_change":"2023-11-15T02:56:51.720753+0000","last_active":"2023-11-15T02:56:51.730642+0000","last_peered":"2023-11-15T02:56:51.730642+0000","last_clean":"2023-11-15T02:56:51.730642+0000","last_became_active":"2023-11-15T02:56:51.720500+0000","last_became_peered":"2023-11-15T02:56:51.720500+0000","last_unstale":"2023-11-15T02:56:51.730642+0000","last_undegraded":"2023-11-15T02:56:51.730642+0000","last_fullsized":"2023-11-15T02:56:51.730642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.711624+0000","last_change":"2023-11-15T02:56:51.280573+0000","last_active":"2023-11-15T02:56:51.711624+0000","last_peered":"2023-11-15T02:56:51.711624+0000","last_clean":"2023-11-15T02:56:51.711624+0000","last_became_active":"2023-11-15T02:56:51.275286+0000","last_became_peered":"2023-11-15T02:56:51.275286+0000","last_unstale":"2023-11-15T02:56:51.711624+0000","last_undegraded":"2023-11-15T02:56:51.711624+0000","last_fullsized":"2023-11-15T02:56:51.711624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.742388+0000","last_change":"2023-11-15T02:56:51.736450+0000","last_active":"2023-11-15T02:56:51.742388+0000","last_peered":"2023-11-15T02:56:51.742388+0000","last_clean":"2023-11-15T02:56:51.742388+0000","last_became_active":"2023-11-15T02:56:51.736143+0000","last_became_peered":"2023-11-15T02:56:51.736143+0000","last_unstale":"2023-11-15T02:56:51.742388+0000","last_undegraded":"2023-11-15T02:56:51.742388+0000","last_fullsized":"2023-11-15T02:56:51.742388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.714425+0000","last_change":"2023-11-15T02:56:51.284434+0000","last_active":"2023-11-15T02:56:51.714425+0000","last_peered":"2023-11-15T02:56:51.714425+0000","last_clean":"2023-11-15T02:56:51.714425+0000","last_became_active":"2023-11-15T02:56:51.284051+0000","last_became_peered":"2023-11-15T02:56:51.284051+0000","last_unstale":"2023-11-15T02:56:51.714425+0000","last_undegraded":"2023-11-15T02:56:51.714425+0000","last_fullsized":"2023-11-15T02:56:51.714425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744584+0000","last_change":"2023-11-15T02:56:51.742247+0000","last_active":"2023-11-15T02:56:51.744584+0000","last_peered":"2023-11-15T02:56:51.744584+0000","last_clean":"2023-11-15T02:56:51.744584+0000","last_became_active":"2023-11-15T02:56:51.741966+0000","last_became_peered":"2023-11-15T02:56:51.741966+0000","last_unstale":"2023-11-15T02:56:51.744584+0000","last_undegraded":"2023-11-15T02:56:51.744584+0000","last_fullsized":"2023-11-15T02:56:51.744584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.714493+0000","last_change":"2023-11-15T02:56:51.277678+0000","last_active":"2023-11-15T02:56:51.714493+0000","last_peered":"2023-11-15T02:56:51.714493+0000","last_clean":"2023-11-15T02:56:51.714493+0000","last_became_active":"2023-11-15T02:56:51.277408+0000","last_became_peered":"2023-11-15T02:56:51.277408+0000","last_unstale":"2023-11-15T02:56:51.714493+0000","last_undegraded":"2023-11-15T02:56:51.714493+0000","last_fullsized":"2023-11-15T02:56:51.714493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.731195+0000","last_change":"2023-11-15T02:56:51.717008+0000","last_active":"2023-11-15T02:56:51.731195+0000","last_peered":"2023-11-15T02:56:51.731195+0000","last_clean":"2023-11-15T02:56:51.731195+0000","last_became_active":"2023-11-15T02:56:51.716661+0000","last_became_peered":"2023-11-15T02:56:51.716661+0000","last_unstale":"2023-11-15T02:56:51.731195+0000","last_undegraded":"2023-11-15T02:56:51.731195+0000","last_fullsized":"2023-11-15T02:56:51.731195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-15T02:56:53.093563+0000","last_change":"2023-11-15T02:56:51.281739+0000","last_active":"2023-11-15T02:56:53.093563+0000","last_peered":"2023-11-15T02:56:53.093563+0000","last_clean":"2023-11-15T02:56:53.093563+0000","last_became_active":"2023-11-15T02:56:51.281484+0000","last_became_peered":"2023-11-15T02:56:51.281484+0000","last_unstale":"2023-11-15T02:56:53.093563+0000","last_undegraded":"2023-11-15T02:56:53.093563+0000","last_fullsized":"2023-11-15T02:56:53.093563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738099+0000","last_change":"2023-11-15T02:56:51.726671+0000","last_active":"2023-11-15T02:56:51.738099+0000","last_peered":"2023-11-15T02:56:51.738099+0000","last_clean":"2023-11-15T02:56:51.738099+0000","last_became_active":"2023-11-15T02:56:51.725108+0000","last_became_peered":"2023-11-15T02:56:51.725108+0000","last_unstale":"2023-11-15T02:56:51.738099+0000","last_undegraded":"2023-11-15T02:56:51.738099+0000","last_fullsized":"2023-11-15T02:56:51.738099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.704634+0000","last_change":"2023-11-15T02:56:51.271748+0000","last_active":"2023-11-15T02:56:51.704634+0000","last_peered":"2023-11-15T02:56:51.704634+0000","last_clean":"2023-11-15T02:56:51.704634+0000","last_became_active":"2023-11-15T02:56:51.271310+0000","last_became_peered":"2023-11-15T02:56:51.271310+0000","last_unstale":"2023-11-15T02:56:51.704634+0000","last_undegraded":"2023-11-15T02:56:51.704634+0000","last_fullsized":"2023-11-15T02:56:51.704634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744179+0000","last_change":"2023-11-15T02:56:51.737961+0000","last_active":"2023-11-15T02:56:51.744179+0000","last_peered":"2023-11-15T02:56:51.744179+0000","last_clean":"2023-11-15T02:56:51.744179+0000","last_became_active":"2023-11-15T02:56:51.737603+0000","last_became_peered":"2023-11-15T02:56:51.737603+0000","last_unstale":"2023-11-15T02:56:51.744179+0000","last_undegraded":"2023-11-15T02:56:51.744179+0000","last_fullsized":"2023-11-15T02:56:51.744179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.711878+0000","last_change":"2023-11-15T02:56:51.286344+0000","last_active":"2023-11-15T02:56:51.711878+0000","last_peered":"2023-11-15T02:56:51.711878+0000","last_clean":"2023-11-15T02:56:51.711878+0000","last_became_active":"2023-11-15T02:56:51.286050+0000","last_became_peered":"2023-11-15T02:56:51.286050+0000","last_unstale":"2023-11-15T02:56:51.711878+0000","last_undegraded":"2023-11-15T02:56:51.711878+0000","last_fullsized":"2023-11-15T02:56:51.711878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.740796+0000","last_change":"2023-11-15T02:56:51.729804+0000","last_active":"2023-11-15T02:56:51.740796+0000","last_peered":"2023-11-15T02:56:51.740796+0000","last_clean":"2023-11-15T02:56:51.740796+0000","last_became_active":"2023-11-15T02:56:51.729569+0000","last_became_peered":"2023-11-15T02:56:51.729569+0000","last_unstale":"2023-11-15T02:56:51.740796+0000","last_undegraded":"2023-11-15T02:56:51.740796+0000","last_fullsized":"2023-11-15T02:56:51.740796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.721025+0000","last_change":"2023-11-15T02:56:51.280678+0000","last_active":"2023-11-15T02:56:51.721025+0000","last_peered":"2023-11-15T02:56:51.721025+0000","last_clean":"2023-11-15T02:56:51.721025+0000","last_became_active":"2023-11-15T02:56:51.279110+0000","last_became_peered":"2023-11-15T02:56:51.279110+0000","last_unstale":"2023-11-15T02:56:51.721025+0000","last_undegraded":"2023-11-15T02:56:51.721025+0000","last_fullsized":"2023-11-15T02:56:51.721025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.728488+0000","last_change":"2023-11-15T02:56:51.726563+0000","last_active":"2023-11-15T02:56:51.728488+0000","last_peered":"2023-11-15T02:56:51.728488+0000","last_clean":"2023-11-15T02:56:51.728488+0000","last_became_active":"2023-11-15T02:56:51.726284+0000","last_became_peered":"2023-11-15T02:56:51.726284+0000","last_unstale":"2023-11-15T02:56:51.728488+0000","last_undegraded":"2023-11-15T02:56:51.728488+0000","last_fullsized":"2023-11-15T02:56:51.728488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.716713+0000","last_change":"2023-11-15T02:56:51.275575+0000","last_active":"2023-11-15T02:56:51.716713+0000","last_peered":"2023-11-15T02:56:51.716713+0000","last_clean":"2023-11-15T02:56:51.716713+0000","last_became_active":"2023-11-15T02:56:51.275267+0000","last_became_peered":"2023-11-15T02:56:51.275267+0000","last_unstale":"2023-11-15T02:56:51.716713+0000","last_undegraded":"2023-11-15T02:56:51.716713+0000","last_fullsized":"2023-11-15T02:56:51.716713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.732898+0000","last_change":"2023-11-15T02:56:51.728149+0000","last_active":"2023-11-15T02:56:51.732898+0000","last_peered":"2023-11-15T02:56:51.732898+0000","last_clean":"2023-11-15T02:56:51.732898+0000","last_became_active":"2023-11-15T02:56:51.727553+0000","last_became_peered":"2023-11-15T02:56:51.727553+0000","last_unstale":"2023-11-15T02:56:51.732898+0000","last_undegraded":"2023-11-15T02:56:51.732898+0000","last_fullsized":"2023-11-15T02:56:51.732898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712886+0000","last_change":"2023-11-15T02:56:51.269197+0000","last_active":"2023-11-15T02:56:51.712886+0000","last_peered":"2023-11-15T02:56:51.712886+0000","last_clean":"2023-11-15T02:56:51.712886+0000","last_became_active":"2023-11-15T02:56:51.268904+0000","last_became_peered":"2023-11-15T02:56:51.268904+0000","last_unstale":"2023-11-15T02:56:51.712886+0000","last_undegraded":"2023-11-15T02:56:51.712886+0000","last_fullsized":"2023-11-15T02:56:51.712886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.724341+0000","last_change":"2023-11-15T02:56:51.715489+0000","last_active":"2023-11-15T02:56:51.724341+0000","last_peered":"2023-11-15T02:56:51.724341+0000","last_clean":"2023-11-15T02:56:51.724341+0000","last_became_active":"2023-11-15T02:56:51.715060+0000","last_became_peered":"2023-11-15T02:56:51.715060+0000","last_unstale":"2023-11-15T02:56:51.724341+0000","last_undegraded":"2023-11-15T02:56:51.724341+0000","last_fullsized":"2023-11-15T02:56:51.724341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712366+0000","last_change":"2023-11-15T02:56:51.276645+0000","last_active":"2023-11-15T02:56:51.712366+0000","last_peered":"2023-11-15T02:56:51.712366+0000","last_clean":"2023-11-15T02:56:51.712366+0000","last_became_active":"2023-11-15T02:56:51.276356+0000","last_became_peered":"2023-11-15T02:56:51.276356+0000","last_unstale":"2023-11-15T02:56:51.712366+0000","last_undegraded":"2023-11-15T02:56:51.712366+0000","last_fullsized":"2023-11-15T02:56:51.712366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.724735+0000","last_change":"2023-11-15T02:56:51.719961+0000","last_active":"2023-11-15T02:56:51.724735+0000","last_peered":"2023-11-15T02:56:51.724735+0000","last_clean":"2023-11-15T02:56:51.724735+0000","last_became_active":"2023-11-15T02:56:51.719598+0000","last_became_peered":"2023-11-15T02:56:51.719598+0000","last_unstale":"2023-11-15T02:56:51.724735+0000","last_undegraded":"2023-11-15T02:56:51.724735+0000","last_fullsized":"2023-11-15T02:56:51.724735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.734513+0000","last_change":"2023-11-15T02:56:51.287275+0000","last_active":"2023-11-15T02:56:51.734513+0000","last_peered":"2023-11-15T02:56:51.734513+0000","last_clean":"2023-11-15T02:56:51.734513+0000","last_became_active":"2023-11-15T02:56:51.286259+0000","last_became_peered":"2023-11-15T02:56:51.286259+0000","last_unstale":"2023-11-15T02:56:51.734513+0000","last_undegraded":"2023-11-15T02:56:51.734513+0000","last_fullsized":"2023-11-15T02:56:51.734513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.751632+0000","last_change":"2023-11-15T02:56:51.748332+0000","last_active":"2023-11-15T02:56:51.751632+0000","last_peered":"2023-11-15T02:56:51.751632+0000","last_clean":"2023-11-15T02:56:51.751632+0000","last_became_active":"2023-11-15T02:56:51.747865+0000","last_became_peered":"2023-11-15T02:56:51.747865+0000","last_unstale":"2023-11-15T02:56:51.751632+0000","last_undegraded":"2023-11-15T02:56:51.751632+0000","last_fullsized":"2023-11-15T02:56:51.751632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746250+0000","last_change":"2023-11-15T02:56:51.274812+0000","last_active":"2023-11-15T02:56:51.746250+0000","last_peered":"2023-11-15T02:56:51.746250+0000","last_clean":"2023-11-15T02:56:51.746250+0000","last_became_active":"2023-11-15T02:56:51.273693+0000","last_became_peered":"2023-11-15T02:56:51.273693+0000","last_unstale":"2023-11-15T02:56:51.746250+0000","last_undegraded":"2023-11-15T02:56:51.746250+0000","last_fullsized":"2023-11-15T02:56:51.746250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.734600+0000","last_change":"2023-11-15T02:56:51.718982+0000","last_active":"2023-11-15T02:56:51.734600+0000","last_peered":"2023-11-15T02:56:51.734600+0000","last_clean":"2023-11-15T02:56:51.734600+0000","last_became_active":"2023-11-15T02:56:51.718702+0000","last_became_peered":"2023-11-15T02:56:51.718702+0000","last_unstale":"2023-11-15T02:56:51.734600+0000","last_undegraded":"2023-11-15T02:56:51.734600+0000","last_fullsized":"2023-11-15T02:56:51.734600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.743614+0000","last_change":"2023-11-15T02:56:51.285434+0000","last_active":"2023-11-15T02:56:51.743614+0000","last_peered":"2023-11-15T02:56:51.743614+0000","last_clean":"2023-11-15T02:56:51.743614+0000","last_became_active":"2023-11-15T02:56:51.284984+0000","last_became_peered":"2023-11-15T02:56:51.284984+0000","last_unstale":"2023-11-15T02:56:51.743614+0000","last_undegraded":"2023-11-15T02:56:51.743614+0000","last_fullsized":"2023-11-15T02:56:51.743614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744214+0000","last_change":"2023-11-15T02:56:51.740534+0000","last_active":"2023-11-15T02:56:51.744214+0000","last_peered":"2023-11-15T02:56:51.744214+0000","last_clean":"2023-11-15T02:56:51.744214+0000","last_became_active":"2023-11-15T02:56:51.740090+0000","last_became_peered":"2023-11-15T02:56:51.740090+0000","last_unstale":"2023-11-15T02:56:51.744214+0000","last_undegraded":"2023-11-15T02:56:51.744214+0000","last_fullsized":"2023-11-15T02:56:51.744214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.714872+0000","last_change":"2023-11-15T02:56:51.279489+0000","last_active":"2023-11-15T02:56:51.714872+0000","last_peered":"2023-11-15T02:56:51.714872+0000","last_clean":"2023-11-15T02:56:51.714872+0000","last_became_active":"2023-11-15T02:56:51.279222+0000","last_became_peered":"2023-11-15T02:56:51.279222+0000","last_unstale":"2023-11-15T02:56:51.714872+0000","last_undegraded":"2023-11-15T02:56:51.714872+0000","last_fullsized":"2023-11-15T02:56:51.714872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.749518+0000","last_change":"2023-11-15T02:56:51.744824+0000","last_active":"2023-11-15T02:56:51.749518+0000","last_peered":"2023-11-15T02:56:51.749518+0000","last_clean":"2023-11-15T02:56:51.749518+0000","last_became_active":"2023-11-15T02:56:51.744533+0000","last_became_peered":"2023-11-15T02:56:51.744533+0000","last_unstale":"2023-11-15T02:56:51.749518+0000","last_undegraded":"2023-11-15T02:56:51.749518+0000","last_fullsized":"2023-11-15T02:56:51.749518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.721137+0000","last_change":"2023-11-15T02:56:51.273142+0000","last_active":"2023-11-15T02:56:51.721137+0000","last_peered":"2023-11-15T02:56:51.721137+0000","last_clean":"2023-11-15T02:56:51.721137+0000","last_became_active":"2023-11-15T02:56:51.272861+0000","last_became_peered":"2023-11-15T02:56:51.272861+0000","last_unstale":"2023-11-15T02:56:51.721137+0000","last_undegraded":"2023-11-15T02:56:51.721137+0000","last_fullsized":"2023-11-15T02:56:51.721137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.735105+0000","last_change":"2023-11-15T02:56:51.728416+0000","last_active":"2023-11-15T02:56:51.735105+0000","last_peered":"2023-11-15T02:56:51.735105+0000","last_clean":"2023-11-15T02:56:51.735105+0000","last_became_active":"2023-11-15T02:56:51.728084+0000","last_became_peered":"2023-11-15T02:56:51.728084+0000","last_unstale":"2023-11-15T02:56:51.735105+0000","last_undegraded":"2023-11-15T02:56:51.735105+0000","last_fullsized":"2023-11-15T02:56:51.735105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.724100+0000","last_change":"2023-11-15T02:56:51.274354+0000","last_active":"2023-11-15T02:56:51.724100+0000","last_peered":"2023-11-15T02:56:51.724100+0000","last_clean":"2023-11-15T02:56:51.724100+0000","last_became_active":"2023-11-15T02:56:51.274083+0000","last_became_peered":"2023-11-15T02:56:51.274083+0000","last_unstale":"2023-11-15T02:56:51.724100+0000","last_undegraded":"2023-11-15T02:56:51.724100+0000","last_fullsized":"2023-11-15T02:56:51.724100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.736974+0000","last_change":"2023-11-15T02:56:51.721714+0000","last_active":"2023-11-15T02:56:51.736974+0000","last_peered":"2023-11-15T02:56:51.736974+0000","last_clean":"2023-11-15T02:56:51.736974+0000","last_became_active":"2023-11-15T02:56:51.721415+0000","last_became_peered":"2023-11-15T02:56:51.721415+0000","last_unstale":"2023-11-15T02:56:51.736974+0000","last_undegraded":"2023-11-15T02:56:51.736974+0000","last_fullsized":"2023-11-15T02:56:51.736974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.704644+0000","last_change":"2023-11-15T02:56:51.283397+0000","last_active":"2023-11-15T02:56:51.704644+0000","last_peered":"2023-11-15T02:56:51.704644+0000","last_clean":"2023-11-15T02:56:51.704644+0000","last_became_active":"2023-11-15T02:56:51.283101+0000","last_became_peered":"2023-11-15T02:56:51.283101+0000","last_unstale":"2023-11-15T02:56:51.704644+0000","last_undegraded":"2023-11-15T02:56:51.704644+0000","last_fullsized":"2023-11-15T02:56:51.704644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.735611+0000","last_change":"2023-11-15T02:56:51.727142+0000","last_active":"2023-11-15T02:56:51.735611+0000","last_peered":"2023-11-15T02:56:51.735611+0000","last_clean":"2023-11-15T02:56:51.735611+0000","last_became_active":"2023-11-15T02:56:51.726872+0000","last_became_peered":"2023-11-15T02:56:51.726872+0000","last_unstale":"2023-11-15T02:56:51.735611+0000","last_undegraded":"2023-11-15T02:56:51.735611+0000","last_fullsized":"2023-11-15T02:56:51.735611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.710839+0000","last_change":"2023-11-15T02:56:51.281104+0000","last_active":"2023-11-15T02:56:51.710839+0000","last_peered":"2023-11-15T02:56:51.710839+0000","last_clean":"2023-11-15T02:56:51.710839+0000","last_became_active":"2023-11-15T02:56:51.280784+0000","last_became_peered":"2023-11-15T02:56:51.280784+0000","last_unstale":"2023-11-15T02:56:51.710839+0000","last_undegraded":"2023-11-15T02:56:51.710839+0000","last_fullsized":"2023-11-15T02:56:51.710839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.737357+0000","last_change":"2023-11-15T02:56:51.719121+0000","last_active":"2023-11-15T02:56:51.737357+0000","last_peered":"2023-11-15T02:56:51.737357+0000","last_clean":"2023-11-15T02:56:51.737357+0000","last_became_active":"2023-11-15T02:56:51.718812+0000","last_became_peered":"2023-11-15T02:56:51.718812+0000","last_unstale":"2023-11-15T02:56:51.737357+0000","last_undegraded":"2023-11-15T02:56:51.737357+0000","last_fullsized":"2023-11-15T02:56:51.737357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.722994+0000","last_change":"2023-11-15T02:56:51.285977+0000","last_active":"2023-11-15T02:56:51.722994+0000","last_peered":"2023-11-15T02:56:51.722994+0000","last_clean":"2023-11-15T02:56:51.722994+0000","last_became_active":"2023-11-15T02:56:51.277431+0000","last_became_peered":"2023-11-15T02:56:51.277431+0000","last_unstale":"2023-11-15T02:56:51.722994+0000","last_undegraded":"2023-11-15T02:56:51.722994+0000","last_fullsized":"2023-11-15T02:56:51.722994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.735350+0000","last_change":"2023-11-15T02:56:51.720775+0000","last_active":"2023-11-15T02:56:51.735350+0000","last_peered":"2023-11-15T02:56:51.735350+0000","last_clean":"2023-11-15T02:56:51.735350+0000","last_became_active":"2023-11-15T02:56:51.720446+0000","last_became_peered":"2023-11-15T02:56:51.720446+0000","last_unstale":"2023-11-15T02:56:51.735350+0000","last_undegraded":"2023-11-15T02:56:51.735350+0000","last_fullsized":"2023-11-15T02:56:51.735350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.715531+0000","last_change":"2023-11-15T02:56:51.268303+0000","last_active":"2023-11-15T02:56:51.715531+0000","last_peered":"2023-11-15T02:56:51.715531+0000","last_clean":"2023-11-15T02:56:51.715531+0000","last_became_active":"2023-11-15T02:56:51.267982+0000","last_became_peered":"2023-11-15T02:56:51.267982+0000","last_unstale":"2023-11-15T02:56:51.715531+0000","last_undegraded":"2023-11-15T02:56:51.715531+0000","last_fullsized":"2023-11-15T02:56:51.715531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738120+0000","last_change":"2023-11-15T02:56:51.726853+0000","last_active":"2023-11-15T02:56:51.738120+0000","last_peered":"2023-11-15T02:56:51.738120+0000","last_clean":"2023-11-15T02:56:51.738120+0000","last_became_active":"2023-11-15T02:56:51.726448+0000","last_became_peered":"2023-11-15T02:56:51.726448+0000","last_unstale":"2023-11-15T02:56:51.738120+0000","last_undegraded":"2023-11-15T02:56:51.738120+0000","last_fullsized":"2023-11-15T02:56:51.738120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745254+0000","last_change":"2023-11-15T02:56:51.291178+0000","last_active":"2023-11-15T02:56:51.745254+0000","last_peered":"2023-11-15T02:56:51.745254+0000","last_clean":"2023-11-15T02:56:51.745254+0000","last_became_active":"2023-11-15T02:56:51.289245+0000","last_became_peered":"2023-11-15T02:56:51.289245+0000","last_unstale":"2023-11-15T02:56:51.745254+0000","last_undegraded":"2023-11-15T02:56:51.745254+0000","last_fullsized":"2023-11-15T02:56:51.745254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.718417+0000","last_change":"2023-11-15T02:56:51.714302+0000","last_active":"2023-11-15T02:56:51.718417+0000","last_peered":"2023-11-15T02:56:51.718417+0000","last_clean":"2023-11-15T02:56:51.718417+0000","last_became_active":"2023-11-15T02:56:51.714023+0000","last_became_peered":"2023-11-15T02:56:51.714023+0000","last_unstale":"2023-11-15T02:56:51.718417+0000","last_undegraded":"2023-11-15T02:56:51.718417+0000","last_fullsized":"2023-11-15T02:56:51.718417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746516+0000","last_change":"2023-11-15T02:56:51.265289+0000","last_active":"2023-11-15T02:56:51.746516+0000","last_peered":"2023-11-15T02:56:51.746516+0000","last_clean":"2023-11-15T02:56:51.746516+0000","last_became_active":"2023-11-15T02:56:51.264945+0000","last_became_peered":"2023-11-15T02:56:51.264945+0000","last_unstale":"2023-11-15T02:56:51.746516+0000","last_undegraded":"2023-11-15T02:56:51.746516+0000","last_fullsized":"2023-11-15T02:56:51.746516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.751653+0000","last_change":"2023-11-15T02:56:51.748696+0000","last_active":"2023-11-15T02:56:51.751653+0000","last_peered":"2023-11-15T02:56:51.751653+0000","last_clean":"2023-11-15T02:56:51.751653+0000","last_became_active":"2023-11-15T02:56:51.748403+0000","last_became_peered":"2023-11-15T02:56:51.748403+0000","last_unstale":"2023-11-15T02:56:51.751653+0000","last_undegraded":"2023-11-15T02:56:51.751653+0000","last_fullsized":"2023-11-15T02:56:51.751653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713322+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T02:56:51.713322+0000","last_peered":"2023-11-15T02:56:51.713322+0000","last_clean":"2023-11-15T02:56:51.713322+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T02:56:51.713322+0000","last_undegraded":"2023-11-15T02:56:51.713322+0000","last_fullsized":"2023-11-15T02:56:51.713322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.722937+0000","last_change":"2023-11-15T02:56:51.270301+0000","last_active":"2023-11-15T02:56:51.722937+0000","last_peered":"2023-11-15T02:56:51.722937+0000","last_clean":"2023-11-15T02:56:51.722937+0000","last_became_active":"2023-11-15T02:56:51.269996+0000","last_became_peered":"2023-11-15T02:56:51.269996+0000","last_unstale":"2023-11-15T02:56:51.722937+0000","last_undegraded":"2023-11-15T02:56:51.722937+0000","last_fullsized":"2023-11-15T02:56:51.722937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.742366+0000","last_change":"2023-11-15T02:56:51.736621+0000","last_active":"2023-11-15T02:56:51.742366+0000","last_peered":"2023-11-15T02:56:51.742366+0000","last_clean":"2023-11-15T02:56:51.742366+0000","last_became_active":"2023-11-15T02:56:51.736319+0000","last_became_peered":"2023-11-15T02:56:51.736319+0000","last_unstale":"2023-11-15T02:56:51.742366+0000","last_undegraded":"2023-11-15T02:56:51.742366+0000","last_fullsized":"2023-11-15T02:56:51.742366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.706577+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T02:56:51.706577+0000","last_peered":"2023-11-15T02:56:51.706577+0000","last_clean":"2023-11-15T02:56:51.706577+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T02:56:51.706577+0000","last_undegraded":"2023-11-15T02:56:51.706577+0000","last_fullsized":"2023-11-15T02:56:51.706577+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745714+0000","last_change":"2023-11-15T02:56:51.273274+0000","last_active":"2023-11-15T02:56:51.745714+0000","last_peered":"2023-11-15T02:56:51.745714+0000","last_clean":"2023-11-15T02:56:51.745714+0000","last_became_active":"2023-11-15T02:56:51.270655+0000","last_became_peered":"2023-11-15T02:56:51.270655+0000","last_unstale":"2023-11-15T02:56:51.745714+0000","last_undegraded":"2023-11-15T02:56:51.745714+0000","last_fullsized":"2023-11-15T02:56:51.745714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738809+0000","last_change":"2023-11-15T02:56:51.734478+0000","last_active":"2023-11-15T02:56:51.738809+0000","last_peered":"2023-11-15T02:56:51.738809+0000","last_clean":"2023-11-15T02:56:51.738809+0000","last_became_active":"2023-11-15T02:56:51.732194+0000","last_became_peered":"2023-11-15T02:56:51.732194+0000","last_unstale":"2023-11-15T02:56:51.738809+0000","last_undegraded":"2023-11-15T02:56:51.738809+0000","last_fullsized":"2023-11-15T02:56:51.738809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730730+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T02:56:51.730730+0000","last_peered":"2023-11-15T02:56:51.730730+0000","last_clean":"2023-11-15T02:56:51.730730+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T02:56:51.730730+0000","last_undegraded":"2023-11-15T02:56:51.730730+0000","last_fullsized":"2023-11-15T02:56:51.730730+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.705716+0000","last_change":"2023-11-15T02:56:51.270161+0000","last_active":"2023-11-15T02:56:51.705716+0000","last_peered":"2023-11-15T02:56:51.705716+0000","last_clean":"2023-11-15T02:56:51.705716+0000","last_became_active":"2023-11-15T02:56:51.268151+0000","last_became_peered":"2023-11-15T02:56:51.268151+0000","last_unstale":"2023-11-15T02:56:51.705716+0000","last_undegraded":"2023-11-15T02:56:51.705716+0000","last_fullsized":"2023-11-15T02:56:51.705716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.737232+0000","last_change":"2023-11-15T02:56:51.731375+0000","last_active":"2023-11-15T02:56:51.737232+0000","last_peered":"2023-11-15T02:56:51.737232+0000","last_clean":"2023-11-15T02:56:51.737232+0000","last_became_active":"2023-11-15T02:56:51.731100+0000","last_became_peered":"2023-11-15T02:56:51.731100+0000","last_unstale":"2023-11-15T02:56:51.737232+0000","last_undegraded":"2023-11-15T02:56:51.737232+0000","last_fullsized":"2023-11-15T02:56:51.737232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.705942+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T02:56:51.705942+0000","last_peered":"2023-11-15T02:56:51.705942+0000","last_clean":"2023-11-15T02:56:51.705942+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T02:56:51.705942+0000","last_undegraded":"2023-11-15T02:56:51.705942+0000","last_fullsized":"2023-11-15T02:56:51.705942+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.715265+0000","last_change":"2023-11-15T02:56:51.275134+0000","last_active":"2023-11-15T02:56:51.715265+0000","last_peered":"2023-11-15T02:56:51.715265+0000","last_clean":"2023-11-15T02:56:51.715265+0000","last_became_active":"2023-11-15T02:56:51.274400+0000","last_became_peered":"2023-11-15T02:56:51.274400+0000","last_unstale":"2023-11-15T02:56:51.715265+0000","last_undegraded":"2023-11-15T02:56:51.715265+0000","last_fullsized":"2023-11-15T02:56:51.715265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.727321+0000","last_change":"2023-11-15T02:56:51.724465+0000","last_active":"2023-11-15T02:56:51.727321+0000","last_peered":"2023-11-15T02:56:51.727321+0000","last_clean":"2023-11-15T02:56:51.727321+0000","last_became_active":"2023-11-15T02:56:51.723590+0000","last_became_peered":"2023-11-15T02:56:51.723590+0000","last_unstale":"2023-11-15T02:56:51.727321+0000","last_undegraded":"2023-11-15T02:56:51.727321+0000","last_fullsized":"2023-11-15T02:56:51.727321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.731884+0000","last_change":"2023-11-15T02:56:51.282484+0000","last_active":"2023-11-15T02:56:51.731884+0000","last_peered":"2023-11-15T02:56:51.731884+0000","last_clean":"2023-11-15T02:56:51.731884+0000","last_became_active":"2023-11-15T02:56:51.282108+0000","last_became_peered":"2023-11-15T02:56:51.282108+0000","last_unstale":"2023-11-15T02:56:51.731884+0000","last_undegraded":"2023-11-15T02:56:51.731884+0000","last_fullsized":"2023-11-15T02:56:51.731884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.716774+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T02:56:51.716774+0000","last_peered":"2023-11-15T02:56:51.716774+0000","last_clean":"2023-11-15T02:56:51.716774+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T02:56:51.716774+0000","last_undegraded":"2023-11-15T02:56:51.716774+0000","last_fullsized":"2023-11-15T02:56:51.716774+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.719253+0000","last_change":"2023-11-15T02:56:51.717765+0000","last_active":"2023-11-15T02:56:51.719253+0000","last_peered":"2023-11-15T02:56:51.719253+0000","last_clean":"2023-11-15T02:56:51.719253+0000","last_became_active":"2023-11-15T02:56:51.717498+0000","last_became_peered":"2023-11-15T02:56:51.717498+0000","last_unstale":"2023-11-15T02:56:51.719253+0000","last_undegraded":"2023-11-15T02:56:51.719253+0000","last_fullsized":"2023-11-15T02:56:51.719253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.704337+0000","last_change":"2023-11-15T02:56:51.267133+0000","last_active":"2023-11-15T02:56:51.704337+0000","last_peered":"2023-11-15T02:56:51.704337+0000","last_clean":"2023-11-15T02:56:51.704337+0000","last_became_active":"2023-11-15T02:56:51.266864+0000","last_became_peered":"2023-11-15T02:56:51.266864+0000","last_unstale":"2023-11-15T02:56:51.704337+0000","last_undegraded":"2023-11-15T02:56:51.704337+0000","last_fullsized":"2023-11-15T02:56:51.704337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712961+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T02:56:51.712961+0000","last_peered":"2023-11-15T02:56:51.712961+0000","last_clean":"2023-11-15T02:56:51.712961+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T02:56:51.712961+0000","last_undegraded":"2023-11-15T02:56:51.712961+0000","last_fullsized":"2023-11-15T02:56:51.712961+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.719804+0000","last_change":"2023-11-15T02:56:51.717744+0000","last_active":"2023-11-15T02:56:51.719804+0000","last_peered":"2023-11-15T02:56:51.719804+0000","last_clean":"2023-11-15T02:56:51.719804+0000","last_became_active":"2023-11-15T02:56:51.717448+0000","last_became_peered":"2023-11-15T02:56:51.717448+0000","last_unstale":"2023-11-15T02:56:51.719804+0000","last_undegraded":"2023-11-15T02:56:51.719804+0000","last_fullsized":"2023-11-15T02:56:51.719804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.718982+0000","last_change":"2023-11-15T02:56:51.273152+0000","last_active":"2023-11-15T02:56:51.718982+0000","last_peered":"2023-11-15T02:56:51.718982+0000","last_clean":"2023-11-15T02:56:51.718982+0000","last_became_active":"2023-11-15T02:56:51.272868+0000","last_became_peered":"2023-11-15T02:56:51.272868+0000","last_unstale":"2023-11-15T02:56:51.718982+0000","last_undegraded":"2023-11-15T02:56:51.718982+0000","last_fullsized":"2023-11-15T02:56:51.718982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.728384+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T02:56:51.728384+0000","last_peered":"2023-11-15T02:56:51.728384+0000","last_clean":"2023-11-15T02:56:51.728384+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T02:56:51.728384+0000","last_undegraded":"2023-11-15T02:56:51.728384+0000","last_fullsized":"2023-11-15T02:56:51.728384+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744430+0000","last_change":"2023-11-15T02:56:51.739786+0000","last_active":"2023-11-15T02:56:51.744430+0000","last_peered":"2023-11-15T02:56:51.744430+0000","last_clean":"2023-11-15T02:56:51.744430+0000","last_became_active":"2023-11-15T02:56:51.737659+0000","last_became_peered":"2023-11-15T02:56:51.737659+0000","last_unstale":"2023-11-15T02:56:51.744430+0000","last_undegraded":"2023-11-15T02:56:51.744430+0000","last_fullsized":"2023-11-15T02:56:51.744430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.711641+0000","last_change":"2023-11-15T02:56:51.276817+0000","last_active":"2023-11-15T02:56:51.711641+0000","last_peered":"2023-11-15T02:56:51.711641+0000","last_clean":"2023-11-15T02:56:51.711641+0000","last_became_active":"2023-11-15T02:56:51.276490+0000","last_became_peered":"2023-11-15T02:56:51.276490+0000","last_unstale":"2023-11-15T02:56:51.711641+0000","last_undegraded":"2023-11-15T02:56:51.711641+0000","last_fullsized":"2023-11-15T02:56:51.711641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713989+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T02:56:51.713989+0000","last_peered":"2023-11-15T02:56:51.713989+0000","last_clean":"2023-11-15T02:56:51.713989+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T02:56:51.713989+0000","last_undegraded":"2023-11-15T02:56:51.713989+0000","last_fullsized":"2023-11-15T02:56:51.713989+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.728465+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T02:56:51.728465+0000","last_peered":"2023-11-15T02:56:51.728465+0000","last_clean":"2023-11-15T02:56:51.728465+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T02:56:51.728465+0000","last_undegraded":"2023-11-15T02:56:51.728465+0000","last_fullsized":"2023-11-15T02:56:51.728465+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.749349+0000","last_change":"2023-11-15T02:56:51.745538+0000","last_active":"2023-11-15T02:56:51.749349+0000","last_peered":"2023-11-15T02:56:51.749349+0000","last_clean":"2023-11-15T02:56:51.749349+0000","last_became_active":"2023-11-15T02:56:51.745369+0000","last_became_peered":"2023-11-15T02:56:51.745369+0000","last_unstale":"2023-11-15T02:56:51.749349+0000","last_undegraded":"2023-11-15T02:56:51.749349+0000","last_fullsized":"2023-11-15T02:56:51.749349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.710979+0000","last_change":"2023-11-15T02:56:51.280484+0000","last_active":"2023-11-15T02:56:51.710979+0000","last_peered":"2023-11-15T02:56:51.710979+0000","last_clean":"2023-11-15T02:56:51.710979+0000","last_became_active":"2023-11-15T02:56:51.272264+0000","last_became_peered":"2023-11-15T02:56:51.272264+0000","last_unstale":"2023-11-15T02:56:51.710979+0000","last_undegraded":"2023-11-15T02:56:51.710979+0000","last_fullsized":"2023-11-15T02:56:51.710979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.724717+0000","last_change":"2023-11-15T02:56:51.719709+0000","last_active":"2023-11-15T02:56:51.724717+0000","last_peered":"2023-11-15T02:56:51.724717+0000","last_clean":"2023-11-15T02:56:51.724717+0000","last_became_active":"2023-11-15T02:56:51.719230+0000","last_became_peered":"2023-11-15T02:56:51.719230+0000","last_unstale":"2023-11-15T02:56:51.724717+0000","last_undegraded":"2023-11-15T02:56:51.724717+0000","last_fullsized":"2023-11-15T02:56:51.724717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.735953+0000","last_change":"2023-11-15T02:56:51.272978+0000","last_active":"2023-11-15T02:56:51.735953+0000","last_peered":"2023-11-15T02:56:51.735953+0000","last_clean":"2023-11-15T02:56:51.735953+0000","last_became_active":"2023-11-15T02:56:51.272650+0000","last_became_peered":"2023-11-15T02:56:51.272650+0000","last_unstale":"2023-11-15T02:56:51.735953+0000","last_undegraded":"2023-11-15T02:56:51.735953+0000","last_fullsized":"2023-11-15T02:56:51.735953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.749492+0000","last_change":"2023-11-15T02:56:51.744603+0000","last_active":"2023-11-15T02:56:51.749492+0000","last_peered":"2023-11-15T02:56:51.749492+0000","last_clean":"2023-11-15T02:56:51.749492+0000","last_became_active":"2023-11-15T02:56:51.744212+0000","last_became_peered":"2023-11-15T02:56:51.744212+0000","last_unstale":"2023-11-15T02:56:51.749492+0000","last_undegraded":"2023-11-15T02:56:51.749492+0000","last_fullsized":"2023-11-15T02:56:51.749492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713620+0000","last_change":"2023-11-15T02:56:51.279629+0000","last_active":"2023-11-15T02:56:51.713620+0000","last_peered":"2023-11-15T02:56:51.713620+0000","last_clean":"2023-11-15T02:56:51.713620+0000","last_became_active":"2023-11-15T02:56:51.279367+0000","last_became_peered":"2023-11-15T02:56:51.279367+0000","last_unstale":"2023-11-15T02:56:51.713620+0000","last_undegraded":"2023-11-15T02:56:51.713620+0000","last_fullsized":"2023-11-15T02:56:51.713620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745035+0000","last_change":"2023-11-15T02:56:51.741271+0000","last_active":"2023-11-15T02:56:51.745035+0000","last_peered":"2023-11-15T02:56:51.745035+0000","last_clean":"2023-11-15T02:56:51.745035+0000","last_became_active":"2023-11-15T02:56:51.740569+0000","last_became_peered":"2023-11-15T02:56:51.740569+0000","last_unstale":"2023-11-15T02:56:51.745035+0000","last_undegraded":"2023-11-15T02:56:51.745035+0000","last_fullsized":"2023-11-15T02:56:51.745035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.740741+0000","last_change":"2023-11-15T02:56:51.278710+0000","last_active":"2023-11-15T02:56:51.740741+0000","last_peered":"2023-11-15T02:56:51.740741+0000","last_clean":"2023-11-15T02:56:51.740741+0000","last_became_active":"2023-11-15T02:56:51.278446+0000","last_became_peered":"2023-11-15T02:56:51.278446+0000","last_unstale":"2023-11-15T02:56:51.740741+0000","last_undegraded":"2023-11-15T02:56:51.740741+0000","last_fullsized":"2023-11-15T02:56:51.740741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745553+0000","last_change":"2023-11-15T02:56:51.740004+0000","last_active":"2023-11-15T02:56:51.745553+0000","last_peered":"2023-11-15T02:56:51.745553+0000","last_clean":"2023-11-15T02:56:51.745553+0000","last_became_active":"2023-11-15T02:56:51.739663+0000","last_became_peered":"2023-11-15T02:56:51.739663+0000","last_unstale":"2023-11-15T02:56:51.745553+0000","last_undegraded":"2023-11-15T02:56:51.745553+0000","last_fullsized":"2023-11-15T02:56:51.745553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.714982+0000","last_change":"2023-11-15T02:56:51.278072+0000","last_active":"2023-11-15T02:56:51.714982+0000","last_peered":"2023-11-15T02:56:51.714982+0000","last_clean":"2023-11-15T02:56:51.714982+0000","last_became_active":"2023-11-15T02:56:51.277779+0000","last_became_peered":"2023-11-15T02:56:51.277779+0000","last_unstale":"2023-11-15T02:56:51.714982+0000","last_undegraded":"2023-11-15T02:56:51.714982+0000","last_fullsized":"2023-11-15T02:56:51.714982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746826+0000","last_change":"2023-11-15T02:56:51.744994+0000","last_active":"2023-11-15T02:56:51.746826+0000","last_peered":"2023-11-15T02:56:51.746826+0000","last_clean":"2023-11-15T02:56:51.746826+0000","last_became_active":"2023-11-15T02:56:51.743902+0000","last_became_peered":"2023-11-15T02:56:51.743902+0000","last_unstale":"2023-11-15T02:56:51.746826+0000","last_undegraded":"2023-11-15T02:56:51.746826+0000","last_fullsized":"2023-11-15T02:56:51.746826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730502+0000","last_change":"2023-11-15T02:56:51.289728+0000","last_active":"2023-11-15T02:56:51.730502+0000","last_peered":"2023-11-15T02:56:51.730502+0000","last_clean":"2023-11-15T02:56:51.730502+0000","last_became_active":"2023-11-15T02:56:51.288566+0000","last_became_peered":"2023-11-15T02:56:51.288566+0000","last_unstale":"2023-11-15T02:56:51.730502+0000","last_undegraded":"2023-11-15T02:56:51.730502+0000","last_fullsized":"2023-11-15T02:56:51.730502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747477+0000","last_change":"2023-11-15T02:56:51.735076+0000","last_active":"2023-11-15T02:56:51.747477+0000","last_peered":"2023-11-15T02:56:51.747477+0000","last_clean":"2023-11-15T02:56:51.747477+0000","last_became_active":"2023-11-15T02:56:51.734802+0000","last_became_peered":"2023-11-15T02:56:51.734802+0000","last_unstale":"2023-11-15T02:56:51.747477+0000","last_undegraded":"2023-11-15T02:56:51.747477+0000","last_fullsized":"2023-11-15T02:56:51.747477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744754+0000","last_change":"2023-11-15T02:56:51.286563+0000","last_active":"2023-11-15T02:56:51.744754+0000","last_peered":"2023-11-15T02:56:51.744754+0000","last_clean":"2023-11-15T02:56:51.744754+0000","last_became_active":"2023-11-15T02:56:51.286245+0000","last_became_peered":"2023-11-15T02:56:51.286245+0000","last_unstale":"2023-11-15T02:56:51.744754+0000","last_undegraded":"2023-11-15T02:56:51.744754+0000","last_fullsized":"2023-11-15T02:56:51.744754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746575+0000","last_change":"2023-11-15T02:56:51.742254+0000","last_active":"2023-11-15T02:56:51.746575+0000","last_peered":"2023-11-15T02:56:51.746575+0000","last_clean":"2023-11-15T02:56:51.746575+0000","last_became_active":"2023-11-15T02:56:51.741950+0000","last_became_peered":"2023-11-15T02:56:51.741950+0000","last_unstale":"2023-11-15T02:56:51.746575+0000","last_undegraded":"2023-11-15T02:56:51.746575+0000","last_fullsized":"2023-11-15T02:56:51.746575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744885+0000","last_change":"2023-11-15T02:56:51.286640+0000","last_active":"2023-11-15T02:56:51.744885+0000","last_peered":"2023-11-15T02:56:51.744885+0000","last_clean":"2023-11-15T02:56:51.744885+0000","last_became_active":"2023-11-15T02:56:51.286290+0000","last_became_peered":"2023-11-15T02:56:51.286290+0000","last_unstale":"2023-11-15T02:56:51.744885+0000","last_undegraded":"2023-11-15T02:56:51.744885+0000","last_fullsized":"2023-11-15T02:56:51.744885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.733131+0000","last_change":"2023-11-15T02:56:51.727797+0000","last_active":"2023-11-15T02:56:51.733131+0000","last_peered":"2023-11-15T02:56:51.733131+0000","last_clean":"2023-11-15T02:56:51.733131+0000","last_became_active":"2023-11-15T02:56:51.727520+0000","last_became_peered":"2023-11-15T02:56:51.727520+0000","last_unstale":"2023-11-15T02:56:51.733131+0000","last_undegraded":"2023-11-15T02:56:51.733131+0000","last_fullsized":"2023-11-15T02:56:51.733131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738551+0000","last_change":"2023-11-15T02:56:51.284828+0000","last_active":"2023-11-15T02:56:51.738551+0000","last_peered":"2023-11-15T02:56:51.738551+0000","last_clean":"2023-11-15T02:56:51.738551+0000","last_became_active":"2023-11-15T02:56:51.276571+0000","last_became_peered":"2023-11-15T02:56:51.276571+0000","last_unstale":"2023-11-15T02:56:51.738551+0000","last_undegraded":"2023-11-15T02:56:51.738551+0000","last_fullsized":"2023-11-15T02:56:51.738551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.739504+0000","last_change":"2023-11-15T02:56:51.732129+0000","last_active":"2023-11-15T02:56:51.739504+0000","last_peered":"2023-11-15T02:56:51.739504+0000","last_clean":"2023-11-15T02:56:51.739504+0000","last_became_active":"2023-11-15T02:56:51.729770+0000","last_became_peered":"2023-11-15T02:56:51.729770+0000","last_unstale":"2023-11-15T02:56:51.739504+0000","last_undegraded":"2023-11-15T02:56:51.739504+0000","last_fullsized":"2023-11-15T02:56:51.739504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.731327+0000","last_change":"2023-11-15T02:56:51.280630+0000","last_active":"2023-11-15T02:56:51.731327+0000","last_peered":"2023-11-15T02:56:51.731327+0000","last_clean":"2023-11-15T02:56:51.731327+0000","last_became_active":"2023-11-15T02:56:51.280353+0000","last_became_peered":"2023-11-15T02:56:51.280353+0000","last_unstale":"2023-11-15T02:56:51.731327+0000","last_undegraded":"2023-11-15T02:56:51.731327+0000","last_fullsized":"2023-11-15T02:56:51.731327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.741375+0000","last_change":"2023-11-15T02:56:51.737689+0000","last_active":"2023-11-15T02:56:51.741375+0000","last_peered":"2023-11-15T02:56:51.741375+0000","last_clean":"2023-11-15T02:56:51.741375+0000","last_became_active":"2023-11-15T02:56:51.737369+0000","last_became_peered":"2023-11-15T02:56:51.737369+0000","last_unstale":"2023-11-15T02:56:51.741375+0000","last_undegraded":"2023-11-15T02:56:51.741375+0000","last_fullsized":"2023-11-15T02:56:51.741375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.719432+0000","last_change":"2023-11-15T02:56:51.285172+0000","last_active":"2023-11-15T02:56:51.719432+0000","last_peered":"2023-11-15T02:56:51.719432+0000","last_clean":"2023-11-15T02:56:51.719432+0000","last_became_active":"2023-11-15T02:56:51.284936+0000","last_became_peered":"2023-11-15T02:56:51.284936+0000","last_unstale":"2023-11-15T02:56:51.719432+0000","last_undegraded":"2023-11-15T02:56:51.719432+0000","last_fullsized":"2023-11-15T02:56:51.719432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.732918+0000","last_change":"2023-11-15T02:56:51.728432+0000","last_active":"2023-11-15T02:56:51.732918+0000","last_peered":"2023-11-15T02:56:51.732918+0000","last_clean":"2023-11-15T02:56:51.732918+0000","last_became_active":"2023-11-15T02:56:51.727880+0000","last_became_peered":"2023-11-15T02:56:51.727880+0000","last_unstale":"2023-11-15T02:56:51.732918+0000","last_undegraded":"2023-11-15T02:56:51.732918+0000","last_fullsized":"2023-11-15T02:56:51.732918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744376+0000","last_change":"2023-11-15T02:56:51.288049+0000","last_active":"2023-11-15T02:56:51.744376+0000","last_peered":"2023-11-15T02:56:51.744376+0000","last_clean":"2023-11-15T02:56:51.744376+0000","last_became_active":"2023-11-15T02:56:51.287794+0000","last_became_peered":"2023-11-15T02:56:51.287794+0000","last_unstale":"2023-11-15T02:56:51.744376+0000","last_undegraded":"2023-11-15T02:56:51.744376+0000","last_fullsized":"2023-11-15T02:56:51.744376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.734683+0000","last_change":"2023-11-15T02:56:51.720823+0000","last_active":"2023-11-15T02:56:51.734683+0000","last_peered":"2023-11-15T02:56:51.734683+0000","last_clean":"2023-11-15T02:56:51.734683+0000","last_became_active":"2023-11-15T02:56:51.720479+0000","last_became_peered":"2023-11-15T02:56:51.720479+0000","last_unstale":"2023-11-15T02:56:51.734683+0000","last_undegraded":"2023-11-15T02:56:51.734683+0000","last_fullsized":"2023-11-15T02:56:51.734683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.718875+0000","last_change":"2023-11-15T02:56:51.278717+0000","last_active":"2023-11-15T02:56:51.718875+0000","last_peered":"2023-11-15T02:56:51.718875+0000","last_clean":"2023-11-15T02:56:51.718875+0000","last_became_active":"2023-11-15T02:56:51.278404+0000","last_became_peered":"2023-11-15T02:56:51.278404+0000","last_unstale":"2023-11-15T02:56:51.718875+0000","last_undegraded":"2023-11-15T02:56:51.718875+0000","last_fullsized":"2023-11-15T02:56:51.718875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745052+0000","last_change":"2023-11-15T02:56:51.741288+0000","last_active":"2023-11-15T02:56:51.745052+0000","last_peered":"2023-11-15T02:56:51.745052+0000","last_clean":"2023-11-15T02:56:51.745052+0000","last_became_active":"2023-11-15T02:56:51.740865+0000","last_became_peered":"2023-11-15T02:56:51.740865+0000","last_unstale":"2023-11-15T02:56:51.745052+0000","last_undegraded":"2023-11-15T02:56:51.745052+0000","last_fullsized":"2023-11-15T02:56:51.745052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746626+0000","last_change":"2023-11-15T02:56:51.274179+0000","last_active":"2023-11-15T02:56:51.746626+0000","last_peered":"2023-11-15T02:56:51.746626+0000","last_clean":"2023-11-15T02:56:51.746626+0000","last_became_active":"2023-11-15T02:56:51.272313+0000","last_became_peered":"2023-11-15T02:56:51.272313+0000","last_unstale":"2023-11-15T02:56:51.746626+0000","last_undegraded":"2023-11-15T02:56:51.746626+0000","last_fullsized":"2023-11-15T02:56:51.746626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.737568+0000","last_change":"2023-11-15T02:56:51.736317+0000","last_active":"2023-11-15T02:56:51.737568+0000","last_peered":"2023-11-15T02:56:51.737568+0000","last_clean":"2023-11-15T02:56:51.737568+0000","last_became_active":"2023-11-15T02:56:51.736060+0000","last_became_peered":"2023-11-15T02:56:51.736060+0000","last_unstale":"2023-11-15T02:56:51.737568+0000","last_undegraded":"2023-11-15T02:56:51.737568+0000","last_fullsized":"2023-11-15T02:56:51.737568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.754612+0000","last_change":"2023-11-15T02:56:51.279248+0000","last_active":"2023-11-15T02:56:51.754612+0000","last_peered":"2023-11-15T02:56:51.754612+0000","last_clean":"2023-11-15T02:56:51.754612+0000","last_became_active":"2023-11-15T02:56:51.278771+0000","last_became_peered":"2023-11-15T02:56:51.278771+0000","last_unstale":"2023-11-15T02:56:51.754612+0000","last_undegraded":"2023-11-15T02:56:51.754612+0000","last_fullsized":"2023-11-15T02:56:51.754612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.727542+0000","last_change":"2023-11-15T02:56:51.716546+0000","last_active":"2023-11-15T02:56:51.727542+0000","last_peered":"2023-11-15T02:56:51.727542+0000","last_clean":"2023-11-15T02:56:51.727542+0000","last_became_active":"2023-11-15T02:56:51.716254+0000","last_became_peered":"2023-11-15T02:56:51.716254+0000","last_unstale":"2023-11-15T02:56:51.727542+0000","last_undegraded":"2023-11-15T02:56:51.727542+0000","last_fullsized":"2023-11-15T02:56:51.727542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712732+0000","last_change":"2023-11-15T02:56:51.286642+0000","last_active":"2023-11-15T02:56:51.712732+0000","last_peered":"2023-11-15T02:56:51.712732+0000","last_clean":"2023-11-15T02:56:51.712732+0000","last_became_active":"2023-11-15T02:56:51.286240+0000","last_became_peered":"2023-11-15T02:56:51.286240+0000","last_unstale":"2023-11-15T02:56:51.712732+0000","last_undegraded":"2023-11-15T02:56:51.712732+0000","last_fullsized":"2023-11-15T02:56:51.712732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.725581+0000","last_change":"2023-11-15T02:56:51.722322+0000","last_active":"2023-11-15T02:56:51.725581+0000","last_peered":"2023-11-15T02:56:51.725581+0000","last_clean":"2023-11-15T02:56:51.725581+0000","last_became_active":"2023-11-15T02:56:51.721931+0000","last_became_peered":"2023-11-15T02:56:51.721931+0000","last_unstale":"2023-11-15T02:56:51.725581+0000","last_undegraded":"2023-11-15T02:56:51.725581+0000","last_fullsized":"2023-11-15T02:56:51.725581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713667+0000","last_change":"2023-11-15T02:56:51.284656+0000","last_active":"2023-11-15T02:56:51.713667+0000","last_peered":"2023-11-15T02:56:51.713667+0000","last_clean":"2023-11-15T02:56:51.713667+0000","last_became_active":"2023-11-15T02:56:51.284378+0000","last_became_peered":"2023-11-15T02:56:51.284378+0000","last_unstale":"2023-11-15T02:56:51.713667+0000","last_undegraded":"2023-11-15T02:56:51.713667+0000","last_fullsized":"2023-11-15T02:56:51.713667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747336+0000","last_change":"2023-11-15T02:56:51.743546+0000","last_active":"2023-11-15T02:56:51.747336+0000","last_peered":"2023-11-15T02:56:51.747336+0000","last_clean":"2023-11-15T02:56:51.747336+0000","last_became_active":"2023-11-15T02:56:51.743276+0000","last_became_peered":"2023-11-15T02:56:51.743276+0000","last_unstale":"2023-11-15T02:56:51.747336+0000","last_undegraded":"2023-11-15T02:56:51.747336+0000","last_fullsized":"2023-11-15T02:56:51.747336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.729927+0000","last_change":"2023-11-15T02:56:51.285769+0000","last_active":"2023-11-15T02:56:51.729927+0000","last_peered":"2023-11-15T02:56:51.729927+0000","last_clean":"2023-11-15T02:56:51.729927+0000","last_became_active":"2023-11-15T02:56:51.285448+0000","last_became_peered":"2023-11-15T02:56:51.285448+0000","last_unstale":"2023-11-15T02:56:51.729927+0000","last_undegraded":"2023-11-15T02:56:51.729927+0000","last_fullsized":"2023-11-15T02:56:51.729927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.742411+0000","last_change":"2023-11-15T02:56:51.736745+0000","last_active":"2023-11-15T02:56:51.742411+0000","last_peered":"2023-11-15T02:56:51.742411+0000","last_clean":"2023-11-15T02:56:51.742411+0000","last_became_active":"2023-11-15T02:56:51.736555+0000","last_became_peered":"2023-11-15T02:56:51.736555+0000","last_unstale":"2023-11-15T02:56:51.742411+0000","last_undegraded":"2023-11-15T02:56:51.742411+0000","last_fullsized":"2023-11-15T02:56:51.742411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730754+0000","last_change":"2023-11-15T02:56:51.722343+0000","last_active":"2023-11-15T02:56:51.730754+0000","last_peered":"2023-11-15T02:56:51.730754+0000","last_clean":"2023-11-15T02:56:51.730754+0000","last_became_active":"2023-11-15T02:56:51.722090+0000","last_became_peered":"2023-11-15T02:56:51.722090+0000","last_unstale":"2023-11-15T02:56:51.730754+0000","last_undegraded":"2023-11-15T02:56:51.730754+0000","last_fullsized":"2023-11-15T02:56:51.730754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.710708+0000","last_change":"2023-11-15T02:56:51.281122+0000","last_active":"2023-11-15T02:56:51.710708+0000","last_peered":"2023-11-15T02:56:51.710708+0000","last_clean":"2023-11-15T02:56:51.710708+0000","last_became_active":"2023-11-15T02:56:51.280798+0000","last_became_peered":"2023-11-15T02:56:51.280798+0000","last_unstale":"2023-11-15T02:56:51.710708+0000","last_undegraded":"2023-11-15T02:56:51.710708+0000","last_fullsized":"2023-11-15T02:56:51.710708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.729313+0000","last_change":"2023-11-15T02:56:51.726756+0000","last_active":"2023-11-15T02:56:51.729313+0000","last_peered":"2023-11-15T02:56:51.729313+0000","last_clean":"2023-11-15T02:56:51.729313+0000","last_became_active":"2023-11-15T02:56:51.726446+0000","last_became_peered":"2023-11-15T02:56:51.726446+0000","last_unstale":"2023-11-15T02:56:51.729313+0000","last_undegraded":"2023-11-15T02:56:51.729313+0000","last_fullsized":"2023-11-15T02:56:51.729313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730560+0000","last_change":"2023-11-15T02:56:51.288389+0000","last_active":"2023-11-15T02:56:51.730560+0000","last_peered":"2023-11-15T02:56:51.730560+0000","last_clean":"2023-11-15T02:56:51.730560+0000","last_became_active":"2023-11-15T02:56:51.287933+0000","last_became_peered":"2023-11-15T02:56:51.287933+0000","last_unstale":"2023-11-15T02:56:51.730560+0000","last_undegraded":"2023-11-15T02:56:51.730560+0000","last_fullsized":"2023-11-15T02:56:51.730560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746552+0000","last_change":"2023-11-15T02:56:51.742907+0000","last_active":"2023-11-15T02:56:51.746552+0000","last_peered":"2023-11-15T02:56:51.746552+0000","last_clean":"2023-11-15T02:56:51.746552+0000","last_became_active":"2023-11-15T02:56:51.742633+0000","last_became_peered":"2023-11-15T02:56:51.742633+0000","last_unstale":"2023-11-15T02:56:51.746552+0000","last_undegraded":"2023-11-15T02:56:51.746552+0000","last_fullsized":"2023-11-15T02:56:51.746552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.736908+0000","last_change":"2023-11-15T02:56:51.278389+0000","last_active":"2023-11-15T02:56:51.736908+0000","last_peered":"2023-11-15T02:56:51.736908+0000","last_clean":"2023-11-15T02:56:51.736908+0000","last_became_active":"2023-11-15T02:56:51.277985+0000","last_became_peered":"2023-11-15T02:56:51.277985+0000","last_unstale":"2023-11-15T02:56:51.736908+0000","last_undegraded":"2023-11-15T02:56:51.736908+0000","last_fullsized":"2023-11-15T02:56:51.736908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.743771+0000","last_change":"2023-11-15T02:56:51.725491+0000","last_active":"2023-11-15T02:56:51.743771+0000","last_peered":"2023-11-15T02:56:51.743771+0000","last_clean":"2023-11-15T02:56:51.743771+0000","last_became_active":"2023-11-15T02:56:51.725072+0000","last_became_peered":"2023-11-15T02:56:51.725072+0000","last_unstale":"2023-11-15T02:56:51.743771+0000","last_undegraded":"2023-11-15T02:56:51.743771+0000","last_fullsized":"2023-11-15T02:56:51.743771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738751+0000","last_change":"2023-11-15T02:56:51.291272+0000","last_active":"2023-11-15T02:56:51.738751+0000","last_peered":"2023-11-15T02:56:51.738751+0000","last_clean":"2023-11-15T02:56:51.738751+0000","last_became_active":"2023-11-15T02:56:51.287508+0000","last_became_peered":"2023-11-15T02:56:51.287508+0000","last_unstale":"2023-11-15T02:56:51.738751+0000","last_undegraded":"2023-11-15T02:56:51.738751+0000","last_fullsized":"2023-11-15T02:56:51.738751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745082+0000","last_change":"2023-11-15T02:56:51.741837+0000","last_active":"2023-11-15T02:56:51.745082+0000","last_peered":"2023-11-15T02:56:51.745082+0000","last_clean":"2023-11-15T02:56:51.745082+0000","last_became_active":"2023-11-15T02:56:51.741487+0000","last_became_peered":"2023-11-15T02:56:51.741487+0000","last_unstale":"2023-11-15T02:56:51.745082+0000","last_undegraded":"2023-11-15T02:56:51.745082+0000","last_fullsized":"2023-11-15T02:56:51.745082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.741740+0000","last_change":"2023-11-15T02:56:51.283591+0000","last_active":"2023-11-15T02:56:51.741740+0000","last_peered":"2023-11-15T02:56:51.741740+0000","last_clean":"2023-11-15T02:56:51.741740+0000","last_became_active":"2023-11-15T02:56:51.283198+0000","last_became_peered":"2023-11-15T02:56:51.283198+0000","last_unstale":"2023-11-15T02:56:51.741740+0000","last_undegraded":"2023-11-15T02:56:51.741740+0000","last_fullsized":"2023-11-15T02:56:51.741740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744198+0000","last_change":"2023-11-15T02:56:51.739304+0000","last_active":"2023-11-15T02:56:51.744198+0000","last_peered":"2023-11-15T02:56:51.744198+0000","last_clean":"2023-11-15T02:56:51.744198+0000","last_became_active":"2023-11-15T02:56:51.738755+0000","last_became_peered":"2023-11-15T02:56:51.738755+0000","last_unstale":"2023-11-15T02:56:51.744198+0000","last_undegraded":"2023-11-15T02:56:51.744198+0000","last_fullsized":"2023-11-15T02:56:51.744198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.722852+0000","last_change":"2023-11-15T02:56:51.269310+0000","last_active":"2023-11-15T02:56:51.722852+0000","last_peered":"2023-11-15T02:56:51.722852+0000","last_clean":"2023-11-15T02:56:51.722852+0000","last_became_active":"2023-11-15T02:56:51.268977+0000","last_became_peered":"2023-11-15T02:56:51.268977+0000","last_unstale":"2023-11-15T02:56:51.722852+0000","last_undegraded":"2023-11-15T02:56:51.722852+0000","last_fullsized":"2023-11-15T02:56:51.722852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747235+0000","last_change":"2023-11-15T02:56:51.742002+0000","last_active":"2023-11-15T02:56:51.747235+0000","last_peered":"2023-11-15T02:56:51.747235+0000","last_clean":"2023-11-15T02:56:51.747235+0000","last_became_active":"2023-11-15T02:56:51.741691+0000","last_became_peered":"2023-11-15T02:56:51.741691+0000","last_unstale":"2023-11-15T02:56:51.747235+0000","last_undegraded":"2023-11-15T02:56:51.747235+0000","last_fullsized":"2023-11-15T02:56:51.747235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.706447+0000","last_change":"2023-11-15T02:56:51.280160+0000","last_active":"2023-11-15T02:56:51.706447+0000","last_peered":"2023-11-15T02:56:51.706447+0000","last_clean":"2023-11-15T02:56:51.706447+0000","last_became_active":"2023-11-15T02:56:51.278364+0000","last_became_peered":"2023-11-15T02:56:51.278364+0000","last_unstale":"2023-11-15T02:56:51.706447+0000","last_undegraded":"2023-11-15T02:56:51.706447+0000","last_fullsized":"2023-11-15T02:56:51.706447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747191+0000","last_change":"2023-11-15T02:56:51.742487+0000","last_active":"2023-11-15T02:56:51.747191+0000","last_peered":"2023-11-15T02:56:51.747191+0000","last_clean":"2023-11-15T02:56:51.747191+0000","last_became_active":"2023-11-15T02:56:51.742056+0000","last_became_peered":"2023-11-15T02:56:51.742056+0000","last_unstale":"2023-11-15T02:56:51.747191+0000","last_undegraded":"2023-11-15T02:56:51.747191+0000","last_fullsized":"2023-11-15T02:56:51.747191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.748052+0000","last_change":"2023-11-15T02:56:51.290964+0000","last_active":"2023-11-15T02:56:51.748052+0000","last_peered":"2023-11-15T02:56:51.748052+0000","last_clean":"2023-11-15T02:56:51.748052+0000","last_became_active":"2023-11-15T02:56:51.290728+0000","last_became_peered":"2023-11-15T02:56:51.290728+0000","last_unstale":"2023-11-15T02:56:51.748052+0000","last_undegraded":"2023-11-15T02:56:51.748052+0000","last_fullsized":"2023-11-15T02:56:51.748052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297332,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074508,"statfs":{"total":96636764160,"available":96332296192,"internally_reserved":0,"allocated":241664,"data_stored":80461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2769999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297312,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074528,"statfs":{"total":96636764160,"available":96332316672,"internally_reserved":0,"allocated":221184,"data_stored":80493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4590000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297324,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074516,"statfs":{"total":96636764160,"available":96332304384,"internally_reserved":0,"allocated":233472,"data_stored":80639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297308,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074532,"statfs":{"total":96636764160,"available":96332320768,"internally_reserved":0,"allocated":217088,"data_stored":79137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.712}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.794}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7769999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":77848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.472}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3140000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5640000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":79047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.341}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":79047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.224}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":79047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T02:57:04.282 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T02:57:04.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.471+0000 7f044d4f1700 1 Processor -- start 2023-11-15T02:57:04.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.471+0000 7f044d4f1700 1 -- start start 2023-11-15T02:57:04.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.475+0000 7f044d4f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04480ac5d0 0x7f04480c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:04.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.475+0000 7f044d4f1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f04480c39b0 0x7f04480c1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:04.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.475+0000 7f044d4f1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0448066950 con 0x7f04480ac1f0 2023-11-15T02:57:04.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.475+0000 7f044d4f1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0448158790 con 0x7f04480c39b0 2023-11-15T02:57:04.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.475+0000 7f044d4f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04480674f0 con 0x7f04480ac5d0 2023-11-15T02:57:04.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.475+0000 7f04477fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f04480c39b0 0x7f04480c1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:04.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.475+0000 7f0446ffd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f04480ac1f0 0x7f0448066de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:59410/0 (socket says 172.21.15.121:59410) 2023-11-15T02:57:04.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.475+0000 7f0446ffd700 1 -- 172.21.15.121:0/3310081887 learned_addr learned my addr 172.21.15.121:0/3310081887 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:04.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.475+0000 7f04467fc700 1 --2- 172.21.15.121:0/3310081887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04480ac5d0 0x7f04480c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:04.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.475+0000 7f0445ffb700 1 -- 172.21.15.121:0/3310081887 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252565751 0 0) 0x7f0448066950 con 0x7f04480ac1f0 2023-11-15T02:57:04.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.475+0000 7f0445ffb700 1 -- 172.21.15.121:0/3310081887 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04300033f0 con 0x7f04480ac1f0 2023-11-15T02:57:04.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.475+0000 7f04477fe700 1 -- 172.21.15.121:0/3310081887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04480ac5d0 msgr2=0x7f04480c34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:04.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.475+0000 7f04477fe700 1 --2- 172.21.15.121:0/3310081887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04480ac5d0 0x7f04480c34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.475+0000 7f04477fe700 1 -- 172.21.15.121:0/3310081887 >> v1:172.21.15.190:6789/0 conn(0x7f04480ac1f0 legacy=0x7f0448066de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:04.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.475+0000 7f04477fe700 1 -- 172.21.15.121:0/3310081887 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0448065d40 con 0x7f04480c39b0 2023-11-15T02:57:04.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.475+0000 7f04477fe700 1 --2- 172.21.15.121:0/3310081887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04480c39b0 0x7f04480c1660 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f0440009e60 tx=0x7f0440009780).ready entity=mon.0 client_cookie=ccd203c02ca3b815 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:04.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.479+0000 7f0445ffb700 1 -- 172.21.15.121:0/3310081887 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044000e070 con 0x7f04480c39b0 2023-11-15T02:57:04.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.479+0000 7f0445ffb700 1 -- 172.21.15.121:0/3310081887 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044000bab0 con 0x7f04480c39b0 2023-11-15T02:57:04.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.479+0000 7f0445ffb700 1 -- 172.21.15.121:0/3310081887 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044000be30 con 0x7f04480c39b0 2023-11-15T02:57:04.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.479+0000 7f044d4f1700 1 -- 172.21.15.121:0/3310081887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04480c39b0 msgr2=0x7f04480c1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:04.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.479+0000 7f044d4f1700 1 --2- 172.21.15.121:0/3310081887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04480c39b0 0x7f04480c1660 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f0440009e60 tx=0x7f0440009780).stop 2023-11-15T02:57:04.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.479+0000 7f044d4f1700 1 -- 172.21.15.121:0/3310081887 shutdown_connections 2023-11-15T02:57:04.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.479+0000 7f044d4f1700 1 --2- 172.21.15.121:0/3310081887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04480c39b0 0x7f04480c1660 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.479+0000 7f044d4f1700 1 --2- 172.21.15.121:0/3310081887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04480ac5d0 0x7f04480c34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.479+0000 7f044d4f1700 1 -- 172.21.15.121:0/3310081887 >> 172.21.15.121:0/3310081887 conn(0x7f0448062190 msgr2=0x7f0448062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:04.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.479+0000 7f044d4f1700 1 -- 172.21.15.121:0/3310081887 shutdown_connections 2023-11-15T02:57:04.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.479+0000 7f044d4f1700 1 -- 172.21.15.121:0/3310081887 wait complete. 2023-11-15T02:57:04.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.483+0000 7f044d4f1700 1 Processor -- start 2023-11-15T02:57:04.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.483+0000 7f044d4f1700 1 -- start start 2023-11-15T02:57:04.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.483+0000 7f044d4f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04480ac1f0 0x7f04480bdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:04.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.483+0000 7f044d4f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04480ac5d0 0x7f04480c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:04.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.483+0000 7f044d4f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04480c39b0 0x7f04480c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:04.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.483+0000 7f044d4f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04480bf1b0 con 0x7f04480c39b0 2023-11-15T02:57:04.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.483+0000 7f044d4f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04480be5a0 con 0x7f04480ac5d0 2023-11-15T02:57:04.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.483+0000 7f044d4f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04480be880 con 0x7f04480ac1f0 2023-11-15T02:57:04.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.483+0000 7f04477fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04480c39b0 0x7f04480c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:04.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.483+0000 7f0446ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04480ac1f0 0x7f04480bdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:04.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.483+0000 7f04477fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04480c39b0 0x7f04480c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33332/0 (socket says 172.21.15.121:33332) 2023-11-15T02:57:04.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.483+0000 7f04477fe700 1 -- 172.21.15.121:0/349020530 learned_addr learned my addr 172.21.15.121:0/349020530 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:04.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.483+0000 7f04467fc700 1 --2- 172.21.15.121:0/349020530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04480ac5d0 0x7f04480c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:04.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.483+0000 7f04477fe700 1 -- 172.21.15.121:0/349020530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04480ac1f0 msgr2=0x7f04480bdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:04.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.483+0000 7f04477fe700 1 --2- 172.21.15.121:0/349020530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04480ac1f0 0x7f04480bdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.483+0000 7f04477fe700 1 -- 172.21.15.121:0/349020530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04480ac5d0 msgr2=0x7f04480c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:04.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.483+0000 7f04477fe700 1 --2- 172.21.15.121:0/349020530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04480ac5d0 0x7f04480c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.483+0000 7f04477fe700 1 -- 172.21.15.121:0/349020530 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0440009a20 con 0x7f04480c39b0 2023-11-15T02:57:04.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.487+0000 7f04477fe700 1 --2- 172.21.15.121:0/349020530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04480c39b0 0x7f04480c5b80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f04480c77b0 tx=0x7f0440009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:04.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.487+0000 7f042effd700 1 -- 172.21.15.121:0/349020530 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440013ea0 con 0x7f04480c39b0 2023-11-15T02:57:04.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.487+0000 7f042effd700 1 -- 172.21.15.121:0/349020530 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044001d630 con 0x7f04480c39b0 2023-11-15T02:57:04.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.487+0000 7f044d4f1700 1 -- 172.21.15.121:0/349020530 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04480beb60 con 0x7f04480c39b0 2023-11-15T02:57:04.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.487+0000 7f042effd700 1 -- 172.21.15.121:0/349020530 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044001d9f0 con 0x7f04480c39b0 2023-11-15T02:57:04.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.487+0000 7f044d4f1700 1 -- 172.21.15.121:0/349020530 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0448157d20 con 0x7f04480c39b0 2023-11-15T02:57:04.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.487+0000 7f044d4f1700 1 -- 172.21.15.121:0/349020530 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f044806bcf0 con 0x7f04480c39b0 2023-11-15T02:57:04.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.487+0000 7f042effd700 1 -- 172.21.15.121:0/349020530 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0440003040 con 0x7f04480c39b0 2023-11-15T02:57:04.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.491+0000 7f042effd700 1 --2- 172.21.15.121:0/349020530 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f043405a510 0x7f043405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:04.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.491+0000 7f042effd700 1 -- 172.21.15.121:0/349020530 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0440076490 con 0x7f04480c39b0 2023-11-15T02:57:04.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.491+0000 7f0446ffd700 1 --2- 172.21.15.121:0/349020530 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f043405a510 0x7f043405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:04.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.491+0000 7f0446ffd700 1 --2- 172.21.15.121:0/349020530 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f043405a510 0x7f043405c9d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f044816b770 tx=0x7f0438009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:04.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.495+0000 7f042effd700 1 -- 172.21.15.121:0/349020530 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0440049d20 con 0x7f04480c39b0 2023-11-15T02:57:04.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.651+0000 7f044d4f1700 1 -- 172.21.15.121:0/349020530 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f04480bf2f0 con 0x7f043405a510 2023-11-15T02:57:04.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.659+0000 7f042effd700 1 -- 172.21.15.121:0/349020530 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336066 (secure 0 0 0) 0x7f0438001510 con 0x7f043405a510 2023-11-15T02:57:04.671 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:04.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.667+0000 7f044d4f1700 1 -- 172.21.15.121:0/349020530 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f043405a510 msgr2=0x7f043405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:04.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.667+0000 7f044d4f1700 1 --2- 172.21.15.121:0/349020530 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f043405a510 0x7f043405c9d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f044816b770 tx=0x7f0438009040).stop 2023-11-15T02:57:04.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.667+0000 7f044d4f1700 1 -- 172.21.15.121:0/349020530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04480c39b0 msgr2=0x7f04480c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:04.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.667+0000 7f044d4f1700 1 --2- 172.21.15.121:0/349020530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04480c39b0 0x7f04480c5b80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f04480c77b0 tx=0x7f0440009780).stop 2023-11-15T02:57:04.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.667+0000 7f044d4f1700 1 -- 172.21.15.121:0/349020530 shutdown_connections 2023-11-15T02:57:04.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.667+0000 7f044d4f1700 1 --2- 172.21.15.121:0/349020530 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f043405a510 0x7f043405c9d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.667+0000 7f044d4f1700 1 --2- 172.21.15.121:0/349020530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04480c39b0 0x7f04480c5b80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.667+0000 7f044d4f1700 1 --2- 172.21.15.121:0/349020530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04480ac5d0 0x7f04480c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.667+0000 7f044d4f1700 1 --2- 172.21.15.121:0/349020530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04480ac1f0 0x7f04480bdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.667+0000 7f044d4f1700 1 -- 172.21.15.121:0/349020530 >> 172.21.15.121:0/349020530 conn(0x7f0448062190 msgr2=0x7f044815a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:04.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.667+0000 7f044d4f1700 1 -- 172.21.15.121:0/349020530 shutdown_connections 2023-11-15T02:57:04.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.667+0000 7f044d4f1700 1 -- 172.21.15.121:0/349020530 wait complete. 2023-11-15T02:57:04.677 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T02:57:04.694 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-15T02:57:03.916329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2378492,"kb_used_data":1724,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752596228,"statfs":{"total":773094113280,"available":770658537472,"internally_reserved":0,"allocated":1765376,"data_stored":635719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003190"},"pg_stats":[{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.736895+0000","last_change":"2023-11-15T02:56:51.733024+0000","last_active":"2023-11-15T02:56:51.736895+0000","last_peered":"2023-11-15T02:56:51.736895+0000","last_clean":"2023-11-15T02:56:51.736895+0000","last_became_active":"2023-11-15T02:56:51.732735+0000","last_became_peered":"2023-11-15T02:56:51.732735+0000","last_unstale":"2023-11-15T02:56:51.736895+0000","last_undegraded":"2023-11-15T02:56:51.736895+0000","last_fullsized":"2023-11-15T02:56:51.736895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.704467+0000","last_change":"2023-11-15T02:56:51.273328+0000","last_active":"2023-11-15T02:56:51.704467+0000","last_peered":"2023-11-15T02:56:51.704467+0000","last_clean":"2023-11-15T02:56:51.704467+0000","last_became_active":"2023-11-15T02:56:51.273044+0000","last_became_peered":"2023-11-15T02:56:51.273044+0000","last_unstale":"2023-11-15T02:56:51.704467+0000","last_undegraded":"2023-11-15T02:56:51.704467+0000","last_fullsized":"2023-11-15T02:56:51.704467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744601+0000","last_change":"2023-11-15T02:56:51.742242+0000","last_active":"2023-11-15T02:56:51.744601+0000","last_peered":"2023-11-15T02:56:51.744601+0000","last_clean":"2023-11-15T02:56:51.744601+0000","last_became_active":"2023-11-15T02:56:51.741604+0000","last_became_peered":"2023-11-15T02:56:51.741604+0000","last_unstale":"2023-11-15T02:56:51.744601+0000","last_undegraded":"2023-11-15T02:56:51.744601+0000","last_fullsized":"2023-11-15T02:56:51.744601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.739512+0000","last_change":"2023-11-15T02:56:51.277577+0000","last_active":"2023-11-15T02:56:51.739512+0000","last_peered":"2023-11-15T02:56:51.739512+0000","last_clean":"2023-11-15T02:56:51.739512+0000","last_became_active":"2023-11-15T02:56:51.276745+0000","last_became_peered":"2023-11-15T02:56:51.276745+0000","last_unstale":"2023-11-15T02:56:51.739512+0000","last_undegraded":"2023-11-15T02:56:51.739512+0000","last_fullsized":"2023-11-15T02:56:51.739512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.716112+0000","last_change":"2023-11-15T02:56:51.269238+0000","last_active":"2023-11-15T02:56:51.716112+0000","last_peered":"2023-11-15T02:56:51.716112+0000","last_clean":"2023-11-15T02:56:51.716112+0000","last_became_active":"2023-11-15T02:56:51.268872+0000","last_became_peered":"2023-11-15T02:56:51.268872+0000","last_unstale":"2023-11-15T02:56:51.716112+0000","last_undegraded":"2023-11-15T02:56:51.716112+0000","last_fullsized":"2023-11-15T02:56:51.716112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747275+0000","last_change":"2023-11-15T02:56:51.742674+0000","last_active":"2023-11-15T02:56:51.747275+0000","last_peered":"2023-11-15T02:56:51.747275+0000","last_clean":"2023-11-15T02:56:51.747275+0000","last_became_active":"2023-11-15T02:56:51.742412+0000","last_became_peered":"2023-11-15T02:56:51.742412+0000","last_unstale":"2023-11-15T02:56:51.747275+0000","last_undegraded":"2023-11-15T02:56:51.747275+0000","last_fullsized":"2023-11-15T02:56:51.747275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712980+0000","last_change":"2023-11-15T02:56:51.279772+0000","last_active":"2023-11-15T02:56:51.712980+0000","last_peered":"2023-11-15T02:56:51.712980+0000","last_clean":"2023-11-15T02:56:51.712980+0000","last_became_active":"2023-11-15T02:56:51.279423+0000","last_became_peered":"2023-11-15T02:56:51.279423+0000","last_unstale":"2023-11-15T02:56:51.712980+0000","last_undegraded":"2023-11-15T02:56:51.712980+0000","last_fullsized":"2023-11-15T02:56:51.712980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738160+0000","last_change":"2023-11-15T02:56:51.731913+0000","last_active":"2023-11-15T02:56:51.738160+0000","last_peered":"2023-11-15T02:56:51.738160+0000","last_clean":"2023-11-15T02:56:51.738160+0000","last_became_active":"2023-11-15T02:56:51.728632+0000","last_became_peered":"2023-11-15T02:56:51.728632+0000","last_unstale":"2023-11-15T02:56:51.738160+0000","last_undegraded":"2023-11-15T02:56:51.738160+0000","last_fullsized":"2023-11-15T02:56:51.738160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744972+0000","last_change":"2023-11-15T02:56:51.289877+0000","last_active":"2023-11-15T02:56:51.744972+0000","last_peered":"2023-11-15T02:56:51.744972+0000","last_clean":"2023-11-15T02:56:51.744972+0000","last_became_active":"2023-11-15T02:56:51.288940+0000","last_became_peered":"2023-11-15T02:56:51.288940+0000","last_unstale":"2023-11-15T02:56:51.744972+0000","last_undegraded":"2023-11-15T02:56:51.744972+0000","last_fullsized":"2023-11-15T02:56:51.744972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.743484+0000","last_change":"2023-11-15T02:56:51.738869+0000","last_active":"2023-11-15T02:56:51.743484+0000","last_peered":"2023-11-15T02:56:51.743484+0000","last_clean":"2023-11-15T02:56:51.743484+0000","last_became_active":"2023-11-15T02:56:51.738159+0000","last_became_peered":"2023-11-15T02:56:51.738159+0000","last_unstale":"2023-11-15T02:56:51.743484+0000","last_undegraded":"2023-11-15T02:56:51.743484+0000","last_fullsized":"2023-11-15T02:56:51.743484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713144+0000","last_change":"2023-11-15T02:56:51.284304+0000","last_active":"2023-11-15T02:56:51.713144+0000","last_peered":"2023-11-15T02:56:51.713144+0000","last_clean":"2023-11-15T02:56:51.713144+0000","last_became_active":"2023-11-15T02:56:51.284043+0000","last_became_peered":"2023-11-15T02:56:51.284043+0000","last_unstale":"2023-11-15T02:56:51.713144+0000","last_undegraded":"2023-11-15T02:56:51.713144+0000","last_fullsized":"2023-11-15T02:56:51.713144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747361+0000","last_change":"2023-11-15T02:56:51.744243+0000","last_active":"2023-11-15T02:56:51.747361+0000","last_peered":"2023-11-15T02:56:51.747361+0000","last_clean":"2023-11-15T02:56:51.747361+0000","last_became_active":"2023-11-15T02:56:51.743943+0000","last_became_peered":"2023-11-15T02:56:51.743943+0000","last_unstale":"2023-11-15T02:56:51.747361+0000","last_undegraded":"2023-11-15T02:56:51.747361+0000","last_fullsized":"2023-11-15T02:56:51.747361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.734769+0000","last_change":"2023-11-15T02:56:51.286898+0000","last_active":"2023-11-15T02:56:51.734769+0000","last_peered":"2023-11-15T02:56:51.734769+0000","last_clean":"2023-11-15T02:56:51.734769+0000","last_became_active":"2023-11-15T02:56:51.286628+0000","last_became_peered":"2023-11-15T02:56:51.286628+0000","last_unstale":"2023-11-15T02:56:51.734769+0000","last_undegraded":"2023-11-15T02:56:51.734769+0000","last_fullsized":"2023-11-15T02:56:51.734769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.749549+0000","last_change":"2023-11-15T02:56:51.745783+0000","last_active":"2023-11-15T02:56:51.749549+0000","last_peered":"2023-11-15T02:56:51.749549+0000","last_clean":"2023-11-15T02:56:51.749549+0000","last_became_active":"2023-11-15T02:56:51.745028+0000","last_became_peered":"2023-11-15T02:56:51.745028+0000","last_unstale":"2023-11-15T02:56:51.749549+0000","last_undegraded":"2023-11-15T02:56:51.749549+0000","last_fullsized":"2023-11-15T02:56:51.749549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.716637+0000","last_change":"2023-11-15T02:56:51.268012+0000","last_active":"2023-11-15T02:56:51.716637+0000","last_peered":"2023-11-15T02:56:51.716637+0000","last_clean":"2023-11-15T02:56:51.716637+0000","last_became_active":"2023-11-15T02:56:51.267734+0000","last_became_peered":"2023-11-15T02:56:51.267734+0000","last_unstale":"2023-11-15T02:56:51.716637+0000","last_undegraded":"2023-11-15T02:56:51.716637+0000","last_fullsized":"2023-11-15T02:56:51.716637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730642+0000","last_change":"2023-11-15T02:56:51.720753+0000","last_active":"2023-11-15T02:56:51.730642+0000","last_peered":"2023-11-15T02:56:51.730642+0000","last_clean":"2023-11-15T02:56:51.730642+0000","last_became_active":"2023-11-15T02:56:51.720500+0000","last_became_peered":"2023-11-15T02:56:51.720500+0000","last_unstale":"2023-11-15T02:56:51.730642+0000","last_undegraded":"2023-11-15T02:56:51.730642+0000","last_fullsized":"2023-11-15T02:56:51.730642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.711624+0000","last_change":"2023-11-15T02:56:51.280573+0000","last_active":"2023-11-15T02:56:51.711624+0000","last_peered":"2023-11-15T02:56:51.711624+0000","last_clean":"2023-11-15T02:56:51.711624+0000","last_became_active":"2023-11-15T02:56:51.275286+0000","last_became_peered":"2023-11-15T02:56:51.275286+0000","last_unstale":"2023-11-15T02:56:51.711624+0000","last_undegraded":"2023-11-15T02:56:51.711624+0000","last_fullsized":"2023-11-15T02:56:51.711624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.742388+0000","last_change":"2023-11-15T02:56:51.736450+0000","last_active":"2023-11-15T02:56:51.742388+0000","last_peered":"2023-11-15T02:56:51.742388+0000","last_clean":"2023-11-15T02:56:51.742388+0000","last_became_active":"2023-11-15T02:56:51.736143+0000","last_became_peered":"2023-11-15T02:56:51.736143+0000","last_unstale":"2023-11-15T02:56:51.742388+0000","last_undegraded":"2023-11-15T02:56:51.742388+0000","last_fullsized":"2023-11-15T02:56:51.742388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.714425+0000","last_change":"2023-11-15T02:56:51.284434+0000","last_active":"2023-11-15T02:56:51.714425+0000","last_peered":"2023-11-15T02:56:51.714425+0000","last_clean":"2023-11-15T02:56:51.714425+0000","last_became_active":"2023-11-15T02:56:51.284051+0000","last_became_peered":"2023-11-15T02:56:51.284051+0000","last_unstale":"2023-11-15T02:56:51.714425+0000","last_undegraded":"2023-11-15T02:56:51.714425+0000","last_fullsized":"2023-11-15T02:56:51.714425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744584+0000","last_change":"2023-11-15T02:56:51.742247+0000","last_active":"2023-11-15T02:56:51.744584+0000","last_peered":"2023-11-15T02:56:51.744584+0000","last_clean":"2023-11-15T02:56:51.744584+0000","last_became_active":"2023-11-15T02:56:51.741966+0000","last_became_peered":"2023-11-15T02:56:51.741966+0000","last_unstale":"2023-11-15T02:56:51.744584+0000","last_undegraded":"2023-11-15T02:56:51.744584+0000","last_fullsized":"2023-11-15T02:56:51.744584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.714493+0000","last_change":"2023-11-15T02:56:51.277678+0000","last_active":"2023-11-15T02:56:51.714493+0000","last_peered":"2023-11-15T02:56:51.714493+0000","last_clean":"2023-11-15T02:56:51.714493+0000","last_became_active":"2023-11-15T02:56:51.277408+0000","last_became_peered":"2023-11-15T02:56:51.277408+0000","last_unstale":"2023-11-15T02:56:51.714493+0000","last_undegraded":"2023-11-15T02:56:51.714493+0000","last_fullsized":"2023-11-15T02:56:51.714493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.731195+0000","last_change":"2023-11-15T02:56:51.717008+0000","last_active":"2023-11-15T02:56:51.731195+0000","last_peered":"2023-11-15T02:56:51.731195+0000","last_clean":"2023-11-15T02:56:51.731195+0000","last_became_active":"2023-11-15T02:56:51.716661+0000","last_became_peered":"2023-11-15T02:56:51.716661+0000","last_unstale":"2023-11-15T02:56:51.731195+0000","last_undegraded":"2023-11-15T02:56:51.731195+0000","last_fullsized":"2023-11-15T02:56:51.731195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-15T02:56:53.093563+0000","last_change":"2023-11-15T02:56:51.281739+0000","last_active":"2023-11-15T02:56:53.093563+0000","last_peered":"2023-11-15T02:56:53.093563+0000","last_clean":"2023-11-15T02:56:53.093563+0000","last_became_active":"2023-11-15T02:56:51.281484+0000","last_became_peered":"2023-11-15T02:56:51.281484+0000","last_unstale":"2023-11-15T02:56:53.093563+0000","last_undegraded":"2023-11-15T02:56:53.093563+0000","last_fullsized":"2023-11-15T02:56:53.093563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738099+0000","last_change":"2023-11-15T02:56:51.726671+0000","last_active":"2023-11-15T02:56:51.738099+0000","last_peered":"2023-11-15T02:56:51.738099+0000","last_clean":"2023-11-15T02:56:51.738099+0000","last_became_active":"2023-11-15T02:56:51.725108+0000","last_became_peered":"2023-11-15T02:56:51.725108+0000","last_unstale":"2023-11-15T02:56:51.738099+0000","last_undegraded":"2023-11-15T02:56:51.738099+0000","last_fullsized":"2023-11-15T02:56:51.738099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.704634+0000","last_change":"2023-11-15T02:56:51.271748+0000","last_active":"2023-11-15T02:56:51.704634+0000","last_peered":"2023-11-15T02:56:51.704634+0000","last_clean":"2023-11-15T02:56:51.704634+0000","last_became_active":"2023-11-15T02:56:51.271310+0000","last_became_peered":"2023-11-15T02:56:51.271310+0000","last_unstale":"2023-11-15T02:56:51.704634+0000","last_undegraded":"2023-11-15T02:56:51.704634+0000","last_fullsized":"2023-11-15T02:56:51.704634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744179+0000","last_change":"2023-11-15T02:56:51.737961+0000","last_active":"2023-11-15T02:56:51.744179+0000","last_peered":"2023-11-15T02:56:51.744179+0000","last_clean":"2023-11-15T02:56:51.744179+0000","last_became_active":"2023-11-15T02:56:51.737603+0000","last_became_peered":"2023-11-15T02:56:51.737603+0000","last_unstale":"2023-11-15T02:56:51.744179+0000","last_undegraded":"2023-11-15T02:56:51.744179+0000","last_fullsized":"2023-11-15T02:56:51.744179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.711878+0000","last_change":"2023-11-15T02:56:51.286344+0000","last_active":"2023-11-15T02:56:51.711878+0000","last_peered":"2023-11-15T02:56:51.711878+0000","last_clean":"2023-11-15T02:56:51.711878+0000","last_became_active":"2023-11-15T02:56:51.286050+0000","last_became_peered":"2023-11-15T02:56:51.286050+0000","last_unstale":"2023-11-15T02:56:51.711878+0000","last_undegraded":"2023-11-15T02:56:51.711878+0000","last_fullsized":"2023-11-15T02:56:51.711878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.740796+0000","last_change":"2023-11-15T02:56:51.729804+0000","last_active":"2023-11-15T02:56:51.740796+0000","last_peered":"2023-11-15T02:56:51.740796+0000","last_clean":"2023-11-15T02:56:51.740796+0000","last_became_active":"2023-11-15T02:56:51.729569+0000","last_became_peered":"2023-11-15T02:56:51.729569+0000","last_unstale":"2023-11-15T02:56:51.740796+0000","last_undegraded":"2023-11-15T02:56:51.740796+0000","last_fullsized":"2023-11-15T02:56:51.740796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.721025+0000","last_change":"2023-11-15T02:56:51.280678+0000","last_active":"2023-11-15T02:56:51.721025+0000","last_peered":"2023-11-15T02:56:51.721025+0000","last_clean":"2023-11-15T02:56:51.721025+0000","last_became_active":"2023-11-15T02:56:51.279110+0000","last_became_peered":"2023-11-15T02:56:51.279110+0000","last_unstale":"2023-11-15T02:56:51.721025+0000","last_undegraded":"2023-11-15T02:56:51.721025+0000","last_fullsized":"2023-11-15T02:56:51.721025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.728488+0000","last_change":"2023-11-15T02:56:51.726563+0000","last_active":"2023-11-15T02:56:51.728488+0000","last_peered":"2023-11-15T02:56:51.728488+0000","last_clean":"2023-11-15T02:56:51.728488+0000","last_became_active":"2023-11-15T02:56:51.726284+0000","last_became_peered":"2023-11-15T02:56:51.726284+0000","last_unstale":"2023-11-15T02:56:51.728488+0000","last_undegraded":"2023-11-15T02:56:51.728488+0000","last_fullsized":"2023-11-15T02:56:51.728488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.716713+0000","last_change":"2023-11-15T02:56:51.275575+0000","last_active":"2023-11-15T02:56:51.716713+0000","last_peered":"2023-11-15T02:56:51.716713+0000","last_clean":"2023-11-15T02:56:51.716713+0000","last_became_active":"2023-11-15T02:56:51.275267+0000","last_became_peered":"2023-11-15T02:56:51.275267+0000","last_unstale":"2023-11-15T02:56:51.716713+0000","last_undegraded":"2023-11-15T02:56:51.716713+0000","last_fullsized":"2023-11-15T02:56:51.716713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.732898+0000","last_change":"2023-11-15T02:56:51.728149+0000","last_active":"2023-11-15T02:56:51.732898+0000","last_peered":"2023-11-15T02:56:51.732898+0000","last_clean":"2023-11-15T02:56:51.732898+0000","last_became_active":"2023-11-15T02:56:51.727553+0000","last_became_peered":"2023-11-15T02:56:51.727553+0000","last_unstale":"2023-11-15T02:56:51.732898+0000","last_undegraded":"2023-11-15T02:56:51.732898+0000","last_fullsized":"2023-11-15T02:56:51.732898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712886+0000","last_change":"2023-11-15T02:56:51.269197+0000","last_active":"2023-11-15T02:56:51.712886+0000","last_peered":"2023-11-15T02:56:51.712886+0000","last_clean":"2023-11-15T02:56:51.712886+0000","last_became_active":"2023-11-15T02:56:51.268904+0000","last_became_peered":"2023-11-15T02:56:51.268904+0000","last_unstale":"2023-11-15T02:56:51.712886+0000","last_undegraded":"2023-11-15T02:56:51.712886+0000","last_fullsized":"2023-11-15T02:56:51.712886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.724341+0000","last_change":"2023-11-15T02:56:51.715489+0000","last_active":"2023-11-15T02:56:51.724341+0000","last_peered":"2023-11-15T02:56:51.724341+0000","last_clean":"2023-11-15T02:56:51.724341+0000","last_became_active":"2023-11-15T02:56:51.715060+0000","last_became_peered":"2023-11-15T02:56:51.715060+0000","last_unstale":"2023-11-15T02:56:51.724341+0000","last_undegraded":"2023-11-15T02:56:51.724341+0000","last_fullsized":"2023-11-15T02:56:51.724341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712366+0000","last_change":"2023-11-15T02:56:51.276645+0000","last_active":"2023-11-15T02:56:51.712366+0000","last_peered":"2023-11-15T02:56:51.712366+0000","last_clean":"2023-11-15T02:56:51.712366+0000","last_became_active":"2023-11-15T02:56:51.276356+0000","last_became_peered":"2023-11-15T02:56:51.276356+0000","last_unstale":"2023-11-15T02:56:51.712366+0000","last_undegraded":"2023-11-15T02:56:51.712366+0000","last_fullsized":"2023-11-15T02:56:51.712366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.724735+0000","last_change":"2023-11-15T02:56:51.719961+0000","last_active":"2023-11-15T02:56:51.724735+0000","last_peered":"2023-11-15T02:56:51.724735+0000","last_clean":"2023-11-15T02:56:51.724735+0000","last_became_active":"2023-11-15T02:56:51.719598+0000","last_became_peered":"2023-11-15T02:56:51.719598+0000","last_unstale":"2023-11-15T02:56:51.724735+0000","last_undegraded":"2023-11-15T02:56:51.724735+0000","last_fullsized":"2023-11-15T02:56:51.724735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.734513+0000","last_change":"2023-11-15T02:56:51.287275+0000","last_active":"2023-11-15T02:56:51.734513+0000","last_peered":"2023-11-15T02:56:51.734513+0000","last_clean":"2023-11-15T02:56:51.734513+0000","last_became_active":"2023-11-15T02:56:51.286259+0000","last_became_peered":"2023-11-15T02:56:51.286259+0000","last_unstale":"2023-11-15T02:56:51.734513+0000","last_undegraded":"2023-11-15T02:56:51.734513+0000","last_fullsized":"2023-11-15T02:56:51.734513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.751632+0000","last_change":"2023-11-15T02:56:51.748332+0000","last_active":"2023-11-15T02:56:51.751632+0000","last_peered":"2023-11-15T02:56:51.751632+0000","last_clean":"2023-11-15T02:56:51.751632+0000","last_became_active":"2023-11-15T02:56:51.747865+0000","last_became_peered":"2023-11-15T02:56:51.747865+0000","last_unstale":"2023-11-15T02:56:51.751632+0000","last_undegraded":"2023-11-15T02:56:51.751632+0000","last_fullsized":"2023-11-15T02:56:51.751632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746250+0000","last_change":"2023-11-15T02:56:51.274812+0000","last_active":"2023-11-15T02:56:51.746250+0000","last_peered":"2023-11-15T02:56:51.746250+0000","last_clean":"2023-11-15T02:56:51.746250+0000","last_became_active":"2023-11-15T02:56:51.273693+0000","last_became_peered":"2023-11-15T02:56:51.273693+0000","last_unstale":"2023-11-15T02:56:51.746250+0000","last_undegraded":"2023-11-15T02:56:51.746250+0000","last_fullsized":"2023-11-15T02:56:51.746250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.734600+0000","last_change":"2023-11-15T02:56:51.718982+0000","last_active":"2023-11-15T02:56:51.734600+0000","last_peered":"2023-11-15T02:56:51.734600+0000","last_clean":"2023-11-15T02:56:51.734600+0000","last_became_active":"2023-11-15T02:56:51.718702+0000","last_became_peered":"2023-11-15T02:56:51.718702+0000","last_unstale":"2023-11-15T02:56:51.734600+0000","last_undegraded":"2023-11-15T02:56:51.734600+0000","last_fullsized":"2023-11-15T02:56:51.734600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.743614+0000","last_change":"2023-11-15T02:56:51.285434+0000","last_active":"2023-11-15T02:56:51.743614+0000","last_peered":"2023-11-15T02:56:51.743614+0000","last_clean":"2023-11-15T02:56:51.743614+0000","last_became_active":"2023-11-15T02:56:51.284984+0000","last_became_peered":"2023-11-15T02:56:51.284984+0000","last_unstale":"2023-11-15T02:56:51.743614+0000","last_undegraded":"2023-11-15T02:56:51.743614+0000","last_fullsized":"2023-11-15T02:56:51.743614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744214+0000","last_change":"2023-11-15T02:56:51.740534+0000","last_active":"2023-11-15T02:56:51.744214+0000","last_peered":"2023-11-15T02:56:51.744214+0000","last_clean":"2023-11-15T02:56:51.744214+0000","last_became_active":"2023-11-15T02:56:51.740090+0000","last_became_peered":"2023-11-15T02:56:51.740090+0000","last_unstale":"2023-11-15T02:56:51.744214+0000","last_undegraded":"2023-11-15T02:56:51.744214+0000","last_fullsized":"2023-11-15T02:56:51.744214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.714872+0000","last_change":"2023-11-15T02:56:51.279489+0000","last_active":"2023-11-15T02:56:51.714872+0000","last_peered":"2023-11-15T02:56:51.714872+0000","last_clean":"2023-11-15T02:56:51.714872+0000","last_became_active":"2023-11-15T02:56:51.279222+0000","last_became_peered":"2023-11-15T02:56:51.279222+0000","last_unstale":"2023-11-15T02:56:51.714872+0000","last_undegraded":"2023-11-15T02:56:51.714872+0000","last_fullsized":"2023-11-15T02:56:51.714872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.749518+0000","last_change":"2023-11-15T02:56:51.744824+0000","last_active":"2023-11-15T02:56:51.749518+0000","last_peered":"2023-11-15T02:56:51.749518+0000","last_clean":"2023-11-15T02:56:51.749518+0000","last_became_active":"2023-11-15T02:56:51.744533+0000","last_became_peered":"2023-11-15T02:56:51.744533+0000","last_unstale":"2023-11-15T02:56:51.749518+0000","last_undegraded":"2023-11-15T02:56:51.749518+0000","last_fullsized":"2023-11-15T02:56:51.749518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.721137+0000","last_change":"2023-11-15T02:56:51.273142+0000","last_active":"2023-11-15T02:56:51.721137+0000","last_peered":"2023-11-15T02:56:51.721137+0000","last_clean":"2023-11-15T02:56:51.721137+0000","last_became_active":"2023-11-15T02:56:51.272861+0000","last_became_peered":"2023-11-15T02:56:51.272861+0000","last_unstale":"2023-11-15T02:56:51.721137+0000","last_undegraded":"2023-11-15T02:56:51.721137+0000","last_fullsized":"2023-11-15T02:56:51.721137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.735105+0000","last_change":"2023-11-15T02:56:51.728416+0000","last_active":"2023-11-15T02:56:51.735105+0000","last_peered":"2023-11-15T02:56:51.735105+0000","last_clean":"2023-11-15T02:56:51.735105+0000","last_became_active":"2023-11-15T02:56:51.728084+0000","last_became_peered":"2023-11-15T02:56:51.728084+0000","last_unstale":"2023-11-15T02:56:51.735105+0000","last_undegraded":"2023-11-15T02:56:51.735105+0000","last_fullsized":"2023-11-15T02:56:51.735105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.724100+0000","last_change":"2023-11-15T02:56:51.274354+0000","last_active":"2023-11-15T02:56:51.724100+0000","last_peered":"2023-11-15T02:56:51.724100+0000","last_clean":"2023-11-15T02:56:51.724100+0000","last_became_active":"2023-11-15T02:56:51.274083+0000","last_became_peered":"2023-11-15T02:56:51.274083+0000","last_unstale":"2023-11-15T02:56:51.724100+0000","last_undegraded":"2023-11-15T02:56:51.724100+0000","last_fullsized":"2023-11-15T02:56:51.724100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.736974+0000","last_change":"2023-11-15T02:56:51.721714+0000","last_active":"2023-11-15T02:56:51.736974+0000","last_peered":"2023-11-15T02:56:51.736974+0000","last_clean":"2023-11-15T02:56:51.736974+0000","last_became_active":"2023-11-15T02:56:51.721415+0000","last_became_peered":"2023-11-15T02:56:51.721415+0000","last_unstale":"2023-11-15T02:56:51.736974+0000","last_undegraded":"2023-11-15T02:56:51.736974+0000","last_fullsized":"2023-11-15T02:56:51.736974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.704644+0000","last_change":"2023-11-15T02:56:51.283397+0000","last_active":"2023-11-15T02:56:51.704644+0000","last_peered":"2023-11-15T02:56:51.704644+0000","last_clean":"2023-11-15T02:56:51.704644+0000","last_became_active":"2023-11-15T02:56:51.283101+0000","last_became_peered":"2023-11-15T02:56:51.283101+0000","last_unstale":"2023-11-15T02:56:51.704644+0000","last_undegraded":"2023-11-15T02:56:51.704644+0000","last_fullsized":"2023-11-15T02:56:51.704644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.735611+0000","last_change":"2023-11-15T02:56:51.727142+0000","last_active":"2023-11-15T02:56:51.735611+0000","last_peered":"2023-11-15T02:56:51.735611+0000","last_clean":"2023-11-15T02:56:51.735611+0000","last_became_active":"2023-11-15T02:56:51.726872+0000","last_became_peered":"2023-11-15T02:56:51.726872+0000","last_unstale":"2023-11-15T02:56:51.735611+0000","last_undegraded":"2023-11-15T02:56:51.735611+0000","last_fullsized":"2023-11-15T02:56:51.735611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.710839+0000","last_change":"2023-11-15T02:56:51.281104+0000","last_active":"2023-11-15T02:56:51.710839+0000","last_peered":"2023-11-15T02:56:51.710839+0000","last_clean":"2023-11-15T02:56:51.710839+0000","last_became_active":"2023-11-15T02:56:51.280784+0000","last_became_peered":"2023-11-15T02:56:51.280784+0000","last_unstale":"2023-11-15T02:56:51.710839+0000","last_undegraded":"2023-11-15T02:56:51.710839+0000","last_fullsized":"2023-11-15T02:56:51.710839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.737357+0000","last_change":"2023-11-15T02:56:51.719121+0000","last_active":"2023-11-15T02:56:51.737357+0000","last_peered":"2023-11-15T02:56:51.737357+0000","last_clean":"2023-11-15T02:56:51.737357+0000","last_became_active":"2023-11-15T02:56:51.718812+0000","last_became_peered":"2023-11-15T02:56:51.718812+0000","last_unstale":"2023-11-15T02:56:51.737357+0000","last_undegraded":"2023-11-15T02:56:51.737357+0000","last_fullsized":"2023-11-15T02:56:51.737357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.722994+0000","last_change":"2023-11-15T02:56:51.285977+0000","last_active":"2023-11-15T02:56:51.722994+0000","last_peered":"2023-11-15T02:56:51.722994+0000","last_clean":"2023-11-15T02:56:51.722994+0000","last_became_active":"2023-11-15T02:56:51.277431+0000","last_became_peered":"2023-11-15T02:56:51.277431+0000","last_unstale":"2023-11-15T02:56:51.722994+0000","last_undegraded":"2023-11-15T02:56:51.722994+0000","last_fullsized":"2023-11-15T02:56:51.722994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.735350+0000","last_change":"2023-11-15T02:56:51.720775+0000","last_active":"2023-11-15T02:56:51.735350+0000","last_peered":"2023-11-15T02:56:51.735350+0000","last_clean":"2023-11-15T02:56:51.735350+0000","last_became_active":"2023-11-15T02:56:51.720446+0000","last_became_peered":"2023-11-15T02:56:51.720446+0000","last_unstale":"2023-11-15T02:56:51.735350+0000","last_undegraded":"2023-11-15T02:56:51.735350+0000","last_fullsized":"2023-11-15T02:56:51.735350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.715531+0000","last_change":"2023-11-15T02:56:51.268303+0000","last_active":"2023-11-15T02:56:51.715531+0000","last_peered":"2023-11-15T02:56:51.715531+0000","last_clean":"2023-11-15T02:56:51.715531+0000","last_became_active":"2023-11-15T02:56:51.267982+0000","last_became_peered":"2023-11-15T02:56:51.267982+0000","last_unstale":"2023-11-15T02:56:51.715531+0000","last_undegraded":"2023-11-15T02:56:51.715531+0000","last_fullsized":"2023-11-15T02:56:51.715531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738120+0000","last_change":"2023-11-15T02:56:51.726853+0000","last_active":"2023-11-15T02:56:51.738120+0000","last_peered":"2023-11-15T02:56:51.738120+0000","last_clean":"2023-11-15T02:56:51.738120+0000","last_became_active":"2023-11-15T02:56:51.726448+0000","last_became_peered":"2023-11-15T02:56:51.726448+0000","last_unstale":"2023-11-15T02:56:51.738120+0000","last_undegraded":"2023-11-15T02:56:51.738120+0000","last_fullsized":"2023-11-15T02:56:51.738120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745254+0000","last_change":"2023-11-15T02:56:51.291178+0000","last_active":"2023-11-15T02:56:51.745254+0000","last_peered":"2023-11-15T02:56:51.745254+0000","last_clean":"2023-11-15T02:56:51.745254+0000","last_became_active":"2023-11-15T02:56:51.289245+0000","last_became_peered":"2023-11-15T02:56:51.289245+0000","last_unstale":"2023-11-15T02:56:51.745254+0000","last_undegraded":"2023-11-15T02:56:51.745254+0000","last_fullsized":"2023-11-15T02:56:51.745254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.718417+0000","last_change":"2023-11-15T02:56:51.714302+0000","last_active":"2023-11-15T02:56:51.718417+0000","last_peered":"2023-11-15T02:56:51.718417+0000","last_clean":"2023-11-15T02:56:51.718417+0000","last_became_active":"2023-11-15T02:56:51.714023+0000","last_became_peered":"2023-11-15T02:56:51.714023+0000","last_unstale":"2023-11-15T02:56:51.718417+0000","last_undegraded":"2023-11-15T02:56:51.718417+0000","last_fullsized":"2023-11-15T02:56:51.718417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746516+0000","last_change":"2023-11-15T02:56:51.265289+0000","last_active":"2023-11-15T02:56:51.746516+0000","last_peered":"2023-11-15T02:56:51.746516+0000","last_clean":"2023-11-15T02:56:51.746516+0000","last_became_active":"2023-11-15T02:56:51.264945+0000","last_became_peered":"2023-11-15T02:56:51.264945+0000","last_unstale":"2023-11-15T02:56:51.746516+0000","last_undegraded":"2023-11-15T02:56:51.746516+0000","last_fullsized":"2023-11-15T02:56:51.746516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.751653+0000","last_change":"2023-11-15T02:56:51.748696+0000","last_active":"2023-11-15T02:56:51.751653+0000","last_peered":"2023-11-15T02:56:51.751653+0000","last_clean":"2023-11-15T02:56:51.751653+0000","last_became_active":"2023-11-15T02:56:51.748403+0000","last_became_peered":"2023-11-15T02:56:51.748403+0000","last_unstale":"2023-11-15T02:56:51.751653+0000","last_undegraded":"2023-11-15T02:56:51.751653+0000","last_fullsized":"2023-11-15T02:56:51.751653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713322+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T02:56:51.713322+0000","last_peered":"2023-11-15T02:56:51.713322+0000","last_clean":"2023-11-15T02:56:51.713322+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T02:56:51.713322+0000","last_undegraded":"2023-11-15T02:56:51.713322+0000","last_fullsized":"2023-11-15T02:56:51.713322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.722937+0000","last_change":"2023-11-15T02:56:51.270301+0000","last_active":"2023-11-15T02:56:51.722937+0000","last_peered":"2023-11-15T02:56:51.722937+0000","last_clean":"2023-11-15T02:56:51.722937+0000","last_became_active":"2023-11-15T02:56:51.269996+0000","last_became_peered":"2023-11-15T02:56:51.269996+0000","last_unstale":"2023-11-15T02:56:51.722937+0000","last_undegraded":"2023-11-15T02:56:51.722937+0000","last_fullsized":"2023-11-15T02:56:51.722937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.742366+0000","last_change":"2023-11-15T02:56:51.736621+0000","last_active":"2023-11-15T02:56:51.742366+0000","last_peered":"2023-11-15T02:56:51.742366+0000","last_clean":"2023-11-15T02:56:51.742366+0000","last_became_active":"2023-11-15T02:56:51.736319+0000","last_became_peered":"2023-11-15T02:56:51.736319+0000","last_unstale":"2023-11-15T02:56:51.742366+0000","last_undegraded":"2023-11-15T02:56:51.742366+0000","last_fullsized":"2023-11-15T02:56:51.742366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.706577+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T02:56:51.706577+0000","last_peered":"2023-11-15T02:56:51.706577+0000","last_clean":"2023-11-15T02:56:51.706577+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T02:56:51.706577+0000","last_undegraded":"2023-11-15T02:56:51.706577+0000","last_fullsized":"2023-11-15T02:56:51.706577+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745714+0000","last_change":"2023-11-15T02:56:51.273274+0000","last_active":"2023-11-15T02:56:51.745714+0000","last_peered":"2023-11-15T02:56:51.745714+0000","last_clean":"2023-11-15T02:56:51.745714+0000","last_became_active":"2023-11-15T02:56:51.270655+0000","last_became_peered":"2023-11-15T02:56:51.270655+0000","last_unstale":"2023-11-15T02:56:51.745714+0000","last_undegraded":"2023-11-15T02:56:51.745714+0000","last_fullsized":"2023-11-15T02:56:51.745714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738809+0000","last_change":"2023-11-15T02:56:51.734478+0000","last_active":"2023-11-15T02:56:51.738809+0000","last_peered":"2023-11-15T02:56:51.738809+0000","last_clean":"2023-11-15T02:56:51.738809+0000","last_became_active":"2023-11-15T02:56:51.732194+0000","last_became_peered":"2023-11-15T02:56:51.732194+0000","last_unstale":"2023-11-15T02:56:51.738809+0000","last_undegraded":"2023-11-15T02:56:51.738809+0000","last_fullsized":"2023-11-15T02:56:51.738809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730730+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T02:56:51.730730+0000","last_peered":"2023-11-15T02:56:51.730730+0000","last_clean":"2023-11-15T02:56:51.730730+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T02:56:51.730730+0000","last_undegraded":"2023-11-15T02:56:51.730730+0000","last_fullsized":"2023-11-15T02:56:51.730730+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.705716+0000","last_change":"2023-11-15T02:56:51.270161+0000","last_active":"2023-11-15T02:56:51.705716+0000","last_peered":"2023-11-15T02:56:51.705716+0000","last_clean":"2023-11-15T02:56:51.705716+0000","last_became_active":"2023-11-15T02:56:51.268151+0000","last_became_peered":"2023-11-15T02:56:51.268151+0000","last_unstale":"2023-11-15T02:56:51.705716+0000","last_undegraded":"2023-11-15T02:56:51.705716+0000","last_fullsized":"2023-11-15T02:56:51.705716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.737232+0000","last_change":"2023-11-15T02:56:51.731375+0000","last_active":"2023-11-15T02:56:51.737232+0000","last_peered":"2023-11-15T02:56:51.737232+0000","last_clean":"2023-11-15T02:56:51.737232+0000","last_became_active":"2023-11-15T02:56:51.731100+0000","last_became_peered":"2023-11-15T02:56:51.731100+0000","last_unstale":"2023-11-15T02:56:51.737232+0000","last_undegraded":"2023-11-15T02:56:51.737232+0000","last_fullsized":"2023-11-15T02:56:51.737232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.705942+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T02:56:51.705942+0000","last_peered":"2023-11-15T02:56:51.705942+0000","last_clean":"2023-11-15T02:56:51.705942+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T02:56:51.705942+0000","last_undegraded":"2023-11-15T02:56:51.705942+0000","last_fullsized":"2023-11-15T02:56:51.705942+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.715265+0000","last_change":"2023-11-15T02:56:51.275134+0000","last_active":"2023-11-15T02:56:51.715265+0000","last_peered":"2023-11-15T02:56:51.715265+0000","last_clean":"2023-11-15T02:56:51.715265+0000","last_became_active":"2023-11-15T02:56:51.274400+0000","last_became_peered":"2023-11-15T02:56:51.274400+0000","last_unstale":"2023-11-15T02:56:51.715265+0000","last_undegraded":"2023-11-15T02:56:51.715265+0000","last_fullsized":"2023-11-15T02:56:51.715265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.727321+0000","last_change":"2023-11-15T02:56:51.724465+0000","last_active":"2023-11-15T02:56:51.727321+0000","last_peered":"2023-11-15T02:56:51.727321+0000","last_clean":"2023-11-15T02:56:51.727321+0000","last_became_active":"2023-11-15T02:56:51.723590+0000","last_became_peered":"2023-11-15T02:56:51.723590+0000","last_unstale":"2023-11-15T02:56:51.727321+0000","last_undegraded":"2023-11-15T02:56:51.727321+0000","last_fullsized":"2023-11-15T02:56:51.727321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.731884+0000","last_change":"2023-11-15T02:56:51.282484+0000","last_active":"2023-11-15T02:56:51.731884+0000","last_peered":"2023-11-15T02:56:51.731884+0000","last_clean":"2023-11-15T02:56:51.731884+0000","last_became_active":"2023-11-15T02:56:51.282108+0000","last_became_peered":"2023-11-15T02:56:51.282108+0000","last_unstale":"2023-11-15T02:56:51.731884+0000","last_undegraded":"2023-11-15T02:56:51.731884+0000","last_fullsized":"2023-11-15T02:56:51.731884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.716774+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T02:56:51.716774+0000","last_peered":"2023-11-15T02:56:51.716774+0000","last_clean":"2023-11-15T02:56:51.716774+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T02:56:51.716774+0000","last_undegraded":"2023-11-15T02:56:51.716774+0000","last_fullsized":"2023-11-15T02:56:51.716774+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.719253+0000","last_change":"2023-11-15T02:56:51.717765+0000","last_active":"2023-11-15T02:56:51.719253+0000","last_peered":"2023-11-15T02:56:51.719253+0000","last_clean":"2023-11-15T02:56:51.719253+0000","last_became_active":"2023-11-15T02:56:51.717498+0000","last_became_peered":"2023-11-15T02:56:51.717498+0000","last_unstale":"2023-11-15T02:56:51.719253+0000","last_undegraded":"2023-11-15T02:56:51.719253+0000","last_fullsized":"2023-11-15T02:56:51.719253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.704337+0000","last_change":"2023-11-15T02:56:51.267133+0000","last_active":"2023-11-15T02:56:51.704337+0000","last_peered":"2023-11-15T02:56:51.704337+0000","last_clean":"2023-11-15T02:56:51.704337+0000","last_became_active":"2023-11-15T02:56:51.266864+0000","last_became_peered":"2023-11-15T02:56:51.266864+0000","last_unstale":"2023-11-15T02:56:51.704337+0000","last_undegraded":"2023-11-15T02:56:51.704337+0000","last_fullsized":"2023-11-15T02:56:51.704337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712961+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T02:56:51.712961+0000","last_peered":"2023-11-15T02:56:51.712961+0000","last_clean":"2023-11-15T02:56:51.712961+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T02:56:51.712961+0000","last_undegraded":"2023-11-15T02:56:51.712961+0000","last_fullsized":"2023-11-15T02:56:51.712961+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.719804+0000","last_change":"2023-11-15T02:56:51.717744+0000","last_active":"2023-11-15T02:56:51.719804+0000","last_peered":"2023-11-15T02:56:51.719804+0000","last_clean":"2023-11-15T02:56:51.719804+0000","last_became_active":"2023-11-15T02:56:51.717448+0000","last_became_peered":"2023-11-15T02:56:51.717448+0000","last_unstale":"2023-11-15T02:56:51.719804+0000","last_undegraded":"2023-11-15T02:56:51.719804+0000","last_fullsized":"2023-11-15T02:56:51.719804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.718982+0000","last_change":"2023-11-15T02:56:51.273152+0000","last_active":"2023-11-15T02:56:51.718982+0000","last_peered":"2023-11-15T02:56:51.718982+0000","last_clean":"2023-11-15T02:56:51.718982+0000","last_became_active":"2023-11-15T02:56:51.272868+0000","last_became_peered":"2023-11-15T02:56:51.272868+0000","last_unstale":"2023-11-15T02:56:51.718982+0000","last_undegraded":"2023-11-15T02:56:51.718982+0000","last_fullsized":"2023-11-15T02:56:51.718982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.728384+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T02:56:51.728384+0000","last_peered":"2023-11-15T02:56:51.728384+0000","last_clean":"2023-11-15T02:56:51.728384+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T02:56:51.728384+0000","last_undegraded":"2023-11-15T02:56:51.728384+0000","last_fullsized":"2023-11-15T02:56:51.728384+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744430+0000","last_change":"2023-11-15T02:56:51.739786+0000","last_active":"2023-11-15T02:56:51.744430+0000","last_peered":"2023-11-15T02:56:51.744430+0000","last_clean":"2023-11-15T02:56:51.744430+0000","last_became_active":"2023-11-15T02:56:51.737659+0000","last_became_peered":"2023-11-15T02:56:51.737659+0000","last_unstale":"2023-11-15T02:56:51.744430+0000","last_undegraded":"2023-11-15T02:56:51.744430+0000","last_fullsized":"2023-11-15T02:56:51.744430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.711641+0000","last_change":"2023-11-15T02:56:51.276817+0000","last_active":"2023-11-15T02:56:51.711641+0000","last_peered":"2023-11-15T02:56:51.711641+0000","last_clean":"2023-11-15T02:56:51.711641+0000","last_became_active":"2023-11-15T02:56:51.276490+0000","last_became_peered":"2023-11-15T02:56:51.276490+0000","last_unstale":"2023-11-15T02:56:51.711641+0000","last_undegraded":"2023-11-15T02:56:51.711641+0000","last_fullsized":"2023-11-15T02:56:51.711641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713989+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T02:56:51.713989+0000","last_peered":"2023-11-15T02:56:51.713989+0000","last_clean":"2023-11-15T02:56:51.713989+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T02:56:51.713989+0000","last_undegraded":"2023-11-15T02:56:51.713989+0000","last_fullsized":"2023-11-15T02:56:51.713989+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.728465+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T02:56:51.728465+0000","last_peered":"2023-11-15T02:56:51.728465+0000","last_clean":"2023-11-15T02:56:51.728465+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T02:56:51.728465+0000","last_undegraded":"2023-11-15T02:56:51.728465+0000","last_fullsized":"2023-11-15T02:56:51.728465+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.749349+0000","last_change":"2023-11-15T02:56:51.745538+0000","last_active":"2023-11-15T02:56:51.749349+0000","last_peered":"2023-11-15T02:56:51.749349+0000","last_clean":"2023-11-15T02:56:51.749349+0000","last_became_active":"2023-11-15T02:56:51.745369+0000","last_became_peered":"2023-11-15T02:56:51.745369+0000","last_unstale":"2023-11-15T02:56:51.749349+0000","last_undegraded":"2023-11-15T02:56:51.749349+0000","last_fullsized":"2023-11-15T02:56:51.749349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.710979+0000","last_change":"2023-11-15T02:56:51.280484+0000","last_active":"2023-11-15T02:56:51.710979+0000","last_peered":"2023-11-15T02:56:51.710979+0000","last_clean":"2023-11-15T02:56:51.710979+0000","last_became_active":"2023-11-15T02:56:51.272264+0000","last_became_peered":"2023-11-15T02:56:51.272264+0000","last_unstale":"2023-11-15T02:56:51.710979+0000","last_undegraded":"2023-11-15T02:56:51.710979+0000","last_fullsized":"2023-11-15T02:56:51.710979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.724717+0000","last_change":"2023-11-15T02:56:51.719709+0000","last_active":"2023-11-15T02:56:51.724717+0000","last_peered":"2023-11-15T02:56:51.724717+0000","last_clean":"2023-11-15T02:56:51.724717+0000","last_became_active":"2023-11-15T02:56:51.719230+0000","last_became_peered":"2023-11-15T02:56:51.719230+0000","last_unstale":"2023-11-15T02:56:51.724717+0000","last_undegraded":"2023-11-15T02:56:51.724717+0000","last_fullsized":"2023-11-15T02:56:51.724717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.735953+0000","last_change":"2023-11-15T02:56:51.272978+0000","last_active":"2023-11-15T02:56:51.735953+0000","last_peered":"2023-11-15T02:56:51.735953+0000","last_clean":"2023-11-15T02:56:51.735953+0000","last_became_active":"2023-11-15T02:56:51.272650+0000","last_became_peered":"2023-11-15T02:56:51.272650+0000","last_unstale":"2023-11-15T02:56:51.735953+0000","last_undegraded":"2023-11-15T02:56:51.735953+0000","last_fullsized":"2023-11-15T02:56:51.735953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.749492+0000","last_change":"2023-11-15T02:56:51.744603+0000","last_active":"2023-11-15T02:56:51.749492+0000","last_peered":"2023-11-15T02:56:51.749492+0000","last_clean":"2023-11-15T02:56:51.749492+0000","last_became_active":"2023-11-15T02:56:51.744212+0000","last_became_peered":"2023-11-15T02:56:51.744212+0000","last_unstale":"2023-11-15T02:56:51.749492+0000","last_undegraded":"2023-11-15T02:56:51.749492+0000","last_fullsized":"2023-11-15T02:56:51.749492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713620+0000","last_change":"2023-11-15T02:56:51.279629+0000","last_active":"2023-11-15T02:56:51.713620+0000","last_peered":"2023-11-15T02:56:51.713620+0000","last_clean":"2023-11-15T02:56:51.713620+0000","last_became_active":"2023-11-15T02:56:51.279367+0000","last_became_peered":"2023-11-15T02:56:51.279367+0000","last_unstale":"2023-11-15T02:56:51.713620+0000","last_undegraded":"2023-11-15T02:56:51.713620+0000","last_fullsized":"2023-11-15T02:56:51.713620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745035+0000","last_change":"2023-11-15T02:56:51.741271+0000","last_active":"2023-11-15T02:56:51.745035+0000","last_peered":"2023-11-15T02:56:51.745035+0000","last_clean":"2023-11-15T02:56:51.745035+0000","last_became_active":"2023-11-15T02:56:51.740569+0000","last_became_peered":"2023-11-15T02:56:51.740569+0000","last_unstale":"2023-11-15T02:56:51.745035+0000","last_undegraded":"2023-11-15T02:56:51.745035+0000","last_fullsized":"2023-11-15T02:56:51.745035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.740741+0000","last_change":"2023-11-15T02:56:51.278710+0000","last_active":"2023-11-15T02:56:51.740741+0000","last_peered":"2023-11-15T02:56:51.740741+0000","last_clean":"2023-11-15T02:56:51.740741+0000","last_became_active":"2023-11-15T02:56:51.278446+0000","last_became_peered":"2023-11-15T02:56:51.278446+0000","last_unstale":"2023-11-15T02:56:51.740741+0000","last_undegraded":"2023-11-15T02:56:51.740741+0000","last_fullsized":"2023-11-15T02:56:51.740741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745553+0000","last_change":"2023-11-15T02:56:51.740004+0000","last_active":"2023-11-15T02:56:51.745553+0000","last_peered":"2023-11-15T02:56:51.745553+0000","last_clean":"2023-11-15T02:56:51.745553+0000","last_became_active":"2023-11-15T02:56:51.739663+0000","last_became_peered":"2023-11-15T02:56:51.739663+0000","last_unstale":"2023-11-15T02:56:51.745553+0000","last_undegraded":"2023-11-15T02:56:51.745553+0000","last_fullsized":"2023-11-15T02:56:51.745553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.714982+0000","last_change":"2023-11-15T02:56:51.278072+0000","last_active":"2023-11-15T02:56:51.714982+0000","last_peered":"2023-11-15T02:56:51.714982+0000","last_clean":"2023-11-15T02:56:51.714982+0000","last_became_active":"2023-11-15T02:56:51.277779+0000","last_became_peered":"2023-11-15T02:56:51.277779+0000","last_unstale":"2023-11-15T02:56:51.714982+0000","last_undegraded":"2023-11-15T02:56:51.714982+0000","last_fullsized":"2023-11-15T02:56:51.714982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746826+0000","last_change":"2023-11-15T02:56:51.744994+0000","last_active":"2023-11-15T02:56:51.746826+0000","last_peered":"2023-11-15T02:56:51.746826+0000","last_clean":"2023-11-15T02:56:51.746826+0000","last_became_active":"2023-11-15T02:56:51.743902+0000","last_became_peered":"2023-11-15T02:56:51.743902+0000","last_unstale":"2023-11-15T02:56:51.746826+0000","last_undegraded":"2023-11-15T02:56:51.746826+0000","last_fullsized":"2023-11-15T02:56:51.746826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730502+0000","last_change":"2023-11-15T02:56:51.289728+0000","last_active":"2023-11-15T02:56:51.730502+0000","last_peered":"2023-11-15T02:56:51.730502+0000","last_clean":"2023-11-15T02:56:51.730502+0000","last_became_active":"2023-11-15T02:56:51.288566+0000","last_became_peered":"2023-11-15T02:56:51.288566+0000","last_unstale":"2023-11-15T02:56:51.730502+0000","last_undegraded":"2023-11-15T02:56:51.730502+0000","last_fullsized":"2023-11-15T02:56:51.730502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747477+0000","last_change":"2023-11-15T02:56:51.735076+0000","last_active":"2023-11-15T02:56:51.747477+0000","last_peered":"2023-11-15T02:56:51.747477+0000","last_clean":"2023-11-15T02:56:51.747477+0000","last_became_active":"2023-11-15T02:56:51.734802+0000","last_became_peered":"2023-11-15T02:56:51.734802+0000","last_unstale":"2023-11-15T02:56:51.747477+0000","last_undegraded":"2023-11-15T02:56:51.747477+0000","last_fullsized":"2023-11-15T02:56:51.747477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744754+0000","last_change":"2023-11-15T02:56:51.286563+0000","last_active":"2023-11-15T02:56:51.744754+0000","last_peered":"2023-11-15T02:56:51.744754+0000","last_clean":"2023-11-15T02:56:51.744754+0000","last_became_active":"2023-11-15T02:56:51.286245+0000","last_became_peered":"2023-11-15T02:56:51.286245+0000","last_unstale":"2023-11-15T02:56:51.744754+0000","last_undegraded":"2023-11-15T02:56:51.744754+0000","last_fullsized":"2023-11-15T02:56:51.744754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746575+0000","last_change":"2023-11-15T02:56:51.742254+0000","last_active":"2023-11-15T02:56:51.746575+0000","last_peered":"2023-11-15T02:56:51.746575+0000","last_clean":"2023-11-15T02:56:51.746575+0000","last_became_active":"2023-11-15T02:56:51.741950+0000","last_became_peered":"2023-11-15T02:56:51.741950+0000","last_unstale":"2023-11-15T02:56:51.746575+0000","last_undegraded":"2023-11-15T02:56:51.746575+0000","last_fullsized":"2023-11-15T02:56:51.746575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744885+0000","last_change":"2023-11-15T02:56:51.286640+0000","last_active":"2023-11-15T02:56:51.744885+0000","last_peered":"2023-11-15T02:56:51.744885+0000","last_clean":"2023-11-15T02:56:51.744885+0000","last_became_active":"2023-11-15T02:56:51.286290+0000","last_became_peered":"2023-11-15T02:56:51.286290+0000","last_unstale":"2023-11-15T02:56:51.744885+0000","last_undegraded":"2023-11-15T02:56:51.744885+0000","last_fullsized":"2023-11-15T02:56:51.744885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.733131+0000","last_change":"2023-11-15T02:56:51.727797+0000","last_active":"2023-11-15T02:56:51.733131+0000","last_peered":"2023-11-15T02:56:51.733131+0000","last_clean":"2023-11-15T02:56:51.733131+0000","last_became_active":"2023-11-15T02:56:51.727520+0000","last_became_peered":"2023-11-15T02:56:51.727520+0000","last_unstale":"2023-11-15T02:56:51.733131+0000","last_undegraded":"2023-11-15T02:56:51.733131+0000","last_fullsized":"2023-11-15T02:56:51.733131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738551+0000","last_change":"2023-11-15T02:56:51.284828+0000","last_active":"2023-11-15T02:56:51.738551+0000","last_peered":"2023-11-15T02:56:51.738551+0000","last_clean":"2023-11-15T02:56:51.738551+0000","last_became_active":"2023-11-15T02:56:51.276571+0000","last_became_peered":"2023-11-15T02:56:51.276571+0000","last_unstale":"2023-11-15T02:56:51.738551+0000","last_undegraded":"2023-11-15T02:56:51.738551+0000","last_fullsized":"2023-11-15T02:56:51.738551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.739504+0000","last_change":"2023-11-15T02:56:51.732129+0000","last_active":"2023-11-15T02:56:51.739504+0000","last_peered":"2023-11-15T02:56:51.739504+0000","last_clean":"2023-11-15T02:56:51.739504+0000","last_became_active":"2023-11-15T02:56:51.729770+0000","last_became_peered":"2023-11-15T02:56:51.729770+0000","last_unstale":"2023-11-15T02:56:51.739504+0000","last_undegraded":"2023-11-15T02:56:51.739504+0000","last_fullsized":"2023-11-15T02:56:51.739504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.731327+0000","last_change":"2023-11-15T02:56:51.280630+0000","last_active":"2023-11-15T02:56:51.731327+0000","last_peered":"2023-11-15T02:56:51.731327+0000","last_clean":"2023-11-15T02:56:51.731327+0000","last_became_active":"2023-11-15T02:56:51.280353+0000","last_became_peered":"2023-11-15T02:56:51.280353+0000","last_unstale":"2023-11-15T02:56:51.731327+0000","last_undegraded":"2023-11-15T02:56:51.731327+0000","last_fullsized":"2023-11-15T02:56:51.731327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.741375+0000","last_change":"2023-11-15T02:56:51.737689+0000","last_active":"2023-11-15T02:56:51.741375+0000","last_peered":"2023-11-15T02:56:51.741375+0000","last_clean":"2023-11-15T02:56:51.741375+0000","last_became_active":"2023-11-15T02:56:51.737369+0000","last_became_peered":"2023-11-15T02:56:51.737369+0000","last_unstale":"2023-11-15T02:56:51.741375+0000","last_undegraded":"2023-11-15T02:56:51.741375+0000","last_fullsized":"2023-11-15T02:56:51.741375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.719432+0000","last_change":"2023-11-15T02:56:51.285172+0000","last_active":"2023-11-15T02:56:51.719432+0000","last_peered":"2023-11-15T02:56:51.719432+0000","last_clean":"2023-11-15T02:56:51.719432+0000","last_became_active":"2023-11-15T02:56:51.284936+0000","last_became_peered":"2023-11-15T02:56:51.284936+0000","last_unstale":"2023-11-15T02:56:51.719432+0000","last_undegraded":"2023-11-15T02:56:51.719432+0000","last_fullsized":"2023-11-15T02:56:51.719432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.732918+0000","last_change":"2023-11-15T02:56:51.728432+0000","last_active":"2023-11-15T02:56:51.732918+0000","last_peered":"2023-11-15T02:56:51.732918+0000","last_clean":"2023-11-15T02:56:51.732918+0000","last_became_active":"2023-11-15T02:56:51.727880+0000","last_became_peered":"2023-11-15T02:56:51.727880+0000","last_unstale":"2023-11-15T02:56:51.732918+0000","last_undegraded":"2023-11-15T02:56:51.732918+0000","last_fullsized":"2023-11-15T02:56:51.732918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744376+0000","last_change":"2023-11-15T02:56:51.288049+0000","last_active":"2023-11-15T02:56:51.744376+0000","last_peered":"2023-11-15T02:56:51.744376+0000","last_clean":"2023-11-15T02:56:51.744376+0000","last_became_active":"2023-11-15T02:56:51.287794+0000","last_became_peered":"2023-11-15T02:56:51.287794+0000","last_unstale":"2023-11-15T02:56:51.744376+0000","last_undegraded":"2023-11-15T02:56:51.744376+0000","last_fullsized":"2023-11-15T02:56:51.744376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.734683+0000","last_change":"2023-11-15T02:56:51.720823+0000","last_active":"2023-11-15T02:56:51.734683+0000","last_peered":"2023-11-15T02:56:51.734683+0000","last_clean":"2023-11-15T02:56:51.734683+0000","last_became_active":"2023-11-15T02:56:51.720479+0000","last_became_peered":"2023-11-15T02:56:51.720479+0000","last_unstale":"2023-11-15T02:56:51.734683+0000","last_undegraded":"2023-11-15T02:56:51.734683+0000","last_fullsized":"2023-11-15T02:56:51.734683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.718875+0000","last_change":"2023-11-15T02:56:51.278717+0000","last_active":"2023-11-15T02:56:51.718875+0000","last_peered":"2023-11-15T02:56:51.718875+0000","last_clean":"2023-11-15T02:56:51.718875+0000","last_became_active":"2023-11-15T02:56:51.278404+0000","last_became_peered":"2023-11-15T02:56:51.278404+0000","last_unstale":"2023-11-15T02:56:51.718875+0000","last_undegraded":"2023-11-15T02:56:51.718875+0000","last_fullsized":"2023-11-15T02:56:51.718875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745052+0000","last_change":"2023-11-15T02:56:51.741288+0000","last_active":"2023-11-15T02:56:51.745052+0000","last_peered":"2023-11-15T02:56:51.745052+0000","last_clean":"2023-11-15T02:56:51.745052+0000","last_became_active":"2023-11-15T02:56:51.740865+0000","last_became_peered":"2023-11-15T02:56:51.740865+0000","last_unstale":"2023-11-15T02:56:51.745052+0000","last_undegraded":"2023-11-15T02:56:51.745052+0000","last_fullsized":"2023-11-15T02:56:51.745052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746626+0000","last_change":"2023-11-15T02:56:51.274179+0000","last_active":"2023-11-15T02:56:51.746626+0000","last_peered":"2023-11-15T02:56:51.746626+0000","last_clean":"2023-11-15T02:56:51.746626+0000","last_became_active":"2023-11-15T02:56:51.272313+0000","last_became_peered":"2023-11-15T02:56:51.272313+0000","last_unstale":"2023-11-15T02:56:51.746626+0000","last_undegraded":"2023-11-15T02:56:51.746626+0000","last_fullsized":"2023-11-15T02:56:51.746626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.737568+0000","last_change":"2023-11-15T02:56:51.736317+0000","last_active":"2023-11-15T02:56:51.737568+0000","last_peered":"2023-11-15T02:56:51.737568+0000","last_clean":"2023-11-15T02:56:51.737568+0000","last_became_active":"2023-11-15T02:56:51.736060+0000","last_became_peered":"2023-11-15T02:56:51.736060+0000","last_unstale":"2023-11-15T02:56:51.737568+0000","last_undegraded":"2023-11-15T02:56:51.737568+0000","last_fullsized":"2023-11-15T02:56:51.737568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.754612+0000","last_change":"2023-11-15T02:56:51.279248+0000","last_active":"2023-11-15T02:56:51.754612+0000","last_peered":"2023-11-15T02:56:51.754612+0000","last_clean":"2023-11-15T02:56:51.754612+0000","last_became_active":"2023-11-15T02:56:51.278771+0000","last_became_peered":"2023-11-15T02:56:51.278771+0000","last_unstale":"2023-11-15T02:56:51.754612+0000","last_undegraded":"2023-11-15T02:56:51.754612+0000","last_fullsized":"2023-11-15T02:56:51.754612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.727542+0000","last_change":"2023-11-15T02:56:51.716546+0000","last_active":"2023-11-15T02:56:51.727542+0000","last_peered":"2023-11-15T02:56:51.727542+0000","last_clean":"2023-11-15T02:56:51.727542+0000","last_became_active":"2023-11-15T02:56:51.716254+0000","last_became_peered":"2023-11-15T02:56:51.716254+0000","last_unstale":"2023-11-15T02:56:51.727542+0000","last_undegraded":"2023-11-15T02:56:51.727542+0000","last_fullsized":"2023-11-15T02:56:51.727542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.712732+0000","last_change":"2023-11-15T02:56:51.286642+0000","last_active":"2023-11-15T02:56:51.712732+0000","last_peered":"2023-11-15T02:56:51.712732+0000","last_clean":"2023-11-15T02:56:51.712732+0000","last_became_active":"2023-11-15T02:56:51.286240+0000","last_became_peered":"2023-11-15T02:56:51.286240+0000","last_unstale":"2023-11-15T02:56:51.712732+0000","last_undegraded":"2023-11-15T02:56:51.712732+0000","last_fullsized":"2023-11-15T02:56:51.712732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.725581+0000","last_change":"2023-11-15T02:56:51.722322+0000","last_active":"2023-11-15T02:56:51.725581+0000","last_peered":"2023-11-15T02:56:51.725581+0000","last_clean":"2023-11-15T02:56:51.725581+0000","last_became_active":"2023-11-15T02:56:51.721931+0000","last_became_peered":"2023-11-15T02:56:51.721931+0000","last_unstale":"2023-11-15T02:56:51.725581+0000","last_undegraded":"2023-11-15T02:56:51.725581+0000","last_fullsized":"2023-11-15T02:56:51.725581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.713667+0000","last_change":"2023-11-15T02:56:51.284656+0000","last_active":"2023-11-15T02:56:51.713667+0000","last_peered":"2023-11-15T02:56:51.713667+0000","last_clean":"2023-11-15T02:56:51.713667+0000","last_became_active":"2023-11-15T02:56:51.284378+0000","last_became_peered":"2023-11-15T02:56:51.284378+0000","last_unstale":"2023-11-15T02:56:51.713667+0000","last_undegraded":"2023-11-15T02:56:51.713667+0000","last_fullsized":"2023-11-15T02:56:51.713667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747336+0000","last_change":"2023-11-15T02:56:51.743546+0000","last_active":"2023-11-15T02:56:51.747336+0000","last_peered":"2023-11-15T02:56:51.747336+0000","last_clean":"2023-11-15T02:56:51.747336+0000","last_became_active":"2023-11-15T02:56:51.743276+0000","last_became_peered":"2023-11-15T02:56:51.743276+0000","last_unstale":"2023-11-15T02:56:51.747336+0000","last_undegraded":"2023-11-15T02:56:51.747336+0000","last_fullsized":"2023-11-15T02:56:51.747336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.729927+0000","last_change":"2023-11-15T02:56:51.285769+0000","last_active":"2023-11-15T02:56:51.729927+0000","last_peered":"2023-11-15T02:56:51.729927+0000","last_clean":"2023-11-15T02:56:51.729927+0000","last_became_active":"2023-11-15T02:56:51.285448+0000","last_became_peered":"2023-11-15T02:56:51.285448+0000","last_unstale":"2023-11-15T02:56:51.729927+0000","last_undegraded":"2023-11-15T02:56:51.729927+0000","last_fullsized":"2023-11-15T02:56:51.729927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.742411+0000","last_change":"2023-11-15T02:56:51.736745+0000","last_active":"2023-11-15T02:56:51.742411+0000","last_peered":"2023-11-15T02:56:51.742411+0000","last_clean":"2023-11-15T02:56:51.742411+0000","last_became_active":"2023-11-15T02:56:51.736555+0000","last_became_peered":"2023-11-15T02:56:51.736555+0000","last_unstale":"2023-11-15T02:56:51.742411+0000","last_undegraded":"2023-11-15T02:56:51.742411+0000","last_fullsized":"2023-11-15T02:56:51.742411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730754+0000","last_change":"2023-11-15T02:56:51.722343+0000","last_active":"2023-11-15T02:56:51.730754+0000","last_peered":"2023-11-15T02:56:51.730754+0000","last_clean":"2023-11-15T02:56:51.730754+0000","last_became_active":"2023-11-15T02:56:51.722090+0000","last_became_peered":"2023-11-15T02:56:51.722090+0000","last_unstale":"2023-11-15T02:56:51.730754+0000","last_undegraded":"2023-11-15T02:56:51.730754+0000","last_fullsized":"2023-11-15T02:56:51.730754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.710708+0000","last_change":"2023-11-15T02:56:51.281122+0000","last_active":"2023-11-15T02:56:51.710708+0000","last_peered":"2023-11-15T02:56:51.710708+0000","last_clean":"2023-11-15T02:56:51.710708+0000","last_became_active":"2023-11-15T02:56:51.280798+0000","last_became_peered":"2023-11-15T02:56:51.280798+0000","last_unstale":"2023-11-15T02:56:51.710708+0000","last_undegraded":"2023-11-15T02:56:51.710708+0000","last_fullsized":"2023-11-15T02:56:51.710708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.729313+0000","last_change":"2023-11-15T02:56:51.726756+0000","last_active":"2023-11-15T02:56:51.729313+0000","last_peered":"2023-11-15T02:56:51.729313+0000","last_clean":"2023-11-15T02:56:51.729313+0000","last_became_active":"2023-11-15T02:56:51.726446+0000","last_became_peered":"2023-11-15T02:56:51.726446+0000","last_unstale":"2023-11-15T02:56:51.729313+0000","last_undegraded":"2023-11-15T02:56:51.729313+0000","last_fullsized":"2023-11-15T02:56:51.729313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.730560+0000","last_change":"2023-11-15T02:56:51.288389+0000","last_active":"2023-11-15T02:56:51.730560+0000","last_peered":"2023-11-15T02:56:51.730560+0000","last_clean":"2023-11-15T02:56:51.730560+0000","last_became_active":"2023-11-15T02:56:51.287933+0000","last_became_peered":"2023-11-15T02:56:51.287933+0000","last_unstale":"2023-11-15T02:56:51.730560+0000","last_undegraded":"2023-11-15T02:56:51.730560+0000","last_fullsized":"2023-11-15T02:56:51.730560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.746552+0000","last_change":"2023-11-15T02:56:51.742907+0000","last_active":"2023-11-15T02:56:51.746552+0000","last_peered":"2023-11-15T02:56:51.746552+0000","last_clean":"2023-11-15T02:56:51.746552+0000","last_became_active":"2023-11-15T02:56:51.742633+0000","last_became_peered":"2023-11-15T02:56:51.742633+0000","last_unstale":"2023-11-15T02:56:51.746552+0000","last_undegraded":"2023-11-15T02:56:51.746552+0000","last_fullsized":"2023-11-15T02:56:51.746552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.736908+0000","last_change":"2023-11-15T02:56:51.278389+0000","last_active":"2023-11-15T02:56:51.736908+0000","last_peered":"2023-11-15T02:56:51.736908+0000","last_clean":"2023-11-15T02:56:51.736908+0000","last_became_active":"2023-11-15T02:56:51.277985+0000","last_became_peered":"2023-11-15T02:56:51.277985+0000","last_unstale":"2023-11-15T02:56:51.736908+0000","last_undegraded":"2023-11-15T02:56:51.736908+0000","last_fullsized":"2023-11-15T02:56:51.736908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.743771+0000","last_change":"2023-11-15T02:56:51.725491+0000","last_active":"2023-11-15T02:56:51.743771+0000","last_peered":"2023-11-15T02:56:51.743771+0000","last_clean":"2023-11-15T02:56:51.743771+0000","last_became_active":"2023-11-15T02:56:51.725072+0000","last_became_peered":"2023-11-15T02:56:51.725072+0000","last_unstale":"2023-11-15T02:56:51.743771+0000","last_undegraded":"2023-11-15T02:56:51.743771+0000","last_fullsized":"2023-11-15T02:56:51.743771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.738751+0000","last_change":"2023-11-15T02:56:51.291272+0000","last_active":"2023-11-15T02:56:51.738751+0000","last_peered":"2023-11-15T02:56:51.738751+0000","last_clean":"2023-11-15T02:56:51.738751+0000","last_became_active":"2023-11-15T02:56:51.287508+0000","last_became_peered":"2023-11-15T02:56:51.287508+0000","last_unstale":"2023-11-15T02:56:51.738751+0000","last_undegraded":"2023-11-15T02:56:51.738751+0000","last_fullsized":"2023-11-15T02:56:51.738751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.745082+0000","last_change":"2023-11-15T02:56:51.741837+0000","last_active":"2023-11-15T02:56:51.745082+0000","last_peered":"2023-11-15T02:56:51.745082+0000","last_clean":"2023-11-15T02:56:51.745082+0000","last_became_active":"2023-11-15T02:56:51.741487+0000","last_became_peered":"2023-11-15T02:56:51.741487+0000","last_unstale":"2023-11-15T02:56:51.745082+0000","last_undegraded":"2023-11-15T02:56:51.745082+0000","last_fullsized":"2023-11-15T02:56:51.745082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.741740+0000","last_change":"2023-11-15T02:56:51.283591+0000","last_active":"2023-11-15T02:56:51.741740+0000","last_peered":"2023-11-15T02:56:51.741740+0000","last_clean":"2023-11-15T02:56:51.741740+0000","last_became_active":"2023-11-15T02:56:51.283198+0000","last_became_peered":"2023-11-15T02:56:51.283198+0000","last_unstale":"2023-11-15T02:56:51.741740+0000","last_undegraded":"2023-11-15T02:56:51.741740+0000","last_fullsized":"2023-11-15T02:56:51.741740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.744198+0000","last_change":"2023-11-15T02:56:51.739304+0000","last_active":"2023-11-15T02:56:51.744198+0000","last_peered":"2023-11-15T02:56:51.744198+0000","last_clean":"2023-11-15T02:56:51.744198+0000","last_became_active":"2023-11-15T02:56:51.738755+0000","last_became_peered":"2023-11-15T02:56:51.738755+0000","last_unstale":"2023-11-15T02:56:51.744198+0000","last_undegraded":"2023-11-15T02:56:51.744198+0000","last_fullsized":"2023-11-15T02:56:51.744198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.722852+0000","last_change":"2023-11-15T02:56:51.269310+0000","last_active":"2023-11-15T02:56:51.722852+0000","last_peered":"2023-11-15T02:56:51.722852+0000","last_clean":"2023-11-15T02:56:51.722852+0000","last_became_active":"2023-11-15T02:56:51.268977+0000","last_became_peered":"2023-11-15T02:56:51.268977+0000","last_unstale":"2023-11-15T02:56:51.722852+0000","last_undegraded":"2023-11-15T02:56:51.722852+0000","last_fullsized":"2023-11-15T02:56:51.722852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747235+0000","last_change":"2023-11-15T02:56:51.742002+0000","last_active":"2023-11-15T02:56:51.747235+0000","last_peered":"2023-11-15T02:56:51.747235+0000","last_clean":"2023-11-15T02:56:51.747235+0000","last_became_active":"2023-11-15T02:56:51.741691+0000","last_became_peered":"2023-11-15T02:56:51.741691+0000","last_unstale":"2023-11-15T02:56:51.747235+0000","last_undegraded":"2023-11-15T02:56:51.747235+0000","last_fullsized":"2023-11-15T02:56:51.747235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.706447+0000","last_change":"2023-11-15T02:56:51.280160+0000","last_active":"2023-11-15T02:56:51.706447+0000","last_peered":"2023-11-15T02:56:51.706447+0000","last_clean":"2023-11-15T02:56:51.706447+0000","last_became_active":"2023-11-15T02:56:51.278364+0000","last_became_peered":"2023-11-15T02:56:51.278364+0000","last_unstale":"2023-11-15T02:56:51.706447+0000","last_undegraded":"2023-11-15T02:56:51.706447+0000","last_fullsized":"2023-11-15T02:56:51.706447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.747191+0000","last_change":"2023-11-15T02:56:51.742487+0000","last_active":"2023-11-15T02:56:51.747191+0000","last_peered":"2023-11-15T02:56:51.747191+0000","last_clean":"2023-11-15T02:56:51.747191+0000","last_became_active":"2023-11-15T02:56:51.742056+0000","last_became_peered":"2023-11-15T02:56:51.742056+0000","last_unstale":"2023-11-15T02:56:51.747191+0000","last_undegraded":"2023-11-15T02:56:51.747191+0000","last_fullsized":"2023-11-15T02:56:51.747191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:51.226965+0000","last_clean_scrub_stamp":"2023-11-15T02:56:51.226965+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-15T02:56:51.748052+0000","last_change":"2023-11-15T02:56:51.290964+0000","last_active":"2023-11-15T02:56:51.748052+0000","last_peered":"2023-11-15T02:56:51.748052+0000","last_clean":"2023-11-15T02:56:51.748052+0000","last_became_active":"2023-11-15T02:56:51.290728+0000","last_became_peered":"2023-11-15T02:56:51.290728+0000","last_unstale":"2023-11-15T02:56:51.748052+0000","last_undegraded":"2023-11-15T02:56:51.748052+0000","last_fullsized":"2023-11-15T02:56:51.748052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:50.213456+0000","last_clean_scrub_stamp":"2023-11-15T02:56:50.213456+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297332,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074508,"statfs":{"total":96636764160,"available":96332296192,"internally_reserved":0,"allocated":241664,"data_stored":80461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2769999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297312,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074528,"statfs":{"total":96636764160,"available":96332316672,"internally_reserved":0,"allocated":221184,"data_stored":80493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4590000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297324,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074516,"statfs":{"total":96636764160,"available":96332304384,"internally_reserved":0,"allocated":233472,"data_stored":80639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297308,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074532,"statfs":{"total":96636764160,"available":96332320768,"internally_reserved":0,"allocated":217088,"data_stored":79137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.712}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.794}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7769999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":77848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.472}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3140000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5640000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":79047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.341}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":79047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.224}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":79047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T02:57:04.701 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-15T02:57:04.701 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-15T02:57:04.701 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-15T02:57:04.702 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-15T02:57:04.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.891+0000 7ff699cb9700 1 Processor -- start 2023-11-15T02:57:04.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.891+0000 7ff699cb9700 1 -- start start 2023-11-15T02:57:04.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.895+0000 7ff699cb9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff6940cabd0 0x7ff6940cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:04.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.895+0000 7ff699cb9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff6940acc90 0x7ff694066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:04.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.895+0000 7ff699cb9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6940ad650 con 0x7ff6940ad0f0 2023-11-15T02:57:04.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.895+0000 7ff699cb9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff694065fc0 con 0x7ff6940cabd0 2023-11-15T02:57:04.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.895+0000 7ff699cb9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6940662a0 con 0x7ff6940acc90 2023-11-15T02:57:04.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.895+0000 7ff698cb7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff6940cabd0 0x7ff6940cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:04.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.895+0000 7ff698cb7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff6940cabd0 0x7ff6940cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49652/0 (socket says 172.21.15.121:49652) 2023-11-15T02:57:04.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.895+0000 7ff6994b8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff6940ad0f0 0x7ff6940ae190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59626/0 (socket says 172.21.15.121:59626) 2023-11-15T02:57:04.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.895+0000 7ff698cb7700 1 -- 172.21.15.121:0/2634947110 learned_addr learned my addr 172.21.15.121:0/2634947110 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:04.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.895+0000 7ff693fff700 1 --2- 172.21.15.121:0/2634947110 >> v2:172.21.15.190:3300/0 conn(0x7ff6940acc90 0x7ff694066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:04.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.895+0000 7ff698cb7700 1 -- 172.21.15.121:0/2634947110 >> v2:172.21.15.190:3300/0 conn(0x7ff6940acc90 msgr2=0x7ff694066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:04.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.899+0000 7ff698cb7700 1 --2- 172.21.15.121:0/2634947110 >> v2:172.21.15.190:3300/0 conn(0x7ff6940acc90 0x7ff694066d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.899+0000 7ff698cb7700 1 -- 172.21.15.121:0/2634947110 >> v1:172.21.15.121:6789/0 conn(0x7ff6940ad0f0 legacy=0x7ff6940ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:04.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.899+0000 7ff6937fe700 1 -- 172.21.15.121:0/2634947110 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56883918 0 0) 0x7ff6940ad650 con 0x7ff6940ad0f0 2023-11-15T02:57:04.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.899+0000 7ff698cb7700 1 -- 172.21.15.121:0/2634947110 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff694066580 con 0x7ff6940cabd0 2023-11-15T02:57:04.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.899+0000 7ff698cb7700 1 --2- 172.21.15.121:0/2634947110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6940cabd0 0x7ff6940cafb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7ff6940c8860 tx=0x7ff6880090f0).ready entity=mon.0 client_cookie=b6cb1d6946c645b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:04.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.899+0000 7ff6937fe700 1 -- 172.21.15.121:0/2634947110 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68800b520 con 0x7ff6940cabd0 2023-11-15T02:57:04.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.899+0000 7ff6937fe700 1 -- 172.21.15.121:0/2634947110 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68800b680 con 0x7ff6940cabd0 2023-11-15T02:57:04.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.899+0000 7ff6937fe700 1 -- 172.21.15.121:0/2634947110 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68800ba00 con 0x7ff6940cabd0 2023-11-15T02:57:04.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.899+0000 7ff699cb9700 1 -- 172.21.15.121:0/2634947110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6940cabd0 msgr2=0x7ff6940cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:04.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.899+0000 7ff699cb9700 1 --2- 172.21.15.121:0/2634947110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6940cabd0 0x7ff6940cafb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7ff6940c8860 tx=0x7ff6880090f0).stop 2023-11-15T02:57:04.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.899+0000 7ff699cb9700 1 -- 172.21.15.121:0/2634947110 shutdown_connections 2023-11-15T02:57:04.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.899+0000 7ff699cb9700 1 --2- 172.21.15.121:0/2634947110 >> v2:172.21.15.190:3300/0 conn(0x7ff6940acc90 0x7ff694066d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.899+0000 7ff699cb9700 1 --2- 172.21.15.121:0/2634947110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6940cabd0 0x7ff6940cafb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.899+0000 7ff699cb9700 1 -- 172.21.15.121:0/2634947110 >> 172.21.15.121:0/2634947110 conn(0x7ff694062190 msgr2=0x7ff6940625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:04.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.899+0000 7ff699cb9700 1 -- 172.21.15.121:0/2634947110 shutdown_connections 2023-11-15T02:57:04.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.899+0000 7ff699cb9700 1 -- 172.21.15.121:0/2634947110 wait complete. 2023-11-15T02:57:04.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff699cb9700 1 Processor -- start 2023-11-15T02:57:04.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff699cb9700 1 -- start start 2023-11-15T02:57:04.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff699cb9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6940acc90 0x7ff694140030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:04.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff699cb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6940ad0f0 0x7ff694062570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:04.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff699cb9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6940cabd0 0x7ff694062a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:04.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff699cb9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff694156cb0 con 0x7ff6940cabd0 2023-11-15T02:57:04.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff699cb9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff694062f50 con 0x7ff6940acc90 2023-11-15T02:57:04.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff699cb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff694140a90 con 0x7ff6940ad0f0 2023-11-15T02:57:04.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff6994b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6940cabd0 0x7ff694062a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:04.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff6994b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6940cabd0 0x7ff694062a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49666/0 (socket says 172.21.15.121:49666) 2023-11-15T02:57:04.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff6994b8700 1 -- 172.21.15.121:0/979992768 learned_addr learned my addr 172.21.15.121:0/979992768 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:04.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff693fff700 1 --2- 172.21.15.121:0/979992768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6940ad0f0 0x7ff694062570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:04.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff698cb7700 1 --2- 172.21.15.121:0/979992768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6940acc90 0x7ff694140030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:04.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff6994b8700 1 -- 172.21.15.121:0/979992768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6940ad0f0 msgr2=0x7ff694062570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:04.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff6994b8700 1 --2- 172.21.15.121:0/979992768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6940ad0f0 0x7ff694062570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff6994b8700 1 -- 172.21.15.121:0/979992768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6940acc90 msgr2=0x7ff694140030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:04.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff6994b8700 1 --2- 172.21.15.121:0/979992768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6940acc90 0x7ff694140030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:04.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff6994b8700 1 -- 172.21.15.121:0/979992768 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff688009370 con 0x7ff6940cabd0 2023-11-15T02:57:04.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff693fff700 1 --2- 172.21.15.121:0/979992768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6940ad0f0 0x7ff694062570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:04.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.903+0000 7ff6994b8700 1 --2- 172.21.15.121:0/979992768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6940cabd0 0x7ff694062a60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7ff68c00c7e0 tx=0x7ff68c007f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:04.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.907+0000 7ff690ff9700 1 -- 172.21.15.121:0/979992768 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c005c70 con 0x7ff6940cabd0 2023-11-15T02:57:04.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.907+0000 7ff690ff9700 1 -- 172.21.15.121:0/979992768 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68c005dd0 con 0x7ff6940cabd0 2023-11-15T02:57:04.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.907+0000 7ff690ff9700 1 -- 172.21.15.121:0/979992768 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c007320 con 0x7ff6940cabd0 2023-11-15T02:57:04.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.907+0000 7ff699cb9700 1 -- 172.21.15.121:0/979992768 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff694154860 con 0x7ff6940cabd0 2023-11-15T02:57:04.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.907+0000 7ff699cb9700 1 -- 172.21.15.121:0/979992768 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6941544f0 con 0x7ff6940cabd0 2023-11-15T02:57:04.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.907+0000 7ff699cb9700 1 -- 172.21.15.121:0/979992768 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6941549f0 con 0x7ff6940cabd0 2023-11-15T02:57:04.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.907+0000 7ff690ff9700 1 -- 172.21.15.121:0/979992768 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff68c007710 con 0x7ff6940cabd0 2023-11-15T02:57:04.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.907+0000 7ff690ff9700 1 --2- 172.21.15.121:0/979992768 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff67805a510 0x7ff67805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:04.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.907+0000 7ff690ff9700 1 -- 172.21.15.121:0/979992768 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff68c074760 con 0x7ff6940cabd0 2023-11-15T02:57:04.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.911+0000 7ff698cb7700 1 --2- 172.21.15.121:0/979992768 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff67805a510 0x7ff67805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:04.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.911+0000 7ff698cb7700 1 --2- 172.21.15.121:0/979992768 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff67805a510 0x7ff67805c9d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7ff688009130 tx=0x7ff688017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:04.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:04.915+0000 7ff690ff9700 1 -- 172.21.15.121:0/979992768 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff68c047f40 con 0x7ff6940cabd0 2023-11-15T02:57:05.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.107+0000 7ff699cb9700 1 -- 172.21.15.121:0/979992768 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff694066860 con 0x7ff6940cabd0 2023-11-15T02:57:05.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.107+0000 7ff690ff9700 1 -- 172.21.15.121:0/979992768 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7ff68c04b560 con 0x7ff6940cabd0 2023-11-15T02:57:05.114 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:05.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.111+0000 7ff699cb9700 1 -- 172.21.15.121:0/979992768 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff67805a510 msgr2=0x7ff67805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:05.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.111+0000 7ff699cb9700 1 --2- 172.21.15.121:0/979992768 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff67805a510 0x7ff67805c9d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7ff688009130 tx=0x7ff688017000).stop 2023-11-15T02:57:05.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.111+0000 7ff699cb9700 1 -- 172.21.15.121:0/979992768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6940cabd0 msgr2=0x7ff694062a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:05.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.111+0000 7ff699cb9700 1 --2- 172.21.15.121:0/979992768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6940cabd0 0x7ff694062a60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7ff68c00c7e0 tx=0x7ff68c007f60).stop 2023-11-15T02:57:05.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.111+0000 7ff699cb9700 1 -- 172.21.15.121:0/979992768 shutdown_connections 2023-11-15T02:57:05.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.111+0000 7ff699cb9700 1 --2- 172.21.15.121:0/979992768 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff67805a510 0x7ff67805c9d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.111+0000 7ff699cb9700 1 --2- 172.21.15.121:0/979992768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6940cabd0 0x7ff694062a60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.111+0000 7ff699cb9700 1 --2- 172.21.15.121:0/979992768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6940ad0f0 0x7ff694062570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.111+0000 7ff699cb9700 1 --2- 172.21.15.121:0/979992768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6940acc90 0x7ff694140030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.111+0000 7ff699cb9700 1 -- 172.21.15.121:0/979992768 >> 172.21.15.121:0/979992768 conn(0x7ff694062190 msgr2=0x7ff6940ae470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:05.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.111+0000 7ff699cb9700 1 -- 172.21.15.121:0/979992768 shutdown_connections 2023-11-15T02:57:05.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.111+0000 7ff699cb9700 1 -- 172.21.15.121:0/979992768 wait complete. 2023-11-15T02:57:05.137 INFO:teuthology.orchestra.run.smithi121.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-15T02:57:05.138 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-15T02:57:05.138 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:57:05.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.299+0000 7f1449d69700 1 Processor -- start 2023-11-15T02:57:05.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.299+0000 7f1449d69700 1 -- start start 2023-11-15T02:57:05.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.303+0000 7f1449d69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14440c4470 0x7f14440c4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:05.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.303+0000 7f1449d69700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f14440c5440 0x7f14440c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:05.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.303+0000 7f1449d69700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14440bc840 con 0x7f14440c23d0 2023-11-15T02:57:05.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.303+0000 7f1449d69700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14440c5bb0 con 0x7f14440c5440 2023-11-15T02:57:05.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.303+0000 7f1449d69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14440c7010 con 0x7f14440c4470 2023-11-15T02:57:05.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.303+0000 7f1442ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f14440c5440 0x7f14440c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:05.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.303+0000 7f1443fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f14440c23d0 0x7f14440bcfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:58080/0 (socket says 172.21.15.121:58080) 2023-11-15T02:57:05.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.303+0000 7f1443fff700 1 -- 172.21.15.121:0/3070204255 learned_addr learned my addr 172.21.15.121:0/3070204255 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:05.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.303+0000 7f14437fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14440c4470 0x7f14440c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:05.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.303+0000 7f1442ffd700 1 --2- 172.21.15.121:0/3070204255 >> v2:172.21.15.190:3300/0 conn(0x7f14440c5440 0x7f14440c1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:55740/0 (socket says 172.21.15.121:55740) 2023-11-15T02:57:05.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.307+0000 7f14427fc700 1 -- 172.21.15.121:0/3070204255 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3073375568 0 0) 0x7f14440bc840 con 0x7f14440c23d0 2023-11-15T02:57:05.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.307+0000 7f14427fc700 1 -- 172.21.15.121:0/3070204255 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f142c0033f0 con 0x7f14440c23d0 2023-11-15T02:57:05.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.307+0000 7f1442ffd700 1 -- 172.21.15.121:0/3070204255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14440c4470 msgr2=0x7f14440c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:05.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.307+0000 7f1442ffd700 1 --2- 172.21.15.121:0/3070204255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14440c4470 0x7f14440c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.307+0000 7f1442ffd700 1 -- 172.21.15.121:0/3070204255 >> v1:172.21.15.190:6789/0 conn(0x7f14440c23d0 legacy=0x7f14440bcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:05.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.307+0000 7f1442ffd700 1 -- 172.21.15.121:0/3070204255 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14440c72b0 con 0x7f14440c5440 2023-11-15T02:57:05.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.307+0000 7f1442ffd700 1 --2- 172.21.15.121:0/3070204255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14440c5440 0x7f14440c1ee0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f1438009790 tx=0x7f1438009060).ready entity=mon.1 client_cookie=d7d69d9f36b59c7e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:05.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.307+0000 7f14427fc700 1 -- 172.21.15.121:0/3070204255 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143800d4a0 con 0x7f14440c5440 2023-11-15T02:57:05.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.307+0000 7f14427fc700 1 -- 172.21.15.121:0/3070204255 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143800d600 con 0x7f14440c5440 2023-11-15T02:57:05.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.307+0000 7f14427fc700 1 -- 172.21.15.121:0/3070204255 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143800d980 con 0x7f14440c5440 2023-11-15T02:57:05.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.307+0000 7f1449d69700 1 -- 172.21.15.121:0/3070204255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14440c5440 msgr2=0x7f14440c1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:05.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.307+0000 7f1449d69700 1 --2- 172.21.15.121:0/3070204255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14440c5440 0x7f14440c1ee0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f1438009790 tx=0x7f1438009060).stop 2023-11-15T02:57:05.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.307+0000 7f1449d69700 1 -- 172.21.15.121:0/3070204255 shutdown_connections 2023-11-15T02:57:05.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.307+0000 7f1449d69700 1 --2- 172.21.15.121:0/3070204255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14440c5440 0x7f14440c1ee0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.307+0000 7f1449d69700 1 --2- 172.21.15.121:0/3070204255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14440c4470 0x7f14440c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.307+0000 7f1449d69700 1 -- 172.21.15.121:0/3070204255 >> 172.21.15.121:0/3070204255 conn(0x7f1444066ee0 msgr2=0x7f1444090f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:05.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.307+0000 7f1449d69700 1 -- 172.21.15.121:0/3070204255 shutdown_connections 2023-11-15T02:57:05.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.307+0000 7f1449d69700 1 -- 172.21.15.121:0/3070204255 wait complete. 2023-11-15T02:57:05.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.311+0000 7f1449d69700 1 Processor -- start 2023-11-15T02:57:05.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.311+0000 7f1449d69700 1 -- start start 2023-11-15T02:57:05.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.311+0000 7f1449d69700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14440c23d0 0x7f14441520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:05.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.311+0000 7f1449d69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14440c4470 0x7f1444153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:05.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.311+0000 7f1449d69700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14440c5440 0x7f14440a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:05.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.311+0000 7f1449d69700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1444159970 con 0x7f14440c23d0 2023-11-15T02:57:05.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.311+0000 7f1449d69700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1444157b00 con 0x7f14440c5440 2023-11-15T02:57:05.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.311+0000 7f1449d69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1444157de0 con 0x7f14440c4470 2023-11-15T02:57:05.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.311+0000 7f14437fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14440c23d0 0x7f14441520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:05.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.311+0000 7f14437fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14440c23d0 0x7f14441520b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49682/0 (socket says 172.21.15.121:49682) 2023-11-15T02:57:05.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.311+0000 7f14437fe700 1 -- 172.21.15.121:0/506652753 learned_addr learned my addr 172.21.15.121:0/506652753 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:05.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.311+0000 7f1442ffd700 1 --2- 172.21.15.121:0/506652753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14440c4470 0x7f1444153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:05.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.311+0000 7f1443fff700 1 --2- 172.21.15.121:0/506652753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14440c5440 0x7f14440a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:05.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.311+0000 7f14437fe700 1 -- 172.21.15.121:0/506652753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14440c4470 msgr2=0x7f1444153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:05.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.311+0000 7f14437fe700 1 --2- 172.21.15.121:0/506652753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14440c4470 0x7f1444153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.311+0000 7f14437fe700 1 -- 172.21.15.121:0/506652753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14440c5440 msgr2=0x7f14440a5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:05.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.311+0000 7f14437fe700 1 --2- 172.21.15.121:0/506652753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14440c5440 0x7f14440a5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.311+0000 7f14437fe700 1 -- 172.21.15.121:0/506652753 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14380092a0 con 0x7f14440c23d0 2023-11-15T02:57:05.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.311+0000 7f14437fe700 1 --2- 172.21.15.121:0/506652753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14440c23d0 0x7f14441520b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f143400e8d0 tx=0x7f1434006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:05.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.315+0000 7f142b7fe700 1 -- 172.21.15.121:0/506652753 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434009450 con 0x7f14440c23d0 2023-11-15T02:57:05.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.315+0000 7f142b7fe700 1 -- 172.21.15.121:0/506652753 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1434012600 con 0x7f14440c23d0 2023-11-15T02:57:05.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.315+0000 7f142b7fe700 1 -- 172.21.15.121:0/506652753 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434012980 con 0x7f14440c23d0 2023-11-15T02:57:05.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.315+0000 7f1449d69700 1 -- 172.21.15.121:0/506652753 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14441580c0 con 0x7f14440c23d0 2023-11-15T02:57:05.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.315+0000 7f1449d69700 1 -- 172.21.15.121:0/506652753 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1444158680 con 0x7f14440c23d0 2023-11-15T02:57:05.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.315+0000 7f1449d69700 1 -- 172.21.15.121:0/506652753 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1444157200 con 0x7f14440c23d0 2023-11-15T02:57:05.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.315+0000 7f142b7fe700 1 -- 172.21.15.121:0/506652753 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1434012cf0 con 0x7f14440c23d0 2023-11-15T02:57:05.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.319+0000 7f142b7fe700 1 --2- 172.21.15.121:0/506652753 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f143005a510 0x7f143005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:05.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.319+0000 7f142b7fe700 1 -- 172.21.15.121:0/506652753 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1434074c40 con 0x7f14440c23d0 2023-11-15T02:57:05.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.319+0000 7f1442ffd700 1 --2- 172.21.15.121:0/506652753 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f143005a510 0x7f143005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:05.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.319+0000 7f1442ffd700 1 --2- 172.21.15.121:0/506652753 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f143005a510 0x7f143005c9d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f1438009790 tx=0x7f1438016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:05.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.327+0000 7f142b7fe700 1 -- 172.21.15.121:0/506652753 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1444157200 con 0x7f14440c23d0 2023-11-15T02:57:05.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.491+0000 7f1449d69700 1 -- 172.21.15.121:0/506652753 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1444157200 con 0x7f14440c23d0 2023-11-15T02:57:05.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.495+0000 7f142b7fe700 1 -- 172.21.15.121:0/506652753 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f1444157200 con 0x7f14440c23d0 2023-11-15T02:57:05.500 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:05.500 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":21,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:52.702668+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:56:49.726353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-15T02:56:50.688098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:57:05.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.495+0000 7f1449d69700 1 -- 172.21.15.121:0/506652753 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f143005a510 msgr2=0x7f143005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:05.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.495+0000 7f1449d69700 1 --2- 172.21.15.121:0/506652753 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f143005a510 0x7f143005c9d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f1438009790 tx=0x7f1438016000).stop 2023-11-15T02:57:05.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.495+0000 7f1449d69700 1 -- 172.21.15.121:0/506652753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14440c23d0 msgr2=0x7f14441520b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:05.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.495+0000 7f1449d69700 1 --2- 172.21.15.121:0/506652753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14440c23d0 0x7f14441520b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f143400e8d0 tx=0x7f1434006f40).stop 2023-11-15T02:57:05.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.495+0000 7f1449d69700 1 -- 172.21.15.121:0/506652753 shutdown_connections 2023-11-15T02:57:05.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.495+0000 7f1449d69700 1 --2- 172.21.15.121:0/506652753 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f143005a510 0x7f143005c9d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.495+0000 7f1449d69700 1 --2- 172.21.15.121:0/506652753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14440c5440 0x7f14440a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.495+0000 7f1449d69700 1 --2- 172.21.15.121:0/506652753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14440c4470 0x7f1444153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.495+0000 7f1449d69700 1 --2- 172.21.15.121:0/506652753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14440c23d0 0x7f14441520b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.495+0000 7f1449d69700 1 -- 172.21.15.121:0/506652753 >> 172.21.15.121:0/506652753 conn(0x7f1444066ee0 msgr2=0x7f14440c27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:05.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.495+0000 7f1449d69700 1 -- 172.21.15.121:0/506652753 shutdown_connections 2023-11-15T02:57:05.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.495+0000 7f1449d69700 1 -- 172.21.15.121:0/506652753 wait complete. 2023-11-15T02:57:05.524 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-15T02:56:49.726353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-15T02:56:50.688098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-15T02:57:05.526 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T02:57:05.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.683+0000 7f6649679700 1 Processor -- start 2023-11-15T02:57:05.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.683+0000 7f6649679700 1 -- start start 2023-11-15T02:57:05.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.687+0000 7f6649679700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f66440a8d40 0x7f66440c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:05.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.687+0000 7f6649679700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66440657f0 con 0x7f6644066520 2023-11-15T02:57:05.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.687+0000 7f6649679700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66440c1b60 con 0x7f66440a8960 2023-11-15T02:57:05.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.687+0000 7f6649679700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66441583d0 con 0x7f66440a8d40 2023-11-15T02:57:05.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.687+0000 7f6648e78700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6644066520 0x7f66440c2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59638/0 (socket says 172.21.15.121:59638) 2023-11-15T02:57:05.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.687+0000 7f6648e78700 1 -- 172.21.15.121:0/2124811917 learned_addr learned my addr 172.21.15.121:0/2124811917 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:05.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.687+0000 7f663bfff700 1 --2- 172.21.15.121:0/2124811917 >> v2:172.21.15.190:3300/0 conn(0x7f66440a8d40 0x7f66440c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:05.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.687+0000 7f66437fe700 1 -- 172.21.15.121:0/2124811917 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020411571 0 0) 0x7f66440657f0 con 0x7f6644066520 2023-11-15T02:57:05.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.687+0000 7f66437fe700 1 -- 172.21.15.121:0/2124811917 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66300033f0 con 0x7f6644066520 2023-11-15T02:57:05.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.687+0000 7f66437fe700 1 -- 172.21.15.121:0/2124811917 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098855497 0 0) 0x7f66440c1b60 con 0x7f66440a8960 2023-11-15T02:57:05.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.687+0000 7f66437fe700 1 -- 172.21.15.121:0/2124811917 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66440657f0 con 0x7f66440a8960 2023-11-15T02:57:05.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.687+0000 7f663bfff700 1 -- 172.21.15.121:0/2124811917 >> v1:172.21.15.190:6789/0 conn(0x7f66440a8960 legacy=0x7f664415c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:05.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.687+0000 7f663bfff700 1 -- 172.21.15.121:0/2124811917 >> v1:172.21.15.121:6789/0 conn(0x7f6644066520 legacy=0x7f66440c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:05.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.687+0000 7f663bfff700 1 -- 172.21.15.121:0/2124811917 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6644065a40 con 0x7f66440a8d40 2023-11-15T02:57:05.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.691+0000 7f663bfff700 1 --2- 172.21.15.121:0/2124811917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66440a8d40 0x7f66440c15c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f662c009760 tx=0x7f662c009060).ready entity=mon.1 client_cookie=13330ecb2d547d06 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:05.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.691+0000 7f66437fe700 1 -- 172.21.15.121:0/2124811917 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c00d4a0 con 0x7f66440a8d40 2023-11-15T02:57:05.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.691+0000 7f66437fe700 1 -- 172.21.15.121:0/2124811917 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662c00d600 con 0x7f66440a8d40 2023-11-15T02:57:05.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.691+0000 7f66437fe700 1 -- 172.21.15.121:0/2124811917 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c00d980 con 0x7f66440a8d40 2023-11-15T02:57:05.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.691+0000 7f6649679700 1 -- 172.21.15.121:0/2124811917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66440a8d40 msgr2=0x7f66440c15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:05.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.691+0000 7f6649679700 1 --2- 172.21.15.121:0/2124811917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66440a8d40 0x7f66440c15c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f662c009760 tx=0x7f662c009060).stop 2023-11-15T02:57:05.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.691+0000 7f6649679700 1 -- 172.21.15.121:0/2124811917 shutdown_connections 2023-11-15T02:57:05.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.691+0000 7f6649679700 1 --2- 172.21.15.121:0/2124811917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66440a8d40 0x7f66440c15c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.691+0000 7f6649679700 1 -- 172.21.15.121:0/2124811917 >> 172.21.15.121:0/2124811917 conn(0x7f6644067a60 msgr2=0x7f6644067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:05.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.691+0000 7f6649679700 1 -- 172.21.15.121:0/2124811917 shutdown_connections 2023-11-15T02:57:05.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.691+0000 7f6649679700 1 -- 172.21.15.121:0/2124811917 wait complete. 2023-11-15T02:57:05.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.691+0000 7f6649679700 1 Processor -- start 2023-11-15T02:57:05.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.691+0000 7f6649679700 1 -- start start 2023-11-15T02:57:05.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f6649679700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6644066520 0x7f6644168110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:05.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f6649679700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66440a8960 0x7f664416af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:05.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f6649679700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66440a8d40 0x7f66440c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:05.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f6649679700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66440bb570 con 0x7f66440a8960 2023-11-15T02:57:05.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f6649679700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6644168600 con 0x7f6644066520 2023-11-15T02:57:05.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f6649679700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66441561d0 con 0x7f66440a8d40 2023-11-15T02:57:05.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f663bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66440a8960 0x7f664416af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:05.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f663bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66440a8960 0x7f664416af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49684/0 (socket says 172.21.15.121:49684) 2023-11-15T02:57:05.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f663bfff700 1 -- 172.21.15.121:0/3229681947 learned_addr learned my addr 172.21.15.121:0/3229681947 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:05.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f6643fff700 1 --2- 172.21.15.121:0/3229681947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6644066520 0x7f6644168110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:05.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f6648e78700 1 --2- 172.21.15.121:0/3229681947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66440a8d40 0x7f66440c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:05.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f663bfff700 1 -- 172.21.15.121:0/3229681947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66440a8d40 msgr2=0x7f66440c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:05.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f663bfff700 1 --2- 172.21.15.121:0/3229681947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66440a8d40 0x7f66440c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f663bfff700 1 -- 172.21.15.121:0/3229681947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6644066520 msgr2=0x7f6644168110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:05.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f663bfff700 1 --2- 172.21.15.121:0/3229681947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6644066520 0x7f6644168110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f663bfff700 1 -- 172.21.15.121:0/3229681947 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662c0092a0 con 0x7f66440a8960 2023-11-15T02:57:05.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f663bfff700 1 --2- 172.21.15.121:0/3229681947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66440a8960 0x7f664416af20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f66440c75a0 tx=0x7f662c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:05.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f6640ff9700 1 -- 172.21.15.121:0/3229681947 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c00dd30 con 0x7f66440a8960 2023-11-15T02:57:05.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f6640ff9700 1 -- 172.21.15.121:0/3229681947 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662c00de90 con 0x7f66440a8960 2023-11-15T02:57:05.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f6640ff9700 1 -- 172.21.15.121:0/3229681947 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c00d980 con 0x7f66440a8960 2023-11-15T02:57:05.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f6649679700 1 -- 172.21.15.121:0/3229681947 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66440bf960 con 0x7f66440a8960 2023-11-15T02:57:05.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.695+0000 7f6649679700 1 -- 172.21.15.121:0/3229681947 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66440bff20 con 0x7f66440a8960 2023-11-15T02:57:05.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.699+0000 7f6649679700 1 -- 172.21.15.121:0/3229681947 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f664406bcf0 con 0x7f66440a8960 2023-11-15T02:57:05.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.699+0000 7f6640ff9700 1 -- 172.21.15.121:0/3229681947 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f662c01d640 con 0x7f66440a8960 2023-11-15T02:57:05.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.699+0000 7f6640ff9700 1 --2- 172.21.15.121:0/3229681947 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f662405a510 0x7f662405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:05.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.699+0000 7f6640ff9700 1 -- 172.21.15.121:0/3229681947 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f662c075df0 con 0x7f66440a8960 2023-11-15T02:57:05.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.699+0000 7f6643fff700 1 --2- 172.21.15.121:0/3229681947 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f662405a510 0x7f662405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:05.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.703+0000 7f6643fff700 1 --2- 172.21.15.121:0/3229681947 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f662405a510 0x7f662405c9d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f6634003930 tx=0x7f6634007bd0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:05.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.707+0000 7f6640ff9700 1 -- 172.21.15.121:0/3229681947 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f662c0495d0 con 0x7f66440a8960 2023-11-15T02:57:05.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.871+0000 7f6649679700 1 -- 172.21.15.121:0/3229681947 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6644156900 con 0x7f66440a8960 2023-11-15T02:57:05.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.871+0000 7f6640ff9700 1 -- 172.21.15.121:0/3229681947 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v21) v1 ==== 110+0+10 (secure 0 0 0) 0x7f662c04cbf0 con 0x7f66440a8960 2023-11-15T02:57:05.877 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T02:57:05.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.871+0000 7f6649679700 1 -- 172.21.15.121:0/3229681947 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f662405a510 msgr2=0x7f662405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:05.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.871+0000 7f6649679700 1 --2- 172.21.15.121:0/3229681947 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f662405a510 0x7f662405c9d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f6634003930 tx=0x7f6634007bd0).stop 2023-11-15T02:57:05.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.871+0000 7f6649679700 1 -- 172.21.15.121:0/3229681947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66440a8960 msgr2=0x7f664416af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:05.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.871+0000 7f6649679700 1 --2- 172.21.15.121:0/3229681947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66440a8960 0x7f664416af20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f66440c75a0 tx=0x7f662c016000).stop 2023-11-15T02:57:05.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.875+0000 7f6649679700 1 -- 172.21.15.121:0/3229681947 shutdown_connections 2023-11-15T02:57:05.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.875+0000 7f6649679700 1 --2- 172.21.15.121:0/3229681947 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f662405a510 0x7f662405c9d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.875+0000 7f6649679700 1 --2- 172.21.15.121:0/3229681947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66440a8d40 0x7f66440c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.875+0000 7f6649679700 1 --2- 172.21.15.121:0/3229681947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66440a8960 0x7f664416af20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.875+0000 7f6649679700 1 --2- 172.21.15.121:0/3229681947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6644066520 0x7f6644168110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:05.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.875+0000 7f6649679700 1 -- 172.21.15.121:0/3229681947 >> 172.21.15.121:0/3229681947 conn(0x7f6644067a60 msgr2=0x7f6644157100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:05.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.875+0000 7f6649679700 1 -- 172.21.15.121:0/3229681947 shutdown_connections 2023-11-15T02:57:05.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:05.875+0000 7f6649679700 1 -- 172.21.15.121:0/3229681947 wait complete. 2023-11-15T02:57:05.902 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T02:57:06.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.063+0000 7f358c8da700 1 Processor -- start 2023-11-15T02:57:06.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.063+0000 7f358c8da700 1 -- start start 2023-11-15T02:57:06.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f358c8da700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3588151af0 0x7f35880a7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:06.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f358c8da700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35880a5720 0x7f35880c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:06.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f358c8da700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35881528b0 con 0x7f35880c3430 2023-11-15T02:57:06.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f358c8da700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3588066d80 con 0x7f3588151af0 2023-11-15T02:57:06.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f358c8da700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3588151ff0 con 0x7f35880a5720 2023-11-15T02:57:06.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f358759e700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f35880c3430 0x7f35880c64a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59646/0 (socket says 172.21.15.121:59646) 2023-11-15T02:57:06.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f358759e700 1 -- 172.21.15.121:0/203726690 learned_addr learned my addr 172.21.15.121:0/203726690 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:06.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f3586d9d700 1 --2- 172.21.15.121:0/203726690 >> v2:172.21.15.190:3300/0 conn(0x7f3588151af0 0x7f35880a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:06.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f358659c700 1 --2- 172.21.15.121:0/203726690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35880a5720 0x7f35880c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:06.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f3585d9b700 1 -- 172.21.15.121:0/203726690 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493154015 0 0) 0x7f35881528b0 con 0x7f35880c3430 2023-11-15T02:57:06.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f3585d9b700 1 -- 172.21.15.121:0/203726690 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35700033f0 con 0x7f35880c3430 2023-11-15T02:57:06.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f3586d9d700 1 -- 172.21.15.121:0/203726690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35880a5720 msgr2=0x7f35880c2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:06.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f3586d9d700 1 --2- 172.21.15.121:0/203726690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35880a5720 0x7f35880c2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f3586d9d700 1 -- 172.21.15.121:0/203726690 >> v1:172.21.15.121:6789/0 conn(0x7f35880c3430 legacy=0x7f35880c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:06.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f3586d9d700 1 -- 172.21.15.121:0/203726690 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35880c3890 con 0x7f3588151af0 2023-11-15T02:57:06.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f3586d9d700 1 --2- 172.21.15.121:0/203726690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3588151af0 0x7f35880a7070 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f3578009760 tx=0x7f3578009060).ready entity=mon.1 client_cookie=b06f48d872749a40 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:06.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f3585d9b700 1 -- 172.21.15.121:0/203726690 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357800d4a0 con 0x7f3588151af0 2023-11-15T02:57:06.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f3585d9b700 1 -- 172.21.15.121:0/203726690 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357800d600 con 0x7f3588151af0 2023-11-15T02:57:06.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f3585d9b700 1 -- 172.21.15.121:0/203726690 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357800d980 con 0x7f3588151af0 2023-11-15T02:57:06.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f358c8da700 1 -- 172.21.15.121:0/203726690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3588151af0 msgr2=0x7f35880a7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:06.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f358c8da700 1 --2- 172.21.15.121:0/203726690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3588151af0 0x7f35880a7070 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f3578009760 tx=0x7f3578009060).stop 2023-11-15T02:57:06.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f358c8da700 1 -- 172.21.15.121:0/203726690 shutdown_connections 2023-11-15T02:57:06.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f358c8da700 1 --2- 172.21.15.121:0/203726690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35880a5720 0x7f35880c2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f358c8da700 1 --2- 172.21.15.121:0/203726690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3588151af0 0x7f35880a7070 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f358c8da700 1 -- 172.21.15.121:0/203726690 >> 172.21.15.121:0/203726690 conn(0x7f3588067a60 msgr2=0x7f3588067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f358c8da700 1 -- 172.21.15.121:0/203726690 shutdown_connections 2023-11-15T02:57:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.067+0000 7f358c8da700 1 -- 172.21.15.121:0/203726690 wait complete. 2023-11-15T02:57:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.071+0000 7f358c8da700 1 Processor -- start 2023-11-15T02:57:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.071+0000 7f358c8da700 1 -- start start 2023-11-15T02:57:06.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.071+0000 7f358c8da700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35880a5720 0x7f3588153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:06.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.071+0000 7f358c8da700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35880c3430 0x7f35881555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:06.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.071+0000 7f358c8da700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3588153b40 0x7f35880c2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:06.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.071+0000 7f358c8da700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3588152f40 con 0x7f35880a5720 2023-11-15T02:57:06.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.071+0000 7f358c8da700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35880c62e0 con 0x7f35880c3430 2023-11-15T02:57:06.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.071+0000 7f358c8da700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35880c65c0 con 0x7f3588153b40 2023-11-15T02:57:06.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.071+0000 7f3586d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35880a5720 0x7f3588153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:06.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.071+0000 7f3586d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35880a5720 0x7f3588153650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49694/0 (socket says 172.21.15.121:49694) 2023-11-15T02:57:06.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.071+0000 7f3586d9d700 1 -- 172.21.15.121:0/889720661 learned_addr learned my addr 172.21.15.121:0/889720661 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:06.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.071+0000 7f358759e700 1 --2- 172.21.15.121:0/889720661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3588153b40 0x7f35880c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:06.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.071+0000 7f358659c700 1 --2- 172.21.15.121:0/889720661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35880c3430 0x7f35881555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:06.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.075+0000 7f3586d9d700 1 -- 172.21.15.121:0/889720661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3588153b40 msgr2=0x7f35880c2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:06.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.075+0000 7f3586d9d700 1 --2- 172.21.15.121:0/889720661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3588153b40 0x7f35880c2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.075+0000 7f3586d9d700 1 -- 172.21.15.121:0/889720661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35880c3430 msgr2=0x7f35881555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:06.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.075+0000 7f3586d9d700 1 --2- 172.21.15.121:0/889720661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35880c3430 0x7f35881555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.075+0000 7f3586d9d700 1 -- 172.21.15.121:0/889720661 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35780092a0 con 0x7f35880a5720 2023-11-15T02:57:06.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.075+0000 7f358659c700 1 --2- 172.21.15.121:0/889720661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35880c3430 0x7f35881555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:06.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.075+0000 7f358759e700 1 --2- 172.21.15.121:0/889720661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3588153b40 0x7f35880c2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:06.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.075+0000 7f3586d9d700 1 --2- 172.21.15.121:0/889720661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35880a5720 0x7f3588153650 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f3578009d50 tx=0x7f3578016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:06.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.075+0000 7f356effd700 1 -- 172.21.15.121:0/889720661 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3578000c90 con 0x7f35880a5720 2023-11-15T02:57:06.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.075+0000 7f356effd700 1 -- 172.21.15.121:0/889720661 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357800d770 con 0x7f35880a5720 2023-11-15T02:57:06.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.075+0000 7f356effd700 1 -- 172.21.15.121:0/889720661 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357801d640 con 0x7f35880a5720 2023-11-15T02:57:06.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.075+0000 7f358c8da700 1 -- 172.21.15.121:0/889720661 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35880c68a0 con 0x7f35880a5720 2023-11-15T02:57:06.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.075+0000 7f358c8da700 1 -- 172.21.15.121:0/889720661 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35880c6e60 con 0x7f35880a5720 2023-11-15T02:57:06.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.075+0000 7f358c8da700 1 -- 172.21.15.121:0/889720661 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3588063070 con 0x7f35880a5720 2023-11-15T02:57:06.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.075+0000 7f356effd700 1 -- 172.21.15.121:0/889720661 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f357801da70 con 0x7f35880a5720 2023-11-15T02:57:06.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.079+0000 7f356effd700 1 --2- 172.21.15.121:0/889720661 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f357405a4e0 0x7f357405c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:06.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.079+0000 7f356effd700 1 -- 172.21.15.121:0/889720661 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3578076020 con 0x7f35880a5720 2023-11-15T02:57:06.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.079+0000 7f358659c700 1 --2- 172.21.15.121:0/889720661 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f357405a4e0 0x7f357405c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:06.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.079+0000 7f358659c700 1 --2- 172.21.15.121:0/889720661 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f357405a4e0 0x7f357405c9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f35880c72e0 tx=0x7f357c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:06.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.083+0000 7f356effd700 1 -- 172.21.15.121:0/889720661 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3588063070 con 0x7f35880a5720 2023-11-15T02:57:06.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.251+0000 7f358c8da700 1 -- 172.21.15.121:0/889720661 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3588063070 con 0x7f35880a5720 2023-11-15T02:57:06.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.251+0000 7f356effd700 1 -- 172.21.15.121:0/889720661 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3588063070 con 0x7f35880a5720 2023-11-15T02:57:06.256 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T02:57:06.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.255+0000 7f358c8da700 1 -- 172.21.15.121:0/889720661 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f357405a4e0 msgr2=0x7f357405c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:06.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.255+0000 7f358c8da700 1 --2- 172.21.15.121:0/889720661 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f357405a4e0 0x7f357405c9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f35880c72e0 tx=0x7f357c008040).stop 2023-11-15T02:57:06.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.255+0000 7f358c8da700 1 -- 172.21.15.121:0/889720661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35880a5720 msgr2=0x7f3588153650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:06.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.255+0000 7f358c8da700 1 --2- 172.21.15.121:0/889720661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35880a5720 0x7f3588153650 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f3578009d50 tx=0x7f3578016000).stop 2023-11-15T02:57:06.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.255+0000 7f358c8da700 1 -- 172.21.15.121:0/889720661 shutdown_connections 2023-11-15T02:57:06.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.255+0000 7f358c8da700 1 --2- 172.21.15.121:0/889720661 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f357405a4e0 0x7f357405c9a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.255+0000 7f358c8da700 1 --2- 172.21.15.121:0/889720661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3588153b40 0x7f35880c2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.255+0000 7f358c8da700 1 --2- 172.21.15.121:0/889720661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35880c3430 0x7f35881555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.255+0000 7f358c8da700 1 --2- 172.21.15.121:0/889720661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35880a5720 0x7f3588153650 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.255+0000 7f358c8da700 1 -- 172.21.15.121:0/889720661 >> 172.21.15.121:0/889720661 conn(0x7f3588067a60 msgr2=0x7f3588062190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:06.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.255+0000 7f358c8da700 1 -- 172.21.15.121:0/889720661 shutdown_connections 2023-11-15T02:57:06.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.255+0000 7f358c8da700 1 -- 172.21.15.121:0/889720661 wait complete. 2023-11-15T02:57:06.279 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-15T02:57:06.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.439+0000 7f48e19db700 1 Processor -- start 2023-11-15T02:57:06.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.439+0000 7f48e19db700 1 -- start start 2023-11-15T02:57:06.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.439+0000 7f48e19db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48dc157ed0 0x7f48dc1582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:06.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.439+0000 7f48e19db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f48dc067020 0x7f48dc0bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:06.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.439+0000 7f48e19db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48dc062da0 con 0x7f48dc067a60 2023-11-15T02:57:06.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.439+0000 7f48e19db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48dc0a93a0 con 0x7f48dc067020 2023-11-15T02:57:06.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.439+0000 7f48e19db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc062190 con 0x7f48dc157ed0 2023-11-15T02:57:06.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48db7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f48dc067020 0x7f48dc0bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:06.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48db7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f48dc067020 0x7f48dc0bc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49710/0 (socket says 172.21.15.121:49710) 2023-11-15T02:57:06.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48db7fe700 1 -- 172.21.15.121:0/258769978 learned_addr learned my addr 172.21.15.121:0/258769978 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:06.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48daffd700 1 --2- 172.21.15.121:0/258769978 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48dc157ed0 0x7f48dc1582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:06.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48db7fe700 1 -- 172.21.15.121:0/258769978 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48dc157ed0 msgr2=0x7f48dc1582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:06.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48db7fe700 1 --2- 172.21.15.121:0/258769978 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48dc157ed0 0x7f48dc1582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48db7fe700 1 -- 172.21.15.121:0/258769978 >> v1:172.21.15.121:6789/0 conn(0x7f48dc067a60 legacy=0x7f48dc067e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:06.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48db7fe700 1 -- 172.21.15.121:0/258769978 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc062470 con 0x7f48dc067020 2023-11-15T02:57:06.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48db7fe700 1 --2- 172.21.15.121:0/258769978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48dc067020 0x7f48dc0bc0d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f48d4009f50 tx=0x7f48d4009840).ready entity=mon.0 client_cookie=d844d01dba66f100 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:06.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48d9ffb700 1 -- 172.21.15.121:0/258769978 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d400e070 con 0x7f48dc067020 2023-11-15T02:57:06.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48d9ffb700 1 -- 172.21.15.121:0/258769978 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d400ba70 con 0x7f48dc067020 2023-11-15T02:57:06.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48d9ffb700 1 -- 172.21.15.121:0/258769978 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d400bdf0 con 0x7f48dc067020 2023-11-15T02:57:06.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48e19db700 1 -- 172.21.15.121:0/258769978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48dc067020 msgr2=0x7f48dc0bc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:06.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48e19db700 1 --2- 172.21.15.121:0/258769978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48dc067020 0x7f48dc0bc0d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f48d4009f50 tx=0x7f48d4009840).stop 2023-11-15T02:57:06.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48e19db700 1 -- 172.21.15.121:0/258769978 shutdown_connections 2023-11-15T02:57:06.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48e19db700 1 --2- 172.21.15.121:0/258769978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48dc067020 0x7f48dc0bc0d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48e19db700 1 --2- 172.21.15.121:0/258769978 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48dc157ed0 0x7f48dc1582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48e19db700 1 -- 172.21.15.121:0/258769978 >> 172.21.15.121:0/258769978 conn(0x7f48dc0591c0 msgr2=0x7f48dc0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:06.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48e19db700 1 -- 172.21.15.121:0/258769978 shutdown_connections 2023-11-15T02:57:06.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.443+0000 7f48e19db700 1 -- 172.21.15.121:0/258769978 wait complete. 2023-11-15T02:57:06.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48e19db700 1 Processor -- start 2023-11-15T02:57:06.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48e19db700 1 -- start start 2023-11-15T02:57:06.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48e19db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48dc067020 0x7f48dc0bea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:06.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48e19db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48dc067a60 0x7f48dc153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:06.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48e19db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48dc157ed0 0x7f48dc159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:06.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48e19db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc0bef10 con 0x7f48dc067020 2023-11-15T02:57:06.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48e19db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc1584e0 con 0x7f48dc067a60 2023-11-15T02:57:06.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48e19db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc0abc20 con 0x7f48dc157ed0 2023-11-15T02:57:06.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48daffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48dc067020 0x7f48dc0bea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:06.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48daffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48dc067020 0x7f48dc0bea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49712/0 (socket says 172.21.15.121:49712) 2023-11-15T02:57:06.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48daffd700 1 -- 172.21.15.121:0/2865335239 learned_addr learned my addr 172.21.15.121:0/2865335239 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:06.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48db7fe700 1 --2- 172.21.15.121:0/2865335239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48dc157ed0 0x7f48dc159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:06.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48daffd700 1 -- 172.21.15.121:0/2865335239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48dc157ed0 msgr2=0x7f48dc159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:06.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48da7fc700 1 --2- 172.21.15.121:0/2865335239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48dc067a60 0x7f48dc153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:06.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48daffd700 1 --2- 172.21.15.121:0/2865335239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48dc157ed0 0x7f48dc159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48daffd700 1 -- 172.21.15.121:0/2865335239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48dc067a60 msgr2=0x7f48dc153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:06.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48daffd700 1 --2- 172.21.15.121:0/2865335239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48dc067a60 0x7f48dc153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48daffd700 1 -- 172.21.15.121:0/2865335239 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d4009ae0 con 0x7f48dc067020 2023-11-15T02:57:06.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48da7fc700 1 --2- 172.21.15.121:0/2865335239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48dc067a60 0x7f48dc153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:57:06.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48daffd700 1 --2- 172.21.15.121:0/2865335239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48dc067020 0x7f48dc0bea20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f48cc00d8c0 tx=0x7f48cc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:06.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48c2ffd700 1 -- 172.21.15.121:0/2865335239 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc0095b0 con 0x7f48dc067020 2023-11-15T02:57:06.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48c2ffd700 1 -- 172.21.15.121:0/2865335239 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48cc009710 con 0x7f48dc067020 2023-11-15T02:57:06.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48c2ffd700 1 -- 172.21.15.121:0/2865335239 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc009a90 con 0x7f48dc067020 2023-11-15T02:57:06.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.447+0000 7f48e19db700 1 -- 172.21.15.121:0/2865335239 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48dc1542f0 con 0x7f48dc067020 2023-11-15T02:57:06.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.451+0000 7f48e19db700 1 -- 172.21.15.121:0/2865335239 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48dc15aa40 con 0x7f48dc067020 2023-11-15T02:57:06.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.451+0000 7f48e19db700 1 -- 172.21.15.121:0/2865335239 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48dc1582b0 con 0x7f48dc067020 2023-11-15T02:57:06.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.451+0000 7f48c2ffd700 1 -- 172.21.15.121:0/2865335239 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f48cc013480 con 0x7f48dc067020 2023-11-15T02:57:06.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.455+0000 7f48c2ffd700 1 --2- 172.21.15.121:0/2865335239 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f48c805a4e0 0x7f48c805c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:06.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.455+0000 7f48c2ffd700 1 -- 172.21.15.121:0/2865335239 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f48cc0744e0 con 0x7f48dc067020 2023-11-15T02:57:06.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.455+0000 7f48da7fc700 1 --2- 172.21.15.121:0/2865335239 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f48c805a4e0 0x7f48c805c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:06.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.455+0000 7f48da7fc700 1 --2- 172.21.15.121:0/2865335239 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f48c805a4e0 0x7f48c805c9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f48dc0a8c10 tx=0x7f48d0009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:06.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.459+0000 7f48c2ffd700 1 -- 172.21.15.121:0/2865335239 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f48dc1582b0 con 0x7f48dc067020 2023-11-15T02:57:06.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.623+0000 7f48e19db700 1 -- 172.21.15.121:0/2865335239 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f48dc1582b0 con 0x7f48dc067020 2023-11-15T02:57:06.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.623+0000 7f48c2ffd700 1 -- 172.21.15.121:0/2865335239 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f48dc1582b0 con 0x7f48dc067020 2023-11-15T02:57:06.628 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T02:57:06.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.627+0000 7f48e19db700 1 -- 172.21.15.121:0/2865335239 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f48c805a4e0 msgr2=0x7f48c805c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:06.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.627+0000 7f48e19db700 1 --2- 172.21.15.121:0/2865335239 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f48c805a4e0 0x7f48c805c9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f48dc0a8c10 tx=0x7f48d0009040).stop 2023-11-15T02:57:06.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.627+0000 7f48e19db700 1 -- 172.21.15.121:0/2865335239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48dc067020 msgr2=0x7f48dc0bea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:06.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.627+0000 7f48e19db700 1 --2- 172.21.15.121:0/2865335239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48dc067020 0x7f48dc0bea20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f48cc00d8c0 tx=0x7f48cc008040).stop 2023-11-15T02:57:06.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.627+0000 7f48e19db700 1 -- 172.21.15.121:0/2865335239 shutdown_connections 2023-11-15T02:57:06.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.627+0000 7f48e19db700 1 --2- 172.21.15.121:0/2865335239 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f48c805a4e0 0x7f48c805c9a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.627+0000 7f48e19db700 1 --2- 172.21.15.121:0/2865335239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48dc157ed0 0x7f48dc159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.627+0000 7f48e19db700 1 --2- 172.21.15.121:0/2865335239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48dc067a60 0x7f48dc153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.627+0000 7f48e19db700 1 --2- 172.21.15.121:0/2865335239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48dc067020 0x7f48dc0bea20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.627+0000 7f48e19db700 1 -- 172.21.15.121:0/2865335239 >> 172.21.15.121:0/2865335239 conn(0x7f48dc0591c0 msgr2=0x7f48dc155b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:06.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.627+0000 7f48e19db700 1 -- 172.21.15.121:0/2865335239 shutdown_connections 2023-11-15T02:57:06.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.627+0000 7f48e19db700 1 -- 172.21.15.121:0/2865335239 wait complete. 2023-11-15T02:57:06.656 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-15T02:57:06.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.807+0000 7f1b2128f700 1 Processor -- start 2023-11-15T02:57:06.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.807+0000 7f1b2128f700 1 -- start start 2023-11-15T02:57:06.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.811+0000 7f1b2128f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1b1c0c5c70 0x7f1b1c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:06.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.811+0000 7f1b2128f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1b1c066330 0x7f1b1c0ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:06.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.811+0000 7f1b2128f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b1c065ea0 con 0x7f1b1c0acd00 2023-11-15T02:57:06.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.811+0000 7f1b2128f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b1c066bb0 con 0x7f1b1c0c5c70 2023-11-15T02:57:06.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.811+0000 7f1b2128f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b1c065890 con 0x7f1b1c066330 2023-11-15T02:57:06.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.811+0000 7f1b1b7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1b1c0c5c70 0x7f1b1c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:06.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.811+0000 7f1b1b7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1b1c0c5c70 0x7f1b1c153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49716/0 (socket says 172.21.15.121:49716) 2023-11-15T02:57:06.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.811+0000 7f1b1b7fe700 1 -- 172.21.15.121:0/3364405665 learned_addr learned my addr 172.21.15.121:0/3364405665 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:06.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.811+0000 7f1b1bfff700 1 --1- 172.21.15.121:0/3364405665 >> v1:172.21.15.121:6789/0 conn(0x7f1b1c0acd00 0x7f1b1c1536c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59662/0 (socket says 172.21.15.121:59662) 2023-11-15T02:57:06.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.811+0000 7f1b20a8e700 1 --2- 172.21.15.121:0/3364405665 >> v2:172.21.15.190:3300/0 conn(0x7f1b1c066330 0x7f1b1c0ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:06.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.811+0000 7f1b1b7fe700 1 -- 172.21.15.121:0/3364405665 >> v2:172.21.15.190:3300/0 conn(0x7f1b1c066330 msgr2=0x7f1b1c0ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:06.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.811+0000 7f1b1b7fe700 1 --2- 172.21.15.121:0/3364405665 >> v2:172.21.15.190:3300/0 conn(0x7f1b1c066330 0x7f1b1c0ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.811+0000 7f1b1b7fe700 1 -- 172.21.15.121:0/3364405665 >> v1:172.21.15.121:6789/0 conn(0x7f1b1c0acd00 legacy=0x7f1b1c1536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:06.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.811+0000 7f1b1b7fe700 1 -- 172.21.15.121:0/3364405665 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c0c64a0 con 0x7f1b1c0c5c70 2023-11-15T02:57:06.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.811+0000 7f1b1affd700 1 -- 172.21.15.121:0/3364405665 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38210332 0 0) 0x7f1b1c065ea0 con 0x7f1b1c0acd00 2023-11-15T02:57:06.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.811+0000 7f1b1b7fe700 1 --2- 172.21.15.121:0/3364405665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1c0c5c70 0x7f1b1c153dd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f1b1c0c4f20 tx=0x7f1b100090f0).ready entity=mon.0 client_cookie=604c449ddd2a190c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:06.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.811+0000 7f1b1affd700 1 -- 172.21.15.121:0/3364405665 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1000b520 con 0x7f1b1c0c5c70 2023-11-15T02:57:06.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.811+0000 7f1b1affd700 1 -- 172.21.15.121:0/3364405665 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1000b680 con 0x7f1b1c0c5c70 2023-11-15T02:57:06.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.811+0000 7f1b1affd700 1 -- 172.21.15.121:0/3364405665 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1000ba00 con 0x7f1b1c0c5c70 2023-11-15T02:57:06.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.815+0000 7f1b2128f700 1 -- 172.21.15.121:0/3364405665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1c0c5c70 msgr2=0x7f1b1c153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:06.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.815+0000 7f1b2128f700 1 --2- 172.21.15.121:0/3364405665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1c0c5c70 0x7f1b1c153dd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f1b1c0c4f20 tx=0x7f1b100090f0).stop 2023-11-15T02:57:06.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.815+0000 7f1b2128f700 1 -- 172.21.15.121:0/3364405665 shutdown_connections 2023-11-15T02:57:06.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.815+0000 7f1b2128f700 1 --2- 172.21.15.121:0/3364405665 >> v2:172.21.15.190:3300/0 conn(0x7f1b1c066330 0x7f1b1c0ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.815+0000 7f1b2128f700 1 --2- 172.21.15.121:0/3364405665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1c0c5c70 0x7f1b1c153dd0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.815+0000 7f1b2128f700 1 -- 172.21.15.121:0/3364405665 >> 172.21.15.121:0/3364405665 conn(0x7f1b1c063fc0 msgr2=0x7f1b1c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:06.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.815+0000 7f1b2128f700 1 -- 172.21.15.121:0/3364405665 shutdown_connections 2023-11-15T02:57:06.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.815+0000 7f1b2128f700 1 -- 172.21.15.121:0/3364405665 wait complete. 2023-11-15T02:57:06.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.815+0000 7f1b2128f700 1 Processor -- start 2023-11-15T02:57:06.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.815+0000 7f1b2128f700 1 -- start start 2023-11-15T02:57:06.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b2128f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b1c066330 0x7f1b1c139c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:06.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b2128f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1c0acd00 0x7f1b1c0c4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:06.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b2128f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b1c0c5c70 0x7f1b1c0c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:06.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b2128f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c155640 con 0x7f1b1c0acd00 2023-11-15T02:57:06.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b2128f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c13a160 con 0x7f1b1c066330 2023-11-15T02:57:06.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b2128f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c154a30 con 0x7f1b1c0c5c70 2023-11-15T02:57:06.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b1b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1c0acd00 0x7f1b1c0c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:06.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b1b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1c0acd00 0x7f1b1c0c4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49718/0 (socket says 172.21.15.121:49718) 2023-11-15T02:57:06.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b1b7fe700 1 -- 172.21.15.121:0/446424852 learned_addr learned my addr 172.21.15.121:0/446424852 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:06.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b1bfff700 1 --2- 172.21.15.121:0/446424852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b1c066330 0x7f1b1c139c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:06.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b20a8e700 1 --2- 172.21.15.121:0/446424852 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b1c0c5c70 0x7f1b1c0c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:06.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b1b7fe700 1 -- 172.21.15.121:0/446424852 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b1c0c5c70 msgr2=0x7f1b1c0c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:06.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b1b7fe700 1 --2- 172.21.15.121:0/446424852 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b1c0c5c70 0x7f1b1c0c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b1b7fe700 1 -- 172.21.15.121:0/446424852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b1c066330 msgr2=0x7f1b1c139c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:06.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b1b7fe700 1 --2- 172.21.15.121:0/446424852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b1c066330 0x7f1b1c139c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b1b7fe700 1 -- 172.21.15.121:0/446424852 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b10009370 con 0x7f1b1c0acd00 2023-11-15T02:57:06.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b1b7fe700 1 --2- 172.21.15.121:0/446424852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1c0acd00 0x7f1b1c0c4170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f1b1c064f20 tx=0x7f1b10017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:06.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b03fff700 1 -- 172.21.15.121:0/446424852 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1000be90 con 0x7f1b1c0acd00 2023-11-15T02:57:06.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b03fff700 1 -- 172.21.15.121:0/446424852 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1000b7f0 con 0x7f1b1c0acd00 2023-11-15T02:57:06.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b2128f700 1 -- 172.21.15.121:0/446424852 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b1c154d10 con 0x7f1b1c0acd00 2023-11-15T02:57:06.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b03fff700 1 -- 172.21.15.121:0/446424852 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1001c670 con 0x7f1b1c0acd00 2023-11-15T02:57:06.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.819+0000 7f1b2128f700 1 -- 172.21.15.121:0/446424852 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1c1552d0 con 0x7f1b1c0acd00 2023-11-15T02:57:06.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.823+0000 7f1b2128f700 1 -- 172.21.15.121:0/446424852 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1c1557b0 con 0x7f1b1c0acd00 2023-11-15T02:57:06.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.823+0000 7f1b03fff700 1 -- 172.21.15.121:0/446424852 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1b1001c9e0 con 0x7f1b1c0acd00 2023-11-15T02:57:06.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.823+0000 7f1b03fff700 1 --2- 172.21.15.121:0/446424852 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1b0805a540 0x7f1b0805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:06.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.823+0000 7f1b03fff700 1 -- 172.21.15.121:0/446424852 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1b100765d0 con 0x7f1b1c0acd00 2023-11-15T02:57:06.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.823+0000 7f1b1bfff700 1 --2- 172.21.15.121:0/446424852 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1b0805a540 0x7f1b0805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:06.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.827+0000 7f1b1bfff700 1 --2- 172.21.15.121:0/446424852 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1b0805a540 0x7f1b0805ca00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f1b0c0012a0 tx=0x7f1b0c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:06.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.827+0000 7f1b03fff700 1 -- 172.21.15.121:0/446424852 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1b10049cf0 con 0x7f1b1c0acd00 2023-11-15T02:57:06.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.987+0000 7f1b2128f700 1 -- 172.21.15.121:0/446424852 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b1c06bcf0 con 0x7f1b1c0acd00 2023-11-15T02:57:06.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.991+0000 7f1b03fff700 1 -- 172.21.15.121:0/446424852 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b10020020 con 0x7f1b1c0acd00 2023-11-15T02:57:06.995 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T02:57:06.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.991+0000 7f1b2128f700 1 -- 172.21.15.121:0/446424852 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1b0805a540 msgr2=0x7f1b0805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:06.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.991+0000 7f1b2128f700 1 --2- 172.21.15.121:0/446424852 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1b0805a540 0x7f1b0805ca00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f1b0c0012a0 tx=0x7f1b0c009040).stop 2023-11-15T02:57:06.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.991+0000 7f1b2128f700 1 -- 172.21.15.121:0/446424852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1c0acd00 msgr2=0x7f1b1c0c4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:06.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.991+0000 7f1b2128f700 1 --2- 172.21.15.121:0/446424852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1c0acd00 0x7f1b1c0c4170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f1b1c064f20 tx=0x7f1b10017000).stop 2023-11-15T02:57:06.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.991+0000 7f1b2128f700 1 -- 172.21.15.121:0/446424852 shutdown_connections 2023-11-15T02:57:06.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.991+0000 7f1b2128f700 1 --2- 172.21.15.121:0/446424852 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1b0805a540 0x7f1b0805ca00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.991+0000 7f1b2128f700 1 --2- 172.21.15.121:0/446424852 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b1c0c5c70 0x7f1b1c0c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.991+0000 7f1b2128f700 1 --2- 172.21.15.121:0/446424852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1c0acd00 0x7f1b1c0c4170 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.991+0000 7f1b2128f700 1 --2- 172.21.15.121:0/446424852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b1c066330 0x7f1b1c139c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:06.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.991+0000 7f1b2128f700 1 -- 172.21.15.121:0/446424852 >> 172.21.15.121:0/446424852 conn(0x7f1b1c063fc0 msgr2=0x7f1b1c14e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:07.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.991+0000 7f1b2128f700 1 -- 172.21.15.121:0/446424852 shutdown_connections 2023-11-15T02:57:07.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:06.991+0000 7f1b2128f700 1 -- 172.21.15.121:0/446424852 wait complete. 2023-11-15T02:57:07.017 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:57:07.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.159+0000 7f33a62db700 1 Processor -- start 2023-11-15T02:57:07.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.163+0000 7f33a62db700 1 -- start start 2023-11-15T02:57:07.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.163+0000 7f33a62db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a01562f0 0x7f33a0155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:07.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.163+0000 7f33a62db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f33a00a57c0 0x7f33a00a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:07.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.163+0000 7f33a62db700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a00c4c30 con 0x7f33a0154ca0 2023-11-15T02:57:07.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.163+0000 7f33a62db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a00a8280 con 0x7f33a00a57c0 2023-11-15T02:57:07.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.163+0000 7f33a62db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a00a6d40 con 0x7f33a01562f0 2023-11-15T02:57:07.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.163+0000 7f33977fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a01562f0 0x7f33a0155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:07.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.163+0000 7f339ffff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f33a0154ca0 0x7f33a00a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:58096/0 (socket says 172.21.15.121:58096) 2023-11-15T02:57:07.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.163+0000 7f339ffff700 1 -- 172.21.15.121:0/3575916996 learned_addr learned my addr 172.21.15.121:0/3575916996 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:07.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.163+0000 7f33a4878700 1 --2- 172.21.15.121:0/3575916996 >> v2:172.21.15.121:3300/0 conn(0x7f33a00a57c0 0x7f33a00a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:07.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.163+0000 7f33a4878700 1 -- 172.21.15.121:0/3575916996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a01562f0 msgr2=0x7f33a0155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:07.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.163+0000 7f33a4878700 1 --2- 172.21.15.121:0/3575916996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a01562f0 0x7f33a0155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:07.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.163+0000 7f33a4878700 1 -- 172.21.15.121:0/3575916996 >> v1:172.21.15.190:6789/0 conn(0x7f33a0154ca0 legacy=0x7f33a00a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:07.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.163+0000 7f33a4878700 1 -- 172.21.15.121:0/3575916996 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a00a6140 con 0x7f33a00a57c0 2023-11-15T02:57:07.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.163+0000 7f33a4878700 1 --2- 172.21.15.121:0/3575916996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33a00a57c0 0x7f33a00a5c50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f3398009e60 tx=0x7f3398009780).ready entity=mon.0 client_cookie=ab85ccef8c8e88dc server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:07.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.167+0000 7f339f7fe700 1 -- 172.21.15.121:0/3575916996 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339800e070 con 0x7f33a00a57c0 2023-11-15T02:57:07.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.167+0000 7f339f7fe700 1 -- 172.21.15.121:0/3575916996 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339800bb60 con 0x7f33a00a57c0 2023-11-15T02:57:07.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.167+0000 7f339f7fe700 1 -- 172.21.15.121:0/3575916996 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398012410 con 0x7f33a00a57c0 2023-11-15T02:57:07.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.167+0000 7f33a62db700 1 -- 172.21.15.121:0/3575916996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33a00a57c0 msgr2=0x7f33a00a5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:07.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.167+0000 7f33a62db700 1 --2- 172.21.15.121:0/3575916996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33a00a57c0 0x7f33a00a5c50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f3398009e60 tx=0x7f3398009780).stop 2023-11-15T02:57:07.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.167+0000 7f33a62db700 1 -- 172.21.15.121:0/3575916996 shutdown_connections 2023-11-15T02:57:07.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.167+0000 7f33a62db700 1 --2- 172.21.15.121:0/3575916996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33a00a57c0 0x7f33a00a5c50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:07.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.167+0000 7f33a62db700 1 --2- 172.21.15.121:0/3575916996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a01562f0 0x7f33a0155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:07.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.167+0000 7f33a62db700 1 -- 172.21.15.121:0/3575916996 >> 172.21.15.121:0/3575916996 conn(0x7f33a0062190 msgr2=0x7f33a00625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:07.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.167+0000 7f33a62db700 1 -- 172.21.15.121:0/3575916996 shutdown_connections 2023-11-15T02:57:07.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.167+0000 7f33a62db700 1 -- 172.21.15.121:0/3575916996 wait complete. 2023-11-15T02:57:07.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.167+0000 7f33a62db700 1 Processor -- start 2023-11-15T02:57:07.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.167+0000 7f33a62db700 1 -- start start 2023-11-15T02:57:07.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f33a62db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33a00a57c0 0x7f33a00aa640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:07.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f33a62db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33a0154ca0 0x7f33a01568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:07.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f33a62db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a01562f0 0x7f33a014c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:07.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f33a62db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a0158da0 con 0x7f33a00a57c0 2023-11-15T02:57:07.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f33a62db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a01598c0 con 0x7f33a0154ca0 2023-11-15T02:57:07.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f33a62db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a00aacb0 con 0x7f33a01562f0 2023-11-15T02:57:07.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f339ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33a00a57c0 0x7f33a00aa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:07.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f339ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33a00a57c0 0x7f33a00aa640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49746/0 (socket says 172.21.15.121:49746) 2023-11-15T02:57:07.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f339ffff700 1 -- 172.21.15.121:0/1322931723 learned_addr learned my addr 172.21.15.121:0/1322931723 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:07.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f33a4878700 1 --2- 172.21.15.121:0/1322931723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a01562f0 0x7f33a014c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:07.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f33977fe700 1 --2- 172.21.15.121:0/1322931723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33a0154ca0 0x7f33a01568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:07.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f339ffff700 1 -- 172.21.15.121:0/1322931723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a01562f0 msgr2=0x7f33a014c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:07.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f339ffff700 1 --2- 172.21.15.121:0/1322931723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a01562f0 0x7f33a014c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:07.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f339ffff700 1 -- 172.21.15.121:0/1322931723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33a0154ca0 msgr2=0x7f33a01568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:07.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f339ffff700 1 --2- 172.21.15.121:0/1322931723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33a0154ca0 0x7f33a01568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:07.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f339ffff700 1 -- 172.21.15.121:0/1322931723 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3398009a20 con 0x7f33a00a57c0 2023-11-15T02:57:07.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f33a4878700 1 --2- 172.21.15.121:0/1322931723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a01562f0 0x7f33a014c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:07.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f33977fe700 1 --2- 172.21.15.121:0/1322931723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33a0154ca0 0x7f33a01568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:07.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f339ffff700 1 --2- 172.21.15.121:0/1322931723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33a00a57c0 0x7f33a00aa640 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f3390007fd0 tx=0x7f3390007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:07.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f339cff9700 1 -- 172.21.15.121:0/1322931723 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33900072e0 con 0x7f33a00a57c0 2023-11-15T02:57:07.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f339cff9700 1 -- 172.21.15.121:0/1322931723 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3390007440 con 0x7f33a00a57c0 2023-11-15T02:57:07.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f33a62db700 1 -- 172.21.15.121:0/1322931723 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a0158fd0 con 0x7f33a00a57c0 2023-11-15T02:57:07.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f339cff9700 1 -- 172.21.15.121:0/1322931723 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33900057a0 con 0x7f33a00a57c0 2023-11-15T02:57:07.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.171+0000 7f33a62db700 1 -- 172.21.15.121:0/1322931723 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a0159270 con 0x7f33a00a57c0 2023-11-15T02:57:07.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.175+0000 7f33a62db700 1 -- 172.21.15.121:0/1322931723 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a00ac300 con 0x7f33a00a57c0 2023-11-15T02:57:07.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.175+0000 7f339cff9700 1 -- 172.21.15.121:0/1322931723 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3390005b90 con 0x7f33a00a57c0 2023-11-15T02:57:07.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.175+0000 7f339cff9700 1 --2- 172.21.15.121:0/1322931723 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f338005a4e0 0x7f338005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:07.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.175+0000 7f339cff9700 1 -- 172.21.15.121:0/1322931723 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3390073fe0 con 0x7f33a00a57c0 2023-11-15T02:57:07.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.179+0000 7f33977fe700 1 --2- 172.21.15.121:0/1322931723 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f338005a4e0 0x7f338005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:07.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.179+0000 7f33977fe700 1 --2- 172.21.15.121:0/1322931723 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f338005a4e0 0x7f338005c9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f33a01678e0 tx=0x7f3388008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:07.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.183+0000 7f339cff9700 1 -- 172.21.15.121:0/1322931723 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3390047780 con 0x7f33a00a57c0 2023-11-15T02:57:07.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.395+0000 7f33a62db700 1 -- 172.21.15.121:0/1322931723 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33a006c150 con 0x7f33a00a57c0 2023-11-15T02:57:07.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.395+0000 7f339cff9700 1 -- 172.21.15.121:0/1322931723 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4489 (secure 0 0 0) 0x7f339004ada0 con 0x7f33a00a57c0 2023-11-15T02:57:07.400 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:07.401 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/802010010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":802010010},{"type":"v1","addr":"172.21.15.190:6835","nonce":802010010}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3991611070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3991611070},{"type":"v1","addr":"172.21.15.121:6836","nonce":3991611070}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3514688785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3514688785},{"type":"v1","addr":"172.21.15.121:6837","nonce":3514688785}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:56:51.695751+0000","modified":"2023-11-15T02:56:54.838464+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6833/742721767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":742721767},{"type":"v1","addr":"172.21.15.190:6833","nonce":742721767}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-15T02:57:07.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.395+0000 7f33a62db700 1 -- 172.21.15.121:0/1322931723 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f338005a4e0 msgr2=0x7f338005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:07.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.395+0000 7f33a62db700 1 --2- 172.21.15.121:0/1322931723 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f338005a4e0 0x7f338005c9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f33a01678e0 tx=0x7f3388008040).stop 2023-11-15T02:57:07.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.395+0000 7f33a62db700 1 -- 172.21.15.121:0/1322931723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33a00a57c0 msgr2=0x7f33a00aa640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:07.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.395+0000 7f33a62db700 1 --2- 172.21.15.121:0/1322931723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33a00a57c0 0x7f33a00aa640 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f3390007fd0 tx=0x7f3390007df0).stop 2023-11-15T02:57:07.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.399+0000 7f33a62db700 1 -- 172.21.15.121:0/1322931723 shutdown_connections 2023-11-15T02:57:07.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.399+0000 7f33a62db700 1 --2- 172.21.15.121:0/1322931723 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f338005a4e0 0x7f338005c9a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:07.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.399+0000 7f33a62db700 1 --2- 172.21.15.121:0/1322931723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a01562f0 0x7f33a014c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:07.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.399+0000 7f33a62db700 1 --2- 172.21.15.121:0/1322931723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33a0154ca0 0x7f33a01568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:07.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.399+0000 7f33a62db700 1 --2- 172.21.15.121:0/1322931723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33a00a57c0 0x7f33a00aa640 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:07.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.399+0000 7f33a62db700 1 -- 172.21.15.121:0/1322931723 >> 172.21.15.121:0/1322931723 conn(0x7f33a0062190 msgr2=0x7f33a0066c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:07.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.399+0000 7f33a62db700 1 -- 172.21.15.121:0/1322931723 shutdown_connections 2023-11-15T02:57:07.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.399+0000 7f33a62db700 1 -- 172.21.15.121:0/1322931723 wait complete. 2023-11-15T02:57:07.406 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 8 2023-11-15T02:57:07.424 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:57:07.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.575+0000 7f02a6ae4700 1 Processor -- start 2023-11-15T02:57:07.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.575+0000 7f02a6ae4700 1 -- start start 2023-11-15T02:57:07.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.579+0000 7f02a6ae4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02a0152850 0x7f02a00c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:07.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.579+0000 7f02a6ae4700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a00c1d50 con 0x7f02a015be30 2023-11-15T02:57:07.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.579+0000 7f02a6ae4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a015cb80 con 0x7f02a00c3280 2023-11-15T02:57:07.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.579+0000 7f02a6ae4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a00c1860 con 0x7f02a0152850 2023-11-15T02:57:07.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.579+0000 7f02a62e3700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f02a015be30 0x7f02a015c280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59674/0 (socket says 172.21.15.121:59674) 2023-11-15T02:57:07.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.579+0000 7f02a62e3700 1 -- 172.21.15.121:0/486639147 learned_addr learned my addr 172.21.15.121:0/486639147 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:07.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.579+0000 7f02a52e1700 1 --2- 172.21.15.121:0/486639147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02a0152850 0x7f02a00c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:07.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.579+0000 7f02a4ae0700 1 -- 172.21.15.121:0/486639147 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130130814 0 0) 0x7f02a00c1d50 con 0x7f02a015be30 2023-11-15T02:57:07.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.579+0000 7f02a4ae0700 1 -- 172.21.15.121:0/486639147 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f028c0033f0 con 0x7f02a015be30 2023-11-15T02:57:07.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.579+0000 7f02a4ae0700 1 -- 172.21.15.121:0/486639147 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4156471617 0 0) 0x7f02a015cb80 con 0x7f02a00c3280 2023-11-15T02:57:07.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.579+0000 7f02a4ae0700 1 -- 172.21.15.121:0/486639147 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02a00c1d50 con 0x7f02a00c3280 2023-11-15T02:57:07.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.583+0000 7f02a52e1700 1 -- 172.21.15.121:0/486639147 >> v1:172.21.15.190:6789/0 conn(0x7f02a00c3280 legacy=0x7f02a0152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:07.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.583+0000 7f02a52e1700 1 -- 172.21.15.121:0/486639147 >> v1:172.21.15.121:6789/0 conn(0x7f02a015be30 legacy=0x7f02a015c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:07.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.583+0000 7f02a52e1700 1 -- 172.21.15.121:0/486639147 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a00a9240 con 0x7f02a0152850 2023-11-15T02:57:07.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.583+0000 7f02a52e1700 1 --2- 172.21.15.121:0/486639147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02a0152850 0x7f02a00c3660 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f0290009790 tx=0x7f0290009060).ready entity=mon.2 client_cookie=7811025ae92102b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:07.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.583+0000 7f02a4ae0700 1 -- 172.21.15.121:0/486639147 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029000f510 con 0x7f02a0152850 2023-11-15T02:57:07.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.583+0000 7f02a4ae0700 1 -- 172.21.15.121:0/486639147 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029000f670 con 0x7f02a0152850 2023-11-15T02:57:07.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.583+0000 7f02a4ae0700 1 -- 172.21.15.121:0/486639147 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029000f9f0 con 0x7f02a0152850 2023-11-15T02:57:07.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.583+0000 7f02a6ae4700 1 -- 172.21.15.121:0/486639147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02a0152850 msgr2=0x7f02a00c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:07.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.583+0000 7f02a6ae4700 1 --2- 172.21.15.121:0/486639147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02a0152850 0x7f02a00c3660 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f0290009790 tx=0x7f0290009060).stop 2023-11-15T02:57:07.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.583+0000 7f02a6ae4700 1 -- 172.21.15.121:0/486639147 shutdown_connections 2023-11-15T02:57:07.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.583+0000 7f02a6ae4700 1 --2- 172.21.15.121:0/486639147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02a0152850 0x7f02a00c3660 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:07.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.583+0000 7f02a6ae4700 1 -- 172.21.15.121:0/486639147 >> 172.21.15.121:0/486639147 conn(0x7f02a0062190 msgr2=0x7f02a0062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:07.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.583+0000 7f02a6ae4700 1 -- 172.21.15.121:0/486639147 shutdown_connections 2023-11-15T02:57:07.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.583+0000 7f02a6ae4700 1 -- 172.21.15.121:0/486639147 wait complete. 2023-11-15T02:57:07.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.587+0000 7f02a6ae4700 1 Processor -- start 2023-11-15T02:57:07.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.587+0000 7f02a6ae4700 1 -- start start 2023-11-15T02:57:07.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.587+0000 7f02a6ae4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02a00c3280 0x7f02a00c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:07.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.587+0000 7f02a6ae4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02a0152850 0x7f02a014d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:07.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.587+0000 7f02a6ae4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02a015be30 0x7f02a0157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:07.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.587+0000 7f02a6ae4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a01537c0 con 0x7f02a0152850 2023-11-15T02:57:07.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.587+0000 7f02a6ae4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a00c5860 con 0x7f02a015be30 2023-11-15T02:57:07.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.587+0000 7f02a6ae4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a00c5b40 con 0x7f02a00c3280 2023-11-15T02:57:07.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.587+0000 7f02a52e1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02a0152850 0x7f02a014d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:07.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.587+0000 7f02a52e1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02a0152850 0x7f02a014d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49758/0 (socket says 172.21.15.121:49758) 2023-11-15T02:57:07.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.587+0000 7f02a52e1700 1 -- 172.21.15.121:0/4140389715 learned_addr learned my addr 172.21.15.121:0/4140389715 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:07.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.587+0000 7f02a62e3700 1 --2- 172.21.15.121:0/4140389715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02a015be30 0x7f02a0157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:07.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.587+0000 7f02a5ae2700 1 --2- 172.21.15.121:0/4140389715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02a00c3280 0x7f02a00c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:07.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.591+0000 7f02a52e1700 1 -- 172.21.15.121:0/4140389715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02a00c3280 msgr2=0x7f02a00c4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:07.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.591+0000 7f02a52e1700 1 --2- 172.21.15.121:0/4140389715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02a00c3280 0x7f02a00c4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:07.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.591+0000 7f02a52e1700 1 -- 172.21.15.121:0/4140389715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02a015be30 msgr2=0x7f02a0157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:07.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.591+0000 7f02a52e1700 1 --2- 172.21.15.121:0/4140389715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02a015be30 0x7f02a0157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:07.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.591+0000 7f02a52e1700 1 -- 172.21.15.121:0/4140389715 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02900092a0 con 0x7f02a0152850 2023-11-15T02:57:07.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.591+0000 7f02a62e3700 1 --2- 172.21.15.121:0/4140389715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02a015be30 0x7f02a0157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:07.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.591+0000 7f02a52e1700 1 --2- 172.21.15.121:0/4140389715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02a0152850 0x7f02a014d970 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f02a00654e0 tx=0x7f0290017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:07.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.591+0000 7f0295ffb700 1 -- 172.21.15.121:0/4140389715 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029000f7e0 con 0x7f02a0152850 2023-11-15T02:57:07.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.591+0000 7f0295ffb700 1 -- 172.21.15.121:0/4140389715 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029000f940 con 0x7f02a0152850 2023-11-15T02:57:07.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.591+0000 7f02a6ae4700 1 -- 172.21.15.121:0/4140389715 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a0153a60 con 0x7f02a0152850 2023-11-15T02:57:07.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.591+0000 7f0295ffb700 1 -- 172.21.15.121:0/4140389715 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029001b660 con 0x7f02a0152850 2023-11-15T02:57:07.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.591+0000 7f02a6ae4700 1 -- 172.21.15.121:0/4140389715 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a00c53e0 con 0x7f02a0152850 2023-11-15T02:57:07.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.591+0000 7f02a6ae4700 1 -- 172.21.15.121:0/4140389715 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a0153c20 con 0x7f02a0152850 2023-11-15T02:57:07.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.595+0000 7f0295ffb700 1 -- 172.21.15.121:0/4140389715 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f029001b9d0 con 0x7f02a0152850 2023-11-15T02:57:07.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.595+0000 7f0295ffb700 1 --2- 172.21.15.121:0/4140389715 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f028405a510 0x7f028405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:07.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.595+0000 7f0295ffb700 1 -- 172.21.15.121:0/4140389715 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f02900762f0 con 0x7f02a0152850 2023-11-15T02:57:07.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.595+0000 7f02a5ae2700 1 --2- 172.21.15.121:0/4140389715 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f028405a510 0x7f028405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:07.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.595+0000 7f02a5ae2700 1 --2- 172.21.15.121:0/4140389715 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f028405a510 0x7f028405c9d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f029c003930 tx=0x7f029c007bd0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:07.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.603+0000 7f0295ffb700 1 -- 172.21.15.121:0/4140389715 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0290049a10 con 0x7f02a0152850 2023-11-15T02:57:07.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.767+0000 7f02a6ae4700 1 -- 172.21.15.121:0/4140389715 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02a006bcf0 con 0x7f02a0152850 2023-11-15T02:57:07.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.771+0000 7f0295ffb700 1 -- 172.21.15.121:0/4140389715 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f029004d030 con 0x7f02a0152850 2023-11-15T02:57:07.776 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:07.776 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":21,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:52.702668+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:56:49.726353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-15T02:56:50.688098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:57:07.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.775+0000 7f02a6ae4700 1 -- 172.21.15.121:0/4140389715 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f028405a510 msgr2=0x7f028405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:07.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.775+0000 7f02a6ae4700 1 --2- 172.21.15.121:0/4140389715 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f028405a510 0x7f028405c9d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f029c003930 tx=0x7f029c007bd0).stop 2023-11-15T02:57:07.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.775+0000 7f02a6ae4700 1 -- 172.21.15.121:0/4140389715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02a0152850 msgr2=0x7f02a014d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:07.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.775+0000 7f02a6ae4700 1 --2- 172.21.15.121:0/4140389715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02a0152850 0x7f02a014d970 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f02a00654e0 tx=0x7f0290017000).stop 2023-11-15T02:57:07.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.775+0000 7f02a6ae4700 1 -- 172.21.15.121:0/4140389715 shutdown_connections 2023-11-15T02:57:07.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.775+0000 7f02a6ae4700 1 --2- 172.21.15.121:0/4140389715 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f028405a510 0x7f028405c9d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:07.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.775+0000 7f02a6ae4700 1 --2- 172.21.15.121:0/4140389715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02a015be30 0x7f02a0157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:07.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.775+0000 7f02a6ae4700 1 --2- 172.21.15.121:0/4140389715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02a0152850 0x7f02a014d970 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:07.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.775+0000 7f02a6ae4700 1 --2- 172.21.15.121:0/4140389715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02a00c3280 0x7f02a00c4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:07.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.775+0000 7f02a6ae4700 1 -- 172.21.15.121:0/4140389715 >> 172.21.15.121:0/4140389715 conn(0x7f02a0062190 msgr2=0x7f02a014f170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:07.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.775+0000 7f02a6ae4700 1 -- 172.21.15.121:0/4140389715 shutdown_connections 2023-11-15T02:57:07.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.775+0000 7f02a6ae4700 1 -- 172.21.15.121:0/4140389715 wait complete. 2023-11-15T02:57:07.801 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-15T02:56:49.726353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-15T02:56:50.688098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-15T02:57:07.802 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T02:57:07.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.979+0000 7f56cccb9700 1 Processor -- start 2023-11-15T02:57:07.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.979+0000 7f56cccb9700 1 -- start start 2023-11-15T02:57:07.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.983+0000 7f56cccb9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f56c80a8960 0x7f56c80c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:07.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.983+0000 7f56cccb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56c8066520 0x7f56c8066990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:07.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.983+0000 7f56cccb9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c80657f0 con 0x7f56c80a8d40 2023-11-15T02:57:07.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.983+0000 7f56cccb9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c81583d0 con 0x7f56c80a8960 2023-11-15T02:57:07.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.983+0000 7f56cccb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80c1b60 con 0x7f56c8066520 2023-11-15T02:57:07.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.983+0000 7f56c77fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f56c80a8960 0x7f56c80c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:07.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.983+0000 7f56c77fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f56c80a8960 0x7f56c80c15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49762/0 (socket says 172.21.15.121:49762) 2023-11-15T02:57:07.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.983+0000 7f56c77fe700 1 -- 172.21.15.121:0/860214651 learned_addr learned my addr 172.21.15.121:0/860214651 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:07.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.983+0000 7f56c7fff700 1 --2- 172.21.15.121:0/860214651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56c8066520 0x7f56c8066990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:07.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.983+0000 7f56c77fe700 1 -- 172.21.15.121:0/860214651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56c8066520 msgr2=0x7f56c8066990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:07.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.983+0000 7f56c77fe700 1 --2- 172.21.15.121:0/860214651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56c8066520 0x7f56c8066990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:07.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.983+0000 7f56c77fe700 1 -- 172.21.15.121:0/860214651 >> v1:172.21.15.190:6789/0 conn(0x7f56c80a8d40 legacy=0x7f56c815c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:07.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.983+0000 7f56c77fe700 1 -- 172.21.15.121:0/860214651 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c8065a40 con 0x7f56c80a8960 2023-11-15T02:57:07.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.983+0000 7f56c77fe700 1 --2- 172.21.15.121:0/860214651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56c80a8960 0x7f56c80c15c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f56b8006fd0 tx=0x7f56b80090f0).ready entity=mon.0 client_cookie=6e025b5f0171bf8c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:07.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.983+0000 7f56c67fc700 1 -- 172.21.15.121:0/860214651 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b800b580 con 0x7f56c80a8960 2023-11-15T02:57:07.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.983+0000 7f56c67fc700 1 -- 172.21.15.121:0/860214651 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b800b6e0 con 0x7f56c80a8960 2023-11-15T02:57:07.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.983+0000 7f56c67fc700 1 -- 172.21.15.121:0/860214651 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b800ba60 con 0x7f56c80a8960 2023-11-15T02:57:07.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.987+0000 7f56cccb9700 1 -- 172.21.15.121:0/860214651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56c80a8960 msgr2=0x7f56c80c15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:07.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.987+0000 7f56cccb9700 1 --2- 172.21.15.121:0/860214651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56c80a8960 0x7f56c80c15c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f56b8006fd0 tx=0x7f56b80090f0).stop 2023-11-15T02:57:07.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.987+0000 7f56cccb9700 1 -- 172.21.15.121:0/860214651 shutdown_connections 2023-11-15T02:57:07.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.987+0000 7f56cccb9700 1 --2- 172.21.15.121:0/860214651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56c8066520 0x7f56c8066990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:07.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.987+0000 7f56cccb9700 1 --2- 172.21.15.121:0/860214651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56c80a8960 0x7f56c80c15c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:07.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.987+0000 7f56cccb9700 1 -- 172.21.15.121:0/860214651 >> 172.21.15.121:0/860214651 conn(0x7f56c8067a60 msgr2=0x7f56c8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:07.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.987+0000 7f56cccb9700 1 -- 172.21.15.121:0/860214651 shutdown_connections 2023-11-15T02:57:07.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.987+0000 7f56cccb9700 1 -- 172.21.15.121:0/860214651 wait complete. 2023-11-15T02:57:07.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.991+0000 7f56cccb9700 1 Processor -- start 2023-11-15T02:57:07.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.991+0000 7f56cccb9700 1 -- start start 2023-11-15T02:57:08.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.991+0000 7f56cccb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56c8066520 0x7f56c8156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:08.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.991+0000 7f56cccb9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56c80a8960 0x7f56c80aa600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:08.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.991+0000 7f56cccb9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c80a8d40 0x7f56c81472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:08.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.991+0000 7f56cccb9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80bb8c0 con 0x7f56c80a8960 2023-11-15T02:57:08.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.991+0000 7f56cccb9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c8157380 con 0x7f56c80a8d40 2023-11-15T02:57:08.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.991+0000 7f56cccb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80c5bf0 con 0x7f56c8066520 2023-11-15T02:57:08.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.991+0000 7f56c6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56c80a8960 0x7f56c80aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:08.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.991+0000 7f56c6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56c80a8960 0x7f56c80aa600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49770/0 (socket says 172.21.15.121:49770) 2023-11-15T02:57:08.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.991+0000 7f56c6ffd700 1 -- 172.21.15.121:0/577177482 learned_addr learned my addr 172.21.15.121:0/577177482 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:08.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.991+0000 7f56c7fff700 1 --2- 172.21.15.121:0/577177482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c80a8d40 0x7f56c81472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:08.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.991+0000 7f56c77fe700 1 --2- 172.21.15.121:0/577177482 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56c8066520 0x7f56c8156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:08.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.991+0000 7f56c6ffd700 1 -- 172.21.15.121:0/577177482 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56c8066520 msgr2=0x7f56c8156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:08.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.991+0000 7f56c6ffd700 1 --2- 172.21.15.121:0/577177482 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56c8066520 0x7f56c8156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.991+0000 7f56c6ffd700 1 -- 172.21.15.121:0/577177482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c80a8d40 msgr2=0x7f56c81472e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:08.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.991+0000 7f56c6ffd700 1 --2- 172.21.15.121:0/577177482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c80a8d40 0x7f56c81472e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.991+0000 7f56c6ffd700 1 -- 172.21.15.121:0/577177482 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b8009370 con 0x7f56c80a8960 2023-11-15T02:57:08.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.991+0000 7f56c7fff700 1 --2- 172.21.15.121:0/577177482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c80a8d40 0x7f56c81472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:08.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.995+0000 7f56c6ffd700 1 --2- 172.21.15.121:0/577177482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56c80a8960 0x7f56c80aa600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f56bc00c440 tx=0x7f56bc007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:08.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.995+0000 7f56af7fe700 1 -- 172.21.15.121:0/577177482 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00ce90 con 0x7f56c80a8960 2023-11-15T02:57:08.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.995+0000 7f56af7fe700 1 -- 172.21.15.121:0/577177482 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc007020 con 0x7f56c80a8960 2023-11-15T02:57:08.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.995+0000 7f56af7fe700 1 -- 172.21.15.121:0/577177482 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc0073c0 con 0x7f56c80a8960 2023-11-15T02:57:08.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.995+0000 7f56cccb9700 1 -- 172.21.15.121:0/577177482 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c816bc60 con 0x7f56c80a8960 2023-11-15T02:57:08.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.995+0000 7f56cccb9700 1 -- 172.21.15.121:0/577177482 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c816c2a0 con 0x7f56c80a8960 2023-11-15T02:57:08.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.995+0000 7f56cccb9700 1 -- 172.21.15.121:0/577177482 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c816c620 con 0x7f56c80a8960 2023-11-15T02:57:08.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.995+0000 7f56af7fe700 1 -- 172.21.15.121:0/577177482 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f56bc0068b0 con 0x7f56c80a8960 2023-11-15T02:57:08.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.999+0000 7f56af7fe700 1 --2- 172.21.15.121:0/577177482 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f56b405a510 0x7f56b405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:08.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.999+0000 7f56af7fe700 1 -- 172.21.15.121:0/577177482 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f56bc073c00 con 0x7f56c80a8960 2023-11-15T02:57:08.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.999+0000 7f56c77fe700 1 --2- 172.21.15.121:0/577177482 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f56b405a510 0x7f56b405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:08.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:07.999+0000 7f56c77fe700 1 --2- 172.21.15.121:0/577177482 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f56b405a510 0x7f56b405c9d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f56b8009130 tx=0x7f56b8017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:08.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.003+0000 7f56af7fe700 1 -- 172.21.15.121:0/577177482 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f56bc047320 con 0x7f56c80a8960 2023-11-15T02:57:08.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.163+0000 7f56cccb9700 1 -- 172.21.15.121:0/577177482 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f56c806bcf0 con 0x7f56c80a8960 2023-11-15T02:57:08.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.167+0000 7f56af7fe700 1 -- 172.21.15.121:0/577177482 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v21) v1 ==== 110+0+10 (secure 0 0 0) 0x7f56bc04a940 con 0x7f56c80a8960 2023-11-15T02:57:08.171 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T02:57:08.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.167+0000 7f56cccb9700 1 -- 172.21.15.121:0/577177482 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f56b405a510 msgr2=0x7f56b405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:08.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.167+0000 7f56cccb9700 1 --2- 172.21.15.121:0/577177482 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f56b405a510 0x7f56b405c9d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f56b8009130 tx=0x7f56b8017000).stop 2023-11-15T02:57:08.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.167+0000 7f56cccb9700 1 -- 172.21.15.121:0/577177482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56c80a8960 msgr2=0x7f56c80aa600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:08.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.167+0000 7f56cccb9700 1 --2- 172.21.15.121:0/577177482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56c80a8960 0x7f56c80aa600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f56bc00c440 tx=0x7f56bc007e00).stop 2023-11-15T02:57:08.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.167+0000 7f56cccb9700 1 -- 172.21.15.121:0/577177482 shutdown_connections 2023-11-15T02:57:08.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.167+0000 7f56cccb9700 1 --2- 172.21.15.121:0/577177482 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f56b405a510 0x7f56b405c9d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.167+0000 7f56cccb9700 1 --2- 172.21.15.121:0/577177482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c80a8d40 0x7f56c81472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.167+0000 7f56cccb9700 1 --2- 172.21.15.121:0/577177482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56c80a8960 0x7f56c80aa600 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.167+0000 7f56cccb9700 1 --2- 172.21.15.121:0/577177482 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56c8066520 0x7f56c8156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.167+0000 7f56cccb9700 1 -- 172.21.15.121:0/577177482 >> 172.21.15.121:0/577177482 conn(0x7f56c8067a60 msgr2=0x7f56c80a5760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:08.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.167+0000 7f56cccb9700 1 -- 172.21.15.121:0/577177482 shutdown_connections 2023-11-15T02:57:08.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.167+0000 7f56cccb9700 1 -- 172.21.15.121:0/577177482 wait complete. 2023-11-15T02:57:08.193 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T02:57:08.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.351+0000 7f0291707700 1 Processor -- start 2023-11-15T02:57:08.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.351+0000 7f0291707700 1 -- start start 2023-11-15T02:57:08.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.355+0000 7f0291707700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f028c14e380 0x7f028c0c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:08.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.355+0000 7f0291707700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f028c0ad270 0x7f028c065680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:08.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.355+0000 7f0291707700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f028c0c7e90 con 0x7f028c0c6040 2023-11-15T02:57:08.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.355+0000 7f0291707700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f028c0c8320 con 0x7f028c0ad270 2023-11-15T02:57:08.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.355+0000 7f0291707700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0c7080 con 0x7f028c14e380 2023-11-15T02:57:08.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.355+0000 7f028b7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f028c0c6040 0x7f028c0acb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59690/0 (socket says 172.21.15.121:59690) 2023-11-15T02:57:08.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.355+0000 7f028b7fe700 1 -- 172.21.15.121:0/1890187448 learned_addr learned my addr 172.21.15.121:0/1890187448 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:08.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.355+0000 7f028bfff700 1 --2- 172.21.15.121:0/1890187448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f028c14e380 0x7f028c0c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:08.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.355+0000 7f0290f06700 1 --2- 172.21.15.121:0/1890187448 >> v2:172.21.15.190:3300/0 conn(0x7f028c0ad270 0x7f028c065680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:08.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.355+0000 7f028affd700 1 -- 172.21.15.121:0/1890187448 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323263871 0 0) 0x7f028c0c7e90 con 0x7f028c0c6040 2023-11-15T02:57:08.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.355+0000 7f028affd700 1 -- 172.21.15.121:0/1890187448 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02740033f0 con 0x7f028c0c6040 2023-11-15T02:57:08.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.355+0000 7f028bfff700 1 -- 172.21.15.121:0/1890187448 >> v2:172.21.15.190:3300/0 conn(0x7f028c0ad270 msgr2=0x7f028c065680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:08.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.355+0000 7f028bfff700 1 --2- 172.21.15.121:0/1890187448 >> v2:172.21.15.190:3300/0 conn(0x7f028c0ad270 0x7f028c065680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.355+0000 7f028bfff700 1 -- 172.21.15.121:0/1890187448 >> v1:172.21.15.121:6789/0 conn(0x7f028c0c6040 legacy=0x7f028c0acb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:08.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.355+0000 7f028bfff700 1 -- 172.21.15.121:0/1890187448 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c14e760 con 0x7f028c14e380 2023-11-15T02:57:08.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.355+0000 7f0290f06700 1 --2- 172.21.15.121:0/1890187448 >> v2:172.21.15.190:3300/0 conn(0x7f028c0ad270 0x7f028c065680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:57:08.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.359+0000 7f028bfff700 1 --2- 172.21.15.121:0/1890187448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f028c14e380 0x7f028c0c5b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f027c009760 tx=0x7f027c009060).ready entity=mon.2 client_cookie=1b4c3ce66733ebc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:08.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.359+0000 7f028affd700 1 -- 172.21.15.121:0/1890187448 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c00d4a0 con 0x7f028c14e380 2023-11-15T02:57:08.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.359+0000 7f028affd700 1 -- 172.21.15.121:0/1890187448 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027c00d600 con 0x7f028c14e380 2023-11-15T02:57:08.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.359+0000 7f028affd700 1 -- 172.21.15.121:0/1890187448 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c00d980 con 0x7f028c14e380 2023-11-15T02:57:08.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.359+0000 7f0291707700 1 -- 172.21.15.121:0/1890187448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f028c14e380 msgr2=0x7f028c0c5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:08.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.359+0000 7f0291707700 1 --2- 172.21.15.121:0/1890187448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f028c14e380 0x7f028c0c5b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f027c009760 tx=0x7f027c009060).stop 2023-11-15T02:57:08.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.359+0000 7f0291707700 1 -- 172.21.15.121:0/1890187448 shutdown_connections 2023-11-15T02:57:08.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.359+0000 7f0291707700 1 --2- 172.21.15.121:0/1890187448 >> v2:172.21.15.190:3300/0 conn(0x7f028c0ad270 0x7f028c065680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.359+0000 7f0291707700 1 --2- 172.21.15.121:0/1890187448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f028c14e380 0x7f028c0c5b50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.359+0000 7f0291707700 1 -- 172.21.15.121:0/1890187448 >> 172.21.15.121:0/1890187448 conn(0x7f028c067a60 msgr2=0x7f028c067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:08.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.359+0000 7f0291707700 1 -- 172.21.15.121:0/1890187448 shutdown_connections 2023-11-15T02:57:08.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.359+0000 7f0291707700 1 -- 172.21.15.121:0/1890187448 wait complete. 2023-11-15T02:57:08.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.359+0000 7f0291707700 1 Processor -- start 2023-11-15T02:57:08.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.359+0000 7f0291707700 1 -- start start 2023-11-15T02:57:08.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f0291707700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f028c0ad270 0x7f028c149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:08.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f0291707700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f028c0c6040 0x7f028c0ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:08.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f0291707700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f028c14e380 0x7f028c1542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:08.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f0291707700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c14a440 con 0x7f028c0c6040 2023-11-15T02:57:08.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f0291707700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0c1a80 con 0x7f028c0ad270 2023-11-15T02:57:08.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f0291707700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0c1d60 con 0x7f028c14e380 2023-11-15T02:57:08.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f028b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f028c0c6040 0x7f028c0ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:08.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f028b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f028c0c6040 0x7f028c0ad8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49782/0 (socket says 172.21.15.121:49782) 2023-11-15T02:57:08.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f028b7fe700 1 -- 172.21.15.121:0/2981204775 learned_addr learned my addr 172.21.15.121:0/2981204775 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:08.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f0290f06700 1 --2- 172.21.15.121:0/2981204775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f028c14e380 0x7f028c1542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:08.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f028bfff700 1 --2- 172.21.15.121:0/2981204775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f028c0ad270 0x7f028c149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:08.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f028b7fe700 1 -- 172.21.15.121:0/2981204775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f028c14e380 msgr2=0x7f028c1542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:08.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f028b7fe700 1 --2- 172.21.15.121:0/2981204775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f028c14e380 0x7f028c1542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f028b7fe700 1 -- 172.21.15.121:0/2981204775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f028c0ad270 msgr2=0x7f028c149f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:08.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f028b7fe700 1 --2- 172.21.15.121:0/2981204775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f028c0ad270 0x7f028c149f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f028b7fe700 1 -- 172.21.15.121:0/2981204775 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0284009a20 con 0x7f028c0c6040 2023-11-15T02:57:08.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f028b7fe700 1 --2- 172.21.15.121:0/2981204775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f028c0c6040 0x7f028c0ad8f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f028000c490 tx=0x7f0280007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:08.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f0273fff700 1 -- 172.21.15.121:0/2981204775 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028000ce00 con 0x7f028c0c6040 2023-11-15T02:57:08.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f0273fff700 1 -- 172.21.15.121:0/2981204775 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02800072e0 con 0x7f028c0c6040 2023-11-15T02:57:08.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f0291707700 1 -- 172.21.15.121:0/2981204775 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027c0092a0 con 0x7f028c0c6040 2023-11-15T02:57:08.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f0273fff700 1 -- 172.21.15.121:0/2981204775 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02800057a0 con 0x7f028c0c6040 2023-11-15T02:57:08.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.363+0000 7f0291707700 1 -- 172.21.15.121:0/2981204775 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028c0c2320 con 0x7f028c0c6040 2023-11-15T02:57:08.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.367+0000 7f0291707700 1 -- 172.21.15.121:0/2981204775 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028c06bcf0 con 0x7f028c0c6040 2023-11-15T02:57:08.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.367+0000 7f0273fff700 1 -- 172.21.15.121:0/2981204775 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0280005b10 con 0x7f028c0c6040 2023-11-15T02:57:08.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.367+0000 7f0273fff700 1 --2- 172.21.15.121:0/2981204775 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f027805a540 0x7f027805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:08.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.367+0000 7f0273fff700 1 -- 172.21.15.121:0/2981204775 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0280073da0 con 0x7f028c0c6040 2023-11-15T02:57:08.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.367+0000 7f028bfff700 1 --2- 172.21.15.121:0/2981204775 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f027805a540 0x7f027805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:08.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.371+0000 7f028bfff700 1 --2- 172.21.15.121:0/2981204775 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f027805a540 0x7f027805ca00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f027c00d260 tx=0x7f027c016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:08.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.375+0000 7f0273fff700 1 -- 172.21.15.121:0/2981204775 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0280047580 con 0x7f028c0c6040 2023-11-15T02:57:08.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.535+0000 7f0291707700 1 -- 172.21.15.121:0/2981204775 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f028c0c7280 con 0x7f028c0c6040 2023-11-15T02:57:08.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.535+0000 7f0273fff700 1 -- 172.21.15.121:0/2981204775 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0280018030 con 0x7f028c0c6040 2023-11-15T02:57:08.543 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T02:57:08.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.539+0000 7f0291707700 1 -- 172.21.15.121:0/2981204775 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f027805a540 msgr2=0x7f027805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:08.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.539+0000 7f0291707700 1 --2- 172.21.15.121:0/2981204775 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f027805a540 0x7f027805ca00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f027c00d260 tx=0x7f027c016000).stop 2023-11-15T02:57:08.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.539+0000 7f0291707700 1 -- 172.21.15.121:0/2981204775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f028c0c6040 msgr2=0x7f028c0ad8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:08.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.539+0000 7f0291707700 1 --2- 172.21.15.121:0/2981204775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f028c0c6040 0x7f028c0ad8f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f028000c490 tx=0x7f0280007c10).stop 2023-11-15T02:57:08.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.539+0000 7f0291707700 1 -- 172.21.15.121:0/2981204775 shutdown_connections 2023-11-15T02:57:08.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.539+0000 7f0291707700 1 --2- 172.21.15.121:0/2981204775 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f027805a540 0x7f027805ca00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.539+0000 7f0291707700 1 --2- 172.21.15.121:0/2981204775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f028c14e380 0x7f028c1542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.539+0000 7f0291707700 1 --2- 172.21.15.121:0/2981204775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f028c0c6040 0x7f028c0ad8f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.539+0000 7f0291707700 1 --2- 172.21.15.121:0/2981204775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f028c0ad270 0x7f028c149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.539+0000 7f0291707700 1 -- 172.21.15.121:0/2981204775 >> 172.21.15.121:0/2981204775 conn(0x7f028c067a60 msgr2=0x7f028c0c8460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:08.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.539+0000 7f0291707700 1 -- 172.21.15.121:0/2981204775 shutdown_connections 2023-11-15T02:57:08.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.539+0000 7f0291707700 1 -- 172.21.15.121:0/2981204775 wait complete. 2023-11-15T02:57:08.564 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-15T02:57:08.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.731+0000 7f12f989b700 1 Processor -- start 2023-11-15T02:57:08.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.731+0000 7f12f989b700 1 -- start start 2023-11-15T02:57:08.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.731+0000 7f12f989b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f12f40a8b80 0x7f12f40a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:08.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.731+0000 7f12f989b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f12f4068170 0x7f12f40bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:08.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.731+0000 7f12f989b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f4152150 con 0x7f12f4065850 2023-11-15T02:57:08.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.731+0000 7f12f989b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12f4148f70 con 0x7f12f40a8b80 2023-11-15T02:57:08.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.731+0000 7f12f989b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12f41523a0 con 0x7f12f4068170 2023-11-15T02:57:08.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.731+0000 7f12f2ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f12f40a8b80 0x7f12f40a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:08.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.731+0000 7f12f2ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f12f40a8b80 0x7f12f40a8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49796/0 (socket says 172.21.15.121:49796) 2023-11-15T02:57:08.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.731+0000 7f12f2ffd700 1 -- 172.21.15.121:0/3762630140 learned_addr learned my addr 172.21.15.121:0/3762630140 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:08.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.731+0000 7f12f37fe700 1 --2- 172.21.15.121:0/3762630140 >> v2:172.21.15.190:3300/0 conn(0x7f12f4068170 0x7f12f40bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:08.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.735+0000 7f12f2ffd700 1 -- 172.21.15.121:0/3762630140 >> v2:172.21.15.190:3300/0 conn(0x7f12f4068170 msgr2=0x7f12f40bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:08.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.735+0000 7f12f2ffd700 1 --2- 172.21.15.121:0/3762630140 >> v2:172.21.15.190:3300/0 conn(0x7f12f4068170 0x7f12f40bbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.735+0000 7f12f2ffd700 1 -- 172.21.15.121:0/3762630140 >> v1:172.21.15.190:6789/0 conn(0x7f12f4065850 legacy=0x7f12f4067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:08.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.735+0000 7f12f2ffd700 1 -- 172.21.15.121:0/3762630140 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f4152680 con 0x7f12f40a8b80 2023-11-15T02:57:08.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.735+0000 7f12f2ffd700 1 --2- 172.21.15.121:0/3762630140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12f40a8b80 0x7f12f40a8f60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f12f4152060 tx=0x7f12e40090f0).ready entity=mon.0 client_cookie=7ae1cae090174caf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:08.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.735+0000 7f12f27fc700 1 -- 172.21.15.121:0/3762630140 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e400b520 con 0x7f12f40a8b80 2023-11-15T02:57:08.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.735+0000 7f12f27fc700 1 -- 172.21.15.121:0/3762630140 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e400b680 con 0x7f12f40a8b80 2023-11-15T02:57:08.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.735+0000 7f12f27fc700 1 -- 172.21.15.121:0/3762630140 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e400ba00 con 0x7f12f40a8b80 2023-11-15T02:57:08.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.735+0000 7f12f989b700 1 -- 172.21.15.121:0/3762630140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12f40a8b80 msgr2=0x7f12f40a8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:08.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.735+0000 7f12f989b700 1 --2- 172.21.15.121:0/3762630140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12f40a8b80 0x7f12f40a8f60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f12f4152060 tx=0x7f12e40090f0).stop 2023-11-15T02:57:08.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.735+0000 7f12f989b700 1 -- 172.21.15.121:0/3762630140 shutdown_connections 2023-11-15T02:57:08.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.735+0000 7f12f989b700 1 --2- 172.21.15.121:0/3762630140 >> v2:172.21.15.190:3300/0 conn(0x7f12f4068170 0x7f12f40bbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.735+0000 7f12f989b700 1 --2- 172.21.15.121:0/3762630140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12f40a8b80 0x7f12f40a8f60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.735+0000 7f12f989b700 1 -- 172.21.15.121:0/3762630140 >> 172.21.15.121:0/3762630140 conn(0x7f12f40591c0 msgr2=0x7f12f40c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:08.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.735+0000 7f12f989b700 1 -- 172.21.15.121:0/3762630140 shutdown_connections 2023-11-15T02:57:08.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.735+0000 7f12f989b700 1 -- 172.21.15.121:0/3762630140 wait complete. 2023-11-15T02:57:08.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.739+0000 7f12f989b700 1 Processor -- start 2023-11-15T02:57:08.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.739+0000 7f12f989b700 1 -- start start 2023-11-15T02:57:08.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.739+0000 7f12f989b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12f4065850 0x7f12f40c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:08.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.739+0000 7f12f989b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12f4068170 0x7f12f4155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:08.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.739+0000 7f12f989b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12f40a8b80 0x7f12f413cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:08.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.739+0000 7f12f989b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f4154090 con 0x7f12f40a8b80 2023-11-15T02:57:08.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.739+0000 7f12f989b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f41632f0 con 0x7f12f4068170 2023-11-15T02:57:08.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.739+0000 7f12f989b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f4153480 con 0x7f12f4065850 2023-11-15T02:57:08.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.739+0000 7f12f37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12f40a8b80 0x7f12f413cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:08.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.739+0000 7f12f37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12f40a8b80 0x7f12f413cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49804/0 (socket says 172.21.15.121:49804) 2023-11-15T02:57:08.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.739+0000 7f12f37fe700 1 -- 172.21.15.121:0/1507127751 learned_addr learned my addr 172.21.15.121:0/1507127751 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:08.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.739+0000 7f12f2ffd700 1 --2- 172.21.15.121:0/1507127751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12f4065850 0x7f12f40c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:08.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.739+0000 7f12ebfff700 1 --2- 172.21.15.121:0/1507127751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12f4068170 0x7f12f4155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:08.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.743+0000 7f12f37fe700 1 -- 172.21.15.121:0/1507127751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12f4065850 msgr2=0x7f12f40c0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:08.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.743+0000 7f12f37fe700 1 --2- 172.21.15.121:0/1507127751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12f4065850 0x7f12f40c0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.743+0000 7f12f37fe700 1 -- 172.21.15.121:0/1507127751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12f4068170 msgr2=0x7f12f4155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:08.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.743+0000 7f12f37fe700 1 --2- 172.21.15.121:0/1507127751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12f4068170 0x7f12f4155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.743+0000 7f12f37fe700 1 -- 172.21.15.121:0/1507127751 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e4009370 con 0x7f12f40a8b80 2023-11-15T02:57:08.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.743+0000 7f12ebfff700 1 --2- 172.21.15.121:0/1507127751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12f4068170 0x7f12f4155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:08.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.743+0000 7f12f37fe700 1 --2- 172.21.15.121:0/1507127751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12f40a8b80 0x7f12f413cde0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f12ec009d70 tx=0x7f12ec00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:08.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.743+0000 7f12eaffd700 1 -- 172.21.15.121:0/1507127751 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec00cb50 con 0x7f12f40a8b80 2023-11-15T02:57:08.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.743+0000 7f12eaffd700 1 -- 172.21.15.121:0/1507127751 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ec00f040 con 0x7f12f40a8b80 2023-11-15T02:57:08.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.743+0000 7f12f989b700 1 -- 172.21.15.121:0/1507127751 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f4153760 con 0x7f12f40a8b80 2023-11-15T02:57:08.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.743+0000 7f12eaffd700 1 -- 172.21.15.121:0/1507127751 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec013400 con 0x7f12f40a8b80 2023-11-15T02:57:08.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.743+0000 7f12f989b700 1 -- 172.21.15.121:0/1507127751 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f4153d20 con 0x7f12f40a8b80 2023-11-15T02:57:08.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.743+0000 7f12f989b700 1 -- 172.21.15.121:0/1507127751 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f4154240 con 0x7f12f40a8b80 2023-11-15T02:57:08.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.743+0000 7f12eaffd700 1 -- 172.21.15.121:0/1507127751 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f12ec0137b0 con 0x7f12f40a8b80 2023-11-15T02:57:08.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.747+0000 7f12eaffd700 1 --2- 172.21.15.121:0/1507127751 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f12d405a510 0x7f12d405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:08.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.747+0000 7f12eaffd700 1 -- 172.21.15.121:0/1507127751 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f12ec074fd0 con 0x7f12f40a8b80 2023-11-15T02:57:08.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.747+0000 7f12f2ffd700 1 --2- 172.21.15.121:0/1507127751 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f12d405a510 0x7f12d405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:08.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.747+0000 7f12f2ffd700 1 --2- 172.21.15.121:0/1507127751 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f12d405a510 0x7f12d405c9d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f12e4009130 tx=0x7f12e4017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:08.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.751+0000 7f12eaffd700 1 -- 172.21.15.121:0/1507127751 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f12ec0486f0 con 0x7f12f40a8b80 2023-11-15T02:57:08.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.915+0000 7f12f989b700 1 -- 172.21.15.121:0/1507127751 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f12f4063b60 con 0x7f12f40a8b80 2023-11-15T02:57:08.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.915+0000 7f12eaffd700 1 -- 172.21.15.121:0/1507127751 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f12ec04bd10 con 0x7f12f40a8b80 2023-11-15T02:57:08.920 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T02:57:08.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.915+0000 7f12f989b700 1 -- 172.21.15.121:0/1507127751 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f12d405a510 msgr2=0x7f12d405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:08.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.915+0000 7f12f989b700 1 --2- 172.21.15.121:0/1507127751 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f12d405a510 0x7f12d405c9d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f12e4009130 tx=0x7f12e4017000).stop 2023-11-15T02:57:08.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.915+0000 7f12f989b700 1 -- 172.21.15.121:0/1507127751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12f40a8b80 msgr2=0x7f12f413cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:08.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.915+0000 7f12f989b700 1 --2- 172.21.15.121:0/1507127751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12f40a8b80 0x7f12f413cde0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f12ec009d70 tx=0x7f12ec00b040).stop 2023-11-15T02:57:08.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.919+0000 7f12f989b700 1 -- 172.21.15.121:0/1507127751 shutdown_connections 2023-11-15T02:57:08.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.919+0000 7f12f989b700 1 --2- 172.21.15.121:0/1507127751 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f12d405a510 0x7f12d405c9d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.919+0000 7f12f989b700 1 --2- 172.21.15.121:0/1507127751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12f40a8b80 0x7f12f413cde0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.919+0000 7f12f989b700 1 --2- 172.21.15.121:0/1507127751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12f4068170 0x7f12f4155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.919+0000 7f12f989b700 1 --2- 172.21.15.121:0/1507127751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12f4065850 0x7f12f40c0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:08.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.919+0000 7f12f989b700 1 -- 172.21.15.121:0/1507127751 >> 172.21.15.121:0/1507127751 conn(0x7f12f40591c0 msgr2=0x7f12f414b120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:08.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.919+0000 7f12f989b700 1 -- 172.21.15.121:0/1507127751 shutdown_connections 2023-11-15T02:57:08.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:08.919+0000 7f12f989b700 1 -- 172.21.15.121:0/1507127751 wait complete. 2023-11-15T02:57:08.942 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-15T02:57:09.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.107+0000 7fba5acb9700 1 Processor -- start 2023-11-15T02:57:09.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.107+0000 7fba5acb9700 1 -- start start 2023-11-15T02:57:09.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.107+0000 7fba5acb9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fba540acd00 0x7fba540ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:09.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.107+0000 7fba5acb9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fba54066330 0x7fba54153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:09.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.107+0000 7fba5acb9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba54065ea0 con 0x7fba540c5c70 2023-11-15T02:57:09.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.107+0000 7fba5acb9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba54066bb0 con 0x7fba54066330 2023-11-15T02:57:09.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.107+0000 7fba5acb9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba54065890 con 0x7fba540acd00 2023-11-15T02:57:09.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.107+0000 7fba5a4b8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fba54066330 0x7fba54153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:09.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.107+0000 7fba5a4b8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fba54066330 0x7fba54153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49808/0 (socket says 172.21.15.121:49808) 2023-11-15T02:57:09.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.107+0000 7fba5a4b8700 1 -- 172.21.15.121:0/805116759 learned_addr learned my addr 172.21.15.121:0/805116759 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:09.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.107+0000 7fba59cb7700 1 --2- 172.21.15.121:0/805116759 >> v2:172.21.15.190:3300/0 conn(0x7fba540acd00 0x7fba540ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:09.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.107+0000 7fba5a4b8700 1 -- 172.21.15.121:0/805116759 >> v2:172.21.15.190:3300/0 conn(0x7fba540acd00 msgr2=0x7fba540ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:09.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.107+0000 7fba5a4b8700 1 --2- 172.21.15.121:0/805116759 >> v2:172.21.15.190:3300/0 conn(0x7fba540acd00 0x7fba540ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.107+0000 7fba5a4b8700 1 -- 172.21.15.121:0/805116759 >> v1:172.21.15.190:6789/0 conn(0x7fba540c5c70 legacy=0x7fba541536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:09.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.107+0000 7fba5a4b8700 1 -- 172.21.15.121:0/805116759 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba540c64a0 con 0x7fba54066330 2023-11-15T02:57:09.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.111+0000 7fba5a4b8700 1 --2- 172.21.15.121:0/805116759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba54066330 0x7fba54153dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fba5406c280 tx=0x7fba4c009840).ready entity=mon.0 client_cookie=98916df6855154cb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:09.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.111+0000 7fba58cb5700 1 -- 172.21.15.121:0/805116759 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c00e070 con 0x7fba54066330 2023-11-15T02:57:09.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.111+0000 7fba58cb5700 1 -- 172.21.15.121:0/805116759 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c00ba20 con 0x7fba54066330 2023-11-15T02:57:09.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.111+0000 7fba58cb5700 1 -- 172.21.15.121:0/805116759 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c00bda0 con 0x7fba54066330 2023-11-15T02:57:09.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.111+0000 7fba5acb9700 1 -- 172.21.15.121:0/805116759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba54066330 msgr2=0x7fba54153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:09.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.111+0000 7fba5acb9700 1 --2- 172.21.15.121:0/805116759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba54066330 0x7fba54153dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fba5406c280 tx=0x7fba4c009840).stop 2023-11-15T02:57:09.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.111+0000 7fba5acb9700 1 -- 172.21.15.121:0/805116759 shutdown_connections 2023-11-15T02:57:09.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.111+0000 7fba5acb9700 1 --2- 172.21.15.121:0/805116759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba54066330 0x7fba54153dd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.111+0000 7fba5acb9700 1 --2- 172.21.15.121:0/805116759 >> v2:172.21.15.190:3300/0 conn(0x7fba540acd00 0x7fba540ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.111+0000 7fba5acb9700 1 -- 172.21.15.121:0/805116759 >> 172.21.15.121:0/805116759 conn(0x7fba54063fc0 msgr2=0x7fba54067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:09.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.111+0000 7fba5acb9700 1 -- 172.21.15.121:0/805116759 shutdown_connections 2023-11-15T02:57:09.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.111+0000 7fba5acb9700 1 -- 172.21.15.121:0/805116759 wait complete. 2023-11-15T02:57:09.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.115+0000 7fba5acb9700 1 Processor -- start 2023-11-15T02:57:09.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.115+0000 7fba5acb9700 1 -- start start 2023-11-15T02:57:09.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.115+0000 7fba5acb9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba54066330 0x7fba541571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:09.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.115+0000 7fba5acb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba540acd00 0x7fba54164b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:09.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.115+0000 7fba5acb9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba540c5c70 0x7fba54154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:09.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.115+0000 7fba5acb9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba54150d30 con 0x7fba54066330 2023-11-15T02:57:09.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.115+0000 7fba5acb9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba54165020 con 0x7fba540c5c70 2023-11-15T02:57:09.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.115+0000 7fba5acb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba54156450 con 0x7fba540acd00 2023-11-15T02:57:09.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.115+0000 7fba59cb7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba54066330 0x7fba541571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:09.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.119+0000 7fba59cb7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba54066330 0x7fba541571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49812/0 (socket says 172.21.15.121:49812) 2023-11-15T02:57:09.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.119+0000 7fba59cb7700 1 -- 172.21.15.121:0/3243433477 learned_addr learned my addr 172.21.15.121:0/3243433477 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:09.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.119+0000 7fba5a4b8700 1 --2- 172.21.15.121:0/3243433477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba540c5c70 0x7fba54154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:09.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.119+0000 7fba594b6700 1 --2- 172.21.15.121:0/3243433477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba540acd00 0x7fba54164b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:09.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.119+0000 7fba59cb7700 1 -- 172.21.15.121:0/3243433477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba540acd00 msgr2=0x7fba54164b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:09.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.119+0000 7fba59cb7700 1 --2- 172.21.15.121:0/3243433477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba540acd00 0x7fba54164b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.119+0000 7fba59cb7700 1 -- 172.21.15.121:0/3243433477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba540c5c70 msgr2=0x7fba54154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:09.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.119+0000 7fba59cb7700 1 --2- 172.21.15.121:0/3243433477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba540c5c70 0x7fba54154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.119+0000 7fba59cb7700 1 -- 172.21.15.121:0/3243433477 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4c009ae0 con 0x7fba54066330 2023-11-15T02:57:09.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.119+0000 7fba59cb7700 1 --2- 172.21.15.121:0/3243433477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba54066330 0x7fba541571a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fba5000e900 tx=0x7fba50006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:09.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.119+0000 7fba49ffb700 1 -- 172.21.15.121:0/3243433477 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba50009450 con 0x7fba54066330 2023-11-15T02:57:09.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.119+0000 7fba49ffb700 1 -- 172.21.15.121:0/3243433477 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba50012600 con 0x7fba54066330 2023-11-15T02:57:09.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.119+0000 7fba49ffb700 1 -- 172.21.15.121:0/3243433477 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba50012980 con 0x7fba54066330 2023-11-15T02:57:09.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.119+0000 7fba5acb9700 1 -- 172.21.15.121:0/3243433477 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba541566c0 con 0x7fba54066330 2023-11-15T02:57:09.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.119+0000 7fba5acb9700 1 -- 172.21.15.121:0/3243433477 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba54156c80 con 0x7fba54066330 2023-11-15T02:57:09.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.119+0000 7fba5acb9700 1 -- 172.21.15.121:0/3243433477 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba5406bcf0 con 0x7fba54066330 2023-11-15T02:57:09.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.123+0000 7fba49ffb700 1 -- 172.21.15.121:0/3243433477 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fba50012cf0 con 0x7fba54066330 2023-11-15T02:57:09.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.123+0000 7fba49ffb700 1 --2- 172.21.15.121:0/3243433477 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fba3805a510 0x7fba3805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:09.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.123+0000 7fba49ffb700 1 -- 172.21.15.121:0/3243433477 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fba50074c70 con 0x7fba54066330 2023-11-15T02:57:09.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.123+0000 7fba594b6700 1 --2- 172.21.15.121:0/3243433477 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fba3805a510 0x7fba3805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:09.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.123+0000 7fba594b6700 1 --2- 172.21.15.121:0/3243433477 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fba3805a510 0x7fba3805c9d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fba540af300 tx=0x7fba44009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:09.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.127+0000 7fba49ffb700 1 -- 172.21.15.121:0/3243433477 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fba50048390 con 0x7fba54066330 2023-11-15T02:57:09.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.291+0000 7fba5acb9700 1 -- 172.21.15.121:0/3243433477 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba540c61c0 con 0x7fba54066330 2023-11-15T02:57:09.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.291+0000 7fba49ffb700 1 -- 172.21.15.121:0/3243433477 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba50016070 con 0x7fba54066330 2023-11-15T02:57:09.298 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T02:57:09.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.295+0000 7fba5acb9700 1 -- 172.21.15.121:0/3243433477 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fba3805a510 msgr2=0x7fba3805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:09.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.295+0000 7fba5acb9700 1 --2- 172.21.15.121:0/3243433477 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fba3805a510 0x7fba3805c9d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fba540af300 tx=0x7fba44009040).stop 2023-11-15T02:57:09.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.295+0000 7fba5acb9700 1 -- 172.21.15.121:0/3243433477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba54066330 msgr2=0x7fba541571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:09.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.295+0000 7fba5acb9700 1 --2- 172.21.15.121:0/3243433477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba54066330 0x7fba541571a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fba5000e900 tx=0x7fba50006f40).stop 2023-11-15T02:57:09.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.295+0000 7fba5acb9700 1 -- 172.21.15.121:0/3243433477 shutdown_connections 2023-11-15T02:57:09.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.295+0000 7fba5acb9700 1 --2- 172.21.15.121:0/3243433477 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fba3805a510 0x7fba3805c9d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.295+0000 7fba5acb9700 1 --2- 172.21.15.121:0/3243433477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba540c5c70 0x7fba54154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.295+0000 7fba5acb9700 1 --2- 172.21.15.121:0/3243433477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba540acd00 0x7fba54164b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.295+0000 7fba5acb9700 1 --2- 172.21.15.121:0/3243433477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba54066330 0x7fba541571a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.295+0000 7fba5acb9700 1 -- 172.21.15.121:0/3243433477 >> 172.21.15.121:0/3243433477 conn(0x7fba54063fc0 msgr2=0x7fba54152a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:09.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.295+0000 7fba5acb9700 1 -- 172.21.15.121:0/3243433477 shutdown_connections 2023-11-15T02:57:09.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.295+0000 7fba5acb9700 1 -- 172.21.15.121:0/3243433477 wait complete. 2023-11-15T02:57:09.321 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:57:09.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.475+0000 7effcd45b700 1 Processor -- start 2023-11-15T02:57:09.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.475+0000 7effcd45b700 1 -- start start 2023-11-15T02:57:09.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.479+0000 7effcd45b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7effc8152780 0x7effc8159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:09.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.479+0000 7effcd45b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7effc80c65d0 0x7effc80c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:09.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.479+0000 7effcd45b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc80c3fa0 con 0x7effc80a5510 2023-11-15T02:57:09.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.479+0000 7effcd45b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc8159630 con 0x7effc8152780 2023-11-15T02:57:09.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.479+0000 7effcd45b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc80c30b0 con 0x7effc80c65d0 2023-11-15T02:57:09.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.479+0000 7effc6ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7effc8152780 0x7effc8159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:09.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.479+0000 7effc6ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7effc8152780 0x7effc8159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49820/0 (socket says 172.21.15.121:49820) 2023-11-15T02:57:09.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.479+0000 7effc6ffd700 1 -- 172.21.15.121:0/45890100 learned_addr learned my addr 172.21.15.121:0/45890100 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:09.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.479+0000 7effbe7fc700 1 --2- 172.21.15.121:0/45890100 >> v2:172.21.15.190:3300/0 conn(0x7effc80c65d0 0x7effc80c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:09.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.479+0000 7effc6ffd700 1 -- 172.21.15.121:0/45890100 >> v2:172.21.15.190:3300/0 conn(0x7effc80c65d0 msgr2=0x7effc80c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:09.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.479+0000 7effc6ffd700 1 --2- 172.21.15.121:0/45890100 >> v2:172.21.15.190:3300/0 conn(0x7effc80c65d0 0x7effc80c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.479+0000 7effc6ffd700 1 -- 172.21.15.121:0/45890100 >> v1:172.21.15.190:6789/0 conn(0x7effc80a5510 legacy=0x7effc80a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:09.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.479+0000 7effc6ffd700 1 -- 172.21.15.121:0/45890100 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc80c3390 con 0x7effc8152780 2023-11-15T02:57:09.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.479+0000 7effc6ffd700 1 --2- 172.21.15.121:0/45890100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc8152780 0x7effc8159140 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7effc8063d40 tx=0x7effb8009840).ready entity=mon.0 client_cookie=1970182db6c6e30b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:09.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.479+0000 7effc67fc700 1 -- 172.21.15.121:0/45890100 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb800e070 con 0x7effc8152780 2023-11-15T02:57:09.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.479+0000 7effc67fc700 1 -- 172.21.15.121:0/45890100 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb800ba50 con 0x7effc8152780 2023-11-15T02:57:09.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.479+0000 7effc67fc700 1 -- 172.21.15.121:0/45890100 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb800bdd0 con 0x7effc8152780 2023-11-15T02:57:09.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.483+0000 7effcd45b700 1 -- 172.21.15.121:0/45890100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc8152780 msgr2=0x7effc8159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:09.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.483+0000 7effcd45b700 1 --2- 172.21.15.121:0/45890100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc8152780 0x7effc8159140 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7effc8063d40 tx=0x7effb8009840).stop 2023-11-15T02:57:09.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.483+0000 7effcd45b700 1 -- 172.21.15.121:0/45890100 shutdown_connections 2023-11-15T02:57:09.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.483+0000 7effcd45b700 1 --2- 172.21.15.121:0/45890100 >> v2:172.21.15.190:3300/0 conn(0x7effc80c65d0 0x7effc80c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.483+0000 7effcd45b700 1 --2- 172.21.15.121:0/45890100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc8152780 0x7effc8159140 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.483+0000 7effcd45b700 1 -- 172.21.15.121:0/45890100 >> 172.21.15.121:0/45890100 conn(0x7effc8062190 msgr2=0x7effc80625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:09.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.483+0000 7effcd45b700 1 -- 172.21.15.121:0/45890100 shutdown_connections 2023-11-15T02:57:09.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.483+0000 7effcd45b700 1 -- 172.21.15.121:0/45890100 wait complete. 2023-11-15T02:57:09.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.483+0000 7effcd45b700 1 Processor -- start 2023-11-15T02:57:09.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.483+0000 7effcd45b700 1 -- start start 2023-11-15T02:57:09.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effcd45b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc80a5510 0x7effc813fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:09.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effcd45b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effc80c65d0 0x7effc813ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:09.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effcd45b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effc8152780 0x7effc80a7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:09.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effcd45b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc8140380 con 0x7effc80a5510 2023-11-15T02:57:09.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effcd45b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc813fa00 con 0x7effc80c65d0 2023-11-15T02:57:09.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effcd45b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc813edf0 con 0x7effc8152780 2023-11-15T02:57:09.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effc6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc80a5510 0x7effc813fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:09.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effc6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc80a5510 0x7effc813fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49834/0 (socket says 172.21.15.121:49834) 2023-11-15T02:57:09.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effc6ffd700 1 -- 172.21.15.121:0/2724729292 learned_addr learned my addr 172.21.15.121:0/2724729292 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:09.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effbe7fc700 1 --2- 172.21.15.121:0/2724729292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effc80c65d0 0x7effc813ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:09.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effc77fe700 1 --2- 172.21.15.121:0/2724729292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effc8152780 0x7effc80a7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:09.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effc6ffd700 1 -- 172.21.15.121:0/2724729292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effc8152780 msgr2=0x7effc80a7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:09.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effc6ffd700 1 --2- 172.21.15.121:0/2724729292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effc8152780 0x7effc80a7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effc6ffd700 1 -- 172.21.15.121:0/2724729292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effc80c65d0 msgr2=0x7effc813ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:09.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effc6ffd700 1 --2- 172.21.15.121:0/2724729292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effc80c65d0 0x7effc813ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effc6ffd700 1 -- 172.21.15.121:0/2724729292 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb8009ae0 con 0x7effc80a5510 2023-11-15T02:57:09.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effc6ffd700 1 --2- 172.21.15.121:0/2724729292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc80a5510 0x7effc813fe90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7effb8009fd0 tx=0x7effb8009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:09.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effbf7fe700 1 -- 172.21.15.121:0/2724729292 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb800bbc0 con 0x7effc80a5510 2023-11-15T02:57:09.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effbf7fe700 1 -- 172.21.15.121:0/2724729292 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb800bd20 con 0x7effc80a5510 2023-11-15T02:57:09.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effcd45b700 1 -- 172.21.15.121:0/2724729292 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc813f0d0 con 0x7effc80a5510 2023-11-15T02:57:09.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effbf7fe700 1 -- 172.21.15.121:0/2724729292 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb801c630 con 0x7effc80a5510 2023-11-15T02:57:09.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.487+0000 7effcd45b700 1 -- 172.21.15.121:0/2724729292 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc813f690 con 0x7effc80a5510 2023-11-15T02:57:09.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.491+0000 7effcd45b700 1 -- 172.21.15.121:0/2724729292 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc806c150 con 0x7effc80a5510 2023-11-15T02:57:09.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.491+0000 7effbf7fe700 1 -- 172.21.15.121:0/2724729292 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7effb801ca60 con 0x7effc80a5510 2023-11-15T02:57:09.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.491+0000 7effbf7fe700 1 --2- 172.21.15.121:0/2724729292 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effa805a510 0x7effa805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:09.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.491+0000 7effbf7fe700 1 -- 172.21.15.121:0/2724729292 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7effb8076020 con 0x7effc80a5510 2023-11-15T02:57:09.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.491+0000 7effbe7fc700 1 --2- 172.21.15.121:0/2724729292 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effa805a510 0x7effa805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:09.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.491+0000 7effbe7fc700 1 --2- 172.21.15.121:0/2724729292 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effa805a510 0x7effa805c9d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7effc8151300 tx=0x7effb0008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:09.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.499+0000 7effbf7fe700 1 -- 172.21.15.121:0/2724729292 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7effb8049800 con 0x7effc80a5510 2023-11-15T02:57:09.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.691+0000 7effcd45b700 1 -- 172.21.15.121:0/2724729292 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effc80c11b0 con 0x7effc80a5510 2023-11-15T02:57:09.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.695+0000 7effbf7fe700 1 -- 172.21.15.121:0/2724729292 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4489 (secure 0 0 0) 0x7effb804ce20 con 0x7effc80a5510 2023-11-15T02:57:09.699 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:09.699 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/802010010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":802010010},{"type":"v1","addr":"172.21.15.190:6835","nonce":802010010}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3991611070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3991611070},{"type":"v1","addr":"172.21.15.121:6836","nonce":3991611070}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3514688785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3514688785},{"type":"v1","addr":"172.21.15.121:6837","nonce":3514688785}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:56:51.695751+0000","modified":"2023-11-15T02:56:54.838464+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6833/742721767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":742721767},{"type":"v1","addr":"172.21.15.190:6833","nonce":742721767}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-15T02:57:09.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.695+0000 7effcd45b700 1 -- 172.21.15.121:0/2724729292 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effa805a510 msgr2=0x7effa805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:09.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.695+0000 7effcd45b700 1 --2- 172.21.15.121:0/2724729292 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effa805a510 0x7effa805c9d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7effc8151300 tx=0x7effb0008040).stop 2023-11-15T02:57:09.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.695+0000 7effcd45b700 1 -- 172.21.15.121:0/2724729292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc80a5510 msgr2=0x7effc813fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:09.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.695+0000 7effcd45b700 1 --2- 172.21.15.121:0/2724729292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc80a5510 0x7effc813fe90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7effb8009fd0 tx=0x7effb8009840).stop 2023-11-15T02:57:09.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.695+0000 7effcd45b700 1 -- 172.21.15.121:0/2724729292 shutdown_connections 2023-11-15T02:57:09.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.695+0000 7effcd45b700 1 --2- 172.21.15.121:0/2724729292 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effa805a510 0x7effa805c9d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.695+0000 7effcd45b700 1 --2- 172.21.15.121:0/2724729292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effc8152780 0x7effc80a7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.695+0000 7effcd45b700 1 --2- 172.21.15.121:0/2724729292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effc80c65d0 0x7effc813ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.695+0000 7effcd45b700 1 --2- 172.21.15.121:0/2724729292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc80a5510 0x7effc813fe90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.695+0000 7effcd45b700 1 -- 172.21.15.121:0/2724729292 >> 172.21.15.121:0/2724729292 conn(0x7effc8062190 msgr2=0x7effc8066370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:09.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.695+0000 7effcd45b700 1 -- 172.21.15.121:0/2724729292 shutdown_connections 2023-11-15T02:57:09.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.695+0000 7effcd45b700 1 -- 172.21.15.121:0/2724729292 wait complete. 2023-11-15T02:57:09.704 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 8 2023-11-15T02:57:09.722 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:57:09.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.883+0000 7f3729ff9700 1 Processor -- start 2023-11-15T02:57:09.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.883+0000 7f3729ff9700 1 -- start start 2023-11-15T02:57:09.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.883+0000 7f3729ff9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f37240a9830 0x7f37240a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:09.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.883+0000 7f3729ff9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37240657f0 con 0x7f3724151870 2023-11-15T02:57:09.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.883+0000 7f3729ff9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3724065ad0 con 0x7f37240c30e0 2023-11-15T02:57:09.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.883+0000 7f3729ff9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37240aa260 con 0x7f37240a9830 2023-11-15T02:57:09.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.883+0000 7f3728ff7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3724151870 0x7f3724151c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59704/0 (socket says 172.21.15.121:59704) 2023-11-15T02:57:09.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.883+0000 7f3728ff7700 1 -- 172.21.15.121:0/1771439597 learned_addr learned my addr 172.21.15.121:0/1771439597 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:09.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.883+0000 7f371bfff700 1 --2- 172.21.15.121:0/1771439597 >> v2:172.21.15.190:3300/0 conn(0x7f37240a9830 0x7f37240a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:09.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.883+0000 7f371b7fe700 1 -- 172.21.15.121:0/1771439597 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1847142652 0 0) 0x7f37240657f0 con 0x7f3724151870 2023-11-15T02:57:09.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.883+0000 7f371b7fe700 1 -- 172.21.15.121:0/1771439597 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f370c0033f0 con 0x7f3724151870 2023-11-15T02:57:09.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.883+0000 7f371b7fe700 1 -- 172.21.15.121:0/1771439597 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2642298005 0 0) 0x7f3724065ad0 con 0x7f37240c30e0 2023-11-15T02:57:09.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.887+0000 7f371b7fe700 1 -- 172.21.15.121:0/1771439597 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37240657f0 con 0x7f37240c30e0 2023-11-15T02:57:09.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.887+0000 7f371b7fe700 1 -- 172.21.15.121:0/1771439597 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3329513293 0 0) 0x7f370c0033f0 con 0x7f3724151870 2023-11-15T02:57:09.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.887+0000 7f371bfff700 1 -- 172.21.15.121:0/1771439597 >> v1:172.21.15.190:6789/0 conn(0x7f37240c30e0 legacy=0x7f3724065f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:09.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.887+0000 7f371bfff700 1 -- 172.21.15.121:0/1771439597 >> v1:172.21.15.121:6789/0 conn(0x7f3724151870 legacy=0x7f3724151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:09.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.887+0000 7f371bfff700 1 -- 172.21.15.121:0/1771439597 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37240aa540 con 0x7f37240a9830 2023-11-15T02:57:09.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.887+0000 7f371bfff700 1 --2- 172.21.15.121:0/1771439597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37240a9830 0x7f37240a9c70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f3714009760 tx=0x7f3714009060).ready entity=mon.1 client_cookie=cae2f6f349fd1bd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:09.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.887+0000 7f371b7fe700 1 -- 172.21.15.121:0/1771439597 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37200017f0 con 0x7f37240a9830 2023-11-15T02:57:09.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.887+0000 7f371b7fe700 1 -- 172.21.15.121:0/1771439597 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3714011400 con 0x7f37240a9830 2023-11-15T02:57:09.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.887+0000 7f371b7fe700 1 -- 172.21.15.121:0/1771439597 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3714011780 con 0x7f37240a9830 2023-11-15T02:57:09.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.887+0000 7f3729ff9700 1 -- 172.21.15.121:0/1771439597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37240a9830 msgr2=0x7f37240a9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:09.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.887+0000 7f3729ff9700 1 --2- 172.21.15.121:0/1771439597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37240a9830 0x7f37240a9c70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f3714009760 tx=0x7f3714009060).stop 2023-11-15T02:57:09.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.887+0000 7f3729ff9700 1 -- 172.21.15.121:0/1771439597 shutdown_connections 2023-11-15T02:57:09.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.887+0000 7f3729ff9700 1 --2- 172.21.15.121:0/1771439597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37240a9830 0x7f37240a9c70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.887+0000 7f3729ff9700 1 -- 172.21.15.121:0/1771439597 >> 172.21.15.121:0/1771439597 conn(0x7f3724062190 msgr2=0x7f3724062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:09.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.887+0000 7f3729ff9700 1 -- 172.21.15.121:0/1771439597 shutdown_connections 2023-11-15T02:57:09.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.887+0000 7f3729ff9700 1 -- 172.21.15.121:0/1771439597 wait complete. 2023-11-15T02:57:09.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.891+0000 7f3729ff9700 1 Processor -- start 2023-11-15T02:57:09.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.891+0000 7f3729ff9700 1 -- start start 2023-11-15T02:57:09.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.891+0000 7f3729ff9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37240a9830 0x7f37240bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:09.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.891+0000 7f3729ff9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37240c30e0 0x7f37241644d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:09.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.891+0000 7f3729ff9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3724151870 0x7f37241649c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:09.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.891+0000 7f3729ff9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3724163430 con 0x7f3724151870 2023-11-15T02:57:09.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.891+0000 7f3729ff9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3724163710 con 0x7f37240a9830 2023-11-15T02:57:09.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.891+0000 7f3729ff9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37241639f0 con 0x7f37240c30e0 2023-11-15T02:57:09.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.891+0000 7f37297f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3724151870 0x7f37241649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:09.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.891+0000 7f37297f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3724151870 0x7f37241649c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49842/0 (socket says 172.21.15.121:49842) 2023-11-15T02:57:09.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.891+0000 7f37297f8700 1 -- 172.21.15.121:0/1021722812 learned_addr learned my addr 172.21.15.121:0/1021722812 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:09.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.891+0000 7f371bfff700 1 --2- 172.21.15.121:0/1021722812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37240c30e0 0x7f37241644d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:09.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.891+0000 7f3728ff7700 1 --2- 172.21.15.121:0/1021722812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37240a9830 0x7f37240bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:09.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.895+0000 7f37297f8700 1 -- 172.21.15.121:0/1021722812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37240c30e0 msgr2=0x7f37241644d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:09.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.895+0000 7f37297f8700 1 --2- 172.21.15.121:0/1021722812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37240c30e0 0x7f37241644d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.895+0000 7f37297f8700 1 -- 172.21.15.121:0/1021722812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37240a9830 msgr2=0x7f37240bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:09.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.895+0000 7f37297f8700 1 --2- 172.21.15.121:0/1021722812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37240a9830 0x7f37240bc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:09.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.895+0000 7f37297f8700 1 -- 172.21.15.121:0/1021722812 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37140092a0 con 0x7f3724151870 2023-11-15T02:57:09.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.895+0000 7f3728ff7700 1 --2- 172.21.15.121:0/1021722812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37240a9830 0x7f37240bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:09.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.895+0000 7f37297f8700 1 --2- 172.21.15.121:0/1021722812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3724151870 0x7f37241649c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f371c00c4c0 tx=0x7f371c007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:09.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.895+0000 7f3718ff9700 1 -- 172.21.15.121:0/1021722812 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c00f470 con 0x7f3724151870 2023-11-15T02:57:09.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.895+0000 7f3718ff9700 1 -- 172.21.15.121:0/1021722812 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371c00f5d0 con 0x7f3724151870 2023-11-15T02:57:09.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.895+0000 7f3718ff9700 1 -- 172.21.15.121:0/1021722812 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c00f950 con 0x7f3724151870 2023-11-15T02:57:09.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.895+0000 7f3729ff9700 1 -- 172.21.15.121:0/1021722812 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3724163cd0 con 0x7f3724151870 2023-11-15T02:57:09.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.895+0000 7f3729ff9700 1 -- 172.21.15.121:0/1021722812 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3724164eb0 con 0x7f3724151870 2023-11-15T02:57:09.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.895+0000 7f3729ff9700 1 -- 172.21.15.121:0/1021722812 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37241641d0 con 0x7f3724151870 2023-11-15T02:57:09.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.895+0000 7f3718ff9700 1 -- 172.21.15.121:0/1021722812 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f371c00fd40 con 0x7f3724151870 2023-11-15T02:57:09.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.899+0000 7f3718ff9700 1 --2- 172.21.15.121:0/1021722812 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f371005a510 0x7f371005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:09.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.899+0000 7f3718ff9700 1 -- 172.21.15.121:0/1021722812 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f371c073c20 con 0x7f3724151870 2023-11-15T02:57:09.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.899+0000 7f3728ff7700 1 --2- 172.21.15.121:0/1021722812 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f371005a510 0x7f371005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:09.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.899+0000 7f3728ff7700 1 --2- 172.21.15.121:0/1021722812 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f371005a510 0x7f371005c9d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f3720001920 tx=0x7f3720005a40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:09.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:09.903+0000 7f3718ff9700 1 -- 172.21.15.121:0/1021722812 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f371c0474b0 con 0x7f3724151870 2023-11-15T02:57:09.979 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T02:57:09.985 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T02:57:10.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.067+0000 7f3729ff9700 1 -- 172.21.15.121:0/1021722812 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f372406bcf0 con 0x7f3724151870 2023-11-15T02:57:10.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.071+0000 7f3718ff9700 1 -- 172.21.15.121:0/1021722812 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f371c017070 con 0x7f3724151870 2023-11-15T02:57:10.076 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:10.076 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":21,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:52.702668+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:56:49.726353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-15T02:56:50.688098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:57:10.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.075+0000 7f3729ff9700 1 -- 172.21.15.121:0/1021722812 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f371005a510 msgr2=0x7f371005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:10.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.075+0000 7f3729ff9700 1 --2- 172.21.15.121:0/1021722812 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f371005a510 0x7f371005c9d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f3720001920 tx=0x7f3720005a40).stop 2023-11-15T02:57:10.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.075+0000 7f3729ff9700 1 -- 172.21.15.121:0/1021722812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3724151870 msgr2=0x7f37241649c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:10.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.075+0000 7f3729ff9700 1 --2- 172.21.15.121:0/1021722812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3724151870 0x7f37241649c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f371c00c4c0 tx=0x7f371c007c00).stop 2023-11-15T02:57:10.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.075+0000 7f3729ff9700 1 -- 172.21.15.121:0/1021722812 shutdown_connections 2023-11-15T02:57:10.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.075+0000 7f3729ff9700 1 --2- 172.21.15.121:0/1021722812 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f371005a510 0x7f371005c9d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:10.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.075+0000 7f3729ff9700 1 --2- 172.21.15.121:0/1021722812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3724151870 0x7f37241649c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:10.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.075+0000 7f3729ff9700 1 --2- 172.21.15.121:0/1021722812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37240c30e0 0x7f37241644d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:10.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.075+0000 7f3729ff9700 1 --2- 172.21.15.121:0/1021722812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37240a9830 0x7f37240bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:10.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.075+0000 7f3729ff9700 1 -- 172.21.15.121:0/1021722812 >> 172.21.15.121:0/1021722812 conn(0x7f3724062190 msgr2=0x7f3724067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:10.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.075+0000 7f3729ff9700 1 -- 172.21.15.121:0/1021722812 shutdown_connections 2023-11-15T02:57:10.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.075+0000 7f3729ff9700 1 -- 172.21.15.121:0/1021722812 wait complete. 2023-11-15T02:57:10.100 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:57:10.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.299+0000 7f313ba1b700 1 Processor -- start 2023-11-15T02:57:10.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.299+0000 7f313ba1b700 1 -- start start 2023-11-15T02:57:10.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.303+0000 7f313ba1b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f31340a5ed0 0x7f3134155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:10.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.303+0000 7f313ba1b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31340c4c30 con 0x7f3134154ca0 2023-11-15T02:57:10.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.303+0000 7f313ba1b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31340a6f50 con 0x7f31341562f0 2023-11-15T02:57:10.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.303+0000 7f313ba1b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31340a8280 con 0x7f31340a5ed0 2023-11-15T02:57:10.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.303+0000 7f31397b7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3134154ca0 0x7f31340a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59716/0 (socket says 172.21.15.121:59716) 2023-11-15T02:57:10.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.303+0000 7f3138fb6700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f31341562f0 0x7f31340a57c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:58170/0 (socket says 172.21.15.121:58170) 2023-11-15T02:57:10.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.303+0000 7f31397b7700 1 -- 172.21.15.121:0/1535709713 learned_addr learned my addr 172.21.15.121:0/1535709713 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:10.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.303+0000 7f3139fb8700 1 --2- 172.21.15.121:0/1535709713 >> v2:172.21.15.190:3300/0 conn(0x7f31340a5ed0 0x7f3134155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:10.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.303+0000 7f312bfff700 1 -- 172.21.15.121:0/1535709713 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3348252893 0 0) 0x7f31340c4c30 con 0x7f3134154ca0 2023-11-15T02:57:10.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.303+0000 7f312bfff700 1 -- 172.21.15.121:0/1535709713 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31200033f0 con 0x7f3134154ca0 2023-11-15T02:57:10.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.303+0000 7f312bfff700 1 -- 172.21.15.121:0/1535709713 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 437720902 0 0) 0x7f31340a6f50 con 0x7f31341562f0 2023-11-15T02:57:10.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.303+0000 7f312bfff700 1 -- 172.21.15.121:0/1535709713 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31340c4c30 con 0x7f31341562f0 2023-11-15T02:57:10.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.307+0000 7f312bfff700 1 -- 172.21.15.121:0/1535709713 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3229003957 0 0) 0x7f31200033f0 con 0x7f3134154ca0 2023-11-15T02:57:10.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.307+0000 7f3139fb8700 1 -- 172.21.15.121:0/1535709713 >> v1:172.21.15.190:6789/0 conn(0x7f31341562f0 legacy=0x7f31340a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:10.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.307+0000 7f3139fb8700 1 -- 172.21.15.121:0/1535709713 >> v1:172.21.15.121:6789/0 conn(0x7f3134154ca0 legacy=0x7f31340a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:10.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.307+0000 7f3139fb8700 1 -- 172.21.15.121:0/1535709713 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31340a7230 con 0x7f31340a5ed0 2023-11-15T02:57:10.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.307+0000 7f3139fb8700 1 --2- 172.21.15.121:0/1535709713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31340a5ed0 0x7f3134155080 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f312c009e60 tx=0x7f312c009780).ready entity=mon.1 client_cookie=ef17e9535952f406 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:10.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.307+0000 7f312bfff700 1 -- 172.21.15.121:0/1535709713 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c00e070 con 0x7f31340a5ed0 2023-11-15T02:57:10.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.307+0000 7f312bfff700 1 -- 172.21.15.121:0/1535709713 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312c00b7a0 con 0x7f31340a5ed0 2023-11-15T02:57:10.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.307+0000 7f312bfff700 1 -- 172.21.15.121:0/1535709713 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3130001470 con 0x7f31340a5ed0 2023-11-15T02:57:10.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.307+0000 7f313ba1b700 1 -- 172.21.15.121:0/1535709713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31340a5ed0 msgr2=0x7f3134155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:10.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.307+0000 7f313ba1b700 1 --2- 172.21.15.121:0/1535709713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31340a5ed0 0x7f3134155080 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f312c009e60 tx=0x7f312c009780).stop 2023-11-15T02:57:10.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.307+0000 7f313ba1b700 1 -- 172.21.15.121:0/1535709713 shutdown_connections 2023-11-15T02:57:10.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.307+0000 7f313ba1b700 1 --2- 172.21.15.121:0/1535709713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31340a5ed0 0x7f3134155080 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:10.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.307+0000 7f313ba1b700 1 -- 172.21.15.121:0/1535709713 >> 172.21.15.121:0/1535709713 conn(0x7f3134062190 msgr2=0x7f31340625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:10.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.307+0000 7f313ba1b700 1 -- 172.21.15.121:0/1535709713 shutdown_connections 2023-11-15T02:57:10.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.307+0000 7f313ba1b700 1 -- 172.21.15.121:0/1535709713 wait complete. 2023-11-15T02:57:10.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.311+0000 7f313ba1b700 1 Processor -- start 2023-11-15T02:57:10.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.311+0000 7f313ba1b700 1 -- start start 2023-11-15T02:57:10.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f313ba1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31340a5ed0 0x7f31340a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:10.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f313ba1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3134154ca0 0x7f31340ab470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:10.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f313ba1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31341562f0 0x7f31340ab960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:10.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f313ba1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3134167730 con 0x7f3134154ca0 2023-11-15T02:57:10.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f313ba1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31340ab190 con 0x7f31341562f0 2023-11-15T02:57:10.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f313ba1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31340abf30 con 0x7f31340a5ed0 2023-11-15T02:57:10.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f3138fb6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3134154ca0 0x7f31340ab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:10.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f3138fb6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3134154ca0 0x7f31340ab470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49852/0 (socket says 172.21.15.121:49852) 2023-11-15T02:57:10.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f3138fb6700 1 -- 172.21.15.121:0/3723310780 learned_addr learned my addr 172.21.15.121:0/3723310780 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:10.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f31397b7700 1 --2- 172.21.15.121:0/3723310780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31340a5ed0 0x7f31340a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:10.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f3139fb8700 1 --2- 172.21.15.121:0/3723310780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31341562f0 0x7f31340ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:10.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f3138fb6700 1 -- 172.21.15.121:0/3723310780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31340a5ed0 msgr2=0x7f31340a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:10.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f3138fb6700 1 --2- 172.21.15.121:0/3723310780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31340a5ed0 0x7f31340a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:10.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f3138fb6700 1 -- 172.21.15.121:0/3723310780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31341562f0 msgr2=0x7f31340ab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:10.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f3138fb6700 1 --2- 172.21.15.121:0/3723310780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31341562f0 0x7f31340ab960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:10.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f3138fb6700 1 -- 172.21.15.121:0/3723310780 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c009a20 con 0x7f3134154ca0 2023-11-15T02:57:10.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f3138fb6700 1 --2- 172.21.15.121:0/3723310780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3134154ca0 0x7f31340ab470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f312400c420 tx=0x7f3124007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:10.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f31297fa700 1 -- 172.21.15.121:0/3723310780 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31240070d0 con 0x7f3134154ca0 2023-11-15T02:57:10.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f31297fa700 1 -- 172.21.15.121:0/3723310780 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3124007230 con 0x7f3134154ca0 2023-11-15T02:57:10.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f31297fa700 1 -- 172.21.15.121:0/3723310780 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31240056e0 con 0x7f3134154ca0 2023-11-15T02:57:10.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f313ba1b700 1 -- 172.21.15.121:0/3723310780 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3134167940 con 0x7f3134154ca0 2023-11-15T02:57:10.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.315+0000 7f313ba1b700 1 -- 172.21.15.121:0/3723310780 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3134167f00 con 0x7f3134154ca0 2023-11-15T02:57:10.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.319+0000 7f313ba1b700 1 -- 172.21.15.121:0/3723310780 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f313406c150 con 0x7f3134154ca0 2023-11-15T02:57:10.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.319+0000 7f31297fa700 1 -- 172.21.15.121:0/3723310780 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3124005a50 con 0x7f3134154ca0 2023-11-15T02:57:10.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.319+0000 7f31297fa700 1 --2- 172.21.15.121:0/3723310780 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f311805a510 0x7f311805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:10.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.319+0000 7f31297fa700 1 -- 172.21.15.121:0/3723310780 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3124073c20 con 0x7f3134154ca0 2023-11-15T02:57:10.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.319+0000 7f31397b7700 1 --2- 172.21.15.121:0/3723310780 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f311805a510 0x7f311805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:10.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.323+0000 7f31397b7700 1 --2- 172.21.15.121:0/3723310780 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f311805a510 0x7f311805c9d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f31300015a0 tx=0x7f3130001ad0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:10.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.327+0000 7f31297fa700 1 -- 172.21.15.121:0/3723310780 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f31240474b0 con 0x7f3134154ca0 2023-11-15T02:57:10.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.523+0000 7f313ba1b700 1 -- 172.21.15.121:0/3723310780 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31340c4500 con 0x7f3134154ca0 2023-11-15T02:57:10.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.523+0000 7f31297fa700 1 -- 172.21.15.121:0/3723310780 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4489 (secure 0 0 0) 0x7f3124007d50 con 0x7f3134154ca0 2023-11-15T02:57:10.530 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:10.531 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/802010010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":802010010},{"type":"v1","addr":"172.21.15.190:6835","nonce":802010010}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3991611070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3991611070},{"type":"v1","addr":"172.21.15.121:6836","nonce":3991611070}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3514688785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3514688785},{"type":"v1","addr":"172.21.15.121:6837","nonce":3514688785}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:56:51.695751+0000","modified":"2023-11-15T02:56:54.838464+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6833/742721767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":742721767},{"type":"v1","addr":"172.21.15.190:6833","nonce":742721767}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-15T02:57:10.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.527+0000 7f313ba1b700 1 -- 172.21.15.121:0/3723310780 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f311805a510 msgr2=0x7f311805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:10.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.527+0000 7f313ba1b700 1 --2- 172.21.15.121:0/3723310780 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f311805a510 0x7f311805c9d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f31300015a0 tx=0x7f3130001ad0).stop 2023-11-15T02:57:10.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.527+0000 7f313ba1b700 1 -- 172.21.15.121:0/3723310780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3134154ca0 msgr2=0x7f31340ab470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:10.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.527+0000 7f313ba1b700 1 --2- 172.21.15.121:0/3723310780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3134154ca0 0x7f31340ab470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f312400c420 tx=0x7f3124007c10).stop 2023-11-15T02:57:10.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.531+0000 7f313ba1b700 1 -- 172.21.15.121:0/3723310780 shutdown_connections 2023-11-15T02:57:10.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.531+0000 7f313ba1b700 1 --2- 172.21.15.121:0/3723310780 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f311805a510 0x7f311805c9d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:10.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.531+0000 7f313ba1b700 1 --2- 172.21.15.121:0/3723310780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31341562f0 0x7f31340ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:10.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.531+0000 7f313ba1b700 1 --2- 172.21.15.121:0/3723310780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3134154ca0 0x7f31340ab470 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:10.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.531+0000 7f313ba1b700 1 --2- 172.21.15.121:0/3723310780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31340a5ed0 0x7f31340a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:10.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.531+0000 7f313ba1b700 1 -- 172.21.15.121:0/3723310780 >> 172.21.15.121:0/3723310780 conn(0x7f3134062190 msgr2=0x7f31340c6050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:10.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.531+0000 7f313ba1b700 1 -- 172.21.15.121:0/3723310780 shutdown_connections 2023-11-15T02:57:10.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:10.531+0000 7f313ba1b700 1 -- 172.21.15.121:0/3723310780 wait complete. 2023-11-15T02:57:10.537 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 8 2023-11-15T02:57:10.958 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T02:56:51.695751+0000', 'modified': '2023-11-15T02:56:54.838464+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4270}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4270': {'gid': 4270, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6833/742721767', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 742721767}, {'type': 'v1', 'addr': '172.21.15.190:6833', 'nonce': 742721767}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T02:57:10.959 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T02:57:10.959 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-15T02:57:11.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.123+0000 7fae40ef1700 1 Processor -- start 2023-11-15T02:57:11.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.123+0000 7fae40ef1700 1 -- start start 2023-11-15T02:57:11.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.123+0000 7fae40ef1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae3c151920 0x7fae3c0bb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.123+0000 7fae40ef1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fae3c067150 0x7fae3c0aba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.123+0000 7fae40ef1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3c155de0 con 0x7fae3c156270 2023-11-15T02:57:11.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.123+0000 7fae40ef1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3c0c4060 con 0x7fae3c067150 2023-11-15T02:57:11.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.123+0000 7fae40ef1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0c3aa0 con 0x7fae3c151920 2023-11-15T02:57:11.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.123+0000 7fae33fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fae3c067150 0x7fae3c0aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.123+0000 7fae33fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fae3c067150 0x7fae3c0aba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49864/0 (socket says 172.21.15.121:49864) 2023-11-15T02:57:11.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.123+0000 7fae33fff700 1 -- 172.21.15.121:0/3825123723 learned_addr learned my addr 172.21.15.121:0/3825123723 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:11.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.123+0000 7fae3a59c700 1 --2- 172.21.15.121:0/3825123723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae3c151920 0x7fae3c0bb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.123+0000 7fae33fff700 1 -- 172.21.15.121:0/3825123723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae3c151920 msgr2=0x7fae3c0bb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:11.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.123+0000 7fae33fff700 1 --2- 172.21.15.121:0/3825123723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae3c151920 0x7fae3c0bb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.123+0000 7fae33fff700 1 -- 172.21.15.121:0/3825123723 >> v1:172.21.15.121:6789/0 conn(0x7fae3c156270 legacy=0x7fae3c0658a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:11.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.123+0000 7fae33fff700 1 -- 172.21.15.121:0/3825123723 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c0c3d80 con 0x7fae3c067150 2023-11-15T02:57:11.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.123+0000 7fae33fff700 1 --2- 172.21.15.121:0/3825123723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae3c067150 0x7fae3c0aba40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fae24006fd0 tx=0x7fae240090f0).ready entity=mon.0 client_cookie=94b59e6c6c93b4f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:11.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.127+0000 7fae39d9b700 1 -- 172.21.15.121:0/3825123723 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2400b600 con 0x7fae3c067150 2023-11-15T02:57:11.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.127+0000 7fae39d9b700 1 -- 172.21.15.121:0/3825123723 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2400b760 con 0x7fae3c067150 2023-11-15T02:57:11.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.127+0000 7fae39d9b700 1 -- 172.21.15.121:0/3825123723 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2400bae0 con 0x7fae3c067150 2023-11-15T02:57:11.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.127+0000 7fae40ef1700 1 -- 172.21.15.121:0/3825123723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae3c067150 msgr2=0x7fae3c0aba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:11.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.127+0000 7fae40ef1700 1 --2- 172.21.15.121:0/3825123723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae3c067150 0x7fae3c0aba40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fae24006fd0 tx=0x7fae240090f0).stop 2023-11-15T02:57:11.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.127+0000 7fae40ef1700 1 -- 172.21.15.121:0/3825123723 shutdown_connections 2023-11-15T02:57:11.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.127+0000 7fae40ef1700 1 --2- 172.21.15.121:0/3825123723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae3c067150 0x7fae3c0aba40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.127+0000 7fae40ef1700 1 --2- 172.21.15.121:0/3825123723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae3c151920 0x7fae3c0bb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.127+0000 7fae40ef1700 1 -- 172.21.15.121:0/3825123723 >> 172.21.15.121:0/3825123723 conn(0x7fae3c067a60 msgr2=0x7fae3c067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:11.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.127+0000 7fae40ef1700 1 -- 172.21.15.121:0/3825123723 shutdown_connections 2023-11-15T02:57:11.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.127+0000 7fae40ef1700 1 -- 172.21.15.121:0/3825123723 wait complete. 2023-11-15T02:57:11.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.127+0000 7fae40ef1700 1 Processor -- start 2023-11-15T02:57:11.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae40ef1700 1 -- start start 2023-11-15T02:57:11.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae40ef1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae3c067150 0x7fae3c148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae40ef1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae3c151920 0x7fae3c0ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae40ef1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae3c156270 0x7fae3c13aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae40ef1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0aa810 con 0x7fae3c151920 2023-11-15T02:57:11.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae40ef1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c148710 con 0x7fae3c067150 2023-11-15T02:57:11.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae40ef1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0aaae0 con 0x7fae3c156270 2023-11-15T02:57:11.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae33fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae3c151920 0x7fae3c0ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae33fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae3c151920 0x7fae3c0ab5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49876/0 (socket says 172.21.15.121:49876) 2023-11-15T02:57:11.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae33fff700 1 -- 172.21.15.121:0/2725703857 learned_addr learned my addr 172.21.15.121:0/2725703857 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:11.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae3ad9d700 1 --2- 172.21.15.121:0/2725703857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae3c156270 0x7fae3c13aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae3a59c700 1 --2- 172.21.15.121:0/2725703857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae3c067150 0x7fae3c148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae33fff700 1 -- 172.21.15.121:0/2725703857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae3c156270 msgr2=0x7fae3c13aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:11.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae33fff700 1 --2- 172.21.15.121:0/2725703857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae3c156270 0x7fae3c13aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae33fff700 1 -- 172.21.15.121:0/2725703857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae3c067150 msgr2=0x7fae3c148150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:11.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae33fff700 1 --2- 172.21.15.121:0/2725703857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae3c067150 0x7fae3c148150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae33fff700 1 -- 172.21.15.121:0/2725703857 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae24009370 con 0x7fae3c151920 2023-11-15T02:57:11.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae3a59c700 1 --2- 172.21.15.121:0/2725703857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae3c067150 0x7fae3c148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:11.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae33fff700 1 --2- 172.21.15.121:0/2725703857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae3c151920 0x7fae3c0ab5c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fae24011000 tx=0x7fae24017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:11.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae327fc700 1 -- 172.21.15.121:0/2725703857 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2400b8d0 con 0x7fae3c151920 2023-11-15T02:57:11.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae327fc700 1 -- 172.21.15.121:0/2725703857 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2400ba30 con 0x7fae3c151920 2023-11-15T02:57:11.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae327fc700 1 -- 172.21.15.121:0/2725703857 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2401c670 con 0x7fae3c151920 2023-11-15T02:57:11.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.131+0000 7fae40ef1700 1 -- 172.21.15.121:0/2725703857 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c0aadc0 con 0x7fae3c151920 2023-11-15T02:57:11.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.135+0000 7fae40ef1700 1 -- 172.21.15.121:0/2725703857 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c13aef0 con 0x7fae3c151920 2023-11-15T02:57:11.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.135+0000 7fae327fc700 1 -- 172.21.15.121:0/2725703857 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fae2401ca60 con 0x7fae3c151920 2023-11-15T02:57:11.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.135+0000 7fae327fc700 1 --2- 172.21.15.121:0/2725703857 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fae1c05a510 0x7fae1c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.135+0000 7fae327fc700 1 -- 172.21.15.121:0/2725703857 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fae24076580 con 0x7fae3c151920 2023-11-15T02:57:11.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.139+0000 7fae3a59c700 1 --2- 172.21.15.121:0/2725703857 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fae1c05a510 0x7fae1c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.139+0000 7fae3a59c700 1 --2- 172.21.15.121:0/2725703857 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fae1c05a510 0x7fae1c05c9d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fae2c00bb60 tx=0x7fae2c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:11.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.139+0000 7fae4b2ab740 1 Processor -- start 2023-11-15T02:57:11.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.139+0000 7fae4b2ab740 1 -- start start 2023-11-15T02:57:11.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.139+0000 7fae4b2ab740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x278e300 0x27d55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.139+0000 7fae4b2ab740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x278d510 0x278e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.139+0000 7fae4b2ab740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27d3ab0 0x27d3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.139+0000 7fae4b2ab740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x270e410 con 0x278d510 2023-11-15T02:57:11.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.139+0000 7fae4b2ab740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x270e550 con 0x278e300 2023-11-15T02:57:11.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.139+0000 7fae4b2ab740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x270ea00 con 0x27d3ab0 2023-11-15T02:57:11.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae33fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x278d510 0x278e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae3ad9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27d3ab0 0x27d3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae3a59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x278e300 0x27d55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae33fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x278d510 0x278e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49892/0 (socket says 172.21.15.121:49892) 2023-11-15T02:57:11.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae33fff700 1 -- 172.21.15.121:0/3459665956 learned_addr learned my addr 172.21.15.121:0/3459665956 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:11.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae33fff700 1 -- 172.21.15.121:0/3459665956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27d3ab0 msgr2=0x27d3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:11.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae33fff700 1 --2- 172.21.15.121:0/3459665956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27d3ab0 0x27d3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae33fff700 1 -- 172.21.15.121:0/3459665956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x278e300 msgr2=0x27d55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:11.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae33fff700 1 --2- 172.21.15.121:0/3459665956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x278e300 0x27d55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae33fff700 1 -- 172.21.15.121:0/3459665956 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2707730 con 0x278d510 2023-11-15T02:57:11.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae33fff700 1 --2- 172.21.15.121:0/3459665956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x278d510 0x278e6e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fae2404d890 tx=0x7fae2400e000).ready entity=mon.0 client_cookie=82b90df27ba3713b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:11.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae1b7fe700 1 -- 172.21.15.121:0/3459665956 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2404d900 con 0x278d510 2023-11-15T02:57:11.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae1b7fe700 1 -- 172.21.15.121:0/3459665956 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae24006af0 con 0x278d510 2023-11-15T02:57:11.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae1b7fe700 1 -- 172.21.15.121:0/3459665956 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24074290 con 0x278d510 2023-11-15T02:57:11.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae4b2ab740 1 -- 172.21.15.121:0/3459665956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x278d510 msgr2=0x278e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:11.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/3459665956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x278d510 0x278e6e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fae2404d890 tx=0x7fae2400e000).stop 2023-11-15T02:57:11.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae4b2ab740 1 -- 172.21.15.121:0/3459665956 shutdown_connections 2023-11-15T02:57:11.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/3459665956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27d3ab0 0x27d3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/3459665956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x278d510 0x278e6e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/3459665956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x278e300 0x27d55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae4b2ab740 1 -- 172.21.15.121:0/3459665956 >> 172.21.15.121:0/3459665956 conn(0x256c550 msgr2=0x27d4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:11.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae4b2ab740 1 -- 172.21.15.121:0/3459665956 shutdown_connections 2023-11-15T02:57:11.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae4b2ab740 1 -- 172.21.15.121:0/3459665956 wait complete. 2023-11-15T02:57:11.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.143+0000 7fae4b2ab740 1 -- start start 2023-11-15T02:57:11.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae4b2ab740 1 Processor -- start 2023-11-15T02:57:11.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae4b2ab740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x278d510 0x27b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae4b2ab740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x278e300 0x27d3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae4b2ab740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27d3ab0 0x27d4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae4b2ab740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x26d9650 con 0x27d3ab0 2023-11-15T02:57:11.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae4b2ab740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x27d4910 con 0x278d510 2023-11-15T02:57:11.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae4b2ab740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x27e1100 con 0x278e300 2023-11-15T02:57:11.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae4b2ab740 10 client.0 ll_register_callbacks cb 0x2746020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:57:11.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae3ad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27d3ab0 0x27d4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae3ad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27d3ab0 0x27d4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49894/0 (socket says 172.21.15.121:49894) 2023-11-15T02:57:11.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae3ad9d700 1 -- 172.21.15.121:0/747638628 learned_addr learned my addr 172.21.15.121:0/747638628 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:11.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae33fff700 1 --2- 172.21.15.121:0/747638628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x278e300 0x27d3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae3a59c700 1 --2- 172.21.15.121:0/747638628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x278d510 0x27b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae3ad9d700 1 -- 172.21.15.121:0/747638628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x278e300 msgr2=0x27d3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:11.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae3ad9d700 1 --2- 172.21.15.121:0/747638628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x278e300 0x27d3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae3ad9d700 1 -- 172.21.15.121:0/747638628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x278d510 msgr2=0x27b8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:11.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae3ad9d700 1 --2- 172.21.15.121:0/747638628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x278d510 0x27b8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae3ad9d700 1 -- 172.21.15.121:0/747638628 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c13aef0 con 0x27d3ab0 2023-11-15T02:57:11.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae3ad9d700 1 --2- 172.21.15.121:0/747638628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27d3ab0 0x27d4420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fae34011650 tx=0x7fae34001460).ready entity=mon.0 client_cookie=5eda28638dbc40cd server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:11.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae18ff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:57:11.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae18ff9700 1 -- 172.21.15.121:0/747638628 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae340027e0 con 0x27d3ab0 2023-11-15T02:57:11.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae18ff9700 1 -- 172.21.15.121:0/747638628 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fae34006c30 con 0x27d3ab0 2023-11-15T02:57:11.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae4b2ab740 1 -- 172.21.15.121:0/747638628 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x27041a0 con 0x27d3ab0 2023-11-15T02:57:11.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae18ff9700 1 -- 172.21.15.121:0/747638628 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae34007030 con 0x27d3ab0 2023-11-15T02:57:11.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae4b2ab740 10 client.4647 fetch_fsmap learned FSMap version 8 2023-11-15T02:57:11.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.147+0000 7fae4b2ab740 1 -- 172.21.15.121:0/747638628 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x2702960 con 0x27d3ab0 2023-11-15T02:57:11.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.151+0000 7fae18ff9700 1 -- 172.21.15.121:0/747638628 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x27041a0 con 0x27d3ab0 2023-11-15T02:57:11.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.151+0000 7fae18ff9700 1 -- 172.21.15.121:0/747638628 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fae34007190 con 0x27d3ab0 2023-11-15T02:57:11.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.151+0000 7fae4b2ab740 10 client.4647 fetch_fsmap finished waiting for FSMap version 8 2023-11-15T02:57:11.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.151+0000 7fae4b2ab740 10 client.4647 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-15T02:57:11.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.151+0000 7fae4b2ab740 20 client.4647 populate_metadata read hostname 'smithi121' 2023-11-15T02:57:11.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.151+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/747638628 >> [v2:172.21.15.190:6832/742721767,v1:172.21.15.190:6833/742721767] conn(0x27e0610 0x27e09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.151+0000 7fae4b2ab740 4 client.4647 mds_command: new command op to 4270 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T02:57:11.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.151+0000 7fae4b2ab740 1 -- 172.21.15.121:0/747638628 --> [v2:172.21.15.190:6832/742721767,v1:172.21.15.190:6833/742721767] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x26fa6f0 con 0x27e0610 2023-11-15T02:57:11.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.151+0000 7fae3a59c700 1 --2- 172.21.15.121:0/747638628 >> [v2:172.21.15.190:6832/742721767,v1:172.21.15.190:6833/742721767] conn(0x27e0610 0x27e09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.151+0000 7fae18ff9700 10 client.4647 ms_handle_connect on v2:172.21.15.190:6832/742721767 2023-11-15T02:57:11.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.151+0000 7fae3a59c700 1 --2- 172.21.15.121:0/747638628 >> [v2:172.21.15.190:6832/742721767,v1:172.21.15.190:6833/742721767] conn(0x27e0610 0x27e09f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=e23e6170d1e87639 server_cookie=9aee52b8bd3afef1 in_seq=0 out_seq=0 2023-11-15T02:57:11.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.155+0000 7fae18ff9700 1 -- 172.21.15.121:0/747638628 <== mds.0 v2:172.21.15.190:6832/742721767 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x26fa6f0 con 0x27e0610 2023-11-15T02:57:11.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.155+0000 7fae18ff9700 10 client.4647 handle_command_reply: tid=0 2023-11-15T02:57:11.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.155+0000 7fae4b2ab740 1 client.4647 shutdown 2023-11-15T02:57:11.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.155+0000 7fae4b2ab740 1 -- 172.21.15.121:0/747638628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27d3ab0 msgr2=0x27d4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:11.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.155+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/747638628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27d3ab0 0x27d4420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fae34011650 tx=0x7fae34001460).stop 2023-11-15T02:57:11.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.155+0000 7fae4b2ab740 1 -- 172.21.15.121:0/747638628 shutdown_connections 2023-11-15T02:57:11.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.155+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/747638628 >> [v2:172.21.15.190:6832/742721767,v1:172.21.15.190:6833/742721767] conn(0x27e0610 0x27e09f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.155+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/747638628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27d3ab0 0x27d4420 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.155+0000 7fae18ff9700 0 client.4647 ms_handle_reset on v2:172.21.15.190:6832/742721767 2023-11-15T02:57:11.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.155+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/747638628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x278e300 0x27d3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.155+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/747638628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x278d510 0x27b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.155+0000 7fae4b2ab740 1 -- 172.21.15.121:0/747638628 >> 172.21.15.121:0/747638628 conn(0x256c550 msgr2=0x27b71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:11.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.155+0000 7fae4b2ab740 1 -- 172.21.15.121:0/747638628 shutdown_connections 2023-11-15T02:57:11.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.155+0000 7fae4b2ab740 1 -- 172.21.15.121:0/747638628 wait complete. 2023-11-15T02:57:11.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.155+0000 7fae4b2ab740 20 client.4647 trim_cache size 0 max 16384 2023-11-15T02:57:11.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae4b2ab740 1 Processor -- start 2023-11-15T02:57:11.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae4b2ab740 1 -- start start 2023-11-15T02:57:11.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae4b2ab740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x278d510 0x2789960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae4b2ab740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x278e300 0x278e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae4b2ab740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27d3ab0 0x278ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae4b2ab740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x27b34b0 con 0x27d3ab0 2023-11-15T02:57:11.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae4b2ab740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x2684f00 con 0x278e300 2023-11-15T02:57:11.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae4b2ab740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x278f0c0 con 0x278d510 2023-11-15T02:57:11.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae3ad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27d3ab0 0x278ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae3a59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x278d510 0x2789960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae3a59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x278d510 0x2789960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:38968/0 (socket says 172.21.15.121:38968) 2023-11-15T02:57:11.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae3a59c700 1 -- 172.21.15.121:0/1754318223 learned_addr learned my addr 172.21.15.121:0/1754318223 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:11.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae3ad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27d3ab0 0x278ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49898/0 (socket says 172.21.15.121:49898) 2023-11-15T02:57:11.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae33fff700 1 --2- 172.21.15.121:0/1754318223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x278e300 0x278e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae3ad9d700 1 -- 172.21.15.121:0/1754318223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x278d510 msgr2=0x2789960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:11.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae3ad9d700 1 --2- 172.21.15.121:0/1754318223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x278d510 0x2789960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae3ad9d700 1 -- 172.21.15.121:0/1754318223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x278e300 msgr2=0x278e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:11.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae3ad9d700 1 --2- 172.21.15.121:0/1754318223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x278e300 0x278e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae3ad9d700 1 -- 172.21.15.121:0/1754318223 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c157620 con 0x27d3ab0 2023-11-15T02:57:11.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.175+0000 7fae33fff700 1 --2- 172.21.15.121:0/1754318223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x278e300 0x278e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:11.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.179+0000 7fae3ad9d700 1 --2- 172.21.15.121:0/1754318223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27d3ab0 0x278ebd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x27b3990 tx=0x7fae3401f000).ready entity=mon.0 client_cookie=8fff38dbb9e12132 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:11.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.179+0000 7fae1bfff700 1 -- 172.21.15.121:0/1754318223 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34007740 con 0x27d3ab0 2023-11-15T02:57:11.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.179+0000 7fae1bfff700 1 -- 172.21.15.121:0/1754318223 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae34007920 con 0x27d3ab0 2023-11-15T02:57:11.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.179+0000 7fae1bfff700 1 -- 172.21.15.121:0/1754318223 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34006c30 con 0x27d3ab0 2023-11-15T02:57:11.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.179+0000 7fae4b2ab740 1 -- 172.21.15.121:0/1754318223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27d3ab0 msgr2=0x278ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:11.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.179+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/1754318223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27d3ab0 0x278ebd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x27b3990 tx=0x7fae3401f000).stop 2023-11-15T02:57:11.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.179+0000 7fae4b2ab740 1 -- 172.21.15.121:0/1754318223 shutdown_connections 2023-11-15T02:57:11.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.179+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/1754318223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27d3ab0 0x278ebd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.179+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/1754318223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x278e300 0x278e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.179+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/1754318223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x278d510 0x2789960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.179+0000 7fae4b2ab740 1 -- 172.21.15.121:0/1754318223 >> 172.21.15.121:0/1754318223 conn(0x256c550 msgr2=0x27d1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:11.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.179+0000 7fae4b2ab740 1 -- 172.21.15.121:0/1754318223 shutdown_connections 2023-11-15T02:57:11.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.179+0000 7fae4b2ab740 1 -- 172.21.15.121:0/1754318223 wait complete. 2023-11-15T02:57:11.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.179+0000 7fae4b2ab740 1 -- start start 2023-11-15T02:57:11.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.179+0000 7fae4b2ab740 1 Processor -- start 2023-11-15T02:57:11.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae4b2ab740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x278d510 0x2835b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae4b2ab740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x278e300 0x2829b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae4b2ab740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27d3ab0 0x27efbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae4b2ab740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2836340 con 0x278e300 2023-11-15T02:57:11.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae4b2ab740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x27d3f90 con 0x27d3ab0 2023-11-15T02:57:11.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae4b2ab740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x278ee90 con 0x278d510 2023-11-15T02:57:11.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae4b2ab740 10 client.0 ll_register_callbacks cb 0x7fae34011270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:57:11.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae33fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x278e300 0x2829b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae33fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x278e300 0x2829b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49912/0 (socket says 172.21.15.121:49912) 2023-11-15T02:57:11.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae33fff700 1 -- 172.21.15.121:0/3412139388 learned_addr learned my addr 172.21.15.121:0/3412139388 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:11.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae3a59c700 1 --2- 172.21.15.121:0/3412139388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x278d510 0x2835b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae3ad9d700 1 --2- 172.21.15.121:0/3412139388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27d3ab0 0x27efbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae33fff700 1 -- 172.21.15.121:0/3412139388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x278d510 msgr2=0x2835b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:11.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae33fff700 1 --2- 172.21.15.121:0/3412139388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x278d510 0x2835b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae33fff700 1 -- 172.21.15.121:0/3412139388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27d3ab0 msgr2=0x27efbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:11.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae33fff700 1 --2- 172.21.15.121:0/3412139388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27d3ab0 0x27efbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae33fff700 1 -- 172.21.15.121:0/3412139388 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2702960 con 0x278e300 2023-11-15T02:57:11.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae33fff700 1 --2- 172.21.15.121:0/3412139388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x278e300 0x2829b10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fae2404d890 tx=0x7fae24030000).ready entity=mon.0 client_cookie=5f681414565db3e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:11.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae18ff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:57:11.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae18ff9700 1 -- 172.21.15.121:0/3412139388 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2404d2c0 con 0x278e300 2023-11-15T02:57:11.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae18ff9700 1 -- 172.21.15.121:0/3412139388 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fae2404d900 con 0x278e300 2023-11-15T02:57:11.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae4b2ab740 1 -- 172.21.15.121:0/3412139388 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae24006af0 con 0x278e300 2023-11-15T02:57:11.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae18ff9700 1 -- 172.21.15.121:0/3412139388 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae240760d0 con 0x278e300 2023-11-15T02:57:11.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae4b2ab740 10 client.4653 fetch_fsmap learned FSMap version 8 2023-11-15T02:57:11.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae4b2ab740 1 -- 172.21.15.121:0/3412139388 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x27026b0 con 0x278e300 2023-11-15T02:57:11.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae18ff9700 1 -- 172.21.15.121:0/3412139388 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae24044020 con 0x278e300 2023-11-15T02:57:11.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae18ff9700 1 -- 172.21.15.121:0/3412139388 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fae2404ce50 con 0x278e300 2023-11-15T02:57:11.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae4b2ab740 10 client.4653 fetch_fsmap finished waiting for FSMap version 8 2023-11-15T02:57:11.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae4b2ab740 10 client.4653 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-15T02:57:11.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae4b2ab740 20 client.4653 populate_metadata read hostname 'smithi121' 2023-11-15T02:57:11.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/3412139388 >> [v2:172.21.15.190:6832/742721767,v1:172.21.15.190:6833/742721767] conn(0x27f00c0 0x27f0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae4b2ab740 4 client.4653 mds_command: new command op to 4270 tid=0[{"prefix": "status"}] 2023-11-15T02:57:11.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.183+0000 7fae4b2ab740 1 -- 172.21.15.121:0/3412139388 --> [v2:172.21.15.190:6832/742721767,v1:172.21.15.190:6833/742721767] -- command(tid 0: {"prefix": "status"}) v1 -- 0x25c34e0 con 0x27f00c0 2023-11-15T02:57:11.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae3a59c700 1 --2- 172.21.15.121:0/3412139388 >> [v2:172.21.15.190:6832/742721767,v1:172.21.15.190:6833/742721767] conn(0x27f00c0 0x27f0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae18ff9700 10 client.4653 ms_handle_connect on v2:172.21.15.190:6832/742721767 2023-11-15T02:57:11.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae3a59c700 1 --2- 172.21.15.121:0/3412139388 >> [v2:172.21.15.190:6832/742721767,v1:172.21.15.190:6833/742721767] conn(0x27f00c0 0x27f0960 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=5cd5c35738c1a39e server_cookie=bd6b57c22ef2b536 in_seq=0 out_seq=0 2023-11-15T02:57:11.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae18ff9700 1 -- 172.21.15.121:0/3412139388 <== mds.0 v2:172.21.15.190:6832/742721767 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x25c34e0 con 0x27f00c0 2023-11-15T02:57:11.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae18ff9700 10 client.4653 handle_command_reply: tid=0 2023-11-15T02:57:11.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae4b2ab740 1 client.4653 shutdown 2023-11-15T02:57:11.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae4b2ab740 1 -- 172.21.15.121:0/3412139388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x278e300 msgr2=0x2829b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:11.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/3412139388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x278e300 0x2829b10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fae2404d890 tx=0x7fae24030000).stop 2023-11-15T02:57:11.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae4b2ab740 1 -- 172.21.15.121:0/3412139388 shutdown_connections 2023-11-15T02:57:11.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/3412139388 >> [v2:172.21.15.190:6832/742721767,v1:172.21.15.190:6833/742721767] conn(0x27f00c0 0x27f0960 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/3412139388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27d3ab0 0x27efbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/3412139388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x278e300 0x2829b10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae4b2ab740 1 --2- 172.21.15.121:0/3412139388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x278d510 0x2835b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae18ff9700 0 client.4653 ms_handle_reset on v2:172.21.15.190:6832/742721767 2023-11-15T02:57:11.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae4b2ab740 1 -- 172.21.15.121:0/3412139388 >> 172.21.15.121:0/3412139388 conn(0x256c550 msgr2=0x27d1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:11.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae4b2ab740 1 -- 172.21.15.121:0/3412139388 shutdown_connections 2023-11-15T02:57:11.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae4b2ab740 1 -- 172.21.15.121:0/3412139388 wait complete. 2023-11-15T02:57:11.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae4b2ab740 20 client.4653 trim_cache size 0 max 16384 2023-11-15T02:57:11.206 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T02:57:11.206 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T02:57:11.206 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T02:57:11.207 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 4270, 2023-11-15T02:57:11.207 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T02:57:11.207 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T02:57:11.207 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T02:57:11.207 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 19.488770076000002, 2023-11-15T02:57:11.208 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 8, 2023-11-15T02:57:11.208 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 21, 2023-11-15T02:57:11.208 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 21, 2023-11-15T02:57:11.208 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 24.840533995000001 2023-11-15T02:57:11.209 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T02:57:11.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae40ef1700 1 -- 172.21.15.121:0/2725703857 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fae1c05a510 msgr2=0x7fae1c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:11.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae40ef1700 1 --2- 172.21.15.121:0/2725703857 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fae1c05a510 0x7fae1c05c9d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fae2c00bb60 tx=0x7fae2c008040).stop 2023-11-15T02:57:11.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae40ef1700 1 -- 172.21.15.121:0/2725703857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae3c151920 msgr2=0x7fae3c0ab5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:11.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae40ef1700 1 --2- 172.21.15.121:0/2725703857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae3c151920 0x7fae3c0ab5c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fae24011000 tx=0x7fae24017000).stop 2023-11-15T02:57:11.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae40ef1700 1 -- 172.21.15.121:0/2725703857 shutdown_connections 2023-11-15T02:57:11.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae40ef1700 1 --2- 172.21.15.121:0/2725703857 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fae1c05a510 0x7fae1c05c9d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae40ef1700 1 --2- 172.21.15.121:0/2725703857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae3c156270 0x7fae3c13aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae40ef1700 1 --2- 172.21.15.121:0/2725703857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae3c151920 0x7fae3c0ab5c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae40ef1700 1 --2- 172.21.15.121:0/2725703857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae3c067150 0x7fae3c148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.187+0000 7fae40ef1700 1 -- 172.21.15.121:0/2725703857 >> 172.21.15.121:0/2725703857 conn(0x7fae3c067a60 msgr2=0x7fae3c14b190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:11.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.191+0000 7fae40ef1700 1 -- 172.21.15.121:0/2725703857 shutdown_connections 2023-11-15T02:57:11.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.191+0000 7fae40ef1700 1 -- 172.21.15.121:0/2725703857 wait complete. 2023-11-15T02:57:11.217 INFO:teuthology.run_tasks:Running task kclient... 2023-11-15T02:57:11.239 INFO:tasks.kclient:Mounting kernel clients... 2023-11-15T02:57:11.240 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-15T02:57:11.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:11.241 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T02:57:11.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:11.262 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link delete ceph-brx 2023-11-15T02:57:11.344 INFO:teuthology.orchestra.run.smithi121.stderr:Cannot find device "ceph-brx" 2023-11-15T02:57:11.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T02:57:11.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:11.349 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T02:57:11.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:11.409 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link delete ceph-brx 2023-11-15T02:57:11.494 INFO:teuthology.orchestra.run.smithi121.stderr:Cannot find device "ceph-brx" 2023-11-15T02:57:11.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T02:57:11.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:11.498 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-11-15T02:57:11.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:11.520 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2023-11-15T02:57:11.604 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2023-11-15T02:57:11.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T02:57:11.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:11.609 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-11-15T02:57:11.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:11.678 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2023-11-15T02:57:11.759 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2023-11-15T02:57:11.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T02:57:11.764 INFO:tasks.kclient:client.0 config is {} 2023-11-15T02:57:11.764 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T02:57:11.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.899+0000 7f9781edb700 1 Processor -- start 2023-11-15T02:57:11.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.899+0000 7f9781edb700 1 -- start start 2023-11-15T02:57:11.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.903+0000 7f9781edb700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f977c0c7520 0x7f977c0c1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.903+0000 7f9781edb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f977c0aa980 0x7f977c0aadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.903+0000 7f9781edb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f977c0c0f70 con 0x7f977c0c7900 2023-11-15T02:57:11.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.903+0000 7f9781edb700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f977c0c5db0 con 0x7f977c0c7520 2023-11-15T02:57:11.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.903+0000 7f9781edb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977c0c6090 con 0x7f977c0aa980 2023-11-15T02:57:11.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.903+0000 7f977affd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f977c0c7900 0x7f977c0c7d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59746/0 (socket says 172.21.15.121:59746) 2023-11-15T02:57:11.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.903+0000 7f977b7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f977c0c7520 0x7f977c0c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.903+0000 7f977affd700 1 -- 172.21.15.121:0/2443959011 learned_addr learned my addr 172.21.15.121:0/2443959011 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:11.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.903+0000 7f977bfff700 1 --2- 172.21.15.121:0/2443959011 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f977c0aa980 0x7f977c0aadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.903+0000 7f977a7fc700 1 -- 172.21.15.121:0/2443959011 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4199781300 0 0) 0x7f977c0c0f70 con 0x7f977c0c7900 2023-11-15T02:57:11.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.903+0000 7f977a7fc700 1 -- 172.21.15.121:0/2443959011 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97640033f0 con 0x7f977c0c7900 2023-11-15T02:57:11.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.903+0000 7f977b7fe700 1 -- 172.21.15.121:0/2443959011 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f977c0aa980 msgr2=0x7f977c0aadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:11.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.903+0000 7f977b7fe700 1 --2- 172.21.15.121:0/2443959011 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f977c0aa980 0x7f977c0aadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.903+0000 7f977b7fe700 1 -- 172.21.15.121:0/2443959011 >> v1:172.21.15.121:6789/0 conn(0x7f977c0c7900 legacy=0x7f977c0c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:11.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.903+0000 7f977b7fe700 1 -- 172.21.15.121:0/2443959011 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977c0c0c00 con 0x7f977c0c7520 2023-11-15T02:57:11.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.903+0000 7f977b7fe700 1 --2- 172.21.15.121:0/2443959011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f977c0c7520 0x7f977c0c1400 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f976c009760 tx=0x7f976c009060).ready entity=mon.0 client_cookie=285875baedd165bd server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:11.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.903+0000 7f977a7fc700 1 -- 172.21.15.121:0/2443959011 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c00d4a0 con 0x7f977c0c7520 2023-11-15T02:57:11.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.903+0000 7f977a7fc700 1 -- 172.21.15.121:0/2443959011 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976c00d600 con 0x7f977c0c7520 2023-11-15T02:57:11.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.903+0000 7f977a7fc700 1 -- 172.21.15.121:0/2443959011 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c00d980 con 0x7f977c0c7520 2023-11-15T02:57:11.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.907+0000 7f9781edb700 1 -- 172.21.15.121:0/2443959011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f977c0c7520 msgr2=0x7f977c0c1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:11.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.907+0000 7f9781edb700 1 --2- 172.21.15.121:0/2443959011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f977c0c7520 0x7f977c0c1400 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f976c009760 tx=0x7f976c009060).stop 2023-11-15T02:57:11.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.907+0000 7f9781edb700 1 -- 172.21.15.121:0/2443959011 shutdown_connections 2023-11-15T02:57:11.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.907+0000 7f9781edb700 1 --2- 172.21.15.121:0/2443959011 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f977c0aa980 0x7f977c0aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.907+0000 7f9781edb700 1 --2- 172.21.15.121:0/2443959011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f977c0c7520 0x7f977c0c1400 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.907+0000 7f9781edb700 1 -- 172.21.15.121:0/2443959011 >> 172.21.15.121:0/2443959011 conn(0x7f977c067780 msgr2=0x7f977c0681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:11.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.907+0000 7f9781edb700 1 -- 172.21.15.121:0/2443959011 shutdown_connections 2023-11-15T02:57:11.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.907+0000 7f9781edb700 1 -- 172.21.15.121:0/2443959011 wait complete. 2023-11-15T02:57:11.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.907+0000 7f9781edb700 1 Processor -- start 2023-11-15T02:57:11.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.907+0000 7f9781edb700 1 -- start start 2023-11-15T02:57:11.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f9781edb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f977c0aa980 0x7f977c062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f9781edb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f977c0c7520 0x7f977c0af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f9781edb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f977c0c7900 0x7f977c15db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f9781edb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c0632d0 con 0x7f977c0c7520 2023-11-15T02:57:11.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f9781edb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c0646c0 con 0x7f977c0c7900 2023-11-15T02:57:11.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f9781edb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977c0b0880 con 0x7f977c0aa980 2023-11-15T02:57:11.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f977affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f977c0c7520 0x7f977c0af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f977affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f977c0c7520 0x7f977c0af800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49930/0 (socket says 172.21.15.121:49930) 2023-11-15T02:57:11.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f977affd700 1 -- 172.21.15.121:0/1953515688 learned_addr learned my addr 172.21.15.121:0/1953515688 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:11.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f977b7fe700 1 --2- 172.21.15.121:0/1953515688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f977c0aa980 0x7f977c062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f977bfff700 1 --2- 172.21.15.121:0/1953515688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f977c0c7900 0x7f977c15db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f977affd700 1 -- 172.21.15.121:0/1953515688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f977c0aa980 msgr2=0x7f977c062de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:11.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f977affd700 1 --2- 172.21.15.121:0/1953515688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f977c0aa980 0x7f977c062de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f977affd700 1 -- 172.21.15.121:0/1953515688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f977c0c7900 msgr2=0x7f977c15db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:11.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f977affd700 1 --2- 172.21.15.121:0/1953515688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f977c0c7900 0x7f977c15db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:11.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f977affd700 1 -- 172.21.15.121:0/1953515688 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976c0092a0 con 0x7f977c0c7520 2023-11-15T02:57:11.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f977affd700 1 --2- 172.21.15.121:0/1953515688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f977c0c7520 0x7f977c0af800 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f977000c420 tx=0x7f9770007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:11.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f97637fe700 1 -- 172.21.15.121:0/1953515688 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97700072e0 con 0x7f977c0c7520 2023-11-15T02:57:11.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f97637fe700 1 -- 172.21.15.121:0/1953515688 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9770007440 con 0x7f977c0c7520 2023-11-15T02:57:11.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f97637fe700 1 -- 172.21.15.121:0/1953515688 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97700057a0 con 0x7f977c0c7520 2023-11-15T02:57:11.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f9781edb700 1 -- 172.21.15.121:0/1953515688 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977c0affe0 con 0x7f977c0c7520 2023-11-15T02:57:11.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.911+0000 7f9781edb700 1 -- 172.21.15.121:0/1953515688 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977c0b05a0 con 0x7f977c0c7520 2023-11-15T02:57:11.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.915+0000 7f9781edb700 1 -- 172.21.15.121:0/1953515688 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f977c064b30 con 0x7f977c0c7520 2023-11-15T02:57:11.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.915+0000 7f97637fe700 1 -- 172.21.15.121:0/1953515688 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9770005bd0 con 0x7f977c0c7520 2023-11-15T02:57:11.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.915+0000 7f97637fe700 1 --2- 172.21.15.121:0/1953515688 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f976805a510 0x7f976805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:11.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.915+0000 7f97637fe700 1 -- 172.21.15.121:0/1953515688 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9770073ff0 con 0x7f977c0c7520 2023-11-15T02:57:11.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.915+0000 7f977b7fe700 1 --2- 172.21.15.121:0/1953515688 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f976805a510 0x7f976805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:11.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.915+0000 7f977b7fe700 1 --2- 172.21.15.121:0/1953515688 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f976805a510 0x7f976805c9d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f976c00d260 tx=0x7f976c016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:11.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:11.923+0000 7f97637fe700 1 -- 172.21.15.121:0/1953515688 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f977c064b30 con 0x7f977c0c7520 2023-11-15T02:57:12.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:12.123+0000 7f9781edb700 1 -- 172.21.15.121:0/1953515688 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f977c064b30 con 0x7f977c0c7520 2023-11-15T02:57:12.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:12.123+0000 7f97637fe700 1 -- 172.21.15.121:0/1953515688 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f977c064b30 con 0x7f977c0c7520 2023-11-15T02:57:12.128 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T02:57:12.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:12.123+0000 7f9781edb700 1 -- 172.21.15.121:0/1953515688 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f976805a510 msgr2=0x7f976805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:12.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:12.123+0000 7f9781edb700 1 --2- 172.21.15.121:0/1953515688 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f976805a510 0x7f976805c9d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f976c00d260 tx=0x7f976c016000).stop 2023-11-15T02:57:12.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:12.123+0000 7f9781edb700 1 -- 172.21.15.121:0/1953515688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f977c0c7520 msgr2=0x7f977c0af800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:12.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:12.123+0000 7f9781edb700 1 --2- 172.21.15.121:0/1953515688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f977c0c7520 0x7f977c0af800 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f977000c420 tx=0x7f9770007c80).stop 2023-11-15T02:57:12.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:12.123+0000 7f9781edb700 1 -- 172.21.15.121:0/1953515688 shutdown_connections 2023-11-15T02:57:12.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:12.123+0000 7f9781edb700 1 --2- 172.21.15.121:0/1953515688 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f976805a510 0x7f976805c9d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:12.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:12.123+0000 7f9781edb700 1 --2- 172.21.15.121:0/1953515688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f977c0c7900 0x7f977c15db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:12.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:12.123+0000 7f9781edb700 1 --2- 172.21.15.121:0/1953515688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f977c0c7520 0x7f977c0af800 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:12.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:12.123+0000 7f9781edb700 1 --2- 172.21.15.121:0/1953515688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f977c0aa980 0x7f977c062de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:12.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:12.123+0000 7f9781edb700 1 -- 172.21.15.121:0/1953515688 >> 172.21.15.121:0/1953515688 conn(0x7f977c067780 msgr2=0x7f977c0bfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:12.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:12.127+0000 7f9781edb700 1 -- 172.21.15.121:0/1953515688 shutdown_connections 2023-11-15T02:57:12.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:12.127+0000 7f9781edb700 1 -- 172.21.15.121:0/1953515688 wait complete. 2023-11-15T02:57:12.150 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-15T02:57:12.150 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T02:57:12.150 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T02:57:12.151 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T02:57:12.151 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T02:57:12.151 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-15T02:57:12.152 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-15T02:57:12.152 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T02:57:12.152 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T02:57:12.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:12.153 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T02:57:12.172 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T02:57:12.173 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T02:57:12.173 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T02:57:12.173 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:12.173 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T02:57:12.174 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:12.174 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:57:12.174 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:12.174 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T02:57:12.174 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:57:12.175 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:12.175 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T02:57:12.175 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:57:12.175 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:12.175 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T02:57:12.176 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:12.176 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T02:57:12.176 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42408sec preferred_lft 42408sec 2023-11-15T02:57:12.176 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T02:57:12.177 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:12.177 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T02:57:12.177 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:12.178 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T02:57:12.178 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:12.179 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-15T02:57:12.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:12.180 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T02:57:12.180 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T02:57:12.181 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add name ceph-brx type bridge 2023-11-15T02:57:12.181 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip addr flush dev ceph-brx 2023-11-15T02:57:12.182 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set ceph-brx up 2023-11-15T02:57:12.183 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-15T02:57:12.183 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T02:57:12.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:12.343 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T02:57:12.379 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T02:57:12.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:12.383 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T02:57:12.461 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T02:57:12.461 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T02:57:12.461 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T02:57:12.462 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T02:57:12.462 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T02:57:12.462 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T02:57:12.462 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T02:57:12.463 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T02:57:12.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:12.464 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T02:57:12.464 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T02:57:12.464 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T02:57:12.465 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T02:57:12.465 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T02:57:12.465 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T02:57:12.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:12.591 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T02:57:12.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:12.612 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list-id 2023-11-15T02:57:12.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:12.681 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T02:57:12.681 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T02:57:12.682 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T02:57:12.682 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-15T02:57:12.682 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T02:57:12.802 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-15T02:57:12.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:12.803 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T02:57:12.804 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T02:57:12.805 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-15T02:57:12.805 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-15T02:57:12.806 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-15T02:57:12.806 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-15T02:57:12.807 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-15T02:57:12.807 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T02:57:12.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:12.983 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T02:57:12.984 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T02:57:12.984 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T02:57:12.984 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set dev brx.0 master ceph-brx 2023-11-15T02:57:12.985 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T02:57:13.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:13.051 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T02:57:13.067 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T02:57:13.067 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:57:13.195 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:57:13.195 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:57:13.328 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:57:13.329 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:57:13.335 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T02:57:13.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:13.335 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T02:57:13.365 INFO:tasks.kclient:client.2 config is {} 2023-11-15T02:57:13.392 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T02:57:13.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.547+0000 7f8f49d69700 1 Processor -- start 2023-11-15T02:57:13.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.547+0000 7f8f49d69700 1 -- start start 2023-11-15T02:57:13.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.547+0000 7f8f49d69700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8f44151890 0x7f8f44063ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:13.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.547+0000 7f8f49d69700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f44156500 con 0x7f8f440c27c0 2023-11-15T02:57:13.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.547+0000 7f8f49d69700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f441558f0 con 0x7f8f44151c70 2023-11-15T02:57:13.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.547+0000 7f8f49d69700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f44155bd0 con 0x7f8f44151890 2023-11-15T02:57:13.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.547+0000 7f8f437fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8f44151890 0x7f8f44063ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:13.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.547+0000 7f8f43fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f8f440c27c0 0x7f8f440bb910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59750/0 (socket says 172.21.15.121:59750) 2023-11-15T02:57:13.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.547+0000 7f8f43fff700 1 -- 172.21.15.121:0/1858739737 learned_addr learned my addr 172.21.15.121:0/1858739737 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:13.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.547+0000 7f8f427fc700 1 -- 172.21.15.121:0/1858739737 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1830302890 0 0) 0x7f8f44156500 con 0x7f8f440c27c0 2023-11-15T02:57:13.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.547+0000 7f8f427fc700 1 -- 172.21.15.121:0/1858739737 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f2c0033f0 con 0x7f8f440c27c0 2023-11-15T02:57:13.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.547+0000 7f8f437fe700 1 -- 172.21.15.121:0/1858739737 >> v1:172.21.15.190:6789/0 conn(0x7f8f44151c70 legacy=0x7f8f440c20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:13.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.547+0000 7f8f437fe700 1 -- 172.21.15.121:0/1858739737 >> v1:172.21.15.121:6789/0 conn(0x7f8f440c27c0 legacy=0x7f8f440bb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:13.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.551+0000 7f8f437fe700 1 -- 172.21.15.121:0/1858739737 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f44155eb0 con 0x7f8f44151890 2023-11-15T02:57:13.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.551+0000 7f8f437fe700 1 --2- 172.21.15.121:0/1858739737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f44151890 0x7f8f44063ab0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f8f34009760 tx=0x7f8f34009060).ready entity=mon.0 client_cookie=3057c00660a18a44 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:13.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.551+0000 7f8f427fc700 1 -- 172.21.15.121:0/1858739737 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400f510 con 0x7f8f44151890 2023-11-15T02:57:13.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.551+0000 7f8f427fc700 1 -- 172.21.15.121:0/1858739737 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3400f670 con 0x7f8f44151890 2023-11-15T02:57:13.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.551+0000 7f8f427fc700 1 -- 172.21.15.121:0/1858739737 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400f9f0 con 0x7f8f44151890 2023-11-15T02:57:13.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.551+0000 7f8f49d69700 1 -- 172.21.15.121:0/1858739737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f44151890 msgr2=0x7f8f44063ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:13.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.551+0000 7f8f49d69700 1 --2- 172.21.15.121:0/1858739737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f44151890 0x7f8f44063ab0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f8f34009760 tx=0x7f8f34009060).stop 2023-11-15T02:57:13.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.551+0000 7f8f49d69700 1 -- 172.21.15.121:0/1858739737 shutdown_connections 2023-11-15T02:57:13.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.551+0000 7f8f49d69700 1 --2- 172.21.15.121:0/1858739737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f44151890 0x7f8f44063ab0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:13.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.551+0000 7f8f49d69700 1 -- 172.21.15.121:0/1858739737 >> 172.21.15.121:0/1858739737 conn(0x7f8f440c66d0 msgr2=0x7f8f440c6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:13.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.551+0000 7f8f49d69700 1 -- 172.21.15.121:0/1858739737 shutdown_connections 2023-11-15T02:57:13.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.551+0000 7f8f49d69700 1 -- 172.21.15.121:0/1858739737 wait complete. 2023-11-15T02:57:13.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f49d69700 1 Processor -- start 2023-11-15T02:57:13.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f49d69700 1 -- start start 2023-11-15T02:57:13.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f49d69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f440c27c0 0x7f8f441654e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:13.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f49d69700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f44151890 0x7f8f4414c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:13.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f49d69700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f440a7a60 0x7f8f440a7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:13.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f49d69700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f44165370 con 0x7f8f44151890 2023-11-15T02:57:13.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f49d69700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f44153f70 con 0x7f8f440a7a60 2023-11-15T02:57:13.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f49d69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f441659d0 con 0x7f8f440c27c0 2023-11-15T02:57:13.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f42ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f44151890 0x7f8f4414c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:13.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f42ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f44151890 0x7f8f4414c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49952/0 (socket says 172.21.15.121:49952) 2023-11-15T02:57:13.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f42ffd700 1 -- 172.21.15.121:0/1021537887 learned_addr learned my addr 172.21.15.121:0/1021537887 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:13.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f43fff700 1 --2- 172.21.15.121:0/1021537887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f440a7a60 0x7f8f440a7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:13.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f437fe700 1 --2- 172.21.15.121:0/1021537887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f440c27c0 0x7f8f441654e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:13.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f42ffd700 1 -- 172.21.15.121:0/1021537887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f440c27c0 msgr2=0x7f8f441654e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:13.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f42ffd700 1 --2- 172.21.15.121:0/1021537887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f440c27c0 0x7f8f441654e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:13.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f42ffd700 1 -- 172.21.15.121:0/1021537887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f440a7a60 msgr2=0x7f8f440a7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:13.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f42ffd700 1 --2- 172.21.15.121:0/1021537887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f440a7a60 0x7f8f440a7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:13.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f42ffd700 1 -- 172.21.15.121:0/1021537887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f340092a0 con 0x7f8f44151890 2023-11-15T02:57:13.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f43fff700 1 --2- 172.21.15.121:0/1021537887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f440a7a60 0x7f8f440a7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:13.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f42ffd700 1 --2- 172.21.15.121:0/1021537887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f44151890 0x7f8f4414c7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f8f3800c470 tx=0x7f8f38007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:13.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f2b7fe700 1 -- 172.21.15.121:0/1021537887 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38007020 con 0x7f8f44151890 2023-11-15T02:57:13.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f2b7fe700 1 -- 172.21.15.121:0/1021537887 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f38007180 con 0x7f8f44151890 2023-11-15T02:57:13.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.555+0000 7f8f49d69700 1 -- 172.21.15.121:0/1021537887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f440a83b0 con 0x7f8f44151890 2023-11-15T02:57:13.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.559+0000 7f8f2b7fe700 1 -- 172.21.15.121:0/1021537887 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38007520 con 0x7f8f44151890 2023-11-15T02:57:13.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.559+0000 7f8f49d69700 1 -- 172.21.15.121:0/1021537887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f44153920 con 0x7f8f44151890 2023-11-15T02:57:13.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.559+0000 7f8f49d69700 1 -- 172.21.15.121:0/1021537887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f44154120 con 0x7f8f44151890 2023-11-15T02:57:13.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.559+0000 7f8f2b7fe700 1 -- 172.21.15.121:0/1021537887 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8f380068b0 con 0x7f8f44151890 2023-11-15T02:57:13.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.563+0000 7f8f2b7fe700 1 --2- 172.21.15.121:0/1021537887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8f3005a510 0x7f8f3005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:13.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.563+0000 7f8f2b7fe700 1 -- 172.21.15.121:0/1021537887 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8f38073af0 con 0x7f8f44151890 2023-11-15T02:57:13.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.563+0000 7f8f437fe700 1 --2- 172.21.15.121:0/1021537887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8f3005a510 0x7f8f3005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:13.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.563+0000 7f8f437fe700 1 --2- 172.21.15.121:0/1021537887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8f3005a510 0x7f8f3005c9d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f8f3400f640 tx=0x7f8f34017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:13.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.567+0000 7f8f2b7fe700 1 -- 172.21.15.121:0/1021537887 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8f44154120 con 0x7f8f44151890 2023-11-15T02:57:13.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.763+0000 7f8f49d69700 1 -- 172.21.15.121:0/1021537887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8f44154120 con 0x7f8f44151890 2023-11-15T02:57:13.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.763+0000 7f8f2b7fe700 1 -- 172.21.15.121:0/1021537887 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8f44154120 con 0x7f8f44151890 2023-11-15T02:57:13.770 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T02:57:13.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.767+0000 7f8f49d69700 1 -- 172.21.15.121:0/1021537887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8f3005a510 msgr2=0x7f8f3005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:13.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.767+0000 7f8f49d69700 1 --2- 172.21.15.121:0/1021537887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8f3005a510 0x7f8f3005c9d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f8f3400f640 tx=0x7f8f34017000).stop 2023-11-15T02:57:13.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.767+0000 7f8f49d69700 1 -- 172.21.15.121:0/1021537887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f44151890 msgr2=0x7f8f4414c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:13.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.767+0000 7f8f49d69700 1 --2- 172.21.15.121:0/1021537887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f44151890 0x7f8f4414c7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f8f3800c470 tx=0x7f8f38007e00).stop 2023-11-15T02:57:13.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.767+0000 7f8f49d69700 1 -- 172.21.15.121:0/1021537887 shutdown_connections 2023-11-15T02:57:13.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.767+0000 7f8f49d69700 1 --2- 172.21.15.121:0/1021537887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8f3005a510 0x7f8f3005c9d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:13.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.767+0000 7f8f49d69700 1 --2- 172.21.15.121:0/1021537887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f440a7a60 0x7f8f440a7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:13.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.767+0000 7f8f49d69700 1 --2- 172.21.15.121:0/1021537887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f44151890 0x7f8f4414c7a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:13.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.767+0000 7f8f49d69700 1 --2- 172.21.15.121:0/1021537887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f440c27c0 0x7f8f441654e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:13.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.767+0000 7f8f49d69700 1 -- 172.21.15.121:0/1021537887 >> 172.21.15.121:0/1021537887 conn(0x7f8f440c66d0 msgr2=0x7f8f4414d170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:13.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.767+0000 7f8f49d69700 1 -- 172.21.15.121:0/1021537887 shutdown_connections 2023-11-15T02:57:13.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:13.767+0000 7f8f49d69700 1 -- 172.21.15.121:0/1021537887 wait complete. 2023-11-15T02:57:13.791 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-15T02:57:13.791 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T02:57:13.792 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T02:57:13.792 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-15T02:57:13.792 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-15T02:57:13.793 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-15T02:57:13.793 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-15T02:57:13.793 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T02:57:13.794 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-15T02:57:13.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:13.794 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T02:57:13.814 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T02:57:13.814 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T02:57:13.815 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T02:57:13.815 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:13.815 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T02:57:13.815 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:13.816 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:57:13.816 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:13.816 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T02:57:13.816 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:57:13.816 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:13.817 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T02:57:13.817 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:57:13.817 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:13.817 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T02:57:13.818 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:13.818 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T02:57:13.818 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42406sec preferred_lft 42406sec 2023-11-15T02:57:13.818 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T02:57:13.819 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:13.819 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T02:57:13.820 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:13.820 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T02:57:13.820 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:13.820 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T02:57:13.821 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:13.821 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T02:57:13.821 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:13.822 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link tentative 2023-11-15T02:57:13.822 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:13.822 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T02:57:13.822 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T02:57:13.822 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link tentative 2023-11-15T02:57:13.823 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:13.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:13.824 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T02:57:13.904 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T02:57:13.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:13.907 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T02:57:13.973 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T02:57:13.974 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T02:57:13.974 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T02:57:13.974 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T02:57:13.974 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T02:57:13.975 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T02:57:13.975 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T02:57:13.975 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T02:57:13.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:13.977 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T02:57:13.977 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T02:57:13.977 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T02:57:13.978 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T02:57:13.978 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T02:57:13.978 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T02:57:14.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:14.115 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T02:57:14.133 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T02:57:14.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:14.135 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list-id 2023-11-15T02:57:14.191 INFO:teuthology.orchestra.run.smithi121.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-15T02:57:14.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:14.193 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T02:57:14.193 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T02:57:14.193 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T02:57:14.193 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-15T02:57:14.194 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T02:57:14.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:14.313 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-15T02:57:14.351 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T02:57:14.351 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T02:57:14.351 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T02:57:14.352 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:14.352 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T02:57:14.352 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:14.352 INFO:teuthology.orchestra.run.smithi121.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T02:57:14.353 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether fa:cb:3b:4f:12:94 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-15T02:57:14.353 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-15T02:57:14.353 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:14.353 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::f8cb:3bff:fe4f:1294/64 scope link tentative 2023-11-15T02:57:14.353 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:14.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:14.356 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-15T02:57:14.440 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T02:57:14.441 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T02:57:14.441 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T02:57:14.441 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:14.442 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T02:57:14.442 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:14.442 INFO:teuthology.orchestra.run.smithi121.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T02:57:14.442 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether fa:cb:3b:4f:12:94 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-15T02:57:14.442 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-15T02:57:14.443 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:14.443 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::f8cb:3bff:fe4f:1294/64 scope link tentative 2023-11-15T02:57:14.443 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:14.444 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-15T02:57:14.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:14.445 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T02:57:14.446 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T02:57:14.446 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-15T02:57:14.446 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-15T02:57:14.447 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-15T02:57:14.447 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-15T02:57:14.447 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-15T02:57:14.447 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T02:57:14.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:14.650 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T02:57:14.650 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T02:57:14.651 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.1 up 2023-11-15T02:57:14.651 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set dev brx.1 master ceph-brx 2023-11-15T02:57:14.651 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T02:57:14.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:14.726 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.2 2023-11-15T02:57:14.743 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T02:57:14.744 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:57:14.865 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:57:14.865 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=2,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:57:14.888 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:57:14.888 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=2,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:57:14.895 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T02:57:14.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:14.896 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-15T02:57:14.927 INFO:tasks.kclient:client.1 config is {} 2023-11-15T02:57:14.927 DEBUG:teuthology.orchestra.run.smithi190:> ceph fs ls 2023-11-15T02:57:15.063 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.056+0000 7f18358f1700 1 Processor -- start 2023-11-15T02:57:15.063 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.056+0000 7f18358f1700 1 -- start start 2023-11-15T02:57:15.066 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.060+0000 7f18358f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18300c13f0 0x7f1830069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:15.066 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.060+0000 7f18358f1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18300bff50 con 0x7f1830158120 2023-11-15T02:57:15.066 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.060+0000 7f18358f1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18300aa280 con 0x7f18300a9ea0 2023-11-15T02:57:15.067 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.060+0000 7f18358f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18300bf920 con 0x7f18300c13f0 2023-11-15T02:57:15.067 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.060+0000 7f182effd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f18300a9ea0 0x7f1830157a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:38890/0 (socket says 172.21.15.190:38890) 2023-11-15T02:57:15.068 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.060+0000 7f182f7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18300c13f0 0x7f1830069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:15.068 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.060+0000 7f182effd700 1 -- 172.21.15.190:0/2516405699 learned_addr learned my addr 172.21.15.190:0/2516405699 (peer_addr_for_me v1:172.21.15.190:0/0) 2023-11-15T02:57:15.068 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.060+0000 7f182f7fe700 1 -- 172.21.15.190:0/2516405699 >> v1:172.21.15.190:6789/0 conn(0x7f18300a9ea0 legacy=0x7f1830157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:15.068 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.060+0000 7f182dffb700 1 -- 172.21.15.190:0/2516405699 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574697035 0 0) 0x7f18300bff50 con 0x7f1830158120 2023-11-15T02:57:15.069 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.060+0000 7f182f7fe700 1 -- 172.21.15.190:0/2516405699 >> v1:172.21.15.121:6789/0 conn(0x7f1830158120 legacy=0x7f1830069780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:15.069 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.060+0000 7f182f7fe700 1 -- 172.21.15.190:0/2516405699 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183006a4b0 con 0x7f18300c13f0 2023-11-15T02:57:15.070 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.064+0000 7f182f7fe700 1 --2- 172.21.15.190:0/2516405699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18300c13f0 0x7f1830069e90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f1828009e10 tx=0x7f1828009780).ready entity=mon.2 client_cookie=4778f9fc10e586c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:15.070 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.064+0000 7f182dffb700 1 -- 172.21.15.190:0/2516405699 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182800e070 con 0x7f18300c13f0 2023-11-15T02:57:15.071 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.064+0000 7f182dffb700 1 -- 172.21.15.190:0/2516405699 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182800bbb0 con 0x7f18300c13f0 2023-11-15T02:57:15.071 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.064+0000 7f182dffb700 1 -- 172.21.15.190:0/2516405699 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828012410 con 0x7f18300c13f0 2023-11-15T02:57:15.071 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.064+0000 7f18358f1700 1 -- 172.21.15.190:0/2516405699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18300c13f0 msgr2=0x7f1830069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:15.071 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.064+0000 7f18358f1700 1 --2- 172.21.15.190:0/2516405699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18300c13f0 0x7f1830069e90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f1828009e10 tx=0x7f1828009780).stop 2023-11-15T02:57:15.071 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.064+0000 7f18358f1700 1 -- 172.21.15.190:0/2516405699 shutdown_connections 2023-11-15T02:57:15.072 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.064+0000 7f18358f1700 1 --2- 172.21.15.190:0/2516405699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18300c13f0 0x7f1830069e90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:15.072 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.064+0000 7f18358f1700 1 -- 172.21.15.190:0/2516405699 >> 172.21.15.190:0/2516405699 conn(0x7f1830068c70 msgr2=0x7f18300ca9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:15.072 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.064+0000 7f18358f1700 1 -- 172.21.15.190:0/2516405699 shutdown_connections 2023-11-15T02:57:15.072 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.064+0000 7f18358f1700 1 -- 172.21.15.190:0/2516405699 wait complete. 2023-11-15T02:57:15.073 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f18358f1700 1 Processor -- start 2023-11-15T02:57:15.073 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f18358f1700 1 -- start start 2023-11-15T02:57:15.075 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f18358f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18300a9ea0 0x7f18300c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:15.076 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f18358f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18300c13f0 0x7f1830168030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:15.076 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f18358f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1830158120 0x7f18301530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:15.076 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f18358f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18300c1d70 con 0x7f18300c13f0 2023-11-15T02:57:15.076 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f18358f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1830168cf0 con 0x7f18300a9ea0 2023-11-15T02:57:15.076 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f18358f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18300c1fc0 con 0x7f1830158120 2023-11-15T02:57:15.077 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f182effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18300a9ea0 0x7f18300c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:15.077 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f182effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18300a9ea0 0x7f18300c45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:46672/0 (socket says 172.21.15.190:46672) 2023-11-15T02:57:15.077 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f182effd700 1 -- 172.21.15.190:0/3649641666 learned_addr learned my addr 172.21.15.190:0/3649641666 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-11-15T02:57:15.077 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f182f7fe700 1 --2- 172.21.15.190:0/3649641666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1830158120 0x7f18301530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:15.077 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f182e7fc700 1 --2- 172.21.15.190:0/3649641666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18300c13f0 0x7f1830168030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:15.078 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f182effd700 1 -- 172.21.15.190:0/3649641666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1830158120 msgr2=0x7f18301530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:15.078 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f182effd700 1 --2- 172.21.15.190:0/3649641666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1830158120 0x7f18301530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:15.078 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f182effd700 1 -- 172.21.15.190:0/3649641666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18300c13f0 msgr2=0x7f1830168030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:15.078 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f182effd700 1 --2- 172.21.15.190:0/3649641666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18300c13f0 0x7f1830168030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:15.078 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f182effd700 1 -- 172.21.15.190:0/3649641666 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1828009a20 con 0x7f18300a9ea0 2023-11-15T02:57:15.079 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f182f7fe700 1 --2- 172.21.15.190:0/3649641666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1830158120 0x7f18301530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:57:15.079 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.068+0000 7f182effd700 1 --2- 172.21.15.190:0/3649641666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18300a9ea0 0x7f18300c45d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f182000c470 tx=0x7f1820007da0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:15.080 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.072+0000 7f1816ffd700 1 -- 172.21.15.190:0/3649641666 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182000ce20 con 0x7f18300a9ea0 2023-11-15T02:57:15.080 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.072+0000 7f1816ffd700 1 -- 172.21.15.190:0/3649641666 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18200072e0 con 0x7f18300a9ea0 2023-11-15T02:57:15.081 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.072+0000 7f18358f1700 1 -- 172.21.15.190:0/3649641666 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18300c2300 con 0x7f18300a9ea0 2023-11-15T02:57:15.081 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.072+0000 7f1816ffd700 1 -- 172.21.15.190:0/3649641666 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820007660 con 0x7f18300a9ea0 2023-11-15T02:57:15.081 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.072+0000 7f18358f1700 1 -- 172.21.15.190:0/3649641666 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18300c2b40 con 0x7f18300a9ea0 2023-11-15T02:57:15.081 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.072+0000 7f18358f1700 1 -- 172.21.15.190:0/3649641666 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18300c2d60 con 0x7f18300a9ea0 2023-11-15T02:57:15.081 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.072+0000 7f1816ffd700 1 -- 172.21.15.190:0/3649641666 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f18200059b0 con 0x7f18300a9ea0 2023-11-15T02:57:15.082 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.072+0000 7f1816ffd700 1 --2- 172.21.15.190:0/3649641666 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f181c05a510 0x7f181c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:15.082 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.076+0000 7f1816ffd700 1 -- 172.21.15.190:0/3649641666 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1820073f80 con 0x7f18300a9ea0 2023-11-15T02:57:15.082 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.076+0000 7f182e7fc700 1 --2- 172.21.15.190:0/3649641666 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f181c05a510 0x7f181c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:15.082 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.076+0000 7f182e7fc700 1 --2- 172.21.15.190:0/3649641666 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f181c05a510 0x7f181c05c9d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f18300c8a10 tx=0x7f1824007c40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:15.084 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.080+0000 7f1816ffd700 1 -- 172.21.15.190:0/3649641666 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f18200476a0 con 0x7f18300a9ea0 2023-11-15T02:57:15.288 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.284+0000 7f18358f1700 1 -- 172.21.15.190:0/3649641666 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f183005eeb0 con 0x7f18300a9ea0 2023-11-15T02:57:15.289 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.284+0000 7f1816ffd700 1 -- 172.21.15.190:0/3649641666 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f182004acc0 con 0x7f18300a9ea0 2023-11-15T02:57:15.290 INFO:teuthology.orchestra.run.smithi190.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T02:57:15.293 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.288+0000 7f18358f1700 1 -- 172.21.15.190:0/3649641666 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f181c05a510 msgr2=0x7f181c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:15.293 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.288+0000 7f18358f1700 1 --2- 172.21.15.190:0/3649641666 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f181c05a510 0x7f181c05c9d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f18300c8a10 tx=0x7f1824007c40).stop 2023-11-15T02:57:15.294 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.288+0000 7f18358f1700 1 -- 172.21.15.190:0/3649641666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18300a9ea0 msgr2=0x7f18300c45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:15.294 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.288+0000 7f18358f1700 1 --2- 172.21.15.190:0/3649641666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18300a9ea0 0x7f18300c45d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f182000c470 tx=0x7f1820007da0).stop 2023-11-15T02:57:15.295 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.288+0000 7f18358f1700 1 -- 172.21.15.190:0/3649641666 shutdown_connections 2023-11-15T02:57:15.295 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.288+0000 7f18358f1700 1 --2- 172.21.15.190:0/3649641666 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f181c05a510 0x7f181c05c9d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:15.295 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.288+0000 7f18358f1700 1 --2- 172.21.15.190:0/3649641666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1830158120 0x7f18301530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:15.296 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.288+0000 7f18358f1700 1 --2- 172.21.15.190:0/3649641666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18300c13f0 0x7f1830168030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:15.296 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.288+0000 7f18358f1700 1 --2- 172.21.15.190:0/3649641666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18300a9ea0 0x7f18300c45d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:15.296 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.288+0000 7f18358f1700 1 -- 172.21.15.190:0/3649641666 >> 172.21.15.190:0/3649641666 conn(0x7f1830068c70 msgr2=0x7f1830155670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:15.297 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.288+0000 7f18358f1700 1 -- 172.21.15.190:0/3649641666 shutdown_connections 2023-11-15T02:57:15.297 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:15.288+0000 7f18358f1700 1 -- 172.21.15.190:0/3649641666 wait complete. 2023-11-15T02:57:15.309 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-15T02:57:15.309 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T02:57:15.310 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2023-11-15T02:57:15.310 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-15T02:57:15.310 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-15T02:57:15.311 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-15T02:57:15.311 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-15T02:57:15.311 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T02:57:15.311 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-15T02:57:15.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:15.312 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2023-11-15T02:57:15.327 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T02:57:15.327 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T02:57:15.328 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T02:57:15.328 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:15.328 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2023-11-15T02:57:15.328 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:15.328 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:57:15.328 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:15.329 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f0 2023-11-15T02:57:15.329 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:57:15.329 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:15.329 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f1 2023-11-15T02:57:15.329 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:57:15.330 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:15.330 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T02:57:15.330 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:15.330 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T02:57:15.330 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42424sec preferred_lft 42424sec 2023-11-15T02:57:15.331 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::ec4:7aff:fe88:7763/64 scope link 2023-11-15T02:57:15.331 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:15.331 INFO:teuthology.orchestra.run.smithi190.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T02:57:15.331 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 02:42:0f:ab:f3:d3 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:15.331 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T02:57:15.332 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:15.333 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-15T02:57:15.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:15.334 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-11-15T02:57:15.334 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-11-15T02:57:15.334 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add name ceph-brx type bridge 2023-11-15T02:57:15.335 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr flush dev ceph-brx 2023-11-15T02:57:15.335 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set ceph-brx up 2023-11-15T02:57:15.335 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-15T02:57:15.335 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-11-15T02:57:15.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:15.488 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T02:57:15.525 INFO:teuthology.orchestra.run.smithi190.stdout:1 2023-11-15T02:57:15.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:15.529 DEBUG:teuthology.orchestra.run.smithi190:> route 2023-11-15T02:57:15.605 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2023-11-15T02:57:15.606 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T02:57:15.606 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T02:57:15.606 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T02:57:15.607 INFO:teuthology.orchestra.run.smithi190.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T02:57:15.607 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T02:57:15.608 INFO:teuthology.orchestra.run.smithi190.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T02:57:15.608 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T02:57:15.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:15.609 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-11-15T02:57:15.610 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-11-15T02:57:15.610 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T02:57:15.610 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T02:57:15.611 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T02:57:15.611 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-11-15T02:57:15.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:15.749 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-11-15T02:57:15.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:15.767 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list-id 2023-11-15T02:57:15.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:15.825 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-11-15T02:57:15.826 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-11-15T02:57:15.826 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-15T02:57:15.826 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-15T02:57:15.827 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-11-15T02:57:15.942 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-15T02:57:15.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:15.943 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-11-15T02:57:15.943 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-11-15T02:57:15.943 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-15T02:57:15.944 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-15T02:57:15.944 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-15T02:57:15.944 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-15T02:57:15.945 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-15T02:57:15.945 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-11-15T02:57:16.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:16.122 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-11-15T02:57:16.122 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-11-15T02:57:16.123 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.0 up 2023-11-15T02:57:16.123 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set dev brx.0 master ceph-brx 2023-11-15T02:57:16.123 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-11-15T02:57:16.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:16.195 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p /home/ubuntu/cephtest/mnt.1 2023-11-15T02:57:16.211 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T02:57:16.212 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:57:16.340 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:57:16.341 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,name=1,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:57:16.467 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:57:16.467 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,name=1,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:57:16.473 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T02:57:16.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:16.474 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-15T02:57:16.508 INFO:tasks.kclient:client.3 config is {} 2023-11-15T02:57:16.509 DEBUG:teuthology.orchestra.run.smithi190:> ceph fs ls 2023-11-15T02:57:16.690 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.684+0000 7f217269b700 1 Processor -- start 2023-11-15T02:57:16.690 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.684+0000 7f217269b700 1 -- start start 2023-11-15T02:57:16.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f217269b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f216c151890 0x7f216c063ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:16.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f217269b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f216c0c27c0 0x7f216c063fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:16.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f217269b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216c156500 con 0x7f216c151c70 2023-11-15T02:57:16.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f217269b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216c1558f0 con 0x7f216c0c27c0 2023-11-15T02:57:16.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f217269b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216c155bd0 con 0x7f216c151890 2023-11-15T02:57:16.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f216bfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f216c151890 0x7f216c063ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:16.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f216bfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f216c151890 0x7f216c063ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:46676/0 (socket says 172.21.15.190:46676) 2023-11-15T02:57:16.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f216bfff700 1 -- 172.21.15.190:0/97545262 learned_addr learned my addr 172.21.15.190:0/97545262 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-11-15T02:57:16.696 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f216bfff700 1 -- 172.21.15.190:0/97545262 >> v2:172.21.15.121:3300/0 conn(0x7f216c0c27c0 msgr2=0x7f216c063fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:16.696 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f216bfff700 1 --2- 172.21.15.190:0/97545262 >> v2:172.21.15.121:3300/0 conn(0x7f216c0c27c0 0x7f216c063fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T02:57:16.696 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f216bfff700 1 -- 172.21.15.190:0/97545262 >> v1:172.21.15.121:6789/0 conn(0x7f216c151c70 legacy=0x7f216c0c20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:16.696 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f216bfff700 1 -- 172.21.15.190:0/97545262 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c155eb0 con 0x7f216c151890 2023-11-15T02:57:16.697 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f216bfff700 1 --2- 172.21.15.190:0/97545262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f216c151890 0x7f216c063ab0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f216c0635c0 tx=0x7f215c0090f0).ready entity=mon.1 client_cookie=dcc76e795f318e93 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:16.697 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f216b7fe700 1 -- 172.21.15.190:0/97545262 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c00b5a0 con 0x7f216c151890 2023-11-15T02:57:16.697 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f216b7fe700 1 -- 172.21.15.190:0/97545262 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c00b700 con 0x7f216c151890 2023-11-15T02:57:16.697 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f216b7fe700 1 -- 172.21.15.190:0/97545262 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c00ba80 con 0x7f216c151890 2023-11-15T02:57:16.697 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f217269b700 1 -- 172.21.15.190:0/97545262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f216c151890 msgr2=0x7f216c063ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:16.698 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f217269b700 1 --2- 172.21.15.190:0/97545262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f216c151890 0x7f216c063ab0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f216c0635c0 tx=0x7f215c0090f0).stop 2023-11-15T02:57:16.698 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f217269b700 1 -- 172.21.15.190:0/97545262 shutdown_connections 2023-11-15T02:57:16.698 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f217269b700 1 --2- 172.21.15.190:0/97545262 >> v2:172.21.15.121:3300/0 conn(0x7f216c0c27c0 0x7f216c063fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T02:57:16.698 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f217269b700 1 --2- 172.21.15.190:0/97545262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f216c151890 0x7f216c063ab0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:16.699 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f217269b700 1 -- 172.21.15.190:0/97545262 >> 172.21.15.190:0/97545262 conn(0x7f216c0c66d0 msgr2=0x7f216c0c6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:16.699 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f217269b700 1 -- 172.21.15.190:0/97545262 shutdown_connections 2023-11-15T02:57:16.699 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.688+0000 7f217269b700 1 -- 172.21.15.190:0/97545262 wait complete. 2023-11-15T02:57:16.700 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.692+0000 7f217269b700 1 Processor -- start 2023-11-15T02:57:16.700 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.692+0000 7f217269b700 1 -- start start 2023-11-15T02:57:16.701 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f217269b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f216c0c27c0 0x7f216c158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:16.701 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f217269b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f216c151890 0x7f216c167c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:16.702 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f217269b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f216c168180 0x7f216c14fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:16.702 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f217269b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c150d70 con 0x7f216c0c27c0 2023-11-15T02:57:16.702 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f217269b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c156e80 con 0x7f216c151890 2023-11-15T02:57:16.702 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f217269b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c168600 con 0x7f216c168180 2023-11-15T02:57:16.702 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f216bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f216c0c27c0 0x7f216c158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:16.703 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f2170c38700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f216c168180 0x7f216c14fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:16.703 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f2163fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f216c151890 0x7f216c167c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:16.703 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f216bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f216c0c27c0 0x7f216c158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.190:50844/0 (socket says 172.21.15.190:50844) 2023-11-15T02:57:16.703 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f216bfff700 1 -- 172.21.15.190:0/690761811 learned_addr learned my addr 172.21.15.190:0/690761811 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-11-15T02:57:16.704 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f216bfff700 1 -- 172.21.15.190:0/690761811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f216c168180 msgr2=0x7f216c14fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:16.705 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f216bfff700 1 --2- 172.21.15.190:0/690761811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f216c168180 0x7f216c14fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:16.705 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f216bfff700 1 -- 172.21.15.190:0/690761811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f216c151890 msgr2=0x7f216c167c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:16.705 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f216bfff700 1 --2- 172.21.15.190:0/690761811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f216c151890 0x7f216c167c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:16.705 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f216bfff700 1 -- 172.21.15.190:0/690761811 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215c009370 con 0x7f216c0c27c0 2023-11-15T02:57:16.706 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f2170c38700 1 --2- 172.21.15.190:0/690761811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f216c168180 0x7f216c14fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:57:16.706 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f2163fff700 1 --2- 172.21.15.190:0/690761811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f216c151890 0x7f216c167c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:57:16.706 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f216bfff700 1 --2- 172.21.15.190:0/690761811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f216c0c27c0 0x7f216c158f60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f215c011fb0 tx=0x7f215c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:16.706 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f2168ff9700 1 -- 172.21.15.190:0/690761811 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c000bc0 con 0x7f216c0c27c0 2023-11-15T02:57:16.707 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f217269b700 1 -- 172.21.15.190:0/690761811 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216c150a90 con 0x7f216c0c27c0 2023-11-15T02:57:16.707 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f2168ff9700 1 -- 172.21.15.190:0/690761811 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c00b870 con 0x7f216c0c27c0 2023-11-15T02:57:16.707 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f2168ff9700 1 -- 172.21.15.190:0/690761811 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c000d20 con 0x7f216c0c27c0 2023-11-15T02:57:16.707 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.696+0000 7f217269b700 1 -- 172.21.15.190:0/690761811 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216c156b10 con 0x7f216c0c27c0 2023-11-15T02:57:16.710 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.700+0000 7f217269b700 1 -- 172.21.15.190:0/690761811 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216c156690 con 0x7f216c0c27c0 2023-11-15T02:57:16.710 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.700+0000 7f2168ff9700 1 -- 172.21.15.190:0/690761811 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f215c021070 con 0x7f216c0c27c0 2023-11-15T02:57:16.710 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.700+0000 7f2168ff9700 1 --2- 172.21.15.190:0/690761811 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f214c05a540 0x7f214c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:16.710 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.700+0000 7f2168ff9700 1 -- 172.21.15.190:0/690761811 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f215c076510 con 0x7f216c0c27c0 2023-11-15T02:57:16.711 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.704+0000 7f2163fff700 1 --2- 172.21.15.190:0/690761811 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f214c05a540 0x7f214c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:16.711 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.704+0000 7f2163fff700 1 --2- 172.21.15.190:0/690761811 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f214c05a540 0x7f214c05ca00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f216c14f080 tx=0x7f2154008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:16.717 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.712+0000 7f2168ff9700 1 -- 172.21.15.190:0/690761811 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f215c049c30 con 0x7f216c0c27c0 2023-11-15T02:57:16.916 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.908+0000 7f217269b700 1 -- 172.21.15.190:0/690761811 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f216c156690 con 0x7f216c0c27c0 2023-11-15T02:57:16.917 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.912+0000 7f2168ff9700 1 -- 172.21.15.190:0/690761811 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f216c156690 con 0x7f216c0c27c0 2023-11-15T02:57:16.917 INFO:teuthology.orchestra.run.smithi190.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T02:57:16.922 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.916+0000 7f217269b700 1 -- 172.21.15.190:0/690761811 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f214c05a540 msgr2=0x7f214c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:16.922 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.916+0000 7f217269b700 1 --2- 172.21.15.190:0/690761811 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f214c05a540 0x7f214c05ca00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f216c14f080 tx=0x7f2154008040).stop 2023-11-15T02:57:16.923 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.916+0000 7f217269b700 1 -- 172.21.15.190:0/690761811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f216c0c27c0 msgr2=0x7f216c158f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:16.925 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.916+0000 7f217269b700 1 --2- 172.21.15.190:0/690761811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f216c0c27c0 0x7f216c158f60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f215c011fb0 tx=0x7f215c015000).stop 2023-11-15T02:57:16.926 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.916+0000 7f217269b700 1 -- 172.21.15.190:0/690761811 shutdown_connections 2023-11-15T02:57:16.926 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.916+0000 7f217269b700 1 --2- 172.21.15.190:0/690761811 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f214c05a540 0x7f214c05ca00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:16.926 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.916+0000 7f217269b700 1 --2- 172.21.15.190:0/690761811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f216c168180 0x7f216c14fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:16.926 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.916+0000 7f217269b700 1 --2- 172.21.15.190:0/690761811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f216c151890 0x7f216c167c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:16.926 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.916+0000 7f217269b700 1 --2- 172.21.15.190:0/690761811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f216c0c27c0 0x7f216c158f60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:16.927 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.916+0000 7f217269b700 1 -- 172.21.15.190:0/690761811 >> 172.21.15.190:0/690761811 conn(0x7f216c0c66d0 msgr2=0x7f216c14d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:16.927 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.916+0000 7f217269b700 1 -- 172.21.15.190:0/690761811 shutdown_connections 2023-11-15T02:57:16.927 INFO:teuthology.orchestra.run.smithi190.stderr:2023-11-15T02:57:16.916+0000 7f217269b700 1 -- 172.21.15.190:0/690761811 wait complete. 2023-11-15T02:57:16.942 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-15T02:57:16.943 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T02:57:16.943 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2023-11-15T02:57:16.943 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-15T02:57:16.943 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-15T02:57:16.944 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-15T02:57:16.944 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-15T02:57:16.944 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T02:57:16.944 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-15T02:57:16.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:16.945 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2023-11-15T02:57:16.963 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T02:57:16.964 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T02:57:16.964 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T02:57:16.964 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:16.964 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2023-11-15T02:57:16.965 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:16.965 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:57:16.965 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:16.965 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f0 2023-11-15T02:57:16.965 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:57:16.966 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:16.966 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f1 2023-11-15T02:57:16.966 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:57:16.966 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:16.966 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T02:57:16.967 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:16.967 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T02:57:16.967 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42423sec preferred_lft 42423sec 2023-11-15T02:57:16.967 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::ec4:7aff:fe88:7763/64 scope link 2023-11-15T02:57:16.968 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:16.968 INFO:teuthology.orchestra.run.smithi190.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T02:57:16.968 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 02:42:0f:ab:f3:d3 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:16.968 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T02:57:16.968 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:16.969 INFO:teuthology.orchestra.run.smithi190.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T02:57:16.969 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 6a:b5:38:67:e3:da brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:16.969 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T02:57:16.969 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:16.969 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::7023:7fff:fe80:2084/64 scope link 2023-11-15T02:57:16.970 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:16.970 INFO:teuthology.orchestra.run.smithi190.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T02:57:16.970 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 6a:b5:38:67:e3:da brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-15T02:57:16.970 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::68b5:38ff:fe67:e3da/64 scope link tentative 2023-11-15T02:57:16.970 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:16.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:16.972 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T02:57:17.049 INFO:teuthology.orchestra.run.smithi190.stdout:1 2023-11-15T02:57:17.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:17.053 DEBUG:teuthology.orchestra.run.smithi190:> route 2023-11-15T02:57:17.120 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2023-11-15T02:57:17.120 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T02:57:17.120 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T02:57:17.121 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T02:57:17.121 INFO:teuthology.orchestra.run.smithi190.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T02:57:17.121 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T02:57:17.121 INFO:teuthology.orchestra.run.smithi190.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T02:57:17.121 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T02:57:17.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:17.123 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-11-15T02:57:17.123 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-11-15T02:57:17.124 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T02:57:17.124 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T02:57:17.124 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T02:57:17.124 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-11-15T02:57:17.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:17.264 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-11-15T02:57:17.278 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-15T02:57:17.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:17.280 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list-id 2023-11-15T02:57:17.348 INFO:teuthology.orchestra.run.smithi190.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-15T02:57:17.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:17.349 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-11-15T02:57:17.350 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-11-15T02:57:17.350 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-15T02:57:17.350 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-15T02:57:17.351 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-11-15T02:57:17.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:17.471 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-15T02:57:17.508 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T02:57:17.509 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T02:57:17.509 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T02:57:17.509 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:17.509 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2023-11-15T02:57:17.509 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:17.510 INFO:teuthology.orchestra.run.smithi190.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T02:57:17.510 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 06:b6:d7:16:b4:a6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-15T02:57:17.510 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-15T02:57:17.511 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:17.511 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::4b6:d7ff:fe16:b4a6/64 scope link tentative 2023-11-15T02:57:17.511 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:17.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:17.513 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-15T02:57:17.594 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T02:57:17.595 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T02:57:17.595 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T02:57:17.595 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:17.595 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2023-11-15T02:57:17.596 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:17.596 INFO:teuthology.orchestra.run.smithi190.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T02:57:17.596 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 06:b6:d7:16:b4:a6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-15T02:57:17.596 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-15T02:57:17.596 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:17.597 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::4b6:d7ff:fe16:b4a6/64 scope link tentative 2023-11-15T02:57:17.597 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:17.598 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-15T02:57:17.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:17.599 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-11-15T02:57:17.599 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-11-15T02:57:17.600 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-15T02:57:17.600 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-15T02:57:17.600 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-15T02:57:17.601 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-15T02:57:17.601 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-15T02:57:17.601 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-11-15T02:57:17.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:17.799 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-11-15T02:57:17.800 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-11-15T02:57:17.800 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.1 up 2023-11-15T02:57:17.800 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set dev brx.1 master ceph-brx 2023-11-15T02:57:17.800 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-11-15T02:57:17.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:17.870 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p /home/ubuntu/cephtest/mnt.3 2023-11-15T02:57:17.885 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T02:57:17.886 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:57:18.001 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:57:18.002 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,name=3,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:57:18.025 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:57:18.025 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,name=3,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:57:18.030 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T02:57:18.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:18.031 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-15T02:57:18.061 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-15T02:57:18.078 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:57:18.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.239+0000 7feb3ae9b700 1 Processor -- start 2023-11-15T02:57:18.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.239+0000 7feb3ae9b700 1 -- start start 2023-11-15T02:57:18.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.243+0000 7feb3ae9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb34152780 0x7feb34159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:18.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.243+0000 7feb3ae9b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb340c3fa0 con 0x7feb340c65d0 2023-11-15T02:57:18.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.243+0000 7feb3ae9b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb34159630 con 0x7feb340a5510 2023-11-15T02:57:18.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.243+0000 7feb3ae9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb340c30b0 con 0x7feb34152780 2023-11-15T02:57:18.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.243+0000 7feb33fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7feb340c65d0 0x7feb340c6a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:48176/0 (socket says 172.21.15.121:48176) 2023-11-15T02:57:18.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.243+0000 7feb33fff700 1 -- 172.21.15.121:0/373659683 learned_addr learned my addr 172.21.15.121:0/373659683 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:18.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.243+0000 7feb38c37700 1 --2- 172.21.15.121:0/373659683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb34152780 0x7feb34159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:18.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.243+0000 7feb337fe700 1 -- 172.21.15.121:0/373659683 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765644289 0 0) 0x7feb340c3fa0 con 0x7feb340c65d0 2023-11-15T02:57:18.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.243+0000 7feb337fe700 1 -- 172.21.15.121:0/373659683 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb240033f0 con 0x7feb340c65d0 2023-11-15T02:57:18.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.243+0000 7feb337fe700 1 -- 172.21.15.121:0/373659683 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371736628 0 0) 0x7feb34159630 con 0x7feb340a5510 2023-11-15T02:57:18.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.243+0000 7feb337fe700 1 -- 172.21.15.121:0/373659683 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb340c3fa0 con 0x7feb340a5510 2023-11-15T02:57:18.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.243+0000 7feb38c37700 1 -- 172.21.15.121:0/373659683 >> v1:172.21.15.190:6789/0 conn(0x7feb340a5510 legacy=0x7feb340a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:18.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.243+0000 7feb337fe700 1 -- 172.21.15.121:0/373659683 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1238202976 0 0) 0x7feb240033f0 con 0x7feb340c65d0 2023-11-15T02:57:18.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.243+0000 7feb38c37700 1 -- 172.21.15.121:0/373659683 >> v1:172.21.15.121:6789/0 conn(0x7feb340c65d0 legacy=0x7feb340c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:18.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.243+0000 7feb38c37700 1 -- 172.21.15.121:0/373659683 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb340c3390 con 0x7feb34152780 2023-11-15T02:57:18.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.243+0000 7feb38c37700 1 --2- 172.21.15.121:0/373659683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb34152780 0x7feb34159140 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7feb28009790 tx=0x7feb28009060).ready entity=mon.2 client_cookie=c79b840747871c42 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:18.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.247+0000 7feb337fe700 1 -- 172.21.15.121:0/373659683 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20001470 con 0x7feb34152780 2023-11-15T02:57:18.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.247+0000 7feb337fe700 1 -- 172.21.15.121:0/373659683 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb28011400 con 0x7feb34152780 2023-11-15T02:57:18.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.247+0000 7feb337fe700 1 -- 172.21.15.121:0/373659683 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28011780 con 0x7feb34152780 2023-11-15T02:57:18.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.247+0000 7feb3ae9b700 1 -- 172.21.15.121:0/373659683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb34152780 msgr2=0x7feb34159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:18.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.247+0000 7feb3ae9b700 1 --2- 172.21.15.121:0/373659683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb34152780 0x7feb34159140 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7feb28009790 tx=0x7feb28009060).stop 2023-11-15T02:57:18.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.247+0000 7feb3ae9b700 1 -- 172.21.15.121:0/373659683 shutdown_connections 2023-11-15T02:57:18.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.247+0000 7feb3ae9b700 1 --2- 172.21.15.121:0/373659683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb34152780 0x7feb34159140 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:18.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.247+0000 7feb3ae9b700 1 -- 172.21.15.121:0/373659683 >> 172.21.15.121:0/373659683 conn(0x7feb34062190 msgr2=0x7feb340625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:18.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.247+0000 7feb3ae9b700 1 -- 172.21.15.121:0/373659683 shutdown_connections 2023-11-15T02:57:18.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.247+0000 7feb3ae9b700 1 -- 172.21.15.121:0/373659683 wait complete. 2023-11-15T02:57:18.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.251+0000 7feb3ae9b700 1 Processor -- start 2023-11-15T02:57:18.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.251+0000 7feb3ae9b700 1 -- start start 2023-11-15T02:57:18.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.251+0000 7feb3ae9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb340a5510 0x7feb34155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:18.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.251+0000 7feb3ae9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb340c65d0 0x7feb340ab980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:18.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.251+0000 7feb3ae9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb34152780 0x7feb340abe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:18.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.251+0000 7feb3ae9b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb340ac360 con 0x7feb340a5510 2023-11-15T02:57:18.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.251+0000 7feb3ae9b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb34153530 con 0x7feb340c65d0 2023-11-15T02:57:18.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.251+0000 7feb3ae9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb341539c0 con 0x7feb34152780 2023-11-15T02:57:18.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.251+0000 7feb38c37700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb340a5510 0x7feb34155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:18.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.251+0000 7feb38c37700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb340a5510 0x7feb34155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39324/0 (socket says 172.21.15.121:39324) 2023-11-15T02:57:18.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.251+0000 7feb38c37700 1 -- 172.21.15.121:0/3025597573 learned_addr learned my addr 172.21.15.121:0/3025597573 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:18.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.251+0000 7feb33fff700 1 --2- 172.21.15.121:0/3025597573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb340c65d0 0x7feb340ab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:18.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.251+0000 7feb39438700 1 --2- 172.21.15.121:0/3025597573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb34152780 0x7feb340abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:18.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.251+0000 7feb38c37700 1 -- 172.21.15.121:0/3025597573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb34152780 msgr2=0x7feb340abe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:18.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.251+0000 7feb38c37700 1 --2- 172.21.15.121:0/3025597573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb34152780 0x7feb340abe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:18.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.251+0000 7feb38c37700 1 -- 172.21.15.121:0/3025597573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb340c65d0 msgr2=0x7feb340ab980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:18.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.251+0000 7feb38c37700 1 --2- 172.21.15.121:0/3025597573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb340c65d0 0x7feb340ab980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:18.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.251+0000 7feb38c37700 1 -- 172.21.15.121:0/3025597573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb280092a0 con 0x7feb340a5510 2023-11-15T02:57:18.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.251+0000 7feb39438700 1 --2- 172.21.15.121:0/3025597573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb34152780 0x7feb340abe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:18.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.255+0000 7feb38c37700 1 --2- 172.21.15.121:0/3025597573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb340a5510 0x7feb34155fa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7feb28009d80 tx=0x7feb28016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:18.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.255+0000 7feb30ff9700 1 -- 172.21.15.121:0/3025597573 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb280107f0 con 0x7feb340a5510 2023-11-15T02:57:18.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.255+0000 7feb30ff9700 1 -- 172.21.15.121:0/3025597573 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb28010950 con 0x7feb340a5510 2023-11-15T02:57:18.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.255+0000 7feb3ae9b700 1 -- 172.21.15.121:0/3025597573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb34169da0 con 0x7feb340a5510 2023-11-15T02:57:18.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.255+0000 7feb30ff9700 1 -- 172.21.15.121:0/3025597573 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28010d10 con 0x7feb340a5510 2023-11-15T02:57:18.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.255+0000 7feb3ae9b700 1 -- 172.21.15.121:0/3025597573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb34152ee0 con 0x7feb340a5510 2023-11-15T02:57:18.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.255+0000 7feb3ae9b700 1 -- 172.21.15.121:0/3025597573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb3406c150 con 0x7feb340a5510 2023-11-15T02:57:18.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.259+0000 7feb30ff9700 1 -- 172.21.15.121:0/3025597573 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7feb280263b0 con 0x7feb340a5510 2023-11-15T02:57:18.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.259+0000 7feb30ff9700 1 --2- 172.21.15.121:0/3025597573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feb1805a560 0x7feb1805ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:18.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.259+0000 7feb30ff9700 1 -- 172.21.15.121:0/3025597573 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7feb28075430 con 0x7feb340a5510 2023-11-15T02:57:18.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.259+0000 7feb33fff700 1 --2- 172.21.15.121:0/3025597573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feb1805a560 0x7feb1805ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:18.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.259+0000 7feb33fff700 1 --2- 172.21.15.121:0/3025597573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feb1805a560 0x7feb1805ca20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7feb340c3e40 tx=0x7feb20000bc0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:18.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.263+0000 7feb30ff9700 1 -- 172.21.15.121:0/3025597573 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7feb28048b70 con 0x7feb340a5510 2023-11-15T02:57:18.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.427+0000 7feb3ae9b700 1 -- 172.21.15.121:0/3025597573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb340c3670 con 0x7feb340a5510 2023-11-15T02:57:18.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.431+0000 7feb30ff9700 1 -- 172.21.15.121:0/3025597573 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7feb28022020 con 0x7feb340a5510 2023-11-15T02:57:18.436 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:18.437 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":21,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:52.702668+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:56:49.726353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-15T02:56:50.688098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:57:18.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.431+0000 7feb3ae9b700 1 -- 172.21.15.121:0/3025597573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feb1805a560 msgr2=0x7feb1805ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:18.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.431+0000 7feb3ae9b700 1 --2- 172.21.15.121:0/3025597573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feb1805a560 0x7feb1805ca20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7feb340c3e40 tx=0x7feb20000bc0).stop 2023-11-15T02:57:18.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.431+0000 7feb3ae9b700 1 -- 172.21.15.121:0/3025597573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb340a5510 msgr2=0x7feb34155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:18.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.431+0000 7feb3ae9b700 1 --2- 172.21.15.121:0/3025597573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb340a5510 0x7feb34155fa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7feb28009d80 tx=0x7feb28016000).stop 2023-11-15T02:57:18.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.431+0000 7feb3ae9b700 1 -- 172.21.15.121:0/3025597573 shutdown_connections 2023-11-15T02:57:18.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.431+0000 7feb3ae9b700 1 --2- 172.21.15.121:0/3025597573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feb1805a560 0x7feb1805ca20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:18.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.431+0000 7feb3ae9b700 1 --2- 172.21.15.121:0/3025597573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb34152780 0x7feb340abe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:18.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.431+0000 7feb3ae9b700 1 --2- 172.21.15.121:0/3025597573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb340c65d0 0x7feb340ab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:18.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.431+0000 7feb3ae9b700 1 --2- 172.21.15.121:0/3025597573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb340a5510 0x7feb34155fa0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:18.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.431+0000 7feb3ae9b700 1 -- 172.21.15.121:0/3025597573 >> 172.21.15.121:0/3025597573 conn(0x7feb34062190 msgr2=0x7feb340a8980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:18.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.435+0000 7feb3ae9b700 1 -- 172.21.15.121:0/3025597573 shutdown_connections 2023-11-15T02:57:18.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.435+0000 7feb3ae9b700 1 -- 172.21.15.121:0/3025597573 wait complete. 2023-11-15T02:57:18.459 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-15T02:56:49.726353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-15T02:56:50.688098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-15T02:57:18.460 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T02:57:18.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.611+0000 7f741d907700 1 Processor -- start 2023-11-15T02:57:18.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.611+0000 7f741d907700 1 -- start start 2023-11-15T02:57:18.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.615+0000 7f741d907700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f741814aff0 0x7f7418066c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:18.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.615+0000 7f741d907700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74180657f0 con 0x7f741814ac10 2023-11-15T02:57:18.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.615+0000 7f741d907700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7418066650 con 0x7f7418067160 2023-11-15T02:57:18.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.615+0000 7f741d907700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74180675b0 con 0x7f741814aff0 2023-11-15T02:57:18.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.615+0000 7f741c905700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f741814ac10 0x7f74180c6bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:48178/0 (socket says 172.21.15.121:48178) 2023-11-15T02:57:18.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.615+0000 7f741c905700 1 -- 172.21.15.121:0/3045554716 learned_addr learned my addr 172.21.15.121:0/3045554716 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:18.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.615+0000 7f7417fff700 1 --2- 172.21.15.121:0/3045554716 >> v2:172.21.15.190:3300/0 conn(0x7f741814aff0 0x7f7418066c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:18.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.619+0000 7f74177fe700 1 -- 172.21.15.121:0/3045554716 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087857106 0 0) 0x7f7418066650 con 0x7f7418067160 2023-11-15T02:57:18.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.619+0000 7f74177fe700 1 -- 172.21.15.121:0/3045554716 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74080033f0 con 0x7f7418067160 2023-11-15T02:57:18.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.619+0000 7f74177fe700 1 -- 172.21.15.121:0/3045554716 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231125480 0 0) 0x7f74180657f0 con 0x7f741814ac10 2023-11-15T02:57:18.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.619+0000 7f74177fe700 1 -- 172.21.15.121:0/3045554716 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7418066650 con 0x7f741814ac10 2023-11-15T02:57:18.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.619+0000 7f7417fff700 1 -- 172.21.15.121:0/3045554716 >> v1:172.21.15.190:6789/0 conn(0x7f7418067160 legacy=0x7f74180a78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:18.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.619+0000 7f7417fff700 1 -- 172.21.15.121:0/3045554716 >> v1:172.21.15.121:6789/0 conn(0x7f741814ac10 legacy=0x7f74180c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:18.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.619+0000 7f7417fff700 1 -- 172.21.15.121:0/3045554716 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7418065a40 con 0x7f741814aff0 2023-11-15T02:57:18.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.619+0000 7f7417fff700 1 --2- 172.21.15.121:0/3045554716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f741814aff0 0x7f7418066c70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f7404009760 tx=0x7f7404009060).ready entity=mon.1 client_cookie=7a60751cbbd17728 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:18.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.619+0000 7f74177fe700 1 -- 172.21.15.121:0/3045554716 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740400d4a0 con 0x7f741814aff0 2023-11-15T02:57:18.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.619+0000 7f74177fe700 1 -- 172.21.15.121:0/3045554716 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740400d600 con 0x7f741814aff0 2023-11-15T02:57:18.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.619+0000 7f74177fe700 1 -- 172.21.15.121:0/3045554716 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740400d980 con 0x7f741814aff0 2023-11-15T02:57:18.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.619+0000 7f741d907700 1 -- 172.21.15.121:0/3045554716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f741814aff0 msgr2=0x7f7418066c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:18.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.619+0000 7f741d907700 1 --2- 172.21.15.121:0/3045554716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f741814aff0 0x7f7418066c70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f7404009760 tx=0x7f7404009060).stop 2023-11-15T02:57:18.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.619+0000 7f741d907700 1 -- 172.21.15.121:0/3045554716 shutdown_connections 2023-11-15T02:57:18.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.619+0000 7f741d907700 1 --2- 172.21.15.121:0/3045554716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f741814aff0 0x7f7418066c70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:18.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.619+0000 7f741d907700 1 -- 172.21.15.121:0/3045554716 >> 172.21.15.121:0/3045554716 conn(0x7f7418067a60 msgr2=0x7f7418067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:18.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.619+0000 7f741d907700 1 -- 172.21.15.121:0/3045554716 shutdown_connections 2023-11-15T02:57:18.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.619+0000 7f741d907700 1 -- 172.21.15.121:0/3045554716 wait complete. 2023-11-15T02:57:18.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.623+0000 7f741d907700 1 Processor -- start 2023-11-15T02:57:18.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.623+0000 7f741d907700 1 -- start start 2023-11-15T02:57:18.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f741d907700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7418067160 0x7f74180ac060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:18.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f741d907700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f741814ac10 0x7f741813ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:18.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f741d907700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f741814aff0 0x7f74180bf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:18.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f741d907700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74180ab4b0 con 0x7f741814ac10 2023-11-15T02:57:18.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f741d907700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7418167d50 con 0x7f741814aff0 2023-11-15T02:57:18.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f741d907700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74180ac550 con 0x7f7418067160 2023-11-15T02:57:18.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f7417fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f741814ac10 0x7f741813ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:18.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f7417fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f741814ac10 0x7f741813ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39332/0 (socket says 172.21.15.121:39332) 2023-11-15T02:57:18.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f7417fff700 1 -- 172.21.15.121:0/2387364195 learned_addr learned my addr 172.21.15.121:0/2387364195 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:18.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f741c905700 1 --2- 172.21.15.121:0/2387364195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7418067160 0x7f74180ac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:18.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f741d106700 1 --2- 172.21.15.121:0/2387364195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f741814aff0 0x7f74180bf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:18.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f7417fff700 1 -- 172.21.15.121:0/2387364195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7418067160 msgr2=0x7f74180ac060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:18.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f7417fff700 1 --2- 172.21.15.121:0/2387364195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7418067160 0x7f74180ac060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:18.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f7417fff700 1 -- 172.21.15.121:0/2387364195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f741814aff0 msgr2=0x7f74180bf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:18.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f7417fff700 1 --2- 172.21.15.121:0/2387364195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f741814aff0 0x7f74180bf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:18.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f7417fff700 1 -- 172.21.15.121:0/2387364195 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74040092a0 con 0x7f741814ac10 2023-11-15T02:57:18.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f741d106700 1 --2- 172.21.15.121:0/2387364195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f741814aff0 0x7f74180bf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:18.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f7417fff700 1 --2- 172.21.15.121:0/2387364195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f741814ac10 0x7f741813ef10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f7404010000 tx=0x7f7404016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:18.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f7414ff9700 1 -- 172.21.15.121:0/2387364195 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740400d770 con 0x7f741814ac10 2023-11-15T02:57:18.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f7414ff9700 1 -- 172.21.15.121:0/2387364195 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740400d8d0 con 0x7f741814ac10 2023-11-15T02:57:18.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f741d907700 1 -- 172.21.15.121:0/2387364195 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74181466e0 con 0x7f741814ac10 2023-11-15T02:57:18.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f7414ff9700 1 -- 172.21.15.121:0/2387364195 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740401d660 con 0x7f741814ac10 2023-11-15T02:57:18.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.627+0000 7f741d907700 1 -- 172.21.15.121:0/2387364195 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7418167700 con 0x7f741814ac10 2023-11-15T02:57:18.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.631+0000 7f741d907700 1 -- 172.21.15.121:0/2387364195 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7418167ee0 con 0x7f741814ac10 2023-11-15T02:57:18.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.631+0000 7f7414ff9700 1 -- 172.21.15.121:0/2387364195 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f740401da50 con 0x7f741814ac10 2023-11-15T02:57:18.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.631+0000 7f7414ff9700 1 --2- 172.21.15.121:0/2387364195 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f73fc05a510 0x7f73fc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:18.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.631+0000 7f7414ff9700 1 -- 172.21.15.121:0/2387364195 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7404075f10 con 0x7f741814ac10 2023-11-15T02:57:18.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.631+0000 7f741c905700 1 --2- 172.21.15.121:0/2387364195 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f73fc05a510 0x7f73fc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:18.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.635+0000 7f741c905700 1 --2- 172.21.15.121:0/2387364195 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f73fc05a510 0x7f73fc05c9d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f740c005750 tx=0x7f740c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:18.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.639+0000 7f7414ff9700 1 -- 172.21.15.121:0/2387364195 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7404049630 con 0x7f741814ac10 2023-11-15T02:57:18.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.803+0000 7f741d907700 1 -- 172.21.15.121:0/2387364195 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7418065d20 con 0x7f741814ac10 2023-11-15T02:57:18.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.803+0000 7f7414ff9700 1 -- 172.21.15.121:0/2387364195 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v21) v1 ==== 110+0+10 (secure 0 0 0) 0x7f740404cc50 con 0x7f741814ac10 2023-11-15T02:57:18.809 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T02:57:18.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.807+0000 7f741d907700 1 -- 172.21.15.121:0/2387364195 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f73fc05a510 msgr2=0x7f73fc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:18.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.807+0000 7f741d907700 1 --2- 172.21.15.121:0/2387364195 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f73fc05a510 0x7f73fc05c9d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f740c005750 tx=0x7f740c009040).stop 2023-11-15T02:57:18.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.807+0000 7f741d907700 1 -- 172.21.15.121:0/2387364195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f741814ac10 msgr2=0x7f741813ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:18.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.807+0000 7f741d907700 1 --2- 172.21.15.121:0/2387364195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f741814ac10 0x7f741813ef10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f7404010000 tx=0x7f7404016000).stop 2023-11-15T02:57:18.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.807+0000 7f741d907700 1 -- 172.21.15.121:0/2387364195 shutdown_connections 2023-11-15T02:57:18.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.807+0000 7f741d907700 1 --2- 172.21.15.121:0/2387364195 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f73fc05a510 0x7f73fc05c9d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:18.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.807+0000 7f741d907700 1 --2- 172.21.15.121:0/2387364195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f741814aff0 0x7f74180bf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:18.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.807+0000 7f741d907700 1 --2- 172.21.15.121:0/2387364195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f741814ac10 0x7f741813ef10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:18.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.807+0000 7f741d907700 1 --2- 172.21.15.121:0/2387364195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7418067160 0x7f74180ac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:18.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.807+0000 7f741d907700 1 -- 172.21.15.121:0/2387364195 >> 172.21.15.121:0/2387364195 conn(0x7f7418067a60 msgr2=0x7f74180c4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:18.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.807+0000 7f741d907700 1 -- 172.21.15.121:0/2387364195 shutdown_connections 2023-11-15T02:57:18.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.807+0000 7f741d907700 1 -- 172.21.15.121:0/2387364195 wait complete. 2023-11-15T02:57:18.835 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T02:57:18.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.995+0000 7f29c2679700 1 Processor -- start 2023-11-15T02:57:18.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.995+0000 7f29c2679700 1 -- start start 2023-11-15T02:57:19.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.995+0000 7f29c2679700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f29bc0a8960 0x7f29bc0c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:19.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.995+0000 7f29c2679700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f29bc0a8d40 0x7f29bc15c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:19.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.995+0000 7f29c2679700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29bc066520 0x7f29bc0669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:19.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.995+0000 7f29c2679700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29bc1583d0 con 0x7f29bc0a8d40 2023-11-15T02:57:19.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.995+0000 7f29c2679700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29bc0657f0 con 0x7f29bc0a8960 2023-11-15T02:57:19.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.995+0000 7f29c2679700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0c1b60 con 0x7f29bc066520 2023-11-15T02:57:19.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.995+0000 7f29c0e76700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f29bc0a8d40 0x7f29bc15c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:19.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.995+0000 7f29c0e76700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f29bc0a8d40 0x7f29bc15c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39340/0 (socket says 172.21.15.121:39340) 2023-11-15T02:57:19.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.995+0000 7f29c0e76700 1 -- 172.21.15.121:0/1605089552 learned_addr learned my addr 172.21.15.121:0/1605089552 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:19.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29c1677700 1 --2- 172.21.15.121:0/1605089552 >> v2:172.21.15.190:3300/0 conn(0x7f29bc0a8960 0x7f29bc0c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:19.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29c1e78700 1 --2- 172.21.15.121:0/1605089552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29bc066520 0x7f29bc0669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:19.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29c0e76700 1 -- 172.21.15.121:0/1605089552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29bc066520 msgr2=0x7f29bc0669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:19.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29c0e76700 1 --2- 172.21.15.121:0/1605089552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29bc066520 0x7f29bc0669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29c0e76700 1 -- 172.21.15.121:0/1605089552 >> v2:172.21.15.190:3300/0 conn(0x7f29bc0a8960 msgr2=0x7f29bc0c15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:19.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29c0e76700 1 --2- 172.21.15.121:0/1605089552 >> v2:172.21.15.190:3300/0 conn(0x7f29bc0a8960 0x7f29bc0c15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29c0e76700 1 -- 172.21.15.121:0/1605089552 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc065a40 con 0x7f29bc0a8d40 2023-11-15T02:57:19.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29c1677700 1 --2- 172.21.15.121:0/1605089552 >> v2:172.21.15.190:3300/0 conn(0x7f29bc0a8960 0x7f29bc0c15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:57:19.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29c1e78700 1 --2- 172.21.15.121:0/1605089552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29bc066520 0x7f29bc0669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:57:19.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29c0e76700 1 --2- 172.21.15.121:0/1605089552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29bc0a8d40 0x7f29bc15c4c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f29bc063c80 tx=0x7f29ac0090f0).ready entity=mon.0 client_cookie=89a56d3770b9e1d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:19.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29b3fff700 1 -- 172.21.15.121:0/1605089552 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac00b600 con 0x7f29bc0a8d40 2023-11-15T02:57:19.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29b3fff700 1 -- 172.21.15.121:0/1605089552 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac00b760 con 0x7f29bc0a8d40 2023-11-15T02:57:19.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29b3fff700 1 -- 172.21.15.121:0/1605089552 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac00bae0 con 0x7f29bc0a8d40 2023-11-15T02:57:19.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29c2679700 1 -- 172.21.15.121:0/1605089552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29bc0a8d40 msgr2=0x7f29bc15c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:19.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29c2679700 1 --2- 172.21.15.121:0/1605089552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29bc0a8d40 0x7f29bc15c4c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f29bc063c80 tx=0x7f29ac0090f0).stop 2023-11-15T02:57:19.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29c2679700 1 -- 172.21.15.121:0/1605089552 shutdown_connections 2023-11-15T02:57:19.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29c2679700 1 --2- 172.21.15.121:0/1605089552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29bc066520 0x7f29bc0669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29c2679700 1 --2- 172.21.15.121:0/1605089552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29bc0a8d40 0x7f29bc15c4c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29c2679700 1 --2- 172.21.15.121:0/1605089552 >> v2:172.21.15.190:3300/0 conn(0x7f29bc0a8960 0x7f29bc0c15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29c2679700 1 -- 172.21.15.121:0/1605089552 >> 172.21.15.121:0/1605089552 conn(0x7f29bc067a60 msgr2=0x7f29bc067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:19.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29c2679700 1 -- 172.21.15.121:0/1605089552 shutdown_connections 2023-11-15T02:57:19.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:18.999+0000 7f29c2679700 1 -- 172.21.15.121:0/1605089552 wait complete. 2023-11-15T02:57:19.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.003+0000 7f29c2679700 1 Processor -- start 2023-11-15T02:57:19.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.003+0000 7f29c2679700 1 -- start start 2023-11-15T02:57:19.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.003+0000 7f29c2679700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29bc066520 0x7f29bc0c00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:19.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.003+0000 7f29c2679700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29bc0a8960 0x7f29bc158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:19.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.003+0000 7f29c2679700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29bc0a8d40 0x7f29bc1682a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:19.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.003+0000 7f29c2679700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0bb570 con 0x7f29bc0a8960 2023-11-15T02:57:19.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.003+0000 7f29c2679700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc168b40 con 0x7f29bc0a8d40 2023-11-15T02:57:19.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.003+0000 7f29c2679700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc168e20 con 0x7f29bc066520 2023-11-15T02:57:19.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.007+0000 7f29c0e76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29bc0a8960 0x7f29bc158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:19.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.007+0000 7f29c0e76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29bc0a8960 0x7f29bc158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39356/0 (socket says 172.21.15.121:39356) 2023-11-15T02:57:19.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.007+0000 7f29c0e76700 1 -- 172.21.15.121:0/3908151005 learned_addr learned my addr 172.21.15.121:0/3908151005 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:19.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.007+0000 7f29c1677700 1 --2- 172.21.15.121:0/3908151005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29bc066520 0x7f29bc0c00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:19.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.007+0000 7f29c1e78700 1 --2- 172.21.15.121:0/3908151005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29bc0a8d40 0x7f29bc1682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:19.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.007+0000 7f29c0e76700 1 -- 172.21.15.121:0/3908151005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29bc066520 msgr2=0x7f29bc0c00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:19.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.007+0000 7f29c0e76700 1 --2- 172.21.15.121:0/3908151005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29bc066520 0x7f29bc0c00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.007+0000 7f29c0e76700 1 -- 172.21.15.121:0/3908151005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29bc0a8d40 msgr2=0x7f29bc1682a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:19.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.007+0000 7f29c0e76700 1 --2- 172.21.15.121:0/3908151005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29bc0a8d40 0x7f29bc1682a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.007+0000 7f29c0e76700 1 -- 172.21.15.121:0/3908151005 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ac009370 con 0x7f29bc0a8960 2023-11-15T02:57:19.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.007+0000 7f29c0e76700 1 --2- 172.21.15.121:0/3908151005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29bc0a8960 0x7f29bc158510 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f29ac009130 tx=0x7f29ac017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:19.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.007+0000 7f29b17fa700 1 -- 172.21.15.121:0/3908151005 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac00b8d0 con 0x7f29bc0a8960 2023-11-15T02:57:19.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.007+0000 7f29b17fa700 1 -- 172.21.15.121:0/3908151005 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac00ba30 con 0x7f29bc0a8960 2023-11-15T02:57:19.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.007+0000 7f29c2679700 1 -- 172.21.15.121:0/3908151005 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29bc169100 con 0x7f29bc0a8960 2023-11-15T02:57:19.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.007+0000 7f29b17fa700 1 -- 172.21.15.121:0/3908151005 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac01c670 con 0x7f29bc0a8960 2023-11-15T02:57:19.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.007+0000 7f29c2679700 1 -- 172.21.15.121:0/3908151005 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29bc158a00 con 0x7f29bc0a8960 2023-11-15T02:57:19.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.011+0000 7f29c2679700 1 -- 172.21.15.121:0/3908151005 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29bc0bb9e0 con 0x7f29bc0a8960 2023-11-15T02:57:19.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.011+0000 7f29b17fa700 1 -- 172.21.15.121:0/3908151005 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f29ac01ca20 con 0x7f29bc0a8960 2023-11-15T02:57:19.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.011+0000 7f29b17fa700 1 --2- 172.21.15.121:0/3908151005 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f29a005a510 0x7f29a005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:19.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.011+0000 7f29b17fa700 1 -- 172.21.15.121:0/3908151005 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f29ac076460 con 0x7f29bc0a8960 2023-11-15T02:57:19.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.011+0000 7f29c1677700 1 --2- 172.21.15.121:0/3908151005 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f29a005a510 0x7f29a005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:19.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.011+0000 7f29c1677700 1 --2- 172.21.15.121:0/3908151005 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f29a005a510 0x7f29a005c9d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f29b800abd0 tx=0x7f29b8008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:19.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.019+0000 7f29b17fa700 1 -- 172.21.15.121:0/3908151005 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f29ac049c40 con 0x7f29bc0a8960 2023-11-15T02:57:19.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.179+0000 7f29c2679700 1 -- 172.21.15.121:0/3908151005 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f29bc06bcf0 con 0x7f29bc0a8960 2023-11-15T02:57:19.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.179+0000 7f29b17fa700 1 -- 172.21.15.121:0/3908151005 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f29ac02a0a0 con 0x7f29bc0a8960 2023-11-15T02:57:19.186 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T02:57:19.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.183+0000 7f29c2679700 1 -- 172.21.15.121:0/3908151005 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f29a005a510 msgr2=0x7f29a005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:19.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.183+0000 7f29c2679700 1 --2- 172.21.15.121:0/3908151005 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f29a005a510 0x7f29a005c9d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f29b800abd0 tx=0x7f29b8008040).stop 2023-11-15T02:57:19.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.183+0000 7f29c2679700 1 -- 172.21.15.121:0/3908151005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29bc0a8960 msgr2=0x7f29bc158510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:19.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.183+0000 7f29c2679700 1 --2- 172.21.15.121:0/3908151005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29bc0a8960 0x7f29bc158510 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f29ac009130 tx=0x7f29ac017000).stop 2023-11-15T02:57:19.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.183+0000 7f29c2679700 1 -- 172.21.15.121:0/3908151005 shutdown_connections 2023-11-15T02:57:19.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.183+0000 7f29c2679700 1 --2- 172.21.15.121:0/3908151005 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f29a005a510 0x7f29a005c9d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.183+0000 7f29c2679700 1 --2- 172.21.15.121:0/3908151005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29bc0a8d40 0x7f29bc1682a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.183+0000 7f29c2679700 1 --2- 172.21.15.121:0/3908151005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29bc0a8960 0x7f29bc158510 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.183+0000 7f29c2679700 1 --2- 172.21.15.121:0/3908151005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29bc066520 0x7f29bc0c00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.183+0000 7f29c2679700 1 -- 172.21.15.121:0/3908151005 >> 172.21.15.121:0/3908151005 conn(0x7f29bc067a60 msgr2=0x7f29bc0aa640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:19.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.187+0000 7f29c2679700 1 -- 172.21.15.121:0/3908151005 shutdown_connections 2023-11-15T02:57:19.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.187+0000 7f29c2679700 1 -- 172.21.15.121:0/3908151005 wait complete. 2023-11-15T02:57:19.211 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-15T02:57:19.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.363+0000 7fb5b011b700 1 Processor -- start 2023-11-15T02:57:19.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.363+0000 7fb5b011b700 1 -- start start 2023-11-15T02:57:19.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5b011b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb5a8157ed0 0x7fb5a81582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:19.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5b011b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb5a8067020 0x7fb5a80bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:19.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5b011b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5a8062da0 con 0x7fb5a8067a60 2023-11-15T02:57:19.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5b011b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a80a93a0 con 0x7fb5a8157ed0 2023-11-15T02:57:19.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5b011b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a8062190 con 0x7fb5a8067020 2023-11-15T02:57:19.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5adeb7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb5a8157ed0 0x7fb5a81582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:19.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5adeb7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb5a8157ed0 0x7fb5a81582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39360/0 (socket says 172.21.15.121:39360) 2023-11-15T02:57:19.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5adeb7700 1 -- 172.21.15.121:0/1711489389 learned_addr learned my addr 172.21.15.121:0/1711489389 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:19.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5ae6b8700 1 --2- 172.21.15.121:0/1711489389 >> v2:172.21.15.190:3300/0 conn(0x7fb5a8067020 0x7fb5a80bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:19.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5adeb7700 1 -- 172.21.15.121:0/1711489389 >> v2:172.21.15.190:3300/0 conn(0x7fb5a8067020 msgr2=0x7fb5a80bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:19.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5adeb7700 1 --2- 172.21.15.121:0/1711489389 >> v2:172.21.15.190:3300/0 conn(0x7fb5a8067020 0x7fb5a80bc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5adeb7700 1 -- 172.21.15.121:0/1711489389 >> v1:172.21.15.190:6789/0 conn(0x7fb5a8067a60 legacy=0x7fb5a8067e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:19.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5adeb7700 1 -- 172.21.15.121:0/1711489389 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a8062470 con 0x7fb5a8157ed0 2023-11-15T02:57:19.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5ae6b8700 1 --2- 172.21.15.121:0/1711489389 >> v2:172.21.15.190:3300/0 conn(0x7fb5a8067020 0x7fb5a80bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:19.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5adeb7700 1 --2- 172.21.15.121:0/1711489389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5a8157ed0 0x7fb5a81582b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fb5a8063690 tx=0x7fb5a40090f0).ready entity=mon.0 client_cookie=8ac3b4bb421048ef server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:19.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5aceb5700 1 -- 172.21.15.121:0/1711489389 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a400b600 con 0x7fb5a8157ed0 2023-11-15T02:57:19.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5aceb5700 1 -- 172.21.15.121:0/1711489389 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a400b760 con 0x7fb5a8157ed0 2023-11-15T02:57:19.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5aceb5700 1 -- 172.21.15.121:0/1711489389 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a400bae0 con 0x7fb5a8157ed0 2023-11-15T02:57:19.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5b011b700 1 -- 172.21.15.121:0/1711489389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5a8157ed0 msgr2=0x7fb5a81582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:19.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5b011b700 1 --2- 172.21.15.121:0/1711489389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5a8157ed0 0x7fb5a81582b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fb5a8063690 tx=0x7fb5a40090f0).stop 2023-11-15T02:57:19.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5b011b700 1 -- 172.21.15.121:0/1711489389 shutdown_connections 2023-11-15T02:57:19.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5b011b700 1 --2- 172.21.15.121:0/1711489389 >> v2:172.21.15.190:3300/0 conn(0x7fb5a8067020 0x7fb5a80bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5b011b700 1 --2- 172.21.15.121:0/1711489389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5a8157ed0 0x7fb5a81582b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5b011b700 1 -- 172.21.15.121:0/1711489389 >> 172.21.15.121:0/1711489389 conn(0x7fb5a80591c0 msgr2=0x7fb5a80c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:19.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.367+0000 7fb5b011b700 1 -- 172.21.15.121:0/1711489389 shutdown_connections 2023-11-15T02:57:19.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.371+0000 7fb5b011b700 1 -- 172.21.15.121:0/1711489389 wait complete. 2023-11-15T02:57:19.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.371+0000 7fb5b011b700 1 Processor -- start 2023-11-15T02:57:19.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.371+0000 7fb5b011b700 1 -- start start 2023-11-15T02:57:19.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb5b011b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5a8067020 0x7fb5a8141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:19.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb5b011b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5a8067a60 0x7fb5a80bf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:19.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb5b011b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5a8157ed0 0x7fb5a80bfe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:19.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb5b011b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a80c0360 con 0x7fb5a8067a60 2023-11-15T02:57:19.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb5b011b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a8143f90 con 0x7fb5a8157ed0 2023-11-15T02:57:19.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb5b011b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a8143380 con 0x7fb5a8067020 2023-11-15T02:57:19.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb5ad6b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5a8067a60 0x7fb5a80bf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:19.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb5ad6b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5a8067a60 0x7fb5a80bf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39374/0 (socket says 172.21.15.121:39374) 2023-11-15T02:57:19.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb5ad6b6700 1 -- 172.21.15.121:0/4098259467 learned_addr learned my addr 172.21.15.121:0/4098259467 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:19.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb5ae6b8700 1 --2- 172.21.15.121:0/4098259467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5a8157ed0 0x7fb5a80bfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:19.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb5adeb7700 1 --2- 172.21.15.121:0/4098259467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5a8067020 0x7fb5a8141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:19.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb5ad6b6700 1 -- 172.21.15.121:0/4098259467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5a8067020 msgr2=0x7fb5a8141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:19.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb5ad6b6700 1 --2- 172.21.15.121:0/4098259467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5a8067020 0x7fb5a8141ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb5ad6b6700 1 -- 172.21.15.121:0/4098259467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5a8157ed0 msgr2=0x7fb5a80bfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:19.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb5ad6b6700 1 --2- 172.21.15.121:0/4098259467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5a8157ed0 0x7fb5a80bfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb5ad6b6700 1 -- 172.21.15.121:0/4098259467 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a4009370 con 0x7fb5a8067a60 2023-11-15T02:57:19.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb5ad6b6700 1 --2- 172.21.15.121:0/4098259467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5a8067a60 0x7fb5a80bf980 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fb59800c4c0 tx=0x7fb598007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:19.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb59dffb700 1 -- 172.21.15.121:0/4098259467 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59800ce70 con 0x7fb5a8067a60 2023-11-15T02:57:19.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb59dffb700 1 -- 172.21.15.121:0/4098259467 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb598001e90 con 0x7fb5a8067a60 2023-11-15T02:57:19.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb59dffb700 1 -- 172.21.15.121:0/4098259467 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb598007250 con 0x7fb5a8067a60 2023-11-15T02:57:19.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb5b011b700 1 -- 172.21.15.121:0/4098259467 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a8143660 con 0x7fb5a8067a60 2023-11-15T02:57:19.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.375+0000 7fb5b011b700 1 -- 172.21.15.121:0/4098259467 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a8143c20 con 0x7fb5a8067a60 2023-11-15T02:57:19.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.379+0000 7fb5b011b700 1 -- 172.21.15.121:0/4098259467 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a8144120 con 0x7fb5a8067a60 2023-11-15T02:57:19.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.379+0000 7fb59dffb700 1 -- 172.21.15.121:0/4098259467 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb5980066a0 con 0x7fb5a8067a60 2023-11-15T02:57:19.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.379+0000 7fb59dffb700 1 --2- 172.21.15.121:0/4098259467 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb58c05a540 0x7fb58c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:19.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.379+0000 7fb59dffb700 1 -- 172.21.15.121:0/4098259467 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb598074100 con 0x7fb5a8067a60 2023-11-15T02:57:19.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.379+0000 7fb5adeb7700 1 --2- 172.21.15.121:0/4098259467 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb58c05a540 0x7fb58c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:19.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.383+0000 7fb5adeb7700 1 --2- 172.21.15.121:0/4098259467 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb58c05a540 0x7fb58c05ca00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fb5a4009fd0 tx=0x7fb5a4017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:19.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.387+0000 7fb59dffb700 1 -- 172.21.15.121:0/4098259467 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb598047820 con 0x7fb5a8067a60 2023-11-15T02:57:19.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.547+0000 7fb5b011b700 1 -- 172.21.15.121:0/4098259467 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5a8062750 con 0x7fb5a8067a60 2023-11-15T02:57:19.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.551+0000 7fb59dffb700 1 -- 172.21.15.121:0/4098259467 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb59804ae40 con 0x7fb5a8067a60 2023-11-15T02:57:19.556 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T02:57:19.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.555+0000 7fb5b011b700 1 -- 172.21.15.121:0/4098259467 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb58c05a540 msgr2=0x7fb58c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:19.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.555+0000 7fb5b011b700 1 --2- 172.21.15.121:0/4098259467 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb58c05a540 0x7fb58c05ca00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fb5a4009fd0 tx=0x7fb5a4017000).stop 2023-11-15T02:57:19.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.555+0000 7fb5b011b700 1 -- 172.21.15.121:0/4098259467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5a8067a60 msgr2=0x7fb5a80bf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:19.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.555+0000 7fb5b011b700 1 --2- 172.21.15.121:0/4098259467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5a8067a60 0x7fb5a80bf980 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fb59800c4c0 tx=0x7fb598007e00).stop 2023-11-15T02:57:19.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.555+0000 7fb5b011b700 1 -- 172.21.15.121:0/4098259467 shutdown_connections 2023-11-15T02:57:19.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.555+0000 7fb5b011b700 1 --2- 172.21.15.121:0/4098259467 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb58c05a540 0x7fb58c05ca00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.555+0000 7fb5b011b700 1 --2- 172.21.15.121:0/4098259467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5a8157ed0 0x7fb5a80bfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.555+0000 7fb5b011b700 1 --2- 172.21.15.121:0/4098259467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5a8067a60 0x7fb5a80bf980 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.555+0000 7fb5b011b700 1 --2- 172.21.15.121:0/4098259467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5a8067020 0x7fb5a8141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.555+0000 7fb5b011b700 1 -- 172.21.15.121:0/4098259467 >> 172.21.15.121:0/4098259467 conn(0x7fb5a80591c0 msgr2=0x7fb5a80c1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:19.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.555+0000 7fb5b011b700 1 -- 172.21.15.121:0/4098259467 shutdown_connections 2023-11-15T02:57:19.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.555+0000 7fb5b011b700 1 -- 172.21.15.121:0/4098259467 wait complete. 2023-11-15T02:57:19.581 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-15T02:57:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.739+0000 7f19262f1700 1 Processor -- start 2023-11-15T02:57:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.739+0000 7f19262f1700 1 -- start start 2023-11-15T02:57:19.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.743+0000 7f19262f1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f19200a8e00 0x7f1920066010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:19.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.743+0000 7f19262f1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1920066500 0x7f1920062830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:19.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.743+0000 7f19262f1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1920154b10 con 0x7f1920067100 2023-11-15T02:57:19.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.743+0000 7f19262f1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1920154df0 con 0x7f1920066500 2023-11-15T02:57:19.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.743+0000 7f19262f1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19201550d0 con 0x7f19200a8e00 2023-11-15T02:57:19.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.743+0000 7f191f7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f19200a8e00 0x7f1920066010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:19.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.743+0000 7f192488e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1920066500 0x7f1920062830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:19.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.743+0000 7f191f7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f19200a8e00 0x7f1920066010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:54890/0 (socket says 172.21.15.121:54890) 2023-11-15T02:57:19.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.743+0000 7f192488e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1920066500 0x7f1920062830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39388/0 (socket says 172.21.15.121:39388) 2023-11-15T02:57:19.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.743+0000 7f191ffff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f1920067100 0x7f1920154310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:57016/0 (socket says 172.21.15.121:57016) 2023-11-15T02:57:19.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.743+0000 7f191f7fe700 1 -- 172.21.15.121:0/2910577633 learned_addr learned my addr 172.21.15.121:0/2910577633 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:19.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.743+0000 7f191f7fe700 1 -- 172.21.15.121:0/2910577633 >> v2:172.21.15.121:3300/0 conn(0x7f1920066500 msgr2=0x7f1920062830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:19.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.743+0000 7f191f7fe700 1 --2- 172.21.15.121:0/2910577633 >> v2:172.21.15.121:3300/0 conn(0x7f1920066500 0x7f1920062830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.743+0000 7f191f7fe700 1 -- 172.21.15.121:0/2910577633 >> v1:172.21.15.190:6789/0 conn(0x7f1920067100 legacy=0x7f1920154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:19.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.743+0000 7f191f7fe700 1 -- 172.21.15.121:0/2910577633 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1920062310 con 0x7f19200a8e00 2023-11-15T02:57:19.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.743+0000 7f192488e700 1 --2- 172.21.15.121:0/2910577633 >> v2:172.21.15.121:3300/0 conn(0x7f1920066500 0x7f1920062830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:57:19.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.743+0000 7f191f7fe700 1 --2- 172.21.15.121:0/2910577633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19200a8e00 0x7f1920066010 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f19200c7800 tx=0x7f19140090f0).ready entity=mon.1 client_cookie=6eeae376c24d1dc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:19.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.747+0000 7f191effd700 1 -- 172.21.15.121:0/2910577633 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191400b520 con 0x7f19200a8e00 2023-11-15T02:57:19.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.747+0000 7f191effd700 1 -- 172.21.15.121:0/2910577633 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191400b680 con 0x7f19200a8e00 2023-11-15T02:57:19.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.747+0000 7f191effd700 1 -- 172.21.15.121:0/2910577633 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191400ba20 con 0x7f19200a8e00 2023-11-15T02:57:19.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.747+0000 7f19262f1700 1 -- 172.21.15.121:0/2910577633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19200a8e00 msgr2=0x7f1920066010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:19.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.747+0000 7f19262f1700 1 --2- 172.21.15.121:0/2910577633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19200a8e00 0x7f1920066010 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f19200c7800 tx=0x7f19140090f0).stop 2023-11-15T02:57:19.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.747+0000 7f19262f1700 1 -- 172.21.15.121:0/2910577633 shutdown_connections 2023-11-15T02:57:19.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.747+0000 7f19262f1700 1 --2- 172.21.15.121:0/2910577633 >> v2:172.21.15.121:3300/0 conn(0x7f1920066500 0x7f1920062830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.747+0000 7f19262f1700 1 --2- 172.21.15.121:0/2910577633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19200a8e00 0x7f1920066010 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.747+0000 7f19262f1700 1 -- 172.21.15.121:0/2910577633 >> 172.21.15.121:0/2910577633 conn(0x7f19200591c0 msgr2=0x7f19200c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:19.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.747+0000 7f19262f1700 1 -- 172.21.15.121:0/2910577633 shutdown_connections 2023-11-15T02:57:19.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.747+0000 7f19262f1700 1 -- 172.21.15.121:0/2910577633 wait complete. 2023-11-15T02:57:19.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.751+0000 7f19262f1700 1 Processor -- start 2023-11-15T02:57:19.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.751+0000 7f19262f1700 1 -- start start 2023-11-15T02:57:19.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.751+0000 7f19262f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1920066500 0x7f1920164100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:19.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.751+0000 7f19262f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1920067100 0x7f19200a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:19.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.751+0000 7f19262f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19200a8e00 0x7f19200a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:19.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.751+0000 7f19262f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19201645f0 con 0x7f1920066500 2023-11-15T02:57:19.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.751+0000 7f19262f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19201650a0 con 0x7f1920067100 2023-11-15T02:57:19.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.751+0000 7f19262f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19200a3f40 con 0x7f19200a8e00 2023-11-15T02:57:19.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.751+0000 7f191ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1920066500 0x7f1920164100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:19.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.751+0000 7f191ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1920066500 0x7f1920164100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39390/0 (socket says 172.21.15.121:39390) 2023-11-15T02:57:19.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.751+0000 7f191ffff700 1 -- 172.21.15.121:0/507893392 learned_addr learned my addr 172.21.15.121:0/507893392 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:19.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.751+0000 7f191f7fe700 1 --2- 172.21.15.121:0/507893392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1920067100 0x7f19200a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:19.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.751+0000 7f192488e700 1 --2- 172.21.15.121:0/507893392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19200a8e00 0x7f19200a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:19.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.751+0000 7f191ffff700 1 -- 172.21.15.121:0/507893392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19200a8e00 msgr2=0x7f19200a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:19.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.751+0000 7f191ffff700 1 --2- 172.21.15.121:0/507893392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19200a8e00 0x7f19200a6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.751+0000 7f191ffff700 1 -- 172.21.15.121:0/507893392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1920067100 msgr2=0x7f19200a3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:19.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.751+0000 7f191ffff700 1 --2- 172.21.15.121:0/507893392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1920067100 0x7f19200a3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.751+0000 7f191ffff700 1 -- 172.21.15.121:0/507893392 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1918009a20 con 0x7f1920066500 2023-11-15T02:57:19.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.755+0000 7f191ffff700 1 --2- 172.21.15.121:0/507893392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1920066500 0x7f1920164100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f191000c4c0 tx=0x7f1910007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:19.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.755+0000 7f1907fff700 1 -- 172.21.15.121:0/507893392 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19100072e0 con 0x7f1920066500 2023-11-15T02:57:19.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.755+0000 7f1907fff700 1 -- 172.21.15.121:0/507893392 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1910007440 con 0x7f1920066500 2023-11-15T02:57:19.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.755+0000 7f1907fff700 1 -- 172.21.15.121:0/507893392 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19100058f0 con 0x7f1920066500 2023-11-15T02:57:19.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.755+0000 7f19262f1700 1 -- 172.21.15.121:0/507893392 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1914009370 con 0x7f1920066500 2023-11-15T02:57:19.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.755+0000 7f19262f1700 1 -- 172.21.15.121:0/507893392 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1920164dc0 con 0x7f1920066500 2023-11-15T02:57:19.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.755+0000 7f19262f1700 1 -- 172.21.15.121:0/507893392 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1920047060 con 0x7f1920066500 2023-11-15T02:57:19.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.755+0000 7f1907fff700 1 -- 172.21.15.121:0/507893392 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1910005ce0 con 0x7f1920066500 2023-11-15T02:57:19.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.759+0000 7f1907fff700 1 --2- 172.21.15.121:0/507893392 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f190c05a560 0x7f190c05ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:19.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.759+0000 7f1907fff700 1 -- 172.21.15.121:0/507893392 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1910073d60 con 0x7f1920066500 2023-11-15T02:57:19.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.759+0000 7f191f7fe700 1 --2- 172.21.15.121:0/507893392 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f190c05a560 0x7f190c05ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:19.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.759+0000 7f191f7fe700 1 --2- 172.21.15.121:0/507893392 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f190c05a560 0x7f190c05ca20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f191400b2e0 tx=0x7f1914017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:19.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.763+0000 7f1907fff700 1 -- 172.21.15.121:0/507893392 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f19100475e0 con 0x7f1920066500 2023-11-15T02:57:19.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.919+0000 7f19262f1700 1 -- 172.21.15.121:0/507893392 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1920068b50 con 0x7f1920066500 2023-11-15T02:57:19.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.923+0000 7f1907fff700 1 -- 172.21.15.121:0/507893392 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f191004ac00 con 0x7f1920066500 2023-11-15T02:57:19.928 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T02:57:19.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.923+0000 7f19262f1700 1 -- 172.21.15.121:0/507893392 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f190c05a560 msgr2=0x7f190c05ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:19.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.923+0000 7f19262f1700 1 --2- 172.21.15.121:0/507893392 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f190c05a560 0x7f190c05ca20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f191400b2e0 tx=0x7f1914017000).stop 2023-11-15T02:57:19.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.923+0000 7f19262f1700 1 -- 172.21.15.121:0/507893392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1920066500 msgr2=0x7f1920164100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:19.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.923+0000 7f19262f1700 1 --2- 172.21.15.121:0/507893392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1920066500 0x7f1920164100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f191000c4c0 tx=0x7f1910007db0).stop 2023-11-15T02:57:19.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.923+0000 7f19262f1700 1 -- 172.21.15.121:0/507893392 shutdown_connections 2023-11-15T02:57:19.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.923+0000 7f19262f1700 1 --2- 172.21.15.121:0/507893392 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f190c05a560 0x7f190c05ca20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.923+0000 7f19262f1700 1 --2- 172.21.15.121:0/507893392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19200a8e00 0x7f19200a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.923+0000 7f19262f1700 1 --2- 172.21.15.121:0/507893392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1920067100 0x7f19200a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.923+0000 7f19262f1700 1 --2- 172.21.15.121:0/507893392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1920066500 0x7f1920164100 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:19.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.923+0000 7f19262f1700 1 -- 172.21.15.121:0/507893392 >> 172.21.15.121:0/507893392 conn(0x7f19200591c0 msgr2=0x7f1920153f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:19.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.927+0000 7f19262f1700 1 -- 172.21.15.121:0/507893392 shutdown_connections 2023-11-15T02:57:19.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:19.927+0000 7f19262f1700 1 -- 172.21.15.121:0/507893392 wait complete. 2023-11-15T02:57:19.950 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:57:20.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.107+0000 7fb79b307700 1 Processor -- start 2023-11-15T02:57:20.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.107+0000 7fb79b307700 1 -- start start 2023-11-15T02:57:20.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.111+0000 7fb79b307700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb7940cabd0 0x7fb7940cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:20.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.111+0000 7fb79b307700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7940ad650 con 0x7fb7940ad0f0 2023-11-15T02:57:20.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.111+0000 7fb79b307700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb794065fc0 con 0x7fb7940acc90 2023-11-15T02:57:20.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.111+0000 7fb79b307700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7940662a0 con 0x7fb7940cabd0 2023-11-15T02:57:20.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.111+0000 7fb79a305700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb7940cabd0 0x7fb7940cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:20.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.111+0000 7fb79a305700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb7940cabd0 0x7fb7940cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:54904/0 (socket says 172.21.15.121:54904) 2023-11-15T02:57:20.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.111+0000 7fb79ab06700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb7940ad0f0 0x7fb7940ae190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:48180/0 (socket says 172.21.15.121:48180) 2023-11-15T02:57:20.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.111+0000 7fb79a305700 1 -- 172.21.15.121:0/4193568180 learned_addr learned my addr 172.21.15.121:0/4193568180 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:20.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.111+0000 7fb799b04700 1 --1- 172.21.15.121:0/4193568180 >> v1:172.21.15.190:6789/0 conn(0x7fb7940acc90 0x7fb794066d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:57022/0 (socket says 172.21.15.121:57022) 2023-11-15T02:57:20.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.111+0000 7fb799303700 1 -- 172.21.15.121:0/4193568180 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618536771 0 0) 0x7fb7940ad650 con 0x7fb7940ad0f0 2023-11-15T02:57:20.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.111+0000 7fb799303700 1 -- 172.21.15.121:0/4193568180 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7800033f0 con 0x7fb7940ad0f0 2023-11-15T02:57:20.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.111+0000 7fb799303700 1 -- 172.21.15.121:0/4193568180 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3089693800 0 0) 0x7fb794065fc0 con 0x7fb7940acc90 2023-11-15T02:57:20.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.111+0000 7fb79a305700 1 -- 172.21.15.121:0/4193568180 >> v1:172.21.15.190:6789/0 conn(0x7fb7940acc90 legacy=0x7fb794066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:20.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.111+0000 7fb79a305700 1 -- 172.21.15.121:0/4193568180 >> v1:172.21.15.121:6789/0 conn(0x7fb7940ad0f0 legacy=0x7fb7940ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:20.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.111+0000 7fb79a305700 1 -- 172.21.15.121:0/4193568180 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb794066580 con 0x7fb7940cabd0 2023-11-15T02:57:20.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.111+0000 7fb79a305700 1 --2- 172.21.15.121:0/4193568180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7940cabd0 0x7fb7940cafb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fb790006fd0 tx=0x7fb7900090f0).ready entity=mon.1 client_cookie=2db8aa49b68587f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:20.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.111+0000 7fb799303700 1 -- 172.21.15.121:0/4193568180 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79000b580 con 0x7fb7940cabd0 2023-11-15T02:57:20.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.111+0000 7fb799303700 1 -- 172.21.15.121:0/4193568180 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79000b6e0 con 0x7fb7940cabd0 2023-11-15T02:57:20.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.111+0000 7fb799303700 1 -- 172.21.15.121:0/4193568180 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79000ba60 con 0x7fb7940cabd0 2023-11-15T02:57:20.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.115+0000 7fb79b307700 1 -- 172.21.15.121:0/4193568180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7940cabd0 msgr2=0x7fb7940cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:20.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.115+0000 7fb79b307700 1 --2- 172.21.15.121:0/4193568180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7940cabd0 0x7fb7940cafb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fb790006fd0 tx=0x7fb7900090f0).stop 2023-11-15T02:57:20.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.115+0000 7fb79b307700 1 -- 172.21.15.121:0/4193568180 shutdown_connections 2023-11-15T02:57:20.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.115+0000 7fb79b307700 1 --2- 172.21.15.121:0/4193568180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7940cabd0 0x7fb7940cafb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:20.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.115+0000 7fb79b307700 1 -- 172.21.15.121:0/4193568180 >> 172.21.15.121:0/4193568180 conn(0x7fb794062190 msgr2=0x7fb7940625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:20.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.115+0000 7fb79b307700 1 -- 172.21.15.121:0/4193568180 shutdown_connections 2023-11-15T02:57:20.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.115+0000 7fb79b307700 1 -- 172.21.15.121:0/4193568180 wait complete. 2023-11-15T02:57:20.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.119+0000 7fb79b307700 1 Processor -- start 2023-11-15T02:57:20.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.119+0000 7fb79b307700 1 -- start start 2023-11-15T02:57:20.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.119+0000 7fb79b307700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7940acc90 0x7fb794144280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:20.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb79b307700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7940ad0f0 0x7fb7940bfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:20.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb79b307700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7940cabd0 0x7fb7940c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:20.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb79b307700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb794155300 con 0x7fb7940cabd0 2023-11-15T02:57:20.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb79b307700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7941555e0 con 0x7fb7940acc90 2023-11-15T02:57:20.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb79b307700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7940c3370 con 0x7fb7940ad0f0 2023-11-15T02:57:20.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb79ab06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7940cabd0 0x7fb7940c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:20.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb79a305700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7940acc90 0x7fb794144280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:20.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb79ab06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7940cabd0 0x7fb7940c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39402/0 (socket says 172.21.15.121:39402) 2023-11-15T02:57:20.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb79ab06700 1 -- 172.21.15.121:0/3138889379 learned_addr learned my addr 172.21.15.121:0/3138889379 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:20.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb799b04700 1 --2- 172.21.15.121:0/3138889379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7940ad0f0 0x7fb7940bfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:20.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb79ab06700 1 -- 172.21.15.121:0/3138889379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7940ad0f0 msgr2=0x7fb7940bfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:20.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb79ab06700 1 --2- 172.21.15.121:0/3138889379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7940ad0f0 0x7fb7940bfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:20.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb79ab06700 1 -- 172.21.15.121:0/3138889379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7940acc90 msgr2=0x7fb794144280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:20.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb79ab06700 1 --2- 172.21.15.121:0/3138889379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7940acc90 0x7fb794144280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:20.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb79ab06700 1 -- 172.21.15.121:0/3138889379 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb790009370 con 0x7fb7940cabd0 2023-11-15T02:57:20.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb79ab06700 1 --2- 172.21.15.121:0/3138889379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7940cabd0 0x7fb7940c93c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fb78c00c7c0 tx=0x7fb78c007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:20.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb78a7fc700 1 -- 172.21.15.121:0/3138889379 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c005c70 con 0x7fb7940cabd0 2023-11-15T02:57:20.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb78a7fc700 1 -- 172.21.15.121:0/3138889379 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78c005dd0 con 0x7fb7940cabd0 2023-11-15T02:57:20.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb79b307700 1 -- 172.21.15.121:0/3138889379 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7940c3650 con 0x7fb7940cabd0 2023-11-15T02:57:20.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb78a7fc700 1 -- 172.21.15.121:0/3138889379 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c007320 con 0x7fb7940cabd0 2023-11-15T02:57:20.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb79b307700 1 -- 172.21.15.121:0/3138889379 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb794144770 con 0x7fb7940cabd0 2023-11-15T02:57:20.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.123+0000 7fb79b307700 1 -- 172.21.15.121:0/3138889379 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7940c31d0 con 0x7fb7940cabd0 2023-11-15T02:57:20.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.127+0000 7fb78a7fc700 1 -- 172.21.15.121:0/3138889379 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb78c007750 con 0x7fb7940cabd0 2023-11-15T02:57:20.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.127+0000 7fb78a7fc700 1 --2- 172.21.15.121:0/3138889379 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb77805a510 0x7fb77805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:20.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.127+0000 7fb78a7fc700 1 -- 172.21.15.121:0/3138889379 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb78c074800 con 0x7fb7940cabd0 2023-11-15T02:57:20.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.127+0000 7fb79a305700 1 --2- 172.21.15.121:0/3138889379 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb77805a510 0x7fb77805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:20.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.131+0000 7fb79a305700 1 --2- 172.21.15.121:0/3138889379 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb77805a510 0x7fb77805c9d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fb790009130 tx=0x7fb790017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:20.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.135+0000 7fb78a7fc700 1 -- 172.21.15.121:0/3138889379 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb7940c31d0 con 0x7fb7940cabd0 2023-11-15T02:57:20.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.299+0000 7fb79b307700 1 -- 172.21.15.121:0/3138889379 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7940c3b50 con 0x7fb7940cabd0 2023-11-15T02:57:20.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.303+0000 7fb78a7fc700 1 -- 172.21.15.121:0/3138889379 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fb78c047f20 con 0x7fb7940cabd0 2023-11-15T02:57:20.309 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:20.309 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":21,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:52.702668+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:56:49.726353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-15T02:56:50.688098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:57:20.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.307+0000 7fb79b307700 1 -- 172.21.15.121:0/3138889379 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb77805a510 msgr2=0x7fb77805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:20.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.307+0000 7fb79b307700 1 --2- 172.21.15.121:0/3138889379 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb77805a510 0x7fb77805c9d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fb790009130 tx=0x7fb790017000).stop 2023-11-15T02:57:20.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.307+0000 7fb79b307700 1 -- 172.21.15.121:0/3138889379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7940cabd0 msgr2=0x7fb7940c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:20.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.307+0000 7fb79b307700 1 --2- 172.21.15.121:0/3138889379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7940cabd0 0x7fb7940c93c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fb78c00c7c0 tx=0x7fb78c007f90).stop 2023-11-15T02:57:20.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.307+0000 7fb79b307700 1 -- 172.21.15.121:0/3138889379 shutdown_connections 2023-11-15T02:57:20.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.307+0000 7fb79b307700 1 --2- 172.21.15.121:0/3138889379 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb77805a510 0x7fb77805c9d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:20.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.307+0000 7fb79b307700 1 --2- 172.21.15.121:0/3138889379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7940cabd0 0x7fb7940c93c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:20.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.307+0000 7fb79b307700 1 --2- 172.21.15.121:0/3138889379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7940ad0f0 0x7fb7940bfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:20.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.307+0000 7fb79b307700 1 --2- 172.21.15.121:0/3138889379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7940acc90 0x7fb794144280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:20.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.307+0000 7fb79b307700 1 -- 172.21.15.121:0/3138889379 >> 172.21.15.121:0/3138889379 conn(0x7fb794062190 msgr2=0x7fb794152710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:20.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.307+0000 7fb79b307700 1 -- 172.21.15.121:0/3138889379 shutdown_connections 2023-11-15T02:57:20.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.307+0000 7fb79b307700 1 -- 172.21.15.121:0/3138889379 wait complete. 2023-11-15T02:57:20.334 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-15T02:56:49.726353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-15T02:56:50.688098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-15T02:57:20.334 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T02:57:20.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.495+0000 7fe09349b700 1 Processor -- start 2023-11-15T02:57:20.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.495+0000 7fe09349b700 1 -- start start 2023-11-15T02:57:20.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.495+0000 7fe09349b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe08c065850 0x7fe08c0a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:20.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.495+0000 7fe09349b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe08c068170 0x7fe08c0bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:20.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.495+0000 7fe09349b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe08c152150 con 0x7fe08c0a8b80 2023-11-15T02:57:20.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.495+0000 7fe09349b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe08c148f70 con 0x7fe08c068170 2023-11-15T02:57:20.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.495+0000 7fe09349b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe08c1523a0 con 0x7fe08c065850 2023-11-15T02:57:20.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.495+0000 7fe091a38700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe08c068170 0x7fe08c0bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:20.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.495+0000 7fe090a36700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe08c065850 0x7fe08c0a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:20.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.499+0000 7fe090a36700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe08c065850 0x7fe08c0a8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:54932/0 (socket says 172.21.15.121:54932) 2023-11-15T02:57:20.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.499+0000 7fe090a36700 1 -- 172.21.15.121:0/2586280055 learned_addr learned my addr 172.21.15.121:0/2586280055 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:20.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.499+0000 7fe090a36700 1 -- 172.21.15.121:0/2586280055 >> v2:172.21.15.121:3300/0 conn(0x7fe08c068170 msgr2=0x7fe08c0bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:20.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.499+0000 7fe090a36700 1 --2- 172.21.15.121:0/2586280055 >> v2:172.21.15.121:3300/0 conn(0x7fe08c068170 0x7fe08c0bbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:20.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.499+0000 7fe090a36700 1 -- 172.21.15.121:0/2586280055 >> v1:172.21.15.190:6789/0 conn(0x7fe08c0a8b80 legacy=0x7fe08c067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:20.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.499+0000 7fe090a36700 1 -- 172.21.15.121:0/2586280055 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c152680 con 0x7fe08c065850 2023-11-15T02:57:20.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.499+0000 7fe090a36700 1 --2- 172.21.15.121:0/2586280055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08c065850 0x7fe08c0a8f60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fe08c0676c0 tx=0x7fe07c0090f0).ready entity=mon.1 client_cookie=452117c56abbfb73 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:20.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.499+0000 7fe083fff700 1 -- 172.21.15.121:0/2586280055 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00b520 con 0x7fe08c065850 2023-11-15T02:57:20.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.499+0000 7fe083fff700 1 -- 172.21.15.121:0/2586280055 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c00b680 con 0x7fe08c065850 2023-11-15T02:57:20.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.499+0000 7fe083fff700 1 -- 172.21.15.121:0/2586280055 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00ba00 con 0x7fe08c065850 2023-11-15T02:57:20.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.499+0000 7fe09349b700 1 -- 172.21.15.121:0/2586280055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08c065850 msgr2=0x7fe08c0a8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:20.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.499+0000 7fe09349b700 1 --2- 172.21.15.121:0/2586280055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08c065850 0x7fe08c0a8f60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fe08c0676c0 tx=0x7fe07c0090f0).stop 2023-11-15T02:57:20.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.499+0000 7fe09349b700 1 -- 172.21.15.121:0/2586280055 shutdown_connections 2023-11-15T02:57:20.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.499+0000 7fe09349b700 1 --2- 172.21.15.121:0/2586280055 >> v2:172.21.15.121:3300/0 conn(0x7fe08c068170 0x7fe08c0bbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:20.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.499+0000 7fe09349b700 1 --2- 172.21.15.121:0/2586280055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08c065850 0x7fe08c0a8f60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:20.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.499+0000 7fe09349b700 1 -- 172.21.15.121:0/2586280055 >> 172.21.15.121:0/2586280055 conn(0x7fe08c0591c0 msgr2=0x7fe08c0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:20.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.499+0000 7fe09349b700 1 -- 172.21.15.121:0/2586280055 shutdown_connections 2023-11-15T02:57:20.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.499+0000 7fe09349b700 1 -- 172.21.15.121:0/2586280055 wait complete. 2023-11-15T02:57:20.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.503+0000 7fe09349b700 1 Processor -- start 2023-11-15T02:57:20.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.503+0000 7fe09349b700 1 -- start start 2023-11-15T02:57:20.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe09349b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe08c065850 0x7fe08c145890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:20.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe09349b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08c068170 0x7fe08c14b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:20.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe09349b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe08c0a8b80 0x7fe08c147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:20.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe09349b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c14b300 con 0x7fe08c065850 2023-11-15T02:57:20.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe09349b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c14a410 con 0x7fe08c068170 2023-11-15T02:57:20.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe09349b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c14a6f0 con 0x7fe08c0a8b80 2023-11-15T02:57:20.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe091237700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe08c065850 0x7fe08c145890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:20.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe091237700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe08c065850 0x7fe08c145890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39428/0 (socket says 172.21.15.121:39428) 2023-11-15T02:57:20.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe091237700 1 -- 172.21.15.121:0/3755671234 learned_addr learned my addr 172.21.15.121:0/3755671234 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:20.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe091a38700 1 --2- 172.21.15.121:0/3755671234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe08c0a8b80 0x7fe08c147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:20.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe090a36700 1 --2- 172.21.15.121:0/3755671234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08c068170 0x7fe08c14b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:20.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe091237700 1 -- 172.21.15.121:0/3755671234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe08c0a8b80 msgr2=0x7fe08c147500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:20.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe091237700 1 --2- 172.21.15.121:0/3755671234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe08c0a8b80 0x7fe08c147500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:20.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe091237700 1 -- 172.21.15.121:0/3755671234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08c068170 msgr2=0x7fe08c14b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:20.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe091237700 1 --2- 172.21.15.121:0/3755671234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08c068170 0x7fe08c14b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:20.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe091237700 1 -- 172.21.15.121:0/3755671234 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07c009370 con 0x7fe08c065850 2023-11-15T02:57:20.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe091237700 1 --2- 172.21.15.121:0/3755671234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe08c065850 0x7fe08c145890 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fe08800c440 tx=0x7fe088007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:20.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe0817fa700 1 -- 172.21.15.121:0/3755671234 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe088007020 con 0x7fe08c065850 2023-11-15T02:57:20.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe09349b700 1 -- 172.21.15.121:0/3755671234 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08c14a9d0 con 0x7fe08c065850 2023-11-15T02:57:20.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe0817fa700 1 -- 172.21.15.121:0/3755671234 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe088007180 con 0x7fe08c065850 2023-11-15T02:57:20.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe0817fa700 1 -- 172.21.15.121:0/3755671234 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe088007520 con 0x7fe08c065850 2023-11-15T02:57:20.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.507+0000 7fe09349b700 1 -- 172.21.15.121:0/3755671234 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08c14af90 con 0x7fe08c065850 2023-11-15T02:57:20.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.511+0000 7fe09349b700 1 -- 172.21.15.121:0/3755671234 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe08c148ba0 con 0x7fe08c065850 2023-11-15T02:57:20.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.511+0000 7fe0817fa700 1 -- 172.21.15.121:0/3755671234 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe0880068b0 con 0x7fe08c065850 2023-11-15T02:57:20.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.515+0000 7fe0817fa700 1 --2- 172.21.15.121:0/3755671234 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe07005a510 0x7fe07005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:20.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.515+0000 7fe0817fa700 1 -- 172.21.15.121:0/3755671234 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe088073c40 con 0x7fe08c065850 2023-11-15T02:57:20.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.515+0000 7fe090a36700 1 --2- 172.21.15.121:0/3755671234 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe07005a510 0x7fe07005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:20.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.515+0000 7fe090a36700 1 --2- 172.21.15.121:0/3755671234 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe07005a510 0x7fe07005c9d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fe07c009130 tx=0x7fe07c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:20.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.519+0000 7fe0817fa700 1 -- 172.21.15.121:0/3755671234 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe08c148ba0 con 0x7fe08c065850 2023-11-15T02:57:20.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.679+0000 7fe09349b700 1 -- 172.21.15.121:0/3755671234 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe08c148ba0 con 0x7fe08c065850 2023-11-15T02:57:20.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.683+0000 7fe0817fa700 1 -- 172.21.15.121:0/3755671234 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v21) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe08c148ba0 con 0x7fe08c065850 2023-11-15T02:57:20.688 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T02:57:20.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.683+0000 7fe09349b700 1 -- 172.21.15.121:0/3755671234 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe07005a510 msgr2=0x7fe07005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:20.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.683+0000 7fe09349b700 1 --2- 172.21.15.121:0/3755671234 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe07005a510 0x7fe07005c9d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fe07c009130 tx=0x7fe07c017000).stop 2023-11-15T02:57:20.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.683+0000 7fe09349b700 1 -- 172.21.15.121:0/3755671234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe08c065850 msgr2=0x7fe08c145890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:20.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.683+0000 7fe09349b700 1 --2- 172.21.15.121:0/3755671234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe08c065850 0x7fe08c145890 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fe08800c440 tx=0x7fe088007e00).stop 2023-11-15T02:57:20.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.683+0000 7fe09349b700 1 -- 172.21.15.121:0/3755671234 shutdown_connections 2023-11-15T02:57:20.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.683+0000 7fe09349b700 1 --2- 172.21.15.121:0/3755671234 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe07005a510 0x7fe07005c9d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:20.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.683+0000 7fe09349b700 1 --2- 172.21.15.121:0/3755671234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe08c0a8b80 0x7fe08c147500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:20.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.683+0000 7fe09349b700 1 --2- 172.21.15.121:0/3755671234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08c068170 0x7fe08c14b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:20.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.683+0000 7fe09349b700 1 --2- 172.21.15.121:0/3755671234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe08c065850 0x7fe08c145890 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:20.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.683+0000 7fe09349b700 1 -- 172.21.15.121:0/3755671234 >> 172.21.15.121:0/3755671234 conn(0x7fe08c0591c0 msgr2=0x7fe08c065c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:20.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.683+0000 7fe09349b700 1 -- 172.21.15.121:0/3755671234 shutdown_connections 2023-11-15T02:57:20.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.683+0000 7fe09349b700 1 -- 172.21.15.121:0/3755671234 wait complete. 2023-11-15T02:57:20.707 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T02:57:20.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.859+0000 7fe181ff9700 1 Processor -- start 2023-11-15T02:57:20.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.859+0000 7fe181ff9700 1 -- start start 2023-11-15T02:57:20.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.863+0000 7fe181ff9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c0c2f30 0x7fe17c15e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:20.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.863+0000 7fe181ff9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17c0c73a0 con 0x7fe17c0c5540 2023-11-15T02:57:20.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.863+0000 7fe181ff9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17c0be540 con 0x7fe17c0a6b00 2023-11-15T02:57:20.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.863+0000 7fe181ff9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0a6ee0 con 0x7fe17c0c2f30 2023-11-15T02:57:20.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.863+0000 7fe173fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe17c0c5540 0x7fe17c0c5960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:48196/0 (socket says 172.21.15.121:48196) 2023-11-15T02:57:20.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.863+0000 7fe173fff700 1 -- 172.21.15.121:0/635600344 learned_addr learned my addr 172.21.15.121:0/635600344 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:20.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.863+0000 7fe1817f8700 1 --2- 172.21.15.121:0/635600344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c0c2f30 0x7fe17c15e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:20.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.863+0000 7fe1737fe700 1 -- 172.21.15.121:0/635600344 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1330862799 0 0) 0x7fe17c0c73a0 con 0x7fe17c0c5540 2023-11-15T02:57:20.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.863+0000 7fe1737fe700 1 -- 172.21.15.121:0/635600344 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1640033f0 con 0x7fe17c0c5540 2023-11-15T02:57:20.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.863+0000 7fe1817f8700 1 -- 172.21.15.121:0/635600344 >> v1:172.21.15.190:6789/0 conn(0x7fe17c0a6b00 legacy=0x7fe17c068430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:20.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.863+0000 7fe1817f8700 1 -- 172.21.15.121:0/635600344 >> v1:172.21.15.121:6789/0 conn(0x7fe17c0c5540 legacy=0x7fe17c0c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:20.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.863+0000 7fe1817f8700 1 -- 172.21.15.121:0/635600344 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c0be260 con 0x7fe17c0c2f30 2023-11-15T02:57:20.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.863+0000 7fe1817f8700 1 --2- 172.21.15.121:0/635600344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c0c2f30 0x7fe17c15e740 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fe174009790 tx=0x7fe174009060).ready entity=mon.2 client_cookie=4a6caa911eb2d3da server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:20.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.863+0000 7fe1737fe700 1 -- 172.21.15.121:0/635600344 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17400f510 con 0x7fe17c0c2f30 2023-11-15T02:57:20.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.863+0000 7fe1737fe700 1 -- 172.21.15.121:0/635600344 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17400f670 con 0x7fe17c0c2f30 2023-11-15T02:57:20.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.863+0000 7fe1737fe700 1 -- 172.21.15.121:0/635600344 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17400f9f0 con 0x7fe17c0c2f30 2023-11-15T02:57:20.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.867+0000 7fe181ff9700 1 -- 172.21.15.121:0/635600344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c0c2f30 msgr2=0x7fe17c15e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:20.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.867+0000 7fe181ff9700 1 --2- 172.21.15.121:0/635600344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c0c2f30 0x7fe17c15e740 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fe174009790 tx=0x7fe174009060).stop 2023-11-15T02:57:20.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.867+0000 7fe181ff9700 1 -- 172.21.15.121:0/635600344 shutdown_connections 2023-11-15T02:57:20.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.867+0000 7fe181ff9700 1 --2- 172.21.15.121:0/635600344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c0c2f30 0x7fe17c15e740 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:20.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.867+0000 7fe181ff9700 1 -- 172.21.15.121:0/635600344 >> 172.21.15.121:0/635600344 conn(0x7fe17c090dd0 msgr2=0x7fe17c069350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:20.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.867+0000 7fe181ff9700 1 -- 172.21.15.121:0/635600344 shutdown_connections 2023-11-15T02:57:20.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.867+0000 7fe181ff9700 1 -- 172.21.15.121:0/635600344 wait complete. 2023-11-15T02:57:20.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe181ff9700 1 Processor -- start 2023-11-15T02:57:20.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe181ff9700 1 -- start start 2023-11-15T02:57:20.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe181ff9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17c0a6b00 0x7fe17c0c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:20.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe181ff9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe17c0c2f30 0x7fe17c164e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:20.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe181ff9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c0c5540 0x7fe17c0ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:20.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe181ff9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0c0b00 con 0x7fe17c0c2f30 2023-11-15T02:57:20.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe181ff9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c165370 con 0x7fe17c0a6b00 2023-11-15T02:57:20.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe181ff9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0c01f0 con 0x7fe17c0c5540 2023-11-15T02:57:20.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe173fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe17c0c2f30 0x7fe17c164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:20.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe173fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe17c0c2f30 0x7fe17c164e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39436/0 (socket says 172.21.15.121:39436) 2023-11-15T02:57:20.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe173fff700 1 -- 172.21.15.121:0/2441515916 learned_addr learned my addr 172.21.15.121:0/2441515916 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:20.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe180ff7700 1 --2- 172.21.15.121:0/2441515916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17c0a6b00 0x7fe17c0c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:20.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe1817f8700 1 --2- 172.21.15.121:0/2441515916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c0c5540 0x7fe17c0ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:20.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe173fff700 1 -- 172.21.15.121:0/2441515916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c0c5540 msgr2=0x7fe17c0ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:20.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe173fff700 1 --2- 172.21.15.121:0/2441515916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c0c5540 0x7fe17c0ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:20.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe173fff700 1 -- 172.21.15.121:0/2441515916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17c0a6b00 msgr2=0x7fe17c0c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:20.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe173fff700 1 --2- 172.21.15.121:0/2441515916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17c0a6b00 0x7fe17c0c0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:20.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe173fff700 1 -- 172.21.15.121:0/2441515916 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1740092a0 con 0x7fe17c0c2f30 2023-11-15T02:57:20.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe173fff700 1 --2- 172.21.15.121:0/2441515916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe17c0c2f30 0x7fe17c164e80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fe16c00c420 tx=0x7fe16c007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:20.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe170ff9700 1 -- 172.21.15.121:0/2441515916 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c0070d0 con 0x7fe17c0c2f30 2023-11-15T02:57:20.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe170ff9700 1 -- 172.21.15.121:0/2441515916 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16c007230 con 0x7fe17c0c2f30 2023-11-15T02:57:20.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe181ff9700 1 -- 172.21.15.121:0/2441515916 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17c0ad510 con 0x7fe17c0c2f30 2023-11-15T02:57:20.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.871+0000 7fe170ff9700 1 -- 172.21.15.121:0/2441515916 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c0075b0 con 0x7fe17c0c2f30 2023-11-15T02:57:20.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.875+0000 7fe181ff9700 1 -- 172.21.15.121:0/2441515916 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17c0c0790 con 0x7fe17c0c2f30 2023-11-15T02:57:20.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.875+0000 7fe181ff9700 1 -- 172.21.15.121:0/2441515916 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe17c0c0cb0 con 0x7fe17c0c2f30 2023-11-15T02:57:20.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.875+0000 7fe170ff9700 1 -- 172.21.15.121:0/2441515916 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe16c0059b0 con 0x7fe17c0c2f30 2023-11-15T02:57:20.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.879+0000 7fe170ff9700 1 --2- 172.21.15.121:0/2441515916 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe16805a510 0x7fe16805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:20.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.879+0000 7fe170ff9700 1 -- 172.21.15.121:0/2441515916 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe16c073d80 con 0x7fe17c0c2f30 2023-11-15T02:57:20.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.879+0000 7fe180ff7700 1 --2- 172.21.15.121:0/2441515916 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe16805a510 0x7fe16805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:20.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.879+0000 7fe180ff7700 1 --2- 172.21.15.121:0/2441515916 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe16805a510 0x7fe16805c9d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fe178004290 tx=0x7fe178001560).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:20.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:20.883+0000 7fe170ff9700 1 -- 172.21.15.121:0/2441515916 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe16c047560 con 0x7fe17c0c2f30 2023-11-15T02:57:21.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.047+0000 7fe181ff9700 1 -- 172.21.15.121:0/2441515916 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe17c06bcf0 con 0x7fe17c0c2f30 2023-11-15T02:57:21.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.047+0000 7fe170ff9700 1 -- 172.21.15.121:0/2441515916 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe16c04ab80 con 0x7fe17c0c2f30 2023-11-15T02:57:21.052 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T02:57:21.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.047+0000 7fe181ff9700 1 -- 172.21.15.121:0/2441515916 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe16805a510 msgr2=0x7fe16805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:21.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.047+0000 7fe181ff9700 1 --2- 172.21.15.121:0/2441515916 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe16805a510 0x7fe16805c9d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fe178004290 tx=0x7fe178001560).stop 2023-11-15T02:57:21.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.047+0000 7fe181ff9700 1 -- 172.21.15.121:0/2441515916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe17c0c2f30 msgr2=0x7fe17c164e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:21.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.047+0000 7fe181ff9700 1 --2- 172.21.15.121:0/2441515916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe17c0c2f30 0x7fe17c164e80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fe16c00c420 tx=0x7fe16c007c80).stop 2023-11-15T02:57:21.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.047+0000 7fe181ff9700 1 -- 172.21.15.121:0/2441515916 shutdown_connections 2023-11-15T02:57:21.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.047+0000 7fe181ff9700 1 --2- 172.21.15.121:0/2441515916 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe16805a510 0x7fe16805c9d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.047+0000 7fe181ff9700 1 --2- 172.21.15.121:0/2441515916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c0c5540 0x7fe17c0ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.047+0000 7fe181ff9700 1 --2- 172.21.15.121:0/2441515916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe17c0c2f30 0x7fe17c164e80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.047+0000 7fe181ff9700 1 --2- 172.21.15.121:0/2441515916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17c0a6b00 0x7fe17c0c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.047+0000 7fe181ff9700 1 -- 172.21.15.121:0/2441515916 >> 172.21.15.121:0/2441515916 conn(0x7fe17c090dd0 msgr2=0x7fe17c0a5700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:21.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.051+0000 7fe181ff9700 1 -- 172.21.15.121:0/2441515916 shutdown_connections 2023-11-15T02:57:21.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.051+0000 7fe181ff9700 1 -- 172.21.15.121:0/2441515916 wait complete. 2023-11-15T02:57:21.073 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-15T02:57:21.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.227+0000 7f475585b700 1 Processor -- start 2023-11-15T02:57:21.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.227+0000 7f475585b700 1 -- start start 2023-11-15T02:57:21.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.231+0000 7f475585b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f47500682a0 0x7f47500c10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:21.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.231+0000 7f475585b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47500c15a0 con 0x7f4750067ec0 2023-11-15T02:57:21.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.231+0000 7f475585b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47500c0c20 con 0x7f4750153700 2023-11-15T02:57:21.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.231+0000 7f475585b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47500686e0 con 0x7f47500682a0 2023-11-15T02:57:21.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.231+0000 7f474effd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4750067ec0 0x7f4750062c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:48202/0 (socket says 172.21.15.121:48202) 2023-11-15T02:57:21.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.231+0000 7f474effd700 1 -- 172.21.15.121:0/615098168 learned_addr learned my addr 172.21.15.121:0/615098168 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:21.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.231+0000 7f474e7fc700 1 --2- 172.21.15.121:0/615098168 >> v2:172.21.15.190:3300/0 conn(0x7f47500682a0 0x7f47500c10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:21.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.231+0000 7f474dffb700 1 -- 172.21.15.121:0/615098168 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 618677941 0 0) 0x7f4740001110 con 0x7f4750067ec0 2023-11-15T02:57:21.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.231+0000 7f474dffb700 1 -- 172.21.15.121:0/615098168 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47380033f0 con 0x7f4750067ec0 2023-11-15T02:57:21.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.231+0000 7f474dffb700 1 -- 172.21.15.121:0/615098168 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761187202 0 0) 0x7f47500c0c20 con 0x7f4750153700 2023-11-15T02:57:21.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.231+0000 7f474dffb700 1 -- 172.21.15.121:0/615098168 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4740001110 con 0x7f4750153700 2023-11-15T02:57:21.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.231+0000 7f474e7fc700 1 -- 172.21.15.121:0/615098168 >> v1:172.21.15.190:6789/0 conn(0x7f4750153700 legacy=0x7f4750153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:21.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.231+0000 7f474dffb700 1 -- 172.21.15.121:0/615098168 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 567250178 0 0) 0x7f47380033f0 con 0x7f4750067ec0 2023-11-15T02:57:21.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.231+0000 7f474e7fc700 1 -- 172.21.15.121:0/615098168 >> v1:172.21.15.121:6789/0 conn(0x7f4750067ec0 legacy=0x7f4750062c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:21.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.231+0000 7f474e7fc700 1 -- 172.21.15.121:0/615098168 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4750158730 con 0x7f47500682a0 2023-11-15T02:57:21.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.231+0000 7f474e7fc700 1 --2- 172.21.15.121:0/615098168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47500682a0 0x7f47500c10b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f4744009760 tx=0x7f4744009060).ready entity=mon.1 client_cookie=937a91c9da0ae536 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:21.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.235+0000 7f474dffb700 1 -- 172.21.15.121:0/615098168 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4740001600 con 0x7f47500682a0 2023-11-15T02:57:21.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.235+0000 7f474dffb700 1 -- 172.21.15.121:0/615098168 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4744011400 con 0x7f47500682a0 2023-11-15T02:57:21.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.235+0000 7f474dffb700 1 -- 172.21.15.121:0/615098168 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744011780 con 0x7f47500682a0 2023-11-15T02:57:21.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.235+0000 7f475585b700 1 -- 172.21.15.121:0/615098168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47500682a0 msgr2=0x7f47500c10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:21.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.235+0000 7f475585b700 1 --2- 172.21.15.121:0/615098168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47500682a0 0x7f47500c10b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f4744009760 tx=0x7f4744009060).stop 2023-11-15T02:57:21.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.235+0000 7f475585b700 1 -- 172.21.15.121:0/615098168 shutdown_connections 2023-11-15T02:57:21.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.235+0000 7f475585b700 1 --2- 172.21.15.121:0/615098168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47500682a0 0x7f47500c10b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.235+0000 7f475585b700 1 -- 172.21.15.121:0/615098168 >> 172.21.15.121:0/615098168 conn(0x7f47500cad40 msgr2=0x7f47500cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:21.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.235+0000 7f475585b700 1 -- 172.21.15.121:0/615098168 shutdown_connections 2023-11-15T02:57:21.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.235+0000 7f475585b700 1 -- 172.21.15.121:0/615098168 wait complete. 2023-11-15T02:57:21.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.239+0000 7f475585b700 1 Processor -- start 2023-11-15T02:57:21.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.239+0000 7f475585b700 1 -- start start 2023-11-15T02:57:21.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.239+0000 7f475585b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4750067ec0 0x7f47500af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:21.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.239+0000 7f475585b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47500682a0 0x7f47501542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:21.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.239+0000 7f475585b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4750153700 0x7f47500ad210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:21.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.239+0000 7f475585b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47501547c0 con 0x7f47500682a0 2023-11-15T02:57:21.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.239+0000 7f475585b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47501643f0 con 0x7f4750153700 2023-11-15T02:57:21.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.239+0000 7f475585b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4750154a00 con 0x7f4750067ec0 2023-11-15T02:57:21.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.239+0000 7f474e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47500682a0 0x7f47501542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:21.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.239+0000 7f474e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47500682a0 0x7f47501542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39438/0 (socket says 172.21.15.121:39438) 2023-11-15T02:57:21.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.239+0000 7f474e7fc700 1 -- 172.21.15.121:0/3988223857 learned_addr learned my addr 172.21.15.121:0/3988223857 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:21.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.239+0000 7f474effd700 1 --2- 172.21.15.121:0/3988223857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4750067ec0 0x7f47500af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:21.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.239+0000 7f474f7fe700 1 --2- 172.21.15.121:0/3988223857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4750153700 0x7f47500ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:21.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.239+0000 7f474e7fc700 1 -- 172.21.15.121:0/3988223857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4750067ec0 msgr2=0x7f47500af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:21.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.239+0000 7f474e7fc700 1 --2- 172.21.15.121:0/3988223857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4750067ec0 0x7f47500af630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.239+0000 7f474e7fc700 1 -- 172.21.15.121:0/3988223857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4750153700 msgr2=0x7f47500ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:21.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.239+0000 7f474e7fc700 1 --2- 172.21.15.121:0/3988223857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4750153700 0x7f47500ad210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.239+0000 7f474e7fc700 1 -- 172.21.15.121:0/3988223857 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47440092a0 con 0x7f47500682a0 2023-11-15T02:57:21.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.239+0000 7f474e7fc700 1 --2- 172.21.15.121:0/3988223857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47500682a0 0x7f47501542d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f47500caab0 tx=0x7f4744016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:21.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.243+0000 7f4736ffd700 1 -- 172.21.15.121:0/3988223857 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744010740 con 0x7f47500682a0 2023-11-15T02:57:21.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.243+0000 7f4736ffd700 1 -- 172.21.15.121:0/3988223857 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47440108a0 con 0x7f47500682a0 2023-11-15T02:57:21.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.243+0000 7f4736ffd700 1 -- 172.21.15.121:0/3988223857 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744010c20 con 0x7f47500682a0 2023-11-15T02:57:21.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.243+0000 7f475585b700 1 -- 172.21.15.121:0/3988223857 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4750154ce0 con 0x7f47500682a0 2023-11-15T02:57:21.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.243+0000 7f475585b700 1 -- 172.21.15.121:0/3988223857 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47501552a0 con 0x7f47500682a0 2023-11-15T02:57:21.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.243+0000 7f475585b700 1 -- 172.21.15.121:0/3988223857 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4750047060 con 0x7f47500682a0 2023-11-15T02:57:21.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.243+0000 7f4736ffd700 1 -- 172.21.15.121:0/3988223857 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f47440263b0 con 0x7f47500682a0 2023-11-15T02:57:21.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.247+0000 7f4736ffd700 1 --2- 172.21.15.121:0/3988223857 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f473c05a540 0x7f473c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:21.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.247+0000 7f4736ffd700 1 -- 172.21.15.121:0/3988223857 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4744075220 con 0x7f47500682a0 2023-11-15T02:57:21.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.247+0000 7f474effd700 1 --2- 172.21.15.121:0/3988223857 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f473c05a540 0x7f473c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:21.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.247+0000 7f474effd700 1 --2- 172.21.15.121:0/3988223857 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f473c05a540 0x7f473c05ca00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f4740000c00 tx=0x7f4740001c60).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:21.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.251+0000 7f4736ffd700 1 -- 172.21.15.121:0/3988223857 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4744048a50 con 0x7f47500682a0 2023-11-15T02:57:21.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.411+0000 7f475585b700 1 -- 172.21.15.121:0/3988223857 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f47500c02f0 con 0x7f47500682a0 2023-11-15T02:57:21.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.411+0000 7f4736ffd700 1 -- 172.21.15.121:0/3988223857 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f474404c070 con 0x7f47500682a0 2023-11-15T02:57:21.418 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T02:57:21.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.415+0000 7f475585b700 1 -- 172.21.15.121:0/3988223857 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f473c05a540 msgr2=0x7f473c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:21.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.415+0000 7f475585b700 1 --2- 172.21.15.121:0/3988223857 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f473c05a540 0x7f473c05ca00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f4740000c00 tx=0x7f4740001c60).stop 2023-11-15T02:57:21.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.415+0000 7f475585b700 1 -- 172.21.15.121:0/3988223857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47500682a0 msgr2=0x7f47501542d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:21.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.415+0000 7f475585b700 1 --2- 172.21.15.121:0/3988223857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47500682a0 0x7f47501542d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f47500caab0 tx=0x7f4744016000).stop 2023-11-15T02:57:21.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.415+0000 7f475585b700 1 -- 172.21.15.121:0/3988223857 shutdown_connections 2023-11-15T02:57:21.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.415+0000 7f475585b700 1 --2- 172.21.15.121:0/3988223857 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f473c05a540 0x7f473c05ca00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.415+0000 7f475585b700 1 --2- 172.21.15.121:0/3988223857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4750153700 0x7f47500ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.415+0000 7f475585b700 1 --2- 172.21.15.121:0/3988223857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47500682a0 0x7f47501542d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.415+0000 7f475585b700 1 --2- 172.21.15.121:0/3988223857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4750067ec0 0x7f47500af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.415+0000 7f475585b700 1 -- 172.21.15.121:0/3988223857 >> 172.21.15.121:0/3988223857 conn(0x7f47500cad40 msgr2=0x7f47500c89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:21.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.415+0000 7f475585b700 1 -- 172.21.15.121:0/3988223857 shutdown_connections 2023-11-15T02:57:21.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.415+0000 7f475585b700 1 -- 172.21.15.121:0/3988223857 wait complete. 2023-11-15T02:57:21.439 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-15T02:57:21.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.603+0000 7fc580979700 1 Processor -- start 2023-11-15T02:57:21.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.603+0000 7fc580979700 1 -- start start 2023-11-15T02:57:21.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.607+0000 7fc580979700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc57c0acd00 0x7fc57c0ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:21.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.607+0000 7fc580979700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc57c066330 0x7fc57c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:21.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.607+0000 7fc580979700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc57c065ea0 con 0x7fc57c0c5c70 2023-11-15T02:57:21.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.607+0000 7fc580979700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc57c066bb0 con 0x7fc57c0acd00 2023-11-15T02:57:21.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.607+0000 7fc580979700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc57c065890 con 0x7fc57c066330 2023-11-15T02:57:21.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.607+0000 7fc57b7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc57c0acd00 0x7fc57c0ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:21.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.607+0000 7fc57b7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc57c0acd00 0x7fc57c0ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39442/0 (socket says 172.21.15.121:39442) 2023-11-15T02:57:21.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.607+0000 7fc57b7fe700 1 -- 172.21.15.121:0/4061133014 learned_addr learned my addr 172.21.15.121:0/4061133014 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:21.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.607+0000 7fc57bfff700 1 --2- 172.21.15.121:0/4061133014 >> v2:172.21.15.190:3300/0 conn(0x7fc57c066330 0x7fc57c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:21.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.607+0000 7fc57b7fe700 1 -- 172.21.15.121:0/4061133014 >> v2:172.21.15.190:3300/0 conn(0x7fc57c066330 msgr2=0x7fc57c153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:21.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.607+0000 7fc57b7fe700 1 --2- 172.21.15.121:0/4061133014 >> v2:172.21.15.190:3300/0 conn(0x7fc57c066330 0x7fc57c153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.607+0000 7fc57b7fe700 1 -- 172.21.15.121:0/4061133014 >> v1:172.21.15.190:6789/0 conn(0x7fc57c0c5c70 legacy=0x7fc57c1536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:21.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.607+0000 7fc57b7fe700 1 -- 172.21.15.121:0/4061133014 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57c0c64a0 con 0x7fc57c0acd00 2023-11-15T02:57:21.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.607+0000 7fc57b7fe700 1 --2- 172.21.15.121:0/4061133014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc57c0acd00 0x7fc57c0ad0e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fc57c0c60d0 tx=0x7fc56c009060).ready entity=mon.0 client_cookie=f8cc74cc8ae239f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:21.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.607+0000 7fc57a7fc700 1 -- 172.21.15.121:0/4061133014 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56c005880 con 0x7fc57c0acd00 2023-11-15T02:57:21.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.607+0000 7fc57a7fc700 1 -- 172.21.15.121:0/4061133014 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56c0059e0 con 0x7fc57c0acd00 2023-11-15T02:57:21.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.607+0000 7fc57a7fc700 1 -- 172.21.15.121:0/4061133014 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56c005d60 con 0x7fc57c0acd00 2023-11-15T02:57:21.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.607+0000 7fc580979700 1 -- 172.21.15.121:0/4061133014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc57c0acd00 msgr2=0x7fc57c0ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:21.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.607+0000 7fc580979700 1 --2- 172.21.15.121:0/4061133014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc57c0acd00 0x7fc57c0ad0e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fc57c0c60d0 tx=0x7fc56c009060).stop 2023-11-15T02:57:21.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.611+0000 7fc580979700 1 -- 172.21.15.121:0/4061133014 shutdown_connections 2023-11-15T02:57:21.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.611+0000 7fc580979700 1 --2- 172.21.15.121:0/4061133014 >> v2:172.21.15.190:3300/0 conn(0x7fc57c066330 0x7fc57c153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.611+0000 7fc580979700 1 --2- 172.21.15.121:0/4061133014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc57c0acd00 0x7fc57c0ad0e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.611+0000 7fc580979700 1 -- 172.21.15.121:0/4061133014 >> 172.21.15.121:0/4061133014 conn(0x7fc57c063fc0 msgr2=0x7fc57c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:21.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.611+0000 7fc580979700 1 -- 172.21.15.121:0/4061133014 shutdown_connections 2023-11-15T02:57:21.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.611+0000 7fc580979700 1 -- 172.21.15.121:0/4061133014 wait complete. 2023-11-15T02:57:21.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.611+0000 7fc580979700 1 Processor -- start 2023-11-15T02:57:21.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.611+0000 7fc580979700 1 -- start start 2023-11-15T02:57:21.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc580979700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc57c066330 0x7fc57c0c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:21.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc580979700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc57c0acd00 0x7fc57c0c36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:21.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc580979700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc57c0c5c70 0x7fc57c164e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:21.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc580979700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c0c48b0 con 0x7fc57c066330 2023-11-15T02:57:21.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc580979700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c0c9120 con 0x7fc57c0acd00 2023-11-15T02:57:21.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc580979700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c0c9400 con 0x7fc57c0c5c70 2023-11-15T02:57:21.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc57b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc57c066330 0x7fc57c0c43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:21.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc57b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc57c066330 0x7fc57c0c43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39452/0 (socket says 172.21.15.121:39452) 2023-11-15T02:57:21.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc57b7fe700 1 -- 172.21.15.121:0/1021158141 learned_addr learned my addr 172.21.15.121:0/1021158141 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:21.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc57affd700 1 --2- 172.21.15.121:0/1021158141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc57c0acd00 0x7fc57c0c36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:21.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc57bfff700 1 --2- 172.21.15.121:0/1021158141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc57c0c5c70 0x7fc57c164e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:21.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc57b7fe700 1 -- 172.21.15.121:0/1021158141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc57c0c5c70 msgr2=0x7fc57c164e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:21.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc57b7fe700 1 --2- 172.21.15.121:0/1021158141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc57c0c5c70 0x7fc57c164e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc57b7fe700 1 -- 172.21.15.121:0/1021158141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc57c0acd00 msgr2=0x7fc57c0c36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:21.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc57b7fe700 1 --2- 172.21.15.121:0/1021158141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc57c0acd00 0x7fc57c0c36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc57b7fe700 1 -- 172.21.15.121:0/1021158141 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c009300 con 0x7fc57c066330 2023-11-15T02:57:21.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc57b7fe700 1 --2- 172.21.15.121:0/1021158141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc57c066330 0x7fc57c0c43c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fc57c064f20 tx=0x7fc56c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:21.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc5637fe700 1 -- 172.21.15.121:0/1021158141 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56c00b580 con 0x7fc57c066330 2023-11-15T02:57:21.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc5637fe700 1 -- 172.21.15.121:0/1021158141 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56c00b6e0 con 0x7fc57c066330 2023-11-15T02:57:21.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc5637fe700 1 -- 172.21.15.121:0/1021158141 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56c00ba80 con 0x7fc57c066330 2023-11-15T02:57:21.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc580979700 1 -- 172.21.15.121:0/1021158141 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57c0c96e0 con 0x7fc57c066330 2023-11-15T02:57:21.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.615+0000 7fc580979700 1 -- 172.21.15.121:0/1021158141 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc57c165ba0 con 0x7fc57c066330 2023-11-15T02:57:21.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.619+0000 7fc580979700 1 -- 172.21.15.121:0/1021158141 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc57c06bcf0 con 0x7fc57c066330 2023-11-15T02:57:21.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.619+0000 7fc5637fe700 1 -- 172.21.15.121:0/1021158141 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc56c018070 con 0x7fc57c066330 2023-11-15T02:57:21.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.619+0000 7fc5637fe700 1 --2- 172.21.15.121:0/1021158141 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc56805a510 0x7fc56805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:21.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.619+0000 7fc5637fe700 1 -- 172.21.15.121:0/1021158141 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc56c0756a0 con 0x7fc57c066330 2023-11-15T02:57:21.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.619+0000 7fc57affd700 1 --2- 172.21.15.121:0/1021158141 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc56805a510 0x7fc56805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:21.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.619+0000 7fc57affd700 1 --2- 172.21.15.121:0/1021158141 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc56805a510 0x7fc56805c9d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fc57c0acab0 tx=0x7fc570007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:21.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.627+0000 7fc5637fe700 1 -- 172.21.15.121:0/1021158141 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc56c048da0 con 0x7fc57c066330 2023-11-15T02:57:21.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.791+0000 7fc580979700 1 -- 172.21.15.121:0/1021158141 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc57c0c81b0 con 0x7fc57c066330 2023-11-15T02:57:21.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.791+0000 7fc5637fe700 1 -- 172.21.15.121:0/1021158141 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc56c04c3c0 con 0x7fc57c066330 2023-11-15T02:57:21.796 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T02:57:21.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.791+0000 7fc580979700 1 -- 172.21.15.121:0/1021158141 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc56805a510 msgr2=0x7fc56805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:21.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.791+0000 7fc580979700 1 --2- 172.21.15.121:0/1021158141 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc56805a510 0x7fc56805c9d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fc57c0acab0 tx=0x7fc570007d40).stop 2023-11-15T02:57:21.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.791+0000 7fc580979700 1 -- 172.21.15.121:0/1021158141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc57c066330 msgr2=0x7fc57c0c43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:21.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.791+0000 7fc580979700 1 --2- 172.21.15.121:0/1021158141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc57c066330 0x7fc57c0c43c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fc57c064f20 tx=0x7fc56c016000).stop 2023-11-15T02:57:21.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.791+0000 7fc580979700 1 -- 172.21.15.121:0/1021158141 shutdown_connections 2023-11-15T02:57:21.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.791+0000 7fc580979700 1 --2- 172.21.15.121:0/1021158141 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc56805a510 0x7fc56805c9d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.791+0000 7fc580979700 1 --2- 172.21.15.121:0/1021158141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc57c0c5c70 0x7fc57c164e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.791+0000 7fc580979700 1 --2- 172.21.15.121:0/1021158141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc57c0acd00 0x7fc57c0c36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.791+0000 7fc580979700 1 --2- 172.21.15.121:0/1021158141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc57c066330 0x7fc57c0c43c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.791+0000 7fc580979700 1 -- 172.21.15.121:0/1021158141 >> 172.21.15.121:0/1021158141 conn(0x7fc57c063fc0 msgr2=0x7fc57c0c0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:21.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.791+0000 7fc580979700 1 -- 172.21.15.121:0/1021158141 shutdown_connections 2023-11-15T02:57:21.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.791+0000 7fc580979700 1 -- 172.21.15.121:0/1021158141 wait complete. 2023-11-15T02:57:21.819 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:57:21.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.971+0000 7f583de8f700 1 Processor -- start 2023-11-15T02:57:21.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.971+0000 7f583de8f700 1 -- start start 2023-11-15T02:57:21.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.975+0000 7f583de8f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58380c3280 0x7f58380c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:21.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.975+0000 7f583de8f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f583815be30 0x7f583815c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:21.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.975+0000 7f583de8f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58380c1d50 con 0x7f5838152140 2023-11-15T02:57:21.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.975+0000 7f583de8f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58380c1860 con 0x7f583815be30 2023-11-15T02:57:21.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.975+0000 7f583de8f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583815cb80 con 0x7f58380c3280 2023-11-15T02:57:21.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.975+0000 7f583d68e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f583815be30 0x7f583815c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:21.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.975+0000 7f583ce8d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58380c3280 0x7f58380c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:21.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.975+0000 7f583d68e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f583815be30 0x7f583815c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39468/0 (socket says 172.21.15.121:39468) 2023-11-15T02:57:21.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.975+0000 7f583d68e700 1 -- 172.21.15.121:0/2819691707 learned_addr learned my addr 172.21.15.121:0/2819691707 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:21.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.975+0000 7f583d68e700 1 -- 172.21.15.121:0/2819691707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58380c3280 msgr2=0x7f58380c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:21.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.975+0000 7f583d68e700 1 --2- 172.21.15.121:0/2819691707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58380c3280 0x7f58380c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.975+0000 7f583d68e700 1 -- 172.21.15.121:0/2819691707 >> v1:172.21.15.121:6789/0 conn(0x7f5838152140 legacy=0x7f58381525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:21.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.975+0000 7f583d68e700 1 -- 172.21.15.121:0/2819691707 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58380a9240 con 0x7f583815be30 2023-11-15T02:57:21.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.979+0000 7f583d68e700 1 --2- 172.21.15.121:0/2819691707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f583815be30 0x7f583815c2a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f5830009f20 tx=0x7f5830009840).ready entity=mon.0 client_cookie=5aedf01e0aa8268a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:21.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.979+0000 7f582f7fe700 1 -- 172.21.15.121:0/2819691707 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583000e070 con 0x7f583815be30 2023-11-15T02:57:21.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.979+0000 7f582f7fe700 1 -- 172.21.15.121:0/2819691707 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583000b9d0 con 0x7f583815be30 2023-11-15T02:57:21.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.979+0000 7f582f7fe700 1 -- 172.21.15.121:0/2819691707 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583000bd50 con 0x7f583815be30 2023-11-15T02:57:21.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.979+0000 7f583de8f700 1 -- 172.21.15.121:0/2819691707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f583815be30 msgr2=0x7f583815c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:21.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.979+0000 7f583de8f700 1 --2- 172.21.15.121:0/2819691707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f583815be30 0x7f583815c2a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f5830009f20 tx=0x7f5830009840).stop 2023-11-15T02:57:21.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.979+0000 7f583de8f700 1 -- 172.21.15.121:0/2819691707 shutdown_connections 2023-11-15T02:57:21.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.979+0000 7f583de8f700 1 --2- 172.21.15.121:0/2819691707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f583815be30 0x7f583815c2a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.979+0000 7f583de8f700 1 --2- 172.21.15.121:0/2819691707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58380c3280 0x7f58380c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.979+0000 7f583de8f700 1 -- 172.21.15.121:0/2819691707 >> 172.21.15.121:0/2819691707 conn(0x7f5838062190 msgr2=0x7f5838062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:21.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.979+0000 7f583de8f700 1 -- 172.21.15.121:0/2819691707 shutdown_connections 2023-11-15T02:57:21.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.979+0000 7f583de8f700 1 -- 172.21.15.121:0/2819691707 wait complete. 2023-11-15T02:57:21.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.979+0000 7f583de8f700 1 Processor -- start 2023-11-15T02:57:21.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f583de8f700 1 -- start start 2023-11-15T02:57:21.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f583de8f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58380c3280 0x7f583814c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:21.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f583de8f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5838152140 0x7f583814c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:21.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f583de8f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583815be30 0x7f58380c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:21.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f583de8f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583814d460 con 0x7f5838152140 2023-11-15T02:57:21.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f583de8f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583816a2a0 con 0x7f58380c3280 2023-11-15T02:57:21.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f583de8f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583814d180 con 0x7f583815be30 2023-11-15T02:57:21.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f582ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5838152140 0x7f583814c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:21.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f582ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5838152140 0x7f583814c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39478/0 (socket says 172.21.15.121:39478) 2023-11-15T02:57:21.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f582ffff700 1 -- 172.21.15.121:0/1055664872 learned_addr learned my addr 172.21.15.121:0/1055664872 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:21.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f583ce8d700 1 --2- 172.21.15.121:0/1055664872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58380c3280 0x7f583814c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:21.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f583d68e700 1 --2- 172.21.15.121:0/1055664872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583815be30 0x7f58380c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:21.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f582ffff700 1 -- 172.21.15.121:0/1055664872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583815be30 msgr2=0x7f58380c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:21.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f582ffff700 1 --2- 172.21.15.121:0/1055664872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583815be30 0x7f58380c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f582ffff700 1 -- 172.21.15.121:0/1055664872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58380c3280 msgr2=0x7f583814c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:21.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f582ffff700 1 --2- 172.21.15.121:0/1055664872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58380c3280 0x7f583814c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:21.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f582ffff700 1 -- 172.21.15.121:0/1055664872 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5830009ae0 con 0x7f5838152140 2023-11-15T02:57:21.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f583d68e700 1 --2- 172.21.15.121:0/1055664872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583815be30 0x7f58380c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:21.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f582ffff700 1 --2- 172.21.15.121:0/1055664872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5838152140 0x7f583814c940 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f582800c4c0 tx=0x7f5828007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:21.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f582cff9700 1 -- 172.21.15.121:0/1055664872 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828007020 con 0x7f5838152140 2023-11-15T02:57:21.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f582cff9700 1 -- 172.21.15.121:0/1055664872 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5828007180 con 0x7f5838152140 2023-11-15T02:57:21.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f582cff9700 1 -- 172.21.15.121:0/1055664872 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828007520 con 0x7f5838152140 2023-11-15T02:57:21.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f583de8f700 1 -- 172.21.15.121:0/1055664872 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5838169c50 con 0x7f5838152140 2023-11-15T02:57:21.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.983+0000 7f583de8f700 1 -- 172.21.15.121:0/1055664872 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583814ce30 con 0x7f5838152140 2023-11-15T02:57:21.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.987+0000 7f583de8f700 1 -- 172.21.15.121:0/1055664872 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5838169ab0 con 0x7f5838152140 2023-11-15T02:57:21.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.987+0000 7f582cff9700 1 -- 172.21.15.121:0/1055664872 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f58280068b0 con 0x7f5838152140 2023-11-15T02:57:21.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.987+0000 7f582cff9700 1 --2- 172.21.15.121:0/1055664872 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f582405a510 0x7f582405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:21.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.991+0000 7f582cff9700 1 -- 172.21.15.121:0/1055664872 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5828073b90 con 0x7f5838152140 2023-11-15T02:57:21.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.991+0000 7f583ce8d700 1 --2- 172.21.15.121:0/1055664872 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f582405a510 0x7f582405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:21.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.991+0000 7f583ce8d700 1 --2- 172.21.15.121:0/1055664872 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f582405a510 0x7f582405c9d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f5834009b90 tx=0x7f5834009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:22.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:21.995+0000 7f582cff9700 1 -- 172.21.15.121:0/1055664872 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5838169ab0 con 0x7f5838152140 2023-11-15T02:57:22.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.159+0000 7f583de8f700 1 -- 172.21.15.121:0/1055664872 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5838169ab0 con 0x7f5838152140 2023-11-15T02:57:22.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.159+0000 7f582cff9700 1 -- 172.21.15.121:0/1055664872 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f5838169ab0 con 0x7f5838152140 2023-11-15T02:57:22.165 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:22.165 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":21,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:52.702668+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:56:49.726353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-15T02:56:50.688098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:57:22.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.163+0000 7f583de8f700 1 -- 172.21.15.121:0/1055664872 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f582405a510 msgr2=0x7f582405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:22.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.163+0000 7f583de8f700 1 --2- 172.21.15.121:0/1055664872 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f582405a510 0x7f582405c9d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f5834009b90 tx=0x7f5834009060).stop 2023-11-15T02:57:22.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.163+0000 7f583de8f700 1 -- 172.21.15.121:0/1055664872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5838152140 msgr2=0x7f583814c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:22.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.163+0000 7f583de8f700 1 --2- 172.21.15.121:0/1055664872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5838152140 0x7f583814c940 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f582800c4c0 tx=0x7f5828007dd0).stop 2023-11-15T02:57:22.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.163+0000 7f583de8f700 1 -- 172.21.15.121:0/1055664872 shutdown_connections 2023-11-15T02:57:22.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.163+0000 7f583de8f700 1 --2- 172.21.15.121:0/1055664872 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f582405a510 0x7f582405c9d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.163+0000 7f583de8f700 1 --2- 172.21.15.121:0/1055664872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583815be30 0x7f58380c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.163+0000 7f583de8f700 1 --2- 172.21.15.121:0/1055664872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5838152140 0x7f583814c940 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.163+0000 7f583de8f700 1 --2- 172.21.15.121:0/1055664872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58380c3280 0x7f583814c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.163+0000 7f583de8f700 1 -- 172.21.15.121:0/1055664872 >> 172.21.15.121:0/1055664872 conn(0x7f5838062190 msgr2=0x7f5838062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:22.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.163+0000 7f583de8f700 1 -- 172.21.15.121:0/1055664872 shutdown_connections 2023-11-15T02:57:22.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.163+0000 7f583de8f700 1 -- 172.21.15.121:0/1055664872 wait complete. 2023-11-15T02:57:22.189 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-15T02:56:49.726353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-15T02:56:50.688098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-15T02:57:22.189 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T02:57:22.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.343+0000 7f464725b700 1 Processor -- start 2023-11-15T02:57:22.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.343+0000 7f464725b700 1 -- start start 2023-11-15T02:57:22.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f464725b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f46400a8b80 0x7f46400a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:22.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f464725b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4640152150 con 0x7f4640068170 2023-11-15T02:57:22.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f464725b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4640148f70 con 0x7f4640065850 2023-11-15T02:57:22.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f464725b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46401523a0 con 0x7f46400a8b80 2023-11-15T02:57:22.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f46457f8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4640068170 0x7f46400bbe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:48232/0 (socket says 172.21.15.121:48232) 2023-11-15T02:57:22.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f46457f8700 1 -- 172.21.15.121:0/1421378900 learned_addr learned my addr 172.21.15.121:0/1421378900 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:22.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f4644ff7700 1 --2- 172.21.15.121:0/1421378900 >> v2:172.21.15.121:3300/0 conn(0x7f46400a8b80 0x7f46400a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:22.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f463f7fe700 1 -- 172.21.15.121:0/1421378900 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 708201960 0 0) 0x7f4640152150 con 0x7f4640068170 2023-11-15T02:57:22.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f463f7fe700 1 -- 172.21.15.121:0/1421378900 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46300033f0 con 0x7f4640068170 2023-11-15T02:57:22.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f4644ff7700 1 -- 172.21.15.121:0/1421378900 >> v1:172.21.15.190:6789/0 conn(0x7f4640065850 legacy=0x7f4640067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:22.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f4644ff7700 1 -- 172.21.15.121:0/1421378900 >> v1:172.21.15.121:6789/0 conn(0x7f4640068170 legacy=0x7f46400bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:22.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f4644ff7700 1 -- 172.21.15.121:0/1421378900 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4640152680 con 0x7f46400a8b80 2023-11-15T02:57:22.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f4644ff7700 1 --2- 172.21.15.121:0/1421378900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46400a8b80 0x7f46400a8f60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f4640149100 tx=0x7f4634009060).ready entity=mon.0 client_cookie=d0190964770e8a7f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:22.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f463f7fe700 1 -- 172.21.15.121:0/1421378900 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463400f510 con 0x7f46400a8b80 2023-11-15T02:57:22.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f463f7fe700 1 -- 172.21.15.121:0/1421378900 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463400f670 con 0x7f46400a8b80 2023-11-15T02:57:22.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f463f7fe700 1 -- 172.21.15.121:0/1421378900 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463400f9f0 con 0x7f46400a8b80 2023-11-15T02:57:22.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f464725b700 1 -- 172.21.15.121:0/1421378900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46400a8b80 msgr2=0x7f46400a8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:22.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f464725b700 1 --2- 172.21.15.121:0/1421378900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46400a8b80 0x7f46400a8f60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f4640149100 tx=0x7f4634009060).stop 2023-11-15T02:57:22.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f464725b700 1 -- 172.21.15.121:0/1421378900 shutdown_connections 2023-11-15T02:57:22.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f464725b700 1 --2- 172.21.15.121:0/1421378900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46400a8b80 0x7f46400a8f60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.347+0000 7f464725b700 1 -- 172.21.15.121:0/1421378900 >> 172.21.15.121:0/1421378900 conn(0x7f46400591c0 msgr2=0x7f46400c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:22.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.351+0000 7f464725b700 1 -- 172.21.15.121:0/1421378900 shutdown_connections 2023-11-15T02:57:22.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.351+0000 7f464725b700 1 -- 172.21.15.121:0/1421378900 wait complete. 2023-11-15T02:57:22.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.351+0000 7f464725b700 1 Processor -- start 2023-11-15T02:57:22.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.351+0000 7f464725b700 1 -- start start 2023-11-15T02:57:22.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f464725b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4640065850 0x7f46400c02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:22.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f464725b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4640068170 0x7f464013b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:22.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f464725b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46400a8b80 0x7f4640154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:22.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f464725b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46400bef10 con 0x7f4640065850 2023-11-15T02:57:22.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f464725b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46400aa7e0 con 0x7f46400a8b80 2023-11-15T02:57:22.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f464725b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46400aaac0 con 0x7f4640068170 2023-11-15T02:57:22.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f4644ff7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4640065850 0x7f46400c02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:22.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f4644ff7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4640065850 0x7f46400c02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39506/0 (socket says 172.21.15.121:39506) 2023-11-15T02:57:22.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f4644ff7700 1 -- 172.21.15.121:0/348085589 learned_addr learned my addr 172.21.15.121:0/348085589 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:22.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f46457f8700 1 --2- 172.21.15.121:0/348085589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46400a8b80 0x7f4640154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:22.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f463ffff700 1 --2- 172.21.15.121:0/348085589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4640068170 0x7f464013b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:22.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f4644ff7700 1 -- 172.21.15.121:0/348085589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4640068170 msgr2=0x7f464013b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:22.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f4644ff7700 1 --2- 172.21.15.121:0/348085589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4640068170 0x7f464013b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f4644ff7700 1 -- 172.21.15.121:0/348085589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46400a8b80 msgr2=0x7f4640154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:22.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f4644ff7700 1 --2- 172.21.15.121:0/348085589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46400a8b80 0x7f4640154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f4644ff7700 1 -- 172.21.15.121:0/348085589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46340092a0 con 0x7f4640065850 2023-11-15T02:57:22.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f46457f8700 1 --2- 172.21.15.121:0/348085589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46400a8b80 0x7f4640154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:57:22.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f463ffff700 1 --2- 172.21.15.121:0/348085589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4640068170 0x7f464013b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:57:22.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f4644ff7700 1 --2- 172.21.15.121:0/348085589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4640065850 0x7f46400c02e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f4634009dd0 tx=0x7f4634017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:22.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f463cff9700 1 -- 172.21.15.121:0/348085589 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46340117e0 con 0x7f4640065850 2023-11-15T02:57:22.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f463cff9700 1 -- 172.21.15.121:0/348085589 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4634011960 con 0x7f4640065850 2023-11-15T02:57:22.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f463cff9700 1 -- 172.21.15.121:0/348085589 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4634011d20 con 0x7f4640065850 2023-11-15T02:57:22.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f464725b700 1 -- 172.21.15.121:0/348085589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46400be690 con 0x7f4640065850 2023-11-15T02:57:22.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.355+0000 7f464725b700 1 -- 172.21.15.121:0/348085589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46400bec30 con 0x7f4640065850 2023-11-15T02:57:22.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.359+0000 7f464725b700 1 -- 172.21.15.121:0/348085589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4640047060 con 0x7f4640065850 2023-11-15T02:57:22.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.359+0000 7f463cff9700 1 -- 172.21.15.121:0/348085589 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4634020070 con 0x7f4640065850 2023-11-15T02:57:22.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.359+0000 7f463cff9700 1 --2- 172.21.15.121:0/348085589 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f462405a540 0x7f462405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:22.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.359+0000 7f463cff9700 1 -- 172.21.15.121:0/348085589 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4634075b10 con 0x7f4640065850 2023-11-15T02:57:22.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.363+0000 7f463ffff700 1 --2- 172.21.15.121:0/348085589 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f462405a540 0x7f462405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:22.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.363+0000 7f463ffff700 1 --2- 172.21.15.121:0/348085589 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f462405a540 0x7f462405ca00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f4640067e70 tx=0x7f462c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:22.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.367+0000 7f463cff9700 1 -- 172.21.15.121:0/348085589 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4634049230 con 0x7f4640065850 2023-11-15T02:57:22.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.527+0000 7f464725b700 1 -- 172.21.15.121:0/348085589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4640063b60 con 0x7f4640065850 2023-11-15T02:57:22.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.527+0000 7f463cff9700 1 -- 172.21.15.121:0/348085589 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v21) v1 ==== 110+0+10 (secure 0 0 0) 0x7f463404c850 con 0x7f4640065850 2023-11-15T02:57:22.532 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T02:57:22.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.531+0000 7f464725b700 1 -- 172.21.15.121:0/348085589 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f462405a540 msgr2=0x7f462405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:22.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.531+0000 7f464725b700 1 --2- 172.21.15.121:0/348085589 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f462405a540 0x7f462405ca00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f4640067e70 tx=0x7f462c009040).stop 2023-11-15T02:57:22.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.531+0000 7f464725b700 1 -- 172.21.15.121:0/348085589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4640065850 msgr2=0x7f46400c02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:22.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.531+0000 7f464725b700 1 --2- 172.21.15.121:0/348085589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4640065850 0x7f46400c02e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f4634009dd0 tx=0x7f4634017000).stop 2023-11-15T02:57:22.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.531+0000 7f464725b700 1 -- 172.21.15.121:0/348085589 shutdown_connections 2023-11-15T02:57:22.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.531+0000 7f464725b700 1 --2- 172.21.15.121:0/348085589 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f462405a540 0x7f462405ca00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.531+0000 7f464725b700 1 --2- 172.21.15.121:0/348085589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46400a8b80 0x7f4640154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.531+0000 7f464725b700 1 --2- 172.21.15.121:0/348085589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4640068170 0x7f464013b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.531+0000 7f464725b700 1 --2- 172.21.15.121:0/348085589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4640065850 0x7f46400c02e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.531+0000 7f464725b700 1 -- 172.21.15.121:0/348085589 >> 172.21.15.121:0/348085589 conn(0x7f46400591c0 msgr2=0x7f4640066190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:22.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.531+0000 7f464725b700 1 -- 172.21.15.121:0/348085589 shutdown_connections 2023-11-15T02:57:22.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.531+0000 7f464725b700 1 -- 172.21.15.121:0/348085589 wait complete. 2023-11-15T02:57:22.558 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T02:57:22.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.703+0000 7f4a4a69d700 1 Processor -- start 2023-11-15T02:57:22.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.707+0000 7f4a4a69d700 1 -- start start 2023-11-15T02:57:22.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.707+0000 7f4a4a69d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4414e380 0x7f4a440c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:22.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.707+0000 7f4a4a69d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4a440c6040 0x7f4a440acb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:22.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.707+0000 7f4a4a69d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4a440ad050 0x7f4a44062190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:22.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.707+0000 7f4a4a69d700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a440c8320 con 0x7f4a440c6040 2023-11-15T02:57:22.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.707+0000 7f4a4a69d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a440c7080 con 0x7f4a440ad050 2023-11-15T02:57:22.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.707+0000 7f4a4a69d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440ad500 con 0x7f4a4414e380 2023-11-15T02:57:22.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.707+0000 7f4a48e9a700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4a440c6040 0x7f4a440acb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:22.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.707+0000 7f4a48e9a700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4a440c6040 0x7f4a440acb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39522/0 (socket says 172.21.15.121:39522) 2023-11-15T02:57:22.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.707+0000 7f4a48e9a700 1 -- 172.21.15.121:0/1812021327 learned_addr learned my addr 172.21.15.121:0/1812021327 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:22.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.707+0000 7f4a49e9c700 1 --2- 172.21.15.121:0/1812021327 >> v2:172.21.15.190:3300/0 conn(0x7f4a440ad050 0x7f4a44062190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:22.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.707+0000 7f4a4969b700 1 --2- 172.21.15.121:0/1812021327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4414e380 0x7f4a440c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:22.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.707+0000 7f4a48e9a700 1 -- 172.21.15.121:0/1812021327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4414e380 msgr2=0x7f4a440c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:22.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.707+0000 7f4a48e9a700 1 --2- 172.21.15.121:0/1812021327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4414e380 0x7f4a440c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.707+0000 7f4a48e9a700 1 -- 172.21.15.121:0/1812021327 >> v2:172.21.15.190:3300/0 conn(0x7f4a440ad050 msgr2=0x7f4a44062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:22.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.707+0000 7f4a48e9a700 1 --2- 172.21.15.121:0/1812021327 >> v2:172.21.15.190:3300/0 conn(0x7f4a440ad050 0x7f4a44062190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.707+0000 7f4a48e9a700 1 -- 172.21.15.121:0/1812021327 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a440c7e90 con 0x7f4a440c6040 2023-11-15T02:57:22.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.707+0000 7f4a48e9a700 1 --2- 172.21.15.121:0/1812021327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a440c6040 0x7f4a440acb60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f4a44069800 tx=0x7f4a340090f0).ready entity=mon.0 client_cookie=5dc0619e8b91fbba server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:22.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.711+0000 7f4a3bfff700 1 -- 172.21.15.121:0/1812021327 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3400b600 con 0x7f4a440c6040 2023-11-15T02:57:22.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.711+0000 7f4a3bfff700 1 -- 172.21.15.121:0/1812021327 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3400b760 con 0x7f4a440c6040 2023-11-15T02:57:22.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.711+0000 7f4a3bfff700 1 -- 172.21.15.121:0/1812021327 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3400bae0 con 0x7f4a440c6040 2023-11-15T02:57:22.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.711+0000 7f4a4a69d700 1 -- 172.21.15.121:0/1812021327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a440c6040 msgr2=0x7f4a440acb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:22.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.711+0000 7f4a4a69d700 1 --2- 172.21.15.121:0/1812021327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a440c6040 0x7f4a440acb60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f4a44069800 tx=0x7f4a340090f0).stop 2023-11-15T02:57:22.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.711+0000 7f4a4a69d700 1 -- 172.21.15.121:0/1812021327 shutdown_connections 2023-11-15T02:57:22.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.711+0000 7f4a4a69d700 1 --2- 172.21.15.121:0/1812021327 >> v2:172.21.15.190:3300/0 conn(0x7f4a440ad050 0x7f4a44062190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.711+0000 7f4a4a69d700 1 --2- 172.21.15.121:0/1812021327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a440c6040 0x7f4a440acb60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.711+0000 7f4a4a69d700 1 --2- 172.21.15.121:0/1812021327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4414e380 0x7f4a440c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.711+0000 7f4a4a69d700 1 -- 172.21.15.121:0/1812021327 >> 172.21.15.121:0/1812021327 conn(0x7f4a44067a60 msgr2=0x7f4a44067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:22.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.711+0000 7f4a4a69d700 1 -- 172.21.15.121:0/1812021327 shutdown_connections 2023-11-15T02:57:22.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.711+0000 7f4a4a69d700 1 -- 172.21.15.121:0/1812021327 wait complete. 2023-11-15T02:57:22.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.715+0000 7f4a4a69d700 1 Processor -- start 2023-11-15T02:57:22.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.715+0000 7f4a4a69d700 1 -- start start 2023-11-15T02:57:22.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.715+0000 7f4a4a69d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a440ad050 0x7f4a44164680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:22.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.715+0000 7f4a4a69d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a440c6040 0x7f4a440aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:22.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.715+0000 7f4a4a69d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4414e380 0x7f4a44155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:22.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.715+0000 7f4a4a69d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4414c110 con 0x7f4a440ad050 2023-11-15T02:57:22.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.715+0000 7f4a4a69d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440c1a70 con 0x7f4a440c6040 2023-11-15T02:57:22.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.715+0000 7f4a4a69d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440c3df0 con 0x7f4a4414e380 2023-11-15T02:57:22.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.715+0000 7f4a4969b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a440ad050 0x7f4a44164680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:22.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.715+0000 7f4a4969b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a440ad050 0x7f4a44164680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39532/0 (socket says 172.21.15.121:39532) 2023-11-15T02:57:22.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.715+0000 7f4a4969b700 1 -- 172.21.15.121:0/2888989777 learned_addr learned my addr 172.21.15.121:0/2888989777 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:22.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.715+0000 7f4a48e9a700 1 --2- 172.21.15.121:0/2888989777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a440c6040 0x7f4a440aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:22.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.715+0000 7f4a49e9c700 1 --2- 172.21.15.121:0/2888989777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4414e380 0x7f4a44155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:22.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.715+0000 7f4a4969b700 1 -- 172.21.15.121:0/2888989777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4414e380 msgr2=0x7f4a44155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:22.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.715+0000 7f4a4969b700 1 --2- 172.21.15.121:0/2888989777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4414e380 0x7f4a44155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.715+0000 7f4a4969b700 1 -- 172.21.15.121:0/2888989777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a440c6040 msgr2=0x7f4a440aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:22.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.715+0000 7f4a4969b700 1 --2- 172.21.15.121:0/2888989777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a440c6040 0x7f4a440aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.715+0000 7f4a4969b700 1 -- 172.21.15.121:0/2888989777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a34009370 con 0x7f4a440ad050 2023-11-15T02:57:22.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.715+0000 7f4a4969b700 1 --2- 172.21.15.121:0/2888989777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a440ad050 0x7f4a44164680 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f4a4000d890 tx=0x7f4a40008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:22.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.719+0000 7f4a397fa700 1 -- 172.21.15.121:0/2888989777 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40011590 con 0x7f4a440ad050 2023-11-15T02:57:22.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.719+0000 7f4a397fa700 1 -- 172.21.15.121:0/2888989777 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a400116f0 con 0x7f4a440ad050 2023-11-15T02:57:22.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.719+0000 7f4a4a69d700 1 -- 172.21.15.121:0/2888989777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a440c40d0 con 0x7f4a440ad050 2023-11-15T02:57:22.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.719+0000 7f4a397fa700 1 -- 172.21.15.121:0/2888989777 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40011a70 con 0x7f4a440ad050 2023-11-15T02:57:22.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.719+0000 7f4a4a69d700 1 -- 172.21.15.121:0/2888989777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a440c4710 con 0x7f4a440ad050 2023-11-15T02:57:22.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.719+0000 7f4a4a69d700 1 -- 172.21.15.121:0/2888989777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4406bcf0 con 0x7f4a440ad050 2023-11-15T02:57:22.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.719+0000 7f4a397fa700 1 -- 172.21.15.121:0/2888989777 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4a40012420 con 0x7f4a440ad050 2023-11-15T02:57:22.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.723+0000 7f4a397fa700 1 --2- 172.21.15.121:0/2888989777 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a2805a510 0x7f4a2805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:22.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.723+0000 7f4a397fa700 1 -- 172.21.15.121:0/2888989777 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4a40074180 con 0x7f4a440ad050 2023-11-15T02:57:22.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.723+0000 7f4a48e9a700 1 --2- 172.21.15.121:0/2888989777 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a2805a510 0x7f4a2805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:22.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.723+0000 7f4a48e9a700 1 --2- 172.21.15.121:0/2888989777 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a2805a510 0x7f4a2805c9d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f4a44069800 tx=0x7f4a34017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:22.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.727+0000 7f4a397fa700 1 -- 172.21.15.121:0/2888989777 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4a40047840 con 0x7f4a440ad050 2023-11-15T02:57:22.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.891+0000 7f4a4a69d700 1 -- 172.21.15.121:0/2888989777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4a4414e760 con 0x7f4a440ad050 2023-11-15T02:57:22.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.891+0000 7f4a397fa700 1 -- 172.21.15.121:0/2888989777 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4a40011e60 con 0x7f4a440ad050 2023-11-15T02:57:22.898 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T02:57:22.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.895+0000 7f4a4a69d700 1 -- 172.21.15.121:0/2888989777 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a2805a510 msgr2=0x7f4a2805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:22.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.895+0000 7f4a4a69d700 1 --2- 172.21.15.121:0/2888989777 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a2805a510 0x7f4a2805c9d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f4a44069800 tx=0x7f4a34017000).stop 2023-11-15T02:57:22.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.895+0000 7f4a4a69d700 1 -- 172.21.15.121:0/2888989777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a440ad050 msgr2=0x7f4a44164680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:22.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.895+0000 7f4a4a69d700 1 --2- 172.21.15.121:0/2888989777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a440ad050 0x7f4a44164680 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f4a4000d890 tx=0x7f4a40008040).stop 2023-11-15T02:57:22.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.895+0000 7f4a4a69d700 1 -- 172.21.15.121:0/2888989777 shutdown_connections 2023-11-15T02:57:22.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.895+0000 7f4a4a69d700 1 --2- 172.21.15.121:0/2888989777 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a2805a510 0x7f4a2805c9d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.895+0000 7f4a4a69d700 1 --2- 172.21.15.121:0/2888989777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4414e380 0x7f4a44155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.895+0000 7f4a4a69d700 1 --2- 172.21.15.121:0/2888989777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a440c6040 0x7f4a440aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.895+0000 7f4a4a69d700 1 --2- 172.21.15.121:0/2888989777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a440ad050 0x7f4a44164680 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:22.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.895+0000 7f4a4a69d700 1 -- 172.21.15.121:0/2888989777 >> 172.21.15.121:0/2888989777 conn(0x7f4a44067a60 msgr2=0x7f4a4414c390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:22.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.895+0000 7f4a4a69d700 1 -- 172.21.15.121:0/2888989777 shutdown_connections 2023-11-15T02:57:22.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:22.895+0000 7f4a4a69d700 1 -- 172.21.15.121:0/2888989777 wait complete. 2023-11-15T02:57:22.920 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-15T02:57:23.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.087+0000 7f48ad979700 1 Processor -- start 2023-11-15T02:57:23.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.087+0000 7f48ad979700 1 -- start start 2023-11-15T02:57:23.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.087+0000 7f48ad979700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f48a81528c0 0x7f48a80bee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.087+0000 7f48ad979700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f48a80c3c10 0x7f48a80c5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.087+0000 7f48ad979700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a80be990 con 0x7f48a80bf310 2023-11-15T02:57:23.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.087+0000 7f48ad979700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a80bdd80 con 0x7f48a81528c0 2023-11-15T02:57:23.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.087+0000 7f48ad979700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a80be060 con 0x7f48a80c3c10 2023-11-15T02:57:23.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.091+0000 7f48ac977700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f48a81528c0 0x7f48a80bee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:23.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.091+0000 7f48a7fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f48a80bf310 0x7f48a80c3500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:57066/0 (socket says 172.21.15.121:57066) 2023-11-15T02:57:23.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.091+0000 7f48ad178700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f48a80c3c10 0x7f48a80c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:23.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.091+0000 7f48a7fff700 1 -- 172.21.15.121:0/937521615 learned_addr learned my addr 172.21.15.121:0/937521615 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:23.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.091+0000 7f48ac977700 1 -- 172.21.15.121:0/937521615 >> v2:172.21.15.190:3300/0 conn(0x7f48a80c3c10 msgr2=0x7f48a80c5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:23.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.091+0000 7f48ac977700 1 --2- 172.21.15.121:0/937521615 >> v2:172.21.15.190:3300/0 conn(0x7f48a80c3c10 0x7f48a80c5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.091+0000 7f48ac977700 1 -- 172.21.15.121:0/937521615 >> v1:172.21.15.190:6789/0 conn(0x7f48a80bf310 legacy=0x7f48a80c3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:23.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.091+0000 7f48ac977700 1 -- 172.21.15.121:0/937521615 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a80be340 con 0x7f48a81528c0 2023-11-15T02:57:23.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.091+0000 7f48ac977700 1 --2- 172.21.15.121:0/937521615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48a81528c0 0x7f48a80bee20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f48a80699d0 tx=0x7f489c009060).ready entity=mon.0 client_cookie=9f6f9587afc84066 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:23.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.091+0000 7f48a77fe700 1 -- 172.21.15.121:0/937521615 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c00f510 con 0x7f48a81528c0 2023-11-15T02:57:23.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.091+0000 7f48a77fe700 1 -- 172.21.15.121:0/937521615 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489c00f670 con 0x7f48a81528c0 2023-11-15T02:57:23.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.091+0000 7f48a77fe700 1 -- 172.21.15.121:0/937521615 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c00f9f0 con 0x7f48a81528c0 2023-11-15T02:57:23.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.091+0000 7f48ad979700 1 -- 172.21.15.121:0/937521615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48a81528c0 msgr2=0x7f48a80bee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:23.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.091+0000 7f48ad979700 1 --2- 172.21.15.121:0/937521615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48a81528c0 0x7f48a80bee20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f48a80699d0 tx=0x7f489c009060).stop 2023-11-15T02:57:23.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.091+0000 7f48ad979700 1 -- 172.21.15.121:0/937521615 shutdown_connections 2023-11-15T02:57:23.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.095+0000 7f48ad979700 1 --2- 172.21.15.121:0/937521615 >> v2:172.21.15.190:3300/0 conn(0x7f48a80c3c10 0x7f48a80c5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.095+0000 7f48ad979700 1 --2- 172.21.15.121:0/937521615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48a81528c0 0x7f48a80bee20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.095+0000 7f48ad979700 1 -- 172.21.15.121:0/937521615 >> 172.21.15.121:0/937521615 conn(0x7f48a8067a60 msgr2=0x7f48a8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:23.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.095+0000 7f48ad979700 1 -- 172.21.15.121:0/937521615 shutdown_connections 2023-11-15T02:57:23.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.095+0000 7f48ad979700 1 -- 172.21.15.121:0/937521615 wait complete. 2023-11-15T02:57:23.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.095+0000 7f48ad979700 1 Processor -- start 2023-11-15T02:57:23.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.095+0000 7f48ad979700 1 -- start start 2023-11-15T02:57:23.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48ad979700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48a80bf310 0x7f48a8163ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48ad979700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48a80c3c10 0x7f48a8062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48ad979700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48a81528c0 0x7f48a8153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48ad979700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a81643b0 con 0x7f48a80c3c10 2023-11-15T02:57:23.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48ad979700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a8167e80 con 0x7f48a80bf310 2023-11-15T02:57:23.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48ad979700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a8168160 con 0x7f48a81528c0 2023-11-15T02:57:23.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48a7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48a80c3c10 0x7f48a8062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:23.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48a7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48a80c3c10 0x7f48a8062f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39556/0 (socket says 172.21.15.121:39556) 2023-11-15T02:57:23.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48a7fff700 1 -- 172.21.15.121:0/2504900091 learned_addr learned my addr 172.21.15.121:0/2504900091 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:23.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48ac977700 1 --2- 172.21.15.121:0/2504900091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48a80bf310 0x7f48a8163ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:23.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48ad178700 1 --2- 172.21.15.121:0/2504900091 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48a81528c0 0x7f48a8153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:23.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48a7fff700 1 -- 172.21.15.121:0/2504900091 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48a81528c0 msgr2=0x7f48a8153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:23.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48a7fff700 1 --2- 172.21.15.121:0/2504900091 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48a81528c0 0x7f48a8153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48a7fff700 1 -- 172.21.15.121:0/2504900091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48a80bf310 msgr2=0x7f48a8163ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:23.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48a7fff700 1 --2- 172.21.15.121:0/2504900091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48a80bf310 0x7f48a8163ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48a7fff700 1 -- 172.21.15.121:0/2504900091 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c0092a0 con 0x7f48a80c3c10 2023-11-15T02:57:23.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48a7fff700 1 --2- 172.21.15.121:0/2504900091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48a80c3c10 0x7f48a8062f20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f4894007fd0 tx=0x7f4894007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:23.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48a4ff9700 1 -- 172.21.15.121:0/2504900091 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489400ce10 con 0x7f48a80c3c10 2023-11-15T02:57:23.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48a4ff9700 1 -- 172.21.15.121:0/2504900091 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48940072e0 con 0x7f48a80c3c10 2023-11-15T02:57:23.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48a4ff9700 1 -- 172.21.15.121:0/2504900091 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894007660 con 0x7f48a80c3c10 2023-11-15T02:57:23.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48ad979700 1 -- 172.21.15.121:0/2504900091 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a8168440 con 0x7f48a80c3c10 2023-11-15T02:57:23.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.099+0000 7f48ad979700 1 -- 172.21.15.121:0/2504900091 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a8167b10 con 0x7f48a80c3c10 2023-11-15T02:57:23.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.103+0000 7f48ad979700 1 -- 172.21.15.121:0/2504900091 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a814e420 con 0x7f48a80c3c10 2023-11-15T02:57:23.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.103+0000 7f48a4ff9700 1 -- 172.21.15.121:0/2504900091 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f48940059b0 con 0x7f48a80c3c10 2023-11-15T02:57:23.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.103+0000 7f48a4ff9700 1 --2- 172.21.15.121:0/2504900091 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f488c05a510 0x7f488c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.103+0000 7f48a4ff9700 1 -- 172.21.15.121:0/2504900091 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4894073ed0 con 0x7f48a80c3c10 2023-11-15T02:57:23.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.103+0000 7f48ac977700 1 --2- 172.21.15.121:0/2504900091 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f488c05a510 0x7f488c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:23.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.103+0000 7f48ac977700 1 --2- 172.21.15.121:0/2504900091 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f488c05a510 0x7f488c05c9d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f489c009630 tx=0x7f489c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:23.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.111+0000 7f48a4ff9700 1 -- 172.21.15.121:0/2504900091 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f48a814e420 con 0x7f48a80c3c10 2023-11-15T02:57:23.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.271+0000 7f48ad979700 1 -- 172.21.15.121:0/2504900091 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f48a814e420 con 0x7f48a80c3c10 2023-11-15T02:57:23.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.271+0000 7f48a4ff9700 1 -- 172.21.15.121:0/2504900091 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f48a814e420 con 0x7f48a80c3c10 2023-11-15T02:57:23.278 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T02:57:23.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.275+0000 7f48ad979700 1 -- 172.21.15.121:0/2504900091 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f488c05a510 msgr2=0x7f488c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:23.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.275+0000 7f48ad979700 1 --2- 172.21.15.121:0/2504900091 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f488c05a510 0x7f488c05c9d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f489c009630 tx=0x7f489c017000).stop 2023-11-15T02:57:23.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.275+0000 7f48ad979700 1 -- 172.21.15.121:0/2504900091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48a80c3c10 msgr2=0x7f48a8062f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:23.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.275+0000 7f48ad979700 1 --2- 172.21.15.121:0/2504900091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48a80c3c10 0x7f48a8062f20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f4894007fd0 tx=0x7f4894007df0).stop 2023-11-15T02:57:23.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.275+0000 7f48ad979700 1 -- 172.21.15.121:0/2504900091 shutdown_connections 2023-11-15T02:57:23.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.275+0000 7f48ad979700 1 --2- 172.21.15.121:0/2504900091 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f488c05a510 0x7f488c05c9d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.275+0000 7f48ad979700 1 --2- 172.21.15.121:0/2504900091 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48a81528c0 0x7f48a8153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.275+0000 7f48ad979700 1 --2- 172.21.15.121:0/2504900091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48a80c3c10 0x7f48a8062f20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.275+0000 7f48ad979700 1 --2- 172.21.15.121:0/2504900091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48a80bf310 0x7f48a8163ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.275+0000 7f48ad979700 1 -- 172.21.15.121:0/2504900091 >> 172.21.15.121:0/2504900091 conn(0x7f48a8067a60 msgr2=0x7f48a80a7980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:23.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.275+0000 7f48ad979700 1 -- 172.21.15.121:0/2504900091 shutdown_connections 2023-11-15T02:57:23.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.275+0000 7f48ad979700 1 -- 172.21.15.121:0/2504900091 wait complete. 2023-11-15T02:57:23.300 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-15T02:57:23.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.455+0000 7fb902e79700 1 Processor -- start 2023-11-15T02:57:23.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.455+0000 7fb902e79700 1 -- start start 2023-11-15T02:57:23.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.455+0000 7fb902e79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8fc066330 0x7fb8fc153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb902e79700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8fc065ea0 con 0x7fb8fc0acd00 2023-11-15T02:57:23.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb902e79700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8fc066bb0 con 0x7fb8fc0c5c70 2023-11-15T02:57:23.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb902e79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc065890 con 0x7fb8fc066330 2023-11-15T02:57:23.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb901e77700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb8fc0acd00 0x7fb8fc1536c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:48244/0 (socket says 172.21.15.121:48244) 2023-11-15T02:57:23.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb901e77700 1 -- 172.21.15.121:0/1216761528 learned_addr learned my addr 172.21.15.121:0/1216761528 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:23.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb902678700 1 --2- 172.21.15.121:0/1216761528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8fc066330 0x7fb8fc153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:23.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb900e75700 1 -- 172.21.15.121:0/1216761528 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3944698033 0 0) 0x7fb8fc065ea0 con 0x7fb8fc0acd00 2023-11-15T02:57:23.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb900e75700 1 -- 172.21.15.121:0/1216761528 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8e80033f0 con 0x7fb8fc0acd00 2023-11-15T02:57:23.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb900e75700 1 -- 172.21.15.121:0/1216761528 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3681837393 0 0) 0x7fb8e80033f0 con 0x7fb8fc0acd00 2023-11-15T02:57:23.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb900e75700 1 -- 172.21.15.121:0/1216761528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8fc066330 msgr2=0x7fb8fc153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:23.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb900e75700 1 --2- 172.21.15.121:0/1216761528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8fc066330 0x7fb8fc153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb900e75700 1 -- 172.21.15.121:0/1216761528 >> v1:172.21.15.190:6789/0 conn(0x7fb8fc0c5c70 legacy=0x7fb8fc1577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:23.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb900e75700 1 -- 172.21.15.121:0/1216761528 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8fc0c64a0 con 0x7fb8fc0acd00 2023-11-15T02:57:23.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb900e75700 1 -- 172.21.15.121:0/1216761528 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fb8f80015f0 con 0x7fb8fc0acd00 2023-11-15T02:57:23.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb900e75700 1 -- 172.21.15.121:0/1216761528 >> v1:172.21.15.121:6789/0 conn(0x7fb8fc0acd00 legacy=0x7fb8fc1536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:23.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb900e75700 1 --2- 172.21.15.121:0/1216761528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8fc066330 0x7fb8e8003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb900e75700 1 --2- 172.21.15.121:0/1216761528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8e8003e00 0x7fb8e8004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb900e75700 1 --2- 172.21.15.121:0/1216761528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8e8004b80 0x7fb8e8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb900e75700 1 -- 172.21.15.121:0/1216761528 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc065890 con 0x7fb8e8004b80 2023-11-15T02:57:23.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb900e75700 1 -- 172.21.15.121:0/1216761528 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e800b8e0 con 0x7fb8e8003e00 2023-11-15T02:57:23.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb900e75700 1 -- 172.21.15.121:0/1216761528 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e800baa0 con 0x7fb8fc066330 2023-11-15T02:57:23.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb902678700 1 --2- 172.21.15.121:0/1216761528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8e8004b80 0x7fb8e8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:23.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb901676700 1 --2- 172.21.15.121:0/1216761528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8e8003e00 0x7fb8e8004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:23.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb902678700 1 -- 172.21.15.121:0/1216761528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8fc066330 msgr2=0x7fb8e8003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:23.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb902678700 1 --2- 172.21.15.121:0/1216761528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8fc066330 0x7fb8e8003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T02:57:23.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb902678700 1 -- 172.21.15.121:0/1216761528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8e8003e00 msgr2=0x7fb8e8004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:23.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb902678700 1 --2- 172.21.15.121:0/1216761528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8e8003e00 0x7fb8e8004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb902678700 1 -- 172.21.15.121:0/1216761528 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e80033f0 con 0x7fb8e8004b80 2023-11-15T02:57:23.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb901676700 1 --2- 172.21.15.121:0/1216761528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8e8003e00 0x7fb8e8004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:57:23.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb902678700 1 --2- 172.21.15.121:0/1216761528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8e8004b80 0x7fb8e8005000 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fb8f400e210 tx=0x7fb8f400b040).ready entity=mon.0 client_cookie=b1b10988f5122a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:23.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb900e75700 1 -- 172.21.15.121:0/1216761528 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4009ac0 con 0x7fb8e8004b80 2023-11-15T02:57:23.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb900e75700 1 -- 172.21.15.121:0/1216761528 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f400f040 con 0x7fb8e8004b80 2023-11-15T02:57:23.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb900e75700 1 -- 172.21.15.121:0/1216761528 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f400ecd0 con 0x7fb8e8004b80 2023-11-15T02:57:23.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb902e79700 1 -- 172.21.15.121:0/1216761528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8e8004b80 msgr2=0x7fb8e8005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:23.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb902e79700 1 --2- 172.21.15.121:0/1216761528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8e8004b80 0x7fb8e8005000 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fb8f400e210 tx=0x7fb8f400b040).stop 2023-11-15T02:57:23.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb902678700 1 -- 172.21.15.121:0/1216761528 reap_dead start 2023-11-15T02:57:23.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb902e79700 1 -- 172.21.15.121:0/1216761528 shutdown_connections 2023-11-15T02:57:23.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.459+0000 7fb902e79700 1 -- 172.21.15.121:0/1216761528 >> 172.21.15.121:0/1216761528 conn(0x7fb8fc063fc0 msgr2=0x7fb8fc067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:23.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.463+0000 7fb902e79700 1 -- 172.21.15.121:0/1216761528 shutdown_connections 2023-11-15T02:57:23.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.463+0000 7fb902e79700 1 -- 172.21.15.121:0/1216761528 wait complete. 2023-11-15T02:57:23.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.463+0000 7fb902e79700 1 Processor -- start 2023-11-15T02:57:23.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.463+0000 7fb902e79700 1 -- start start 2023-11-15T02:57:23.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.463+0000 7fb902e79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8e8004b80 0x7fb8fc13ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.463+0000 7fb902e79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8fc152970 0x7fb8fc0c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.463+0000 7fb902e79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8fc0c8c60 0x7fb8fc150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.463+0000 7fb902e79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc0c90e0 con 0x7fb8fc0c8c60 2023-11-15T02:57:23.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.463+0000 7fb902e79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc1518d0 con 0x7fb8fc152970 2023-11-15T02:57:23.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.463+0000 7fb902e79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc151bb0 con 0x7fb8e8004b80 2023-11-15T02:57:23.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.463+0000 7fb902678700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8fc0c8c60 0x7fb8fc150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:23.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.463+0000 7fb902678700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8fc0c8c60 0x7fb8fc150b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39568/0 (socket says 172.21.15.121:39568) 2023-11-15T02:57:23.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.463+0000 7fb902678700 1 -- 172.21.15.121:0/3013410181 learned_addr learned my addr 172.21.15.121:0/3013410181 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:23.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.467+0000 7fb901676700 1 --2- 172.21.15.121:0/3013410181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8fc152970 0x7fb8fc0c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:23.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.467+0000 7fb901e77700 1 --2- 172.21.15.121:0/3013410181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8e8004b80 0x7fb8fc13ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:23.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.467+0000 7fb902678700 1 -- 172.21.15.121:0/3013410181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8e8004b80 msgr2=0x7fb8fc13ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:23.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.467+0000 7fb902678700 1 --2- 172.21.15.121:0/3013410181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8e8004b80 0x7fb8fc13ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.467+0000 7fb902678700 1 -- 172.21.15.121:0/3013410181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8fc152970 msgr2=0x7fb8fc0c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:23.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.467+0000 7fb902678700 1 --2- 172.21.15.121:0/3013410181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8fc152970 0x7fb8fc0c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.467+0000 7fb902678700 1 -- 172.21.15.121:0/3013410181 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8fc151e90 con 0x7fb8fc0c8c60 2023-11-15T02:57:23.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.467+0000 7fb901676700 1 --2- 172.21.15.121:0/3013410181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8fc152970 0x7fb8fc0c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:57:23.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.467+0000 7fb901e77700 1 --2- 172.21.15.121:0/3013410181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8e8004b80 0x7fb8fc13ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:57:23.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.467+0000 7fb902678700 1 --2- 172.21.15.121:0/3013410181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8fc0c8c60 0x7fb8fc150b80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fb8e8004610 tx=0x7fb8f401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:23.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.467+0000 7fb8f1ffb700 1 -- 172.21.15.121:0/3013410181 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4021030 con 0x7fb8fc0c8c60 2023-11-15T02:57:23.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.467+0000 7fb8f1ffb700 1 -- 172.21.15.121:0/3013410181 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f4009ac0 con 0x7fb8fc0c8c60 2023-11-15T02:57:23.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.467+0000 7fb8f1ffb700 1 -- 172.21.15.121:0/3013410181 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4013e70 con 0x7fb8fc0c8c60 2023-11-15T02:57:23.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.467+0000 7fb902e79700 1 -- 172.21.15.121:0/3013410181 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8fc152170 con 0x7fb8fc0c8c60 2023-11-15T02:57:23.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.467+0000 7fb902e79700 1 -- 172.21.15.121:0/3013410181 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8fc13b200 con 0x7fb8fc0c8c60 2023-11-15T02:57:23.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.467+0000 7fb902e79700 1 -- 172.21.15.121:0/3013410181 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8fc13b3f0 con 0x7fb8fc0c8c60 2023-11-15T02:57:23.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.471+0000 7fb8f1ffb700 1 -- 172.21.15.121:0/3013410181 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb8f4024070 con 0x7fb8fc0c8c60 2023-11-15T02:57:23.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.471+0000 7fb8f1ffb700 1 --2- 172.21.15.121:0/3013410181 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb8e005a510 0x7fb8e005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.471+0000 7fb8f1ffb700 1 -- 172.21.15.121:0/3013410181 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb8f4022070 con 0x7fb8fc0c8c60 2023-11-15T02:57:23.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.471+0000 7fb901e77700 1 --2- 172.21.15.121:0/3013410181 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb8e005a510 0x7fb8e005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:23.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.471+0000 7fb901e77700 1 --2- 172.21.15.121:0/3013410181 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb8e005a510 0x7fb8e005c9d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fb8f8001190 tx=0x7fb8f8009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:23.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.479+0000 7fb8f1ffb700 1 -- 172.21.15.121:0/3013410181 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb8fc13b3f0 con 0x7fb8fc0c8c60 2023-11-15T02:57:23.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.639+0000 7fb902e79700 1 -- 172.21.15.121:0/3013410181 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb8fc13b3f0 con 0x7fb8fc0c8c60 2023-11-15T02:57:23.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.643+0000 7fb8f1ffb700 1 -- 172.21.15.121:0/3013410181 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb8fc13b3f0 con 0x7fb8fc0c8c60 2023-11-15T02:57:23.647 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T02:57:23.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.643+0000 7fb902e79700 1 -- 172.21.15.121:0/3013410181 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb8e005a510 msgr2=0x7fb8e005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:23.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.643+0000 7fb902e79700 1 --2- 172.21.15.121:0/3013410181 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb8e005a510 0x7fb8e005c9d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fb8f8001190 tx=0x7fb8f8009040).stop 2023-11-15T02:57:23.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.643+0000 7fb902e79700 1 -- 172.21.15.121:0/3013410181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8fc0c8c60 msgr2=0x7fb8fc150b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:23.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.643+0000 7fb902e79700 1 --2- 172.21.15.121:0/3013410181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8fc0c8c60 0x7fb8fc150b80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fb8e8004610 tx=0x7fb8f401a000).stop 2023-11-15T02:57:23.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.643+0000 7fb902e79700 1 -- 172.21.15.121:0/3013410181 shutdown_connections 2023-11-15T02:57:23.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.643+0000 7fb902e79700 1 --2- 172.21.15.121:0/3013410181 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb8e005a510 0x7fb8e005c9d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.643+0000 7fb902e79700 1 --2- 172.21.15.121:0/3013410181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8fc0c8c60 0x7fb8fc150b80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.643+0000 7fb902e79700 1 --2- 172.21.15.121:0/3013410181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8fc152970 0x7fb8fc0c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.643+0000 7fb902e79700 1 --2- 172.21.15.121:0/3013410181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8e8004b80 0x7fb8fc13ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.643+0000 7fb902e79700 1 -- 172.21.15.121:0/3013410181 >> 172.21.15.121:0/3013410181 conn(0x7fb8fc063fc0 msgr2=0x7fb8fc14f360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:23.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.643+0000 7fb902e79700 1 -- 172.21.15.121:0/3013410181 shutdown_connections 2023-11-15T02:57:23.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.643+0000 7fb902e79700 1 -- 172.21.15.121:0/3013410181 wait complete. 2023-11-15T02:57:23.669 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:57:23.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.819+0000 7f77db379700 1 Processor -- start 2023-11-15T02:57:23.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.823+0000 7f77db379700 1 -- start start 2023-11-15T02:57:23.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.823+0000 7f77db379700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f77d40a9830 0x7f77d40a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.823+0000 7f77db379700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f77d40c30e0 0x7f77d40c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.823+0000 7f77db379700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d40657f0 con 0x7f77d4151870 2023-11-15T02:57:23.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.823+0000 7f77db379700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d4065ad0 con 0x7f77d40c30e0 2023-11-15T02:57:23.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.823+0000 7f77db379700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d40c3a60 con 0x7f77d40a9830 2023-11-15T02:57:23.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.823+0000 7f77da377700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f77d4151870 0x7f77d4151c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:48256/0 (socket says 172.21.15.121:48256) 2023-11-15T02:57:23.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.823+0000 7f77da377700 1 -- 172.21.15.121:0/2204969375 learned_addr learned my addr 172.21.15.121:0/2204969375 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:23.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.823+0000 7f77d9b76700 1 --2- 172.21.15.121:0/2204969375 >> v2:172.21.15.190:3300/0 conn(0x7f77d40a9830 0x7f77d40a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:23.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.823+0000 7f77dab78700 1 --2- 172.21.15.121:0/2204969375 >> v2:172.21.15.121:3300/0 conn(0x7f77d40c30e0 0x7f77d40c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:23.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.823+0000 7f77d9375700 1 -- 172.21.15.121:0/2204969375 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039799782 0 0) 0x7f77d40657f0 con 0x7f77d4151870 2023-11-15T02:57:23.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.823+0000 7f77d9375700 1 -- 172.21.15.121:0/2204969375 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77c00033f0 con 0x7f77d4151870 2023-11-15T02:57:23.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.823+0000 7f77d9b76700 1 -- 172.21.15.121:0/2204969375 >> v2:172.21.15.121:3300/0 conn(0x7f77d40c30e0 msgr2=0x7f77d40c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:23.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.823+0000 7f77d9b76700 1 --2- 172.21.15.121:0/2204969375 >> v2:172.21.15.121:3300/0 conn(0x7f77d40c30e0 0x7f77d40c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.823+0000 7f77d9b76700 1 -- 172.21.15.121:0/2204969375 >> v1:172.21.15.121:6789/0 conn(0x7f77d4151870 legacy=0x7f77d4151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:23.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.823+0000 7f77d9b76700 1 -- 172.21.15.121:0/2204969375 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d40aa260 con 0x7f77d40a9830 2023-11-15T02:57:23.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.827+0000 7f77d9b76700 1 --2- 172.21.15.121:0/2204969375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77d40a9830 0x7f77d40a9c70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f77d4063d00 tx=0x7f77c4009060).ready entity=mon.1 client_cookie=cf3973a04cc9f2f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:23.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.827+0000 7f77d9375700 1 -- 172.21.15.121:0/2204969375 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c400d4a0 con 0x7f77d40a9830 2023-11-15T02:57:23.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.827+0000 7f77d9375700 1 -- 172.21.15.121:0/2204969375 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c400d600 con 0x7f77d40a9830 2023-11-15T02:57:23.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.827+0000 7f77d9375700 1 -- 172.21.15.121:0/2204969375 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c400d980 con 0x7f77d40a9830 2023-11-15T02:57:23.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.827+0000 7f77db379700 1 -- 172.21.15.121:0/2204969375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77d40a9830 msgr2=0x7f77d40a9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:23.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.827+0000 7f77db379700 1 --2- 172.21.15.121:0/2204969375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77d40a9830 0x7f77d40a9c70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f77d4063d00 tx=0x7f77c4009060).stop 2023-11-15T02:57:23.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.827+0000 7f77db379700 1 -- 172.21.15.121:0/2204969375 shutdown_connections 2023-11-15T02:57:23.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.827+0000 7f77db379700 1 --2- 172.21.15.121:0/2204969375 >> v2:172.21.15.121:3300/0 conn(0x7f77d40c30e0 0x7f77d40c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.827+0000 7f77db379700 1 --2- 172.21.15.121:0/2204969375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77d40a9830 0x7f77d40a9c70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.827+0000 7f77db379700 1 -- 172.21.15.121:0/2204969375 >> 172.21.15.121:0/2204969375 conn(0x7f77d4062190 msgr2=0x7f77d4062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:23.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.827+0000 7f77db379700 1 -- 172.21.15.121:0/2204969375 shutdown_connections 2023-11-15T02:57:23.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.827+0000 7f77db379700 1 -- 172.21.15.121:0/2204969375 wait complete. 2023-11-15T02:57:23.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77db379700 1 Processor -- start 2023-11-15T02:57:23.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77db379700 1 -- start start 2023-11-15T02:57:23.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77db379700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d40a9830 0x7f77d4157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77db379700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77d40c30e0 0x7f77d41582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77db379700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77d4151870 0x7f77d4153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77db379700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d4159460 con 0x7f77d40a9830 2023-11-15T02:57:23.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77db379700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d4158850 con 0x7f77d4151870 2023-11-15T02:57:23.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77db379700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d4158b30 con 0x7f77d40c30e0 2023-11-15T02:57:23.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77da377700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d40a9830 0x7f77d4157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:23.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77da377700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d40a9830 0x7f77d4157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39582/0 (socket says 172.21.15.121:39582) 2023-11-15T02:57:23.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77da377700 1 -- 172.21.15.121:0/1536304444 learned_addr learned my addr 172.21.15.121:0/1536304444 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:23.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77dab78700 1 --2- 172.21.15.121:0/1536304444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77d4151870 0x7f77d4153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:23.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77d9b76700 1 --2- 172.21.15.121:0/1536304444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77d40c30e0 0x7f77d41582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:23.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77da377700 1 -- 172.21.15.121:0/1536304444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77d40c30e0 msgr2=0x7f77d41582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:23.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77da377700 1 --2- 172.21.15.121:0/1536304444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77d40c30e0 0x7f77d41582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77da377700 1 -- 172.21.15.121:0/1536304444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77d4151870 msgr2=0x7f77d4153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:23.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77da377700 1 --2- 172.21.15.121:0/1536304444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77d4151870 0x7f77d4153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:23.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77da377700 1 -- 172.21.15.121:0/1536304444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c40092a0 con 0x7f77d40a9830 2023-11-15T02:57:23.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77da377700 1 --2- 172.21.15.121:0/1536304444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d40a9830 0x7f77d4157dc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f77d000c410 tx=0x7f77d0007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:23.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77ca7fc700 1 -- 172.21.15.121:0/1536304444 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d00070d0 con 0x7f77d40a9830 2023-11-15T02:57:23.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77ca7fc700 1 -- 172.21.15.121:0/1536304444 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d0007230 con 0x7f77d40a9830 2023-11-15T02:57:23.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77db379700 1 -- 172.21.15.121:0/1536304444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d4158e10 con 0x7f77d40a9830 2023-11-15T02:57:23.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77ca7fc700 1 -- 172.21.15.121:0/1536304444 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d00075d0 con 0x7f77d40a9830 2023-11-15T02:57:23.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.831+0000 7f77db379700 1 -- 172.21.15.121:0/1536304444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d4154f40 con 0x7f77d40a9830 2023-11-15T02:57:23.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.835+0000 7f77db379700 1 -- 172.21.15.121:0/1536304444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d406bcf0 con 0x7f77d40a9830 2023-11-15T02:57:23.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.835+0000 7f77ca7fc700 1 -- 172.21.15.121:0/1536304444 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f77d00059b0 con 0x7f77d40a9830 2023-11-15T02:57:23.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.835+0000 7f77ca7fc700 1 --2- 172.21.15.121:0/1536304444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f77b805a4e0 0x7f77b805c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:23.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.835+0000 7f77ca7fc700 1 -- 172.21.15.121:0/1536304444 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f77d0073d20 con 0x7f77d40a9830 2023-11-15T02:57:23.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.835+0000 7f77d9b76700 1 --2- 172.21.15.121:0/1536304444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f77b805a4e0 0x7f77b805c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:23.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.839+0000 7f77d9b76700 1 --2- 172.21.15.121:0/1536304444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f77b805a4e0 0x7f77b805c9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f77c4009fd0 tx=0x7f77c4016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:23.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:23.843+0000 7f77ca7fc700 1 -- 172.21.15.121:0/1536304444 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f77d0047500 con 0x7f77d40a9830 2023-11-15T02:57:24.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.007+0000 7f77db379700 1 -- 172.21.15.121:0/1536304444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77d40c4750 con 0x7f77d40a9830 2023-11-15T02:57:24.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.011+0000 7f77ca7fc700 1 -- 172.21.15.121:0/1536304444 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f77d0018030 con 0x7f77d40a9830 2023-11-15T02:57:24.015 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:24.015 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":21,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:52.702668+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:56:49.726353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-15T02:56:50.688098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:57:24.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.011+0000 7f77db379700 1 -- 172.21.15.121:0/1536304444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f77b805a4e0 msgr2=0x7f77b805c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:24.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.011+0000 7f77db379700 1 --2- 172.21.15.121:0/1536304444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f77b805a4e0 0x7f77b805c9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f77c4009fd0 tx=0x7f77c4016000).stop 2023-11-15T02:57:24.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.011+0000 7f77db379700 1 -- 172.21.15.121:0/1536304444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d40a9830 msgr2=0x7f77d4157dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:24.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.011+0000 7f77db379700 1 --2- 172.21.15.121:0/1536304444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d40a9830 0x7f77d4157dc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f77d000c410 tx=0x7f77d0007c70).stop 2023-11-15T02:57:24.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.011+0000 7f77db379700 1 -- 172.21.15.121:0/1536304444 shutdown_connections 2023-11-15T02:57:24.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.011+0000 7f77db379700 1 --2- 172.21.15.121:0/1536304444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f77b805a4e0 0x7f77b805c9a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.011+0000 7f77db379700 1 --2- 172.21.15.121:0/1536304444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77d4151870 0x7f77d4153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.011+0000 7f77db379700 1 --2- 172.21.15.121:0/1536304444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77d40c30e0 0x7f77d41582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.011+0000 7f77db379700 1 --2- 172.21.15.121:0/1536304444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d40a9830 0x7f77d4157dc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.011+0000 7f77db379700 1 -- 172.21.15.121:0/1536304444 >> 172.21.15.121:0/1536304444 conn(0x7f77d4062190 msgr2=0x7f77d414e360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:24.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.011+0000 7f77db379700 1 -- 172.21.15.121:0/1536304444 shutdown_connections 2023-11-15T02:57:24.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.011+0000 7f77db379700 1 -- 172.21.15.121:0/1536304444 wait complete. 2023-11-15T02:57:24.040 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-15T02:56:49.726353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-15T02:56:50.688098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-15T02:57:24.041 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T02:57:24.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.207+0000 7fca30a8f700 1 Processor -- start 2023-11-15T02:57:24.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.207+0000 7fca30a8f700 1 -- start start 2023-11-15T02:57:24.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.207+0000 7fca30a8f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fca2c14e380 0x7fca2c0c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.207+0000 7fca30a8f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fca2c0ad270 0x7fca2c065680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.207+0000 7fca30a8f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2c0c7e90 con 0x7fca2c0c6040 2023-11-15T02:57:24.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.207+0000 7fca30a8f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca2c0c8320 con 0x7fca2c0ad270 2023-11-15T02:57:24.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.207+0000 7fca30a8f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca2c0c7080 con 0x7fca2c14e380 2023-11-15T02:57:24.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca2bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fca2c0ad270 0x7fca2c065680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca2bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fca2c0ad270 0x7fca2c065680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39584/0 (socket says 172.21.15.121:39584) 2023-11-15T02:57:24.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca2bfff700 1 -- 172.21.15.121:0/1281370839 learned_addr learned my addr 172.21.15.121:0/1281370839 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:24.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca2b7fe700 1 --2- 172.21.15.121:0/1281370839 >> v2:172.21.15.190:3300/0 conn(0x7fca2c14e380 0x7fca2c0c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca2bfff700 1 -- 172.21.15.121:0/1281370839 >> v2:172.21.15.190:3300/0 conn(0x7fca2c14e380 msgr2=0x7fca2c0c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:24.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca2bfff700 1 --2- 172.21.15.121:0/1281370839 >> v2:172.21.15.190:3300/0 conn(0x7fca2c14e380 0x7fca2c0c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca2bfff700 1 -- 172.21.15.121:0/1281370839 >> v1:172.21.15.190:6789/0 conn(0x7fca2c0c6040 legacy=0x7fca2c0acb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:24.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca2bfff700 1 -- 172.21.15.121:0/1281370839 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c14e760 con 0x7fca2c0ad270 2023-11-15T02:57:24.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca2bfff700 1 --2- 172.21.15.121:0/1281370839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca2c0ad270 0x7fca2c065680 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fca2c065190 tx=0x7fca24009840).ready entity=mon.0 client_cookie=8d01975d12ecb410 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:24.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca2a7fc700 1 -- 172.21.15.121:0/1281370839 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2400e070 con 0x7fca2c0ad270 2023-11-15T02:57:24.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca2a7fc700 1 -- 172.21.15.121:0/1281370839 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2400ba50 con 0x7fca2c0ad270 2023-11-15T02:57:24.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca2a7fc700 1 -- 172.21.15.121:0/1281370839 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2400bdd0 con 0x7fca2c0ad270 2023-11-15T02:57:24.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca30a8f700 1 -- 172.21.15.121:0/1281370839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca2c0ad270 msgr2=0x7fca2c065680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:24.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca30a8f700 1 --2- 172.21.15.121:0/1281370839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca2c0ad270 0x7fca2c065680 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fca2c065190 tx=0x7fca24009840).stop 2023-11-15T02:57:24.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca30a8f700 1 -- 172.21.15.121:0/1281370839 shutdown_connections 2023-11-15T02:57:24.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca30a8f700 1 --2- 172.21.15.121:0/1281370839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca2c0ad270 0x7fca2c065680 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca30a8f700 1 --2- 172.21.15.121:0/1281370839 >> v2:172.21.15.190:3300/0 conn(0x7fca2c14e380 0x7fca2c0c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca30a8f700 1 -- 172.21.15.121:0/1281370839 >> 172.21.15.121:0/1281370839 conn(0x7fca2c067a60 msgr2=0x7fca2c067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:24.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca30a8f700 1 -- 172.21.15.121:0/1281370839 shutdown_connections 2023-11-15T02:57:24.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.211+0000 7fca30a8f700 1 -- 172.21.15.121:0/1281370839 wait complete. 2023-11-15T02:57:24.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.215+0000 7fca30a8f700 1 Processor -- start 2023-11-15T02:57:24.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.215+0000 7fca30a8f700 1 -- start start 2023-11-15T02:57:24.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca30a8f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca2c0ad270 0x7fca2c1424a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca30a8f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca2c0c6040 0x7fca2c14c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca30a8f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca2c14e380 0x7fca2c1542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca30a8f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c142990 con 0x7fca2c0ad270 2023-11-15T02:57:24.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca30a8f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c0b0200 con 0x7fca2c0c6040 2023-11-15T02:57:24.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca30a8f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c0af5f0 con 0x7fca2c14e380 2023-11-15T02:57:24.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca2b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca2c0ad270 0x7fca2c1424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca2b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca2c0ad270 0x7fca2c1424a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39586/0 (socket says 172.21.15.121:39586) 2023-11-15T02:57:24.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca2b7fe700 1 -- 172.21.15.121:0/2453939031 learned_addr learned my addr 172.21.15.121:0/2453939031 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:24.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca2bfff700 1 --2- 172.21.15.121:0/2453939031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca2c14e380 0x7fca2c1542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca2affd700 1 --2- 172.21.15.121:0/2453939031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca2c0c6040 0x7fca2c14c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca2b7fe700 1 -- 172.21.15.121:0/2453939031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca2c14e380 msgr2=0x7fca2c1542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:24.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca2b7fe700 1 --2- 172.21.15.121:0/2453939031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca2c14e380 0x7fca2c1542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca2b7fe700 1 -- 172.21.15.121:0/2453939031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca2c0c6040 msgr2=0x7fca2c14c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:24.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca2b7fe700 1 --2- 172.21.15.121:0/2453939031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca2c0c6040 0x7fca2c14c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca2b7fe700 1 -- 172.21.15.121:0/2453939031 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca24009ae0 con 0x7fca2c0ad270 2023-11-15T02:57:24.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca2b7fe700 1 --2- 172.21.15.121:0/2453939031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca2c0ad270 0x7fca2c1424a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fca1c00d890 tx=0x7fca1c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:24.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca137fe700 1 -- 172.21.15.121:0/2453939031 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c011590 con 0x7fca2c0ad270 2023-11-15T02:57:24.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca137fe700 1 -- 172.21.15.121:0/2453939031 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c0116f0 con 0x7fca2c0ad270 2023-11-15T02:57:24.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca30a8f700 1 -- 172.21.15.121:0/2453939031 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2c0af8d0 con 0x7fca2c0ad270 2023-11-15T02:57:24.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca137fe700 1 -- 172.21.15.121:0/2453939031 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c011a70 con 0x7fca2c0ad270 2023-11-15T02:57:24.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.219+0000 7fca30a8f700 1 -- 172.21.15.121:0/2453939031 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca2c0afe90 con 0x7fca2c0ad270 2023-11-15T02:57:24.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.223+0000 7fca30a8f700 1 -- 172.21.15.121:0/2453939031 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2c06bcf0 con 0x7fca2c0ad270 2023-11-15T02:57:24.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.223+0000 7fca137fe700 1 -- 172.21.15.121:0/2453939031 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fca1c01c460 con 0x7fca2c0ad270 2023-11-15T02:57:24.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.223+0000 7fca137fe700 1 --2- 172.21.15.121:0/2453939031 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fca1805a510 0x7fca1805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.223+0000 7fca137fe700 1 -- 172.21.15.121:0/2453939031 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fca1c074600 con 0x7fca2c0ad270 2023-11-15T02:57:24.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.223+0000 7fca2affd700 1 --2- 172.21.15.121:0/2453939031 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fca1805a510 0x7fca1805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.227+0000 7fca2affd700 1 --2- 172.21.15.121:0/2453939031 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fca1805a510 0x7fca1805c9d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fca2c166f20 tx=0x7fca20009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:24.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.231+0000 7fca137fe700 1 -- 172.21.15.121:0/2453939031 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fca1c047cc0 con 0x7fca2c0ad270 2023-11-15T02:57:24.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.391+0000 7fca30a8f700 1 -- 172.21.15.121:0/2453939031 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fca2c0c7280 con 0x7fca2c0ad270 2023-11-15T02:57:24.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.391+0000 7fca137fe700 1 -- 172.21.15.121:0/2453939031 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v21) v1 ==== 110+0+10 (secure 0 0 0) 0x7fca1c04b2e0 con 0x7fca2c0ad270 2023-11-15T02:57:24.399 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T02:57:24.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.399+0000 7fca30a8f700 1 -- 172.21.15.121:0/2453939031 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fca1805a510 msgr2=0x7fca1805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:24.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.399+0000 7fca30a8f700 1 --2- 172.21.15.121:0/2453939031 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fca1805a510 0x7fca1805c9d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fca2c166f20 tx=0x7fca20009040).stop 2023-11-15T02:57:24.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.399+0000 7fca30a8f700 1 -- 172.21.15.121:0/2453939031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca2c0ad270 msgr2=0x7fca2c1424a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:24.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.399+0000 7fca30a8f700 1 --2- 172.21.15.121:0/2453939031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca2c0ad270 0x7fca2c1424a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fca1c00d890 tx=0x7fca1c008040).stop 2023-11-15T02:57:24.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.399+0000 7fca30a8f700 1 -- 172.21.15.121:0/2453939031 shutdown_connections 2023-11-15T02:57:24.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.399+0000 7fca30a8f700 1 --2- 172.21.15.121:0/2453939031 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fca1805a510 0x7fca1805c9d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.399+0000 7fca30a8f700 1 --2- 172.21.15.121:0/2453939031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca2c14e380 0x7fca2c1542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.399+0000 7fca30a8f700 1 --2- 172.21.15.121:0/2453939031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca2c0c6040 0x7fca2c14c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.399+0000 7fca30a8f700 1 --2- 172.21.15.121:0/2453939031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca2c0ad270 0x7fca2c1424a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.399+0000 7fca30a8f700 1 -- 172.21.15.121:0/2453939031 >> 172.21.15.121:0/2453939031 conn(0x7fca2c067a60 msgr2=0x7fca2c0acb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:24.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.399+0000 7fca30a8f700 1 -- 172.21.15.121:0/2453939031 shutdown_connections 2023-11-15T02:57:24.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.399+0000 7fca30a8f700 1 -- 172.21.15.121:0/2453939031 wait complete. 2023-11-15T02:57:24.423 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T02:57:24.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.587+0000 7fbbef48f700 1 Processor -- start 2023-11-15T02:57:24.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.587+0000 7fbbef48f700 1 -- start start 2023-11-15T02:57:24.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.591+0000 7fbbef48f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbbe80acd00 0x7fbbe80ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.591+0000 7fbbef48f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbbe80c5c70 0x7fbbe81536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.591+0000 7fbbef48f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe8065ea0 con 0x7fbbe8066330 2023-11-15T02:57:24.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.591+0000 7fbbef48f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe8066bb0 con 0x7fbbe80c5c70 2023-11-15T02:57:24.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.591+0000 7fbbef48f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe8065890 con 0x7fbbe80acd00 2023-11-15T02:57:24.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.591+0000 7fbbedc8c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbbe80c5c70 0x7fbbe81536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.591+0000 7fbbedc8c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbbe80c5c70 0x7fbbe81536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39598/0 (socket says 172.21.15.121:39598) 2023-11-15T02:57:24.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.591+0000 7fbbedc8c700 1 -- 172.21.15.121:0/1012184268 learned_addr learned my addr 172.21.15.121:0/1012184268 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:24.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.591+0000 7fbbee48d700 1 --2- 172.21.15.121:0/1012184268 >> v2:172.21.15.190:3300/0 conn(0x7fbbe80acd00 0x7fbbe80ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.591+0000 7fbbedc8c700 1 -- 172.21.15.121:0/1012184268 >> v2:172.21.15.190:3300/0 conn(0x7fbbe80acd00 msgr2=0x7fbbe80ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:24.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.591+0000 7fbbedc8c700 1 --2- 172.21.15.121:0/1012184268 >> v2:172.21.15.190:3300/0 conn(0x7fbbe80acd00 0x7fbbe80ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.591+0000 7fbbedc8c700 1 -- 172.21.15.121:0/1012184268 >> v1:172.21.15.190:6789/0 conn(0x7fbbe8066330 legacy=0x7fbbe8153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:24.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.591+0000 7fbbedc8c700 1 -- 172.21.15.121:0/1012184268 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe80c64a0 con 0x7fbbe80c5c70 2023-11-15T02:57:24.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.591+0000 7fbbedc8c700 1 --2- 172.21.15.121:0/1012184268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbe80c5c70 0x7fbbe81536c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fbbe806c280 tx=0x7fbbd80090f0).ready entity=mon.0 client_cookie=9f0955c0bf96fb77 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:24.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.591+0000 7fbbed48b700 1 -- 172.21.15.121:0/1012184268 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd800b600 con 0x7fbbe80c5c70 2023-11-15T02:57:24.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.591+0000 7fbbed48b700 1 -- 172.21.15.121:0/1012184268 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd800b760 con 0x7fbbe80c5c70 2023-11-15T02:57:24.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.591+0000 7fbbed48b700 1 -- 172.21.15.121:0/1012184268 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd800bae0 con 0x7fbbe80c5c70 2023-11-15T02:57:24.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.591+0000 7fbbef48f700 1 -- 172.21.15.121:0/1012184268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbe80c5c70 msgr2=0x7fbbe81536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:24.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.591+0000 7fbbef48f700 1 --2- 172.21.15.121:0/1012184268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbe80c5c70 0x7fbbe81536c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fbbe806c280 tx=0x7fbbd80090f0).stop 2023-11-15T02:57:24.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.595+0000 7fbbef48f700 1 -- 172.21.15.121:0/1012184268 shutdown_connections 2023-11-15T02:57:24.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.595+0000 7fbbef48f700 1 --2- 172.21.15.121:0/1012184268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbe80c5c70 0x7fbbe81536c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.595+0000 7fbbef48f700 1 --2- 172.21.15.121:0/1012184268 >> v2:172.21.15.190:3300/0 conn(0x7fbbe80acd00 0x7fbbe80ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.595+0000 7fbbef48f700 1 -- 172.21.15.121:0/1012184268 >> 172.21.15.121:0/1012184268 conn(0x7fbbe8063fc0 msgr2=0x7fbbe8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:24.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.595+0000 7fbbef48f700 1 -- 172.21.15.121:0/1012184268 shutdown_connections 2023-11-15T02:57:24.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.595+0000 7fbbef48f700 1 -- 172.21.15.121:0/1012184268 wait complete. 2023-11-15T02:57:24.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.595+0000 7fbbef48f700 1 Processor -- start 2023-11-15T02:57:24.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.595+0000 7fbbef48f700 1 -- start start 2023-11-15T02:57:24.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbef48f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbe8066330 0x7fbbe8156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbef48f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbe80acd00 0x7fbbe81643a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbef48f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbe80c5c70 0x7fbbe80caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbef48f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe80c1d80 con 0x7fbbe8066330 2023-11-15T02:57:24.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbef48f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe8150850 con 0x7fbbe80c5c70 2023-11-15T02:57:24.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbef48f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe80c41e0 con 0x7fbbe80acd00 2023-11-15T02:57:24.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbee48d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbe8066330 0x7fbbe8156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbee48d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbe8066330 0x7fbbe8156e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39608/0 (socket says 172.21.15.121:39608) 2023-11-15T02:57:24.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbee48d700 1 -- 172.21.15.121:0/2405213202 learned_addr learned my addr 172.21.15.121:0/2405213202 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:24.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbedc8c700 1 --2- 172.21.15.121:0/2405213202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbe80acd00 0x7fbbe81643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbeec8e700 1 --2- 172.21.15.121:0/2405213202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbe80c5c70 0x7fbbe80caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbee48d700 1 -- 172.21.15.121:0/2405213202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbe80acd00 msgr2=0x7fbbe81643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:24.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbee48d700 1 --2- 172.21.15.121:0/2405213202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbe80acd00 0x7fbbe81643a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbee48d700 1 -- 172.21.15.121:0/2405213202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbe80c5c70 msgr2=0x7fbbe80caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:24.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbee48d700 1 --2- 172.21.15.121:0/2405213202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbe80c5c70 0x7fbbe80caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbee48d700 1 -- 172.21.15.121:0/2405213202 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd8009370 con 0x7fbbe8066330 2023-11-15T02:57:24.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbee48d700 1 --2- 172.21.15.121:0/2405213202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbe8066330 0x7fbbe8156e50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fbbe400d860 tx=0x7fbbe4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:24.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbde7fc700 1 -- 172.21.15.121:0/2405213202 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4011590 con 0x7fbbe8066330 2023-11-15T02:57:24.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbde7fc700 1 -- 172.21.15.121:0/2405213202 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe40116f0 con 0x7fbbe8066330 2023-11-15T02:57:24.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbef48f700 1 -- 172.21.15.121:0/2405213202 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe80c44c0 con 0x7fbbe8066330 2023-11-15T02:57:24.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbde7fc700 1 -- 172.21.15.121:0/2405213202 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4011a70 con 0x7fbbe8066330 2023-11-15T02:57:24.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.599+0000 7fbbef48f700 1 -- 172.21.15.121:0/2405213202 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe81504e0 con 0x7fbbe8066330 2023-11-15T02:57:24.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.603+0000 7fbbef48f700 1 -- 172.21.15.121:0/2405213202 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe806bcf0 con 0x7fbbe8066330 2023-11-15T02:57:24.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.603+0000 7fbbde7fc700 1 -- 172.21.15.121:0/2405213202 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fbbe401c460 con 0x7fbbe8066330 2023-11-15T02:57:24.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.603+0000 7fbbde7fc700 1 --2- 172.21.15.121:0/2405213202 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbbcc05a510 0x7fbbcc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.603+0000 7fbbde7fc700 1 -- 172.21.15.121:0/2405213202 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fbbe4074740 con 0x7fbbe8066330 2023-11-15T02:57:24.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.603+0000 7fbbedc8c700 1 --2- 172.21.15.121:0/2405213202 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbbcc05a510 0x7fbbcc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.607+0000 7fbbedc8c700 1 --2- 172.21.15.121:0/2405213202 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbbcc05a510 0x7fbbcc05c9d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fbbd8009130 tx=0x7fbbd8017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:24.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.611+0000 7fbbde7fc700 1 -- 172.21.15.121:0/2405213202 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbbe4047e80 con 0x7fbbe8066330 2023-11-15T02:57:24.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.771+0000 7fbbef48f700 1 -- 172.21.15.121:0/2405213202 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbbe80c61c0 con 0x7fbbe8066330 2023-11-15T02:57:24.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.771+0000 7fbbde7fc700 1 -- 172.21.15.121:0/2405213202 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbbe404b4a0 con 0x7fbbe8066330 2023-11-15T02:57:24.777 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T02:57:24.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.775+0000 7fbbef48f700 1 -- 172.21.15.121:0/2405213202 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbbcc05a510 msgr2=0x7fbbcc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:24.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.775+0000 7fbbef48f700 1 --2- 172.21.15.121:0/2405213202 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbbcc05a510 0x7fbbcc05c9d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fbbd8009130 tx=0x7fbbd8017000).stop 2023-11-15T02:57:24.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.775+0000 7fbbef48f700 1 -- 172.21.15.121:0/2405213202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbe8066330 msgr2=0x7fbbe8156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:24.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.775+0000 7fbbef48f700 1 --2- 172.21.15.121:0/2405213202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbe8066330 0x7fbbe8156e50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fbbe400d860 tx=0x7fbbe4008040).stop 2023-11-15T02:57:24.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.775+0000 7fbbef48f700 1 -- 172.21.15.121:0/2405213202 shutdown_connections 2023-11-15T02:57:24.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.775+0000 7fbbef48f700 1 --2- 172.21.15.121:0/2405213202 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbbcc05a510 0x7fbbcc05c9d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.775+0000 7fbbef48f700 1 --2- 172.21.15.121:0/2405213202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbe80c5c70 0x7fbbe80caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.775+0000 7fbbef48f700 1 --2- 172.21.15.121:0/2405213202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbe80acd00 0x7fbbe81643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.775+0000 7fbbef48f700 1 --2- 172.21.15.121:0/2405213202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbe8066330 0x7fbbe8156e50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.775+0000 7fbbef48f700 1 -- 172.21.15.121:0/2405213202 >> 172.21.15.121:0/2405213202 conn(0x7fbbe8063fc0 msgr2=0x7fbbe80c8890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:24.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.775+0000 7fbbef48f700 1 -- 172.21.15.121:0/2405213202 shutdown_connections 2023-11-15T02:57:24.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.775+0000 7fbbef48f700 1 -- 172.21.15.121:0/2405213202 wait complete. 2023-11-15T02:57:24.802 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-15T02:57:24.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.959+0000 7f32423b9700 1 Processor -- start 2023-11-15T02:57:24.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.959+0000 7f32423b9700 1 -- start start 2023-11-15T02:57:24.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f32423b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f323c0c5c70 0x7f323c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f32423b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323c065ea0 con 0x7f323c0acd00 2023-11-15T02:57:24.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f32423b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323c066bb0 con 0x7f323c066330 2023-11-15T02:57:24.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f32423b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323c065890 con 0x7f323c0c5c70 2023-11-15T02:57:24.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f32413b7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f323c0acd00 0x7f323c1536c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39130/0 (socket says 172.21.15.121:39130) 2023-11-15T02:57:24.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f32413b7700 1 -- 172.21.15.121:0/464742758 learned_addr learned my addr 172.21.15.121:0/464742758 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:24.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3240bb6700 1 --2- 172.21.15.121:0/464742758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f323c0c5c70 0x7f323c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3233fff700 1 -- 172.21.15.121:0/464742758 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3662672250 0 0) 0x7f323c065ea0 con 0x7f323c0acd00 2023-11-15T02:57:24.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3233fff700 1 -- 172.21.15.121:0/464742758 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32280033f0 con 0x7f323c0acd00 2023-11-15T02:57:24.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3233fff700 1 -- 172.21.15.121:0/464742758 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3072720328 0 0) 0x7f32280033f0 con 0x7f323c0acd00 2023-11-15T02:57:24.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3233fff700 1 -- 172.21.15.121:0/464742758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f323c0c5c70 msgr2=0x7f323c153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:24.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3233fff700 1 --2- 172.21.15.121:0/464742758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f323c0c5c70 0x7f323c153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3233fff700 1 -- 172.21.15.121:0/464742758 >> v1:172.21.15.190:6789/0 conn(0x7f323c066330 legacy=0x7f323c1577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:24.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3233fff700 1 -- 172.21.15.121:0/464742758 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c0c64a0 con 0x7f323c0acd00 2023-11-15T02:57:24.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3233fff700 1 -- 172.21.15.121:0/464742758 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f32380015f0 con 0x7f323c0acd00 2023-11-15T02:57:24.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3233fff700 1 -- 172.21.15.121:0/464742758 >> v1:172.21.15.121:6789/0 conn(0x7f323c0acd00 legacy=0x7f323c1536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:24.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3233fff700 1 --2- 172.21.15.121:0/464742758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f323c0c5c70 0x7f3228003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3233fff700 1 --2- 172.21.15.121:0/464742758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3228003e00 0x7f3228004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3233fff700 1 --2- 172.21.15.121:0/464742758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3228004b80 0x7f3228005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3233fff700 1 -- 172.21.15.121:0/464742758 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c065890 con 0x7f3228003e00 2023-11-15T02:57:24.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3233fff700 1 -- 172.21.15.121:0/464742758 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322800b8e0 con 0x7f3228004b80 2023-11-15T02:57:24.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3233fff700 1 -- 172.21.15.121:0/464742758 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322800baa0 con 0x7f323c0c5c70 2023-11-15T02:57:24.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f32413b7700 1 --2- 172.21.15.121:0/464742758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f323c0c5c70 0x7f3228003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3240bb6700 1 --2- 172.21.15.121:0/464742758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3228003e00 0x7f3228004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3241bb8700 1 --2- 172.21.15.121:0/464742758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3228004b80 0x7f3228005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3240bb6700 1 -- 172.21.15.121:0/464742758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f323c0c5c70 msgr2=0x7f3228003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:24.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3240bb6700 1 --2- 172.21.15.121:0/464742758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f323c0c5c70 0x7f3228003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3240bb6700 1 -- 172.21.15.121:0/464742758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3228004b80 msgr2=0x7f3228005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:24.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3240bb6700 1 --2- 172.21.15.121:0/464742758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3228004b80 0x7f3228005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.963+0000 7f3240bb6700 1 -- 172.21.15.121:0/464742758 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32280033f0 con 0x7f3228003e00 2023-11-15T02:57:24.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.967+0000 7f3240bb6700 1 --2- 172.21.15.121:0/464742758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3228003e00 0x7f3228004690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f322c009a80 tx=0x7f322c009060).ready entity=mon.0 client_cookie=a35b3af6e790b5b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:24.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.967+0000 7f3233fff700 1 -- 172.21.15.121:0/464742758 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c009430 con 0x7f3228003e00 2023-11-15T02:57:24.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.967+0000 7f3233fff700 1 -- 172.21.15.121:0/464742758 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322c00e640 con 0x7f3228003e00 2023-11-15T02:57:24.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.967+0000 7f3233fff700 1 -- 172.21.15.121:0/464742758 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c00e9c0 con 0x7f3228003e00 2023-11-15T02:57:24.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.967+0000 7f32423b9700 1 -- 172.21.15.121:0/464742758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3228003e00 msgr2=0x7f3228004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:24.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.967+0000 7f32423b9700 1 --2- 172.21.15.121:0/464742758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3228003e00 0x7f3228004690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f322c009a80 tx=0x7f322c009060).stop 2023-11-15T02:57:24.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.967+0000 7f3241bb8700 1 -- 172.21.15.121:0/464742758 reap_dead start 2023-11-15T02:57:24.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.967+0000 7f32423b9700 1 -- 172.21.15.121:0/464742758 shutdown_connections 2023-11-15T02:57:24.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.967+0000 7f32423b9700 1 -- 172.21.15.121:0/464742758 >> 172.21.15.121:0/464742758 conn(0x7f323c063fc0 msgr2=0x7f323c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:24.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.967+0000 7f32423b9700 1 -- 172.21.15.121:0/464742758 shutdown_connections 2023-11-15T02:57:24.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.967+0000 7f32423b9700 1 -- 172.21.15.121:0/464742758 wait complete. 2023-11-15T02:57:24.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f32423b9700 1 Processor -- start 2023-11-15T02:57:24.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f32423b9700 1 -- start start 2023-11-15T02:57:24.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f32423b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3228003e00 0x7f323c0af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f32423b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f323c14f730 0x7f323c14fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f32423b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f323c166ad0 0x7f323c166f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f32423b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c0aff20 con 0x7f323c166ad0 2023-11-15T02:57:24.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f32423b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c0c1a30 con 0x7f323c14f730 2023-11-15T02:57:24.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f32423b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323c0c1d10 con 0x7f3228003e00 2023-11-15T02:57:24.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f3241bb8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f323c166ad0 0x7f323c166f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f3241bb8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f323c166ad0 0x7f323c166f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37408/0 (socket says 172.21.15.121:37408) 2023-11-15T02:57:24.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f3241bb8700 1 -- 172.21.15.121:0/1376092568 learned_addr learned my addr 172.21.15.121:0/1376092568 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:24.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f3240bb6700 1 --2- 172.21.15.121:0/1376092568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f323c14f730 0x7f323c14fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f32413b7700 1 --2- 172.21.15.121:0/1376092568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3228003e00 0x7f323c0af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f3241bb8700 1 -- 172.21.15.121:0/1376092568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3228003e00 msgr2=0x7f323c0af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:24.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f3241bb8700 1 --2- 172.21.15.121:0/1376092568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3228003e00 0x7f323c0af350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f3241bb8700 1 -- 172.21.15.121:0/1376092568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f323c14f730 msgr2=0x7f323c14fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:24.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f3241bb8700 1 --2- 172.21.15.121:0/1376092568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f323c14f730 0x7f323c14fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:24.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f3241bb8700 1 -- 172.21.15.121:0/1376092568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c0c1ff0 con 0x7f323c166ad0 2023-11-15T02:57:24.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f32413b7700 1 --2- 172.21.15.121:0/1376092568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3228003e00 0x7f323c0af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:24.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f3241bb8700 1 --2- 172.21.15.121:0/1376092568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f323c166ad0 0x7f323c166f50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f323c06c2b0 tx=0x7f3234007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:24.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f32317fa700 1 -- 172.21.15.121:0/1376092568 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3234004100 con 0x7f323c166ad0 2023-11-15T02:57:24.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f32317fa700 1 -- 172.21.15.121:0/1376092568 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3234012590 con 0x7f323c166ad0 2023-11-15T02:57:24.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f32423b9700 1 -- 172.21.15.121:0/1376092568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323c0c22d0 con 0x7f323c166ad0 2023-11-15T02:57:24.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f32317fa700 1 -- 172.21.15.121:0/1376092568 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3234012910 con 0x7f323c166ad0 2023-11-15T02:57:24.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.971+0000 7f32423b9700 1 -- 172.21.15.121:0/1376092568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323c0c2890 con 0x7f323c166ad0 2023-11-15T02:57:24.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.975+0000 7f32423b9700 1 -- 172.21.15.121:0/1376092568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323c06bcf0 con 0x7f323c166ad0 2023-11-15T02:57:24.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.975+0000 7f32317fa700 1 -- 172.21.15.121:0/1376092568 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3234012d00 con 0x7f323c166ad0 2023-11-15T02:57:24.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.979+0000 7f32317fa700 1 --2- 172.21.15.121:0/1376092568 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f322005a510 0x7f322005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:24.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.979+0000 7f32317fa700 1 -- 172.21.15.121:0/1376092568 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3234074990 con 0x7f323c166ad0 2023-11-15T02:57:24.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.979+0000 7f32413b7700 1 --2- 172.21.15.121:0/1376092568 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f322005a510 0x7f322005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:24.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.979+0000 7f32413b7700 1 --2- 172.21.15.121:0/1376092568 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f322005a510 0x7f322005c9d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f3238000e10 tx=0x7f3238009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:24.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:24.983+0000 7f32317fa700 1 -- 172.21.15.121:0/1376092568 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f32340480b0 con 0x7f323c166ad0 2023-11-15T02:57:25.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.143+0000 7f32423b9700 1 -- 172.21.15.121:0/1376092568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f323c0c8980 con 0x7f323c166ad0 2023-11-15T02:57:25.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.143+0000 7f32317fa700 1 -- 172.21.15.121:0/1376092568 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f323404b6d0 con 0x7f323c166ad0 2023-11-15T02:57:25.151 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T02:57:25.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.147+0000 7f32423b9700 1 -- 172.21.15.121:0/1376092568 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f322005a510 msgr2=0x7f322005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:25.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.147+0000 7f32423b9700 1 --2- 172.21.15.121:0/1376092568 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f322005a510 0x7f322005c9d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f3238000e10 tx=0x7f3238009040).stop 2023-11-15T02:57:25.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.147+0000 7f32423b9700 1 -- 172.21.15.121:0/1376092568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f323c166ad0 msgr2=0x7f323c166f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:25.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.147+0000 7f32423b9700 1 --2- 172.21.15.121:0/1376092568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f323c166ad0 0x7f323c166f50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f323c06c2b0 tx=0x7f3234007f10).stop 2023-11-15T02:57:25.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.147+0000 7f32423b9700 1 -- 172.21.15.121:0/1376092568 shutdown_connections 2023-11-15T02:57:25.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.147+0000 7f32423b9700 1 --2- 172.21.15.121:0/1376092568 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f322005a510 0x7f322005c9d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:25.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.147+0000 7f32423b9700 1 --2- 172.21.15.121:0/1376092568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f323c166ad0 0x7f323c166f50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:25.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.147+0000 7f32423b9700 1 --2- 172.21.15.121:0/1376092568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f323c14f730 0x7f323c14fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:25.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.147+0000 7f32423b9700 1 --2- 172.21.15.121:0/1376092568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3228003e00 0x7f323c0af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:25.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.147+0000 7f32423b9700 1 -- 172.21.15.121:0/1376092568 >> 172.21.15.121:0/1376092568 conn(0x7f323c063fc0 msgr2=0x7f323c152750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:25.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.147+0000 7f32423b9700 1 -- 172.21.15.121:0/1376092568 shutdown_connections 2023-11-15T02:57:25.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.147+0000 7f32423b9700 1 -- 172.21.15.121:0/1376092568 wait complete. 2023-11-15T02:57:25.173 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-15T02:57:25.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.331+0000 7f5f21a79700 1 Processor -- start 2023-11-15T02:57:25.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.331+0000 7f5f21a79700 1 -- start start 2023-11-15T02:57:25.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.335+0000 7f5f21a79700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5f1c14da00 0x7f5f1c151130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:25.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.335+0000 7f5f21a79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1c066080 0x7f5f1c0664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:25.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.335+0000 7f5f21a79700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1c0c2040 con 0x7f5f1c0c4bc0 2023-11-15T02:57:25.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.335+0000 7f5f21a79700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1c0c2300 con 0x7f5f1c14da00 2023-11-15T02:57:25.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.335+0000 7f5f21a79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0669e0 con 0x7f5f1c066080 2023-11-15T02:57:25.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.335+0000 7f5f20a77700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5f1c14da00 0x7f5f1c151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:25.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.335+0000 7f5f20a77700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5f1c14da00 0x7f5f1c151130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:56434/0 (socket says 172.21.15.121:56434) 2023-11-15T02:57:25.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.335+0000 7f5f20a77700 1 -- 172.21.15.121:0/3729316088 learned_addr learned my addr 172.21.15.121:0/3729316088 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:25.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.335+0000 7f5f21278700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1c066080 0x7f5f1c0664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:25.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.335+0000 7f5f1bfff700 1 --1- 172.21.15.121:0/3729316088 >> v1:172.21.15.190:6789/0 conn(0x7f5f1c0c4bc0 0x7f5f1c067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:33678/0 (socket says 172.21.15.121:33678) 2023-11-15T02:57:25.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.335+0000 7f5f20a77700 1 -- 172.21.15.121:0/3729316088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1c066080 msgr2=0x7f5f1c0664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:25.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.335+0000 7f5f1b7fe700 1 -- 172.21.15.121:0/3729316088 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101493991 0 0) 0x7f5f1c0c2040 con 0x7f5f1c0c4bc0 2023-11-15T02:57:25.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.335+0000 7f5f20a77700 1 --2- 172.21.15.121:0/3729316088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1c066080 0x7f5f1c0664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:25.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.335+0000 7f5f20a77700 1 -- 172.21.15.121:0/3729316088 >> v1:172.21.15.190:6789/0 conn(0x7f5f1c0c4bc0 legacy=0x7f5f1c067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:25.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.335+0000 7f5f20a77700 1 -- 172.21.15.121:0/3729316088 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c068b30 con 0x7f5f1c14da00 2023-11-15T02:57:25.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.339+0000 7f5f20a77700 1 --2- 172.21.15.121:0/3729316088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f1c14da00 0x7f5f1c151130 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f5f10006fd0 tx=0x7f5f1000a2b0).ready entity=mon.1 client_cookie=186ce769c9e67d59 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:25.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.339+0000 7f5f1b7fe700 1 -- 172.21.15.121:0/3729316088 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1000ae30 con 0x7f5f1c14da00 2023-11-15T02:57:25.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.339+0000 7f5f1b7fe700 1 -- 172.21.15.121:0/3729316088 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1000c970 con 0x7f5f1c14da00 2023-11-15T02:57:25.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.339+0000 7f5f1b7fe700 1 -- 172.21.15.121:0/3729316088 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1000ccf0 con 0x7f5f1c14da00 2023-11-15T02:57:25.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.339+0000 7f5f21a79700 1 -- 172.21.15.121:0/3729316088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f1c14da00 msgr2=0x7f5f1c151130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:25.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.339+0000 7f5f21a79700 1 --2- 172.21.15.121:0/3729316088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f1c14da00 0x7f5f1c151130 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f5f10006fd0 tx=0x7f5f1000a2b0).stop 2023-11-15T02:57:25.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.339+0000 7f5f21a79700 1 -- 172.21.15.121:0/3729316088 shutdown_connections 2023-11-15T02:57:25.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.339+0000 7f5f21a79700 1 --2- 172.21.15.121:0/3729316088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1c066080 0x7f5f1c0664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:25.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.339+0000 7f5f21a79700 1 --2- 172.21.15.121:0/3729316088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f1c14da00 0x7f5f1c151130 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:25.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.339+0000 7f5f21a79700 1 -- 172.21.15.121:0/3729316088 >> 172.21.15.121:0/3729316088 conn(0x7f5f1c062c00 msgr2=0x7f5f1c0c6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:25.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.339+0000 7f5f21a79700 1 -- 172.21.15.121:0/3729316088 shutdown_connections 2023-11-15T02:57:25.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.339+0000 7f5f21a79700 1 -- 172.21.15.121:0/3729316088 wait complete. 2023-11-15T02:57:25.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.343+0000 7f5f21a79700 1 Processor -- start 2023-11-15T02:57:25.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.343+0000 7f5f21a79700 1 -- start start 2023-11-15T02:57:25.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.343+0000 7f5f21a79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f1c066080 0x7f5f1c1362c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:25.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.343+0000 7f5f21a79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f1c0c4bc0 0x7f5f1c14aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:25.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.343+0000 7f5f21a79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1c14da00 0x7f5f1c0a9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:25.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.343+0000 7f5f21a79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c14c080 con 0x7f5f1c0c4bc0 2023-11-15T02:57:25.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.343+0000 7f5f21a79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c14b470 con 0x7f5f1c066080 2023-11-15T02:57:25.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.343+0000 7f5f21a79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c14b750 con 0x7f5f1c14da00 2023-11-15T02:57:25.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.343+0000 7f5f1bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f1c0c4bc0 0x7f5f1c14aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:25.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.343+0000 7f5f1bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f1c0c4bc0 0x7f5f1c14aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37424/0 (socket says 172.21.15.121:37424) 2023-11-15T02:57:25.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.343+0000 7f5f1bfff700 1 -- 172.21.15.121:0/4093707396 learned_addr learned my addr 172.21.15.121:0/4093707396 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:25.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.343+0000 7f5f21278700 1 --2- 172.21.15.121:0/4093707396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1c14da00 0x7f5f1c0a9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:25.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.343+0000 7f5f20a77700 1 --2- 172.21.15.121:0/4093707396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f1c066080 0x7f5f1c1362c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.343+0000 7f5f1bfff700 1 -- 172.21.15.121:0/4093707396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1c14da00 msgr2=0x7f5f1c0a9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.343+0000 7f5f1bfff700 1 --2- 172.21.15.121:0/4093707396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1c14da00 0x7f5f1c0a9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.343+0000 7f5f1bfff700 1 -- 172.21.15.121:0/4093707396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f1c066080 msgr2=0x7f5f1c1362c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.343+0000 7f5f1bfff700 1 --2- 172.21.15.121:0/4093707396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f1c066080 0x7f5f1c1362c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:25.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.343+0000 7f5f1bfff700 1 -- 172.21.15.121:0/4093707396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1000a510 con 0x7f5f1c0c4bc0 2023-11-15T02:57:25.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.343+0000 7f5f1bfff700 1 --2- 172.21.15.121:0/4093707396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f1c0c4bc0 0x7f5f1c14aa20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f5f0800c490 tx=0x7f5f08007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:25.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.347+0000 7f5f18ff9700 1 -- 172.21.15.121:0/4093707396 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0800ce00 con 0x7f5f1c0c4bc0 2023-11-15T02:57:25.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.347+0000 7f5f18ff9700 1 -- 172.21.15.121:0/4093707396 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f08007170 con 0x7f5f1c0c4bc0 2023-11-15T02:57:25.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.347+0000 7f5f21a79700 1 -- 172.21.15.121:0/4093707396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c14ba30 con 0x7f5f1c0c4bc0 2023-11-15T02:57:25.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.347+0000 7f5f18ff9700 1 -- 172.21.15.121:0/4093707396 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08007510 con 0x7f5f1c0c4bc0 2023-11-15T02:57:25.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.347+0000 7f5f21a79700 1 -- 172.21.15.121:0/4093707396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c1636f0 con 0x7f5f1c0c4bc0 2023-11-15T02:57:25.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.347+0000 7f5f21a79700 1 -- 172.21.15.121:0/4093707396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c06bcf0 con 0x7f5f1c0c4bc0 2023-11-15T02:57:25.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.347+0000 7f5f18ff9700 1 -- 172.21.15.121:0/4093707396 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5f08004ad0 con 0x7f5f1c0c4bc0 2023-11-15T02:57:25.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.351+0000 7f5f18ff9700 1 --2- 172.21.15.121:0/4093707396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5f0005a510 0x7f5f0005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:25.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.351+0000 7f5f18ff9700 1 -- 172.21.15.121:0/4093707396 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5f08073f40 con 0x7f5f1c0c4bc0 2023-11-15T02:57:25.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.351+0000 7f5f20a77700 1 --2- 172.21.15.121:0/4093707396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5f0005a510 0x7f5f0005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:25.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.351+0000 7f5f20a77700 1 --2- 172.21.15.121:0/4093707396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5f0005a510 0x7f5f0005c9d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f5f10008010 tx=0x7f5f10019000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:25.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.359+0000 7f5f18ff9700 1 -- 172.21.15.121:0/4093707396 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5f08047720 con 0x7f5f1c0c4bc0 2023-11-15T02:57:25.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.523+0000 7f5f21a79700 1 -- 172.21.15.121:0/4093707396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5f1c059ad0 con 0x7f5f1c0c4bc0 2023-11-15T02:57:25.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.523+0000 7f5f18ff9700 1 -- 172.21.15.121:0/4093707396 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5f0804ad40 con 0x7f5f1c0c4bc0 2023-11-15T02:57:25.530 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T02:57:25.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.527+0000 7f5f21a79700 1 -- 172.21.15.121:0/4093707396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5f0005a510 msgr2=0x7f5f0005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:25.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.527+0000 7f5f21a79700 1 --2- 172.21.15.121:0/4093707396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5f0005a510 0x7f5f0005c9d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f5f10008010 tx=0x7f5f10019000).stop 2023-11-15T02:57:25.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.527+0000 7f5f21a79700 1 -- 172.21.15.121:0/4093707396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f1c0c4bc0 msgr2=0x7f5f1c14aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:25.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.527+0000 7f5f21a79700 1 --2- 172.21.15.121:0/4093707396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f1c0c4bc0 0x7f5f1c14aa20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f5f0800c490 tx=0x7f5f08007c10).stop 2023-11-15T02:57:25.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.527+0000 7f5f21a79700 1 -- 172.21.15.121:0/4093707396 shutdown_connections 2023-11-15T02:57:25.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.527+0000 7f5f21a79700 1 --2- 172.21.15.121:0/4093707396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5f0005a510 0x7f5f0005c9d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:25.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.527+0000 7f5f21a79700 1 --2- 172.21.15.121:0/4093707396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1c14da00 0x7f5f1c0a9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:25.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.527+0000 7f5f21a79700 1 --2- 172.21.15.121:0/4093707396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f1c0c4bc0 0x7f5f1c14aa20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:25.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.527+0000 7f5f21a79700 1 --2- 172.21.15.121:0/4093707396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f1c066080 0x7f5f1c1362c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:25.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.527+0000 7f5f21a79700 1 -- 172.21.15.121:0/4093707396 >> 172.21.15.121:0/4093707396 conn(0x7f5f1c062c00 msgr2=0x7f5f1c0c5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:25.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.527+0000 7f5f21a79700 1 -- 172.21.15.121:0/4093707396 shutdown_connections 2023-11-15T02:57:25.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.527+0000 7f5f21a79700 1 -- 172.21.15.121:0/4093707396 wait complete. 2023-11-15T02:57:25.568 INFO:tasks.cephfs_test_runner:Starting test: test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) 2023-11-15T02:57:25.569 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry' 2023-11-15T02:57:25.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.727+0000 7fa6ff307700 1 Processor -- start 2023-11-15T02:57:25.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.727+0000 7fa6ff307700 1 -- start start 2023-11-15T02:57:25.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.727+0000 7fa6ff307700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f80c37f0 0x7fa6f80a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:25.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.727+0000 7fa6ff307700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f80657f0 con 0x7fa6f80a9830 2023-11-15T02:57:25.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.727+0000 7fa6ff307700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f8065ad0 con 0x7fa6f8151870 2023-11-15T02:57:25.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.727+0000 7fa6ff307700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80aa260 con 0x7fa6f80c37f0 2023-11-15T02:57:25.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.731+0000 7fa6fdb04700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa6f80a9830 0x7fa6f80c30e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39142/0 (socket says 172.21.15.121:39142) 2023-11-15T02:57:25.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.731+0000 7fa6fdb04700 1 -- 172.21.15.121:0/3752191485 learned_addr learned my addr 172.21.15.121:0/3752191485 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:25.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.731+0000 7fa6feb06700 1 --2- 172.21.15.121:0/3752191485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f80c37f0 0x7fa6f80a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:25.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.731+0000 7fa6fd303700 1 -- 172.21.15.121:0/3752191485 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 483292742 0 0) 0x7fa6f80657f0 con 0x7fa6f80a9830 2023-11-15T02:57:25.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.731+0000 7fa6fd303700 1 -- 172.21.15.121:0/3752191485 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e40033f0 con 0x7fa6f80a9830 2023-11-15T02:57:25.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.731+0000 7fa6fd303700 1 -- 172.21.15.121:0/3752191485 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382828303 0 0) 0x7fa6f8065ad0 con 0x7fa6f8151870 2023-11-15T02:57:25.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.731+0000 7fa6fd303700 1 -- 172.21.15.121:0/3752191485 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f80657f0 con 0x7fa6f8151870 2023-11-15T02:57:25.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.731+0000 7fa6feb06700 1 -- 172.21.15.121:0/3752191485 >> v1:172.21.15.190:6789/0 conn(0x7fa6f8151870 legacy=0x7fa6f8151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:25.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.731+0000 7fa6feb06700 1 -- 172.21.15.121:0/3752191485 >> v1:172.21.15.121:6789/0 conn(0x7fa6f80a9830 legacy=0x7fa6f80c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:25.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.731+0000 7fa6feb06700 1 -- 172.21.15.121:0/3752191485 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f80aa540 con 0x7fa6f80c37f0 2023-11-15T02:57:25.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.731+0000 7fa6feb06700 1 --2- 172.21.15.121:0/3752191485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f80c37f0 0x7fa6f80a9c70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fa6f0009e90 tx=0x7fa6f0009780).ready entity=mon.2 client_cookie=1c434ffd1d64366c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:25.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.731+0000 7fa6fd303700 1 -- 172.21.15.121:0/3752191485 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000e070 con 0x7fa6f80c37f0 2023-11-15T02:57:25.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.731+0000 7fa6fd303700 1 -- 172.21.15.121:0/3752191485 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f000bab0 con 0x7fa6f80c37f0 2023-11-15T02:57:25.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.731+0000 7fa6fd303700 1 -- 172.21.15.121:0/3752191485 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000be30 con 0x7fa6f80c37f0 2023-11-15T02:57:25.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.731+0000 7fa6ff307700 1 -- 172.21.15.121:0/3752191485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f80c37f0 msgr2=0x7fa6f80a9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:25.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.731+0000 7fa6ff307700 1 --2- 172.21.15.121:0/3752191485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f80c37f0 0x7fa6f80a9c70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fa6f0009e90 tx=0x7fa6f0009780).stop 2023-11-15T02:57:25.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.731+0000 7fa6ff307700 1 -- 172.21.15.121:0/3752191485 shutdown_connections 2023-11-15T02:57:25.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.731+0000 7fa6ff307700 1 --2- 172.21.15.121:0/3752191485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f80c37f0 0x7fa6f80a9c70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:25.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.735+0000 7fa6ff307700 1 -- 172.21.15.121:0/3752191485 >> 172.21.15.121:0/3752191485 conn(0x7fa6f8062190 msgr2=0x7fa6f8062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:25.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.735+0000 7fa6ff307700 1 -- 172.21.15.121:0/3752191485 shutdown_connections 2023-11-15T02:57:25.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.735+0000 7fa6ff307700 1 -- 172.21.15.121:0/3752191485 wait complete. 2023-11-15T02:57:25.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.735+0000 7fa6ff307700 1 Processor -- start 2023-11-15T02:57:25.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.735+0000 7fa6ff307700 1 -- start start 2023-11-15T02:57:25.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6ff307700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f80a9830 0x7fa6f80c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:25.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6ff307700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6f80c37f0 0x7fa6f813e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:25.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6ff307700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f8151870 0x7fa6f814ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:25.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6ff307700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f8157280 con 0x7fa6f80c37f0 2023-11-15T02:57:25.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6ff307700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f814a8c0 con 0x7fa6f80a9830 2023-11-15T02:57:25.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6ff307700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80c74d0 con 0x7fa6f8151870 2023-11-15T02:57:25.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6fdb04700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6f80c37f0 0x7fa6f813e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:25.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6fdb04700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6f80c37f0 0x7fa6f813e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37436/0 (socket says 172.21.15.121:37436) 2023-11-15T02:57:25.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6fdb04700 1 -- 172.21.15.121:0/1346054256 learned_addr learned my addr 172.21.15.121:0/1346054256 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:25.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6fe305700 1 --2- 172.21.15.121:0/1346054256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f80a9830 0x7fa6f80c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:25.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6feb06700 1 --2- 172.21.15.121:0/1346054256 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f8151870 0x7fa6f814ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:25.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6fdb04700 1 -- 172.21.15.121:0/1346054256 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f8151870 msgr2=0x7fa6f814ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:25.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6fdb04700 1 --2- 172.21.15.121:0/1346054256 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f8151870 0x7fa6f814ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:25.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6fdb04700 1 -- 172.21.15.121:0/1346054256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f80a9830 msgr2=0x7fa6f80c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:25.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6fdb04700 1 --2- 172.21.15.121:0/1346054256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f80a9830 0x7fa6f80c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:25.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6fdb04700 1 -- 172.21.15.121:0/1346054256 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f0009a20 con 0x7fa6f80c37f0 2023-11-15T02:57:25.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6fdb04700 1 --2- 172.21.15.121:0/1346054256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6f80c37f0 0x7fa6f813e0e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fa6e800c560 tx=0x7fa6e8007d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:25.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6ee7fc700 1 -- 172.21.15.121:0/1346054256 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e80072e0 con 0x7fa6f80c37f0 2023-11-15T02:57:25.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6ee7fc700 1 -- 172.21.15.121:0/1346054256 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e8007440 con 0x7fa6f80c37f0 2023-11-15T02:57:25.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6ff307700 1 -- 172.21.15.121:0/1346054256 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f8156670 con 0x7fa6f80c37f0 2023-11-15T02:57:25.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6ee7fc700 1 -- 172.21.15.121:0/1346054256 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e80057a0 con 0x7fa6f80c37f0 2023-11-15T02:57:25.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.739+0000 7fa6ff307700 1 -- 172.21.15.121:0/1346054256 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f8156c30 con 0x7fa6f80c37f0 2023-11-15T02:57:25.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.743+0000 7fa6ff307700 1 -- 172.21.15.121:0/1346054256 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f806bcf0 con 0x7fa6f80c37f0 2023-11-15T02:57:25.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.743+0000 7fa6ee7fc700 1 -- 172.21.15.121:0/1346054256 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa6e8005b90 con 0x7fa6f80c37f0 2023-11-15T02:57:25.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.743+0000 7fa6ee7fc700 1 --2- 172.21.15.121:0/1346054256 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa6dc05a510 0x7fa6dc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:25.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.743+0000 7fa6ee7fc700 1 -- 172.21.15.121:0/1346054256 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fa6e8074100 con 0x7fa6f80c37f0 2023-11-15T02:57:25.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.743+0000 7fa6fe305700 1 --2- 172.21.15.121:0/1346054256 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa6dc05a510 0x7fa6dc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:25.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.747+0000 7fa6fe305700 1 --2- 172.21.15.121:0/1346054256 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa6dc05a510 0x7fa6dc05c9d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fa6f8157a10 tx=0x7fa6f4009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:25.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.751+0000 7fa6ee7fc700 1 -- 172.21.15.121:0/1346054256 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa6e8047820 con 0x7fa6f80c37f0 2023-11-15T02:57:25.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:25.963+0000 7fa6ff307700 1 -- 172.21.15.121:0/1346054256 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]} v 0) v1 -- 0x7fa6f80aa820 con 0x7fa6f80c37f0 2023-11-15T02:57:26.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:26.915+0000 7fa6ee7fc700 1 -- 172.21.15.121:0/1346054256 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]}]=0 v52) v1 ==== 137+0+0 (secure 0 0 0) 0x7fa6e804ae40 con 0x7fa6f80c37f0 2023-11-15T02:57:26.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:26.919+0000 7fa6ff307700 1 -- 172.21.15.121:0/1346054256 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa6dc05a510 msgr2=0x7fa6dc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:26.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:26.919+0000 7fa6ff307700 1 --2- 172.21.15.121:0/1346054256 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa6dc05a510 0x7fa6dc05c9d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fa6f8157a10 tx=0x7fa6f4009040).stop 2023-11-15T02:57:26.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:26.919+0000 7fa6ff307700 1 -- 172.21.15.121:0/1346054256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6f80c37f0 msgr2=0x7fa6f813e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:26.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:26.919+0000 7fa6ff307700 1 --2- 172.21.15.121:0/1346054256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6f80c37f0 0x7fa6f813e0e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fa6e800c560 tx=0x7fa6e8007d20).stop 2023-11-15T02:57:26.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:26.919+0000 7fa6ff307700 1 -- 172.21.15.121:0/1346054256 shutdown_connections 2023-11-15T02:57:26.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:26.919+0000 7fa6ff307700 1 --2- 172.21.15.121:0/1346054256 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa6dc05a510 0x7fa6dc05c9d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:26.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:26.919+0000 7fa6ff307700 1 --2- 172.21.15.121:0/1346054256 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f8151870 0x7fa6f814ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:26.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:26.919+0000 7fa6ff307700 1 --2- 172.21.15.121:0/1346054256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6f80c37f0 0x7fa6f813e0e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:26.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:26.919+0000 7fa6ff307700 1 --2- 172.21.15.121:0/1346054256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f80a9830 0x7fa6f80c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:26.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:26.919+0000 7fa6ff307700 1 -- 172.21.15.121:0/1346054256 >> 172.21.15.121:0/1346054256 conn(0x7fa6f8062190 msgr2=0x7fa6f80c2580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:26.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:26.919+0000 7fa6ff307700 1 -- 172.21.15.121:0/1346054256 shutdown_connections 2023-11-15T02:57:26.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:26.919+0000 7fa6ff307700 1 -- 172.21.15.121:0/1346054256 wait complete. 2023-11-15T02:57:26.945 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-15T02:57:27.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.111+0000 7f8e5129b700 1 Processor -- start 2023-11-15T02:57:27.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.115+0000 7f8e5129b700 1 -- start start 2023-11-15T02:57:27.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.115+0000 7f8e5129b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8e4c158420 0x7f8e4c0657f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:27.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.115+0000 7f8e5129b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e4c065ce0 0x7f8e4c0c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:27.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.115+0000 7f8e5129b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4c0c7280 con 0x7f8e4c153e60 2023-11-15T02:57:27.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.115+0000 7f8e5129b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4c1553c0 con 0x7f8e4c158420 2023-11-15T02:57:27.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.115+0000 7f8e5129b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c1588d0 con 0x7f8e4c065ce0 2023-11-15T02:57:27.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.115+0000 7f8e4affd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f8e4c153e60 0x7f8e4c154240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39150/0 (socket says 172.21.15.121:39150) 2023-11-15T02:57:27.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.115+0000 7f8e4affd700 1 -- 172.21.15.121:0/2030246598 learned_addr learned my addr 172.21.15.121:0/2030246598 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:27.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.115+0000 7f8e4a7fc700 1 --2- 172.21.15.121:0/2030246598 >> v2:172.21.15.121:3300/0 conn(0x7f8e4c158420 0x7f8e4c0657f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:27.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.115+0000 7f8e4b7fe700 1 --2- 172.21.15.121:0/2030246598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e4c065ce0 0x7f8e4c0c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:27.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.115+0000 7f8e4a7fc700 1 -- 172.21.15.121:0/2030246598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e4c065ce0 msgr2=0x7f8e4c0c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:27.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.115+0000 7f8e4a7fc700 1 --2- 172.21.15.121:0/2030246598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e4c065ce0 0x7f8e4c0c5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:27.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.115+0000 7f8e4a7fc700 1 -- 172.21.15.121:0/2030246598 >> v1:172.21.15.121:6789/0 conn(0x7f8e4c153e60 legacy=0x7f8e4c154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:27.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.115+0000 7f8e4a7fc700 1 -- 172.21.15.121:0/2030246598 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c154b20 con 0x7f8e4c158420 2023-11-15T02:57:27.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.115+0000 7f8e4a7fc700 1 --2- 172.21.15.121:0/2030246598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e4c158420 0x7f8e4c0657f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f8e40009760 tx=0x7f8e40009060).ready entity=mon.0 client_cookie=1cdaab753cf30fb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:27.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.115+0000 7f8e49ffb700 1 -- 172.21.15.121:0/2030246598 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000b580 con 0x7f8e4c158420 2023-11-15T02:57:27.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.115+0000 7f8e49ffb700 1 -- 172.21.15.121:0/2030246598 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4000b6e0 con 0x7f8e4c158420 2023-11-15T02:57:27.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.115+0000 7f8e49ffb700 1 -- 172.21.15.121:0/2030246598 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000ba60 con 0x7f8e4c158420 2023-11-15T02:57:27.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.119+0000 7f8e5129b700 1 -- 172.21.15.121:0/2030246598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e4c158420 msgr2=0x7f8e4c0657f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:27.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.119+0000 7f8e5129b700 1 --2- 172.21.15.121:0/2030246598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e4c158420 0x7f8e4c0657f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f8e40009760 tx=0x7f8e40009060).stop 2023-11-15T02:57:27.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.119+0000 7f8e5129b700 1 -- 172.21.15.121:0/2030246598 shutdown_connections 2023-11-15T02:57:27.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.119+0000 7f8e5129b700 1 --2- 172.21.15.121:0/2030246598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e4c065ce0 0x7f8e4c0c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:27.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.119+0000 7f8e5129b700 1 --2- 172.21.15.121:0/2030246598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e4c158420 0x7f8e4c0657f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:27.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.119+0000 7f8e5129b700 1 -- 172.21.15.121:0/2030246598 >> 172.21.15.121:0/2030246598 conn(0x7f8e4c067a60 msgr2=0x7f8e4c067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:27.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.119+0000 7f8e5129b700 1 -- 172.21.15.121:0/2030246598 shutdown_connections 2023-11-15T02:57:27.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.119+0000 7f8e5129b700 1 -- 172.21.15.121:0/2030246598 wait complete. 2023-11-15T02:57:27.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.119+0000 7f8e5129b700 1 Processor -- start 2023-11-15T02:57:27.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.119+0000 7f8e5129b700 1 -- start start 2023-11-15T02:57:27.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.119+0000 7f8e5129b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e4c065ce0 0x7f8e4c0aa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:27.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.119+0000 7f8e5129b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e4c153e60 0x7f8e4c0af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:27.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.119+0000 7f8e5129b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e4c158420 0x7f8e4c0abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:27.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.119+0000 7f8e5129b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c0c22e0 con 0x7f8e4c158420 2023-11-15T02:57:27.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.119+0000 7f8e5129b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c0b0100 con 0x7f8e4c153e60 2023-11-15T02:57:27.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.119+0000 7f8e5129b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c0b03e0 con 0x7f8e4c065ce0 2023-11-15T02:57:27.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.123+0000 7f8e4b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e4c158420 0x7f8e4c0abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:27.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.123+0000 7f8e4affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e4c065ce0 0x7f8e4c0aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:27.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.123+0000 7f8e4a7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e4c153e60 0x7f8e4c0af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:27.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.123+0000 7f8e4b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e4c158420 0x7f8e4c0abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37462/0 (socket says 172.21.15.121:37462) 2023-11-15T02:57:27.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.123+0000 7f8e4b7fe700 1 -- 172.21.15.121:0/512573418 learned_addr learned my addr 172.21.15.121:0/512573418 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:27.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.123+0000 7f8e4affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e4c065ce0 0x7f8e4c0aa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:53786/0 (socket says 172.21.15.121:53786) 2023-11-15T02:57:27.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.123+0000 7f8e4a7fc700 1 --2- 172.21.15.121:0/512573418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e4c153e60 0x7f8e4c0af920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:56454/0 (socket says 172.21.15.121:56454) 2023-11-15T02:57:27.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.123+0000 7f8e4b7fe700 1 -- 172.21.15.121:0/512573418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e4c065ce0 msgr2=0x7f8e4c0aa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:27.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.123+0000 7f8e4b7fe700 1 --2- 172.21.15.121:0/512573418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e4c065ce0 0x7f8e4c0aa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:27.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.123+0000 7f8e4b7fe700 1 -- 172.21.15.121:0/512573418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e4c153e60 msgr2=0x7f8e4c0af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:27.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.123+0000 7f8e4b7fe700 1 --2- 172.21.15.121:0/512573418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e4c153e60 0x7f8e4c0af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:27.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.123+0000 7f8e4b7fe700 1 -- 172.21.15.121:0/512573418 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e400092a0 con 0x7f8e4c158420 2023-11-15T02:57:27.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.123+0000 7f8e4b7fe700 1 --2- 172.21.15.121:0/512573418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e4c158420 0x7f8e4c0abcf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f8e44007f20 tx=0x7f8e44009860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:27.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.123+0000 7f8e32ffd700 1 -- 172.21.15.121:0/512573418 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4400eb90 con 0x7f8e4c158420 2023-11-15T02:57:27.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.123+0000 7f8e32ffd700 1 -- 172.21.15.121:0/512573418 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4400ecf0 con 0x7f8e4c158420 2023-11-15T02:57:27.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.123+0000 7f8e32ffd700 1 -- 172.21.15.121:0/512573418 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44014610 con 0x7f8e4c158420 2023-11-15T02:57:27.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.123+0000 7f8e5129b700 1 -- 172.21.15.121:0/512573418 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4c0b06c0 con 0x7f8e4c158420 2023-11-15T02:57:27.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.123+0000 7f8e5129b700 1 -- 172.21.15.121:0/512573418 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4c141260 con 0x7f8e4c158420 2023-11-15T02:57:27.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.127+0000 7f8e5129b700 1 -- 172.21.15.121:0/512573418 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4c0c2730 con 0x7f8e4c158420 2023-11-15T02:57:27.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.127+0000 7f8e32ffd700 1 -- 172.21.15.121:0/512573418 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8e44014980 con 0x7f8e4c158420 2023-11-15T02:57:27.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.127+0000 7f8e32ffd700 1 --2- 172.21.15.121:0/512573418 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8e3805a510 0x7f8e3805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:27.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.127+0000 7f8e32ffd700 1 -- 172.21.15.121:0/512573418 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8e44074c70 con 0x7f8e4c158420 2023-11-15T02:57:27.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.127+0000 7f8e4affd700 1 --2- 172.21.15.121:0/512573418 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8e3805a510 0x7f8e3805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:27.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.131+0000 7f8e4affd700 1 --2- 172.21.15.121:0/512573418 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8e3805a510 0x7f8e3805c9d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f8e4c068a40 tx=0x7f8e3c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:27.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.135+0000 7f8e32ffd700 1 -- 172.21.15.121:0/512573418 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8e44048390 con 0x7f8e4c158420 2023-11-15T02:57:27.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.303+0000 7f8e5129b700 1 -- 172.21.15.121:0/512573418 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8e4c06bcf0 con 0x7f8e4c158420 2023-11-15T02:57:27.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.311+0000 7f8e32ffd700 1 -- 172.21.15.121:0/512573418 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8e4400ee60 con 0x7f8e4c158420 2023-11-15T02:57:27.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.315+0000 7f8e5129b700 1 -- 172.21.15.121:0/512573418 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8e3805a510 msgr2=0x7f8e3805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:27.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.315+0000 7f8e5129b700 1 --2- 172.21.15.121:0/512573418 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8e3805a510 0x7f8e3805c9d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f8e4c068a40 tx=0x7f8e3c009040).stop 2023-11-15T02:57:27.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.315+0000 7f8e5129b700 1 -- 172.21.15.121:0/512573418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e4c158420 msgr2=0x7f8e4c0abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:27.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.315+0000 7f8e5129b700 1 --2- 172.21.15.121:0/512573418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e4c158420 0x7f8e4c0abcf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f8e44007f20 tx=0x7f8e44009860).stop 2023-11-15T02:57:27.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.315+0000 7f8e5129b700 1 -- 172.21.15.121:0/512573418 shutdown_connections 2023-11-15T02:57:27.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.315+0000 7f8e5129b700 1 --2- 172.21.15.121:0/512573418 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8e3805a510 0x7f8e3805c9d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:27.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.315+0000 7f8e5129b700 1 --2- 172.21.15.121:0/512573418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e4c158420 0x7f8e4c0abcf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:27.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.315+0000 7f8e5129b700 1 --2- 172.21.15.121:0/512573418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e4c153e60 0x7f8e4c0af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:27.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.315+0000 7f8e5129b700 1 --2- 172.21.15.121:0/512573418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e4c065ce0 0x7f8e4c0aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:27.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.315+0000 7f8e5129b700 1 -- 172.21.15.121:0/512573418 >> 172.21.15.121:0/512573418 conn(0x7f8e4c067a60 msgr2=0x7f8e4c153280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:27.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.315+0000 7f8e5129b700 1 -- 172.21.15.121:0/512573418 shutdown_connections 2023-11-15T02:57:27.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:27.315+0000 7f8e5129b700 1 -- 172.21.15.121:0/512573418 wait complete. 2023-11-15T02:57:27.340 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-15T02:57:27.341 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T02:57:27.384 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T02:57:27.430 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-15T02:57:27.430 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T02:57:27.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:57:27.431 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-15T02:57:27.490 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T02:57:27.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:27.491 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T02:57:27.520 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-15T02:57:27.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:57:27.521 DEBUG:teuthology.orchestra.run.smithi190:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-15T02:57:27.572 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2023-11-15T02:57:27.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:27.573 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-15T02:57:27.645 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-15T02:57:27.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:57:27.645 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-15T02:57:27.689 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T02:57:27.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:27.690 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-15T02:57:27.757 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-15T02:57:27.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:57:27.758 DEBUG:teuthology.orchestra.run.smithi190:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-15T02:57:27.802 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2023-11-15T02:57:27.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:27.803 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-15T02:57:27.865 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-15T02:57:27.865 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:57:28.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.023+0000 7fd58ca9b700 1 Processor -- start 2023-11-15T02:57:28.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.027+0000 7fd58ca9b700 1 -- start start 2023-11-15T02:57:28.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.027+0000 7fd58ca9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd588153bb0 0x7fd5880a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:28.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.027+0000 7fd58ca9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd5880aa020 0x7fd588067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:28.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.027+0000 7fd58ca9b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd588068400 con 0x7fd588067f50 2023-11-15T02:57:28.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.027+0000 7fd58ca9b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5880aa6f0 con 0x7fd5880aa020 2023-11-15T02:57:28.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.027+0000 7fd58ca9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5880c5900 con 0x7fd588153bb0 2023-11-15T02:57:28.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.027+0000 7fd57ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd5880aa020 0x7fd588067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:28.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.031+0000 7fd57ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd5880aa020 0x7fd588067a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37476/0 (socket says 172.21.15.121:37476) 2023-11-15T02:57:28.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.031+0000 7fd57ffff700 1 -- 172.21.15.121:0/2890780980 learned_addr learned my addr 172.21.15.121:0/2890780980 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:28.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.031+0000 7fd58659c700 1 --2- 172.21.15.121:0/2890780980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd588153bb0 0x7fd5880a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:28.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.031+0000 7fd57ffff700 1 -- 172.21.15.121:0/2890780980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd588153bb0 msgr2=0x7fd5880a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:28.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.031+0000 7fd57ffff700 1 --2- 172.21.15.121:0/2890780980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd588153bb0 0x7fd5880a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:28.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.031+0000 7fd57ffff700 1 -- 172.21.15.121:0/2890780980 >> v1:172.21.15.190:6789/0 conn(0x7fd588067f50 legacy=0x7fd5881578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:28.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.031+0000 7fd57ffff700 1 -- 172.21.15.121:0/2890780980 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5880cb5c0 con 0x7fd5880aa020 2023-11-15T02:57:28.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.031+0000 7fd57ffff700 1 --2- 172.21.15.121:0/2890780980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5880aa020 0x7fd588067a60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fd570006fd0 tx=0x7fd5700090f0).ready entity=mon.0 client_cookie=41f74e661e9c25bf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:28.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.031+0000 7fd585d9b700 1 -- 172.21.15.121:0/2890780980 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57000b600 con 0x7fd5880aa020 2023-11-15T02:57:28.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.031+0000 7fd585d9b700 1 -- 172.21.15.121:0/2890780980 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57000b760 con 0x7fd5880aa020 2023-11-15T02:57:28.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.031+0000 7fd585d9b700 1 -- 172.21.15.121:0/2890780980 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57000bae0 con 0x7fd5880aa020 2023-11-15T02:57:28.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.031+0000 7fd58ca9b700 1 -- 172.21.15.121:0/2890780980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5880aa020 msgr2=0x7fd588067a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:28.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.031+0000 7fd58ca9b700 1 --2- 172.21.15.121:0/2890780980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5880aa020 0x7fd588067a60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fd570006fd0 tx=0x7fd5700090f0).stop 2023-11-15T02:57:28.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.031+0000 7fd58ca9b700 1 -- 172.21.15.121:0/2890780980 shutdown_connections 2023-11-15T02:57:28.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.031+0000 7fd58ca9b700 1 --2- 172.21.15.121:0/2890780980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5880aa020 0x7fd588067a60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:28.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.031+0000 7fd58ca9b700 1 --2- 172.21.15.121:0/2890780980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd588153bb0 0x7fd5880a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:28.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.031+0000 7fd58ca9b700 1 -- 172.21.15.121:0/2890780980 >> 172.21.15.121:0/2890780980 conn(0x7fd588058c50 msgr2=0x7fd588059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:28.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.031+0000 7fd58ca9b700 1 -- 172.21.15.121:0/2890780980 shutdown_connections 2023-11-15T02:57:28.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.031+0000 7fd58ca9b700 1 -- 172.21.15.121:0/2890780980 wait complete. 2023-11-15T02:57:28.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd58ca9b700 1 Processor -- start 2023-11-15T02:57:28.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd58ca9b700 1 -- start start 2023-11-15T02:57:28.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd58ca9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd588067f50 0x7fd58813cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:28.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd58ca9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5880aa020 0x7fd588156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:28.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd58ca9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd588153bb0 0x7fd588165180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:28.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd58ca9b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5880abed0 con 0x7fd588067f50 2023-11-15T02:57:28.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd58ca9b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd588157130 con 0x7fd588153bb0 2023-11-15T02:57:28.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd58ca9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd588157410 con 0x7fd5880aa020 2023-11-15T02:57:28.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd58659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd588067f50 0x7fd58813cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:28.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd58659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd588067f50 0x7fd58813cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37490/0 (socket says 172.21.15.121:37490) 2023-11-15T02:57:28.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd58659c700 1 -- 172.21.15.121:0/3122643245 learned_addr learned my addr 172.21.15.121:0/3122643245 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:28.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd58659c700 1 -- 172.21.15.121:0/3122643245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5880aa020 msgr2=0x7fd588156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:28.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd58659c700 1 --2- 172.21.15.121:0/3122643245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5880aa020 0x7fd588156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T02:57:28.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd58659c700 1 -- 172.21.15.121:0/3122643245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd588153bb0 msgr2=0x7fd588165180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:28.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd58659c700 1 --2- 172.21.15.121:0/3122643245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd588153bb0 0x7fd588165180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T02:57:28.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd58659c700 1 -- 172.21.15.121:0/3122643245 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd570009370 con 0x7fd588067f50 2023-11-15T02:57:28.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd58659c700 1 --2- 172.21.15.121:0/3122643245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd588067f50 0x7fd58813cd60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fd57800d890 tx=0x7fd578008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:28.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd57e7fc700 1 -- 172.21.15.121:0/3122643245 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5780125f0 con 0x7fd588067f50 2023-11-15T02:57:28.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd57e7fc700 1 -- 172.21.15.121:0/3122643245 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd578012750 con 0x7fd588067f50 2023-11-15T02:57:28.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd57e7fc700 1 -- 172.21.15.121:0/3122643245 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578012ad0 con 0x7fd588067f50 2023-11-15T02:57:28.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd58ca9b700 1 -- 172.21.15.121:0/3122643245 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5880abb60 con 0x7fd588067f50 2023-11-15T02:57:28.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.035+0000 7fd58ca9b700 1 -- 172.21.15.121:0/3122643245 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd588165670 con 0x7fd588067f50 2023-11-15T02:57:28.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.039+0000 7fd58ca9b700 1 -- 172.21.15.121:0/3122643245 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5880ab9c0 con 0x7fd588067f50 2023-11-15T02:57:28.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.039+0000 7fd57e7fc700 1 -- 172.21.15.121:0/3122643245 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd578013480 con 0x7fd588067f50 2023-11-15T02:57:28.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.039+0000 7fd57e7fc700 1 --2- 172.21.15.121:0/3122643245 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd56805a510 0x7fd56805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:28.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.043+0000 7fd57e7fc700 1 -- 172.21.15.121:0/3122643245 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd5780745e0 con 0x7fd588067f50 2023-11-15T02:57:28.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.043+0000 7fd57ffff700 1 --2- 172.21.15.121:0/3122643245 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd56805a510 0x7fd56805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:28.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.043+0000 7fd57ffff700 1 --2- 172.21.15.121:0/3122643245 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd56805a510 0x7fd56805c9d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fd570006fd0 tx=0x7fd570017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:28.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.047+0000 7fd57e7fc700 1 -- 172.21.15.121:0/3122643245 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd5880ab9c0 con 0x7fd588067f50 2023-11-15T02:57:28.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.243+0000 7fd58ca9b700 1 -- 172.21.15.121:0/3122643245 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5880ab9c0 con 0x7fd588067f50 2023-11-15T02:57:28.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.243+0000 7fd57e7fc700 1 -- 172.21.15.121:0/3122643245 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4489 (secure 0 0 0) 0x7fd5880ab9c0 con 0x7fd588067f50 2023-11-15T02:57:28.250 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:28.250 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/802010010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":802010010},{"type":"v1","addr":"172.21.15.190:6835","nonce":802010010}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3991611070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3991611070},{"type":"v1","addr":"172.21.15.121:6836","nonce":3991611070}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3514688785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3514688785},{"type":"v1","addr":"172.21.15.121:6837","nonce":3514688785}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:56:51.695751+0000","modified":"2023-11-15T02:56:54.838464+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6833/742721767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":742721767},{"type":"v1","addr":"172.21.15.190:6833","nonce":742721767}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-15T02:57:28.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.247+0000 7fd58ca9b700 1 -- 172.21.15.121:0/3122643245 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd56805a510 msgr2=0x7fd56805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:28.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.247+0000 7fd58ca9b700 1 --2- 172.21.15.121:0/3122643245 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd56805a510 0x7fd56805c9d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fd570006fd0 tx=0x7fd570017000).stop 2023-11-15T02:57:28.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.247+0000 7fd58ca9b700 1 -- 172.21.15.121:0/3122643245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd588067f50 msgr2=0x7fd58813cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:28.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.247+0000 7fd58ca9b700 1 --2- 172.21.15.121:0/3122643245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd588067f50 0x7fd58813cd60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fd57800d890 tx=0x7fd578008040).stop 2023-11-15T02:57:28.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.247+0000 7fd58ca9b700 1 -- 172.21.15.121:0/3122643245 shutdown_connections 2023-11-15T02:57:28.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.247+0000 7fd58ca9b700 1 --2- 172.21.15.121:0/3122643245 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd56805a510 0x7fd56805c9d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:28.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.247+0000 7fd58ca9b700 1 --2- 172.21.15.121:0/3122643245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd588153bb0 0x7fd588165180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T02:57:28.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.247+0000 7fd58ca9b700 1 --2- 172.21.15.121:0/3122643245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5880aa020 0x7fd588156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T02:57:28.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.247+0000 7fd58ca9b700 1 --2- 172.21.15.121:0/3122643245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd588067f50 0x7fd58813cd60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:28.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.247+0000 7fd58ca9b700 1 -- 172.21.15.121:0/3122643245 >> 172.21.15.121:0/3122643245 conn(0x7fd588058c50 msgr2=0x7fd5880c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:28.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.247+0000 7fd58ca9b700 1 -- 172.21.15.121:0/3122643245 shutdown_connections 2023-11-15T02:57:28.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.247+0000 7fd58ca9b700 1 -- 172.21.15.121:0/3122643245 wait complete. 2023-11-15T02:57:28.256 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 8 2023-11-15T02:57:28.272 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:57:28.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.427+0000 7fe99421b700 1 Processor -- start 2023-11-15T02:57:28.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.427+0000 7fe99421b700 1 -- start start 2023-11-15T02:57:28.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.431+0000 7fe99421b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe98c0c28d0 0x7fe98c159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:28.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.431+0000 7fe99421b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe98c0c3fa0 con 0x7fe98c0c6ce0 2023-11-15T02:57:28.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.431+0000 7fe99421b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe98c159630 con 0x7fe98c152780 2023-11-15T02:57:28.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.431+0000 7fe99421b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe98c0c30b0 con 0x7fe98c0c28d0 2023-11-15T02:57:28.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.431+0000 7fe9917b6700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe98c0c6ce0 0x7fe98c0a5510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39160/0 (socket says 172.21.15.121:39160) 2023-11-15T02:57:28.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.431+0000 7fe9917b6700 1 -- 172.21.15.121:0/2436665014 learned_addr learned my addr 172.21.15.121:0/2436665014 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:28.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.431+0000 7fe9927b8700 1 --2- 172.21.15.121:0/2436665014 >> v2:172.21.15.121:3300/0 conn(0x7fe98c0c28d0 0x7fe98c159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:28.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.431+0000 7fe990fb5700 1 -- 172.21.15.121:0/2436665014 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793592776 0 0) 0x7fe98c0c3fa0 con 0x7fe98c0c6ce0 2023-11-15T02:57:28.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.431+0000 7fe990fb5700 1 -- 172.21.15.121:0/2436665014 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9780033f0 con 0x7fe98c0c6ce0 2023-11-15T02:57:28.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.431+0000 7fe9927b8700 1 -- 172.21.15.121:0/2436665014 >> v1:172.21.15.190:6789/0 conn(0x7fe98c152780 legacy=0x7fe98c0c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:28.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.431+0000 7fe990fb5700 1 -- 172.21.15.121:0/2436665014 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003687285 0 0) 0x7fe98c159630 con 0x7fe98c152780 2023-11-15T02:57:28.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.435+0000 7fe9927b8700 1 -- 172.21.15.121:0/2436665014 >> v1:172.21.15.121:6789/0 conn(0x7fe98c0c6ce0 legacy=0x7fe98c0a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:28.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.435+0000 7fe9927b8700 1 -- 172.21.15.121:0/2436665014 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c0c3390 con 0x7fe98c0c28d0 2023-11-15T02:57:28.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.435+0000 7fe9927b8700 1 --2- 172.21.15.121:0/2436665014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98c0c28d0 0x7fe98c159140 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fe984009e60 tx=0x7fe984009780).ready entity=mon.0 client_cookie=3d3b1b2671fb4ebc server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:28.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.435+0000 7fe990fb5700 1 -- 172.21.15.121:0/2436665014 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98400e070 con 0x7fe98c0c28d0 2023-11-15T02:57:28.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.435+0000 7fe990fb5700 1 -- 172.21.15.121:0/2436665014 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98400b850 con 0x7fe98c0c28d0 2023-11-15T02:57:28.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.435+0000 7fe990fb5700 1 -- 172.21.15.121:0/2436665014 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97c0013f0 con 0x7fe98c0c28d0 2023-11-15T02:57:28.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.435+0000 7fe99421b700 1 -- 172.21.15.121:0/2436665014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98c0c28d0 msgr2=0x7fe98c159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:28.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.435+0000 7fe99421b700 1 --2- 172.21.15.121:0/2436665014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98c0c28d0 0x7fe98c159140 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fe984009e60 tx=0x7fe984009780).stop 2023-11-15T02:57:28.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.435+0000 7fe99421b700 1 -- 172.21.15.121:0/2436665014 shutdown_connections 2023-11-15T02:57:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.435+0000 7fe99421b700 1 --2- 172.21.15.121:0/2436665014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98c0c28d0 0x7fe98c159140 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.435+0000 7fe99421b700 1 -- 172.21.15.121:0/2436665014 >> 172.21.15.121:0/2436665014 conn(0x7fe98c062190 msgr2=0x7fe98c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.435+0000 7fe99421b700 1 -- 172.21.15.121:0/2436665014 shutdown_connections 2023-11-15T02:57:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.435+0000 7fe99421b700 1 -- 172.21.15.121:0/2436665014 wait complete. 2023-11-15T02:57:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe99421b700 1 Processor -- start 2023-11-15T02:57:28.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe99421b700 1 -- start start 2023-11-15T02:57:28.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe99421b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98c0c28d0 0x7fe98c0ab090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:28.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe99421b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe98c0c6ce0 0x7fe98c0aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:28.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe99421b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe98c152780 0x7fe98c168770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:28.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe99421b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0ac440 con 0x7fe98c0c28d0 2023-11-15T02:57:28.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe99421b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0ab580 con 0x7fe98c0c6ce0 2023-11-15T02:57:28.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe99421b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0ab830 con 0x7fe98c152780 2023-11-15T02:57:28.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe991fb7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98c0c28d0 0x7fe98c0ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:28.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe991fb7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98c0c28d0 0x7fe98c0ab090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37504/0 (socket says 172.21.15.121:37504) 2023-11-15T02:57:28.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe991fb7700 1 -- 172.21.15.121:0/3020136477 learned_addr learned my addr 172.21.15.121:0/3020136477 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:28.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe9927b8700 1 --2- 172.21.15.121:0/3020136477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe98c152780 0x7fe98c168770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:28.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe991fb7700 1 -- 172.21.15.121:0/3020136477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe98c152780 msgr2=0x7fe98c168770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:28.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe9917b6700 1 --2- 172.21.15.121:0/3020136477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe98c0c6ce0 0x7fe98c0aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:28.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe991fb7700 1 --2- 172.21.15.121:0/3020136477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe98c152780 0x7fe98c168770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:28.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe991fb7700 1 -- 172.21.15.121:0/3020136477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe98c0c6ce0 msgr2=0x7fe98c0aa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:28.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe991fb7700 1 --2- 172.21.15.121:0/3020136477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe98c0c6ce0 0x7fe98c0aa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:28.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe991fb7700 1 -- 172.21.15.121:0/3020136477 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe984009a20 con 0x7fe98c0c28d0 2023-11-15T02:57:28.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe9917b6700 1 --2- 172.21.15.121:0/3020136477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe98c0c6ce0 0x7fe98c0aa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:57:28.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe991fb7700 1 --2- 172.21.15.121:0/3020136477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98c0c28d0 0x7fe98c0ab090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fe98800c410 tx=0x7fe988007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:28.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe981ffb700 1 -- 172.21.15.121:0/3020136477 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988007020 con 0x7fe98c0c28d0 2023-11-15T02:57:28.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.439+0000 7fe981ffb700 1 -- 172.21.15.121:0/3020136477 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9880071a0 con 0x7fe98c0c28d0 2023-11-15T02:57:28.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.443+0000 7fe981ffb700 1 -- 172.21.15.121:0/3020136477 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988007560 con 0x7fe98c0c28d0 2023-11-15T02:57:28.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.443+0000 7fe99421b700 1 -- 172.21.15.121:0/3020136477 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98c0abb10 con 0x7fe98c0c28d0 2023-11-15T02:57:28.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.443+0000 7fe99421b700 1 -- 172.21.15.121:0/3020136477 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98c0ac0d0 con 0x7fe98c0c28d0 2023-11-15T02:57:28.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.443+0000 7fe99421b700 1 -- 172.21.15.121:0/3020136477 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98c0ac7b0 con 0x7fe98c0c28d0 2023-11-15T02:57:28.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.443+0000 7fe981ffb700 1 -- 172.21.15.121:0/3020136477 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe9880068b0 con 0x7fe98c0c28d0 2023-11-15T02:57:28.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.447+0000 7fe981ffb700 1 --2- 172.21.15.121:0/3020136477 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe97005a510 0x7fe97005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:28.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.447+0000 7fe981ffb700 1 -- 172.21.15.121:0/3020136477 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe988073c00 con 0x7fe98c0c28d0 2023-11-15T02:57:28.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.447+0000 7fe9917b6700 1 --2- 172.21.15.121:0/3020136477 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe97005a510 0x7fe97005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:28.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.447+0000 7fe9917b6700 1 --2- 172.21.15.121:0/3020136477 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe97005a510 0x7fe97005c9d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fe97c0011e0 tx=0x7fe97c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:28.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.451+0000 7fe981ffb700 1 -- 172.21.15.121:0/3020136477 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe98807a080 con 0x7fe98c0c28d0 2023-11-15T02:57:28.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.615+0000 7fe99421b700 1 -- 172.21.15.121:0/3020136477 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe98c06c150 con 0x7fe98c0c28d0 2023-11-15T02:57:28.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.615+0000 7fe981ffb700 1 -- 172.21.15.121:0/3020136477 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fe9880473e0 con 0x7fe98c0c28d0 2023-11-15T02:57:28.622 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:28.622 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":21,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:52.702668+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:56:49.726353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-15T02:56:50.688098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:57:28.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.619+0000 7fe99421b700 1 -- 172.21.15.121:0/3020136477 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe97005a510 msgr2=0x7fe97005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:28.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.619+0000 7fe99421b700 1 --2- 172.21.15.121:0/3020136477 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe97005a510 0x7fe97005c9d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fe97c0011e0 tx=0x7fe97c009040).stop 2023-11-15T02:57:28.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.619+0000 7fe99421b700 1 -- 172.21.15.121:0/3020136477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98c0c28d0 msgr2=0x7fe98c0ab090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:28.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.619+0000 7fe99421b700 1 --2- 172.21.15.121:0/3020136477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98c0c28d0 0x7fe98c0ab090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fe98800c410 tx=0x7fe988007e00).stop 2023-11-15T02:57:28.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.623+0000 7fe99421b700 1 -- 172.21.15.121:0/3020136477 shutdown_connections 2023-11-15T02:57:28.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.623+0000 7fe99421b700 1 --2- 172.21.15.121:0/3020136477 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe97005a510 0x7fe97005c9d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:28.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.623+0000 7fe99421b700 1 --2- 172.21.15.121:0/3020136477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe98c152780 0x7fe98c168770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:28.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.623+0000 7fe99421b700 1 --2- 172.21.15.121:0/3020136477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe98c0c6ce0 0x7fe98c0aa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:28.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.623+0000 7fe99421b700 1 --2- 172.21.15.121:0/3020136477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe98c0c28d0 0x7fe98c0ab090 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:28.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.623+0000 7fe99421b700 1 -- 172.21.15.121:0/3020136477 >> 172.21.15.121:0/3020136477 conn(0x7fe98c062190 msgr2=0x7fe98c0a69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:28.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.623+0000 7fe99421b700 1 -- 172.21.15.121:0/3020136477 shutdown_connections 2023-11-15T02:57:28.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.623+0000 7fe99421b700 1 -- 172.21.15.121:0/3020136477 wait complete. 2023-11-15T02:57:28.646 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-15T02:56:49.726353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-15T02:56:50.688098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-15T02:57:28.647 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T02:57:28.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.815+0000 7f5659d07700 1 Processor -- start 2023-11-15T02:57:28.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.815+0000 7f5659d07700 1 -- start start 2023-11-15T02:57:28.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.815+0000 7f5659d07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f565414ac10 0x7f56540c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:28.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.815+0000 7f5659d07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5654067380 0x7f56540a78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:28.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.815+0000 7f5659d07700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56540657f0 con 0x7f565414aff0 2023-11-15T02:57:28.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.815+0000 7f5659d07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5654066650 con 0x7f5654067380 2023-11-15T02:57:28.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.815+0000 7f5659d07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5654065a40 con 0x7f565414ac10 2023-11-15T02:57:28.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.819+0000 7f5653fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f565414aff0 0x7f5654066c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39172/0 (socket says 172.21.15.121:39172) 2023-11-15T02:57:28.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.819+0000 7f5653fff700 1 -- 172.21.15.121:0/3773461406 learned_addr learned my addr 172.21.15.121:0/3773461406 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:28.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.819+0000 7f5659506700 1 --2- 172.21.15.121:0/3773461406 >> v2:172.21.15.190:3300/0 conn(0x7f5654067380 0x7f56540a78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:28.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.819+0000 7f5658d05700 1 --2- 172.21.15.121:0/3773461406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f565414ac10 0x7f56540c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:28.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.819+0000 7f56537fe700 1 -- 172.21.15.121:0/3773461406 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485124361 0 0) 0x7f56540657f0 con 0x7f565414aff0 2023-11-15T02:57:28.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.819+0000 7f56537fe700 1 -- 172.21.15.121:0/3773461406 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56440033f0 con 0x7f565414aff0 2023-11-15T02:57:28.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.819+0000 7f5659506700 1 -- 172.21.15.121:0/3773461406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f565414ac10 msgr2=0x7f56540c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:28.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.819+0000 7f5659506700 1 --2- 172.21.15.121:0/3773461406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f565414ac10 0x7f56540c6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:28.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.819+0000 7f5659506700 1 -- 172.21.15.121:0/3773461406 >> v1:172.21.15.121:6789/0 conn(0x7f565414aff0 legacy=0x7f5654066c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:28.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.819+0000 7f5659506700 1 -- 172.21.15.121:0/3773461406 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5654065d20 con 0x7f5654067380 2023-11-15T02:57:28.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.819+0000 7f5659506700 1 --2- 172.21.15.121:0/3773461406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5654067380 0x7f56540a78a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f564c009e60 tx=0x7f564c009780).ready entity=mon.1 client_cookie=934bc457d9622c12 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:28.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.819+0000 7f56537fe700 1 -- 172.21.15.121:0/3773461406 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c00e070 con 0x7f5654067380 2023-11-15T02:57:28.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.819+0000 7f56537fe700 1 -- 172.21.15.121:0/3773461406 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c00bab0 con 0x7f5654067380 2023-11-15T02:57:28.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.819+0000 7f56537fe700 1 -- 172.21.15.121:0/3773461406 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c00be30 con 0x7f5654067380 2023-11-15T02:57:28.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.823+0000 7f5659d07700 1 -- 172.21.15.121:0/3773461406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5654067380 msgr2=0x7f56540a78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:28.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.823+0000 7f5659d07700 1 --2- 172.21.15.121:0/3773461406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5654067380 0x7f56540a78a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f564c009e60 tx=0x7f564c009780).stop 2023-11-15T02:57:28.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.823+0000 7f5659d07700 1 -- 172.21.15.121:0/3773461406 shutdown_connections 2023-11-15T02:57:28.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.823+0000 7f5659d07700 1 --2- 172.21.15.121:0/3773461406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5654067380 0x7f56540a78a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:28.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.823+0000 7f5659d07700 1 --2- 172.21.15.121:0/3773461406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f565414ac10 0x7f56540c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:28.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.823+0000 7f5659d07700 1 -- 172.21.15.121:0/3773461406 >> 172.21.15.121:0/3773461406 conn(0x7f5654067a60 msgr2=0x7f5654067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:28.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.823+0000 7f5659d07700 1 -- 172.21.15.121:0/3773461406 shutdown_connections 2023-11-15T02:57:28.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.823+0000 7f5659d07700 1 -- 172.21.15.121:0/3773461406 wait complete. 2023-11-15T02:57:28.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.823+0000 7f5659d07700 1 Processor -- start 2023-11-15T02:57:28.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.823+0000 7f5659d07700 1 -- start start 2023-11-15T02:57:28.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5659d07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5654067380 0x7f5654163be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:28.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5659d07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f565414ac10 0x7f5654152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:28.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5659d07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f565414aff0 0x7f5654149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:28.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5659d07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5654151ef0 con 0x7f565414ac10 2023-11-15T02:57:28.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5659d07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56541640d0 con 0x7f565414aff0 2023-11-15T02:57:28.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5659d07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5654151000 con 0x7f5654067380 2023-11-15T02:57:28.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5653fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f565414ac10 0x7f5654152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:28.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5653fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f565414ac10 0x7f5654152380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37506/0 (socket says 172.21.15.121:37506) 2023-11-15T02:57:28.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5653fff700 1 -- 172.21.15.121:0/2441846222 learned_addr learned my addr 172.21.15.121:0/2441846222 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:28.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5658d05700 1 --2- 172.21.15.121:0/2441846222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5654067380 0x7f5654163be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:28.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5659506700 1 --2- 172.21.15.121:0/2441846222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f565414aff0 0x7f5654149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:28.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5653fff700 1 -- 172.21.15.121:0/2441846222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5654067380 msgr2=0x7f5654163be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:28.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5653fff700 1 --2- 172.21.15.121:0/2441846222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5654067380 0x7f5654163be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:28.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5653fff700 1 -- 172.21.15.121:0/2441846222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f565414aff0 msgr2=0x7f5654149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:28.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5653fff700 1 --2- 172.21.15.121:0/2441846222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f565414aff0 0x7f5654149160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:28.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5653fff700 1 -- 172.21.15.121:0/2441846222 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c009a20 con 0x7f565414ac10 2023-11-15T02:57:28.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5653fff700 1 --2- 172.21.15.121:0/2441846222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f565414ac10 0x7f5654152380 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f564000c450 tx=0x7f5640007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:28.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5650ff9700 1 -- 172.21.15.121:0/2441846222 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56400070d0 con 0x7f565414ac10 2023-11-15T02:57:28.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5650ff9700 1 -- 172.21.15.121:0/2441846222 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5640007230 con 0x7f565414ac10 2023-11-15T02:57:28.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5659d07700 1 -- 172.21.15.121:0/2441846222 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56541512e0 con 0x7f565414ac10 2023-11-15T02:57:28.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5650ff9700 1 -- 172.21.15.121:0/2441846222 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56400075f0 con 0x7f565414ac10 2023-11-15T02:57:28.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.827+0000 7f5659d07700 1 -- 172.21.15.121:0/2441846222 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56541518a0 con 0x7f565414ac10 2023-11-15T02:57:28.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.831+0000 7f5659d07700 1 -- 172.21.15.121:0/2441846222 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56541520a0 con 0x7f565414ac10 2023-11-15T02:57:28.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.831+0000 7f5650ff9700 1 -- 172.21.15.121:0/2441846222 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f56400059b0 con 0x7f565414ac10 2023-11-15T02:57:28.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.835+0000 7f5650ff9700 1 --2- 172.21.15.121:0/2441846222 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f563805a510 0x7f563805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:28.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.835+0000 7f5650ff9700 1 -- 172.21.15.121:0/2441846222 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5640073f40 con 0x7f565414ac10 2023-11-15T02:57:28.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.835+0000 7f5658d05700 1 --2- 172.21.15.121:0/2441846222 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f563805a510 0x7f563805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:28.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.835+0000 7f5658d05700 1 --2- 172.21.15.121:0/2441846222 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f563805a510 0x7f563805c9d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f5648009c30 tx=0x7f5648009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:28.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:28.839+0000 7f5650ff9700 1 -- 172.21.15.121:0/2441846222 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f56541520a0 con 0x7f565414ac10 2023-11-15T02:57:29.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.003+0000 7f5659d07700 1 -- 172.21.15.121:0/2441846222 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f56541520a0 con 0x7f565414ac10 2023-11-15T02:57:29.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.007+0000 7f5650ff9700 1 -- 172.21.15.121:0/2441846222 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v21) v1 ==== 110+0+10 (secure 0 0 0) 0x7f56541520a0 con 0x7f565414ac10 2023-11-15T02:57:29.012 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T02:57:29.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.007+0000 7f5659d07700 1 -- 172.21.15.121:0/2441846222 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f563805a510 msgr2=0x7f563805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.007+0000 7f5659d07700 1 --2- 172.21.15.121:0/2441846222 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f563805a510 0x7f563805c9d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f5648009c30 tx=0x7f5648009060).stop 2023-11-15T02:57:29.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.007+0000 7f5659d07700 1 -- 172.21.15.121:0/2441846222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f565414ac10 msgr2=0x7f5654152380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.007+0000 7f5659d07700 1 --2- 172.21.15.121:0/2441846222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f565414ac10 0x7f5654152380 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f564000c450 tx=0x7f5640007c50).stop 2023-11-15T02:57:29.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.007+0000 7f5659d07700 1 -- 172.21.15.121:0/2441846222 shutdown_connections 2023-11-15T02:57:29.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.007+0000 7f5659d07700 1 --2- 172.21.15.121:0/2441846222 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f563805a510 0x7f563805c9d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.007+0000 7f5659d07700 1 --2- 172.21.15.121:0/2441846222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f565414aff0 0x7f5654149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.007+0000 7f5659d07700 1 --2- 172.21.15.121:0/2441846222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f565414ac10 0x7f5654152380 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.007+0000 7f5659d07700 1 --2- 172.21.15.121:0/2441846222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5654067380 0x7f5654163be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.007+0000 7f5659d07700 1 -- 172.21.15.121:0/2441846222 >> 172.21.15.121:0/2441846222 conn(0x7f5654067a60 msgr2=0x7f5654067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:29.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.007+0000 7f5659d07700 1 -- 172.21.15.121:0/2441846222 shutdown_connections 2023-11-15T02:57:29.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.011+0000 7f5659d07700 1 -- 172.21.15.121:0/2441846222 wait complete. 2023-11-15T02:57:29.033 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T02:57:29.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.187+0000 7fb07e7f9700 1 Processor -- start 2023-11-15T02:57:29.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.191+0000 7fb07e7f9700 1 -- start start 2023-11-15T02:57:29.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.191+0000 7fb07e7f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb078066330 0x7fb078153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:29.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.191+0000 7fb07e7f9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb078065ea0 con 0x7fb0780acd00 2023-11-15T02:57:29.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.195+0000 7fb07e7f9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb078066bb0 con 0x7fb0780c5c70 2023-11-15T02:57:29.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.195+0000 7fb07e7f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb078065890 con 0x7fb078066330 2023-11-15T02:57:29.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.195+0000 7fb07d7f7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb0780acd00 0x7fb0781536c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39178/0 (socket says 172.21.15.121:39178) 2023-11-15T02:57:29.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.195+0000 7fb07d7f7700 1 -- 172.21.15.121:0/2857222262 learned_addr learned my addr 172.21.15.121:0/2857222262 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:29.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.195+0000 7fb07dff8700 1 --2- 172.21.15.121:0/2857222262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb078066330 0x7fb078153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:29.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.195+0000 7fb06ffff700 1 -- 172.21.15.121:0/2857222262 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753175950 0 0) 0x7fb078065ea0 con 0x7fb0780acd00 2023-11-15T02:57:29.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.195+0000 7fb06ffff700 1 -- 172.21.15.121:0/2857222262 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0640033f0 con 0x7fb0780acd00 2023-11-15T02:57:29.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.195+0000 7fb07dff8700 1 -- 172.21.15.121:0/2857222262 >> v1:172.21.15.190:6789/0 conn(0x7fb0780c5c70 legacy=0x7fb0781577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.195+0000 7fb07dff8700 1 -- 172.21.15.121:0/2857222262 >> v1:172.21.15.121:6789/0 conn(0x7fb0780acd00 legacy=0x7fb0781536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.195+0000 7fb07dff8700 1 -- 172.21.15.121:0/2857222262 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0780c64a0 con 0x7fb078066330 2023-11-15T02:57:29.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.195+0000 7fb07dff8700 1 --2- 172.21.15.121:0/2857222262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb078066330 0x7fb078153dd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fb070009e90 tx=0x7fb070009780).ready entity=mon.2 client_cookie=fb7804629af3c1b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:29.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.195+0000 7fb06ffff700 1 -- 172.21.15.121:0/2857222262 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07000e070 con 0x7fb078066330 2023-11-15T02:57:29.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.195+0000 7fb06ffff700 1 -- 172.21.15.121:0/2857222262 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07000bb60 con 0x7fb078066330 2023-11-15T02:57:29.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.195+0000 7fb06ffff700 1 -- 172.21.15.121:0/2857222262 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb070012410 con 0x7fb078066330 2023-11-15T02:57:29.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.195+0000 7fb07e7f9700 1 -- 172.21.15.121:0/2857222262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb078066330 msgr2=0x7fb078153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.195+0000 7fb07e7f9700 1 --2- 172.21.15.121:0/2857222262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb078066330 0x7fb078153dd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fb070009e90 tx=0x7fb070009780).stop 2023-11-15T02:57:29.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.195+0000 7fb07e7f9700 1 -- 172.21.15.121:0/2857222262 shutdown_connections 2023-11-15T02:57:29.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.195+0000 7fb07e7f9700 1 --2- 172.21.15.121:0/2857222262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb078066330 0x7fb078153dd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.195+0000 7fb07e7f9700 1 -- 172.21.15.121:0/2857222262 >> 172.21.15.121:0/2857222262 conn(0x7fb078063fc0 msgr2=0x7fb078067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:29.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.199+0000 7fb07e7f9700 1 -- 172.21.15.121:0/2857222262 shutdown_connections 2023-11-15T02:57:29.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.199+0000 7fb07e7f9700 1 -- 172.21.15.121:0/2857222262 wait complete. 2023-11-15T02:57:29.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.199+0000 7fb07e7f9700 1 Processor -- start 2023-11-15T02:57:29.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.199+0000 7fb07e7f9700 1 -- start start 2023-11-15T02:57:29.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07e7f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb078066330 0x7fb078141670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:29.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07e7f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0780acd00 0x7fb0780ad960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:29.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07e7f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0780c5c70 0x7fb0780ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:29.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07e7f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0780ae340 con 0x7fb078066330 2023-11-15T02:57:29.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07e7f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb078164e70 con 0x7fb0780acd00 2023-11-15T02:57:29.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07e7f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0780c2ba0 con 0x7fb0780c5c70 2023-11-15T02:57:29.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07d7f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb078066330 0x7fb078141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:29.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07d7f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb078066330 0x7fb078141670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37516/0 (socket says 172.21.15.121:37516) 2023-11-15T02:57:29.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07d7f7700 1 -- 172.21.15.121:0/3154478483 learned_addr learned my addr 172.21.15.121:0/3154478483 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:29.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07dff8700 1 --2- 172.21.15.121:0/3154478483 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0780c5c70 0x7fb0780ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:29.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07cff6700 1 --2- 172.21.15.121:0/3154478483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0780acd00 0x7fb0780ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:29.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07d7f7700 1 -- 172.21.15.121:0/3154478483 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0780c5c70 msgr2=0x7fb0780ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07d7f7700 1 --2- 172.21.15.121:0/3154478483 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0780c5c70 0x7fb0780ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07d7f7700 1 -- 172.21.15.121:0/3154478483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0780acd00 msgr2=0x7fb0780ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07d7f7700 1 --2- 172.21.15.121:0/3154478483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0780acd00 0x7fb0780ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07d7f7700 1 -- 172.21.15.121:0/3154478483 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb070009a20 con 0x7fb078066330 2023-11-15T02:57:29.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07cff6700 1 --2- 172.21.15.121:0/3154478483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0780acd00 0x7fb0780ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:29.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07d7f7700 1 --2- 172.21.15.121:0/3154478483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb078066330 0x7fb078141670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fb07400c4b0 tx=0x7fb074007b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:29.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb06d7fa700 1 -- 172.21.15.121:0/3154478483 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0740072e0 con 0x7fb078066330 2023-11-15T02:57:29.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb06d7fa700 1 -- 172.21.15.121:0/3154478483 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb074007440 con 0x7fb078066330 2023-11-15T02:57:29.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07e7f9700 1 -- 172.21.15.121:0/3154478483 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0780c2e80 con 0x7fb078066330 2023-11-15T02:57:29.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb06d7fa700 1 -- 172.21.15.121:0/3154478483 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb074005880 con 0x7fb078066330 2023-11-15T02:57:29.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.203+0000 7fb07e7f9700 1 -- 172.21.15.121:0/3154478483 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb078141b60 con 0x7fb078066330 2023-11-15T02:57:29.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.207+0000 7fb07e7f9700 1 -- 172.21.15.121:0/3154478483 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0780c2a00 con 0x7fb078066330 2023-11-15T02:57:29.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.207+0000 7fb06d7fa700 1 -- 172.21.15.121:0/3154478483 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb074005c70 con 0x7fb078066330 2023-11-15T02:57:29.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.211+0000 7fb06d7fa700 1 --2- 172.21.15.121:0/3154478483 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb05c05a510 0x7fb05c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:29.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.211+0000 7fb06d7fa700 1 -- 172.21.15.121:0/3154478483 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb074074060 con 0x7fb078066330 2023-11-15T02:57:29.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.211+0000 7fb07cff6700 1 --2- 172.21.15.121:0/3154478483 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb05c05a510 0x7fb05c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:29.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.211+0000 7fb07cff6700 1 --2- 172.21.15.121:0/3154478483 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb05c05a510 0x7fb05c05c9d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fb0780cb790 tx=0x7fb068009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:29.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.215+0000 7fb06d7fa700 1 -- 172.21.15.121:0/3154478483 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb0780c2a00 con 0x7fb078066330 2023-11-15T02:57:29.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.383+0000 7fb07e7f9700 1 -- 172.21.15.121:0/3154478483 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb0780c2a00 con 0x7fb078066330 2023-11-15T02:57:29.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.383+0000 7fb06d7fa700 1 -- 172.21.15.121:0/3154478483 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb0780c2a00 con 0x7fb078066330 2023-11-15T02:57:29.388 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T02:57:29.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.383+0000 7fb07e7f9700 1 -- 172.21.15.121:0/3154478483 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb05c05a510 msgr2=0x7fb05c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.383+0000 7fb07e7f9700 1 --2- 172.21.15.121:0/3154478483 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb05c05a510 0x7fb05c05c9d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fb0780cb790 tx=0x7fb068009040).stop 2023-11-15T02:57:29.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.387+0000 7fb07e7f9700 1 -- 172.21.15.121:0/3154478483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb078066330 msgr2=0x7fb078141670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.387+0000 7fb07e7f9700 1 --2- 172.21.15.121:0/3154478483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb078066330 0x7fb078141670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fb07400c4b0 tx=0x7fb074007b00).stop 2023-11-15T02:57:29.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.387+0000 7fb07e7f9700 1 -- 172.21.15.121:0/3154478483 shutdown_connections 2023-11-15T02:57:29.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.387+0000 7fb07e7f9700 1 --2- 172.21.15.121:0/3154478483 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb05c05a510 0x7fb05c05c9d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.387+0000 7fb07e7f9700 1 --2- 172.21.15.121:0/3154478483 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0780c5c70 0x7fb0780ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.387+0000 7fb07e7f9700 1 --2- 172.21.15.121:0/3154478483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0780acd00 0x7fb0780ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.387+0000 7fb07e7f9700 1 --2- 172.21.15.121:0/3154478483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb078066330 0x7fb078141670 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.387+0000 7fb07e7f9700 1 -- 172.21.15.121:0/3154478483 >> 172.21.15.121:0/3154478483 conn(0x7fb078063fc0 msgr2=0x7fb0780cacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:29.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.387+0000 7fb07e7f9700 1 -- 172.21.15.121:0/3154478483 shutdown_connections 2023-11-15T02:57:29.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.387+0000 7fb07e7f9700 1 -- 172.21.15.121:0/3154478483 wait complete. 2023-11-15T02:57:29.413 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-15T02:57:29.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.571+0000 7f15bd85b700 1 Processor -- start 2023-11-15T02:57:29.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.571+0000 7f15bd85b700 1 -- start start 2023-11-15T02:57:29.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.575+0000 7f15bd85b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15b8067020 0x7f15b81582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:29.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.575+0000 7f15bd85b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f15b8068170 0x7f15b80bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:29.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.575+0000 7f15bd85b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b8062da0 con 0x7f15b8157ed0 2023-11-15T02:57:29.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.575+0000 7f15bd85b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b80a93a0 con 0x7f15b8068170 2023-11-15T02:57:29.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.575+0000 7f15bd85b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b8062190 con 0x7f15b8067020 2023-11-15T02:57:29.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.575+0000 7f15b77fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f15b8068170 0x7f15b80bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:29.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.575+0000 7f15affff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15b8067020 0x7f15b81582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:29.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.575+0000 7f15b6ffd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f15b8157ed0 0x7f15b8067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:33724/0 (socket says 172.21.15.121:33724) 2023-11-15T02:57:29.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.575+0000 7f15b6ffd700 1 -- 172.21.15.121:0/4202564457 learned_addr learned my addr 172.21.15.121:0/4202564457 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:29.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.575+0000 7f15affff700 1 -- 172.21.15.121:0/4202564457 >> v2:172.21.15.121:3300/0 conn(0x7f15b8068170 msgr2=0x7f15b80bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:29.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.575+0000 7f15affff700 1 --2- 172.21.15.121:0/4202564457 >> v2:172.21.15.121:3300/0 conn(0x7f15b8068170 0x7f15b80bc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.575+0000 7f15affff700 1 -- 172.21.15.121:0/4202564457 >> v1:172.21.15.190:6789/0 conn(0x7f15b8157ed0 legacy=0x7f15b8067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.575+0000 7f15affff700 1 -- 172.21.15.121:0/4202564457 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b8062470 con 0x7f15b8067020 2023-11-15T02:57:29.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.575+0000 7f15affff700 1 --2- 172.21.15.121:0/4202564457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15b8067020 0x7f15b81582b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f15a0009760 tx=0x7f15a0009060).ready entity=mon.2 client_cookie=2fe04cc31e1eadc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:29.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.575+0000 7f15b67fc700 1 -- 172.21.15.121:0/4202564457 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a000f510 con 0x7f15b8067020 2023-11-15T02:57:29.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.575+0000 7f15b67fc700 1 -- 172.21.15.121:0/4202564457 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a000f670 con 0x7f15b8067020 2023-11-15T02:57:29.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.575+0000 7f15b67fc700 1 -- 172.21.15.121:0/4202564457 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a000f9f0 con 0x7f15b8067020 2023-11-15T02:57:29.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.575+0000 7f15bd85b700 1 -- 172.21.15.121:0/4202564457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15b8067020 msgr2=0x7f15b81582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.575+0000 7f15bd85b700 1 --2- 172.21.15.121:0/4202564457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15b8067020 0x7f15b81582b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f15a0009760 tx=0x7f15a0009060).stop 2023-11-15T02:57:29.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.579+0000 7f15bd85b700 1 -- 172.21.15.121:0/4202564457 shutdown_connections 2023-11-15T02:57:29.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.579+0000 7f15bd85b700 1 --2- 172.21.15.121:0/4202564457 >> v2:172.21.15.121:3300/0 conn(0x7f15b8068170 0x7f15b80bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.579+0000 7f15bd85b700 1 --2- 172.21.15.121:0/4202564457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15b8067020 0x7f15b81582b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.579+0000 7f15bd85b700 1 -- 172.21.15.121:0/4202564457 >> 172.21.15.121:0/4202564457 conn(0x7f15b80591c0 msgr2=0x7f15b80c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:29.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.579+0000 7f15bd85b700 1 -- 172.21.15.121:0/4202564457 shutdown_connections 2023-11-15T02:57:29.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.579+0000 7f15bd85b700 1 -- 172.21.15.121:0/4202564457 wait complete. 2023-11-15T02:57:29.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15bd85b700 1 Processor -- start 2023-11-15T02:57:29.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15bd85b700 1 -- start start 2023-11-15T02:57:29.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15bd85b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15b8067020 0x7f15b816b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:29.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15bd85b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15b8068170 0x7f15b8169230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:29.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15bd85b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15b8157ed0 0x7f15b80c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:29.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15bd85b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b816bae0 con 0x7f15b8068170 2023-11-15T02:57:29.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15bd85b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b816c140 con 0x7f15b8157ed0 2023-11-15T02:57:29.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15bd85b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b816c420 con 0x7f15b8067020 2023-11-15T02:57:29.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15affff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15b8068170 0x7f15b8169230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:29.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15b6ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15b8067020 0x7f15b816b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:29.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15affff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15b8068170 0x7f15b8169230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37526/0 (socket says 172.21.15.121:37526) 2023-11-15T02:57:29.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15affff700 1 -- 172.21.15.121:0/617707801 learned_addr learned my addr 172.21.15.121:0/617707801 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:29.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15b77fe700 1 --2- 172.21.15.121:0/617707801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15b8157ed0 0x7f15b80c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:29.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15affff700 1 -- 172.21.15.121:0/617707801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15b8067020 msgr2=0x7f15b816b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15affff700 1 --2- 172.21.15.121:0/617707801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15b8067020 0x7f15b816b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15affff700 1 -- 172.21.15.121:0/617707801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15b8157ed0 msgr2=0x7f15b80c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15affff700 1 --2- 172.21.15.121:0/617707801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15b8157ed0 0x7f15b80c45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15affff700 1 -- 172.21.15.121:0/617707801 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a00092a0 con 0x7f15b8068170 2023-11-15T02:57:29.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15b77fe700 1 --2- 172.21.15.121:0/617707801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15b8157ed0 0x7f15b80c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:29.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15affff700 1 --2- 172.21.15.121:0/617707801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15b8068170 0x7f15b8169230 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f15b80622d0 tx=0x7f15a0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:29.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15aeffd700 1 -- 172.21.15.121:0/617707801 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a000f7e0 con 0x7f15b8068170 2023-11-15T02:57:29.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.583+0000 7f15aeffd700 1 -- 172.21.15.121:0/617707801 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a000f940 con 0x7f15b8068170 2023-11-15T02:57:29.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.587+0000 7f15bd85b700 1 -- 172.21.15.121:0/617707801 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b816c700 con 0x7f15b8068170 2023-11-15T02:57:29.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.587+0000 7f15aeffd700 1 -- 172.21.15.121:0/617707801 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a001b660 con 0x7f15b8068170 2023-11-15T02:57:29.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.587+0000 7f15bd85b700 1 -- 172.21.15.121:0/617707801 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b8169720 con 0x7f15b8068170 2023-11-15T02:57:29.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.587+0000 7f15bd85b700 1 -- 172.21.15.121:0/617707801 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b8047060 con 0x7f15b8068170 2023-11-15T02:57:29.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.587+0000 7f15aeffd700 1 -- 172.21.15.121:0/617707801 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f15a001ba10 con 0x7f15b8068170 2023-11-15T02:57:29.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.591+0000 7f15aeffd700 1 --2- 172.21.15.121:0/617707801 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f159805a510 0x7f159805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:29.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.591+0000 7f15aeffd700 1 -- 172.21.15.121:0/617707801 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f15a0075fb0 con 0x7f15b8068170 2023-11-15T02:57:29.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.591+0000 7f15b6ffd700 1 --2- 172.21.15.121:0/617707801 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f159805a510 0x7f159805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:29.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.591+0000 7f15b6ffd700 1 --2- 172.21.15.121:0/617707801 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f159805a510 0x7f159805c9d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f15a8007fd0 tx=0x7f15a8009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:29.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.595+0000 7f15aeffd700 1 -- 172.21.15.121:0/617707801 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f15a00a4af0 con 0x7f15b8068170 2023-11-15T02:57:29.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.763+0000 7f15bd85b700 1 -- 172.21.15.121:0/617707801 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f15b81699e0 con 0x7f15b8068170 2023-11-15T02:57:29.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.763+0000 7f15aeffd700 1 -- 172.21.15.121:0/617707801 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f15a00290a0 con 0x7f15b8068170 2023-11-15T02:57:29.768 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T02:57:29.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.767+0000 7f15bd85b700 1 -- 172.21.15.121:0/617707801 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f159805a510 msgr2=0x7f159805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.767+0000 7f15bd85b700 1 --2- 172.21.15.121:0/617707801 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f159805a510 0x7f159805c9d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f15a8007fd0 tx=0x7f15a8009040).stop 2023-11-15T02:57:29.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.767+0000 7f15bd85b700 1 -- 172.21.15.121:0/617707801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15b8068170 msgr2=0x7f15b8169230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.767+0000 7f15bd85b700 1 --2- 172.21.15.121:0/617707801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15b8068170 0x7f15b8169230 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f15b80622d0 tx=0x7f15a0017000).stop 2023-11-15T02:57:29.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.767+0000 7f15bd85b700 1 -- 172.21.15.121:0/617707801 shutdown_connections 2023-11-15T02:57:29.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.767+0000 7f15bd85b700 1 --2- 172.21.15.121:0/617707801 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f159805a510 0x7f159805c9d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.767+0000 7f15bd85b700 1 --2- 172.21.15.121:0/617707801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15b8157ed0 0x7f15b80c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.767+0000 7f15bd85b700 1 --2- 172.21.15.121:0/617707801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15b8068170 0x7f15b8169230 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.767+0000 7f15bd85b700 1 --2- 172.21.15.121:0/617707801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15b8067020 0x7f15b816b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.767+0000 7f15bd85b700 1 -- 172.21.15.121:0/617707801 >> 172.21.15.121:0/617707801 conn(0x7f15b80591c0 msgr2=0x7f15b8156af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:29.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.767+0000 7f15bd85b700 1 -- 172.21.15.121:0/617707801 shutdown_connections 2023-11-15T02:57:29.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.767+0000 7f15bd85b700 1 -- 172.21.15.121:0/617707801 wait complete. 2023-11-15T02:57:29.792 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-15T02:57:29.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.955+0000 7f9e6929d700 1 Processor -- start 2023-11-15T02:57:29.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.955+0000 7f9e6929d700 1 -- start start 2023-11-15T02:57:29.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.959+0000 7f9e6929d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9e640c15c0 0x7f9e640c2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:29.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.959+0000 7f9e6929d700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e640657f0 con 0x7f9e640a8960 2023-11-15T02:57:29.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.959+0000 7f9e6929d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e640c1b60 con 0x7f9e640a8d40 2023-11-15T02:57:29.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.959+0000 7f9e6929d700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e641583d0 con 0x7f9e640c15c0 2023-11-15T02:57:29.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.959+0000 7f9e63fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9e640a8960 0x7f9e6415c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39192/0 (socket says 172.21.15.121:39192) 2023-11-15T02:57:29.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.959+0000 7f9e63fff700 1 -- 172.21.15.121:0/3635288696 learned_addr learned my addr 172.21.15.121:0/3635288696 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:29.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.959+0000 7f9e68a9c700 1 --2- 172.21.15.121:0/3635288696 >> v2:172.21.15.121:3300/0 conn(0x7f9e640c15c0 0x7f9e640c2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:29.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.959+0000 7f9e637fe700 1 -- 172.21.15.121:0/3635288696 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2671762119 0 0) 0x7f9e640657f0 con 0x7f9e640a8960 2023-11-15T02:57:29.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.959+0000 7f9e637fe700 1 -- 172.21.15.121:0/3635288696 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e500033f0 con 0x7f9e640a8960 2023-11-15T02:57:29.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.959+0000 7f9e637fe700 1 -- 172.21.15.121:0/3635288696 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091639102 0 0) 0x7f9e640c1b60 con 0x7f9e640a8d40 2023-11-15T02:57:29.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.959+0000 7f9e637fe700 1 -- 172.21.15.121:0/3635288696 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e640657f0 con 0x7f9e640a8d40 2023-11-15T02:57:29.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.959+0000 7f9e68a9c700 1 -- 172.21.15.121:0/3635288696 >> v1:172.21.15.190:6789/0 conn(0x7f9e640a8d40 legacy=0x7f9e64066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.959+0000 7f9e68a9c700 1 -- 172.21.15.121:0/3635288696 >> v1:172.21.15.121:6789/0 conn(0x7f9e640a8960 legacy=0x7f9e6415c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.959+0000 7f9e68a9c700 1 -- 172.21.15.121:0/3635288696 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e64065a40 con 0x7f9e640c15c0 2023-11-15T02:57:29.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.959+0000 7f9e68a9c700 1 --2- 172.21.15.121:0/3635288696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e640c15c0 0x7f9e640c2e40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f9e5c009760 tx=0x7f9e5c009060).ready entity=mon.0 client_cookie=280b5a59628496bf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:29.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.959+0000 7f9e637fe700 1 -- 172.21.15.121:0/3635288696 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c00d4a0 con 0x7f9e640c15c0 2023-11-15T02:57:29.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.963+0000 7f9e637fe700 1 -- 172.21.15.121:0/3635288696 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5c00d600 con 0x7f9e640c15c0 2023-11-15T02:57:29.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.963+0000 7f9e637fe700 1 -- 172.21.15.121:0/3635288696 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c00d980 con 0x7f9e640c15c0 2023-11-15T02:57:29.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.963+0000 7f9e6929d700 1 -- 172.21.15.121:0/3635288696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e640c15c0 msgr2=0x7f9e640c2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.963+0000 7f9e6929d700 1 --2- 172.21.15.121:0/3635288696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e640c15c0 0x7f9e640c2e40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f9e5c009760 tx=0x7f9e5c009060).stop 2023-11-15T02:57:29.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.963+0000 7f9e6929d700 1 -- 172.21.15.121:0/3635288696 shutdown_connections 2023-11-15T02:57:29.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.963+0000 7f9e6929d700 1 --2- 172.21.15.121:0/3635288696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e640c15c0 0x7f9e640c2e40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.963+0000 7f9e6929d700 1 -- 172.21.15.121:0/3635288696 >> 172.21.15.121:0/3635288696 conn(0x7f9e64067a60 msgr2=0x7f9e64067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:29.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.963+0000 7f9e6929d700 1 -- 172.21.15.121:0/3635288696 shutdown_connections 2023-11-15T02:57:29.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.963+0000 7f9e6929d700 1 -- 172.21.15.121:0/3635288696 wait complete. 2023-11-15T02:57:29.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.963+0000 7f9e6929d700 1 Processor -- start 2023-11-15T02:57:29.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.963+0000 7f9e6929d700 1 -- start start 2023-11-15T02:57:29.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e6929d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e640a8960 0x7f9e64155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:29.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e6929d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e640a8d40 0x7f9e640c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:29.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e6929d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e640c15c0 0x7f9e640c4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:29.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e6929d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640c4cf0 con 0x7f9e640a8960 2023-11-15T02:57:29.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e6929d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e641595c0 con 0x7f9e640c15c0 2023-11-15T02:57:29.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e6929d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640c70f0 con 0x7f9e640a8d40 2023-11-15T02:57:29.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e63fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e640a8960 0x7f9e64155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:29.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e63fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e640a8960 0x7f9e64155ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37544/0 (socket says 172.21.15.121:37544) 2023-11-15T02:57:29.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e63fff700 1 -- 172.21.15.121:0/217719752 learned_addr learned my addr 172.21.15.121:0/217719752 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:29.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e5b7fe700 1 --2- 172.21.15.121:0/217719752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e640a8d40 0x7f9e640c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:29.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e68a9c700 1 --2- 172.21.15.121:0/217719752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e640c15c0 0x7f9e640c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:29.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e63fff700 1 -- 172.21.15.121:0/217719752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e640a8d40 msgr2=0x7f9e640c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e63fff700 1 --2- 172.21.15.121:0/217719752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e640a8d40 0x7f9e640c4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e63fff700 1 -- 172.21.15.121:0/217719752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e640c15c0 msgr2=0x7f9e640c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:29.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e63fff700 1 --2- 172.21.15.121:0/217719752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e640c15c0 0x7f9e640c4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:29.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e63fff700 1 -- 172.21.15.121:0/217719752 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5c0092a0 con 0x7f9e640a8960 2023-11-15T02:57:29.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e68a9c700 1 --2- 172.21.15.121:0/217719752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e640c15c0 0x7f9e640c4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:29.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e5b7fe700 1 --2- 172.21.15.121:0/217719752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e640a8d40 0x7f9e640c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:29.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e63fff700 1 --2- 172.21.15.121:0/217719752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e640a8960 0x7f9e64155ea0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f9e5400c560 tx=0x7f9e54007d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:29.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e60ff9700 1 -- 172.21.15.121:0/217719752 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e540072e0 con 0x7f9e640a8960 2023-11-15T02:57:29.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e60ff9700 1 -- 172.21.15.121:0/217719752 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e54007440 con 0x7f9e640a8960 2023-11-15T02:57:29.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e6929d700 1 -- 172.21.15.121:0/217719752 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e64158c90 con 0x7f9e640a8960 2023-11-15T02:57:29.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e60ff9700 1 -- 172.21.15.121:0/217719752 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e540057a0 con 0x7f9e640a8960 2023-11-15T02:57:29.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.967+0000 7f9e6929d700 1 -- 172.21.15.121:0/217719752 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e64159250 con 0x7f9e640a8960 2023-11-15T02:57:29.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.971+0000 7f9e6929d700 1 -- 172.21.15.121:0/217719752 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e6406bcf0 con 0x7f9e640a8960 2023-11-15T02:57:29.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.971+0000 7f9e60ff9700 1 -- 172.21.15.121:0/217719752 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9e54005b90 con 0x7f9e640a8960 2023-11-15T02:57:29.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.971+0000 7f9e60ff9700 1 --2- 172.21.15.121:0/217719752 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9e4405a540 0x7f9e4405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:29.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.971+0000 7f9e60ff9700 1 -- 172.21.15.121:0/217719752 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9e54073f80 con 0x7f9e640a8960 2023-11-15T02:57:29.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.975+0000 7f9e5b7fe700 1 --2- 172.21.15.121:0/217719752 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9e4405a540 0x7f9e4405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:29.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.975+0000 7f9e5b7fe700 1 --2- 172.21.15.121:0/217719752 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9e4405a540 0x7f9e4405ca00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f9e640a8030 tx=0x7f9e4c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:29.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:29.979+0000 7f9e60ff9700 1 -- 172.21.15.121:0/217719752 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9e540476a0 con 0x7f9e640a8960 2023-11-15T02:57:30.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.139+0000 7f9e6929d700 1 -- 172.21.15.121:0/217719752 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9e64068a70 con 0x7f9e640a8960 2023-11-15T02:57:30.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.139+0000 7f9e60ff9700 1 -- 172.21.15.121:0/217719752 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9e54007e60 con 0x7f9e640a8960 2023-11-15T02:57:30.147 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T02:57:30.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.143+0000 7f9e6929d700 1 -- 172.21.15.121:0/217719752 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9e4405a540 msgr2=0x7f9e4405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:30.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.143+0000 7f9e6929d700 1 --2- 172.21.15.121:0/217719752 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9e4405a540 0x7f9e4405ca00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f9e640a8030 tx=0x7f9e4c009040).stop 2023-11-15T02:57:30.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.143+0000 7f9e6929d700 1 -- 172.21.15.121:0/217719752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e640a8960 msgr2=0x7f9e64155ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:30.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.143+0000 7f9e6929d700 1 --2- 172.21.15.121:0/217719752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e640a8960 0x7f9e64155ea0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f9e5400c560 tx=0x7f9e54007d20).stop 2023-11-15T02:57:30.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.143+0000 7f9e6929d700 1 -- 172.21.15.121:0/217719752 shutdown_connections 2023-11-15T02:57:30.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.143+0000 7f9e6929d700 1 --2- 172.21.15.121:0/217719752 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9e4405a540 0x7f9e4405ca00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.143+0000 7f9e6929d700 1 --2- 172.21.15.121:0/217719752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e640c15c0 0x7f9e640c4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.143+0000 7f9e6929d700 1 --2- 172.21.15.121:0/217719752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e640a8d40 0x7f9e640c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.143+0000 7f9e6929d700 1 --2- 172.21.15.121:0/217719752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e640a8960 0x7f9e64155ea0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.143+0000 7f9e6929d700 1 -- 172.21.15.121:0/217719752 >> 172.21.15.121:0/217719752 conn(0x7f9e64067a60 msgr2=0x7f9e640a98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:30.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.143+0000 7f9e6929d700 1 -- 172.21.15.121:0/217719752 shutdown_connections 2023-11-15T02:57:30.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.143+0000 7f9e6929d700 1 -- 172.21.15.121:0/217719752 wait complete. 2023-11-15T02:57:30.170 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:57:30.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.327+0000 7fda97f1b700 1 Processor -- start 2023-11-15T02:57:30.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.331+0000 7fda97f1b700 1 -- start start 2023-11-15T02:57:30.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.331+0000 7fda97f1b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fda901539d0 0x7fda900c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:30.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.331+0000 7fda97f1b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fda900c2670 0x7fda900c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:30.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.331+0000 7fda97f1b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda900c1530 con 0x7fda90153db0 2023-11-15T02:57:30.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.331+0000 7fda97f1b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda901541f0 con 0x7fda900c2670 2023-11-15T02:57:30.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.331+0000 7fda97f1b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda900c2ac0 con 0x7fda901539d0 2023-11-15T02:57:30.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda964b8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fda900c2670 0x7fda900c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:30.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda964b8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fda900c2670 0x7fda900c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37548/0 (socket says 172.21.15.121:37548) 2023-11-15T02:57:30.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda964b8700 1 -- 172.21.15.121:0/855954590 learned_addr learned my addr 172.21.15.121:0/855954590 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:30.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda95cb7700 1 --2- 172.21.15.121:0/855954590 >> v2:172.21.15.190:3300/0 conn(0x7fda901539d0 0x7fda900c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:30.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda964b8700 1 -- 172.21.15.121:0/855954590 >> v2:172.21.15.190:3300/0 conn(0x7fda901539d0 msgr2=0x7fda900c06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:30.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda964b8700 1 --2- 172.21.15.121:0/855954590 >> v2:172.21.15.190:3300/0 conn(0x7fda901539d0 0x7fda900c06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda964b8700 1 -- 172.21.15.121:0/855954590 >> v1:172.21.15.121:6789/0 conn(0x7fda90153db0 legacy=0x7fda900c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:30.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda964b8700 1 -- 172.21.15.121:0/855954590 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda900c2d80 con 0x7fda900c2670 2023-11-15T02:57:30.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda94cb5700 1 -- 172.21.15.121:0/855954590 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2304674034 0 0) 0x7fda900c1530 con 0x7fda90153db0 2023-11-15T02:57:30.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda964b8700 1 --2- 172.21.15.121:0/855954590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda900c2670 0x7fda900c3190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fda900c4270 tx=0x7fda880090f0).ready entity=mon.0 client_cookie=7907c035f39af600 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:30.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda94cb5700 1 -- 172.21.15.121:0/855954590 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8800b520 con 0x7fda900c2670 2023-11-15T02:57:30.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda94cb5700 1 -- 172.21.15.121:0/855954590 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8800b680 con 0x7fda900c2670 2023-11-15T02:57:30.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda94cb5700 1 -- 172.21.15.121:0/855954590 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8800ba00 con 0x7fda900c2670 2023-11-15T02:57:30.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda97f1b700 1 -- 172.21.15.121:0/855954590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda900c2670 msgr2=0x7fda900c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:30.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda97f1b700 1 --2- 172.21.15.121:0/855954590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda900c2670 0x7fda900c3190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fda900c4270 tx=0x7fda880090f0).stop 2023-11-15T02:57:30.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda97f1b700 1 -- 172.21.15.121:0/855954590 shutdown_connections 2023-11-15T02:57:30.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda97f1b700 1 --2- 172.21.15.121:0/855954590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda900c2670 0x7fda900c3190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda97f1b700 1 --2- 172.21.15.121:0/855954590 >> v2:172.21.15.190:3300/0 conn(0x7fda901539d0 0x7fda900c06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda97f1b700 1 -- 172.21.15.121:0/855954590 >> 172.21.15.121:0/855954590 conn(0x7fda90062190 msgr2=0x7fda900625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:30.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda97f1b700 1 -- 172.21.15.121:0/855954590 shutdown_connections 2023-11-15T02:57:30.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.335+0000 7fda97f1b700 1 -- 172.21.15.121:0/855954590 wait complete. 2023-11-15T02:57:30.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.339+0000 7fda97f1b700 1 Processor -- start 2023-11-15T02:57:30.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.339+0000 7fda97f1b700 1 -- start start 2023-11-15T02:57:30.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.339+0000 7fda97f1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda900c2670 0x7fda900aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:30.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.339+0000 7fda97f1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda901539d0 0x7fda9014e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:30.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.339+0000 7fda97f1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda90153db0 0x7fda90154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:30.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.339+0000 7fda97f1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda900ab1b0 con 0x7fda901539d0 2023-11-15T02:57:30.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.339+0000 7fda97f1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda90156f20 con 0x7fda90153db0 2023-11-15T02:57:30.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.339+0000 7fda97f1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda901570b0 con 0x7fda900c2670 2023-11-15T02:57:30.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.339+0000 7fda954b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda901539d0 0x7fda9014e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:30.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.339+0000 7fda95cb7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda900c2670 0x7fda900aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:30.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.339+0000 7fda954b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda901539d0 0x7fda9014e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37554/0 (socket says 172.21.15.121:37554) 2023-11-15T02:57:30.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.339+0000 7fda954b6700 1 -- 172.21.15.121:0/536993164 learned_addr learned my addr 172.21.15.121:0/536993164 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:30.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.343+0000 7fda964b8700 1 --2- 172.21.15.121:0/536993164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda90153db0 0x7fda90154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:30.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.343+0000 7fda954b6700 1 -- 172.21.15.121:0/536993164 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda900c2670 msgr2=0x7fda900aacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:30.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.343+0000 7fda954b6700 1 --2- 172.21.15.121:0/536993164 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda900c2670 0x7fda900aacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.343+0000 7fda954b6700 1 -- 172.21.15.121:0/536993164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda90153db0 msgr2=0x7fda90154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:30.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.343+0000 7fda954b6700 1 --2- 172.21.15.121:0/536993164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda90153db0 0x7fda90154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.343+0000 7fda954b6700 1 -- 172.21.15.121:0/536993164 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda88009370 con 0x7fda901539d0 2023-11-15T02:57:30.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.343+0000 7fda954b6700 1 --2- 172.21.15.121:0/536993164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda901539d0 0x7fda9014e8f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fda80001eb0 tx=0x7fda80007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:30.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.343+0000 7fda85ffb700 1 -- 172.21.15.121:0/536993164 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8000ce60 con 0x7fda901539d0 2023-11-15T02:57:30.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.343+0000 7fda85ffb700 1 -- 172.21.15.121:0/536993164 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8000f470 con 0x7fda901539d0 2023-11-15T02:57:30.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.343+0000 7fda97f1b700 1 -- 172.21.15.121:0/536993164 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda900a9da0 con 0x7fda901539d0 2023-11-15T02:57:30.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.343+0000 7fda85ffb700 1 -- 172.21.15.121:0/536993164 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8000f810 con 0x7fda901539d0 2023-11-15T02:57:30.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.343+0000 7fda97f1b700 1 -- 172.21.15.121:0/536993164 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda900aa130 con 0x7fda901539d0 2023-11-15T02:57:30.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.343+0000 7fda97f1b700 1 -- 172.21.15.121:0/536993164 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda90157220 con 0x7fda901539d0 2023-11-15T02:57:30.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.347+0000 7fda85ffb700 1 -- 172.21.15.121:0/536993164 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fda8000fb80 con 0x7fda901539d0 2023-11-15T02:57:30.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.347+0000 7fda85ffb700 1 --2- 172.21.15.121:0/536993164 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fda7405a510 0x7fda7405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:30.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.347+0000 7fda85ffb700 1 -- 172.21.15.121:0/536993164 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fda80073d90 con 0x7fda901539d0 2023-11-15T02:57:30.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.347+0000 7fda95cb7700 1 --2- 172.21.15.121:0/536993164 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fda7405a510 0x7fda7405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:30.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.347+0000 7fda95cb7700 1 --2- 172.21.15.121:0/536993164 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fda7405a510 0x7fda7405c9d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fda8c00f570 tx=0x7fda8c00b040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:30.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.355+0000 7fda85ffb700 1 -- 172.21.15.121:0/536993164 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fda800474b0 con 0x7fda901539d0 2023-11-15T02:57:30.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.555+0000 7fda97f1b700 1 -- 172.21.15.121:0/536993164 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda900a61c0 con 0x7fda901539d0 2023-11-15T02:57:30.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.555+0000 7fda85ffb700 1 -- 172.21.15.121:0/536993164 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4489 (secure 0 0 0) 0x7fda8004aad0 con 0x7fda901539d0 2023-11-15T02:57:30.562 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:30.563 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/802010010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":802010010},{"type":"v1","addr":"172.21.15.190:6835","nonce":802010010}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3991611070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3991611070},{"type":"v1","addr":"172.21.15.121:6836","nonce":3991611070}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3514688785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3514688785},{"type":"v1","addr":"172.21.15.121:6837","nonce":3514688785}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:56:51.695751+0000","modified":"2023-11-15T02:56:54.838464+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6833/742721767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":742721767},{"type":"v1","addr":"172.21.15.190:6833","nonce":742721767}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-15T02:57:30.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.559+0000 7fda97f1b700 1 -- 172.21.15.121:0/536993164 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fda7405a510 msgr2=0x7fda7405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:30.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.559+0000 7fda97f1b700 1 --2- 172.21.15.121:0/536993164 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fda7405a510 0x7fda7405c9d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fda8c00f570 tx=0x7fda8c00b040).stop 2023-11-15T02:57:30.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.559+0000 7fda97f1b700 1 -- 172.21.15.121:0/536993164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda901539d0 msgr2=0x7fda9014e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:30.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.559+0000 7fda97f1b700 1 --2- 172.21.15.121:0/536993164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda901539d0 0x7fda9014e8f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fda80001eb0 tx=0x7fda80007c70).stop 2023-11-15T02:57:30.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.559+0000 7fda97f1b700 1 -- 172.21.15.121:0/536993164 shutdown_connections 2023-11-15T02:57:30.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.559+0000 7fda97f1b700 1 --2- 172.21.15.121:0/536993164 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fda7405a510 0x7fda7405c9d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.559+0000 7fda97f1b700 1 --2- 172.21.15.121:0/536993164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda90153db0 0x7fda90154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.559+0000 7fda97f1b700 1 --2- 172.21.15.121:0/536993164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda901539d0 0x7fda9014e8f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.559+0000 7fda97f1b700 1 --2- 172.21.15.121:0/536993164 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda900c2670 0x7fda900aacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.559+0000 7fda97f1b700 1 -- 172.21.15.121:0/536993164 >> 172.21.15.121:0/536993164 conn(0x7fda90062190 msgr2=0x7fda9014f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:30.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.559+0000 7fda97f1b700 1 -- 172.21.15.121:0/536993164 shutdown_connections 2023-11-15T02:57:30.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.559+0000 7fda97f1b700 1 -- 172.21.15.121:0/536993164 wait complete. 2023-11-15T02:57:30.568 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 8 2023-11-15T02:57:30.582 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:57:30.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.739+0000 7f6b217a9700 1 Processor -- start 2023-11-15T02:57:30.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.739+0000 7f6b217a9700 1 -- start start 2023-11-15T02:57:30.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b217a9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6b1c152780 0x7f6b1c159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:30.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b217a9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6b1c0c65d0 0x7f6b1c0c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:30.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b217a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b1c0a5510 0x7f6b1c0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:30.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b217a9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1c0c3fa0 con 0x7f6b1c152780 2023-11-15T02:57:30.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b217a9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1c159630 con 0x7f6b1c0c65d0 2023-11-15T02:57:30.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b217a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0c30b0 con 0x7f6b1c0a5510 2023-11-15T02:57:30.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b1affd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6b1c152780 0x7f6b1c159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:30.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b1affd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6b1c152780 0x7f6b1c159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37568/0 (socket says 172.21.15.121:37568) 2023-11-15T02:57:30.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b1affd700 1 -- 172.21.15.121:0/3308915270 learned_addr learned my addr 172.21.15.121:0/3308915270 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:30.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b1a7fc700 1 --2- 172.21.15.121:0/3308915270 >> v2:172.21.15.190:3300/0 conn(0x7f6b1c0c65d0 0x7f6b1c0c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:30.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b1b7fe700 1 --2- 172.21.15.121:0/3308915270 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b1c0a5510 0x7f6b1c0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:30.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b1affd700 1 -- 172.21.15.121:0/3308915270 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b1c0a5510 msgr2=0x7f6b1c0a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:30.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b1affd700 1 --2- 172.21.15.121:0/3308915270 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b1c0a5510 0x7f6b1c0a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b1affd700 1 -- 172.21.15.121:0/3308915270 >> v2:172.21.15.190:3300/0 conn(0x7f6b1c0c65d0 msgr2=0x7f6b1c0c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:30.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b1affd700 1 --2- 172.21.15.121:0/3308915270 >> v2:172.21.15.190:3300/0 conn(0x7f6b1c0c65d0 0x7f6b1c0c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b1affd700 1 -- 172.21.15.121:0/3308915270 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c0c3390 con 0x7f6b1c152780 2023-11-15T02:57:30.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b1affd700 1 --2- 172.21.15.121:0/3308915270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b1c152780 0x7f6b1c159140 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f6b1c063d40 tx=0x7f6b0c0090f0).ready entity=mon.0 client_cookie=fe3c4564d4f2951d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:30.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b19ffb700 1 -- 172.21.15.121:0/3308915270 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00b600 con 0x7f6b1c152780 2023-11-15T02:57:30.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b19ffb700 1 -- 172.21.15.121:0/3308915270 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0c00b760 con 0x7f6b1c152780 2023-11-15T02:57:30.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b19ffb700 1 -- 172.21.15.121:0/3308915270 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00bae0 con 0x7f6b1c152780 2023-11-15T02:57:30.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b217a9700 1 -- 172.21.15.121:0/3308915270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b1c152780 msgr2=0x7f6b1c159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:30.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b217a9700 1 --2- 172.21.15.121:0/3308915270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b1c152780 0x7f6b1c159140 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f6b1c063d40 tx=0x7f6b0c0090f0).stop 2023-11-15T02:57:30.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b217a9700 1 -- 172.21.15.121:0/3308915270 shutdown_connections 2023-11-15T02:57:30.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b217a9700 1 --2- 172.21.15.121:0/3308915270 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b1c0a5510 0x7f6b1c0a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b217a9700 1 --2- 172.21.15.121:0/3308915270 >> v2:172.21.15.190:3300/0 conn(0x7f6b1c0c65d0 0x7f6b1c0c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b217a9700 1 --2- 172.21.15.121:0/3308915270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b1c152780 0x7f6b1c159140 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.743+0000 7f6b217a9700 1 -- 172.21.15.121:0/3308915270 >> 172.21.15.121:0/3308915270 conn(0x7f6b1c062190 msgr2=0x7f6b1c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:30.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.747+0000 7f6b217a9700 1 -- 172.21.15.121:0/3308915270 shutdown_connections 2023-11-15T02:57:30.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.747+0000 7f6b217a9700 1 -- 172.21.15.121:0/3308915270 wait complete. 2023-11-15T02:57:30.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.747+0000 7f6b217a9700 1 Processor -- start 2023-11-15T02:57:30.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.747+0000 7f6b217a9700 1 -- start start 2023-11-15T02:57:30.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b217a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b1c0a5510 0x7f6b1c14f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:30.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b217a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b1c0c65d0 0x7f6b1c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:30.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b217a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b1c152780 0x7f6b1c136820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:30.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b217a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0ab2d0 con 0x7f6b1c0a5510 2023-11-15T02:57:30.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b217a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c14fcc0 con 0x7f6b1c0c65d0 2023-11-15T02:57:30.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b217a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0aa660 con 0x7f6b1c152780 2023-11-15T02:57:30.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b1affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b1c0a5510 0x7f6b1c14f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:30.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b1affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b1c0a5510 0x7f6b1c14f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37574/0 (socket says 172.21.15.121:37574) 2023-11-15T02:57:30.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b1affd700 1 -- 172.21.15.121:0/366754638 learned_addr learned my addr 172.21.15.121:0/366754638 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:30.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b1b7fe700 1 --2- 172.21.15.121:0/366754638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b1c152780 0x7f6b1c136820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:30.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b1a7fc700 1 --2- 172.21.15.121:0/366754638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b1c0c65d0 0x7f6b1c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:30.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b1affd700 1 -- 172.21.15.121:0/366754638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b1c152780 msgr2=0x7f6b1c136820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:30.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b1affd700 1 --2- 172.21.15.121:0/366754638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b1c152780 0x7f6b1c136820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b1affd700 1 -- 172.21.15.121:0/366754638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b1c0c65d0 msgr2=0x7f6b1c0abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:30.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b1affd700 1 --2- 172.21.15.121:0/366754638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b1c0c65d0 0x7f6b1c0abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b1affd700 1 -- 172.21.15.121:0/366754638 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c009370 con 0x7f6b1c0a5510 2023-11-15T02:57:30.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b1affd700 1 --2- 172.21.15.121:0/366754638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b1c0a5510 0x7f6b1c14f7d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f6b1c064f10 tx=0x7f6b0c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:30.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b02ffd700 1 -- 172.21.15.121:0/366754638 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00bea0 con 0x7f6b1c0a5510 2023-11-15T02:57:30.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b217a9700 1 -- 172.21.15.121:0/366754638 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1c0ab4b0 con 0x7f6b1c0a5510 2023-11-15T02:57:30.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b217a9700 1 -- 172.21.15.121:0/366754638 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1c0aba70 con 0x7f6b1c0a5510 2023-11-15T02:57:30.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b02ffd700 1 -- 172.21.15.121:0/366754638 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0c006b10 con 0x7f6b1c0a5510 2023-11-15T02:57:30.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.751+0000 7f6b02ffd700 1 -- 172.21.15.121:0/366754638 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00bae0 con 0x7f6b1c0a5510 2023-11-15T02:57:30.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.755+0000 7f6b217a9700 1 -- 172.21.15.121:0/366754638 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6afc005120 con 0x7f6b1c0a5510 2023-11-15T02:57:30.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.755+0000 7f6b02ffd700 1 -- 172.21.15.121:0/366754638 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6b0c027460 con 0x7f6b1c0a5510 2023-11-15T02:57:30.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.755+0000 7f6b02ffd700 1 --2- 172.21.15.121:0/366754638 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6b0805a510 0x7f6b0805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:30.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.755+0000 7f6b02ffd700 1 -- 172.21.15.121:0/366754638 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6b0c076210 con 0x7f6b1c0a5510 2023-11-15T02:57:30.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.755+0000 7f6b1a7fc700 1 --2- 172.21.15.121:0/366754638 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6b0805a510 0x7f6b0805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:30.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.755+0000 7f6b1a7fc700 1 --2- 172.21.15.121:0/366754638 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6b0805a510 0x7f6b0805c9d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f6b1c0a8c70 tx=0x7f6b10008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:30.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.763+0000 7f6b02ffd700 1 -- 172.21.15.121:0/366754638 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6b0c0498d0 con 0x7f6b1c0a5510 2023-11-15T02:57:30.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.923+0000 7f6b217a9700 1 -- 172.21.15.121:0/366754638 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6afc0053d0 con 0x7f6b1c0a5510 2023-11-15T02:57:30.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.927+0000 7f6b02ffd700 1 -- 172.21.15.121:0/366754638 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f6b0c04cef0 con 0x7f6b1c0a5510 2023-11-15T02:57:30.932 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:30.933 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":21,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:52.702668+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:56:49.726353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-15T02:56:50.688098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:57:30.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.931+0000 7f6b217a9700 1 -- 172.21.15.121:0/366754638 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6b0805a510 msgr2=0x7f6b0805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:30.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.931+0000 7f6b217a9700 1 --2- 172.21.15.121:0/366754638 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6b0805a510 0x7f6b0805c9d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f6b1c0a8c70 tx=0x7f6b10008040).stop 2023-11-15T02:57:30.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.931+0000 7f6b217a9700 1 -- 172.21.15.121:0/366754638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b1c0a5510 msgr2=0x7f6b1c14f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:30.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.931+0000 7f6b217a9700 1 --2- 172.21.15.121:0/366754638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b1c0a5510 0x7f6b1c14f7d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f6b1c064f10 tx=0x7f6b0c017000).stop 2023-11-15T02:57:30.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.931+0000 7f6b217a9700 1 -- 172.21.15.121:0/366754638 shutdown_connections 2023-11-15T02:57:30.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.931+0000 7f6b217a9700 1 --2- 172.21.15.121:0/366754638 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6b0805a510 0x7f6b0805c9d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.931+0000 7f6b217a9700 1 --2- 172.21.15.121:0/366754638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b1c152780 0x7f6b1c136820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.931+0000 7f6b217a9700 1 --2- 172.21.15.121:0/366754638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b1c0c65d0 0x7f6b1c0abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.931+0000 7f6b217a9700 1 --2- 172.21.15.121:0/366754638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b1c0a5510 0x7f6b1c14f7d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:30.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.931+0000 7f6b217a9700 1 -- 172.21.15.121:0/366754638 >> 172.21.15.121:0/366754638 conn(0x7f6b1c062190 msgr2=0x7f6b1c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:30.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.931+0000 7f6b217a9700 1 -- 172.21.15.121:0/366754638 shutdown_connections 2023-11-15T02:57:30.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:30.931+0000 7f6b217a9700 1 -- 172.21.15.121:0/366754638 wait complete. 2023-11-15T02:57:30.956 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-15T02:57:30.957 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:57:31.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.107+0000 7f019c4f1700 1 Processor -- start 2023-11-15T02:57:31.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.111+0000 7f019c4f1700 1 -- start start 2023-11-15T02:57:31.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.111+0000 7f019c4f1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0194150870 0x7f01940c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:31.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.111+0000 7f019c4f1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0194153d20 0x7f01940c3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:31.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.111+0000 7f019c4f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01940c6730 0x7f0194150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:31.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.111+0000 7f019c4f1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01940c3b70 con 0x7f0194150870 2023-11-15T02:57:31.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.111+0000 7f019c4f1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01940c6350 con 0x7f0194153d20 2023-11-15T02:57:31.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.111+0000 7f019c4f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01941541b0 con 0x7f01940c6730 2023-11-15T02:57:31.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.111+0000 7f019a28d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0194150870 0x7f01940c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:31.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.111+0000 7f019a28d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0194150870 0x7f01940c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37590/0 (socket says 172.21.15.121:37590) 2023-11-15T02:57:31.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.111+0000 7f019a28d700 1 -- 172.21.15.121:0/2909665374 learned_addr learned my addr 172.21.15.121:0/2909665374 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:31.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.111+0000 7f019aa8e700 1 --2- 172.21.15.121:0/2909665374 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01940c6730 0x7f0194150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:31.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.111+0000 7f0199a8c700 1 --2- 172.21.15.121:0/2909665374 >> v2:172.21.15.190:3300/0 conn(0x7f0194153d20 0x7f01940c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:31.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.111+0000 7f019a28d700 1 -- 172.21.15.121:0/2909665374 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01940c6730 msgr2=0x7f0194150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:31.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.111+0000 7f019a28d700 1 --2- 172.21.15.121:0/2909665374 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01940c6730 0x7f0194150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.111+0000 7f019a28d700 1 -- 172.21.15.121:0/2909665374 >> v2:172.21.15.190:3300/0 conn(0x7f0194153d20 msgr2=0x7f01940c3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:31.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.111+0000 7f019a28d700 1 --2- 172.21.15.121:0/2909665374 >> v2:172.21.15.190:3300/0 conn(0x7f0194153d20 0x7f01940c3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.111+0000 7f019a28d700 1 -- 172.21.15.121:0/2909665374 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0194154490 con 0x7f0194150870 2023-11-15T02:57:31.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.111+0000 7f019a28d700 1 --2- 172.21.15.121:0/2909665374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0194150870 0x7f01940c3190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f0194063170 tx=0x7f01900090f0).ready entity=mon.0 client_cookie=19cf3dab8276f233 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:31.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.111+0000 7f019928b700 1 -- 172.21.15.121:0/2909665374 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019000b520 con 0x7f0194150870 2023-11-15T02:57:31.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.115+0000 7f019928b700 1 -- 172.21.15.121:0/2909665374 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019000b680 con 0x7f0194150870 2023-11-15T02:57:31.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.115+0000 7f019928b700 1 -- 172.21.15.121:0/2909665374 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019000ba00 con 0x7f0194150870 2023-11-15T02:57:31.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.115+0000 7f019c4f1700 1 -- 172.21.15.121:0/2909665374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0194150870 msgr2=0x7f01940c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:31.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.115+0000 7f019c4f1700 1 --2- 172.21.15.121:0/2909665374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0194150870 0x7f01940c3190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f0194063170 tx=0x7f01900090f0).stop 2023-11-15T02:57:31.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.115+0000 7f019c4f1700 1 -- 172.21.15.121:0/2909665374 shutdown_connections 2023-11-15T02:57:31.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.115+0000 7f019c4f1700 1 --2- 172.21.15.121:0/2909665374 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01940c6730 0x7f0194150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.115+0000 7f019c4f1700 1 --2- 172.21.15.121:0/2909665374 >> v2:172.21.15.190:3300/0 conn(0x7f0194153d20 0x7f01940c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.115+0000 7f019c4f1700 1 --2- 172.21.15.121:0/2909665374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0194150870 0x7f01940c3190 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.115+0000 7f019c4f1700 1 -- 172.21.15.121:0/2909665374 >> 172.21.15.121:0/2909665374 conn(0x7f0194062190 msgr2=0x7f01940625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:31.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.115+0000 7f019c4f1700 1 -- 172.21.15.121:0/2909665374 shutdown_connections 2023-11-15T02:57:31.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.115+0000 7f019c4f1700 1 -- 172.21.15.121:0/2909665374 wait complete. 2023-11-15T02:57:31.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.115+0000 7f019c4f1700 1 Processor -- start 2023-11-15T02:57:31.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.115+0000 7f019c4f1700 1 -- start start 2023-11-15T02:57:31.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f019c4f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01940c6730 0x7f01940bb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:31.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f019c4f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0194150870 0x7f0194141250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:31.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f019c4f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01940a9830 0x7f01941458f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:31.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f019c4f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01940ab390 con 0x7f0194150870 2023-11-15T02:57:31.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f019c4f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0194143a30 con 0x7f01940c6730 2023-11-15T02:57:31.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f019c4f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01940ab650 con 0x7f01940a9830 2023-11-15T02:57:31.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f0199a8c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0194150870 0x7f0194141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:31.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f0199a8c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0194150870 0x7f0194141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37592/0 (socket says 172.21.15.121:37592) 2023-11-15T02:57:31.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f0199a8c700 1 -- 172.21.15.121:0/4087139677 learned_addr learned my addr 172.21.15.121:0/4087139677 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:31.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f019a28d700 1 --2- 172.21.15.121:0/4087139677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01940c6730 0x7f01940bb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:31.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f019aa8e700 1 --2- 172.21.15.121:0/4087139677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01940a9830 0x7f01941458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:31.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f0199a8c700 1 -- 172.21.15.121:0/4087139677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01940a9830 msgr2=0x7f01941458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:31.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f0199a8c700 1 --2- 172.21.15.121:0/4087139677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01940a9830 0x7f01941458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f0199a8c700 1 -- 172.21.15.121:0/4087139677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01940c6730 msgr2=0x7f01940bb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:31.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f0199a8c700 1 --2- 172.21.15.121:0/4087139677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01940c6730 0x7f01940bb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f0199a8c700 1 -- 172.21.15.121:0/4087139677 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0190009370 con 0x7f0194150870 2023-11-15T02:57:31.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f019a28d700 1 --2- 172.21.15.121:0/4087139677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01940c6730 0x7f01940bb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:31.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f0199a8c700 1 --2- 172.21.15.121:0/4087139677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0194150870 0x7f0194141250 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f018400d8c0 tx=0x7f0184008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:31.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f018a7fc700 1 -- 172.21.15.121:0/4087139677 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01840095b0 con 0x7f0194150870 2023-11-15T02:57:31.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f018a7fc700 1 -- 172.21.15.121:0/4087139677 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0184009710 con 0x7f0194150870 2023-11-15T02:57:31.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f019c4f1700 1 -- 172.21.15.121:0/4087139677 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0194141740 con 0x7f0194150870 2023-11-15T02:57:31.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f018a7fc700 1 -- 172.21.15.121:0/4087139677 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184009a90 con 0x7f0194150870 2023-11-15T02:57:31.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.119+0000 7f019c4f1700 1 -- 172.21.15.121:0/4087139677 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0194143750 con 0x7f0194150870 2023-11-15T02:57:31.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.123+0000 7f019c4f1700 1 -- 172.21.15.121:0/4087139677 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f019414ffb0 con 0x7f0194150870 2023-11-15T02:57:31.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.123+0000 7f018a7fc700 1 -- 172.21.15.121:0/4087139677 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0184013480 con 0x7f0194150870 2023-11-15T02:57:31.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.127+0000 7f018a7fc700 1 --2- 172.21.15.121:0/4087139677 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f017805a590 0x7f017805ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:31.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.127+0000 7f018a7fc700 1 -- 172.21.15.121:0/4087139677 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f01840743e0 con 0x7f0194150870 2023-11-15T02:57:31.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.127+0000 7f019a28d700 1 --2- 172.21.15.121:0/4087139677 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f017805a590 0x7f017805ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:31.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.127+0000 7f019a28d700 1 --2- 172.21.15.121:0/4087139677 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f017805a590 0x7f017805ca50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f0190009130 tx=0x7f0190015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:31.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.131+0000 7f018a7fc700 1 -- 172.21.15.121:0/4087139677 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0184047c00 con 0x7f0194150870 2023-11-15T02:57:31.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.327+0000 7f019c4f1700 1 -- 172.21.15.121:0/4087139677 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f019406c150 con 0x7f0194150870 2023-11-15T02:57:31.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.331+0000 7f018a7fc700 1 -- 172.21.15.121:0/4087139677 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4489 (secure 0 0 0) 0x7f018404b220 con 0x7f0194150870 2023-11-15T02:57:31.336 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:31.336 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/802010010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":802010010},{"type":"v1","addr":"172.21.15.190:6835","nonce":802010010}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3991611070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3991611070},{"type":"v1","addr":"172.21.15.121:6836","nonce":3991611070}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3514688785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3514688785},{"type":"v1","addr":"172.21.15.121:6837","nonce":3514688785}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:56:51.695751+0000","modified":"2023-11-15T02:56:54.838464+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6833/742721767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":742721767},{"type":"v1","addr":"172.21.15.190:6833","nonce":742721767}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-15T02:57:31.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.331+0000 7f019c4f1700 1 -- 172.21.15.121:0/4087139677 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f017805a590 msgr2=0x7f017805ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:31.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.331+0000 7f019c4f1700 1 --2- 172.21.15.121:0/4087139677 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f017805a590 0x7f017805ca50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f0190009130 tx=0x7f0190015000).stop 2023-11-15T02:57:31.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.331+0000 7f019c4f1700 1 -- 172.21.15.121:0/4087139677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0194150870 msgr2=0x7f0194141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:31.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.331+0000 7f019c4f1700 1 --2- 172.21.15.121:0/4087139677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0194150870 0x7f0194141250 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f018400d8c0 tx=0x7f0184008040).stop 2023-11-15T02:57:31.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.335+0000 7f019c4f1700 1 -- 172.21.15.121:0/4087139677 shutdown_connections 2023-11-15T02:57:31.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.335+0000 7f019c4f1700 1 --2- 172.21.15.121:0/4087139677 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f017805a590 0x7f017805ca50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.335+0000 7f019c4f1700 1 --2- 172.21.15.121:0/4087139677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01940a9830 0x7f01941458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.335+0000 7f019c4f1700 1 --2- 172.21.15.121:0/4087139677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0194150870 0x7f0194141250 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.335+0000 7f019c4f1700 1 --2- 172.21.15.121:0/4087139677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01940c6730 0x7f01940bb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.335+0000 7f019c4f1700 1 -- 172.21.15.121:0/4087139677 >> 172.21.15.121:0/4087139677 conn(0x7f0194062190 msgr2=0x7f0194065750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:31.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.335+0000 7f019c4f1700 1 -- 172.21.15.121:0/4087139677 shutdown_connections 2023-11-15T02:57:31.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.335+0000 7f019c4f1700 1 -- 172.21.15.121:0/4087139677 wait complete. 2023-11-15T02:57:31.342 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 8 2023-11-15T02:57:31.356 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:57:31.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.507+0000 7f1e65839700 1 Processor -- start 2023-11-15T02:57:31.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.507+0000 7f1e65839700 1 -- start start 2023-11-15T02:57:31.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.507+0000 7f1e65839700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1e600cabd0 0x7f1e600cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:31.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.507+0000 7f1e65839700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e600ad650 con 0x7f1e600ad0f0 2023-11-15T02:57:31.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.507+0000 7f1e65839700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e60065fc0 con 0x7f1e600acc90 2023-11-15T02:57:31.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.507+0000 7f1e65839700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e600662a0 con 0x7f1e600cabd0 2023-11-15T02:57:31.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.507+0000 7f1e64837700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1e600cabd0 0x7f1e600cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:31.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.511+0000 7f1e65038700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f1e600ad0f0 0x7f1e600ae190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39220/0 (socket says 172.21.15.121:39220) 2023-11-15T02:57:31.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.511+0000 7f1e65038700 1 -- 172.21.15.121:0/3576130732 learned_addr learned my addr 172.21.15.121:0/3576130732 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:31.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.511+0000 7f1e64837700 1 -- 172.21.15.121:0/3576130732 >> v1:172.21.15.190:6789/0 conn(0x7f1e600acc90 legacy=0x7f1e60066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:31.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.511+0000 7f1e64837700 1 -- 172.21.15.121:0/3576130732 >> v1:172.21.15.121:6789/0 conn(0x7f1e600ad0f0 legacy=0x7f1e600ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:31.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.511+0000 7f1e64837700 1 -- 172.21.15.121:0/3576130732 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e60066580 con 0x7f1e600cabd0 2023-11-15T02:57:31.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.511+0000 7f1e64837700 1 --2- 172.21.15.121:0/3576130732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e600cabd0 0x7f1e600cafb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f1e54009760 tx=0x7f1e54009060).ready entity=mon.0 client_cookie=582ac7776074859 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:31.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.511+0000 7f1e5f7fe700 1 -- 172.21.15.121:0/3576130732 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5400b580 con 0x7f1e600cabd0 2023-11-15T02:57:31.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.511+0000 7f1e5f7fe700 1 -- 172.21.15.121:0/3576130732 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5400b6e0 con 0x7f1e600cabd0 2023-11-15T02:57:31.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.511+0000 7f1e5f7fe700 1 -- 172.21.15.121:0/3576130732 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5400ba60 con 0x7f1e600cabd0 2023-11-15T02:57:31.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.511+0000 7f1e65839700 1 -- 172.21.15.121:0/3576130732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e600cabd0 msgr2=0x7f1e600cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:31.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.511+0000 7f1e65839700 1 --2- 172.21.15.121:0/3576130732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e600cabd0 0x7f1e600cafb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f1e54009760 tx=0x7f1e54009060).stop 2023-11-15T02:57:31.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.511+0000 7f1e65839700 1 -- 172.21.15.121:0/3576130732 shutdown_connections 2023-11-15T02:57:31.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.511+0000 7f1e65839700 1 --2- 172.21.15.121:0/3576130732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e600cabd0 0x7f1e600cafb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.511+0000 7f1e65839700 1 -- 172.21.15.121:0/3576130732 >> 172.21.15.121:0/3576130732 conn(0x7f1e60062190 msgr2=0x7f1e600625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:31.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.511+0000 7f1e65839700 1 -- 172.21.15.121:0/3576130732 shutdown_connections 2023-11-15T02:57:31.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.511+0000 7f1e65839700 1 -- 172.21.15.121:0/3576130732 wait complete. 2023-11-15T02:57:31.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e65839700 1 Processor -- start 2023-11-15T02:57:31.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e65839700 1 -- start start 2023-11-15T02:57:31.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e65839700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e600acc90 0x7f1e601493c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:31.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e65839700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e600ad0f0 0x7f1e600837d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:31.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e65839700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e601498b0 0x7f1e60083cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:31.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e65839700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e60156fa0 con 0x7f1e600acc90 2023-11-15T02:57:31.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e65839700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e600841b0 con 0x7f1e600ad0f0 2023-11-15T02:57:31.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e65839700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e60082a70 con 0x7f1e601498b0 2023-11-15T02:57:31.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e64837700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e600acc90 0x7f1e601493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:31.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e64837700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e600acc90 0x7f1e601493c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37600/0 (socket says 172.21.15.121:37600) 2023-11-15T02:57:31.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e64837700 1 -- 172.21.15.121:0/3024444687 learned_addr learned my addr 172.21.15.121:0/3024444687 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:31.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e5ffff700 1 --2- 172.21.15.121:0/3024444687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e600ad0f0 0x7f1e600837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:31.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e65038700 1 --2- 172.21.15.121:0/3024444687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e601498b0 0x7f1e60083cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:31.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e64837700 1 -- 172.21.15.121:0/3024444687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e601498b0 msgr2=0x7f1e60083cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:31.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e64837700 1 --2- 172.21.15.121:0/3024444687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e601498b0 0x7f1e60083cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e64837700 1 -- 172.21.15.121:0/3024444687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e600ad0f0 msgr2=0x7f1e600837d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:31.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e64837700 1 --2- 172.21.15.121:0/3024444687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e600ad0f0 0x7f1e600837d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e64837700 1 -- 172.21.15.121:0/3024444687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e540092a0 con 0x7f1e600acc90 2023-11-15T02:57:31.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e65038700 1 --2- 172.21.15.121:0/3024444687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e601498b0 0x7f1e60083cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:57:31.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e64837700 1 --2- 172.21.15.121:0/3024444687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e600acc90 0x7f1e601493c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f1e60064f20 tx=0x7f1e54017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:31.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e5cff9700 1 -- 172.21.15.121:0/3024444687 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5400b850 con 0x7f1e600acc90 2023-11-15T02:57:31.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e5cff9700 1 -- 172.21.15.121:0/3024444687 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5400b9b0 con 0x7f1e600acc90 2023-11-15T02:57:31.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e65839700 1 -- 172.21.15.121:0/3024444687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e600843b0 con 0x7f1e600acc90 2023-11-15T02:57:31.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e5cff9700 1 -- 172.21.15.121:0/3024444687 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5401b620 con 0x7f1e600acc90 2023-11-15T02:57:31.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.515+0000 7f1e65839700 1 -- 172.21.15.121:0/3024444687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e60082420 con 0x7f1e600acc90 2023-11-15T02:57:31.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.519+0000 7f1e65839700 1 -- 172.21.15.121:0/3024444687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e6006bcf0 con 0x7f1e600acc90 2023-11-15T02:57:31.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.519+0000 7f1e5cff9700 1 -- 172.21.15.121:0/3024444687 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1e5401b9d0 con 0x7f1e600acc90 2023-11-15T02:57:31.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.519+0000 7f1e5cff9700 1 --2- 172.21.15.121:0/3024444687 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1e4c05a4e0 0x7f1e4c05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:31.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.519+0000 7f1e5cff9700 1 -- 172.21.15.121:0/3024444687 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1e540760a0 con 0x7f1e600acc90 2023-11-15T02:57:31.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.523+0000 7f1e5ffff700 1 --2- 172.21.15.121:0/3024444687 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1e4c05a4e0 0x7f1e4c05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:31.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.523+0000 7f1e5ffff700 1 --2- 172.21.15.121:0/3024444687 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1e4c05a4e0 0x7f1e4c05c9a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f1e60157260 tx=0x7f1e50009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:31.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.527+0000 7f1e5cff9700 1 -- 172.21.15.121:0/3024444687 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1e54049880 con 0x7f1e600acc90 2023-11-15T02:57:31.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.723+0000 7f1e65839700 1 -- 172.21.15.121:0/3024444687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e60066860 con 0x7f1e600acc90 2023-11-15T02:57:31.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.723+0000 7f1e5cff9700 1 -- 172.21.15.121:0/3024444687 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4489 (secure 0 0 0) 0x7f1e5404cea0 con 0x7f1e600acc90 2023-11-15T02:57:31.729 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:31.729 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/802010010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":802010010},{"type":"v1","addr":"172.21.15.190:6835","nonce":802010010}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3991611070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3991611070},{"type":"v1","addr":"172.21.15.121:6836","nonce":3991611070}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3514688785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3514688785},{"type":"v1","addr":"172.21.15.121:6837","nonce":3514688785}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:56:51.695751+0000","modified":"2023-11-15T02:56:54.838464+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6833/742721767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":742721767},{"type":"v1","addr":"172.21.15.190:6833","nonce":742721767}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-15T02:57:31.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.727+0000 7f1e65839700 1 -- 172.21.15.121:0/3024444687 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1e4c05a4e0 msgr2=0x7f1e4c05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:31.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.727+0000 7f1e65839700 1 --2- 172.21.15.121:0/3024444687 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1e4c05a4e0 0x7f1e4c05c9a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f1e60157260 tx=0x7f1e50009040).stop 2023-11-15T02:57:31.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.727+0000 7f1e65839700 1 -- 172.21.15.121:0/3024444687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e600acc90 msgr2=0x7f1e601493c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:31.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.727+0000 7f1e65839700 1 --2- 172.21.15.121:0/3024444687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e600acc90 0x7f1e601493c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f1e60064f20 tx=0x7f1e54017000).stop 2023-11-15T02:57:31.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.727+0000 7f1e65839700 1 -- 172.21.15.121:0/3024444687 shutdown_connections 2023-11-15T02:57:31.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.727+0000 7f1e65839700 1 --2- 172.21.15.121:0/3024444687 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1e4c05a4e0 0x7f1e4c05c9a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.727+0000 7f1e65839700 1 --2- 172.21.15.121:0/3024444687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e601498b0 0x7f1e60083cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.727+0000 7f1e65839700 1 --2- 172.21.15.121:0/3024444687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e600ad0f0 0x7f1e600837d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.727+0000 7f1e65839700 1 --2- 172.21.15.121:0/3024444687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e600acc90 0x7f1e601493c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.727+0000 7f1e65839700 1 -- 172.21.15.121:0/3024444687 >> 172.21.15.121:0/3024444687 conn(0x7f1e60062190 msgr2=0x7f1e600ae560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:31.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.727+0000 7f1e65839700 1 -- 172.21.15.121:0/3024444687 shutdown_connections 2023-11-15T02:57:31.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.727+0000 7f1e65839700 1 -- 172.21.15.121:0/3024444687 wait complete. 2023-11-15T02:57:31.734 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 8 2023-11-15T02:57:31.749 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:57:31.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.895+0000 7f65813db700 1 Processor -- start 2023-11-15T02:57:31.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.895+0000 7f65813db700 1 -- start start 2023-11-15T02:57:31.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.895+0000 7f65813db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f657c153bb0 0x7f657c0a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:31.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.895+0000 7f65813db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f657c0aa020 0x7f657c067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:31.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.895+0000 7f65813db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657c0cb5c0 con 0x7f657c067f50 2023-11-15T02:57:31.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.895+0000 7f65813db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f657c0aa6f0 con 0x7f657c153bb0 2023-11-15T02:57:31.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.895+0000 7f65813db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0c5900 con 0x7f657c0aa020 2023-11-15T02:57:31.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f657affd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f657c153bb0 0x7f657c0a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:31.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f657affd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f657c153bb0 0x7f657c0a9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37602/0 (socket says 172.21.15.121:37602) 2023-11-15T02:57:31.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f657b7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f657c067f50 0x7f657c1578a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39230/0 (socket says 172.21.15.121:39230) 2023-11-15T02:57:31.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f657affd700 1 -- 172.21.15.121:0/2139423030 learned_addr learned my addr 172.21.15.121:0/2139423030 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:31.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f657a7fc700 1 --2- 172.21.15.121:0/2139423030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f657c0aa020 0x7f657c067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:31.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f657affd700 1 -- 172.21.15.121:0/2139423030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f657c0aa020 msgr2=0x7f657c067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:31.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f657affd700 1 --2- 172.21.15.121:0/2139423030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f657c0aa020 0x7f657c067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f657affd700 1 -- 172.21.15.121:0/2139423030 >> v1:172.21.15.121:6789/0 conn(0x7f657c067f50 legacy=0x7f657c1578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:31.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f6579ffb700 1 -- 172.21.15.121:0/2139423030 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4208657351 0 0) 0x7f657c0cb5c0 con 0x7f657c067f50 2023-11-15T02:57:31.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f657affd700 1 -- 172.21.15.121:0/2139423030 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c0683e0 con 0x7f657c153bb0 2023-11-15T02:57:31.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f657affd700 1 --2- 172.21.15.121:0/2139423030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f657c153bb0 0x7f657c0a9b30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f656c006fd0 tx=0x7f656c0090f0).ready entity=mon.0 client_cookie=edd6c1e3a54f3c5c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:31.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f6579ffb700 1 -- 172.21.15.121:0/2139423030 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c00b580 con 0x7f657c153bb0 2023-11-15T02:57:31.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f6579ffb700 1 -- 172.21.15.121:0/2139423030 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656c00b6e0 con 0x7f657c153bb0 2023-11-15T02:57:31.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f6579ffb700 1 -- 172.21.15.121:0/2139423030 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c00ba60 con 0x7f657c153bb0 2023-11-15T02:57:31.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f65813db700 1 -- 172.21.15.121:0/2139423030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f657c153bb0 msgr2=0x7f657c0a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:31.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f65813db700 1 --2- 172.21.15.121:0/2139423030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f657c153bb0 0x7f657c0a9b30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f656c006fd0 tx=0x7f656c0090f0).stop 2023-11-15T02:57:31.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f65813db700 1 -- 172.21.15.121:0/2139423030 shutdown_connections 2023-11-15T02:57:31.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f65813db700 1 --2- 172.21.15.121:0/2139423030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f657c0aa020 0x7f657c067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f65813db700 1 --2- 172.21.15.121:0/2139423030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f657c153bb0 0x7f657c0a9b30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f65813db700 1 -- 172.21.15.121:0/2139423030 >> 172.21.15.121:0/2139423030 conn(0x7f657c058c50 msgr2=0x7f657c059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:31.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f65813db700 1 -- 172.21.15.121:0/2139423030 shutdown_connections 2023-11-15T02:57:31.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.899+0000 7f65813db700 1 -- 172.21.15.121:0/2139423030 wait complete. 2023-11-15T02:57:31.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.903+0000 7f65813db700 1 Processor -- start 2023-11-15T02:57:31.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.903+0000 7f65813db700 1 -- start start 2023-11-15T02:57:31.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.903+0000 7f65813db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f657c067f50 0x7f657c0ab420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:31.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.903+0000 7f65813db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f657c0aa020 0x7f657c152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:31.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.903+0000 7f65813db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f657c153bb0 0x7f657c15b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:31.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.903+0000 7f65813db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c153050 con 0x7f657c067f50 2023-11-15T02:57:31.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.903+0000 7f65813db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0ac960 con 0x7f657c0aa020 2023-11-15T02:57:31.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.903+0000 7f65813db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0ac030 con 0x7f657c153bb0 2023-11-15T02:57:31.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.903+0000 7f657affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f657c067f50 0x7f657c0ab420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:31.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.903+0000 7f657affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f657c067f50 0x7f657c0ab420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37612/0 (socket says 172.21.15.121:37612) 2023-11-15T02:57:31.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.903+0000 7f657affd700 1 -- 172.21.15.121:0/2826410032 learned_addr learned my addr 172.21.15.121:0/2826410032 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:31.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.903+0000 7f657a7fc700 1 --2- 172.21.15.121:0/2826410032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f657c0aa020 0x7f657c152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:31.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.903+0000 7f657b7fe700 1 --2- 172.21.15.121:0/2826410032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f657c153bb0 0x7f657c15b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:31.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.903+0000 7f657affd700 1 -- 172.21.15.121:0/2826410032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f657c153bb0 msgr2=0x7f657c15b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:31.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.903+0000 7f657affd700 1 --2- 172.21.15.121:0/2826410032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f657c153bb0 0x7f657c15b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.903+0000 7f657affd700 1 -- 172.21.15.121:0/2826410032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f657c0aa020 msgr2=0x7f657c152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:31.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.903+0000 7f657affd700 1 --2- 172.21.15.121:0/2826410032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f657c0aa020 0x7f657c152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:31.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.903+0000 7f657affd700 1 -- 172.21.15.121:0/2826410032 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c009370 con 0x7f657c067f50 2023-11-15T02:57:31.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.907+0000 7f657affd700 1 --2- 172.21.15.121:0/2826410032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f657c067f50 0x7f657c0ab420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f656c009130 tx=0x7f656c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:31.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.907+0000 7f6562ffd700 1 -- 172.21.15.121:0/2826410032 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c000bc0 con 0x7f657c067f50 2023-11-15T02:57:31.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.907+0000 7f6562ffd700 1 -- 172.21.15.121:0/2826410032 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656c00b850 con 0x7f657c067f50 2023-11-15T02:57:31.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.907+0000 7f65813db700 1 -- 172.21.15.121:0/2826410032 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657c0ac310 con 0x7f657c067f50 2023-11-15T02:57:31.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.907+0000 7f6562ffd700 1 -- 172.21.15.121:0/2826410032 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c000d20 con 0x7f657c067f50 2023-11-15T02:57:31.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.907+0000 7f65813db700 1 -- 172.21.15.121:0/2826410032 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657c152cf0 con 0x7f657c067f50 2023-11-15T02:57:31.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.907+0000 7f65813db700 1 -- 172.21.15.121:0/2826410032 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657c06c150 con 0x7f657c067f50 2023-11-15T02:57:31.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.907+0000 7f6562ffd700 1 -- 172.21.15.121:0/2826410032 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f656c01b8f0 con 0x7f657c067f50 2023-11-15T02:57:31.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.911+0000 7f6562ffd700 1 --2- 172.21.15.121:0/2826410032 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f656805a540 0x7f656805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:31.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.911+0000 7f6562ffd700 1 -- 172.21.15.121:0/2826410032 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f656c075f30 con 0x7f657c067f50 2023-11-15T02:57:31.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.911+0000 7f657a7fc700 1 --2- 172.21.15.121:0/2826410032 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f656805a540 0x7f656805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:31.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.911+0000 7f657a7fc700 1 --2- 172.21.15.121:0/2826410032 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f656805a540 0x7f656805ca00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f657c15c210 tx=0x7f6570009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:31.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:31.915+0000 7f6562ffd700 1 -- 172.21.15.121:0/2826410032 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f656c049710 con 0x7f657c067f50 2023-11-15T02:57:32.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.083+0000 7f65813db700 1 -- 172.21.15.121:0/2826410032 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f657c153420 con 0x7f657c067f50 2023-11-15T02:57:32.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.087+0000 7f6562ffd700 1 -- 172.21.15.121:0/2826410032 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f656c04cd30 con 0x7f657c067f50 2023-11-15T02:57:32.092 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:32.093 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":21,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:56:52.702668+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:56:49.726353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-15T02:56:50.688098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:57:32.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.091+0000 7f65813db700 1 -- 172.21.15.121:0/2826410032 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f656805a540 msgr2=0x7f656805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:32.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.091+0000 7f65813db700 1 --2- 172.21.15.121:0/2826410032 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f656805a540 0x7f656805ca00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f657c15c210 tx=0x7f6570009060).stop 2023-11-15T02:57:32.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.091+0000 7f65813db700 1 -- 172.21.15.121:0/2826410032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f657c067f50 msgr2=0x7f657c0ab420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:32.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.091+0000 7f65813db700 1 --2- 172.21.15.121:0/2826410032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f657c067f50 0x7f657c0ab420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f656c009130 tx=0x7f656c017000).stop 2023-11-15T02:57:32.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.091+0000 7f65813db700 1 -- 172.21.15.121:0/2826410032 shutdown_connections 2023-11-15T02:57:32.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.091+0000 7f65813db700 1 --2- 172.21.15.121:0/2826410032 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f656805a540 0x7f656805ca00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:32.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.091+0000 7f65813db700 1 --2- 172.21.15.121:0/2826410032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f657c153bb0 0x7f657c15b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:32.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.091+0000 7f65813db700 1 --2- 172.21.15.121:0/2826410032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f657c0aa020 0x7f657c152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:32.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.091+0000 7f65813db700 1 --2- 172.21.15.121:0/2826410032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f657c067f50 0x7f657c0ab420 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:32.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.091+0000 7f65813db700 1 -- 172.21.15.121:0/2826410032 >> 172.21.15.121:0/2826410032 conn(0x7f657c058c50 msgr2=0x7f657c0bfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:32.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.091+0000 7f65813db700 1 -- 172.21.15.121:0/2826410032 shutdown_connections 2023-11-15T02:57:32.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.091+0000 7f65813db700 1 -- 172.21.15.121:0/2826410032 wait complete. 2023-11-15T02:57:32.115 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T02:57:32.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.267+0000 7f3f1c45b700 1 Processor -- start 2023-11-15T02:57:32.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.267+0000 7f3f1c45b700 1 -- start start 2023-11-15T02:57:32.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.271+0000 7f3f1c45b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f140628a0 0x7f3f14066420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:32.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.271+0000 7f3f1c45b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f14065f90 con 0x7f3f141516a0 2023-11-15T02:57:32.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.271+0000 7f3f1c45b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f14147ea0 con 0x7f3f140a8de0 2023-11-15T02:57:32.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.271+0000 7f3f1c45b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f14151e30 con 0x7f3f140628a0 2023-11-15T02:57:32.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.271+0000 7f3f199f6700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3f141516a0 0x7f3f14062190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39238/0 (socket says 172.21.15.121:39238) 2023-11-15T02:57:32.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.271+0000 7f3f199f6700 1 -- 172.21.15.121:0/992489484 learned_addr learned my addr 172.21.15.121:0/992489484 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:32.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.275+0000 7f3f1a9f8700 1 --2- 172.21.15.121:0/992489484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f140628a0 0x7f3f14066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:32.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.275+0000 7f3f191f5700 1 -- 172.21.15.121:0/992489484 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1121220839 0 0) 0x7f3f14065f90 con 0x7f3f141516a0 2023-11-15T02:57:32.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.275+0000 7f3f191f5700 1 -- 172.21.15.121:0/992489484 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f000033f0 con 0x7f3f141516a0 2023-11-15T02:57:32.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.275+0000 7f3f191f5700 1 -- 172.21.15.121:0/992489484 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 246245999 0 0) 0x7f3f14147ea0 con 0x7f3f140a8de0 2023-11-15T02:57:32.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.275+0000 7f3f191f5700 1 -- 172.21.15.121:0/992489484 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f14065f90 con 0x7f3f140a8de0 2023-11-15T02:57:32.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.275+0000 7f3f191f5700 1 -- 172.21.15.121:0/992489484 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 670209885 0 0) 0x7f3f000033f0 con 0x7f3f141516a0 2023-11-15T02:57:32.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.275+0000 7f3f1a9f8700 1 -- 172.21.15.121:0/992489484 >> v1:172.21.15.190:6789/0 conn(0x7f3f140a8de0 legacy=0x7f3f14150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:32.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.275+0000 7f3f1a9f8700 1 -- 172.21.15.121:0/992489484 >> v1:172.21.15.121:6789/0 conn(0x7f3f141516a0 legacy=0x7f3f14062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:32.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.275+0000 7f3f1a9f8700 1 -- 172.21.15.121:0/992489484 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f140671f0 con 0x7f3f140628a0 2023-11-15T02:57:32.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.275+0000 7f3f1a9f8700 1 --2- 172.21.15.121:0/992489484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f140628a0 0x7f3f14066420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f3f0c009e90 tx=0x7f3f0c009780).ready entity=mon.2 client_cookie=3cdef1d12e30dccc server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:32.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.275+0000 7f3f191f5700 1 -- 172.21.15.121:0/992489484 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c00e070 con 0x7f3f140628a0 2023-11-15T02:57:32.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.275+0000 7f3f191f5700 1 -- 172.21.15.121:0/992489484 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0c00b7a0 con 0x7f3f140628a0 2023-11-15T02:57:32.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.275+0000 7f3f191f5700 1 -- 172.21.15.121:0/992489484 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04001470 con 0x7f3f140628a0 2023-11-15T02:57:32.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.275+0000 7f3f1c45b700 1 -- 172.21.15.121:0/992489484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f140628a0 msgr2=0x7f3f14066420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:32.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.275+0000 7f3f1c45b700 1 --2- 172.21.15.121:0/992489484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f140628a0 0x7f3f14066420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f3f0c009e90 tx=0x7f3f0c009780).stop 2023-11-15T02:57:32.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.275+0000 7f3f1c45b700 1 -- 172.21.15.121:0/992489484 shutdown_connections 2023-11-15T02:57:32.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.275+0000 7f3f1c45b700 1 --2- 172.21.15.121:0/992489484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f140628a0 0x7f3f14066420 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:32.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.275+0000 7f3f1c45b700 1 -- 172.21.15.121:0/992489484 >> 172.21.15.121:0/992489484 conn(0x7f3f14067a60 msgr2=0x7f3f14067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:32.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.279+0000 7f3f1c45b700 1 -- 172.21.15.121:0/992489484 shutdown_connections 2023-11-15T02:57:32.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.279+0000 7f3f1c45b700 1 -- 172.21.15.121:0/992489484 wait complete. 2023-11-15T02:57:32.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.279+0000 7f3f1c45b700 1 Processor -- start 2023-11-15T02:57:32.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.279+0000 7f3f1c45b700 1 -- start start 2023-11-15T02:57:32.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f1c45b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f140628a0 0x7f3f14164580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:32.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f1c45b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f140a8de0 0x7f3f14152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:32.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f1c45b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f141516a0 0x7f3f140c1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:32.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f1c45b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f14165570 con 0x7f3f141516a0 2023-11-15T02:57:32.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f1c45b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f140c05e0 con 0x7f3f140628a0 2023-11-15T02:57:32.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f1c45b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f14164a70 con 0x7f3f140a8de0 2023-11-15T02:57:32.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f1a9f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f141516a0 0x7f3f140c1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:32.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f1a9f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f141516a0 0x7f3f140c1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37624/0 (socket says 172.21.15.121:37624) 2023-11-15T02:57:32.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f1a9f8700 1 -- 172.21.15.121:0/3413265641 learned_addr learned my addr 172.21.15.121:0/3413265641 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:32.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f199f6700 1 --2- 172.21.15.121:0/3413265641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f140a8de0 0x7f3f14152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:32.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f1a1f7700 1 --2- 172.21.15.121:0/3413265641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f140628a0 0x7f3f14164580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:32.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f1a9f8700 1 -- 172.21.15.121:0/3413265641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f140a8de0 msgr2=0x7f3f14152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:32.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f1a9f8700 1 --2- 172.21.15.121:0/3413265641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f140a8de0 0x7f3f14152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:32.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f1a9f8700 1 -- 172.21.15.121:0/3413265641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f140628a0 msgr2=0x7f3f14164580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:32.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f1a9f8700 1 --2- 172.21.15.121:0/3413265641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f140628a0 0x7f3f14164580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:32.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f1a9f8700 1 -- 172.21.15.121:0/3413265641 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c009a20 con 0x7f3f141516a0 2023-11-15T02:57:32.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f1a1f7700 1 --2- 172.21.15.121:0/3413265641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f140628a0 0x7f3f14164580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:32.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f1a9f8700 1 --2- 172.21.15.121:0/3413265641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f141516a0 0x7f3f140c1ac0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f3f0c00b610 tx=0x7f3f0c009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:32.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f0a7fc700 1 -- 172.21.15.121:0/3413265641 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c01d440 con 0x7f3f141516a0 2023-11-15T02:57:32.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f1c45b700 1 -- 172.21.15.121:0/3413265641 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f14164cd0 con 0x7f3f141516a0 2023-11-15T02:57:32.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f0a7fc700 1 -- 172.21.15.121:0/3413265641 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0c01d5a0 con 0x7f3f141516a0 2023-11-15T02:57:32.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f0a7fc700 1 -- 172.21.15.121:0/3413265641 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c01d940 con 0x7f3f141516a0 2023-11-15T02:57:32.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.283+0000 7f3f1c45b700 1 -- 172.21.15.121:0/3413265641 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f14165290 con 0x7f3f141516a0 2023-11-15T02:57:32.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.287+0000 7f3f1c45b700 1 -- 172.21.15.121:0/3413265641 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f140c0750 con 0x7f3f141516a0 2023-11-15T02:57:32.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.287+0000 7f3f0a7fc700 1 -- 172.21.15.121:0/3413265641 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3f0c01dd30 con 0x7f3f141516a0 2023-11-15T02:57:32.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.287+0000 7f3f0a7fc700 1 --2- 172.21.15.121:0/3413265641 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ef805a4e0 0x7f3ef805c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:32.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.287+0000 7f3f0a7fc700 1 -- 172.21.15.121:0/3413265641 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3f0c076870 con 0x7f3f141516a0 2023-11-15T02:57:32.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.287+0000 7f3f1a1f7700 1 --2- 172.21.15.121:0/3413265641 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ef805a4e0 0x7f3ef805c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:32.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.291+0000 7f3f1a1f7700 1 --2- 172.21.15.121:0/3413265641 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ef805a4e0 0x7f3ef805c9a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f3f10007fd0 tx=0x7f3f10009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:32.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.295+0000 7f3f0a7fc700 1 -- 172.21.15.121:0/3413265641 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3f0c04a050 con 0x7f3f141516a0 2023-11-15T02:57:32.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.499+0000 7f3f1c45b700 1 -- 172.21.15.121:0/3413265641 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3f14047060 con 0x7f3f141516a0 2023-11-15T02:57:32.961 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T02:57:32.955+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T02:57:32.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.971+0000 7f3f0a7fc700 1 -- 172.21.15.121:0/3413265641 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3f0c04d630 con 0x7f3f141516a0 2023-11-15T02:57:32.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.975+0000 7f3f1c45b700 1 -- 172.21.15.121:0/3413265641 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ef805a4e0 msgr2=0x7f3ef805c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:32.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.975+0000 7f3f1c45b700 1 --2- 172.21.15.121:0/3413265641 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ef805a4e0 0x7f3ef805c9a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f3f10007fd0 tx=0x7f3f10009040).stop 2023-11-15T02:57:32.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.975+0000 7f3f1c45b700 1 -- 172.21.15.121:0/3413265641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f141516a0 msgr2=0x7f3f140c1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:32.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.975+0000 7f3f1c45b700 1 --2- 172.21.15.121:0/3413265641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f141516a0 0x7f3f140c1ac0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f3f0c00b610 tx=0x7f3f0c009780).stop 2023-11-15T02:57:32.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.975+0000 7f3f1c45b700 1 -- 172.21.15.121:0/3413265641 shutdown_connections 2023-11-15T02:57:32.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.975+0000 7f3f1c45b700 1 --2- 172.21.15.121:0/3413265641 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ef805a4e0 0x7f3ef805c9a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:32.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.975+0000 7f3f1c45b700 1 --2- 172.21.15.121:0/3413265641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f141516a0 0x7f3f140c1ac0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:32.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.975+0000 7f3f1c45b700 1 --2- 172.21.15.121:0/3413265641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f140a8de0 0x7f3f14152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:32.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.975+0000 7f3f1c45b700 1 --2- 172.21.15.121:0/3413265641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f140628a0 0x7f3f14164580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:32.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.975+0000 7f3f1c45b700 1 -- 172.21.15.121:0/3413265641 >> 172.21.15.121:0/3413265641 conn(0x7f3f14067a60 msgr2=0x7f3f14150d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:32.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.975+0000 7f3f1c45b700 1 -- 172.21.15.121:0/3413265641 shutdown_connections 2023-11-15T02:57:32.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:32.975+0000 7f3f1c45b700 1 -- 172.21.15.121:0/3413265641 wait complete. 2023-11-15T02:57:32.983 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T02:57:32.986 INFO:tasks.ceph.mds.b.smithi190.stderr: -1908> 2023-11-15T02:56:57.368+0000 7f210b499700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T02:57:33.004 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-15T02:57:33.030 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-11-15T02:57:33.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.159+0000 7fa279ba9700 1 Processor -- start 2023-11-15T02:57:33.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.159+0000 7fa279ba9700 1 -- start start 2023-11-15T02:57:33.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa279ba9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa274065850 0x7fa2740a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:33.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa279ba9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa274068170 0x7fa2740bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:33.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa279ba9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa274152150 con 0x7fa2740a8b80 2023-11-15T02:57:33.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa279ba9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa274148f70 con 0x7fa274068170 2023-11-15T02:57:33.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa279ba9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2741523a0 con 0x7fa274065850 2023-11-15T02:57:33.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa273fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa274068170 0x7fa2740bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:33.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa2737fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa2740a8b80 0x7fa274067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39242/0 (socket says 172.21.15.121:39242) 2023-11-15T02:57:33.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa2737fe700 1 -- 172.21.15.121:0/2999157698 learned_addr learned my addr 172.21.15.121:0/2999157698 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:33.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa272ffd700 1 --2- 172.21.15.121:0/2999157698 >> v2:172.21.15.190:3300/0 conn(0x7fa274065850 0x7fa2740a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:33.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa273fff700 1 -- 172.21.15.121:0/2999157698 >> v2:172.21.15.190:3300/0 conn(0x7fa274065850 msgr2=0x7fa2740a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:33.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa273fff700 1 --2- 172.21.15.121:0/2999157698 >> v2:172.21.15.190:3300/0 conn(0x7fa274065850 0x7fa2740a8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:33.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa273fff700 1 -- 172.21.15.121:0/2999157698 >> v1:172.21.15.121:6789/0 conn(0x7fa2740a8b80 legacy=0x7fa274067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:33.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa273fff700 1 -- 172.21.15.121:0/2999157698 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa274152680 con 0x7fa274068170 2023-11-15T02:57:33.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa2727fc700 1 -- 172.21.15.121:0/2999157698 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132487796 0 0) 0x7fa274152150 con 0x7fa2740a8b80 2023-11-15T02:57:33.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa273fff700 1 --2- 172.21.15.121:0/2999157698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa274068170 0x7fa2740bbe20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fa2740676c0 tx=0x7fa26c009780).ready entity=mon.0 client_cookie=2d336ef562a66521 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:33.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa2727fc700 1 -- 172.21.15.121:0/2999157698 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c00e070 con 0x7fa274068170 2023-11-15T02:57:33.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa2727fc700 1 -- 172.21.15.121:0/2999157698 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26c00bb60 con 0x7fa274068170 2023-11-15T02:57:33.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa2727fc700 1 -- 172.21.15.121:0/2999157698 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c012410 con 0x7fa274068170 2023-11-15T02:57:33.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa279ba9700 1 -- 172.21.15.121:0/2999157698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa274068170 msgr2=0x7fa2740bbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:33.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa279ba9700 1 --2- 172.21.15.121:0/2999157698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa274068170 0x7fa2740bbe20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fa2740676c0 tx=0x7fa26c009780).stop 2023-11-15T02:57:33.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa279ba9700 1 -- 172.21.15.121:0/2999157698 shutdown_connections 2023-11-15T02:57:33.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa279ba9700 1 --2- 172.21.15.121:0/2999157698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa274068170 0x7fa2740bbe20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:33.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa279ba9700 1 --2- 172.21.15.121:0/2999157698 >> v2:172.21.15.190:3300/0 conn(0x7fa274065850 0x7fa2740a8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:33.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa279ba9700 1 -- 172.21.15.121:0/2999157698 >> 172.21.15.121:0/2999157698 conn(0x7fa2740591c0 msgr2=0x7fa2740c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:33.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa279ba9700 1 -- 172.21.15.121:0/2999157698 shutdown_connections 2023-11-15T02:57:33.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.163+0000 7fa279ba9700 1 -- 172.21.15.121:0/2999157698 wait complete. 2023-11-15T02:57:33.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.167+0000 7fa279ba9700 1 Processor -- start 2023-11-15T02:57:33.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.167+0000 7fa279ba9700 1 -- start start 2023-11-15T02:57:33.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa279ba9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa274065850 0x7fa274149110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:33.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa279ba9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa274068170 0x7fa2741472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:33.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa279ba9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2740a8b80 0x7fa274151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:33.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa279ba9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2740c0130 con 0x7fa274068170 2023-11-15T02:57:33.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa279ba9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2740bd960 con 0x7fa274065850 2023-11-15T02:57:33.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa279ba9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa274149600 con 0x7fa2740a8b80 2023-11-15T02:57:33.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa272ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa274068170 0x7fa2741472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:33.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa272ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa274068170 0x7fa2741472e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37648/0 (socket says 172.21.15.121:37648) 2023-11-15T02:57:33.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa272ffd700 1 -- 172.21.15.121:0/429053312 learned_addr learned my addr 172.21.15.121:0/429053312 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:33.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa273fff700 1 --2- 172.21.15.121:0/429053312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2740a8b80 0x7fa274151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:33.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa2737fe700 1 --2- 172.21.15.121:0/429053312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa274065850 0x7fa274149110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:33.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa272ffd700 1 -- 172.21.15.121:0/429053312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2740a8b80 msgr2=0x7fa274151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:33.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa272ffd700 1 --2- 172.21.15.121:0/429053312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2740a8b80 0x7fa274151530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:33.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa272ffd700 1 -- 172.21.15.121:0/429053312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa274065850 msgr2=0x7fa274149110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:33.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa272ffd700 1 --2- 172.21.15.121:0/429053312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa274065850 0x7fa274149110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:33.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa272ffd700 1 -- 172.21.15.121:0/429053312 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c009a20 con 0x7fa274068170 2023-11-15T02:57:33.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa273fff700 1 --2- 172.21.15.121:0/429053312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2740a8b80 0x7fa274151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:57:33.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa272ffd700 1 --2- 172.21.15.121:0/429053312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa274068170 0x7fa2741472e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fa26800e900 tx=0x7fa268009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:33.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa25b7fe700 1 -- 172.21.15.121:0/429053312 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26800c640 con 0x7fa274068170 2023-11-15T02:57:33.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa25b7fe700 1 -- 172.21.15.121:0/429053312 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26800c7a0 con 0x7fa274068170 2023-11-15T02:57:33.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa279ba9700 1 -- 172.21.15.121:0/429053312 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa274149880 con 0x7fa274068170 2023-11-15T02:57:33.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa25b7fe700 1 -- 172.21.15.121:0/429053312 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26800cb20 con 0x7fa274068170 2023-11-15T02:57:33.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.171+0000 7fa279ba9700 1 -- 172.21.15.121:0/429053312 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa274149e40 con 0x7fa274068170 2023-11-15T02:57:33.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.175+0000 7fa279ba9700 1 -- 172.21.15.121:0/429053312 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2740bdaf0 con 0x7fa274068170 2023-11-15T02:57:33.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.175+0000 7fa25b7fe700 1 -- 172.21.15.121:0/429053312 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa26801b400 con 0x7fa274068170 2023-11-15T02:57:33.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.175+0000 7fa25b7fe700 1 --2- 172.21.15.121:0/429053312 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa26005a540 0x7fa26005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:33.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.175+0000 7fa25b7fe700 1 -- 172.21.15.121:0/429053312 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fa268074c60 con 0x7fa274068170 2023-11-15T02:57:33.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.175+0000 7fa2737fe700 1 --2- 172.21.15.121:0/429053312 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa26005a540 0x7fa26005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:33.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.179+0000 7fa2737fe700 1 --2- 172.21.15.121:0/429053312 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa26005a540 0x7fa26005ca00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fa26400c7d0 tx=0x7fa264009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:33.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.183+0000 7fa25b7fe700 1 -- 172.21.15.121:0/429053312 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa268048440 con 0x7fa274068170 2023-11-15T02:57:33.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.379+0000 7fa279ba9700 1 -- 172.21.15.121:0/429053312 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa274063b60 con 0x7fa274068170 2023-11-15T02:57:33.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.983+0000 7fa25b7fe700 1 -- 172.21.15.121:0/429053312 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa268016070 con 0x7fa274068170 2023-11-15T02:57:33.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.987+0000 7fa279ba9700 1 -- 172.21.15.121:0/429053312 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa26005a540 msgr2=0x7fa26005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:33.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.987+0000 7fa279ba9700 1 --2- 172.21.15.121:0/429053312 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa26005a540 0x7fa26005ca00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fa26400c7d0 tx=0x7fa264009040).stop 2023-11-15T02:57:33.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.987+0000 7fa279ba9700 1 -- 172.21.15.121:0/429053312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa274068170 msgr2=0x7fa2741472e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:33.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.987+0000 7fa279ba9700 1 --2- 172.21.15.121:0/429053312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa274068170 0x7fa2741472e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fa26800e900 tx=0x7fa268009060).stop 2023-11-15T02:57:33.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.987+0000 7fa279ba9700 1 -- 172.21.15.121:0/429053312 shutdown_connections 2023-11-15T02:57:33.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.987+0000 7fa279ba9700 1 --2- 172.21.15.121:0/429053312 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa26005a540 0x7fa26005ca00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:33.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.987+0000 7fa279ba9700 1 --2- 172.21.15.121:0/429053312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2740a8b80 0x7fa274151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:33.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.987+0000 7fa279ba9700 1 --2- 172.21.15.121:0/429053312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa274068170 0x7fa2741472e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:33.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.987+0000 7fa279ba9700 1 --2- 172.21.15.121:0/429053312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa274065850 0x7fa274149110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:33.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.987+0000 7fa279ba9700 1 -- 172.21.15.121:0/429053312 >> 172.21.15.121:0/429053312 conn(0x7fa2740591c0 msgr2=0x7fa2740c51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:33.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.987+0000 7fa279ba9700 1 -- 172.21.15.121:0/429053312 shutdown_connections 2023-11-15T02:57:33.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:33.987+0000 7fa279ba9700 1 -- 172.21.15.121:0/429053312 wait complete. 2023-11-15T02:57:34.015 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-15T02:57:34.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.183+0000 7fd3b0569700 1 Processor -- start 2023-11-15T02:57:34.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.183+0000 7fd3b0569700 1 -- start start 2023-11-15T02:57:34.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.187+0000 7fd3b0569700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd3a8150800 0x7fd3a8066ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:34.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.187+0000 7fd3b0569700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd3a80bb780 0x7fd3a80bbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:34.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.187+0000 7fd3b0569700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a80bc0e0 con 0x7fd3a8150be0 2023-11-15T02:57:34.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.187+0000 7fd3b0569700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a8065950 con 0x7fd3a8150800 2023-11-15T02:57:34.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.187+0000 7fd3b0569700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a8065c30 con 0x7fd3a80bb780 2023-11-15T02:57:34.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.187+0000 7fd3ae305700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd3a8150800 0x7fd3a8066ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:34.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.187+0000 7fd3adb04700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fd3a8150be0 0x7fd3a80660c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39256/0 (socket says 172.21.15.121:39256) 2023-11-15T02:57:34.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.187+0000 7fd3adb04700 1 -- 172.21.15.121:0/1449258978 learned_addr learned my addr 172.21.15.121:0/1449258978 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:34.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.187+0000 7fd3aeb06700 1 --2- 172.21.15.121:0/1449258978 >> v2:172.21.15.190:3300/0 conn(0x7fd3a80bb780 0x7fd3a80bbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:34.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.187+0000 7fd3ae305700 1 -- 172.21.15.121:0/1449258978 >> v2:172.21.15.190:3300/0 conn(0x7fd3a80bb780 msgr2=0x7fd3a80bbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:34.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.187+0000 7fd3ae305700 1 --2- 172.21.15.121:0/1449258978 >> v2:172.21.15.190:3300/0 conn(0x7fd3a80bb780 0x7fd3a80bbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:34.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.187+0000 7fd3ae305700 1 -- 172.21.15.121:0/1449258978 >> v1:172.21.15.121:6789/0 conn(0x7fd3a8150be0 legacy=0x7fd3a80660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:34.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.187+0000 7fd3ae305700 1 -- 172.21.15.121:0/1449258978 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a80667d0 con 0x7fd3a8150800 2023-11-15T02:57:34.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.187+0000 7fd3ad303700 1 -- 172.21.15.121:0/1449258978 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1488858031 0 0) 0x7fd398001110 con 0x7fd3a8150be0 2023-11-15T02:57:34.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.187+0000 7fd3ae305700 1 --2- 172.21.15.121:0/1449258978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3a8150800 0x7fd3a8066ef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fd3a80658e0 tx=0x7fd3a4009060).ready entity=mon.0 client_cookie=8b161c5f04eb2aac server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:34.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.187+0000 7fd3ad303700 1 -- 172.21.15.121:0/1449258978 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a400f510 con 0x7fd3a8150800 2023-11-15T02:57:34.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.187+0000 7fd3ad303700 1 -- 172.21.15.121:0/1449258978 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a400f670 con 0x7fd3a8150800 2023-11-15T02:57:34.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.187+0000 7fd3ad303700 1 -- 172.21.15.121:0/1449258978 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a400f9f0 con 0x7fd3a8150800 2023-11-15T02:57:34.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.191+0000 7fd3b0569700 1 -- 172.21.15.121:0/1449258978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3a8150800 msgr2=0x7fd3a8066ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:34.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.191+0000 7fd3b0569700 1 --2- 172.21.15.121:0/1449258978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3a8150800 0x7fd3a8066ef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fd3a80658e0 tx=0x7fd3a4009060).stop 2023-11-15T02:57:34.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.191+0000 7fd3b0569700 1 -- 172.21.15.121:0/1449258978 shutdown_connections 2023-11-15T02:57:34.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.191+0000 7fd3b0569700 1 --2- 172.21.15.121:0/1449258978 >> v2:172.21.15.190:3300/0 conn(0x7fd3a80bb780 0x7fd3a80bbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:34.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.191+0000 7fd3b0569700 1 --2- 172.21.15.121:0/1449258978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3a8150800 0x7fd3a8066ef0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:34.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.191+0000 7fd3b0569700 1 -- 172.21.15.121:0/1449258978 >> 172.21.15.121:0/1449258978 conn(0x7fd3a8068500 msgr2=0x7fd3a80c6780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:34.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.191+0000 7fd3b0569700 1 -- 172.21.15.121:0/1449258978 shutdown_connections 2023-11-15T02:57:34.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.191+0000 7fd3b0569700 1 -- 172.21.15.121:0/1449258978 wait complete. 2023-11-15T02:57:34.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.191+0000 7fd3b0569700 1 Processor -- start 2023-11-15T02:57:34.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.191+0000 7fd3b0569700 1 -- start start 2023-11-15T02:57:34.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd3b0569700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3a80bb780 0x7fd3a81649e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:34.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd3b0569700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3a8150800 0x7fd3a814f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:34.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd3b0569700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3a8150be0 0x7fd3a80a9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:34.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd3b0569700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a8164ed0 con 0x7fd3a80bb780 2023-11-15T02:57:34.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd3b0569700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a80ab990 con 0x7fd3a8150be0 2023-11-15T02:57:34.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd3b0569700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a80abc70 con 0x7fd3a8150800 2023-11-15T02:57:34.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd3ae305700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3a80bb780 0x7fd3a81649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:34.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd3ae305700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3a80bb780 0x7fd3a81649e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37664/0 (socket says 172.21.15.121:37664) 2023-11-15T02:57:34.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd3ae305700 1 -- 172.21.15.121:0/3755774114 learned_addr learned my addr 172.21.15.121:0/3755774114 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:34.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd3aeb06700 1 --2- 172.21.15.121:0/3755774114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3a8150be0 0x7fd3a80a9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:34.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd3adb04700 1 --2- 172.21.15.121:0/3755774114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3a8150800 0x7fd3a814f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:34.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd3ae305700 1 -- 172.21.15.121:0/3755774114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3a8150800 msgr2=0x7fd3a814f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:34.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd3ae305700 1 --2- 172.21.15.121:0/3755774114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3a8150800 0x7fd3a814f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:34.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd3ae305700 1 -- 172.21.15.121:0/3755774114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3a8150be0 msgr2=0x7fd3a80a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:34.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd3ae305700 1 --2- 172.21.15.121:0/3755774114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3a8150be0 0x7fd3a80a9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:34.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd3ae305700 1 -- 172.21.15.121:0/3755774114 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a40092a0 con 0x7fd3a80bb780 2023-11-15T02:57:34.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd3ae305700 1 --2- 172.21.15.121:0/3755774114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3a80bb780 0x7fd3a81649e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fd3a4009dd0 tx=0x7fd3a4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:34.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd39e7fc700 1 -- 172.21.15.121:0/3755774114 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a400f7e0 con 0x7fd3a80bb780 2023-11-15T02:57:34.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd39e7fc700 1 -- 172.21.15.121:0/3755774114 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a400f940 con 0x7fd3a80bb780 2023-11-15T02:57:34.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd39e7fc700 1 -- 172.21.15.121:0/3755774114 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a401b660 con 0x7fd3a80bb780 2023-11-15T02:57:34.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd3b0569700 1 -- 172.21.15.121:0/3755774114 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a80abf50 con 0x7fd3a80bb780 2023-11-15T02:57:34.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.195+0000 7fd3b0569700 1 -- 172.21.15.121:0/3755774114 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a80ac510 con 0x7fd3a80bb780 2023-11-15T02:57:34.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.199+0000 7fd3b0569700 1 -- 172.21.15.121:0/3755774114 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a80ac760 con 0x7fd3a80bb780 2023-11-15T02:57:34.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.199+0000 7fd39e7fc700 1 -- 172.21.15.121:0/3755774114 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd3a401ba10 con 0x7fd3a80bb780 2023-11-15T02:57:34.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.199+0000 7fd39e7fc700 1 --2- 172.21.15.121:0/3755774114 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd38c05a510 0x7fd38c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:34.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.199+0000 7fd39e7fc700 1 -- 172.21.15.121:0/3755774114 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fd3a4076300 con 0x7fd3a80bb780 2023-11-15T02:57:34.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.199+0000 7fd3adb04700 1 --2- 172.21.15.121:0/3755774114 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd38c05a510 0x7fd38c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:34.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.203+0000 7fd3adb04700 1 --2- 172.21.15.121:0/3755774114 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd38c05a510 0x7fd38c05c9d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fd39800c7d0 tx=0x7fd398009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:34.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.207+0000 7fd39e7fc700 1 -- 172.21.15.121:0/3755774114 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd3a4049a90 con 0x7fd3a80bb780 2023-11-15T02:57:34.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.379+0000 7fd3b0569700 1 -- 172.21.15.121:0/3755774114 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd3a806c150 con 0x7fd3a80bb780 2023-11-15T02:57:34.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.987+0000 7fd39e7fc700 1 -- 172.21.15.121:0/3755774114 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd3a4028020 con 0x7fd3a80bb780 2023-11-15T02:57:34.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.991+0000 7fd3b0569700 1 -- 172.21.15.121:0/3755774114 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd38c05a510 msgr2=0x7fd38c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:34.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.991+0000 7fd3b0569700 1 --2- 172.21.15.121:0/3755774114 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd38c05a510 0x7fd38c05c9d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fd39800c7d0 tx=0x7fd398009040).stop 2023-11-15T02:57:34.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.991+0000 7fd3b0569700 1 -- 172.21.15.121:0/3755774114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3a80bb780 msgr2=0x7fd3a81649e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:34.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.991+0000 7fd3b0569700 1 --2- 172.21.15.121:0/3755774114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3a80bb780 0x7fd3a81649e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fd3a4009dd0 tx=0x7fd3a4017000).stop 2023-11-15T02:57:34.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.991+0000 7fd3b0569700 1 -- 172.21.15.121:0/3755774114 shutdown_connections 2023-11-15T02:57:34.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.991+0000 7fd3b0569700 1 --2- 172.21.15.121:0/3755774114 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd38c05a510 0x7fd38c05c9d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:34.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.991+0000 7fd3b0569700 1 --2- 172.21.15.121:0/3755774114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3a8150be0 0x7fd3a80a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:35.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.991+0000 7fd3b0569700 1 --2- 172.21.15.121:0/3755774114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3a8150800 0x7fd3a814f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:35.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.991+0000 7fd3b0569700 1 --2- 172.21.15.121:0/3755774114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3a80bb780 0x7fd3a81649e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:35.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.991+0000 7fd3b0569700 1 -- 172.21.15.121:0/3755774114 >> 172.21.15.121:0/3755774114 conn(0x7fd3a8068500 msgr2=0x7fd3a80a4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:35.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.991+0000 7fd3b0569700 1 -- 172.21.15.121:0/3755774114 shutdown_connections 2023-11-15T02:57:35.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:34.995+0000 7fd3b0569700 1 -- 172.21.15.121:0/3755774114 wait complete. 2023-11-15T02:57:35.001 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_metadata' removed 2023-11-15T02:57:35.020 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-15T02:57:35.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.187+0000 7f0eb759e700 1 Processor -- start 2023-11-15T02:57:35.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.187+0000 7f0eb759e700 1 -- start start 2023-11-15T02:57:35.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.191+0000 7f0eb759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eb80ad1b0 0x7f0eb8065710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:35.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.191+0000 7f0eb759e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0eb8065c00 0x7f0eb80add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:35.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.191+0000 7f0eb759e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb8066c90 con 0x7f0eb80ab0d0 2023-11-15T02:57:35.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.191+0000 7f0eb759e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb8066080 con 0x7f0eb8065c00 2023-11-15T02:57:35.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.191+0000 7f0eb759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb8066360 con 0x7f0eb80ad1b0 2023-11-15T02:57:35.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.191+0000 7f0eb659c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f0eb80ab0d0 0x7f0eb80ab4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56180/0 (socket says 172.21.15.121:56180) 2023-11-15T02:57:35.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.191+0000 7f0eb659c700 1 -- 172.21.15.121:0/2852495228 learned_addr learned my addr 172.21.15.121:0/2852495228 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:35.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.191+0000 7f0eb6d9d700 1 --2- 172.21.15.121:0/2852495228 >> v2:172.21.15.121:3300/0 conn(0x7f0eb8065c00 0x7f0eb80add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:35.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.191+0000 7f0eb5d9b700 1 --2- 172.21.15.121:0/2852495228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eb80ad1b0 0x7f0eb8065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:35.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.191+0000 7f0eb559a700 1 -- 172.21.15.121:0/2852495228 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316870297 0 0) 0x7f0eb8066c90 con 0x7f0eb80ab0d0 2023-11-15T02:57:35.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.191+0000 7f0eb559a700 1 -- 172.21.15.121:0/2852495228 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ea00033f0 con 0x7f0eb80ab0d0 2023-11-15T02:57:35.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.191+0000 7f0eb6d9d700 1 -- 172.21.15.121:0/2852495228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eb80ad1b0 msgr2=0x7f0eb8065710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:35.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.191+0000 7f0eb6d9d700 1 --2- 172.21.15.121:0/2852495228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eb80ad1b0 0x7f0eb8065710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:35.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.191+0000 7f0eb6d9d700 1 -- 172.21.15.121:0/2852495228 >> v1:172.21.15.121:6789/0 conn(0x7f0eb80ab0d0 legacy=0x7f0eb80ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:35.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.191+0000 7f0eb6d9d700 1 -- 172.21.15.121:0/2852495228 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb8066640 con 0x7f0eb8065c00 2023-11-15T02:57:35.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.191+0000 7f0eb6d9d700 1 --2- 172.21.15.121:0/2852495228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb8065c00 0x7f0eb80add40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f0eb0009e60 tx=0x7f0eb0009780).ready entity=mon.0 client_cookie=22a22a827ea1b80c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:35.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.195+0000 7f0eb559a700 1 -- 172.21.15.121:0/2852495228 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000e070 con 0x7f0eb8065c00 2023-11-15T02:57:35.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.195+0000 7f0eb559a700 1 -- 172.21.15.121:0/2852495228 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb000bab0 con 0x7f0eb8065c00 2023-11-15T02:57:35.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.195+0000 7f0eb559a700 1 -- 172.21.15.121:0/2852495228 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000be30 con 0x7f0eb8065c00 2023-11-15T02:57:35.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.195+0000 7f0eb759e700 1 -- 172.21.15.121:0/2852495228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb8065c00 msgr2=0x7f0eb80add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:35.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.195+0000 7f0eb759e700 1 --2- 172.21.15.121:0/2852495228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb8065c00 0x7f0eb80add40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f0eb0009e60 tx=0x7f0eb0009780).stop 2023-11-15T02:57:35.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.195+0000 7f0eb759e700 1 -- 172.21.15.121:0/2852495228 shutdown_connections 2023-11-15T02:57:35.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.195+0000 7f0eb759e700 1 --2- 172.21.15.121:0/2852495228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb8065c00 0x7f0eb80add40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:35.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.195+0000 7f0eb759e700 1 --2- 172.21.15.121:0/2852495228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eb80ad1b0 0x7f0eb8065710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:35.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.195+0000 7f0eb759e700 1 -- 172.21.15.121:0/2852495228 >> 172.21.15.121:0/2852495228 conn(0x7f0eb8067a60 msgr2=0x7f0eb8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:35.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.195+0000 7f0eb759e700 1 -- 172.21.15.121:0/2852495228 shutdown_connections 2023-11-15T02:57:35.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.195+0000 7f0eb759e700 1 -- 172.21.15.121:0/2852495228 wait complete. 2023-11-15T02:57:35.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.195+0000 7f0eb759e700 1 Processor -- start 2023-11-15T02:57:35.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.195+0000 7f0eb759e700 1 -- start start 2023-11-15T02:57:35.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0eb759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0eb8065c00 0x7f0eb80ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:35.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0eb759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb80ab0d0 0x7f0eb80c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:35.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0eb759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eb80ad1b0 0x7f0eb80c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:35.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0eb759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb80ca520 con 0x7f0eb80ab0d0 2023-11-15T02:57:35.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0eb759e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb80ca730 con 0x7f0eb8065c00 2023-11-15T02:57:35.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0eb759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb80a38c0 con 0x7f0eb80ad1b0 2023-11-15T02:57:35.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0eb5d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb80ab0d0 0x7f0eb80c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:35.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0eb5d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb80ab0d0 0x7f0eb80c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36678/0 (socket says 172.21.15.121:36678) 2023-11-15T02:57:35.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0eb5d9b700 1 -- 172.21.15.121:0/3223536646 learned_addr learned my addr 172.21.15.121:0/3223536646 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:35.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0eb659c700 1 --2- 172.21.15.121:0/3223536646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0eb8065c00 0x7f0eb80ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:35.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0eb6d9d700 1 --2- 172.21.15.121:0/3223536646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eb80ad1b0 0x7f0eb80c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:35.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0eb5d9b700 1 -- 172.21.15.121:0/3223536646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eb80ad1b0 msgr2=0x7f0eb80c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:35.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0eb5d9b700 1 --2- 172.21.15.121:0/3223536646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eb80ad1b0 0x7f0eb80c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:35.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0eb5d9b700 1 -- 172.21.15.121:0/3223536646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0eb8065c00 msgr2=0x7f0eb80ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:35.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0eb5d9b700 1 --2- 172.21.15.121:0/3223536646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0eb8065c00 0x7f0eb80ca030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:35.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0eb5d9b700 1 -- 172.21.15.121:0/3223536646 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb0009a20 con 0x7f0eb80ab0d0 2023-11-15T02:57:35.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0eb5d9b700 1 --2- 172.21.15.121:0/3223536646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb80ab0d0 0x7f0eb80c8350 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f0eac00e900 tx=0x7f0eac009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:35.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0ea67fc700 1 -- 172.21.15.121:0/3223536646 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac012600 con 0x7f0eb80ab0d0 2023-11-15T02:57:35.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0ea67fc700 1 -- 172.21.15.121:0/3223536646 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac012760 con 0x7f0eb80ab0d0 2023-11-15T02:57:35.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0ea67fc700 1 -- 172.21.15.121:0/3223536646 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac012ae0 con 0x7f0eb80ab0d0 2023-11-15T02:57:35.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0eb759e700 1 -- 172.21.15.121:0/3223536646 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb80a3ba0 con 0x7f0eb80ab0d0 2023-11-15T02:57:35.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.199+0000 7f0eb759e700 1 -- 172.21.15.121:0/3223536646 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb80a4160 con 0x7f0eb80ab0d0 2023-11-15T02:57:35.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.203+0000 7f0eb759e700 1 -- 172.21.15.121:0/3223536646 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb806bcf0 con 0x7f0eb80ab0d0 2023-11-15T02:57:35.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.203+0000 7f0ea67fc700 1 -- 172.21.15.121:0/3223536646 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0eac01b400 con 0x7f0eb80ab0d0 2023-11-15T02:57:35.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.203+0000 7f0ea67fc700 1 --2- 172.21.15.121:0/3223536646 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e9805a540 0x7f0e9805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:35.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.207+0000 7f0ea67fc700 1 -- 172.21.15.121:0/3223536646 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5769+0+0 (secure 0 0 0) 0x7f0eac074a40 con 0x7f0eb80ab0d0 2023-11-15T02:57:35.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.207+0000 7f0eb659c700 1 --2- 172.21.15.121:0/3223536646 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e9805a540 0x7f0e9805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:35.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.207+0000 7f0eb659c700 1 --2- 172.21.15.121:0/3223536646 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e9805a540 0x7f0e9805ca00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f0ea8000c00 tx=0x7f0ea8009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:35.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.211+0000 7f0ea67fc700 1 -- 172.21.15.121:0/3223536646 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0eac076e60 con 0x7f0eb80ab0d0 2023-11-15T02:57:35.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.383+0000 7f0eb759e700 1 -- 172.21.15.121:0/3223536646 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0eb80c7160 con 0x7f0eb80ab0d0 2023-11-15T02:57:36.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:35.999+0000 7f0ea67fc700 1 -- 172.21.15.121:0/3223536646 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0eac016030 con 0x7f0eb80ab0d0 2023-11-15T02:57:36.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.003+0000 7f0eb759e700 1 -- 172.21.15.121:0/3223536646 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e9805a540 msgr2=0x7f0e9805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:36.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.003+0000 7f0eb759e700 1 --2- 172.21.15.121:0/3223536646 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e9805a540 0x7f0e9805ca00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f0ea8000c00 tx=0x7f0ea8009040).stop 2023-11-15T02:57:36.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.003+0000 7f0eb759e700 1 -- 172.21.15.121:0/3223536646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb80ab0d0 msgr2=0x7f0eb80c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:36.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.003+0000 7f0eb759e700 1 --2- 172.21.15.121:0/3223536646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb80ab0d0 0x7f0eb80c8350 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f0eac00e900 tx=0x7f0eac009060).stop 2023-11-15T02:57:36.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.007+0000 7f0eb759e700 1 -- 172.21.15.121:0/3223536646 shutdown_connections 2023-11-15T02:57:36.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.007+0000 7f0eb759e700 1 --2- 172.21.15.121:0/3223536646 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e9805a540 0x7f0e9805ca00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:36.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.007+0000 7f0eb759e700 1 --2- 172.21.15.121:0/3223536646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eb80ad1b0 0x7f0eb80c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:36.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.007+0000 7f0eb759e700 1 --2- 172.21.15.121:0/3223536646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb80ab0d0 0x7f0eb80c8350 secure :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0x7f0eac00e900 tx=0x7f0eac009060).stop 2023-11-15T02:57:36.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.007+0000 7f0eb759e700 1 --2- 172.21.15.121:0/3223536646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0eb8065c00 0x7f0eb80ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:36.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.007+0000 7f0eb759e700 1 -- 172.21.15.121:0/3223536646 >> 172.21.15.121:0/3223536646 conn(0x7f0eb8067a60 msgr2=0x7f0eb8153380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:36.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.007+0000 7f0eb759e700 1 -- 172.21.15.121:0/3223536646 shutdown_connections 2023-11-15T02:57:36.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.007+0000 7f0eb759e700 1 -- 172.21.15.121:0/3223536646 wait complete. 2023-11-15T02:57:36.014 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_data' removed 2023-11-15T02:57:36.034 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-15T02:57:36.034 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-15T02:57:36.035 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-15T02:57:36.035 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T02:57:36.035 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-15T02:57:36.036 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-15T02:57:36.036 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-15T02:57:36.036 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T02:57:36.037 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-15T02:57:36.037 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-15T02:57:36.037 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-15T02:57:36.037 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T02:57:36.038 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-15T02:57:36.038 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-15T02:57:36.038 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-15T02:57:36.039 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T02:57:36.055 INFO:tasks.ceph.mds.c:Stopped 2023-11-15T02:57:36.055 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-15T02:57:36.057 INFO:tasks.ceph.mds.a:Stopped 2023-11-15T02:57:36.057 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-15T02:57:36.060 INFO:tasks.ceph.mds.d:Stopped 2023-11-15T02:57:36.061 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-15T02:57:36.064 INFO:tasks.ceph.mds.b:Stopped 2023-11-15T02:57:36.064 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-15T02:57:36.108 INFO:tasks.ceph.mds.c:Started 2023-11-15T02:57:36.109 DEBUG:teuthology.parallel:result is None 2023-11-15T02:57:36.114 INFO:tasks.ceph.mds.a:Started 2023-11-15T02:57:36.115 DEBUG:teuthology.parallel:result is None 2023-11-15T02:57:36.117 INFO:tasks.ceph.mds.d:Started 2023-11-15T02:57:36.118 DEBUG:teuthology.parallel:result is None 2023-11-15T02:57:36.121 INFO:tasks.ceph.mds.b:Started 2023-11-15T02:57:36.121 DEBUG:teuthology.parallel:result is None 2023-11-15T02:57:36.122 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-15T02:57:36.216 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T02:57:36.219 INFO:tasks.ceph.mds.c.smithi121.stdout:starting mds.c at 2023-11-15T02:57:36.229 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-11-15T02:57:36.241 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-11-15T02:57:36.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.283+0000 7fc0d58e4700 1 Processor -- start 2023-11-15T02:57:36.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.283+0000 7fc0d58e4700 1 -- start start 2023-11-15T02:57:36.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0d58e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0d01511f0 0x7fc0d0066570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:36.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0d58e4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc0d00bcab0 0x7fc0d01515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:36.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0d58e4700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0d0066a60 con 0x7fc0d0151ac0 2023-11-15T02:57:36.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0d58e4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d00660e0 con 0x7fc0d00bcab0 2023-11-15T02:57:36.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0d58e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d00bd390 con 0x7fc0d01511f0 2023-11-15T02:57:36.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0d50e3700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc0d0151ac0 0x7fc0d00a8750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56208/0 (socket says 172.21.15.121:56208) 2023-11-15T02:57:36.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0d50e3700 1 -- 172.21.15.121:0/3569439087 learned_addr learned my addr 172.21.15.121:0/3569439087 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:36.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0d48e2700 1 --2- 172.21.15.121:0/3569439087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0d01511f0 0x7fc0d0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:36.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cffff700 1 --2- 172.21.15.121:0/3569439087 >> v2:172.21.15.190:3300/0 conn(0x7fc0d00bcab0 0x7fc0d01515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:36.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cf7fe700 1 -- 172.21.15.121:0/3569439087 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2112575319 0 0) 0x7fc0d0066a60 con 0x7fc0d0151ac0 2023-11-15T02:57:36.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cf7fe700 1 -- 172.21.15.121:0/3569439087 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0c00033f0 con 0x7fc0d0151ac0 2023-11-15T02:57:36.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cf7fe700 1 -- 172.21.15.121:0/3569439087 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1106631743 0 0) 0x7fc0c00033f0 con 0x7fc0d0151ac0 2023-11-15T02:57:36.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cf7fe700 1 -- 172.21.15.121:0/3569439087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0d01511f0 msgr2=0x7fc0d0066570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:36.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cf7fe700 1 --2- 172.21.15.121:0/3569439087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0d01511f0 0x7fc0d0066570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:36.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cf7fe700 1 -- 172.21.15.121:0/3569439087 >> v2:172.21.15.190:3300/0 conn(0x7fc0d00bcab0 msgr2=0x7fc0d01515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:36.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cf7fe700 1 --2- 172.21.15.121:0/3569439087 >> v2:172.21.15.190:3300/0 conn(0x7fc0d00bcab0 0x7fc0d01515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:36.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cf7fe700 1 -- 172.21.15.121:0/3569439087 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d00657b0 con 0x7fc0d0151ac0 2023-11-15T02:57:36.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cf7fe700 1 -- 172.21.15.121:0/3569439087 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fc0c80017f0 con 0x7fc0d0151ac0 2023-11-15T02:57:36.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cf7fe700 1 -- 172.21.15.121:0/3569439087 >> v1:172.21.15.121:6789/0 conn(0x7fc0d0151ac0 legacy=0x7fc0d00a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:36.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cf7fe700 1 --2- 172.21.15.121:0/3569439087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c00038e0 0x7fc0c0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:36.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cf7fe700 1 --2- 172.21.15.121:0/3569439087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0d01511f0 0x7fc0c0004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:36.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cf7fe700 1 --2- 172.21.15.121:0/3569439087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c0004b80 0x7fc0c0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:36.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cf7fe700 1 -- 172.21.15.121:0/3569439087 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d00660e0 con 0x7fc0c0004b80 2023-11-15T02:57:36.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cf7fe700 1 -- 172.21.15.121:0/3569439087 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d00bd390 con 0x7fc0c00038e0 2023-11-15T02:57:36.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cf7fe700 1 -- 172.21.15.121:0/3569439087 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c000b8e0 con 0x7fc0d01511f0 2023-11-15T02:57:36.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0d50e3700 1 --2- 172.21.15.121:0/3569439087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c0004b80 0x7fc0c0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:36.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0d48e2700 1 --2- 172.21.15.121:0/3569439087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c00038e0 0x7fc0c0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:36.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cffff700 1 --2- 172.21.15.121:0/3569439087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0d01511f0 0x7fc0c0004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:36.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0d50e3700 1 -- 172.21.15.121:0/3569439087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0d01511f0 msgr2=0x7fc0c0004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:36.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0d50e3700 1 --2- 172.21.15.121:0/3569439087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0d01511f0 0x7fc0c0004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:36.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0d50e3700 1 -- 172.21.15.121:0/3569439087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c00038e0 msgr2=0x7fc0c0003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:36.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0d50e3700 1 --2- 172.21.15.121:0/3569439087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c00038e0 0x7fc0c0003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:36.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0d50e3700 1 -- 172.21.15.121:0/3569439087 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c00033f0 con 0x7fc0c0004b80 2023-11-15T02:57:36.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0d50e3700 1 --2- 172.21.15.121:0/3569439087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c0004b80 0x7fc0c0005000 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fc0c800e740 tx=0x7fc0c8002210).ready entity=mon.0 client_cookie=9e2e4615673fb083 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:36.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cf7fe700 1 -- 172.21.15.121:0/3569439087 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8007de0 con 0x7fc0c0004b80 2023-11-15T02:57:36.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cf7fe700 1 -- 172.21.15.121:0/3569439087 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c8005b20 con 0x7fc0c0004b80 2023-11-15T02:57:36.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.287+0000 7fc0cf7fe700 1 -- 172.21.15.121:0/3569439087 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c80070d0 con 0x7fc0c0004b80 2023-11-15T02:57:36.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.291+0000 7fc0d58e4700 1 -- 172.21.15.121:0/3569439087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c0004b80 msgr2=0x7fc0c0005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:36.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.291+0000 7fc0d58e4700 1 --2- 172.21.15.121:0/3569439087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c0004b80 0x7fc0c0005000 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fc0c800e740 tx=0x7fc0c8002210).stop 2023-11-15T02:57:36.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.291+0000 7fc0d50e3700 1 -- 172.21.15.121:0/3569439087 reap_dead start 2023-11-15T02:57:36.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.291+0000 7fc0d58e4700 1 -- 172.21.15.121:0/3569439087 shutdown_connections 2023-11-15T02:57:36.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.291+0000 7fc0d58e4700 1 -- 172.21.15.121:0/3569439087 >> 172.21.15.121:0/3569439087 conn(0x7fc0d0067a60 msgr2=0x7fc0d0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:36.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.291+0000 7fc0d58e4700 1 -- 172.21.15.121:0/3569439087 shutdown_connections 2023-11-15T02:57:36.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.291+0000 7fc0d58e4700 1 -- 172.21.15.121:0/3569439087 wait complete. 2023-11-15T02:57:36.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.291+0000 7fc0d58e4700 1 Processor -- start 2023-11-15T02:57:36.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.291+0000 7fc0d58e4700 1 -- start start 2023-11-15T02:57:36.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0d58e4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c0004b80 0x7fc0d00abca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:36.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0d58e4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0d014fca0 0x7fc0d00aa220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:36.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0d58e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0d0150120 0x7fc0d0151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:36.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0d58e4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d00ac190 con 0x7fc0c0004b80 2023-11-15T02:57:36.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0d58e4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d00ac370 con 0x7fc0d014fca0 2023-11-15T02:57:36.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0d58e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d00a9740 con 0x7fc0d0150120 2023-11-15T02:57:36.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0d48e2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c0004b80 0x7fc0d00abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:36.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0d48e2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c0004b80 0x7fc0d00abca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36716/0 (socket says 172.21.15.121:36716) 2023-11-15T02:57:36.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0d48e2700 1 -- 172.21.15.121:0/116179745 learned_addr learned my addr 172.21.15.121:0/116179745 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:36.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0d50e3700 1 --2- 172.21.15.121:0/116179745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0d0150120 0x7fc0d0151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:36.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0cffff700 1 --2- 172.21.15.121:0/116179745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0d014fca0 0x7fc0d00aa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:36.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0d48e2700 1 -- 172.21.15.121:0/116179745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0d0150120 msgr2=0x7fc0d0151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:36.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0d48e2700 1 --2- 172.21.15.121:0/116179745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0d0150120 0x7fc0d0151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:36.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0d48e2700 1 -- 172.21.15.121:0/116179745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0d014fca0 msgr2=0x7fc0d00aa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:36.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0d48e2700 1 --2- 172.21.15.121:0/116179745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0d014fca0 0x7fc0d00aa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:36.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0d48e2700 1 -- 172.21.15.121:0/116179745 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d00a9a20 con 0x7fc0c0004b80 2023-11-15T02:57:36.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0d48e2700 1 --2- 172.21.15.121:0/116179745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c0004b80 0x7fc0d00abca0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fc0c40119c0 tx=0x7fc0c4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:36.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0ccff9700 1 -- 172.21.15.121:0/116179745 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c400f8e0 con 0x7fc0c0004b80 2023-11-15T02:57:36.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0ccff9700 1 -- 172.21.15.121:0/116179745 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c400fa40 con 0x7fc0c0004b80 2023-11-15T02:57:36.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0ccff9700 1 -- 172.21.15.121:0/116179745 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c400fdc0 con 0x7fc0c0004b80 2023-11-15T02:57:36.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0d58e4700 1 -- 172.21.15.121:0/116179745 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d00a9d00 con 0x7fc0c0004b80 2023-11-15T02:57:36.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.295+0000 7fc0d58e4700 1 -- 172.21.15.121:0/116179745 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d00aa8b0 con 0x7fc0c0004b80 2023-11-15T02:57:36.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.299+0000 7fc0d58e4700 1 -- 172.21.15.121:0/116179745 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d006bcf0 con 0x7fc0c0004b80 2023-11-15T02:57:36.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.299+0000 7fc0ccff9700 1 -- 172.21.15.121:0/116179745 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc0c4014650 con 0x7fc0c0004b80 2023-11-15T02:57:36.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.299+0000 7fc0ccff9700 1 --2- 172.21.15.121:0/116179745 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc0b405a540 0x7fc0b405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:36.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.299+0000 7fc0ccff9700 1 -- 172.21.15.121:0/116179745 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5355+0+0 (secure 0 0 0) 0x7fc0c40750d0 con 0x7fc0c0004b80 2023-11-15T02:57:36.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.299+0000 7fc0cffff700 1 --2- 172.21.15.121:0/116179745 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc0b405a540 0x7fc0b405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:36.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.303+0000 7fc0cffff700 1 --2- 172.21.15.121:0/116179745 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc0b405a540 0x7fc0b405ca00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fc0bc006f10 tx=0x7fc0bc008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:36.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.307+0000 7fc0ccff9700 1 -- 172.21.15.121:0/116179745 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc0c4048a90 con 0x7fc0c0004b80 2023-11-15T02:57:36.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:36.479+0000 7fc0d58e4700 1 -- 172.21.15.121:0/116179745 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc0d0068a70 con 0x7fc0c0004b80 2023-11-15T02:57:37.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.267+0000 7fc0ccff9700 1 -- 172.21.15.121:0/116179745 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc0c404c0b0 con 0x7fc0c0004b80 2023-11-15T02:57:37.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.271+0000 7fc0d58e4700 1 -- 172.21.15.121:0/116179745 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc0b405a540 msgr2=0x7fc0b405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:37.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.271+0000 7fc0d58e4700 1 --2- 172.21.15.121:0/116179745 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc0b405a540 0x7fc0b405ca00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fc0bc006f10 tx=0x7fc0bc008040).stop 2023-11-15T02:57:37.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.271+0000 7fc0d58e4700 1 -- 172.21.15.121:0/116179745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c0004b80 msgr2=0x7fc0d00abca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:37.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.271+0000 7fc0d58e4700 1 --2- 172.21.15.121:0/116179745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c0004b80 0x7fc0d00abca0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fc0c40119c0 tx=0x7fc0c4009060).stop 2023-11-15T02:57:37.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.271+0000 7fc0d58e4700 1 -- 172.21.15.121:0/116179745 shutdown_connections 2023-11-15T02:57:37.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.271+0000 7fc0d58e4700 1 --2- 172.21.15.121:0/116179745 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc0b405a540 0x7fc0b405ca00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:37.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.271+0000 7fc0d58e4700 1 --2- 172.21.15.121:0/116179745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0d0150120 0x7fc0d0151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:37.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.271+0000 7fc0d58e4700 1 --2- 172.21.15.121:0/116179745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0d014fca0 0x7fc0d00aa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:37.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.271+0000 7fc0d58e4700 1 --2- 172.21.15.121:0/116179745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c0004b80 0x7fc0d00abca0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:37.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.271+0000 7fc0d58e4700 1 -- 172.21.15.121:0/116179745 >> 172.21.15.121:0/116179745 conn(0x7fc0d0067a60 msgr2=0x7fc0d00c2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:37.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.271+0000 7fc0d58e4700 1 -- 172.21.15.121:0/116179745 shutdown_connections 2023-11-15T02:57:37.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.271+0000 7fc0d58e4700 1 -- 172.21.15.121:0/116179745 wait complete. 2023-11-15T02:57:37.278 INFO:teuthology.orchestra.run.smithi121.stderr: removed all blocklist entries 2023-11-15T02:57:37.297 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-15T02:57:37.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.455+0000 7f419c61b700 1 Processor -- start 2023-11-15T02:57:37.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.455+0000 7f419c61b700 1 -- start start 2023-11-15T02:57:37.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.459+0000 7f419c61b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4194150ff0 0x7f4194151470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:37.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.459+0000 7f419c61b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f41940c6640 0x7f41940c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:37.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.459+0000 7f419c61b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4194152000 con 0x7f419414af80 2023-11-15T02:57:37.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.459+0000 7f419c61b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41940c1270 con 0x7f41940c6640 2023-11-15T02:57:37.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.459+0000 7f419c61b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41940c6fc0 con 0x7f4194150ff0 2023-11-15T02:57:37.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.459+0000 7f419abb8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f41940c6640 0x7f41940c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:37.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.459+0000 7f419abb8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f41940c6640 0x7f41940c6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36730/0 (socket says 172.21.15.121:36730) 2023-11-15T02:57:37.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.459+0000 7f419abb8700 1 -- 172.21.15.121:0/4233125820 learned_addr learned my addr 172.21.15.121:0/4233125820 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:37.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.459+0000 7f4199bb6700 1 --2- 172.21.15.121:0/4233125820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4194150ff0 0x7f4194151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:37.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.459+0000 7f419abb8700 1 -- 172.21.15.121:0/4233125820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4194150ff0 msgr2=0x7f4194151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:37.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.459+0000 7f419abb8700 1 --2- 172.21.15.121:0/4233125820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4194150ff0 0x7f4194151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:37.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.459+0000 7f419abb8700 1 -- 172.21.15.121:0/4233125820 >> v1:172.21.15.190:6789/0 conn(0x7f419414af80 legacy=0x7f41940c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:37.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.459+0000 7f419abb8700 1 -- 172.21.15.121:0/4233125820 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41940c1e10 con 0x7f41940c6640 2023-11-15T02:57:37.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.459+0000 7f4199bb6700 1 --2- 172.21.15.121:0/4233125820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4194150ff0 0x7f4194151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:57:37.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.459+0000 7f419abb8700 1 --2- 172.21.15.121:0/4233125820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41940c6640 0x7f41940c6ad0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f418c009f20 tx=0x7f418c009840).ready entity=mon.0 client_cookie=4cdae4223a04ad42 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:37.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.463+0000 7f41993b5700 1 -- 172.21.15.121:0/4233125820 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c00e070 con 0x7f41940c6640 2023-11-15T02:57:37.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.463+0000 7f41993b5700 1 -- 172.21.15.121:0/4233125820 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418c00ba80 con 0x7f41940c6640 2023-11-15T02:57:37.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.463+0000 7f41993b5700 1 -- 172.21.15.121:0/4233125820 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c00be00 con 0x7f41940c6640 2023-11-15T02:57:37.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.463+0000 7f419c61b700 1 -- 172.21.15.121:0/4233125820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41940c6640 msgr2=0x7f41940c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:37.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.463+0000 7f419c61b700 1 --2- 172.21.15.121:0/4233125820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41940c6640 0x7f41940c6ad0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f418c009f20 tx=0x7f418c009840).stop 2023-11-15T02:57:37.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.463+0000 7f419c61b700 1 -- 172.21.15.121:0/4233125820 shutdown_connections 2023-11-15T02:57:37.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.463+0000 7f419c61b700 1 --2- 172.21.15.121:0/4233125820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41940c6640 0x7f41940c6ad0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:37.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.463+0000 7f419c61b700 1 --2- 172.21.15.121:0/4233125820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4194150ff0 0x7f4194151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:37.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.463+0000 7f419c61b700 1 -- 172.21.15.121:0/4233125820 >> 172.21.15.121:0/4233125820 conn(0x7f4194062190 msgr2=0x7f41940625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:37.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.463+0000 7f419c61b700 1 -- 172.21.15.121:0/4233125820 shutdown_connections 2023-11-15T02:57:37.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.463+0000 7f419c61b700 1 -- 172.21.15.121:0/4233125820 wait complete. 2023-11-15T02:57:37.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.467+0000 7f419c61b700 1 Processor -- start 2023-11-15T02:57:37.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.467+0000 7f419c61b700 1 -- start start 2023-11-15T02:57:37.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.467+0000 7f419c61b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41940c6640 0x7f41940abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:37.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.467+0000 7f419c61b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f419414af80 0x7f419414fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:37.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.467+0000 7f419c61b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4194150ff0 0x7f41941502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:37.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.467+0000 7f419c61b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41941507a0 con 0x7f4194150ff0 2023-11-15T02:57:37.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.467+0000 7f419c61b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4194155300 con 0x7f419414af80 2023-11-15T02:57:37.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.467+0000 7f419c61b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41941546f0 con 0x7f41940c6640 2023-11-15T02:57:37.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.467+0000 7f419abb8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4194150ff0 0x7f41941502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:37.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.467+0000 7f419abb8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4194150ff0 0x7f41941502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36742/0 (socket says 172.21.15.121:36742) 2023-11-15T02:57:37.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.467+0000 7f419abb8700 1 -- 172.21.15.121:0/1721327728 learned_addr learned my addr 172.21.15.121:0/1721327728 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:37.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.467+0000 7f419a3b7700 1 --2- 172.21.15.121:0/1721327728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41940c6640 0x7f41940abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:37.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.467+0000 7f4199bb6700 1 --2- 172.21.15.121:0/1721327728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f419414af80 0x7f419414fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:37.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.467+0000 7f419abb8700 1 -- 172.21.15.121:0/1721327728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41940c6640 msgr2=0x7f41940abfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:37.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.467+0000 7f419abb8700 1 --2- 172.21.15.121:0/1721327728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41940c6640 0x7f41940abfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:37.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.467+0000 7f419abb8700 1 -- 172.21.15.121:0/1721327728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f419414af80 msgr2=0x7f419414fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:37.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.467+0000 7f419abb8700 1 --2- 172.21.15.121:0/1721327728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f419414af80 0x7f419414fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:37.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.467+0000 7f419abb8700 1 -- 172.21.15.121:0/1721327728 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418c009ae0 con 0x7f4194150ff0 2023-11-15T02:57:37.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.467+0000 7f4199bb6700 1 --2- 172.21.15.121:0/1721327728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f419414af80 0x7f419414fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:37.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.471+0000 7f419abb8700 1 --2- 172.21.15.121:0/1721327728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4194150ff0 0x7f41941502b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f418c00bf20 tx=0x7f418c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:37.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.471+0000 7f418a7fc700 1 -- 172.21.15.121:0/1721327728 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c00b6f0 con 0x7f4194150ff0 2023-11-15T02:57:37.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.471+0000 7f418a7fc700 1 -- 172.21.15.121:0/1721327728 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418c01e470 con 0x7f4194150ff0 2023-11-15T02:57:37.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.471+0000 7f419c61b700 1 -- 172.21.15.121:0/1721327728 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41941549d0 con 0x7f4194150ff0 2023-11-15T02:57:37.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.471+0000 7f418a7fc700 1 -- 172.21.15.121:0/1721327728 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c01e7f0 con 0x7f4194150ff0 2023-11-15T02:57:37.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.471+0000 7f419c61b700 1 -- 172.21.15.121:0/1721327728 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4194154f90 con 0x7f4194150ff0 2023-11-15T02:57:37.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.471+0000 7f419c61b700 1 -- 172.21.15.121:0/1721327728 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4194155490 con 0x7f4194150ff0 2023-11-15T02:57:37.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.471+0000 7f418a7fc700 1 -- 172.21.15.121:0/1721327728 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f418c003040 con 0x7f4194150ff0 2023-11-15T02:57:37.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.475+0000 7f418a7fc700 1 --2- 172.21.15.121:0/1721327728 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f417805a540 0x7f417805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:37.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.475+0000 7f418a7fc700 1 -- 172.21.15.121:0/1721327728 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f418c075b10 con 0x7f4194150ff0 2023-11-15T02:57:37.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.475+0000 7f419a3b7700 1 --2- 172.21.15.121:0/1721327728 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f417805a540 0x7f417805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:37.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.475+0000 7f419a3b7700 1 --2- 172.21.15.121:0/1721327728 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f417805a540 0x7f417805ca00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f419414d530 tx=0x7f4190009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:37.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.479+0000 7f418a7fc700 1 -- 172.21.15.121:0/1721327728 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f418c0496f0 con 0x7f4194150ff0 2023-11-15T02:57:37.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.703+0000 7f419c61b700 1 -- 172.21.15.121:0/1721327728 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4194151a40 con 0x7f4194150ff0 2023-11-15T02:57:37.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.707+0000 7f418a7fc700 1 -- 172.21.15.121:0/1721327728 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f418c023030 con 0x7f4194150ff0 2023-11-15T02:57:37.712 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:37.712 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T02:57:37.713 INFO:teuthology.orchestra.run.smithi121.stdout: "auth_dump": [ 2023-11-15T02:57:37.713 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.713 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.a", 2023-11-15T02:57:37.713 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1Rlzy3zOhAAeJ3COoWQP/Kw+P/0AwHbwA==", 2023-11-15T02:57:37.714 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.714 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:57:37.714 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T02:57:37.714 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T02:57:37.715 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:57:37.715 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.715 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.715 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.716 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.b", 2023-11-15T02:57:37.716 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlUglaERAAC6WUS9ftCfr+KbPpiEIyqw==", 2023-11-15T02:57:37.716 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.716 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:57:37.716 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T02:57:37.717 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T02:57:37.717 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:57:37.717 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.717 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.717 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.718 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.c", 2023-11-15T02:57:37.718 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RldFcvCRAATIouBNyHGip5hFrH+z4v5A==", 2023-11-15T02:57:37.718 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.718 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:57:37.719 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T02:57:37.719 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T02:57:37.719 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:57:37.719 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.720 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.720 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.720 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.d", 2023-11-15T02:57:37.720 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1Rlg5iZGxAADwdL2RrtU0kDeb9EmNJU6A==", 2023-11-15T02:57:37.721 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.721 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:57:37.721 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T02:57:37.721 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T02:57:37.721 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:57:37.722 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.722 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.722 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.722 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.0", 2023-11-15T02:57:37.723 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAzM1RldPH9CBAAUCF/a47aFbbNsWV1r+u3Tw==", 2023-11-15T02:57:37.723 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.723 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:57:37.723 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:57:37.723 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:57:37.724 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.724 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.724 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.724 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.1", 2023-11-15T02:57:37.724 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQA2M1RltoxwMhAAZN3bgOjSaoK3ZxE5ezZ3IA==", 2023-11-15T02:57:37.725 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.725 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:57:37.725 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:57:37.725 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:57:37.725 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.726 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.726 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.726 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.2", 2023-11-15T02:57:37.726 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQA6M1RlgltjHxAAx/m5gm+qs8LiEFyZL+uLzQ==", 2023-11-15T02:57:37.727 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.727 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:57:37.727 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:57:37.727 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:57:37.727 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.728 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.728 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.728 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.3", 2023-11-15T02:57:37.728 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQA+M1RlhtQcDRAA+g11PmSNKazpk+VoK17EEQ==", 2023-11-15T02:57:37.728 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.729 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:57:37.729 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:57:37.729 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:57:37.729 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.729 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.730 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.730 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.4", 2023-11-15T02:57:37.730 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBFM1RlJ7hnBBAAjskKfD63N/61T4GSPqOiRg==", 2023-11-15T02:57:37.730 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.730 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:57:37.731 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:57:37.731 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:57:37.731 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.731 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.732 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.732 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.5", 2023-11-15T02:57:37.732 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBIM1RlI1HPLRAAgVbiNq1tmgK6PaWP+c2JFw==", 2023-11-15T02:57:37.732 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.732 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:57:37.733 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:57:37.733 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:57:37.733 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.733 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.733 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.734 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.6", 2023-11-15T02:57:37.734 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBMM1RlPDshDRAAVgIqxklHSQcFb5sM6GUfyQ==", 2023-11-15T02:57:37.734 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.734 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:57:37.734 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:57:37.735 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:57:37.735 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.735 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.735 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.735 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.7", 2023-11-15T02:57:37.736 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBPM1RlN8IuJxAA2rN2848kaB2eOM6zDb+eJg==", 2023-11-15T02:57:37.736 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.736 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:57:37.736 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:57:37.737 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:57:37.737 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.737 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.737 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.737 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.0", 2023-11-15T02:57:37.738 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlfHPEIRAAJ4YV46zzBQ7v7y0kD4P2ZA==", 2023-11-15T02:57:37.738 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.738 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:57:37.738 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2023-11-15T02:57:37.739 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2023-11-15T02:57:37.739 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2023-11-15T02:57:37.739 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.739 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.739 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.740 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.1", 2023-11-15T02:57:37.740 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlibaULBAA+y1EyhVOt+wXSk2uRajJ3g==", 2023-11-15T02:57:37.740 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.740 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:57:37.740 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2023-11-15T02:57:37.741 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2023-11-15T02:57:37.741 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2023-11-15T02:57:37.741 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.741 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.741 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.742 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.2", 2023-11-15T02:57:37.742 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlHHbvJhAAX5TE6UvcpBeJPSCZkfqD8w==", 2023-11-15T02:57:37.742 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.742 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:57:37.742 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2023-11-15T02:57:37.743 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2023-11-15T02:57:37.743 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2023-11-15T02:57:37.743 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.743 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.743 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.744 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.3", 2023-11-15T02:57:37.744 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlQtQXMhAAGAvtnvJLSoUKuEiZDsSxww==", 2023-11-15T02:57:37.744 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.744 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:57:37.745 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2023-11-15T02:57:37.745 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2023-11-15T02:57:37.745 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2023-11-15T02:57:37.745 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.745 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.746 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.746 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.admin", 2023-11-15T02:57:37.746 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1Rl67qqEhAAI48kZ9D7T807bTVoUmwajg==", 2023-11-15T02:57:37.746 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.746 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2023-11-15T02:57:37.747 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T02:57:37.747 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T02:57:37.747 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:57:37.747 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.747 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.748 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.748 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.bootstrap-mds", 2023-11-15T02:57:37.748 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBhM1RlcK9EBBAAqo0mlbMjkeYDpbHwK6ikUw==", 2023-11-15T02:57:37.748 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.748 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile bootstrap-mds" 2023-11-15T02:57:37.749 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.749 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.749 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.749 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.bootstrap-mgr", 2023-11-15T02:57:37.749 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBhM1Rlr75EBBAAaQ6weRQ8gjf19YPAYP6gfg==", 2023-11-15T02:57:37.750 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.750 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-15T02:57:37.750 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.750 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.751 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.751 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.bootstrap-osd", 2023-11-15T02:57:37.751 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBhM1Rl/cxEBBAAltMS49gNSpCiD67jFQ0cdA==", 2023-11-15T02:57:37.751 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.751 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile bootstrap-osd" 2023-11-15T02:57:37.752 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.752 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.752 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.752 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.bootstrap-rbd", 2023-11-15T02:57:37.752 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBhM1RlyNpEBBAAvbMpcx/xvJg4E0e0exdY9Q==", 2023-11-15T02:57:37.753 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.753 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-15T02:57:37.753 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.753 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.753 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.754 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-15T02:57:37.754 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBhM1RlO+xEBBAA+Ntq1aIj0QCW76qU5R7LmQ==", 2023-11-15T02:57:37.754 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.754 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-15T02:57:37.754 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.755 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.755 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.755 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.bootstrap-rgw", 2023-11-15T02:57:37.755 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBhM1Rlbf1EBBAAk1nvhI1gL4xEIp4VOTISzA==", 2023-11-15T02:57:37.755 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.756 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-15T02:57:37.756 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.756 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.756 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.756 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mgr.x", 2023-11-15T02:57:37.756 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1RlJQFPNRAAy7E5mrn8Xh98339fWSkX9Q==", 2023-11-15T02:57:37.757 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.757 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2023-11-15T02:57:37.757 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile mgr", 2023-11-15T02:57:37.757 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:57:37.757 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.758 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:37.758 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:37.758 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mgr.y", 2023-11-15T02:57:37.758 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1RldQJlLhAAbc46liEP03yBNeVhZCb7IQ==", 2023-11-15T02:57:37.758 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:57:37.759 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2023-11-15T02:57:37.759 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile mgr", 2023-11-15T02:57:37.759 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:57:37.759 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.760 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:37.760 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2023-11-15T02:57:37.760 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T02:57:37.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.711+0000 7f419c61b700 1 -- 172.21.15.121:0/1721327728 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f417805a540 msgr2=0x7f417805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:37.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.711+0000 7f419c61b700 1 --2- 172.21.15.121:0/1721327728 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f417805a540 0x7f417805ca00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f419414d530 tx=0x7f4190009040).stop 2023-11-15T02:57:37.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.711+0000 7f419c61b700 1 -- 172.21.15.121:0/1721327728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4194150ff0 msgr2=0x7f41941502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:37.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.711+0000 7f419c61b700 1 --2- 172.21.15.121:0/1721327728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4194150ff0 0x7f41941502b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f418c00bf20 tx=0x7f418c009840).stop 2023-11-15T02:57:37.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.711+0000 7f419c61b700 1 -- 172.21.15.121:0/1721327728 shutdown_connections 2023-11-15T02:57:37.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.711+0000 7f419c61b700 1 --2- 172.21.15.121:0/1721327728 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f417805a540 0x7f417805ca00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:37.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.711+0000 7f419c61b700 1 --2- 172.21.15.121:0/1721327728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4194150ff0 0x7f41941502b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:37.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.711+0000 7f419c61b700 1 --2- 172.21.15.121:0/1721327728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f419414af80 0x7f419414fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:37.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.711+0000 7f419c61b700 1 --2- 172.21.15.121:0/1721327728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41940c6640 0x7f41940abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:37.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.711+0000 7f419c61b700 1 -- 172.21.15.121:0/1721327728 >> 172.21.15.121:0/1721327728 conn(0x7f4194062190 msgr2=0x7f4194149e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:37.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.711+0000 7f419c61b700 1 -- 172.21.15.121:0/1721327728 shutdown_connections 2023-11-15T02:57:37.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.711+0000 7f419c61b700 1 -- 172.21.15.121:0/1721327728 wait complete. 2023-11-15T02:57:37.764 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-15T02:57:37.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.919+0000 7f2e5001b700 1 Processor -- start 2023-11-15T02:57:37.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.919+0000 7f2e5001b700 1 -- start start 2023-11-15T02:57:37.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.919+0000 7f2e5001b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2e48062330 0x7f2e48062710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:37.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.919+0000 7f2e5001b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e480ac150 0x7f2e4814ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:37.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.919+0000 7f2e5001b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e480aac50 con 0x7f2e480aa130 2023-11-15T02:57:37.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.919+0000 7f2e5001b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e480aa6b0 con 0x7f2e48062330 2023-11-15T02:57:37.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.919+0000 7f2e5001b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e480aa970 con 0x7f2e480ac150 2023-11-15T02:57:37.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e4e5b8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e480ac150 0x7f2e4814ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:37.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e4e5b8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e480ac150 0x7f2e4814ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:37658/0 (socket says 172.21.15.121:37658) 2023-11-15T02:57:37.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e4e5b8700 1 -- 172.21.15.121:0/3072832124 learned_addr learned my addr 172.21.15.121:0/3072832124 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:37.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e4ddb7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2e48062330 0x7f2e48062710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:37.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e4d5b6700 1 --1- 172.21.15.121:0/3072832124 >> v1:172.21.15.190:6789/0 conn(0x7f2e480aa130 0x7f2e48154c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:52178/0 (socket says 172.21.15.121:52178) 2023-11-15T02:57:37.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e4e5b8700 1 -- 172.21.15.121:0/3072832124 >> v2:172.21.15.190:3300/0 conn(0x7f2e48062330 msgr2=0x7f2e48062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:37.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e4e5b8700 1 --2- 172.21.15.121:0/3072832124 >> v2:172.21.15.190:3300/0 conn(0x7f2e48062330 0x7f2e48062710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:37.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e4cdb5700 1 -- 172.21.15.121:0/3072832124 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684171719 0 0) 0x7f2e480aac50 con 0x7f2e480aa130 2023-11-15T02:57:37.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e4e5b8700 1 -- 172.21.15.121:0/3072832124 >> v1:172.21.15.190:6789/0 conn(0x7f2e480aa130 legacy=0x7f2e48154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:37.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e4e5b8700 1 -- 172.21.15.121:0/3072832124 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e48154410 con 0x7f2e480ac150 2023-11-15T02:57:37.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e4e5b8700 1 --2- 172.21.15.121:0/3072832124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e480ac150 0x7f2e4814ddd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f2e40009f20 tx=0x7f2e40009840).ready entity=mon.2 client_cookie=f83099131c64c935 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:37.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e4cdb5700 1 -- 172.21.15.121:0/3072832124 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4000e070 con 0x7f2e480ac150 2023-11-15T02:57:37.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e4cdb5700 1 -- 172.21.15.121:0/3072832124 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4000b9d0 con 0x7f2e480ac150 2023-11-15T02:57:37.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e4cdb5700 1 -- 172.21.15.121:0/3072832124 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4000bd50 con 0x7f2e480ac150 2023-11-15T02:57:37.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e5001b700 1 -- 172.21.15.121:0/3072832124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e480ac150 msgr2=0x7f2e4814ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:37.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e5001b700 1 --2- 172.21.15.121:0/3072832124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e480ac150 0x7f2e4814ddd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f2e40009f20 tx=0x7f2e40009840).stop 2023-11-15T02:57:37.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e5001b700 1 -- 172.21.15.121:0/3072832124 shutdown_connections 2023-11-15T02:57:37.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e5001b700 1 --2- 172.21.15.121:0/3072832124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e480ac150 0x7f2e4814ddd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:37.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e5001b700 1 --2- 172.21.15.121:0/3072832124 >> v2:172.21.15.190:3300/0 conn(0x7f2e48062330 0x7f2e48062710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:37.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.923+0000 7f2e5001b700 1 -- 172.21.15.121:0/3072832124 >> 172.21.15.121:0/3072832124 conn(0x7f2e48067a60 msgr2=0x7f2e48067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:37.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.927+0000 7f2e5001b700 1 -- 172.21.15.121:0/3072832124 shutdown_connections 2023-11-15T02:57:37.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.927+0000 7f2e5001b700 1 -- 172.21.15.121:0/3072832124 wait complete. 2023-11-15T02:57:37.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.927+0000 7f2e5001b700 1 Processor -- start 2023-11-15T02:57:37.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.927+0000 7f2e5001b700 1 -- start start 2023-11-15T02:57:37.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.927+0000 7f2e5001b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e48062330 0x7f2e480c3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:37.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.927+0000 7f2e5001b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e480aa130 0x7f2e48159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:37.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.927+0000 7f2e5001b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e480ac150 0x7f2e480c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:37.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.927+0000 7f2e5001b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e480c3920 con 0x7f2e480aa130 2023-11-15T02:57:37.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.927+0000 7f2e5001b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4815a420 con 0x7f2e48062330 2023-11-15T02:57:37.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.927+0000 7f2e5001b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e480c20c0 con 0x7f2e480ac150 2023-11-15T02:57:37.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.931+0000 7f2e4d5b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e480aa130 0x7f2e48159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:37.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.931+0000 7f2e4d5b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e480aa130 0x7f2e48159f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36752/0 (socket says 172.21.15.121:36752) 2023-11-15T02:57:37.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.931+0000 7f2e4d5b6700 1 -- 172.21.15.121:0/1658262094 learned_addr learned my addr 172.21.15.121:0/1658262094 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:37.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.931+0000 7f2e4e5b8700 1 --2- 172.21.15.121:0/1658262094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e480ac150 0x7f2e480c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:37.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.931+0000 7f2e4ddb7700 1 --2- 172.21.15.121:0/1658262094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e48062330 0x7f2e480c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:37.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.931+0000 7f2e4d5b6700 1 -- 172.21.15.121:0/1658262094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e480ac150 msgr2=0x7f2e480c1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:37.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.931+0000 7f2e4d5b6700 1 --2- 172.21.15.121:0/1658262094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e480ac150 0x7f2e480c1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:37.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.931+0000 7f2e4d5b6700 1 -- 172.21.15.121:0/1658262094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e48062330 msgr2=0x7f2e480c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:37.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.931+0000 7f2e4d5b6700 1 --2- 172.21.15.121:0/1658262094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e48062330 0x7f2e480c3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:37.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.931+0000 7f2e4d5b6700 1 -- 172.21.15.121:0/1658262094 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e40009ae0 con 0x7f2e480aa130 2023-11-15T02:57:37.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.931+0000 7f2e4d5b6700 1 --2- 172.21.15.121:0/1658262094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e480aa130 0x7f2e48159f30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f2e3800c590 tx=0x7f2e38007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:37.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.931+0000 7f2e3dffb700 1 -- 172.21.15.121:0/1658262094 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3800cea0 con 0x7f2e480aa130 2023-11-15T02:57:37.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.931+0000 7f2e3dffb700 1 -- 172.21.15.121:0/1658262094 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e380072e0 con 0x7f2e480aa130 2023-11-15T02:57:37.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.931+0000 7f2e3dffb700 1 -- 172.21.15.121:0/1658262094 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380056e0 con 0x7f2e480aa130 2023-11-15T02:57:37.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.931+0000 7f2e5001b700 1 -- 172.21.15.121:0/1658262094 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e480c2390 con 0x7f2e480aa130 2023-11-15T02:57:37.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.931+0000 7f2e5001b700 1 -- 172.21.15.121:0/1658262094 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e480c2950 con 0x7f2e480aa130 2023-11-15T02:57:37.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.935+0000 7f2e5001b700 1 -- 172.21.15.121:0/1658262094 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e48047060 con 0x7f2e480aa130 2023-11-15T02:57:37.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.935+0000 7f2e3dffb700 1 -- 172.21.15.121:0/1658262094 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2e38005ad0 con 0x7f2e480aa130 2023-11-15T02:57:37.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.935+0000 7f2e3dffb700 1 --2- 172.21.15.121:0/1658262094 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2e2c05a540 0x7f2e2c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:37.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.935+0000 7f2e3dffb700 1 -- 172.21.15.121:0/1658262094 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f2e38073ba0 con 0x7f2e480aa130 2023-11-15T02:57:37.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.935+0000 7f2e4ddb7700 1 --2- 172.21.15.121:0/1658262094 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2e2c05a540 0x7f2e2c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:37.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.935+0000 7f2e4ddb7700 1 --2- 172.21.15.121:0/1658262094 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2e2c05a540 0x7f2e2c05ca00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f2e44009c30 tx=0x7f2e44009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:37.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:37.943+0000 7f2e3dffb700 1 -- 172.21.15.121:0/1658262094 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2e38047610 con 0x7f2e480aa130 2023-11-15T02:57:38.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.167+0000 7f2e5001b700 1 -- 172.21.15.121:0/1658262094 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f2e48068b20 con 0x7f2e480aa130 2023-11-15T02:57:38.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.175+0000 7f2e3dffb700 1 -- 172.21.15.121:0/1658262094 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2e3804ac30 con 0x7f2e480aa130 2023-11-15T02:57:38.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.179+0000 7f2e5001b700 1 -- 172.21.15.121:0/1658262094 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2e2c05a540 msgr2=0x7f2e2c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:38.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.179+0000 7f2e5001b700 1 --2- 172.21.15.121:0/1658262094 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2e2c05a540 0x7f2e2c05ca00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f2e44009c30 tx=0x7f2e44009060).stop 2023-11-15T02:57:38.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.179+0000 7f2e5001b700 1 -- 172.21.15.121:0/1658262094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e480aa130 msgr2=0x7f2e48159f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:38.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.179+0000 7f2e5001b700 1 --2- 172.21.15.121:0/1658262094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e480aa130 0x7f2e48159f30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f2e3800c590 tx=0x7f2e38007c90).stop 2023-11-15T02:57:38.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.179+0000 7f2e5001b700 1 -- 172.21.15.121:0/1658262094 shutdown_connections 2023-11-15T02:57:38.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.179+0000 7f2e5001b700 1 --2- 172.21.15.121:0/1658262094 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2e2c05a540 0x7f2e2c05ca00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:38.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.179+0000 7f2e5001b700 1 --2- 172.21.15.121:0/1658262094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e480ac150 0x7f2e480c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:38.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.179+0000 7f2e5001b700 1 --2- 172.21.15.121:0/1658262094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e480aa130 0x7f2e48159f30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:38.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.179+0000 7f2e5001b700 1 --2- 172.21.15.121:0/1658262094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e48062330 0x7f2e480c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:38.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.179+0000 7f2e5001b700 1 -- 172.21.15.121:0/1658262094 >> 172.21.15.121:0/1658262094 conn(0x7f2e48067a60 msgr2=0x7f2e480c4600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:38.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.179+0000 7f2e5001b700 1 -- 172.21.15.121:0/1658262094 shutdown_connections 2023-11-15T02:57:38.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.179+0000 7f2e5001b700 1 -- 172.21.15.121:0/1658262094 wait complete. 2023-11-15T02:57:38.186 INFO:teuthology.orchestra.run.smithi121.stderr:updated 2023-11-15T02:57:38.204 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-15T02:57:38.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.359+0000 7f1d89ba9700 1 Processor -- start 2023-11-15T02:57:38.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.359+0000 7f1d89ba9700 1 -- start start 2023-11-15T02:57:38.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.363+0000 7f1d89ba9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1d841500d0 0x7f1d841504b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:38.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.363+0000 7f1d89ba9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1d84066820 0x7f1d84066c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:38.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.363+0000 7f1d89ba9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d840a98d0 con 0x7f1d84065ad0 2023-11-15T02:57:38.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.363+0000 7f1d89ba9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d840a9b70 con 0x7f1d84066820 2023-11-15T02:57:38.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.363+0000 7f1d89ba9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d840a9e50 con 0x7f1d841500d0 2023-11-15T02:57:38.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.363+0000 7f1d83fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1d84066820 0x7f1d84066c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:38.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.363+0000 7f1d83fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1d84066820 0x7f1d84066c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36754/0 (socket says 172.21.15.121:36754) 2023-11-15T02:57:38.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.363+0000 7f1d83fff700 1 -- 172.21.15.121:0/2185862483 learned_addr learned my addr 172.21.15.121:0/2185862483 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:38.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.363+0000 7f1d837fe700 1 --2- 172.21.15.121:0/2185862483 >> v2:172.21.15.190:3300/0 conn(0x7f1d841500d0 0x7f1d841504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:38.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.363+0000 7f1d83fff700 1 -- 172.21.15.121:0/2185862483 >> v2:172.21.15.190:3300/0 conn(0x7f1d841500d0 msgr2=0x7f1d841504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:38.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.363+0000 7f1d83fff700 1 --2- 172.21.15.121:0/2185862483 >> v2:172.21.15.190:3300/0 conn(0x7f1d841500d0 0x7f1d841504b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:38.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.363+0000 7f1d83fff700 1 -- 172.21.15.121:0/2185862483 >> v1:172.21.15.190:6789/0 conn(0x7f1d84065ad0 legacy=0x7f1d84065f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:38.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.363+0000 7f1d83fff700 1 -- 172.21.15.121:0/2185862483 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d840aa130 con 0x7f1d84066820 2023-11-15T02:57:38.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.363+0000 7f1d83fff700 1 --2- 172.21.15.121:0/2185862483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d84066820 0x7f1d84066c60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f1d840c7730 tx=0x7f1d7c009840).ready entity=mon.0 client_cookie=13aa2aed432846fa server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:38.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.363+0000 7f1d827fc700 1 -- 172.21.15.121:0/2185862483 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c00e070 con 0x7f1d84066820 2023-11-15T02:57:38.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.363+0000 7f1d827fc700 1 -- 172.21.15.121:0/2185862483 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7c00ba50 con 0x7f1d84066820 2023-11-15T02:57:38.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.363+0000 7f1d827fc700 1 -- 172.21.15.121:0/2185862483 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c00bdd0 con 0x7f1d84066820 2023-11-15T02:57:38.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.363+0000 7f1d89ba9700 1 -- 172.21.15.121:0/2185862483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d84066820 msgr2=0x7f1d84066c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:38.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.363+0000 7f1d89ba9700 1 --2- 172.21.15.121:0/2185862483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d84066820 0x7f1d84066c60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f1d840c7730 tx=0x7f1d7c009840).stop 2023-11-15T02:57:38.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.367+0000 7f1d89ba9700 1 -- 172.21.15.121:0/2185862483 shutdown_connections 2023-11-15T02:57:38.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.367+0000 7f1d89ba9700 1 --2- 172.21.15.121:0/2185862483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d84066820 0x7f1d84066c60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:38.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.367+0000 7f1d89ba9700 1 --2- 172.21.15.121:0/2185862483 >> v2:172.21.15.190:3300/0 conn(0x7f1d841500d0 0x7f1d841504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:38.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.367+0000 7f1d89ba9700 1 -- 172.21.15.121:0/2185862483 >> 172.21.15.121:0/2185862483 conn(0x7f1d84067a60 msgr2=0x7f1d84067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:38.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.367+0000 7f1d89ba9700 1 -- 172.21.15.121:0/2185862483 shutdown_connections 2023-11-15T02:57:38.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.367+0000 7f1d89ba9700 1 -- 172.21.15.121:0/2185862483 wait complete. 2023-11-15T02:57:38.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.367+0000 7f1d89ba9700 1 Processor -- start 2023-11-15T02:57:38.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.371+0000 7f1d89ba9700 1 -- start start 2023-11-15T02:57:38.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.371+0000 7f1d89ba9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d84065ad0 0x7f1d84143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:38.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.371+0000 7f1d89ba9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d84066820 0x7f1d84167bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:38.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.371+0000 7f1d89ba9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d841500d0 0x7f1d840c2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:38.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.371+0000 7f1d89ba9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d840c2180 con 0x7f1d841500d0 2023-11-15T02:57:38.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.371+0000 7f1d89ba9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d84168b00 con 0x7f1d84065ad0 2023-11-15T02:57:38.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.371+0000 7f1d89ba9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d84168540 con 0x7f1d84066820 2023-11-15T02:57:38.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.371+0000 7f1d83fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d841500d0 0x7f1d840c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:38.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.371+0000 7f1d837fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d84065ad0 0x7f1d84143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:38.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.371+0000 7f1d83fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d841500d0 0x7f1d840c2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36768/0 (socket says 172.21.15.121:36768) 2023-11-15T02:57:38.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.371+0000 7f1d83fff700 1 -- 172.21.15.121:0/1787808056 learned_addr learned my addr 172.21.15.121:0/1787808056 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:38.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.371+0000 7f1d82ffd700 1 --2- 172.21.15.121:0/1787808056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d84066820 0x7f1d84167bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:38.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.371+0000 7f1d83fff700 1 -- 172.21.15.121:0/1787808056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d84066820 msgr2=0x7f1d84167bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:38.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.371+0000 7f1d83fff700 1 --2- 172.21.15.121:0/1787808056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d84066820 0x7f1d84167bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:38.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.371+0000 7f1d83fff700 1 -- 172.21.15.121:0/1787808056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d84065ad0 msgr2=0x7f1d84143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:38.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.371+0000 7f1d83fff700 1 --2- 172.21.15.121:0/1787808056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d84065ad0 0x7f1d84143530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:38.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.371+0000 7f1d83fff700 1 -- 172.21.15.121:0/1787808056 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c009ae0 con 0x7f1d841500d0 2023-11-15T02:57:38.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.371+0000 7f1d837fe700 1 --2- 172.21.15.121:0/1787808056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d84065ad0 0x7f1d84143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:57:38.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.375+0000 7f1d83fff700 1 --2- 172.21.15.121:0/1787808056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d841500d0 0x7f1d840c2610 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f1d840c7760 tx=0x7f1d840a9a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:38.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.375+0000 7f1d6b7fe700 1 -- 172.21.15.121:0/1787808056 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c0092e0 con 0x7f1d841500d0 2023-11-15T02:57:38.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.375+0000 7f1d6b7fe700 1 -- 172.21.15.121:0/1787808056 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7c013e30 con 0x7f1d841500d0 2023-11-15T02:57:38.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.375+0000 7f1d89ba9700 1 -- 172.21.15.121:0/1787808056 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d84168820 con 0x7f1d841500d0 2023-11-15T02:57:38.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.375+0000 7f1d6b7fe700 1 -- 172.21.15.121:0/1787808056 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c01d620 con 0x7f1d841500d0 2023-11-15T02:57:38.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.375+0000 7f1d89ba9700 1 -- 172.21.15.121:0/1787808056 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d84143a20 con 0x7f1d841500d0 2023-11-15T02:57:38.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.375+0000 7f1d89ba9700 1 -- 172.21.15.121:0/1787808056 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d840c2330 con 0x7f1d841500d0 2023-11-15T02:57:38.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.379+0000 7f1d6b7fe700 1 -- 172.21.15.121:0/1787808056 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1d7c01da10 con 0x7f1d841500d0 2023-11-15T02:57:38.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.379+0000 7f1d6b7fe700 1 --2- 172.21.15.121:0/1787808056 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d7005a540 0x7f1d7005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:38.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.379+0000 7f1d6b7fe700 1 -- 172.21.15.121:0/1787808056 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f1d7c075e80 con 0x7f1d841500d0 2023-11-15T02:57:38.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.379+0000 7f1d837fe700 1 --2- 172.21.15.121:0/1787808056 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d7005a540 0x7f1d7005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:38.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.379+0000 7f1d837fe700 1 --2- 172.21.15.121:0/1787808056 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d7005a540 0x7f1d7005ca00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f1d7400bf10 tx=0x7f1d74008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:38.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.383+0000 7f1d6b7fe700 1 -- 172.21.15.121:0/1787808056 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1d7c0499b0 con 0x7f1d841500d0 2023-11-15T02:57:38.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.595+0000 7f1d89ba9700 1 -- 172.21.15.121:0/1787808056 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f1d84047060 con 0x7f1d841500d0 2023-11-15T02:57:38.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.603+0000 7f1d6b7fe700 1 -- 172.21.15.121:0/1787808056 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1d7c04cfd0 con 0x7f1d841500d0 2023-11-15T02:57:38.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.607+0000 7f1d89ba9700 1 -- 172.21.15.121:0/1787808056 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d7005a540 msgr2=0x7f1d7005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:38.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.607+0000 7f1d89ba9700 1 --2- 172.21.15.121:0/1787808056 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d7005a540 0x7f1d7005ca00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f1d7400bf10 tx=0x7f1d74008040).stop 2023-11-15T02:57:38.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.607+0000 7f1d89ba9700 1 -- 172.21.15.121:0/1787808056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d841500d0 msgr2=0x7f1d840c2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:38.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.607+0000 7f1d89ba9700 1 --2- 172.21.15.121:0/1787808056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d841500d0 0x7f1d840c2610 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f1d840c7760 tx=0x7f1d840a9a60).stop 2023-11-15T02:57:38.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.607+0000 7f1d89ba9700 1 -- 172.21.15.121:0/1787808056 shutdown_connections 2023-11-15T02:57:38.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.607+0000 7f1d89ba9700 1 --2- 172.21.15.121:0/1787808056 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d7005a540 0x7f1d7005ca00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:38.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.607+0000 7f1d89ba9700 1 --2- 172.21.15.121:0/1787808056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d841500d0 0x7f1d840c2610 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:38.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.607+0000 7f1d89ba9700 1 --2- 172.21.15.121:0/1787808056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d84066820 0x7f1d84167bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:38.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.607+0000 7f1d89ba9700 1 --2- 172.21.15.121:0/1787808056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d84065ad0 0x7f1d84143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:38.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.607+0000 7f1d89ba9700 1 -- 172.21.15.121:0/1787808056 >> 172.21.15.121:0/1787808056 conn(0x7f1d84067a60 msgr2=0x7f1d840c48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:38.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.607+0000 7f1d89ba9700 1 -- 172.21.15.121:0/1787808056 shutdown_connections 2023-11-15T02:57:38.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.607+0000 7f1d89ba9700 1 -- 172.21.15.121:0/1787808056 wait complete. 2023-11-15T02:57:38.614 INFO:teuthology.orchestra.run.smithi121.stderr:updated 2023-11-15T02:57:38.632 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-15T02:57:38.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.783+0000 7f2611279700 1 Processor -- start 2023-11-15T02:57:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.783+0000 7f2611279700 1 -- start start 2023-11-15T02:57:38.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f2611279700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260c1511f0 0x7f260c066570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:38.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f2611279700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f260c0bcab0 0x7f260c1515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:38.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f2611279700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f260c066a60 con 0x7f260c151ac0 2023-11-15T02:57:38.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f2611279700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f260c0660e0 con 0x7f260c0bcab0 2023-11-15T02:57:38.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f2611279700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260c0bd390 con 0x7f260c1511f0 2023-11-15T02:57:38.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f2610a78700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f260c151ac0 0x7f260c0a8750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56220/0 (socket says 172.21.15.121:56220) 2023-11-15T02:57:38.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f2610a78700 1 -- 172.21.15.121:0/2665351307 learned_addr learned my addr 172.21.15.121:0/2665351307 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:38.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f26037fe700 1 --2- 172.21.15.121:0/2665351307 >> v2:172.21.15.121:3300/0 conn(0x7f260c0bcab0 0x7f260c1515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:38.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f260b7fe700 1 -- 172.21.15.121:0/2665351307 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404562084 0 0) 0x7f260c066a60 con 0x7f260c151ac0 2023-11-15T02:57:38.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f260bfff700 1 --2- 172.21.15.121:0/2665351307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260c1511f0 0x7f260c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:38.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f260b7fe700 1 -- 172.21.15.121:0/2665351307 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25f80033f0 con 0x7f260c151ac0 2023-11-15T02:57:38.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f260b7fe700 1 -- 172.21.15.121:0/2665351307 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1969746095 0 0) 0x7f25f80033f0 con 0x7f260c151ac0 2023-11-15T02:57:38.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f260b7fe700 1 -- 172.21.15.121:0/2665351307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260c1511f0 msgr2=0x7f260c066570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:38.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f260b7fe700 1 --2- 172.21.15.121:0/2665351307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260c1511f0 0x7f260c066570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:38.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f260b7fe700 1 -- 172.21.15.121:0/2665351307 >> v2:172.21.15.121:3300/0 conn(0x7f260c0bcab0 msgr2=0x7f260c1515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:38.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f260b7fe700 1 --2- 172.21.15.121:0/2665351307 >> v2:172.21.15.121:3300/0 conn(0x7f260c0bcab0 0x7f260c1515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:38.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f260b7fe700 1 -- 172.21.15.121:0/2665351307 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c0657b0 con 0x7f260c151ac0 2023-11-15T02:57:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f260b7fe700 1 -- 172.21.15.121:0/2665351307 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f26040017f0 con 0x7f260c151ac0 2023-11-15T02:57:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f260b7fe700 1 -- 172.21.15.121:0/2665351307 >> v1:172.21.15.121:6789/0 conn(0x7f260c151ac0 legacy=0x7f260c0a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f260b7fe700 1 --2- 172.21.15.121:0/2665351307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25f80038e0 0x7f25f8003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f260b7fe700 1 --2- 172.21.15.121:0/2665351307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25f8004690 0x7f25f8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f260b7fe700 1 --2- 172.21.15.121:0/2665351307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260c1511f0 0x7f25f8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:38.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f260b7fe700 1 -- 172.21.15.121:0/2665351307 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c0660e0 con 0x7f25f80038e0 2023-11-15T02:57:38.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f260b7fe700 1 -- 172.21.15.121:0/2665351307 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c0bd390 con 0x7f25f8004690 2023-11-15T02:57:38.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f260b7fe700 1 -- 172.21.15.121:0/2665351307 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f800b8e0 con 0x7f260c1511f0 2023-11-15T02:57:38.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.787+0000 7f260bfff700 1 --2- 172.21.15.121:0/2665351307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25f80038e0 0x7f25f8003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:38.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f26037fe700 1 --2- 172.21.15.121:0/2665351307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25f8004690 0x7f25f8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:38.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f260bfff700 1 -- 172.21.15.121:0/2665351307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260c1511f0 msgr2=0x7f25f8005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:38.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f2610a78700 1 --2- 172.21.15.121:0/2665351307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260c1511f0 0x7f25f8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:38.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f260bfff700 1 --2- 172.21.15.121:0/2665351307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260c1511f0 0x7f25f8005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:38.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f260bfff700 1 -- 172.21.15.121:0/2665351307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25f8004690 msgr2=0x7f25f8004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:38.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f260bfff700 1 --2- 172.21.15.121:0/2665351307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25f8004690 0x7f25f8004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:38.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f260bfff700 1 -- 172.21.15.121:0/2665351307 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f80033f0 con 0x7f25f80038e0 2023-11-15T02:57:38.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f2610a78700 1 --2- 172.21.15.121:0/2665351307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260c1511f0 0x7f25f8005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:57:38.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f26037fe700 1 --2- 172.21.15.121:0/2665351307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25f8004690 0x7f25f8004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:57:38.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f260bfff700 1 --2- 172.21.15.121:0/2665351307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25f80038e0 0x7f25f8003d00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f25fc00ba00 tx=0x7f25fc008040).ready entity=mon.0 client_cookie=12a758f486e528f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:38.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f260b7fe700 1 -- 172.21.15.121:0/2665351307 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc00b340 con 0x7f25f80038e0 2023-11-15T02:57:38.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f260b7fe700 1 -- 172.21.15.121:0/2665351307 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25fc013480 con 0x7f25f80038e0 2023-11-15T02:57:38.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f260b7fe700 1 -- 172.21.15.121:0/2665351307 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc013800 con 0x7f25f80038e0 2023-11-15T02:57:38.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f2611279700 1 -- 172.21.15.121:0/2665351307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25f80038e0 msgr2=0x7f25f8003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:38.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f2611279700 1 --2- 172.21.15.121:0/2665351307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25f80038e0 0x7f25f8003d00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f25fc00ba00 tx=0x7f25fc008040).stop 2023-11-15T02:57:38.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f2610a78700 1 -- 172.21.15.121:0/2665351307 reap_dead start 2023-11-15T02:57:38.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f2611279700 1 -- 172.21.15.121:0/2665351307 shutdown_connections 2023-11-15T02:57:38.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f2611279700 1 -- 172.21.15.121:0/2665351307 >> 172.21.15.121:0/2665351307 conn(0x7f260c067a60 msgr2=0x7f260c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:38.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f2611279700 1 -- 172.21.15.121:0/2665351307 shutdown_connections 2023-11-15T02:57:38.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.791+0000 7f2611279700 1 -- 172.21.15.121:0/2665351307 wait complete. 2023-11-15T02:57:38.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f2611279700 1 Processor -- start 2023-11-15T02:57:38.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f2611279700 1 -- start start 2023-11-15T02:57:38.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f2611279700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25f80038e0 0x7f260c14e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:38.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f2611279700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260c1645f0 0x7f260c0aad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:38.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f2611279700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f260c0ab270 0x7f260c152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:38.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f2611279700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c164a70 con 0x7f260c0ab270 2023-11-15T02:57:38.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f2611279700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c0ab6f0 con 0x7f25f80038e0 2023-11-15T02:57:38.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f2611279700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260c165570 con 0x7f260c1645f0 2023-11-15T02:57:38.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f2610a78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f260c0ab270 0x7f260c152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:38.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f2610a78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f260c0ab270 0x7f260c152620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36792/0 (socket says 172.21.15.121:36792) 2023-11-15T02:57:38.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f2610a78700 1 -- 172.21.15.121:0/1764956593 learned_addr learned my addr 172.21.15.121:0/1764956593 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:38.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f260bfff700 1 --2- 172.21.15.121:0/1764956593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25f80038e0 0x7f260c14e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:38.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f26037fe700 1 --2- 172.21.15.121:0/1764956593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260c1645f0 0x7f260c0aad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:38.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f2610a78700 1 -- 172.21.15.121:0/1764956593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260c1645f0 msgr2=0x7f260c0aad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:38.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f2610a78700 1 --2- 172.21.15.121:0/1764956593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260c1645f0 0x7f260c0aad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:38.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f2610a78700 1 -- 172.21.15.121:0/1764956593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25f80038e0 msgr2=0x7f260c14e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:38.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f2610a78700 1 --2- 172.21.15.121:0/1764956593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25f80038e0 0x7f260c14e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:38.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f2610a78700 1 -- 172.21.15.121:0/1764956593 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c164cd0 con 0x7f260c0ab270 2023-11-15T02:57:38.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f260bfff700 1 --2- 172.21.15.121:0/1764956593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25f80038e0 0x7f260c14e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:38.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.795+0000 7f2610a78700 1 --2- 172.21.15.121:0/1764956593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f260c0ab270 0x7f260c152620 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f260c0688e0 tx=0x7f2604009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:38.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.799+0000 7f2608ff9700 1 -- 172.21.15.121:0/1764956593 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260400dec0 con 0x7f260c0ab270 2023-11-15T02:57:38.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.799+0000 7f2608ff9700 1 -- 172.21.15.121:0/1764956593 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2604004c40 con 0x7f260c0ab270 2023-11-15T02:57:38.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.799+0000 7f2611279700 1 -- 172.21.15.121:0/1764956593 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f260c164fb0 con 0x7f260c0ab270 2023-11-15T02:57:38.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.799+0000 7f2608ff9700 1 -- 172.21.15.121:0/1764956593 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2604007170 con 0x7f260c0ab270 2023-11-15T02:57:38.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.799+0000 7f2611279700 1 -- 172.21.15.121:0/1764956593 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f260c152100 con 0x7f260c0ab270 2023-11-15T02:57:38.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.799+0000 7f2611279700 1 -- 172.21.15.121:0/1764956593 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260c06bcf0 con 0x7f260c0ab270 2023-11-15T02:57:38.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.799+0000 7f2608ff9700 1 -- 172.21.15.121:0/1764956593 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2604007560 con 0x7f260c0ab270 2023-11-15T02:57:38.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.803+0000 7f2608ff9700 1 --2- 172.21.15.121:0/1764956593 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25ec05a540 0x7f25ec05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:38.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.803+0000 7f2608ff9700 1 -- 172.21.15.121:0/1764956593 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f2604073d30 con 0x7f260c0ab270 2023-11-15T02:57:38.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.803+0000 7f260bfff700 1 --2- 172.21.15.121:0/1764956593 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25ec05a540 0x7f25ec05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:38.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.803+0000 7f260bfff700 1 --2- 172.21.15.121:0/1764956593 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25ec05a540 0x7f25ec05ca00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f260c150610 tx=0x7f25fc008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:38.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:38.807+0000 7f2608ff9700 1 -- 172.21.15.121:0/1764956593 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2604047740 con 0x7f260c0ab270 2023-11-15T02:57:39.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.023+0000 7f2611279700 1 -- 172.21.15.121:0/1764956593 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f260c0a9190 con 0x7f260c0ab270 2023-11-15T02:57:39.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.031+0000 7f2608ff9700 1 -- 172.21.15.121:0/1764956593 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f260401c030 con 0x7f260c0ab270 2023-11-15T02:57:39.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.031+0000 7f2611279700 1 -- 172.21.15.121:0/1764956593 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25ec05a540 msgr2=0x7f25ec05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:39.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.031+0000 7f2611279700 1 --2- 172.21.15.121:0/1764956593 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25ec05a540 0x7f25ec05ca00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f260c150610 tx=0x7f25fc008040).stop 2023-11-15T02:57:39.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.031+0000 7f2611279700 1 -- 172.21.15.121:0/1764956593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f260c0ab270 msgr2=0x7f260c152620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:39.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.031+0000 7f2611279700 1 --2- 172.21.15.121:0/1764956593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f260c0ab270 0x7f260c152620 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f260c0688e0 tx=0x7f2604009040).stop 2023-11-15T02:57:39.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.035+0000 7f2611279700 1 -- 172.21.15.121:0/1764956593 shutdown_connections 2023-11-15T02:57:39.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.035+0000 7f2611279700 1 --2- 172.21.15.121:0/1764956593 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25ec05a540 0x7f25ec05ca00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.035+0000 7f2611279700 1 --2- 172.21.15.121:0/1764956593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f260c0ab270 0x7f260c152620 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.035+0000 7f2611279700 1 --2- 172.21.15.121:0/1764956593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260c1645f0 0x7f260c0aad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.035+0000 7f2611279700 1 --2- 172.21.15.121:0/1764956593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25f80038e0 0x7f260c14e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.035+0000 7f2611279700 1 -- 172.21.15.121:0/1764956593 >> 172.21.15.121:0/1764956593 conn(0x7f260c067a60 msgr2=0x7f260c0bfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:39.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.035+0000 7f2611279700 1 -- 172.21.15.121:0/1764956593 shutdown_connections 2023-11-15T02:57:39.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.035+0000 7f2611279700 1 -- 172.21.15.121:0/1764956593 wait complete. 2023-11-15T02:57:39.041 INFO:teuthology.orchestra.run.smithi121.stderr:updated 2023-11-15T02:57:39.063 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-15T02:57:39.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.215+0000 7fceff59e700 1 Processor -- start 2023-11-15T02:57:39.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.215+0000 7fceff59e700 1 -- start start 2023-11-15T02:57:39.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.219+0000 7fceff59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcf00151720 0x7fcf00152b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:39.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.219+0000 7fceff59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcf00151b00 0x7fcf001554f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:39.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.219+0000 7fceff59e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf000bbd50 con 0x7fcf00068a70 2023-11-15T02:57:39.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.219+0000 7fceff59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf000bc030 con 0x7fcf00151b00 2023-11-15T02:57:39.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.219+0000 7fceff59e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf00155d20 con 0x7fcf00151720 2023-11-15T02:57:39.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.219+0000 7fcefdd9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcf00151b00 0x7fcf001554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:39.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.219+0000 7fcefed9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fcf00068a70 0x7fcf00062b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56234/0 (socket says 172.21.15.121:56234) 2023-11-15T02:57:39.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.219+0000 7fcefed9d700 1 -- 172.21.15.121:0/1867547832 learned_addr learned my addr 172.21.15.121:0/1867547832 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:39.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.219+0000 7fcefe59c700 1 --2- 172.21.15.121:0/1867547832 >> v2:172.21.15.190:3300/0 conn(0x7fcf00151720 0x7fcf00152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:39.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.219+0000 7fcefd59a700 1 -- 172.21.15.121:0/1867547832 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4028409573 0 0) 0x7fcf000bbd50 con 0x7fcf00068a70 2023-11-15T02:57:39.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.219+0000 7fcefd59a700 1 -- 172.21.15.121:0/1867547832 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcee80033f0 con 0x7fcf00068a70 2023-11-15T02:57:39.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.219+0000 7fcefdd9b700 1 -- 172.21.15.121:0/1867547832 >> v2:172.21.15.190:3300/0 conn(0x7fcf00151720 msgr2=0x7fcf00152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:39.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.219+0000 7fcefdd9b700 1 --2- 172.21.15.121:0/1867547832 >> v2:172.21.15.190:3300/0 conn(0x7fcf00151720 0x7fcf00152b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.219+0000 7fcefdd9b700 1 -- 172.21.15.121:0/1867547832 >> v1:172.21.15.121:6789/0 conn(0x7fcf00068a70 legacy=0x7fcf00062b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:39.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.219+0000 7fcefd59a700 1 -- 172.21.15.121:0/1867547832 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 405544783 0 0) 0x7fcee80033f0 con 0x7fcf00068a70 2023-11-15T02:57:39.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.219+0000 7fcefdd9b700 1 -- 172.21.15.121:0/1867547832 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf00156000 con 0x7fcf00151b00 2023-11-15T02:57:39.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.219+0000 7fcefdd9b700 1 --2- 172.21.15.121:0/1867547832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf00151b00 0x7fcf001554f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fcf00065690 tx=0x7fcef4009060).ready entity=mon.0 client_cookie=e19c76ca48076df2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:39.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.223+0000 7fcefd59a700 1 -- 172.21.15.121:0/1867547832 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef400d4a0 con 0x7fcf00151b00 2023-11-15T02:57:39.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.223+0000 7fcefd59a700 1 -- 172.21.15.121:0/1867547832 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef400d600 con 0x7fcf00151b00 2023-11-15T02:57:39.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.223+0000 7fcefd59a700 1 -- 172.21.15.121:0/1867547832 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef400d980 con 0x7fcf00151b00 2023-11-15T02:57:39.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.223+0000 7fceff59e700 1 -- 172.21.15.121:0/1867547832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf00151b00 msgr2=0x7fcf001554f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:39.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.223+0000 7fceff59e700 1 --2- 172.21.15.121:0/1867547832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf00151b00 0x7fcf001554f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fcf00065690 tx=0x7fcef4009060).stop 2023-11-15T02:57:39.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.223+0000 7fceff59e700 1 -- 172.21.15.121:0/1867547832 shutdown_connections 2023-11-15T02:57:39.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.223+0000 7fceff59e700 1 --2- 172.21.15.121:0/1867547832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf00151b00 0x7fcf001554f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.223+0000 7fceff59e700 1 --2- 172.21.15.121:0/1867547832 >> v2:172.21.15.190:3300/0 conn(0x7fcf00151720 0x7fcf00152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.223+0000 7fceff59e700 1 -- 172.21.15.121:0/1867547832 >> 172.21.15.121:0/1867547832 conn(0x7fcf00065fa0 msgr2=0x7fcf000663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:39.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.223+0000 7fceff59e700 1 -- 172.21.15.121:0/1867547832 shutdown_connections 2023-11-15T02:57:39.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.223+0000 7fceff59e700 1 -- 172.21.15.121:0/1867547832 wait complete. 2023-11-15T02:57:39.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.223+0000 7fceff59e700 1 Processor -- start 2023-11-15T02:57:39.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.223+0000 7fceff59e700 1 -- start start 2023-11-15T02:57:39.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.223+0000 7fceff59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf00068a70 0x7fcf00164b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:39.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fceff59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf00151720 0x7fcf000a8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:39.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fceff59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf00151b00 0x7fcf000c0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:39.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fceff59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000a7cb0 con 0x7fcf00151b00 2023-11-15T02:57:39.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fceff59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf00165030 con 0x7fcf00068a70 2023-11-15T02:57:39.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fceff59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000a6dc0 con 0x7fcf00151720 2023-11-15T02:57:39.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fcefed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf00151b00 0x7fcf000c0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:39.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fcefed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf00151b00 0x7fcf000c0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36812/0 (socket says 172.21.15.121:36812) 2023-11-15T02:57:39.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fcefed9d700 1 -- 172.21.15.121:0/3821154788 learned_addr learned my addr 172.21.15.121:0/3821154788 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:39.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fcefe59c700 1 --2- 172.21.15.121:0/3821154788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf00068a70 0x7fcf00164b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:39.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fcefdd9b700 1 --2- 172.21.15.121:0/3821154788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf00151720 0x7fcf000a8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:39.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fcefed9d700 1 -- 172.21.15.121:0/3821154788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf00151720 msgr2=0x7fcf000a8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:39.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fcefed9d700 1 --2- 172.21.15.121:0/3821154788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf00151720 0x7fcf000a8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fcefed9d700 1 -- 172.21.15.121:0/3821154788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf00068a70 msgr2=0x7fcf00164b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:39.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fcefed9d700 1 --2- 172.21.15.121:0/3821154788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf00068a70 0x7fcf00164b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fcefed9d700 1 -- 172.21.15.121:0/3821154788 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef40092a0 con 0x7fcf00151b00 2023-11-15T02:57:39.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fcefed9d700 1 --2- 172.21.15.121:0/3821154788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf00151b00 0x7fcf000c0170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7fcef8007fd0 tx=0x7fcef8007cf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:39.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fceee7fc700 1 -- 172.21.15.121:0/3821154788 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef8004c40 con 0x7fcf00151b00 2023-11-15T02:57:39.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fceee7fc700 1 -- 172.21.15.121:0/3821154788 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef8004da0 con 0x7fcf00151b00 2023-11-15T02:57:39.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fceee7fc700 1 -- 172.21.15.121:0/3821154788 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef80073a0 con 0x7fcf00151b00 2023-11-15T02:57:39.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fceff59e700 1 -- 172.21.15.121:0/3821154788 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf000a70a0 con 0x7fcf00151b00 2023-11-15T02:57:39.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fceff59e700 1 -- 172.21.15.121:0/3821154788 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf000a7660 con 0x7fcf00151b00 2023-11-15T02:57:39.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.227+0000 7fceff59e700 1 -- 172.21.15.121:0/3821154788 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf000a7e20 con 0x7fcf00151b00 2023-11-15T02:57:39.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.231+0000 7fceee7fc700 1 -- 172.21.15.121:0/3821154788 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fcef8007750 con 0x7fcf00151b00 2023-11-15T02:57:39.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.231+0000 7fceee7fc700 1 --2- 172.21.15.121:0/3821154788 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcee005a510 0x7fcee005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:39.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.231+0000 7fceee7fc700 1 -- 172.21.15.121:0/3821154788 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fcef8073f20 con 0x7fcf00151b00 2023-11-15T02:57:39.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.231+0000 7fcefe59c700 1 --2- 172.21.15.121:0/3821154788 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcee005a510 0x7fcee005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:39.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.231+0000 7fcefe59c700 1 --2- 172.21.15.121:0/3821154788 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcee005a510 0x7fcee005c9d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fcef0009c30 tx=0x7fcef0009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:39.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.239+0000 7fceee7fc700 1 -- 172.21.15.121:0/3821154788 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcef8047990 con 0x7fcf00151b00 2023-11-15T02:57:39.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.459+0000 7fceff59e700 1 -- 172.21.15.121:0/3821154788 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fcf0006bcf0 con 0x7fcf00151b00 2023-11-15T02:57:39.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.467+0000 7fceee7fc700 1 -- 172.21.15.121:0/3821154788 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcef804afb0 con 0x7fcf00151b00 2023-11-15T02:57:39.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.471+0000 7fceff59e700 1 -- 172.21.15.121:0/3821154788 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcee005a510 msgr2=0x7fcee005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:39.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.471+0000 7fceff59e700 1 --2- 172.21.15.121:0/3821154788 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcee005a510 0x7fcee005c9d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fcef0009c30 tx=0x7fcef0009060).stop 2023-11-15T02:57:39.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.471+0000 7fceff59e700 1 -- 172.21.15.121:0/3821154788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf00151b00 msgr2=0x7fcf000c0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:39.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.471+0000 7fceff59e700 1 --2- 172.21.15.121:0/3821154788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf00151b00 0x7fcf000c0170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7fcef8007fd0 tx=0x7fcef8007cf0).stop 2023-11-15T02:57:39.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.471+0000 7fceff59e700 1 -- 172.21.15.121:0/3821154788 shutdown_connections 2023-11-15T02:57:39.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.471+0000 7fceff59e700 1 --2- 172.21.15.121:0/3821154788 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcee005a510 0x7fcee005c9d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.471+0000 7fceff59e700 1 --2- 172.21.15.121:0/3821154788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf00151b00 0x7fcf000c0170 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.471+0000 7fceff59e700 1 --2- 172.21.15.121:0/3821154788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf00151720 0x7fcf000a8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.471+0000 7fceff59e700 1 --2- 172.21.15.121:0/3821154788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf00068a70 0x7fcf00164b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.471+0000 7fceff59e700 1 -- 172.21.15.121:0/3821154788 >> 172.21.15.121:0/3821154788 conn(0x7fcf00065fa0 msgr2=0x7fcf00151f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:39.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.471+0000 7fceff59e700 1 -- 172.21.15.121:0/3821154788 shutdown_connections 2023-11-15T02:57:39.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.471+0000 7fceff59e700 1 -- 172.21.15.121:0/3821154788 wait complete. 2023-11-15T02:57:39.480 INFO:teuthology.orchestra.run.smithi121.stderr:updated 2023-11-15T02:57:39.497 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-15T02:57:39.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.651+0000 7f0dcfa9b700 1 Processor -- start 2023-11-15T02:57:39.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.651+0000 7f0dcfa9b700 1 -- start start 2023-11-15T02:57:39.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dcfa9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0dc81516a0 0x7f0dc8066420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:39.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dcfa9b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc8065f90 con 0x7f0dc8062190 2023-11-15T02:57:39.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dcfa9b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc8147ea0 con 0x7f0dc80a8de0 2023-11-15T02:57:39.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dcfa9b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc8151e30 con 0x7f0dc81516a0 2023-11-15T02:57:39.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dcd036700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0dc81516a0 0x7f0dc8066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:39.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dce038700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f0dc8062190 0x7f0dc8152080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56242/0 (socket says 172.21.15.121:56242) 2023-11-15T02:57:39.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dcd837700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f0dc80a8de0 0x7f0dc8150f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:52200/0 (socket says 172.21.15.121:52200) 2023-11-15T02:57:39.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dce038700 1 -- 172.21.15.121:0/1735667412 learned_addr learned my addr 172.21.15.121:0/1735667412 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:39.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dcd036700 1 -- 172.21.15.121:0/1735667412 >> v1:172.21.15.190:6789/0 conn(0x7f0dc80a8de0 legacy=0x7f0dc8150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:39.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dcd036700 1 -- 172.21.15.121:0/1735667412 >> v1:172.21.15.121:6789/0 conn(0x7f0dc8062190 legacy=0x7f0dc8152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:39.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dcc835700 1 -- 172.21.15.121:0/1735667412 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280409517 0 0) 0x7f0dc8147ea0 con 0x7f0dc80a8de0 2023-11-15T02:57:39.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dcd036700 1 -- 172.21.15.121:0/1735667412 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc80671f0 con 0x7f0dc81516a0 2023-11-15T02:57:39.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dcd036700 1 --2- 172.21.15.121:0/1735667412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc81516a0 0x7f0dc8066420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f0db8009760 tx=0x7f0db8009060).ready entity=mon.0 client_cookie=965dd26279c99dc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:39.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dcc835700 1 -- 172.21.15.121:0/1735667412 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db80090d0 con 0x7f0dc81516a0 2023-11-15T02:57:39.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dcc835700 1 -- 172.21.15.121:0/1735667412 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db800f510 con 0x7f0dc81516a0 2023-11-15T02:57:39.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dcc835700 1 -- 172.21.15.121:0/1735667412 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db800f890 con 0x7f0dc81516a0 2023-11-15T02:57:39.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dcfa9b700 1 -- 172.21.15.121:0/1735667412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc81516a0 msgr2=0x7f0dc8066420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:39.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dcfa9b700 1 --2- 172.21.15.121:0/1735667412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc81516a0 0x7f0dc8066420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f0db8009760 tx=0x7f0db8009060).stop 2023-11-15T02:57:39.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dcfa9b700 1 -- 172.21.15.121:0/1735667412 shutdown_connections 2023-11-15T02:57:39.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dcfa9b700 1 --2- 172.21.15.121:0/1735667412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc81516a0 0x7f0dc8066420 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.655+0000 7f0dcfa9b700 1 -- 172.21.15.121:0/1735667412 >> 172.21.15.121:0/1735667412 conn(0x7f0dc8067a60 msgr2=0x7f0dc8067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:39.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.659+0000 7f0dcfa9b700 1 -- 172.21.15.121:0/1735667412 shutdown_connections 2023-11-15T02:57:39.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.659+0000 7f0dcfa9b700 1 -- 172.21.15.121:0/1735667412 wait complete. 2023-11-15T02:57:39.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.659+0000 7f0dcfa9b700 1 Processor -- start 2023-11-15T02:57:39.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.659+0000 7f0dcfa9b700 1 -- start start 2023-11-15T02:57:39.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dcfa9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dc8062190 0x7f0dc80ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:39.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dcfa9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc80a8de0 0x7f0dc8138040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:39.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dcfa9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dc81516a0 0x7f0dc814faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:39.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dcfa9b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc814d250 con 0x7f0dc80a8de0 2023-11-15T02:57:39.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dcfa9b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc8148ac0 con 0x7f0dc81516a0 2023-11-15T02:57:39.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dcfa9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc80a9cd0 con 0x7f0dc8062190 2023-11-15T02:57:39.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dcd036700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc80a8de0 0x7f0dc8138040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:39.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dcd036700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc80a8de0 0x7f0dc8138040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36834/0 (socket says 172.21.15.121:36834) 2023-11-15T02:57:39.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dcd036700 1 -- 172.21.15.121:0/3722876897 learned_addr learned my addr 172.21.15.121:0/3722876897 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:39.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dcd837700 1 --2- 172.21.15.121:0/3722876897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dc8062190 0x7f0dc80ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:39.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dce038700 1 --2- 172.21.15.121:0/3722876897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dc81516a0 0x7f0dc814faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:39.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dcd036700 1 -- 172.21.15.121:0/3722876897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dc8062190 msgr2=0x7f0dc80ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:39.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dcd036700 1 --2- 172.21.15.121:0/3722876897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dc8062190 0x7f0dc80ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dcd036700 1 -- 172.21.15.121:0/3722876897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dc81516a0 msgr2=0x7f0dc814faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:39.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dcd036700 1 --2- 172.21.15.121:0/3722876897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dc81516a0 0x7f0dc814faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dcd036700 1 -- 172.21.15.121:0/3722876897 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db80092a0 con 0x7f0dc80a8de0 2023-11-15T02:57:39.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dcd036700 1 --2- 172.21.15.121:0/3722876897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc80a8de0 0x7f0dc8138040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f0dc8063db0 tx=0x7f0db8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:39.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dbdffb700 1 -- 172.21.15.121:0/3722876897 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db800fea0 con 0x7f0dc80a8de0 2023-11-15T02:57:39.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dbdffb700 1 -- 172.21.15.121:0/3722876897 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db8006990 con 0x7f0dc80a8de0 2023-11-15T02:57:39.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dcfa9b700 1 -- 172.21.15.121:0/3722876897 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc80a9fb0 con 0x7f0dc80a8de0 2023-11-15T02:57:39.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dbdffb700 1 -- 172.21.15.121:0/3722876897 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db800f890 con 0x7f0dc80a8de0 2023-11-15T02:57:39.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.663+0000 7f0dcfa9b700 1 -- 172.21.15.121:0/3722876897 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc80aa570 con 0x7f0dc80a8de0 2023-11-15T02:57:39.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.667+0000 7f0dcfa9b700 1 -- 172.21.15.121:0/3722876897 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc8148890 con 0x7f0dc80a8de0 2023-11-15T02:57:39.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.667+0000 7f0dbdffb700 1 -- 172.21.15.121:0/3722876897 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0db801b620 con 0x7f0dc80a8de0 2023-11-15T02:57:39.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.667+0000 7f0dbdffb700 1 --2- 172.21.15.121:0/3722876897 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0dac05a510 0x7f0dac05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:39.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.667+0000 7f0dbdffb700 1 -- 172.21.15.121:0/3722876897 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f0db8075c40 con 0x7f0dc80a8de0 2023-11-15T02:57:39.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.671+0000 7f0dcd837700 1 --2- 172.21.15.121:0/3722876897 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0dac05a510 0x7f0dac05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:39.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.671+0000 7f0dcd837700 1 --2- 172.21.15.121:0/3722876897 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0dac05a510 0x7f0dac05c9d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f0dc40011e0 tx=0x7f0dc4007180).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:39.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.675+0000 7f0dbdffb700 1 -- 172.21.15.121:0/3722876897 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0dc8148890 con 0x7f0dc80a8de0 2023-11-15T02:57:39.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.887+0000 7f0dcfa9b700 1 -- 172.21.15.121:0/3722876897 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f0dc8148890 con 0x7f0dc80a8de0 2023-11-15T02:57:39.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.895+0000 7f0dbdffb700 1 -- 172.21.15.121:0/3722876897 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0dc8148890 con 0x7f0dc80a8de0 2023-11-15T02:57:39.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.895+0000 7f0dcfa9b700 1 -- 172.21.15.121:0/3722876897 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0dac05a510 msgr2=0x7f0dac05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:39.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.895+0000 7f0dcfa9b700 1 --2- 172.21.15.121:0/3722876897 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0dac05a510 0x7f0dac05c9d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f0dc40011e0 tx=0x7f0dc4007180).stop 2023-11-15T02:57:39.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.895+0000 7f0dcfa9b700 1 -- 172.21.15.121:0/3722876897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc80a8de0 msgr2=0x7f0dc8138040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:39.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.895+0000 7f0dcfa9b700 1 --2- 172.21.15.121:0/3722876897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc80a8de0 0x7f0dc8138040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f0dc8063db0 tx=0x7f0db8017000).stop 2023-11-15T02:57:39.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.895+0000 7f0dcfa9b700 1 -- 172.21.15.121:0/3722876897 shutdown_connections 2023-11-15T02:57:39.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.895+0000 7f0dcfa9b700 1 --2- 172.21.15.121:0/3722876897 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0dac05a510 0x7f0dac05c9d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.895+0000 7f0dcfa9b700 1 --2- 172.21.15.121:0/3722876897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dc81516a0 0x7f0dc814faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.895+0000 7f0dcfa9b700 1 --2- 172.21.15.121:0/3722876897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc80a8de0 0x7f0dc8138040 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.895+0000 7f0dcfa9b700 1 --2- 172.21.15.121:0/3722876897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dc8062190 0x7f0dc80ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:39.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.895+0000 7f0dcfa9b700 1 -- 172.21.15.121:0/3722876897 >> 172.21.15.121:0/3722876897 conn(0x7f0dc8067a60 msgr2=0x7f0dc8152d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:39.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.899+0000 7f0dcfa9b700 1 -- 172.21.15.121:0/3722876897 shutdown_connections 2023-11-15T02:57:39.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:39.899+0000 7f0dcfa9b700 1 -- 172.21.15.121:0/3722876897 wait complete. 2023-11-15T02:57:39.905 INFO:teuthology.orchestra.run.smithi121.stderr:updated 2023-11-15T02:57:39.921 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-15T02:57:40.069 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T02:57:40.076 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T02:57:40.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.075+0000 7fb57965b700 1 Processor -- start 2023-11-15T02:57:40.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.075+0000 7fb57965b700 1 -- start start 2023-11-15T02:57:40.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.079+0000 7fb57965b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb5740c04d0 0x7fb5740c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:40.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.079+0000 7fb57965b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb5740c8fb0 0x7fb5740c9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:40.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.079+0000 7fb57965b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5740c5720 con 0x7fb574153670 2023-11-15T02:57:40.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.079+0000 7fb57965b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5740c0da0 con 0x7fb5740c04d0 2023-11-15T02:57:40.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.079+0000 7fb57965b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5740ca8a0 con 0x7fb5740c8fb0 2023-11-15T02:57:40.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.079+0000 7fb572ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb5740c04d0 0x7fb5740c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:40.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.079+0000 7fb572ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb5740c04d0 0x7fb5740c08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36850/0 (socket says 172.21.15.121:36850) 2023-11-15T02:57:40.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.079+0000 7fb572ffd700 1 -- 172.21.15.121:0/2469423237 learned_addr learned my addr 172.21.15.121:0/2469423237 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:40.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.079+0000 7fb5727fc700 1 --1- 172.21.15.121:0/2469423237 >> v1:172.21.15.121:6789/0 conn(0x7fb574153670 0x7fb574153b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56256/0 (socket says 172.21.15.121:56256) 2023-11-15T02:57:40.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.079+0000 7fb5737fe700 1 --2- 172.21.15.121:0/2469423237 >> v2:172.21.15.190:3300/0 conn(0x7fb5740c8fb0 0x7fb5740c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:40.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.079+0000 7fb572ffd700 1 -- 172.21.15.121:0/2469423237 >> v2:172.21.15.190:3300/0 conn(0x7fb5740c8fb0 msgr2=0x7fb5740c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:40.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.079+0000 7fb572ffd700 1 --2- 172.21.15.121:0/2469423237 >> v2:172.21.15.190:3300/0 conn(0x7fb5740c8fb0 0x7fb5740c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.079+0000 7fb572ffd700 1 -- 172.21.15.121:0/2469423237 >> v1:172.21.15.121:6789/0 conn(0x7fb574153670 legacy=0x7fb574153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:40.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.079+0000 7fb572ffd700 1 -- 172.21.15.121:0/2469423237 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5740c9d20 con 0x7fb5740c04d0 2023-11-15T02:57:40.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.079+0000 7fb5737fe700 1 --2- 172.21.15.121:0/2469423237 >> v2:172.21.15.190:3300/0 conn(0x7fb5740c8fb0 0x7fb5740c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:57:40.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.079+0000 7fb572ffd700 1 --2- 172.21.15.121:0/2469423237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5740c04d0 0x7fb5740c08b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7fb5740c1a20 tx=0x7fb5640090f0).ready entity=mon.0 client_cookie=19702d67c1a9b3cf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:40.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.083+0000 7fb571ffb700 1 -- 172.21.15.121:0/2469423237 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56400b600 con 0x7fb5740c04d0 2023-11-15T02:57:40.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.083+0000 7fb571ffb700 1 -- 172.21.15.121:0/2469423237 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56400b760 con 0x7fb5740c04d0 2023-11-15T02:57:40.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.083+0000 7fb571ffb700 1 -- 172.21.15.121:0/2469423237 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56400bb00 con 0x7fb5740c04d0 2023-11-15T02:57:40.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.083+0000 7fb57965b700 1 -- 172.21.15.121:0/2469423237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5740c04d0 msgr2=0x7fb5740c08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:40.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.083+0000 7fb57965b700 1 --2- 172.21.15.121:0/2469423237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5740c04d0 0x7fb5740c08b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7fb5740c1a20 tx=0x7fb5640090f0).stop 2023-11-15T02:57:40.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.083+0000 7fb57965b700 1 -- 172.21.15.121:0/2469423237 shutdown_connections 2023-11-15T02:57:40.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.083+0000 7fb57965b700 1 --2- 172.21.15.121:0/2469423237 >> v2:172.21.15.190:3300/0 conn(0x7fb5740c8fb0 0x7fb5740c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.083+0000 7fb57965b700 1 --2- 172.21.15.121:0/2469423237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5740c04d0 0x7fb5740c08b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.083+0000 7fb57965b700 1 -- 172.21.15.121:0/2469423237 >> 172.21.15.121:0/2469423237 conn(0x7fb574067a60 msgr2=0x7fb574067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:40.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.083+0000 7fb57965b700 1 -- 172.21.15.121:0/2469423237 shutdown_connections 2023-11-15T02:57:40.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.083+0000 7fb57965b700 1 -- 172.21.15.121:0/2469423237 wait complete. 2023-11-15T02:57:40.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.087+0000 7fb57965b700 1 Processor -- start 2023-11-15T02:57:40.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.087+0000 7fb57965b700 1 -- start start 2023-11-15T02:57:40.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.087+0000 7fb57965b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5740c04d0 0x7fb574165010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:40.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.087+0000 7fb57965b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5740c8fb0 0x7fb57414b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:40.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.087+0000 7fb57965b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb574153670 0x7fb5740c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:40.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.087+0000 7fb57965b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb574168900 con 0x7fb574153670 2023-11-15T02:57:40.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.087+0000 7fb57965b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb574157640 con 0x7fb5740c8fb0 2023-11-15T02:57:40.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.087+0000 7fb57965b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb574168620 con 0x7fb5740c04d0 2023-11-15T02:57:40.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.087+0000 7fb5737fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb574153670 0x7fb5740c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:40.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.087+0000 7fb5737fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb574153670 0x7fb5740c3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36866/0 (socket says 172.21.15.121:36866) 2023-11-15T02:57:40.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.087+0000 7fb5737fe700 1 -- 172.21.15.121:0/2304751782 learned_addr learned my addr 172.21.15.121:0/2304751782 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:40.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.087+0000 7fb572ffd700 1 --2- 172.21.15.121:0/2304751782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5740c04d0 0x7fb574165010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:40.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.087+0000 7fb5727fc700 1 --2- 172.21.15.121:0/2304751782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5740c8fb0 0x7fb57414b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:40.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.091+0000 7fb5737fe700 1 -- 172.21.15.121:0/2304751782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5740c04d0 msgr2=0x7fb574165010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:40.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.091+0000 7fb5737fe700 1 --2- 172.21.15.121:0/2304751782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5740c04d0 0x7fb574165010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.091+0000 7fb5737fe700 1 -- 172.21.15.121:0/2304751782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5740c8fb0 msgr2=0x7fb57414b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:40.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.091+0000 7fb5737fe700 1 --2- 172.21.15.121:0/2304751782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5740c8fb0 0x7fb57414b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.091+0000 7fb5737fe700 1 -- 172.21.15.121:0/2304751782 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb564009370 con 0x7fb574153670 2023-11-15T02:57:40.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.091+0000 7fb5737fe700 1 --2- 172.21.15.121:0/2304751782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb574153670 0x7fb5740c3ca0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fb56c00a380 tx=0x7fb56c007900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:40.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.091+0000 7fb55affd700 1 -- 172.21.15.121:0/2304751782 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c016470 con 0x7fb574153670 2023-11-15T02:57:40.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.091+0000 7fb55affd700 1 -- 172.21.15.121:0/2304751782 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56c0165d0 con 0x7fb574153670 2023-11-15T02:57:40.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.091+0000 7fb57965b700 1 -- 172.21.15.121:0/2304751782 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb574164550 con 0x7fb574153670 2023-11-15T02:57:40.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.091+0000 7fb55affd700 1 -- 172.21.15.121:0/2304751782 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c016950 con 0x7fb574153670 2023-11-15T02:57:40.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.091+0000 7fb57965b700 1 -- 172.21.15.121:0/2304751782 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5741648e0 con 0x7fb574153670 2023-11-15T02:57:40.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.091+0000 7fb55affd700 1 -- 172.21.15.121:0/2304751782 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb56c013070 con 0x7fb574153670 2023-11-15T02:57:40.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.091+0000 7fb57965b700 1 -- 172.21.15.121:0/2304751782 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb574047060 con 0x7fb574153670 2023-11-15T02:57:40.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.095+0000 7fb55affd700 1 --2- 172.21.15.121:0/2304751782 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb56005a510 0x7fb56005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:40.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.095+0000 7fb55affd700 1 -- 172.21.15.121:0/2304751782 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fb56c074e00 con 0x7fb574153670 2023-11-15T02:57:40.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.095+0000 7fb572ffd700 1 --2- 172.21.15.121:0/2304751782 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb56005a510 0x7fb56005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:40.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.095+0000 7fb572ffd700 1 --2- 172.21.15.121:0/2304751782 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb56005a510 0x7fb56005c9d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fb564009fd0 tx=0x7fb564017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:40.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.095+0000 7fb55affd700 1 -- 172.21.15.121:0/2304751782 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb56c048870 con 0x7fb574153670 2023-11-15T02:57:40.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.311+0000 7fb57965b700 1 -- 172.21.15.121:0/2304751782 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fb5740ca000 con 0x7fb574153670 2023-11-15T02:57:40.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.319+0000 7fb55affd700 1 -- 172.21.15.121:0/2304751782 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v12) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb56c01b030 con 0x7fb574153670 2023-11-15T02:57:40.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.323+0000 7fb57965b700 1 -- 172.21.15.121:0/2304751782 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb56005a510 msgr2=0x7fb56005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:40.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.323+0000 7fb57965b700 1 --2- 172.21.15.121:0/2304751782 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb56005a510 0x7fb56005c9d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fb564009fd0 tx=0x7fb564017000).stop 2023-11-15T02:57:40.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.323+0000 7fb57965b700 1 -- 172.21.15.121:0/2304751782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb574153670 msgr2=0x7fb5740c3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:40.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.323+0000 7fb57965b700 1 --2- 172.21.15.121:0/2304751782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb574153670 0x7fb5740c3ca0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fb56c00a380 tx=0x7fb56c007900).stop 2023-11-15T02:57:40.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.323+0000 7fb57965b700 1 -- 172.21.15.121:0/2304751782 shutdown_connections 2023-11-15T02:57:40.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.323+0000 7fb57965b700 1 --2- 172.21.15.121:0/2304751782 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb56005a510 0x7fb56005c9d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.323+0000 7fb57965b700 1 --2- 172.21.15.121:0/2304751782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb574153670 0x7fb5740c3ca0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.323+0000 7fb57965b700 1 --2- 172.21.15.121:0/2304751782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5740c8fb0 0x7fb57414b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.323+0000 7fb57965b700 1 --2- 172.21.15.121:0/2304751782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5740c04d0 0x7fb574165010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.323+0000 7fb57965b700 1 -- 172.21.15.121:0/2304751782 >> 172.21.15.121:0/2304751782 conn(0x7fb574067a60 msgr2=0x7fb574062190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:40.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.327+0000 7fb57965b700 1 -- 172.21.15.121:0/2304751782 shutdown_connections 2023-11-15T02:57:40.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.327+0000 7fb57965b700 1 -- 172.21.15.121:0/2304751782 wait complete. 2023-11-15T02:57:40.333 INFO:teuthology.orchestra.run.smithi121.stderr:updated 2023-11-15T02:57:40.352 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:57:40.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.503+0000 7f510759e700 1 Processor -- start 2023-11-15T02:57:40.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.503+0000 7f510759e700 1 -- start start 2023-11-15T02:57:40.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.507+0000 7f510759e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5108152850 0x7f51080c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:40.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.507+0000 7f510759e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f510815be30 0x7f510815c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:40.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.507+0000 7f510759e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51080c1d50 con 0x7f51080c3280 2023-11-15T02:57:40.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.507+0000 7f510759e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51080c1860 con 0x7f5108152850 2023-11-15T02:57:40.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.507+0000 7f510759e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510815cb80 con 0x7f510815be30 2023-11-15T02:57:40.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.507+0000 7f50fdd9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5108152850 0x7f51080c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:40.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.507+0000 7f50fdd9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5108152850 0x7f51080c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36882/0 (socket says 172.21.15.121:36882) 2023-11-15T02:57:40.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.507+0000 7f50fdd9b700 1 -- 172.21.15.121:0/3187439849 learned_addr learned my addr 172.21.15.121:0/3187439849 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:40.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.507+0000 7f5106d9d700 1 --2- 172.21.15.121:0/3187439849 >> v2:172.21.15.190:3300/0 conn(0x7f510815be30 0x7f510815c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:40.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.507+0000 7f50fdd9b700 1 -- 172.21.15.121:0/3187439849 >> v2:172.21.15.190:3300/0 conn(0x7f510815be30 msgr2=0x7f510815c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:40.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.507+0000 7f50fdd9b700 1 --2- 172.21.15.121:0/3187439849 >> v2:172.21.15.190:3300/0 conn(0x7f510815be30 0x7f510815c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.507+0000 7f50fdd9b700 1 -- 172.21.15.121:0/3187439849 >> v1:172.21.15.190:6789/0 conn(0x7f51080c3280 legacy=0x7f5108152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:40.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.507+0000 7f50fdd9b700 1 -- 172.21.15.121:0/3187439849 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51080a9240 con 0x7f5108152850 2023-11-15T02:57:40.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.507+0000 7f50fdd9b700 1 --2- 172.21.15.121:0/3187439849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5108152850 0x7f51080c3660 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f51080c0d80 tx=0x7f50f00090f0).ready entity=mon.0 client_cookie=55063a0d43830e37 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:40.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.507+0000 7f5105d9b700 1 -- 172.21.15.121:0/3187439849 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f000b520 con 0x7f5108152850 2023-11-15T02:57:40.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.507+0000 7f5105d9b700 1 -- 172.21.15.121:0/3187439849 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f000b680 con 0x7f5108152850 2023-11-15T02:57:40.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.507+0000 7f5105d9b700 1 -- 172.21.15.121:0/3187439849 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f000ba00 con 0x7f5108152850 2023-11-15T02:57:40.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.507+0000 7f510759e700 1 -- 172.21.15.121:0/3187439849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5108152850 msgr2=0x7f51080c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:40.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.507+0000 7f510759e700 1 --2- 172.21.15.121:0/3187439849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5108152850 0x7f51080c3660 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f51080c0d80 tx=0x7f50f00090f0).stop 2023-11-15T02:57:40.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.511+0000 7f510759e700 1 -- 172.21.15.121:0/3187439849 shutdown_connections 2023-11-15T02:57:40.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.511+0000 7f510759e700 1 --2- 172.21.15.121:0/3187439849 >> v2:172.21.15.190:3300/0 conn(0x7f510815be30 0x7f510815c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.511+0000 7f510759e700 1 --2- 172.21.15.121:0/3187439849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5108152850 0x7f51080c3660 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.511+0000 7f510759e700 1 -- 172.21.15.121:0/3187439849 >> 172.21.15.121:0/3187439849 conn(0x7f5108062190 msgr2=0x7f5108062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:40.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.511+0000 7f510759e700 1 -- 172.21.15.121:0/3187439849 shutdown_connections 2023-11-15T02:57:40.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.511+0000 7f510759e700 1 -- 172.21.15.121:0/3187439849 wait complete. 2023-11-15T02:57:40.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.511+0000 7f510759e700 1 Processor -- start 2023-11-15T02:57:40.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.511+0000 7f510759e700 1 -- start start 2023-11-15T02:57:40.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f510759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51080c3280 0x7f510814d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:40.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f510759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5108152850 0x7f5108159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:40.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f510759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f510815be30 0x7f510815a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:40.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f510759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510815a610 con 0x7f510815be30 2023-11-15T02:57:40.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f510759e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51080bc3b0 con 0x7f51080c3280 2023-11-15T02:57:40.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f510759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51080bb7e0 con 0x7f5108152850 2023-11-15T02:57:40.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f5106d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f510815be30 0x7f510815a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:40.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f5106d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f510815be30 0x7f510815a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36890/0 (socket says 172.21.15.121:36890) 2023-11-15T02:57:40.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f5106d9d700 1 -- 172.21.15.121:0/29354380 learned_addr learned my addr 172.21.15.121:0/29354380 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:40.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f50fdd9b700 1 --2- 172.21.15.121:0/29354380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5108152850 0x7f5108159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:40.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f510659c700 1 --2- 172.21.15.121:0/29354380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51080c3280 0x7f510814d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:40.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f5106d9d700 1 -- 172.21.15.121:0/29354380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5108152850 msgr2=0x7f5108159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:40.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f5106d9d700 1 --2- 172.21.15.121:0/29354380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5108152850 0x7f5108159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f5106d9d700 1 -- 172.21.15.121:0/29354380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51080c3280 msgr2=0x7f510814d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:40.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f5106d9d700 1 --2- 172.21.15.121:0/29354380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51080c3280 0x7f510814d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f5106d9d700 1 -- 172.21.15.121:0/29354380 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f0009370 con 0x7f510815be30 2023-11-15T02:57:40.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f510659c700 1 --2- 172.21.15.121:0/29354380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51080c3280 0x7f510814d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:40.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f5106d9d700 1 --2- 172.21.15.121:0/29354380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f510815be30 0x7f510815a120 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f5100009e90 tx=0x7f510000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:40.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f50feffd700 1 -- 172.21.15.121:0/29354380 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510000cb50 con 0x7f510815be30 2023-11-15T02:57:40.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f50feffd700 1 -- 172.21.15.121:0/29354380 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510000f040 con 0x7f510815be30 2023-11-15T02:57:40.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f50feffd700 1 -- 172.21.15.121:0/29354380 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510000cec0 con 0x7f510815be30 2023-11-15T02:57:40.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f510759e700 1 -- 172.21.15.121:0/29354380 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51080bba80 con 0x7f510815be30 2023-11-15T02:57:40.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.515+0000 7f510759e700 1 -- 172.21.15.121:0/29354380 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51080bc040 con 0x7f510815be30 2023-11-15T02:57:40.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.519+0000 7f510759e700 1 -- 172.21.15.121:0/29354380 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f510806bcf0 con 0x7f510815be30 2023-11-15T02:57:40.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.519+0000 7f50feffd700 1 -- 172.21.15.121:0/29354380 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5100013610 con 0x7f510815be30 2023-11-15T02:57:40.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.519+0000 7f50feffd700 1 --2- 172.21.15.121:0/29354380 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50e805a510 0x7f50e805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:40.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.519+0000 7f50feffd700 1 -- 172.21.15.121:0/29354380 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f5100074e30 con 0x7f510815be30 2023-11-15T02:57:40.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.519+0000 7f510659c700 1 --2- 172.21.15.121:0/29354380 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50e805a510 0x7f50e805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:40.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.523+0000 7f510659c700 1 --2- 172.21.15.121:0/29354380 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50e805a510 0x7f50e805c9d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f50f8001690 tx=0x7f50f8009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:40.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.527+0000 7f50feffd700 1 -- 172.21.15.121:0/29354380 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f51000488a0 con 0x7f510815be30 2023-11-15T02:57:40.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.695+0000 7f510759e700 1 -- 172.21.15.121:0/29354380 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51080a7cc0 con 0x7f510815be30 2023-11-15T02:57:40.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.695+0000 7f50feffd700 1 -- 172.21.15.121:0/29354380 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+13760 (secure 0 0 0) 0x7f510004bec0 con 0x7f510815be30 2023-11-15T02:57:40.700 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:40.701 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":29,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:57:37.262188+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:57:40.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.695+0000 7f510759e700 1 -- 172.21.15.121:0/29354380 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50e805a510 msgr2=0x7f50e805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:40.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.699+0000 7f510759e700 1 --2- 172.21.15.121:0/29354380 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50e805a510 0x7f50e805c9d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f50f8001690 tx=0x7f50f8009040).stop 2023-11-15T02:57:40.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.699+0000 7f510759e700 1 -- 172.21.15.121:0/29354380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f510815be30 msgr2=0x7f510815a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:40.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.699+0000 7f510759e700 1 --2- 172.21.15.121:0/29354380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f510815be30 0x7f510815a120 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f5100009e90 tx=0x7f510000b040).stop 2023-11-15T02:57:40.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.699+0000 7f510759e700 1 -- 172.21.15.121:0/29354380 shutdown_connections 2023-11-15T02:57:40.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.699+0000 7f510759e700 1 --2- 172.21.15.121:0/29354380 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50e805a510 0x7f50e805c9d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.699+0000 7f510759e700 1 --2- 172.21.15.121:0/29354380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f510815be30 0x7f510815a120 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.699+0000 7f510759e700 1 --2- 172.21.15.121:0/29354380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5108152850 0x7f5108159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.699+0000 7f510759e700 1 --2- 172.21.15.121:0/29354380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51080c3280 0x7f510814d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.699+0000 7f510759e700 1 -- 172.21.15.121:0/29354380 >> 172.21.15.121:0/29354380 conn(0x7f5108062190 msgr2=0x7f510814e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:40.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.699+0000 7f510759e700 1 -- 172.21.15.121:0/29354380 shutdown_connections 2023-11-15T02:57:40.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.699+0000 7f510759e700 1 -- 172.21.15.121:0/29354380 wait complete. 2023-11-15T02:57:40.723 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-15T02:57:40.723 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T02:57:40.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.883+0000 7fa81759e700 1 Processor -- start 2023-11-15T02:57:40.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.883+0000 7fa81759e700 1 -- start start 2023-11-15T02:57:40.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.883+0000 7fa81759e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa8180acb60 0x7fa8180acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:40.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa81759e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8180c7e90 con 0x7fa81814e380 2023-11-15T02:57:40.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa81759e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8180ad4d0 con 0x7fa818065680 2023-11-15T02:57:40.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa81759e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8180c8320 con 0x7fa8180acb60 2023-11-15T02:57:40.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa815d9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa8180acb60 0x7fa8180acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:40.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa815d9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa8180acb60 0x7fa8180acfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36900/0 (socket says 172.21.15.121:36900) 2023-11-15T02:57:40.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa815d9b700 1 -- 172.21.15.121:0/396066477 learned_addr learned my addr 172.21.15.121:0/396066477 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:40.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa815d9b700 1 -- 172.21.15.121:0/396066477 >> v1:172.21.15.190:6789/0 conn(0x7fa818065680 legacy=0x7fa818065af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:40.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa815d9b700 1 -- 172.21.15.121:0/396066477 >> v1:172.21.15.121:6789/0 conn(0x7fa81814e380 legacy=0x7fa8180c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:40.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa815d9b700 1 -- 172.21.15.121:0/396066477 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81814e760 con 0x7fa8180acb60 2023-11-15T02:57:40.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa815d9b700 1 --2- 172.21.15.121:0/396066477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8180acb60 0x7fa8180acfe0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fa80c006fd0 tx=0x7fa80c0090f0).ready entity=mon.0 client_cookie=d76c15f0947bd67a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:40.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa81559a700 1 -- 172.21.15.121:0/396066477 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c00b600 con 0x7fa8180acb60 2023-11-15T02:57:40.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa81559a700 1 -- 172.21.15.121:0/396066477 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80c00b760 con 0x7fa8180acb60 2023-11-15T02:57:40.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa81559a700 1 -- 172.21.15.121:0/396066477 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c00bae0 con 0x7fa8180acb60 2023-11-15T02:57:40.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa81759e700 1 -- 172.21.15.121:0/396066477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8180acb60 msgr2=0x7fa8180acfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:40.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa81759e700 1 --2- 172.21.15.121:0/396066477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8180acb60 0x7fa8180acfe0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fa80c006fd0 tx=0x7fa80c0090f0).stop 2023-11-15T02:57:40.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa81759e700 1 -- 172.21.15.121:0/396066477 shutdown_connections 2023-11-15T02:57:40.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa81759e700 1 --2- 172.21.15.121:0/396066477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8180acb60 0x7fa8180acfe0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa81759e700 1 -- 172.21.15.121:0/396066477 >> 172.21.15.121:0/396066477 conn(0x7fa818067a60 msgr2=0x7fa818067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:40.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa81759e700 1 -- 172.21.15.121:0/396066477 shutdown_connections 2023-11-15T02:57:40.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.887+0000 7fa81759e700 1 -- 172.21.15.121:0/396066477 wait complete. 2023-11-15T02:57:40.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa81759e700 1 Processor -- start 2023-11-15T02:57:40.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa81759e700 1 -- start start 2023-11-15T02:57:40.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa81759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa818065680 0x7fa8180b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:40.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa81759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8180acb60 0x7fa8180c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:40.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa81759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa81814e380 0x7fa8180c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:40.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa81759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8180c33e0 con 0x7fa81814e380 2023-11-15T02:57:40.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa81759e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8180adb20 con 0x7fa818065680 2023-11-15T02:57:40.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa81759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8180c2b40 con 0x7fa8180acb60 2023-11-15T02:57:40.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa816d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa81814e380 0x7fa8180c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:40.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa816d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa81814e380 0x7fa8180c25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36908/0 (socket says 172.21.15.121:36908) 2023-11-15T02:57:40.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa816d9d700 1 -- 172.21.15.121:0/828418044 learned_addr learned my addr 172.21.15.121:0/828418044 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:40.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa81659c700 1 --2- 172.21.15.121:0/828418044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa818065680 0x7fa8180b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:40.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa815d9b700 1 --2- 172.21.15.121:0/828418044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8180acb60 0x7fa8180c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:40.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa816d9d700 1 -- 172.21.15.121:0/828418044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8180acb60 msgr2=0x7fa8180c20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:40.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa816d9d700 1 --2- 172.21.15.121:0/828418044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8180acb60 0x7fa8180c20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa816d9d700 1 -- 172.21.15.121:0/828418044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa818065680 msgr2=0x7fa8180b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:40.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa816d9d700 1 --2- 172.21.15.121:0/828418044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa818065680 0x7fa8180b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:40.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa816d9d700 1 -- 172.21.15.121:0/828418044 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80c009370 con 0x7fa81814e380 2023-11-15T02:57:40.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa81659c700 1 --2- 172.21.15.121:0/828418044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa818065680 0x7fa8180b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:40.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.891+0000 7fa816d9d700 1 --2- 172.21.15.121:0/828418044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa81814e380 0x7fa8180c25a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fa81000c7f0 tx=0x7fa810001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:40.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.895+0000 7fa8067fc700 1 -- 172.21.15.121:0/828418044 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810010510 con 0x7fa81814e380 2023-11-15T02:57:40.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.895+0000 7fa8067fc700 1 -- 172.21.15.121:0/828418044 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa810010670 con 0x7fa81814e380 2023-11-15T02:57:40.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.895+0000 7fa8067fc700 1 -- 172.21.15.121:0/828418044 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8100109f0 con 0x7fa81814e380 2023-11-15T02:57:40.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.895+0000 7fa81759e700 1 -- 172.21.15.121:0/828418044 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8180c2e20 con 0x7fa81814e380 2023-11-15T02:57:40.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.895+0000 7fa81759e700 1 -- 172.21.15.121:0/828418044 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa818167ab0 con 0x7fa81814e380 2023-11-15T02:57:40.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.895+0000 7fa81759e700 1 -- 172.21.15.121:0/828418044 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8180ad8f0 con 0x7fa81814e380 2023-11-15T02:57:40.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.895+0000 7fa8067fc700 1 -- 172.21.15.121:0/828418044 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa81001b400 con 0x7fa81814e380 2023-11-15T02:57:40.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.899+0000 7fa8067fc700 1 --2- 172.21.15.121:0/828418044 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa7f805a510 0x7fa7f805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:40.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.899+0000 7fa8067fc700 1 -- 172.21.15.121:0/828418044 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fa8100741e0 con 0x7fa81814e380 2023-11-15T02:57:40.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.899+0000 7fa81659c700 1 --2- 172.21.15.121:0/828418044 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa7f805a510 0x7fa7f805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:40.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.899+0000 7fa81659c700 1 --2- 172.21.15.121:0/828418044 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa7f805a510 0x7fa7f805c9d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fa8180c36e0 tx=0x7fa808009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:40.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:40.903+0000 7fa8067fc700 1 -- 172.21.15.121:0/828418044 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa8180ad8f0 con 0x7fa81814e380 2023-11-15T02:57:41.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.059+0000 7fa81759e700 1 -- 172.21.15.121:0/828418044 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa8180ad8f0 con 0x7fa81814e380 2023-11-15T02:57:41.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.059+0000 7fa8067fc700 1 -- 172.21.15.121:0/828418044 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v29) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa8180ad8f0 con 0x7fa81814e380 2023-11-15T02:57:41.066 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T02:57:41.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.063+0000 7fa81759e700 1 -- 172.21.15.121:0/828418044 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa7f805a510 msgr2=0x7fa7f805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:41.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.063+0000 7fa81759e700 1 --2- 172.21.15.121:0/828418044 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa7f805a510 0x7fa7f805c9d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fa8180c36e0 tx=0x7fa808009040).stop 2023-11-15T02:57:41.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.063+0000 7fa81759e700 1 -- 172.21.15.121:0/828418044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa81814e380 msgr2=0x7fa8180c25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:41.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.063+0000 7fa81759e700 1 --2- 172.21.15.121:0/828418044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa81814e380 0x7fa8180c25a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fa81000c7f0 tx=0x7fa810001510).stop 2023-11-15T02:57:41.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.063+0000 7fa81759e700 1 -- 172.21.15.121:0/828418044 shutdown_connections 2023-11-15T02:57:41.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.063+0000 7fa81759e700 1 --2- 172.21.15.121:0/828418044 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa7f805a510 0x7fa7f805c9d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.063+0000 7fa81759e700 1 --2- 172.21.15.121:0/828418044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa81814e380 0x7fa8180c25a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.063+0000 7fa81759e700 1 --2- 172.21.15.121:0/828418044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8180acb60 0x7fa8180c20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.063+0000 7fa81759e700 1 --2- 172.21.15.121:0/828418044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa818065680 0x7fa8180b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.063+0000 7fa81759e700 1 -- 172.21.15.121:0/828418044 >> 172.21.15.121:0/828418044 conn(0x7fa818067a60 msgr2=0x7fa8180c1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:41.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.063+0000 7fa81759e700 1 -- 172.21.15.121:0/828418044 shutdown_connections 2023-11-15T02:57:41.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.063+0000 7fa81759e700 1 -- 172.21.15.121:0/828418044 wait complete. 2023-11-15T02:57:41.088 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T02:57:41.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.247+0000 7f043a08f700 1 Processor -- start 2023-11-15T02:57:41.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.247+0000 7f043a08f700 1 -- start start 2023-11-15T02:57:41.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.251+0000 7f043a08f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04340acb60 0x7f04340acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:41.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.251+0000 7f043a08f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04340c7e90 con 0x7f043414e380 2023-11-15T02:57:41.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.251+0000 7f043a08f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04340ad4d0 con 0x7f0434065680 2023-11-15T02:57:41.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.251+0000 7f043a08f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04340c8320 con 0x7f04340acb60 2023-11-15T02:57:41.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.251+0000 7f043908d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f043414e380 0x7f04340c5b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56286/0 (socket says 172.21.15.121:56286) 2023-11-15T02:57:41.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.251+0000 7f043908d700 1 -- 172.21.15.121:0/898656575 learned_addr learned my addr 172.21.15.121:0/898656575 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:41.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.251+0000 7f043888c700 1 --2- 172.21.15.121:0/898656575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04340acb60 0x7f04340acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:41.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.251+0000 7f042bfff700 1 -- 172.21.15.121:0/898656575 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413223137 0 0) 0x7f04340c7e90 con 0x7f043414e380 2023-11-15T02:57:41.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.251+0000 7f042bfff700 1 -- 172.21.15.121:0/898656575 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04200033f0 con 0x7f043414e380 2023-11-15T02:57:41.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.251+0000 7f043888c700 1 -- 172.21.15.121:0/898656575 >> v1:172.21.15.190:6789/0 conn(0x7f0434065680 legacy=0x7f0434065b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:41.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.251+0000 7f042bfff700 1 -- 172.21.15.121:0/898656575 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2954532333 0 0) 0x7f04340ad4d0 con 0x7f0434065680 2023-11-15T02:57:41.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.251+0000 7f043888c700 1 -- 172.21.15.121:0/898656575 >> v1:172.21.15.121:6789/0 conn(0x7f043414e380 legacy=0x7f04340c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:41.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.251+0000 7f043888c700 1 -- 172.21.15.121:0/898656575 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043414e760 con 0x7f04340acb60 2023-11-15T02:57:41.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.255+0000 7f043888c700 1 --2- 172.21.15.121:0/898656575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04340acb60 0x7f04340acfe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f0424009790 tx=0x7f0424009060).ready entity=mon.2 client_cookie=a23817159809604d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:41.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.255+0000 7f042bfff700 1 -- 172.21.15.121:0/898656575 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042400f510 con 0x7f04340acb60 2023-11-15T02:57:41.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.255+0000 7f042bfff700 1 -- 172.21.15.121:0/898656575 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042400f670 con 0x7f04340acb60 2023-11-15T02:57:41.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.255+0000 7f042bfff700 1 -- 172.21.15.121:0/898656575 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042400f9f0 con 0x7f04340acb60 2023-11-15T02:57:41.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.255+0000 7f043a08f700 1 -- 172.21.15.121:0/898656575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04340acb60 msgr2=0x7f04340acfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:41.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.255+0000 7f043a08f700 1 --2- 172.21.15.121:0/898656575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04340acb60 0x7f04340acfe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f0424009790 tx=0x7f0424009060).stop 2023-11-15T02:57:41.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.255+0000 7f043a08f700 1 -- 172.21.15.121:0/898656575 shutdown_connections 2023-11-15T02:57:41.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.255+0000 7f043a08f700 1 --2- 172.21.15.121:0/898656575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04340acb60 0x7f04340acfe0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.255+0000 7f043a08f700 1 -- 172.21.15.121:0/898656575 >> 172.21.15.121:0/898656575 conn(0x7f0434067a60 msgr2=0x7f0434067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:41.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.255+0000 7f043a08f700 1 -- 172.21.15.121:0/898656575 shutdown_connections 2023-11-15T02:57:41.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.255+0000 7f043a08f700 1 -- 172.21.15.121:0/898656575 wait complete. 2023-11-15T02:57:41.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.259+0000 7f043a08f700 1 Processor -- start 2023-11-15T02:57:41.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.259+0000 7f043a08f700 1 -- start start 2023-11-15T02:57:41.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f043a08f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0434065680 0x7f043414c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:41.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f043a08f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04340acb60 0x7f04341435b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:41.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f043a08f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f043414e380 0x7f043414a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:41.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f043a08f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043414ae60 con 0x7f043414e380 2023-11-15T02:57:41.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f043a08f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043414b140 con 0x7f0434065680 2023-11-15T02:57:41.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f043a08f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043414b420 con 0x7f04340acb60 2023-11-15T02:57:41.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f043988e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f043414e380 0x7f043414a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:41.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f043988e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f043414e380 0x7f043414a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36924/0 (socket says 172.21.15.121:36924) 2023-11-15T02:57:41.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f043988e700 1 -- 172.21.15.121:0/2972867697 learned_addr learned my addr 172.21.15.121:0/2972867697 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:41.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f043908d700 1 --2- 172.21.15.121:0/2972867697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0434065680 0x7f043414c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:41.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f043888c700 1 --2- 172.21.15.121:0/2972867697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04340acb60 0x7f04341435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:41.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f043988e700 1 -- 172.21.15.121:0/2972867697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04340acb60 msgr2=0x7f04341435b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:41.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f043988e700 1 --2- 172.21.15.121:0/2972867697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04340acb60 0x7f04341435b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f043988e700 1 -- 172.21.15.121:0/2972867697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0434065680 msgr2=0x7f043414c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:41.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f043988e700 1 --2- 172.21.15.121:0/2972867697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0434065680 0x7f043414c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f043988e700 1 -- 172.21.15.121:0/2972867697 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04240092a0 con 0x7f043414e380 2023-11-15T02:57:41.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f043988e700 1 --2- 172.21.15.121:0/2972867697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f043414e380 0x7f043414a680 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f042c00c810 tx=0x7f042c001500).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:41.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f04297fa700 1 -- 172.21.15.121:0/2972867697 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c00e4a0 con 0x7f043414e380 2023-11-15T02:57:41.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f04297fa700 1 -- 172.21.15.121:0/2972867697 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042c00e600 con 0x7f043414e380 2023-11-15T02:57:41.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f043a08f700 1 -- 172.21.15.121:0/2972867697 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043414b700 con 0x7f043414e380 2023-11-15T02:57:41.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f04297fa700 1 -- 172.21.15.121:0/2972867697 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c00e9a0 con 0x7f043414e380 2023-11-15T02:57:41.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.263+0000 7f043a08f700 1 -- 172.21.15.121:0/2972867697 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043414ab70 con 0x7f043414e380 2023-11-15T02:57:41.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.267+0000 7f043a08f700 1 -- 172.21.15.121:0/2972867697 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043406bcf0 con 0x7f043414e380 2023-11-15T02:57:41.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.267+0000 7f04297fa700 1 -- 172.21.15.121:0/2972867697 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f042c012400 con 0x7f043414e380 2023-11-15T02:57:41.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.267+0000 7f04297fa700 1 --2- 172.21.15.121:0/2972867697 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f041805a510 0x7f041805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:41.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.267+0000 7f04297fa700 1 -- 172.21.15.121:0/2972867697 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f042c073b50 con 0x7f043414e380 2023-11-15T02:57:41.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.267+0000 7f043908d700 1 --2- 172.21.15.121:0/2972867697 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f041805a510 0x7f041805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:41.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.267+0000 7f043908d700 1 --2- 172.21.15.121:0/2972867697 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f041805a510 0x7f041805c9d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f0430005750 tx=0x7f0430009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:41.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.275+0000 7f04297fa700 1 -- 172.21.15.121:0/2972867697 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f042c047610 con 0x7f043414e380 2023-11-15T02:57:41.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.451+0000 7f043a08f700 1 -- 172.21.15.121:0/2972867697 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0434068a70 con 0x7f043414e380 2023-11-15T02:57:41.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.455+0000 7f04297fa700 1 -- 172.21.15.121:0/2972867697 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f042c017030 con 0x7f043414e380 2023-11-15T02:57:41.460 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T02:57:41.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.455+0000 7f043a08f700 1 -- 172.21.15.121:0/2972867697 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f041805a510 msgr2=0x7f041805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:41.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.455+0000 7f043a08f700 1 --2- 172.21.15.121:0/2972867697 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f041805a510 0x7f041805c9d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f0430005750 tx=0x7f0430009040).stop 2023-11-15T02:57:41.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.455+0000 7f043a08f700 1 -- 172.21.15.121:0/2972867697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f043414e380 msgr2=0x7f043414a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:41.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.455+0000 7f043a08f700 1 --2- 172.21.15.121:0/2972867697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f043414e380 0x7f043414a680 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f042c00c810 tx=0x7f042c001500).stop 2023-11-15T02:57:41.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.455+0000 7f043a08f700 1 -- 172.21.15.121:0/2972867697 shutdown_connections 2023-11-15T02:57:41.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.455+0000 7f043a08f700 1 --2- 172.21.15.121:0/2972867697 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f041805a510 0x7f041805c9d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.455+0000 7f043a08f700 1 --2- 172.21.15.121:0/2972867697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f043414e380 0x7f043414a680 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.455+0000 7f043a08f700 1 --2- 172.21.15.121:0/2972867697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04340acb60 0x7f04341435b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.455+0000 7f043a08f700 1 --2- 172.21.15.121:0/2972867697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0434065680 0x7f043414c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.455+0000 7f043a08f700 1 -- 172.21.15.121:0/2972867697 >> 172.21.15.121:0/2972867697 conn(0x7f0434067a60 msgr2=0x7f04341565e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:41.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.455+0000 7f043a08f700 1 -- 172.21.15.121:0/2972867697 shutdown_connections 2023-11-15T02:57:41.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.455+0000 7f043a08f700 1 -- 172.21.15.121:0/2972867697 wait complete. 2023-11-15T02:57:41.481 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-15T02:57:41.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.635+0000 7f4715769700 1 Processor -- start 2023-11-15T02:57:41.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.635+0000 7f4715769700 1 -- start start 2023-11-15T02:57:41.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.635+0000 7f4715769700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4710067a60 0x7f47100a9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:41.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.635+0000 7f4715769700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4710067f60 con 0x7f47100cb9f0 2023-11-15T02:57:41.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.635+0000 7f4715769700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4710068240 con 0x7f4710153a70 2023-11-15T02:57:41.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.635+0000 7f4715769700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4710068520 con 0x7f4710067a60 2023-11-15T02:57:41.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.635+0000 7f470e7fc700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f47100cb9f0 0x7f47100a8f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56288/0 (socket says 172.21.15.121:56288) 2023-11-15T02:57:41.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.635+0000 7f470e7fc700 1 -- 172.21.15.121:0/1704027038 learned_addr learned my addr 172.21.15.121:0/1704027038 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:41.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.635+0000 7f470dffb700 1 -- 172.21.15.121:0/1704027038 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3142551574 0 0) 0x7f4710067f60 con 0x7f47100cb9f0 2023-11-15T02:57:41.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.635+0000 7f470ffff700 1 --2- 172.21.15.121:0/1704027038 >> v2:172.21.15.190:3300/0 conn(0x7f4710067a60 0x7f47100a9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:41.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.635+0000 7f470dffb700 1 -- 172.21.15.121:0/1704027038 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46f80033f0 con 0x7f47100cb9f0 2023-11-15T02:57:41.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.635+0000 7f470dffb700 1 -- 172.21.15.121:0/1704027038 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1673536161 0 0) 0x7f46f80033f0 con 0x7f47100cb9f0 2023-11-15T02:57:41.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.635+0000 7f470dffb700 1 -- 172.21.15.121:0/1704027038 >> v2:172.21.15.190:3300/0 conn(0x7f4710067a60 msgr2=0x7f47100a9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:41.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.635+0000 7f470dffb700 1 --2- 172.21.15.121:0/1704027038 >> v2:172.21.15.190:3300/0 conn(0x7f4710067a60 0x7f47100a9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.635+0000 7f470dffb700 1 -- 172.21.15.121:0/1704027038 >> v1:172.21.15.190:6789/0 conn(0x7f4710153a70 legacy=0x7f47100cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:41.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470dffb700 1 -- 172.21.15.121:0/1704027038 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4710068800 con 0x7f47100cb9f0 2023-11-15T02:57:41.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470dffb700 1 -- 172.21.15.121:0/1704027038 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f4704001470 con 0x7f47100cb9f0 2023-11-15T02:57:41.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470dffb700 1 -- 172.21.15.121:0/1704027038 >> v1:172.21.15.121:6789/0 conn(0x7f47100cb9f0 legacy=0x7f47100a8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:41.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470dffb700 1 --2- 172.21.15.121:0/1704027038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f80038e0 0x7f46f8003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:41.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470dffb700 1 --2- 172.21.15.121:0/1704027038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46f8004690 0x7f46f8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:41.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470dffb700 1 --2- 172.21.15.121:0/1704027038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46f8005000 0x7f46f800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:41.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470dffb700 1 -- 172.21.15.121:0/1704027038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4710068520 con 0x7f46f8005000 2023-11-15T02:57:41.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470dffb700 1 -- 172.21.15.121:0/1704027038 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f800bee0 con 0x7f46f8004690 2023-11-15T02:57:41.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470dffb700 1 -- 172.21.15.121:0/1704027038 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f800c0a0 con 0x7f46f80038e0 2023-11-15T02:57:41.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470ffff700 1 --2- 172.21.15.121:0/1704027038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46f8005000 0x7f46f800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:41.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470e7fc700 1 --2- 172.21.15.121:0/1704027038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46f8004690 0x7f46f8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:41.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470ffff700 1 -- 172.21.15.121:0/1704027038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f80038e0 msgr2=0x7f46f8003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:41.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470effd700 1 --2- 172.21.15.121:0/1704027038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f80038e0 0x7f46f8003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:41.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470ffff700 1 --2- 172.21.15.121:0/1704027038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f80038e0 0x7f46f8003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470ffff700 1 -- 172.21.15.121:0/1704027038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46f8004690 msgr2=0x7f46f8004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:41.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470ffff700 1 --2- 172.21.15.121:0/1704027038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46f8004690 0x7f46f8004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470ffff700 1 -- 172.21.15.121:0/1704027038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f80033f0 con 0x7f46f8005000 2023-11-15T02:57:41.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470e7fc700 1 --2- 172.21.15.121:0/1704027038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46f8004690 0x7f46f8004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:57:41.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470effd700 1 --2- 172.21.15.121:0/1704027038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f80038e0 0x7f46f8003ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:57:41.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470ffff700 1 -- 172.21.15.121:0/1704027038 reap_dead start 2023-11-15T02:57:41.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470ffff700 1 --2- 172.21.15.121:0/1704027038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46f8005000 0x7f46f800b8e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f4710067e90 tx=0x7f470800b040).ready entity=mon.0 client_cookie=c9bab73b5bbfc91d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:41.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470dffb700 1 -- 172.21.15.121:0/1704027038 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708009ac0 con 0x7f46f8005000 2023-11-15T02:57:41.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470dffb700 1 -- 172.21.15.121:0/1704027038 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470800f040 con 0x7f46f8005000 2023-11-15T02:57:41.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f470dffb700 1 -- 172.21.15.121:0/1704027038 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470800ece0 con 0x7f46f8005000 2023-11-15T02:57:41.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f4715769700 1 -- 172.21.15.121:0/1704027038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46f8005000 msgr2=0x7f46f800b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:41.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f4715769700 1 --2- 172.21.15.121:0/1704027038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46f8005000 0x7f46f800b8e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f4710067e90 tx=0x7f470800b040).stop 2023-11-15T02:57:41.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f4715769700 1 -- 172.21.15.121:0/1704027038 shutdown_connections 2023-11-15T02:57:41.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f4715769700 1 --2- 172.21.15.121:0/1704027038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46f8005000 0x7f46f800b8e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f4715769700 1 -- 172.21.15.121:0/1704027038 >> 172.21.15.121:0/1704027038 conn(0x7f4710095840 msgr2=0x7f4710069820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:41.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f4715769700 1 -- 172.21.15.121:0/1704027038 shutdown_connections 2023-11-15T02:57:41.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.639+0000 7f4715769700 1 -- 172.21.15.121:0/1704027038 wait complete. 2023-11-15T02:57:41.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.643+0000 7f4715769700 1 Processor -- start 2023-11-15T02:57:41.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.643+0000 7f4715769700 1 -- start start 2023-11-15T02:57:41.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.643+0000 7f4715769700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46f8005000 0x7f47101524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:41.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.643+0000 7f4715769700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4710151fe0 0x7f47101517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:41.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.643+0000 7f4715769700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4710156110 0x7f47100ace50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:41.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.643+0000 7f4715769700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4710095c20 con 0x7f4710151fe0 2023-11-15T02:57:41.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.643+0000 7f4715769700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4710154e20 con 0x7f46f8005000 2023-11-15T02:57:41.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.643+0000 7f4715769700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4710155c80 con 0x7f4710156110 2023-11-15T02:57:41.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.647+0000 7f470e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4710151fe0 0x7f47101517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:41.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.647+0000 7f470e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4710151fe0 0x7f47101517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36940/0 (socket says 172.21.15.121:36940) 2023-11-15T02:57:41.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.647+0000 7f470e7fc700 1 -- 172.21.15.121:0/1215105283 learned_addr learned my addr 172.21.15.121:0/1215105283 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:41.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.647+0000 7f470effd700 1 --2- 172.21.15.121:0/1215105283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46f8005000 0x7f47101524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:41.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.647+0000 7f470ffff700 1 --2- 172.21.15.121:0/1215105283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4710156110 0x7f47100ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:41.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.647+0000 7f470e7fc700 1 -- 172.21.15.121:0/1215105283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4710156110 msgr2=0x7f47100ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:41.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.647+0000 7f470e7fc700 1 --2- 172.21.15.121:0/1215105283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4710156110 0x7f47100ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.647+0000 7f470e7fc700 1 -- 172.21.15.121:0/1215105283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46f8005000 msgr2=0x7f47101524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:41.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.647+0000 7f470e7fc700 1 --2- 172.21.15.121:0/1215105283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46f8005000 0x7f47101524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.647+0000 7f470e7fc700 1 -- 172.21.15.121:0/1215105283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4710157120 con 0x7f4710151fe0 2023-11-15T02:57:41.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.647+0000 7f470e7fc700 1 --2- 172.21.15.121:0/1215105283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4710151fe0 0x7f47101517a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f470400d830 tx=0x7f4704009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:41.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.647+0000 7f46f6ffd700 1 -- 172.21.15.121:0/1215105283 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704007170 con 0x7f4710151fe0 2023-11-15T02:57:41.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.647+0000 7f46f6ffd700 1 -- 172.21.15.121:0/1215105283 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47040072d0 con 0x7f4710151fe0 2023-11-15T02:57:41.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.647+0000 7f4715769700 1 -- 172.21.15.121:0/1215105283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4710155070 con 0x7f4710151fe0 2023-11-15T02:57:41.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.647+0000 7f46f6ffd700 1 -- 172.21.15.121:0/1215105283 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47040048c0 con 0x7f4710151fe0 2023-11-15T02:57:41.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.647+0000 7f4715769700 1 -- 172.21.15.121:0/1215105283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4710155630 con 0x7f4710151fe0 2023-11-15T02:57:41.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.647+0000 7f4715769700 1 -- 172.21.15.121:0/1215105283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4710047060 con 0x7f4710151fe0 2023-11-15T02:57:41.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.651+0000 7f46f6ffd700 1 -- 172.21.15.121:0/1215105283 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4704004c70 con 0x7f4710151fe0 2023-11-15T02:57:41.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.651+0000 7f46f6ffd700 1 --2- 172.21.15.121:0/1215105283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f46fc05a510 0x7f46fc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:41.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.651+0000 7f46f6ffd700 1 -- 172.21.15.121:0/1215105283 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f4704073ea0 con 0x7f4710151fe0 2023-11-15T02:57:41.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.651+0000 7f470effd700 1 --2- 172.21.15.121:0/1215105283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f46fc05a510 0x7f46fc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:41.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.651+0000 7f470effd700 1 --2- 172.21.15.121:0/1215105283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f46fc05a510 0x7f46fc05c9d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f4700000c00 tx=0x7f4700009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:41.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.655+0000 7f46f6ffd700 1 -- 172.21.15.121:0/1215105283 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f47040479d0 con 0x7f4710151fe0 2023-11-15T02:57:41.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.815+0000 7f4715769700 1 -- 172.21.15.121:0/1215105283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4710155dc0 con 0x7f4710151fe0 2023-11-15T02:57:41.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.819+0000 7f46f6ffd700 1 -- 172.21.15.121:0/1215105283 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+143 (secure 0 0 0) 0x7f470404aff0 con 0x7f4710151fe0 2023-11-15T02:57:41.823 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:41.824 INFO:teuthology.orchestra.run.smithi121.stdout:[ 2023-11-15T02:57:41.824 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:41.824 INFO:teuthology.orchestra.run.smithi121.stdout: "poolnum": 1, 2023-11-15T02:57:41.824 INFO:teuthology.orchestra.run.smithi121.stdout: "poolname": "device_health_metrics" 2023-11-15T02:57:41.824 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:57:41.825 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:57:41.825 INFO:teuthology.orchestra.run.smithi121.stdout: "poolnum": 2, 2023-11-15T02:57:41.825 INFO:teuthology.orchestra.run.smithi121.stdout: "poolname": "rbd" 2023-11-15T02:57:41.825 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:57:41.825 INFO:teuthology.orchestra.run.smithi121.stdout:] 2023-11-15T02:57:41.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.819+0000 7f4715769700 1 -- 172.21.15.121:0/1215105283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f46fc05a510 msgr2=0x7f46fc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:41.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.819+0000 7f4715769700 1 --2- 172.21.15.121:0/1215105283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f46fc05a510 0x7f46fc05c9d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f4700000c00 tx=0x7f4700009040).stop 2023-11-15T02:57:41.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.819+0000 7f4715769700 1 -- 172.21.15.121:0/1215105283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4710151fe0 msgr2=0x7f47101517a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:41.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.819+0000 7f4715769700 1 --2- 172.21.15.121:0/1215105283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4710151fe0 0x7f47101517a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f470400d830 tx=0x7f4704009040).stop 2023-11-15T02:57:41.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.819+0000 7f4715769700 1 -- 172.21.15.121:0/1215105283 shutdown_connections 2023-11-15T02:57:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.819+0000 7f4715769700 1 --2- 172.21.15.121:0/1215105283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f46fc05a510 0x7f46fc05c9d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.819+0000 7f4715769700 1 --2- 172.21.15.121:0/1215105283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4710156110 0x7f47100ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.819+0000 7f4715769700 1 --2- 172.21.15.121:0/1215105283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4710151fe0 0x7f47101517a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.819+0000 7f4715769700 1 --2- 172.21.15.121:0/1215105283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46f8005000 0x7f47101524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.819+0000 7f4715769700 1 -- 172.21.15.121:0/1215105283 >> 172.21.15.121:0/1215105283 conn(0x7f4710095840 msgr2=0x7f4710152cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:41.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.819+0000 7f4715769700 1 -- 172.21.15.121:0/1215105283 shutdown_connections 2023-11-15T02:57:41.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:41.819+0000 7f4715769700 1 -- 172.21.15.121:0/1215105283 wait complete. 2023-11-15T02:57:41.846 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-15T02:57:41.846 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-15T02:57:42.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.003+0000 7fd96bfff700 1 Processor -- start 2023-11-15T02:57:42.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.007+0000 7fd96bfff700 1 -- start start 2023-11-15T02:57:42.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.007+0000 7fd96bfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd96c0669b0 0x7fd96c158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:42.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.007+0000 7fd96bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd96c0a8750 0x7fd96c0a8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:42.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.007+0000 7fd96bfff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd96c0bb910 con 0x7fd96c157d90 2023-11-15T02:57:42.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.007+0000 7fd96bfff700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd96c0c1fb0 con 0x7fd96c0a8750 2023-11-15T02:57:42.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.007+0000 7fd96bfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd96c15bbd0 con 0x7fd96c0669b0 2023-11-15T02:57:42.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd96a7fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd96c0669b0 0x7fd96c158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:42.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd96b7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd96c0a8750 0x7fd96c0a8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:42.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd96a7fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd96c0669b0 0x7fd96c158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:55842/0 (socket says 172.21.15.121:55842) 2023-11-15T02:57:42.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd96a7fc700 1 -- 172.21.15.121:0/2923945014 learned_addr learned my addr 172.21.15.121:0/2923945014 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:42.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd96b7fe700 1 -- 172.21.15.121:0/2923945014 >> v2:172.21.15.190:3300/0 conn(0x7fd96c0669b0 msgr2=0x7fd96c158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:42.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd96b7fe700 1 --2- 172.21.15.121:0/2923945014 >> v2:172.21.15.190:3300/0 conn(0x7fd96c0669b0 0x7fd96c158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:42.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd96b7fe700 1 -- 172.21.15.121:0/2923945014 >> v1:172.21.15.190:6789/0 conn(0x7fd96c157d90 legacy=0x7fd96c0c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:42.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd96b7fe700 1 -- 172.21.15.121:0/2923945014 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96c0a9090 con 0x7fd96c0a8750 2023-11-15T02:57:42.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd96a7fc700 1 --2- 172.21.15.121:0/2923945014 >> v2:172.21.15.190:3300/0 conn(0x7fd96c0669b0 0x7fd96c158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:57:42.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd96b7fe700 1 --2- 172.21.15.121:0/2923945014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd96c0a8750 0x7fd96c0a8ba0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fd96c15bd10 tx=0x7fd964009780).ready entity=mon.0 client_cookie=a2a60b557bcebb3e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:42.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd969ffb700 1 -- 172.21.15.121:0/2923945014 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96400e070 con 0x7fd96c0a8750 2023-11-15T02:57:42.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd969ffb700 1 -- 172.21.15.121:0/2923945014 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96400bb30 con 0x7fd96c0a8750 2023-11-15T02:57:42.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd969ffb700 1 -- 172.21.15.121:0/2923945014 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd964012410 con 0x7fd96c0a8750 2023-11-15T02:57:42.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd96bfff700 1 -- 172.21.15.121:0/2923945014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd96c0a8750 msgr2=0x7fd96c0a8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:42.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd96bfff700 1 --2- 172.21.15.121:0/2923945014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd96c0a8750 0x7fd96c0a8ba0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fd96c15bd10 tx=0x7fd964009780).stop 2023-11-15T02:57:42.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd96bfff700 1 -- 172.21.15.121:0/2923945014 shutdown_connections 2023-11-15T02:57:42.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd96bfff700 1 --2- 172.21.15.121:0/2923945014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd96c0a8750 0x7fd96c0a8ba0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:42.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd96bfff700 1 --2- 172.21.15.121:0/2923945014 >> v2:172.21.15.190:3300/0 conn(0x7fd96c0669b0 0x7fd96c158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:42.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd96bfff700 1 -- 172.21.15.121:0/2923945014 >> 172.21.15.121:0/2923945014 conn(0x7fd96c067a60 msgr2=0x7fd96c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:42.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd96bfff700 1 -- 172.21.15.121:0/2923945014 shutdown_connections 2023-11-15T02:57:42.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.011+0000 7fd96bfff700 1 -- 172.21.15.121:0/2923945014 wait complete. 2023-11-15T02:57:42.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.015+0000 7fd96bfff700 1 Processor -- start 2023-11-15T02:57:42.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.015+0000 7fd96bfff700 1 -- start start 2023-11-15T02:57:42.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd96bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd96c0669b0 0x7fd96c1556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:42.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd96bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd96c0a8750 0x7fd96c155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:42.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd96bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd96c157d90 0x7fd96c16ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:42.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd96bfff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c1560a0 con 0x7fd96c157d90 2023-11-15T02:57:42.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd96bfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c159ea0 con 0x7fd96c0669b0 2023-11-15T02:57:42.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd96bfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c15a180 con 0x7fd96c0a8750 2023-11-15T02:57:42.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd96b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd96c157d90 0x7fd96c16ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:42.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd96b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd96c157d90 0x7fd96c16ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36948/0 (socket says 172.21.15.121:36948) 2023-11-15T02:57:42.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd96b7fe700 1 -- 172.21.15.121:0/2399200894 learned_addr learned my addr 172.21.15.121:0/2399200894 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:42.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd96affd700 1 --2- 172.21.15.121:0/2399200894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd96c0669b0 0x7fd96c1556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:42.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd96a7fc700 1 --2- 172.21.15.121:0/2399200894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd96c0a8750 0x7fd96c155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:42.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd96b7fe700 1 -- 172.21.15.121:0/2399200894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd96c0a8750 msgr2=0x7fd96c155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:42.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd96b7fe700 1 --2- 172.21.15.121:0/2399200894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd96c0a8750 0x7fd96c155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:42.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd96b7fe700 1 -- 172.21.15.121:0/2399200894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd96c0669b0 msgr2=0x7fd96c1556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:42.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd96b7fe700 1 --2- 172.21.15.121:0/2399200894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd96c0669b0 0x7fd96c1556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:42.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd96b7fe700 1 -- 172.21.15.121:0/2399200894 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd960009300 con 0x7fd96c157d90 2023-11-15T02:57:42.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd96affd700 1 --2- 172.21.15.121:0/2399200894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd96c0669b0 0x7fd96c1556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:57:42.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd96b7fe700 1 --2- 172.21.15.121:0/2399200894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd96c157d90 0x7fd96c16ba60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fd96c0689c0 tx=0x7fd96c063a10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:42.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd952ffd700 1 -- 172.21.15.121:0/2399200894 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd964012410 con 0x7fd96c157d90 2023-11-15T02:57:42.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd952ffd700 1 -- 172.21.15.121:0/2399200894 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96401d630 con 0x7fd96c157d90 2023-11-15T02:57:42.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd96bfff700 1 -- 172.21.15.121:0/2399200894 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd964009a20 con 0x7fd96c157d90 2023-11-15T02:57:42.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.019+0000 7fd952ffd700 1 -- 172.21.15.121:0/2399200894 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96401d9f0 con 0x7fd96c157d90 2023-11-15T02:57:42.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.023+0000 7fd96bfff700 1 -- 172.21.15.121:0/2399200894 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96c15a740 con 0x7fd96c157d90 2023-11-15T02:57:42.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.023+0000 7fd96bfff700 1 -- 172.21.15.121:0/2399200894 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd96c06bcf0 con 0x7fd96c157d90 2023-11-15T02:57:42.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.023+0000 7fd952ffd700 1 -- 172.21.15.121:0/2399200894 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd964003040 con 0x7fd96c157d90 2023-11-15T02:57:42.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.027+0000 7fd952ffd700 1 --2- 172.21.15.121:0/2399200894 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd95805a510 0x7fd95805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:42.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.027+0000 7fd952ffd700 1 -- 172.21.15.121:0/2399200894 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fd964076240 con 0x7fd96c157d90 2023-11-15T02:57:42.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.027+0000 7fd96affd700 1 --2- 172.21.15.121:0/2399200894 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd95805a510 0x7fd95805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:42.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.027+0000 7fd96affd700 1 --2- 172.21.15.121:0/2399200894 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd95805a510 0x7fd95805c9d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fd95c001ba0 tx=0x7fd95c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:42.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.035+0000 7fd952ffd700 1 -- 172.21.15.121:0/2399200894 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd964049cb0 con 0x7fd96c157d90 2023-11-15T02:57:42.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:42.207+0000 7fd96bfff700 1 -- 172.21.15.121:0/2399200894 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fd96c0c3c70 con 0x7fd96c157d90 2023-11-15T02:57:43.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.055+0000 7fd952ffd700 1 -- 172.21.15.121:0/2399200894 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 152+0+0 (secure 0 0 0) 0x7fd96404d2d0 con 0x7fd96c157d90 2023-11-15T02:57:43.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.059+0000 7fd96bfff700 1 -- 172.21.15.121:0/2399200894 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd95805a510 msgr2=0x7fd95805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:43.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.059+0000 7fd96bfff700 1 --2- 172.21.15.121:0/2399200894 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd95805a510 0x7fd95805c9d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fd95c001ba0 tx=0x7fd95c009040).stop 2023-11-15T02:57:43.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.063+0000 7fd96bfff700 1 -- 172.21.15.121:0/2399200894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd96c157d90 msgr2=0x7fd96c16ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:43.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.063+0000 7fd96bfff700 1 --2- 172.21.15.121:0/2399200894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd96c157d90 0x7fd96c16ba60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fd96c0689c0 tx=0x7fd96c063a10).stop 2023-11-15T02:57:43.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.063+0000 7fd96bfff700 1 -- 172.21.15.121:0/2399200894 shutdown_connections 2023-11-15T02:57:43.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.063+0000 7fd96bfff700 1 --2- 172.21.15.121:0/2399200894 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd95805a510 0x7fd95805c9d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:43.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.063+0000 7fd96bfff700 1 --2- 172.21.15.121:0/2399200894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd96c157d90 0x7fd96c16ba60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:43.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.063+0000 7fd96bfff700 1 --2- 172.21.15.121:0/2399200894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd96c0a8750 0x7fd96c155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:43.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.063+0000 7fd96bfff700 1 --2- 172.21.15.121:0/2399200894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd96c0669b0 0x7fd96c1556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:43.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.063+0000 7fd96bfff700 1 -- 172.21.15.121:0/2399200894 >> 172.21.15.121:0/2399200894 conn(0x7fd96c067a60 msgr2=0x7fd96c0bcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:43.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.063+0000 7fd96bfff700 1 -- 172.21.15.121:0/2399200894 shutdown_connections 2023-11-15T02:57:43.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.063+0000 7fd96bfff700 1 -- 172.21.15.121:0/2399200894 wait complete. 2023-11-15T02:57:43.077 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_metadata' created 2023-11-15T02:57:43.095 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-15T02:57:43.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.271+0000 7f5d07cdb700 1 Processor -- start 2023-11-15T02:57:43.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.271+0000 7f5d07cdb700 1 -- start start 2023-11-15T02:57:43.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.275+0000 7f5d07cdb700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5d00158f40 0x7f5d00159320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:43.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.275+0000 7f5d07cdb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d000a6290 con 0x7f5d000bc170 2023-11-15T02:57:43.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.275+0000 7f5d07cdb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d00066b90 con 0x7f5d00067020 2023-11-15T02:57:43.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.275+0000 7f5d07cdb700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d00158640 con 0x7f5d00158f40 2023-11-15T02:57:43.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.275+0000 7f5d05a77700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5d00158f40 0x7f5d00159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:43.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.275+0000 7f5d06278700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f5d000bc170 0x7f5d000bc5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56302/0 (socket says 172.21.15.121:56302) 2023-11-15T02:57:43.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.275+0000 7f5d06278700 1 -- 172.21.15.121:0/208351581 learned_addr learned my addr 172.21.15.121:0/208351581 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:43.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.279+0000 7f5d05a77700 1 -- 172.21.15.121:0/208351581 >> v1:172.21.15.190:6789/0 conn(0x7f5d00067020 legacy=0x7f5d000a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:43.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.279+0000 7f5d05a77700 1 -- 172.21.15.121:0/208351581 >> v1:172.21.15.121:6789/0 conn(0x7f5d000bc170 legacy=0x7f5d000bc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:43.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.279+0000 7f5d05a77700 1 -- 172.21.15.121:0/208351581 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d00066260 con 0x7f5d00158f40 2023-11-15T02:57:43.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.279+0000 7f5d04a75700 1 -- 172.21.15.121:0/208351581 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357208954 0 0) 0x7f5d00066b90 con 0x7f5d00067020 2023-11-15T02:57:43.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.279+0000 7f5d05a77700 1 --2- 172.21.15.121:0/208351581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d00158f40 0x7f5d00159320 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f5cfc009760 tx=0x7f5cfc009060).ready entity=mon.0 client_cookie=e72e8ce5a591ea1c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:43.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.279+0000 7f5d04a75700 1 -- 172.21.15.121:0/208351581 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc00f510 con 0x7f5d00158f40 2023-11-15T02:57:43.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.279+0000 7f5d04a75700 1 -- 172.21.15.121:0/208351581 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cfc00f670 con 0x7f5d00158f40 2023-11-15T02:57:43.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.279+0000 7f5d04a75700 1 -- 172.21.15.121:0/208351581 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc00f9f0 con 0x7f5d00158f40 2023-11-15T02:57:43.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.279+0000 7f5d07cdb700 1 -- 172.21.15.121:0/208351581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d00158f40 msgr2=0x7f5d00159320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:43.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.279+0000 7f5d07cdb700 1 --2- 172.21.15.121:0/208351581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d00158f40 0x7f5d00159320 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f5cfc009760 tx=0x7f5cfc009060).stop 2023-11-15T02:57:43.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.279+0000 7f5d07cdb700 1 -- 172.21.15.121:0/208351581 shutdown_connections 2023-11-15T02:57:43.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.279+0000 7f5d07cdb700 1 --2- 172.21.15.121:0/208351581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d00158f40 0x7f5d00159320 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:43.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.279+0000 7f5d07cdb700 1 -- 172.21.15.121:0/208351581 >> 172.21.15.121:0/208351581 conn(0x7f5d00068500 msgr2=0x7f5d000c6780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:43.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.279+0000 7f5d07cdb700 1 -- 172.21.15.121:0/208351581 shutdown_connections 2023-11-15T02:57:43.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.279+0000 7f5d07cdb700 1 -- 172.21.15.121:0/208351581 wait complete. 2023-11-15T02:57:43.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.279+0000 7f5d07cdb700 1 Processor -- start 2023-11-15T02:57:43.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.279+0000 7f5d07cdb700 1 -- start start 2023-11-15T02:57:43.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5d07cdb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d00067020 0x7f5d000c0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:43.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5d07cdb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d000bc170 0x7f5d000c3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:43.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5d07cdb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d00158f40 0x7f5d000c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:43.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5d07cdb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d000c3ee0 con 0x7f5d00067020 2023-11-15T02:57:43.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5d07cdb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0016cec0 con 0x7f5d000bc170 2023-11-15T02:57:43.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5d07cdb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d00169bd0 con 0x7f5d00158f40 2023-11-15T02:57:43.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5d05a77700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d00067020 0x7f5d000c0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:43.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5d05a77700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d00067020 0x7f5d000c0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36962/0 (socket says 172.21.15.121:36962) 2023-11-15T02:57:43.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5d05a77700 1 -- 172.21.15.121:0/3679333038 learned_addr learned my addr 172.21.15.121:0/3679333038 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:43.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5d06278700 1 --2- 172.21.15.121:0/3679333038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d00158f40 0x7f5d000c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:43.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5d05276700 1 --2- 172.21.15.121:0/3679333038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d000bc170 0x7f5d000c3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:43.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5d05a77700 1 -- 172.21.15.121:0/3679333038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d00158f40 msgr2=0x7f5d000c39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:43.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5d05a77700 1 --2- 172.21.15.121:0/3679333038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d00158f40 0x7f5d000c39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:43.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5d05a77700 1 -- 172.21.15.121:0/3679333038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d000bc170 msgr2=0x7f5d000c3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:43.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5d05a77700 1 --2- 172.21.15.121:0/3679333038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d000bc170 0x7f5d000c3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:43.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5d05a77700 1 -- 172.21.15.121:0/3679333038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc0092a0 con 0x7f5d00067020 2023-11-15T02:57:43.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5d05a77700 1 --2- 172.21.15.121:0/3679333038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d00067020 0x7f5d000c0070 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f5cfc011e50 tx=0x7f5cfc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:43.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5cf5ffb700 1 -- 172.21.15.121:0/3679333038 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc00f7e0 con 0x7f5d00067020 2023-11-15T02:57:43.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5cf5ffb700 1 -- 172.21.15.121:0/3679333038 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cfc00f940 con 0x7f5d00067020 2023-11-15T02:57:43.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5cf5ffb700 1 -- 172.21.15.121:0/3679333038 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc01b640 con 0x7f5d00067020 2023-11-15T02:57:43.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5d07cdb700 1 -- 172.21.15.121:0/3679333038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0016c100 con 0x7f5d00067020 2023-11-15T02:57:43.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.283+0000 7f5d07cdb700 1 -- 172.21.15.121:0/3679333038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0016c6c0 con 0x7f5d00067020 2023-11-15T02:57:43.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.287+0000 7f5d07cdb700 1 -- 172.21.15.121:0/3679333038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0006c150 con 0x7f5d00067020 2023-11-15T02:57:43.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.287+0000 7f5cf5ffb700 1 -- 172.21.15.121:0/3679333038 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5cfc01b9b0 con 0x7f5d00067020 2023-11-15T02:57:43.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.287+0000 7f5cf5ffb700 1 --2- 172.21.15.121:0/3679333038 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5ce405a510 0x7f5ce405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:43.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.287+0000 7f5cf5ffb700 1 -- 172.21.15.121:0/3679333038 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f5cfc075e80 con 0x7f5d00067020 2023-11-15T02:57:43.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.291+0000 7f5d05276700 1 --2- 172.21.15.121:0/3679333038 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5ce405a510 0x7f5ce405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:43.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.291+0000 7f5d05276700 1 --2- 172.21.15.121:0/3679333038 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5ce405a510 0x7f5ce405c9d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f5cf0007fd0 tx=0x7f5cf0007bb0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:43.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.295+0000 7f5cf5ffb700 1 -- 172.21.15.121:0/3679333038 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5cfc049840 con 0x7f5d00067020 2023-11-15T02:57:43.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:43.463+0000 7f5d07cdb700 1 -- 172.21.15.121:0/3679333038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5d00065950 con 0x7f5d00067020 2023-11-15T02:57:44.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.067+0000 7f5cf5ffb700 1 -- 172.21.15.121:0/3679333038 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5cfc029070 con 0x7f5d00067020 2023-11-15T02:57:44.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.071+0000 7f5d07cdb700 1 -- 172.21.15.121:0/3679333038 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5ce405a510 msgr2=0x7f5ce405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:44.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.071+0000 7f5d07cdb700 1 --2- 172.21.15.121:0/3679333038 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5ce405a510 0x7f5ce405c9d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f5cf0007fd0 tx=0x7f5cf0007bb0).stop 2023-11-15T02:57:44.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.071+0000 7f5d07cdb700 1 -- 172.21.15.121:0/3679333038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d00067020 msgr2=0x7f5d000c0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:44.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.071+0000 7f5d07cdb700 1 --2- 172.21.15.121:0/3679333038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d00067020 0x7f5d000c0070 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f5cfc011e50 tx=0x7f5cfc017000).stop 2023-11-15T02:57:44.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.071+0000 7f5d07cdb700 1 -- 172.21.15.121:0/3679333038 shutdown_connections 2023-11-15T02:57:44.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.071+0000 7f5d07cdb700 1 --2- 172.21.15.121:0/3679333038 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5ce405a510 0x7f5ce405c9d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.071+0000 7f5d07cdb700 1 --2- 172.21.15.121:0/3679333038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d00158f40 0x7f5d000c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.071+0000 7f5d07cdb700 1 --2- 172.21.15.121:0/3679333038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d000bc170 0x7f5d000c3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.071+0000 7f5d07cdb700 1 --2- 172.21.15.121:0/3679333038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d00067020 0x7f5d000c0070 secure :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0x7f5cfc011e50 tx=0x7f5cfc017000).stop 2023-11-15T02:57:44.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.071+0000 7f5d07cdb700 1 -- 172.21.15.121:0/3679333038 >> 172.21.15.121:0/3679333038 conn(0x7f5d00068500 msgr2=0x7f5d00063020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:44.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.079+0000 7f5d07cdb700 1 -- 172.21.15.121:0/3679333038 shutdown_connections 2023-11-15T02:57:44.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.083+0000 7f5d07cdb700 1 -- 172.21.15.121:0/3679333038 wait complete. 2023-11-15T02:57:44.152 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_data' created 2023-11-15T02:57:44.152 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-15T02:57:44.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.311+0000 7f7234bb9700 1 Processor -- start 2023-11-15T02:57:44.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.311+0000 7f7234bb9700 1 -- start start 2023-11-15T02:57:44.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.315+0000 7f7234bb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72300c5c70 0x7f7230153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:44.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.315+0000 7f7234bb9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7230066330 0x7f72300ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:44.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.315+0000 7f7234bb9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7230065ea0 con 0x7f72300acd00 2023-11-15T02:57:44.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.315+0000 7f7234bb9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7230066bb0 con 0x7f7230066330 2023-11-15T02:57:44.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.315+0000 7f7234bb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7230065890 con 0x7f72300c5c70 2023-11-15T02:57:44.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.315+0000 7f722f7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f72300acd00 0x7f72301536c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56316/0 (socket says 172.21.15.121:56316) 2023-11-15T02:57:44.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.315+0000 7f722f7fe700 1 -- 172.21.15.121:0/3691664844 learned_addr learned my addr 172.21.15.121:0/3691664844 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:44.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.315+0000 7f722ffff700 1 --2- 172.21.15.121:0/3691664844 >> v2:172.21.15.190:3300/0 conn(0x7f7230066330 0x7f72300ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:44.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.315+0000 7f722effd700 1 --2- 172.21.15.121:0/3691664844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72300c5c70 0x7f7230153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:44.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.315+0000 7f722e7fc700 1 -- 172.21.15.121:0/3691664844 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3900506270 0 0) 0x7f7230065ea0 con 0x7f72300acd00 2023-11-15T02:57:44.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.315+0000 7f722e7fc700 1 -- 172.21.15.121:0/3691664844 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72180033f0 con 0x7f72300acd00 2023-11-15T02:57:44.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.315+0000 7f722ffff700 1 -- 172.21.15.121:0/3691664844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72300c5c70 msgr2=0x7f7230153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:44.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.315+0000 7f722ffff700 1 --2- 172.21.15.121:0/3691664844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72300c5c70 0x7f7230153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.315+0000 7f722ffff700 1 -- 172.21.15.121:0/3691664844 >> v1:172.21.15.121:6789/0 conn(0x7f72300acd00 legacy=0x7f72301536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:44.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.315+0000 7f722ffff700 1 -- 172.21.15.121:0/3691664844 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72300c64a0 con 0x7f7230066330 2023-11-15T02:57:44.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.319+0000 7f722ffff700 1 --2- 172.21.15.121:0/3691664844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7230066330 0x7f72300ad0e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f7228009e60 tx=0x7f7228009780).ready entity=mon.1 client_cookie=bbef8ca1e839617 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:44.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.319+0000 7f722e7fc700 1 -- 172.21.15.121:0/3691664844 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722800e070 con 0x7f7230066330 2023-11-15T02:57:44.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.319+0000 7f722e7fc700 1 -- 172.21.15.121:0/3691664844 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722800bab0 con 0x7f7230066330 2023-11-15T02:57:44.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.319+0000 7f722e7fc700 1 -- 172.21.15.121:0/3691664844 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722800be30 con 0x7f7230066330 2023-11-15T02:57:44.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.319+0000 7f7234bb9700 1 -- 172.21.15.121:0/3691664844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7230066330 msgr2=0x7f72300ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:44.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.319+0000 7f7234bb9700 1 --2- 172.21.15.121:0/3691664844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7230066330 0x7f72300ad0e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f7228009e60 tx=0x7f7228009780).stop 2023-11-15T02:57:44.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.319+0000 7f7234bb9700 1 -- 172.21.15.121:0/3691664844 shutdown_connections 2023-11-15T02:57:44.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.319+0000 7f7234bb9700 1 --2- 172.21.15.121:0/3691664844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7230066330 0x7f72300ad0e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.319+0000 7f7234bb9700 1 --2- 172.21.15.121:0/3691664844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72300c5c70 0x7f7230153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.319+0000 7f7234bb9700 1 -- 172.21.15.121:0/3691664844 >> 172.21.15.121:0/3691664844 conn(0x7f7230063fc0 msgr2=0x7f7230067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:44.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.319+0000 7f7234bb9700 1 -- 172.21.15.121:0/3691664844 shutdown_connections 2023-11-15T02:57:44.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.319+0000 7f7234bb9700 1 -- 172.21.15.121:0/3691664844 wait complete. 2023-11-15T02:57:44.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.319+0000 7f7234bb9700 1 Processor -- start 2023-11-15T02:57:44.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f7234bb9700 1 -- start start 2023-11-15T02:57:44.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f7234bb9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7230066330 0x7f723014edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:44.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f7234bb9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72300acd00 0x7f72300ac180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:44.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f7234bb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72300c5c70 0x7f7230151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:44.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f7234bb9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72300c1d30 con 0x7f72300acd00 2023-11-15T02:57:44.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f7234bb9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7230168de0 con 0x7f7230066330 2023-11-15T02:57:44.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f7234bb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7230167f60 con 0x7f72300c5c70 2023-11-15T02:57:44.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f722effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72300acd00 0x7f72300ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:44.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f722effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72300acd00 0x7f72300ac180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36964/0 (socket says 172.21.15.121:36964) 2023-11-15T02:57:44.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f722effd700 1 -- 172.21.15.121:0/695724961 learned_addr learned my addr 172.21.15.121:0/695724961 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:44.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f722f7fe700 1 --2- 172.21.15.121:0/695724961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7230066330 0x7f723014edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:44.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f722ffff700 1 --2- 172.21.15.121:0/695724961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72300c5c70 0x7f7230151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:44.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f722effd700 1 -- 172.21.15.121:0/695724961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72300c5c70 msgr2=0x7f7230151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:44.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f722effd700 1 --2- 172.21.15.121:0/695724961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72300c5c70 0x7f7230151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f722effd700 1 -- 172.21.15.121:0/695724961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7230066330 msgr2=0x7f723014edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:44.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f722effd700 1 --2- 172.21.15.121:0/695724961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7230066330 0x7f723014edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f722effd700 1 -- 172.21.15.121:0/695724961 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7228009a20 con 0x7f72300acd00 2023-11-15T02:57:44.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f722f7fe700 1 --2- 172.21.15.121:0/695724961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7230066330 0x7f723014edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:44.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f722effd700 1 --2- 172.21.15.121:0/695724961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72300acd00 0x7f72300ac180 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f722400e900 tx=0x7f7224009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:44.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f72177fe700 1 -- 172.21.15.121:0/695724961 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722400c640 con 0x7f72300acd00 2023-11-15T02:57:44.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f72177fe700 1 -- 172.21.15.121:0/695724961 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722400c7a0 con 0x7f72300acd00 2023-11-15T02:57:44.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f7234bb9700 1 -- 172.21.15.121:0/695724961 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72301681d0 con 0x7f72300acd00 2023-11-15T02:57:44.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f72177fe700 1 -- 172.21.15.121:0/695724961 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722400cb20 con 0x7f72300acd00 2023-11-15T02:57:44.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.323+0000 7f7234bb9700 1 -- 172.21.15.121:0/695724961 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7230168790 con 0x7f72300acd00 2023-11-15T02:57:44.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.327+0000 7f7234bb9700 1 -- 172.21.15.121:0/695724961 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7230168f90 con 0x7f72300acd00 2023-11-15T02:57:44.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.327+0000 7f72177fe700 1 -- 172.21.15.121:0/695724961 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f722401b400 con 0x7f72300acd00 2023-11-15T02:57:44.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.327+0000 7f72177fe700 1 --2- 172.21.15.121:0/695724961 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f721c05a510 0x7f721c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:44.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.331+0000 7f72177fe700 1 -- 172.21.15.121:0/695724961 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6021+0+0 (secure 0 0 0) 0x7f7224074c50 con 0x7f72300acd00 2023-11-15T02:57:44.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.331+0000 7f722f7fe700 1 --2- 172.21.15.121:0/695724961 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f721c05a510 0x7f721c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:44.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.331+0000 7f722f7fe700 1 --2- 172.21.15.121:0/695724961 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f721c05a510 0x7f721c05c9d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f7220000c00 tx=0x7f7220009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:44.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.339+0000 7f72177fe700 1 -- 172.21.15.121:0/695724961 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7224048370 con 0x7f72300acd00 2023-11-15T02:57:44.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.535+0000 7f7234bb9700 1 -- 172.21.15.121:0/695724961 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f723006bcf0 con 0x7f72300acd00 2023-11-15T02:57:44.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.543+0000 7f72177fe700 1 -- 172.21.15.121:0/695724961 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f7224016030 con 0x7f72300acd00 2023-11-15T02:57:44.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.551+0000 7f7234bb9700 1 -- 172.21.15.121:0/695724961 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f721c05a510 msgr2=0x7f721c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:44.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.551+0000 7f7234bb9700 1 --2- 172.21.15.121:0/695724961 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f721c05a510 0x7f721c05c9d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f7220000c00 tx=0x7f7220009040).stop 2023-11-15T02:57:44.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.551+0000 7f7234bb9700 1 -- 172.21.15.121:0/695724961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72300acd00 msgr2=0x7f72300ac180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:44.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.551+0000 7f7234bb9700 1 --2- 172.21.15.121:0/695724961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72300acd00 0x7f72300ac180 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f722400e900 tx=0x7f7224009060).stop 2023-11-15T02:57:44.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.551+0000 7f7234bb9700 1 -- 172.21.15.121:0/695724961 shutdown_connections 2023-11-15T02:57:44.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.551+0000 7f7234bb9700 1 --2- 172.21.15.121:0/695724961 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f721c05a510 0x7f721c05c9d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.551+0000 7f7234bb9700 1 --2- 172.21.15.121:0/695724961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72300c5c70 0x7f7230151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.551+0000 7f7234bb9700 1 --2- 172.21.15.121:0/695724961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72300acd00 0x7f72300ac180 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.551+0000 7f7234bb9700 1 --2- 172.21.15.121:0/695724961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7230066330 0x7f723014edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.551+0000 7f7234bb9700 1 -- 172.21.15.121:0/695724961 >> 172.21.15.121:0/695724961 conn(0x7f7230063fc0 msgr2=0x7f72300afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:44.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.555+0000 7f7234bb9700 1 -- 172.21.15.121:0/695724961 shutdown_connections 2023-11-15T02:57:44.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.555+0000 7f7234bb9700 1 -- 172.21.15.121:0/695724961 wait complete. 2023-11-15T02:57:44.568 INFO:teuthology.orchestra.run.smithi121.stderr:new fs with metadata pool 5 and data pool 6 2023-11-15T02:57:44.592 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:57:44.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.739+0000 7f25542f1700 1 Processor -- start 2023-11-15T02:57:44.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.739+0000 7f25542f1700 1 -- start start 2023-11-15T02:57:44.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.739+0000 7f25542f1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f254c1539d0 0x7f254c0c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:44.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.739+0000 7f25542f1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f254c153db0 0x7f254c0c2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:44.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.739+0000 7f25542f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f254c0c19c0 0x7f254c0c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:44.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.739+0000 7f25542f1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f254c1541f0 con 0x7f254c153db0 2023-11-15T02:57:44.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.739+0000 7f25542f1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f254c0c2b60 con 0x7f254c1539d0 2023-11-15T02:57:44.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.739+0000 7f25542f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0c1530 con 0x7f254c0c19c0 2023-11-15T02:57:44.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.739+0000 7f255188c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f254c153db0 0x7f254c0c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:44.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.739+0000 7f255188c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f254c153db0 0x7f254c0c2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56310/0 (socket says 172.21.15.121:56310) 2023-11-15T02:57:44.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.739+0000 7f255188c700 1 -- 172.21.15.121:0/2465075575 learned_addr learned my addr 172.21.15.121:0/2465075575 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:44.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f255208d700 1 --2- 172.21.15.121:0/2465075575 >> v2:172.21.15.190:3300/0 conn(0x7f254c1539d0 0x7f254c0c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:44.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f255288e700 1 --2- 172.21.15.121:0/2465075575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f254c0c19c0 0x7f254c0c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:44.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f255188c700 1 -- 172.21.15.121:0/2465075575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f254c0c19c0 msgr2=0x7f254c0c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:44.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f255188c700 1 --2- 172.21.15.121:0/2465075575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f254c0c19c0 0x7f254c0c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f255188c700 1 -- 172.21.15.121:0/2465075575 >> v2:172.21.15.190:3300/0 conn(0x7f254c1539d0 msgr2=0x7f254c0c06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:44.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f255188c700 1 --2- 172.21.15.121:0/2465075575 >> v2:172.21.15.190:3300/0 conn(0x7f254c1539d0 0x7f254c0c06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f255188c700 1 -- 172.21.15.121:0/2465075575 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c0c2d80 con 0x7f254c153db0 2023-11-15T02:57:44.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f255288e700 1 --2- 172.21.15.121:0/2465075575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f254c0c19c0 0x7f254c0c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:57:44.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f255188c700 1 --2- 172.21.15.121:0/2465075575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f254c153db0 0x7f254c0c2670 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f254c0c4270 tx=0x7f253c0090f0).ready entity=mon.0 client_cookie=e2e315b7c9c4e868 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:44.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f255108b700 1 -- 172.21.15.121:0/2465075575 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c00b600 con 0x7f254c153db0 2023-11-15T02:57:44.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f255108b700 1 -- 172.21.15.121:0/2465075575 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253c00b760 con 0x7f254c153db0 2023-11-15T02:57:44.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f255108b700 1 -- 172.21.15.121:0/2465075575 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c00bae0 con 0x7f254c153db0 2023-11-15T02:57:44.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f25542f1700 1 -- 172.21.15.121:0/2465075575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f254c153db0 msgr2=0x7f254c0c2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:44.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f25542f1700 1 --2- 172.21.15.121:0/2465075575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f254c153db0 0x7f254c0c2670 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f254c0c4270 tx=0x7f253c0090f0).stop 2023-11-15T02:57:44.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f25542f1700 1 -- 172.21.15.121:0/2465075575 shutdown_connections 2023-11-15T02:57:44.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f25542f1700 1 --2- 172.21.15.121:0/2465075575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f254c0c19c0 0x7f254c0c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f25542f1700 1 --2- 172.21.15.121:0/2465075575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f254c153db0 0x7f254c0c2670 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f25542f1700 1 --2- 172.21.15.121:0/2465075575 >> v2:172.21.15.190:3300/0 conn(0x7f254c1539d0 0x7f254c0c06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f25542f1700 1 -- 172.21.15.121:0/2465075575 >> 172.21.15.121:0/2465075575 conn(0x7f254c062190 msgr2=0x7f254c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:44.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f25542f1700 1 -- 172.21.15.121:0/2465075575 shutdown_connections 2023-11-15T02:57:44.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.743+0000 7f25542f1700 1 -- 172.21.15.121:0/2465075575 wait complete. 2023-11-15T02:57:44.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.747+0000 7f25542f1700 1 Processor -- start 2023-11-15T02:57:44.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.747+0000 7f25542f1700 1 -- start start 2023-11-15T02:57:44.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.747+0000 7f25542f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f254c0c19c0 0x7f254c1514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:44.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f25542f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f254c1539d0 0x7f254c13e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:44.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f25542f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f254c153db0 0x7f254c065750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:44.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f25542f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c151050 con 0x7f254c1539d0 2023-11-15T02:57:44.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f25542f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c13e730 con 0x7f254c0c19c0 2023-11-15T02:57:44.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f25542f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c150440 con 0x7f254c153db0 2023-11-15T02:57:44.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f255188c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f254c1539d0 0x7f254c13e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:44.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f255188c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f254c1539d0 0x7f254c13e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56314/0 (socket says 172.21.15.121:56314) 2023-11-15T02:57:44.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f255188c700 1 -- 172.21.15.121:0/4280571953 learned_addr learned my addr 172.21.15.121:0/4280571953 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:44.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f255208d700 1 --2- 172.21.15.121:0/4280571953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f254c0c19c0 0x7f254c1514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:44.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f255288e700 1 --2- 172.21.15.121:0/4280571953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f254c153db0 0x7f254c065750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:44.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f255188c700 1 -- 172.21.15.121:0/4280571953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f254c153db0 msgr2=0x7f254c065750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:44.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f255188c700 1 --2- 172.21.15.121:0/4280571953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f254c153db0 0x7f254c065750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f255188c700 1 -- 172.21.15.121:0/4280571953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f254c0c19c0 msgr2=0x7f254c1514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:44.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f255188c700 1 --2- 172.21.15.121:0/4280571953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f254c0c19c0 0x7f254c1514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f255188c700 1 -- 172.21.15.121:0/4280571953 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c009370 con 0x7f254c1539d0 2023-11-15T02:57:44.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f255288e700 1 --2- 172.21.15.121:0/4280571953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f254c153db0 0x7f254c065750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:44.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f255188c700 1 --2- 172.21.15.121:0/4280571953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f254c1539d0 0x7f254c13e240 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f253c009130 tx=0x7f253c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:44.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f25427fc700 1 -- 172.21.15.121:0/4280571953 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c00b8d0 con 0x7f254c1539d0 2023-11-15T02:57:44.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f25427fc700 1 -- 172.21.15.121:0/4280571953 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253c00ba30 con 0x7f254c1539d0 2023-11-15T02:57:44.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f25542f1700 1 -- 172.21.15.121:0/4280571953 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254c150720 con 0x7f254c1539d0 2023-11-15T02:57:44.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f25427fc700 1 -- 172.21.15.121:0/4280571953 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c01c670 con 0x7f254c1539d0 2023-11-15T02:57:44.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f25542f1700 1 -- 172.21.15.121:0/4280571953 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254c150ce0 con 0x7f254c1539d0 2023-11-15T02:57:44.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.751+0000 7f25542f1700 1 -- 172.21.15.121:0/4280571953 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254c06c150 con 0x7f254c1539d0 2023-11-15T02:57:44.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.755+0000 7f25427fc700 1 -- 172.21.15.121:0/4280571953 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f253c01ca20 con 0x7f254c1539d0 2023-11-15T02:57:44.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.759+0000 7f25427fc700 1 --2- 172.21.15.121:0/4280571953 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f253005a510 0x7f253005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:44.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.759+0000 7f25427fc700 1 -- 172.21.15.121:0/4280571953 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f253c076400 con 0x7f254c1539d0 2023-11-15T02:57:44.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.759+0000 7f255208d700 1 --2- 172.21.15.121:0/4280571953 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f253005a510 0x7f253005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:44.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.759+0000 7f255208d700 1 --2- 172.21.15.121:0/4280571953 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f253005a510 0x7f253005c9d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f254800bbc0 tx=0x7f2548008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:44.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.763+0000 7f25427fc700 1 -- 172.21.15.121:0/4280571953 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f253c049b20 con 0x7f254c1539d0 2023-11-15T02:57:44.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.927+0000 7f25542f1700 1 -- 172.21.15.121:0/4280571953 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f254c0bffe0 con 0x7f254c1539d0 2023-11-15T02:57:44.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.927+0000 7f25427fc700 1 -- 172.21.15.121:0/4280571953 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16989 (secure 0 0 0) 0x7f253c04d140 con 0x7f254c1539d0 2023-11-15T02:57:44.935 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:44.936 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":32,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:57:44.540462+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:57:42.277674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-15T02:57:43.540361+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:57:44.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.931+0000 7f25542f1700 1 -- 172.21.15.121:0/4280571953 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f253005a510 msgr2=0x7f253005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:44.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.931+0000 7f25542f1700 1 --2- 172.21.15.121:0/4280571953 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f253005a510 0x7f253005c9d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f254800bbc0 tx=0x7f2548008040).stop 2023-11-15T02:57:44.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.931+0000 7f25542f1700 1 -- 172.21.15.121:0/4280571953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f254c1539d0 msgr2=0x7f254c13e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:44.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.931+0000 7f25542f1700 1 --2- 172.21.15.121:0/4280571953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f254c1539d0 0x7f254c13e240 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f253c009130 tx=0x7f253c017000).stop 2023-11-15T02:57:44.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.931+0000 7f25542f1700 1 -- 172.21.15.121:0/4280571953 shutdown_connections 2023-11-15T02:57:44.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.931+0000 7f25542f1700 1 --2- 172.21.15.121:0/4280571953 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f253005a510 0x7f253005c9d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.931+0000 7f25542f1700 1 --2- 172.21.15.121:0/4280571953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f254c153db0 0x7f254c065750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.931+0000 7f25542f1700 1 --2- 172.21.15.121:0/4280571953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f254c1539d0 0x7f254c13e240 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.931+0000 7f25542f1700 1 --2- 172.21.15.121:0/4280571953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f254c0c19c0 0x7f254c1514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:44.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.931+0000 7f25542f1700 1 -- 172.21.15.121:0/4280571953 >> 172.21.15.121:0/4280571953 conn(0x7f254c062190 msgr2=0x7f254c066c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:44.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.931+0000 7f25542f1700 1 -- 172.21.15.121:0/4280571953 shutdown_connections 2023-11-15T02:57:44.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:44.931+0000 7f25542f1700 1 -- 172.21.15.121:0/4280571953 wait complete. 2023-11-15T02:57:44.956 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:57:45.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.115+0000 7f71cf8f1700 1 Processor -- start 2023-11-15T02:57:45.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.119+0000 7f71cf8f1700 1 -- start start 2023-11-15T02:57:45.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.119+0000 7f71cf8f1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f71c81562f0 0x7f71c8155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:45.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.119+0000 7f71cf8f1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c80c4c30 con 0x7f71c80a57c0 2023-11-15T02:57:45.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.119+0000 7f71cf8f1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c80a6f50 con 0x7f71c8154ca0 2023-11-15T02:57:45.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.119+0000 7f71cf8f1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c80a8280 con 0x7f71c81562f0 2023-11-15T02:57:45.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.119+0000 7f71cde8e700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f71c80a57c0 0x7f71c80c50c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40634/0 (socket says 172.21.15.121:40634) 2023-11-15T02:57:45.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.119+0000 7f71cde8e700 1 -- 172.21.15.121:0/1274565587 learned_addr learned my addr 172.21.15.121:0/1274565587 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:45.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.119+0000 7f71cce8c700 1 --2- 172.21.15.121:0/1274565587 >> v2:172.21.15.190:3300/0 conn(0x7f71c81562f0 0x7f71c8155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:45.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.119+0000 7f71bffff700 1 -- 172.21.15.121:0/1274565587 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299501806 0 0) 0x7f71c80c4c30 con 0x7f71c80a57c0 2023-11-15T02:57:45.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.119+0000 7f71bffff700 1 -- 172.21.15.121:0/1274565587 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71b40033f0 con 0x7f71c80a57c0 2023-11-15T02:57:45.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.119+0000 7f71bffff700 1 -- 172.21.15.121:0/1274565587 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 800079615 0 0) 0x7f71c80a6f50 con 0x7f71c8154ca0 2023-11-15T02:57:45.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71bffff700 1 -- 172.21.15.121:0/1274565587 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71c80c4c30 con 0x7f71c8154ca0 2023-11-15T02:57:45.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71bffff700 1 -- 172.21.15.121:0/1274565587 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3520092902 0 0) 0x7f71b40033f0 con 0x7f71c80a57c0 2023-11-15T02:57:45.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71bffff700 1 -- 172.21.15.121:0/1274565587 >> v2:172.21.15.190:3300/0 conn(0x7f71c81562f0 msgr2=0x7f71c8155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:45.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71bffff700 1 --2- 172.21.15.121:0/1274565587 >> v2:172.21.15.190:3300/0 conn(0x7f71c81562f0 0x7f71c8155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:45.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71bffff700 1 -- 172.21.15.121:0/1274565587 >> v1:172.21.15.190:6789/0 conn(0x7f71c8154ca0 legacy=0x7f71c80a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:45.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71bffff700 1 -- 172.21.15.121:0/1274565587 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c80a7230 con 0x7f71c80a57c0 2023-11-15T02:57:45.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71bffff700 1 -- 172.21.15.121:0/1274565587 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f71c00017f0 con 0x7f71c80a57c0 2023-11-15T02:57:45.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71bffff700 1 -- 172.21.15.121:0/1274565587 >> v1:172.21.15.121:6789/0 conn(0x7f71c80a57c0 legacy=0x7f71c80c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:45.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71bffff700 1 --2- 172.21.15.121:0/1274565587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71b4003ac0 0x7f71b4003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:45.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71bffff700 1 --2- 172.21.15.121:0/1274565587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71b4004690 0x7f71b4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:45.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71bffff700 1 --2- 172.21.15.121:0/1274565587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71b4005000 0x7f71b400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:45.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71bffff700 1 -- 172.21.15.121:0/1274565587 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80a8280 con 0x7f71b4003ac0 2023-11-15T02:57:45.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71bffff700 1 -- 172.21.15.121:0/1274565587 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b400beb0 con 0x7f71b4004690 2023-11-15T02:57:45.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71bffff700 1 -- 172.21.15.121:0/1274565587 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b400c070 con 0x7f71b4005000 2023-11-15T02:57:45.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71cce8c700 1 --2- 172.21.15.121:0/1274565587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71b4004690 0x7f71b4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:45.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71cd68d700 1 --2- 172.21.15.121:0/1274565587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71b4003ac0 0x7f71b4003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:45.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71cde8e700 1 --2- 172.21.15.121:0/1274565587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71b4005000 0x7f71b400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:45.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71cd68d700 1 -- 172.21.15.121:0/1274565587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71b4005000 msgr2=0x7f71b400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:45.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71cd68d700 1 --2- 172.21.15.121:0/1274565587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71b4005000 0x7f71b400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:45.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71cd68d700 1 -- 172.21.15.121:0/1274565587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71b4004690 msgr2=0x7f71b4004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:45.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71cd68d700 1 --2- 172.21.15.121:0/1274565587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71b4004690 0x7f71b4004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:45.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71cd68d700 1 -- 172.21.15.121:0/1274565587 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b40033f0 con 0x7f71b4003ac0 2023-11-15T02:57:45.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71cde8e700 1 -- 172.21.15.121:0/1274565587 reap_dead start 2023-11-15T02:57:45.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71cd68d700 1 --2- 172.21.15.121:0/1274565587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71b4003ac0 0x7f71b4003ec0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f71c4001890 tx=0x7f71c4007bb0).ready entity=mon.0 client_cookie=2fa63351d24e7fe1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:45.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71bffff700 1 -- 172.21.15.121:0/1274565587 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4007020 con 0x7f71b4003ac0 2023-11-15T02:57:45.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71bffff700 1 -- 172.21.15.121:0/1274565587 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c4007180 con 0x7f71b4003ac0 2023-11-15T02:57:45.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71bffff700 1 -- 172.21.15.121:0/1274565587 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4007500 con 0x7f71b4003ac0 2023-11-15T02:57:45.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71cf8f1700 1 -- 172.21.15.121:0/1274565587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71b4003ac0 msgr2=0x7f71b4003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:45.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71cf8f1700 1 --2- 172.21.15.121:0/1274565587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71b4003ac0 0x7f71b4003ec0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f71c4001890 tx=0x7f71c4007bb0).stop 2023-11-15T02:57:45.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71cf8f1700 1 -- 172.21.15.121:0/1274565587 shutdown_connections 2023-11-15T02:57:45.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71cf8f1700 1 --2- 172.21.15.121:0/1274565587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71b4003ac0 0x7f71b4003ec0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:45.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71cf8f1700 1 -- 172.21.15.121:0/1274565587 >> 172.21.15.121:0/1274565587 conn(0x7f71c8062190 msgr2=0x7f71c80625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:45.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.123+0000 7f71cf8f1700 1 -- 172.21.15.121:0/1274565587 shutdown_connections 2023-11-15T02:57:45.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.127+0000 7f71cf8f1700 1 -- 172.21.15.121:0/1274565587 wait complete. 2023-11-15T02:57:45.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.127+0000 7f71cf8f1700 1 Processor -- start 2023-11-15T02:57:45.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.127+0000 7f71cf8f1700 1 -- start start 2023-11-15T02:57:45.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cf8f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71b4003ac0 0x7f71c80aab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:45.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cf8f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c80ab010 0x7f71c8151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:45.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cf8f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c8151ff0 0x7f71c8152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:45.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cf8f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8152470 con 0x7f71c80ab010 2023-11-15T02:57:45.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cf8f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80a9830 con 0x7f71b4003ac0 2023-11-15T02:57:45.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cf8f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80aa690 con 0x7f71c8151ff0 2023-11-15T02:57:45.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cce8c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c80ab010 0x7f71c8151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:45.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cd68d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71b4003ac0 0x7f71c80aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:45.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cd68d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71b4003ac0 0x7f71c80aab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:48570/0 (socket says 172.21.15.121:48570) 2023-11-15T02:57:45.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cd68d700 1 -- 172.21.15.121:0/577729414 learned_addr learned my addr 172.21.15.121:0/577729414 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:45.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cde8e700 1 --2- 172.21.15.121:0/577729414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c8151ff0 0x7f71c8152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:45.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cce8c700 1 -- 172.21.15.121:0/577729414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c8151ff0 msgr2=0x7f71c8152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:45.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cce8c700 1 --2- 172.21.15.121:0/577729414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c8151ff0 0x7f71c8152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:45.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cce8c700 1 -- 172.21.15.121:0/577729414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71b4003ac0 msgr2=0x7f71c80aab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:45.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cce8c700 1 --2- 172.21.15.121:0/577729414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71b4003ac0 0x7f71c80aab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:45.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cce8c700 1 -- 172.21.15.121:0/577729414 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c80a9a80 con 0x7f71c80ab010 2023-11-15T02:57:45.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cd68d700 1 --2- 172.21.15.121:0/577729414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71b4003ac0 0x7f71c80aab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:57:45.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cce8c700 1 --2- 172.21.15.121:0/577729414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c80ab010 0x7f71c8151b00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f71b80119c0 tx=0x7f71b8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:45.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71bd7fa700 1 -- 172.21.15.121:0/577729414 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8011e50 con 0x7f71c80ab010 2023-11-15T02:57:45.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71bd7fa700 1 -- 172.21.15.121:0/577729414 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b800f8e0 con 0x7f71c80ab010 2023-11-15T02:57:45.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cf8f1700 1 -- 172.21.15.121:0/577729414 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c80a9d60 con 0x7f71c80ab010 2023-11-15T02:57:45.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71bd7fa700 1 -- 172.21.15.121:0/577729414 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b800fc60 con 0x7f71c80ab010 2023-11-15T02:57:45.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.131+0000 7f71cf8f1700 1 -- 172.21.15.121:0/577729414 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c80aa320 con 0x7f71c80ab010 2023-11-15T02:57:45.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.135+0000 7f71cf8f1700 1 -- 172.21.15.121:0/577729414 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c806c150 con 0x7f71c80ab010 2023-11-15T02:57:45.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.135+0000 7f71bd7fa700 1 -- 172.21.15.121:0/577729414 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f71b8014650 con 0x7f71c80ab010 2023-11-15T02:57:45.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.135+0000 7f71bd7fa700 1 --2- 172.21.15.121:0/577729414 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71ac05a510 0x7f71ac05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:45.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.135+0000 7f71bd7fa700 1 -- 172.21.15.121:0/577729414 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f71b8074c40 con 0x7f71c80ab010 2023-11-15T02:57:45.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.135+0000 7f71cd68d700 1 --2- 172.21.15.121:0/577729414 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71ac05a510 0x7f71ac05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:45.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.135+0000 7f71cd68d700 1 --2- 172.21.15.121:0/577729414 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71ac05a510 0x7f71ac05c9d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f71c4008fd0 tx=0x7f71c4015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:45.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.139+0000 7f71bd7fa700 1 -- 172.21.15.121:0/577729414 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f71b8048420 con 0x7f71c80ab010 2023-11-15T02:57:45.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.303+0000 7f71cf8f1700 1 -- 172.21.15.121:0/577729414 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71c80c4500 con 0x7f71c80ab010 2023-11-15T02:57:45.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.303+0000 7f71bd7fa700 1 -- 172.21.15.121:0/577729414 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16989 (secure 0 0 0) 0x7f71b804ba40 con 0x7f71c80ab010 2023-11-15T02:57:45.310 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:45.310 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":32,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:57:44.540462+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:57:42.277674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-15T02:57:43.540361+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:57:45.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.307+0000 7f71cf8f1700 1 -- 172.21.15.121:0/577729414 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71ac05a510 msgr2=0x7f71ac05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:45.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.307+0000 7f71cf8f1700 1 --2- 172.21.15.121:0/577729414 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71ac05a510 0x7f71ac05c9d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f71c4008fd0 tx=0x7f71c4015000).stop 2023-11-15T02:57:45.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.307+0000 7f71cf8f1700 1 -- 172.21.15.121:0/577729414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c80ab010 msgr2=0x7f71c8151b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:45.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.307+0000 7f71cf8f1700 1 --2- 172.21.15.121:0/577729414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c80ab010 0x7f71c8151b00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f71b80119c0 tx=0x7f71b8009060).stop 2023-11-15T02:57:45.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.307+0000 7f71cf8f1700 1 -- 172.21.15.121:0/577729414 shutdown_connections 2023-11-15T02:57:45.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.307+0000 7f71cf8f1700 1 --2- 172.21.15.121:0/577729414 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71ac05a510 0x7f71ac05c9d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:45.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.307+0000 7f71cf8f1700 1 --2- 172.21.15.121:0/577729414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c8151ff0 0x7f71c8152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:45.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.307+0000 7f71cf8f1700 1 --2- 172.21.15.121:0/577729414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c80ab010 0x7f71c8151b00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:45.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.307+0000 7f71cf8f1700 1 --2- 172.21.15.121:0/577729414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71b4003ac0 0x7f71c80aab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:45.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.307+0000 7f71cf8f1700 1 -- 172.21.15.121:0/577729414 >> 172.21.15.121:0/577729414 conn(0x7f71c8062190 msgr2=0x7f71c8146860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:45.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.307+0000 7f71cf8f1700 1 -- 172.21.15.121:0/577729414 shutdown_connections 2023-11-15T02:57:45.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.307+0000 7f71cf8f1700 1 -- 172.21.15.121:0/577729414 wait complete. 2023-11-15T02:57:45.336 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-15T02:57:45.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.503+0000 7fb713d1b700 1 Processor -- start 2023-11-15T02:57:45.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.503+0000 7fb713d1b700 1 -- start start 2023-11-15T02:57:45.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.503+0000 7fb713d1b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb70c067020 0x7fb70c1582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:45.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.503+0000 7fb713d1b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb70c068170 0x7fb70c0bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:45.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.503+0000 7fb713d1b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb70c062da0 con 0x7fb70c157ed0 2023-11-15T02:57:45.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.503+0000 7fb713d1b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb70c0a93a0 con 0x7fb70c067020 2023-11-15T02:57:45.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.503+0000 7fb713d1b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb70c062190 con 0x7fb70c068170 2023-11-15T02:57:45.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb711ab7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb70c157ed0 0x7fb70c067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40642/0 (socket says 172.21.15.121:40642) 2023-11-15T02:57:45.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb711ab7700 1 -- 172.21.15.121:0/3626140038 learned_addr learned my addr 172.21.15.121:0/3626140038 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:45.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb7112b6700 1 --2- 172.21.15.121:0/3626140038 >> v2:172.21.15.121:3300/0 conn(0x7fb70c067020 0x7fb70c1582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:45.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb7122b8700 1 --2- 172.21.15.121:0/3626140038 >> v2:172.21.15.190:3300/0 conn(0x7fb70c068170 0x7fb70c0bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:45.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb710ab5700 1 -- 172.21.15.121:0/3626140038 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2151619072 0 0) 0x7fb70c062da0 con 0x7fb70c157ed0 2023-11-15T02:57:45.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb710ab5700 1 -- 172.21.15.121:0/3626140038 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6f80033f0 con 0x7fb70c157ed0 2023-11-15T02:57:45.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb7112b6700 1 -- 172.21.15.121:0/3626140038 >> v2:172.21.15.190:3300/0 conn(0x7fb70c068170 msgr2=0x7fb70c0bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:45.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb7112b6700 1 --2- 172.21.15.121:0/3626140038 >> v2:172.21.15.190:3300/0 conn(0x7fb70c068170 0x7fb70c0bc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:45.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb7112b6700 1 -- 172.21.15.121:0/3626140038 >> v1:172.21.15.121:6789/0 conn(0x7fb70c157ed0 legacy=0x7fb70c067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:45.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb7112b6700 1 -- 172.21.15.121:0/3626140038 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c062470 con 0x7fb70c067020 2023-11-15T02:57:45.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb7112b6700 1 --2- 172.21.15.121:0/3626140038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb70c067020 0x7fb70c1582b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fb70c063690 tx=0x7fb6fc009060).ready entity=mon.0 client_cookie=44dc4cb8a2f1e410 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:45.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb710ab5700 1 -- 172.21.15.121:0/3626140038 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00d4a0 con 0x7fb70c067020 2023-11-15T02:57:45.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb710ab5700 1 -- 172.21.15.121:0/3626140038 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc00d600 con 0x7fb70c067020 2023-11-15T02:57:45.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb710ab5700 1 -- 172.21.15.121:0/3626140038 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00d980 con 0x7fb70c067020 2023-11-15T02:57:45.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb713d1b700 1 -- 172.21.15.121:0/3626140038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb70c067020 msgr2=0x7fb70c1582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:45.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb713d1b700 1 --2- 172.21.15.121:0/3626140038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb70c067020 0x7fb70c1582b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fb70c063690 tx=0x7fb6fc009060).stop 2023-11-15T02:57:45.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb713d1b700 1 -- 172.21.15.121:0/3626140038 shutdown_connections 2023-11-15T02:57:45.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb713d1b700 1 --2- 172.21.15.121:0/3626140038 >> v2:172.21.15.190:3300/0 conn(0x7fb70c068170 0x7fb70c0bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:45.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb713d1b700 1 --2- 172.21.15.121:0/3626140038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb70c067020 0x7fb70c1582b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:45.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb713d1b700 1 -- 172.21.15.121:0/3626140038 >> 172.21.15.121:0/3626140038 conn(0x7fb70c0591c0 msgr2=0x7fb70c0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:45.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.507+0000 7fb713d1b700 1 -- 172.21.15.121:0/3626140038 shutdown_connections 2023-11-15T02:57:45.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.511+0000 7fb713d1b700 1 -- 172.21.15.121:0/3626140038 wait complete. 2023-11-15T02:57:45.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.511+0000 7fb713d1b700 1 Processor -- start 2023-11-15T02:57:45.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.511+0000 7fb713d1b700 1 -- start start 2023-11-15T02:57:45.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb713d1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb70c067020 0x7fb70c15b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:45.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb713d1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb70c068170 0x7fb70c0aadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:45.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb713d1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb70c157ed0 0x7fb70c156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:45.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb713d1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c09ff40 con 0x7fb70c157ed0 2023-11-15T02:57:45.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb713d1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0a04c0 con 0x7fb70c068170 2023-11-15T02:57:45.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb713d1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0ab2a0 con 0x7fb70c067020 2023-11-15T02:57:45.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb7122b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb70c157ed0 0x7fb70c156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:45.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb7122b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb70c157ed0 0x7fb70c156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56350/0 (socket says 172.21.15.121:56350) 2023-11-15T02:57:45.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb7122b8700 1 -- 172.21.15.121:0/865908780 learned_addr learned my addr 172.21.15.121:0/865908780 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:45.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb711ab7700 1 --2- 172.21.15.121:0/865908780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb70c067020 0x7fb70c15b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:45.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb7112b6700 1 --2- 172.21.15.121:0/865908780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb70c068170 0x7fb70c0aadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:45.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb7122b8700 1 -- 172.21.15.121:0/865908780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb70c067020 msgr2=0x7fb70c15b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:45.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb7122b8700 1 --2- 172.21.15.121:0/865908780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb70c067020 0x7fb70c15b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:45.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb7122b8700 1 -- 172.21.15.121:0/865908780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb70c068170 msgr2=0x7fb70c0aadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:45.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb7122b8700 1 --2- 172.21.15.121:0/865908780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb70c068170 0x7fb70c0aadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:45.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb7122b8700 1 -- 172.21.15.121:0/865908780 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6fc0092a0 con 0x7fb70c157ed0 2023-11-15T02:57:45.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb7122b8700 1 --2- 172.21.15.121:0/865908780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb70c157ed0 0x7fb70c156a70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fb704009ef0 tx=0x7fb70400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:45.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb701ffb700 1 -- 172.21.15.121:0/865908780 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70400cb70 con 0x7fb70c157ed0 2023-11-15T02:57:45.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb701ffb700 1 -- 172.21.15.121:0/865908780 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70400f040 con 0x7fb70c157ed0 2023-11-15T02:57:45.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb701ffb700 1 -- 172.21.15.121:0/865908780 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb704013400 con 0x7fb70c157ed0 2023-11-15T02:57:45.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.515+0000 7fb713d1b700 1 -- 172.21.15.121:0/865908780 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb70c0a01e0 con 0x7fb70c157ed0 2023-11-15T02:57:45.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.519+0000 7fb713d1b700 1 -- 172.21.15.121:0/865908780 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb70c15b140 con 0x7fb70c157ed0 2023-11-15T02:57:45.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.519+0000 7fb713d1b700 1 -- 172.21.15.121:0/865908780 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70c15aca0 con 0x7fb70c157ed0 2023-11-15T02:57:45.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.519+0000 7fb701ffb700 1 -- 172.21.15.121:0/865908780 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb7040137b0 con 0x7fb70c157ed0 2023-11-15T02:57:45.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.523+0000 7fb701ffb700 1 --2- 172.21.15.121:0/865908780 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb6f005a510 0x7fb6f005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:45.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.523+0000 7fb701ffb700 1 -- 172.21.15.121:0/865908780 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb7040750a0 con 0x7fb70c157ed0 2023-11-15T02:57:45.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.523+0000 7fb711ab7700 1 --2- 172.21.15.121:0/865908780 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb6f005a510 0x7fb6f005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:45.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.523+0000 7fb711ab7700 1 --2- 172.21.15.121:0/865908780 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb6f005a510 0x7fb6f005c9d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fb708007fd0 tx=0x7fb708009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:45.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.527+0000 7fb701ffb700 1 -- 172.21.15.121:0/865908780 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb7040487c0 con 0x7fb70c157ed0 2023-11-15T02:57:45.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:45.731+0000 7fb713d1b700 1 -- 172.21.15.121:0/865908780 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb70c15b330 con 0x7fb70c157ed0 2023-11-15T02:57:46.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.563+0000 7fb701ffb700 1 -- 172.21.15.121:0/865908780 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb70404bde0 con 0x7fb70c157ed0 2023-11-15T02:57:46.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.571+0000 7fb713d1b700 1 -- 172.21.15.121:0/865908780 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb6f005a510 msgr2=0x7fb6f005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:46.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.571+0000 7fb713d1b700 1 --2- 172.21.15.121:0/865908780 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb6f005a510 0x7fb6f005c9d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fb708007fd0 tx=0x7fb708009040).stop 2023-11-15T02:57:46.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.571+0000 7fb713d1b700 1 -- 172.21.15.121:0/865908780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb70c157ed0 msgr2=0x7fb70c156a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:46.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.571+0000 7fb713d1b700 1 --2- 172.21.15.121:0/865908780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb70c157ed0 0x7fb70c156a70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fb704009ef0 tx=0x7fb70400b040).stop 2023-11-15T02:57:46.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.571+0000 7fb713d1b700 1 -- 172.21.15.121:0/865908780 shutdown_connections 2023-11-15T02:57:46.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.571+0000 7fb713d1b700 1 --2- 172.21.15.121:0/865908780 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb6f005a510 0x7fb6f005c9d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:46.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.571+0000 7fb713d1b700 1 --2- 172.21.15.121:0/865908780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb70c157ed0 0x7fb70c156a70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:46.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.571+0000 7fb713d1b700 1 --2- 172.21.15.121:0/865908780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb70c068170 0x7fb70c0aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:46.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.571+0000 7fb713d1b700 1 --2- 172.21.15.121:0/865908780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb70c067020 0x7fb70c15b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:46.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.571+0000 7fb713d1b700 1 -- 172.21.15.121:0/865908780 >> 172.21.15.121:0/865908780 conn(0x7fb70c0591c0 msgr2=0x7fb70c0a8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:46.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.571+0000 7fb713d1b700 1 -- 172.21.15.121:0/865908780 shutdown_connections 2023-11-15T02:57:46.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.571+0000 7fb713d1b700 1 -- 172.21.15.121:0/865908780 wait complete. 2023-11-15T02:57:46.598 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-15T02:57:46.599 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-15T02:57:46.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.763+0000 7f8b9b59e700 1 Processor -- start 2023-11-15T02:57:46.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.763+0000 7f8b9b59e700 1 -- start start 2023-11-15T02:57:46.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.767+0000 7f8b9b59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8b9c068c10 0x7f8b9c1574e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:46.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.767+0000 7f8b9b59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b9c064600 0x7f8b9c068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:46.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.767+0000 7f8b9b59e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b9c063560 con 0x7f8b9c062c80 2023-11-15T02:57:46.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.767+0000 7f8b9b59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b9c063840 con 0x7f8b9c068c10 2023-11-15T02:57:46.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.767+0000 7f8b9b59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c063b20 con 0x7f8b9c064600 2023-11-15T02:57:46.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.767+0000 7f8b9a59c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8b9c068c10 0x7f8b9c1574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:46.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.767+0000 7f8b9a59c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8b9c068c10 0x7f8b9c1574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56362/0 (socket says 172.21.15.121:56362) 2023-11-15T02:57:46.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.767+0000 7f8b9a59c700 1 -- 172.21.15.121:0/4133996343 learned_addr learned my addr 172.21.15.121:0/4133996343 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:46.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.767+0000 7f8b99d9b700 1 --2- 172.21.15.121:0/4133996343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b9c064600 0x7f8b9c068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:46.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.771+0000 7f8b9a59c700 1 -- 172.21.15.121:0/4133996343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b9c064600 msgr2=0x7f8b9c068ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:46.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.771+0000 7f8b9a59c700 1 --2- 172.21.15.121:0/4133996343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b9c064600 0x7f8b9c068ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:46.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.771+0000 7f8b9a59c700 1 -- 172.21.15.121:0/4133996343 >> v1:172.21.15.190:6789/0 conn(0x7f8b9c062c80 legacy=0x7f8b9c0a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:46.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.771+0000 7f8b9a59c700 1 -- 172.21.15.121:0/4133996343 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9c063e00 con 0x7f8b9c068c10 2023-11-15T02:57:46.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.771+0000 7f8b9a59c700 1 --2- 172.21.15.121:0/4133996343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c068c10 0x7f8b9c1574e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f8b8c009830 tx=0x7f8b8c0090f0).ready entity=mon.0 client_cookie=5c8920516b2e7193 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:46.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.771+0000 7f8b9959a700 1 -- 172.21.15.121:0/4133996343 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c00b580 con 0x7f8b9c068c10 2023-11-15T02:57:46.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.771+0000 7f8b9959a700 1 -- 172.21.15.121:0/4133996343 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c00b6e0 con 0x7f8b9c068c10 2023-11-15T02:57:46.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.771+0000 7f8b9959a700 1 -- 172.21.15.121:0/4133996343 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c00ba60 con 0x7f8b9c068c10 2023-11-15T02:57:46.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.771+0000 7f8b9b59e700 1 -- 172.21.15.121:0/4133996343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c068c10 msgr2=0x7f8b9c1574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:46.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.771+0000 7f8b9b59e700 1 --2- 172.21.15.121:0/4133996343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c068c10 0x7f8b9c1574e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f8b8c009830 tx=0x7f8b8c0090f0).stop 2023-11-15T02:57:46.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.771+0000 7f8b9b59e700 1 -- 172.21.15.121:0/4133996343 shutdown_connections 2023-11-15T02:57:46.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.771+0000 7f8b9b59e700 1 --2- 172.21.15.121:0/4133996343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b9c064600 0x7f8b9c068ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:46.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.771+0000 7f8b9b59e700 1 --2- 172.21.15.121:0/4133996343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c068c10 0x7f8b9c1574e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:46.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.771+0000 7f8b9b59e700 1 -- 172.21.15.121:0/4133996343 >> 172.21.15.121:0/4133996343 conn(0x7f8b9c066010 msgr2=0x7f8b9c066420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:46.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.771+0000 7f8b9b59e700 1 -- 172.21.15.121:0/4133996343 shutdown_connections 2023-11-15T02:57:46.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.771+0000 7f8b9b59e700 1 -- 172.21.15.121:0/4133996343 wait complete. 2023-11-15T02:57:46.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.775+0000 7f8b9b59e700 1 Processor -- start 2023-11-15T02:57:46.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.775+0000 7f8b9b59e700 1 -- start start 2023-11-15T02:57:46.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.775+0000 7f8b9b59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b9c062c80 0x7f8b9c0aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:46.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.775+0000 7f8b9b59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c068c10 0x7f8b9c14e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:46.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.775+0000 7f8b9b59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b9c164400 0x7f8b9c164860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:46.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.775+0000 7f8b9b59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c0aab30 con 0x7f8b9c068c10 2023-11-15T02:57:46.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.775+0000 7f8b9b59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c164e10 con 0x7f8b9c062c80 2023-11-15T02:57:46.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.775+0000 7f8b9b59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c1650f0 con 0x7f8b9c164400 2023-11-15T02:57:46.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.775+0000 7f8b99d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c068c10 0x7f8b9c14e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:46.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.775+0000 7f8b99d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c068c10 0x7f8b9c14e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56378/0 (socket says 172.21.15.121:56378) 2023-11-15T02:57:46.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.775+0000 7f8b99d9b700 1 -- 172.21.15.121:0/301961506 learned_addr learned my addr 172.21.15.121:0/301961506 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:46.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.775+0000 7f8b9a59c700 1 --2- 172.21.15.121:0/301961506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b9c062c80 0x7f8b9c0aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:46.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.775+0000 7f8b9ad9d700 1 --2- 172.21.15.121:0/301961506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b9c164400 0x7f8b9c164860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:46.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.775+0000 7f8b99d9b700 1 -- 172.21.15.121:0/301961506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b9c164400 msgr2=0x7f8b9c164860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:46.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.775+0000 7f8b99d9b700 1 --2- 172.21.15.121:0/301961506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b9c164400 0x7f8b9c164860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:46.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.775+0000 7f8b99d9b700 1 -- 172.21.15.121:0/301961506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b9c062c80 msgr2=0x7f8b9c0aafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:46.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.775+0000 7f8b99d9b700 1 --2- 172.21.15.121:0/301961506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b9c062c80 0x7f8b9c0aafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:46.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.775+0000 7f8b99d9b700 1 -- 172.21.15.121:0/301961506 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c009370 con 0x7f8b9c068c10 2023-11-15T02:57:46.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.779+0000 7f8b99d9b700 1 --2- 172.21.15.121:0/301961506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c068c10 0x7f8b9c14e820 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f8b9000e930 tx=0x7f8b90009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:46.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.779+0000 7f8b8a7fc700 1 -- 172.21.15.121:0/301961506 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b90012600 con 0x7f8b9c068c10 2023-11-15T02:57:46.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.779+0000 7f8b8a7fc700 1 -- 172.21.15.121:0/301961506 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b90012760 con 0x7f8b9c068c10 2023-11-15T02:57:46.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.779+0000 7f8b9b59e700 1 -- 172.21.15.121:0/301961506 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b9c0aa7c0 con 0x7f8b9c068c10 2023-11-15T02:57:46.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.779+0000 7f8b8a7fc700 1 -- 172.21.15.121:0/301961506 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b90012ae0 con 0x7f8b9c068c10 2023-11-15T02:57:46.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.779+0000 7f8b9b59e700 1 -- 172.21.15.121:0/301961506 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9c14ed10 con 0x7f8b9c068c10 2023-11-15T02:57:46.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.779+0000 7f8b9b59e700 1 -- 172.21.15.121:0/301961506 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b9c06bcf0 con 0x7f8b9c068c10 2023-11-15T02:57:46.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.779+0000 7f8b8a7fc700 1 -- 172.21.15.121:0/301961506 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8b9001b400 con 0x7f8b9c068c10 2023-11-15T02:57:46.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.783+0000 7f8b8a7fc700 1 --2- 172.21.15.121:0/301961506 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8b7c05a540 0x7f8b7c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:46.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.783+0000 7f8b8a7fc700 1 -- 172.21.15.121:0/301961506 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8b90074c40 con 0x7f8b9c068c10 2023-11-15T02:57:46.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.783+0000 7f8b9a59c700 1 --2- 172.21.15.121:0/301961506 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8b7c05a540 0x7f8b7c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:46.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.783+0000 7f8b9a59c700 1 --2- 172.21.15.121:0/301961506 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8b7c05a540 0x7f8b7c05ca00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f8b8c009130 tx=0x7f8b8c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:46.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.787+0000 7f8b8a7fc700 1 -- 172.21.15.121:0/301961506 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8b90048300 con 0x7f8b9c068c10 2023-11-15T02:57:47.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:46.999+0000 7f8b9b59e700 1 -- 172.21.15.121:0/301961506 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8b9c0640e0 con 0x7f8b9c068c10 2023-11-15T02:57:47.021 INFO:tasks.ceph.mds.c.smithi121.stderr:2023-11-15T02:57:47.015+0000 7fb9c16dc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T02:57:47.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.571+0000 7f8b8a7fc700 1 -- 172.21.15.121:0/301961506 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8b9004b920 con 0x7f8b9c068c10 2023-11-15T02:57:47.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.575+0000 7f8b9b59e700 1 -- 172.21.15.121:0/301961506 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8b7c05a540 msgr2=0x7f8b7c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:47.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.575+0000 7f8b9b59e700 1 --2- 172.21.15.121:0/301961506 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8b7c05a540 0x7f8b7c05ca00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f8b8c009130 tx=0x7f8b8c017000).stop 2023-11-15T02:57:47.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.575+0000 7f8b9b59e700 1 -- 172.21.15.121:0/301961506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c068c10 msgr2=0x7f8b9c14e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:47.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.575+0000 7f8b9b59e700 1 --2- 172.21.15.121:0/301961506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c068c10 0x7f8b9c14e820 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f8b9000e930 tx=0x7f8b90009060).stop 2023-11-15T02:57:47.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.575+0000 7f8b9b59e700 1 -- 172.21.15.121:0/301961506 shutdown_connections 2023-11-15T02:57:47.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.575+0000 7f8b9b59e700 1 --2- 172.21.15.121:0/301961506 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8b7c05a540 0x7f8b7c05ca00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:47.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.575+0000 7f8b9b59e700 1 --2- 172.21.15.121:0/301961506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b9c164400 0x7f8b9c164860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:47.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.575+0000 7f8b9b59e700 1 --2- 172.21.15.121:0/301961506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c068c10 0x7f8b9c14e820 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:47.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.575+0000 7f8b9b59e700 1 --2- 172.21.15.121:0/301961506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b9c062c80 0x7f8b9c0aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:47.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.575+0000 7f8b9b59e700 1 -- 172.21.15.121:0/301961506 >> 172.21.15.121:0/301961506 conn(0x7f8b9c066010 msgr2=0x7f8b9c14f880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:47.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.575+0000 7f8b9b59e700 1 -- 172.21.15.121:0/301961506 shutdown_connections 2023-11-15T02:57:47.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.575+0000 7f8b9b59e700 1 -- 172.21.15.121:0/301961506 wait complete. 2023-11-15T02:57:47.604 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-15T02:57:47.604 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:57:47.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.767+0000 7f04f759e700 1 Processor -- start 2023-11-15T02:57:47.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.767+0000 7f04f759e700 1 -- start start 2023-11-15T02:57:47.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.771+0000 7f04f759e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f04f81508f0 0x7f04f80c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:47.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.771+0000 7f04f759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f8069350 0x7f04f80a4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:47.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.771+0000 7f04f759e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04f80c2ca0 con 0x7f04f80a4a10 2023-11-15T02:57:47.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.771+0000 7f04f759e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04f80c4460 con 0x7f04f81508f0 2023-11-15T02:57:47.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.771+0000 7f04f759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f80bdd70 con 0x7f04f8069350 2023-11-15T02:57:47.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.771+0000 7f04f659c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f04f81508f0 0x7f04f80c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:47.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.775+0000 7f04f659c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f04f81508f0 0x7f04f80c3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:48598/0 (socket says 172.21.15.121:48598) 2023-11-15T02:57:47.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.775+0000 7f04f659c700 1 -- 172.21.15.121:0/1548262787 learned_addr learned my addr 172.21.15.121:0/1548262787 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:47.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.775+0000 7f04f5d9b700 1 --2- 172.21.15.121:0/1548262787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f8069350 0x7f04f80a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:47.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.775+0000 7f04f659c700 1 -- 172.21.15.121:0/1548262787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f8069350 msgr2=0x7f04f80a4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:47.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.775+0000 7f04f659c700 1 --2- 172.21.15.121:0/1548262787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f8069350 0x7f04f80a4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:47.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.775+0000 7f04f659c700 1 -- 172.21.15.121:0/1548262787 >> v1:172.21.15.190:6789/0 conn(0x7f04f80a4a10 legacy=0x7f04f80a6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:47.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.775+0000 7f04f659c700 1 -- 172.21.15.121:0/1548262787 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f80c4ca0 con 0x7f04f81508f0 2023-11-15T02:57:47.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.775+0000 7f04f659c700 1 --2- 172.21.15.121:0/1548262787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f81508f0 0x7f04f80c3f70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f04e8009810 tx=0x7f04e8009060).ready entity=mon.1 client_cookie=7b7b89e1e946483c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:47.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.775+0000 7f04f559a700 1 -- 172.21.15.121:0/1548262787 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e800f510 con 0x7f04f81508f0 2023-11-15T02:57:47.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.775+0000 7f04f559a700 1 -- 172.21.15.121:0/1548262787 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e800f670 con 0x7f04f81508f0 2023-11-15T02:57:47.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.775+0000 7f04f559a700 1 -- 172.21.15.121:0/1548262787 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e800f9f0 con 0x7f04f81508f0 2023-11-15T02:57:47.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.775+0000 7f04f759e700 1 -- 172.21.15.121:0/1548262787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f81508f0 msgr2=0x7f04f80c3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:47.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.775+0000 7f04f759e700 1 --2- 172.21.15.121:0/1548262787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f81508f0 0x7f04f80c3f70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f04e8009810 tx=0x7f04e8009060).stop 2023-11-15T02:57:47.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.775+0000 7f04f759e700 1 -- 172.21.15.121:0/1548262787 shutdown_connections 2023-11-15T02:57:47.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.775+0000 7f04f759e700 1 --2- 172.21.15.121:0/1548262787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f8069350 0x7f04f80a4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:47.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.775+0000 7f04f759e700 1 --2- 172.21.15.121:0/1548262787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f81508f0 0x7f04f80c3f70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:47.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.775+0000 7f04f759e700 1 -- 172.21.15.121:0/1548262787 >> 172.21.15.121:0/1548262787 conn(0x7f04f8066e00 msgr2=0x7f04f8090df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:47.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.775+0000 7f04f759e700 1 -- 172.21.15.121:0/1548262787 shutdown_connections 2023-11-15T02:57:47.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.775+0000 7f04f759e700 1 -- 172.21.15.121:0/1548262787 wait complete. 2023-11-15T02:57:47.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.779+0000 7f04f759e700 1 Processor -- start 2023-11-15T02:57:47.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.779+0000 7f04f759e700 1 -- start start 2023-11-15T02:57:47.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.779+0000 7f04f759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f8069350 0x7f04f814e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:47.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.779+0000 7f04f759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f80a4a10 0x7f04f814c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:47.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.779+0000 7f04f759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f81508f0 0x7f04f814b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:47.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.779+0000 7f04f759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f814ebd0 con 0x7f04f8069350 2023-11-15T02:57:47.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.779+0000 7f04f759e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f8151280 con 0x7f04f80a4a10 2023-11-15T02:57:47.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.779+0000 7f04f759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f814ee30 con 0x7f04f81508f0 2023-11-15T02:57:47.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.783+0000 7f04f659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f8069350 0x7f04f814e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:47.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.783+0000 7f04f659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f8069350 0x7f04f814e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56388/0 (socket says 172.21.15.121:56388) 2023-11-15T02:57:47.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.783+0000 7f04f5d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f80a4a10 0x7f04f814c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:47.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.783+0000 7f04f659c700 1 -- 172.21.15.121:0/934672055 learned_addr learned my addr 172.21.15.121:0/934672055 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:47.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.783+0000 7f04f6d9d700 1 --2- 172.21.15.121:0/934672055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f81508f0 0x7f04f814b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:47.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.783+0000 7f04f5d9b700 1 -- 172.21.15.121:0/934672055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f81508f0 msgr2=0x7f04f814b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:47.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.783+0000 7f04f5d9b700 1 --2- 172.21.15.121:0/934672055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f81508f0 0x7f04f814b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:47.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.783+0000 7f04f5d9b700 1 -- 172.21.15.121:0/934672055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f8069350 msgr2=0x7f04f814e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:47.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.783+0000 7f04f5d9b700 1 --2- 172.21.15.121:0/934672055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f8069350 0x7f04f814e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:47.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.783+0000 7f04f5d9b700 1 -- 172.21.15.121:0/934672055 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e8009300 con 0x7f04f80a4a10 2023-11-15T02:57:47.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.783+0000 7f04f659c700 1 --2- 172.21.15.121:0/934672055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f8069350 0x7f04f814e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:57:47.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.783+0000 7f04f5d9b700 1 --2- 172.21.15.121:0/934672055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f80a4a10 0x7f04f814c4b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f04ec00e6f0 tx=0x7f04ec009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:47.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.783+0000 7f04e67fc700 1 -- 172.21.15.121:0/934672055 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ec012600 con 0x7f04f80a4a10 2023-11-15T02:57:47.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.783+0000 7f04e67fc700 1 -- 172.21.15.121:0/934672055 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04ec012760 con 0x7f04f80a4a10 2023-11-15T02:57:47.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.783+0000 7f04e67fc700 1 -- 172.21.15.121:0/934672055 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ec012ae0 con 0x7f04f80a4a10 2023-11-15T02:57:47.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.783+0000 7f04f759e700 1 -- 172.21.15.121:0/934672055 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f81528b0 con 0x7f04f80a4a10 2023-11-15T02:57:47.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.783+0000 7f04f759e700 1 -- 172.21.15.121:0/934672055 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f8152e70 con 0x7f04f80a4a10 2023-11-15T02:57:47.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.787+0000 7f04f759e700 1 -- 172.21.15.121:0/934672055 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f806bcf0 con 0x7f04f80a4a10 2023-11-15T02:57:47.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.787+0000 7f04e67fc700 1 -- 172.21.15.121:0/934672055 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f04ec01b400 con 0x7f04f80a4a10 2023-11-15T02:57:47.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.787+0000 7f04e67fc700 1 --2- 172.21.15.121:0/934672055 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f04d805a510 0x7f04d805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:47.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.787+0000 7f04e67fc700 1 -- 172.21.15.121:0/934672055 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f04ec0749e0 con 0x7f04f80a4a10 2023-11-15T02:57:47.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.787+0000 7f04f659c700 1 --2- 172.21.15.121:0/934672055 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f04d805a510 0x7f04d805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:47.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.791+0000 7f04f659c700 1 --2- 172.21.15.121:0/934672055 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f04d805a510 0x7f04d805c9d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f04e8009150 tx=0x7f04e8017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:47.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:47.799+0000 7f04e67fc700 1 -- 172.21.15.121:0/934672055 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f04ec048160 con 0x7f04f80a4a10 2023-11-15T02:57:48.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.003+0000 7f04f759e700 1 -- 172.21.15.121:0/934672055 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04f80c46e0 con 0x7f04f80a4a10 2023-11-15T02:57:48.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.007+0000 7f04e67fc700 1 -- 172.21.15.121:0/934672055 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4501 (secure 0 0 0) 0x7f04ec04b780 con 0x7f04f80a4a10 2023-11-15T02:57:48.011 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:48.011 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2544127447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2544127447},{"type":"v1","addr":"172.21.15.190:6833","nonce":2544127447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1815834193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1815834193},{"type":"v1","addr":"172.21.15.190:6835","nonce":1815834193}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4950,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3229917854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3229917854},{"type":"v1","addr":"172.21.15.121:6835","nonce":3229917854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:57:44.541243+0000","modified":"2023-11-15T02:57:47.567757+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.121:6837/3788932715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3788932715},{"type":"v1","addr":"172.21.15.121:6837","nonce":3788932715}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-15T02:57:48.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.007+0000 7f04f759e700 1 -- 172.21.15.121:0/934672055 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f04d805a510 msgr2=0x7f04d805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:48.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.007+0000 7f04f759e700 1 --2- 172.21.15.121:0/934672055 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f04d805a510 0x7f04d805c9d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f04e8009150 tx=0x7f04e8017000).stop 2023-11-15T02:57:48.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.007+0000 7f04f759e700 1 -- 172.21.15.121:0/934672055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f80a4a10 msgr2=0x7f04f814c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:48.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.007+0000 7f04f759e700 1 --2- 172.21.15.121:0/934672055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f80a4a10 0x7f04f814c4b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f04ec00e6f0 tx=0x7f04ec009060).stop 2023-11-15T02:57:48.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.007+0000 7f04f759e700 1 -- 172.21.15.121:0/934672055 shutdown_connections 2023-11-15T02:57:48.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.007+0000 7f04f759e700 1 --2- 172.21.15.121:0/934672055 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f04d805a510 0x7f04d805c9d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.007+0000 7f04f759e700 1 --2- 172.21.15.121:0/934672055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f81508f0 0x7f04f814b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.007+0000 7f04f759e700 1 --2- 172.21.15.121:0/934672055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f80a4a10 0x7f04f814c4b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.007+0000 7f04f759e700 1 --2- 172.21.15.121:0/934672055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f8069350 0x7f04f814e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.007+0000 7f04f759e700 1 -- 172.21.15.121:0/934672055 >> 172.21.15.121:0/934672055 conn(0x7f04f8066e00 msgr2=0x7f04f80ad3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:48.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.007+0000 7f04f759e700 1 -- 172.21.15.121:0/934672055 shutdown_connections 2023-11-15T02:57:48.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.007+0000 7f04f759e700 1 -- 172.21.15.121:0/934672055 wait complete. 2023-11-15T02:57:48.016 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 16 2023-11-15T02:57:48.037 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:57:48.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.199+0000 7f846129b700 1 Processor -- start 2023-11-15T02:57:48.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.199+0000 7f846129b700 1 -- start start 2023-11-15T02:57:48.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.203+0000 7f846129b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845c1539d0 0x7f845c0c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:48.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.203+0000 7f846129b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f845c153db0 0x7f845c0c2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:48.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.203+0000 7f846129b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845c0c1530 con 0x7f845c0c19c0 2023-11-15T02:57:48.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.203+0000 7f846129b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845c1541f0 con 0x7f845c153db0 2023-11-15T02:57:48.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.203+0000 7f846129b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845c0c2b60 con 0x7f845c1539d0 2023-11-15T02:57:48.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.203+0000 7f84527fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f845c153db0 0x7f845c0c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:48.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.203+0000 7f845affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845c1539d0 0x7f845c0c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:48.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.203+0000 7f84527fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f845c153db0 0x7f845c0c2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:48606/0 (socket says 172.21.15.121:48606) 2023-11-15T02:57:48.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.203+0000 7f84527fc700 1 -- 172.21.15.121:0/1189672345 learned_addr learned my addr 172.21.15.121:0/1189672345 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:48.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.203+0000 7f84527fc700 1 -- 172.21.15.121:0/1189672345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845c1539d0 msgr2=0x7f845c0c06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:48.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.203+0000 7f84527fc700 1 --2- 172.21.15.121:0/1189672345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845c1539d0 0x7f845c0c06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.203+0000 7f84527fc700 1 -- 172.21.15.121:0/1189672345 >> v1:172.21.15.190:6789/0 conn(0x7f845c0c19c0 legacy=0x7f845c0c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:48.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.203+0000 7f84527fc700 1 -- 172.21.15.121:0/1189672345 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845c0c2d80 con 0x7f845c153db0 2023-11-15T02:57:48.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.203+0000 7f84527fc700 1 --2- 172.21.15.121:0/1189672345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845c153db0 0x7f845c0c2670 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f8444006fd0 tx=0x7f84440090f0).ready entity=mon.1 client_cookie=7e4728236a1d8e89 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:48.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.203+0000 7f845a7fc700 1 -- 172.21.15.121:0/1189672345 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844400b580 con 0x7f845c153db0 2023-11-15T02:57:48.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.203+0000 7f845a7fc700 1 -- 172.21.15.121:0/1189672345 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844400b6e0 con 0x7f845c153db0 2023-11-15T02:57:48.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.203+0000 7f845a7fc700 1 -- 172.21.15.121:0/1189672345 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844400ba60 con 0x7f845c153db0 2023-11-15T02:57:48.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.207+0000 7f846129b700 1 -- 172.21.15.121:0/1189672345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845c153db0 msgr2=0x7f845c0c2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:48.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.207+0000 7f846129b700 1 --2- 172.21.15.121:0/1189672345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845c153db0 0x7f845c0c2670 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f8444006fd0 tx=0x7f84440090f0).stop 2023-11-15T02:57:48.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.207+0000 7f846129b700 1 -- 172.21.15.121:0/1189672345 shutdown_connections 2023-11-15T02:57:48.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.207+0000 7f846129b700 1 --2- 172.21.15.121:0/1189672345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845c153db0 0x7f845c0c2670 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.207+0000 7f846129b700 1 --2- 172.21.15.121:0/1189672345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845c1539d0 0x7f845c0c06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.207+0000 7f846129b700 1 -- 172.21.15.121:0/1189672345 >> 172.21.15.121:0/1189672345 conn(0x7f845c062190 msgr2=0x7f845c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:48.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.207+0000 7f846129b700 1 -- 172.21.15.121:0/1189672345 shutdown_connections 2023-11-15T02:57:48.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.207+0000 7f846129b700 1 -- 172.21.15.121:0/1189672345 wait complete. 2023-11-15T02:57:48.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.211+0000 7f846129b700 1 Processor -- start 2023-11-15T02:57:48.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.211+0000 7f846129b700 1 -- start start 2023-11-15T02:57:48.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.211+0000 7f846129b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845c0c19c0 0x7f845c14f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:48.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.211+0000 7f846129b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f845c1539d0 0x7f845c164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:48.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.211+0000 7f846129b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845c153db0 0x7f845c152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:48.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.211+0000 7f846129b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845c14f840 con 0x7f845c1539d0 2023-11-15T02:57:48.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.211+0000 7f846129b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845c1652c0 con 0x7f845c0c19c0 2023-11-15T02:57:48.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.211+0000 7f846129b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845c150380 con 0x7f845c153db0 2023-11-15T02:57:48.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.211+0000 7f84527fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f845c1539d0 0x7f845c164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:48.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.211+0000 7f84527fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f845c1539d0 0x7f845c164dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56398/0 (socket says 172.21.15.121:56398) 2023-11-15T02:57:48.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.211+0000 7f84527fc700 1 -- 172.21.15.121:0/701452454 learned_addr learned my addr 172.21.15.121:0/701452454 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:48.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.211+0000 7f845affd700 1 --2- 172.21.15.121:0/701452454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845c0c19c0 0x7f845c14f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:48.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.211+0000 7f845b7fe700 1 --2- 172.21.15.121:0/701452454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845c153db0 0x7f845c152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:48.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.211+0000 7f84527fc700 1 -- 172.21.15.121:0/701452454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845c153db0 msgr2=0x7f845c152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:48.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.211+0000 7f84527fc700 1 --2- 172.21.15.121:0/701452454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845c153db0 0x7f845c152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.211+0000 7f84527fc700 1 -- 172.21.15.121:0/701452454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845c0c19c0 msgr2=0x7f845c14f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:48.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.211+0000 7f84527fc700 1 --2- 172.21.15.121:0/701452454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845c0c19c0 0x7f845c14f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.215+0000 7f84527fc700 1 -- 172.21.15.121:0/701452454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8444009370 con 0x7f845c1539d0 2023-11-15T02:57:48.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.215+0000 7f84527fc700 1 --2- 172.21.15.121:0/701452454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f845c1539d0 0x7f845c164dd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f8444011000 tx=0x7f8444017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:48.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.215+0000 7f84537fe700 1 -- 172.21.15.121:0/701452454 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8444000bc0 con 0x7f845c1539d0 2023-11-15T02:57:48.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.215+0000 7f84537fe700 1 -- 172.21.15.121:0/701452454 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844400b850 con 0x7f845c1539d0 2023-11-15T02:57:48.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.215+0000 7f84537fe700 1 -- 172.21.15.121:0/701452454 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8444000d20 con 0x7f845c1539d0 2023-11-15T02:57:48.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.215+0000 7f846129b700 1 -- 172.21.15.121:0/701452454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f845c150660 con 0x7f845c1539d0 2023-11-15T02:57:48.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.215+0000 7f846129b700 1 -- 172.21.15.121:0/701452454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845c14fd30 con 0x7f845c1539d0 2023-11-15T02:57:48.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.215+0000 7f846129b700 1 -- 172.21.15.121:0/701452454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f845c1654b0 con 0x7f845c1539d0 2023-11-15T02:57:48.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.219+0000 7f84537fe700 1 -- 172.21.15.121:0/701452454 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f844401b8f0 con 0x7f845c1539d0 2023-11-15T02:57:48.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.219+0000 7f84537fe700 1 --2- 172.21.15.121:0/701452454 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f844005a510 0x7f844005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:48.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.219+0000 7f84537fe700 1 -- 172.21.15.121:0/701452454 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8444075eb0 con 0x7f845c1539d0 2023-11-15T02:57:48.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.219+0000 7f845affd700 1 --2- 172.21.15.121:0/701452454 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f844005a510 0x7f844005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:48.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.219+0000 7f845affd700 1 --2- 172.21.15.121:0/701452454 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f844005a510 0x7f844005c9d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f845c14c900 tx=0x7f844c009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:48.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.227+0000 7f84537fe700 1 -- 172.21.15.121:0/701452454 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f845c1654b0 con 0x7f845c1539d0 2023-11-15T02:57:48.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.387+0000 7f846129b700 1 -- 172.21.15.121:0/701452454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f845c06c150 con 0x7f845c1539d0 2023-11-15T02:57:48.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.387+0000 7f84537fe700 1 -- 172.21.15.121:0/701452454 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f8444049690 con 0x7f845c1539d0 2023-11-15T02:57:48.395 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:48.395 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":33,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:57:45.546467+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:57:42.277674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-15T02:57:43.540361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:57:48.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.391+0000 7f846129b700 1 -- 172.21.15.121:0/701452454 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f844005a510 msgr2=0x7f844005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:48.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.391+0000 7f846129b700 1 --2- 172.21.15.121:0/701452454 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f844005a510 0x7f844005c9d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f845c14c900 tx=0x7f844c009060).stop 2023-11-15T02:57:48.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.391+0000 7f846129b700 1 -- 172.21.15.121:0/701452454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f845c1539d0 msgr2=0x7f845c164dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:48.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.391+0000 7f846129b700 1 --2- 172.21.15.121:0/701452454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f845c1539d0 0x7f845c164dd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f8444011000 tx=0x7f8444017000).stop 2023-11-15T02:57:48.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.391+0000 7f846129b700 1 -- 172.21.15.121:0/701452454 shutdown_connections 2023-11-15T02:57:48.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.391+0000 7f846129b700 1 --2- 172.21.15.121:0/701452454 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f844005a510 0x7f844005c9d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.391+0000 7f846129b700 1 --2- 172.21.15.121:0/701452454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845c153db0 0x7f845c152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.391+0000 7f846129b700 1 --2- 172.21.15.121:0/701452454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f845c1539d0 0x7f845c164dd0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.391+0000 7f846129b700 1 --2- 172.21.15.121:0/701452454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845c0c19c0 0x7f845c14f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.391+0000 7f846129b700 1 -- 172.21.15.121:0/701452454 >> 172.21.15.121:0/701452454 conn(0x7f845c062190 msgr2=0x7f845c066890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:48.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.391+0000 7f846129b700 1 -- 172.21.15.121:0/701452454 shutdown_connections 2023-11-15T02:57:48.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.391+0000 7f846129b700 1 -- 172.21.15.121:0/701452454 wait complete. 2023-11-15T02:57:48.418 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-15T02:57:48.418 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T02:57:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.579+0000 7f898025b700 1 Processor -- start 2023-11-15T02:57:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.579+0000 7f898025b700 1 -- start start 2023-11-15T02:57:48.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.583+0000 7f898025b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8978153bb0 0x7f89780a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:48.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.583+0000 7f898025b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f89780aa020 0x7f8978067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:48.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.583+0000 7f898025b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8978068400 con 0x7f8978067f50 2023-11-15T02:57:48.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.583+0000 7f898025b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89780aa6f0 con 0x7f89780aa020 2023-11-15T02:57:48.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.583+0000 7f898025b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89780c5900 con 0x7f8978153bb0 2023-11-15T02:57:48.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.583+0000 7f897dff7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8978153bb0 0x7f89780a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:48.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.583+0000 7f897e7f8700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f8978067f50 0x7f89781578a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:44046/0 (socket says 172.21.15.121:44046) 2023-11-15T02:57:48.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.583+0000 7f897d7f6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f89780aa020 0x7f8978067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:48.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.583+0000 7f897e7f8700 1 -- 172.21.15.121:0/4268127257 learned_addr learned my addr 172.21.15.121:0/4268127257 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:48.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.583+0000 7f897cff5700 1 -- 172.21.15.121:0/4268127257 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 187224927 0 0) 0x7f8970001490 con 0x7f8978067f50 2023-11-15T02:57:48.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.583+0000 7f897cff5700 1 -- 172.21.15.121:0/4268127257 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89640033f0 con 0x7f8978067f50 2023-11-15T02:57:48.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.583+0000 7f897dff7700 1 -- 172.21.15.121:0/4268127257 >> v2:172.21.15.190:3300/0 conn(0x7f89780aa020 msgr2=0x7f8978067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:48.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.583+0000 7f897dff7700 1 --2- 172.21.15.121:0/4268127257 >> v2:172.21.15.190:3300/0 conn(0x7f89780aa020 0x7f8978067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.583+0000 7f897dff7700 1 -- 172.21.15.121:0/4268127257 >> v1:172.21.15.190:6789/0 conn(0x7f8978067f50 legacy=0x7f89781578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:48.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.583+0000 7f897dff7700 1 -- 172.21.15.121:0/4268127257 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89780cb5c0 con 0x7f8978153bb0 2023-11-15T02:57:48.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.583+0000 7f897dff7700 1 --2- 172.21.15.121:0/4268127257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8978153bb0 0x7f89780a9b30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f8974009760 tx=0x7f8974009060).ready entity=mon.2 client_cookie=31dd4de9a8d7b871 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:48.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.587+0000 7f897cff5700 1 -- 172.21.15.121:0/4268127257 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897400d4a0 con 0x7f8978153bb0 2023-11-15T02:57:48.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.587+0000 7f897cff5700 1 -- 172.21.15.121:0/4268127257 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897400d600 con 0x7f8978153bb0 2023-11-15T02:57:48.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.587+0000 7f897cff5700 1 -- 172.21.15.121:0/4268127257 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897400d980 con 0x7f8978153bb0 2023-11-15T02:57:48.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.587+0000 7f898025b700 1 -- 172.21.15.121:0/4268127257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8978153bb0 msgr2=0x7f89780a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:48.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.587+0000 7f898025b700 1 --2- 172.21.15.121:0/4268127257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8978153bb0 0x7f89780a9b30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f8974009760 tx=0x7f8974009060).stop 2023-11-15T02:57:48.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.587+0000 7f898025b700 1 -- 172.21.15.121:0/4268127257 shutdown_connections 2023-11-15T02:57:48.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.587+0000 7f898025b700 1 --2- 172.21.15.121:0/4268127257 >> v2:172.21.15.190:3300/0 conn(0x7f89780aa020 0x7f8978067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.587+0000 7f898025b700 1 --2- 172.21.15.121:0/4268127257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8978153bb0 0x7f89780a9b30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.587+0000 7f898025b700 1 -- 172.21.15.121:0/4268127257 >> 172.21.15.121:0/4268127257 conn(0x7f8978058c50 msgr2=0x7f8978059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:48.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.587+0000 7f898025b700 1 -- 172.21.15.121:0/4268127257 shutdown_connections 2023-11-15T02:57:48.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.587+0000 7f898025b700 1 -- 172.21.15.121:0/4268127257 wait complete. 2023-11-15T02:57:48.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.591+0000 7f898025b700 1 Processor -- start 2023-11-15T02:57:48.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.591+0000 7f898025b700 1 -- start start 2023-11-15T02:57:48.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.591+0000 7f898025b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8978067f50 0x7f8978164eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:48.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.591+0000 7f898025b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89780aa020 0x7f8978151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:48.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.591+0000 7f898025b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8978153bb0 0x7f8978151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:48.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.591+0000 7f898025b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8978152990 con 0x7f89780aa020 2023-11-15T02:57:48.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.591+0000 7f898025b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897814d230 con 0x7f8978153bb0 2023-11-15T02:57:48.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.591+0000 7f898025b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89780c2e90 con 0x7f8978067f50 2023-11-15T02:57:48.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.591+0000 7f897d7f6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89780aa020 0x7f8978151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:48.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.591+0000 7f897dff7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8978067f50 0x7f8978164eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:48.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.591+0000 7f897d7f6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89780aa020 0x7f8978151550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56410/0 (socket says 172.21.15.121:56410) 2023-11-15T02:57:48.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.591+0000 7f897d7f6700 1 -- 172.21.15.121:0/4044205297 learned_addr learned my addr 172.21.15.121:0/4044205297 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:48.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.591+0000 7f897e7f8700 1 --2- 172.21.15.121:0/4044205297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8978153bb0 0x7f8978151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:48.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.591+0000 7f897d7f6700 1 -- 172.21.15.121:0/4044205297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8978067f50 msgr2=0x7f8978164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:48.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.591+0000 7f897d7f6700 1 --2- 172.21.15.121:0/4044205297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8978067f50 0x7f8978164eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.591+0000 7f897d7f6700 1 -- 172.21.15.121:0/4044205297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8978153bb0 msgr2=0x7f8978151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:48.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.591+0000 7f897d7f6700 1 --2- 172.21.15.121:0/4044205297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8978153bb0 0x7f8978151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.591+0000 7f897d7f6700 1 -- 172.21.15.121:0/4044205297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89740092a0 con 0x7f89780aa020 2023-11-15T02:57:48.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.591+0000 7f897dff7700 1 --2- 172.21.15.121:0/4044205297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8978067f50 0x7f8978164eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:57:48.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.595+0000 7f897d7f6700 1 --2- 172.21.15.121:0/4044205297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89780aa020 0x7f8978151550 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f896800e900 tx=0x7f8968009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:48.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.595+0000 7f896dffb700 1 -- 172.21.15.121:0/4044205297 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8968012600 con 0x7f89780aa020 2023-11-15T02:57:48.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.595+0000 7f896dffb700 1 -- 172.21.15.121:0/4044205297 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8968012760 con 0x7f89780aa020 2023-11-15T02:57:48.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.595+0000 7f896dffb700 1 -- 172.21.15.121:0/4044205297 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8968012ae0 con 0x7f89780aa020 2023-11-15T02:57:48.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.595+0000 7f898025b700 1 -- 172.21.15.121:0/4044205297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89781520f0 con 0x7f89780aa020 2023-11-15T02:57:48.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.595+0000 7f898025b700 1 -- 172.21.15.121:0/4044205297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89781526b0 con 0x7f89780aa020 2023-11-15T02:57:48.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.595+0000 7f898025b700 1 -- 172.21.15.121:0/4044205297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89780c2c60 con 0x7f89780aa020 2023-11-15T02:57:48.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.599+0000 7f896dffb700 1 -- 172.21.15.121:0/4044205297 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f896801b400 con 0x7f89780aa020 2023-11-15T02:57:48.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.599+0000 7f896dffb700 1 --2- 172.21.15.121:0/4044205297 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f895c05a510 0x7f895c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:48.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.599+0000 7f896dffb700 1 -- 172.21.15.121:0/4044205297 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8968074b90 con 0x7f89780aa020 2023-11-15T02:57:48.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.599+0000 7f897dff7700 1 --2- 172.21.15.121:0/4044205297 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f895c05a510 0x7f895c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:48.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.599+0000 7f897dff7700 1 --2- 172.21.15.121:0/4044205297 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f895c05a510 0x7f895c05c9d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f897400d260 tx=0x7f8974016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:48.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.603+0000 7f896dffb700 1 -- 172.21.15.121:0/4044205297 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f89780c2c60 con 0x7f89780aa020 2023-11-15T02:57:48.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.767+0000 7f898025b700 1 -- 172.21.15.121:0/4044205297 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f89780c2c60 con 0x7f895c05a510 2023-11-15T02:57:48.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.783+0000 7f896dffb700 1 -- 172.21.15.121:0/4044205297 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+338132 (secure 0 0 0) 0x7f89780c2c60 con 0x7f895c05a510 2023-11-15T02:57:48.793 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:48.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.791+0000 7f898025b700 1 -- 172.21.15.121:0/4044205297 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f895c05a510 msgr2=0x7f895c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:48.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.791+0000 7f898025b700 1 --2- 172.21.15.121:0/4044205297 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f895c05a510 0x7f895c05c9d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f897400d260 tx=0x7f8974016000).stop 2023-11-15T02:57:48.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.791+0000 7f898025b700 1 -- 172.21.15.121:0/4044205297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89780aa020 msgr2=0x7f8978151550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:48.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.791+0000 7f898025b700 1 --2- 172.21.15.121:0/4044205297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89780aa020 0x7f8978151550 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f896800e900 tx=0x7f8968009060).stop 2023-11-15T02:57:48.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.791+0000 7f898025b700 1 -- 172.21.15.121:0/4044205297 shutdown_connections 2023-11-15T02:57:48.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.791+0000 7f898025b700 1 --2- 172.21.15.121:0/4044205297 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f895c05a510 0x7f895c05c9d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.791+0000 7f898025b700 1 --2- 172.21.15.121:0/4044205297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8978153bb0 0x7f8978151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.791+0000 7f898025b700 1 --2- 172.21.15.121:0/4044205297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89780aa020 0x7f8978151550 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.791+0000 7f898025b700 1 --2- 172.21.15.121:0/4044205297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8978067f50 0x7f8978164eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:48.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.791+0000 7f898025b700 1 -- 172.21.15.121:0/4044205297 >> 172.21.15.121:0/4044205297 conn(0x7f8978058c50 msgr2=0x7f89780b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:48.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.791+0000 7f898025b700 1 -- 172.21.15.121:0/4044205297 shutdown_connections 2023-11-15T02:57:48.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:48.791+0000 7f898025b700 1 -- 172.21.15.121:0/4044205297 wait complete. 2023-11-15T02:57:48.800 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T02:57:48.816 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2023-11-15T02:57:47.931491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1302,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":262,"acting":262,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2379448,"kb_used_data":2680,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752595272,"statfs":{"total":773094113280,"available":770657558528,"internally_reserved":0,"allocated":2744320,"data_stored":1102302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4297,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.436357"},"pg_stats":[{"pgid":"6.3c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.576426+0000","last_change":"2023-11-15T02:57:44.576426+0000","last_active":"2023-11-15T02:57:44.576426+0000","last_peered":"2023-11-15T02:57:44.576426+0000","last_clean":"2023-11-15T02:57:44.576426+0000","last_became_active":"2023-11-15T02:57:44.572509+0000","last_became_peered":"2023-11-15T02:57:44.572509+0000","last_unstale":"2023-11-15T02:57:44.576426+0000","last_undegraded":"2023-11-15T02:57:44.576426+0000","last_fullsized":"2023-11-15T02:57:44.576426+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3f","version":"32'3","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.587176+0000","last_change":"2023-11-15T02:57:44.105284+0000","last_active":"2023-11-15T02:57:44.587176+0000","last_peered":"2023-11-15T02:57:44.587176+0000","last_clean":"2023-11-15T02:57:44.587176+0000","last_became_active":"2023-11-15T02:57:44.104935+0000","last_became_peered":"2023-11-15T02:57:44.104935+0000","last_unstale":"2023-11-15T02:57:44.587176+0000","last_undegraded":"2023-11-15T02:57:44.587176+0000","last_fullsized":"2023-11-15T02:57:44.587176+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566808+0000","last_change":"2023-11-15T02:57:44.566808+0000","last_active":"2023-11-15T02:57:44.566808+0000","last_peered":"2023-11-15T02:57:44.566808+0000","last_clean":"2023-11-15T02:57:44.566808+0000","last_became_active":"2023-11-15T02:57:44.566228+0000","last_became_peered":"2023-11-15T02:57:44.566228+0000","last_unstale":"2023-11-15T02:57:44.566808+0000","last_undegraded":"2023-11-15T02:57:44.566808+0000","last_fullsized":"2023-11-15T02:57:44.566808+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109248+0000","last_change":"2023-11-15T02:57:44.101869+0000","last_active":"2023-11-15T02:57:44.109248+0000","last_peered":"2023-11-15T02:57:44.109248+0000","last_clean":"2023-11-15T02:57:44.109248+0000","last_became_active":"2023-11-15T02:57:44.101602+0000","last_became_peered":"2023-11-15T02:57:44.101602+0000","last_unstale":"2023-11-15T02:57:44.109248+0000","last_undegraded":"2023-11-15T02:57:44.109248+0000","last_fullsized":"2023-11-15T02:57:44.109248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.564204+0000","last_change":"2023-11-15T02:57:44.564204+0000","last_active":"2023-11-15T02:57:44.564204+0000","last_peered":"2023-11-15T02:57:44.564204+0000","last_clean":"2023-11-15T02:57:44.564204+0000","last_became_active":"2023-11-15T02:57:44.563976+0000","last_became_peered":"2023-11-15T02:57:44.563976+0000","last_unstale":"2023-11-15T02:57:44.564204+0000","last_undegraded":"2023-11-15T02:57:44.564204+0000","last_fullsized":"2023-11-15T02:57:44.564204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.096033+0000","last_change":"2023-11-15T02:57:44.096033+0000","last_active":"2023-11-15T02:57:44.096033+0000","last_peered":"2023-11-15T02:57:44.096033+0000","last_clean":"2023-11-15T02:57:44.096033+0000","last_became_active":"2023-11-15T02:57:44.095626+0000","last_became_peered":"2023-11-15T02:57:44.095626+0000","last_unstale":"2023-11-15T02:57:44.096033+0000","last_undegraded":"2023-11-15T02:57:44.096033+0000","last_fullsized":"2023-11-15T02:57:44.096033+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.568907+0000","last_change":"2023-11-15T02:57:44.568907+0000","last_active":"2023-11-15T02:57:44.568907+0000","last_peered":"2023-11-15T02:57:44.568907+0000","last_clean":"2023-11-15T02:57:44.568907+0000","last_became_active":"2023-11-15T02:57:44.568621+0000","last_became_peered":"2023-11-15T02:57:44.568621+0000","last_unstale":"2023-11-15T02:57:44.568907+0000","last_undegraded":"2023-11-15T02:57:44.568907+0000","last_fullsized":"2023-11-15T02:57:44.568907+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.065783+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.570582+0000","last_change":"2023-11-15T02:57:44.570582+0000","last_active":"2023-11-15T02:57:44.570582+0000","last_peered":"2023-11-15T02:57:44.570582+0000","last_clean":"2023-11-15T02:57:44.570582+0000","last_became_active":"2023-11-15T02:57:44.570314+0000","last_became_peered":"2023-11-15T02:57:44.570314+0000","last_unstale":"2023-11-15T02:57:44.570582+0000","last_undegraded":"2023-11-15T02:57:44.570582+0000","last_fullsized":"2023-11-15T02:57:44.570582+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3b","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560605+0000","last_change":"2023-11-15T02:57:44.104587+0000","last_active":"2023-11-15T02:57:44.560605+0000","last_peered":"2023-11-15T02:57:44.560605+0000","last_clean":"2023-11-15T02:57:44.560605+0000","last_became_active":"2023-11-15T02:57:44.104223+0000","last_became_peered":"2023-11-15T02:57:44.104223+0000","last_unstale":"2023-11-15T02:57:44.560605+0000","last_undegraded":"2023-11-15T02:57:44.560605+0000","last_fullsized":"2023-11-15T02:57:44.560605+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.077573+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.553452+0000","last_change":"2023-11-15T02:57:44.102587+0000","last_active":"2023-11-15T02:57:44.553452+0000","last_peered":"2023-11-15T02:57:44.553452+0000","last_clean":"2023-11-15T02:57:44.553452+0000","last_became_active":"2023-11-15T02:57:44.102311+0000","last_became_peered":"2023-11-15T02:57:44.102311+0000","last_unstale":"2023-11-15T02:57:44.553452+0000","last_undegraded":"2023-11-15T02:57:44.553452+0000","last_fullsized":"2023-11-15T02:57:44.553452+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.054985+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.110877+0000","last_change":"2023-11-15T02:57:44.105206+0000","last_active":"2023-11-15T02:57:44.110877+0000","last_peered":"2023-11-15T02:57:44.110877+0000","last_clean":"2023-11-15T02:57:44.110877+0000","last_became_active":"2023-11-15T02:57:44.104936+0000","last_became_peered":"2023-11-15T02:57:44.104936+0000","last_unstale":"2023-11-15T02:57:44.110877+0000","last_undegraded":"2023-11-15T02:57:44.110877+0000","last_fullsized":"2023-11-15T02:57:44.110877+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566763+0000","last_change":"2023-11-15T02:57:44.566763+0000","last_active":"2023-11-15T02:57:44.566763+0000","last_peered":"2023-11-15T02:57:44.566763+0000","last_clean":"2023-11-15T02:57:44.566763+0000","last_became_active":"2023-11-15T02:57:44.566510+0000","last_became_peered":"2023-11-15T02:57:44.566510+0000","last_unstale":"2023-11-15T02:57:44.566763+0000","last_undegraded":"2023-11-15T02:57:44.566763+0000","last_fullsized":"2023-11-15T02:57:44.566763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.110691+0000","last_change":"2023-11-15T02:57:44.106233+0000","last_active":"2023-11-15T02:57:44.110691+0000","last_peered":"2023-11-15T02:57:44.110691+0000","last_clean":"2023-11-15T02:57:44.110691+0000","last_became_active":"2023-11-15T02:57:44.105985+0000","last_became_peered":"2023-11-15T02:57:44.105985+0000","last_unstale":"2023-11-15T02:57:44.110691+0000","last_undegraded":"2023-11-15T02:57:44.110691+0000","last_fullsized":"2023-11-15T02:57:44.110691+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577960+0000","last_change":"2023-11-15T02:57:44.577960+0000","last_active":"2023-11-15T02:57:44.577960+0000","last_peered":"2023-11-15T02:57:44.577960+0000","last_clean":"2023-11-15T02:57:44.577960+0000","last_became_active":"2023-11-15T02:57:44.577405+0000","last_became_peered":"2023-11-15T02:57:44.577405+0000","last_unstale":"2023-11-15T02:57:44.577960+0000","last_undegraded":"2023-11-15T02:57:44.577960+0000","last_fullsized":"2023-11-15T02:57:44.577960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.099195+0000","last_change":"2023-11-15T02:57:44.091037+0000","last_active":"2023-11-15T02:57:44.099195+0000","last_peered":"2023-11-15T02:57:44.099195+0000","last_clean":"2023-11-15T02:57:44.099195+0000","last_became_active":"2023-11-15T02:57:44.090808+0000","last_became_peered":"2023-11-15T02:57:44.090808+0000","last_unstale":"2023-11-15T02:57:44.099195+0000","last_undegraded":"2023-11-15T02:57:44.099195+0000","last_fullsized":"2023-11-15T02:57:44.099195+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.077670+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.063700+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.075254+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109579+0000","last_change":"2023-11-15T02:57:44.102619+0000","last_active":"2023-11-15T02:57:44.109579+0000","last_peered":"2023-11-15T02:57:44.109579+0000","last_clean":"2023-11-15T02:57:44.109579+0000","last_became_active":"2023-11-15T02:57:44.102328+0000","last_became_peered":"2023-11-15T02:57:44.102328+0000","last_unstale":"2023-11-15T02:57:44.109579+0000","last_undegraded":"2023-11-15T02:57:44.109579+0000","last_fullsized":"2023-11-15T02:57:44.109579+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.579068+0000","last_change":"2023-11-15T02:57:44.579068+0000","last_active":"2023-11-15T02:57:44.579068+0000","last_peered":"2023-11-15T02:57:44.579068+0000","last_clean":"2023-11-15T02:57:44.579068+0000","last_became_active":"2023-11-15T02:57:44.578815+0000","last_became_peered":"2023-11-15T02:57:44.578815+0000","last_unstale":"2023-11-15T02:57:44.579068+0000","last_undegraded":"2023-11-15T02:57:44.579068+0000","last_fullsized":"2023-11-15T02:57:44.579068+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.113636+0000","last_change":"2023-11-15T02:57:44.113636+0000","last_active":"2023-11-15T02:57:44.113636+0000","last_peered":"2023-11-15T02:57:44.113636+0000","last_clean":"2023-11-15T02:57:44.113636+0000","last_became_active":"2023-11-15T02:57:44.113283+0000","last_became_peered":"2023-11-15T02:57:44.113283+0000","last_unstale":"2023-11-15T02:57:44.113636+0000","last_undegraded":"2023-11-15T02:57:44.113636+0000","last_fullsized":"2023-11-15T02:57:44.113636+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.584681+0000","last_change":"2023-11-15T02:57:44.584681+0000","last_active":"2023-11-15T02:57:44.584681+0000","last_peered":"2023-11-15T02:57:44.584681+0000","last_clean":"2023-11-15T02:57:44.584681+0000","last_became_active":"2023-11-15T02:57:44.584395+0000","last_became_peered":"2023-11-15T02:57:44.584395+0000","last_unstale":"2023-11-15T02:57:44.584681+0000","last_undegraded":"2023-11-15T02:57:44.584681+0000","last_fullsized":"2023-11-15T02:57:44.584681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.33","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560690+0000","last_change":"2023-11-15T02:57:44.086072+0000","last_active":"2023-11-15T02:57:44.560690+0000","last_peered":"2023-11-15T02:57:44.560690+0000","last_clean":"2023-11-15T02:57:44.560690+0000","last_became_active":"2023-11-15T02:57:44.085574+0000","last_became_peered":"2023-11-15T02:57:44.085574+0000","last_unstale":"2023-11-15T02:57:44.560690+0000","last_undegraded":"2023-11-15T02:57:44.560690+0000","last_fullsized":"2023-11-15T02:57:44.560690+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.076869+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.107875+0000","last_change":"2023-11-15T02:57:44.094640+0000","last_active":"2023-11-15T02:57:44.107875+0000","last_peered":"2023-11-15T02:57:44.107875+0000","last_clean":"2023-11-15T02:57:44.107875+0000","last_became_active":"2023-11-15T02:57:44.094279+0000","last_became_peered":"2023-11-15T02:57:44.094279+0000","last_unstale":"2023-11-15T02:57:44.107875+0000","last_undegraded":"2023-11-15T02:57:44.107875+0000","last_fullsized":"2023-11-15T02:57:44.107875+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.576818+0000","last_change":"2023-11-15T02:57:44.576818+0000","last_active":"2023-11-15T02:57:44.576818+0000","last_peered":"2023-11-15T02:57:44.576818+0000","last_clean":"2023-11-15T02:57:44.576818+0000","last_became_active":"2023-11-15T02:57:44.576074+0000","last_became_peered":"2023-11-15T02:57:44.576074+0000","last_unstale":"2023-11-15T02:57:44.576818+0000","last_undegraded":"2023-11-15T02:57:44.576818+0000","last_fullsized":"2023-11-15T02:57:44.576818+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109239+0000","last_change":"2023-11-15T02:57:44.109239+0000","last_active":"2023-11-15T02:57:44.109239+0000","last_peered":"2023-11-15T02:57:44.109239+0000","last_clean":"2023-11-15T02:57:44.109239+0000","last_became_active":"2023-11-15T02:57:44.103686+0000","last_became_peered":"2023-11-15T02:57:44.103686+0000","last_unstale":"2023-11-15T02:57:44.109239+0000","last_undegraded":"2023-11-15T02:57:44.109239+0000","last_fullsized":"2023-11-15T02:57:44.109239+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.074718+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"32'4","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585116+0000","last_change":"2023-11-15T02:57:44.113817+0000","last_active":"2023-11-15T02:57:44.585116+0000","last_peered":"2023-11-15T02:57:44.585116+0000","last_clean":"2023-11-15T02:57:44.585116+0000","last_became_active":"2023-11-15T02:57:44.112534+0000","last_became_peered":"2023-11-15T02:57:44.112534+0000","last_unstale":"2023-11-15T02:57:44.585116+0000","last_undegraded":"2023-11-15T02:57:44.585116+0000","last_fullsized":"2023-11-15T02:57:44.585116+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.575459+0000","last_change":"2023-11-15T02:57:44.575459+0000","last_active":"2023-11-15T02:57:44.575459+0000","last_peered":"2023-11-15T02:57:44.575459+0000","last_clean":"2023-11-15T02:57:44.575459+0000","last_became_active":"2023-11-15T02:57:44.575123+0000","last_became_peered":"2023-11-15T02:57:44.575123+0000","last_unstale":"2023-11-15T02:57:44.575459+0000","last_undegraded":"2023-11-15T02:57:44.575459+0000","last_fullsized":"2023-11-15T02:57:44.575459+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.111343+0000","last_change":"2023-11-15T02:57:44.107752+0000","last_active":"2023-11-15T02:57:44.111343+0000","last_peered":"2023-11-15T02:57:44.111343+0000","last_clean":"2023-11-15T02:57:44.111343+0000","last_became_active":"2023-11-15T02:57:44.107413+0000","last_became_peered":"2023-11-15T02:57:44.107413+0000","last_unstale":"2023-11-15T02:57:44.111343+0000","last_undegraded":"2023-11-15T02:57:44.111343+0000","last_fullsized":"2023-11-15T02:57:44.111343+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.572196+0000","last_change":"2023-11-15T02:57:44.572196+0000","last_active":"2023-11-15T02:57:44.572196+0000","last_peered":"2023-11-15T02:57:44.572196+0000","last_clean":"2023-11-15T02:57:44.572196+0000","last_became_active":"2023-11-15T02:57:44.571963+0000","last_became_peered":"2023-11-15T02:57:44.571963+0000","last_unstale":"2023-11-15T02:57:44.572196+0000","last_undegraded":"2023-11-15T02:57:44.572196+0000","last_fullsized":"2023-11-15T02:57:44.572196+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.106664+0000","last_change":"2023-11-15T02:57:44.097768+0000","last_active":"2023-11-15T02:57:44.106664+0000","last_peered":"2023-11-15T02:57:44.106664+0000","last_clean":"2023-11-15T02:57:44.106664+0000","last_became_active":"2023-11-15T02:57:44.097449+0000","last_became_peered":"2023-11-15T02:57:44.097449+0000","last_unstale":"2023-11-15T02:57:44.106664+0000","last_undegraded":"2023-11-15T02:57:44.106664+0000","last_fullsized":"2023-11-15T02:57:44.106664+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.588091+0000","last_change":"2023-11-15T02:57:44.588091+0000","last_active":"2023-11-15T02:57:44.588091+0000","last_peered":"2023-11-15T02:57:44.588091+0000","last_clean":"2023-11-15T02:57:44.588091+0000","last_became_active":"2023-11-15T02:57:44.587721+0000","last_became_peered":"2023-11-15T02:57:44.587721+0000","last_unstale":"2023-11-15T02:57:44.588091+0000","last_undegraded":"2023-11-15T02:57:44.588091+0000","last_fullsized":"2023-11-15T02:57:44.588091+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2d","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.551810+0000","last_change":"2023-11-15T02:57:44.097948+0000","last_active":"2023-11-15T02:57:44.551810+0000","last_peered":"2023-11-15T02:57:44.551810+0000","last_clean":"2023-11-15T02:57:44.551810+0000","last_became_active":"2023-11-15T02:57:44.097642+0000","last_became_peered":"2023-11-15T02:57:44.097642+0000","last_unstale":"2023-11-15T02:57:44.551810+0000","last_undegraded":"2023-11-15T02:57:44.551810+0000","last_fullsized":"2023-11-15T02:57:44.551810+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582841+0000","last_change":"2023-11-15T02:57:44.582841+0000","last_active":"2023-11-15T02:57:44.582841+0000","last_peered":"2023-11-15T02:57:44.582841+0000","last_clean":"2023-11-15T02:57:44.582841+0000","last_became_active":"2023-11-15T02:57:44.582592+0000","last_became_peered":"2023-11-15T02:57:44.582592+0000","last_unstale":"2023-11-15T02:57:44.582841+0000","last_undegraded":"2023-11-15T02:57:44.582841+0000","last_fullsized":"2023-11-15T02:57:44.582841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.101447+0000","last_change":"2023-11-15T02:57:44.095430+0000","last_active":"2023-11-15T02:57:44.101447+0000","last_peered":"2023-11-15T02:57:44.101447+0000","last_clean":"2023-11-15T02:57:44.101447+0000","last_became_active":"2023-11-15T02:57:44.094060+0000","last_became_peered":"2023-11-15T02:57:44.094060+0000","last_unstale":"2023-11-15T02:57:44.101447+0000","last_undegraded":"2023-11-15T02:57:44.101447+0000","last_fullsized":"2023-11-15T02:57:44.101447+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.571866+0000","last_change":"2023-11-15T02:57:44.571866+0000","last_active":"2023-11-15T02:57:44.571866+0000","last_peered":"2023-11-15T02:57:44.571866+0000","last_clean":"2023-11-15T02:57:44.571866+0000","last_became_active":"2023-11-15T02:57:44.571594+0000","last_became_peered":"2023-11-15T02:57:44.571594+0000","last_unstale":"2023-11-15T02:57:44.571866+0000","last_undegraded":"2023-11-15T02:57:44.571866+0000","last_fullsized":"2023-11-15T02:57:44.571866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.561370+0000","last_change":"2023-11-15T02:57:44.111471+0000","last_active":"2023-11-15T02:57:44.561370+0000","last_peered":"2023-11-15T02:57:44.561370+0000","last_clean":"2023-11-15T02:57:44.561370+0000","last_became_active":"2023-11-15T02:57:44.110158+0000","last_became_peered":"2023-11-15T02:57:44.110158+0000","last_unstale":"2023-11-15T02:57:44.561370+0000","last_undegraded":"2023-11-15T02:57:44.561370+0000","last_fullsized":"2023-11-15T02:57:44.561370+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.589275+0000","last_change":"2023-11-15T02:57:44.589275+0000","last_active":"2023-11-15T02:57:44.589275+0000","last_peered":"2023-11-15T02:57:44.589275+0000","last_clean":"2023-11-15T02:57:44.589275+0000","last_became_active":"2023-11-15T02:57:44.588964+0000","last_became_peered":"2023-11-15T02:57:44.588964+0000","last_unstale":"2023-11-15T02:57:44.589275+0000","last_undegraded":"2023-11-15T02:57:44.589275+0000","last_fullsized":"2023-11-15T02:57:44.589275+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.103617+0000","last_change":"2023-11-15T02:57:44.098842+0000","last_active":"2023-11-15T02:57:44.103617+0000","last_peered":"2023-11-15T02:57:44.103617+0000","last_clean":"2023-11-15T02:57:44.103617+0000","last_became_active":"2023-11-15T02:57:44.098539+0000","last_became_peered":"2023-11-15T02:57:44.098539+0000","last_unstale":"2023-11-15T02:57:44.103617+0000","last_undegraded":"2023-11-15T02:57:44.103617+0000","last_fullsized":"2023-11-15T02:57:44.103617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.571505+0000","last_change":"2023-11-15T02:57:44.571505+0000","last_active":"2023-11-15T02:57:44.571505+0000","last_peered":"2023-11-15T02:57:44.571505+0000","last_clean":"2023-11-15T02:57:44.571505+0000","last_became_active":"2023-11-15T02:57:44.570032+0000","last_became_peered":"2023-11-15T02:57:44.570032+0000","last_unstale":"2023-11-15T02:57:44.571505+0000","last_undegraded":"2023-11-15T02:57:44.571505+0000","last_fullsized":"2023-11-15T02:57:44.571505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.112480+0000","last_change":"2023-11-15T02:57:44.102777+0000","last_active":"2023-11-15T02:57:44.112480+0000","last_peered":"2023-11-15T02:57:44.112480+0000","last_clean":"2023-11-15T02:57:44.112480+0000","last_became_active":"2023-11-15T02:57:44.102485+0000","last_became_peered":"2023-11-15T02:57:44.102485+0000","last_unstale":"2023-11-15T02:57:44.112480+0000","last_undegraded":"2023-11-15T02:57:44.112480+0000","last_fullsized":"2023-11-15T02:57:44.112480+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.076093+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.100059+0000","last_change":"2023-11-15T02:57:44.100059+0000","last_active":"2023-11-15T02:57:44.100059+0000","last_peered":"2023-11-15T02:57:44.100059+0000","last_clean":"2023-11-15T02:57:44.100059+0000","last_became_active":"2023-11-15T02:57:44.099755+0000","last_became_peered":"2023-11-15T02:57:44.099755+0000","last_unstale":"2023-11-15T02:57:44.100059+0000","last_undegraded":"2023-11-15T02:57:44.100059+0000","last_fullsized":"2023-11-15T02:57:44.100059+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.076365+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.063419+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577168+0000","last_change":"2023-11-15T02:57:44.577168+0000","last_active":"2023-11-15T02:57:44.577168+0000","last_peered":"2023-11-15T02:57:44.577168+0000","last_clean":"2023-11-15T02:57:44.577168+0000","last_became_active":"2023-11-15T02:57:44.576902+0000","last_became_peered":"2023-11-15T02:57:44.576902+0000","last_unstale":"2023-11-15T02:57:44.577168+0000","last_undegraded":"2023-11-15T02:57:44.577168+0000","last_fullsized":"2023-11-15T02:57:44.577168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.106529+0000","last_change":"2023-11-15T02:57:44.097229+0000","last_active":"2023-11-15T02:57:44.106529+0000","last_peered":"2023-11-15T02:57:44.106529+0000","last_clean":"2023-11-15T02:57:44.106529+0000","last_became_active":"2023-11-15T02:57:44.096765+0000","last_became_peered":"2023-11-15T02:57:44.096765+0000","last_unstale":"2023-11-15T02:57:44.106529+0000","last_undegraded":"2023-11-15T02:57:44.106529+0000","last_fullsized":"2023-11-15T02:57:44.106529+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.054985+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.063627+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.576398+0000","last_change":"2023-11-15T02:57:44.576398+0000","last_active":"2023-11-15T02:57:44.576398+0000","last_peered":"2023-11-15T02:57:44.576398+0000","last_clean":"2023-11-15T02:57:44.576398+0000","last_became_active":"2023-11-15T02:57:44.576039+0000","last_became_peered":"2023-11-15T02:57:44.576039+0000","last_unstale":"2023-11-15T02:57:44.576398+0000","last_undegraded":"2023-11-15T02:57:44.576398+0000","last_fullsized":"2023-11-15T02:57:44.576398+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.098514+0000","last_change":"2023-11-15T02:57:44.098514+0000","last_active":"2023-11-15T02:57:44.098514+0000","last_peered":"2023-11-15T02:57:44.098514+0000","last_clean":"2023-11-15T02:57:44.098514+0000","last_became_active":"2023-11-15T02:57:44.098120+0000","last_became_peered":"2023-11-15T02:57:44.098120+0000","last_unstale":"2023-11-15T02:57:44.098514+0000","last_undegraded":"2023-11-15T02:57:44.098514+0000","last_fullsized":"2023-11-15T02:57:44.098514+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.054985+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.108729+0000","last_change":"2023-11-15T02:57:44.101472+0000","last_active":"2023-11-15T02:57:44.108729+0000","last_peered":"2023-11-15T02:57:44.108729+0000","last_clean":"2023-11-15T02:57:44.108729+0000","last_became_active":"2023-11-15T02:57:44.101190+0000","last_became_peered":"2023-11-15T02:57:44.101190+0000","last_unstale":"2023-11-15T02:57:44.108729+0000","last_undegraded":"2023-11-15T02:57:44.108729+0000","last_fullsized":"2023-11-15T02:57:44.108729+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.589081+0000","last_change":"2023-11-15T02:57:44.589081+0000","last_active":"2023-11-15T02:57:44.589081+0000","last_peered":"2023-11-15T02:57:44.589081+0000","last_clean":"2023-11-15T02:57:44.589081+0000","last_became_active":"2023-11-15T02:57:44.588602+0000","last_became_peered":"2023-11-15T02:57:44.588602+0000","last_unstale":"2023-11-15T02:57:44.589081+0000","last_undegraded":"2023-11-15T02:57:44.589081+0000","last_fullsized":"2023-11-15T02:57:44.589081+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.108688+0000","last_change":"2023-11-15T02:57:44.101411+0000","last_active":"2023-11-15T02:57:44.108688+0000","last_peered":"2023-11-15T02:57:44.108688+0000","last_clean":"2023-11-15T02:57:44.108688+0000","last_became_active":"2023-11-15T02:57:44.099981+0000","last_became_peered":"2023-11-15T02:57:44.099981+0000","last_unstale":"2023-11-15T02:57:44.108688+0000","last_undegraded":"2023-11-15T02:57:44.108688+0000","last_fullsized":"2023-11-15T02:57:44.108688+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.588317+0000","last_change":"2023-11-15T02:57:44.588317+0000","last_active":"2023-11-15T02:57:44.588317+0000","last_peered":"2023-11-15T02:57:44.588317+0000","last_clean":"2023-11-15T02:57:44.588317+0000","last_became_active":"2023-11-15T02:57:44.586643+0000","last_became_peered":"2023-11-15T02:57:44.586643+0000","last_unstale":"2023-11-15T02:57:44.588317+0000","last_undegraded":"2023-11-15T02:57:44.588317+0000","last_fullsized":"2023-11-15T02:57:44.588317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.104386+0000","last_change":"2023-11-15T02:57:44.097212+0000","last_active":"2023-11-15T02:57:44.104386+0000","last_peered":"2023-11-15T02:57:44.104386+0000","last_clean":"2023-11-15T02:57:44.104386+0000","last_became_active":"2023-11-15T02:57:44.096903+0000","last_became_peered":"2023-11-15T02:57:44.096903+0000","last_unstale":"2023-11-15T02:57:44.104386+0000","last_undegraded":"2023-11-15T02:57:44.104386+0000","last_fullsized":"2023-11-15T02:57:44.104386+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.578257+0000","last_change":"2023-11-15T02:57:44.578257+0000","last_active":"2023-11-15T02:57:44.578257+0000","last_peered":"2023-11-15T02:57:44.578257+0000","last_clean":"2023-11-15T02:57:44.578257+0000","last_became_active":"2023-11-15T02:57:44.571560+0000","last_became_peered":"2023-11-15T02:57:44.571560+0000","last_unstale":"2023-11-15T02:57:44.578257+0000","last_undegraded":"2023-11-15T02:57:44.578257+0000","last_fullsized":"2023-11-15T02:57:44.578257+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.108660+0000","last_change":"2023-11-15T02:57:44.102174+0000","last_active":"2023-11-15T02:57:44.108660+0000","last_peered":"2023-11-15T02:57:44.108660+0000","last_clean":"2023-11-15T02:57:44.108660+0000","last_became_active":"2023-11-15T02:57:44.101794+0000","last_became_peered":"2023-11-15T02:57:44.101794+0000","last_unstale":"2023-11-15T02:57:44.108660+0000","last_undegraded":"2023-11-15T02:57:44.108660+0000","last_fullsized":"2023-11-15T02:57:44.108660+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.587132+0000","last_change":"2023-11-15T02:57:44.587132+0000","last_active":"2023-11-15T02:57:44.587132+0000","last_peered":"2023-11-15T02:57:44.587132+0000","last_clean":"2023-11-15T02:57:44.587132+0000","last_became_active":"2023-11-15T02:57:44.586820+0000","last_became_peered":"2023-11-15T02:57:44.586820+0000","last_unstale":"2023-11-15T02:57:44.587132+0000","last_undegraded":"2023-11-15T02:57:44.587132+0000","last_fullsized":"2023-11-15T02:57:44.587132+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.574296+0000","last_change":"2023-11-15T02:57:44.102584+0000","last_active":"2023-11-15T02:57:44.574296+0000","last_peered":"2023-11-15T02:57:44.574296+0000","last_clean":"2023-11-15T02:57:44.574296+0000","last_became_active":"2023-11-15T02:57:44.102267+0000","last_became_peered":"2023-11-15T02:57:44.102267+0000","last_unstale":"2023-11-15T02:57:44.574296+0000","last_undegraded":"2023-11-15T02:57:44.574296+0000","last_fullsized":"2023-11-15T02:57:44.574296+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.054985+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.561164+0000","last_change":"2023-11-15T02:57:44.102680+0000","last_active":"2023-11-15T02:57:44.561164+0000","last_peered":"2023-11-15T02:57:44.561164+0000","last_clean":"2023-11-15T02:57:44.561164+0000","last_became_active":"2023-11-15T02:57:44.102394+0000","last_became_peered":"2023-11-15T02:57:44.102394+0000","last_unstale":"2023-11-15T02:57:44.561164+0000","last_undegraded":"2023-11-15T02:57:44.561164+0000","last_fullsized":"2023-11-15T02:57:44.561164+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.586592+0000","last_change":"2023-11-15T02:57:44.586592+0000","last_active":"2023-11-15T02:57:44.586592+0000","last_peered":"2023-11-15T02:57:44.586592+0000","last_clean":"2023-11-15T02:57:44.586592+0000","last_became_active":"2023-11-15T02:57:44.586211+0000","last_became_peered":"2023-11-15T02:57:44.586211+0000","last_unstale":"2023-11-15T02:57:44.586592+0000","last_undegraded":"2023-11-15T02:57:44.586592+0000","last_fullsized":"2023-11-15T02:57:44.586592+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.063052+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.579124+0000","last_change":"2023-11-15T02:57:44.579124+0000","last_active":"2023-11-15T02:57:44.579124+0000","last_peered":"2023-11-15T02:57:44.579124+0000","last_clean":"2023-11-15T02:57:44.579124+0000","last_became_active":"2023-11-15T02:57:44.578824+0000","last_became_peered":"2023-11-15T02:57:44.578824+0000","last_unstale":"2023-11-15T02:57:44.579124+0000","last_undegraded":"2023-11-15T02:57:44.579124+0000","last_fullsized":"2023-11-15T02:57:44.579124+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.108976+0000","last_change":"2023-11-15T02:57:44.101425+0000","last_active":"2023-11-15T02:57:44.108976+0000","last_peered":"2023-11-15T02:57:44.108976+0000","last_clean":"2023-11-15T02:57:44.108976+0000","last_became_active":"2023-11-15T02:57:44.101139+0000","last_became_peered":"2023-11-15T02:57:44.101139+0000","last_unstale":"2023-11-15T02:57:44.108976+0000","last_undegraded":"2023-11-15T02:57:44.108976+0000","last_fullsized":"2023-11-15T02:57:44.108976+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.574408+0000","last_change":"2023-11-15T02:57:44.574408+0000","last_active":"2023-11-15T02:57:44.574408+0000","last_peered":"2023-11-15T02:57:44.574408+0000","last_clean":"2023-11-15T02:57:44.574408+0000","last_became_active":"2023-11-15T02:57:44.573651+0000","last_became_peered":"2023-11-15T02:57:44.573651+0000","last_unstale":"2023-11-15T02:57:44.574408+0000","last_undegraded":"2023-11-15T02:57:44.574408+0000","last_fullsized":"2023-11-15T02:57:44.574408+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.112809+0000","last_change":"2023-11-15T02:57:44.112809+0000","last_active":"2023-11-15T02:57:44.112809+0000","last_peered":"2023-11-15T02:57:44.112809+0000","last_clean":"2023-11-15T02:57:44.112809+0000","last_became_active":"2023-11-15T02:57:44.111134+0000","last_became_peered":"2023-11-15T02:57:44.111134+0000","last_unstale":"2023-11-15T02:57:44.112809+0000","last_undegraded":"2023-11-15T02:57:44.112809+0000","last_fullsized":"2023-11-15T02:57:44.112809+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.076060+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.104280+0000","last_change":"2023-11-15T02:57:44.099996+0000","last_active":"2023-11-15T02:57:44.104280+0000","last_peered":"2023-11-15T02:57:44.104280+0000","last_clean":"2023-11-15T02:57:44.104280+0000","last_became_active":"2023-11-15T02:57:44.099695+0000","last_became_peered":"2023-11-15T02:57:44.099695+0000","last_unstale":"2023-11-15T02:57:44.104280+0000","last_undegraded":"2023-11-15T02:57:44.104280+0000","last_fullsized":"2023-11-15T02:57:44.104280+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577040+0000","last_change":"2023-11-15T02:57:44.577040+0000","last_active":"2023-11-15T02:57:44.577040+0000","last_peered":"2023-11-15T02:57:44.577040+0000","last_clean":"2023-11-15T02:57:44.577040+0000","last_became_active":"2023-11-15T02:57:44.576592+0000","last_became_peered":"2023-11-15T02:57:44.576592+0000","last_unstale":"2023-11-15T02:57:44.577040+0000","last_undegraded":"2023-11-15T02:57:44.577040+0000","last_fullsized":"2023-11-15T02:57:44.577040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.064186+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.571061+0000","last_change":"2023-11-15T02:57:44.571061+0000","last_active":"2023-11-15T02:57:44.571061+0000","last_peered":"2023-11-15T02:57:44.571061+0000","last_clean":"2023-11-15T02:57:44.571061+0000","last_became_active":"2023-11-15T02:57:44.570802+0000","last_became_peered":"2023-11-15T02:57:44.570802+0000","last_unstale":"2023-11-15T02:57:44.571061+0000","last_undegraded":"2023-11-15T02:57:44.571061+0000","last_fullsized":"2023-11-15T02:57:44.571061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.103937+0000","last_change":"2023-11-15T02:57:44.099371+0000","last_active":"2023-11-15T02:57:44.103937+0000","last_peered":"2023-11-15T02:57:44.103937+0000","last_clean":"2023-11-15T02:57:44.103937+0000","last_became_active":"2023-11-15T02:57:44.096689+0000","last_became_peered":"2023-11-15T02:57:44.096689+0000","last_unstale":"2023-11-15T02:57:44.103937+0000","last_undegraded":"2023-11-15T02:57:44.103937+0000","last_fullsized":"2023-11-15T02:57:44.103937+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.572799+0000","last_change":"2023-11-15T02:57:44.572799+0000","last_active":"2023-11-15T02:57:44.572799+0000","last_peered":"2023-11-15T02:57:44.572799+0000","last_clean":"2023-11-15T02:57:44.572799+0000","last_became_active":"2023-11-15T02:57:44.572147+0000","last_became_peered":"2023-11-15T02:57:44.572147+0000","last_unstale":"2023-11-15T02:57:44.572799+0000","last_undegraded":"2023-11-15T02:57:44.572799+0000","last_fullsized":"2023-11-15T02:57:44.572799+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.105678+0000","last_change":"2023-11-15T02:57:44.094718+0000","last_active":"2023-11-15T02:57:44.105678+0000","last_peered":"2023-11-15T02:57:44.105678+0000","last_clean":"2023-11-15T02:57:44.105678+0000","last_became_active":"2023-11-15T02:57:44.094236+0000","last_became_peered":"2023-11-15T02:57:44.094236+0000","last_unstale":"2023-11-15T02:57:44.105678+0000","last_undegraded":"2023-11-15T02:57:44.105678+0000","last_fullsized":"2023-11-15T02:57:44.105678+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577901+0000","last_change":"2023-11-15T02:57:44.577901+0000","last_active":"2023-11-15T02:57:44.577901+0000","last_peered":"2023-11-15T02:57:44.577901+0000","last_clean":"2023-11-15T02:57:44.577901+0000","last_became_active":"2023-11-15T02:57:44.577370+0000","last_became_peered":"2023-11-15T02:57:44.577370+0000","last_unstale":"2023-11-15T02:57:44.577901+0000","last_undegraded":"2023-11-15T02:57:44.577901+0000","last_fullsized":"2023-11-15T02:57:44.577901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109398+0000","last_change":"2023-11-15T02:57:44.109398+0000","last_active":"2023-11-15T02:57:44.109398+0000","last_peered":"2023-11-15T02:57:44.109398+0000","last_clean":"2023-11-15T02:57:44.109398+0000","last_became_active":"2023-11-15T02:57:44.104880+0000","last_became_peered":"2023-11-15T02:57:44.104880+0000","last_unstale":"2023-11-15T02:57:44.109398+0000","last_undegraded":"2023-11-15T02:57:44.109398+0000","last_fullsized":"2023-11-15T02:57:44.109398+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.570258+0000","last_change":"2023-11-15T02:57:44.570258+0000","last_active":"2023-11-15T02:57:44.570258+0000","last_peered":"2023-11-15T02:57:44.570258+0000","last_clean":"2023-11-15T02:57:44.570258+0000","last_became_active":"2023-11-15T02:57:44.569983+0000","last_became_peered":"2023-11-15T02:57:44.569983+0000","last_unstale":"2023-11-15T02:57:44.570258+0000","last_undegraded":"2023-11-15T02:57:44.570258+0000","last_fullsized":"2023-11-15T02:57:44.570258+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.096037+0000","last_change":"2023-11-15T02:57:44.096037+0000","last_active":"2023-11-15T02:57:44.096037+0000","last_peered":"2023-11-15T02:57:44.096037+0000","last_clean":"2023-11-15T02:57:44.096037+0000","last_became_active":"2023-11-15T02:57:44.095669+0000","last_became_peered":"2023-11-15T02:57:44.095669+0000","last_unstale":"2023-11-15T02:57:44.096037+0000","last_undegraded":"2023-11-15T02:57:44.096037+0000","last_fullsized":"2023-11-15T02:57:44.096037+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585203+0000","last_change":"2023-11-15T02:57:44.585203+0000","last_active":"2023-11-15T02:57:44.585203+0000","last_peered":"2023-11-15T02:57:44.585203+0000","last_clean":"2023-11-15T02:57:44.585203+0000","last_became_active":"2023-11-15T02:57:44.583007+0000","last_became_peered":"2023-11-15T02:57:44.583007+0000","last_unstale":"2023-11-15T02:57:44.585203+0000","last_undegraded":"2023-11-15T02:57:44.585203+0000","last_fullsized":"2023-11-15T02:57:44.585203+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.581336+0000","last_change":"2023-11-15T02:57:44.096764+0000","last_active":"2023-11-15T02:57:44.581336+0000","last_peered":"2023-11-15T02:57:44.581336+0000","last_clean":"2023-11-15T02:57:44.581336+0000","last_became_active":"2023-11-15T02:57:44.096348+0000","last_became_peered":"2023-11-15T02:57:44.096348+0000","last_unstale":"2023-11-15T02:57:44.581336+0000","last_undegraded":"2023-11-15T02:57:44.581336+0000","last_fullsized":"2023-11-15T02:57:44.581336+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566200+0000","last_change":"2023-11-15T02:57:44.566200+0000","last_active":"2023-11-15T02:57:44.566200+0000","last_peered":"2023-11-15T02:57:44.566200+0000","last_clean":"2023-11-15T02:57:44.566200+0000","last_became_active":"2023-11-15T02:57:44.565943+0000","last_became_peered":"2023-11-15T02:57:44.565943+0000","last_unstale":"2023-11-15T02:57:44.566200+0000","last_undegraded":"2023-11-15T02:57:44.566200+0000","last_fullsized":"2023-11-15T02:57:44.566200+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.553334+0000","last_change":"2023-11-15T02:57:44.109346+0000","last_active":"2023-11-15T02:57:44.553334+0000","last_peered":"2023-11-15T02:57:44.553334+0000","last_clean":"2023-11-15T02:57:44.553334+0000","last_became_active":"2023-11-15T02:57:44.109023+0000","last_became_peered":"2023-11-15T02:57:44.109023+0000","last_unstale":"2023-11-15T02:57:44.553334+0000","last_undegraded":"2023-11-15T02:57:44.553334+0000","last_fullsized":"2023-11-15T02:57:44.553334+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566248+0000","last_change":"2023-11-15T02:57:44.566248+0000","last_active":"2023-11-15T02:57:44.566248+0000","last_peered":"2023-11-15T02:57:44.566248+0000","last_clean":"2023-11-15T02:57:44.566248+0000","last_became_active":"2023-11-15T02:57:44.566002+0000","last_became_peered":"2023-11-15T02:57:44.566002+0000","last_unstale":"2023-11-15T02:57:44.566248+0000","last_undegraded":"2023-11-15T02:57:44.566248+0000","last_fullsized":"2023-11-15T02:57:44.566248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.062537+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.588350+0000","last_change":"2023-11-15T02:57:44.588350+0000","last_active":"2023-11-15T02:57:44.588350+0000","last_peered":"2023-11-15T02:57:44.588350+0000","last_clean":"2023-11-15T02:57:44.588350+0000","last_became_active":"2023-11-15T02:57:44.588073+0000","last_became_peered":"2023-11-15T02:57:44.588073+0000","last_unstale":"2023-11-15T02:57:44.588350+0000","last_undegraded":"2023-11-15T02:57:44.588350+0000","last_fullsized":"2023-11-15T02:57:44.588350+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109706+0000","last_change":"2023-11-15T02:57:44.101276+0000","last_active":"2023-11-15T02:57:44.109706+0000","last_peered":"2023-11-15T02:57:44.109706+0000","last_clean":"2023-11-15T02:57:44.109706+0000","last_became_active":"2023-11-15T02:57:44.100908+0000","last_became_peered":"2023-11-15T02:57:44.100908+0000","last_unstale":"2023-11-15T02:57:44.109706+0000","last_undegraded":"2023-11-15T02:57:44.109706+0000","last_fullsized":"2023-11-15T02:57:44.109706+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566177+0000","last_change":"2023-11-15T02:57:44.566177+0000","last_active":"2023-11-15T02:57:44.566177+0000","last_peered":"2023-11-15T02:57:44.566177+0000","last_clean":"2023-11-15T02:57:44.566177+0000","last_became_active":"2023-11-15T02:57:44.565917+0000","last_became_peered":"2023-11-15T02:57:44.565917+0000","last_unstale":"2023-11-15T02:57:44.566177+0000","last_undegraded":"2023-11-15T02:57:44.566177+0000","last_fullsized":"2023-11-15T02:57:44.566177+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"32'4","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.586189+0000","last_change":"2023-11-15T02:57:44.102229+0000","last_active":"2023-11-15T02:57:44.586189+0000","last_peered":"2023-11-15T02:57:44.586189+0000","last_clean":"2023-11-15T02:57:44.586189+0000","last_became_active":"2023-11-15T02:57:44.101858+0000","last_became_peered":"2023-11-15T02:57:44.101858+0000","last_unstale":"2023-11-15T02:57:44.586189+0000","last_undegraded":"2023-11-15T02:57:44.586189+0000","last_fullsized":"2023-11-15T02:57:44.586189+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.578142+0000","last_change":"2023-11-15T02:57:44.578142+0000","last_active":"2023-11-15T02:57:44.578142+0000","last_peered":"2023-11-15T02:57:44.578142+0000","last_clean":"2023-11-15T02:57:44.578142+0000","last_became_active":"2023-11-15T02:57:44.575328+0000","last_became_peered":"2023-11-15T02:57:44.575328+0000","last_unstale":"2023-11-15T02:57:44.578142+0000","last_undegraded":"2023-11-15T02:57:44.578142+0000","last_fullsized":"2023-11-15T02:57:44.578142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.100857+0000","last_change":"2023-11-15T02:57:44.100857+0000","last_active":"2023-11-15T02:57:44.100857+0000","last_peered":"2023-11-15T02:57:44.100857+0000","last_clean":"2023-11-15T02:57:44.100857+0000","last_became_active":"2023-11-15T02:57:44.099959+0000","last_became_peered":"2023-11-15T02:57:44.099959+0000","last_unstale":"2023-11-15T02:57:44.100857+0000","last_undegraded":"2023-11-15T02:57:44.100857+0000","last_fullsized":"2023-11-15T02:57:44.100857+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.077201+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585609+0000","last_change":"2023-11-15T02:57:44.096009+0000","last_active":"2023-11-15T02:57:44.585609+0000","last_peered":"2023-11-15T02:57:44.585609+0000","last_clean":"2023-11-15T02:57:44.585609+0000","last_became_active":"2023-11-15T02:57:44.095541+0000","last_became_peered":"2023-11-15T02:57:44.095541+0000","last_unstale":"2023-11-15T02:57:44.585609+0000","last_undegraded":"2023-11-15T02:57:44.585609+0000","last_fullsized":"2023-11-15T02:57:44.585609+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.583166+0000","last_change":"2023-11-15T02:57:44.583166+0000","last_active":"2023-11-15T02:57:44.583166+0000","last_peered":"2023-11-15T02:57:44.583166+0000","last_clean":"2023-11-15T02:57:44.583166+0000","last_became_active":"2023-11-15T02:57:44.582842+0000","last_became_peered":"2023-11-15T02:57:44.582842+0000","last_unstale":"2023-11-15T02:57:44.583166+0000","last_undegraded":"2023-11-15T02:57:44.583166+0000","last_fullsized":"2023-11-15T02:57:44.583166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109552+0000","last_change":"2023-11-15T02:57:44.101593+0000","last_active":"2023-11-15T02:57:44.109552+0000","last_peered":"2023-11-15T02:57:44.109552+0000","last_clean":"2023-11-15T02:57:44.109552+0000","last_became_active":"2023-11-15T02:57:44.101261+0000","last_became_peered":"2023-11-15T02:57:44.101261+0000","last_unstale":"2023-11-15T02:57:44.109552+0000","last_undegraded":"2023-11-15T02:57:44.109552+0000","last_fullsized":"2023-11-15T02:57:44.109552+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.584677+0000","last_change":"2023-11-15T02:57:44.584677+0000","last_active":"2023-11-15T02:57:44.584677+0000","last_peered":"2023-11-15T02:57:44.584677+0000","last_clean":"2023-11-15T02:57:44.584677+0000","last_became_active":"2023-11-15T02:57:44.584394+0000","last_became_peered":"2023-11-15T02:57:44.584394+0000","last_unstale":"2023-11-15T02:57:44.584677+0000","last_undegraded":"2023-11-15T02:57:44.584677+0000","last_fullsized":"2023-11-15T02:57:44.584677+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.064181+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.590180+0000","last_change":"2023-11-15T02:57:44.590180+0000","last_active":"2023-11-15T02:57:44.590180+0000","last_peered":"2023-11-15T02:57:44.590180+0000","last_clean":"2023-11-15T02:57:44.590180+0000","last_became_active":"2023-11-15T02:57:44.589752+0000","last_became_peered":"2023-11-15T02:57:44.589752+0000","last_unstale":"2023-11-15T02:57:44.590180+0000","last_undegraded":"2023-11-15T02:57:44.590180+0000","last_fullsized":"2023-11-15T02:57:44.590180+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560777+0000","last_change":"2023-11-15T02:57:44.109252+0000","last_active":"2023-11-15T02:57:44.560777+0000","last_peered":"2023-11-15T02:57:44.560777+0000","last_clean":"2023-11-15T02:57:44.560777+0000","last_became_active":"2023-11-15T02:57:44.108907+0000","last_became_peered":"2023-11-15T02:57:44.108907+0000","last_unstale":"2023-11-15T02:57:44.560777+0000","last_undegraded":"2023-11-15T02:57:44.560777+0000","last_fullsized":"2023-11-15T02:57:44.560777+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.568542+0000","last_change":"2023-11-15T02:57:44.568542+0000","last_active":"2023-11-15T02:57:44.568542+0000","last_peered":"2023-11-15T02:57:44.568542+0000","last_clean":"2023-11-15T02:57:44.568542+0000","last_became_active":"2023-11-15T02:57:44.568269+0000","last_became_peered":"2023-11-15T02:57:44.568269+0000","last_unstale":"2023-11-15T02:57:44.568542+0000","last_undegraded":"2023-11-15T02:57:44.568542+0000","last_fullsized":"2023-11-15T02:57:44.568542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.103890+0000","last_change":"2023-11-15T02:57:44.097937+0000","last_active":"2023-11-15T02:57:44.103890+0000","last_peered":"2023-11-15T02:57:44.103890+0000","last_clean":"2023-11-15T02:57:44.103890+0000","last_became_active":"2023-11-15T02:57:44.097630+0000","last_became_peered":"2023-11-15T02:57:44.097630+0000","last_unstale":"2023-11-15T02:57:44.103890+0000","last_undegraded":"2023-11-15T02:57:44.103890+0000","last_fullsized":"2023-11-15T02:57:44.103890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.078891+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.111403+0000","last_change":"2023-11-15T02:57:44.108255+0000","last_active":"2023-11-15T02:57:44.111403+0000","last_peered":"2023-11-15T02:57:44.111403+0000","last_clean":"2023-11-15T02:57:44.111403+0000","last_became_active":"2023-11-15T02:57:44.107961+0000","last_became_peered":"2023-11-15T02:57:44.107961+0000","last_unstale":"2023-11-15T02:57:44.111403+0000","last_undegraded":"2023-11-15T02:57:44.111403+0000","last_fullsized":"2023-11-15T02:57:44.111403+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.570926+0000","last_change":"2023-11-15T02:57:44.570926+0000","last_active":"2023-11-15T02:57:44.570926+0000","last_peered":"2023-11-15T02:57:44.570926+0000","last_clean":"2023-11-15T02:57:44.570926+0000","last_became_active":"2023-11-15T02:57:44.570646+0000","last_became_peered":"2023-11-15T02:57:44.570646+0000","last_unstale":"2023-11-15T02:57:44.570926+0000","last_undegraded":"2023-11-15T02:57:44.570926+0000","last_fullsized":"2023-11-15T02:57:44.570926+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.111257+0000","last_change":"2023-11-15T02:57:44.105564+0000","last_active":"2023-11-15T02:57:44.111257+0000","last_peered":"2023-11-15T02:57:44.111257+0000","last_clean":"2023-11-15T02:57:44.111257+0000","last_became_active":"2023-11-15T02:57:44.105312+0000","last_became_peered":"2023-11-15T02:57:44.105312+0000","last_unstale":"2023-11-15T02:57:44.111257+0000","last_undegraded":"2023-11-15T02:57:44.111257+0000","last_fullsized":"2023-11-15T02:57:44.111257+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.580419+0000","last_change":"2023-11-15T02:57:44.580419+0000","last_active":"2023-11-15T02:57:44.580419+0000","last_peered":"2023-11-15T02:57:44.580419+0000","last_clean":"2023-11-15T02:57:44.580419+0000","last_became_active":"2023-11-15T02:57:44.580057+0000","last_became_peered":"2023-11-15T02:57:44.580057+0000","last_unstale":"2023-11-15T02:57:44.580419+0000","last_undegraded":"2023-11-15T02:57:44.580419+0000","last_fullsized":"2023-11-15T02:57:44.580419+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.107016+0000","last_change":"2023-11-15T02:57:44.097433+0000","last_active":"2023-11-15T02:57:44.107016+0000","last_peered":"2023-11-15T02:57:44.107016+0000","last_clean":"2023-11-15T02:57:44.107016+0000","last_became_active":"2023-11-15T02:57:44.097149+0000","last_became_peered":"2023-11-15T02:57:44.097149+0000","last_unstale":"2023-11-15T02:57:44.107016+0000","last_undegraded":"2023-11-15T02:57:44.107016+0000","last_fullsized":"2023-11-15T02:57:44.107016+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.090441+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T02:57:44.090441+0000","last_peered":"2023-11-15T02:57:44.090441+0000","last_clean":"2023-11-15T02:57:44.090441+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T02:57:44.090441+0000","last_undegraded":"2023-11-15T02:57:44.090441+0000","last_fullsized":"2023-11-15T02:57:44.090441+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.584602+0000","last_change":"2023-11-15T02:57:44.584602+0000","last_active":"2023-11-15T02:57:44.584602+0000","last_peered":"2023-11-15T02:57:44.584602+0000","last_clean":"2023-11-15T02:57:44.584602+0000","last_became_active":"2023-11-15T02:57:44.584333+0000","last_became_peered":"2023-11-15T02:57:44.584333+0000","last_unstale":"2023-11-15T02:57:44.584602+0000","last_undegraded":"2023-11-15T02:57:44.584602+0000","last_fullsized":"2023-11-15T02:57:44.584602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.104126+0000","last_change":"2023-11-15T02:57:44.099697+0000","last_active":"2023-11-15T02:57:44.104126+0000","last_peered":"2023-11-15T02:57:44.104126+0000","last_clean":"2023-11-15T02:57:44.104126+0000","last_became_active":"2023-11-15T02:57:44.099388+0000","last_became_peered":"2023-11-15T02:57:44.099388+0000","last_unstale":"2023-11-15T02:57:44.104126+0000","last_undegraded":"2023-11-15T02:57:44.104126+0000","last_fullsized":"2023-11-15T02:57:44.104126+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-11-15T02:57:38.672457+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T02:57:38.672457+0000","last_peered":"2023-11-15T02:57:38.672457+0000","last_clean":"2023-11-15T02:57:38.672457+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T02:57:38.672457+0000","last_undegraded":"2023-11-15T02:57:38.672457+0000","last_fullsized":"2023-11-15T02:57:38.672457+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.054985+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.105522+0000","last_change":"2023-11-15T02:57:44.097789+0000","last_active":"2023-11-15T02:57:44.105522+0000","last_peered":"2023-11-15T02:57:44.105522+0000","last_clean":"2023-11-15T02:57:44.105522+0000","last_became_active":"2023-11-15T02:57:44.097133+0000","last_became_peered":"2023-11-15T02:57:44.097133+0000","last_unstale":"2023-11-15T02:57:44.105522+0000","last_undegraded":"2023-11-15T02:57:44.105522+0000","last_fullsized":"2023-11-15T02:57:44.105522+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.090466+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T02:57:44.090466+0000","last_peered":"2023-11-15T02:57:44.090466+0000","last_clean":"2023-11-15T02:57:44.090466+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T02:57:44.090466+0000","last_undegraded":"2023-11-15T02:57:44.090466+0000","last_fullsized":"2023-11-15T02:57:44.090466+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582488+0000","last_change":"2023-11-15T02:57:44.582488+0000","last_active":"2023-11-15T02:57:44.582488+0000","last_peered":"2023-11-15T02:57:44.582488+0000","last_clean":"2023-11-15T02:57:44.582488+0000","last_became_active":"2023-11-15T02:57:44.582160+0000","last_became_peered":"2023-11-15T02:57:44.582160+0000","last_unstale":"2023-11-15T02:57:44.582488+0000","last_undegraded":"2023-11-15T02:57:44.582488+0000","last_fullsized":"2023-11-15T02:57:44.582488+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.113534+0000","last_change":"2023-11-15T02:57:44.110737+0000","last_active":"2023-11-15T02:57:44.113534+0000","last_peered":"2023-11-15T02:57:44.113534+0000","last_clean":"2023-11-15T02:57:44.113534+0000","last_became_active":"2023-11-15T02:57:44.110423+0000","last_became_peered":"2023-11-15T02:57:44.110423+0000","last_unstale":"2023-11-15T02:57:44.113534+0000","last_undegraded":"2023-11-15T02:57:44.113534+0000","last_fullsized":"2023-11-15T02:57:44.113534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-11-15T02:57:38.672509+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T02:57:38.672509+0000","last_peered":"2023-11-15T02:57:38.672509+0000","last_clean":"2023-11-15T02:57:38.672509+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T02:57:38.672509+0000","last_undegraded":"2023-11-15T02:57:38.672509+0000","last_fullsized":"2023-11-15T02:57:38.672509+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577142+0000","last_change":"2023-11-15T02:57:44.577142+0000","last_active":"2023-11-15T02:57:44.577142+0000","last_peered":"2023-11-15T02:57:44.577142+0000","last_clean":"2023-11-15T02:57:44.577142+0000","last_became_active":"2023-11-15T02:57:44.576883+0000","last_became_peered":"2023-11-15T02:57:44.576883+0000","last_unstale":"2023-11-15T02:57:44.577142+0000","last_undegraded":"2023-11-15T02:57:44.577142+0000","last_fullsized":"2023-11-15T02:57:44.577142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.113548+0000","last_change":"2023-11-15T02:57:44.111033+0000","last_active":"2023-11-15T02:57:44.113548+0000","last_peered":"2023-11-15T02:57:44.113548+0000","last_clean":"2023-11-15T02:57:44.113548+0000","last_became_active":"2023-11-15T02:57:44.109168+0000","last_became_peered":"2023-11-15T02:57:44.109168+0000","last_unstale":"2023-11-15T02:57:44.113548+0000","last_undegraded":"2023-11-15T02:57:44.113548+0000","last_fullsized":"2023-11-15T02:57:44.113548+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-11-15T02:57:43.068175+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T02:57:43.068175+0000","last_peered":"2023-11-15T02:57:43.068175+0000","last_clean":"2023-11-15T02:57:43.068175+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T02:57:43.068175+0000","last_undegraded":"2023-11-15T02:57:43.068175+0000","last_fullsized":"2023-11-15T02:57:43.068175+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577944+0000","last_change":"2023-11-15T02:57:44.577944+0000","last_active":"2023-11-15T02:57:44.577944+0000","last_peered":"2023-11-15T02:57:44.577944+0000","last_clean":"2023-11-15T02:57:44.577944+0000","last_became_active":"2023-11-15T02:57:44.577738+0000","last_became_peered":"2023-11-15T02:57:44.577738+0000","last_unstale":"2023-11-15T02:57:44.577944+0000","last_undegraded":"2023-11-15T02:57:44.577944+0000","last_fullsized":"2023-11-15T02:57:44.577944+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.105276+0000","last_change":"2023-11-15T02:57:44.098264+0000","last_active":"2023-11-15T02:57:44.105276+0000","last_peered":"2023-11-15T02:57:44.105276+0000","last_clean":"2023-11-15T02:57:44.105276+0000","last_became_active":"2023-11-15T02:57:44.095009+0000","last_became_peered":"2023-11-15T02:57:44.095009+0000","last_unstale":"2023-11-15T02:57:44.105276+0000","last_undegraded":"2023-11-15T02:57:44.105276+0000","last_fullsized":"2023-11-15T02:57:44.105276+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.078017+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T02:57:44.078017+0000","last_peered":"2023-11-15T02:57:44.078017+0000","last_clean":"2023-11-15T02:57:44.078017+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T02:57:44.078017+0000","last_undegraded":"2023-11-15T02:57:44.078017+0000","last_fullsized":"2023-11-15T02:57:44.078017+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.054985+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.095125+0000","last_change":"2023-11-15T02:57:44.095125+0000","last_active":"2023-11-15T02:57:44.095125+0000","last_peered":"2023-11-15T02:57:44.095125+0000","last_clean":"2023-11-15T02:57:44.095125+0000","last_became_active":"2023-11-15T02:57:44.094525+0000","last_became_peered":"2023-11-15T02:57:44.094525+0000","last_unstale":"2023-11-15T02:57:44.095125+0000","last_undegraded":"2023-11-15T02:57:44.095125+0000","last_fullsized":"2023-11-15T02:57:44.095125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.088566+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T02:57:44.088566+0000","last_peered":"2023-11-15T02:57:44.088566+0000","last_clean":"2023-11-15T02:57:44.088566+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T02:57:44.088566+0000","last_undegraded":"2023-11-15T02:57:44.088566+0000","last_fullsized":"2023-11-15T02:57:44.088566+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.077301+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.112596+0000","last_change":"2023-11-15T02:57:44.111330+0000","last_active":"2023-11-15T02:57:44.112596+0000","last_peered":"2023-11-15T02:57:44.112596+0000","last_clean":"2023-11-15T02:57:44.112596+0000","last_became_active":"2023-11-15T02:57:44.111040+0000","last_became_peered":"2023-11-15T02:57:44.111040+0000","last_unstale":"2023-11-15T02:57:44.112596+0000","last_undegraded":"2023-11-15T02:57:44.112596+0000","last_fullsized":"2023-11-15T02:57:44.112596+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-11-15T02:57:43.068311+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T02:57:43.068311+0000","last_peered":"2023-11-15T02:57:43.068311+0000","last_clean":"2023-11-15T02:57:43.068311+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T02:57:43.068311+0000","last_undegraded":"2023-11-15T02:57:43.068311+0000","last_fullsized":"2023-11-15T02:57:43.068311+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.088937+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T02:57:44.088937+0000","last_peered":"2023-11-15T02:57:44.088937+0000","last_clean":"2023-11-15T02:57:44.088937+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T02:57:44.088937+0000","last_undegraded":"2023-11-15T02:57:44.088937+0000","last_fullsized":"2023-11-15T02:57:44.088937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":1302,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297432,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074408,"statfs":{"total":96636764160,"available":96332193792,"internally_reserved":0,"allocated":344064,"data_stored":138527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4970000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.369}]}]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":360448,"data_stored":146260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":360448,"data_stored":145728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297432,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074408,"statfs":{"total":96636764160,"available":96332193792,"internally_reserved":0,"allocated":344064,"data_stored":138527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.512}]}]},{"osd":2,"up_from":14,"seq":60129542159,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297428,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074412,"statfs":{"total":96636764160,"available":96332197888,"internally_reserved":0,"allocated":339968,"data_stored":137258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]}]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297432,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074408,"statfs":{"total":96636764160,"available":96332193792,"internally_reserved":0,"allocated":344064,"data_stored":138527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":1.0549999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.2839999999999998,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.81799999999999995}]},{"osd":3,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":3.54,"5min":3.54,"15min":3.54},"last":1.093}]},{"osd":4,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.85599999999999998}]},{"osd":5,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":3.6760000000000002,"5min":3.6760000000000002,"15min":3.6760000000000002},"last":1.149}]},{"osd":6,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":3.71,"5min":3.71,"15min":3.71},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":1.024,"5min":1.024,"15min":1.024},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.165}]},{"osd":7,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":1.032,"5min":1.032,"15min":1.032},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.238},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.2090000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297420,"kb_used_data":324,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074420,"statfs":{"total":96636764160,"available":96332206080,"internally_reserved":0,"allocated":331776,"data_stored":131848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3370000000000002}]},{"osd":1,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":3.6349999999999998,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":3.6349999999999998},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.516,"5min":3.516,"15min":3.516},"last":3.516}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.4180000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.141}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3879999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.2309999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3660000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297408,"kb_used_data":312,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074432,"statfs":{"total":96636764160,"available":96332218368,"internally_reserved":0,"allocated":319488,"data_stored":125627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T02:57:48.824 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T02:57:49.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.015+0000 7fe7c6c39700 1 Processor -- start 2023-11-15T02:57:49.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.015+0000 7fe7c6c39700 1 -- start start 2023-11-15T02:57:49.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.019+0000 7fe7c6c39700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c0152850 0x7fe7c00c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.019+0000 7fe7c6c39700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe7c015be30 0x7fe7c015c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.019+0000 7fe7c6c39700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c00c1d50 con 0x7fe7c00c3280 2023-11-15T02:57:49.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.019+0000 7fe7c6c39700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c00c1860 con 0x7fe7c015be30 2023-11-15T02:57:49.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.019+0000 7fe7c6c39700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c015cb80 con 0x7fe7c0152850 2023-11-15T02:57:49.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.019+0000 7fe7c5c37700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe7c00c3280 0x7fe7c0152140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40658/0 (socket says 172.21.15.121:40658) 2023-11-15T02:57:49.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.019+0000 7fe7c5c37700 1 -- 172.21.15.121:0/2029149711 learned_addr learned my addr 172.21.15.121:0/2029149711 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:49.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.019+0000 7fe7c5436700 1 --2- 172.21.15.121:0/2029149711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c0152850 0x7fe7c00c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.019+0000 7fe7c6438700 1 --2- 172.21.15.121:0/2029149711 >> v2:172.21.15.190:3300/0 conn(0x7fe7c015be30 0x7fe7c015c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.019+0000 7fe7c4c35700 1 -- 172.21.15.121:0/2029149711 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 664306671 0 0) 0x7fe7c00c1d50 con 0x7fe7c00c3280 2023-11-15T02:57:49.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.019+0000 7fe7c4c35700 1 -- 172.21.15.121:0/2029149711 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7ac0033f0 con 0x7fe7c00c3280 2023-11-15T02:57:49.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c4c35700 1 -- 172.21.15.121:0/2029149711 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3456167479 0 0) 0x7fe7ac0033f0 con 0x7fe7c00c3280 2023-11-15T02:57:49.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c4c35700 1 -- 172.21.15.121:0/2029149711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c0152850 msgr2=0x7fe7c00c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:49.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c4c35700 1 --2- 172.21.15.121:0/2029149711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c0152850 0x7fe7c00c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c4c35700 1 -- 172.21.15.121:0/2029149711 >> v2:172.21.15.190:3300/0 conn(0x7fe7c015be30 msgr2=0x7fe7c015c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:49.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c4c35700 1 --2- 172.21.15.121:0/2029149711 >> v2:172.21.15.190:3300/0 conn(0x7fe7c015be30 0x7fe7c015c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c4c35700 1 -- 172.21.15.121:0/2029149711 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c00a9240 con 0x7fe7c00c3280 2023-11-15T02:57:49.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c4c35700 1 -- 172.21.15.121:0/2029149711 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fe7bc001470 con 0x7fe7c00c3280 2023-11-15T02:57:49.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c4c35700 1 -- 172.21.15.121:0/2029149711 >> v1:172.21.15.121:6789/0 conn(0x7fe7c00c3280 legacy=0x7fe7c0152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:49.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c4c35700 1 --2- 172.21.15.121:0/2029149711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7ac0038e0 0x7fe7ac003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c4c35700 1 --2- 172.21.15.121:0/2029149711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7ac004690 0x7fe7ac004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c4c35700 1 --2- 172.21.15.121:0/2029149711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7ac005000 0x7fe7ac00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c4c35700 1 -- 172.21.15.121:0/2029149711 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c00c1860 con 0x7fe7ac005000 2023-11-15T02:57:49.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c4c35700 1 -- 172.21.15.121:0/2029149711 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c015cb80 con 0x7fe7ac004690 2023-11-15T02:57:49.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c4c35700 1 -- 172.21.15.121:0/2029149711 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac00bdd0 con 0x7fe7ac0038e0 2023-11-15T02:57:49.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c5436700 1 --2- 172.21.15.121:0/2029149711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c0152850 0x7fe7c00c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:57:49.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c6438700 1 --2- 172.21.15.121:0/2029149711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7ac005000 0x7fe7ac00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c5436700 1 --2- 172.21.15.121:0/2029149711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7ac004690 0x7fe7ac004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c5c37700 1 --2- 172.21.15.121:0/2029149711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7ac0038e0 0x7fe7ac003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c6438700 1 -- 172.21.15.121:0/2029149711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7ac0038e0 msgr2=0x7fe7ac003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:49.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c6438700 1 --2- 172.21.15.121:0/2029149711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7ac0038e0 0x7fe7ac003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c6438700 1 -- 172.21.15.121:0/2029149711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7ac004690 msgr2=0x7fe7ac004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:49.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c6438700 1 --2- 172.21.15.121:0/2029149711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7ac004690 0x7fe7ac004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c6438700 1 -- 172.21.15.121:0/2029149711 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac0033f0 con 0x7fe7ac005000 2023-11-15T02:57:49.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c6438700 1 --2- 172.21.15.121:0/2029149711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7ac005000 0x7fe7ac00b8e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fe7b800e210 tx=0x7fe7b800b040).ready entity=mon.0 client_cookie=2002379a631884a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:49.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c4c35700 1 -- 172.21.15.121:0/2029149711 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8009ac0 con 0x7fe7ac005000 2023-11-15T02:57:49.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c4c35700 1 -- 172.21.15.121:0/2029149711 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b800f040 con 0x7fe7ac005000 2023-11-15T02:57:49.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c4c35700 1 -- 172.21.15.121:0/2029149711 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b800ecd0 con 0x7fe7ac005000 2023-11-15T02:57:49.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c6c39700 1 -- 172.21.15.121:0/2029149711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7ac005000 msgr2=0x7fe7ac00b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:49.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c6c39700 1 --2- 172.21.15.121:0/2029149711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7ac005000 0x7fe7ac00b8e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fe7b800e210 tx=0x7fe7b800b040).stop 2023-11-15T02:57:49.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c6438700 1 -- 172.21.15.121:0/2029149711 reap_dead start 2023-11-15T02:57:49.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c6c39700 1 -- 172.21.15.121:0/2029149711 shutdown_connections 2023-11-15T02:57:49.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.023+0000 7fe7c6c39700 1 -- 172.21.15.121:0/2029149711 >> 172.21.15.121:0/2029149711 conn(0x7fe7c0062190 msgr2=0x7fe7c0062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:49.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.027+0000 7fe7c6c39700 1 -- 172.21.15.121:0/2029149711 shutdown_connections 2023-11-15T02:57:49.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.027+0000 7fe7c6c39700 1 -- 172.21.15.121:0/2029149711 wait complete. 2023-11-15T02:57:49.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.027+0000 7fe7c6c39700 1 Processor -- start 2023-11-15T02:57:49.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.027+0000 7fe7c6c39700 1 -- start start 2023-11-15T02:57:49.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7c6c39700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c00bd170 0x7fe7c00bd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7c6c39700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c0168ef0 0x7fe7c014e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7c6c39700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7c0151150 0x7fe7c01515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7c6c39700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c01684b0 con 0x7fe7c0151150 2023-11-15T02:57:49.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7c6c39700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c0168a60 con 0x7fe7c00bd170 2023-11-15T02:57:49.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7c6c39700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c0151ac0 con 0x7fe7c0168ef0 2023-11-15T02:57:49.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7c6438700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7c0151150 0x7fe7c01515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7c6438700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7c0151150 0x7fe7c01515d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56438/0 (socket says 172.21.15.121:56438) 2023-11-15T02:57:49.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7c6438700 1 -- 172.21.15.121:0/3925672539 learned_addr learned my addr 172.21.15.121:0/3925672539 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:49.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7c5c37700 1 --2- 172.21.15.121:0/3925672539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c00bd170 0x7fe7c00bd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7c5436700 1 --2- 172.21.15.121:0/3925672539 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c0168ef0 0x7fe7c014e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7c6438700 1 -- 172.21.15.121:0/3925672539 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c0168ef0 msgr2=0x7fe7c014e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:49.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7c6438700 1 --2- 172.21.15.121:0/3925672539 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c0168ef0 0x7fe7c014e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7c6438700 1 -- 172.21.15.121:0/3925672539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c00bd170 msgr2=0x7fe7c00bd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:49.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7c6438700 1 --2- 172.21.15.121:0/3925672539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c00bd170 0x7fe7c00bd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7c6438700 1 -- 172.21.15.121:0/3925672539 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c014ebb0 con 0x7fe7c0151150 2023-11-15T02:57:49.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7c6438700 1 --2- 172.21.15.121:0/3925672539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7c0151150 0x7fe7c01515d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fe7ac0044b0 tx=0x7fe7b801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:49.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7b5ffb700 1 -- 172.21.15.121:0/3925672539 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8021030 con 0x7fe7c0151150 2023-11-15T02:57:49.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7b5ffb700 1 -- 172.21.15.121:0/3925672539 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b8009ac0 con 0x7fe7c0151150 2023-11-15T02:57:49.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7c6c39700 1 -- 172.21.15.121:0/3925672539 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c00bd550 con 0x7fe7c0151150 2023-11-15T02:57:49.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7b5ffb700 1 -- 172.21.15.121:0/3925672539 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8013dc0 con 0x7fe7c0151150 2023-11-15T02:57:49.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.031+0000 7fe7c6c39700 1 -- 172.21.15.121:0/3925672539 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c00be2f0 con 0x7fe7c0151150 2023-11-15T02:57:49.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.035+0000 7fe7c6c39700 1 -- 172.21.15.121:0/3925672539 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c00be510 con 0x7fe7c0151150 2023-11-15T02:57:49.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.035+0000 7fe7b5ffb700 1 -- 172.21.15.121:0/3925672539 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe7b8024070 con 0x7fe7c0151150 2023-11-15T02:57:49.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.035+0000 7fe7b5ffb700 1 --2- 172.21.15.121:0/3925672539 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe7a405a540 0x7fe7a405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.035+0000 7fe7b5ffb700 1 -- 172.21.15.121:0/3925672539 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe7b8022070 con 0x7fe7c0151150 2023-11-15T02:57:49.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.035+0000 7fe7c5c37700 1 --2- 172.21.15.121:0/3925672539 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe7a405a540 0x7fe7a405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.039+0000 7fe7c5c37700 1 --2- 172.21.15.121:0/3925672539 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe7a405a540 0x7fe7a405ca00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fe7bc000c00 tx=0x7fe7bc007040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:49.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.039+0000 7fe7b5ffb700 1 -- 172.21.15.121:0/3925672539 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe7b8049500 con 0x7fe7c0151150 2023-11-15T02:57:49.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.199+0000 7fe7c6c39700 1 -- 172.21.15.121:0/3925672539 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe7c00a7cc0 con 0x7fe7a405a540 2023-11-15T02:57:49.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.219+0000 7fe7b5ffb700 1 -- 172.21.15.121:0/3925672539 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+338132 (secure 0 0 0) 0x7fe7c00a7cc0 con 0x7fe7a405a540 2023-11-15T02:57:49.231 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:49.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.227+0000 7fe7c6c39700 1 -- 172.21.15.121:0/3925672539 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe7a405a540 msgr2=0x7fe7a405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:49.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.227+0000 7fe7c6c39700 1 --2- 172.21.15.121:0/3925672539 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe7a405a540 0x7fe7a405ca00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fe7bc000c00 tx=0x7fe7bc007040).stop 2023-11-15T02:57:49.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.227+0000 7fe7c6c39700 1 -- 172.21.15.121:0/3925672539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7c0151150 msgr2=0x7fe7c01515d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:49.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.227+0000 7fe7c6c39700 1 --2- 172.21.15.121:0/3925672539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7c0151150 0x7fe7c01515d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fe7ac0044b0 tx=0x7fe7b801a000).stop 2023-11-15T02:57:49.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.227+0000 7fe7c6c39700 1 -- 172.21.15.121:0/3925672539 shutdown_connections 2023-11-15T02:57:49.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.227+0000 7fe7c6c39700 1 --2- 172.21.15.121:0/3925672539 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe7a405a540 0x7fe7a405ca00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.227+0000 7fe7c6c39700 1 --2- 172.21.15.121:0/3925672539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7c0151150 0x7fe7c01515d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.227+0000 7fe7c6c39700 1 --2- 172.21.15.121:0/3925672539 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c0168ef0 0x7fe7c014e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.227+0000 7fe7c6c39700 1 --2- 172.21.15.121:0/3925672539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c00bd170 0x7fe7c00bd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.227+0000 7fe7c6c39700 1 -- 172.21.15.121:0/3925672539 >> 172.21.15.121:0/3925672539 conn(0x7fe7c0062190 msgr2=0x7fe7c0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:49.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.227+0000 7fe7c6c39700 1 -- 172.21.15.121:0/3925672539 shutdown_connections 2023-11-15T02:57:49.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.227+0000 7fe7c6c39700 1 -- 172.21.15.121:0/3925672539 wait complete. 2023-11-15T02:57:49.237 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T02:57:49.255 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2023-11-15T02:57:47.931491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1302,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":262,"acting":262,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2379448,"kb_used_data":2680,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752595272,"statfs":{"total":773094113280,"available":770657558528,"internally_reserved":0,"allocated":2744320,"data_stored":1102302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4297,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.436357"},"pg_stats":[{"pgid":"6.3c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.576426+0000","last_change":"2023-11-15T02:57:44.576426+0000","last_active":"2023-11-15T02:57:44.576426+0000","last_peered":"2023-11-15T02:57:44.576426+0000","last_clean":"2023-11-15T02:57:44.576426+0000","last_became_active":"2023-11-15T02:57:44.572509+0000","last_became_peered":"2023-11-15T02:57:44.572509+0000","last_unstale":"2023-11-15T02:57:44.576426+0000","last_undegraded":"2023-11-15T02:57:44.576426+0000","last_fullsized":"2023-11-15T02:57:44.576426+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3f","version":"32'3","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.587176+0000","last_change":"2023-11-15T02:57:44.105284+0000","last_active":"2023-11-15T02:57:44.587176+0000","last_peered":"2023-11-15T02:57:44.587176+0000","last_clean":"2023-11-15T02:57:44.587176+0000","last_became_active":"2023-11-15T02:57:44.104935+0000","last_became_peered":"2023-11-15T02:57:44.104935+0000","last_unstale":"2023-11-15T02:57:44.587176+0000","last_undegraded":"2023-11-15T02:57:44.587176+0000","last_fullsized":"2023-11-15T02:57:44.587176+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566808+0000","last_change":"2023-11-15T02:57:44.566808+0000","last_active":"2023-11-15T02:57:44.566808+0000","last_peered":"2023-11-15T02:57:44.566808+0000","last_clean":"2023-11-15T02:57:44.566808+0000","last_became_active":"2023-11-15T02:57:44.566228+0000","last_became_peered":"2023-11-15T02:57:44.566228+0000","last_unstale":"2023-11-15T02:57:44.566808+0000","last_undegraded":"2023-11-15T02:57:44.566808+0000","last_fullsized":"2023-11-15T02:57:44.566808+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109248+0000","last_change":"2023-11-15T02:57:44.101869+0000","last_active":"2023-11-15T02:57:44.109248+0000","last_peered":"2023-11-15T02:57:44.109248+0000","last_clean":"2023-11-15T02:57:44.109248+0000","last_became_active":"2023-11-15T02:57:44.101602+0000","last_became_peered":"2023-11-15T02:57:44.101602+0000","last_unstale":"2023-11-15T02:57:44.109248+0000","last_undegraded":"2023-11-15T02:57:44.109248+0000","last_fullsized":"2023-11-15T02:57:44.109248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.564204+0000","last_change":"2023-11-15T02:57:44.564204+0000","last_active":"2023-11-15T02:57:44.564204+0000","last_peered":"2023-11-15T02:57:44.564204+0000","last_clean":"2023-11-15T02:57:44.564204+0000","last_became_active":"2023-11-15T02:57:44.563976+0000","last_became_peered":"2023-11-15T02:57:44.563976+0000","last_unstale":"2023-11-15T02:57:44.564204+0000","last_undegraded":"2023-11-15T02:57:44.564204+0000","last_fullsized":"2023-11-15T02:57:44.564204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.096033+0000","last_change":"2023-11-15T02:57:44.096033+0000","last_active":"2023-11-15T02:57:44.096033+0000","last_peered":"2023-11-15T02:57:44.096033+0000","last_clean":"2023-11-15T02:57:44.096033+0000","last_became_active":"2023-11-15T02:57:44.095626+0000","last_became_peered":"2023-11-15T02:57:44.095626+0000","last_unstale":"2023-11-15T02:57:44.096033+0000","last_undegraded":"2023-11-15T02:57:44.096033+0000","last_fullsized":"2023-11-15T02:57:44.096033+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.568907+0000","last_change":"2023-11-15T02:57:44.568907+0000","last_active":"2023-11-15T02:57:44.568907+0000","last_peered":"2023-11-15T02:57:44.568907+0000","last_clean":"2023-11-15T02:57:44.568907+0000","last_became_active":"2023-11-15T02:57:44.568621+0000","last_became_peered":"2023-11-15T02:57:44.568621+0000","last_unstale":"2023-11-15T02:57:44.568907+0000","last_undegraded":"2023-11-15T02:57:44.568907+0000","last_fullsized":"2023-11-15T02:57:44.568907+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.065783+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.570582+0000","last_change":"2023-11-15T02:57:44.570582+0000","last_active":"2023-11-15T02:57:44.570582+0000","last_peered":"2023-11-15T02:57:44.570582+0000","last_clean":"2023-11-15T02:57:44.570582+0000","last_became_active":"2023-11-15T02:57:44.570314+0000","last_became_peered":"2023-11-15T02:57:44.570314+0000","last_unstale":"2023-11-15T02:57:44.570582+0000","last_undegraded":"2023-11-15T02:57:44.570582+0000","last_fullsized":"2023-11-15T02:57:44.570582+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3b","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560605+0000","last_change":"2023-11-15T02:57:44.104587+0000","last_active":"2023-11-15T02:57:44.560605+0000","last_peered":"2023-11-15T02:57:44.560605+0000","last_clean":"2023-11-15T02:57:44.560605+0000","last_became_active":"2023-11-15T02:57:44.104223+0000","last_became_peered":"2023-11-15T02:57:44.104223+0000","last_unstale":"2023-11-15T02:57:44.560605+0000","last_undegraded":"2023-11-15T02:57:44.560605+0000","last_fullsized":"2023-11-15T02:57:44.560605+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.077573+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.553452+0000","last_change":"2023-11-15T02:57:44.102587+0000","last_active":"2023-11-15T02:57:44.553452+0000","last_peered":"2023-11-15T02:57:44.553452+0000","last_clean":"2023-11-15T02:57:44.553452+0000","last_became_active":"2023-11-15T02:57:44.102311+0000","last_became_peered":"2023-11-15T02:57:44.102311+0000","last_unstale":"2023-11-15T02:57:44.553452+0000","last_undegraded":"2023-11-15T02:57:44.553452+0000","last_fullsized":"2023-11-15T02:57:44.553452+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.054985+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.110877+0000","last_change":"2023-11-15T02:57:44.105206+0000","last_active":"2023-11-15T02:57:44.110877+0000","last_peered":"2023-11-15T02:57:44.110877+0000","last_clean":"2023-11-15T02:57:44.110877+0000","last_became_active":"2023-11-15T02:57:44.104936+0000","last_became_peered":"2023-11-15T02:57:44.104936+0000","last_unstale":"2023-11-15T02:57:44.110877+0000","last_undegraded":"2023-11-15T02:57:44.110877+0000","last_fullsized":"2023-11-15T02:57:44.110877+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566763+0000","last_change":"2023-11-15T02:57:44.566763+0000","last_active":"2023-11-15T02:57:44.566763+0000","last_peered":"2023-11-15T02:57:44.566763+0000","last_clean":"2023-11-15T02:57:44.566763+0000","last_became_active":"2023-11-15T02:57:44.566510+0000","last_became_peered":"2023-11-15T02:57:44.566510+0000","last_unstale":"2023-11-15T02:57:44.566763+0000","last_undegraded":"2023-11-15T02:57:44.566763+0000","last_fullsized":"2023-11-15T02:57:44.566763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.110691+0000","last_change":"2023-11-15T02:57:44.106233+0000","last_active":"2023-11-15T02:57:44.110691+0000","last_peered":"2023-11-15T02:57:44.110691+0000","last_clean":"2023-11-15T02:57:44.110691+0000","last_became_active":"2023-11-15T02:57:44.105985+0000","last_became_peered":"2023-11-15T02:57:44.105985+0000","last_unstale":"2023-11-15T02:57:44.110691+0000","last_undegraded":"2023-11-15T02:57:44.110691+0000","last_fullsized":"2023-11-15T02:57:44.110691+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577960+0000","last_change":"2023-11-15T02:57:44.577960+0000","last_active":"2023-11-15T02:57:44.577960+0000","last_peered":"2023-11-15T02:57:44.577960+0000","last_clean":"2023-11-15T02:57:44.577960+0000","last_became_active":"2023-11-15T02:57:44.577405+0000","last_became_peered":"2023-11-15T02:57:44.577405+0000","last_unstale":"2023-11-15T02:57:44.577960+0000","last_undegraded":"2023-11-15T02:57:44.577960+0000","last_fullsized":"2023-11-15T02:57:44.577960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.099195+0000","last_change":"2023-11-15T02:57:44.091037+0000","last_active":"2023-11-15T02:57:44.099195+0000","last_peered":"2023-11-15T02:57:44.099195+0000","last_clean":"2023-11-15T02:57:44.099195+0000","last_became_active":"2023-11-15T02:57:44.090808+0000","last_became_peered":"2023-11-15T02:57:44.090808+0000","last_unstale":"2023-11-15T02:57:44.099195+0000","last_undegraded":"2023-11-15T02:57:44.099195+0000","last_fullsized":"2023-11-15T02:57:44.099195+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.077670+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.063700+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.075254+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109579+0000","last_change":"2023-11-15T02:57:44.102619+0000","last_active":"2023-11-15T02:57:44.109579+0000","last_peered":"2023-11-15T02:57:44.109579+0000","last_clean":"2023-11-15T02:57:44.109579+0000","last_became_active":"2023-11-15T02:57:44.102328+0000","last_became_peered":"2023-11-15T02:57:44.102328+0000","last_unstale":"2023-11-15T02:57:44.109579+0000","last_undegraded":"2023-11-15T02:57:44.109579+0000","last_fullsized":"2023-11-15T02:57:44.109579+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.579068+0000","last_change":"2023-11-15T02:57:44.579068+0000","last_active":"2023-11-15T02:57:44.579068+0000","last_peered":"2023-11-15T02:57:44.579068+0000","last_clean":"2023-11-15T02:57:44.579068+0000","last_became_active":"2023-11-15T02:57:44.578815+0000","last_became_peered":"2023-11-15T02:57:44.578815+0000","last_unstale":"2023-11-15T02:57:44.579068+0000","last_undegraded":"2023-11-15T02:57:44.579068+0000","last_fullsized":"2023-11-15T02:57:44.579068+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.113636+0000","last_change":"2023-11-15T02:57:44.113636+0000","last_active":"2023-11-15T02:57:44.113636+0000","last_peered":"2023-11-15T02:57:44.113636+0000","last_clean":"2023-11-15T02:57:44.113636+0000","last_became_active":"2023-11-15T02:57:44.113283+0000","last_became_peered":"2023-11-15T02:57:44.113283+0000","last_unstale":"2023-11-15T02:57:44.113636+0000","last_undegraded":"2023-11-15T02:57:44.113636+0000","last_fullsized":"2023-11-15T02:57:44.113636+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.584681+0000","last_change":"2023-11-15T02:57:44.584681+0000","last_active":"2023-11-15T02:57:44.584681+0000","last_peered":"2023-11-15T02:57:44.584681+0000","last_clean":"2023-11-15T02:57:44.584681+0000","last_became_active":"2023-11-15T02:57:44.584395+0000","last_became_peered":"2023-11-15T02:57:44.584395+0000","last_unstale":"2023-11-15T02:57:44.584681+0000","last_undegraded":"2023-11-15T02:57:44.584681+0000","last_fullsized":"2023-11-15T02:57:44.584681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.33","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560690+0000","last_change":"2023-11-15T02:57:44.086072+0000","last_active":"2023-11-15T02:57:44.560690+0000","last_peered":"2023-11-15T02:57:44.560690+0000","last_clean":"2023-11-15T02:57:44.560690+0000","last_became_active":"2023-11-15T02:57:44.085574+0000","last_became_peered":"2023-11-15T02:57:44.085574+0000","last_unstale":"2023-11-15T02:57:44.560690+0000","last_undegraded":"2023-11-15T02:57:44.560690+0000","last_fullsized":"2023-11-15T02:57:44.560690+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.076869+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.107875+0000","last_change":"2023-11-15T02:57:44.094640+0000","last_active":"2023-11-15T02:57:44.107875+0000","last_peered":"2023-11-15T02:57:44.107875+0000","last_clean":"2023-11-15T02:57:44.107875+0000","last_became_active":"2023-11-15T02:57:44.094279+0000","last_became_peered":"2023-11-15T02:57:44.094279+0000","last_unstale":"2023-11-15T02:57:44.107875+0000","last_undegraded":"2023-11-15T02:57:44.107875+0000","last_fullsized":"2023-11-15T02:57:44.107875+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.576818+0000","last_change":"2023-11-15T02:57:44.576818+0000","last_active":"2023-11-15T02:57:44.576818+0000","last_peered":"2023-11-15T02:57:44.576818+0000","last_clean":"2023-11-15T02:57:44.576818+0000","last_became_active":"2023-11-15T02:57:44.576074+0000","last_became_peered":"2023-11-15T02:57:44.576074+0000","last_unstale":"2023-11-15T02:57:44.576818+0000","last_undegraded":"2023-11-15T02:57:44.576818+0000","last_fullsized":"2023-11-15T02:57:44.576818+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109239+0000","last_change":"2023-11-15T02:57:44.109239+0000","last_active":"2023-11-15T02:57:44.109239+0000","last_peered":"2023-11-15T02:57:44.109239+0000","last_clean":"2023-11-15T02:57:44.109239+0000","last_became_active":"2023-11-15T02:57:44.103686+0000","last_became_peered":"2023-11-15T02:57:44.103686+0000","last_unstale":"2023-11-15T02:57:44.109239+0000","last_undegraded":"2023-11-15T02:57:44.109239+0000","last_fullsized":"2023-11-15T02:57:44.109239+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.074718+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"32'4","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585116+0000","last_change":"2023-11-15T02:57:44.113817+0000","last_active":"2023-11-15T02:57:44.585116+0000","last_peered":"2023-11-15T02:57:44.585116+0000","last_clean":"2023-11-15T02:57:44.585116+0000","last_became_active":"2023-11-15T02:57:44.112534+0000","last_became_peered":"2023-11-15T02:57:44.112534+0000","last_unstale":"2023-11-15T02:57:44.585116+0000","last_undegraded":"2023-11-15T02:57:44.585116+0000","last_fullsized":"2023-11-15T02:57:44.585116+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.575459+0000","last_change":"2023-11-15T02:57:44.575459+0000","last_active":"2023-11-15T02:57:44.575459+0000","last_peered":"2023-11-15T02:57:44.575459+0000","last_clean":"2023-11-15T02:57:44.575459+0000","last_became_active":"2023-11-15T02:57:44.575123+0000","last_became_peered":"2023-11-15T02:57:44.575123+0000","last_unstale":"2023-11-15T02:57:44.575459+0000","last_undegraded":"2023-11-15T02:57:44.575459+0000","last_fullsized":"2023-11-15T02:57:44.575459+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.111343+0000","last_change":"2023-11-15T02:57:44.107752+0000","last_active":"2023-11-15T02:57:44.111343+0000","last_peered":"2023-11-15T02:57:44.111343+0000","last_clean":"2023-11-15T02:57:44.111343+0000","last_became_active":"2023-11-15T02:57:44.107413+0000","last_became_peered":"2023-11-15T02:57:44.107413+0000","last_unstale":"2023-11-15T02:57:44.111343+0000","last_undegraded":"2023-11-15T02:57:44.111343+0000","last_fullsized":"2023-11-15T02:57:44.111343+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.572196+0000","last_change":"2023-11-15T02:57:44.572196+0000","last_active":"2023-11-15T02:57:44.572196+0000","last_peered":"2023-11-15T02:57:44.572196+0000","last_clean":"2023-11-15T02:57:44.572196+0000","last_became_active":"2023-11-15T02:57:44.571963+0000","last_became_peered":"2023-11-15T02:57:44.571963+0000","last_unstale":"2023-11-15T02:57:44.572196+0000","last_undegraded":"2023-11-15T02:57:44.572196+0000","last_fullsized":"2023-11-15T02:57:44.572196+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.106664+0000","last_change":"2023-11-15T02:57:44.097768+0000","last_active":"2023-11-15T02:57:44.106664+0000","last_peered":"2023-11-15T02:57:44.106664+0000","last_clean":"2023-11-15T02:57:44.106664+0000","last_became_active":"2023-11-15T02:57:44.097449+0000","last_became_peered":"2023-11-15T02:57:44.097449+0000","last_unstale":"2023-11-15T02:57:44.106664+0000","last_undegraded":"2023-11-15T02:57:44.106664+0000","last_fullsized":"2023-11-15T02:57:44.106664+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.588091+0000","last_change":"2023-11-15T02:57:44.588091+0000","last_active":"2023-11-15T02:57:44.588091+0000","last_peered":"2023-11-15T02:57:44.588091+0000","last_clean":"2023-11-15T02:57:44.588091+0000","last_became_active":"2023-11-15T02:57:44.587721+0000","last_became_peered":"2023-11-15T02:57:44.587721+0000","last_unstale":"2023-11-15T02:57:44.588091+0000","last_undegraded":"2023-11-15T02:57:44.588091+0000","last_fullsized":"2023-11-15T02:57:44.588091+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2d","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.551810+0000","last_change":"2023-11-15T02:57:44.097948+0000","last_active":"2023-11-15T02:57:44.551810+0000","last_peered":"2023-11-15T02:57:44.551810+0000","last_clean":"2023-11-15T02:57:44.551810+0000","last_became_active":"2023-11-15T02:57:44.097642+0000","last_became_peered":"2023-11-15T02:57:44.097642+0000","last_unstale":"2023-11-15T02:57:44.551810+0000","last_undegraded":"2023-11-15T02:57:44.551810+0000","last_fullsized":"2023-11-15T02:57:44.551810+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582841+0000","last_change":"2023-11-15T02:57:44.582841+0000","last_active":"2023-11-15T02:57:44.582841+0000","last_peered":"2023-11-15T02:57:44.582841+0000","last_clean":"2023-11-15T02:57:44.582841+0000","last_became_active":"2023-11-15T02:57:44.582592+0000","last_became_peered":"2023-11-15T02:57:44.582592+0000","last_unstale":"2023-11-15T02:57:44.582841+0000","last_undegraded":"2023-11-15T02:57:44.582841+0000","last_fullsized":"2023-11-15T02:57:44.582841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.101447+0000","last_change":"2023-11-15T02:57:44.095430+0000","last_active":"2023-11-15T02:57:44.101447+0000","last_peered":"2023-11-15T02:57:44.101447+0000","last_clean":"2023-11-15T02:57:44.101447+0000","last_became_active":"2023-11-15T02:57:44.094060+0000","last_became_peered":"2023-11-15T02:57:44.094060+0000","last_unstale":"2023-11-15T02:57:44.101447+0000","last_undegraded":"2023-11-15T02:57:44.101447+0000","last_fullsized":"2023-11-15T02:57:44.101447+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.571866+0000","last_change":"2023-11-15T02:57:44.571866+0000","last_active":"2023-11-15T02:57:44.571866+0000","last_peered":"2023-11-15T02:57:44.571866+0000","last_clean":"2023-11-15T02:57:44.571866+0000","last_became_active":"2023-11-15T02:57:44.571594+0000","last_became_peered":"2023-11-15T02:57:44.571594+0000","last_unstale":"2023-11-15T02:57:44.571866+0000","last_undegraded":"2023-11-15T02:57:44.571866+0000","last_fullsized":"2023-11-15T02:57:44.571866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.561370+0000","last_change":"2023-11-15T02:57:44.111471+0000","last_active":"2023-11-15T02:57:44.561370+0000","last_peered":"2023-11-15T02:57:44.561370+0000","last_clean":"2023-11-15T02:57:44.561370+0000","last_became_active":"2023-11-15T02:57:44.110158+0000","last_became_peered":"2023-11-15T02:57:44.110158+0000","last_unstale":"2023-11-15T02:57:44.561370+0000","last_undegraded":"2023-11-15T02:57:44.561370+0000","last_fullsized":"2023-11-15T02:57:44.561370+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.589275+0000","last_change":"2023-11-15T02:57:44.589275+0000","last_active":"2023-11-15T02:57:44.589275+0000","last_peered":"2023-11-15T02:57:44.589275+0000","last_clean":"2023-11-15T02:57:44.589275+0000","last_became_active":"2023-11-15T02:57:44.588964+0000","last_became_peered":"2023-11-15T02:57:44.588964+0000","last_unstale":"2023-11-15T02:57:44.589275+0000","last_undegraded":"2023-11-15T02:57:44.589275+0000","last_fullsized":"2023-11-15T02:57:44.589275+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.103617+0000","last_change":"2023-11-15T02:57:44.098842+0000","last_active":"2023-11-15T02:57:44.103617+0000","last_peered":"2023-11-15T02:57:44.103617+0000","last_clean":"2023-11-15T02:57:44.103617+0000","last_became_active":"2023-11-15T02:57:44.098539+0000","last_became_peered":"2023-11-15T02:57:44.098539+0000","last_unstale":"2023-11-15T02:57:44.103617+0000","last_undegraded":"2023-11-15T02:57:44.103617+0000","last_fullsized":"2023-11-15T02:57:44.103617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.571505+0000","last_change":"2023-11-15T02:57:44.571505+0000","last_active":"2023-11-15T02:57:44.571505+0000","last_peered":"2023-11-15T02:57:44.571505+0000","last_clean":"2023-11-15T02:57:44.571505+0000","last_became_active":"2023-11-15T02:57:44.570032+0000","last_became_peered":"2023-11-15T02:57:44.570032+0000","last_unstale":"2023-11-15T02:57:44.571505+0000","last_undegraded":"2023-11-15T02:57:44.571505+0000","last_fullsized":"2023-11-15T02:57:44.571505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.112480+0000","last_change":"2023-11-15T02:57:44.102777+0000","last_active":"2023-11-15T02:57:44.112480+0000","last_peered":"2023-11-15T02:57:44.112480+0000","last_clean":"2023-11-15T02:57:44.112480+0000","last_became_active":"2023-11-15T02:57:44.102485+0000","last_became_peered":"2023-11-15T02:57:44.102485+0000","last_unstale":"2023-11-15T02:57:44.112480+0000","last_undegraded":"2023-11-15T02:57:44.112480+0000","last_fullsized":"2023-11-15T02:57:44.112480+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.076093+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.100059+0000","last_change":"2023-11-15T02:57:44.100059+0000","last_active":"2023-11-15T02:57:44.100059+0000","last_peered":"2023-11-15T02:57:44.100059+0000","last_clean":"2023-11-15T02:57:44.100059+0000","last_became_active":"2023-11-15T02:57:44.099755+0000","last_became_peered":"2023-11-15T02:57:44.099755+0000","last_unstale":"2023-11-15T02:57:44.100059+0000","last_undegraded":"2023-11-15T02:57:44.100059+0000","last_fullsized":"2023-11-15T02:57:44.100059+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.076365+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.063419+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577168+0000","last_change":"2023-11-15T02:57:44.577168+0000","last_active":"2023-11-15T02:57:44.577168+0000","last_peered":"2023-11-15T02:57:44.577168+0000","last_clean":"2023-11-15T02:57:44.577168+0000","last_became_active":"2023-11-15T02:57:44.576902+0000","last_became_peered":"2023-11-15T02:57:44.576902+0000","last_unstale":"2023-11-15T02:57:44.577168+0000","last_undegraded":"2023-11-15T02:57:44.577168+0000","last_fullsized":"2023-11-15T02:57:44.577168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.106529+0000","last_change":"2023-11-15T02:57:44.097229+0000","last_active":"2023-11-15T02:57:44.106529+0000","last_peered":"2023-11-15T02:57:44.106529+0000","last_clean":"2023-11-15T02:57:44.106529+0000","last_became_active":"2023-11-15T02:57:44.096765+0000","last_became_peered":"2023-11-15T02:57:44.096765+0000","last_unstale":"2023-11-15T02:57:44.106529+0000","last_undegraded":"2023-11-15T02:57:44.106529+0000","last_fullsized":"2023-11-15T02:57:44.106529+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.054985+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.063627+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.576398+0000","last_change":"2023-11-15T02:57:44.576398+0000","last_active":"2023-11-15T02:57:44.576398+0000","last_peered":"2023-11-15T02:57:44.576398+0000","last_clean":"2023-11-15T02:57:44.576398+0000","last_became_active":"2023-11-15T02:57:44.576039+0000","last_became_peered":"2023-11-15T02:57:44.576039+0000","last_unstale":"2023-11-15T02:57:44.576398+0000","last_undegraded":"2023-11-15T02:57:44.576398+0000","last_fullsized":"2023-11-15T02:57:44.576398+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.098514+0000","last_change":"2023-11-15T02:57:44.098514+0000","last_active":"2023-11-15T02:57:44.098514+0000","last_peered":"2023-11-15T02:57:44.098514+0000","last_clean":"2023-11-15T02:57:44.098514+0000","last_became_active":"2023-11-15T02:57:44.098120+0000","last_became_peered":"2023-11-15T02:57:44.098120+0000","last_unstale":"2023-11-15T02:57:44.098514+0000","last_undegraded":"2023-11-15T02:57:44.098514+0000","last_fullsized":"2023-11-15T02:57:44.098514+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.054985+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.108729+0000","last_change":"2023-11-15T02:57:44.101472+0000","last_active":"2023-11-15T02:57:44.108729+0000","last_peered":"2023-11-15T02:57:44.108729+0000","last_clean":"2023-11-15T02:57:44.108729+0000","last_became_active":"2023-11-15T02:57:44.101190+0000","last_became_peered":"2023-11-15T02:57:44.101190+0000","last_unstale":"2023-11-15T02:57:44.108729+0000","last_undegraded":"2023-11-15T02:57:44.108729+0000","last_fullsized":"2023-11-15T02:57:44.108729+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.589081+0000","last_change":"2023-11-15T02:57:44.589081+0000","last_active":"2023-11-15T02:57:44.589081+0000","last_peered":"2023-11-15T02:57:44.589081+0000","last_clean":"2023-11-15T02:57:44.589081+0000","last_became_active":"2023-11-15T02:57:44.588602+0000","last_became_peered":"2023-11-15T02:57:44.588602+0000","last_unstale":"2023-11-15T02:57:44.589081+0000","last_undegraded":"2023-11-15T02:57:44.589081+0000","last_fullsized":"2023-11-15T02:57:44.589081+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.108688+0000","last_change":"2023-11-15T02:57:44.101411+0000","last_active":"2023-11-15T02:57:44.108688+0000","last_peered":"2023-11-15T02:57:44.108688+0000","last_clean":"2023-11-15T02:57:44.108688+0000","last_became_active":"2023-11-15T02:57:44.099981+0000","last_became_peered":"2023-11-15T02:57:44.099981+0000","last_unstale":"2023-11-15T02:57:44.108688+0000","last_undegraded":"2023-11-15T02:57:44.108688+0000","last_fullsized":"2023-11-15T02:57:44.108688+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.588317+0000","last_change":"2023-11-15T02:57:44.588317+0000","last_active":"2023-11-15T02:57:44.588317+0000","last_peered":"2023-11-15T02:57:44.588317+0000","last_clean":"2023-11-15T02:57:44.588317+0000","last_became_active":"2023-11-15T02:57:44.586643+0000","last_became_peered":"2023-11-15T02:57:44.586643+0000","last_unstale":"2023-11-15T02:57:44.588317+0000","last_undegraded":"2023-11-15T02:57:44.588317+0000","last_fullsized":"2023-11-15T02:57:44.588317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.104386+0000","last_change":"2023-11-15T02:57:44.097212+0000","last_active":"2023-11-15T02:57:44.104386+0000","last_peered":"2023-11-15T02:57:44.104386+0000","last_clean":"2023-11-15T02:57:44.104386+0000","last_became_active":"2023-11-15T02:57:44.096903+0000","last_became_peered":"2023-11-15T02:57:44.096903+0000","last_unstale":"2023-11-15T02:57:44.104386+0000","last_undegraded":"2023-11-15T02:57:44.104386+0000","last_fullsized":"2023-11-15T02:57:44.104386+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.578257+0000","last_change":"2023-11-15T02:57:44.578257+0000","last_active":"2023-11-15T02:57:44.578257+0000","last_peered":"2023-11-15T02:57:44.578257+0000","last_clean":"2023-11-15T02:57:44.578257+0000","last_became_active":"2023-11-15T02:57:44.571560+0000","last_became_peered":"2023-11-15T02:57:44.571560+0000","last_unstale":"2023-11-15T02:57:44.578257+0000","last_undegraded":"2023-11-15T02:57:44.578257+0000","last_fullsized":"2023-11-15T02:57:44.578257+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.108660+0000","last_change":"2023-11-15T02:57:44.102174+0000","last_active":"2023-11-15T02:57:44.108660+0000","last_peered":"2023-11-15T02:57:44.108660+0000","last_clean":"2023-11-15T02:57:44.108660+0000","last_became_active":"2023-11-15T02:57:44.101794+0000","last_became_peered":"2023-11-15T02:57:44.101794+0000","last_unstale":"2023-11-15T02:57:44.108660+0000","last_undegraded":"2023-11-15T02:57:44.108660+0000","last_fullsized":"2023-11-15T02:57:44.108660+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.587132+0000","last_change":"2023-11-15T02:57:44.587132+0000","last_active":"2023-11-15T02:57:44.587132+0000","last_peered":"2023-11-15T02:57:44.587132+0000","last_clean":"2023-11-15T02:57:44.587132+0000","last_became_active":"2023-11-15T02:57:44.586820+0000","last_became_peered":"2023-11-15T02:57:44.586820+0000","last_unstale":"2023-11-15T02:57:44.587132+0000","last_undegraded":"2023-11-15T02:57:44.587132+0000","last_fullsized":"2023-11-15T02:57:44.587132+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.574296+0000","last_change":"2023-11-15T02:57:44.102584+0000","last_active":"2023-11-15T02:57:44.574296+0000","last_peered":"2023-11-15T02:57:44.574296+0000","last_clean":"2023-11-15T02:57:44.574296+0000","last_became_active":"2023-11-15T02:57:44.102267+0000","last_became_peered":"2023-11-15T02:57:44.102267+0000","last_unstale":"2023-11-15T02:57:44.574296+0000","last_undegraded":"2023-11-15T02:57:44.574296+0000","last_fullsized":"2023-11-15T02:57:44.574296+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.054985+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.561164+0000","last_change":"2023-11-15T02:57:44.102680+0000","last_active":"2023-11-15T02:57:44.561164+0000","last_peered":"2023-11-15T02:57:44.561164+0000","last_clean":"2023-11-15T02:57:44.561164+0000","last_became_active":"2023-11-15T02:57:44.102394+0000","last_became_peered":"2023-11-15T02:57:44.102394+0000","last_unstale":"2023-11-15T02:57:44.561164+0000","last_undegraded":"2023-11-15T02:57:44.561164+0000","last_fullsized":"2023-11-15T02:57:44.561164+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.586592+0000","last_change":"2023-11-15T02:57:44.586592+0000","last_active":"2023-11-15T02:57:44.586592+0000","last_peered":"2023-11-15T02:57:44.586592+0000","last_clean":"2023-11-15T02:57:44.586592+0000","last_became_active":"2023-11-15T02:57:44.586211+0000","last_became_peered":"2023-11-15T02:57:44.586211+0000","last_unstale":"2023-11-15T02:57:44.586592+0000","last_undegraded":"2023-11-15T02:57:44.586592+0000","last_fullsized":"2023-11-15T02:57:44.586592+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.063052+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.579124+0000","last_change":"2023-11-15T02:57:44.579124+0000","last_active":"2023-11-15T02:57:44.579124+0000","last_peered":"2023-11-15T02:57:44.579124+0000","last_clean":"2023-11-15T02:57:44.579124+0000","last_became_active":"2023-11-15T02:57:44.578824+0000","last_became_peered":"2023-11-15T02:57:44.578824+0000","last_unstale":"2023-11-15T02:57:44.579124+0000","last_undegraded":"2023-11-15T02:57:44.579124+0000","last_fullsized":"2023-11-15T02:57:44.579124+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.108976+0000","last_change":"2023-11-15T02:57:44.101425+0000","last_active":"2023-11-15T02:57:44.108976+0000","last_peered":"2023-11-15T02:57:44.108976+0000","last_clean":"2023-11-15T02:57:44.108976+0000","last_became_active":"2023-11-15T02:57:44.101139+0000","last_became_peered":"2023-11-15T02:57:44.101139+0000","last_unstale":"2023-11-15T02:57:44.108976+0000","last_undegraded":"2023-11-15T02:57:44.108976+0000","last_fullsized":"2023-11-15T02:57:44.108976+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.574408+0000","last_change":"2023-11-15T02:57:44.574408+0000","last_active":"2023-11-15T02:57:44.574408+0000","last_peered":"2023-11-15T02:57:44.574408+0000","last_clean":"2023-11-15T02:57:44.574408+0000","last_became_active":"2023-11-15T02:57:44.573651+0000","last_became_peered":"2023-11-15T02:57:44.573651+0000","last_unstale":"2023-11-15T02:57:44.574408+0000","last_undegraded":"2023-11-15T02:57:44.574408+0000","last_fullsized":"2023-11-15T02:57:44.574408+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.112809+0000","last_change":"2023-11-15T02:57:44.112809+0000","last_active":"2023-11-15T02:57:44.112809+0000","last_peered":"2023-11-15T02:57:44.112809+0000","last_clean":"2023-11-15T02:57:44.112809+0000","last_became_active":"2023-11-15T02:57:44.111134+0000","last_became_peered":"2023-11-15T02:57:44.111134+0000","last_unstale":"2023-11-15T02:57:44.112809+0000","last_undegraded":"2023-11-15T02:57:44.112809+0000","last_fullsized":"2023-11-15T02:57:44.112809+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.076060+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.104280+0000","last_change":"2023-11-15T02:57:44.099996+0000","last_active":"2023-11-15T02:57:44.104280+0000","last_peered":"2023-11-15T02:57:44.104280+0000","last_clean":"2023-11-15T02:57:44.104280+0000","last_became_active":"2023-11-15T02:57:44.099695+0000","last_became_peered":"2023-11-15T02:57:44.099695+0000","last_unstale":"2023-11-15T02:57:44.104280+0000","last_undegraded":"2023-11-15T02:57:44.104280+0000","last_fullsized":"2023-11-15T02:57:44.104280+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577040+0000","last_change":"2023-11-15T02:57:44.577040+0000","last_active":"2023-11-15T02:57:44.577040+0000","last_peered":"2023-11-15T02:57:44.577040+0000","last_clean":"2023-11-15T02:57:44.577040+0000","last_became_active":"2023-11-15T02:57:44.576592+0000","last_became_peered":"2023-11-15T02:57:44.576592+0000","last_unstale":"2023-11-15T02:57:44.577040+0000","last_undegraded":"2023-11-15T02:57:44.577040+0000","last_fullsized":"2023-11-15T02:57:44.577040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.064186+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.571061+0000","last_change":"2023-11-15T02:57:44.571061+0000","last_active":"2023-11-15T02:57:44.571061+0000","last_peered":"2023-11-15T02:57:44.571061+0000","last_clean":"2023-11-15T02:57:44.571061+0000","last_became_active":"2023-11-15T02:57:44.570802+0000","last_became_peered":"2023-11-15T02:57:44.570802+0000","last_unstale":"2023-11-15T02:57:44.571061+0000","last_undegraded":"2023-11-15T02:57:44.571061+0000","last_fullsized":"2023-11-15T02:57:44.571061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.103937+0000","last_change":"2023-11-15T02:57:44.099371+0000","last_active":"2023-11-15T02:57:44.103937+0000","last_peered":"2023-11-15T02:57:44.103937+0000","last_clean":"2023-11-15T02:57:44.103937+0000","last_became_active":"2023-11-15T02:57:44.096689+0000","last_became_peered":"2023-11-15T02:57:44.096689+0000","last_unstale":"2023-11-15T02:57:44.103937+0000","last_undegraded":"2023-11-15T02:57:44.103937+0000","last_fullsized":"2023-11-15T02:57:44.103937+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.572799+0000","last_change":"2023-11-15T02:57:44.572799+0000","last_active":"2023-11-15T02:57:44.572799+0000","last_peered":"2023-11-15T02:57:44.572799+0000","last_clean":"2023-11-15T02:57:44.572799+0000","last_became_active":"2023-11-15T02:57:44.572147+0000","last_became_peered":"2023-11-15T02:57:44.572147+0000","last_unstale":"2023-11-15T02:57:44.572799+0000","last_undegraded":"2023-11-15T02:57:44.572799+0000","last_fullsized":"2023-11-15T02:57:44.572799+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.105678+0000","last_change":"2023-11-15T02:57:44.094718+0000","last_active":"2023-11-15T02:57:44.105678+0000","last_peered":"2023-11-15T02:57:44.105678+0000","last_clean":"2023-11-15T02:57:44.105678+0000","last_became_active":"2023-11-15T02:57:44.094236+0000","last_became_peered":"2023-11-15T02:57:44.094236+0000","last_unstale":"2023-11-15T02:57:44.105678+0000","last_undegraded":"2023-11-15T02:57:44.105678+0000","last_fullsized":"2023-11-15T02:57:44.105678+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577901+0000","last_change":"2023-11-15T02:57:44.577901+0000","last_active":"2023-11-15T02:57:44.577901+0000","last_peered":"2023-11-15T02:57:44.577901+0000","last_clean":"2023-11-15T02:57:44.577901+0000","last_became_active":"2023-11-15T02:57:44.577370+0000","last_became_peered":"2023-11-15T02:57:44.577370+0000","last_unstale":"2023-11-15T02:57:44.577901+0000","last_undegraded":"2023-11-15T02:57:44.577901+0000","last_fullsized":"2023-11-15T02:57:44.577901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109398+0000","last_change":"2023-11-15T02:57:44.109398+0000","last_active":"2023-11-15T02:57:44.109398+0000","last_peered":"2023-11-15T02:57:44.109398+0000","last_clean":"2023-11-15T02:57:44.109398+0000","last_became_active":"2023-11-15T02:57:44.104880+0000","last_became_peered":"2023-11-15T02:57:44.104880+0000","last_unstale":"2023-11-15T02:57:44.109398+0000","last_undegraded":"2023-11-15T02:57:44.109398+0000","last_fullsized":"2023-11-15T02:57:44.109398+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.570258+0000","last_change":"2023-11-15T02:57:44.570258+0000","last_active":"2023-11-15T02:57:44.570258+0000","last_peered":"2023-11-15T02:57:44.570258+0000","last_clean":"2023-11-15T02:57:44.570258+0000","last_became_active":"2023-11-15T02:57:44.569983+0000","last_became_peered":"2023-11-15T02:57:44.569983+0000","last_unstale":"2023-11-15T02:57:44.570258+0000","last_undegraded":"2023-11-15T02:57:44.570258+0000","last_fullsized":"2023-11-15T02:57:44.570258+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.096037+0000","last_change":"2023-11-15T02:57:44.096037+0000","last_active":"2023-11-15T02:57:44.096037+0000","last_peered":"2023-11-15T02:57:44.096037+0000","last_clean":"2023-11-15T02:57:44.096037+0000","last_became_active":"2023-11-15T02:57:44.095669+0000","last_became_peered":"2023-11-15T02:57:44.095669+0000","last_unstale":"2023-11-15T02:57:44.096037+0000","last_undegraded":"2023-11-15T02:57:44.096037+0000","last_fullsized":"2023-11-15T02:57:44.096037+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585203+0000","last_change":"2023-11-15T02:57:44.585203+0000","last_active":"2023-11-15T02:57:44.585203+0000","last_peered":"2023-11-15T02:57:44.585203+0000","last_clean":"2023-11-15T02:57:44.585203+0000","last_became_active":"2023-11-15T02:57:44.583007+0000","last_became_peered":"2023-11-15T02:57:44.583007+0000","last_unstale":"2023-11-15T02:57:44.585203+0000","last_undegraded":"2023-11-15T02:57:44.585203+0000","last_fullsized":"2023-11-15T02:57:44.585203+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.581336+0000","last_change":"2023-11-15T02:57:44.096764+0000","last_active":"2023-11-15T02:57:44.581336+0000","last_peered":"2023-11-15T02:57:44.581336+0000","last_clean":"2023-11-15T02:57:44.581336+0000","last_became_active":"2023-11-15T02:57:44.096348+0000","last_became_peered":"2023-11-15T02:57:44.096348+0000","last_unstale":"2023-11-15T02:57:44.581336+0000","last_undegraded":"2023-11-15T02:57:44.581336+0000","last_fullsized":"2023-11-15T02:57:44.581336+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566200+0000","last_change":"2023-11-15T02:57:44.566200+0000","last_active":"2023-11-15T02:57:44.566200+0000","last_peered":"2023-11-15T02:57:44.566200+0000","last_clean":"2023-11-15T02:57:44.566200+0000","last_became_active":"2023-11-15T02:57:44.565943+0000","last_became_peered":"2023-11-15T02:57:44.565943+0000","last_unstale":"2023-11-15T02:57:44.566200+0000","last_undegraded":"2023-11-15T02:57:44.566200+0000","last_fullsized":"2023-11-15T02:57:44.566200+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.553334+0000","last_change":"2023-11-15T02:57:44.109346+0000","last_active":"2023-11-15T02:57:44.553334+0000","last_peered":"2023-11-15T02:57:44.553334+0000","last_clean":"2023-11-15T02:57:44.553334+0000","last_became_active":"2023-11-15T02:57:44.109023+0000","last_became_peered":"2023-11-15T02:57:44.109023+0000","last_unstale":"2023-11-15T02:57:44.553334+0000","last_undegraded":"2023-11-15T02:57:44.553334+0000","last_fullsized":"2023-11-15T02:57:44.553334+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566248+0000","last_change":"2023-11-15T02:57:44.566248+0000","last_active":"2023-11-15T02:57:44.566248+0000","last_peered":"2023-11-15T02:57:44.566248+0000","last_clean":"2023-11-15T02:57:44.566248+0000","last_became_active":"2023-11-15T02:57:44.566002+0000","last_became_peered":"2023-11-15T02:57:44.566002+0000","last_unstale":"2023-11-15T02:57:44.566248+0000","last_undegraded":"2023-11-15T02:57:44.566248+0000","last_fullsized":"2023-11-15T02:57:44.566248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.062537+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.588350+0000","last_change":"2023-11-15T02:57:44.588350+0000","last_active":"2023-11-15T02:57:44.588350+0000","last_peered":"2023-11-15T02:57:44.588350+0000","last_clean":"2023-11-15T02:57:44.588350+0000","last_became_active":"2023-11-15T02:57:44.588073+0000","last_became_peered":"2023-11-15T02:57:44.588073+0000","last_unstale":"2023-11-15T02:57:44.588350+0000","last_undegraded":"2023-11-15T02:57:44.588350+0000","last_fullsized":"2023-11-15T02:57:44.588350+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109706+0000","last_change":"2023-11-15T02:57:44.101276+0000","last_active":"2023-11-15T02:57:44.109706+0000","last_peered":"2023-11-15T02:57:44.109706+0000","last_clean":"2023-11-15T02:57:44.109706+0000","last_became_active":"2023-11-15T02:57:44.100908+0000","last_became_peered":"2023-11-15T02:57:44.100908+0000","last_unstale":"2023-11-15T02:57:44.109706+0000","last_undegraded":"2023-11-15T02:57:44.109706+0000","last_fullsized":"2023-11-15T02:57:44.109706+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566177+0000","last_change":"2023-11-15T02:57:44.566177+0000","last_active":"2023-11-15T02:57:44.566177+0000","last_peered":"2023-11-15T02:57:44.566177+0000","last_clean":"2023-11-15T02:57:44.566177+0000","last_became_active":"2023-11-15T02:57:44.565917+0000","last_became_peered":"2023-11-15T02:57:44.565917+0000","last_unstale":"2023-11-15T02:57:44.566177+0000","last_undegraded":"2023-11-15T02:57:44.566177+0000","last_fullsized":"2023-11-15T02:57:44.566177+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"32'4","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.586189+0000","last_change":"2023-11-15T02:57:44.102229+0000","last_active":"2023-11-15T02:57:44.586189+0000","last_peered":"2023-11-15T02:57:44.586189+0000","last_clean":"2023-11-15T02:57:44.586189+0000","last_became_active":"2023-11-15T02:57:44.101858+0000","last_became_peered":"2023-11-15T02:57:44.101858+0000","last_unstale":"2023-11-15T02:57:44.586189+0000","last_undegraded":"2023-11-15T02:57:44.586189+0000","last_fullsized":"2023-11-15T02:57:44.586189+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.578142+0000","last_change":"2023-11-15T02:57:44.578142+0000","last_active":"2023-11-15T02:57:44.578142+0000","last_peered":"2023-11-15T02:57:44.578142+0000","last_clean":"2023-11-15T02:57:44.578142+0000","last_became_active":"2023-11-15T02:57:44.575328+0000","last_became_peered":"2023-11-15T02:57:44.575328+0000","last_unstale":"2023-11-15T02:57:44.578142+0000","last_undegraded":"2023-11-15T02:57:44.578142+0000","last_fullsized":"2023-11-15T02:57:44.578142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.100857+0000","last_change":"2023-11-15T02:57:44.100857+0000","last_active":"2023-11-15T02:57:44.100857+0000","last_peered":"2023-11-15T02:57:44.100857+0000","last_clean":"2023-11-15T02:57:44.100857+0000","last_became_active":"2023-11-15T02:57:44.099959+0000","last_became_peered":"2023-11-15T02:57:44.099959+0000","last_unstale":"2023-11-15T02:57:44.100857+0000","last_undegraded":"2023-11-15T02:57:44.100857+0000","last_fullsized":"2023-11-15T02:57:44.100857+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.077201+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585609+0000","last_change":"2023-11-15T02:57:44.096009+0000","last_active":"2023-11-15T02:57:44.585609+0000","last_peered":"2023-11-15T02:57:44.585609+0000","last_clean":"2023-11-15T02:57:44.585609+0000","last_became_active":"2023-11-15T02:57:44.095541+0000","last_became_peered":"2023-11-15T02:57:44.095541+0000","last_unstale":"2023-11-15T02:57:44.585609+0000","last_undegraded":"2023-11-15T02:57:44.585609+0000","last_fullsized":"2023-11-15T02:57:44.585609+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.583166+0000","last_change":"2023-11-15T02:57:44.583166+0000","last_active":"2023-11-15T02:57:44.583166+0000","last_peered":"2023-11-15T02:57:44.583166+0000","last_clean":"2023-11-15T02:57:44.583166+0000","last_became_active":"2023-11-15T02:57:44.582842+0000","last_became_peered":"2023-11-15T02:57:44.582842+0000","last_unstale":"2023-11-15T02:57:44.583166+0000","last_undegraded":"2023-11-15T02:57:44.583166+0000","last_fullsized":"2023-11-15T02:57:44.583166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109552+0000","last_change":"2023-11-15T02:57:44.101593+0000","last_active":"2023-11-15T02:57:44.109552+0000","last_peered":"2023-11-15T02:57:44.109552+0000","last_clean":"2023-11-15T02:57:44.109552+0000","last_became_active":"2023-11-15T02:57:44.101261+0000","last_became_peered":"2023-11-15T02:57:44.101261+0000","last_unstale":"2023-11-15T02:57:44.109552+0000","last_undegraded":"2023-11-15T02:57:44.109552+0000","last_fullsized":"2023-11-15T02:57:44.109552+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.584677+0000","last_change":"2023-11-15T02:57:44.584677+0000","last_active":"2023-11-15T02:57:44.584677+0000","last_peered":"2023-11-15T02:57:44.584677+0000","last_clean":"2023-11-15T02:57:44.584677+0000","last_became_active":"2023-11-15T02:57:44.584394+0000","last_became_peered":"2023-11-15T02:57:44.584394+0000","last_unstale":"2023-11-15T02:57:44.584677+0000","last_undegraded":"2023-11-15T02:57:44.584677+0000","last_fullsized":"2023-11-15T02:57:44.584677+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.064181+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.590180+0000","last_change":"2023-11-15T02:57:44.590180+0000","last_active":"2023-11-15T02:57:44.590180+0000","last_peered":"2023-11-15T02:57:44.590180+0000","last_clean":"2023-11-15T02:57:44.590180+0000","last_became_active":"2023-11-15T02:57:44.589752+0000","last_became_peered":"2023-11-15T02:57:44.589752+0000","last_unstale":"2023-11-15T02:57:44.590180+0000","last_undegraded":"2023-11-15T02:57:44.590180+0000","last_fullsized":"2023-11-15T02:57:44.590180+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560777+0000","last_change":"2023-11-15T02:57:44.109252+0000","last_active":"2023-11-15T02:57:44.560777+0000","last_peered":"2023-11-15T02:57:44.560777+0000","last_clean":"2023-11-15T02:57:44.560777+0000","last_became_active":"2023-11-15T02:57:44.108907+0000","last_became_peered":"2023-11-15T02:57:44.108907+0000","last_unstale":"2023-11-15T02:57:44.560777+0000","last_undegraded":"2023-11-15T02:57:44.560777+0000","last_fullsized":"2023-11-15T02:57:44.560777+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.568542+0000","last_change":"2023-11-15T02:57:44.568542+0000","last_active":"2023-11-15T02:57:44.568542+0000","last_peered":"2023-11-15T02:57:44.568542+0000","last_clean":"2023-11-15T02:57:44.568542+0000","last_became_active":"2023-11-15T02:57:44.568269+0000","last_became_peered":"2023-11-15T02:57:44.568269+0000","last_unstale":"2023-11-15T02:57:44.568542+0000","last_undegraded":"2023-11-15T02:57:44.568542+0000","last_fullsized":"2023-11-15T02:57:44.568542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.103890+0000","last_change":"2023-11-15T02:57:44.097937+0000","last_active":"2023-11-15T02:57:44.103890+0000","last_peered":"2023-11-15T02:57:44.103890+0000","last_clean":"2023-11-15T02:57:44.103890+0000","last_became_active":"2023-11-15T02:57:44.097630+0000","last_became_peered":"2023-11-15T02:57:44.097630+0000","last_unstale":"2023-11-15T02:57:44.103890+0000","last_undegraded":"2023-11-15T02:57:44.103890+0000","last_fullsized":"2023-11-15T02:57:44.103890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.078891+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.111403+0000","last_change":"2023-11-15T02:57:44.108255+0000","last_active":"2023-11-15T02:57:44.111403+0000","last_peered":"2023-11-15T02:57:44.111403+0000","last_clean":"2023-11-15T02:57:44.111403+0000","last_became_active":"2023-11-15T02:57:44.107961+0000","last_became_peered":"2023-11-15T02:57:44.107961+0000","last_unstale":"2023-11-15T02:57:44.111403+0000","last_undegraded":"2023-11-15T02:57:44.111403+0000","last_fullsized":"2023-11-15T02:57:44.111403+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.570926+0000","last_change":"2023-11-15T02:57:44.570926+0000","last_active":"2023-11-15T02:57:44.570926+0000","last_peered":"2023-11-15T02:57:44.570926+0000","last_clean":"2023-11-15T02:57:44.570926+0000","last_became_active":"2023-11-15T02:57:44.570646+0000","last_became_peered":"2023-11-15T02:57:44.570646+0000","last_unstale":"2023-11-15T02:57:44.570926+0000","last_undegraded":"2023-11-15T02:57:44.570926+0000","last_fullsized":"2023-11-15T02:57:44.570926+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.111257+0000","last_change":"2023-11-15T02:57:44.105564+0000","last_active":"2023-11-15T02:57:44.111257+0000","last_peered":"2023-11-15T02:57:44.111257+0000","last_clean":"2023-11-15T02:57:44.111257+0000","last_became_active":"2023-11-15T02:57:44.105312+0000","last_became_peered":"2023-11-15T02:57:44.105312+0000","last_unstale":"2023-11-15T02:57:44.111257+0000","last_undegraded":"2023-11-15T02:57:44.111257+0000","last_fullsized":"2023-11-15T02:57:44.111257+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.580419+0000","last_change":"2023-11-15T02:57:44.580419+0000","last_active":"2023-11-15T02:57:44.580419+0000","last_peered":"2023-11-15T02:57:44.580419+0000","last_clean":"2023-11-15T02:57:44.580419+0000","last_became_active":"2023-11-15T02:57:44.580057+0000","last_became_peered":"2023-11-15T02:57:44.580057+0000","last_unstale":"2023-11-15T02:57:44.580419+0000","last_undegraded":"2023-11-15T02:57:44.580419+0000","last_fullsized":"2023-11-15T02:57:44.580419+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.107016+0000","last_change":"2023-11-15T02:57:44.097433+0000","last_active":"2023-11-15T02:57:44.107016+0000","last_peered":"2023-11-15T02:57:44.107016+0000","last_clean":"2023-11-15T02:57:44.107016+0000","last_became_active":"2023-11-15T02:57:44.097149+0000","last_became_peered":"2023-11-15T02:57:44.097149+0000","last_unstale":"2023-11-15T02:57:44.107016+0000","last_undegraded":"2023-11-15T02:57:44.107016+0000","last_fullsized":"2023-11-15T02:57:44.107016+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.090441+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T02:57:44.090441+0000","last_peered":"2023-11-15T02:57:44.090441+0000","last_clean":"2023-11-15T02:57:44.090441+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T02:57:44.090441+0000","last_undegraded":"2023-11-15T02:57:44.090441+0000","last_fullsized":"2023-11-15T02:57:44.090441+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.584602+0000","last_change":"2023-11-15T02:57:44.584602+0000","last_active":"2023-11-15T02:57:44.584602+0000","last_peered":"2023-11-15T02:57:44.584602+0000","last_clean":"2023-11-15T02:57:44.584602+0000","last_became_active":"2023-11-15T02:57:44.584333+0000","last_became_peered":"2023-11-15T02:57:44.584333+0000","last_unstale":"2023-11-15T02:57:44.584602+0000","last_undegraded":"2023-11-15T02:57:44.584602+0000","last_fullsized":"2023-11-15T02:57:44.584602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.104126+0000","last_change":"2023-11-15T02:57:44.099697+0000","last_active":"2023-11-15T02:57:44.104126+0000","last_peered":"2023-11-15T02:57:44.104126+0000","last_clean":"2023-11-15T02:57:44.104126+0000","last_became_active":"2023-11-15T02:57:44.099388+0000","last_became_peered":"2023-11-15T02:57:44.099388+0000","last_unstale":"2023-11-15T02:57:44.104126+0000","last_undegraded":"2023-11-15T02:57:44.104126+0000","last_fullsized":"2023-11-15T02:57:44.104126+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-11-15T02:57:38.672457+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T02:57:38.672457+0000","last_peered":"2023-11-15T02:57:38.672457+0000","last_clean":"2023-11-15T02:57:38.672457+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T02:57:38.672457+0000","last_undegraded":"2023-11-15T02:57:38.672457+0000","last_fullsized":"2023-11-15T02:57:38.672457+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.054985+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.105522+0000","last_change":"2023-11-15T02:57:44.097789+0000","last_active":"2023-11-15T02:57:44.105522+0000","last_peered":"2023-11-15T02:57:44.105522+0000","last_clean":"2023-11-15T02:57:44.105522+0000","last_became_active":"2023-11-15T02:57:44.097133+0000","last_became_peered":"2023-11-15T02:57:44.097133+0000","last_unstale":"2023-11-15T02:57:44.105522+0000","last_undegraded":"2023-11-15T02:57:44.105522+0000","last_fullsized":"2023-11-15T02:57:44.105522+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.090466+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T02:57:44.090466+0000","last_peered":"2023-11-15T02:57:44.090466+0000","last_clean":"2023-11-15T02:57:44.090466+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T02:57:44.090466+0000","last_undegraded":"2023-11-15T02:57:44.090466+0000","last_fullsized":"2023-11-15T02:57:44.090466+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582488+0000","last_change":"2023-11-15T02:57:44.582488+0000","last_active":"2023-11-15T02:57:44.582488+0000","last_peered":"2023-11-15T02:57:44.582488+0000","last_clean":"2023-11-15T02:57:44.582488+0000","last_became_active":"2023-11-15T02:57:44.582160+0000","last_became_peered":"2023-11-15T02:57:44.582160+0000","last_unstale":"2023-11-15T02:57:44.582488+0000","last_undegraded":"2023-11-15T02:57:44.582488+0000","last_fullsized":"2023-11-15T02:57:44.582488+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.113534+0000","last_change":"2023-11-15T02:57:44.110737+0000","last_active":"2023-11-15T02:57:44.113534+0000","last_peered":"2023-11-15T02:57:44.113534+0000","last_clean":"2023-11-15T02:57:44.113534+0000","last_became_active":"2023-11-15T02:57:44.110423+0000","last_became_peered":"2023-11-15T02:57:44.110423+0000","last_unstale":"2023-11-15T02:57:44.113534+0000","last_undegraded":"2023-11-15T02:57:44.113534+0000","last_fullsized":"2023-11-15T02:57:44.113534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-11-15T02:57:38.672509+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T02:57:38.672509+0000","last_peered":"2023-11-15T02:57:38.672509+0000","last_clean":"2023-11-15T02:57:38.672509+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T02:57:38.672509+0000","last_undegraded":"2023-11-15T02:57:38.672509+0000","last_fullsized":"2023-11-15T02:57:38.672509+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577142+0000","last_change":"2023-11-15T02:57:44.577142+0000","last_active":"2023-11-15T02:57:44.577142+0000","last_peered":"2023-11-15T02:57:44.577142+0000","last_clean":"2023-11-15T02:57:44.577142+0000","last_became_active":"2023-11-15T02:57:44.576883+0000","last_became_peered":"2023-11-15T02:57:44.576883+0000","last_unstale":"2023-11-15T02:57:44.577142+0000","last_undegraded":"2023-11-15T02:57:44.577142+0000","last_fullsized":"2023-11-15T02:57:44.577142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.113548+0000","last_change":"2023-11-15T02:57:44.111033+0000","last_active":"2023-11-15T02:57:44.113548+0000","last_peered":"2023-11-15T02:57:44.113548+0000","last_clean":"2023-11-15T02:57:44.113548+0000","last_became_active":"2023-11-15T02:57:44.109168+0000","last_became_peered":"2023-11-15T02:57:44.109168+0000","last_unstale":"2023-11-15T02:57:44.113548+0000","last_undegraded":"2023-11-15T02:57:44.113548+0000","last_fullsized":"2023-11-15T02:57:44.113548+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-11-15T02:57:43.068175+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T02:57:43.068175+0000","last_peered":"2023-11-15T02:57:43.068175+0000","last_clean":"2023-11-15T02:57:43.068175+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T02:57:43.068175+0000","last_undegraded":"2023-11-15T02:57:43.068175+0000","last_fullsized":"2023-11-15T02:57:43.068175+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577944+0000","last_change":"2023-11-15T02:57:44.577944+0000","last_active":"2023-11-15T02:57:44.577944+0000","last_peered":"2023-11-15T02:57:44.577944+0000","last_clean":"2023-11-15T02:57:44.577944+0000","last_became_active":"2023-11-15T02:57:44.577738+0000","last_became_peered":"2023-11-15T02:57:44.577738+0000","last_unstale":"2023-11-15T02:57:44.577944+0000","last_undegraded":"2023-11-15T02:57:44.577944+0000","last_fullsized":"2023-11-15T02:57:44.577944+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.105276+0000","last_change":"2023-11-15T02:57:44.098264+0000","last_active":"2023-11-15T02:57:44.105276+0000","last_peered":"2023-11-15T02:57:44.105276+0000","last_clean":"2023-11-15T02:57:44.105276+0000","last_became_active":"2023-11-15T02:57:44.095009+0000","last_became_peered":"2023-11-15T02:57:44.095009+0000","last_unstale":"2023-11-15T02:57:44.105276+0000","last_undegraded":"2023-11-15T02:57:44.105276+0000","last_fullsized":"2023-11-15T02:57:44.105276+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.078017+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T02:57:44.078017+0000","last_peered":"2023-11-15T02:57:44.078017+0000","last_clean":"2023-11-15T02:57:44.078017+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T02:57:44.078017+0000","last_undegraded":"2023-11-15T02:57:44.078017+0000","last_fullsized":"2023-11-15T02:57:44.078017+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.054985+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.095125+0000","last_change":"2023-11-15T02:57:44.095125+0000","last_active":"2023-11-15T02:57:44.095125+0000","last_peered":"2023-11-15T02:57:44.095125+0000","last_clean":"2023-11-15T02:57:44.095125+0000","last_became_active":"2023-11-15T02:57:44.094525+0000","last_became_peered":"2023-11-15T02:57:44.094525+0000","last_unstale":"2023-11-15T02:57:44.095125+0000","last_undegraded":"2023-11-15T02:57:44.095125+0000","last_fullsized":"2023-11-15T02:57:44.095125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.088566+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T02:57:44.088566+0000","last_peered":"2023-11-15T02:57:44.088566+0000","last_clean":"2023-11-15T02:57:44.088566+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T02:57:44.088566+0000","last_undegraded":"2023-11-15T02:57:44.088566+0000","last_fullsized":"2023-11-15T02:57:44.088566+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.077301+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.112596+0000","last_change":"2023-11-15T02:57:44.111330+0000","last_active":"2023-11-15T02:57:44.112596+0000","last_peered":"2023-11-15T02:57:44.112596+0000","last_clean":"2023-11-15T02:57:44.112596+0000","last_became_active":"2023-11-15T02:57:44.111040+0000","last_became_peered":"2023-11-15T02:57:44.111040+0000","last_unstale":"2023-11-15T02:57:44.112596+0000","last_undegraded":"2023-11-15T02:57:44.112596+0000","last_fullsized":"2023-11-15T02:57:44.112596+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-11-15T02:57:43.068311+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T02:57:43.068311+0000","last_peered":"2023-11-15T02:57:43.068311+0000","last_clean":"2023-11-15T02:57:43.068311+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T02:57:43.068311+0000","last_undegraded":"2023-11-15T02:57:43.068311+0000","last_fullsized":"2023-11-15T02:57:43.068311+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.088937+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T02:57:44.088937+0000","last_peered":"2023-11-15T02:57:44.088937+0000","last_clean":"2023-11-15T02:57:44.088937+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T02:57:44.088937+0000","last_undegraded":"2023-11-15T02:57:44.088937+0000","last_fullsized":"2023-11-15T02:57:44.088937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":1302,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297432,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074408,"statfs":{"total":96636764160,"available":96332193792,"internally_reserved":0,"allocated":344064,"data_stored":138527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4970000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.369}]}]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":360448,"data_stored":146260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":360448,"data_stored":145728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297432,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074408,"statfs":{"total":96636764160,"available":96332193792,"internally_reserved":0,"allocated":344064,"data_stored":138527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.512}]}]},{"osd":2,"up_from":14,"seq":60129542159,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297428,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074412,"statfs":{"total":96636764160,"available":96332197888,"internally_reserved":0,"allocated":339968,"data_stored":137258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]}]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297432,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074408,"statfs":{"total":96636764160,"available":96332193792,"internally_reserved":0,"allocated":344064,"data_stored":138527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":1.0549999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.2839999999999998,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.81799999999999995}]},{"osd":3,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":3.54,"5min":3.54,"15min":3.54},"last":1.093}]},{"osd":4,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.85599999999999998}]},{"osd":5,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":3.6760000000000002,"5min":3.6760000000000002,"15min":3.6760000000000002},"last":1.149}]},{"osd":6,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":3.71,"5min":3.71,"15min":3.71},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":1.024,"5min":1.024,"15min":1.024},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.165}]},{"osd":7,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":1.032,"5min":1.032,"15min":1.032},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.238},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.2090000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297420,"kb_used_data":324,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074420,"statfs":{"total":96636764160,"available":96332206080,"internally_reserved":0,"allocated":331776,"data_stored":131848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3370000000000002}]},{"osd":1,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":3.6349999999999998,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":3.6349999999999998},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.516,"5min":3.516,"15min":3.516},"last":3.516}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.4180000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.141}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3879999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.2309999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3660000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297408,"kb_used_data":312,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074432,"statfs":{"total":96636764160,"available":96332218368,"internally_reserved":0,"allocated":319488,"data_stored":125627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T02:57:49.262 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T02:57:49.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.451+0000 7f0e9649b700 1 Processor -- start 2023-11-15T02:57:49.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.455+0000 7f0e9649b700 1 -- start start 2023-11-15T02:57:49.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.455+0000 7f0e9649b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0e90154ca0 0x7f0e90155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.455+0000 7f0e9649b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e900c4c30 con 0x7f0e901562f0 2023-11-15T02:57:49.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.455+0000 7f0e9649b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e900a6f50 con 0x7f0e900a57c0 2023-11-15T02:57:49.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.455+0000 7f0e9649b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e900a8280 con 0x7f0e90154ca0 2023-11-15T02:57:49.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.455+0000 7f0e8ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0e90154ca0 0x7f0e90155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.455+0000 7f0e8ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0e90154ca0 0x7f0e90155080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56446/0 (socket says 172.21.15.121:56446) 2023-11-15T02:57:49.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.455+0000 7f0e8ffff700 1 -- 172.21.15.121:0/2292080235 learned_addr learned my addr 172.21.15.121:0/2292080235 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:49.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.455+0000 7f0e8ffff700 1 -- 172.21.15.121:0/2292080235 >> v1:172.21.15.190:6789/0 conn(0x7f0e900a57c0 legacy=0x7f0e900c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:49.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.455+0000 7f0e8ffff700 1 -- 172.21.15.121:0/2292080235 >> v1:172.21.15.121:6789/0 conn(0x7f0e901562f0 legacy=0x7f0e900a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:49.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.455+0000 7f0e8ffff700 1 -- 172.21.15.121:0/2292080235 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e900a7230 con 0x7f0e90154ca0 2023-11-15T02:57:49.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.455+0000 7f0e8ffff700 1 --2- 172.21.15.121:0/2292080235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e90154ca0 0x7f0e90155080 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f0e80009f20 tx=0x7f0e80009840).ready entity=mon.0 client_cookie=4cf5e36592720161 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:49.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.459+0000 7f0e8f7fe700 1 -- 172.21.15.121:0/2292080235 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8000e070 con 0x7f0e90154ca0 2023-11-15T02:57:49.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.459+0000 7f0e8f7fe700 1 -- 172.21.15.121:0/2292080235 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8000ba40 con 0x7f0e90154ca0 2023-11-15T02:57:49.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.459+0000 7f0e8f7fe700 1 -- 172.21.15.121:0/2292080235 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8000bdc0 con 0x7f0e90154ca0 2023-11-15T02:57:49.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.459+0000 7f0e9649b700 1 -- 172.21.15.121:0/2292080235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e90154ca0 msgr2=0x7f0e90155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:49.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.459+0000 7f0e9649b700 1 --2- 172.21.15.121:0/2292080235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e90154ca0 0x7f0e90155080 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f0e80009f20 tx=0x7f0e80009840).stop 2023-11-15T02:57:49.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.459+0000 7f0e9649b700 1 -- 172.21.15.121:0/2292080235 shutdown_connections 2023-11-15T02:57:49.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.459+0000 7f0e9649b700 1 --2- 172.21.15.121:0/2292080235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e90154ca0 0x7f0e90155080 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.459+0000 7f0e9649b700 1 -- 172.21.15.121:0/2292080235 >> 172.21.15.121:0/2292080235 conn(0x7f0e90062190 msgr2=0x7f0e900625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:49.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.459+0000 7f0e9649b700 1 -- 172.21.15.121:0/2292080235 shutdown_connections 2023-11-15T02:57:49.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.459+0000 7f0e9649b700 1 -- 172.21.15.121:0/2292080235 wait complete. 2023-11-15T02:57:49.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.459+0000 7f0e9649b700 1 Processor -- start 2023-11-15T02:57:49.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.459+0000 7f0e9649b700 1 -- start start 2023-11-15T02:57:49.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e9649b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e900a57c0 0x7f0e90143040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e9649b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e90154ca0 0x7f0e9014efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e9649b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e901562f0 0x7f0e9014f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e9649b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9014f9c0 con 0x7f0e90154ca0 2023-11-15T02:57:49.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e9649b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e900bc530 con 0x7f0e901562f0 2023-11-15T02:57:49.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e9649b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e900bb920 con 0x7f0e900a57c0 2023-11-15T02:57:49.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e877fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e90154ca0 0x7f0e9014efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e877fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e90154ca0 0x7f0e9014efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56456/0 (socket says 172.21.15.121:56456) 2023-11-15T02:57:49.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e877fe700 1 -- 172.21.15.121:0/2426576298 learned_addr learned my addr 172.21.15.121:0/2426576298 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:49.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e8ffff700 1 --2- 172.21.15.121:0/2426576298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e900a57c0 0x7f0e90143040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e94a38700 1 --2- 172.21.15.121:0/2426576298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e901562f0 0x7f0e9014f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e877fe700 1 -- 172.21.15.121:0/2426576298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e900a57c0 msgr2=0x7f0e90143040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:49.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e877fe700 1 --2- 172.21.15.121:0/2426576298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e900a57c0 0x7f0e90143040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e877fe700 1 -- 172.21.15.121:0/2426576298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e901562f0 msgr2=0x7f0e9014f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:49.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e877fe700 1 --2- 172.21.15.121:0/2426576298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e901562f0 0x7f0e9014f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e877fe700 1 -- 172.21.15.121:0/2426576298 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e80009ae0 con 0x7f0e90154ca0 2023-11-15T02:57:49.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e877fe700 1 --2- 172.21.15.121:0/2426576298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e90154ca0 0x7f0e9014efe0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f0e7800c440 tx=0x7f0e78007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:49.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e8cff9700 1 -- 172.21.15.121:0/2426576298 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e78007020 con 0x7f0e90154ca0 2023-11-15T02:57:49.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e8cff9700 1 -- 172.21.15.121:0/2426576298 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e78007180 con 0x7f0e90154ca0 2023-11-15T02:57:49.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e9649b700 1 -- 172.21.15.121:0/2426576298 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e900bbc00 con 0x7f0e90154ca0 2023-11-15T02:57:49.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e8cff9700 1 -- 172.21.15.121:0/2426576298 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e78007520 con 0x7f0e90154ca0 2023-11-15T02:57:49.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.463+0000 7f0e9649b700 1 -- 172.21.15.121:0/2426576298 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e900bc1c0 con 0x7f0e90154ca0 2023-11-15T02:57:49.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.467+0000 7f0e9649b700 1 -- 172.21.15.121:0/2426576298 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e9006c150 con 0x7f0e90154ca0 2023-11-15T02:57:49.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.467+0000 7f0e8cff9700 1 -- 172.21.15.121:0/2426576298 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0e780068b0 con 0x7f0e90154ca0 2023-11-15T02:57:49.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.467+0000 7f0e8cff9700 1 --2- 172.21.15.121:0/2426576298 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e7005a540 0x7f0e7005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.467+0000 7f0e8cff9700 1 -- 172.21.15.121:0/2426576298 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0e78074320 con 0x7f0e90154ca0 2023-11-15T02:57:49.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.467+0000 7f0e8ffff700 1 --2- 172.21.15.121:0/2426576298 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e7005a540 0x7f0e7005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.471+0000 7f0e8ffff700 1 --2- 172.21.15.121:0/2426576298 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e7005a540 0x7f0e7005ca00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f0e8000ba10 tx=0x7f0e80009840).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:49.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.475+0000 7f0e8cff9700 1 -- 172.21.15.121:0/2426576298 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0e78047a40 con 0x7f0e90154ca0 2023-11-15T02:57:49.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.631+0000 7f0e9649b700 1 -- 172.21.15.121:0/2426576298 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e900c4500 con 0x7f0e7005a540 2023-11-15T02:57:49.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.647+0000 7f0e8cff9700 1 -- 172.21.15.121:0/2426576298 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+338132 (secure 0 0 0) 0x7f0e900c4500 con 0x7f0e7005a540 2023-11-15T02:57:49.659 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:49.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.655+0000 7f0e9649b700 1 -- 172.21.15.121:0/2426576298 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e7005a540 msgr2=0x7f0e7005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:49.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.655+0000 7f0e9649b700 1 --2- 172.21.15.121:0/2426576298 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e7005a540 0x7f0e7005ca00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f0e8000ba10 tx=0x7f0e80009840).stop 2023-11-15T02:57:49.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.655+0000 7f0e9649b700 1 -- 172.21.15.121:0/2426576298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e90154ca0 msgr2=0x7f0e9014efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:49.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.655+0000 7f0e9649b700 1 --2- 172.21.15.121:0/2426576298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e90154ca0 0x7f0e9014efe0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f0e7800c440 tx=0x7f0e78007dd0).stop 2023-11-15T02:57:49.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.655+0000 7f0e9649b700 1 -- 172.21.15.121:0/2426576298 shutdown_connections 2023-11-15T02:57:49.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.655+0000 7f0e9649b700 1 --2- 172.21.15.121:0/2426576298 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e7005a540 0x7f0e7005ca00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.655+0000 7f0e9649b700 1 --2- 172.21.15.121:0/2426576298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e901562f0 0x7f0e9014f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.655+0000 7f0e9649b700 1 --2- 172.21.15.121:0/2426576298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e90154ca0 0x7f0e9014efe0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.655+0000 7f0e9649b700 1 --2- 172.21.15.121:0/2426576298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e900a57c0 0x7f0e90143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.655+0000 7f0e9649b700 1 -- 172.21.15.121:0/2426576298 >> 172.21.15.121:0/2426576298 conn(0x7f0e90062190 msgr2=0x7f0e900ab840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:49.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.655+0000 7f0e9649b700 1 -- 172.21.15.121:0/2426576298 shutdown_connections 2023-11-15T02:57:49.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.655+0000 7f0e9649b700 1 -- 172.21.15.121:0/2426576298 wait complete. 2023-11-15T02:57:49.665 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T02:57:49.682 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2023-11-15T02:57:47.931491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1302,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":262,"acting":262,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2379448,"kb_used_data":2680,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752595272,"statfs":{"total":773094113280,"available":770657558528,"internally_reserved":0,"allocated":2744320,"data_stored":1102302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4297,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.436357"},"pg_stats":[{"pgid":"6.3c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.576426+0000","last_change":"2023-11-15T02:57:44.576426+0000","last_active":"2023-11-15T02:57:44.576426+0000","last_peered":"2023-11-15T02:57:44.576426+0000","last_clean":"2023-11-15T02:57:44.576426+0000","last_became_active":"2023-11-15T02:57:44.572509+0000","last_became_peered":"2023-11-15T02:57:44.572509+0000","last_unstale":"2023-11-15T02:57:44.576426+0000","last_undegraded":"2023-11-15T02:57:44.576426+0000","last_fullsized":"2023-11-15T02:57:44.576426+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3f","version":"32'3","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.587176+0000","last_change":"2023-11-15T02:57:44.105284+0000","last_active":"2023-11-15T02:57:44.587176+0000","last_peered":"2023-11-15T02:57:44.587176+0000","last_clean":"2023-11-15T02:57:44.587176+0000","last_became_active":"2023-11-15T02:57:44.104935+0000","last_became_peered":"2023-11-15T02:57:44.104935+0000","last_unstale":"2023-11-15T02:57:44.587176+0000","last_undegraded":"2023-11-15T02:57:44.587176+0000","last_fullsized":"2023-11-15T02:57:44.587176+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566808+0000","last_change":"2023-11-15T02:57:44.566808+0000","last_active":"2023-11-15T02:57:44.566808+0000","last_peered":"2023-11-15T02:57:44.566808+0000","last_clean":"2023-11-15T02:57:44.566808+0000","last_became_active":"2023-11-15T02:57:44.566228+0000","last_became_peered":"2023-11-15T02:57:44.566228+0000","last_unstale":"2023-11-15T02:57:44.566808+0000","last_undegraded":"2023-11-15T02:57:44.566808+0000","last_fullsized":"2023-11-15T02:57:44.566808+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109248+0000","last_change":"2023-11-15T02:57:44.101869+0000","last_active":"2023-11-15T02:57:44.109248+0000","last_peered":"2023-11-15T02:57:44.109248+0000","last_clean":"2023-11-15T02:57:44.109248+0000","last_became_active":"2023-11-15T02:57:44.101602+0000","last_became_peered":"2023-11-15T02:57:44.101602+0000","last_unstale":"2023-11-15T02:57:44.109248+0000","last_undegraded":"2023-11-15T02:57:44.109248+0000","last_fullsized":"2023-11-15T02:57:44.109248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.564204+0000","last_change":"2023-11-15T02:57:44.564204+0000","last_active":"2023-11-15T02:57:44.564204+0000","last_peered":"2023-11-15T02:57:44.564204+0000","last_clean":"2023-11-15T02:57:44.564204+0000","last_became_active":"2023-11-15T02:57:44.563976+0000","last_became_peered":"2023-11-15T02:57:44.563976+0000","last_unstale":"2023-11-15T02:57:44.564204+0000","last_undegraded":"2023-11-15T02:57:44.564204+0000","last_fullsized":"2023-11-15T02:57:44.564204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.096033+0000","last_change":"2023-11-15T02:57:44.096033+0000","last_active":"2023-11-15T02:57:44.096033+0000","last_peered":"2023-11-15T02:57:44.096033+0000","last_clean":"2023-11-15T02:57:44.096033+0000","last_became_active":"2023-11-15T02:57:44.095626+0000","last_became_peered":"2023-11-15T02:57:44.095626+0000","last_unstale":"2023-11-15T02:57:44.096033+0000","last_undegraded":"2023-11-15T02:57:44.096033+0000","last_fullsized":"2023-11-15T02:57:44.096033+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.568907+0000","last_change":"2023-11-15T02:57:44.568907+0000","last_active":"2023-11-15T02:57:44.568907+0000","last_peered":"2023-11-15T02:57:44.568907+0000","last_clean":"2023-11-15T02:57:44.568907+0000","last_became_active":"2023-11-15T02:57:44.568621+0000","last_became_peered":"2023-11-15T02:57:44.568621+0000","last_unstale":"2023-11-15T02:57:44.568907+0000","last_undegraded":"2023-11-15T02:57:44.568907+0000","last_fullsized":"2023-11-15T02:57:44.568907+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.065783+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.570582+0000","last_change":"2023-11-15T02:57:44.570582+0000","last_active":"2023-11-15T02:57:44.570582+0000","last_peered":"2023-11-15T02:57:44.570582+0000","last_clean":"2023-11-15T02:57:44.570582+0000","last_became_active":"2023-11-15T02:57:44.570314+0000","last_became_peered":"2023-11-15T02:57:44.570314+0000","last_unstale":"2023-11-15T02:57:44.570582+0000","last_undegraded":"2023-11-15T02:57:44.570582+0000","last_fullsized":"2023-11-15T02:57:44.570582+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3b","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560605+0000","last_change":"2023-11-15T02:57:44.104587+0000","last_active":"2023-11-15T02:57:44.560605+0000","last_peered":"2023-11-15T02:57:44.560605+0000","last_clean":"2023-11-15T02:57:44.560605+0000","last_became_active":"2023-11-15T02:57:44.104223+0000","last_became_peered":"2023-11-15T02:57:44.104223+0000","last_unstale":"2023-11-15T02:57:44.560605+0000","last_undegraded":"2023-11-15T02:57:44.560605+0000","last_fullsized":"2023-11-15T02:57:44.560605+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.077573+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.553452+0000","last_change":"2023-11-15T02:57:44.102587+0000","last_active":"2023-11-15T02:57:44.553452+0000","last_peered":"2023-11-15T02:57:44.553452+0000","last_clean":"2023-11-15T02:57:44.553452+0000","last_became_active":"2023-11-15T02:57:44.102311+0000","last_became_peered":"2023-11-15T02:57:44.102311+0000","last_unstale":"2023-11-15T02:57:44.553452+0000","last_undegraded":"2023-11-15T02:57:44.553452+0000","last_fullsized":"2023-11-15T02:57:44.553452+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.054985+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.110877+0000","last_change":"2023-11-15T02:57:44.105206+0000","last_active":"2023-11-15T02:57:44.110877+0000","last_peered":"2023-11-15T02:57:44.110877+0000","last_clean":"2023-11-15T02:57:44.110877+0000","last_became_active":"2023-11-15T02:57:44.104936+0000","last_became_peered":"2023-11-15T02:57:44.104936+0000","last_unstale":"2023-11-15T02:57:44.110877+0000","last_undegraded":"2023-11-15T02:57:44.110877+0000","last_fullsized":"2023-11-15T02:57:44.110877+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566763+0000","last_change":"2023-11-15T02:57:44.566763+0000","last_active":"2023-11-15T02:57:44.566763+0000","last_peered":"2023-11-15T02:57:44.566763+0000","last_clean":"2023-11-15T02:57:44.566763+0000","last_became_active":"2023-11-15T02:57:44.566510+0000","last_became_peered":"2023-11-15T02:57:44.566510+0000","last_unstale":"2023-11-15T02:57:44.566763+0000","last_undegraded":"2023-11-15T02:57:44.566763+0000","last_fullsized":"2023-11-15T02:57:44.566763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.110691+0000","last_change":"2023-11-15T02:57:44.106233+0000","last_active":"2023-11-15T02:57:44.110691+0000","last_peered":"2023-11-15T02:57:44.110691+0000","last_clean":"2023-11-15T02:57:44.110691+0000","last_became_active":"2023-11-15T02:57:44.105985+0000","last_became_peered":"2023-11-15T02:57:44.105985+0000","last_unstale":"2023-11-15T02:57:44.110691+0000","last_undegraded":"2023-11-15T02:57:44.110691+0000","last_fullsized":"2023-11-15T02:57:44.110691+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577960+0000","last_change":"2023-11-15T02:57:44.577960+0000","last_active":"2023-11-15T02:57:44.577960+0000","last_peered":"2023-11-15T02:57:44.577960+0000","last_clean":"2023-11-15T02:57:44.577960+0000","last_became_active":"2023-11-15T02:57:44.577405+0000","last_became_peered":"2023-11-15T02:57:44.577405+0000","last_unstale":"2023-11-15T02:57:44.577960+0000","last_undegraded":"2023-11-15T02:57:44.577960+0000","last_fullsized":"2023-11-15T02:57:44.577960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.099195+0000","last_change":"2023-11-15T02:57:44.091037+0000","last_active":"2023-11-15T02:57:44.099195+0000","last_peered":"2023-11-15T02:57:44.099195+0000","last_clean":"2023-11-15T02:57:44.099195+0000","last_became_active":"2023-11-15T02:57:44.090808+0000","last_became_peered":"2023-11-15T02:57:44.090808+0000","last_unstale":"2023-11-15T02:57:44.099195+0000","last_undegraded":"2023-11-15T02:57:44.099195+0000","last_fullsized":"2023-11-15T02:57:44.099195+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.077670+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.063700+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.075254+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109579+0000","last_change":"2023-11-15T02:57:44.102619+0000","last_active":"2023-11-15T02:57:44.109579+0000","last_peered":"2023-11-15T02:57:44.109579+0000","last_clean":"2023-11-15T02:57:44.109579+0000","last_became_active":"2023-11-15T02:57:44.102328+0000","last_became_peered":"2023-11-15T02:57:44.102328+0000","last_unstale":"2023-11-15T02:57:44.109579+0000","last_undegraded":"2023-11-15T02:57:44.109579+0000","last_fullsized":"2023-11-15T02:57:44.109579+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.579068+0000","last_change":"2023-11-15T02:57:44.579068+0000","last_active":"2023-11-15T02:57:44.579068+0000","last_peered":"2023-11-15T02:57:44.579068+0000","last_clean":"2023-11-15T02:57:44.579068+0000","last_became_active":"2023-11-15T02:57:44.578815+0000","last_became_peered":"2023-11-15T02:57:44.578815+0000","last_unstale":"2023-11-15T02:57:44.579068+0000","last_undegraded":"2023-11-15T02:57:44.579068+0000","last_fullsized":"2023-11-15T02:57:44.579068+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.113636+0000","last_change":"2023-11-15T02:57:44.113636+0000","last_active":"2023-11-15T02:57:44.113636+0000","last_peered":"2023-11-15T02:57:44.113636+0000","last_clean":"2023-11-15T02:57:44.113636+0000","last_became_active":"2023-11-15T02:57:44.113283+0000","last_became_peered":"2023-11-15T02:57:44.113283+0000","last_unstale":"2023-11-15T02:57:44.113636+0000","last_undegraded":"2023-11-15T02:57:44.113636+0000","last_fullsized":"2023-11-15T02:57:44.113636+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.584681+0000","last_change":"2023-11-15T02:57:44.584681+0000","last_active":"2023-11-15T02:57:44.584681+0000","last_peered":"2023-11-15T02:57:44.584681+0000","last_clean":"2023-11-15T02:57:44.584681+0000","last_became_active":"2023-11-15T02:57:44.584395+0000","last_became_peered":"2023-11-15T02:57:44.584395+0000","last_unstale":"2023-11-15T02:57:44.584681+0000","last_undegraded":"2023-11-15T02:57:44.584681+0000","last_fullsized":"2023-11-15T02:57:44.584681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.33","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560690+0000","last_change":"2023-11-15T02:57:44.086072+0000","last_active":"2023-11-15T02:57:44.560690+0000","last_peered":"2023-11-15T02:57:44.560690+0000","last_clean":"2023-11-15T02:57:44.560690+0000","last_became_active":"2023-11-15T02:57:44.085574+0000","last_became_peered":"2023-11-15T02:57:44.085574+0000","last_unstale":"2023-11-15T02:57:44.560690+0000","last_undegraded":"2023-11-15T02:57:44.560690+0000","last_fullsized":"2023-11-15T02:57:44.560690+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.076869+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.107875+0000","last_change":"2023-11-15T02:57:44.094640+0000","last_active":"2023-11-15T02:57:44.107875+0000","last_peered":"2023-11-15T02:57:44.107875+0000","last_clean":"2023-11-15T02:57:44.107875+0000","last_became_active":"2023-11-15T02:57:44.094279+0000","last_became_peered":"2023-11-15T02:57:44.094279+0000","last_unstale":"2023-11-15T02:57:44.107875+0000","last_undegraded":"2023-11-15T02:57:44.107875+0000","last_fullsized":"2023-11-15T02:57:44.107875+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.576818+0000","last_change":"2023-11-15T02:57:44.576818+0000","last_active":"2023-11-15T02:57:44.576818+0000","last_peered":"2023-11-15T02:57:44.576818+0000","last_clean":"2023-11-15T02:57:44.576818+0000","last_became_active":"2023-11-15T02:57:44.576074+0000","last_became_peered":"2023-11-15T02:57:44.576074+0000","last_unstale":"2023-11-15T02:57:44.576818+0000","last_undegraded":"2023-11-15T02:57:44.576818+0000","last_fullsized":"2023-11-15T02:57:44.576818+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109239+0000","last_change":"2023-11-15T02:57:44.109239+0000","last_active":"2023-11-15T02:57:44.109239+0000","last_peered":"2023-11-15T02:57:44.109239+0000","last_clean":"2023-11-15T02:57:44.109239+0000","last_became_active":"2023-11-15T02:57:44.103686+0000","last_became_peered":"2023-11-15T02:57:44.103686+0000","last_unstale":"2023-11-15T02:57:44.109239+0000","last_undegraded":"2023-11-15T02:57:44.109239+0000","last_fullsized":"2023-11-15T02:57:44.109239+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.074718+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"32'4","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585116+0000","last_change":"2023-11-15T02:57:44.113817+0000","last_active":"2023-11-15T02:57:44.585116+0000","last_peered":"2023-11-15T02:57:44.585116+0000","last_clean":"2023-11-15T02:57:44.585116+0000","last_became_active":"2023-11-15T02:57:44.112534+0000","last_became_peered":"2023-11-15T02:57:44.112534+0000","last_unstale":"2023-11-15T02:57:44.585116+0000","last_undegraded":"2023-11-15T02:57:44.585116+0000","last_fullsized":"2023-11-15T02:57:44.585116+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.575459+0000","last_change":"2023-11-15T02:57:44.575459+0000","last_active":"2023-11-15T02:57:44.575459+0000","last_peered":"2023-11-15T02:57:44.575459+0000","last_clean":"2023-11-15T02:57:44.575459+0000","last_became_active":"2023-11-15T02:57:44.575123+0000","last_became_peered":"2023-11-15T02:57:44.575123+0000","last_unstale":"2023-11-15T02:57:44.575459+0000","last_undegraded":"2023-11-15T02:57:44.575459+0000","last_fullsized":"2023-11-15T02:57:44.575459+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.111343+0000","last_change":"2023-11-15T02:57:44.107752+0000","last_active":"2023-11-15T02:57:44.111343+0000","last_peered":"2023-11-15T02:57:44.111343+0000","last_clean":"2023-11-15T02:57:44.111343+0000","last_became_active":"2023-11-15T02:57:44.107413+0000","last_became_peered":"2023-11-15T02:57:44.107413+0000","last_unstale":"2023-11-15T02:57:44.111343+0000","last_undegraded":"2023-11-15T02:57:44.111343+0000","last_fullsized":"2023-11-15T02:57:44.111343+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.572196+0000","last_change":"2023-11-15T02:57:44.572196+0000","last_active":"2023-11-15T02:57:44.572196+0000","last_peered":"2023-11-15T02:57:44.572196+0000","last_clean":"2023-11-15T02:57:44.572196+0000","last_became_active":"2023-11-15T02:57:44.571963+0000","last_became_peered":"2023-11-15T02:57:44.571963+0000","last_unstale":"2023-11-15T02:57:44.572196+0000","last_undegraded":"2023-11-15T02:57:44.572196+0000","last_fullsized":"2023-11-15T02:57:44.572196+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.106664+0000","last_change":"2023-11-15T02:57:44.097768+0000","last_active":"2023-11-15T02:57:44.106664+0000","last_peered":"2023-11-15T02:57:44.106664+0000","last_clean":"2023-11-15T02:57:44.106664+0000","last_became_active":"2023-11-15T02:57:44.097449+0000","last_became_peered":"2023-11-15T02:57:44.097449+0000","last_unstale":"2023-11-15T02:57:44.106664+0000","last_undegraded":"2023-11-15T02:57:44.106664+0000","last_fullsized":"2023-11-15T02:57:44.106664+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.588091+0000","last_change":"2023-11-15T02:57:44.588091+0000","last_active":"2023-11-15T02:57:44.588091+0000","last_peered":"2023-11-15T02:57:44.588091+0000","last_clean":"2023-11-15T02:57:44.588091+0000","last_became_active":"2023-11-15T02:57:44.587721+0000","last_became_peered":"2023-11-15T02:57:44.587721+0000","last_unstale":"2023-11-15T02:57:44.588091+0000","last_undegraded":"2023-11-15T02:57:44.588091+0000","last_fullsized":"2023-11-15T02:57:44.588091+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2d","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.551810+0000","last_change":"2023-11-15T02:57:44.097948+0000","last_active":"2023-11-15T02:57:44.551810+0000","last_peered":"2023-11-15T02:57:44.551810+0000","last_clean":"2023-11-15T02:57:44.551810+0000","last_became_active":"2023-11-15T02:57:44.097642+0000","last_became_peered":"2023-11-15T02:57:44.097642+0000","last_unstale":"2023-11-15T02:57:44.551810+0000","last_undegraded":"2023-11-15T02:57:44.551810+0000","last_fullsized":"2023-11-15T02:57:44.551810+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582841+0000","last_change":"2023-11-15T02:57:44.582841+0000","last_active":"2023-11-15T02:57:44.582841+0000","last_peered":"2023-11-15T02:57:44.582841+0000","last_clean":"2023-11-15T02:57:44.582841+0000","last_became_active":"2023-11-15T02:57:44.582592+0000","last_became_peered":"2023-11-15T02:57:44.582592+0000","last_unstale":"2023-11-15T02:57:44.582841+0000","last_undegraded":"2023-11-15T02:57:44.582841+0000","last_fullsized":"2023-11-15T02:57:44.582841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.101447+0000","last_change":"2023-11-15T02:57:44.095430+0000","last_active":"2023-11-15T02:57:44.101447+0000","last_peered":"2023-11-15T02:57:44.101447+0000","last_clean":"2023-11-15T02:57:44.101447+0000","last_became_active":"2023-11-15T02:57:44.094060+0000","last_became_peered":"2023-11-15T02:57:44.094060+0000","last_unstale":"2023-11-15T02:57:44.101447+0000","last_undegraded":"2023-11-15T02:57:44.101447+0000","last_fullsized":"2023-11-15T02:57:44.101447+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.571866+0000","last_change":"2023-11-15T02:57:44.571866+0000","last_active":"2023-11-15T02:57:44.571866+0000","last_peered":"2023-11-15T02:57:44.571866+0000","last_clean":"2023-11-15T02:57:44.571866+0000","last_became_active":"2023-11-15T02:57:44.571594+0000","last_became_peered":"2023-11-15T02:57:44.571594+0000","last_unstale":"2023-11-15T02:57:44.571866+0000","last_undegraded":"2023-11-15T02:57:44.571866+0000","last_fullsized":"2023-11-15T02:57:44.571866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.561370+0000","last_change":"2023-11-15T02:57:44.111471+0000","last_active":"2023-11-15T02:57:44.561370+0000","last_peered":"2023-11-15T02:57:44.561370+0000","last_clean":"2023-11-15T02:57:44.561370+0000","last_became_active":"2023-11-15T02:57:44.110158+0000","last_became_peered":"2023-11-15T02:57:44.110158+0000","last_unstale":"2023-11-15T02:57:44.561370+0000","last_undegraded":"2023-11-15T02:57:44.561370+0000","last_fullsized":"2023-11-15T02:57:44.561370+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.589275+0000","last_change":"2023-11-15T02:57:44.589275+0000","last_active":"2023-11-15T02:57:44.589275+0000","last_peered":"2023-11-15T02:57:44.589275+0000","last_clean":"2023-11-15T02:57:44.589275+0000","last_became_active":"2023-11-15T02:57:44.588964+0000","last_became_peered":"2023-11-15T02:57:44.588964+0000","last_unstale":"2023-11-15T02:57:44.589275+0000","last_undegraded":"2023-11-15T02:57:44.589275+0000","last_fullsized":"2023-11-15T02:57:44.589275+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.103617+0000","last_change":"2023-11-15T02:57:44.098842+0000","last_active":"2023-11-15T02:57:44.103617+0000","last_peered":"2023-11-15T02:57:44.103617+0000","last_clean":"2023-11-15T02:57:44.103617+0000","last_became_active":"2023-11-15T02:57:44.098539+0000","last_became_peered":"2023-11-15T02:57:44.098539+0000","last_unstale":"2023-11-15T02:57:44.103617+0000","last_undegraded":"2023-11-15T02:57:44.103617+0000","last_fullsized":"2023-11-15T02:57:44.103617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.571505+0000","last_change":"2023-11-15T02:57:44.571505+0000","last_active":"2023-11-15T02:57:44.571505+0000","last_peered":"2023-11-15T02:57:44.571505+0000","last_clean":"2023-11-15T02:57:44.571505+0000","last_became_active":"2023-11-15T02:57:44.570032+0000","last_became_peered":"2023-11-15T02:57:44.570032+0000","last_unstale":"2023-11-15T02:57:44.571505+0000","last_undegraded":"2023-11-15T02:57:44.571505+0000","last_fullsized":"2023-11-15T02:57:44.571505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.112480+0000","last_change":"2023-11-15T02:57:44.102777+0000","last_active":"2023-11-15T02:57:44.112480+0000","last_peered":"2023-11-15T02:57:44.112480+0000","last_clean":"2023-11-15T02:57:44.112480+0000","last_became_active":"2023-11-15T02:57:44.102485+0000","last_became_peered":"2023-11-15T02:57:44.102485+0000","last_unstale":"2023-11-15T02:57:44.112480+0000","last_undegraded":"2023-11-15T02:57:44.112480+0000","last_fullsized":"2023-11-15T02:57:44.112480+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.076093+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.100059+0000","last_change":"2023-11-15T02:57:44.100059+0000","last_active":"2023-11-15T02:57:44.100059+0000","last_peered":"2023-11-15T02:57:44.100059+0000","last_clean":"2023-11-15T02:57:44.100059+0000","last_became_active":"2023-11-15T02:57:44.099755+0000","last_became_peered":"2023-11-15T02:57:44.099755+0000","last_unstale":"2023-11-15T02:57:44.100059+0000","last_undegraded":"2023-11-15T02:57:44.100059+0000","last_fullsized":"2023-11-15T02:57:44.100059+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.076365+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.063419+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577168+0000","last_change":"2023-11-15T02:57:44.577168+0000","last_active":"2023-11-15T02:57:44.577168+0000","last_peered":"2023-11-15T02:57:44.577168+0000","last_clean":"2023-11-15T02:57:44.577168+0000","last_became_active":"2023-11-15T02:57:44.576902+0000","last_became_peered":"2023-11-15T02:57:44.576902+0000","last_unstale":"2023-11-15T02:57:44.577168+0000","last_undegraded":"2023-11-15T02:57:44.577168+0000","last_fullsized":"2023-11-15T02:57:44.577168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.106529+0000","last_change":"2023-11-15T02:57:44.097229+0000","last_active":"2023-11-15T02:57:44.106529+0000","last_peered":"2023-11-15T02:57:44.106529+0000","last_clean":"2023-11-15T02:57:44.106529+0000","last_became_active":"2023-11-15T02:57:44.096765+0000","last_became_peered":"2023-11-15T02:57:44.096765+0000","last_unstale":"2023-11-15T02:57:44.106529+0000","last_undegraded":"2023-11-15T02:57:44.106529+0000","last_fullsized":"2023-11-15T02:57:44.106529+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.054985+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.063627+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.576398+0000","last_change":"2023-11-15T02:57:44.576398+0000","last_active":"2023-11-15T02:57:44.576398+0000","last_peered":"2023-11-15T02:57:44.576398+0000","last_clean":"2023-11-15T02:57:44.576398+0000","last_became_active":"2023-11-15T02:57:44.576039+0000","last_became_peered":"2023-11-15T02:57:44.576039+0000","last_unstale":"2023-11-15T02:57:44.576398+0000","last_undegraded":"2023-11-15T02:57:44.576398+0000","last_fullsized":"2023-11-15T02:57:44.576398+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.098514+0000","last_change":"2023-11-15T02:57:44.098514+0000","last_active":"2023-11-15T02:57:44.098514+0000","last_peered":"2023-11-15T02:57:44.098514+0000","last_clean":"2023-11-15T02:57:44.098514+0000","last_became_active":"2023-11-15T02:57:44.098120+0000","last_became_peered":"2023-11-15T02:57:44.098120+0000","last_unstale":"2023-11-15T02:57:44.098514+0000","last_undegraded":"2023-11-15T02:57:44.098514+0000","last_fullsized":"2023-11-15T02:57:44.098514+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.054985+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.108729+0000","last_change":"2023-11-15T02:57:44.101472+0000","last_active":"2023-11-15T02:57:44.108729+0000","last_peered":"2023-11-15T02:57:44.108729+0000","last_clean":"2023-11-15T02:57:44.108729+0000","last_became_active":"2023-11-15T02:57:44.101190+0000","last_became_peered":"2023-11-15T02:57:44.101190+0000","last_unstale":"2023-11-15T02:57:44.108729+0000","last_undegraded":"2023-11-15T02:57:44.108729+0000","last_fullsized":"2023-11-15T02:57:44.108729+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.589081+0000","last_change":"2023-11-15T02:57:44.589081+0000","last_active":"2023-11-15T02:57:44.589081+0000","last_peered":"2023-11-15T02:57:44.589081+0000","last_clean":"2023-11-15T02:57:44.589081+0000","last_became_active":"2023-11-15T02:57:44.588602+0000","last_became_peered":"2023-11-15T02:57:44.588602+0000","last_unstale":"2023-11-15T02:57:44.589081+0000","last_undegraded":"2023-11-15T02:57:44.589081+0000","last_fullsized":"2023-11-15T02:57:44.589081+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.108688+0000","last_change":"2023-11-15T02:57:44.101411+0000","last_active":"2023-11-15T02:57:44.108688+0000","last_peered":"2023-11-15T02:57:44.108688+0000","last_clean":"2023-11-15T02:57:44.108688+0000","last_became_active":"2023-11-15T02:57:44.099981+0000","last_became_peered":"2023-11-15T02:57:44.099981+0000","last_unstale":"2023-11-15T02:57:44.108688+0000","last_undegraded":"2023-11-15T02:57:44.108688+0000","last_fullsized":"2023-11-15T02:57:44.108688+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.588317+0000","last_change":"2023-11-15T02:57:44.588317+0000","last_active":"2023-11-15T02:57:44.588317+0000","last_peered":"2023-11-15T02:57:44.588317+0000","last_clean":"2023-11-15T02:57:44.588317+0000","last_became_active":"2023-11-15T02:57:44.586643+0000","last_became_peered":"2023-11-15T02:57:44.586643+0000","last_unstale":"2023-11-15T02:57:44.588317+0000","last_undegraded":"2023-11-15T02:57:44.588317+0000","last_fullsized":"2023-11-15T02:57:44.588317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.104386+0000","last_change":"2023-11-15T02:57:44.097212+0000","last_active":"2023-11-15T02:57:44.104386+0000","last_peered":"2023-11-15T02:57:44.104386+0000","last_clean":"2023-11-15T02:57:44.104386+0000","last_became_active":"2023-11-15T02:57:44.096903+0000","last_became_peered":"2023-11-15T02:57:44.096903+0000","last_unstale":"2023-11-15T02:57:44.104386+0000","last_undegraded":"2023-11-15T02:57:44.104386+0000","last_fullsized":"2023-11-15T02:57:44.104386+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.578257+0000","last_change":"2023-11-15T02:57:44.578257+0000","last_active":"2023-11-15T02:57:44.578257+0000","last_peered":"2023-11-15T02:57:44.578257+0000","last_clean":"2023-11-15T02:57:44.578257+0000","last_became_active":"2023-11-15T02:57:44.571560+0000","last_became_peered":"2023-11-15T02:57:44.571560+0000","last_unstale":"2023-11-15T02:57:44.578257+0000","last_undegraded":"2023-11-15T02:57:44.578257+0000","last_fullsized":"2023-11-15T02:57:44.578257+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.108660+0000","last_change":"2023-11-15T02:57:44.102174+0000","last_active":"2023-11-15T02:57:44.108660+0000","last_peered":"2023-11-15T02:57:44.108660+0000","last_clean":"2023-11-15T02:57:44.108660+0000","last_became_active":"2023-11-15T02:57:44.101794+0000","last_became_peered":"2023-11-15T02:57:44.101794+0000","last_unstale":"2023-11-15T02:57:44.108660+0000","last_undegraded":"2023-11-15T02:57:44.108660+0000","last_fullsized":"2023-11-15T02:57:44.108660+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.587132+0000","last_change":"2023-11-15T02:57:44.587132+0000","last_active":"2023-11-15T02:57:44.587132+0000","last_peered":"2023-11-15T02:57:44.587132+0000","last_clean":"2023-11-15T02:57:44.587132+0000","last_became_active":"2023-11-15T02:57:44.586820+0000","last_became_peered":"2023-11-15T02:57:44.586820+0000","last_unstale":"2023-11-15T02:57:44.587132+0000","last_undegraded":"2023-11-15T02:57:44.587132+0000","last_fullsized":"2023-11-15T02:57:44.587132+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.574296+0000","last_change":"2023-11-15T02:57:44.102584+0000","last_active":"2023-11-15T02:57:44.574296+0000","last_peered":"2023-11-15T02:57:44.574296+0000","last_clean":"2023-11-15T02:57:44.574296+0000","last_became_active":"2023-11-15T02:57:44.102267+0000","last_became_peered":"2023-11-15T02:57:44.102267+0000","last_unstale":"2023-11-15T02:57:44.574296+0000","last_undegraded":"2023-11-15T02:57:44.574296+0000","last_fullsized":"2023-11-15T02:57:44.574296+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.054985+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.561164+0000","last_change":"2023-11-15T02:57:44.102680+0000","last_active":"2023-11-15T02:57:44.561164+0000","last_peered":"2023-11-15T02:57:44.561164+0000","last_clean":"2023-11-15T02:57:44.561164+0000","last_became_active":"2023-11-15T02:57:44.102394+0000","last_became_peered":"2023-11-15T02:57:44.102394+0000","last_unstale":"2023-11-15T02:57:44.561164+0000","last_undegraded":"2023-11-15T02:57:44.561164+0000","last_fullsized":"2023-11-15T02:57:44.561164+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.586592+0000","last_change":"2023-11-15T02:57:44.586592+0000","last_active":"2023-11-15T02:57:44.586592+0000","last_peered":"2023-11-15T02:57:44.586592+0000","last_clean":"2023-11-15T02:57:44.586592+0000","last_became_active":"2023-11-15T02:57:44.586211+0000","last_became_peered":"2023-11-15T02:57:44.586211+0000","last_unstale":"2023-11-15T02:57:44.586592+0000","last_undegraded":"2023-11-15T02:57:44.586592+0000","last_fullsized":"2023-11-15T02:57:44.586592+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.063052+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.579124+0000","last_change":"2023-11-15T02:57:44.579124+0000","last_active":"2023-11-15T02:57:44.579124+0000","last_peered":"2023-11-15T02:57:44.579124+0000","last_clean":"2023-11-15T02:57:44.579124+0000","last_became_active":"2023-11-15T02:57:44.578824+0000","last_became_peered":"2023-11-15T02:57:44.578824+0000","last_unstale":"2023-11-15T02:57:44.579124+0000","last_undegraded":"2023-11-15T02:57:44.579124+0000","last_fullsized":"2023-11-15T02:57:44.579124+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.108976+0000","last_change":"2023-11-15T02:57:44.101425+0000","last_active":"2023-11-15T02:57:44.108976+0000","last_peered":"2023-11-15T02:57:44.108976+0000","last_clean":"2023-11-15T02:57:44.108976+0000","last_became_active":"2023-11-15T02:57:44.101139+0000","last_became_peered":"2023-11-15T02:57:44.101139+0000","last_unstale":"2023-11-15T02:57:44.108976+0000","last_undegraded":"2023-11-15T02:57:44.108976+0000","last_fullsized":"2023-11-15T02:57:44.108976+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.574408+0000","last_change":"2023-11-15T02:57:44.574408+0000","last_active":"2023-11-15T02:57:44.574408+0000","last_peered":"2023-11-15T02:57:44.574408+0000","last_clean":"2023-11-15T02:57:44.574408+0000","last_became_active":"2023-11-15T02:57:44.573651+0000","last_became_peered":"2023-11-15T02:57:44.573651+0000","last_unstale":"2023-11-15T02:57:44.574408+0000","last_undegraded":"2023-11-15T02:57:44.574408+0000","last_fullsized":"2023-11-15T02:57:44.574408+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.112809+0000","last_change":"2023-11-15T02:57:44.112809+0000","last_active":"2023-11-15T02:57:44.112809+0000","last_peered":"2023-11-15T02:57:44.112809+0000","last_clean":"2023-11-15T02:57:44.112809+0000","last_became_active":"2023-11-15T02:57:44.111134+0000","last_became_peered":"2023-11-15T02:57:44.111134+0000","last_unstale":"2023-11-15T02:57:44.112809+0000","last_undegraded":"2023-11-15T02:57:44.112809+0000","last_fullsized":"2023-11-15T02:57:44.112809+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.076060+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.104280+0000","last_change":"2023-11-15T02:57:44.099996+0000","last_active":"2023-11-15T02:57:44.104280+0000","last_peered":"2023-11-15T02:57:44.104280+0000","last_clean":"2023-11-15T02:57:44.104280+0000","last_became_active":"2023-11-15T02:57:44.099695+0000","last_became_peered":"2023-11-15T02:57:44.099695+0000","last_unstale":"2023-11-15T02:57:44.104280+0000","last_undegraded":"2023-11-15T02:57:44.104280+0000","last_fullsized":"2023-11-15T02:57:44.104280+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577040+0000","last_change":"2023-11-15T02:57:44.577040+0000","last_active":"2023-11-15T02:57:44.577040+0000","last_peered":"2023-11-15T02:57:44.577040+0000","last_clean":"2023-11-15T02:57:44.577040+0000","last_became_active":"2023-11-15T02:57:44.576592+0000","last_became_peered":"2023-11-15T02:57:44.576592+0000","last_unstale":"2023-11-15T02:57:44.577040+0000","last_undegraded":"2023-11-15T02:57:44.577040+0000","last_fullsized":"2023-11-15T02:57:44.577040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.064186+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.571061+0000","last_change":"2023-11-15T02:57:44.571061+0000","last_active":"2023-11-15T02:57:44.571061+0000","last_peered":"2023-11-15T02:57:44.571061+0000","last_clean":"2023-11-15T02:57:44.571061+0000","last_became_active":"2023-11-15T02:57:44.570802+0000","last_became_peered":"2023-11-15T02:57:44.570802+0000","last_unstale":"2023-11-15T02:57:44.571061+0000","last_undegraded":"2023-11-15T02:57:44.571061+0000","last_fullsized":"2023-11-15T02:57:44.571061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.103937+0000","last_change":"2023-11-15T02:57:44.099371+0000","last_active":"2023-11-15T02:57:44.103937+0000","last_peered":"2023-11-15T02:57:44.103937+0000","last_clean":"2023-11-15T02:57:44.103937+0000","last_became_active":"2023-11-15T02:57:44.096689+0000","last_became_peered":"2023-11-15T02:57:44.096689+0000","last_unstale":"2023-11-15T02:57:44.103937+0000","last_undegraded":"2023-11-15T02:57:44.103937+0000","last_fullsized":"2023-11-15T02:57:44.103937+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.572799+0000","last_change":"2023-11-15T02:57:44.572799+0000","last_active":"2023-11-15T02:57:44.572799+0000","last_peered":"2023-11-15T02:57:44.572799+0000","last_clean":"2023-11-15T02:57:44.572799+0000","last_became_active":"2023-11-15T02:57:44.572147+0000","last_became_peered":"2023-11-15T02:57:44.572147+0000","last_unstale":"2023-11-15T02:57:44.572799+0000","last_undegraded":"2023-11-15T02:57:44.572799+0000","last_fullsized":"2023-11-15T02:57:44.572799+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.105678+0000","last_change":"2023-11-15T02:57:44.094718+0000","last_active":"2023-11-15T02:57:44.105678+0000","last_peered":"2023-11-15T02:57:44.105678+0000","last_clean":"2023-11-15T02:57:44.105678+0000","last_became_active":"2023-11-15T02:57:44.094236+0000","last_became_peered":"2023-11-15T02:57:44.094236+0000","last_unstale":"2023-11-15T02:57:44.105678+0000","last_undegraded":"2023-11-15T02:57:44.105678+0000","last_fullsized":"2023-11-15T02:57:44.105678+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577901+0000","last_change":"2023-11-15T02:57:44.577901+0000","last_active":"2023-11-15T02:57:44.577901+0000","last_peered":"2023-11-15T02:57:44.577901+0000","last_clean":"2023-11-15T02:57:44.577901+0000","last_became_active":"2023-11-15T02:57:44.577370+0000","last_became_peered":"2023-11-15T02:57:44.577370+0000","last_unstale":"2023-11-15T02:57:44.577901+0000","last_undegraded":"2023-11-15T02:57:44.577901+0000","last_fullsized":"2023-11-15T02:57:44.577901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109398+0000","last_change":"2023-11-15T02:57:44.109398+0000","last_active":"2023-11-15T02:57:44.109398+0000","last_peered":"2023-11-15T02:57:44.109398+0000","last_clean":"2023-11-15T02:57:44.109398+0000","last_became_active":"2023-11-15T02:57:44.104880+0000","last_became_peered":"2023-11-15T02:57:44.104880+0000","last_unstale":"2023-11-15T02:57:44.109398+0000","last_undegraded":"2023-11-15T02:57:44.109398+0000","last_fullsized":"2023-11-15T02:57:44.109398+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.570258+0000","last_change":"2023-11-15T02:57:44.570258+0000","last_active":"2023-11-15T02:57:44.570258+0000","last_peered":"2023-11-15T02:57:44.570258+0000","last_clean":"2023-11-15T02:57:44.570258+0000","last_became_active":"2023-11-15T02:57:44.569983+0000","last_became_peered":"2023-11-15T02:57:44.569983+0000","last_unstale":"2023-11-15T02:57:44.570258+0000","last_undegraded":"2023-11-15T02:57:44.570258+0000","last_fullsized":"2023-11-15T02:57:44.570258+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.096037+0000","last_change":"2023-11-15T02:57:44.096037+0000","last_active":"2023-11-15T02:57:44.096037+0000","last_peered":"2023-11-15T02:57:44.096037+0000","last_clean":"2023-11-15T02:57:44.096037+0000","last_became_active":"2023-11-15T02:57:44.095669+0000","last_became_peered":"2023-11-15T02:57:44.095669+0000","last_unstale":"2023-11-15T02:57:44.096037+0000","last_undegraded":"2023-11-15T02:57:44.096037+0000","last_fullsized":"2023-11-15T02:57:44.096037+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585203+0000","last_change":"2023-11-15T02:57:44.585203+0000","last_active":"2023-11-15T02:57:44.585203+0000","last_peered":"2023-11-15T02:57:44.585203+0000","last_clean":"2023-11-15T02:57:44.585203+0000","last_became_active":"2023-11-15T02:57:44.583007+0000","last_became_peered":"2023-11-15T02:57:44.583007+0000","last_unstale":"2023-11-15T02:57:44.585203+0000","last_undegraded":"2023-11-15T02:57:44.585203+0000","last_fullsized":"2023-11-15T02:57:44.585203+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.581336+0000","last_change":"2023-11-15T02:57:44.096764+0000","last_active":"2023-11-15T02:57:44.581336+0000","last_peered":"2023-11-15T02:57:44.581336+0000","last_clean":"2023-11-15T02:57:44.581336+0000","last_became_active":"2023-11-15T02:57:44.096348+0000","last_became_peered":"2023-11-15T02:57:44.096348+0000","last_unstale":"2023-11-15T02:57:44.581336+0000","last_undegraded":"2023-11-15T02:57:44.581336+0000","last_fullsized":"2023-11-15T02:57:44.581336+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566200+0000","last_change":"2023-11-15T02:57:44.566200+0000","last_active":"2023-11-15T02:57:44.566200+0000","last_peered":"2023-11-15T02:57:44.566200+0000","last_clean":"2023-11-15T02:57:44.566200+0000","last_became_active":"2023-11-15T02:57:44.565943+0000","last_became_peered":"2023-11-15T02:57:44.565943+0000","last_unstale":"2023-11-15T02:57:44.566200+0000","last_undegraded":"2023-11-15T02:57:44.566200+0000","last_fullsized":"2023-11-15T02:57:44.566200+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.553334+0000","last_change":"2023-11-15T02:57:44.109346+0000","last_active":"2023-11-15T02:57:44.553334+0000","last_peered":"2023-11-15T02:57:44.553334+0000","last_clean":"2023-11-15T02:57:44.553334+0000","last_became_active":"2023-11-15T02:57:44.109023+0000","last_became_peered":"2023-11-15T02:57:44.109023+0000","last_unstale":"2023-11-15T02:57:44.553334+0000","last_undegraded":"2023-11-15T02:57:44.553334+0000","last_fullsized":"2023-11-15T02:57:44.553334+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566248+0000","last_change":"2023-11-15T02:57:44.566248+0000","last_active":"2023-11-15T02:57:44.566248+0000","last_peered":"2023-11-15T02:57:44.566248+0000","last_clean":"2023-11-15T02:57:44.566248+0000","last_became_active":"2023-11-15T02:57:44.566002+0000","last_became_peered":"2023-11-15T02:57:44.566002+0000","last_unstale":"2023-11-15T02:57:44.566248+0000","last_undegraded":"2023-11-15T02:57:44.566248+0000","last_fullsized":"2023-11-15T02:57:44.566248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.062537+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.588350+0000","last_change":"2023-11-15T02:57:44.588350+0000","last_active":"2023-11-15T02:57:44.588350+0000","last_peered":"2023-11-15T02:57:44.588350+0000","last_clean":"2023-11-15T02:57:44.588350+0000","last_became_active":"2023-11-15T02:57:44.588073+0000","last_became_peered":"2023-11-15T02:57:44.588073+0000","last_unstale":"2023-11-15T02:57:44.588350+0000","last_undegraded":"2023-11-15T02:57:44.588350+0000","last_fullsized":"2023-11-15T02:57:44.588350+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109706+0000","last_change":"2023-11-15T02:57:44.101276+0000","last_active":"2023-11-15T02:57:44.109706+0000","last_peered":"2023-11-15T02:57:44.109706+0000","last_clean":"2023-11-15T02:57:44.109706+0000","last_became_active":"2023-11-15T02:57:44.100908+0000","last_became_peered":"2023-11-15T02:57:44.100908+0000","last_unstale":"2023-11-15T02:57:44.109706+0000","last_undegraded":"2023-11-15T02:57:44.109706+0000","last_fullsized":"2023-11-15T02:57:44.109706+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566177+0000","last_change":"2023-11-15T02:57:44.566177+0000","last_active":"2023-11-15T02:57:44.566177+0000","last_peered":"2023-11-15T02:57:44.566177+0000","last_clean":"2023-11-15T02:57:44.566177+0000","last_became_active":"2023-11-15T02:57:44.565917+0000","last_became_peered":"2023-11-15T02:57:44.565917+0000","last_unstale":"2023-11-15T02:57:44.566177+0000","last_undegraded":"2023-11-15T02:57:44.566177+0000","last_fullsized":"2023-11-15T02:57:44.566177+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"32'4","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.586189+0000","last_change":"2023-11-15T02:57:44.102229+0000","last_active":"2023-11-15T02:57:44.586189+0000","last_peered":"2023-11-15T02:57:44.586189+0000","last_clean":"2023-11-15T02:57:44.586189+0000","last_became_active":"2023-11-15T02:57:44.101858+0000","last_became_peered":"2023-11-15T02:57:44.101858+0000","last_unstale":"2023-11-15T02:57:44.586189+0000","last_undegraded":"2023-11-15T02:57:44.586189+0000","last_fullsized":"2023-11-15T02:57:44.586189+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.578142+0000","last_change":"2023-11-15T02:57:44.578142+0000","last_active":"2023-11-15T02:57:44.578142+0000","last_peered":"2023-11-15T02:57:44.578142+0000","last_clean":"2023-11-15T02:57:44.578142+0000","last_became_active":"2023-11-15T02:57:44.575328+0000","last_became_peered":"2023-11-15T02:57:44.575328+0000","last_unstale":"2023-11-15T02:57:44.578142+0000","last_undegraded":"2023-11-15T02:57:44.578142+0000","last_fullsized":"2023-11-15T02:57:44.578142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.100857+0000","last_change":"2023-11-15T02:57:44.100857+0000","last_active":"2023-11-15T02:57:44.100857+0000","last_peered":"2023-11-15T02:57:44.100857+0000","last_clean":"2023-11-15T02:57:44.100857+0000","last_became_active":"2023-11-15T02:57:44.099959+0000","last_became_peered":"2023-11-15T02:57:44.099959+0000","last_unstale":"2023-11-15T02:57:44.100857+0000","last_undegraded":"2023-11-15T02:57:44.100857+0000","last_fullsized":"2023-11-15T02:57:44.100857+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.077201+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585609+0000","last_change":"2023-11-15T02:57:44.096009+0000","last_active":"2023-11-15T02:57:44.585609+0000","last_peered":"2023-11-15T02:57:44.585609+0000","last_clean":"2023-11-15T02:57:44.585609+0000","last_became_active":"2023-11-15T02:57:44.095541+0000","last_became_peered":"2023-11-15T02:57:44.095541+0000","last_unstale":"2023-11-15T02:57:44.585609+0000","last_undegraded":"2023-11-15T02:57:44.585609+0000","last_fullsized":"2023-11-15T02:57:44.585609+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.583166+0000","last_change":"2023-11-15T02:57:44.583166+0000","last_active":"2023-11-15T02:57:44.583166+0000","last_peered":"2023-11-15T02:57:44.583166+0000","last_clean":"2023-11-15T02:57:44.583166+0000","last_became_active":"2023-11-15T02:57:44.582842+0000","last_became_peered":"2023-11-15T02:57:44.582842+0000","last_unstale":"2023-11-15T02:57:44.583166+0000","last_undegraded":"2023-11-15T02:57:44.583166+0000","last_fullsized":"2023-11-15T02:57:44.583166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109552+0000","last_change":"2023-11-15T02:57:44.101593+0000","last_active":"2023-11-15T02:57:44.109552+0000","last_peered":"2023-11-15T02:57:44.109552+0000","last_clean":"2023-11-15T02:57:44.109552+0000","last_became_active":"2023-11-15T02:57:44.101261+0000","last_became_peered":"2023-11-15T02:57:44.101261+0000","last_unstale":"2023-11-15T02:57:44.109552+0000","last_undegraded":"2023-11-15T02:57:44.109552+0000","last_fullsized":"2023-11-15T02:57:44.109552+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.584677+0000","last_change":"2023-11-15T02:57:44.584677+0000","last_active":"2023-11-15T02:57:44.584677+0000","last_peered":"2023-11-15T02:57:44.584677+0000","last_clean":"2023-11-15T02:57:44.584677+0000","last_became_active":"2023-11-15T02:57:44.584394+0000","last_became_peered":"2023-11-15T02:57:44.584394+0000","last_unstale":"2023-11-15T02:57:44.584677+0000","last_undegraded":"2023-11-15T02:57:44.584677+0000","last_fullsized":"2023-11-15T02:57:44.584677+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:43.048700+0000","last_change":"2023-11-15T02:57:43.064181+0000","last_active":"2023-11-15T02:57:43.048700+0000","last_peered":"2023-11-15T02:57:43.048700+0000","last_clean":"2023-11-15T02:57:43.048700+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:43.048700+0000","last_undegraded":"2023-11-15T02:57:43.048700+0000","last_fullsized":"2023-11-15T02:57:43.048700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.590180+0000","last_change":"2023-11-15T02:57:44.590180+0000","last_active":"2023-11-15T02:57:44.590180+0000","last_peered":"2023-11-15T02:57:44.590180+0000","last_clean":"2023-11-15T02:57:44.590180+0000","last_became_active":"2023-11-15T02:57:44.589752+0000","last_became_peered":"2023-11-15T02:57:44.589752+0000","last_unstale":"2023-11-15T02:57:44.590180+0000","last_undegraded":"2023-11-15T02:57:44.590180+0000","last_fullsized":"2023-11-15T02:57:44.590180+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560777+0000","last_change":"2023-11-15T02:57:44.109252+0000","last_active":"2023-11-15T02:57:44.560777+0000","last_peered":"2023-11-15T02:57:44.560777+0000","last_clean":"2023-11-15T02:57:44.560777+0000","last_became_active":"2023-11-15T02:57:44.108907+0000","last_became_peered":"2023-11-15T02:57:44.108907+0000","last_unstale":"2023-11-15T02:57:44.560777+0000","last_undegraded":"2023-11-15T02:57:44.560777+0000","last_fullsized":"2023-11-15T02:57:44.560777+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.568542+0000","last_change":"2023-11-15T02:57:44.568542+0000","last_active":"2023-11-15T02:57:44.568542+0000","last_peered":"2023-11-15T02:57:44.568542+0000","last_clean":"2023-11-15T02:57:44.568542+0000","last_became_active":"2023-11-15T02:57:44.568269+0000","last_became_peered":"2023-11-15T02:57:44.568269+0000","last_unstale":"2023-11-15T02:57:44.568542+0000","last_undegraded":"2023-11-15T02:57:44.568542+0000","last_fullsized":"2023-11-15T02:57:44.568542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.103890+0000","last_change":"2023-11-15T02:57:44.097937+0000","last_active":"2023-11-15T02:57:44.103890+0000","last_peered":"2023-11-15T02:57:44.103890+0000","last_clean":"2023-11-15T02:57:44.103890+0000","last_became_active":"2023-11-15T02:57:44.097630+0000","last_became_peered":"2023-11-15T02:57:44.097630+0000","last_unstale":"2023-11-15T02:57:44.103890+0000","last_undegraded":"2023-11-15T02:57:44.103890+0000","last_fullsized":"2023-11-15T02:57:44.103890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.078891+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.111403+0000","last_change":"2023-11-15T02:57:44.108255+0000","last_active":"2023-11-15T02:57:44.111403+0000","last_peered":"2023-11-15T02:57:44.111403+0000","last_clean":"2023-11-15T02:57:44.111403+0000","last_became_active":"2023-11-15T02:57:44.107961+0000","last_became_peered":"2023-11-15T02:57:44.107961+0000","last_unstale":"2023-11-15T02:57:44.111403+0000","last_undegraded":"2023-11-15T02:57:44.111403+0000","last_fullsized":"2023-11-15T02:57:44.111403+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.570926+0000","last_change":"2023-11-15T02:57:44.570926+0000","last_active":"2023-11-15T02:57:44.570926+0000","last_peered":"2023-11-15T02:57:44.570926+0000","last_clean":"2023-11-15T02:57:44.570926+0000","last_became_active":"2023-11-15T02:57:44.570646+0000","last_became_peered":"2023-11-15T02:57:44.570646+0000","last_unstale":"2023-11-15T02:57:44.570926+0000","last_undegraded":"2023-11-15T02:57:44.570926+0000","last_fullsized":"2023-11-15T02:57:44.570926+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.111257+0000","last_change":"2023-11-15T02:57:44.105564+0000","last_active":"2023-11-15T02:57:44.111257+0000","last_peered":"2023-11-15T02:57:44.111257+0000","last_clean":"2023-11-15T02:57:44.111257+0000","last_became_active":"2023-11-15T02:57:44.105312+0000","last_became_peered":"2023-11-15T02:57:44.105312+0000","last_unstale":"2023-11-15T02:57:44.111257+0000","last_undegraded":"2023-11-15T02:57:44.111257+0000","last_fullsized":"2023-11-15T02:57:44.111257+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.580419+0000","last_change":"2023-11-15T02:57:44.580419+0000","last_active":"2023-11-15T02:57:44.580419+0000","last_peered":"2023-11-15T02:57:44.580419+0000","last_clean":"2023-11-15T02:57:44.580419+0000","last_became_active":"2023-11-15T02:57:44.580057+0000","last_became_peered":"2023-11-15T02:57:44.580057+0000","last_unstale":"2023-11-15T02:57:44.580419+0000","last_undegraded":"2023-11-15T02:57:44.580419+0000","last_fullsized":"2023-11-15T02:57:44.580419+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.107016+0000","last_change":"2023-11-15T02:57:44.097433+0000","last_active":"2023-11-15T02:57:44.107016+0000","last_peered":"2023-11-15T02:57:44.107016+0000","last_clean":"2023-11-15T02:57:44.107016+0000","last_became_active":"2023-11-15T02:57:44.097149+0000","last_became_peered":"2023-11-15T02:57:44.097149+0000","last_unstale":"2023-11-15T02:57:44.107016+0000","last_undegraded":"2023-11-15T02:57:44.107016+0000","last_fullsized":"2023-11-15T02:57:44.107016+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.090441+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T02:57:44.090441+0000","last_peered":"2023-11-15T02:57:44.090441+0000","last_clean":"2023-11-15T02:57:44.090441+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T02:57:44.090441+0000","last_undegraded":"2023-11-15T02:57:44.090441+0000","last_fullsized":"2023-11-15T02:57:44.090441+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.584602+0000","last_change":"2023-11-15T02:57:44.584602+0000","last_active":"2023-11-15T02:57:44.584602+0000","last_peered":"2023-11-15T02:57:44.584602+0000","last_clean":"2023-11-15T02:57:44.584602+0000","last_became_active":"2023-11-15T02:57:44.584333+0000","last_became_peered":"2023-11-15T02:57:44.584333+0000","last_unstale":"2023-11-15T02:57:44.584602+0000","last_undegraded":"2023-11-15T02:57:44.584602+0000","last_fullsized":"2023-11-15T02:57:44.584602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.104126+0000","last_change":"2023-11-15T02:57:44.099697+0000","last_active":"2023-11-15T02:57:44.104126+0000","last_peered":"2023-11-15T02:57:44.104126+0000","last_clean":"2023-11-15T02:57:44.104126+0000","last_became_active":"2023-11-15T02:57:44.099388+0000","last_became_peered":"2023-11-15T02:57:44.099388+0000","last_unstale":"2023-11-15T02:57:44.104126+0000","last_undegraded":"2023-11-15T02:57:44.104126+0000","last_fullsized":"2023-11-15T02:57:44.104126+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-11-15T02:57:38.672457+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T02:57:38.672457+0000","last_peered":"2023-11-15T02:57:38.672457+0000","last_clean":"2023-11-15T02:57:38.672457+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T02:57:38.672457+0000","last_undegraded":"2023-11-15T02:57:38.672457+0000","last_fullsized":"2023-11-15T02:57:38.672457+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.054985+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.105522+0000","last_change":"2023-11-15T02:57:44.097789+0000","last_active":"2023-11-15T02:57:44.105522+0000","last_peered":"2023-11-15T02:57:44.105522+0000","last_clean":"2023-11-15T02:57:44.105522+0000","last_became_active":"2023-11-15T02:57:44.097133+0000","last_became_peered":"2023-11-15T02:57:44.097133+0000","last_unstale":"2023-11-15T02:57:44.105522+0000","last_undegraded":"2023-11-15T02:57:44.105522+0000","last_fullsized":"2023-11-15T02:57:44.105522+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.090466+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T02:57:44.090466+0000","last_peered":"2023-11-15T02:57:44.090466+0000","last_clean":"2023-11-15T02:57:44.090466+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T02:57:44.090466+0000","last_undegraded":"2023-11-15T02:57:44.090466+0000","last_fullsized":"2023-11-15T02:57:44.090466+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582488+0000","last_change":"2023-11-15T02:57:44.582488+0000","last_active":"2023-11-15T02:57:44.582488+0000","last_peered":"2023-11-15T02:57:44.582488+0000","last_clean":"2023-11-15T02:57:44.582488+0000","last_became_active":"2023-11-15T02:57:44.582160+0000","last_became_peered":"2023-11-15T02:57:44.582160+0000","last_unstale":"2023-11-15T02:57:44.582488+0000","last_undegraded":"2023-11-15T02:57:44.582488+0000","last_fullsized":"2023-11-15T02:57:44.582488+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.113534+0000","last_change":"2023-11-15T02:57:44.110737+0000","last_active":"2023-11-15T02:57:44.113534+0000","last_peered":"2023-11-15T02:57:44.113534+0000","last_clean":"2023-11-15T02:57:44.113534+0000","last_became_active":"2023-11-15T02:57:44.110423+0000","last_became_peered":"2023-11-15T02:57:44.110423+0000","last_unstale":"2023-11-15T02:57:44.113534+0000","last_undegraded":"2023-11-15T02:57:44.113534+0000","last_fullsized":"2023-11-15T02:57:44.113534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-11-15T02:57:38.672509+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T02:57:38.672509+0000","last_peered":"2023-11-15T02:57:38.672509+0000","last_clean":"2023-11-15T02:57:38.672509+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T02:57:38.672509+0000","last_undegraded":"2023-11-15T02:57:38.672509+0000","last_fullsized":"2023-11-15T02:57:38.672509+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577142+0000","last_change":"2023-11-15T02:57:44.577142+0000","last_active":"2023-11-15T02:57:44.577142+0000","last_peered":"2023-11-15T02:57:44.577142+0000","last_clean":"2023-11-15T02:57:44.577142+0000","last_became_active":"2023-11-15T02:57:44.576883+0000","last_became_peered":"2023-11-15T02:57:44.576883+0000","last_unstale":"2023-11-15T02:57:44.577142+0000","last_undegraded":"2023-11-15T02:57:44.577142+0000","last_fullsized":"2023-11-15T02:57:44.577142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.113548+0000","last_change":"2023-11-15T02:57:44.111033+0000","last_active":"2023-11-15T02:57:44.113548+0000","last_peered":"2023-11-15T02:57:44.113548+0000","last_clean":"2023-11-15T02:57:44.113548+0000","last_became_active":"2023-11-15T02:57:44.109168+0000","last_became_peered":"2023-11-15T02:57:44.109168+0000","last_unstale":"2023-11-15T02:57:44.113548+0000","last_undegraded":"2023-11-15T02:57:44.113548+0000","last_fullsized":"2023-11-15T02:57:44.113548+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-11-15T02:57:43.068175+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T02:57:43.068175+0000","last_peered":"2023-11-15T02:57:43.068175+0000","last_clean":"2023-11-15T02:57:43.068175+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T02:57:43.068175+0000","last_undegraded":"2023-11-15T02:57:43.068175+0000","last_fullsized":"2023-11-15T02:57:43.068175+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577944+0000","last_change":"2023-11-15T02:57:44.577944+0000","last_active":"2023-11-15T02:57:44.577944+0000","last_peered":"2023-11-15T02:57:44.577944+0000","last_clean":"2023-11-15T02:57:44.577944+0000","last_became_active":"2023-11-15T02:57:44.577738+0000","last_became_peered":"2023-11-15T02:57:44.577738+0000","last_unstale":"2023-11-15T02:57:44.577944+0000","last_undegraded":"2023-11-15T02:57:44.577944+0000","last_fullsized":"2023-11-15T02:57:44.577944+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.105276+0000","last_change":"2023-11-15T02:57:44.098264+0000","last_active":"2023-11-15T02:57:44.105276+0000","last_peered":"2023-11-15T02:57:44.105276+0000","last_clean":"2023-11-15T02:57:44.105276+0000","last_became_active":"2023-11-15T02:57:44.095009+0000","last_became_peered":"2023-11-15T02:57:44.095009+0000","last_unstale":"2023-11-15T02:57:44.105276+0000","last_undegraded":"2023-11-15T02:57:44.105276+0000","last_fullsized":"2023-11-15T02:57:44.105276+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.078017+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T02:57:44.078017+0000","last_peered":"2023-11-15T02:57:44.078017+0000","last_clean":"2023-11-15T02:57:44.078017+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T02:57:44.078017+0000","last_undegraded":"2023-11-15T02:57:44.078017+0000","last_fullsized":"2023-11-15T02:57:44.078017+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.054985+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.095125+0000","last_change":"2023-11-15T02:57:44.095125+0000","last_active":"2023-11-15T02:57:44.095125+0000","last_peered":"2023-11-15T02:57:44.095125+0000","last_clean":"2023-11-15T02:57:44.095125+0000","last_became_active":"2023-11-15T02:57:44.094525+0000","last_became_peered":"2023-11-15T02:57:44.094525+0000","last_unstale":"2023-11-15T02:57:44.095125+0000","last_undegraded":"2023-11-15T02:57:44.095125+0000","last_fullsized":"2023-11-15T02:57:44.095125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.088566+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T02:57:44.088566+0000","last_peered":"2023-11-15T02:57:44.088566+0000","last_clean":"2023-11-15T02:57:44.088566+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T02:57:44.088566+0000","last_undegraded":"2023-11-15T02:57:44.088566+0000","last_fullsized":"2023-11-15T02:57:44.088566+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:57:44.054985+0000","last_change":"2023-11-15T02:57:44.077301+0000","last_active":"2023-11-15T02:57:44.054985+0000","last_peered":"2023-11-15T02:57:44.054985+0000","last_clean":"2023-11-15T02:57:44.054985+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:57:44.054985+0000","last_undegraded":"2023-11-15T02:57:44.054985+0000","last_fullsized":"2023-11-15T02:57:44.054985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.112596+0000","last_change":"2023-11-15T02:57:44.111330+0000","last_active":"2023-11-15T02:57:44.112596+0000","last_peered":"2023-11-15T02:57:44.112596+0000","last_clean":"2023-11-15T02:57:44.112596+0000","last_became_active":"2023-11-15T02:57:44.111040+0000","last_became_peered":"2023-11-15T02:57:44.111040+0000","last_unstale":"2023-11-15T02:57:44.112596+0000","last_undegraded":"2023-11-15T02:57:44.112596+0000","last_fullsized":"2023-11-15T02:57:44.112596+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-11-15T02:57:43.068311+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T02:57:43.068311+0000","last_peered":"2023-11-15T02:57:43.068311+0000","last_clean":"2023-11-15T02:57:43.068311+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T02:57:43.068311+0000","last_undegraded":"2023-11-15T02:57:43.068311+0000","last_fullsized":"2023-11-15T02:57:43.068311+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.088937+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T02:57:44.088937+0000","last_peered":"2023-11-15T02:57:44.088937+0000","last_clean":"2023-11-15T02:57:44.088937+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T02:57:44.088937+0000","last_undegraded":"2023-11-15T02:57:44.088937+0000","last_fullsized":"2023-11-15T02:57:44.088937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":1302,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297432,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074408,"statfs":{"total":96636764160,"available":96332193792,"internally_reserved":0,"allocated":344064,"data_stored":138527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4970000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.369}]}]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":360448,"data_stored":146260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":360448,"data_stored":145728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297432,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074408,"statfs":{"total":96636764160,"available":96332193792,"internally_reserved":0,"allocated":344064,"data_stored":138527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.512}]}]},{"osd":2,"up_from":14,"seq":60129542159,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297428,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074412,"statfs":{"total":96636764160,"available":96332197888,"internally_reserved":0,"allocated":339968,"data_stored":137258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]}]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297432,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074408,"statfs":{"total":96636764160,"available":96332193792,"internally_reserved":0,"allocated":344064,"data_stored":138527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":1.0549999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.2839999999999998,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.81799999999999995}]},{"osd":3,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":3.54,"5min":3.54,"15min":3.54},"last":1.093}]},{"osd":4,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.85599999999999998}]},{"osd":5,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":3.6760000000000002,"5min":3.6760000000000002,"15min":3.6760000000000002},"last":1.149}]},{"osd":6,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":3.71,"5min":3.71,"15min":3.71},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":1.024,"5min":1.024,"15min":1.024},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.165}]},{"osd":7,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":1.032,"5min":1.032,"15min":1.032},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.238},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.2090000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297420,"kb_used_data":324,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074420,"statfs":{"total":96636764160,"available":96332206080,"internally_reserved":0,"allocated":331776,"data_stored":131848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3370000000000002}]},{"osd":1,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":3.6349999999999998,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":3.6349999999999998},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.516,"5min":3.516,"15min":3.516},"last":3.516}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.4180000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.141}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3879999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.2309999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3660000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297408,"kb_used_data":312,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074432,"statfs":{"total":96636764160,"available":96332218368,"internally_reserved":0,"allocated":319488,"data_stored":125627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T02:57:49.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2023-11-15T02:57:49.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:57:43.048700+0000', 'last_change': '2023-11-15T02:57:43.065783+0000', 'last_active': '2023-11-15T02:57:43.048700+0000', 'last_peered': '2023-11-15T02:57:43.048700+0000', 'last_clean': '2023-11-15T02:57:43.048700+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:43.048700+0000', 'last_undegraded': '2023-11-15T02:57:43.048700+0000', 'last_fullsized': '2023-11-15T02:57:43.048700+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-15T02:57:49.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2023-11-15T02:57:49.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:57:44.054985+0000', 'last_change': '2023-11-15T02:57:44.077573+0000', 'last_active': '2023-11-15T02:57:44.054985+0000', 'last_peered': '2023-11-15T02:57:44.054985+0000', 'last_clean': '2023-11-15T02:57:44.054985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:44.054985+0000', 'last_undegraded': '2023-11-15T02:57:44.054985+0000', 'last_fullsized': '2023-11-15T02:57:44.054985+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-15T02:57:49.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2023-11-15T02:57:49.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:57:44.054985+0000', 'last_change': '2023-11-15T02:57:44.054985+0000', 'last_active': '2023-11-15T02:57:44.054985+0000', 'last_peered': '2023-11-15T02:57:44.054985+0000', 'last_clean': '2023-11-15T02:57:44.054985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:44.054985+0000', 'last_undegraded': '2023-11-15T02:57:44.054985+0000', 'last_fullsized': '2023-11-15T02:57:44.054985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:57:49.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2023-11-15T02:57:49.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:57:44.054985+0000', 'last_change': '2023-11-15T02:57:44.077670+0000', 'last_active': '2023-11-15T02:57:44.054985+0000', 'last_peered': '2023-11-15T02:57:44.054985+0000', 'last_clean': '2023-11-15T02:57:44.054985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:44.054985+0000', 'last_undegraded': '2023-11-15T02:57:44.054985+0000', 'last_fullsized': '2023-11-15T02:57:44.054985+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-15T02:57:49.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2023-11-15T02:57:49.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:57:43.048700+0000', 'last_change': '2023-11-15T02:57:43.063700+0000', 'last_active': '2023-11-15T02:57:43.048700+0000', 'last_peered': '2023-11-15T02:57:43.048700+0000', 'last_clean': '2023-11-15T02:57:43.048700+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:43.048700+0000', 'last_undegraded': '2023-11-15T02:57:43.048700+0000', 'last_fullsized': '2023-11-15T02:57:43.048700+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-15T02:57:49.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2023-11-15T02:57:49.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:57:44.054985+0000', 'last_change': '2023-11-15T02:57:44.075254+0000', 'last_active': '2023-11-15T02:57:44.054985+0000', 'last_peered': '2023-11-15T02:57:44.054985+0000', 'last_clean': '2023-11-15T02:57:44.054985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:44.054985+0000', 'last_undegraded': '2023-11-15T02:57:44.054985+0000', 'last_fullsized': '2023-11-15T02:57:44.054985+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-15T02:57:49.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2023-11-15T02:57:49.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:57:44.054985+0000', 'last_change': '2023-11-15T02:57:44.076869+0000', 'last_active': '2023-11-15T02:57:44.054985+0000', 'last_peered': '2023-11-15T02:57:44.054985+0000', 'last_clean': '2023-11-15T02:57:44.054985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:44.054985+0000', 'last_undegraded': '2023-11-15T02:57:44.054985+0000', 'last_fullsized': '2023-11-15T02:57:44.054985+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-15T02:57:49.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2023-11-15T02:57:49.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:57:44.054985+0000', 'last_change': '2023-11-15T02:57:44.074718+0000', 'last_active': '2023-11-15T02:57:44.054985+0000', 'last_peered': '2023-11-15T02:57:44.054985+0000', 'last_clean': '2023-11-15T02:57:44.054985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:44.054985+0000', 'last_undegraded': '2023-11-15T02:57:44.054985+0000', 'last_fullsized': '2023-11-15T02:57:44.054985+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-15T02:57:49.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2023-11-15T02:57:49.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:57:44.054985+0000', 'last_change': '2023-11-15T02:57:44.076093+0000', 'last_active': '2023-11-15T02:57:44.054985+0000', 'last_peered': '2023-11-15T02:57:44.054985+0000', 'last_clean': '2023-11-15T02:57:44.054985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:44.054985+0000', 'last_undegraded': '2023-11-15T02:57:44.054985+0000', 'last_fullsized': '2023-11-15T02:57:44.054985+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-15T02:57:49.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2023-11-15T02:57:49.697 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:57:44.054985+0000', 'last_change': '2023-11-15T02:57:44.076365+0000', 'last_active': '2023-11-15T02:57:44.054985+0000', 'last_peered': '2023-11-15T02:57:44.054985+0000', 'last_clean': '2023-11-15T02:57:44.054985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:44.054985+0000', 'last_undegraded': '2023-11-15T02:57:44.054985+0000', 'last_fullsized': '2023-11-15T02:57:44.054985+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-15T02:57:49.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2023-11-15T02:57:49.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:57:43.048700+0000', 'last_change': '2023-11-15T02:57:43.063419+0000', 'last_active': '2023-11-15T02:57:43.048700+0000', 'last_peered': '2023-11-15T02:57:43.048700+0000', 'last_clean': '2023-11-15T02:57:43.048700+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:43.048700+0000', 'last_undegraded': '2023-11-15T02:57:43.048700+0000', 'last_fullsized': '2023-11-15T02:57:43.048700+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-15T02:57:49.698 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2023-11-15T02:57:49.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:57:44.054985+0000', 'last_change': '2023-11-15T02:57:44.054985+0000', 'last_active': '2023-11-15T02:57:44.054985+0000', 'last_peered': '2023-11-15T02:57:44.054985+0000', 'last_clean': '2023-11-15T02:57:44.054985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:44.054985+0000', 'last_undegraded': '2023-11-15T02:57:44.054985+0000', 'last_fullsized': '2023-11-15T02:57:44.054985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:57:49.699 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2023-11-15T02:57:49.699 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:57:43.048700+0000', 'last_change': '2023-11-15T02:57:43.063627+0000', 'last_active': '2023-11-15T02:57:43.048700+0000', 'last_peered': '2023-11-15T02:57:43.048700+0000', 'last_clean': '2023-11-15T02:57:43.048700+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:43.048700+0000', 'last_undegraded': '2023-11-15T02:57:43.048700+0000', 'last_fullsized': '2023-11-15T02:57:43.048700+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-15T02:57:49.700 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2023-11-15T02:57:49.700 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:57:44.054985+0000', 'last_change': '2023-11-15T02:57:44.054985+0000', 'last_active': '2023-11-15T02:57:44.054985+0000', 'last_peered': '2023-11-15T02:57:44.054985+0000', 'last_clean': '2023-11-15T02:57:44.054985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:44.054985+0000', 'last_undegraded': '2023-11-15T02:57:44.054985+0000', 'last_fullsized': '2023-11-15T02:57:44.054985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:57:49.700 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2023-11-15T02:57:49.701 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:57:44.054985+0000', 'last_change': '2023-11-15T02:57:44.054985+0000', 'last_active': '2023-11-15T02:57:44.054985+0000', 'last_peered': '2023-11-15T02:57:44.054985+0000', 'last_clean': '2023-11-15T02:57:44.054985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:44.054985+0000', 'last_undegraded': '2023-11-15T02:57:44.054985+0000', 'last_fullsized': '2023-11-15T02:57:44.054985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:57:49.701 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2023-11-15T02:57:49.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:57:43.048700+0000', 'last_change': '2023-11-15T02:57:43.063052+0000', 'last_active': '2023-11-15T02:57:43.048700+0000', 'last_peered': '2023-11-15T02:57:43.048700+0000', 'last_clean': '2023-11-15T02:57:43.048700+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:43.048700+0000', 'last_undegraded': '2023-11-15T02:57:43.048700+0000', 'last_fullsized': '2023-11-15T02:57:43.048700+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-15T02:57:49.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2023-11-15T02:57:49.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:57:44.054985+0000', 'last_change': '2023-11-15T02:57:44.076060+0000', 'last_active': '2023-11-15T02:57:44.054985+0000', 'last_peered': '2023-11-15T02:57:44.054985+0000', 'last_clean': '2023-11-15T02:57:44.054985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:44.054985+0000', 'last_undegraded': '2023-11-15T02:57:44.054985+0000', 'last_fullsized': '2023-11-15T02:57:44.054985+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-15T02:57:49.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2023-11-15T02:57:49.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:57:43.048700+0000', 'last_change': '2023-11-15T02:57:43.064186+0000', 'last_active': '2023-11-15T02:57:43.048700+0000', 'last_peered': '2023-11-15T02:57:43.048700+0000', 'last_clean': '2023-11-15T02:57:43.048700+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:43.048700+0000', 'last_undegraded': '2023-11-15T02:57:43.048700+0000', 'last_fullsized': '2023-11-15T02:57:43.048700+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-15T02:57:49.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2023-11-15T02:57:49.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:57:43.048700+0000', 'last_change': '2023-11-15T02:57:43.062537+0000', 'last_active': '2023-11-15T02:57:43.048700+0000', 'last_peered': '2023-11-15T02:57:43.048700+0000', 'last_clean': '2023-11-15T02:57:43.048700+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:43.048700+0000', 'last_undegraded': '2023-11-15T02:57:43.048700+0000', 'last_fullsized': '2023-11-15T02:57:43.048700+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-15T02:57:49.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2023-11-15T02:57:49.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:57:44.054985+0000', 'last_change': '2023-11-15T02:57:44.077201+0000', 'last_active': '2023-11-15T02:57:44.054985+0000', 'last_peered': '2023-11-15T02:57:44.054985+0000', 'last_clean': '2023-11-15T02:57:44.054985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:44.054985+0000', 'last_undegraded': '2023-11-15T02:57:44.054985+0000', 'last_fullsized': '2023-11-15T02:57:44.054985+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-15T02:57:49.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2023-11-15T02:57:49.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:57:43.048700+0000', 'last_change': '2023-11-15T02:57:43.064181+0000', 'last_active': '2023-11-15T02:57:43.048700+0000', 'last_peered': '2023-11-15T02:57:43.048700+0000', 'last_clean': '2023-11-15T02:57:43.048700+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:43.048700+0000', 'last_undegraded': '2023-11-15T02:57:43.048700+0000', 'last_fullsized': '2023-11-15T02:57:43.048700+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:43.048700+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-15T02:57:49.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2023-11-15T02:57:49.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:57:44.054985+0000', 'last_change': '2023-11-15T02:57:44.078891+0000', 'last_active': '2023-11-15T02:57:44.054985+0000', 'last_peered': '2023-11-15T02:57:44.054985+0000', 'last_clean': '2023-11-15T02:57:44.054985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:44.054985+0000', 'last_undegraded': '2023-11-15T02:57:44.054985+0000', 'last_fullsized': '2023-11-15T02:57:44.054985+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-15T02:57:49.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2023-11-15T02:57:49.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:57:44.054985+0000', 'last_change': '2023-11-15T02:57:44.054985+0000', 'last_active': '2023-11-15T02:57:44.054985+0000', 'last_peered': '2023-11-15T02:57:44.054985+0000', 'last_clean': '2023-11-15T02:57:44.054985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:44.054985+0000', 'last_undegraded': '2023-11-15T02:57:44.054985+0000', 'last_fullsized': '2023-11-15T02:57:44.054985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:57:49.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2023-11-15T02:57:49.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:57:44.054985+0000', 'last_change': '2023-11-15T02:57:44.054985+0000', 'last_active': '2023-11-15T02:57:44.054985+0000', 'last_peered': '2023-11-15T02:57:44.054985+0000', 'last_clean': '2023-11-15T02:57:44.054985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:44.054985+0000', 'last_undegraded': '2023-11-15T02:57:44.054985+0000', 'last_fullsized': '2023-11-15T02:57:44.054985+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:57:49.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2023-11-15T02:57:49.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:57:44.054985+0000', 'last_change': '2023-11-15T02:57:44.077301+0000', 'last_active': '2023-11-15T02:57:44.054985+0000', 'last_peered': '2023-11-15T02:57:44.054985+0000', 'last_clean': '2023-11-15T02:57:44.054985+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:57:44.054985+0000', 'last_undegraded': '2023-11-15T02:57:44.054985+0000', 'last_fullsized': '2023-11-15T02:57:44.054985+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'last_clean_scrub_stamp': '2023-11-15T02:57:44.054985+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-15T02:57:49.710 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-15T02:57:49.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.887+0000 7f5800b47700 1 Processor -- start 2023-11-15T02:57:49.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.887+0000 7f5800b47700 1 -- start start 2023-11-15T02:57:49.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.887+0000 7f5800b47700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f57fc0a9830 0x7f57fc0a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.887+0000 7f5800b47700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57fc0c30e0 0x7f57fc0c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.887+0000 7f5800b47700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57fc0657f0 con 0x7f57fc151870 2023-11-15T02:57:49.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.887+0000 7f5800b47700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57fc065ad0 con 0x7f57fc0a9830 2023-11-15T02:57:49.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.887+0000 7f5800b47700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc0c3a60 con 0x7f57fc0c30e0 2023-11-15T02:57:49.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.887+0000 7f57fbfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57fc0c30e0 0x7f57fc0c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.887+0000 7f57f3fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f57fc0a9830 0x7f57fc0a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.887+0000 7f57f3fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f57fc0a9830 0x7f57fc0a9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:48662/0 (socket says 172.21.15.121:48662) 2023-11-15T02:57:49.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.887+0000 7f57fbfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57fc0c30e0 0x7f57fc0c3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:41788/0 (socket says 172.21.15.121:41788) 2023-11-15T02:57:49.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.887+0000 7f57f3fff700 1 -- 172.21.15.121:0/1283861174 learned_addr learned my addr 172.21.15.121:0/1283861174 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:49.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.891+0000 7f57f3fff700 1 -- 172.21.15.121:0/1283861174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57fc0c30e0 msgr2=0x7f57fc0c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:49.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.891+0000 7f57f3fff700 1 --2- 172.21.15.121:0/1283861174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57fc0c30e0 0x7f57fc0c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.891+0000 7f57f3fff700 1 -- 172.21.15.121:0/1283861174 >> v1:172.21.15.190:6789/0 conn(0x7f57fc151870 legacy=0x7f57fc151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:49.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.891+0000 7f57f3fff700 1 -- 172.21.15.121:0/1283861174 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57fc0aa260 con 0x7f57fc0a9830 2023-11-15T02:57:49.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.891+0000 7f57f3fff700 1 --2- 172.21.15.121:0/1283861174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57fc0a9830 0x7f57fc0a9c70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f57e4006fd0 tx=0x7f57e40090f0).ready entity=mon.1 client_cookie=384771e6d3f0decd server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:49.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.891+0000 7f57faffd700 1 -- 172.21.15.121:0/1283861174 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e400b580 con 0x7f57fc0a9830 2023-11-15T02:57:49.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.891+0000 7f57faffd700 1 -- 172.21.15.121:0/1283861174 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e400b6e0 con 0x7f57fc0a9830 2023-11-15T02:57:49.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.891+0000 7f57faffd700 1 -- 172.21.15.121:0/1283861174 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e400ba60 con 0x7f57fc0a9830 2023-11-15T02:57:49.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.891+0000 7f5800b47700 1 -- 172.21.15.121:0/1283861174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57fc0a9830 msgr2=0x7f57fc0a9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:49.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.891+0000 7f5800b47700 1 --2- 172.21.15.121:0/1283861174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57fc0a9830 0x7f57fc0a9c70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f57e4006fd0 tx=0x7f57e40090f0).stop 2023-11-15T02:57:49.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.891+0000 7f5800b47700 1 -- 172.21.15.121:0/1283861174 shutdown_connections 2023-11-15T02:57:49.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.891+0000 7f5800b47700 1 --2- 172.21.15.121:0/1283861174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57fc0c30e0 0x7f57fc0c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.891+0000 7f5800b47700 1 --2- 172.21.15.121:0/1283861174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57fc0a9830 0x7f57fc0a9c70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.891+0000 7f5800b47700 1 -- 172.21.15.121:0/1283861174 >> 172.21.15.121:0/1283861174 conn(0x7f57fc062190 msgr2=0x7f57fc062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:49.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.891+0000 7f5800b47700 1 -- 172.21.15.121:0/1283861174 shutdown_connections 2023-11-15T02:57:49.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.891+0000 7f5800b47700 1 -- 172.21.15.121:0/1283861174 wait complete. 2023-11-15T02:57:49.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.895+0000 7f5800b47700 1 Processor -- start 2023-11-15T02:57:49.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.895+0000 7f5800b47700 1 -- start start 2023-11-15T02:57:49.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f5800b47700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57fc0a9830 0x7f57fc141170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f5800b47700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57fc0c30e0 0x7f57fc0a7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f5800b47700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57fc151870 0x7f57fc0a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f5800b47700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc0a86a0 con 0x7f57fc0a9830 2023-11-15T02:57:49.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f5800b47700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc0bf240 con 0x7f57fc151870 2023-11-15T02:57:49.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f5800b47700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc0be630 con 0x7f57fc0c30e0 2023-11-15T02:57:49.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f57fb7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57fc0a9830 0x7f57fc141170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f57fb7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57fc0a9830 0x7f57fc141170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56470/0 (socket says 172.21.15.121:56470) 2023-11-15T02:57:49.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f57fb7fe700 1 -- 172.21.15.121:0/2216814488 learned_addr learned my addr 172.21.15.121:0/2216814488 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:49.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f57fbfff700 1 --2- 172.21.15.121:0/2216814488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57fc151870 0x7f57fc0a81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f57f3fff700 1 --2- 172.21.15.121:0/2216814488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57fc0c30e0 0x7f57fc0a7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f57fb7fe700 1 -- 172.21.15.121:0/2216814488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57fc0c30e0 msgr2=0x7f57fc0a7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:49.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f57fb7fe700 1 --2- 172.21.15.121:0/2216814488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57fc0c30e0 0x7f57fc0a7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f57fb7fe700 1 -- 172.21.15.121:0/2216814488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57fc151870 msgr2=0x7f57fc0a81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:49.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f57fb7fe700 1 --2- 172.21.15.121:0/2216814488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57fc151870 0x7f57fc0a81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:49.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f57fb7fe700 1 -- 172.21.15.121:0/2216814488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e4009370 con 0x7f57fc0a9830 2023-11-15T02:57:49.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f57fb7fe700 1 --2- 172.21.15.121:0/2216814488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57fc0a9830 0x7f57fc141170 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f57ec00c4c0 tx=0x7f57ec007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:49.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f57f37fe700 1 -- 172.21.15.121:0/2216814488 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec00ce70 con 0x7f57fc0a9830 2023-11-15T02:57:49.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f57f37fe700 1 -- 172.21.15.121:0/2216814488 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec001e90 con 0x7f57fc0a9830 2023-11-15T02:57:49.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f57f37fe700 1 -- 172.21.15.121:0/2216814488 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec007250 con 0x7f57fc0a9830 2023-11-15T02:57:49.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f5800b47700 1 -- 172.21.15.121:0/2216814488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57fc0be910 con 0x7f57fc0a9830 2023-11-15T02:57:49.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.899+0000 7f5800b47700 1 -- 172.21.15.121:0/2216814488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc0beed0 con 0x7f57fc0a9830 2023-11-15T02:57:49.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.903+0000 7f5800b47700 1 -- 172.21.15.121:0/2216814488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57fc06bcf0 con 0x7f57fc0a9830 2023-11-15T02:57:49.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.903+0000 7f57f37fe700 1 -- 172.21.15.121:0/2216814488 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f57ec0066a0 con 0x7f57fc0a9830 2023-11-15T02:57:49.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.903+0000 7f57f37fe700 1 --2- 172.21.15.121:0/2216814488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f57e005a540 0x7f57e005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:49.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.903+0000 7f57f37fe700 1 -- 172.21.15.121:0/2216814488 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f57ec074100 con 0x7f57fc0a9830 2023-11-15T02:57:49.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.903+0000 7f57f3fff700 1 --2- 172.21.15.121:0/2216814488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f57e005a540 0x7f57e005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:49.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.907+0000 7f57f3fff700 1 --2- 172.21.15.121:0/2216814488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f57e005a540 0x7f57e005ca00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f57e4009130 tx=0x7f57e4017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:49.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:49.911+0000 7f57f37fe700 1 -- 172.21.15.121:0/2216814488 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f57ec047820 con 0x7f57fc0a9830 2023-11-15T02:57:50.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.135+0000 7f5800b47700 1 -- 172.21.15.121:0/2216814488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f57fc068a70 con 0x7f57fc0a9830 2023-11-15T02:57:50.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.139+0000 7f57f37fe700 1 -- 172.21.15.121:0/2216814488 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1661 (secure 0 0 0) 0x7f57ec04ae40 con 0x7f57fc0a9830 2023-11-15T02:57:50.143 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:50.144 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":77,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1700017004,"num_in_osds":8,"osd_in_since":1700016996,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":112},{"state_name":"creating+peering","count":19},{"state_name":"unknown","count":6}],"num_pgs":137,"num_pools":4,"num_objects":13,"data_bytes":1302,"bytes_used":2436554752,"bytes_avail":770657558528,"bytes_total":773094113280,"unknown_pgs_ratio":0.043795619159936905,"inactive_pgs_ratio":0.13868613541126251},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":4953}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-11-15T02:57:45.930917+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-15T02:57:50.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.139+0000 7f5800b47700 1 -- 172.21.15.121:0/2216814488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f57e005a540 msgr2=0x7f57e005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:50.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.139+0000 7f5800b47700 1 --2- 172.21.15.121:0/2216814488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f57e005a540 0x7f57e005ca00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f57e4009130 tx=0x7f57e4017000).stop 2023-11-15T02:57:50.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.139+0000 7f5800b47700 1 -- 172.21.15.121:0/2216814488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57fc0a9830 msgr2=0x7f57fc141170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:50.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.139+0000 7f5800b47700 1 --2- 172.21.15.121:0/2216814488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57fc0a9830 0x7f57fc141170 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f57ec00c4c0 tx=0x7f57ec007e00).stop 2023-11-15T02:57:50.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.139+0000 7f5800b47700 1 -- 172.21.15.121:0/2216814488 shutdown_connections 2023-11-15T02:57:50.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.139+0000 7f5800b47700 1 --2- 172.21.15.121:0/2216814488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f57e005a540 0x7f57e005ca00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:50.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.139+0000 7f5800b47700 1 --2- 172.21.15.121:0/2216814488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57fc151870 0x7f57fc0a81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:50.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.139+0000 7f5800b47700 1 --2- 172.21.15.121:0/2216814488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57fc0c30e0 0x7f57fc0a7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:50.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.139+0000 7f5800b47700 1 --2- 172.21.15.121:0/2216814488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57fc0a9830 0x7f57fc141170 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:50.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.139+0000 7f5800b47700 1 -- 172.21.15.121:0/2216814488 >> 172.21.15.121:0/2216814488 conn(0x7f57fc062190 msgr2=0x7f57fc150330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:50.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.139+0000 7f5800b47700 1 -- 172.21.15.121:0/2216814488 shutdown_connections 2023-11-15T02:57:50.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.139+0000 7f5800b47700 1 -- 172.21.15.121:0/2216814488 wait complete. 2023-11-15T02:57:50.165 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-15T02:57:50.166 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T02:57:50.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.315+0000 7f152f279700 1 Processor -- start 2023-11-15T02:57:50.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.315+0000 7f152f279700 1 -- start start 2023-11-15T02:57:50.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.319+0000 7f152f279700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1528157ef0 0x7f1528067140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:50.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.319+0000 7f152f279700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1528066370 0x7f15281535c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:50.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.319+0000 7f152f279700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1528066800 con 0x7f1528066d60 2023-11-15T02:57:50.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.319+0000 7f152f279700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15280c00f0 con 0x7f1528157ef0 2023-11-15T02:57:50.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.319+0000 7f152f279700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15281583f0 con 0x7f1528066370 2023-11-15T02:57:50.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.319+0000 7f152da76700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1528157ef0 0x7f1528067140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:50.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.319+0000 7f152da76700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1528157ef0 0x7f1528067140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:48684/0 (socket says 172.21.15.121:48684) 2023-11-15T02:57:50.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.319+0000 7f152da76700 1 -- 172.21.15.121:0/3602640608 learned_addr learned my addr 172.21.15.121:0/3602640608 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:50.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.319+0000 7f152ea78700 1 --2- 172.21.15.121:0/3602640608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1528066370 0x7f15281535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:50.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.319+0000 7f152da76700 1 -- 172.21.15.121:0/3602640608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1528066370 msgr2=0x7f15281535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:50.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.319+0000 7f152da76700 1 --2- 172.21.15.121:0/3602640608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1528066370 0x7f15281535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:50.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.319+0000 7f152da76700 1 -- 172.21.15.121:0/3602640608 >> v1:172.21.15.190:6789/0 conn(0x7f1528066d60 legacy=0x7f15281577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:50.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.319+0000 7f152da76700 1 -- 172.21.15.121:0/3602640608 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15281586d0 con 0x7f1528157ef0 2023-11-15T02:57:50.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.319+0000 7f152d275700 1 -- 172.21.15.121:0/3602640608 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3553486916 0 0) 0x7f1528066800 con 0x7f1528066d60 2023-11-15T02:57:50.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.319+0000 7f152da76700 1 --2- 172.21.15.121:0/3602640608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1528157ef0 0x7f1528067140 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f1518009830 tx=0x7f15180090f0).ready entity=mon.1 client_cookie=31cbdc9436ce3528 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:50.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.319+0000 7f152d275700 1 -- 172.21.15.121:0/3602640608 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151800b580 con 0x7f1528157ef0 2023-11-15T02:57:50.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.319+0000 7f152d275700 1 -- 172.21.15.121:0/3602640608 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151800b6e0 con 0x7f1528157ef0 2023-11-15T02:57:50.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.319+0000 7f152d275700 1 -- 172.21.15.121:0/3602640608 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151800ba60 con 0x7f1528157ef0 2023-11-15T02:57:50.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.323+0000 7f152f279700 1 -- 172.21.15.121:0/3602640608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1528157ef0 msgr2=0x7f1528067140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:50.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.323+0000 7f152f279700 1 --2- 172.21.15.121:0/3602640608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1528157ef0 0x7f1528067140 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f1518009830 tx=0x7f15180090f0).stop 2023-11-15T02:57:50.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.323+0000 7f152f279700 1 -- 172.21.15.121:0/3602640608 shutdown_connections 2023-11-15T02:57:50.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.323+0000 7f152f279700 1 --2- 172.21.15.121:0/3602640608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1528066370 0x7f15281535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:50.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.323+0000 7f152f279700 1 --2- 172.21.15.121:0/3602640608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1528157ef0 0x7f1528067140 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:50.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.323+0000 7f152f279700 1 -- 172.21.15.121:0/3602640608 >> 172.21.15.121:0/3602640608 conn(0x7f1528062190 msgr2=0x7f1528062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:50.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.323+0000 7f152f279700 1 -- 172.21.15.121:0/3602640608 shutdown_connections 2023-11-15T02:57:50.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.323+0000 7f152f279700 1 -- 172.21.15.121:0/3602640608 wait complete. 2023-11-15T02:57:50.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.327+0000 7f152f279700 1 Processor -- start 2023-11-15T02:57:50.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.327+0000 7f152f279700 1 -- start start 2023-11-15T02:57:50.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.327+0000 7f152f279700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1528066370 0x7f15280c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:50.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.327+0000 7f152f279700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1528066d60 0x7f15280c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:50.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.327+0000 7f152f279700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1528157ef0 0x7f15280c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:50.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.327+0000 7f152f279700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15280c65b0 con 0x7f1528066370 2023-11-15T02:57:50.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.327+0000 7f152f279700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1528168960 con 0x7f1528066d60 2023-11-15T02:57:50.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.327+0000 7f152f279700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15281542d0 con 0x7f1528157ef0 2023-11-15T02:57:50.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.327+0000 7f152e277700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1528066370 0x7f15280c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:50.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.327+0000 7f152ea78700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1528157ef0 0x7f15280c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:50.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.327+0000 7f152da76700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1528066d60 0x7f15280c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:50.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.327+0000 7f152e277700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1528066370 0x7f15280c4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56472/0 (socket says 172.21.15.121:56472) 2023-11-15T02:57:50.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.327+0000 7f152e277700 1 -- 172.21.15.121:0/57274613 learned_addr learned my addr 172.21.15.121:0/57274613 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:50.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.327+0000 7f152e277700 1 -- 172.21.15.121:0/57274613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1528157ef0 msgr2=0x7f15280c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:50.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.327+0000 7f152e277700 1 --2- 172.21.15.121:0/57274613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1528157ef0 0x7f15280c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:50.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.331+0000 7f152e277700 1 -- 172.21.15.121:0/57274613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1528066d60 msgr2=0x7f15280c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:50.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.331+0000 7f152e277700 1 --2- 172.21.15.121:0/57274613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1528066d60 0x7f15280c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:50.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.331+0000 7f152e277700 1 -- 172.21.15.121:0/57274613 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1518009370 con 0x7f1528066370 2023-11-15T02:57:50.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.331+0000 7f152e277700 1 --2- 172.21.15.121:0/57274613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1528066370 0x7f15280c4a30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f1524001eb0 tx=0x7f1524007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:50.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.331+0000 7f151e7fc700 1 -- 172.21.15.121:0/57274613 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152400ce60 con 0x7f1528066370 2023-11-15T02:57:50.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.331+0000 7f151e7fc700 1 -- 172.21.15.121:0/57274613 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152400f470 con 0x7f1528066370 2023-11-15T02:57:50.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.331+0000 7f152f279700 1 -- 172.21.15.121:0/57274613 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15281545b0 con 0x7f1528066370 2023-11-15T02:57:50.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.331+0000 7f151e7fc700 1 -- 172.21.15.121:0/57274613 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152400f810 con 0x7f1528066370 2023-11-15T02:57:50.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.331+0000 7f152f279700 1 -- 172.21.15.121:0/57274613 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15280c4f20 con 0x7f1528066370 2023-11-15T02:57:50.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.331+0000 7f152f279700 1 -- 172.21.15.121:0/57274613 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15280c6330 con 0x7f1528066370 2023-11-15T02:57:50.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.335+0000 7f151e7fc700 1 -- 172.21.15.121:0/57274613 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f152400fb80 con 0x7f1528066370 2023-11-15T02:57:50.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.335+0000 7f151e7fc700 1 --2- 172.21.15.121:0/57274613 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f150c05a510 0x7f150c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:50.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.335+0000 7f151e7fc700 1 -- 172.21.15.121:0/57274613 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1524073dc0 con 0x7f1528066370 2023-11-15T02:57:50.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.335+0000 7f152da76700 1 --2- 172.21.15.121:0/57274613 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f150c05a510 0x7f150c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:50.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.335+0000 7f152da76700 1 --2- 172.21.15.121:0/57274613 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f150c05a510 0x7f150c05c9d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f1528158320 tx=0x7f1518017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:50.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.339+0000 7f151e7fc700 1 -- 172.21.15.121:0/57274613 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f15240474e0 con 0x7f1528066370 2023-11-15T02:57:50.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.503+0000 7f152f279700 1 -- 172.21.15.121:0/57274613 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f152806bcf0 con 0x7f150c05a510 2023-11-15T02:57:50.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.523+0000 7f151e7fc700 1 -- 172.21.15.121:0/57274613 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344359 (secure 0 0 0) 0x7f152806bcf0 con 0x7f150c05a510 2023-11-15T02:57:50.531 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:50.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.531+0000 7f152f279700 1 -- 172.21.15.121:0/57274613 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f150c05a510 msgr2=0x7f150c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:50.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.531+0000 7f152f279700 1 --2- 172.21.15.121:0/57274613 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f150c05a510 0x7f150c05c9d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f1528158320 tx=0x7f1518017000).stop 2023-11-15T02:57:50.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.531+0000 7f152f279700 1 -- 172.21.15.121:0/57274613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1528066370 msgr2=0x7f15280c4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:50.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.531+0000 7f152f279700 1 --2- 172.21.15.121:0/57274613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1528066370 0x7f15280c4a30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f1524001eb0 tx=0x7f1524007c70).stop 2023-11-15T02:57:50.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.531+0000 7f152f279700 1 -- 172.21.15.121:0/57274613 shutdown_connections 2023-11-15T02:57:50.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.531+0000 7f152f279700 1 --2- 172.21.15.121:0/57274613 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f150c05a510 0x7f150c05c9d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:50.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.531+0000 7f152f279700 1 --2- 172.21.15.121:0/57274613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1528157ef0 0x7f15280c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:50.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.531+0000 7f152f279700 1 --2- 172.21.15.121:0/57274613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1528066d60 0x7f15280c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:50.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.531+0000 7f152f279700 1 --2- 172.21.15.121:0/57274613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1528066370 0x7f15280c4a30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:50.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.531+0000 7f152f279700 1 -- 172.21.15.121:0/57274613 >> 172.21.15.121:0/57274613 conn(0x7f1528062190 msgr2=0x7f1528152b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:50.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.531+0000 7f152f279700 1 -- 172.21.15.121:0/57274613 shutdown_connections 2023-11-15T02:57:50.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:50.531+0000 7f152f279700 1 -- 172.21.15.121:0/57274613 wait complete. 2023-11-15T02:57:50.540 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T02:57:50.557 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2023-11-15T02:57:49.932050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2379556,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752595164,"statfs":{"total":773094113280,"available":770657447936,"internally_reserved":0,"allocated":2854912,"data_stored":1151365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3857,"num_objects":-14,"num_object_clones":0,"num_object_copies":-28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-36,"num_write_kb":-20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.436442"},"pg_stats":[{"pgid":"6.3c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.576426+0000","last_change":"2023-11-15T02:57:44.576426+0000","last_active":"2023-11-15T02:57:44.576426+0000","last_peered":"2023-11-15T02:57:44.576426+0000","last_clean":"2023-11-15T02:57:44.576426+0000","last_became_active":"2023-11-15T02:57:44.572509+0000","last_became_peered":"2023-11-15T02:57:44.572509+0000","last_unstale":"2023-11-15T02:57:44.576426+0000","last_undegraded":"2023-11-15T02:57:44.576426+0000","last_fullsized":"2023-11-15T02:57:44.576426+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3f","version":"32'3","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.587176+0000","last_change":"2023-11-15T02:57:44.105284+0000","last_active":"2023-11-15T02:57:44.587176+0000","last_peered":"2023-11-15T02:57:44.587176+0000","last_clean":"2023-11-15T02:57:44.587176+0000","last_became_active":"2023-11-15T02:57:44.104935+0000","last_became_peered":"2023-11-15T02:57:44.104935+0000","last_unstale":"2023-11-15T02:57:44.587176+0000","last_undegraded":"2023-11-15T02:57:44.587176+0000","last_fullsized":"2023-11-15T02:57:44.587176+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566808+0000","last_change":"2023-11-15T02:57:44.566808+0000","last_active":"2023-11-15T02:57:44.566808+0000","last_peered":"2023-11-15T02:57:44.566808+0000","last_clean":"2023-11-15T02:57:44.566808+0000","last_became_active":"2023-11-15T02:57:44.566228+0000","last_became_peered":"2023-11-15T02:57:44.566228+0000","last_unstale":"2023-11-15T02:57:44.566808+0000","last_undegraded":"2023-11-15T02:57:44.566808+0000","last_fullsized":"2023-11-15T02:57:44.566808+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109248+0000","last_change":"2023-11-15T02:57:44.101869+0000","last_active":"2023-11-15T02:57:44.109248+0000","last_peered":"2023-11-15T02:57:44.109248+0000","last_clean":"2023-11-15T02:57:44.109248+0000","last_became_active":"2023-11-15T02:57:44.101602+0000","last_became_peered":"2023-11-15T02:57:44.101602+0000","last_unstale":"2023-11-15T02:57:44.109248+0000","last_undegraded":"2023-11-15T02:57:44.109248+0000","last_fullsized":"2023-11-15T02:57:44.109248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.564204+0000","last_change":"2023-11-15T02:57:44.564204+0000","last_active":"2023-11-15T02:57:44.564204+0000","last_peered":"2023-11-15T02:57:44.564204+0000","last_clean":"2023-11-15T02:57:44.564204+0000","last_became_active":"2023-11-15T02:57:44.563976+0000","last_became_peered":"2023-11-15T02:57:44.563976+0000","last_unstale":"2023-11-15T02:57:44.564204+0000","last_undegraded":"2023-11-15T02:57:44.564204+0000","last_fullsized":"2023-11-15T02:57:44.564204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.556695+0000","last_change":"2023-11-15T02:57:44.096033+0000","last_active":"2023-11-15T02:57:44.556695+0000","last_peered":"2023-11-15T02:57:44.556695+0000","last_clean":"2023-11-15T02:57:44.556695+0000","last_became_active":"2023-11-15T02:57:44.095626+0000","last_became_peered":"2023-11-15T02:57:44.095626+0000","last_unstale":"2023-11-15T02:57:44.556695+0000","last_undegraded":"2023-11-15T02:57:44.556695+0000","last_fullsized":"2023-11-15T02:57:44.556695+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.568907+0000","last_change":"2023-11-15T02:57:44.568907+0000","last_active":"2023-11-15T02:57:44.568907+0000","last_peered":"2023-11-15T02:57:44.568907+0000","last_clean":"2023-11-15T02:57:44.568907+0000","last_became_active":"2023-11-15T02:57:44.568621+0000","last_became_peered":"2023-11-15T02:57:44.568621+0000","last_unstale":"2023-11-15T02:57:44.568907+0000","last_undegraded":"2023-11-15T02:57:44.568907+0000","last_fullsized":"2023-11-15T02:57:44.568907+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3c","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.588835+0000","last_change":"2023-11-15T02:57:44.103658+0000","last_active":"2023-11-15T02:57:44.588835+0000","last_peered":"2023-11-15T02:57:44.588835+0000","last_clean":"2023-11-15T02:57:44.588835+0000","last_became_active":"2023-11-15T02:57:44.101877+0000","last_became_peered":"2023-11-15T02:57:44.101877+0000","last_unstale":"2023-11-15T02:57:44.588835+0000","last_undegraded":"2023-11-15T02:57:44.588835+0000","last_fullsized":"2023-11-15T02:57:44.588835+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.570582+0000","last_change":"2023-11-15T02:57:44.570582+0000","last_active":"2023-11-15T02:57:44.570582+0000","last_peered":"2023-11-15T02:57:44.570582+0000","last_clean":"2023-11-15T02:57:44.570582+0000","last_became_active":"2023-11-15T02:57:44.570314+0000","last_became_peered":"2023-11-15T02:57:44.570314+0000","last_unstale":"2023-11-15T02:57:44.570582+0000","last_undegraded":"2023-11-15T02:57:44.570582+0000","last_fullsized":"2023-11-15T02:57:44.570582+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3b","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560605+0000","last_change":"2023-11-15T02:57:44.104587+0000","last_active":"2023-11-15T02:57:44.560605+0000","last_peered":"2023-11-15T02:57:44.560605+0000","last_clean":"2023-11-15T02:57:44.560605+0000","last_became_active":"2023-11-15T02:57:44.104223+0000","last_became_peered":"2023-11-15T02:57:44.104223+0000","last_unstale":"2023-11-15T02:57:44.560605+0000","last_undegraded":"2023-11-15T02:57:44.560605+0000","last_fullsized":"2023-11-15T02:57:44.560605+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.593390+0000","last_change":"2023-11-15T02:57:44.584427+0000","last_active":"2023-11-15T02:57:44.593390+0000","last_peered":"2023-11-15T02:57:44.593390+0000","last_clean":"2023-11-15T02:57:44.593390+0000","last_became_active":"2023-11-15T02:57:44.584121+0000","last_became_peered":"2023-11-15T02:57:44.584121+0000","last_unstale":"2023-11-15T02:57:44.593390+0000","last_undegraded":"2023-11-15T02:57:44.593390+0000","last_fullsized":"2023-11-15T02:57:44.593390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.553452+0000","last_change":"2023-11-15T02:57:44.102587+0000","last_active":"2023-11-15T02:57:44.553452+0000","last_peered":"2023-11-15T02:57:44.553452+0000","last_clean":"2023-11-15T02:57:44.553452+0000","last_became_active":"2023-11-15T02:57:44.102311+0000","last_became_peered":"2023-11-15T02:57:44.102311+0000","last_unstale":"2023-11-15T02:57:44.553452+0000","last_undegraded":"2023-11-15T02:57:44.553452+0000","last_fullsized":"2023-11-15T02:57:44.553452+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.589914+0000","last_change":"2023-11-15T02:57:44.585731+0000","last_active":"2023-11-15T02:57:44.589914+0000","last_peered":"2023-11-15T02:57:44.589914+0000","last_clean":"2023-11-15T02:57:44.589914+0000","last_became_active":"2023-11-15T02:57:44.585470+0000","last_became_peered":"2023-11-15T02:57:44.585470+0000","last_unstale":"2023-11-15T02:57:44.589914+0000","last_undegraded":"2023-11-15T02:57:44.589914+0000","last_fullsized":"2023-11-15T02:57:44.589914+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.110877+0000","last_change":"2023-11-15T02:57:44.105206+0000","last_active":"2023-11-15T02:57:44.110877+0000","last_peered":"2023-11-15T02:57:44.110877+0000","last_clean":"2023-11-15T02:57:44.110877+0000","last_became_active":"2023-11-15T02:57:44.104936+0000","last_became_peered":"2023-11-15T02:57:44.104936+0000","last_unstale":"2023-11-15T02:57:44.110877+0000","last_undegraded":"2023-11-15T02:57:44.110877+0000","last_fullsized":"2023-11-15T02:57:44.110877+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566763+0000","last_change":"2023-11-15T02:57:44.566763+0000","last_active":"2023-11-15T02:57:44.566763+0000","last_peered":"2023-11-15T02:57:44.566763+0000","last_clean":"2023-11-15T02:57:44.566763+0000","last_became_active":"2023-11-15T02:57:44.566510+0000","last_became_peered":"2023-11-15T02:57:44.566510+0000","last_unstale":"2023-11-15T02:57:44.566763+0000","last_undegraded":"2023-11-15T02:57:44.566763+0000","last_fullsized":"2023-11-15T02:57:44.566763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.110691+0000","last_change":"2023-11-15T02:57:44.106233+0000","last_active":"2023-11-15T02:57:44.110691+0000","last_peered":"2023-11-15T02:57:44.110691+0000","last_clean":"2023-11-15T02:57:44.110691+0000","last_became_active":"2023-11-15T02:57:44.105985+0000","last_became_peered":"2023-11-15T02:57:44.105985+0000","last_unstale":"2023-11-15T02:57:44.110691+0000","last_undegraded":"2023-11-15T02:57:44.110691+0000","last_fullsized":"2023-11-15T02:57:44.110691+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577960+0000","last_change":"2023-11-15T02:57:44.577960+0000","last_active":"2023-11-15T02:57:44.577960+0000","last_peered":"2023-11-15T02:57:44.577960+0000","last_clean":"2023-11-15T02:57:44.577960+0000","last_became_active":"2023-11-15T02:57:44.577405+0000","last_became_peered":"2023-11-15T02:57:44.577405+0000","last_unstale":"2023-11-15T02:57:44.577960+0000","last_undegraded":"2023-11-15T02:57:44.577960+0000","last_fullsized":"2023-11-15T02:57:44.577960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.099195+0000","last_change":"2023-11-15T02:57:44.091037+0000","last_active":"2023-11-15T02:57:44.099195+0000","last_peered":"2023-11-15T02:57:44.099195+0000","last_clean":"2023-11-15T02:57:44.099195+0000","last_became_active":"2023-11-15T02:57:44.090808+0000","last_became_peered":"2023-11-15T02:57:44.090808+0000","last_unstale":"2023-11-15T02:57:44.099195+0000","last_undegraded":"2023-11-15T02:57:44.099195+0000","last_fullsized":"2023-11-15T02:57:44.099195+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.583352+0000","last_change":"2023-11-15T02:57:44.572175+0000","last_active":"2023-11-15T02:57:44.583352+0000","last_peered":"2023-11-15T02:57:44.583352+0000","last_clean":"2023-11-15T02:57:44.583352+0000","last_became_active":"2023-11-15T02:57:44.571339+0000","last_became_peered":"2023-11-15T02:57:44.571339+0000","last_unstale":"2023-11-15T02:57:44.583352+0000","last_undegraded":"2023-11-15T02:57:44.583352+0000","last_fullsized":"2023-11-15T02:57:44.583352+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.570535+0000","last_change":"2023-11-15T02:57:44.104971+0000","last_active":"2023-11-15T02:57:44.570535+0000","last_peered":"2023-11-15T02:57:44.570535+0000","last_clean":"2023-11-15T02:57:44.570535+0000","last_became_active":"2023-11-15T02:57:44.104610+0000","last_became_peered":"2023-11-15T02:57:44.104610+0000","last_unstale":"2023-11-15T02:57:44.570535+0000","last_undegraded":"2023-11-15T02:57:44.570535+0000","last_fullsized":"2023-11-15T02:57:44.570535+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.576019+0000","last_change":"2023-11-15T02:57:44.569668+0000","last_active":"2023-11-15T02:57:44.576019+0000","last_peered":"2023-11-15T02:57:44.576019+0000","last_clean":"2023-11-15T02:57:44.576019+0000","last_became_active":"2023-11-15T02:57:44.569379+0000","last_became_peered":"2023-11-15T02:57:44.569379+0000","last_unstale":"2023-11-15T02:57:44.576019+0000","last_undegraded":"2023-11-15T02:57:44.576019+0000","last_fullsized":"2023-11-15T02:57:44.576019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109579+0000","last_change":"2023-11-15T02:57:44.102619+0000","last_active":"2023-11-15T02:57:44.109579+0000","last_peered":"2023-11-15T02:57:44.109579+0000","last_clean":"2023-11-15T02:57:44.109579+0000","last_became_active":"2023-11-15T02:57:44.102328+0000","last_became_peered":"2023-11-15T02:57:44.102328+0000","last_unstale":"2023-11-15T02:57:44.109579+0000","last_undegraded":"2023-11-15T02:57:44.109579+0000","last_fullsized":"2023-11-15T02:57:44.109579+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.579068+0000","last_change":"2023-11-15T02:57:44.579068+0000","last_active":"2023-11-15T02:57:44.579068+0000","last_peered":"2023-11-15T02:57:44.579068+0000","last_clean":"2023-11-15T02:57:44.579068+0000","last_became_active":"2023-11-15T02:57:44.578815+0000","last_became_peered":"2023-11-15T02:57:44.578815+0000","last_unstale":"2023-11-15T02:57:44.579068+0000","last_undegraded":"2023-11-15T02:57:44.579068+0000","last_fullsized":"2023-11-15T02:57:44.579068+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"32'4","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.598725+0000","last_change":"2023-11-15T02:57:44.113636+0000","last_active":"2023-11-15T02:57:44.598725+0000","last_peered":"2023-11-15T02:57:44.598725+0000","last_clean":"2023-11-15T02:57:44.598725+0000","last_became_active":"2023-11-15T02:57:44.113283+0000","last_became_peered":"2023-11-15T02:57:44.113283+0000","last_unstale":"2023-11-15T02:57:44.598725+0000","last_undegraded":"2023-11-15T02:57:44.598725+0000","last_fullsized":"2023-11-15T02:57:44.598725+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.584681+0000","last_change":"2023-11-15T02:57:44.584681+0000","last_active":"2023-11-15T02:57:44.584681+0000","last_peered":"2023-11-15T02:57:44.584681+0000","last_clean":"2023-11-15T02:57:44.584681+0000","last_became_active":"2023-11-15T02:57:44.584395+0000","last_became_peered":"2023-11-15T02:57:44.584395+0000","last_unstale":"2023-11-15T02:57:44.584681+0000","last_undegraded":"2023-11-15T02:57:44.584681+0000","last_fullsized":"2023-11-15T02:57:44.584681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.33","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560690+0000","last_change":"2023-11-15T02:57:44.086072+0000","last_active":"2023-11-15T02:57:44.560690+0000","last_peered":"2023-11-15T02:57:44.560690+0000","last_clean":"2023-11-15T02:57:44.560690+0000","last_became_active":"2023-11-15T02:57:44.085574+0000","last_became_peered":"2023-11-15T02:57:44.085574+0000","last_unstale":"2023-11-15T02:57:44.560690+0000","last_undegraded":"2023-11-15T02:57:44.560690+0000","last_fullsized":"2023-11-15T02:57:44.560690+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.590256+0000","last_change":"2023-11-15T02:57:44.580438+0000","last_active":"2023-11-15T02:57:44.590256+0000","last_peered":"2023-11-15T02:57:44.590256+0000","last_clean":"2023-11-15T02:57:44.590256+0000","last_became_active":"2023-11-15T02:57:44.579977+0000","last_became_peered":"2023-11-15T02:57:44.579977+0000","last_unstale":"2023-11-15T02:57:44.590256+0000","last_undegraded":"2023-11-15T02:57:44.590256+0000","last_fullsized":"2023-11-15T02:57:44.590256+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.107875+0000","last_change":"2023-11-15T02:57:44.094640+0000","last_active":"2023-11-15T02:57:44.107875+0000","last_peered":"2023-11-15T02:57:44.107875+0000","last_clean":"2023-11-15T02:57:44.107875+0000","last_became_active":"2023-11-15T02:57:44.094279+0000","last_became_peered":"2023-11-15T02:57:44.094279+0000","last_unstale":"2023-11-15T02:57:44.107875+0000","last_undegraded":"2023-11-15T02:57:44.107875+0000","last_fullsized":"2023-11-15T02:57:44.107875+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.576818+0000","last_change":"2023-11-15T02:57:44.576818+0000","last_active":"2023-11-15T02:57:44.576818+0000","last_peered":"2023-11-15T02:57:44.576818+0000","last_clean":"2023-11-15T02:57:44.576818+0000","last_became_active":"2023-11-15T02:57:44.576074+0000","last_became_peered":"2023-11-15T02:57:44.576074+0000","last_unstale":"2023-11-15T02:57:44.576818+0000","last_undegraded":"2023-11-15T02:57:44.576818+0000","last_fullsized":"2023-11-15T02:57:44.576818+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.556584+0000","last_change":"2023-11-15T02:57:44.109239+0000","last_active":"2023-11-15T02:57:44.556584+0000","last_peered":"2023-11-15T02:57:44.556584+0000","last_clean":"2023-11-15T02:57:44.556584+0000","last_became_active":"2023-11-15T02:57:44.103686+0000","last_became_peered":"2023-11-15T02:57:44.103686+0000","last_unstale":"2023-11-15T02:57:44.556584+0000","last_undegraded":"2023-11-15T02:57:44.556584+0000","last_fullsized":"2023-11-15T02:57:44.556584+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.579784+0000","last_change":"2023-11-15T02:57:44.572530+0000","last_active":"2023-11-15T02:57:44.579784+0000","last_peered":"2023-11-15T02:57:44.579784+0000","last_clean":"2023-11-15T02:57:44.579784+0000","last_became_active":"2023-11-15T02:57:44.572016+0000","last_became_peered":"2023-11-15T02:57:44.572016+0000","last_unstale":"2023-11-15T02:57:44.579784+0000","last_undegraded":"2023-11-15T02:57:44.579784+0000","last_fullsized":"2023-11-15T02:57:44.579784+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"32'4","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585116+0000","last_change":"2023-11-15T02:57:44.113817+0000","last_active":"2023-11-15T02:57:44.585116+0000","last_peered":"2023-11-15T02:57:44.585116+0000","last_clean":"2023-11-15T02:57:44.585116+0000","last_became_active":"2023-11-15T02:57:44.112534+0000","last_became_peered":"2023-11-15T02:57:44.112534+0000","last_unstale":"2023-11-15T02:57:44.585116+0000","last_undegraded":"2023-11-15T02:57:44.585116+0000","last_fullsized":"2023-11-15T02:57:44.585116+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.575459+0000","last_change":"2023-11-15T02:57:44.575459+0000","last_active":"2023-11-15T02:57:44.575459+0000","last_peered":"2023-11-15T02:57:44.575459+0000","last_clean":"2023-11-15T02:57:44.575459+0000","last_became_active":"2023-11-15T02:57:44.575123+0000","last_became_peered":"2023-11-15T02:57:44.575123+0000","last_unstale":"2023-11-15T02:57:44.575459+0000","last_undegraded":"2023-11-15T02:57:44.575459+0000","last_fullsized":"2023-11-15T02:57:44.575459+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.111343+0000","last_change":"2023-11-15T02:57:44.107752+0000","last_active":"2023-11-15T02:57:44.111343+0000","last_peered":"2023-11-15T02:57:44.111343+0000","last_clean":"2023-11-15T02:57:44.111343+0000","last_became_active":"2023-11-15T02:57:44.107413+0000","last_became_peered":"2023-11-15T02:57:44.107413+0000","last_unstale":"2023-11-15T02:57:44.111343+0000","last_undegraded":"2023-11-15T02:57:44.111343+0000","last_fullsized":"2023-11-15T02:57:44.111343+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.572196+0000","last_change":"2023-11-15T02:57:44.572196+0000","last_active":"2023-11-15T02:57:44.572196+0000","last_peered":"2023-11-15T02:57:44.572196+0000","last_clean":"2023-11-15T02:57:44.572196+0000","last_became_active":"2023-11-15T02:57:44.571963+0000","last_became_peered":"2023-11-15T02:57:44.571963+0000","last_unstale":"2023-11-15T02:57:44.572196+0000","last_undegraded":"2023-11-15T02:57:44.572196+0000","last_fullsized":"2023-11-15T02:57:44.572196+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.106664+0000","last_change":"2023-11-15T02:57:44.097768+0000","last_active":"2023-11-15T02:57:44.106664+0000","last_peered":"2023-11-15T02:57:44.106664+0000","last_clean":"2023-11-15T02:57:44.106664+0000","last_became_active":"2023-11-15T02:57:44.097449+0000","last_became_peered":"2023-11-15T02:57:44.097449+0000","last_unstale":"2023-11-15T02:57:44.106664+0000","last_undegraded":"2023-11-15T02:57:44.106664+0000","last_fullsized":"2023-11-15T02:57:44.106664+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.588091+0000","last_change":"2023-11-15T02:57:44.588091+0000","last_active":"2023-11-15T02:57:44.588091+0000","last_peered":"2023-11-15T02:57:44.588091+0000","last_clean":"2023-11-15T02:57:44.588091+0000","last_became_active":"2023-11-15T02:57:44.587721+0000","last_became_peered":"2023-11-15T02:57:44.587721+0000","last_unstale":"2023-11-15T02:57:44.588091+0000","last_undegraded":"2023-11-15T02:57:44.588091+0000","last_fullsized":"2023-11-15T02:57:44.588091+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2d","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.551810+0000","last_change":"2023-11-15T02:57:44.097948+0000","last_active":"2023-11-15T02:57:44.551810+0000","last_peered":"2023-11-15T02:57:44.551810+0000","last_clean":"2023-11-15T02:57:44.551810+0000","last_became_active":"2023-11-15T02:57:44.097642+0000","last_became_peered":"2023-11-15T02:57:44.097642+0000","last_unstale":"2023-11-15T02:57:44.551810+0000","last_undegraded":"2023-11-15T02:57:44.551810+0000","last_fullsized":"2023-11-15T02:57:44.551810+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582841+0000","last_change":"2023-11-15T02:57:44.582841+0000","last_active":"2023-11-15T02:57:44.582841+0000","last_peered":"2023-11-15T02:57:44.582841+0000","last_clean":"2023-11-15T02:57:44.582841+0000","last_became_active":"2023-11-15T02:57:44.582592+0000","last_became_peered":"2023-11-15T02:57:44.582592+0000","last_unstale":"2023-11-15T02:57:44.582841+0000","last_undegraded":"2023-11-15T02:57:44.582841+0000","last_fullsized":"2023-11-15T02:57:44.582841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.101447+0000","last_change":"2023-11-15T02:57:44.095430+0000","last_active":"2023-11-15T02:57:44.101447+0000","last_peered":"2023-11-15T02:57:44.101447+0000","last_clean":"2023-11-15T02:57:44.101447+0000","last_became_active":"2023-11-15T02:57:44.094060+0000","last_became_peered":"2023-11-15T02:57:44.094060+0000","last_unstale":"2023-11-15T02:57:44.101447+0000","last_undegraded":"2023-11-15T02:57:44.101447+0000","last_fullsized":"2023-11-15T02:57:44.101447+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.571866+0000","last_change":"2023-11-15T02:57:44.571866+0000","last_active":"2023-11-15T02:57:44.571866+0000","last_peered":"2023-11-15T02:57:44.571866+0000","last_clean":"2023-11-15T02:57:44.571866+0000","last_became_active":"2023-11-15T02:57:44.571594+0000","last_became_peered":"2023-11-15T02:57:44.571594+0000","last_unstale":"2023-11-15T02:57:44.571866+0000","last_undegraded":"2023-11-15T02:57:44.571866+0000","last_fullsized":"2023-11-15T02:57:44.571866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.561370+0000","last_change":"2023-11-15T02:57:44.111471+0000","last_active":"2023-11-15T02:57:44.561370+0000","last_peered":"2023-11-15T02:57:44.561370+0000","last_clean":"2023-11-15T02:57:44.561370+0000","last_became_active":"2023-11-15T02:57:44.110158+0000","last_became_peered":"2023-11-15T02:57:44.110158+0000","last_unstale":"2023-11-15T02:57:44.561370+0000","last_undegraded":"2023-11-15T02:57:44.561370+0000","last_fullsized":"2023-11-15T02:57:44.561370+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.589275+0000","last_change":"2023-11-15T02:57:44.589275+0000","last_active":"2023-11-15T02:57:44.589275+0000","last_peered":"2023-11-15T02:57:44.589275+0000","last_clean":"2023-11-15T02:57:44.589275+0000","last_became_active":"2023-11-15T02:57:44.588964+0000","last_became_peered":"2023-11-15T02:57:44.588964+0000","last_unstale":"2023-11-15T02:57:44.589275+0000","last_undegraded":"2023-11-15T02:57:44.589275+0000","last_fullsized":"2023-11-15T02:57:44.589275+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.103617+0000","last_change":"2023-11-15T02:57:44.098842+0000","last_active":"2023-11-15T02:57:44.103617+0000","last_peered":"2023-11-15T02:57:44.103617+0000","last_clean":"2023-11-15T02:57:44.103617+0000","last_became_active":"2023-11-15T02:57:44.098539+0000","last_became_peered":"2023-11-15T02:57:44.098539+0000","last_unstale":"2023-11-15T02:57:44.103617+0000","last_undegraded":"2023-11-15T02:57:44.103617+0000","last_fullsized":"2023-11-15T02:57:44.103617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.571505+0000","last_change":"2023-11-15T02:57:44.571505+0000","last_active":"2023-11-15T02:57:44.571505+0000","last_peered":"2023-11-15T02:57:44.571505+0000","last_clean":"2023-11-15T02:57:44.571505+0000","last_became_active":"2023-11-15T02:57:44.570032+0000","last_became_peered":"2023-11-15T02:57:44.570032+0000","last_unstale":"2023-11-15T02:57:44.571505+0000","last_undegraded":"2023-11-15T02:57:44.571505+0000","last_fullsized":"2023-11-15T02:57:44.571505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.112480+0000","last_change":"2023-11-15T02:57:44.102777+0000","last_active":"2023-11-15T02:57:44.112480+0000","last_peered":"2023-11-15T02:57:44.112480+0000","last_clean":"2023-11-15T02:57:44.112480+0000","last_became_active":"2023-11-15T02:57:44.102485+0000","last_became_peered":"2023-11-15T02:57:44.102485+0000","last_unstale":"2023-11-15T02:57:44.112480+0000","last_undegraded":"2023-11-15T02:57:44.112480+0000","last_fullsized":"2023-11-15T02:57:44.112480+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582377+0000","last_change":"2023-11-15T02:57:44.575964+0000","last_active":"2023-11-15T02:57:44.582377+0000","last_peered":"2023-11-15T02:57:44.582377+0000","last_clean":"2023-11-15T02:57:44.582377+0000","last_became_active":"2023-11-15T02:57:44.572566+0000","last_became_peered":"2023-11-15T02:57:44.572566+0000","last_unstale":"2023-11-15T02:57:44.582377+0000","last_undegraded":"2023-11-15T02:57:44.582377+0000","last_fullsized":"2023-11-15T02:57:44.582377+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.555946+0000","last_change":"2023-11-15T02:57:44.100059+0000","last_active":"2023-11-15T02:57:44.555946+0000","last_peered":"2023-11-15T02:57:44.555946+0000","last_clean":"2023-11-15T02:57:44.555946+0000","last_became_active":"2023-11-15T02:57:44.099755+0000","last_became_peered":"2023-11-15T02:57:44.099755+0000","last_unstale":"2023-11-15T02:57:44.555946+0000","last_undegraded":"2023-11-15T02:57:44.555946+0000","last_fullsized":"2023-11-15T02:57:44.555946+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.593351+0000","last_change":"2023-11-15T02:57:44.584133+0000","last_active":"2023-11-15T02:57:44.593351+0000","last_peered":"2023-11-15T02:57:44.593351+0000","last_clean":"2023-11-15T02:57:44.593351+0000","last_became_active":"2023-11-15T02:57:44.583602+0000","last_became_peered":"2023-11-15T02:57:44.583602+0000","last_unstale":"2023-11-15T02:57:44.593351+0000","last_undegraded":"2023-11-15T02:57:44.593351+0000","last_fullsized":"2023-11-15T02:57:44.593351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.570688+0000","last_change":"2023-11-15T02:57:44.098313+0000","last_active":"2023-11-15T02:57:44.570688+0000","last_peered":"2023-11-15T02:57:44.570688+0000","last_clean":"2023-11-15T02:57:44.570688+0000","last_became_active":"2023-11-15T02:57:44.096267+0000","last_became_peered":"2023-11-15T02:57:44.096267+0000","last_unstale":"2023-11-15T02:57:44.570688+0000","last_undegraded":"2023-11-15T02:57:44.570688+0000","last_fullsized":"2023-11-15T02:57:44.570688+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577168+0000","last_change":"2023-11-15T02:57:44.577168+0000","last_active":"2023-11-15T02:57:44.577168+0000","last_peered":"2023-11-15T02:57:44.577168+0000","last_clean":"2023-11-15T02:57:44.577168+0000","last_became_active":"2023-11-15T02:57:44.576902+0000","last_became_peered":"2023-11-15T02:57:44.576902+0000","last_unstale":"2023-11-15T02:57:44.577168+0000","last_undegraded":"2023-11-15T02:57:44.577168+0000","last_fullsized":"2023-11-15T02:57:44.577168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.106529+0000","last_change":"2023-11-15T02:57:44.097229+0000","last_active":"2023-11-15T02:57:44.106529+0000","last_peered":"2023-11-15T02:57:44.106529+0000","last_clean":"2023-11-15T02:57:44.106529+0000","last_became_active":"2023-11-15T02:57:44.096765+0000","last_became_peered":"2023-11-15T02:57:44.096765+0000","last_unstale":"2023-11-15T02:57:44.106529+0000","last_undegraded":"2023-11-15T02:57:44.106529+0000","last_fullsized":"2023-11-15T02:57:44.106529+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.587387+0000","last_change":"2023-11-15T02:57:44.580512+0000","last_active":"2023-11-15T02:57:44.587387+0000","last_peered":"2023-11-15T02:57:44.587387+0000","last_clean":"2023-11-15T02:57:44.587387+0000","last_became_active":"2023-11-15T02:57:44.580249+0000","last_became_peered":"2023-11-15T02:57:44.580249+0000","last_unstale":"2023-11-15T02:57:44.587387+0000","last_undegraded":"2023-11-15T02:57:44.587387+0000","last_fullsized":"2023-11-15T02:57:44.587387+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.569612+0000","last_change":"2023-11-15T02:57:44.103956+0000","last_active":"2023-11-15T02:57:44.569612+0000","last_peered":"2023-11-15T02:57:44.569612+0000","last_clean":"2023-11-15T02:57:44.569612+0000","last_became_active":"2023-11-15T02:57:44.103187+0000","last_became_peered":"2023-11-15T02:57:44.103187+0000","last_unstale":"2023-11-15T02:57:44.569612+0000","last_undegraded":"2023-11-15T02:57:44.569612+0000","last_fullsized":"2023-11-15T02:57:44.569612+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.576398+0000","last_change":"2023-11-15T02:57:44.576398+0000","last_active":"2023-11-15T02:57:44.576398+0000","last_peered":"2023-11-15T02:57:44.576398+0000","last_clean":"2023-11-15T02:57:44.576398+0000","last_became_active":"2023-11-15T02:57:44.576039+0000","last_became_peered":"2023-11-15T02:57:44.576039+0000","last_unstale":"2023-11-15T02:57:44.576398+0000","last_undegraded":"2023-11-15T02:57:44.576398+0000","last_fullsized":"2023-11-15T02:57:44.576398+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.556438+0000","last_change":"2023-11-15T02:57:44.098514+0000","last_active":"2023-11-15T02:57:44.556438+0000","last_peered":"2023-11-15T02:57:44.556438+0000","last_clean":"2023-11-15T02:57:44.556438+0000","last_became_active":"2023-11-15T02:57:44.098120+0000","last_became_peered":"2023-11-15T02:57:44.098120+0000","last_unstale":"2023-11-15T02:57:44.556438+0000","last_undegraded":"2023-11-15T02:57:44.556438+0000","last_fullsized":"2023-11-15T02:57:44.556438+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.581009+0000","last_change":"2023-11-15T02:57:44.578170+0000","last_active":"2023-11-15T02:57:44.581009+0000","last_peered":"2023-11-15T02:57:44.581009+0000","last_clean":"2023-11-15T02:57:44.581009+0000","last_became_active":"2023-11-15T02:57:44.577800+0000","last_became_peered":"2023-11-15T02:57:44.577800+0000","last_unstale":"2023-11-15T02:57:44.581009+0000","last_undegraded":"2023-11-15T02:57:44.581009+0000","last_fullsized":"2023-11-15T02:57:44.581009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.108729+0000","last_change":"2023-11-15T02:57:44.101472+0000","last_active":"2023-11-15T02:57:44.108729+0000","last_peered":"2023-11-15T02:57:44.108729+0000","last_clean":"2023-11-15T02:57:44.108729+0000","last_became_active":"2023-11-15T02:57:44.101190+0000","last_became_peered":"2023-11-15T02:57:44.101190+0000","last_unstale":"2023-11-15T02:57:44.108729+0000","last_undegraded":"2023-11-15T02:57:44.108729+0000","last_fullsized":"2023-11-15T02:57:44.108729+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.589081+0000","last_change":"2023-11-15T02:57:44.589081+0000","last_active":"2023-11-15T02:57:44.589081+0000","last_peered":"2023-11-15T02:57:44.589081+0000","last_clean":"2023-11-15T02:57:44.589081+0000","last_became_active":"2023-11-15T02:57:44.588602+0000","last_became_peered":"2023-11-15T02:57:44.588602+0000","last_unstale":"2023-11-15T02:57:44.589081+0000","last_undegraded":"2023-11-15T02:57:44.589081+0000","last_fullsized":"2023-11-15T02:57:44.589081+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.108688+0000","last_change":"2023-11-15T02:57:44.101411+0000","last_active":"2023-11-15T02:57:44.108688+0000","last_peered":"2023-11-15T02:57:44.108688+0000","last_clean":"2023-11-15T02:57:44.108688+0000","last_became_active":"2023-11-15T02:57:44.099981+0000","last_became_peered":"2023-11-15T02:57:44.099981+0000","last_unstale":"2023-11-15T02:57:44.108688+0000","last_undegraded":"2023-11-15T02:57:44.108688+0000","last_fullsized":"2023-11-15T02:57:44.108688+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.588317+0000","last_change":"2023-11-15T02:57:44.588317+0000","last_active":"2023-11-15T02:57:44.588317+0000","last_peered":"2023-11-15T02:57:44.588317+0000","last_clean":"2023-11-15T02:57:44.588317+0000","last_became_active":"2023-11-15T02:57:44.586643+0000","last_became_peered":"2023-11-15T02:57:44.586643+0000","last_unstale":"2023-11-15T02:57:44.588317+0000","last_undegraded":"2023-11-15T02:57:44.588317+0000","last_fullsized":"2023-11-15T02:57:44.588317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.104386+0000","last_change":"2023-11-15T02:57:44.097212+0000","last_active":"2023-11-15T02:57:44.104386+0000","last_peered":"2023-11-15T02:57:44.104386+0000","last_clean":"2023-11-15T02:57:44.104386+0000","last_became_active":"2023-11-15T02:57:44.096903+0000","last_became_peered":"2023-11-15T02:57:44.096903+0000","last_unstale":"2023-11-15T02:57:44.104386+0000","last_undegraded":"2023-11-15T02:57:44.104386+0000","last_fullsized":"2023-11-15T02:57:44.104386+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.578257+0000","last_change":"2023-11-15T02:57:44.578257+0000","last_active":"2023-11-15T02:57:44.578257+0000","last_peered":"2023-11-15T02:57:44.578257+0000","last_clean":"2023-11-15T02:57:44.578257+0000","last_became_active":"2023-11-15T02:57:44.571560+0000","last_became_peered":"2023-11-15T02:57:44.571560+0000","last_unstale":"2023-11-15T02:57:44.578257+0000","last_undegraded":"2023-11-15T02:57:44.578257+0000","last_fullsized":"2023-11-15T02:57:44.578257+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.108660+0000","last_change":"2023-11-15T02:57:44.102174+0000","last_active":"2023-11-15T02:57:44.108660+0000","last_peered":"2023-11-15T02:57:44.108660+0000","last_clean":"2023-11-15T02:57:44.108660+0000","last_became_active":"2023-11-15T02:57:44.101794+0000","last_became_peered":"2023-11-15T02:57:44.101794+0000","last_unstale":"2023-11-15T02:57:44.108660+0000","last_undegraded":"2023-11-15T02:57:44.108660+0000","last_fullsized":"2023-11-15T02:57:44.108660+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.587132+0000","last_change":"2023-11-15T02:57:44.587132+0000","last_active":"2023-11-15T02:57:44.587132+0000","last_peered":"2023-11-15T02:57:44.587132+0000","last_clean":"2023-11-15T02:57:44.587132+0000","last_became_active":"2023-11-15T02:57:44.586820+0000","last_became_peered":"2023-11-15T02:57:44.586820+0000","last_unstale":"2023-11-15T02:57:44.587132+0000","last_undegraded":"2023-11-15T02:57:44.587132+0000","last_fullsized":"2023-11-15T02:57:44.587132+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.574296+0000","last_change":"2023-11-15T02:57:44.102584+0000","last_active":"2023-11-15T02:57:44.574296+0000","last_peered":"2023-11-15T02:57:44.574296+0000","last_clean":"2023-11-15T02:57:44.574296+0000","last_became_active":"2023-11-15T02:57:44.102267+0000","last_became_peered":"2023-11-15T02:57:44.102267+0000","last_unstale":"2023-11-15T02:57:44.574296+0000","last_undegraded":"2023-11-15T02:57:44.574296+0000","last_fullsized":"2023-11-15T02:57:44.574296+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.595125+0000","last_change":"2023-11-15T02:57:44.591963+0000","last_active":"2023-11-15T02:57:44.595125+0000","last_peered":"2023-11-15T02:57:44.595125+0000","last_clean":"2023-11-15T02:57:44.595125+0000","last_became_active":"2023-11-15T02:57:44.591725+0000","last_became_peered":"2023-11-15T02:57:44.591725+0000","last_unstale":"2023-11-15T02:57:44.595125+0000","last_undegraded":"2023-11-15T02:57:44.595125+0000","last_fullsized":"2023-11-15T02:57:44.595125+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.561164+0000","last_change":"2023-11-15T02:57:44.102680+0000","last_active":"2023-11-15T02:57:44.561164+0000","last_peered":"2023-11-15T02:57:44.561164+0000","last_clean":"2023-11-15T02:57:44.561164+0000","last_became_active":"2023-11-15T02:57:44.102394+0000","last_became_peered":"2023-11-15T02:57:44.102394+0000","last_unstale":"2023-11-15T02:57:44.561164+0000","last_undegraded":"2023-11-15T02:57:44.561164+0000","last_fullsized":"2023-11-15T02:57:44.561164+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.586592+0000","last_change":"2023-11-15T02:57:44.586592+0000","last_active":"2023-11-15T02:57:44.586592+0000","last_peered":"2023-11-15T02:57:44.586592+0000","last_clean":"2023-11-15T02:57:44.586592+0000","last_became_active":"2023-11-15T02:57:44.586211+0000","last_became_peered":"2023-11-15T02:57:44.586211+0000","last_unstale":"2023-11-15T02:57:44.586592+0000","last_undegraded":"2023-11-15T02:57:44.586592+0000","last_fullsized":"2023-11-15T02:57:44.586592+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.589223+0000","last_change":"2023-11-15T02:57:44.092034+0000","last_active":"2023-11-15T02:57:44.589223+0000","last_peered":"2023-11-15T02:57:44.589223+0000","last_clean":"2023-11-15T02:57:44.589223+0000","last_became_active":"2023-11-15T02:57:44.090952+0000","last_became_peered":"2023-11-15T02:57:44.090952+0000","last_unstale":"2023-11-15T02:57:44.589223+0000","last_undegraded":"2023-11-15T02:57:44.589223+0000","last_fullsized":"2023-11-15T02:57:44.589223+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.579124+0000","last_change":"2023-11-15T02:57:44.579124+0000","last_active":"2023-11-15T02:57:44.579124+0000","last_peered":"2023-11-15T02:57:44.579124+0000","last_clean":"2023-11-15T02:57:44.579124+0000","last_became_active":"2023-11-15T02:57:44.578824+0000","last_became_peered":"2023-11-15T02:57:44.578824+0000","last_unstale":"2023-11-15T02:57:44.579124+0000","last_undegraded":"2023-11-15T02:57:44.579124+0000","last_fullsized":"2023-11-15T02:57:44.579124+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.108976+0000","last_change":"2023-11-15T02:57:44.101425+0000","last_active":"2023-11-15T02:57:44.108976+0000","last_peered":"2023-11-15T02:57:44.108976+0000","last_clean":"2023-11-15T02:57:44.108976+0000","last_became_active":"2023-11-15T02:57:44.101139+0000","last_became_peered":"2023-11-15T02:57:44.101139+0000","last_unstale":"2023-11-15T02:57:44.108976+0000","last_undegraded":"2023-11-15T02:57:44.108976+0000","last_fullsized":"2023-11-15T02:57:44.108976+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.574408+0000","last_change":"2023-11-15T02:57:44.574408+0000","last_active":"2023-11-15T02:57:44.574408+0000","last_peered":"2023-11-15T02:57:44.574408+0000","last_clean":"2023-11-15T02:57:44.574408+0000","last_became_active":"2023-11-15T02:57:44.573651+0000","last_became_peered":"2023-11-15T02:57:44.573651+0000","last_unstale":"2023-11-15T02:57:44.574408+0000","last_undegraded":"2023-11-15T02:57:44.574408+0000","last_fullsized":"2023-11-15T02:57:44.574408+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.556340+0000","last_change":"2023-11-15T02:57:44.112809+0000","last_active":"2023-11-15T02:57:44.556340+0000","last_peered":"2023-11-15T02:57:44.556340+0000","last_clean":"2023-11-15T02:57:44.556340+0000","last_became_active":"2023-11-15T02:57:44.111134+0000","last_became_peered":"2023-11-15T02:57:44.111134+0000","last_unstale":"2023-11-15T02:57:44.556340+0000","last_undegraded":"2023-11-15T02:57:44.556340+0000","last_fullsized":"2023-11-15T02:57:44.556340+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.580330+0000","last_change":"2023-11-15T02:57:44.572751+0000","last_active":"2023-11-15T02:57:44.580330+0000","last_peered":"2023-11-15T02:57:44.580330+0000","last_clean":"2023-11-15T02:57:44.580330+0000","last_became_active":"2023-11-15T02:57:44.572322+0000","last_became_peered":"2023-11-15T02:57:44.572322+0000","last_unstale":"2023-11-15T02:57:44.580330+0000","last_undegraded":"2023-11-15T02:57:44.580330+0000","last_fullsized":"2023-11-15T02:57:44.580330+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.104280+0000","last_change":"2023-11-15T02:57:44.099996+0000","last_active":"2023-11-15T02:57:44.104280+0000","last_peered":"2023-11-15T02:57:44.104280+0000","last_clean":"2023-11-15T02:57:44.104280+0000","last_became_active":"2023-11-15T02:57:44.099695+0000","last_became_peered":"2023-11-15T02:57:44.099695+0000","last_unstale":"2023-11-15T02:57:44.104280+0000","last_undegraded":"2023-11-15T02:57:44.104280+0000","last_fullsized":"2023-11-15T02:57:44.104280+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577040+0000","last_change":"2023-11-15T02:57:44.577040+0000","last_active":"2023-11-15T02:57:44.577040+0000","last_peered":"2023-11-15T02:57:44.577040+0000","last_clean":"2023-11-15T02:57:44.577040+0000","last_became_active":"2023-11-15T02:57:44.576592+0000","last_became_peered":"2023-11-15T02:57:44.576592+0000","last_unstale":"2023-11-15T02:57:44.577040+0000","last_undegraded":"2023-11-15T02:57:44.577040+0000","last_fullsized":"2023-11-15T02:57:44.577040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566837+0000","last_change":"2023-11-15T02:57:44.086947+0000","last_active":"2023-11-15T02:57:44.566837+0000","last_peered":"2023-11-15T02:57:44.566837+0000","last_clean":"2023-11-15T02:57:44.566837+0000","last_became_active":"2023-11-15T02:57:44.086415+0000","last_became_peered":"2023-11-15T02:57:44.086415+0000","last_unstale":"2023-11-15T02:57:44.566837+0000","last_undegraded":"2023-11-15T02:57:44.566837+0000","last_fullsized":"2023-11-15T02:57:44.566837+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.571061+0000","last_change":"2023-11-15T02:57:44.571061+0000","last_active":"2023-11-15T02:57:44.571061+0000","last_peered":"2023-11-15T02:57:44.571061+0000","last_clean":"2023-11-15T02:57:44.571061+0000","last_became_active":"2023-11-15T02:57:44.570802+0000","last_became_peered":"2023-11-15T02:57:44.570802+0000","last_unstale":"2023-11-15T02:57:44.571061+0000","last_undegraded":"2023-11-15T02:57:44.571061+0000","last_fullsized":"2023-11-15T02:57:44.571061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.103937+0000","last_change":"2023-11-15T02:57:44.099371+0000","last_active":"2023-11-15T02:57:44.103937+0000","last_peered":"2023-11-15T02:57:44.103937+0000","last_clean":"2023-11-15T02:57:44.103937+0000","last_became_active":"2023-11-15T02:57:44.096689+0000","last_became_peered":"2023-11-15T02:57:44.096689+0000","last_unstale":"2023-11-15T02:57:44.103937+0000","last_undegraded":"2023-11-15T02:57:44.103937+0000","last_fullsized":"2023-11-15T02:57:44.103937+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.572799+0000","last_change":"2023-11-15T02:57:44.572799+0000","last_active":"2023-11-15T02:57:44.572799+0000","last_peered":"2023-11-15T02:57:44.572799+0000","last_clean":"2023-11-15T02:57:44.572799+0000","last_became_active":"2023-11-15T02:57:44.572147+0000","last_became_peered":"2023-11-15T02:57:44.572147+0000","last_unstale":"2023-11-15T02:57:44.572799+0000","last_undegraded":"2023-11-15T02:57:44.572799+0000","last_fullsized":"2023-11-15T02:57:44.572799+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.105678+0000","last_change":"2023-11-15T02:57:44.094718+0000","last_active":"2023-11-15T02:57:44.105678+0000","last_peered":"2023-11-15T02:57:44.105678+0000","last_clean":"2023-11-15T02:57:44.105678+0000","last_became_active":"2023-11-15T02:57:44.094236+0000","last_became_peered":"2023-11-15T02:57:44.094236+0000","last_unstale":"2023-11-15T02:57:44.105678+0000","last_undegraded":"2023-11-15T02:57:44.105678+0000","last_fullsized":"2023-11-15T02:57:44.105678+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577901+0000","last_change":"2023-11-15T02:57:44.577901+0000","last_active":"2023-11-15T02:57:44.577901+0000","last_peered":"2023-11-15T02:57:44.577901+0000","last_clean":"2023-11-15T02:57:44.577901+0000","last_became_active":"2023-11-15T02:57:44.577370+0000","last_became_peered":"2023-11-15T02:57:44.577370+0000","last_unstale":"2023-11-15T02:57:44.577901+0000","last_undegraded":"2023-11-15T02:57:44.577901+0000","last_fullsized":"2023-11-15T02:57:44.577901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.556365+0000","last_change":"2023-11-15T02:57:44.109398+0000","last_active":"2023-11-15T02:57:44.556365+0000","last_peered":"2023-11-15T02:57:44.556365+0000","last_clean":"2023-11-15T02:57:44.556365+0000","last_became_active":"2023-11-15T02:57:44.104880+0000","last_became_peered":"2023-11-15T02:57:44.104880+0000","last_unstale":"2023-11-15T02:57:44.556365+0000","last_undegraded":"2023-11-15T02:57:44.556365+0000","last_fullsized":"2023-11-15T02:57:44.556365+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.570258+0000","last_change":"2023-11-15T02:57:44.570258+0000","last_active":"2023-11-15T02:57:44.570258+0000","last_peered":"2023-11-15T02:57:44.570258+0000","last_clean":"2023-11-15T02:57:44.570258+0000","last_became_active":"2023-11-15T02:57:44.569983+0000","last_became_peered":"2023-11-15T02:57:44.569983+0000","last_unstale":"2023-11-15T02:57:44.570258+0000","last_undegraded":"2023-11-15T02:57:44.570258+0000","last_fullsized":"2023-11-15T02:57:44.570258+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"32'3","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.598694+0000","last_change":"2023-11-15T02:57:44.096037+0000","last_active":"2023-11-15T02:57:44.598694+0000","last_peered":"2023-11-15T02:57:44.598694+0000","last_clean":"2023-11-15T02:57:44.598694+0000","last_became_active":"2023-11-15T02:57:44.095669+0000","last_became_peered":"2023-11-15T02:57:44.095669+0000","last_unstale":"2023-11-15T02:57:44.598694+0000","last_undegraded":"2023-11-15T02:57:44.598694+0000","last_fullsized":"2023-11-15T02:57:44.598694+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585203+0000","last_change":"2023-11-15T02:57:44.585203+0000","last_active":"2023-11-15T02:57:44.585203+0000","last_peered":"2023-11-15T02:57:44.585203+0000","last_clean":"2023-11-15T02:57:44.585203+0000","last_became_active":"2023-11-15T02:57:44.583007+0000","last_became_peered":"2023-11-15T02:57:44.583007+0000","last_unstale":"2023-11-15T02:57:44.585203+0000","last_undegraded":"2023-11-15T02:57:44.585203+0000","last_fullsized":"2023-11-15T02:57:44.585203+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.581336+0000","last_change":"2023-11-15T02:57:44.096764+0000","last_active":"2023-11-15T02:57:44.581336+0000","last_peered":"2023-11-15T02:57:44.581336+0000","last_clean":"2023-11-15T02:57:44.581336+0000","last_became_active":"2023-11-15T02:57:44.096348+0000","last_became_peered":"2023-11-15T02:57:44.096348+0000","last_unstale":"2023-11-15T02:57:44.581336+0000","last_undegraded":"2023-11-15T02:57:44.581336+0000","last_fullsized":"2023-11-15T02:57:44.581336+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566200+0000","last_change":"2023-11-15T02:57:44.566200+0000","last_active":"2023-11-15T02:57:44.566200+0000","last_peered":"2023-11-15T02:57:44.566200+0000","last_clean":"2023-11-15T02:57:44.566200+0000","last_became_active":"2023-11-15T02:57:44.565943+0000","last_became_peered":"2023-11-15T02:57:44.565943+0000","last_unstale":"2023-11-15T02:57:44.566200+0000","last_undegraded":"2023-11-15T02:57:44.566200+0000","last_fullsized":"2023-11-15T02:57:44.566200+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.553334+0000","last_change":"2023-11-15T02:57:44.109346+0000","last_active":"2023-11-15T02:57:44.553334+0000","last_peered":"2023-11-15T02:57:44.553334+0000","last_clean":"2023-11-15T02:57:44.553334+0000","last_became_active":"2023-11-15T02:57:44.109023+0000","last_became_peered":"2023-11-15T02:57:44.109023+0000","last_unstale":"2023-11-15T02:57:44.553334+0000","last_undegraded":"2023-11-15T02:57:44.553334+0000","last_fullsized":"2023-11-15T02:57:44.553334+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566248+0000","last_change":"2023-11-15T02:57:44.566248+0000","last_active":"2023-11-15T02:57:44.566248+0000","last_peered":"2023-11-15T02:57:44.566248+0000","last_clean":"2023-11-15T02:57:44.566248+0000","last_became_active":"2023-11-15T02:57:44.566002+0000","last_became_peered":"2023-11-15T02:57:44.566002+0000","last_unstale":"2023-11-15T02:57:44.566248+0000","last_undegraded":"2023-11-15T02:57:44.566248+0000","last_fullsized":"2023-11-15T02:57:44.566248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.567066+0000","last_change":"2023-11-15T02:57:44.093738+0000","last_active":"2023-11-15T02:57:44.567066+0000","last_peered":"2023-11-15T02:57:44.567066+0000","last_clean":"2023-11-15T02:57:44.567066+0000","last_became_active":"2023-11-15T02:57:44.093452+0000","last_became_peered":"2023-11-15T02:57:44.093452+0000","last_unstale":"2023-11-15T02:57:44.567066+0000","last_undegraded":"2023-11-15T02:57:44.567066+0000","last_fullsized":"2023-11-15T02:57:44.567066+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.588350+0000","last_change":"2023-11-15T02:57:44.588350+0000","last_active":"2023-11-15T02:57:44.588350+0000","last_peered":"2023-11-15T02:57:44.588350+0000","last_clean":"2023-11-15T02:57:44.588350+0000","last_became_active":"2023-11-15T02:57:44.588073+0000","last_became_peered":"2023-11-15T02:57:44.588073+0000","last_unstale":"2023-11-15T02:57:44.588350+0000","last_undegraded":"2023-11-15T02:57:44.588350+0000","last_fullsized":"2023-11-15T02:57:44.588350+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109706+0000","last_change":"2023-11-15T02:57:44.101276+0000","last_active":"2023-11-15T02:57:44.109706+0000","last_peered":"2023-11-15T02:57:44.109706+0000","last_clean":"2023-11-15T02:57:44.109706+0000","last_became_active":"2023-11-15T02:57:44.100908+0000","last_became_peered":"2023-11-15T02:57:44.100908+0000","last_unstale":"2023-11-15T02:57:44.109706+0000","last_undegraded":"2023-11-15T02:57:44.109706+0000","last_fullsized":"2023-11-15T02:57:44.109706+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566177+0000","last_change":"2023-11-15T02:57:44.566177+0000","last_active":"2023-11-15T02:57:44.566177+0000","last_peered":"2023-11-15T02:57:44.566177+0000","last_clean":"2023-11-15T02:57:44.566177+0000","last_became_active":"2023-11-15T02:57:44.565917+0000","last_became_peered":"2023-11-15T02:57:44.565917+0000","last_unstale":"2023-11-15T02:57:44.566177+0000","last_undegraded":"2023-11-15T02:57:44.566177+0000","last_fullsized":"2023-11-15T02:57:44.566177+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"32'4","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.586189+0000","last_change":"2023-11-15T02:57:44.102229+0000","last_active":"2023-11-15T02:57:44.586189+0000","last_peered":"2023-11-15T02:57:44.586189+0000","last_clean":"2023-11-15T02:57:44.586189+0000","last_became_active":"2023-11-15T02:57:44.101858+0000","last_became_peered":"2023-11-15T02:57:44.101858+0000","last_unstale":"2023-11-15T02:57:44.586189+0000","last_undegraded":"2023-11-15T02:57:44.586189+0000","last_fullsized":"2023-11-15T02:57:44.586189+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.578142+0000","last_change":"2023-11-15T02:57:44.578142+0000","last_active":"2023-11-15T02:57:44.578142+0000","last_peered":"2023-11-15T02:57:44.578142+0000","last_clean":"2023-11-15T02:57:44.578142+0000","last_became_active":"2023-11-15T02:57:44.575328+0000","last_became_peered":"2023-11-15T02:57:44.575328+0000","last_unstale":"2023-11-15T02:57:44.578142+0000","last_undegraded":"2023-11-15T02:57:44.578142+0000","last_fullsized":"2023-11-15T02:57:44.578142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.593940+0000","last_change":"2023-11-15T02:57:44.100857+0000","last_active":"2023-11-15T02:57:44.593940+0000","last_peered":"2023-11-15T02:57:44.593940+0000","last_clean":"2023-11-15T02:57:44.593940+0000","last_became_active":"2023-11-15T02:57:44.099959+0000","last_became_peered":"2023-11-15T02:57:44.099959+0000","last_unstale":"2023-11-15T02:57:44.593940+0000","last_undegraded":"2023-11-15T02:57:44.593940+0000","last_fullsized":"2023-11-15T02:57:44.593940+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.583371+0000","last_change":"2023-11-15T02:57:44.576443+0000","last_active":"2023-11-15T02:57:44.583371+0000","last_peered":"2023-11-15T02:57:44.583371+0000","last_clean":"2023-11-15T02:57:44.583371+0000","last_became_active":"2023-11-15T02:57:44.576085+0000","last_became_peered":"2023-11-15T02:57:44.576085+0000","last_unstale":"2023-11-15T02:57:44.583371+0000","last_undegraded":"2023-11-15T02:57:44.583371+0000","last_fullsized":"2023-11-15T02:57:44.583371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585609+0000","last_change":"2023-11-15T02:57:44.096009+0000","last_active":"2023-11-15T02:57:44.585609+0000","last_peered":"2023-11-15T02:57:44.585609+0000","last_clean":"2023-11-15T02:57:44.585609+0000","last_became_active":"2023-11-15T02:57:44.095541+0000","last_became_peered":"2023-11-15T02:57:44.095541+0000","last_unstale":"2023-11-15T02:57:44.585609+0000","last_undegraded":"2023-11-15T02:57:44.585609+0000","last_fullsized":"2023-11-15T02:57:44.585609+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.583166+0000","last_change":"2023-11-15T02:57:44.583166+0000","last_active":"2023-11-15T02:57:44.583166+0000","last_peered":"2023-11-15T02:57:44.583166+0000","last_clean":"2023-11-15T02:57:44.583166+0000","last_became_active":"2023-11-15T02:57:44.582842+0000","last_became_peered":"2023-11-15T02:57:44.582842+0000","last_unstale":"2023-11-15T02:57:44.583166+0000","last_undegraded":"2023-11-15T02:57:44.583166+0000","last_fullsized":"2023-11-15T02:57:44.583166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.109552+0000","last_change":"2023-11-15T02:57:44.101593+0000","last_active":"2023-11-15T02:57:44.109552+0000","last_peered":"2023-11-15T02:57:44.109552+0000","last_clean":"2023-11-15T02:57:44.109552+0000","last_became_active":"2023-11-15T02:57:44.101261+0000","last_became_peered":"2023-11-15T02:57:44.101261+0000","last_unstale":"2023-11-15T02:57:44.109552+0000","last_undegraded":"2023-11-15T02:57:44.109552+0000","last_fullsized":"2023-11-15T02:57:44.109552+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.584677+0000","last_change":"2023-11-15T02:57:44.584677+0000","last_active":"2023-11-15T02:57:44.584677+0000","last_peered":"2023-11-15T02:57:44.584677+0000","last_clean":"2023-11-15T02:57:44.584677+0000","last_became_active":"2023-11-15T02:57:44.584394+0000","last_became_peered":"2023-11-15T02:57:44.584394+0000","last_unstale":"2023-11-15T02:57:44.584677+0000","last_undegraded":"2023-11-15T02:57:44.584677+0000","last_fullsized":"2023-11-15T02:57:44.584677+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.567985+0000","last_change":"2023-11-15T02:57:44.104181+0000","last_active":"2023-11-15T02:57:44.567985+0000","last_peered":"2023-11-15T02:57:44.567985+0000","last_clean":"2023-11-15T02:57:44.567985+0000","last_became_active":"2023-11-15T02:57:44.100596+0000","last_became_peered":"2023-11-15T02:57:44.100596+0000","last_unstale":"2023-11-15T02:57:44.567985+0000","last_undegraded":"2023-11-15T02:57:44.567985+0000","last_fullsized":"2023-11-15T02:57:44.567985+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.590180+0000","last_change":"2023-11-15T02:57:44.590180+0000","last_active":"2023-11-15T02:57:44.590180+0000","last_peered":"2023-11-15T02:57:44.590180+0000","last_clean":"2023-11-15T02:57:44.590180+0000","last_became_active":"2023-11-15T02:57:44.589752+0000","last_became_peered":"2023-11-15T02:57:44.589752+0000","last_unstale":"2023-11-15T02:57:44.590180+0000","last_undegraded":"2023-11-15T02:57:44.590180+0000","last_fullsized":"2023-11-15T02:57:44.590180+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560777+0000","last_change":"2023-11-15T02:57:44.109252+0000","last_active":"2023-11-15T02:57:44.560777+0000","last_peered":"2023-11-15T02:57:44.560777+0000","last_clean":"2023-11-15T02:57:44.560777+0000","last_became_active":"2023-11-15T02:57:44.108907+0000","last_became_peered":"2023-11-15T02:57:44.108907+0000","last_unstale":"2023-11-15T02:57:44.560777+0000","last_undegraded":"2023-11-15T02:57:44.560777+0000","last_fullsized":"2023-11-15T02:57:44.560777+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.568542+0000","last_change":"2023-11-15T02:57:44.568542+0000","last_active":"2023-11-15T02:57:44.568542+0000","last_peered":"2023-11-15T02:57:44.568542+0000","last_clean":"2023-11-15T02:57:44.568542+0000","last_became_active":"2023-11-15T02:57:44.568269+0000","last_became_peered":"2023-11-15T02:57:44.568269+0000","last_unstale":"2023-11-15T02:57:44.568542+0000","last_undegraded":"2023-11-15T02:57:44.568542+0000","last_fullsized":"2023-11-15T02:57:44.568542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.103890+0000","last_change":"2023-11-15T02:57:44.097937+0000","last_active":"2023-11-15T02:57:44.103890+0000","last_peered":"2023-11-15T02:57:44.103890+0000","last_clean":"2023-11-15T02:57:44.103890+0000","last_became_active":"2023-11-15T02:57:44.097630+0000","last_became_peered":"2023-11-15T02:57:44.097630+0000","last_unstale":"2023-11-15T02:57:44.103890+0000","last_undegraded":"2023-11-15T02:57:44.103890+0000","last_fullsized":"2023-11-15T02:57:44.103890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582522+0000","last_change":"2023-11-15T02:57:44.579127+0000","last_active":"2023-11-15T02:57:44.582522+0000","last_peered":"2023-11-15T02:57:44.582522+0000","last_clean":"2023-11-15T02:57:44.582522+0000","last_became_active":"2023-11-15T02:57:44.578854+0000","last_became_peered":"2023-11-15T02:57:44.578854+0000","last_unstale":"2023-11-15T02:57:44.582522+0000","last_undegraded":"2023-11-15T02:57:44.582522+0000","last_fullsized":"2023-11-15T02:57:44.582522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.111403+0000","last_change":"2023-11-15T02:57:44.108255+0000","last_active":"2023-11-15T02:57:44.111403+0000","last_peered":"2023-11-15T02:57:44.111403+0000","last_clean":"2023-11-15T02:57:44.111403+0000","last_became_active":"2023-11-15T02:57:44.107961+0000","last_became_peered":"2023-11-15T02:57:44.107961+0000","last_unstale":"2023-11-15T02:57:44.111403+0000","last_undegraded":"2023-11-15T02:57:44.111403+0000","last_fullsized":"2023-11-15T02:57:44.111403+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.570926+0000","last_change":"2023-11-15T02:57:44.570926+0000","last_active":"2023-11-15T02:57:44.570926+0000","last_peered":"2023-11-15T02:57:44.570926+0000","last_clean":"2023-11-15T02:57:44.570926+0000","last_became_active":"2023-11-15T02:57:44.570646+0000","last_became_peered":"2023-11-15T02:57:44.570646+0000","last_unstale":"2023-11-15T02:57:44.570926+0000","last_undegraded":"2023-11-15T02:57:44.570926+0000","last_fullsized":"2023-11-15T02:57:44.570926+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.111257+0000","last_change":"2023-11-15T02:57:44.105564+0000","last_active":"2023-11-15T02:57:44.111257+0000","last_peered":"2023-11-15T02:57:44.111257+0000","last_clean":"2023-11-15T02:57:44.111257+0000","last_became_active":"2023-11-15T02:57:44.105312+0000","last_became_peered":"2023-11-15T02:57:44.105312+0000","last_unstale":"2023-11-15T02:57:44.111257+0000","last_undegraded":"2023-11-15T02:57:44.111257+0000","last_fullsized":"2023-11-15T02:57:44.111257+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.580419+0000","last_change":"2023-11-15T02:57:44.580419+0000","last_active":"2023-11-15T02:57:44.580419+0000","last_peered":"2023-11-15T02:57:44.580419+0000","last_clean":"2023-11-15T02:57:44.580419+0000","last_became_active":"2023-11-15T02:57:44.580057+0000","last_became_peered":"2023-11-15T02:57:44.580057+0000","last_unstale":"2023-11-15T02:57:44.580419+0000","last_undegraded":"2023-11-15T02:57:44.580419+0000","last_fullsized":"2023-11-15T02:57:44.580419+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.107016+0000","last_change":"2023-11-15T02:57:44.097433+0000","last_active":"2023-11-15T02:57:44.107016+0000","last_peered":"2023-11-15T02:57:44.107016+0000","last_clean":"2023-11-15T02:57:44.107016+0000","last_became_active":"2023-11-15T02:57:44.097149+0000","last_became_peered":"2023-11-15T02:57:44.097149+0000","last_unstale":"2023-11-15T02:57:44.107016+0000","last_undegraded":"2023-11-15T02:57:44.107016+0000","last_fullsized":"2023-11-15T02:57:44.107016+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.090441+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T02:57:44.090441+0000","last_peered":"2023-11-15T02:57:44.090441+0000","last_clean":"2023-11-15T02:57:44.090441+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T02:57:44.090441+0000","last_undegraded":"2023-11-15T02:57:44.090441+0000","last_fullsized":"2023-11-15T02:57:44.090441+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.584602+0000","last_change":"2023-11-15T02:57:44.584602+0000","last_active":"2023-11-15T02:57:44.584602+0000","last_peered":"2023-11-15T02:57:44.584602+0000","last_clean":"2023-11-15T02:57:44.584602+0000","last_became_active":"2023-11-15T02:57:44.584333+0000","last_became_peered":"2023-11-15T02:57:44.584333+0000","last_unstale":"2023-11-15T02:57:44.584602+0000","last_undegraded":"2023-11-15T02:57:44.584602+0000","last_fullsized":"2023-11-15T02:57:44.584602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.104126+0000","last_change":"2023-11-15T02:57:44.099697+0000","last_active":"2023-11-15T02:57:44.104126+0000","last_peered":"2023-11-15T02:57:44.104126+0000","last_clean":"2023-11-15T02:57:44.104126+0000","last_became_active":"2023-11-15T02:57:44.099388+0000","last_became_peered":"2023-11-15T02:57:44.099388+0000","last_unstale":"2023-11-15T02:57:44.104126+0000","last_undegraded":"2023-11-15T02:57:44.104126+0000","last_fullsized":"2023-11-15T02:57:44.104126+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.570736+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T02:57:44.570736+0000","last_peered":"2023-11-15T02:57:44.570736+0000","last_clean":"2023-11-15T02:57:44.570736+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T02:57:44.570736+0000","last_undegraded":"2023-11-15T02:57:44.570736+0000","last_fullsized":"2023-11-15T02:57:44.570736+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577326+0000","last_change":"2023-11-15T02:57:44.573248+0000","last_active":"2023-11-15T02:57:44.577326+0000","last_peered":"2023-11-15T02:57:44.577326+0000","last_clean":"2023-11-15T02:57:44.577326+0000","last_became_active":"2023-11-15T02:57:44.572993+0000","last_became_peered":"2023-11-15T02:57:44.572993+0000","last_unstale":"2023-11-15T02:57:44.577326+0000","last_undegraded":"2023-11-15T02:57:44.577326+0000","last_fullsized":"2023-11-15T02:57:44.577326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.105522+0000","last_change":"2023-11-15T02:57:44.097789+0000","last_active":"2023-11-15T02:57:44.105522+0000","last_peered":"2023-11-15T02:57:44.105522+0000","last_clean":"2023-11-15T02:57:44.105522+0000","last_became_active":"2023-11-15T02:57:44.097133+0000","last_became_peered":"2023-11-15T02:57:44.097133+0000","last_unstale":"2023-11-15T02:57:44.105522+0000","last_undegraded":"2023-11-15T02:57:44.105522+0000","last_fullsized":"2023-11-15T02:57:44.105522+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.090466+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T02:57:44.090466+0000","last_peered":"2023-11-15T02:57:44.090466+0000","last_clean":"2023-11-15T02:57:44.090466+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T02:57:44.090466+0000","last_undegraded":"2023-11-15T02:57:44.090466+0000","last_fullsized":"2023-11-15T02:57:44.090466+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582488+0000","last_change":"2023-11-15T02:57:44.582488+0000","last_active":"2023-11-15T02:57:44.582488+0000","last_peered":"2023-11-15T02:57:44.582488+0000","last_clean":"2023-11-15T02:57:44.582488+0000","last_became_active":"2023-11-15T02:57:44.582160+0000","last_became_peered":"2023-11-15T02:57:44.582160+0000","last_unstale":"2023-11-15T02:57:44.582488+0000","last_undegraded":"2023-11-15T02:57:44.582488+0000","last_fullsized":"2023-11-15T02:57:44.582488+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.113534+0000","last_change":"2023-11-15T02:57:44.110737+0000","last_active":"2023-11-15T02:57:44.113534+0000","last_peered":"2023-11-15T02:57:44.113534+0000","last_clean":"2023-11-15T02:57:44.113534+0000","last_became_active":"2023-11-15T02:57:44.110423+0000","last_became_peered":"2023-11-15T02:57:44.110423+0000","last_unstale":"2023-11-15T02:57:44.113534+0000","last_undegraded":"2023-11-15T02:57:44.113534+0000","last_fullsized":"2023-11-15T02:57:44.113534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.568173+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T02:57:44.568173+0000","last_peered":"2023-11-15T02:57:44.568173+0000","last_clean":"2023-11-15T02:57:44.568173+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T02:57:44.568173+0000","last_undegraded":"2023-11-15T02:57:44.568173+0000","last_fullsized":"2023-11-15T02:57:44.568173+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577142+0000","last_change":"2023-11-15T02:57:44.577142+0000","last_active":"2023-11-15T02:57:44.577142+0000","last_peered":"2023-11-15T02:57:44.577142+0000","last_clean":"2023-11-15T02:57:44.577142+0000","last_became_active":"2023-11-15T02:57:44.576883+0000","last_became_peered":"2023-11-15T02:57:44.576883+0000","last_unstale":"2023-11-15T02:57:44.577142+0000","last_undegraded":"2023-11-15T02:57:44.577142+0000","last_fullsized":"2023-11-15T02:57:44.577142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.113548+0000","last_change":"2023-11-15T02:57:44.111033+0000","last_active":"2023-11-15T02:57:44.113548+0000","last_peered":"2023-11-15T02:57:44.113548+0000","last_clean":"2023-11-15T02:57:44.113548+0000","last_became_active":"2023-11-15T02:57:44.109168+0000","last_became_peered":"2023-11-15T02:57:44.109168+0000","last_unstale":"2023-11-15T02:57:44.113548+0000","last_undegraded":"2023-11-15T02:57:44.113548+0000","last_fullsized":"2023-11-15T02:57:44.113548+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.556390+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T02:57:44.556390+0000","last_peered":"2023-11-15T02:57:44.556390+0000","last_clean":"2023-11-15T02:57:44.556390+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T02:57:44.556390+0000","last_undegraded":"2023-11-15T02:57:44.556390+0000","last_fullsized":"2023-11-15T02:57:44.556390+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577944+0000","last_change":"2023-11-15T02:57:44.577944+0000","last_active":"2023-11-15T02:57:44.577944+0000","last_peered":"2023-11-15T02:57:44.577944+0000","last_clean":"2023-11-15T02:57:44.577944+0000","last_became_active":"2023-11-15T02:57:44.577738+0000","last_became_peered":"2023-11-15T02:57:44.577738+0000","last_unstale":"2023-11-15T02:57:44.577944+0000","last_undegraded":"2023-11-15T02:57:44.577944+0000","last_fullsized":"2023-11-15T02:57:44.577944+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.105276+0000","last_change":"2023-11-15T02:57:44.098264+0000","last_active":"2023-11-15T02:57:44.105276+0000","last_peered":"2023-11-15T02:57:44.105276+0000","last_clean":"2023-11-15T02:57:44.105276+0000","last_became_active":"2023-11-15T02:57:44.095009+0000","last_became_peered":"2023-11-15T02:57:44.095009+0000","last_unstale":"2023-11-15T02:57:44.105276+0000","last_undegraded":"2023-11-15T02:57:44.105276+0000","last_fullsized":"2023-11-15T02:57:44.105276+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.078017+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T02:57:44.078017+0000","last_peered":"2023-11-15T02:57:44.078017+0000","last_clean":"2023-11-15T02:57:44.078017+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T02:57:44.078017+0000","last_undegraded":"2023-11-15T02:57:44.078017+0000","last_fullsized":"2023-11-15T02:57:44.078017+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.587338+0000","last_change":"2023-11-15T02:57:44.581053+0000","last_active":"2023-11-15T02:57:44.587338+0000","last_peered":"2023-11-15T02:57:44.587338+0000","last_clean":"2023-11-15T02:57:44.587338+0000","last_became_active":"2023-11-15T02:57:44.580765+0000","last_became_peered":"2023-11-15T02:57:44.580765+0000","last_unstale":"2023-11-15T02:57:44.587338+0000","last_undegraded":"2023-11-15T02:57:44.587338+0000","last_fullsized":"2023-11-15T02:57:44.587338+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.596704+0000","last_change":"2023-11-15T02:57:44.095125+0000","last_active":"2023-11-15T02:57:44.596704+0000","last_peered":"2023-11-15T02:57:44.596704+0000","last_clean":"2023-11-15T02:57:44.596704+0000","last_became_active":"2023-11-15T02:57:44.094525+0000","last_became_peered":"2023-11-15T02:57:44.094525+0000","last_unstale":"2023-11-15T02:57:44.596704+0000","last_undegraded":"2023-11-15T02:57:44.596704+0000","last_fullsized":"2023-11-15T02:57:44.596704+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.088566+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T02:57:44.088566+0000","last_peered":"2023-11-15T02:57:44.088566+0000","last_clean":"2023-11-15T02:57:44.088566+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T02:57:44.088566+0000","last_undegraded":"2023-11-15T02:57:44.088566+0000","last_fullsized":"2023-11-15T02:57:44.088566+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.593320+0000","last_change":"2023-11-15T02:57:44.584057+0000","last_active":"2023-11-15T02:57:44.593320+0000","last_peered":"2023-11-15T02:57:44.593320+0000","last_clean":"2023-11-15T02:57:44.593320+0000","last_became_active":"2023-11-15T02:57:44.583712+0000","last_became_peered":"2023-11-15T02:57:44.583712+0000","last_unstale":"2023-11-15T02:57:44.593320+0000","last_undegraded":"2023-11-15T02:57:44.593320+0000","last_fullsized":"2023-11-15T02:57:44.593320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.112596+0000","last_change":"2023-11-15T02:57:44.111330+0000","last_active":"2023-11-15T02:57:44.112596+0000","last_peered":"2023-11-15T02:57:44.112596+0000","last_clean":"2023-11-15T02:57:44.112596+0000","last_became_active":"2023-11-15T02:57:44.111040+0000","last_became_peered":"2023-11-15T02:57:44.111040+0000","last_unstale":"2023-11-15T02:57:44.112596+0000","last_undegraded":"2023-11-15T02:57:44.112596+0000","last_fullsized":"2023-11-15T02:57:44.112596+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.556502+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T02:57:44.556502+0000","last_peered":"2023-11-15T02:57:44.556502+0000","last_clean":"2023-11-15T02:57:44.556502+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T02:57:44.556502+0000","last_undegraded":"2023-11-15T02:57:44.556502+0000","last_fullsized":"2023-11-15T02:57:44.556502+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.088937+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T02:57:44.088937+0000","last_peered":"2023-11-15T02:57:44.088937+0000","last_clean":"2023-11-15T02:57:44.088937+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T02:57:44.088937+0000","last_undegraded":"2023-11-15T02:57:44.088937+0000","last_fullsized":"2023-11-15T02:57:44.088937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297432,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074408,"statfs":{"total":96636764160,"available":96332193792,"internally_reserved":0,"allocated":344064,"data_stored":138527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4970000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.369}]}]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":360448,"data_stored":146260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":360448,"data_stored":145728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297432,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074408,"statfs":{"total":96636764160,"available":96332193792,"internally_reserved":0,"allocated":344064,"data_stored":138527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.512}]}]},{"osd":2,"up_from":14,"seq":60129542159,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297428,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074412,"statfs":{"total":96636764160,"available":96332197888,"internally_reserved":0,"allocated":339968,"data_stored":137258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]}]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297432,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074408,"statfs":{"total":96636764160,"available":96332193792,"internally_reserved":0,"allocated":344064,"data_stored":138527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":1.0549999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.2839999999999998,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.81799999999999995}]},{"osd":3,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":3.54,"5min":3.54,"15min":3.54},"last":1.093}]},{"osd":4,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.85599999999999998}]},{"osd":5,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":3.6760000000000002,"5min":3.6760000000000002,"15min":3.6760000000000002},"last":1.149}]},{"osd":6,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":3.71,"5min":3.71,"15min":3.71},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":1.024,"5min":1.024,"15min":1.024},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.165}]},{"osd":7,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":1.032,"5min":1.032,"15min":1.032},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.238},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.2090000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297472,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074368,"statfs":{"total":96636764160,"available":96332152832,"internally_reserved":0,"allocated":385024,"data_stored":153968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":3.46,"5min":3.46,"15min":3.46},"last":0.86399999999999999}]},{"osd":1,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":3.6349999999999998,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":1.105},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.516,"5min":3.516,"15min":3.516},"last":0.99099999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":5.4180000000000001,"5min":5.4180000000000001,"15min":5.4180000000000001},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":3.5859999999999999,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":1.073}]},{"osd":3,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":1.208},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":4.2729999999999997,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":1.286}]},{"osd":4,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":3.706,"5min":3.706,"15min":3.706},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.659,"5min":1.659,"15min":1.659},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":6.1180000000000003,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":1.159}]},{"osd":6,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.607,"5min":1.607,"15min":1.607},"min":{"1min":1.123,"5min":1.123,"15min":1.123},"max":{"1min":3.641,"5min":3.641,"15min":3.641},"last":1.1299999999999999}]},{"osd":7,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"min":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":3.516,"5min":3.516,"15min":3.516},"last":1.054},{"interface":"front","average":{"1min":1.468,"5min":1.468,"15min":1.468},"min":{"1min":1.022,"5min":1.022,"15min":1.022},"max":{"1min":3.754,"5min":3.754,"15min":3.754},"last":1.2270000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297464,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074376,"statfs":{"total":96636764160,"available":96332161024,"internally_reserved":0,"allocated":376832,"data_stored":152570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":1.016}]},{"osd":2,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.96799999999999997}]},{"osd":3,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.173},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.141}]},{"osd":4,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":1.2809999999999999}]},{"osd":5,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.4609999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.2310000000000001}]},{"osd":6,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.601}]},{"osd":7,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.671}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T02:57:53.566 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T02:57:53.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.723+0000 7f2b95e9b700 1 Processor -- start 2023-11-15T02:57:53.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.723+0000 7f2b95e9b700 1 -- start start 2023-11-15T02:57:53.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.727+0000 7f2b95e9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b90154ca0 0x7f2b90155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:53.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.727+0000 7f2b95e9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2b900a57c0 0x7f2b900a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:53.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.727+0000 7f2b95e9b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b900a6160 con 0x7f2b901562f0 2023-11-15T02:57:53.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.727+0000 7f2b95e9b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b900a8280 con 0x7f2b900a57c0 2023-11-15T02:57:53.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.727+0000 7f2b95e9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900a6d40 con 0x7f2b90154ca0 2023-11-15T02:57:53.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.727+0000 7f2b8effd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f2b901562f0 0x7f2b900a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40676/0 (socket says 172.21.15.121:40676) 2023-11-15T02:57:53.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.727+0000 7f2b8effd700 1 -- 172.21.15.121:0/2090380967 learned_addr learned my addr 172.21.15.121:0/2090380967 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:53.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.727+0000 7f2b8ffff700 1 --2- 172.21.15.121:0/2090380967 >> v2:172.21.15.121:3300/0 conn(0x7f2b900a57c0 0x7f2b900a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:53.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.727+0000 7f2b8f7fe700 1 --2- 172.21.15.121:0/2090380967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b90154ca0 0x7f2b90155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:53.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.727+0000 7f2b8e7fc700 1 -- 172.21.15.121:0/2090380967 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951227942 0 0) 0x7f2b84001110 con 0x7f2b901562f0 2023-11-15T02:57:53.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.727+0000 7f2b8e7fc700 1 -- 172.21.15.121:0/2090380967 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b780033f0 con 0x7f2b901562f0 2023-11-15T02:57:53.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.731+0000 7f2b8ffff700 1 -- 172.21.15.121:0/2090380967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b90154ca0 msgr2=0x7f2b90155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:53.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.731+0000 7f2b8ffff700 1 --2- 172.21.15.121:0/2090380967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b90154ca0 0x7f2b90155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:53.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.731+0000 7f2b8ffff700 1 -- 172.21.15.121:0/2090380967 >> v1:172.21.15.121:6789/0 conn(0x7f2b901562f0 legacy=0x7f2b900a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:53.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.731+0000 7f2b8ffff700 1 -- 172.21.15.121:0/2090380967 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b900c4c30 con 0x7f2b900a57c0 2023-11-15T02:57:53.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.731+0000 7f2b8ffff700 1 --2- 172.21.15.121:0/2090380967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b900a57c0 0x7f2b900a5c70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f2b88009760 tx=0x7f2b88009060).ready entity=mon.0 client_cookie=1c57049cb8840b1b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:53.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.731+0000 7f2b8e7fc700 1 -- 172.21.15.121:0/2090380967 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8800d4a0 con 0x7f2b900a57c0 2023-11-15T02:57:53.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.731+0000 7f2b8e7fc700 1 -- 172.21.15.121:0/2090380967 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8800d600 con 0x7f2b900a57c0 2023-11-15T02:57:53.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.731+0000 7f2b8e7fc700 1 -- 172.21.15.121:0/2090380967 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8800d980 con 0x7f2b900a57c0 2023-11-15T02:57:53.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.731+0000 7f2b95e9b700 1 -- 172.21.15.121:0/2090380967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b900a57c0 msgr2=0x7f2b900a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:53.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.731+0000 7f2b95e9b700 1 --2- 172.21.15.121:0/2090380967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b900a57c0 0x7f2b900a5c70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f2b88009760 tx=0x7f2b88009060).stop 2023-11-15T02:57:53.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.731+0000 7f2b95e9b700 1 -- 172.21.15.121:0/2090380967 shutdown_connections 2023-11-15T02:57:53.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.731+0000 7f2b95e9b700 1 --2- 172.21.15.121:0/2090380967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b900a57c0 0x7f2b900a5c70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:53.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.731+0000 7f2b95e9b700 1 --2- 172.21.15.121:0/2090380967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b90154ca0 0x7f2b90155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:53.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.731+0000 7f2b95e9b700 1 -- 172.21.15.121:0/2090380967 >> 172.21.15.121:0/2090380967 conn(0x7f2b90062190 msgr2=0x7f2b900625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:53.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.731+0000 7f2b95e9b700 1 -- 172.21.15.121:0/2090380967 shutdown_connections 2023-11-15T02:57:53.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.731+0000 7f2b95e9b700 1 -- 172.21.15.121:0/2090380967 wait complete. 2023-11-15T02:57:53.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.731+0000 7f2b95e9b700 1 Processor -- start 2023-11-15T02:57:53.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b95e9b700 1 -- start start 2023-11-15T02:57:53.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b95e9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b900a57c0 0x7f2b90136660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:53.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b95e9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b90154ca0 0x7f2b901513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:53.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b95e9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b901562f0 0x7f2b9014ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:53.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b95e9b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b90168d80 con 0x7f2b90154ca0 2023-11-15T02:57:53.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b95e9b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900a9e70 con 0x7f2b900a57c0 2023-11-15T02:57:53.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b95e9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b90136b50 con 0x7f2b901562f0 2023-11-15T02:57:53.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b8effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b90154ca0 0x7f2b901513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:53.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b8effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b90154ca0 0x7f2b901513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56500/0 (socket says 172.21.15.121:56500) 2023-11-15T02:57:53.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b8effd700 1 -- 172.21.15.121:0/1602710623 learned_addr learned my addr 172.21.15.121:0/1602710623 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:53.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b8f7fe700 1 --2- 172.21.15.121:0/1602710623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b900a57c0 0x7f2b90136660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:53.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b8ffff700 1 --2- 172.21.15.121:0/1602710623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b901562f0 0x7f2b9014ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:53.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b8effd700 1 -- 172.21.15.121:0/1602710623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b901562f0 msgr2=0x7f2b9014ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:53.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b8effd700 1 --2- 172.21.15.121:0/1602710623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b901562f0 0x7f2b9014ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:53.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b8effd700 1 -- 172.21.15.121:0/1602710623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b900a57c0 msgr2=0x7f2b90136660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:53.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b8effd700 1 --2- 172.21.15.121:0/1602710623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b900a57c0 0x7f2b90136660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:53.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b8effd700 1 -- 172.21.15.121:0/1602710623 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b880092a0 con 0x7f2b90154ca0 2023-11-15T02:57:53.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b8f7fe700 1 --2- 172.21.15.121:0/1602710623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b900a57c0 0x7f2b90136660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:53.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b8effd700 1 --2- 172.21.15.121:0/1602710623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b90154ca0 0x7f2b901513a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f2b8400c410 tx=0x7f2b84007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:53.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b777fe700 1 -- 172.21.15.121:0/1602710623 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b840072e0 con 0x7f2b90154ca0 2023-11-15T02:57:53.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b777fe700 1 -- 172.21.15.121:0/1602710623 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b84007440 con 0x7f2b90154ca0 2023-11-15T02:57:53.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b95e9b700 1 -- 172.21.15.121:0/1602710623 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b90168100 con 0x7f2b90154ca0 2023-11-15T02:57:53.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b777fe700 1 -- 172.21.15.121:0/1602710623 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b840057a0 con 0x7f2b90154ca0 2023-11-15T02:57:53.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.735+0000 7f2b95e9b700 1 -- 172.21.15.121:0/1602710623 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b901686c0 con 0x7f2b90154ca0 2023-11-15T02:57:53.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.739+0000 7f2b95e9b700 1 -- 172.21.15.121:0/1602710623 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b90168ec0 con 0x7f2b90154ca0 2023-11-15T02:57:53.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.739+0000 7f2b777fe700 1 -- 172.21.15.121:0/1602710623 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2b84005b50 con 0x7f2b90154ca0 2023-11-15T02:57:53.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.739+0000 7f2b777fe700 1 --2- 172.21.15.121:0/1602710623 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2b7c05a510 0x7f2b7c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:53.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.739+0000 7f2b777fe700 1 -- 172.21.15.121:0/1602710623 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2b84073e60 con 0x7f2b90154ca0 2023-11-15T02:57:53.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.739+0000 7f2b8f7fe700 1 --2- 172.21.15.121:0/1602710623 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2b7c05a510 0x7f2b7c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:53.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.739+0000 7f2b8f7fe700 1 --2- 172.21.15.121:0/1602710623 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2b7c05a510 0x7f2b7c05c9d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f2b800076c0 tx=0x7f2b8000b040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:53.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.747+0000 7f2b777fe700 1 -- 172.21.15.121:0/1602710623 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2b84047580 con 0x7f2b90154ca0 2023-11-15T02:57:53.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.903+0000 7f2b95e9b700 1 -- 172.21.15.121:0/1602710623 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2b900a6160 con 0x7f2b7c05a510 2023-11-15T02:57:53.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.915+0000 7f2b777fe700 1 -- 172.21.15.121:0/1602710623 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+358104 (secure 0 0 0) 0x7f2b900a6160 con 0x7f2b7c05a510 2023-11-15T02:57:53.923 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:53.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.923+0000 7f2b95e9b700 1 -- 172.21.15.121:0/1602710623 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2b7c05a510 msgr2=0x7f2b7c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:53.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.923+0000 7f2b95e9b700 1 --2- 172.21.15.121:0/1602710623 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2b7c05a510 0x7f2b7c05c9d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f2b800076c0 tx=0x7f2b8000b040).stop 2023-11-15T02:57:53.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.923+0000 7f2b95e9b700 1 -- 172.21.15.121:0/1602710623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b90154ca0 msgr2=0x7f2b901513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:53.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.923+0000 7f2b95e9b700 1 --2- 172.21.15.121:0/1602710623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b90154ca0 0x7f2b901513a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f2b8400c410 tx=0x7f2b84007ad0).stop 2023-11-15T02:57:53.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.923+0000 7f2b95e9b700 1 -- 172.21.15.121:0/1602710623 shutdown_connections 2023-11-15T02:57:53.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.923+0000 7f2b95e9b700 1 --2- 172.21.15.121:0/1602710623 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2b7c05a510 0x7f2b7c05c9d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:53.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.923+0000 7f2b95e9b700 1 --2- 172.21.15.121:0/1602710623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b901562f0 0x7f2b9014ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:53.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.923+0000 7f2b95e9b700 1 --2- 172.21.15.121:0/1602710623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b90154ca0 0x7f2b901513a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:53.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.923+0000 7f2b95e9b700 1 --2- 172.21.15.121:0/1602710623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b900a57c0 0x7f2b90136660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:53.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.923+0000 7f2b95e9b700 1 -- 172.21.15.121:0/1602710623 >> 172.21.15.121:0/1602710623 conn(0x7f2b90062190 msgr2=0x7f2b90066df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:53.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.923+0000 7f2b95e9b700 1 -- 172.21.15.121:0/1602710623 shutdown_connections 2023-11-15T02:57:53.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:53.923+0000 7f2b95e9b700 1 -- 172.21.15.121:0/1602710623 wait complete. 2023-11-15T02:57:53.933 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T02:57:53.950 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2023-11-15T02:57:51.932842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":270,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2379708,"kb_used_data":2940,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752595012,"statfs":{"total":773094113280,"available":770657292288,"internally_reserved":0,"allocated":3010560,"data_stored":1222623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3857,"num_objects":-13,"num_object_clones":0,"num_object_copies":-26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-35,"num_write_kb":-20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.852015"},"pg_stats":[{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.587058+0000","last_change":"2023-11-15T02:57:44.576426+0000","last_active":"2023-11-15T02:57:44.587058+0000","last_peered":"2023-11-15T02:57:44.587058+0000","last_clean":"2023-11-15T02:57:44.587058+0000","last_became_active":"2023-11-15T02:57:44.572509+0000","last_became_peered":"2023-11-15T02:57:44.572509+0000","last_unstale":"2023-11-15T02:57:44.587058+0000","last_undegraded":"2023-11-15T02:57:44.587058+0000","last_fullsized":"2023-11-15T02:57:44.587058+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3f","version":"32'3","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.587303+0000","last_change":"2023-11-15T02:57:44.105284+0000","last_active":"2023-11-15T02:57:44.587303+0000","last_peered":"2023-11-15T02:57:44.587303+0000","last_clean":"2023-11-15T02:57:44.587303+0000","last_became_active":"2023-11-15T02:57:44.104935+0000","last_became_peered":"2023-11-15T02:57:44.104935+0000","last_unstale":"2023-11-15T02:57:44.587303+0000","last_undegraded":"2023-11-15T02:57:44.587303+0000","last_fullsized":"2023-11-15T02:57:44.587303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.569548+0000","last_change":"2023-11-15T02:57:44.566808+0000","last_active":"2023-11-15T02:57:44.569548+0000","last_peered":"2023-11-15T02:57:44.569548+0000","last_clean":"2023-11-15T02:57:44.569548+0000","last_became_active":"2023-11-15T02:57:44.566228+0000","last_became_peered":"2023-11-15T02:57:44.566228+0000","last_unstale":"2023-11-15T02:57:44.569548+0000","last_undegraded":"2023-11-15T02:57:44.569548+0000","last_fullsized":"2023-11-15T02:57:44.569548+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.552637+0000","last_change":"2023-11-15T02:57:44.101869+0000","last_active":"2023-11-15T02:57:44.552637+0000","last_peered":"2023-11-15T02:57:44.552637+0000","last_clean":"2023-11-15T02:57:44.552637+0000","last_became_active":"2023-11-15T02:57:44.101602+0000","last_became_peered":"2023-11-15T02:57:44.101602+0000","last_unstale":"2023-11-15T02:57:44.552637+0000","last_undegraded":"2023-11-15T02:57:44.552637+0000","last_fullsized":"2023-11-15T02:57:44.552637+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.581954+0000","last_change":"2023-11-15T02:57:44.564204+0000","last_active":"2023-11-15T02:57:44.581954+0000","last_peered":"2023-11-15T02:57:44.581954+0000","last_clean":"2023-11-15T02:57:44.581954+0000","last_became_active":"2023-11-15T02:57:44.563976+0000","last_became_peered":"2023-11-15T02:57:44.563976+0000","last_unstale":"2023-11-15T02:57:44.581954+0000","last_undegraded":"2023-11-15T02:57:44.581954+0000","last_fullsized":"2023-11-15T02:57:44.581954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.556695+0000","last_change":"2023-11-15T02:57:44.096033+0000","last_active":"2023-11-15T02:57:44.556695+0000","last_peered":"2023-11-15T02:57:44.556695+0000","last_clean":"2023-11-15T02:57:44.556695+0000","last_became_active":"2023-11-15T02:57:44.095626+0000","last_became_peered":"2023-11-15T02:57:44.095626+0000","last_unstale":"2023-11-15T02:57:44.556695+0000","last_undegraded":"2023-11-15T02:57:44.556695+0000","last_fullsized":"2023-11-15T02:57:44.556695+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582219+0000","last_change":"2023-11-15T02:57:44.568907+0000","last_active":"2023-11-15T02:57:44.582219+0000","last_peered":"2023-11-15T02:57:44.582219+0000","last_clean":"2023-11-15T02:57:44.582219+0000","last_became_active":"2023-11-15T02:57:44.568621+0000","last_became_peered":"2023-11-15T02:57:44.568621+0000","last_unstale":"2023-11-15T02:57:44.582219+0000","last_undegraded":"2023-11-15T02:57:44.582219+0000","last_fullsized":"2023-11-15T02:57:44.582219+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3c","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.588835+0000","last_change":"2023-11-15T02:57:44.103658+0000","last_active":"2023-11-15T02:57:44.588835+0000","last_peered":"2023-11-15T02:57:44.588835+0000","last_clean":"2023-11-15T02:57:44.588835+0000","last_became_active":"2023-11-15T02:57:44.101877+0000","last_became_peered":"2023-11-15T02:57:44.101877+0000","last_unstale":"2023-11-15T02:57:44.588835+0000","last_undegraded":"2023-11-15T02:57:44.588835+0000","last_fullsized":"2023-11-15T02:57:44.588835+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582346+0000","last_change":"2023-11-15T02:57:44.570582+0000","last_active":"2023-11-15T02:57:44.582346+0000","last_peered":"2023-11-15T02:57:44.582346+0000","last_clean":"2023-11-15T02:57:44.582346+0000","last_became_active":"2023-11-15T02:57:44.570314+0000","last_became_peered":"2023-11-15T02:57:44.570314+0000","last_unstale":"2023-11-15T02:57:44.582346+0000","last_undegraded":"2023-11-15T02:57:44.582346+0000","last_fullsized":"2023-11-15T02:57:44.582346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3b","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585119+0000","last_change":"2023-11-15T02:57:44.104587+0000","last_active":"2023-11-15T02:57:44.585119+0000","last_peered":"2023-11-15T02:57:44.585119+0000","last_clean":"2023-11-15T02:57:44.585119+0000","last_became_active":"2023-11-15T02:57:44.104223+0000","last_became_peered":"2023-11-15T02:57:44.104223+0000","last_unstale":"2023-11-15T02:57:44.585119+0000","last_undegraded":"2023-11-15T02:57:44.585119+0000","last_fullsized":"2023-11-15T02:57:44.585119+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.593390+0000","last_change":"2023-11-15T02:57:44.584427+0000","last_active":"2023-11-15T02:57:44.593390+0000","last_peered":"2023-11-15T02:57:44.593390+0000","last_clean":"2023-11-15T02:57:44.593390+0000","last_became_active":"2023-11-15T02:57:44.584121+0000","last_became_peered":"2023-11-15T02:57:44.584121+0000","last_unstale":"2023-11-15T02:57:44.593390+0000","last_undegraded":"2023-11-15T02:57:44.593390+0000","last_fullsized":"2023-11-15T02:57:44.593390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.580646+0000","last_change":"2023-11-15T02:57:44.102587+0000","last_active":"2023-11-15T02:57:44.580646+0000","last_peered":"2023-11-15T02:57:44.580646+0000","last_clean":"2023-11-15T02:57:44.580646+0000","last_became_active":"2023-11-15T02:57:44.102311+0000","last_became_peered":"2023-11-15T02:57:44.102311+0000","last_unstale":"2023-11-15T02:57:44.580646+0000","last_undegraded":"2023-11-15T02:57:44.580646+0000","last_fullsized":"2023-11-15T02:57:44.580646+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.589914+0000","last_change":"2023-11-15T02:57:44.585731+0000","last_active":"2023-11-15T02:57:44.589914+0000","last_peered":"2023-11-15T02:57:44.589914+0000","last_clean":"2023-11-15T02:57:44.589914+0000","last_became_active":"2023-11-15T02:57:44.585470+0000","last_became_peered":"2023-11-15T02:57:44.585470+0000","last_unstale":"2023-11-15T02:57:44.589914+0000","last_undegraded":"2023-11-15T02:57:44.589914+0000","last_fullsized":"2023-11-15T02:57:44.589914+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560771+0000","last_change":"2023-11-15T02:57:44.105206+0000","last_active":"2023-11-15T02:57:44.560771+0000","last_peered":"2023-11-15T02:57:44.560771+0000","last_clean":"2023-11-15T02:57:44.560771+0000","last_became_active":"2023-11-15T02:57:44.104936+0000","last_became_peered":"2023-11-15T02:57:44.104936+0000","last_unstale":"2023-11-15T02:57:44.560771+0000","last_undegraded":"2023-11-15T02:57:44.560771+0000","last_fullsized":"2023-11-15T02:57:44.560771+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585016+0000","last_change":"2023-11-15T02:57:44.566763+0000","last_active":"2023-11-15T02:57:44.585016+0000","last_peered":"2023-11-15T02:57:44.585016+0000","last_clean":"2023-11-15T02:57:44.585016+0000","last_became_active":"2023-11-15T02:57:44.566510+0000","last_became_peered":"2023-11-15T02:57:44.566510+0000","last_unstale":"2023-11-15T02:57:44.585016+0000","last_undegraded":"2023-11-15T02:57:44.585016+0000","last_fullsized":"2023-11-15T02:57:44.585016+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560184+0000","last_change":"2023-11-15T02:57:44.106233+0000","last_active":"2023-11-15T02:57:44.560184+0000","last_peered":"2023-11-15T02:57:44.560184+0000","last_clean":"2023-11-15T02:57:44.560184+0000","last_became_active":"2023-11-15T02:57:44.105985+0000","last_became_peered":"2023-11-15T02:57:44.105985+0000","last_unstale":"2023-11-15T02:57:44.560184+0000","last_undegraded":"2023-11-15T02:57:44.560184+0000","last_fullsized":"2023-11-15T02:57:44.560184+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.580558+0000","last_change":"2023-11-15T02:57:44.577960+0000","last_active":"2023-11-15T02:57:44.580558+0000","last_peered":"2023-11-15T02:57:44.580558+0000","last_clean":"2023-11-15T02:57:44.580558+0000","last_became_active":"2023-11-15T02:57:44.577405+0000","last_became_peered":"2023-11-15T02:57:44.577405+0000","last_unstale":"2023-11-15T02:57:44.580558+0000","last_undegraded":"2023-11-15T02:57:44.580558+0000","last_fullsized":"2023-11-15T02:57:44.580558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.563378+0000","last_change":"2023-11-15T02:57:44.091037+0000","last_active":"2023-11-15T02:57:44.563378+0000","last_peered":"2023-11-15T02:57:44.563378+0000","last_clean":"2023-11-15T02:57:44.563378+0000","last_became_active":"2023-11-15T02:57:44.090808+0000","last_became_peered":"2023-11-15T02:57:44.090808+0000","last_unstale":"2023-11-15T02:57:44.563378+0000","last_undegraded":"2023-11-15T02:57:44.563378+0000","last_fullsized":"2023-11-15T02:57:44.563378+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.583352+0000","last_change":"2023-11-15T02:57:44.572175+0000","last_active":"2023-11-15T02:57:44.583352+0000","last_peered":"2023-11-15T02:57:44.583352+0000","last_clean":"2023-11-15T02:57:44.583352+0000","last_became_active":"2023-11-15T02:57:44.571339+0000","last_became_peered":"2023-11-15T02:57:44.571339+0000","last_unstale":"2023-11-15T02:57:44.583352+0000","last_undegraded":"2023-11-15T02:57:44.583352+0000","last_fullsized":"2023-11-15T02:57:44.583352+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.570535+0000","last_change":"2023-11-15T02:57:44.104971+0000","last_active":"2023-11-15T02:57:44.570535+0000","last_peered":"2023-11-15T02:57:44.570535+0000","last_clean":"2023-11-15T02:57:44.570535+0000","last_became_active":"2023-11-15T02:57:44.104610+0000","last_became_peered":"2023-11-15T02:57:44.104610+0000","last_unstale":"2023-11-15T02:57:44.570535+0000","last_undegraded":"2023-11-15T02:57:44.570535+0000","last_fullsized":"2023-11-15T02:57:44.570535+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.576019+0000","last_change":"2023-11-15T02:57:44.569668+0000","last_active":"2023-11-15T02:57:44.576019+0000","last_peered":"2023-11-15T02:57:44.576019+0000","last_clean":"2023-11-15T02:57:44.576019+0000","last_became_active":"2023-11-15T02:57:44.569379+0000","last_became_peered":"2023-11-15T02:57:44.569379+0000","last_unstale":"2023-11-15T02:57:44.576019+0000","last_undegraded":"2023-11-15T02:57:44.576019+0000","last_fullsized":"2023-11-15T02:57:44.576019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.564356+0000","last_change":"2023-11-15T02:57:44.102619+0000","last_active":"2023-11-15T02:57:44.564356+0000","last_peered":"2023-11-15T02:57:44.564356+0000","last_clean":"2023-11-15T02:57:44.564356+0000","last_became_active":"2023-11-15T02:57:44.102328+0000","last_became_peered":"2023-11-15T02:57:44.102328+0000","last_unstale":"2023-11-15T02:57:44.564356+0000","last_undegraded":"2023-11-15T02:57:44.564356+0000","last_fullsized":"2023-11-15T02:57:44.564356+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.581231+0000","last_change":"2023-11-15T02:57:44.579068+0000","last_active":"2023-11-15T02:57:44.581231+0000","last_peered":"2023-11-15T02:57:44.581231+0000","last_clean":"2023-11-15T02:57:44.581231+0000","last_became_active":"2023-11-15T02:57:44.578815+0000","last_became_peered":"2023-11-15T02:57:44.578815+0000","last_unstale":"2023-11-15T02:57:44.581231+0000","last_undegraded":"2023-11-15T02:57:44.581231+0000","last_fullsized":"2023-11-15T02:57:44.581231+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"32'4","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.598725+0000","last_change":"2023-11-15T02:57:44.113636+0000","last_active":"2023-11-15T02:57:44.598725+0000","last_peered":"2023-11-15T02:57:44.598725+0000","last_clean":"2023-11-15T02:57:44.598725+0000","last_became_active":"2023-11-15T02:57:44.113283+0000","last_became_peered":"2023-11-15T02:57:44.113283+0000","last_unstale":"2023-11-15T02:57:44.598725+0000","last_undegraded":"2023-11-15T02:57:44.598725+0000","last_fullsized":"2023-11-15T02:57:44.598725+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.587561+0000","last_change":"2023-11-15T02:57:44.584681+0000","last_active":"2023-11-15T02:57:44.587561+0000","last_peered":"2023-11-15T02:57:44.587561+0000","last_clean":"2023-11-15T02:57:44.587561+0000","last_became_active":"2023-11-15T02:57:44.584395+0000","last_became_peered":"2023-11-15T02:57:44.584395+0000","last_unstale":"2023-11-15T02:57:44.587561+0000","last_undegraded":"2023-11-15T02:57:44.587561+0000","last_fullsized":"2023-11-15T02:57:44.587561+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.33","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585088+0000","last_change":"2023-11-15T02:57:44.086072+0000","last_active":"2023-11-15T02:57:44.585088+0000","last_peered":"2023-11-15T02:57:44.585088+0000","last_clean":"2023-11-15T02:57:44.585088+0000","last_became_active":"2023-11-15T02:57:44.085574+0000","last_became_peered":"2023-11-15T02:57:44.085574+0000","last_unstale":"2023-11-15T02:57:44.585088+0000","last_undegraded":"2023-11-15T02:57:44.585088+0000","last_fullsized":"2023-11-15T02:57:44.585088+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.590256+0000","last_change":"2023-11-15T02:57:44.580438+0000","last_active":"2023-11-15T02:57:44.590256+0000","last_peered":"2023-11-15T02:57:44.590256+0000","last_clean":"2023-11-15T02:57:44.590256+0000","last_became_active":"2023-11-15T02:57:44.579977+0000","last_became_peered":"2023-11-15T02:57:44.579977+0000","last_unstale":"2023-11-15T02:57:44.590256+0000","last_undegraded":"2023-11-15T02:57:44.590256+0000","last_fullsized":"2023-11-15T02:57:44.590256+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.553609+0000","last_change":"2023-11-15T02:57:44.094640+0000","last_active":"2023-11-15T02:57:44.553609+0000","last_peered":"2023-11-15T02:57:44.553609+0000","last_clean":"2023-11-15T02:57:44.553609+0000","last_became_active":"2023-11-15T02:57:44.094279+0000","last_became_peered":"2023-11-15T02:57:44.094279+0000","last_unstale":"2023-11-15T02:57:44.553609+0000","last_undegraded":"2023-11-15T02:57:44.553609+0000","last_fullsized":"2023-11-15T02:57:44.553609+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585823+0000","last_change":"2023-11-15T02:57:44.576818+0000","last_active":"2023-11-15T02:57:44.585823+0000","last_peered":"2023-11-15T02:57:44.585823+0000","last_clean":"2023-11-15T02:57:44.585823+0000","last_became_active":"2023-11-15T02:57:44.576074+0000","last_became_peered":"2023-11-15T02:57:44.576074+0000","last_unstale":"2023-11-15T02:57:44.585823+0000","last_undegraded":"2023-11-15T02:57:44.585823+0000","last_fullsized":"2023-11-15T02:57:44.585823+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.556584+0000","last_change":"2023-11-15T02:57:44.109239+0000","last_active":"2023-11-15T02:57:44.556584+0000","last_peered":"2023-11-15T02:57:44.556584+0000","last_clean":"2023-11-15T02:57:44.556584+0000","last_became_active":"2023-11-15T02:57:44.103686+0000","last_became_peered":"2023-11-15T02:57:44.103686+0000","last_unstale":"2023-11-15T02:57:44.556584+0000","last_undegraded":"2023-11-15T02:57:44.556584+0000","last_fullsized":"2023-11-15T02:57:44.556584+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.579784+0000","last_change":"2023-11-15T02:57:44.572530+0000","last_active":"2023-11-15T02:57:44.579784+0000","last_peered":"2023-11-15T02:57:44.579784+0000","last_clean":"2023-11-15T02:57:44.579784+0000","last_became_active":"2023-11-15T02:57:44.572016+0000","last_became_peered":"2023-11-15T02:57:44.572016+0000","last_unstale":"2023-11-15T02:57:44.579784+0000","last_undegraded":"2023-11-15T02:57:44.579784+0000","last_fullsized":"2023-11-15T02:57:44.579784+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"32'4","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585881+0000","last_change":"2023-11-15T02:57:44.113817+0000","last_active":"2023-11-15T02:57:44.585881+0000","last_peered":"2023-11-15T02:57:44.585881+0000","last_clean":"2023-11-15T02:57:44.585881+0000","last_became_active":"2023-11-15T02:57:44.112534+0000","last_became_peered":"2023-11-15T02:57:44.112534+0000","last_unstale":"2023-11-15T02:57:44.585881+0000","last_undegraded":"2023-11-15T02:57:44.585881+0000","last_fullsized":"2023-11-15T02:57:44.585881+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.581190+0000","last_change":"2023-11-15T02:57:44.575459+0000","last_active":"2023-11-15T02:57:44.581190+0000","last_peered":"2023-11-15T02:57:44.581190+0000","last_clean":"2023-11-15T02:57:44.581190+0000","last_became_active":"2023-11-15T02:57:44.575123+0000","last_became_peered":"2023-11-15T02:57:44.575123+0000","last_unstale":"2023-11-15T02:57:44.581190+0000","last_undegraded":"2023-11-15T02:57:44.581190+0000","last_fullsized":"2023-11-15T02:57:44.581190+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.565132+0000","last_change":"2023-11-15T02:57:44.107752+0000","last_active":"2023-11-15T02:57:44.565132+0000","last_peered":"2023-11-15T02:57:44.565132+0000","last_clean":"2023-11-15T02:57:44.565132+0000","last_became_active":"2023-11-15T02:57:44.107413+0000","last_became_peered":"2023-11-15T02:57:44.107413+0000","last_unstale":"2023-11-15T02:57:44.565132+0000","last_undegraded":"2023-11-15T02:57:44.565132+0000","last_fullsized":"2023-11-15T02:57:44.565132+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.584952+0000","last_change":"2023-11-15T02:57:44.572196+0000","last_active":"2023-11-15T02:57:44.584952+0000","last_peered":"2023-11-15T02:57:44.584952+0000","last_clean":"2023-11-15T02:57:44.584952+0000","last_became_active":"2023-11-15T02:57:44.571963+0000","last_became_peered":"2023-11-15T02:57:44.571963+0000","last_unstale":"2023-11-15T02:57:44.584952+0000","last_undegraded":"2023-11-15T02:57:44.584952+0000","last_fullsized":"2023-11-15T02:57:44.584952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.553177+0000","last_change":"2023-11-15T02:57:44.097768+0000","last_active":"2023-11-15T02:57:44.553177+0000","last_peered":"2023-11-15T02:57:44.553177+0000","last_clean":"2023-11-15T02:57:44.553177+0000","last_became_active":"2023-11-15T02:57:44.097449+0000","last_became_peered":"2023-11-15T02:57:44.097449+0000","last_unstale":"2023-11-15T02:57:44.553177+0000","last_undegraded":"2023-11-15T02:57:44.553177+0000","last_fullsized":"2023-11-15T02:57:44.553177+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.590853+0000","last_change":"2023-11-15T02:57:44.588091+0000","last_active":"2023-11-15T02:57:44.590853+0000","last_peered":"2023-11-15T02:57:44.590853+0000","last_clean":"2023-11-15T02:57:44.590853+0000","last_became_active":"2023-11-15T02:57:44.587721+0000","last_became_peered":"2023-11-15T02:57:44.587721+0000","last_unstale":"2023-11-15T02:57:44.590853+0000","last_undegraded":"2023-11-15T02:57:44.590853+0000","last_fullsized":"2023-11-15T02:57:44.590853+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2d","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.578229+0000","last_change":"2023-11-15T02:57:44.097948+0000","last_active":"2023-11-15T02:57:44.578229+0000","last_peered":"2023-11-15T02:57:44.578229+0000","last_clean":"2023-11-15T02:57:44.578229+0000","last_became_active":"2023-11-15T02:57:44.097642+0000","last_became_peered":"2023-11-15T02:57:44.097642+0000","last_unstale":"2023-11-15T02:57:44.578229+0000","last_undegraded":"2023-11-15T02:57:44.578229+0000","last_fullsized":"2023-11-15T02:57:44.578229+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585366+0000","last_change":"2023-11-15T02:57:44.582841+0000","last_active":"2023-11-15T02:57:44.585366+0000","last_peered":"2023-11-15T02:57:44.585366+0000","last_clean":"2023-11-15T02:57:44.585366+0000","last_became_active":"2023-11-15T02:57:44.582592+0000","last_became_peered":"2023-11-15T02:57:44.582592+0000","last_unstale":"2023-11-15T02:57:44.585366+0000","last_undegraded":"2023-11-15T02:57:44.585366+0000","last_fullsized":"2023-11-15T02:57:44.585366+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.562142+0000","last_change":"2023-11-15T02:57:44.095430+0000","last_active":"2023-11-15T02:57:44.562142+0000","last_peered":"2023-11-15T02:57:44.562142+0000","last_clean":"2023-11-15T02:57:44.562142+0000","last_became_active":"2023-11-15T02:57:44.094060+0000","last_became_peered":"2023-11-15T02:57:44.094060+0000","last_unstale":"2023-11-15T02:57:44.562142+0000","last_undegraded":"2023-11-15T02:57:44.562142+0000","last_fullsized":"2023-11-15T02:57:44.562142+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585041+0000","last_change":"2023-11-15T02:57:44.571866+0000","last_active":"2023-11-15T02:57:44.585041+0000","last_peered":"2023-11-15T02:57:44.585041+0000","last_clean":"2023-11-15T02:57:44.585041+0000","last_became_active":"2023-11-15T02:57:44.571594+0000","last_became_peered":"2023-11-15T02:57:44.571594+0000","last_unstale":"2023-11-15T02:57:44.585041+0000","last_undegraded":"2023-11-15T02:57:44.585041+0000","last_fullsized":"2023-11-15T02:57:44.585041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.580896+0000","last_change":"2023-11-15T02:57:44.111471+0000","last_active":"2023-11-15T02:57:44.580896+0000","last_peered":"2023-11-15T02:57:44.580896+0000","last_clean":"2023-11-15T02:57:44.580896+0000","last_became_active":"2023-11-15T02:57:44.110158+0000","last_became_peered":"2023-11-15T02:57:44.110158+0000","last_unstale":"2023-11-15T02:57:44.580896+0000","last_undegraded":"2023-11-15T02:57:44.580896+0000","last_fullsized":"2023-11-15T02:57:44.580896+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.591730+0000","last_change":"2023-11-15T02:57:44.589275+0000","last_active":"2023-11-15T02:57:44.591730+0000","last_peered":"2023-11-15T02:57:44.591730+0000","last_clean":"2023-11-15T02:57:44.591730+0000","last_became_active":"2023-11-15T02:57:44.588964+0000","last_became_peered":"2023-11-15T02:57:44.588964+0000","last_unstale":"2023-11-15T02:57:44.591730+0000","last_undegraded":"2023-11-15T02:57:44.591730+0000","last_fullsized":"2023-11-15T02:57:44.591730+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.561577+0000","last_change":"2023-11-15T02:57:44.098842+0000","last_active":"2023-11-15T02:57:44.561577+0000","last_peered":"2023-11-15T02:57:44.561577+0000","last_clean":"2023-11-15T02:57:44.561577+0000","last_became_active":"2023-11-15T02:57:44.098539+0000","last_became_peered":"2023-11-15T02:57:44.098539+0000","last_unstale":"2023-11-15T02:57:44.561577+0000","last_undegraded":"2023-11-15T02:57:44.561577+0000","last_fullsized":"2023-11-15T02:57:44.561577+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.581101+0000","last_change":"2023-11-15T02:57:44.571505+0000","last_active":"2023-11-15T02:57:44.581101+0000","last_peered":"2023-11-15T02:57:44.581101+0000","last_clean":"2023-11-15T02:57:44.581101+0000","last_became_active":"2023-11-15T02:57:44.570032+0000","last_became_peered":"2023-11-15T02:57:44.570032+0000","last_unstale":"2023-11-15T02:57:44.581101+0000","last_undegraded":"2023-11-15T02:57:44.581101+0000","last_fullsized":"2023-11-15T02:57:44.581101+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.559105+0000","last_change":"2023-11-15T02:57:44.102777+0000","last_active":"2023-11-15T02:57:44.559105+0000","last_peered":"2023-11-15T02:57:44.559105+0000","last_clean":"2023-11-15T02:57:44.559105+0000","last_became_active":"2023-11-15T02:57:44.102485+0000","last_became_peered":"2023-11-15T02:57:44.102485+0000","last_unstale":"2023-11-15T02:57:44.559105+0000","last_undegraded":"2023-11-15T02:57:44.559105+0000","last_fullsized":"2023-11-15T02:57:44.559105+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582377+0000","last_change":"2023-11-15T02:57:44.575964+0000","last_active":"2023-11-15T02:57:44.582377+0000","last_peered":"2023-11-15T02:57:44.582377+0000","last_clean":"2023-11-15T02:57:44.582377+0000","last_became_active":"2023-11-15T02:57:44.572566+0000","last_became_peered":"2023-11-15T02:57:44.572566+0000","last_unstale":"2023-11-15T02:57:44.582377+0000","last_undegraded":"2023-11-15T02:57:44.582377+0000","last_fullsized":"2023-11-15T02:57:44.582377+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.555946+0000","last_change":"2023-11-15T02:57:44.100059+0000","last_active":"2023-11-15T02:57:44.555946+0000","last_peered":"2023-11-15T02:57:44.555946+0000","last_clean":"2023-11-15T02:57:44.555946+0000","last_became_active":"2023-11-15T02:57:44.099755+0000","last_became_peered":"2023-11-15T02:57:44.099755+0000","last_unstale":"2023-11-15T02:57:44.555946+0000","last_undegraded":"2023-11-15T02:57:44.555946+0000","last_fullsized":"2023-11-15T02:57:44.555946+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.593351+0000","last_change":"2023-11-15T02:57:44.584133+0000","last_active":"2023-11-15T02:57:44.593351+0000","last_peered":"2023-11-15T02:57:44.593351+0000","last_clean":"2023-11-15T02:57:44.593351+0000","last_became_active":"2023-11-15T02:57:44.583602+0000","last_became_peered":"2023-11-15T02:57:44.583602+0000","last_unstale":"2023-11-15T02:57:44.593351+0000","last_undegraded":"2023-11-15T02:57:44.593351+0000","last_fullsized":"2023-11-15T02:57:44.593351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.570688+0000","last_change":"2023-11-15T02:57:44.098313+0000","last_active":"2023-11-15T02:57:44.570688+0000","last_peered":"2023-11-15T02:57:44.570688+0000","last_clean":"2023-11-15T02:57:44.570688+0000","last_became_active":"2023-11-15T02:57:44.096267+0000","last_became_peered":"2023-11-15T02:57:44.096267+0000","last_unstale":"2023-11-15T02:57:44.570688+0000","last_undegraded":"2023-11-15T02:57:44.570688+0000","last_fullsized":"2023-11-15T02:57:44.570688+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.581305+0000","last_change":"2023-11-15T02:57:44.577168+0000","last_active":"2023-11-15T02:57:44.581305+0000","last_peered":"2023-11-15T02:57:44.581305+0000","last_clean":"2023-11-15T02:57:44.581305+0000","last_became_active":"2023-11-15T02:57:44.576902+0000","last_became_peered":"2023-11-15T02:57:44.576902+0000","last_unstale":"2023-11-15T02:57:44.581305+0000","last_undegraded":"2023-11-15T02:57:44.581305+0000","last_fullsized":"2023-11-15T02:57:44.581305+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.26","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-15T02:57:46.235464+0000","last_change":"2023-11-15T02:57:44.097229+0000","last_active":"2023-11-15T02:57:46.235464+0000","last_peered":"2023-11-15T02:57:46.235464+0000","last_clean":"2023-11-15T02:57:46.235464+0000","last_became_active":"2023-11-15T02:57:44.096765+0000","last_became_peered":"2023-11-15T02:57:44.096765+0000","last_unstale":"2023-11-15T02:57:46.235464+0000","last_undegraded":"2023-11-15T02:57:46.235464+0000","last_fullsized":"2023-11-15T02:57:46.235464+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.587387+0000","last_change":"2023-11-15T02:57:44.580512+0000","last_active":"2023-11-15T02:57:44.587387+0000","last_peered":"2023-11-15T02:57:44.587387+0000","last_clean":"2023-11-15T02:57:44.587387+0000","last_became_active":"2023-11-15T02:57:44.580249+0000","last_became_peered":"2023-11-15T02:57:44.580249+0000","last_unstale":"2023-11-15T02:57:44.587387+0000","last_undegraded":"2023-11-15T02:57:44.587387+0000","last_fullsized":"2023-11-15T02:57:44.587387+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.569612+0000","last_change":"2023-11-15T02:57:44.103956+0000","last_active":"2023-11-15T02:57:44.569612+0000","last_peered":"2023-11-15T02:57:44.569612+0000","last_clean":"2023-11-15T02:57:44.569612+0000","last_became_active":"2023-11-15T02:57:44.103187+0000","last_became_peered":"2023-11-15T02:57:44.103187+0000","last_unstale":"2023-11-15T02:57:44.569612+0000","last_undegraded":"2023-11-15T02:57:44.569612+0000","last_fullsized":"2023-11-15T02:57:44.569612+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.586335+0000","last_change":"2023-11-15T02:57:44.576398+0000","last_active":"2023-11-15T02:57:44.586335+0000","last_peered":"2023-11-15T02:57:44.586335+0000","last_clean":"2023-11-15T02:57:44.586335+0000","last_became_active":"2023-11-15T02:57:44.576039+0000","last_became_peered":"2023-11-15T02:57:44.576039+0000","last_unstale":"2023-11-15T02:57:44.586335+0000","last_undegraded":"2023-11-15T02:57:44.586335+0000","last_fullsized":"2023-11-15T02:57:44.586335+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.556438+0000","last_change":"2023-11-15T02:57:44.098514+0000","last_active":"2023-11-15T02:57:44.556438+0000","last_peered":"2023-11-15T02:57:44.556438+0000","last_clean":"2023-11-15T02:57:44.556438+0000","last_became_active":"2023-11-15T02:57:44.098120+0000","last_became_peered":"2023-11-15T02:57:44.098120+0000","last_unstale":"2023-11-15T02:57:44.556438+0000","last_undegraded":"2023-11-15T02:57:44.556438+0000","last_fullsized":"2023-11-15T02:57:44.556438+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.581009+0000","last_change":"2023-11-15T02:57:44.578170+0000","last_active":"2023-11-15T02:57:44.581009+0000","last_peered":"2023-11-15T02:57:44.581009+0000","last_clean":"2023-11-15T02:57:44.581009+0000","last_became_active":"2023-11-15T02:57:44.577800+0000","last_became_peered":"2023-11-15T02:57:44.577800+0000","last_unstale":"2023-11-15T02:57:44.581009+0000","last_undegraded":"2023-11-15T02:57:44.581009+0000","last_fullsized":"2023-11-15T02:57:44.581009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.561460+0000","last_change":"2023-11-15T02:57:44.101472+0000","last_active":"2023-11-15T02:57:44.561460+0000","last_peered":"2023-11-15T02:57:44.561460+0000","last_clean":"2023-11-15T02:57:44.561460+0000","last_became_active":"2023-11-15T02:57:44.101190+0000","last_became_peered":"2023-11-15T02:57:44.101190+0000","last_unstale":"2023-11-15T02:57:44.561460+0000","last_undegraded":"2023-11-15T02:57:44.561460+0000","last_fullsized":"2023-11-15T02:57:44.561460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.591709+0000","last_change":"2023-11-15T02:57:44.589081+0000","last_active":"2023-11-15T02:57:44.591709+0000","last_peered":"2023-11-15T02:57:44.591709+0000","last_clean":"2023-11-15T02:57:44.591709+0000","last_became_active":"2023-11-15T02:57:44.588602+0000","last_became_peered":"2023-11-15T02:57:44.588602+0000","last_unstale":"2023-11-15T02:57:44.591709+0000","last_undegraded":"2023-11-15T02:57:44.591709+0000","last_fullsized":"2023-11-15T02:57:44.591709+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.564141+0000","last_change":"2023-11-15T02:57:44.101411+0000","last_active":"2023-11-15T02:57:44.564141+0000","last_peered":"2023-11-15T02:57:44.564141+0000","last_clean":"2023-11-15T02:57:44.564141+0000","last_became_active":"2023-11-15T02:57:44.099981+0000","last_became_peered":"2023-11-15T02:57:44.099981+0000","last_unstale":"2023-11-15T02:57:44.564141+0000","last_undegraded":"2023-11-15T02:57:44.564141+0000","last_fullsized":"2023-11-15T02:57:44.564141+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.590872+0000","last_change":"2023-11-15T02:57:44.588317+0000","last_active":"2023-11-15T02:57:44.590872+0000","last_peered":"2023-11-15T02:57:44.590872+0000","last_clean":"2023-11-15T02:57:44.590872+0000","last_became_active":"2023-11-15T02:57:44.586643+0000","last_became_peered":"2023-11-15T02:57:44.586643+0000","last_unstale":"2023-11-15T02:57:44.590872+0000","last_undegraded":"2023-11-15T02:57:44.590872+0000","last_fullsized":"2023-11-15T02:57:44.590872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.573720+0000","last_change":"2023-11-15T02:57:44.097212+0000","last_active":"2023-11-15T02:57:44.573720+0000","last_peered":"2023-11-15T02:57:44.573720+0000","last_clean":"2023-11-15T02:57:44.573720+0000","last_became_active":"2023-11-15T02:57:44.096903+0000","last_became_peered":"2023-11-15T02:57:44.096903+0000","last_unstale":"2023-11-15T02:57:44.573720+0000","last_undegraded":"2023-11-15T02:57:44.573720+0000","last_fullsized":"2023-11-15T02:57:44.573720+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.581336+0000","last_change":"2023-11-15T02:57:44.578257+0000","last_active":"2023-11-15T02:57:44.581336+0000","last_peered":"2023-11-15T02:57:44.581336+0000","last_clean":"2023-11-15T02:57:44.581336+0000","last_became_active":"2023-11-15T02:57:44.571560+0000","last_became_peered":"2023-11-15T02:57:44.571560+0000","last_unstale":"2023-11-15T02:57:44.581336+0000","last_undegraded":"2023-11-15T02:57:44.581336+0000","last_fullsized":"2023-11-15T02:57:44.581336+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560288+0000","last_change":"2023-11-15T02:57:44.102174+0000","last_active":"2023-11-15T02:57:44.560288+0000","last_peered":"2023-11-15T02:57:44.560288+0000","last_clean":"2023-11-15T02:57:44.560288+0000","last_became_active":"2023-11-15T02:57:44.101794+0000","last_became_peered":"2023-11-15T02:57:44.101794+0000","last_unstale":"2023-11-15T02:57:44.560288+0000","last_undegraded":"2023-11-15T02:57:44.560288+0000","last_fullsized":"2023-11-15T02:57:44.560288+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.588989+0000","last_change":"2023-11-15T02:57:44.587132+0000","last_active":"2023-11-15T02:57:44.588989+0000","last_peered":"2023-11-15T02:57:44.588989+0000","last_clean":"2023-11-15T02:57:44.588989+0000","last_became_active":"2023-11-15T02:57:44.586820+0000","last_became_peered":"2023-11-15T02:57:44.586820+0000","last_unstale":"2023-11-15T02:57:44.588989+0000","last_undegraded":"2023-11-15T02:57:44.588989+0000","last_fullsized":"2023-11-15T02:57:44.588989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.590960+0000","last_change":"2023-11-15T02:57:44.102584+0000","last_active":"2023-11-15T02:57:44.590960+0000","last_peered":"2023-11-15T02:57:44.590960+0000","last_clean":"2023-11-15T02:57:44.590960+0000","last_became_active":"2023-11-15T02:57:44.102267+0000","last_became_peered":"2023-11-15T02:57:44.102267+0000","last_unstale":"2023-11-15T02:57:44.590960+0000","last_undegraded":"2023-11-15T02:57:44.590960+0000","last_fullsized":"2023-11-15T02:57:44.590960+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.595125+0000","last_change":"2023-11-15T02:57:44.591963+0000","last_active":"2023-11-15T02:57:44.595125+0000","last_peered":"2023-11-15T02:57:44.595125+0000","last_clean":"2023-11-15T02:57:44.595125+0000","last_became_active":"2023-11-15T02:57:44.591725+0000","last_became_peered":"2023-11-15T02:57:44.591725+0000","last_unstale":"2023-11-15T02:57:44.595125+0000","last_undegraded":"2023-11-15T02:57:44.595125+0000","last_fullsized":"2023-11-15T02:57:44.595125+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.571853+0000","last_change":"2023-11-15T02:57:44.102680+0000","last_active":"2023-11-15T02:57:44.571853+0000","last_peered":"2023-11-15T02:57:44.571853+0000","last_clean":"2023-11-15T02:57:44.571853+0000","last_became_active":"2023-11-15T02:57:44.102394+0000","last_became_peered":"2023-11-15T02:57:44.102394+0000","last_unstale":"2023-11-15T02:57:44.571853+0000","last_undegraded":"2023-11-15T02:57:44.571853+0000","last_fullsized":"2023-11-15T02:57:44.571853+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.589207+0000","last_change":"2023-11-15T02:57:44.586592+0000","last_active":"2023-11-15T02:57:44.589207+0000","last_peered":"2023-11-15T02:57:44.589207+0000","last_clean":"2023-11-15T02:57:44.589207+0000","last_became_active":"2023-11-15T02:57:44.586211+0000","last_became_peered":"2023-11-15T02:57:44.586211+0000","last_unstale":"2023-11-15T02:57:44.589207+0000","last_undegraded":"2023-11-15T02:57:44.589207+0000","last_fullsized":"2023-11-15T02:57:44.589207+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.589223+0000","last_change":"2023-11-15T02:57:44.092034+0000","last_active":"2023-11-15T02:57:44.589223+0000","last_peered":"2023-11-15T02:57:44.589223+0000","last_clean":"2023-11-15T02:57:44.589223+0000","last_became_active":"2023-11-15T02:57:44.090952+0000","last_became_peered":"2023-11-15T02:57:44.090952+0000","last_unstale":"2023-11-15T02:57:44.589223+0000","last_undegraded":"2023-11-15T02:57:44.589223+0000","last_fullsized":"2023-11-15T02:57:44.589223+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.584009+0000","last_change":"2023-11-15T02:57:44.579124+0000","last_active":"2023-11-15T02:57:44.584009+0000","last_peered":"2023-11-15T02:57:44.584009+0000","last_clean":"2023-11-15T02:57:44.584009+0000","last_became_active":"2023-11-15T02:57:44.578824+0000","last_became_peered":"2023-11-15T02:57:44.578824+0000","last_unstale":"2023-11-15T02:57:44.584009+0000","last_undegraded":"2023-11-15T02:57:44.584009+0000","last_fullsized":"2023-11-15T02:57:44.584009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.561782+0000","last_change":"2023-11-15T02:57:44.101425+0000","last_active":"2023-11-15T02:57:44.561782+0000","last_peered":"2023-11-15T02:57:44.561782+0000","last_clean":"2023-11-15T02:57:44.561782+0000","last_became_active":"2023-11-15T02:57:44.101139+0000","last_became_peered":"2023-11-15T02:57:44.101139+0000","last_unstale":"2023-11-15T02:57:44.561782+0000","last_undegraded":"2023-11-15T02:57:44.561782+0000","last_fullsized":"2023-11-15T02:57:44.561782+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.580844+0000","last_change":"2023-11-15T02:57:44.574408+0000","last_active":"2023-11-15T02:57:44.580844+0000","last_peered":"2023-11-15T02:57:44.580844+0000","last_clean":"2023-11-15T02:57:44.580844+0000","last_became_active":"2023-11-15T02:57:44.573651+0000","last_became_peered":"2023-11-15T02:57:44.573651+0000","last_unstale":"2023-11-15T02:57:44.580844+0000","last_undegraded":"2023-11-15T02:57:44.580844+0000","last_fullsized":"2023-11-15T02:57:44.580844+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.556340+0000","last_change":"2023-11-15T02:57:44.112809+0000","last_active":"2023-11-15T02:57:44.556340+0000","last_peered":"2023-11-15T02:57:44.556340+0000","last_clean":"2023-11-15T02:57:44.556340+0000","last_became_active":"2023-11-15T02:57:44.111134+0000","last_became_peered":"2023-11-15T02:57:44.111134+0000","last_unstale":"2023-11-15T02:57:44.556340+0000","last_undegraded":"2023-11-15T02:57:44.556340+0000","last_fullsized":"2023-11-15T02:57:44.556340+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.580330+0000","last_change":"2023-11-15T02:57:44.572751+0000","last_active":"2023-11-15T02:57:44.580330+0000","last_peered":"2023-11-15T02:57:44.580330+0000","last_clean":"2023-11-15T02:57:44.580330+0000","last_became_active":"2023-11-15T02:57:44.572322+0000","last_became_peered":"2023-11-15T02:57:44.572322+0000","last_unstale":"2023-11-15T02:57:44.580330+0000","last_undegraded":"2023-11-15T02:57:44.580330+0000","last_fullsized":"2023-11-15T02:57:44.580330+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.564267+0000","last_change":"2023-11-15T02:57:44.099996+0000","last_active":"2023-11-15T02:57:44.564267+0000","last_peered":"2023-11-15T02:57:44.564267+0000","last_clean":"2023-11-15T02:57:44.564267+0000","last_became_active":"2023-11-15T02:57:44.099695+0000","last_became_peered":"2023-11-15T02:57:44.099695+0000","last_unstale":"2023-11-15T02:57:44.564267+0000","last_undegraded":"2023-11-15T02:57:44.564267+0000","last_fullsized":"2023-11-15T02:57:44.564267+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585839+0000","last_change":"2023-11-15T02:57:44.577040+0000","last_active":"2023-11-15T02:57:44.585839+0000","last_peered":"2023-11-15T02:57:44.585839+0000","last_clean":"2023-11-15T02:57:44.585839+0000","last_became_active":"2023-11-15T02:57:44.576592+0000","last_became_peered":"2023-11-15T02:57:44.576592+0000","last_unstale":"2023-11-15T02:57:44.585839+0000","last_undegraded":"2023-11-15T02:57:44.585839+0000","last_fullsized":"2023-11-15T02:57:44.585839+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.566837+0000","last_change":"2023-11-15T02:57:44.086947+0000","last_active":"2023-11-15T02:57:44.566837+0000","last_peered":"2023-11-15T02:57:44.566837+0000","last_clean":"2023-11-15T02:57:44.566837+0000","last_became_active":"2023-11-15T02:57:44.086415+0000","last_became_peered":"2023-11-15T02:57:44.086415+0000","last_unstale":"2023-11-15T02:57:44.566837+0000","last_undegraded":"2023-11-15T02:57:44.566837+0000","last_fullsized":"2023-11-15T02:57:44.566837+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582445+0000","last_change":"2023-11-15T02:57:44.571061+0000","last_active":"2023-11-15T02:57:44.582445+0000","last_peered":"2023-11-15T02:57:44.582445+0000","last_clean":"2023-11-15T02:57:44.582445+0000","last_became_active":"2023-11-15T02:57:44.570802+0000","last_became_peered":"2023-11-15T02:57:44.570802+0000","last_unstale":"2023-11-15T02:57:44.582445+0000","last_undegraded":"2023-11-15T02:57:44.582445+0000","last_fullsized":"2023-11-15T02:57:44.582445+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560795+0000","last_change":"2023-11-15T02:57:44.099371+0000","last_active":"2023-11-15T02:57:44.560795+0000","last_peered":"2023-11-15T02:57:44.560795+0000","last_clean":"2023-11-15T02:57:44.560795+0000","last_became_active":"2023-11-15T02:57:44.096689+0000","last_became_peered":"2023-11-15T02:57:44.096689+0000","last_unstale":"2023-11-15T02:57:44.560795+0000","last_undegraded":"2023-11-15T02:57:44.560795+0000","last_fullsized":"2023-11-15T02:57:44.560795+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.587040+0000","last_change":"2023-11-15T02:57:44.572799+0000","last_active":"2023-11-15T02:57:44.587040+0000","last_peered":"2023-11-15T02:57:44.587040+0000","last_clean":"2023-11-15T02:57:44.587040+0000","last_became_active":"2023-11-15T02:57:44.572147+0000","last_became_peered":"2023-11-15T02:57:44.572147+0000","last_unstale":"2023-11-15T02:57:44.587040+0000","last_undegraded":"2023-11-15T02:57:44.587040+0000","last_fullsized":"2023-11-15T02:57:44.587040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.553097+0000","last_change":"2023-11-15T02:57:44.094718+0000","last_active":"2023-11-15T02:57:44.553097+0000","last_peered":"2023-11-15T02:57:44.553097+0000","last_clean":"2023-11-15T02:57:44.553097+0000","last_became_active":"2023-11-15T02:57:44.094236+0000","last_became_peered":"2023-11-15T02:57:44.094236+0000","last_unstale":"2023-11-15T02:57:44.553097+0000","last_undegraded":"2023-11-15T02:57:44.553097+0000","last_fullsized":"2023-11-15T02:57:44.553097+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.580950+0000","last_change":"2023-11-15T02:57:44.577901+0000","last_active":"2023-11-15T02:57:44.580950+0000","last_peered":"2023-11-15T02:57:44.580950+0000","last_clean":"2023-11-15T02:57:44.580950+0000","last_became_active":"2023-11-15T02:57:44.577370+0000","last_became_peered":"2023-11-15T02:57:44.577370+0000","last_unstale":"2023-11-15T02:57:44.580950+0000","last_undegraded":"2023-11-15T02:57:44.580950+0000","last_fullsized":"2023-11-15T02:57:44.580950+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.556365+0000","last_change":"2023-11-15T02:57:44.109398+0000","last_active":"2023-11-15T02:57:44.556365+0000","last_peered":"2023-11-15T02:57:44.556365+0000","last_clean":"2023-11-15T02:57:44.556365+0000","last_became_active":"2023-11-15T02:57:44.104880+0000","last_became_peered":"2023-11-15T02:57:44.104880+0000","last_unstale":"2023-11-15T02:57:44.556365+0000","last_undegraded":"2023-11-15T02:57:44.556365+0000","last_fullsized":"2023-11-15T02:57:44.556365+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.574382+0000","last_change":"2023-11-15T02:57:44.570258+0000","last_active":"2023-11-15T02:57:44.574382+0000","last_peered":"2023-11-15T02:57:44.574382+0000","last_clean":"2023-11-15T02:57:44.574382+0000","last_became_active":"2023-11-15T02:57:44.569983+0000","last_became_peered":"2023-11-15T02:57:44.569983+0000","last_unstale":"2023-11-15T02:57:44.574382+0000","last_undegraded":"2023-11-15T02:57:44.574382+0000","last_fullsized":"2023-11-15T02:57:44.574382+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"32'3","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.598694+0000","last_change":"2023-11-15T02:57:44.096037+0000","last_active":"2023-11-15T02:57:44.598694+0000","last_peered":"2023-11-15T02:57:44.598694+0000","last_clean":"2023-11-15T02:57:44.598694+0000","last_became_active":"2023-11-15T02:57:44.095669+0000","last_became_peered":"2023-11-15T02:57:44.095669+0000","last_unstale":"2023-11-15T02:57:44.598694+0000","last_undegraded":"2023-11-15T02:57:44.598694+0000","last_fullsized":"2023-11-15T02:57:44.598694+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.589470+0000","last_change":"2023-11-15T02:57:44.585203+0000","last_active":"2023-11-15T02:57:44.589470+0000","last_peered":"2023-11-15T02:57:44.589470+0000","last_clean":"2023-11-15T02:57:44.589470+0000","last_became_active":"2023-11-15T02:57:44.583007+0000","last_became_peered":"2023-11-15T02:57:44.583007+0000","last_unstale":"2023-11-15T02:57:44.589470+0000","last_undegraded":"2023-11-15T02:57:44.589470+0000","last_fullsized":"2023-11-15T02:57:44.589470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.581428+0000","last_change":"2023-11-15T02:57:44.096764+0000","last_active":"2023-11-15T02:57:44.581428+0000","last_peered":"2023-11-15T02:57:44.581428+0000","last_clean":"2023-11-15T02:57:44.581428+0000","last_became_active":"2023-11-15T02:57:44.096348+0000","last_became_peered":"2023-11-15T02:57:44.096348+0000","last_unstale":"2023-11-15T02:57:44.581428+0000","last_undegraded":"2023-11-15T02:57:44.581428+0000","last_fullsized":"2023-11-15T02:57:44.581428+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.569510+0000","last_change":"2023-11-15T02:57:44.566200+0000","last_active":"2023-11-15T02:57:44.569510+0000","last_peered":"2023-11-15T02:57:44.569510+0000","last_clean":"2023-11-15T02:57:44.569510+0000","last_became_active":"2023-11-15T02:57:44.565943+0000","last_became_peered":"2023-11-15T02:57:44.565943+0000","last_unstale":"2023-11-15T02:57:44.569510+0000","last_undegraded":"2023-11-15T02:57:44.569510+0000","last_fullsized":"2023-11-15T02:57:44.569510+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.580735+0000","last_change":"2023-11-15T02:57:44.109346+0000","last_active":"2023-11-15T02:57:44.580735+0000","last_peered":"2023-11-15T02:57:44.580735+0000","last_clean":"2023-11-15T02:57:44.580735+0000","last_became_active":"2023-11-15T02:57:44.109023+0000","last_became_peered":"2023-11-15T02:57:44.109023+0000","last_unstale":"2023-11-15T02:57:44.580735+0000","last_undegraded":"2023-11-15T02:57:44.580735+0000","last_fullsized":"2023-11-15T02:57:44.580735+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582194+0000","last_change":"2023-11-15T02:57:44.566248+0000","last_active":"2023-11-15T02:57:44.582194+0000","last_peered":"2023-11-15T02:57:44.582194+0000","last_clean":"2023-11-15T02:57:44.582194+0000","last_became_active":"2023-11-15T02:57:44.566002+0000","last_became_peered":"2023-11-15T02:57:44.566002+0000","last_unstale":"2023-11-15T02:57:44.582194+0000","last_undegraded":"2023-11-15T02:57:44.582194+0000","last_fullsized":"2023-11-15T02:57:44.582194+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.567066+0000","last_change":"2023-11-15T02:57:44.093738+0000","last_active":"2023-11-15T02:57:44.567066+0000","last_peered":"2023-11-15T02:57:44.567066+0000","last_clean":"2023-11-15T02:57:44.567066+0000","last_became_active":"2023-11-15T02:57:44.093452+0000","last_became_peered":"2023-11-15T02:57:44.093452+0000","last_unstale":"2023-11-15T02:57:44.567066+0000","last_undegraded":"2023-11-15T02:57:44.567066+0000","last_fullsized":"2023-11-15T02:57:44.567066+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.589952+0000","last_change":"2023-11-15T02:57:44.588350+0000","last_active":"2023-11-15T02:57:44.589952+0000","last_peered":"2023-11-15T02:57:44.589952+0000","last_clean":"2023-11-15T02:57:44.589952+0000","last_became_active":"2023-11-15T02:57:44.588073+0000","last_became_peered":"2023-11-15T02:57:44.588073+0000","last_unstale":"2023-11-15T02:57:44.589952+0000","last_undegraded":"2023-11-15T02:57:44.589952+0000","last_fullsized":"2023-11-15T02:57:44.589952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.573815+0000","last_change":"2023-11-15T02:57:44.101276+0000","last_active":"2023-11-15T02:57:44.573815+0000","last_peered":"2023-11-15T02:57:44.573815+0000","last_clean":"2023-11-15T02:57:44.573815+0000","last_became_active":"2023-11-15T02:57:44.100908+0000","last_became_peered":"2023-11-15T02:57:44.100908+0000","last_unstale":"2023-11-15T02:57:44.573815+0000","last_undegraded":"2023-11-15T02:57:44.573815+0000","last_fullsized":"2023-11-15T02:57:44.573815+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582096+0000","last_change":"2023-11-15T02:57:44.566177+0000","last_active":"2023-11-15T02:57:44.582096+0000","last_peered":"2023-11-15T02:57:44.582096+0000","last_clean":"2023-11-15T02:57:44.582096+0000","last_became_active":"2023-11-15T02:57:44.565917+0000","last_became_peered":"2023-11-15T02:57:44.565917+0000","last_unstale":"2023-11-15T02:57:44.582096+0000","last_undegraded":"2023-11-15T02:57:44.582096+0000","last_fullsized":"2023-11-15T02:57:44.582096+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"32'4","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.587437+0000","last_change":"2023-11-15T02:57:44.102229+0000","last_active":"2023-11-15T02:57:44.587437+0000","last_peered":"2023-11-15T02:57:44.587437+0000","last_clean":"2023-11-15T02:57:44.587437+0000","last_became_active":"2023-11-15T02:57:44.101858+0000","last_became_peered":"2023-11-15T02:57:44.101858+0000","last_unstale":"2023-11-15T02:57:44.587437+0000","last_undegraded":"2023-11-15T02:57:44.587437+0000","last_fullsized":"2023-11-15T02:57:44.587437+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582784+0000","last_change":"2023-11-15T02:57:44.578142+0000","last_active":"2023-11-15T02:57:44.582784+0000","last_peered":"2023-11-15T02:57:44.582784+0000","last_clean":"2023-11-15T02:57:44.582784+0000","last_became_active":"2023-11-15T02:57:44.575328+0000","last_became_peered":"2023-11-15T02:57:44.575328+0000","last_unstale":"2023-11-15T02:57:44.582784+0000","last_undegraded":"2023-11-15T02:57:44.582784+0000","last_fullsized":"2023-11-15T02:57:44.582784+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.593940+0000","last_change":"2023-11-15T02:57:44.100857+0000","last_active":"2023-11-15T02:57:44.593940+0000","last_peered":"2023-11-15T02:57:44.593940+0000","last_clean":"2023-11-15T02:57:44.593940+0000","last_became_active":"2023-11-15T02:57:44.099959+0000","last_became_peered":"2023-11-15T02:57:44.099959+0000","last_unstale":"2023-11-15T02:57:44.593940+0000","last_undegraded":"2023-11-15T02:57:44.593940+0000","last_fullsized":"2023-11-15T02:57:44.593940+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.583371+0000","last_change":"2023-11-15T02:57:44.576443+0000","last_active":"2023-11-15T02:57:44.583371+0000","last_peered":"2023-11-15T02:57:44.583371+0000","last_clean":"2023-11-15T02:57:44.583371+0000","last_became_active":"2023-11-15T02:57:44.576085+0000","last_became_peered":"2023-11-15T02:57:44.576085+0000","last_unstale":"2023-11-15T02:57:44.583371+0000","last_undegraded":"2023-11-15T02:57:44.583371+0000","last_fullsized":"2023-11-15T02:57:44.583371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585761+0000","last_change":"2023-11-15T02:57:44.096009+0000","last_active":"2023-11-15T02:57:44.585761+0000","last_peered":"2023-11-15T02:57:44.585761+0000","last_clean":"2023-11-15T02:57:44.585761+0000","last_became_active":"2023-11-15T02:57:44.095541+0000","last_became_peered":"2023-11-15T02:57:44.095541+0000","last_unstale":"2023-11-15T02:57:44.585761+0000","last_undegraded":"2023-11-15T02:57:44.585761+0000","last_fullsized":"2023-11-15T02:57:44.585761+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.587385+0000","last_change":"2023-11-15T02:57:44.583166+0000","last_active":"2023-11-15T02:57:44.587385+0000","last_peered":"2023-11-15T02:57:44.587385+0000","last_clean":"2023-11-15T02:57:44.587385+0000","last_became_active":"2023-11-15T02:57:44.582842+0000","last_became_peered":"2023-11-15T02:57:44.582842+0000","last_unstale":"2023-11-15T02:57:44.587385+0000","last_undegraded":"2023-11-15T02:57:44.587385+0000","last_fullsized":"2023-11-15T02:57:44.587385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.561968+0000","last_change":"2023-11-15T02:57:44.101593+0000","last_active":"2023-11-15T02:57:44.561968+0000","last_peered":"2023-11-15T02:57:44.561968+0000","last_clean":"2023-11-15T02:57:44.561968+0000","last_became_active":"2023-11-15T02:57:44.101261+0000","last_became_peered":"2023-11-15T02:57:44.101261+0000","last_unstale":"2023-11-15T02:57:44.561968+0000","last_undegraded":"2023-11-15T02:57:44.561968+0000","last_fullsized":"2023-11-15T02:57:44.561968+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.589707+0000","last_change":"2023-11-15T02:57:44.584677+0000","last_active":"2023-11-15T02:57:44.589707+0000","last_peered":"2023-11-15T02:57:44.589707+0000","last_clean":"2023-11-15T02:57:44.589707+0000","last_became_active":"2023-11-15T02:57:44.584394+0000","last_became_peered":"2023-11-15T02:57:44.584394+0000","last_unstale":"2023-11-15T02:57:44.589707+0000","last_undegraded":"2023-11-15T02:57:44.589707+0000","last_fullsized":"2023-11-15T02:57:44.589707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.567985+0000","last_change":"2023-11-15T02:57:44.104181+0000","last_active":"2023-11-15T02:57:44.567985+0000","last_peered":"2023-11-15T02:57:44.567985+0000","last_clean":"2023-11-15T02:57:44.567985+0000","last_became_active":"2023-11-15T02:57:44.100596+0000","last_became_peered":"2023-11-15T02:57:44.100596+0000","last_unstale":"2023-11-15T02:57:44.567985+0000","last_undegraded":"2023-11-15T02:57:44.567985+0000","last_fullsized":"2023-11-15T02:57:44.567985+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.591569+0000","last_change":"2023-11-15T02:57:44.590180+0000","last_active":"2023-11-15T02:57:44.591569+0000","last_peered":"2023-11-15T02:57:44.591569+0000","last_clean":"2023-11-15T02:57:44.591569+0000","last_became_active":"2023-11-15T02:57:44.589752+0000","last_became_peered":"2023-11-15T02:57:44.589752+0000","last_unstale":"2023-11-15T02:57:44.591569+0000","last_undegraded":"2023-11-15T02:57:44.591569+0000","last_fullsized":"2023-11-15T02:57:44.591569+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.587117+0000","last_change":"2023-11-15T02:57:44.109252+0000","last_active":"2023-11-15T02:57:44.587117+0000","last_peered":"2023-11-15T02:57:44.587117+0000","last_clean":"2023-11-15T02:57:44.587117+0000","last_became_active":"2023-11-15T02:57:44.108907+0000","last_became_peered":"2023-11-15T02:57:44.108907+0000","last_unstale":"2023-11-15T02:57:44.587117+0000","last_undegraded":"2023-11-15T02:57:44.587117+0000","last_fullsized":"2023-11-15T02:57:44.587117+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.573670+0000","last_change":"2023-11-15T02:57:44.568542+0000","last_active":"2023-11-15T02:57:44.573670+0000","last_peered":"2023-11-15T02:57:44.573670+0000","last_clean":"2023-11-15T02:57:44.573670+0000","last_became_active":"2023-11-15T02:57:44.568269+0000","last_became_peered":"2023-11-15T02:57:44.568269+0000","last_unstale":"2023-11-15T02:57:44.573670+0000","last_undegraded":"2023-11-15T02:57:44.573670+0000","last_fullsized":"2023-11-15T02:57:44.573670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560477+0000","last_change":"2023-11-15T02:57:44.097937+0000","last_active":"2023-11-15T02:57:44.560477+0000","last_peered":"2023-11-15T02:57:44.560477+0000","last_clean":"2023-11-15T02:57:44.560477+0000","last_became_active":"2023-11-15T02:57:44.097630+0000","last_became_peered":"2023-11-15T02:57:44.097630+0000","last_unstale":"2023-11-15T02:57:44.560477+0000","last_undegraded":"2023-11-15T02:57:44.560477+0000","last_fullsized":"2023-11-15T02:57:44.560477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582522+0000","last_change":"2023-11-15T02:57:44.579127+0000","last_active":"2023-11-15T02:57:44.582522+0000","last_peered":"2023-11-15T02:57:44.582522+0000","last_clean":"2023-11-15T02:57:44.582522+0000","last_became_active":"2023-11-15T02:57:44.578854+0000","last_became_peered":"2023-11-15T02:57:44.578854+0000","last_unstale":"2023-11-15T02:57:44.582522+0000","last_undegraded":"2023-11-15T02:57:44.582522+0000","last_fullsized":"2023-11-15T02:57:44.582522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560883+0000","last_change":"2023-11-15T02:57:44.108255+0000","last_active":"2023-11-15T02:57:44.560883+0000","last_peered":"2023-11-15T02:57:44.560883+0000","last_clean":"2023-11-15T02:57:44.560883+0000","last_became_active":"2023-11-15T02:57:44.107961+0000","last_became_peered":"2023-11-15T02:57:44.107961+0000","last_unstale":"2023-11-15T02:57:44.560883+0000","last_undegraded":"2023-11-15T02:57:44.560883+0000","last_fullsized":"2023-11-15T02:57:44.560883+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582428+0000","last_change":"2023-11-15T02:57:44.570926+0000","last_active":"2023-11-15T02:57:44.582428+0000","last_peered":"2023-11-15T02:57:44.582428+0000","last_clean":"2023-11-15T02:57:44.582428+0000","last_became_active":"2023-11-15T02:57:44.570646+0000","last_became_peered":"2023-11-15T02:57:44.570646+0000","last_unstale":"2023-11-15T02:57:44.582428+0000","last_undegraded":"2023-11-15T02:57:44.582428+0000","last_fullsized":"2023-11-15T02:57:44.582428+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.562251+0000","last_change":"2023-11-15T02:57:44.105564+0000","last_active":"2023-11-15T02:57:44.562251+0000","last_peered":"2023-11-15T02:57:44.562251+0000","last_clean":"2023-11-15T02:57:44.562251+0000","last_became_active":"2023-11-15T02:57:44.105312+0000","last_became_peered":"2023-11-15T02:57:44.105312+0000","last_unstale":"2023-11-15T02:57:44.562251+0000","last_undegraded":"2023-11-15T02:57:44.562251+0000","last_fullsized":"2023-11-15T02:57:44.562251+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585305+0000","last_change":"2023-11-15T02:57:44.580419+0000","last_active":"2023-11-15T02:57:44.585305+0000","last_peered":"2023-11-15T02:57:44.585305+0000","last_clean":"2023-11-15T02:57:44.585305+0000","last_became_active":"2023-11-15T02:57:44.580057+0000","last_became_peered":"2023-11-15T02:57:44.580057+0000","last_unstale":"2023-11-15T02:57:44.585305+0000","last_undegraded":"2023-11-15T02:57:44.585305+0000","last_fullsized":"2023-11-15T02:57:44.585305+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.551763+0000","last_change":"2023-11-15T02:57:44.097433+0000","last_active":"2023-11-15T02:57:44.551763+0000","last_peered":"2023-11-15T02:57:44.551763+0000","last_clean":"2023-11-15T02:57:44.551763+0000","last_became_active":"2023-11-15T02:57:44.097149+0000","last_became_peered":"2023-11-15T02:57:44.097149+0000","last_unstale":"2023-11-15T02:57:44.551763+0000","last_undegraded":"2023-11-15T02:57:44.551763+0000","last_fullsized":"2023-11-15T02:57:44.551763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.574489+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T02:57:44.574489+0000","last_peered":"2023-11-15T02:57:44.574489+0000","last_clean":"2023-11-15T02:57:44.574489+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T02:57:44.574489+0000","last_undegraded":"2023-11-15T02:57:44.574489+0000","last_fullsized":"2023-11-15T02:57:44.574489+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.589179+0000","last_change":"2023-11-15T02:57:44.584602+0000","last_active":"2023-11-15T02:57:44.589179+0000","last_peered":"2023-11-15T02:57:44.589179+0000","last_clean":"2023-11-15T02:57:44.589179+0000","last_became_active":"2023-11-15T02:57:44.584333+0000","last_became_peered":"2023-11-15T02:57:44.584333+0000","last_unstale":"2023-11-15T02:57:44.589179+0000","last_undegraded":"2023-11-15T02:57:44.589179+0000","last_fullsized":"2023-11-15T02:57:44.589179+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.561148+0000","last_change":"2023-11-15T02:57:44.099697+0000","last_active":"2023-11-15T02:57:44.561148+0000","last_peered":"2023-11-15T02:57:44.561148+0000","last_clean":"2023-11-15T02:57:44.561148+0000","last_became_active":"2023-11-15T02:57:44.099388+0000","last_became_peered":"2023-11-15T02:57:44.099388+0000","last_unstale":"2023-11-15T02:57:44.561148+0000","last_undegraded":"2023-11-15T02:57:44.561148+0000","last_fullsized":"2023-11-15T02:57:44.561148+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.570736+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T02:57:44.570736+0000","last_peered":"2023-11-15T02:57:44.570736+0000","last_clean":"2023-11-15T02:57:44.570736+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T02:57:44.570736+0000","last_undegraded":"2023-11-15T02:57:44.570736+0000","last_fullsized":"2023-11-15T02:57:44.570736+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.577326+0000","last_change":"2023-11-15T02:57:44.573248+0000","last_active":"2023-11-15T02:57:44.577326+0000","last_peered":"2023-11-15T02:57:44.577326+0000","last_clean":"2023-11-15T02:57:44.577326+0000","last_became_active":"2023-11-15T02:57:44.572993+0000","last_became_peered":"2023-11-15T02:57:44.572993+0000","last_unstale":"2023-11-15T02:57:44.577326+0000","last_undegraded":"2023-11-15T02:57:44.577326+0000","last_fullsized":"2023-11-15T02:57:44.577326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.575526+0000","last_change":"2023-11-15T02:57:44.097789+0000","last_active":"2023-11-15T02:57:44.575526+0000","last_peered":"2023-11-15T02:57:44.575526+0000","last_clean":"2023-11-15T02:57:44.575526+0000","last_became_active":"2023-11-15T02:57:44.097133+0000","last_became_peered":"2023-11-15T02:57:44.097133+0000","last_unstale":"2023-11-15T02:57:44.575526+0000","last_undegraded":"2023-11-15T02:57:44.575526+0000","last_fullsized":"2023-11-15T02:57:44.575526+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.564074+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T02:57:44.564074+0000","last_peered":"2023-11-15T02:57:44.564074+0000","last_clean":"2023-11-15T02:57:44.564074+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T02:57:44.564074+0000","last_undegraded":"2023-11-15T02:57:44.564074+0000","last_fullsized":"2023-11-15T02:57:44.564074+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.585291+0000","last_change":"2023-11-15T02:57:44.582488+0000","last_active":"2023-11-15T02:57:44.585291+0000","last_peered":"2023-11-15T02:57:44.585291+0000","last_clean":"2023-11-15T02:57:44.585291+0000","last_became_active":"2023-11-15T02:57:44.582160+0000","last_became_peered":"2023-11-15T02:57:44.582160+0000","last_unstale":"2023-11-15T02:57:44.585291+0000","last_undegraded":"2023-11-15T02:57:44.585291+0000","last_fullsized":"2023-11-15T02:57:44.585291+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560320+0000","last_change":"2023-11-15T02:57:44.110737+0000","last_active":"2023-11-15T02:57:44.560320+0000","last_peered":"2023-11-15T02:57:44.560320+0000","last_clean":"2023-11-15T02:57:44.560320+0000","last_became_active":"2023-11-15T02:57:44.110423+0000","last_became_peered":"2023-11-15T02:57:44.110423+0000","last_unstale":"2023-11-15T02:57:44.560320+0000","last_undegraded":"2023-11-15T02:57:44.560320+0000","last_fullsized":"2023-11-15T02:57:44.560320+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.568173+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T02:57:44.568173+0000","last_peered":"2023-11-15T02:57:44.568173+0000","last_clean":"2023-11-15T02:57:44.568173+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T02:57:44.568173+0000","last_undegraded":"2023-11-15T02:57:44.568173+0000","last_fullsized":"2023-11-15T02:57:44.568173+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.582658+0000","last_change":"2023-11-15T02:57:44.577142+0000","last_active":"2023-11-15T02:57:44.582658+0000","last_peered":"2023-11-15T02:57:44.582658+0000","last_clean":"2023-11-15T02:57:44.582658+0000","last_became_active":"2023-11-15T02:57:44.576883+0000","last_became_peered":"2023-11-15T02:57:44.576883+0000","last_unstale":"2023-11-15T02:57:44.582658+0000","last_undegraded":"2023-11-15T02:57:44.582658+0000","last_fullsized":"2023-11-15T02:57:44.582658+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.559540+0000","last_change":"2023-11-15T02:57:44.111033+0000","last_active":"2023-11-15T02:57:44.559540+0000","last_peered":"2023-11-15T02:57:44.559540+0000","last_clean":"2023-11-15T02:57:44.559540+0000","last_became_active":"2023-11-15T02:57:44.109168+0000","last_became_peered":"2023-11-15T02:57:44.109168+0000","last_unstale":"2023-11-15T02:57:44.559540+0000","last_undegraded":"2023-11-15T02:57:44.559540+0000","last_fullsized":"2023-11-15T02:57:44.559540+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.556390+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T02:57:44.556390+0000","last_peered":"2023-11-15T02:57:44.556390+0000","last_clean":"2023-11-15T02:57:44.556390+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T02:57:44.556390+0000","last_undegraded":"2023-11-15T02:57:44.556390+0000","last_fullsized":"2023-11-15T02:57:44.556390+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.580969+0000","last_change":"2023-11-15T02:57:44.577944+0000","last_active":"2023-11-15T02:57:44.580969+0000","last_peered":"2023-11-15T02:57:44.580969+0000","last_clean":"2023-11-15T02:57:44.580969+0000","last_became_active":"2023-11-15T02:57:44.577738+0000","last_became_peered":"2023-11-15T02:57:44.577738+0000","last_unstale":"2023-11-15T02:57:44.580969+0000","last_undegraded":"2023-11-15T02:57:44.580969+0000","last_fullsized":"2023-11-15T02:57:44.580969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.561951+0000","last_change":"2023-11-15T02:57:44.098264+0000","last_active":"2023-11-15T02:57:44.561951+0000","last_peered":"2023-11-15T02:57:44.561951+0000","last_clean":"2023-11-15T02:57:44.561951+0000","last_became_active":"2023-11-15T02:57:44.095009+0000","last_became_peered":"2023-11-15T02:57:44.095009+0000","last_unstale":"2023-11-15T02:57:44.561951+0000","last_undegraded":"2023-11-15T02:57:44.561951+0000","last_fullsized":"2023-11-15T02:57:44.561951+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560296+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T02:57:44.560296+0000","last_peered":"2023-11-15T02:57:44.560296+0000","last_clean":"2023-11-15T02:57:44.560296+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T02:57:44.560296+0000","last_undegraded":"2023-11-15T02:57:44.560296+0000","last_fullsized":"2023-11-15T02:57:44.560296+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.587338+0000","last_change":"2023-11-15T02:57:44.581053+0000","last_active":"2023-11-15T02:57:44.587338+0000","last_peered":"2023-11-15T02:57:44.587338+0000","last_clean":"2023-11-15T02:57:44.587338+0000","last_became_active":"2023-11-15T02:57:44.580765+0000","last_became_peered":"2023-11-15T02:57:44.580765+0000","last_unstale":"2023-11-15T02:57:44.587338+0000","last_undegraded":"2023-11-15T02:57:44.587338+0000","last_fullsized":"2023-11-15T02:57:44.587338+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.596704+0000","last_change":"2023-11-15T02:57:44.095125+0000","last_active":"2023-11-15T02:57:44.596704+0000","last_peered":"2023-11-15T02:57:44.596704+0000","last_clean":"2023-11-15T02:57:44.596704+0000","last_became_active":"2023-11-15T02:57:44.094525+0000","last_became_peered":"2023-11-15T02:57:44.094525+0000","last_unstale":"2023-11-15T02:57:44.596704+0000","last_undegraded":"2023-11-15T02:57:44.596704+0000","last_fullsized":"2023-11-15T02:57:44.596704+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.560867+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T02:57:44.560867+0000","last_peered":"2023-11-15T02:57:44.560867+0000","last_clean":"2023-11-15T02:57:44.560867+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T02:57:44.560867+0000","last_undegraded":"2023-11-15T02:57:44.560867+0000","last_fullsized":"2023-11-15T02:57:44.560867+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.593320+0000","last_change":"2023-11-15T02:57:44.584057+0000","last_active":"2023-11-15T02:57:44.593320+0000","last_peered":"2023-11-15T02:57:44.593320+0000","last_clean":"2023-11-15T02:57:44.593320+0000","last_became_active":"2023-11-15T02:57:44.583712+0000","last_became_peered":"2023-11-15T02:57:44.583712+0000","last_unstale":"2023-11-15T02:57:44.593320+0000","last_undegraded":"2023-11-15T02:57:44.593320+0000","last_fullsized":"2023-11-15T02:57:44.593320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:44.054985+0000","last_clean_scrub_stamp":"2023-11-15T02:57:44.054985+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.563584+0000","last_change":"2023-11-15T02:57:44.111330+0000","last_active":"2023-11-15T02:57:44.563584+0000","last_peered":"2023-11-15T02:57:44.563584+0000","last_clean":"2023-11-15T02:57:44.563584+0000","last_became_active":"2023-11-15T02:57:44.111040+0000","last_became_peered":"2023-11-15T02:57:44.111040+0000","last_unstale":"2023-11-15T02:57:44.563584+0000","last_undegraded":"2023-11-15T02:57:44.563584+0000","last_fullsized":"2023-11-15T02:57:44.563584+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:57:43.048700+0000","last_clean_scrub_stamp":"2023-11-15T02:57:43.048700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.556502+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T02:57:44.556502+0000","last_peered":"2023-11-15T02:57:44.556502+0000","last_clean":"2023-11-15T02:57:44.556502+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T02:57:44.556502+0000","last_undegraded":"2023-11-15T02:57:44.556502+0000","last_fullsized":"2023-11-15T02:57:44.556502+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-15T02:57:44.561414+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T02:57:44.561414+0000","last_peered":"2023-11-15T02:57:44.561414+0000","last_clean":"2023-11-15T02:57:44.561414+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T02:57:44.561414+0000","last_undegraded":"2023-11-15T02:57:44.561414+0000","last_fullsized":"2023-11-15T02:57:44.561414+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542159,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297464,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074376,"statfs":{"total":96636764160,"available":96332161024,"internally_reserved":0,"allocated":376832,"data_stored":153054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:57:48 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.196,"15min":1.196},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.952,"5min":3.952,"15min":3.952},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":0.93500000000000005}]},{"osd":1,"last update":"Wed Nov 15 02:57:48 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.087},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.294}]},{"osd":2,"last update":"Wed Nov 15 02:57:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.6829999999999998,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.5939999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":1.1459999999999999}]},{"osd":3,"last update":"Wed Nov 15 02:57:48 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":4.1159999999999997,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.29,"15min":1.29},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.335}]},{"osd":4,"last update":"Wed Nov 15 02:57:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.5649999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":4.226,"5min":4.226,"15min":4.226},"last":1.357}]},{"osd":5,"last update":"Wed Nov 15 02:57:48 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":1.056,"5min":1.056,"15min":1.056},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":1.266},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.4750000000000001}]},{"osd":6,"last update":"Wed Nov 15 02:57:48 2023","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":1.262,"5min":1.262,"15min":1.262},"max":{"1min":3.2789999999999999,"5min":3.2789999999999999,"15min":3.2789999999999999},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"min":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"max":{"1min":4.7800000000000002,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":1.2170000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297460,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074380,"statfs":{"total":96636764160,"available":96332165120,"internally_reserved":0,"allocated":372736,"data_stored":152964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":4.3159999999999998,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":1.343},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.88200000000000001}]},{"osd":1,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":1.389},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.8570000000000002,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.532}]},{"osd":2,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.621,"5min":2.621,"15min":2.621},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":1.4970000000000001}]},{"osd":3,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.8359999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.474}]},{"osd":4,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":4.5350000000000001,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":1.224},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.0149999999999999}]},{"osd":5,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.9460000000000002,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":1.456}]},{"osd":7,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":3.1989999999999998,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.663},{"interface":"front","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.9350000000000001,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":1.6479999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297460,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074380,"statfs":{"total":96636764160,"available":96332165120,"internally_reserved":0,"allocated":372736,"data_stored":152432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.089},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":1.0309999999999999}]},{"osd":1,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":2.617,"5min":2.617,"15min":2.617},"last":1.778},{"interface":"front","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":1.29}]},{"osd":2,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.131}]},{"osd":3,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.875},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.5630000000000002,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":1.923}]},{"osd":5,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.5720000000000001}]},{"osd":6,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":2.6640000000000001,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":2.0059999999999998},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.968}]},{"osd":7,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":1.833},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.7669999999999999,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":2.0939999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":389120,"data_stored":154050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.79300000000000004}]},{"osd":1,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":1.1950000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":1.375}]},{"osd":2,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.1279999999999999}]},{"osd":4,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.262}]},{"osd":5,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.504},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.4590000000000001}]},{"osd":6,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":1.048,"5min":1.048,"15min":1.048},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"min":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.677}]},{"osd":7,"last update":"Wed Nov 15 02:57:46 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":1.883,"5min":1.883,"15min":1.883},"last":1.6339999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":1.085,"5min":1.085,"15min":1.085},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.429}]}]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297452,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074388,"statfs":{"total":96636764160,"available":96332173312,"internally_reserved":0,"allocated":364544,"data_stored":151141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:57:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.91200000000000003}]},{"osd":1,"last update":"Wed Nov 15 02:57:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.875},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.026}]},{"osd":3,"last update":"Wed Nov 15 02:57:45 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":1.147},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.171}]},{"osd":4,"last update":"Wed Nov 15 02:57:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.226,"15min":1.226},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.3109999999999999}]},{"osd":5,"last update":"Wed Nov 15 02:57:45 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.425},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":1.379}]},{"osd":6,"last update":"Wed Nov 15 02:57:45 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":1.119,"5min":1.119,"15min":1.119},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.258}]},{"osd":7,"last update":"Wed Nov 15 02:57:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.397},{"interface":"front","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.341}]}]},{"osd":0,"up_from":14,"seq":60129542160,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297460,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074380,"statfs":{"total":96636764160,"available":96332165120,"internally_reserved":0,"allocated":372736,"data_stored":152444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":1.0549999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.2839999999999998,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.81799999999999995}]},{"osd":3,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":3.54,"5min":3.54,"15min":3.54},"last":1.093}]},{"osd":4,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.85599999999999998}]},{"osd":5,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":3.6760000000000002,"5min":3.6760000000000002,"15min":3.6760000000000002},"last":1.149}]},{"osd":6,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":3.71,"5min":3.71,"15min":3.71},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":1.024,"5min":1.024,"15min":1.024},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.165}]},{"osd":7,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":1.032,"5min":1.032,"15min":1.032},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.238},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.2090000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297472,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074368,"statfs":{"total":96636764160,"available":96332152832,"internally_reserved":0,"allocated":385024,"data_stored":153968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":3.46,"5min":3.46,"15min":3.46},"last":0.86399999999999999}]},{"osd":1,"last update":"Wed Nov 15 02:57:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":3.6349999999999998,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":1.105},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.516,"5min":3.516,"15min":3.516},"last":0.99099999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":5.4180000000000001,"5min":5.4180000000000001,"15min":5.4180000000000001},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":3.5859999999999999,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":1.073}]},{"osd":3,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":1.208},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":4.2729999999999997,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":1.286}]},{"osd":4,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":3.706,"5min":3.706,"15min":3.706},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.659,"5min":1.659,"15min":1.659},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":6.1180000000000003,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":1.159}]},{"osd":6,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.607,"5min":1.607,"15min":1.607},"min":{"1min":1.123,"5min":1.123,"15min":1.123},"max":{"1min":3.641,"5min":3.641,"15min":3.641},"last":1.1299999999999999}]},{"osd":7,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"min":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":3.516,"5min":3.516,"15min":3.516},"last":1.054},{"interface":"front","average":{"1min":1.468,"5min":1.468,"15min":1.468},"min":{"1min":1.022,"5min":1.022,"15min":1.022},"max":{"1min":3.754,"5min":3.754,"15min":3.754},"last":1.2270000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297464,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074376,"statfs":{"total":96636764160,"available":96332161024,"internally_reserved":0,"allocated":376832,"data_stored":152570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":1.016}]},{"osd":2,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.96799999999999997}]},{"osd":3,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.173},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.141}]},{"osd":4,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":1.2809999999999999}]},{"osd":5,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.4609999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.2310000000000001}]},{"osd":6,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.601}]},{"osd":7,"last update":"Wed Nov 15 02:57:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.671}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T02:57:53.958 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-15T02:57:53.961 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-15T02:57:54.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.159+0000 7f511f59e700 1 Processor -- start 2023-11-15T02:57:54.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.159+0000 7f511f59e700 1 -- start start 2023-11-15T02:57:54.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.163+0000 7f511f59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51200a9f80 0x7f5120158260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:54.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.163+0000 7f511f59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f51200c7a90 0x7f5120154b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:54.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.163+0000 7f511f59e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51200c6170 con 0x7f51200c5a00 2023-11-15T02:57:54.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.163+0000 7f511f59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5120154000 con 0x7f51200c7a90 2023-11-15T02:57:54.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.163+0000 7f511f59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5120157dd0 con 0x7f51200a9f80 2023-11-15T02:57:54.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.163+0000 7f511ed9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f51200c7a90 0x7f5120154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:54.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.163+0000 7f511ed9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f51200c7a90 0x7f5120154b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56502/0 (socket says 172.21.15.121:56502) 2023-11-15T02:57:54.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.163+0000 7f511ed9d700 1 -- 172.21.15.121:0/1023058081 learned_addr learned my addr 172.21.15.121:0/1023058081 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:54.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.163+0000 7f511dd9b700 1 --1- 172.21.15.121:0/1023058081 >> v1:172.21.15.190:6789/0 conn(0x7f51200c5a00 0x7f51200c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:44090/0 (socket says 172.21.15.121:44090) 2023-11-15T02:57:54.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.163+0000 7f511e59c700 1 --2- 172.21.15.121:0/1023058081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51200a9f80 0x7f5120158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:54.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.167+0000 7f511d59a700 1 -- 172.21.15.121:0/1023058081 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2556291355 0 0) 0x7f51200c6170 con 0x7f51200c5a00 2023-11-15T02:57:54.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.167+0000 7f511d59a700 1 -- 172.21.15.121:0/1023058081 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51080033f0 con 0x7f51200c5a00 2023-11-15T02:57:54.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.167+0000 7f511ed9d700 1 -- 172.21.15.121:0/1023058081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51200a9f80 msgr2=0x7f5120158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:54.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.167+0000 7f511ed9d700 1 --2- 172.21.15.121:0/1023058081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51200a9f80 0x7f5120158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:54.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.167+0000 7f511ed9d700 1 -- 172.21.15.121:0/1023058081 >> v1:172.21.15.190:6789/0 conn(0x7f51200c5a00 legacy=0x7f51200c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:54.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.167+0000 7f511ed9d700 1 -- 172.21.15.121:0/1023058081 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5120155080 con 0x7f51200c7a90 2023-11-15T02:57:54.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.167+0000 7f511ed9d700 1 --2- 172.21.15.121:0/1023058081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51200c7a90 0x7f5120154b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f5118009f20 tx=0x7f5118009840).ready entity=mon.0 client_cookie=60fc205c8d725bf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:54.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.167+0000 7f511d59a700 1 -- 172.21.15.121:0/1023058081 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511800e070 con 0x7f51200c7a90 2023-11-15T02:57:54.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.167+0000 7f511d59a700 1 -- 172.21.15.121:0/1023058081 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511800b970 con 0x7f51200c7a90 2023-11-15T02:57:54.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.167+0000 7f511d59a700 1 -- 172.21.15.121:0/1023058081 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511800bcf0 con 0x7f51200c7a90 2023-11-15T02:57:54.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.167+0000 7f511f59e700 1 -- 172.21.15.121:0/1023058081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51200c7a90 msgr2=0x7f5120154b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:54.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.167+0000 7f511f59e700 1 --2- 172.21.15.121:0/1023058081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51200c7a90 0x7f5120154b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f5118009f20 tx=0x7f5118009840).stop 2023-11-15T02:57:54.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.167+0000 7f511f59e700 1 -- 172.21.15.121:0/1023058081 shutdown_connections 2023-11-15T02:57:54.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.167+0000 7f511f59e700 1 --2- 172.21.15.121:0/1023058081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51200c7a90 0x7f5120154b90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:54.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.167+0000 7f511f59e700 1 --2- 172.21.15.121:0/1023058081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51200a9f80 0x7f5120158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:54.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.167+0000 7f511f59e700 1 -- 172.21.15.121:0/1023058081 >> 172.21.15.121:0/1023058081 conn(0x7f5120067a60 msgr2=0x7f5120067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:54.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.167+0000 7f511f59e700 1 -- 172.21.15.121:0/1023058081 shutdown_connections 2023-11-15T02:57:54.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.167+0000 7f511f59e700 1 -- 172.21.15.121:0/1023058081 wait complete. 2023-11-15T02:57:54.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511f59e700 1 Processor -- start 2023-11-15T02:57:54.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511f59e700 1 -- start start 2023-11-15T02:57:54.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511f59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51200a9f80 0x7f51201432e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:54.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511f59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51200c7a90 0x7f5120066130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:54.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511f59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5120066620 0x7f5120163980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:54.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511f59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51201652d0 con 0x7f51200c7a90 2023-11-15T02:57:54.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511f59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5120066aa0 con 0x7f5120066620 2023-11-15T02:57:54.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511f59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51201646c0 con 0x7f51200a9f80 2023-11-15T02:57:54.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511dd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51200c7a90 0x7f5120066130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:54.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511dd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51200c7a90 0x7f5120066130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56514/0 (socket says 172.21.15.121:56514) 2023-11-15T02:57:54.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511dd9b700 1 -- 172.21.15.121:0/2293134427 learned_addr learned my addr 172.21.15.121:0/2293134427 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:54.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511ed9d700 1 --2- 172.21.15.121:0/2293134427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5120066620 0x7f5120163980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:54.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511e59c700 1 --2- 172.21.15.121:0/2293134427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51200a9f80 0x7f51201432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:54.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511dd9b700 1 -- 172.21.15.121:0/2293134427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51200a9f80 msgr2=0x7f51201432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:54.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511dd9b700 1 --2- 172.21.15.121:0/2293134427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51200a9f80 0x7f51201432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:54.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511dd9b700 1 -- 172.21.15.121:0/2293134427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5120066620 msgr2=0x7f5120163980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:54.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511dd9b700 1 --2- 172.21.15.121:0/2293134427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5120066620 0x7f5120163980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:54.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511dd9b700 1 -- 172.21.15.121:0/2293134427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5118009ae0 con 0x7f51200c7a90 2023-11-15T02:57:54.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511dd9b700 1 --2- 172.21.15.121:0/2293134427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51200c7a90 0x7f5120066130 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f511400c4f0 tx=0x7f5114007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:54.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f510e7fc700 1 -- 172.21.15.121:0/2293134427 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51140072e0 con 0x7f51200c7a90 2023-11-15T02:57:54.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f510e7fc700 1 -- 172.21.15.121:0/2293134427 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5114007440 con 0x7f51200c7a90 2023-11-15T02:57:54.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511f59e700 1 -- 172.21.15.121:0/2293134427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51201649a0 con 0x7f51200c7a90 2023-11-15T02:57:54.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f510e7fc700 1 -- 172.21.15.121:0/2293134427 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114005910 con 0x7f51200c7a90 2023-11-15T02:57:54.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.171+0000 7f511f59e700 1 -- 172.21.15.121:0/2293134427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5120164f60 con 0x7f51200c7a90 2023-11-15T02:57:54.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.175+0000 7f511f59e700 1 -- 172.21.15.121:0/2293134427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5120165480 con 0x7f51200c7a90 2023-11-15T02:57:54.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.175+0000 7f510e7fc700 1 -- 172.21.15.121:0/2293134427 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5114005d00 con 0x7f51200c7a90 2023-11-15T02:57:54.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.175+0000 7f510e7fc700 1 --2- 172.21.15.121:0/2293134427 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f510005a510 0x7f510005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:54.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.175+0000 7f510e7fc700 1 -- 172.21.15.121:0/2293134427 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5114074020 con 0x7f51200c7a90 2023-11-15T02:57:54.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.179+0000 7f511e59c700 1 --2- 172.21.15.121:0/2293134427 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f510005a510 0x7f510005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:54.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.179+0000 7f511e59c700 1 --2- 172.21.15.121:0/2293134427 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f510005a510 0x7f510005c9d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f5110009bc0 tx=0x7f5110008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:54.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.183+0000 7f510e7fc700 1 -- 172.21.15.121:0/2293134427 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5120165480 con 0x7f51200c7a90 2023-11-15T02:57:54.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.399+0000 7f511f59e700 1 -- 172.21.15.121:0/2293134427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5120165480 con 0x7f51200c7a90 2023-11-15T02:57:54.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.407+0000 7f510e7fc700 1 -- 172.21.15.121:0/2293134427 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f5120165480 con 0x7f51200c7a90 2023-11-15T02:57:54.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.411+0000 7f511f59e700 1 -- 172.21.15.121:0/2293134427 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f510005a510 msgr2=0x7f510005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:54.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.411+0000 7f511f59e700 1 --2- 172.21.15.121:0/2293134427 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f510005a510 0x7f510005c9d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f5110009bc0 tx=0x7f5110008040).stop 2023-11-15T02:57:54.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.411+0000 7f511f59e700 1 -- 172.21.15.121:0/2293134427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51200c7a90 msgr2=0x7f5120066130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:54.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.411+0000 7f511f59e700 1 --2- 172.21.15.121:0/2293134427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51200c7a90 0x7f5120066130 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f511400c4f0 tx=0x7f5114007c50).stop 2023-11-15T02:57:54.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.411+0000 7f511f59e700 1 -- 172.21.15.121:0/2293134427 shutdown_connections 2023-11-15T02:57:54.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.411+0000 7f511f59e700 1 --2- 172.21.15.121:0/2293134427 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f510005a510 0x7f510005c9d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:54.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.411+0000 7f511f59e700 1 --2- 172.21.15.121:0/2293134427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5120066620 0x7f5120163980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:54.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.411+0000 7f511f59e700 1 --2- 172.21.15.121:0/2293134427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51200c7a90 0x7f5120066130 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:54.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.411+0000 7f511f59e700 1 --2- 172.21.15.121:0/2293134427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51200a9f80 0x7f51201432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:54.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.411+0000 7f511f59e700 1 -- 172.21.15.121:0/2293134427 >> 172.21.15.121:0/2293134427 conn(0x7f5120067a60 msgr2=0x7f51200c86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:54.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.411+0000 7f511f59e700 1 -- 172.21.15.121:0/2293134427 shutdown_connections 2023-11-15T02:57:54.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.411+0000 7f511f59e700 1 -- 172.21.15.121:0/2293134427 wait complete. 2023-11-15T02:57:54.419 INFO:teuthology.orchestra.run.smithi121.stderr:updated caps for client.0 2023-11-15T02:57:54.438 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:57:54.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.595+0000 7f0efb079700 1 Processor -- start 2023-11-15T02:57:54.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.595+0000 7f0efb079700 1 -- start start 2023-11-15T02:57:54.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.595+0000 7f0efb079700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0ef40c3280 0x7f0ef40c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:54.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.595+0000 7f0efb079700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef415be30 0x7f0ef415c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:54.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.595+0000 7f0efb079700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef40c1d50 con 0x7f0ef4152140 2023-11-15T02:57:54.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.595+0000 7f0efb079700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ef40c1860 con 0x7f0ef40c3280 2023-11-15T02:57:54.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.595+0000 7f0efb079700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef415cb80 con 0x7f0ef415be30 2023-11-15T02:57:54.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.595+0000 7f0efa077700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0ef40c3280 0x7f0ef40c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:54.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.595+0000 7f0ef9876700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f0ef4152140 0x7f0ef41525f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:44098/0 (socket says 172.21.15.121:44098) 2023-11-15T02:57:54.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.595+0000 7f0ef9876700 1 -- 172.21.15.121:0/100473564 learned_addr learned my addr 172.21.15.121:0/100473564 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:54.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.595+0000 7f0efa077700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0ef40c3280 0x7f0ef40c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:48722/0 (socket says 172.21.15.121:48722) 2023-11-15T02:57:54.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.595+0000 7f0efa878700 1 --2- 172.21.15.121:0/100473564 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef415be30 0x7f0ef415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:54.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.595+0000 7f0efa077700 1 -- 172.21.15.121:0/100473564 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef415be30 msgr2=0x7f0ef415c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:54.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.595+0000 7f0efa077700 1 --2- 172.21.15.121:0/100473564 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef415be30 0x7f0ef415c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:54.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.595+0000 7f0efa077700 1 -- 172.21.15.121:0/100473564 >> v1:172.21.15.190:6789/0 conn(0x7f0ef4152140 legacy=0x7f0ef41525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:54.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.595+0000 7f0efa077700 1 -- 172.21.15.121:0/100473564 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef40a9240 con 0x7f0ef40c3280 2023-11-15T02:57:54.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.599+0000 7f0efa077700 1 --2- 172.21.15.121:0/100473564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef40c3280 0x7f0ef40c3660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f0ef0009760 tx=0x7f0ef0009060).ready entity=mon.1 client_cookie=f0f9254c261e7548 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:54.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.599+0000 7f0ef9075700 1 -- 172.21.15.121:0/100473564 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef000f510 con 0x7f0ef40c3280 2023-11-15T02:57:54.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.599+0000 7f0ef9075700 1 -- 172.21.15.121:0/100473564 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef000f670 con 0x7f0ef40c3280 2023-11-15T02:57:54.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.599+0000 7f0ef9075700 1 -- 172.21.15.121:0/100473564 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef000f9f0 con 0x7f0ef40c3280 2023-11-15T02:57:54.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.599+0000 7f0efb079700 1 -- 172.21.15.121:0/100473564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef40c3280 msgr2=0x7f0ef40c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:54.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.599+0000 7f0efb079700 1 --2- 172.21.15.121:0/100473564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef40c3280 0x7f0ef40c3660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f0ef0009760 tx=0x7f0ef0009060).stop 2023-11-15T02:57:54.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.599+0000 7f0efb079700 1 -- 172.21.15.121:0/100473564 shutdown_connections 2023-11-15T02:57:54.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.599+0000 7f0efb079700 1 --2- 172.21.15.121:0/100473564 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef415be30 0x7f0ef415c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:54.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.599+0000 7f0efb079700 1 --2- 172.21.15.121:0/100473564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef40c3280 0x7f0ef40c3660 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:54.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.599+0000 7f0efb079700 1 -- 172.21.15.121:0/100473564 >> 172.21.15.121:0/100473564 conn(0x7f0ef4062190 msgr2=0x7f0ef4062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:54.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.599+0000 7f0efb079700 1 -- 172.21.15.121:0/100473564 shutdown_connections 2023-11-15T02:57:54.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.599+0000 7f0efb079700 1 -- 172.21.15.121:0/100473564 wait complete. 2023-11-15T02:57:54.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.603+0000 7f0efb079700 1 Processor -- start 2023-11-15T02:57:54.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.603+0000 7f0efb079700 1 -- start start 2023-11-15T02:57:54.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.603+0000 7f0efb079700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ef40c3280 0x7f0ef40c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:54.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.603+0000 7f0efb079700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef4152140 0x7f0ef4154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:54.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.603+0000 7f0efb079700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef415be30 0x7f0ef4154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:54.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.603+0000 7f0efb079700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef4153bf0 con 0x7f0ef40c3280 2023-11-15T02:57:54.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.603+0000 7f0efb079700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef414e860 con 0x7f0ef4152140 2023-11-15T02:57:54.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.603+0000 7f0efb079700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef414e580 con 0x7f0ef415be30 2023-11-15T02:57:54.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.603+0000 7f0efa077700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ef40c3280 0x7f0ef40c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:54.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.603+0000 7f0efa077700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ef40c3280 0x7f0ef40c5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56528/0 (socket says 172.21.15.121:56528) 2023-11-15T02:57:54.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.603+0000 7f0efa077700 1 -- 172.21.15.121:0/4123762772 learned_addr learned my addr 172.21.15.121:0/4123762772 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:54.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.603+0000 7f0ef9876700 1 --2- 172.21.15.121:0/4123762772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef4152140 0x7f0ef4154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:54.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.603+0000 7f0efa878700 1 --2- 172.21.15.121:0/4123762772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef415be30 0x7f0ef4154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:54.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.607+0000 7f0efa077700 1 -- 172.21.15.121:0/4123762772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef415be30 msgr2=0x7f0ef4154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:54.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.607+0000 7f0efa077700 1 --2- 172.21.15.121:0/4123762772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef415be30 0x7f0ef4154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:54.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.607+0000 7f0efa077700 1 -- 172.21.15.121:0/4123762772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef4152140 msgr2=0x7f0ef4154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:54.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.607+0000 7f0efa077700 1 --2- 172.21.15.121:0/4123762772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef4152140 0x7f0ef4154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:54.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.607+0000 7f0efa077700 1 -- 172.21.15.121:0/4123762772 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef00092a0 con 0x7f0ef40c3280 2023-11-15T02:57:54.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.607+0000 7f0efa077700 1 --2- 172.21.15.121:0/4123762772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ef40c3280 0x7f0ef40c5230 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f0ef0009e00 tx=0x7f0ef0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:54.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.607+0000 7f0eea7fc700 1 -- 172.21.15.121:0/4123762772 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef000f7e0 con 0x7f0ef40c3280 2023-11-15T02:57:54.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.607+0000 7f0eea7fc700 1 -- 172.21.15.121:0/4123762772 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef000f940 con 0x7f0ef40c3280 2023-11-15T02:57:54.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.607+0000 7f0efb079700 1 -- 172.21.15.121:0/4123762772 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef4153530 con 0x7f0ef40c3280 2023-11-15T02:57:54.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.607+0000 7f0eea7fc700 1 -- 172.21.15.121:0/4123762772 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef001b660 con 0x7f0ef40c3280 2023-11-15T02:57:54.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.607+0000 7f0efb079700 1 -- 172.21.15.121:0/4123762772 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef4168710 con 0x7f0ef40c3280 2023-11-15T02:57:54.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.607+0000 7f0efb079700 1 -- 172.21.15.121:0/4123762772 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef4153e60 con 0x7f0ef40c3280 2023-11-15T02:57:54.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.611+0000 7f0eea7fc700 1 -- 172.21.15.121:0/4123762772 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0ef001ba10 con 0x7f0ef40c3280 2023-11-15T02:57:54.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.611+0000 7f0eea7fc700 1 --2- 172.21.15.121:0/4123762772 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0ed805a540 0x7f0ed805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:54.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.611+0000 7f0eea7fc700 1 -- 172.21.15.121:0/4123762772 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0ef00761d0 con 0x7f0ef40c3280 2023-11-15T02:57:54.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.611+0000 7f0ef9876700 1 --2- 172.21.15.121:0/4123762772 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0ed805a540 0x7f0ed805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:54.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.611+0000 7f0ef9876700 1 --2- 172.21.15.121:0/4123762772 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0ed805a540 0x7f0ed805ca00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f0ef4152b20 tx=0x7f0ee4009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:54.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.619+0000 7f0eea7fc700 1 -- 172.21.15.121:0/4123762772 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0ef00499f0 con 0x7f0ef40c3280 2023-11-15T02:57:54.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.819+0000 7f0efb079700 1 -- 172.21.15.121:0/4123762772 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ef406bcf0 con 0x7f0ef40c3280 2023-11-15T02:57:54.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.819+0000 7f0eea7fc700 1 -- 172.21.15.121:0/4123762772 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4501 (secure 0 0 0) 0x7f0ef004cfd0 con 0x7f0ef40c3280 2023-11-15T02:57:54.825 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:54.825 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2544127447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2544127447},{"type":"v1","addr":"172.21.15.190:6833","nonce":2544127447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1815834193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1815834193},{"type":"v1","addr":"172.21.15.190:6835","nonce":1815834193}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4950,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3229917854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3229917854},{"type":"v1","addr":"172.21.15.121:6835","nonce":3229917854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:57:44.541243+0000","modified":"2023-11-15T02:57:47.567757+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.121:6837/3788932715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3788932715},{"type":"v1","addr":"172.21.15.121:6837","nonce":3788932715}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-15T02:57:54.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.823+0000 7f0efb079700 1 -- 172.21.15.121:0/4123762772 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0ed805a540 msgr2=0x7f0ed805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:54.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.823+0000 7f0efb079700 1 --2- 172.21.15.121:0/4123762772 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0ed805a540 0x7f0ed805ca00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f0ef4152b20 tx=0x7f0ee4009040).stop 2023-11-15T02:57:54.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.823+0000 7f0efb079700 1 -- 172.21.15.121:0/4123762772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ef40c3280 msgr2=0x7f0ef40c5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:54.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.823+0000 7f0efb079700 1 --2- 172.21.15.121:0/4123762772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ef40c3280 0x7f0ef40c5230 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f0ef0009e00 tx=0x7f0ef0017000).stop 2023-11-15T02:57:54.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.823+0000 7f0efb079700 1 -- 172.21.15.121:0/4123762772 shutdown_connections 2023-11-15T02:57:54.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.823+0000 7f0efb079700 1 --2- 172.21.15.121:0/4123762772 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0ed805a540 0x7f0ed805ca00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:54.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.823+0000 7f0efb079700 1 --2- 172.21.15.121:0/4123762772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef415be30 0x7f0ef4154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:54.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.823+0000 7f0efb079700 1 --2- 172.21.15.121:0/4123762772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef4152140 0x7f0ef4154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:54.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.823+0000 7f0efb079700 1 --2- 172.21.15.121:0/4123762772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ef40c3280 0x7f0ef40c5230 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:54.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.823+0000 7f0efb079700 1 -- 172.21.15.121:0/4123762772 >> 172.21.15.121:0/4123762772 conn(0x7f0ef4062190 msgr2=0x7f0ef414fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:54.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.823+0000 7f0efb079700 1 -- 172.21.15.121:0/4123762772 shutdown_connections 2023-11-15T02:57:54.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:54.823+0000 7f0efb079700 1 -- 172.21.15.121:0/4123762772 wait complete. 2023-11-15T02:57:54.831 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 16 2023-11-15T02:57:55.248 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T02:57:44.541243+0000', 'modified': '2023-11-15T02:57:47.567757+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4953}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4953': {'gid': 4953, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.121:6837/3788932715', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6836', 'nonce': 3788932715}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 3788932715}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T02:57:55.248 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T02:57:55.249 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-15T02:57:55.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.407+0000 7f50c6f79700 1 Processor -- start 2023-11-15T02:57:55.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.407+0000 7f50c6f79700 1 -- start start 2023-11-15T02:57:55.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.411+0000 7f50c6f79700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f50c01511f0 0x7f50c0066570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.411+0000 7f50c6f79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c00a8750 0x7f50c00a8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.411+0000 7f50c6f79700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c0066a60 con 0x7f50c00bcab0 2023-11-15T02:57:55.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.411+0000 7f50c6f79700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c00660e0 con 0x7f50c01511f0 2023-11-15T02:57:55.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.411+0000 7f50c6f79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c00a90b0 con 0x7f50c00a8750 2023-11-15T02:57:55.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.411+0000 7f50c5f77700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f50c01511f0 0x7f50c0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.411+0000 7f50c6778700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c00a8750 0x7f50c00a8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.411+0000 7f50c5f77700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f50c01511f0 0x7f50c0066570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:34832/0 (socket says 172.21.15.121:34832) 2023-11-15T02:57:55.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.411+0000 7f50c5f77700 1 -- 172.21.15.121:0/2556668458 learned_addr learned my addr 172.21.15.121:0/2556668458 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:55.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.411+0000 7f50c5776700 1 --1- 172.21.15.121:0/2556668458 >> v1:172.21.15.190:6789/0 conn(0x7f50c00bcab0 0x7f50c01515d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:52972/0 (socket says 172.21.15.121:52972) 2023-11-15T02:57:55.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.411+0000 7f50c5f77700 1 -- 172.21.15.121:0/2556668458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c00a8750 msgr2=0x7f50c00a8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:55.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.411+0000 7f50c5f77700 1 --2- 172.21.15.121:0/2556668458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c00a8750 0x7f50c00a8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.411+0000 7f50c5f77700 1 -- 172.21.15.121:0/2556668458 >> v1:172.21.15.190:6789/0 conn(0x7f50c00bcab0 legacy=0x7f50c01515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:55.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.411+0000 7f50c5f77700 1 -- 172.21.15.121:0/2556668458 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c00bd390 con 0x7f50c01511f0 2023-11-15T02:57:55.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.411+0000 7f50c5f77700 1 --2- 172.21.15.121:0/2556668458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c01511f0 0x7f50c0066570 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f50bc006fd0 tx=0x7f50bc00b3b0).ready entity=mon.1 client_cookie=53f9fb3931d4b6c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:55.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.411+0000 7f50c4f75700 1 -- 172.21.15.121:0/2556668458 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc00da70 con 0x7f50c01511f0 2023-11-15T02:57:55.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.415+0000 7f50c4f75700 1 -- 172.21.15.121:0/2556668458 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50bc00dbd0 con 0x7f50c01511f0 2023-11-15T02:57:55.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.415+0000 7f50c4f75700 1 -- 172.21.15.121:0/2556668458 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc010470 con 0x7f50c01511f0 2023-11-15T02:57:55.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.415+0000 7f50c6f79700 1 -- 172.21.15.121:0/2556668458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c01511f0 msgr2=0x7f50c0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:55.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.415+0000 7f50c6f79700 1 --2- 172.21.15.121:0/2556668458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c01511f0 0x7f50c0066570 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f50bc006fd0 tx=0x7f50bc00b3b0).stop 2023-11-15T02:57:55.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.415+0000 7f50c6f79700 1 -- 172.21.15.121:0/2556668458 shutdown_connections 2023-11-15T02:57:55.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.415+0000 7f50c6f79700 1 --2- 172.21.15.121:0/2556668458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c00a8750 0x7f50c00a8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.415+0000 7f50c6f79700 1 --2- 172.21.15.121:0/2556668458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c01511f0 0x7f50c0066570 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.415+0000 7f50c6f79700 1 -- 172.21.15.121:0/2556668458 >> 172.21.15.121:0/2556668458 conn(0x7f50c0067a60 msgr2=0x7f50c0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:55.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.415+0000 7f50c6f79700 1 -- 172.21.15.121:0/2556668458 shutdown_connections 2023-11-15T02:57:55.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.415+0000 7f50c6f79700 1 -- 172.21.15.121:0/2556668458 wait complete. 2023-11-15T02:57:55.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.419+0000 7f50c6f79700 1 Processor -- start 2023-11-15T02:57:55.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.419+0000 7f50c6f79700 1 -- start start 2023-11-15T02:57:55.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.419+0000 7f50c6f79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c00a8750 0x7f50c0144370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.419+0000 7f50c6f79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c00bcab0 0x7f50c014d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.419+0000 7f50c6f79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50c01511f0 0x7f50c014de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.419+0000 7f50c6f79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c00bfae0 con 0x7f50c01511f0 2023-11-15T02:57:55.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.419+0000 7f50c6f79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c00ab8c0 con 0x7f50c00a8750 2023-11-15T02:57:55.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.419+0000 7f50c6f79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c0144860 con 0x7f50c00bcab0 2023-11-15T02:57:55.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.419+0000 7f50c6778700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50c01511f0 0x7f50c014de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.419+0000 7f50c6778700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50c01511f0 0x7f50c014de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34466/0 (socket says 172.21.15.121:34466) 2023-11-15T02:57:55.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.419+0000 7f50c6778700 1 -- 172.21.15.121:0/2999853323 learned_addr learned my addr 172.21.15.121:0/2999853323 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:55.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.419+0000 7f50c5776700 1 --2- 172.21.15.121:0/2999853323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c00bcab0 0x7f50c014d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.419+0000 7f50c5f77700 1 --2- 172.21.15.121:0/2999853323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c00a8750 0x7f50c0144370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.419+0000 7f50c6778700 1 -- 172.21.15.121:0/2999853323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c00bcab0 msgr2=0x7f50c014d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:55.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.419+0000 7f50c6778700 1 --2- 172.21.15.121:0/2999853323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c00bcab0 0x7f50c014d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.419+0000 7f50c6778700 1 -- 172.21.15.121:0/2999853323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c00a8750 msgr2=0x7f50c0144370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:55.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.419+0000 7f50c6778700 1 --2- 172.21.15.121:0/2999853323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c00a8750 0x7f50c0144370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.419+0000 7f50c6778700 1 -- 172.21.15.121:0/2999853323 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc00b610 con 0x7f50c01511f0 2023-11-15T02:57:55.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.423+0000 7f50c6778700 1 --2- 172.21.15.121:0/2999853323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50c01511f0 0x7f50c014de50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f50b8009df0 tx=0x7f50b800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:55.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.423+0000 7f50b5ffb700 1 -- 172.21.15.121:0/2999853323 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b800caa0 con 0x7f50c01511f0 2023-11-15T02:57:55.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.423+0000 7f50b5ffb700 1 -- 172.21.15.121:0/2999853323 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b800f040 con 0x7f50c01511f0 2023-11-15T02:57:55.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.423+0000 7f50c6f79700 1 -- 172.21.15.121:0/2999853323 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c00a9ad0 con 0x7f50c01511f0 2023-11-15T02:57:55.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.423+0000 7f50b5ffb700 1 -- 172.21.15.121:0/2999853323 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b800ce10 con 0x7f50c01511f0 2023-11-15T02:57:55.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.423+0000 7f50c6f79700 1 -- 172.21.15.121:0/2999853323 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c00be550 con 0x7f50c01511f0 2023-11-15T02:57:55.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.427+0000 7f50b5ffb700 1 -- 172.21.15.121:0/2999853323 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f50b8013610 con 0x7f50c01511f0 2023-11-15T02:57:55.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.427+0000 7f50b5ffb700 1 --2- 172.21.15.121:0/2999853323 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50a405a510 0x7f50a405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.427+0000 7f50b5ffb700 1 -- 172.21.15.121:0/2999853323 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f50b8074fb0 con 0x7f50c01511f0 2023-11-15T02:57:55.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.427+0000 7f50c5f77700 1 --2- 172.21.15.121:0/2999853323 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50a405a510 0x7f50a405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.427+0000 7f50d256c740 1 Processor -- start 2023-11-15T02:57:55.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50c5f77700 1 --2- 172.21.15.121:0/2999853323 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50a405a510 0x7f50a405c9d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f50bc00bfd0 tx=0x7f50bc00b3b0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:55.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50d256c740 1 -- start start 2023-11-15T02:57:55.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50d256c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x25c1ad0 0x25e8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50d256c740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x25c1eb0 0x260e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50d256c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x260e630 0x260eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50d256c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2542410 con 0x260e630 2023-11-15T02:57:55.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50d256c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x2542550 con 0x25c1ad0 2023-11-15T02:57:55.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50d256c740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2542a00 con 0x25c1eb0 2023-11-15T02:57:55.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50c6778700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x260e630 0x260eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50c5f77700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x25c1ad0 0x25e8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50c6778700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x260e630 0x260eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34480/0 (socket says 172.21.15.121:34480) 2023-11-15T02:57:55.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50c6778700 1 -- 172.21.15.121:0/3567473309 learned_addr learned my addr 172.21.15.121:0/3567473309 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:55.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50c5776700 1 --2- 172.21.15.121:0/3567473309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x25c1eb0 0x260e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50c6778700 1 -- 172.21.15.121:0/3567473309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x25c1eb0 msgr2=0x260e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:55.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50c6778700 1 --2- 172.21.15.121:0/3567473309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x25c1eb0 0x260e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50c6778700 1 -- 172.21.15.121:0/3567473309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x25c1ad0 msgr2=0x25e8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:55.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50c6778700 1 --2- 172.21.15.121:0/3567473309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x25c1ad0 0x25e8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50c6778700 1 -- 172.21.15.121:0/3567473309 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x253b730 con 0x260e630 2023-11-15T02:57:55.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50c5f77700 1 --2- 172.21.15.121:0/3567473309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x25c1ad0 0x25e8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:55.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50c5776700 1 --2- 172.21.15.121:0/3567473309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x25c1eb0 0x260e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:55.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50c6778700 1 --2- 172.21.15.121:0/3567473309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x260e630 0x260eab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f50b804bed0 tx=0x7f50b8009780).ready entity=mon.0 client_cookie=499dd05c893ee259 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:55.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50aaffd700 1 -- 172.21.15.121:0/3567473309 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b804a730 con 0x260e630 2023-11-15T02:57:55.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50aaffd700 1 -- 172.21.15.121:0/3567473309 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b8048aa0 con 0x260e630 2023-11-15T02:57:55.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.431+0000 7f50aaffd700 1 -- 172.21.15.121:0/3567473309 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b804ba90 con 0x260e630 2023-11-15T02:57:55.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.435+0000 7f50d256c740 1 -- 172.21.15.121:0/3567473309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x260e630 msgr2=0x260eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:55.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.435+0000 7f50d256c740 1 --2- 172.21.15.121:0/3567473309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x260e630 0x260eab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f50b804bed0 tx=0x7f50b8009780).stop 2023-11-15T02:57:55.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.435+0000 7f50d256c740 1 -- 172.21.15.121:0/3567473309 shutdown_connections 2023-11-15T02:57:55.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.435+0000 7f50d256c740 1 --2- 172.21.15.121:0/3567473309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x260e630 0x260eab0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.435+0000 7f50d256c740 1 --2- 172.21.15.121:0/3567473309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x25c1eb0 0x260e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.435+0000 7f50d256c740 1 --2- 172.21.15.121:0/3567473309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x25c1ad0 0x25e8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.435+0000 7f50d256c740 1 -- 172.21.15.121:0/3567473309 >> 172.21.15.121:0/3567473309 conn(0x23a0550 msgr2=0x25c07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:55.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.435+0000 7f50d256c740 1 -- 172.21.15.121:0/3567473309 shutdown_connections 2023-11-15T02:57:55.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.435+0000 7f50d256c740 1 -- 172.21.15.121:0/3567473309 wait complete. 2023-11-15T02:57:55.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.435+0000 7f50d256c740 1 -- start start 2023-11-15T02:57:55.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.435+0000 7f50d256c740 1 Processor -- start 2023-11-15T02:57:55.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.435+0000 7f50d256c740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x25c1ad0 0x25e74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.435+0000 7f50d256c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 0x25ead40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.435+0000 7f50d256c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x260e630 0x25eb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.435+0000 7f50d256c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x250d650 con 0x25c1eb0 2023-11-15T02:57:55.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.435+0000 7f50d256c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x25ed590 con 0x260e630 2023-11-15T02:57:55.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.435+0000 7f50d256c740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2609fe0 con 0x25c1ad0 2023-11-15T02:57:55.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.435+0000 7f50d256c740 10 client.0 ll_register_callbacks cb 0x257a020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:57:55.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50c5776700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 0x25ead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50c5776700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 0x25ead40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34482/0 (socket says 172.21.15.121:34482) 2023-11-15T02:57:55.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50c5776700 1 -- 172.21.15.121:0/1722948392 learned_addr learned my addr 172.21.15.121:0/1722948392 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:55.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50c6778700 1 --2- 172.21.15.121:0/1722948392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x260e630 0x25eb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50c5f77700 1 --2- 172.21.15.121:0/1722948392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x25c1ad0 0x25e74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50c5776700 1 -- 172.21.15.121:0/1722948392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x25c1ad0 msgr2=0x25e74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:55.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50c5776700 1 --2- 172.21.15.121:0/1722948392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x25c1ad0 0x25e74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50c5776700 1 -- 172.21.15.121:0/1722948392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x260e630 msgr2=0x25eb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:55.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50c5776700 1 --2- 172.21.15.121:0/1722948392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x260e630 0x25eb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50c5776700 1 -- 172.21.15.121:0/1722948392 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c00be550 con 0x25c1eb0 2023-11-15T02:57:55.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50c6778700 1 --2- 172.21.15.121:0/1722948392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x260e630 0x25eb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:57:55.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50c5776700 1 --2- 172.21.15.121:0/1722948392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 0x25ead40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f50b00076b0 tx=0x7f50b0009040).ready entity=mon.0 client_cookie=6610310549c56b61 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:55.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f509ffff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:57:55.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f509ffff700 1 -- 172.21.15.121:0/1722948392 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b00071f0 con 0x25c1eb0 2023-11-15T02:57:55.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f509ffff700 1 -- 172.21.15.121:0/1722948392 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f50b0011d30 con 0x25c1eb0 2023-11-15T02:57:55.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50d256c740 1 -- 172.21.15.121:0/1722948392 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x25381a0 con 0x25c1eb0 2023-11-15T02:57:55.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f509ffff700 1 -- 172.21.15.121:0/1722948392 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50b00143d0 con 0x25c1eb0 2023-11-15T02:57:55.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50d256c740 10 client.5142 fetch_fsmap learned FSMap version 16 2023-11-15T02:57:55.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50d256c740 1 -- 172.21.15.121:0/1722948392 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x2536960 con 0x25c1eb0 2023-11-15T02:57:55.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f509ffff700 1 -- 172.21.15.121:0/1722948392 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x25381a0 con 0x25c1eb0 2023-11-15T02:57:55.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f509ffff700 1 -- 172.21.15.121:0/1722948392 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f50b0014530 con 0x25c1eb0 2023-11-15T02:57:55.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50d256c740 10 client.5142 fetch_fsmap finished waiting for FSMap version 16 2023-11-15T02:57:55.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50d256c740 10 client.5142 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-15T02:57:55.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50d256c740 20 client.5142 populate_metadata read hostname 'smithi121' 2023-11-15T02:57:55.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50d256c740 1 --2- 172.21.15.121:0/1722948392 >> [v2:172.21.15.121:6836/3788932715,v1:172.21.15.121:6837/3788932715] conn(0x25c2b40 0x260c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50d256c740 4 client.5142 mds_command: new command op to 4953 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T02:57:55.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50d256c740 1 -- 172.21.15.121:0/1722948392 --> [v2:172.21.15.121:6836/3788932715,v1:172.21.15.121:6837/3788932715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x252e6f0 con 0x25c2b40 2023-11-15T02:57:55.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.439+0000 7f50c5f77700 1 --2- 172.21.15.121:0/1722948392 >> [v2:172.21.15.121:6836/3788932715,v1:172.21.15.121:6837/3788932715] conn(0x25c2b40 0x260c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.443+0000 7f509ffff700 10 client.5142 ms_handle_connect on v2:172.21.15.121:6836/3788932715 2023-11-15T02:57:55.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.443+0000 7f50c5f77700 1 --2- 172.21.15.121:0/1722948392 >> [v2:172.21.15.121:6836/3788932715,v1:172.21.15.121:6837/3788932715] conn(0x25c2b40 0x260c660 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=21659021201d3218 server_cookie=f71114f728ae859c in_seq=0 out_seq=0 2023-11-15T02:57:55.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.443+0000 7f509ffff700 1 -- 172.21.15.121:0/1722948392 <== mds.0 v2:172.21.15.121:6836/3788932715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x252e6f0 con 0x25c2b40 2023-11-15T02:57:55.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.443+0000 7f509ffff700 10 client.5142 handle_command_reply: tid=0 2023-11-15T02:57:55.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.443+0000 7f50d256c740 1 client.5142 shutdown 2023-11-15T02:57:55.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.443+0000 7f50d256c740 1 -- 172.21.15.121:0/1722948392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 msgr2=0x25ead40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:55.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.443+0000 7f50d256c740 1 --2- 172.21.15.121:0/1722948392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 0x25ead40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f50b00076b0 tx=0x7f50b0009040).stop 2023-11-15T02:57:55.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.443+0000 7f50d256c740 1 -- 172.21.15.121:0/1722948392 shutdown_connections 2023-11-15T02:57:55.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.443+0000 7f50d256c740 1 --2- 172.21.15.121:0/1722948392 >> [v2:172.21.15.121:6836/3788932715,v1:172.21.15.121:6837/3788932715] conn(0x25c2b40 0x260c660 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.443+0000 7f50d256c740 1 --2- 172.21.15.121:0/1722948392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x260e630 0x25eb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.443+0000 7f509ffff700 0 client.5142 ms_handle_reset on v2:172.21.15.121:6836/3788932715 2023-11-15T02:57:55.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.443+0000 7f50d256c740 1 --2- 172.21.15.121:0/1722948392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 0x25ead40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.443+0000 7f50d256c740 1 --2- 172.21.15.121:0/1722948392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x25c1ad0 0x25e74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.443+0000 7f50d256c740 1 -- 172.21.15.121:0/1722948392 >> 172.21.15.121:0/1722948392 conn(0x23a0550 msgr2=0x2609580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:55.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.443+0000 7f50d256c740 1 -- 172.21.15.121:0/1722948392 shutdown_connections 2023-11-15T02:57:55.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.443+0000 7f50d256c740 1 -- 172.21.15.121:0/1722948392 wait complete. 2023-11-15T02:57:55.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.443+0000 7f50d256c740 20 client.5142 trim_cache size 0 max 16384 2023-11-15T02:57:55.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50d256c740 1 Processor -- start 2023-11-15T02:57:55.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50d256c740 1 -- start start 2023-11-15T02:57:55.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50d256c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x25c1ad0 0x25ff150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50d256c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 0x25eae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50d256c740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x260e630 0x2665750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50d256c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2517c60 con 0x25c1eb0 2023-11-15T02:57:55.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50d256c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x25e6eb0 con 0x25c1ad0 2023-11-15T02:57:55.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50d256c740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x261eab0 con 0x260e630 2023-11-15T02:57:55.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50c5776700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 0x25eae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50c5776700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 0x25eae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34494/0 (socket says 172.21.15.121:34494) 2023-11-15T02:57:55.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50c5776700 1 -- 172.21.15.121:0/706398598 learned_addr learned my addr 172.21.15.121:0/706398598 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:55.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50c5776700 1 -- 172.21.15.121:0/706398598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x260e630 msgr2=0x2665750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:55.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50c5776700 1 --2- 172.21.15.121:0/706398598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x260e630 0x2665750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T02:57:55.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50c5776700 1 -- 172.21.15.121:0/706398598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x25c1ad0 msgr2=0x25ff150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:55.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50c5776700 1 --2- 172.21.15.121:0/706398598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x25c1ad0 0x25ff150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T02:57:55.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50c5776700 1 -- 172.21.15.121:0/706398598 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c014cd70 con 0x25c1eb0 2023-11-15T02:57:55.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50c5776700 1 --2- 172.21.15.121:0/706398598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 0x25eae40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f50b0007110 tx=0x7f50b0015000).ready entity=mon.0 client_cookie=3856b30226aac350 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:55.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50ab7fe700 1 -- 172.21.15.121:0/706398598 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0014d20 con 0x25c1eb0 2023-11-15T02:57:55.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50ab7fe700 1 -- 172.21.15.121:0/706398598 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b0014e80 con 0x25c1eb0 2023-11-15T02:57:55.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.463+0000 7f50ab7fe700 1 -- 172.21.15.121:0/706398598 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0000c30 con 0x25c1eb0 2023-11-15T02:57:55.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.467+0000 7f50d256c740 1 -- 172.21.15.121:0/706398598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 msgr2=0x25eae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:55.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.467+0000 7f50d256c740 1 --2- 172.21.15.121:0/706398598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 0x25eae40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f50b0007110 tx=0x7f50b0015000).stop 2023-11-15T02:57:55.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.467+0000 7f50d256c740 1 -- 172.21.15.121:0/706398598 shutdown_connections 2023-11-15T02:57:55.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.467+0000 7f50d256c740 1 --2- 172.21.15.121:0/706398598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x260e630 0x2665750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T02:57:55.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.467+0000 7f50d256c740 1 --2- 172.21.15.121:0/706398598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 0x25eae40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.467+0000 7f50d256c740 1 --2- 172.21.15.121:0/706398598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x25c1ad0 0x25ff150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T02:57:55.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.467+0000 7f50d256c740 1 -- 172.21.15.121:0/706398598 >> 172.21.15.121:0/706398598 conn(0x23a0550 msgr2=0x25be180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:55.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.467+0000 7f50d256c740 1 -- 172.21.15.121:0/706398598 shutdown_connections 2023-11-15T02:57:55.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.467+0000 7f50d256c740 1 -- 172.21.15.121:0/706398598 wait complete. 2023-11-15T02:57:55.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.467+0000 7f50d256c740 1 -- start start 2023-11-15T02:57:55.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.467+0000 7f50d256c740 1 Processor -- start 2023-11-15T02:57:55.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.467+0000 7f50d256c740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x25c1ad0 0x25eb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.467+0000 7f50d256c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 0x265f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.467+0000 7f50d256c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x260e630 0x265f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.467+0000 7f50d256c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x25deba0 con 0x25c1eb0 2023-11-15T02:57:55.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.467+0000 7f50d256c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x2663430 con 0x260e630 2023-11-15T02:57:55.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.467+0000 7f50d256c740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x25de710 con 0x25c1ad0 2023-11-15T02:57:55.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.467+0000 7f50d256c740 10 client.0 ll_register_callbacks cb 0x7f50b0004b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:57:55.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50c5776700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 0x265f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50c5776700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 0x265f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34502/0 (socket says 172.21.15.121:34502) 2023-11-15T02:57:55.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50c5776700 1 -- 172.21.15.121:0/1002692124 learned_addr learned my addr 172.21.15.121:0/1002692124 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:55.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50c5f77700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x25c1ad0 0x25eb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50c6778700 1 --2- 172.21.15.121:0/1002692124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x260e630 0x265f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50c5776700 1 -- 172.21.15.121:0/1002692124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x25c1ad0 msgr2=0x25eb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:55.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50c5776700 1 --2- 172.21.15.121:0/1002692124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x25c1ad0 0x25eb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50c5776700 1 -- 172.21.15.121:0/1002692124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x260e630 msgr2=0x265f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:55.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50c5776700 1 --2- 172.21.15.121:0/1002692124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x260e630 0x265f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50c5776700 1 -- 172.21.15.121:0/1002692124 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2536960 con 0x25c1eb0 2023-11-15T02:57:55.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50c5776700 1 --2- 172.21.15.121:0/1002692124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 0x265f180 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x25ff960 tx=0x7f50b0023040).ready entity=mon.0 client_cookie=b897d13b8bf6b83c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:55.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f509ffff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:57:55.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f509ffff700 1 -- 172.21.15.121:0/1002692124 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b00143d0 con 0x25c1eb0 2023-11-15T02:57:55.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f509ffff700 1 -- 172.21.15.121:0/1002692124 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f50b0014530 con 0x25c1eb0 2023-11-15T02:57:55.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50d256c740 1 -- 172.21.15.121:0/1002692124 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x23f74e0 con 0x25c1eb0 2023-11-15T02:57:55.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f509ffff700 1 -- 172.21.15.121:0/1002692124 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50b00146f0 con 0x25c1eb0 2023-11-15T02:57:55.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50d256c740 10 client.5148 fetch_fsmap learned FSMap version 16 2023-11-15T02:57:55.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50d256c740 1 -- 172.21.15.121:0/1002692124 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x25366b0 con 0x25c1eb0 2023-11-15T02:57:55.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f509ffff700 1 -- 172.21.15.121:0/1002692124 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x23f74e0 con 0x25c1eb0 2023-11-15T02:57:55.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f509ffff700 1 -- 172.21.15.121:0/1002692124 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f50b0027030 con 0x25c1eb0 2023-11-15T02:57:55.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50d256c740 10 client.5148 fetch_fsmap finished waiting for FSMap version 16 2023-11-15T02:57:55.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50d256c740 10 client.5148 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-15T02:57:55.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50d256c740 20 client.5148 populate_metadata read hostname 'smithi121' 2023-11-15T02:57:55.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50d256c740 1 --2- 172.21.15.121:0/1002692124 >> [v2:172.21.15.121:6836/3788932715,v1:172.21.15.121:6837/3788932715] conn(0x26622c0 0x26626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50d256c740 4 client.5148 mds_command: new command op to 4953 tid=0[{"prefix": "status"}] 2023-11-15T02:57:55.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50d256c740 1 -- 172.21.15.121:0/1002692124 --> [v2:172.21.15.121:6836/3788932715,v1:172.21.15.121:6837/3788932715] -- command(tid 0: {"prefix": "status"}) v1 -- 0x23ad260 con 0x26622c0 2023-11-15T02:57:55.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50c5f77700 1 --2- 172.21.15.121:0/1002692124 >> [v2:172.21.15.121:6836/3788932715,v1:172.21.15.121:6837/3788932715] conn(0x26622c0 0x26626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f50c5f77700 1 --2- 172.21.15.121:0/1002692124 >> [v2:172.21.15.121:6836/3788932715,v1:172.21.15.121:6837/3788932715] conn(0x26622c0 0x26626a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=a5ec32dd54508244 server_cookie=c31f1e3bd37a8855 in_seq=0 out_seq=0 2023-11-15T02:57:55.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.471+0000 7f509ffff700 10 client.5148 ms_handle_connect on v2:172.21.15.121:6836/3788932715 2023-11-15T02:57:55.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f509ffff700 1 -- 172.21.15.121:0/1002692124 <== mds.0 v2:172.21.15.121:6836/3788932715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x23ad260 con 0x26622c0 2023-11-15T02:57:55.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f509ffff700 10 client.5148 handle_command_reply: tid=0 2023-11-15T02:57:55.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50d256c740 1 client.5148 shutdown 2023-11-15T02:57:55.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50d256c740 1 -- 172.21.15.121:0/1002692124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 msgr2=0x265f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:55.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50d256c740 1 --2- 172.21.15.121:0/1002692124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 0x265f180 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x25ff960 tx=0x7f50b0023040).stop 2023-11-15T02:57:55.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50d256c740 1 -- 172.21.15.121:0/1002692124 shutdown_connections 2023-11-15T02:57:55.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50d256c740 1 --2- 172.21.15.121:0/1002692124 >> [v2:172.21.15.121:6836/3788932715,v1:172.21.15.121:6837/3788932715] conn(0x26622c0 0x26626a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50d256c740 1 --2- 172.21.15.121:0/1002692124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x260e630 0x265f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50d256c740 1 --2- 172.21.15.121:0/1002692124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x25c1eb0 0x265f180 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50d256c740 1 --2- 172.21.15.121:0/1002692124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x25c1ad0 0x25eb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f509ffff700 0 client.5148 ms_handle_reset on v2:172.21.15.121:6836/3788932715 2023-11-15T02:57:55.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50d256c740 1 -- 172.21.15.121:0/1002692124 >> 172.21.15.121:0/1002692124 conn(0x23a0550 msgr2=0x25be180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:55.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50d256c740 1 -- 172.21.15.121:0/1002692124 shutdown_connections 2023-11-15T02:57:55.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50d256c740 1 -- 172.21.15.121:0/1002692124 wait complete. 2023-11-15T02:57:55.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50d256c740 20 client.5148 trim_cache size 0 max 16384 2023-11-15T02:57:55.490 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T02:57:55.490 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T02:57:55.490 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T02:57:55.491 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 4953, 2023-11-15T02:57:55.491 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T02:57:55.491 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T02:57:55.491 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T02:57:55.492 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 10.931268983000001, 2023-11-15T02:57:55.492 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 16, 2023-11-15T02:57:55.492 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 33, 2023-11-15T02:57:55.492 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 33, 2023-11-15T02:57:55.492 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 19.254313493000002 2023-11-15T02:57:55.492 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T02:57:55.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50c6f79700 1 -- 172.21.15.121:0/2999853323 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50a405a510 msgr2=0x7f50a405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:55.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50c6f79700 1 --2- 172.21.15.121:0/2999853323 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50a405a510 0x7f50a405c9d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f50bc00bfd0 tx=0x7f50bc00b3b0).stop 2023-11-15T02:57:55.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50c6f79700 1 -- 172.21.15.121:0/2999853323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50c01511f0 msgr2=0x7f50c014de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:55.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50c6f79700 1 --2- 172.21.15.121:0/2999853323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50c01511f0 0x7f50c014de50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f50b8009df0 tx=0x7f50b800b040).stop 2023-11-15T02:57:55.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50c6f79700 1 -- 172.21.15.121:0/2999853323 shutdown_connections 2023-11-15T02:57:55.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50c6f79700 1 --2- 172.21.15.121:0/2999853323 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50a405a510 0x7f50a405c9d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50c6f79700 1 --2- 172.21.15.121:0/2999853323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50c01511f0 0x7f50c014de50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50c6f79700 1 --2- 172.21.15.121:0/2999853323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c00bcab0 0x7f50c014d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50c6f79700 1 --2- 172.21.15.121:0/2999853323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c00a8750 0x7f50c0144370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50c6f79700 1 -- 172.21.15.121:0/2999853323 >> 172.21.15.121:0/2999853323 conn(0x7f50c0067a60 msgr2=0x7f50c00c4390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:55.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50c6f79700 1 -- 172.21.15.121:0/2999853323 shutdown_connections 2023-11-15T02:57:55.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.475+0000 7f50c6f79700 1 -- 172.21.15.121:0/2999853323 wait complete. 2023-11-15T02:57:55.502 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T02:57:55.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.631+0000 7fbb597db700 1 Processor -- start 2023-11-15T02:57:55.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.635+0000 7fbb597db700 1 -- start start 2023-11-15T02:57:55.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.635+0000 7fbb597db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbb540aa2c0 0x7fbb540c3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.635+0000 7fbb597db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb54155450 0x7fbb540685a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.635+0000 7fbb597db700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb54065620 con 0x7fbb541558d0 2023-11-15T02:57:55.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.635+0000 7fbb597db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb541563f0 con 0x7fbb540aa2c0 2023-11-15T02:57:55.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.635+0000 7fbb597db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb54155e30 con 0x7fbb54155450 2023-11-15T02:57:55.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.635+0000 7fbb4a7fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb54155450 0x7fbb540685a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.635+0000 7fbb4a7fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb54155450 0x7fbb540685a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:35676/0 (socket says 172.21.15.121:35676) 2023-11-15T02:57:55.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.635+0000 7fbb4a7fc700 1 -- 172.21.15.121:0/1789505857 learned_addr learned my addr 172.21.15.121:0/1789505857 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:55.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.635+0000 7fbb52ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbb540aa2c0 0x7fbb540c3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.639+0000 7fbb4a7fc700 1 -- 172.21.15.121:0/1789505857 >> v2:172.21.15.190:3300/0 conn(0x7fbb540aa2c0 msgr2=0x7fbb540c3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:55.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.639+0000 7fbb4a7fc700 1 --2- 172.21.15.121:0/1789505857 >> v2:172.21.15.190:3300/0 conn(0x7fbb540aa2c0 0x7fbb540c3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.639+0000 7fbb4a7fc700 1 -- 172.21.15.121:0/1789505857 >> v1:172.21.15.190:6789/0 conn(0x7fbb541558d0 legacy=0x7fbb54068a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:55.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.639+0000 7fbb4a7fc700 1 -- 172.21.15.121:0/1789505857 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb54156110 con 0x7fbb54155450 2023-11-15T02:57:55.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.639+0000 7fbb4a7fc700 1 --2- 172.21.15.121:0/1789505857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb54155450 0x7fbb540685a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fbb3c009f20 tx=0x7fbb3c009840).ready entity=mon.2 client_cookie=bcd6e8470efe8a10 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:55.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.639+0000 7fbb527fc700 1 -- 172.21.15.121:0/1789505857 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c00e070 con 0x7fbb54155450 2023-11-15T02:57:55.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.639+0000 7fbb527fc700 1 -- 172.21.15.121:0/1789505857 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3c00b9d0 con 0x7fbb54155450 2023-11-15T02:57:55.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.639+0000 7fbb527fc700 1 -- 172.21.15.121:0/1789505857 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c00bd50 con 0x7fbb54155450 2023-11-15T02:57:55.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.639+0000 7fbb597db700 1 -- 172.21.15.121:0/1789505857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb54155450 msgr2=0x7fbb540685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:55.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.639+0000 7fbb597db700 1 --2- 172.21.15.121:0/1789505857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb54155450 0x7fbb540685a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fbb3c009f20 tx=0x7fbb3c009840).stop 2023-11-15T02:57:55.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.639+0000 7fbb597db700 1 -- 172.21.15.121:0/1789505857 shutdown_connections 2023-11-15T02:57:55.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.639+0000 7fbb597db700 1 --2- 172.21.15.121:0/1789505857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb54155450 0x7fbb540685a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.639+0000 7fbb597db700 1 --2- 172.21.15.121:0/1789505857 >> v2:172.21.15.190:3300/0 conn(0x7fbb540aa2c0 0x7fbb540c3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.639+0000 7fbb597db700 1 -- 172.21.15.121:0/1789505857 >> 172.21.15.121:0/1789505857 conn(0x7fbb54069960 msgr2=0x7fbb54069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:55.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.639+0000 7fbb597db700 1 -- 172.21.15.121:0/1789505857 shutdown_connections 2023-11-15T02:57:55.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.639+0000 7fbb597db700 1 -- 172.21.15.121:0/1789505857 wait complete. 2023-11-15T02:57:55.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.643+0000 7fbb597db700 1 Processor -- start 2023-11-15T02:57:55.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.643+0000 7fbb597db700 1 -- start start 2023-11-15T02:57:55.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.643+0000 7fbb597db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb540aa2c0 0x7fbb54135640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.643+0000 7fbb597db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb54155450 0x7fbb540aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.643+0000 7fbb597db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb541558d0 0x7fbb541658b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.643+0000 7fbb597db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb541568a0 con 0x7fbb541558d0 2023-11-15T02:57:55.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.643+0000 7fbb597db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540a8780 con 0x7fbb54155450 2023-11-15T02:57:55.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.643+0000 7fbb597db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540a8a60 con 0x7fbb540aa2c0 2023-11-15T02:57:55.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.643+0000 7fbb53fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb541558d0 0x7fbb541658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.643+0000 7fbb53fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb541558d0 0x7fbb541658b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34504/0 (socket says 172.21.15.121:34504) 2023-11-15T02:57:55.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.643+0000 7fbb53fff700 1 -- 172.21.15.121:0/819411236 learned_addr learned my addr 172.21.15.121:0/819411236 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:55.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.643+0000 7fbb4a7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb54155450 0x7fbb540aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.643+0000 7fbb52ffd700 1 --2- 172.21.15.121:0/819411236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb540aa2c0 0x7fbb54135640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.643+0000 7fbb52ffd700 1 -- 172.21.15.121:0/819411236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb54155450 msgr2=0x7fbb540aadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:55.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.643+0000 7fbb52ffd700 1 --2- 172.21.15.121:0/819411236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb54155450 0x7fbb540aadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.643+0000 7fbb52ffd700 1 -- 172.21.15.121:0/819411236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb541558d0 msgr2=0x7fbb541658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:55.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.643+0000 7fbb52ffd700 1 --2- 172.21.15.121:0/819411236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb541558d0 0x7fbb541658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.643+0000 7fbb52ffd700 1 -- 172.21.15.121:0/819411236 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c009ae0 con 0x7fbb540aa2c0 2023-11-15T02:57:55.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.643+0000 7fbb53fff700 1 --2- 172.21.15.121:0/819411236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb541558d0 0x7fbb541658b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:57:55.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.647+0000 7fbb52ffd700 1 --2- 172.21.15.121:0/819411236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb540aa2c0 0x7fbb54135640 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fbb4400e900 tx=0x7fbb44009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:55.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.647+0000 7fbb4b7fe700 1 -- 172.21.15.121:0/819411236 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44009450 con 0x7fbb540aa2c0 2023-11-15T02:57:55.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.647+0000 7fbb4b7fe700 1 -- 172.21.15.121:0/819411236 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb44012600 con 0x7fbb540aa2c0 2023-11-15T02:57:55.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.647+0000 7fbb597db700 1 -- 172.21.15.121:0/819411236 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb540a8d40 con 0x7fbb540aa2c0 2023-11-15T02:57:55.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.647+0000 7fbb4b7fe700 1 -- 172.21.15.121:0/819411236 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44012980 con 0x7fbb540aa2c0 2023-11-15T02:57:55.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.647+0000 7fbb597db700 1 -- 172.21.15.121:0/819411236 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb54135b30 con 0x7fbb540aa2c0 2023-11-15T02:57:55.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.647+0000 7fbb597db700 1 -- 172.21.15.121:0/819411236 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb5405eeb0 con 0x7fbb540aa2c0 2023-11-15T02:57:55.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.651+0000 7fbb4b7fe700 1 -- 172.21.15.121:0/819411236 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fbb44012d30 con 0x7fbb540aa2c0 2023-11-15T02:57:55.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.651+0000 7fbb4b7fe700 1 --2- 172.21.15.121:0/819411236 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbb3405a530 0x7fbb3405c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:55.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.651+0000 7fbb4b7fe700 1 -- 172.21.15.121:0/819411236 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fbb44074b10 con 0x7fbb540aa2c0 2023-11-15T02:57:55.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.651+0000 7fbb4a7fc700 1 --2- 172.21.15.121:0/819411236 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbb3405a530 0x7fbb3405c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:55.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.651+0000 7fbb4a7fc700 1 --2- 172.21.15.121:0/819411236 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbb3405a530 0x7fbb3405c9f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fbb3c009540 tx=0x7fbb3c009840).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:55.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.663+0000 7fbb4b7fe700 1 -- 172.21.15.121:0/819411236 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbb44048230 con 0x7fbb540aa2c0 2023-11-15T02:57:55.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.859+0000 7fbb597db700 1 -- 172.21.15.121:0/819411236 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbb540652b0 con 0x7fbb540aa2c0 2023-11-15T02:57:55.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.859+0000 7fbb4b7fe700 1 -- 172.21.15.121:0/819411236 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbb4404b850 con 0x7fbb540aa2c0 2023-11-15T02:57:55.864 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T02:57:55.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.859+0000 7fbb597db700 1 -- 172.21.15.121:0/819411236 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbb3405a530 msgr2=0x7fbb3405c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:55.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.859+0000 7fbb597db700 1 --2- 172.21.15.121:0/819411236 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbb3405a530 0x7fbb3405c9f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fbb3c009540 tx=0x7fbb3c009840).stop 2023-11-15T02:57:55.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.863+0000 7fbb597db700 1 -- 172.21.15.121:0/819411236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb540aa2c0 msgr2=0x7fbb54135640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:55.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.863+0000 7fbb597db700 1 --2- 172.21.15.121:0/819411236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb540aa2c0 0x7fbb54135640 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fbb4400e900 tx=0x7fbb44009060).stop 2023-11-15T02:57:55.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.863+0000 7fbb597db700 1 -- 172.21.15.121:0/819411236 shutdown_connections 2023-11-15T02:57:55.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.863+0000 7fbb597db700 1 --2- 172.21.15.121:0/819411236 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbb3405a530 0x7fbb3405c9f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.863+0000 7fbb597db700 1 --2- 172.21.15.121:0/819411236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb541558d0 0x7fbb541658b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.863+0000 7fbb597db700 1 --2- 172.21.15.121:0/819411236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb54155450 0x7fbb540aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.863+0000 7fbb597db700 1 --2- 172.21.15.121:0/819411236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb540aa2c0 0x7fbb54135640 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:55.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.863+0000 7fbb597db700 1 -- 172.21.15.121:0/819411236 >> 172.21.15.121:0/819411236 conn(0x7fbb54069960 msgr2=0x7fbb540bd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:55.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.863+0000 7fbb597db700 1 -- 172.21.15.121:0/819411236 shutdown_connections 2023-11-15T02:57:55.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:55.863+0000 7fbb597db700 1 -- 172.21.15.121:0/819411236 wait complete. 2023-11-15T02:57:55.882 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T02:57:55.883 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T02:57:55.883 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T02:57:55.883 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T02:57:55.884 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T02:57:55.884 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T02:57:55.884 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T02:57:55.884 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T02:57:55.885 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T02:57:55.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:55.886 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T02:57:55.906 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T02:57:55.907 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T02:57:55.907 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T02:57:55.907 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:55.907 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T02:57:55.907 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:55.908 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:57:55.908 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:55.908 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T02:57:55.908 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:57:55.909 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:55.909 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T02:57:55.909 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:57:55.909 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:55.909 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T02:57:55.910 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:55.910 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T02:57:55.910 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42364sec preferred_lft 42364sec 2023-11-15T02:57:55.910 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T02:57:55.910 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:55.911 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T02:57:55.911 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:55.911 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T02:57:55.911 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:55.911 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T02:57:55.912 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T02:57:55.912 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T02:57:55.912 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:55.912 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T02:57:55.912 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:55.913 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T02:57:55.914 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T02:57:55.914 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T02:57:55.914 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:55.914 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T02:57:55.914 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T02:57:55.915 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T02:57:55.915 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:57:55.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:55.916 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T02:57:55.993 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T02:57:55.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:55.996 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T02:57:56.059 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T02:57:56.059 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T02:57:56.059 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T02:57:56.060 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T02:57:56.060 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T02:57:56.060 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T02:57:56.060 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T02:57:56.060 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T02:57:56.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:56.062 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T02:57:56.062 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T02:57:56.063 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T02:57:56.063 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T02:57:56.064 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T02:57:56.064 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T02:57:56.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:56.199 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T02:57:56.215 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T02:57:56.216 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T02:57:56.217 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T02:57:56.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:56.217 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T02:57:56.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:56.291 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T02:57:56.353 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T02:57:56.353 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:57:56.475 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:57:56.476 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:57:56.499 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:57:56.499 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:57:56.505 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T02:57:56.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:56.506 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T02:57:56.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:57:56.540 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2023-11-15T02:57:56.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:57:56.622 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2023-11-15T02:57:56.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:57:56.690 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2023-11-15T02:57:56.753 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-15T02:57:56.753 DEBUG:teuthology.orchestra.run.smithi121:> import os 2023-11-15T02:57:56.754 DEBUG:teuthology.orchestra.run.smithi121:> import stat 2023-11-15T02:57:56.754 DEBUG:teuthology.orchestra.run.smithi121:> 2023-11-15T02:57:56.755 DEBUG:teuthology.orchestra.run.smithi121:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2023-11-15T02:57:56.755 DEBUG:teuthology.orchestra.run.smithi121:> ' 2023-11-15T02:57:56.892 INFO:teuthology.orchestra.run.smithi121.stdout:1099511627776 2023-11-15T02:57:57.084 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T02:57:57.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:57:57.085 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-15T02:57:57.215 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T02:57:57.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:57:57.216 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T02:57:57.234 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:57:57.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.431+0000 7f367d5db700 1 Processor -- start 2023-11-15T02:57:57.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.431+0000 7f367d5db700 1 -- start start 2023-11-15T02:57:57.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.435+0000 7f367d5db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36781539a0 0x7f3678153d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:57.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.435+0000 7f367d5db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f36780be6a0 0x7f36780c2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:57.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.435+0000 7f367d5db700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36780c2e40 con 0x7f36780c23f0 2023-11-15T02:57:57.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.435+0000 7f367d5db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36780c3120 con 0x7f36780be6a0 2023-11-15T02:57:57.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.435+0000 7f367d5db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36780c3400 con 0x7f36781539a0 2023-11-15T02:57:57.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.435+0000 7f3677fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f36780be6a0 0x7f36780c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:57.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.435+0000 7f36767fc700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f36780c23f0 0x7f36780bdf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:52986/0 (socket says 172.21.15.121:52986) 2023-11-15T02:57:57.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.435+0000 7f36767fc700 1 -- 172.21.15.121:0/3518019006 learned_addr learned my addr 172.21.15.121:0/3518019006 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:57.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.435+0000 7f3676ffd700 1 --2- 172.21.15.121:0/3518019006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36781539a0 0x7f3678153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:57.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.435+0000 7f3677fff700 1 -- 172.21.15.121:0/3518019006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36781539a0 msgr2=0x7f3678153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:57.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.435+0000 7f3677fff700 1 --2- 172.21.15.121:0/3518019006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36781539a0 0x7f3678153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:57.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.435+0000 7f3677fff700 1 -- 172.21.15.121:0/3518019006 >> v1:172.21.15.190:6789/0 conn(0x7f36780c23f0 legacy=0x7f36780bdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:57.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.435+0000 7f3675ffb700 1 -- 172.21.15.121:0/3518019006 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1277148742 0 0) 0x7f36780c2e40 con 0x7f36780c23f0 2023-11-15T02:57:57.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.435+0000 7f3677fff700 1 -- 172.21.15.121:0/3518019006 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36780c36e0 con 0x7f36780be6a0 2023-11-15T02:57:57.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.435+0000 7f3677fff700 1 --2- 172.21.15.121:0/3518019006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36780be6a0 0x7f36780c2850 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f3670009e60 tx=0x7f3670009780).ready entity=mon.0 client_cookie=4a1fd50a37390585 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:57.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.439+0000 7f3675ffb700 1 -- 172.21.15.121:0/3518019006 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367000e070 con 0x7f36780be6a0 2023-11-15T02:57:57.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.439+0000 7f3675ffb700 1 -- 172.21.15.121:0/3518019006 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367000bb60 con 0x7f36780be6a0 2023-11-15T02:57:57.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.439+0000 7f3675ffb700 1 -- 172.21.15.121:0/3518019006 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670012410 con 0x7f36780be6a0 2023-11-15T02:57:57.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.439+0000 7f367d5db700 1 -- 172.21.15.121:0/3518019006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36780be6a0 msgr2=0x7f36780c2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:57.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.439+0000 7f367d5db700 1 --2- 172.21.15.121:0/3518019006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36780be6a0 0x7f36780c2850 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f3670009e60 tx=0x7f3670009780).stop 2023-11-15T02:57:57.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.439+0000 7f367d5db700 1 -- 172.21.15.121:0/3518019006 shutdown_connections 2023-11-15T02:57:57.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.439+0000 7f367d5db700 1 --2- 172.21.15.121:0/3518019006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36780be6a0 0x7f36780c2850 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:57.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.439+0000 7f367d5db700 1 --2- 172.21.15.121:0/3518019006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36781539a0 0x7f3678153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:57.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.439+0000 7f367d5db700 1 -- 172.21.15.121:0/3518019006 >> 172.21.15.121:0/3518019006 conn(0x7f3678069ac0 msgr2=0x7f3678090a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:57.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.439+0000 7f367d5db700 1 -- 172.21.15.121:0/3518019006 shutdown_connections 2023-11-15T02:57:57.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.439+0000 7f367d5db700 1 -- 172.21.15.121:0/3518019006 wait complete. 2023-11-15T02:57:57.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.439+0000 7f367d5db700 1 Processor -- start 2023-11-15T02:57:57.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.439+0000 7f367d5db700 1 -- start start 2023-11-15T02:57:57.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f367d5db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36780be6a0 0x7f3678155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:57.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f367d5db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36780c23f0 0x7f3678167e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:57.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f367d5db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36781539a0 0x7f3678152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:57.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f367d5db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3678156a10 con 0x7f36780be6a0 2023-11-15T02:57:57.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f367d5db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36780ad710 con 0x7f36780c23f0 2023-11-15T02:57:57.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f367d5db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3678156ea0 con 0x7f36781539a0 2023-11-15T02:57:57.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f3676ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36780be6a0 0x7f3678155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:57.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f3676ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36780be6a0 0x7f3678155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34528/0 (socket says 172.21.15.121:34528) 2023-11-15T02:57:57.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f3676ffd700 1 -- 172.21.15.121:0/278498489 learned_addr learned my addr 172.21.15.121:0/278498489 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:57.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f36767fc700 1 --2- 172.21.15.121:0/278498489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36780c23f0 0x7f3678167e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:57.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f3677fff700 1 --2- 172.21.15.121:0/278498489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36781539a0 0x7f3678152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:57.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f3676ffd700 1 -- 172.21.15.121:0/278498489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36781539a0 msgr2=0x7f3678152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:57.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f3676ffd700 1 --2- 172.21.15.121:0/278498489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36781539a0 0x7f3678152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:57.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f3676ffd700 1 -- 172.21.15.121:0/278498489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36780c23f0 msgr2=0x7f3678167e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:57.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f3676ffd700 1 --2- 172.21.15.121:0/278498489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36780c23f0 0x7f3678167e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:57.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f3676ffd700 1 -- 172.21.15.121:0/278498489 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3670009a20 con 0x7f36780be6a0 2023-11-15T02:57:57.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f3676ffd700 1 --2- 172.21.15.121:0/278498489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36780be6a0 0x7f3678155800 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f366800e960 tx=0x7f3668009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:57.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f365effd700 1 -- 172.21.15.121:0/278498489 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366800c850 con 0x7f36780be6a0 2023-11-15T02:57:57.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f365effd700 1 -- 172.21.15.121:0/278498489 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366800c9b0 con 0x7f36780be6a0 2023-11-15T02:57:57.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f367d5db700 1 -- 172.21.15.121:0/278498489 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3678168360 con 0x7f36780be6a0 2023-11-15T02:57:57.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f365effd700 1 -- 172.21.15.121:0/278498489 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366800cd30 con 0x7f36780be6a0 2023-11-15T02:57:57.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.443+0000 7f367d5db700 1 -- 172.21.15.121:0/278498489 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3678155e00 con 0x7f36780be6a0 2023-11-15T02:57:57.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.447+0000 7f367d5db700 1 -- 172.21.15.121:0/278498489 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3678047060 con 0x7f36780be6a0 2023-11-15T02:57:57.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.447+0000 7f365effd700 1 -- 172.21.15.121:0/278498489 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3668012600 con 0x7f36780be6a0 2023-11-15T02:57:57.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.447+0000 7f365effd700 1 --2- 172.21.15.121:0/278498489 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f366405a540 0x7f366405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:57.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.451+0000 7f365effd700 1 -- 172.21.15.121:0/278498489 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3668074dd0 con 0x7f36780be6a0 2023-11-15T02:57:57.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.451+0000 7f36767fc700 1 --2- 172.21.15.121:0/278498489 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f366405a540 0x7f366405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:57.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.451+0000 7f36767fc700 1 --2- 172.21.15.121:0/278498489 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f366405a540 0x7f366405ca00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f36780aa050 tx=0x7f366c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:57.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.455+0000 7f365effd700 1 -- 172.21.15.121:0/278498489 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f36680484f0 con 0x7f36780be6a0 2023-11-15T02:57:57.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.651+0000 7f367d5db700 1 -- 172.21.15.121:0/278498489 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3678066eb0 con 0x7f36780be6a0 2023-11-15T02:57:57.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.651+0000 7f365effd700 1 -- 172.21.15.121:0/278498489 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4501 (secure 0 0 0) 0x7f366804bb10 con 0x7f36780be6a0 2023-11-15T02:57:57.658 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:57:57.659 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2544127447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2544127447},{"type":"v1","addr":"172.21.15.190:6833","nonce":2544127447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1815834193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1815834193},{"type":"v1","addr":"172.21.15.190:6835","nonce":1815834193}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4950,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3229917854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3229917854},{"type":"v1","addr":"172.21.15.121:6835","nonce":3229917854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:57:44.541243+0000","modified":"2023-11-15T02:57:47.567757+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.121:6837/3788932715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3788932715},{"type":"v1","addr":"172.21.15.121:6837","nonce":3788932715}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-15T02:57:57.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.655+0000 7f367d5db700 1 -- 172.21.15.121:0/278498489 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f366405a540 msgr2=0x7f366405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:57.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.655+0000 7f367d5db700 1 --2- 172.21.15.121:0/278498489 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f366405a540 0x7f366405ca00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f36780aa050 tx=0x7f366c009040).stop 2023-11-15T02:57:57.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.655+0000 7f367d5db700 1 -- 172.21.15.121:0/278498489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36780be6a0 msgr2=0x7f3678155800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:57.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.655+0000 7f367d5db700 1 --2- 172.21.15.121:0/278498489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36780be6a0 0x7f3678155800 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f366800e960 tx=0x7f3668009060).stop 2023-11-15T02:57:57.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.655+0000 7f367d5db700 1 -- 172.21.15.121:0/278498489 shutdown_connections 2023-11-15T02:57:57.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.655+0000 7f367d5db700 1 --2- 172.21.15.121:0/278498489 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f366405a540 0x7f366405ca00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:57.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.655+0000 7f367d5db700 1 --2- 172.21.15.121:0/278498489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36781539a0 0x7f3678152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:57.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.655+0000 7f367d5db700 1 --2- 172.21.15.121:0/278498489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36780c23f0 0x7f3678167e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:57.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.655+0000 7f367d5db700 1 --2- 172.21.15.121:0/278498489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36780be6a0 0x7f3678155800 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:57.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.655+0000 7f367d5db700 1 -- 172.21.15.121:0/278498489 >> 172.21.15.121:0/278498489 conn(0x7f3678069ac0 msgr2=0x7f36780c4420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:57.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.655+0000 7f367d5db700 1 -- 172.21.15.121:0/278498489 shutdown_connections 2023-11-15T02:57:57.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:57.655+0000 7f367d5db700 1 -- 172.21.15.121:0/278498489 wait complete. 2023-11-15T02:57:57.663 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 16 2023-11-15T02:57:57.681 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-11-15T02:57:57.866 INFO:teuthology.orchestra.run.smithi121.stdout:{"message":"","return_code":0} 2023-11-15T02:57:57.866 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-15T02:57:57.867 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T02:57:58.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.015+0000 7fcda5947700 1 Processor -- start 2023-11-15T02:57:58.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.019+0000 7fcda5947700 1 -- start start 2023-11-15T02:57:58.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.019+0000 7fcda5947700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcda0153d70 0x7fcda00bb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:58.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.019+0000 7fcda5947700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcda00c2ec0 0x7fcda00c3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:58.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.019+0000 7fcda5947700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda00a8b20 con 0x7fcda0153990 2023-11-15T02:57:58.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.019+0000 7fcda5947700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda00c1760 con 0x7fcda0153d70 2023-11-15T02:57:58.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.019+0000 7fcda5947700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda00a8d50 con 0x7fcda00c2ec0 2023-11-15T02:57:58.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.019+0000 7fcd9ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcda0153d70 0x7fcda00bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:58.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcda4945700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fcda0153990 0x7fcda01581e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:54510/0 (socket says 172.21.15.121:54510) 2023-11-15T02:57:58.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcd9ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcda0153d70 0x7fcda00bb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34532/0 (socket says 172.21.15.121:34532) 2023-11-15T02:57:58.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcda4945700 1 -- 172.21.15.121:0/2462281866 learned_addr learned my addr 172.21.15.121:0/2462281866 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:57:58.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcda5146700 1 --2- 172.21.15.121:0/2462281866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcda00c2ec0 0x7fcda00c3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:58.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcd9f7fe700 1 -- 172.21.15.121:0/2462281866 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473903233 0 0) 0x7fcda00a8b20 con 0x7fcda0153990 2023-11-15T02:57:58.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcd9f7fe700 1 -- 172.21.15.121:0/2462281866 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd900033f0 con 0x7fcda0153990 2023-11-15T02:57:58.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcd9ffff700 1 -- 172.21.15.121:0/2462281866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcda00c2ec0 msgr2=0x7fcda00c3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:58.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcd9ffff700 1 --2- 172.21.15.121:0/2462281866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcda00c2ec0 0x7fcda00c3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:58.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcd9ffff700 1 -- 172.21.15.121:0/2462281866 >> v1:172.21.15.121:6789/0 conn(0x7fcda0153990 legacy=0x7fcda01581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:58.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcd9ffff700 1 -- 172.21.15.121:0/2462281866 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda00a9030 con 0x7fcda0153d70 2023-11-15T02:57:58.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcd9ffff700 1 --2- 172.21.15.121:0/2462281866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcda0153d70 0x7fcda00bb910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fcd8c009750 tx=0x7fcd8c009060).ready entity=mon.0 client_cookie=20109b684453c840 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:58.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcd9f7fe700 1 -- 172.21.15.121:0/2462281866 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c00d4a0 con 0x7fcda0153d70 2023-11-15T02:57:58.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcd9f7fe700 1 -- 172.21.15.121:0/2462281866 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c00d600 con 0x7fcda0153d70 2023-11-15T02:57:58.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcd9f7fe700 1 -- 172.21.15.121:0/2462281866 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c00d980 con 0x7fcda0153d70 2023-11-15T02:57:58.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcda5947700 1 -- 172.21.15.121:0/2462281866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcda0153d70 msgr2=0x7fcda00bb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:58.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcda5947700 1 --2- 172.21.15.121:0/2462281866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcda0153d70 0x7fcda00bb910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fcd8c009750 tx=0x7fcd8c009060).stop 2023-11-15T02:57:58.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcda5947700 1 -- 172.21.15.121:0/2462281866 shutdown_connections 2023-11-15T02:57:58.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcda5947700 1 --2- 172.21.15.121:0/2462281866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcda00c2ec0 0x7fcda00c3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:58.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcda5947700 1 --2- 172.21.15.121:0/2462281866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcda0153d70 0x7fcda00bb910 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:58.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcda5947700 1 -- 172.21.15.121:0/2462281866 >> 172.21.15.121:0/2462281866 conn(0x7fcda0067a60 msgr2=0x7fcda0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:58.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcda5947700 1 -- 172.21.15.121:0/2462281866 shutdown_connections 2023-11-15T02:57:58.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.023+0000 7fcda5947700 1 -- 172.21.15.121:0/2462281866 wait complete. 2023-11-15T02:57:58.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.027+0000 7fcda5947700 1 Processor -- start 2023-11-15T02:57:58.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.027+0000 7fcda5947700 1 -- start start 2023-11-15T02:57:58.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.027+0000 7fcda5947700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcda00c2ec0 0x7fcda01503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:58.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.027+0000 7fcda5947700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcda0153990 0x7fcda00bc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:58.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.027+0000 7fcda5947700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcda0153d70 0x7fcda00bc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:58.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.027+0000 7fcda5947700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda0154570 con 0x7fcda0153990 2023-11-15T02:57:58.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.027+0000 7fcda5947700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda0156df0 con 0x7fcda0153d70 2023-11-15T02:57:58.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.027+0000 7fcda5947700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda00bca70 con 0x7fcda00c2ec0 2023-11-15T02:57:58.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.031+0000 7fcd9ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcda0153990 0x7fcda00bc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:58.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.031+0000 7fcd9ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcda0153990 0x7fcda00bc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34536/0 (socket says 172.21.15.121:34536) 2023-11-15T02:57:58.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.031+0000 7fcd9ffff700 1 -- 172.21.15.121:0/820895143 learned_addr learned my addr 172.21.15.121:0/820895143 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:58.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.031+0000 7fcda4945700 1 --2- 172.21.15.121:0/820895143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcda00c2ec0 0x7fcda01503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:58.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.031+0000 7fcda5146700 1 --2- 172.21.15.121:0/820895143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcda0153d70 0x7fcda00bc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:58.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.031+0000 7fcd9ffff700 1 -- 172.21.15.121:0/820895143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcda00c2ec0 msgr2=0x7fcda01503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:58.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.031+0000 7fcd9ffff700 1 --2- 172.21.15.121:0/820895143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcda00c2ec0 0x7fcda01503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:58.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.031+0000 7fcd9ffff700 1 -- 172.21.15.121:0/820895143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcda0153d70 msgr2=0x7fcda00bc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:58.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.031+0000 7fcd9ffff700 1 --2- 172.21.15.121:0/820895143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcda0153d70 0x7fcda00bc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:58.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.031+0000 7fcd9ffff700 1 -- 172.21.15.121:0/820895143 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c009240 con 0x7fcda0153990 2023-11-15T02:57:58.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.031+0000 7fcda5146700 1 --2- 172.21.15.121:0/820895143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcda0153d70 0x7fcda00bc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:57:58.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.031+0000 7fcd9ffff700 1 --2- 172.21.15.121:0/820895143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcda0153990 0x7fcda00bc090 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fcda0064eb0 tx=0x7fcd8c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:58.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.031+0000 7fcd9cff9700 1 -- 172.21.15.121:0/820895143 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c00d770 con 0x7fcda0153990 2023-11-15T02:57:58.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.031+0000 7fcd9cff9700 1 -- 172.21.15.121:0/820895143 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c00d8d0 con 0x7fcda0153990 2023-11-15T02:57:58.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.031+0000 7fcda5947700 1 -- 172.21.15.121:0/820895143 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda014b7e0 con 0x7fcda0153990 2023-11-15T02:57:58.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.031+0000 7fcd9cff9700 1 -- 172.21.15.121:0/820895143 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c01c670 con 0x7fcda0153990 2023-11-15T02:57:58.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.031+0000 7fcda5947700 1 -- 172.21.15.121:0/820895143 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda014b500 con 0x7fcda0153990 2023-11-15T02:57:58.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.031+0000 7fcda5947700 1 -- 172.21.15.121:0/820895143 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda006bcf0 con 0x7fcda0153990 2023-11-15T02:57:58.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.035+0000 7fcd9cff9700 1 -- 172.21.15.121:0/820895143 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fcd8c01ca60 con 0x7fcda0153990 2023-11-15T02:57:58.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.035+0000 7fcd9cff9700 1 --2- 172.21.15.121:0/820895143 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd8405a560 0x7fcd8405ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:58.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.035+0000 7fcd9cff9700 1 -- 172.21.15.121:0/820895143 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fcd8c075d50 con 0x7fcda0153990 2023-11-15T02:57:58.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.035+0000 7fcda4945700 1 --2- 172.21.15.121:0/820895143 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd8405a560 0x7fcd8405ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:58.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.035+0000 7fcda4945700 1 --2- 172.21.15.121:0/820895143 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd8405a560 0x7fcd8405ca20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fcd94005750 tx=0x7fcd94009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:58.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.039+0000 7fcd9cff9700 1 -- 172.21.15.121:0/820895143 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcd8c049570 con 0x7fcda0153990 2023-11-15T02:57:58.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:58.235+0000 7fcda5947700 1 -- 172.21.15.121:0/820895143 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcda00c19b0 con 0x7fcda0153990 2023-11-15T02:57:59.014 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T02:57:59.007+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T02:57:59.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.019+0000 7fcd9cff9700 1 -- 172.21.15.121:0/820895143 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcd8c01a310 con 0x7fcda0153990 2023-11-15T02:57:59.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.027+0000 7fcda5947700 1 -- 172.21.15.121:0/820895143 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd8405a560 msgr2=0x7fcd8405ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:59.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.027+0000 7fcda5947700 1 --2- 172.21.15.121:0/820895143 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd8405a560 0x7fcd8405ca20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fcd94005750 tx=0x7fcd94009040).stop 2023-11-15T02:57:59.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.027+0000 7fcda5947700 1 -- 172.21.15.121:0/820895143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcda0153990 msgr2=0x7fcda00bc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:59.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.027+0000 7fcda5947700 1 --2- 172.21.15.121:0/820895143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcda0153990 0x7fcda00bc090 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fcda0064eb0 tx=0x7fcd8c016000).stop 2023-11-15T02:57:59.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.027+0000 7fcda5947700 1 -- 172.21.15.121:0/820895143 shutdown_connections 2023-11-15T02:57:59.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.027+0000 7fcda5947700 1 --2- 172.21.15.121:0/820895143 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd8405a560 0x7fcd8405ca20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:59.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.027+0000 7fcda5947700 1 --2- 172.21.15.121:0/820895143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcda0153d70 0x7fcda00bc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:59.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.027+0000 7fcda5947700 1 --2- 172.21.15.121:0/820895143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcda0153990 0x7fcda00bc090 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:59.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.027+0000 7fcda5947700 1 --2- 172.21.15.121:0/820895143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcda00c2ec0 0x7fcda01503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:59.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.027+0000 7fcda5947700 1 -- 172.21.15.121:0/820895143 >> 172.21.15.121:0/820895143 conn(0x7fcda0067a60 msgr2=0x7fcda01574b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:59.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.027+0000 7fcda5947700 1 -- 172.21.15.121:0/820895143 shutdown_connections 2023-11-15T02:57:59.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.027+0000 7fcda5947700 1 -- 172.21.15.121:0/820895143 wait complete. 2023-11-15T02:57:59.035 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T02:57:59.037 INFO:tasks.ceph.mds.c.smithi121.stderr: -3354> 2023-11-15T02:57:47.015+0000 7fb9c16dc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T02:57:59.053 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-11-15T02:57:59.085 INFO:tasks.ceph.mds.c.smithi121.stdout:starting mds.c at 2023-11-15T02:57:59.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18c993fd00 1 Processor -- start 2023-11-15T02:57:59.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18c993fd00 1 -- start start 2023-11-15T02:57:59.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18c993fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18ac0030b0 0x55a51c599ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:59.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18c993fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a51c61fcf0 0x55a51c59a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:59.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18c993fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a51c595430 0x55a51c595890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:59.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18c993fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51c4fbcc0 con 0x7f18ac0030b0 2023-11-15T02:57:59.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18c993fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51c4c6220 con 0x55a51c61fcf0 2023-11-15T02:57:59.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18c993fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a51c61c630 con 0x55a51c595430 2023-11-15T02:57:59.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18be019700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18ac0030b0 0x55a51c599ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:59.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18be019700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18ac0030b0 0x55a51c599ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34562/0 (socket says 172.21.15.121:34562) 2023-11-15T02:57:59.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18be019700 1 -- 172.21.15.121:0/2622293400 learned_addr learned my addr 172.21.15.121:0/2622293400 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:59.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18b7fff700 1 --2- 172.21.15.121:0/2622293400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a51c61fcf0 0x55a51c59a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:59.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18be81a700 1 --2- 172.21.15.121:0/2622293400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a51c595430 0x55a51c595890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:59.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18be019700 1 -- 172.21.15.121:0/2622293400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a51c595430 msgr2=0x55a51c595890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:59.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18be019700 1 --2- 172.21.15.121:0/2622293400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a51c595430 0x55a51c595890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:59.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18be019700 1 -- 172.21.15.121:0/2622293400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a51c61fcf0 msgr2=0x55a51c59a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:59.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18be019700 1 --2- 172.21.15.121:0/2622293400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a51c61fcf0 0x55a51c59a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:59.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18be019700 1 -- 172.21.15.121:0/2622293400 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a51c4bea70 con 0x7f18ac0030b0 2023-11-15T02:57:59.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18b7fff700 1 --2- 172.21.15.121:0/2622293400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a51c61fcf0 0x55a51c59a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:59.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18be019700 1 --2- 172.21.15.121:0/2622293400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18ac0030b0 0x55a51c599ff0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f18b000d310 tx=0x7f18b0001a90).ready entity=mon.0 client_cookie=7b71f96e01972aaa server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:59.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18bd017700 1 -- 172.21.15.121:0/2622293400 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b000dda0 con 0x7f18ac0030b0 2023-11-15T02:57:59.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18bd017700 1 -- 172.21.15.121:0/2622293400 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b0008040 con 0x7f18ac0030b0 2023-11-15T02:57:59.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.115+0000 7f18bd017700 1 -- 172.21.15.121:0/2622293400 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b000f550 con 0x7f18ac0030b0 2023-11-15T02:57:59.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.119+0000 7f18c993fd00 1 -- 172.21.15.121:0/2622293400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18ac0030b0 msgr2=0x55a51c599ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:59.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.119+0000 7f18c993fd00 1 --2- 172.21.15.121:0/2622293400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18ac0030b0 0x55a51c599ff0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f18b000d310 tx=0x7f18b0001a90).stop 2023-11-15T02:57:59.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.119+0000 7f18c993fd00 1 -- 172.21.15.121:0/2622293400 shutdown_connections 2023-11-15T02:57:59.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.119+0000 7f18c993fd00 1 --2- 172.21.15.121:0/2622293400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a51c595430 0x55a51c595890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:59.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.119+0000 7f18c993fd00 1 --2- 172.21.15.121:0/2622293400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a51c61fcf0 0x55a51c59a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:59.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.119+0000 7f18c993fd00 1 --2- 172.21.15.121:0/2622293400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18ac0030b0 0x55a51c599ff0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:59.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.119+0000 7f18c993fd00 1 -- 172.21.15.121:0/2622293400 >> 172.21.15.121:0/2622293400 conn(0x55a51c59d090 msgr2=0x55a51c628b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:59.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.119+0000 7f18c993fd00 1 -- 172.21.15.121:0/2622293400 shutdown_connections 2023-11-15T02:57:59.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.119+0000 7f18c993fd00 1 -- 172.21.15.121:0/2622293400 wait complete. 2023-11-15T02:57:59.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.119+0000 7f18c993fd00 1 Processor -- start 2023-11-15T02:57:59.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.119+0000 7f18c993fd00 1 -- start start 2023-11-15T02:57:59.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18c993fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a51c595430 0x55a51c61a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:59.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18c993fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a51c61fcf0 0x55a51c628b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:59.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18c993fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18ac0030b0 0x55a51c629040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:59.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18c993fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51c629530 con 0x55a51c61fcf0 2023-11-15T02:57:59.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18c993fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51c596b70 con 0x7f18ac0030b0 2023-11-15T02:57:59.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18c993fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a51c57c910 con 0x55a51c595430 2023-11-15T02:57:59.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18b7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a51c61fcf0 0x55a51c628b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:59.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18b7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a51c61fcf0 0x55a51c628b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34576/0 (socket says 172.21.15.121:34576) 2023-11-15T02:57:59.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18b7fff700 1 -- 172.21.15.121:0/231223634 learned_addr learned my addr 172.21.15.121:0/231223634 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:59.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18be019700 1 --2- 172.21.15.121:0/231223634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a51c595430 0x55a51c61a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:59.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18be81a700 1 --2- 172.21.15.121:0/231223634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18ac0030b0 0x55a51c629040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:59.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18b7fff700 1 -- 172.21.15.121:0/231223634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a51c595430 msgr2=0x55a51c61a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:59.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18b7fff700 1 --2- 172.21.15.121:0/231223634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a51c595430 0x55a51c61a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:59.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18b7fff700 1 -- 172.21.15.121:0/231223634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18ac0030b0 msgr2=0x55a51c629040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:59.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18b7fff700 1 --2- 172.21.15.121:0/231223634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18ac0030b0 0x55a51c629040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:59.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18b7fff700 1 -- 172.21.15.121:0/231223634 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b000a040 con 0x55a51c61fcf0 2023-11-15T02:57:59.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18be81a700 1 --2- 172.21.15.121:0/231223634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18ac0030b0 0x55a51c629040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:59.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18b7fff700 1 --2- 172.21.15.121:0/231223634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a51c61fcf0 0x55a51c628b50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f18ac00a920 tx=0x7f18a8009270).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:59.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18b6ffd700 1 -- 172.21.15.121:0/231223634 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8020070 con 0x55a51c61fcf0 2023-11-15T02:57:59.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18b6ffd700 1 -- 172.21.15.121:0/231223634 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a8000ed0 con 0x55a51c61fcf0 2023-11-15T02:57:59.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18b6ffd700 1 -- 172.21.15.121:0/231223634 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a800e680 con 0x55a51c61fcf0 2023-11-15T02:57:59.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18c993fd00 1 -- 172.21.15.121:0/231223634 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a51c4ca010 con 0x55a51c61fcf0 2023-11-15T02:57:59.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.123+0000 7f18c993fd00 1 -- 172.21.15.121:0/231223634 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a51c57d000 con 0x55a51c61fcf0 2023-11-15T02:57:59.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.127+0000 7f18b6ffd700 1 -- 172.21.15.121:0/231223634 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f18a8023070 con 0x55a51c61fcf0 2023-11-15T02:57:59.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.127+0000 7f18b6ffd700 1 --2- 172.21.15.121:0/231223634 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f18a0059020 0x7f18a005b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:59.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.127+0000 7f18b6ffd700 1 -- 172.21.15.121:0/231223634 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f18a8074800 con 0x55a51c61fcf0 2023-11-15T02:57:59.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.127+0000 7f18be019700 1 --2- 172.21.15.121:0/231223634 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f18a0059020 0x7f18a005b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:59.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.127+0000 7f18c993fd00 1 --2- 172.21.15.121:0/231223634 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a51c61aee0 0x55a51c62c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:59.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.127+0000 7f18c993fd00 1 -- 172.21.15.121:0/231223634 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 5.30 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected e34) v8 -- 0x55a51c57eaf0 con 0x55a51c61aee0 2023-11-15T02:57:59.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.127+0000 7f18be019700 1 --2- 172.21.15.121:0/231223634 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f18a0059020 0x7f18a005b4e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f18b000d310 tx=0x7f18b0001a90).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:59.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.131+0000 7f18be81a700 1 --2- 172.21.15.121:0/231223634 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a51c61aee0 0x55a51c62c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:59.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.131+0000 7f18be81a700 1 --2- 172.21.15.121:0/231223634 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a51c61aee0 0x55a51c62c450 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:59.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.139+0000 7f18be81a700 1 -- 172.21.15.121:0/231223634 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v34'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f18b800fab0 con 0x55a51c61aee0 2023-11-15T02:57:59.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.139+0000 7f18c993fd00 1 -- 172.21.15.121:0/231223634 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a51c61aee0 msgr2=0x55a51c62c450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:59.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.139+0000 7f18c993fd00 1 --2- 172.21.15.121:0/231223634 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a51c61aee0 0x55a51c62c450 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:59.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.139+0000 7f18c993fd00 1 -- 172.21.15.121:0/231223634 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f18a0059020 msgr2=0x7f18a005b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:59.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.139+0000 7f18c993fd00 1 --2- 172.21.15.121:0/231223634 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f18a0059020 0x7f18a005b4e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f18b000d310 tx=0x7f18b0001a90).stop 2023-11-15T02:57:59.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.139+0000 7f18c993fd00 1 -- 172.21.15.121:0/231223634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a51c61fcf0 msgr2=0x55a51c628b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:59.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.139+0000 7f18c993fd00 1 --2- 172.21.15.121:0/231223634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a51c61fcf0 0x55a51c628b50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f18ac00a920 tx=0x7f18a8009270).stop 2023-11-15T02:57:59.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.139+0000 7f18be81a700 1 -- 172.21.15.121:0/231223634 reap_dead start 2023-11-15T02:57:59.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.139+0000 7f18c993fd00 1 -- 172.21.15.121:0/231223634 shutdown_connections 2023-11-15T02:57:59.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.139+0000 7f18c993fd00 1 -- 172.21.15.121:0/231223634 >> 172.21.15.121:0/231223634 conn(0x55a51c59d090 msgr2=0x55a51c5971e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:59.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.139+0000 7f18c993fd00 1 -- 172.21.15.121:0/231223634 shutdown_connections 2023-11-15T02:57:59.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.139+0000 7f18c993fd00 1 -- 172.21.15.121:0/231223634 wait complete. 2023-11-15T02:57:59.150 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T02:57:59.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.355+0000 7f30ea7b9700 1 Processor -- start 2023-11-15T02:57:59.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.355+0000 7f30ea7b9700 1 -- start start 2023-11-15T02:57:59.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.359+0000 7f30ea7b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30e40c5c70 0x7f30e4153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:59.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.359+0000 7f30ea7b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f30e4066330 0x7f30e40ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:59.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.359+0000 7f30ea7b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e4065ea0 con 0x7f30e40acd00 2023-11-15T02:57:59.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.359+0000 7f30ea7b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e4066bb0 con 0x7f30e4066330 2023-11-15T02:57:59.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.359+0000 7f30ea7b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e4065890 con 0x7f30e40c5c70 2023-11-15T02:57:59.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.359+0000 7f30e8fb6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30e40c5c70 0x7f30e4153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:59.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.359+0000 7f30e8fb6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30e40c5c70 0x7f30e4153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:35770/0 (socket says 172.21.15.121:35770) 2023-11-15T02:57:59.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.359+0000 7f30e8fb6700 1 -- 172.21.15.121:0/4241111041 learned_addr learned my addr 172.21.15.121:0/4241111041 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:59.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.359+0000 7f30e9fb8700 1 --2- 172.21.15.121:0/4241111041 >> v2:172.21.15.190:3300/0 conn(0x7f30e4066330 0x7f30e40ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:59.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.359+0000 7f30e8fb6700 1 -- 172.21.15.121:0/4241111041 >> v2:172.21.15.190:3300/0 conn(0x7f30e4066330 msgr2=0x7f30e40ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:57:59.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.359+0000 7f30e8fb6700 1 --2- 172.21.15.121:0/4241111041 >> v2:172.21.15.190:3300/0 conn(0x7f30e4066330 0x7f30e40ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:59.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.359+0000 7f30e8fb6700 1 -- 172.21.15.121:0/4241111041 >> v1:172.21.15.190:6789/0 conn(0x7f30e40acd00 legacy=0x7f30e41536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:59.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.359+0000 7f30e8fb6700 1 -- 172.21.15.121:0/4241111041 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e40c64a0 con 0x7f30e40c5c70 2023-11-15T02:57:59.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.359+0000 7f30e8fb6700 1 --2- 172.21.15.121:0/4241111041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30e40c5c70 0x7f30e4153dd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f30d4006fd0 tx=0x7f30d40090f0).ready entity=mon.2 client_cookie=57e78fa367719087 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:59.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.363+0000 7f30dbfff700 1 -- 172.21.15.121:0/4241111041 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d400b580 con 0x7f30e40c5c70 2023-11-15T02:57:59.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.363+0000 7f30dbfff700 1 -- 172.21.15.121:0/4241111041 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d400b6e0 con 0x7f30e40c5c70 2023-11-15T02:57:59.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.363+0000 7f30dbfff700 1 -- 172.21.15.121:0/4241111041 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d400ba60 con 0x7f30e40c5c70 2023-11-15T02:57:59.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.363+0000 7f30ea7b9700 1 -- 172.21.15.121:0/4241111041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30e40c5c70 msgr2=0x7f30e4153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:59.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.363+0000 7f30ea7b9700 1 --2- 172.21.15.121:0/4241111041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30e40c5c70 0x7f30e4153dd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f30d4006fd0 tx=0x7f30d40090f0).stop 2023-11-15T02:57:59.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.363+0000 7f30ea7b9700 1 -- 172.21.15.121:0/4241111041 shutdown_connections 2023-11-15T02:57:59.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.363+0000 7f30ea7b9700 1 --2- 172.21.15.121:0/4241111041 >> v2:172.21.15.190:3300/0 conn(0x7f30e4066330 0x7f30e40ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:59.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.363+0000 7f30ea7b9700 1 --2- 172.21.15.121:0/4241111041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30e40c5c70 0x7f30e4153dd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:59.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.363+0000 7f30ea7b9700 1 -- 172.21.15.121:0/4241111041 >> 172.21.15.121:0/4241111041 conn(0x7f30e4063fc0 msgr2=0x7f30e4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:57:59.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.363+0000 7f30ea7b9700 1 -- 172.21.15.121:0/4241111041 shutdown_connections 2023-11-15T02:57:59.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.363+0000 7f30ea7b9700 1 -- 172.21.15.121:0/4241111041 wait complete. 2023-11-15T02:57:59.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.367+0000 7f30ea7b9700 1 Processor -- start 2023-11-15T02:57:59.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.367+0000 7f30ea7b9700 1 -- start start 2023-11-15T02:57:59.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.367+0000 7f30ea7b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30e4066330 0x7f30e41415e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:59.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.367+0000 7f30ea7b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30e40acd00 0x7f30e4154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:59.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.367+0000 7f30ea7b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30e40c5c70 0x7f30e40afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:59.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.367+0000 7f30ea7b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e4164ff0 con 0x7f30e40acd00 2023-11-15T02:57:59.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.367+0000 7f30ea7b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e4154ad0 con 0x7f30e4066330 2023-11-15T02:57:59.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.367+0000 7f30ea7b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e4164d10 con 0x7f30e40c5c70 2023-11-15T02:57:59.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.367+0000 7f30e8fb6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30e40acd00 0x7f30e4154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:59.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.367+0000 7f30e8fb6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30e40acd00 0x7f30e4154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34580/0 (socket says 172.21.15.121:34580) 2023-11-15T02:57:59.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.367+0000 7f30e8fb6700 1 -- 172.21.15.121:0/1239734770 learned_addr learned my addr 172.21.15.121:0/1239734770 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:57:59.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.371+0000 7f30e9fb8700 1 --2- 172.21.15.121:0/1239734770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30e40c5c70 0x7f30e40afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:59.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.367+0000 7f30e97b7700 1 --2- 172.21.15.121:0/1239734770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30e4066330 0x7f30e41415e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:59.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.371+0000 7f30e8fb6700 1 -- 172.21.15.121:0/1239734770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30e40c5c70 msgr2=0x7f30e40afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:59.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.371+0000 7f30e8fb6700 1 --2- 172.21.15.121:0/1239734770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30e40c5c70 0x7f30e40afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:59.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.371+0000 7f30e8fb6700 1 -- 172.21.15.121:0/1239734770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30e4066330 msgr2=0x7f30e41415e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:57:59.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.371+0000 7f30e8fb6700 1 --2- 172.21.15.121:0/1239734770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30e4066330 0x7f30e41415e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:57:59.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.371+0000 7f30e8fb6700 1 -- 172.21.15.121:0/1239734770 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d4009370 con 0x7f30e40acd00 2023-11-15T02:57:59.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.371+0000 7f30e97b7700 1 --2- 172.21.15.121:0/1239734770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30e4066330 0x7f30e41415e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:59.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.371+0000 7f30e9fb8700 1 --2- 172.21.15.121:0/1239734770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30e40c5c70 0x7f30e40afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:57:59.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.371+0000 7f30e8fb6700 1 --2- 172.21.15.121:0/1239734770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30e40acd00 0x7f30e4154f60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f30d4009ea0 tx=0x7f30d4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:59.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.371+0000 7f30d97fa700 1 -- 172.21.15.121:0/1239734770 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d400b850 con 0x7f30e40acd00 2023-11-15T02:57:59.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.371+0000 7f30d97fa700 1 -- 172.21.15.121:0/1239734770 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d400b9b0 con 0x7f30e40acd00 2023-11-15T02:57:59.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.371+0000 7f30ea7b9700 1 -- 172.21.15.121:0/1239734770 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e4154480 con 0x7f30e40acd00 2023-11-15T02:57:59.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.371+0000 7f30d97fa700 1 -- 172.21.15.121:0/1239734770 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d401c670 con 0x7f30e40acd00 2023-11-15T02:57:59.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.371+0000 7f30ea7b9700 1 -- 172.21.15.121:0/1239734770 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e40affa0 con 0x7f30e40acd00 2023-11-15T02:57:59.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.371+0000 7f30ea7b9700 1 -- 172.21.15.121:0/1239734770 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e41542c0 con 0x7f30e40acd00 2023-11-15T02:57:59.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.375+0000 7f30d97fa700 1 -- 172.21.15.121:0/1239734770 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f30d401ca60 con 0x7f30e40acd00 2023-11-15T02:57:59.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.375+0000 7f30d97fa700 1 --2- 172.21.15.121:0/1239734770 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f30c805a510 0x7f30c805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:57:59.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.375+0000 7f30d97fa700 1 -- 172.21.15.121:0/1239734770 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f30d4076160 con 0x7f30e40acd00 2023-11-15T02:57:59.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.375+0000 7f30e97b7700 1 --2- 172.21.15.121:0/1239734770 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f30c805a510 0x7f30c805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:57:59.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.375+0000 7f30e97b7700 1 --2- 172.21.15.121:0/1239734770 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f30c805a510 0x7f30c805c9d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f30e4167550 tx=0x7f30e0009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:57:59.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.383+0000 7f30d97fa700 1 -- 172.21.15.121:0/1239734770 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f30d4049830 con 0x7f30e40acd00 2023-11-15T02:57:59.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:57:59.579+0000 7f30ea7b9700 1 -- 172.21.15.121:0/1239734770 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f30e40c61c0 con 0x7f30e40acd00 2023-11-15T02:58:00.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.027+0000 7f30d97fa700 1 -- 172.21.15.121:0/1239734770 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v18) v1 ==== 162+0+0 (secure 0 0 0) 0x7f30d4020020 con 0x7f30e40acd00 2023-11-15T02:58:00.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.031+0000 7f30ea7b9700 1 -- 172.21.15.121:0/1239734770 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f30c805a510 msgr2=0x7f30c805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:00.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.031+0000 7f30ea7b9700 1 --2- 172.21.15.121:0/1239734770 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f30c805a510 0x7f30c805c9d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f30e4167550 tx=0x7f30e0009040).stop 2023-11-15T02:58:00.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.031+0000 7f30ea7b9700 1 -- 172.21.15.121:0/1239734770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30e40acd00 msgr2=0x7f30e4154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:00.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.031+0000 7f30ea7b9700 1 --2- 172.21.15.121:0/1239734770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30e40acd00 0x7f30e4154f60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f30d4009ea0 tx=0x7f30d4017000).stop 2023-11-15T02:58:00.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.031+0000 7f30ea7b9700 1 -- 172.21.15.121:0/1239734770 shutdown_connections 2023-11-15T02:58:00.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.031+0000 7f30ea7b9700 1 --2- 172.21.15.121:0/1239734770 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f30c805a510 0x7f30c805c9d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:00.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.031+0000 7f30ea7b9700 1 --2- 172.21.15.121:0/1239734770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30e40c5c70 0x7f30e40afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:00.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.031+0000 7f30ea7b9700 1 --2- 172.21.15.121:0/1239734770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30e40acd00 0x7f30e4154f60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:00.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.031+0000 7f30ea7b9700 1 --2- 172.21.15.121:0/1239734770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30e4066330 0x7f30e41415e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:00.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.031+0000 7f30ea7b9700 1 -- 172.21.15.121:0/1239734770 >> 172.21.15.121:0/1239734770 conn(0x7f30e4063fc0 msgr2=0x7f30e4062190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:00.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.031+0000 7f30ea7b9700 1 -- 172.21.15.121:0/1239734770 shutdown_connections 2023-11-15T02:58:00.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.031+0000 7f30ea7b9700 1 -- 172.21.15.121:0/1239734770 wait complete. 2023-11-15T02:58:00.041 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T02:58:00.060 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:58:00.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.207+0000 7fc99af07700 1 Processor -- start 2023-11-15T02:58:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.207+0000 7fc99af07700 1 -- start start 2023-11-15T02:58:00.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.211+0000 7fc99af07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc9940c3280 0x7fc9940c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:00.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.211+0000 7fc99af07700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc994152140 0x7fc9941525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:00.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.211+0000 7fc99af07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc99415be30 0x7fc99415c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:00.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.211+0000 7fc99af07700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9940c1860 con 0x7fc994152140 2023-11-15T02:58:00.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.211+0000 7fc99af07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9940c1d50 con 0x7fc9940c3280 2023-11-15T02:58:00.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.211+0000 7fc99af07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99415cb80 con 0x7fc99415be30 2023-11-15T02:58:00.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.211+0000 7fc999704700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc994152140 0x7fc9941525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:00.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.211+0000 7fc999704700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc994152140 0x7fc9941525f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34596/0 (socket says 172.21.15.121:34596) 2023-11-15T02:58:00.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.211+0000 7fc999f05700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc9940c3280 0x7fc9940c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:00.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.211+0000 7fc999704700 1 -- 172.21.15.121:0/4012059662 learned_addr learned my addr 172.21.15.121:0/4012059662 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:00.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.211+0000 7fc99a706700 1 --2- 172.21.15.121:0/4012059662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc99415be30 0x7fc99415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:00.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.211+0000 7fc99a706700 1 -- 172.21.15.121:0/4012059662 >> v2:172.21.15.190:3300/0 conn(0x7fc9940c3280 msgr2=0x7fc9940c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:00.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.211+0000 7fc99a706700 1 --2- 172.21.15.121:0/4012059662 >> v2:172.21.15.190:3300/0 conn(0x7fc9940c3280 0x7fc9940c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:00.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.211+0000 7fc99a706700 1 -- 172.21.15.121:0/4012059662 >> v2:172.21.15.121:3300/0 conn(0x7fc994152140 msgr2=0x7fc9941525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:00.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.211+0000 7fc99a706700 1 --2- 172.21.15.121:0/4012059662 >> v2:172.21.15.121:3300/0 conn(0x7fc994152140 0x7fc9941525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:00.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.211+0000 7fc99a706700 1 -- 172.21.15.121:0/4012059662 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9940c1f80 con 0x7fc99415be30 2023-11-15T02:58:00.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.211+0000 7fc999f05700 1 --2- 172.21.15.121:0/4012059662 >> v2:172.21.15.190:3300/0 conn(0x7fc9940c3280 0x7fc9940c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:58:00.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.215+0000 7fc99a706700 1 --2- 172.21.15.121:0/4012059662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc99415be30 0x7fc99415c280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fc9940c7720 tx=0x7fc98c009780).ready entity=mon.2 client_cookie=8f5b391d1e9ab7f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:00.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.215+0000 7fc998f03700 1 -- 172.21.15.121:0/4012059662 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c00e070 con 0x7fc99415be30 2023-11-15T02:58:00.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.215+0000 7fc998f03700 1 -- 172.21.15.121:0/4012059662 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98c00ba70 con 0x7fc99415be30 2023-11-15T02:58:00.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.215+0000 7fc998f03700 1 -- 172.21.15.121:0/4012059662 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c00bdf0 con 0x7fc99415be30 2023-11-15T02:58:00.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.215+0000 7fc99af07700 1 -- 172.21.15.121:0/4012059662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc99415be30 msgr2=0x7fc99415c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:00.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.215+0000 7fc99af07700 1 --2- 172.21.15.121:0/4012059662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc99415be30 0x7fc99415c280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fc9940c7720 tx=0x7fc98c009780).stop 2023-11-15T02:58:00.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.215+0000 7fc99af07700 1 -- 172.21.15.121:0/4012059662 shutdown_connections 2023-11-15T02:58:00.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.215+0000 7fc99af07700 1 --2- 172.21.15.121:0/4012059662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc99415be30 0x7fc99415c280 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:00.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.215+0000 7fc99af07700 1 --2- 172.21.15.121:0/4012059662 >> v2:172.21.15.121:3300/0 conn(0x7fc994152140 0x7fc9941525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:00.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.215+0000 7fc99af07700 1 --2- 172.21.15.121:0/4012059662 >> v2:172.21.15.190:3300/0 conn(0x7fc9940c3280 0x7fc9940c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:00.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.215+0000 7fc99af07700 1 -- 172.21.15.121:0/4012059662 >> 172.21.15.121:0/4012059662 conn(0x7fc994062190 msgr2=0x7fc994062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:00.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.215+0000 7fc99af07700 1 -- 172.21.15.121:0/4012059662 shutdown_connections 2023-11-15T02:58:00.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.215+0000 7fc99af07700 1 -- 172.21.15.121:0/4012059662 wait complete. 2023-11-15T02:58:00.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.219+0000 7fc99af07700 1 Processor -- start 2023-11-15T02:58:00.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.219+0000 7fc99af07700 1 -- start start 2023-11-15T02:58:00.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.219+0000 7fc99af07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9940c3280 0x7fc99414dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:00.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.219+0000 7fc99af07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc994152140 0x7fc9940c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:00.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.219+0000 7fc99af07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc99415be30 0x7fc9940ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:00.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.219+0000 7fc99af07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994159320 con 0x7fc994152140 2023-11-15T02:58:00.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.219+0000 7fc99af07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994157830 con 0x7fc9940c3280 2023-11-15T02:58:00.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.219+0000 7fc99af07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9941573a0 con 0x7fc99415be30 2023-11-15T02:58:00.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.219+0000 7fc999704700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc994152140 0x7fc9940c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:00.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.219+0000 7fc999704700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc994152140 0x7fc9940c6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34612/0 (socket says 172.21.15.121:34612) 2023-11-15T02:58:00.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.219+0000 7fc999704700 1 -- 172.21.15.121:0/1130357084 learned_addr learned my addr 172.21.15.121:0/1130357084 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:00.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.223+0000 7fc999f05700 1 --2- 172.21.15.121:0/1130357084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9940c3280 0x7fc99414dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:00.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.223+0000 7fc99a706700 1 --2- 172.21.15.121:0/1130357084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc99415be30 0x7fc9940ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:00.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.223+0000 7fc999704700 1 -- 172.21.15.121:0/1130357084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc99415be30 msgr2=0x7fc9940ab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:00.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.223+0000 7fc999704700 1 --2- 172.21.15.121:0/1130357084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc99415be30 0x7fc9940ab8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:00.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.223+0000 7fc999704700 1 -- 172.21.15.121:0/1130357084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9940c3280 msgr2=0x7fc99414dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:00.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.223+0000 7fc999704700 1 --2- 172.21.15.121:0/1130357084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9940c3280 0x7fc99414dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:00.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.223+0000 7fc999704700 1 -- 172.21.15.121:0/1130357084 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9900092a0 con 0x7fc994152140 2023-11-15T02:58:00.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.223+0000 7fc999704700 1 --2- 172.21.15.121:0/1130357084 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc994152140 0x7fc9940c6d20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fc98400e930 tx=0x7fc9840090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:00.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.223+0000 7fc989ffb700 1 -- 172.21.15.121:0/1130357084 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984012600 con 0x7fc994152140 2023-11-15T02:58:00.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.223+0000 7fc989ffb700 1 -- 172.21.15.121:0/1130357084 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc984012760 con 0x7fc994152140 2023-11-15T02:58:00.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.223+0000 7fc99af07700 1 -- 172.21.15.121:0/1130357084 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98c009a20 con 0x7fc994152140 2023-11-15T02:58:00.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.223+0000 7fc989ffb700 1 -- 172.21.15.121:0/1130357084 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984012ae0 con 0x7fc994152140 2023-11-15T02:58:00.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.223+0000 7fc99af07700 1 -- 172.21.15.121:0/1130357084 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc994156d50 con 0x7fc994152140 2023-11-15T02:58:00.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.223+0000 7fc99af07700 1 -- 172.21.15.121:0/1130357084 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99406bcf0 con 0x7fc994152140 2023-11-15T02:58:00.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.227+0000 7fc989ffb700 1 -- 172.21.15.121:0/1130357084 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc98401b400 con 0x7fc994152140 2023-11-15T02:58:00.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.227+0000 7fc989ffb700 1 --2- 172.21.15.121:0/1130357084 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc97805a540 0x7fc97805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:00.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.227+0000 7fc989ffb700 1 -- 172.21.15.121:0/1130357084 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fc984074b70 con 0x7fc994152140 2023-11-15T02:58:00.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.227+0000 7fc999f05700 1 --2- 172.21.15.121:0/1130357084 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc97805a540 0x7fc97805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:00.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.231+0000 7fc999f05700 1 --2- 172.21.15.121:0/1130357084 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc97805a540 0x7fc97805ca00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fc990009dc0 tx=0x7fc990009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:00.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.235+0000 7fc989ffb700 1 -- 172.21.15.121:0/1130357084 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc984048220 con 0x7fc994152140 2023-11-15T02:58:00.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.431+0000 7fc99af07700 1 -- 172.21.15.121:0/1130357084 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9941574e0 con 0x7fc994152140 2023-11-15T02:58:00.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.435+0000 7fc989ffb700 1 -- 172.21.15.121:0/1130357084 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4502 (secure 0 0 0) 0x7fc98404b840 con 0x7fc994152140 2023-11-15T02:58:00.440 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:58:00.440 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2544127447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2544127447},{"type":"v1","addr":"172.21.15.190:6833","nonce":2544127447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1815834193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1815834193},{"type":"v1","addr":"172.21.15.190:6835","nonce":1815834193}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4950,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3229917854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3229917854},{"type":"v1","addr":"172.21.15.121:6835","nonce":3229917854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:57:44.541243+0000","modified":"2023-11-15T02:58:00.032532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5178":{"gid":5178,"name":"c","rank":0,"incarnation":19,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6837/3305580606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3305580606},{"type":"v1","addr":"172.21.15.121:6837","nonce":3305580606}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-15T02:58:00.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.435+0000 7fc99af07700 1 -- 172.21.15.121:0/1130357084 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc97805a540 msgr2=0x7fc97805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:00.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.435+0000 7fc99af07700 1 --2- 172.21.15.121:0/1130357084 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc97805a540 0x7fc97805ca00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fc990009dc0 tx=0x7fc990009060).stop 2023-11-15T02:58:00.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.435+0000 7fc99af07700 1 -- 172.21.15.121:0/1130357084 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc994152140 msgr2=0x7fc9940c6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:00.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.435+0000 7fc99af07700 1 --2- 172.21.15.121:0/1130357084 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc994152140 0x7fc9940c6d20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fc98400e930 tx=0x7fc9840090f0).stop 2023-11-15T02:58:00.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.435+0000 7fc99af07700 1 -- 172.21.15.121:0/1130357084 shutdown_connections 2023-11-15T02:58:00.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.435+0000 7fc99af07700 1 --2- 172.21.15.121:0/1130357084 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc97805a540 0x7fc97805ca00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:00.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.435+0000 7fc99af07700 1 --2- 172.21.15.121:0/1130357084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc99415be30 0x7fc9940ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:00.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.435+0000 7fc99af07700 1 --2- 172.21.15.121:0/1130357084 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc994152140 0x7fc9940c6d20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:00.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.435+0000 7fc99af07700 1 --2- 172.21.15.121:0/1130357084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9940c3280 0x7fc99414dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:00.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.435+0000 7fc99af07700 1 -- 172.21.15.121:0/1130357084 >> 172.21.15.121:0/1130357084 conn(0x7fc994062190 msgr2=0x7fc99414ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:00.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.435+0000 7fc99af07700 1 -- 172.21.15.121:0/1130357084 shutdown_connections 2023-11-15T02:58:00.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:00.439+0000 7fc99af07700 1 -- 172.21.15.121:0/1130357084 wait complete. 2023-11-15T02:58:00.445 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 19 2023-11-15T02:58:00.864 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T02:57:44.541243+0000', 'modified': '2023-11-15T02:58:00.032532+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 34, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5178}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5178': {'gid': 5178, 'name': 'c', 'rank': 0, 'incarnation': 19, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.121:6837/3305580606', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6836', 'nonce': 3305580606}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 3305580606}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T02:58:00.865 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5178:up:replay 2023-11-15T02:58:01.866 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:58:02.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.019+0000 7f0210879700 1 Processor -- start 2023-11-15T02:58:02.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.023+0000 7f0210879700 1 -- start start 2023-11-15T02:58:02.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.023+0000 7f0210879700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f020c151870 0x7f020c151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:02.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.023+0000 7f0210879700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c0657f0 con 0x7f020c0c30e0 2023-11-15T02:58:02.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.023+0000 7f0210879700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c065ad0 con 0x7f020c152140 2023-11-15T02:58:02.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.023+0000 7f0210879700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c0aa260 con 0x7f020c151870 2023-11-15T02:58:02.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.023+0000 7f020b7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f020c151870 0x7f020c151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:02.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.023+0000 7f020bfff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f020c0c30e0 0x7f020c065f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:54524/0 (socket says 172.21.15.121:54524) 2023-11-15T02:58:02.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.023+0000 7f020bfff700 1 -- 172.21.15.121:0/2408180647 learned_addr learned my addr 172.21.15.121:0/2408180647 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:02.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.023+0000 7f020b7fe700 1 -- 172.21.15.121:0/2408180647 >> v1:172.21.15.190:6789/0 conn(0x7f020c152140 legacy=0x7f020c0a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:02.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.023+0000 7f020b7fe700 1 -- 172.21.15.121:0/2408180647 >> v1:172.21.15.121:6789/0 conn(0x7f020c0c30e0 legacy=0x7f020c065f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:02.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.023+0000 7f020b7fe700 1 -- 172.21.15.121:0/2408180647 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0aa540 con 0x7f020c151870 2023-11-15T02:58:02.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.023+0000 7f020affd700 1 -- 172.21.15.121:0/2408180647 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181054369 0 0) 0x7f020c0657f0 con 0x7f020c0c30e0 2023-11-15T02:58:02.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.023+0000 7f020b7fe700 1 --2- 172.21.15.121:0/2408180647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f020c151870 0x7f020c151c50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f01f4009760 tx=0x7f01f4009060).ready entity=mon.0 client_cookie=5dffe11d1c2ad3c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:02.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.027+0000 7f020affd700 1 -- 172.21.15.121:0/2408180647 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f400b580 con 0x7f020c151870 2023-11-15T02:58:02.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.027+0000 7f020affd700 1 -- 172.21.15.121:0/2408180647 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f400b6e0 con 0x7f020c151870 2023-11-15T02:58:02.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.027+0000 7f020affd700 1 -- 172.21.15.121:0/2408180647 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f400ba60 con 0x7f020c151870 2023-11-15T02:58:02.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.027+0000 7f0210879700 1 -- 172.21.15.121:0/2408180647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f020c151870 msgr2=0x7f020c151c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:02.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.027+0000 7f0210879700 1 --2- 172.21.15.121:0/2408180647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f020c151870 0x7f020c151c50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f01f4009760 tx=0x7f01f4009060).stop 2023-11-15T02:58:02.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.027+0000 7f0210879700 1 -- 172.21.15.121:0/2408180647 shutdown_connections 2023-11-15T02:58:02.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.027+0000 7f0210879700 1 --2- 172.21.15.121:0/2408180647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f020c151870 0x7f020c151c50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:02.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.027+0000 7f0210879700 1 -- 172.21.15.121:0/2408180647 >> 172.21.15.121:0/2408180647 conn(0x7f020c062190 msgr2=0x7f020c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:02.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.027+0000 7f0210879700 1 -- 172.21.15.121:0/2408180647 shutdown_connections 2023-11-15T02:58:02.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.027+0000 7f0210879700 1 -- 172.21.15.121:0/2408180647 wait complete. 2023-11-15T02:58:02.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.027+0000 7f0210879700 1 Processor -- start 2023-11-15T02:58:02.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0210879700 1 -- start start 2023-11-15T02:58:02.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0210879700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f020c0c30e0 0x7f020c0c1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:02.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0210879700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f020c151870 0x7f020c0c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:02.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0210879700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f020c152140 0x7f020c1640a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:02.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0210879700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0c58a0 con 0x7f020c151870 2023-11-15T02:58:02.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0210879700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c156ab0 con 0x7f020c152140 2023-11-15T02:58:02.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0210879700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c156d90 con 0x7f020c0c30e0 2023-11-15T02:58:02.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0202ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f020c151870 0x7f020c0c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:02.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0202ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f020c151870 0x7f020c0c53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34638/0 (socket says 172.21.15.121:34638) 2023-11-15T02:58:02.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0202ffd700 1 -- 172.21.15.121:0/3035387849 learned_addr learned my addr 172.21.15.121:0/3035387849 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:02.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f020b7fe700 1 --2- 172.21.15.121:0/3035387849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f020c0c30e0 0x7f020c0c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:02.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f020bfff700 1 --2- 172.21.15.121:0/3035387849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f020c152140 0x7f020c1640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:02.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0202ffd700 1 -- 172.21.15.121:0/3035387849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f020c0c30e0 msgr2=0x7f020c0c1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:02.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0202ffd700 1 --2- 172.21.15.121:0/3035387849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f020c0c30e0 0x7f020c0c1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:02.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0202ffd700 1 -- 172.21.15.121:0/3035387849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f020c152140 msgr2=0x7f020c1640a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:02.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0202ffd700 1 --2- 172.21.15.121:0/3035387849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f020c152140 0x7f020c1640a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:02.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0202ffd700 1 -- 172.21.15.121:0/3035387849 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f40092a0 con 0x7f020c151870 2023-11-15T02:58:02.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f020bfff700 1 --2- 172.21.15.121:0/3035387849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f020c152140 0x7f020c1640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:02.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0202ffd700 1 --2- 172.21.15.121:0/3035387849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f020c151870 0x7f020c0c53b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f01fc00c440 tx=0x7f01fc007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:02.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0203fff700 1 -- 172.21.15.121:0/3035387849 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc00ce40 con 0x7f020c151870 2023-11-15T02:58:02.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0203fff700 1 -- 172.21.15.121:0/3035387849 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc007020 con 0x7f020c151870 2023-11-15T02:58:02.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0203fff700 1 -- 172.21.15.121:0/3035387849 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc0073c0 con 0x7f020c151870 2023-11-15T02:58:02.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0210879700 1 -- 172.21.15.121:0/3035387849 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020c1648d0 con 0x7f020c151870 2023-11-15T02:58:02.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.031+0000 7f0210879700 1 -- 172.21.15.121:0/3035387849 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020c164f10 con 0x7f020c151870 2023-11-15T02:58:02.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.035+0000 7f0210879700 1 -- 172.21.15.121:0/3035387849 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020c156f40 con 0x7f020c151870 2023-11-15T02:58:02.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.035+0000 7f0203fff700 1 -- 172.21.15.121:0/3035387849 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f01fc0068b0 con 0x7f020c151870 2023-11-15T02:58:02.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.039+0000 7f0203fff700 1 --2- 172.21.15.121:0/3035387849 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f01ec05a510 0x7f01ec05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:02.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.039+0000 7f0203fff700 1 -- 172.21.15.121:0/3035387849 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f01fc0742c0 con 0x7f020c151870 2023-11-15T02:58:02.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.039+0000 7f020b7fe700 1 --2- 172.21.15.121:0/3035387849 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f01ec05a510 0x7f01ec05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:02.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.039+0000 7f020b7fe700 1 --2- 172.21.15.121:0/3035387849 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f01ec05a510 0x7f01ec05c9d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f020c0c2470 tx=0x7f01f4017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:02.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.043+0000 7f0203fff700 1 -- 172.21.15.121:0/3035387849 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f020c156f40 con 0x7f020c151870 2023-11-15T02:58:02.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.243+0000 7f0210879700 1 -- 172.21.15.121:0/3035387849 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f020c156f00 con 0x7f020c151870 2023-11-15T02:58:02.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.243+0000 7f0203fff700 1 -- 172.21.15.121:0/3035387849 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4502 (secure 0 0 0) 0x7f01fc047990 con 0x7f020c151870 2023-11-15T02:58:02.251 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:58:02.251 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2544127447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2544127447},{"type":"v1","addr":"172.21.15.190:6833","nonce":2544127447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1815834193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1815834193},{"type":"v1","addr":"172.21.15.190:6835","nonce":1815834193}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4950,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3229917854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3229917854},{"type":"v1","addr":"172.21.15.121:6835","nonce":3229917854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:57:44.541243+0000","modified":"2023-11-15T02:58:02.047148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5178":{"gid":5178,"name":"c","rank":0,"incarnation":19,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.121:6837/3305580606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3305580606},{"type":"v1","addr":"172.21.15.121:6837","nonce":3305580606}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-15T02:58:02.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.247+0000 7f0210879700 1 -- 172.21.15.121:0/3035387849 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f01ec05a510 msgr2=0x7f01ec05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:02.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.247+0000 7f0210879700 1 --2- 172.21.15.121:0/3035387849 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f01ec05a510 0x7f01ec05c9d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f020c0c2470 tx=0x7f01f4017000).stop 2023-11-15T02:58:02.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.247+0000 7f0210879700 1 -- 172.21.15.121:0/3035387849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f020c151870 msgr2=0x7f020c0c53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:02.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.247+0000 7f0210879700 1 --2- 172.21.15.121:0/3035387849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f020c151870 0x7f020c0c53b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f01fc00c440 tx=0x7f01fc007dd0).stop 2023-11-15T02:58:02.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.247+0000 7f0210879700 1 -- 172.21.15.121:0/3035387849 shutdown_connections 2023-11-15T02:58:02.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.247+0000 7f0210879700 1 --2- 172.21.15.121:0/3035387849 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f01ec05a510 0x7f01ec05c9d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:02.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.247+0000 7f0210879700 1 --2- 172.21.15.121:0/3035387849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f020c152140 0x7f020c1640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:02.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.247+0000 7f0210879700 1 --2- 172.21.15.121:0/3035387849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f020c151870 0x7f020c0c53b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:02.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.247+0000 7f0210879700 1 --2- 172.21.15.121:0/3035387849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f020c0c30e0 0x7f020c0c1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:02.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.247+0000 7f0210879700 1 -- 172.21.15.121:0/3035387849 >> 172.21.15.121:0/3035387849 conn(0x7f020c062190 msgr2=0x7f020c14f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:02.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.247+0000 7f0210879700 1 -- 172.21.15.121:0/3035387849 shutdown_connections 2023-11-15T02:58:02.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:02.247+0000 7f0210879700 1 -- 172.21.15.121:0/3035387849 wait complete. 2023-11-15T02:58:02.256 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 21 2023-11-15T02:58:02.675 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T02:57:44.541243+0000', 'modified': '2023-11-15T02:58:02.047148+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 34, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5178}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5178': {'gid': 5178, 'name': 'c', 'rank': 0, 'incarnation': 19, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.121:6837/3305580606', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6836', 'nonce': 3305580606}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 3305580606}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T02:58:02.675 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5178:up:rejoin 2023-11-15T02:58:03.676 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:58:03.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.839+0000 7f8c16adb700 1 Processor -- start 2023-11-15T02:58:03.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.839+0000 7f8c16adb700 1 -- start start 2023-11-15T02:58:03.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.839+0000 7f8c16adb700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8c100a9ff0 0x7f8c100c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:03.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.839+0000 7f8c16adb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c100cb820 0x7f8c1015c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:03.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.839+0000 7f8c16adb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8c1015c930 0x7f8c10067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:03.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.839+0000 7f8c16adb700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c100c6610 con 0x7f8c100a9ff0 2023-11-15T02:58:03.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.839+0000 7f8c16adb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c100c5ce0 con 0x7f8c1015c930 2023-11-15T02:58:03.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.839+0000 7f8c16adb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100c5fc0 con 0x7f8c100cb820 2023-11-15T02:58:03.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c14877700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8c100a9ff0 0x7f8c100c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:03.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c14877700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8c100a9ff0 0x7f8c100c6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34644/0 (socket says 172.21.15.121:34644) 2023-11-15T02:58:03.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c14877700 1 -- 172.21.15.121:0/3064494726 learned_addr learned my addr 172.21.15.121:0/3064494726 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:03.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c07fff700 1 --2- 172.21.15.121:0/3064494726 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c100cb820 0x7f8c1015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:03.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c15078700 1 --2- 172.21.15.121:0/3064494726 >> v2:172.21.15.190:3300/0 conn(0x7f8c1015c930 0x7f8c10067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:03.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c14877700 1 -- 172.21.15.121:0/3064494726 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c100cb820 msgr2=0x7f8c1015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:03.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c14877700 1 --2- 172.21.15.121:0/3064494726 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c100cb820 0x7f8c1015c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:03.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c14877700 1 -- 172.21.15.121:0/3064494726 >> v2:172.21.15.190:3300/0 conn(0x7f8c1015c930 msgr2=0x7f8c10067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:03.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c14877700 1 --2- 172.21.15.121:0/3064494726 >> v2:172.21.15.190:3300/0 conn(0x7f8c1015c930 0x7f8c10067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:03.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c14877700 1 -- 172.21.15.121:0/3064494726 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c100c62a0 con 0x7f8c100a9ff0 2023-11-15T02:58:03.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c14877700 1 --2- 172.21.15.121:0/3064494726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c100a9ff0 0x7f8c100c6aa0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f8c10065800 tx=0x7f8c0c0090f0).ready entity=mon.0 client_cookie=2b78d42e6581d0d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:03.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c077fe700 1 -- 172.21.15.121:0/3064494726 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c00b600 con 0x7f8c100a9ff0 2023-11-15T02:58:03.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c077fe700 1 -- 172.21.15.121:0/3064494726 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c00b760 con 0x7f8c100a9ff0 2023-11-15T02:58:03.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c077fe700 1 -- 172.21.15.121:0/3064494726 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c00bae0 con 0x7f8c100a9ff0 2023-11-15T02:58:03.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c16adb700 1 -- 172.21.15.121:0/3064494726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c100a9ff0 msgr2=0x7f8c100c6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:03.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c16adb700 1 --2- 172.21.15.121:0/3064494726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c100a9ff0 0x7f8c100c6aa0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f8c10065800 tx=0x7f8c0c0090f0).stop 2023-11-15T02:58:03.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c16adb700 1 -- 172.21.15.121:0/3064494726 shutdown_connections 2023-11-15T02:58:03.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c16adb700 1 --2- 172.21.15.121:0/3064494726 >> v2:172.21.15.190:3300/0 conn(0x7f8c1015c930 0x7f8c10067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:03.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c16adb700 1 --2- 172.21.15.121:0/3064494726 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c100cb820 0x7f8c1015c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:03.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c16adb700 1 --2- 172.21.15.121:0/3064494726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c100a9ff0 0x7f8c100c6aa0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:03.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c16adb700 1 -- 172.21.15.121:0/3064494726 >> 172.21.15.121:0/3064494726 conn(0x7f8c10058c50 msgr2=0x7f8c10059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:03.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c16adb700 1 -- 172.21.15.121:0/3064494726 shutdown_connections 2023-11-15T02:58:03.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.843+0000 7f8c16adb700 1 -- 172.21.15.121:0/3064494726 wait complete. 2023-11-15T02:58:03.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.847+0000 7f8c16adb700 1 Processor -- start 2023-11-15T02:58:03.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.847+0000 7f8c16adb700 1 -- start start 2023-11-15T02:58:03.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.851+0000 7f8c16adb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c100a9ff0 0x7f8c10159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:03.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.851+0000 7f8c16adb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c100cb820 0x7f8c100c3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:03.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.851+0000 7f8c16adb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c1015c930 0x7f8c100c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:03.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.851+0000 7f8c16adb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100c1d20 con 0x7f8c100cb820 2023-11-15T02:58:03.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.851+0000 7f8c16adb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100c4130 con 0x7f8c1015c930 2023-11-15T02:58:03.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.851+0000 7f8c16adb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100c4410 con 0x7f8c100a9ff0 2023-11-15T02:58:03.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.851+0000 7f8c07fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c100cb820 0x7f8c100c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:03.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.851+0000 7f8c07fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c100cb820 0x7f8c100c3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34648/0 (socket says 172.21.15.121:34648) 2023-11-15T02:58:03.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.851+0000 7f8c07fff700 1 -- 172.21.15.121:0/1057746349 learned_addr learned my addr 172.21.15.121:0/1057746349 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:03.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.851+0000 7f8c14877700 1 --2- 172.21.15.121:0/1057746349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c100a9ff0 0x7f8c10159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:03.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.851+0000 7f8c15078700 1 --2- 172.21.15.121:0/1057746349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c1015c930 0x7f8c100c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:03.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.851+0000 7f8c07fff700 1 -- 172.21.15.121:0/1057746349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c100a9ff0 msgr2=0x7f8c10159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:03.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.851+0000 7f8c07fff700 1 --2- 172.21.15.121:0/1057746349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c100a9ff0 0x7f8c10159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:03.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.851+0000 7f8c07fff700 1 -- 172.21.15.121:0/1057746349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c1015c930 msgr2=0x7f8c100c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:03.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.851+0000 7f8c07fff700 1 --2- 172.21.15.121:0/1057746349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c1015c930 0x7f8c100c3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:03.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.851+0000 7f8c07fff700 1 -- 172.21.15.121:0/1057746349 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0c009370 con 0x7f8c100cb820 2023-11-15T02:58:03.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.851+0000 7f8c15078700 1 --2- 172.21.15.121:0/1057746349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c1015c930 0x7f8c100c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:03.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.851+0000 7f8c07fff700 1 --2- 172.21.15.121:0/1057746349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c100cb820 0x7f8c100c3670 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f8c0000d890 tx=0x7f8c00008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:03.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.851+0000 7f8c04ff9700 1 -- 172.21.15.121:0/1057746349 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c000125f0 con 0x7f8c100cb820 2023-11-15T02:58:03.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.855+0000 7f8c04ff9700 1 -- 172.21.15.121:0/1057746349 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c00012750 con 0x7f8c100cb820 2023-11-15T02:58:03.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.855+0000 7f8c16adb700 1 -- 172.21.15.121:0/1057746349 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c100c46f0 con 0x7f8c100cb820 2023-11-15T02:58:03.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.855+0000 7f8c04ff9700 1 -- 172.21.15.121:0/1057746349 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00012ad0 con 0x7f8c100cb820 2023-11-15T02:58:03.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.855+0000 7f8c16adb700 1 -- 172.21.15.121:0/1057746349 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c100b0930 con 0x7f8c100cb820 2023-11-15T02:58:03.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.855+0000 7f8c16adb700 1 -- 172.21.15.121:0/1057746349 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1015c6b0 con 0x7f8c100cb820 2023-11-15T02:58:03.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.855+0000 7f8c04ff9700 1 -- 172.21.15.121:0/1057746349 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8c00013480 con 0x7f8c100cb820 2023-11-15T02:58:03.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.859+0000 7f8c04ff9700 1 --2- 172.21.15.121:0/1057746349 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8bfc05a510 0x7f8bfc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:03.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.859+0000 7f8c04ff9700 1 -- 172.21.15.121:0/1057746349 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f8c000745c0 con 0x7f8c100cb820 2023-11-15T02:58:03.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.859+0000 7f8c14877700 1 --2- 172.21.15.121:0/1057746349 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8bfc05a510 0x7f8bfc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:03.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.859+0000 7f8c14877700 1 --2- 172.21.15.121:0/1057746349 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8bfc05a510 0x7f8bfc05c9d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f8c0c009130 tx=0x7f8c0c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:03.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:03.863+0000 7f8c04ff9700 1 -- 172.21.15.121:0/1057746349 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8c00047c30 con 0x7f8c100cb820 2023-11-15T02:58:04.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.067+0000 7f8c16adb700 1 -- 172.21.15.121:0/1057746349 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c1006c150 con 0x7f8c100cb820 2023-11-15T02:58:04.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.071+0000 7f8c04ff9700 1 -- 172.21.15.121:0/1057746349 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4502 (secure 0 0 0) 0x7f8c0004b250 con 0x7f8c100cb820 2023-11-15T02:58:04.076 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:58:04.076 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2544127447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2544127447},{"type":"v1","addr":"172.21.15.190:6833","nonce":2544127447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1815834193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1815834193},{"type":"v1","addr":"172.21.15.190:6835","nonce":1815834193}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4950,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3229917854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3229917854},{"type":"v1","addr":"172.21.15.121:6835","nonce":3229917854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:57:44.541243+0000","modified":"2023-11-15T02:58:03.057548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5178":{"gid":5178,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.121:6837/3305580606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3305580606},{"type":"v1","addr":"172.21.15.121:6837","nonce":3305580606}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-15T02:58:04.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.071+0000 7f8c16adb700 1 -- 172.21.15.121:0/1057746349 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8bfc05a510 msgr2=0x7f8bfc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:04.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.071+0000 7f8c16adb700 1 --2- 172.21.15.121:0/1057746349 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8bfc05a510 0x7f8bfc05c9d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f8c0c009130 tx=0x7f8c0c017000).stop 2023-11-15T02:58:04.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.071+0000 7f8c16adb700 1 -- 172.21.15.121:0/1057746349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c100cb820 msgr2=0x7f8c100c3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:04.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.071+0000 7f8c16adb700 1 --2- 172.21.15.121:0/1057746349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c100cb820 0x7f8c100c3670 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f8c0000d890 tx=0x7f8c00008040).stop 2023-11-15T02:58:04.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.071+0000 7f8c16adb700 1 -- 172.21.15.121:0/1057746349 shutdown_connections 2023-11-15T02:58:04.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.071+0000 7f8c16adb700 1 --2- 172.21.15.121:0/1057746349 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8bfc05a510 0x7f8bfc05c9d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.071+0000 7f8c16adb700 1 --2- 172.21.15.121:0/1057746349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c1015c930 0x7f8c100c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.071+0000 7f8c16adb700 1 --2- 172.21.15.121:0/1057746349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c100cb820 0x7f8c100c3670 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.071+0000 7f8c16adb700 1 --2- 172.21.15.121:0/1057746349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c100a9ff0 0x7f8c10159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.071+0000 7f8c16adb700 1 -- 172.21.15.121:0/1057746349 >> 172.21.15.121:0/1057746349 conn(0x7f8c10058c50 msgr2=0x7f8c10157bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:04.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.071+0000 7f8c16adb700 1 -- 172.21.15.121:0/1057746349 shutdown_connections 2023-11-15T02:58:04.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.071+0000 7f8c16adb700 1 -- 172.21.15.121:0/1057746349 wait complete. 2023-11-15T02:58:04.081 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 22 2023-11-15T02:58:04.502 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T02:57:44.541243+0000', 'modified': '2023-11-15T02:58:03.057548+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 34, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5178}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5178': {'gid': 5178, 'name': 'c', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.121:6837/3305580606', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6836', 'nonce': 3305580606}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 3305580606}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T02:58:04.503 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T02:58:04.503 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-15T02:58:04.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.663+0000 7fbb719b9700 1 Processor -- start 2023-11-15T02:58:04.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.663+0000 7fbb719b9700 1 -- start start 2023-11-15T02:58:04.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb719b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb6c0c8e70 0x7fbb6c0c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb719b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6c0a9b40 con 0x7fbb6c0c43b0 2023-11-15T02:58:04.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb719b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6c0a8040 con 0x7fbb6c156f10 2023-11-15T02:58:04.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb719b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0c3730 con 0x7fbb6c0c8e70 2023-11-15T02:58:04.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb711b8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fbb6c0c43b0 0x7fbb6c0a9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58392/0 (socket says 172.21.15.121:58392) 2023-11-15T02:58:04.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb711b8700 1 -- 172.21.15.121:0/2393150036 learned_addr learned my addr 172.21.15.121:0/2393150036 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:04.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb6bfff700 1 --2- 172.21.15.121:0/2393150036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb6c0c8e70 0x7fbb6c0c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb6b7fe700 1 -- 172.21.15.121:0/2393150036 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293798802 0 0) 0x7fbb6c0a9b40 con 0x7fbb6c0c43b0 2023-11-15T02:58:04.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb6b7fe700 1 -- 172.21.15.121:0/2393150036 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb5c0033f0 con 0x7fbb6c0c43b0 2023-11-15T02:58:04.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb6b7fe700 1 -- 172.21.15.121:0/2393150036 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3270106632 0 0) 0x7fbb5c0033f0 con 0x7fbb6c0c43b0 2023-11-15T02:58:04.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb6b7fe700 1 -- 172.21.15.121:0/2393150036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb6c0c8e70 msgr2=0x7fbb6c0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:04.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb6b7fe700 1 --2- 172.21.15.121:0/2393150036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb6c0c8e70 0x7fbb6c0c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb6b7fe700 1 -- 172.21.15.121:0/2393150036 >> v1:172.21.15.190:6789/0 conn(0x7fbb6c156f10 legacy=0x7fbb6c0bd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:04.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb6b7fe700 1 -- 172.21.15.121:0/2393150036 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c0c3910 con 0x7fbb6c0c43b0 2023-11-15T02:58:04.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb6b7fe700 1 -- 172.21.15.121:0/2393150036 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fbb640017f0 con 0x7fbb6c0c43b0 2023-11-15T02:58:04.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb6b7fe700 1 -- 172.21.15.121:0/2393150036 >> v1:172.21.15.121:6789/0 conn(0x7fbb6c0c43b0 legacy=0x7fbb6c0a9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:04.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb6b7fe700 1 --2- 172.21.15.121:0/2393150036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb5c003910 0x7fbb5c003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb6b7fe700 1 --2- 172.21.15.121:0/2393150036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb6c0c8e70 0x7fbb5c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb6b7fe700 1 --2- 172.21.15.121:0/2393150036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb5c004b80 0x7fbb5c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb6b7fe700 1 -- 172.21.15.121:0/2393150036 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0c3730 con 0x7fbb5c003910 2023-11-15T02:58:04.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb6b7fe700 1 -- 172.21.15.121:0/2393150036 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c00b8e0 con 0x7fbb5c004b80 2023-11-15T02:58:04.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb6b7fe700 1 -- 172.21.15.121:0/2393150036 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c00ba50 con 0x7fbb6c0c8e70 2023-11-15T02:58:04.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb709b7700 1 --2- 172.21.15.121:0/2393150036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb5c003910 0x7fbb5c003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb711b8700 1 --2- 172.21.15.121:0/2393150036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb5c004b80 0x7fbb5c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.667+0000 7fbb6bfff700 1 --2- 172.21.15.121:0/2393150036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb6c0c8e70 0x7fbb5c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.671+0000 7fbb709b7700 1 -- 172.21.15.121:0/2393150036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb6c0c8e70 msgr2=0x7fbb5c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:04.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.671+0000 7fbb709b7700 1 --2- 172.21.15.121:0/2393150036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb6c0c8e70 0x7fbb5c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.671+0000 7fbb709b7700 1 -- 172.21.15.121:0/2393150036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb5c004b80 msgr2=0x7fbb5c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:04.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.671+0000 7fbb709b7700 1 --2- 172.21.15.121:0/2393150036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb5c004b80 0x7fbb5c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.671+0000 7fbb709b7700 1 -- 172.21.15.121:0/2393150036 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c0033f0 con 0x7fbb5c003910 2023-11-15T02:58:04.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.671+0000 7fbb709b7700 1 --2- 172.21.15.121:0/2393150036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb5c003910 0x7fbb5c003cf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fbb60001820 tx=0x7fbb60007b90).ready entity=mon.0 client_cookie=62e7adfea50334c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:04.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.671+0000 7fbb6b7fe700 1 -- 172.21.15.121:0/2393150036 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb60007ea0 con 0x7fbb5c003910 2023-11-15T02:58:04.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.671+0000 7fbb6b7fe700 1 -- 172.21.15.121:0/2393150036 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb60012400 con 0x7fbb5c003910 2023-11-15T02:58:04.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.671+0000 7fbb6b7fe700 1 -- 172.21.15.121:0/2393150036 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb600127c0 con 0x7fbb5c003910 2023-11-15T02:58:04.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.671+0000 7fbb719b9700 1 -- 172.21.15.121:0/2393150036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb5c003910 msgr2=0x7fbb5c003cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:04.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.671+0000 7fbb719b9700 1 --2- 172.21.15.121:0/2393150036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb5c003910 0x7fbb5c003cf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fbb60001820 tx=0x7fbb60007b90).stop 2023-11-15T02:58:04.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.671+0000 7fbb711b8700 1 -- 172.21.15.121:0/2393150036 reap_dead start 2023-11-15T02:58:04.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.671+0000 7fbb719b9700 1 -- 172.21.15.121:0/2393150036 shutdown_connections 2023-11-15T02:58:04.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.671+0000 7fbb719b9700 1 -- 172.21.15.121:0/2393150036 >> 172.21.15.121:0/2393150036 conn(0x7fbb6c0684f0 msgr2=0x7fbb6c0c8760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:04.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.671+0000 7fbb719b9700 1 -- 172.21.15.121:0/2393150036 shutdown_connections 2023-11-15T02:58:04.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.671+0000 7fbb719b9700 1 -- 172.21.15.121:0/2393150036 wait complete. 2023-11-15T02:58:04.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb719b9700 1 Processor -- start 2023-11-15T02:58:04.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb719b9700 1 -- start start 2023-11-15T02:58:04.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb719b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb5c003910 0x7fbb6c1500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb719b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb6c1506d0 0x7fbb6c14f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb719b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb6c14fcc0 0x7fbb6c159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb719b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c062190 con 0x7fbb6c1506d0 2023-11-15T02:58:04.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb719b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c158120 con 0x7fbb6c14fcc0 2023-11-15T02:58:04.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb719b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c158400 con 0x7fbb5c003910 2023-11-15T02:58:04.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb6bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb6c1506d0 0x7fbb6c14f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb6bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb6c1506d0 0x7fbb6c14f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49974/0 (socket says 172.21.15.121:49974) 2023-11-15T02:58:04.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb6bfff700 1 -- 172.21.15.121:0/3653249409 learned_addr learned my addr 172.21.15.121:0/3653249409 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:04.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb709b7700 1 --2- 172.21.15.121:0/3653249409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb5c003910 0x7fbb6c1500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb711b8700 1 --2- 172.21.15.121:0/3653249409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb6c14fcc0 0x7fbb6c159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb6bfff700 1 -- 172.21.15.121:0/3653249409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb5c003910 msgr2=0x7fbb6c1500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:04.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb6bfff700 1 --2- 172.21.15.121:0/3653249409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb5c003910 0x7fbb6c1500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb6bfff700 1 -- 172.21.15.121:0/3653249409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb6c14fcc0 msgr2=0x7fbb6c159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:04.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb6bfff700 1 --2- 172.21.15.121:0/3653249409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb6c14fcc0 0x7fbb6c159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb6bfff700 1 -- 172.21.15.121:0/3653249409 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c1586e0 con 0x7fbb6c1506d0 2023-11-15T02:58:04.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb709b7700 1 --2- 172.21.15.121:0/3653249409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb5c003910 0x7fbb6c1500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:04.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.675+0000 7fbb6bfff700 1 --2- 172.21.15.121:0/3653249409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb6c1506d0 0x7fbb6c14f7d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7fbb580116e0 tx=0x7fbb58008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:04.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.679+0000 7fbb68ff9700 1 -- 172.21.15.121:0/3653249409 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58013620 con 0x7fbb6c1506d0 2023-11-15T02:58:04.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.679+0000 7fbb68ff9700 1 -- 172.21.15.121:0/3653249409 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb58013780 con 0x7fbb6c1506d0 2023-11-15T02:58:04.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.679+0000 7fbb719b9700 1 -- 172.21.15.121:0/3653249409 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6c1589c0 con 0x7fbb6c1506d0 2023-11-15T02:58:04.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.679+0000 7fbb68ff9700 1 -- 172.21.15.121:0/3653249409 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58013b00 con 0x7fbb6c1506d0 2023-11-15T02:58:04.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.679+0000 7fbb719b9700 1 -- 172.21.15.121:0/3653249409 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6c159c20 con 0x7fbb6c1506d0 2023-11-15T02:58:04.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.683+0000 7fbb68ff9700 1 -- 172.21.15.121:0/3653249409 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fbb58015480 con 0x7fbb6c1506d0 2023-11-15T02:58:04.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.683+0000 7fbb68ff9700 1 --2- 172.21.15.121:0/3653249409 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbb5005a540 0x7fbb5005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.683+0000 7fbb68ff9700 1 -- 172.21.15.121:0/3653249409 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fbb58074bc0 con 0x7fbb6c1506d0 2023-11-15T02:58:04.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.683+0000 7fbb709b7700 1 --2- 172.21.15.121:0/3653249409 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbb5005a540 0x7fbb5005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.683+0000 7fbb709b7700 1 --2- 172.21.15.121:0/3653249409 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbb5005a540 0x7fbb5005ca00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fbb6c15a6b0 tx=0x7fbb60016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:04.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.683+0000 7fbb7cf92740 1 Processor -- start 2023-11-15T02:58:04.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.683+0000 7fbb7cf92740 1 -- start start 2023-11-15T02:58:04.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb7cf92740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x24b5ad0 0x24dc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb7cf92740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x24b5eb0 0x2502140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb7cf92740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2502630 0x2502ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb7cf92740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2436410 con 0x24b5ad0 2023-11-15T02:58:04.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb7cf92740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x2436550 con 0x24b5eb0 2023-11-15T02:58:04.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb7cf92740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2436a00 con 0x2502630 2023-11-15T02:58:04.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb709b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x24b5ad0 0x24dc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb709b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x24b5ad0 0x24dc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49984/0 (socket says 172.21.15.121:49984) 2023-11-15T02:58:04.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb709b7700 1 -- 172.21.15.121:0/3903693560 learned_addr learned my addr 172.21.15.121:0/3903693560 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:04.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb6bfff700 1 --2- 172.21.15.121:0/3903693560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x24b5eb0 0x2502140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb711b8700 1 --2- 172.21.15.121:0/3903693560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2502630 0x2502ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb709b7700 1 -- 172.21.15.121:0/3903693560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2502630 msgr2=0x2502ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:04.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb709b7700 1 --2- 172.21.15.121:0/3903693560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2502630 0x2502ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb709b7700 1 -- 172.21.15.121:0/3903693560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x24b5eb0 msgr2=0x2502140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:04.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb709b7700 1 --2- 172.21.15.121:0/3903693560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x24b5eb0 0x2502140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb709b7700 1 -- 172.21.15.121:0/3903693560 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x242f730 con 0x24b5ad0 2023-11-15T02:58:04.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb709b7700 1 --2- 172.21.15.121:0/3903693560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x24b5ad0 0x24dc090 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fbb60000e50 tx=0x7fbb6001f000).ready entity=mon.0 client_cookie=94dac4bcf9ebf9dd server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:04.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb55ffb700 1 -- 172.21.15.121:0/3903693560 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb60007ea0 con 0x24b5ad0 2023-11-15T02:58:04.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb55ffb700 1 -- 172.21.15.121:0/3903693560 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb60007850 con 0x24b5ad0 2023-11-15T02:58:04.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.687+0000 7fbb55ffb700 1 -- 172.21.15.121:0/3903693560 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6001f8f0 con 0x24b5ad0 2023-11-15T02:58:04.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.691+0000 7fbb7cf92740 1 -- 172.21.15.121:0/3903693560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x24b5ad0 msgr2=0x24dc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:04.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.691+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/3903693560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x24b5ad0 0x24dc090 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fbb60000e50 tx=0x7fbb6001f000).stop 2023-11-15T02:58:04.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.691+0000 7fbb7cf92740 1 -- 172.21.15.121:0/3903693560 shutdown_connections 2023-11-15T02:58:04.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.691+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/3903693560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2502630 0x2502ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.691+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/3903693560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x24b5eb0 0x2502140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.691+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/3903693560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x24b5ad0 0x24dc090 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.691+0000 7fbb7cf92740 1 -- 172.21.15.121:0/3903693560 >> 172.21.15.121:0/3903693560 conn(0x2294550 msgr2=0x24b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:04.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.691+0000 7fbb7cf92740 1 -- 172.21.15.121:0/3903693560 shutdown_connections 2023-11-15T02:58:04.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.691+0000 7fbb7cf92740 1 -- 172.21.15.121:0/3903693560 wait complete. 2023-11-15T02:58:04.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.691+0000 7fbb7cf92740 1 -- start start 2023-11-15T02:58:04.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.691+0000 7fbb7cf92740 1 Processor -- start 2023-11-15T02:58:04.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb7cf92740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x24b5ad0 0x24af800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb7cf92740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x24b5eb0 0x24ff330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb7cf92740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2502630 0x24ff820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb7cf92740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2401650 con 0x2502630 2023-11-15T02:58:04.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb7cf92740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x24e1140 con 0x24b5ad0 2023-11-15T02:58:04.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb7cf92740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x24f6a30 con 0x24b5eb0 2023-11-15T02:58:04.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb7cf92740 10 client.0 ll_register_callbacks cb 0x246e020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:58:04.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb711b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2502630 0x24ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb711b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2502630 0x24ff820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49994/0 (socket says 172.21.15.121:49994) 2023-11-15T02:58:04.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb711b8700 1 -- 172.21.15.121:0/2334770878 learned_addr learned my addr 172.21.15.121:0/2334770878 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:04.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb709b7700 1 --2- 172.21.15.121:0/2334770878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x24b5ad0 0x24af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb6bfff700 1 --2- 172.21.15.121:0/2334770878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x24b5eb0 0x24ff330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb711b8700 1 -- 172.21.15.121:0/2334770878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x24b5eb0 msgr2=0x24ff330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:04.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb711b8700 1 --2- 172.21.15.121:0/2334770878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x24b5eb0 0x24ff330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb711b8700 1 -- 172.21.15.121:0/2334770878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x24b5ad0 msgr2=0x24af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:04.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb711b8700 1 --2- 172.21.15.121:0/2334770878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x24b5ad0 0x24af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb711b8700 1 -- 172.21.15.121:0/2334770878 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5c0033f0 con 0x2502630 2023-11-15T02:58:04.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb711b8700 1 --2- 172.21.15.121:0/2334770878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2502630 0x24ff820 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fbb6c06a9c0 tx=0x7fbb64001d90).ready entity=mon.0 client_cookie=e29bcb2bba03b7a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:04.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb4affd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:58:04.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb4affd700 1 -- 172.21.15.121:0/2334770878 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64001250 con 0x2502630 2023-11-15T02:58:04.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb4affd700 1 -- 172.21.15.121:0/2334770878 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fbb64015620 con 0x2502630 2023-11-15T02:58:04.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb7cf92740 1 -- 172.21.15.121:0/2334770878 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x242c1a0 con 0x2502630 2023-11-15T02:58:04.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb4affd700 1 -- 172.21.15.121:0/2334770878 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb640159f0 con 0x2502630 2023-11-15T02:58:04.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb7cf92740 10 client.5223 fetch_fsmap learned FSMap version 22 2023-11-15T02:58:04.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb7cf92740 1 -- 172.21.15.121:0/2334770878 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x242a960 con 0x2502630 2023-11-15T02:58:04.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.695+0000 7fbb4affd700 1 -- 172.21.15.121:0/2334770878 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x242c1a0 con 0x2502630 2023-11-15T02:58:04.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb4affd700 1 -- 172.21.15.121:0/2334770878 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 22) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fbb64015b50 con 0x2502630 2023-11-15T02:58:04.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb7cf92740 10 client.5223 fetch_fsmap finished waiting for FSMap version 22 2023-11-15T02:58:04.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb7cf92740 10 client.5223 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-15T02:58:04.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb7cf92740 20 client.5223 populate_metadata read hostname 'smithi121' 2023-11-15T02:58:04.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/2334770878 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x250bf70 0x250c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb7cf92740 4 client.5223 mds_command: new command op to 5178 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T02:58:04.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb7cf92740 1 -- 172.21.15.121:0/2334770878 --> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x24226f0 con 0x250bf70 2023-11-15T02:58:04.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb709b7700 1 --2- 172.21.15.121:0/2334770878 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x250bf70 0x250c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb709b7700 1 --2- 172.21.15.121:0/2334770878 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x250bf70 0x250c350 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=61907719a0b62631 server_cookie=61b091429760b12c in_seq=0 out_seq=0 2023-11-15T02:58:04.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb4affd700 10 client.5223 ms_handle_connect on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:04.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb4affd700 1 -- 172.21.15.121:0/2334770878 <== mds.0 v2:172.21.15.121:6836/3305580606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x24226f0 con 0x250bf70 2023-11-15T02:58:04.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb4affd700 10 client.5223 handle_command_reply: tid=0 2023-11-15T02:58:04.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb7cf92740 1 client.5223 shutdown 2023-11-15T02:58:04.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb7cf92740 1 -- 172.21.15.121:0/2334770878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2502630 msgr2=0x24ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:04.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/2334770878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2502630 0x24ff820 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fbb6c06a9c0 tx=0x7fbb64001d90).stop 2023-11-15T02:58:04.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb7cf92740 1 -- 172.21.15.121:0/2334770878 shutdown_connections 2023-11-15T02:58:04.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/2334770878 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x250bf70 0x250c350 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/2334770878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2502630 0x24ff820 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/2334770878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x24b5eb0 0x24ff330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/2334770878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x24b5ad0 0x24af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb4affd700 0 client.5223 ms_handle_reset on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:04.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb7cf92740 1 -- 172.21.15.121:0/2334770878 >> 172.21.15.121:0/2334770878 conn(0x2294550 msgr2=0x24b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:04.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb7cf92740 1 -- 172.21.15.121:0/2334770878 shutdown_connections 2023-11-15T02:58:04.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb7cf92740 1 -- 172.21.15.121:0/2334770878 wait complete. 2023-11-15T02:58:04.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.699+0000 7fbb7cf92740 20 client.5223 trim_cache size 0 max 16384 2023-11-15T02:58:04.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.719+0000 7fbb7cf92740 1 Processor -- start 2023-11-15T02:58:04.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.719+0000 7fbb7cf92740 1 -- start start 2023-11-15T02:58:04.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb7cf92740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x24b5ad0 0x24df450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb7cf92740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x24b5eb0 0x24ff0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb7cf92740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2502630 0x24f3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb7cf92740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x24dbc90 con 0x24b5eb0 2023-11-15T02:58:04.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb7cf92740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x24fe530 con 0x2502630 2023-11-15T02:58:04.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb7cf92740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x24b03b0 con 0x24b5ad0 2023-11-15T02:58:04.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb6bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x24b5eb0 0x24ff0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb6bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x24b5eb0 0x24ff0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49998/0 (socket says 172.21.15.121:49998) 2023-11-15T02:58:04.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb6bfff700 1 -- 172.21.15.121:0/4175344105 learned_addr learned my addr 172.21.15.121:0/4175344105 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:04.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb711b8700 1 --2- 172.21.15.121:0/4175344105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2502630 0x24f3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb709b7700 1 --2- 172.21.15.121:0/4175344105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x24b5ad0 0x24df450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb6bfff700 1 -- 172.21.15.121:0/4175344105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x24b5ad0 msgr2=0x24df450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:04.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb6bfff700 1 --2- 172.21.15.121:0/4175344105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x24b5ad0 0x24df450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb6bfff700 1 -- 172.21.15.121:0/4175344105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2502630 msgr2=0x24f3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:04.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb6bfff700 1 --2- 172.21.15.121:0/4175344105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2502630 0x24f3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb6bfff700 1 -- 172.21.15.121:0/4175344105 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c0be130 con 0x24b5eb0 2023-11-15T02:58:04.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb709b7700 1 --2- 172.21.15.121:0/4175344105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x24b5ad0 0x24df450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:58:04.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb711b8700 1 --2- 172.21.15.121:0/4175344105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2502630 0x24f3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:58:04.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb6bfff700 1 --2- 172.21.15.121:0/4175344105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x24b5eb0 0x24ff0f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fbb580484b0 tx=0x7fbb58009060).ready entity=mon.0 client_cookie=2c6db4027ad40118 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:04.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb567fc700 1 -- 172.21.15.121:0/4175344105 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb580516f0 con 0x24b5eb0 2023-11-15T02:58:04.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb567fc700 1 -- 172.21.15.121:0/4175344105 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5804a0f0 con 0x24b5eb0 2023-11-15T02:58:04.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb567fc700 1 -- 172.21.15.121:0/4175344105 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5804a350 con 0x24b5eb0 2023-11-15T02:58:04.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb7cf92740 1 -- 172.21.15.121:0/4175344105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x24b5eb0 msgr2=0x24ff0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:04.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/4175344105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x24b5eb0 0x24ff0f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fbb580484b0 tx=0x7fbb58009060).stop 2023-11-15T02:58:04.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb7cf92740 1 -- 172.21.15.121:0/4175344105 shutdown_connections 2023-11-15T02:58:04.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/4175344105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2502630 0x24f3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/4175344105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x24b5eb0 0x24ff0f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/4175344105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x24b5ad0 0x24df450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb7cf92740 1 -- 172.21.15.121:0/4175344105 >> 172.21.15.121:0/4175344105 conn(0x2294550 msgr2=0x24af800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:04.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.723+0000 7fbb7cf92740 1 -- 172.21.15.121:0/4175344105 shutdown_connections 2023-11-15T02:58:04.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.727+0000 7fbb7cf92740 1 -- 172.21.15.121:0/4175344105 wait complete. 2023-11-15T02:58:04.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.727+0000 7fbb7cf92740 1 -- start start 2023-11-15T02:58:04.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.727+0000 7fbb7cf92740 1 Processor -- start 2023-11-15T02:58:04.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb7cf92740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x24b5ad0 0x2566380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb7cf92740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x24b5eb0 0x24ff0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb7cf92740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2502630 0x24ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb7cf92740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x24ffad0 con 0x2502630 2023-11-15T02:58:04.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb7cf92740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x24fb560 con 0x24b5ad0 2023-11-15T02:58:04.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb7cf92740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x24f9d90 con 0x24b5eb0 2023-11-15T02:58:04.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb7cf92740 10 client.0 ll_register_callbacks cb 0x7fbb640072f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:58:04.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb711b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2502630 0x24ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb709b7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x24b5ad0 0x2566380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb6bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x24b5eb0 0x24ff0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb711b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2502630 0x24ff5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50002/0 (socket says 172.21.15.121:50002) 2023-11-15T02:58:04.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb711b8700 1 -- 172.21.15.121:0/2642324458 learned_addr learned my addr 172.21.15.121:0/2642324458 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:04.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb711b8700 1 -- 172.21.15.121:0/2642324458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x24b5eb0 msgr2=0x24ff0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:04.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb711b8700 1 --2- 172.21.15.121:0/2642324458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x24b5eb0 0x24ff0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb711b8700 1 -- 172.21.15.121:0/2642324458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x24b5ad0 msgr2=0x2566380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:04.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb711b8700 1 --2- 172.21.15.121:0/2642324458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x24b5ad0 0x2566380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb711b8700 1 -- 172.21.15.121:0/2642324458 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6c159c20 con 0x2502630 2023-11-15T02:58:04.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb711b8700 1 --2- 172.21.15.121:0/2642324458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2502630 0x24ff5e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fbb64001840 tx=0x7fbb6401e000).ready entity=mon.0 client_cookie=4582af597782ca45 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:04.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb4affd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:58:04.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb4affd700 1 -- 172.21.15.121:0/2642324458 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64028070 con 0x2502630 2023-11-15T02:58:04.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb4affd700 1 -- 172.21.15.121:0/2642324458 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fbb64015620 con 0x2502630 2023-11-15T02:58:04.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb7cf92740 1 -- 172.21.15.121:0/2642324458 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb60007850 con 0x2502630 2023-11-15T02:58:04.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb4affd700 1 -- 172.21.15.121:0/2642324458 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb6402b070 con 0x2502630 2023-11-15T02:58:04.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb7cf92740 10 client.5229 fetch_fsmap learned FSMap version 22 2023-11-15T02:58:04.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb7cf92740 1 -- 172.21.15.121:0/2642324458 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x242a6b0 con 0x2502630 2023-11-15T02:58:04.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb4affd700 1 -- 172.21.15.121:0/2642324458 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb60007850 con 0x2502630 2023-11-15T02:58:04.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb4affd700 1 -- 172.21.15.121:0/2642324458 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 22) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fbb6402e020 con 0x2502630 2023-11-15T02:58:04.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb7cf92740 10 client.5229 fetch_fsmap finished waiting for FSMap version 22 2023-11-15T02:58:04.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb7cf92740 10 client.5229 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-15T02:58:04.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb7cf92740 20 client.5229 populate_metadata read hostname 'smithi121' 2023-11-15T02:58:04.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/2642324458 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x24b66e0 0x24b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb7cf92740 4 client.5229 mds_command: new command op to 5178 tid=0[{"prefix": "status"}] 2023-11-15T02:58:04.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb7cf92740 1 -- 172.21.15.121:0/2642324458 --> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] -- command(tid 0: {"prefix": "status"}) v1 -- 0x22eb4e0 con 0x24b66e0 2023-11-15T02:58:04.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.731+0000 7fbb709b7700 1 --2- 172.21.15.121:0/2642324458 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x24b66e0 0x24b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb4affd700 10 client.5229 ms_handle_connect on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:04.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb709b7700 1 --2- 172.21.15.121:0/2642324458 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x24b66e0 0x24b6ac0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=2e4fb9752ef32d9f server_cookie=32691aca5b18150b in_seq=0 out_seq=0 2023-11-15T02:58:04.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb4affd700 1 -- 172.21.15.121:0/2642324458 <== mds.0 v2:172.21.15.121:6836/3305580606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x22eb4e0 con 0x24b66e0 2023-11-15T02:58:04.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb4affd700 10 client.5229 handle_command_reply: tid=0 2023-11-15T02:58:04.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb7cf92740 1 client.5229 shutdown 2023-11-15T02:58:04.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb7cf92740 1 -- 172.21.15.121:0/2642324458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2502630 msgr2=0x24ff5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:04.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/2642324458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2502630 0x24ff5e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fbb64001840 tx=0x7fbb6401e000).stop 2023-11-15T02:58:04.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb7cf92740 1 -- 172.21.15.121:0/2642324458 shutdown_connections 2023-11-15T02:58:04.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/2642324458 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x24b66e0 0x24b6ac0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/2642324458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2502630 0x24ff5e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/2642324458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x24b5eb0 0x24ff0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb7cf92740 1 --2- 172.21.15.121:0/2642324458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x24b5ad0 0x2566380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb4affd700 0 client.5229 ms_handle_reset on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:04.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb7cf92740 1 -- 172.21.15.121:0/2642324458 >> 172.21.15.121:0/2642324458 conn(0x2294550 msgr2=0x25126e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:04.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb7cf92740 1 -- 172.21.15.121:0/2642324458 shutdown_connections 2023-11-15T02:58:04.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb7cf92740 1 -- 172.21.15.121:0/2642324458 wait complete. 2023-11-15T02:58:04.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb7cf92740 20 client.5229 trim_cache size 0 max 16384 2023-11-15T02:58:04.751 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T02:58:04.751 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T02:58:04.751 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T02:58:04.751 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 5178, 2023-11-15T02:58:04.751 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T02:58:04.752 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T02:58:04.752 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T02:58:04.752 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 4.6977333779999997, 2023-11-15T02:58:04.752 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 22, 2023-11-15T02:58:04.753 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 34, 2023-11-15T02:58:04.753 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 34, 2023-11-15T02:58:04.753 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 5.6488480240000003 2023-11-15T02:58:04.753 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T02:58:04.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb719b9700 1 -- 172.21.15.121:0/3653249409 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbb5005a540 msgr2=0x7fbb5005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:04.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb719b9700 1 --2- 172.21.15.121:0/3653249409 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbb5005a540 0x7fbb5005ca00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fbb6c15a6b0 tx=0x7fbb60016000).stop 2023-11-15T02:58:04.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb719b9700 1 -- 172.21.15.121:0/3653249409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb6c1506d0 msgr2=0x7fbb6c14f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:04.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb719b9700 1 --2- 172.21.15.121:0/3653249409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb6c1506d0 0x7fbb6c14f7d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7fbb580116e0 tx=0x7fbb58008040).stop 2023-11-15T02:58:04.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb719b9700 1 -- 172.21.15.121:0/3653249409 shutdown_connections 2023-11-15T02:58:04.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb719b9700 1 --2- 172.21.15.121:0/3653249409 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbb5005a540 0x7fbb5005ca00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb719b9700 1 --2- 172.21.15.121:0/3653249409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb6c14fcc0 0x7fbb6c159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb719b9700 1 --2- 172.21.15.121:0/3653249409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb6c1506d0 0x7fbb6c14f7d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb719b9700 1 --2- 172.21.15.121:0/3653249409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb5c003910 0x7fbb6c1500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.735+0000 7fbb719b9700 1 -- 172.21.15.121:0/3653249409 >> 172.21.15.121:0/3653249409 conn(0x7fbb6c0684f0 msgr2=0x7fbb6c0bd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:04.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.739+0000 7fbb719b9700 1 -- 172.21.15.121:0/3653249409 shutdown_connections 2023-11-15T02:58:04.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.739+0000 7fbb719b9700 1 -- 172.21.15.121:0/3653249409 wait complete. 2023-11-15T02:58:04.765 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T02:58:04.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.899+0000 7f8b24e39700 1 Processor -- start 2023-11-15T02:58:04.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b24e39700 1 -- start start 2023-11-15T02:58:04.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b24e39700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8b2006a0c0 0x7f8b2006a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b24e39700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8b20068330 0x7f8b20151510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b24e39700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b20151a00 0x7f8b20063aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b24e39700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b20065dc0 con 0x7f8b20068330 2023-11-15T02:58:04.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b24e39700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b201566c0 con 0x7f8b2006a0c0 2023-11-15T02:58:04.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b24e39700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b20155b40 con 0x7f8b20151a00 2023-11-15T02:58:04.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b16ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8b20068330 0x7f8b20151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b1f7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8b2006a0c0 0x7f8b2006a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b1f7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8b2006a0c0 0x7f8b2006a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:38980/0 (socket says 172.21.15.121:38980) 2023-11-15T02:58:04.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b16ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8b20068330 0x7f8b20151510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50010/0 (socket says 172.21.15.121:50010) 2023-11-15T02:58:04.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b1f7fe700 1 -- 172.21.15.121:0/1724540173 learned_addr learned my addr 172.21.15.121:0/1724540173 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:04.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b1ffff700 1 --2- 172.21.15.121:0/1724540173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b20151a00 0x7f8b20063aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b16ffd700 1 -- 172.21.15.121:0/1724540173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b20151a00 msgr2=0x7f8b20063aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:04.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b16ffd700 1 --2- 172.21.15.121:0/1724540173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b20151a00 0x7f8b20063aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b16ffd700 1 -- 172.21.15.121:0/1724540173 >> v2:172.21.15.190:3300/0 conn(0x7f8b2006a0c0 msgr2=0x7f8b2006a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:04.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b16ffd700 1 --2- 172.21.15.121:0/1724540173 >> v2:172.21.15.190:3300/0 conn(0x7f8b2006a0c0 0x7f8b2006a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b16ffd700 1 -- 172.21.15.121:0/1724540173 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b20155e20 con 0x7f8b20068330 2023-11-15T02:58:04.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b16ffd700 1 --2- 172.21.15.121:0/1724540173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b20068330 0x7f8b20151510 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f8b200638a0 tx=0x7f8b08009060).ready entity=mon.0 client_cookie=feb32c734f8dcb48 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:04.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.903+0000 7f8b1effd700 1 -- 172.21.15.121:0/1724540173 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0800d4a0 con 0x7f8b20068330 2023-11-15T02:58:04.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.907+0000 7f8b1effd700 1 -- 172.21.15.121:0/1724540173 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0800d600 con 0x7f8b20068330 2023-11-15T02:58:04.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.907+0000 7f8b1effd700 1 -- 172.21.15.121:0/1724540173 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0800d980 con 0x7f8b20068330 2023-11-15T02:58:04.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.907+0000 7f8b24e39700 1 -- 172.21.15.121:0/1724540173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b20068330 msgr2=0x7f8b20151510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:04.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.907+0000 7f8b24e39700 1 --2- 172.21.15.121:0/1724540173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b20068330 0x7f8b20151510 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f8b200638a0 tx=0x7f8b08009060).stop 2023-11-15T02:58:04.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.907+0000 7f8b24e39700 1 -- 172.21.15.121:0/1724540173 shutdown_connections 2023-11-15T02:58:04.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.907+0000 7f8b24e39700 1 --2- 172.21.15.121:0/1724540173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b20151a00 0x7f8b20063aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.907+0000 7f8b24e39700 1 --2- 172.21.15.121:0/1724540173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b20068330 0x7f8b20151510 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.907+0000 7f8b24e39700 1 --2- 172.21.15.121:0/1724540173 >> v2:172.21.15.190:3300/0 conn(0x7f8b2006a0c0 0x7f8b2006a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.907+0000 7f8b24e39700 1 -- 172.21.15.121:0/1724540173 >> 172.21.15.121:0/1724540173 conn(0x7f8b200677c0 msgr2=0x7f8b200c65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:04.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.907+0000 7f8b24e39700 1 -- 172.21.15.121:0/1724540173 shutdown_connections 2023-11-15T02:58:04.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.907+0000 7f8b24e39700 1 -- 172.21.15.121:0/1724540173 wait complete. 2023-11-15T02:58:04.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b24e39700 1 Processor -- start 2023-11-15T02:58:04.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b24e39700 1 -- start start 2023-11-15T02:58:04.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b24e39700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b20068330 0x7f8b2014c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b24e39700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b2006a0c0 0x7f8b2016bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b24e39700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b20151a00 0x7f8b2016c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b24e39700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2016c780 con 0x7f8b20068330 2023-11-15T02:58:04.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b24e39700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2014fab0 con 0x7f8b2006a0c0 2023-11-15T02:58:04.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b24e39700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2014cab0 con 0x7f8b20151a00 2023-11-15T02:58:04.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b1f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b20068330 0x7f8b2014c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b1f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b20068330 0x7f8b2014c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50020/0 (socket says 172.21.15.121:50020) 2023-11-15T02:58:04.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b1f7fe700 1 -- 172.21.15.121:0/1125950181 learned_addr learned my addr 172.21.15.121:0/1125950181 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:04.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b16ffd700 1 --2- 172.21.15.121:0/1125950181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b2006a0c0 0x7f8b2016bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b1ffff700 1 --2- 172.21.15.121:0/1125950181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b20151a00 0x7f8b2016c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b1f7fe700 1 -- 172.21.15.121:0/1125950181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b20151a00 msgr2=0x7f8b2016c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:04.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b1f7fe700 1 --2- 172.21.15.121:0/1125950181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b20151a00 0x7f8b2016c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b1f7fe700 1 -- 172.21.15.121:0/1125950181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b2006a0c0 msgr2=0x7f8b2016bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:04.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b1f7fe700 1 --2- 172.21.15.121:0/1125950181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b2006a0c0 0x7f8b2016bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:04.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b1f7fe700 1 -- 172.21.15.121:0/1125950181 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b080092a0 con 0x7f8b20068330 2023-11-15T02:58:04.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b16ffd700 1 --2- 172.21.15.121:0/1125950181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b2006a0c0 0x7f8b2016bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:04.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b1f7fe700 1 --2- 172.21.15.121:0/1125950181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b20068330 0x7f8b2014c5c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f8b1000e900 tx=0x7f8b100090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:04.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b17fff700 1 -- 172.21.15.121:0/1125950181 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1000ee40 con 0x7f8b20068330 2023-11-15T02:58:04.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b17fff700 1 -- 172.21.15.121:0/1125950181 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1000c850 con 0x7f8b20068330 2023-11-15T02:58:04.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b17fff700 1 -- 172.21.15.121:0/1125950181 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1000cbd0 con 0x7f8b20068330 2023-11-15T02:58:04.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.911+0000 7f8b24e39700 1 -- 172.21.15.121:0/1125950181 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2014ff40 con 0x7f8b20068330 2023-11-15T02:58:04.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.915+0000 7f8b24e39700 1 -- 172.21.15.121:0/1125950181 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b2014f180 con 0x7f8b20068330 2023-11-15T02:58:04.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.915+0000 7f8b24e39700 1 -- 172.21.15.121:0/1125950181 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b200657e0 con 0x7f8b20068330 2023-11-15T02:58:04.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.915+0000 7f8b17fff700 1 -- 172.21.15.121:0/1125950181 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8b1001b400 con 0x7f8b20068330 2023-11-15T02:58:04.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.919+0000 7f8b17fff700 1 --2- 172.21.15.121:0/1125950181 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8b0005a510 0x7f8b0005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:04.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.919+0000 7f8b17fff700 1 -- 172.21.15.121:0/1125950181 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f8b10074d00 con 0x7f8b20068330 2023-11-15T02:58:04.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.919+0000 7f8b16ffd700 1 --2- 172.21.15.121:0/1125950181 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8b0005a510 0x7f8b0005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:04.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.919+0000 7f8b16ffd700 1 --2- 172.21.15.121:0/1125950181 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8b0005a510 0x7f8b0005c9d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f8b08009fd0 tx=0x7f8b08016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:04.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:04.923+0000 7f8b17fff700 1 -- 172.21.15.121:0/1125950181 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8b200657e0 con 0x7f8b20068330 2023-11-15T02:58:05.045 INFO:tasks.ceph.mds.c.smithi121.stderr:2023-11-15T02:58:05.039+0000 7f7e57a71700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T02:58:05.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:05.123+0000 7f8b24e39700 1 -- 172.21.15.121:0/1125950181 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8b200657e0 con 0x7f8b20068330 2023-11-15T02:58:05.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:05.123+0000 7f8b17fff700 1 -- 172.21.15.121:0/1125950181 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8b200657e0 con 0x7f8b20068330 2023-11-15T02:58:05.128 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T02:58:05.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:05.123+0000 7f8b24e39700 1 -- 172.21.15.121:0/1125950181 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8b0005a510 msgr2=0x7f8b0005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:05.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:05.123+0000 7f8b24e39700 1 --2- 172.21.15.121:0/1125950181 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8b0005a510 0x7f8b0005c9d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f8b08009fd0 tx=0x7f8b08016000).stop 2023-11-15T02:58:05.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:05.123+0000 7f8b24e39700 1 -- 172.21.15.121:0/1125950181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b20068330 msgr2=0x7f8b2014c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:05.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:05.123+0000 7f8b24e39700 1 --2- 172.21.15.121:0/1125950181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b20068330 0x7f8b2014c5c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f8b1000e900 tx=0x7f8b100090f0).stop 2023-11-15T02:58:05.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:05.123+0000 7f8b24e39700 1 -- 172.21.15.121:0/1125950181 shutdown_connections 2023-11-15T02:58:05.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:05.123+0000 7f8b24e39700 1 --2- 172.21.15.121:0/1125950181 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8b0005a510 0x7f8b0005c9d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:05.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:05.123+0000 7f8b24e39700 1 --2- 172.21.15.121:0/1125950181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b20151a00 0x7f8b2016c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:05.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:05.123+0000 7f8b24e39700 1 --2- 172.21.15.121:0/1125950181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b2006a0c0 0x7f8b2016bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:05.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:05.123+0000 7f8b24e39700 1 --2- 172.21.15.121:0/1125950181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b20068330 0x7f8b2014c5c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:05.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:05.123+0000 7f8b24e39700 1 -- 172.21.15.121:0/1125950181 >> 172.21.15.121:0/1125950181 conn(0x7f8b200677c0 msgr2=0x7f8b200c46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:05.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:05.127+0000 7f8b24e39700 1 -- 172.21.15.121:0/1125950181 shutdown_connections 2023-11-15T02:58:05.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:05.127+0000 7f8b24e39700 1 -- 172.21.15.121:0/1125950181 wait complete. 2023-11-15T02:58:05.149 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T02:58:05.150 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T02:58:05.150 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T02:58:05.150 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T02:58:05.151 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T02:58:05.151 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T02:58:05.151 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T02:58:05.152 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T02:58:05.152 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T02:58:05.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:58:05.153 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T02:58:05.173 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T02:58:05.174 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T02:58:05.174 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T02:58:05.174 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:58:05.174 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T02:58:05.174 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:58:05.175 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:58:05.175 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:58:05.175 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T02:58:05.175 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:58:05.176 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:58:05.176 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T02:58:05.176 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:58:05.176 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:58:05.176 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T02:58:05.176 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:58:05.177 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T02:58:05.177 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42355sec preferred_lft 42355sec 2023-11-15T02:58:05.177 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T02:58:05.177 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:58:05.177 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T02:58:05.178 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:58:05.178 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T02:58:05.178 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:58:05.178 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T02:58:05.178 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T02:58:05.179 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T02:58:05.179 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:58:05.179 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T02:58:05.179 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:58:05.179 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T02:58:05.180 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T02:58:05.180 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T02:58:05.180 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:58:05.180 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T02:58:05.180 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T02:58:05.181 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T02:58:05.181 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:58:05.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:58:05.182 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T02:58:05.261 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T02:58:05.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:58:05.264 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T02:58:05.327 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T02:58:05.328 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T02:58:05.328 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T02:58:05.328 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T02:58:05.328 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T02:58:05.329 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T02:58:05.329 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T02:58:05.329 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T02:58:05.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:58:05.330 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T02:58:05.330 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T02:58:05.331 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T02:58:05.331 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T02:58:05.331 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T02:58:05.332 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T02:58:05.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:58:05.472 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T02:58:05.487 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T02:58:05.487 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T02:58:05.488 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T02:58:05.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:58:05.489 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T02:58:05.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:58:05.567 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T02:58:05.627 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T02:58:05.628 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:58:05.750 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:58:05.751 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:58:05.771 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:58:05.771 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:58:05.776 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T02:58:05.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:58:05.777 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T02:58:05.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:58:05.808 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2023-11-15T02:58:05.874 INFO:teuthology.orchestra.run.smithi121.stdout:file_undamaged 2023-11-15T02:58:05.876 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-15T02:58:05.877 DEBUG:teuthology.orchestra.run.smithi121:> import os 2023-11-15T02:58:05.877 DEBUG:teuthology.orchestra.run.smithi121:> import stat 2023-11-15T02:58:05.877 DEBUG:teuthology.orchestra.run.smithi121:> import json 2023-11-15T02:58:05.877 DEBUG:teuthology.orchestra.run.smithi121:> import sys 2023-11-15T02:58:05.878 DEBUG:teuthology.orchestra.run.smithi121:> 2023-11-15T02:58:05.878 DEBUG:teuthology.orchestra.run.smithi121:> try: 2023-11-15T02:58:05.878 DEBUG:teuthology.orchestra.run.smithi121:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2023-11-15T02:58:05.879 DEBUG:teuthology.orchestra.run.smithi121:> except OSError as e: 2023-11-15T02:58:05.879 DEBUG:teuthology.orchestra.run.smithi121:> sys.exit(e.errno) 2023-11-15T02:58:05.879 DEBUG:teuthology.orchestra.run.smithi121:> 2023-11-15T02:58:05.880 DEBUG:teuthology.orchestra.run.smithi121:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-11-15T02:58:05.880 DEBUG:teuthology.orchestra.run.smithi121:> print(json.dumps( 2023-11-15T02:58:05.880 DEBUG:teuthology.orchestra.run.smithi121:> dict([(a, getattr(s, a)) for a in attrs]), 2023-11-15T02:58:05.881 DEBUG:teuthology.orchestra.run.smithi121:> indent=2)) 2023-11-15T02:58:05.881 DEBUG:teuthology.orchestra.run.smithi121:> ' 2023-11-15T02:58:06.192 INFO:teuthology.orchestra.run.smithi121.stderr:daemon-helper: command failed with exit status 2 2023-11-15T02:58:06.208 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-11-15T02:58:06.209 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:58:06.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.415+0000 7fe1d9e79700 1 Processor -- start 2023-11-15T02:58:06.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.415+0000 7fe1d9e79700 1 -- start start 2023-11-15T02:58:06.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d9e79700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe1d4067a60 0x7fe1d4067ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d9e79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1d40bc8f0 0x7fe1d40bcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d9e79700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d40c4c90 con 0x7fe1d40a6b60 2023-11-15T02:58:06.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d9e79700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d40683b0 con 0x7fe1d4067a60 2023-11-15T02:58:06.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d9e79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d40c72d0 con 0x7fe1d40bc8f0 2023-11-15T02:58:06.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d3fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe1d4067a60 0x7fe1d4067ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d3fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe1d4067a60 0x7fe1d4067ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:38994/0 (socket says 172.21.15.121:38994) 2023-11-15T02:58:06.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d3fff700 1 -- 172.21.15.121:0/857192567 learned_addr learned my addr 172.21.15.121:0/857192567 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:06.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d8e77700 1 --1- 172.21.15.121:0/857192567 >> v1:172.21.15.190:6789/0 conn(0x7fe1d40a6b60 0x7fe1d40c6bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:38040/0 (socket says 172.21.15.121:38040) 2023-11-15T02:58:06.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d9678700 1 --2- 172.21.15.121:0/857192567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1d40bc8f0 0x7fe1d40bcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d3fff700 1 -- 172.21.15.121:0/857192567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1d40bc8f0 msgr2=0x7fe1d40bcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:06.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d3fff700 1 --2- 172.21.15.121:0/857192567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1d40bc8f0 0x7fe1d40bcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d3fff700 1 -- 172.21.15.121:0/857192567 >> v1:172.21.15.190:6789/0 conn(0x7fe1d40a6b60 legacy=0x7fe1d40c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:06.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d3fff700 1 -- 172.21.15.121:0/857192567 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d40c4360 con 0x7fe1d4067a60 2023-11-15T02:58:06.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d9678700 1 --2- 172.21.15.121:0/857192567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1d40bc8f0 0x7fe1d40bcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:06.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d3fff700 1 --2- 172.21.15.121:0/857192567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1d4067a60 0x7fe1d4067ec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fe1c0006fd0 tx=0x7fe1c00090f0).ready entity=mon.1 client_cookie=b54a2c44dc1ae218 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:06.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d37fe700 1 -- 172.21.15.121:0/857192567 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000b600 con 0x7fe1d4067a60 2023-11-15T02:58:06.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d37fe700 1 -- 172.21.15.121:0/857192567 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c000b760 con 0x7fe1d4067a60 2023-11-15T02:58:06.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d37fe700 1 -- 172.21.15.121:0/857192567 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000bae0 con 0x7fe1d4067a60 2023-11-15T02:58:06.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d9e79700 1 -- 172.21.15.121:0/857192567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1d4067a60 msgr2=0x7fe1d4067ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:06.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.419+0000 7fe1d9e79700 1 --2- 172.21.15.121:0/857192567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1d4067a60 0x7fe1d4067ec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fe1c0006fd0 tx=0x7fe1c00090f0).stop 2023-11-15T02:58:06.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.423+0000 7fe1d9e79700 1 -- 172.21.15.121:0/857192567 shutdown_connections 2023-11-15T02:58:06.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.423+0000 7fe1d9e79700 1 --2- 172.21.15.121:0/857192567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1d40bc8f0 0x7fe1d40bcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.423+0000 7fe1d9e79700 1 --2- 172.21.15.121:0/857192567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1d4067a60 0x7fe1d4067ec0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.423+0000 7fe1d9e79700 1 -- 172.21.15.121:0/857192567 >> 172.21.15.121:0/857192567 conn(0x7fe1d4069350 msgr2=0x7fe1d4090a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:06.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.423+0000 7fe1d9e79700 1 -- 172.21.15.121:0/857192567 shutdown_connections 2023-11-15T02:58:06.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.423+0000 7fe1d9e79700 1 -- 172.21.15.121:0/857192567 wait complete. 2023-11-15T02:58:06.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.423+0000 7fe1d9e79700 1 Processor -- start 2023-11-15T02:58:06.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.423+0000 7fe1d9e79700 1 -- start start 2023-11-15T02:58:06.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d9e79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1d4067a60 0x7fe1d4163dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d9e79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1d40a6b60 0x7fe1d4150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d9e79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1d40bc8f0 0x7fe1d40c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d9e79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d4155680 con 0x7fe1d4067a60 2023-11-15T02:58:06.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d9e79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d41646a0 con 0x7fe1d40bc8f0 2023-11-15T02:58:06.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d9e79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d4164980 con 0x7fe1d40a6b60 2023-11-15T02:58:06.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d8e77700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1d4067a60 0x7fe1d4163dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d8e77700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1d4067a60 0x7fe1d4163dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50022/0 (socket says 172.21.15.121:50022) 2023-11-15T02:58:06.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d8e77700 1 -- 172.21.15.121:0/3134389963 learned_addr learned my addr 172.21.15.121:0/3134389963 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:06.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d9678700 1 --2- 172.21.15.121:0/3134389963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1d40bc8f0 0x7fe1d40c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d3fff700 1 --2- 172.21.15.121:0/3134389963 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1d40a6b60 0x7fe1d4150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d8e77700 1 -- 172.21.15.121:0/3134389963 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1d40a6b60 msgr2=0x7fe1d4150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:06.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d8e77700 1 --2- 172.21.15.121:0/3134389963 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1d40a6b60 0x7fe1d4150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d8e77700 1 -- 172.21.15.121:0/3134389963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1d40bc8f0 msgr2=0x7fe1d40c54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:06.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d8e77700 1 --2- 172.21.15.121:0/3134389963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1d40bc8f0 0x7fe1d40c54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d8e77700 1 -- 172.21.15.121:0/3134389963 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c0009370 con 0x7fe1d4067a60 2023-11-15T02:58:06.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d8e77700 1 --2- 172.21.15.121:0/3134389963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1d4067a60 0x7fe1d4163dc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fe1c8007fd0 tx=0x7fe1c8007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:06.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d0ff9700 1 -- 172.21.15.121:0/3134389963 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c80072e0 con 0x7fe1d4067a60 2023-11-15T02:58:06.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d0ff9700 1 -- 172.21.15.121:0/3134389963 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c8007440 con 0x7fe1d4067a60 2023-11-15T02:58:06.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d9e79700 1 -- 172.21.15.121:0/3134389963 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d4164c60 con 0x7fe1d4067a60 2023-11-15T02:58:06.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d0ff9700 1 -- 172.21.15.121:0/3134389963 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c80058f0 con 0x7fe1d4067a60 2023-11-15T02:58:06.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.427+0000 7fe1d9e79700 1 -- 172.21.15.121:0/3134389963 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d41642b0 con 0x7fe1d4067a60 2023-11-15T02:58:06.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.431+0000 7fe1d9e79700 1 -- 172.21.15.121:0/3134389963 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d406bcf0 con 0x7fe1d4067a60 2023-11-15T02:58:06.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.431+0000 7fe1d0ff9700 1 -- 172.21.15.121:0/3134389963 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe1c8005ce0 con 0x7fe1d4067a60 2023-11-15T02:58:06.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.431+0000 7fe1d0ff9700 1 --2- 172.21.15.121:0/3134389963 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe1b805a510 0x7fe1b805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.431+0000 7fe1d0ff9700 1 -- 172.21.15.121:0/3134389963 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fe1c8073fa0 con 0x7fe1d4067a60 2023-11-15T02:58:06.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.431+0000 7fe1d3fff700 1 --2- 172.21.15.121:0/3134389963 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe1b805a510 0x7fe1b805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.431+0000 7fe1d3fff700 1 --2- 172.21.15.121:0/3134389963 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe1b805a510 0x7fe1b805c9d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fe1c0009130 tx=0x7fe1c0017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:06.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.439+0000 7fe1d0ff9700 1 -- 172.21.15.121:0/3134389963 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe1c80477d0 con 0x7fe1d4067a60 2023-11-15T02:58:06.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.639+0000 7fe1d9e79700 1 -- 172.21.15.121:0/3134389963 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1d40c38e0 con 0x7fe1d4067a60 2023-11-15T02:58:06.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.639+0000 7fe1d0ff9700 1 -- 172.21.15.121:0/3134389963 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4502 (secure 0 0 0) 0x7fe1c804adf0 con 0x7fe1d4067a60 2023-11-15T02:58:06.646 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:58:06.646 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2544127447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2544127447},{"type":"v1","addr":"172.21.15.190:6833","nonce":2544127447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1815834193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1815834193},{"type":"v1","addr":"172.21.15.190:6835","nonce":1815834193}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4950,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3229917854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3229917854},{"type":"v1","addr":"172.21.15.121:6835","nonce":3229917854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:57:44.541243+0000","modified":"2023-11-15T02:58:03.057548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5178":{"gid":5178,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.121:6837/3305580606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3305580606},{"type":"v1","addr":"172.21.15.121:6837","nonce":3305580606}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-15T02:58:06.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.643+0000 7fe1d9e79700 1 -- 172.21.15.121:0/3134389963 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe1b805a510 msgr2=0x7fe1b805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:06.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.643+0000 7fe1d9e79700 1 --2- 172.21.15.121:0/3134389963 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe1b805a510 0x7fe1b805c9d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fe1c0009130 tx=0x7fe1c0017000).stop 2023-11-15T02:58:06.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.643+0000 7fe1d9e79700 1 -- 172.21.15.121:0/3134389963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1d4067a60 msgr2=0x7fe1d4163dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:06.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.643+0000 7fe1d9e79700 1 --2- 172.21.15.121:0/3134389963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1d4067a60 0x7fe1d4163dc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fe1c8007fd0 tx=0x7fe1c8007df0).stop 2023-11-15T02:58:06.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.643+0000 7fe1d9e79700 1 -- 172.21.15.121:0/3134389963 shutdown_connections 2023-11-15T02:58:06.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.643+0000 7fe1d9e79700 1 --2- 172.21.15.121:0/3134389963 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe1b805a510 0x7fe1b805c9d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.643+0000 7fe1d9e79700 1 --2- 172.21.15.121:0/3134389963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1d40bc8f0 0x7fe1d40c54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.643+0000 7fe1d9e79700 1 --2- 172.21.15.121:0/3134389963 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1d40a6b60 0x7fe1d4150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.643+0000 7fe1d9e79700 1 --2- 172.21.15.121:0/3134389963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1d4067a60 0x7fe1d4163dc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.643+0000 7fe1d9e79700 1 -- 172.21.15.121:0/3134389963 >> 172.21.15.121:0/3134389963 conn(0x7fe1d4069350 msgr2=0x7fe1d4153440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:06.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.643+0000 7fe1d9e79700 1 -- 172.21.15.121:0/3134389963 shutdown_connections 2023-11-15T02:58:06.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.643+0000 7fe1d9e79700 1 -- 172.21.15.121:0/3134389963 wait complete. 2023-11-15T02:58:06.651 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 22 2023-11-15T02:58:06.670 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2023-11-15T02:58:06.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.831+0000 7fc523fff700 1 Processor -- start 2023-11-15T02:58:06.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.831+0000 7fc523fff700 1 -- start start 2023-11-15T02:58:06.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc523fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc5240c15c0 0x7fc5240c2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc523fff700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5240657f0 con 0x7fc5240a8960 2023-11-15T02:58:06.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc523fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5240c1b60 con 0x7fc5240a8d40 2023-11-15T02:58:06.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc523fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5241583d0 con 0x7fc5240c15c0 2023-11-15T02:58:06.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc522ffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc5240a8960 0x7fc52415c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58406/0 (socket says 172.21.15.121:58406) 2023-11-15T02:58:06.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc522ffd700 1 -- 172.21.15.121:0/2078307093 learned_addr learned my addr 172.21.15.121:0/2078307093 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:06.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc5237fe700 1 --2- 172.21.15.121:0/2078307093 >> v2:172.21.15.190:3300/0 conn(0x7fc5240c15c0 0x7fc5240c2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc5227fc700 1 -- 172.21.15.121:0/2078307093 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 575679556 0 0) 0x7fc5240657f0 con 0x7fc5240a8960 2023-11-15T02:58:06.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc5227fc700 1 -- 172.21.15.121:0/2078307093 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5100033f0 con 0x7fc5240a8960 2023-11-15T02:58:06.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc5227fc700 1 -- 172.21.15.121:0/2078307093 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277589879 0 0) 0x7fc5240c1b60 con 0x7fc5240a8d40 2023-11-15T02:58:06.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc5227fc700 1 -- 172.21.15.121:0/2078307093 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5240657f0 con 0x7fc5240a8d40 2023-11-15T02:58:06.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc5227fc700 1 -- 172.21.15.121:0/2078307093 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3983293312 0 0) 0x7fc5100033f0 con 0x7fc5240a8960 2023-11-15T02:58:06.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc5227fc700 1 -- 172.21.15.121:0/2078307093 >> v2:172.21.15.190:3300/0 conn(0x7fc5240c15c0 msgr2=0x7fc5240c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:06.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc5227fc700 1 --2- 172.21.15.121:0/2078307093 >> v2:172.21.15.190:3300/0 conn(0x7fc5240c15c0 0x7fc5240c2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc5227fc700 1 -- 172.21.15.121:0/2078307093 >> v1:172.21.15.190:6789/0 conn(0x7fc5240a8d40 legacy=0x7fc524066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:06.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc5227fc700 1 -- 172.21.15.121:0/2078307093 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc524065a40 con 0x7fc5240a8960 2023-11-15T02:58:06.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc5227fc700 1 -- 172.21.15.121:0/2078307093 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fc514001470 con 0x7fc5240a8960 2023-11-15T02:58:06.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc5227fc700 1 -- 172.21.15.121:0/2078307093 >> v1:172.21.15.121:6789/0 conn(0x7fc5240a8960 legacy=0x7fc52415c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:06.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc5227fc700 1 --2- 172.21.15.121:0/2078307093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc510003a70 0x7fc510003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.835+0000 7fc5227fc700 1 --2- 172.21.15.121:0/2078307093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc510004690 0x7fc510004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc5227fc700 1 --2- 172.21.15.121:0/2078307093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc510005000 0x7fc51000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc5227fc700 1 -- 172.21.15.121:0/2078307093 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5241583d0 con 0x7fc510005000 2023-11-15T02:58:06.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc5227fc700 1 -- 172.21.15.121:0/2078307093 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51000bf60 con 0x7fc510004690 2023-11-15T02:58:06.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc5227fc700 1 -- 172.21.15.121:0/2078307093 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51000c0d0 con 0x7fc510003a70 2023-11-15T02:58:06.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc5237fe700 1 --2- 172.21.15.121:0/2078307093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc510005000 0x7fc51000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc522ffd700 1 --2- 172.21.15.121:0/2078307093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc510003a70 0x7fc510003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc51a7fc700 1 --2- 172.21.15.121:0/2078307093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc510004690 0x7fc510004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc5237fe700 1 -- 172.21.15.121:0/2078307093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc510003a70 msgr2=0x7fc510003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:06.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc5237fe700 1 --2- 172.21.15.121:0/2078307093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc510003a70 0x7fc510003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc5237fe700 1 -- 172.21.15.121:0/2078307093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc510004690 msgr2=0x7fc510004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:06.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc5237fe700 1 --2- 172.21.15.121:0/2078307093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc510004690 0x7fc510004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc5237fe700 1 -- 172.21.15.121:0/2078307093 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5100033f0 con 0x7fc510005000 2023-11-15T02:58:06.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc5237fe700 1 -- 172.21.15.121:0/2078307093 reap_dead start 2023-11-15T02:58:06.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc5237fe700 1 --2- 172.21.15.121:0/2078307093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc510005000 0x7fc51000b8e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fc51c009a80 tx=0x7fc51c009060).ready entity=mon.0 client_cookie=1fed991d5008bd6e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:06.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc5227fc700 1 -- 172.21.15.121:0/2078307093 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c009430 con 0x7fc510005000 2023-11-15T02:58:06.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc5227fc700 1 -- 172.21.15.121:0/2078307093 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51c00e640 con 0x7fc510005000 2023-11-15T02:58:06.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc5227fc700 1 -- 172.21.15.121:0/2078307093 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00e9c0 con 0x7fc510005000 2023-11-15T02:58:06.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc523fff700 1 -- 172.21.15.121:0/2078307093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc510005000 msgr2=0x7fc51000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:06.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc523fff700 1 --2- 172.21.15.121:0/2078307093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc510005000 0x7fc51000b8e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fc51c009a80 tx=0x7fc51c009060).stop 2023-11-15T02:58:06.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc523fff700 1 -- 172.21.15.121:0/2078307093 shutdown_connections 2023-11-15T02:58:06.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc523fff700 1 --2- 172.21.15.121:0/2078307093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc510005000 0x7fc51000b8e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc523fff700 1 -- 172.21.15.121:0/2078307093 >> 172.21.15.121:0/2078307093 conn(0x7fc524067a60 msgr2=0x7fc524067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:06.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc523fff700 1 -- 172.21.15.121:0/2078307093 shutdown_connections 2023-11-15T02:58:06.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.839+0000 7fc523fff700 1 -- 172.21.15.121:0/2078307093 wait complete. 2023-11-15T02:58:06.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.843+0000 7fc523fff700 1 Processor -- start 2023-11-15T02:58:06.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.843+0000 7fc523fff700 1 -- start start 2023-11-15T02:58:06.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc523fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc510005000 0x7fc524155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc523fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5240bb570 0x7fc5240bb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc523fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5240ab320 0x7fc5240ab760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc523fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5240bbee0 con 0x7fc5240bb570 2023-11-15T02:58:06.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc523fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5240bccf0 con 0x7fc5240ab320 2023-11-15T02:58:06.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc523fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52415a1d0 con 0x7fc510005000 2023-11-15T02:58:06.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc51a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5240bb570 0x7fc5240bb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc51a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5240bb570 0x7fc5240bb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50046/0 (socket says 172.21.15.121:50046) 2023-11-15T02:58:06.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc51a7fc700 1 -- 172.21.15.121:0/151257301 learned_addr learned my addr 172.21.15.121:0/151257301 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:06.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc5237fe700 1 --2- 172.21.15.121:0/151257301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5240ab320 0x7fc5240ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc522ffd700 1 --2- 172.21.15.121:0/151257301 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc510005000 0x7fc524155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc51a7fc700 1 -- 172.21.15.121:0/151257301 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc510005000 msgr2=0x7fc524155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:06.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc51a7fc700 1 --2- 172.21.15.121:0/151257301 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc510005000 0x7fc524155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc51a7fc700 1 -- 172.21.15.121:0/151257301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5240ab320 msgr2=0x7fc5240ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:06.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc51a7fc700 1 --2- 172.21.15.121:0/151257301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5240ab320 0x7fc5240ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc51a7fc700 1 -- 172.21.15.121:0/151257301 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5240abc50 con 0x7fc5240bb570 2023-11-15T02:58:06.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc51a7fc700 1 --2- 172.21.15.121:0/151257301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5240bb570 0x7fc5240bb9f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fc50c00e860 tx=0x7fc50c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:06.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc51b7fe700 1 -- 172.21.15.121:0/151257301 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c003320 con 0x7fc5240bb570 2023-11-15T02:58:06.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc51b7fe700 1 -- 172.21.15.121:0/151257301 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50c010040 con 0x7fc5240bb570 2023-11-15T02:58:06.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc523fff700 1 -- 172.21.15.121:0/151257301 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc524155570 con 0x7fc5240bb570 2023-11-15T02:58:06.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc51b7fe700 1 -- 172.21.15.121:0/151257301 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0016e0 con 0x7fc5240bb570 2023-11-15T02:58:06.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.847+0000 7fc523fff700 1 -- 172.21.15.121:0/151257301 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5240bc0e0 con 0x7fc5240bb570 2023-11-15T02:58:06.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.851+0000 7fc51b7fe700 1 -- 172.21.15.121:0/151257301 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc50c001a50 con 0x7fc5240bb570 2023-11-15T02:58:06.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.851+0000 7fc51b7fe700 1 --2- 172.21.15.121:0/151257301 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc50405a510 0x7fc50405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.851+0000 7fc51b7fe700 1 -- 172.21.15.121:0/151257301 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fc50c0747c0 con 0x7fc5240bb570 2023-11-15T02:58:06.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.851+0000 7fc522ffd700 1 --2- 172.21.15.121:0/151257301 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc50405a510 0x7fc50405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc522ffd700 1 --2- 172.21.15.121:0/151257301 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc50405a510 0x7fc50405c9d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fc514000e10 tx=0x7fc514009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:06.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc533d5c740 1 Processor -- start 2023-11-15T02:58:06.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc533d5c740 1 -- start start 2023-11-15T02:58:06.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc533d5c740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1b27810 0x1b27bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc533d5c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b2b810 0x1ae3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc533d5c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ade110 0x1ade590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc533d5c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a1f3f0 con 0x1b2b810 2023-11-15T02:58:06.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc533d5c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a42580 con 0x1ade110 2023-11-15T02:58:06.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc533d5c740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a0bd50 con 0x1b27810 2023-11-15T02:58:06.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc51a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b2b810 0x1ae3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc51a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b2b810 0x1ae3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50054/0 (socket says 172.21.15.121:50054) 2023-11-15T02:58:06.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc51a7fc700 1 -- 172.21.15.121:0/2899143003 learned_addr learned my addr 172.21.15.121:0/2899143003 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:06.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc522ffd700 1 --2- 172.21.15.121:0/2899143003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1b27810 0x1b27bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc5237fe700 1 --2- 172.21.15.121:0/2899143003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ade110 0x1ade590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc51a7fc700 1 -- 172.21.15.121:0/2899143003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1b27810 msgr2=0x1b27bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:06.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc51a7fc700 1 --2- 172.21.15.121:0/2899143003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1b27810 0x1b27bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc51a7fc700 1 -- 172.21.15.121:0/2899143003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ade110 msgr2=0x1ade590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:06.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc51a7fc700 1 --2- 172.21.15.121:0/2899143003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ade110 0x1ade590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc51a7fc700 1 -- 172.21.15.121:0/2899143003 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1a5f830 con 0x1b2b810 2023-11-15T02:58:06.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc51a7fc700 1 --2- 172.21.15.121:0/2899143003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b2b810 0x1ae3310 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fc50c0036d0 tx=0x7fc50c011000).ready entity=mon.0 client_cookie=b858865d965b79cf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:06.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc503fff700 1 -- 172.21.15.121:0/2899143003 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c050070 con 0x1b2b810 2023-11-15T02:58:06.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc503fff700 1 -- 172.21.15.121:0/2899143003 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50c072e90 con 0x1b2b810 2023-11-15T02:58:06.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.855+0000 7fc503fff700 1 -- 172.21.15.121:0/2899143003 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c051230 con 0x1b2b810 2023-11-15T02:58:06.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.859+0000 7fc533d5c740 1 -- 172.21.15.121:0/2899143003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b2b810 msgr2=0x1ae3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:06.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.859+0000 7fc533d5c740 1 --2- 172.21.15.121:0/2899143003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b2b810 0x1ae3310 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fc50c0036d0 tx=0x7fc50c011000).stop 2023-11-15T02:58:06.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.859+0000 7fc533d5c740 1 -- 172.21.15.121:0/2899143003 shutdown_connections 2023-11-15T02:58:06.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.859+0000 7fc533d5c740 1 --2- 172.21.15.121:0/2899143003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ade110 0x1ade590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.859+0000 7fc533d5c740 1 --2- 172.21.15.121:0/2899143003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b2b810 0x1ae3310 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.859+0000 7fc533d5c740 1 --2- 172.21.15.121:0/2899143003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1b27810 0x1b27bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.859+0000 7fc533d5c740 1 -- 172.21.15.121:0/2899143003 >> 172.21.15.121:0/2899143003 conn(0x18c4790 msgr2=0x1b2bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:06.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.859+0000 7fc533d5c740 1 -- 172.21.15.121:0/2899143003 shutdown_connections 2023-11-15T02:58:06.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.859+0000 7fc533d5c740 1 -- 172.21.15.121:0/2899143003 wait complete. 2023-11-15T02:58:06.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.859+0000 7fc533d5c740 1 -- start start 2023-11-15T02:58:06.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.859+0000 7fc533d5c740 1 Processor -- start 2023-11-15T02:58:06.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc533d5c740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ade110 0x1b25c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc533d5c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1b27810 0x1b38880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc533d5c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b2b810 0x1b263f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc533d5c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b0c8a0 con 0x1b2b810 2023-11-15T02:58:06.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc533d5c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b29330 con 0x1b27810 2023-11-15T02:58:06.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc533d5c740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b28720 con 0x1ade110 2023-11-15T02:58:06.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc533d5c740 10 client.0 ll_register_callbacks cb 0x1a5c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:58:06.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc5237fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b2b810 0x1b263f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc5237fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b2b810 0x1b263f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50056/0 (socket says 172.21.15.121:50056) 2023-11-15T02:58:06.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc5237fe700 1 -- 172.21.15.121:0/1153652589 learned_addr learned my addr 172.21.15.121:0/1153652589 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:06.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc51a7fc700 1 --2- 172.21.15.121:0/1153652589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1b27810 0x1b38880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc522ffd700 1 --2- 172.21.15.121:0/1153652589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ade110 0x1b25c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc5237fe700 1 -- 172.21.15.121:0/1153652589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ade110 msgr2=0x1b25c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:06.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc5237fe700 1 --2- 172.21.15.121:0/1153652589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ade110 0x1b25c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc5237fe700 1 -- 172.21.15.121:0/1153652589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1b27810 msgr2=0x1b38880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:06.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc5237fe700 1 --2- 172.21.15.121:0/1153652589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1b27810 0x1b38880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc5237fe700 1 -- 172.21.15.121:0/1153652589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5240bc0e0 con 0x1b2b810 2023-11-15T02:58:06.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc5237fe700 1 --2- 172.21.15.121:0/1153652589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b2b810 0x1b263f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7fc5240c7620 tx=0x7fc51c019000).ready entity=mon.0 client_cookie=ebb4da9d2e294c4b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:06.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc5017fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:58:06.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc5017fa700 1 -- 172.21.15.121:0/1153652589 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00e9c0 con 0x1b2b810 2023-11-15T02:58:06.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc5017fa700 1 -- 172.21.15.121:0/1153652589 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fc51c028cf0 con 0x1b2b810 2023-11-15T02:58:06.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc533d5c740 1 -- 172.21.15.121:0/1153652589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1a5c2a0 con 0x1b2b810 2023-11-15T02:58:06.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc5017fa700 1 -- 172.21.15.121:0/1153652589 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc51c01c660 con 0x1b2b810 2023-11-15T02:58:06.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc533d5c740 10 client.5253 fetch_fsmap learned FSMap version 22 2023-11-15T02:58:06.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc533d5c740 1 -- 172.21.15.121:0/1153652589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x1a5aa60 con 0x1b2b810 2023-11-15T02:58:06.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc5017fa700 1 -- 172.21.15.121:0/1153652589 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1a5c2a0 con 0x1b2b810 2023-11-15T02:58:06.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc5017fa700 1 -- 172.21.15.121:0/1153652589 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 22) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc51c011070 con 0x1b2b810 2023-11-15T02:58:06.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc533d5c740 10 client.5253 fetch_fsmap finished waiting for FSMap version 22 2023-11-15T02:58:06.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.863+0000 7fc533d5c740 10 client.5253 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-15T02:58:06.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc533d5c740 20 client.5253 populate_metadata read hostname 'smithi121' 2023-11-15T02:58:06.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc533d5c740 1 --2- 172.21.15.121:0/1153652589 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x1b3b3d0 0x1b45610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc533d5c740 4 client.5253 mds_command: new command op to 5178 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T02:58:06.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc533d5c740 1 -- 172.21.15.121:0/1153652589 --> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1a527f0 con 0x1b3b3d0 2023-11-15T02:58:06.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc522ffd700 1 --2- 172.21.15.121:0/1153652589 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x1b3b3d0 0x1b45610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc522ffd700 1 --2- 172.21.15.121:0/1153652589 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x1b3b3d0 0x1b45610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=a6507b344251f0d1 server_cookie=9258eeb34fdd3c0c in_seq=0 out_seq=0 2023-11-15T02:58:06.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc5017fa700 10 client.5253 ms_handle_connect on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:06.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc5017fa700 1 -- 172.21.15.121:0/1153652589 <== mds.0 v2:172.21.15.121:6836/3305580606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x1a527f0 con 0x1b3b3d0 2023-11-15T02:58:06.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc5017fa700 10 client.5253 handle_command_reply: tid=0 2023-11-15T02:58:06.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc533d5c740 1 client.5253 shutdown 2023-11-15T02:58:06.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc533d5c740 1 -- 172.21.15.121:0/1153652589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b2b810 msgr2=0x1b263f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:06.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc533d5c740 1 --2- 172.21.15.121:0/1153652589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b2b810 0x1b263f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7fc5240c7620 tx=0x7fc51c019000).stop 2023-11-15T02:58:06.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc533d5c740 1 -- 172.21.15.121:0/1153652589 shutdown_connections 2023-11-15T02:58:06.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc533d5c740 1 --2- 172.21.15.121:0/1153652589 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x1b3b3d0 0x1b45610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc533d5c740 1 --2- 172.21.15.121:0/1153652589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b2b810 0x1b263f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc533d5c740 1 --2- 172.21.15.121:0/1153652589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1b27810 0x1b38880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc533d5c740 1 --2- 172.21.15.121:0/1153652589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ade110 0x1b25c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc5017fa700 0 client.5253 ms_handle_reset on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:06.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc533d5c740 1 -- 172.21.15.121:0/1153652589 >> 172.21.15.121:0/1153652589 conn(0x18c4790 msgr2=0x1b0a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:06.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc533d5c740 1 -- 172.21.15.121:0/1153652589 shutdown_connections 2023-11-15T02:58:06.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc533d5c740 1 -- 172.21.15.121:0/1153652589 wait complete. 2023-11-15T02:58:06.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.867+0000 7fc533d5c740 20 client.5253 trim_cache size 0 max 16384 2023-11-15T02:58:06.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.883+0000 7fc533d5c740 1 Processor -- start 2023-11-15T02:58:06.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.883+0000 7fc533d5c740 1 -- start start 2023-11-15T02:58:06.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc533d5c740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ade110 0x1ae4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc533d5c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b27810 0x1ae2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc533d5c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1b2b810 0x1b0a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc533d5c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ae4370 con 0x1b27810 2023-11-15T02:58:06.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc533d5c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b0e700 con 0x1b2b810 2023-11-15T02:58:06.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc533d5c740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b29a40 con 0x1ade110 2023-11-15T02:58:06.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc51a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b27810 0x1ae2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc51a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b27810 0x1ae2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50064/0 (socket says 172.21.15.121:50064) 2023-11-15T02:58:06.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc51a7fc700 1 -- 172.21.15.121:0/3653446514 learned_addr learned my addr 172.21.15.121:0/3653446514 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:06.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc522ffd700 1 --2- 172.21.15.121:0/3653446514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ade110 0x1ae4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc5237fe700 1 --2- 172.21.15.121:0/3653446514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1b2b810 0x1b0a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc51a7fc700 1 -- 172.21.15.121:0/3653446514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ade110 msgr2=0x1ae4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:06.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc51a7fc700 1 --2- 172.21.15.121:0/3653446514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ade110 0x1ae4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc51a7fc700 1 -- 172.21.15.121:0/3653446514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1b2b810 msgr2=0x1b0a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:06.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc51a7fc700 1 --2- 172.21.15.121:0/3653446514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1b2b810 0x1b0a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc51a7fc700 1 -- 172.21.15.121:0/3653446514 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5240681a0 con 0x1b27810 2023-11-15T02:58:06.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc51a7fc700 1 --2- 172.21.15.121:0/3653446514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b27810 0x1ae2670 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fc50c049ea0 tx=0x7fc50c055000).ready entity=mon.0 client_cookie=c0ed1c4a9a07a36b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:06.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc518ff9700 1 -- 172.21.15.121:0/3653446514 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c051230 con 0x1b27810 2023-11-15T02:58:06.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc518ff9700 1 -- 172.21.15.121:0/3653446514 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50c001840 con 0x1b27810 2023-11-15T02:58:06.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc518ff9700 1 -- 172.21.15.121:0/3653446514 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c072480 con 0x1b27810 2023-11-15T02:58:06.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc533d5c740 1 -- 172.21.15.121:0/3653446514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b27810 msgr2=0x1ae2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:06.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc533d5c740 1 --2- 172.21.15.121:0/3653446514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b27810 0x1ae2670 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fc50c049ea0 tx=0x7fc50c055000).stop 2023-11-15T02:58:06.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc533d5c740 1 -- 172.21.15.121:0/3653446514 shutdown_connections 2023-11-15T02:58:06.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc533d5c740 1 --2- 172.21.15.121:0/3653446514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1b2b810 0x1b0a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc533d5c740 1 --2- 172.21.15.121:0/3653446514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b27810 0x1ae2670 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc533d5c740 1 --2- 172.21.15.121:0/3653446514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ade110 0x1ae4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc533d5c740 1 -- 172.21.15.121:0/3653446514 >> 172.21.15.121:0/3653446514 conn(0x18c4790 msgr2=0x1add8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:06.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc533d5c740 1 -- 172.21.15.121:0/3653446514 shutdown_connections 2023-11-15T02:58:06.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc533d5c740 1 -- 172.21.15.121:0/3653446514 wait complete. 2023-11-15T02:58:06.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.887+0000 7fc533d5c740 1 -- start start 2023-11-15T02:58:06.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc533d5c740 1 Processor -- start 2023-11-15T02:58:06.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc533d5c740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ade110 0x1b0a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc533d5c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b27810 0x1b0ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc533d5c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1b2b810 0x1b10c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc533d5c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b0b050 con 0x1b27810 2023-11-15T02:58:06.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc533d5c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b3e640 con 0x1b2b810 2023-11-15T02:58:06.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc533d5c740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b09a20 con 0x1ade110 2023-11-15T02:58:06.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc533d5c740 10 client.0 ll_register_callbacks cb 0x1a5c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:58:06.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc51a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b27810 0x1b0ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc51a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b27810 0x1b0ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50066/0 (socket says 172.21.15.121:50066) 2023-11-15T02:58:06.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc51a7fc700 1 -- 172.21.15.121:0/806402570 learned_addr learned my addr 172.21.15.121:0/806402570 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:06.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc522ffd700 1 --2- 172.21.15.121:0/806402570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ade110 0x1b0a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc5237fe700 1 --2- 172.21.15.121:0/806402570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1b2b810 0x1b10c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc51a7fc700 1 -- 172.21.15.121:0/806402570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ade110 msgr2=0x1b0a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:06.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc51a7fc700 1 --2- 172.21.15.121:0/806402570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ade110 0x1b0a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc51a7fc700 1 -- 172.21.15.121:0/806402570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1b2b810 msgr2=0x1b10c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:06.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc51a7fc700 1 --2- 172.21.15.121:0/806402570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1b2b810 0x1b10c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc51a7fc700 1 -- 172.21.15.121:0/806402570 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1a5f830 con 0x1b27810 2023-11-15T02:58:06.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc5237fe700 1 --2- 172.21.15.121:0/806402570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1b2b810 0x1b10c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:06.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc51a7fc700 1 --2- 172.21.15.121:0/806402570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b27810 0x1b0ab60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7fc50c04ba30 tx=0x7fc50c060040).ready entity=mon.0 client_cookie=e9750424dcf5e34d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:06.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc5017fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:58:06.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc5017fa700 1 -- 172.21.15.121:0/806402570 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c072480 con 0x1b27810 2023-11-15T02:58:06.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc533d5c740 1 -- 172.21.15.121:0/806402570 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1b2b250 con 0x1b27810 2023-11-15T02:58:06.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.891+0000 7fc5017fa700 1 -- 172.21.15.121:0/806402570 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fc50c0725e0 con 0x1b27810 2023-11-15T02:58:06.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc5017fa700 1 -- 172.21.15.121:0/806402570 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc50c013350 con 0x1b27810 2023-11-15T02:58:06.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 10 client.5259 fetch_fsmap learned FSMap version 22 2023-11-15T02:58:06.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 1 -- 172.21.15.121:0/806402570 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x1a5a7b0 con 0x1b27810 2023-11-15T02:58:06.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc5017fa700 1 -- 172.21.15.121:0/806402570 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1b2b250 con 0x1b27810 2023-11-15T02:58:06.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc5017fa700 1 -- 172.21.15.121:0/806402570 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 22) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc50c071070 con 0x1b27810 2023-11-15T02:58:06.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 10 client.5259 fetch_fsmap finished waiting for FSMap version 22 2023-11-15T02:58:06.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 10 client.5259 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-15T02:58:06.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 20 client.5259 populate_metadata read hostname 'smithi121' 2023-11-15T02:58:06.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 1 --2- 172.21.15.121:0/806402570 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x1b0de30 0x1b11120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:06.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 4 client.5259 mds_command: new command op to 5178 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-11-15T02:58:06.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 1 -- 172.21.15.121:0/806402570 --> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x7fc50c072e90 con 0x1b0de30 2023-11-15T02:58:06.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc522ffd700 1 --2- 172.21.15.121:0/806402570 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x1b0de30 0x1b11120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:06.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc5017fa700 10 client.5259 ms_handle_connect on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:06.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc522ffd700 1 --2- 172.21.15.121:0/806402570 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x1b0de30 0x1b11120 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=44d194a2a3a56ba9 server_cookie=fecd20931efabca3 in_seq=0 out_seq=0 2023-11-15T02:58:06.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc5017fa700 1 -- 172.21.15.121:0/806402570 <== mds.0 v2:172.21.15.121:6836/3305580606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+234 (crc 0 0 0) 0x7fc50c072e90 con 0x1b0de30 2023-11-15T02:58:06.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc5017fa700 10 client.5259 handle_command_reply: tid=0 2023-11-15T02:58:06.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 1 client.5259 shutdown 2023-11-15T02:58:06.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 1 -- 172.21.15.121:0/806402570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b27810 msgr2=0x1b0ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:06.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 1 --2- 172.21.15.121:0/806402570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b27810 0x1b0ab60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7fc50c04ba30 tx=0x7fc50c060040).stop 2023-11-15T02:58:06.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 1 -- 172.21.15.121:0/806402570 shutdown_connections 2023-11-15T02:58:06.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 1 --2- 172.21.15.121:0/806402570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1b2b810 0x1b10c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 1 --2- 172.21.15.121:0/806402570 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x1b0de30 0x1b11120 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 1 --2- 172.21.15.121:0/806402570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1b27810 0x1b0ab60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 1 --2- 172.21.15.121:0/806402570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ade110 0x1b0a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc5017fa700 0 client.5259 ms_handle_reset on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:06.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 1 -- 172.21.15.121:0/806402570 >> 172.21.15.121:0/806402570 conn(0x18c4790 msgr2=0x1add8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:06.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 1 -- 172.21.15.121:0/806402570 shutdown_connections 2023-11-15T02:58:06.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 1 -- 172.21.15.121:0/806402570 wait complete. 2023-11-15T02:58:06.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.895+0000 7fc533d5c740 20 client.5259 trim_cache size 0 max 16384 2023-11-15T02:58:06.914 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:58:06.914 INFO:teuthology.orchestra.run.smithi121.stdout:[ 2023-11-15T02:58:06.914 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:06.914 INFO:teuthology.orchestra.run.smithi121.stdout: "damage_type": "dentry", 2023-11-15T02:58:06.915 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 4012680, 2023-11-15T02:58:06.915 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 1099511627776, 2023-11-15T02:58:06.915 INFO:teuthology.orchestra.run.smithi121.stdout: "frag": "*", 2023-11-15T02:58:06.915 INFO:teuthology.orchestra.run.smithi121.stdout: "dname": "file_to_be_damaged", 2023-11-15T02:58:06.916 INFO:teuthology.orchestra.run.smithi121.stdout: "snap_id": "head", 2023-11-15T02:58:06.916 INFO:teuthology.orchestra.run.smithi121.stdout: "path": "/subdir/file_to_be_damaged" 2023-11-15T02:58:06.916 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:06.916 INFO:teuthology.orchestra.run.smithi121.stdout:] 2023-11-15T02:58:06.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.899+0000 7fc523fff700 1 -- 172.21.15.121:0/151257301 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc50405a510 msgr2=0x7fc50405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:06.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.899+0000 7fc523fff700 1 --2- 172.21.15.121:0/151257301 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc50405a510 0x7fc50405c9d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fc514000e10 tx=0x7fc514009040).stop 2023-11-15T02:58:06.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.899+0000 7fc523fff700 1 -- 172.21.15.121:0/151257301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5240bb570 msgr2=0x7fc5240bb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:06.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.899+0000 7fc523fff700 1 --2- 172.21.15.121:0/151257301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5240bb570 0x7fc5240bb9f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fc50c00e860 tx=0x7fc50c009040).stop 2023-11-15T02:58:06.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.899+0000 7fc523fff700 1 -- 172.21.15.121:0/151257301 shutdown_connections 2023-11-15T02:58:06.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.899+0000 7fc523fff700 1 --2- 172.21.15.121:0/151257301 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc50405a510 0x7fc50405c9d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.899+0000 7fc523fff700 1 --2- 172.21.15.121:0/151257301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5240ab320 0x7fc5240ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.899+0000 7fc523fff700 1 --2- 172.21.15.121:0/151257301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5240bb570 0x7fc5240bb9f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.899+0000 7fc523fff700 1 --2- 172.21.15.121:0/151257301 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc510005000 0x7fc524155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:06.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.899+0000 7fc523fff700 1 -- 172.21.15.121:0/151257301 >> 172.21.15.121:0/151257301 conn(0x7fc524067a60 msgr2=0x7fc5240a7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:06.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.899+0000 7fc523fff700 1 -- 172.21.15.121:0/151257301 shutdown_connections 2023-11-15T02:58:06.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:06.899+0000 7fc523fff700 1 -- 172.21.15.121:0/151257301 wait complete. 2023-11-15T02:58:06.925 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-15T02:58:06.925 DEBUG:teuthology.orchestra.run.smithi121:> import sys 2023-11-15T02:58:06.926 DEBUG:teuthology.orchestra.run.smithi121:> import errno 2023-11-15T02:58:06.926 DEBUG:teuthology.orchestra.run.smithi121:> 2023-11-15T02:58:06.927 DEBUG:teuthology.orchestra.run.smithi121:> try: 2023-11-15T02:58:06.927 DEBUG:teuthology.orchestra.run.smithi121:> f = open("/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged", "w") 2023-11-15T02:58:06.927 DEBUG:teuthology.orchestra.run.smithi121:> f.close() 2023-11-15T02:58:06.927 DEBUG:teuthology.orchestra.run.smithi121:> except IOError as e: 2023-11-15T02:58:06.928 DEBUG:teuthology.orchestra.run.smithi121:> sys.exit(errno.EIO) 2023-11-15T02:58:06.928 DEBUG:teuthology.orchestra.run.smithi121:> ' 2023-11-15T02:58:07.205 INFO:teuthology.orchestra.run.smithi121.stderr:daemon-helper: command failed with exit status 5 2023-11-15T02:58:07.219 DEBUG:teuthology.orchestra.run:got remote process result: 5 2023-11-15T02:58:07.221 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-15T02:58:07.221 DEBUG:teuthology.orchestra.run.smithi121:> import os 2023-11-15T02:58:07.221 DEBUG:teuthology.orchestra.run.smithi121:> import stat 2023-11-15T02:58:07.222 DEBUG:teuthology.orchestra.run.smithi121:> import json 2023-11-15T02:58:07.222 DEBUG:teuthology.orchestra.run.smithi121:> import sys 2023-11-15T02:58:07.222 DEBUG:teuthology.orchestra.run.smithi121:> 2023-11-15T02:58:07.223 DEBUG:teuthology.orchestra.run.smithi121:> try: 2023-11-15T02:58:07.223 DEBUG:teuthology.orchestra.run.smithi121:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2023-11-15T02:58:07.223 DEBUG:teuthology.orchestra.run.smithi121:> except OSError as e: 2023-11-15T02:58:07.224 DEBUG:teuthology.orchestra.run.smithi121:> sys.exit(e.errno) 2023-11-15T02:58:07.224 DEBUG:teuthology.orchestra.run.smithi121:> 2023-11-15T02:58:07.224 DEBUG:teuthology.orchestra.run.smithi121:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-11-15T02:58:07.225 DEBUG:teuthology.orchestra.run.smithi121:> print(json.dumps( 2023-11-15T02:58:07.225 DEBUG:teuthology.orchestra.run.smithi121:> dict([(a, getattr(s, a)) for a in attrs]), 2023-11-15T02:58:07.226 DEBUG:teuthology.orchestra.run.smithi121:> indent=2)) 2023-11-15T02:58:07.226 DEBUG:teuthology.orchestra.run.smithi121:> ' 2023-11-15T02:58:07.535 INFO:teuthology.orchestra.run.smithi121.stderr:daemon-helper: command failed with exit status 5 2023-11-15T02:58:07.549 DEBUG:teuthology.orchestra.run:got remote process result: 5 2023-11-15T02:58:07.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:58:07.551 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2023-11-15T02:58:07.623 INFO:teuthology.orchestra.run.smithi121.stdout:2 2023-11-15T02:58:07.623 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T02:58:07.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:58:07.624 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-15T02:58:07.716 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T02:58:07.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:58:07.717 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T02:58:07.776 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub start /subdir repair 2023-11-15T02:58:08.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:07.995+0000 7f146525b700 1 Processor -- start 2023-11-15T02:58:08.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:07.995+0000 7f146525b700 1 -- start start 2023-11-15T02:58:08.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:07.999+0000 7f146525b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14601570b0 0x7f1460066860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:07.999+0000 7f146525b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f14600c5c80 0x7f1460063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:07.999+0000 7f146525b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1460062da0 con 0x7f1460157490 2023-11-15T02:58:08.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:07.999+0000 7f146525b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14600bfb90 con 0x7f14600c5c80 2023-11-15T02:58:08.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:07.999+0000 7f146525b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14600c6140 con 0x7f14601570b0 2023-11-15T02:58:08.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:07.999+0000 7f145ed9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14601570b0 0x7f1460066860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:07.999+0000 7f145f59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f14600c5c80 0x7f1460063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:07.999+0000 7f145f59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f14600c5c80 0x7f1460063230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:39092/0 (socket says 172.21.15.121:39092) 2023-11-15T02:58:08.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:07.999+0000 7f145ed9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14601570b0 0x7f1460066860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:54652/0 (socket says 172.21.15.121:54652) 2023-11-15T02:58:08.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:07.999+0000 7f145f59e700 1 -- 172.21.15.121:0/3798786984 learned_addr learned my addr 172.21.15.121:0/3798786984 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:08.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:07.999+0000 7f145f59e700 1 -- 172.21.15.121:0/3798786984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14601570b0 msgr2=0x7f1460066860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:08.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:07.999+0000 7f145f59e700 1 --2- 172.21.15.121:0/3798786984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14601570b0 0x7f1460066860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:07.999+0000 7f145f59e700 1 -- 172.21.15.121:0/3798786984 >> v1:172.21.15.190:6789/0 conn(0x7f1460157490 legacy=0x7f1460065950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:07.999+0000 7f145f59e700 1 -- 172.21.15.121:0/3798786984 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14600c6420 con 0x7f14600c5c80 2023-11-15T02:58:08.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:07.999+0000 7f145f59e700 1 --2- 172.21.15.121:0/3798786984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14600c5c80 0x7f1460063230 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f1458009f20 tx=0x7f1458009840).ready entity=mon.1 client_cookie=5c5a1236495c8c95 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:08.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:07.999+0000 7f145dd9b700 1 -- 172.21.15.121:0/3798786984 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145800e070 con 0x7f14600c5c80 2023-11-15T02:58:08.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.003+0000 7f145dd9b700 1 -- 172.21.15.121:0/3798786984 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145800b9d0 con 0x7f14600c5c80 2023-11-15T02:58:08.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.003+0000 7f145dd9b700 1 -- 172.21.15.121:0/3798786984 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145800bd50 con 0x7f14600c5c80 2023-11-15T02:58:08.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.003+0000 7f146525b700 1 -- 172.21.15.121:0/3798786984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14600c5c80 msgr2=0x7f1460063230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.003+0000 7f146525b700 1 --2- 172.21.15.121:0/3798786984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14600c5c80 0x7f1460063230 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f1458009f20 tx=0x7f1458009840).stop 2023-11-15T02:58:08.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.003+0000 7f146525b700 1 -- 172.21.15.121:0/3798786984 shutdown_connections 2023-11-15T02:58:08.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.003+0000 7f146525b700 1 --2- 172.21.15.121:0/3798786984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14600c5c80 0x7f1460063230 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.003+0000 7f146525b700 1 --2- 172.21.15.121:0/3798786984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14601570b0 0x7f1460066860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.003+0000 7f146525b700 1 -- 172.21.15.121:0/3798786984 >> 172.21.15.121:0/3798786984 conn(0x7f14600684f0 msgr2=0x7f14600cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:08.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.003+0000 7f146525b700 1 -- 172.21.15.121:0/3798786984 shutdown_connections 2023-11-15T02:58:08.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.003+0000 7f146525b700 1 -- 172.21.15.121:0/3798786984 wait complete. 2023-11-15T02:58:08.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.003+0000 7f146525b700 1 Processor -- start 2023-11-15T02:58:08.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.003+0000 7f146525b700 1 -- start start 2023-11-15T02:58:08.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f146525b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14600c5c80 0x7f146014a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f146525b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14601570b0 0x7f1460167580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f146525b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14601650b0 0x7f1460158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f146525b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1460167a70 con 0x7f14601650b0 2023-11-15T02:58:08.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f146525b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146014b5d0 con 0x7f14600c5c80 2023-11-15T02:58:08.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f146525b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14600b09a0 con 0x7f14601570b0 2023-11-15T02:58:08.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f145f59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14601650b0 0x7f1460158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f145f59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14601650b0 0x7f1460158c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50076/0 (socket says 172.21.15.121:50076) 2023-11-15T02:58:08.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f145f59e700 1 -- 172.21.15.121:0/3187741676 learned_addr learned my addr 172.21.15.121:0/3187741676 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:08.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f145ed9d700 1 --2- 172.21.15.121:0/3187741676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14600c5c80 0x7f146014a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f145e59c700 1 --2- 172.21.15.121:0/3187741676 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14601570b0 0x7f1460167580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f145f59e700 1 -- 172.21.15.121:0/3187741676 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14601570b0 msgr2=0x7f1460167580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f145f59e700 1 --2- 172.21.15.121:0/3187741676 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14601570b0 0x7f1460167580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f145f59e700 1 -- 172.21.15.121:0/3187741676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14600c5c80 msgr2=0x7f146014a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f145f59e700 1 --2- 172.21.15.121:0/3187741676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14600c5c80 0x7f146014a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f145f59e700 1 -- 172.21.15.121:0/3187741676 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1458009ae0 con 0x7f14601650b0 2023-11-15T02:58:08.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f145f59e700 1 --2- 172.21.15.121:0/3187741676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14601650b0 0x7f1460158c90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f1458010000 tx=0x7f1460062d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:08.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f1446ffd700 1 -- 172.21.15.121:0/3187741676 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14580092e0 con 0x7f14601650b0 2023-11-15T02:58:08.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f1446ffd700 1 -- 172.21.15.121:0/3187741676 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145801e470 con 0x7f14601650b0 2023-11-15T02:58:08.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f146525b700 1 -- 172.21.15.121:0/3187741676 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146014a9c0 con 0x7f14601650b0 2023-11-15T02:58:08.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f1446ffd700 1 -- 172.21.15.121:0/3187741676 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145801e830 con 0x7f14601650b0 2023-11-15T02:58:08.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.007+0000 7f146525b700 1 -- 172.21.15.121:0/3187741676 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146014af80 con 0x7f14601650b0 2023-11-15T02:58:08.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.011+0000 7f1446ffd700 1 -- 172.21.15.121:0/3187741676 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1458003040 con 0x7f14601650b0 2023-11-15T02:58:08.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.011+0000 7f1446ffd700 1 --2- 172.21.15.121:0/3187741676 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f144c05a510 0x7f144c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.011+0000 7f1446ffd700 1 -- 172.21.15.121:0/3187741676 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f1458075d60 con 0x7f14601650b0 2023-11-15T02:58:08.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.011+0000 7f145ed9d700 1 --2- 172.21.15.121:0/3187741676 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f144c05a510 0x7f144c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.015+0000 7f145ed9d700 1 --2- 172.21.15.121:0/3187741676 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f144c05a510 0x7f144c05c9d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f1460158c30 tx=0x7f1450009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:08.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.015+0000 7f146f5ce740 1 Processor -- start 2023-11-15T02:58:08.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.015+0000 7f146f5ce740 1 -- start start 2023-11-15T02:58:08.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.015+0000 7f146f5ce740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1328ca0 0x12e1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.015+0000 7f146f5ce740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x130d560 0x1329080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.015+0000 7f146f5ce740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x12e3d70 0x12e41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.015+0000 7f146f5ce740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1222160 con 0x1328ca0 2023-11-15T02:58:08.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.015+0000 7f146f5ce740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1222400 con 0x12e3d70 2023-11-15T02:58:08.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.015+0000 7f146f5ce740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x120ee00 con 0x130d560 2023-11-15T02:58:08.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.015+0000 7f145ed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1328ca0 0x12e1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.015+0000 7f145ed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1328ca0 0x12e1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50090/0 (socket says 172.21.15.121:50090) 2023-11-15T02:58:08.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.015+0000 7f145ed9d700 1 -- 172.21.15.121:0/4005397338 learned_addr learned my addr 172.21.15.121:0/4005397338 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:08.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f145e59c700 1 --2- 172.21.15.121:0/4005397338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x130d560 0x1329080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f145f59e700 1 --2- 172.21.15.121:0/4005397338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x12e3d70 0x12e41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f145ed9d700 1 -- 172.21.15.121:0/4005397338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x130d560 msgr2=0x1329080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:08.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f145ed9d700 1 --2- 172.21.15.121:0/4005397338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x130d560 0x1329080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f145ed9d700 1 -- 172.21.15.121:0/4005397338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x12e3d70 msgr2=0x12e41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:08.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f145ed9d700 1 --2- 172.21.15.121:0/4005397338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x12e3d70 0x12e41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f145ed9d700 1 -- 172.21.15.121:0/4005397338 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x12628e0 con 0x1328ca0 2023-11-15T02:58:08.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f145ed9d700 1 --2- 172.21.15.121:0/4005397338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1328ca0 0x12e1f80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f1450011d80 tx=0x7f1450017000).ready entity=mon.0 client_cookie=88fffe7d993c20d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:08.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f143ffff700 1 -- 172.21.15.121:0/4005397338 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1450011e80 con 0x1328ca0 2023-11-15T02:58:08.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f143ffff700 1 -- 172.21.15.121:0/4005397338 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1450014c60 con 0x1328ca0 2023-11-15T02:58:08.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f143ffff700 1 -- 172.21.15.121:0/4005397338 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145001e3f0 con 0x1328ca0 2023-11-15T02:58:08.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f146f5ce740 1 -- 172.21.15.121:0/4005397338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1328ca0 msgr2=0x12e1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f146f5ce740 1 --2- 172.21.15.121:0/4005397338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1328ca0 0x12e1f80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f1450011d80 tx=0x7f1450017000).stop 2023-11-15T02:58:08.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f146f5ce740 1 -- 172.21.15.121:0/4005397338 shutdown_connections 2023-11-15T02:58:08.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f146f5ce740 1 --2- 172.21.15.121:0/4005397338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x12e3d70 0x12e41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f146f5ce740 1 --2- 172.21.15.121:0/4005397338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x130d560 0x1329080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f146f5ce740 1 --2- 172.21.15.121:0/4005397338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1328ca0 0x12e1f80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f146f5ce740 1 -- 172.21.15.121:0/4005397338 >> 172.21.15.121:0/4005397338 conn(0x10c77c0 msgr2=0x12e7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:08.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f146f5ce740 1 -- 172.21.15.121:0/4005397338 shutdown_connections 2023-11-15T02:58:08.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.019+0000 7f146f5ce740 1 -- 172.21.15.121:0/4005397338 wait complete. 2023-11-15T02:58:08.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f146f5ce740 1 -- start start 2023-11-15T02:58:08.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f146f5ce740 1 Processor -- start 2023-11-15T02:58:08.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f146f5ce740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12e3d70 0x132b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f146f5ce740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x130d560 0x133a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f146f5ce740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1328ca0 0x133a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f146f5ce740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x133b2c0 con 0x12e3d70 2023-11-15T02:58:08.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f146f5ce740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x133b450 con 0x1328ca0 2023-11-15T02:58:08.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f146f5ce740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x133b870 con 0x130d560 2023-11-15T02:58:08.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f146f5ce740 10 client.0 ll_register_callbacks cb 0x129efa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:58:08.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f145ed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12e3d70 0x132b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f145ed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12e3d70 0x132b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50092/0 (socket says 172.21.15.121:50092) 2023-11-15T02:58:08.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f145ed9d700 1 -- 172.21.15.121:0/2166667670 learned_addr learned my addr 172.21.15.121:0/2166667670 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:08.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f145e59c700 1 --2- 172.21.15.121:0/2166667670 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x130d560 0x133a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f145f59e700 1 --2- 172.21.15.121:0/2166667670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1328ca0 0x133a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f145ed9d700 1 -- 172.21.15.121:0/2166667670 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x130d560 msgr2=0x133a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:08.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f145ed9d700 1 --2- 172.21.15.121:0/2166667670 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x130d560 0x133a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f145ed9d700 1 -- 172.21.15.121:0/2166667670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1328ca0 msgr2=0x133a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:08.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f145ed9d700 1 --2- 172.21.15.121:0/2166667670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1328ca0 0x133a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f145ed9d700 1 -- 172.21.15.121:0/2166667670 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14500119d0 con 0x12e3d70 2023-11-15T02:58:08.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f145ed9d700 1 --2- 172.21.15.121:0/2166667670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12e3d70 0x132b050 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f1450009170 tx=0x7f1450024000).ready entity=mon.0 client_cookie=77be049ad94a68e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:08.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f143d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:58:08.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f143d7fa700 1 -- 172.21.15.121:0/2166667670 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145001eb40 con 0x12e3d70 2023-11-15T02:58:08.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f143d7fa700 1 -- 172.21.15.121:0/2166667670 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f145001eca0 con 0x12e3d70 2023-11-15T02:58:08.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.023+0000 7f146f5ce740 1 -- 172.21.15.121:0/2166667670 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x125f350 con 0x12e3d70 2023-11-15T02:58:08.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f143d7fa700 1 -- 172.21.15.121:0/2166667670 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f145001ee60 con 0x12e3d70 2023-11-15T02:58:08.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f146f5ce740 10 client.5268 fetch_fsmap learned FSMap version 22 2023-11-15T02:58:08.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f146f5ce740 1 -- 172.21.15.121:0/2166667670 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x125db10 con 0x12e3d70 2023-11-15T02:58:08.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f143d7fa700 1 -- 172.21.15.121:0/2166667670 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x125f350 con 0x12e3d70 2023-11-15T02:58:08.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f143d7fa700 1 -- 172.21.15.121:0/2166667670 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 22) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1450027450 con 0x12e3d70 2023-11-15T02:58:08.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f146f5ce740 10 client.5268 fetch_fsmap finished waiting for FSMap version 22 2023-11-15T02:58:08.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f146f5ce740 10 client.5268 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-15T02:58:08.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f146f5ce740 20 client.5268 populate_metadata read hostname 'smithi121' 2023-11-15T02:58:08.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f146f5ce740 1 --2- 172.21.15.121:0/2166667670 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x133cf00 0x133d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f146f5ce740 4 client.5268 mds_command: new command op to 5178 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T02:58:08.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f146f5ce740 1 -- 172.21.15.121:0/2166667670 --> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x12558a0 con 0x133cf00 2023-11-15T02:58:08.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f145e59c700 1 --2- 172.21.15.121:0/2166667670 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x133cf00 0x133d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f145e59c700 1 --2- 172.21.15.121:0/2166667670 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x133cf00 0x133d2e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=453992fe78de06ac server_cookie=f6a81ae7d2604e35 in_seq=0 out_seq=0 2023-11-15T02:58:08.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f143d7fa700 10 client.5268 ms_handle_connect on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:08.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f143d7fa700 1 -- 172.21.15.121:0/2166667670 <== mds.0 v2:172.21.15.121:6836/3305580606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x12558a0 con 0x133cf00 2023-11-15T02:58:08.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f143d7fa700 10 client.5268 handle_command_reply: tid=0 2023-11-15T02:58:08.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f146f5ce740 1 client.5268 shutdown 2023-11-15T02:58:08.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f146f5ce740 1 -- 172.21.15.121:0/2166667670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12e3d70 msgr2=0x132b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f146f5ce740 1 --2- 172.21.15.121:0/2166667670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12e3d70 0x132b050 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f1450009170 tx=0x7f1450024000).stop 2023-11-15T02:58:08.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f146f5ce740 1 -- 172.21.15.121:0/2166667670 shutdown_connections 2023-11-15T02:58:08.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f146f5ce740 1 --2- 172.21.15.121:0/2166667670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1328ca0 0x133a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f146f5ce740 1 --2- 172.21.15.121:0/2166667670 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x130d560 0x133a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f146f5ce740 1 --2- 172.21.15.121:0/2166667670 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x133cf00 0x133d2e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f146f5ce740 1 --2- 172.21.15.121:0/2166667670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12e3d70 0x132b050 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f143d7fa700 0 client.5268 ms_handle_reset on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:08.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.027+0000 7f146f5ce740 1 -- 172.21.15.121:0/2166667670 >> 172.21.15.121:0/2166667670 conn(0x10c77c0 msgr2=0x12e6200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:08.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.031+0000 7f146f5ce740 1 -- 172.21.15.121:0/2166667670 shutdown_connections 2023-11-15T02:58:08.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.031+0000 7f146f5ce740 1 -- 172.21.15.121:0/2166667670 wait complete. 2023-11-15T02:58:08.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.031+0000 7f146f5ce740 20 client.5268 trim_cache size 0 max 16384 2023-11-15T02:58:08.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.047+0000 7f146f5ce740 1 Processor -- start 2023-11-15T02:58:08.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.047+0000 7f146f5ce740 1 -- start start 2023-11-15T02:58:08.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f146f5ce740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12e3d70 0x12e21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f146f5ce740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x130d560 0x130ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f146f5ce740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1328ca0 0x13388d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f146f5ce740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1310d20 con 0x12e3d70 2023-11-15T02:58:08.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f146f5ce740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x12dcb30 con 0x1328ca0 2023-11-15T02:58:08.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f146f5ce740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x130edf0 con 0x130d560 2023-11-15T02:58:08.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f145ed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12e3d70 0x12e21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f145ed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12e3d70 0x12e21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50104/0 (socket says 172.21.15.121:50104) 2023-11-15T02:58:08.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f145ed9d700 1 -- 172.21.15.121:0/1172165507 learned_addr learned my addr 172.21.15.121:0/1172165507 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:08.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f145ed9d700 1 -- 172.21.15.121:0/1172165507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x130d560 msgr2=0x130ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:08.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f145f59e700 1 --2- 172.21.15.121:0/1172165507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1328ca0 0x13388d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f145e59c700 1 --2- 172.21.15.121:0/1172165507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x130d560 0x130ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f145ed9d700 1 --2- 172.21.15.121:0/1172165507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x130d560 0x130ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f145ed9d700 1 -- 172.21.15.121:0/1172165507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1328ca0 msgr2=0x13388d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:08.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f145ed9d700 1 --2- 172.21.15.121:0/1172165507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1328ca0 0x13388d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f145ed9d700 1 -- 172.21.15.121:0/1172165507 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14601594e0 con 0x12e3d70 2023-11-15T02:58:08.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f145e59c700 1 --2- 172.21.15.121:0/1172165507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x130d560 0x130ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:58:08.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f145f59e700 1 --2- 172.21.15.121:0/1172165507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1328ca0 0x13388d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:58:08.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f145ed9d700 1 --2- 172.21.15.121:0/1172165507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12e3d70 0x12e21b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f145000b010 tx=0x7f145002c040).ready entity=mon.0 client_cookie=8c4ab49d4806e7d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:08.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f1444ff9700 1 -- 172.21.15.121:0/1172165507 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14500279b0 con 0x12e3d70 2023-11-15T02:58:08.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f1444ff9700 1 -- 172.21.15.121:0/1172165507 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1450027450 con 0x12e3d70 2023-11-15T02:58:08.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f1444ff9700 1 -- 172.21.15.121:0/1172165507 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145001ed50 con 0x12e3d70 2023-11-15T02:58:08.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f146f5ce740 1 -- 172.21.15.121:0/1172165507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12e3d70 msgr2=0x12e21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f146f5ce740 1 --2- 172.21.15.121:0/1172165507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12e3d70 0x12e21b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f145000b010 tx=0x7f145002c040).stop 2023-11-15T02:58:08.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f146f5ce740 1 -- 172.21.15.121:0/1172165507 shutdown_connections 2023-11-15T02:58:08.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f146f5ce740 1 --2- 172.21.15.121:0/1172165507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1328ca0 0x13388d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f146f5ce740 1 --2- 172.21.15.121:0/1172165507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x130d560 0x130ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f146f5ce740 1 --2- 172.21.15.121:0/1172165507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12e3d70 0x12e21b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f146f5ce740 1 -- 172.21.15.121:0/1172165507 >> 172.21.15.121:0/1172165507 conn(0x10c77c0 msgr2=0x12e0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:08.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f146f5ce740 1 -- 172.21.15.121:0/1172165507 shutdown_connections 2023-11-15T02:58:08.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.051+0000 7f146f5ce740 1 -- 172.21.15.121:0/1172165507 wait complete. 2023-11-15T02:58:08.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f146f5ce740 1 -- start start 2023-11-15T02:58:08.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f146f5ce740 1 Processor -- start 2023-11-15T02:58:08.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f146f5ce740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x12e3d70 0x138a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f146f5ce740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x130d560 0x1326820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f146f5ce740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1328ca0 0x1333cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f146f5ce740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1328850 con 0x130d560 2023-11-15T02:58:08.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f146f5ce740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x138ad70 con 0x12e3d70 2023-11-15T02:58:08.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f146f5ce740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1327090 con 0x1328ca0 2023-11-15T02:58:08.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f146f5ce740 10 client.0 ll_register_callbacks cb 0x7f1450011890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:58:08.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f145e59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x130d560 0x1326820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f145e59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x130d560 0x1326820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50112/0 (socket says 172.21.15.121:50112) 2023-11-15T02:58:08.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f145e59c700 1 -- 172.21.15.121:0/2489619824 learned_addr learned my addr 172.21.15.121:0/2489619824 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:08.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f145ed9d700 1 --2- 172.21.15.121:0/2489619824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x12e3d70 0x138a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f145f59e700 1 --2- 172.21.15.121:0/2489619824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1328ca0 0x1333cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f145e59c700 1 -- 172.21.15.121:0/2489619824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1328ca0 msgr2=0x1333cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:08.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f145e59c700 1 --2- 172.21.15.121:0/2489619824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1328ca0 0x1333cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f145e59c700 1 -- 172.21.15.121:0/2489619824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x12e3d70 msgr2=0x138a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:08.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f145e59c700 1 --2- 172.21.15.121:0/2489619824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x12e3d70 0x138a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f145e59c700 1 -- 172.21.15.121:0/2489619824 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x125db10 con 0x130d560 2023-11-15T02:58:08.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f145e59c700 1 --2- 172.21.15.121:0/2489619824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x130d560 0x1326820 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f14600c54d0 tx=0x7f145400ff90).ready entity=mon.0 client_cookie=612373e6388fde5e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:08.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.055+0000 7f143d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:58:08.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f143d7fa700 1 -- 172.21.15.121:0/2489619824 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145401d070 con 0x130d560 2023-11-15T02:58:08.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f143d7fa700 1 -- 172.21.15.121:0/2489619824 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f14540180d0 con 0x130d560 2023-11-15T02:58:08.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f146f5ce740 1 -- 172.21.15.121:0/2489619824 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1450014c60 con 0x130d560 2023-11-15T02:58:08.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f143d7fa700 1 -- 172.21.15.121:0/2489619824 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1454018290 con 0x130d560 2023-11-15T02:58:08.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f146f5ce740 10 client.5274 fetch_fsmap learned FSMap version 22 2023-11-15T02:58:08.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f146f5ce740 1 -- 172.21.15.121:0/2489619824 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x125d860 con 0x130d560 2023-11-15T02:58:08.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f143d7fa700 1 -- 172.21.15.121:0/2489619824 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1450014c60 con 0x130d560 2023-11-15T02:58:08.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f143d7fa700 1 -- 172.21.15.121:0/2489619824 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 22) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f14540189a0 con 0x130d560 2023-11-15T02:58:08.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f146f5ce740 10 client.5274 fetch_fsmap finished waiting for FSMap version 22 2023-11-15T02:58:08.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f146f5ce740 10 client.5274 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-15T02:58:08.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f146f5ce740 20 client.5274 populate_metadata read hostname 'smithi121' 2023-11-15T02:58:08.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f146f5ce740 1 --2- 172.21.15.121:0/2489619824 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x133a560 0x1338f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f146f5ce740 4 client.5274 mds_command: new command op to 5178 tid=0[{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair"]}] 2023-11-15T02:58:08.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f146f5ce740 1 -- 172.21.15.121:0/2489619824 --> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair"]}) v1 -- 0x111e750 con 0x133a560 2023-11-15T02:58:08.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f145ed9d700 1 --2- 172.21.15.121:0/2489619824 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x133a560 0x1338f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f143d7fa700 10 client.5274 ms_handle_connect on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:08.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f145ed9d700 1 --2- 172.21.15.121:0/2489619824 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x133a560 0x1338f40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=a0a8a52d3d4877fe server_cookie=10bceaef02d488e8 in_seq=0 out_seq=0 2023-11-15T02:58:08.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f143d7fa700 1 -- 172.21.15.121:0/2489619824 <== mds.0 v2:172.21.15.121:6836/3305580606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x111e750 con 0x133a560 2023-11-15T02:58:08.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f143d7fa700 10 client.5274 handle_command_reply: tid=0 2023-11-15T02:58:08.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f146f5ce740 1 client.5274 shutdown 2023-11-15T02:58:08.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f146f5ce740 1 -- 172.21.15.121:0/2489619824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x130d560 msgr2=0x1326820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.059+0000 7f146f5ce740 1 --2- 172.21.15.121:0/2489619824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x130d560 0x1326820 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f14600c54d0 tx=0x7f145400ff90).stop 2023-11-15T02:58:08.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146f5ce740 1 -- 172.21.15.121:0/2489619824 shutdown_connections 2023-11-15T02:58:08.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146f5ce740 1 --2- 172.21.15.121:0/2489619824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1328ca0 0x1333cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146f5ce740 1 --2- 172.21.15.121:0/2489619824 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x133a560 0x1338f40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146f5ce740 1 --2- 172.21.15.121:0/2489619824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x130d560 0x1326820 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146f5ce740 1 --2- 172.21.15.121:0/2489619824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x12e3d70 0x138a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146f5ce740 1 -- 172.21.15.121:0/2489619824 >> 172.21.15.121:0/2489619824 conn(0x10c77c0 msgr2=0x1313830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:08.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f143d7fa700 0 client.5274 ms_handle_reset on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:08.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146f5ce740 1 -- 172.21.15.121:0/2489619824 shutdown_connections 2023-11-15T02:58:08.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146f5ce740 1 -- 172.21.15.121:0/2489619824 wait complete. 2023-11-15T02:58:08.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146f5ce740 20 client.5274 trim_cache size 0 max 16384 2023-11-15T02:58:08.078 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T02:58:08.078 INFO:teuthology.orchestra.run.smithi121.stdout: "return_code": 0, 2023-11-15T02:58:08.078 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_tag": "4c60cbb0-674a-4f3a-8f8f-3258f75817b1", 2023-11-15T02:58:08.079 INFO:teuthology.orchestra.run.smithi121.stdout: "mode": "asynchronous" 2023-11-15T02:58:08.079 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T02:58:08.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146525b700 1 -- 172.21.15.121:0/3187741676 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f144c05a510 msgr2=0x7f144c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146525b700 1 --2- 172.21.15.121:0/3187741676 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f144c05a510 0x7f144c05c9d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f1460158c30 tx=0x7f1450009060).stop 2023-11-15T02:58:08.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146525b700 1 -- 172.21.15.121:0/3187741676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14601650b0 msgr2=0x7f1460158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146525b700 1 --2- 172.21.15.121:0/3187741676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14601650b0 0x7f1460158c90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f1458010000 tx=0x7f1460062d10).stop 2023-11-15T02:58:08.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146525b700 1 -- 172.21.15.121:0/3187741676 shutdown_connections 2023-11-15T02:58:08.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146525b700 1 --2- 172.21.15.121:0/3187741676 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f144c05a510 0x7f144c05c9d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146525b700 1 --2- 172.21.15.121:0/3187741676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14601650b0 0x7f1460158c90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146525b700 1 --2- 172.21.15.121:0/3187741676 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14601570b0 0x7f1460167580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146525b700 1 --2- 172.21.15.121:0/3187741676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14600c5c80 0x7f146014a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146525b700 1 -- 172.21.15.121:0/3187741676 >> 172.21.15.121:0/3187741676 conn(0x7f14600684f0 msgr2=0x7f1460063ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:08.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146525b700 1 -- 172.21.15.121:0/3187741676 shutdown_connections 2023-11-15T02:58:08.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.063+0000 7f146525b700 1 -- 172.21.15.121:0/3187741676 wait complete. 2023-11-15T02:58:08.089 INFO:tasks.cephfs.test_damage:{ "return_code": 0, "scrub_tag": "4c60cbb0-674a-4f3a-8f8f-3258f75817b1", "mode": "asynchronous" } 2023-11-15T02:58:08.090 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2023-11-15T02:58:08.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.243+0000 7f00e49b9700 1 Processor -- start 2023-11-15T02:58:08.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.243+0000 7f00e49b9700 1 -- start start 2023-11-15T02:58:08.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.247+0000 7f00e49b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00e014e380 0x7f00e00c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.247+0000 7f00e49b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00e00c7e90 con 0x7f00e00ad270 2023-11-15T02:58:08.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.247+0000 7f00e49b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00e014e760 con 0x7f00e00c6040 2023-11-15T02:58:08.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.247+0000 7f00e49b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e00c8320 con 0x7f00e014e380 2023-11-15T02:58:08.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.247+0000 7f00dffff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f00e00ad270 0x7f00e0065680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58408/0 (socket says 172.21.15.121:58408) 2023-11-15T02:58:08.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.247+0000 7f00dffff700 1 -- 172.21.15.121:0/1415701484 learned_addr learned my addr 172.21.15.121:0/1415701484 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:08.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.247+0000 7f00df7fe700 1 --2- 172.21.15.121:0/1415701484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00e014e380 0x7f00e00c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.247+0000 7f00deffd700 1 -- 172.21.15.121:0/1415701484 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771908266 0 0) 0x7f00e00c7e90 con 0x7f00e00ad270 2023-11-15T02:58:08.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.247+0000 7f00deffd700 1 -- 172.21.15.121:0/1415701484 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00cc0033f0 con 0x7f00e00ad270 2023-11-15T02:58:08.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.247+0000 7f00deffd700 1 -- 172.21.15.121:0/1415701484 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5058004 0 0) 0x7f00e014e760 con 0x7f00e00c6040 2023-11-15T02:58:08.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.247+0000 7f00deffd700 1 -- 172.21.15.121:0/1415701484 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00e00c7e90 con 0x7f00e00c6040 2023-11-15T02:58:08.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.247+0000 7f00deffd700 1 -- 172.21.15.121:0/1415701484 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2660575998 0 0) 0x7f00cc0033f0 con 0x7f00e00ad270 2023-11-15T02:58:08.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.247+0000 7f00df7fe700 1 -- 172.21.15.121:0/1415701484 >> v1:172.21.15.190:6789/0 conn(0x7f00e00c6040 legacy=0x7f00e00acb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.247+0000 7f00df7fe700 1 -- 172.21.15.121:0/1415701484 >> v1:172.21.15.121:6789/0 conn(0x7f00e00ad270 legacy=0x7f00e0065680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.247+0000 7f00df7fe700 1 -- 172.21.15.121:0/1415701484 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e00c7280 con 0x7f00e014e380 2023-11-15T02:58:08.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.251+0000 7f00df7fe700 1 --2- 172.21.15.121:0/1415701484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00e014e380 0x7f00e00c5b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f00d0009790 tx=0x7f00d0009060).ready entity=mon.2 client_cookie=8212dd0ec094a2a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:08.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.251+0000 7f00deffd700 1 -- 172.21.15.121:0/1415701484 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d80017f0 con 0x7f00e014e380 2023-11-15T02:58:08.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.251+0000 7f00deffd700 1 -- 172.21.15.121:0/1415701484 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d0011400 con 0x7f00e014e380 2023-11-15T02:58:08.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.251+0000 7f00deffd700 1 -- 172.21.15.121:0/1415701484 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d0011780 con 0x7f00e014e380 2023-11-15T02:58:08.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.251+0000 7f00e49b9700 1 -- 172.21.15.121:0/1415701484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00e014e380 msgr2=0x7f00e00c5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.251+0000 7f00e49b9700 1 --2- 172.21.15.121:0/1415701484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00e014e380 0x7f00e00c5b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f00d0009790 tx=0x7f00d0009060).stop 2023-11-15T02:58:08.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.251+0000 7f00e49b9700 1 -- 172.21.15.121:0/1415701484 shutdown_connections 2023-11-15T02:58:08.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.251+0000 7f00e49b9700 1 --2- 172.21.15.121:0/1415701484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00e014e380 0x7f00e00c5b50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.251+0000 7f00e49b9700 1 -- 172.21.15.121:0/1415701484 >> 172.21.15.121:0/1415701484 conn(0x7f00e0067a60 msgr2=0x7f00e0067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:08.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.251+0000 7f00e49b9700 1 -- 172.21.15.121:0/1415701484 shutdown_connections 2023-11-15T02:58:08.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.251+0000 7f00e49b9700 1 -- 172.21.15.121:0/1415701484 wait complete. 2023-11-15T02:58:08.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.255+0000 7f00e49b9700 1 Processor -- start 2023-11-15T02:58:08.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.255+0000 7f00e49b9700 1 -- start start 2023-11-15T02:58:08.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.255+0000 7f00e49b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00e00ad270 0x7f00e014d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.255+0000 7f00e49b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00e00c6040 0x7f00e00ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.255+0000 7f00e49b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00e014e380 0x7f00e00cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.255+0000 7f00e49b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e00adde0 con 0x7f00e014e380 2023-11-15T02:58:08.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.255+0000 7f00e49b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e00c25f0 con 0x7f00e00ad270 2023-11-15T02:58:08.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.255+0000 7f00e49b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e00c28b0 con 0x7f00e00c6040 2023-11-15T02:58:08.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.255+0000 7f00dffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00e014e380 0x7f00e00cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.255+0000 7f00dffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00e014e380 0x7f00e00cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50114/0 (socket says 172.21.15.121:50114) 2023-11-15T02:58:08.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.255+0000 7f00dffff700 1 -- 172.21.15.121:0/893728458 learned_addr learned my addr 172.21.15.121:0/893728458 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:08.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.255+0000 7f00df7fe700 1 --2- 172.21.15.121:0/893728458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00e00ad270 0x7f00e014d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.255+0000 7f00d6ffd700 1 --2- 172.21.15.121:0/893728458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00e00c6040 0x7f00e00ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.255+0000 7f00dffff700 1 -- 172.21.15.121:0/893728458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00e00c6040 msgr2=0x7f00e00ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.255+0000 7f00dffff700 1 --2- 172.21.15.121:0/893728458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00e00c6040 0x7f00e00ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.255+0000 7f00dffff700 1 -- 172.21.15.121:0/893728458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00e00ad270 msgr2=0x7f00e014d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.255+0000 7f00dffff700 1 --2- 172.21.15.121:0/893728458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00e00ad270 0x7f00e014d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.255+0000 7f00dffff700 1 -- 172.21.15.121:0/893728458 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d00092a0 con 0x7f00e014e380 2023-11-15T02:58:08.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.255+0000 7f00df7fe700 1 --2- 172.21.15.121:0/893728458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00e00ad270 0x7f00e014d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:08.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.259+0000 7f00dffff700 1 --2- 172.21.15.121:0/893728458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00e014e380 0x7f00e00cae70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f00d800cb60 tx=0x7f00d8001e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:08.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.259+0000 7f00d7fff700 1 -- 172.21.15.121:0/893728458 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d8004c40 con 0x7f00e014e380 2023-11-15T02:58:08.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.259+0000 7f00d7fff700 1 -- 172.21.15.121:0/893728458 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d8004da0 con 0x7f00e014e380 2023-11-15T02:58:08.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.259+0000 7f00e49b9700 1 -- 172.21.15.121:0/893728458 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e00c2b90 con 0x7f00e014e380 2023-11-15T02:58:08.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.259+0000 7f00d7fff700 1 -- 172.21.15.121:0/893728458 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d80073a0 con 0x7f00e014e380 2023-11-15T02:58:08.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.259+0000 7f00e49b9700 1 -- 172.21.15.121:0/893728458 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e00cb360 con 0x7f00e014e380 2023-11-15T02:58:08.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.263+0000 7f00d7fff700 1 -- 172.21.15.121:0/893728458 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f00d8007710 con 0x7f00e014e380 2023-11-15T02:58:08.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.263+0000 7f00d7fff700 1 --2- 172.21.15.121:0/893728458 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f00c005a510 0x7f00c005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.263+0000 7f00d7fff700 1 -- 172.21.15.121:0/893728458 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f00d8074770 con 0x7f00e014e380 2023-11-15T02:58:08.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.263+0000 7f00df7fe700 1 --2- 172.21.15.121:0/893728458 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f00c005a510 0x7f00c005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.263+0000 7f00df7fe700 1 --2- 172.21.15.121:0/893728458 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f00c005a510 0x7f00c005c9d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f00e00c1330 tx=0x7f00d0016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:08.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.263+0000 7f00eff8f740 1 Processor -- start 2023-11-15T02:58:08.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00eff8f740 1 -- start start 2023-11-15T02:58:08.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00eff8f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe26930 0xe26d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00eff8f740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xde4bd0 0xddf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00eff8f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde5ac0 0xddf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00eff8f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0xd42540 con 0xde5ac0 2023-11-15T02:58:08.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00eff8f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0xd05e90 con 0xe26930 2023-11-15T02:58:08.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00eff8f740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0xd0bd90 con 0xde4bd0 2023-11-15T02:58:08.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00dffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde5ac0 0xddf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00dffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde5ac0 0xddf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50124/0 (socket says 172.21.15.121:50124) 2023-11-15T02:58:08.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00dffff700 1 -- 172.21.15.121:0/3853078562 learned_addr learned my addr 172.21.15.121:0/3853078562 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:08.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00d6ffd700 1 --2- 172.21.15.121:0/3853078562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xde4bd0 0xddf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00df7fe700 1 --2- 172.21.15.121:0/3853078562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe26930 0xe26d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00dffff700 1 -- 172.21.15.121:0/3853078562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xde4bd0 msgr2=0xddf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:08.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00dffff700 1 --2- 172.21.15.121:0/3853078562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xde4bd0 0xddf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00dffff700 1 -- 172.21.15.121:0/3853078562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe26930 msgr2=0xe26d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:08.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00dffff700 1 --2- 172.21.15.121:0/3853078562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe26930 0xe26d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00dffff700 1 -- 172.21.15.121:0/3853078562 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xd704c0 con 0xde5ac0 2023-11-15T02:58:08.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00df7fe700 1 --2- 172.21.15.121:0/3853078562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe26930 0xe26d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:58:08.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00dffff700 1 --2- 172.21.15.121:0/3853078562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde5ac0 0xddf640 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f00d804b800 tx=0x7f00d800f000).ready entity=mon.0 client_cookie=66a02af3204874c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:08.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00d4ff9700 1 -- 172.21.15.121:0/3853078562 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d8055030 con 0xde5ac0 2023-11-15T02:58:08.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00d4ff9700 1 -- 172.21.15.121:0/3853078562 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d801de00 con 0xde5ac0 2023-11-15T02:58:08.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00d4ff9700 1 -- 172.21.15.121:0/3853078562 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d8051150 con 0xde5ac0 2023-11-15T02:58:08.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00eff8f740 1 -- 172.21.15.121:0/3853078562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde5ac0 msgr2=0xddf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.267+0000 7f00eff8f740 1 --2- 172.21.15.121:0/3853078562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde5ac0 0xddf640 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f00d804b800 tx=0x7f00d800f000).stop 2023-11-15T02:58:08.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00eff8f740 1 -- 172.21.15.121:0/3853078562 shutdown_connections 2023-11-15T02:58:08.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00eff8f740 1 --2- 172.21.15.121:0/3853078562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde5ac0 0xddf640 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00eff8f740 1 --2- 172.21.15.121:0/3853078562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xde4bd0 0xddf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00eff8f740 1 --2- 172.21.15.121:0/3853078562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe26930 0xe26d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00eff8f740 1 -- 172.21.15.121:0/3853078562 >> 172.21.15.121:0/3853078562 conn(0xbc4680 msgr2=0xe2c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:08.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00eff8f740 1 -- 172.21.15.121:0/3853078562 shutdown_connections 2023-11-15T02:58:08.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00eff8f740 1 -- 172.21.15.121:0/3853078562 wait complete. 2023-11-15T02:58:08.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00eff8f740 1 -- start start 2023-11-15T02:58:08.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00eff8f740 1 Processor -- start 2023-11-15T02:58:08.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00eff8f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 0xe0b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00eff8f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xde5ac0 0xe0e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00eff8f740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe26930 0xe247d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00eff8f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0xe25370 con 0xde4bd0 2023-11-15T02:58:08.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00eff8f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0xe3bf30 con 0xde5ac0 2023-11-15T02:58:08.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00eff8f740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0xe3c180 con 0xe26930 2023-11-15T02:58:08.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00eff8f740 10 client.0 ll_register_callbacks cb 0xd9c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:58:08.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00df7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 0xe0b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00df7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 0xe0b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50130/0 (socket says 172.21.15.121:50130) 2023-11-15T02:58:08.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00df7fe700 1 -- 172.21.15.121:0/4184813150 learned_addr learned my addr 172.21.15.121:0/4184813150 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:08.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00d6ffd700 1 --2- 172.21.15.121:0/4184813150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xde5ac0 0xe0e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00dffff700 1 --2- 172.21.15.121:0/4184813150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe26930 0xe247d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00df7fe700 1 -- 172.21.15.121:0/4184813150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe26930 msgr2=0xe247d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:08.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00df7fe700 1 --2- 172.21.15.121:0/4184813150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe26930 0xe247d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00df7fe700 1 -- 172.21.15.121:0/4184813150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xde5ac0 msgr2=0xe0e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:08.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00df7fe700 1 --2- 172.21.15.121:0/4184813150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xde5ac0 0xe0e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.271+0000 7f00df7fe700 1 -- 172.21.15.121:0/4184813150 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e00cb360 con 0xde4bd0 2023-11-15T02:58:08.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.275+0000 7f00df7fe700 1 --2- 172.21.15.121:0/4184813150 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 0xe0b250 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f00e0064eb0 tx=0x7f00d001f000).ready entity=mon.0 client_cookie=585b988a698a58c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:08.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.275+0000 7f00bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:58:08.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.275+0000 7f00bdffb700 1 -- 172.21.15.121:0/4184813150 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d002d070 con 0xde4bd0 2023-11-15T02:58:08.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.275+0000 7f00bdffb700 1 -- 172.21.15.121:0/4184813150 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f00d001ee00 con 0xde4bd0 2023-11-15T02:58:08.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.275+0000 7f00eff8f740 1 -- 172.21.15.121:0/4184813150 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xd5c230 con 0xde4bd0 2023-11-15T02:58:08.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.275+0000 7f00bdffb700 1 -- 172.21.15.121:0/4184813150 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00d0000d20 con 0xde4bd0 2023-11-15T02:58:08.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.275+0000 7f00eff8f740 10 client.5286 fetch_fsmap learned FSMap version 22 2023-11-15T02:58:08.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.275+0000 7f00eff8f740 1 -- 172.21.15.121:0/4184813150 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0xd5f7c0 con 0xde4bd0 2023-11-15T02:58:08.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.275+0000 7f00bdffb700 1 -- 172.21.15.121:0/4184813150 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xd5c230 con 0xde4bd0 2023-11-15T02:58:08.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.275+0000 7f00bdffb700 1 -- 172.21.15.121:0/4184813150 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 22) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f00d0030020 con 0xde4bd0 2023-11-15T02:58:08.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.275+0000 7f00eff8f740 10 client.5286 fetch_fsmap finished waiting for FSMap version 22 2023-11-15T02:58:08.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.275+0000 7f00eff8f740 10 client.5286 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-15T02:58:08.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.275+0000 7f00eff8f740 20 client.5286 populate_metadata read hostname 'smithi121' 2023-11-15T02:58:08.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.275+0000 7f00eff8f740 1 --2- 172.21.15.121:0/4184813150 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0xe3b950 0xde3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.275+0000 7f00eff8f740 4 client.5286 mds_command: new command op to 5178 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T02:58:08.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.275+0000 7f00eff8f740 1 -- 172.21.15.121:0/4184813150 --> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xd52780 con 0xe3b950 2023-11-15T02:58:08.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.275+0000 7f00d6ffd700 1 --2- 172.21.15.121:0/4184813150 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0xe3b950 0xde3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.275+0000 7f00d6ffd700 1 --2- 172.21.15.121:0/4184813150 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0xe3b950 0xde3430 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ee2f16f5726009eb server_cookie=ebd643e1d0b00960 in_seq=0 out_seq=0 2023-11-15T02:58:08.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.275+0000 7f00bdffb700 10 client.5286 ms_handle_connect on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:08.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.279+0000 7f00bdffb700 1 -- 172.21.15.121:0/4184813150 <== mds.0 v2:172.21.15.121:6836/3305580606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0xd52780 con 0xe3b950 2023-11-15T02:58:08.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.279+0000 7f00bdffb700 10 client.5286 handle_command_reply: tid=0 2023-11-15T02:58:08.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.279+0000 7f00eff8f740 1 client.5286 shutdown 2023-11-15T02:58:08.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.279+0000 7f00eff8f740 1 -- 172.21.15.121:0/4184813150 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 msgr2=0xe0b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.279+0000 7f00eff8f740 1 --2- 172.21.15.121:0/4184813150 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 0xe0b250 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f00e0064eb0 tx=0x7f00d001f000).stop 2023-11-15T02:58:08.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.279+0000 7f00eff8f740 1 -- 172.21.15.121:0/4184813150 shutdown_connections 2023-11-15T02:58:08.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.279+0000 7f00eff8f740 1 --2- 172.21.15.121:0/4184813150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe26930 0xe247d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.279+0000 7f00eff8f740 1 --2- 172.21.15.121:0/4184813150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xde5ac0 0xe0e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.279+0000 7f00eff8f740 1 --2- 172.21.15.121:0/4184813150 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0xe3b950 0xde3430 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.279+0000 7f00eff8f740 1 --2- 172.21.15.121:0/4184813150 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 0xe0b250 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.279+0000 7f00eff8f740 1 -- 172.21.15.121:0/4184813150 >> 172.21.15.121:0/4184813150 conn(0xbc4680 msgr2=0xe26d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:08.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.279+0000 7f00bdffb700 0 client.5286 ms_handle_reset on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:08.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.279+0000 7f00eff8f740 1 -- 172.21.15.121:0/4184813150 shutdown_connections 2023-11-15T02:58:08.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.279+0000 7f00eff8f740 1 -- 172.21.15.121:0/4184813150 wait complete. 2023-11-15T02:58:08.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.279+0000 7f00eff8f740 20 client.5286 trim_cache size 0 max 16384 2023-11-15T02:58:08.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.295+0000 7f00eff8f740 1 Processor -- start 2023-11-15T02:58:08.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.295+0000 7f00eff8f740 1 -- start start 2023-11-15T02:58:08.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00eff8f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 0xe2c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00eff8f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xde5ac0 0xe2c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00eff8f740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe26930 0xe0caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00eff8f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0xdb0df0 con 0xde4bd0 2023-11-15T02:58:08.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00eff8f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0xe0ac90 con 0xde5ac0 2023-11-15T02:58:08.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00eff8f740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0xd6c970 con 0xe26930 2023-11-15T02:58:08.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00df7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 0xe2c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00d6ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xde5ac0 0xe2c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00df7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 0xe2c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50140/0 (socket says 172.21.15.121:50140) 2023-11-15T02:58:08.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00df7fe700 1 -- 172.21.15.121:0/1093800674 learned_addr learned my addr 172.21.15.121:0/1093800674 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:08.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00dffff700 1 --2- 172.21.15.121:0/1093800674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe26930 0xe0caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00d6ffd700 1 -- 172.21.15.121:0/1093800674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe26930 msgr2=0xe0caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:08.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00d6ffd700 1 --2- 172.21.15.121:0/1093800674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe26930 0xe0caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00d6ffd700 1 -- 172.21.15.121:0/1093800674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 msgr2=0xe2c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:08.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00d6ffd700 1 --2- 172.21.15.121:0/1093800674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 0xe2c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00d6ffd700 1 -- 172.21.15.121:0/1093800674 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xd5a9f0 con 0xde5ac0 2023-11-15T02:58:08.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00d6ffd700 1 --2- 172.21.15.121:0/1093800674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xde5ac0 0xe2c5f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f00e014de30 tx=0x7f00c801c040).ready entity=mon.1 client_cookie=e064c069eb99534d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:08.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00d57fa700 1 -- 172.21.15.121:0/1093800674 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c8003de0 con 0xde5ac0 2023-11-15T02:58:08.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00d57fa700 1 -- 172.21.15.121:0/1093800674 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c801a110 con 0xde5ac0 2023-11-15T02:58:08.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00d57fa700 1 -- 172.21.15.121:0/1093800674 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c801ab20 con 0xde5ac0 2023-11-15T02:58:08.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00eff8f740 1 -- 172.21.15.121:0/1093800674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xde5ac0 msgr2=0xe2c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00eff8f740 1 --2- 172.21.15.121:0/1093800674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xde5ac0 0xe2c5f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f00e014de30 tx=0x7f00c801c040).stop 2023-11-15T02:58:08.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00eff8f740 1 -- 172.21.15.121:0/1093800674 shutdown_connections 2023-11-15T02:58:08.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00eff8f740 1 --2- 172.21.15.121:0/1093800674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe26930 0xe0caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00eff8f740 1 --2- 172.21.15.121:0/1093800674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xde5ac0 0xe2c5f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00eff8f740 1 --2- 172.21.15.121:0/1093800674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 0xe2c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00eff8f740 1 -- 172.21.15.121:0/1093800674 >> 172.21.15.121:0/1093800674 conn(0xbc4680 msgr2=0xe27ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:08.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00eff8f740 1 -- 172.21.15.121:0/1093800674 shutdown_connections 2023-11-15T02:58:08.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.299+0000 7f00eff8f740 1 -- 172.21.15.121:0/1093800674 wait complete. 2023-11-15T02:58:08.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00eff8f740 1 -- start start 2023-11-15T02:58:08.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00eff8f740 1 Processor -- start 2023-11-15T02:58:08.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00eff8f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 0xe3f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00eff8f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe26930 0xe0c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00eff8f740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe2c810 0xe392c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00eff8f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0xe3ed80 con 0xde4bd0 2023-11-15T02:58:08.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00eff8f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0xe40350 con 0xe26930 2023-11-15T02:58:08.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00eff8f740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0xe270d0 con 0xe2c810 2023-11-15T02:58:08.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00eff8f740 10 client.0 ll_register_callbacks cb 0x7f00d0011ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:58:08.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00df7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 0xe3f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00df7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 0xe3f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50144/0 (socket says 172.21.15.121:50144) 2023-11-15T02:58:08.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00df7fe700 1 -- 172.21.15.121:0/823035919 learned_addr learned my addr 172.21.15.121:0/823035919 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:08.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00dffff700 1 --2- 172.21.15.121:0/823035919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe2c810 0xe392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00d6ffd700 1 --2- 172.21.15.121:0/823035919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe26930 0xe0c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00df7fe700 1 -- 172.21.15.121:0/823035919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe2c810 msgr2=0xe392c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:08.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00df7fe700 1 --2- 172.21.15.121:0/823035919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe2c810 0xe392c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00df7fe700 1 -- 172.21.15.121:0/823035919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe26930 msgr2=0xe0c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:08.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00df7fe700 1 --2- 172.21.15.121:0/823035919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe26930 0xe0c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00df7fe700 1 -- 172.21.15.121:0/823035919 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c800f040 con 0xde4bd0 2023-11-15T02:58:08.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00d6ffd700 1 --2- 172.21.15.121:0/823035919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe26930 0xe0c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:58:08.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00df7fe700 1 --2- 172.21.15.121:0/823035919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 0xe3f940 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f00d0006c00 tx=0x7f00d0029000).ready entity=mon.0 client_cookie=2d5da146869863b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:08.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:58:08.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00bdffb700 1 -- 172.21.15.121:0/823035919 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d002d070 con 0xde4bd0 2023-11-15T02:58:08.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00bdffb700 1 -- 172.21.15.121:0/823035919 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f00d0039600 con 0xde4bd0 2023-11-15T02:58:08.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00eff8f740 1 -- 172.21.15.121:0/823035919 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00d801de00 con 0xde4bd0 2023-11-15T02:58:08.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.303+0000 7f00bdffb700 1 -- 172.21.15.121:0/823035919 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00d00397c0 con 0xde4bd0 2023-11-15T02:58:08.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 10 client.5292 fetch_fsmap learned FSMap version 22 2023-11-15T02:58:08.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 1 -- 172.21.15.121:0/823035919 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0xd5a740 con 0xde4bd0 2023-11-15T02:58:08.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00bdffb700 1 -- 172.21.15.121:0/823035919 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d801de00 con 0xde4bd0 2023-11-15T02:58:08.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00bdffb700 1 -- 172.21.15.121:0/823035919 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 22) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f00d0030070 con 0xde4bd0 2023-11-15T02:58:08.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 10 client.5292 fetch_fsmap finished waiting for FSMap version 22 2023-11-15T02:58:08.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 10 client.5292 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-15T02:58:08.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 20 client.5292 populate_metadata read hostname 'smithi121' 2023-11-15T02:58:08.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 1 --2- 172.21.15.121:0/823035919 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0xe397b0 0xe39bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:08.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 4 client.5292 mds_command: new command op to 5178 tid=0[{"prefix": "scrub status"}] 2023-11-15T02:58:08.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 1 -- 172.21.15.121:0/823035919 --> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0xc1b610 con 0xe397b0 2023-11-15T02:58:08.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00d6ffd700 1 --2- 172.21.15.121:0/823035919 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0xe397b0 0xe39bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:08.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00bdffb700 10 client.5292 ms_handle_connect on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:08.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00d6ffd700 1 --2- 172.21.15.121:0/823035919 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0xe397b0 0xe39bc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=8184bfdb5e8ed6b3 server_cookie=eaeeb6f010a1747e in_seq=0 out_seq=0 2023-11-15T02:58:08.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00bdffb700 1 -- 172.21.15.121:0/823035919 <== mds.0 v2:172.21.15.121:6836/3305580606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+256 (crc 0 0 0) 0xc1b610 con 0xe397b0 2023-11-15T02:58:08.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00bdffb700 10 client.5292 handle_command_reply: tid=0 2023-11-15T02:58:08.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 1 client.5292 shutdown 2023-11-15T02:58:08.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 1 -- 172.21.15.121:0/823035919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 msgr2=0xe3f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 1 --2- 172.21.15.121:0/823035919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 0xe3f940 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f00d0006c00 tx=0x7f00d0029000).stop 2023-11-15T02:58:08.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 1 -- 172.21.15.121:0/823035919 shutdown_connections 2023-11-15T02:58:08.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 1 --2- 172.21.15.121:0/823035919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe2c810 0xe392c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 1 --2- 172.21.15.121:0/823035919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe26930 0xe0c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 1 --2- 172.21.15.121:0/823035919 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0xe397b0 0xe39bc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 1 --2- 172.21.15.121:0/823035919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xde4bd0 0xe3f940 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 1 -- 172.21.15.121:0/823035919 >> 172.21.15.121:0/823035919 conn(0xbc4680 msgr2=0xe2c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:08.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00bdffb700 0 client.5292 ms_handle_reset on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:08.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 1 -- 172.21.15.121:0/823035919 shutdown_connections 2023-11-15T02:58:08.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 1 -- 172.21.15.121:0/823035919 wait complete. 2023-11-15T02:58:08.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00eff8f740 20 client.5292 trim_cache size 0 max 16384 2023-11-15T02:58:08.328 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T02:58:08.328 INFO:teuthology.orchestra.run.smithi121.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-15T02:58:08.328 INFO:teuthology.orchestra.run.smithi121.stdout: "scrubs": { 2023-11-15T02:58:08.328 INFO:teuthology.orchestra.run.smithi121.stdout: "4c60cbb0-674a-4f3a-8f8f-3258f75817b1": { 2023-11-15T02:58:08.329 INFO:teuthology.orchestra.run.smithi121.stdout: "path": "/subdir", 2023-11-15T02:58:08.329 INFO:teuthology.orchestra.run.smithi121.stdout: "tag": "4c60cbb0-674a-4f3a-8f8f-3258f75817b1", 2023-11-15T02:58:08.329 INFO:teuthology.orchestra.run.smithi121.stdout: "options": "repair" 2023-11-15T02:58:08.329 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:08.329 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:08.330 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T02:58:08.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00e49b9700 1 -- 172.21.15.121:0/893728458 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f00c005a510 msgr2=0x7f00c005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00e49b9700 1 --2- 172.21.15.121:0/893728458 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f00c005a510 0x7f00c005c9d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f00e00c1330 tx=0x7f00d0016000).stop 2023-11-15T02:58:08.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00e49b9700 1 -- 172.21.15.121:0/893728458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00e014e380 msgr2=0x7f00e00cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:08.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.307+0000 7f00e49b9700 1 --2- 172.21.15.121:0/893728458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00e014e380 0x7f00e00cae70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f00d800cb60 tx=0x7f00d8001e50).stop 2023-11-15T02:58:08.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.311+0000 7f00e49b9700 1 -- 172.21.15.121:0/893728458 shutdown_connections 2023-11-15T02:58:08.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.311+0000 7f00e49b9700 1 --2- 172.21.15.121:0/893728458 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f00c005a510 0x7f00c005c9d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.311+0000 7f00e49b9700 1 --2- 172.21.15.121:0/893728458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00e014e380 0x7f00e00cae70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.311+0000 7f00e49b9700 1 --2- 172.21.15.121:0/893728458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00e00c6040 0x7f00e00ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.311+0000 7f00e49b9700 1 --2- 172.21.15.121:0/893728458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00e00ad270 0x7f00e014d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:08.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.311+0000 7f00e49b9700 1 -- 172.21.15.121:0/893728458 >> 172.21.15.121:0/893728458 conn(0x7f00e0067a60 msgr2=0x7f00e014c270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:08.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.311+0000 7f00e49b9700 1 -- 172.21.15.121:0/893728458 shutdown_connections 2023-11-15T02:58:08.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:08.311+0000 7f00e49b9700 1 -- 172.21.15.121:0/893728458 wait complete. 2023-11-15T02:58:08.335 INFO:tasks.cephfs.filesystem:scrub status for tag:4c60cbb0-674a-4f3a-8f8f-3258f75817b1 - {'path': '/subdir', 'tag': '4c60cbb0-674a-4f3a-8f8f-3258f75817b1', 'options': 'repair'} 2023-11-15T02:58:10.150 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T02:58:10.158 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T02:58:10.998 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T02:58:10.991+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-11-15T02:58:38.337 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2023-11-15T02:58:38.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.498+0000 7f8ebeff9700 1 Processor -- start 2023-11-15T02:58:38.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.498+0000 7f8ebeff9700 1 -- start start 2023-11-15T02:58:38.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.502+0000 7f8ebeff9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8eb80acd00 0x7f8eb80ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.502+0000 7f8ebeff9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8eb80c5c70 0x7f8eb81536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.502+0000 7f8ebeff9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb8065ea0 con 0x7f8eb8066330 2023-11-15T02:58:38.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.502+0000 7f8ebeff9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb8066bb0 con 0x7f8eb80acd00 2023-11-15T02:58:38.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.502+0000 7f8ebeff9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb8065890 con 0x7f8eb80c5c70 2023-11-15T02:58:38.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.502+0000 7f8ebdff7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8eb80acd00 0x7f8eb80ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.502+0000 7f8ebdff7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8eb80acd00 0x7f8eb80ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32848/0 (socket says 172.21.15.121:32848) 2023-11-15T02:58:38.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.502+0000 7f8ebdff7700 1 -- 172.21.15.121:0/3996827329 learned_addr learned my addr 172.21.15.121:0/3996827329 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:38.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.502+0000 7f8ebd7f6700 1 --2- 172.21.15.121:0/3996827329 >> v2:172.21.15.190:3300/0 conn(0x7f8eb80c5c70 0x7f8eb81536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.502+0000 7f8ebdff7700 1 -- 172.21.15.121:0/3996827329 >> v2:172.21.15.190:3300/0 conn(0x7f8eb80c5c70 msgr2=0x7f8eb81536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:38.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.502+0000 7f8ebdff7700 1 --2- 172.21.15.121:0/3996827329 >> v2:172.21.15.190:3300/0 conn(0x7f8eb80c5c70 0x7f8eb81536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.502+0000 7f8ebdff7700 1 -- 172.21.15.121:0/3996827329 >> v1:172.21.15.190:6789/0 conn(0x7f8eb8066330 legacy=0x7f8eb8153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:38.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.502+0000 7f8ebdff7700 1 -- 172.21.15.121:0/3996827329 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb80c64a0 con 0x7f8eb80acd00 2023-11-15T02:58:38.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.502+0000 7f8ebd7f6700 1 --2- 172.21.15.121:0/3996827329 >> v2:172.21.15.190:3300/0 conn(0x7f8eb80c5c70 0x7f8eb81536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:38.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.502+0000 7f8ebdff7700 1 --2- 172.21.15.121:0/3996827329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8eb80acd00 0x7f8eb80ad0e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f8eb80cb9c0 tx=0x7f8eb40090f0).ready entity=mon.0 client_cookie=3be3cf79e30627ff server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:38.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.506+0000 7f8ebcff5700 1 -- 172.21.15.121:0/3996827329 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb400b600 con 0x7f8eb80acd00 2023-11-15T02:58:38.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.506+0000 7f8ebcff5700 1 -- 172.21.15.121:0/3996827329 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb400b760 con 0x7f8eb80acd00 2023-11-15T02:58:38.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.506+0000 7f8ebcff5700 1 -- 172.21.15.121:0/3996827329 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb400bae0 con 0x7f8eb80acd00 2023-11-15T02:58:38.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.506+0000 7f8ebeff9700 1 -- 172.21.15.121:0/3996827329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8eb80acd00 msgr2=0x7f8eb80ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:38.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.506+0000 7f8ebeff9700 1 --2- 172.21.15.121:0/3996827329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8eb80acd00 0x7f8eb80ad0e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f8eb80cb9c0 tx=0x7f8eb40090f0).stop 2023-11-15T02:58:38.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.506+0000 7f8ebeff9700 1 -- 172.21.15.121:0/3996827329 shutdown_connections 2023-11-15T02:58:38.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.506+0000 7f8ebeff9700 1 --2- 172.21.15.121:0/3996827329 >> v2:172.21.15.190:3300/0 conn(0x7f8eb80c5c70 0x7f8eb81536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.506+0000 7f8ebeff9700 1 --2- 172.21.15.121:0/3996827329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8eb80acd00 0x7f8eb80ad0e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.506+0000 7f8ebeff9700 1 -- 172.21.15.121:0/3996827329 >> 172.21.15.121:0/3996827329 conn(0x7f8eb8063fc0 msgr2=0x7f8eb8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:38.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.506+0000 7f8ebeff9700 1 -- 172.21.15.121:0/3996827329 shutdown_connections 2023-11-15T02:58:38.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.506+0000 7f8ebeff9700 1 -- 172.21.15.121:0/3996827329 wait complete. 2023-11-15T02:58:38.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebeff9700 1 Processor -- start 2023-11-15T02:58:38.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebeff9700 1 -- start start 2023-11-15T02:58:38.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebeff9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8eb8066330 0x7f8eb80c1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebeff9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8eb80acd00 0x7f8eb814bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebeff9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8eb80c5c70 0x7f8eb80c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebeff9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb814cbf0 con 0x7f8eb8066330 2023-11-15T02:58:38.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebeff9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80c8d90 con 0x7f8eb80c5c70 2023-11-15T02:58:38.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebeff9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80c9070 con 0x7f8eb80acd00 2023-11-15T02:58:38.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebdff7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8eb8066330 0x7f8eb80c1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebdff7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8eb8066330 0x7f8eb80c1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32860/0 (socket says 172.21.15.121:32860) 2023-11-15T02:58:38.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebdff7700 1 -- 172.21.15.121:0/854101107 learned_addr learned my addr 172.21.15.121:0/854101107 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:38.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebd7f6700 1 --2- 172.21.15.121:0/854101107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8eb80acd00 0x7f8eb814bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebe7f8700 1 --2- 172.21.15.121:0/854101107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8eb80c5c70 0x7f8eb80c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebdff7700 1 -- 172.21.15.121:0/854101107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8eb80acd00 msgr2=0x7f8eb814bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:38.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebdff7700 1 --2- 172.21.15.121:0/854101107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8eb80acd00 0x7f8eb814bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebdff7700 1 -- 172.21.15.121:0/854101107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8eb80c5c70 msgr2=0x7f8eb80c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:38.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebdff7700 1 --2- 172.21.15.121:0/854101107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8eb80c5c70 0x7f8eb80c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebdff7700 1 -- 172.21.15.121:0/854101107 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb4009310 con 0x7f8eb8066330 2023-11-15T02:58:38.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebd7f6700 1 --2- 172.21.15.121:0/854101107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8eb80acd00 0x7f8eb814bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:38.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebdff7700 1 --2- 172.21.15.121:0/854101107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8eb8066330 0x7f8eb80c1a30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f8eb8064f20 tx=0x7f8eb4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:38.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8eadffb700 1 -- 172.21.15.121:0/854101107 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb400b8d0 con 0x7f8eb8066330 2023-11-15T02:58:38.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8eadffb700 1 -- 172.21.15.121:0/854101107 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb400ba30 con 0x7f8eb8066330 2023-11-15T02:58:38.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.510+0000 7f8ebeff9700 1 -- 172.21.15.121:0/854101107 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb80c9350 con 0x7f8eb8066330 2023-11-15T02:58:38.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.514+0000 7f8eadffb700 1 -- 172.21.15.121:0/854101107 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb401c670 con 0x7f8eb8066330 2023-11-15T02:58:38.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.514+0000 7f8ebeff9700 1 -- 172.21.15.121:0/854101107 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb80c1f20 con 0x7f8eb8066330 2023-11-15T02:58:38.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.514+0000 7f8eadffb700 1 -- 172.21.15.121:0/854101107 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8eb401caa0 con 0x7f8eb8066330 2023-11-15T02:58:38.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.518+0000 7f8eadffb700 1 --2- 172.21.15.121:0/854101107 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8e9c05a510 0x7f8e9c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.518+0000 7f8eadffb700 1 -- 172.21.15.121:0/854101107 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f8eb40761f0 con 0x7f8eb8066330 2023-11-15T02:58:38.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.518+0000 7f8ebd7f6700 1 --2- 172.21.15.121:0/854101107 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8e9c05a510 0x7f8e9c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.518+0000 7f8eca5ef740 1 Processor -- start 2023-11-15T02:58:38.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.518+0000 7f8eca5ef740 1 -- start start 2023-11-15T02:58:38.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.518+0000 7f8ebd7f6700 1 --2- 172.21.15.121:0/854101107 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8e9c05a510 0x7f8e9c05c9d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f8ea8006d80 tx=0x7f8ea8008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:38.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.518+0000 7f8eca5ef740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1df1930 0x1df1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.518+0000 7f8eca5ef740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1dafbd0 0x1daa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.518+0000 7f8eca5ef740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1db0ac0 0x1daa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.518+0000 7f8eca5ef740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d0d540 con 0x1df1930 2023-11-15T02:58:38.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.518+0000 7f8eca5ef740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cd0e90 con 0x1db0ac0 2023-11-15T02:58:38.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.518+0000 7f8eca5ef740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1cd6d90 con 0x1dafbd0 2023-11-15T02:58:38.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.518+0000 7f8ebdff7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1df1930 0x1df1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.518+0000 7f8ebdff7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1df1930 0x1df1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32876/0 (socket says 172.21.15.121:32876) 2023-11-15T02:58:38.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.518+0000 7f8ebdff7700 1 -- 172.21.15.121:0/3709735776 learned_addr learned my addr 172.21.15.121:0/3709735776 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:38.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8ebd7f6700 1 --2- 172.21.15.121:0/3709735776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1dafbd0 0x1daa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8ebe7f8700 1 --2- 172.21.15.121:0/3709735776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1db0ac0 0x1daa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8ebdff7700 1 -- 172.21.15.121:0/3709735776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1dafbd0 msgr2=0x1daa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:38.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8ebdff7700 1 --2- 172.21.15.121:0/3709735776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1dafbd0 0x1daa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8ebdff7700 1 -- 172.21.15.121:0/3709735776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1db0ac0 msgr2=0x1daa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:38.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8ebdff7700 1 --2- 172.21.15.121:0/3709735776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1db0ac0 0x1daa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8ebdff7700 1 -- 172.21.15.121:0/3709735776 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1d3b4c0 con 0x1df1930 2023-11-15T02:58:38.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8ebdff7700 1 --2- 172.21.15.121:0/3709735776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1df1930 0x1df1d10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f8eb4007fe0 tx=0x7f8eb400e000).ready entity=mon.0 client_cookie=2c26ba2f9478d0d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:38.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8ea2ffd700 1 -- 172.21.15.121:0/3709735776 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb404cee0 con 0x1df1930 2023-11-15T02:58:38.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8ea2ffd700 1 -- 172.21.15.121:0/3709735776 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb404d520 con 0x1df1930 2023-11-15T02:58:38.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8ea2ffd700 1 -- 172.21.15.121:0/3709735776 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb401cd50 con 0x1df1930 2023-11-15T02:58:38.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8eca5ef740 1 -- 172.21.15.121:0/3709735776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1df1930 msgr2=0x1df1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:38.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/3709735776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1df1930 0x1df1d10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f8eb4007fe0 tx=0x7f8eb400e000).stop 2023-11-15T02:58:38.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8eca5ef740 1 -- 172.21.15.121:0/3709735776 shutdown_connections 2023-11-15T02:58:38.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/3709735776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1db0ac0 0x1daa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/3709735776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1dafbd0 0x1daa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/3709735776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1df1930 0x1df1d10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8eca5ef740 1 -- 172.21.15.121:0/3709735776 >> 172.21.15.121:0/3709735776 conn(0x1b8f680 msgr2=0x1df7180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:38.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8eca5ef740 1 -- 172.21.15.121:0/3709735776 shutdown_connections 2023-11-15T02:58:38.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8eca5ef740 1 -- 172.21.15.121:0/3709735776 wait complete. 2023-11-15T02:58:38.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8eca5ef740 1 -- start start 2023-11-15T02:58:38.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.522+0000 7f8eca5ef740 1 Processor -- start 2023-11-15T02:58:38.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8eca5ef740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1dafbd0 0x1def1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8eca5ef740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1db0ac0 0x1daa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8eca5ef740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1df1930 0x1daae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8eca5ef740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dab320 con 0x1db0ac0 2023-11-15T02:58:38.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8eca5ef740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dda670 con 0x1dafbd0 2023-11-15T02:58:38.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8eca5ef740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1def9b0 con 0x1df1930 2023-11-15T02:58:38.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8eca5ef740 10 client.0 ll_register_callbacks cb 0x1d675d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:58:38.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8ebd7f6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1db0ac0 0x1daa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8ebd7f6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1db0ac0 0x1daa940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32878/0 (socket says 172.21.15.121:32878) 2023-11-15T02:58:38.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8ebd7f6700 1 -- 172.21.15.121:0/3756570992 learned_addr learned my addr 172.21.15.121:0/3756570992 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:38.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8ebdff7700 1 --2- 172.21.15.121:0/3756570992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1dafbd0 0x1def1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8ebe7f8700 1 --2- 172.21.15.121:0/3756570992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1df1930 0x1daae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8ebd7f6700 1 -- 172.21.15.121:0/3756570992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1df1930 msgr2=0x1daae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:38.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8ebd7f6700 1 --2- 172.21.15.121:0/3756570992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1df1930 0x1daae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8ebd7f6700 1 -- 172.21.15.121:0/3756570992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1dafbd0 msgr2=0x1def1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:38.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8ebd7f6700 1 --2- 172.21.15.121:0/3756570992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1dafbd0 0x1def1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8ebd7f6700 1 -- 172.21.15.121:0/3756570992 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb80c1f20 con 0x1db0ac0 2023-11-15T02:58:38.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8ebdff7700 1 --2- 172.21.15.121:0/3756570992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1dafbd0 0x1def1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:58:38.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8ebe7f8700 1 --2- 172.21.15.121:0/3756570992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1df1930 0x1daae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:58:38.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8ebd7f6700 1 --2- 172.21.15.121:0/3756570992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1db0ac0 0x1daa940 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f8ea8013550 tx=0x7f8ea8016040).ready entity=mon.0 client_cookie=f569ce261516d6da server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:38.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8e97fff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:58:38.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8e97fff700 1 -- 172.21.15.121:0/3756570992 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8012570 con 0x1db0ac0 2023-11-15T02:58:38.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8eca5ef740 1 -- 172.21.15.121:0/3756570992 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1d27230 con 0x1db0ac0 2023-11-15T02:58:38.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8e97fff700 1 -- 172.21.15.121:0/3756570992 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f8ea80126d0 con 0x1db0ac0 2023-11-15T02:58:38.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8e97fff700 1 -- 172.21.15.121:0/3756570992 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ea8012aa0 con 0x1db0ac0 2023-11-15T02:58:38.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8eca5ef740 10 client.5304 fetch_fsmap learned FSMap version 23 2023-11-15T02:58:38.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.526+0000 7f8eca5ef740 1 -- 172.21.15.121:0/3756570992 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x1d2a7c0 con 0x1db0ac0 2023-11-15T02:58:38.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8e97fff700 1 -- 172.21.15.121:0/3756570992 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1d27230 con 0x1db0ac0 2023-11-15T02:58:38.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8e97fff700 1 -- 172.21.15.121:0/3756570992 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 23) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8ea8012c00 con 0x1db0ac0 2023-11-15T02:58:38.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8eca5ef740 10 client.5304 fetch_fsmap finished waiting for FSMap version 23 2023-11-15T02:58:38.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8eca5ef740 10 client.5304 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-15T02:58:38.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8eca5ef740 20 client.5304 populate_metadata read hostname 'smithi121' 2023-11-15T02:58:38.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/3756570992 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x1ddcb50 0x1ddcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8eca5ef740 4 client.5304 mds_command: new command op to 5178 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T02:58:38.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8eca5ef740 1 -- 172.21.15.121:0/3756570992 --> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1d1d780 con 0x1ddcb50 2023-11-15T02:58:38.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8ebdff7700 1 --2- 172.21.15.121:0/3756570992 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x1ddcb50 0x1ddcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8e97fff700 10 client.5304 ms_handle_connect on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:38.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8ebdff7700 1 --2- 172.21.15.121:0/3756570992 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x1ddcb50 0x1ddcf60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=1bf44fbd196d639b server_cookie=d248afd4fb003ac2 in_seq=0 out_seq=0 2023-11-15T02:58:38.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8e97fff700 1 -- 172.21.15.121:0/3756570992 <== mds.0 v2:172.21.15.121:6836/3305580606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x1d1d780 con 0x1ddcb50 2023-11-15T02:58:38.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8e97fff700 10 client.5304 handle_command_reply: tid=0 2023-11-15T02:58:38.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8eca5ef740 1 client.5304 shutdown 2023-11-15T02:58:38.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8eca5ef740 1 -- 172.21.15.121:0/3756570992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1db0ac0 msgr2=0x1daa940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:38.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/3756570992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1db0ac0 0x1daa940 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f8ea8013550 tx=0x7f8ea8016040).stop 2023-11-15T02:58:38.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8eca5ef740 1 -- 172.21.15.121:0/3756570992 shutdown_connections 2023-11-15T02:58:38.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/3756570992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1df1930 0x1daae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/3756570992 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x1ddcb50 0x1ddcf60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/3756570992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1db0ac0 0x1daa940 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8e97fff700 0 client.5304 ms_handle_reset on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:38.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/3756570992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1dafbd0 0x1def1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8eca5ef740 1 -- 172.21.15.121:0/3756570992 >> 172.21.15.121:0/3756570992 conn(0x1b8f680 msgr2=0x1df6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:38.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8eca5ef740 1 -- 172.21.15.121:0/3756570992 shutdown_connections 2023-11-15T02:58:38.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8eca5ef740 1 -- 172.21.15.121:0/3756570992 wait complete. 2023-11-15T02:58:38.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.530+0000 7f8eca5ef740 20 client.5304 trim_cache size 0 max 16384 2023-11-15T02:58:38.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.550+0000 7f8eca5ef740 1 Processor -- start 2023-11-15T02:58:38.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.550+0000 7f8eca5ef740 1 -- start start 2023-11-15T02:58:38.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.550+0000 7f8eca5ef740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1dafbd0 0x1dae230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.550+0000 7f8eca5ef740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1df1930 0x1ddbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.550+0000 7f8eca5ef740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1dabf70 0x1dd9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.550+0000 7f8eca5ef740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e02e30 con 0x1dafbd0 2023-11-15T02:58:38.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.550+0000 7f8eca5ef740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1daec60 con 0x1dabf70 2023-11-15T02:58:38.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.550+0000 7f8eca5ef740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1df7ce0 con 0x1df1930 2023-11-15T02:58:38.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.550+0000 7f8ebdff7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1dafbd0 0x1dae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.550+0000 7f8ebdff7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1dafbd0 0x1dae230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32882/0 (socket says 172.21.15.121:32882) 2023-11-15T02:58:38.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.550+0000 7f8ebdff7700 1 -- 172.21.15.121:0/2485336936 learned_addr learned my addr 172.21.15.121:0/2485336936 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:38.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8ebd7f6700 1 --2- 172.21.15.121:0/2485336936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1df1930 0x1ddbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8ebe7f8700 1 --2- 172.21.15.121:0/2485336936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1dabf70 0x1dd9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8ebdff7700 1 -- 172.21.15.121:0/2485336936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1df1930 msgr2=0x1ddbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:38.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8ebdff7700 1 --2- 172.21.15.121:0/2485336936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1df1930 0x1ddbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8ebdff7700 1 -- 172.21.15.121:0/2485336936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1dabf70 msgr2=0x1dd9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:38.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8ebdff7700 1 --2- 172.21.15.121:0/2485336936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1dabf70 0x1dd9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8ebdff7700 1 -- 172.21.15.121:0/2485336936 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb8159630 con 0x1dafbd0 2023-11-15T02:58:38.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8ebdff7700 1 --2- 172.21.15.121:0/2485336936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1dafbd0 0x1dae230 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f8eb404b8d0 tx=0x7f8eb4031000).ready entity=mon.0 client_cookie=8f24f7584b442775 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:38.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8ea37fe700 1 -- 172.21.15.121:0/2485336936 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb404b920 con 0x1dafbd0 2023-11-15T02:58:38.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8ea37fe700 1 -- 172.21.15.121:0/2485336936 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb401cd50 con 0x1dafbd0 2023-11-15T02:58:38.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8ea37fe700 1 -- 172.21.15.121:0/2485336936 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4000bc0 con 0x1dafbd0 2023-11-15T02:58:38.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8eca5ef740 1 -- 172.21.15.121:0/2485336936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1dafbd0 msgr2=0x1dae230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:38.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/2485336936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1dafbd0 0x1dae230 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f8eb404b8d0 tx=0x7f8eb4031000).stop 2023-11-15T02:58:38.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8eca5ef740 1 -- 172.21.15.121:0/2485336936 shutdown_connections 2023-11-15T02:58:38.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/2485336936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1dabf70 0x1dd9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/2485336936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1df1930 0x1ddbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/2485336936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1dafbd0 0x1dae230 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8eca5ef740 1 -- 172.21.15.121:0/2485336936 >> 172.21.15.121:0/2485336936 conn(0x1b8f680 msgr2=0x1df3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:38.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8eca5ef740 1 -- 172.21.15.121:0/2485336936 shutdown_connections 2023-11-15T02:58:38.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8eca5ef740 1 -- 172.21.15.121:0/2485336936 wait complete. 2023-11-15T02:58:38.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8eca5ef740 1 -- start start 2023-11-15T02:58:38.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.554+0000 7f8eca5ef740 1 Processor -- start 2023-11-15T02:58:38.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8eca5ef740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1dabf70 0x1da9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8eca5ef740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1dafbd0 0x1daa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8eca5ef740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1df1930 0x1ddb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8eca5ef740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1df7f30 con 0x1df1930 2023-11-15T02:58:38.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8eca5ef740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1daa860 con 0x1dabf70 2023-11-15T02:58:38.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8eca5ef740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e135a0 con 0x1dafbd0 2023-11-15T02:58:38.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8eca5ef740 10 client.0 ll_register_callbacks cb 0x7f8ea800fbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:58:38.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8ebe7f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1df1930 0x1ddb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8ebe7f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1df1930 0x1ddb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32884/0 (socket says 172.21.15.121:32884) 2023-11-15T02:58:38.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8ebe7f8700 1 -- 172.21.15.121:0/1575057824 learned_addr learned my addr 172.21.15.121:0/1575057824 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:38.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8ebdff7700 1 --2- 172.21.15.121:0/1575057824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1dabf70 0x1da9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8ebd7f6700 1 --2- 172.21.15.121:0/1575057824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1dafbd0 0x1daa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8ebe7f8700 1 -- 172.21.15.121:0/1575057824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1dafbd0 msgr2=0x1daa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:38.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8ebe7f8700 1 --2- 172.21.15.121:0/1575057824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1dafbd0 0x1daa370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8ebe7f8700 1 -- 172.21.15.121:0/1575057824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1dabf70 msgr2=0x1da9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:38.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8ebe7f8700 1 --2- 172.21.15.121:0/1575057824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1dabf70 0x1da9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8ebe7f8700 1 -- 172.21.15.121:0/1575057824 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1d3b4c0 con 0x1df1930 2023-11-15T02:58:38.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8ebdff7700 1 --2- 172.21.15.121:0/1575057824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1dabf70 0x1da9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:38.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8ebe7f8700 1 --2- 172.21.15.121:0/1575057824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1df1930 0x1ddb180 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f8eb0013820 tx=0x7f8eb0009040).ready entity=mon.0 client_cookie=6716493933eb0776 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:38.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8e97fff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:58:38.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8e97fff700 1 -- 172.21.15.121:0/1575057824 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0002900 con 0x1df1930 2023-11-15T02:58:38.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8eca5ef740 1 -- 172.21.15.121:0/1575057824 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8eb404d520 con 0x1df1930 2023-11-15T02:58:38.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8e97fff700 1 -- 172.21.15.121:0/1575057824 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f8eb0015630 con 0x1df1930 2023-11-15T02:58:38.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8e97fff700 1 -- 172.21.15.121:0/1575057824 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8eb0015a00 con 0x1df1930 2023-11-15T02:58:38.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8eca5ef740 10 client.5310 fetch_fsmap learned FSMap version 23 2023-11-15T02:58:38.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8eca5ef740 1 -- 172.21.15.121:0/1575057824 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x1d259f0 con 0x1df1930 2023-11-15T02:58:38.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8e97fff700 1 -- 172.21.15.121:0/1575057824 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb404d520 con 0x1df1930 2023-11-15T02:58:38.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8e97fff700 1 -- 172.21.15.121:0/1575057824 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 23) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8eb0015b60 con 0x1df1930 2023-11-15T02:58:38.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8eca5ef740 10 client.5310 fetch_fsmap finished waiting for FSMap version 23 2023-11-15T02:58:38.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8eca5ef740 10 client.5310 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-15T02:58:38.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8eca5ef740 20 client.5310 populate_metadata read hostname 'smithi121' 2023-11-15T02:58:38.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/1575057824 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x1ddb670 0x1ddba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8eca5ef740 4 client.5310 mds_command: new command op to 5178 tid=0[{"prefix": "scrub status"}] 2023-11-15T02:58:38.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.558+0000 7f8eca5ef740 1 -- 172.21.15.121:0/1575057824 --> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x1be6610 con 0x1ddb670 2023-11-15T02:58:38.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8ebdff7700 1 --2- 172.21.15.121:0/1575057824 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x1ddb670 0x1ddba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8ebdff7700 1 --2- 172.21.15.121:0/1575057824 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x1ddb670 0x1ddba80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=bedb06813fa83f3 server_cookie=e97dcb88dd87b23e in_seq=0 out_seq=0 2023-11-15T02:58:38.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8e97fff700 10 client.5310 ms_handle_connect on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:38.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8e97fff700 1 -- 172.21.15.121:0/1575057824 <== mds.0 v2:172.21.15.121:6836/3305580606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x1be6610 con 0x1ddb670 2023-11-15T02:58:38.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8e97fff700 10 client.5310 handle_command_reply: tid=0 2023-11-15T02:58:38.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8eca5ef740 1 client.5310 shutdown 2023-11-15T02:58:38.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8eca5ef740 1 -- 172.21.15.121:0/1575057824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1df1930 msgr2=0x1ddb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:38.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/1575057824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1df1930 0x1ddb180 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f8eb0013820 tx=0x7f8eb0009040).stop 2023-11-15T02:58:38.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8eca5ef740 1 -- 172.21.15.121:0/1575057824 shutdown_connections 2023-11-15T02:58:38.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/1575057824 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x1ddb670 0x1ddba80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/1575057824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1df1930 0x1ddb180 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/1575057824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1dafbd0 0x1daa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8eca5ef740 1 --2- 172.21.15.121:0/1575057824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1dabf70 0x1da9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8e97fff700 0 client.5310 ms_handle_reset on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:38.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8eca5ef740 1 -- 172.21.15.121:0/1575057824 >> 172.21.15.121:0/1575057824 conn(0x1b8f680 msgr2=0x1dd7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:38.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8eca5ef740 1 -- 172.21.15.121:0/1575057824 shutdown_connections 2023-11-15T02:58:38.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8eca5ef740 1 -- 172.21.15.121:0/1575057824 wait complete. 2023-11-15T02:58:38.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8eca5ef740 20 client.5310 trim_cache size 0 max 16384 2023-11-15T02:58:38.583 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T02:58:38.583 INFO:teuthology.orchestra.run.smithi121.stdout: "status": "no active scrubs running", 2023-11-15T02:58:38.583 INFO:teuthology.orchestra.run.smithi121.stdout: "scrubs": {} 2023-11-15T02:58:38.583 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T02:58:38.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8ebeff9700 1 -- 172.21.15.121:0/854101107 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8e9c05a510 msgr2=0x7f8e9c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:38.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8ebeff9700 1 --2- 172.21.15.121:0/854101107 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8e9c05a510 0x7f8e9c05c9d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f8ea8006d80 tx=0x7f8ea8008040).stop 2023-11-15T02:58:38.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8ebeff9700 1 -- 172.21.15.121:0/854101107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8eb8066330 msgr2=0x7f8eb80c1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:38.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8ebeff9700 1 --2- 172.21.15.121:0/854101107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8eb8066330 0x7f8eb80c1a30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f8eb8064f20 tx=0x7f8eb4017000).stop 2023-11-15T02:58:38.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8ebeff9700 1 -- 172.21.15.121:0/854101107 shutdown_connections 2023-11-15T02:58:38.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8ebeff9700 1 --2- 172.21.15.121:0/854101107 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8e9c05a510 0x7f8e9c05c9d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8ebeff9700 1 --2- 172.21.15.121:0/854101107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8eb80c5c70 0x7f8eb80c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8ebeff9700 1 --2- 172.21.15.121:0/854101107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8eb80acd00 0x7f8eb814bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8ebeff9700 1 --2- 172.21.15.121:0/854101107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8eb8066330 0x7f8eb80c1a30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8ebeff9700 1 -- 172.21.15.121:0/854101107 >> 172.21.15.121:0/854101107 conn(0x7f8eb8063fc0 msgr2=0x7f8eb814e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:38.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8ebeff9700 1 -- 172.21.15.121:0/854101107 shutdown_connections 2023-11-15T02:58:38.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.562+0000 7f8ebeff9700 1 -- 172.21.15.121:0/854101107 wait complete. 2023-11-15T02:58:38.591 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-15T02:58:38.592 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T02:58:38.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.726+0000 7ff64759e700 1 Processor -- start 2023-11-15T02:58:38.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.726+0000 7ff64759e700 1 -- start start 2023-11-15T02:58:38.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.726+0000 7ff64759e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff6480c2890 0x7ff648064d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.726+0000 7ff64759e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff64806a0d0 0x7ff648065550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.726+0000 7ff64759e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6480c3080 con 0x7ff6480c2c70 2023-11-15T02:58:38.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.726+0000 7ff64759e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff64806a560 con 0x7ff64806a0d0 2023-11-15T02:58:38.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.726+0000 7ff64759e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff648065a40 con 0x7ff6480c2890 2023-11-15T02:58:38.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.726+0000 7ff646d9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff64806a0d0 0x7ff648065550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.726+0000 7ff646d9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff64806a0d0 0x7ff648065550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32900/0 (socket says 172.21.15.121:32900) 2023-11-15T02:58:38.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.726+0000 7ff646d9d700 1 -- 172.21.15.121:0/3212690807 learned_addr learned my addr 172.21.15.121:0/3212690807 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:38.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.730+0000 7ff646d9d700 1 -- 172.21.15.121:0/3212690807 >> v2:172.21.15.190:3300/0 conn(0x7ff6480c2890 msgr2=0x7ff648064d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:38.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.730+0000 7ff64659c700 1 --2- 172.21.15.121:0/3212690807 >> v2:172.21.15.190:3300/0 conn(0x7ff6480c2890 0x7ff648064d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.730+0000 7ff646d9d700 1 --2- 172.21.15.121:0/3212690807 >> v2:172.21.15.190:3300/0 conn(0x7ff6480c2890 0x7ff648064d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.730+0000 7ff646d9d700 1 -- 172.21.15.121:0/3212690807 >> v1:172.21.15.190:6789/0 conn(0x7ff6480c2c70 legacy=0x7ff6481578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:38.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.730+0000 7ff646d9d700 1 -- 172.21.15.121:0/3212690807 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff648065c60 con 0x7ff64806a0d0 2023-11-15T02:58:38.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.730+0000 7ff64659c700 1 --2- 172.21.15.121:0/3212690807 >> v2:172.21.15.190:3300/0 conn(0x7ff6480c2890 0x7ff648064d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:58:38.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.730+0000 7ff646d9d700 1 --2- 172.21.15.121:0/3212690807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff64806a0d0 0x7ff648065550 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7ff64806bce0 tx=0x7ff6400090f0).ready entity=mon.0 client_cookie=f99c8974419a2b5a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:38.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.730+0000 7ff645d9b700 1 -- 172.21.15.121:0/3212690807 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64000b5a0 con 0x7ff64806a0d0 2023-11-15T02:58:38.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.730+0000 7ff645d9b700 1 -- 172.21.15.121:0/3212690807 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64000b700 con 0x7ff64806a0d0 2023-11-15T02:58:38.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.730+0000 7ff645d9b700 1 -- 172.21.15.121:0/3212690807 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64000ba80 con 0x7ff64806a0d0 2023-11-15T02:58:38.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.730+0000 7ff64759e700 1 -- 172.21.15.121:0/3212690807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff64806a0d0 msgr2=0x7ff648065550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:38.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.730+0000 7ff64759e700 1 --2- 172.21.15.121:0/3212690807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff64806a0d0 0x7ff648065550 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7ff64806bce0 tx=0x7ff6400090f0).stop 2023-11-15T02:58:38.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.730+0000 7ff64759e700 1 -- 172.21.15.121:0/3212690807 shutdown_connections 2023-11-15T02:58:38.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.730+0000 7ff64759e700 1 --2- 172.21.15.121:0/3212690807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff64806a0d0 0x7ff648065550 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.730+0000 7ff64759e700 1 --2- 172.21.15.121:0/3212690807 >> v2:172.21.15.190:3300/0 conn(0x7ff6480c2890 0x7ff648064d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.730+0000 7ff64759e700 1 -- 172.21.15.121:0/3212690807 >> 172.21.15.121:0/3212690807 conn(0x7ff6480c38a0 msgr2=0x7ff6480c3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:38.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.730+0000 7ff64759e700 1 -- 172.21.15.121:0/3212690807 shutdown_connections 2023-11-15T02:58:38.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.730+0000 7ff64759e700 1 -- 172.21.15.121:0/3212690807 wait complete. 2023-11-15T02:58:38.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff64759e700 1 Processor -- start 2023-11-15T02:58:38.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff64759e700 1 -- start start 2023-11-15T02:58:38.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff64759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff64806a0d0 0x7ff64814a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff64759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6480c2890 0x7ff648152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff64759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6480c2c70 0x7ff64814e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff64759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff648154d20 con 0x7ff64806a0d0 2023-11-15T02:58:38.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff64759e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff648152f60 con 0x7ff6480c2890 2023-11-15T02:58:38.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff64759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff648151230 con 0x7ff6480c2c70 2023-11-15T02:58:38.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff64659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff64806a0d0 0x7ff64814a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff64659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff64806a0d0 0x7ff64814a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32906/0 (socket says 172.21.15.121:32906) 2023-11-15T02:58:38.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff64659c700 1 -- 172.21.15.121:0/4673950 learned_addr learned my addr 172.21.15.121:0/4673950 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:38.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff646d9d700 1 --2- 172.21.15.121:0/4673950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6480c2c70 0x7ff64814e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff63dd9b700 1 --2- 172.21.15.121:0/4673950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6480c2890 0x7ff648152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff64659c700 1 -- 172.21.15.121:0/4673950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6480c2c70 msgr2=0x7ff64814e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:38.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff64659c700 1 --2- 172.21.15.121:0/4673950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6480c2c70 0x7ff64814e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff64659c700 1 -- 172.21.15.121:0/4673950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6480c2890 msgr2=0x7ff648152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:38.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff64659c700 1 --2- 172.21.15.121:0/4673950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6480c2890 0x7ff648152a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff64659c700 1 -- 172.21.15.121:0/4673950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff640009370 con 0x7ff64806a0d0 2023-11-15T02:58:38.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff64659c700 1 --2- 172.21.15.121:0/4673950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff64806a0d0 0x7ff64814a610 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7ff63800a400 tx=0x7ff638007910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:38.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff63effd700 1 -- 172.21.15.121:0/4673950 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63800d650 con 0x7ff64806a0d0 2023-11-15T02:58:38.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff63effd700 1 -- 172.21.15.121:0/4673950 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63800d7b0 con 0x7ff64806a0d0 2023-11-15T02:58:38.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff63effd700 1 -- 172.21.15.121:0/4673950 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63800db30 con 0x7ff64806a0d0 2023-11-15T02:58:38.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff64759e700 1 -- 172.21.15.121:0/4673950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff648151510 con 0x7ff64806a0d0 2023-11-15T02:58:38.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.734+0000 7ff64759e700 1 -- 172.21.15.121:0/4673950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff648151ad0 con 0x7ff64806a0d0 2023-11-15T02:58:38.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.738+0000 7ff64759e700 1 -- 172.21.15.121:0/4673950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff648063b40 con 0x7ff64806a0d0 2023-11-15T02:58:38.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.738+0000 7ff63effd700 1 -- 172.21.15.121:0/4673950 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff638013070 con 0x7ff64806a0d0 2023-11-15T02:58:38.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.738+0000 7ff63effd700 1 --2- 172.21.15.121:0/4673950 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff62805a510 0x7ff62805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:38.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.738+0000 7ff63effd700 1 -- 172.21.15.121:0/4673950 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ff638075450 con 0x7ff64806a0d0 2023-11-15T02:58:38.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.738+0000 7ff63dd9b700 1 --2- 172.21.15.121:0/4673950 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff62805a510 0x7ff62805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:38.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.742+0000 7ff63dd9b700 1 --2- 172.21.15.121:0/4673950 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff62805a510 0x7ff62805c9d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7ff6480acf40 tx=0x7ff630008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:38.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.746+0000 7ff63effd700 1 -- 172.21.15.121:0/4673950 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff638048b20 con 0x7ff64806a0d0 2023-11-15T02:58:38.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.946+0000 7ff64759e700 1 -- 172.21.15.121:0/4673950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff648064240 con 0x7ff64806a0d0 2023-11-15T02:58:38.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.950+0000 7ff63effd700 1 -- 172.21.15.121:0/4673950 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff63804c140 con 0x7ff64806a0d0 2023-11-15T02:58:38.955 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T02:58:38.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.950+0000 7ff64759e700 1 -- 172.21.15.121:0/4673950 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff62805a510 msgr2=0x7ff62805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:38.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.950+0000 7ff64759e700 1 --2- 172.21.15.121:0/4673950 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff62805a510 0x7ff62805c9d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7ff6480acf40 tx=0x7ff630008040).stop 2023-11-15T02:58:38.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.950+0000 7ff64759e700 1 -- 172.21.15.121:0/4673950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff64806a0d0 msgr2=0x7ff64814a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:38.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.950+0000 7ff64759e700 1 --2- 172.21.15.121:0/4673950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff64806a0d0 0x7ff64814a610 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7ff63800a400 tx=0x7ff638007910).stop 2023-11-15T02:58:38.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.950+0000 7ff64759e700 1 -- 172.21.15.121:0/4673950 shutdown_connections 2023-11-15T02:58:38.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.950+0000 7ff64759e700 1 --2- 172.21.15.121:0/4673950 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff62805a510 0x7ff62805c9d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.950+0000 7ff64759e700 1 --2- 172.21.15.121:0/4673950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6480c2c70 0x7ff64814e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.950+0000 7ff64759e700 1 --2- 172.21.15.121:0/4673950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6480c2890 0x7ff648152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.950+0000 7ff64759e700 1 --2- 172.21.15.121:0/4673950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff64806a0d0 0x7ff64814a610 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:38.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.950+0000 7ff64759e700 1 -- 172.21.15.121:0/4673950 >> 172.21.15.121:0/4673950 conn(0x7ff6480c38a0 msgr2=0x7ff64814cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:38.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.950+0000 7ff64759e700 1 -- 172.21.15.121:0/4673950 shutdown_connections 2023-11-15T02:58:38.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:38.950+0000 7ff64759e700 1 -- 172.21.15.121:0/4673950 wait complete. 2023-11-15T02:58:38.974 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T02:58:38.975 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T02:58:38.975 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T02:58:38.976 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T02:58:38.976 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T02:58:38.976 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T02:58:38.977 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T02:58:38.977 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T02:58:38.977 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T02:58:38.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:58:38.978 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T02:58:38.997 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T02:58:38.998 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T02:58:38.998 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T02:58:38.999 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:58:38.999 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T02:58:38.999 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:58:38.999 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:58:39.000 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:58:39.000 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T02:58:39.000 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:58:39.001 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:58:39.001 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T02:58:39.001 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:58:39.002 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:58:39.002 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T02:58:39.002 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:58:39.003 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T02:58:39.003 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42321sec preferred_lft 42321sec 2023-11-15T02:58:39.003 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T02:58:39.003 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:58:39.004 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T02:58:39.004 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:58:39.004 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T02:58:39.005 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:58:39.005 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T02:58:39.005 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T02:58:39.006 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T02:58:39.007 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:58:39.007 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T02:58:39.008 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:58:39.009 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T02:58:39.009 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T02:58:39.009 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T02:58:39.009 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:58:39.010 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T02:58:39.010 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T02:58:39.010 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T02:58:39.010 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:58:39.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:58:39.012 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T02:58:39.091 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T02:58:39.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:58:39.095 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T02:58:39.160 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T02:58:39.160 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T02:58:39.160 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T02:58:39.161 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T02:58:39.161 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T02:58:39.162 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T02:58:39.162 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T02:58:39.163 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T02:58:39.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:58:39.164 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T02:58:39.164 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T02:58:39.165 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T02:58:39.165 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T02:58:39.166 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T02:58:39.166 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T02:58:39.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:58:39.296 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T02:58:39.312 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T02:58:39.313 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T02:58:39.314 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T02:58:39.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:58:39.315 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T02:58:39.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:58:39.393 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T02:58:39.453 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T02:58:39.454 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:58:39.575 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:58:39.575 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:58:39.597 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:58:39.598 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:58:39.602 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T02:58:39.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:58:39.603 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T02:58:39.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:58:39.630 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2023-11-15T02:58:39.694 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T02:58:39.695 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-11-15T02:58:39.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.790+0000 7f8c55d37d00 1 Processor -- start 2023-11-15T02:58:39.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.794+0000 7f8c55d37d00 1 -- start start 2023-11-15T02:58:39.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.794+0000 7f8c55d37d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bc63e3bc0 0x562bc6416dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:39.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.794+0000 7f8c55d37d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562bc642db10 0x562bc6412bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:39.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.794+0000 7f8c55d37d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562bc64c0660 0x562bc64130e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:39.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.794+0000 7f8c55d37d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc635c220 con 0x562bc63e3bc0 2023-11-15T02:58:39.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.794+0000 7f8c55d37d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc6418d00 con 0x562bc642db10 2023-11-15T02:58:39.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.794+0000 7f8c55d37d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x562bc6430880 con 0x562bc64c0660 2023-11-15T02:58:39.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.794+0000 7f8c4a366700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bc63e3bc0 0x562bc6416dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:39.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.794+0000 7f8c4a366700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bc63e3bc0 0x562bc6416dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32914/0 (socket says 172.21.15.121:32914) 2023-11-15T02:58:39.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.794+0000 7f8c4a366700 1 -- 172.21.15.121:0/2712745882 learned_addr learned my addr 172.21.15.121:0/2712745882 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:39.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.794+0000 7f8c41b65700 1 --2- 172.21.15.121:0/2712745882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562bc642db10 0x562bc6412bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:39.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.794+0000 7f8c4ab67700 1 --2- 172.21.15.121:0/2712745882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562bc64c0660 0x562bc64130e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:39.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.794+0000 7f8c4a366700 1 -- 172.21.15.121:0/2712745882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562bc64c0660 msgr2=0x562bc64130e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:39.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.794+0000 7f8c4a366700 1 --2- 172.21.15.121:0/2712745882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562bc64c0660 0x562bc64130e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:39.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.794+0000 7f8c4a366700 1 -- 172.21.15.121:0/2712745882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562bc642db10 msgr2=0x562bc6412bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:39.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.794+0000 7f8c4a366700 1 --2- 172.21.15.121:0/2712745882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562bc642db10 0x562bc6412bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:39.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.794+0000 7f8c4a366700 1 -- 172.21.15.121:0/2712745882 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4400a0e0 con 0x562bc63e3bc0 2023-11-15T02:58:39.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.794+0000 7f8c4a366700 1 --2- 172.21.15.121:0/2712745882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bc63e3bc0 0x562bc6416dd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f8c3c001d60 tx=0x7f8c3c001810).ready entity=mon.0 client_cookie=1f4738e3fcb37fa4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:39.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.798+0000 7f8c49364700 1 -- 172.21.15.121:0/2712745882 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00bc80 con 0x562bc63e3bc0 2023-11-15T02:58:39.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.798+0000 7f8c49364700 1 -- 172.21.15.121:0/2712745882 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3c00bde0 con 0x562bc63e3bc0 2023-11-15T02:58:39.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.798+0000 7f8c49364700 1 -- 172.21.15.121:0/2712745882 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00e680 con 0x562bc63e3bc0 2023-11-15T02:58:39.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.798+0000 7f8c55d37d00 1 -- 172.21.15.121:0/2712745882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bc63e3bc0 msgr2=0x562bc6416dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:39.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.798+0000 7f8c55d37d00 1 --2- 172.21.15.121:0/2712745882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bc63e3bc0 0x562bc6416dd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f8c3c001d60 tx=0x7f8c3c001810).stop 2023-11-15T02:58:39.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.798+0000 7f8c55d37d00 1 -- 172.21.15.121:0/2712745882 shutdown_connections 2023-11-15T02:58:39.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.798+0000 7f8c55d37d00 1 --2- 172.21.15.121:0/2712745882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562bc64c0660 0x562bc64130e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:39.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.798+0000 7f8c55d37d00 1 --2- 172.21.15.121:0/2712745882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562bc642db10 0x562bc6412bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:39.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.798+0000 7f8c55d37d00 1 --2- 172.21.15.121:0/2712745882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bc63e3bc0 0x562bc6416dd0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:39.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.798+0000 7f8c55d37d00 1 -- 172.21.15.121:0/2712745882 >> 172.21.15.121:0/2712745882 conn(0x562bc6433090 msgr2=0x562bc6427ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:39.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.798+0000 7f8c55d37d00 1 -- 172.21.15.121:0/2712745882 shutdown_connections 2023-11-15T02:58:39.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.798+0000 7f8c55d37d00 1 -- 172.21.15.121:0/2712745882 wait complete. 2023-11-15T02:58:39.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.798+0000 7f8c55d37d00 1 Processor -- start 2023-11-15T02:58:39.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.798+0000 7f8c55d37d00 1 -- start start 2023-11-15T02:58:39.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c55d37d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562bc63e3bc0 0x562bc64aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:39.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c55d37d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bc642db10 0x562bc64b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:39.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c55d37d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562bc64c0660 0x562bc64291b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:39.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c55d37d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc64b00a0 con 0x562bc642db10 2023-11-15T02:58:39.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c55d37d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc64af520 con 0x562bc64c0660 2023-11-15T02:58:39.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c55d37d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x562bc64af800 con 0x562bc63e3bc0 2023-11-15T02:58:39.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c41b65700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bc642db10 0x562bc64b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:39.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c41b65700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bc642db10 0x562bc64b60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32928/0 (socket says 172.21.15.121:32928) 2023-11-15T02:58:39.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c41b65700 1 -- 172.21.15.121:0/4059175845 learned_addr learned my addr 172.21.15.121:0/4059175845 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:39.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c4a366700 1 --2- 172.21.15.121:0/4059175845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562bc63e3bc0 0x562bc64aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:39.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c4ab67700 1 --2- 172.21.15.121:0/4059175845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562bc64c0660 0x562bc64291b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:39.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c41b65700 1 -- 172.21.15.121:0/4059175845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562bc63e3bc0 msgr2=0x562bc64aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:39.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c41b65700 1 --2- 172.21.15.121:0/4059175845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562bc63e3bc0 0x562bc64aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:39.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c41b65700 1 -- 172.21.15.121:0/4059175845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562bc64c0660 msgr2=0x562bc64291b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:39.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c41b65700 1 --2- 172.21.15.121:0/4059175845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562bc64c0660 0x562bc64291b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:39.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c41b65700 1 -- 172.21.15.121:0/4059175845 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c38001e10 con 0x562bc642db10 2023-11-15T02:58:39.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c4ab67700 1 --2- 172.21.15.121:0/4059175845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562bc64c0660 0x562bc64291b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:58:39.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c41b65700 1 --2- 172.21.15.121:0/4059175845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bc642db10 0x562bc64b60e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f8c34008b50 tx=0x7f8c3400c040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:39.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c437fe700 1 -- 172.21.15.121:0/4059175845 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3400f030 con 0x562bc642db10 2023-11-15T02:58:39.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c55d37d00 1 -- 172.21.15.121:0/4059175845 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562bc6385770 con 0x562bc642db10 2023-11-15T02:58:39.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c437fe700 1 -- 172.21.15.121:0/4059175845 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c34010040 con 0x562bc642db10 2023-11-15T02:58:39.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c437fe700 1 -- 172.21.15.121:0/4059175845 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c34006cb0 con 0x562bc642db10 2023-11-15T02:58:39.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.802+0000 7f8c55d37d00 1 -- 172.21.15.121:0/4059175845 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562bc6354a70 con 0x562bc642db10 2023-11-15T02:58:39.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.806+0000 7f8c437fe700 1 -- 172.21.15.121:0/4059175845 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8c34014610 con 0x562bc642db10 2023-11-15T02:58:39.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.806+0000 7f8c437fe700 1 --2- 172.21.15.121:0/4059175845 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8c2c058ff0 0x7f8c2c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:39.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.806+0000 7f8c437fe700 1 -- 172.21.15.121:0/4059175845 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f8c34073350 con 0x562bc642db10 2023-11-15T02:58:39.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.806+0000 7f8c4a366700 1 --2- 172.21.15.121:0/4059175845 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8c2c058ff0 0x7f8c2c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:39.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.806+0000 7f8c55d37d00 1 --2- 172.21.15.121:0/4059175845 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562bc64acff0 0x562bc64b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:39.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.806+0000 7f8c55d37d00 1 -- 172.21.15.121:0/4059175845 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 5.30 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected e35) v8 -- 0x562bc64b1f00 con 0x562bc64acff0 2023-11-15T02:58:39.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.806+0000 7f8c4a366700 1 --2- 172.21.15.121:0/4059175845 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8c2c058ff0 0x7f8c2c05b4b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f8c3c001870 tx=0x7f8c3c014000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:39.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.810+0000 7f8c4ab67700 1 --2- 172.21.15.121:0/4059175845 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562bc64acff0 0x562bc64b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:39.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.810+0000 7f8c4ab67700 1 --2- 172.21.15.121:0/4059175845 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562bc64acff0 0x562bc64b1a10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:39.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.814+0000 7f8c4ab67700 1 -- 172.21.15.121:0/4059175845 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v35'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8c4400d740 con 0x562bc64acff0 2023-11-15T02:58:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.814+0000 7f8c55d37d00 1 -- 172.21.15.121:0/4059175845 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562bc64acff0 msgr2=0x562bc64b1a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.814+0000 7f8c55d37d00 1 --2- 172.21.15.121:0/4059175845 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562bc64acff0 0x562bc64b1a10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.814+0000 7f8c55d37d00 1 -- 172.21.15.121:0/4059175845 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8c2c058ff0 msgr2=0x7f8c2c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.814+0000 7f8c55d37d00 1 --2- 172.21.15.121:0/4059175845 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8c2c058ff0 0x7f8c2c05b4b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f8c3c001870 tx=0x7f8c3c014000).stop 2023-11-15T02:58:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.814+0000 7f8c55d37d00 1 -- 172.21.15.121:0/4059175845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bc642db10 msgr2=0x562bc64b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:39.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.814+0000 7f8c55d37d00 1 --2- 172.21.15.121:0/4059175845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bc642db10 0x562bc64b60e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f8c34008b50 tx=0x7f8c3400c040).stop 2023-11-15T02:58:39.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.814+0000 7f8c4ab67700 1 -- 172.21.15.121:0/4059175845 reap_dead start 2023-11-15T02:58:39.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.814+0000 7f8c55d37d00 1 -- 172.21.15.121:0/4059175845 shutdown_connections 2023-11-15T02:58:39.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.814+0000 7f8c55d37d00 1 -- 172.21.15.121:0/4059175845 >> 172.21.15.121:0/4059175845 conn(0x562bc6433090 msgr2=0x562bc6416dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:39.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.814+0000 7f8c55d37d00 1 -- 172.21.15.121:0/4059175845 shutdown_connections 2023-11-15T02:58:39.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.814+0000 7f8c55d37d00 1 -- 172.21.15.121:0/4059175845 wait complete. 2023-11-15T02:58:39.825 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:58:40.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.998+0000 7f796ec1b700 1 Processor -- start 2023-11-15T02:58:40.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:39.998+0000 7f796ec1b700 1 -- start start 2023-11-15T02:58:40.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.002+0000 7f796ec1b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f79681539d0 0x7f79680c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.002+0000 7f796ec1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79680c2670 0x7f79680c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.002+0000 7f796ec1b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79680c1530 con 0x7f7968153db0 2023-11-15T02:58:40.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.002+0000 7f796ec1b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79681541f0 con 0x7f79681539d0 2023-11-15T02:58:40.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.002+0000 7f796ec1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79680c2b00 con 0x7f79680c2670 2023-11-15T02:58:40.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.002+0000 7f796d1b8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79680c2670 0x7f79680c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.002+0000 7f796d1b8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79680c2670 0x7f79680c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:50682/0 (socket says 172.21.15.121:50682) 2023-11-15T02:58:40.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.002+0000 7f796d1b8700 1 -- 172.21.15.121:0/387260816 learned_addr learned my addr 172.21.15.121:0/387260816 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:40.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.002+0000 7f796c9b7700 1 --2- 172.21.15.121:0/387260816 >> v2:172.21.15.190:3300/0 conn(0x7f79681539d0 0x7f79680c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.002+0000 7f796c9b7700 1 -- 172.21.15.121:0/387260816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79680c2670 msgr2=0x7f79680c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:40.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.002+0000 7f796c9b7700 1 --2- 172.21.15.121:0/387260816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79680c2670 0x7f79680c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.002+0000 7f796c9b7700 1 -- 172.21.15.121:0/387260816 >> v1:172.21.15.190:6789/0 conn(0x7f7968153db0 legacy=0x7f79680c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:40.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.002+0000 7f796c9b7700 1 -- 172.21.15.121:0/387260816 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79680c2d80 con 0x7f79681539d0 2023-11-15T02:58:40.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.002+0000 7f796c9b7700 1 --2- 172.21.15.121:0/387260816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79681539d0 0x7f79680c06e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f795c009e60 tx=0x7f795c009780).ready entity=mon.1 client_cookie=1b937d40f1bad5ba server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:40.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.002+0000 7f79677fe700 1 -- 172.21.15.121:0/387260816 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c00e070 con 0x7f79681539d0 2023-11-15T02:58:40.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.006+0000 7f79677fe700 1 -- 172.21.15.121:0/387260816 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795c00bb60 con 0x7f79681539d0 2023-11-15T02:58:40.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.006+0000 7f79677fe700 1 -- 172.21.15.121:0/387260816 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c012410 con 0x7f79681539d0 2023-11-15T02:58:40.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.006+0000 7f796ec1b700 1 -- 172.21.15.121:0/387260816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79681539d0 msgr2=0x7f79680c06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:40.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.006+0000 7f796ec1b700 1 --2- 172.21.15.121:0/387260816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79681539d0 0x7f79680c06e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f795c009e60 tx=0x7f795c009780).stop 2023-11-15T02:58:40.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.006+0000 7f796ec1b700 1 -- 172.21.15.121:0/387260816 shutdown_connections 2023-11-15T02:58:40.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.006+0000 7f796ec1b700 1 --2- 172.21.15.121:0/387260816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79680c2670 0x7f79680c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.006+0000 7f796ec1b700 1 --2- 172.21.15.121:0/387260816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79681539d0 0x7f79680c06e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.006+0000 7f796ec1b700 1 -- 172.21.15.121:0/387260816 >> 172.21.15.121:0/387260816 conn(0x7f7968062190 msgr2=0x7f79680625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:40.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.006+0000 7f796ec1b700 1 -- 172.21.15.121:0/387260816 shutdown_connections 2023-11-15T02:58:40.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.006+0000 7f796ec1b700 1 -- 172.21.15.121:0/387260816 wait complete. 2023-11-15T02:58:40.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.006+0000 7f796ec1b700 1 Processor -- start 2023-11-15T02:58:40.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.006+0000 7f796ec1b700 1 -- start start 2023-11-15T02:58:40.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f796ec1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79680c2670 0x7f796814c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f796ec1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79681539d0 0x7f79680ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f796ec1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7968153db0 0x7f7968151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f796ec1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796814c890 con 0x7f79680c2670 2023-11-15T02:58:40.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f796ec1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7968150dd0 con 0x7f7968153db0 2023-11-15T02:58:40.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f796ec1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79681510b0 con 0x7f79681539d0 2023-11-15T02:58:40.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f796c9b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79680c2670 0x7f796814c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f796c9b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79680c2670 0x7f796814c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32934/0 (socket says 172.21.15.121:32934) 2023-11-15T02:58:40.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f796c9b7700 1 -- 172.21.15.121:0/4062150353 learned_addr learned my addr 172.21.15.121:0/4062150353 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:40.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f7967fff700 1 --2- 172.21.15.121:0/4062150353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79681539d0 0x7f79680ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f796d1b8700 1 --2- 172.21.15.121:0/4062150353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7968153db0 0x7f7968151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f796c9b7700 1 -- 172.21.15.121:0/4062150353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79681539d0 msgr2=0x7f79680ac1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:40.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f796c9b7700 1 --2- 172.21.15.121:0/4062150353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79681539d0 0x7f79680ac1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f796c9b7700 1 -- 172.21.15.121:0/4062150353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7968153db0 msgr2=0x7f7968151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:40.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f796c9b7700 1 --2- 172.21.15.121:0/4062150353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7968153db0 0x7f7968151980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f796c9b7700 1 -- 172.21.15.121:0/4062150353 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795c009a20 con 0x7f79680c2670 2023-11-15T02:58:40.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f796c9b7700 1 --2- 172.21.15.121:0/4062150353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79680c2670 0x7f796814c3a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f795c00b6c0 tx=0x7f795c009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:40.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f7964ff9700 1 -- 172.21.15.121:0/4062150353 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c01d420 con 0x7f79680c2670 2023-11-15T02:58:40.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f7964ff9700 1 -- 172.21.15.121:0/4062150353 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795c01d580 con 0x7f79680c2670 2023-11-15T02:58:40.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f796ec1b700 1 -- 172.21.15.121:0/4062150353 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7968151390 con 0x7f79680c2670 2023-11-15T02:58:40.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f7964ff9700 1 -- 172.21.15.121:0/4062150353 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c01d940 con 0x7f79680c2670 2023-11-15T02:58:40.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.010+0000 7f796ec1b700 1 -- 172.21.15.121:0/4062150353 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79680a98d0 con 0x7f79680c2670 2023-11-15T02:58:40.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.014+0000 7f796ec1b700 1 -- 172.21.15.121:0/4062150353 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f796806c150 con 0x7f79680c2670 2023-11-15T02:58:40.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.014+0000 7f7964ff9700 1 -- 172.21.15.121:0/4062150353 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f795c003040 con 0x7f79680c2670 2023-11-15T02:58:40.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.014+0000 7f7964ff9700 1 --2- 172.21.15.121:0/4062150353 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f794c05a510 0x7f794c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.014+0000 7f7964ff9700 1 -- 172.21.15.121:0/4062150353 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f795c076580 con 0x7f79680c2670 2023-11-15T02:58:40.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.014+0000 7f7967fff700 1 --2- 172.21.15.121:0/4062150353 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f794c05a510 0x7f794c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.018+0000 7f7967fff700 1 --2- 172.21.15.121:0/4062150353 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f794c05a510 0x7f794c05c9d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f7968152d40 tx=0x7f7954007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:40.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.022+0000 7f7964ff9700 1 -- 172.21.15.121:0/4062150353 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f795c049cf0 con 0x7f79680c2670 2023-11-15T02:58:40.222 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T02:58:40.229 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T02:58:40.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.242+0000 7f796ec1b700 1 -- 172.21.15.121:0/4062150353 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7968068ae0 con 0x7f79680c2670 2023-11-15T02:58:40.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.242+0000 7f7964ff9700 1 -- 172.21.15.121:0/4062150353 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4502 (secure 0 0 0) 0x7f795c04d310 con 0x7f79680c2670 2023-11-15T02:58:40.248 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:58:40.248 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2544127447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2544127447},{"type":"v1","addr":"172.21.15.190:6833","nonce":2544127447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1815834193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1815834193},{"type":"v1","addr":"172.21.15.190:6835","nonce":1815834193}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4950,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3229917854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3229917854},{"type":"v1","addr":"172.21.15.121:6835","nonce":3229917854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:57:44.541243+0000","modified":"2023-11-15T02:58:10.996523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5178":{"gid":5178,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6837/3305580606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3305580606},{"type":"v1","addr":"172.21.15.121:6837","nonce":3305580606}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-15T02:58:40.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.242+0000 7f796ec1b700 1 -- 172.21.15.121:0/4062150353 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f794c05a510 msgr2=0x7f794c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:40.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.242+0000 7f796ec1b700 1 --2- 172.21.15.121:0/4062150353 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f794c05a510 0x7f794c05c9d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f7968152d40 tx=0x7f7954007d40).stop 2023-11-15T02:58:40.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.242+0000 7f796ec1b700 1 -- 172.21.15.121:0/4062150353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79680c2670 msgr2=0x7f796814c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:40.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.242+0000 7f796ec1b700 1 --2- 172.21.15.121:0/4062150353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79680c2670 0x7f796814c3a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f795c00b6c0 tx=0x7f795c009780).stop 2023-11-15T02:58:40.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.242+0000 7f796ec1b700 1 -- 172.21.15.121:0/4062150353 shutdown_connections 2023-11-15T02:58:40.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.242+0000 7f796ec1b700 1 --2- 172.21.15.121:0/4062150353 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f794c05a510 0x7f794c05c9d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.242+0000 7f796ec1b700 1 --2- 172.21.15.121:0/4062150353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7968153db0 0x7f7968151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.242+0000 7f796ec1b700 1 --2- 172.21.15.121:0/4062150353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79681539d0 0x7f79680ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.242+0000 7f796ec1b700 1 --2- 172.21.15.121:0/4062150353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79680c2670 0x7f796814c3a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.242+0000 7f796ec1b700 1 -- 172.21.15.121:0/4062150353 >> 172.21.15.121:0/4062150353 conn(0x7f7968062190 msgr2=0x7f79680c19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:40.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.242+0000 7f796ec1b700 1 -- 172.21.15.121:0/4062150353 shutdown_connections 2023-11-15T02:58:40.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.242+0000 7f796ec1b700 1 -- 172.21.15.121:0/4062150353 wait complete. 2023-11-15T02:58:40.253 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 23 2023-11-15T02:58:40.268 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 4012680 2023-11-15T02:58:40.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.498+0000 7f7f8ccdb700 1 Processor -- start 2023-11-15T02:58:40.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.502+0000 7f7f8ccdb700 1 -- start start 2023-11-15T02:58:40.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.502+0000 7f7f8ccdb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7f88153c10 0x7f7f880bcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.502+0000 7f7f8ccdb700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7f880bb6f0 0x7f7f88153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.502+0000 7f7f8ccdb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f880a9250 con 0x7f7f88067060 2023-11-15T02:58:40.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.502+0000 7f7f8ccdb700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f880bd5a0 con 0x7f7f880bb6f0 2023-11-15T02:58:40.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.502+0000 7f7f8ccdb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f880a8920 con 0x7f7f88153c10 2023-11-15T02:58:40.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.502+0000 7f7f86d9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f7f88067060 0x7f7f880632a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39872/0 (socket says 172.21.15.121:39872) 2023-11-15T02:58:40.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.502+0000 7f7f7ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7f880bb6f0 0x7f7f88153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.502+0000 7f7f86d9d700 1 -- 172.21.15.121:0/571821612 learned_addr learned my addr 172.21.15.121:0/571821612 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:40.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.502+0000 7f7f8659c700 1 --2- 172.21.15.121:0/571821612 >> v2:172.21.15.190:3300/0 conn(0x7f7f88153c10 0x7f7f880bcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.502+0000 7f7f7ffff700 1 -- 172.21.15.121:0/571821612 >> v2:172.21.15.190:3300/0 conn(0x7f7f88153c10 msgr2=0x7f7f880bcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:40.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.502+0000 7f7f7ffff700 1 --2- 172.21.15.121:0/571821612 >> v2:172.21.15.190:3300/0 conn(0x7f7f88153c10 0x7f7f880bcfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.502+0000 7f7f7ffff700 1 -- 172.21.15.121:0/571821612 >> v1:172.21.15.121:6789/0 conn(0x7f7f88067060 legacy=0x7f7f880632a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:40.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.502+0000 7f7f7ffff700 1 -- 172.21.15.121:0/571821612 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f880a8c00 con 0x7f7f880bb6f0 2023-11-15T02:58:40.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.506+0000 7f7f7ffff700 1 --2- 172.21.15.121:0/571821612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f880bb6f0 0x7f7f88153ff0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f7f88063e40 tx=0x7f7f70009060).ready entity=mon.0 client_cookie=441944b89d799348 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:40.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.506+0000 7f7f85d9b700 1 -- 172.21.15.121:0/571821612 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7000f510 con 0x7f7f880bb6f0 2023-11-15T02:58:40.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.506+0000 7f7f85d9b700 1 -- 172.21.15.121:0/571821612 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7000f670 con 0x7f7f880bb6f0 2023-11-15T02:58:40.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.506+0000 7f7f85d9b700 1 -- 172.21.15.121:0/571821612 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7000f9f0 con 0x7f7f880bb6f0 2023-11-15T02:58:40.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.506+0000 7f7f8ccdb700 1 -- 172.21.15.121:0/571821612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f880bb6f0 msgr2=0x7f7f88153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:40.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.506+0000 7f7f8ccdb700 1 --2- 172.21.15.121:0/571821612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f880bb6f0 0x7f7f88153ff0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f7f88063e40 tx=0x7f7f70009060).stop 2023-11-15T02:58:40.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.506+0000 7f7f8ccdb700 1 -- 172.21.15.121:0/571821612 shutdown_connections 2023-11-15T02:58:40.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.506+0000 7f7f8ccdb700 1 --2- 172.21.15.121:0/571821612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f880bb6f0 0x7f7f88153ff0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.506+0000 7f7f8ccdb700 1 --2- 172.21.15.121:0/571821612 >> v2:172.21.15.190:3300/0 conn(0x7f7f88153c10 0x7f7f880bcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.506+0000 7f7f8ccdb700 1 -- 172.21.15.121:0/571821612 >> 172.21.15.121:0/571821612 conn(0x7f7f880591c0 msgr2=0x7f7f880c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:40.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.506+0000 7f7f8ccdb700 1 -- 172.21.15.121:0/571821612 shutdown_connections 2023-11-15T02:58:40.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.506+0000 7f7f8ccdb700 1 -- 172.21.15.121:0/571821612 wait complete. 2023-11-15T02:58:40.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.510+0000 7f7f8ccdb700 1 Processor -- start 2023-11-15T02:58:40.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.510+0000 7f7f8ccdb700 1 -- start start 2023-11-15T02:58:40.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.510+0000 7f7f8ccdb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f88067060 0x7f7f8814e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.510+0000 7f7f8ccdb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f880bb6f0 0x7f7f88156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.510+0000 7f7f8ccdb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f88153c10 0x7f7f88157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.510+0000 7f7f8ccdb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f880c0190 con 0x7f7f88153c10 2023-11-15T02:58:40.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.510+0000 7f7f8ccdb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8814e950 con 0x7f7f880bb6f0 2023-11-15T02:58:40.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.510+0000 7f7f8ccdb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f880bf510 con 0x7f7f88067060 2023-11-15T02:58:40.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.510+0000 7f7f86d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f88153c10 0x7f7f88157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.510+0000 7f7f86d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f88153c10 0x7f7f88157100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32948/0 (socket says 172.21.15.121:32948) 2023-11-15T02:58:40.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.510+0000 7f7f86d9d700 1 -- 172.21.15.121:0/1803266396 learned_addr learned my addr 172.21.15.121:0/1803266396 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:40.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.514+0000 7f7f8659c700 1 --2- 172.21.15.121:0/1803266396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f88067060 0x7f7f8814e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.514+0000 7f7f7ffff700 1 --2- 172.21.15.121:0/1803266396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f880bb6f0 0x7f7f88156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.514+0000 7f7f86d9d700 1 -- 172.21.15.121:0/1803266396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f88067060 msgr2=0x7f7f8814e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:40.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.514+0000 7f7f86d9d700 1 --2- 172.21.15.121:0/1803266396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f88067060 0x7f7f8814e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.514+0000 7f7f86d9d700 1 -- 172.21.15.121:0/1803266396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f880bb6f0 msgr2=0x7f7f88156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:40.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.514+0000 7f7f86d9d700 1 --2- 172.21.15.121:0/1803266396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f880bb6f0 0x7f7f88156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.514+0000 7f7f86d9d700 1 -- 172.21.15.121:0/1803266396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f700092a0 con 0x7f7f88153c10 2023-11-15T02:58:40.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.514+0000 7f7f8659c700 1 --2- 172.21.15.121:0/1803266396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f88067060 0x7f7f8814e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:40.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.514+0000 7f7f86d9d700 1 --2- 172.21.15.121:0/1803266396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f88153c10 0x7f7f88157100 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f7f8000c7f0 tx=0x7f7f80001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:40.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.514+0000 7f7f7e7fc700 1 -- 172.21.15.121:0/1803266396 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f80005d30 con 0x7f7f88153c10 2023-11-15T02:58:40.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.514+0000 7f7f7e7fc700 1 -- 172.21.15.121:0/1803266396 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f80005e90 con 0x7f7f88153c10 2023-11-15T02:58:40.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.514+0000 7f7f8ccdb700 1 -- 172.21.15.121:0/1803266396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f880bf7f0 con 0x7f7f88153c10 2023-11-15T02:58:40.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.514+0000 7f7f7e7fc700 1 -- 172.21.15.121:0/1803266396 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f80007320 con 0x7f7f88153c10 2023-11-15T02:58:40.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.514+0000 7f7f8ccdb700 1 -- 172.21.15.121:0/1803266396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f880bfdb0 con 0x7f7f88153c10 2023-11-15T02:58:40.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.518+0000 7f7f7e7fc700 1 -- 172.21.15.121:0/1803266396 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7f80007750 con 0x7f7f88153c10 2023-11-15T02:58:40.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.518+0000 7f7f7e7fc700 1 --2- 172.21.15.121:0/1803266396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7f6805a540 0x7f7f6805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.518+0000 7f7f7e7fc700 1 -- 172.21.15.121:0/1803266396 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f7f800747b0 con 0x7f7f88153c10 2023-11-15T02:58:40.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.518+0000 7f7f8659c700 1 --2- 172.21.15.121:0/1803266396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7f6805a540 0x7f7f6805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f97056740 1 Processor -- start 2023-11-15T02:58:40.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f8659c700 1 --2- 172.21.15.121:0/1803266396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7f6805a540 0x7f7f6805ca00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f7f78009c00 tx=0x7f7f78008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:40.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f97056740 1 -- start start 2023-11-15T02:58:40.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f97056740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22afb50 0x22aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f97056740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226d210 0x22921f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f97056740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2267e50 0x2268ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f97056740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x21a60e0 con 0x22afb50 2023-11-15T02:58:40.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f97056740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x21a6340 con 0x2267e50 2023-11-15T02:58:40.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f97056740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2192d10 con 0x226d210 2023-11-15T02:58:40.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f8659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22afb50 0x22aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f8659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22afb50 0x22aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32956/0 (socket says 172.21.15.121:32956) 2023-11-15T02:58:40.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f8659c700 1 -- 172.21.15.121:0/1946764891 learned_addr learned my addr 172.21.15.121:0/1946764891 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:40.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f7ffff700 1 --2- 172.21.15.121:0/1946764891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226d210 0x22921f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f86d9d700 1 --2- 172.21.15.121:0/1946764891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2267e50 0x2268ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f8659c700 1 -- 172.21.15.121:0/1946764891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226d210 msgr2=0x22921f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:40.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f8659c700 1 --2- 172.21.15.121:0/1946764891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226d210 0x22921f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f8659c700 1 -- 172.21.15.121:0/1946764891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2267e50 msgr2=0x2268ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:40.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f8659c700 1 --2- 172.21.15.121:0/1946764891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2267e50 0x2268ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f8659c700 1 -- 172.21.15.121:0/1946764891 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x21f7d00 con 0x22afb50 2023-11-15T02:58:40.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f86d9d700 1 --2- 172.21.15.121:0/1946764891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2267e50 0x2268ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:40.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.522+0000 7f7f8659c700 1 --2- 172.21.15.121:0/1946764891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22afb50 0x22aff30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f7f78011cc0 tx=0x7f7f78016000).ready entity=mon.0 client_cookie=66d0e1511baf461f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:40.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.526+0000 7f7f677fe700 1 -- 172.21.15.121:0/1946764891 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f78015a90 con 0x22afb50 2023-11-15T02:58:40.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.526+0000 7f7f677fe700 1 -- 172.21.15.121:0/1946764891 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f78015bf0 con 0x22afb50 2023-11-15T02:58:40.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.526+0000 7f7f677fe700 1 -- 172.21.15.121:0/1946764891 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f78020400 con 0x22afb50 2023-11-15T02:58:40.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.526+0000 7f7f97056740 1 -- 172.21.15.121:0/1946764891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22afb50 msgr2=0x22aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:40.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.526+0000 7f7f97056740 1 --2- 172.21.15.121:0/1946764891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22afb50 0x22aff30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f7f78011cc0 tx=0x7f7f78016000).stop 2023-11-15T02:58:40.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.526+0000 7f7f97056740 1 -- 172.21.15.121:0/1946764891 shutdown_connections 2023-11-15T02:58:40.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.526+0000 7f7f97056740 1 --2- 172.21.15.121:0/1946764891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2267e50 0x2268ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.526+0000 7f7f97056740 1 --2- 172.21.15.121:0/1946764891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226d210 0x22921f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.526+0000 7f7f97056740 1 --2- 172.21.15.121:0/1946764891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22afb50 0x22aff30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.526+0000 7f7f97056740 1 -- 172.21.15.121:0/1946764891 >> 172.21.15.121:0/1946764891 conn(0x204b6d0 msgr2=0x226cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:40.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.526+0000 7f7f97056740 1 -- 172.21.15.121:0/1946764891 shutdown_connections 2023-11-15T02:58:40.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.526+0000 7f7f97056740 1 -- 172.21.15.121:0/1946764891 wait complete. 2023-11-15T02:58:40.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.526+0000 7f7f97056740 1 -- start start 2023-11-15T02:58:40.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.526+0000 7f7f97056740 1 Processor -- start 2023-11-15T02:58:40.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f97056740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2267e50 0x22bfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f97056740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226d210 0x22c0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f97056740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22afb50 0x22bea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f97056740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2291ce0 con 0x2267e50 2023-11-15T02:58:40.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f97056740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x22c0cb0 con 0x22afb50 2023-11-15T02:58:40.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f97056740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x226b4b0 con 0x226d210 2023-11-15T02:58:40.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f97056740 10 client.0 ll_register_callbacks cb 0x2223e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:58:40.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f8659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2267e50 0x22bfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f8659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2267e50 0x22bfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32968/0 (socket says 172.21.15.121:32968) 2023-11-15T02:58:40.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f8659c700 1 -- 172.21.15.121:0/790175007 learned_addr learned my addr 172.21.15.121:0/790175007 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:40.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f7ffff700 1 --2- 172.21.15.121:0/790175007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226d210 0x22c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f8659c700 1 -- 172.21.15.121:0/790175007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226d210 msgr2=0x22c0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:40.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f8659c700 1 --2- 172.21.15.121:0/790175007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226d210 0x22c0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f8659c700 1 -- 172.21.15.121:0/790175007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22afb50 msgr2=0x22bea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:40.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f8659c700 1 --2- 172.21.15.121:0/790175007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22afb50 0x22bea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T02:58:40.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f8659c700 1 -- 172.21.15.121:0/790175007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f780119b0 con 0x2267e50 2023-11-15T02:58:40.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f7ffff700 1 --2- 172.21.15.121:0/790175007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226d210 0x22c0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:58:40.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f8659c700 1 --2- 172.21.15.121:0/790175007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2267e50 0x22bfc50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f7f78012010 tx=0x7f7f78020000).ready entity=mon.0 client_cookie=7903e1b5f7a124fc server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:40.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f64ff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:58:40.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f64ff9700 1 -- 172.21.15.121:0/790175007 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f78006cd0 con 0x2267e50 2023-11-15T02:58:40.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f64ff9700 1 -- 172.21.15.121:0/790175007 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f7f78029640 con 0x2267e50 2023-11-15T02:58:40.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f97056740 1 -- 172.21.15.121:0/790175007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x21e3260 con 0x2267e50 2023-11-15T02:58:40.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f64ff9700 1 -- 172.21.15.121:0/790175007 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f78029a10 con 0x2267e50 2023-11-15T02:58:40.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f97056740 10 client.5349 fetch_fsmap learned FSMap version 23 2023-11-15T02:58:40.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f97056740 1 -- 172.21.15.121:0/790175007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x21e67f0 con 0x2267e50 2023-11-15T02:58:40.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f64ff9700 1 -- 172.21.15.121:0/790175007 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x21e3260 con 0x2267e50 2023-11-15T02:58:40.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f64ff9700 1 -- 172.21.15.121:0/790175007 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 23) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7f78029b70 con 0x2267e50 2023-11-15T02:58:40.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f97056740 10 client.5349 fetch_fsmap finished waiting for FSMap version 23 2023-11-15T02:58:40.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f97056740 10 client.5349 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-15T02:58:40.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f97056740 20 client.5349 populate_metadata read hostname 'smithi121' 2023-11-15T02:58:40.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f97056740 1 --2- 172.21.15.121:0/790175007 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x22bf500 0x22c2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f97056740 4 client.5349 mds_command: new command op to 5178 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T02:58:40.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f97056740 1 -- 172.21.15.121:0/790175007 --> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x21d97b0 con 0x22bf500 2023-11-15T02:58:40.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.530+0000 7f7f7ffff700 1 --2- 172.21.15.121:0/790175007 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x22bf500 0x22c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.534+0000 7f7f7ffff700 1 --2- 172.21.15.121:0/790175007 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x22bf500 0x22c2050 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=2dfda421964f48a8 server_cookie=3bade77bbdba1e5c in_seq=0 out_seq=0 2023-11-15T02:58:40.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.534+0000 7f7f64ff9700 10 client.5349 ms_handle_connect on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:40.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.534+0000 7f7f64ff9700 1 -- 172.21.15.121:0/790175007 <== mds.0 v2:172.21.15.121:6836/3305580606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x21d97b0 con 0x22bf500 2023-11-15T02:58:40.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.534+0000 7f7f64ff9700 10 client.5349 handle_command_reply: tid=0 2023-11-15T02:58:40.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.534+0000 7f7f97056740 1 client.5349 shutdown 2023-11-15T02:58:40.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.534+0000 7f7f97056740 1 -- 172.21.15.121:0/790175007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2267e50 msgr2=0x22bfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:40.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.534+0000 7f7f97056740 1 --2- 172.21.15.121:0/790175007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2267e50 0x22bfc50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f7f78012010 tx=0x7f7f78020000).stop 2023-11-15T02:58:40.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.534+0000 7f7f97056740 1 -- 172.21.15.121:0/790175007 shutdown_connections 2023-11-15T02:58:40.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.534+0000 7f7f97056740 1 --2- 172.21.15.121:0/790175007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22afb50 0x22bea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T02:58:40.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.534+0000 7f7f97056740 1 --2- 172.21.15.121:0/790175007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226d210 0x22c0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.534+0000 7f7f97056740 1 --2- 172.21.15.121:0/790175007 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x22bf500 0x22c2050 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.534+0000 7f7f97056740 1 --2- 172.21.15.121:0/790175007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2267e50 0x22bfc50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.534+0000 7f7f64ff9700 0 client.5349 ms_handle_reset on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:40.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.534+0000 7f7f97056740 1 -- 172.21.15.121:0/790175007 >> 172.21.15.121:0/790175007 conn(0x204b6d0 msgr2=0x2295650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:40.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.534+0000 7f7f97056740 1 -- 172.21.15.121:0/790175007 shutdown_connections 2023-11-15T02:58:40.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.534+0000 7f7f97056740 1 -- 172.21.15.121:0/790175007 wait complete. 2023-11-15T02:58:40.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.534+0000 7f7f97056740 20 client.5349 trim_cache size 0 max 16384 2023-11-15T02:58:40.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.550+0000 7f7f97056740 1 Processor -- start 2023-11-15T02:58:40.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.550+0000 7f7f97056740 1 -- start start 2023-11-15T02:58:40.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.550+0000 7f7f97056740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2267e50 0x226a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.550+0000 7f7f97056740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226d210 0x226ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.550+0000 7f7f97056740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22afb50 0x2291ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.550+0000 7f7f97056740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x22af5e0 con 0x2267e50 2023-11-15T02:58:40.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.550+0000 7f7f97056740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x22c9060 con 0x226d210 2023-11-15T02:58:40.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.550+0000 7f7f97056740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x22b06e0 con 0x22afb50 2023-11-15T02:58:40.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.550+0000 7f7f8659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2267e50 0x226a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f8659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2267e50 0x226a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32984/0 (socket says 172.21.15.121:32984) 2023-11-15T02:58:40.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f8659c700 1 -- 172.21.15.121:0/474734986 learned_addr learned my addr 172.21.15.121:0/474734986 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:40.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f86d9d700 1 --2- 172.21.15.121:0/474734986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22afb50 0x2291ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f7ffff700 1 --2- 172.21.15.121:0/474734986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226d210 0x226ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f8659c700 1 -- 172.21.15.121:0/474734986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22afb50 msgr2=0x2291ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:40.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f8659c700 1 --2- 172.21.15.121:0/474734986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22afb50 0x2291ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f8659c700 1 -- 172.21.15.121:0/474734986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226d210 msgr2=0x226ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:40.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f8659c700 1 --2- 172.21.15.121:0/474734986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226d210 0x226ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f8659c700 1 -- 172.21.15.121:0/474734986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f88067dc0 con 0x2267e50 2023-11-15T02:58:40.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f86d9d700 1 --2- 172.21.15.121:0/474734986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22afb50 0x2291ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:40.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f7ffff700 1 --2- 172.21.15.121:0/474734986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226d210 0x226ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:40.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f8659c700 1 --2- 172.21.15.121:0/474734986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2267e50 0x226a8f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f7f78008010 tx=0x7f7f78028000).ready entity=mon.0 client_cookie=ec11c98357464ecc server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:40.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f67fff700 1 -- 172.21.15.121:0/474734986 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f78029640 con 0x2267e50 2023-11-15T02:58:40.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f67fff700 1 -- 172.21.15.121:0/474734986 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f78014430 con 0x2267e50 2023-11-15T02:58:40.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f67fff700 1 -- 172.21.15.121:0/474734986 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f78029a10 con 0x2267e50 2023-11-15T02:58:40.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f97056740 1 -- 172.21.15.121:0/474734986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2267e50 msgr2=0x226a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:40.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f97056740 1 --2- 172.21.15.121:0/474734986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2267e50 0x226a8f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f7f78008010 tx=0x7f7f78028000).stop 2023-11-15T02:58:40.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f97056740 1 -- 172.21.15.121:0/474734986 shutdown_connections 2023-11-15T02:58:40.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f97056740 1 --2- 172.21.15.121:0/474734986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22afb50 0x2291ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f97056740 1 --2- 172.21.15.121:0/474734986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226d210 0x226ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f97056740 1 --2- 172.21.15.121:0/474734986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2267e50 0x226a8f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f97056740 1 -- 172.21.15.121:0/474734986 >> 172.21.15.121:0/474734986 conn(0x204b6d0 msgr2=0x2266010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:40.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f97056740 1 -- 172.21.15.121:0/474734986 shutdown_connections 2023-11-15T02:58:40.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f97056740 1 -- 172.21.15.121:0/474734986 wait complete. 2023-11-15T02:58:40.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f97056740 1 -- start start 2023-11-15T02:58:40.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.554+0000 7f7f97056740 1 Processor -- start 2023-11-15T02:58:40.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f97056740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226d210 0x22a9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f97056740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22afb50 0x2291e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f97056740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2319160 0x2292310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f97056740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2292800 con 0x2319160 2023-11-15T02:58:40.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f97056740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x22cc470 con 0x22afb50 2023-11-15T02:58:40.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f97056740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2292c70 con 0x226d210 2023-11-15T02:58:40.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f97056740 10 client.0 ll_register_callbacks cb 0x7f7f78015480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:58:40.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f86d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2319160 0x2292310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f7ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22afb50 0x2291e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f86d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2319160 0x2292310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33000/0 (socket says 172.21.15.121:33000) 2023-11-15T02:58:40.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f86d9d700 1 -- 172.21.15.121:0/592185749 learned_addr learned my addr 172.21.15.121:0/592185749 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:40.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f8659c700 1 --2- 172.21.15.121:0/592185749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226d210 0x22a9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f86d9d700 1 -- 172.21.15.121:0/592185749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226d210 msgr2=0x22a9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:40.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f86d9d700 1 --2- 172.21.15.121:0/592185749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226d210 0x22a9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f86d9d700 1 -- 172.21.15.121:0/592185749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22afb50 msgr2=0x2291e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:40.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f86d9d700 1 --2- 172.21.15.121:0/592185749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22afb50 0x2291e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f86d9d700 1 -- 172.21.15.121:0/592185749 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x21e67f0 con 0x2319160 2023-11-15T02:58:40.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f86d9d700 1 --2- 172.21.15.121:0/592185749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2319160 0x2292310 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f7f8004bb40 tx=0x7f7f80074040).ready entity=mon.0 client_cookie=90ea1ca0b054c815 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:40.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f64ff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:58:40.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f64ff9700 1 -- 172.21.15.121:0/592185749 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f80048230 con 0x2319160 2023-11-15T02:58:40.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f97056740 1 -- 172.21.15.121:0/592185749 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f78015bf0 con 0x2319160 2023-11-15T02:58:40.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f64ff9700 1 -- 172.21.15.121:0/592185749 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f7f80048390 con 0x2319160 2023-11-15T02:58:40.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f64ff9700 1 -- 172.21.15.121:0/592185749 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f8004b220 con 0x2319160 2023-11-15T02:58:40.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f97056740 10 client.5355 fetch_fsmap learned FSMap version 23 2023-11-15T02:58:40.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f97056740 1 -- 172.21.15.121:0/592185749 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x21e1a20 con 0x2319160 2023-11-15T02:58:40.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.558+0000 7f7f64ff9700 1 -- 172.21.15.121:0/592185749 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f78015bf0 con 0x2319160 2023-11-15T02:58:40.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f64ff9700 1 -- 172.21.15.121:0/592185749 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 23) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7f8001abd0 con 0x2319160 2023-11-15T02:58:40.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f97056740 10 client.5355 fetch_fsmap finished waiting for FSMap version 23 2023-11-15T02:58:40.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f97056740 10 client.5355 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-15T02:58:40.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f97056740 20 client.5355 populate_metadata read hostname 'smithi121' 2023-11-15T02:58:40.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f97056740 1 --2- 172.21.15.121:0/592185749 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x2314ab0 0x2314e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:40.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f97056740 4 client.5355 mds_command: new command op to 5178 tid=0[{"prefix": "damage rm", "damage_id": 4012680}] 2023-11-15T02:58:40.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f97056740 1 -- 172.21.15.121:0/592185749 --> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] -- command(tid 0: {"prefix": "damage rm", "damage_id": 4012680}) v1 -- 0x20a2660 con 0x2314ab0 2023-11-15T02:58:40.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f8659c700 1 --2- 172.21.15.121:0/592185749 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x2314ab0 0x2314e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:40.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f8659c700 1 --2- 172.21.15.121:0/592185749 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x2314ab0 0x2314e90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=7c6aa56f897da898 server_cookie=f56445c33eb302dc in_seq=0 out_seq=0 2023-11-15T02:58:40.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f64ff9700 10 client.5355 ms_handle_connect on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:40.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f64ff9700 1 -- 172.21.15.121:0/592185749 <== mds.0 v2:172.21.15.121:6836/3305580606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x20a2660 con 0x2314ab0 2023-11-15T02:58:40.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f64ff9700 10 client.5355 handle_command_reply: tid=0 2023-11-15T02:58:40.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f97056740 1 client.5355 shutdown 2023-11-15T02:58:40.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f97056740 1 -- 172.21.15.121:0/592185749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2319160 msgr2=0x2292310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:40.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f97056740 1 --2- 172.21.15.121:0/592185749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2319160 0x2292310 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f7f8004bb40 tx=0x7f7f80074040).stop 2023-11-15T02:58:40.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f97056740 1 -- 172.21.15.121:0/592185749 shutdown_connections 2023-11-15T02:58:40.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f97056740 1 --2- 172.21.15.121:0/592185749 >> [v2:172.21.15.121:6836/3305580606,v1:172.21.15.121:6837/3305580606] conn(0x2314ab0 0x2314e90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f97056740 1 --2- 172.21.15.121:0/592185749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2319160 0x2292310 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f64ff9700 0 client.5355 ms_handle_reset on v2:172.21.15.121:6836/3305580606 2023-11-15T02:58:40.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f97056740 1 --2- 172.21.15.121:0/592185749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22afb50 0x2291e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f97056740 1 --2- 172.21.15.121:0/592185749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226d210 0x22a9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f97056740 1 -- 172.21.15.121:0/592185749 >> 172.21.15.121:0/592185749 conn(0x204b6d0 msgr2=0x226ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:40.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f97056740 1 -- 172.21.15.121:0/592185749 shutdown_connections 2023-11-15T02:58:40.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f97056740 1 -- 172.21.15.121:0/592185749 wait complete. 2023-11-15T02:58:40.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.562+0000 7f7f97056740 20 client.5355 trim_cache size 0 max 16384 2023-11-15T02:58:40.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.566+0000 7f7f8ccdb700 1 -- 172.21.15.121:0/1803266396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7f6805a540 msgr2=0x7f7f6805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:40.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.566+0000 7f7f8ccdb700 1 --2- 172.21.15.121:0/1803266396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7f6805a540 0x7f7f6805ca00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f7f78009c00 tx=0x7f7f78008040).stop 2023-11-15T02:58:40.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.566+0000 7f7f8ccdb700 1 -- 172.21.15.121:0/1803266396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f88153c10 msgr2=0x7f7f88157100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:40.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.566+0000 7f7f8ccdb700 1 --2- 172.21.15.121:0/1803266396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f88153c10 0x7f7f88157100 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f7f8000c7f0 tx=0x7f7f80001510).stop 2023-11-15T02:58:40.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.566+0000 7f7f8ccdb700 1 -- 172.21.15.121:0/1803266396 shutdown_connections 2023-11-15T02:58:40.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.566+0000 7f7f8ccdb700 1 --2- 172.21.15.121:0/1803266396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7f6805a540 0x7f7f6805ca00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.566+0000 7f7f8ccdb700 1 --2- 172.21.15.121:0/1803266396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f88153c10 0x7f7f88157100 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.566+0000 7f7f8ccdb700 1 --2- 172.21.15.121:0/1803266396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f880bb6f0 0x7f7f88156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.566+0000 7f7f8ccdb700 1 --2- 172.21.15.121:0/1803266396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f88067060 0x7f7f8814e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:40.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.566+0000 7f7f8ccdb700 1 -- 172.21.15.121:0/1803266396 >> 172.21.15.121:0/1803266396 conn(0x7f7f880591c0 msgr2=0x7f7f880c3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:40.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.566+0000 7f7f8ccdb700 1 -- 172.21.15.121:0/1803266396 shutdown_connections 2023-11-15T02:58:40.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:40.566+0000 7f7f8ccdb700 1 -- 172.21.15.121:0/1803266396 wait complete. 2023-11-15T02:58:40.592 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-15T02:58:40.592 DEBUG:teuthology.orchestra.run.smithi121:> import sys 2023-11-15T02:58:40.593 DEBUG:teuthology.orchestra.run.smithi121:> import errno 2023-11-15T02:58:40.593 DEBUG:teuthology.orchestra.run.smithi121:> 2023-11-15T02:58:40.593 DEBUG:teuthology.orchestra.run.smithi121:> try: 2023-11-15T02:58:40.594 DEBUG:teuthology.orchestra.run.smithi121:> f = open("/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged", "w") 2023-11-15T02:58:40.594 DEBUG:teuthology.orchestra.run.smithi121:> f.close() 2023-11-15T02:58:40.594 DEBUG:teuthology.orchestra.run.smithi121:> except IOError as e: 2023-11-15T02:58:40.595 DEBUG:teuthology.orchestra.run.smithi121:> sys.exit(errno.EIO) 2023-11-15T02:58:40.595 DEBUG:teuthology.orchestra.run.smithi121:> ' 2023-11-15T02:58:40.885 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-15T02:58:40.885 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T02:58:40.969 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T02:58:41.015 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-15T02:58:41.015 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T02:58:41.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:58:41.016 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-15T02:58:41.063 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T02:58:41.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:58:41.064 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T02:58:41.128 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:58:41.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.330+0000 7f137945b700 1 Processor -- start 2023-11-15T02:58:41.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.330+0000 7f137945b700 1 -- start start 2023-11-15T02:58:41.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.334+0000 7f137945b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1374153d80 0x7f13740bdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:41.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.334+0000 7f137945b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f13740be480 0x7f13740c4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:41.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.334+0000 7f137945b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13740c2e40 con 0x7f13741539a0 2023-11-15T02:58:41.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.334+0000 7f137945b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13740c3120 con 0x7f13740be480 2023-11-15T02:58:41.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.334+0000 7f137945b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13740c3400 con 0x7f1374153d80 2023-11-15T02:58:41.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.334+0000 7f1373fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f13740be480 0x7f13740c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:41.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.334+0000 7f1373fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f13740be480 0x7f13740c4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33016/0 (socket says 172.21.15.121:33016) 2023-11-15T02:58:41.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.334+0000 7f1373fff700 1 -- 172.21.15.121:0/595813441 learned_addr learned my addr 172.21.15.121:0/595813441 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:41.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.334+0000 7f1372ffd700 1 --1- 172.21.15.121:0/595813441 >> v1:172.21.15.121:6789/0 conn(0x7f13741539a0 0x7f13740c23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39878/0 (socket says 172.21.15.121:39878) 2023-11-15T02:58:41.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.334+0000 7f13727fc700 1 --2- 172.21.15.121:0/595813441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1374153d80 0x7f13740bdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:41.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.334+0000 7f1373fff700 1 -- 172.21.15.121:0/595813441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1374153d80 msgr2=0x7f13740bdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:41.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.334+0000 7f1373fff700 1 --2- 172.21.15.121:0/595813441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1374153d80 0x7f13740bdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:41.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.334+0000 7f1373fff700 1 -- 172.21.15.121:0/595813441 >> v1:172.21.15.121:6789/0 conn(0x7f13741539a0 legacy=0x7f13740c23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:41.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.334+0000 7f1373fff700 1 -- 172.21.15.121:0/595813441 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13740c36e0 con 0x7f13740be480 2023-11-15T02:58:41.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.334+0000 7f1373fff700 1 --2- 172.21.15.121:0/595813441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13740be480 0x7f13740c4420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f136c009f20 tx=0x7f136c009840).ready entity=mon.0 client_cookie=a8216b8933b2ea40 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:41.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.334+0000 7f1371ffb700 1 -- 172.21.15.121:0/595813441 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c00e070 con 0x7f13740be480 2023-11-15T02:58:41.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.334+0000 7f1371ffb700 1 -- 172.21.15.121:0/595813441 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c00ba80 con 0x7f13740be480 2023-11-15T02:58:41.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.334+0000 7f1371ffb700 1 -- 172.21.15.121:0/595813441 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c00be00 con 0x7f13740be480 2023-11-15T02:58:41.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.338+0000 7f137945b700 1 -- 172.21.15.121:0/595813441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13740be480 msgr2=0x7f13740c4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:41.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.338+0000 7f137945b700 1 --2- 172.21.15.121:0/595813441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13740be480 0x7f13740c4420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f136c009f20 tx=0x7f136c009840).stop 2023-11-15T02:58:41.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.338+0000 7f137945b700 1 -- 172.21.15.121:0/595813441 shutdown_connections 2023-11-15T02:58:41.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.338+0000 7f137945b700 1 --2- 172.21.15.121:0/595813441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13740be480 0x7f13740c4420 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:41.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.338+0000 7f137945b700 1 --2- 172.21.15.121:0/595813441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1374153d80 0x7f13740bdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:41.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.338+0000 7f137945b700 1 -- 172.21.15.121:0/595813441 >> 172.21.15.121:0/595813441 conn(0x7f1374069ac0 msgr2=0x7f1374090a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:41.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.338+0000 7f137945b700 1 -- 172.21.15.121:0/595813441 shutdown_connections 2023-11-15T02:58:41.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.338+0000 7f137945b700 1 -- 172.21.15.121:0/595813441 wait complete. 2023-11-15T02:58:41.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.342+0000 7f137945b700 1 Processor -- start 2023-11-15T02:58:41.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.342+0000 7f137945b700 1 -- start start 2023-11-15T02:58:41.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.342+0000 7f137945b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13740be480 0x7f1374151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:41.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.342+0000 7f137945b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13741539a0 0x7f13740c1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:41.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.342+0000 7f137945b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1374153d80 0x7f13740a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:41.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.342+0000 7f137945b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13740aad00 con 0x7f1374153d80 2023-11-15T02:58:41.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.342+0000 7f137945b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13740aafe0 con 0x7f13741539a0 2023-11-15T02:58:41.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.342+0000 7f137945b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13740bedd0 con 0x7f13740be480 2023-11-15T02:58:41.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.342+0000 7f1373fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1374153d80 0x7f13740a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:41.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.342+0000 7f1373fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1374153d80 0x7f13740a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33018/0 (socket says 172.21.15.121:33018) 2023-11-15T02:58:41.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.342+0000 7f1372ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13740be480 0x7f1374151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:41.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.342+0000 7f1373fff700 1 -- 172.21.15.121:0/1107816290 learned_addr learned my addr 172.21.15.121:0/1107816290 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:41.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.342+0000 7f13727fc700 1 --2- 172.21.15.121:0/1107816290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13741539a0 0x7f13740c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:41.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.346+0000 7f1372ffd700 1 -- 172.21.15.121:0/1107816290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13741539a0 msgr2=0x7f13740c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:41.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.346+0000 7f1372ffd700 1 --2- 172.21.15.121:0/1107816290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13741539a0 0x7f13740c1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:41.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.346+0000 7f1372ffd700 1 -- 172.21.15.121:0/1107816290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1374153d80 msgr2=0x7f13740a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:41.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.346+0000 7f1372ffd700 1 --2- 172.21.15.121:0/1107816290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1374153d80 0x7f13740a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:41.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.346+0000 7f1372ffd700 1 -- 172.21.15.121:0/1107816290 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c009ae0 con 0x7f13740be480 2023-11-15T02:58:41.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.346+0000 7f1373fff700 1 --2- 172.21.15.121:0/1107816290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1374153d80 0x7f13740a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:58:41.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.346+0000 7f1372ffd700 1 --2- 172.21.15.121:0/1107816290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13740be480 0x7f1374151d20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f136400c440 tx=0x7f1364007d30).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:41.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.346+0000 7f135affd700 1 -- 172.21.15.121:0/1107816290 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13640070d0 con 0x7f13740be480 2023-11-15T02:58:41.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.346+0000 7f135affd700 1 -- 172.21.15.121:0/1107816290 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1364007230 con 0x7f13740be480 2023-11-15T02:58:41.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.346+0000 7f137945b700 1 -- 172.21.15.121:0/1107816290 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13740bf0b0 con 0x7f13740be480 2023-11-15T02:58:41.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.346+0000 7f135affd700 1 -- 172.21.15.121:0/1107816290 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13640075b0 con 0x7f13740be480 2023-11-15T02:58:41.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.346+0000 7f137945b700 1 -- 172.21.15.121:0/1107816290 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1374152210 con 0x7f13740be480 2023-11-15T02:58:41.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.346+0000 7f137945b700 1 -- 172.21.15.121:0/1107816290 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1374047060 con 0x7f13740be480 2023-11-15T02:58:41.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.350+0000 7f135affd700 1 -- 172.21.15.121:0/1107816290 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1364030a40 con 0x7f13740be480 2023-11-15T02:58:41.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.350+0000 7f135affd700 1 --2- 172.21.15.121:0/1107816290 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f136005a510 0x7f136005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:41.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.350+0000 7f135affd700 1 -- 172.21.15.121:0/1107816290 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f1364074200 con 0x7f13740be480 2023-11-15T02:58:41.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.350+0000 7f13727fc700 1 --2- 172.21.15.121:0/1107816290 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f136005a510 0x7f136005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:41.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.350+0000 7f13727fc700 1 --2- 172.21.15.121:0/1107816290 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f136005a510 0x7f136005c9d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f13740aded0 tx=0x7f1368008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:41.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.358+0000 7f135affd700 1 -- 172.21.15.121:0/1107816290 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1364047930 con 0x7f13740be480 2023-11-15T02:58:41.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.558+0000 7f137945b700 1 -- 172.21.15.121:0/1107816290 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1374124f10 con 0x7f13740be480 2023-11-15T02:58:41.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.558+0000 7f135affd700 1 -- 172.21.15.121:0/1107816290 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4502 (secure 0 0 0) 0x7f1364018070 con 0x7f13740be480 2023-11-15T02:58:41.565 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:58:41.565 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2544127447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2544127447},{"type":"v1","addr":"172.21.15.190:6833","nonce":2544127447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1815834193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1815834193},{"type":"v1","addr":"172.21.15.190:6835","nonce":1815834193}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4950,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3229917854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3229917854},{"type":"v1","addr":"172.21.15.121:6835","nonce":3229917854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:57:44.541243+0000","modified":"2023-11-15T02:58:10.996523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5178":{"gid":5178,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6837/3305580606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3305580606},{"type":"v1","addr":"172.21.15.121:6837","nonce":3305580606}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-15T02:58:41.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.562+0000 7f137945b700 1 -- 172.21.15.121:0/1107816290 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f136005a510 msgr2=0x7f136005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:41.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.562+0000 7f137945b700 1 --2- 172.21.15.121:0/1107816290 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f136005a510 0x7f136005c9d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f13740aded0 tx=0x7f1368008040).stop 2023-11-15T02:58:41.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.562+0000 7f137945b700 1 -- 172.21.15.121:0/1107816290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13740be480 msgr2=0x7f1374151d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:41.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.562+0000 7f137945b700 1 --2- 172.21.15.121:0/1107816290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13740be480 0x7f1374151d20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f136400c440 tx=0x7f1364007d30).stop 2023-11-15T02:58:41.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.562+0000 7f137945b700 1 -- 172.21.15.121:0/1107816290 shutdown_connections 2023-11-15T02:58:41.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.562+0000 7f137945b700 1 --2- 172.21.15.121:0/1107816290 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f136005a510 0x7f136005c9d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:41.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.562+0000 7f137945b700 1 --2- 172.21.15.121:0/1107816290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1374153d80 0x7f13740a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:41.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.562+0000 7f137945b700 1 --2- 172.21.15.121:0/1107816290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13741539a0 0x7f13740c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:41.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.562+0000 7f137945b700 1 --2- 172.21.15.121:0/1107816290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13740be480 0x7f1374151d20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:41.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.562+0000 7f137945b700 1 -- 172.21.15.121:0/1107816290 >> 172.21.15.121:0/1107816290 conn(0x7f1374069ac0 msgr2=0x7f1374152ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:41.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.562+0000 7f137945b700 1 -- 172.21.15.121:0/1107816290 shutdown_connections 2023-11-15T02:58:41.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.562+0000 7f137945b700 1 -- 172.21.15.121:0/1107816290 wait complete. 2023-11-15T02:58:41.570 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 23 2023-11-15T02:58:41.587 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:58:41.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.738+0000 7f590edf9700 1 Processor -- start 2023-11-15T02:58:41.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.738+0000 7f590edf9700 1 -- start start 2023-11-15T02:58:41.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.742+0000 7f590edf9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f59080cafb0 0x7f5908066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:41.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.742+0000 7f590edf9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5908067270 0x7f59080ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:41.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.742+0000 7f590edf9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59080ad3a0 con 0x7f59080cabd0 2023-11-15T02:58:41.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.742+0000 7f590edf9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59080cb410 con 0x7f59080cafb0 2023-11-15T02:58:41.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.742+0000 7f590edf9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59080ad650 con 0x7f5908067270 2023-11-15T02:58:41.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.742+0000 7f590ddf7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f59080cabd0 0x7f59080acc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39880/0 (socket says 172.21.15.121:39880) 2023-11-15T02:58:41.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.742+0000 7f590ddf7700 1 -- 172.21.15.121:0/385901870 learned_addr learned my addr 172.21.15.121:0/385901870 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:41.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.742+0000 7f590d5f6700 1 --2- 172.21.15.121:0/385901870 >> v2:172.21.15.121:3300/0 conn(0x7f59080cafb0 0x7f5908066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:41.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.742+0000 7f590e5f8700 1 --2- 172.21.15.121:0/385901870 >> v2:172.21.15.190:3300/0 conn(0x7f5908067270 0x7f59080ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:41.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.742+0000 7f590cdf5700 1 -- 172.21.15.121:0/385901870 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2866155770 0 0) 0x7f59080ad3a0 con 0x7f59080cabd0 2023-11-15T02:58:41.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.742+0000 7f590cdf5700 1 -- 172.21.15.121:0/385901870 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58f40033f0 con 0x7f59080cabd0 2023-11-15T02:58:41.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.742+0000 7f590d5f6700 1 -- 172.21.15.121:0/385901870 >> v2:172.21.15.190:3300/0 conn(0x7f5908067270 msgr2=0x7f59080ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:41.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.742+0000 7f590d5f6700 1 --2- 172.21.15.121:0/385901870 >> v2:172.21.15.190:3300/0 conn(0x7f5908067270 0x7f59080ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:41.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.742+0000 7f590d5f6700 1 -- 172.21.15.121:0/385901870 >> v1:172.21.15.121:6789/0 conn(0x7f59080cabd0 legacy=0x7f59080acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:41.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.742+0000 7f590d5f6700 1 -- 172.21.15.121:0/385901870 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5908065fc0 con 0x7f59080cafb0 2023-11-15T02:58:41.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.742+0000 7f590d5f6700 1 --2- 172.21.15.121:0/385901870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59080cafb0 0x7f5908066d80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f59080c7000 tx=0x7f58f8009060).ready entity=mon.0 client_cookie=b516fa199aa21ed0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:41.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.742+0000 7f590cdf5700 1 -- 172.21.15.121:0/385901870 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f800d4a0 con 0x7f59080cafb0 2023-11-15T02:58:41.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.746+0000 7f590cdf5700 1 -- 172.21.15.121:0/385901870 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f800d600 con 0x7f59080cafb0 2023-11-15T02:58:41.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.746+0000 7f590cdf5700 1 -- 172.21.15.121:0/385901870 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f800d980 con 0x7f59080cafb0 2023-11-15T02:58:41.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.746+0000 7f590edf9700 1 -- 172.21.15.121:0/385901870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59080cafb0 msgr2=0x7f5908066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:41.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.746+0000 7f590edf9700 1 --2- 172.21.15.121:0/385901870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59080cafb0 0x7f5908066d80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f59080c7000 tx=0x7f58f8009060).stop 2023-11-15T02:58:41.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.746+0000 7f590edf9700 1 -- 172.21.15.121:0/385901870 shutdown_connections 2023-11-15T02:58:41.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.746+0000 7f590edf9700 1 --2- 172.21.15.121:0/385901870 >> v2:172.21.15.190:3300/0 conn(0x7f5908067270 0x7f59080ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:41.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.746+0000 7f590edf9700 1 --2- 172.21.15.121:0/385901870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59080cafb0 0x7f5908066d80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:41.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.746+0000 7f590edf9700 1 -- 172.21.15.121:0/385901870 >> 172.21.15.121:0/385901870 conn(0x7f5908062190 msgr2=0x7f59080625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:41.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.746+0000 7f590edf9700 1 -- 172.21.15.121:0/385901870 shutdown_connections 2023-11-15T02:58:41.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.746+0000 7f590edf9700 1 -- 172.21.15.121:0/385901870 wait complete. 2023-11-15T02:58:41.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.746+0000 7f590edf9700 1 Processor -- start 2023-11-15T02:58:41.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f590edf9700 1 -- start start 2023-11-15T02:58:41.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f590edf9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5908067270 0x7f590816b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:41.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f590edf9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59080cabd0 0x7f5908081fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:41.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f590edf9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59080cafb0 0x7f59080824d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:41.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f590edf9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5908084b60 con 0x7f59080cabd0 2023-11-15T02:58:41.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f590edf9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590816b610 con 0x7f5908067270 2023-11-15T02:58:41.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f590edf9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59080829c0 con 0x7f59080cafb0 2023-11-15T02:58:41.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f590d5f6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59080cabd0 0x7f5908081fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:41.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f590d5f6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59080cabd0 0x7f5908081fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33032/0 (socket says 172.21.15.121:33032) 2023-11-15T02:58:41.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f590d5f6700 1 -- 172.21.15.121:0/1134416567 learned_addr learned my addr 172.21.15.121:0/1134416567 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:41.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f590ddf7700 1 --2- 172.21.15.121:0/1134416567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5908067270 0x7f590816b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:41.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f590e5f8700 1 --2- 172.21.15.121:0/1134416567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59080cafb0 0x7f59080824d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:41.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f590d5f6700 1 -- 172.21.15.121:0/1134416567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59080cafb0 msgr2=0x7f59080824d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:41.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f590d5f6700 1 --2- 172.21.15.121:0/1134416567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59080cafb0 0x7f59080824d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:41.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f590d5f6700 1 -- 172.21.15.121:0/1134416567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5908067270 msgr2=0x7f590816b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:41.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f590d5f6700 1 --2- 172.21.15.121:0/1134416567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5908067270 0x7f590816b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:41.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f590d5f6700 1 -- 172.21.15.121:0/1134416567 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f80092a0 con 0x7f59080cabd0 2023-11-15T02:58:41.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f590ddf7700 1 --2- 172.21.15.121:0/1134416567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5908067270 0x7f590816b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:41.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f590d5f6700 1 --2- 172.21.15.121:0/1134416567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59080cabd0 0x7f5908081fe0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f5908064f20 tx=0x7f58f8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:41.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f58fdffb700 1 -- 172.21.15.121:0/1134416567 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f8000c90 con 0x7f59080cabd0 2023-11-15T02:58:41.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f58fdffb700 1 -- 172.21.15.121:0/1134416567 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f800d770 con 0x7f59080cabd0 2023-11-15T02:58:41.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.750+0000 7f58fdffb700 1 -- 172.21.15.121:0/1134416567 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f801d430 con 0x7f59080cabd0 2023-11-15T02:58:41.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.754+0000 7f590edf9700 1 -- 172.21.15.121:0/1134416567 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5908084230 con 0x7f59080cabd0 2023-11-15T02:58:41.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.754+0000 7f590edf9700 1 -- 172.21.15.121:0/1134416567 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59080847f0 con 0x7f59080cabd0 2023-11-15T02:58:41.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.754+0000 7f590edf9700 1 -- 172.21.15.121:0/1134416567 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5908084cd0 con 0x7f59080cabd0 2023-11-15T02:58:41.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.754+0000 7f58fdffb700 1 -- 172.21.15.121:0/1134416567 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f58f8027440 con 0x7f59080cabd0 2023-11-15T02:58:41.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.754+0000 7f58fdffb700 1 --2- 172.21.15.121:0/1134416567 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58ec05a510 0x7f58ec05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:41.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.754+0000 7f58fdffb700 1 -- 172.21.15.121:0/1134416567 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f58f8075b20 con 0x7f59080cabd0 2023-11-15T02:58:41.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.758+0000 7f590ddf7700 1 --2- 172.21.15.121:0/1134416567 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58ec05a510 0x7f58ec05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:41.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.758+0000 7f590ddf7700 1 --2- 172.21.15.121:0/1134416567 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58ec05a510 0x7f58ec05c9d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f5908169420 tx=0x7f5904009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:41.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.762+0000 7f58fdffb700 1 -- 172.21.15.121:0/1134416567 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f58f8049230 con 0x7f59080cabd0 2023-11-15T02:58:41.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.926+0000 7f590edf9700 1 -- 172.21.15.121:0/1134416567 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f590806bcf0 con 0x7f59080cabd0 2023-11-15T02:58:41.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.926+0000 7f58fdffb700 1 -- 172.21.15.121:0/1134416567 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+17156 (secure 0 0 0) 0x7f58f8022030 con 0x7f59080cabd0 2023-11-15T02:58:41.934 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:58:41.935 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":36,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:58:39.889913+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:57:42.277674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-15T02:57:43.540361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.26","mappings":[{"from":5,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6837/3788932715":"2023-11-16T02:57:58.241271+0000","172.21.15.121:6836/3788932715":"2023-11-16T02:57:58.241271+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:58:41.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.930+0000 7f590edf9700 1 -- 172.21.15.121:0/1134416567 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58ec05a510 msgr2=0x7f58ec05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:41.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.930+0000 7f590edf9700 1 --2- 172.21.15.121:0/1134416567 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58ec05a510 0x7f58ec05c9d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f5908169420 tx=0x7f5904009040).stop 2023-11-15T02:58:41.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.930+0000 7f590edf9700 1 -- 172.21.15.121:0/1134416567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59080cabd0 msgr2=0x7f5908081fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:41.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.930+0000 7f590edf9700 1 --2- 172.21.15.121:0/1134416567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59080cabd0 0x7f5908081fe0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f5908064f20 tx=0x7f58f8016000).stop 2023-11-15T02:58:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.934+0000 7f590edf9700 1 -- 172.21.15.121:0/1134416567 shutdown_connections 2023-11-15T02:58:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.934+0000 7f590edf9700 1 --2- 172.21.15.121:0/1134416567 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58ec05a510 0x7f58ec05c9d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.934+0000 7f590edf9700 1 --2- 172.21.15.121:0/1134416567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59080cafb0 0x7f59080824d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.934+0000 7f590edf9700 1 --2- 172.21.15.121:0/1134416567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59080cabd0 0x7f5908081fe0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.934+0000 7f590edf9700 1 --2- 172.21.15.121:0/1134416567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5908067270 0x7f590816b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:41.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.934+0000 7f590edf9700 1 -- 172.21.15.121:0/1134416567 >> 172.21.15.121:0/1134416567 conn(0x7f5908062190 msgr2=0x7f5908151670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:41.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.934+0000 7f590edf9700 1 -- 172.21.15.121:0/1134416567 shutdown_connections 2023-11-15T02:58:41.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:41.934+0000 7f590edf9700 1 -- 172.21.15.121:0/1134416567 wait complete. 2023-11-15T02:58:41.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-15T02:57:42.277674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-15T02:57:43.540361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-15T02:58:41.958 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T02:58:42.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.114+0000 7f9699479700 1 Processor -- start 2023-11-15T02:58:42.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.114+0000 7f9699479700 1 -- start start 2023-11-15T02:58:42.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f9699479700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9694153830 0x7f96940acb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:42.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f9699479700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96940c7e10 0x7f9694153c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:42.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f9699479700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9694158160 0x7f9694062190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:42.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f9699479700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9694157710 con 0x7f9694153830 2023-11-15T02:58:42.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f9699479700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9694157cd0 con 0x7f9694158160 2023-11-15T02:58:42.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f9699479700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96940ad140 con 0x7f96940c7e10 2023-11-15T02:58:42.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f9693fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9694153830 0x7f96940acb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:42.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f9693fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9694153830 0x7f96940acb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33044/0 (socket says 172.21.15.121:33044) 2023-11-15T02:58:42.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f9693fff700 1 -- 172.21.15.121:0/3831428088 learned_addr learned my addr 172.21.15.121:0/3831428088 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:42.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f968b7fe700 1 --2- 172.21.15.121:0/3831428088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96940c7e10 0x7f9694153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:42.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f9698c78700 1 --2- 172.21.15.121:0/3831428088 >> v2:172.21.15.190:3300/0 conn(0x7f9694158160 0x7f9694062190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:42.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f9693fff700 1 -- 172.21.15.121:0/3831428088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96940c7e10 msgr2=0x7f9694153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:42.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f9693fff700 1 --2- 172.21.15.121:0/3831428088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96940c7e10 0x7f9694153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f9693fff700 1 -- 172.21.15.121:0/3831428088 >> v2:172.21.15.190:3300/0 conn(0x7f9694158160 msgr2=0x7f9694062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:42.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f9693fff700 1 --2- 172.21.15.121:0/3831428088 >> v2:172.21.15.190:3300/0 conn(0x7f9694158160 0x7f9694062190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f9693fff700 1 -- 172.21.15.121:0/3831428088 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96940ad420 con 0x7f9694153830 2023-11-15T02:58:42.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f9698c78700 1 --2- 172.21.15.121:0/3831428088 >> v2:172.21.15.190:3300/0 conn(0x7f9694158160 0x7f9694062190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:42.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f9693fff700 1 --2- 172.21.15.121:0/3831428088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9694153830 0x7f96940acb60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f9694157c40 tx=0x7f96840090f0).ready entity=mon.0 client_cookie=b10d1f4b64d17120 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:42.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f96937fe700 1 -- 172.21.15.121:0/3831428088 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968400b520 con 0x7f9694153830 2023-11-15T02:58:42.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f96937fe700 1 -- 172.21.15.121:0/3831428088 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968400b680 con 0x7f9694153830 2023-11-15T02:58:42.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f96937fe700 1 -- 172.21.15.121:0/3831428088 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968400ba00 con 0x7f9694153830 2023-11-15T02:58:42.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f9699479700 1 -- 172.21.15.121:0/3831428088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9694153830 msgr2=0x7f96940acb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:42.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.118+0000 7f9699479700 1 --2- 172.21.15.121:0/3831428088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9694153830 0x7f96940acb60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f9694157c40 tx=0x7f96840090f0).stop 2023-11-15T02:58:42.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.122+0000 7f9699479700 1 -- 172.21.15.121:0/3831428088 shutdown_connections 2023-11-15T02:58:42.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.122+0000 7f9699479700 1 --2- 172.21.15.121:0/3831428088 >> v2:172.21.15.190:3300/0 conn(0x7f9694158160 0x7f9694062190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.122+0000 7f9699479700 1 --2- 172.21.15.121:0/3831428088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96940c7e10 0x7f9694153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.122+0000 7f9699479700 1 --2- 172.21.15.121:0/3831428088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9694153830 0x7f96940acb60 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.122+0000 7f9699479700 1 -- 172.21.15.121:0/3831428088 >> 172.21.15.121:0/3831428088 conn(0x7f9694063570 msgr2=0x7f9694063950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:42.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.122+0000 7f9699479700 1 -- 172.21.15.121:0/3831428088 shutdown_connections 2023-11-15T02:58:42.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.122+0000 7f9699479700 1 -- 172.21.15.121:0/3831428088 wait complete. 2023-11-15T02:58:42.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.122+0000 7f9699479700 1 Processor -- start 2023-11-15T02:58:42.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f9699479700 1 -- start start 2023-11-15T02:58:42.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f9699479700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96940c7e10 0x7f96940c4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:42.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f9699479700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9694153830 0x7f96940ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:42.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f9699479700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9694158160 0x7f96940aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:42.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f9699479700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96940af2c0 con 0x7f9694153830 2023-11-15T02:58:42.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f9699479700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9694165200 con 0x7f9694158160 2023-11-15T02:58:42.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f9699479700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9694167c40 con 0x7f96940c7e10 2023-11-15T02:58:42.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f968b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9694153830 0x7f96940ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:42.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f968b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9694153830 0x7f96940ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33046/0 (socket says 172.21.15.121:33046) 2023-11-15T02:58:42.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f968b7fe700 1 -- 172.21.15.121:0/3459817072 learned_addr learned my addr 172.21.15.121:0/3459817072 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:42.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f9693fff700 1 --2- 172.21.15.121:0/3459817072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96940c7e10 0x7f96940c4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:42.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f9698c78700 1 --2- 172.21.15.121:0/3459817072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9694158160 0x7f96940aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:42.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f968b7fe700 1 -- 172.21.15.121:0/3459817072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96940c7e10 msgr2=0x7f96940c4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:42.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f968b7fe700 1 --2- 172.21.15.121:0/3459817072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96940c7e10 0x7f96940c4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f968b7fe700 1 -- 172.21.15.121:0/3459817072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9694158160 msgr2=0x7f96940aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:42.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f968b7fe700 1 --2- 172.21.15.121:0/3459817072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9694158160 0x7f96940aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f968b7fe700 1 -- 172.21.15.121:0/3459817072 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9684009370 con 0x7f9694153830 2023-11-15T02:58:42.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f9698c78700 1 --2- 172.21.15.121:0/3459817072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9694158160 0x7f96940aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:42.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f968b7fe700 1 --2- 172.21.15.121:0/3459817072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9694153830 0x7f96940ae8e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f967c00e950 tx=0x7f967c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:42.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f9690ff9700 1 -- 172.21.15.121:0/3459817072 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c00c6a0 con 0x7f9694153830 2023-11-15T02:58:42.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f9690ff9700 1 -- 172.21.15.121:0/3459817072 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c00c800 con 0x7f9694153830 2023-11-15T02:58:42.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f9690ff9700 1 -- 172.21.15.121:0/3459817072 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c00cb80 con 0x7f9694153830 2023-11-15T02:58:42.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f9699479700 1 -- 172.21.15.121:0/3459817072 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9694167f20 con 0x7f9694153830 2023-11-15T02:58:42.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.126+0000 7f9699479700 1 -- 172.21.15.121:0/3459817072 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96940c4570 con 0x7f9694153830 2023-11-15T02:58:42.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.130+0000 7f9699479700 1 -- 172.21.15.121:0/3459817072 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9694167aa0 con 0x7f9694153830 2023-11-15T02:58:42.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.130+0000 7f9690ff9700 1 -- 172.21.15.121:0/3459817072 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f967c01b400 con 0x7f9694153830 2023-11-15T02:58:42.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.134+0000 7f9690ff9700 1 --2- 172.21.15.121:0/3459817072 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f967405a540 0x7f967405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:42.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.134+0000 7f9690ff9700 1 -- 172.21.15.121:0/3459817072 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f967c074d20 con 0x7f9694153830 2023-11-15T02:58:42.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.134+0000 7f9693fff700 1 --2- 172.21.15.121:0/3459817072 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f967405a540 0x7f967405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:42.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.134+0000 7f9693fff700 1 --2- 172.21.15.121:0/3459817072 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f967405a540 0x7f967405ca00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f9684009130 tx=0x7f9684017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:42.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.138+0000 7f9690ff9700 1 -- 172.21.15.121:0/3459817072 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9694167aa0 con 0x7f9694153830 2023-11-15T02:58:42.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.298+0000 7f9699479700 1 -- 172.21.15.121:0/3459817072 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9694167aa0 con 0x7f9694153830 2023-11-15T02:58:42.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.298+0000 7f9690ff9700 1 -- 172.21.15.121:0/3459817072 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v36) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9694167aa0 con 0x7f9694153830 2023-11-15T02:58:42.304 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T02:58:42.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.298+0000 7f9699479700 1 -- 172.21.15.121:0/3459817072 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f967405a540 msgr2=0x7f967405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:42.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.298+0000 7f9699479700 1 --2- 172.21.15.121:0/3459817072 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f967405a540 0x7f967405ca00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f9684009130 tx=0x7f9684017000).stop 2023-11-15T02:58:42.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.298+0000 7f9699479700 1 -- 172.21.15.121:0/3459817072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9694153830 msgr2=0x7f96940ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:42.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.298+0000 7f9699479700 1 --2- 172.21.15.121:0/3459817072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9694153830 0x7f96940ae8e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f967c00e950 tx=0x7f967c009060).stop 2023-11-15T02:58:42.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.302+0000 7f9699479700 1 -- 172.21.15.121:0/3459817072 shutdown_connections 2023-11-15T02:58:42.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.302+0000 7f9699479700 1 --2- 172.21.15.121:0/3459817072 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f967405a540 0x7f967405ca00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.302+0000 7f9699479700 1 --2- 172.21.15.121:0/3459817072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9694158160 0x7f96940aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.302+0000 7f9699479700 1 --2- 172.21.15.121:0/3459817072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9694153830 0x7f96940ae8e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.302+0000 7f9699479700 1 --2- 172.21.15.121:0/3459817072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96940c7e10 0x7f96940c4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.302+0000 7f9699479700 1 -- 172.21.15.121:0/3459817072 >> 172.21.15.121:0/3459817072 conn(0x7f9694063570 msgr2=0x7f9694152530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:42.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.302+0000 7f9699479700 1 -- 172.21.15.121:0/3459817072 shutdown_connections 2023-11-15T02:58:42.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.302+0000 7f9699479700 1 -- 172.21.15.121:0/3459817072 wait complete. 2023-11-15T02:58:42.326 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T02:58:42.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.474+0000 7f9749ce4700 1 Processor -- start 2023-11-15T02:58:42.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.474+0000 7f9749ce4700 1 -- start start 2023-11-15T02:58:42.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.478+0000 7f9749ce4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f97440acd00 0x7f97440ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:42.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.478+0000 7f9749ce4700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9744066330 0x7f9744153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:42.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.478+0000 7f9749ce4700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9744065ea0 con 0x7f97440c5c70 2023-11-15T02:58:42.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.478+0000 7f9749ce4700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9744066bb0 con 0x7f9744066330 2023-11-15T02:58:42.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.478+0000 7f9749ce4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9744065890 con 0x7f97440acd00 2023-11-15T02:58:42.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.478+0000 7f9743fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f97440c5c70 0x7f97441536c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39882/0 (socket says 172.21.15.121:39882) 2023-11-15T02:58:42.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.478+0000 7f9748ce2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f97440acd00 0x7f97440ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:42.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.478+0000 7f9743fff700 1 -- 172.21.15.121:0/3930854529 learned_addr learned my addr 172.21.15.121:0/3930854529 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:42.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.478+0000 7f97494e3700 1 --2- 172.21.15.121:0/3930854529 >> v2:172.21.15.121:3300/0 conn(0x7f9744066330 0x7f9744153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:42.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.478+0000 7f97437fe700 1 -- 172.21.15.121:0/3930854529 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545976940 0 0) 0x7f9744065ea0 con 0x7f97440c5c70 2023-11-15T02:58:42.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.478+0000 7f97437fe700 1 -- 172.21.15.121:0/3930854529 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97340033f0 con 0x7f97440c5c70 2023-11-15T02:58:42.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.478+0000 7f9748ce2700 1 -- 172.21.15.121:0/3930854529 >> v2:172.21.15.121:3300/0 conn(0x7f9744066330 msgr2=0x7f9744153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:42.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.478+0000 7f9748ce2700 1 --2- 172.21.15.121:0/3930854529 >> v2:172.21.15.121:3300/0 conn(0x7f9744066330 0x7f9744153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.478+0000 7f9748ce2700 1 -- 172.21.15.121:0/3930854529 >> v1:172.21.15.121:6789/0 conn(0x7f97440c5c70 legacy=0x7f97441536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:42.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.478+0000 7f9748ce2700 1 -- 172.21.15.121:0/3930854529 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97440c64a0 con 0x7f97440acd00 2023-11-15T02:58:42.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.478+0000 7f97494e3700 1 --2- 172.21.15.121:0/3930854529 >> v2:172.21.15.121:3300/0 conn(0x7f9744066330 0x7f9744153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:58:42.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.478+0000 7f9748ce2700 1 --2- 172.21.15.121:0/3930854529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97440acd00 0x7f97440ad0e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f974406c280 tx=0x7f9738006f40).ready entity=mon.1 client_cookie=803effeecf2f5588 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:42.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.482+0000 7f97437fe700 1 -- 172.21.15.121:0/3930854529 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738005880 con 0x7f97440acd00 2023-11-15T02:58:42.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.482+0000 7f97437fe700 1 -- 172.21.15.121:0/3930854529 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97380059e0 con 0x7f97440acd00 2023-11-15T02:58:42.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.482+0000 7f97437fe700 1 -- 172.21.15.121:0/3930854529 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738005d60 con 0x7f97440acd00 2023-11-15T02:58:42.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.482+0000 7f9749ce4700 1 -- 172.21.15.121:0/3930854529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97440acd00 msgr2=0x7f97440ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:42.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.482+0000 7f9749ce4700 1 --2- 172.21.15.121:0/3930854529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97440acd00 0x7f97440ad0e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f974406c280 tx=0x7f9738006f40).stop 2023-11-15T02:58:42.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.482+0000 7f9749ce4700 1 -- 172.21.15.121:0/3930854529 shutdown_connections 2023-11-15T02:58:42.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.482+0000 7f9749ce4700 1 --2- 172.21.15.121:0/3930854529 >> v2:172.21.15.121:3300/0 conn(0x7f9744066330 0x7f9744153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.482+0000 7f9749ce4700 1 --2- 172.21.15.121:0/3930854529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97440acd00 0x7f97440ad0e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.482+0000 7f9749ce4700 1 -- 172.21.15.121:0/3930854529 >> 172.21.15.121:0/3930854529 conn(0x7f9744063fc0 msgr2=0x7f9744067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:42.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.482+0000 7f9749ce4700 1 -- 172.21.15.121:0/3930854529 shutdown_connections 2023-11-15T02:58:42.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.482+0000 7f9749ce4700 1 -- 172.21.15.121:0/3930854529 wait complete. 2023-11-15T02:58:42.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.482+0000 7f9749ce4700 1 Processor -- start 2023-11-15T02:58:42.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.482+0000 7f9749ce4700 1 -- start start 2023-11-15T02:58:42.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9749ce4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9744066330 0x7f9744150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:42.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9749ce4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97440acd00 0x7f97440c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:42.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9749ce4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97440c5c70 0x7f97440c4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:42.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9749ce4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97440c2120 con 0x7f97440acd00 2023-11-15T02:58:42.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9749ce4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97441651e0 con 0x7f97440c5c70 2023-11-15T02:58:42.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9749ce4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9744164c60 con 0x7f9744066330 2023-11-15T02:58:42.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9743fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97440acd00 0x7f97440c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:42.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9743fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97440acd00 0x7f97440c3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33064/0 (socket says 172.21.15.121:33064) 2023-11-15T02:58:42.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9743fff700 1 -- 172.21.15.121:0/3955682076 learned_addr learned my addr 172.21.15.121:0/3955682076 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:42.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f97494e3700 1 --2- 172.21.15.121:0/3955682076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97440c5c70 0x7f97440c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:42.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9748ce2700 1 --2- 172.21.15.121:0/3955682076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9744066330 0x7f9744150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:42.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9743fff700 1 -- 172.21.15.121:0/3955682076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9744066330 msgr2=0x7f9744150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:42.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9743fff700 1 --2- 172.21.15.121:0/3955682076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9744066330 0x7f9744150910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9743fff700 1 -- 172.21.15.121:0/3955682076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97440c5c70 msgr2=0x7f97440c4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:42.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9743fff700 1 --2- 172.21.15.121:0/3955682076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97440c5c70 0x7f97440c4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9743fff700 1 -- 172.21.15.121:0/3955682076 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c009a20 con 0x7f97440acd00 2023-11-15T02:58:42.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9743fff700 1 --2- 172.21.15.121:0/3955682076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97440acd00 0x7f97440c3e10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f973000c420 tx=0x7f9730007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:42.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9740ff9700 1 -- 172.21.15.121:0/3955682076 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97300072e0 con 0x7f97440acd00 2023-11-15T02:58:42.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9740ff9700 1 -- 172.21.15.121:0/3955682076 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9730007440 con 0x7f97440acd00 2023-11-15T02:58:42.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9749ce4700 1 -- 172.21.15.121:0/3955682076 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9738009250 con 0x7f97440acd00 2023-11-15T02:58:42.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9740ff9700 1 -- 172.21.15.121:0/3955682076 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97300057a0 con 0x7f97440acd00 2023-11-15T02:58:42.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.486+0000 7f9749ce4700 1 -- 172.21.15.121:0/3955682076 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97440c1b60 con 0x7f97440acd00 2023-11-15T02:58:42.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.490+0000 7f9749ce4700 1 -- 172.21.15.121:0/3955682076 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974406bcf0 con 0x7f97440acd00 2023-11-15T02:58:42.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.490+0000 7f9740ff9700 1 -- 172.21.15.121:0/3955682076 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9730005bd0 con 0x7f97440acd00 2023-11-15T02:58:42.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.490+0000 7f9740ff9700 1 --2- 172.21.15.121:0/3955682076 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f972805a510 0x7f972805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:42.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.490+0000 7f9740ff9700 1 -- 172.21.15.121:0/3955682076 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f9730073d80 con 0x7f97440acd00 2023-11-15T02:58:42.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.490+0000 7f9748ce2700 1 --2- 172.21.15.121:0/3955682076 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f972805a510 0x7f972805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:42.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.490+0000 7f9748ce2700 1 --2- 172.21.15.121:0/3955682076 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f972805a510 0x7f972805c9d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f9738006fa0 tx=0x7f9738015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:42.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.494+0000 7f9740ff9700 1 -- 172.21.15.121:0/3955682076 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9730047430 con 0x7f97440acd00 2023-11-15T02:58:42.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.658+0000 7f9749ce4700 1 -- 172.21.15.121:0/3955682076 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f97440c81b0 con 0x7f97440acd00 2023-11-15T02:58:42.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.658+0000 7f9740ff9700 1 -- 172.21.15.121:0/3955682076 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v36) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9730018070 con 0x7f97440acd00 2023-11-15T02:58:42.665 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T02:58:42.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.662+0000 7f9749ce4700 1 -- 172.21.15.121:0/3955682076 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f972805a510 msgr2=0x7f972805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:42.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.662+0000 7f9749ce4700 1 --2- 172.21.15.121:0/3955682076 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f972805a510 0x7f972805c9d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f9738006fa0 tx=0x7f9738015000).stop 2023-11-15T02:58:42.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.662+0000 7f9749ce4700 1 -- 172.21.15.121:0/3955682076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97440acd00 msgr2=0x7f97440c3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:42.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.662+0000 7f9749ce4700 1 --2- 172.21.15.121:0/3955682076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97440acd00 0x7f97440c3e10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f973000c420 tx=0x7f9730007c80).stop 2023-11-15T02:58:42.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.662+0000 7f9749ce4700 1 -- 172.21.15.121:0/3955682076 shutdown_connections 2023-11-15T02:58:42.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.662+0000 7f9749ce4700 1 --2- 172.21.15.121:0/3955682076 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f972805a510 0x7f972805c9d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.662+0000 7f9749ce4700 1 --2- 172.21.15.121:0/3955682076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97440c5c70 0x7f97440c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.662+0000 7f9749ce4700 1 --2- 172.21.15.121:0/3955682076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97440acd00 0x7f97440c3e10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.662+0000 7f9749ce4700 1 --2- 172.21.15.121:0/3955682076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9744066330 0x7f9744150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.662+0000 7f9749ce4700 1 -- 172.21.15.121:0/3955682076 >> 172.21.15.121:0/3955682076 conn(0x7f9744063fc0 msgr2=0x7f9744067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:42.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.662+0000 7f9749ce4700 1 -- 172.21.15.121:0/3955682076 shutdown_connections 2023-11-15T02:58:42.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.662+0000 7f9749ce4700 1 -- 172.21.15.121:0/3955682076 wait complete. 2023-11-15T02:58:42.688 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-15T02:58:42.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.846+0000 7f818b039700 1 Processor -- start 2023-11-15T02:58:42.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.850+0000 7f818b039700 1 -- start start 2023-11-15T02:58:42.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.850+0000 7f818b039700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f818414e380 0x7f81840c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:42.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f818b039700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f81840c6040 0x7f81840acb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:42.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f818b039700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81840ad500 con 0x7f81840ad050 2023-11-15T02:58:42.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f818b039700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81840c8320 con 0x7f81840c6040 2023-11-15T02:58:42.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f818b039700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81840c7080 con 0x7f818414e380 2023-11-15T02:58:42.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f8189836700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f81840c6040 0x7f81840acb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:42.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f8189836700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f81840c6040 0x7f81840acb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33070/0 (socket says 172.21.15.121:33070) 2023-11-15T02:58:42.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f8189836700 1 -- 172.21.15.121:0/855769480 learned_addr learned my addr 172.21.15.121:0/855769480 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:42.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f818a037700 1 --2- 172.21.15.121:0/855769480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f818414e380 0x7f81840c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:42.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f8189836700 1 -- 172.21.15.121:0/855769480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f818414e380 msgr2=0x7f81840c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:42.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f8189836700 1 --2- 172.21.15.121:0/855769480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f818414e380 0x7f81840c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f8189836700 1 -- 172.21.15.121:0/855769480 >> v1:172.21.15.190:6789/0 conn(0x7f81840ad050 legacy=0x7f8184062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:42.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f8189836700 1 -- 172.21.15.121:0/855769480 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81840c7e90 con 0x7f81840c6040 2023-11-15T02:58:42.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f8189836700 1 --2- 172.21.15.121:0/855769480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81840c6040 0x7f81840acb60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f8174006fd0 tx=0x7f81740090f0).ready entity=mon.0 client_cookie=ef818e8c8768e1ec server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:42.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f8189035700 1 -- 172.21.15.121:0/855769480 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817400b600 con 0x7f81840c6040 2023-11-15T02:58:42.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f8189035700 1 -- 172.21.15.121:0/855769480 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817400b760 con 0x7f81840c6040 2023-11-15T02:58:42.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f8189035700 1 -- 172.21.15.121:0/855769480 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817400bae0 con 0x7f81840c6040 2023-11-15T02:58:42.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f818b039700 1 -- 172.21.15.121:0/855769480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81840c6040 msgr2=0x7f81840acb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:42.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f818b039700 1 --2- 172.21.15.121:0/855769480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81840c6040 0x7f81840acb60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f8174006fd0 tx=0x7f81740090f0).stop 2023-11-15T02:58:42.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f818b039700 1 -- 172.21.15.121:0/855769480 shutdown_connections 2023-11-15T02:58:42.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f818b039700 1 --2- 172.21.15.121:0/855769480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81840c6040 0x7f81840acb60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f818b039700 1 --2- 172.21.15.121:0/855769480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f818414e380 0x7f81840c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f818b039700 1 -- 172.21.15.121:0/855769480 >> 172.21.15.121:0/855769480 conn(0x7f8184067a60 msgr2=0x7f8184067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:42.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f818b039700 1 -- 172.21.15.121:0/855769480 shutdown_connections 2023-11-15T02:58:42.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.854+0000 7f818b039700 1 -- 172.21.15.121:0/855769480 wait complete. 2023-11-15T02:58:42.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.858+0000 7f818b039700 1 Processor -- start 2023-11-15T02:58:42.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.858+0000 7f818b039700 1 -- start start 2023-11-15T02:58:42.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.858+0000 7f818b039700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81840ad050 0x7f81840c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:42.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.858+0000 7f818b039700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81840c6040 0x7f818414a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:42.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.858+0000 7f818b039700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f818414e380 0x7f81840afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:42.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.858+0000 7f818b039700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81840adc20 con 0x7f81840ad050 2023-11-15T02:58:42.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.858+0000 7f818b039700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81840aeeb0 con 0x7f818414e380 2023-11-15T02:58:42.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.858+0000 7f818b039700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81840af120 con 0x7f81840c6040 2023-11-15T02:58:42.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.858+0000 7f818a037700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81840ad050 0x7f81840c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:42.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.858+0000 7f818a037700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81840ad050 0x7f81840c3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33082/0 (socket says 172.21.15.121:33082) 2023-11-15T02:58:42.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.858+0000 7f818a037700 1 -- 172.21.15.121:0/319686012 learned_addr learned my addr 172.21.15.121:0/319686012 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:42.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.858+0000 7f818a838700 1 --2- 172.21.15.121:0/319686012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f818414e380 0x7f81840afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:42.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.858+0000 7f8189836700 1 --2- 172.21.15.121:0/319686012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81840c6040 0x7f818414a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:42.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.858+0000 7f818a037700 1 -- 172.21.15.121:0/319686012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81840c6040 msgr2=0x7f818414a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:42.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.858+0000 7f818a037700 1 --2- 172.21.15.121:0/319686012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81840c6040 0x7f818414a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.858+0000 7f818a037700 1 -- 172.21.15.121:0/319686012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f818414e380 msgr2=0x7f81840afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:42.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.858+0000 7f818a037700 1 --2- 172.21.15.121:0/319686012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f818414e380 0x7f81840afc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:42.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.858+0000 7f818a037700 1 -- 172.21.15.121:0/319686012 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8174009370 con 0x7f81840ad050 2023-11-15T02:58:42.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.858+0000 7f818a037700 1 --2- 172.21.15.121:0/319686012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81840ad050 0x7f81840c3f10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f818000d8c0 tx=0x7f8180008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:42.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.862+0000 7f817a7fc700 1 -- 172.21.15.121:0/319686012 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180011590 con 0x7f81840ad050 2023-11-15T02:58:42.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.862+0000 7f817a7fc700 1 -- 172.21.15.121:0/319686012 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81800116f0 con 0x7f81840ad050 2023-11-15T02:58:42.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.862+0000 7f817a7fc700 1 -- 172.21.15.121:0/319686012 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180011a70 con 0x7f81840ad050 2023-11-15T02:58:42.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.862+0000 7f818b039700 1 -- 172.21.15.121:0/319686012 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81840af400 con 0x7f81840ad050 2023-11-15T02:58:42.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.862+0000 7f818b039700 1 -- 172.21.15.121:0/319686012 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81840c4760 con 0x7f81840ad050 2023-11-15T02:58:42.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.862+0000 7f818b039700 1 -- 172.21.15.121:0/319686012 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81840af880 con 0x7f81840ad050 2023-11-15T02:58:42.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.862+0000 7f817a7fc700 1 -- 172.21.15.121:0/319686012 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8180012420 con 0x7f81840ad050 2023-11-15T02:58:42.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.866+0000 7f817a7fc700 1 --2- 172.21.15.121:0/319686012 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f816805a510 0x7f816805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:42.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.866+0000 7f817a7fc700 1 -- 172.21.15.121:0/319686012 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f81800741d0 con 0x7f81840ad050 2023-11-15T02:58:42.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.866+0000 7f8189836700 1 --2- 172.21.15.121:0/319686012 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f816805a510 0x7f816805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:42.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.866+0000 7f8189836700 1 --2- 172.21.15.121:0/319686012 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f816805a510 0x7f816805c9d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f8174006fd0 tx=0x7f8174017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:42.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:42.870+0000 7f817a7fc700 1 -- 172.21.15.121:0/319686012 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8180047820 con 0x7f81840ad050 2023-11-15T02:58:43.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.038+0000 7f818b039700 1 -- 172.21.15.121:0/319686012 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f818406bcf0 con 0x7f81840ad050 2023-11-15T02:58:43.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.038+0000 7f817a7fc700 1 -- 172.21.15.121:0/319686012 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8180011e60 con 0x7f81840ad050 2023-11-15T02:58:43.045 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T02:58:43.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.042+0000 7f818b039700 1 -- 172.21.15.121:0/319686012 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f816805a510 msgr2=0x7f816805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:43.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.042+0000 7f818b039700 1 --2- 172.21.15.121:0/319686012 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f816805a510 0x7f816805c9d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f8174006fd0 tx=0x7f8174017000).stop 2023-11-15T02:58:43.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.042+0000 7f818b039700 1 -- 172.21.15.121:0/319686012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81840ad050 msgr2=0x7f81840c3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:43.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.042+0000 7f818b039700 1 --2- 172.21.15.121:0/319686012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81840ad050 0x7f81840c3f10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f818000d8c0 tx=0x7f8180008040).stop 2023-11-15T02:58:43.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.042+0000 7f818b039700 1 -- 172.21.15.121:0/319686012 shutdown_connections 2023-11-15T02:58:43.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.042+0000 7f818b039700 1 --2- 172.21.15.121:0/319686012 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f816805a510 0x7f816805c9d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.042+0000 7f818b039700 1 --2- 172.21.15.121:0/319686012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f818414e380 0x7f81840afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.042+0000 7f818b039700 1 --2- 172.21.15.121:0/319686012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81840c6040 0x7f818414a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.042+0000 7f818b039700 1 --2- 172.21.15.121:0/319686012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81840ad050 0x7f81840c3f10 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.042+0000 7f818b039700 1 -- 172.21.15.121:0/319686012 >> 172.21.15.121:0/319686012 conn(0x7f8184067a60 msgr2=0x7f818414c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:43.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.042+0000 7f818b039700 1 -- 172.21.15.121:0/319686012 shutdown_connections 2023-11-15T02:58:43.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.042+0000 7f818b039700 1 -- 172.21.15.121:0/319686012 wait complete. 2023-11-15T02:58:43.070 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-15T02:58:43.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.226+0000 7f4b99839700 1 Processor -- start 2023-11-15T02:58:43.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.226+0000 7f4b99839700 1 -- start start 2023-11-15T02:58:43.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.230+0000 7f4b99839700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4b940acd00 0x7f4b940ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:43.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.230+0000 7f4b99839700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b94066330 0x7f4b94153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:43.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.230+0000 7f4b99839700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b94065ea0 con 0x7f4b940c5c70 2023-11-15T02:58:43.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.230+0000 7f4b99839700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b94066bb0 con 0x7f4b940acd00 2023-11-15T02:58:43.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.230+0000 7f4b99839700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b94065890 con 0x7f4b94066330 2023-11-15T02:58:43.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b93fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4b940c5c70 0x7f4b941536c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39884/0 (socket says 172.21.15.121:39884) 2023-11-15T02:58:43.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b93fff700 1 -- 172.21.15.121:0/4072396447 learned_addr learned my addr 172.21.15.121:0/4072396447 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:43.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b98837700 1 --2- 172.21.15.121:0/4072396447 >> v2:172.21.15.121:3300/0 conn(0x7f4b940acd00 0x7f4b940ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:43.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b99038700 1 --2- 172.21.15.121:0/4072396447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b94066330 0x7f4b94153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:43.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b937fe700 1 -- 172.21.15.121:0/4072396447 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107730009 0 0) 0x7f4b94065ea0 con 0x7f4b940c5c70 2023-11-15T02:58:43.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b937fe700 1 -- 172.21.15.121:0/4072396447 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b840033f0 con 0x7f4b940c5c70 2023-11-15T02:58:43.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b98837700 1 -- 172.21.15.121:0/4072396447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b94066330 msgr2=0x7f4b94153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:43.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b98837700 1 --2- 172.21.15.121:0/4072396447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b94066330 0x7f4b94153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b98837700 1 -- 172.21.15.121:0/4072396447 >> v1:172.21.15.121:6789/0 conn(0x7f4b940c5c70 legacy=0x7f4b941536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:43.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b98837700 1 -- 172.21.15.121:0/4072396447 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b940c64a0 con 0x7f4b940acd00 2023-11-15T02:58:43.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b98837700 1 --2- 172.21.15.121:0/4072396447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b940acd00 0x7f4b940ad0e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f4b88009750 tx=0x7f4b88009060).ready entity=mon.0 client_cookie=6df6d567988f5f75 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:43.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b937fe700 1 -- 172.21.15.121:0/4072396447 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8800d4a0 con 0x7f4b940acd00 2023-11-15T02:58:43.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b937fe700 1 -- 172.21.15.121:0/4072396447 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8800d600 con 0x7f4b940acd00 2023-11-15T02:58:43.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b937fe700 1 -- 172.21.15.121:0/4072396447 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8800d980 con 0x7f4b940acd00 2023-11-15T02:58:43.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b99839700 1 -- 172.21.15.121:0/4072396447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b940acd00 msgr2=0x7f4b940ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:43.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b99839700 1 --2- 172.21.15.121:0/4072396447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b940acd00 0x7f4b940ad0e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f4b88009750 tx=0x7f4b88009060).stop 2023-11-15T02:58:43.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b99839700 1 -- 172.21.15.121:0/4072396447 shutdown_connections 2023-11-15T02:58:43.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b99839700 1 --2- 172.21.15.121:0/4072396447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b94066330 0x7f4b94153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b99839700 1 --2- 172.21.15.121:0/4072396447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b940acd00 0x7f4b940ad0e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b99839700 1 -- 172.21.15.121:0/4072396447 >> 172.21.15.121:0/4072396447 conn(0x7f4b94063fc0 msgr2=0x7f4b94067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:43.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b99839700 1 -- 172.21.15.121:0/4072396447 shutdown_connections 2023-11-15T02:58:43.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.234+0000 7f4b99839700 1 -- 172.21.15.121:0/4072396447 wait complete. 2023-11-15T02:58:43.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.238+0000 7f4b99839700 1 Processor -- start 2023-11-15T02:58:43.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.238+0000 7f4b99839700 1 -- start start 2023-11-15T02:58:43.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b99839700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b94066330 0x7f4b94150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:43.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b99839700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b940acd00 0x7f4b94158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:43.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b99839700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b940c5c70 0x7f4b94158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:43.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b99839700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b94158c10 con 0x7f4b94066330 2023-11-15T02:58:43.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b99839700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b94157090 con 0x7f4b940c5c70 2023-11-15T02:58:43.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b99839700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b941561e0 con 0x7f4b940acd00 2023-11-15T02:58:43.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b98837700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b94066330 0x7f4b94150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:43.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b98837700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b94066330 0x7f4b94150830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33092/0 (socket says 172.21.15.121:33092) 2023-11-15T02:58:43.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b98837700 1 -- 172.21.15.121:0/1554380559 learned_addr learned my addr 172.21.15.121:0/1554380559 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:43.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b99038700 1 --2- 172.21.15.121:0/1554380559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b940c5c70 0x7f4b94158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:43.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b93fff700 1 --2- 172.21.15.121:0/1554380559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b940acd00 0x7f4b94158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:43.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b98837700 1 -- 172.21.15.121:0/1554380559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b940acd00 msgr2=0x7f4b94158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:43.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b98837700 1 --2- 172.21.15.121:0/1554380559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b940acd00 0x7f4b94158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b98837700 1 -- 172.21.15.121:0/1554380559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b940c5c70 msgr2=0x7f4b94158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:43.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b98837700 1 --2- 172.21.15.121:0/1554380559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b940c5c70 0x7f4b94158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b98837700 1 -- 172.21.15.121:0/1554380559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b88009240 con 0x7f4b94066330 2023-11-15T02:58:43.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b98837700 1 --2- 172.21.15.121:0/1554380559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b94066330 0x7f4b94150830 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f4b88010000 tx=0x7f4b88016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:43.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b90ff9700 1 -- 172.21.15.121:0/1554380559 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b88000c90 con 0x7f4b94066330 2023-11-15T02:58:43.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b90ff9700 1 -- 172.21.15.121:0/1554380559 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8800d770 con 0x7f4b94066330 2023-11-15T02:58:43.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b90ff9700 1 -- 172.21.15.121:0/1554380559 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8801c420 con 0x7f4b94066330 2023-11-15T02:58:43.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.242+0000 7f4b99839700 1 -- 172.21.15.121:0/1554380559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b94156480 con 0x7f4b94066330 2023-11-15T02:58:43.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.246+0000 7f4b99839700 1 -- 172.21.15.121:0/1554380559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b94156a40 con 0x7f4b94066330 2023-11-15T02:58:43.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.246+0000 7f4b99839700 1 -- 172.21.15.121:0/1554380559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b9406bcf0 con 0x7f4b94066330 2023-11-15T02:58:43.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.250+0000 7f4b90ff9700 1 -- 172.21.15.121:0/1554380559 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4b8801c850 con 0x7f4b94066330 2023-11-15T02:58:43.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.250+0000 7f4b90ff9700 1 --2- 172.21.15.121:0/1554380559 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4b7805a4e0 0x7f4b7805c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:43.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.250+0000 7f4b90ff9700 1 -- 172.21.15.121:0/1554380559 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f4b88076840 con 0x7f4b94066330 2023-11-15T02:58:43.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.250+0000 7f4b93fff700 1 --2- 172.21.15.121:0/1554380559 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4b7805a4e0 0x7f4b7805c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:43.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.250+0000 7f4b93fff700 1 --2- 172.21.15.121:0/1554380559 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4b7805a4e0 0x7f4b7805c9a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f4b80005750 tx=0x7f4b80009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:43.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.254+0000 7f4b90ff9700 1 -- 172.21.15.121:0/1554380559 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4b88049fb0 con 0x7f4b94066330 2023-11-15T02:58:43.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.422+0000 7f4b99839700 1 -- 172.21.15.121:0/1554380559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4b940c61c0 con 0x7f4b94066330 2023-11-15T02:58:43.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.426+0000 7f4b90ff9700 1 -- 172.21.15.121:0/1554380559 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4b8804d5d0 con 0x7f4b94066330 2023-11-15T02:58:43.431 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T02:58:43.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.426+0000 7f4b99839700 1 -- 172.21.15.121:0/1554380559 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4b7805a4e0 msgr2=0x7f4b7805c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:43.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.426+0000 7f4b99839700 1 --2- 172.21.15.121:0/1554380559 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4b7805a4e0 0x7f4b7805c9a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f4b80005750 tx=0x7f4b80009040).stop 2023-11-15T02:58:43.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.426+0000 7f4b99839700 1 -- 172.21.15.121:0/1554380559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b94066330 msgr2=0x7f4b94150830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:43.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.426+0000 7f4b99839700 1 --2- 172.21.15.121:0/1554380559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b94066330 0x7f4b94150830 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f4b88010000 tx=0x7f4b88016000).stop 2023-11-15T02:58:43.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.426+0000 7f4b99839700 1 -- 172.21.15.121:0/1554380559 shutdown_connections 2023-11-15T02:58:43.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.426+0000 7f4b99839700 1 --2- 172.21.15.121:0/1554380559 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4b7805a4e0 0x7f4b7805c9a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.426+0000 7f4b99839700 1 --2- 172.21.15.121:0/1554380559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b940c5c70 0x7f4b94158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.426+0000 7f4b99839700 1 --2- 172.21.15.121:0/1554380559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b940acd00 0x7f4b94158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.426+0000 7f4b99839700 1 --2- 172.21.15.121:0/1554380559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b94066330 0x7f4b94150830 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.426+0000 7f4b99839700 1 -- 172.21.15.121:0/1554380559 >> 172.21.15.121:0/1554380559 conn(0x7f4b94063fc0 msgr2=0x7f4b94152750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:43.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.426+0000 7f4b99839700 1 -- 172.21.15.121:0/1554380559 shutdown_connections 2023-11-15T02:58:43.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.426+0000 7f4b99839700 1 -- 172.21.15.121:0/1554380559 wait complete. 2023-11-15T02:58:43.455 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:58:43.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.610+0000 7f2bd7f1b700 1 Processor -- start 2023-11-15T02:58:43.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.610+0000 7f2bd7f1b700 1 -- start start 2023-11-15T02:58:43.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.610+0000 7f2bd7f1b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2bd0153bb0 0x7f2bd00a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:43.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd7f1b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2bd00aa020 0x7f2bd0067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:43.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd7f1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bd0067f50 0x7f2bd01578a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:43.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd7f1b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd00aa6f0 con 0x7f2bd00aa020 2023-11-15T02:58:43.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd7f1b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd00c5900 con 0x7f2bd0153bb0 2023-11-15T02:58:43.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd7f1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd00cb5c0 con 0x7f2bd0067f50 2023-11-15T02:58:43.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd54b6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2bd00aa020 0x7f2bd0067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:43.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd54b6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2bd00aa020 0x7f2bd0067a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33094/0 (socket says 172.21.15.121:33094) 2023-11-15T02:58:43.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd54b6700 1 -- 172.21.15.121:0/2210841698 learned_addr learned my addr 172.21.15.121:0/2210841698 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:43.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd5cb7700 1 --2- 172.21.15.121:0/2210841698 >> v2:172.21.15.190:3300/0 conn(0x7f2bd0153bb0 0x7f2bd00a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:43.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd54b6700 1 -- 172.21.15.121:0/2210841698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bd0067f50 msgr2=0x7f2bd01578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:43.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd64b8700 1 --2- 172.21.15.121:0/2210841698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bd0067f50 0x7f2bd01578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:43.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd54b6700 1 --2- 172.21.15.121:0/2210841698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bd0067f50 0x7f2bd01578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd54b6700 1 -- 172.21.15.121:0/2210841698 >> v2:172.21.15.190:3300/0 conn(0x7f2bd0153bb0 msgr2=0x7f2bd00a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:43.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd54b6700 1 --2- 172.21.15.121:0/2210841698 >> v2:172.21.15.190:3300/0 conn(0x7f2bd0153bb0 0x7f2bd00a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd54b6700 1 -- 172.21.15.121:0/2210841698 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd00683e0 con 0x7f2bd00aa020 2023-11-15T02:58:43.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd5cb7700 1 --2- 172.21.15.121:0/2210841698 >> v2:172.21.15.190:3300/0 conn(0x7f2bd0153bb0 0x7f2bd00a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:58:43.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd64b8700 1 --2- 172.21.15.121:0/2210841698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bd0067f50 0x7f2bd01578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:58:43.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd54b6700 1 --2- 172.21.15.121:0/2210841698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bd00aa020 0x7f2bd0067a60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f2bd0069f00 tx=0x7f2bc00090f0).ready entity=mon.0 client_cookie=9ed96592a9d403fa server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:43.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd4cb5700 1 -- 172.21.15.121:0/2210841698 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc000b600 con 0x7f2bd00aa020 2023-11-15T02:58:43.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd4cb5700 1 -- 172.21.15.121:0/2210841698 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc000b760 con 0x7f2bd00aa020 2023-11-15T02:58:43.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd4cb5700 1 -- 172.21.15.121:0/2210841698 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc000bae0 con 0x7f2bd00aa020 2023-11-15T02:58:43.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd7f1b700 1 -- 172.21.15.121:0/2210841698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bd00aa020 msgr2=0x7f2bd0067a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:43.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd7f1b700 1 --2- 172.21.15.121:0/2210841698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bd00aa020 0x7f2bd0067a60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f2bd0069f00 tx=0x7f2bc00090f0).stop 2023-11-15T02:58:43.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd7f1b700 1 -- 172.21.15.121:0/2210841698 shutdown_connections 2023-11-15T02:58:43.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd7f1b700 1 --2- 172.21.15.121:0/2210841698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bd0067f50 0x7f2bd01578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd7f1b700 1 --2- 172.21.15.121:0/2210841698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bd00aa020 0x7f2bd0067a60 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd7f1b700 1 --2- 172.21.15.121:0/2210841698 >> v2:172.21.15.190:3300/0 conn(0x7f2bd0153bb0 0x7f2bd00a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd7f1b700 1 -- 172.21.15.121:0/2210841698 >> 172.21.15.121:0/2210841698 conn(0x7f2bd0058c50 msgr2=0x7f2bd0059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:43.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd7f1b700 1 -- 172.21.15.121:0/2210841698 shutdown_connections 2023-11-15T02:58:43.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.614+0000 7f2bd7f1b700 1 -- 172.21.15.121:0/2210841698 wait complete. 2023-11-15T02:58:43.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.618+0000 7f2bd7f1b700 1 Processor -- start 2023-11-15T02:58:43.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.618+0000 7f2bd7f1b700 1 -- start start 2023-11-15T02:58:43.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.618+0000 7f2bd7f1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bd0067f50 0x7f2bd00ac010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:43.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.618+0000 7f2bd7f1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bd00aa020 0x7f2bd014d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:43.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.618+0000 7f2bd7f1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bd0153bb0 0x7f2bd00affd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:43.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.618+0000 7f2bd7f1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd0150c70 con 0x7f2bd0153bb0 2023-11-15T02:58:43.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.618+0000 7f2bd7f1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd00ab500 con 0x7f2bd0067f50 2023-11-15T02:58:43.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.618+0000 7f2bd7f1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd00ab7e0 con 0x7f2bd00aa020 2023-11-15T02:58:43.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.618+0000 7f2bd64b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bd0153bb0 0x7f2bd00affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:43.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.618+0000 7f2bd64b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bd0153bb0 0x7f2bd00affd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33106/0 (socket says 172.21.15.121:33106) 2023-11-15T02:58:43.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.618+0000 7f2bd64b8700 1 -- 172.21.15.121:0/2111945212 learned_addr learned my addr 172.21.15.121:0/2111945212 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:43.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.622+0000 7f2bd5cb7700 1 --2- 172.21.15.121:0/2111945212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bd0067f50 0x7f2bd00ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:43.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.622+0000 7f2bd54b6700 1 --2- 172.21.15.121:0/2111945212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bd00aa020 0x7f2bd014d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:43.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.622+0000 7f2bd64b8700 1 -- 172.21.15.121:0/2111945212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bd00aa020 msgr2=0x7f2bd014d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:43.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.622+0000 7f2bd64b8700 1 --2- 172.21.15.121:0/2111945212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bd00aa020 0x7f2bd014d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.622+0000 7f2bd64b8700 1 -- 172.21.15.121:0/2111945212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bd0067f50 msgr2=0x7f2bd00ac010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:43.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.622+0000 7f2bd64b8700 1 --2- 172.21.15.121:0/2111945212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bd0067f50 0x7f2bd00ac010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.622+0000 7f2bd64b8700 1 -- 172.21.15.121:0/2111945212 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc0009370 con 0x7f2bd0153bb0 2023-11-15T02:58:43.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.622+0000 7f2bd64b8700 1 --2- 172.21.15.121:0/2111945212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bd0153bb0 0x7f2bd00affd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f2bc800a380 tx=0x7f2bc8007900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:43.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.622+0000 7f2bc5ffb700 1 -- 172.21.15.121:0/2111945212 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800ae10 con 0x7f2bd0153bb0 2023-11-15T02:58:43.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.622+0000 7f2bc5ffb700 1 -- 172.21.15.121:0/2111945212 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc800d650 con 0x7f2bd0153bb0 2023-11-15T02:58:43.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.622+0000 7f2bc5ffb700 1 -- 172.21.15.121:0/2111945212 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800d9d0 con 0x7f2bd0153bb0 2023-11-15T02:58:43.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.622+0000 7f2bd7f1b700 1 -- 172.21.15.121:0/2111945212 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd00abac0 con 0x7f2bd0153bb0 2023-11-15T02:58:43.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.622+0000 7f2bd7f1b700 1 -- 172.21.15.121:0/2111945212 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd014db60 con 0x7f2bd0153bb0 2023-11-15T02:58:43.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.622+0000 7f2bd7f1b700 1 -- 172.21.15.121:0/2111945212 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd0153460 con 0x7f2bd0153bb0 2023-11-15T02:58:43.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.626+0000 7f2bc5ffb700 1 -- 172.21.15.121:0/2111945212 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2bc8013070 con 0x7f2bd0153bb0 2023-11-15T02:58:43.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.626+0000 7f2bc5ffb700 1 --2- 172.21.15.121:0/2111945212 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2bb405a510 0x7f2bb405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:43.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.626+0000 7f2bc5ffb700 1 -- 172.21.15.121:0/2111945212 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f2bc8075420 con 0x7f2bd0153bb0 2023-11-15T02:58:43.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.626+0000 7f2bd5cb7700 1 --2- 172.21.15.121:0/2111945212 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2bb405a510 0x7f2bb405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:43.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.626+0000 7f2bd5cb7700 1 --2- 172.21.15.121:0/2111945212 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2bb405a510 0x7f2bb405c9d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f2bcc00ab40 tx=0x7f2bcc008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:43.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.634+0000 7f2bc5ffb700 1 -- 172.21.15.121:0/2111945212 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2bc8048ad0 con 0x7f2bd0153bb0 2023-11-15T02:58:43.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.826+0000 7f2bd7f1b700 1 -- 172.21.15.121:0/2111945212 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bd0150760 con 0x7f2bd0153bb0 2023-11-15T02:58:43.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.826+0000 7f2bc5ffb700 1 -- 172.21.15.121:0/2111945212 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4502 (secure 0 0 0) 0x7f2bd0150760 con 0x7f2bd0153bb0 2023-11-15T02:58:43.835 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:58:43.835 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2544127447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2544127447},{"type":"v1","addr":"172.21.15.190:6833","nonce":2544127447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1815834193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1815834193},{"type":"v1","addr":"172.21.15.190:6835","nonce":1815834193}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4950,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3229917854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3229917854},{"type":"v1","addr":"172.21.15.121:6835","nonce":3229917854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:57:44.541243+0000","modified":"2023-11-15T02:58:10.996523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5178":{"gid":5178,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6837/3305580606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3305580606},{"type":"v1","addr":"172.21.15.121:6837","nonce":3305580606}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-15T02:58:43.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.830+0000 7f2bd7f1b700 1 -- 172.21.15.121:0/2111945212 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2bb405a510 msgr2=0x7f2bb405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:43.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.830+0000 7f2bd7f1b700 1 --2- 172.21.15.121:0/2111945212 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2bb405a510 0x7f2bb405c9d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f2bcc00ab40 tx=0x7f2bcc008040).stop 2023-11-15T02:58:43.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.830+0000 7f2bd7f1b700 1 -- 172.21.15.121:0/2111945212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bd0153bb0 msgr2=0x7f2bd00affd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:43.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.830+0000 7f2bd7f1b700 1 --2- 172.21.15.121:0/2111945212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bd0153bb0 0x7f2bd00affd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f2bc800a380 tx=0x7f2bc8007900).stop 2023-11-15T02:58:43.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.830+0000 7f2bd7f1b700 1 -- 172.21.15.121:0/2111945212 shutdown_connections 2023-11-15T02:58:43.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.830+0000 7f2bd7f1b700 1 --2- 172.21.15.121:0/2111945212 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2bb405a510 0x7f2bb405c9d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.830+0000 7f2bd7f1b700 1 --2- 172.21.15.121:0/2111945212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bd0153bb0 0x7f2bd00affd0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.830+0000 7f2bd7f1b700 1 --2- 172.21.15.121:0/2111945212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bd00aa020 0x7f2bd014d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.830+0000 7f2bd7f1b700 1 --2- 172.21.15.121:0/2111945212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bd0067f50 0x7f2bd00ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:43.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.830+0000 7f2bd7f1b700 1 -- 172.21.15.121:0/2111945212 >> 172.21.15.121:0/2111945212 conn(0x7f2bd0058c50 msgr2=0x7f2bd00c5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:43.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.830+0000 7f2bd7f1b700 1 -- 172.21.15.121:0/2111945212 shutdown_connections 2023-11-15T02:58:43.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:43.830+0000 7f2bd7f1b700 1 -- 172.21.15.121:0/2111945212 wait complete. 2023-11-15T02:58:43.840 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 23 2023-11-15T02:58:43.857 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:58:44.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.018+0000 7f7fc359e700 1 Processor -- start 2023-11-15T02:58:44.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.018+0000 7f7fc359e700 1 -- start start 2023-11-15T02:58:44.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.018+0000 7f7fc359e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7fc40cabd0 0x7f7fc40cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:44.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.018+0000 7f7fc359e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc40ad650 con 0x7f7fc40ad0f0 2023-11-15T02:58:44.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.018+0000 7f7fc359e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc4065fc0 con 0x7f7fc40acc90 2023-11-15T02:58:44.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.018+0000 7f7fc359e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc40662a0 con 0x7f7fc40cabd0 2023-11-15T02:58:44.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.022+0000 7f7fc2d9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f7fc40ad0f0 0x7f7fc40ae190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39896/0 (socket says 172.21.15.121:39896) 2023-11-15T02:58:44.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.022+0000 7f7fc2d9d700 1 -- 172.21.15.121:0/501322104 learned_addr learned my addr 172.21.15.121:0/501322104 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:44.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.022+0000 7f7fc259c700 1 --2- 172.21.15.121:0/501322104 >> v2:172.21.15.190:3300/0 conn(0x7f7fc40cabd0 0x7f7fc40cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:44.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.022+0000 7f7fc1d9b700 1 -- 172.21.15.121:0/501322104 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 374537768 0 0) 0x7f7fc40ad650 con 0x7f7fc40ad0f0 2023-11-15T02:58:44.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.022+0000 7f7fc1d9b700 1 -- 172.21.15.121:0/501322104 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fb00033f0 con 0x7f7fc40ad0f0 2023-11-15T02:58:44.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.022+0000 7f7fc1d9b700 1 -- 172.21.15.121:0/501322104 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1869632287 0 0) 0x7f7fc4065fc0 con 0x7f7fc40acc90 2023-11-15T02:58:44.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.022+0000 7f7fc1d9b700 1 -- 172.21.15.121:0/501322104 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fc40ad650 con 0x7f7fc40acc90 2023-11-15T02:58:44.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.022+0000 7f7fc259c700 1 -- 172.21.15.121:0/501322104 >> v1:172.21.15.190:6789/0 conn(0x7f7fc40acc90 legacy=0x7f7fc4066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:44.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.022+0000 7f7fc259c700 1 -- 172.21.15.121:0/501322104 >> v1:172.21.15.121:6789/0 conn(0x7f7fc40ad0f0 legacy=0x7f7fc40ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:44.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.022+0000 7f7fc259c700 1 -- 172.21.15.121:0/501322104 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc4066580 con 0x7f7fc40cabd0 2023-11-15T02:58:44.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.022+0000 7f7fc259c700 1 --2- 172.21.15.121:0/501322104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fc40cabd0 0x7f7fc40cafb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f7fb4009760 tx=0x7f7fb4009060).ready entity=mon.1 client_cookie=aa1a1d5a089445b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:44.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.022+0000 7f7fc1d9b700 1 -- 172.21.15.121:0/501322104 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb400d4a0 con 0x7f7fc40cabd0 2023-11-15T02:58:44.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.022+0000 7f7fc1d9b700 1 -- 172.21.15.121:0/501322104 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb400d600 con 0x7f7fc40cabd0 2023-11-15T02:58:44.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.022+0000 7f7fc1d9b700 1 -- 172.21.15.121:0/501322104 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb400d980 con 0x7f7fc40cabd0 2023-11-15T02:58:44.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.022+0000 7f7fc359e700 1 -- 172.21.15.121:0/501322104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fc40cabd0 msgr2=0x7f7fc40cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:44.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.022+0000 7f7fc359e700 1 --2- 172.21.15.121:0/501322104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fc40cabd0 0x7f7fc40cafb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f7fb4009760 tx=0x7f7fb4009060).stop 2023-11-15T02:58:44.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.022+0000 7f7fc359e700 1 -- 172.21.15.121:0/501322104 shutdown_connections 2023-11-15T02:58:44.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.022+0000 7f7fc359e700 1 --2- 172.21.15.121:0/501322104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fc40cabd0 0x7f7fc40cafb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.026+0000 7f7fc359e700 1 -- 172.21.15.121:0/501322104 >> 172.21.15.121:0/501322104 conn(0x7f7fc4062190 msgr2=0x7f7fc40625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:44.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.026+0000 7f7fc359e700 1 -- 172.21.15.121:0/501322104 shutdown_connections 2023-11-15T02:58:44.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.026+0000 7f7fc359e700 1 -- 172.21.15.121:0/501322104 wait complete. 2023-11-15T02:58:44.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.026+0000 7f7fc359e700 1 Processor -- start 2023-11-15T02:58:44.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.026+0000 7f7fc359e700 1 -- start start 2023-11-15T02:58:44.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.026+0000 7f7fc359e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7fc40acc90 0x7f7fc4164bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:44.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.026+0000 7f7fc359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fc40ad0f0 0x7f7fc413f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:44.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.026+0000 7f7fc359e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fc40cabd0 0x7f7fc40af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:44.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.026+0000 7f7fc359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc41571f0 con 0x7f7fc40ad0f0 2023-11-15T02:58:44.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.026+0000 7f7fc359e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc4165d00 con 0x7f7fc40cabd0 2023-11-15T02:58:44.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.026+0000 7f7fc359e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc414ec50 con 0x7f7fc40acc90 2023-11-15T02:58:44.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.026+0000 7f7fb9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fc40ad0f0 0x7f7fc413f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:44.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.030+0000 7f7fb9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fc40ad0f0 0x7f7fc413f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33120/0 (socket says 172.21.15.121:33120) 2023-11-15T02:58:44.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.030+0000 7f7fb9d9b700 1 -- 172.21.15.121:0/249030923 learned_addr learned my addr 172.21.15.121:0/249030923 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:44.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.030+0000 7f7fc2d9d700 1 --2- 172.21.15.121:0/249030923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fc40cabd0 0x7f7fc40af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:44.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.030+0000 7f7fc259c700 1 --2- 172.21.15.121:0/249030923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7fc40acc90 0x7f7fc4164bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:44.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.030+0000 7f7fb9d9b700 1 -- 172.21.15.121:0/249030923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7fc40acc90 msgr2=0x7f7fc4164bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:44.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.030+0000 7f7fb9d9b700 1 --2- 172.21.15.121:0/249030923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7fc40acc90 0x7f7fc4164bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.030+0000 7f7fb9d9b700 1 -- 172.21.15.121:0/249030923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fc40cabd0 msgr2=0x7f7fc40af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:44.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.030+0000 7f7fb9d9b700 1 --2- 172.21.15.121:0/249030923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fc40cabd0 0x7f7fc40af2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.030+0000 7f7fb9d9b700 1 -- 172.21.15.121:0/249030923 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb40092a0 con 0x7f7fc40ad0f0 2023-11-15T02:58:44.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.030+0000 7f7fb9d9b700 1 --2- 172.21.15.121:0/249030923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fc40ad0f0 0x7f7fc413f440 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f7fac00c4c0 tx=0x7f7fac007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:44.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.030+0000 7f7fbaffd700 1 -- 172.21.15.121:0/249030923 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac00ce30 con 0x7f7fc40ad0f0 2023-11-15T02:58:44.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.030+0000 7f7fbaffd700 1 -- 172.21.15.121:0/249030923 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fac00f470 con 0x7f7fc40ad0f0 2023-11-15T02:58:44.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.030+0000 7f7fc359e700 1 -- 172.21.15.121:0/249030923 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc41653d0 con 0x7f7fc40ad0f0 2023-11-15T02:58:44.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.030+0000 7f7fbaffd700 1 -- 172.21.15.121:0/249030923 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac00f810 con 0x7f7fc40ad0f0 2023-11-15T02:58:44.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.030+0000 7f7fc359e700 1 -- 172.21.15.121:0/249030923 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc4165990 con 0x7f7fc40ad0f0 2023-11-15T02:58:44.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.030+0000 7f7fc359e700 1 -- 172.21.15.121:0/249030923 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc406bcf0 con 0x7f7fc40ad0f0 2023-11-15T02:58:44.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.034+0000 7f7fbaffd700 1 -- 172.21.15.121:0/249030923 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7fac00fbc0 con 0x7f7fc40ad0f0 2023-11-15T02:58:44.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.034+0000 7f7fbaffd700 1 --2- 172.21.15.121:0/249030923 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7fa405a540 0x7f7fa405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:44.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.034+0000 7f7fbaffd700 1 -- 172.21.15.121:0/249030923 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f7fac073cf0 con 0x7f7fc40ad0f0 2023-11-15T02:58:44.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.034+0000 7f7fc259c700 1 --2- 172.21.15.121:0/249030923 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7fa405a540 0x7f7fa405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:44.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.034+0000 7f7fc259c700 1 --2- 172.21.15.121:0/249030923 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7fa405a540 0x7f7fa405ca00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f7fb4006d60 tx=0x7f7fb4016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:44.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.042+0000 7f7fbaffd700 1 -- 172.21.15.121:0/249030923 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7fac078070 con 0x7f7fc40ad0f0 2023-11-15T02:58:44.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.210+0000 7f7fc359e700 1 -- 172.21.15.121:0/249030923 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fc4066860 con 0x7f7fc40ad0f0 2023-11-15T02:58:44.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.210+0000 7f7fbaffd700 1 -- 172.21.15.121:0/249030923 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+17156 (secure 0 0 0) 0x7f7fac047460 con 0x7f7fc40ad0f0 2023-11-15T02:58:44.218 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:58:44.219 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":36,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:58:39.889913+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:57:42.277674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-15T02:57:43.540361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.26","mappings":[{"from":5,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6837/3788932715":"2023-11-16T02:57:58.241271+0000","172.21.15.121:6836/3788932715":"2023-11-16T02:57:58.241271+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:58:44.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.214+0000 7f7fc359e700 1 -- 172.21.15.121:0/249030923 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7fa405a540 msgr2=0x7f7fa405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:44.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.214+0000 7f7fc359e700 1 --2- 172.21.15.121:0/249030923 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7fa405a540 0x7f7fa405ca00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f7fb4006d60 tx=0x7f7fb4016000).stop 2023-11-15T02:58:44.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.214+0000 7f7fc359e700 1 -- 172.21.15.121:0/249030923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fc40ad0f0 msgr2=0x7f7fc413f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:44.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.214+0000 7f7fc359e700 1 --2- 172.21.15.121:0/249030923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fc40ad0f0 0x7f7fc413f440 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f7fac00c4c0 tx=0x7f7fac007c90).stop 2023-11-15T02:58:44.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.214+0000 7f7fc359e700 1 -- 172.21.15.121:0/249030923 shutdown_connections 2023-11-15T02:58:44.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.214+0000 7f7fc359e700 1 --2- 172.21.15.121:0/249030923 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7fa405a540 0x7f7fa405ca00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.214+0000 7f7fc359e700 1 --2- 172.21.15.121:0/249030923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fc40cabd0 0x7f7fc40af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.214+0000 7f7fc359e700 1 --2- 172.21.15.121:0/249030923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fc40ad0f0 0x7f7fc413f440 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.214+0000 7f7fc359e700 1 --2- 172.21.15.121:0/249030923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7fc40acc90 0x7f7fc4164bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.214+0000 7f7fc359e700 1 -- 172.21.15.121:0/249030923 >> 172.21.15.121:0/249030923 conn(0x7f7fc4062190 msgr2=0x7f7fc4151240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:44.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.214+0000 7f7fc359e700 1 -- 172.21.15.121:0/249030923 shutdown_connections 2023-11-15T02:58:44.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.214+0000 7f7fc359e700 1 -- 172.21.15.121:0/249030923 wait complete. 2023-11-15T02:58:44.241 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-15T02:58:44.242 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:58:44.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.398+0000 7f0d8c546700 1 Processor -- start 2023-11-15T02:58:44.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.398+0000 7f0d8c546700 1 -- start start 2023-11-15T02:58:44.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.402+0000 7f0d8c546700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0d840c2e20 0x7f0d840bd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:44.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.402+0000 7f0d8c546700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d840c6a30 0x7f0d840bd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:44.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.402+0000 7f0d8c546700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d840c3ee0 con 0x7f0d840c4de0 2023-11-15T02:58:44.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.402+0000 7f0d8c546700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d84150f80 con 0x7f0d840c2e20 2023-11-15T02:58:44.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.402+0000 7f0d8c546700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d840bdc30 con 0x7f0d840c6a30 2023-11-15T02:58:44.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.402+0000 7f0d89ae1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0d840c2e20 0x7f0d840bd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:44.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.402+0000 7f0d8a2e2700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f0d840c4de0 0x7f0d840c2710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39902/0 (socket says 172.21.15.121:39902) 2023-11-15T02:58:44.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.402+0000 7f0d8a2e2700 1 -- 172.21.15.121:0/1455078831 learned_addr learned my addr 172.21.15.121:0/1455078831 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:44.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.406+0000 7f0d8aae3700 1 --2- 172.21.15.121:0/1455078831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d840c6a30 0x7f0d840bd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:44.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.406+0000 7f0d892e0700 1 -- 172.21.15.121:0/1455078831 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284644402 0 0) 0x7f0d840c3ee0 con 0x7f0d840c4de0 2023-11-15T02:58:44.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.406+0000 7f0d892e0700 1 -- 172.21.15.121:0/1455078831 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d700033f0 con 0x7f0d840c4de0 2023-11-15T02:58:44.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.406+0000 7f0d89ae1700 1 -- 172.21.15.121:0/1455078831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d840c6a30 msgr2=0x7f0d840bd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:44.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.406+0000 7f0d89ae1700 1 --2- 172.21.15.121:0/1455078831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d840c6a30 0x7f0d840bd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.406+0000 7f0d89ae1700 1 -- 172.21.15.121:0/1455078831 >> v1:172.21.15.121:6789/0 conn(0x7f0d840c4de0 legacy=0x7f0d840c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:44.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.406+0000 7f0d89ae1700 1 -- 172.21.15.121:0/1455078831 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d840c32d0 con 0x7f0d840c2e20 2023-11-15T02:58:44.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.406+0000 7f0d89ae1700 1 --2- 172.21.15.121:0/1455078831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d840c2e20 0x7f0d840bd250 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f0d74009760 tx=0x7f0d74009060).ready entity=mon.0 client_cookie=8191ba0306519e51 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:44.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.406+0000 7f0d892e0700 1 -- 172.21.15.121:0/1455078831 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7400d4a0 con 0x7f0d840c2e20 2023-11-15T02:58:44.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.406+0000 7f0d892e0700 1 -- 172.21.15.121:0/1455078831 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7400d600 con 0x7f0d840c2e20 2023-11-15T02:58:44.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.406+0000 7f0d892e0700 1 -- 172.21.15.121:0/1455078831 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7400d980 con 0x7f0d840c2e20 2023-11-15T02:58:44.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.406+0000 7f0d8c546700 1 -- 172.21.15.121:0/1455078831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d840c2e20 msgr2=0x7f0d840bd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:44.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.406+0000 7f0d8c546700 1 --2- 172.21.15.121:0/1455078831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d840c2e20 0x7f0d840bd250 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f0d74009760 tx=0x7f0d74009060).stop 2023-11-15T02:58:44.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.406+0000 7f0d8c546700 1 -- 172.21.15.121:0/1455078831 shutdown_connections 2023-11-15T02:58:44.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.406+0000 7f0d8c546700 1 --2- 172.21.15.121:0/1455078831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d840c6a30 0x7f0d840bd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.406+0000 7f0d8c546700 1 --2- 172.21.15.121:0/1455078831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d840c2e20 0x7f0d840bd250 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.406+0000 7f0d8c546700 1 -- 172.21.15.121:0/1455078831 >> 172.21.15.121:0/1455078831 conn(0x7f0d84066ee0 msgr2=0x7f0d84090ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:44.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.406+0000 7f0d8c546700 1 -- 172.21.15.121:0/1455078831 shutdown_connections 2023-11-15T02:58:44.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.406+0000 7f0d8c546700 1 -- 172.21.15.121:0/1455078831 wait complete. 2023-11-15T02:58:44.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.410+0000 7f0d8c546700 1 Processor -- start 2023-11-15T02:58:44.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.410+0000 7f0d8c546700 1 -- start start 2023-11-15T02:58:44.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.410+0000 7f0d8c546700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d840c2e20 0x7f0d8415abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:44.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.410+0000 7f0d8c546700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d840c4de0 0x7f0d840ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:44.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.410+0000 7f0d8c546700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d840c6a30 0x7f0d84093670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:44.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.410+0000 7f0d8c546700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d840ad310 con 0x7f0d840c6a30 2023-11-15T02:58:44.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.410+0000 7f0d8c546700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d840ab480 con 0x7f0d840c4de0 2023-11-15T02:58:44.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.410+0000 7f0d8c546700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d840aa870 con 0x7f0d840c2e20 2023-11-15T02:58:44.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.410+0000 7f0d8aae3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d840c6a30 0x7f0d84093670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:44.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.410+0000 7f0d8aae3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d840c6a30 0x7f0d84093670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33136/0 (socket says 172.21.15.121:33136) 2023-11-15T02:58:44.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.410+0000 7f0d8aae3700 1 -- 172.21.15.121:0/2493919971 learned_addr learned my addr 172.21.15.121:0/2493919971 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:44.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.410+0000 7f0d8a2e2700 1 --2- 172.21.15.121:0/2493919971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d840c2e20 0x7f0d8415abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:44.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.410+0000 7f0d89ae1700 1 --2- 172.21.15.121:0/2493919971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d840c4de0 0x7f0d840ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:44.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.414+0000 7f0d8aae3700 1 -- 172.21.15.121:0/2493919971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d840c2e20 msgr2=0x7f0d8415abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:44.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.414+0000 7f0d8aae3700 1 --2- 172.21.15.121:0/2493919971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d840c2e20 0x7f0d8415abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.414+0000 7f0d8aae3700 1 -- 172.21.15.121:0/2493919971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d840c4de0 msgr2=0x7f0d840ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:44.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.414+0000 7f0d8aae3700 1 --2- 172.21.15.121:0/2493919971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d840c4de0 0x7f0d840ace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.414+0000 7f0d8aae3700 1 -- 172.21.15.121:0/2493919971 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d740092a0 con 0x7f0d840c6a30 2023-11-15T02:58:44.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.414+0000 7f0d8aae3700 1 --2- 172.21.15.121:0/2493919971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d840c6a30 0x7f0d84093670 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f0d7c009d70 tx=0x7f0d7c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:44.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.414+0000 7f0d7a7fc700 1 -- 172.21.15.121:0/2493919971 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c00cb50 con 0x7f0d840c6a30 2023-11-15T02:58:44.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.414+0000 7f0d7a7fc700 1 -- 172.21.15.121:0/2493919971 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c00f040 con 0x7f0d840c6a30 2023-11-15T02:58:44.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.414+0000 7f0d8c546700 1 -- 172.21.15.121:0/2493919971 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d840aab50 con 0x7f0d840c6a30 2023-11-15T02:58:44.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.414+0000 7f0d7a7fc700 1 -- 172.21.15.121:0/2493919971 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c013400 con 0x7f0d840c6a30 2023-11-15T02:58:44.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.414+0000 7f0d8c546700 1 -- 172.21.15.121:0/2493919971 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d840ab110 con 0x7f0d840c6a30 2023-11-15T02:58:44.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.414+0000 7f0d8c546700 1 -- 172.21.15.121:0/2493919971 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8406c150 con 0x7f0d840c6a30 2023-11-15T02:58:44.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.414+0000 7f0d7a7fc700 1 -- 172.21.15.121:0/2493919971 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0d7c013770 con 0x7f0d840c6a30 2023-11-15T02:58:44.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.418+0000 7f0d7a7fc700 1 --2- 172.21.15.121:0/2493919971 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0d6805a560 0x7f0d6805ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:44.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.418+0000 7f0d7a7fc700 1 -- 172.21.15.121:0/2493919971 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f0d7c074fb0 con 0x7f0d840c6a30 2023-11-15T02:58:44.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.418+0000 7f0d8a2e2700 1 --2- 172.21.15.121:0/2493919971 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0d6805a560 0x7f0d6805ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:44.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.418+0000 7f0d8a2e2700 1 --2- 172.21.15.121:0/2493919971 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0d6805a560 0x7f0d6805ca20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f0d80001360 tx=0x7f0d80009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:44.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.422+0000 7f0d7a7fc700 1 -- 172.21.15.121:0/2493919971 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0d7c0487c0 con 0x7f0d840c6a30 2023-11-15T02:58:44.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.618+0000 7f0d8c546700 1 -- 172.21.15.121:0/2493919971 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d840c35b0 con 0x7f0d840c6a30 2023-11-15T02:58:44.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.618+0000 7f0d7a7fc700 1 -- 172.21.15.121:0/2493919971 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4502 (secure 0 0 0) 0x7f0d7c04bde0 con 0x7f0d840c6a30 2023-11-15T02:58:44.625 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:58:44.625 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2544127447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2544127447},{"type":"v1","addr":"172.21.15.190:6833","nonce":2544127447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1815834193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1815834193},{"type":"v1","addr":"172.21.15.190:6835","nonce":1815834193}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4950,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3229917854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3229917854},{"type":"v1","addr":"172.21.15.121:6835","nonce":3229917854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:57:44.541243+0000","modified":"2023-11-15T02:58:10.996523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5178":{"gid":5178,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6837/3305580606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3305580606},{"type":"v1","addr":"172.21.15.121:6837","nonce":3305580606}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-15T02:58:44.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.622+0000 7f0d8c546700 1 -- 172.21.15.121:0/2493919971 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0d6805a560 msgr2=0x7f0d6805ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:44.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.622+0000 7f0d8c546700 1 --2- 172.21.15.121:0/2493919971 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0d6805a560 0x7f0d6805ca20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f0d80001360 tx=0x7f0d80009040).stop 2023-11-15T02:58:44.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.622+0000 7f0d8c546700 1 -- 172.21.15.121:0/2493919971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d840c6a30 msgr2=0x7f0d84093670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:44.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.622+0000 7f0d8c546700 1 --2- 172.21.15.121:0/2493919971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d840c6a30 0x7f0d84093670 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f0d7c009d70 tx=0x7f0d7c00b040).stop 2023-11-15T02:58:44.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.622+0000 7f0d8c546700 1 -- 172.21.15.121:0/2493919971 shutdown_connections 2023-11-15T02:58:44.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.622+0000 7f0d8c546700 1 --2- 172.21.15.121:0/2493919971 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0d6805a560 0x7f0d6805ca20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.622+0000 7f0d8c546700 1 --2- 172.21.15.121:0/2493919971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d840c6a30 0x7f0d84093670 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.622+0000 7f0d8c546700 1 --2- 172.21.15.121:0/2493919971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d840c4de0 0x7f0d840ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.622+0000 7f0d8c546700 1 --2- 172.21.15.121:0/2493919971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d840c2e20 0x7f0d8415abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.622+0000 7f0d8c546700 1 -- 172.21.15.121:0/2493919971 >> 172.21.15.121:0/2493919971 conn(0x7f0d84066ee0 msgr2=0x7f0d84062d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:44.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.622+0000 7f0d8c546700 1 -- 172.21.15.121:0/2493919971 shutdown_connections 2023-11-15T02:58:44.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.622+0000 7f0d8c546700 1 -- 172.21.15.121:0/2493919971 wait complete. 2023-11-15T02:58:44.630 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 23 2023-11-15T02:58:44.646 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:58:44.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.802+0000 7fe38a907700 1 Processor -- start 2023-11-15T02:58:44.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.802+0000 7fe38a907700 1 -- start start 2023-11-15T02:58:44.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.802+0000 7fe38a907700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe384066320 0x7fe384154110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:44.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.802+0000 7fe38a907700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3840acf10 0x7fe384062190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:44.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.802+0000 7fe38a907700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3840c73e0 con 0x7fe384066700 2023-11-15T02:58:44.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.802+0000 7fe38a907700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3841546c0 con 0x7fe384066320 2023-11-15T02:58:44.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.802+0000 7fe38a907700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3841549a0 con 0x7fe3840acf10 2023-11-15T02:58:44.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe389905700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe384066320 0x7fe384154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:44.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe389905700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe384066320 0x7fe384154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38990/0 (socket says 172.21.15.121:38990) 2023-11-15T02:58:44.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe389905700 1 -- 172.21.15.121:0/167034371 learned_addr learned my addr 172.21.15.121:0/167034371 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:44.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe38a106700 1 --2- 172.21.15.121:0/167034371 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3840acf10 0x7fe384062190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:44.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe389905700 1 -- 172.21.15.121:0/167034371 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3840acf10 msgr2=0x7fe384062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:44.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe389905700 1 --2- 172.21.15.121:0/167034371 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3840acf10 0x7fe384062190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe389905700 1 -- 172.21.15.121:0/167034371 >> v1:172.21.15.190:6789/0 conn(0x7fe384066700 legacy=0x7fe3840656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:44.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe389905700 1 -- 172.21.15.121:0/167034371 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe384154c80 con 0x7fe384066320 2023-11-15T02:58:44.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe389905700 1 --2- 172.21.15.121:0/167034371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe384066320 0x7fe384154110 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fe380006fd0 tx=0x7fe3800090f0).ready entity=mon.0 client_cookie=b37254d9e07ada46 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:44.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe388903700 1 -- 172.21.15.121:0/167034371 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38000b580 con 0x7fe384066320 2023-11-15T02:58:44.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe388903700 1 -- 172.21.15.121:0/167034371 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38000b6e0 con 0x7fe384066320 2023-11-15T02:58:44.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe388903700 1 -- 172.21.15.121:0/167034371 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38000ba60 con 0x7fe384066320 2023-11-15T02:58:44.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe38a907700 1 -- 172.21.15.121:0/167034371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe384066320 msgr2=0x7fe384154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:44.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe38a907700 1 --2- 172.21.15.121:0/167034371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe384066320 0x7fe384154110 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fe380006fd0 tx=0x7fe3800090f0).stop 2023-11-15T02:58:44.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe38a907700 1 -- 172.21.15.121:0/167034371 shutdown_connections 2023-11-15T02:58:44.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe38a907700 1 --2- 172.21.15.121:0/167034371 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3840acf10 0x7fe384062190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe38a907700 1 --2- 172.21.15.121:0/167034371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe384066320 0x7fe384154110 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe38a907700 1 -- 172.21.15.121:0/167034371 >> 172.21.15.121:0/167034371 conn(0x7fe384067a60 msgr2=0x7fe384067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:44.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe38a907700 1 -- 172.21.15.121:0/167034371 shutdown_connections 2023-11-15T02:58:44.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.806+0000 7fe38a907700 1 -- 172.21.15.121:0/167034371 wait complete. 2023-11-15T02:58:44.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.810+0000 7fe38a907700 1 Processor -- start 2023-11-15T02:58:44.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.810+0000 7fe38a907700 1 -- start start 2023-11-15T02:58:44.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.810+0000 7fe38a907700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe384066320 0x7fe38414d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:44.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.810+0000 7fe38a907700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe384066700 0x7fe3840a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:44.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.810+0000 7fe38a907700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3840acf10 0x7fe3840a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:44.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.810+0000 7fe38a907700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3840a4130 con 0x7fe384066700 2023-11-15T02:58:44.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.810+0000 7fe38a907700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3840a3e00 con 0x7fe384066320 2023-11-15T02:58:44.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.810+0000 7fe38a907700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3840a6900 con 0x7fe3840acf10 2023-11-15T02:58:44.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.810+0000 7fe389104700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe384066700 0x7fe3840a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:44.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.810+0000 7fe389104700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe384066700 0x7fe3840a58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38998/0 (socket says 172.21.15.121:38998) 2023-11-15T02:58:44.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.810+0000 7fe389104700 1 -- 172.21.15.121:0/1920345433 learned_addr learned my addr 172.21.15.121:0/1920345433 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:44.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.810+0000 7fe38a106700 1 --2- 172.21.15.121:0/1920345433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3840acf10 0x7fe3840a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:44.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.810+0000 7fe389905700 1 --2- 172.21.15.121:0/1920345433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe384066320 0x7fe38414d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:44.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.814+0000 7fe389104700 1 -- 172.21.15.121:0/1920345433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3840acf10 msgr2=0x7fe3840a7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:44.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.814+0000 7fe389104700 1 --2- 172.21.15.121:0/1920345433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3840acf10 0x7fe3840a7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.814+0000 7fe389104700 1 -- 172.21.15.121:0/1920345433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe384066320 msgr2=0x7fe38414d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:44.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.814+0000 7fe389104700 1 --2- 172.21.15.121:0/1920345433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe384066320 0x7fe38414d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:44.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.814+0000 7fe389104700 1 -- 172.21.15.121:0/1920345433 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe380009370 con 0x7fe384066700 2023-11-15T02:58:44.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.814+0000 7fe38a106700 1 --2- 172.21.15.121:0/1920345433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3840acf10 0x7fe3840a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:44.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.814+0000 7fe389104700 1 --2- 172.21.15.121:0/1920345433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe384066700 0x7fe3840a58f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7fe37400c4c0 tx=0x7fe374007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:44.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.814+0000 7fe379ffb700 1 -- 172.21.15.121:0/1920345433 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374007020 con 0x7fe384066700 2023-11-15T02:58:44.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.814+0000 7fe379ffb700 1 -- 172.21.15.121:0/1920345433 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe374007180 con 0x7fe384066700 2023-11-15T02:58:44.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.814+0000 7fe379ffb700 1 -- 172.21.15.121:0/1920345433 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374007520 con 0x7fe384066700 2023-11-15T02:58:44.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.814+0000 7fe38a907700 1 -- 172.21.15.121:0/1920345433 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38414dc60 con 0x7fe384066700 2023-11-15T02:58:44.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.814+0000 7fe38a907700 1 -- 172.21.15.121:0/1920345433 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3840a6620 con 0x7fe384066700 2023-11-15T02:58:44.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.814+0000 7fe38a907700 1 -- 172.21.15.121:0/1920345433 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3840a3c60 con 0x7fe384066700 2023-11-15T02:58:44.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.814+0000 7fe379ffb700 1 -- 172.21.15.121:0/1920345433 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe3740068b0 con 0x7fe384066700 2023-11-15T02:58:44.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.818+0000 7fe379ffb700 1 --2- 172.21.15.121:0/1920345433 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe36805a510 0x7fe36805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:44.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.818+0000 7fe379ffb700 1 -- 172.21.15.121:0/1920345433 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6236+0+0 (secure 0 0 0) 0x7fe374073be0 con 0x7fe384066700 2023-11-15T02:58:44.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.818+0000 7fe389905700 1 --2- 172.21.15.121:0/1920345433 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe36805a510 0x7fe36805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:44.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.818+0000 7fe389905700 1 --2- 172.21.15.121:0/1920345433 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe36805a510 0x7fe36805c9d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fe380009fd0 tx=0x7fe380017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:44.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:44.822+0000 7fe379ffb700 1 -- 172.21.15.121:0/1920345433 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe3840a3c60 con 0x7fe384066700 2023-11-15T02:58:45.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.022+0000 7fe38a907700 1 -- 172.21.15.121:0/1920345433 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3840a3c60 con 0x7fe384066700 2023-11-15T02:58:45.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.022+0000 7fe379ffb700 1 -- 172.21.15.121:0/1920345433 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4502 (secure 0 0 0) 0x7fe3840a3c60 con 0x7fe384066700 2023-11-15T02:58:45.030 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:58:45.030 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2544127447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2544127447},{"type":"v1","addr":"172.21.15.190:6833","nonce":2544127447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1815834193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1815834193},{"type":"v1","addr":"172.21.15.190:6835","nonce":1815834193}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4950,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3229917854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3229917854},{"type":"v1","addr":"172.21.15.121:6835","nonce":3229917854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:57:44.541243+0000","modified":"2023-11-15T02:58:10.996523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5178":{"gid":5178,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6837/3305580606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3305580606},{"type":"v1","addr":"172.21.15.121:6837","nonce":3305580606}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-15T02:58:45.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.030+0000 7fe38a907700 1 -- 172.21.15.121:0/1920345433 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe36805a510 msgr2=0x7fe36805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:45.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.030+0000 7fe38a907700 1 --2- 172.21.15.121:0/1920345433 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe36805a510 0x7fe36805c9d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fe380009fd0 tx=0x7fe380017000).stop 2023-11-15T02:58:45.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.030+0000 7fe38a907700 1 -- 172.21.15.121:0/1920345433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe384066700 msgr2=0x7fe3840a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:45.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.030+0000 7fe38a907700 1 --2- 172.21.15.121:0/1920345433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe384066700 0x7fe3840a58f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7fe37400c4c0 tx=0x7fe374007dd0).stop 2023-11-15T02:58:45.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.030+0000 7fe38a907700 1 -- 172.21.15.121:0/1920345433 shutdown_connections 2023-11-15T02:58:45.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.030+0000 7fe38a907700 1 --2- 172.21.15.121:0/1920345433 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe36805a510 0x7fe36805c9d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:45.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.030+0000 7fe38a907700 1 --2- 172.21.15.121:0/1920345433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3840acf10 0x7fe3840a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:45.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.030+0000 7fe38a907700 1 --2- 172.21.15.121:0/1920345433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe384066700 0x7fe3840a58f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:45.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.030+0000 7fe38a907700 1 --2- 172.21.15.121:0/1920345433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe384066320 0x7fe38414d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:45.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.030+0000 7fe38a907700 1 -- 172.21.15.121:0/1920345433 >> 172.21.15.121:0/1920345433 conn(0x7fe384067a60 msgr2=0x7fe384068050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:45.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.030+0000 7fe38a907700 1 -- 172.21.15.121:0/1920345433 shutdown_connections 2023-11-15T02:58:45.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.030+0000 7fe38a907700 1 -- 172.21.15.121:0/1920345433 wait complete. 2023-11-15T02:58:45.038 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 23 2023-11-15T02:58:45.059 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:58:45.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.210+0000 7fb3d61a9700 1 Processor -- start 2023-11-15T02:58:45.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.210+0000 7fb3d61a9700 1 -- start start 2023-11-15T02:58:45.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.210+0000 7fb3d61a9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb3d0154ca0 0x7fb3d0155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:45.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.210+0000 7fb3d61a9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb3d01562f0 0x7fb3d00a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:45.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.210+0000 7fb3d61a9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d00c4c30 con 0x7fb3d00a8f40 2023-11-15T02:58:45.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.210+0000 7fb3d61a9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d00a8280 con 0x7fb3d01562f0 2023-11-15T02:58:45.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.210+0000 7fb3d61a9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d00a6d40 con 0x7fb3d0154ca0 2023-11-15T02:58:45.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3ceffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb3d01562f0 0x7fb3d00a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:45.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3cffff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb3d00a8f40 0x7fb3d00a57c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:48948/0 (socket says 172.21.15.121:48948) 2023-11-15T02:58:45.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3cffff700 1 -- 172.21.15.121:0/2790287696 learned_addr learned my addr 172.21.15.121:0/2790287696 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:45.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3cf7fe700 1 --2- 172.21.15.121:0/2790287696 >> v2:172.21.15.190:3300/0 conn(0x7fb3d0154ca0 0x7fb3d0155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:45.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3ce7fc700 1 -- 172.21.15.121:0/2790287696 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1040223582 0 0) 0x7fb3d00c4c30 con 0x7fb3d00a8f40 2023-11-15T02:58:45.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3ce7fc700 1 -- 172.21.15.121:0/2790287696 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3b80033f0 con 0x7fb3d00a8f40 2023-11-15T02:58:45.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3ceffd700 1 -- 172.21.15.121:0/2790287696 >> v2:172.21.15.190:3300/0 conn(0x7fb3d0154ca0 msgr2=0x7fb3d0155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:45.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3ceffd700 1 --2- 172.21.15.121:0/2790287696 >> v2:172.21.15.190:3300/0 conn(0x7fb3d0154ca0 0x7fb3d0155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:45.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3ceffd700 1 -- 172.21.15.121:0/2790287696 >> v1:172.21.15.121:6789/0 conn(0x7fb3d00a8f40 legacy=0x7fb3d00a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:45.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3ceffd700 1 -- 172.21.15.121:0/2790287696 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d00a6f50 con 0x7fb3d01562f0 2023-11-15T02:58:45.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3cf7fe700 1 --2- 172.21.15.121:0/2790287696 >> v2:172.21.15.190:3300/0 conn(0x7fb3d0154ca0 0x7fb3d0155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:45.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3ceffd700 1 --2- 172.21.15.121:0/2790287696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3d01562f0 0x7fb3d00a8a50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fb3d00a6e80 tx=0x7fb3c4009060).ready entity=mon.0 client_cookie=e7a9c436d680684 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:45.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3ce7fc700 1 -- 172.21.15.121:0/2790287696 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c400f510 con 0x7fb3d01562f0 2023-11-15T02:58:45.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3ce7fc700 1 -- 172.21.15.121:0/2790287696 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c400f670 con 0x7fb3d01562f0 2023-11-15T02:58:45.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3ce7fc700 1 -- 172.21.15.121:0/2790287696 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c400f9f0 con 0x7fb3d01562f0 2023-11-15T02:58:45.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3d61a9700 1 -- 172.21.15.121:0/2790287696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3d01562f0 msgr2=0x7fb3d00a8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:45.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3d61a9700 1 --2- 172.21.15.121:0/2790287696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3d01562f0 0x7fb3d00a8a50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fb3d00a6e80 tx=0x7fb3c4009060).stop 2023-11-15T02:58:45.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3d61a9700 1 -- 172.21.15.121:0/2790287696 shutdown_connections 2023-11-15T02:58:45.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3d61a9700 1 --2- 172.21.15.121:0/2790287696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3d01562f0 0x7fb3d00a8a50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:45.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3d61a9700 1 --2- 172.21.15.121:0/2790287696 >> v2:172.21.15.190:3300/0 conn(0x7fb3d0154ca0 0x7fb3d0155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:45.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3d61a9700 1 -- 172.21.15.121:0/2790287696 >> 172.21.15.121:0/2790287696 conn(0x7fb3d0062190 msgr2=0x7fb3d00625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:45.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3d61a9700 1 -- 172.21.15.121:0/2790287696 shutdown_connections 2023-11-15T02:58:45.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.214+0000 7fb3d61a9700 1 -- 172.21.15.121:0/2790287696 wait complete. 2023-11-15T02:58:45.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.218+0000 7fb3d61a9700 1 Processor -- start 2023-11-15T02:58:45.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.218+0000 7fb3d61a9700 1 -- start start 2023-11-15T02:58:45.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3d61a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d00a8f40 0x7fb3d014d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:45.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3d61a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3d0154ca0 0x7fb3d01517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:45.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3d61a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d01562f0 0x7fb3d0151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:45.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3d61a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d01543d0 con 0x7fb3d0154ca0 2023-11-15T02:58:45.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3d61a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0151010 con 0x7fb3d01562f0 2023-11-15T02:58:45.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3d61a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d01537c0 con 0x7fb3d00a8f40 2023-11-15T02:58:45.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3ceffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3d0154ca0 0x7fb3d01517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:45.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3ceffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3d0154ca0 0x7fb3d01517c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39024/0 (socket says 172.21.15.121:39024) 2023-11-15T02:58:45.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3ceffd700 1 -- 172.21.15.121:0/3269859637 learned_addr learned my addr 172.21.15.121:0/3269859637 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:45.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3cf7fe700 1 --2- 172.21.15.121:0/3269859637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d00a8f40 0x7fb3d014d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:45.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3cffff700 1 --2- 172.21.15.121:0/3269859637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d01562f0 0x7fb3d0151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:45.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3ceffd700 1 -- 172.21.15.121:0/3269859637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d00a8f40 msgr2=0x7fb3d014d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:45.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3ceffd700 1 --2- 172.21.15.121:0/3269859637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d00a8f40 0x7fb3d014d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:45.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3ceffd700 1 -- 172.21.15.121:0/3269859637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d01562f0 msgr2=0x7fb3d0151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:45.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3ceffd700 1 --2- 172.21.15.121:0/3269859637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d01562f0 0x7fb3d0151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:45.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3ceffd700 1 -- 172.21.15.121:0/3269859637 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c40092a0 con 0x7fb3d0154ca0 2023-11-15T02:58:45.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3ceffd700 1 --2- 172.21.15.121:0/3269859637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3d0154ca0 0x7fb3d01517c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7fb3d00c0d10 tx=0x7fb3c4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:45.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3b77fe700 1 -- 172.21.15.121:0/3269859637 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4002f50 con 0x7fb3d0154ca0 2023-11-15T02:58:45.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3b77fe700 1 -- 172.21.15.121:0/3269859637 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c400f7e0 con 0x7fb3d0154ca0 2023-11-15T02:58:45.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3b77fe700 1 -- 172.21.15.121:0/3269859637 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c401b410 con 0x7fb3d0154ca0 2023-11-15T02:58:45.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3d61a9700 1 -- 172.21.15.121:0/3269859637 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d0153aa0 con 0x7fb3d0154ca0 2023-11-15T02:58:45.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3d61a9700 1 -- 172.21.15.121:0/3269859637 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d0154060 con 0x7fb3d0154ca0 2023-11-15T02:58:45.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.222+0000 7fb3d61a9700 1 -- 172.21.15.121:0/3269859637 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d0154540 con 0x7fb3d0154ca0 2023-11-15T02:58:45.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.226+0000 7fb3b77fe700 1 -- 172.21.15.121:0/3269859637 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb3c401b7c0 con 0x7fb3d0154ca0 2023-11-15T02:58:45.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.226+0000 7fb3b77fe700 1 --2- 172.21.15.121:0/3269859637 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3bc05a540 0x7fb3bc05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:45.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.226+0000 7fb3b77fe700 1 -- 172.21.15.121:0/3269859637 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6236+0+0 (secure 0 0 0) 0x7fb3c4076090 con 0x7fb3d0154ca0 2023-11-15T02:58:45.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.226+0000 7fb3cf7fe700 1 --2- 172.21.15.121:0/3269859637 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3bc05a540 0x7fb3bc05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:45.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.226+0000 7fb3cf7fe700 1 --2- 172.21.15.121:0/3269859637 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3bc05a540 0x7fb3bc05ca00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fb3c0006fd0 tx=0x7fb3c0008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:45.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.234+0000 7fb3b77fe700 1 -- 172.21.15.121:0/3269859637 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb3c4049740 con 0x7fb3d0154ca0 2023-11-15T02:58:45.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.398+0000 7fb3d61a9700 1 -- 172.21.15.121:0/3269859637 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3d00a7230 con 0x7fb3d0154ca0 2023-11-15T02:58:45.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.398+0000 7fb3b77fe700 1 -- 172.21.15.121:0/3269859637 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+17156 (secure 0 0 0) 0x7fb3c404cd60 con 0x7fb3d0154ca0 2023-11-15T02:58:45.406 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:58:45.406 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":36,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:58:39.889913+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:57:42.277674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-15T02:57:43.540361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.26","mappings":[{"from":5,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6837/3788932715":"2023-11-16T02:57:58.241271+0000","172.21.15.121:6836/3788932715":"2023-11-16T02:57:58.241271+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:58:45.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.402+0000 7fb3d61a9700 1 -- 172.21.15.121:0/3269859637 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3bc05a540 msgr2=0x7fb3bc05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:45.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.402+0000 7fb3d61a9700 1 --2- 172.21.15.121:0/3269859637 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3bc05a540 0x7fb3bc05ca00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fb3c0006fd0 tx=0x7fb3c0008040).stop 2023-11-15T02:58:45.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.402+0000 7fb3d61a9700 1 -- 172.21.15.121:0/3269859637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3d0154ca0 msgr2=0x7fb3d01517c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:45.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.402+0000 7fb3d61a9700 1 --2- 172.21.15.121:0/3269859637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3d0154ca0 0x7fb3d01517c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7fb3d00c0d10 tx=0x7fb3c4017000).stop 2023-11-15T02:58:45.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.402+0000 7fb3d61a9700 1 -- 172.21.15.121:0/3269859637 shutdown_connections 2023-11-15T02:58:45.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.402+0000 7fb3d61a9700 1 --2- 172.21.15.121:0/3269859637 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3bc05a540 0x7fb3bc05ca00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:45.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.402+0000 7fb3d61a9700 1 --2- 172.21.15.121:0/3269859637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d01562f0 0x7fb3d0151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:45.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.402+0000 7fb3d61a9700 1 --2- 172.21.15.121:0/3269859637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3d0154ca0 0x7fb3d01517c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:45.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.402+0000 7fb3d61a9700 1 --2- 172.21.15.121:0/3269859637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d00a8f40 0x7fb3d014d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:45.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.402+0000 7fb3d61a9700 1 -- 172.21.15.121:0/3269859637 >> 172.21.15.121:0/3269859637 conn(0x7fb3d0062190 msgr2=0x7fb3d014e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:45.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.402+0000 7fb3d61a9700 1 -- 172.21.15.121:0/3269859637 shutdown_connections 2023-11-15T02:58:45.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.402+0000 7fb3d61a9700 1 -- 172.21.15.121:0/3269859637 wait complete. 2023-11-15T02:58:45.428 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T02:58:45.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.582+0000 7f0197f46700 1 Processor -- start 2023-11-15T02:58:45.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.582+0000 7f0197f46700 1 -- start start 2023-11-15T02:58:45.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.586+0000 7f0197f46700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0190153aa0 0x7f0190066680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:45.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.586+0000 7f0197f46700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01900c0490 0x7f01900c0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:45.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.586+0000 7f0197f46700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01900c5c50 con 0x7f0190153e80 2023-11-15T02:58:45.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.586+0000 7f0197f46700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0190066b70 con 0x7f0190153aa0 2023-11-15T02:58:45.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.586+0000 7f0197f46700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01900c0e30 con 0x7f01900c0490 2023-11-15T02:58:45.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.586+0000 7f0195ce2700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0190153aa0 0x7f0190066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:45.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.586+0000 7f0195ce2700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0190153aa0 0x7f0190066680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39032/0 (socket says 172.21.15.121:39032) 2023-11-15T02:58:45.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.586+0000 7f0195ce2700 1 -- 172.21.15.121:0/1135753103 learned_addr learned my addr 172.21.15.121:0/1135753103 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:45.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.586+0000 7f01964e3700 1 --2- 172.21.15.121:0/1135753103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01900c0490 0x7f01900c0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:45.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.586+0000 7f0195ce2700 1 -- 172.21.15.121:0/1135753103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01900c0490 msgr2=0x7f01900c0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:45.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.586+0000 7f0195ce2700 1 --2- 172.21.15.121:0/1135753103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01900c0490 0x7f01900c0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:45.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.586+0000 7f0195ce2700 1 -- 172.21.15.121:0/1135753103 >> v1:172.21.15.190:6789/0 conn(0x7f0190153e80 legacy=0x7f01900ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:45.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.586+0000 7f0195ce2700 1 -- 172.21.15.121:0/1135753103 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01900c5920 con 0x7f0190153aa0 2023-11-15T02:58:45.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.586+0000 7f0195ce2700 1 --2- 172.21.15.121:0/1135753103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0190153aa0 0x7f0190066680 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f018c006fd0 tx=0x7f018c0090f0).ready entity=mon.0 client_cookie=ddcf1ce019a8ae68 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:45.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.586+0000 7f0194ce0700 1 -- 172.21.15.121:0/1135753103 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c00b580 con 0x7f0190153aa0 2023-11-15T02:58:45.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.586+0000 7f0194ce0700 1 -- 172.21.15.121:0/1135753103 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018c00b6e0 con 0x7f0190153aa0 2023-11-15T02:58:45.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.586+0000 7f0194ce0700 1 -- 172.21.15.121:0/1135753103 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c00ba60 con 0x7f0190153aa0 2023-11-15T02:58:45.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.590+0000 7f0197f46700 1 -- 172.21.15.121:0/1135753103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0190153aa0 msgr2=0x7f0190066680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:45.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.590+0000 7f0197f46700 1 --2- 172.21.15.121:0/1135753103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0190153aa0 0x7f0190066680 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f018c006fd0 tx=0x7f018c0090f0).stop 2023-11-15T02:58:45.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.590+0000 7f0197f46700 1 -- 172.21.15.121:0/1135753103 shutdown_connections 2023-11-15T02:58:45.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.590+0000 7f0197f46700 1 --2- 172.21.15.121:0/1135753103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01900c0490 0x7f01900c0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:45.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.590+0000 7f0197f46700 1 --2- 172.21.15.121:0/1135753103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0190153aa0 0x7f0190066680 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:45.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.590+0000 7f0197f46700 1 -- 172.21.15.121:0/1135753103 >> 172.21.15.121:0/1135753103 conn(0x7f0190067a60 msgr2=0x7f0190067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:45.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.590+0000 7f0197f46700 1 -- 172.21.15.121:0/1135753103 shutdown_connections 2023-11-15T02:58:45.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.590+0000 7f0197f46700 1 -- 172.21.15.121:0/1135753103 wait complete. 2023-11-15T02:58:45.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.590+0000 7f0197f46700 1 Processor -- start 2023-11-15T02:58:45.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.590+0000 7f0197f46700 1 -- start start 2023-11-15T02:58:45.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0197f46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01900c0490 0x7f0190164cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:45.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0197f46700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0190153aa0 0x7f01900c2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:45.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0197f46700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0190153e80 0x7f01901461e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:45.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0197f46700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01900c4360 con 0x7f01900c0490 2023-11-15T02:58:45.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0197f46700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01900c37e0 con 0x7f0190153aa0 2023-11-15T02:58:45.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0197f46700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01900c3ac0 con 0x7f0190153e80 2023-11-15T02:58:45.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0195ce2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01900c0490 0x7f0190164cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:45.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0195ce2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01900c0490 0x7f0190164cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39038/0 (socket says 172.21.15.121:39038) 2023-11-15T02:58:45.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0195ce2700 1 -- 172.21.15.121:0/2130827615 learned_addr learned my addr 172.21.15.121:0/2130827615 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:45.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0195ce2700 1 -- 172.21.15.121:0/2130827615 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0190153e80 msgr2=0x7f01901461e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:45.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0195ce2700 1 --2- 172.21.15.121:0/2130827615 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0190153e80 0x7f01901461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T02:58:45.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0195ce2700 1 -- 172.21.15.121:0/2130827615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0190153aa0 msgr2=0x7f01900c2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:45.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0195ce2700 1 --2- 172.21.15.121:0/2130827615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0190153aa0 0x7f01900c2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T02:58:45.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0195ce2700 1 -- 172.21.15.121:0/2130827615 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018c009370 con 0x7f01900c0490 2023-11-15T02:58:45.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0195ce2700 1 --2- 172.21.15.121:0/2130827615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01900c0490 0x7f0190164cb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f018c009130 tx=0x7f018c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:45.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0185ffb700 1 -- 172.21.15.121:0/2130827615 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c00b850 con 0x7f01900c0490 2023-11-15T02:58:45.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0185ffb700 1 -- 172.21.15.121:0/2130827615 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018c00b9b0 con 0x7f01900c0490 2023-11-15T02:58:45.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0197f46700 1 -- 172.21.15.121:0/2130827615 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01900c3da0 con 0x7f01900c0490 2023-11-15T02:58:45.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0185ffb700 1 -- 172.21.15.121:0/2130827615 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c01c670 con 0x7f01900c0490 2023-11-15T02:58:45.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.594+0000 7f0197f46700 1 -- 172.21.15.121:0/2130827615 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01900c3480 con 0x7f01900c0490 2023-11-15T02:58:45.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.598+0000 7f0197f46700 1 -- 172.21.15.121:0/2130827615 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01901543f0 con 0x7f01900c0490 2023-11-15T02:58:45.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.598+0000 7f0185ffb700 1 -- 172.21.15.121:0/2130827615 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f018c01ca60 con 0x7f01900c0490 2023-11-15T02:58:45.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.598+0000 7f0185ffb700 1 --2- 172.21.15.121:0/2130827615 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f017405a510 0x7f017405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:45.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.598+0000 7f0185ffb700 1 -- 172.21.15.121:0/2130827615 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f018c0764c0 con 0x7f01900c0490 2023-11-15T02:58:45.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.598+0000 7f01954e1700 1 --2- 172.21.15.121:0/2130827615 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f017405a510 0x7f017405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:45.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.602+0000 7f01954e1700 1 --2- 172.21.15.121:0/2130827615 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f017405a510 0x7f017405c9d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f01900c4a00 tx=0x7f01800079c0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:45.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.606+0000 7f0185ffb700 1 -- 172.21.15.121:0/2130827615 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f01901543f0 con 0x7f01900c0490 2023-11-15T02:58:45.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.798+0000 7f0197f46700 1 -- 172.21.15.121:0/2130827615 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f01901543f0 con 0x7f01900c0490 2023-11-15T02:58:45.948 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T02:58:45.942+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T02:58:45.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.954+0000 7f0185ffb700 1 -- 172.21.15.121:0/2130827615 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7f01901543f0 con 0x7f01900c0490 2023-11-15T02:58:45.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.958+0000 7f0197f46700 1 -- 172.21.15.121:0/2130827615 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f017405a510 msgr2=0x7f017405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:45.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.958+0000 7f0197f46700 1 --2- 172.21.15.121:0/2130827615 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f017405a510 0x7f017405c9d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f01900c4a00 tx=0x7f01800079c0).stop 2023-11-15T02:58:45.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.958+0000 7f0197f46700 1 -- 172.21.15.121:0/2130827615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01900c0490 msgr2=0x7f0190164cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:45.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.958+0000 7f0197f46700 1 --2- 172.21.15.121:0/2130827615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01900c0490 0x7f0190164cb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f018c009130 tx=0x7f018c017000).stop 2023-11-15T02:58:45.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.958+0000 7f0197f46700 1 -- 172.21.15.121:0/2130827615 shutdown_connections 2023-11-15T02:58:45.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.958+0000 7f0197f46700 1 --2- 172.21.15.121:0/2130827615 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f017405a510 0x7f017405c9d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:45.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.958+0000 7f0197f46700 1 --2- 172.21.15.121:0/2130827615 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0190153e80 0x7f01901461e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T02:58:45.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.958+0000 7f0197f46700 1 --2- 172.21.15.121:0/2130827615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0190153aa0 0x7f01900c2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T02:58:45.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.958+0000 7f0197f46700 1 --2- 172.21.15.121:0/2130827615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01900c0490 0x7f0190164cb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:45.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.958+0000 7f0197f46700 1 -- 172.21.15.121:0/2130827615 >> 172.21.15.121:0/2130827615 conn(0x7f0190067a60 msgr2=0x7f01900c6900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:45.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.958+0000 7f0197f46700 1 -- 172.21.15.121:0/2130827615 shutdown_connections 2023-11-15T02:58:45.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:45.958+0000 7f0197f46700 1 -- 172.21.15.121:0/2130827615 wait complete. 2023-11-15T02:58:45.967 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T02:58:45.972 INFO:tasks.ceph.mds.c.smithi121.stderr: -2796> 2023-11-15T02:58:05.039+0000 7f7e57a71700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T02:58:45.989 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-15T02:58:46.019 INFO:tasks.ceph.mds.c.smithi121.stdout:starting mds.c at 2023-11-15T02:58:46.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.138+0000 7f3c6b2db700 1 Processor -- start 2023-11-15T02:58:46.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.142+0000 7f3c6b2db700 1 -- start start 2023-11-15T02:58:46.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.142+0000 7f3c6b2db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3c640bc9b0 0x7f3c640670a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:46.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.142+0000 7f3c6b2db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c640bcd90 0x7f3c64153d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:46.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.142+0000 7f3c6b2db700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c640bd170 con 0x7f3c64062190 2023-11-15T02:58:46.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.142+0000 7f3c6b2db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c64154530 con 0x7f3c640bc9b0 2023-11-15T02:58:46.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.142+0000 7f3c6b2db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c640c1dd0 con 0x7f3c640bcd90 2023-11-15T02:58:46.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.142+0000 7f3c69077700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3c640bc9b0 0x7f3c640670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:46.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.142+0000 7f3c69878700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f3c64062190 0x7f3c640c2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:55338/0 (socket says 172.21.15.121:55338) 2023-11-15T02:58:46.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.142+0000 7f3c69878700 1 -- 172.21.15.121:0/1393952873 learned_addr learned my addr 172.21.15.121:0/1393952873 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:46.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.142+0000 7f3c68876700 1 --2- 172.21.15.121:0/1393952873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c640bcd90 0x7f3c64153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:46.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.146+0000 7f3c69077700 1 -- 172.21.15.121:0/1393952873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c640bcd90 msgr2=0x7f3c64153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:46.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.146+0000 7f3c69077700 1 --2- 172.21.15.121:0/1393952873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c640bcd90 0x7f3c64153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:46.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.146+0000 7f3c69077700 1 -- 172.21.15.121:0/1393952873 >> v1:172.21.15.190:6789/0 conn(0x7f3c64062190 legacy=0x7f3c640c2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:46.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.146+0000 7f3c69077700 1 -- 172.21.15.121:0/1393952873 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c641547d0 con 0x7f3c640bc9b0 2023-11-15T02:58:46.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.146+0000 7f3c68876700 1 --2- 172.21.15.121:0/1393952873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c640bcd90 0x7f3c64153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:58:46.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.146+0000 7f3c69077700 1 --2- 172.21.15.121:0/1393952873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c640bc9b0 0x7f3c640670a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f3c60009760 tx=0x7f3c60009060).ready entity=mon.1 client_cookie=c0dbcccab9582859 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:46.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.146+0000 7f3c5bfff700 1 -- 172.21.15.121:0/1393952873 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c600090d0 con 0x7f3c640bc9b0 2023-11-15T02:58:46.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.146+0000 7f3c5bfff700 1 -- 172.21.15.121:0/1393952873 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6000f510 con 0x7f3c640bc9b0 2023-11-15T02:58:46.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.146+0000 7f3c5bfff700 1 -- 172.21.15.121:0/1393952873 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6000f890 con 0x7f3c640bc9b0 2023-11-15T02:58:46.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.146+0000 7f3c6b2db700 1 -- 172.21.15.121:0/1393952873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c640bc9b0 msgr2=0x7f3c640670a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:46.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.146+0000 7f3c6b2db700 1 --2- 172.21.15.121:0/1393952873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c640bc9b0 0x7f3c640670a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f3c60009760 tx=0x7f3c60009060).stop 2023-11-15T02:58:46.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.146+0000 7f3c6b2db700 1 -- 172.21.15.121:0/1393952873 shutdown_connections 2023-11-15T02:58:46.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.146+0000 7f3c6b2db700 1 --2- 172.21.15.121:0/1393952873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c640bcd90 0x7f3c64153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:46.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.146+0000 7f3c6b2db700 1 --2- 172.21.15.121:0/1393952873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c640bc9b0 0x7f3c640670a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:46.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.146+0000 7f3c6b2db700 1 -- 172.21.15.121:0/1393952873 >> 172.21.15.121:0/1393952873 conn(0x7f3c640591c0 msgr2=0x7f3c640c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:46.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.146+0000 7f3c6b2db700 1 -- 172.21.15.121:0/1393952873 shutdown_connections 2023-11-15T02:58:46.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.146+0000 7f3c6b2db700 1 -- 172.21.15.121:0/1393952873 wait complete. 2023-11-15T02:58:46.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.150+0000 7f3c6b2db700 1 Processor -- start 2023-11-15T02:58:46.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.150+0000 7f3c6b2db700 1 -- start start 2023-11-15T02:58:46.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.150+0000 7f3c6b2db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c64062190 0x7f3c641412a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:46.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.150+0000 7f3c6b2db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c640bc9b0 0x7f3c641577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:46.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.150+0000 7f3c6b2db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c640bcd90 0x7f3c64165a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:46.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.150+0000 7f3c6b2db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c64164a70 con 0x7f3c640bc9b0 2023-11-15T02:58:46.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.150+0000 7f3c6b2db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c64141790 con 0x7f3c64062190 2023-11-15T02:58:46.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.150+0000 7f3c6b2db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c64158590 con 0x7f3c640bcd90 2023-11-15T02:58:46.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.150+0000 7f3c68876700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c640bc9b0 0x7f3c641577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:46.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.150+0000 7f3c68876700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c640bc9b0 0x7f3c641577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39076/0 (socket says 172.21.15.121:39076) 2023-11-15T02:58:46.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.150+0000 7f3c68876700 1 -- 172.21.15.121:0/3089412884 learned_addr learned my addr 172.21.15.121:0/3089412884 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:46.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.150+0000 7f3c69878700 1 --2- 172.21.15.121:0/3089412884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c640bcd90 0x7f3c64165a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:46.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.150+0000 7f3c69077700 1 --2- 172.21.15.121:0/3089412884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c64062190 0x7f3c641412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:46.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.150+0000 7f3c68876700 1 -- 172.21.15.121:0/3089412884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c640bcd90 msgr2=0x7f3c64165a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:46.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.150+0000 7f3c68876700 1 --2- 172.21.15.121:0/3089412884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c640bcd90 0x7f3c64165a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:46.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.150+0000 7f3c68876700 1 -- 172.21.15.121:0/3089412884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c64062190 msgr2=0x7f3c641412a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:46.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.150+0000 7f3c68876700 1 --2- 172.21.15.121:0/3089412884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c64062190 0x7f3c641412a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:46.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.150+0000 7f3c68876700 1 -- 172.21.15.121:0/3089412884 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5400a3c0 con 0x7f3c640bc9b0 2023-11-15T02:58:46.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.150+0000 7f3c69878700 1 --2- 172.21.15.121:0/3089412884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c640bcd90 0x7f3c64165a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:58:46.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.154+0000 7f3c68876700 1 --2- 172.21.15.121:0/3089412884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c640bc9b0 0x7f3c641577c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f3c5400f970 tx=0x7f3c54009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:46.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.154+0000 7f3c597fa700 1 -- 172.21.15.121:0/3089412884 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54005a90 con 0x7f3c640bc9b0 2023-11-15T02:58:46.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.154+0000 7f3c597fa700 1 -- 172.21.15.121:0/3089412884 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c54005bf0 con 0x7f3c640bc9b0 2023-11-15T02:58:46.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.154+0000 7f3c597fa700 1 -- 172.21.15.121:0/3089412884 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54007110 con 0x7f3c640bc9b0 2023-11-15T02:58:46.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.154+0000 7f3c6b2db700 1 -- 172.21.15.121:0/3089412884 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c600092a0 con 0x7f3c640bc9b0 2023-11-15T02:58:46.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.154+0000 7f3c6b2db700 1 -- 172.21.15.121:0/3089412884 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c64158b50 con 0x7f3c640bc9b0 2023-11-15T02:58:46.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.154+0000 7f3c6b2db700 1 -- 172.21.15.121:0/3089412884 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c64047060 con 0x7f3c640bc9b0 2023-11-15T02:58:46.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.154+0000 7f3c597fa700 1 -- 172.21.15.121:0/3089412884 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3c540074c0 con 0x7f3c640bc9b0 2023-11-15T02:58:46.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.158+0000 7f3c597fa700 1 --2- 172.21.15.121:0/3089412884 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3c4805a510 0x7f3c4805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:46.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.158+0000 7f3c597fa700 1 -- 172.21.15.121:0/3089412884 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f3c540743f0 con 0x7f3c640bc9b0 2023-11-15T02:58:46.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.158+0000 7f3c69077700 1 --2- 172.21.15.121:0/3089412884 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3c4805a510 0x7f3c4805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:46.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.158+0000 7f3c69077700 1 --2- 172.21.15.121:0/3089412884 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3c4805a510 0x7f3c4805c9d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f3c60006f70 tx=0x7f3c60017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:46.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.162+0000 7f3c597fa700 1 -- 172.21.15.121:0/3089412884 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3c54047bb0 con 0x7f3c640bc9b0 2023-11-15T02:58:46.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.362+0000 7f3c6b2db700 1 -- 172.21.15.121:0/3089412884 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3c64154ab0 con 0x7f3c640bc9b0 2023-11-15T02:58:46.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.962+0000 7f3c597fa700 1 -- 172.21.15.121:0/3089412884 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3c540190a0 con 0x7f3c640bc9b0 2023-11-15T02:58:46.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.966+0000 7f3c6b2db700 1 -- 172.21.15.121:0/3089412884 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3c4805a510 msgr2=0x7f3c4805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:46.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.966+0000 7f3c6b2db700 1 --2- 172.21.15.121:0/3089412884 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3c4805a510 0x7f3c4805c9d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f3c60006f70 tx=0x7f3c60017000).stop 2023-11-15T02:58:46.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.966+0000 7f3c6b2db700 1 -- 172.21.15.121:0/3089412884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c640bc9b0 msgr2=0x7f3c641577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:46.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.966+0000 7f3c6b2db700 1 --2- 172.21.15.121:0/3089412884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c640bc9b0 0x7f3c641577c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f3c5400f970 tx=0x7f3c54009040).stop 2023-11-15T02:58:46.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.970+0000 7f3c6b2db700 1 -- 172.21.15.121:0/3089412884 shutdown_connections 2023-11-15T02:58:46.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.970+0000 7f3c6b2db700 1 --2- 172.21.15.121:0/3089412884 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3c4805a510 0x7f3c4805c9d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:46.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.970+0000 7f3c6b2db700 1 --2- 172.21.15.121:0/3089412884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c640bcd90 0x7f3c64165a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:46.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.970+0000 7f3c6b2db700 1 --2- 172.21.15.121:0/3089412884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c640bc9b0 0x7f3c641577c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:46.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.970+0000 7f3c6b2db700 1 --2- 172.21.15.121:0/3089412884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c64062190 0x7f3c641412a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:46.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.970+0000 7f3c6b2db700 1 -- 172.21.15.121:0/3089412884 >> 172.21.15.121:0/3089412884 conn(0x7f3c640591c0 msgr2=0x7f3c640c3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:46.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.970+0000 7f3c6b2db700 1 -- 172.21.15.121:0/3089412884 shutdown_connections 2023-11-15T02:58:46.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:46.970+0000 7f3c6b2db700 1 -- 172.21.15.121:0/3089412884 wait complete. 2023-11-15T02:58:46.997 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-15T02:58:47.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.210+0000 7f7da71a9700 1 Processor -- start 2023-11-15T02:58:47.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.210+0000 7f7da71a9700 1 -- start start 2023-11-15T02:58:47.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.214+0000 7f7da71a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da0158f40 0x7f7da0159320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:47.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.214+0000 7f7da71a9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7da0067020 0x7f7da00a5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:47.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.214+0000 7f7da71a9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da00a6290 con 0x7f7da00a5c10 2023-11-15T02:58:47.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.214+0000 7f7da71a9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da0158640 con 0x7f7da0067020 2023-11-15T02:58:47.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.214+0000 7f7da71a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da0066b90 con 0x7f7da0158f40 2023-11-15T02:58:47.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.214+0000 7f7d9ffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7da0067020 0x7f7da00a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:47.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.214+0000 7f7d9ffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7da0067020 0x7f7da00a5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:52404/0 (socket says 172.21.15.121:52404) 2023-11-15T02:58:47.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.214+0000 7f7d9ffff700 1 -- 172.21.15.121:0/1768051198 learned_addr learned my addr 172.21.15.121:0/1768051198 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:47.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.214+0000 7f7da4f45700 1 --2- 172.21.15.121:0/1768051198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da0158f40 0x7f7da0159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:47.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.214+0000 7f7d9ffff700 1 -- 172.21.15.121:0/1768051198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da0158f40 msgr2=0x7f7da0159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:47.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.214+0000 7f7d9ffff700 1 --2- 172.21.15.121:0/1768051198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da0158f40 0x7f7da0159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:47.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.214+0000 7f7d9ffff700 1 -- 172.21.15.121:0/1768051198 >> v1:172.21.15.190:6789/0 conn(0x7f7da00a5c10 legacy=0x7f7da00bc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:47.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.214+0000 7f7d9ffff700 1 -- 172.21.15.121:0/1768051198 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da0066260 con 0x7f7da0067020 2023-11-15T02:58:47.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.214+0000 7f7d9ffff700 1 --2- 172.21.15.121:0/1768051198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7da0067020 0x7f7da00a5720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f7d8c006fd0 tx=0x7f7d8c0090f0).ready entity=mon.1 client_cookie=b12193fb62713c15 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:47.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.214+0000 7f7d9f7fe700 1 -- 172.21.15.121:0/1768051198 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c00b580 con 0x7f7da0067020 2023-11-15T02:58:47.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.214+0000 7f7d9f7fe700 1 -- 172.21.15.121:0/1768051198 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8c00b6e0 con 0x7f7da0067020 2023-11-15T02:58:47.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.214+0000 7f7d9f7fe700 1 -- 172.21.15.121:0/1768051198 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c00ba60 con 0x7f7da0067020 2023-11-15T02:58:47.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.214+0000 7f7da71a9700 1 -- 172.21.15.121:0/1768051198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7da0067020 msgr2=0x7f7da00a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:47.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.214+0000 7f7da71a9700 1 --2- 172.21.15.121:0/1768051198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7da0067020 0x7f7da00a5720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f7d8c006fd0 tx=0x7f7d8c0090f0).stop 2023-11-15T02:58:47.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.218+0000 7f7da71a9700 1 -- 172.21.15.121:0/1768051198 shutdown_connections 2023-11-15T02:58:47.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.218+0000 7f7da71a9700 1 --2- 172.21.15.121:0/1768051198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7da0067020 0x7f7da00a5720 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:47.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.218+0000 7f7da71a9700 1 --2- 172.21.15.121:0/1768051198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da0158f40 0x7f7da0159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:47.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.218+0000 7f7da71a9700 1 -- 172.21.15.121:0/1768051198 >> 172.21.15.121:0/1768051198 conn(0x7f7da0068500 msgr2=0x7f7da00c6780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:47.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.218+0000 7f7da71a9700 1 -- 172.21.15.121:0/1768051198 shutdown_connections 2023-11-15T02:58:47.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.218+0000 7f7da71a9700 1 -- 172.21.15.121:0/1768051198 wait complete. 2023-11-15T02:58:47.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.218+0000 7f7da71a9700 1 Processor -- start 2023-11-15T02:58:47.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.218+0000 7f7da71a9700 1 -- start start 2023-11-15T02:58:47.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7da71a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7da0067020 0x7f7da00bfb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:47.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7da71a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da00a5c10 0x7f7da00aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:47.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7da71a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7da0158f40 0x7f7da00aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:47.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7da71a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da00a8080 con 0x7f7da0067020 2023-11-15T02:58:47.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7da71a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da00a74e0 con 0x7f7da0158f40 2023-11-15T02:58:47.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7da71a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da00aabc0 con 0x7f7da00a5c10 2023-11-15T02:58:47.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7da4f45700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7da0067020 0x7f7da00bfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:47.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7da4f45700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7da0067020 0x7f7da00bfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39090/0 (socket says 172.21.15.121:39090) 2023-11-15T02:58:47.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7da4f45700 1 -- 172.21.15.121:0/3363069486 learned_addr learned my addr 172.21.15.121:0/3363069486 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:47.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7d9ffff700 1 --2- 172.21.15.121:0/3363069486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da00a5c10 0x7f7da00aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:47.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7da5746700 1 --2- 172.21.15.121:0/3363069486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7da0158f40 0x7f7da00aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:47.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7da4f45700 1 -- 172.21.15.121:0/3363069486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da00a5c10 msgr2=0x7f7da00aa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:47.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7da4f45700 1 --2- 172.21.15.121:0/3363069486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da00a5c10 0x7f7da00aa1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:47.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7da4f45700 1 -- 172.21.15.121:0/3363069486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7da0158f40 msgr2=0x7f7da00aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:47.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7da4f45700 1 --2- 172.21.15.121:0/3363069486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7da0158f40 0x7f7da00aa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:47.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7da4f45700 1 -- 172.21.15.121:0/3363069486 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8c009370 con 0x7f7da0067020 2023-11-15T02:58:47.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7d9ffff700 1 --2- 172.21.15.121:0/3363069486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da00a5c10 0x7f7da00aa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:47.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7da4f45700 1 --2- 172.21.15.121:0/3363069486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7da0067020 0x7f7da00bfb60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f7d9400e900 tx=0x7f7d94009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:47.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7d9cff9700 1 -- 172.21.15.121:0/3363069486 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9400c640 con 0x7f7da0067020 2023-11-15T02:58:47.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7d9cff9700 1 -- 172.21.15.121:0/3363069486 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9400c7a0 con 0x7f7da0067020 2023-11-15T02:58:47.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7da71a9700 1 -- 172.21.15.121:0/3363069486 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da00ac3d0 con 0x7f7da0067020 2023-11-15T02:58:47.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7d9cff9700 1 -- 172.21.15.121:0/3363069486 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9400cb20 con 0x7f7da0067020 2023-11-15T02:58:47.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.222+0000 7f7da71a9700 1 -- 172.21.15.121:0/3363069486 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da00a7a30 con 0x7f7da0067020 2023-11-15T02:58:47.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.226+0000 7f7da71a9700 1 -- 172.21.15.121:0/3363069486 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da006c150 con 0x7f7da0067020 2023-11-15T02:58:47.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.226+0000 7f7d9cff9700 1 -- 172.21.15.121:0/3363069486 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7d9401b400 con 0x7f7da0067020 2023-11-15T02:58:47.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.230+0000 7f7d9cff9700 1 --2- 172.21.15.121:0/3363069486 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7d8405a540 0x7f7d8405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:47.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.230+0000 7f7d9cff9700 1 -- 172.21.15.121:0/3363069486 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f7d94074c70 con 0x7f7da0067020 2023-11-15T02:58:47.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.230+0000 7f7d9ffff700 1 --2- 172.21.15.121:0/3363069486 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7d8405a540 0x7f7d8405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:47.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.230+0000 7f7d9ffff700 1 --2- 172.21.15.121:0/3363069486 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7d8405a540 0x7f7d8405ca00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f7d8c006fd0 tx=0x7f7d8c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:47.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.234+0000 7f7d9cff9700 1 -- 172.21.15.121:0/3363069486 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7d94048320 con 0x7f7da0067020 2023-11-15T02:58:47.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.398+0000 7f7da71a9700 1 -- 172.21.15.121:0/3363069486 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7da0158900 con 0x7f7da0067020 2023-11-15T02:58:47.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.970+0000 7f7d9cff9700 1 -- 172.21.15.121:0/3363069486 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7d94016030 con 0x7f7da0067020 2023-11-15T02:58:47.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.974+0000 7f7da71a9700 1 -- 172.21.15.121:0/3363069486 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7d8405a540 msgr2=0x7f7d8405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:47.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.974+0000 7f7da71a9700 1 --2- 172.21.15.121:0/3363069486 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7d8405a540 0x7f7d8405ca00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f7d8c006fd0 tx=0x7f7d8c017000).stop 2023-11-15T02:58:47.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.974+0000 7f7da71a9700 1 -- 172.21.15.121:0/3363069486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7da0067020 msgr2=0x7f7da00bfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:47.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.974+0000 7f7da71a9700 1 --2- 172.21.15.121:0/3363069486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7da0067020 0x7f7da00bfb60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f7d9400e900 tx=0x7f7d94009060).stop 2023-11-15T02:58:47.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.978+0000 7f7da71a9700 1 -- 172.21.15.121:0/3363069486 shutdown_connections 2023-11-15T02:58:47.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.978+0000 7f7da71a9700 1 --2- 172.21.15.121:0/3363069486 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7d8405a540 0x7f7d8405ca00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:47.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.978+0000 7f7da71a9700 1 --2- 172.21.15.121:0/3363069486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7da0158f40 0x7f7da00aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:47.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.978+0000 7f7da71a9700 1 --2- 172.21.15.121:0/3363069486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da00a5c10 0x7f7da00aa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:47.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.978+0000 7f7da71a9700 1 --2- 172.21.15.121:0/3363069486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7da0067020 0x7f7da00bfb60 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:47.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.978+0000 7f7da71a9700 1 -- 172.21.15.121:0/3363069486 >> 172.21.15.121:0/3363069486 conn(0x7f7da0068500 msgr2=0x7f7da00c3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:47.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.978+0000 7f7da71a9700 1 -- 172.21.15.121:0/3363069486 shutdown_connections 2023-11-15T02:58:47.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:47.978+0000 7f7da71a9700 1 -- 172.21.15.121:0/3363069486 wait complete. 2023-11-15T02:58:47.986 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_metadata' removed 2023-11-15T02:58:48.006 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-15T02:58:48.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.170+0000 7f326489a700 1 Processor -- start 2023-11-15T02:58:48.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.170+0000 7f326489a700 1 -- start start 2023-11-15T02:58:48.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.174+0000 7f326489a700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f32601511f0 0x7f3260066570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:48.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.174+0000 7f326489a700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3260066a60 con 0x7f32600bcab0 2023-11-15T02:58:48.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.174+0000 7f326489a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32600660e0 con 0x7f32600a8750 2023-11-15T02:58:48.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.174+0000 7f326489a700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32600bd390 con 0x7f32601511f0 2023-11-15T02:58:48.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.174+0000 7f325e59c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f32600bcab0 0x7f32601515d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:48956/0 (socket says 172.21.15.121:48956) 2023-11-15T02:58:48.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.174+0000 7f325e59c700 1 -- 172.21.15.121:0/1104810746 learned_addr learned my addr 172.21.15.121:0/1104810746 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:48.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.174+0000 7f325ed9d700 1 --2- 172.21.15.121:0/1104810746 >> v2:172.21.15.121:3300/0 conn(0x7f32601511f0 0x7f3260066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:48.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.174+0000 7f325dd9b700 1 -- 172.21.15.121:0/1104810746 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1472374585 0 0) 0x7f3260066a60 con 0x7f32600bcab0 2023-11-15T02:58:48.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.174+0000 7f325dd9b700 1 -- 172.21.15.121:0/1104810746 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32480033f0 con 0x7f32600bcab0 2023-11-15T02:58:48.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.174+0000 7f325ed9d700 1 -- 172.21.15.121:0/1104810746 >> v1:172.21.15.190:6789/0 conn(0x7f32600a8750 legacy=0x7f32600a8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:48.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.174+0000 7f325ed9d700 1 -- 172.21.15.121:0/1104810746 >> v1:172.21.15.121:6789/0 conn(0x7f32600bcab0 legacy=0x7f32601515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:48.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.174+0000 7f325ed9d700 1 -- 172.21.15.121:0/1104810746 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32600657b0 con 0x7f32601511f0 2023-11-15T02:58:48.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.174+0000 7f325ed9d700 1 --2- 172.21.15.121:0/1104810746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32601511f0 0x7f3260066570 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f3250009760 tx=0x7f3250009060).ready entity=mon.0 client_cookie=d0a679cc80764120 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:48.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.178+0000 7f325dd9b700 1 -- 172.21.15.121:0/1104810746 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325000f510 con 0x7f32601511f0 2023-11-15T02:58:48.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.178+0000 7f325dd9b700 1 -- 172.21.15.121:0/1104810746 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325000f670 con 0x7f32601511f0 2023-11-15T02:58:48.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.178+0000 7f325dd9b700 1 -- 172.21.15.121:0/1104810746 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325000f9f0 con 0x7f32601511f0 2023-11-15T02:58:48.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.178+0000 7f326489a700 1 -- 172.21.15.121:0/1104810746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32601511f0 msgr2=0x7f3260066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:48.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.178+0000 7f326489a700 1 --2- 172.21.15.121:0/1104810746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32601511f0 0x7f3260066570 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f3250009760 tx=0x7f3250009060).stop 2023-11-15T02:58:48.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.178+0000 7f326489a700 1 -- 172.21.15.121:0/1104810746 shutdown_connections 2023-11-15T02:58:48.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.178+0000 7f326489a700 1 --2- 172.21.15.121:0/1104810746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32601511f0 0x7f3260066570 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:48.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.178+0000 7f326489a700 1 -- 172.21.15.121:0/1104810746 >> 172.21.15.121:0/1104810746 conn(0x7f3260067a60 msgr2=0x7f3260067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:48.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.178+0000 7f326489a700 1 -- 172.21.15.121:0/1104810746 shutdown_connections 2023-11-15T02:58:48.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.178+0000 7f326489a700 1 -- 172.21.15.121:0/1104810746 wait complete. 2023-11-15T02:58:48.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f326489a700 1 Processor -- start 2023-11-15T02:58:48.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f326489a700 1 -- start start 2023-11-15T02:58:48.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f326489a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32600a8750 0x7f326014ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:48.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f326489a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32600bcab0 0x7f3260062580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:48.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f326489a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32601511f0 0x7f3260162a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:48.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f326489a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3260062a70 con 0x7f32600a8750 2023-11-15T02:58:48.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f326489a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326014d060 con 0x7f32601511f0 2023-11-15T02:58:48.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f326489a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326014d340 con 0x7f32600bcab0 2023-11-15T02:58:48.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f325ed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32600a8750 0x7f326014ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:48.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f325ed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32600a8750 0x7f326014ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39096/0 (socket says 172.21.15.121:39096) 2023-11-15T02:58:48.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f325ed9d700 1 -- 172.21.15.121:0/300243549 learned_addr learned my addr 172.21.15.121:0/300243549 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:48.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f325e59c700 1 --2- 172.21.15.121:0/300243549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32600bcab0 0x7f3260062580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:48.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f325f59e700 1 --2- 172.21.15.121:0/300243549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32601511f0 0x7f3260162a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:48.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f325ed9d700 1 -- 172.21.15.121:0/300243549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32600bcab0 msgr2=0x7f3260062580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:48.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f325ed9d700 1 --2- 172.21.15.121:0/300243549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32600bcab0 0x7f3260062580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:48.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f325ed9d700 1 -- 172.21.15.121:0/300243549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32601511f0 msgr2=0x7f3260162a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:48.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f325ed9d700 1 --2- 172.21.15.121:0/300243549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32601511f0 0x7f3260162a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:48.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f325ed9d700 1 -- 172.21.15.121:0/300243549 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32500092a0 con 0x7f32600a8750 2023-11-15T02:58:48.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f325e59c700 1 --2- 172.21.15.121:0/300243549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32600bcab0 0x7f3260062580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:48.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f325f59e700 1 --2- 172.21.15.121:0/300243549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32601511f0 0x7f3260162a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:48.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.182+0000 7f325ed9d700 1 --2- 172.21.15.121:0/300243549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32600a8750 0x7f326014ca80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f3250009e00 tx=0x7f3250017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:48.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.186+0000 7f3246ffd700 1 -- 172.21.15.121:0/300243549 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325000f7e0 con 0x7f32600a8750 2023-11-15T02:58:48.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.186+0000 7f3246ffd700 1 -- 172.21.15.121:0/300243549 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325000f940 con 0x7f32600a8750 2023-11-15T02:58:48.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.186+0000 7f3246ffd700 1 -- 172.21.15.121:0/300243549 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325001b690 con 0x7f32600a8750 2023-11-15T02:58:48.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.186+0000 7f326489a700 1 -- 172.21.15.121:0/300243549 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326014d620 con 0x7f32600a8750 2023-11-15T02:58:48.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.186+0000 7f326489a700 1 -- 172.21.15.121:0/300243549 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326014dbe0 con 0x7f32600a8750 2023-11-15T02:58:48.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.186+0000 7f3246ffd700 1 -- 172.21.15.121:0/300243549 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f325001ba40 con 0x7f32600a8750 2023-11-15T02:58:48.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.186+0000 7f326489a700 1 -- 172.21.15.121:0/300243549 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326006bcf0 con 0x7f32600a8750 2023-11-15T02:58:48.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.186+0000 7f3246ffd700 1 --2- 172.21.15.121:0/300243549 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f324c05a510 0x7f324c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:48.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.186+0000 7f3246ffd700 1 -- 172.21.15.121:0/300243549 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5884+0+0 (secure 0 0 0) 0x7f3250075ea0 con 0x7f32600a8750 2023-11-15T02:58:48.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.186+0000 7f325e59c700 1 --2- 172.21.15.121:0/300243549 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f324c05a510 0x7f324c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:48.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.190+0000 7f325e59c700 1 --2- 172.21.15.121:0/300243549 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f324c05a510 0x7f324c05c9d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f3260165ed0 tx=0x7f3254009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:48.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.194+0000 7f3246ffd700 1 -- 172.21.15.121:0/300243549 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3250049770 con 0x7f32600a8750 2023-11-15T02:58:48.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.362+0000 7f326489a700 1 -- 172.21.15.121:0/300243549 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f32601553a0 con 0x7f32600a8750 2023-11-15T02:58:48.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.982+0000 7f3246ffd700 1 -- 172.21.15.121:0/300243549 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) v1 ==== 169+0+0 (secure 0 0 0) 0x7f325004cd90 con 0x7f32600a8750 2023-11-15T02:58:48.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.986+0000 7f326489a700 1 -- 172.21.15.121:0/300243549 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f324c05a510 msgr2=0x7f324c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:48.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.986+0000 7f326489a700 1 --2- 172.21.15.121:0/300243549 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f324c05a510 0x7f324c05c9d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f3260165ed0 tx=0x7f3254009040).stop 2023-11-15T02:58:48.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.986+0000 7f326489a700 1 -- 172.21.15.121:0/300243549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32600a8750 msgr2=0x7f326014ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:48.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.986+0000 7f326489a700 1 --2- 172.21.15.121:0/300243549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32600a8750 0x7f326014ca80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f3250009e00 tx=0x7f3250017000).stop 2023-11-15T02:58:48.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.986+0000 7f326489a700 1 -- 172.21.15.121:0/300243549 shutdown_connections 2023-11-15T02:58:48.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.986+0000 7f326489a700 1 --2- 172.21.15.121:0/300243549 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f324c05a510 0x7f324c05c9d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:48.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.986+0000 7f326489a700 1 --2- 172.21.15.121:0/300243549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32601511f0 0x7f3260162a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:48.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.986+0000 7f326489a700 1 --2- 172.21.15.121:0/300243549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32600bcab0 0x7f3260062580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:48.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.986+0000 7f326489a700 1 --2- 172.21.15.121:0/300243549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32600a8750 0x7f326014ca80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:48.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.986+0000 7f326489a700 1 -- 172.21.15.121:0/300243549 >> 172.21.15.121:0/300243549 conn(0x7f3260067a60 msgr2=0x7f32600bfcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:48.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.986+0000 7f326489a700 1 -- 172.21.15.121:0/300243549 shutdown_connections 2023-11-15T02:58:48.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:48.990+0000 7f326489a700 1 -- 172.21.15.121:0/300243549 wait complete. 2023-11-15T02:58:48.997 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_data' removed 2023-11-15T02:58:49.015 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-15T02:58:49.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.174+0000 7ff47edf9700 1 Processor -- start 2023-11-15T02:58:49.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.174+0000 7ff47edf9700 1 -- start start 2023-11-15T02:58:49.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47edf9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4780c5c70 0x7ff478153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:49.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47edf9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff478066330 0x7ff4780ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:49.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47edf9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff478065ea0 con 0x7ff4780acd00 2023-11-15T02:58:49.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47edf9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff478066bb0 con 0x7ff478066330 2023-11-15T02:58:49.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47edf9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff478065890 con 0x7ff4780c5c70 2023-11-15T02:58:49.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47ddf7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff4780acd00 0x7ff4781536c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:48960/0 (socket says 172.21.15.121:48960) 2023-11-15T02:58:49.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47ddf7700 1 -- 172.21.15.121:0/4294480756 learned_addr learned my addr 172.21.15.121:0/4294480756 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:49.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47e5f8700 1 --2- 172.21.15.121:0/4294480756 >> v2:172.21.15.190:3300/0 conn(0x7ff478066330 0x7ff4780ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:49.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47d5f6700 1 --2- 172.21.15.121:0/4294480756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4780c5c70 0x7ff478153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:49.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47cdf5700 1 -- 172.21.15.121:0/4294480756 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548607021 0 0) 0x7ff478065ea0 con 0x7ff4780acd00 2023-11-15T02:58:49.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47cdf5700 1 -- 172.21.15.121:0/4294480756 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4640033f0 con 0x7ff4780acd00 2023-11-15T02:58:49.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47cdf5700 1 -- 172.21.15.121:0/4294480756 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4119777374 0 0) 0x7ff4640033f0 con 0x7ff4780acd00 2023-11-15T02:58:49.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47cdf5700 1 -- 172.21.15.121:0/4294480756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4780c5c70 msgr2=0x7ff478153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:49.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47cdf5700 1 --2- 172.21.15.121:0/4294480756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4780c5c70 0x7ff478153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:49.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47cdf5700 1 -- 172.21.15.121:0/4294480756 >> v2:172.21.15.190:3300/0 conn(0x7ff478066330 msgr2=0x7ff4780ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:49.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47cdf5700 1 --2- 172.21.15.121:0/4294480756 >> v2:172.21.15.190:3300/0 conn(0x7ff478066330 0x7ff4780ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:49.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47cdf5700 1 -- 172.21.15.121:0/4294480756 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4780c64a0 con 0x7ff4780acd00 2023-11-15T02:58:49.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47cdf5700 1 -- 172.21.15.121:0/4294480756 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7ff4740015f0 con 0x7ff4780acd00 2023-11-15T02:58:49.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47cdf5700 1 -- 172.21.15.121:0/4294480756 >> v1:172.21.15.121:6789/0 conn(0x7ff4780acd00 legacy=0x7ff4781536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:49.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47cdf5700 1 --2- 172.21.15.121:0/4294480756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4640038e0 0x7ff464003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:49.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47cdf5700 1 --2- 172.21.15.121:0/4294480756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff464004690 0x7ff464004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:49.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47cdf5700 1 --2- 172.21.15.121:0/4294480756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4780c5c70 0x7ff464005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:49.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47cdf5700 1 -- 172.21.15.121:0/4294480756 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff478066bb0 con 0x7ff4640038e0 2023-11-15T02:58:49.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47cdf5700 1 -- 172.21.15.121:0/4294480756 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff478065890 con 0x7ff464004690 2023-11-15T02:58:49.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47cdf5700 1 -- 172.21.15.121:0/4294480756 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff46400b8e0 con 0x7ff4780c5c70 2023-11-15T02:58:49.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47ddf7700 1 --2- 172.21.15.121:0/4294480756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4640038e0 0x7ff464003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:49.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47d5f6700 1 --2- 172.21.15.121:0/4294480756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff464004690 0x7ff464004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:49.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.178+0000 7ff47e5f8700 1 --2- 172.21.15.121:0/4294480756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4780c5c70 0x7ff464005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:49.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.182+0000 7ff47ddf7700 1 -- 172.21.15.121:0/4294480756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4780c5c70 msgr2=0x7ff464005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:49.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.182+0000 7ff47ddf7700 1 --2- 172.21.15.121:0/4294480756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4780c5c70 0x7ff464005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:49.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.182+0000 7ff47ddf7700 1 -- 172.21.15.121:0/4294480756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff464004690 msgr2=0x7ff464004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:49.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.182+0000 7ff47ddf7700 1 --2- 172.21.15.121:0/4294480756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff464004690 0x7ff464004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:49.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.182+0000 7ff47ddf7700 1 -- 172.21.15.121:0/4294480756 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4640033f0 con 0x7ff4640038e0 2023-11-15T02:58:49.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.182+0000 7ff47ddf7700 1 --2- 172.21.15.121:0/4294480756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4640038e0 0x7ff464003d00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7ff47400c190 tx=0x7ff478065e10).ready entity=mon.0 client_cookie=ad146a6cfa5f493c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:49.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.182+0000 7ff47cdf5700 1 -- 172.21.15.121:0/4294480756 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff474007a40 con 0x7ff4640038e0 2023-11-15T02:58:49.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.182+0000 7ff47cdf5700 1 -- 172.21.15.121:0/4294480756 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47400cac0 con 0x7ff4640038e0 2023-11-15T02:58:49.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.182+0000 7ff47cdf5700 1 -- 172.21.15.121:0/4294480756 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47400ce40 con 0x7ff4640038e0 2023-11-15T02:58:49.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.182+0000 7ff47edf9700 1 -- 172.21.15.121:0/4294480756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4640038e0 msgr2=0x7ff464003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:49.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.182+0000 7ff47edf9700 1 --2- 172.21.15.121:0/4294480756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4640038e0 0x7ff464003d00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7ff47400c190 tx=0x7ff478065e10).stop 2023-11-15T02:58:49.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.182+0000 7ff47e5f8700 1 -- 172.21.15.121:0/4294480756 reap_dead start 2023-11-15T02:58:49.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.182+0000 7ff47edf9700 1 -- 172.21.15.121:0/4294480756 shutdown_connections 2023-11-15T02:58:49.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.182+0000 7ff47edf9700 1 -- 172.21.15.121:0/4294480756 >> 172.21.15.121:0/4294480756 conn(0x7ff478063fc0 msgr2=0x7ff478067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:49.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.182+0000 7ff47edf9700 1 -- 172.21.15.121:0/4294480756 shutdown_connections 2023-11-15T02:58:49.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.182+0000 7ff47edf9700 1 -- 172.21.15.121:0/4294480756 wait complete. 2023-11-15T02:58:49.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47edf9700 1 Processor -- start 2023-11-15T02:58:49.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47edf9700 1 -- start start 2023-11-15T02:58:49.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47edf9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4640038e0 0x7ff4781591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:49.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47edf9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4780ac640 0x7ff4780c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:49.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47edf9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4780c4960 0x7ff4780c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:49.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47edf9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff478154eb0 con 0x7ff4780c4960 2023-11-15T02:58:49.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47edf9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4781542a0 con 0x7ff4640038e0 2023-11-15T02:58:49.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47edf9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff478154580 con 0x7ff4780ac640 2023-11-15T02:58:49.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47e5f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4780c4960 0x7ff4780c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:49.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47e5f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4780c4960 0x7ff4780c25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39128/0 (socket says 172.21.15.121:39128) 2023-11-15T02:58:49.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47e5f8700 1 -- 172.21.15.121:0/494964932 learned_addr learned my addr 172.21.15.121:0/494964932 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:49.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47ddf7700 1 --2- 172.21.15.121:0/494964932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4640038e0 0x7ff4781591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:49.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47d5f6700 1 --2- 172.21.15.121:0/494964932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4780ac640 0x7ff4780c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:49.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47e5f8700 1 -- 172.21.15.121:0/494964932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4780ac640 msgr2=0x7ff4780c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:49.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47e5f8700 1 --2- 172.21.15.121:0/494964932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4780ac640 0x7ff4780c4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:49.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47e5f8700 1 -- 172.21.15.121:0/494964932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4640038e0 msgr2=0x7ff4781591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:49.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47e5f8700 1 --2- 172.21.15.121:0/494964932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4640038e0 0x7ff4781591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:49.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47e5f8700 1 -- 172.21.15.121:0/494964932 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff478154860 con 0x7ff4780c4960 2023-11-15T02:58:49.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47d5f6700 1 --2- 172.21.15.121:0/494964932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4780ac640 0x7ff4780c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:58:49.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff47e5f8700 1 --2- 172.21.15.121:0/494964932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4780c4960 0x7ff4780c25d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7ff4780c60d0 tx=0x7ff47000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:49.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.186+0000 7ff46dffb700 1 -- 172.21.15.121:0/494964932 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47000e040 con 0x7ff4780c4960 2023-11-15T02:58:49.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.190+0000 7ff46dffb700 1 -- 172.21.15.121:0/494964932 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47000f040 con 0x7ff4780c4960 2023-11-15T02:58:49.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.190+0000 7ff47edf9700 1 -- 172.21.15.121:0/494964932 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff478154b40 con 0x7ff4780c4960 2023-11-15T02:58:49.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.190+0000 7ff46dffb700 1 -- 172.21.15.121:0/494964932 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47000fe20 con 0x7ff4780c4960 2023-11-15T02:58:49.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.190+0000 7ff47edf9700 1 -- 172.21.15.121:0/494964932 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4780c2ac0 con 0x7ff4780c4960 2023-11-15T02:58:49.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.190+0000 7ff47edf9700 1 -- 172.21.15.121:0/494964932 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff478155020 con 0x7ff4780c4960 2023-11-15T02:58:49.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.190+0000 7ff46dffb700 1 -- 172.21.15.121:0/494964932 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff470015660 con 0x7ff4780c4960 2023-11-15T02:58:49.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.194+0000 7ff46dffb700 1 --2- 172.21.15.121:0/494964932 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff45c05a4e0 0x7ff45c05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:49.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.194+0000 7ff46dffb700 1 -- 172.21.15.121:0/494964932 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5441+0+0 (secure 0 0 0) 0x7ff470074a90 con 0x7ff4780c4960 2023-11-15T02:58:49.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.194+0000 7ff47ddf7700 1 --2- 172.21.15.121:0/494964932 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff45c05a4e0 0x7ff45c05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:49.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.194+0000 7ff47ddf7700 1 --2- 172.21.15.121:0/494964932 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff45c05a4e0 0x7ff45c05c9a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7ff474000ef0 tx=0x7ff474015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:49.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.198+0000 7ff46dffb700 1 -- 172.21.15.121:0/494964932 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff4700484b0 con 0x7ff4780c4960 2023-11-15T02:58:49.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.370+0000 7ff47edf9700 1 -- 172.21.15.121:0/494964932 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff47806bcf0 con 0x7ff4780c4960 2023-11-15T02:58:49.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.378+0000 7ff46dffb700 1 -- 172.21.15.121:0/494964932 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff47001c020 con 0x7ff4780c4960 2023-11-15T02:58:49.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.382+0000 7ff47edf9700 1 -- 172.21.15.121:0/494964932 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff45c05a4e0 msgr2=0x7ff45c05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:49.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.382+0000 7ff47edf9700 1 --2- 172.21.15.121:0/494964932 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff45c05a4e0 0x7ff45c05c9a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7ff474000ef0 tx=0x7ff474015000).stop 2023-11-15T02:58:49.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.382+0000 7ff47edf9700 1 -- 172.21.15.121:0/494964932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4780c4960 msgr2=0x7ff4780c25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:49.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.382+0000 7ff47edf9700 1 --2- 172.21.15.121:0/494964932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4780c4960 0x7ff4780c25d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7ff4780c60d0 tx=0x7ff47000b040).stop 2023-11-15T02:58:49.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.382+0000 7ff47edf9700 1 -- 172.21.15.121:0/494964932 shutdown_connections 2023-11-15T02:58:49.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.382+0000 7ff47edf9700 1 --2- 172.21.15.121:0/494964932 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff45c05a4e0 0x7ff45c05c9a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:49.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.382+0000 7ff47edf9700 1 --2- 172.21.15.121:0/494964932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4780c4960 0x7ff4780c25d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:49.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.382+0000 7ff47edf9700 1 --2- 172.21.15.121:0/494964932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4780ac640 0x7ff4780c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:49.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.382+0000 7ff47edf9700 1 --2- 172.21.15.121:0/494964932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4640038e0 0x7ff4781591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:49.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.382+0000 7ff47edf9700 1 -- 172.21.15.121:0/494964932 >> 172.21.15.121:0/494964932 conn(0x7ff478063fc0 msgr2=0x7ff4780b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:49.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.382+0000 7ff47edf9700 1 -- 172.21.15.121:0/494964932 shutdown_connections 2023-11-15T02:58:49.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.382+0000 7ff47edf9700 1 -- 172.21.15.121:0/494964932 wait complete. 2023-11-15T02:58:49.409 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry' 2023-11-15T02:58:49.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.574+0000 7efd5965b700 1 Processor -- start 2023-11-15T02:58:49.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.574+0000 7efd5965b700 1 -- start start 2023-11-15T02:58:49.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd5965b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efd540c6670 0x7efd540c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:49.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd5965b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efd54150d20 0x7efd541511b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:49.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd5965b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd540657f0 con 0x7efd54150940 2023-11-15T02:58:49.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd5965b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd54065db0 con 0x7efd540c6670 2023-11-15T02:58:49.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd5965b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd540c6fc0 con 0x7efd54150d20 2023-11-15T02:58:49.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd527fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efd540c6670 0x7efd540c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:49.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd52ffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7efd54150940 0x7efd54066240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:48966/0 (socket says 172.21.15.121:48966) 2023-11-15T02:58:49.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd52ffd700 1 -- 172.21.15.121:0/3875624033 learned_addr learned my addr 172.21.15.121:0/3875624033 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:49.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd537fe700 1 --2- 172.21.15.121:0/3875624033 >> v2:172.21.15.190:3300/0 conn(0x7efd54150d20 0x7efd541511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:49.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd527fc700 1 -- 172.21.15.121:0/3875624033 >> v2:172.21.15.190:3300/0 conn(0x7efd54150d20 msgr2=0x7efd541511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:49.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd527fc700 1 --2- 172.21.15.121:0/3875624033 >> v2:172.21.15.190:3300/0 conn(0x7efd54150d20 0x7efd541511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:49.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd527fc700 1 -- 172.21.15.121:0/3875624033 >> v1:172.21.15.121:6789/0 conn(0x7efd54150940 legacy=0x7efd54066240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:49.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd527fc700 1 -- 172.21.15.121:0/3875624033 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd54067390 con 0x7efd540c6670 2023-11-15T02:58:49.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd527fc700 1 --2- 172.21.15.121:0/3875624033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd540c6670 0x7efd540c6ad0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7efd54069a60 tx=0x7efd48009060).ready entity=mon.0 client_cookie=f775a2cd3a5407df server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:49.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd51ffb700 1 -- 172.21.15.121:0/3875624033 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd480090d0 con 0x7efd540c6670 2023-11-15T02:58:49.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd51ffb700 1 -- 172.21.15.121:0/3875624033 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4800b580 con 0x7efd540c6670 2023-11-15T02:58:49.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd51ffb700 1 -- 172.21.15.121:0/3875624033 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4800b900 con 0x7efd540c6670 2023-11-15T02:58:49.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd5965b700 1 -- 172.21.15.121:0/3875624033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd540c6670 msgr2=0x7efd540c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:49.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd5965b700 1 --2- 172.21.15.121:0/3875624033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd540c6670 0x7efd540c6ad0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7efd54069a60 tx=0x7efd48009060).stop 2023-11-15T02:58:49.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd5965b700 1 -- 172.21.15.121:0/3875624033 shutdown_connections 2023-11-15T02:58:49.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd5965b700 1 --2- 172.21.15.121:0/3875624033 >> v2:172.21.15.190:3300/0 conn(0x7efd54150d20 0x7efd541511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:49.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd5965b700 1 --2- 172.21.15.121:0/3875624033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd540c6670 0x7efd540c6ad0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:49.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.578+0000 7efd5965b700 1 -- 172.21.15.121:0/3875624033 >> 172.21.15.121:0/3875624033 conn(0x7efd54062190 msgr2=0x7efd54062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:49.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.582+0000 7efd5965b700 1 -- 172.21.15.121:0/3875624033 shutdown_connections 2023-11-15T02:58:49.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.582+0000 7efd5965b700 1 -- 172.21.15.121:0/3875624033 wait complete. 2023-11-15T02:58:49.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.582+0000 7efd5965b700 1 Processor -- start 2023-11-15T02:58:49.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.582+0000 7efd5965b700 1 -- start start 2023-11-15T02:58:49.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd5965b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd540c6670 0x7efd54144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:49.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd5965b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd54150940 0x7efd540c3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:49.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd5965b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd54150d20 0x7efd540c3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:49.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd5965b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd540c3b60 con 0x7efd54150d20 2023-11-15T02:58:49.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd5965b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd541464a0 con 0x7efd54150940 2023-11-15T02:58:49.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd5965b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd54145890 con 0x7efd540c6670 2023-11-15T02:58:49.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd537fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd54150d20 0x7efd540c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:49.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd537fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd54150d20 0x7efd540c3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39150/0 (socket says 172.21.15.121:39150) 2023-11-15T02:58:49.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd537fe700 1 -- 172.21.15.121:0/202969903 learned_addr learned my addr 172.21.15.121:0/202969903 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:49.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd527fc700 1 --2- 172.21.15.121:0/202969903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd54150940 0x7efd540c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:49.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd52ffd700 1 --2- 172.21.15.121:0/202969903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd540c6670 0x7efd54144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:49.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd537fe700 1 -- 172.21.15.121:0/202969903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd540c6670 msgr2=0x7efd54144030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:49.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd537fe700 1 --2- 172.21.15.121:0/202969903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd540c6670 0x7efd54144030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:49.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd537fe700 1 -- 172.21.15.121:0/202969903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd54150940 msgr2=0x7efd540c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:49.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd537fe700 1 --2- 172.21.15.121:0/202969903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd54150940 0x7efd540c3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:49.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd537fe700 1 -- 172.21.15.121:0/202969903 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd480092a0 con 0x7efd54150d20 2023-11-15T02:58:49.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd537fe700 1 --2- 172.21.15.121:0/202969903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd54150d20 0x7efd540c3670 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7efd4c007f20 tx=0x7efd4c009860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:49.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd3affd700 1 -- 172.21.15.121:0/202969903 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c00eb90 con 0x7efd54150d20 2023-11-15T02:58:49.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd3affd700 1 -- 172.21.15.121:0/202969903 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4c00ecf0 con 0x7efd54150d20 2023-11-15T02:58:49.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd5965b700 1 -- 172.21.15.121:0/202969903 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd54145b70 con 0x7efd54150d20 2023-11-15T02:58:49.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd3affd700 1 -- 172.21.15.121:0/202969903 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c014610 con 0x7efd54150d20 2023-11-15T02:58:49.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.586+0000 7efd5965b700 1 -- 172.21.15.121:0/202969903 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd54146130 con 0x7efd54150d20 2023-11-15T02:58:49.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.590+0000 7efd5965b700 1 -- 172.21.15.121:0/202969903 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd54146630 con 0x7efd54150d20 2023-11-15T02:58:49.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.590+0000 7efd3affd700 1 -- 172.21.15.121:0/202969903 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7efd4c014980 con 0x7efd54150d20 2023-11-15T02:58:49.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.594+0000 7efd3affd700 1 --2- 172.21.15.121:0/202969903 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efd4005a510 0x7efd4005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:49.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.594+0000 7efd3affd700 1 -- 172.21.15.121:0/202969903 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5441+0+0 (secure 0 0 0) 0x7efd4c074990 con 0x7efd54150d20 2023-11-15T02:58:49.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.594+0000 7efd52ffd700 1 --2- 172.21.15.121:0/202969903 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efd4005a510 0x7efd4005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:49.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.594+0000 7efd52ffd700 1 --2- 172.21.15.121:0/202969903 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efd4005a510 0x7efd4005c9d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7efd44007fd0 tx=0x7efd44007d30).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:49.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.598+0000 7efd3affd700 1 -- 172.21.15.121:0/202969903 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efd4c048350 con 0x7efd54150d20 2023-11-15T02:58:49.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.810+0000 7efd5965b700 1 -- 172.21.15.121:0/202969903 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]} v 0) v1 -- 0x7efd54151b10 con 0x7efd54150d20 2023-11-15T02:58:49.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.990+0000 7efd3affd700 1 -- 172.21.15.121:0/202969903 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]}]=0 v119) v1 ==== 134+0+0 (secure 0 0 0) 0x7efd4c00ee60 con 0x7efd54150d20 2023-11-15T02:58:49.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.994+0000 7efd5965b700 1 -- 172.21.15.121:0/202969903 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efd4005a510 msgr2=0x7efd4005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:49.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.994+0000 7efd5965b700 1 --2- 172.21.15.121:0/202969903 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efd4005a510 0x7efd4005c9d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7efd44007fd0 tx=0x7efd44007d30).stop 2023-11-15T02:58:50.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.994+0000 7efd5965b700 1 -- 172.21.15.121:0/202969903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd54150d20 msgr2=0x7efd540c3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:50.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.994+0000 7efd5965b700 1 --2- 172.21.15.121:0/202969903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd54150d20 0x7efd540c3670 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7efd4c007f20 tx=0x7efd4c009860).stop 2023-11-15T02:58:50.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.994+0000 7efd5965b700 1 -- 172.21.15.121:0/202969903 shutdown_connections 2023-11-15T02:58:50.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.994+0000 7efd5965b700 1 --2- 172.21.15.121:0/202969903 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efd4005a510 0x7efd4005c9d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:50.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.994+0000 7efd5965b700 1 --2- 172.21.15.121:0/202969903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd54150d20 0x7efd540c3670 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:50.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.994+0000 7efd5965b700 1 --2- 172.21.15.121:0/202969903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd54150940 0x7efd540c3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:50.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.994+0000 7efd5965b700 1 --2- 172.21.15.121:0/202969903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd540c6670 0x7efd54144030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:50.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.994+0000 7efd5965b700 1 -- 172.21.15.121:0/202969903 >> 172.21.15.121:0/202969903 conn(0x7efd54062190 msgr2=0x7efd540aafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:50.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.994+0000 7efd5965b700 1 -- 172.21.15.121:0/202969903 shutdown_connections 2023-11-15T02:58:50.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:49.994+0000 7efd5965b700 1 -- 172.21.15.121:0/202969903 wait complete. 2023-11-15T02:58:50.022 INFO:tasks.cephfs_test_runner:test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) ... ok 2023-11-15T02:58:50.022 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_existing (tasks.cephfs.test_damage.TestDamage) That the MDS won't abort when the dentry is already known to be damaged. 2023-11-15T02:58:50.023 INFO:tasks.cephfs_test_runner:test_dentry_first_existing (tasks.cephfs.test_damage.TestDamage) 2023-11-15T02:58:50.023 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing' 2023-11-15T02:58:50.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.174+0000 7fa66a8f1700 1 Processor -- start 2023-11-15T02:58:50.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.174+0000 7fa66a8f1700 1 -- start start 2023-11-15T02:58:50.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.178+0000 7fa66a8f1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa664153bb0 0x7fa6640a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:50.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.178+0000 7fa66a8f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6640aa020 0x7fa664067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:50.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.178+0000 7fa66a8f1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6640cb5c0 con 0x7fa664067f50 2023-11-15T02:58:50.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.178+0000 7fa66a8f1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6640aa6f0 con 0x7fa664153bb0 2023-11-15T02:58:50.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.178+0000 7fa66a8f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6640c5900 con 0x7fa6640aa020 2023-11-15T02:58:50.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.178+0000 7fa663fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa664153bb0 0x7fa6640a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:50.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.178+0000 7fa663fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa664153bb0 0x7fa6640a9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39166/0 (socket says 172.21.15.121:39166) 2023-11-15T02:58:50.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.178+0000 7fa663fff700 1 -- 172.21.15.121:0/3385160573 learned_addr learned my addr 172.21.15.121:0/3385160573 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:50.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.178+0000 7fa65b7fe700 1 --2- 172.21.15.121:0/3385160573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6640aa020 0x7fa664067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:50.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.178+0000 7fa663fff700 1 -- 172.21.15.121:0/3385160573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6640aa020 msgr2=0x7fa664067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:50.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.178+0000 7fa663fff700 1 --2- 172.21.15.121:0/3385160573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6640aa020 0x7fa664067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:50.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.178+0000 7fa6637fe700 1 -- 172.21.15.121:0/3385160573 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3564754697 0 0) 0x7fa6640cb5c0 con 0x7fa664067f50 2023-11-15T02:58:50.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.178+0000 7fa663fff700 1 -- 172.21.15.121:0/3385160573 >> v1:172.21.15.190:6789/0 conn(0x7fa664067f50 legacy=0x7fa6641578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:50.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.178+0000 7fa663fff700 1 -- 172.21.15.121:0/3385160573 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6640683e0 con 0x7fa664153bb0 2023-11-15T02:58:50.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.178+0000 7fa663fff700 1 --2- 172.21.15.121:0/3385160573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa664153bb0 0x7fa6640a9b30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fa654006fd0 tx=0x7fa6540090f0).ready entity=mon.0 client_cookie=8da4d8e24ca4f5cf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:50.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.178+0000 7fa6637fe700 1 -- 172.21.15.121:0/3385160573 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65400b580 con 0x7fa664153bb0 2023-11-15T02:58:50.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.178+0000 7fa6637fe700 1 -- 172.21.15.121:0/3385160573 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65400b6e0 con 0x7fa664153bb0 2023-11-15T02:58:50.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.178+0000 7fa6637fe700 1 -- 172.21.15.121:0/3385160573 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65400ba60 con 0x7fa664153bb0 2023-11-15T02:58:50.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.182+0000 7fa66a8f1700 1 -- 172.21.15.121:0/3385160573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa664153bb0 msgr2=0x7fa6640a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:50.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.182+0000 7fa66a8f1700 1 --2- 172.21.15.121:0/3385160573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa664153bb0 0x7fa6640a9b30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fa654006fd0 tx=0x7fa6540090f0).stop 2023-11-15T02:58:50.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.182+0000 7fa66a8f1700 1 -- 172.21.15.121:0/3385160573 shutdown_connections 2023-11-15T02:58:50.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.182+0000 7fa66a8f1700 1 --2- 172.21.15.121:0/3385160573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6640aa020 0x7fa664067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:50.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.182+0000 7fa66a8f1700 1 --2- 172.21.15.121:0/3385160573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa664153bb0 0x7fa6640a9b30 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:50.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.182+0000 7fa66a8f1700 1 -- 172.21.15.121:0/3385160573 >> 172.21.15.121:0/3385160573 conn(0x7fa664058c50 msgr2=0x7fa664059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:50.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.182+0000 7fa66a8f1700 1 -- 172.21.15.121:0/3385160573 shutdown_connections 2023-11-15T02:58:50.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.182+0000 7fa66a8f1700 1 -- 172.21.15.121:0/3385160573 wait complete. 2023-11-15T02:58:50.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.182+0000 7fa66a8f1700 1 Processor -- start 2023-11-15T02:58:50.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.182+0000 7fa66a8f1700 1 -- start start 2023-11-15T02:58:50.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa66a8f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa664067f50 0x7fa6641644c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:50.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa66a8f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6640aa020 0x7fa66414ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:50.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa66a8f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa664153bb0 0x7fa6641529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:50.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa66a8f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66414ed50 con 0x7fa6640aa020 2023-11-15T02:58:50.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa66a8f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66414e2b0 con 0x7fa664067f50 2023-11-15T02:58:50.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa66a8f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6640ac5c0 con 0x7fa664153bb0 2023-11-15T02:58:50.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa65b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6640aa020 0x7fa66414ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:50.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa65b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6640aa020 0x7fa66414ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39180/0 (socket says 172.21.15.121:39180) 2023-11-15T02:58:50.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa65b7fe700 1 -- 172.21.15.121:0/283316998 learned_addr learned my addr 172.21.15.121:0/283316998 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:50.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa663fff700 1 --2- 172.21.15.121:0/283316998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa664067f50 0x7fa6641644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:50.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa668e8e700 1 --2- 172.21.15.121:0/283316998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa664153bb0 0x7fa6641529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:50.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa65b7fe700 1 -- 172.21.15.121:0/283316998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa664153bb0 msgr2=0x7fa6641529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:50.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa65b7fe700 1 --2- 172.21.15.121:0/283316998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa664153bb0 0x7fa6641529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:50.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa65b7fe700 1 -- 172.21.15.121:0/283316998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa664067f50 msgr2=0x7fa6641644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:50.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa65b7fe700 1 --2- 172.21.15.121:0/283316998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa664067f50 0x7fa6641644c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:50.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa65b7fe700 1 -- 172.21.15.121:0/283316998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa654009370 con 0x7fa6640aa020 2023-11-15T02:58:50.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa65b7fe700 1 --2- 172.21.15.121:0/283316998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6640aa020 0x7fa66414ddc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fa64c00e960 tx=0x7fa64c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:50.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa660ff9700 1 -- 172.21.15.121:0/283316998 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c009510 con 0x7fa6640aa020 2023-11-15T02:58:50.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa660ff9700 1 -- 172.21.15.121:0/283316998 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64c012600 con 0x7fa6640aa020 2023-11-15T02:58:50.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa660ff9700 1 -- 172.21.15.121:0/283316998 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c012980 con 0x7fa6640aa020 2023-11-15T02:58:50.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa66a8f1700 1 -- 172.21.15.121:0/283316998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6641532b0 con 0x7fa6640aa020 2023-11-15T02:58:50.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.186+0000 7fa66a8f1700 1 -- 172.21.15.121:0/283316998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6640abf70 con 0x7fa6640aa020 2023-11-15T02:58:50.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.190+0000 7fa66a8f1700 1 -- 172.21.15.121:0/283316998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa66406c150 con 0x7fa6640aa020 2023-11-15T02:58:50.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.190+0000 7fa660ff9700 1 -- 172.21.15.121:0/283316998 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa64c012d30 con 0x7fa6640aa020 2023-11-15T02:58:50.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.190+0000 7fa660ff9700 1 --2- 172.21.15.121:0/283316998 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa64405a510 0x7fa64405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:50.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.190+0000 7fa660ff9700 1 -- 172.21.15.121:0/283316998 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5441+0+0 (secure 0 0 0) 0x7fa64c074980 con 0x7fa6640aa020 2023-11-15T02:58:50.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.190+0000 7fa663fff700 1 --2- 172.21.15.121:0/283316998 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa64405a510 0x7fa64405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:50.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.190+0000 7fa663fff700 1 --2- 172.21.15.121:0/283316998 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa64405a510 0x7fa64405c9d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fa654009fd0 tx=0x7fa654017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:50.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.198+0000 7fa660ff9700 1 -- 172.21.15.121:0/283316998 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa64c048340 con 0x7fa6640aa020 2023-11-15T02:58:50.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.418+0000 7fa66a8f1700 1 -- 172.21.15.121:0/283316998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]} v 0) v1 -- 0x7fa664068ae0 con 0x7fa6640aa020 2023-11-15T02:58:50.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.994+0000 7fa660ff9700 1 -- 172.21.15.121:0/283316998 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]}]=0 v120) v1 ==== 144+0+0 (secure 0 0 0) 0x7fa64c04b960 con 0x7fa6640aa020 2023-11-15T02:58:51.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.998+0000 7fa66a8f1700 1 -- 172.21.15.121:0/283316998 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa64405a510 msgr2=0x7fa64405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:51.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.998+0000 7fa66a8f1700 1 --2- 172.21.15.121:0/283316998 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa64405a510 0x7fa64405c9d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fa654009fd0 tx=0x7fa654017000).stop 2023-11-15T02:58:51.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.998+0000 7fa66a8f1700 1 -- 172.21.15.121:0/283316998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6640aa020 msgr2=0x7fa66414ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:51.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.998+0000 7fa66a8f1700 1 --2- 172.21.15.121:0/283316998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6640aa020 0x7fa66414ddc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fa64c00e960 tx=0x7fa64c009060).stop 2023-11-15T02:58:51.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.998+0000 7fa66a8f1700 1 -- 172.21.15.121:0/283316998 shutdown_connections 2023-11-15T02:58:51.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.998+0000 7fa66a8f1700 1 --2- 172.21.15.121:0/283316998 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa64405a510 0x7fa64405c9d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.998+0000 7fa66a8f1700 1 --2- 172.21.15.121:0/283316998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa664153bb0 0x7fa6641529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.998+0000 7fa66a8f1700 1 --2- 172.21.15.121:0/283316998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6640aa020 0x7fa66414ddc0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.998+0000 7fa66a8f1700 1 --2- 172.21.15.121:0/283316998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa664067f50 0x7fa6641644c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.998+0000 7fa66a8f1700 1 -- 172.21.15.121:0/283316998 >> 172.21.15.121:0/283316998 conn(0x7fa664058c50 msgr2=0x7fa6640b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:51.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.998+0000 7fa66a8f1700 1 -- 172.21.15.121:0/283316998 shutdown_connections 2023-11-15T02:58:51.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:50.998+0000 7fa66a8f1700 1 -- 172.21.15.121:0/283316998 wait complete. 2023-11-15T02:58:51.025 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-15T02:58:51.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.178+0000 7f4c6cfb9700 1 Processor -- start 2023-11-15T02:58:51.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.178+0000 7f4c6cfb9700 1 -- start start 2023-11-15T02:58:51.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.182+0000 7f4c6cfb9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4c680c5c70 0x7f4c68153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:51.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.182+0000 7f4c6cfb9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c68065ea0 con 0x7f4c680acd00 2023-11-15T02:58:51.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.182+0000 7f4c6cfb9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c68066bb0 con 0x7f4c68066330 2023-11-15T02:58:51.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.182+0000 7f4c6cfb9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c68065890 con 0x7f4c680c5c70 2023-11-15T02:58:51.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.182+0000 7f4c66ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4c680c5c70 0x7f4c68153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:51.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.182+0000 7f4c677fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4c680acd00 0x7f4c681536c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:48970/0 (socket says 172.21.15.121:48970) 2023-11-15T02:58:51.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.182+0000 7f4c66ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4c680c5c70 0x7f4c68153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39182/0 (socket says 172.21.15.121:39182) 2023-11-15T02:58:51.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.182+0000 7f4c677fe700 1 -- 172.21.15.121:0/2549804206 learned_addr learned my addr 172.21.15.121:0/2549804206 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:51.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.182+0000 7f4c66ffd700 1 -- 172.21.15.121:0/2549804206 >> v1:172.21.15.190:6789/0 conn(0x7f4c68066330 legacy=0x7f4c681577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:51.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.182+0000 7f4c66ffd700 1 -- 172.21.15.121:0/2549804206 >> v1:172.21.15.121:6789/0 conn(0x7f4c680acd00 legacy=0x7f4c681536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:51.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.182+0000 7f4c66ffd700 1 -- 172.21.15.121:0/2549804206 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c680c64a0 con 0x7f4c680c5c70 2023-11-15T02:58:51.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.182+0000 7f4c667fc700 1 -- 172.21.15.121:0/2549804206 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 972984053 0 0) 0x7f4c68065ea0 con 0x7f4c680acd00 2023-11-15T02:58:51.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.186+0000 7f4c66ffd700 1 --2- 172.21.15.121:0/2549804206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c680c5c70 0x7f4c68153dd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f4c5c009760 tx=0x7f4c5c009060).ready entity=mon.0 client_cookie=ae72bac7a2c0d44c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:51.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.186+0000 7f4c667fc700 1 -- 172.21.15.121:0/2549804206 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c00b580 con 0x7f4c680c5c70 2023-11-15T02:58:51.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.186+0000 7f4c667fc700 1 -- 172.21.15.121:0/2549804206 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5c00b6e0 con 0x7f4c680c5c70 2023-11-15T02:58:51.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.186+0000 7f4c667fc700 1 -- 172.21.15.121:0/2549804206 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c00ba60 con 0x7f4c680c5c70 2023-11-15T02:58:51.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.186+0000 7f4c6cfb9700 1 -- 172.21.15.121:0/2549804206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c680c5c70 msgr2=0x7f4c68153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:51.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.186+0000 7f4c6cfb9700 1 --2- 172.21.15.121:0/2549804206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c680c5c70 0x7f4c68153dd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f4c5c009760 tx=0x7f4c5c009060).stop 2023-11-15T02:58:51.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.186+0000 7f4c6cfb9700 1 -- 172.21.15.121:0/2549804206 shutdown_connections 2023-11-15T02:58:51.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.186+0000 7f4c6cfb9700 1 --2- 172.21.15.121:0/2549804206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c680c5c70 0x7f4c68153dd0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.186+0000 7f4c6cfb9700 1 -- 172.21.15.121:0/2549804206 >> 172.21.15.121:0/2549804206 conn(0x7f4c68063fc0 msgr2=0x7f4c68067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:51.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.186+0000 7f4c6cfb9700 1 -- 172.21.15.121:0/2549804206 shutdown_connections 2023-11-15T02:58:51.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.186+0000 7f4c6cfb9700 1 -- 172.21.15.121:0/2549804206 wait complete. 2023-11-15T02:58:51.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.186+0000 7f4c6cfb9700 1 Processor -- start 2023-11-15T02:58:51.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c6cfb9700 1 -- start start 2023-11-15T02:58:51.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c6cfb9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c68066330 0x7f4c681414e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:51.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c6cfb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c680acd00 0x7f4c681544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:51.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c6cfb9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c680c5c70 0x7f4c68155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:51.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c6cfb9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c681419d0 con 0x7f4c680c5c70 2023-11-15T02:58:51.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c6cfb9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680c1e60 con 0x7f4c68066330 2023-11-15T02:58:51.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c6cfb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680caa70 con 0x7f4c680acd00 2023-11-15T02:58:51.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c67fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c680c5c70 0x7f4c68155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:51.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c67fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c680c5c70 0x7f4c68155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39186/0 (socket says 172.21.15.121:39186) 2023-11-15T02:58:51.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c67fff700 1 -- 172.21.15.121:0/1236835068 learned_addr learned my addr 172.21.15.121:0/1236835068 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:51.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c677fe700 1 --2- 172.21.15.121:0/1236835068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c68066330 0x7f4c681414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:51.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c66ffd700 1 --2- 172.21.15.121:0/1236835068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c680acd00 0x7f4c681544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:51.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c67fff700 1 -- 172.21.15.121:0/1236835068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c680acd00 msgr2=0x7f4c681544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:51.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c67fff700 1 --2- 172.21.15.121:0/1236835068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c680acd00 0x7f4c681544b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c67fff700 1 -- 172.21.15.121:0/1236835068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c68066330 msgr2=0x7f4c681414e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:51.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c67fff700 1 --2- 172.21.15.121:0/1236835068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c68066330 0x7f4c681414e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c67fff700 1 -- 172.21.15.121:0/1236835068 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5c0092a0 con 0x7f4c680c5c70 2023-11-15T02:58:51.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c67fff700 1 --2- 172.21.15.121:0/1236835068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c680c5c70 0x7f4c68155ee0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f4c60002230 tx=0x7f4c600015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:51.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c4f7fe700 1 -- 172.21.15.121:0/1236835068 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60010510 con 0x7f4c680c5c70 2023-11-15T02:58:51.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c4f7fe700 1 -- 172.21.15.121:0/1236835068 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c60010670 con 0x7f4c680c5c70 2023-11-15T02:58:51.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c6cfb9700 1 -- 172.21.15.121:0/1236835068 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c680c2100 con 0x7f4c680c5c70 2023-11-15T02:58:51.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c4f7fe700 1 -- 172.21.15.121:0/1236835068 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c600109f0 con 0x7f4c680c5c70 2023-11-15T02:58:51.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.190+0000 7f4c6cfb9700 1 -- 172.21.15.121:0/1236835068 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c680cad50 con 0x7f4c680c5c70 2023-11-15T02:58:51.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.194+0000 7f4c6cfb9700 1 -- 172.21.15.121:0/1236835068 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c680c2600 con 0x7f4c680c5c70 2023-11-15T02:58:51.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.194+0000 7f4c4f7fe700 1 -- 172.21.15.121:0/1236835068 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4c6001b400 con 0x7f4c680c5c70 2023-11-15T02:58:51.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.194+0000 7f4c4f7fe700 1 --2- 172.21.15.121:0/1236835068 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4c5405a510 0x7f4c5405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:51.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.194+0000 7f4c4f7fe700 1 -- 172.21.15.121:0/1236835068 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f4c60074320 con 0x7f4c680c5c70 2023-11-15T02:58:51.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.194+0000 7f4c677fe700 1 --2- 172.21.15.121:0/1236835068 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4c5405a510 0x7f4c5405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:51.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.198+0000 7f4c677fe700 1 --2- 172.21.15.121:0/1236835068 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4c5405a510 0x7f4c5405c9d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f4c58001360 tx=0x7f4c58009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:51.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.202+0000 7f4c4f7fe700 1 -- 172.21.15.121:0/1236835068 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4c60047d40 con 0x7f4c680c5c70 2023-11-15T02:58:51.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.370+0000 7f4c6cfb9700 1 -- 172.21.15.121:0/1236835068 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4c6806bcf0 con 0x7f4c680c5c70 2023-11-15T02:58:51.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.374+0000 7f4c4f7fe700 1 -- 172.21.15.121:0/1236835068 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4c6004b360 con 0x7f4c680c5c70 2023-11-15T02:58:51.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.378+0000 7f4c6cfb9700 1 -- 172.21.15.121:0/1236835068 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4c5405a510 msgr2=0x7f4c5405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:51.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.378+0000 7f4c6cfb9700 1 --2- 172.21.15.121:0/1236835068 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4c5405a510 0x7f4c5405c9d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f4c58001360 tx=0x7f4c58009040).stop 2023-11-15T02:58:51.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.378+0000 7f4c6cfb9700 1 -- 172.21.15.121:0/1236835068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c680c5c70 msgr2=0x7f4c68155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:51.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.378+0000 7f4c6cfb9700 1 --2- 172.21.15.121:0/1236835068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c680c5c70 0x7f4c68155ee0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f4c60002230 tx=0x7f4c600015e0).stop 2023-11-15T02:58:51.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.378+0000 7f4c6cfb9700 1 -- 172.21.15.121:0/1236835068 shutdown_connections 2023-11-15T02:58:51.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.378+0000 7f4c6cfb9700 1 --2- 172.21.15.121:0/1236835068 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4c5405a510 0x7f4c5405c9d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.378+0000 7f4c6cfb9700 1 --2- 172.21.15.121:0/1236835068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4c680c5c70 0x7f4c68155ee0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.378+0000 7f4c6cfb9700 1 --2- 172.21.15.121:0/1236835068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c680acd00 0x7f4c681544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.378+0000 7f4c6cfb9700 1 --2- 172.21.15.121:0/1236835068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c68066330 0x7f4c681414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.378+0000 7f4c6cfb9700 1 -- 172.21.15.121:0/1236835068 >> 172.21.15.121:0/1236835068 conn(0x7f4c68063fc0 msgr2=0x7f4c68152930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:51.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.378+0000 7f4c6cfb9700 1 -- 172.21.15.121:0/1236835068 shutdown_connections 2023-11-15T02:58:51.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.378+0000 7f4c6cfb9700 1 -- 172.21.15.121:0/1236835068 wait complete. 2023-11-15T02:58:51.407 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-15T02:58:51.408 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T02:58:51.450 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T02:58:51.495 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-15T02:58:51.496 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-15T02:58:51.496 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:58:51.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.650+0000 7fef80a5b700 1 Processor -- start 2023-11-15T02:58:51.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.650+0000 7fef80a5b700 1 -- start start 2023-11-15T02:58:51.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.650+0000 7fef80a5b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fef7c154ca0 0x7fef7c155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:51.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.650+0000 7fef80a5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c0a57c0 0x7fef7c0a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:51.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.650+0000 7fef80a5b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef7c0c4c30 con 0x7fef7c1562f0 2023-11-15T02:58:51.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.650+0000 7fef80a5b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef7c0a8280 con 0x7fef7c154ca0 2023-11-15T02:58:51.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.650+0000 7fef80a5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c0a6d40 con 0x7fef7c0a57c0 2023-11-15T02:58:51.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef71d9b700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fef7c1562f0 0x7fef7c0a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:48982/0 (socket says 172.21.15.121:48982) 2023-11-15T02:58:51.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef71d9b700 1 -- 172.21.15.121:0/1261577508 learned_addr learned my addr 172.21.15.121:0/1261577508 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:51.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef7a59c700 1 --2- 172.21.15.121:0/1261577508 >> v2:172.21.15.190:3300/0 conn(0x7fef7c154ca0 0x7fef7c155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:51.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef7ad9d700 1 --2- 172.21.15.121:0/1261577508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c0a57c0 0x7fef7c0a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:51.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef79d9b700 1 -- 172.21.15.121:0/1261577508 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 874559322 0 0) 0x7fef7c0c4c30 con 0x7fef7c1562f0 2023-11-15T02:58:51.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef79d9b700 1 -- 172.21.15.121:0/1261577508 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef680033f0 con 0x7fef7c1562f0 2023-11-15T02:58:51.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef7ad9d700 1 -- 172.21.15.121:0/1261577508 >> v2:172.21.15.190:3300/0 conn(0x7fef7c154ca0 msgr2=0x7fef7c155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:51.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef7ad9d700 1 --2- 172.21.15.121:0/1261577508 >> v2:172.21.15.190:3300/0 conn(0x7fef7c154ca0 0x7fef7c155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef7ad9d700 1 -- 172.21.15.121:0/1261577508 >> v1:172.21.15.121:6789/0 conn(0x7fef7c1562f0 legacy=0x7fef7c0a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:51.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef7ad9d700 1 -- 172.21.15.121:0/1261577508 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c0a6140 con 0x7fef7c0a57c0 2023-11-15T02:58:51.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef7ad9d700 1 --2- 172.21.15.121:0/1261577508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c0a57c0 0x7fef7c0a5c50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fef74009760 tx=0x7fef74009060).ready entity=mon.2 client_cookie=48d08a0cc2ed9e66 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:51.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef79d9b700 1 -- 172.21.15.121:0/1261577508 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7400d4a0 con 0x7fef7c0a57c0 2023-11-15T02:58:51.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef79d9b700 1 -- 172.21.15.121:0/1261577508 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7400d600 con 0x7fef7c0a57c0 2023-11-15T02:58:51.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef79d9b700 1 -- 172.21.15.121:0/1261577508 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7400d980 con 0x7fef7c0a57c0 2023-11-15T02:58:51.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef80a5b700 1 -- 172.21.15.121:0/1261577508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c0a57c0 msgr2=0x7fef7c0a5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:51.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef80a5b700 1 --2- 172.21.15.121:0/1261577508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c0a57c0 0x7fef7c0a5c50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fef74009760 tx=0x7fef74009060).stop 2023-11-15T02:58:51.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef80a5b700 1 -- 172.21.15.121:0/1261577508 shutdown_connections 2023-11-15T02:58:51.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef80a5b700 1 --2- 172.21.15.121:0/1261577508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c0a57c0 0x7fef7c0a5c50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef80a5b700 1 --2- 172.21.15.121:0/1261577508 >> v2:172.21.15.190:3300/0 conn(0x7fef7c154ca0 0x7fef7c155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef80a5b700 1 -- 172.21.15.121:0/1261577508 >> 172.21.15.121:0/1261577508 conn(0x7fef7c062190 msgr2=0x7fef7c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:51.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef80a5b700 1 -- 172.21.15.121:0/1261577508 shutdown_connections 2023-11-15T02:58:51.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.654+0000 7fef80a5b700 1 -- 172.21.15.121:0/1261577508 wait complete. 2023-11-15T02:58:51.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.658+0000 7fef80a5b700 1 Processor -- start 2023-11-15T02:58:51.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.658+0000 7fef80a5b700 1 -- start start 2023-11-15T02:58:51.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.658+0000 7fef80a5b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef7c0a57c0 0x7fef7c141c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:51.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.658+0000 7fef80a5b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef7c154ca0 0x7fef7c16b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:51.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.658+0000 7fef80a5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c1562f0 0x7fef7c16bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:51.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.658+0000 7fef80a5b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c16c220 con 0x7fef7c154ca0 2023-11-15T02:58:51.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.658+0000 7fef80a5b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c1417b0 con 0x7fef7c0a57c0 2023-11-15T02:58:51.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.658+0000 7fef80a5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c140ba0 con 0x7fef7c1562f0 2023-11-15T02:58:51.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.658+0000 7fef71d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef7c154ca0 0x7fef7c16b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:51.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.658+0000 7fef71d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef7c154ca0 0x7fef7c16b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39198/0 (socket says 172.21.15.121:39198) 2023-11-15T02:58:51.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.658+0000 7fef71d9b700 1 -- 172.21.15.121:0/2124115067 learned_addr learned my addr 172.21.15.121:0/2124115067 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:51.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.658+0000 7fef7a59c700 1 --2- 172.21.15.121:0/2124115067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef7c0a57c0 0x7fef7c141c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:51.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.662+0000 7fef7ad9d700 1 --2- 172.21.15.121:0/2124115067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c1562f0 0x7fef7c16bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:51.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.662+0000 7fef71d9b700 1 -- 172.21.15.121:0/2124115067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c1562f0 msgr2=0x7fef7c16bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:51.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.662+0000 7fef71d9b700 1 --2- 172.21.15.121:0/2124115067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c1562f0 0x7fef7c16bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.662+0000 7fef71d9b700 1 -- 172.21.15.121:0/2124115067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef7c0a57c0 msgr2=0x7fef7c141c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:51.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.662+0000 7fef71d9b700 1 --2- 172.21.15.121:0/2124115067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef7c0a57c0 0x7fef7c141c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.662+0000 7fef71d9b700 1 -- 172.21.15.121:0/2124115067 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef740092a0 con 0x7fef7c154ca0 2023-11-15T02:58:51.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.662+0000 7fef7a59c700 1 --2- 172.21.15.121:0/2124115067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef7c0a57c0 0x7fef7c141c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:58:51.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.662+0000 7fef71d9b700 1 --2- 172.21.15.121:0/2124115067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef7c154ca0 0x7fef7c16b840 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fef6400c420 tx=0x7fef64007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:51.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.662+0000 7fef72ffd700 1 -- 172.21.15.121:0/2124115067 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef640070d0 con 0x7fef7c154ca0 2023-11-15T02:58:51.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.662+0000 7fef72ffd700 1 -- 172.21.15.121:0/2124115067 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef64007230 con 0x7fef7c154ca0 2023-11-15T02:58:51.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.662+0000 7fef72ffd700 1 -- 172.21.15.121:0/2124115067 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef640075d0 con 0x7fef7c154ca0 2023-11-15T02:58:51.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.662+0000 7fef80a5b700 1 -- 172.21.15.121:0/2124115067 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7c140e80 con 0x7fef7c154ca0 2023-11-15T02:58:51.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.662+0000 7fef80a5b700 1 -- 172.21.15.121:0/2124115067 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c141440 con 0x7fef7c154ca0 2023-11-15T02:58:51.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.662+0000 7fef80a5b700 1 -- 172.21.15.121:0/2124115067 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7c141960 con 0x7fef7c154ca0 2023-11-15T02:58:51.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.666+0000 7fef72ffd700 1 -- 172.21.15.121:0/2124115067 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fef640059b0 con 0x7fef7c154ca0 2023-11-15T02:58:51.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.666+0000 7fef72ffd700 1 --2- 172.21.15.121:0/2124115067 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef5c05a510 0x7fef5c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:51.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.666+0000 7fef72ffd700 1 -- 172.21.15.121:0/2124115067 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5441+0+0 (secure 0 0 0) 0x7fef64073ca0 con 0x7fef7c154ca0 2023-11-15T02:58:51.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.666+0000 7fef7a59c700 1 --2- 172.21.15.121:0/2124115067 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef5c05a510 0x7fef5c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:51.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.666+0000 7fef7a59c700 1 --2- 172.21.15.121:0/2124115067 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef5c05a510 0x7fef5c05c9d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fef6c00e950 tx=0x7fef6c00b040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:51.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.670+0000 7fef72ffd700 1 -- 172.21.15.121:0/2124115067 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fef7c141960 con 0x7fef7c154ca0 2023-11-15T02:58:51.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.870+0000 7fef80a5b700 1 -- 172.21.15.121:0/2124115067 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef7c141920 con 0x7fef7c154ca0 2023-11-15T02:58:51.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.870+0000 7fef72ffd700 1 -- 172.21.15.121:0/2124115067 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+3541 (secure 0 0 0) 0x7fef64047660 con 0x7fef7c154ca0 2023-11-15T02:58:51.876 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:58:51.877 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2544127447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2544127447},{"type":"v1","addr":"172.21.15.190:6833","nonce":2544127447}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1815834193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1815834193},{"type":"v1","addr":"172.21.15.190:6835","nonce":1815834193}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4950,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3229917854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3229917854},{"type":"v1","addr":"172.21.15.121:6835","nonce":3229917854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2501106899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2501106899},{"type":"v1","addr":"172.21.15.121:6837","nonce":2501106899}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[]} 2023-11-15T02:58:51.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.870+0000 7fef80a5b700 1 -- 172.21.15.121:0/2124115067 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef5c05a510 msgr2=0x7fef5c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:51.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.870+0000 7fef80a5b700 1 --2- 172.21.15.121:0/2124115067 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef5c05a510 0x7fef5c05c9d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fef6c00e950 tx=0x7fef6c00b040).stop 2023-11-15T02:58:51.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.870+0000 7fef80a5b700 1 -- 172.21.15.121:0/2124115067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef7c154ca0 msgr2=0x7fef7c16b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:51.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.870+0000 7fef80a5b700 1 --2- 172.21.15.121:0/2124115067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef7c154ca0 0x7fef7c16b840 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fef6400c420 tx=0x7fef64007c50).stop 2023-11-15T02:58:51.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.874+0000 7fef80a5b700 1 -- 172.21.15.121:0/2124115067 shutdown_connections 2023-11-15T02:58:51.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.874+0000 7fef80a5b700 1 --2- 172.21.15.121:0/2124115067 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef5c05a510 0x7fef5c05c9d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.874+0000 7fef80a5b700 1 --2- 172.21.15.121:0/2124115067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c1562f0 0x7fef7c16bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.874+0000 7fef80a5b700 1 --2- 172.21.15.121:0/2124115067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef7c154ca0 0x7fef7c16b840 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.874+0000 7fef80a5b700 1 --2- 172.21.15.121:0/2124115067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef7c0a57c0 0x7fef7c141c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:51.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.874+0000 7fef80a5b700 1 -- 172.21.15.121:0/2124115067 >> 172.21.15.121:0/2124115067 conn(0x7fef7c062190 msgr2=0x7fef7c0c52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:51.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.874+0000 7fef80a5b700 1 -- 172.21.15.121:0/2124115067 shutdown_connections 2023-11-15T02:58:51.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:51.874+0000 7fef80a5b700 1 -- 172.21.15.121:0/2124115067 wait complete. 2023-11-15T02:58:51.882 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 25 2023-11-15T02:58:51.897 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-15T02:58:51.897 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-15T02:58:51.897 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-15T02:58:51.898 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T02:58:51.898 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-15T02:58:51.898 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-15T02:58:51.898 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-15T02:58:51.899 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T02:58:51.899 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-15T02:58:51.899 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-15T02:58:51.900 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-15T02:58:51.900 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T02:58:51.900 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-15T02:58:51.901 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-15T02:58:51.901 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-15T02:58:51.901 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T02:58:51.915 INFO:tasks.ceph.mds.a:Stopped 2023-11-15T02:58:51.916 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-15T02:58:51.919 INFO:tasks.ceph.mds.c:Stopped 2023-11-15T02:58:51.919 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-15T02:58:51.920 INFO:tasks.ceph.mds.b:Stopped 2023-11-15T02:58:51.920 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-15T02:58:51.925 INFO:tasks.ceph.mds.d:Stopped 2023-11-15T02:58:51.926 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-15T02:58:51.972 INFO:tasks.ceph.mds.a:Started 2023-11-15T02:58:51.973 DEBUG:teuthology.parallel:result is None 2023-11-15T02:58:51.973 INFO:tasks.ceph.mds.b:Started 2023-11-15T02:58:51.974 DEBUG:teuthology.parallel:result is None 2023-11-15T02:58:51.978 INFO:tasks.ceph.mds.c:Started 2023-11-15T02:58:51.978 DEBUG:teuthology.parallel:result is None 2023-11-15T02:58:51.981 INFO:tasks.ceph.mds.d:Started 2023-11-15T02:58:51.981 DEBUG:teuthology.parallel:result is None 2023-11-15T02:58:51.982 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-15T02:58:52.075 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T02:58:52.078 INFO:tasks.ceph.mds.c.smithi121.stdout:starting mds.c at 2023-11-15T02:58:52.081 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-11-15T02:58:52.087 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-11-15T02:58:52.128 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-11-15T02:58:52.128 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-15T02:58:52.128 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-11-15T02:58:52.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.130+0000 7f6f63479700 1 Processor -- start 2023-11-15T02:58:52.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.130+0000 7f6f63479700 1 -- start start 2023-11-15T02:58:52.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.134+0000 7f6f63479700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6f5c066570 0x7f6f5c1515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:52.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.134+0000 7f6f63479700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f5c0660e0 con 0x7f6f5c151ac0 2023-11-15T02:58:52.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.134+0000 7f6f63479700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f5c066a20 con 0x7f6f5c1511f0 2023-11-15T02:58:52.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.134+0000 7f6f63479700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f5c151ef0 con 0x7f6f5c066570 2023-11-15T02:58:52.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.134+0000 7f6f62c78700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6f5c151ac0 0x7f6f5c0a8750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49000/0 (socket says 172.21.15.121:49000) 2023-11-15T02:58:52.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.134+0000 7f6f62c78700 1 -- 172.21.15.121:0/2748867202 learned_addr learned my addr 172.21.15.121:0/2748867202 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:52.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.134+0000 7f6f61c76700 1 --2- 172.21.15.121:0/2748867202 >> v2:172.21.15.190:3300/0 conn(0x7f6f5c066570 0x7f6f5c1515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:52.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.134+0000 7f6f61475700 1 -- 172.21.15.121:0/2748867202 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154093903 0 0) 0x7f6f5c0660e0 con 0x7f6f5c151ac0 2023-11-15T02:58:52.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.134+0000 7f6f61475700 1 -- 172.21.15.121:0/2748867202 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f480033f0 con 0x7f6f5c151ac0 2023-11-15T02:58:52.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.134+0000 7f6f61c76700 1 -- 172.21.15.121:0/2748867202 >> v1:172.21.15.190:6789/0 conn(0x7f6f5c1511f0 legacy=0x7f6f5c0bcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:52.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.134+0000 7f6f61475700 1 -- 172.21.15.121:0/2748867202 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1785609405 0 0) 0x7f6f480033f0 con 0x7f6f5c151ac0 2023-11-15T02:58:52.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.134+0000 7f6f61c76700 1 -- 172.21.15.121:0/2748867202 >> v1:172.21.15.121:6789/0 conn(0x7f6f5c151ac0 legacy=0x7f6f5c0a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:52.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.134+0000 7f6f61c76700 1 -- 172.21.15.121:0/2748867202 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5c0bd390 con 0x7f6f5c066570 2023-11-15T02:58:52.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.134+0000 7f6f61c76700 1 --2- 172.21.15.121:0/2748867202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f5c066570 0x7f6f5c1515d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f6f4c006fd0 tx=0x7f6f4c006f40).ready entity=mon.1 client_cookie=9039a9d63295f957 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:52.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.138+0000 7f6f61475700 1 -- 172.21.15.121:0/2748867202 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f540017f0 con 0x7f6f5c066570 2023-11-15T02:58:52.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.138+0000 7f6f61475700 1 -- 172.21.15.121:0/2748867202 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4c005880 con 0x7f6f5c066570 2023-11-15T02:58:52.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.138+0000 7f6f61475700 1 -- 172.21.15.121:0/2748867202 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c005c00 con 0x7f6f5c066570 2023-11-15T02:58:52.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.138+0000 7f6f63479700 1 -- 172.21.15.121:0/2748867202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f5c066570 msgr2=0x7f6f5c1515d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:52.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.138+0000 7f6f63479700 1 --2- 172.21.15.121:0/2748867202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f5c066570 0x7f6f5c1515d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f6f4c006fd0 tx=0x7f6f4c006f40).stop 2023-11-15T02:58:52.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.138+0000 7f6f63479700 1 -- 172.21.15.121:0/2748867202 shutdown_connections 2023-11-15T02:58:52.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.138+0000 7f6f63479700 1 --2- 172.21.15.121:0/2748867202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f5c066570 0x7f6f5c1515d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:52.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.138+0000 7f6f63479700 1 -- 172.21.15.121:0/2748867202 >> 172.21.15.121:0/2748867202 conn(0x7f6f5c067a60 msgr2=0x7f6f5c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:52.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.138+0000 7f6f63479700 1 -- 172.21.15.121:0/2748867202 shutdown_connections 2023-11-15T02:58:52.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.138+0000 7f6f63479700 1 -- 172.21.15.121:0/2748867202 wait complete. 2023-11-15T02:58:52.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.142+0000 7f6f63479700 1 Processor -- start 2023-11-15T02:58:52.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.142+0000 7f6f63479700 1 -- start start 2023-11-15T02:58:52.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.142+0000 7f6f63479700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f5c066570 0x7f6f5c160cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:52.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.142+0000 7f6f63479700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f5c1511f0 0x7f6f5c0bdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:52.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.142+0000 7f6f63479700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f5c151ac0 0x7f6f5c0be0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:52.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.142+0000 7f6f63479700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c0a9f70 con 0x7f6f5c1511f0 2023-11-15T02:58:52.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.142+0000 7f6f63479700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c0aa250 con 0x7f6f5c151ac0 2023-11-15T02:58:52.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.142+0000 7f6f63479700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c0aa530 con 0x7f6f5c066570 2023-11-15T02:58:52.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.142+0000 7f6f61c76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f5c1511f0 0x7f6f5c0bdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:52.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.142+0000 7f6f61c76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f5c1511f0 0x7f6f5c0bdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39242/0 (socket says 172.21.15.121:39242) 2023-11-15T02:58:52.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.142+0000 7f6f61c76700 1 -- 172.21.15.121:0/606699058 learned_addr learned my addr 172.21.15.121:0/606699058 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:52.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.142+0000 7f6f62477700 1 --2- 172.21.15.121:0/606699058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f5c066570 0x7f6f5c160cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:52.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.142+0000 7f6f62c78700 1 --2- 172.21.15.121:0/606699058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f5c151ac0 0x7f6f5c0be0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:52.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.142+0000 7f6f62477700 1 -- 172.21.15.121:0/606699058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f5c151ac0 msgr2=0x7f6f5c0be0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:52.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.142+0000 7f6f62477700 1 --2- 172.21.15.121:0/606699058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f5c151ac0 0x7f6f5c0be0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:52.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.142+0000 7f6f62477700 1 -- 172.21.15.121:0/606699058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f5c1511f0 msgr2=0x7f6f5c0bdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:52.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.142+0000 7f6f62477700 1 --2- 172.21.15.121:0/606699058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f5c1511f0 0x7f6f5c0bdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:52.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.142+0000 7f6f62477700 1 -- 172.21.15.121:0/606699058 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4c009250 con 0x7f6f5c066570 2023-11-15T02:58:52.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.142+0000 7f6f61c76700 1 --2- 172.21.15.121:0/606699058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f5c1511f0 0x7f6f5c0bdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:58:52.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.146+0000 7f6f62477700 1 --2- 172.21.15.121:0/606699058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f5c066570 0x7f6f5c160cc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f6f5800c440 tx=0x7f6f58007d40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:52.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.146+0000 7f6f527fc700 1 -- 172.21.15.121:0/606699058 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f58007020 con 0x7f6f5c066570 2023-11-15T02:58:52.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.146+0000 7f6f527fc700 1 -- 172.21.15.121:0/606699058 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f58007180 con 0x7f6f5c066570 2023-11-15T02:58:52.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.146+0000 7f6f527fc700 1 -- 172.21.15.121:0/606699058 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f58007520 con 0x7f6f5c066570 2023-11-15T02:58:52.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.146+0000 7f6f63479700 1 -- 172.21.15.121:0/606699058 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f5c0aa810 con 0x7f6f5c066570 2023-11-15T02:58:52.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.146+0000 7f6f63479700 1 -- 172.21.15.121:0/606699058 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f5c0ab010 con 0x7f6f5c066570 2023-11-15T02:58:52.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.146+0000 7f6f63479700 1 -- 172.21.15.121:0/606699058 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f5c06bcf0 con 0x7f6f5c066570 2023-11-15T02:58:52.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.146+0000 7f6f527fc700 1 -- 172.21.15.121:0/606699058 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6f580068b0 con 0x7f6f5c066570 2023-11-15T02:58:52.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.146+0000 7f6f527fc700 1 --2- 172.21.15.121:0/606699058 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6f4005a510 0x7f6f4005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:52.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.150+0000 7f6f527fc700 1 -- 172.21.15.121:0/606699058 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f6f58073eb0 con 0x7f6f5c066570 2023-11-15T02:58:52.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.150+0000 7f6f61c76700 1 --2- 172.21.15.121:0/606699058 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6f4005a510 0x7f6f4005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:52.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.150+0000 7f6f61c76700 1 --2- 172.21.15.121:0/606699058 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6f4005a510 0x7f6f4005c9d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f6f5c0ac700 tx=0x7f6f4c015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:52.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.154+0000 7f6f527fc700 1 -- 172.21.15.121:0/606699058 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6f58047870 con 0x7f6f5c066570 2023-11-15T02:58:52.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:52.326+0000 7f6f63479700 1 -- 172.21.15.121:0/606699058 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6f5c068a70 con 0x7f6f5c066570 2023-11-15T02:58:53.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.122+0000 7f6f527fc700 1 -- 172.21.15.121:0/606699058 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v44) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6f5804ae90 con 0x7f6f5c066570 2023-11-15T02:58:53.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.130+0000 7f6f63479700 1 -- 172.21.15.121:0/606699058 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6f4005a510 msgr2=0x7f6f4005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:53.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.130+0000 7f6f63479700 1 --2- 172.21.15.121:0/606699058 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6f4005a510 0x7f6f4005c9d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f6f5c0ac700 tx=0x7f6f4c015000).stop 2023-11-15T02:58:53.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.130+0000 7f6f63479700 1 -- 172.21.15.121:0/606699058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f5c066570 msgr2=0x7f6f5c160cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:53.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.130+0000 7f6f63479700 1 --2- 172.21.15.121:0/606699058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f5c066570 0x7f6f5c160cc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f6f5800c440 tx=0x7f6f58007d40).stop 2023-11-15T02:58:53.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.130+0000 7f6f63479700 1 -- 172.21.15.121:0/606699058 shutdown_connections 2023-11-15T02:58:53.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.130+0000 7f6f63479700 1 --2- 172.21.15.121:0/606699058 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6f4005a510 0x7f6f4005c9d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.130+0000 7f6f63479700 1 --2- 172.21.15.121:0/606699058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f5c151ac0 0x7f6f5c0be0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.130+0000 7f6f63479700 1 --2- 172.21.15.121:0/606699058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f5c1511f0 0x7f6f5c0bdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.130+0000 7f6f63479700 1 --2- 172.21.15.121:0/606699058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f5c066570 0x7f6f5c160cc0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.130+0000 7f6f63479700 1 -- 172.21.15.121:0/606699058 >> 172.21.15.121:0/606699058 conn(0x7f6f5c067a60 msgr2=0x7f6f5c062190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:53.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.130+0000 7f6f63479700 1 -- 172.21.15.121:0/606699058 shutdown_connections 2023-11-15T02:58:53.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.130+0000 7f6f63479700 1 -- 172.21.15.121:0/606699058 wait complete. 2023-11-15T02:58:53.138 INFO:teuthology.orchestra.run.smithi121.stderr: removed all blocklist entries 2023-11-15T02:58:53.156 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-15T02:58:53.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.310+0000 7fdddb65b700 1 Processor -- start 2023-11-15T02:58:53.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.314+0000 7fdddb65b700 1 -- start start 2023-11-15T02:58:53.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.314+0000 7fdddb65b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fddd4152780 0x7fddd4159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:53.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.314+0000 7fdddb65b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddd40c3fa0 con 0x7fddd40a5510 2023-11-15T02:58:53.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fdddb65b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddd4159630 con 0x7fddd40c65d0 2023-11-15T02:58:53.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fdddb65b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd40c30b0 con 0x7fddd4152780 2023-11-15T02:58:53.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fddd9bf8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fddd40a5510 0x7fddd40a59a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49012/0 (socket says 172.21.15.121:49012) 2023-11-15T02:58:53.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fddd9bf8700 1 -- 172.21.15.121:0/3878398406 learned_addr learned my addr 172.21.15.121:0/3878398406 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:53.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fddd93f7700 1 --2- 172.21.15.121:0/3878398406 >> v2:172.21.15.121:3300/0 conn(0x7fddd4152780 0x7fddd4159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:53.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fddcbfff700 1 -- 172.21.15.121:0/3878398406 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244813516 0 0) 0x7fddd40c3fa0 con 0x7fddd40a5510 2023-11-15T02:58:53.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fddcbfff700 1 -- 172.21.15.121:0/3878398406 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddc00033f0 con 0x7fddd40a5510 2023-11-15T02:58:53.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fddd93f7700 1 -- 172.21.15.121:0/3878398406 >> v1:172.21.15.190:6789/0 conn(0x7fddd40c65d0 legacy=0x7fddd40c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:53.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fddd93f7700 1 -- 172.21.15.121:0/3878398406 >> v1:172.21.15.121:6789/0 conn(0x7fddd40a5510 legacy=0x7fddd40a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:53.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fddd93f7700 1 -- 172.21.15.121:0/3878398406 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd40c3390 con 0x7fddd4152780 2023-11-15T02:58:53.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fddd93f7700 1 --2- 172.21.15.121:0/3878398406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddd4152780 0x7fddd4159140 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fddd0009760 tx=0x7fddd0009060).ready entity=mon.0 client_cookie=6f9119577fead47a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:53.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fddcbfff700 1 -- 172.21.15.121:0/3878398406 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd000f510 con 0x7fddd4152780 2023-11-15T02:58:53.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fddcbfff700 1 -- 172.21.15.121:0/3878398406 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd000f670 con 0x7fddd4152780 2023-11-15T02:58:53.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fddcbfff700 1 -- 172.21.15.121:0/3878398406 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd000f9f0 con 0x7fddd4152780 2023-11-15T02:58:53.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fdddb65b700 1 -- 172.21.15.121:0/3878398406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddd4152780 msgr2=0x7fddd4159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:53.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fdddb65b700 1 --2- 172.21.15.121:0/3878398406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddd4152780 0x7fddd4159140 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fddd0009760 tx=0x7fddd0009060).stop 2023-11-15T02:58:53.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fdddb65b700 1 -- 172.21.15.121:0/3878398406 shutdown_connections 2023-11-15T02:58:53.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fdddb65b700 1 --2- 172.21.15.121:0/3878398406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddd4152780 0x7fddd4159140 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fdddb65b700 1 -- 172.21.15.121:0/3878398406 >> 172.21.15.121:0/3878398406 conn(0x7fddd4062190 msgr2=0x7fddd40625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:53.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fdddb65b700 1 -- 172.21.15.121:0/3878398406 shutdown_connections 2023-11-15T02:58:53.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.318+0000 7fdddb65b700 1 -- 172.21.15.121:0/3878398406 wait complete. 2023-11-15T02:58:53.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.322+0000 7fdddb65b700 1 Processor -- start 2023-11-15T02:58:53.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.322+0000 7fdddb65b700 1 -- start start 2023-11-15T02:58:53.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fdddb65b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddd40a5510 0x7fddd41402e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:53.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fdddb65b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddd40c65d0 0x7fddd4156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:53.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fdddb65b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddd4152780 0x7fddd4066050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:53.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fdddb65b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd416caa0 con 0x7fddd40a5510 2023-11-15T02:58:53.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fdddb65b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd4154d20 con 0x7fddd4152780 2023-11-15T02:58:53.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fdddb65b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd41543f0 con 0x7fddd40c65d0 2023-11-15T02:58:53.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fddd93f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddd40a5510 0x7fddd41402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:53.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fddd93f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddd40a5510 0x7fddd41402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39264/0 (socket says 172.21.15.121:39264) 2023-11-15T02:58:53.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fddd93f7700 1 -- 172.21.15.121:0/1610782303 learned_addr learned my addr 172.21.15.121:0/1610782303 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:53.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fddd8bf6700 1 --2- 172.21.15.121:0/1610782303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddd40c65d0 0x7fddd4156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:53.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fddd9bf8700 1 --2- 172.21.15.121:0/1610782303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddd4152780 0x7fddd4066050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:53.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fddd93f7700 1 -- 172.21.15.121:0/1610782303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddd40c65d0 msgr2=0x7fddd4156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:53.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fddd93f7700 1 --2- 172.21.15.121:0/1610782303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddd40c65d0 0x7fddd4156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fddd93f7700 1 -- 172.21.15.121:0/1610782303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddd4152780 msgr2=0x7fddd4066050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:53.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fddd93f7700 1 --2- 172.21.15.121:0/1610782303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddd4152780 0x7fddd4066050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fddd93f7700 1 -- 172.21.15.121:0/1610782303 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd00092a0 con 0x7fddd40a5510 2023-11-15T02:58:53.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fddd9bf8700 1 --2- 172.21.15.121:0/1610782303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddd4152780 0x7fddd4066050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:58:53.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fddd93f7700 1 --2- 172.21.15.121:0/1610782303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddd40a5510 0x7fddd41402e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7fddd0009e00 tx=0x7fddd0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:53.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fddc97fa700 1 -- 172.21.15.121:0/1610782303 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd000f7e0 con 0x7fddd40a5510 2023-11-15T02:58:53.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fddc97fa700 1 -- 172.21.15.121:0/1610782303 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd000f940 con 0x7fddd40a5510 2023-11-15T02:58:53.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fdddb65b700 1 -- 172.21.15.121:0/1610782303 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd41546d0 con 0x7fddd40a5510 2023-11-15T02:58:53.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fddc97fa700 1 -- 172.21.15.121:0/1610782303 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd0022650 con 0x7fddd40a5510 2023-11-15T02:58:53.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.326+0000 7fdddb65b700 1 -- 172.21.15.121:0/1610782303 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd4156f40 con 0x7fddd40a5510 2023-11-15T02:58:53.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.330+0000 7fdddb65b700 1 -- 172.21.15.121:0/1610782303 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd406c150 con 0x7fddd40a5510 2023-11-15T02:58:53.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.330+0000 7fddc97fa700 1 -- 172.21.15.121:0/1610782303 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fddd0022a00 con 0x7fddd40a5510 2023-11-15T02:58:53.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.330+0000 7fddc97fa700 1 --2- 172.21.15.121:0/1610782303 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fddb805a510 0x7fddb805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:53.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.330+0000 7fddc97fa700 1 -- 172.21.15.121:0/1610782303 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fddd0075b90 con 0x7fddd40a5510 2023-11-15T02:58:53.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.330+0000 7fddd8bf6700 1 --2- 172.21.15.121:0/1610782303 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fddb805a510 0x7fddb805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:53.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.334+0000 7fddd8bf6700 1 --2- 172.21.15.121:0/1610782303 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fddb805a510 0x7fddb805c9d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fddd4063080 tx=0x7fddc4009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:53.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.338+0000 7fddc97fa700 1 -- 172.21.15.121:0/1610782303 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fddd0049680 con 0x7fddd40a5510 2023-11-15T02:58:53.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.558+0000 7fdddb65b700 1 -- 172.21.15.121:0/1610782303 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fddd40c3670 con 0x7fddd40a5510 2023-11-15T02:58:53.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.558+0000 7fddc97fa700 1 -- 172.21.15.121:0/1610782303 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fddd0022cb0 con 0x7fddd40a5510 2023-11-15T02:58:53.565 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:58:53.565 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T02:58:53.566 INFO:teuthology.orchestra.run.smithi121.stdout: "auth_dump": [ 2023-11-15T02:58:53.566 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:53.566 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.a", 2023-11-15T02:58:53.566 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1Rlzy3zOhAAeJ3COoWQP/Kw+P/0AwHbwA==", 2023-11-15T02:58:53.566 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:58:53.567 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:58:53.567 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T02:58:53.567 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T02:58:53.567 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:58:53.568 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.568 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:58:53.568 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:53.568 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.b", 2023-11-15T02:58:53.568 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlUglaERAAC6WUS9ftCfr+KbPpiEIyqw==", 2023-11-15T02:58:53.569 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:58:53.569 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:58:53.569 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T02:58:53.569 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T02:58:53.569 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:58:53.570 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.570 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:58:53.570 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:53.570 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.c", 2023-11-15T02:58:53.570 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RldFcvCRAATIouBNyHGip5hFrH+z4v5A==", 2023-11-15T02:58:53.571 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:58:53.571 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:58:53.571 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T02:58:53.571 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T02:58:53.571 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:58:53.572 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.572 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:58:53.572 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:53.572 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.d", 2023-11-15T02:58:53.572 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1Rlg5iZGxAADwdL2RrtU0kDeb9EmNJU6A==", 2023-11-15T02:58:53.573 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:58:53.573 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:58:53.573 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T02:58:53.573 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T02:58:53.573 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:58:53.574 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.574 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:58:53.574 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:53.574 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.0", 2023-11-15T02:58:53.574 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAzM1RldPH9CBAAUCF/a47aFbbNsWV1r+u3Tw==", 2023-11-15T02:58:53.575 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:58:53.575 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:58:53.575 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:58:53.575 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:58:53.575 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.576 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:58:53.576 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:53.576 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.1", 2023-11-15T02:58:53.576 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQA2M1RltoxwMhAAZN3bgOjSaoK3ZxE5ezZ3IA==", 2023-11-15T02:58:53.576 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:58:53.577 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:58:53.577 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:58:53.577 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:58:53.577 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.577 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:58:53.577 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:53.578 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.2", 2023-11-15T02:58:53.578 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQA6M1RlgltjHxAAx/m5gm+qs8LiEFyZL+uLzQ==", 2023-11-15T02:58:53.578 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:58:53.578 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:58:53.578 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:58:53.579 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:58:53.579 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.579 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:58:53.579 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:53.579 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.3", 2023-11-15T02:58:53.580 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQA+M1RlhtQcDRAA+g11PmSNKazpk+VoK17EEQ==", 2023-11-15T02:58:53.580 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:58:53.580 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:58:53.580 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:58:53.580 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:58:53.581 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.581 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:58:53.581 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:53.581 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.4", 2023-11-15T02:58:53.581 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBFM1RlJ7hnBBAAjskKfD63N/61T4GSPqOiRg==", 2023-11-15T02:58:53.581 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:58:53.582 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:58:53.582 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:58:53.582 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:58:53.582 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.582 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:58:53.583 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:53.583 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.5", 2023-11-15T02:58:53.583 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBIM1RlI1HPLRAAgVbiNq1tmgK6PaWP+c2JFw==", 2023-11-15T02:58:53.583 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:58:53.583 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:58:53.584 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:58:53.584 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:58:53.584 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.584 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:58:53.584 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:53.585 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.6", 2023-11-15T02:58:53.585 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBMM1RlPDshDRAAVgIqxklHSQcFb5sM6GUfyQ==", 2023-11-15T02:58:53.585 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:58:53.585 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:58:53.585 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:58:53.586 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:58:53.586 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.586 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:58:53.586 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:53.586 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.7", 2023-11-15T02:58:53.587 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBPM1RlN8IuJxAA2rN2848kaB2eOM6zDb+eJg==", 2023-11-15T02:58:53.587 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:58:53.587 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:58:53.587 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:58:53.587 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:58:53.588 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.588 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:58:53.588 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:53.588 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.0", 2023-11-15T02:58:53.588 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlfHPEIRAAJ4YV46zzBQ7v7y0kD4P2ZA==", 2023-11-15T02:58:53.589 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:58:53.589 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:58:53.589 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow r", 2023-11-15T02:58:53.589 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-15T02:58:53.589 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.590 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:58:53.590 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:53.590 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.1", 2023-11-15T02:58:53.590 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlibaULBAA+y1EyhVOt+wXSk2uRajJ3g==", 2023-11-15T02:58:53.590 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:58:53.591 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:58:53.591 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2023-11-15T02:58:53.591 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2023-11-15T02:58:53.591 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2023-11-15T02:58:53.591 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.592 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:58:53.592 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:53.592 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.2", 2023-11-15T02:58:53.592 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlHHbvJhAAX5TE6UvcpBeJPSCZkfqD8w==", 2023-11-15T02:58:53.592 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:58:53.593 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:58:53.593 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2023-11-15T02:58:53.593 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2023-11-15T02:58:53.593 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2023-11-15T02:58:53.593 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.593 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:58:53.594 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:53.594 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.3", 2023-11-15T02:58:53.594 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlQtQXMhAAGAvtnvJLSoUKuEiZDsSxww==", 2023-11-15T02:58:53.594 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:58:53.594 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:58:53.595 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2023-11-15T02:58:53.595 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2023-11-15T02:58:53.595 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2023-11-15T02:58:53.595 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.595 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:58:53.596 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:53.596 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.admin", 2023-11-15T02:58:53.596 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1Rl67qqEhAAI48kZ9D7T807bTVoUmwajg==", 2023-11-15T02:58:53.596 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:58:53.596 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2023-11-15T02:58:53.596 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T02:58:53.597 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T02:58:53.597 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:58:53.597 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.597 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:58:53.597 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:53.598 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mgr.x", 2023-11-15T02:58:53.598 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1RlJQFPNRAAy7E5mrn8Xh98339fWSkX9Q==", 2023-11-15T02:58:53.598 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:58:53.598 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2023-11-15T02:58:53.598 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile mgr", 2023-11-15T02:58:53.599 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:58:53.599 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.599 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:58:53.599 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:53.599 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mgr.y", 2023-11-15T02:58:53.600 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1RldQJlLhAAbc46liEP03yBNeVhZCb7IQ==", 2023-11-15T02:58:53.600 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:58:53.600 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2023-11-15T02:58:53.600 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile mgr", 2023-11-15T02:58:53.600 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:58:53.600 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.601 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:53.601 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2023-11-15T02:58:53.601 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T02:58:53.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.562+0000 7fdddb65b700 1 -- 172.21.15.121:0/1610782303 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fddb805a510 msgr2=0x7fddb805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:53.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.562+0000 7fdddb65b700 1 --2- 172.21.15.121:0/1610782303 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fddb805a510 0x7fddb805c9d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fddd4063080 tx=0x7fddc4009040).stop 2023-11-15T02:58:53.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.562+0000 7fdddb65b700 1 -- 172.21.15.121:0/1610782303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddd40a5510 msgr2=0x7fddd41402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:53.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.562+0000 7fdddb65b700 1 --2- 172.21.15.121:0/1610782303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddd40a5510 0x7fddd41402e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7fddd0009e00 tx=0x7fddd0017000).stop 2023-11-15T02:58:53.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.562+0000 7fdddb65b700 1 -- 172.21.15.121:0/1610782303 shutdown_connections 2023-11-15T02:58:53.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.562+0000 7fdddb65b700 1 --2- 172.21.15.121:0/1610782303 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fddb805a510 0x7fddb805c9d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.562+0000 7fdddb65b700 1 --2- 172.21.15.121:0/1610782303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddd4152780 0x7fddd4066050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.562+0000 7fdddb65b700 1 --2- 172.21.15.121:0/1610782303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddd40c65d0 0x7fddd4156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.562+0000 7fdddb65b700 1 --2- 172.21.15.121:0/1610782303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddd40a5510 0x7fddd41402e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.562+0000 7fdddb65b700 1 -- 172.21.15.121:0/1610782303 >> 172.21.15.121:0/1610782303 conn(0x7fddd4062190 msgr2=0x7fddd40a8980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:53.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.562+0000 7fdddb65b700 1 -- 172.21.15.121:0/1610782303 shutdown_connections 2023-11-15T02:58:53.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.562+0000 7fdddb65b700 1 -- 172.21.15.121:0/1610782303 wait complete. 2023-11-15T02:58:53.606 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:58:53.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.770+0000 7f19b759e700 1 Processor -- start 2023-11-15T02:58:53.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.770+0000 7f19b759e700 1 -- start start 2023-11-15T02:58:53.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.770+0000 7f19b759e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f19b8067a60 0x7f19b8067ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:53.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.770+0000 7f19b759e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b80c4c90 con 0x7f19b80a6b60 2023-11-15T02:58:53.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.770+0000 7f19b759e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b80683b0 con 0x7f19b80bc8f0 2023-11-15T02:58:53.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.770+0000 7f19b759e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19b80c72d0 con 0x7f19b8067a60 2023-11-15T02:58:53.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.770+0000 7f19b659c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f19b80a6b60 0x7f19b80c6bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49028/0 (socket says 172.21.15.121:49028) 2023-11-15T02:58:53.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.770+0000 7f19b659c700 1 -- 172.21.15.121:0/3563384526 learned_addr learned my addr 172.21.15.121:0/3563384526 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:53.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b5d9b700 1 --2- 172.21.15.121:0/3563384526 >> v2:172.21.15.190:3300/0 conn(0x7f19b8067a60 0x7f19b8067ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:53.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b559a700 1 -- 172.21.15.121:0/3563384526 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378225573 0 0) 0x7f19b80c4c90 con 0x7f19b80a6b60 2023-11-15T02:58:53.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b559a700 1 -- 172.21.15.121:0/3563384526 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19a00033f0 con 0x7f19b80a6b60 2023-11-15T02:58:53.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b559a700 1 -- 172.21.15.121:0/3563384526 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2528347575 0 0) 0x7f19a00033f0 con 0x7f19b80a6b60 2023-11-15T02:58:53.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b559a700 1 -- 172.21.15.121:0/3563384526 >> v2:172.21.15.190:3300/0 conn(0x7f19b8067a60 msgr2=0x7f19b8067ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:53.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b559a700 1 --2- 172.21.15.121:0/3563384526 >> v2:172.21.15.190:3300/0 conn(0x7f19b8067a60 0x7f19b8067ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b559a700 1 -- 172.21.15.121:0/3563384526 >> v1:172.21.15.190:6789/0 conn(0x7f19b80bc8f0 legacy=0x7f19b80bcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:53.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b559a700 1 -- 172.21.15.121:0/3563384526 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b80c4360 con 0x7f19b80a6b60 2023-11-15T02:58:53.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b559a700 1 -- 172.21.15.121:0/3563384526 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f19a8001470 con 0x7f19b80a6b60 2023-11-15T02:58:53.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b559a700 1 -- 172.21.15.121:0/3563384526 >> v1:172.21.15.121:6789/0 conn(0x7f19b80a6b60 legacy=0x7f19b80c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b559a700 1 --2- 172.21.15.121:0/3563384526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19a00038e0 0x7f19a0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b559a700 1 --2- 172.21.15.121:0/3563384526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19a0004690 0x7f19a0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b559a700 1 --2- 172.21.15.121:0/3563384526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19a0005000 0x7f19a000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b559a700 1 -- 172.21.15.121:0/3563384526 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b80c72d0 con 0x7f19a00038e0 2023-11-15T02:58:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b559a700 1 -- 172.21.15.121:0/3563384526 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a000bf60 con 0x7f19a0005000 2023-11-15T02:58:53.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b559a700 1 -- 172.21.15.121:0/3563384526 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a000c0f0 con 0x7f19a0004690 2023-11-15T02:58:53.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b659c700 1 --2- 172.21.15.121:0/3563384526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19a00038e0 0x7f19a0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:53.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b5d9b700 1 --2- 172.21.15.121:0/3563384526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19a0004690 0x7f19a0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:53.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b6d9d700 1 --2- 172.21.15.121:0/3563384526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19a0005000 0x7f19a000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:53.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b659c700 1 -- 172.21.15.121:0/3563384526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19a0004690 msgr2=0x7f19a0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b659c700 1 --2- 172.21.15.121:0/3563384526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19a0004690 0x7f19a0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b659c700 1 -- 172.21.15.121:0/3563384526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19a0005000 msgr2=0x7f19a000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b659c700 1 --2- 172.21.15.121:0/3563384526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19a0005000 0x7f19a000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b659c700 1 -- 172.21.15.121:0/3563384526 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a00033f0 con 0x7f19a00038e0 2023-11-15T02:58:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b5d9b700 1 --2- 172.21.15.121:0/3563384526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19a0004690 0x7f19a0004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:53.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b6d9d700 1 -- 172.21.15.121:0/3563384526 reap_dead start 2023-11-15T02:58:53.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b659c700 1 --2- 172.21.15.121:0/3563384526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19a00038e0 0x7f19a0003d00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f19a800c190 tx=0x7f19a8001e50).ready entity=mon.0 client_cookie=11d086755b155264 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:53.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b559a700 1 -- 172.21.15.121:0/3563384526 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a8007a20 con 0x7f19a00038e0 2023-11-15T02:58:53.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b559a700 1 -- 172.21.15.121:0/3563384526 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a800cac0 con 0x7f19a00038e0 2023-11-15T02:58:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b559a700 1 -- 172.21.15.121:0/3563384526 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a800ce40 con 0x7f19a00038e0 2023-11-15T02:58:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b759e700 1 -- 172.21.15.121:0/3563384526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19a00038e0 msgr2=0x7f19a0003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b759e700 1 --2- 172.21.15.121:0/3563384526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19a00038e0 0x7f19a0003d00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f19a800c190 tx=0x7f19a8001e50).stop 2023-11-15T02:58:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b759e700 1 -- 172.21.15.121:0/3563384526 shutdown_connections 2023-11-15T02:58:53.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b759e700 1 --2- 172.21.15.121:0/3563384526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19a00038e0 0x7f19a0003d00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b759e700 1 -- 172.21.15.121:0/3563384526 >> 172.21.15.121:0/3563384526 conn(0x7f19b8069350 msgr2=0x7f19b8090a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:53.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b759e700 1 -- 172.21.15.121:0/3563384526 shutdown_connections 2023-11-15T02:58:53.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.774+0000 7f19b759e700 1 -- 172.21.15.121:0/3563384526 wait complete. 2023-11-15T02:58:53.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.778+0000 7f19b759e700 1 Processor -- start 2023-11-15T02:58:53.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.778+0000 7f19b759e700 1 -- start start 2023-11-15T02:58:53.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.778+0000 7f19b759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19a00038e0 0x7f19b8163d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:53.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.778+0000 7f19b759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19b8164210 0x7f19b81525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:53.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.778+0000 7f19b759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19b8152ac0 0x7f19b80bf140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:53.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.778+0000 7f19b759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b8168a30 con 0x7f19a00038e0 2023-11-15T02:58:53.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.778+0000 7f19b759e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b8152f00 con 0x7f19b8152ac0 2023-11-15T02:58:53.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.778+0000 7f19b759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b8167db0 con 0x7f19b8164210 2023-11-15T02:58:53.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.782+0000 7f19b659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19a00038e0 0x7f19b8163d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:53.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.782+0000 7f19b659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19a00038e0 0x7f19b8163d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39296/0 (socket says 172.21.15.121:39296) 2023-11-15T02:58:53.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.782+0000 7f19b659c700 1 -- 172.21.15.121:0/2482470427 learned_addr learned my addr 172.21.15.121:0/2482470427 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:53.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.782+0000 7f19b5d9b700 1 --2- 172.21.15.121:0/2482470427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19b8164210 0x7f19b81525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:53.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.782+0000 7f19b6d9d700 1 --2- 172.21.15.121:0/2482470427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19b8152ac0 0x7f19b80bf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:53.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.782+0000 7f19b659c700 1 -- 172.21.15.121:0/2482470427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19b8164210 msgr2=0x7f19b81525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:53.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.782+0000 7f19b659c700 1 --2- 172.21.15.121:0/2482470427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19b8164210 0x7f19b81525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.782+0000 7f19b659c700 1 -- 172.21.15.121:0/2482470427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19b8152ac0 msgr2=0x7f19b80bf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:53.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.782+0000 7f19b659c700 1 --2- 172.21.15.121:0/2482470427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19b8152ac0 0x7f19b80bf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.782+0000 7f19b659c700 1 -- 172.21.15.121:0/2482470427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b8168090 con 0x7f19a00038e0 2023-11-15T02:58:53.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.782+0000 7f19b659c700 1 --2- 172.21.15.121:0/2482470427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19a00038e0 0x7f19b8163d20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f19a0005640 tx=0x7f19a8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:53.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.782+0000 7f19a67fc700 1 -- 172.21.15.121:0/2482470427 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a80073b0 con 0x7f19a00038e0 2023-11-15T02:58:53.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.782+0000 7f19a67fc700 1 -- 172.21.15.121:0/2482470427 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a8007510 con 0x7f19a00038e0 2023-11-15T02:58:53.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.782+0000 7f19a67fc700 1 -- 172.21.15.121:0/2482470427 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a80048e0 con 0x7f19a00038e0 2023-11-15T02:58:53.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.782+0000 7f19b759e700 1 -- 172.21.15.121:0/2482470427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b8168370 con 0x7f19a00038e0 2023-11-15T02:58:53.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.782+0000 7f19b759e700 1 -- 172.21.15.121:0/2482470427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b80bf630 con 0x7f19a00038e0 2023-11-15T02:58:53.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.782+0000 7f19b759e700 1 -- 172.21.15.121:0/2482470427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b8168b70 con 0x7f19a00038e0 2023-11-15T02:58:53.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.786+0000 7f19a67fc700 1 -- 172.21.15.121:0/2482470427 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f19a801c070 con 0x7f19a00038e0 2023-11-15T02:58:53.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.786+0000 7f19a67fc700 1 --2- 172.21.15.121:0/2482470427 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f199805a510 0x7f199805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:53.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.786+0000 7f19a67fc700 1 -- 172.21.15.121:0/2482470427 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f19a8074a30 con 0x7f19a00038e0 2023-11-15T02:58:53.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.786+0000 7f19b5d9b700 1 --2- 172.21.15.121:0/2482470427 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f199805a510 0x7f199805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:53.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.786+0000 7f19b5d9b700 1 --2- 172.21.15.121:0/2482470427 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f199805a510 0x7f199805c9d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f19b80c4820 tx=0x7f19ac008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:53.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.794+0000 7f19a67fc700 1 -- 172.21.15.121:0/2482470427 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f19b8168b70 con 0x7f19a00038e0 2023-11-15T02:58:53.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.958+0000 7f19b759e700 1 -- 172.21.15.121:0/2482470427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19b8168b70 con 0x7f19a00038e0 2023-11-15T02:58:53.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.962+0000 7f19a67fc700 1 -- 172.21.15.121:0/2482470427 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+13760 (secure 0 0 0) 0x7f19b8168b70 con 0x7f19a00038e0 2023-11-15T02:58:53.967 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:58:53.967 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":44,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:58:53.118766+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:58:53.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.962+0000 7f19b759e700 1 -- 172.21.15.121:0/2482470427 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f199805a510 msgr2=0x7f199805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:53.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.962+0000 7f19b759e700 1 --2- 172.21.15.121:0/2482470427 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f199805a510 0x7f199805c9d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f19b80c4820 tx=0x7f19ac008040).stop 2023-11-15T02:58:53.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.962+0000 7f19b759e700 1 -- 172.21.15.121:0/2482470427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19a00038e0 msgr2=0x7f19b8163d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:53.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.962+0000 7f19b759e700 1 --2- 172.21.15.121:0/2482470427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19a00038e0 0x7f19b8163d20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f19a0005640 tx=0x7f19a8015000).stop 2023-11-15T02:58:53.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.962+0000 7f19b759e700 1 -- 172.21.15.121:0/2482470427 shutdown_connections 2023-11-15T02:58:53.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.962+0000 7f19b759e700 1 --2- 172.21.15.121:0/2482470427 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f199805a510 0x7f199805c9d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.962+0000 7f19b759e700 1 --2- 172.21.15.121:0/2482470427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19b8152ac0 0x7f19b80bf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.962+0000 7f19b759e700 1 --2- 172.21.15.121:0/2482470427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19b8164210 0x7f19b81525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.962+0000 7f19b759e700 1 --2- 172.21.15.121:0/2482470427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19a00038e0 0x7f19b8163d20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:53.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.962+0000 7f19b759e700 1 -- 172.21.15.121:0/2482470427 >> 172.21.15.121:0/2482470427 conn(0x7f19b8069350 msgr2=0x7f19b80c54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:53.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.962+0000 7f19b759e700 1 -- 172.21.15.121:0/2482470427 shutdown_connections 2023-11-15T02:58:53.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:53.962+0000 7f19b759e700 1 -- 172.21.15.121:0/2482470427 wait complete. 2023-11-15T02:58:53.988 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-15T02:58:53.989 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T02:58:54.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.146+0000 7eff2cc39700 1 Processor -- start 2023-11-15T02:58:54.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.146+0000 7eff2cc39700 1 -- start start 2023-11-15T02:58:54.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.146+0000 7eff2cc39700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7eff281521a0 0x7eff280bf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:54.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.146+0000 7eff2cc39700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7eff28152580 0x7eff281529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:54.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.146+0000 7eff2cc39700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff28152ed0 con 0x7eff280c4e70 2023-11-15T02:58:54.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.146+0000 7eff2cc39700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff280c4b40 con 0x7eff28152580 2023-11-15T02:58:54.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.146+0000 7eff2cc39700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff280bebf0 con 0x7eff281521a0 2023-11-15T02:58:54.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff26ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7eff28152580 0x7eff281529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:54.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff27fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7eff280c4e70 0x7eff280c52c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49042/0 (socket says 172.21.15.121:49042) 2023-11-15T02:58:54.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff27fff700 1 -- 172.21.15.121:0/2128384607 learned_addr learned my addr 172.21.15.121:0/2128384607 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:54.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff277fe700 1 --2- 172.21.15.121:0/2128384607 >> v2:172.21.15.190:3300/0 conn(0x7eff281521a0 0x7eff280bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:54.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff26ffd700 1 -- 172.21.15.121:0/2128384607 >> v2:172.21.15.190:3300/0 conn(0x7eff281521a0 msgr2=0x7eff280bf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:54.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff26ffd700 1 --2- 172.21.15.121:0/2128384607 >> v2:172.21.15.190:3300/0 conn(0x7eff281521a0 0x7eff280bf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff26ffd700 1 -- 172.21.15.121:0/2128384607 >> v1:172.21.15.121:6789/0 conn(0x7eff280c4e70 legacy=0x7eff280c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:54.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff26ffd700 1 -- 172.21.15.121:0/2128384607 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff280bdfe0 con 0x7eff28152580 2023-11-15T02:58:54.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff26ffd700 1 --2- 172.21.15.121:0/2128384607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff28152580 0x7eff281529e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7eff280654a0 tx=0x7eff1c009060).ready entity=mon.0 client_cookie=ad1e0c886b1e035 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:54.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff267fc700 1 -- 172.21.15.121:0/2128384607 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c00f510 con 0x7eff28152580 2023-11-15T02:58:54.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff267fc700 1 -- 172.21.15.121:0/2128384607 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1c00f670 con 0x7eff28152580 2023-11-15T02:58:54.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff267fc700 1 -- 172.21.15.121:0/2128384607 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c00f9f0 con 0x7eff28152580 2023-11-15T02:58:54.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff2cc39700 1 -- 172.21.15.121:0/2128384607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff28152580 msgr2=0x7eff281529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:54.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff2cc39700 1 --2- 172.21.15.121:0/2128384607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff28152580 0x7eff281529e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7eff280654a0 tx=0x7eff1c009060).stop 2023-11-15T02:58:54.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff2cc39700 1 -- 172.21.15.121:0/2128384607 shutdown_connections 2023-11-15T02:58:54.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff2cc39700 1 --2- 172.21.15.121:0/2128384607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff28152580 0x7eff281529e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff2cc39700 1 --2- 172.21.15.121:0/2128384607 >> v2:172.21.15.190:3300/0 conn(0x7eff281521a0 0x7eff280bf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff2cc39700 1 -- 172.21.15.121:0/2128384607 >> 172.21.15.121:0/2128384607 conn(0x7eff28067a60 msgr2=0x7eff28067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:54.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff2cc39700 1 -- 172.21.15.121:0/2128384607 shutdown_connections 2023-11-15T02:58:54.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.150+0000 7eff2cc39700 1 -- 172.21.15.121:0/2128384607 wait complete. 2023-11-15T02:58:54.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.154+0000 7eff2cc39700 1 Processor -- start 2023-11-15T02:58:54.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.154+0000 7eff2cc39700 1 -- start start 2023-11-15T02:58:54.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff2cc39700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff280c4e70 0x7eff28161780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:54.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff2cc39700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff281521a0 0x7eff280a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:54.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff2cc39700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff28152580 0x7eff280a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:54.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff2cc39700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff280a1fc0 con 0x7eff280c4e70 2023-11-15T02:58:54.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff2cc39700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2809fec0 con 0x7eff28152580 2023-11-15T02:58:54.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff2cc39700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff280adf70 con 0x7eff281521a0 2023-11-15T02:58:54.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff277fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff280c4e70 0x7eff28161780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:54.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff277fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff280c4e70 0x7eff28161780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39314/0 (socket says 172.21.15.121:39314) 2023-11-15T02:58:54.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff277fe700 1 -- 172.21.15.121:0/844185151 learned_addr learned my addr 172.21.15.121:0/844185151 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:54.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff277fe700 1 -- 172.21.15.121:0/844185151 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff281521a0 msgr2=0x7eff280a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:54.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff26ffd700 1 --2- 172.21.15.121:0/844185151 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff281521a0 0x7eff280a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:54.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff277fe700 1 --2- 172.21.15.121:0/844185151 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff281521a0 0x7eff280a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff27fff700 1 --2- 172.21.15.121:0/844185151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff28152580 0x7eff280a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:54.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff277fe700 1 -- 172.21.15.121:0/844185151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff28152580 msgr2=0x7eff280a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:54.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff277fe700 1 --2- 172.21.15.121:0/844185151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff28152580 0x7eff280a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff277fe700 1 -- 172.21.15.121:0/844185151 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1c0092a0 con 0x7eff280c4e70 2023-11-15T02:58:54.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff26ffd700 1 --2- 172.21.15.121:0/844185151 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff281521a0 0x7eff280a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:58:54.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff27fff700 1 --2- 172.21.15.121:0/844185151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff28152580 0x7eff280a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:58:54.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff277fe700 1 --2- 172.21.15.121:0/844185151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff280c4e70 0x7eff28161780 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7eff1800e960 tx=0x7eff18009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:54.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff0f7fe700 1 -- 172.21.15.121:0/844185151 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1800c850 con 0x7eff280c4e70 2023-11-15T02:58:54.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff0f7fe700 1 -- 172.21.15.121:0/844185151 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1800c9b0 con 0x7eff280c4e70 2023-11-15T02:58:54.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff2cc39700 1 -- 172.21.15.121:0/844185151 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2809f590 con 0x7eff280c4e70 2023-11-15T02:58:54.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff0f7fe700 1 -- 172.21.15.121:0/844185151 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1800cd30 con 0x7eff280c4e70 2023-11-15T02:58:54.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.158+0000 7eff2cc39700 1 -- 172.21.15.121:0/844185151 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2809fb50 con 0x7eff280c4e70 2023-11-15T02:58:54.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.162+0000 7eff2cc39700 1 -- 172.21.15.121:0/844185151 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff2806bcf0 con 0x7eff280c4e70 2023-11-15T02:58:54.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.162+0000 7eff0f7fe700 1 -- 172.21.15.121:0/844185151 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7eff18012600 con 0x7eff280c4e70 2023-11-15T02:58:54.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.162+0000 7eff0f7fe700 1 --2- 172.21.15.121:0/844185151 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7eff1405a510 0x7eff1405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:54.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.162+0000 7eff0f7fe700 1 -- 172.21.15.121:0/844185151 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5269+0+0 (secure 0 0 0) 0x7eff18074a00 con 0x7eff280c4e70 2023-11-15T02:58:54.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.162+0000 7eff26ffd700 1 --2- 172.21.15.121:0/844185151 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7eff1405a510 0x7eff1405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:54.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.166+0000 7eff26ffd700 1 --2- 172.21.15.121:0/844185151 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7eff1405a510 0x7eff1405c9d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7eff28150b10 tx=0x7eff1c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:54.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.166+0000 7eff0f7fe700 1 -- 172.21.15.121:0/844185151 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7eff18048530 con 0x7eff280c4e70 2023-11-15T02:58:54.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.326+0000 7eff2cc39700 1 -- 172.21.15.121:0/844185151 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7eff280c5790 con 0x7eff280c4e70 2023-11-15T02:58:54.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.330+0000 7eff0f7fe700 1 -- 172.21.15.121:0/844185151 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v44) v1 ==== 110+0+10 (secure 0 0 0) 0x7eff1804bb50 con 0x7eff280c4e70 2023-11-15T02:58:54.335 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T02:58:54.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.330+0000 7eff2cc39700 1 -- 172.21.15.121:0/844185151 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7eff1405a510 msgr2=0x7eff1405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:54.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.330+0000 7eff2cc39700 1 --2- 172.21.15.121:0/844185151 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7eff1405a510 0x7eff1405c9d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7eff28150b10 tx=0x7eff1c017000).stop 2023-11-15T02:58:54.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.330+0000 7eff2cc39700 1 -- 172.21.15.121:0/844185151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff280c4e70 msgr2=0x7eff28161780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:54.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.330+0000 7eff2cc39700 1 --2- 172.21.15.121:0/844185151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff280c4e70 0x7eff28161780 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7eff1800e960 tx=0x7eff18009060).stop 2023-11-15T02:58:54.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.330+0000 7eff2cc39700 1 -- 172.21.15.121:0/844185151 shutdown_connections 2023-11-15T02:58:54.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.330+0000 7eff2cc39700 1 --2- 172.21.15.121:0/844185151 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7eff1405a510 0x7eff1405c9d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.330+0000 7eff2cc39700 1 --2- 172.21.15.121:0/844185151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff28152580 0x7eff280a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.330+0000 7eff2cc39700 1 --2- 172.21.15.121:0/844185151 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff281521a0 0x7eff280a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.330+0000 7eff2cc39700 1 --2- 172.21.15.121:0/844185151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff280c4e70 0x7eff28161780 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.330+0000 7eff2cc39700 1 -- 172.21.15.121:0/844185151 >> 172.21.15.121:0/844185151 conn(0x7eff28067a60 msgr2=0x7eff280a8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:54.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.330+0000 7eff2cc39700 1 -- 172.21.15.121:0/844185151 shutdown_connections 2023-11-15T02:58:54.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.330+0000 7eff2cc39700 1 -- 172.21.15.121:0/844185151 wait complete. 2023-11-15T02:58:54.360 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T02:58:54.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.514+0000 7ffb8069b700 1 Processor -- start 2023-11-15T02:58:54.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.514+0000 7ffb8069b700 1 -- start start 2023-11-15T02:58:54.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.518+0000 7ffb8069b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb78067ec0 0x7ffb780682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:54.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.518+0000 7ffb8069b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb780c0c20 con 0x7ffb78153700 2023-11-15T02:58:54.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.518+0000 7ffb8069b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb78158730 con 0x7ffb780c10b0 2023-11-15T02:58:54.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.518+0000 7ffb8069b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780c02f0 con 0x7ffb78067ec0 2023-11-15T02:58:54.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.518+0000 7ffb7ec38700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ffb78153700 0x7ffb78153bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49044/0 (socket says 172.21.15.121:49044) 2023-11-15T02:58:54.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.518+0000 7ffb7ec38700 1 -- 172.21.15.121:0/2107318096 learned_addr learned my addr 172.21.15.121:0/2107318096 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:54.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.518+0000 7ffb7e437700 1 --2- 172.21.15.121:0/2107318096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb78067ec0 0x7ffb780682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:54.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.518+0000 7ffb7d435700 1 -- 172.21.15.121:0/2107318096 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 933146148 0 0) 0x7ffb780c0c20 con 0x7ffb78153700 2023-11-15T02:58:54.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.518+0000 7ffb7d435700 1 -- 172.21.15.121:0/2107318096 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb640033f0 con 0x7ffb78153700 2023-11-15T02:58:54.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.518+0000 7ffb7d435700 1 -- 172.21.15.121:0/2107318096 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 236460331 0 0) 0x7ffb78158730 con 0x7ffb780c10b0 2023-11-15T02:58:54.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.518+0000 7ffb7e437700 1 -- 172.21.15.121:0/2107318096 >> v1:172.21.15.190:6789/0 conn(0x7ffb780c10b0 legacy=0x7ffb78062c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:54.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.518+0000 7ffb7e437700 1 -- 172.21.15.121:0/2107318096 >> v1:172.21.15.121:6789/0 conn(0x7ffb78153700 legacy=0x7ffb78153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:54.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.518+0000 7ffb7e437700 1 -- 172.21.15.121:0/2107318096 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb780c05d0 con 0x7ffb78067ec0 2023-11-15T02:58:54.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.522+0000 7ffb7e437700 1 --2- 172.21.15.121:0/2107318096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb78067ec0 0x7ffb780682a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7ffb74009790 tx=0x7ffb74009060).ready entity=mon.2 client_cookie=b39cea87b76228be server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:54.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.522+0000 7ffb7d435700 1 -- 172.21.15.121:0/2107318096 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7400f510 con 0x7ffb78067ec0 2023-11-15T02:58:54.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.522+0000 7ffb7d435700 1 -- 172.21.15.121:0/2107318096 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7400f670 con 0x7ffb78067ec0 2023-11-15T02:58:54.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.522+0000 7ffb7d435700 1 -- 172.21.15.121:0/2107318096 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7400f9f0 con 0x7ffb78067ec0 2023-11-15T02:58:54.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.522+0000 7ffb8069b700 1 -- 172.21.15.121:0/2107318096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb78067ec0 msgr2=0x7ffb780682a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:54.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.522+0000 7ffb8069b700 1 --2- 172.21.15.121:0/2107318096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb78067ec0 0x7ffb780682a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7ffb74009790 tx=0x7ffb74009060).stop 2023-11-15T02:58:54.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.522+0000 7ffb8069b700 1 -- 172.21.15.121:0/2107318096 shutdown_connections 2023-11-15T02:58:54.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.522+0000 7ffb8069b700 1 --2- 172.21.15.121:0/2107318096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb78067ec0 0x7ffb780682a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.522+0000 7ffb8069b700 1 -- 172.21.15.121:0/2107318096 >> 172.21.15.121:0/2107318096 conn(0x7ffb780cad40 msgr2=0x7ffb780cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:54.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.522+0000 7ffb8069b700 1 -- 172.21.15.121:0/2107318096 shutdown_connections 2023-11-15T02:58:54.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.522+0000 7ffb8069b700 1 -- 172.21.15.121:0/2107318096 wait complete. 2023-11-15T02:58:54.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.522+0000 7ffb8069b700 1 Processor -- start 2023-11-15T02:58:54.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb8069b700 1 -- start start 2023-11-15T02:58:54.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb8069b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb78067ec0 0x7ffb7813c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:54.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb8069b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb780c10b0 0x7ffb78158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:54.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb8069b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb78153700 0x7ffb78158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:54.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb8069b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb78159300 con 0x7ffb780c10b0 2023-11-15T02:58:54.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb8069b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb78167fa0 con 0x7ffb78067ec0 2023-11-15T02:58:54.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb8069b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb78167670 con 0x7ffb78153700 2023-11-15T02:58:54.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb7dc36700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb780c10b0 0x7ffb78158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:54.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb7dc36700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb780c10b0 0x7ffb78158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39324/0 (socket says 172.21.15.121:39324) 2023-11-15T02:58:54.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb7dc36700 1 -- 172.21.15.121:0/2876448573 learned_addr learned my addr 172.21.15.121:0/2876448573 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:54.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb7e437700 1 --2- 172.21.15.121:0/2876448573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb78067ec0 0x7ffb7813c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:54.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb7ec38700 1 --2- 172.21.15.121:0/2876448573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb78153700 0x7ffb78158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:54.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb7dc36700 1 -- 172.21.15.121:0/2876448573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb78153700 msgr2=0x7ffb78158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:54.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb7dc36700 1 --2- 172.21.15.121:0/2876448573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb78153700 0x7ffb78158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb7dc36700 1 -- 172.21.15.121:0/2876448573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb78067ec0 msgr2=0x7ffb7813c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:54.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb7dc36700 1 --2- 172.21.15.121:0/2876448573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb78067ec0 0x7ffb7813c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb7dc36700 1 -- 172.21.15.121:0/2876448573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb740092a0 con 0x7ffb780c10b0 2023-11-15T02:58:54.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb7dc36700 1 --2- 172.21.15.121:0/2876448573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb780c10b0 0x7ffb78158920 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7ffb68001eb0 tx=0x7ffb68007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:54.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb6e7fc700 1 -- 172.21.15.121:0/2876448573 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6800f470 con 0x7ffb780c10b0 2023-11-15T02:58:54.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb6e7fc700 1 -- 172.21.15.121:0/2876448573 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6800f5f0 con 0x7ffb780c10b0 2023-11-15T02:58:54.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb8069b700 1 -- 172.21.15.121:0/2876448573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb78167950 con 0x7ffb780c10b0 2023-11-15T02:58:54.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb6e7fc700 1 -- 172.21.15.121:0/2876448573 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6800f9b0 con 0x7ffb780c10b0 2023-11-15T02:58:54.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.526+0000 7ffb8069b700 1 -- 172.21.15.121:0/2876448573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb7813c6f0 con 0x7ffb780c10b0 2023-11-15T02:58:54.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.530+0000 7ffb8069b700 1 -- 172.21.15.121:0/2876448573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb78047060 con 0x7ffb780c10b0 2023-11-15T02:58:54.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.530+0000 7ffb6e7fc700 1 -- 172.21.15.121:0/2876448573 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ffb6800fd20 con 0x7ffb780c10b0 2023-11-15T02:58:54.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.530+0000 7ffb6e7fc700 1 --2- 172.21.15.121:0/2876448573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffb5c05a510 0x7ffb5c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:54.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.530+0000 7ffb6e7fc700 1 -- 172.21.15.121:0/2876448573 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5269+0+0 (secure 0 0 0) 0x7ffb68073840 con 0x7ffb780c10b0 2023-11-15T02:58:54.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.530+0000 7ffb7e437700 1 --2- 172.21.15.121:0/2876448573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffb5c05a510 0x7ffb5c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:54.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.534+0000 7ffb7e437700 1 --2- 172.21.15.121:0/2876448573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffb5c05a510 0x7ffb5c05c9d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7ffb740096e0 tx=0x7ffb74017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:54.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.534+0000 7ffb6e7fc700 1 -- 172.21.15.121:0/2876448573 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ffb68047420 con 0x7ffb780c10b0 2023-11-15T02:58:54.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.694+0000 7ffb8069b700 1 -- 172.21.15.121:0/2876448573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ffb780bffc0 con 0x7ffb780c10b0 2023-11-15T02:58:54.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.698+0000 7ffb6e7fc700 1 -- 172.21.15.121:0/2876448573 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v44) v1 ==== 92+0+10 (secure 0 0 0) 0x7ffb6804aa40 con 0x7ffb780c10b0 2023-11-15T02:58:54.703 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T02:58:54.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.698+0000 7ffb8069b700 1 -- 172.21.15.121:0/2876448573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffb5c05a510 msgr2=0x7ffb5c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:54.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.698+0000 7ffb8069b700 1 --2- 172.21.15.121:0/2876448573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffb5c05a510 0x7ffb5c05c9d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7ffb740096e0 tx=0x7ffb74017000).stop 2023-11-15T02:58:54.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.698+0000 7ffb8069b700 1 -- 172.21.15.121:0/2876448573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb780c10b0 msgr2=0x7ffb78158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:54.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.698+0000 7ffb8069b700 1 --2- 172.21.15.121:0/2876448573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb780c10b0 0x7ffb78158920 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7ffb68001eb0 tx=0x7ffb68007c40).stop 2023-11-15T02:58:54.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.698+0000 7ffb8069b700 1 -- 172.21.15.121:0/2876448573 shutdown_connections 2023-11-15T02:58:54.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.698+0000 7ffb8069b700 1 --2- 172.21.15.121:0/2876448573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffb5c05a510 0x7ffb5c05c9d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.698+0000 7ffb8069b700 1 --2- 172.21.15.121:0/2876448573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb78153700 0x7ffb78158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.698+0000 7ffb8069b700 1 --2- 172.21.15.121:0/2876448573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb780c10b0 0x7ffb78158920 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.698+0000 7ffb8069b700 1 --2- 172.21.15.121:0/2876448573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb78067ec0 0x7ffb7813c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.698+0000 7ffb8069b700 1 -- 172.21.15.121:0/2876448573 >> 172.21.15.121:0/2876448573 conn(0x7ffb780cad40 msgr2=0x7ffb78066100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:54.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.698+0000 7ffb8069b700 1 -- 172.21.15.121:0/2876448573 shutdown_connections 2023-11-15T02:58:54.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.698+0000 7ffb8069b700 1 -- 172.21.15.121:0/2876448573 wait complete. 2023-11-15T02:58:54.723 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-15T02:58:54.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.878+0000 7f94eb29b700 1 Processor -- start 2023-11-15T02:58:54.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.882+0000 7f94eb29b700 1 -- start start 2023-11-15T02:58:54.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.882+0000 7f94eb29b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f94e40aa240 0x7f94e4067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:54.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.882+0000 7f94eb29b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94e4067f50 0x7f94e4065d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:54.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.882+0000 7f94eb29b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e40cb5c0 con 0x7f94e4153bb0 2023-11-15T02:58:54.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.882+0000 7f94eb29b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e40aa6f0 con 0x7f94e40aa240 2023-11-15T02:58:54.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.882+0000 7f94eb29b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40c5900 con 0x7f94e4067f50 2023-11-15T02:58:54.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.882+0000 7f94e8836700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f94e40aa240 0x7f94e4067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:54.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.882+0000 7f94e9838700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94e4067f50 0x7f94e4065d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:54.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.882+0000 7f94e8836700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f94e40aa240 0x7f94e4067a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:56070/0 (socket says 172.21.15.121:56070) 2023-11-15T02:58:54.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.882+0000 7f94e8836700 1 -- 172.21.15.121:0/4070766606 learned_addr learned my addr 172.21.15.121:0/4070766606 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:54.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.882+0000 7f94e9037700 1 --1- 172.21.15.121:0/4070766606 >> v1:172.21.15.190:6789/0 conn(0x7f94e4153bb0 0x7f94e40a9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:48416/0 (socket says 172.21.15.121:48416) 2023-11-15T02:58:54.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.886+0000 7f94e9838700 1 -- 172.21.15.121:0/4070766606 >> v2:172.21.15.190:3300/0 conn(0x7f94e40aa240 msgr2=0x7f94e4067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:54.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.886+0000 7f94e9838700 1 --2- 172.21.15.121:0/4070766606 >> v2:172.21.15.190:3300/0 conn(0x7f94e40aa240 0x7f94e4067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.886+0000 7f94e9838700 1 -- 172.21.15.121:0/4070766606 >> v1:172.21.15.190:6789/0 conn(0x7f94e4153bb0 legacy=0x7f94e40a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:54.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.886+0000 7f94e9838700 1 -- 172.21.15.121:0/4070766606 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e40c9ca0 con 0x7f94e4067f50 2023-11-15T02:58:54.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.886+0000 7f94e9838700 1 --2- 172.21.15.121:0/4070766606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94e4067f50 0x7f94e4065d30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f94e40c5a40 tx=0x7f94dc009780).ready entity=mon.2 client_cookie=fed0a387ec01b4d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:54.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.886+0000 7f94dbfff700 1 -- 172.21.15.121:0/4070766606 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc00e070 con 0x7f94e4067f50 2023-11-15T02:58:54.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.886+0000 7f94dbfff700 1 -- 172.21.15.121:0/4070766606 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94dc00bb60 con 0x7f94e4067f50 2023-11-15T02:58:54.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.886+0000 7f94dbfff700 1 -- 172.21.15.121:0/4070766606 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc012410 con 0x7f94e4067f50 2023-11-15T02:58:54.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.886+0000 7f94eb29b700 1 -- 172.21.15.121:0/4070766606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94e4067f50 msgr2=0x7f94e4065d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:54.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.886+0000 7f94eb29b700 1 --2- 172.21.15.121:0/4070766606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94e4067f50 0x7f94e4065d30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f94e40c5a40 tx=0x7f94dc009780).stop 2023-11-15T02:58:54.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.886+0000 7f94eb29b700 1 -- 172.21.15.121:0/4070766606 shutdown_connections 2023-11-15T02:58:54.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.886+0000 7f94eb29b700 1 --2- 172.21.15.121:0/4070766606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94e4067f50 0x7f94e4065d30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.886+0000 7f94eb29b700 1 --2- 172.21.15.121:0/4070766606 >> v2:172.21.15.190:3300/0 conn(0x7f94e40aa240 0x7f94e4067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.886+0000 7f94eb29b700 1 -- 172.21.15.121:0/4070766606 >> 172.21.15.121:0/4070766606 conn(0x7f94e4058c50 msgr2=0x7f94e4059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:54.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.886+0000 7f94eb29b700 1 -- 172.21.15.121:0/4070766606 shutdown_connections 2023-11-15T02:58:54.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.886+0000 7f94eb29b700 1 -- 172.21.15.121:0/4070766606 wait complete. 2023-11-15T02:58:54.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.890+0000 7f94eb29b700 1 Processor -- start 2023-11-15T02:58:54.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.890+0000 7f94eb29b700 1 -- start start 2023-11-15T02:58:54.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94eb29b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94e4067f50 0x7f94e41656f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:54.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94eb29b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94e40aa240 0x7f94e41671c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:54.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94eb29b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94e4153bb0 0x7f94e40acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:54.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94eb29b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40c3720 con 0x7f94e4067f50 2023-11-15T02:58:54.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94eb29b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40c2e10 con 0x7f94e40aa240 2023-11-15T02:58:54.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94eb29b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40c30d0 con 0x7f94e4153bb0 2023-11-15T02:58:54.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94e9037700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94e4067f50 0x7f94e41656f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:54.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94e9037700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94e4067f50 0x7f94e41656f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37932/0 (socket says 172.21.15.121:37932) 2023-11-15T02:58:54.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94e9037700 1 -- 172.21.15.121:0/3378704854 learned_addr learned my addr 172.21.15.121:0/3378704854 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:54.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94e8836700 1 --2- 172.21.15.121:0/3378704854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94e40aa240 0x7f94e41671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:54.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94e9838700 1 --2- 172.21.15.121:0/3378704854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94e4153bb0 0x7f94e40acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:54.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94e9037700 1 -- 172.21.15.121:0/3378704854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94e4153bb0 msgr2=0x7f94e40acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:54.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94e9037700 1 --2- 172.21.15.121:0/3378704854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94e4153bb0 0x7f94e40acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94e9037700 1 -- 172.21.15.121:0/3378704854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94e40aa240 msgr2=0x7f94e41671c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:54.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94e9037700 1 --2- 172.21.15.121:0/3378704854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94e40aa240 0x7f94e41671c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:54.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94e9037700 1 -- 172.21.15.121:0/3378704854 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94dc009a20 con 0x7f94e4067f50 2023-11-15T02:58:54.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94e8836700 1 --2- 172.21.15.121:0/3378704854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94e40aa240 0x7f94e41671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:54.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94e9037700 1 --2- 172.21.15.121:0/3378704854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94e4067f50 0x7f94e41656f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f94e000c410 tx=0x7f94e0007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:54.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94d97fa700 1 -- 172.21.15.121:0/3378704854 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e000ce10 con 0x7f94e4067f50 2023-11-15T02:58:54.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94d97fa700 1 -- 172.21.15.121:0/3378704854 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e00072e0 con 0x7f94e4067f50 2023-11-15T02:58:54.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94eb29b700 1 -- 172.21.15.121:0/3378704854 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e40c33b0 con 0x7f94e4067f50 2023-11-15T02:58:54.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94d97fa700 1 -- 172.21.15.121:0/3378704854 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e00056e0 con 0x7f94e4067f50 2023-11-15T02:58:54.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.894+0000 7f94eb29b700 1 -- 172.21.15.121:0/3378704854 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e41689e0 con 0x7f94e4067f50 2023-11-15T02:58:54.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.898+0000 7f94eb29b700 1 -- 172.21.15.121:0/3378704854 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e406c150 con 0x7f94e4067f50 2023-11-15T02:58:54.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.898+0000 7f94d97fa700 1 -- 172.21.15.121:0/3378704854 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f94e0005a50 con 0x7f94e4067f50 2023-11-15T02:58:54.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.898+0000 7f94d97fa700 1 --2- 172.21.15.121:0/3378704854 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f94c805a510 0x7f94c805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:54.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.898+0000 7f94d97fa700 1 -- 172.21.15.121:0/3378704854 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f94e0073a80 con 0x7f94e4067f50 2023-11-15T02:58:54.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.898+0000 7f94e8836700 1 --2- 172.21.15.121:0/3378704854 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f94c805a510 0x7f94c805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:54.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.902+0000 7f94e8836700 1 --2- 172.21.15.121:0/3378704854 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f94c805a510 0x7f94c805c9d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f94d4006bc0 tx=0x7f94d4008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:54.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:54.906+0000 7f94d97fa700 1 -- 172.21.15.121:0/3378704854 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f94e00474f0 con 0x7f94e4067f50 2023-11-15T02:58:55.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.070+0000 7f94eb29b700 1 -- 172.21.15.121:0/3378704854 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f94e40c7870 con 0x7f94e4067f50 2023-11-15T02:58:55.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.070+0000 7f94d97fa700 1 -- 172.21.15.121:0/3378704854 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v44) v1 ==== 84+0+143 (secure 0 0 0) 0x7f94e004ab10 con 0x7f94e4067f50 2023-11-15T02:58:55.078 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:58:55.079 INFO:teuthology.orchestra.run.smithi121.stdout:[ 2023-11-15T02:58:55.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:55.079 INFO:teuthology.orchestra.run.smithi121.stdout: "poolnum": 1, 2023-11-15T02:58:55.079 INFO:teuthology.orchestra.run.smithi121.stdout: "poolname": "device_health_metrics" 2023-11-15T02:58:55.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:58:55.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:58:55.080 INFO:teuthology.orchestra.run.smithi121.stdout: "poolnum": 2, 2023-11-15T02:58:55.080 INFO:teuthology.orchestra.run.smithi121.stdout: "poolname": "rbd" 2023-11-15T02:58:55.080 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:58:55.080 INFO:teuthology.orchestra.run.smithi121.stdout:] 2023-11-15T02:58:55.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.074+0000 7f94eb29b700 1 -- 172.21.15.121:0/3378704854 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f94c805a510 msgr2=0x7f94c805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:55.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.074+0000 7f94eb29b700 1 --2- 172.21.15.121:0/3378704854 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f94c805a510 0x7f94c805c9d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f94d4006bc0 tx=0x7f94d4008040).stop 2023-11-15T02:58:55.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.074+0000 7f94eb29b700 1 -- 172.21.15.121:0/3378704854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94e4067f50 msgr2=0x7f94e41656f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:55.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.074+0000 7f94eb29b700 1 --2- 172.21.15.121:0/3378704854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94e4067f50 0x7f94e41656f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f94e000c410 tx=0x7f94e0007e10).stop 2023-11-15T02:58:55.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.074+0000 7f94eb29b700 1 -- 172.21.15.121:0/3378704854 shutdown_connections 2023-11-15T02:58:55.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.074+0000 7f94eb29b700 1 --2- 172.21.15.121:0/3378704854 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f94c805a510 0x7f94c805c9d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:55.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.074+0000 7f94eb29b700 1 --2- 172.21.15.121:0/3378704854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94e4153bb0 0x7f94e40acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:55.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.074+0000 7f94eb29b700 1 --2- 172.21.15.121:0/3378704854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94e40aa240 0x7f94e41671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:55.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.074+0000 7f94eb29b700 1 --2- 172.21.15.121:0/3378704854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94e4067f50 0x7f94e41656f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:55.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.074+0000 7f94eb29b700 1 -- 172.21.15.121:0/3378704854 >> 172.21.15.121:0/3378704854 conn(0x7f94e4058c50 msgr2=0x7f94e40b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:55.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.074+0000 7f94eb29b700 1 -- 172.21.15.121:0/3378704854 shutdown_connections 2023-11-15T02:58:55.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.074+0000 7f94eb29b700 1 -- 172.21.15.121:0/3378704854 wait complete. 2023-11-15T02:58:55.100 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-15T02:58:55.101 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-15T02:58:55.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.266+0000 7f94d5707700 1 Processor -- start 2023-11-15T02:58:55.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.266+0000 7f94d5707700 1 -- start start 2023-11-15T02:58:55.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.270+0000 7f94d5707700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f94d00ab0d0 0x7f94d00ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:55.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.270+0000 7f94d5707700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00add40 0x7f94d00ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:55.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.270+0000 7f94d5707700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d0066c90 con 0x7f94d00ad1b0 2023-11-15T02:58:55.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.270+0000 7f94d5707700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d0065e20 con 0x7f94d00ab0d0 2023-11-15T02:58:55.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.270+0000 7f94d5707700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d0066080 con 0x7f94d00add40 2023-11-15T02:58:55.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.270+0000 7f94cffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f94d00ab0d0 0x7f94d00ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:55.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.270+0000 7f94d4f06700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00add40 0x7f94d00ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:55.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.270+0000 7f94cf7fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f94d00ad1b0 0x7f94d0065710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:48432/0 (socket says 172.21.15.121:48432) 2023-11-15T02:58:55.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.270+0000 7f94cffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f94d00ab0d0 0x7f94d00ab4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:56076/0 (socket says 172.21.15.121:56076) 2023-11-15T02:58:55.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.270+0000 7f94cf7fe700 1 -- 172.21.15.121:0/830737354 learned_addr learned my addr 172.21.15.121:0/830737354 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:55.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.270+0000 7f94d4f06700 1 -- 172.21.15.121:0/830737354 >> v2:172.21.15.190:3300/0 conn(0x7f94d00ab0d0 msgr2=0x7f94d00ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:55.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.270+0000 7f94d4f06700 1 --2- 172.21.15.121:0/830737354 >> v2:172.21.15.190:3300/0 conn(0x7f94d00ab0d0 0x7f94d00ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:55.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.270+0000 7f94d4f06700 1 -- 172.21.15.121:0/830737354 >> v1:172.21.15.190:6789/0 conn(0x7f94d00ad1b0 legacy=0x7f94d0065710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:55.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.270+0000 7f94ceffd700 1 -- 172.21.15.121:0/830737354 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880107779 0 0) 0x7f94d0066c90 con 0x7f94d00ad1b0 2023-11-15T02:58:55.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.270+0000 7f94d4f06700 1 -- 172.21.15.121:0/830737354 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d0066360 con 0x7f94d00add40 2023-11-15T02:58:55.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.270+0000 7f94cffff700 1 --2- 172.21.15.121:0/830737354 >> v2:172.21.15.190:3300/0 conn(0x7f94d00ab0d0 0x7f94d00ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:58:55.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.270+0000 7f94d4f06700 1 --2- 172.21.15.121:0/830737354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00add40 0x7f94d00ae1f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f94c8009e60 tx=0x7f94c8009780).ready entity=mon.2 client_cookie=471f24d17edb778 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:55.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.274+0000 7f94ceffd700 1 -- 172.21.15.121:0/830737354 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c800e070 con 0x7f94d00add40 2023-11-15T02:58:55.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.274+0000 7f94ceffd700 1 -- 172.21.15.121:0/830737354 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c800bb60 con 0x7f94d00add40 2023-11-15T02:58:55.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.274+0000 7f94ceffd700 1 -- 172.21.15.121:0/830737354 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c8012410 con 0x7f94d00add40 2023-11-15T02:58:55.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.274+0000 7f94d5707700 1 -- 172.21.15.121:0/830737354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00add40 msgr2=0x7f94d00ae1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:55.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.274+0000 7f94d5707700 1 --2- 172.21.15.121:0/830737354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00add40 0x7f94d00ae1f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f94c8009e60 tx=0x7f94c8009780).stop 2023-11-15T02:58:55.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.274+0000 7f94d5707700 1 -- 172.21.15.121:0/830737354 shutdown_connections 2023-11-15T02:58:55.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.274+0000 7f94d5707700 1 --2- 172.21.15.121:0/830737354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00add40 0x7f94d00ae1f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:55.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.274+0000 7f94d5707700 1 --2- 172.21.15.121:0/830737354 >> v2:172.21.15.190:3300/0 conn(0x7f94d00ab0d0 0x7f94d00ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:55.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.274+0000 7f94d5707700 1 -- 172.21.15.121:0/830737354 >> 172.21.15.121:0/830737354 conn(0x7f94d0067a60 msgr2=0x7f94d0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:55.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.274+0000 7f94d5707700 1 -- 172.21.15.121:0/830737354 shutdown_connections 2023-11-15T02:58:55.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.274+0000 7f94d5707700 1 -- 172.21.15.121:0/830737354 wait complete. 2023-11-15T02:58:55.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.278+0000 7f94d5707700 1 Processor -- start 2023-11-15T02:58:55.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.278+0000 7f94d5707700 1 -- start start 2023-11-15T02:58:55.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.278+0000 7f94d5707700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00ab0d0 0x7f94d0151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:55.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.278+0000 7f94d5707700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d00ad1b0 0x7f94d00a8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:55.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.278+0000 7f94d5707700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94d00add40 0x7f94d00afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:55.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.278+0000 7f94d5707700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d00a4890 con 0x7f94d00add40 2023-11-15T02:58:55.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.278+0000 7f94d5707700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d00a56f0 con 0x7f94d00ad1b0 2023-11-15T02:58:55.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.278+0000 7f94d5707700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d00a4ae0 con 0x7f94d00ab0d0 2023-11-15T02:58:55.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.278+0000 7f94d4f06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94d00add40 0x7f94d00afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:55.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.278+0000 7f94d4f06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94d00add40 0x7f94d00afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37938/0 (socket says 172.21.15.121:37938) 2023-11-15T02:58:55.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.278+0000 7f94d4f06700 1 -- 172.21.15.121:0/631697123 learned_addr learned my addr 172.21.15.121:0/631697123 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:55.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.278+0000 7f94cffff700 1 --2- 172.21.15.121:0/631697123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00ab0d0 0x7f94d0151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:55.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.278+0000 7f94cf7fe700 1 --2- 172.21.15.121:0/631697123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d00ad1b0 0x7f94d00a8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:55.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.278+0000 7f94d4f06700 1 -- 172.21.15.121:0/631697123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00ab0d0 msgr2=0x7f94d0151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:55.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.278+0000 7f94d4f06700 1 --2- 172.21.15.121:0/631697123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00ab0d0 0x7f94d0151ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:55.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.278+0000 7f94d4f06700 1 -- 172.21.15.121:0/631697123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d00ad1b0 msgr2=0x7f94d00a8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:55.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.278+0000 7f94d4f06700 1 --2- 172.21.15.121:0/631697123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d00ad1b0 0x7f94d00a8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:55.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.278+0000 7f94d4f06700 1 -- 172.21.15.121:0/631697123 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c0009250 con 0x7f94d00add40 2023-11-15T02:58:55.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.278+0000 7f94cf7fe700 1 --2- 172.21.15.121:0/631697123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d00ad1b0 0x7f94d00a8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:55.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.282+0000 7f94d4f06700 1 --2- 172.21.15.121:0/631697123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94d00add40 0x7f94d00afea0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f94c800b6c0 tx=0x7f94c8009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:55.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.282+0000 7f94b7fff700 1 -- 172.21.15.121:0/631697123 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c801d700 con 0x7f94d00add40 2023-11-15T02:58:55.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.282+0000 7f94b7fff700 1 -- 172.21.15.121:0/631697123 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c801d860 con 0x7f94d00add40 2023-11-15T02:58:55.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.282+0000 7f94d5707700 1 -- 172.21.15.121:0/631697123 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c8009a20 con 0x7f94d00add40 2023-11-15T02:58:55.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.282+0000 7f94b7fff700 1 -- 172.21.15.121:0/631697123 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c801dbe0 con 0x7f94d00add40 2023-11-15T02:58:55.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.282+0000 7f94d5707700 1 -- 172.21.15.121:0/631697123 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d00a50a0 con 0x7f94d00add40 2023-11-15T02:58:55.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.282+0000 7f94d5707700 1 -- 172.21.15.121:0/631697123 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d006bcf0 con 0x7f94d00add40 2023-11-15T02:58:55.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.282+0000 7f94b7fff700 1 -- 172.21.15.121:0/631697123 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f94c8003040 con 0x7f94d00add40 2023-11-15T02:58:55.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.286+0000 7f94b7fff700 1 --2- 172.21.15.121:0/631697123 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f94bc05a510 0x7f94bc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:55.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.286+0000 7f94b7fff700 1 -- 172.21.15.121:0/631697123 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f94c8076510 con 0x7f94d00add40 2023-11-15T02:58:55.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.286+0000 7f94cffff700 1 --2- 172.21.15.121:0/631697123 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f94bc05a510 0x7f94bc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:55.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.286+0000 7f94cffff700 1 --2- 172.21.15.121:0/631697123 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f94bc05a510 0x7f94bc05c9d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f94d006a9f0 tx=0x7f94c0008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:55.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.290+0000 7f94b7fff700 1 -- 172.21.15.121:0/631697123 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f94c804a080 con 0x7f94d00add40 2023-11-15T02:58:55.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:55.454+0000 7f94d5707700 1 -- 172.21.15.121:0/631697123 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f94d0064090 con 0x7f94d00add40 2023-11-15T02:58:56.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.142+0000 7f94b7fff700 1 -- 172.21.15.121:0/631697123 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v45) v1 ==== 152+0+0 (secure 0 0 0) 0x7f94c804d660 con 0x7f94d00add40 2023-11-15T02:58:56.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.146+0000 7f94d5707700 1 -- 172.21.15.121:0/631697123 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f94bc05a510 msgr2=0x7f94bc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:56.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.146+0000 7f94d5707700 1 --2- 172.21.15.121:0/631697123 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f94bc05a510 0x7f94bc05c9d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f94d006a9f0 tx=0x7f94c0008040).stop 2023-11-15T02:58:56.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.146+0000 7f94d5707700 1 -- 172.21.15.121:0/631697123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94d00add40 msgr2=0x7f94d00afea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:56.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.146+0000 7f94d5707700 1 --2- 172.21.15.121:0/631697123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94d00add40 0x7f94d00afea0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f94c800b6c0 tx=0x7f94c8009780).stop 2023-11-15T02:58:56.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.146+0000 7f94d5707700 1 -- 172.21.15.121:0/631697123 shutdown_connections 2023-11-15T02:58:56.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.146+0000 7f94d5707700 1 --2- 172.21.15.121:0/631697123 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f94bc05a510 0x7f94bc05c9d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:56.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.146+0000 7f94d5707700 1 --2- 172.21.15.121:0/631697123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94d00add40 0x7f94d00afea0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:56.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.146+0000 7f94d5707700 1 --2- 172.21.15.121:0/631697123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d00ad1b0 0x7f94d00a8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:56.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.146+0000 7f94d5707700 1 --2- 172.21.15.121:0/631697123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00ab0d0 0x7f94d0151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:56.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.150+0000 7f94d5707700 1 -- 172.21.15.121:0/631697123 >> 172.21.15.121:0/631697123 conn(0x7f94d0067a60 msgr2=0x7f94d00ae150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:56.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.150+0000 7f94d5707700 1 -- 172.21.15.121:0/631697123 shutdown_connections 2023-11-15T02:58:56.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.150+0000 7f94d5707700 1 -- 172.21.15.121:0/631697123 wait complete. 2023-11-15T02:58:56.160 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_metadata' created 2023-11-15T02:58:56.180 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-15T02:58:56.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.342+0000 7effc5507700 1 Processor -- start 2023-11-15T02:58:56.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.342+0000 7effc5507700 1 -- start start 2023-11-15T02:58:56.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.346+0000 7effc5507700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7effc00abaa0 0x7effc00abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:56.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.346+0000 7effc5507700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc00c2c00 con 0x7effc00bb510 2023-11-15T02:58:56.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.346+0000 7effc5507700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc00657b0 con 0x7effc00a8880 2023-11-15T02:58:56.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.346+0000 7effc5507700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc0065a90 con 0x7effc00abaa0 2023-11-15T02:58:56.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.346+0000 7effbffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7effc00abaa0 0x7effc00abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:56.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.346+0000 7effbffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7effc00abaa0 0x7effc00abe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37952/0 (socket says 172.21.15.121:37952) 2023-11-15T02:58:56.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.346+0000 7effbffff700 1 -- 172.21.15.121:0/520440119 learned_addr learned my addr 172.21.15.121:0/520440119 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:56.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.346+0000 7effbffff700 1 -- 172.21.15.121:0/520440119 >> v1:172.21.15.190:6789/0 conn(0x7effc00a8880 legacy=0x7effc0151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:56.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.346+0000 7effbffff700 1 -- 172.21.15.121:0/520440119 >> v1:172.21.15.121:6789/0 conn(0x7effc00bb510 legacy=0x7effc0063570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:56.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.346+0000 7effbffff700 1 -- 172.21.15.121:0/520440119 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc00c2890 con 0x7effc00abaa0 2023-11-15T02:58:56.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.346+0000 7effbffff700 1 --2- 172.21.15.121:0/520440119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc00abaa0 0x7effc00abe80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7effb0006fd0 tx=0x7effb00090f0).ready entity=mon.0 client_cookie=1bb759bc55ebc412 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:56.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.346+0000 7effbeffd700 1 -- 172.21.15.121:0/520440119 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb000b600 con 0x7effc00abaa0 2023-11-15T02:58:56.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.346+0000 7effbeffd700 1 -- 172.21.15.121:0/520440119 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb000b760 con 0x7effc00abaa0 2023-11-15T02:58:56.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.346+0000 7effbeffd700 1 -- 172.21.15.121:0/520440119 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb000bae0 con 0x7effc00abaa0 2023-11-15T02:58:56.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.350+0000 7effc5507700 1 -- 172.21.15.121:0/520440119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc00abaa0 msgr2=0x7effc00abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:56.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.350+0000 7effc5507700 1 --2- 172.21.15.121:0/520440119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc00abaa0 0x7effc00abe80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7effb0006fd0 tx=0x7effb00090f0).stop 2023-11-15T02:58:56.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.350+0000 7effc5507700 1 -- 172.21.15.121:0/520440119 shutdown_connections 2023-11-15T02:58:56.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.350+0000 7effc5507700 1 --2- 172.21.15.121:0/520440119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc00abaa0 0x7effc00abe80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:56.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.350+0000 7effc5507700 1 -- 172.21.15.121:0/520440119 >> 172.21.15.121:0/520440119 conn(0x7effc0067a60 msgr2=0x7effc0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:56.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.350+0000 7effc5507700 1 -- 172.21.15.121:0/520440119 shutdown_connections 2023-11-15T02:58:56.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.350+0000 7effc5507700 1 -- 172.21.15.121:0/520440119 wait complete. 2023-11-15T02:58:56.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.350+0000 7effc5507700 1 Processor -- start 2023-11-15T02:58:56.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.350+0000 7effc5507700 1 -- start start 2023-11-15T02:58:56.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effc5507700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effc00a8880 0x7effc014d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:56.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effc5507700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc00abaa0 0x7effc014a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:56.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effc5507700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effc00bb510 0x7effc00bfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:56.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effc5507700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc00bf660 con 0x7effc00abaa0 2023-11-15T02:58:56.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effc5507700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc014dc30 con 0x7effc00bb510 2023-11-15T02:58:56.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effc5507700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc00be770 con 0x7effc00a8880 2023-11-15T02:58:56.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effbf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc00abaa0 0x7effc014a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:56.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effbf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc00abaa0 0x7effc014a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37954/0 (socket says 172.21.15.121:37954) 2023-11-15T02:58:56.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effbf7fe700 1 -- 172.21.15.121:0/2146986592 learned_addr learned my addr 172.21.15.121:0/2146986592 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:56.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effbffff700 1 --2- 172.21.15.121:0/2146986592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effc00a8880 0x7effc014d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:56.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effc4d06700 1 --2- 172.21.15.121:0/2146986592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effc00bb510 0x7effc00bfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:56.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effbf7fe700 1 -- 172.21.15.121:0/2146986592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effc00a8880 msgr2=0x7effc014d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:56.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effbf7fe700 1 --2- 172.21.15.121:0/2146986592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effc00a8880 0x7effc014d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:56.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effbf7fe700 1 -- 172.21.15.121:0/2146986592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effc00bb510 msgr2=0x7effc00bfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:56.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effbf7fe700 1 --2- 172.21.15.121:0/2146986592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effc00bb510 0x7effc00bfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:56.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effbf7fe700 1 -- 172.21.15.121:0/2146986592 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb0009370 con 0x7effc00abaa0 2023-11-15T02:58:56.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effbf7fe700 1 --2- 172.21.15.121:0/2146986592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc00abaa0 0x7effc014a170 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7effb400c4c0 tx=0x7effb4007d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:56.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effa7fff700 1 -- 172.21.15.121:0/2146986592 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4013450 con 0x7effc00abaa0 2023-11-15T02:58:56.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effa7fff700 1 -- 172.21.15.121:0/2146986592 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb40135b0 con 0x7effc00abaa0 2023-11-15T02:58:56.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effc5507700 1 -- 172.21.15.121:0/2146986592 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc00bea50 con 0x7effc00abaa0 2023-11-15T02:58:56.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effa7fff700 1 -- 172.21.15.121:0/2146986592 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4013930 con 0x7effc00abaa0 2023-11-15T02:58:56.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.354+0000 7effc5507700 1 -- 172.21.15.121:0/2146986592 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc00bf010 con 0x7effc00abaa0 2023-11-15T02:58:56.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.358+0000 7effc5507700 1 -- 172.21.15.121:0/2146986592 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc00bf810 con 0x7effc00abaa0 2023-11-15T02:58:56.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.358+0000 7effa7fff700 1 -- 172.21.15.121:0/2146986592 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7effb401c3f0 con 0x7effc00abaa0 2023-11-15T02:58:56.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.362+0000 7effa7fff700 1 --2- 172.21.15.121:0/2146986592 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effac05a540 0x7effac05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:56.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.362+0000 7effa7fff700 1 -- 172.21.15.121:0/2146986592 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5639+0+0 (secure 0 0 0) 0x7effb4073ec0 con 0x7effc00abaa0 2023-11-15T02:58:56.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.362+0000 7effbffff700 1 --2- 172.21.15.121:0/2146986592 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effac05a540 0x7effac05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:56.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.362+0000 7effbffff700 1 --2- 172.21.15.121:0/2146986592 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effac05a540 0x7effac05ca00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7effb0009130 tx=0x7effb0017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:56.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.366+0000 7effa7fff700 1 -- 172.21.15.121:0/2146986592 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7effc00bf810 con 0x7effc00abaa0 2023-11-15T02:58:56.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:56.526+0000 7effc5507700 1 -- 172.21.15.121:0/2146986592 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7effc006bcf0 con 0x7effc00abaa0 2023-11-15T02:58:57.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.154+0000 7effa7fff700 1 -- 172.21.15.121:0/2146986592 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v46) v1 ==== 170+0+0 (secure 0 0 0) 0x7effb4018030 con 0x7effc00abaa0 2023-11-15T02:58:57.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.166+0000 7effc5507700 1 -- 172.21.15.121:0/2146986592 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effac05a540 msgr2=0x7effac05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:57.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.166+0000 7effc5507700 1 --2- 172.21.15.121:0/2146986592 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effac05a540 0x7effac05ca00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7effb0009130 tx=0x7effb0017000).stop 2023-11-15T02:58:57.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.166+0000 7effc5507700 1 -- 172.21.15.121:0/2146986592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc00abaa0 msgr2=0x7effc014a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:57.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.166+0000 7effc5507700 1 --2- 172.21.15.121:0/2146986592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc00abaa0 0x7effc014a170 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7effb400c4c0 tx=0x7effb4007d80).stop 2023-11-15T02:58:57.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.170+0000 7effc5507700 1 -- 172.21.15.121:0/2146986592 shutdown_connections 2023-11-15T02:58:57.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.170+0000 7effc5507700 1 --2- 172.21.15.121:0/2146986592 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effac05a540 0x7effac05ca00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:57.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.170+0000 7effc5507700 1 --2- 172.21.15.121:0/2146986592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effc00bb510 0x7effc00bfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:57.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.170+0000 7effc5507700 1 --2- 172.21.15.121:0/2146986592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effc00abaa0 0x7effc014a170 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:57.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.170+0000 7effc5507700 1 --2- 172.21.15.121:0/2146986592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effc00a8880 0x7effc014d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:57.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.174+0000 7effc5507700 1 -- 172.21.15.121:0/2146986592 >> 172.21.15.121:0/2146986592 conn(0x7effc0067a60 msgr2=0x7effc00a7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:57.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.178+0000 7effc5507700 1 -- 172.21.15.121:0/2146986592 shutdown_connections 2023-11-15T02:58:57.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.178+0000 7effc5507700 1 -- 172.21.15.121:0/2146986592 wait complete. 2023-11-15T02:58:57.193 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_data' created 2023-11-15T02:58:57.215 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-15T02:58:57.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.378+0000 7f05b7347700 1 Processor -- start 2023-11-15T02:58:57.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.378+0000 7f05b7347700 1 -- start start 2023-11-15T02:58:57.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.378+0000 7f05b7347700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f05b00acd00 0x7f05b00ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:57.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.378+0000 7f05b7347700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05b0066330 0x7f05b0153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:57.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.378+0000 7f05b7347700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b0065ea0 con 0x7f05b00c5c70 2023-11-15T02:58:57.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.378+0000 7f05b7347700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05b0066bb0 con 0x7f05b00acd00 2023-11-15T02:58:57.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.378+0000 7f05b7347700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b0065890 con 0x7f05b0066330 2023-11-15T02:58:57.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.382+0000 7f05b6b46700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05b0066330 0x7f05b0153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:57.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.382+0000 7f05b6345700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f05b00acd00 0x7f05b00ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:57.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.382+0000 7f05b6b46700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05b0066330 0x7f05b0153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:36960/0 (socket says 172.21.15.121:36960) 2023-11-15T02:58:57.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.382+0000 7f05b6b46700 1 -- 172.21.15.121:0/580829734 learned_addr learned my addr 172.21.15.121:0/580829734 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:57.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.382+0000 7f05b6b46700 1 -- 172.21.15.121:0/580829734 >> v2:172.21.15.190:3300/0 conn(0x7f05b00acd00 msgr2=0x7f05b00ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:57.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.382+0000 7f05b6b46700 1 --2- 172.21.15.121:0/580829734 >> v2:172.21.15.190:3300/0 conn(0x7f05b00acd00 0x7f05b00ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:57.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.382+0000 7f05b6b46700 1 -- 172.21.15.121:0/580829734 >> v1:172.21.15.190:6789/0 conn(0x7f05b00c5c70 legacy=0x7f05b01536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:57.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.382+0000 7f05b6b46700 1 -- 172.21.15.121:0/580829734 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b00c64a0 con 0x7f05b0066330 2023-11-15T02:58:57.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.382+0000 7f05b6b46700 1 --2- 172.21.15.121:0/580829734 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05b0066330 0x7f05b0153dd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f05a8009f20 tx=0x7f05a8009840).ready entity=mon.2 client_cookie=215ace1b1beb55 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:57.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.382+0000 7f05b5343700 1 -- 172.21.15.121:0/580829734 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a800e070 con 0x7f05b0066330 2023-11-15T02:58:57.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.382+0000 7f05b5343700 1 -- 172.21.15.121:0/580829734 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a800b9d0 con 0x7f05b0066330 2023-11-15T02:58:57.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.382+0000 7f05b5343700 1 -- 172.21.15.121:0/580829734 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a800bd50 con 0x7f05b0066330 2023-11-15T02:58:57.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.382+0000 7f05b7347700 1 -- 172.21.15.121:0/580829734 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05b0066330 msgr2=0x7f05b0153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:57.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.382+0000 7f05b7347700 1 --2- 172.21.15.121:0/580829734 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05b0066330 0x7f05b0153dd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f05a8009f20 tx=0x7f05a8009840).stop 2023-11-15T02:58:57.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.382+0000 7f05b7347700 1 -- 172.21.15.121:0/580829734 shutdown_connections 2023-11-15T02:58:57.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.382+0000 7f05b7347700 1 --2- 172.21.15.121:0/580829734 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05b0066330 0x7f05b0153dd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:57.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.382+0000 7f05b7347700 1 --2- 172.21.15.121:0/580829734 >> v2:172.21.15.190:3300/0 conn(0x7f05b00acd00 0x7f05b00ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:57.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.382+0000 7f05b7347700 1 -- 172.21.15.121:0/580829734 >> 172.21.15.121:0/580829734 conn(0x7f05b0063fc0 msgr2=0x7f05b0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:57.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.382+0000 7f05b7347700 1 -- 172.21.15.121:0/580829734 shutdown_connections 2023-11-15T02:58:57.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.386+0000 7f05b7347700 1 -- 172.21.15.121:0/580829734 wait complete. 2023-11-15T02:58:57.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.386+0000 7f05b7347700 1 Processor -- start 2023-11-15T02:58:57.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.386+0000 7f05b7347700 1 -- start start 2023-11-15T02:58:57.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b7347700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05b0066330 0x7f05b00caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:57.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b7347700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05b00acd00 0x7f05b00c4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:57.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b7347700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05b00c5c70 0x7f05b00c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:57.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b7347700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b013af40 con 0x7f05b0066330 2023-11-15T02:58:57.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b7347700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b00c1d30 con 0x7f05b00c5c70 2023-11-15T02:58:57.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b7347700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b0156440 con 0x7f05b00acd00 2023-11-15T02:58:57.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b6345700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05b0066330 0x7f05b00caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:57.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b6345700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05b0066330 0x7f05b00caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37956/0 (socket says 172.21.15.121:37956) 2023-11-15T02:58:57.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b6345700 1 -- 172.21.15.121:0/2137003843 learned_addr learned my addr 172.21.15.121:0/2137003843 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:57.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b5b44700 1 --2- 172.21.15.121:0/2137003843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05b00acd00 0x7f05b00c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:57.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b6b46700 1 --2- 172.21.15.121:0/2137003843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05b00c5c70 0x7f05b00c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:57.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b6345700 1 -- 172.21.15.121:0/2137003843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05b00acd00 msgr2=0x7f05b00c4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:57.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b6345700 1 --2- 172.21.15.121:0/2137003843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05b00acd00 0x7f05b00c4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:57.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b6345700 1 -- 172.21.15.121:0/2137003843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05b00c5c70 msgr2=0x7f05b00c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:57.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b6345700 1 --2- 172.21.15.121:0/2137003843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05b00c5c70 0x7f05b00c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:57.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b6345700 1 -- 172.21.15.121:0/2137003843 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a8009ae0 con 0x7f05b0066330 2023-11-15T02:58:57.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b6b46700 1 --2- 172.21.15.121:0/2137003843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05b00c5c70 0x7f05b00c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:58:57.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b6345700 1 --2- 172.21.15.121:0/2137003843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05b0066330 0x7f05b00caf70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f05ac00e8d0 tx=0x7f05ac006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:57.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05a67fc700 1 -- 172.21.15.121:0/2137003843 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac012600 con 0x7f05b0066330 2023-11-15T02:58:57.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05a67fc700 1 -- 172.21.15.121:0/2137003843 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac012760 con 0x7f05b0066330 2023-11-15T02:58:57.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05a67fc700 1 -- 172.21.15.121:0/2137003843 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac012ae0 con 0x7f05b0066330 2023-11-15T02:58:57.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b7347700 1 -- 172.21.15.121:0/2137003843 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b0156720 con 0x7f05b0066330 2023-11-15T02:58:57.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.390+0000 7f05b7347700 1 -- 172.21.15.121:0/2137003843 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b00c85d0 con 0x7f05b0066330 2023-11-15T02:58:57.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.394+0000 7f05b7347700 1 -- 172.21.15.121:0/2137003843 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b00c1ae0 con 0x7f05b0066330 2023-11-15T02:58:57.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.398+0000 7f05a67fc700 1 -- 172.21.15.121:0/2137003843 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f05ac01b400 con 0x7f05b0066330 2023-11-15T02:58:57.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.398+0000 7f05a67fc700 1 --2- 172.21.15.121:0/2137003843 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f059405a540 0x7f059405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:57.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.398+0000 7f05a67fc700 1 -- 172.21.15.121:0/2137003843 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6021+0+0 (secure 0 0 0) 0x7f05ac0751e0 con 0x7f05b0066330 2023-11-15T02:58:57.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.398+0000 7f05b5b44700 1 --2- 172.21.15.121:0/2137003843 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f059405a540 0x7f059405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:57.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.398+0000 7f05b5b44700 1 --2- 172.21.15.121:0/2137003843 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f059405a540 0x7f059405ca00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f05b00aeff0 tx=0x7f05a0007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:57.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.402+0000 7f05a67fc700 1 -- 172.21.15.121:0/2137003843 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f05ac048900 con 0x7f05b0066330 2023-11-15T02:58:57.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.602+0000 7f05b7347700 1 -- 172.21.15.121:0/2137003843 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f05b006bd60 con 0x7f05b0066330 2023-11-15T02:58:57.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.614+0000 7f05a67fc700 1 -- 172.21.15.121:0/2137003843 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v27) v1 ==== 172+0+0 (secure 0 0 0) 0x7f05b006bd60 con 0x7f05b0066330 2023-11-15T02:58:57.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.622+0000 7f05b7347700 1 -- 172.21.15.121:0/2137003843 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f059405a540 msgr2=0x7f059405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:57.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.622+0000 7f05b7347700 1 --2- 172.21.15.121:0/2137003843 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f059405a540 0x7f059405ca00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f05b00aeff0 tx=0x7f05a0007d40).stop 2023-11-15T02:58:57.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.622+0000 7f05b7347700 1 -- 172.21.15.121:0/2137003843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05b0066330 msgr2=0x7f05b00caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:57.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.622+0000 7f05b7347700 1 --2- 172.21.15.121:0/2137003843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05b0066330 0x7f05b00caf70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f05ac00e8d0 tx=0x7f05ac006f40).stop 2023-11-15T02:58:57.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.626+0000 7f05b7347700 1 -- 172.21.15.121:0/2137003843 shutdown_connections 2023-11-15T02:58:57.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.626+0000 7f05b7347700 1 --2- 172.21.15.121:0/2137003843 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f059405a540 0x7f059405ca00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:57.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.626+0000 7f05b7347700 1 --2- 172.21.15.121:0/2137003843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05b00c5c70 0x7f05b00c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:57.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.626+0000 7f05b7347700 1 --2- 172.21.15.121:0/2137003843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05b00acd00 0x7f05b00c4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:57.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.626+0000 7f05b7347700 1 --2- 172.21.15.121:0/2137003843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05b0066330 0x7f05b00caf70 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:57.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.626+0000 7f05b7347700 1 -- 172.21.15.121:0/2137003843 >> 172.21.15.121:0/2137003843 conn(0x7f05b0063fc0 msgr2=0x7f05b014f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:57.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.626+0000 7f05b7347700 1 -- 172.21.15.121:0/2137003843 shutdown_connections 2023-11-15T02:58:57.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.626+0000 7f05b7347700 1 -- 172.21.15.121:0/2137003843 wait complete. 2023-11-15T02:58:57.642 INFO:teuthology.orchestra.run.smithi121.stderr:new fs with metadata pool 7 and data pool 8 2023-11-15T02:58:57.665 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:58:57.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.818+0000 7f84755a9700 1 Processor -- start 2023-11-15T02:58:57.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.818+0000 7f84755a9700 1 -- start start 2023-11-15T02:58:57.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.818+0000 7f84755a9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f84701517c0 0x7f8470151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:57.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.818+0000 7f84755a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84700ab2c0 0x7f84700a8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:57.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f84755a9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8470065d00 con 0x7f8470155630 2023-11-15T02:58:57.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f84755a9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84700c6bf0 con 0x7f84701517c0 2023-11-15T02:58:57.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f84755a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84700c6e90 con 0x7f84700ab2c0 2023-11-15T02:58:57.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f846effd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f84701517c0 0x7f8470151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:57.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f846effd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f84701517c0 0x7f8470151ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37962/0 (socket says 172.21.15.121:37962) 2023-11-15T02:58:57.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f846effd700 1 -- 172.21.15.121:0/3814435703 learned_addr learned my addr 172.21.15.121:0/3814435703 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:57.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f846effd700 1 -- 172.21.15.121:0/3814435703 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84700ab2c0 msgr2=0x7f84700a8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:57.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f846effd700 1 --2- 172.21.15.121:0/3814435703 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84700ab2c0 0x7f84700a8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T02:58:57.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f846effd700 1 -- 172.21.15.121:0/3814435703 >> v1:172.21.15.190:6789/0 conn(0x7f8470155630 legacy=0x7f84700c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:57.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f846effd700 1 -- 172.21.15.121:0/3814435703 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84700c7170 con 0x7f84701517c0 2023-11-15T02:58:57.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f846effd700 1 --2- 172.21.15.121:0/3814435703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84701517c0 0x7f8470151ba0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f8460006fd0 tx=0x7f84600090f0).ready entity=mon.0 client_cookie=ee33b399cfdd7efd server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:57.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f846e7fc700 1 -- 172.21.15.121:0/3814435703 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846000b600 con 0x7f84701517c0 2023-11-15T02:58:57.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f846e7fc700 1 -- 172.21.15.121:0/3814435703 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846000b760 con 0x7f84701517c0 2023-11-15T02:58:57.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f846e7fc700 1 -- 172.21.15.121:0/3814435703 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846000bae0 con 0x7f84701517c0 2023-11-15T02:58:57.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f84755a9700 1 -- 172.21.15.121:0/3814435703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84701517c0 msgr2=0x7f8470151ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:57.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f84755a9700 1 --2- 172.21.15.121:0/3814435703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84701517c0 0x7f8470151ba0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f8460006fd0 tx=0x7f84600090f0).stop 2023-11-15T02:58:57.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f84755a9700 1 -- 172.21.15.121:0/3814435703 shutdown_connections 2023-11-15T02:58:57.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f84755a9700 1 --2- 172.21.15.121:0/3814435703 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84700ab2c0 0x7f84700a8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T02:58:57.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f84755a9700 1 --2- 172.21.15.121:0/3814435703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84701517c0 0x7f8470151ba0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:57.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f84755a9700 1 -- 172.21.15.121:0/3814435703 >> 172.21.15.121:0/3814435703 conn(0x7f84700c89c0 msgr2=0x7f84700c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:57.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f84755a9700 1 -- 172.21.15.121:0/3814435703 shutdown_connections 2023-11-15T02:58:57.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.822+0000 7f84755a9700 1 -- 172.21.15.121:0/3814435703 wait complete. 2023-11-15T02:58:57.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.826+0000 7f84755a9700 1 Processor -- start 2023-11-15T02:58:57.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.826+0000 7f84755a9700 1 -- start start 2023-11-15T02:58:57.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f84755a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84700ab2c0 0x7f84700abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:57.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f84755a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84701517c0 0x7f847014f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:57.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f84755a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8470155630 0x7f847014bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:57.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f84755a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8470151ba0 con 0x7f8470155630 2023-11-15T02:58:57.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f84755a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8470152720 con 0x7f84700ab2c0 2023-11-15T02:58:57.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f84755a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8470151df0 con 0x7f84701517c0 2023-11-15T02:58:57.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f846f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8470155630 0x7f847014bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:57.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f846f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8470155630 0x7f847014bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37970/0 (socket says 172.21.15.121:37970) 2023-11-15T02:58:57.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f846f7fe700 1 -- 172.21.15.121:0/3668162203 learned_addr learned my addr 172.21.15.121:0/3668162203 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:57.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f846effd700 1 --2- 172.21.15.121:0/3668162203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84700ab2c0 0x7f84700abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:57.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f8467fff700 1 --2- 172.21.15.121:0/3668162203 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84701517c0 0x7f847014f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:57.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f846f7fe700 1 -- 172.21.15.121:0/3668162203 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84701517c0 msgr2=0x7f847014f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:57.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f846f7fe700 1 --2- 172.21.15.121:0/3668162203 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84701517c0 0x7f847014f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:57.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f846f7fe700 1 -- 172.21.15.121:0/3668162203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84700ab2c0 msgr2=0x7f84700abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:57.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f846f7fe700 1 --2- 172.21.15.121:0/3668162203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84700ab2c0 0x7f84700abb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:57.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f846f7fe700 1 -- 172.21.15.121:0/3668162203 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8460009370 con 0x7f8470155630 2023-11-15T02:58:57.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f846f7fe700 1 --2- 172.21.15.121:0/3668162203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8470155630 0x7f847014bd20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f846800b840 tx=0x7f8468009270).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:57.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f8466ffd700 1 -- 172.21.15.121:0/3668162203 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846800d4a0 con 0x7f8470155630 2023-11-15T02:58:57.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f8466ffd700 1 -- 172.21.15.121:0/3668162203 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846800d640 con 0x7f8470155630 2023-11-15T02:58:57.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f84755a9700 1 -- 172.21.15.121:0/3668162203 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84701520d0 con 0x7f8470155630 2023-11-15T02:58:57.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f8466ffd700 1 -- 172.21.15.121:0/3668162203 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846800da00 con 0x7f8470155630 2023-11-15T02:58:57.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.830+0000 7f84755a9700 1 -- 172.21.15.121:0/3668162203 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847014fc20 con 0x7f8470155630 2023-11-15T02:58:57.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.834+0000 7f84755a9700 1 -- 172.21.15.121:0/3668162203 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84701528b0 con 0x7f8470155630 2023-11-15T02:58:57.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.834+0000 7f8466ffd700 1 -- 172.21.15.121:0/3668162203 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8468011400 con 0x7f8470155630 2023-11-15T02:58:57.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.834+0000 7f8466ffd700 1 --2- 172.21.15.121:0/3668162203 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f845005a510 0x7f845005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:57.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.834+0000 7f8466ffd700 1 -- 172.21.15.121:0/3668162203 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8468073f70 con 0x7f8470155630 2023-11-15T02:58:57.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.834+0000 7f846effd700 1 --2- 172.21.15.121:0/3668162203 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f845005a510 0x7f845005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:57.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.838+0000 7f846effd700 1 --2- 172.21.15.121:0/3668162203 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f845005a510 0x7f845005c9d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f8460009130 tx=0x7f8460017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:57.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:57.842+0000 7f8466ffd700 1 -- 172.21.15.121:0/3668162203 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f84680a2af0 con 0x7f8470155630 2023-11-15T02:58:58.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.010+0000 7f84755a9700 1 -- 172.21.15.121:0/3668162203 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f847006c150 con 0x7f8470155630 2023-11-15T02:58:58.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.010+0000 7f8466ffd700 1 -- 172.21.15.121:0/3668162203 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16989 (secure 0 0 0) 0x7f84680477a0 con 0x7f8470155630 2023-11-15T02:58:58.018 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:58:58.018 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":47,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:58:57.608659+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:58:55.530878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-15T02:58:56.602591+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:58:58.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.014+0000 7f84755a9700 1 -- 172.21.15.121:0/3668162203 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f845005a510 msgr2=0x7f845005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:58.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.014+0000 7f84755a9700 1 --2- 172.21.15.121:0/3668162203 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f845005a510 0x7f845005c9d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f8460009130 tx=0x7f8460017000).stop 2023-11-15T02:58:58.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.014+0000 7f84755a9700 1 -- 172.21.15.121:0/3668162203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8470155630 msgr2=0x7f847014bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:58.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.014+0000 7f84755a9700 1 --2- 172.21.15.121:0/3668162203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8470155630 0x7f847014bd20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f846800b840 tx=0x7f8468009270).stop 2023-11-15T02:58:58.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.014+0000 7f84755a9700 1 -- 172.21.15.121:0/3668162203 shutdown_connections 2023-11-15T02:58:58.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.014+0000 7f84755a9700 1 --2- 172.21.15.121:0/3668162203 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f845005a510 0x7f845005c9d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:58.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.014+0000 7f84755a9700 1 --2- 172.21.15.121:0/3668162203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8470155630 0x7f847014bd20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:58.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.014+0000 7f84755a9700 1 --2- 172.21.15.121:0/3668162203 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84701517c0 0x7f847014f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:58.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.014+0000 7f84755a9700 1 --2- 172.21.15.121:0/3668162203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84700ab2c0 0x7f84700abb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:58.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.014+0000 7f84755a9700 1 -- 172.21.15.121:0/3668162203 >> 172.21.15.121:0/3668162203 conn(0x7f84700c89c0 msgr2=0x7f847014c710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:58.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.014+0000 7f84755a9700 1 -- 172.21.15.121:0/3668162203 shutdown_connections 2023-11-15T02:58:58.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.014+0000 7f84755a9700 1 -- 172.21.15.121:0/3668162203 wait complete. 2023-11-15T02:58:58.023 INFO:tasks.ceph.mds.c.smithi121.stderr:2023-11-15T02:58:58.014+0000 7feaa9e67700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T02:58:58.042 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:58:58.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.190+0000 7f02f161b700 1 Processor -- start 2023-11-15T02:58:58.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.190+0000 7f02f161b700 1 -- start start 2023-11-15T02:58:58.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.190+0000 7f02f161b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ec151d00 0x7f02ec0c4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:58.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.190+0000 7f02f161b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f02ec0c4c10 0x7f02ec0a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:58.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.190+0000 7f02f161b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02ec068430 con 0x7f02ec151920 2023-11-15T02:58:58.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.190+0000 7f02f161b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ec0c50a0 con 0x7f02ec0c4c10 2023-11-15T02:58:58.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.190+0000 7f02f161b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec064b50 con 0x7f02ec151d00 2023-11-15T02:58:58.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.190+0000 7f02eaffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f02ec151920 0x7f02ec063fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44278/0 (socket says 172.21.15.121:44278) 2023-11-15T02:58:58.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.190+0000 7f02eaffd700 1 -- 172.21.15.121:0/58251799 learned_addr learned my addr 172.21.15.121:0/58251799 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:58.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02ebfff700 1 --2- 172.21.15.121:0/58251799 >> v2:172.21.15.190:3300/0 conn(0x7f02ec0c4c10 0x7f02ec0a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:58.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02e3fff700 1 --2- 172.21.15.121:0/58251799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ec151d00 0x7f02ec0c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:58.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02ea7fc700 1 -- 172.21.15.121:0/58251799 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936049100 0 0) 0x7f02ec068430 con 0x7f02ec151920 2023-11-15T02:58:58.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02ea7fc700 1 -- 172.21.15.121:0/58251799 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02d80033f0 con 0x7f02ec151920 2023-11-15T02:58:58.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02ebfff700 1 -- 172.21.15.121:0/58251799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ec151d00 msgr2=0x7f02ec0c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:58.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02ebfff700 1 --2- 172.21.15.121:0/58251799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ec151d00 0x7f02ec0c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:58.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02ebfff700 1 -- 172.21.15.121:0/58251799 >> v1:172.21.15.121:6789/0 conn(0x7f02ec151920 legacy=0x7f02ec063fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:58.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02ebfff700 1 -- 172.21.15.121:0/58251799 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ec0bec30 con 0x7f02ec0c4c10 2023-11-15T02:58:58.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02e3fff700 1 --2- 172.21.15.121:0/58251799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ec151d00 0x7f02ec0c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:58:58.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02ebfff700 1 --2- 172.21.15.121:0/58251799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02ec0c4c10 0x7f02ec0a4b00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f02e4009e60 tx=0x7f02e4009780).ready entity=mon.1 client_cookie=46cee4ca56e2cf9e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:58.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02ea7fc700 1 -- 172.21.15.121:0/58251799 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e400e070 con 0x7f02ec0c4c10 2023-11-15T02:58:58.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02ea7fc700 1 -- 172.21.15.121:0/58251799 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e400bab0 con 0x7f02ec0c4c10 2023-11-15T02:58:58.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02ea7fc700 1 -- 172.21.15.121:0/58251799 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e400be30 con 0x7f02ec0c4c10 2023-11-15T02:58:58.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02f161b700 1 -- 172.21.15.121:0/58251799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02ec0c4c10 msgr2=0x7f02ec0a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:58.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02f161b700 1 --2- 172.21.15.121:0/58251799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02ec0c4c10 0x7f02ec0a4b00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f02e4009e60 tx=0x7f02e4009780).stop 2023-11-15T02:58:58.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02f161b700 1 -- 172.21.15.121:0/58251799 shutdown_connections 2023-11-15T02:58:58.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02f161b700 1 --2- 172.21.15.121:0/58251799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02ec0c4c10 0x7f02ec0a4b00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:58.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02f161b700 1 --2- 172.21.15.121:0/58251799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ec151d00 0x7f02ec0c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:58.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02f161b700 1 -- 172.21.15.121:0/58251799 >> 172.21.15.121:0/58251799 conn(0x7f02ec069ac0 msgr2=0x7f02ec090a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:58.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02f161b700 1 -- 172.21.15.121:0/58251799 shutdown_connections 2023-11-15T02:58:58.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.194+0000 7f02f161b700 1 -- 172.21.15.121:0/58251799 wait complete. 2023-11-15T02:58:58.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.198+0000 7f02f161b700 1 Processor -- start 2023-11-15T02:58:58.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.198+0000 7f02f161b700 1 -- start start 2023-11-15T02:58:58.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.198+0000 7f02f161b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02ec0c4c10 0x7f02ec1427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:58.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.198+0000 7f02f161b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ec151920 0x7f02ec0acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:58.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.198+0000 7f02f161b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02ec151d00 0x7f02ec0ad270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:58.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.198+0000 7f02f161b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec0ad760 con 0x7f02ec0c4c10 2023-11-15T02:58:58.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.198+0000 7f02f161b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec167840 con 0x7f02ec151d00 2023-11-15T02:58:58.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.198+0000 7f02f161b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec142cb0 con 0x7f02ec151920 2023-11-15T02:58:58.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.198+0000 7f02eaffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02ec0c4c10 0x7f02ec1427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:58.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.198+0000 7f02eaffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02ec0c4c10 0x7f02ec1427c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37980/0 (socket says 172.21.15.121:37980) 2023-11-15T02:58:58.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.198+0000 7f02eaffd700 1 -- 172.21.15.121:0/3931861186 learned_addr learned my addr 172.21.15.121:0/3931861186 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:58.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.198+0000 7f02ebfff700 1 --2- 172.21.15.121:0/3931861186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02ec151d00 0x7f02ec0ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:58.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.198+0000 7f02e3fff700 1 --2- 172.21.15.121:0/3931861186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ec151920 0x7f02ec0acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:58.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.202+0000 7f02eaffd700 1 -- 172.21.15.121:0/3931861186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ec151920 msgr2=0x7f02ec0acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:58.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.202+0000 7f02eaffd700 1 --2- 172.21.15.121:0/3931861186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ec151920 0x7f02ec0acd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:58.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.202+0000 7f02eaffd700 1 -- 172.21.15.121:0/3931861186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02ec151d00 msgr2=0x7f02ec0ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:58.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.202+0000 7f02eaffd700 1 --2- 172.21.15.121:0/3931861186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02ec151d00 0x7f02ec0ad270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:58.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.202+0000 7f02eaffd700 1 -- 172.21.15.121:0/3931861186 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d40092a0 con 0x7f02ec0c4c10 2023-11-15T02:58:58.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.202+0000 7f02eaffd700 1 --2- 172.21.15.121:0/3931861186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02ec0c4c10 0x7f02ec1427c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f02dc00c590 tx=0x7f02dc007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:58.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.202+0000 7f02e2ffd700 1 -- 172.21.15.121:0/3931861186 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc0072e0 con 0x7f02ec0c4c10 2023-11-15T02:58:58.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.202+0000 7f02e2ffd700 1 -- 172.21.15.121:0/3931861186 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02dc007440 con 0x7f02ec0c4c10 2023-11-15T02:58:58.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.202+0000 7f02f161b700 1 -- 172.21.15.121:0/3931861186 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e4009a20 con 0x7f02ec0c4c10 2023-11-15T02:58:58.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.202+0000 7f02e2ffd700 1 -- 172.21.15.121:0/3931861186 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc0057a0 con 0x7f02ec0c4c10 2023-11-15T02:58:58.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.202+0000 7f02f161b700 1 -- 172.21.15.121:0/3931861186 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ec1671f0 con 0x7f02ec0c4c10 2023-11-15T02:58:58.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.202+0000 7f02f161b700 1 -- 172.21.15.121:0/3931861186 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ec167050 con 0x7f02ec0c4c10 2023-11-15T02:58:58.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.206+0000 7f02e2ffd700 1 -- 172.21.15.121:0/3931861186 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f02dc005bd0 con 0x7f02ec0c4c10 2023-11-15T02:58:58.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.206+0000 7f02e2ffd700 1 --2- 172.21.15.121:0/3931861186 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f02cc05a510 0x7f02cc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:58.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.206+0000 7f02e2ffd700 1 -- 172.21.15.121:0/3931861186 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f02dc073d40 con 0x7f02ec0c4c10 2023-11-15T02:58:58.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.206+0000 7f02e3fff700 1 --2- 172.21.15.121:0/3931861186 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f02cc05a510 0x7f02cc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:58.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.206+0000 7f02e3fff700 1 --2- 172.21.15.121:0/3931861186 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f02cc05a510 0x7f02cc05c9d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f02d4009270 tx=0x7f02d4008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:58.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.214+0000 7f02e2ffd700 1 -- 172.21.15.121:0/3931861186 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f02ec167050 con 0x7f02ec0c4c10 2023-11-15T02:58:58.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.374+0000 7f02f161b700 1 -- 172.21.15.121:0/3931861186 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02ec167050 con 0x7f02ec0c4c10 2023-11-15T02:58:58.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.374+0000 7f02e2ffd700 1 -- 172.21.15.121:0/3931861186 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16989 (secure 0 0 0) 0x7f02ec167050 con 0x7f02ec0c4c10 2023-11-15T02:58:58.383 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:58:58.383 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":47,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:58:57.608659+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:58:55.530878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-15T02:58:56.602591+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:58:58.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.378+0000 7f02f161b700 1 -- 172.21.15.121:0/3931861186 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f02cc05a510 msgr2=0x7f02cc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:58.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.378+0000 7f02f161b700 1 --2- 172.21.15.121:0/3931861186 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f02cc05a510 0x7f02cc05c9d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f02d4009270 tx=0x7f02d4008040).stop 2023-11-15T02:58:58.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.378+0000 7f02f161b700 1 -- 172.21.15.121:0/3931861186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02ec0c4c10 msgr2=0x7f02ec1427c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:58.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.378+0000 7f02f161b700 1 --2- 172.21.15.121:0/3931861186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02ec0c4c10 0x7f02ec1427c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f02dc00c590 tx=0x7f02dc007c80).stop 2023-11-15T02:58:58.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.378+0000 7f02f161b700 1 -- 172.21.15.121:0/3931861186 shutdown_connections 2023-11-15T02:58:58.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.378+0000 7f02f161b700 1 --2- 172.21.15.121:0/3931861186 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f02cc05a510 0x7f02cc05c9d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:58.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.378+0000 7f02f161b700 1 --2- 172.21.15.121:0/3931861186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02ec151d00 0x7f02ec0ad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:58.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.378+0000 7f02f161b700 1 --2- 172.21.15.121:0/3931861186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ec151920 0x7f02ec0acd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:58.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.378+0000 7f02f161b700 1 --2- 172.21.15.121:0/3931861186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02ec0c4c10 0x7f02ec1427c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:58.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.378+0000 7f02f161b700 1 -- 172.21.15.121:0/3931861186 >> 172.21.15.121:0/3931861186 conn(0x7f02ec069ac0 msgr2=0x7f02ec1507b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:58.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.378+0000 7f02f161b700 1 -- 172.21.15.121:0/3931861186 shutdown_connections 2023-11-15T02:58:58.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.378+0000 7f02f161b700 1 -- 172.21.15.121:0/3931861186 wait complete. 2023-11-15T02:58:58.404 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-15T02:58:58.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.558+0000 7fbf75507700 1 Processor -- start 2023-11-15T02:58:58.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.558+0000 7fbf75507700 1 -- start start 2023-11-15T02:58:58.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.562+0000 7fbf75507700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf700c5c70 0x7fbf70153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:58.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.562+0000 7fbf75507700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbf70066330 0x7fbf700ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:58.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.562+0000 7fbf75507700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf70065ea0 con 0x7fbf700acd00 2023-11-15T02:58:58.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.562+0000 7fbf75507700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf70066bb0 con 0x7fbf70066330 2023-11-15T02:58:58.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.562+0000 7fbf75507700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf70065890 con 0x7fbf700c5c70 2023-11-15T02:58:58.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.562+0000 7fbf6ffff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fbf700acd00 0x7fbf701536c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44294/0 (socket says 172.21.15.121:44294) 2023-11-15T02:58:58.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.562+0000 7fbf6ffff700 1 -- 172.21.15.121:0/40877059 learned_addr learned my addr 172.21.15.121:0/40877059 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:58.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.562+0000 7fbf6f7fe700 1 --2- 172.21.15.121:0/40877059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf700c5c70 0x7fbf70153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:58.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.562+0000 7fbf74d06700 1 --2- 172.21.15.121:0/40877059 >> v2:172.21.15.190:3300/0 conn(0x7fbf70066330 0x7fbf700ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:58.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.562+0000 7fbf6effd700 1 -- 172.21.15.121:0/40877059 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217897413 0 0) 0x7fbf70065ea0 con 0x7fbf700acd00 2023-11-15T02:58:58.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.562+0000 7fbf6effd700 1 -- 172.21.15.121:0/40877059 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf580033f0 con 0x7fbf700acd00 2023-11-15T02:58:58.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.562+0000 7fbf6effd700 1 -- 172.21.15.121:0/40877059 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1536071182 0 0) 0x7fbf580033f0 con 0x7fbf700acd00 2023-11-15T02:58:58.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.562+0000 7fbf6effd700 1 -- 172.21.15.121:0/40877059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf700c5c70 msgr2=0x7fbf70153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:58.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.562+0000 7fbf6effd700 1 --2- 172.21.15.121:0/40877059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf700c5c70 0x7fbf70153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:58.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.562+0000 7fbf6effd700 1 -- 172.21.15.121:0/40877059 >> v2:172.21.15.190:3300/0 conn(0x7fbf70066330 msgr2=0x7fbf700ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:58.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.562+0000 7fbf6effd700 1 --2- 172.21.15.121:0/40877059 >> v2:172.21.15.190:3300/0 conn(0x7fbf70066330 0x7fbf700ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:58.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.562+0000 7fbf6effd700 1 -- 172.21.15.121:0/40877059 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf700c64a0 con 0x7fbf700acd00 2023-11-15T02:58:58.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.562+0000 7fbf6effd700 1 -- 172.21.15.121:0/40877059 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fbf600015f0 con 0x7fbf700acd00 2023-11-15T02:58:58.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.562+0000 7fbf6effd700 1 -- 172.21.15.121:0/40877059 >> v1:172.21.15.121:6789/0 conn(0x7fbf700acd00 legacy=0x7fbf701536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:58.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf6effd700 1 --2- 172.21.15.121:0/40877059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf700c5c70 0x7fbf580038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:58.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf6effd700 1 --2- 172.21.15.121:0/40877059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf58003e10 0x7fbf58004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:58.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf6effd700 1 --2- 172.21.15.121:0/40877059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf58004b80 0x7fbf58005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:58.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf6effd700 1 -- 172.21.15.121:0/40877059 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf70066bb0 con 0x7fbf58003e10 2023-11-15T02:58:58.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf6effd700 1 -- 172.21.15.121:0/40877059 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf70065890 con 0x7fbf58004b80 2023-11-15T02:58:58.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf6effd700 1 -- 172.21.15.121:0/40877059 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5800b8e0 con 0x7fbf700c5c70 2023-11-15T02:58:58.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf6f7fe700 1 --2- 172.21.15.121:0/40877059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf58003e10 0x7fbf58004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:58.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf6ffff700 1 --2- 172.21.15.121:0/40877059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf700c5c70 0x7fbf580038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:58.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf74d06700 1 --2- 172.21.15.121:0/40877059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf58004b80 0x7fbf58005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:58.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf6f7fe700 1 -- 172.21.15.121:0/40877059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf700c5c70 msgr2=0x7fbf580038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:58.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf6f7fe700 1 --2- 172.21.15.121:0/40877059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf700c5c70 0x7fbf580038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:58.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf6f7fe700 1 -- 172.21.15.121:0/40877059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf58004b80 msgr2=0x7fbf58005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:58.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf6f7fe700 1 --2- 172.21.15.121:0/40877059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf58004b80 0x7fbf58005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:58.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf6f7fe700 1 -- 172.21.15.121:0/40877059 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf580033f0 con 0x7fbf58003e10 2023-11-15T02:58:58.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf6f7fe700 1 --2- 172.21.15.121:0/40877059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf58003e10 0x7fbf58004690 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fbf64009ab0 tx=0x7fbf64009060).ready entity=mon.0 client_cookie=1b24d958b3dfbc39 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:58.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf6effd700 1 -- 172.21.15.121:0/40877059 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf64009430 con 0x7fbf58003e10 2023-11-15T02:58:58.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf6effd700 1 -- 172.21.15.121:0/40877059 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6400e640 con 0x7fbf58003e10 2023-11-15T02:58:58.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf6effd700 1 -- 172.21.15.121:0/40877059 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6400e9c0 con 0x7fbf58003e10 2023-11-15T02:58:58.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf75507700 1 -- 172.21.15.121:0/40877059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf58003e10 msgr2=0x7fbf58004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:58.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf75507700 1 --2- 172.21.15.121:0/40877059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf58003e10 0x7fbf58004690 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fbf64009ab0 tx=0x7fbf64009060).stop 2023-11-15T02:58:58.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf74d06700 1 -- 172.21.15.121:0/40877059 reap_dead start 2023-11-15T02:58:58.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf75507700 1 -- 172.21.15.121:0/40877059 shutdown_connections 2023-11-15T02:58:58.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf75507700 1 -- 172.21.15.121:0/40877059 >> 172.21.15.121:0/40877059 conn(0x7fbf70063fc0 msgr2=0x7fbf70067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:58.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf75507700 1 -- 172.21.15.121:0/40877059 shutdown_connections 2023-11-15T02:58:58.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.566+0000 7fbf75507700 1 -- 172.21.15.121:0/40877059 wait complete. 2023-11-15T02:58:58.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.570+0000 7fbf75507700 1 Processor -- start 2023-11-15T02:58:58.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.570+0000 7fbf75507700 1 -- start start 2023-11-15T02:58:58.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.570+0000 7fbf75507700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf58003e10 0x7fbf70147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:58.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.570+0000 7fbf75507700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf700c2fe0 0x7fbf70144f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:58.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.570+0000 7fbf75507700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf70143b70 0x7fbf70143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:58.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.570+0000 7fbf75507700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf700c3440 con 0x7fbf58003e10 2023-11-15T02:58:58.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.570+0000 7fbf75507700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf70146400 con 0x7fbf70143b70 2023-11-15T02:58:58.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.570+0000 7fbf75507700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf701457f0 con 0x7fbf700c2fe0 2023-11-15T02:58:58.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.574+0000 7fbf6ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf58003e10 0x7fbf70147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:58.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.574+0000 7fbf6ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf58003e10 0x7fbf70147b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37996/0 (socket says 172.21.15.121:37996) 2023-11-15T02:58:58.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.574+0000 7fbf6ffff700 1 -- 172.21.15.121:0/3249596309 learned_addr learned my addr 172.21.15.121:0/3249596309 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:58.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.574+0000 7fbf6f7fe700 1 --2- 172.21.15.121:0/3249596309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf700c2fe0 0x7fbf70144f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:58.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.574+0000 7fbf74d06700 1 --2- 172.21.15.121:0/3249596309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf70143b70 0x7fbf70143ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:58.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.574+0000 7fbf6ffff700 1 -- 172.21.15.121:0/3249596309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf700c2fe0 msgr2=0x7fbf70144f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:58.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.574+0000 7fbf6ffff700 1 --2- 172.21.15.121:0/3249596309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf700c2fe0 0x7fbf70144f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:58.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.574+0000 7fbf6ffff700 1 -- 172.21.15.121:0/3249596309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf70143b70 msgr2=0x7fbf70143ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:58.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.574+0000 7fbf6ffff700 1 --2- 172.21.15.121:0/3249596309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf70143b70 0x7fbf70143ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:58.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.574+0000 7fbf6ffff700 1 -- 172.21.15.121:0/3249596309 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf70145ad0 con 0x7fbf58003e10 2023-11-15T02:58:58.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.574+0000 7fbf6ffff700 1 --2- 172.21.15.121:0/3249596309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf58003e10 0x7fbf70147b60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fbf60001020 tx=0x7fbf60009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:58.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.574+0000 7fbf57fff700 1 -- 172.21.15.121:0/3249596309 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6000fee0 con 0x7fbf58003e10 2023-11-15T02:58:58.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.574+0000 7fbf57fff700 1 -- 172.21.15.121:0/3249596309 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf600070d0 con 0x7fbf58003e10 2023-11-15T02:58:58.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.574+0000 7fbf57fff700 1 -- 172.21.15.121:0/3249596309 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60007450 con 0x7fbf58003e10 2023-11-15T02:58:58.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.574+0000 7fbf75507700 1 -- 172.21.15.121:0/3249596309 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf70145db0 con 0x7fbf58003e10 2023-11-15T02:58:58.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.574+0000 7fbf75507700 1 -- 172.21.15.121:0/3249596309 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf70145480 con 0x7fbf58003e10 2023-11-15T02:58:58.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.574+0000 7fbf57fff700 1 -- 172.21.15.121:0/3249596309 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fbf60005a90 con 0x7fbf58003e10 2023-11-15T02:58:58.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.574+0000 7fbf75507700 1 -- 172.21.15.121:0/3249596309 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf70146230 con 0x7fbf58003e10 2023-11-15T02:58:58.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.578+0000 7fbf57fff700 1 --2- 172.21.15.121:0/3249596309 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbf5c05a4e0 0x7fbf5c05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:58.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.578+0000 7fbf57fff700 1 -- 172.21.15.121:0/3249596309 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fbf600740a0 con 0x7fbf58003e10 2023-11-15T02:58:58.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.578+0000 7fbf6f7fe700 1 --2- 172.21.15.121:0/3249596309 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbf5c05a4e0 0x7fbf5c05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:58.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.578+0000 7fbf6f7fe700 1 --2- 172.21.15.121:0/3249596309 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbf5c05a4e0 0x7fbf5c05c9a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fbf70167890 tx=0x7fbf6401a000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:58.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.582+0000 7fbf57fff700 1 -- 172.21.15.121:0/3249596309 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbf600478d0 con 0x7fbf58003e10 2023-11-15T02:58:58.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:58.786+0000 7fbf75507700 1 -- 172.21.15.121:0/3249596309 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbf7006bcf0 con 0x7fbf58003e10 2023-11-15T02:58:58.841 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-11-15T02:58:58.842 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-15T02:58:58.842 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-11-15T02:58:59.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.618+0000 7fbf57fff700 1 -- 172.21.15.121:0/3249596309 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v29) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbf6004aef0 con 0x7fbf58003e10 2023-11-15T02:58:59.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.622+0000 7fbf75507700 1 -- 172.21.15.121:0/3249596309 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbf5c05a4e0 msgr2=0x7fbf5c05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:59.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.622+0000 7fbf75507700 1 --2- 172.21.15.121:0/3249596309 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbf5c05a4e0 0x7fbf5c05c9a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fbf70167890 tx=0x7fbf6401a000).stop 2023-11-15T02:58:59.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.622+0000 7fbf75507700 1 -- 172.21.15.121:0/3249596309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf58003e10 msgr2=0x7fbf70147b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:59.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.622+0000 7fbf75507700 1 --2- 172.21.15.121:0/3249596309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf58003e10 0x7fbf70147b60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fbf60001020 tx=0x7fbf60009040).stop 2023-11-15T02:58:59.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.626+0000 7fbf75507700 1 -- 172.21.15.121:0/3249596309 shutdown_connections 2023-11-15T02:58:59.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.626+0000 7fbf75507700 1 --2- 172.21.15.121:0/3249596309 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbf5c05a4e0 0x7fbf5c05c9a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:59.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.626+0000 7fbf75507700 1 --2- 172.21.15.121:0/3249596309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf70143b70 0x7fbf70143ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:59.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.626+0000 7fbf75507700 1 --2- 172.21.15.121:0/3249596309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf700c2fe0 0x7fbf70144f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:59.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.626+0000 7fbf75507700 1 --2- 172.21.15.121:0/3249596309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf58003e10 0x7fbf70147b60 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:59.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.626+0000 7fbf75507700 1 -- 172.21.15.121:0/3249596309 >> 172.21.15.121:0/3249596309 conn(0x7fbf70063fc0 msgr2=0x7fbf700cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:59.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.626+0000 7fbf75507700 1 -- 172.21.15.121:0/3249596309 shutdown_connections 2023-11-15T02:58:59.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.626+0000 7fbf75507700 1 -- 172.21.15.121:0/3249596309 wait complete. 2023-11-15T02:58:59.654 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-15T02:58:59.654 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-15T02:58:59.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.814+0000 7f41e1e5b700 1 Processor -- start 2023-11-15T02:58:59.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.814+0000 7f41e1e5b700 1 -- start start 2023-11-15T02:58:59.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.814+0000 7f41e1e5b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f41dc153b90 0x7f41dc062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:59.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41e1e5b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41dc158200 con 0x7f41dc153f70 2023-11-15T02:58:59.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41e1e5b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41dc0c7220 con 0x7f41dc068430 2023-11-15T02:58:59.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41e1e5b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41dc063680 con 0x7f41dc153b90 2023-11-15T02:58:59.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41daffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f41dc153f70 0x7f41dc067d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44298/0 (socket says 172.21.15.121:44298) 2023-11-15T02:58:59.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41daffd700 1 -- 172.21.15.121:0/319958076 learned_addr learned my addr 172.21.15.121:0/319958076 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:58:59.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41db7fe700 1 --2- 172.21.15.121:0/319958076 >> v2:172.21.15.121:3300/0 conn(0x7f41dc153b90 0x7f41dc062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:59.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41da7fc700 1 -- 172.21.15.121:0/319958076 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1292077333 0 0) 0x7f41dc158200 con 0x7f41dc153f70 2023-11-15T02:58:59.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41da7fc700 1 -- 172.21.15.121:0/319958076 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41c40033f0 con 0x7f41dc153f70 2023-11-15T02:58:59.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41da7fc700 1 -- 172.21.15.121:0/319958076 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1012954842 0 0) 0x7f41c40033f0 con 0x7f41dc153f70 2023-11-15T02:58:59.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41da7fc700 1 -- 172.21.15.121:0/319958076 >> v2:172.21.15.121:3300/0 conn(0x7f41dc153b90 msgr2=0x7f41dc062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:59.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41da7fc700 1 --2- 172.21.15.121:0/319958076 >> v2:172.21.15.121:3300/0 conn(0x7f41dc153b90 0x7f41dc062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:59.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41da7fc700 1 -- 172.21.15.121:0/319958076 >> v1:172.21.15.190:6789/0 conn(0x7f41dc068430 legacy=0x7f41dc0c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:59.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41da7fc700 1 -- 172.21.15.121:0/319958076 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc1578d0 con 0x7f41dc153f70 2023-11-15T02:58:59.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41da7fc700 1 -- 172.21.15.121:0/319958076 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f41d0001470 con 0x7f41dc153f70 2023-11-15T02:58:59.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41da7fc700 1 -- 172.21.15.121:0/319958076 >> v1:172.21.15.121:6789/0 conn(0x7f41dc153f70 legacy=0x7f41dc067d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:59.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41da7fc700 1 --2- 172.21.15.121:0/319958076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41c40038e0 0x7f41c4003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:59.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41da7fc700 1 --2- 172.21.15.121:0/319958076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41c4004690 0x7f41c4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:59.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41da7fc700 1 --2- 172.21.15.121:0/319958076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41c4005000 0x7f41c400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:59.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41da7fc700 1 -- 172.21.15.121:0/319958076 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc063680 con 0x7f41c4005000 2023-11-15T02:58:59.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41da7fc700 1 -- 172.21.15.121:0/319958076 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c400beb0 con 0x7f41c4004690 2023-11-15T02:58:59.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41da7fc700 1 -- 172.21.15.121:0/319958076 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c400c070 con 0x7f41c40038e0 2023-11-15T02:58:59.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41dbfff700 1 --2- 172.21.15.121:0/319958076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41c4005000 0x7f41c400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:59.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41db7fe700 1 --2- 172.21.15.121:0/319958076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41c40038e0 0x7f41c4003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:59.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41daffd700 1 --2- 172.21.15.121:0/319958076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41c4004690 0x7f41c4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:59.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41dbfff700 1 -- 172.21.15.121:0/319958076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41c40038e0 msgr2=0x7f41c4003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:59.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41dbfff700 1 --2- 172.21.15.121:0/319958076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41c40038e0 0x7f41c4003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:59.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41dbfff700 1 -- 172.21.15.121:0/319958076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41c4004690 msgr2=0x7f41c4004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:58:59.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41dbfff700 1 --2- 172.21.15.121:0/319958076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41c4004690 0x7f41c4004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:59.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41dbfff700 1 -- 172.21.15.121:0/319958076 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c40033f0 con 0x7f41c4005000 2023-11-15T02:58:59.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41dbfff700 1 -- 172.21.15.121:0/319958076 reap_dead start 2023-11-15T02:58:59.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41dbfff700 1 --2- 172.21.15.121:0/319958076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41c4005000 0x7f41c400b8e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f41d4001ef0 tx=0x7f41d4007f10).ready entity=mon.0 client_cookie=bd9e2bdec0312bb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:59.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.818+0000 7f41da7fc700 1 -- 172.21.15.121:0/319958076 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d40017e0 con 0x7f41c4005000 2023-11-15T02:58:59.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.822+0000 7f41da7fc700 1 -- 172.21.15.121:0/319958076 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d400c6a0 con 0x7f41c4005000 2023-11-15T02:58:59.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.822+0000 7f41da7fc700 1 -- 172.21.15.121:0/319958076 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d400ca40 con 0x7f41c4005000 2023-11-15T02:58:59.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.822+0000 7f41e1e5b700 1 -- 172.21.15.121:0/319958076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41c4005000 msgr2=0x7f41c400b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:59.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.822+0000 7f41e1e5b700 1 --2- 172.21.15.121:0/319958076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41c4005000 0x7f41c400b8e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f41d4001ef0 tx=0x7f41d4007f10).stop 2023-11-15T02:58:59.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.822+0000 7f41e1e5b700 1 -- 172.21.15.121:0/319958076 shutdown_connections 2023-11-15T02:58:59.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.822+0000 7f41e1e5b700 1 --2- 172.21.15.121:0/319958076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41c4005000 0x7f41c400b8e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:59.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.822+0000 7f41e1e5b700 1 -- 172.21.15.121:0/319958076 >> 172.21.15.121:0/319958076 conn(0x7f41dc0cad00 msgr2=0x7f41dc0cb110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:58:59.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.822+0000 7f41e1e5b700 1 -- 172.21.15.121:0/319958076 shutdown_connections 2023-11-15T02:58:59.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.822+0000 7f41e1e5b700 1 -- 172.21.15.121:0/319958076 wait complete. 2023-11-15T02:58:59.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.822+0000 7f41e1e5b700 1 Processor -- start 2023-11-15T02:58:59.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41e1e5b700 1 -- start start 2023-11-15T02:58:59.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41e1e5b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41c4005000 0x7f41dc14e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:59.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41e1e5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41dc155680 0x7f41dc14f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:59.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41e1e5b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41dc155b00 0x7f41dc14f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:59.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41e1e5b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc14fe10 con 0x7f41c4005000 2023-11-15T02:58:59.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41e1e5b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc152df0 con 0x7f41dc155b00 2023-11-15T02:58:59.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41e1e5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc0af780 con 0x7f41dc155680 2023-11-15T02:58:59.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41db7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41c4005000 0x7f41dc14e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:59.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41db7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41c4005000 0x7f41dc14e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38014/0 (socket says 172.21.15.121:38014) 2023-11-15T02:58:59.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41db7fe700 1 -- 172.21.15.121:0/2897551602 learned_addr learned my addr 172.21.15.121:0/2897551602 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:58:59.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41daffd700 1 --2- 172.21.15.121:0/2897551602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41dc155680 0x7f41dc14f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:59.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41dbfff700 1 --2- 172.21.15.121:0/2897551602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41dc155b00 0x7f41dc14f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:59.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41db7fe700 1 -- 172.21.15.121:0/2897551602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41dc155680 msgr2=0x7f41dc14f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:59.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41db7fe700 1 --2- 172.21.15.121:0/2897551602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41dc155680 0x7f41dc14f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:59.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41db7fe700 1 -- 172.21.15.121:0/2897551602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41dc155b00 msgr2=0x7f41dc14f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:58:59.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41db7fe700 1 --2- 172.21.15.121:0/2897551602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41dc155b00 0x7f41dc14f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:58:59.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41db7fe700 1 -- 172.21.15.121:0/2897551602 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc0aee90 con 0x7f41c4005000 2023-11-15T02:58:59.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41db7fe700 1 --2- 172.21.15.121:0/2897551602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41c4005000 0x7f41dc14e6e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f41cc0115f0 tx=0x7f41cc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:59.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41c37fe700 1 -- 172.21.15.121:0/2897551602 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc0145f0 con 0x7f41c4005000 2023-11-15T02:58:59.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41c37fe700 1 -- 172.21.15.121:0/2897551602 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41cc014750 con 0x7f41c4005000 2023-11-15T02:58:59.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41e1e5b700 1 -- 172.21.15.121:0/2897551602 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41dc0af130 con 0x7f41c4005000 2023-11-15T02:58:59.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41c37fe700 1 -- 172.21.15.121:0/2897551602 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc014ad0 con 0x7f41c4005000 2023-11-15T02:58:59.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.826+0000 7f41e1e5b700 1 -- 172.21.15.121:0/2897551602 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41dc14ebd0 con 0x7f41c4005000 2023-11-15T02:58:59.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.830+0000 7f41e1e5b700 1 -- 172.21.15.121:0/2897551602 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41dc047060 con 0x7f41c4005000 2023-11-15T02:58:59.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.830+0000 7f41c37fe700 1 -- 172.21.15.121:0/2897551602 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f41cc015480 con 0x7f41c4005000 2023-11-15T02:58:59.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.830+0000 7f41c37fe700 1 --2- 172.21.15.121:0/2897551602 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41c805a540 0x7f41c805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:58:59.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.830+0000 7f41c37fe700 1 -- 172.21.15.121:0/2897551602 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f41cc074cd0 con 0x7f41c4005000 2023-11-15T02:58:59.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.834+0000 7f41daffd700 1 --2- 172.21.15.121:0/2897551602 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41c805a540 0x7f41c805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:58:59.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.834+0000 7f41daffd700 1 --2- 172.21.15.121:0/2897551602 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41c805a540 0x7f41c805ca00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f41dc0ca910 tx=0x7f41d0007040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:58:59.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:58:59.838+0000 7f41c37fe700 1 -- 172.21.15.121:0/2897551602 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f41cc048450 con 0x7f41c4005000 2023-11-15T02:59:00.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.082+0000 7f41e1e5b700 1 -- 172.21.15.121:0/2897551602 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f41dc157bb0 con 0x7f41c4005000 2023-11-15T02:59:00.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.630+0000 7f41c37fe700 1 -- 172.21.15.121:0/2897551602 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f41cc04ba70 con 0x7f41c4005000 2023-11-15T02:59:00.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.630+0000 7f41e1e5b700 1 -- 172.21.15.121:0/2897551602 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41c805a540 msgr2=0x7f41c805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:00.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.630+0000 7f41e1e5b700 1 --2- 172.21.15.121:0/2897551602 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41c805a540 0x7f41c805ca00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f41dc0ca910 tx=0x7f41d0007040).stop 2023-11-15T02:59:00.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.630+0000 7f41e1e5b700 1 -- 172.21.15.121:0/2897551602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41c4005000 msgr2=0x7f41dc14e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:00.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.630+0000 7f41e1e5b700 1 --2- 172.21.15.121:0/2897551602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41c4005000 0x7f41dc14e6e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f41cc0115f0 tx=0x7f41cc008040).stop 2023-11-15T02:59:00.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.630+0000 7f41e1e5b700 1 -- 172.21.15.121:0/2897551602 shutdown_connections 2023-11-15T02:59:00.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.630+0000 7f41e1e5b700 1 --2- 172.21.15.121:0/2897551602 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41c805a540 0x7f41c805ca00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:00.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.630+0000 7f41e1e5b700 1 --2- 172.21.15.121:0/2897551602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41dc155b00 0x7f41dc14f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:00.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.630+0000 7f41e1e5b700 1 --2- 172.21.15.121:0/2897551602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41dc155680 0x7f41dc14f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:00.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.630+0000 7f41e1e5b700 1 --2- 172.21.15.121:0/2897551602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41c4005000 0x7f41dc14e6e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:00.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.630+0000 7f41e1e5b700 1 -- 172.21.15.121:0/2897551602 >> 172.21.15.121:0/2897551602 conn(0x7f41dc0cad00 msgr2=0x7f41dc0c8970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:00.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.630+0000 7f41e1e5b700 1 -- 172.21.15.121:0/2897551602 shutdown_connections 2023-11-15T02:59:00.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.634+0000 7f41e1e5b700 1 -- 172.21.15.121:0/2897551602 wait complete. 2023-11-15T02:59:00.659 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-15T02:59:00.660 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:59:00.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.810+0000 7f502009b700 1 Processor -- start 2023-11-15T02:59:00.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.814+0000 7f502009b700 1 -- start start 2023-11-15T02:59:00.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.814+0000 7f502009b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f501814af80 0x7f50180c1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:00.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.814+0000 7f502009b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f50180c1bf0 0x7f5018150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:00.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.814+0000 7f502009b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5018152000 con 0x7f50181514e0 2023-11-15T02:59:00.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.814+0000 7f502009b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50180c1270 con 0x7f50180c1bf0 2023-11-15T02:59:00.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.814+0000 7f502009b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5018151a40 con 0x7f501814af80 2023-11-15T02:59:00.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.814+0000 7f501d636700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f50180c1bf0 0x7f5018150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:00.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.818+0000 7f501d636700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f50180c1bf0 0x7f5018150ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38016/0 (socket says 172.21.15.121:38016) 2023-11-15T02:59:00.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.818+0000 7f501d636700 1 -- 172.21.15.121:0/988695377 learned_addr learned my addr 172.21.15.121:0/988695377 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:00.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.818+0000 7f501de37700 1 --2- 172.21.15.121:0/988695377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f501814af80 0x7f50180c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:00.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.818+0000 7f501d636700 1 -- 172.21.15.121:0/988695377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f501814af80 msgr2=0x7f50180c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:00.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.818+0000 7f501d636700 1 --2- 172.21.15.121:0/988695377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f501814af80 0x7f50180c1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:00.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.818+0000 7f501d636700 1 -- 172.21.15.121:0/988695377 >> v1:172.21.15.190:6789/0 conn(0x7f50181514e0 legacy=0x7f50180c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:00.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.818+0000 7f501d636700 1 -- 172.21.15.121:0/988695377 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5018151d20 con 0x7f50180c1bf0 2023-11-15T02:59:00.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.818+0000 7f501d636700 1 --2- 172.21.15.121:0/988695377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50180c1bf0 0x7f5018150ff0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f5008006fd0 tx=0x7f50080090f0).ready entity=mon.0 client_cookie=ac768be183db91c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:00.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.818+0000 7f501ce35700 1 -- 172.21.15.121:0/988695377 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500800b580 con 0x7f50180c1bf0 2023-11-15T02:59:00.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.818+0000 7f501ce35700 1 -- 172.21.15.121:0/988695377 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500800b6e0 con 0x7f50180c1bf0 2023-11-15T02:59:00.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.818+0000 7f501ce35700 1 -- 172.21.15.121:0/988695377 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500800ba60 con 0x7f50180c1bf0 2023-11-15T02:59:00.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.818+0000 7f502009b700 1 -- 172.21.15.121:0/988695377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50180c1bf0 msgr2=0x7f5018150ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:00.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.818+0000 7f502009b700 1 --2- 172.21.15.121:0/988695377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50180c1bf0 0x7f5018150ff0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f5008006fd0 tx=0x7f50080090f0).stop 2023-11-15T02:59:00.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.818+0000 7f502009b700 1 -- 172.21.15.121:0/988695377 shutdown_connections 2023-11-15T02:59:00.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.818+0000 7f502009b700 1 --2- 172.21.15.121:0/988695377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50180c1bf0 0x7f5018150ff0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:00.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.818+0000 7f502009b700 1 --2- 172.21.15.121:0/988695377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f501814af80 0x7f50180c1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:00.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.818+0000 7f502009b700 1 -- 172.21.15.121:0/988695377 >> 172.21.15.121:0/988695377 conn(0x7f5018062190 msgr2=0x7f50180625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:00.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.818+0000 7f502009b700 1 -- 172.21.15.121:0/988695377 shutdown_connections 2023-11-15T02:59:00.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.818+0000 7f502009b700 1 -- 172.21.15.121:0/988695377 wait complete. 2023-11-15T02:59:00.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.822+0000 7f502009b700 1 Processor -- start 2023-11-15T02:59:00.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.822+0000 7f502009b700 1 -- start start 2023-11-15T02:59:00.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.822+0000 7f502009b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50180c1bf0 0x7f50181687c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:00.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.822+0000 7f502009b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f501814af80 0x7f50180c0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:00.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.822+0000 7f502009b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50181514e0 0x7f5018153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:00.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.822+0000 7f502009b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50181529f0 con 0x7f50180c1bf0 2023-11-15T02:59:00.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.822+0000 7f502009b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5018152cd0 con 0x7f50181514e0 2023-11-15T02:59:00.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.822+0000 7f502009b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5018152fb0 con 0x7f501814af80 2023-11-15T02:59:00.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.822+0000 7f501de37700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50180c1bf0 0x7f50181687c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:00.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.822+0000 7f501de37700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50180c1bf0 0x7f50181687c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38022/0 (socket says 172.21.15.121:38022) 2023-11-15T02:59:00.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.822+0000 7f501de37700 1 -- 172.21.15.121:0/194083023 learned_addr learned my addr 172.21.15.121:0/194083023 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:00.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.822+0000 7f501d636700 1 --2- 172.21.15.121:0/194083023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f501814af80 0x7f50180c0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:00.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.822+0000 7f501e638700 1 --2- 172.21.15.121:0/194083023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50181514e0 0x7f5018153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:00.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.826+0000 7f501de37700 1 -- 172.21.15.121:0/194083023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f501814af80 msgr2=0x7f50180c0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:00.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.826+0000 7f501de37700 1 --2- 172.21.15.121:0/194083023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f501814af80 0x7f50180c0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:00.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.826+0000 7f501de37700 1 -- 172.21.15.121:0/194083023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50181514e0 msgr2=0x7f5018153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:00.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.826+0000 7f501de37700 1 --2- 172.21.15.121:0/194083023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50181514e0 0x7f5018153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:00.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.826+0000 7f501de37700 1 -- 172.21.15.121:0/194083023 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5008009370 con 0x7f50180c1bf0 2023-11-15T02:59:00.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.826+0000 7f501de37700 1 --2- 172.21.15.121:0/194083023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50180c1bf0 0x7f50181687c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f501400e930 tx=0x7f5014009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:00.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.826+0000 7f500dffb700 1 -- 172.21.15.121:0/194083023 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501400eed0 con 0x7f50180c1bf0 2023-11-15T02:59:00.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.826+0000 7f500dffb700 1 -- 172.21.15.121:0/194083023 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501400c850 con 0x7f50180c1bf0 2023-11-15T02:59:00.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.826+0000 7f500dffb700 1 -- 172.21.15.121:0/194083023 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501400cbd0 con 0x7f50180c1bf0 2023-11-15T02:59:00.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.826+0000 7f502009b700 1 -- 172.21.15.121:0/194083023 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5018153290 con 0x7f50180c1bf0 2023-11-15T02:59:00.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.826+0000 7f502009b700 1 -- 172.21.15.121:0/194083023 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5018153f80 con 0x7f50180c1bf0 2023-11-15T02:59:00.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.826+0000 7f502009b700 1 -- 172.21.15.121:0/194083023 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5018153710 con 0x7f50180c1bf0 2023-11-15T02:59:00.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.830+0000 7f500dffb700 1 -- 172.21.15.121:0/194083023 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f501401b400 con 0x7f50180c1bf0 2023-11-15T02:59:00.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.830+0000 7f500dffb700 1 --2- 172.21.15.121:0/194083023 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4ffc05a510 0x7f4ffc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:00.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.830+0000 7f500dffb700 1 -- 172.21.15.121:0/194083023 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5014074e50 con 0x7f50180c1bf0 2023-11-15T02:59:00.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.830+0000 7f501d636700 1 --2- 172.21.15.121:0/194083023 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4ffc05a510 0x7f4ffc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:00.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.830+0000 7f501d636700 1 --2- 172.21.15.121:0/194083023 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4ffc05a510 0x7f4ffc05c9d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f50180c57f0 tx=0x7f5008017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:00.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:00.834+0000 7f500dffb700 1 -- 172.21.15.121:0/194083023 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5014048510 con 0x7f50180c1bf0 2023-11-15T02:59:01.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.038+0000 7f502009b700 1 -- 172.21.15.121:0/194083023 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5018065240 con 0x7f50180c1bf0 2023-11-15T02:59:01.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.038+0000 7f500dffb700 1 -- 172.21.15.121:0/194083023 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4501 (secure 0 0 0) 0x7f501404bb30 con 0x7f50180c1bf0 2023-11-15T02:59:01.044 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:01.044 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":30,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3403224816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3403224816},{"type":"v1","addr":"172.21.15.190:6835","nonce":3403224816}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2088953798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2088953798},{"type":"v1","addr":"172.21.15.190:6833","nonce":2088953798}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5511,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2190514138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2190514138},{"type":"v1","addr":"172.21.15.121:6835","nonce":2190514138}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:58:57.609524+0000","modified":"2023-11-15T02:59:00.621955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5514":{"gid":5514,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6837/1945477051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1945477051},{"type":"v1","addr":"172.21.15.121:6837","nonce":1945477051}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-15T02:59:01.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.038+0000 7f502009b700 1 -- 172.21.15.121:0/194083023 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4ffc05a510 msgr2=0x7f4ffc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:01.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.038+0000 7f502009b700 1 --2- 172.21.15.121:0/194083023 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4ffc05a510 0x7f4ffc05c9d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f50180c57f0 tx=0x7f5008017000).stop 2023-11-15T02:59:01.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.038+0000 7f502009b700 1 -- 172.21.15.121:0/194083023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50180c1bf0 msgr2=0x7f50181687c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:01.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.038+0000 7f502009b700 1 --2- 172.21.15.121:0/194083023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50180c1bf0 0x7f50181687c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f501400e930 tx=0x7f5014009060).stop 2023-11-15T02:59:01.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.038+0000 7f502009b700 1 -- 172.21.15.121:0/194083023 shutdown_connections 2023-11-15T02:59:01.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.038+0000 7f502009b700 1 --2- 172.21.15.121:0/194083023 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4ffc05a510 0x7f4ffc05c9d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.038+0000 7f502009b700 1 --2- 172.21.15.121:0/194083023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50181514e0 0x7f5018153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.038+0000 7f502009b700 1 --2- 172.21.15.121:0/194083023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f501814af80 0x7f50180c0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.038+0000 7f502009b700 1 --2- 172.21.15.121:0/194083023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50180c1bf0 0x7f50181687c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.038+0000 7f502009b700 1 -- 172.21.15.121:0/194083023 >> 172.21.15.121:0/194083023 conn(0x7f5018062190 msgr2=0x7f50180c3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:01.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.038+0000 7f502009b700 1 -- 172.21.15.121:0/194083023 shutdown_connections 2023-11-15T02:59:01.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.038+0000 7f502009b700 1 -- 172.21.15.121:0/194083023 wait complete. 2023-11-15T02:59:01.049 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 30 2023-11-15T02:59:01.064 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:59:01.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.218+0000 7fa7408f1700 1 Processor -- start 2023-11-15T02:59:01.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.218+0000 7fa7408f1700 1 -- start start 2023-11-15T02:59:01.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.222+0000 7fa7408f1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa73c150ff0 0x7fa73c151470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:01.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.222+0000 7fa7408f1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa73c152000 con 0x7fa73c0c6640 2023-11-15T02:59:01.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.222+0000 7fa7408f1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa73c0c1270 con 0x7fa73c14af80 2023-11-15T02:59:01.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.222+0000 7fa7408f1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa73c0c1e10 con 0x7fa73c150ff0 2023-11-15T02:59:01.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.222+0000 7fa73ad9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa73c0c6640 0x7fa73c0a79d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44302/0 (socket says 172.21.15.121:44302) 2023-11-15T02:59:01.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.222+0000 7fa73ad9d700 1 -- 172.21.15.121:0/3716240764 learned_addr learned my addr 172.21.15.121:0/3716240764 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:01.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.222+0000 7fa739d9b700 1 --2- 172.21.15.121:0/3716240764 >> v2:172.21.15.190:3300/0 conn(0x7fa73c150ff0 0x7fa73c151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:01.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.222+0000 7fa73959a700 1 -- 172.21.15.121:0/3716240764 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3821891556 0 0) 0x7fa734001490 con 0x7fa73c0c6640 2023-11-15T02:59:01.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.222+0000 7fa73959a700 1 -- 172.21.15.121:0/3716240764 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7240033f0 con 0x7fa73c0c6640 2023-11-15T02:59:01.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.222+0000 7fa739d9b700 1 -- 172.21.15.121:0/3716240764 >> v1:172.21.15.190:6789/0 conn(0x7fa73c14af80 legacy=0x7fa73c0c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:01.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.222+0000 7fa739d9b700 1 -- 172.21.15.121:0/3716240764 >> v1:172.21.15.121:6789/0 conn(0x7fa73c0c6640 legacy=0x7fa73c0a79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:01.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.222+0000 7fa739d9b700 1 -- 172.21.15.121:0/3716240764 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c151a40 con 0x7fa73c150ff0 2023-11-15T02:59:01.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.222+0000 7fa739d9b700 1 --2- 172.21.15.121:0/3716240764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa73c150ff0 0x7fa73c151470 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fa730009760 tx=0x7fa730009060).ready entity=mon.1 client_cookie=544d693b7b3b3ac5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:01.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.222+0000 7fa73959a700 1 -- 172.21.15.121:0/3716240764 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73000f510 con 0x7fa73c150ff0 2023-11-15T02:59:01.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.222+0000 7fa73959a700 1 -- 172.21.15.121:0/3716240764 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73000f670 con 0x7fa73c150ff0 2023-11-15T02:59:01.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.222+0000 7fa73959a700 1 -- 172.21.15.121:0/3716240764 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73000f9f0 con 0x7fa73c150ff0 2023-11-15T02:59:01.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.226+0000 7fa7408f1700 1 -- 172.21.15.121:0/3716240764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa73c150ff0 msgr2=0x7fa73c151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:01.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.226+0000 7fa7408f1700 1 --2- 172.21.15.121:0/3716240764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa73c150ff0 0x7fa73c151470 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fa730009760 tx=0x7fa730009060).stop 2023-11-15T02:59:01.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.226+0000 7fa7408f1700 1 -- 172.21.15.121:0/3716240764 shutdown_connections 2023-11-15T02:59:01.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.226+0000 7fa7408f1700 1 --2- 172.21.15.121:0/3716240764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa73c150ff0 0x7fa73c151470 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.226+0000 7fa7408f1700 1 -- 172.21.15.121:0/3716240764 >> 172.21.15.121:0/3716240764 conn(0x7fa73c062190 msgr2=0x7fa73c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:01.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.226+0000 7fa7408f1700 1 -- 172.21.15.121:0/3716240764 shutdown_connections 2023-11-15T02:59:01.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.226+0000 7fa7408f1700 1 -- 172.21.15.121:0/3716240764 wait complete. 2023-11-15T02:59:01.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.226+0000 7fa7408f1700 1 Processor -- start 2023-11-15T02:59:01.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.226+0000 7fa7408f1700 1 -- start start 2023-11-15T02:59:01.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.226+0000 7fa7408f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa73c0c6640 0x7fa73c13f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:01.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa7408f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa73c14af80 0x7fa73c0aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:01.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa7408f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa73c150ff0 0x7fa73c0aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:01.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa7408f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0abe40 con 0x7fa73c0c6640 2023-11-15T02:59:01.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa7408f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0ab3c0 con 0x7fa73c150ff0 2023-11-15T02:59:01.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa7408f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c14f6b0 con 0x7fa73c14af80 2023-11-15T02:59:01.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa73a59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa73c0c6640 0x7fa73c13f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:01.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa73a59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa73c0c6640 0x7fa73c13f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38024/0 (socket says 172.21.15.121:38024) 2023-11-15T02:59:01.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa73a59c700 1 -- 172.21.15.121:0/3274650351 learned_addr learned my addr 172.21.15.121:0/3274650351 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:01.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa73ad9d700 1 --2- 172.21.15.121:0/3274650351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa73c150ff0 0x7fa73c0aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:01.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa739d9b700 1 --2- 172.21.15.121:0/3274650351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa73c14af80 0x7fa73c0aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:01.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa73a59c700 1 -- 172.21.15.121:0/3274650351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa73c14af80 msgr2=0x7fa73c0aa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:01.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa73a59c700 1 --2- 172.21.15.121:0/3274650351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa73c14af80 0x7fa73c0aa9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa73a59c700 1 -- 172.21.15.121:0/3274650351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa73c150ff0 msgr2=0x7fa73c0aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:01.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa73a59c700 1 --2- 172.21.15.121:0/3274650351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa73c150ff0 0x7fa73c0aaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa73a59c700 1 -- 172.21.15.121:0/3274650351 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7300092a0 con 0x7fa73c0c6640 2023-11-15T02:59:01.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa73a59c700 1 --2- 172.21.15.121:0/3274650351 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa73c0c6640 0x7fa73c13f3d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fa72c00c4c0 tx=0x7fa72c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:01.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa72a7fc700 1 -- 172.21.15.121:0/3274650351 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c00ce70 con 0x7fa73c0c6640 2023-11-15T02:59:01.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa72a7fc700 1 -- 172.21.15.121:0/3274650351 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c001e90 con 0x7fa73c0c6640 2023-11-15T02:59:01.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa72a7fc700 1 -- 172.21.15.121:0/3274650351 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c007250 con 0x7fa73c0c6640 2023-11-15T02:59:01.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa7408f1700 1 -- 172.21.15.121:0/3274650351 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73c0ab5a0 con 0x7fa73c0c6640 2023-11-15T02:59:01.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa7408f1700 1 -- 172.21.15.121:0/3274650351 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73c0abb60 con 0x7fa73c0c6640 2023-11-15T02:59:01.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.230+0000 7fa7408f1700 1 -- 172.21.15.121:0/3274650351 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73c14f480 con 0x7fa73c0c6640 2023-11-15T02:59:01.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.234+0000 7fa72a7fc700 1 -- 172.21.15.121:0/3274650351 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa72c0066a0 con 0x7fa73c0c6640 2023-11-15T02:59:01.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.234+0000 7fa72a7fc700 1 --2- 172.21.15.121:0/3274650351 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa71c05a540 0x7fa71c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:01.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.234+0000 7fa72a7fc700 1 -- 172.21.15.121:0/3274650351 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fa72c074040 con 0x7fa73c0c6640 2023-11-15T02:59:01.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.234+0000 7fa739d9b700 1 --2- 172.21.15.121:0/3274650351 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa71c05a540 0x7fa71c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:01.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.238+0000 7fa739d9b700 1 --2- 172.21.15.121:0/3274650351 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa71c05a540 0x7fa71c05ca00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fa73c0c48f0 tx=0x7fa730017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:01.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.242+0000 7fa72a7fc700 1 -- 172.21.15.121:0/3274650351 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa73c14f480 con 0x7fa73c0c6640 2023-11-15T02:59:01.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.406+0000 7fa7408f1700 1 -- 172.21.15.121:0/3274650351 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa73c14f480 con 0x7fa73c0c6640 2023-11-15T02:59:01.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.410+0000 7fa72a7fc700 1 -- 172.21.15.121:0/3274650351 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fa73c14f480 con 0x7fa73c0c6640 2023-11-15T02:59:01.416 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:01.417 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":48,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:58:58.613415+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:58:55.530878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-15T02:58:56.602591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:59:01.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.414+0000 7fa7408f1700 1 -- 172.21.15.121:0/3274650351 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa71c05a540 msgr2=0x7fa71c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:01.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.414+0000 7fa7408f1700 1 --2- 172.21.15.121:0/3274650351 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa71c05a540 0x7fa71c05ca00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fa73c0c48f0 tx=0x7fa730017000).stop 2023-11-15T02:59:01.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.414+0000 7fa7408f1700 1 -- 172.21.15.121:0/3274650351 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa73c0c6640 msgr2=0x7fa73c13f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:01.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.414+0000 7fa7408f1700 1 --2- 172.21.15.121:0/3274650351 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa73c0c6640 0x7fa73c13f3d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fa72c00c4c0 tx=0x7fa72c007e00).stop 2023-11-15T02:59:01.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.414+0000 7fa7408f1700 1 -- 172.21.15.121:0/3274650351 shutdown_connections 2023-11-15T02:59:01.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.414+0000 7fa7408f1700 1 --2- 172.21.15.121:0/3274650351 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa71c05a540 0x7fa71c05ca00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.414+0000 7fa7408f1700 1 --2- 172.21.15.121:0/3274650351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa73c150ff0 0x7fa73c0aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.414+0000 7fa7408f1700 1 --2- 172.21.15.121:0/3274650351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa73c14af80 0x7fa73c0aa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.414+0000 7fa7408f1700 1 --2- 172.21.15.121:0/3274650351 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa73c0c6640 0x7fa73c13f3d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.414+0000 7fa7408f1700 1 -- 172.21.15.121:0/3274650351 >> 172.21.15.121:0/3274650351 conn(0x7fa73c062190 msgr2=0x7fa73c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:01.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.414+0000 7fa7408f1700 1 -- 172.21.15.121:0/3274650351 shutdown_connections 2023-11-15T02:59:01.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.414+0000 7fa7408f1700 1 -- 172.21.15.121:0/3274650351 wait complete. 2023-11-15T02:59:01.440 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-15T02:59:01.441 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T02:59:01.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.598+0000 7efd49ff9700 1 Processor -- start 2023-11-15T02:59:01.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.598+0000 7efd49ff9700 1 -- start start 2023-11-15T02:59:01.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.598+0000 7efd49ff9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efd440cabd0 0x7efd440cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:01.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.598+0000 7efd49ff9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efd440ad0f0 0x7efd440ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:01.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd49ff9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd440ad650 con 0x7efd440acc90 2023-11-15T02:59:01.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd49ff9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd44065fc0 con 0x7efd440cabd0 2023-11-15T02:59:01.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd49ff9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd440662a0 con 0x7efd440ad0f0 2023-11-15T02:59:01.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd48ff7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efd440cabd0 0x7efd440cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:01.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd48ff7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efd440cabd0 0x7efd440cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38036/0 (socket says 172.21.15.121:38036) 2023-11-15T02:59:01.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd48ff7700 1 -- 172.21.15.121:0/3833118860 learned_addr learned my addr 172.21.15.121:0/3833118860 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:01.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd3bfff700 1 --1- 172.21.15.121:0/3833118860 >> v1:172.21.15.121:6789/0 conn(0x7efd440acc90 0x7efd44066d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44312/0 (socket says 172.21.15.121:44312) 2023-11-15T02:59:01.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd497f8700 1 --2- 172.21.15.121:0/3833118860 >> v2:172.21.15.190:3300/0 conn(0x7efd440ad0f0 0x7efd440ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:01.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd48ff7700 1 -- 172.21.15.121:0/3833118860 >> v2:172.21.15.190:3300/0 conn(0x7efd440ad0f0 msgr2=0x7efd440ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:01.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd48ff7700 1 --2- 172.21.15.121:0/3833118860 >> v2:172.21.15.190:3300/0 conn(0x7efd440ad0f0 0x7efd440ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd48ff7700 1 -- 172.21.15.121:0/3833118860 >> v1:172.21.15.121:6789/0 conn(0x7efd440acc90 legacy=0x7efd44066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:01.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd48ff7700 1 -- 172.21.15.121:0/3833118860 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd44066580 con 0x7efd440cabd0 2023-11-15T02:59:01.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd48ff7700 1 --2- 172.21.15.121:0/3833118860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd440cabd0 0x7efd440cafb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7efd440c7000 tx=0x7efd400090f0).ready entity=mon.0 client_cookie=e111d6292fc29bda server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:01.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd3b7fe700 1 -- 172.21.15.121:0/3833118860 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4000b600 con 0x7efd440cabd0 2023-11-15T02:59:01.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd3b7fe700 1 -- 172.21.15.121:0/3833118860 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4000b760 con 0x7efd440cabd0 2023-11-15T02:59:01.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd3b7fe700 1 -- 172.21.15.121:0/3833118860 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4000bae0 con 0x7efd440cabd0 2023-11-15T02:59:01.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd49ff9700 1 -- 172.21.15.121:0/3833118860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd440cabd0 msgr2=0x7efd440cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:01.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd49ff9700 1 --2- 172.21.15.121:0/3833118860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd440cabd0 0x7efd440cafb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7efd440c7000 tx=0x7efd400090f0).stop 2023-11-15T02:59:01.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd49ff9700 1 -- 172.21.15.121:0/3833118860 shutdown_connections 2023-11-15T02:59:01.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd49ff9700 1 --2- 172.21.15.121:0/3833118860 >> v2:172.21.15.190:3300/0 conn(0x7efd440ad0f0 0x7efd440ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd49ff9700 1 --2- 172.21.15.121:0/3833118860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd440cabd0 0x7efd440cafb0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd49ff9700 1 -- 172.21.15.121:0/3833118860 >> 172.21.15.121:0/3833118860 conn(0x7efd44062190 msgr2=0x7efd440625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:01.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd49ff9700 1 -- 172.21.15.121:0/3833118860 shutdown_connections 2023-11-15T02:59:01.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.602+0000 7efd49ff9700 1 -- 172.21.15.121:0/3833118860 wait complete. 2023-11-15T02:59:01.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.606+0000 7efd49ff9700 1 Processor -- start 2023-11-15T02:59:01.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.606+0000 7efd49ff9700 1 -- start start 2023-11-15T02:59:01.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd49ff9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd440acc90 0x7efd44152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:01.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd49ff9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd440ad0f0 0x7efd44146480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:01.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd49ff9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd440cabd0 0x7efd44158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:01.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd49ff9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd441566c0 con 0x7efd440ad0f0 2023-11-15T02:59:01.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd49ff9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd44151d90 con 0x7efd440acc90 2023-11-15T02:59:01.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd49ff9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd44151180 con 0x7efd440cabd0 2023-11-15T02:59:01.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd3bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd440ad0f0 0x7efd44146480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:01.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd3bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd440ad0f0 0x7efd44146480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38044/0 (socket says 172.21.15.121:38044) 2023-11-15T02:59:01.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd3bfff700 1 -- 172.21.15.121:0/1955283698 learned_addr learned my addr 172.21.15.121:0/1955283698 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:01.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd48ff7700 1 --2- 172.21.15.121:0/1955283698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd440acc90 0x7efd44152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:01.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd497f8700 1 --2- 172.21.15.121:0/1955283698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd440cabd0 0x7efd44158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:01.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd3bfff700 1 -- 172.21.15.121:0/1955283698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd440cabd0 msgr2=0x7efd44158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:01.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd3bfff700 1 --2- 172.21.15.121:0/1955283698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd440cabd0 0x7efd44158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd3bfff700 1 -- 172.21.15.121:0/1955283698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd440acc90 msgr2=0x7efd44152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:01.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd3bfff700 1 --2- 172.21.15.121:0/1955283698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd440acc90 0x7efd44152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd3bfff700 1 -- 172.21.15.121:0/1955283698 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd40009370 con 0x7efd440ad0f0 2023-11-15T02:59:01.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd3bfff700 1 --2- 172.21.15.121:0/1955283698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd440ad0f0 0x7efd44146480 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7efd34007fd0 tx=0x7efd34007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:01.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd38ff9700 1 -- 172.21.15.121:0/1955283698 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340072e0 con 0x7efd440ad0f0 2023-11-15T02:59:01.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd38ff9700 1 -- 172.21.15.121:0/1955283698 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd34007440 con 0x7efd440ad0f0 2023-11-15T02:59:01.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd49ff9700 1 -- 172.21.15.121:0/1955283698 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd44151460 con 0x7efd440ad0f0 2023-11-15T02:59:01.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd38ff9700 1 -- 172.21.15.121:0/1955283698 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340058f0 con 0x7efd440ad0f0 2023-11-15T02:59:01.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.610+0000 7efd49ff9700 1 -- 172.21.15.121:0/1955283698 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd44151a20 con 0x7efd440ad0f0 2023-11-15T02:59:01.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.614+0000 7efd49ff9700 1 -- 172.21.15.121:0/1955283698 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd44151f40 con 0x7efd440ad0f0 2023-11-15T02:59:01.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.614+0000 7efd38ff9700 1 -- 172.21.15.121:0/1955283698 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7efd34005ce0 con 0x7efd440ad0f0 2023-11-15T02:59:01.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.614+0000 7efd38ff9700 1 --2- 172.21.15.121:0/1955283698 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efd3005a4e0 0x7efd3005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:01.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.614+0000 7efd38ff9700 1 -- 172.21.15.121:0/1955283698 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efd34073ff0 con 0x7efd440ad0f0 2023-11-15T02:59:01.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.614+0000 7efd48ff7700 1 --2- 172.21.15.121:0/1955283698 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efd3005a4e0 0x7efd3005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:01.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.618+0000 7efd48ff7700 1 --2- 172.21.15.121:0/1955283698 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efd3005a4e0 0x7efd3005c9a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7efd40009130 tx=0x7efd40017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:01.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.622+0000 7efd38ff9700 1 -- 172.21.15.121:0/1955283698 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efd34047750 con 0x7efd440ad0f0 2023-11-15T02:59:01.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.782+0000 7efd49ff9700 1 -- 172.21.15.121:0/1955283698 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd4406bcf0 con 0x7efd3005a4e0 2023-11-15T02:59:01.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.802+0000 7efd38ff9700 1 -- 172.21.15.121:0/1955283698 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+349141 (secure 0 0 0) 0x7efd4406bcf0 con 0x7efd3005a4e0 2023-11-15T02:59:01.810 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:01.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.810+0000 7efd49ff9700 1 -- 172.21.15.121:0/1955283698 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efd3005a4e0 msgr2=0x7efd3005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:01.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.810+0000 7efd49ff9700 1 --2- 172.21.15.121:0/1955283698 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efd3005a4e0 0x7efd3005c9a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7efd40009130 tx=0x7efd40017000).stop 2023-11-15T02:59:01.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.810+0000 7efd49ff9700 1 -- 172.21.15.121:0/1955283698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd440ad0f0 msgr2=0x7efd44146480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:01.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.810+0000 7efd49ff9700 1 --2- 172.21.15.121:0/1955283698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd440ad0f0 0x7efd44146480 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7efd34007fd0 tx=0x7efd34007df0).stop 2023-11-15T02:59:01.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.810+0000 7efd49ff9700 1 -- 172.21.15.121:0/1955283698 shutdown_connections 2023-11-15T02:59:01.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.810+0000 7efd49ff9700 1 --2- 172.21.15.121:0/1955283698 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efd3005a4e0 0x7efd3005c9a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.810+0000 7efd49ff9700 1 --2- 172.21.15.121:0/1955283698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd440cabd0 0x7efd44158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.810+0000 7efd49ff9700 1 --2- 172.21.15.121:0/1955283698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd440ad0f0 0x7efd44146480 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.810+0000 7efd49ff9700 1 --2- 172.21.15.121:0/1955283698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd440acc90 0x7efd44152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:01.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.810+0000 7efd49ff9700 1 -- 172.21.15.121:0/1955283698 >> 172.21.15.121:0/1955283698 conn(0x7efd44062190 msgr2=0x7efd44156be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:01.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.814+0000 7efd49ff9700 1 -- 172.21.15.121:0/1955283698 shutdown_connections 2023-11-15T02:59:01.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:01.814+0000 7efd49ff9700 1 -- 172.21.15.121:0/1955283698 wait complete. 2023-11-15T02:59:01.822 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T02:59:01.838 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2023-11-15T02:58:59.957417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":892,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":70,"acting":70,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2380840,"kb_used_data":4072,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752593880,"statfs":{"total":773094113280,"available":770656133120,"internally_reserved":0,"allocated":4169728,"data_stored":1792226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25831,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-54,"num_read_kb":-56,"num_write":-80,"num_write_kb":-54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.371806"},"pg_stats":[{"pgid":"8.30","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.632390+0000","last_change":"2023-11-15T02:58:57.632390+0000","last_active":"2023-11-15T02:58:57.632390+0000","last_peered":"2023-11-15T02:58:57.632390+0000","last_clean":"2023-11-15T02:58:57.632390+0000","last_became_active":"2023-11-15T02:58:57.632126+0000","last_became_peered":"2023-11-15T02:58:57.632126+0000","last_unstale":"2023-11-15T02:58:57.632390+0000","last_undegraded":"2023-11-15T02:58:57.632390+0000","last_fullsized":"2023-11-15T02:58:57.632390+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.622234+0000","last_change":"2023-11-15T02:58:57.194667+0000","last_active":"2023-11-15T02:58:57.622234+0000","last_peered":"2023-11-15T02:58:57.622234+0000","last_clean":"2023-11-15T02:58:57.622234+0000","last_became_active":"2023-11-15T02:58:57.194280+0000","last_became_peered":"2023-11-15T02:58:57.194280+0000","last_unstale":"2023-11-15T02:58:57.622234+0000","last_undegraded":"2023-11-15T02:58:57.622234+0000","last_fullsized":"2023-11-15T02:58:57.622234+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.632654+0000","last_change":"2023-11-15T02:58:57.632654+0000","last_active":"2023-11-15T02:58:57.632654+0000","last_peered":"2023-11-15T02:58:57.632654+0000","last_clean":"2023-11-15T02:58:57.632654+0000","last_became_active":"2023-11-15T02:58:57.632380+0000","last_became_peered":"2023-11-15T02:58:57.632380+0000","last_unstale":"2023-11-15T02:58:57.632654+0000","last_undegraded":"2023-11-15T02:58:57.632654+0000","last_fullsized":"2023-11-15T02:58:57.632654+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"47'1","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.641614+0000","last_change":"2023-11-15T02:58:57.195055+0000","last_active":"2023-11-15T02:58:57.641614+0000","last_peered":"2023-11-15T02:58:57.641614+0000","last_clean":"2023-11-15T02:58:57.641614+0000","last_became_active":"2023-11-15T02:58:57.194269+0000","last_became_peered":"2023-11-15T02:58:57.194269+0000","last_unstale":"2023-11-15T02:58:57.641614+0000","last_undegraded":"2023-11-15T02:58:57.641614+0000","last_fullsized":"2023-11-15T02:58:57.641614+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.637020+0000","last_change":"2023-11-15T02:58:57.637020+0000","last_active":"2023-11-15T02:58:57.637020+0000","last_peered":"2023-11-15T02:58:57.637020+0000","last_clean":"2023-11-15T02:58:57.637020+0000","last_became_active":"2023-11-15T02:58:57.636627+0000","last_became_peered":"2023-11-15T02:58:57.636627+0000","last_unstale":"2023-11-15T02:58:57.637020+0000","last_undegraded":"2023-11-15T02:58:57.637020+0000","last_fullsized":"2023-11-15T02:58:57.637020+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.653843+0000","last_change":"2023-11-15T02:58:57.653843+0000","last_active":"2023-11-15T02:58:57.653843+0000","last_peered":"2023-11-15T02:58:57.653843+0000","last_clean":"2023-11-15T02:58:57.653843+0000","last_became_active":"2023-11-15T02:58:57.653352+0000","last_became_peered":"2023-11-15T02:58:57.653352+0000","last_unstale":"2023-11-15T02:58:57.653843+0000","last_undegraded":"2023-11-15T02:58:57.653843+0000","last_fullsized":"2023-11-15T02:58:57.653843+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"47'4","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.658044+0000","last_change":"2023-11-15T02:58:57.183738+0000","last_active":"2023-11-15T02:58:57.658044+0000","last_peered":"2023-11-15T02:58:57.658044+0000","last_clean":"2023-11-15T02:58:57.658044+0000","last_became_active":"2023-11-15T02:58:57.179352+0000","last_became_peered":"2023-11-15T02:58:57.179352+0000","last_unstale":"2023-11-15T02:58:57.658044+0000","last_undegraded":"2023-11-15T02:58:57.658044+0000","last_fullsized":"2023-11-15T02:58:57.658044+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.632675+0000","last_change":"2023-11-15T02:58:57.632675+0000","last_active":"2023-11-15T02:58:57.632675+0000","last_peered":"2023-11-15T02:58:57.632675+0000","last_clean":"2023-11-15T02:58:57.632675+0000","last_became_active":"2023-11-15T02:58:57.632390+0000","last_became_peered":"2023-11-15T02:58:57.632390+0000","last_unstale":"2023-11-15T02:58:57.632675+0000","last_undegraded":"2023-11-15T02:58:57.632675+0000","last_fullsized":"2023-11-15T02:58:57.632675+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.637366+0000","last_change":"2023-11-15T02:58:57.635264+0000","last_active":"2023-11-15T02:58:57.637366+0000","last_peered":"2023-11-15T02:58:57.637366+0000","last_clean":"2023-11-15T02:58:57.637366+0000","last_became_active":"2023-11-15T02:58:57.634849+0000","last_became_peered":"2023-11-15T02:58:57.634849+0000","last_unstale":"2023-11-15T02:58:57.637366+0000","last_undegraded":"2023-11-15T02:58:57.637366+0000","last_fullsized":"2023-11-15T02:58:57.637366+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.644118+0000","last_change":"2023-11-15T02:58:57.644118+0000","last_active":"2023-11-15T02:58:57.644118+0000","last_peered":"2023-11-15T02:58:57.644118+0000","last_clean":"2023-11-15T02:58:57.644118+0000","last_became_active":"2023-11-15T02:58:57.643849+0000","last_became_peered":"2023-11-15T02:58:57.643849+0000","last_unstale":"2023-11-15T02:58:57.644118+0000","last_undegraded":"2023-11-15T02:58:57.644118+0000","last_fullsized":"2023-11-15T02:58:57.644118+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.625321+0000","last_change":"2023-11-15T02:58:57.181901+0000","last_active":"2023-11-15T02:58:57.625321+0000","last_peered":"2023-11-15T02:58:57.625321+0000","last_clean":"2023-11-15T02:58:57.625321+0000","last_became_active":"2023-11-15T02:58:57.178952+0000","last_became_peered":"2023-11-15T02:58:57.178952+0000","last_unstale":"2023-11-15T02:58:57.625321+0000","last_undegraded":"2023-11-15T02:58:57.625321+0000","last_fullsized":"2023-11-15T02:58:57.625321+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.656731+0000","last_change":"2023-11-15T02:58:57.653842+0000","last_active":"2023-11-15T02:58:57.656731+0000","last_peered":"2023-11-15T02:58:57.656731+0000","last_clean":"2023-11-15T02:58:57.656731+0000","last_became_active":"2023-11-15T02:58:57.653543+0000","last_became_peered":"2023-11-15T02:58:57.653543+0000","last_unstale":"2023-11-15T02:58:57.656731+0000","last_undegraded":"2023-11-15T02:58:57.656731+0000","last_fullsized":"2023-11-15T02:58:57.656731+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"47'1","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.628131+0000","last_change":"2023-11-15T02:58:57.195494+0000","last_active":"2023-11-15T02:58:57.628131+0000","last_peered":"2023-11-15T02:58:57.628131+0000","last_clean":"2023-11-15T02:58:57.628131+0000","last_became_active":"2023-11-15T02:58:57.195196+0000","last_became_peered":"2023-11-15T02:58:57.195196+0000","last_unstale":"2023-11-15T02:58:57.628131+0000","last_undegraded":"2023-11-15T02:58:57.628131+0000","last_fullsized":"2023-11-15T02:58:57.628131+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"47'2","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.648975+0000","last_change":"2023-11-15T02:58:57.190329+0000","last_active":"2023-11-15T02:58:57.648975+0000","last_peered":"2023-11-15T02:58:57.648975+0000","last_clean":"2023-11-15T02:58:57.648975+0000","last_became_active":"2023-11-15T02:58:57.190059+0000","last_became_peered":"2023-11-15T02:58:57.190059+0000","last_unstale":"2023-11-15T02:58:57.648975+0000","last_undegraded":"2023-11-15T02:58:57.648975+0000","last_fullsized":"2023-11-15T02:58:57.648975+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.625423+0000","last_change":"2023-11-15T02:58:57.204873+0000","last_active":"2023-11-15T02:58:57.625423+0000","last_peered":"2023-11-15T02:58:57.625423+0000","last_clean":"2023-11-15T02:58:57.625423+0000","last_became_active":"2023-11-15T02:58:57.204576+0000","last_became_peered":"2023-11-15T02:58:57.204576+0000","last_unstale":"2023-11-15T02:58:57.625423+0000","last_undegraded":"2023-11-15T02:58:57.625423+0000","last_fullsized":"2023-11-15T02:58:57.625423+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.635033+0000","last_change":"2023-11-15T02:58:57.630393+0000","last_active":"2023-11-15T02:58:57.635033+0000","last_peered":"2023-11-15T02:58:57.635033+0000","last_clean":"2023-11-15T02:58:57.635033+0000","last_became_active":"2023-11-15T02:58:57.629556+0000","last_became_peered":"2023-11-15T02:58:57.629556+0000","last_unstale":"2023-11-15T02:58:57.635033+0000","last_undegraded":"2023-11-15T02:58:57.635033+0000","last_fullsized":"2023-11-15T02:58:57.635033+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.651446+0000","last_change":"2023-11-15T02:58:57.647677+0000","last_active":"2023-11-15T02:58:57.651446+0000","last_peered":"2023-11-15T02:58:57.651446+0000","last_clean":"2023-11-15T02:58:57.651446+0000","last_became_active":"2023-11-15T02:58:57.647396+0000","last_became_peered":"2023-11-15T02:58:57.647396+0000","last_unstale":"2023-11-15T02:58:57.651446+0000","last_undegraded":"2023-11-15T02:58:57.651446+0000","last_fullsized":"2023-11-15T02:58:57.651446+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.642369+0000","last_change":"2023-11-15T02:58:57.642369+0000","last_active":"2023-11-15T02:58:57.642369+0000","last_peered":"2023-11-15T02:58:57.642369+0000","last_clean":"2023-11-15T02:58:57.642369+0000","last_became_active":"2023-11-15T02:58:57.642090+0000","last_became_peered":"2023-11-15T02:58:57.642090+0000","last_unstale":"2023-11-15T02:58:57.642369+0000","last_undegraded":"2023-11-15T02:58:57.642369+0000","last_fullsized":"2023-11-15T02:58:57.642369+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.642380+0000","last_change":"2023-11-15T02:58:57.642380+0000","last_active":"2023-11-15T02:58:57.642380+0000","last_peered":"2023-11-15T02:58:57.642380+0000","last_clean":"2023-11-15T02:58:57.642380+0000","last_became_active":"2023-11-15T02:58:57.640187+0000","last_became_peered":"2023-11-15T02:58:57.640187+0000","last_unstale":"2023-11-15T02:58:57.642380+0000","last_undegraded":"2023-11-15T02:58:57.642380+0000","last_fullsized":"2023-11-15T02:58:57.642380+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.635066+0000","last_change":"2023-11-15T02:58:57.630783+0000","last_active":"2023-11-15T02:58:57.635066+0000","last_peered":"2023-11-15T02:58:57.635066+0000","last_clean":"2023-11-15T02:58:57.635066+0000","last_became_active":"2023-11-15T02:58:57.630491+0000","last_became_peered":"2023-11-15T02:58:57.630491+0000","last_unstale":"2023-11-15T02:58:57.635066+0000","last_undegraded":"2023-11-15T02:58:57.635066+0000","last_fullsized":"2023-11-15T02:58:57.635066+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"47'1","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.627547+0000","last_change":"2023-11-15T02:58:57.181792+0000","last_active":"2023-11-15T02:58:57.627547+0000","last_peered":"2023-11-15T02:58:57.627547+0000","last_clean":"2023-11-15T02:58:57.627547+0000","last_became_active":"2023-11-15T02:58:57.181505+0000","last_became_peered":"2023-11-15T02:58:57.181505+0000","last_unstale":"2023-11-15T02:58:57.627547+0000","last_undegraded":"2023-11-15T02:58:57.627547+0000","last_fullsized":"2023-11-15T02:58:57.627547+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.641367+0000","last_change":"2023-11-15T02:58:57.641367+0000","last_active":"2023-11-15T02:58:57.641367+0000","last_peered":"2023-11-15T02:58:57.641367+0000","last_clean":"2023-11-15T02:58:57.641367+0000","last_became_active":"2023-11-15T02:58:57.641112+0000","last_became_peered":"2023-11-15T02:58:57.641112+0000","last_unstale":"2023-11-15T02:58:57.641367+0000","last_undegraded":"2023-11-15T02:58:57.641367+0000","last_fullsized":"2023-11-15T02:58:57.641367+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.656078+0000","last_change":"2023-11-15T02:58:57.652641+0000","last_active":"2023-11-15T02:58:57.656078+0000","last_peered":"2023-11-15T02:58:57.656078+0000","last_clean":"2023-11-15T02:58:57.656078+0000","last_became_active":"2023-11-15T02:58:57.652308+0000","last_became_peered":"2023-11-15T02:58:57.652308+0000","last_unstale":"2023-11-15T02:58:57.656078+0000","last_undegraded":"2023-11-15T02:58:57.656078+0000","last_fullsized":"2023-11-15T02:58:57.656078+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-15T02:58:52.124423+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T02:58:52.124423+0000","last_peered":"2023-11-15T02:58:52.124423+0000","last_clean":"2023-11-15T02:58:52.124423+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T02:58:52.124423+0000","last_undegraded":"2023-11-15T02:58:52.124423+0000","last_fullsized":"2023-11-15T02:58:52.124423+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.165470+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T02:58:57.165470+0000","last_peered":"2023-11-15T02:58:57.165470+0000","last_clean":"2023-11-15T02:58:57.165470+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T02:58:57.165470+0000","last_undegraded":"2023-11-15T02:58:57.165470+0000","last_fullsized":"2023-11-15T02:58:57.165470+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.623049+0000","last_change":"2023-11-15T02:58:57.188147+0000","last_active":"2023-11-15T02:58:57.623049+0000","last_peered":"2023-11-15T02:58:57.623049+0000","last_clean":"2023-11-15T02:58:57.623049+0000","last_became_active":"2023-11-15T02:58:57.187878+0000","last_became_peered":"2023-11-15T02:58:57.187878+0000","last_unstale":"2023-11-15T02:58:57.623049+0000","last_undegraded":"2023-11-15T02:58:57.623049+0000","last_fullsized":"2023-11-15T02:58:57.623049+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":30,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-15T02:58:52.758583+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T02:58:52.758583+0000","last_peered":"2023-11-15T02:58:52.758583+0000","last_clean":"2023-11-15T02:58:52.758583+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T02:58:52.758583+0000","last_undegraded":"2023-11-15T02:58:52.758583+0000","last_fullsized":"2023-11-15T02:58:52.758583+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.164926+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T02:58:57.164926+0000","last_peered":"2023-11-15T02:58:57.164926+0000","last_clean":"2023-11-15T02:58:57.164926+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T02:58:57.164926+0000","last_undegraded":"2023-11-15T02:58:57.164926+0000","last_fullsized":"2023-11-15T02:58:57.164926+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.637346+0000","last_change":"2023-11-15T02:58:57.635240+0000","last_active":"2023-11-15T02:58:57.637346+0000","last_peered":"2023-11-15T02:58:57.637346+0000","last_clean":"2023-11-15T02:58:57.637346+0000","last_became_active":"2023-11-15T02:58:57.634833+0000","last_became_peered":"2023-11-15T02:58:57.634833+0000","last_unstale":"2023-11-15T02:58:57.637346+0000","last_undegraded":"2023-11-15T02:58:57.637346+0000","last_fullsized":"2023-11-15T02:58:57.637346+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.621848+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T02:58:57.621848+0000","last_peered":"2023-11-15T02:58:57.621848+0000","last_clean":"2023-11-15T02:58:57.621848+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T02:58:57.621848+0000","last_undegraded":"2023-11-15T02:58:57.621848+0000","last_fullsized":"2023-11-15T02:58:57.621848+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-15T02:58:52.121449+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T02:58:52.121449+0000","last_peered":"2023-11-15T02:58:52.121449+0000","last_clean":"2023-11-15T02:58:52.121449+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T02:58:52.121449+0000","last_undegraded":"2023-11-15T02:58:52.121449+0000","last_fullsized":"2023-11-15T02:58:52.121449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.646610+0000","last_change":"2023-11-15T02:58:57.646610+0000","last_active":"2023-11-15T02:58:57.646610+0000","last_peered":"2023-11-15T02:58:57.646610+0000","last_clean":"2023-11-15T02:58:57.646610+0000","last_became_active":"2023-11-15T02:58:57.645676+0000","last_became_peered":"2023-11-15T02:58:57.645676+0000","last_unstale":"2023-11-15T02:58:57.646610+0000","last_undegraded":"2023-11-15T02:58:57.646610+0000","last_fullsized":"2023-11-15T02:58:57.646610+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-15T02:58:52.758593+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T02:58:52.758593+0000","last_peered":"2023-11-15T02:58:52.758593+0000","last_clean":"2023-11-15T02:58:52.758593+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T02:58:52.758593+0000","last_undegraded":"2023-11-15T02:58:52.758593+0000","last_fullsized":"2023-11-15T02:58:52.758593+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:57.144906+0000","last_change":"2023-11-15T02:58:57.144906+0000","last_active":"2023-11-15T02:58:57.144906+0000","last_peered":"2023-11-15T02:58:57.144906+0000","last_clean":"2023-11-15T02:58:57.144906+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:57.144906+0000","last_undegraded":"2023-11-15T02:58:57.144906+0000","last_fullsized":"2023-11-15T02:58:57.144906+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:58:56.131661+0000","last_change":"2023-11-15T02:58:56.131661+0000","last_active":"2023-11-15T02:58:56.131661+0000","last_peered":"2023-11-15T02:58:56.131661+0000","last_clean":"2023-11-15T02:58:56.131661+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:58:56.131661+0000","last_undegraded":"2023-11-15T02:58:56.131661+0000","last_fullsized":"2023-11-15T02:58:56.131661+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.623060+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T02:58:57.623060+0000","last_peered":"2023-11-15T02:58:57.623060+0000","last_clean":"2023-11-15T02:58:57.623060+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T02:58:57.623060+0000","last_undegraded":"2023-11-15T02:58:57.623060+0000","last_fullsized":"2023-11-15T02:58:57.623060+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-15T02:58:52.758541+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T02:58:52.758541+0000","last_peered":"2023-11-15T02:58:52.758541+0000","last_clean":"2023-11-15T02:58:52.758541+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T02:58:52.758541+0000","last_undegraded":"2023-11-15T02:58:52.758541+0000","last_fullsized":"2023-11-15T02:58:52.758541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":34,"acting":34,"num_store_stats":2},{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":892,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":18,"acting":18,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542172,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297596,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074244,"statfs":{"total":96636764160,"available":96332025856,"internally_reserved":0,"allocated":512000,"data_stored":219115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.1679999999999999,"15min":1.1859999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.458,"5min":3.952,"15min":3.952},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.046,"5min":1.1799999999999999,"15min":1.202},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4419999999999999,"5min":2.5,"15min":2.5},"last":0.96199999999999997}]},{"osd":1,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.137,"15min":1.145},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.3740000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1739999999999999,"15min":1.1779999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.415,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.139}]},{"osd":2,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2150000000000001,"15min":1.2330000000000001},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.458,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":1.083},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2949999999999999,"15min":1.3069999999999999},"min":{"1min":0.97899999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.5149999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":1.2}]},{"osd":3,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.3400000000000001,"15min":1.3740000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.381,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.175,"5min":1.2669999999999999,"15min":1.282},"min":{"1min":0.90600000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.351,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.0309999999999999}]},{"osd":4,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.2689999999999999,"15min":1.3},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.4359999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.448,"15min":1.4990000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.5029999999999999,"5min":4.226,"15min":4.226},"last":1.1679999999999999}]},{"osd":5,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.329,"15min":1.343},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.556,"5min":2.024,"15min":2.024},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.234,"5min":1.4399999999999999,"15min":1.474},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.653,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.236}]},{"osd":6,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.5620000000000001,"15min":1.601},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":1.5660000000000001,"5min":3.2789999999999999,"15min":3.2789999999999999},"last":1.484},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.591,"15min":1.645},"min":{"1min":1.123,"5min":1.123,"15min":1.123},"max":{"1min":1.4399999999999999,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":1.264}]}]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297596,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074244,"statfs":{"total":96636764160,"available":96332025856,"internally_reserved":0,"allocated":512000,"data_stored":219115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.125,"15min":1.133},"min":{"1min":0.69799999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.353,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":1.238},{"interface":"front","average":{"1min":1.083,"5min":1.0469999999999999,"15min":1.0409999999999999},"min":{"1min":0.72499999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.484,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.484}]},{"osd":1,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0920000000000001,"15min":1.0840000000000001},"min":{"1min":0.876,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.415,"5min":1.623,"15min":1.623},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.173,"5min":1.169,"15min":1.169},"min":{"1min":0.80800000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.609,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.3169999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.1870000000000001,"15min":1.1779999999999999},"min":{"1min":0.84399999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.496,"5min":2.621,"15min":2.621},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.0960000000000001,"15min":1.0840000000000001},"min":{"1min":0.91700000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.425,"5min":1.5,"15min":1.5},"last":1.425}]},{"osd":3,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.167,"15min":1.157},"min":{"1min":1.0029999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.492,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.345},{"interface":"front","average":{"1min":1.24,"5min":1.26,"15min":1.2629999999999999},"min":{"1min":0.99299999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.6850000000000001,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.196}]},{"osd":4,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.173,"15min":1.1910000000000001},"min":{"1min":0.81399999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.4099999999999999,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.95599999999999996,"5min":0.625,"15min":0.625},"max":{"1min":1.4850000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.2769999999999999}]},{"osd":5,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.417,"15min":1.411},"min":{"1min":1.155,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.772,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.3109999999999999,"15min":1.319},"min":{"1min":0.877,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6599999999999999,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":1.1140000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2809999999999999,"15min":1.3009999999999999},"min":{"1min":0.82199999999999995,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.395,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.153},{"interface":"front","average":{"1min":1.401,"5min":1.448,"15min":1.456},"min":{"1min":1.046,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8540000000000001,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":1.4419999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297596,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074244,"statfs":{"total":96636764160,"available":96332025856,"internally_reserved":0,"allocated":512000,"data_stored":219115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.107,"15min":1.1120000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.516,"5min":1.532,"15min":1.532},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0069999999999999,"15min":1.0029999999999999},"min":{"1min":0.80700000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":1.377}]},{"osd":1,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2250000000000001,"15min":1.2370000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4770000000000001,"5min":2.617,"15min":2.617},"last":1.121},{"interface":"front","average":{"1min":1.115,"5min":1.1559999999999999,"15min":1.163},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.425,"5min":1.5,"15min":1.5},"last":1.657}]},{"osd":2,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.1519999999999999,"15min":1.141},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.462,"5min":1.909,"15min":1.909},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1599999999999999,"15min":1.1639999999999999},"min":{"1min":0.88800000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.399,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.026}]},{"osd":3,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1519999999999999,"15min":1.1599999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.462,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.625},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.294,"15min":1.3},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.5840000000000001,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":1.518}]},{"osd":5,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2430000000000001,"15min":1.2529999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5780000000000001,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.2290000000000001,"15min":1.2350000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.5049999999999999,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.2709999999999999}]},{"osd":6,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.333,"15min":1.343},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6619999999999999,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":1.913},{"interface":"front","average":{"1min":1.296,"5min":1.3979999999999999,"15min":1.415},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.7450000000000001,"5min":2.173,"15min":2.173},"last":1.7070000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.369,"15min":1.381},"min":{"1min":1.069,"5min":0.87,"15min":0.87},"max":{"1min":1.5820000000000001,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":1.8600000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.4350000000000001,"15min":1.4450000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.587,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":1.7989999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542173,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297596,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074244,"statfs":{"total":96636764160,"available":96332025856,"internally_reserved":0,"allocated":512000,"data_stored":219115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.77400000000000002,"15min":0.75900000000000001},"min":{"1min":0.64900000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.84199999999999997,"15min":0.82899999999999996},"min":{"1min":0.73599999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1779999999999999,"5min":1.26,"15min":1.26},"last":0.64100000000000001}]},{"osd":1,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.90000000000000002,"15min":0.90100000000000002},"min":{"1min":0.621,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.97999999999999998,"15min":0.96999999999999997},"min":{"1min":0.83299999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2749999999999999,"5min":1.375,"15min":1.375},"last":0.94799999999999995}]},{"osd":2,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97699999999999998,"15min":0.96799999999999997},"min":{"1min":0.76700000000000002,"5min":0.63,"15min":0.63},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0649999999999999,"15min":1.052},"min":{"1min":0.85899999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.01}]},{"osd":4,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1859999999999999,"15min":1.1799999999999999},"min":{"1min":0.97899999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.456,"5min":1.837,"15min":1.837},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.22,"15min":1.226},"min":{"1min":1.0249999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.492,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.0800000000000001}]},{"osd":5,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.27,"15min":1.2669999999999999},"min":{"1min":0.996,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.6459999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.127},{"interface":"front","average":{"1min":1.345,"5min":1.2789999999999999,"15min":1.268},"min":{"1min":1.01,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":1.3169999999999999}]},{"osd":6,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3120000000000001,"15min":1.306},"min":{"1min":1.1060000000000001,"5min":1.048,"15min":1.048},"max":{"1min":1.552,"5min":1.794,"15min":1.794},"last":1.262},{"interface":"front","average":{"1min":1.391,"5min":1.3580000000000001,"15min":1.3520000000000001},"min":{"1min":1.129,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":1.647,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.1830000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.3919999999999999,"15min":1.3899999999999999},"min":{"1min":1.083,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.228},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.365,"15min":1.357},"min":{"1min":1.1819999999999999,"5min":1.085,"15min":1.085},"max":{"1min":1.665,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.2030000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542173,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":496,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":507904,"data_stored":217846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76700000000000002,"15min":0.76400000000000001},"min":{"1min":0.61699999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88700000000000001,"15min":0.88200000000000001},"min":{"1min":0.68300000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.81100000000000005}]},{"osd":1,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89600000000000002,"15min":0.89200000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3120000000000001,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.96099999999999997,"15min":0.94899999999999995},"min":{"1min":0.83499999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.66500000000000004}]},{"osd":3,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.002,"15min":1},"min":{"1min":0.79200000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.387,"5min":1.472,"15min":1.472},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1259999999999999,"15min":1.1200000000000001},"min":{"1min":0.93000000000000005,"5min":0.622,"15min":0.622},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.1100000000000001}]},{"osd":4,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.208,"15min":1.21},"min":{"1min":1.0149999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.48,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.177},{"interface":"front","average":{"1min":1.286,"5min":1.238,"15min":1.23},"min":{"1min":1.0760000000000001,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.5429999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.90400000000000003}]},{"osd":5,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.262,"15min":1.264},"min":{"1min":1.0149999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":1.613,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2989999999999999,"15min":1.2969999999999999},"min":{"1min":1.0700000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.6890000000000001,"5min":1.857,"15min":1.857},"last":1.2210000000000001}]},{"osd":6,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.367,"15min":1.373},"min":{"1min":1.056,"5min":1.056,"15min":1.056},"max":{"1min":1.72,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.347,"15min":1.353},"min":{"1min":1.046,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":1.573,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.081}]},{"osd":7,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3939999999999999,"15min":1.391},"min":{"1min":1.1719999999999999,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":1.8069999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.3799999999999999,"15min":1.3919999999999999},"min":{"1min":1.097,"5min":1.097,"15min":1.097},"max":{"1min":1.5529999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.1870000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542173,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297596,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074244,"statfs":{"total":96636764160,"available":96332025856,"internally_reserved":0,"allocated":512000,"data_stored":219115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.84899999999999998,"15min":0.86899999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1990000000000001,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.755},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.89700000000000002,"15min":0.89600000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1619999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.80200000000000005}]},{"osd":2,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.99399999999999999,"15min":1.0129999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.147,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.99199999999999999,"15min":0.98799999999999999},"min":{"1min":0.68000000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.2529999999999999,"5min":1.847,"15min":1.847},"last":1.1659999999999999}]},{"osd":3,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.96499999999999997,"15min":0.96199999999999997},"min":{"1min":0.76600000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.401,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.157,"15min":1.175},"min":{"1min":0.68899999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2210000000000001,"5min":3.54,"15min":3.54},"last":1.1120000000000001}]},{"osd":4,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2490000000000001,"15min":1.2470000000000001},"min":{"1min":1.0720000000000001,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.6319999999999999,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.175,"15min":1.175},"min":{"1min":1.006,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.4850000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.2250000000000001}]},{"osd":5,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2170000000000001,"15min":1.218},"min":{"1min":0.98899999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.393,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.3420000000000001,"15min":1.359},"min":{"1min":1.083,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.375,"5min":3.6760000000000002,"15min":3.6760000000000002},"last":1.0429999999999999}]},{"osd":6,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.411,"15min":1.429},"min":{"1min":1.113,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":1.4730000000000001,"5min":3.71,"15min":3.71},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.3340000000000001,"15min":1.343},"min":{"1min":1.05,"5min":1.024,"15min":1.024},"max":{"1min":1.4970000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.52}]},{"osd":7,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.3640000000000001,"15min":1.379},"min":{"1min":1.018,"5min":1.018,"15min":1.018},"max":{"1min":1.554,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2649999999999999,"15min":1.2609999999999999},"min":{"1min":1.1040000000000001,"5min":0.876,"15min":0.876},"max":{"1min":1.5449999999999999,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.278}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":565248,"data_stored":246790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.05,"15min":1.046},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4359999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.216,"15min":1.2370000000000001},"min":{"1min":0.69699999999999995,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.4630000000000001,"5min":3.46,"15min":3.46},"last":0.97699999999999998}]},{"osd":1,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.2310000000000001,"15min":1.2589999999999999},"min":{"1min":0.82699999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.47,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":1.052},{"interface":"front","average":{"1min":1.091,"5min":1.1739999999999999,"15min":1.1879999999999999},"min":{"1min":0.80300000000000005,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.4570000000000001,"5min":3.516,"15min":3.516},"last":1.099}]},{"osd":2,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.2070000000000001,"15min":1.2130000000000001},"min":{"1min":0.91600000000000004,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.3640000000000001,"5min":5.4180000000000001,"15min":5.4180000000000001},"last":1.179},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3069999999999999,"15min":1.321},"min":{"1min":0.94799999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.5089999999999999,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":0.90200000000000002}]},{"osd":3,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.29,"15min":1.3080000000000001},"min":{"1min":1.022,"5min":0.879,"15min":0.879},"max":{"1min":1.4850000000000001,"5min":2.141,"15min":2.141},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.448,"15min":1.464},"min":{"1min":1.1839999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":1.583,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":1.274}]},{"osd":4,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3680000000000001,"15min":1.3879999999999999},"min":{"1min":0.84199999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.5309999999999999,"5min":3.706,"15min":3.706},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.597,"15min":1.6379999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6479999999999999,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":1.1419999999999999}]},{"osd":6,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.327,"15min":1.3420000000000001},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.6439999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":1.347},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.546,"15min":1.5860000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.613,"5min":3.641,"15min":3.641},"last":1.22}]},{"osd":7,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4259999999999999,"15min":1.4370000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.762,"5min":3.516,"15min":3.516},"last":1.321},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.46,"15min":1.4650000000000001},"min":{"1min":1.139,"5min":1.022,"15min":1.022},"max":{"1min":1.722,"5min":3.754,"15min":3.754},"last":1.411}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297620,"kb_used_data":524,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074220,"statfs":{"total":96636764160,"available":96332001280,"internally_reserved":0,"allocated":536576,"data_stored":232015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.84999999999999998,"15min":0.84599999999999997},"min":{"1min":0.63600000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2150000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.72199999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.155,"5min":1.419,"15min":1.419},"last":0.64000000000000001}]},{"osd":2,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.85799999999999998,"15min":0.84799999999999998},"min":{"1min":0.495,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":1.1000000000000001},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.90700000000000003,"15min":0.91000000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.161,"5min":1.341,"15min":1.341},"last":0.96799999999999997}]},{"osd":3,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.0620000000000001,"15min":1.054},"min":{"1min":0.54100000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.98899999999999999,"15min":0.97699999999999998},"min":{"1min":0.67800000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.92700000000000005}]},{"osd":4,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.175,"15min":1.165},"min":{"1min":0.75700000000000001,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.49,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.248,"15min":1.2589999999999999},"min":{"1min":0.92000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.5609999999999999,"5min":2.206,"15min":2.206},"last":1.365}]},{"osd":5,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.26,"15min":1.2649999999999999},"min":{"1min":1.0089999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.4299999999999999,"5min":1.899,"15min":1.899},"last":1.153},{"interface":"front","average":{"1min":1.236,"5min":1.2829999999999999,"15min":1.2909999999999999},"min":{"1min":0.92800000000000005,"5min":0.871,"15min":0.871},"max":{"1min":1.5029999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.242}]},{"osd":6,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.323,"15min":1.325},"min":{"1min":0.98299999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.5229999999999999,"5min":1.825,"15min":1.825},"last":1.27},{"interface":"front","average":{"1min":1.401,"5min":1.4099999999999999,"15min":1.4119999999999999},"min":{"1min":1.1870000000000001,"5min":0.87,"15min":0.87},"max":{"1min":1.5529999999999999,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.5269999999999999}]},{"osd":7,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3200000000000001,"15min":1.3169999999999999},"min":{"1min":0.93999999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.6839999999999999,"5min":1.772,"15min":1.772},"last":1.415},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3540000000000001,"15min":1.3600000000000001},"min":{"1min":0.95499999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.6120000000000001,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.452}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T02:59:01.848 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T02:59:02.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.038+0000 7f48134b9700 1 Processor -- start 2023-11-15T02:59:02.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.038+0000 7f48134b9700 1 -- start start 2023-11-15T02:59:02.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.042+0000 7f48134b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f480c151870 0x7f480c151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.042+0000 7f48134b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f480c152140 0x7f480c0a9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.042+0000 7f48134b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f480c0657f0 con 0x7f480c0a9d20 2023-11-15T02:59:02.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.042+0000 7f48134b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f480c065ad0 con 0x7f480c151870 2023-11-15T02:59:02.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.042+0000 7f48134b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480c0aa260 con 0x7f480c152140 2023-11-15T02:59:02.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.042+0000 7f48124b7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f480c151870 0x7f480c151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.042+0000 7f4811cb6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f480c152140 0x7f480c0a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.042+0000 7f48124b7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f480c151870 0x7f480c151c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:56238/0 (socket says 172.21.15.121:56238) 2023-11-15T02:59:02.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.042+0000 7f48124b7700 1 -- 172.21.15.121:0/2963518095 learned_addr learned my addr 172.21.15.121:0/2963518095 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:02.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.042+0000 7f4812cb8700 1 --1- 172.21.15.121:0/2963518095 >> v1:172.21.15.190:6789/0 conn(0x7f480c0a9d20 0x7f480c0c30e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:48486/0 (socket says 172.21.15.121:48486) 2023-11-15T02:59:02.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.042+0000 7f48124b7700 1 -- 172.21.15.121:0/2963518095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f480c152140 msgr2=0x7f480c0a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:02.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.042+0000 7f48124b7700 1 --2- 172.21.15.121:0/2963518095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f480c152140 0x7f480c0a9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.042+0000 7f48124b7700 1 -- 172.21.15.121:0/2963518095 >> v1:172.21.15.190:6789/0 conn(0x7f480c0a9d20 legacy=0x7f480c0c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:02.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.042+0000 7f48124b7700 1 -- 172.21.15.121:0/2963518095 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480c0aa540 con 0x7f480c151870 2023-11-15T02:59:02.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.042+0000 7f48124b7700 1 --2- 172.21.15.121:0/2963518095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f480c151870 0x7f480c151c50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f4808006fd0 tx=0x7f48080090f0).ready entity=mon.1 client_cookie=daf7afa00a7b1558 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:02.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.042+0000 7f48114b5700 1 -- 172.21.15.121:0/2963518095 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480800b580 con 0x7f480c151870 2023-11-15T02:59:02.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.042+0000 7f48114b5700 1 -- 172.21.15.121:0/2963518095 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480800b6e0 con 0x7f480c151870 2023-11-15T02:59:02.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.042+0000 7f48114b5700 1 -- 172.21.15.121:0/2963518095 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480800ba60 con 0x7f480c151870 2023-11-15T02:59:02.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.046+0000 7f48134b9700 1 -- 172.21.15.121:0/2963518095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f480c151870 msgr2=0x7f480c151c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:02.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.046+0000 7f48134b9700 1 --2- 172.21.15.121:0/2963518095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f480c151870 0x7f480c151c50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f4808006fd0 tx=0x7f48080090f0).stop 2023-11-15T02:59:02.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.046+0000 7f48134b9700 1 -- 172.21.15.121:0/2963518095 shutdown_connections 2023-11-15T02:59:02.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.046+0000 7f48134b9700 1 --2- 172.21.15.121:0/2963518095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f480c152140 0x7f480c0a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.046+0000 7f48134b9700 1 --2- 172.21.15.121:0/2963518095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f480c151870 0x7f480c151c50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.046+0000 7f48134b9700 1 -- 172.21.15.121:0/2963518095 >> 172.21.15.121:0/2963518095 conn(0x7f480c062190 msgr2=0x7f480c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:02.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.046+0000 7f48134b9700 1 -- 172.21.15.121:0/2963518095 shutdown_connections 2023-11-15T02:59:02.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.046+0000 7f48134b9700 1 -- 172.21.15.121:0/2963518095 wait complete. 2023-11-15T02:59:02.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.046+0000 7f48134b9700 1 Processor -- start 2023-11-15T02:59:02.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.046+0000 7f48134b9700 1 -- start start 2023-11-15T02:59:02.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f48134b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f480c0a9d20 0x7f480c14ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f48134b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f480c151870 0x7f480c0c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f48134b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f480c152140 0x7f480c146740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f48134b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480c14b170 con 0x7f480c0a9d20 2023-11-15T02:59:02.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f48134b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480c0c4c40 con 0x7f480c152140 2023-11-15T02:59:02.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f48134b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480c0c72b0 con 0x7f480c151870 2023-11-15T02:59:02.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f48124b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f480c0a9d20 0x7f480c14ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f48124b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f480c0a9d20 0x7f480c14ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38046/0 (socket says 172.21.15.121:38046) 2023-11-15T02:59:02.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f48124b7700 1 -- 172.21.15.121:0/1581557543 learned_addr learned my addr 172.21.15.121:0/1581557543 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:02.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f4811cb6700 1 --2- 172.21.15.121:0/1581557543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f480c151870 0x7f480c0c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f4812cb8700 1 --2- 172.21.15.121:0/1581557543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f480c152140 0x7f480c146740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f48124b7700 1 -- 172.21.15.121:0/1581557543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f480c151870 msgr2=0x7f480c0c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:02.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f48124b7700 1 --2- 172.21.15.121:0/1581557543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f480c151870 0x7f480c0c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f48124b7700 1 -- 172.21.15.121:0/1581557543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f480c152140 msgr2=0x7f480c146740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:02.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f48124b7700 1 --2- 172.21.15.121:0/1581557543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f480c152140 0x7f480c146740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f48124b7700 1 -- 172.21.15.121:0/1581557543 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4808009370 con 0x7f480c0a9d20 2023-11-15T02:59:02.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f4811cb6700 1 --2- 172.21.15.121:0/1581557543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f480c151870 0x7f480c0c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:59:02.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f48124b7700 1 --2- 172.21.15.121:0/1581557543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f480c0a9d20 0x7f480c14ac80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f4808009130 tx=0x7f4808017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:02.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f48027fc700 1 -- 172.21.15.121:0/1581557543 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808000bc0 con 0x7f480c0a9d20 2023-11-15T02:59:02.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f48027fc700 1 -- 172.21.15.121:0/1581557543 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480800b850 con 0x7f480c0a9d20 2023-11-15T02:59:02.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f48027fc700 1 -- 172.21.15.121:0/1581557543 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808000d20 con 0x7f480c0a9d20 2023-11-15T02:59:02.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.050+0000 7f48134b9700 1 -- 172.21.15.121:0/1581557543 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f480c149490 con 0x7f480c0a9d20 2023-11-15T02:59:02.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.054+0000 7f48134b9700 1 -- 172.21.15.121:0/1581557543 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480c0c48d0 con 0x7f480c0a9d20 2023-11-15T02:59:02.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.054+0000 7f48134b9700 1 -- 172.21.15.121:0/1581557543 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f480c06bcf0 con 0x7f480c0a9d20 2023-11-15T02:59:02.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.054+0000 7f48027fc700 1 -- 172.21.15.121:0/1581557543 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f480801cad0 con 0x7f480c0a9d20 2023-11-15T02:59:02.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.058+0000 7f48027fc700 1 --2- 172.21.15.121:0/1581557543 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f47f005a510 0x7f47f005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.058+0000 7f48027fc700 1 -- 172.21.15.121:0/1581557543 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f48080765e0 con 0x7f480c0a9d20 2023-11-15T02:59:02.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.058+0000 7f4811cb6700 1 --2- 172.21.15.121:0/1581557543 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f47f005a510 0x7f47f005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.058+0000 7f4811cb6700 1 --2- 172.21.15.121:0/1581557543 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f47f005a510 0x7f47f005c9d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f480c148040 tx=0x7f47fc009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:02.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.062+0000 7f48027fc700 1 -- 172.21.15.121:0/1581557543 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4808049d00 con 0x7f480c0a9d20 2023-11-15T02:59:02.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.222+0000 7f48134b9700 1 -- 172.21.15.121:0/1581557543 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f480c0aa820 con 0x7f47f005a510 2023-11-15T02:59:02.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.242+0000 7f48027fc700 1 -- 172.21.15.121:0/1581557543 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+357898 (secure 0 0 0) 0x7f480c0aa820 con 0x7f47f005a510 2023-11-15T02:59:02.253 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:02.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.250+0000 7f48134b9700 1 -- 172.21.15.121:0/1581557543 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f47f005a510 msgr2=0x7f47f005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:02.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.250+0000 7f48134b9700 1 --2- 172.21.15.121:0/1581557543 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f47f005a510 0x7f47f005c9d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f480c148040 tx=0x7f47fc009060).stop 2023-11-15T02:59:02.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.250+0000 7f48134b9700 1 -- 172.21.15.121:0/1581557543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f480c0a9d20 msgr2=0x7f480c14ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:02.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.250+0000 7f48134b9700 1 --2- 172.21.15.121:0/1581557543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f480c0a9d20 0x7f480c14ac80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f4808009130 tx=0x7f4808017000).stop 2023-11-15T02:59:02.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.250+0000 7f48134b9700 1 -- 172.21.15.121:0/1581557543 shutdown_connections 2023-11-15T02:59:02.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.250+0000 7f48134b9700 1 --2- 172.21.15.121:0/1581557543 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f47f005a510 0x7f47f005c9d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.250+0000 7f48134b9700 1 --2- 172.21.15.121:0/1581557543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f480c152140 0x7f480c146740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.250+0000 7f48134b9700 1 --2- 172.21.15.121:0/1581557543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f480c151870 0x7f480c0c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.250+0000 7f48134b9700 1 --2- 172.21.15.121:0/1581557543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f480c0a9d20 0x7f480c14ac80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.250+0000 7f48134b9700 1 -- 172.21.15.121:0/1581557543 >> 172.21.15.121:0/1581557543 conn(0x7f480c062190 msgr2=0x7f480c0c16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:02.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.250+0000 7f48134b9700 1 -- 172.21.15.121:0/1581557543 shutdown_connections 2023-11-15T02:59:02.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.250+0000 7f48134b9700 1 -- 172.21.15.121:0/1581557543 wait complete. 2023-11-15T02:59:02.259 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T02:59:02.277 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2023-11-15T02:59:01.958103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381172,"kb_used_data":4404,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752593548,"statfs":{"total":773094113280,"available":770655793152,"internally_reserved":0,"allocated":4509696,"data_stored":1949850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29762,"num_objects":-6,"num_object_clones":0,"num_object_copies":-12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6,"num_whiteouts":0,"num_read":-54,"num_read_kb":-56,"num_write":-48,"num_write_kb":-50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.512563"},"pg_stats":[{"pgid":"8.30","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.632390+0000","last_change":"2023-11-15T02:58:57.632390+0000","last_active":"2023-11-15T02:58:57.632390+0000","last_peered":"2023-11-15T02:58:57.632390+0000","last_clean":"2023-11-15T02:58:57.632390+0000","last_became_active":"2023-11-15T02:58:57.632126+0000","last_became_peered":"2023-11-15T02:58:57.632126+0000","last_unstale":"2023-11-15T02:58:57.632390+0000","last_undegraded":"2023-11-15T02:58:57.632390+0000","last_fullsized":"2023-11-15T02:58:57.632390+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3f","version":"47'3","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.660337+0000","last_change":"2023-11-15T02:58:57.195836+0000","last_active":"2023-11-15T02:58:57.660337+0000","last_peered":"2023-11-15T02:58:57.660337+0000","last_clean":"2023-11-15T02:58:57.660337+0000","last_became_active":"2023-11-15T02:58:57.195598+0000","last_became_peered":"2023-11-15T02:58:57.195598+0000","last_unstale":"2023-11-15T02:58:57.660337+0000","last_undegraded":"2023-11-15T02:58:57.660337+0000","last_fullsized":"2023-11-15T02:58:57.660337+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.661146+0000","last_change":"2023-11-15T02:58:57.654729+0000","last_active":"2023-11-15T02:58:57.661146+0000","last_peered":"2023-11-15T02:58:57.661146+0000","last_clean":"2023-11-15T02:58:57.661146+0000","last_became_active":"2023-11-15T02:58:57.654460+0000","last_became_peered":"2023-11-15T02:58:57.654460+0000","last_unstale":"2023-11-15T02:58:57.661146+0000","last_undegraded":"2023-11-15T02:58:57.661146+0000","last_fullsized":"2023-11-15T02:58:57.661146+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.622234+0000","last_change":"2023-11-15T02:58:57.194667+0000","last_active":"2023-11-15T02:58:57.622234+0000","last_peered":"2023-11-15T02:58:57.622234+0000","last_clean":"2023-11-15T02:58:57.622234+0000","last_became_active":"2023-11-15T02:58:57.194280+0000","last_became_peered":"2023-11-15T02:58:57.194280+0000","last_unstale":"2023-11-15T02:58:57.622234+0000","last_undegraded":"2023-11-15T02:58:57.622234+0000","last_fullsized":"2023-11-15T02:58:57.622234+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.660324+0000","last_change":"2023-11-15T02:58:57.648322+0000","last_active":"2023-11-15T02:58:57.660324+0000","last_peered":"2023-11-15T02:58:57.660324+0000","last_clean":"2023-11-15T02:58:57.660324+0000","last_became_active":"2023-11-15T02:58:57.648045+0000","last_became_peered":"2023-11-15T02:58:57.648045+0000","last_unstale":"2023-11-15T02:58:57.660324+0000","last_undegraded":"2023-11-15T02:58:57.660324+0000","last_fullsized":"2023-11-15T02:58:57.660324+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.625774+0000","last_change":"2023-11-15T02:58:57.198423+0000","last_active":"2023-11-15T02:58:57.625774+0000","last_peered":"2023-11-15T02:58:57.625774+0000","last_clean":"2023-11-15T02:58:57.625774+0000","last_became_active":"2023-11-15T02:58:57.198150+0000","last_became_peered":"2023-11-15T02:58:57.198150+0000","last_unstale":"2023-11-15T02:58:57.625774+0000","last_undegraded":"2023-11-15T02:58:57.625774+0000","last_fullsized":"2023-11-15T02:58:57.625774+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.632654+0000","last_change":"2023-11-15T02:58:57.632654+0000","last_active":"2023-11-15T02:58:57.632654+0000","last_peered":"2023-11-15T02:58:57.632654+0000","last_clean":"2023-11-15T02:58:57.632654+0000","last_became_active":"2023-11-15T02:58:57.632380+0000","last_became_peered":"2023-11-15T02:58:57.632380+0000","last_unstale":"2023-11-15T02:58:57.632654+0000","last_undegraded":"2023-11-15T02:58:57.632654+0000","last_fullsized":"2023-11-15T02:58:57.632654+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.647398+0000","last_change":"2023-11-15T02:58:57.190310+0000","last_active":"2023-11-15T02:58:57.647398+0000","last_peered":"2023-11-15T02:58:57.647398+0000","last_clean":"2023-11-15T02:58:57.647398+0000","last_became_active":"2023-11-15T02:58:57.190003+0000","last_became_peered":"2023-11-15T02:58:57.190003+0000","last_unstale":"2023-11-15T02:58:57.647398+0000","last_undegraded":"2023-11-15T02:58:57.647398+0000","last_fullsized":"2023-11-15T02:58:57.647398+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.649814+0000","last_change":"2023-11-15T02:58:57.645739+0000","last_active":"2023-11-15T02:58:57.649814+0000","last_peered":"2023-11-15T02:58:57.649814+0000","last_clean":"2023-11-15T02:58:57.649814+0000","last_became_active":"2023-11-15T02:58:57.645447+0000","last_became_peered":"2023-11-15T02:58:57.645447+0000","last_unstale":"2023-11-15T02:58:57.649814+0000","last_undegraded":"2023-11-15T02:58:57.649814+0000","last_fullsized":"2023-11-15T02:58:57.649814+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3b","version":"47'1","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.641614+0000","last_change":"2023-11-15T02:58:57.195055+0000","last_active":"2023-11-15T02:58:57.641614+0000","last_peered":"2023-11-15T02:58:57.641614+0000","last_clean":"2023-11-15T02:58:57.641614+0000","last_became_active":"2023-11-15T02:58:57.194269+0000","last_became_peered":"2023-11-15T02:58:57.194269+0000","last_unstale":"2023-11-15T02:58:57.641614+0000","last_undegraded":"2023-11-15T02:58:57.641614+0000","last_fullsized":"2023-11-15T02:58:57.641614+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.659588+0000","last_change":"2023-11-15T02:58:57.654040+0000","last_active":"2023-11-15T02:58:57.659588+0000","last_peered":"2023-11-15T02:58:57.659588+0000","last_clean":"2023-11-15T02:58:57.659588+0000","last_became_active":"2023-11-15T02:58:57.653628+0000","last_became_peered":"2023-11-15T02:58:57.653628+0000","last_unstale":"2023-11-15T02:58:57.659588+0000","last_undegraded":"2023-11-15T02:58:57.659588+0000","last_fullsized":"2023-11-15T02:58:57.659588+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3a","version":"47'1","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.659784+0000","last_change":"2023-11-15T02:58:57.200595+0000","last_active":"2023-11-15T02:58:57.659784+0000","last_peered":"2023-11-15T02:58:57.659784+0000","last_clean":"2023-11-15T02:58:57.659784+0000","last_became_active":"2023-11-15T02:58:57.200298+0000","last_became_peered":"2023-11-15T02:58:57.200298+0000","last_unstale":"2023-11-15T02:58:57.659784+0000","last_undegraded":"2023-11-15T02:58:57.659784+0000","last_fullsized":"2023-11-15T02:58:57.659784+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.637020+0000","last_change":"2023-11-15T02:58:57.637020+0000","last_active":"2023-11-15T02:58:57.637020+0000","last_peered":"2023-11-15T02:58:57.637020+0000","last_clean":"2023-11-15T02:58:57.637020+0000","last_became_active":"2023-11-15T02:58:57.636627+0000","last_became_peered":"2023-11-15T02:58:57.636627+0000","last_unstale":"2023-11-15T02:58:57.637020+0000","last_undegraded":"2023-11-15T02:58:57.637020+0000","last_fullsized":"2023-11-15T02:58:57.637020+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.635012+0000","last_change":"2023-11-15T02:58:57.189601+0000","last_active":"2023-11-15T02:58:57.635012+0000","last_peered":"2023-11-15T02:58:57.635012+0000","last_clean":"2023-11-15T02:58:57.635012+0000","last_became_active":"2023-11-15T02:58:57.188820+0000","last_became_peered":"2023-11-15T02:58:57.188820+0000","last_unstale":"2023-11-15T02:58:57.635012+0000","last_undegraded":"2023-11-15T02:58:57.635012+0000","last_fullsized":"2023-11-15T02:58:57.635012+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.660263+0000","last_change":"2023-11-15T02:58:57.652107+0000","last_active":"2023-11-15T02:58:57.660263+0000","last_peered":"2023-11-15T02:58:57.660263+0000","last_clean":"2023-11-15T02:58:57.660263+0000","last_became_active":"2023-11-15T02:58:57.651646+0000","last_became_peered":"2023-11-15T02:58:57.651646+0000","last_unstale":"2023-11-15T02:58:57.660263+0000","last_undegraded":"2023-11-15T02:58:57.660263+0000","last_fullsized":"2023-11-15T02:58:57.660263+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.636612+0000","last_change":"2023-11-15T02:58:57.187999+0000","last_active":"2023-11-15T02:58:57.636612+0000","last_peered":"2023-11-15T02:58:57.636612+0000","last_clean":"2023-11-15T02:58:57.636612+0000","last_became_active":"2023-11-15T02:58:57.187538+0000","last_became_peered":"2023-11-15T02:58:57.187538+0000","last_unstale":"2023-11-15T02:58:57.636612+0000","last_undegraded":"2023-11-15T02:58:57.636612+0000","last_fullsized":"2023-11-15T02:58:57.636612+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.650768+0000","last_change":"2023-11-15T02:58:57.647526+0000","last_active":"2023-11-15T02:58:57.650768+0000","last_peered":"2023-11-15T02:58:57.650768+0000","last_clean":"2023-11-15T02:58:57.650768+0000","last_became_active":"2023-11-15T02:58:57.647263+0000","last_became_peered":"2023-11-15T02:58:57.647263+0000","last_unstale":"2023-11-15T02:58:57.650768+0000","last_undegraded":"2023-11-15T02:58:57.650768+0000","last_fullsized":"2023-11-15T02:58:57.650768+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.642348+0000","last_change":"2023-11-15T02:58:57.191151+0000","last_active":"2023-11-15T02:58:57.642348+0000","last_peered":"2023-11-15T02:58:57.642348+0000","last_clean":"2023-11-15T02:58:57.642348+0000","last_became_active":"2023-11-15T02:58:57.190800+0000","last_became_peered":"2023-11-15T02:58:57.190800+0000","last_unstale":"2023-11-15T02:58:57.642348+0000","last_undegraded":"2023-11-15T02:58:57.642348+0000","last_fullsized":"2023-11-15T02:58:57.642348+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.660659+0000","last_change":"2023-11-15T02:58:57.652915+0000","last_active":"2023-11-15T02:58:57.660659+0000","last_peered":"2023-11-15T02:58:57.660659+0000","last_clean":"2023-11-15T02:58:57.660659+0000","last_became_active":"2023-11-15T02:58:57.652370+0000","last_became_peered":"2023-11-15T02:58:57.652370+0000","last_unstale":"2023-11-15T02:58:57.660659+0000","last_undegraded":"2023-11-15T02:58:57.660659+0000","last_fullsized":"2023-11-15T02:58:57.660659+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.640283+0000","last_change":"2023-11-15T02:58:57.184331+0000","last_active":"2023-11-15T02:58:57.640283+0000","last_peered":"2023-11-15T02:58:57.640283+0000","last_clean":"2023-11-15T02:58:57.640283+0000","last_became_active":"2023-11-15T02:58:57.184009+0000","last_became_peered":"2023-11-15T02:58:57.184009+0000","last_unstale":"2023-11-15T02:58:57.640283+0000","last_undegraded":"2023-11-15T02:58:57.640283+0000","last_fullsized":"2023-11-15T02:58:57.640283+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.653843+0000","last_change":"2023-11-15T02:58:57.653843+0000","last_active":"2023-11-15T02:58:57.653843+0000","last_peered":"2023-11-15T02:58:57.653843+0000","last_clean":"2023-11-15T02:58:57.653843+0000","last_became_active":"2023-11-15T02:58:57.653352+0000","last_became_peered":"2023-11-15T02:58:57.653352+0000","last_unstale":"2023-11-15T02:58:57.653843+0000","last_undegraded":"2023-11-15T02:58:57.653843+0000","last_fullsized":"2023-11-15T02:58:57.653843+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.639232+0000","last_change":"2023-11-15T02:58:57.186920+0000","last_active":"2023-11-15T02:58:57.639232+0000","last_peered":"2023-11-15T02:58:57.639232+0000","last_clean":"2023-11-15T02:58:57.639232+0000","last_became_active":"2023-11-15T02:58:57.185983+0000","last_became_peered":"2023-11-15T02:58:57.185983+0000","last_unstale":"2023-11-15T02:58:57.639232+0000","last_undegraded":"2023-11-15T02:58:57.639232+0000","last_fullsized":"2023-11-15T02:58:57.639232+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.650011+0000","last_change":"2023-11-15T02:58:57.647479+0000","last_active":"2023-11-15T02:58:57.650011+0000","last_peered":"2023-11-15T02:58:57.650011+0000","last_clean":"2023-11-15T02:58:57.650011+0000","last_became_active":"2023-11-15T02:58:57.647165+0000","last_became_peered":"2023-11-15T02:58:57.647165+0000","last_unstale":"2023-11-15T02:58:57.650011+0000","last_undegraded":"2023-11-15T02:58:57.650011+0000","last_fullsized":"2023-11-15T02:58:57.650011+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.34","version":"47'4","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.658044+0000","last_change":"2023-11-15T02:58:57.183738+0000","last_active":"2023-11-15T02:58:57.658044+0000","last_peered":"2023-11-15T02:58:57.658044+0000","last_clean":"2023-11-15T02:58:57.658044+0000","last_became_active":"2023-11-15T02:58:57.179352+0000","last_became_peered":"2023-11-15T02:58:57.179352+0000","last_unstale":"2023-11-15T02:58:57.658044+0000","last_undegraded":"2023-11-15T02:58:57.658044+0000","last_fullsized":"2023-11-15T02:58:57.658044+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.661109+0000","last_change":"2023-11-15T02:58:57.654131+0000","last_active":"2023-11-15T02:58:57.661109+0000","last_peered":"2023-11-15T02:58:57.661109+0000","last_clean":"2023-11-15T02:58:57.661109+0000","last_became_active":"2023-11-15T02:58:57.653784+0000","last_became_peered":"2023-11-15T02:58:57.653784+0000","last_unstale":"2023-11-15T02:58:57.661109+0000","last_undegraded":"2023-11-15T02:58:57.661109+0000","last_fullsized":"2023-11-15T02:58:57.661109+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.33","version":"47'1","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.641730+0000","last_change":"2023-11-15T02:58:57.197387+0000","last_active":"2023-11-15T02:58:57.641730+0000","last_peered":"2023-11-15T02:58:57.641730+0000","last_clean":"2023-11-15T02:58:57.641730+0000","last_became_active":"2023-11-15T02:58:57.197070+0000","last_became_peered":"2023-11-15T02:58:57.197070+0000","last_unstale":"2023-11-15T02:58:57.641730+0000","last_undegraded":"2023-11-15T02:58:57.641730+0000","last_fullsized":"2023-11-15T02:58:57.641730+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.660605+0000","last_change":"2023-11-15T02:58:57.651996+0000","last_active":"2023-11-15T02:58:57.660605+0000","last_peered":"2023-11-15T02:58:57.660605+0000","last_clean":"2023-11-15T02:58:57.660605+0000","last_became_active":"2023-11-15T02:58:57.651180+0000","last_became_peered":"2023-11-15T02:58:57.651180+0000","last_unstale":"2023-11-15T02:58:57.660605+0000","last_undegraded":"2023-11-15T02:58:57.660605+0000","last_fullsized":"2023-11-15T02:58:57.660605+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.637911+0000","last_change":"2023-11-15T02:58:57.186841+0000","last_active":"2023-11-15T02:58:57.637911+0000","last_peered":"2023-11-15T02:58:57.637911+0000","last_clean":"2023-11-15T02:58:57.637911+0000","last_became_active":"2023-11-15T02:58:57.186477+0000","last_became_peered":"2023-11-15T02:58:57.186477+0000","last_unstale":"2023-11-15T02:58:57.637911+0000","last_undegraded":"2023-11-15T02:58:57.637911+0000","last_fullsized":"2023-11-15T02:58:57.637911+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.661441+0000","last_change":"2023-11-15T02:58:57.656477+0000","last_active":"2023-11-15T02:58:57.661441+0000","last_peered":"2023-11-15T02:58:57.661441+0000","last_clean":"2023-11-15T02:58:57.661441+0000","last_became_active":"2023-11-15T02:58:57.656007+0000","last_became_peered":"2023-11-15T02:58:57.656007+0000","last_unstale":"2023-11-15T02:58:57.661441+0000","last_undegraded":"2023-11-15T02:58:57.661441+0000","last_fullsized":"2023-11-15T02:58:57.661441+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.619118+0000","last_change":"2023-11-15T02:58:57.199099+0000","last_active":"2023-11-15T02:58:57.619118+0000","last_peered":"2023-11-15T02:58:57.619118+0000","last_clean":"2023-11-15T02:58:57.619118+0000","last_became_active":"2023-11-15T02:58:57.198839+0000","last_became_peered":"2023-11-15T02:58:57.198839+0000","last_unstale":"2023-11-15T02:58:57.619118+0000","last_undegraded":"2023-11-15T02:58:57.619118+0000","last_fullsized":"2023-11-15T02:58:57.619118+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.632675+0000","last_change":"2023-11-15T02:58:57.632675+0000","last_active":"2023-11-15T02:58:57.632675+0000","last_peered":"2023-11-15T02:58:57.632675+0000","last_clean":"2023-11-15T02:58:57.632675+0000","last_became_active":"2023-11-15T02:58:57.632390+0000","last_became_peered":"2023-11-15T02:58:57.632390+0000","last_unstale":"2023-11-15T02:58:57.632675+0000","last_undegraded":"2023-11-15T02:58:57.632675+0000","last_fullsized":"2023-11-15T02:58:57.632675+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.30","version":"47'4","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.658571+0000","last_change":"2023-11-15T02:58:57.195228+0000","last_active":"2023-11-15T02:58:57.658571+0000","last_peered":"2023-11-15T02:58:57.658571+0000","last_clean":"2023-11-15T02:58:57.658571+0000","last_became_active":"2023-11-15T02:58:57.194961+0000","last_became_peered":"2023-11-15T02:58:57.194961+0000","last_unstale":"2023-11-15T02:58:57.658571+0000","last_undegraded":"2023-11-15T02:58:57.658571+0000","last_fullsized":"2023-11-15T02:58:57.658571+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.637366+0000","last_change":"2023-11-15T02:58:57.635264+0000","last_active":"2023-11-15T02:58:57.637366+0000","last_peered":"2023-11-15T02:58:57.637366+0000","last_clean":"2023-11-15T02:58:57.637366+0000","last_became_active":"2023-11-15T02:58:57.634849+0000","last_became_peered":"2023-11-15T02:58:57.634849+0000","last_unstale":"2023-11-15T02:58:57.637366+0000","last_undegraded":"2023-11-15T02:58:57.637366+0000","last_fullsized":"2023-11-15T02:58:57.637366+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.626197+0000","last_change":"2023-11-15T02:58:57.196191+0000","last_active":"2023-11-15T02:58:57.626197+0000","last_peered":"2023-11-15T02:58:57.626197+0000","last_clean":"2023-11-15T02:58:57.626197+0000","last_became_active":"2023-11-15T02:58:57.193278+0000","last_became_peered":"2023-11-15T02:58:57.193278+0000","last_unstale":"2023-11-15T02:58:57.626197+0000","last_undegraded":"2023-11-15T02:58:57.626197+0000","last_fullsized":"2023-11-15T02:58:57.626197+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.660622+0000","last_change":"2023-11-15T02:58:57.651995+0000","last_active":"2023-11-15T02:58:57.660622+0000","last_peered":"2023-11-15T02:58:57.660622+0000","last_clean":"2023-11-15T02:58:57.660622+0000","last_became_active":"2023-11-15T02:58:57.651717+0000","last_became_peered":"2023-11-15T02:58:57.651717+0000","last_unstale":"2023-11-15T02:58:57.660622+0000","last_undegraded":"2023-11-15T02:58:57.660622+0000","last_fullsized":"2023-11-15T02:58:57.660622+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.625947+0000","last_change":"2023-11-15T02:58:57.197615+0000","last_active":"2023-11-15T02:58:57.625947+0000","last_peered":"2023-11-15T02:58:57.625947+0000","last_clean":"2023-11-15T02:58:57.625947+0000","last_became_active":"2023-11-15T02:58:57.197297+0000","last_became_peered":"2023-11-15T02:58:57.197297+0000","last_unstale":"2023-11-15T02:58:57.625947+0000","last_undegraded":"2023-11-15T02:58:57.625947+0000","last_fullsized":"2023-11-15T02:58:57.625947+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.661053+0000","last_change":"2023-11-15T02:58:57.652748+0000","last_active":"2023-11-15T02:58:57.661053+0000","last_peered":"2023-11-15T02:58:57.661053+0000","last_clean":"2023-11-15T02:58:57.661053+0000","last_became_active":"2023-11-15T02:58:57.652464+0000","last_became_peered":"2023-11-15T02:58:57.652464+0000","last_unstale":"2023-11-15T02:58:57.661053+0000","last_undegraded":"2023-11-15T02:58:57.661053+0000","last_fullsized":"2023-11-15T02:58:57.661053+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2d","version":"47'1","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.658336+0000","last_change":"2023-11-15T02:58:57.200442+0000","last_active":"2023-11-15T02:58:57.658336+0000","last_peered":"2023-11-15T02:58:57.658336+0000","last_clean":"2023-11-15T02:58:57.658336+0000","last_became_active":"2023-11-15T02:58:57.200116+0000","last_became_peered":"2023-11-15T02:58:57.200116+0000","last_unstale":"2023-11-15T02:58:57.658336+0000","last_undegraded":"2023-11-15T02:58:57.658336+0000","last_fullsized":"2023-11-15T02:58:57.658336+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.660167+0000","last_change":"2023-11-15T02:58:57.645121+0000","last_active":"2023-11-15T02:58:57.660167+0000","last_peered":"2023-11-15T02:58:57.660167+0000","last_clean":"2023-11-15T02:58:57.660167+0000","last_became_active":"2023-11-15T02:58:57.644828+0000","last_became_peered":"2023-11-15T02:58:57.644828+0000","last_unstale":"2023-11-15T02:58:57.660167+0000","last_undegraded":"2023-11-15T02:58:57.660167+0000","last_fullsized":"2023-11-15T02:58:57.660167+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.628283+0000","last_change":"2023-11-15T02:58:57.191374+0000","last_active":"2023-11-15T02:58:57.628283+0000","last_peered":"2023-11-15T02:58:57.628283+0000","last_clean":"2023-11-15T02:58:57.628283+0000","last_became_active":"2023-11-15T02:58:57.191051+0000","last_became_peered":"2023-11-15T02:58:57.191051+0000","last_unstale":"2023-11-15T02:58:57.628283+0000","last_undegraded":"2023-11-15T02:58:57.628283+0000","last_fullsized":"2023-11-15T02:58:57.628283+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.656127+0000","last_change":"2023-11-15T02:58:57.654816+0000","last_active":"2023-11-15T02:58:57.656127+0000","last_peered":"2023-11-15T02:58:57.656127+0000","last_clean":"2023-11-15T02:58:57.656127+0000","last_became_active":"2023-11-15T02:58:57.654390+0000","last_became_peered":"2023-11-15T02:58:57.654390+0000","last_unstale":"2023-11-15T02:58:57.656127+0000","last_undegraded":"2023-11-15T02:58:57.656127+0000","last_fullsized":"2023-11-15T02:58:57.656127+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2b","version":"47'1","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.656884+0000","last_change":"2023-11-15T02:58:57.194868+0000","last_active":"2023-11-15T02:58:57.656884+0000","last_peered":"2023-11-15T02:58:57.656884+0000","last_clean":"2023-11-15T02:58:57.656884+0000","last_became_active":"2023-11-15T02:58:57.194571+0000","last_became_peered":"2023-11-15T02:58:57.194571+0000","last_unstale":"2023-11-15T02:58:57.656884+0000","last_undegraded":"2023-11-15T02:58:57.656884+0000","last_fullsized":"2023-11-15T02:58:57.656884+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.644118+0000","last_change":"2023-11-15T02:58:57.644118+0000","last_active":"2023-11-15T02:58:57.644118+0000","last_peered":"2023-11-15T02:58:57.644118+0000","last_clean":"2023-11-15T02:58:57.644118+0000","last_became_active":"2023-11-15T02:58:57.643849+0000","last_became_peered":"2023-11-15T02:58:57.643849+0000","last_unstale":"2023-11-15T02:58:57.644118+0000","last_undegraded":"2023-11-15T02:58:57.644118+0000","last_fullsized":"2023-11-15T02:58:57.644118+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.638024+0000","last_change":"2023-11-15T02:58:57.191467+0000","last_active":"2023-11-15T02:58:57.638024+0000","last_peered":"2023-11-15T02:58:57.638024+0000","last_clean":"2023-11-15T02:58:57.638024+0000","last_became_active":"2023-11-15T02:58:57.191188+0000","last_became_peered":"2023-11-15T02:58:57.191188+0000","last_unstale":"2023-11-15T02:58:57.638024+0000","last_undegraded":"2023-11-15T02:58:57.638024+0000","last_fullsized":"2023-11-15T02:58:57.638024+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.651527+0000","last_change":"2023-11-15T02:58:57.648633+0000","last_active":"2023-11-15T02:58:57.651527+0000","last_peered":"2023-11-15T02:58:57.651527+0000","last_clean":"2023-11-15T02:58:57.651527+0000","last_became_active":"2023-11-15T02:58:57.648303+0000","last_became_peered":"2023-11-15T02:58:57.648303+0000","last_unstale":"2023-11-15T02:58:57.651527+0000","last_undegraded":"2023-11-15T02:58:57.651527+0000","last_fullsized":"2023-11-15T02:58:57.651527+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.638539+0000","last_change":"2023-11-15T02:58:57.186986+0000","last_active":"2023-11-15T02:58:57.638539+0000","last_peered":"2023-11-15T02:58:57.638539+0000","last_clean":"2023-11-15T02:58:57.638539+0000","last_became_active":"2023-11-15T02:58:57.186694+0000","last_became_peered":"2023-11-15T02:58:57.186694+0000","last_unstale":"2023-11-15T02:58:57.638539+0000","last_undegraded":"2023-11-15T02:58:57.638539+0000","last_fullsized":"2023-11-15T02:58:57.638539+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.660469+0000","last_change":"2023-11-15T02:58:57.650040+0000","last_active":"2023-11-15T02:58:57.660469+0000","last_peered":"2023-11-15T02:58:57.660469+0000","last_clean":"2023-11-15T02:58:57.660469+0000","last_became_active":"2023-11-15T02:58:57.649231+0000","last_became_peered":"2023-11-15T02:58:57.649231+0000","last_unstale":"2023-11-15T02:58:57.660469+0000","last_undegraded":"2023-11-15T02:58:57.660469+0000","last_fullsized":"2023-11-15T02:58:57.660469+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.633149+0000","last_change":"2023-11-15T02:58:57.194015+0000","last_active":"2023-11-15T02:58:57.633149+0000","last_peered":"2023-11-15T02:58:57.633149+0000","last_clean":"2023-11-15T02:58:57.633149+0000","last_became_active":"2023-11-15T02:58:57.193220+0000","last_became_peered":"2023-11-15T02:58:57.193220+0000","last_unstale":"2023-11-15T02:58:57.633149+0000","last_undegraded":"2023-11-15T02:58:57.633149+0000","last_fullsized":"2023-11-15T02:58:57.633149+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.660301+0000","last_change":"2023-11-15T02:58:57.645840+0000","last_active":"2023-11-15T02:58:57.660301+0000","last_peered":"2023-11-15T02:58:57.660301+0000","last_clean":"2023-11-15T02:58:57.660301+0000","last_became_active":"2023-11-15T02:58:57.645614+0000","last_became_peered":"2023-11-15T02:58:57.645614+0000","last_unstale":"2023-11-15T02:58:57.660301+0000","last_undegraded":"2023-11-15T02:58:57.660301+0000","last_fullsized":"2023-11-15T02:58:57.660301+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.629921+0000","last_change":"2023-11-15T02:58:57.189809+0000","last_active":"2023-11-15T02:58:57.629921+0000","last_peered":"2023-11-15T02:58:57.629921+0000","last_clean":"2023-11-15T02:58:57.629921+0000","last_became_active":"2023-11-15T02:58:57.189498+0000","last_became_peered":"2023-11-15T02:58:57.189498+0000","last_unstale":"2023-11-15T02:58:57.629921+0000","last_undegraded":"2023-11-15T02:58:57.629921+0000","last_fullsized":"2023-11-15T02:58:57.629921+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.659585+0000","last_change":"2023-11-15T02:58:57.654208+0000","last_active":"2023-11-15T02:58:57.659585+0000","last_peered":"2023-11-15T02:58:57.659585+0000","last_clean":"2023-11-15T02:58:57.659585+0000","last_became_active":"2023-11-15T02:58:57.653710+0000","last_became_peered":"2023-11-15T02:58:57.653710+0000","last_unstale":"2023-11-15T02:58:57.659585+0000","last_undegraded":"2023-11-15T02:58:57.659585+0000","last_fullsized":"2023-11-15T02:58:57.659585+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.629670+0000","last_change":"2023-11-15T02:58:57.185785+0000","last_active":"2023-11-15T02:58:57.629670+0000","last_peered":"2023-11-15T02:58:57.629670+0000","last_clean":"2023-11-15T02:58:57.629670+0000","last_became_active":"2023-11-15T02:58:57.185471+0000","last_became_peered":"2023-11-15T02:58:57.185471+0000","last_unstale":"2023-11-15T02:58:57.629670+0000","last_undegraded":"2023-11-15T02:58:57.629670+0000","last_fullsized":"2023-11-15T02:58:57.629670+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.649910+0000","last_change":"2023-11-15T02:58:57.646306+0000","last_active":"2023-11-15T02:58:57.649910+0000","last_peered":"2023-11-15T02:58:57.649910+0000","last_clean":"2023-11-15T02:58:57.649910+0000","last_became_active":"2023-11-15T02:58:57.646034+0000","last_became_peered":"2023-11-15T02:58:57.646034+0000","last_unstale":"2023-11-15T02:58:57.649910+0000","last_undegraded":"2023-11-15T02:58:57.649910+0000","last_fullsized":"2023-11-15T02:58:57.649910+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.625903+0000","last_change":"2023-11-15T02:58:57.197811+0000","last_active":"2023-11-15T02:58:57.625903+0000","last_peered":"2023-11-15T02:58:57.625903+0000","last_clean":"2023-11-15T02:58:57.625903+0000","last_became_active":"2023-11-15T02:58:57.189992+0000","last_became_peered":"2023-11-15T02:58:57.189992+0000","last_unstale":"2023-11-15T02:58:57.625903+0000","last_undegraded":"2023-11-15T02:58:57.625903+0000","last_fullsized":"2023-11-15T02:58:57.625903+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.661701+0000","last_change":"2023-11-15T02:58:57.658672+0000","last_active":"2023-11-15T02:58:57.661701+0000","last_peered":"2023-11-15T02:58:57.661701+0000","last_clean":"2023-11-15T02:58:57.661701+0000","last_became_active":"2023-11-15T02:58:57.658332+0000","last_became_peered":"2023-11-15T02:58:57.658332+0000","last_unstale":"2023-11-15T02:58:57.661701+0000","last_undegraded":"2023-11-15T02:58:57.661701+0000","last_fullsized":"2023-11-15T02:58:57.661701+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.625321+0000","last_change":"2023-11-15T02:58:57.181901+0000","last_active":"2023-11-15T02:58:57.625321+0000","last_peered":"2023-11-15T02:58:57.625321+0000","last_clean":"2023-11-15T02:58:57.625321+0000","last_became_active":"2023-11-15T02:58:57.178952+0000","last_became_peered":"2023-11-15T02:58:57.178952+0000","last_unstale":"2023-11-15T02:58:57.625321+0000","last_undegraded":"2023-11-15T02:58:57.625321+0000","last_fullsized":"2023-11-15T02:58:57.625321+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.654941+0000","last_change":"2023-11-15T02:58:57.649205+0000","last_active":"2023-11-15T02:58:57.654941+0000","last_peered":"2023-11-15T02:58:57.654941+0000","last_clean":"2023-11-15T02:58:57.654941+0000","last_became_active":"2023-11-15T02:58:57.648893+0000","last_became_peered":"2023-11-15T02:58:57.648893+0000","last_unstale":"2023-11-15T02:58:57.654941+0000","last_undegraded":"2023-11-15T02:58:57.654941+0000","last_fullsized":"2023-11-15T02:58:57.654941+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.631021+0000","last_change":"2023-11-15T02:58:57.189107+0000","last_active":"2023-11-15T02:58:57.631021+0000","last_peered":"2023-11-15T02:58:57.631021+0000","last_clean":"2023-11-15T02:58:57.631021+0000","last_became_active":"2023-11-15T02:58:57.188858+0000","last_became_peered":"2023-11-15T02:58:57.188858+0000","last_unstale":"2023-11-15T02:58:57.631021+0000","last_undegraded":"2023-11-15T02:58:57.631021+0000","last_fullsized":"2023-11-15T02:58:57.631021+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.660636+0000","last_change":"2023-11-15T02:58:57.652177+0000","last_active":"2023-11-15T02:58:57.660636+0000","last_peered":"2023-11-15T02:58:57.660636+0000","last_clean":"2023-11-15T02:58:57.660636+0000","last_became_active":"2023-11-15T02:58:57.651490+0000","last_became_peered":"2023-11-15T02:58:57.651490+0000","last_unstale":"2023-11-15T02:58:57.660636+0000","last_undegraded":"2023-11-15T02:58:57.660636+0000","last_fullsized":"2023-11-15T02:58:57.660636+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.625663+0000","last_change":"2023-11-15T02:58:57.193584+0000","last_active":"2023-11-15T02:58:57.625663+0000","last_peered":"2023-11-15T02:58:57.625663+0000","last_clean":"2023-11-15T02:58:57.625663+0000","last_became_active":"2023-11-15T02:58:57.193038+0000","last_became_peered":"2023-11-15T02:58:57.193038+0000","last_unstale":"2023-11-15T02:58:57.625663+0000","last_undegraded":"2023-11-15T02:58:57.625663+0000","last_fullsized":"2023-11-15T02:58:57.625663+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.661590+0000","last_change":"2023-11-15T02:58:57.658134+0000","last_active":"2023-11-15T02:58:57.661590+0000","last_peered":"2023-11-15T02:58:57.661590+0000","last_clean":"2023-11-15T02:58:57.661590+0000","last_became_active":"2023-11-15T02:58:57.657849+0000","last_became_peered":"2023-11-15T02:58:57.657849+0000","last_unstale":"2023-11-15T02:58:57.661590+0000","last_undegraded":"2023-11-15T02:58:57.661590+0000","last_fullsized":"2023-11-15T02:58:57.661590+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.619276+0000","last_change":"2023-11-15T02:58:57.198984+0000","last_active":"2023-11-15T02:58:57.619276+0000","last_peered":"2023-11-15T02:58:57.619276+0000","last_clean":"2023-11-15T02:58:57.619276+0000","last_became_active":"2023-11-15T02:58:57.198558+0000","last_became_peered":"2023-11-15T02:58:57.198558+0000","last_unstale":"2023-11-15T02:58:57.619276+0000","last_undegraded":"2023-11-15T02:58:57.619276+0000","last_fullsized":"2023-11-15T02:58:57.619276+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.660525+0000","last_change":"2023-11-15T02:58:57.651483+0000","last_active":"2023-11-15T02:58:57.660525+0000","last_peered":"2023-11-15T02:58:57.660525+0000","last_clean":"2023-11-15T02:58:57.660525+0000","last_became_active":"2023-11-15T02:58:57.651201+0000","last_became_peered":"2023-11-15T02:58:57.651201+0000","last_unstale":"2023-11-15T02:58:57.660525+0000","last_undegraded":"2023-11-15T02:58:57.660525+0000","last_fullsized":"2023-11-15T02:58:57.660525+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.626954+0000","last_change":"2023-11-15T02:58:57.180939+0000","last_active":"2023-11-15T02:58:57.626954+0000","last_peered":"2023-11-15T02:58:57.626954+0000","last_clean":"2023-11-15T02:58:57.626954+0000","last_became_active":"2023-11-15T02:58:57.180662+0000","last_became_peered":"2023-11-15T02:58:57.180662+0000","last_unstale":"2023-11-15T02:58:57.626954+0000","last_undegraded":"2023-11-15T02:58:57.626954+0000","last_fullsized":"2023-11-15T02:58:57.626954+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.661151+0000","last_change":"2023-11-15T02:58:57.654864+0000","last_active":"2023-11-15T02:58:57.661151+0000","last_peered":"2023-11-15T02:58:57.661151+0000","last_clean":"2023-11-15T02:58:57.661151+0000","last_became_active":"2023-11-15T02:58:57.654603+0000","last_became_peered":"2023-11-15T02:58:57.654603+0000","last_unstale":"2023-11-15T02:58:57.661151+0000","last_undegraded":"2023-11-15T02:58:57.661151+0000","last_fullsized":"2023-11-15T02:58:57.661151+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"47'1","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.639596+0000","last_change":"2023-11-15T02:58:57.196012+0000","last_active":"2023-11-15T02:58:57.639596+0000","last_peered":"2023-11-15T02:58:57.639596+0000","last_clean":"2023-11-15T02:58:57.639596+0000","last_became_active":"2023-11-15T02:58:57.195666+0000","last_became_peered":"2023-11-15T02:58:57.195666+0000","last_unstale":"2023-11-15T02:58:57.639596+0000","last_undegraded":"2023-11-15T02:58:57.639596+0000","last_fullsized":"2023-11-15T02:58:57.639596+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.656731+0000","last_change":"2023-11-15T02:58:57.653842+0000","last_active":"2023-11-15T02:58:57.656731+0000","last_peered":"2023-11-15T02:58:57.656731+0000","last_clean":"2023-11-15T02:58:57.656731+0000","last_became_active":"2023-11-15T02:58:57.653543+0000","last_became_peered":"2023-11-15T02:58:57.653543+0000","last_unstale":"2023-11-15T02:58:57.656731+0000","last_undegraded":"2023-11-15T02:58:57.656731+0000","last_fullsized":"2023-11-15T02:58:57.656731+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"47'1","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.628131+0000","last_change":"2023-11-15T02:58:57.195494+0000","last_active":"2023-11-15T02:58:57.628131+0000","last_peered":"2023-11-15T02:58:57.628131+0000","last_clean":"2023-11-15T02:58:57.628131+0000","last_became_active":"2023-11-15T02:58:57.195196+0000","last_became_peered":"2023-11-15T02:58:57.195196+0000","last_unstale":"2023-11-15T02:58:57.628131+0000","last_undegraded":"2023-11-15T02:58:57.628131+0000","last_fullsized":"2023-11-15T02:58:57.628131+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.661355+0000","last_change":"2023-11-15T02:58:57.657804+0000","last_active":"2023-11-15T02:58:57.661355+0000","last_peered":"2023-11-15T02:58:57.661355+0000","last_clean":"2023-11-15T02:58:57.661355+0000","last_became_active":"2023-11-15T02:58:57.657411+0000","last_became_peered":"2023-11-15T02:58:57.657411+0000","last_unstale":"2023-11-15T02:58:57.661355+0000","last_undegraded":"2023-11-15T02:58:57.661355+0000","last_fullsized":"2023-11-15T02:58:57.661355+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"47'2","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.648975+0000","last_change":"2023-11-15T02:58:57.190329+0000","last_active":"2023-11-15T02:58:57.648975+0000","last_peered":"2023-11-15T02:58:57.648975+0000","last_clean":"2023-11-15T02:58:57.648975+0000","last_became_active":"2023-11-15T02:58:57.190059+0000","last_became_peered":"2023-11-15T02:58:57.190059+0000","last_unstale":"2023-11-15T02:58:57.648975+0000","last_undegraded":"2023-11-15T02:58:57.648975+0000","last_fullsized":"2023-11-15T02:58:57.648975+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.647405+0000","last_change":"2023-11-15T02:58:57.641814+0000","last_active":"2023-11-15T02:58:57.647405+0000","last_peered":"2023-11-15T02:58:57.647405+0000","last_clean":"2023-11-15T02:58:57.647405+0000","last_became_active":"2023-11-15T02:58:57.641546+0000","last_became_peered":"2023-11-15T02:58:57.641546+0000","last_unstale":"2023-11-15T02:58:57.647405+0000","last_undegraded":"2023-11-15T02:58:57.647405+0000","last_fullsized":"2023-11-15T02:58:57.647405+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.625423+0000","last_change":"2023-11-15T02:58:57.204873+0000","last_active":"2023-11-15T02:58:57.625423+0000","last_peered":"2023-11-15T02:58:57.625423+0000","last_clean":"2023-11-15T02:58:57.625423+0000","last_became_active":"2023-11-15T02:58:57.204576+0000","last_became_peered":"2023-11-15T02:58:57.204576+0000","last_unstale":"2023-11-15T02:58:57.625423+0000","last_undegraded":"2023-11-15T02:58:57.625423+0000","last_fullsized":"2023-11-15T02:58:57.625423+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.652272+0000","last_change":"2023-11-15T02:58:57.650781+0000","last_active":"2023-11-15T02:58:57.652272+0000","last_peered":"2023-11-15T02:58:57.652272+0000","last_clean":"2023-11-15T02:58:57.652272+0000","last_became_active":"2023-11-15T02:58:57.650461+0000","last_became_peered":"2023-11-15T02:58:57.650461+0000","last_unstale":"2023-11-15T02:58:57.652272+0000","last_undegraded":"2023-11-15T02:58:57.652272+0000","last_fullsized":"2023-11-15T02:58:57.652272+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.638734+0000","last_change":"2023-11-15T02:58:57.183559+0000","last_active":"2023-11-15T02:58:57.638734+0000","last_peered":"2023-11-15T02:58:57.638734+0000","last_clean":"2023-11-15T02:58:57.638734+0000","last_became_active":"2023-11-15T02:58:57.183270+0000","last_became_peered":"2023-11-15T02:58:57.183270+0000","last_unstale":"2023-11-15T02:58:57.638734+0000","last_undegraded":"2023-11-15T02:58:57.638734+0000","last_fullsized":"2023-11-15T02:58:57.638734+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.661228+0000","last_change":"2023-11-15T02:58:57.655419+0000","last_active":"2023-11-15T02:58:57.661228+0000","last_peered":"2023-11-15T02:58:57.661228+0000","last_clean":"2023-11-15T02:58:57.661228+0000","last_became_active":"2023-11-15T02:58:57.655104+0000","last_became_peered":"2023-11-15T02:58:57.655104+0000","last_unstale":"2023-11-15T02:58:57.661228+0000","last_undegraded":"2023-11-15T02:58:57.661228+0000","last_fullsized":"2023-11-15T02:58:57.661228+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.625784+0000","last_change":"2023-11-15T02:58:57.196840+0000","last_active":"2023-11-15T02:58:57.625784+0000","last_peered":"2023-11-15T02:58:57.625784+0000","last_clean":"2023-11-15T02:58:57.625784+0000","last_became_active":"2023-11-15T02:58:57.196549+0000","last_became_peered":"2023-11-15T02:58:57.196549+0000","last_unstale":"2023-11-15T02:58:57.625784+0000","last_undegraded":"2023-11-15T02:58:57.625784+0000","last_fullsized":"2023-11-15T02:58:57.625784+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.648715+0000","last_change":"2023-11-15T02:58:57.640783+0000","last_active":"2023-11-15T02:58:57.648715+0000","last_peered":"2023-11-15T02:58:57.648715+0000","last_clean":"2023-11-15T02:58:57.648715+0000","last_became_active":"2023-11-15T02:58:57.640505+0000","last_became_peered":"2023-11-15T02:58:57.640505+0000","last_unstale":"2023-11-15T02:58:57.648715+0000","last_undegraded":"2023-11-15T02:58:57.648715+0000","last_fullsized":"2023-11-15T02:58:57.648715+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.638444+0000","last_change":"2023-11-15T02:58:57.187185+0000","last_active":"2023-11-15T02:58:57.638444+0000","last_peered":"2023-11-15T02:58:57.638444+0000","last_clean":"2023-11-15T02:58:57.638444+0000","last_became_active":"2023-11-15T02:58:57.186840+0000","last_became_peered":"2023-11-15T02:58:57.186840+0000","last_unstale":"2023-11-15T02:58:57.638444+0000","last_undegraded":"2023-11-15T02:58:57.638444+0000","last_fullsized":"2023-11-15T02:58:57.638444+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.662338+0000","last_change":"2023-11-15T02:58:57.661411+0000","last_active":"2023-11-15T02:58:57.662338+0000","last_peered":"2023-11-15T02:58:57.662338+0000","last_clean":"2023-11-15T02:58:57.662338+0000","last_became_active":"2023-11-15T02:58:57.661066+0000","last_became_peered":"2023-11-15T02:58:57.661066+0000","last_unstale":"2023-11-15T02:58:57.662338+0000","last_undegraded":"2023-11-15T02:58:57.662338+0000","last_fullsized":"2023-11-15T02:58:57.662338+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.636706+0000","last_change":"2023-11-15T02:58:57.187827+0000","last_active":"2023-11-15T02:58:57.636706+0000","last_peered":"2023-11-15T02:58:57.636706+0000","last_clean":"2023-11-15T02:58:57.636706+0000","last_became_active":"2023-11-15T02:58:57.187143+0000","last_became_peered":"2023-11-15T02:58:57.187143+0000","last_unstale":"2023-11-15T02:58:57.636706+0000","last_undegraded":"2023-11-15T02:58:57.636706+0000","last_fullsized":"2023-11-15T02:58:57.636706+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.646995+0000","last_change":"2023-11-15T02:58:57.636451+0000","last_active":"2023-11-15T02:58:57.646995+0000","last_peered":"2023-11-15T02:58:57.646995+0000","last_clean":"2023-11-15T02:58:57.646995+0000","last_became_active":"2023-11-15T02:58:57.635723+0000","last_became_peered":"2023-11-15T02:58:57.635723+0000","last_unstale":"2023-11-15T02:58:57.646995+0000","last_undegraded":"2023-11-15T02:58:57.646995+0000","last_fullsized":"2023-11-15T02:58:57.646995+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.639903+0000","last_change":"2023-11-15T02:58:57.185933+0000","last_active":"2023-11-15T02:58:57.639903+0000","last_peered":"2023-11-15T02:58:57.639903+0000","last_clean":"2023-11-15T02:58:57.639903+0000","last_became_active":"2023-11-15T02:58:57.185590+0000","last_became_peered":"2023-11-15T02:58:57.185590+0000","last_unstale":"2023-11-15T02:58:57.639903+0000","last_undegraded":"2023-11-15T02:58:57.639903+0000","last_fullsized":"2023-11-15T02:58:57.639903+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.635033+0000","last_change":"2023-11-15T02:58:57.630393+0000","last_active":"2023-11-15T02:58:57.635033+0000","last_peered":"2023-11-15T02:58:57.635033+0000","last_clean":"2023-11-15T02:58:57.635033+0000","last_became_active":"2023-11-15T02:58:57.629556+0000","last_became_peered":"2023-11-15T02:58:57.629556+0000","last_unstale":"2023-11-15T02:58:57.635033+0000","last_undegraded":"2023-11-15T02:58:57.635033+0000","last_fullsized":"2023-11-15T02:58:57.635033+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.626067+0000","last_change":"2023-11-15T02:58:57.199863+0000","last_active":"2023-11-15T02:58:57.626067+0000","last_peered":"2023-11-15T02:58:57.626067+0000","last_clean":"2023-11-15T02:58:57.626067+0000","last_became_active":"2023-11-15T02:58:57.199557+0000","last_became_peered":"2023-11-15T02:58:57.199557+0000","last_unstale":"2023-11-15T02:58:57.626067+0000","last_undegraded":"2023-11-15T02:58:57.626067+0000","last_fullsized":"2023-11-15T02:58:57.626067+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.659371+0000","last_change":"2023-11-15T02:58:57.644455+0000","last_active":"2023-11-15T02:58:57.659371+0000","last_peered":"2023-11-15T02:58:57.659371+0000","last_clean":"2023-11-15T02:58:57.659371+0000","last_became_active":"2023-11-15T02:58:57.644115+0000","last_became_peered":"2023-11-15T02:58:57.644115+0000","last_unstale":"2023-11-15T02:58:57.659371+0000","last_undegraded":"2023-11-15T02:58:57.659371+0000","last_fullsized":"2023-11-15T02:58:57.659371+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"47'3","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.660244+0000","last_change":"2023-11-15T02:58:57.191943+0000","last_active":"2023-11-15T02:58:57.660244+0000","last_peered":"2023-11-15T02:58:57.660244+0000","last_clean":"2023-11-15T02:58:57.660244+0000","last_became_active":"2023-11-15T02:58:57.191643+0000","last_became_peered":"2023-11-15T02:58:57.191643+0000","last_unstale":"2023-11-15T02:58:57.660244+0000","last_undegraded":"2023-11-15T02:58:57.660244+0000","last_fullsized":"2023-11-15T02:58:57.660244+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.647594+0000","last_change":"2023-11-15T02:58:57.639414+0000","last_active":"2023-11-15T02:58:57.647594+0000","last_peered":"2023-11-15T02:58:57.647594+0000","last_clean":"2023-11-15T02:58:57.647594+0000","last_became_active":"2023-11-15T02:58:57.639107+0000","last_became_peered":"2023-11-15T02:58:57.639107+0000","last_unstale":"2023-11-15T02:58:57.647594+0000","last_undegraded":"2023-11-15T02:58:57.647594+0000","last_fullsized":"2023-11-15T02:58:57.647594+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.658501+0000","last_change":"2023-11-15T02:58:57.198795+0000","last_active":"2023-11-15T02:58:57.658501+0000","last_peered":"2023-11-15T02:58:57.658501+0000","last_clean":"2023-11-15T02:58:57.658501+0000","last_became_active":"2023-11-15T02:58:57.198523+0000","last_became_peered":"2023-11-15T02:58:57.198523+0000","last_unstale":"2023-11-15T02:58:57.658501+0000","last_undegraded":"2023-11-15T02:58:57.658501+0000","last_fullsized":"2023-11-15T02:58:57.658501+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.651446+0000","last_change":"2023-11-15T02:58:57.647677+0000","last_active":"2023-11-15T02:58:57.651446+0000","last_peered":"2023-11-15T02:58:57.651446+0000","last_clean":"2023-11-15T02:58:57.651446+0000","last_became_active":"2023-11-15T02:58:57.647396+0000","last_became_peered":"2023-11-15T02:58:57.647396+0000","last_unstale":"2023-11-15T02:58:57.651446+0000","last_undegraded":"2023-11-15T02:58:57.651446+0000","last_fullsized":"2023-11-15T02:58:57.651446+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"47'1","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.657752+0000","last_change":"2023-11-15T02:58:57.192885+0000","last_active":"2023-11-15T02:58:57.657752+0000","last_peered":"2023-11-15T02:58:57.657752+0000","last_clean":"2023-11-15T02:58:57.657752+0000","last_became_active":"2023-11-15T02:58:57.192623+0000","last_became_peered":"2023-11-15T02:58:57.192623+0000","last_unstale":"2023-11-15T02:58:57.657752+0000","last_undegraded":"2023-11-15T02:58:57.657752+0000","last_fullsized":"2023-11-15T02:58:57.657752+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.660654+0000","last_change":"2023-11-15T02:58:57.651772+0000","last_active":"2023-11-15T02:58:57.660654+0000","last_peered":"2023-11-15T02:58:57.660654+0000","last_clean":"2023-11-15T02:58:57.660654+0000","last_became_active":"2023-11-15T02:58:57.650851+0000","last_became_peered":"2023-11-15T02:58:57.650851+0000","last_unstale":"2023-11-15T02:58:57.660654+0000","last_undegraded":"2023-11-15T02:58:57.660654+0000","last_fullsized":"2023-11-15T02:58:57.660654+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.632009+0000","last_change":"2023-11-15T02:58:57.200538+0000","last_active":"2023-11-15T02:58:57.632009+0000","last_peered":"2023-11-15T02:58:57.632009+0000","last_clean":"2023-11-15T02:58:57.632009+0000","last_became_active":"2023-11-15T02:58:57.199264+0000","last_became_peered":"2023-11-15T02:58:57.199264+0000","last_unstale":"2023-11-15T02:58:57.632009+0000","last_undegraded":"2023-11-15T02:58:57.632009+0000","last_fullsized":"2023-11-15T02:58:57.632009+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.652899+0000","last_change":"2023-11-15T02:58:57.649191+0000","last_active":"2023-11-15T02:58:57.652899+0000","last_peered":"2023-11-15T02:58:57.652899+0000","last_clean":"2023-11-15T02:58:57.652899+0000","last_became_active":"2023-11-15T02:58:57.648957+0000","last_became_peered":"2023-11-15T02:58:57.648957+0000","last_unstale":"2023-11-15T02:58:57.652899+0000","last_undegraded":"2023-11-15T02:58:57.652899+0000","last_fullsized":"2023-11-15T02:58:57.652899+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.635124+0000","last_change":"2023-11-15T02:58:57.187356+0000","last_active":"2023-11-15T02:58:57.635124+0000","last_peered":"2023-11-15T02:58:57.635124+0000","last_clean":"2023-11-15T02:58:57.635124+0000","last_became_active":"2023-11-15T02:58:57.187031+0000","last_became_peered":"2023-11-15T02:58:57.187031+0000","last_unstale":"2023-11-15T02:58:57.635124+0000","last_undegraded":"2023-11-15T02:58:57.635124+0000","last_fullsized":"2023-11-15T02:58:57.635124+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.660487+0000","last_change":"2023-11-15T02:58:57.651239+0000","last_active":"2023-11-15T02:58:57.660487+0000","last_peered":"2023-11-15T02:58:57.660487+0000","last_clean":"2023-11-15T02:58:57.660487+0000","last_became_active":"2023-11-15T02:58:57.650989+0000","last_became_peered":"2023-11-15T02:58:57.650989+0000","last_unstale":"2023-11-15T02:58:57.660487+0000","last_undegraded":"2023-11-15T02:58:57.660487+0000","last_fullsized":"2023-11-15T02:58:57.660487+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"47'4","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.652959+0000","last_change":"2023-11-15T02:58:57.202348+0000","last_active":"2023-11-15T02:58:57.652959+0000","last_peered":"2023-11-15T02:58:57.652959+0000","last_clean":"2023-11-15T02:58:57.652959+0000","last_became_active":"2023-11-15T02:58:57.201416+0000","last_became_peered":"2023-11-15T02:58:57.201416+0000","last_unstale":"2023-11-15T02:58:57.652959+0000","last_undegraded":"2023-11-15T02:58:57.652959+0000","last_fullsized":"2023-11-15T02:58:57.652959+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.639358+0000","last_change":"2023-11-15T02:58:57.632880+0000","last_active":"2023-11-15T02:58:57.639358+0000","last_peered":"2023-11-15T02:58:57.639358+0000","last_clean":"2023-11-15T02:58:57.639358+0000","last_became_active":"2023-11-15T02:58:57.631665+0000","last_became_peered":"2023-11-15T02:58:57.631665+0000","last_unstale":"2023-11-15T02:58:57.639358+0000","last_undegraded":"2023-11-15T02:58:57.639358+0000","last_fullsized":"2023-11-15T02:58:57.639358+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"47'1","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.662284+0000","last_change":"2023-11-15T02:58:57.203704+0000","last_active":"2023-11-15T02:58:57.662284+0000","last_peered":"2023-11-15T02:58:57.662284+0000","last_clean":"2023-11-15T02:58:57.662284+0000","last_became_active":"2023-11-15T02:58:57.203346+0000","last_became_peered":"2023-11-15T02:58:57.203346+0000","last_unstale":"2023-11-15T02:58:57.662284+0000","last_undegraded":"2023-11-15T02:58:57.662284+0000","last_fullsized":"2023-11-15T02:58:57.662284+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.642369+0000","last_change":"2023-11-15T02:58:57.642369+0000","last_active":"2023-11-15T02:58:57.642369+0000","last_peered":"2023-11-15T02:58:57.642369+0000","last_clean":"2023-11-15T02:58:57.642369+0000","last_became_active":"2023-11-15T02:58:57.642090+0000","last_became_peered":"2023-11-15T02:58:57.642090+0000","last_unstale":"2023-11-15T02:58:57.642369+0000","last_undegraded":"2023-11-15T02:58:57.642369+0000","last_fullsized":"2023-11-15T02:58:57.642369+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.641792+0000","last_change":"2023-11-15T02:58:57.180459+0000","last_active":"2023-11-15T02:58:57.641792+0000","last_peered":"2023-11-15T02:58:57.641792+0000","last_clean":"2023-11-15T02:58:57.641792+0000","last_became_active":"2023-11-15T02:58:57.178814+0000","last_became_peered":"2023-11-15T02:58:57.178814+0000","last_unstale":"2023-11-15T02:58:57.641792+0000","last_undegraded":"2023-11-15T02:58:57.641792+0000","last_fullsized":"2023-11-15T02:58:57.641792+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.647761+0000","last_change":"2023-11-15T02:58:57.640944+0000","last_active":"2023-11-15T02:58:57.647761+0000","last_peered":"2023-11-15T02:58:57.647761+0000","last_clean":"2023-11-15T02:58:57.647761+0000","last_became_active":"2023-11-15T02:58:57.640625+0000","last_became_peered":"2023-11-15T02:58:57.640625+0000","last_unstale":"2023-11-15T02:58:57.647761+0000","last_undegraded":"2023-11-15T02:58:57.647761+0000","last_fullsized":"2023-11-15T02:58:57.647761+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.638551+0000","last_change":"2023-11-15T02:58:57.187987+0000","last_active":"2023-11-15T02:58:57.638551+0000","last_peered":"2023-11-15T02:58:57.638551+0000","last_clean":"2023-11-15T02:58:57.638551+0000","last_became_active":"2023-11-15T02:58:57.187718+0000","last_became_peered":"2023-11-15T02:58:57.187718+0000","last_unstale":"2023-11-15T02:58:57.638551+0000","last_undegraded":"2023-11-15T02:58:57.638551+0000","last_fullsized":"2023-11-15T02:58:57.638551+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.642380+0000","last_change":"2023-11-15T02:58:57.642380+0000","last_active":"2023-11-15T02:58:57.642380+0000","last_peered":"2023-11-15T02:58:57.642380+0000","last_clean":"2023-11-15T02:58:57.642380+0000","last_became_active":"2023-11-15T02:58:57.640187+0000","last_became_peered":"2023-11-15T02:58:57.640187+0000","last_unstale":"2023-11-15T02:58:57.642380+0000","last_undegraded":"2023-11-15T02:58:57.642380+0000","last_fullsized":"2023-11-15T02:58:57.642380+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.634956+0000","last_change":"2023-11-15T02:58:57.204652+0000","last_active":"2023-11-15T02:58:57.634956+0000","last_peered":"2023-11-15T02:58:57.634956+0000","last_clean":"2023-11-15T02:58:57.634956+0000","last_became_active":"2023-11-15T02:58:57.204345+0000","last_became_peered":"2023-11-15T02:58:57.204345+0000","last_unstale":"2023-11-15T02:58:57.634956+0000","last_undegraded":"2023-11-15T02:58:57.634956+0000","last_fullsized":"2023-11-15T02:58:57.634956+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.635066+0000","last_change":"2023-11-15T02:58:57.630783+0000","last_active":"2023-11-15T02:58:57.635066+0000","last_peered":"2023-11-15T02:58:57.635066+0000","last_clean":"2023-11-15T02:58:57.635066+0000","last_became_active":"2023-11-15T02:58:57.630491+0000","last_became_peered":"2023-11-15T02:58:57.630491+0000","last_unstale":"2023-11-15T02:58:57.635066+0000","last_undegraded":"2023-11-15T02:58:57.635066+0000","last_fullsized":"2023-11-15T02:58:57.635066+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"47'1","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.627547+0000","last_change":"2023-11-15T02:58:57.181792+0000","last_active":"2023-11-15T02:58:57.627547+0000","last_peered":"2023-11-15T02:58:57.627547+0000","last_clean":"2023-11-15T02:58:57.627547+0000","last_became_active":"2023-11-15T02:58:57.181505+0000","last_became_peered":"2023-11-15T02:58:57.181505+0000","last_unstale":"2023-11-15T02:58:57.627547+0000","last_undegraded":"2023-11-15T02:58:57.627547+0000","last_fullsized":"2023-11-15T02:58:57.627547+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.654300+0000","last_change":"2023-11-15T02:58:57.651063+0000","last_active":"2023-11-15T02:58:57.654300+0000","last_peered":"2023-11-15T02:58:57.654300+0000","last_clean":"2023-11-15T02:58:57.654300+0000","last_became_active":"2023-11-15T02:58:57.650309+0000","last_became_peered":"2023-11-15T02:58:57.650309+0000","last_unstale":"2023-11-15T02:58:57.654300+0000","last_undegraded":"2023-11-15T02:58:57.654300+0000","last_fullsized":"2023-11-15T02:58:57.654300+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.619641+0000","last_change":"2023-11-15T02:58:57.202787+0000","last_active":"2023-11-15T02:58:57.619641+0000","last_peered":"2023-11-15T02:58:57.619641+0000","last_clean":"2023-11-15T02:58:57.619641+0000","last_became_active":"2023-11-15T02:58:57.202442+0000","last_became_peered":"2023-11-15T02:58:57.202442+0000","last_unstale":"2023-11-15T02:58:57.619641+0000","last_undegraded":"2023-11-15T02:58:57.619641+0000","last_fullsized":"2023-11-15T02:58:57.619641+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.660221+0000","last_change":"2023-11-15T02:58:57.645124+0000","last_active":"2023-11-15T02:58:57.660221+0000","last_peered":"2023-11-15T02:58:57.660221+0000","last_clean":"2023-11-15T02:58:57.660221+0000","last_became_active":"2023-11-15T02:58:57.644861+0000","last_became_peered":"2023-11-15T02:58:57.644861+0000","last_unstale":"2023-11-15T02:58:57.660221+0000","last_undegraded":"2023-11-15T02:58:57.660221+0000","last_fullsized":"2023-11-15T02:58:57.660221+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.637541+0000","last_change":"2023-11-15T02:58:57.191482+0000","last_active":"2023-11-15T02:58:57.637541+0000","last_peered":"2023-11-15T02:58:57.637541+0000","last_clean":"2023-11-15T02:58:57.637541+0000","last_became_active":"2023-11-15T02:58:57.190137+0000","last_became_peered":"2023-11-15T02:58:57.190137+0000","last_unstale":"2023-11-15T02:58:57.637541+0000","last_undegraded":"2023-11-15T02:58:57.637541+0000","last_fullsized":"2023-11-15T02:58:57.637541+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.641367+0000","last_change":"2023-11-15T02:58:57.641367+0000","last_active":"2023-11-15T02:58:57.641367+0000","last_peered":"2023-11-15T02:58:57.641367+0000","last_clean":"2023-11-15T02:58:57.641367+0000","last_became_active":"2023-11-15T02:58:57.641112+0000","last_became_peered":"2023-11-15T02:58:57.641112+0000","last_unstale":"2023-11-15T02:58:57.641367+0000","last_undegraded":"2023-11-15T02:58:57.641367+0000","last_fullsized":"2023-11-15T02:58:57.641367+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.627049+0000","last_change":"2023-11-15T02:58:57.179251+0000","last_active":"2023-11-15T02:58:57.627049+0000","last_peered":"2023-11-15T02:58:57.627049+0000","last_clean":"2023-11-15T02:58:57.627049+0000","last_became_active":"2023-11-15T02:58:57.178899+0000","last_became_peered":"2023-11-15T02:58:57.178899+0000","last_unstale":"2023-11-15T02:58:57.627049+0000","last_undegraded":"2023-11-15T02:58:57.627049+0000","last_fullsized":"2023-11-15T02:58:57.627049+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.656078+0000","last_change":"2023-11-15T02:58:57.652641+0000","last_active":"2023-11-15T02:58:57.656078+0000","last_peered":"2023-11-15T02:58:57.656078+0000","last_clean":"2023-11-15T02:58:57.656078+0000","last_became_active":"2023-11-15T02:58:57.652308+0000","last_became_peered":"2023-11-15T02:58:57.652308+0000","last_unstale":"2023-11-15T02:58:57.656078+0000","last_undegraded":"2023-11-15T02:58:57.656078+0000","last_fullsized":"2023-11-15T02:58:57.656078+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.619774+0000","last_change":"2023-11-15T02:58:57.194048+0000","last_active":"2023-11-15T02:58:57.619774+0000","last_peered":"2023-11-15T02:58:57.619774+0000","last_clean":"2023-11-15T02:58:57.619774+0000","last_became_active":"2023-11-15T02:58:57.193743+0000","last_became_peered":"2023-11-15T02:58:57.193743+0000","last_unstale":"2023-11-15T02:58:57.619774+0000","last_undegraded":"2023-11-15T02:58:57.619774+0000","last_fullsized":"2023-11-15T02:58:57.619774+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":34,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.639744+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T02:58:57.639744+0000","last_peered":"2023-11-15T02:58:57.639744+0000","last_clean":"2023-11-15T02:58:57.639744+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T02:58:57.639744+0000","last_undegraded":"2023-11-15T02:58:57.639744+0000","last_fullsized":"2023-11-15T02:58:57.639744+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.660721+0000","last_change":"2023-11-15T02:58:57.654213+0000","last_active":"2023-11-15T02:58:57.660721+0000","last_peered":"2023-11-15T02:58:57.660721+0000","last_clean":"2023-11-15T02:58:57.660721+0000","last_became_active":"2023-11-15T02:58:57.653691+0000","last_became_peered":"2023-11-15T02:58:57.653691+0000","last_unstale":"2023-11-15T02:58:57.660721+0000","last_undegraded":"2023-11-15T02:58:57.660721+0000","last_fullsized":"2023-11-15T02:58:57.660721+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.620849+0000","last_change":"2023-11-15T02:58:57.189737+0000","last_active":"2023-11-15T02:58:57.620849+0000","last_peered":"2023-11-15T02:58:57.620849+0000","last_clean":"2023-11-15T02:58:57.620849+0000","last_became_active":"2023-11-15T02:58:57.189365+0000","last_became_peered":"2023-11-15T02:58:57.189365+0000","last_unstale":"2023-11-15T02:58:57.620849+0000","last_undegraded":"2023-11-15T02:58:57.620849+0000","last_fullsized":"2023-11-15T02:58:57.620849+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.165470+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T02:58:57.165470+0000","last_peered":"2023-11-15T02:58:57.165470+0000","last_clean":"2023-11-15T02:58:57.165470+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T02:58:57.165470+0000","last_undegraded":"2023-11-15T02:58:57.165470+0000","last_fullsized":"2023-11-15T02:58:57.165470+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.638934+0000","last_change":"2023-11-15T02:58:57.632652+0000","last_active":"2023-11-15T02:58:57.638934+0000","last_peered":"2023-11-15T02:58:57.638934+0000","last_clean":"2023-11-15T02:58:57.638934+0000","last_became_active":"2023-11-15T02:58:57.632350+0000","last_became_peered":"2023-11-15T02:58:57.632350+0000","last_unstale":"2023-11-15T02:58:57.638934+0000","last_undegraded":"2023-11-15T02:58:57.638934+0000","last_fullsized":"2023-11-15T02:58:57.638934+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.623049+0000","last_change":"2023-11-15T02:58:57.188147+0000","last_active":"2023-11-15T02:58:57.623049+0000","last_peered":"2023-11-15T02:58:57.623049+0000","last_clean":"2023-11-15T02:58:57.623049+0000","last_became_active":"2023-11-15T02:58:57.187878+0000","last_became_peered":"2023-11-15T02:58:57.187878+0000","last_unstale":"2023-11-15T02:58:57.623049+0000","last_undegraded":"2023-11-15T02:58:57.623049+0000","last_fullsized":"2023-11-15T02:58:57.623049+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":34,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.642347+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T02:58:57.642347+0000","last_peered":"2023-11-15T02:58:57.642347+0000","last_clean":"2023-11-15T02:58:57.642347+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T02:58:57.642347+0000","last_undegraded":"2023-11-15T02:58:57.642347+0000","last_fullsized":"2023-11-15T02:58:57.642347+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.660257+0000","last_change":"2023-11-15T02:58:57.648180+0000","last_active":"2023-11-15T02:58:57.660257+0000","last_peered":"2023-11-15T02:58:57.660257+0000","last_clean":"2023-11-15T02:58:57.660257+0000","last_became_active":"2023-11-15T02:58:57.647910+0000","last_became_peered":"2023-11-15T02:58:57.647910+0000","last_unstale":"2023-11-15T02:58:57.660257+0000","last_undegraded":"2023-11-15T02:58:57.660257+0000","last_fullsized":"2023-11-15T02:58:57.660257+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.625716+0000","last_change":"2023-11-15T02:58:57.205048+0000","last_active":"2023-11-15T02:58:57.625716+0000","last_peered":"2023-11-15T02:58:57.625716+0000","last_clean":"2023-11-15T02:58:57.625716+0000","last_became_active":"2023-11-15T02:58:57.204006+0000","last_became_peered":"2023-11-15T02:58:57.204006+0000","last_unstale":"2023-11-15T02:58:57.625716+0000","last_undegraded":"2023-11-15T02:58:57.625716+0000","last_fullsized":"2023-11-15T02:58:57.625716+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.164926+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T02:58:57.164926+0000","last_peered":"2023-11-15T02:58:57.164926+0000","last_clean":"2023-11-15T02:58:57.164926+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T02:58:57.164926+0000","last_undegraded":"2023-11-15T02:58:57.164926+0000","last_fullsized":"2023-11-15T02:58:57.164926+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.637346+0000","last_change":"2023-11-15T02:58:57.635240+0000","last_active":"2023-11-15T02:58:57.637346+0000","last_peered":"2023-11-15T02:58:57.637346+0000","last_clean":"2023-11-15T02:58:57.637346+0000","last_became_active":"2023-11-15T02:58:57.634833+0000","last_became_peered":"2023-11-15T02:58:57.634833+0000","last_unstale":"2023-11-15T02:58:57.637346+0000","last_undegraded":"2023-11-15T02:58:57.637346+0000","last_fullsized":"2023-11-15T02:58:57.637346+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.658562+0000","last_change":"2023-11-15T02:58:57.195865+0000","last_active":"2023-11-15T02:58:57.658562+0000","last_peered":"2023-11-15T02:58:57.658562+0000","last_clean":"2023-11-15T02:58:57.658562+0000","last_became_active":"2023-11-15T02:58:57.195335+0000","last_became_peered":"2023-11-15T02:58:57.195335+0000","last_unstale":"2023-11-15T02:58:57.658562+0000","last_undegraded":"2023-11-15T02:58:57.658562+0000","last_fullsized":"2023-11-15T02:58:57.658562+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.621848+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T02:58:57.621848+0000","last_peered":"2023-11-15T02:58:57.621848+0000","last_clean":"2023-11-15T02:58:57.621848+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T02:58:57.621848+0000","last_undegraded":"2023-11-15T02:58:57.621848+0000","last_fullsized":"2023-11-15T02:58:57.621848+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.633043+0000","last_change":"2023-11-15T02:58:57.629787+0000","last_active":"2023-11-15T02:58:57.633043+0000","last_peered":"2023-11-15T02:58:57.633043+0000","last_clean":"2023-11-15T02:58:57.633043+0000","last_became_active":"2023-11-15T02:58:57.629506+0000","last_became_peered":"2023-11-15T02:58:57.629506+0000","last_unstale":"2023-11-15T02:58:57.633043+0000","last_undegraded":"2023-11-15T02:58:57.633043+0000","last_fullsized":"2023-11-15T02:58:57.633043+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.637660+0000","last_change":"2023-11-15T02:58:57.186609+0000","last_active":"2023-11-15T02:58:57.637660+0000","last_peered":"2023-11-15T02:58:57.637660+0000","last_clean":"2023-11-15T02:58:57.637660+0000","last_became_active":"2023-11-15T02:58:57.186300+0000","last_became_peered":"2023-11-15T02:58:57.186300+0000","last_unstale":"2023-11-15T02:58:57.637660+0000","last_undegraded":"2023-11-15T02:58:57.637660+0000","last_fullsized":"2023-11-15T02:58:57.637660+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":34,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.627243+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T02:58:57.627243+0000","last_peered":"2023-11-15T02:58:57.627243+0000","last_clean":"2023-11-15T02:58:57.627243+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T02:58:57.627243+0000","last_undegraded":"2023-11-15T02:58:57.627243+0000","last_fullsized":"2023-11-15T02:58:57.627243+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":8,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.646610+0000","last_change":"2023-11-15T02:58:57.646610+0000","last_active":"2023-11-15T02:58:57.646610+0000","last_peered":"2023-11-15T02:58:57.646610+0000","last_clean":"2023-11-15T02:58:57.646610+0000","last_became_active":"2023-11-15T02:58:57.645676+0000","last_became_peered":"2023-11-15T02:58:57.645676+0000","last_unstale":"2023-11-15T02:58:57.646610+0000","last_undegraded":"2023-11-15T02:58:57.646610+0000","last_fullsized":"2023-11-15T02:58:57.646610+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.642269+0000","last_change":"2023-11-15T02:58:57.192517+0000","last_active":"2023-11-15T02:58:57.642269+0000","last_peered":"2023-11-15T02:58:57.642269+0000","last_clean":"2023-11-15T02:58:57.642269+0000","last_became_active":"2023-11-15T02:58:57.192244+0000","last_became_peered":"2023-11-15T02:58:57.192244+0000","last_unstale":"2023-11-15T02:58:57.642269+0000","last_undegraded":"2023-11-15T02:58:57.642269+0000","last_fullsized":"2023-11-15T02:58:57.642269+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.632453+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T02:58:57.632453+0000","last_peered":"2023-11-15T02:58:57.632453+0000","last_clean":"2023-11-15T02:58:57.632453+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T02:58:57.632453+0000","last_undegraded":"2023-11-15T02:58:57.632453+0000","last_fullsized":"2023-11-15T02:58:57.632453+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.654267+0000","last_change":"2023-11-15T02:58:57.649584+0000","last_active":"2023-11-15T02:58:57.654267+0000","last_peered":"2023-11-15T02:58:57.654267+0000","last_clean":"2023-11-15T02:58:57.654267+0000","last_became_active":"2023-11-15T02:58:57.649267+0000","last_became_peered":"2023-11-15T02:58:57.649267+0000","last_unstale":"2023-11-15T02:58:57.654267+0000","last_undegraded":"2023-11-15T02:58:57.654267+0000","last_fullsized":"2023-11-15T02:58:57.654267+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:57.144906+0000","last_clean_scrub_stamp":"2023-11-15T02:58:57.144906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.640397+0000","last_change":"2023-11-15T02:58:57.192510+0000","last_active":"2023-11-15T02:58:57.640397+0000","last_peered":"2023-11-15T02:58:57.640397+0000","last_clean":"2023-11-15T02:58:57.640397+0000","last_became_active":"2023-11-15T02:58:57.192204+0000","last_became_peered":"2023-11-15T02:58:57.192204+0000","last_unstale":"2023-11-15T02:58:57.640397+0000","last_undegraded":"2023-11-15T02:58:57.640397+0000","last_fullsized":"2023-11-15T02:58:57.640397+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:58:56.131661+0000","last_clean_scrub_stamp":"2023-11-15T02:58:56.131661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.623060+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T02:58:57.623060+0000","last_peered":"2023-11-15T02:58:57.623060+0000","last_clean":"2023-11-15T02:58:57.623060+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T02:58:57.623060+0000","last_undegraded":"2023-11-15T02:58:57.623060+0000","last_fullsized":"2023-11-15T02:58:57.623060+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":35,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-15T02:58:57.632538+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T02:58:57.632538+0000","last_peered":"2023-11-15T02:58:57.632538+0000","last_clean":"2023-11-15T02:58:57.632538+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T02:58:57.632538+0000","last_undegraded":"2023-11-15T02:58:57.632538+0000","last_fullsized":"2023-11-15T02:58:57.632538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542173,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":561152,"data_stored":239872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.1679999999999999,"15min":1.1859999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.458,"5min":3.952,"15min":3.952},"last":10.961},{"interface":"front","average":{"1min":1.046,"5min":1.1799999999999999,"15min":1.202},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4419999999999999,"5min":2.5,"15min":2.5},"last":3.4910000000000001}]},{"osd":1,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.137,"15min":1.145},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.3740000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":5.4930000000000003},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1739999999999999,"15min":1.1779999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.415,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":6.8239999999999998}]},{"osd":2,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2150000000000001,"15min":1.2330000000000001},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.458,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":5.7000000000000002},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2949999999999999,"15min":1.3069999999999999},"min":{"1min":0.97899999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.5149999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":2.75}]},{"osd":3,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.3400000000000001,"15min":1.3740000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.381,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":12.907},{"interface":"front","average":{"1min":1.175,"5min":1.2669999999999999,"15min":1.282},"min":{"1min":0.90600000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.351,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":9.9909999999999997}]},{"osd":4,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.2689999999999999,"15min":1.3},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.4359999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.448,"15min":1.4990000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.5029999999999999,"5min":4.226,"15min":4.226},"last":0.56000000000000005}]},{"osd":5,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.329,"15min":1.343},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.556,"5min":2.024,"15min":2.024},"last":2.6890000000000001},{"interface":"front","average":{"1min":1.234,"5min":1.4399999999999999,"15min":1.474},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.653,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.98099999999999998}]},{"osd":6,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.5620000000000001,"15min":1.601},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":1.5660000000000001,"5min":3.2789999999999999,"15min":3.2789999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.591,"15min":1.645},"min":{"1min":1.123,"5min":1.123,"15min":1.123},"max":{"1min":1.4399999999999999,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.63600000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542173,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":565248,"data_stored":245968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.125,"15min":1.133},"min":{"1min":0.69799999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.353,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.083,"5min":1.0469999999999999,"15min":1.0409999999999999},"min":{"1min":0.72499999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.484,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.018}]},{"osd":1,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0920000000000001,"15min":1.0840000000000001},"min":{"1min":0.876,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.415,"5min":1.623,"15min":1.623},"last":1.395},{"interface":"front","average":{"1min":1.173,"5min":1.169,"15min":1.169},"min":{"1min":0.80800000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.609,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.74199999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.1870000000000001,"15min":1.1779999999999999},"min":{"1min":0.84399999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.496,"5min":2.621,"15min":2.621},"last":1.46},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.0960000000000001,"15min":1.0840000000000001},"min":{"1min":0.91700000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.425,"5min":1.5,"15min":1.5},"last":1.2390000000000001}]},{"osd":3,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.167,"15min":1.157},"min":{"1min":1.0029999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.492,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.139},{"interface":"front","average":{"1min":1.24,"5min":1.26,"15min":1.2629999999999999},"min":{"1min":0.99299999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.6850000000000001,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.3260000000000001}]},{"osd":4,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.173,"15min":1.1910000000000001},"min":{"1min":0.81399999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.4099999999999999,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.95599999999999996,"5min":0.625,"15min":0.625},"max":{"1min":1.4850000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.2010000000000001}]},{"osd":5,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.417,"15min":1.411},"min":{"1min":1.155,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.772,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.589},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.3109999999999999,"15min":1.319},"min":{"1min":0.877,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6599999999999999,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":1.1619999999999999}]},{"osd":7,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2809999999999999,"15min":1.3009999999999999},"min":{"1min":0.82199999999999995,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.395,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.448,"15min":1.456},"min":{"1min":1.046,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8540000000000001,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":1.5109999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542174,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":565248,"data_stored":246452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.107,"15min":1.1120000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.516,"5min":1.532,"15min":1.532},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0069999999999999,"15min":1.0029999999999999},"min":{"1min":0.80700000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":1.1830000000000001}]},{"osd":1,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2250000000000001,"15min":1.2370000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4770000000000001,"5min":2.617,"15min":2.617},"last":1.145},{"interface":"front","average":{"1min":1.115,"5min":1.1559999999999999,"15min":1.163},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.425,"5min":1.5,"15min":1.5},"last":1.274}]},{"osd":2,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.1519999999999999,"15min":1.141},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.462,"5min":1.909,"15min":1.909},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1599999999999999,"15min":1.1639999999999999},"min":{"1min":0.88800000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.399,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.91700000000000004}]},{"osd":3,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1519999999999999,"15min":1.1599999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.462,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.05},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.294,"15min":1.3},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.5840000000000001,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":1.2450000000000001}]},{"osd":5,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2430000000000001,"15min":1.2529999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5780000000000001,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.2290000000000001,"15min":1.2350000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.5049999999999999,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.5489999999999999}]},{"osd":6,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.333,"15min":1.343},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6619999999999999,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":1.413},{"interface":"front","average":{"1min":1.296,"5min":1.3979999999999999,"15min":1.415},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.7450000000000001,"5min":2.173,"15min":2.173},"last":1.2030000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.369,"15min":1.381},"min":{"1min":1.069,"5min":0.87,"15min":0.87},"max":{"1min":1.5820000000000001,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.4350000000000001,"15min":1.4450000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.587,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":1.335}]}]},{"osd":3,"up_from":14,"seq":60129542174,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":573440,"data_stored":246564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.77400000000000002,"15min":0.75900000000000001},"min":{"1min":0.64900000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":1.121},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.84199999999999997,"15min":0.82899999999999996},"min":{"1min":0.73599999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1779999999999999,"5min":1.26,"15min":1.26},"last":1.256}]},{"osd":1,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.90000000000000002,"15min":0.90100000000000002},"min":{"1min":0.621,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.66300000000000003},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.97999999999999998,"15min":0.96999999999999997},"min":{"1min":0.83299999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2749999999999999,"5min":1.375,"15min":1.375},"last":0.91500000000000004}]},{"osd":2,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97699999999999998,"15min":0.96799999999999997},"min":{"1min":0.76700000000000002,"5min":0.63,"15min":0.63},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0649999999999999,"15min":1.052},"min":{"1min":0.85899999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.995}]},{"osd":4,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1859999999999999,"15min":1.1799999999999999},"min":{"1min":0.97899999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.456,"5min":1.837,"15min":1.837},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.22,"15min":1.226},"min":{"1min":1.0249999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.492,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.21}]},{"osd":5,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.27,"15min":1.2669999999999999},"min":{"1min":0.996,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.6459999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.327},{"interface":"front","average":{"1min":1.345,"5min":1.2789999999999999,"15min":1.268},"min":{"1min":1.01,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":1.458}]},{"osd":6,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3120000000000001,"15min":1.306},"min":{"1min":1.1060000000000001,"5min":1.048,"15min":1.048},"max":{"1min":1.552,"5min":1.794,"15min":1.794},"last":1.369},{"interface":"front","average":{"1min":1.391,"5min":1.3580000000000001,"15min":1.3520000000000001},"min":{"1min":1.129,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":1.647,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.425}]},{"osd":7,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.3919999999999999,"15min":1.3899999999999999},"min":{"1min":1.083,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.365,"15min":1.357},"min":{"1min":1.1819999999999999,"5min":1.085,"15min":1.085},"max":{"1min":1.665,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.298}]}]},{"osd":2,"up_from":14,"seq":60129542174,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297652,"kb_used_data":556,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074188,"statfs":{"total":96636764160,"available":96331968512,"internally_reserved":0,"allocated":569344,"data_stored":246165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76700000000000002,"15min":0.76400000000000001},"min":{"1min":0.61699999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":1.409},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88700000000000001,"15min":0.88200000000000001},"min":{"1min":0.68300000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.70699999999999996}]},{"osd":1,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89600000000000002,"15min":0.89200000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3120000000000001,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.96099999999999997,"15min":0.94899999999999995},"min":{"1min":0.83499999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":1.1180000000000001}]},{"osd":3,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.002,"15min":1},"min":{"1min":0.79200000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.387,"5min":1.472,"15min":1.472},"last":1.24},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1259999999999999,"15min":1.1200000000000001},"min":{"1min":0.93000000000000005,"5min":0.622,"15min":0.622},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.4990000000000001}]},{"osd":4,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.208,"15min":1.21},"min":{"1min":1.0149999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.48,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.238,"15min":1.23},"min":{"1min":1.0760000000000001,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.5429999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.282}]},{"osd":5,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.262,"15min":1.264},"min":{"1min":1.0149999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":1.613,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2989999999999999,"15min":1.2969999999999999},"min":{"1min":1.0700000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.6890000000000001,"5min":1.857,"15min":1.857},"last":1.3340000000000001}]},{"osd":6,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.367,"15min":1.373},"min":{"1min":1.056,"5min":1.056,"15min":1.056},"max":{"1min":1.72,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.202},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.347,"15min":1.353},"min":{"1min":1.046,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":1.573,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.3200000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3939999999999999,"15min":1.391},"min":{"1min":1.1719999999999999,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":1.8069999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.3799999999999999,"15min":1.3919999999999999},"min":{"1min":1.097,"5min":1.097,"15min":1.097},"max":{"1min":1.5529999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.032}]}]},{"osd":0,"up_from":14,"seq":60129542174,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":573440,"data_stored":246024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.84899999999999998,"15min":0.86899999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1990000000000001,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.89700000000000002,"15min":0.89600000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1619999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":1.0509999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.99399999999999999,"15min":1.0129999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.147,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.99199999999999999,"15min":0.98799999999999999},"min":{"1min":0.68000000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.2529999999999999,"5min":1.847,"15min":1.847},"last":0.92000000000000004}]},{"osd":3,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.96499999999999997,"15min":0.96199999999999997},"min":{"1min":0.76600000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.401,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.157,"15min":1.175},"min":{"1min":0.68899999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2210000000000001,"5min":3.54,"15min":3.54},"last":1.083}]},{"osd":4,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2490000000000001,"15min":1.2470000000000001},"min":{"1min":1.0720000000000001,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.6319999999999999,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.175,"15min":1.175},"min":{"1min":1.006,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.4850000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.347}]},{"osd":5,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2170000000000001,"15min":1.218},"min":{"1min":0.98899999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.393,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.3420000000000001,"15min":1.359},"min":{"1min":1.083,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.375,"5min":3.6760000000000002,"15min":3.6760000000000002},"last":1.1579999999999999}]},{"osd":6,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.411,"15min":1.429},"min":{"1min":1.113,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":1.4730000000000001,"5min":3.71,"15min":3.71},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.3340000000000001,"15min":1.343},"min":{"1min":1.05,"5min":1.024,"15min":1.024},"max":{"1min":1.4970000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.2370000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.3640000000000001,"15min":1.379},"min":{"1min":1.018,"5min":1.018,"15min":1.018},"max":{"1min":1.554,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.119},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2649999999999999,"15min":1.2609999999999999},"min":{"1min":1.1040000000000001,"5min":0.876,"15min":0.876},"max":{"1min":1.5449999999999999,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.2070000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":565248,"data_stored":246790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.05,"15min":1.046},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4359999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.216,"15min":1.2370000000000001},"min":{"1min":0.69699999999999995,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.4630000000000001,"5min":3.46,"15min":3.46},"last":0.97699999999999998}]},{"osd":1,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.2310000000000001,"15min":1.2589999999999999},"min":{"1min":0.82699999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.47,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":1.052},{"interface":"front","average":{"1min":1.091,"5min":1.1739999999999999,"15min":1.1879999999999999},"min":{"1min":0.80300000000000005,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.4570000000000001,"5min":3.516,"15min":3.516},"last":1.099}]},{"osd":2,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.2070000000000001,"15min":1.2130000000000001},"min":{"1min":0.91600000000000004,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.3640000000000001,"5min":5.4180000000000001,"15min":5.4180000000000001},"last":1.179},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3069999999999999,"15min":1.321},"min":{"1min":0.94799999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.5089999999999999,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":0.90200000000000002}]},{"osd":3,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.29,"15min":1.3080000000000001},"min":{"1min":1.022,"5min":0.879,"15min":0.879},"max":{"1min":1.4850000000000001,"5min":2.141,"15min":2.141},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.448,"15min":1.464},"min":{"1min":1.1839999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":1.583,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":1.274}]},{"osd":4,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3680000000000001,"15min":1.3879999999999999},"min":{"1min":0.84199999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.5309999999999999,"5min":3.706,"15min":3.706},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.597,"15min":1.6379999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6479999999999999,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":1.1419999999999999}]},{"osd":6,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.327,"15min":1.3420000000000001},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.6439999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":1.347},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.546,"15min":1.5860000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.613,"5min":3.641,"15min":3.641},"last":1.22}]},{"osd":7,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4259999999999999,"15min":1.4370000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.762,"5min":3.516,"15min":3.516},"last":1.321},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.46,"15min":1.4650000000000001},"min":{"1min":1.139,"5min":1.022,"15min":1.022},"max":{"1min":1.722,"5min":3.754,"15min":3.754},"last":1.411}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297620,"kb_used_data":524,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074220,"statfs":{"total":96636764160,"available":96332001280,"internally_reserved":0,"allocated":536576,"data_stored":232015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.84999999999999998,"15min":0.84599999999999997},"min":{"1min":0.63600000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2150000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.72199999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.155,"5min":1.419,"15min":1.419},"last":0.64000000000000001}]},{"osd":2,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.85799999999999998,"15min":0.84799999999999998},"min":{"1min":0.495,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":1.1000000000000001},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.90700000000000003,"15min":0.91000000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.161,"5min":1.341,"15min":1.341},"last":0.96799999999999997}]},{"osd":3,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.0620000000000001,"15min":1.054},"min":{"1min":0.54100000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.98899999999999999,"15min":0.97699999999999998},"min":{"1min":0.67800000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.92700000000000005}]},{"osd":4,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.175,"15min":1.165},"min":{"1min":0.75700000000000001,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.49,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.248,"15min":1.2589999999999999},"min":{"1min":0.92000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.5609999999999999,"5min":2.206,"15min":2.206},"last":1.365}]},{"osd":5,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.26,"15min":1.2649999999999999},"min":{"1min":1.0089999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.4299999999999999,"5min":1.899,"15min":1.899},"last":1.153},{"interface":"front","average":{"1min":1.236,"5min":1.2829999999999999,"15min":1.2909999999999999},"min":{"1min":0.92800000000000005,"5min":0.871,"15min":0.871},"max":{"1min":1.5029999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.242}]},{"osd":6,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.323,"15min":1.325},"min":{"1min":0.98299999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.5229999999999999,"5min":1.825,"15min":1.825},"last":1.27},{"interface":"front","average":{"1min":1.401,"5min":1.4099999999999999,"15min":1.4119999999999999},"min":{"1min":1.1870000000000001,"5min":0.87,"15min":0.87},"max":{"1min":1.5529999999999999,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.5269999999999999}]},{"osd":7,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3200000000000001,"15min":1.3169999999999999},"min":{"1min":0.93999999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.6839999999999999,"5min":1.772,"15min":1.772},"last":1.415},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3540000000000001,"15min":1.3600000000000001},"min":{"1min":0.95499999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.6120000000000001,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.452}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T02:59:02.289 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-15T02:59:02.291 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-15T02:59:02.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.486+0000 7f25d5ef1700 1 Processor -- start 2023-11-15T02:59:02.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.486+0000 7f25d5ef1700 1 -- start start 2023-11-15T02:59:02.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.490+0000 7f25d5ef1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25d01570b0 0x7f25d0066860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.490+0000 7f25d5ef1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f25d0157490 0x7f25d0065950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.490+0000 7f25d5ef1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d0062da0 con 0x7f25d00c5c80 2023-11-15T02:59:02.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.490+0000 7f25d5ef1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d00bfb90 con 0x7f25d0157490 2023-11-15T02:59:02.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.490+0000 7f25d5ef1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00c6140 con 0x7f25d01570b0 2023-11-15T02:59:02.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.490+0000 7f25cf7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25d01570b0 0x7f25d0066860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.490+0000 7f25ceffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f25d0157490 0x7f25d0065950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.490+0000 7f25cffff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f25d00c5c80 0x7f25d01578f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:48488/0 (socket says 172.21.15.121:48488) 2023-11-15T02:59:02.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.490+0000 7f25cffff700 1 -- 172.21.15.121:0/1620921288 learned_addr learned my addr 172.21.15.121:0/1620921288 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:02.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.490+0000 7f25ceffd700 1 -- 172.21.15.121:0/1620921288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25d01570b0 msgr2=0x7f25d0066860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:02.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.490+0000 7f25ceffd700 1 --2- 172.21.15.121:0/1620921288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25d01570b0 0x7f25d0066860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.490+0000 7f25ceffd700 1 -- 172.21.15.121:0/1620921288 >> v1:172.21.15.190:6789/0 conn(0x7f25d00c5c80 legacy=0x7f25d01578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:02.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.490+0000 7f25ceffd700 1 -- 172.21.15.121:0/1620921288 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d00c6420 con 0x7f25d0157490 2023-11-15T02:59:02.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.490+0000 7f25cf7fe700 1 --2- 172.21.15.121:0/1620921288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25d01570b0 0x7f25d0066860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:59:02.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.490+0000 7f25ce7fc700 1 -- 172.21.15.121:0/1620921288 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 145485065 0 0) 0x7f25d0062da0 con 0x7f25d00c5c80 2023-11-15T02:59:02.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.494+0000 7f25ceffd700 1 --2- 172.21.15.121:0/1620921288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25d0157490 0x7f25d0065950 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f25c4009760 tx=0x7f25c4009060).ready entity=mon.1 client_cookie=24159588f9c2fcea server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:02.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.494+0000 7f25ce7fc700 1 -- 172.21.15.121:0/1620921288 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c400f510 con 0x7f25d0157490 2023-11-15T02:59:02.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.494+0000 7f25ce7fc700 1 -- 172.21.15.121:0/1620921288 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c400f670 con 0x7f25d0157490 2023-11-15T02:59:02.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.494+0000 7f25ce7fc700 1 -- 172.21.15.121:0/1620921288 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c400f9f0 con 0x7f25d0157490 2023-11-15T02:59:02.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.494+0000 7f25d5ef1700 1 -- 172.21.15.121:0/1620921288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25d0157490 msgr2=0x7f25d0065950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:02.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.494+0000 7f25d5ef1700 1 --2- 172.21.15.121:0/1620921288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25d0157490 0x7f25d0065950 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f25c4009760 tx=0x7f25c4009060).stop 2023-11-15T02:59:02.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.494+0000 7f25d5ef1700 1 -- 172.21.15.121:0/1620921288 shutdown_connections 2023-11-15T02:59:02.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.494+0000 7f25d5ef1700 1 --2- 172.21.15.121:0/1620921288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25d0157490 0x7f25d0065950 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.494+0000 7f25d5ef1700 1 --2- 172.21.15.121:0/1620921288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25d01570b0 0x7f25d0066860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.494+0000 7f25d5ef1700 1 -- 172.21.15.121:0/1620921288 >> 172.21.15.121:0/1620921288 conn(0x7f25d00684f0 msgr2=0x7f25d00cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:02.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.494+0000 7f25d5ef1700 1 -- 172.21.15.121:0/1620921288 shutdown_connections 2023-11-15T02:59:02.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.494+0000 7f25d5ef1700 1 -- 172.21.15.121:0/1620921288 wait complete. 2023-11-15T02:59:02.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.494+0000 7f25d5ef1700 1 Processor -- start 2023-11-15T02:59:02.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.494+0000 7f25d5ef1700 1 -- start start 2023-11-15T02:59:02.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25d5ef1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25d00c5c80 0x7f25d00c16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25d5ef1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25d01570b0 0x7f25d00ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25d5ef1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25d00c2ee0 0x7f25d00ae630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25d5ef1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00adf70 con 0x7f25d01570b0 2023-11-15T02:59:02.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25d5ef1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00c1bc0 con 0x7f25d00c5c80 2023-11-15T02:59:02.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25d5ef1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00c1e40 con 0x7f25d00c2ee0 2023-11-15T02:59:02.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25ceffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25d01570b0 0x7f25d00ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25ceffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25d01570b0 0x7f25d00ae140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38056/0 (socket says 172.21.15.121:38056) 2023-11-15T02:59:02.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25ceffd700 1 -- 172.21.15.121:0/1298799501 learned_addr learned my addr 172.21.15.121:0/1298799501 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:02.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25cf7fe700 1 --2- 172.21.15.121:0/1298799501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25d00c5c80 0x7f25d00c16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25cffff700 1 --2- 172.21.15.121:0/1298799501 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25d00c2ee0 0x7f25d00ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25ceffd700 1 -- 172.21.15.121:0/1298799501 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25d00c2ee0 msgr2=0x7f25d00ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:02.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25ceffd700 1 --2- 172.21.15.121:0/1298799501 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25d00c2ee0 0x7f25d00ae630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25ceffd700 1 -- 172.21.15.121:0/1298799501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25d00c5c80 msgr2=0x7f25d00c16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:02.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25ceffd700 1 --2- 172.21.15.121:0/1298799501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25d00c5c80 0x7f25d00c16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25ceffd700 1 -- 172.21.15.121:0/1298799501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c00092a0 con 0x7f25d01570b0 2023-11-15T02:59:02.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25cf7fe700 1 --2- 172.21.15.121:0/1298799501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25d00c5c80 0x7f25d00c16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:59:02.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25ceffd700 1 --2- 172.21.15.121:0/1298799501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25d01570b0 0x7f25d00ae140 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f25c4011000 tx=0x7f25c4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:02.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25b77fe700 1 -- 172.21.15.121:0/1298799501 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c400f7e0 con 0x7f25d01570b0 2023-11-15T02:59:02.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25b77fe700 1 -- 172.21.15.121:0/1298799501 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c400f940 con 0x7f25d01570b0 2023-11-15T02:59:02.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25b77fe700 1 -- 172.21.15.121:0/1298799501 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c401b620 con 0x7f25d01570b0 2023-11-15T02:59:02.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25d5ef1700 1 -- 172.21.15.121:0/1298799501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c40092a0 con 0x7f25d01570b0 2023-11-15T02:59:02.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.498+0000 7f25d5ef1700 1 -- 172.21.15.121:0/1298799501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d00c2400 con 0x7f25d01570b0 2023-11-15T02:59:02.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.502+0000 7f25d5ef1700 1 -- 172.21.15.121:0/1298799501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d0062f50 con 0x7f25d01570b0 2023-11-15T02:59:02.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.502+0000 7f25b77fe700 1 -- 172.21.15.121:0/1298799501 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f25c401b9d0 con 0x7f25d01570b0 2023-11-15T02:59:02.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.502+0000 7f25b77fe700 1 --2- 172.21.15.121:0/1298799501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25bc05a540 0x7f25bc05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.502+0000 7f25b77fe700 1 -- 172.21.15.121:0/1298799501 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f25c40760a0 con 0x7f25d01570b0 2023-11-15T02:59:02.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.502+0000 7f25cf7fe700 1 --2- 172.21.15.121:0/1298799501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25bc05a540 0x7f25bc05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.506+0000 7f25cf7fe700 1 --2- 172.21.15.121:0/1298799501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25bc05a540 0x7f25bc05ca00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f25c00114c0 tx=0x7f25c0009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:02.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.510+0000 7f25b77fe700 1 -- 172.21.15.121:0/1298799501 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f25d0062f50 con 0x7f25d01570b0 2023-11-15T02:59:02.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.726+0000 7f25d5ef1700 1 -- 172.21.15.121:0/1298799501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f25d0062f50 con 0x7f25d01570b0 2023-11-15T02:59:02.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.734+0000 7f25b77fe700 1 -- 172.21.15.121:0/1298799501 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f25d0062f50 con 0x7f25d01570b0 2023-11-15T02:59:02.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.738+0000 7f25d5ef1700 1 -- 172.21.15.121:0/1298799501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25bc05a540 msgr2=0x7f25bc05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:02.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.738+0000 7f25d5ef1700 1 --2- 172.21.15.121:0/1298799501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25bc05a540 0x7f25bc05ca00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f25c00114c0 tx=0x7f25c0009060).stop 2023-11-15T02:59:02.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.738+0000 7f25d5ef1700 1 -- 172.21.15.121:0/1298799501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25d01570b0 msgr2=0x7f25d00ae140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:02.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.738+0000 7f25d5ef1700 1 --2- 172.21.15.121:0/1298799501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25d01570b0 0x7f25d00ae140 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f25c4011000 tx=0x7f25c4017000).stop 2023-11-15T02:59:02.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.738+0000 7f25d5ef1700 1 -- 172.21.15.121:0/1298799501 shutdown_connections 2023-11-15T02:59:02.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.738+0000 7f25d5ef1700 1 --2- 172.21.15.121:0/1298799501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25bc05a540 0x7f25bc05ca00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.738+0000 7f25d5ef1700 1 --2- 172.21.15.121:0/1298799501 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25d00c2ee0 0x7f25d00ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.738+0000 7f25d5ef1700 1 --2- 172.21.15.121:0/1298799501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25d01570b0 0x7f25d00ae140 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.738+0000 7f25d5ef1700 1 --2- 172.21.15.121:0/1298799501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25d00c5c80 0x7f25d00c16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.738+0000 7f25d5ef1700 1 -- 172.21.15.121:0/1298799501 >> 172.21.15.121:0/1298799501 conn(0x7f25d00684f0 msgr2=0x7f25d00aa080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:02.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.738+0000 7f25d5ef1700 1 -- 172.21.15.121:0/1298799501 shutdown_connections 2023-11-15T02:59:02.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.738+0000 7f25d5ef1700 1 -- 172.21.15.121:0/1298799501 wait complete. 2023-11-15T02:59:02.749 INFO:teuthology.orchestra.run.smithi121.stderr:updated caps for client.0 2023-11-15T02:59:02.768 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:59:02.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.918+0000 7f4dfa507700 1 Processor -- start 2023-11-15T02:59:02.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.918+0000 7f4dfa507700 1 -- start start 2023-11-15T02:59:02.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.922+0000 7f4dfa507700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df4066d70 0x7f4df40a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.922+0000 7f4dfa507700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df40a9350 con 0x7f4df40c29a0 2023-11-15T02:59:02.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.922+0000 7f4dfa507700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df4151420 con 0x7f4df4067150 2023-11-15T02:59:02.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.922+0000 7f4dfa507700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df4150ed0 con 0x7f4df4066d70 2023-11-15T02:59:02.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.922+0000 7f4df9d06700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4df40c29a0 0x7f4df414e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44322/0 (socket says 172.21.15.121:44322) 2023-11-15T02:59:02.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.922+0000 7f4df9d06700 1 -- 172.21.15.121:0/2421415572 learned_addr learned my addr 172.21.15.121:0/2421415572 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:02.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.922+0000 7f4df9505700 1 --2- 172.21.15.121:0/2421415572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df4066d70 0x7f4df40a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4debfff700 1 -- 172.21.15.121:0/2421415572 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2966728677 0 0) 0x7f4df40a9350 con 0x7f4df40c29a0 2023-11-15T02:59:02.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4debfff700 1 -- 172.21.15.121:0/2421415572 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4de00033f0 con 0x7f4df40c29a0 2023-11-15T02:59:02.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4debfff700 1 -- 172.21.15.121:0/2421415572 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1997648979 0 0) 0x7f4de00033f0 con 0x7f4df40c29a0 2023-11-15T02:59:02.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4debfff700 1 -- 172.21.15.121:0/2421415572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df4066d70 msgr2=0x7f4df40a87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:02.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4debfff700 1 --2- 172.21.15.121:0/2421415572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df4066d70 0x7f4df40a87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4debfff700 1 -- 172.21.15.121:0/2421415572 >> v1:172.21.15.190:6789/0 conn(0x7f4df4067150 legacy=0x7f4df41518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:02.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4debfff700 1 -- 172.21.15.121:0/2421415572 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df40a8d90 con 0x7f4df40c29a0 2023-11-15T02:59:02.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4debfff700 1 -- 172.21.15.121:0/2421415572 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f4dec0017f0 con 0x7f4df40c29a0 2023-11-15T02:59:02.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4debfff700 1 -- 172.21.15.121:0/2421415572 >> v1:172.21.15.121:6789/0 conn(0x7f4df40c29a0 legacy=0x7f4df414e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:02.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4debfff700 1 --2- 172.21.15.121:0/2421415572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df4066d70 0x7f4de0003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4debfff700 1 --2- 172.21.15.121:0/2421415572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4de0003e40 0x7f4de0004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4debfff700 1 --2- 172.21.15.121:0/2421415572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4de0004b80 0x7f4de0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4debfff700 1 -- 172.21.15.121:0/2421415572 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df4150ed0 con 0x7f4de0004b80 2023-11-15T02:59:02.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4debfff700 1 -- 172.21.15.121:0/2421415572 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de000b990 con 0x7f4de0003e40 2023-11-15T02:59:02.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4debfff700 1 -- 172.21.15.121:0/2421415572 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de000bb50 con 0x7f4df4066d70 2023-11-15T02:59:02.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4df9d06700 1 --2- 172.21.15.121:0/2421415572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4de0004b80 0x7f4de0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4df9505700 1 --2- 172.21.15.121:0/2421415572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df4066d70 0x7f4de0003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4df8d04700 1 --2- 172.21.15.121:0/2421415572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4de0003e40 0x7f4de0004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4df9d06700 1 -- 172.21.15.121:0/2421415572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df4066d70 msgr2=0x7f4de0003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:02.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4df9d06700 1 --2- 172.21.15.121:0/2421415572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df4066d70 0x7f4de0003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4df9d06700 1 -- 172.21.15.121:0/2421415572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4de0003e40 msgr2=0x7f4de0004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:02.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4df9d06700 1 --2- 172.21.15.121:0/2421415572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4de0003e40 0x7f4de0004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4df9d06700 1 -- 172.21.15.121:0/2421415572 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de00033f0 con 0x7f4de0004b80 2023-11-15T02:59:02.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4df9d06700 1 --2- 172.21.15.121:0/2421415572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4de0004b80 0x7f4de0005000 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f4dec002130 tx=0x7f4df4151390).ready entity=mon.0 client_cookie=36d687e9a190818b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:02.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4debfff700 1 -- 172.21.15.121:0/2421415572 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec00ce90 con 0x7f4de0004b80 2023-11-15T02:59:02.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4debfff700 1 -- 172.21.15.121:0/2421415572 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec005b20 con 0x7f4de0004b80 2023-11-15T02:59:02.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4debfff700 1 -- 172.21.15.121:0/2421415572 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec005ea0 con 0x7f4de0004b80 2023-11-15T02:59:02.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4dfa507700 1 -- 172.21.15.121:0/2421415572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4de0004b80 msgr2=0x7f4de0005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:02.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4dfa507700 1 --2- 172.21.15.121:0/2421415572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4de0004b80 0x7f4de0005000 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f4dec002130 tx=0x7f4df4151390).stop 2023-11-15T02:59:02.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4df9d06700 1 -- 172.21.15.121:0/2421415572 reap_dead start 2023-11-15T02:59:02.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4dfa507700 1 -- 172.21.15.121:0/2421415572 shutdown_connections 2023-11-15T02:59:02.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4dfa507700 1 -- 172.21.15.121:0/2421415572 >> 172.21.15.121:0/2421415572 conn(0x7f4df4062190 msgr2=0x7f4df4062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:02.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4dfa507700 1 -- 172.21.15.121:0/2421415572 shutdown_connections 2023-11-15T02:59:02.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.926+0000 7f4dfa507700 1 -- 172.21.15.121:0/2421415572 wait complete. 2023-11-15T02:59:02.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.930+0000 7f4dfa507700 1 Processor -- start 2023-11-15T02:59:02.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.930+0000 7f4dfa507700 1 -- start start 2023-11-15T02:59:02.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.930+0000 7f4dfa507700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4de0004b80 0x7f4df4065de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.930+0000 7f4dfa507700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4df4152b90 0x7f4df413f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.930+0000 7f4dfa507700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4df40662d0 0x7f4df413f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.930+0000 7f4dfa507700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df40675d0 con 0x7f4df4152b90 2023-11-15T02:59:02.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.930+0000 7f4dfa507700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df4141bf0 con 0x7f4df40662d0 2023-11-15T02:59:02.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.930+0000 7f4dfa507700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df413f9e0 con 0x7f4de0004b80 2023-11-15T02:59:02.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.930+0000 7f4df8d04700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4df4152b90 0x7f4df413f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.930+0000 7f4df8d04700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4df4152b90 0x7f4df413f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38064/0 (socket says 172.21.15.121:38064) 2023-11-15T02:59:02.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.930+0000 7f4df8d04700 1 -- 172.21.15.121:0/295219171 learned_addr learned my addr 172.21.15.121:0/295219171 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:02.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.934+0000 7f4df9505700 1 --2- 172.21.15.121:0/295219171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4de0004b80 0x7f4df4065de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.934+0000 7f4df9d06700 1 --2- 172.21.15.121:0/295219171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4df40662d0 0x7f4df413f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.934+0000 7f4df8d04700 1 -- 172.21.15.121:0/295219171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4de0004b80 msgr2=0x7f4df4065de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:02.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.934+0000 7f4df8d04700 1 --2- 172.21.15.121:0/295219171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4de0004b80 0x7f4df4065de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.934+0000 7f4df8d04700 1 -- 172.21.15.121:0/295219171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4df40662d0 msgr2=0x7f4df413f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:02.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.934+0000 7f4df8d04700 1 --2- 172.21.15.121:0/295219171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4df40662d0 0x7f4df413f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:02.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.934+0000 7f4df8d04700 1 -- 172.21.15.121:0/295219171 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df4140fe0 con 0x7f4df4152b90 2023-11-15T02:59:02.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.934+0000 7f4df8d04700 1 --2- 172.21.15.121:0/295219171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4df4152b90 0x7f4df413f000 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f4de400e430 tx=0x7f4de4009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:02.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.934+0000 7f4de97fa700 1 -- 172.21.15.121:0/295219171 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de4015400 con 0x7f4df4152b90 2023-11-15T02:59:02.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.934+0000 7f4de97fa700 1 -- 172.21.15.121:0/295219171 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de4010040 con 0x7f4df4152b90 2023-11-15T02:59:02.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.934+0000 7f4de97fa700 1 -- 172.21.15.121:0/295219171 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de4015770 con 0x7f4df4152b90 2023-11-15T02:59:02.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.934+0000 7f4dfa507700 1 -- 172.21.15.121:0/295219171 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df41412c0 con 0x7f4df4152b90 2023-11-15T02:59:02.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.934+0000 7f4dfa507700 1 -- 172.21.15.121:0/295219171 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df4141880 con 0x7f4df4152b90 2023-11-15T02:59:02.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.934+0000 7f4dfa507700 1 -- 172.21.15.121:0/295219171 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df4141d60 con 0x7f4df4152b90 2023-11-15T02:59:02.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.938+0000 7f4de97fa700 1 -- 172.21.15.121:0/295219171 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4de4015b20 con 0x7f4df4152b90 2023-11-15T02:59:02.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.938+0000 7f4de97fa700 1 --2- 172.21.15.121:0/295219171 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4dd805a540 0x7f4dd805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:02.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.938+0000 7f4de97fa700 1 -- 172.21.15.121:0/295219171 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4de4074400 con 0x7f4df4152b90 2023-11-15T02:59:02.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.938+0000 7f4df9505700 1 --2- 172.21.15.121:0/295219171 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4dd805a540 0x7f4dd805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:02.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.938+0000 7f4df9505700 1 --2- 172.21.15.121:0/295219171 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4dd805a540 0x7f4dd805ca00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f4df0009c30 tx=0x7f4df0008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:02.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:02.946+0000 7f4de97fa700 1 -- 172.21.15.121:0/295219171 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4de4047ac0 con 0x7f4df4152b90 2023-11-15T02:59:03.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.142+0000 7f4dfa507700 1 -- 172.21.15.121:0/295219171 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4df40ac280 con 0x7f4df4152b90 2023-11-15T02:59:03.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.142+0000 7f4de97fa700 1 -- 172.21.15.121:0/295219171 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4501 (secure 0 0 0) 0x7f4de404b0e0 con 0x7f4df4152b90 2023-11-15T02:59:03.150 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:03.150 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":30,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3403224816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3403224816},{"type":"v1","addr":"172.21.15.190:6835","nonce":3403224816}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2088953798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2088953798},{"type":"v1","addr":"172.21.15.190:6833","nonce":2088953798}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5511,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2190514138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2190514138},{"type":"v1","addr":"172.21.15.121:6835","nonce":2190514138}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:58:57.609524+0000","modified":"2023-11-15T02:59:00.621955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5514":{"gid":5514,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6837/1945477051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1945477051},{"type":"v1","addr":"172.21.15.121:6837","nonce":1945477051}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-15T02:59:03.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.146+0000 7f4dfa507700 1 -- 172.21.15.121:0/295219171 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4dd805a540 msgr2=0x7f4dd805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:03.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.146+0000 7f4dfa507700 1 --2- 172.21.15.121:0/295219171 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4dd805a540 0x7f4dd805ca00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f4df0009c30 tx=0x7f4df0008040).stop 2023-11-15T02:59:03.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.146+0000 7f4dfa507700 1 -- 172.21.15.121:0/295219171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4df4152b90 msgr2=0x7f4df413f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:03.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.146+0000 7f4dfa507700 1 --2- 172.21.15.121:0/295219171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4df4152b90 0x7f4df413f000 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f4de400e430 tx=0x7f4de4009040).stop 2023-11-15T02:59:03.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.146+0000 7f4dfa507700 1 -- 172.21.15.121:0/295219171 shutdown_connections 2023-11-15T02:59:03.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.146+0000 7f4dfa507700 1 --2- 172.21.15.121:0/295219171 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4dd805a540 0x7f4dd805ca00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.146+0000 7f4dfa507700 1 --2- 172.21.15.121:0/295219171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4df40662d0 0x7f4df413f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.146+0000 7f4dfa507700 1 --2- 172.21.15.121:0/295219171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4df4152b90 0x7f4df413f000 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.146+0000 7f4dfa507700 1 --2- 172.21.15.121:0/295219171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4de0004b80 0x7f4df4065de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.146+0000 7f4dfa507700 1 -- 172.21.15.121:0/295219171 >> 172.21.15.121:0/295219171 conn(0x7f4df4062190 msgr2=0x7f4df414fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:03.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.146+0000 7f4dfa507700 1 -- 172.21.15.121:0/295219171 shutdown_connections 2023-11-15T02:59:03.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.146+0000 7f4dfa507700 1 -- 172.21.15.121:0/295219171 wait complete. 2023-11-15T02:59:03.157 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 30 2023-11-15T02:59:03.578 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T02:58:57.609524+0000', 'modified': '2023-11-15T02:59:00.621955+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5514}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5514': {'gid': 5514, 'name': 'c', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.121:6837/1945477051', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6836', 'nonce': 1945477051}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 1945477051}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T02:59:03.578 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T02:59:03.579 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-15T02:59:03.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.738+0000 7f9a106db700 1 Processor -- start 2023-11-15T02:59:03.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.738+0000 7f9a106db700 1 -- start start 2023-11-15T02:59:03.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.738+0000 7f9a106db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9a08157e30 0x7f9a08158210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.738+0000 7f9a106db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a080ac420 0x7f9a080667f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.738+0000 7f9a106db700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a080aaf80 con 0x7f9a08062190 2023-11-15T02:59:03.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.738+0000 7f9a106db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a080aa970 con 0x7f9a08157e30 2023-11-15T02:59:03.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.738+0000 7f9a106db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a08066e10 con 0x7f9a080ac420 2023-11-15T02:59:03.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.742+0000 7f9a0e477700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9a08157e30 0x7f9a08158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.742+0000 7f9a0ec78700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a080ac420 0x7f9a080667f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.742+0000 7f9a0ec78700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a080ac420 0x7f9a080667f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:37146/0 (socket says 172.21.15.121:37146) 2023-11-15T02:59:03.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.742+0000 7f9a0e477700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9a08157e30 0x7f9a08158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:56290/0 (socket says 172.21.15.121:56290) 2023-11-15T02:59:03.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.742+0000 7f9a0ec78700 1 -- 172.21.15.121:0/4247616786 learned_addr learned my addr 172.21.15.121:0/4247616786 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:03.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.742+0000 7f9a0dc76700 1 --1- 172.21.15.121:0/4247616786 >> v1:172.21.15.190:6789/0 conn(0x7f9a08062190 0x7f9a080660e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:48496/0 (socket says 172.21.15.121:48496) 2023-11-15T02:59:03.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.742+0000 7f9a0ec78700 1 -- 172.21.15.121:0/4247616786 >> v2:172.21.15.190:3300/0 conn(0x7f9a08157e30 msgr2=0x7f9a08158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:03.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.742+0000 7f9a0ec78700 1 --2- 172.21.15.121:0/4247616786 >> v2:172.21.15.190:3300/0 conn(0x7f9a08157e30 0x7f9a08158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.742+0000 7f9a0ec78700 1 -- 172.21.15.121:0/4247616786 >> v1:172.21.15.190:6789/0 conn(0x7f9a08062190 legacy=0x7f9a080660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:03.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.742+0000 7f9a0ec78700 1 -- 172.21.15.121:0/4247616786 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a080670f0 con 0x7f9a080ac420 2023-11-15T02:59:03.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.742+0000 7f9a0ec78700 1 --2- 172.21.15.121:0/4247616786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a080ac420 0x7f9a080667f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f9a00009f20 tx=0x7f9a00009840).ready entity=mon.2 client_cookie=6b6b9cf09e6c2573 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:03.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.742+0000 7f9a0d475700 1 -- 172.21.15.121:0/4247616786 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0000e070 con 0x7f9a080ac420 2023-11-15T02:59:03.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.742+0000 7f9a0d475700 1 -- 172.21.15.121:0/4247616786 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0000b9d0 con 0x7f9a080ac420 2023-11-15T02:59:03.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.742+0000 7f9a0d475700 1 -- 172.21.15.121:0/4247616786 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0000bd50 con 0x7f9a080ac420 2023-11-15T02:59:03.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.742+0000 7f9a106db700 1 -- 172.21.15.121:0/4247616786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a080ac420 msgr2=0x7f9a080667f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:03.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.742+0000 7f9a106db700 1 --2- 172.21.15.121:0/4247616786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a080ac420 0x7f9a080667f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f9a00009f20 tx=0x7f9a00009840).stop 2023-11-15T02:59:03.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.746+0000 7f9a106db700 1 -- 172.21.15.121:0/4247616786 shutdown_connections 2023-11-15T02:59:03.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.746+0000 7f9a106db700 1 --2- 172.21.15.121:0/4247616786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a080ac420 0x7f9a080667f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.746+0000 7f9a106db700 1 --2- 172.21.15.121:0/4247616786 >> v2:172.21.15.190:3300/0 conn(0x7f9a08157e30 0x7f9a08158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.746+0000 7f9a106db700 1 -- 172.21.15.121:0/4247616786 >> 172.21.15.121:0/4247616786 conn(0x7f9a08067a60 msgr2=0x7f9a08067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:03.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.746+0000 7f9a106db700 1 -- 172.21.15.121:0/4247616786 shutdown_connections 2023-11-15T02:59:03.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.746+0000 7f9a106db700 1 -- 172.21.15.121:0/4247616786 wait complete. 2023-11-15T02:59:03.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.746+0000 7f9a106db700 1 Processor -- start 2023-11-15T02:59:03.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a106db700 1 -- start start 2023-11-15T02:59:03.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a106db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a08062190 0x7f9a0814e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a106db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a080ac420 0x7f9a0814c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a106db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a08157e30 0x7f9a0814de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a106db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0814d6c0 con 0x7f9a080ac420 2023-11-15T02:59:03.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a106db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0814cab0 con 0x7f9a08157e30 2023-11-15T02:59:03.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a106db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0814cd90 con 0x7f9a08062190 2023-11-15T02:59:03.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a0dc76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a080ac420 0x7f9a0814c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a0dc76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a080ac420 0x7f9a0814c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38070/0 (socket says 172.21.15.121:38070) 2023-11-15T02:59:03.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a0dc76700 1 -- 172.21.15.121:0/122141029 learned_addr learned my addr 172.21.15.121:0/122141029 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:03.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a0e477700 1 --2- 172.21.15.121:0/122141029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a08062190 0x7f9a0814e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a0ec78700 1 --2- 172.21.15.121:0/122141029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a08157e30 0x7f9a0814de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a0dc76700 1 -- 172.21.15.121:0/122141029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a08062190 msgr2=0x7f9a0814e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:03.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a0dc76700 1 --2- 172.21.15.121:0/122141029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a08062190 0x7f9a0814e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a0dc76700 1 -- 172.21.15.121:0/122141029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a08157e30 msgr2=0x7f9a0814de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:03.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a0dc76700 1 --2- 172.21.15.121:0/122141029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a08157e30 0x7f9a0814de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a0dc76700 1 -- 172.21.15.121:0/122141029 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a00009ae0 con 0x7f9a080ac420 2023-11-15T02:59:03.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a0dc76700 1 --2- 172.21.15.121:0/122141029 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a080ac420 0x7f9a0814c310 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f99f8007fd0 tx=0x7f99f8007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:03.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f99fe7fc700 1 -- 172.21.15.121:0/122141029 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f800ce10 con 0x7f9a080ac420 2023-11-15T02:59:03.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a106db700 1 -- 172.21.15.121:0/122141029 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0814d070 con 0x7f9a080ac420 2023-11-15T02:59:03.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f99fe7fc700 1 -- 172.21.15.121:0/122141029 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f80072e0 con 0x7f9a080ac420 2023-11-15T02:59:03.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f99fe7fc700 1 -- 172.21.15.121:0/122141029 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f80056e0 con 0x7f9a080ac420 2023-11-15T02:59:03.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.750+0000 7f9a106db700 1 -- 172.21.15.121:0/122141029 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0814c800 con 0x7f9a080ac420 2023-11-15T02:59:03.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.754+0000 7f99fe7fc700 1 -- 172.21.15.121:0/122141029 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f99f8005ad0 con 0x7f9a080ac420 2023-11-15T02:59:03.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.754+0000 7f99fe7fc700 1 --2- 172.21.15.121:0/122141029 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f99ec05a540 0x7f99ec05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.754+0000 7f99fe7fc700 1 -- 172.21.15.121:0/122141029 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f99f8073ea0 con 0x7f9a080ac420 2023-11-15T02:59:03.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.758+0000 7f9a0e477700 1 --2- 172.21.15.121:0/122141029 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f99ec05a540 0x7f99ec05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.758+0000 7f9a0e477700 1 --2- 172.21.15.121:0/122141029 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f99ec05a540 0x7f99ec05ca00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f9a04009b90 tx=0x7f9a04009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:03.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.758+0000 7f9a1aa38740 1 Processor -- start 2023-11-15T02:59:03.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.758+0000 7f9a1aa38740 1 -- start start 2023-11-15T02:59:03.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.758+0000 7f9a1aa38740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2290300 0x22d75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.758+0000 7f9a1aa38740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x228f510 0x22906e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.758+0000 7f9a1aa38740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22d5ab0 0x22d5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.758+0000 7f9a1aa38740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2210410 con 0x22d5ab0 2023-11-15T02:59:03.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.758+0000 7f9a1aa38740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x2210550 con 0x228f510 2023-11-15T02:59:03.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.758+0000 7f9a1aa38740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2210a00 con 0x2290300 2023-11-15T02:59:03.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.758+0000 7f9a0ec78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22d5ab0 0x22d5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.758+0000 7f9a0ec78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22d5ab0 0x22d5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38074/0 (socket says 172.21.15.121:38074) 2023-11-15T02:59:03.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.758+0000 7f9a0ec78700 1 -- 172.21.15.121:0/3258673904 learned_addr learned my addr 172.21.15.121:0/3258673904 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:03.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a0e477700 1 --2- 172.21.15.121:0/3258673904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2290300 0x22d75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a0dc76700 1 --2- 172.21.15.121:0/3258673904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x228f510 0x22906e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a0ec78700 1 -- 172.21.15.121:0/3258673904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2290300 msgr2=0x22d75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:03.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a0ec78700 1 --2- 172.21.15.121:0/3258673904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2290300 0x22d75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a0ec78700 1 -- 172.21.15.121:0/3258673904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x228f510 msgr2=0x22906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:03.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a0ec78700 1 --2- 172.21.15.121:0/3258673904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x228f510 0x22906e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a0ec78700 1 -- 172.21.15.121:0/3258673904 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2209730 con 0x22d5ab0 2023-11-15T02:59:03.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a0dc76700 1 --2- 172.21.15.121:0/3258673904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x228f510 0x22906e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:03.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a0ec78700 1 --2- 172.21.15.121:0/3258673904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22d5ab0 0x22d5f30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f9a00010000 tx=0x7f9a00009840).ready entity=mon.0 client_cookie=c49c13fc335b1501 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:03.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f99eb7fe700 1 -- 172.21.15.121:0/3258673904 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00023070 con 0x22d5ab0 2023-11-15T02:59:03.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f99eb7fe700 1 -- 172.21.15.121:0/3258673904 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0000bc10 con 0x22d5ab0 2023-11-15T02:59:03.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f99eb7fe700 1 -- 172.21.15.121:0/3258673904 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0000bd80 con 0x22d5ab0 2023-11-15T02:59:03.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a1aa38740 1 -- 172.21.15.121:0/3258673904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22d5ab0 msgr2=0x22d5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:03.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/3258673904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22d5ab0 0x22d5f30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f9a00010000 tx=0x7f9a00009840).stop 2023-11-15T02:59:03.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a1aa38740 1 -- 172.21.15.121:0/3258673904 shutdown_connections 2023-11-15T02:59:03.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/3258673904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22d5ab0 0x22d5f30 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/3258673904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x228f510 0x22906e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/3258673904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2290300 0x22d75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a1aa38740 1 -- 172.21.15.121:0/3258673904 >> 172.21.15.121:0/3258673904 conn(0x206e550 msgr2=0x22d6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:03.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a1aa38740 1 -- 172.21.15.121:0/3258673904 shutdown_connections 2023-11-15T02:59:03.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a1aa38740 1 -- 172.21.15.121:0/3258673904 wait complete. 2023-11-15T02:59:03.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a1aa38740 1 -- start start 2023-11-15T02:59:03.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.762+0000 7f9a1aa38740 1 Processor -- start 2023-11-15T02:59:03.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a1aa38740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 0x22bbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a1aa38740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2290300 0x22cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a1aa38740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22d5ab0 0x22baea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a1aa38740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x21db650 con 0x228f510 2023-11-15T02:59:03.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a1aa38740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x22cefd0 con 0x22d5ab0 2023-11-15T02:59:03.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a1aa38740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x22b6a70 con 0x2290300 2023-11-15T02:59:03.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a1aa38740 10 client.0 ll_register_callbacks cb 0x2248020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:59:03.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a0e477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 0x22bbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a0e477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 0x22bbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38080/0 (socket says 172.21.15.121:38080) 2023-11-15T02:59:03.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a0e477700 1 -- 172.21.15.121:0/1249818878 learned_addr learned my addr 172.21.15.121:0/1249818878 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:03.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a0ec78700 1 --2- 172.21.15.121:0/1249818878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22d5ab0 0x22baea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a0dc76700 1 --2- 172.21.15.121:0/1249818878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2290300 0x22cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a0e477700 1 -- 172.21.15.121:0/1249818878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2290300 msgr2=0x22cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:03.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a0e477700 1 --2- 172.21.15.121:0/1249818878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2290300 0x22cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a0e477700 1 -- 172.21.15.121:0/1249818878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22d5ab0 msgr2=0x22baea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:03.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a0e477700 1 --2- 172.21.15.121:0/1249818878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22d5ab0 0x22baea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a0e477700 1 -- 172.21.15.121:0/1249818878 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a080670f0 con 0x228f510 2023-11-15T02:59:03.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a0dc76700 1 --2- 172.21.15.121:0/1249818878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2290300 0x22cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:03.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a0ec78700 1 --2- 172.21.15.121:0/1249818878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22d5ab0 0x22baea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:03.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a0e477700 1 --2- 172.21.15.121:0/1249818878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 0x22bbf30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f9a04014970 tx=0x7f9a04017000).ready entity=mon.0 client_cookie=c591076852b9b734 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:03.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f99e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:59:03.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f99e8ff9700 1 -- 172.21.15.121:0/1249818878 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a040118b0 con 0x228f510 2023-11-15T02:59:03.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a1aa38740 1 -- 172.21.15.121:0/1249818878 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x22061a0 con 0x228f510 2023-11-15T02:59:03.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f99e8ff9700 1 -- 172.21.15.121:0/1249818878 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9a04013830 con 0x228f510 2023-11-15T02:59:03.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f99e8ff9700 1 -- 172.21.15.121:0/1249818878 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a04013c00 con 0x228f510 2023-11-15T02:59:03.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a1aa38740 10 client.5643 fetch_fsmap learned FSMap version 30 2023-11-15T02:59:03.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a1aa38740 1 -- 172.21.15.121:0/1249818878 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x2204960 con 0x228f510 2023-11-15T02:59:03.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f99e8ff9700 1 -- 172.21.15.121:0/1249818878 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x22061a0 con 0x228f510 2023-11-15T02:59:03.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f99e8ff9700 1 -- 172.21.15.121:0/1249818878 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9a04020470 con 0x228f510 2023-11-15T02:59:03.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a1aa38740 10 client.5643 fetch_fsmap finished waiting for FSMap version 30 2023-11-15T02:59:03.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a1aa38740 10 client.5643 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-15T02:59:03.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a1aa38740 20 client.5643 populate_metadata read hostname 'smithi121' 2023-11-15T02:59:03.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/1249818878 >> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] conn(0x22e5fa0 0x22e6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a1aa38740 4 client.5643 mds_command: new command op to 5514 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T02:59:03.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a1aa38740 1 -- 172.21.15.121:0/1249818878 --> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x21fc6f0 con 0x22e5fa0 2023-11-15T02:59:03.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.766+0000 7f9a0dc76700 1 --2- 172.21.15.121:0/1249818878 >> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] conn(0x22e5fa0 0x22e6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.770+0000 7f99e8ff9700 10 client.5643 ms_handle_connect on v2:172.21.15.121:6836/1945477051 2023-11-15T02:59:03.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.770+0000 7f9a0dc76700 1 --2- 172.21.15.121:0/1249818878 >> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] conn(0x22e5fa0 0x22e6380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=d73e37538e4f25e9 server_cookie=c7c88fb261853e84 in_seq=0 out_seq=0 2023-11-15T02:59:03.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.770+0000 7f99e8ff9700 1 -- 172.21.15.121:0/1249818878 <== mds.0 v2:172.21.15.121:6836/1945477051 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x21fc6f0 con 0x22e5fa0 2023-11-15T02:59:03.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.770+0000 7f99e8ff9700 10 client.5643 handle_command_reply: tid=0 2023-11-15T02:59:03.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.770+0000 7f9a1aa38740 1 client.5643 shutdown 2023-11-15T02:59:03.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.770+0000 7f9a1aa38740 1 -- 172.21.15.121:0/1249818878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 msgr2=0x22bbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:03.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.770+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/1249818878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 0x22bbf30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f9a04014970 tx=0x7f9a04017000).stop 2023-11-15T02:59:03.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.770+0000 7f9a1aa38740 1 -- 172.21.15.121:0/1249818878 shutdown_connections 2023-11-15T02:59:03.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.770+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/1249818878 >> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] conn(0x22e5fa0 0x22e6380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.770+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/1249818878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22d5ab0 0x22baea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.770+0000 7f99e8ff9700 0 client.5643 ms_handle_reset on v2:172.21.15.121:6836/1945477051 2023-11-15T02:59:03.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.770+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/1249818878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2290300 0x22cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.770+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/1249818878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 0x22bbf30 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.770+0000 7f9a1aa38740 1 -- 172.21.15.121:0/1249818878 >> 172.21.15.121:0/1249818878 conn(0x206e550 msgr2=0x22d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:03.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.770+0000 7f9a1aa38740 1 -- 172.21.15.121:0/1249818878 shutdown_connections 2023-11-15T02:59:03.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.770+0000 7f9a1aa38740 1 -- 172.21.15.121:0/1249818878 wait complete. 2023-11-15T02:59:03.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.770+0000 7f9a1aa38740 20 client.5643 trim_cache size 0 max 16384 2023-11-15T02:59:03.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.790+0000 7f9a1aa38740 1 Processor -- start 2023-11-15T02:59:03.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.790+0000 7f9a1aa38740 1 -- start start 2023-11-15T02:59:03.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.790+0000 7f9a1aa38740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 0x22baea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.790+0000 7f9a1aa38740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2290300 0x228b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.790+0000 7f9a1aa38740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22d5ab0 0x228bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.790+0000 7f9a1aa38740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x225c2c0 con 0x228f510 2023-11-15T02:59:03.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.790+0000 7f9a1aa38740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x21e5c60 con 0x2290300 2023-11-15T02:59:03.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.790+0000 7f9a1aa38740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x226be40 con 0x22d5ab0 2023-11-15T02:59:03.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.790+0000 7f9a0e477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 0x22baea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.790+0000 7f9a0e477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 0x22baea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38096/0 (socket says 172.21.15.121:38096) 2023-11-15T02:59:03.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.790+0000 7f9a0e477700 1 -- 172.21.15.121:0/994269892 learned_addr learned my addr 172.21.15.121:0/994269892 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:03.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a0dc76700 1 --2- 172.21.15.121:0/994269892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2290300 0x228b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a0ec78700 1 --2- 172.21.15.121:0/994269892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22d5ab0 0x228bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a0e477700 1 -- 172.21.15.121:0/994269892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22d5ab0 msgr2=0x228bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:03.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a0e477700 1 --2- 172.21.15.121:0/994269892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22d5ab0 0x228bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a0e477700 1 -- 172.21.15.121:0/994269892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2290300 msgr2=0x228b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:03.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a0e477700 1 --2- 172.21.15.121:0/994269892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2290300 0x228b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a0e477700 1 -- 172.21.15.121:0/994269892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0814f230 con 0x228f510 2023-11-15T02:59:03.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a0dc76700 1 --2- 172.21.15.121:0/994269892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2290300 0x228b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:03.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a0ec78700 1 --2- 172.21.15.121:0/994269892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22d5ab0 0x228bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:03.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a0e477700 1 --2- 172.21.15.121:0/994269892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 0x22baea0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f9a04014940 tx=0x7f9a04025000).ready entity=mon.0 client_cookie=a7c829026cdadb84 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:03.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f99ebfff700 1 -- 172.21.15.121:0/994269892 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04020470 con 0x228f510 2023-11-15T02:59:03.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f99ebfff700 1 -- 172.21.15.121:0/994269892 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a040205d0 con 0x228f510 2023-11-15T02:59:03.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f99ebfff700 1 -- 172.21.15.121:0/994269892 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04013680 con 0x228f510 2023-11-15T02:59:03.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a1aa38740 1 -- 172.21.15.121:0/994269892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 msgr2=0x22baea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:03.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/994269892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 0x22baea0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f9a04014940 tx=0x7f9a04025000).stop 2023-11-15T02:59:03.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a1aa38740 1 -- 172.21.15.121:0/994269892 shutdown_connections 2023-11-15T02:59:03.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/994269892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22d5ab0 0x228bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/994269892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2290300 0x228b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/994269892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 0x22baea0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a1aa38740 1 -- 172.21.15.121:0/994269892 >> 172.21.15.121:0/994269892 conn(0x206e550 msgr2=0x22ec920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:03.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a1aa38740 1 -- 172.21.15.121:0/994269892 shutdown_connections 2023-11-15T02:59:03.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a1aa38740 1 -- 172.21.15.121:0/994269892 wait complete. 2023-11-15T02:59:03.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a1aa38740 1 -- start start 2023-11-15T02:59:03.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.794+0000 7f9a1aa38740 1 Processor -- start 2023-11-15T02:59:03.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a1aa38740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 0x232f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a1aa38740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2290300 0x23381d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a1aa38740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22d5ab0 0x22b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a1aa38740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2338c80 con 0x228f510 2023-11-15T02:59:03.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a1aa38740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x22a67a0 con 0x22d5ab0 2023-11-15T02:59:03.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a1aa38740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x232fb20 con 0x2290300 2023-11-15T02:59:03.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a1aa38740 10 client.0 ll_register_callbacks cb 0x7f9a04014440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:59:03.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a0e477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 0x232f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a0e477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 0x232f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38098/0 (socket says 172.21.15.121:38098) 2023-11-15T02:59:03.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a0e477700 1 -- 172.21.15.121:0/2728617856 learned_addr learned my addr 172.21.15.121:0/2728617856 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:03.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a0dc76700 1 --2- 172.21.15.121:0/2728617856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2290300 0x23381d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a0ec78700 1 --2- 172.21.15.121:0/2728617856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22d5ab0 0x22b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a0e477700 1 -- 172.21.15.121:0/2728617856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2290300 msgr2=0x23381d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:03.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a0e477700 1 --2- 172.21.15.121:0/2728617856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2290300 0x23381d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a0e477700 1 -- 172.21.15.121:0/2728617856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22d5ab0 msgr2=0x22b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:03.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a0e477700 1 --2- 172.21.15.121:0/2728617856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22d5ab0 0x22b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a0e477700 1 -- 172.21.15.121:0/2728617856 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2204960 con 0x228f510 2023-11-15T02:59:03.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a0dc76700 1 --2- 172.21.15.121:0/2728617856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2290300 0x23381d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:03.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a0e477700 1 --2- 172.21.15.121:0/2728617856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 0x232f630 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f9a04011da0 tx=0x7f9a04034040).ready entity=mon.0 client_cookie=55c123b41f1acc97 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:03.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f99e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:59:03.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f99e8ff9700 1 -- 172.21.15.121:0/2728617856 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04041070 con 0x228f510 2023-11-15T02:59:03.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f99e8ff9700 1 -- 172.21.15.121:0/2728617856 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9a04013e10 con 0x228f510 2023-11-15T02:59:03.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a1aa38740 1 -- 172.21.15.121:0/2728617856 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x20c54e0 con 0x228f510 2023-11-15T02:59:03.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f99e8ff9700 1 -- 172.21.15.121:0/2728617856 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a04013890 con 0x228f510 2023-11-15T02:59:03.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a1aa38740 10 client.5649 fetch_fsmap learned FSMap version 30 2023-11-15T02:59:03.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a1aa38740 1 -- 172.21.15.121:0/2728617856 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x22046b0 con 0x228f510 2023-11-15T02:59:03.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f99e8ff9700 1 -- 172.21.15.121:0/2728617856 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x20c54e0 con 0x228f510 2023-11-15T02:59:03.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f99e8ff9700 1 -- 172.21.15.121:0/2728617856 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9a0403b430 con 0x228f510 2023-11-15T02:59:03.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a1aa38740 10 client.5649 fetch_fsmap finished waiting for FSMap version 30 2023-11-15T02:59:03.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a1aa38740 10 client.5649 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-15T02:59:03.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a1aa38740 20 client.5649 populate_metadata read hostname 'smithi121' 2023-11-15T02:59:03.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/2728617856 >> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] conn(0x22a4510 0x22a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a1aa38740 4 client.5649 mds_command: new command op to 5514 tid=0[{"prefix": "status"}] 2023-11-15T02:59:03.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a1aa38740 1 -- 172.21.15.121:0/2728617856 --> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] -- command(tid 0: {"prefix": "status"}) v1 -- 0x207b260 con 0x22a4510 2023-11-15T02:59:03.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.798+0000 7f9a0dc76700 1 --2- 172.21.15.121:0/2728617856 >> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] conn(0x22a4510 0x22a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a0dc76700 1 --2- 172.21.15.121:0/2728617856 >> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] conn(0x22a4510 0x22a48f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=a3d3671e88e31f8d server_cookie=33c51e9a55a4a090 in_seq=0 out_seq=0 2023-11-15T02:59:03.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f99e8ff9700 10 client.5649 ms_handle_connect on v2:172.21.15.121:6836/1945477051 2023-11-15T02:59:03.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f99e8ff9700 1 -- 172.21.15.121:0/2728617856 <== mds.0 v2:172.21.15.121:6836/1945477051 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x207b260 con 0x22a4510 2023-11-15T02:59:03.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f99e8ff9700 10 client.5649 handle_command_reply: tid=0 2023-11-15T02:59:03.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a1aa38740 1 client.5649 shutdown 2023-11-15T02:59:03.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a1aa38740 1 -- 172.21.15.121:0/2728617856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 msgr2=0x232f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:03.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/2728617856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 0x232f630 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f9a04011da0 tx=0x7f9a04034040).stop 2023-11-15T02:59:03.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a1aa38740 1 -- 172.21.15.121:0/2728617856 shutdown_connections 2023-11-15T02:59:03.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/2728617856 >> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] conn(0x22a4510 0x22a48f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/2728617856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22d5ab0 0x22b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/2728617856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2290300 0x23381d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a1aa38740 1 --2- 172.21.15.121:0/2728617856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x228f510 0x232f630 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a1aa38740 1 -- 172.21.15.121:0/2728617856 >> 172.21.15.121:0/2728617856 conn(0x206e550 msgr2=0x228baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:03.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f99e8ff9700 0 client.5649 ms_handle_reset on v2:172.21.15.121:6836/1945477051 2023-11-15T02:59:03.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a1aa38740 1 -- 172.21.15.121:0/2728617856 shutdown_connections 2023-11-15T02:59:03.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a1aa38740 1 -- 172.21.15.121:0/2728617856 wait complete. 2023-11-15T02:59:03.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a1aa38740 20 client.5649 trim_cache size 0 max 16384 2023-11-15T02:59:03.820 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T02:59:03.820 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T02:59:03.821 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T02:59:03.821 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 5514, 2023-11-15T02:59:03.821 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T02:59:03.821 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T02:59:03.821 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T02:59:03.822 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 6.1917187230000001, 2023-11-15T02:59:03.822 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 30, 2023-11-15T02:59:03.822 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 48, 2023-11-15T02:59:03.822 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 48, 2023-11-15T02:59:03.822 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 11.722774883 2023-11-15T02:59:03.823 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T02:59:03.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a106db700 1 -- 172.21.15.121:0/122141029 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f99ec05a540 msgr2=0x7f99ec05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:03.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a106db700 1 --2- 172.21.15.121:0/122141029 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f99ec05a540 0x7f99ec05ca00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f9a04009b90 tx=0x7f9a04009060).stop 2023-11-15T02:59:03.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a106db700 1 -- 172.21.15.121:0/122141029 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a080ac420 msgr2=0x7f9a0814c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:03.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a106db700 1 --2- 172.21.15.121:0/122141029 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a080ac420 0x7f9a0814c310 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f99f8007fd0 tx=0x7f99f8007df0).stop 2023-11-15T02:59:03.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a106db700 1 -- 172.21.15.121:0/122141029 shutdown_connections 2023-11-15T02:59:03.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a106db700 1 --2- 172.21.15.121:0/122141029 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f99ec05a540 0x7f99ec05ca00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a106db700 1 --2- 172.21.15.121:0/122141029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a08157e30 0x7f9a0814de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a106db700 1 --2- 172.21.15.121:0/122141029 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a080ac420 0x7f9a0814c310 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a106db700 1 --2- 172.21.15.121:0/122141029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a08062190 0x7f9a0814e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a106db700 1 -- 172.21.15.121:0/122141029 >> 172.21.15.121:0/122141029 conn(0x7f9a08067a60 msgr2=0x7f9a08153870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:03.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a106db700 1 -- 172.21.15.121:0/122141029 shutdown_connections 2023-11-15T02:59:03.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.802+0000 7f9a106db700 1 -- 172.21.15.121:0/122141029 wait complete. 2023-11-15T02:59:03.829 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T02:59:03.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.958+0000 7ffa3f169700 1 Processor -- start 2023-11-15T02:59:03.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.958+0000 7ffa3f169700 1 -- start start 2023-11-15T02:59:03.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa3f169700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa38151890 0x7ffa38063ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa3f169700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffa38151c70 0x7ffa38063fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa3f169700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa380c20b0 0x7ffa380c2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa3f169700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa38156500 con 0x7ffa38151890 2023-11-15T02:59:03.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa3f169700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa380c2a30 con 0x7ffa38151c70 2023-11-15T02:59:03.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa3f169700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa381558f0 con 0x7ffa380c20b0 2023-11-15T02:59:03.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa3cf05700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa38151890 0x7ffa38063ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa3cf05700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa38151890 0x7ffa38063ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38110/0 (socket says 172.21.15.121:38110) 2023-11-15T02:59:03.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa3cf05700 1 -- 172.21.15.121:0/1210924557 learned_addr learned my addr 172.21.15.121:0/1210924557 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:03.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa3d706700 1 --2- 172.21.15.121:0/1210924557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa380c20b0 0x7ffa380c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa2ffff700 1 --2- 172.21.15.121:0/1210924557 >> v2:172.21.15.190:3300/0 conn(0x7ffa38151c70 0x7ffa38063fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa3cf05700 1 -- 172.21.15.121:0/1210924557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa380c20b0 msgr2=0x7ffa380c2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:03.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa3cf05700 1 --2- 172.21.15.121:0/1210924557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa380c20b0 0x7ffa380c2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa3cf05700 1 -- 172.21.15.121:0/1210924557 >> v2:172.21.15.190:3300/0 conn(0x7ffa38151c70 msgr2=0x7ffa38063fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:03.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa3cf05700 1 --2- 172.21.15.121:0/1210924557 >> v2:172.21.15.190:3300/0 conn(0x7ffa38151c70 0x7ffa38063fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa3cf05700 1 -- 172.21.15.121:0/1210924557 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa38155bd0 con 0x7ffa38151890 2023-11-15T02:59:03.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa3cf05700 1 --2- 172.21.15.121:0/1210924557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa38151890 0x7ffa38063ab0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7ffa380c0c00 tx=0x7ffa340090f0).ready entity=mon.0 client_cookie=658bcaed63419fe9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:03.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa2f7fe700 1 -- 172.21.15.121:0/1210924557 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3400f510 con 0x7ffa38151890 2023-11-15T02:59:03.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa2f7fe700 1 -- 172.21.15.121:0/1210924557 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3400f670 con 0x7ffa38151890 2023-11-15T02:59:03.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.962+0000 7ffa2f7fe700 1 -- 172.21.15.121:0/1210924557 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3400f9f0 con 0x7ffa38151890 2023-11-15T02:59:03.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.966+0000 7ffa3f169700 1 -- 172.21.15.121:0/1210924557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa38151890 msgr2=0x7ffa38063ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:03.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.966+0000 7ffa3f169700 1 --2- 172.21.15.121:0/1210924557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa38151890 0x7ffa38063ab0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7ffa380c0c00 tx=0x7ffa340090f0).stop 2023-11-15T02:59:03.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.966+0000 7ffa3f169700 1 -- 172.21.15.121:0/1210924557 shutdown_connections 2023-11-15T02:59:03.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.966+0000 7ffa3f169700 1 --2- 172.21.15.121:0/1210924557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa380c20b0 0x7ffa380c2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.966+0000 7ffa3f169700 1 --2- 172.21.15.121:0/1210924557 >> v2:172.21.15.190:3300/0 conn(0x7ffa38151c70 0x7ffa38063fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.966+0000 7ffa3f169700 1 --2- 172.21.15.121:0/1210924557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa38151890 0x7ffa38063ab0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.966+0000 7ffa3f169700 1 -- 172.21.15.121:0/1210924557 >> 172.21.15.121:0/1210924557 conn(0x7ffa380c66d0 msgr2=0x7ffa380c6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:03.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.966+0000 7ffa3f169700 1 -- 172.21.15.121:0/1210924557 shutdown_connections 2023-11-15T02:59:03.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.966+0000 7ffa3f169700 1 -- 172.21.15.121:0/1210924557 wait complete. 2023-11-15T02:59:03.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.966+0000 7ffa3f169700 1 Processor -- start 2023-11-15T02:59:03.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.966+0000 7ffa3f169700 1 -- start start 2023-11-15T02:59:03.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa3f169700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa380c20b0 0x7ffa380a76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa3f169700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa38151890 0x7ffa3814e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa3f169700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa380a7bd0 0x7ffa3814ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa3f169700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa380be4e0 con 0x7ffa380c20b0 2023-11-15T02:59:03.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa3f169700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa380bd910 con 0x7ffa380a7bd0 2023-11-15T02:59:03.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa3f169700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa380bdbb0 con 0x7ffa38151890 2023-11-15T02:59:03.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa3cf05700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa380c20b0 0x7ffa380a76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa3cf05700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa380c20b0 0x7ffa380a76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38120/0 (socket says 172.21.15.121:38120) 2023-11-15T02:59:03.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa3cf05700 1 -- 172.21.15.121:0/3772324809 learned_addr learned my addr 172.21.15.121:0/3772324809 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:03.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa3d706700 1 --2- 172.21.15.121:0/3772324809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa380a7bd0 0x7ffa3814ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa2ffff700 1 --2- 172.21.15.121:0/3772324809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa38151890 0x7ffa3814e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa3cf05700 1 -- 172.21.15.121:0/3772324809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa38151890 msgr2=0x7ffa3814e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:03.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa3cf05700 1 --2- 172.21.15.121:0/3772324809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa38151890 0x7ffa3814e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa3cf05700 1 -- 172.21.15.121:0/3772324809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa380a7bd0 msgr2=0x7ffa3814ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:03.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa3cf05700 1 --2- 172.21.15.121:0/3772324809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa380a7bd0 0x7ffa3814ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:03.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa3cf05700 1 -- 172.21.15.121:0/3772324809 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa34009370 con 0x7ffa380c20b0 2023-11-15T02:59:03.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa3cf05700 1 --2- 172.21.15.121:0/3772324809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa380c20b0 0x7ffa380a76e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7ffa38046990 tx=0x7ffa34017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:03.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa2cff9700 1 -- 172.21.15.121:0/3772324809 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3400f7e0 con 0x7ffa380c20b0 2023-11-15T02:59:03.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa2cff9700 1 -- 172.21.15.121:0/3772324809 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3400f940 con 0x7ffa380c20b0 2023-11-15T02:59:03.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa3f169700 1 -- 172.21.15.121:0/3772324809 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa380bde90 con 0x7ffa380c20b0 2023-11-15T02:59:03.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa2cff9700 1 -- 172.21.15.121:0/3772324809 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3401c670 con 0x7ffa380c20b0 2023-11-15T02:59:03.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.970+0000 7ffa3f169700 1 -- 172.21.15.121:0/3772324809 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa3814dfc0 con 0x7ffa380c20b0 2023-11-15T02:59:03.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.974+0000 7ffa3f169700 1 -- 172.21.15.121:0/3772324809 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa3805eeb0 con 0x7ffa380c20b0 2023-11-15T02:59:03.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.974+0000 7ffa2cff9700 1 -- 172.21.15.121:0/3772324809 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ffa3401ca60 con 0x7ffa380c20b0 2023-11-15T02:59:03.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.974+0000 7ffa2cff9700 1 --2- 172.21.15.121:0/3772324809 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa2405a510 0x7ffa2405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:03.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.974+0000 7ffa2cff9700 1 -- 172.21.15.121:0/3772324809 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ffa34076380 con 0x7ffa380c20b0 2023-11-15T02:59:03.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.974+0000 7ffa2ffff700 1 --2- 172.21.15.121:0/3772324809 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa2405a510 0x7ffa2405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:03.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.978+0000 7ffa2ffff700 1 --2- 172.21.15.121:0/3772324809 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa2405a510 0x7ffa2405c9d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7ffa3814cb30 tx=0x7ffa28009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:03.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:03.982+0000 7ffa2cff9700 1 -- 172.21.15.121:0/3772324809 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ffa34049aa0 con 0x7ffa380c20b0 2023-11-15T02:59:04.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:04.174+0000 7ffa3f169700 1 -- 172.21.15.121:0/3772324809 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffa38155eb0 con 0x7ffa380c20b0 2023-11-15T02:59:04.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:04.174+0000 7ffa2cff9700 1 -- 172.21.15.121:0/3772324809 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) v1 ==== 53+0+73 (secure 0 0 0) 0x7ffa3402a070 con 0x7ffa380c20b0 2023-11-15T02:59:04.182 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T02:59:04.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:04.178+0000 7ffa3f169700 1 -- 172.21.15.121:0/3772324809 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa2405a510 msgr2=0x7ffa2405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:04.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:04.178+0000 7ffa3f169700 1 --2- 172.21.15.121:0/3772324809 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa2405a510 0x7ffa2405c9d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7ffa3814cb30 tx=0x7ffa28009060).stop 2023-11-15T02:59:04.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:04.178+0000 7ffa3f169700 1 -- 172.21.15.121:0/3772324809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa380c20b0 msgr2=0x7ffa380a76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:04.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:04.178+0000 7ffa3f169700 1 --2- 172.21.15.121:0/3772324809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa380c20b0 0x7ffa380a76e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7ffa38046990 tx=0x7ffa34017000).stop 2023-11-15T02:59:04.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:04.178+0000 7ffa3f169700 1 -- 172.21.15.121:0/3772324809 shutdown_connections 2023-11-15T02:59:04.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:04.178+0000 7ffa3f169700 1 --2- 172.21.15.121:0/3772324809 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa2405a510 0x7ffa2405c9d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:04.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:04.178+0000 7ffa3f169700 1 --2- 172.21.15.121:0/3772324809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa380a7bd0 0x7ffa3814ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:04.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:04.178+0000 7ffa3f169700 1 --2- 172.21.15.121:0/3772324809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa38151890 0x7ffa3814e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:04.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:04.178+0000 7ffa3f169700 1 --2- 172.21.15.121:0/3772324809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa380c20b0 0x7ffa380a76e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:04.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:04.178+0000 7ffa3f169700 1 -- 172.21.15.121:0/3772324809 >> 172.21.15.121:0/3772324809 conn(0x7ffa380c66d0 msgr2=0x7ffa380ab670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:04.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:04.178+0000 7ffa3f169700 1 -- 172.21.15.121:0/3772324809 shutdown_connections 2023-11-15T02:59:04.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:04.178+0000 7ffa3f169700 1 -- 172.21.15.121:0/3772324809 wait complete. 2023-11-15T02:59:04.205 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T02:59:04.205 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T02:59:04.206 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T02:59:04.206 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T02:59:04.206 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T02:59:04.206 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T02:59:04.207 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T02:59:04.207 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T02:59:04.207 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T02:59:04.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:04.208 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T02:59:04.229 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T02:59:04.229 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T02:59:04.229 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T02:59:04.230 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:04.230 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T02:59:04.230 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:04.230 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:59:04.230 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:59:04.231 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T02:59:04.231 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:59:04.231 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:59:04.231 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T02:59:04.231 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:59:04.232 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:59:04.232 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T02:59:04.232 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:59:04.232 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T02:59:04.232 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42296sec preferred_lft 42296sec 2023-11-15T02:59:04.233 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T02:59:04.233 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:04.233 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T02:59:04.233 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:59:04.233 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T02:59:04.234 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:04.234 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T02:59:04.234 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T02:59:04.234 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T02:59:04.234 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:04.235 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T02:59:04.235 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:04.235 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T02:59:04.235 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T02:59:04.235 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T02:59:04.236 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:04.236 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T02:59:04.236 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T02:59:04.236 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T02:59:04.236 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:04.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:04.238 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T02:59:04.314 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T02:59:04.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:04.318 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T02:59:04.382 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T02:59:04.382 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T02:59:04.383 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T02:59:04.383 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T02:59:04.383 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T02:59:04.383 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T02:59:04.384 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T02:59:04.384 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T02:59:04.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:04.385 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T02:59:04.386 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T02:59:04.386 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T02:59:04.387 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T02:59:04.387 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T02:59:04.387 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T02:59:04.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:04.517 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T02:59:04.535 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T02:59:04.535 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T02:59:04.536 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T02:59:04.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:04.537 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T02:59:04.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:04.620 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T02:59:04.678 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T02:59:04.679 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:59:04.801 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:59:04.801 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:59:04.826 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:59:04.827 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:59:04.832 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T02:59:04.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:04.833 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T02:59:04.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:59:04.877 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'mkdir -p a/b') 2023-11-15T02:59:04.960 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2023-11-15T02:59:05.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.158+0000 7fc7d06db700 1 Processor -- start 2023-11-15T02:59:05.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.158+0000 7fc7d06db700 1 -- start start 2023-11-15T02:59:05.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.162+0000 7fc7d06db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc7c8153c10 0x7fc7c80bcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.162+0000 7fc7d06db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c80a9250 con 0x7fc7c80639b0 2023-11-15T02:59:05.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.162+0000 7fc7d06db700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c80bd5a0 con 0x7fc7c80bb6f0 2023-11-15T02:59:05.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.162+0000 7fc7d06db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c80a8920 con 0x7fc7c8153c10 2023-11-15T02:59:05.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.162+0000 7fc7cec78700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc7c80639b0 0x7fc7c8154770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:42916/0 (socket says 172.21.15.121:42916) 2023-11-15T02:59:05.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.162+0000 7fc7cec78700 1 -- 172.21.15.121:0/2866203579 learned_addr learned my addr 172.21.15.121:0/2866203579 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:05.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.162+0000 7fc7ce477700 1 --2- 172.21.15.121:0/2866203579 >> v2:172.21.15.190:3300/0 conn(0x7fc7c8153c10 0x7fc7c80bcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.162+0000 7fc7cd475700 1 -- 172.21.15.121:0/2866203579 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181880347 0 0) 0x7fc7c80a9250 con 0x7fc7c80639b0 2023-11-15T02:59:05.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.162+0000 7fc7cd475700 1 -- 172.21.15.121:0/2866203579 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7b40033f0 con 0x7fc7c80639b0 2023-11-15T02:59:05.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.162+0000 7fc7cd475700 1 -- 172.21.15.121:0/2866203579 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1283395505 0 0) 0x7fc7b40033f0 con 0x7fc7c80639b0 2023-11-15T02:59:05.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.162+0000 7fc7ce477700 1 -- 172.21.15.121:0/2866203579 >> v1:172.21.15.190:6789/0 conn(0x7fc7c80bb6f0 legacy=0x7fc7c80632a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.162+0000 7fc7ce477700 1 -- 172.21.15.121:0/2866203579 >> v1:172.21.15.121:6789/0 conn(0x7fc7c80639b0 legacy=0x7fc7c8154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.162+0000 7fc7ce477700 1 -- 172.21.15.121:0/2866203579 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80a8c00 con 0x7fc7c8153c10 2023-11-15T02:59:05.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.162+0000 7fc7ce477700 1 --2- 172.21.15.121:0/2866203579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7c8153c10 0x7fc7c80bcfe0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fc7c4009e60 tx=0x7fc7c4009780).ready entity=mon.1 client_cookie=3b1c60b29bb04cad server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:05.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.166+0000 7fc7cd475700 1 -- 172.21.15.121:0/2866203579 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c400e070 con 0x7fc7c8153c10 2023-11-15T02:59:05.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.166+0000 7fc7cd475700 1 -- 172.21.15.121:0/2866203579 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c400b850 con 0x7fc7c8153c10 2023-11-15T02:59:05.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.166+0000 7fc7cd475700 1 -- 172.21.15.121:0/2866203579 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0001470 con 0x7fc7c8153c10 2023-11-15T02:59:05.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.166+0000 7fc7d06db700 1 -- 172.21.15.121:0/2866203579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7c8153c10 msgr2=0x7fc7c80bcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.166+0000 7fc7d06db700 1 --2- 172.21.15.121:0/2866203579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7c8153c10 0x7fc7c80bcfe0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fc7c4009e60 tx=0x7fc7c4009780).stop 2023-11-15T02:59:05.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.166+0000 7fc7d06db700 1 -- 172.21.15.121:0/2866203579 shutdown_connections 2023-11-15T02:59:05.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.166+0000 7fc7d06db700 1 --2- 172.21.15.121:0/2866203579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7c8153c10 0x7fc7c80bcfe0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.166+0000 7fc7d06db700 1 -- 172.21.15.121:0/2866203579 >> 172.21.15.121:0/2866203579 conn(0x7fc7c80591c0 msgr2=0x7fc7c80c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:05.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.166+0000 7fc7d06db700 1 -- 172.21.15.121:0/2866203579 shutdown_connections 2023-11-15T02:59:05.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.166+0000 7fc7d06db700 1 -- 172.21.15.121:0/2866203579 wait complete. 2023-11-15T02:59:05.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.170+0000 7fc7d06db700 1 Processor -- start 2023-11-15T02:59:05.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.170+0000 7fc7d06db700 1 -- start start 2023-11-15T02:59:05.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.170+0000 7fc7d06db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7c80639b0 0x7fc7c8151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.170+0000 7fc7d06db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7c80bb6f0 0x7fc7c81640c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.170+0000 7fc7d06db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7c8153c10 0x7fc7c80bf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.170+0000 7fc7d06db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80bec10 con 0x7fc7c80639b0 2023-11-15T02:59:05.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.170+0000 7fc7d06db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8164980 con 0x7fc7c80bb6f0 2023-11-15T02:59:05.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.170+0000 7fc7d06db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8164c60 con 0x7fc7c8153c10 2023-11-15T02:59:05.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.174+0000 7fc7ce477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7c80639b0 0x7fc7c8151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.174+0000 7fc7ce477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7c80639b0 0x7fc7c8151370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51402/0 (socket says 172.21.15.121:51402) 2023-11-15T02:59:05.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.174+0000 7fc7ce477700 1 -- 172.21.15.121:0/2530730252 learned_addr learned my addr 172.21.15.121:0/2530730252 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:05.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.174+0000 7fc7cec78700 1 --2- 172.21.15.121:0/2530730252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7c8153c10 0x7fc7c80bf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.174+0000 7fc7cdc76700 1 --2- 172.21.15.121:0/2530730252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7c80bb6f0 0x7fc7c81640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.174+0000 7fc7ce477700 1 -- 172.21.15.121:0/2530730252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7c8153c10 msgr2=0x7fc7c80bf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.174+0000 7fc7ce477700 1 --2- 172.21.15.121:0/2530730252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7c8153c10 0x7fc7c80bf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.174+0000 7fc7ce477700 1 -- 172.21.15.121:0/2530730252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7c80bb6f0 msgr2=0x7fc7c81640c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.174+0000 7fc7ce477700 1 --2- 172.21.15.121:0/2530730252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7c80bb6f0 0x7fc7c81640c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.174+0000 7fc7ce477700 1 -- 172.21.15.121:0/2530730252 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c4009a20 con 0x7fc7c80639b0 2023-11-15T02:59:05.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.174+0000 7fc7ce477700 1 --2- 172.21.15.121:0/2530730252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7c80639b0 0x7fc7c8151370 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7fc7c400b6c0 tx=0x7fc7c4009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:05.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.174+0000 7fc7be7fc700 1 -- 172.21.15.121:0/2530730252 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4009460 con 0x7fc7c80639b0 2023-11-15T02:59:05.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.174+0000 7fc7be7fc700 1 -- 172.21.15.121:0/2530730252 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c4012e30 con 0x7fc7c80639b0 2023-11-15T02:59:05.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.174+0000 7fc7d06db700 1 -- 172.21.15.121:0/2530730252 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c8164f40 con 0x7fc7c80639b0 2023-11-15T02:59:05.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.174+0000 7fc7be7fc700 1 -- 172.21.15.121:0/2530730252 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c401d670 con 0x7fc7c80639b0 2023-11-15T02:59:05.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.174+0000 7fc7d06db700 1 -- 172.21.15.121:0/2530730252 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c8151860 con 0x7fc7c80639b0 2023-11-15T02:59:05.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.178+0000 7fc7be7fc700 1 -- 172.21.15.121:0/2530730252 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc7c401da20 con 0x7fc7c80639b0 2023-11-15T02:59:05.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.178+0000 7fc7be7fc700 1 --2- 172.21.15.121:0/2530730252 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc7ac05a510 0x7fc7ac05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.182+0000 7fc7be7fc700 1 -- 172.21.15.121:0/2530730252 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc7c40764e0 con 0x7fc7c80639b0 2023-11-15T02:59:05.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.182+0000 7fc7cdc76700 1 --2- 172.21.15.121:0/2530730252 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc7ac05a510 0x7fc7ac05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.182+0000 7fc7cdc76700 1 --2- 172.21.15.121:0/2530730252 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc7ac05a510 0x7fc7ac05c9d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fc7c8151c60 tx=0x7fc7b8007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:05.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.182+0000 7fc7daa7a740 1 Processor -- start 2023-11-15T02:59:05.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.182+0000 7fc7daa7a740 1 -- start start 2023-11-15T02:59:05.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.182+0000 7fc7daa7a740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1256a70 0x1213180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.182+0000 7fc7daa7a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1213990 0x1256e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.182+0000 7fc7daa7a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x120d150 0x120d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.182+0000 7fc7daa7a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1170540 con 0x1213990 2023-11-15T02:59:05.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.182+0000 7fc7daa7a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1133e90 con 0x120d150 2023-11-15T02:59:05.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.182+0000 7fc7daa7a740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1139d90 con 0x1256a70 2023-11-15T02:59:05.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.182+0000 7fc7cdc76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1213990 0x1256e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.182+0000 7fc7cec78700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x120d150 0x120d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.182+0000 7fc7ce477700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1256a70 0x1213180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.182+0000 7fc7cec78700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x120d150 0x120d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:58172/0 (socket says 172.21.15.121:58172) 2023-11-15T02:59:05.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.182+0000 7fc7cec78700 1 -- 172.21.15.121:0/78605633 learned_addr learned my addr 172.21.15.121:0/78605633 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:05.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.186+0000 7fc7cec78700 1 -- 172.21.15.121:0/78605633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1256a70 msgr2=0x1213180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:05.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.186+0000 7fc7cec78700 1 --2- 172.21.15.121:0/78605633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1256a70 0x1213180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.186+0000 7fc7cec78700 1 -- 172.21.15.121:0/78605633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1213990 msgr2=0x1256e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:05.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.186+0000 7fc7cec78700 1 --2- 172.21.15.121:0/78605633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1213990 0x1256e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.186+0000 7fc7cec78700 1 -- 172.21.15.121:0/78605633 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x119e4c0 con 0x120d150 2023-11-15T02:59:05.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.186+0000 7fc7cec78700 1 --2- 172.21.15.121:0/78605633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x120d150 0x120d5d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fc7c0004b70 tx=0x7fc7c0001a10).ready entity=mon.1 client_cookie=320a9e9b663c2214 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:05.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.186+0000 7fc7ab7fe700 1 -- 172.21.15.121:0/78605633 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c000f8d0 con 0x120d150 2023-11-15T02:59:05.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.186+0000 7fc7ab7fe700 1 -- 172.21.15.121:0/78605633 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c000fa30 con 0x120d150 2023-11-15T02:59:05.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.186+0000 7fc7ab7fe700 1 -- 172.21.15.121:0/78605633 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c000fdb0 con 0x120d150 2023-11-15T02:59:05.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.186+0000 7fc7daa7a740 1 -- 172.21.15.121:0/78605633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x120d150 msgr2=0x120d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.186+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/78605633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x120d150 0x120d5d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fc7c0004b70 tx=0x7fc7c0001a10).stop 2023-11-15T02:59:05.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.186+0000 7fc7daa7a740 1 -- 172.21.15.121:0/78605633 shutdown_connections 2023-11-15T02:59:05.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.186+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/78605633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x120d150 0x120d5d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.186+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/78605633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1213990 0x1256e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.186+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/78605633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1256a70 0x1213180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.186+0000 7fc7daa7a740 1 -- 172.21.15.121:0/78605633 >> 172.21.15.121:0/78605633 conn(0xff2680 msgr2=0x1211aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:05.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.186+0000 7fc7daa7a740 1 -- 172.21.15.121:0/78605633 shutdown_connections 2023-11-15T02:59:05.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.186+0000 7fc7daa7a740 1 -- 172.21.15.121:0/78605633 wait complete. 2023-11-15T02:59:05.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7daa7a740 1 -- start start 2023-11-15T02:59:05.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7daa7a740 1 Processor -- start 2023-11-15T02:59:05.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7daa7a740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x120d150 0x1250390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7daa7a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1213990 0x1251490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7daa7a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1256a70 0x1259d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7daa7a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1252680 con 0x1213990 2023-11-15T02:59:05.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7daa7a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x12530e0 con 0x1256a70 2023-11-15T02:59:05.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7daa7a740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1266160 con 0x120d150 2023-11-15T02:59:05.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7daa7a740 10 client.0 ll_register_callbacks cb 0x11ca5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:59:05.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7cdc76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1213990 0x1251490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7cdc76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1213990 0x1251490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51426/0 (socket says 172.21.15.121:51426) 2023-11-15T02:59:05.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7cdc76700 1 -- 172.21.15.121:0/308249183 learned_addr learned my addr 172.21.15.121:0/308249183 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:05.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7cec78700 1 --2- 172.21.15.121:0/308249183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1256a70 0x1259d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7ce477700 1 --2- 172.21.15.121:0/308249183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x120d150 0x1250390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7cdc76700 1 -- 172.21.15.121:0/308249183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x120d150 msgr2=0x1250390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:05.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7cdc76700 1 --2- 172.21.15.121:0/308249183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x120d150 0x1250390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7cdc76700 1 -- 172.21.15.121:0/308249183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1256a70 msgr2=0x1259d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:05.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7cdc76700 1 --2- 172.21.15.121:0/308249183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1256a70 0x1259d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7cdc76700 1 -- 172.21.15.121:0/308249183 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c0004770 con 0x1213990 2023-11-15T02:59:05.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7cdc76700 1 --2- 172.21.15.121:0/308249183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1213990 0x1251490 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fc7b8016470 tx=0x7fc7b800a000).ready entity=mon.0 client_cookie=8b1a3bff224b2ef0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:05.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:59:05.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7a8ff9700 1 -- 172.21.15.121:0/308249183 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b801c030 con 0x1213990 2023-11-15T02:59:05.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7a8ff9700 1 -- 172.21.15.121:0/308249183 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc7b8023c70 con 0x1213990 2023-11-15T02:59:05.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.190+0000 7fc7daa7a740 1 -- 172.21.15.121:0/308249183 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x118a230 con 0x1213990 2023-11-15T02:59:05.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7a8ff9700 1 -- 172.21.15.121:0/308249183 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7b8003040 con 0x1213990 2023-11-15T02:59:05.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7daa7a740 10 client.5667 fetch_fsmap learned FSMap version 30 2023-11-15T02:59:05.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7daa7a740 1 -- 172.21.15.121:0/308249183 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x118d7c0 con 0x1213990 2023-11-15T02:59:05.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7a8ff9700 1 -- 172.21.15.121:0/308249183 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x118a230 con 0x1213990 2023-11-15T02:59:05.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7a8ff9700 1 -- 172.21.15.121:0/308249183 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc7b8022410 con 0x1213990 2023-11-15T02:59:05.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7daa7a740 10 client.5667 fetch_fsmap finished waiting for FSMap version 30 2023-11-15T02:59:05.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7daa7a740 10 client.5667 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-15T02:59:05.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7daa7a740 20 client.5667 populate_metadata read hostname 'smithi121' 2023-11-15T02:59:05.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/308249183 >> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] conn(0x1253550 0x1253960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7daa7a740 4 client.5667 mds_command: new command op to 5514 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T02:59:05.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7daa7a740 1 -- 172.21.15.121:0/308249183 --> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1180780 con 0x1253550 2023-11-15T02:59:05.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7ce477700 1 --2- 172.21.15.121:0/308249183 >> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] conn(0x1253550 0x1253960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7ce477700 1 --2- 172.21.15.121:0/308249183 >> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] conn(0x1253550 0x1253960 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=29191d833b5d3e52 server_cookie=dcc109f797ea3c9e in_seq=0 out_seq=0 2023-11-15T02:59:05.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7a8ff9700 10 client.5667 ms_handle_connect on v2:172.21.15.121:6836/1945477051 2023-11-15T02:59:05.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7a8ff9700 1 -- 172.21.15.121:0/308249183 <== mds.0 v2:172.21.15.121:6836/1945477051 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x1180780 con 0x1253550 2023-11-15T02:59:05.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7a8ff9700 10 client.5667 handle_command_reply: tid=0 2023-11-15T02:59:05.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7daa7a740 1 client.5667 shutdown 2023-11-15T02:59:05.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7daa7a740 1 -- 172.21.15.121:0/308249183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1213990 msgr2=0x1251490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/308249183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1213990 0x1251490 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fc7b8016470 tx=0x7fc7b800a000).stop 2023-11-15T02:59:05.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7daa7a740 1 -- 172.21.15.121:0/308249183 shutdown_connections 2023-11-15T02:59:05.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/308249183 >> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] conn(0x1253550 0x1253960 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/308249183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1256a70 0x1259d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/308249183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1213990 0x1251490 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/308249183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x120d150 0x1250390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7a8ff9700 0 client.5667 ms_handle_reset on v2:172.21.15.121:6836/1945477051 2023-11-15T02:59:05.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7daa7a740 1 -- 172.21.15.121:0/308249183 >> 172.21.15.121:0/308249183 conn(0xff2680 msgr2=0x1211820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:05.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7daa7a740 1 -- 172.21.15.121:0/308249183 shutdown_connections 2023-11-15T02:59:05.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.194+0000 7fc7daa7a740 1 -- 172.21.15.121:0/308249183 wait complete. 2023-11-15T02:59:05.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.198+0000 7fc7daa7a740 20 client.5667 trim_cache size 0 max 16384 2023-11-15T02:59:05.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.210+0000 7fc7daa7a740 1 Processor -- start 2023-11-15T02:59:05.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.210+0000 7fc7daa7a740 1 -- start start 2023-11-15T02:59:05.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7daa7a740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x120d150 0x1268300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7daa7a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1213990 0x12584d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7daa7a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1256a70 0x12589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7daa7a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x123f180 con 0x1213990 2023-11-15T02:59:05.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7daa7a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1258eb0 con 0x1256a70 2023-11-15T02:59:05.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7daa7a740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1213760 con 0x120d150 2023-11-15T02:59:05.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7cdc76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1213990 0x12584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7cdc76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1213990 0x12584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51434/0 (socket says 172.21.15.121:51434) 2023-11-15T02:59:05.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7cdc76700 1 -- 172.21.15.121:0/1462474038 learned_addr learned my addr 172.21.15.121:0/1462474038 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:05.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7cdc76700 1 -- 172.21.15.121:0/1462474038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x120d150 msgr2=0x1268300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:05.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7cdc76700 1 --2- 172.21.15.121:0/1462474038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x120d150 0x1268300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T02:59:05.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7cdc76700 1 -- 172.21.15.121:0/1462474038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1256a70 msgr2=0x12589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:05.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7cdc76700 1 --2- 172.21.15.121:0/1462474038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1256a70 0x12589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T02:59:05.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7cdc76700 1 -- 172.21.15.121:0/1462474038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80c4b30 con 0x1213990 2023-11-15T02:59:05.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7cdc76700 1 --2- 172.21.15.121:0/1462474038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1213990 0x12584d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fc7b8007aa0 tx=0x7fc7b8020000).ready entity=mon.0 client_cookie=501d330f1992c81c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:05.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7abfff700 1 -- 172.21.15.121:0/1462474038 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b801c040 con 0x1213990 2023-11-15T02:59:05.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7abfff700 1 -- 172.21.15.121:0/1462474038 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b801d040 con 0x1213990 2023-11-15T02:59:05.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7abfff700 1 -- 172.21.15.121:0/1462474038 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b80228d0 con 0x1213990 2023-11-15T02:59:05.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7daa7a740 1 -- 172.21.15.121:0/1462474038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1213990 msgr2=0x12584d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/1462474038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1213990 0x12584d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fc7b8007aa0 tx=0x7fc7b8020000).stop 2023-11-15T02:59:05.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7daa7a740 1 -- 172.21.15.121:0/1462474038 shutdown_connections 2023-11-15T02:59:05.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/1462474038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1256a70 0x12589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T02:59:05.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/1462474038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1213990 0x12584d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/1462474038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x120d150 0x1268300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T02:59:05.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7daa7a740 1 -- 172.21.15.121:0/1462474038 >> 172.21.15.121:0/1462474038 conn(0xff2680 msgr2=0x12560b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:05.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7daa7a740 1 -- 172.21.15.121:0/1462474038 shutdown_connections 2023-11-15T02:59:05.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7daa7a740 1 -- 172.21.15.121:0/1462474038 wait complete. 2023-11-15T02:59:05.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7daa7a740 1 -- start start 2023-11-15T02:59:05.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.214+0000 7fc7daa7a740 1 Processor -- start 2023-11-15T02:59:05.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7daa7a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x120d150 0x1273f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7daa7a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1256a70 0x1274400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7daa7a740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1240460 0x123b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7daa7a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x12748f0 con 0x1256a70 2023-11-15T02:59:05.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7daa7a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1277a60 con 0x120d150 2023-11-15T02:59:05.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7daa7a740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1276290 con 0x1240460 2023-11-15T02:59:05.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7daa7a740 10 client.0 ll_register_callbacks cb 0x11ca5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:59:05.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7cdc76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1256a70 0x1274400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7cdc76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1256a70 0x1274400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51450/0 (socket says 172.21.15.121:51450) 2023-11-15T02:59:05.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7cdc76700 1 -- 172.21.15.121:0/1188714811 learned_addr learned my addr 172.21.15.121:0/1188714811 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:05.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7ce477700 1 --2- 172.21.15.121:0/1188714811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x120d150 0x1273f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7cec78700 1 --2- 172.21.15.121:0/1188714811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1240460 0x123b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7cdc76700 1 -- 172.21.15.121:0/1188714811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1240460 msgr2=0x123b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:05.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7cdc76700 1 --2- 172.21.15.121:0/1188714811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1240460 0x123b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7cdc76700 1 -- 172.21.15.121:0/1188714811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x120d150 msgr2=0x1273f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:05.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7cdc76700 1 --2- 172.21.15.121:0/1188714811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x120d150 0x1273f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7cdc76700 1 -- 172.21.15.121:0/1188714811 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x118d7c0 con 0x1256a70 2023-11-15T02:59:05.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7ce477700 1 --2- 172.21.15.121:0/1188714811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x120d150 0x1273f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:05.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7cec78700 1 --2- 172.21.15.121:0/1188714811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1240460 0x123b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:05.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7cdc76700 1 --2- 172.21.15.121:0/1188714811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1256a70 0x1274400 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fc7b8016470 tx=0x7fc7b8018040).ready entity=mon.0 client_cookie=143d0cc0972c878e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:05.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:59:05.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7a8ff9700 1 -- 172.21.15.121:0/1188714811 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b801c040 con 0x1256a70 2023-11-15T02:59:05.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7daa7a740 1 -- 172.21.15.121:0/1188714811 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7c000fa30 con 0x1256a70 2023-11-15T02:59:05.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7a8ff9700 1 -- 172.21.15.121:0/1188714811 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc7b8035470 con 0x1256a70 2023-11-15T02:59:05.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7a8ff9700 1 -- 172.21.15.121:0/1188714811 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7b8035840 con 0x1256a70 2023-11-15T02:59:05.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7daa7a740 10 client.5673 fetch_fsmap learned FSMap version 30 2023-11-15T02:59:05.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7daa7a740 1 -- 172.21.15.121:0/1188714811 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x11889f0 con 0x1256a70 2023-11-15T02:59:05.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7a8ff9700 1 -- 172.21.15.121:0/1188714811 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c000fa30 con 0x1256a70 2023-11-15T02:59:05.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.218+0000 7fc7a8ff9700 1 -- 172.21.15.121:0/1188714811 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc7b80359a0 con 0x1256a70 2023-11-15T02:59:05.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.222+0000 7fc7daa7a740 10 client.5673 fetch_fsmap finished waiting for FSMap version 30 2023-11-15T02:59:05.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.222+0000 7fc7daa7a740 10 client.5673 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-15T02:59:05.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.222+0000 7fc7daa7a740 20 client.5673 populate_metadata read hostname 'smithi121' 2023-11-15T02:59:05.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.222+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/1188714811 >> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] conn(0x126fcb0 0x123bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.222+0000 7fc7daa7a740 4 client.5673 mds_command: new command op to 5514 tid=0[{"prefix": "flush journal"}] 2023-11-15T02:59:05.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.222+0000 7fc7daa7a740 1 -- 172.21.15.121:0/1188714811 --> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x1049610 con 0x126fcb0 2023-11-15T02:59:05.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.222+0000 7fc7ce477700 1 --2- 172.21.15.121:0/1188714811 >> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] conn(0x126fcb0 0x123bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.222+0000 7fc7a8ff9700 10 client.5673 ms_handle_connect on v2:172.21.15.121:6836/1945477051 2023-11-15T02:59:05.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.222+0000 7fc7ce477700 1 --2- 172.21.15.121:0/1188714811 >> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] conn(0x126fcb0 0x123bdc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=6e88846b4f3b0333 server_cookie=7d000106228867d8 in_seq=0 out_seq=0 2023-11-15T02:59:05.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7a8ff9700 1 -- 172.21.15.121:0/1188714811 <== mds.0 v2:172.21.15.121:6836/1945477051 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x1049610 con 0x126fcb0 2023-11-15T02:59:05.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7a8ff9700 10 client.5673 handle_command_reply: tid=0 2023-11-15T02:59:05.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7daa7a740 1 client.5673 shutdown 2023-11-15T02:59:05.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7daa7a740 1 -- 172.21.15.121:0/1188714811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1256a70 msgr2=0x1274400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/1188714811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1256a70 0x1274400 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fc7b8016470 tx=0x7fc7b8018040).stop 2023-11-15T02:59:05.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7daa7a740 1 -- 172.21.15.121:0/1188714811 shutdown_connections 2023-11-15T02:59:05.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/1188714811 >> [v2:172.21.15.121:6836/1945477051,v1:172.21.15.121:6837/1945477051] conn(0x126fcb0 0x123bdc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/1188714811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1240460 0x123b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/1188714811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1256a70 0x1274400 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7a8ff9700 0 client.5673 ms_handle_reset on v2:172.21.15.121:6836/1945477051 2023-11-15T02:59:05.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7daa7a740 1 --2- 172.21.15.121:0/1188714811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x120d150 0x1273f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7daa7a740 1 -- 172.21.15.121:0/1188714811 >> 172.21.15.121:0/1188714811 conn(0xff2680 msgr2=0x12560b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:05.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7daa7a740 1 -- 172.21.15.121:0/1188714811 shutdown_connections 2023-11-15T02:59:05.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7daa7a740 1 -- 172.21.15.121:0/1188714811 wait complete. 2023-11-15T02:59:05.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7daa7a740 20 client.5673 trim_cache size 0 max 16384 2023-11-15T02:59:05.248 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T02:59:05.248 INFO:teuthology.orchestra.run.smithi121.stdout: "message": "", 2023-11-15T02:59:05.248 INFO:teuthology.orchestra.run.smithi121.stdout: "return_code": 0 2023-11-15T02:59:05.249 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T02:59:05.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7d06db700 1 -- 172.21.15.121:0/2530730252 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc7ac05a510 msgr2=0x7fc7ac05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7d06db700 1 --2- 172.21.15.121:0/2530730252 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc7ac05a510 0x7fc7ac05c9d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fc7c8151c60 tx=0x7fc7b8007d40).stop 2023-11-15T02:59:05.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7d06db700 1 -- 172.21.15.121:0/2530730252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7c80639b0 msgr2=0x7fc7c8151370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7d06db700 1 --2- 172.21.15.121:0/2530730252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7c80639b0 0x7fc7c8151370 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7fc7c400b6c0 tx=0x7fc7c4009780).stop 2023-11-15T02:59:05.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7d06db700 1 -- 172.21.15.121:0/2530730252 shutdown_connections 2023-11-15T02:59:05.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7d06db700 1 --2- 172.21.15.121:0/2530730252 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc7ac05a510 0x7fc7ac05c9d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7d06db700 1 --2- 172.21.15.121:0/2530730252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7c8153c10 0x7fc7c80bf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7d06db700 1 --2- 172.21.15.121:0/2530730252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7c80bb6f0 0x7fc7c81640c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7d06db700 1 --2- 172.21.15.121:0/2530730252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7c80639b0 0x7fc7c8151370 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7d06db700 1 -- 172.21.15.121:0/2530730252 >> 172.21.15.121:0/2530730252 conn(0x7fc7c80591c0 msgr2=0x7fc7c80ab770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:05.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7d06db700 1 -- 172.21.15.121:0/2530730252 shutdown_connections 2023-11-15T02:59:05.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.238+0000 7fc7d06db700 1 -- 172.21.15.121:0/2530730252 wait complete. 2023-11-15T02:59:05.265 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_abort_on_newly_corrupt_dentry False 2023-11-15T02:59:05.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.422+0000 7ff4b6ce4700 1 Processor -- start 2023-11-15T02:59:05.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.422+0000 7ff4b6ce4700 1 -- start start 2023-11-15T02:59:05.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.426+0000 7ff4b6ce4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b00a8960 0x7ff4b00c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.426+0000 7ff4b6ce4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff4b00a8d40 0x7ff4b015c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.426+0000 7ff4b6ce4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b00657f0 con 0x7ff4b0066520 2023-11-15T02:59:05.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.426+0000 7ff4b6ce4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b01583d0 con 0x7ff4b00a8d40 2023-11-15T02:59:05.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.426+0000 7ff4b6ce4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00c1b60 con 0x7ff4b00a8960 2023-11-15T02:59:05.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.426+0000 7ff4b54e1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff4b00a8d40 0x7ff4b015c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.426+0000 7ff4b54e1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff4b00a8d40 0x7ff4b015c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:58204/0 (socket says 172.21.15.121:58204) 2023-11-15T02:59:05.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.426+0000 7ff4b54e1700 1 -- 172.21.15.121:0/1828772874 learned_addr learned my addr 172.21.15.121:0/1828772874 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:05.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.426+0000 7ff4b64e3700 1 --1- 172.21.15.121:0/1828772874 >> v1:172.21.15.190:6789/0 conn(0x7ff4b0066520 0x7ff4b00c2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:47836/0 (socket says 172.21.15.121:47836) 2023-11-15T02:59:05.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.426+0000 7ff4b5ce2700 1 --2- 172.21.15.121:0/1828772874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b00a8960 0x7ff4b00c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.426+0000 7ff4b5ce2700 1 -- 172.21.15.121:0/1828772874 >> v2:172.21.15.190:3300/0 conn(0x7ff4b00a8d40 msgr2=0x7ff4b015c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:05.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.426+0000 7ff4b5ce2700 1 --2- 172.21.15.121:0/1828772874 >> v2:172.21.15.190:3300/0 conn(0x7ff4b00a8d40 0x7ff4b015c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.426+0000 7ff4b5ce2700 1 -- 172.21.15.121:0/1828772874 >> v1:172.21.15.190:6789/0 conn(0x7ff4b0066520 legacy=0x7ff4b00c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.426+0000 7ff4b5ce2700 1 -- 172.21.15.121:0/1828772874 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b0065a40 con 0x7ff4b00a8960 2023-11-15T02:59:05.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.426+0000 7ff4b4ce0700 1 -- 172.21.15.121:0/1828772874 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3396817995 0 0) 0x7ff4b00657f0 con 0x7ff4b0066520 2023-11-15T02:59:05.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.426+0000 7ff4b54e1700 1 --2- 172.21.15.121:0/1828772874 >> v2:172.21.15.190:3300/0 conn(0x7ff4b00a8d40 0x7ff4b015c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:59:05.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.426+0000 7ff4b5ce2700 1 --2- 172.21.15.121:0/1828772874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b00a8960 0x7ff4b00c15c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7ff4ac009760 tx=0x7ff4ac009060).ready entity=mon.2 client_cookie=abd3c236079a830d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:05.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.430+0000 7ff4b4ce0700 1 -- 172.21.15.121:0/1828772874 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac00f510 con 0x7ff4b00a8960 2023-11-15T02:59:05.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.430+0000 7ff4b4ce0700 1 -- 172.21.15.121:0/1828772874 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac00f670 con 0x7ff4b00a8960 2023-11-15T02:59:05.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.430+0000 7ff4b4ce0700 1 -- 172.21.15.121:0/1828772874 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac00f9f0 con 0x7ff4b00a8960 2023-11-15T02:59:05.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.430+0000 7ff4b6ce4700 1 -- 172.21.15.121:0/1828772874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b00a8960 msgr2=0x7ff4b00c15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.430+0000 7ff4b6ce4700 1 --2- 172.21.15.121:0/1828772874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b00a8960 0x7ff4b00c15c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7ff4ac009760 tx=0x7ff4ac009060).stop 2023-11-15T02:59:05.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.430+0000 7ff4b6ce4700 1 -- 172.21.15.121:0/1828772874 shutdown_connections 2023-11-15T02:59:05.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.430+0000 7ff4b6ce4700 1 --2- 172.21.15.121:0/1828772874 >> v2:172.21.15.190:3300/0 conn(0x7ff4b00a8d40 0x7ff4b015c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.430+0000 7ff4b6ce4700 1 --2- 172.21.15.121:0/1828772874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b00a8960 0x7ff4b00c15c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.430+0000 7ff4b6ce4700 1 -- 172.21.15.121:0/1828772874 >> 172.21.15.121:0/1828772874 conn(0x7ff4b0067a60 msgr2=0x7ff4b0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:05.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.430+0000 7ff4b6ce4700 1 -- 172.21.15.121:0/1828772874 shutdown_connections 2023-11-15T02:59:05.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.430+0000 7ff4b6ce4700 1 -- 172.21.15.121:0/1828772874 wait complete. 2023-11-15T02:59:05.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.430+0000 7ff4b6ce4700 1 Processor -- start 2023-11-15T02:59:05.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.430+0000 7ff4b6ce4700 1 -- start start 2023-11-15T02:59:05.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4b6ce4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4b0066520 0x7ff4b0168a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4b6ce4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b00a8960 0x7ff4b015cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4b6ce4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b00a8d40 0x7ff4b015d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4b6ce4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b0159850 con 0x7ff4b0066520 2023-11-15T02:59:05.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4b6ce4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b015d900 con 0x7ff4b00a8960 2023-11-15T02:59:05.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4b6ce4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b015dbe0 con 0x7ff4b00a8d40 2023-11-15T02:59:05.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4b5ce2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4b0066520 0x7ff4b0168a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4b5ce2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4b0066520 0x7ff4b0168a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51452/0 (socket says 172.21.15.121:51452) 2023-11-15T02:59:05.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4b5ce2700 1 -- 172.21.15.121:0/218683399 learned_addr learned my addr 172.21.15.121:0/218683399 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:05.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4b64e3700 1 --2- 172.21.15.121:0/218683399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b00a8d40 0x7ff4b015d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4b54e1700 1 --2- 172.21.15.121:0/218683399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b00a8960 0x7ff4b015cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4b5ce2700 1 -- 172.21.15.121:0/218683399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b00a8d40 msgr2=0x7ff4b015d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4b5ce2700 1 --2- 172.21.15.121:0/218683399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b00a8d40 0x7ff4b015d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4b5ce2700 1 -- 172.21.15.121:0/218683399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b00a8960 msgr2=0x7ff4b015cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4b5ce2700 1 --2- 172.21.15.121:0/218683399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b00a8960 0x7ff4b015cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4b5ce2700 1 -- 172.21.15.121:0/218683399 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a0009370 con 0x7ff4b0066520 2023-11-15T02:59:05.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4b5ce2700 1 --2- 172.21.15.121:0/218683399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4b0066520 0x7ff4b0168a80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7ff4b00c1ca0 tx=0x7ff4ac017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:05.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4a5ffb700 1 -- 172.21.15.121:0/218683399 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac01a4d0 con 0x7ff4b0066520 2023-11-15T02:59:05.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4a5ffb700 1 -- 172.21.15.121:0/218683399 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac01e070 con 0x7ff4b0066520 2023-11-15T02:59:05.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4b6ce4700 1 -- 172.21.15.121:0/218683399 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4ac0092a0 con 0x7ff4b0066520 2023-11-15T02:59:05.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4a5ffb700 1 -- 172.21.15.121:0/218683399 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac01a8a0 con 0x7ff4b0066520 2023-11-15T02:59:05.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.434+0000 7ff4b6ce4700 1 -- 172.21.15.121:0/218683399 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b015e1a0 con 0x7ff4b0066520 2023-11-15T02:59:05.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.438+0000 7ff4b6ce4700 1 -- 172.21.15.121:0/218683399 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b006bcf0 con 0x7ff4b0066520 2023-11-15T02:59:05.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.438+0000 7ff4a5ffb700 1 -- 172.21.15.121:0/218683399 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff4ac020030 con 0x7ff4b0066520 2023-11-15T02:59:05.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.442+0000 7ff4a5ffb700 1 --2- 172.21.15.121:0/218683399 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff49405a510 0x7ff49405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.442+0000 7ff4a5ffb700 1 -- 172.21.15.121:0/218683399 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff4ac075710 con 0x7ff4b0066520 2023-11-15T02:59:05.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.442+0000 7ff4b54e1700 1 --2- 172.21.15.121:0/218683399 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff49405a510 0x7ff49405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.442+0000 7ff4b54e1700 1 --2- 172.21.15.121:0/218683399 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff49405a510 0x7ff49405c9d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7ff4b00c0470 tx=0x7ff4a00090f0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:05.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.446+0000 7ff4a5ffb700 1 -- 172.21.15.121:0/218683399 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff4ac048ef0 con 0x7ff4b0066520 2023-11-15T02:59:05.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.610+0000 7ff4b6ce4700 1 -- 172.21.15.121:0/218683399 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}] v 0) v1 -- 0x7ff4b0068a70 con 0x7ff4b0066520 2023-11-15T02:59:05.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.622+0000 7ff4a5ffb700 1 -- 172.21.15.121:0/218683399 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}]=0 v5)=0 v5) v1 ==== 135+0+0 (secure 0 0 0) 0x7ff4ac028020 con 0x7ff4b0066520 2023-11-15T02:59:05.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.626+0000 7ff4b6ce4700 1 -- 172.21.15.121:0/218683399 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff49405a510 msgr2=0x7ff49405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.626+0000 7ff4b6ce4700 1 --2- 172.21.15.121:0/218683399 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff49405a510 0x7ff49405c9d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7ff4b00c0470 tx=0x7ff4a00090f0).stop 2023-11-15T02:59:05.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.626+0000 7ff4b6ce4700 1 -- 172.21.15.121:0/218683399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4b0066520 msgr2=0x7ff4b0168a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.626+0000 7ff4b6ce4700 1 --2- 172.21.15.121:0/218683399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4b0066520 0x7ff4b0168a80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7ff4b00c1ca0 tx=0x7ff4ac017000).stop 2023-11-15T02:59:05.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.626+0000 7ff4b6ce4700 1 -- 172.21.15.121:0/218683399 shutdown_connections 2023-11-15T02:59:05.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.626+0000 7ff4b6ce4700 1 --2- 172.21.15.121:0/218683399 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff49405a510 0x7ff49405c9d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.626+0000 7ff4b6ce4700 1 --2- 172.21.15.121:0/218683399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4b00a8d40 0x7ff4b015d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.626+0000 7ff4b6ce4700 1 --2- 172.21.15.121:0/218683399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b00a8960 0x7ff4b015cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.626+0000 7ff4b6ce4700 1 --2- 172.21.15.121:0/218683399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4b0066520 0x7ff4b0168a80 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.626+0000 7ff4b6ce4700 1 -- 172.21.15.121:0/218683399 >> 172.21.15.121:0/218683399 conn(0x7ff4b0067a60 msgr2=0x7ff4b00a7350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:05.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.626+0000 7ff4b6ce4700 1 -- 172.21.15.121:0/218683399 shutdown_connections 2023-11-15T02:59:05.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.626+0000 7ff4b6ce4700 1 -- 172.21.15.121:0/218683399 wait complete. 2023-11-15T02:59:05.656 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_rename_corrupt_dentry_first 1.0 2023-11-15T02:59:05.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.814+0000 7fa138907700 1 Processor -- start 2023-11-15T02:59:05.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.814+0000 7fa138907700 1 -- start start 2023-11-15T02:59:05.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.818+0000 7fa138907700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa13414f2a0 0x7fa1340a8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.818+0000 7fa138907700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa134155ba0 0x7fa1340c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.818+0000 7fa138907700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa134152ab0 con 0x7fa13414f680 2023-11-15T02:59:05.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.818+0000 7fa138907700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa134152180 con 0x7fa134155ba0 2023-11-15T02:59:05.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.818+0000 7fa138907700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa134152460 con 0x7fa13414f2a0 2023-11-15T02:59:05.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.818+0000 7fa133fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa134155ba0 0x7fa1340c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.818+0000 7fa1337fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa13414f2a0 0x7fa1340a8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.818+0000 7fa132ffd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fa13414f680 0x7fa134155490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:47842/0 (socket says 172.21.15.121:47842) 2023-11-15T02:59:05.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.818+0000 7fa132ffd700 1 -- 172.21.15.121:0/108442282 learned_addr learned my addr 172.21.15.121:0/108442282 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:05.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.818+0000 7fa133fff700 1 -- 172.21.15.121:0/108442282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa13414f2a0 msgr2=0x7fa1340a8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:05.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.818+0000 7fa133fff700 1 --2- 172.21.15.121:0/108442282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa13414f2a0 0x7fa1340a8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.818+0000 7fa133fff700 1 -- 172.21.15.121:0/108442282 >> v1:172.21.15.190:6789/0 conn(0x7fa13414f680 legacy=0x7fa134155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.818+0000 7fa133fff700 1 -- 172.21.15.121:0/108442282 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa134152740 con 0x7fa134155ba0 2023-11-15T02:59:05.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.818+0000 7fa133fff700 1 --2- 172.21.15.121:0/108442282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa134155ba0 0x7fa1340c2f40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7fa12c009e60 tx=0x7fa12c009780).ready entity=mon.0 client_cookie=933bddbf31c938c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:05.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.818+0000 7fa1327fc700 1 -- 172.21.15.121:0/108442282 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c00e070 con 0x7fa134155ba0 2023-11-15T02:59:05.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.818+0000 7fa1327fc700 1 -- 172.21.15.121:0/108442282 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12c00bb60 con 0x7fa134155ba0 2023-11-15T02:59:05.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.818+0000 7fa1327fc700 1 -- 172.21.15.121:0/108442282 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c012410 con 0x7fa134155ba0 2023-11-15T02:59:05.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.822+0000 7fa138907700 1 -- 172.21.15.121:0/108442282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa134155ba0 msgr2=0x7fa1340c2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.822+0000 7fa138907700 1 --2- 172.21.15.121:0/108442282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa134155ba0 0x7fa1340c2f40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7fa12c009e60 tx=0x7fa12c009780).stop 2023-11-15T02:59:05.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.822+0000 7fa138907700 1 -- 172.21.15.121:0/108442282 shutdown_connections 2023-11-15T02:59:05.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.822+0000 7fa138907700 1 --2- 172.21.15.121:0/108442282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa134155ba0 0x7fa1340c2f40 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.822+0000 7fa138907700 1 --2- 172.21.15.121:0/108442282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa13414f2a0 0x7fa1340a8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.822+0000 7fa138907700 1 -- 172.21.15.121:0/108442282 >> 172.21.15.121:0/108442282 conn(0x7fa134067a60 msgr2=0x7fa134067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:05.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.822+0000 7fa138907700 1 -- 172.21.15.121:0/108442282 shutdown_connections 2023-11-15T02:59:05.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.822+0000 7fa138907700 1 -- 172.21.15.121:0/108442282 wait complete. 2023-11-15T02:59:05.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.822+0000 7fa138907700 1 Processor -- start 2023-11-15T02:59:05.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.822+0000 7fa138907700 1 -- start start 2023-11-15T02:59:05.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa138907700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa13414f2a0 0x7fa1340aab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa138907700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa134155ba0 0x7fa1340c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa138907700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa13414c2b0 0x7fa13414c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa138907700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13414bd20 con 0x7fa134155ba0 2023-11-15T02:59:05.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa138907700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340c6820 con 0x7fa13414c2b0 2023-11-15T02:59:05.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa138907700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340c5c70 con 0x7fa13414f2a0 2023-11-15T02:59:05.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa132ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa134155ba0 0x7fa1340c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa132ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa134155ba0 0x7fa1340c6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51466/0 (socket says 172.21.15.121:51466) 2023-11-15T02:59:05.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa132ffd700 1 -- 172.21.15.121:0/3499811556 learned_addr learned my addr 172.21.15.121:0/3499811556 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:05.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa133fff700 1 --2- 172.21.15.121:0/3499811556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa13414c2b0 0x7fa13414c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa1337fe700 1 --2- 172.21.15.121:0/3499811556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa13414f2a0 0x7fa1340aab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa132ffd700 1 -- 172.21.15.121:0/3499811556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa13414f2a0 msgr2=0x7fa1340aab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa132ffd700 1 --2- 172.21.15.121:0/3499811556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa13414f2a0 0x7fa1340aab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa132ffd700 1 -- 172.21.15.121:0/3499811556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa13414c2b0 msgr2=0x7fa13414c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:05.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa132ffd700 1 --2- 172.21.15.121:0/3499811556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa13414c2b0 0x7fa13414c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:05.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa132ffd700 1 -- 172.21.15.121:0/3499811556 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c009a20 con 0x7fa134155ba0 2023-11-15T02:59:05.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa132ffd700 1 --2- 172.21.15.121:0/3499811556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa134155ba0 0x7fa1340c6cb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7fa12800c410 tx=0x7fa128007d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:05.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa11b7fe700 1 -- 172.21.15.121:0/3499811556 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1280070d0 con 0x7fa134155ba0 2023-11-15T02:59:05.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa11b7fe700 1 -- 172.21.15.121:0/3499811556 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa128007230 con 0x7fa134155ba0 2023-11-15T02:59:05.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa11b7fe700 1 -- 172.21.15.121:0/3499811556 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1280075b0 con 0x7fa134155ba0 2023-11-15T02:59:05.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa138907700 1 -- 172.21.15.121:0/3499811556 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13414cc20 con 0x7fa134155ba0 2023-11-15T02:59:05.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.826+0000 7fa138907700 1 -- 172.21.15.121:0/3499811556 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1340c5ef0 con 0x7fa134155ba0 2023-11-15T02:59:05.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.830+0000 7fa138907700 1 -- 172.21.15.121:0/3499811556 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1340c69d0 con 0x7fa134155ba0 2023-11-15T02:59:05.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.830+0000 7fa11b7fe700 1 -- 172.21.15.121:0/3499811556 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa1280059b0 con 0x7fa134155ba0 2023-11-15T02:59:05.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.830+0000 7fa11b7fe700 1 --2- 172.21.15.121:0/3499811556 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa12005a510 0x7fa12005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:05.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.830+0000 7fa11b7fe700 1 -- 172.21.15.121:0/3499811556 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fa128073dd0 con 0x7fa134155ba0 2023-11-15T02:59:05.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.830+0000 7fa1337fe700 1 --2- 172.21.15.121:0/3499811556 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa12005a510 0x7fa12005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:05.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.834+0000 7fa1337fe700 1 --2- 172.21.15.121:0/3499811556 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa12005a510 0x7fa12005c9d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fa124009c30 tx=0x7fa124009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:05.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:05.838+0000 7fa11b7fe700 1 -- 172.21.15.121:0/3499811556 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa1340c69d0 con 0x7fa134155ba0 2023-11-15T02:59:06.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:06.002+0000 7fa138907700 1 -- 172.21.15.121:0/3499811556 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}] v 0) v1 -- 0x7fa1340c69d0 con 0x7fa134155ba0 2023-11-15T02:59:06.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:06.010+0000 7fa11b7fe700 1 -- 172.21.15.121:0/3499811556 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}]=0 v6)=0 v6) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa1340c69d0 con 0x7fa134155ba0 2023-11-15T02:59:06.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:06.014+0000 7fa138907700 1 -- 172.21.15.121:0/3499811556 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa12005a510 msgr2=0x7fa12005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:06.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:06.014+0000 7fa138907700 1 --2- 172.21.15.121:0/3499811556 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa12005a510 0x7fa12005c9d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fa124009c30 tx=0x7fa124009060).stop 2023-11-15T02:59:06.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:06.014+0000 7fa138907700 1 -- 172.21.15.121:0/3499811556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa134155ba0 msgr2=0x7fa1340c6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:06.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:06.014+0000 7fa138907700 1 --2- 172.21.15.121:0/3499811556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa134155ba0 0x7fa1340c6cb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7fa12800c410 tx=0x7fa128007d50).stop 2023-11-15T02:59:06.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:06.014+0000 7fa138907700 1 -- 172.21.15.121:0/3499811556 shutdown_connections 2023-11-15T02:59:06.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:06.014+0000 7fa138907700 1 --2- 172.21.15.121:0/3499811556 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa12005a510 0x7fa12005c9d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:06.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:06.014+0000 7fa138907700 1 --2- 172.21.15.121:0/3499811556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa13414c2b0 0x7fa13414c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:06.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:06.014+0000 7fa138907700 1 --2- 172.21.15.121:0/3499811556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa134155ba0 0x7fa1340c6cb0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:06.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:06.014+0000 7fa138907700 1 --2- 172.21.15.121:0/3499811556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa13414f2a0 0x7fa1340aab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:06.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:06.014+0000 7fa138907700 1 -- 172.21.15.121:0/3499811556 >> 172.21.15.121:0/3499811556 conn(0x7fa134067a60 msgr2=0x7fa13414e580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:06.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:06.014+0000 7fa138907700 1 -- 172.21.15.121:0/3499811556 shutdown_connections 2023-11-15T02:59:06.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:06.014+0000 7fa138907700 1 -- 172.21.15.121:0/3499811556 wait complete. 2023-11-15T02:59:10.302 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T02:59:10.310 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T02:59:11.042 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:59:11.046 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'mv a/b a/c; sync .') 2023-11-15T02:59:11.077 INFO:tasks.ceph.mds.c.smithi121.stderr:2023-11-15T02:59:11.074+0000 7feaac66c700 -1 mds.0.cache.den(0x10000000000 c) newly corrupt dentry to be committed: [dentry #0x1/a/c [fffffffffffffff6,head] auth NULL (dn xlock x=1 by 0x5640eb7fac00) (dversion lock w=1 last_client=5608) pv=4 v=3 ap=2 ino=(nil) state=1073741888|bottomlru | request=1 lock=2 authpin=1 0x5640eb7e7900] 2023-11-15T02:59:11.099 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T02:59:11.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:59:11.159 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-15T02:59:11.216 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T02:59:11.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:11.217 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T02:59:11.234 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:59:11.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.434+0000 7f6939d69700 1 Processor -- start 2023-11-15T02:59:11.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.434+0000 7f6939d69700 1 -- start start 2023-11-15T02:59:11.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.434+0000 7f6939d69700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6934153bb0 0x7f69340a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:11.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.434+0000 7f6939d69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69340aa020 0x7f6934067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:11.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.434+0000 7f6939d69700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6934067f50 0x7f69341578a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:11.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.434+0000 7f6939d69700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69340aa6f0 con 0x7f6934067f50 2023-11-15T02:59:11.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.434+0000 7f6939d69700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69340c5900 con 0x7f6934153bb0 2023-11-15T02:59:11.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.434+0000 7f6939d69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69340cb5c0 con 0x7f69340aa020 2023-11-15T02:59:11.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.434+0000 7f6933fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6934067f50 0x7f69341578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:11.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6933fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6934067f50 0x7f69341578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51482/0 (socket says 172.21.15.121:51482) 2023-11-15T02:59:11.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6933fff700 1 -- 172.21.15.121:0/79791032 learned_addr learned my addr 172.21.15.121:0/79791032 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:11.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f69337fe700 1 --2- 172.21.15.121:0/79791032 >> v2:172.21.15.190:3300/0 conn(0x7f6934153bb0 0x7f69340a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:11.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f692affd700 1 --2- 172.21.15.121:0/79791032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69340aa020 0x7f6934067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:11.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6933fff700 1 -- 172.21.15.121:0/79791032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69340aa020 msgr2=0x7f6934067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:11.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6933fff700 1 --2- 172.21.15.121:0/79791032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69340aa020 0x7f6934067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:11.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6933fff700 1 -- 172.21.15.121:0/79791032 >> v2:172.21.15.190:3300/0 conn(0x7f6934153bb0 msgr2=0x7f69340a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:11.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6933fff700 1 --2- 172.21.15.121:0/79791032 >> v2:172.21.15.190:3300/0 conn(0x7f6934153bb0 0x7f69340a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:11.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6933fff700 1 -- 172.21.15.121:0/79791032 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69340683e0 con 0x7f6934067f50 2023-11-15T02:59:11.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6933fff700 1 --2- 172.21.15.121:0/79791032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6934067f50 0x7f69341578a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f6934069f00 tx=0x7f692c0090f0).ready entity=mon.0 client_cookie=b87372d0652477c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:11.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6932ffd700 1 -- 172.21.15.121:0/79791032 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692c00b520 con 0x7f6934067f50 2023-11-15T02:59:11.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6932ffd700 1 -- 172.21.15.121:0/79791032 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692c00b680 con 0x7f6934067f50 2023-11-15T02:59:11.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6932ffd700 1 -- 172.21.15.121:0/79791032 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692c00ba00 con 0x7f6934067f50 2023-11-15T02:59:11.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6939d69700 1 -- 172.21.15.121:0/79791032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6934067f50 msgr2=0x7f69341578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:11.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6939d69700 1 --2- 172.21.15.121:0/79791032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6934067f50 0x7f69341578a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f6934069f00 tx=0x7f692c0090f0).stop 2023-11-15T02:59:11.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6939d69700 1 -- 172.21.15.121:0/79791032 shutdown_connections 2023-11-15T02:59:11.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6939d69700 1 --2- 172.21.15.121:0/79791032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6934067f50 0x7f69341578a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:11.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6939d69700 1 --2- 172.21.15.121:0/79791032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69340aa020 0x7f6934067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:11.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6939d69700 1 --2- 172.21.15.121:0/79791032 >> v2:172.21.15.190:3300/0 conn(0x7f6934153bb0 0x7f69340a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:11.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6939d69700 1 -- 172.21.15.121:0/79791032 >> 172.21.15.121:0/79791032 conn(0x7f6934058c50 msgr2=0x7f6934059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:11.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6939d69700 1 -- 172.21.15.121:0/79791032 shutdown_connections 2023-11-15T02:59:11.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.438+0000 7f6939d69700 1 -- 172.21.15.121:0/79791032 wait complete. 2023-11-15T02:59:11.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.442+0000 7f6939d69700 1 Processor -- start 2023-11-15T02:59:11.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.442+0000 7f6939d69700 1 -- start start 2023-11-15T02:59:11.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.442+0000 7f6939d69700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6934067f50 0x7f6934152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:11.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.442+0000 7f6939d69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69340aa020 0x7f69341542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:11.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.442+0000 7f6939d69700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6934153bb0 0x7f69340c1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:11.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.442+0000 7f6939d69700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6934155830 con 0x7f6934067f50 2023-11-15T02:59:11.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.442+0000 7f6939d69700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6934154c20 con 0x7f6934153bb0 2023-11-15T02:59:11.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.442+0000 7f6939d69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6934154f00 con 0x7f69340aa020 2023-11-15T02:59:11.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.442+0000 7f69337fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6934067f50 0x7f6934152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:11.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.442+0000 7f69337fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6934067f50 0x7f6934152420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51488/0 (socket says 172.21.15.121:51488) 2023-11-15T02:59:11.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.442+0000 7f69337fe700 1 -- 172.21.15.121:0/1919852226 learned_addr learned my addr 172.21.15.121:0/1919852226 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:11.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.442+0000 7f6933fff700 1 --2- 172.21.15.121:0/1919852226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6934153bb0 0x7f69340c1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:11.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.442+0000 7f692affd700 1 --2- 172.21.15.121:0/1919852226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69340aa020 0x7f69341542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:11.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.442+0000 7f69337fe700 1 -- 172.21.15.121:0/1919852226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69340aa020 msgr2=0x7f69341542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:11.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.442+0000 7f69337fe700 1 --2- 172.21.15.121:0/1919852226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69340aa020 0x7f69341542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:11.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.442+0000 7f69337fe700 1 -- 172.21.15.121:0/1919852226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6934153bb0 msgr2=0x7f69340c1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:11.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.442+0000 7f69337fe700 1 --2- 172.21.15.121:0/1919852226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6934153bb0 0x7f69340c1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:11.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.442+0000 7f69337fe700 1 -- 172.21.15.121:0/1919852226 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692c009370 con 0x7f6934067f50 2023-11-15T02:59:11.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.446+0000 7f69337fe700 1 --2- 172.21.15.121:0/1919852226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6934067f50 0x7f6934152420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f6924009e50 tx=0x7f692400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:11.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.446+0000 7f692bfff700 1 -- 172.21.15.121:0/1919852226 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692400cc20 con 0x7f6934067f50 2023-11-15T02:59:11.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.446+0000 7f692bfff700 1 -- 172.21.15.121:0/1919852226 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692400f040 con 0x7f6934067f50 2023-11-15T02:59:11.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.446+0000 7f692bfff700 1 -- 172.21.15.121:0/1919852226 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924013400 con 0x7f6934067f50 2023-11-15T02:59:11.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.446+0000 7f6939d69700 1 -- 172.21.15.121:0/1919852226 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69341551e0 con 0x7f6934067f50 2023-11-15T02:59:11.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.446+0000 7f6939d69700 1 -- 172.21.15.121:0/1919852226 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6934152910 con 0x7f6934067f50 2023-11-15T02:59:11.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.446+0000 7f6939d69700 1 -- 172.21.15.121:0/1919852226 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f693406c150 con 0x7f6934067f50 2023-11-15T02:59:11.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.446+0000 7f692bfff700 1 -- 172.21.15.121:0/1919852226 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6924013770 con 0x7f6934067f50 2023-11-15T02:59:11.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.450+0000 7f692bfff700 1 --2- 172.21.15.121:0/1919852226 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f691405a510 0x7f691405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:11.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.450+0000 7f692bfff700 1 -- 172.21.15.121:0/1919852226 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6924075040 con 0x7f6934067f50 2023-11-15T02:59:11.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.450+0000 7f692affd700 1 --2- 172.21.15.121:0/1919852226 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f691405a510 0x7f691405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:11.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.450+0000 7f692affd700 1 --2- 172.21.15.121:0/1919852226 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f691405a510 0x7f691405c9d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f691c006fd0 tx=0x7f691c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:11.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.454+0000 7f692bfff700 1 -- 172.21.15.121:0/1919852226 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6924048820 con 0x7f6934067f50 2023-11-15T02:59:11.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.650+0000 7f6939d69700 1 -- 172.21.15.121:0/1919852226 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69340c83c0 con 0x7f6934067f50 2023-11-15T02:59:11.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.650+0000 7f692bfff700 1 -- 172.21.15.121:0/1919852226 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4501 (secure 0 0 0) 0x7f692404be40 con 0x7f6934067f50 2023-11-15T02:59:11.659 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:11.659 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":30,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3403224816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3403224816},{"type":"v1","addr":"172.21.15.190:6835","nonce":3403224816}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2088953798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2088953798},{"type":"v1","addr":"172.21.15.190:6833","nonce":2088953798}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5511,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2190514138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2190514138},{"type":"v1","addr":"172.21.15.121:6835","nonce":2190514138}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:58:57.609524+0000","modified":"2023-11-15T02:59:00.621955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5514":{"gid":5514,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6837/1945477051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1945477051},{"type":"v1","addr":"172.21.15.121:6837","nonce":1945477051}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-15T02:59:11.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.654+0000 7f6939d69700 1 -- 172.21.15.121:0/1919852226 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f691405a510 msgr2=0x7f691405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:11.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.654+0000 7f6939d69700 1 --2- 172.21.15.121:0/1919852226 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f691405a510 0x7f691405c9d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f691c006fd0 tx=0x7f691c008040).stop 2023-11-15T02:59:11.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.654+0000 7f6939d69700 1 -- 172.21.15.121:0/1919852226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6934067f50 msgr2=0x7f6934152420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:11.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.654+0000 7f6939d69700 1 --2- 172.21.15.121:0/1919852226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6934067f50 0x7f6934152420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f6924009e50 tx=0x7f692400b040).stop 2023-11-15T02:59:11.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.654+0000 7f6939d69700 1 -- 172.21.15.121:0/1919852226 shutdown_connections 2023-11-15T02:59:11.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.654+0000 7f6939d69700 1 --2- 172.21.15.121:0/1919852226 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f691405a510 0x7f691405c9d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:11.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.654+0000 7f6939d69700 1 --2- 172.21.15.121:0/1919852226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6934153bb0 0x7f69340c1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:11.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.654+0000 7f6939d69700 1 --2- 172.21.15.121:0/1919852226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69340aa020 0x7f69341542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:11.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.654+0000 7f6939d69700 1 --2- 172.21.15.121:0/1919852226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6934067f50 0x7f6934152420 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:11.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.654+0000 7f6939d69700 1 -- 172.21.15.121:0/1919852226 >> 172.21.15.121:0/1919852226 conn(0x7f6934058c50 msgr2=0x7f6934158fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:11.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.654+0000 7f6939d69700 1 -- 172.21.15.121:0/1919852226 shutdown_connections 2023-11-15T02:59:11.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.654+0000 7f6939d69700 1 -- 172.21.15.121:0/1919852226 wait complete. 2023-11-15T02:59:11.664 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 30 2023-11-15T02:59:11.683 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump tree /a 0 2023-11-15T02:59:11.846 INFO:teuthology.orchestra.run.smithi121.stdout:[{"ino":1099511627776,"rdev":0,"ctime":"2023-11-15T02:59:11.066259+0000","btime":"2023-11-15T02:59:04.942382+0000","mode":16893,"uid":1000,"gid":1267,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2023-11-15T02:59:11.066259+0000","atime":"2023-11-15T02:59:04.942382+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2023-11-15T02:59:11.066259+0000","num_files":0,"num_subdirs":1,"change_attr":3},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2023-11-15T02:59:11.066259+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2023-11-15T02:59:11.066259+0000"},"version":9,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"dirfrag":1,"caps":0,"dirtyparent":0,"dirty":1,"authpin":0},"nref":2,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"states":["auth","dirty"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/a","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"7","version":"7","committing_version":"3","committed_version":"3","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"a/c","path_ino":1,"snap_first":18446744073709551606,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511627777,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"a/b","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":false,"is_remote":false,"is_null":true,"is_new":false,"inode":0,"remote_type":"","version":5,"projected_version":5,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":0,"dirty":1,"authpin":0},"nref":1,"lock":{},"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}]}] 2023-11-15T02:59:11.847 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2023-11-15T02:59:11.066259+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "atime": "2023-11-15T02:59:04.942382+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2023-11-15T02:59:04.942382+0000", "change_attr": 3, "client_caps": [], "client_ranges": [], "ctime": "2023-11-15T02:59:11.066259+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "3", "committing_version": "3", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511627777, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "a/c", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 18446744073709551606, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 0, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": true, "is_primary": false, "is_remote": false, "lock": {}, "nref": 1, "path": "a/b", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 0, "lock": 0, "request": 0 }, "projected_version": 5, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 5, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/a", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2023-11-15T02:59:11.066259+0000", "num_files": 0, "num_subdirs": 1, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1267, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16893, "mtime": "2023-11-15T02:59:11.066259+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "nlink": 1, "nref": 2, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 0, "dirfrag": 1, "dirty": 1, "dirtyparent": 0, "lock": 0, "request": 0 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2023-11-15T02:59:11.066259+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 9, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } ] 2023-11-15T02:59:11.847 DEBUG:tasks.cephfs.test_damage:[{'ino': 1099511627776, 'rdev': 0, 'ctime': '2023-11-15T02:59:11.066259+0000', 'btime': '2023-11-15T02:59:04.942382+0000', 'mode': 16893, 'uid': 1000, 'gid': 1267, 'nlink': 1, 'dir_layout': {'dir_hash': 2, 'unused1': 0, 'unused2': 0, 'unused3': 0}, 'layout': {'stripe_unit': 0, 'stripe_count': 0, 'object_size': 0, 'pool_id': -1, 'pool_ns': ''}, 'old_pools': [], 'size': 0, 'truncate_seq': 1, 'truncate_size': 18446744073709551615, 'truncate_from': 0, 'truncate_pending': 0, 'mtime': '2023-11-15T02:59:11.066259+0000', 'atime': '2023-11-15T02:59:04.942382+0000', 'time_warp_seq': 0, 'change_attr': 3, 'export_pin': -1, 'export_ephemeral_random_pin': 0, 'export_ephemeral_distributed_pin': False, 'client_ranges': [], 'dirstat': {'version': 0, 'mtime': '2023-11-15T02:59:11.066259+0000', 'num_files': 0, 'num_subdirs': 1, 'change_attr': 3}, 'rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2023-11-15T02:59:11.066259+0000'}, 'accounted_rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2023-11-15T02:59:11.066259+0000'}, 'version': 9, 'file_data_version': 0, 'xattr_version': 1, 'backtrace_version': 2, 'stray_prior_path': '', 'max_size_ever': 0, 'quota': {'max_bytes': 0, 'max_files': 0}, 'last_scrub_stamp': '0.000000', 'last_scrub_version': 0, 'symlink': '', 'xattrs': [], 'dirfragtree': {'splits': []}, 'old_inodes': [], 'oldest_snap': 18446744073709551614, 'damage_flags': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'dirfrag': 1, 'caps': 0, 'dirtyparent': 0, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'versionlock': {'gather_set': [], 'state': 'lock', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'authlock': {}, 'linklock': {}, 'dirfragtreelock': {}, 'filelock': {}, 'xattrlock': {}, 'snaplock': {}, 'nestlock': {'gather_set': [], 'state': 'lock', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'flocklock': {}, 'policylock': {}, 'states': ['auth', 'dirty'], 'client_caps': [], 'loner': -1, 'want_loner': -1, 'mds_caps_wanted': [], 'dirfrags': [{'path': '/a', 'dirfrag': '0x10000000000', 'snapid_first': 2, 'projected_version': '7', 'version': '7', 'committing_version': '3', 'committed_version': '3', 'is_rep': False, 'dir_auth': '', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'dentries': [{'path': 'a/c', 'path_ino': 1, 'snap_first': 18446744073709551606, 'snap_last': 18446744073709551614, 'is_primary': True, 'is_remote': False, 'is_null': False, 'is_new': False, 'inode': 1099511627777, 'remote_type': '', 'version': 4, 'projected_version': 4, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'inodepin': 1, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}, {'path': 'a/b', 'path_ino': 1, 'snap_first': 2, 'snap_last': 18446744073709551614, 'is_primary': False, 'is_remote': False, 'is_null': True, 'is_new': False, 'inode': 0, 'remote_type': '', 'version': 5, 'projected_version': 5, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'inodepin': 0, 'dirty': 1, 'authpin': 0}, 'nref': 1, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}]}]}] 2023-11-15T02:59:11.848 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T02:59:12.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.998+0000 7f628b59e700 1 Processor -- start 2023-11-15T02:59:12.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.998+0000 7f628b59e700 1 -- start start 2023-11-15T02:59:12.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:11.998+0000 7f628b59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f628c0a8880 0x7f628c151b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:12.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f628b59e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628c0c2c00 con 0x7f628c0bb510 2023-11-15T02:59:12.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f628b59e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628c0657b0 con 0x7f628c0abaa0 2023-11-15T02:59:12.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f628b59e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f628c065a90 con 0x7f628c0a8880 2023-11-15T02:59:12.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6283fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f628c0bb510 0x7f628c151400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:42920/0 (socket says 172.21.15.121:42920) 2023-11-15T02:59:12.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6283fff700 1 -- 172.21.15.121:0/1725650116 learned_addr learned my addr 172.21.15.121:0/1725650116 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:12.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f628ad9d700 1 --2- 172.21.15.121:0/1725650116 >> v2:172.21.15.190:3300/0 conn(0x7f628c0a8880 0x7f628c151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:12.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6289d9b700 1 -- 172.21.15.121:0/1725650116 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321958100 0 0) 0x7f628c0c2c00 con 0x7f628c0bb510 2023-11-15T02:59:12.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6289d9b700 1 -- 172.21.15.121:0/1725650116 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f626c0033f0 con 0x7f628c0bb510 2023-11-15T02:59:12.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6289d9b700 1 -- 172.21.15.121:0/1725650116 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3735350040 0 0) 0x7f626c0033f0 con 0x7f628c0bb510 2023-11-15T02:59:12.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6289d9b700 1 -- 172.21.15.121:0/1725650116 >> v2:172.21.15.190:3300/0 conn(0x7f628c0a8880 msgr2=0x7f628c151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:12.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6289d9b700 1 --2- 172.21.15.121:0/1725650116 >> v2:172.21.15.190:3300/0 conn(0x7f628c0a8880 0x7f628c151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:12.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6289d9b700 1 -- 172.21.15.121:0/1725650116 >> v1:172.21.15.190:6789/0 conn(0x7f628c0abaa0 legacy=0x7f628c063570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:12.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6289d9b700 1 -- 172.21.15.121:0/1725650116 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0c2890 con 0x7f628c0bb510 2023-11-15T02:59:12.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6289d9b700 1 -- 172.21.15.121:0/1725650116 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f62740015f0 con 0x7f628c0bb510 2023-11-15T02:59:12.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6289d9b700 1 -- 172.21.15.121:0/1725650116 >> v1:172.21.15.121:6789/0 conn(0x7f628c0bb510 legacy=0x7f628c151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:12.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6289d9b700 1 --2- 172.21.15.121:0/1725650116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f626c0038e0 0x7f626c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:12.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6289d9b700 1 --2- 172.21.15.121:0/1725650116 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626c004690 0x7f626c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:12.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6289d9b700 1 --2- 172.21.15.121:0/1725650116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f626c005000 0x7f626c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:12.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6289d9b700 1 -- 172.21.15.121:0/1725650116 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c065a90 con 0x7f626c005000 2023-11-15T02:59:12.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6289d9b700 1 -- 172.21.15.121:0/1725650116 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c00bf60 con 0x7f626c0038e0 2023-11-15T02:59:12.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6289d9b700 1 -- 172.21.15.121:0/1725650116 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626c00c0f0 con 0x7f626c004690 2023-11-15T02:59:12.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f628ad9d700 1 --2- 172.21.15.121:0/1725650116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f626c005000 0x7f626c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:12.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f628a59c700 1 --2- 172.21.15.121:0/1725650116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f626c0038e0 0x7f626c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:12.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6283fff700 1 --2- 172.21.15.121:0/1725650116 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626c004690 0x7f626c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:12.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f628ad9d700 1 -- 172.21.15.121:0/1725650116 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626c004690 msgr2=0x7f626c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:12.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f628ad9d700 1 --2- 172.21.15.121:0/1725650116 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626c004690 0x7f626c004b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:12.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f628ad9d700 1 -- 172.21.15.121:0/1725650116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f626c0038e0 msgr2=0x7f626c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:12.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f628ad9d700 1 --2- 172.21.15.121:0/1725650116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f626c0038e0 0x7f626c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:12.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f628ad9d700 1 -- 172.21.15.121:0/1725650116 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626c0033f0 con 0x7f626c005000 2023-11-15T02:59:12.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f628a59c700 1 --2- 172.21.15.121:0/1725650116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f626c0038e0 0x7f626c003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:12.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f628ad9d700 1 -- 172.21.15.121:0/1725650116 reap_dead start 2023-11-15T02:59:12.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f628ad9d700 1 --2- 172.21.15.121:0/1725650116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f626c005000 0x7f626c00b8e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f628c152000 tx=0x7f628400b040).ready entity=mon.0 client_cookie=d9f05c3efeb25740 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:12.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6289d9b700 1 -- 172.21.15.121:0/1725650116 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6284017070 con 0x7f626c005000 2023-11-15T02:59:12.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6289d9b700 1 -- 172.21.15.121:0/1725650116 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628400f040 con 0x7f626c005000 2023-11-15T02:59:12.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.002+0000 7f6289d9b700 1 -- 172.21.15.121:0/1725650116 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628400ece0 con 0x7f626c005000 2023-11-15T02:59:12.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.006+0000 7f628b59e700 1 -- 172.21.15.121:0/1725650116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f626c005000 msgr2=0x7f626c00b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:12.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.006+0000 7f628b59e700 1 --2- 172.21.15.121:0/1725650116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f626c005000 0x7f626c00b8e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f628c152000 tx=0x7f628400b040).stop 2023-11-15T02:59:12.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.006+0000 7f628b59e700 1 -- 172.21.15.121:0/1725650116 shutdown_connections 2023-11-15T02:59:12.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.006+0000 7f628b59e700 1 --2- 172.21.15.121:0/1725650116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f626c005000 0x7f626c00b8e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:12.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.006+0000 7f628b59e700 1 -- 172.21.15.121:0/1725650116 >> 172.21.15.121:0/1725650116 conn(0x7f628c067a60 msgr2=0x7f628c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:12.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.006+0000 7f628b59e700 1 -- 172.21.15.121:0/1725650116 shutdown_connections 2023-11-15T02:59:12.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.006+0000 7f628b59e700 1 -- 172.21.15.121:0/1725650116 wait complete. 2023-11-15T02:59:12.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.010+0000 7f628b59e700 1 Processor -- start 2023-11-15T02:59:12.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.010+0000 7f628b59e700 1 -- start start 2023-11-15T02:59:12.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.010+0000 7f628b59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626c005000 0x7f628c0c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:12.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.010+0000 7f628b59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f628c168530 0x7f628c163ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:12.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.010+0000 7f628b59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f628c1641d0 0x7f628c14ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:12.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.010+0000 7f628b59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0c4360 con 0x7f628c1641d0 2023-11-15T02:59:12.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.010+0000 7f628b59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0c6e00 con 0x7f628c168530 2023-11-15T02:59:12.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.010+0000 7f628b59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0c61f0 con 0x7f626c005000 2023-11-15T02:59:12.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.010+0000 7f628ad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f628c1641d0 0x7f628c14ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:12.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.010+0000 7f628ad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f628c1641d0 0x7f628c14ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51504/0 (socket says 172.21.15.121:51504) 2023-11-15T02:59:12.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.010+0000 7f628ad9d700 1 -- 172.21.15.121:0/2973976702 learned_addr learned my addr 172.21.15.121:0/2973976702 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:12.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.010+0000 7f6283fff700 1 --2- 172.21.15.121:0/2973976702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f628c168530 0x7f628c163ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:12.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.010+0000 7f628a59c700 1 --2- 172.21.15.121:0/2973976702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626c005000 0x7f628c0c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:12.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.014+0000 7f628ad9d700 1 -- 172.21.15.121:0/2973976702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626c005000 msgr2=0x7f628c0c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:12.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.014+0000 7f628ad9d700 1 --2- 172.21.15.121:0/2973976702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626c005000 0x7f628c0c5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:12.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.014+0000 7f628ad9d700 1 -- 172.21.15.121:0/2973976702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f628c168530 msgr2=0x7f628c163ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:12.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.014+0000 7f628ad9d700 1 --2- 172.21.15.121:0/2973976702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f628c168530 0x7f628c163ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:12.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.014+0000 7f628ad9d700 1 -- 172.21.15.121:0/2973976702 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0c64d0 con 0x7f628c1641d0 2023-11-15T02:59:12.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.014+0000 7f628ad9d700 1 --2- 172.21.15.121:0/2973976702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f628c1641d0 0x7f628c14ef70 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f628c0c2b70 tx=0x7f6284019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:12.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.014+0000 7f62827fc700 1 -- 172.21.15.121:0/2973976702 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628400ebd0 con 0x7f628c1641d0 2023-11-15T02:59:12.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.014+0000 7f62827fc700 1 -- 172.21.15.121:0/2973976702 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628400ed30 con 0x7f628c1641d0 2023-11-15T02:59:12.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.014+0000 7f62827fc700 1 -- 172.21.15.121:0/2973976702 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628401e680 con 0x7f628c1641d0 2023-11-15T02:59:12.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.014+0000 7f628b59e700 1 -- 172.21.15.121:0/2973976702 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628c0c67b0 con 0x7f628c1641d0 2023-11-15T02:59:12.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.014+0000 7f628b59e700 1 -- 172.21.15.121:0/2973976702 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628c1689b0 con 0x7f628c1641d0 2023-11-15T02:59:12.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.014+0000 7f628b59e700 1 -- 172.21.15.121:0/2973976702 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628c0c6fb0 con 0x7f628c1641d0 2023-11-15T02:59:12.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.018+0000 7f62827fc700 1 -- 172.21.15.121:0/2973976702 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6284003040 con 0x7f628c1641d0 2023-11-15T02:59:12.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.018+0000 7f62827fc700 1 --2- 172.21.15.121:0/2973976702 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f627005a510 0x7f627005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:12.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.018+0000 7f62827fc700 1 -- 172.21.15.121:0/2973976702 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6284076470 con 0x7f628c1641d0 2023-11-15T02:59:12.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.018+0000 7f628a59c700 1 --2- 172.21.15.121:0/2973976702 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f627005a510 0x7f627005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:12.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.018+0000 7f628a59c700 1 --2- 172.21.15.121:0/2973976702 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f627005a510 0x7f627005c9d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f627c000c00 tx=0x7f627c00c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:12.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.026+0000 7f62827fc700 1 -- 172.21.15.121:0/2973976702 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f628c0c6fb0 con 0x7f628c1641d0 2023-11-15T02:59:12.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:12.222+0000 7f628b59e700 1 -- 172.21.15.121:0/2973976702 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f628c0c6f70 con 0x7f628c1641d0 2023-11-15T02:59:13.038 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T02:59:13.030+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T02:59:13.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.042+0000 7f62827fc700 1 -- 172.21.15.121:0/2973976702 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7f628404d680 con 0x7f628c1641d0 2023-11-15T02:59:13.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.046+0000 7f628b59e700 1 -- 172.21.15.121:0/2973976702 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f627005a510 msgr2=0x7f627005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:13.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.046+0000 7f628b59e700 1 --2- 172.21.15.121:0/2973976702 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f627005a510 0x7f627005c9d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f627c000c00 tx=0x7f627c00c040).stop 2023-11-15T02:59:13.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.046+0000 7f628b59e700 1 -- 172.21.15.121:0/2973976702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f628c1641d0 msgr2=0x7f628c14ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:13.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.046+0000 7f628b59e700 1 --2- 172.21.15.121:0/2973976702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f628c1641d0 0x7f628c14ef70 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f628c0c2b70 tx=0x7f6284019000).stop 2023-11-15T02:59:13.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.046+0000 7f628b59e700 1 -- 172.21.15.121:0/2973976702 shutdown_connections 2023-11-15T02:59:13.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.046+0000 7f628b59e700 1 --2- 172.21.15.121:0/2973976702 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f627005a510 0x7f627005c9d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.050+0000 7f628b59e700 1 --2- 172.21.15.121:0/2973976702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f628c1641d0 0x7f628c14ef70 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.050+0000 7f628b59e700 1 --2- 172.21.15.121:0/2973976702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f628c168530 0x7f628c163ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.050+0000 7f628b59e700 1 --2- 172.21.15.121:0/2973976702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626c005000 0x7f628c0c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.050+0000 7f628b59e700 1 -- 172.21.15.121:0/2973976702 >> 172.21.15.121:0/2973976702 conn(0x7f628c067a60 msgr2=0x7f628c1483b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:13.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.050+0000 7f628b59e700 1 -- 172.21.15.121:0/2973976702 shutdown_connections 2023-11-15T02:59:13.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.050+0000 7f628b59e700 1 -- 172.21.15.121:0/2973976702 wait complete. 2023-11-15T02:59:13.057 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T02:59:13.062 INFO:tasks.ceph.mds.c.smithi121.stderr: -2734> 2023-11-15T02:58:58.014+0000 7feaa9e67700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T02:59:13.068 INFO:tasks.ceph.mds.c.smithi121.stderr: -315> 2023-11-15T02:59:11.074+0000 7feaac66c700 -1 mds.0.cache.den(0x10000000000 c) newly corrupt dentry to be committed: [dentry #0x1/a/c [fffffffffffffff6,head] auth NULL (dn xlock x=1 by 0x5640eb7fac00) (dversion lock w=1 last_client=5608) pv=4 v=3 ap=2 ino=(nil) state=1073741888|bottomlru | request=1 lock=2 authpin=1 0x5640eb7e7900] 2023-11-15T02:59:13.078 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2023-11-15T02:59:13.111 INFO:tasks.ceph.mds.c.smithi121.stdout:starting mds.c at 2023-11-15T02:59:13.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.234+0000 7f7bb4b07700 1 Processor -- start 2023-11-15T02:59:13.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.234+0000 7f7bb4b07700 1 -- start start 2023-11-15T02:59:13.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.234+0000 7f7bb4b07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb0151af0 0x7f7bb00a7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:13.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.234+0000 7f7bb4b07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7bb00a5720 0x7f7bb00c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:13.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.234+0000 7f7bb4b07700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7bb00c3430 0x7f7bb00c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:13.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.234+0000 7f7bb4b07700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb00c38e0 con 0x7f7bb00c3430 2023-11-15T02:59:13.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.234+0000 7f7bb4b07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb01528b0 con 0x7f7bb00a5720 2023-11-15T02:59:13.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.234+0000 7f7bb4b07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb0151ff0 con 0x7f7bb0151af0 2023-11-15T02:59:13.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.234+0000 7f7baffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7bb00c3430 0x7f7bb00c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:13.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.234+0000 7f7baeffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7bb00a5720 0x7f7bb00c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:13.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.234+0000 7f7baf7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb0151af0 0x7f7bb00a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:13.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.234+0000 7f7baffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7bb00c3430 0x7f7bb00c64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51524/0 (socket says 172.21.15.121:51524) 2023-11-15T02:59:13.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.234+0000 7f7baffff700 1 -- 172.21.15.121:0/4104403769 learned_addr learned my addr 172.21.15.121:0/4104403769 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:13.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.238+0000 7f7baffff700 1 -- 172.21.15.121:0/4104403769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb0151af0 msgr2=0x7f7bb00a7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:13.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.238+0000 7f7baffff700 1 --2- 172.21.15.121:0/4104403769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb0151af0 0x7f7bb00a7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.238+0000 7f7baffff700 1 -- 172.21.15.121:0/4104403769 >> v2:172.21.15.190:3300/0 conn(0x7f7bb00a5720 msgr2=0x7f7bb00c2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:13.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.238+0000 7f7baffff700 1 --2- 172.21.15.121:0/4104403769 >> v2:172.21.15.190:3300/0 conn(0x7f7bb00a5720 0x7f7bb00c2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.238+0000 7f7baffff700 1 -- 172.21.15.121:0/4104403769 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb0152260 con 0x7f7bb00c3430 2023-11-15T02:59:13.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.238+0000 7f7baeffd700 1 --2- 172.21.15.121:0/4104403769 >> v2:172.21.15.190:3300/0 conn(0x7f7bb00a5720 0x7f7bb00c2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:59:13.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.238+0000 7f7baffff700 1 --2- 172.21.15.121:0/4104403769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bb00c3430 0x7f7bb00c64a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f7bb00652a0 tx=0x7f7ba8009840).ready entity=mon.0 client_cookie=8fc960381c74d738 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:13.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.238+0000 7f7bae7fc700 1 -- 172.21.15.121:0/4104403769 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba800e070 con 0x7f7bb00c3430 2023-11-15T02:59:13.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.238+0000 7f7bae7fc700 1 -- 172.21.15.121:0/4104403769 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba800b970 con 0x7f7bb00c3430 2023-11-15T02:59:13.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.238+0000 7f7bae7fc700 1 -- 172.21.15.121:0/4104403769 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba800bcf0 con 0x7f7bb00c3430 2023-11-15T02:59:13.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.238+0000 7f7bb4b07700 1 -- 172.21.15.121:0/4104403769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bb00c3430 msgr2=0x7f7bb00c64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:13.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.238+0000 7f7bb4b07700 1 --2- 172.21.15.121:0/4104403769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bb00c3430 0x7f7bb00c64a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f7bb00652a0 tx=0x7f7ba8009840).stop 2023-11-15T02:59:13.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.238+0000 7f7bb4b07700 1 -- 172.21.15.121:0/4104403769 shutdown_connections 2023-11-15T02:59:13.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.238+0000 7f7bb4b07700 1 --2- 172.21.15.121:0/4104403769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bb00c3430 0x7f7bb00c64a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.238+0000 7f7bb4b07700 1 --2- 172.21.15.121:0/4104403769 >> v2:172.21.15.190:3300/0 conn(0x7f7bb00a5720 0x7f7bb00c2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.238+0000 7f7bb4b07700 1 --2- 172.21.15.121:0/4104403769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb0151af0 0x7f7bb00a7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.238+0000 7f7bb4b07700 1 -- 172.21.15.121:0/4104403769 >> 172.21.15.121:0/4104403769 conn(0x7f7bb0067a60 msgr2=0x7f7bb0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:13.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.238+0000 7f7bb4b07700 1 -- 172.21.15.121:0/4104403769 shutdown_connections 2023-11-15T02:59:13.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.238+0000 7f7bb4b07700 1 -- 172.21.15.121:0/4104403769 wait complete. 2023-11-15T02:59:13.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.242+0000 7f7bb4b07700 1 Processor -- start 2023-11-15T02:59:13.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.242+0000 7f7bb4b07700 1 -- start start 2023-11-15T02:59:13.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.242+0000 7f7bb4b07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb00a5720 0x7f7bb0139690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:13.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.242+0000 7f7bb4b07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bb00c3430 0x7f7bb00c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:13.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.242+0000 7f7bb4b07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb00c6670 0x7f7bb00c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:13.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.242+0000 7f7bb4b07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb0156ba0 con 0x7f7bb00c3430 2023-11-15T02:59:13.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.242+0000 7f7bb4b07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb00bf980 con 0x7f7bb00c6670 2023-11-15T02:59:13.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.242+0000 7f7bb4b07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb00c07e0 con 0x7f7bb00a5720 2023-11-15T02:59:13.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.246+0000 7f7baeffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bb00c3430 0x7f7bb00c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:13.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.246+0000 7f7baeffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bb00c3430 0x7f7bb00c6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51538/0 (socket says 172.21.15.121:51538) 2023-11-15T02:59:13.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.246+0000 7f7baeffd700 1 -- 172.21.15.121:0/2157182719 learned_addr learned my addr 172.21.15.121:0/2157182719 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:13.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.246+0000 7f7baf7fe700 1 --2- 172.21.15.121:0/2157182719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb00a5720 0x7f7bb0139690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:13.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.246+0000 7f7baffff700 1 --2- 172.21.15.121:0/2157182719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb00c6670 0x7f7bb00c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:13.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.246+0000 7f7baeffd700 1 -- 172.21.15.121:0/2157182719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb00a5720 msgr2=0x7f7bb0139690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:13.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.246+0000 7f7baeffd700 1 --2- 172.21.15.121:0/2157182719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb00a5720 0x7f7bb0139690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.246+0000 7f7baeffd700 1 -- 172.21.15.121:0/2157182719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb00c6670 msgr2=0x7f7bb00c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:13.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.246+0000 7f7baeffd700 1 --2- 172.21.15.121:0/2157182719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb00c6670 0x7f7bb00c5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.246+0000 7f7baeffd700 1 -- 172.21.15.121:0/2157182719 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba40092a0 con 0x7f7bb00c3430 2023-11-15T02:59:13.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.246+0000 7f7baf7fe700 1 --2- 172.21.15.121:0/2157182719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb00a5720 0x7f7bb0139690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:13.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.246+0000 7f7baffff700 1 --2- 172.21.15.121:0/2157182719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb00c6670 0x7f7bb00c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:13.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.246+0000 7f7baeffd700 1 --2- 172.21.15.121:0/2157182719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bb00c3430 0x7f7bb00c6180 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f7ba400e930 tx=0x7f7ba4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:13.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.246+0000 7f7b977fe700 1 -- 172.21.15.121:0/2157182719 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400eed0 con 0x7f7bb00c3430 2023-11-15T02:59:13.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.246+0000 7f7b977fe700 1 -- 172.21.15.121:0/2157182719 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba400c850 con 0x7f7bb00c3430 2023-11-15T02:59:13.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.246+0000 7f7bb4b07700 1 -- 172.21.15.121:0/2157182719 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba8009ae0 con 0x7f7bb00c3430 2023-11-15T02:59:13.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.246+0000 7f7b977fe700 1 -- 172.21.15.121:0/2157182719 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400cbd0 con 0x7f7bb00c3430 2023-11-15T02:59:13.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.246+0000 7f7bb4b07700 1 -- 172.21.15.121:0/2157182719 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb00bfeb0 con 0x7f7bb00c3430 2023-11-15T02:59:13.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.246+0000 7f7bb4b07700 1 -- 172.21.15.121:0/2157182719 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb006bcf0 con 0x7f7bb00c3430 2023-11-15T02:59:13.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.250+0000 7f7b977fe700 1 -- 172.21.15.121:0/2157182719 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7ba401b400 con 0x7f7bb00c3430 2023-11-15T02:59:13.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.250+0000 7f7b977fe700 1 --2- 172.21.15.121:0/2157182719 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7b9c05a510 0x7f7b9c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:13.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.250+0000 7f7b977fe700 1 -- 172.21.15.121:0/2157182719 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f7ba4074d70 con 0x7f7bb00c3430 2023-11-15T02:59:13.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.250+0000 7f7baf7fe700 1 --2- 172.21.15.121:0/2157182719 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7b9c05a510 0x7f7b9c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:13.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.254+0000 7f7baf7fe700 1 --2- 172.21.15.121:0/2157182719 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7b9c05a510 0x7f7b9c05c9d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f7ba0009c00 tx=0x7f7ba0008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:13.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.258+0000 7f7b977fe700 1 -- 172.21.15.121:0/2157182719 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7ba40484a0 con 0x7f7bb00c3430 2023-11-15T02:59:13.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.422+0000 7f7bb4b07700 1 -- 172.21.15.121:0/2157182719 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) v1 -- 0x7f7bb0068a70 con 0x7f7bb00c3430 2023-11-15T02:59:13.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.430+0000 7f7b977fe700 1 -- 172.21.15.121:0/2157182719 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v7) v1 ==== 121+0+0 (secure 0 0 0) 0x7f7ba4016070 con 0x7f7bb00c3430 2023-11-15T02:59:13.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.434+0000 7f7bb4b07700 1 -- 172.21.15.121:0/2157182719 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7b9c05a510 msgr2=0x7f7b9c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:13.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.434+0000 7f7bb4b07700 1 --2- 172.21.15.121:0/2157182719 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7b9c05a510 0x7f7b9c05c9d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f7ba0009c00 tx=0x7f7ba0008040).stop 2023-11-15T02:59:13.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.434+0000 7f7bb4b07700 1 -- 172.21.15.121:0/2157182719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bb00c3430 msgr2=0x7f7bb00c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:13.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.434+0000 7f7bb4b07700 1 --2- 172.21.15.121:0/2157182719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bb00c3430 0x7f7bb00c6180 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f7ba400e930 tx=0x7f7ba4009060).stop 2023-11-15T02:59:13.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.434+0000 7f7bb4b07700 1 -- 172.21.15.121:0/2157182719 shutdown_connections 2023-11-15T02:59:13.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.434+0000 7f7bb4b07700 1 --2- 172.21.15.121:0/2157182719 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7b9c05a510 0x7f7b9c05c9d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.434+0000 7f7bb4b07700 1 --2- 172.21.15.121:0/2157182719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb00c6670 0x7f7bb00c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.434+0000 7f7bb4b07700 1 --2- 172.21.15.121:0/2157182719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bb00c3430 0x7f7bb00c6180 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.434+0000 7f7bb4b07700 1 --2- 172.21.15.121:0/2157182719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb00a5720 0x7f7bb0139690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.434+0000 7f7bb4b07700 1 -- 172.21.15.121:0/2157182719 >> 172.21.15.121:0/2157182719 conn(0x7f7bb0067a60 msgr2=0x7f7bb0150380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:13.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.434+0000 7f7bb4b07700 1 -- 172.21.15.121:0/2157182719 shutdown_connections 2023-11-15T02:59:13.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.434+0000 7f7bb4b07700 1 -- 172.21.15.121:0/2157182719 wait complete. 2023-11-15T02:59:13.463 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_abort_on_newly_corrupt_dentry False 2023-11-15T02:59:13.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.666+0000 7f78f71f9700 1 Processor -- start 2023-11-15T02:59:13.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.666+0000 7f78f71f9700 1 -- start start 2023-11-15T02:59:13.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.670+0000 7f78f71f9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f78f00acd00 0x7f78f00ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:13.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.670+0000 7f78f71f9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f0065ea0 con 0x7f78f0066330 2023-11-15T02:59:13.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.670+0000 7f78f71f9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f0066bb0 con 0x7f78f00c5c70 2023-11-15T02:59:13.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.670+0000 7f78f71f9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f0065890 con 0x7f78f00acd00 2023-11-15T02:59:13.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.670+0000 7f78f61f7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f78f00acd00 0x7f78f00ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:13.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.670+0000 7f78f61f7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f78f00acd00 0x7f78f00ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51548/0 (socket says 172.21.15.121:51548) 2023-11-15T02:59:13.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.670+0000 7f78f61f7700 1 -- 172.21.15.121:0/692744635 learned_addr learned my addr 172.21.15.121:0/692744635 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:13.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.670+0000 7f78f69f8700 1 --1- 172.21.15.121:0/692744635 >> v1:172.21.15.121:6789/0 conn(0x7f78f0066330 0x7f78f01577e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:42924/0 (socket says 172.21.15.121:42924) 2023-11-15T02:59:13.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.670+0000 7f78f61f7700 1 -- 172.21.15.121:0/692744635 >> v1:172.21.15.190:6789/0 conn(0x7f78f00c5c70 legacy=0x7f78f01536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:13.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.670+0000 7f78f61f7700 1 -- 172.21.15.121:0/692744635 >> v1:172.21.15.121:6789/0 conn(0x7f78f0066330 legacy=0x7f78f01577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:13.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.670+0000 7f78f61f7700 1 -- 172.21.15.121:0/692744635 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f00c64a0 con 0x7f78f00acd00 2023-11-15T02:59:13.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.670+0000 7f78f51f5700 1 -- 172.21.15.121:0/692744635 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1595189568 0 0) 0x7f78f0065ea0 con 0x7f78f0066330 2023-11-15T02:59:13.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.670+0000 7f78f61f7700 1 --2- 172.21.15.121:0/692744635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78f00acd00 0x7f78f00ad0e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f78ec009810 tx=0x7f78ec009060).ready entity=mon.0 client_cookie=829b2aed27d40105 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:13.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.670+0000 7f78f51f5700 1 -- 172.21.15.121:0/692744635 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec005880 con 0x7f78f00acd00 2023-11-15T02:59:13.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.670+0000 7f78f51f5700 1 -- 172.21.15.121:0/692744635 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec0059e0 con 0x7f78f00acd00 2023-11-15T02:59:13.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.670+0000 7f78f51f5700 1 -- 172.21.15.121:0/692744635 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec005d60 con 0x7f78f00acd00 2023-11-15T02:59:13.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.674+0000 7f78f71f9700 1 -- 172.21.15.121:0/692744635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78f00acd00 msgr2=0x7f78f00ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:13.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.674+0000 7f78f71f9700 1 --2- 172.21.15.121:0/692744635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78f00acd00 0x7f78f00ad0e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f78ec009810 tx=0x7f78ec009060).stop 2023-11-15T02:59:13.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.674+0000 7f78f71f9700 1 -- 172.21.15.121:0/692744635 shutdown_connections 2023-11-15T02:59:13.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.674+0000 7f78f71f9700 1 --2- 172.21.15.121:0/692744635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78f00acd00 0x7f78f00ad0e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.674+0000 7f78f71f9700 1 -- 172.21.15.121:0/692744635 >> 172.21.15.121:0/692744635 conn(0x7f78f0063fc0 msgr2=0x7f78f0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:13.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.674+0000 7f78f71f9700 1 -- 172.21.15.121:0/692744635 shutdown_connections 2023-11-15T02:59:13.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.674+0000 7f78f71f9700 1 -- 172.21.15.121:0/692744635 wait complete. 2023-11-15T02:59:13.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f71f9700 1 Processor -- start 2023-11-15T02:59:13.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f71f9700 1 -- start start 2023-11-15T02:59:13.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f71f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78f0066330 0x7f78f00c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:13.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f71f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78f00acd00 0x7f78f013e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:13.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f71f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78f00c5c70 0x7f78f0150e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:13.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f71f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f00c3f00 con 0x7f78f0066330 2023-11-15T02:59:13.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f71f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f00c32f0 con 0x7f78f00acd00 2023-11-15T02:59:13.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f71f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f00c35d0 con 0x7f78f00c5c70 2023-11-15T02:59:13.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f61f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78f0066330 0x7f78f00c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:13.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f61f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78f0066330 0x7f78f00c4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51554/0 (socket says 172.21.15.121:51554) 2023-11-15T02:59:13.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f61f7700 1 -- 172.21.15.121:0/655383035 learned_addr learned my addr 172.21.15.121:0/655383035 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:13.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f59f6700 1 --2- 172.21.15.121:0/655383035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78f00acd00 0x7f78f013e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:13.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f69f8700 1 --2- 172.21.15.121:0/655383035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78f00c5c70 0x7f78f0150e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:13.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f61f7700 1 -- 172.21.15.121:0/655383035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78f00c5c70 msgr2=0x7f78f0150e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:13.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f61f7700 1 --2- 172.21.15.121:0/655383035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78f00c5c70 0x7f78f0150e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f61f7700 1 -- 172.21.15.121:0/655383035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78f00acd00 msgr2=0x7f78f013e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:13.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f61f7700 1 --2- 172.21.15.121:0/655383035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78f00acd00 0x7f78f013e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f61f7700 1 -- 172.21.15.121:0/655383035 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec009300 con 0x7f78f0066330 2023-11-15T02:59:13.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f69f8700 1 --2- 172.21.15.121:0/655383035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78f00c5c70 0x7f78f0150e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:13.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.678+0000 7f78f61f7700 1 --2- 172.21.15.121:0/655383035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78f0066330 0x7f78f00c4390 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f78ec009fd0 tx=0x7f78ec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:13.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.682+0000 7f78e67fc700 1 -- 172.21.15.121:0/655383035 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec00b850 con 0x7f78f0066330 2023-11-15T02:59:13.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.682+0000 7f78e67fc700 1 -- 172.21.15.121:0/655383035 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec00b9b0 con 0x7f78f0066330 2023-11-15T02:59:13.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.682+0000 7f78f71f9700 1 -- 172.21.15.121:0/655383035 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f00c38b0 con 0x7f78f0066330 2023-11-15T02:59:13.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.682+0000 7f78e67fc700 1 -- 172.21.15.121:0/655383035 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec00bd30 con 0x7f78f0066330 2023-11-15T02:59:13.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.682+0000 7f78f71f9700 1 -- 172.21.15.121:0/655383035 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f00c2f10 con 0x7f78f0066330 2023-11-15T02:59:13.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.682+0000 7f78f71f9700 1 -- 172.21.15.121:0/655383035 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f00c4880 con 0x7f78f0066330 2023-11-15T02:59:13.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.682+0000 7f78e67fc700 1 -- 172.21.15.121:0/655383035 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f78ec018030 con 0x7f78f0066330 2023-11-15T02:59:13.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.686+0000 7f78e67fc700 1 --2- 172.21.15.121:0/655383035 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f78d405a510 0x7f78d405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:13.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.686+0000 7f78e67fc700 1 -- 172.21.15.121:0/655383035 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f78ec075990 con 0x7f78f0066330 2023-11-15T02:59:13.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.686+0000 7f78f59f6700 1 --2- 172.21.15.121:0/655383035 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f78d405a510 0x7f78d405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:13.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.686+0000 7f78f59f6700 1 --2- 172.21.15.121:0/655383035 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f78d405a510 0x7f78d405c9d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f78f0153330 tx=0x7f78e0009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:13.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.690+0000 7f78e67fc700 1 -- 172.21.15.121:0/655383035 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f78ec049060 con 0x7f78f0066330 2023-11-15T02:59:13.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.854+0000 7f78f71f9700 1 -- 172.21.15.121:0/655383035 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}] v 0) v1 -- 0x7f78f006bcf0 con 0x7f78f0066330 2023-11-15T02:59:13.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.854+0000 7f78e67fc700 1 -- 172.21.15.121:0/655383035 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}]=0 v7)=0 v7) v1 ==== 135+0+0 (secure 0 0 0) 0x7f78ec028070 con 0x7f78f0066330 2023-11-15T02:59:13.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.858+0000 7f78f71f9700 1 -- 172.21.15.121:0/655383035 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f78d405a510 msgr2=0x7f78d405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:13.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.858+0000 7f78f71f9700 1 --2- 172.21.15.121:0/655383035 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f78d405a510 0x7f78d405c9d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f78f0153330 tx=0x7f78e0009040).stop 2023-11-15T02:59:13.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.858+0000 7f78f71f9700 1 -- 172.21.15.121:0/655383035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78f0066330 msgr2=0x7f78f00c4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:13.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.858+0000 7f78f71f9700 1 --2- 172.21.15.121:0/655383035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78f0066330 0x7f78f00c4390 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f78ec009fd0 tx=0x7f78ec016000).stop 2023-11-15T02:59:13.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.858+0000 7f78f71f9700 1 -- 172.21.15.121:0/655383035 shutdown_connections 2023-11-15T02:59:13.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.858+0000 7f78f71f9700 1 --2- 172.21.15.121:0/655383035 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f78d405a510 0x7f78d405c9d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.858+0000 7f78f71f9700 1 --2- 172.21.15.121:0/655383035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78f00c5c70 0x7f78f0150e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.858+0000 7f78f71f9700 1 --2- 172.21.15.121:0/655383035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78f00acd00 0x7f78f013e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.858+0000 7f78f71f9700 1 --2- 172.21.15.121:0/655383035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78f0066330 0x7f78f00c4390 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:13.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.858+0000 7f78f71f9700 1 -- 172.21.15.121:0/655383035 >> 172.21.15.121:0/655383035 conn(0x7f78f0063fc0 msgr2=0x7f78f00caf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:13.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.858+0000 7f78f71f9700 1 -- 172.21.15.121:0/655383035 shutdown_connections 2023-11-15T02:59:13.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:13.858+0000 7f78f71f9700 1 -- 172.21.15.121:0/655383035 wait complete. 2023-11-15T02:59:13.885 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T02:59:14.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.034+0000 7fb585169700 1 Processor -- start 2023-11-15T02:59:14.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.038+0000 7fb585169700 1 -- start start 2023-11-15T02:59:14.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.042+0000 7fb585169700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5800682a0 0x7fb5800c10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:14.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.042+0000 7fb585169700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5800c15a0 con 0x7fb580067ec0 2023-11-15T02:59:14.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.042+0000 7fb585169700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5800c0c20 con 0x7fb580153700 2023-11-15T02:59:14.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.042+0000 7fb585169700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5800686e0 con 0x7fb5800682a0 2023-11-15T02:59:14.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.042+0000 7fb57ed9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb580067ec0 0x7fb580062c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:42938/0 (socket says 172.21.15.121:42938) 2023-11-15T02:59:14.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.042+0000 7fb57ed9d700 1 -- 172.21.15.121:0/1249116481 learned_addr learned my addr 172.21.15.121:0/1249116481 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:14.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.042+0000 7fb57e59c700 1 --2- 172.21.15.121:0/1249116481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5800682a0 0x7fb5800c10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:14.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.042+0000 7fb57e59c700 1 -- 172.21.15.121:0/1249116481 >> v1:172.21.15.190:6789/0 conn(0x7fb580153700 legacy=0x7fb580153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:14.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.042+0000 7fb57e59c700 1 -- 172.21.15.121:0/1249116481 >> v1:172.21.15.121:6789/0 conn(0x7fb580067ec0 legacy=0x7fb580062c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:14.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.042+0000 7fb57e59c700 1 -- 172.21.15.121:0/1249116481 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb580158730 con 0x7fb5800682a0 2023-11-15T02:59:14.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.042+0000 7fb57e59c700 1 --2- 172.21.15.121:0/1249116481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5800682a0 0x7fb5800c10b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fb574009790 tx=0x7fb574009060).ready entity=mon.2 client_cookie=88ddca368e98af30 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:14.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.046+0000 7fb57dd9b700 1 -- 172.21.15.121:0/1249116481 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57400b5a0 con 0x7fb5800682a0 2023-11-15T02:59:14.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.046+0000 7fb57dd9b700 1 -- 172.21.15.121:0/1249116481 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57400b700 con 0x7fb5800682a0 2023-11-15T02:59:14.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.046+0000 7fb57dd9b700 1 -- 172.21.15.121:0/1249116481 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57400ba80 con 0x7fb5800682a0 2023-11-15T02:59:14.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.046+0000 7fb585169700 1 -- 172.21.15.121:0/1249116481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5800682a0 msgr2=0x7fb5800c10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:14.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.046+0000 7fb585169700 1 --2- 172.21.15.121:0/1249116481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5800682a0 0x7fb5800c10b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fb574009790 tx=0x7fb574009060).stop 2023-11-15T02:59:14.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.046+0000 7fb585169700 1 -- 172.21.15.121:0/1249116481 shutdown_connections 2023-11-15T02:59:14.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.046+0000 7fb585169700 1 --2- 172.21.15.121:0/1249116481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5800682a0 0x7fb5800c10b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:14.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.046+0000 7fb585169700 1 -- 172.21.15.121:0/1249116481 >> 172.21.15.121:0/1249116481 conn(0x7fb5800cad40 msgr2=0x7fb5800cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:14.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.046+0000 7fb585169700 1 -- 172.21.15.121:0/1249116481 shutdown_connections 2023-11-15T02:59:14.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.046+0000 7fb585169700 1 -- 172.21.15.121:0/1249116481 wait complete. 2023-11-15T02:59:14.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.050+0000 7fb585169700 1 Processor -- start 2023-11-15T02:59:14.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.050+0000 7fb585169700 1 -- start start 2023-11-15T02:59:14.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.050+0000 7fb585169700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb580067ec0 0x7fb580165460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:14.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.050+0000 7fb585169700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5800682a0 0x7fb580166fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:14.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.050+0000 7fb585169700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb580153700 0x7fb580164290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:14.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.050+0000 7fb585169700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5801505b0 con 0x7fb580153700 2023-11-15T02:59:14.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.050+0000 7fb585169700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb580164e50 con 0x7fb5800682a0 2023-11-15T02:59:14.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.050+0000 7fb585169700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb580165130 con 0x7fb580067ec0 2023-11-15T02:59:14.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.054+0000 7fb57ed9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb580067ec0 0x7fb580165460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:14.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.054+0000 7fb57f59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb580153700 0x7fb580164290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:14.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.054+0000 7fb57ed9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb580067ec0 0x7fb580165460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:41686/0 (socket says 172.21.15.121:41686) 2023-11-15T02:59:14.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.054+0000 7fb57ed9d700 1 -- 172.21.15.121:0/1088036750 learned_addr learned my addr 172.21.15.121:0/1088036750 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:14.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.054+0000 7fb57e59c700 1 --2- 172.21.15.121:0/1088036750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5800682a0 0x7fb580166fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:14.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.054+0000 7fb57f59e700 1 -- 172.21.15.121:0/1088036750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb580067ec0 msgr2=0x7fb580165460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:14.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.054+0000 7fb57f59e700 1 --2- 172.21.15.121:0/1088036750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb580067ec0 0x7fb580165460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:14.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.054+0000 7fb57f59e700 1 -- 172.21.15.121:0/1088036750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5800682a0 msgr2=0x7fb580166fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:14.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.054+0000 7fb57f59e700 1 --2- 172.21.15.121:0/1088036750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5800682a0 0x7fb580166fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:14.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.054+0000 7fb57f59e700 1 -- 172.21.15.121:0/1088036750 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5740092a0 con 0x7fb580153700 2023-11-15T02:59:14.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.054+0000 7fb57f59e700 1 --2- 172.21.15.121:0/1088036750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb580153700 0x7fb580164290 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fb5801542a0 tx=0x7fb578001460).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:14.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.054+0000 7fb566ffd700 1 -- 172.21.15.121:0/1088036750 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57800f470 con 0x7fb580153700 2023-11-15T02:59:14.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.054+0000 7fb566ffd700 1 -- 172.21.15.121:0/1088036750 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57800f5d0 con 0x7fb580153700 2023-11-15T02:59:14.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.054+0000 7fb585169700 1 -- 172.21.15.121:0/1088036750 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5801542d0 con 0x7fb580153700 2023-11-15T02:59:14.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.054+0000 7fb566ffd700 1 -- 172.21.15.121:0/1088036750 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57800f950 con 0x7fb580153700 2023-11-15T02:59:14.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.054+0000 7fb585169700 1 -- 172.21.15.121:0/1088036750 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb580154890 con 0x7fb580153700 2023-11-15T02:59:14.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.054+0000 7fb585169700 1 -- 172.21.15.121:0/1088036750 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb580154ab0 con 0x7fb580153700 2023-11-15T02:59:14.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.058+0000 7fb566ffd700 1 -- 172.21.15.121:0/1088036750 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb57800fd40 con 0x7fb580153700 2023-11-15T02:59:14.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.058+0000 7fb566ffd700 1 --2- 172.21.15.121:0/1088036750 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb56c05a510 0x7fb56c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:14.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.058+0000 7fb566ffd700 1 -- 172.21.15.121:0/1088036750 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fb578073ee0 con 0x7fb580153700 2023-11-15T02:59:14.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.058+0000 7fb57ed9d700 1 --2- 172.21.15.121:0/1088036750 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb56c05a510 0x7fb56c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:14.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.058+0000 7fb57ed9d700 1 --2- 172.21.15.121:0/1088036750 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb56c05a510 0x7fb56c05c9d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fb570007fd0 tx=0x7fb570007df0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:14.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.062+0000 7fb566ffd700 1 -- 172.21.15.121:0/1088036750 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb578047670 con 0x7fb580153700 2023-11-15T02:59:14.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:14.258+0000 7fb585169700 1 -- 172.21.15.121:0/1088036750 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fb580047060 con 0x7fb580153700 2023-11-15T02:59:15.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.062+0000 7fb566ffd700 1 -- 172.21.15.121:0/1088036750 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v33) v1 ==== 162+0+0 (secure 0 0 0) 0x7fb57804ac90 con 0x7fb580153700 2023-11-15T02:59:15.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.066+0000 7fb585169700 1 -- 172.21.15.121:0/1088036750 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb56c05a510 msgr2=0x7fb56c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.066+0000 7fb585169700 1 --2- 172.21.15.121:0/1088036750 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb56c05a510 0x7fb56c05c9d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fb570007fd0 tx=0x7fb570007df0).stop 2023-11-15T02:59:15.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.066+0000 7fb585169700 1 -- 172.21.15.121:0/1088036750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb580153700 msgr2=0x7fb580164290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.066+0000 7fb585169700 1 --2- 172.21.15.121:0/1088036750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb580153700 0x7fb580164290 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fb5801542a0 tx=0x7fb578001460).stop 2023-11-15T02:59:15.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.066+0000 7fb585169700 1 -- 172.21.15.121:0/1088036750 shutdown_connections 2023-11-15T02:59:15.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.066+0000 7fb585169700 1 --2- 172.21.15.121:0/1088036750 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb56c05a510 0x7fb56c05c9d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.066+0000 7fb585169700 1 --2- 172.21.15.121:0/1088036750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb580153700 0x7fb580164290 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.066+0000 7fb585169700 1 --2- 172.21.15.121:0/1088036750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5800682a0 0x7fb580166fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.066+0000 7fb585169700 1 --2- 172.21.15.121:0/1088036750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb580067ec0 0x7fb580165460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.066+0000 7fb585169700 1 -- 172.21.15.121:0/1088036750 >> 172.21.15.121:0/1088036750 conn(0x7fb5800cad40 msgr2=0x7fb5800c9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:15.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.066+0000 7fb585169700 1 -- 172.21.15.121:0/1088036750 shutdown_connections 2023-11-15T02:59:15.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.066+0000 7fb585169700 1 -- 172.21.15.121:0/1088036750 wait complete. 2023-11-15T02:59:15.077 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T02:59:15.096 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:59:15.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.254+0000 7fad149a9700 1 Processor -- start 2023-11-15T02:59:15.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.258+0000 7fad149a9700 1 -- start start 2023-11-15T02:59:15.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.258+0000 7fad149a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad10154ca0 0x7fad10155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.258+0000 7fad149a9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fad100a57c0 0x7fad100a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.258+0000 7fad149a9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad100a6160 con 0x7fad101562f0 2023-11-15T02:59:15.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.258+0000 7fad149a9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad100a8280 con 0x7fad100a57c0 2023-11-15T02:59:15.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.258+0000 7fad149a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad100a6d40 con 0x7fad10154ca0 2023-11-15T02:59:15.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.258+0000 7fad0ed9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fad100a57c0 0x7fad100a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.258+0000 7fad0ed9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fad100a57c0 0x7fad100a5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35794/0 (socket says 172.21.15.121:35794) 2023-11-15T02:59:15.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.258+0000 7fad0e59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad10154ca0 0x7fad10155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.258+0000 7fad0ed9d700 1 -- 172.21.15.121:0/3137539790 learned_addr learned my addr 172.21.15.121:0/3137539790 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:15.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.258+0000 7fad0ed9d700 1 -- 172.21.15.121:0/3137539790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad10154ca0 msgr2=0x7fad10155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:15.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.258+0000 7fad0ed9d700 1 --2- 172.21.15.121:0/3137539790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad10154ca0 0x7fad10155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.258+0000 7fad0ed9d700 1 -- 172.21.15.121:0/3137539790 >> v1:172.21.15.190:6789/0 conn(0x7fad101562f0 legacy=0x7fad100a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.258+0000 7fad0ed9d700 1 -- 172.21.15.121:0/3137539790 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad100c4c30 con 0x7fad100a57c0 2023-11-15T02:59:15.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.262+0000 7fad0ed9d700 1 --2- 172.21.15.121:0/3137539790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad100a57c0 0x7fad100a5c70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fad08009f20 tx=0x7fad08009840).ready entity=mon.0 client_cookie=9a37f1ff3506aca server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:15.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.262+0000 7fad0dd9b700 1 -- 172.21.15.121:0/3137539790 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0800e070 con 0x7fad100a57c0 2023-11-15T02:59:15.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.262+0000 7fad0dd9b700 1 -- 172.21.15.121:0/3137539790 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0800b9d0 con 0x7fad100a57c0 2023-11-15T02:59:15.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.262+0000 7fad0dd9b700 1 -- 172.21.15.121:0/3137539790 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0800bd50 con 0x7fad100a57c0 2023-11-15T02:59:15.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.262+0000 7fad149a9700 1 -- 172.21.15.121:0/3137539790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad100a57c0 msgr2=0x7fad100a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.262+0000 7fad149a9700 1 --2- 172.21.15.121:0/3137539790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad100a57c0 0x7fad100a5c70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fad08009f20 tx=0x7fad08009840).stop 2023-11-15T02:59:15.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.262+0000 7fad149a9700 1 -- 172.21.15.121:0/3137539790 shutdown_connections 2023-11-15T02:59:15.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.262+0000 7fad149a9700 1 --2- 172.21.15.121:0/3137539790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad100a57c0 0x7fad100a5c70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.262+0000 7fad149a9700 1 --2- 172.21.15.121:0/3137539790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad10154ca0 0x7fad10155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.262+0000 7fad149a9700 1 -- 172.21.15.121:0/3137539790 >> 172.21.15.121:0/3137539790 conn(0x7fad10062190 msgr2=0x7fad100625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:15.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.262+0000 7fad149a9700 1 -- 172.21.15.121:0/3137539790 shutdown_connections 2023-11-15T02:59:15.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.262+0000 7fad149a9700 1 -- 172.21.15.121:0/3137539790 wait complete. 2023-11-15T02:59:15.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad149a9700 1 Processor -- start 2023-11-15T02:59:15.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad149a9700 1 -- start start 2023-11-15T02:59:15.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad149a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad100a57c0 0x7fad1014a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad149a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad10154ca0 0x7fad1014c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad149a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad101562f0 0x7fad1013c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad149a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1014db40 con 0x7fad101562f0 2023-11-15T02:59:15.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad149a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1014ccb0 con 0x7fad100a57c0 2023-11-15T02:59:15.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad149a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1013cb30 con 0x7fad10154ca0 2023-11-15T02:59:15.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad0ed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad101562f0 0x7fad1013c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad0ed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad101562f0 0x7fad1013c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35798/0 (socket says 172.21.15.121:35798) 2023-11-15T02:59:15.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad0ed9d700 1 -- 172.21.15.121:0/4080789317 learned_addr learned my addr 172.21.15.121:0/4080789317 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:15.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad0e59c700 1 --2- 172.21.15.121:0/4080789317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad100a57c0 0x7fad1014a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad07fff700 1 --2- 172.21.15.121:0/4080789317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad10154ca0 0x7fad1014c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad0ed9d700 1 -- 172.21.15.121:0/4080789317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad10154ca0 msgr2=0x7fad1014c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad0ed9d700 1 --2- 172.21.15.121:0/4080789317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad10154ca0 0x7fad1014c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad0ed9d700 1 -- 172.21.15.121:0/4080789317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad100a57c0 msgr2=0x7fad1014a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad0ed9d700 1 --2- 172.21.15.121:0/4080789317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad100a57c0 0x7fad1014a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad0ed9d700 1 -- 172.21.15.121:0/4080789317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad08009ae0 con 0x7fad101562f0 2023-11-15T02:59:15.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad0ed9d700 1 --2- 172.21.15.121:0/4080789317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad101562f0 0x7fad1013c640 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7fad0800b610 tx=0x7fad100c4ba0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:15.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad067fc700 1 -- 172.21.15.121:0/4080789317 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0800bd50 con 0x7fad101562f0 2023-11-15T02:59:15.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad067fc700 1 -- 172.21.15.121:0/4080789317 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad080092e0 con 0x7fad101562f0 2023-11-15T02:59:15.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad149a9700 1 -- 172.21.15.121:0/4080789317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1014cf30 con 0x7fad101562f0 2023-11-15T02:59:15.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.266+0000 7fad067fc700 1 -- 172.21.15.121:0/4080789317 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0801c650 con 0x7fad101562f0 2023-11-15T02:59:15.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.270+0000 7fad149a9700 1 -- 172.21.15.121:0/4080789317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1014d4f0 con 0x7fad101562f0 2023-11-15T02:59:15.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.270+0000 7fad149a9700 1 -- 172.21.15.121:0/4080789317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1014dcf0 con 0x7fad101562f0 2023-11-15T02:59:15.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.270+0000 7fad067fc700 1 -- 172.21.15.121:0/4080789317 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fad08003040 con 0x7fad101562f0 2023-11-15T02:59:15.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.274+0000 7fad067fc700 1 --2- 172.21.15.121:0/4080789317 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7facf005a510 0x7facf005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.274+0000 7fad067fc700 1 -- 172.21.15.121:0/4080789317 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fad080757d0 con 0x7fad101562f0 2023-11-15T02:59:15.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.274+0000 7fad0e59c700 1 --2- 172.21.15.121:0/4080789317 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7facf005a510 0x7facf005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.274+0000 7fad0e59c700 1 --2- 172.21.15.121:0/4080789317 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7facf005a510 0x7facf005c9d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fad00009bc0 tx=0x7fad00008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:15.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.278+0000 7fad067fc700 1 -- 172.21.15.121:0/4080789317 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fad1014dcf0 con 0x7fad101562f0 2023-11-15T02:59:15.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.474+0000 7fad149a9700 1 -- 172.21.15.121:0/4080789317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad1014dcf0 con 0x7fad101562f0 2023-11-15T02:59:15.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.474+0000 7fad067fc700 1 -- 172.21.15.121:0/4080789317 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4502 (secure 0 0 0) 0x7fad1014dcf0 con 0x7fad101562f0 2023-11-15T02:59:15.482 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:15.482 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":34,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3403224816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3403224816},{"type":"v1","addr":"172.21.15.190:6835","nonce":3403224816}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2088953798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2088953798},{"type":"v1","addr":"172.21.15.190:6833","nonce":2088953798}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5511,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2190514138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2190514138},{"type":"v1","addr":"172.21.15.121:6835","nonce":2190514138}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:58:57.609524+0000","modified":"2023-11-15T02:59:15.067097+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5700},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5700":{"gid":5700,"name":"c","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6837/3406398738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3406398738},{"type":"v1","addr":"172.21.15.121:6837","nonce":3406398738}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-15T02:59:15.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.478+0000 7fad149a9700 1 -- 172.21.15.121:0/4080789317 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7facf005a510 msgr2=0x7facf005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.478+0000 7fad149a9700 1 --2- 172.21.15.121:0/4080789317 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7facf005a510 0x7facf005c9d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fad00009bc0 tx=0x7fad00008040).stop 2023-11-15T02:59:15.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.478+0000 7fad149a9700 1 -- 172.21.15.121:0/4080789317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad101562f0 msgr2=0x7fad1013c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.478+0000 7fad149a9700 1 --2- 172.21.15.121:0/4080789317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad101562f0 0x7fad1013c640 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7fad0800b610 tx=0x7fad100c4ba0).stop 2023-11-15T02:59:15.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.478+0000 7fad149a9700 1 -- 172.21.15.121:0/4080789317 shutdown_connections 2023-11-15T02:59:15.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.478+0000 7fad149a9700 1 --2- 172.21.15.121:0/4080789317 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7facf005a510 0x7facf005c9d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.478+0000 7fad149a9700 1 --2- 172.21.15.121:0/4080789317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad101562f0 0x7fad1013c640 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.478+0000 7fad149a9700 1 --2- 172.21.15.121:0/4080789317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad10154ca0 0x7fad1014c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.478+0000 7fad149a9700 1 --2- 172.21.15.121:0/4080789317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad100a57c0 0x7fad1014a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.478+0000 7fad149a9700 1 -- 172.21.15.121:0/4080789317 >> 172.21.15.121:0/4080789317 conn(0x7fad10062190 msgr2=0x7fad10155610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:15.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.478+0000 7fad149a9700 1 -- 172.21.15.121:0/4080789317 shutdown_connections 2023-11-15T02:59:15.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.478+0000 7fad149a9700 1 -- 172.21.15.121:0/4080789317 wait complete. 2023-11-15T02:59:15.488 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 34 2023-11-15T02:59:15.505 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2023-11-15T02:59:15.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.666+0000 7efc0a49b700 1 Processor -- start 2023-11-15T02:59:15.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.670+0000 7efc0a49b700 1 -- start start 2023-11-15T02:59:15.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.670+0000 7efc0a49b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efc04153c10 0x7efc040bcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.670+0000 7efc0a49b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc040bb6f0 0x7efc04153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.670+0000 7efc0a49b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc040a9250 con 0x7efc04067060 2023-11-15T02:59:15.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.670+0000 7efc0a49b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc040bd5a0 con 0x7efc04153c10 2023-11-15T02:59:15.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.670+0000 7efc0a49b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc040a8920 con 0x7efc040bb6f0 2023-11-15T02:59:15.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.670+0000 7efc03fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efc04153c10 0x7efc040bcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.670+0000 7efc03fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efc04153c10 0x7efc040bcfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35800/0 (socket says 172.21.15.121:35800) 2023-11-15T02:59:15.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.670+0000 7efc03fff700 1 -- 172.21.15.121:0/714822988 learned_addr learned my addr 172.21.15.121:0/714822988 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:15.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.670+0000 7efbfb7fe700 1 --2- 172.21.15.121:0/714822988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc040bb6f0 0x7efc04153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.674+0000 7efbfb7fe700 1 -- 172.21.15.121:0/714822988 >> v2:172.21.15.121:3300/0 conn(0x7efc04153c10 msgr2=0x7efc040bcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:15.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.674+0000 7efbfb7fe700 1 --2- 172.21.15.121:0/714822988 >> v2:172.21.15.121:3300/0 conn(0x7efc04153c10 0x7efc040bcfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.674+0000 7efbfb7fe700 1 -- 172.21.15.121:0/714822988 >> v1:172.21.15.190:6789/0 conn(0x7efc04067060 legacy=0x7efc040632a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.674+0000 7efbfb7fe700 1 -- 172.21.15.121:0/714822988 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc040a8c00 con 0x7efc040bb6f0 2023-11-15T02:59:15.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.674+0000 7efc03fff700 1 --2- 172.21.15.121:0/714822988 >> v2:172.21.15.121:3300/0 conn(0x7efc04153c10 0x7efc040bcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:59:15.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.674+0000 7efbfb7fe700 1 --2- 172.21.15.121:0/714822988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc040bb6f0 0x7efc04153ff0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7efbec009760 tx=0x7efbec009060).ready entity=mon.2 client_cookie=9024e37f8b1660c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:15.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.674+0000 7efc037fe700 1 -- 172.21.15.121:0/714822988 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbec00f510 con 0x7efc040bb6f0 2023-11-15T02:59:15.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.674+0000 7efc037fe700 1 -- 172.21.15.121:0/714822988 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbec00f670 con 0x7efc040bb6f0 2023-11-15T02:59:15.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.674+0000 7efc037fe700 1 -- 172.21.15.121:0/714822988 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbec00f9f0 con 0x7efc040bb6f0 2023-11-15T02:59:15.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.674+0000 7efc0a49b700 1 -- 172.21.15.121:0/714822988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc040bb6f0 msgr2=0x7efc04153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.674+0000 7efc0a49b700 1 --2- 172.21.15.121:0/714822988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc040bb6f0 0x7efc04153ff0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7efbec009760 tx=0x7efbec009060).stop 2023-11-15T02:59:15.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.674+0000 7efc0a49b700 1 -- 172.21.15.121:0/714822988 shutdown_connections 2023-11-15T02:59:15.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.674+0000 7efc0a49b700 1 --2- 172.21.15.121:0/714822988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc040bb6f0 0x7efc04153ff0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.674+0000 7efc0a49b700 1 --2- 172.21.15.121:0/714822988 >> v2:172.21.15.121:3300/0 conn(0x7efc04153c10 0x7efc040bcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.674+0000 7efc0a49b700 1 -- 172.21.15.121:0/714822988 >> 172.21.15.121:0/714822988 conn(0x7efc040591c0 msgr2=0x7efc040c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:15.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.674+0000 7efc0a49b700 1 -- 172.21.15.121:0/714822988 shutdown_connections 2023-11-15T02:59:15.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.674+0000 7efc0a49b700 1 -- 172.21.15.121:0/714822988 wait complete. 2023-11-15T02:59:15.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.678+0000 7efc0a49b700 1 Processor -- start 2023-11-15T02:59:15.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.678+0000 7efc0a49b700 1 -- start start 2023-11-15T02:59:15.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc0a49b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc04067060 0x7efc04159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc0a49b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc040bb6f0 0x7efc04157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc0a49b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc04153c10 0x7efc04152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc0a49b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc04158830 con 0x7efc04067060 2023-11-15T02:59:15.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc0a49b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc04159700 con 0x7efc04153c10 2023-11-15T02:59:15.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc0a49b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc04157d30 con 0x7efc040bb6f0 2023-11-15T02:59:15.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc03fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc04067060 0x7efc04159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc03fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc04067060 0x7efc04159210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35808/0 (socket says 172.21.15.121:35808) 2023-11-15T02:59:15.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc03fff700 1 -- 172.21.15.121:0/80880663 learned_addr learned my addr 172.21.15.121:0/80880663 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:15.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efbfb7fe700 1 --2- 172.21.15.121:0/80880663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc040bb6f0 0x7efc04157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc08a38700 1 --2- 172.21.15.121:0/80880663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc04153c10 0x7efc04152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc03fff700 1 -- 172.21.15.121:0/80880663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc040bb6f0 msgr2=0x7efc04157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc03fff700 1 --2- 172.21.15.121:0/80880663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc040bb6f0 0x7efc04157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc03fff700 1 -- 172.21.15.121:0/80880663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc04153c10 msgr2=0x7efc04152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc03fff700 1 --2- 172.21.15.121:0/80880663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc04153c10 0x7efc04152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc03fff700 1 -- 172.21.15.121:0/80880663 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf4009370 con 0x7efc04067060 2023-11-15T02:59:15.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc03fff700 1 --2- 172.21.15.121:0/80880663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc04067060 0x7efc04159210 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7efbf4006d00 tx=0x7efbf40090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:15.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc00ff9700 1 -- 172.21.15.121:0/80880663 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf40095b0 con 0x7efc04067060 2023-11-15T02:59:15.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc00ff9700 1 -- 172.21.15.121:0/80880663 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf4012600 con 0x7efc04067060 2023-11-15T02:59:15.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc0a49b700 1 -- 172.21.15.121:0/80880663 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbec0092a0 con 0x7efc04067060 2023-11-15T02:59:15.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc00ff9700 1 -- 172.21.15.121:0/80880663 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4012980 con 0x7efc04067060 2023-11-15T02:59:15.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.682+0000 7efc0a49b700 1 -- 172.21.15.121:0/80880663 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc04158270 con 0x7efc04067060 2023-11-15T02:59:15.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.686+0000 7efc00ff9700 1 -- 172.21.15.121:0/80880663 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7efbf401b400 con 0x7efc04067060 2023-11-15T02:59:15.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.690+0000 7efc00ff9700 1 --2- 172.21.15.121:0/80880663 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efbe405a540 0x7efbe405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.690+0000 7efc00ff9700 1 -- 172.21.15.121:0/80880663 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7efbf4074ba0 con 0x7efc04067060 2023-11-15T02:59:15.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.690+0000 7efbfb7fe700 1 --2- 172.21.15.121:0/80880663 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efbe405a540 0x7efbe405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.690+0000 7efbfb7fe700 1 --2- 172.21.15.121:0/80880663 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efbe405a540 0x7efbe405ca00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7efbec009fd0 tx=0x7efbec017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:15.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.690+0000 7efc14823740 1 Processor -- start 2023-11-15T02:59:15.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.690+0000 7efc14823740 1 -- start start 2023-11-15T02:59:15.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.690+0000 7efc14823740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x10e8a70 0x10a5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.690+0000 7efc14823740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10a5990 0x10e8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.690+0000 7efc14823740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x109f150 0x109f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.690+0000 7efc14823740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1002540 con 0x109f150 2023-11-15T02:59:15.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.690+0000 7efc14823740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0xfc5e90 con 0x10a5990 2023-11-15T02:59:15.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.690+0000 7efc14823740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0xfcbd90 con 0x10e8a70 2023-11-15T02:59:15.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc08a38700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x109f150 0x109f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc03fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x10e8a70 0x10a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efbfb7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10a5990 0x10e8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc03fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x10e8a70 0x10a5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:58548/0 (socket says 172.21.15.121:58548) 2023-11-15T02:59:15.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc03fff700 1 -- 172.21.15.121:0/3591827899 learned_addr learned my addr 172.21.15.121:0/3591827899 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:15.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc08a38700 1 -- 172.21.15.121:0/3591827899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x10e8a70 msgr2=0x10a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:15.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc08a38700 1 --2- 172.21.15.121:0/3591827899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x10e8a70 0x10a5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc08a38700 1 -- 172.21.15.121:0/3591827899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10a5990 msgr2=0x10e8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:15.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc08a38700 1 --2- 172.21.15.121:0/3591827899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10a5990 0x10e8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc08a38700 1 -- 172.21.15.121:0/3591827899 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x10304c0 con 0x109f150 2023-11-15T02:59:15.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc03fff700 1 --2- 172.21.15.121:0/3591827899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x10e8a70 0x10a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:59:15.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc08a38700 1 --2- 172.21.15.121:0/3591827899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x109f150 0x109f5d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7efbfc007fd0 tx=0x7efbfc001530).ready entity=mon.0 client_cookie=82d16788f8855a55 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:15.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efbf97fa700 1 -- 172.21.15.121:0/3591827899 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc002240 con 0x109f150 2023-11-15T02:59:15.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efbf97fa700 1 -- 172.21.15.121:0/3591827899 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbfc0023a0 con 0x109f150 2023-11-15T02:59:15.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efbf97fa700 1 -- 172.21.15.121:0/3591827899 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc0017b0 con 0x109f150 2023-11-15T02:59:15.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc14823740 1 -- 172.21.15.121:0/3591827899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x109f150 msgr2=0x109f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc14823740 1 --2- 172.21.15.121:0/3591827899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x109f150 0x109f5d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7efbfc007fd0 tx=0x7efbfc001530).stop 2023-11-15T02:59:15.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc14823740 1 -- 172.21.15.121:0/3591827899 shutdown_connections 2023-11-15T02:59:15.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc14823740 1 --2- 172.21.15.121:0/3591827899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x109f150 0x109f5d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc14823740 1 --2- 172.21.15.121:0/3591827899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10a5990 0x10e8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc14823740 1 --2- 172.21.15.121:0/3591827899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x10e8a70 0x10a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc14823740 1 -- 172.21.15.121:0/3591827899 >> 172.21.15.121:0/3591827899 conn(0xe84680 msgr2=0x10a3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:15.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc14823740 1 -- 172.21.15.121:0/3591827899 shutdown_connections 2023-11-15T02:59:15.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.694+0000 7efc14823740 1 -- 172.21.15.121:0/3591827899 wait complete. 2023-11-15T02:59:15.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.698+0000 7efc14823740 1 -- start start 2023-11-15T02:59:15.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.698+0000 7efc14823740 1 Processor -- start 2023-11-15T02:59:15.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.698+0000 7efc14823740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x109f150 0x10e3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.698+0000 7efc14823740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x10a5990 0x10ec0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.698+0000 7efc14823740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10e8a70 0x10e1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.698+0000 7efc14823740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x10cf3a0 con 0x109f150 2023-11-15T02:59:15.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.698+0000 7efc14823740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x10e44b0 con 0x10e8a70 2023-11-15T02:59:15.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.698+0000 7efc14823740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x10d1070 con 0x10a5990 2023-11-15T02:59:15.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.698+0000 7efc14823740 10 client.0 ll_register_callbacks cb 0x105c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:59:15.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.698+0000 7efc03fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x109f150 0x10e3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.698+0000 7efc03fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x109f150 0x10e3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35816/0 (socket says 172.21.15.121:35816) 2023-11-15T02:59:15.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.698+0000 7efc03fff700 1 -- 172.21.15.121:0/3492212791 learned_addr learned my addr 172.21.15.121:0/3492212791 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:15.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.698+0000 7efbfb7fe700 1 --2- 172.21.15.121:0/3492212791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x10a5990 0x10ec0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.698+0000 7efc08a38700 1 --2- 172.21.15.121:0/3492212791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10e8a70 0x10e1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.698+0000 7efc03fff700 1 -- 172.21.15.121:0/3492212791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x10a5990 msgr2=0x10ec0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:15.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.698+0000 7efc03fff700 1 --2- 172.21.15.121:0/3492212791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x10a5990 0x10ec0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efc03fff700 1 -- 172.21.15.121:0/3492212791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10e8a70 msgr2=0x10e1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:15.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efc03fff700 1 --2- 172.21.15.121:0/3492212791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10e8a70 0x10e1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efc03fff700 1 -- 172.21.15.121:0/3492212791 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc04158270 con 0x109f150 2023-11-15T02:59:15.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efc08a38700 1 --2- 172.21.15.121:0/3492212791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10e8a70 0x10e1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:15.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efc03fff700 1 --2- 172.21.15.121:0/3492212791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x109f150 0x10e3fc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7efbf4048490 tx=0x7efbf401d040).ready entity=mon.0 client_cookie=2b893c89cfa68c33 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:15.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efbe27fc700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:59:15.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efbe27fc700 1 -- 172.21.15.121:0/3492212791 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf40486a0 con 0x109f150 2023-11-15T02:59:15.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efbe27fc700 1 -- 172.21.15.121:0/3492212791 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7efbf4048800 con 0x109f150 2023-11-15T02:59:15.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efc14823740 1 -- 172.21.15.121:0/3492212791 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x101c230 con 0x109f150 2023-11-15T02:59:15.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efbe27fc700 1 -- 172.21.15.121:0/3492212791 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7efbf404b690 con 0x109f150 2023-11-15T02:59:15.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efc14823740 10 client.5736 fetch_fsmap learned FSMap version 34 2023-11-15T02:59:15.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efc14823740 1 -- 172.21.15.121:0/3492212791 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x7efbfc00f040 con 0x109f150 2023-11-15T02:59:15.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efbe27fc700 1 -- 172.21.15.121:0/3492212791 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x101c230 con 0x109f150 2023-11-15T02:59:15.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efbe27fc700 1 -- 172.21.15.121:0/3492212791 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 34) v1 ==== 3100+0+0 (secure 0 0 0) 0x7efbf4050040 con 0x109f150 2023-11-15T02:59:15.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efc14823740 10 client.5736 fetch_fsmap finished waiting for FSMap version 34 2023-11-15T02:59:15.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efc14823740 10 client.5736 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-15T02:59:15.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efc14823740 20 client.5736 populate_metadata read hostname 'smithi121' 2023-11-15T02:59:15.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efc14823740 1 --2- 172.21.15.121:0/3492212791 >> [v2:172.21.15.121:6836/3406398738,v1:172.21.15.121:6837/3406398738] conn(0x10cf7a0 0x10cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efc14823740 4 client.5736 mds_command: new command op to 5700 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T02:59:15.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efc14823740 1 -- 172.21.15.121:0/3492212791 --> [v2:172.21.15.121:6836/3406398738,v1:172.21.15.121:6837/3406398738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1012780 con 0x10cf7a0 2023-11-15T02:59:15.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efbfb7fe700 1 --2- 172.21.15.121:0/3492212791 >> [v2:172.21.15.121:6836/3406398738,v1:172.21.15.121:6837/3406398738] conn(0x10cf7a0 0x10cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efbe27fc700 10 client.5736 ms_handle_connect on v2:172.21.15.121:6836/3406398738 2023-11-15T02:59:15.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.702+0000 7efbfb7fe700 1 --2- 172.21.15.121:0/3492212791 >> [v2:172.21.15.121:6836/3406398738,v1:172.21.15.121:6837/3406398738] conn(0x10cf7a0 0x10cfbb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=94866f09b9b09ac9 server_cookie=62d89a516995bc0c in_seq=0 out_seq=0 2023-11-15T02:59:15.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.706+0000 7efbe27fc700 1 -- 172.21.15.121:0/3492212791 <== mds.0 v2:172.21.15.121:6836/3406398738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x1012780 con 0x10cf7a0 2023-11-15T02:59:15.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.706+0000 7efbe27fc700 10 client.5736 handle_command_reply: tid=0 2023-11-15T02:59:15.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.706+0000 7efc14823740 1 client.5736 shutdown 2023-11-15T02:59:15.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.706+0000 7efc14823740 1 -- 172.21.15.121:0/3492212791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x109f150 msgr2=0x10e3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.706+0000 7efc14823740 1 --2- 172.21.15.121:0/3492212791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x109f150 0x10e3fc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7efbf4048490 tx=0x7efbf401d040).stop 2023-11-15T02:59:15.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.706+0000 7efc14823740 1 -- 172.21.15.121:0/3492212791 shutdown_connections 2023-11-15T02:59:15.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.706+0000 7efc14823740 1 --2- 172.21.15.121:0/3492212791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10e8a70 0x10e1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.706+0000 7efc14823740 1 --2- 172.21.15.121:0/3492212791 >> [v2:172.21.15.121:6836/3406398738,v1:172.21.15.121:6837/3406398738] conn(0x10cf7a0 0x10cfbb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.706+0000 7efc14823740 1 --2- 172.21.15.121:0/3492212791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x10a5990 0x10ec0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.706+0000 7efc14823740 1 --2- 172.21.15.121:0/3492212791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x109f150 0x10e3fc0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.706+0000 7efc14823740 1 -- 172.21.15.121:0/3492212791 >> 172.21.15.121:0/3492212791 conn(0xe84680 msgr2=0x10a3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:15.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.706+0000 7efbe27fc700 0 client.5736 ms_handle_reset on v2:172.21.15.121:6836/3406398738 2023-11-15T02:59:15.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.706+0000 7efc14823740 1 -- 172.21.15.121:0/3492212791 shutdown_connections 2023-11-15T02:59:15.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.706+0000 7efc14823740 1 -- 172.21.15.121:0/3492212791 wait complete. 2023-11-15T02:59:15.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.706+0000 7efc14823740 20 client.5736 trim_cache size 0 max 16384 2023-11-15T02:59:15.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.722+0000 7efc14823740 1 Processor -- start 2023-11-15T02:59:15.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.722+0000 7efc14823740 1 -- start start 2023-11-15T02:59:15.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efc14823740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x109f150 0x10f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efc14823740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x10a5990 0x10f82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efc14823740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10e8a70 0x11490b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efc14823740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x10ce940 con 0x10a5990 2023-11-15T02:59:15.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efc14823740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x109af70 con 0x10e8a70 2023-11-15T02:59:15.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efc14823740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x10ec2c0 con 0x109f150 2023-11-15T02:59:15.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efbfb7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x10a5990 0x10f82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efbfb7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x10a5990 0x10f82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35824/0 (socket says 172.21.15.121:35824) 2023-11-15T02:59:15.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efbfb7fe700 1 -- 172.21.15.121:0/3560033470 learned_addr learned my addr 172.21.15.121:0/3560033470 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:15.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efc03fff700 1 --2- 172.21.15.121:0/3560033470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x109f150 0x10f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efc08a38700 1 --2- 172.21.15.121:0/3560033470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10e8a70 0x11490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efbfb7fe700 1 -- 172.21.15.121:0/3560033470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x109f150 msgr2=0x10f7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:15.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efbfb7fe700 1 --2- 172.21.15.121:0/3560033470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x109f150 0x10f7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efbfb7fe700 1 -- 172.21.15.121:0/3560033470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10e8a70 msgr2=0x11490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:15.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efbfb7fe700 1 --2- 172.21.15.121:0/3560033470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10e8a70 0x11490b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efbfb7fe700 1 -- 172.21.15.121:0/3560033470 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x10ec450 con 0x10a5990 2023-11-15T02:59:15.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efc08a38700 1 --2- 172.21.15.121:0/3560033470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10e8a70 0x11490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:15.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efbfb7fe700 1 --2- 172.21.15.121:0/3560033470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x10a5990 0x10f82d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7efbec006c10 tx=0x7efbec028040).ready entity=mon.0 client_cookie=f86e9df9ead4693b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:15.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efbf9ffb700 1 -- 172.21.15.121:0/3560033470 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbec02b040 con 0x10a5990 2023-11-15T02:59:15.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efbf9ffb700 1 -- 172.21.15.121:0/3560033470 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbec031a20 con 0x10a5990 2023-11-15T02:59:15.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efbf9ffb700 1 -- 172.21.15.121:0/3560033470 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbec031b90 con 0x10a5990 2023-11-15T02:59:15.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efc14823740 1 -- 172.21.15.121:0/3560033470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x10a5990 msgr2=0x10f82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efc14823740 1 --2- 172.21.15.121:0/3560033470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x10a5990 0x10f82d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7efbec006c10 tx=0x7efbec028040).stop 2023-11-15T02:59:15.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efc14823740 1 -- 172.21.15.121:0/3560033470 shutdown_connections 2023-11-15T02:59:15.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efc14823740 1 --2- 172.21.15.121:0/3560033470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10e8a70 0x11490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efc14823740 1 --2- 172.21.15.121:0/3560033470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x10a5990 0x10f82d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efc14823740 1 --2- 172.21.15.121:0/3560033470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x109f150 0x10f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efc14823740 1 -- 172.21.15.121:0/3560033470 >> 172.21.15.121:0/3560033470 conn(0xe84680 msgr2=0x10e78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:15.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efc14823740 1 -- 172.21.15.121:0/3560033470 shutdown_connections 2023-11-15T02:59:15.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efc14823740 1 -- 172.21.15.121:0/3560033470 wait complete. 2023-11-15T02:59:15.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.726+0000 7efc14823740 1 -- start start 2023-11-15T02:59:15.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc14823740 1 Processor -- start 2023-11-15T02:59:15.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc14823740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x109f150 0x1148a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc14823740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10a5990 0x11059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc14823740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x10e8a70 0x1105e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc14823740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x10ea500 con 0x10e8a70 2023-11-15T02:59:15.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc14823740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1106380 con 0x10a5990 2023-11-15T02:59:15.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc14823740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1109120 con 0x109f150 2023-11-15T02:59:15.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc14823740 10 client.0 ll_register_callbacks cb 0x7efbf4012db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:59:15.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc08a38700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x10e8a70 0x1105e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efbfb7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10a5990 0x11059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc08a38700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x10e8a70 0x1105e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35836/0 (socket says 172.21.15.121:35836) 2023-11-15T02:59:15.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc08a38700 1 -- 172.21.15.121:0/219109626 learned_addr learned my addr 172.21.15.121:0/219109626 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:15.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc03fff700 1 --2- 172.21.15.121:0/219109626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x109f150 0x1148a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc08a38700 1 -- 172.21.15.121:0/219109626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x109f150 msgr2=0x1148a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:15.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc08a38700 1 --2- 172.21.15.121:0/219109626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x109f150 0x1148a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc08a38700 1 -- 172.21.15.121:0/219109626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10a5990 msgr2=0x11059a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:15.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc08a38700 1 --2- 172.21.15.121:0/219109626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10a5990 0x11059a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc08a38700 1 -- 172.21.15.121:0/219109626 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc040a8c00 con 0x10e8a70 2023-11-15T02:59:15.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efbfb7fe700 1 --2- 172.21.15.121:0/219109626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10a5990 0x11059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:59:15.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc08a38700 1 --2- 172.21.15.121:0/219109626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x10e8a70 0x1105e90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7efbfc012000 tx=0x7efbfc015000).ready entity=mon.0 client_cookie=2db65ad387312dd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:15.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efbe27fc700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:59:15.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efbe27fc700 1 -- 172.21.15.121:0/219109626 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc001b60 con 0x10e8a70 2023-11-15T02:59:15.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efbe27fc700 1 -- 172.21.15.121:0/219109626 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7efbfc001cc0 con 0x10e8a70 2023-11-15T02:59:15.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc14823740 1 -- 172.21.15.121:0/219109626 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efbfc0023a0 con 0x10e8a70 2023-11-15T02:59:15.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efbe27fc700 1 -- 172.21.15.121:0/219109626 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7efbfc001e80 con 0x10e8a70 2023-11-15T02:59:15.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc14823740 10 client.5742 fetch_fsmap learned FSMap version 34 2023-11-15T02:59:15.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.730+0000 7efc14823740 1 -- 172.21.15.121:0/219109626 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x7efc040c6520 con 0x10e8a70 2023-11-15T02:59:15.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efbe27fc700 1 -- 172.21.15.121:0/219109626 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbfc0023a0 con 0x10e8a70 2023-11-15T02:59:15.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efbe27fc700 1 -- 172.21.15.121:0/219109626 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 34) v1 ==== 3100+0+0 (secure 0 0 0) 0x7efbfc014650 con 0x10e8a70 2023-11-15T02:59:15.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc14823740 10 client.5742 fetch_fsmap finished waiting for FSMap version 34 2023-11-15T02:59:15.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc14823740 10 client.5742 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-15T02:59:15.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc14823740 20 client.5742 populate_metadata read hostname 'smithi121' 2023-11-15T02:59:15.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc14823740 1 --2- 172.21.15.121:0/219109626 >> [v2:172.21.15.121:6836/3406398738,v1:172.21.15.121:6837/3406398738] conn(0x10d2690 0x10e2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc14823740 4 client.5742 mds_command: new command op to 5700 tid=0[{"prefix": "flush journal"}] 2023-11-15T02:59:15.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc14823740 1 -- 172.21.15.121:0/219109626 --> [v2:172.21.15.121:6836/3406398738,v1:172.21.15.121:6837/3406398738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0xedb610 con 0x10d2690 2023-11-15T02:59:15.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc03fff700 1 --2- 172.21.15.121:0/219109626 >> [v2:172.21.15.121:6836/3406398738,v1:172.21.15.121:6837/3406398738] conn(0x10d2690 0x10e2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc03fff700 1 --2- 172.21.15.121:0/219109626 >> [v2:172.21.15.121:6836/3406398738,v1:172.21.15.121:6837/3406398738] conn(0x10d2690 0x10e2ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=fb8c9743280f182d server_cookie=fdabb5a6319a8bf8 in_seq=0 out_seq=0 2023-11-15T02:59:15.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efbe27fc700 10 client.5742 ms_handle_connect on v2:172.21.15.121:6836/3406398738 2023-11-15T02:59:15.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efbe27fc700 1 -- 172.21.15.121:0/219109626 <== mds.0 v2:172.21.15.121:6836/3406398738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0xedb610 con 0x10d2690 2023-11-15T02:59:15.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efbe27fc700 10 client.5742 handle_command_reply: tid=0 2023-11-15T02:59:15.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc14823740 1 client.5742 shutdown 2023-11-15T02:59:15.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc14823740 1 -- 172.21.15.121:0/219109626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x10e8a70 msgr2=0x1105e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc14823740 1 --2- 172.21.15.121:0/219109626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x10e8a70 0x1105e90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7efbfc012000 tx=0x7efbfc015000).stop 2023-11-15T02:59:15.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc14823740 1 -- 172.21.15.121:0/219109626 shutdown_connections 2023-11-15T02:59:15.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc14823740 1 --2- 172.21.15.121:0/219109626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x10e8a70 0x1105e90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc14823740 1 --2- 172.21.15.121:0/219109626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x10a5990 0x11059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc14823740 1 --2- 172.21.15.121:0/219109626 >> [v2:172.21.15.121:6836/3406398738,v1:172.21.15.121:6837/3406398738] conn(0x10d2690 0x10e2ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc14823740 1 --2- 172.21.15.121:0/219109626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x109f150 0x1148a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efbe27fc700 0 client.5742 ms_handle_reset on v2:172.21.15.121:6836/3406398738 2023-11-15T02:59:15.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc14823740 1 -- 172.21.15.121:0/219109626 >> 172.21.15.121:0/219109626 conn(0xe84680 msgr2=0x10f7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:15.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc14823740 1 -- 172.21.15.121:0/219109626 shutdown_connections 2023-11-15T02:59:15.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc14823740 1 -- 172.21.15.121:0/219109626 wait complete. 2023-11-15T02:59:15.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc14823740 20 client.5742 trim_cache size 0 max 16384 2023-11-15T02:59:15.752 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T02:59:15.753 INFO:teuthology.orchestra.run.smithi121.stdout: "message": "", 2023-11-15T02:59:15.753 INFO:teuthology.orchestra.run.smithi121.stdout: "return_code": 0 2023-11-15T02:59:15.753 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T02:59:15.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc0a49b700 1 -- 172.21.15.121:0/80880663 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efbe405a540 msgr2=0x7efbe405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc0a49b700 1 --2- 172.21.15.121:0/80880663 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efbe405a540 0x7efbe405ca00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7efbec009fd0 tx=0x7efbec017000).stop 2023-11-15T02:59:15.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc0a49b700 1 -- 172.21.15.121:0/80880663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc04067060 msgr2=0x7efc04159210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc0a49b700 1 --2- 172.21.15.121:0/80880663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc04067060 0x7efc04159210 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7efbf4006d00 tx=0x7efbf40090f0).stop 2023-11-15T02:59:15.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc0a49b700 1 -- 172.21.15.121:0/80880663 shutdown_connections 2023-11-15T02:59:15.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.734+0000 7efc0a49b700 1 --2- 172.21.15.121:0/80880663 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efbe405a540 0x7efbe405ca00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.738+0000 7efc0a49b700 1 --2- 172.21.15.121:0/80880663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc04153c10 0x7efc04152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.738+0000 7efc0a49b700 1 --2- 172.21.15.121:0/80880663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc040bb6f0 0x7efc04157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.738+0000 7efc0a49b700 1 --2- 172.21.15.121:0/80880663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc04067060 0x7efc04159210 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.738+0000 7efc0a49b700 1 -- 172.21.15.121:0/80880663 >> 172.21.15.121:0/80880663 conn(0x7efc040591c0 msgr2=0x7efc04152bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:15.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.738+0000 7efc0a49b700 1 -- 172.21.15.121:0/80880663 shutdown_connections 2023-11-15T02:59:15.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.738+0000 7efc0a49b700 1 -- 172.21.15.121:0/80880663 wait complete. 2023-11-15T02:59:15.763 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:59:15.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.914+0000 7f4f6431b700 1 Processor -- start 2023-11-15T02:59:15.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.918+0000 7f4f6431b700 1 -- start start 2023-11-15T02:59:15.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.918+0000 7f4f6431b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4f5c154ca0 0x7f4f5c155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.918+0000 7f4f6431b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4f5c1562f0 0x7f4f5c0a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.918+0000 7f4f6431b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f5c0c4c30 con 0x7f4f5c0a8f40 2023-11-15T02:59:15.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.918+0000 7f4f6431b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f5c0a8280 con 0x7f4f5c1562f0 2023-11-15T02:59:15.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.918+0000 7f4f6431b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f5c0a6d40 con 0x7f4f5c154ca0 2023-11-15T02:59:15.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f618b6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4f5c1562f0 0x7f4f5c0a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f618b6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4f5c1562f0 0x7f4f5c0a8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35848/0 (socket says 172.21.15.121:35848) 2023-11-15T02:59:15.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f618b6700 1 -- 172.21.15.121:0/1946920176 learned_addr learned my addr 172.21.15.121:0/1946920176 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:15.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f620b7700 1 --2- 172.21.15.121:0/1946920176 >> v2:172.21.15.190:3300/0 conn(0x7f4f5c154ca0 0x7f4f5c155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f618b6700 1 -- 172.21.15.121:0/1946920176 >> v2:172.21.15.190:3300/0 conn(0x7f4f5c154ca0 msgr2=0x7f4f5c155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:15.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f618b6700 1 --2- 172.21.15.121:0/1946920176 >> v2:172.21.15.190:3300/0 conn(0x7f4f5c154ca0 0x7f4f5c155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f618b6700 1 -- 172.21.15.121:0/1946920176 >> v1:172.21.15.190:6789/0 conn(0x7f4f5c0a8f40 legacy=0x7f4f5c0a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f618b6700 1 -- 172.21.15.121:0/1946920176 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5c0a6f50 con 0x7f4f5c1562f0 2023-11-15T02:59:15.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f618b6700 1 --2- 172.21.15.121:0/1946920176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f5c1562f0 0x7f4f5c0a8a50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f4f5c0a6e80 tx=0x7f4f4c0090f0).ready entity=mon.0 client_cookie=873b0d4fcee3ddbc server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:15.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f610b5700 1 -- 172.21.15.121:0/1946920176 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c00b520 con 0x7f4f5c1562f0 2023-11-15T02:59:15.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f610b5700 1 -- 172.21.15.121:0/1946920176 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4c00b680 con 0x7f4f5c1562f0 2023-11-15T02:59:15.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f610b5700 1 -- 172.21.15.121:0/1946920176 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c00ba00 con 0x7f4f5c1562f0 2023-11-15T02:59:15.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f6431b700 1 -- 172.21.15.121:0/1946920176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f5c1562f0 msgr2=0x7f4f5c0a8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f6431b700 1 --2- 172.21.15.121:0/1946920176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f5c1562f0 0x7f4f5c0a8a50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f4f5c0a6e80 tx=0x7f4f4c0090f0).stop 2023-11-15T02:59:15.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f6431b700 1 -- 172.21.15.121:0/1946920176 shutdown_connections 2023-11-15T02:59:15.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f6431b700 1 --2- 172.21.15.121:0/1946920176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f5c1562f0 0x7f4f5c0a8a50 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f6431b700 1 --2- 172.21.15.121:0/1946920176 >> v2:172.21.15.190:3300/0 conn(0x7f4f5c154ca0 0x7f4f5c155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f6431b700 1 -- 172.21.15.121:0/1946920176 >> 172.21.15.121:0/1946920176 conn(0x7f4f5c062190 msgr2=0x7f4f5c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:15.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f6431b700 1 -- 172.21.15.121:0/1946920176 shutdown_connections 2023-11-15T02:59:15.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.922+0000 7f4f6431b700 1 -- 172.21.15.121:0/1946920176 wait complete. 2023-11-15T02:59:15.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.926+0000 7f4f6431b700 1 Processor -- start 2023-11-15T02:59:15.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.926+0000 7f4f6431b700 1 -- start start 2023-11-15T02:59:15.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.926+0000 7f4f6431b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f5c0a8f40 0x7f4f5c152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.926+0000 7f4f6431b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f5c154ca0 0x7f4f5c153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.926+0000 7f4f6431b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f5c1562f0 0x7f4f5c0aa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.926+0000 7f4f6431b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5c153770 con 0x7f4f5c1562f0 2023-11-15T02:59:15.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.926+0000 7f4f6431b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5c0ab680 con 0x7f4f5c154ca0 2023-11-15T02:59:15.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.926+0000 7f4f6431b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5c0ab960 con 0x7f4f5c0a8f40 2023-11-15T02:59:15.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.930+0000 7f4f628b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f5c1562f0 0x7f4f5c0aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.930+0000 7f4f618b6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f5c154ca0 0x7f4f5c153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.930+0000 7f4f618b6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f5c154ca0 0x7f4f5c153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:42866/0 (socket says 172.21.15.121:42866) 2023-11-15T02:59:15.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.930+0000 7f4f618b6700 1 -- 172.21.15.121:0/3452455419 learned_addr learned my addr 172.21.15.121:0/3452455419 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:15.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.930+0000 7f4f620b7700 1 --2- 172.21.15.121:0/3452455419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f5c0a8f40 0x7f4f5c152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.930+0000 7f4f628b8700 1 -- 172.21.15.121:0/3452455419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f5c0a8f40 msgr2=0x7f4f5c152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.930+0000 7f4f628b8700 1 --2- 172.21.15.121:0/3452455419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f5c0a8f40 0x7f4f5c152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.930+0000 7f4f628b8700 1 -- 172.21.15.121:0/3452455419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f5c154ca0 msgr2=0x7f4f5c153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:15.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.930+0000 7f4f628b8700 1 --2- 172.21.15.121:0/3452455419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f5c154ca0 0x7f4f5c153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:15.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.930+0000 7f4f628b8700 1 -- 172.21.15.121:0/3452455419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c009370 con 0x7f4f5c1562f0 2023-11-15T02:59:15.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.930+0000 7f4f618b6700 1 --2- 172.21.15.121:0/3452455419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f5c154ca0 0x7f4f5c153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:59:15.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.930+0000 7f4f620b7700 1 --2- 172.21.15.121:0/3452455419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f5c0a8f40 0x7f4f5c152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:59:15.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.930+0000 7f4f628b8700 1 --2- 172.21.15.121:0/3452455419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f5c1562f0 0x7f4f5c0aa950 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f4f540013d0 tx=0x7f4f54001530).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:15.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.930+0000 7f4f527fc700 1 -- 172.21.15.121:0/3452455419 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5400e4a0 con 0x7f4f5c1562f0 2023-11-15T02:59:15.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.930+0000 7f4f527fc700 1 -- 172.21.15.121:0/3452455419 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5400e600 con 0x7f4f5c1562f0 2023-11-15T02:59:15.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.930+0000 7f4f6431b700 1 -- 172.21.15.121:0/3452455419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f5c0abc40 con 0x7f4f5c1562f0 2023-11-15T02:59:15.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.930+0000 7f4f527fc700 1 -- 172.21.15.121:0/3452455419 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5400e980 con 0x7f4f5c1562f0 2023-11-15T02:59:15.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.930+0000 7f4f6431b700 1 -- 172.21.15.121:0/3452455419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f5c14e350 con 0x7f4f5c1562f0 2023-11-15T02:59:15.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.930+0000 7f4f6431b700 1 -- 172.21.15.121:0/3452455419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f5c14e540 con 0x7f4f5c1562f0 2023-11-15T02:59:15.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.934+0000 7f4f527fc700 1 -- 172.21.15.121:0/3452455419 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4f540123f0 con 0x7f4f5c1562f0 2023-11-15T02:59:15.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.934+0000 7f4f527fc700 1 --2- 172.21.15.121:0/3452455419 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4f4005a540 0x7f4f4005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:15.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.934+0000 7f4f527fc700 1 -- 172.21.15.121:0/3452455419 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f4f54073fb0 con 0x7f4f5c1562f0 2023-11-15T02:59:15.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.934+0000 7f4f620b7700 1 --2- 172.21.15.121:0/3452455419 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4f4005a540 0x7f4f4005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:15.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.938+0000 7f4f620b7700 1 --2- 172.21.15.121:0/3452455419 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4f4005a540 0x7f4f4005ca00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f4f5c063170 tx=0x7f4f58009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:15.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:15.942+0000 7f4f527fc700 1 -- 172.21.15.121:0/3452455419 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4f540a2af0 con 0x7f4f5c1562f0 2023-11-15T02:59:16.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.146+0000 7f4f6431b700 1 -- 172.21.15.121:0/3452455419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f5c0a7230 con 0x7f4f5c1562f0 2023-11-15T02:59:16.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.146+0000 7f4f527fc700 1 -- 172.21.15.121:0/3452455419 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4505 (secure 0 0 0) 0x7f4f540476e0 con 0x7f4f5c1562f0 2023-11-15T02:59:16.155 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:16.155 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":35,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3403224816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3403224816},{"type":"v1","addr":"172.21.15.190:6835","nonce":3403224816}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2088953798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2088953798},{"type":"v1","addr":"172.21.15.190:6833","nonce":2088953798}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5511,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2190514138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2190514138},{"type":"v1","addr":"172.21.15.121:6835","nonce":2190514138}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:58:57.609524+0000","modified":"2023-11-15T02:59:16.074548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5700},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5700":{"gid":5700,"name":"c","rank":0,"incarnation":34,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.121:6837/3406398738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3406398738},{"type":"v1","addr":"172.21.15.121:6837","nonce":3406398738}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-15T02:59:16.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.150+0000 7f4f6431b700 1 -- 172.21.15.121:0/3452455419 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4f4005a540 msgr2=0x7f4f4005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:16.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.150+0000 7f4f6431b700 1 --2- 172.21.15.121:0/3452455419 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4f4005a540 0x7f4f4005ca00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f4f5c063170 tx=0x7f4f58009060).stop 2023-11-15T02:59:16.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.150+0000 7f4f6431b700 1 -- 172.21.15.121:0/3452455419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f5c1562f0 msgr2=0x7f4f5c0aa950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:16.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.150+0000 7f4f6431b700 1 --2- 172.21.15.121:0/3452455419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f5c1562f0 0x7f4f5c0aa950 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f4f540013d0 tx=0x7f4f54001530).stop 2023-11-15T02:59:16.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.150+0000 7f4f6431b700 1 -- 172.21.15.121:0/3452455419 shutdown_connections 2023-11-15T02:59:16.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.150+0000 7f4f6431b700 1 --2- 172.21.15.121:0/3452455419 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4f4005a540 0x7f4f4005ca00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:16.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.150+0000 7f4f6431b700 1 --2- 172.21.15.121:0/3452455419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f5c1562f0 0x7f4f5c0aa950 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:16.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.150+0000 7f4f6431b700 1 --2- 172.21.15.121:0/3452455419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f5c154ca0 0x7f4f5c153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:16.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.150+0000 7f4f6431b700 1 --2- 172.21.15.121:0/3452455419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f5c0a8f40 0x7f4f5c152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:16.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.150+0000 7f4f6431b700 1 -- 172.21.15.121:0/3452455419 >> 172.21.15.121:0/3452455419 conn(0x7f4f5c062190 msgr2=0x7f4f5c0c69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:16.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.150+0000 7f4f6431b700 1 -- 172.21.15.121:0/3452455419 shutdown_connections 2023-11-15T02:59:16.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.150+0000 7f4f6431b700 1 -- 172.21.15.121:0/3452455419 wait complete. 2023-11-15T02:59:16.160 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 35 2023-11-15T02:59:16.180 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:59:16.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.330+0000 7fedce947700 1 Processor -- start 2023-11-15T02:59:16.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.330+0000 7fedce947700 1 -- start start 2023-11-15T02:59:16.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.334+0000 7fedce947700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fedc8152850 0x7fedc80c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:16.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.334+0000 7fedce947700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc80c1d50 con 0x7fedc80c3280 2023-11-15T02:59:16.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.334+0000 7fedce947700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc815cb80 con 0x7fedc815be30 2023-11-15T02:59:16.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.334+0000 7fedce947700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc80c1860 con 0x7fedc8152850 2023-11-15T02:59:16.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.334+0000 7fedcd945700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fedc80c3280 0x7fedc8152140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60854/0 (socket says 172.21.15.121:60854) 2023-11-15T02:59:16.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.334+0000 7fedcd144700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fedc8152850 0x7fedc80c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:16.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.334+0000 7fedcd945700 1 -- 172.21.15.121:0/1742727611 learned_addr learned my addr 172.21.15.121:0/1742727611 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:16.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.334+0000 7fedcd144700 1 -- 172.21.15.121:0/1742727611 >> v1:172.21.15.190:6789/0 conn(0x7fedc815be30 legacy=0x7fedc815c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:16.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.334+0000 7fedcd144700 1 -- 172.21.15.121:0/1742727611 >> v1:172.21.15.121:6789/0 conn(0x7fedc80c3280 legacy=0x7fedc8152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:16.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.334+0000 7fedcc943700 1 -- 172.21.15.121:0/1742727611 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665655779 0 0) 0x7fedc80c1d50 con 0x7fedc80c3280 2023-11-15T02:59:16.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.334+0000 7fedcd144700 1 -- 172.21.15.121:0/1742727611 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc80a9240 con 0x7fedc8152850 2023-11-15T02:59:16.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.338+0000 7fedcd144700 1 --2- 172.21.15.121:0/1742727611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedc8152850 0x7fedc80c3660 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7fedb8009760 tx=0x7fedb8009060).ready entity=mon.0 client_cookie=c69a90b900bfdcaf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:16.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.338+0000 7fedcc943700 1 -- 172.21.15.121:0/1742727611 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb800b580 con 0x7fedc8152850 2023-11-15T02:59:16.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.338+0000 7fedcc943700 1 -- 172.21.15.121:0/1742727611 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb800b6e0 con 0x7fedc8152850 2023-11-15T02:59:16.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.338+0000 7fedcc943700 1 -- 172.21.15.121:0/1742727611 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb800ba60 con 0x7fedc8152850 2023-11-15T02:59:16.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.338+0000 7fedce947700 1 -- 172.21.15.121:0/1742727611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedc8152850 msgr2=0x7fedc80c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:16.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.338+0000 7fedce947700 1 --2- 172.21.15.121:0/1742727611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedc8152850 0x7fedc80c3660 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7fedb8009760 tx=0x7fedb8009060).stop 2023-11-15T02:59:16.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.338+0000 7fedce947700 1 -- 172.21.15.121:0/1742727611 shutdown_connections 2023-11-15T02:59:16.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.338+0000 7fedce947700 1 --2- 172.21.15.121:0/1742727611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedc8152850 0x7fedc80c3660 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:16.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.338+0000 7fedce947700 1 -- 172.21.15.121:0/1742727611 >> 172.21.15.121:0/1742727611 conn(0x7fedc8062190 msgr2=0x7fedc8062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:16.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.338+0000 7fedce947700 1 -- 172.21.15.121:0/1742727611 shutdown_connections 2023-11-15T02:59:16.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.338+0000 7fedce947700 1 -- 172.21.15.121:0/1742727611 wait complete. 2023-11-15T02:59:16.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.342+0000 7fedce947700 1 Processor -- start 2023-11-15T02:59:16.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.342+0000 7fedce947700 1 -- start start 2023-11-15T02:59:16.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.342+0000 7fedce947700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedc80c3280 0x7fedc8158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:16.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.342+0000 7fedce947700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedc8152850 0x7fedc81571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:16.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedce947700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedc815be30 0x7fedc8062570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:16.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedce947700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc816ca00 con 0x7fedc815be30 2023-11-15T02:59:16.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedce947700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8156d50 con 0x7fedc8152850 2023-11-15T02:59:16.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedce947700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8156420 con 0x7fedc80c3280 2023-11-15T02:59:16.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedce146700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedc815be30 0x7fedc8062570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:16.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedce146700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedc815be30 0x7fedc8062570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35872/0 (socket says 172.21.15.121:35872) 2023-11-15T02:59:16.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedce146700 1 -- 172.21.15.121:0/3213977480 learned_addr learned my addr 172.21.15.121:0/3213977480 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:16.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedcd144700 1 --2- 172.21.15.121:0/3213977480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedc8152850 0x7fedc81571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:16.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedcd945700 1 --2- 172.21.15.121:0/3213977480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedc80c3280 0x7fedc8158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:16.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedce146700 1 -- 172.21.15.121:0/3213977480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedc80c3280 msgr2=0x7fedc8158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:16.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedce146700 1 --2- 172.21.15.121:0/3213977480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedc80c3280 0x7fedc8158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:16.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedce146700 1 -- 172.21.15.121:0/3213977480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedc8152850 msgr2=0x7fedc81571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:16.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedce146700 1 --2- 172.21.15.121:0/3213977480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedc8152850 0x7fedc81571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:16.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedce146700 1 -- 172.21.15.121:0/3213977480 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb80092a0 con 0x7fedc815be30 2023-11-15T02:59:16.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedcd144700 1 --2- 172.21.15.121:0/3213977480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedc8152850 0x7fedc81571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:16.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedce146700 1 --2- 172.21.15.121:0/3213977480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedc815be30 0x7fedc8062570 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7fedc000c7c0 tx=0x7fedc0001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:16.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedbdffb700 1 -- 172.21.15.121:0/3213977480 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0010510 con 0x7fedc815be30 2023-11-15T02:59:16.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedbdffb700 1 -- 172.21.15.121:0/3213977480 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc0010670 con 0x7fedc815be30 2023-11-15T02:59:16.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedce947700 1 -- 172.21.15.121:0/3213977480 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc8156700 con 0x7fedc815be30 2023-11-15T02:59:16.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedbdffb700 1 -- 172.21.15.121:0/3213977480 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc00109f0 con 0x7fedc815be30 2023-11-15T02:59:16.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedce947700 1 -- 172.21.15.121:0/3213977480 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc81593e0 con 0x7fedc815be30 2023-11-15T02:59:16.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.346+0000 7fedce947700 1 -- 172.21.15.121:0/3213977480 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc806bcf0 con 0x7fedc815be30 2023-11-15T02:59:16.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.350+0000 7fedbdffb700 1 -- 172.21.15.121:0/3213977480 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fedc001b400 con 0x7fedc815be30 2023-11-15T02:59:16.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.350+0000 7fedbdffb700 1 --2- 172.21.15.121:0/3213977480 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fedac05a510 0x7fedac05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:16.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.350+0000 7fedbdffb700 1 -- 172.21.15.121:0/3213977480 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fedc0074690 con 0x7fedc815be30 2023-11-15T02:59:16.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.350+0000 7fedcd945700 1 --2- 172.21.15.121:0/3213977480 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fedac05a510 0x7fedac05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:16.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.350+0000 7fedcd945700 1 --2- 172.21.15.121:0/3213977480 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fedac05a510 0x7fedac05c9d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fedc40011e0 tx=0x7fedc4009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:16.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.358+0000 7fedbdffb700 1 -- 172.21.15.121:0/3213977480 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fedc0047d00 con 0x7fedc815be30 2023-11-15T02:59:16.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.550+0000 7fedce947700 1 -- 172.21.15.121:0/3213977480 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedc80c1f80 con 0x7fedc815be30 2023-11-15T02:59:16.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.550+0000 7fedbdffb700 1 -- 172.21.15.121:0/3213977480 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4505 (secure 0 0 0) 0x7fedc004b320 con 0x7fedc815be30 2023-11-15T02:59:16.557 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:16.557 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":35,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3403224816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3403224816},{"type":"v1","addr":"172.21.15.190:6835","nonce":3403224816}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2088953798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2088953798},{"type":"v1","addr":"172.21.15.190:6833","nonce":2088953798}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5511,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2190514138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2190514138},{"type":"v1","addr":"172.21.15.121:6835","nonce":2190514138}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:58:57.609524+0000","modified":"2023-11-15T02:59:16.074548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5700},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5700":{"gid":5700,"name":"c","rank":0,"incarnation":34,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.121:6837/3406398738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3406398738},{"type":"v1","addr":"172.21.15.121:6837","nonce":3406398738}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-15T02:59:16.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.554+0000 7fedce947700 1 -- 172.21.15.121:0/3213977480 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fedac05a510 msgr2=0x7fedac05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:16.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.554+0000 7fedce947700 1 --2- 172.21.15.121:0/3213977480 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fedac05a510 0x7fedac05c9d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fedc40011e0 tx=0x7fedc4009040).stop 2023-11-15T02:59:16.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.554+0000 7fedce947700 1 -- 172.21.15.121:0/3213977480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedc815be30 msgr2=0x7fedc8062570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:16.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.554+0000 7fedce947700 1 --2- 172.21.15.121:0/3213977480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedc815be30 0x7fedc8062570 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7fedc000c7c0 tx=0x7fedc0001510).stop 2023-11-15T02:59:16.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.554+0000 7fedce947700 1 -- 172.21.15.121:0/3213977480 shutdown_connections 2023-11-15T02:59:16.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.554+0000 7fedce947700 1 --2- 172.21.15.121:0/3213977480 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fedac05a510 0x7fedac05c9d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:16.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.554+0000 7fedce947700 1 --2- 172.21.15.121:0/3213977480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedc815be30 0x7fedc8062570 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:16.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.554+0000 7fedce947700 1 --2- 172.21.15.121:0/3213977480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedc8152850 0x7fedc81571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:16.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.554+0000 7fedce947700 1 --2- 172.21.15.121:0/3213977480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedc80c3280 0x7fedc8158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:16.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.554+0000 7fedce947700 1 -- 172.21.15.121:0/3213977480 >> 172.21.15.121:0/3213977480 conn(0x7fedc8062190 msgr2=0x7fedc81597a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:16.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.554+0000 7fedce947700 1 -- 172.21.15.121:0/3213977480 shutdown_connections 2023-11-15T02:59:16.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:16.554+0000 7fedce947700 1 -- 172.21.15.121:0/3213977480 wait complete. 2023-11-15T02:59:16.564 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 35 2023-11-15T02:59:16.584 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump tree /a 0 2023-11-15T02:59:16.754 INFO:teuthology.orchestra.run.smithi121.stdout:[{"ino":1099511627776,"rdev":0,"ctime":"2023-11-15T02:59:11.066259+0000","btime":"2023-11-15T02:59:04.942382+0000","mode":16893,"uid":1000,"gid":1267,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2023-11-15T02:59:11.066259+0000","atime":"2023-11-15T02:59:04.942382+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2023-11-15T02:59:11.066259+0000","num_files":0,"num_subdirs":1,"change_attr":3},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2023-11-15T02:59:11.066259+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2023-11-15T02:59:11.066259+0000"},"version":9,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirfrag":1,"dirty":1},"nref":2,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{},"flocklock":{},"policylock":{},"states":["auth","dirty"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/a","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"7","version":"7","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1},"nref":1,"dentries":[{"path":"a/c","path_ino":1,"snap_first":18446744073709551606,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511627777,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"inodepin":1,"dirty":1},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"a/b","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":false,"is_remote":false,"is_null":true,"is_new":false,"inode":0,"remote_type":"","version":5,"projected_version":5,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirty":1},"nref":1,"lock":{},"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}]}] 2023-11-15T02:59:16.755 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2023-11-15T02:59:11.066259+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "atime": "2023-11-15T02:59:04.942382+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2023-11-15T02:59:04.942382+0000", "change_attr": 3, "client_caps": [], "client_ranges": [], "ctime": "2023-11-15T02:59:11.066259+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511627777, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "a/c", "path_ino": 1, "pins": { "dirty": 1, "inodepin": 1 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 18446744073709551606, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 0, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": true, "is_primary": false, "is_remote": false, "lock": {}, "nref": 1, "path": "a/b", "path_ino": 1, "pins": { "dirty": 1 }, "projected_version": 5, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 5, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/a", "pins": { "child": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2023-11-15T02:59:11.066259+0000", "num_files": 0, "num_subdirs": 1, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1267, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16893, "mtime": "2023-11-15T02:59:11.066259+0000", "nestlock": {}, "nlink": 1, "nref": 2, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "dirfrag": 1, "dirty": 1 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2023-11-15T02:59:11.066259+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 9, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } ] 2023-11-15T02:59:16.756 DEBUG:tasks.cephfs.test_damage:[{'ino': 1099511627776, 'rdev': 0, 'ctime': '2023-11-15T02:59:11.066259+0000', 'btime': '2023-11-15T02:59:04.942382+0000', 'mode': 16893, 'uid': 1000, 'gid': 1267, 'nlink': 1, 'dir_layout': {'dir_hash': 2, 'unused1': 0, 'unused2': 0, 'unused3': 0}, 'layout': {'stripe_unit': 0, 'stripe_count': 0, 'object_size': 0, 'pool_id': -1, 'pool_ns': ''}, 'old_pools': [], 'size': 0, 'truncate_seq': 1, 'truncate_size': 18446744073709551615, 'truncate_from': 0, 'truncate_pending': 0, 'mtime': '2023-11-15T02:59:11.066259+0000', 'atime': '2023-11-15T02:59:04.942382+0000', 'time_warp_seq': 0, 'change_attr': 3, 'export_pin': -1, 'export_ephemeral_random_pin': 0, 'export_ephemeral_distributed_pin': False, 'client_ranges': [], 'dirstat': {'version': 0, 'mtime': '2023-11-15T02:59:11.066259+0000', 'num_files': 0, 'num_subdirs': 1, 'change_attr': 3}, 'rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2023-11-15T02:59:11.066259+0000'}, 'accounted_rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2023-11-15T02:59:11.066259+0000'}, 'version': 9, 'file_data_version': 0, 'xattr_version': 1, 'backtrace_version': 2, 'stray_prior_path': '', 'max_size_ever': 0, 'quota': {'max_bytes': 0, 'max_files': 0}, 'last_scrub_stamp': '0.000000', 'last_scrub_version': 0, 'symlink': '', 'xattrs': [], 'dirfragtree': {'splits': []}, 'old_inodes': [], 'oldest_snap': 18446744073709551614, 'damage_flags': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'dirfrag': 1, 'dirty': 1}, 'nref': 2, 'versionlock': {'gather_set': [], 'state': 'lock', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'authlock': {}, 'linklock': {}, 'dirfragtreelock': {}, 'filelock': {}, 'xattrlock': {}, 'snaplock': {}, 'nestlock': {}, 'flocklock': {}, 'policylock': {}, 'states': ['auth', 'dirty'], 'client_caps': [], 'loner': -1, 'want_loner': -1, 'mds_caps_wanted': [], 'dirfrags': [{'path': '/a', 'dirfrag': '0x10000000000', 'snapid_first': 2, 'projected_version': '7', 'version': '7', 'committing_version': '0', 'committed_version': '0', 'is_rep': False, 'dir_auth': '', 'states': ['auth'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1}, 'nref': 1, 'dentries': [{'path': 'a/c', 'path_ino': 1, 'snap_first': 18446744073709551606, 'snap_last': 18446744073709551614, 'is_primary': True, 'is_remote': False, 'is_null': False, 'is_new': False, 'inode': 1099511627777, 'remote_type': '', 'version': 4, 'projected_version': 4, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'inodepin': 1, 'dirty': 1}, 'nref': 2, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}, {'path': 'a/b', 'path_ino': 1, 'snap_first': 2, 'snap_last': 18446744073709551614, 'is_primary': False, 'is_remote': False, 'is_null': True, 'is_new': False, 'inode': 0, 'remote_type': '', 'version': 5, 'projected_version': 5, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'dirty': 1}, 'nref': 1, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}]}]}] 2023-11-15T02:59:16.756 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-15T02:59:16.757 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T02:59:16.795 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T02:59:16.841 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-15T02:59:16.841 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:59:17.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.010+0000 7fc9f32b9700 1 Processor -- start 2023-11-15T02:59:17.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.010+0000 7fc9f32b9700 1 -- start start 2023-11-15T02:59:17.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.014+0000 7fc9f32b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc9ec0a9260 0x7fc9ec063c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:17.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.014+0000 7fc9f32b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc9ec062220 0x7fc9ec068a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:17.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.014+0000 7fc9f32b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ec151300 con 0x7fc9ec0a9640 2023-11-15T02:59:17.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.014+0000 7fc9f32b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ec0649b0 con 0x7fc9ec0a9260 2023-11-15T02:59:17.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.014+0000 7fc9f32b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ec150d80 con 0x7fc9ec062220 2023-11-15T02:59:17.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.014+0000 7fc9f22b7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc9ec0a9260 0x7fc9ec063c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:17.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.014+0000 7fc9f22b7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc9ec0a9260 0x7fc9ec063c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35888/0 (socket says 172.21.15.121:35888) 2023-11-15T02:59:17.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.014+0000 7fc9f22b7700 1 -- 172.21.15.121:0/4225033322 learned_addr learned my addr 172.21.15.121:0/4225033322 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:17.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.014+0000 7fc9f2ab8700 1 --2- 172.21.15.121:0/4225033322 >> v2:172.21.15.190:3300/0 conn(0x7fc9ec062220 0x7fc9ec068a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:17.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.014+0000 7fc9f22b7700 1 -- 172.21.15.121:0/4225033322 >> v2:172.21.15.190:3300/0 conn(0x7fc9ec062220 msgr2=0x7fc9ec068a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:17.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.014+0000 7fc9f22b7700 1 --2- 172.21.15.121:0/4225033322 >> v2:172.21.15.190:3300/0 conn(0x7fc9ec062220 0x7fc9ec068a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.014+0000 7fc9f22b7700 1 -- 172.21.15.121:0/4225033322 >> v1:172.21.15.190:6789/0 conn(0x7fc9ec0a9640 legacy=0x7fc9ec151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:17.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.014+0000 7fc9f22b7700 1 -- 172.21.15.121:0/4225033322 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ec064150 con 0x7fc9ec0a9260 2023-11-15T02:59:17.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.014+0000 7fc9f22b7700 1 --2- 172.21.15.121:0/4225033322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9ec0a9260 0x7fc9ec063c60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7fc9ec069f40 tx=0x7fc9e80090f0).ready entity=mon.0 client_cookie=8f262a3468a36f0e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:17.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.014+0000 7fc9f12b5700 1 -- 172.21.15.121:0/4225033322 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e800b600 con 0x7fc9ec0a9260 2023-11-15T02:59:17.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.014+0000 7fc9f12b5700 1 -- 172.21.15.121:0/4225033322 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e800b760 con 0x7fc9ec0a9260 2023-11-15T02:59:17.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.014+0000 7fc9f12b5700 1 -- 172.21.15.121:0/4225033322 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e800bae0 con 0x7fc9ec0a9260 2023-11-15T02:59:17.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.014+0000 7fc9f32b9700 1 -- 172.21.15.121:0/4225033322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9ec0a9260 msgr2=0x7fc9ec063c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:17.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.014+0000 7fc9f32b9700 1 --2- 172.21.15.121:0/4225033322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9ec0a9260 0x7fc9ec063c60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7fc9ec069f40 tx=0x7fc9e80090f0).stop 2023-11-15T02:59:17.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.018+0000 7fc9f32b9700 1 -- 172.21.15.121:0/4225033322 shutdown_connections 2023-11-15T02:59:17.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.018+0000 7fc9f32b9700 1 --2- 172.21.15.121:0/4225033322 >> v2:172.21.15.190:3300/0 conn(0x7fc9ec062220 0x7fc9ec068a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.018+0000 7fc9f32b9700 1 --2- 172.21.15.121:0/4225033322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9ec0a9260 0x7fc9ec063c60 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.018+0000 7fc9f32b9700 1 -- 172.21.15.121:0/4225033322 >> 172.21.15.121:0/4225033322 conn(0x7fc9ec065f10 msgr2=0x7fc9ec0662f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:17.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.018+0000 7fc9f32b9700 1 -- 172.21.15.121:0/4225033322 shutdown_connections 2023-11-15T02:59:17.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.018+0000 7fc9f32b9700 1 -- 172.21.15.121:0/4225033322 wait complete. 2023-11-15T02:59:17.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.018+0000 7fc9f32b9700 1 Processor -- start 2023-11-15T02:59:17.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.018+0000 7fc9f32b9700 1 -- start start 2023-11-15T02:59:17.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9f32b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9ec062220 0x7fc9ec144cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:17.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9f32b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9ec0a9260 0x7fc9ec0ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:17.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9f32b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9ec0a9640 0x7fc9ec067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:17.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9f32b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec0bfa70 con 0x7fc9ec062220 2023-11-15T02:59:17.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9f32b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec0c4f10 con 0x7fc9ec0a9260 2023-11-15T02:59:17.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9f32b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec0c51f0 con 0x7fc9ec0a9640 2023-11-15T02:59:17.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9f22b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9ec062220 0x7fc9ec144cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:17.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9f22b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9ec062220 0x7fc9ec144cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35892/0 (socket says 172.21.15.121:35892) 2023-11-15T02:59:17.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9f22b7700 1 -- 172.21.15.121:0/3242096850 learned_addr learned my addr 172.21.15.121:0/3242096850 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:17.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9f2ab8700 1 --2- 172.21.15.121:0/3242096850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9ec0a9640 0x7fc9ec067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:17.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9f1ab6700 1 --2- 172.21.15.121:0/3242096850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9ec0a9260 0x7fc9ec0ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:17.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9f22b7700 1 -- 172.21.15.121:0/3242096850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9ec0a9640 msgr2=0x7fc9ec067f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:17.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9f22b7700 1 --2- 172.21.15.121:0/3242096850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9ec0a9640 0x7fc9ec067f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9f22b7700 1 -- 172.21.15.121:0/3242096850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9ec0a9260 msgr2=0x7fc9ec0ac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:17.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9f22b7700 1 --2- 172.21.15.121:0/3242096850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9ec0a9260 0x7fc9ec0ac7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9f22b7700 1 -- 172.21.15.121:0/3242096850 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e8009370 con 0x7fc9ec062220 2023-11-15T02:59:17.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9f22b7700 1 --2- 172.21.15.121:0/3242096850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9ec062220 0x7fc9ec144cb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fc9e800b3c0 tx=0x7fc9e8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:17.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9e27fc700 1 -- 172.21.15.121:0/3242096850 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e800b8d0 con 0x7fc9ec062220 2023-11-15T02:59:17.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9e27fc700 1 -- 172.21.15.121:0/3242096850 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e800ba30 con 0x7fc9ec062220 2023-11-15T02:59:17.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9e27fc700 1 -- 172.21.15.121:0/3242096850 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e801c670 con 0x7fc9ec062220 2023-11-15T02:59:17.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.022+0000 7fc9f32b9700 1 -- 172.21.15.121:0/3242096850 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ec0c54d0 con 0x7fc9ec062220 2023-11-15T02:59:17.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.026+0000 7fc9f32b9700 1 -- 172.21.15.121:0/3242096850 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ec0acca0 con 0x7fc9ec062220 2023-11-15T02:59:17.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.026+0000 7fc9f32b9700 1 -- 172.21.15.121:0/3242096850 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ec06bcf0 con 0x7fc9ec062220 2023-11-15T02:59:17.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.026+0000 7fc9e27fc700 1 -- 172.21.15.121:0/3242096850 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc9e801ca20 con 0x7fc9ec062220 2023-11-15T02:59:17.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.026+0000 7fc9e27fc700 1 --2- 172.21.15.121:0/3242096850 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc9d005a510 0x7fc9d005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:17.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.026+0000 7fc9e27fc700 1 -- 172.21.15.121:0/3242096850 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fc9e8076560 con 0x7fc9ec062220 2023-11-15T02:59:17.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.026+0000 7fc9f1ab6700 1 --2- 172.21.15.121:0/3242096850 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc9d005a510 0x7fc9d005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:17.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.030+0000 7fc9f1ab6700 1 --2- 172.21.15.121:0/3242096850 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc9d005a510 0x7fc9d005c9d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fc9ec0c6530 tx=0x7fc9dc009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:17.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.034+0000 7fc9e27fc700 1 -- 172.21.15.121:0/3242096850 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc9e8049cf0 con 0x7fc9ec062220 2023-11-15T02:59:17.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.234+0000 7fc9f32b9700 1 -- 172.21.15.121:0/3242096850 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9ec0bf840 con 0x7fc9ec062220 2023-11-15T02:59:17.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.234+0000 7fc9e27fc700 1 -- 172.21.15.121:0/3242096850 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4502 (secure 0 0 0) 0x7fc9e802a020 con 0x7fc9ec062220 2023-11-15T02:59:17.240 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:17.241 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3403224816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3403224816},{"type":"v1","addr":"172.21.15.190:6835","nonce":3403224816}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2088953798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2088953798},{"type":"v1","addr":"172.21.15.190:6833","nonce":2088953798}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5511,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2190514138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2190514138},{"type":"v1","addr":"172.21.15.121:6835","nonce":2190514138}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:58:57.609524+0000","modified":"2023-11-15T02:59:17.084252+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5700},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5700":{"gid":5700,"name":"c","rank":0,"incarnation":34,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.121:6837/3406398738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3406398738},{"type":"v1","addr":"172.21.15.121:6837","nonce":3406398738}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-15T02:59:17.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.234+0000 7fc9f32b9700 1 -- 172.21.15.121:0/3242096850 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc9d005a510 msgr2=0x7fc9d005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:17.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.234+0000 7fc9f32b9700 1 --2- 172.21.15.121:0/3242096850 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc9d005a510 0x7fc9d005c9d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fc9ec0c6530 tx=0x7fc9dc009040).stop 2023-11-15T02:59:17.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.234+0000 7fc9f32b9700 1 -- 172.21.15.121:0/3242096850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9ec062220 msgr2=0x7fc9ec144cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:17.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.234+0000 7fc9f32b9700 1 --2- 172.21.15.121:0/3242096850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9ec062220 0x7fc9ec144cb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fc9e800b3c0 tx=0x7fc9e8017000).stop 2023-11-15T02:59:17.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.234+0000 7fc9f32b9700 1 -- 172.21.15.121:0/3242096850 shutdown_connections 2023-11-15T02:59:17.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.234+0000 7fc9f32b9700 1 --2- 172.21.15.121:0/3242096850 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc9d005a510 0x7fc9d005c9d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.238+0000 7fc9f32b9700 1 --2- 172.21.15.121:0/3242096850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9ec0a9640 0x7fc9ec067f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.238+0000 7fc9f32b9700 1 --2- 172.21.15.121:0/3242096850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9ec0a9260 0x7fc9ec0ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.238+0000 7fc9f32b9700 1 --2- 172.21.15.121:0/3242096850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9ec062220 0x7fc9ec144cb0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.238+0000 7fc9f32b9700 1 -- 172.21.15.121:0/3242096850 >> 172.21.15.121:0/3242096850 conn(0x7fc9ec065f10 msgr2=0x7fc9ec0c5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:17.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.238+0000 7fc9f32b9700 1 -- 172.21.15.121:0/3242096850 shutdown_connections 2023-11-15T02:59:17.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.238+0000 7fc9f32b9700 1 -- 172.21.15.121:0/3242096850 wait complete. 2023-11-15T02:59:17.246 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 36 2023-11-15T02:59:17.261 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:59:17.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.418+0000 7f17e3307700 1 Processor -- start 2023-11-15T02:59:17.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.418+0000 7f17e3307700 1 -- start start 2023-11-15T02:59:17.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.418+0000 7f17e3307700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f17dc153cd0 0x7f17dc067140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:17.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.418+0000 7f17e3307700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17dc1583f0 con 0x7f17dc157ef0 2023-11-15T02:59:17.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.418+0000 7f17e3307700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17dc1586d0 con 0x7f17dc066d60 2023-11-15T02:59:17.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.418+0000 7f17e3307700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17dc0c00f0 con 0x7f17dc153cd0 2023-11-15T02:59:17.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.422+0000 7f17e1b04700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f17dc157ef0 0x7f17dc1535c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60864/0 (socket says 172.21.15.121:60864) 2023-11-15T02:59:17.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.422+0000 7f17e1b04700 1 -- 172.21.15.121:0/2865566387 learned_addr learned my addr 172.21.15.121:0/2865566387 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:17.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.422+0000 7f17e2b06700 1 --2- 172.21.15.121:0/2865566387 >> v2:172.21.15.190:3300/0 conn(0x7f17dc153cd0 0x7f17dc067140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:17.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.422+0000 7f17e1303700 1 -- 172.21.15.121:0/2865566387 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559879090 0 0) 0x7f17dc1583f0 con 0x7f17dc157ef0 2023-11-15T02:59:17.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.422+0000 7f17e1303700 1 -- 172.21.15.121:0/2865566387 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17c80033f0 con 0x7f17dc157ef0 2023-11-15T02:59:17.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.422+0000 7f17e2b06700 1 -- 172.21.15.121:0/2865566387 >> v1:172.21.15.190:6789/0 conn(0x7f17dc066d60 legacy=0x7f17dc1577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:17.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.422+0000 7f17e2b06700 1 -- 172.21.15.121:0/2865566387 >> v1:172.21.15.121:6789/0 conn(0x7f17dc157ef0 legacy=0x7f17dc1535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:17.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.422+0000 7f17e2b06700 1 -- 172.21.15.121:0/2865566387 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17dc065890 con 0x7f17dc153cd0 2023-11-15T02:59:17.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.422+0000 7f17e2b06700 1 --2- 172.21.15.121:0/2865566387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc153cd0 0x7f17dc067140 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f17d4009e90 tx=0x7f17d4009780).ready entity=mon.1 client_cookie=d07a11c03a61c7d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:17.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.422+0000 7f17e1303700 1 -- 172.21.15.121:0/2865566387 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d400e070 con 0x7f17dc153cd0 2023-11-15T02:59:17.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.422+0000 7f17e1303700 1 -- 172.21.15.121:0/2865566387 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d400bb20 con 0x7f17dc153cd0 2023-11-15T02:59:17.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.422+0000 7f17e1303700 1 -- 172.21.15.121:0/2865566387 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d400bea0 con 0x7f17dc153cd0 2023-11-15T02:59:17.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.422+0000 7f17e3307700 1 -- 172.21.15.121:0/2865566387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc153cd0 msgr2=0x7f17dc067140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:17.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.422+0000 7f17e3307700 1 --2- 172.21.15.121:0/2865566387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc153cd0 0x7f17dc067140 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f17d4009e90 tx=0x7f17d4009780).stop 2023-11-15T02:59:17.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.422+0000 7f17e3307700 1 -- 172.21.15.121:0/2865566387 shutdown_connections 2023-11-15T02:59:17.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.422+0000 7f17e3307700 1 --2- 172.21.15.121:0/2865566387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc153cd0 0x7f17dc067140 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.422+0000 7f17e3307700 1 -- 172.21.15.121:0/2865566387 >> 172.21.15.121:0/2865566387 conn(0x7f17dc062190 msgr2=0x7f17dc062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:17.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.422+0000 7f17e3307700 1 -- 172.21.15.121:0/2865566387 shutdown_connections 2023-11-15T02:59:17.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.422+0000 7f17e3307700 1 -- 172.21.15.121:0/2865566387 wait complete. 2023-11-15T02:59:17.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.426+0000 7f17e3307700 1 Processor -- start 2023-11-15T02:59:17.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.426+0000 7f17e3307700 1 -- start start 2023-11-15T02:59:17.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e3307700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17dc066d60 0x7f17dc146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:17.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e3307700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc153cd0 0x7f17dc0c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:17.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e3307700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17dc157ef0 0x7f17dc156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:17.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e3307700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc1474e0 con 0x7f17dc066d60 2023-11-15T02:59:17.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e3307700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc155cb0 con 0x7f17dc153cd0 2023-11-15T02:59:17.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e3307700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc1550a0 con 0x7f17dc157ef0 2023-11-15T02:59:17.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e2305700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17dc066d60 0x7f17dc146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:17.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e2305700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17dc066d60 0x7f17dc146bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35900/0 (socket says 172.21.15.121:35900) 2023-11-15T02:59:17.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e2305700 1 -- 172.21.15.121:0/1308207595 learned_addr learned my addr 172.21.15.121:0/1308207595 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:17.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e1b04700 1 --2- 172.21.15.121:0/1308207595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc153cd0 0x7f17dc0c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:17.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e2b06700 1 --2- 172.21.15.121:0/1308207595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17dc157ef0 0x7f17dc156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:17.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e2305700 1 -- 172.21.15.121:0/1308207595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17dc157ef0 msgr2=0x7f17dc156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:17.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e2305700 1 --2- 172.21.15.121:0/1308207595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17dc157ef0 0x7f17dc156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e2305700 1 -- 172.21.15.121:0/1308207595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc153cd0 msgr2=0x7f17dc0c6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:17.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e2305700 1 --2- 172.21.15.121:0/1308207595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc153cd0 0x7f17dc0c6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e2305700 1 -- 172.21.15.121:0/1308207595 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d4009a20 con 0x7f17dc066d60 2023-11-15T02:59:17.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e1b04700 1 --2- 172.21.15.121:0/1308207595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc153cd0 0x7f17dc0c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:17.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e2305700 1 --2- 172.21.15.121:0/1308207595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17dc066d60 0x7f17dc146bc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f17d800c440 tx=0x7f17d8007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:17.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17d27fc700 1 -- 172.21.15.121:0/1308207595 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d800f470 con 0x7f17dc066d60 2023-11-15T02:59:17.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e3307700 1 -- 172.21.15.121:0/1308207595 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17dc155380 con 0x7f17dc066d60 2023-11-15T02:59:17.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17d27fc700 1 -- 172.21.15.121:0/1308207595 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d800f5d0 con 0x7f17dc066d60 2023-11-15T02:59:17.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17d27fc700 1 -- 172.21.15.121:0/1308207595 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d800f970 con 0x7f17dc066d60 2023-11-15T02:59:17.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e3307700 1 -- 172.21.15.121:0/1308207595 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17dc155940 con 0x7f17dc066d60 2023-11-15T02:59:17.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.430+0000 7f17e3307700 1 -- 172.21.15.121:0/1308207595 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17dc06bcf0 con 0x7f17dc066d60 2023-11-15T02:59:17.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.434+0000 7f17d27fc700 1 -- 172.21.15.121:0/1308207595 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f17d800fd20 con 0x7f17dc066d60 2023-11-15T02:59:17.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.434+0000 7f17d27fc700 1 --2- 172.21.15.121:0/1308207595 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f17c005a510 0x7f17c005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:17.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.434+0000 7f17d27fc700 1 -- 172.21.15.121:0/1308207595 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f17d8073e80 con 0x7f17dc066d60 2023-11-15T02:59:17.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.434+0000 7f17e1b04700 1 --2- 172.21.15.121:0/1308207595 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f17c005a510 0x7f17c005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:17.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.438+0000 7f17e1b04700 1 --2- 172.21.15.121:0/1308207595 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f17c005a510 0x7f17c005c9d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f17cc007fd0 tx=0x7f17cc009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:17.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.442+0000 7f17d27fc700 1 -- 172.21.15.121:0/1308207595 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f17d80475d0 con 0x7f17dc066d60 2023-11-15T02:59:17.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.610+0000 7f17e3307700 1 -- 172.21.15.121:0/1308207595 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17dc0ac810 con 0x7f17dc066d60 2023-11-15T02:59:17.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.610+0000 7f17d27fc700 1 -- 172.21.15.121:0/1308207595 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+17107 (secure 0 0 0) 0x7f17d8017030 con 0x7f17dc066d60 2023-11-15T02:59:17.618 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:17.618 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":49,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:59:12.230103+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:58:55.530878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-15T02:58:56.602591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6837/1945477051":"2023-11-16T02:59:12.230061+0000","172.21.15.121:6836/1945477051":"2023-11-16T02:59:12.230061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:59:17.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.614+0000 7f17e3307700 1 -- 172.21.15.121:0/1308207595 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f17c005a510 msgr2=0x7f17c005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:17.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.614+0000 7f17e3307700 1 --2- 172.21.15.121:0/1308207595 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f17c005a510 0x7f17c005c9d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f17cc007fd0 tx=0x7f17cc009040).stop 2023-11-15T02:59:17.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.614+0000 7f17e3307700 1 -- 172.21.15.121:0/1308207595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17dc066d60 msgr2=0x7f17dc146bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:17.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.614+0000 7f17e3307700 1 --2- 172.21.15.121:0/1308207595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17dc066d60 0x7f17dc146bc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f17d800c440 tx=0x7f17d8007dc0).stop 2023-11-15T02:59:17.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.614+0000 7f17e3307700 1 -- 172.21.15.121:0/1308207595 shutdown_connections 2023-11-15T02:59:17.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.614+0000 7f17e3307700 1 --2- 172.21.15.121:0/1308207595 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f17c005a510 0x7f17c005c9d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.614+0000 7f17e3307700 1 --2- 172.21.15.121:0/1308207595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17dc157ef0 0x7f17dc156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.614+0000 7f17e3307700 1 --2- 172.21.15.121:0/1308207595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc153cd0 0x7f17dc0c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.614+0000 7f17e3307700 1 --2- 172.21.15.121:0/1308207595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17dc066d60 0x7f17dc146bc0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.614+0000 7f17e3307700 1 -- 172.21.15.121:0/1308207595 >> 172.21.15.121:0/1308207595 conn(0x7f17dc062190 msgr2=0x7f17dc14e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:17.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.614+0000 7f17e3307700 1 -- 172.21.15.121:0/1308207595 shutdown_connections 2023-11-15T02:59:17.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.614+0000 7f17e3307700 1 -- 172.21.15.121:0/1308207595 wait complete. 2023-11-15T02:59:17.643 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-15T02:58:55.530878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-15T02:58:56.602591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-15T02:59:17.644 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T02:59:17.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.802+0000 7f9aef59e700 1 Processor -- start 2023-11-15T02:59:17.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.802+0000 7f9aef59e700 1 -- start start 2023-11-15T02:59:17.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aef59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9af00a8d40 0x7f9af00c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:17.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aef59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9af0066520 0x7f9af0066980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:17.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aef59e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af00657f0 con 0x7f9af00a8960 2023-11-15T02:59:17.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aef59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af01583d0 con 0x7f9af0066520 2023-11-15T02:59:17.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aef59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af00c1b60 con 0x7f9af00a8d40 2023-11-15T02:59:17.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aee59c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9af00a8960 0x7f9af015c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60870/0 (socket says 172.21.15.121:60870) 2023-11-15T02:59:17.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aeed9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9af0066520 0x7f9af0066980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:17.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aee59c700 1 -- 172.21.15.121:0/2090819547 learned_addr learned my addr 172.21.15.121:0/2090819547 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:17.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aedd9b700 1 --2- 172.21.15.121:0/2090819547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9af00a8d40 0x7f9af00c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:17.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aeed9d700 1 -- 172.21.15.121:0/2090819547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9af00a8d40 msgr2=0x7f9af00c15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:17.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aeed9d700 1 --2- 172.21.15.121:0/2090819547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9af00a8d40 0x7f9af00c15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aeed9d700 1 -- 172.21.15.121:0/2090819547 >> v1:172.21.15.121:6789/0 conn(0x7f9af00a8960 legacy=0x7f9af015c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:17.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aed59a700 1 -- 172.21.15.121:0/2090819547 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3293990564 0 0) 0x7f9af00657f0 con 0x7f9af00a8960 2023-11-15T02:59:17.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aeed9d700 1 -- 172.21.15.121:0/2090819547 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af0065a40 con 0x7f9af0066520 2023-11-15T02:59:17.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aeed9d700 1 --2- 172.21.15.121:0/2090819547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9af0066520 0x7f9af0066980 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f9ae8009e60 tx=0x7f9ae8009780).ready entity=mon.0 client_cookie=c1c2acf6a0cdc8e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:17.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aed59a700 1 -- 172.21.15.121:0/2090819547 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae800e070 con 0x7f9af0066520 2023-11-15T02:59:17.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aed59a700 1 -- 172.21.15.121:0/2090819547 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae800bb60 con 0x7f9af0066520 2023-11-15T02:59:17.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aed59a700 1 -- 172.21.15.121:0/2090819547 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8012410 con 0x7f9af0066520 2023-11-15T02:59:17.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aef59e700 1 -- 172.21.15.121:0/2090819547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9af0066520 msgr2=0x7f9af0066980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:17.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.806+0000 7f9aef59e700 1 --2- 172.21.15.121:0/2090819547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9af0066520 0x7f9af0066980 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f9ae8009e60 tx=0x7f9ae8009780).stop 2023-11-15T02:59:17.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.810+0000 7f9aef59e700 1 -- 172.21.15.121:0/2090819547 shutdown_connections 2023-11-15T02:59:17.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.810+0000 7f9aef59e700 1 --2- 172.21.15.121:0/2090819547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9af0066520 0x7f9af0066980 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.810+0000 7f9aef59e700 1 --2- 172.21.15.121:0/2090819547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9af00a8d40 0x7f9af00c15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.810+0000 7f9aef59e700 1 -- 172.21.15.121:0/2090819547 >> 172.21.15.121:0/2090819547 conn(0x7f9af0067a60 msgr2=0x7f9af0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:17.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.810+0000 7f9aef59e700 1 -- 172.21.15.121:0/2090819547 shutdown_connections 2023-11-15T02:59:17.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.810+0000 7f9aef59e700 1 -- 172.21.15.121:0/2090819547 wait complete. 2023-11-15T02:59:17.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.810+0000 7f9aef59e700 1 Processor -- start 2023-11-15T02:59:17.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.810+0000 7f9aef59e700 1 -- start start 2023-11-15T02:59:17.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9aef59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9af0066520 0x7f9af00c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:17.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9aef59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9af00a8960 0x7f9af0159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:17.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9aef59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9af00a8d40 0x7f9af0152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:17.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9aef59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af00c63e0 con 0x7f9af00a8960 2023-11-15T02:59:17.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9aef59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af00c5530 con 0x7f9af00a8d40 2023-11-15T02:59:17.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9aef59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af00c57d0 con 0x7f9af0066520 2023-11-15T02:59:17.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9aedd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9af00a8960 0x7f9af0159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:17.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9aedd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9af00a8960 0x7f9af0159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35910/0 (socket says 172.21.15.121:35910) 2023-11-15T02:59:17.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9aedd9b700 1 -- 172.21.15.121:0/4046146766 learned_addr learned my addr 172.21.15.121:0/4046146766 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:17.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9aee59c700 1 --2- 172.21.15.121:0/4046146766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9af0066520 0x7f9af00c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:17.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9aeed9d700 1 --2- 172.21.15.121:0/4046146766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9af00a8d40 0x7f9af0152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:17.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9aedd9b700 1 -- 172.21.15.121:0/4046146766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9af0066520 msgr2=0x7f9af00c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:17.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9aedd9b700 1 --2- 172.21.15.121:0/4046146766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9af0066520 0x7f9af00c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9aedd9b700 1 -- 172.21.15.121:0/4046146766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9af00a8d40 msgr2=0x7f9af0152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:17.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9aedd9b700 1 --2- 172.21.15.121:0/4046146766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9af00a8d40 0x7f9af0152bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:17.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9aedd9b700 1 -- 172.21.15.121:0/4046146766 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae8009a20 con 0x7f9af00a8960 2023-11-15T02:59:17.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9aedd9b700 1 --2- 172.21.15.121:0/4046146766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9af00a8960 0x7f9af0159ef0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f9ae400e900 tx=0x7f9ae4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:17.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9ade7fc700 1 -- 172.21.15.121:0/4046146766 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae400eea0 con 0x7f9af00a8960 2023-11-15T02:59:17.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9ade7fc700 1 -- 172.21.15.121:0/4046146766 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae400c850 con 0x7f9af00a8960 2023-11-15T02:59:17.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9aef59e700 1 -- 172.21.15.121:0/4046146766 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af00c5ab0 con 0x7f9af00a8960 2023-11-15T02:59:17.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9ade7fc700 1 -- 172.21.15.121:0/4046146766 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae400cbd0 con 0x7f9af00a8960 2023-11-15T02:59:17.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.814+0000 7f9aef59e700 1 -- 172.21.15.121:0/4046146766 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af00c6070 con 0x7f9af00a8960 2023-11-15T02:59:17.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.818+0000 7f9aef59e700 1 -- 172.21.15.121:0/4046146766 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af006bcf0 con 0x7f9af00a8960 2023-11-15T02:59:17.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.818+0000 7f9ade7fc700 1 -- 172.21.15.121:0/4046146766 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9ae401b400 con 0x7f9af00a8960 2023-11-15T02:59:17.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.818+0000 7f9ade7fc700 1 --2- 172.21.15.121:0/4046146766 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ad005a510 0x7f9ad005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:17.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.818+0000 7f9ade7fc700 1 -- 172.21.15.121:0/4046146766 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f9ae4074e00 con 0x7f9af00a8960 2023-11-15T02:59:17.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.818+0000 7f9aee59c700 1 --2- 172.21.15.121:0/4046146766 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ad005a510 0x7f9ad005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:17.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.822+0000 7f9aee59c700 1 --2- 172.21.15.121:0/4046146766 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ad005a510 0x7f9ad005c9d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f9ae00011e0 tx=0x7f9ae0009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:17.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.826+0000 7f9ade7fc700 1 -- 172.21.15.121:0/4046146766 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9ae4048520 con 0x7f9af00a8960 2023-11-15T02:59:18.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.994+0000 7f9aef59e700 1 -- 172.21.15.121:0/4046146766 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9af0155e90 con 0x7f9af00a8960 2023-11-15T02:59:18.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.994+0000 7f9ade7fc700 1 -- 172.21.15.121:0/4046146766 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v49) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9ae4016070 con 0x7f9af00a8960 2023-11-15T02:59:18.001 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T02:59:18.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.998+0000 7f9aef59e700 1 -- 172.21.15.121:0/4046146766 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ad005a510 msgr2=0x7f9ad005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:18.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.998+0000 7f9aef59e700 1 --2- 172.21.15.121:0/4046146766 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ad005a510 0x7f9ad005c9d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f9ae00011e0 tx=0x7f9ae0009040).stop 2023-11-15T02:59:18.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.998+0000 7f9aef59e700 1 -- 172.21.15.121:0/4046146766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9af00a8960 msgr2=0x7f9af0159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:18.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.998+0000 7f9aef59e700 1 --2- 172.21.15.121:0/4046146766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9af00a8960 0x7f9af0159ef0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f9ae400e900 tx=0x7f9ae4009060).stop 2023-11-15T02:59:18.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.998+0000 7f9aef59e700 1 -- 172.21.15.121:0/4046146766 shutdown_connections 2023-11-15T02:59:18.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.998+0000 7f9aef59e700 1 --2- 172.21.15.121:0/4046146766 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ad005a510 0x7f9ad005c9d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.998+0000 7f9aef59e700 1 --2- 172.21.15.121:0/4046146766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9af00a8d40 0x7f9af0152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.998+0000 7f9aef59e700 1 --2- 172.21.15.121:0/4046146766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9af00a8960 0x7f9af0159ef0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.998+0000 7f9aef59e700 1 --2- 172.21.15.121:0/4046146766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9af0066520 0x7f9af00c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.998+0000 7f9aef59e700 1 -- 172.21.15.121:0/4046146766 >> 172.21.15.121:0/4046146766 conn(0x7f9af0067a60 msgr2=0x7f9af00aa2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:18.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.998+0000 7f9aef59e700 1 -- 172.21.15.121:0/4046146766 shutdown_connections 2023-11-15T02:59:18.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:17.998+0000 7f9aef59e700 1 -- 172.21.15.121:0/4046146766 wait complete. 2023-11-15T02:59:18.022 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T02:59:18.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.182+0000 7f697ed69700 1 Processor -- start 2023-11-15T02:59:18.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.182+0000 7f697ed69700 1 -- start start 2023-11-15T02:59:18.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.186+0000 7f697ed69700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f69780a8b80 0x7f69780a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.186+0000 7f697ed69700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6978065850 0x7f6978067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.186+0000 7f697ed69700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6978152150 con 0x7f6978067f50 2023-11-15T02:59:18.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.186+0000 7f697ed69700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6978148f70 con 0x7f69780a8b80 2023-11-15T02:59:18.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.186+0000 7f697ed69700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69780683e0 con 0x7f6978065850 2023-11-15T02:59:18.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.186+0000 7f697cb05700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f69780a8b80 0x7f69780a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.186+0000 7f697d306700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6978067f50 0x7f69780bbe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60872/0 (socket says 172.21.15.121:60872) 2023-11-15T02:59:18.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.186+0000 7f697d306700 1 -- 172.21.15.121:0/669799725 learned_addr learned my addr 172.21.15.121:0/669799725 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:18.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.186+0000 7f6977fff700 1 --2- 172.21.15.121:0/669799725 >> v2:172.21.15.190:3300/0 conn(0x7f6978065850 0x7f6978067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.186+0000 7f697cb05700 1 -- 172.21.15.121:0/669799725 >> v2:172.21.15.190:3300/0 conn(0x7f6978065850 msgr2=0x7f6978067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:18.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.186+0000 7f697cb05700 1 --2- 172.21.15.121:0/669799725 >> v2:172.21.15.190:3300/0 conn(0x7f6978065850 0x7f6978067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.186+0000 7f697cb05700 1 -- 172.21.15.121:0/669799725 >> v1:172.21.15.121:6789/0 conn(0x7f6978067f50 legacy=0x7f69780bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:18.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.186+0000 7f697cb05700 1 -- 172.21.15.121:0/669799725 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69781523a0 con 0x7f69780a8b80 2023-11-15T02:59:18.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.186+0000 7f697cb05700 1 --2- 172.21.15.121:0/669799725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69780a8b80 0x7f69780a8f60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f69780676c0 tx=0x7f696c009060).ready entity=mon.0 client_cookie=13791d3d1af90042 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:18.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.190+0000 7f69777fe700 1 -- 172.21.15.121:0/669799725 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c00b580 con 0x7f69780a8b80 2023-11-15T02:59:18.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.190+0000 7f69777fe700 1 -- 172.21.15.121:0/669799725 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696c00b6e0 con 0x7f69780a8b80 2023-11-15T02:59:18.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.190+0000 7f69777fe700 1 -- 172.21.15.121:0/669799725 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c00ba60 con 0x7f69780a8b80 2023-11-15T02:59:18.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.190+0000 7f697ed69700 1 -- 172.21.15.121:0/669799725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69780a8b80 msgr2=0x7f69780a8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:18.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.190+0000 7f697ed69700 1 --2- 172.21.15.121:0/669799725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69780a8b80 0x7f69780a8f60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f69780676c0 tx=0x7f696c009060).stop 2023-11-15T02:59:18.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.190+0000 7f697ed69700 1 -- 172.21.15.121:0/669799725 shutdown_connections 2023-11-15T02:59:18.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.190+0000 7f697ed69700 1 --2- 172.21.15.121:0/669799725 >> v2:172.21.15.190:3300/0 conn(0x7f6978065850 0x7f6978067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.190+0000 7f697ed69700 1 --2- 172.21.15.121:0/669799725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69780a8b80 0x7f69780a8f60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.190+0000 7f697ed69700 1 -- 172.21.15.121:0/669799725 >> 172.21.15.121:0/669799725 conn(0x7f69780591c0 msgr2=0x7f69780c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:18.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.190+0000 7f697ed69700 1 -- 172.21.15.121:0/669799725 shutdown_connections 2023-11-15T02:59:18.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.190+0000 7f697ed69700 1 -- 172.21.15.121:0/669799725 wait complete. 2023-11-15T02:59:18.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.190+0000 7f697ed69700 1 Processor -- start 2023-11-15T02:59:18.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.190+0000 7f697ed69700 1 -- start start 2023-11-15T02:59:18.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f697ed69700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6978065850 0x7f697814b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f697ed69700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6978067f50 0x7f697813f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f697ed69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69780a8b80 0x7f69780bfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f697ed69700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697814a090 con 0x7f6978067f50 2023-11-15T02:59:18.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f697ed69700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697814b900 con 0x7f6978065850 2023-11-15T02:59:18.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f697ed69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6978149480 con 0x7f69780a8b80 2023-11-15T02:59:18.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f6977fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6978067f50 0x7f697813f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f6977fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6978067f50 0x7f697813f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35938/0 (socket says 172.21.15.121:35938) 2023-11-15T02:59:18.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f6977fff700 1 -- 172.21.15.121:0/3657403263 learned_addr learned my addr 172.21.15.121:0/3657403263 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:18.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f697d306700 1 --2- 172.21.15.121:0/3657403263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69780a8b80 0x7f69780bfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f697cb05700 1 --2- 172.21.15.121:0/3657403263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6978065850 0x7f697814b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f6977fff700 1 -- 172.21.15.121:0/3657403263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69780a8b80 msgr2=0x7f69780bfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:18.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f6977fff700 1 --2- 172.21.15.121:0/3657403263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69780a8b80 0x7f69780bfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f6977fff700 1 -- 172.21.15.121:0/3657403263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6978065850 msgr2=0x7f697814b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:18.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f6977fff700 1 --2- 172.21.15.121:0/3657403263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6978065850 0x7f697814b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f6977fff700 1 -- 172.21.15.121:0/3657403263 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696c0092a0 con 0x7f6978067f50 2023-11-15T02:59:18.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f6977fff700 1 --2- 172.21.15.121:0/3657403263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6978067f50 0x7f697813f1e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f696400d890 tx=0x7f6964008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:18.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f6974ff9700 1 -- 172.21.15.121:0/3657403263 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69640125f0 con 0x7f6978067f50 2023-11-15T02:59:18.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f6974ff9700 1 -- 172.21.15.121:0/3657403263 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6964012750 con 0x7f6978067f50 2023-11-15T02:59:18.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f6974ff9700 1 -- 172.21.15.121:0/3657403263 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6964012ad0 con 0x7f6978067f50 2023-11-15T02:59:18.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f697ed69700 1 -- 172.21.15.121:0/3657403263 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6978149760 con 0x7f6978067f50 2023-11-15T02:59:18.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.194+0000 7f697ed69700 1 -- 172.21.15.121:0/3657403263 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6978149d20 con 0x7f6978067f50 2023-11-15T02:59:18.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.198+0000 7f697ed69700 1 -- 172.21.15.121:0/3657403263 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6978047060 con 0x7f6978067f50 2023-11-15T02:59:18.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.198+0000 7f6974ff9700 1 -- 172.21.15.121:0/3657403263 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6964013480 con 0x7f6978067f50 2023-11-15T02:59:18.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.198+0000 7f6974ff9700 1 --2- 172.21.15.121:0/3657403263 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f695c05a510 0x7f695c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.198+0000 7f6974ff9700 1 -- 172.21.15.121:0/3657403263 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f6964074630 con 0x7f6978067f50 2023-11-15T02:59:18.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.202+0000 7f697cb05700 1 --2- 172.21.15.121:0/3657403263 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f695c05a510 0x7f695c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.202+0000 7f697cb05700 1 --2- 172.21.15.121:0/3657403263 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f695c05a510 0x7f695c05c9d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f696c009fd0 tx=0x7f696c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:18.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.206+0000 7f6974ff9700 1 -- 172.21.15.121:0/3657403263 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6964047ca0 con 0x7f6978067f50 2023-11-15T02:59:18.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.370+0000 7f697ed69700 1 -- 172.21.15.121:0/3657403263 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6978063b60 con 0x7f6978067f50 2023-11-15T02:59:18.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.370+0000 7f6974ff9700 1 -- 172.21.15.121:0/3657403263 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v49) v1 ==== 92+0+10 (secure 0 0 0) 0x7f696404b2c0 con 0x7f6978067f50 2023-11-15T02:59:18.379 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T02:59:18.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.374+0000 7f697ed69700 1 -- 172.21.15.121:0/3657403263 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f695c05a510 msgr2=0x7f695c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:18.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.374+0000 7f697ed69700 1 --2- 172.21.15.121:0/3657403263 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f695c05a510 0x7f695c05c9d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f696c009fd0 tx=0x7f696c017000).stop 2023-11-15T02:59:18.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.374+0000 7f697ed69700 1 -- 172.21.15.121:0/3657403263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6978067f50 msgr2=0x7f697813f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:18.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.374+0000 7f697ed69700 1 --2- 172.21.15.121:0/3657403263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6978067f50 0x7f697813f1e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f696400d890 tx=0x7f6964008040).stop 2023-11-15T02:59:18.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.374+0000 7f697ed69700 1 -- 172.21.15.121:0/3657403263 shutdown_connections 2023-11-15T02:59:18.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.374+0000 7f697ed69700 1 --2- 172.21.15.121:0/3657403263 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f695c05a510 0x7f695c05c9d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.374+0000 7f697ed69700 1 --2- 172.21.15.121:0/3657403263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69780a8b80 0x7f69780bfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.374+0000 7f697ed69700 1 --2- 172.21.15.121:0/3657403263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6978067f50 0x7f697813f1e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.374+0000 7f697ed69700 1 --2- 172.21.15.121:0/3657403263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6978065850 0x7f697814b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.374+0000 7f697ed69700 1 -- 172.21.15.121:0/3657403263 >> 172.21.15.121:0/3657403263 conn(0x7f69780591c0 msgr2=0x7f6978062190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:18.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.374+0000 7f697ed69700 1 -- 172.21.15.121:0/3657403263 shutdown_connections 2023-11-15T02:59:18.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.374+0000 7f697ed69700 1 -- 172.21.15.121:0/3657403263 wait complete. 2023-11-15T02:59:18.402 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-15T02:59:18.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.566+0000 7f096e769700 1 Processor -- start 2023-11-15T02:59:18.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.566+0000 7f096e769700 1 -- start start 2023-11-15T02:59:18.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.570+0000 7f096e769700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f09680682a0 0x7f09680c10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.570+0000 7f096e769700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0968153700 0x7f0968153b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.570+0000 7f096e769700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09680c15a0 con 0x7f0968067ec0 2023-11-15T02:59:18.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.570+0000 7f096e769700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09680686e0 con 0x7f09680682a0 2023-11-15T02:59:18.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.570+0000 7f096e769700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09680c0c20 con 0x7f0968153700 2023-11-15T02:59:18.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.570+0000 7f09677fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f09680682a0 0x7f09680c10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.570+0000 7f096cd06700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0968153700 0x7f0968153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.570+0000 7f0967fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f0968067ec0 0x7f0968062c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:40392/0 (socket says 172.21.15.121:40392) 2023-11-15T02:59:18.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.570+0000 7f0967fff700 1 -- 172.21.15.121:0/3291175026 learned_addr learned my addr 172.21.15.121:0/3291175026 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:18.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.570+0000 7f0966ffd700 1 -- 172.21.15.121:0/3291175026 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 228252454 0 0) 0x7f0958001110 con 0x7f0968067ec0 2023-11-15T02:59:18.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.570+0000 7f0966ffd700 1 -- 172.21.15.121:0/3291175026 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09500033f0 con 0x7f0968067ec0 2023-11-15T02:59:18.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.570+0000 7f09677fe700 1 -- 172.21.15.121:0/3291175026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0968153700 msgr2=0x7f0968153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:18.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.570+0000 7f09677fe700 1 --2- 172.21.15.121:0/3291175026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0968153700 0x7f0968153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.570+0000 7f09677fe700 1 -- 172.21.15.121:0/3291175026 >> v1:172.21.15.190:6789/0 conn(0x7f0968067ec0 legacy=0x7f0968062c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:18.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.570+0000 7f09677fe700 1 -- 172.21.15.121:0/3291175026 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0968158730 con 0x7f09680682a0 2023-11-15T02:59:18.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.570+0000 7f09677fe700 1 --2- 172.21.15.121:0/3291175026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09680682a0 0x7f09680c10b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f095c009760 tx=0x7f095c009060).ready entity=mon.1 client_cookie=3cd7c4e1001ecdbf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:18.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.574+0000 7f0966ffd700 1 -- 172.21.15.121:0/3291175026 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c00d4a0 con 0x7f09680682a0 2023-11-15T02:59:18.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.574+0000 7f0966ffd700 1 -- 172.21.15.121:0/3291175026 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095c00d600 con 0x7f09680682a0 2023-11-15T02:59:18.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.574+0000 7f0966ffd700 1 -- 172.21.15.121:0/3291175026 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c00d980 con 0x7f09680682a0 2023-11-15T02:59:18.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.574+0000 7f096e769700 1 -- 172.21.15.121:0/3291175026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09680682a0 msgr2=0x7f09680c10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:18.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.574+0000 7f096e769700 1 --2- 172.21.15.121:0/3291175026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09680682a0 0x7f09680c10b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f095c009760 tx=0x7f095c009060).stop 2023-11-15T02:59:18.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.574+0000 7f096e769700 1 -- 172.21.15.121:0/3291175026 shutdown_connections 2023-11-15T02:59:18.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.574+0000 7f096e769700 1 --2- 172.21.15.121:0/3291175026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0968153700 0x7f0968153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.574+0000 7f096e769700 1 --2- 172.21.15.121:0/3291175026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09680682a0 0x7f09680c10b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.574+0000 7f096e769700 1 -- 172.21.15.121:0/3291175026 >> 172.21.15.121:0/3291175026 conn(0x7f09680cad40 msgr2=0x7f09680cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:18.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.574+0000 7f096e769700 1 -- 172.21.15.121:0/3291175026 shutdown_connections 2023-11-15T02:59:18.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.574+0000 7f096e769700 1 -- 172.21.15.121:0/3291175026 wait complete. 2023-11-15T02:59:18.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.578+0000 7f096e769700 1 Processor -- start 2023-11-15T02:59:18.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.578+0000 7f096e769700 1 -- start start 2023-11-15T02:59:18.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.578+0000 7f096e769700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0968067ec0 0x7f0968145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.578+0000 7f096e769700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09680682a0 0x7f0968156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.578+0000 7f096e769700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0968153700 0x7f0968156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.578+0000 7f096e769700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0968156f20 con 0x7f0968067ec0 2023-11-15T02:59:18.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.582+0000 7f096e769700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09680c45e0 con 0x7f09680682a0 2023-11-15T02:59:18.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.582+0000 7f096e769700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09680c3a60 con 0x7f0968153700 2023-11-15T02:59:18.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.582+0000 7f0967fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0968067ec0 0x7f0968145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.582+0000 7f0967fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0968067ec0 0x7f0968145530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35950/0 (socket says 172.21.15.121:35950) 2023-11-15T02:59:18.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.582+0000 7f0967fff700 1 -- 172.21.15.121:0/4028967699 learned_addr learned my addr 172.21.15.121:0/4028967699 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:18.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.582+0000 7f09677fe700 1 --2- 172.21.15.121:0/4028967699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09680682a0 0x7f0968156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.582+0000 7f096cd06700 1 --2- 172.21.15.121:0/4028967699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0968153700 0x7f0968156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.582+0000 7f0967fff700 1 -- 172.21.15.121:0/4028967699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0968153700 msgr2=0x7f0968156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:18.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.582+0000 7f0967fff700 1 --2- 172.21.15.121:0/4028967699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0968153700 0x7f0968156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.582+0000 7f0967fff700 1 -- 172.21.15.121:0/4028967699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09680682a0 msgr2=0x7f0968156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:18.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.582+0000 7f0967fff700 1 --2- 172.21.15.121:0/4028967699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09680682a0 0x7f0968156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.582+0000 7f0967fff700 1 -- 172.21.15.121:0/4028967699 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095c0092a0 con 0x7f0968067ec0 2023-11-15T02:59:18.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.582+0000 7f0967fff700 1 --2- 172.21.15.121:0/4028967699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0968067ec0 0x7f0968145530 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f095800c4d0 tx=0x7f0958007af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:18.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.582+0000 7f094ffff700 1 -- 172.21.15.121:0/4028967699 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095800ce40 con 0x7f0968067ec0 2023-11-15T02:59:18.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.582+0000 7f094ffff700 1 -- 172.21.15.121:0/4028967699 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09580072e0 con 0x7f0968067ec0 2023-11-15T02:59:18.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.582+0000 7f096e769700 1 -- 172.21.15.121:0/4028967699 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09680c3d40 con 0x7f0968067ec0 2023-11-15T02:59:18.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.582+0000 7f094ffff700 1 -- 172.21.15.121:0/4028967699 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09580057a0 con 0x7f0968067ec0 2023-11-15T02:59:18.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.582+0000 7f096e769700 1 -- 172.21.15.121:0/4028967699 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09680c4300 con 0x7f0968067ec0 2023-11-15T02:59:18.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.586+0000 7f096e769700 1 -- 172.21.15.121:0/4028967699 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0968153d70 con 0x7f0968067ec0 2023-11-15T02:59:18.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.586+0000 7f094ffff700 1 -- 172.21.15.121:0/4028967699 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0958005bd0 con 0x7f0968067ec0 2023-11-15T02:59:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.586+0000 7f094ffff700 1 --2- 172.21.15.121:0/4028967699 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f095405a510 0x7f095405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.586+0000 7f094ffff700 1 -- 172.21.15.121:0/4028967699 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f0958073d90 con 0x7f0968067ec0 2023-11-15T02:59:18.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.586+0000 7f09677fe700 1 --2- 172.21.15.121:0/4028967699 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f095405a510 0x7f095405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.590+0000 7f09677fe700 1 --2- 172.21.15.121:0/4028967699 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f095405a510 0x7f095405c9d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f095c00d260 tx=0x7f095c016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:18.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.594+0000 7f094ffff700 1 -- 172.21.15.121:0/4028967699 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0968153d70 con 0x7f0968067ec0 2023-11-15T02:59:18.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.766+0000 7f096e769700 1 -- 172.21.15.121:0/4028967699 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f09680c15a0 con 0x7f0968067ec0 2023-11-15T02:59:18.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.770+0000 7f094ffff700 1 -- 172.21.15.121:0/4028967699 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f09680c15a0 con 0x7f0968067ec0 2023-11-15T02:59:18.776 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T02:59:18.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.770+0000 7f096e769700 1 -- 172.21.15.121:0/4028967699 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f095405a510 msgr2=0x7f095405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:18.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.770+0000 7f096e769700 1 --2- 172.21.15.121:0/4028967699 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f095405a510 0x7f095405c9d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f095c00d260 tx=0x7f095c016000).stop 2023-11-15T02:59:18.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.770+0000 7f096e769700 1 -- 172.21.15.121:0/4028967699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0968067ec0 msgr2=0x7f0968145530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:18.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.770+0000 7f096e769700 1 --2- 172.21.15.121:0/4028967699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0968067ec0 0x7f0968145530 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f095800c4d0 tx=0x7f0958007af0).stop 2023-11-15T02:59:18.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.770+0000 7f096e769700 1 -- 172.21.15.121:0/4028967699 shutdown_connections 2023-11-15T02:59:18.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.770+0000 7f096e769700 1 --2- 172.21.15.121:0/4028967699 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f095405a510 0x7f095405c9d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.770+0000 7f096e769700 1 --2- 172.21.15.121:0/4028967699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0968153700 0x7f0968156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.770+0000 7f096e769700 1 --2- 172.21.15.121:0/4028967699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09680682a0 0x7f0968156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.770+0000 7f096e769700 1 --2- 172.21.15.121:0/4028967699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0968067ec0 0x7f0968145530 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.770+0000 7f096e769700 1 -- 172.21.15.121:0/4028967699 >> 172.21.15.121:0/4028967699 conn(0x7f09680cad40 msgr2=0x7f09680c9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:18.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.774+0000 7f096e769700 1 -- 172.21.15.121:0/4028967699 shutdown_connections 2023-11-15T02:59:18.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.774+0000 7f096e769700 1 -- 172.21.15.121:0/4028967699 wait complete. 2023-11-15T02:59:18.800 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-15T02:59:18.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.954+0000 7f05394ff700 1 Processor -- start 2023-11-15T02:59:18.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.954+0000 7f05394ff700 1 -- start start 2023-11-15T02:59:18.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f05394ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0534153e60 0x7f0534158420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f05394ff700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0534158910 con 0x7f05340c5ba0 2023-11-15T02:59:18.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f05394ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05340c7280 con 0x7f0534154240 2023-11-15T02:59:18.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f05394ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05341553c0 con 0x7f0534153e60 2023-11-15T02:59:18.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f05337fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f05340c5ba0 0x7f05340c7a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60880/0 (socket says 172.21.15.121:60880) 2023-11-15T02:59:18.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f05337fe700 1 -- 172.21.15.121:0/1297982697 learned_addr learned my addr 172.21.15.121:0/1297982697 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:18.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0532ffd700 1 --2- 172.21.15.121:0/1297982697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0534153e60 0x7f0534158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0531ffb700 1 -- 172.21.15.121:0/1297982697 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2341539654 0 0) 0x7f052c001490 con 0x7f05340c5ba0 2023-11-15T02:59:18.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0531ffb700 1 -- 172.21.15.121:0/1297982697 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f051c0033f0 con 0x7f05340c5ba0 2023-11-15T02:59:18.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0531ffb700 1 -- 172.21.15.121:0/1297982697 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2316584713 0 0) 0x7f05340c7280 con 0x7f0534154240 2023-11-15T02:59:18.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0531ffb700 1 -- 172.21.15.121:0/1297982697 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f052c001490 con 0x7f0534154240 2023-11-15T02:59:18.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0531ffb700 1 -- 172.21.15.121:0/1297982697 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2786118960 0 0) 0x7f051c0033f0 con 0x7f05340c5ba0 2023-11-15T02:59:18.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0531ffb700 1 -- 172.21.15.121:0/1297982697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0534153e60 msgr2=0x7f0534158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:18.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0531ffb700 1 --2- 172.21.15.121:0/1297982697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0534153e60 0x7f0534158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0531ffb700 1 -- 172.21.15.121:0/1297982697 >> v1:172.21.15.190:6789/0 conn(0x7f0534154240 legacy=0x7f05340657f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:18.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0531ffb700 1 -- 172.21.15.121:0/1297982697 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0534154b20 con 0x7f05340c5ba0 2023-11-15T02:59:18.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0531ffb700 1 -- 172.21.15.121:0/1297982697 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f052c001980 con 0x7f05340c5ba0 2023-11-15T02:59:18.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0532ffd700 1 --2- 172.21.15.121:0/1297982697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0534153e60 0x7f0534158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:59:18.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0531ffb700 1 -- 172.21.15.121:0/1297982697 >> v1:172.21.15.121:6789/0 conn(0x7f05340c5ba0 legacy=0x7f05340c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:18.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0531ffb700 1 --2- 172.21.15.121:0/1297982697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f051c003aa0 0x7f051c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0531ffb700 1 --2- 172.21.15.121:0/1297982697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0534153e60 0x7f051c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0531ffb700 1 --2- 172.21.15.121:0/1297982697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f051c004b80 0x7f051c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0531ffb700 1 -- 172.21.15.121:0/1297982697 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05341553c0 con 0x7f051c004b80 2023-11-15T02:59:18.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0531ffb700 1 -- 172.21.15.121:0/1297982697 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051c00b8e0 con 0x7f051c003aa0 2023-11-15T02:59:18.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0531ffb700 1 -- 172.21.15.121:0/1297982697 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051c00ba70 con 0x7f0534153e60 2023-11-15T02:59:18.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f05337fe700 1 --2- 172.21.15.121:0/1297982697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f051c004b80 0x7f051c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f0532ffd700 1 --2- 172.21.15.121:0/1297982697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f051c003aa0 0x7f051c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.958+0000 7f05327fc700 1 --2- 172.21.15.121:0/1297982697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0534153e60 0x7f051c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.962+0000 7f05337fe700 1 -- 172.21.15.121:0/1297982697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0534153e60 msgr2=0x7f051c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:18.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.962+0000 7f05337fe700 1 --2- 172.21.15.121:0/1297982697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0534153e60 0x7f051c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.962+0000 7f05337fe700 1 -- 172.21.15.121:0/1297982697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f051c003aa0 msgr2=0x7f051c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:18.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.962+0000 7f05337fe700 1 --2- 172.21.15.121:0/1297982697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f051c003aa0 0x7f051c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.962+0000 7f05337fe700 1 -- 172.21.15.121:0/1297982697 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05240092a0 con 0x7f051c004b80 2023-11-15T02:59:18.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.962+0000 7f05337fe700 1 --2- 172.21.15.121:0/1297982697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f051c004b80 0x7f051c005000 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f052c00c630 tx=0x7f052c007c00).ready entity=mon.0 client_cookie=a62566c42a68b845 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:18.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.962+0000 7f0531ffb700 1 -- 172.21.15.121:0/1297982697 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c005b20 con 0x7f051c004b80 2023-11-15T02:59:18.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.962+0000 7f0531ffb700 1 -- 172.21.15.121:0/1297982697 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c005c80 con 0x7f051c004b80 2023-11-15T02:59:18.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.962+0000 7f0531ffb700 1 -- 172.21.15.121:0/1297982697 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c0070d0 con 0x7f051c004b80 2023-11-15T02:59:18.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.962+0000 7f05394ff700 1 -- 172.21.15.121:0/1297982697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f051c004b80 msgr2=0x7f051c005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:18.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.962+0000 7f05394ff700 1 --2- 172.21.15.121:0/1297982697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f051c004b80 0x7f051c005000 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f052c00c630 tx=0x7f052c007c00).stop 2023-11-15T02:59:18.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.962+0000 7f05337fe700 1 -- 172.21.15.121:0/1297982697 reap_dead start 2023-11-15T02:59:18.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.962+0000 7f05394ff700 1 -- 172.21.15.121:0/1297982697 shutdown_connections 2023-11-15T02:59:18.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.962+0000 7f05394ff700 1 -- 172.21.15.121:0/1297982697 >> 172.21.15.121:0/1297982697 conn(0x7f0534067a60 msgr2=0x7f0534067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:18.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.962+0000 7f05394ff700 1 -- 172.21.15.121:0/1297982697 shutdown_connections 2023-11-15T02:59:18.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.962+0000 7f05394ff700 1 -- 172.21.15.121:0/1297982697 wait complete. 2023-11-15T02:59:18.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f05394ff700 1 Processor -- start 2023-11-15T02:59:18.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f05394ff700 1 -- start start 2023-11-15T02:59:18.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f05394ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f051c004b80 0x7f05340c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f05394ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05340845b0 0x7f05341693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f05394ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0534082290 0x7f05340826f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f05394ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0534082be0 con 0x7f05340845b0 2023-11-15T02:59:18.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f05394ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0534084a90 con 0x7f0534082290 2023-11-15T02:59:18.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f05394ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0534084d70 con 0x7f051c004b80 2023-11-15T02:59:18.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f05327fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05340845b0 0x7f05341693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f05327fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05340845b0 0x7f05341693c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35982/0 (socket says 172.21.15.121:35982) 2023-11-15T02:59:18.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f05327fc700 1 -- 172.21.15.121:0/1414342709 learned_addr learned my addr 172.21.15.121:0/1414342709 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:18.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f0532ffd700 1 --2- 172.21.15.121:0/1414342709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f051c004b80 0x7f05340c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f05327fc700 1 -- 172.21.15.121:0/1414342709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f051c004b80 msgr2=0x7f05340c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:18.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f05337fe700 1 --2- 172.21.15.121:0/1414342709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0534082290 0x7f05340826f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f05327fc700 1 --2- 172.21.15.121:0/1414342709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f051c004b80 0x7f05340c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f05327fc700 1 -- 172.21.15.121:0/1414342709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0534082290 msgr2=0x7f05340826f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:18.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f05327fc700 1 --2- 172.21.15.121:0/1414342709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0534082290 0x7f05340826f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:18.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f05327fc700 1 -- 172.21.15.121:0/1414342709 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0534085050 con 0x7f05340845b0 2023-11-15T02:59:18.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f0532ffd700 1 --2- 172.21.15.121:0/1414342709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f051c004b80 0x7f05340c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:18.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f05327fc700 1 --2- 172.21.15.121:0/1414342709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05340845b0 0x7f05341693c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f0528007cc0 tx=0x7f0528007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:18.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f051affd700 1 -- 172.21.15.121:0/1414342709 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052800f8a0 con 0x7f05340845b0 2023-11-15T02:59:18.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f051affd700 1 -- 172.21.15.121:0/1414342709 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052800fa20 con 0x7f05340845b0 2023-11-15T02:59:18.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f05394ff700 1 -- 172.21.15.121:0/1414342709 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0534085330 con 0x7f05340845b0 2023-11-15T02:59:18.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.966+0000 7f051affd700 1 -- 172.21.15.121:0/1414342709 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052800fde0 con 0x7f05340845b0 2023-11-15T02:59:18.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.970+0000 7f05394ff700 1 -- 172.21.15.121:0/1414342709 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05340ab300 con 0x7f05340845b0 2023-11-15T02:59:18.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.970+0000 7f05394ff700 1 -- 172.21.15.121:0/1414342709 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053406bcf0 con 0x7f05340845b0 2023-11-15T02:59:18.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.970+0000 7f051affd700 1 -- 172.21.15.121:0/1414342709 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f052801d3b0 con 0x7f05340845b0 2023-11-15T02:59:18.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.974+0000 7f051affd700 1 --2- 172.21.15.121:0/1414342709 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f052005a510 0x7f052005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:18.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.974+0000 7f051affd700 1 -- 172.21.15.121:0/1414342709 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f05280747f0 con 0x7f05340845b0 2023-11-15T02:59:18.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.974+0000 7f0532ffd700 1 --2- 172.21.15.121:0/1414342709 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f052005a510 0x7f052005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:18.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.974+0000 7f0532ffd700 1 --2- 172.21.15.121:0/1414342709 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f052005a510 0x7f052005c9d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f0524009270 tx=0x7f0524008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:18.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:18.978+0000 7f051affd700 1 -- 172.21.15.121:0/1414342709 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0528047f20 con 0x7f05340845b0 2023-11-15T02:59:19.061 INFO:tasks.ceph.mds.c.smithi121.stderr:2023-11-15T02:59:19.054+0000 7f21e20dc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T02:59:19.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.138+0000 7f05394ff700 1 -- 172.21.15.121:0/1414342709 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0534154e00 con 0x7f05340845b0 2023-11-15T02:59:19.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.138+0000 7f051affd700 1 -- 172.21.15.121:0/1414342709 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f052804b540 con 0x7f05340845b0 2023-11-15T02:59:19.147 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T02:59:19.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.142+0000 7f05394ff700 1 -- 172.21.15.121:0/1414342709 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f052005a510 msgr2=0x7f052005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:19.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.146+0000 7f05394ff700 1 --2- 172.21.15.121:0/1414342709 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f052005a510 0x7f052005c9d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f0524009270 tx=0x7f0524008040).stop 2023-11-15T02:59:19.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.146+0000 7f05394ff700 1 -- 172.21.15.121:0/1414342709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05340845b0 msgr2=0x7f05341693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:19.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.146+0000 7f05394ff700 1 --2- 172.21.15.121:0/1414342709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05340845b0 0x7f05341693c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f0528007cc0 tx=0x7f0528007bb0).stop 2023-11-15T02:59:19.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.146+0000 7f05394ff700 1 -- 172.21.15.121:0/1414342709 shutdown_connections 2023-11-15T02:59:19.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.146+0000 7f05394ff700 1 --2- 172.21.15.121:0/1414342709 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f052005a510 0x7f052005c9d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.146+0000 7f05394ff700 1 --2- 172.21.15.121:0/1414342709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0534082290 0x7f05340826f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.146+0000 7f05394ff700 1 --2- 172.21.15.121:0/1414342709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05340845b0 0x7f05341693c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.146+0000 7f05394ff700 1 --2- 172.21.15.121:0/1414342709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f051c004b80 0x7f05340c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.146+0000 7f05394ff700 1 -- 172.21.15.121:0/1414342709 >> 172.21.15.121:0/1414342709 conn(0x7f0534067a60 msgr2=0x7f05341534a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:19.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.146+0000 7f05394ff700 1 -- 172.21.15.121:0/1414342709 shutdown_connections 2023-11-15T02:59:19.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.146+0000 7f05394ff700 1 -- 172.21.15.121:0/1414342709 wait complete. 2023-11-15T02:59:19.175 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:59:19.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.322+0000 7f0342039700 1 Processor -- start 2023-11-15T02:59:19.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.326+0000 7f0342039700 1 -- start start 2023-11-15T02:59:19.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.326+0000 7f0342039700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f033c157ef0 0x7f033c067140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:19.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.326+0000 7f0342039700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f033c066370 0x7f033c1535c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:19.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.326+0000 7f0342039700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033c066800 con 0x7f033c066d60 2023-11-15T02:59:19.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.326+0000 7f0342039700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033c0c00f0 con 0x7f033c066370 2023-11-15T02:59:19.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.326+0000 7f0342039700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c1583f0 con 0x7f033c157ef0 2023-11-15T02:59:19.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.326+0000 7f0340836700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f033c157ef0 0x7f033c067140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:19.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.326+0000 7f0341838700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f033c066370 0x7f033c1535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:19.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.326+0000 7f0340836700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f033c157ef0 0x7f033c067140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:58684/0 (socket says 172.21.15.121:58684) 2023-11-15T02:59:19.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.326+0000 7f0340836700 1 -- 172.21.15.121:0/4249630462 learned_addr learned my addr 172.21.15.121:0/4249630462 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:19.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.326+0000 7f0340836700 1 -- 172.21.15.121:0/4249630462 >> v2:172.21.15.190:3300/0 conn(0x7f033c066370 msgr2=0x7f033c1535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:19.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.326+0000 7f0340836700 1 --2- 172.21.15.121:0/4249630462 >> v2:172.21.15.190:3300/0 conn(0x7f033c066370 0x7f033c1535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.326+0000 7f0340836700 1 -- 172.21.15.121:0/4249630462 >> v1:172.21.15.190:6789/0 conn(0x7f033c066d60 legacy=0x7f033c1577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:19.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.330+0000 7f0340836700 1 -- 172.21.15.121:0/4249630462 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c1586d0 con 0x7f033c157ef0 2023-11-15T02:59:19.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.326+0000 7f0333fff700 1 -- 172.21.15.121:0/4249630462 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857208889 0 0) 0x7f033c066800 con 0x7f033c066d60 2023-11-15T02:59:19.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.330+0000 7f0340836700 1 --2- 172.21.15.121:0/4249630462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f033c157ef0 0x7f033c067140 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f032c006fd0 tx=0x7f032c0090f0).ready entity=mon.2 client_cookie=ad7865e240f2d874 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:19.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.330+0000 7f0333fff700 1 -- 172.21.15.121:0/4249630462 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c00b580 con 0x7f033c157ef0 2023-11-15T02:59:19.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.330+0000 7f0333fff700 1 -- 172.21.15.121:0/4249630462 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c00b6e0 con 0x7f033c157ef0 2023-11-15T02:59:19.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.330+0000 7f0333fff700 1 -- 172.21.15.121:0/4249630462 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c00ba60 con 0x7f033c157ef0 2023-11-15T02:59:19.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.330+0000 7f0342039700 1 -- 172.21.15.121:0/4249630462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f033c157ef0 msgr2=0x7f033c067140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:19.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.330+0000 7f0342039700 1 --2- 172.21.15.121:0/4249630462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f033c157ef0 0x7f033c067140 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f032c006fd0 tx=0x7f032c0090f0).stop 2023-11-15T02:59:19.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.330+0000 7f0342039700 1 -- 172.21.15.121:0/4249630462 shutdown_connections 2023-11-15T02:59:19.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.330+0000 7f0342039700 1 --2- 172.21.15.121:0/4249630462 >> v2:172.21.15.190:3300/0 conn(0x7f033c066370 0x7f033c1535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.330+0000 7f0342039700 1 --2- 172.21.15.121:0/4249630462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f033c157ef0 0x7f033c067140 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.330+0000 7f0342039700 1 -- 172.21.15.121:0/4249630462 >> 172.21.15.121:0/4249630462 conn(0x7f033c062190 msgr2=0x7f033c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:19.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.330+0000 7f0342039700 1 -- 172.21.15.121:0/4249630462 shutdown_connections 2023-11-15T02:59:19.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.330+0000 7f0342039700 1 -- 172.21.15.121:0/4249630462 wait complete. 2023-11-15T02:59:19.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.334+0000 7f0342039700 1 Processor -- start 2023-11-15T02:59:19.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.334+0000 7f0342039700 1 -- start start 2023-11-15T02:59:19.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.334+0000 7f0342039700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f033c066370 0x7f033c13d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:19.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.334+0000 7f0342039700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f033c066d60 0x7f033c0c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:19.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.334+0000 7f0342039700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f033c157ef0 0x7f033c0aa590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:19.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.334+0000 7f0342039700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c1592c0 con 0x7f033c157ef0 2023-11-15T02:59:19.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.334+0000 7f0342039700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c159df0 con 0x7f033c066370 2023-11-15T02:59:19.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.334+0000 7f0342039700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c15a280 con 0x7f033c066d60 2023-11-15T02:59:19.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.334+0000 7f0341838700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f033c157ef0 0x7f033c0aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:19.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.334+0000 7f0341838700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f033c157ef0 0x7f033c0aa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35992/0 (socket says 172.21.15.121:35992) 2023-11-15T02:59:19.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.334+0000 7f0341838700 1 -- 172.21.15.121:0/2147918463 learned_addr learned my addr 172.21.15.121:0/2147918463 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:19.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.334+0000 7f0340836700 1 --2- 172.21.15.121:0/2147918463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f033c066d60 0x7f033c0c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:19.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.334+0000 7f0341037700 1 --2- 172.21.15.121:0/2147918463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f033c066370 0x7f033c13d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:19.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.338+0000 7f0341838700 1 -- 172.21.15.121:0/2147918463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f033c066d60 msgr2=0x7f033c0c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:19.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.338+0000 7f0341838700 1 --2- 172.21.15.121:0/2147918463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f033c066d60 0x7f033c0c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.338+0000 7f0341838700 1 -- 172.21.15.121:0/2147918463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f033c066370 msgr2=0x7f033c13d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:19.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.338+0000 7f0341838700 1 --2- 172.21.15.121:0/2147918463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f033c066370 0x7f033c13d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.338+0000 7f0341838700 1 -- 172.21.15.121:0/2147918463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032c009370 con 0x7f033c157ef0 2023-11-15T02:59:19.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.338+0000 7f0341838700 1 --2- 172.21.15.121:0/2147918463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f033c157ef0 0x7f033c0aa590 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f0334009e90 tx=0x7f033400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:19.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.338+0000 7f03317fa700 1 -- 172.21.15.121:0/2147918463 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033400caa0 con 0x7f033c157ef0 2023-11-15T02:59:19.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.338+0000 7f03317fa700 1 -- 172.21.15.121:0/2147918463 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033400f040 con 0x7f033c157ef0 2023-11-15T02:59:19.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.338+0000 7f03317fa700 1 -- 172.21.15.121:0/2147918463 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033400ce10 con 0x7f033c157ef0 2023-11-15T02:59:19.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.338+0000 7f0342039700 1 -- 172.21.15.121:0/2147918463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033c13d670 con 0x7f033c157ef0 2023-11-15T02:59:19.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.338+0000 7f0342039700 1 -- 172.21.15.121:0/2147918463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033c1597a0 con 0x7f033c157ef0 2023-11-15T02:59:19.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.338+0000 7f0342039700 1 -- 172.21.15.121:0/2147918463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033c159f80 con 0x7f033c157ef0 2023-11-15T02:59:19.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.342+0000 7f03317fa700 1 -- 172.21.15.121:0/2147918463 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0334013610 con 0x7f033c157ef0 2023-11-15T02:59:19.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.342+0000 7f03317fa700 1 --2- 172.21.15.121:0/2147918463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f032005a560 0x7f032005ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:19.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.342+0000 7f03317fa700 1 -- 172.21.15.121:0/2147918463 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f0334075030 con 0x7f033c157ef0 2023-11-15T02:59:19.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.342+0000 7f0341037700 1 --2- 172.21.15.121:0/2147918463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f032005a560 0x7f032005ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:19.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.342+0000 7f0341037700 1 --2- 172.21.15.121:0/2147918463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f032005a560 0x7f032005ca20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f0338007fd0 tx=0x7f0338007bb0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:19.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.350+0000 7f03317fa700 1 -- 172.21.15.121:0/2147918463 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f033404c290 con 0x7f033c157ef0 2023-11-15T02:59:19.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.542+0000 7f0342039700 1 -- 172.21.15.121:0/2147918463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f033c065890 con 0x7f033c157ef0 2023-11-15T02:59:19.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.542+0000 7f03317fa700 1 -- 172.21.15.121:0/2147918463 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4502 (secure 0 0 0) 0x7f033404be80 con 0x7f033c157ef0 2023-11-15T02:59:19.551 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:19.551 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3403224816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3403224816},{"type":"v1","addr":"172.21.15.190:6835","nonce":3403224816}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2088953798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2088953798},{"type":"v1","addr":"172.21.15.190:6833","nonce":2088953798}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5511,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2190514138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2190514138},{"type":"v1","addr":"172.21.15.121:6835","nonce":2190514138}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:58:57.609524+0000","modified":"2023-11-15T02:59:18.096576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5700},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5700":{"gid":5700,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":4,"addr":"172.21.15.121:6837/3406398738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3406398738},{"type":"v1","addr":"172.21.15.121:6837","nonce":3406398738}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-15T02:59:19.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.546+0000 7f0342039700 1 -- 172.21.15.121:0/2147918463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f032005a560 msgr2=0x7f032005ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:19.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.546+0000 7f0342039700 1 --2- 172.21.15.121:0/2147918463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f032005a560 0x7f032005ca20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f0338007fd0 tx=0x7f0338007bb0).stop 2023-11-15T02:59:19.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.546+0000 7f0342039700 1 -- 172.21.15.121:0/2147918463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f033c157ef0 msgr2=0x7f033c0aa590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:19.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.546+0000 7f0342039700 1 --2- 172.21.15.121:0/2147918463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f033c157ef0 0x7f033c0aa590 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f0334009e90 tx=0x7f033400b040).stop 2023-11-15T02:59:19.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.546+0000 7f0342039700 1 -- 172.21.15.121:0/2147918463 shutdown_connections 2023-11-15T02:59:19.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.546+0000 7f0342039700 1 --2- 172.21.15.121:0/2147918463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f032005a560 0x7f032005ca20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.546+0000 7f0342039700 1 --2- 172.21.15.121:0/2147918463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f033c157ef0 0x7f033c0aa590 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.546+0000 7f0342039700 1 --2- 172.21.15.121:0/2147918463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f033c066d60 0x7f033c0c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.546+0000 7f0342039700 1 --2- 172.21.15.121:0/2147918463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f033c066370 0x7f033c13d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.546+0000 7f0342039700 1 -- 172.21.15.121:0/2147918463 >> 172.21.15.121:0/2147918463 conn(0x7f033c062190 msgr2=0x7f033c14aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:19.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.546+0000 7f0342039700 1 -- 172.21.15.121:0/2147918463 shutdown_connections 2023-11-15T02:59:19.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.546+0000 7f0342039700 1 -- 172.21.15.121:0/2147918463 wait complete. 2023-11-15T02:59:19.557 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 37 2023-11-15T02:59:19.572 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:59:19.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.730+0000 7fcd5a907700 1 Processor -- start 2023-11-15T02:59:19.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.730+0000 7fcd5a907700 1 -- start start 2023-11-15T02:59:19.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.734+0000 7fcd5a907700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd540c2010 0x7fcd540c23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:19.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.734+0000 7fcd5a907700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcd54153030 0x7fcd541534c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:19.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.734+0000 7fcd5a907700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd54154b90 con 0x7fcd54154750 2023-11-15T02:59:19.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.734+0000 7fcd5a907700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd54153c90 con 0x7fcd54153030 2023-11-15T02:59:19.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.734+0000 7fcd5a907700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540c1810 con 0x7fcd540c2010 2023-11-15T02:59:19.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.734+0000 7fcd59104700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fcd54154750 0x7fcd540c0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60894/0 (socket says 172.21.15.121:60894) 2023-11-15T02:59:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.734+0000 7fcd59104700 1 -- 172.21.15.121:0/3453561298 learned_addr learned my addr 172.21.15.121:0/3453561298 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.734+0000 7fcd5a106700 1 --2- 172.21.15.121:0/3453561298 >> v2:172.21.15.190:3300/0 conn(0x7fcd54153030 0x7fcd541534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.734+0000 7fcd59905700 1 --2- 172.21.15.121:0/3453561298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd540c2010 0x7fcd540c23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.734+0000 7fcd58903700 1 -- 172.21.15.121:0/3453561298 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217618769 0 0) 0x7fcd54154b90 con 0x7fcd54154750 2023-11-15T02:59:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.734+0000 7fcd58903700 1 -- 172.21.15.121:0/3453561298 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd400033f0 con 0x7fcd54154750 2023-11-15T02:59:19.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.734+0000 7fcd5a106700 1 -- 172.21.15.121:0/3453561298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd540c2010 msgr2=0x7fcd540c23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:19.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.734+0000 7fcd5a106700 1 --2- 172.21.15.121:0/3453561298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd540c2010 0x7fcd540c23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.738+0000 7fcd5a106700 1 -- 172.21.15.121:0/3453561298 >> v1:172.21.15.121:6789/0 conn(0x7fcd54154750 legacy=0x7fcd540c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:19.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.738+0000 7fcd5a106700 1 -- 172.21.15.121:0/3453561298 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd540c1af0 con 0x7fcd54153030 2023-11-15T02:59:19.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.738+0000 7fcd5a106700 1 --2- 172.21.15.121:0/3453561298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd54153030 0x7fcd541534c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fcd4c00af70 tx=0x7fcd4c00a890).ready entity=mon.1 client_cookie=aa6c9d8758049bac server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:19.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.738+0000 7fcd58903700 1 -- 172.21.15.121:0/3453561298 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c00a2e0 con 0x7fcd54153030 2023-11-15T02:59:19.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.738+0000 7fcd58903700 1 -- 172.21.15.121:0/3453561298 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4c00c910 con 0x7fcd54153030 2023-11-15T02:59:19.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.738+0000 7fcd58903700 1 -- 172.21.15.121:0/3453561298 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c00cc90 con 0x7fcd54153030 2023-11-15T02:59:19.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.738+0000 7fcd5a907700 1 -- 172.21.15.121:0/3453561298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd54153030 msgr2=0x7fcd541534c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:19.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.738+0000 7fcd5a907700 1 --2- 172.21.15.121:0/3453561298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd54153030 0x7fcd541534c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fcd4c00af70 tx=0x7fcd4c00a890).stop 2023-11-15T02:59:19.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.738+0000 7fcd5a907700 1 -- 172.21.15.121:0/3453561298 shutdown_connections 2023-11-15T02:59:19.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.738+0000 7fcd5a907700 1 --2- 172.21.15.121:0/3453561298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd54153030 0x7fcd541534c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.738+0000 7fcd5a907700 1 --2- 172.21.15.121:0/3453561298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd540c2010 0x7fcd540c23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.738+0000 7fcd5a907700 1 -- 172.21.15.121:0/3453561298 >> 172.21.15.121:0/3453561298 conn(0x7fcd54062190 msgr2=0x7fcd54062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:19.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.738+0000 7fcd5a907700 1 -- 172.21.15.121:0/3453561298 shutdown_connections 2023-11-15T02:59:19.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.738+0000 7fcd5a907700 1 -- 172.21.15.121:0/3453561298 wait complete. 2023-11-15T02:59:19.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.738+0000 7fcd5a907700 1 Processor -- start 2023-11-15T02:59:19.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.738+0000 7fcd5a907700 1 -- start start 2023-11-15T02:59:19.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd5a907700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd540c2010 0x7fcd5413eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:19.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd5a907700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd54153030 0x7fcd5414ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:19.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd5a907700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd54154750 0x7fcd5414b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:19.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd5a907700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5414c550 con 0x7fcd54153030 2023-11-15T02:59:19.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd5a907700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5414b6c0 con 0x7fcd54154750 2023-11-15T02:59:19.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd5a907700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5414b940 con 0x7fcd540c2010 2023-11-15T02:59:19.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd59104700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd54153030 0x7fcd5414ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:19.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd5a106700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd54154750 0x7fcd5414b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:19.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd59104700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd54153030 0x7fcd5414ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35998/0 (socket says 172.21.15.121:35998) 2023-11-15T02:59:19.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd59104700 1 -- 172.21.15.121:0/2589654853 learned_addr learned my addr 172.21.15.121:0/2589654853 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:19.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd59905700 1 --2- 172.21.15.121:0/2589654853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd540c2010 0x7fcd5413eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:19.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd59104700 1 -- 172.21.15.121:0/2589654853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd540c2010 msgr2=0x7fcd5413eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:19.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd59104700 1 --2- 172.21.15.121:0/2589654853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd540c2010 0x7fcd5413eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd59104700 1 -- 172.21.15.121:0/2589654853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd54154750 msgr2=0x7fcd5414b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:19.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd59104700 1 --2- 172.21.15.121:0/2589654853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd54154750 0x7fcd5414b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd59104700 1 -- 172.21.15.121:0/2589654853 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4c00ab30 con 0x7fcd54153030 2023-11-15T02:59:19.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd59104700 1 --2- 172.21.15.121:0/2589654853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd54153030 0x7fcd5414ace0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7fcd4400c490 tx=0x7fcd44007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:19.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd49ffb700 1 -- 172.21.15.121:0/2589654853 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd440070d0 con 0x7fcd54153030 2023-11-15T02:59:19.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd49ffb700 1 -- 172.21.15.121:0/2589654853 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd44007230 con 0x7fcd54153030 2023-11-15T02:59:19.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd5a907700 1 -- 172.21.15.121:0/2589654853 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5414bc20 con 0x7fcd54153030 2023-11-15T02:59:19.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd49ffb700 1 -- 172.21.15.121:0/2589654853 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd440075d0 con 0x7fcd54153030 2023-11-15T02:59:19.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.742+0000 7fcd5a907700 1 -- 172.21.15.121:0/2589654853 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5414c1e0 con 0x7fcd54153030 2023-11-15T02:59:19.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.746+0000 7fcd5a907700 1 -- 172.21.15.121:0/2589654853 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5406bcf0 con 0x7fcd54153030 2023-11-15T02:59:19.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.746+0000 7fcd49ffb700 1 -- 172.21.15.121:0/2589654853 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fcd44005bc0 con 0x7fcd54153030 2023-11-15T02:59:19.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.746+0000 7fcd49ffb700 1 --2- 172.21.15.121:0/2589654853 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd3805a540 0x7fcd3805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:19.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.746+0000 7fcd49ffb700 1 -- 172.21.15.121:0/2589654853 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fcd44074020 con 0x7fcd54153030 2023-11-15T02:59:19.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.750+0000 7fcd59905700 1 --2- 172.21.15.121:0/2589654853 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd3805a540 0x7fcd3805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:19.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.750+0000 7fcd59905700 1 --2- 172.21.15.121:0/2589654853 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd3805a540 0x7fcd3805ca00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fcd50009b90 tx=0x7fcd50009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:19.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.758+0000 7fcd49ffb700 1 -- 172.21.15.121:0/2589654853 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcd440476f0 con 0x7fcd54153030 2023-11-15T02:59:19.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.914+0000 7fcd5a907700 1 -- 172.21.15.121:0/2589654853 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd54068a70 con 0x7fcd54153030 2023-11-15T02:59:19.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.914+0000 7fcd49ffb700 1 -- 172.21.15.121:0/2589654853 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+17107 (secure 0 0 0) 0x7fcd44018030 con 0x7fcd54153030 2023-11-15T02:59:19.921 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:19.922 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":49,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:59:12.230103+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:58:55.530878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-15T02:58:56.602591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6837/1945477051":"2023-11-16T02:59:12.230061+0000","172.21.15.121:6836/1945477051":"2023-11-16T02:59:12.230061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:59:19.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.918+0000 7fcd5a907700 1 -- 172.21.15.121:0/2589654853 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd3805a540 msgr2=0x7fcd3805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:19.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.918+0000 7fcd5a907700 1 --2- 172.21.15.121:0/2589654853 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd3805a540 0x7fcd3805ca00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fcd50009b90 tx=0x7fcd50009060).stop 2023-11-15T02:59:19.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.918+0000 7fcd5a907700 1 -- 172.21.15.121:0/2589654853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd54153030 msgr2=0x7fcd5414ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:19.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.918+0000 7fcd5a907700 1 --2- 172.21.15.121:0/2589654853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd54153030 0x7fcd5414ace0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7fcd4400c490 tx=0x7fcd44007bf0).stop 2023-11-15T02:59:19.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.918+0000 7fcd5a907700 1 -- 172.21.15.121:0/2589654853 shutdown_connections 2023-11-15T02:59:19.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.918+0000 7fcd5a907700 1 --2- 172.21.15.121:0/2589654853 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd3805a540 0x7fcd3805ca00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.918+0000 7fcd5a907700 1 --2- 172.21.15.121:0/2589654853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd54154750 0x7fcd5414b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.918+0000 7fcd5a907700 1 --2- 172.21.15.121:0/2589654853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd54153030 0x7fcd5414ace0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.918+0000 7fcd5a907700 1 --2- 172.21.15.121:0/2589654853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd540c2010 0x7fcd5413eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:19.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.918+0000 7fcd5a907700 1 -- 172.21.15.121:0/2589654853 >> 172.21.15.121:0/2589654853 conn(0x7fcd54062190 msgr2=0x7fcd540aaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:19.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.918+0000 7fcd5a907700 1 -- 172.21.15.121:0/2589654853 shutdown_connections 2023-11-15T02:59:19.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:19.918+0000 7fcd5a907700 1 -- 172.21.15.121:0/2589654853 wait complete. 2023-11-15T02:59:19.945 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-15T02:59:19.946 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:59:20.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.098+0000 7fd90f439700 1 Processor -- start 2023-11-15T02:59:20.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.098+0000 7fd90f439700 1 -- start start 2023-11-15T02:59:20.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.102+0000 7fd90f439700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd908153a20 0x7fd9080c4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:20.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.102+0000 7fd90f439700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9080c3b90 con 0x7fd9080c8f30 2023-11-15T02:59:20.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.102+0000 7fd90f439700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd908062fa0 con 0x7fd9080c4510 2023-11-15T02:59:20.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.102+0000 7fd90f439700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd908063280 con 0x7fd908153a20 2023-11-15T02:59:20.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.102+0000 7fd90e437700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd908153a20 0x7fd9080c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:20.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.102+0000 7fd90e437700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd908153a20 0x7fd9080c4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36002/0 (socket says 172.21.15.121:36002) 2023-11-15T02:59:20.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.102+0000 7fd90e437700 1 -- 172.21.15.121:0/2001057868 learned_addr learned my addr 172.21.15.121:0/2001057868 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:20.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.102+0000 7fd90e437700 1 -- 172.21.15.121:0/2001057868 >> v1:172.21.15.190:6789/0 conn(0x7fd9080c4510 legacy=0x7fd9080c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:20.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.102+0000 7fd90e437700 1 -- 172.21.15.121:0/2001057868 >> v1:172.21.15.121:6789/0 conn(0x7fd9080c8f30 legacy=0x7fd9080bd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:20.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.102+0000 7fd90e437700 1 -- 172.21.15.121:0/2001057868 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9080c3540 con 0x7fd908153a20 2023-11-15T02:59:20.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.102+0000 7fd90e437700 1 --2- 172.21.15.121:0/2001057868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd908153a20 0x7fd9080c4020 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fd904009810 tx=0x7fd904009060).ready entity=mon.0 client_cookie=3b5184c9dc848089 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:20.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.106+0000 7fd90d435700 1 -- 172.21.15.121:0/2001057868 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904005880 con 0x7fd908153a20 2023-11-15T02:59:20.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.106+0000 7fd90d435700 1 -- 172.21.15.121:0/2001057868 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9040059e0 con 0x7fd908153a20 2023-11-15T02:59:20.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.106+0000 7fd90d435700 1 -- 172.21.15.121:0/2001057868 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904005d60 con 0x7fd908153a20 2023-11-15T02:59:20.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.106+0000 7fd90f439700 1 -- 172.21.15.121:0/2001057868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd908153a20 msgr2=0x7fd9080c4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:20.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.106+0000 7fd90f439700 1 --2- 172.21.15.121:0/2001057868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd908153a20 0x7fd9080c4020 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fd904009810 tx=0x7fd904009060).stop 2023-11-15T02:59:20.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.106+0000 7fd90f439700 1 -- 172.21.15.121:0/2001057868 shutdown_connections 2023-11-15T02:59:20.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.106+0000 7fd90f439700 1 --2- 172.21.15.121:0/2001057868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd908153a20 0x7fd9080c4020 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:20.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.106+0000 7fd90f439700 1 -- 172.21.15.121:0/2001057868 >> 172.21.15.121:0/2001057868 conn(0x7fd9080643a0 msgr2=0x7fd9080647b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:20.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.106+0000 7fd90f439700 1 -- 172.21.15.121:0/2001057868 shutdown_connections 2023-11-15T02:59:20.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.106+0000 7fd90f439700 1 -- 172.21.15.121:0/2001057868 wait complete. 2023-11-15T02:59:20.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90f439700 1 Processor -- start 2023-11-15T02:59:20.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90f439700 1 -- start start 2023-11-15T02:59:20.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90f439700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9080c4510 0x7fd9080c0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:20.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90f439700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9080c8f30 0x7fd90814c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:20.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90f439700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd908153a20 0x7fd9081547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:20.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90f439700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080c0a00 con 0x7fd9080c4510 2023-11-15T02:59:20.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90f439700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080bfdf0 con 0x7fd9080c8f30 2023-11-15T02:59:20.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90f439700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080c00d0 con 0x7fd908153a20 2023-11-15T02:59:20.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90e437700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9080c4510 0x7fd9080c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:20.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90e437700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9080c4510 0x7fd9080c0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36012/0 (socket says 172.21.15.121:36012) 2023-11-15T02:59:20.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90e437700 1 -- 172.21.15.121:0/3896029912 learned_addr learned my addr 172.21.15.121:0/3896029912 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:20.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90dc36700 1 --2- 172.21.15.121:0/3896029912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9080c8f30 0x7fd90814c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:20.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90ec38700 1 --2- 172.21.15.121:0/3896029912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd908153a20 0x7fd9081547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:20.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90e437700 1 -- 172.21.15.121:0/3896029912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd908153a20 msgr2=0x7fd9081547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:20.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90e437700 1 --2- 172.21.15.121:0/3896029912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd908153a20 0x7fd9081547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:20.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90e437700 1 -- 172.21.15.121:0/3896029912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9080c8f30 msgr2=0x7fd90814c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:20.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90e437700 1 --2- 172.21.15.121:0/3896029912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9080c8f30 0x7fd90814c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:20.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90e437700 1 -- 172.21.15.121:0/3896029912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd904009300 con 0x7fd9080c4510 2023-11-15T02:59:20.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90dc36700 1 --2- 172.21.15.121:0/3896029912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9080c8f30 0x7fd90814c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:20.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd90e437700 1 --2- 172.21.15.121:0/3896029912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9080c4510 0x7fd9080c0e90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7fd90400b400 tx=0x7fd904016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:20.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.110+0000 7fd8fe7fc700 1 -- 172.21.15.121:0/3896029912 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90400b850 con 0x7fd9080c4510 2023-11-15T02:59:20.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.114+0000 7fd8fe7fc700 1 -- 172.21.15.121:0/3896029912 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90400b9b0 con 0x7fd9080c4510 2023-11-15T02:59:20.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.114+0000 7fd90f439700 1 -- 172.21.15.121:0/3896029912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9080c03b0 con 0x7fd9080c4510 2023-11-15T02:59:20.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.114+0000 7fd8fe7fc700 1 -- 172.21.15.121:0/3896029912 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90400bd50 con 0x7fd9080c4510 2023-11-15T02:59:20.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.114+0000 7fd90f439700 1 -- 172.21.15.121:0/3896029912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9080c1380 con 0x7fd9080c4510 2023-11-15T02:59:20.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.114+0000 7fd90f439700 1 -- 172.21.15.121:0/3896029912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90806bcf0 con 0x7fd9080c4510 2023-11-15T02:59:20.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.114+0000 7fd8fe7fc700 1 -- 172.21.15.121:0/3896029912 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd904018030 con 0x7fd9080c4510 2023-11-15T02:59:20.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.114+0000 7fd8fe7fc700 1 --2- 172.21.15.121:0/3896029912 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd8ec05a510 0x7fd8ec05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:20.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.114+0000 7fd8fe7fc700 1 -- 172.21.15.121:0/3896029912 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fd9040759d0 con 0x7fd9080c4510 2023-11-15T02:59:20.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.114+0000 7fd90dc36700 1 --2- 172.21.15.121:0/3896029912 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd8ec05a510 0x7fd8ec05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:20.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.118+0000 7fd90dc36700 1 --2- 172.21.15.121:0/3896029912 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd8ec05a510 0x7fd8ec05c9d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fd9080c1c80 tx=0x7fd8f8009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:20.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.122+0000 7fd8fe7fc700 1 -- 172.21.15.121:0/3896029912 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd9040490a0 con 0x7fd9080c4510 2023-11-15T02:59:20.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.318+0000 7fd90f439700 1 -- 172.21.15.121:0/3896029912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9080c3820 con 0x7fd9080c4510 2023-11-15T02:59:20.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.318+0000 7fd8fe7fc700 1 -- 172.21.15.121:0/3896029912 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4502 (secure 0 0 0) 0x7fd904028070 con 0x7fd9080c4510 2023-11-15T02:59:20.325 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:20.325 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3403224816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3403224816},{"type":"v1","addr":"172.21.15.190:6835","nonce":3403224816}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2088953798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2088953798},{"type":"v1","addr":"172.21.15.190:6833","nonce":2088953798}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5511,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2190514138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2190514138},{"type":"v1","addr":"172.21.15.121:6835","nonce":2190514138}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:58:57.609524+0000","modified":"2023-11-15T02:59:18.096576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5700},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5700":{"gid":5700,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":4,"addr":"172.21.15.121:6837/3406398738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3406398738},{"type":"v1","addr":"172.21.15.121:6837","nonce":3406398738}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-15T02:59:20.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.322+0000 7fd90f439700 1 -- 172.21.15.121:0/3896029912 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd8ec05a510 msgr2=0x7fd8ec05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:20.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.322+0000 7fd90f439700 1 --2- 172.21.15.121:0/3896029912 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd8ec05a510 0x7fd8ec05c9d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fd9080c1c80 tx=0x7fd8f8009040).stop 2023-11-15T02:59:20.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.322+0000 7fd90f439700 1 -- 172.21.15.121:0/3896029912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9080c4510 msgr2=0x7fd9080c0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:20.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.322+0000 7fd90f439700 1 --2- 172.21.15.121:0/3896029912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9080c4510 0x7fd9080c0e90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7fd90400b400 tx=0x7fd904016000).stop 2023-11-15T02:59:20.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.322+0000 7fd90f439700 1 -- 172.21.15.121:0/3896029912 shutdown_connections 2023-11-15T02:59:20.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.322+0000 7fd90f439700 1 --2- 172.21.15.121:0/3896029912 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd8ec05a510 0x7fd8ec05c9d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:20.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.322+0000 7fd90f439700 1 --2- 172.21.15.121:0/3896029912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd908153a20 0x7fd9081547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:20.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.322+0000 7fd90f439700 1 --2- 172.21.15.121:0/3896029912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9080c8f30 0x7fd90814c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:20.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.322+0000 7fd90f439700 1 --2- 172.21.15.121:0/3896029912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9080c4510 0x7fd9080c0e90 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:20.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.322+0000 7fd90f439700 1 -- 172.21.15.121:0/3896029912 >> 172.21.15.121:0/3896029912 conn(0x7fd9080643a0 msgr2=0x7fd9080c8450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:20.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.322+0000 7fd90f439700 1 -- 172.21.15.121:0/3896029912 shutdown_connections 2023-11-15T02:59:20.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.322+0000 7fd90f439700 1 -- 172.21.15.121:0/3896029912 wait complete. 2023-11-15T02:59:20.330 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 37 2023-11-15T02:59:20.349 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:59:20.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.510+0000 7fe677edb700 1 Processor -- start 2023-11-15T02:59:20.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.510+0000 7fe677edb700 1 -- start start 2023-11-15T02:59:20.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.514+0000 7fe677edb700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe670068170 0x7fe67015ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:20.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.514+0000 7fe677edb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6700cb5c0 con 0x7fe670153bb0 2023-11-15T02:59:20.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.514+0000 7fe677edb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6700c9ca0 con 0x7fe6700aa240 2023-11-15T02:59:20.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.514+0000 7fe677edb700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6700aa6f0 con 0x7fe670068170 2023-11-15T02:59:20.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.514+0000 7fe675c77700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe670153bb0 0x7fe6700a9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60910/0 (socket says 172.21.15.121:60910) 2023-11-15T02:59:20.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.514+0000 7fe675c77700 1 -- 172.21.15.121:0/712309447 learned_addr learned my addr 172.21.15.121:0/712309447 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:20.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.514+0000 7fe676478700 1 --2- 172.21.15.121:0/712309447 >> v2:172.21.15.121:3300/0 conn(0x7fe670068170 0x7fe67015ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:20.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.514+0000 7fe674c75700 1 -- 172.21.15.121:0/712309447 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2391922772 0 0) 0x7fe6700cb5c0 con 0x7fe670153bb0 2023-11-15T02:59:20.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.514+0000 7fe674c75700 1 -- 172.21.15.121:0/712309447 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe65c0033f0 con 0x7fe670153bb0 2023-11-15T02:59:20.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.514+0000 7fe674c75700 1 -- 172.21.15.121:0/712309447 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3347819514 0 0) 0x7fe6700c9ca0 con 0x7fe6700aa240 2023-11-15T02:59:20.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.514+0000 7fe674c75700 1 -- 172.21.15.121:0/712309447 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6700cb5c0 con 0x7fe6700aa240 2023-11-15T02:59:20.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.514+0000 7fe676478700 1 -- 172.21.15.121:0/712309447 >> v1:172.21.15.190:6789/0 conn(0x7fe6700aa240 legacy=0x7fe670067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:20.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.514+0000 7fe676478700 1 -- 172.21.15.121:0/712309447 >> v1:172.21.15.121:6789/0 conn(0x7fe670153bb0 legacy=0x7fe6700a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:20.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.514+0000 7fe676478700 1 -- 172.21.15.121:0/712309447 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6700caa20 con 0x7fe670068170 2023-11-15T02:59:20.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.514+0000 7fe676478700 1 --2- 172.21.15.121:0/712309447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe670068170 0x7fe67015ec40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fe668009760 tx=0x7fe668009060).ready entity=mon.0 client_cookie=750d4e5324eedd4d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:20.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.514+0000 7fe674c75700 1 -- 172.21.15.121:0/712309447 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66800d4a0 con 0x7fe670068170 2023-11-15T02:59:20.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.514+0000 7fe674c75700 1 -- 172.21.15.121:0/712309447 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66800d600 con 0x7fe670068170 2023-11-15T02:59:20.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.514+0000 7fe674c75700 1 -- 172.21.15.121:0/712309447 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66800d980 con 0x7fe670068170 2023-11-15T02:59:20.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.518+0000 7fe677edb700 1 -- 172.21.15.121:0/712309447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe670068170 msgr2=0x7fe67015ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:20.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.518+0000 7fe677edb700 1 --2- 172.21.15.121:0/712309447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe670068170 0x7fe67015ec40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fe668009760 tx=0x7fe668009060).stop 2023-11-15T02:59:20.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.518+0000 7fe677edb700 1 -- 172.21.15.121:0/712309447 shutdown_connections 2023-11-15T02:59:20.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.518+0000 7fe677edb700 1 --2- 172.21.15.121:0/712309447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe670068170 0x7fe67015ec40 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:20.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.518+0000 7fe677edb700 1 -- 172.21.15.121:0/712309447 >> 172.21.15.121:0/712309447 conn(0x7fe670058c50 msgr2=0x7fe670059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:20.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.518+0000 7fe677edb700 1 -- 172.21.15.121:0/712309447 shutdown_connections 2023-11-15T02:59:20.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.518+0000 7fe677edb700 1 -- 172.21.15.121:0/712309447 wait complete. 2023-11-15T02:59:20.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.518+0000 7fe677edb700 1 Processor -- start 2023-11-15T02:59:20.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe677edb700 1 -- start start 2023-11-15T02:59:20.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe677edb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe670068170 0x7fe6700c1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:20.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe677edb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6700aa240 0x7fe670065d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:20.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe677edb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe670153bb0 0x7fe670066220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:20.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe677edb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe670066710 con 0x7fe6700aa240 2023-11-15T02:59:20.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe677edb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67013f530 con 0x7fe670153bb0 2023-11-15T02:59:20.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe677edb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6700c5000 con 0x7fe670068170 2023-11-15T02:59:20.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe675476700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6700aa240 0x7fe670065d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:20.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe675476700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6700aa240 0x7fe670065d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36028/0 (socket says 172.21.15.121:36028) 2023-11-15T02:59:20.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe675476700 1 -- 172.21.15.121:0/3633092005 learned_addr learned my addr 172.21.15.121:0/3633092005 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:20.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe676478700 1 --2- 172.21.15.121:0/3633092005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe670153bb0 0x7fe670066220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:20.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe675c77700 1 --2- 172.21.15.121:0/3633092005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe670068170 0x7fe6700c1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:20.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe675476700 1 -- 172.21.15.121:0/3633092005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe670068170 msgr2=0x7fe6700c1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:20.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe675476700 1 --2- 172.21.15.121:0/3633092005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe670068170 0x7fe6700c1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:20.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe675476700 1 -- 172.21.15.121:0/3633092005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe670153bb0 msgr2=0x7fe670066220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:20.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe675476700 1 --2- 172.21.15.121:0/3633092005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe670153bb0 0x7fe670066220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:20.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe675476700 1 -- 172.21.15.121:0/3633092005 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6680092a0 con 0x7fe6700aa240 2023-11-15T02:59:20.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe676478700 1 --2- 172.21.15.121:0/3633092005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe670153bb0 0x7fe670066220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:20.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe675476700 1 --2- 172.21.15.121:0/3633092005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6700aa240 0x7fe670065d30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fe66000c490 tx=0x7fe660007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:20.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.522+0000 7fe665ffb700 1 -- 172.21.15.121:0/3633092005 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66000f470 con 0x7fe6700aa240 2023-11-15T02:59:20.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.526+0000 7fe665ffb700 1 -- 172.21.15.121:0/3633092005 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66000f5d0 con 0x7fe6700aa240 2023-11-15T02:59:20.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.526+0000 7fe677edb700 1 -- 172.21.15.121:0/3633092005 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6700c4a40 con 0x7fe6700aa240 2023-11-15T02:59:20.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.526+0000 7fe665ffb700 1 -- 172.21.15.121:0/3633092005 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66000f990 con 0x7fe6700aa240 2023-11-15T02:59:20.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.526+0000 7fe677edb700 1 -- 172.21.15.121:0/3633092005 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67013f1c0 con 0x7fe6700aa240 2023-11-15T02:59:20.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.526+0000 7fe677edb700 1 -- 172.21.15.121:0/3633092005 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67006c150 con 0x7fe6700aa240 2023-11-15T02:59:20.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.526+0000 7fe665ffb700 1 -- 172.21.15.121:0/3633092005 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe66000fd00 con 0x7fe6700aa240 2023-11-15T02:59:20.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.526+0000 7fe665ffb700 1 --2- 172.21.15.121:0/3633092005 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe65405a540 0x7fe65405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:20.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.530+0000 7fe665ffb700 1 -- 172.21.15.121:0/3633092005 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fe660073e40 con 0x7fe6700aa240 2023-11-15T02:59:20.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.530+0000 7fe675c77700 1 --2- 172.21.15.121:0/3633092005 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe65405a540 0x7fe65405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:20.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.530+0000 7fe675c77700 1 --2- 172.21.15.121:0/3633092005 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe65405a540 0x7fe65405ca00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fe66c005ad0 tx=0x7fe66c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:20.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.534+0000 7fe665ffb700 1 -- 172.21.15.121:0/3633092005 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe6600475d0 con 0x7fe6700aa240 2023-11-15T02:59:20.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.730+0000 7fe677edb700 1 -- 172.21.15.121:0/3633092005 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe670068ae0 con 0x7fe6700aa240 2023-11-15T02:59:20.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.734+0000 7fe665ffb700 1 -- 172.21.15.121:0/3633092005 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4502 (secure 0 0 0) 0x7fe660007d80 con 0x7fe6700aa240 2023-11-15T02:59:20.739 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:20.740 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3403224816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3403224816},{"type":"v1","addr":"172.21.15.190:6835","nonce":3403224816}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2088953798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2088953798},{"type":"v1","addr":"172.21.15.190:6833","nonce":2088953798}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5511,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2190514138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2190514138},{"type":"v1","addr":"172.21.15.121:6835","nonce":2190514138}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:58:57.609524+0000","modified":"2023-11-15T02:59:18.096576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5700},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5700":{"gid":5700,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":4,"addr":"172.21.15.121:6837/3406398738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3406398738},{"type":"v1","addr":"172.21.15.121:6837","nonce":3406398738}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-15T02:59:20.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.734+0000 7fe677edb700 1 -- 172.21.15.121:0/3633092005 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe65405a540 msgr2=0x7fe65405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:20.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.734+0000 7fe677edb700 1 --2- 172.21.15.121:0/3633092005 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe65405a540 0x7fe65405ca00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fe66c005ad0 tx=0x7fe66c009040).stop 2023-11-15T02:59:20.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.734+0000 7fe677edb700 1 -- 172.21.15.121:0/3633092005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6700aa240 msgr2=0x7fe670065d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:20.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.734+0000 7fe677edb700 1 --2- 172.21.15.121:0/3633092005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6700aa240 0x7fe670065d30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fe66000c490 tx=0x7fe660007c60).stop 2023-11-15T02:59:20.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.734+0000 7fe677edb700 1 -- 172.21.15.121:0/3633092005 shutdown_connections 2023-11-15T02:59:20.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.734+0000 7fe677edb700 1 --2- 172.21.15.121:0/3633092005 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe65405a540 0x7fe65405ca00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:20.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.734+0000 7fe677edb700 1 --2- 172.21.15.121:0/3633092005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe670153bb0 0x7fe670066220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:20.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.734+0000 7fe677edb700 1 --2- 172.21.15.121:0/3633092005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6700aa240 0x7fe670065d30 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:20.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.734+0000 7fe677edb700 1 --2- 172.21.15.121:0/3633092005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe670068170 0x7fe6700c1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:20.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.734+0000 7fe677edb700 1 -- 172.21.15.121:0/3633092005 >> 172.21.15.121:0/3633092005 conn(0x7fe670058c50 msgr2=0x7fe6700bfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:20.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.738+0000 7fe677edb700 1 -- 172.21.15.121:0/3633092005 shutdown_connections 2023-11-15T02:59:20.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.738+0000 7fe677edb700 1 -- 172.21.15.121:0/3633092005 wait complete. 2023-11-15T02:59:20.745 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 37 2023-11-15T02:59:20.765 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:59:20.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.926+0000 7f9cd6439700 1 Processor -- start 2023-11-15T02:59:20.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.926+0000 7f9cd6439700 1 -- start start 2023-11-15T02:59:20.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.930+0000 7f9cd6439700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9cd014e4c0 0x7f9cd00a6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:20.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.930+0000 7f9cd6439700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd0151b00 con 0x7f9cd00a6b00 2023-11-15T02:59:20.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.930+0000 7f9cd6439700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd0150f10 con 0x7f9cd00c3040 2023-11-15T02:59:20.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.930+0000 7f9cd6439700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd01511d0 con 0x7f9cd014e4c0 2023-11-15T02:59:20.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.930+0000 7f9cd4c36700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9cd00a6b00 0x7f9cd00c2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60924/0 (socket says 172.21.15.121:60924) 2023-11-15T02:59:20.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.930+0000 7f9cd4c36700 1 -- 172.21.15.121:0/3173735537 learned_addr learned my addr 172.21.15.121:0/3173735537 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:20.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.930+0000 7f9cd5437700 1 --2- 172.21.15.121:0/3173735537 >> v2:172.21.15.190:3300/0 conn(0x7f9cd014e4c0 0x7f9cd00a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:20.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.930+0000 7f9cc7fff700 1 -- 172.21.15.121:0/3173735537 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690144133 0 0) 0x7f9cd0151b00 con 0x7f9cd00a6b00 2023-11-15T02:59:20.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.930+0000 7f9cc7fff700 1 -- 172.21.15.121:0/3173735537 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cbc0033f0 con 0x7f9cd00a6b00 2023-11-15T02:59:20.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.930+0000 7f9cc7fff700 1 -- 172.21.15.121:0/3173735537 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 597650301 0 0) 0x7f9cd0150f10 con 0x7f9cd00c3040 2023-11-15T02:59:20.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.930+0000 7f9cc7fff700 1 -- 172.21.15.121:0/3173735537 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cd0151b00 con 0x7f9cd00c3040 2023-11-15T02:59:20.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.930+0000 7f9cc7fff700 1 -- 172.21.15.121:0/3173735537 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 959273852 0 0) 0x7f9cbc0033f0 con 0x7f9cd00a6b00 2023-11-15T02:59:20.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.930+0000 7f9cd5437700 1 -- 172.21.15.121:0/3173735537 >> v1:172.21.15.190:6789/0 conn(0x7f9cd00c3040 legacy=0x7f9cd00bcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:20.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.930+0000 7f9cd5437700 1 -- 172.21.15.121:0/3173735537 >> v1:172.21.15.121:6789/0 conn(0x7f9cd00a6b00 legacy=0x7f9cd00c2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:20.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.930+0000 7f9cd5437700 1 -- 172.21.15.121:0/3173735537 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd01514b0 con 0x7f9cd014e4c0 2023-11-15T02:59:20.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.934+0000 7f9cd5437700 1 --2- 172.21.15.121:0/3173735537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cd014e4c0 0x7f9cd00a6610 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f9ccc009760 tx=0x7f9ccc009060).ready entity=mon.1 client_cookie=60bfdef9a691cc1b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:20.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.934+0000 7f9cc7fff700 1 -- 172.21.15.121:0/3173735537 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0001470 con 0x7f9cd014e4c0 2023-11-15T02:59:20.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.934+0000 7f9cc7fff700 1 -- 172.21.15.121:0/3173735537 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ccc011400 con 0x7f9cd014e4c0 2023-11-15T02:59:20.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.934+0000 7f9cc7fff700 1 -- 172.21.15.121:0/3173735537 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc011780 con 0x7f9cd014e4c0 2023-11-15T02:59:20.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.934+0000 7f9cd6439700 1 -- 172.21.15.121:0/3173735537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cd014e4c0 msgr2=0x7f9cd00a6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:20.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.934+0000 7f9cd6439700 1 --2- 172.21.15.121:0/3173735537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cd014e4c0 0x7f9cd00a6610 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f9ccc009760 tx=0x7f9ccc009060).stop 2023-11-15T02:59:20.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.934+0000 7f9cd6439700 1 -- 172.21.15.121:0/3173735537 shutdown_connections 2023-11-15T02:59:20.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.934+0000 7f9cd6439700 1 --2- 172.21.15.121:0/3173735537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cd014e4c0 0x7f9cd00a6610 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:20.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.934+0000 7f9cd6439700 1 -- 172.21.15.121:0/3173735537 >> 172.21.15.121:0/3173735537 conn(0x7f9cd0069350 msgr2=0x7f9cd0090a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:20.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.934+0000 7f9cd6439700 1 -- 172.21.15.121:0/3173735537 shutdown_connections 2023-11-15T02:59:20.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.934+0000 7f9cd6439700 1 -- 172.21.15.121:0/3173735537 wait complete. 2023-11-15T02:59:20.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cd6439700 1 Processor -- start 2023-11-15T02:59:20.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cd6439700 1 -- start start 2023-11-15T02:59:20.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cd6439700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cd00a6b00 0x7f9cd0063c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:20.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cd6439700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cd00c3040 0x7f9cd0153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:20.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cd6439700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd014e4c0 0x7f9cd0152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:20.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cd6439700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd00a7230 con 0x7f9cd014e4c0 2023-11-15T02:59:20.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cd6439700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd00a7510 con 0x7f9cd00a6b00 2023-11-15T02:59:20.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cd6439700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd00a77f0 con 0x7f9cd00c3040 2023-11-15T02:59:20.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cd5c38700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd014e4c0 0x7f9cd0152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:20.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cd5437700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cd00a6b00 0x7f9cd0063c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:20.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cd5c38700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd014e4c0 0x7f9cd0152040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36040/0 (socket says 172.21.15.121:36040) 2023-11-15T02:59:20.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cd5c38700 1 -- 172.21.15.121:0/898157741 learned_addr learned my addr 172.21.15.121:0/898157741 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:20.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cd4c36700 1 --2- 172.21.15.121:0/898157741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cd00c3040 0x7f9cd0153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:20.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cd5437700 1 -- 172.21.15.121:0/898157741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cd00c3040 msgr2=0x7f9cd0153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:20.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cd5437700 1 --2- 172.21.15.121:0/898157741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cd00c3040 0x7f9cd0153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:20.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cd5437700 1 -- 172.21.15.121:0/898157741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd014e4c0 msgr2=0x7f9cd0152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:20.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cd5437700 1 --2- 172.21.15.121:0/898157741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd014e4c0 0x7f9cd0152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:20.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cd5437700 1 -- 172.21.15.121:0/898157741 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc0092a0 con 0x7f9cd00a6b00 2023-11-15T02:59:20.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cd5437700 1 --2- 172.21.15.121:0/898157741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cd00a6b00 0x7f9cd0063c10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f9ccc013000 tx=0x7f9ccc016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:20.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cc57fa700 1 -- 172.21.15.121:0/898157741 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc0106c0 con 0x7f9cd00a6b00 2023-11-15T02:59:20.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cc57fa700 1 -- 172.21.15.121:0/898157741 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ccc010820 con 0x7f9cd00a6b00 2023-11-15T02:59:20.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.938+0000 7f9cc57fa700 1 -- 172.21.15.121:0/898157741 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc010bc0 con 0x7f9cd00a6b00 2023-11-15T02:59:20.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.942+0000 7f9cd6439700 1 -- 172.21.15.121:0/898157741 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd00a7ad0 con 0x7f9cd00a6b00 2023-11-15T02:59:20.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.942+0000 7f9cd6439700 1 -- 172.21.15.121:0/898157741 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd0153f10 con 0x7f9cd00a6b00 2023-11-15T02:59:20.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.942+0000 7f9cd6439700 1 -- 172.21.15.121:0/898157741 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd01540a0 con 0x7f9cd00a6b00 2023-11-15T02:59:20.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.942+0000 7f9cc57fa700 1 -- 172.21.15.121:0/898157741 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9ccc027460 con 0x7f9cd00a6b00 2023-11-15T02:59:20.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.946+0000 7f9cc57fa700 1 --2- 172.21.15.121:0/898157741 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9cb405a510 0x7f9cb405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:20.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.946+0000 7f9cc57fa700 1 -- 172.21.15.121:0/898157741 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f9ccc075bd0 con 0x7f9cd00a6b00 2023-11-15T02:59:20.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.946+0000 7f9cd4c36700 1 --2- 172.21.15.121:0/898157741 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9cb405a510 0x7f9cb405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:20.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.946+0000 7f9cd4c36700 1 --2- 172.21.15.121:0/898157741 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9cb405a510 0x7f9cb405c9d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f9cc0001ed0 tx=0x7f9cc0001990).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:20.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:20.954+0000 7f9cc57fa700 1 -- 172.21.15.121:0/898157741 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9ccc049340 con 0x7f9cd00a6b00 2023-11-15T02:59:21.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.118+0000 7f9cd6439700 1 -- 172.21.15.121:0/898157741 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cd00a7fd0 con 0x7f9cd00a6b00 2023-11-15T02:59:21.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.118+0000 7f9cc57fa700 1 -- 172.21.15.121:0/898157741 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+17107 (secure 0 0 0) 0x7f9ccc04c920 con 0x7f9cd00a6b00 2023-11-15T02:59:21.127 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:21.127 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":49,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:59:12.230103+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:58:55.530878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-15T02:58:56.602591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6837/1945477051":"2023-11-16T02:59:12.230061+0000","172.21.15.121:6836/1945477051":"2023-11-16T02:59:12.230061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:59:21.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.122+0000 7f9cd6439700 1 -- 172.21.15.121:0/898157741 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9cb405a510 msgr2=0x7f9cb405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:21.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.122+0000 7f9cd6439700 1 --2- 172.21.15.121:0/898157741 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9cb405a510 0x7f9cb405c9d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f9cc0001ed0 tx=0x7f9cc0001990).stop 2023-11-15T02:59:21.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.122+0000 7f9cd6439700 1 -- 172.21.15.121:0/898157741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cd00a6b00 msgr2=0x7f9cd0063c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:21.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.122+0000 7f9cd6439700 1 --2- 172.21.15.121:0/898157741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cd00a6b00 0x7f9cd0063c10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f9ccc013000 tx=0x7f9ccc016000).stop 2023-11-15T02:59:21.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.122+0000 7f9cd6439700 1 -- 172.21.15.121:0/898157741 shutdown_connections 2023-11-15T02:59:21.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.122+0000 7f9cd6439700 1 --2- 172.21.15.121:0/898157741 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9cb405a510 0x7f9cb405c9d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:21.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.122+0000 7f9cd6439700 1 --2- 172.21.15.121:0/898157741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd014e4c0 0x7f9cd0152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:21.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.122+0000 7f9cd6439700 1 --2- 172.21.15.121:0/898157741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cd00c3040 0x7f9cd0153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:21.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.122+0000 7f9cd6439700 1 --2- 172.21.15.121:0/898157741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cd00a6b00 0x7f9cd0063c10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:21.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.122+0000 7f9cd6439700 1 -- 172.21.15.121:0/898157741 >> 172.21.15.121:0/898157741 conn(0x7f9cd0069350 msgr2=0x7f9cd00c5420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:21.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.122+0000 7f9cd6439700 1 -- 172.21.15.121:0/898157741 shutdown_connections 2023-11-15T02:59:21.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.122+0000 7f9cd6439700 1 -- 172.21.15.121:0/898157741 wait complete. 2023-11-15T02:59:21.149 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T02:59:21.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.306+0000 7f004cadb700 1 Processor -- start 2023-11-15T02:59:21.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.306+0000 7f004cadb700 1 -- start start 2023-11-15T02:59:21.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.306+0000 7f004cadb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00481508e0 0x7f00480c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:21.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.306+0000 7f004cadb700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0048063f70 0x7f00480643d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:21.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.306+0000 7f004cadb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00480bd100 con 0x7f0048094ca0 2023-11-15T02:59:21.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.306+0000 7f004cadb700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0048062970 con 0x7f0048063f70 2023-11-15T02:59:21.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.306+0000 7f004cadb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0048062c50 con 0x7f00481508e0 2023-11-15T02:59:21.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f0045d9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0048063f70 0x7f00480643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:21.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f0045d9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0048063f70 0x7f00480643d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36052/0 (socket says 172.21.15.121:36052) 2023-11-15T02:59:21.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f0045d9b700 1 -- 172.21.15.121:0/3552925602 learned_addr learned my addr 172.21.15.121:0/3552925602 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:21.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f004659c700 1 --2- 172.21.15.121:0/3552925602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00481508e0 0x7f00480c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:21.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f0045d9b700 1 -- 172.21.15.121:0/3552925602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00481508e0 msgr2=0x7f00480c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:21.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f0045d9b700 1 --2- 172.21.15.121:0/3552925602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00481508e0 0x7f00480c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:21.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f0045d9b700 1 -- 172.21.15.121:0/3552925602 >> v1:172.21.15.190:6789/0 conn(0x7f0048094ca0 legacy=0x7f00480630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:21.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f0045d9b700 1 -- 172.21.15.121:0/3552925602 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00480648c0 con 0x7f0048063f70 2023-11-15T02:59:21.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f0045d9b700 1 --2- 172.21.15.121:0/3552925602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0048063f70 0x7f00480643d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f003c006fd0 tx=0x7f003c0090f0).ready entity=mon.0 client_cookie=da6ee94d65758973 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:21.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f004559a700 1 -- 172.21.15.121:0/3552925602 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c00f6b0 con 0x7f0048063f70 2023-11-15T02:59:21.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f004559a700 1 -- 172.21.15.121:0/3552925602 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c00f810 con 0x7f0048063f70 2023-11-15T02:59:21.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f004559a700 1 -- 172.21.15.121:0/3552925602 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c00fb90 con 0x7f0048063f70 2023-11-15T02:59:21.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f004cadb700 1 -- 172.21.15.121:0/3552925602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0048063f70 msgr2=0x7f00480643d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:21.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f004cadb700 1 --2- 172.21.15.121:0/3552925602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0048063f70 0x7f00480643d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f003c006fd0 tx=0x7f003c0090f0).stop 2023-11-15T02:59:21.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f004cadb700 1 -- 172.21.15.121:0/3552925602 shutdown_connections 2023-11-15T02:59:21.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f004cadb700 1 --2- 172.21.15.121:0/3552925602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0048063f70 0x7f00480643d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:21.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f004cadb700 1 --2- 172.21.15.121:0/3552925602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00481508e0 0x7f00480c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:21.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f004cadb700 1 -- 172.21.15.121:0/3552925602 >> 172.21.15.121:0/3552925602 conn(0x7f0048067480 msgr2=0x7f0048068b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:21.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f004cadb700 1 -- 172.21.15.121:0/3552925602 shutdown_connections 2023-11-15T02:59:21.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.310+0000 7f004cadb700 1 -- 172.21.15.121:0/3552925602 wait complete. 2023-11-15T02:59:21.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f004cadb700 1 Processor -- start 2023-11-15T02:59:21.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f004cadb700 1 -- start start 2023-11-15T02:59:21.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f004cadb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0048063f70 0x7f00480abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:21.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f004cadb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0048094ca0 0x7f004814b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:21.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f004cadb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00481508e0 0x7f00481648c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:21.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f004cadb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004814bda0 con 0x7f0048094ca0 2023-11-15T02:59:21.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f004cadb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00480c5600 con 0x7f0048063f70 2023-11-15T02:59:21.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f004cadb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00480c58e0 con 0x7f00481508e0 2023-11-15T02:59:21.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f0045d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0048094ca0 0x7f004814b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:21.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f0045d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0048094ca0 0x7f004814b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36064/0 (socket says 172.21.15.121:36064) 2023-11-15T02:59:21.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f0045d9b700 1 -- 172.21.15.121:0/4176759362 learned_addr learned my addr 172.21.15.121:0/4176759362 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:21.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f004659c700 1 --2- 172.21.15.121:0/4176759362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0048063f70 0x7f00480abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:21.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f0046d9d700 1 --2- 172.21.15.121:0/4176759362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00481508e0 0x7f00481648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:21.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f0045d9b700 1 -- 172.21.15.121:0/4176759362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00481508e0 msgr2=0x7f00481648c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:21.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f0045d9b700 1 --2- 172.21.15.121:0/4176759362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00481508e0 0x7f00481648c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:21.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f0045d9b700 1 -- 172.21.15.121:0/4176759362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0048063f70 msgr2=0x7f00480abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:21.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f0045d9b700 1 --2- 172.21.15.121:0/4176759362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0048063f70 0x7f00480abde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:21.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f0045d9b700 1 -- 172.21.15.121:0/4176759362 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c009370 con 0x7f0048094ca0 2023-11-15T02:59:21.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f004659c700 1 --2- 172.21.15.121:0/4176759362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0048063f70 0x7f00480abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:21.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f0045d9b700 1 --2- 172.21.15.121:0/4176759362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0048094ca0 0x7f004814b8b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f003c009130 tx=0x7f003c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:21.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f00367fc700 1 -- 172.21.15.121:0/4176759362 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c00fb90 con 0x7f0048094ca0 2023-11-15T02:59:21.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f00367fc700 1 -- 172.21.15.121:0/4176759362 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c000d10 con 0x7f0048094ca0 2023-11-15T02:59:21.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f004cadb700 1 -- 172.21.15.121:0/4176759362 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00481650e0 con 0x7f0048094ca0 2023-11-15T02:59:21.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f00367fc700 1 -- 172.21.15.121:0/4176759362 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c0196a0 con 0x7f0048094ca0 2023-11-15T02:59:21.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.314+0000 7f004cadb700 1 -- 172.21.15.121:0/4176759362 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00481656a0 con 0x7f0048094ca0 2023-11-15T02:59:21.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.318+0000 7f004cadb700 1 -- 172.21.15.121:0/4176759362 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00480c5a70 con 0x7f0048094ca0 2023-11-15T02:59:21.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.318+0000 7f00367fc700 1 -- 172.21.15.121:0/4176759362 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f003c019a90 con 0x7f0048094ca0 2023-11-15T02:59:21.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.318+0000 7f00367fc700 1 --2- 172.21.15.121:0/4176759362 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f002805a510 0x7f002805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:21.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.318+0000 7f00367fc700 1 -- 172.21.15.121:0/4176759362 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f003c075fb0 con 0x7f0048094ca0 2023-11-15T02:59:21.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.322+0000 7f004659c700 1 --2- 172.21.15.121:0/4176759362 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f002805a510 0x7f002805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:21.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.322+0000 7f004659c700 1 --2- 172.21.15.121:0/4176759362 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f002805a510 0x7f002805c9d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f0048062eb0 tx=0x7f0038008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:21.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.322+0000 7f00367fc700 1 -- 172.21.15.121:0/4176759362 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f003c049680 con 0x7f0048094ca0 2023-11-15T02:59:21.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:21.518+0000 7f004cadb700 1 -- 172.21.15.121:0/4176759362 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f00480bd350 con 0x7f0048094ca0 2023-11-15T02:59:22.120 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T02:59:22.114+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T02:59:22.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.130+0000 7f00367fc700 1 -- 172.21.15.121:0/4176759362 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) v1 ==== 161+0+0 (secure 0 0 0) 0x7f003c019d40 con 0x7f0048094ca0 2023-11-15T02:59:22.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.130+0000 7f004cadb700 1 -- 172.21.15.121:0/4176759362 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f002805a510 msgr2=0x7f002805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:22.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.130+0000 7f004cadb700 1 --2- 172.21.15.121:0/4176759362 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f002805a510 0x7f002805c9d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f0048062eb0 tx=0x7f0038008040).stop 2023-11-15T02:59:22.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.130+0000 7f004cadb700 1 -- 172.21.15.121:0/4176759362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0048094ca0 msgr2=0x7f004814b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:22.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.130+0000 7f004cadb700 1 --2- 172.21.15.121:0/4176759362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0048094ca0 0x7f004814b8b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f003c009130 tx=0x7f003c017000).stop 2023-11-15T02:59:22.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.130+0000 7f004cadb700 1 -- 172.21.15.121:0/4176759362 shutdown_connections 2023-11-15T02:59:22.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.130+0000 7f004cadb700 1 --2- 172.21.15.121:0/4176759362 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f002805a510 0x7f002805c9d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:22.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.130+0000 7f004cadb700 1 --2- 172.21.15.121:0/4176759362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00481508e0 0x7f00481648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:22.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.130+0000 7f004cadb700 1 --2- 172.21.15.121:0/4176759362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0048094ca0 0x7f004814b8b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:22.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.130+0000 7f004cadb700 1 --2- 172.21.15.121:0/4176759362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0048063f70 0x7f00480abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:22.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.130+0000 7f004cadb700 1 -- 172.21.15.121:0/4176759362 >> 172.21.15.121:0/4176759362 conn(0x7f0048067480 msgr2=0x7f00480a69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:22.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.130+0000 7f004cadb700 1 -- 172.21.15.121:0/4176759362 shutdown_connections 2023-11-15T02:59:22.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.130+0000 7f004cadb700 1 -- 172.21.15.121:0/4176759362 wait complete. 2023-11-15T02:59:22.141 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T02:59:22.144 INFO:tasks.ceph.mds.c.smithi121.stderr: -88> 2023-11-15T02:59:19.054+0000 7f21e20dc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T02:59:22.159 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-15T02:59:22.195 INFO:tasks.ceph.mds.c.smithi121.stdout:starting mds.c at 2023-11-15T02:59:22.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.314+0000 7feee7569700 1 Processor -- start 2023-11-15T02:59:22.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.314+0000 7feee7569700 1 -- start start 2023-11-15T02:59:22.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.318+0000 7feee7569700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feee00a8b80 0x7feee00a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:22.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.318+0000 7feee7569700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7feee0065850 0x7feee0067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:22.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.318+0000 7feee7569700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee0152150 con 0x7feee0067f50 2023-11-15T02:59:22.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.318+0000 7feee7569700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee0148f70 con 0x7feee0065850 2023-11-15T02:59:22.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.318+0000 7feee7569700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee00683d0 con 0x7feee00a8b80 2023-11-15T02:59:22.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.318+0000 7feee4b04700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7feee0065850 0x7feee0067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:22.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.318+0000 7feee4b04700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7feee0065850 0x7feee0067a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36082/0 (socket says 172.21.15.121:36082) 2023-11-15T02:59:22.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.318+0000 7feee4b04700 1 -- 172.21.15.121:0/191005595 learned_addr learned my addr 172.21.15.121:0/191005595 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:22.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.318+0000 7feee5305700 1 --2- 172.21.15.121:0/191005595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feee00a8b80 0x7feee00a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:22.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.318+0000 7feee4b04700 1 -- 172.21.15.121:0/191005595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feee00a8b80 msgr2=0x7feee00a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:22.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.318+0000 7feee4b04700 1 --2- 172.21.15.121:0/191005595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feee00a8b80 0x7feee00a8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:22.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.318+0000 7feee4b04700 1 -- 172.21.15.121:0/191005595 >> v1:172.21.15.190:6789/0 conn(0x7feee0067f50 legacy=0x7feee00bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:22.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.318+0000 7feee4b04700 1 -- 172.21.15.121:0/191005595 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee01523a0 con 0x7feee0065850 2023-11-15T02:59:22.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.318+0000 7feee4b04700 1 --2- 172.21.15.121:0/191005595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feee0065850 0x7feee0067a60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7feed0006fd0 tx=0x7feed00090f0).ready entity=mon.0 client_cookie=e45f7bf0e009a168 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:22.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.318+0000 7feed7fff700 1 -- 172.21.15.121:0/191005595 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed000b580 con 0x7feee0065850 2023-11-15T02:59:22.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.318+0000 7feed7fff700 1 -- 172.21.15.121:0/191005595 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed000b6e0 con 0x7feee0065850 2023-11-15T02:59:22.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.318+0000 7feed7fff700 1 -- 172.21.15.121:0/191005595 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed000ba60 con 0x7feee0065850 2023-11-15T02:59:22.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.322+0000 7feee7569700 1 -- 172.21.15.121:0/191005595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feee0065850 msgr2=0x7feee0067a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:22.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.322+0000 7feee7569700 1 --2- 172.21.15.121:0/191005595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feee0065850 0x7feee0067a60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7feed0006fd0 tx=0x7feed00090f0).stop 2023-11-15T02:59:22.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.322+0000 7feee7569700 1 -- 172.21.15.121:0/191005595 shutdown_connections 2023-11-15T02:59:22.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.322+0000 7feee7569700 1 --2- 172.21.15.121:0/191005595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feee0065850 0x7feee0067a60 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:22.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.322+0000 7feee7569700 1 --2- 172.21.15.121:0/191005595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feee00a8b80 0x7feee00a8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:22.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.322+0000 7feee7569700 1 -- 172.21.15.121:0/191005595 >> 172.21.15.121:0/191005595 conn(0x7feee00591c0 msgr2=0x7feee00c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:22.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.322+0000 7feee7569700 1 -- 172.21.15.121:0/191005595 shutdown_connections 2023-11-15T02:59:22.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.322+0000 7feee7569700 1 -- 172.21.15.121:0/191005595 wait complete. 2023-11-15T02:59:22.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.322+0000 7feee7569700 1 Processor -- start 2023-11-15T02:59:22.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.322+0000 7feee7569700 1 -- start start 2023-11-15T02:59:22.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee7569700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feee0065850 0x7feee01479f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:22.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee7569700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feee0067f50 0x7feee0149b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:22.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee7569700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feee00a8b80 0x7feee00c0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:22.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee7569700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee0149110 con 0x7feee0067f50 2023-11-15T02:59:22.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee7569700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee01496d0 con 0x7feee0065850 2023-11-15T02:59:22.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee7569700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee00c2770 con 0x7feee00a8b80 2023-11-15T02:59:22.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee4b04700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feee0067f50 0x7feee0149b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:22.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee4b04700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feee0067f50 0x7feee0149b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36086/0 (socket says 172.21.15.121:36086) 2023-11-15T02:59:22.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee4b04700 1 -- 172.21.15.121:0/704717894 learned_addr learned my addr 172.21.15.121:0/704717894 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:22.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee5305700 1 --2- 172.21.15.121:0/704717894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feee0065850 0x7feee01479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:22.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee5b06700 1 --2- 172.21.15.121:0/704717894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feee00a8b80 0x7feee00c0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:22.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee4b04700 1 -- 172.21.15.121:0/704717894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feee00a8b80 msgr2=0x7feee00c0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:22.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee4b04700 1 --2- 172.21.15.121:0/704717894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feee00a8b80 0x7feee00c0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:22.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee4b04700 1 -- 172.21.15.121:0/704717894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feee0065850 msgr2=0x7feee01479f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:22.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee4b04700 1 --2- 172.21.15.121:0/704717894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feee0065850 0x7feee01479f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:22.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee4b04700 1 -- 172.21.15.121:0/704717894 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed0009370 con 0x7feee0067f50 2023-11-15T02:59:22.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee5305700 1 --2- 172.21.15.121:0/704717894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feee0065850 0x7feee01479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:22.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee4b04700 1 --2- 172.21.15.121:0/704717894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feee0067f50 0x7feee0149b60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7feed0009130 tx=0x7feed0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:22.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feed57fa700 1 -- 172.21.15.121:0/704717894 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed000b850 con 0x7feee0067f50 2023-11-15T02:59:22.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feed57fa700 1 -- 172.21.15.121:0/704717894 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed000b9b0 con 0x7feee0067f50 2023-11-15T02:59:22.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feed57fa700 1 -- 172.21.15.121:0/704717894 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed001b660 con 0x7feee0067f50 2023-11-15T02:59:22.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee7569700 1 -- 172.21.15.121:0/704717894 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee00c1970 con 0x7feee0067f50 2023-11-15T02:59:22.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.326+0000 7feee7569700 1 -- 172.21.15.121:0/704717894 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee0149360 con 0x7feee0067f50 2023-11-15T02:59:22.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.330+0000 7feee7569700 1 -- 172.21.15.121:0/704717894 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee0047060 con 0x7feee0067f50 2023-11-15T02:59:22.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.330+0000 7feed57fa700 1 -- 172.21.15.121:0/704717894 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7feed001ba50 con 0x7feee0067f50 2023-11-15T02:59:22.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.330+0000 7feed57fa700 1 --2- 172.21.15.121:0/704717894 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feec405a510 0x7feec405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:22.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.330+0000 7feed57fa700 1 -- 172.21.15.121:0/704717894 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6293+0+0 (secure 0 0 0) 0x7feed0076220 con 0x7feee0067f50 2023-11-15T02:59:22.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.330+0000 7feee5305700 1 --2- 172.21.15.121:0/704717894 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feec405a510 0x7feec405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:22.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.334+0000 7feee5305700 1 --2- 172.21.15.121:0/704717894 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feec405a510 0x7feec405c9d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7feedc009b60 tx=0x7feedc008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:22.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.338+0000 7feed57fa700 1 -- 172.21.15.121:0/704717894 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7feed0049950 con 0x7feee0067f50 2023-11-15T02:59:22.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:22.534+0000 7feee7569700 1 -- 172.21.15.121:0/704717894 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7feee0068b50 con 0x7feee0067f50 2023-11-15T02:59:23.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.134+0000 7feed57fa700 1 -- 172.21.15.121:0/704717894 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) v1 ==== 104+0+0 (secure 0 0 0) 0x7feed00290a0 con 0x7feee0067f50 2023-11-15T02:59:23.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.138+0000 7feee7569700 1 -- 172.21.15.121:0/704717894 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feec405a510 msgr2=0x7feec405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:23.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.138+0000 7feee7569700 1 --2- 172.21.15.121:0/704717894 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feec405a510 0x7feec405c9d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7feedc009b60 tx=0x7feedc008040).stop 2023-11-15T02:59:23.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.138+0000 7feee7569700 1 -- 172.21.15.121:0/704717894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feee0067f50 msgr2=0x7feee0149b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:23.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.138+0000 7feee7569700 1 --2- 172.21.15.121:0/704717894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feee0067f50 0x7feee0149b60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7feed0009130 tx=0x7feed0017000).stop 2023-11-15T02:59:23.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.138+0000 7feee7569700 1 -- 172.21.15.121:0/704717894 shutdown_connections 2023-11-15T02:59:23.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.138+0000 7feee7569700 1 --2- 172.21.15.121:0/704717894 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feec405a510 0x7feec405c9d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:23.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.138+0000 7feee7569700 1 --2- 172.21.15.121:0/704717894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feee00a8b80 0x7feee00c0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:23.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.138+0000 7feee7569700 1 --2- 172.21.15.121:0/704717894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feee0067f50 0x7feee0149b60 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:23.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.138+0000 7feee7569700 1 --2- 172.21.15.121:0/704717894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feee0065850 0x7feee01479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:23.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.138+0000 7feee7569700 1 -- 172.21.15.121:0/704717894 >> 172.21.15.121:0/704717894 conn(0x7feee00591c0 msgr2=0x7feee00c31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:23.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.138+0000 7feee7569700 1 -- 172.21.15.121:0/704717894 shutdown_connections 2023-11-15T02:59:23.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.138+0000 7feee7569700 1 -- 172.21.15.121:0/704717894 wait complete. 2023-11-15T02:59:23.166 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-15T02:59:23.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.370+0000 7fcd86747700 1 Processor -- start 2023-11-15T02:59:23.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.370+0000 7fcd86747700 1 -- start start 2023-11-15T02:59:23.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.374+0000 7fcd86747700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcd800abaa0 0x7fcd800abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:23.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.374+0000 7fcd86747700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd800a8880 0x7fcd80151400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:23.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.374+0000 7fcd86747700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd800c2c00 con 0x7fcd800bb510 2023-11-15T02:59:23.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.374+0000 7fcd86747700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd800657b0 con 0x7fcd800abaa0 2023-11-15T02:59:23.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.374+0000 7fcd86747700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd80065a90 con 0x7fcd800a8880 2023-11-15T02:59:23.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.374+0000 7fcd85745700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcd800abaa0 0x7fcd800abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:23.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.374+0000 7fcd85745700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcd800abaa0 0x7fcd800abe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36094/0 (socket says 172.21.15.121:36094) 2023-11-15T02:59:23.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.374+0000 7fcd85745700 1 -- 172.21.15.121:0/1604666542 learned_addr learned my addr 172.21.15.121:0/1604666542 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:23.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.374+0000 7fcd85f46700 1 --2- 172.21.15.121:0/1604666542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd800a8880 0x7fcd80151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:23.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.374+0000 7fcd85745700 1 -- 172.21.15.121:0/1604666542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd800a8880 msgr2=0x7fcd80151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:23.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.374+0000 7fcd85745700 1 --2- 172.21.15.121:0/1604666542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd800a8880 0x7fcd80151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:23.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.374+0000 7fcd85745700 1 -- 172.21.15.121:0/1604666542 >> v1:172.21.15.190:6789/0 conn(0x7fcd800bb510 legacy=0x7fcd80063570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:23.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.374+0000 7fcd85745700 1 -- 172.21.15.121:0/1604666542 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd800c2890 con 0x7fcd800abaa0 2023-11-15T02:59:23.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.374+0000 7fcd85f46700 1 --2- 172.21.15.121:0/1604666542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd800a8880 0x7fcd80151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:23.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.374+0000 7fcd85745700 1 --2- 172.21.15.121:0/1604666542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd800abaa0 0x7fcd800abe80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7fcd7c006fd0 tx=0x7fcd7c0090f0).ready entity=mon.0 client_cookie=87dae7304996bac5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:23.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.374+0000 7fcd77fff700 1 -- 172.21.15.121:0/1604666542 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c00b600 con 0x7fcd800abaa0 2023-11-15T02:59:23.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.378+0000 7fcd77fff700 1 -- 172.21.15.121:0/1604666542 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7c00b760 con 0x7fcd800abaa0 2023-11-15T02:59:23.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.378+0000 7fcd77fff700 1 -- 172.21.15.121:0/1604666542 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c00bb00 con 0x7fcd800abaa0 2023-11-15T02:59:23.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.378+0000 7fcd86747700 1 -- 172.21.15.121:0/1604666542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd800abaa0 msgr2=0x7fcd800abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:23.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.378+0000 7fcd86747700 1 --2- 172.21.15.121:0/1604666542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd800abaa0 0x7fcd800abe80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7fcd7c006fd0 tx=0x7fcd7c0090f0).stop 2023-11-15T02:59:23.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.378+0000 7fcd86747700 1 -- 172.21.15.121:0/1604666542 shutdown_connections 2023-11-15T02:59:23.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.378+0000 7fcd86747700 1 --2- 172.21.15.121:0/1604666542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd800a8880 0x7fcd80151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:23.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.378+0000 7fcd86747700 1 --2- 172.21.15.121:0/1604666542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd800abaa0 0x7fcd800abe80 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:23.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.378+0000 7fcd86747700 1 -- 172.21.15.121:0/1604666542 >> 172.21.15.121:0/1604666542 conn(0x7fcd80067a60 msgr2=0x7fcd80067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:23.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.378+0000 7fcd86747700 1 -- 172.21.15.121:0/1604666542 shutdown_connections 2023-11-15T02:59:23.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.378+0000 7fcd86747700 1 -- 172.21.15.121:0/1604666542 wait complete. 2023-11-15T02:59:23.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.378+0000 7fcd86747700 1 Processor -- start 2023-11-15T02:59:23.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.378+0000 7fcd86747700 1 -- start start 2023-11-15T02:59:23.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd86747700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd800a8880 0x7fcd801523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:23.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd86747700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd800abaa0 0x7fcd801528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:23.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd86747700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd800bb510 0x7fcd800be770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:23.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd86747700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd80153d40 con 0x7fcd800bb510 2023-11-15T02:59:23.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd86747700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd80153130 con 0x7fcd800a8880 2023-11-15T02:59:23.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd86747700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd80153410 con 0x7fcd800abaa0 2023-11-15T02:59:23.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd85f46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd800bb510 0x7fcd800be770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:23.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd85f46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd800bb510 0x7fcd800be770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36106/0 (socket says 172.21.15.121:36106) 2023-11-15T02:59:23.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd85f46700 1 -- 172.21.15.121:0/2516509976 learned_addr learned my addr 172.21.15.121:0/2516509976 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:23.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd85745700 1 --2- 172.21.15.121:0/2516509976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd800a8880 0x7fcd801523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:23.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd84f44700 1 --2- 172.21.15.121:0/2516509976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd800abaa0 0x7fcd801528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:23.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd85f46700 1 -- 172.21.15.121:0/2516509976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd800abaa0 msgr2=0x7fcd801528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:23.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd85f46700 1 --2- 172.21.15.121:0/2516509976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd800abaa0 0x7fcd801528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:23.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd85f46700 1 -- 172.21.15.121:0/2516509976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd800a8880 msgr2=0x7fcd801523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:23.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd85f46700 1 --2- 172.21.15.121:0/2516509976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd800a8880 0x7fcd801523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:23.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd85f46700 1 -- 172.21.15.121:0/2516509976 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c009370 con 0x7fcd800bb510 2023-11-15T02:59:23.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd85f46700 1 --2- 172.21.15.121:0/2516509976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd800bb510 0x7fcd800be770 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fcd7800cef0 tx=0x7fcd78007860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:23.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd757fa700 1 -- 172.21.15.121:0/2516509976 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7800ad20 con 0x7fcd800bb510 2023-11-15T02:59:23.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd757fa700 1 -- 172.21.15.121:0/2516509976 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7800ae80 con 0x7fcd800bb510 2023-11-15T02:59:23.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd757fa700 1 -- 172.21.15.121:0/2516509976 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd78016610 con 0x7fcd800bb510 2023-11-15T02:59:23.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd86747700 1 -- 172.21.15.121:0/2516509976 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd801536f0 con 0x7fcd800bb510 2023-11-15T02:59:23.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.382+0000 7fcd86747700 1 -- 172.21.15.121:0/2516509976 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd80152da0 con 0x7fcd800bb510 2023-11-15T02:59:23.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.386+0000 7fcd86747700 1 -- 172.21.15.121:0/2516509976 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd801487c0 con 0x7fcd800bb510 2023-11-15T02:59:23.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.386+0000 7fcd757fa700 1 -- 172.21.15.121:0/2516509976 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fcd780203f0 con 0x7fcd800bb510 2023-11-15T02:59:23.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.386+0000 7fcd757fa700 1 --2- 172.21.15.121:0/2516509976 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd6405a4e0 0x7fcd6405c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:23.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.386+0000 7fcd757fa700 1 -- 172.21.15.121:0/2516509976 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fcd78075290 con 0x7fcd800bb510 2023-11-15T02:59:23.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.390+0000 7fcd85745700 1 --2- 172.21.15.121:0/2516509976 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd6405a4e0 0x7fcd6405c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:23.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.390+0000 7fcd85745700 1 --2- 172.21.15.121:0/2516509976 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd6405a4e0 0x7fcd6405c9a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7fcd7c006fd0 tx=0x7fcd7c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:23.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.394+0000 7fcd757fa700 1 -- 172.21.15.121:0/2516509976 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcd780488a0 con 0x7fcd800bb510 2023-11-15T02:59:23.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:23.558+0000 7fcd86747700 1 -- 172.21.15.121:0/2516509976 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcd80153e80 con 0x7fcd800bb510 2023-11-15T02:59:24.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.142+0000 7fcd757fa700 1 -- 172.21.15.121:0/2516509976 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v51) v1 ==== 181+0+0 (secure 0 0 0) 0x7fcd7804bec0 con 0x7fcd800bb510 2023-11-15T02:59:24.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.146+0000 7fcd86747700 1 -- 172.21.15.121:0/2516509976 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd6405a4e0 msgr2=0x7fcd6405c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:24.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.146+0000 7fcd86747700 1 --2- 172.21.15.121:0/2516509976 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd6405a4e0 0x7fcd6405c9a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7fcd7c006fd0 tx=0x7fcd7c017000).stop 2023-11-15T02:59:24.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.146+0000 7fcd86747700 1 -- 172.21.15.121:0/2516509976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd800bb510 msgr2=0x7fcd800be770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:24.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.146+0000 7fcd86747700 1 --2- 172.21.15.121:0/2516509976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd800bb510 0x7fcd800be770 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fcd7800cef0 tx=0x7fcd78007860).stop 2023-11-15T02:59:24.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.146+0000 7fcd86747700 1 -- 172.21.15.121:0/2516509976 shutdown_connections 2023-11-15T02:59:24.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.146+0000 7fcd86747700 1 --2- 172.21.15.121:0/2516509976 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd6405a4e0 0x7fcd6405c9a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:24.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.146+0000 7fcd86747700 1 --2- 172.21.15.121:0/2516509976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd800bb510 0x7fcd800be770 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:24.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.146+0000 7fcd86747700 1 --2- 172.21.15.121:0/2516509976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd800abaa0 0x7fcd801528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:24.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.146+0000 7fcd86747700 1 --2- 172.21.15.121:0/2516509976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd800a8880 0x7fcd801523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:24.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.146+0000 7fcd86747700 1 -- 172.21.15.121:0/2516509976 >> 172.21.15.121:0/2516509976 conn(0x7fcd80067a60 msgr2=0x7fcd800a7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:24.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.146+0000 7fcd86747700 1 -- 172.21.15.121:0/2516509976 shutdown_connections 2023-11-15T02:59:24.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.146+0000 7fcd86747700 1 -- 172.21.15.121:0/2516509976 wait complete. 2023-11-15T02:59:24.158 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_metadata' removed 2023-11-15T02:59:24.176 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-15T02:59:24.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.354+0000 7f2c69d69700 1 Processor -- start 2023-11-15T02:59:24.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.354+0000 7f2c69d69700 1 -- start start 2023-11-15T02:59:24.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.358+0000 7f2c69d69700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2c641570b0 0x7f2c64066860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:24.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.358+0000 7f2c69d69700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2c64157490 0x7f2c64065950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:24.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.358+0000 7f2c69d69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c640c5c80 0x7f2c64065e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:24.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.358+0000 7f2c69d69700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c64066330 con 0x7f2c64157490 2023-11-15T02:59:24.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.358+0000 7f2c69d69700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c64062da0 con 0x7f2c641570b0 2023-11-15T02:59:24.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.358+0000 7f2c69d69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c640bfb90 con 0x7f2c640c5c80 2023-11-15T02:59:24.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.362+0000 7f2c62ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2c64157490 0x7f2c64065950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:24.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.362+0000 7f2c62ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2c64157490 0x7f2c64065950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36118/0 (socket says 172.21.15.121:36118) 2023-11-15T02:59:24.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.362+0000 7f2c62ffd700 1 -- 172.21.15.121:0/1404734673 learned_addr learned my addr 172.21.15.121:0/1404734673 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:24.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.362+0000 7f2c637fe700 1 --2- 172.21.15.121:0/1404734673 >> v2:172.21.15.190:3300/0 conn(0x7f2c641570b0 0x7f2c64066860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:24.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.362+0000 7f2c63fff700 1 --2- 172.21.15.121:0/1404734673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c640c5c80 0x7f2c64065e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:24.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.362+0000 7f2c637fe700 1 -- 172.21.15.121:0/1404734673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c640c5c80 msgr2=0x7f2c64065e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:24.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.362+0000 7f2c637fe700 1 --2- 172.21.15.121:0/1404734673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c640c5c80 0x7f2c64065e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:24.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.362+0000 7f2c637fe700 1 -- 172.21.15.121:0/1404734673 >> v2:172.21.15.121:3300/0 conn(0x7f2c64157490 msgr2=0x7f2c64065950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:24.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.362+0000 7f2c637fe700 1 --2- 172.21.15.121:0/1404734673 >> v2:172.21.15.121:3300/0 conn(0x7f2c64157490 0x7f2c64065950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:24.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.362+0000 7f2c637fe700 1 -- 172.21.15.121:0/1404734673 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c640c6140 con 0x7f2c641570b0 2023-11-15T02:59:24.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.362+0000 7f2c62ffd700 1 --2- 172.21.15.121:0/1404734673 >> v2:172.21.15.121:3300/0 conn(0x7f2c64157490 0x7f2c64065950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:24.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.362+0000 7f2c637fe700 1 --2- 172.21.15.121:0/1404734673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c641570b0 0x7f2c64066860 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f2c640cb6a0 tx=0x7f2c54009060).ready entity=mon.1 client_cookie=eae7563848015746 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:24.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.366+0000 7f2c627fc700 1 -- 172.21.15.121:0/1404734673 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5400f510 con 0x7f2c641570b0 2023-11-15T02:59:24.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.366+0000 7f2c627fc700 1 -- 172.21.15.121:0/1404734673 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5400f670 con 0x7f2c641570b0 2023-11-15T02:59:24.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.366+0000 7f2c627fc700 1 -- 172.21.15.121:0/1404734673 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5400f9f0 con 0x7f2c641570b0 2023-11-15T02:59:24.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.366+0000 7f2c69d69700 1 -- 172.21.15.121:0/1404734673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c641570b0 msgr2=0x7f2c64066860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:24.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.366+0000 7f2c69d69700 1 --2- 172.21.15.121:0/1404734673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c641570b0 0x7f2c64066860 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f2c640cb6a0 tx=0x7f2c54009060).stop 2023-11-15T02:59:24.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.366+0000 7f2c69d69700 1 -- 172.21.15.121:0/1404734673 shutdown_connections 2023-11-15T02:59:24.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.366+0000 7f2c69d69700 1 --2- 172.21.15.121:0/1404734673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c640c5c80 0x7f2c64065e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:24.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.366+0000 7f2c69d69700 1 --2- 172.21.15.121:0/1404734673 >> v2:172.21.15.121:3300/0 conn(0x7f2c64157490 0x7f2c64065950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:24.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.366+0000 7f2c69d69700 1 --2- 172.21.15.121:0/1404734673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c641570b0 0x7f2c64066860 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:24.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.366+0000 7f2c69d69700 1 -- 172.21.15.121:0/1404734673 >> 172.21.15.121:0/1404734673 conn(0x7f2c640684f0 msgr2=0x7f2c640cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:24.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.366+0000 7f2c69d69700 1 -- 172.21.15.121:0/1404734673 shutdown_connections 2023-11-15T02:59:24.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.366+0000 7f2c69d69700 1 -- 172.21.15.121:0/1404734673 wait complete. 2023-11-15T02:59:24.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c69d69700 1 Processor -- start 2023-11-15T02:59:24.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c69d69700 1 -- start start 2023-11-15T02:59:24.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c69d69700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c640c5c80 0x7f2c64152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:24.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c69d69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c641570b0 0x7f2c640ab810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:24.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c69d69700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c640abd00 0x7f2c640c2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:24.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c69d69700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c640ac160 con 0x7f2c640c5c80 2023-11-15T02:59:24.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c69d69700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c640acf90 con 0x7f2c640abd00 2023-11-15T02:59:24.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c69d69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c640c3a80 con 0x7f2c641570b0 2023-11-15T02:59:24.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c637fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c640c5c80 0x7f2c64152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:24.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c637fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c640c5c80 0x7f2c64152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36132/0 (socket says 172.21.15.121:36132) 2023-11-15T02:59:24.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c637fe700 1 -- 172.21.15.121:0/1984025093 learned_addr learned my addr 172.21.15.121:0/1984025093 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:24.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c63fff700 1 --2- 172.21.15.121:0/1984025093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c640abd00 0x7f2c640c2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:24.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c62ffd700 1 --2- 172.21.15.121:0/1984025093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c641570b0 0x7f2c640ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:24.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c637fe700 1 -- 172.21.15.121:0/1984025093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c641570b0 msgr2=0x7f2c640ab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:24.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c637fe700 1 --2- 172.21.15.121:0/1984025093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c641570b0 0x7f2c640ab810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:24.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c637fe700 1 -- 172.21.15.121:0/1984025093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c640abd00 msgr2=0x7f2c640c2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:24.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c637fe700 1 --2- 172.21.15.121:0/1984025093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c640abd00 0x7f2c640c2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:24.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c637fe700 1 -- 172.21.15.121:0/1984025093 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c540092a0 con 0x7f2c640c5c80 2023-11-15T02:59:24.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c62ffd700 1 --2- 172.21.15.121:0/1984025093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c641570b0 0x7f2c640ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:24.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c63fff700 1 --2- 172.21.15.121:0/1984025093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c640abd00 0x7f2c640c2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:24.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c637fe700 1 --2- 172.21.15.121:0/1984025093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c640c5c80 0x7f2c64152610 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f2c640c8740 tx=0x7f2c54017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:24.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.370+0000 7f2c4b7fe700 1 -- 172.21.15.121:0/1984025093 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5400f7e0 con 0x7f2c640c5c80 2023-11-15T02:59:24.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.374+0000 7f2c4b7fe700 1 -- 172.21.15.121:0/1984025093 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5400f960 con 0x7f2c640c5c80 2023-11-15T02:59:24.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.374+0000 7f2c4b7fe700 1 -- 172.21.15.121:0/1984025093 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5401b690 con 0x7f2c640c5c80 2023-11-15T02:59:24.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.374+0000 7f2c69d69700 1 -- 172.21.15.121:0/1984025093 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c640ac380 con 0x7f2c640c5c80 2023-11-15T02:59:24.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.374+0000 7f2c69d69700 1 -- 172.21.15.121:0/1984025093 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c640ac940 con 0x7f2c640c5c80 2023-11-15T02:59:24.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.374+0000 7f2c69d69700 1 -- 172.21.15.121:0/1984025093 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6406c150 con 0x7f2c640c5c80 2023-11-15T02:59:24.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.374+0000 7f2c4b7fe700 1 -- 172.21.15.121:0/1984025093 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2c5401ba00 con 0x7f2c640c5c80 2023-11-15T02:59:24.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.374+0000 7f2c4b7fe700 1 --2- 172.21.15.121:0/1984025093 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c5005a510 0x7f2c5005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:24.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.374+0000 7f2c4b7fe700 1 -- 172.21.15.121:0/1984025093 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5855+0+0 (secure 0 0 0) 0x7f2c54075d50 con 0x7f2c640c5c80 2023-11-15T02:59:24.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.374+0000 7f2c62ffd700 1 --2- 172.21.15.121:0/1984025093 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c5005a510 0x7f2c5005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:24.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.378+0000 7f2c62ffd700 1 --2- 172.21.15.121:0/1984025093 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c5005a510 0x7f2c5005c9d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f2c640c9e60 tx=0x7f2c58008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:24.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.378+0000 7f2c4b7fe700 1 -- 172.21.15.121:0/1984025093 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2c54079030 con 0x7f2c640c5c80 2023-11-15T02:59:24.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:24.542+0000 7f2c69d69700 1 -- 172.21.15.121:0/1984025093 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2c64068b50 con 0x7f2c640c5c80 2023-11-15T02:59:25.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.150+0000 7f2c4b7fe700 1 -- 172.21.15.121:0/1984025093 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2c540280d0 con 0x7f2c640c5c80 2023-11-15T02:59:25.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.154+0000 7f2c69d69700 1 -- 172.21.15.121:0/1984025093 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c5005a510 msgr2=0x7f2c5005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:25.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.154+0000 7f2c69d69700 1 --2- 172.21.15.121:0/1984025093 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c5005a510 0x7f2c5005c9d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f2c640c9e60 tx=0x7f2c58008040).stop 2023-11-15T02:59:25.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.154+0000 7f2c69d69700 1 -- 172.21.15.121:0/1984025093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c640c5c80 msgr2=0x7f2c64152610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:25.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.154+0000 7f2c69d69700 1 --2- 172.21.15.121:0/1984025093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c640c5c80 0x7f2c64152610 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f2c640c8740 tx=0x7f2c54017000).stop 2023-11-15T02:59:25.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.154+0000 7f2c69d69700 1 -- 172.21.15.121:0/1984025093 shutdown_connections 2023-11-15T02:59:25.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.154+0000 7f2c69d69700 1 --2- 172.21.15.121:0/1984025093 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c5005a510 0x7f2c5005c9d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.154+0000 7f2c69d69700 1 --2- 172.21.15.121:0/1984025093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c640abd00 0x7f2c640c2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.154+0000 7f2c69d69700 1 --2- 172.21.15.121:0/1984025093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c641570b0 0x7f2c640ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.154+0000 7f2c69d69700 1 --2- 172.21.15.121:0/1984025093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c640c5c80 0x7f2c64152610 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.154+0000 7f2c69d69700 1 -- 172.21.15.121:0/1984025093 >> 172.21.15.121:0/1984025093 conn(0x7f2c640684f0 msgr2=0x7f2c64155540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:25.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.154+0000 7f2c69d69700 1 -- 172.21.15.121:0/1984025093 shutdown_connections 2023-11-15T02:59:25.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.154+0000 7f2c69d69700 1 -- 172.21.15.121:0/1984025093 wait complete. 2023-11-15T02:59:25.165 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_data' removed 2023-11-15T02:59:25.183 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-15T02:59:25.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.342+0000 7f0286839700 1 Processor -- start 2023-11-15T02:59:25.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.346+0000 7f0286839700 1 -- start start 2023-11-15T02:59:25.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.346+0000 7f0286839700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02800acb60 0x7f02800acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:25.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.346+0000 7f0286839700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0280065680 0x7f0280065b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:25.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.346+0000 7f0286839700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02800c7e90 con 0x7f028014e380 2023-11-15T02:59:25.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0286839700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02800c8320 con 0x7f0280065680 2023-11-15T02:59:25.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0286839700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02800c7080 con 0x7f02800acb60 2023-11-15T02:59:25.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0286038700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0280065680 0x7f0280065b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:25.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0285036700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02800acb60 0x7f02800acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:25.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0286038700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0280065680 0x7f0280065b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:60910/0 (socket says 172.21.15.121:60910) 2023-11-15T02:59:25.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0286038700 1 -- 172.21.15.121:0/3497128196 learned_addr learned my addr 172.21.15.121:0/3497128196 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:25.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0285837700 1 --1- 172.21.15.121:0/3497128196 >> v1:172.21.15.190:6789/0 conn(0x7f028014e380 0x7f02800c5b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:41184/0 (socket says 172.21.15.121:41184) 2023-11-15T02:59:25.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0286038700 1 -- 172.21.15.121:0/3497128196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02800acb60 msgr2=0x7f02800acfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:25.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0286038700 1 --2- 172.21.15.121:0/3497128196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02800acb60 0x7f02800acfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0286038700 1 -- 172.21.15.121:0/3497128196 >> v1:172.21.15.190:6789/0 conn(0x7f028014e380 legacy=0x7f02800c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:25.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0286038700 1 -- 172.21.15.121:0/3497128196 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02800ad4d0 con 0x7f0280065680 2023-11-15T02:59:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0286038700 1 --2- 172.21.15.121:0/3497128196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0280065680 0x7f0280065b10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f0278009f20 tx=0x7f0278009840).ready entity=mon.1 client_cookie=e136486ce0c42e02 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:25.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0284835700 1 -- 172.21.15.121:0/3497128196 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027800e070 con 0x7f0280065680 2023-11-15T02:59:25.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0284835700 1 -- 172.21.15.121:0/3497128196 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027800b9d0 con 0x7f0280065680 2023-11-15T02:59:25.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0284835700 1 -- 172.21.15.121:0/3497128196 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027800bd50 con 0x7f0280065680 2023-11-15T02:59:25.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0286839700 1 -- 172.21.15.121:0/3497128196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0280065680 msgr2=0x7f0280065b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:25.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0286839700 1 --2- 172.21.15.121:0/3497128196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0280065680 0x7f0280065b10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f0278009f20 tx=0x7f0278009840).stop 2023-11-15T02:59:25.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0286839700 1 -- 172.21.15.121:0/3497128196 shutdown_connections 2023-11-15T02:59:25.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0286839700 1 --2- 172.21.15.121:0/3497128196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0280065680 0x7f0280065b10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0286839700 1 --2- 172.21.15.121:0/3497128196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02800acb60 0x7f02800acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.350+0000 7f0286839700 1 -- 172.21.15.121:0/3497128196 >> 172.21.15.121:0/3497128196 conn(0x7f0280067a60 msgr2=0x7f0280067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:25.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.354+0000 7f0286839700 1 -- 172.21.15.121:0/3497128196 shutdown_connections 2023-11-15T02:59:25.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.354+0000 7f0286839700 1 -- 172.21.15.121:0/3497128196 wait complete. 2023-11-15T02:59:25.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.354+0000 7f0286839700 1 Processor -- start 2023-11-15T02:59:25.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.354+0000 7f0286839700 1 -- start start 2023-11-15T02:59:25.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.354+0000 7f0286839700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0280065680 0x7f0280154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:25.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.354+0000 7f0286839700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02800acb60 0x7f02800c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:25.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.354+0000 7f0286839700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f028014e380 0x7f02800ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:25.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.354+0000 7f0286839700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0280155b00 con 0x7f02800acb60 2023-11-15T02:59:25.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.354+0000 7f0286839700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0280155f90 con 0x7f0280065680 2023-11-15T02:59:25.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.354+0000 7f0286839700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0280154ef0 con 0x7f028014e380 2023-11-15T02:59:25.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.354+0000 7f0285036700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02800acb60 0x7f02800c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:25.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.354+0000 7f0285036700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02800acb60 0x7f02800c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43224/0 (socket says 172.21.15.121:43224) 2023-11-15T02:59:25.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.354+0000 7f0285036700 1 -- 172.21.15.121:0/14341016 learned_addr learned my addr 172.21.15.121:0/14341016 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:25.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.354+0000 7f0286038700 1 --2- 172.21.15.121:0/14341016 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f028014e380 0x7f02800ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:25.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.354+0000 7f0285837700 1 --2- 172.21.15.121:0/14341016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0280065680 0x7f0280154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:25.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.358+0000 7f0285036700 1 -- 172.21.15.121:0/14341016 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f028014e380 msgr2=0x7f02800ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:25.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.358+0000 7f0285036700 1 --2- 172.21.15.121:0/14341016 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f028014e380 0x7f02800ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.358+0000 7f0285036700 1 -- 172.21.15.121:0/14341016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0280065680 msgr2=0x7f0280154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:25.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.358+0000 7f0285036700 1 --2- 172.21.15.121:0/14341016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0280065680 0x7f0280154580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.358+0000 7f0285036700 1 -- 172.21.15.121:0/14341016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0278009ae0 con 0x7f02800acb60 2023-11-15T02:59:25.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.358+0000 7f0285036700 1 --2- 172.21.15.121:0/14341016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02800acb60 0x7f02800c40c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f027000e930 tx=0x7f0270009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:25.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.358+0000 7f0275ffb700 1 -- 172.21.15.121:0/14341016 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270012600 con 0x7f02800acb60 2023-11-15T02:59:25.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.358+0000 7f0275ffb700 1 -- 172.21.15.121:0/14341016 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0270012760 con 0x7f02800acb60 2023-11-15T02:59:25.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.358+0000 7f0286839700 1 -- 172.21.15.121:0/14341016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02801551d0 con 0x7f02800acb60 2023-11-15T02:59:25.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.358+0000 7f0275ffb700 1 -- 172.21.15.121:0/14341016 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270012ae0 con 0x7f02800acb60 2023-11-15T02:59:25.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.358+0000 7f0286839700 1 -- 172.21.15.121:0/14341016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0280155790 con 0x7f02800acb60 2023-11-15T02:59:25.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.358+0000 7f0286839700 1 -- 172.21.15.121:0/14341016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0280155c70 con 0x7f02800acb60 2023-11-15T02:59:25.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.362+0000 7f0275ffb700 1 -- 172.21.15.121:0/14341016 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f027001b400 con 0x7f02800acb60 2023-11-15T02:59:25.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.362+0000 7f0275ffb700 1 --2- 172.21.15.121:0/14341016 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f026405a510 0x7f026405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:25.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.362+0000 7f0275ffb700 1 -- 172.21.15.121:0/14341016 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f0270074a50 con 0x7f02800acb60 2023-11-15T02:59:25.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.362+0000 7f0285837700 1 --2- 172.21.15.121:0/14341016 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f026405a510 0x7f026405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:25.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.362+0000 7f0285837700 1 --2- 172.21.15.121:0/14341016 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f026405a510 0x7f026405c9d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f027c007fd0 tx=0x7f027c007d30).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:25.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.366+0000 7f0275ffb700 1 -- 172.21.15.121:0/14341016 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f02700483b0 con 0x7f02800acb60 2023-11-15T02:59:25.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.530+0000 7f0286839700 1 -- 172.21.15.121:0/14341016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f028014e760 con 0x7f02800acb60 2023-11-15T02:59:25.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.538+0000 7f0275ffb700 1 -- 172.21.15.121:0/14341016 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f027004b9d0 con 0x7f02800acb60 2023-11-15T02:59:25.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.542+0000 7f0286839700 1 -- 172.21.15.121:0/14341016 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f026405a510 msgr2=0x7f026405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:25.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.542+0000 7f0286839700 1 --2- 172.21.15.121:0/14341016 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f026405a510 0x7f026405c9d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f027c007fd0 tx=0x7f027c007d30).stop 2023-11-15T02:59:25.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.542+0000 7f0286839700 1 -- 172.21.15.121:0/14341016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02800acb60 msgr2=0x7f02800c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:25.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.542+0000 7f0286839700 1 --2- 172.21.15.121:0/14341016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02800acb60 0x7f02800c40c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f027000e930 tx=0x7f0270009060).stop 2023-11-15T02:59:25.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.542+0000 7f0286839700 1 -- 172.21.15.121:0/14341016 shutdown_connections 2023-11-15T02:59:25.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.542+0000 7f0286839700 1 --2- 172.21.15.121:0/14341016 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f026405a510 0x7f026405c9d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.542+0000 7f0286839700 1 --2- 172.21.15.121:0/14341016 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f028014e380 0x7f02800ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.542+0000 7f0286839700 1 --2- 172.21.15.121:0/14341016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02800acb60 0x7f02800c40c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.542+0000 7f0286839700 1 --2- 172.21.15.121:0/14341016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0280065680 0x7f0280154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.542+0000 7f0286839700 1 -- 172.21.15.121:0/14341016 >> 172.21.15.121:0/14341016 conn(0x7f0280067a60 msgr2=0x7f02800c0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:25.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.542+0000 7f0286839700 1 -- 172.21.15.121:0/14341016 shutdown_connections 2023-11-15T02:59:25.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.542+0000 7f0286839700 1 -- 172.21.15.121:0/14341016 wait complete. 2023-11-15T02:59:25.569 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_abort_on_newly_corrupt_dentry 2023-11-15T02:59:25.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.722+0000 7fdb8b59e700 1 Processor -- start 2023-11-15T02:59:25.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.722+0000 7fdb8b59e700 1 -- start start 2023-11-15T02:59:25.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.726+0000 7fdb8b59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdb8c0acd00 0x7fdb8c0ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:25.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.726+0000 7fdb8b59e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8c065ea0 con 0x7fdb8c066330 2023-11-15T02:59:25.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.726+0000 7fdb8b59e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8c066bb0 con 0x7fdb8c0c5c70 2023-11-15T02:59:25.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.726+0000 7fdb8b59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb8c065890 con 0x7fdb8c0acd00 2023-11-15T02:59:25.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.726+0000 7fdb8a59c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdb8c0acd00 0x7fdb8c0ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:25.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.726+0000 7fdb8ad9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fdb8c066330 0x7fdb8c1577e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58648/0 (socket says 172.21.15.121:58648) 2023-11-15T02:59:25.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.726+0000 7fdb8ad9d700 1 -- 172.21.15.121:0/1410091623 learned_addr learned my addr 172.21.15.121:0/1410091623 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:25.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.726+0000 7fdb8a59c700 1 -- 172.21.15.121:0/1410091623 >> v1:172.21.15.190:6789/0 conn(0x7fdb8c0c5c70 legacy=0x7fdb8c1536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:25.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.726+0000 7fdb8a59c700 1 -- 172.21.15.121:0/1410091623 >> v1:172.21.15.121:6789/0 conn(0x7fdb8c066330 legacy=0x7fdb8c1577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:25.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.726+0000 7fdb8959a700 1 -- 172.21.15.121:0/1410091623 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 252758183 0 0) 0x7fdb8c065ea0 con 0x7fdb8c066330 2023-11-15T02:59:25.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.726+0000 7fdb8a59c700 1 -- 172.21.15.121:0/1410091623 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c0c64a0 con 0x7fdb8c0acd00 2023-11-15T02:59:25.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.726+0000 7fdb8a59c700 1 --2- 172.21.15.121:0/1410091623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c0acd00 0x7fdb8c0ad0e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fdb7c006fd0 tx=0x7fdb7c006f40).ready entity=mon.0 client_cookie=4f45279fcd45ac29 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:25.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.730+0000 7fdb8959a700 1 -- 172.21.15.121:0/1410091623 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c005880 con 0x7fdb8c0acd00 2023-11-15T02:59:25.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.730+0000 7fdb8959a700 1 -- 172.21.15.121:0/1410091623 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7c0059e0 con 0x7fdb8c0acd00 2023-11-15T02:59:25.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.730+0000 7fdb8959a700 1 -- 172.21.15.121:0/1410091623 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c005d60 con 0x7fdb8c0acd00 2023-11-15T02:59:25.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.730+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1410091623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c0acd00 msgr2=0x7fdb8c0ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:25.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.730+0000 7fdb8b59e700 1 --2- 172.21.15.121:0/1410091623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c0acd00 0x7fdb8c0ad0e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fdb7c006fd0 tx=0x7fdb7c006f40).stop 2023-11-15T02:59:25.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.730+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1410091623 shutdown_connections 2023-11-15T02:59:25.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.730+0000 7fdb8b59e700 1 --2- 172.21.15.121:0/1410091623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c0acd00 0x7fdb8c0ad0e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.730+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1410091623 >> 172.21.15.121:0/1410091623 conn(0x7fdb8c063fc0 msgr2=0x7fdb8c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:25.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.730+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1410091623 shutdown_connections 2023-11-15T02:59:25.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.730+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1410091623 wait complete. 2023-11-15T02:59:25.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb8b59e700 1 Processor -- start 2023-11-15T02:59:25.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb8b59e700 1 -- start start 2023-11-15T02:59:25.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb8b59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c066330 0x7fdb8c169150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:25.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb8b59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb8c0acd00 0x7fdb8c14d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:25.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb8b59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb8c0c5c70 0x7fdb8c150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:25.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb8b59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c1500e0 con 0x7fdb8c066330 2023-11-15T02:59:25.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb8b59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c156d10 con 0x7fdb8c0acd00 2023-11-15T02:59:25.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb8b59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c156fb0 con 0x7fdb8c0c5c70 2023-11-15T02:59:25.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb8a59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c066330 0x7fdb8c169150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:25.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb8a59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c066330 0x7fdb8c169150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43248/0 (socket says 172.21.15.121:43248) 2023-11-15T02:59:25.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb8a59c700 1 -- 172.21.15.121:0/1835274667 learned_addr learned my addr 172.21.15.121:0/1835274667 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:25.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb89d9b700 1 --2- 172.21.15.121:0/1835274667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb8c0acd00 0x7fdb8c14d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:25.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb8ad9d700 1 --2- 172.21.15.121:0/1835274667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb8c0c5c70 0x7fdb8c150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:25.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb8a59c700 1 -- 172.21.15.121:0/1835274667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb8c0c5c70 msgr2=0x7fdb8c150570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:25.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb8a59c700 1 --2- 172.21.15.121:0/1835274667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb8c0c5c70 0x7fdb8c150570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb8a59c700 1 -- 172.21.15.121:0/1835274667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb8c0acd00 msgr2=0x7fdb8c14d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:25.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb8a59c700 1 --2- 172.21.15.121:0/1835274667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb8c0acd00 0x7fdb8c14d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb8a59c700 1 -- 172.21.15.121:0/1835274667 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c009250 con 0x7fdb8c066330 2023-11-15T02:59:25.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb8ad9d700 1 --2- 172.21.15.121:0/1835274667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb8c0c5c70 0x7fdb8c150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:25.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb8a59c700 1 --2- 172.21.15.121:0/1835274667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c066330 0x7fdb8c169150 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fdb7c006fa0 tx=0x7fdb7c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:25.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.734+0000 7fdb7a7fc700 1 -- 172.21.15.121:0/1835274667 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c00b790 con 0x7fdb8c066330 2023-11-15T02:59:25.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.738+0000 7fdb7a7fc700 1 -- 172.21.15.121:0/1835274667 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7c00b8f0 con 0x7fdb8c066330 2023-11-15T02:59:25.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.738+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1835274667 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8c157290 con 0x7fdb8c066330 2023-11-15T02:59:25.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.738+0000 7fdb7a7fc700 1 -- 172.21.15.121:0/1835274667 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c00bc70 con 0x7fdb8c066330 2023-11-15T02:59:25.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.738+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1835274667 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8c14db20 con 0x7fdb8c066330 2023-11-15T02:59:25.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.738+0000 7fdb7a7fc700 1 -- 172.21.15.121:0/1835274667 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fdb7c011650 con 0x7fdb8c066330 2023-11-15T02:59:25.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.738+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1835274667 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb8c06bcf0 con 0x7fdb8c066330 2023-11-15T02:59:25.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.738+0000 7fdb7a7fc700 1 --2- 172.21.15.121:0/1835274667 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdb6c05a510 0x7fdb6c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:25.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.738+0000 7fdb7a7fc700 1 -- 172.21.15.121:0/1835274667 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5441+0+0 (secure 0 0 0) 0x7fdb7c075ca0 con 0x7fdb8c066330 2023-11-15T02:59:25.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.738+0000 7fdb89d9b700 1 --2- 172.21.15.121:0/1835274667 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdb6c05a510 0x7fdb6c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:25.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.742+0000 7fdb89d9b700 1 --2- 172.21.15.121:0/1835274667 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdb6c05a510 0x7fdb6c05c9d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fdb8c0b0470 tx=0x7fdb80007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:25.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.742+0000 7fdb7a7fc700 1 -- 172.21.15.121:0/1835274667 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdb7c07a070 con 0x7fdb8c066330 2023-11-15T02:59:25.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.906+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1835274667 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_abort_on_newly_corrupt_dentry"} v 0) v1 -- 0x7fdb8c0c61c0 con 0x7fdb8c066330 2023-11-15T02:59:25.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.914+0000 7fdb7a7fc700 1 -- 172.21.15.121:0/1835274667 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_abort_on_newly_corrupt_dentry"}]=0 v9) v1 ==== 116+0+0 (secure 0 0 0) 0x7fdb7c0497d0 con 0x7fdb8c066330 2023-11-15T02:59:25.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.918+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1835274667 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdb6c05a510 msgr2=0x7fdb6c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:25.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.918+0000 7fdb8b59e700 1 --2- 172.21.15.121:0/1835274667 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdb6c05a510 0x7fdb6c05c9d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fdb8c0b0470 tx=0x7fdb80007d40).stop 2023-11-15T02:59:25.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.918+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1835274667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c066330 msgr2=0x7fdb8c169150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:25.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.918+0000 7fdb8b59e700 1 --2- 172.21.15.121:0/1835274667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c066330 0x7fdb8c169150 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fdb7c006fa0 tx=0x7fdb7c016000).stop 2023-11-15T02:59:25.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.918+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1835274667 shutdown_connections 2023-11-15T02:59:25.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.918+0000 7fdb8b59e700 1 --2- 172.21.15.121:0/1835274667 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdb6c05a510 0x7fdb6c05c9d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.918+0000 7fdb8b59e700 1 --2- 172.21.15.121:0/1835274667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb8c0c5c70 0x7fdb8c150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.918+0000 7fdb8b59e700 1 --2- 172.21.15.121:0/1835274667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb8c0acd00 0x7fdb8c14d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.918+0000 7fdb8b59e700 1 --2- 172.21.15.121:0/1835274667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c066330 0x7fdb8c169150 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:25.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.918+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1835274667 >> 172.21.15.121:0/1835274667 conn(0x7fdb8c063fc0 msgr2=0x7fdb8c0caf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:25.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.918+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1835274667 shutdown_connections 2023-11-15T02:59:25.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:25.918+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1835274667 wait complete. 2023-11-15T02:59:25.943 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2023-11-15T02:59:26.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.102+0000 7f910d6db700 1 Processor -- start 2023-11-15T02:59:26.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.102+0000 7f910d6db700 1 -- start start 2023-11-15T02:59:26.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f910d6db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9108151460 0x7f91080c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:26.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f910d6db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9108151840 0x7f91080c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:26.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f910d6db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f91080c3a90 0x7f9108066140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:26.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f910d6db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91080a3ae0 con 0x7f91080c3a90 2023-11-15T02:59:26.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f910d6db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91080c1d30 con 0x7f9108151840 2023-11-15T02:59:26.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f910d6db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91080c1470 con 0x7f9108151460 2023-11-15T02:59:26.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f91077fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f91080c3a90 0x7f9108066140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:26.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f91077fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f91080c3a90 0x7f9108066140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43250/0 (socket says 172.21.15.121:43250) 2023-11-15T02:59:26.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f91077fe700 1 -- 172.21.15.121:0/485526563 learned_addr learned my addr 172.21.15.121:0/485526563 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:26.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f9106ffd700 1 --2- 172.21.15.121:0/485526563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9108151460 0x7f91080c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:26.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f91067fc700 1 --2- 172.21.15.121:0/485526563 >> v2:172.21.15.190:3300/0 conn(0x7f9108151840 0x7f91080c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:26.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f91077fe700 1 -- 172.21.15.121:0/485526563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9108151460 msgr2=0x7f91080c21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:26.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f91077fe700 1 --2- 172.21.15.121:0/485526563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9108151460 0x7f91080c21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:26.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f91077fe700 1 -- 172.21.15.121:0/485526563 >> v2:172.21.15.190:3300/0 conn(0x7f9108151840 msgr2=0x7f91080c35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:26.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f91077fe700 1 --2- 172.21.15.121:0/485526563 >> v2:172.21.15.190:3300/0 conn(0x7f9108151840 0x7f91080c35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:26.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f91077fe700 1 -- 172.21.15.121:0/485526563 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91080c16e0 con 0x7f91080c3a90 2023-11-15T02:59:26.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f91077fe700 1 --2- 172.21.15.121:0/485526563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91080c3a90 0x7f9108066140 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f91080659e0 tx=0x7f9100009840).ready entity=mon.0 client_cookie=87712f7e9207cc8a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:26.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f9105ffb700 1 -- 172.21.15.121:0/485526563 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910000e070 con 0x7f91080c3a90 2023-11-15T02:59:26.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f9105ffb700 1 -- 172.21.15.121:0/485526563 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910000bab0 con 0x7f91080c3a90 2023-11-15T02:59:26.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.106+0000 7f9105ffb700 1 -- 172.21.15.121:0/485526563 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910000be30 con 0x7f91080c3a90 2023-11-15T02:59:26.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f910d6db700 1 -- 172.21.15.121:0/485526563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91080c3a90 msgr2=0x7f9108066140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:26.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f910d6db700 1 --2- 172.21.15.121:0/485526563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91080c3a90 0x7f9108066140 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f91080659e0 tx=0x7f9100009840).stop 2023-11-15T02:59:26.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f910d6db700 1 -- 172.21.15.121:0/485526563 shutdown_connections 2023-11-15T02:59:26.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f910d6db700 1 --2- 172.21.15.121:0/485526563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91080c3a90 0x7f9108066140 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:26.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f910d6db700 1 --2- 172.21.15.121:0/485526563 >> v2:172.21.15.190:3300/0 conn(0x7f9108151840 0x7f91080c35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:26.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f910d6db700 1 --2- 172.21.15.121:0/485526563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9108151460 0x7f91080c21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:26.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f910d6db700 1 -- 172.21.15.121:0/485526563 >> 172.21.15.121:0/485526563 conn(0x7f9108062190 msgr2=0x7f91080625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:26.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f910d6db700 1 -- 172.21.15.121:0/485526563 shutdown_connections 2023-11-15T02:59:26.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f910d6db700 1 -- 172.21.15.121:0/485526563 wait complete. 2023-11-15T02:59:26.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f910d6db700 1 Processor -- start 2023-11-15T02:59:26.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f910d6db700 1 -- start start 2023-11-15T02:59:26.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f910d6db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91080c3a90 0x7f91081451e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:26.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f910d6db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9108151460 0x7f91080aa740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:26.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f910d6db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9108151840 0x7f91080c0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:26.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f910d6db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91080bfb70 con 0x7f9108151460 2023-11-15T02:59:26.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f910d6db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91080bef60 con 0x7f91080c3a90 2023-11-15T02:59:26.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f910d6db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91080bf240 con 0x7f9108151840 2023-11-15T02:59:26.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f91067fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9108151460 0x7f91080aa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:26.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f91067fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9108151460 0x7f91080aa740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43258/0 (socket says 172.21.15.121:43258) 2023-11-15T02:59:26.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.110+0000 7f91067fc700 1 -- 172.21.15.121:0/3250879809 learned_addr learned my addr 172.21.15.121:0/3250879809 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:26.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.114+0000 7f9106ffd700 1 --2- 172.21.15.121:0/3250879809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91080c3a90 0x7f91081451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:26.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.114+0000 7f91077fe700 1 --2- 172.21.15.121:0/3250879809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9108151840 0x7f91080c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:26.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.114+0000 7f91067fc700 1 -- 172.21.15.121:0/3250879809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9108151840 msgr2=0x7f91080c0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:26.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.114+0000 7f91067fc700 1 --2- 172.21.15.121:0/3250879809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9108151840 0x7f91080c0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:26.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.114+0000 7f91067fc700 1 -- 172.21.15.121:0/3250879809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91080c3a90 msgr2=0x7f91081451e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:26.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.114+0000 7f91067fc700 1 --2- 172.21.15.121:0/3250879809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91080c3a90 0x7f91081451e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:26.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.114+0000 7f91067fc700 1 -- 172.21.15.121:0/3250879809 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9100009ae0 con 0x7f9108151460 2023-11-15T02:59:26.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.114+0000 7f91067fc700 1 --2- 172.21.15.121:0/3250879809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9108151460 0x7f91080aa740 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f90fc00d890 tx=0x7f90fc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:26.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.114+0000 7f90eeffd700 1 -- 172.21.15.121:0/3250879809 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc00f520 con 0x7f9108151460 2023-11-15T02:59:26.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.114+0000 7f90eeffd700 1 -- 172.21.15.121:0/3250879809 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90fc00f680 con 0x7f9108151460 2023-11-15T02:59:26.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.114+0000 7f90eeffd700 1 -- 172.21.15.121:0/3250879809 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc00fa00 con 0x7f9108151460 2023-11-15T02:59:26.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.114+0000 7f910d6db700 1 -- 172.21.15.121:0/3250879809 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91080bf520 con 0x7f9108151460 2023-11-15T02:59:26.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.114+0000 7f910d6db700 1 -- 172.21.15.121:0/3250879809 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91081456d0 con 0x7f9108151460 2023-11-15T02:59:26.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.114+0000 7f910d6db700 1 -- 172.21.15.121:0/3250879809 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91080bfce0 con 0x7f9108151460 2023-11-15T02:59:26.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.118+0000 7f90eeffd700 1 -- 172.21.15.121:0/3250879809 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f90fc013480 con 0x7f9108151460 2023-11-15T02:59:26.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.118+0000 7f90eeffd700 1 --2- 172.21.15.121:0/3250879809 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f90f405a540 0x7f90f405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:26.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.118+0000 7f90eeffd700 1 -- 172.21.15.121:0/3250879809 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f90fc074090 con 0x7f9108151460 2023-11-15T02:59:26.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.118+0000 7f9106ffd700 1 --2- 172.21.15.121:0/3250879809 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f90f405a540 0x7f90f405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:26.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.118+0000 7f9106ffd700 1 --2- 172.21.15.121:0/3250879809 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f90f405a540 0x7f90f405ca00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f91080c0b80 tx=0x7f90f8009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:26.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.126+0000 7f90eeffd700 1 -- 172.21.15.121:0/3250879809 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f90fc0479f0 con 0x7f9108151460 2023-11-15T02:59:26.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.290+0000 7f910d6db700 1 -- 172.21.15.121:0/3250879809 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) v1 -- 0x7f9108058f60 con 0x7f9108151460 2023-11-15T02:59:26.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.294+0000 7f90eeffd700 1 -- 172.21.15.121:0/3250879809 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v9) v1 ==== 121+0+0 (secure 0 0 0) 0x7f90fc04b010 con 0x7f9108151460 2023-11-15T02:59:26.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.298+0000 7f910d6db700 1 -- 172.21.15.121:0/3250879809 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f90f405a540 msgr2=0x7f90f405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:26.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.298+0000 7f910d6db700 1 --2- 172.21.15.121:0/3250879809 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f90f405a540 0x7f90f405ca00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f91080c0b80 tx=0x7f90f8009060).stop 2023-11-15T02:59:26.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.298+0000 7f910d6db700 1 -- 172.21.15.121:0/3250879809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9108151460 msgr2=0x7f91080aa740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:26.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.298+0000 7f910d6db700 1 --2- 172.21.15.121:0/3250879809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9108151460 0x7f91080aa740 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f90fc00d890 tx=0x7f90fc008040).stop 2023-11-15T02:59:26.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.298+0000 7f910d6db700 1 -- 172.21.15.121:0/3250879809 shutdown_connections 2023-11-15T02:59:26.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.298+0000 7f910d6db700 1 --2- 172.21.15.121:0/3250879809 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f90f405a540 0x7f90f405ca00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:26.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.298+0000 7f910d6db700 1 --2- 172.21.15.121:0/3250879809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9108151840 0x7f91080c0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:26.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.298+0000 7f910d6db700 1 --2- 172.21.15.121:0/3250879809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9108151460 0x7f91080aa740 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:26.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.298+0000 7f910d6db700 1 --2- 172.21.15.121:0/3250879809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91080c3a90 0x7f91081451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:26.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.298+0000 7f910d6db700 1 -- 172.21.15.121:0/3250879809 >> 172.21.15.121:0/3250879809 conn(0x7f9108062190 msgr2=0x7f91080a5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:26.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.298+0000 7f910d6db700 1 -- 172.21.15.121:0/3250879809 shutdown_connections 2023-11-15T02:59:26.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.298+0000 7f910d6db700 1 -- 172.21.15.121:0/3250879809 wait complete. 2023-11-15T02:59:26.325 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing' 2023-11-15T02:59:26.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.482+0000 7f82be569700 1 Processor -- start 2023-11-15T02:59:26.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.486+0000 7f82be569700 1 -- start start 2023-11-15T02:59:26.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.486+0000 7f82be569700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f82b814af80 0x7f82b80c1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:26.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.486+0000 7f82be569700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b8152000 con 0x7f82b80c6640 2023-11-15T02:59:26.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.486+0000 7f82be569700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b80c1270 con 0x7f82b80c1bf0 2023-11-15T02:59:26.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.486+0000 7f82be569700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b8151a40 con 0x7f82b814af80 2023-11-15T02:59:26.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.486+0000 7f82bcb06700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f82b80c6640 0x7f82b80a79d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58658/0 (socket says 172.21.15.121:58658) 2023-11-15T02:59:26.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.486+0000 7f82bcb06700 1 -- 172.21.15.121:0/4290656375 learned_addr learned my addr 172.21.15.121:0/4290656375 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:26.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.486+0000 7f82b7fff700 1 --2- 172.21.15.121:0/4290656375 >> v2:172.21.15.190:3300/0 conn(0x7f82b814af80 0x7f82b80c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:26.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.490+0000 7f82b77fe700 1 -- 172.21.15.121:0/4290656375 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000736213 0 0) 0x7f82b0001490 con 0x7f82b80c6640 2023-11-15T02:59:26.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.490+0000 7f82b77fe700 1 -- 172.21.15.121:0/4290656375 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82a40033f0 con 0x7f82b80c6640 2023-11-15T02:59:26.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.490+0000 7f82b77fe700 1 -- 172.21.15.121:0/4290656375 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443743228 0 0) 0x7f82b80c1270 con 0x7f82b80c1bf0 2023-11-15T02:59:26.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.490+0000 7f82b77fe700 1 -- 172.21.15.121:0/4290656375 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82b0001490 con 0x7f82b80c1bf0 2023-11-15T02:59:26.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.490+0000 7f82b7fff700 1 -- 172.21.15.121:0/4290656375 >> v1:172.21.15.190:6789/0 conn(0x7f82b80c1bf0 legacy=0x7f82b8150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:26.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.490+0000 7f82b7fff700 1 -- 172.21.15.121:0/4290656375 >> v1:172.21.15.121:6789/0 conn(0x7f82b80c6640 legacy=0x7f82b80a79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:26.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.490+0000 7f82b7fff700 1 -- 172.21.15.121:0/4290656375 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b8151d20 con 0x7f82b814af80 2023-11-15T02:59:26.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.490+0000 7f82b7fff700 1 --2- 172.21.15.121:0/4290656375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82b814af80 0x7f82b80c1700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f82a8009760 tx=0x7f82a8009060).ready entity=mon.1 client_cookie=319e965efecacaa8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:26.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.490+0000 7f82b77fe700 1 -- 172.21.15.121:0/4290656375 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a800d4a0 con 0x7f82b814af80 2023-11-15T02:59:26.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.490+0000 7f82b77fe700 1 -- 172.21.15.121:0/4290656375 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a800d600 con 0x7f82b814af80 2023-11-15T02:59:26.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.490+0000 7f82b77fe700 1 -- 172.21.15.121:0/4290656375 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a800d980 con 0x7f82b814af80 2023-11-15T02:59:26.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.490+0000 7f82be569700 1 -- 172.21.15.121:0/4290656375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82b814af80 msgr2=0x7f82b80c1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:26.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.490+0000 7f82be569700 1 --2- 172.21.15.121:0/4290656375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82b814af80 0x7f82b80c1700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f82a8009760 tx=0x7f82a8009060).stop 2023-11-15T02:59:26.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.490+0000 7f82be569700 1 -- 172.21.15.121:0/4290656375 shutdown_connections 2023-11-15T02:59:26.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.490+0000 7f82be569700 1 --2- 172.21.15.121:0/4290656375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82b814af80 0x7f82b80c1700 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:26.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.490+0000 7f82be569700 1 -- 172.21.15.121:0/4290656375 >> 172.21.15.121:0/4290656375 conn(0x7f82b8062190 msgr2=0x7f82b80625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:26.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.490+0000 7f82be569700 1 -- 172.21.15.121:0/4290656375 shutdown_connections 2023-11-15T02:59:26.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.490+0000 7f82be569700 1 -- 172.21.15.121:0/4290656375 wait complete. 2023-11-15T02:59:26.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.494+0000 7f82be569700 1 Processor -- start 2023-11-15T02:59:26.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.494+0000 7f82be569700 1 -- start start 2023-11-15T02:59:26.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82be569700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82b80c1bf0 0x7f82b80aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:26.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82be569700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82b80c6640 0x7f82b81498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:26.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82be569700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82b814af80 0x7f82b80c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:26.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82be569700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b80aa850 con 0x7f82b80c1bf0 2023-11-15T02:59:26.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82be569700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b814e640 con 0x7f82b80c6640 2023-11-15T02:59:26.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82be569700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b81521a0 con 0x7f82b814af80 2023-11-15T02:59:26.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82b7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82b80c1bf0 0x7f82b80aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:26.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82b7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82b80c1bf0 0x7f82b80aa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43266/0 (socket says 172.21.15.121:43266) 2023-11-15T02:59:26.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82b7fff700 1 -- 172.21.15.121:0/2847495623 learned_addr learned my addr 172.21.15.121:0/2847495623 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:26.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82af7fe700 1 --2- 172.21.15.121:0/2847495623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82b80c6640 0x7f82b81498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:26.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82bcb06700 1 --2- 172.21.15.121:0/2847495623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82b814af80 0x7f82b80c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:26.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82b7fff700 1 -- 172.21.15.121:0/2847495623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82b814af80 msgr2=0x7f82b80c47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:26.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82b7fff700 1 --2- 172.21.15.121:0/2847495623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82b814af80 0x7f82b80c47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:26.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82b7fff700 1 -- 172.21.15.121:0/2847495623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82b80c6640 msgr2=0x7f82b81498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:26.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82b7fff700 1 --2- 172.21.15.121:0/2847495623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82b80c6640 0x7f82b81498a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:26.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82b7fff700 1 -- 172.21.15.121:0/2847495623 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a80092a0 con 0x7f82b80c1bf0 2023-11-15T02:59:26.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82b7fff700 1 --2- 172.21.15.121:0/2847495623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82b80c1bf0 0x7f82b80aa0e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f82b8064f10 tx=0x7f82a8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:26.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82b4ff9700 1 -- 172.21.15.121:0/2847495623 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a800d770 con 0x7f82b80c1bf0 2023-11-15T02:59:26.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82b4ff9700 1 -- 172.21.15.121:0/2847495623 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a800d8d0 con 0x7f82b80c1bf0 2023-11-15T02:59:26.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82b4ff9700 1 -- 172.21.15.121:0/2847495623 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a801d660 con 0x7f82b80c1bf0 2023-11-15T02:59:26.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82be569700 1 -- 172.21.15.121:0/2847495623 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b8152d20 con 0x7f82b80c1bf0 2023-11-15T02:59:26.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82be569700 1 -- 172.21.15.121:0/2847495623 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b81523f0 con 0x7f82b80c1bf0 2023-11-15T02:59:26.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.498+0000 7f82be569700 1 -- 172.21.15.121:0/2847495623 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b806c150 con 0x7f82b80c1bf0 2023-11-15T02:59:26.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.502+0000 7f82b4ff9700 1 -- 172.21.15.121:0/2847495623 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f82a801da50 con 0x7f82b80c1bf0 2023-11-15T02:59:26.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.502+0000 7f82b4ff9700 1 --2- 172.21.15.121:0/2847495623 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f829805a510 0x7f829805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:26.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.502+0000 7f82b4ff9700 1 -- 172.21.15.121:0/2847495623 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f82a8075c50 con 0x7f82b80c1bf0 2023-11-15T02:59:26.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.502+0000 7f82af7fe700 1 --2- 172.21.15.121:0/2847495623 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f829805a510 0x7f829805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:26.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.506+0000 7f82af7fe700 1 --2- 172.21.15.121:0/2847495623 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f829805a510 0x7f829805c9d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f82b80a8650 tx=0x7f82a0009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:26.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.510+0000 7f82b4ff9700 1 -- 172.21.15.121:0/2847495623 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f82a807b020 con 0x7f82b80c1bf0 2023-11-15T02:59:26.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:26.726+0000 7f82be569700 1 -- 172.21.15.121:0/2847495623 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]} v 0) v1 -- 0x7f82b80c0f00 con 0x7f82b80c1bf0 2023-11-15T02:59:27.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.170+0000 7f82b4ff9700 1 -- 172.21.15.121:0/2847495623 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]}]=0 v152) v1 ==== 141+0+0 (secure 0 0 0) 0x7f82a80496d0 con 0x7f82b80c1bf0 2023-11-15T02:59:27.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.174+0000 7f82be569700 1 -- 172.21.15.121:0/2847495623 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f829805a510 msgr2=0x7f829805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:27.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.174+0000 7f82be569700 1 --2- 172.21.15.121:0/2847495623 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f829805a510 0x7f829805c9d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f82b80a8650 tx=0x7f82a0009040).stop 2023-11-15T02:59:27.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.174+0000 7f82be569700 1 -- 172.21.15.121:0/2847495623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82b80c1bf0 msgr2=0x7f82b80aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:27.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.174+0000 7f82be569700 1 --2- 172.21.15.121:0/2847495623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82b80c1bf0 0x7f82b80aa0e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f82b8064f10 tx=0x7f82a8016000).stop 2023-11-15T02:59:27.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.174+0000 7f82be569700 1 -- 172.21.15.121:0/2847495623 shutdown_connections 2023-11-15T02:59:27.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.174+0000 7f82be569700 1 --2- 172.21.15.121:0/2847495623 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f829805a510 0x7f829805c9d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:27.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.174+0000 7f82be569700 1 --2- 172.21.15.121:0/2847495623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82b814af80 0x7f82b80c47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:27.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.174+0000 7f82be569700 1 --2- 172.21.15.121:0/2847495623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82b80c6640 0x7f82b81498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:27.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.174+0000 7f82be569700 1 --2- 172.21.15.121:0/2847495623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82b80c1bf0 0x7f82b80aa0e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:27.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.174+0000 7f82be569700 1 -- 172.21.15.121:0/2847495623 >> 172.21.15.121:0/2847495623 conn(0x7f82b8062190 msgr2=0x7f82b814a250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:27.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.174+0000 7f82be569700 1 -- 172.21.15.121:0/2847495623 shutdown_connections 2023-11-15T02:59:27.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.174+0000 7f82be569700 1 -- 172.21.15.121:0/2847495623 wait complete. 2023-11-15T02:59:27.204 INFO:tasks.cephfs_test_runner:That the MDS won't abort when the dentry is already known to be damaged. ... ok 2023-11-15T02:59:27.204 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_precommit (tasks.cephfs.test_damage.TestDamage) That the MDS won't write a dentry with new damage to CDentry::first 2023-11-15T02:59:27.205 INFO:tasks.cephfs_test_runner:test_dentry_first_precommit (tasks.cephfs.test_damage.TestDamage) 2023-11-15T02:59:27.205 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit' 2023-11-15T02:59:27.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.366+0000 7f3396907700 1 Processor -- start 2023-11-15T02:59:27.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.370+0000 7f3396907700 1 -- start start 2023-11-15T02:59:27.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3396907700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3390152850 0x7f33900c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:27.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3396907700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f339015be30 0x7f339015c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:27.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3396907700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33900c1d50 con 0x7f33900c3280 2023-11-15T02:59:27.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3396907700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33900c1860 con 0x7f339015be30 2023-11-15T02:59:27.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3396907700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339015cb80 con 0x7f3390152850 2023-11-15T02:59:27.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3396106700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f339015be30 0x7f339015c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:27.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3395905700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f33900c3280 0x7f3390152140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:41208/0 (socket says 172.21.15.121:41208) 2023-11-15T02:59:27.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3395905700 1 -- 172.21.15.121:0/3727637017 learned_addr learned my addr 172.21.15.121:0/3727637017 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:27.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3395104700 1 --2- 172.21.15.121:0/3727637017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3390152850 0x7f33900c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:27.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3394903700 1 -- 172.21.15.121:0/3727637017 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310621328 0 0) 0x7f33900c1d50 con 0x7f33900c3280 2023-11-15T02:59:27.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3394903700 1 -- 172.21.15.121:0/3727637017 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f337c0033f0 con 0x7f33900c3280 2023-11-15T02:59:27.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3396106700 1 -- 172.21.15.121:0/3727637017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3390152850 msgr2=0x7f33900c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:27.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3396106700 1 --2- 172.21.15.121:0/3727637017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3390152850 0x7f33900c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:27.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3396106700 1 -- 172.21.15.121:0/3727637017 >> v1:172.21.15.190:6789/0 conn(0x7f33900c3280 legacy=0x7f3390152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:27.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3396106700 1 -- 172.21.15.121:0/3727637017 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33900a9240 con 0x7f339015be30 2023-11-15T02:59:27.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3396106700 1 --2- 172.21.15.121:0/3727637017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f339015be30 0x7f339015c280 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f3388009e60 tx=0x7f3388009780).ready entity=mon.0 client_cookie=25f31ec6610764d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:27.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3394903700 1 -- 172.21.15.121:0/3727637017 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338800e070 con 0x7f339015be30 2023-11-15T02:59:27.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3394903700 1 -- 172.21.15.121:0/3727637017 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338800bab0 con 0x7f339015be30 2023-11-15T02:59:27.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3394903700 1 -- 172.21.15.121:0/3727637017 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338800be30 con 0x7f339015be30 2023-11-15T02:59:27.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3396907700 1 -- 172.21.15.121:0/3727637017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f339015be30 msgr2=0x7f339015c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:27.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.374+0000 7f3396907700 1 --2- 172.21.15.121:0/3727637017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f339015be30 0x7f339015c280 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f3388009e60 tx=0x7f3388009780).stop 2023-11-15T02:59:27.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.378+0000 7f3396907700 1 -- 172.21.15.121:0/3727637017 shutdown_connections 2023-11-15T02:59:27.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.378+0000 7f3396907700 1 --2- 172.21.15.121:0/3727637017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f339015be30 0x7f339015c280 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:27.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.378+0000 7f3396907700 1 --2- 172.21.15.121:0/3727637017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3390152850 0x7f33900c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:27.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.378+0000 7f3396907700 1 -- 172.21.15.121:0/3727637017 >> 172.21.15.121:0/3727637017 conn(0x7f3390062190 msgr2=0x7f3390062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:27.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.378+0000 7f3396907700 1 -- 172.21.15.121:0/3727637017 shutdown_connections 2023-11-15T02:59:27.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.378+0000 7f3396907700 1 -- 172.21.15.121:0/3727637017 wait complete. 2023-11-15T02:59:27.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.378+0000 7f3396907700 1 Processor -- start 2023-11-15T02:59:27.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.378+0000 7f3396907700 1 -- start start 2023-11-15T02:59:27.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3396907700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33900c3280 0x7f33901431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:27.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3396907700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3390152850 0x7f3390168320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:27.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3396907700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f339015be30 0x7f33900bc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:27.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3396907700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33901436b0 con 0x7f3390152850 2023-11-15T02:59:27.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3396907700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33900bcce0 con 0x7f33900c3280 2023-11-15T02:59:27.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3396907700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33900c44e0 con 0x7f339015be30 2023-11-15T02:59:27.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3395104700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3390152850 0x7f3390168320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:27.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3395104700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3390152850 0x7f3390168320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43280/0 (socket says 172.21.15.121:43280) 2023-11-15T02:59:27.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3395104700 1 -- 172.21.15.121:0/2601837445 learned_addr learned my addr 172.21.15.121:0/2601837445 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:27.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3395905700 1 --2- 172.21.15.121:0/2601837445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33900c3280 0x7f33901431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:27.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3396106700 1 --2- 172.21.15.121:0/2601837445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f339015be30 0x7f33900bc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:27.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3395104700 1 -- 172.21.15.121:0/2601837445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f339015be30 msgr2=0x7f33900bc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:27.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3395104700 1 --2- 172.21.15.121:0/2601837445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f339015be30 0x7f33900bc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:27.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3395104700 1 -- 172.21.15.121:0/2601837445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33900c3280 msgr2=0x7f33901431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:27.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3395104700 1 --2- 172.21.15.121:0/2601837445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33900c3280 0x7f33901431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:27.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3395104700 1 -- 172.21.15.121:0/2601837445 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3388009a20 con 0x7f3390152850 2023-11-15T02:59:27.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3396106700 1 --2- 172.21.15.121:0/2601837445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f339015be30 0x7f33900bc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:27.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3395104700 1 --2- 172.21.15.121:0/2601837445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3390152850 0x7f3390168320 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f338000e930 tx=0x7f3380009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:27.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3385ffb700 1 -- 172.21.15.121:0/2601837445 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338000c670 con 0x7f3390152850 2023-11-15T02:59:27.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3385ffb700 1 -- 172.21.15.121:0/2601837445 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338000c7d0 con 0x7f3390152850 2023-11-15T02:59:27.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3385ffb700 1 -- 172.21.15.121:0/2601837445 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338000cb50 con 0x7f3390152850 2023-11-15T02:59:27.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.382+0000 7f3396907700 1 -- 172.21.15.121:0/2601837445 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33900c47c0 con 0x7f3390152850 2023-11-15T02:59:27.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.386+0000 7f3396907700 1 -- 172.21.15.121:0/2601837445 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33900c4d80 con 0x7f3390152850 2023-11-15T02:59:27.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.386+0000 7f3396907700 1 -- 172.21.15.121:0/2601837445 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339006bcf0 con 0x7f3390152850 2023-11-15T02:59:27.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.386+0000 7f3385ffb700 1 -- 172.21.15.121:0/2601837445 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f338001b400 con 0x7f3390152850 2023-11-15T02:59:27.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.386+0000 7f3385ffb700 1 --2- 172.21.15.121:0/2601837445 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f337405a510 0x7f337405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:27.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.386+0000 7f3385ffb700 1 -- 172.21.15.121:0/2601837445 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f3380074a20 con 0x7f3390152850 2023-11-15T02:59:27.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.390+0000 7f3395905700 1 --2- 172.21.15.121:0/2601837445 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f337405a510 0x7f337405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:27.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.390+0000 7f3395905700 1 --2- 172.21.15.121:0/2601837445 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f337405a510 0x7f337405c9d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f338c007fd0 tx=0x7f338c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:27.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.394+0000 7f3385ffb700 1 -- 172.21.15.121:0/2601837445 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f33800484f0 con 0x7f3390152850 2023-11-15T02:59:27.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:27.606+0000 7f3396907700 1 -- 172.21.15.121:0/2601837445 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]} v 0) v1 -- 0x7f3390068a70 con 0x7f3390152850 2023-11-15T02:59:28.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.174+0000 7f3385ffb700 1 -- 172.21.15.121:0/2601837445 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]}]=0 v153) v1 ==== 145+0+0 (secure 0 0 0) 0x7f338004bb10 con 0x7f3390152850 2023-11-15T02:59:28.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.178+0000 7f3396907700 1 -- 172.21.15.121:0/2601837445 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f337405a510 msgr2=0x7f337405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:28.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.178+0000 7f3396907700 1 --2- 172.21.15.121:0/2601837445 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f337405a510 0x7f337405c9d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f338c007fd0 tx=0x7f338c009040).stop 2023-11-15T02:59:28.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.178+0000 7f3396907700 1 -- 172.21.15.121:0/2601837445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3390152850 msgr2=0x7f3390168320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:28.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.178+0000 7f3396907700 1 --2- 172.21.15.121:0/2601837445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3390152850 0x7f3390168320 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f338000e930 tx=0x7f3380009060).stop 2023-11-15T02:59:28.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.178+0000 7f3396907700 1 -- 172.21.15.121:0/2601837445 shutdown_connections 2023-11-15T02:59:28.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.178+0000 7f3396907700 1 --2- 172.21.15.121:0/2601837445 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f337405a510 0x7f337405c9d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:28.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.178+0000 7f3396907700 1 --2- 172.21.15.121:0/2601837445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f339015be30 0x7f33900bc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:28.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.178+0000 7f3396907700 1 --2- 172.21.15.121:0/2601837445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3390152850 0x7f3390168320 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:28.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.178+0000 7f3396907700 1 --2- 172.21.15.121:0/2601837445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33900c3280 0x7f33901431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:28.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.178+0000 7f3396907700 1 -- 172.21.15.121:0/2601837445 >> 172.21.15.121:0/2601837445 conn(0x7f3390062190 msgr2=0x7f33900ac030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:28.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.178+0000 7f3396907700 1 -- 172.21.15.121:0/2601837445 shutdown_connections 2023-11-15T02:59:28.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.178+0000 7f3396907700 1 -- 172.21.15.121:0/2601837445 wait complete. 2023-11-15T02:59:28.210 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-15T02:59:28.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.378+0000 7f2fd3cdb700 1 Processor -- start 2023-11-15T02:59:28.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.378+0000 7f2fd3cdb700 1 -- start start 2023-11-15T02:59:28.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd3cdb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2fcc0682a0 0x7f2fcc0c10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:28.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd3cdb700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2fcc153700 0x7f2fcc153b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:28.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd3cdb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fcc0c15a0 con 0x7f2fcc067ec0 2023-11-15T02:59:28.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd3cdb700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fcc0686e0 con 0x7f2fcc153700 2023-11-15T02:59:28.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd3cdb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fcc0c0c20 con 0x7f2fcc0682a0 2023-11-15T02:59:28.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd1276700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2fcc0682a0 0x7f2fcc0c10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:28.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd2278700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2fcc153700 0x7f2fcc153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:28.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd1276700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2fcc0682a0 0x7f2fcc0c10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:60976/0 (socket says 172.21.15.121:60976) 2023-11-15T02:59:28.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd1276700 1 -- 172.21.15.121:0/3326585067 learned_addr learned my addr 172.21.15.121:0/3326585067 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:28.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd1a77700 1 --1- 172.21.15.121:0/3326585067 >> v1:172.21.15.121:6789/0 conn(0x7f2fcc067ec0 0x7f2fcc062c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58672/0 (socket says 172.21.15.121:58672) 2023-11-15T02:59:28.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd2278700 1 -- 172.21.15.121:0/3326585067 >> v2:172.21.15.190:3300/0 conn(0x7f2fcc0682a0 msgr2=0x7f2fcc0c10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:28.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd2278700 1 --2- 172.21.15.121:0/3326585067 >> v2:172.21.15.190:3300/0 conn(0x7f2fcc0682a0 0x7f2fcc0c10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:28.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd2278700 1 -- 172.21.15.121:0/3326585067 >> v1:172.21.15.121:6789/0 conn(0x7f2fcc067ec0 legacy=0x7f2fcc062c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:28.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd2278700 1 -- 172.21.15.121:0/3326585067 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc158730 con 0x7f2fcc153700 2023-11-15T02:59:28.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd0a75700 1 -- 172.21.15.121:0/3326585067 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2525500239 0 0) 0x7f2fc8001110 con 0x7f2fcc067ec0 2023-11-15T02:59:28.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd2278700 1 --2- 172.21.15.121:0/3326585067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fcc153700 0x7f2fcc153b70 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f2fcc068a10 tx=0x7f2fc4009780).ready entity=mon.0 client_cookie=d89a48d30fe074a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:28.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd0a75700 1 -- 172.21.15.121:0/3326585067 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc400e070 con 0x7f2fcc153700 2023-11-15T02:59:28.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd0a75700 1 -- 172.21.15.121:0/3326585067 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc400bb60 con 0x7f2fcc153700 2023-11-15T02:59:28.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd0a75700 1 -- 172.21.15.121:0/3326585067 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4012410 con 0x7f2fcc153700 2023-11-15T02:59:28.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd3cdb700 1 -- 172.21.15.121:0/3326585067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fcc153700 msgr2=0x7f2fcc153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:28.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd3cdb700 1 --2- 172.21.15.121:0/3326585067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fcc153700 0x7f2fcc153b70 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f2fcc068a10 tx=0x7f2fc4009780).stop 2023-11-15T02:59:28.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd3cdb700 1 -- 172.21.15.121:0/3326585067 shutdown_connections 2023-11-15T02:59:28.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd3cdb700 1 --2- 172.21.15.121:0/3326585067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fcc153700 0x7f2fcc153b70 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:28.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd3cdb700 1 --2- 172.21.15.121:0/3326585067 >> v2:172.21.15.190:3300/0 conn(0x7f2fcc0682a0 0x7f2fcc0c10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:28.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.382+0000 7f2fd3cdb700 1 -- 172.21.15.121:0/3326585067 >> 172.21.15.121:0/3326585067 conn(0x7f2fcc0cad40 msgr2=0x7f2fcc0cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:28.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.386+0000 7f2fd3cdb700 1 -- 172.21.15.121:0/3326585067 shutdown_connections 2023-11-15T02:59:28.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.386+0000 7f2fd3cdb700 1 -- 172.21.15.121:0/3326585067 wait complete. 2023-11-15T02:59:28.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.386+0000 7f2fd3cdb700 1 Processor -- start 2023-11-15T02:59:28.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.386+0000 7f2fd3cdb700 1 -- start start 2023-11-15T02:59:28.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd3cdb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fcc067ec0 0x7f2fcc153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:28.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd3cdb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fcc0682a0 0x7f2fcc1571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:28.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd3cdb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fcc153700 0x7f2fcc150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:28.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd3cdb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc154d00 con 0x7f2fcc067ec0 2023-11-15T02:59:28.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd3cdb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc150d30 con 0x7f2fcc0682a0 2023-11-15T02:59:28.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd3cdb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc1540f0 con 0x7f2fcc153700 2023-11-15T02:59:28.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd1a77700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fcc067ec0 0x7f2fcc153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:28.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd1a77700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fcc067ec0 0x7f2fcc153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43302/0 (socket says 172.21.15.121:43302) 2023-11-15T02:59:28.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd1a77700 1 -- 172.21.15.121:0/2298846769 learned_addr learned my addr 172.21.15.121:0/2298846769 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:28.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd2278700 1 --2- 172.21.15.121:0/2298846769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fcc153700 0x7f2fcc150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:28.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd1276700 1 --2- 172.21.15.121:0/2298846769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fcc0682a0 0x7f2fcc1571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:28.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd1a77700 1 -- 172.21.15.121:0/2298846769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fcc153700 msgr2=0x7f2fcc150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:28.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd1a77700 1 --2- 172.21.15.121:0/2298846769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fcc153700 0x7f2fcc150840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:28.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd1a77700 1 -- 172.21.15.121:0/2298846769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fcc0682a0 msgr2=0x7f2fcc1571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:28.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd1a77700 1 --2- 172.21.15.121:0/2298846769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fcc0682a0 0x7f2fcc1571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:28.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd1a77700 1 -- 172.21.15.121:0/2298846769 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc4009a20 con 0x7f2fcc067ec0 2023-11-15T02:59:28.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd2278700 1 --2- 172.21.15.121:0/2298846769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fcc153700 0x7f2fcc150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:28.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd1a77700 1 --2- 172.21.15.121:0/2298846769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fcc067ec0 0x7f2fcc153b10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f2fc800c510 tx=0x7f2fc8007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:28.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fc1ffb700 1 -- 172.21.15.121:0/2298846769 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc80072e0 con 0x7f2fcc067ec0 2023-11-15T02:59:28.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fc1ffb700 1 -- 172.21.15.121:0/2298846769 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc8007440 con 0x7f2fcc067ec0 2023-11-15T02:59:28.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd3cdb700 1 -- 172.21.15.121:0/2298846769 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fcc1543d0 con 0x7f2fcc067ec0 2023-11-15T02:59:28.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fc1ffb700 1 -- 172.21.15.121:0/2298846769 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc80058f0 con 0x7f2fcc067ec0 2023-11-15T02:59:28.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.390+0000 7f2fd3cdb700 1 -- 172.21.15.121:0/2298846769 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fcc154990 con 0x7f2fcc067ec0 2023-11-15T02:59:28.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.394+0000 7f2fd3cdb700 1 -- 172.21.15.121:0/2298846769 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fcc047060 con 0x7f2fcc067ec0 2023-11-15T02:59:28.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.394+0000 7f2fc1ffb700 1 -- 172.21.15.121:0/2298846769 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2fc8005ce0 con 0x7f2fcc067ec0 2023-11-15T02:59:28.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.394+0000 7f2fc1ffb700 1 --2- 172.21.15.121:0/2298846769 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2fb005a510 0x7f2fb005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:28.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.394+0000 7f2fc1ffb700 1 -- 172.21.15.121:0/2298846769 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f2fc8073bb0 con 0x7f2fcc067ec0 2023-11-15T02:59:28.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.398+0000 7f2fd1276700 1 --2- 172.21.15.121:0/2298846769 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2fb005a510 0x7f2fb005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:28.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.398+0000 7f2fd1276700 1 --2- 172.21.15.121:0/2298846769 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2fb005a510 0x7f2fb005c9d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f2fcc0c4420 tx=0x7f2fbc0090f0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:28.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.402+0000 7f2fc1ffb700 1 -- 172.21.15.121:0/2298846769 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2fc8047570 con 0x7f2fcc067ec0 2023-11-15T02:59:28.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.562+0000 7f2fd3cdb700 1 -- 172.21.15.121:0/2298846769 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2fcc0c15a0 con 0x7f2fcc067ec0 2023-11-15T02:59:28.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.570+0000 7f2fc1ffb700 1 -- 172.21.15.121:0/2298846769 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2fcc0c15a0 con 0x7f2fcc067ec0 2023-11-15T02:59:28.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.574+0000 7f2fd3cdb700 1 -- 172.21.15.121:0/2298846769 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2fb005a510 msgr2=0x7f2fb005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:28.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.574+0000 7f2fd3cdb700 1 --2- 172.21.15.121:0/2298846769 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2fb005a510 0x7f2fb005c9d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f2fcc0c4420 tx=0x7f2fbc0090f0).stop 2023-11-15T02:59:28.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.574+0000 7f2fd3cdb700 1 -- 172.21.15.121:0/2298846769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fcc067ec0 msgr2=0x7f2fcc153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:28.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.574+0000 7f2fd3cdb700 1 --2- 172.21.15.121:0/2298846769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fcc067ec0 0x7f2fcc153b10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f2fc800c510 tx=0x7f2fc8007ad0).stop 2023-11-15T02:59:28.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.574+0000 7f2fd3cdb700 1 -- 172.21.15.121:0/2298846769 shutdown_connections 2023-11-15T02:59:28.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.574+0000 7f2fd3cdb700 1 --2- 172.21.15.121:0/2298846769 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2fb005a510 0x7f2fb005c9d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:28.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.574+0000 7f2fd3cdb700 1 --2- 172.21.15.121:0/2298846769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fcc153700 0x7f2fcc150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:28.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.574+0000 7f2fd3cdb700 1 --2- 172.21.15.121:0/2298846769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fcc0682a0 0x7f2fcc1571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:28.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.574+0000 7f2fd3cdb700 1 --2- 172.21.15.121:0/2298846769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fcc067ec0 0x7f2fcc153b10 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:28.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.574+0000 7f2fd3cdb700 1 -- 172.21.15.121:0/2298846769 >> 172.21.15.121:0/2298846769 conn(0x7f2fcc0cad40 msgr2=0x7f2fcc0ca190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:28.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.574+0000 7f2fd3cdb700 1 -- 172.21.15.121:0/2298846769 shutdown_connections 2023-11-15T02:59:28.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.574+0000 7f2fd3cdb700 1 -- 172.21.15.121:0/2298846769 wait complete. 2023-11-15T02:59:28.601 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-15T02:59:28.602 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T02:59:28.645 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T02:59:28.694 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-15T02:59:28.694 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-15T02:59:28.695 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:59:28.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.850+0000 7f41e8879700 1 Processor -- start 2023-11-15T02:59:28.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.850+0000 7f41e8879700 1 -- start start 2023-11-15T02:59:28.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.850+0000 7f41e8879700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41e40cafb0 0x7f41e4066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:28.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.850+0000 7f41e8879700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e40ad3a0 con 0x7f41e40cabd0 2023-11-15T02:59:28.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.850+0000 7f41e8879700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e40ad650 con 0x7f41e4067270 2023-11-15T02:59:28.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.850+0000 7f41e8879700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e40cb410 con 0x7f41e40cafb0 2023-11-15T02:59:28.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.850+0000 7f41e37fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f41e40cabd0 0x7f41e40acc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58684/0 (socket says 172.21.15.121:58684) 2023-11-15T02:59:28.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.850+0000 7f41e37fe700 1 -- 172.21.15.121:0/2014793360 learned_addr learned my addr 172.21.15.121:0/2014793360 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:28.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.850+0000 7f41e2ffd700 1 --2- 172.21.15.121:0/2014793360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41e40cafb0 0x7f41e4066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:28.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.850+0000 7f41e27fc700 1 -- 172.21.15.121:0/2014793360 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823659511 0 0) 0x7f41e40ad3a0 con 0x7f41e40cabd0 2023-11-15T02:59:28.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.850+0000 7f41e27fc700 1 -- 172.21.15.121:0/2014793360 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41cc0033f0 con 0x7f41e40cabd0 2023-11-15T02:59:28.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.850+0000 7f41e27fc700 1 -- 172.21.15.121:0/2014793360 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768279013 0 0) 0x7f41e40ad650 con 0x7f41e4067270 2023-11-15T02:59:28.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.850+0000 7f41e27fc700 1 -- 172.21.15.121:0/2014793360 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41e40ad3a0 con 0x7f41e4067270 2023-11-15T02:59:28.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.854+0000 7f41e2ffd700 1 -- 172.21.15.121:0/2014793360 >> v1:172.21.15.190:6789/0 conn(0x7f41e4067270 legacy=0x7f41e40ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:28.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.854+0000 7f41e27fc700 1 -- 172.21.15.121:0/2014793360 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2399925098 0 0) 0x7f41cc0033f0 con 0x7f41e40cabd0 2023-11-15T02:59:28.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.854+0000 7f41e2ffd700 1 -- 172.21.15.121:0/2014793360 >> v1:172.21.15.121:6789/0 conn(0x7f41e40cabd0 legacy=0x7f41e40acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:28.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.854+0000 7f41e2ffd700 1 -- 172.21.15.121:0/2014793360 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e4065fc0 con 0x7f41e40cafb0 2023-11-15T02:59:28.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.854+0000 7f41e2ffd700 1 --2- 172.21.15.121:0/2014793360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41e40cafb0 0x7f41e4066d80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f41d8009790 tx=0x7f41d8009060).ready entity=mon.2 client_cookie=aa4c8d48e66fedfa server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:28.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.854+0000 7f41e27fc700 1 -- 172.21.15.121:0/2014793360 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d4001470 con 0x7f41e40cafb0 2023-11-15T02:59:28.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.854+0000 7f41e27fc700 1 -- 172.21.15.121:0/2014793360 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d8011400 con 0x7f41e40cafb0 2023-11-15T02:59:28.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.854+0000 7f41e27fc700 1 -- 172.21.15.121:0/2014793360 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d8011780 con 0x7f41e40cafb0 2023-11-15T02:59:28.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.854+0000 7f41e8879700 1 -- 172.21.15.121:0/2014793360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41e40cafb0 msgr2=0x7f41e4066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:28.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.854+0000 7f41e8879700 1 --2- 172.21.15.121:0/2014793360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41e40cafb0 0x7f41e4066d80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f41d8009790 tx=0x7f41d8009060).stop 2023-11-15T02:59:28.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.854+0000 7f41e8879700 1 -- 172.21.15.121:0/2014793360 shutdown_connections 2023-11-15T02:59:28.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.854+0000 7f41e8879700 1 --2- 172.21.15.121:0/2014793360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41e40cafb0 0x7f41e4066d80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:28.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.854+0000 7f41e8879700 1 -- 172.21.15.121:0/2014793360 >> 172.21.15.121:0/2014793360 conn(0x7f41e4062190 msgr2=0x7f41e40625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:28.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.854+0000 7f41e8879700 1 -- 172.21.15.121:0/2014793360 shutdown_connections 2023-11-15T02:59:28.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.854+0000 7f41e8879700 1 -- 172.21.15.121:0/2014793360 wait complete. 2023-11-15T02:59:28.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.858+0000 7f41e8879700 1 Processor -- start 2023-11-15T02:59:28.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.858+0000 7f41e8879700 1 -- start start 2023-11-15T02:59:28.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.858+0000 7f41e8879700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41e4067270 0x7f41e416c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:28.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.858+0000 7f41e8879700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41e40cabd0 0x7f41e414fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:28.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.858+0000 7f41e8879700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41e40cafb0 0x7f41e414fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:28.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.858+0000 7f41e8879700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e41504e0 con 0x7f41e40cafb0 2023-11-15T02:59:28.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.858+0000 7f41e8879700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e416bde0 con 0x7f41e4067270 2023-11-15T02:59:28.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.858+0000 7f41e8879700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e41683e0 con 0x7f41e40cabd0 2023-11-15T02:59:28.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.862+0000 7f41e3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41e40cafb0 0x7f41e414fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:28.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.862+0000 7f41e3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41e40cafb0 0x7f41e414fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43318/0 (socket says 172.21.15.121:43318) 2023-11-15T02:59:28.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.862+0000 7f41e3fff700 1 -- 172.21.15.121:0/1347664171 learned_addr learned my addr 172.21.15.121:0/1347664171 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:28.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.862+0000 7f41e37fe700 1 --2- 172.21.15.121:0/1347664171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41e4067270 0x7f41e416c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:28.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.862+0000 7f41e2ffd700 1 --2- 172.21.15.121:0/1347664171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41e40cabd0 0x7f41e414fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:28.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.862+0000 7f41e3fff700 1 -- 172.21.15.121:0/1347664171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41e40cabd0 msgr2=0x7f41e414fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:28.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.862+0000 7f41e3fff700 1 --2- 172.21.15.121:0/1347664171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41e40cabd0 0x7f41e414fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:28.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.862+0000 7f41e3fff700 1 -- 172.21.15.121:0/1347664171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41e4067270 msgr2=0x7f41e416c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:28.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.862+0000 7f41e3fff700 1 --2- 172.21.15.121:0/1347664171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41e4067270 0x7f41e416c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:28.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.862+0000 7f41e3fff700 1 -- 172.21.15.121:0/1347664171 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d80092a0 con 0x7f41e40cafb0 2023-11-15T02:59:28.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.862+0000 7f41e3fff700 1 --2- 172.21.15.121:0/1347664171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41e40cafb0 0x7f41e414fff0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f41dc00c710 tx=0x7f41dc00ca20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:28.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.862+0000 7f41cb7fe700 1 -- 172.21.15.121:0/1347664171 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc00e4a0 con 0x7f41e40cafb0 2023-11-15T02:59:28.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.862+0000 7f41cb7fe700 1 -- 172.21.15.121:0/1347664171 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41dc00e600 con 0x7f41e40cafb0 2023-11-15T02:59:28.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.862+0000 7f41e8879700 1 -- 172.21.15.121:0/1347664171 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e4155760 con 0x7f41e40cafb0 2023-11-15T02:59:28.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.862+0000 7f41cb7fe700 1 -- 172.21.15.121:0/1347664171 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc00e980 con 0x7f41e40cafb0 2023-11-15T02:59:28.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.862+0000 7f41e8879700 1 -- 172.21.15.121:0/1347664171 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e4155fa0 con 0x7f41e40cafb0 2023-11-15T02:59:28.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.862+0000 7f41e8879700 1 -- 172.21.15.121:0/1347664171 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e406bcf0 con 0x7f41e40cafb0 2023-11-15T02:59:28.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.866+0000 7f41cb7fe700 1 -- 172.21.15.121:0/1347664171 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f41dc012400 con 0x7f41e40cafb0 2023-11-15T02:59:28.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.866+0000 7f41cb7fe700 1 --2- 172.21.15.121:0/1347664171 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41d005a4e0 0x7f41d005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:28.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.866+0000 7f41cb7fe700 1 -- 172.21.15.121:0/1347664171 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f41dc073c40 con 0x7f41e40cafb0 2023-11-15T02:59:28.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.866+0000 7f41e37fe700 1 --2- 172.21.15.121:0/1347664171 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41d005a4e0 0x7f41d005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:28.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.866+0000 7f41e37fe700 1 --2- 172.21.15.121:0/1347664171 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41d005a4e0 0x7f41d005c9a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f41d40011e0 tx=0x7f41d4000bc0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:28.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:28.870+0000 7f41cb7fe700 1 -- 172.21.15.121:0/1347664171 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f41dc047710 con 0x7f41e40cafb0 2023-11-15T02:59:29.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.070+0000 7f41e8879700 1 -- 172.21.15.121:0/1347664171 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41e40662a0 con 0x7f41e40cafb0 2023-11-15T02:59:29.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.070+0000 7f41cb7fe700 1 -- 172.21.15.121:0/1347664171 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f41dc04ad30 con 0x7f41e40cafb0 2023-11-15T02:59:29.077 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:29.077 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":39,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3403224816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3403224816},{"type":"v1","addr":"172.21.15.190:6835","nonce":3403224816}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2088953798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2088953798},{"type":"v1","addr":"172.21.15.190:6833","nonce":2088953798}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5511,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2190514138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2190514138},{"type":"v1","addr":"172.21.15.121:6835","nonce":2190514138}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/53608860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":53608860},{"type":"v1","addr":"172.21.15.121:6837","nonce":53608860}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[]} 2023-11-15T02:59:29.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.074+0000 7f41e8879700 1 -- 172.21.15.121:0/1347664171 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41d005a4e0 msgr2=0x7f41d005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:29.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.074+0000 7f41e8879700 1 --2- 172.21.15.121:0/1347664171 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41d005a4e0 0x7f41d005c9a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f41d40011e0 tx=0x7f41d4000bc0).stop 2023-11-15T02:59:29.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.074+0000 7f41e8879700 1 -- 172.21.15.121:0/1347664171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41e40cafb0 msgr2=0x7f41e414fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:29.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.074+0000 7f41e8879700 1 --2- 172.21.15.121:0/1347664171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41e40cafb0 0x7f41e414fff0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f41dc00c710 tx=0x7f41dc00ca20).stop 2023-11-15T02:59:29.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.074+0000 7f41e8879700 1 -- 172.21.15.121:0/1347664171 shutdown_connections 2023-11-15T02:59:29.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.074+0000 7f41e8879700 1 --2- 172.21.15.121:0/1347664171 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41d005a4e0 0x7f41d005c9a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:29.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.074+0000 7f41e8879700 1 --2- 172.21.15.121:0/1347664171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41e40cafb0 0x7f41e414fff0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:29.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.074+0000 7f41e8879700 1 --2- 172.21.15.121:0/1347664171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41e40cabd0 0x7f41e414fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:29.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.074+0000 7f41e8879700 1 --2- 172.21.15.121:0/1347664171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41e4067270 0x7f41e416c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:29.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.074+0000 7f41e8879700 1 -- 172.21.15.121:0/1347664171 >> 172.21.15.121:0/1347664171 conn(0x7f41e4062190 msgr2=0x7f41e41529a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:29.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.074+0000 7f41e8879700 1 -- 172.21.15.121:0/1347664171 shutdown_connections 2023-11-15T02:59:29.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.074+0000 7f41e8879700 1 -- 172.21.15.121:0/1347664171 wait complete. 2023-11-15T02:59:29.085 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 39 2023-11-15T02:59:29.101 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-15T02:59:29.101 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-15T02:59:29.101 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-15T02:59:29.102 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T02:59:29.102 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-15T02:59:29.102 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-15T02:59:29.103 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-15T02:59:29.103 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T02:59:29.103 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-15T02:59:29.104 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-15T02:59:29.104 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-15T02:59:29.104 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T02:59:29.104 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-15T02:59:29.105 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-15T02:59:29.105 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-15T02:59:29.105 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T02:59:29.120 INFO:tasks.ceph.mds.c:Stopped 2023-11-15T02:59:29.121 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-15T02:59:29.122 INFO:tasks.ceph.mds.a:Stopped 2023-11-15T02:59:29.122 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-15T02:59:29.128 INFO:tasks.ceph.mds.b:Stopped 2023-11-15T02:59:29.129 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-15T02:59:29.130 INFO:tasks.ceph.mds.d:Stopped 2023-11-15T02:59:29.131 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-15T02:59:29.169 INFO:tasks.ceph.mds.c:Started 2023-11-15T02:59:29.169 DEBUG:teuthology.parallel:result is None 2023-11-15T02:59:29.172 INFO:tasks.ceph.mds.a:Started 2023-11-15T02:59:29.172 DEBUG:teuthology.parallel:result is None 2023-11-15T02:59:29.178 INFO:tasks.ceph.mds.b:Started 2023-11-15T02:59:29.178 DEBUG:teuthology.parallel:result is None 2023-11-15T02:59:29.182 INFO:tasks.ceph.mds.d:Started 2023-11-15T02:59:29.183 DEBUG:teuthology.parallel:result is None 2023-11-15T02:59:29.183 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-15T02:59:29.270 INFO:tasks.ceph.mds.c.smithi121.stdout:starting mds.c at 2023-11-15T02:59:29.270 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T02:59:29.288 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-11-15T02:59:29.295 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-11-15T02:59:29.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.350+0000 7f32039db700 1 Processor -- start 2023-11-15T02:59:29.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.350+0000 7f32039db700 1 -- start start 2023-11-15T02:59:29.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f32039db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f31fc153e80 0x7f31fc066680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:29.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f32039db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31fc0c0490 0x7f31fc0c0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:29.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f32039db700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31fc0c5c50 con 0x7f31fc153aa0 2023-11-15T02:59:29.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f32039db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31fc066b70 con 0x7f31fc153e80 2023-11-15T02:59:29.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f32039db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc0c0e10 con 0x7f31fc0c0490 2023-11-15T02:59:29.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f3200f76700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f31fc153e80 0x7f31fc066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:29.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f3200f76700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f31fc153e80 0x7f31fc066680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43362/0 (socket says 172.21.15.121:43362) 2023-11-15T02:59:29.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f3200f76700 1 -- 172.21.15.121:0/1327531100 learned_addr learned my addr 172.21.15.121:0/1327531100 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:29.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f3201f78700 1 --2- 172.21.15.121:0/1327531100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31fc0c0490 0x7f31fc0c0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:29.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f3200f76700 1 -- 172.21.15.121:0/1327531100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31fc0c0490 msgr2=0x7f31fc0c0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:29.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f3200f76700 1 --2- 172.21.15.121:0/1327531100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31fc0c0490 0x7f31fc0c0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:29.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f3200f76700 1 -- 172.21.15.121:0/1327531100 >> v1:172.21.15.190:6789/0 conn(0x7f31fc153aa0 legacy=0x7f31fc0ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:29.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f3200f76700 1 -- 172.21.15.121:0/1327531100 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31fc0c5920 con 0x7f31fc153e80 2023-11-15T02:59:29.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f3201f78700 1 --2- 172.21.15.121:0/1327531100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31fc0c0490 0x7f31fc0c0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:29.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f3200f76700 1 --2- 172.21.15.121:0/1327531100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31fc153e80 0x7f31fc066680 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f31ec006fd0 tx=0x7f31ec0090f0).ready entity=mon.0 client_cookie=692ccd2ed3f3cf11 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:29.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f31f3fff700 1 -- 172.21.15.121:0/1327531100 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec00b600 con 0x7f31fc153e80 2023-11-15T02:59:29.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f31f3fff700 1 -- 172.21.15.121:0/1327531100 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec00b760 con 0x7f31fc153e80 2023-11-15T02:59:29.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f31f3fff700 1 -- 172.21.15.121:0/1327531100 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec00bae0 con 0x7f31fc153e80 2023-11-15T02:59:29.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f32039db700 1 -- 172.21.15.121:0/1327531100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31fc153e80 msgr2=0x7f31fc066680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:29.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.354+0000 7f32039db700 1 --2- 172.21.15.121:0/1327531100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31fc153e80 0x7f31fc066680 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f31ec006fd0 tx=0x7f31ec0090f0).stop 2023-11-15T02:59:29.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.358+0000 7f32039db700 1 -- 172.21.15.121:0/1327531100 shutdown_connections 2023-11-15T02:59:29.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.358+0000 7f32039db700 1 --2- 172.21.15.121:0/1327531100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31fc0c0490 0x7f31fc0c0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:29.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.358+0000 7f32039db700 1 --2- 172.21.15.121:0/1327531100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31fc153e80 0x7f31fc066680 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:29.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.358+0000 7f32039db700 1 -- 172.21.15.121:0/1327531100 >> 172.21.15.121:0/1327531100 conn(0x7f31fc067a60 msgr2=0x7f31fc067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:29.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.358+0000 7f32039db700 1 -- 172.21.15.121:0/1327531100 shutdown_connections 2023-11-15T02:59:29.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.358+0000 7f32039db700 1 -- 172.21.15.121:0/1327531100 wait complete. 2023-11-15T02:59:29.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.358+0000 7f32039db700 1 Processor -- start 2023-11-15T02:59:29.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.362+0000 7f32039db700 1 -- start start 2023-11-15T02:59:29.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.362+0000 7f32039db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31fc0c0490 0x7f31fc168e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:29.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.362+0000 7f32039db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31fc153aa0 0x7f31fc156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:29.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.362+0000 7f32039db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31fc153e80 0x7f31fc0ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:29.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.362+0000 7f32039db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc169370 con 0x7f31fc153e80 2023-11-15T02:59:29.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.362+0000 7f32039db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc0b0a70 con 0x7f31fc153aa0 2023-11-15T02:59:29.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.362+0000 7f32039db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc0afe60 con 0x7f31fc0c0490 2023-11-15T02:59:29.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.362+0000 7f3201f78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31fc153e80 0x7f31fc0ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:29.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.362+0000 7f3201f78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31fc153e80 0x7f31fc0ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43372/0 (socket says 172.21.15.121:43372) 2023-11-15T02:59:29.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.362+0000 7f3201f78700 1 -- 172.21.15.121:0/3724229704 learned_addr learned my addr 172.21.15.121:0/3724229704 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:29.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.362+0000 7f3201777700 1 --2- 172.21.15.121:0/3724229704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31fc0c0490 0x7f31fc168e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:29.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.362+0000 7f3200f76700 1 --2- 172.21.15.121:0/3724229704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31fc153aa0 0x7f31fc156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:29.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.366+0000 7f3201f78700 1 -- 172.21.15.121:0/3724229704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31fc0c0490 msgr2=0x7f31fc168e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:29.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.366+0000 7f3201f78700 1 --2- 172.21.15.121:0/3724229704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31fc0c0490 0x7f31fc168e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:29.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.366+0000 7f3201f78700 1 -- 172.21.15.121:0/3724229704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31fc153aa0 msgr2=0x7f31fc156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:29.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.366+0000 7f3201f78700 1 --2- 172.21.15.121:0/3724229704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31fc153aa0 0x7f31fc156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:29.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.366+0000 7f3201f78700 1 -- 172.21.15.121:0/3724229704 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ec009370 con 0x7f31fc153e80 2023-11-15T02:59:29.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.366+0000 7f3201777700 1 --2- 172.21.15.121:0/3724229704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31fc0c0490 0x7f31fc168e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:59:29.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.366+0000 7f3201f78700 1 --2- 172.21.15.121:0/3724229704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31fc153e80 0x7f31fc0ae0d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f31f400d180 tx=0x7f31f4009a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:29.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.366+0000 7f31f17fa700 1 -- 172.21.15.121:0/3724229704 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f400dcc0 con 0x7f31fc153e80 2023-11-15T02:59:29.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.366+0000 7f31f17fa700 1 -- 172.21.15.121:0/3724229704 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f400f040 con 0x7f31fc153e80 2023-11-15T02:59:29.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.366+0000 7f32039db700 1 -- 172.21.15.121:0/3724229704 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31fc0b0140 con 0x7f31fc153e80 2023-11-15T02:59:29.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.366+0000 7f31f17fa700 1 -- 172.21.15.121:0/3724229704 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4013600 con 0x7f31fc153e80 2023-11-15T02:59:29.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.366+0000 7f32039db700 1 -- 172.21.15.121:0/3724229704 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31fc0b0700 con 0x7f31fc153e80 2023-11-15T02:59:29.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.366+0000 7f32039db700 1 -- 172.21.15.121:0/3724229704 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31fc0b0c00 con 0x7f31fc153e80 2023-11-15T02:59:29.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.370+0000 7f31f17fa700 1 -- 172.21.15.121:0/3724229704 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f31f40139b0 con 0x7f31fc153e80 2023-11-15T02:59:29.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.370+0000 7f31f17fa700 1 --2- 172.21.15.121:0/3724229704 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f31e005a510 0x7f31e005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:29.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.370+0000 7f31f17fa700 1 -- 172.21.15.121:0/3724229704 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f31f4074c80 con 0x7f31fc153e80 2023-11-15T02:59:29.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.370+0000 7f3201777700 1 --2- 172.21.15.121:0/3724229704 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f31e005a510 0x7f31e005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:29.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.374+0000 7f3201777700 1 --2- 172.21.15.121:0/3724229704 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f31e005a510 0x7f31e005c9d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f31f8001bb0 tx=0x7f31f8009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:29.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.374+0000 7f31f17fa700 1 -- 172.21.15.121:0/3724229704 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f31f40485e0 con 0x7f31fc153e80 2023-11-15T02:59:29.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:29.554+0000 7f32039db700 1 -- 172.21.15.121:0/3724229704 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f31fc066d50 con 0x7f31fc153e80 2023-11-15T02:59:30.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.330+0000 7f31f17fa700 1 -- 172.21.15.121:0/3724229704 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v57) v1 ==== 97+0+0 (secure 0 0 0) 0x7f31f404bc00 con 0x7f31fc153e80 2023-11-15T02:59:30.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.334+0000 7f32039db700 1 -- 172.21.15.121:0/3724229704 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f31e005a510 msgr2=0x7f31e005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:30.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.334+0000 7f32039db700 1 --2- 172.21.15.121:0/3724229704 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f31e005a510 0x7f31e005c9d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f31f8001bb0 tx=0x7f31f8009040).stop 2023-11-15T02:59:30.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.334+0000 7f32039db700 1 -- 172.21.15.121:0/3724229704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31fc153e80 msgr2=0x7f31fc0ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:30.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.334+0000 7f32039db700 1 --2- 172.21.15.121:0/3724229704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31fc153e80 0x7f31fc0ae0d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f31f400d180 tx=0x7f31f4009a50).stop 2023-11-15T02:59:30.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.334+0000 7f32039db700 1 -- 172.21.15.121:0/3724229704 shutdown_connections 2023-11-15T02:59:30.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.334+0000 7f32039db700 1 --2- 172.21.15.121:0/3724229704 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f31e005a510 0x7f31e005c9d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:30.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.334+0000 7f32039db700 1 --2- 172.21.15.121:0/3724229704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31fc153e80 0x7f31fc0ae0d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:30.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.334+0000 7f32039db700 1 --2- 172.21.15.121:0/3724229704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31fc153aa0 0x7f31fc156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:30.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.334+0000 7f32039db700 1 --2- 172.21.15.121:0/3724229704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31fc0c0490 0x7f31fc168e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:30.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.334+0000 7f32039db700 1 -- 172.21.15.121:0/3724229704 >> 172.21.15.121:0/3724229704 conn(0x7f31fc067a60 msgr2=0x7f31fc062190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:30.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.334+0000 7f32039db700 1 -- 172.21.15.121:0/3724229704 shutdown_connections 2023-11-15T02:59:30.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.334+0000 7f32039db700 1 -- 172.21.15.121:0/3724229704 wait complete. 2023-11-15T02:59:30.343 INFO:teuthology.orchestra.run.smithi121.stderr: removed all blocklist entries 2023-11-15T02:59:30.364 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-15T02:59:30.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.518+0000 7f7dcbcdb700 1 Processor -- start 2023-11-15T02:59:30.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.522+0000 7f7dcbcdb700 1 -- start start 2023-11-15T02:59:30.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.522+0000 7f7dcbcdb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7dc4150ff0 0x7f7dc4151470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:30.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.522+0000 7f7dcbcdb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc4152000 con 0x7f7dc414af80 2023-11-15T02:59:30.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.522+0000 7f7dcbcdb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc40c1270 con 0x7f7dc40c6640 2023-11-15T02:59:30.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.522+0000 7f7dcbcdb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc40c1e10 con 0x7f7dc4150ff0 2023-11-15T02:59:30.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.522+0000 7f7dc9a77700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f7dc414af80 0x7f7dc40c1700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58706/0 (socket says 172.21.15.121:58706) 2023-11-15T02:59:30.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.522+0000 7f7dc9a77700 1 -- 172.21.15.121:0/3589625897 learned_addr learned my addr 172.21.15.121:0/3589625897 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:30.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc9276700 1 --2- 172.21.15.121:0/3589625897 >> v2:172.21.15.190:3300/0 conn(0x7f7dc4150ff0 0x7f7dc4151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:30.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 -- 172.21.15.121:0/3589625897 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2097506715 0 0) 0x7f7dc0001110 con 0x7f7dc414af80 2023-11-15T02:59:30.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 -- 172.21.15.121:0/3589625897 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7db00033f0 con 0x7f7dc414af80 2023-11-15T02:59:30.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 -- 172.21.15.121:0/3589625897 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942049325 0 0) 0x7f7dc40c1270 con 0x7f7dc40c6640 2023-11-15T02:59:30.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 -- 172.21.15.121:0/3589625897 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dc0001110 con 0x7f7dc40c6640 2023-11-15T02:59:30.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 -- 172.21.15.121:0/3589625897 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1637589287 0 0) 0x7f7db00033f0 con 0x7f7dc414af80 2023-11-15T02:59:30.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 -- 172.21.15.121:0/3589625897 >> v2:172.21.15.190:3300/0 conn(0x7f7dc4150ff0 msgr2=0x7f7dc4151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:30.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 --2- 172.21.15.121:0/3589625897 >> v2:172.21.15.190:3300/0 conn(0x7f7dc4150ff0 0x7f7dc4151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:30.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 -- 172.21.15.121:0/3589625897 >> v1:172.21.15.190:6789/0 conn(0x7f7dc40c6640 legacy=0x7f7dc40a79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:30.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 -- 172.21.15.121:0/3589625897 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc4151a40 con 0x7f7dc414af80 2023-11-15T02:59:30.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 -- 172.21.15.121:0/3589625897 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f7dc0001600 con 0x7f7dc414af80 2023-11-15T02:59:30.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc9276700 1 --2- 172.21.15.121:0/3589625897 >> v2:172.21.15.190:3300/0 conn(0x7f7dc4150ff0 0x7f7dc4151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:59:30.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 -- 172.21.15.121:0/3589625897 >> v1:172.21.15.121:6789/0 conn(0x7f7dc414af80 legacy=0x7f7dc40c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:30.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 --2- 172.21.15.121:0/3589625897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db0003a70 0x7f7db0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:30.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 --2- 172.21.15.121:0/3589625897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db0004690 0x7f7db0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:30.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 --2- 172.21.15.121:0/3589625897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db0005000 0x7f7db000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:30.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 -- 172.21.15.121:0/3589625897 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc40c1e10 con 0x7f7db0003a70 2023-11-15T02:59:30.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 -- 172.21.15.121:0/3589625897 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db000be80 con 0x7f7db0005000 2023-11-15T02:59:30.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 -- 172.21.15.121:0/3589625897 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db000c040 con 0x7f7db0004690 2023-11-15T02:59:30.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc9a77700 1 --2- 172.21.15.121:0/3589625897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db0003a70 0x7f7db0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:30.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dca278700 1 --2- 172.21.15.121:0/3589625897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db0005000 0x7f7db000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:30.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc9276700 1 --2- 172.21.15.121:0/3589625897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db0004690 0x7f7db0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:30.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc9a77700 1 -- 172.21.15.121:0/3589625897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db0004690 msgr2=0x7f7db0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:30.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc9a77700 1 --2- 172.21.15.121:0/3589625897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db0004690 0x7f7db0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:30.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc9a77700 1 -- 172.21.15.121:0/3589625897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db0005000 msgr2=0x7f7db000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:30.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc9a77700 1 --2- 172.21.15.121:0/3589625897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db0005000 0x7f7db000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:30.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc9a77700 1 -- 172.21.15.121:0/3589625897 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db00033f0 con 0x7f7db0003a70 2023-11-15T02:59:30.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dca278700 1 --2- 172.21.15.121:0/3589625897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db0005000 0x7f7db000b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:30.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc9276700 1 --2- 172.21.15.121:0/3589625897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db0004690 0x7f7db0004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:30.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dca278700 1 -- 172.21.15.121:0/3589625897 reap_dead start 2023-11-15T02:59:30.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc9a77700 1 --2- 172.21.15.121:0/3589625897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db0003a70 0x7f7db0003e70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f7dc0007c90 tx=0x7f7dc0007880).ready entity=mon.0 client_cookie=7b58bdea10d9360 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:30.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 -- 172.21.15.121:0/3589625897 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc000cba0 con 0x7f7db0003a70 2023-11-15T02:59:30.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 -- 172.21.15.121:0/3589625897 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc000cd00 con 0x7f7db0003a70 2023-11-15T02:59:30.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.526+0000 7f7dc8a75700 1 -- 172.21.15.121:0/3589625897 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00072e0 con 0x7f7db0003a70 2023-11-15T02:59:30.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.530+0000 7f7dcbcdb700 1 -- 172.21.15.121:0/3589625897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db0003a70 msgr2=0x7f7db0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:30.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.530+0000 7f7dcbcdb700 1 --2- 172.21.15.121:0/3589625897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db0003a70 0x7f7db0003e70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f7dc0007c90 tx=0x7f7dc0007880).stop 2023-11-15T02:59:30.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.530+0000 7f7dcbcdb700 1 -- 172.21.15.121:0/3589625897 shutdown_connections 2023-11-15T02:59:30.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.530+0000 7f7dcbcdb700 1 --2- 172.21.15.121:0/3589625897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db0003a70 0x7f7db0003e70 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:30.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.530+0000 7f7dcbcdb700 1 -- 172.21.15.121:0/3589625897 >> 172.21.15.121:0/3589625897 conn(0x7f7dc4062190 msgr2=0x7f7dc40625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:30.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.530+0000 7f7dcbcdb700 1 -- 172.21.15.121:0/3589625897 shutdown_connections 2023-11-15T02:59:30.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.530+0000 7f7dcbcdb700 1 -- 172.21.15.121:0/3589625897 wait complete. 2023-11-15T02:59:30.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.530+0000 7f7dcbcdb700 1 Processor -- start 2023-11-15T02:59:30.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.530+0000 7f7dcbcdb700 1 -- start start 2023-11-15T02:59:30.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7dcbcdb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db0003a70 0x7f7dc40c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:30.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7dcbcdb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dc40aa200 0x7f7dc40aba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:30.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7dcbcdb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc40abf60 0x7f7dc4149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:30.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7dcbcdb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc4152850 con 0x7f7dc40aa200 2023-11-15T02:59:30.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7dcbcdb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc4152240 con 0x7f7db0003a70 2023-11-15T02:59:30.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7dcbcdb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc40aaa20 con 0x7f7dc40abf60 2023-11-15T02:59:30.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7dc9276700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dc40aa200 0x7f7dc40aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:30.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7dc9276700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dc40aa200 0x7f7dc40aba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43386/0 (socket says 172.21.15.121:43386) 2023-11-15T02:59:30.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7dc9276700 1 -- 172.21.15.121:0/2391030010 learned_addr learned my addr 172.21.15.121:0/2391030010 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:30.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7dc9a77700 1 --2- 172.21.15.121:0/2391030010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db0003a70 0x7f7dc40c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:30.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7dca278700 1 --2- 172.21.15.121:0/2391030010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc40abf60 0x7f7dc4149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:30.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7dc9276700 1 -- 172.21.15.121:0/2391030010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc40abf60 msgr2=0x7f7dc4149640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:30.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7dc9276700 1 --2- 172.21.15.121:0/2391030010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc40abf60 0x7f7dc4149640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:30.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7dc9276700 1 -- 172.21.15.121:0/2391030010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db0003a70 msgr2=0x7f7dc40c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:30.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7dc9276700 1 --2- 172.21.15.121:0/2391030010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db0003a70 0x7f7dc40c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:30.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7dc9276700 1 -- 172.21.15.121:0/2391030010 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc41524e0 con 0x7f7dc40aa200 2023-11-15T02:59:30.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7dc9276700 1 --2- 172.21.15.121:0/2391030010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dc40aa200 0x7f7dc40aba70 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f7db4010920 tx=0x7f7db4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:30.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7db9ffb700 1 -- 172.21.15.121:0/2391030010 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4014620 con 0x7f7dc40aa200 2023-11-15T02:59:30.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7db9ffb700 1 -- 172.21.15.121:0/2391030010 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db4014780 con 0x7f7dc40aa200 2023-11-15T02:59:30.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7dcbcdb700 1 -- 172.21.15.121:0/2391030010 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc40aad00 con 0x7f7dc40aa200 2023-11-15T02:59:30.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7db9ffb700 1 -- 172.21.15.121:0/2391030010 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4014b00 con 0x7f7dc40aa200 2023-11-15T02:59:30.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.534+0000 7f7dcbcdb700 1 -- 172.21.15.121:0/2391030010 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc414c270 con 0x7f7dc40aa200 2023-11-15T02:59:30.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.538+0000 7f7dcbcdb700 1 -- 172.21.15.121:0/2391030010 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc406c150 con 0x7f7dc40aa200 2023-11-15T02:59:30.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.538+0000 7f7db9ffb700 1 -- 172.21.15.121:0/2391030010 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7db400e450 con 0x7f7dc40aa200 2023-11-15T02:59:30.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.538+0000 7f7db9ffb700 1 --2- 172.21.15.121:0/2391030010 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7da805a540 0x7f7da805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:30.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.538+0000 7f7db9ffb700 1 -- 172.21.15.121:0/2391030010 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f7db4074920 con 0x7f7dc40aa200 2023-11-15T02:59:30.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.538+0000 7f7dc9a77700 1 --2- 172.21.15.121:0/2391030010 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7da805a540 0x7f7da805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:30.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.538+0000 7f7dc9a77700 1 --2- 172.21.15.121:0/2391030010 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7da805a540 0x7f7da805ca00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f7dc0007790 tx=0x7f7dc0017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:30.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.546+0000 7f7db9ffb700 1 -- 172.21.15.121:0/2391030010 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7db40483f0 con 0x7f7dc40aa200 2023-11-15T02:59:30.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.766+0000 7f7dcbcdb700 1 -- 172.21.15.121:0/2391030010 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7dc4151d20 con 0x7f7dc40aa200 2023-11-15T02:59:30.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.766+0000 7f7db9ffb700 1 -- 172.21.15.121:0/2391030010 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7db4018020 con 0x7f7dc40aa200 2023-11-15T02:59:30.774 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:30.774 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T02:59:30.774 INFO:teuthology.orchestra.run.smithi121.stdout: "auth_dump": [ 2023-11-15T02:59:30.774 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:30.775 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.a", 2023-11-15T02:59:30.775 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1Rlzy3zOhAAeJ3COoWQP/Kw+P/0AwHbwA==", 2023-11-15T02:59:30.775 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:59:30.775 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:59:30.775 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T02:59:30.776 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T02:59:30.776 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:59:30.776 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.776 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:59:30.776 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:30.777 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.b", 2023-11-15T02:59:30.777 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlUglaERAAC6WUS9ftCfr+KbPpiEIyqw==", 2023-11-15T02:59:30.777 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:59:30.777 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:59:30.777 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T02:59:30.778 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T02:59:30.778 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:59:30.778 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.778 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:59:30.778 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:30.779 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.c", 2023-11-15T02:59:30.779 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RldFcvCRAATIouBNyHGip5hFrH+z4v5A==", 2023-11-15T02:59:30.779 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:59:30.779 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:59:30.779 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T02:59:30.780 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T02:59:30.780 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:59:30.780 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.780 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:59:30.780 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:30.781 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.d", 2023-11-15T02:59:30.781 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1Rlg5iZGxAADwdL2RrtU0kDeb9EmNJU6A==", 2023-11-15T02:59:30.781 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:59:30.781 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:59:30.781 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T02:59:30.781 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T02:59:30.782 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:59:30.782 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.782 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:59:30.782 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:30.782 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.0", 2023-11-15T02:59:30.783 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAzM1RldPH9CBAAUCF/a47aFbbNsWV1r+u3Tw==", 2023-11-15T02:59:30.783 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:59:30.783 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:59:30.783 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:59:30.783 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:59:30.784 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.784 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:59:30.784 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:30.784 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.1", 2023-11-15T02:59:30.784 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQA2M1RltoxwMhAAZN3bgOjSaoK3ZxE5ezZ3IA==", 2023-11-15T02:59:30.785 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:59:30.785 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:59:30.785 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:59:30.785 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:59:30.785 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.786 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:59:30.786 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:30.786 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.2", 2023-11-15T02:59:30.786 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQA6M1RlgltjHxAAx/m5gm+qs8LiEFyZL+uLzQ==", 2023-11-15T02:59:30.786 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:59:30.787 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:59:30.787 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:59:30.787 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:59:30.787 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.787 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:59:30.787 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:30.788 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.3", 2023-11-15T02:59:30.788 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQA+M1RlhtQcDRAA+g11PmSNKazpk+VoK17EEQ==", 2023-11-15T02:59:30.788 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:59:30.788 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:59:30.789 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:59:30.789 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:59:30.789 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.789 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:59:30.789 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:30.790 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.4", 2023-11-15T02:59:30.790 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBFM1RlJ7hnBBAAjskKfD63N/61T4GSPqOiRg==", 2023-11-15T02:59:30.790 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:59:30.790 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:59:30.790 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:59:30.791 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:59:30.791 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.791 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:59:30.791 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:30.791 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.5", 2023-11-15T02:59:30.792 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBIM1RlI1HPLRAAgVbiNq1tmgK6PaWP+c2JFw==", 2023-11-15T02:59:30.792 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:59:30.792 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:59:30.792 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:59:30.792 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:59:30.793 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.793 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:59:30.793 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:30.793 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.6", 2023-11-15T02:59:30.793 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBMM1RlPDshDRAAVgIqxklHSQcFb5sM6GUfyQ==", 2023-11-15T02:59:30.794 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:59:30.794 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:59:30.794 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:59:30.794 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:59:30.794 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.795 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:59:30.795 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:30.795 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.7", 2023-11-15T02:59:30.795 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBPM1RlN8IuJxAA2rN2848kaB2eOM6zDb+eJg==", 2023-11-15T02:59:30.795 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:59:30.796 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T02:59:30.796 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T02:59:30.796 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:59:30.796 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.796 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:59:30.796 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:30.797 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.0", 2023-11-15T02:59:30.797 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlfHPEIRAAJ4YV46zzBQ7v7y0kD4P2ZA==", 2023-11-15T02:59:30.797 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:59:30.797 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:59:30.797 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow r", 2023-11-15T02:59:30.798 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-15T02:59:30.798 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.798 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:59:30.798 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:30.798 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.1", 2023-11-15T02:59:30.799 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlibaULBAA+y1EyhVOt+wXSk2uRajJ3g==", 2023-11-15T02:59:30.799 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:59:30.799 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:59:30.799 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2023-11-15T02:59:30.799 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2023-11-15T02:59:30.800 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2023-11-15T02:59:30.800 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.800 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:59:30.800 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:30.800 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.2", 2023-11-15T02:59:30.800 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlHHbvJhAAX5TE6UvcpBeJPSCZkfqD8w==", 2023-11-15T02:59:30.801 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:59:30.801 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:59:30.801 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2023-11-15T02:59:30.801 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2023-11-15T02:59:30.801 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2023-11-15T02:59:30.802 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.802 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:59:30.802 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:30.802 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.3", 2023-11-15T02:59:30.802 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlQtQXMhAAGAvtnvJLSoUKuEiZDsSxww==", 2023-11-15T02:59:30.803 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:59:30.803 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T02:59:30.803 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2023-11-15T02:59:30.803 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2023-11-15T02:59:30.803 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2023-11-15T02:59:30.804 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.804 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:59:30.804 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:30.804 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.admin", 2023-11-15T02:59:30.804 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1Rl67qqEhAAI48kZ9D7T807bTVoUmwajg==", 2023-11-15T02:59:30.805 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:59:30.805 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2023-11-15T02:59:30.805 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T02:59:30.805 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T02:59:30.805 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:59:30.805 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.806 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:59:30.806 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:30.806 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mgr.x", 2023-11-15T02:59:30.806 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1RlJQFPNRAAy7E5mrn8Xh98339fWSkX9Q==", 2023-11-15T02:59:30.806 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:59:30.807 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2023-11-15T02:59:30.807 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile mgr", 2023-11-15T02:59:30.807 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:59:30.807 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.807 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:59:30.808 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:30.808 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mgr.y", 2023-11-15T02:59:30.808 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1RldQJlLhAAbc46liEP03yBNeVhZCb7IQ==", 2023-11-15T02:59:30.808 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T02:59:30.808 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2023-11-15T02:59:30.809 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile mgr", 2023-11-15T02:59:30.809 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T02:59:30.809 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.809 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:30.809 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2023-11-15T02:59:30.809 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T02:59:30.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.770+0000 7f7dcbcdb700 1 -- 172.21.15.121:0/2391030010 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7da805a540 msgr2=0x7f7da805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:30.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.770+0000 7f7dcbcdb700 1 --2- 172.21.15.121:0/2391030010 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7da805a540 0x7f7da805ca00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f7dc0007790 tx=0x7f7dc0017000).stop 2023-11-15T02:59:30.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.770+0000 7f7dcbcdb700 1 -- 172.21.15.121:0/2391030010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dc40aa200 msgr2=0x7f7dc40aba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:30.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.770+0000 7f7dcbcdb700 1 --2- 172.21.15.121:0/2391030010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dc40aa200 0x7f7dc40aba70 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f7db4010920 tx=0x7f7db4008040).stop 2023-11-15T02:59:30.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.770+0000 7f7dcbcdb700 1 -- 172.21.15.121:0/2391030010 shutdown_connections 2023-11-15T02:59:30.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.770+0000 7f7dcbcdb700 1 --2- 172.21.15.121:0/2391030010 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7da805a540 0x7f7da805ca00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:30.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.770+0000 7f7dcbcdb700 1 --2- 172.21.15.121:0/2391030010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc40abf60 0x7f7dc4149640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:30.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.770+0000 7f7dcbcdb700 1 --2- 172.21.15.121:0/2391030010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dc40aa200 0x7f7dc40aba70 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:30.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.770+0000 7f7dcbcdb700 1 --2- 172.21.15.121:0/2391030010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db0003a70 0x7f7dc40c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:30.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.770+0000 7f7dcbcdb700 1 -- 172.21.15.121:0/2391030010 >> 172.21.15.121:0/2391030010 conn(0x7f7dc4062190 msgr2=0x7f7dc40a8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:30.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.770+0000 7f7dcbcdb700 1 -- 172.21.15.121:0/2391030010 shutdown_connections 2023-11-15T02:59:30.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.770+0000 7f7dcbcdb700 1 -- 172.21.15.121:0/2391030010 wait complete. 2023-11-15T02:59:30.814 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:59:30.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.970+0000 7ff64d839700 1 Processor -- start 2023-11-15T02:59:30.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.970+0000 7ff64d839700 1 -- start start 2023-11-15T02:59:30.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.974+0000 7ff64d839700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff648151cf0 0x7ff6480bd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:30.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.974+0000 7ff64d839700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6480beb70 con 0x7ff648151910 2023-11-15T02:59:30.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.974+0000 7ff64d839700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6480bdf60 con 0x7ff6480bf000 2023-11-15T02:59:30.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.974+0000 7ff64d839700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6480be240 con 0x7ff648151cf0 2023-11-15T02:59:30.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.974+0000 7ff647fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff648151cf0 0x7ff6480bd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:30.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.974+0000 7ff64c837700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff648151910 0x7ff6480c5c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58714/0 (socket says 172.21.15.121:58714) 2023-11-15T02:59:30.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.974+0000 7ff64c837700 1 -- 172.21.15.121:0/2732390023 learned_addr learned my addr 172.21.15.121:0/2732390023 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:30.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.974+0000 7ff647fff700 1 -- 172.21.15.121:0/2732390023 >> v1:172.21.15.190:6789/0 conn(0x7ff6480bf000 legacy=0x7ff6480c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:30.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.974+0000 7ff647fff700 1 -- 172.21.15.121:0/2732390023 >> v1:172.21.15.121:6789/0 conn(0x7ff648151910 legacy=0x7ff6480c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:30.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.974+0000 7ff647fff700 1 -- 172.21.15.121:0/2732390023 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6480be520 con 0x7ff648151cf0 2023-11-15T02:59:30.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.974+0000 7ff647fff700 1 --2- 172.21.15.121:0/2732390023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff648151cf0 0x7ff6480bd8d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7ff634009760 tx=0x7ff634009060).ready entity=mon.0 client_cookie=415c1bd106dd470e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:30.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.974+0000 7ff6477fe700 1 -- 172.21.15.121:0/2732390023 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63400b580 con 0x7ff648151cf0 2023-11-15T02:59:30.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.974+0000 7ff6477fe700 1 -- 172.21.15.121:0/2732390023 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63400b6e0 con 0x7ff648151cf0 2023-11-15T02:59:30.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.974+0000 7ff6477fe700 1 -- 172.21.15.121:0/2732390023 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63400ba60 con 0x7ff648151cf0 2023-11-15T02:59:30.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.978+0000 7ff64d839700 1 -- 172.21.15.121:0/2732390023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff648151cf0 msgr2=0x7ff6480bd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:30.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.978+0000 7ff64d839700 1 --2- 172.21.15.121:0/2732390023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff648151cf0 0x7ff6480bd8d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7ff634009760 tx=0x7ff634009060).stop 2023-11-15T02:59:30.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.978+0000 7ff64d839700 1 -- 172.21.15.121:0/2732390023 shutdown_connections 2023-11-15T02:59:30.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.978+0000 7ff64d839700 1 --2- 172.21.15.121:0/2732390023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff648151cf0 0x7ff6480bd8d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:30.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.978+0000 7ff64d839700 1 -- 172.21.15.121:0/2732390023 >> 172.21.15.121:0/2732390023 conn(0x7ff6480643a0 msgr2=0x7ff6480647b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:30.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.978+0000 7ff64d839700 1 -- 172.21.15.121:0/2732390023 shutdown_connections 2023-11-15T02:59:30.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.978+0000 7ff64d839700 1 -- 172.21.15.121:0/2732390023 wait complete. 2023-11-15T02:59:30.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.982+0000 7ff64d839700 1 Processor -- start 2023-11-15T02:59:30.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.982+0000 7ff64d839700 1 -- start start 2023-11-15T02:59:30.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.982+0000 7ff64d839700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6480bf000 0x7ff648152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:30.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.982+0000 7ff64d839700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff648151910 0x7ff648143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:30.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.982+0000 7ff64d839700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff648151cf0 0x7ff6480bf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:30.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.982+0000 7ff64d839700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff648153870 con 0x7ff648151910 2023-11-15T02:59:30.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.982+0000 7ff64d839700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff648150750 con 0x7ff648151cf0 2023-11-15T02:59:30.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.982+0000 7ff64d839700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff648150a30 con 0x7ff6480bf000 2023-11-15T02:59:30.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.982+0000 7ff647fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff648151910 0x7ff648143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:30.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.982+0000 7ff647fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff648151910 0x7ff648143200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43410/0 (socket says 172.21.15.121:43410) 2023-11-15T02:59:30.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.982+0000 7ff64c837700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6480bf000 0x7ff648152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:30.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.982+0000 7ff647fff700 1 -- 172.21.15.121:0/2632479751 learned_addr learned my addr 172.21.15.121:0/2632479751 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:30.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.986+0000 7ff64d038700 1 --2- 172.21.15.121:0/2632479751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff648151cf0 0x7ff6480bf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:30.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.986+0000 7ff64c837700 1 -- 172.21.15.121:0/2632479751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff648151cf0 msgr2=0x7ff6480bf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:30.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.986+0000 7ff64c837700 1 --2- 172.21.15.121:0/2632479751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff648151cf0 0x7ff6480bf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:30.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.986+0000 7ff64c837700 1 -- 172.21.15.121:0/2632479751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff648151910 msgr2=0x7ff648143200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:30.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.986+0000 7ff64c837700 1 --2- 172.21.15.121:0/2632479751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff648151910 0x7ff648143200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:30.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.986+0000 7ff64c837700 1 -- 172.21.15.121:0/2632479751 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6340092a0 con 0x7ff6480bf000 2023-11-15T02:59:30.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.986+0000 7ff647fff700 1 --2- 172.21.15.121:0/2632479751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff648151910 0x7ff648143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:59:30.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.986+0000 7ff64c837700 1 --2- 172.21.15.121:0/2632479751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6480bf000 0x7ff648152b00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7ff63c00c3f0 tx=0x7ff63c007ce0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:30.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.986+0000 7ff644ff9700 1 -- 172.21.15.121:0/2632479751 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c0070d0 con 0x7ff6480bf000 2023-11-15T02:59:30.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.986+0000 7ff644ff9700 1 -- 172.21.15.121:0/2632479751 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c007230 con 0x7ff6480bf000 2023-11-15T02:59:30.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.986+0000 7ff64d839700 1 -- 172.21.15.121:0/2632479751 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff648152ff0 con 0x7ff6480bf000 2023-11-15T02:59:30.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.986+0000 7ff644ff9700 1 -- 172.21.15.121:0/2632479751 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c0075d0 con 0x7ff6480bf000 2023-11-15T02:59:30.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.986+0000 7ff64d839700 1 -- 172.21.15.121:0/2632479751 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff648150510 con 0x7ff6480bf000 2023-11-15T02:59:30.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.986+0000 7ff64d839700 1 -- 172.21.15.121:0/2632479751 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6481535d0 con 0x7ff6480bf000 2023-11-15T02:59:30.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.990+0000 7ff644ff9700 1 -- 172.21.15.121:0/2632479751 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff63c0059b0 con 0x7ff6480bf000 2023-11-15T02:59:30.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.990+0000 7ff644ff9700 1 --2- 172.21.15.121:0/2632479751 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff62c05a510 0x7ff62c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:30.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.990+0000 7ff644ff9700 1 -- 172.21.15.121:0/2632479751 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5269+0+0 (secure 0 0 0) 0x7ff63c073dd0 con 0x7ff6480bf000 2023-11-15T02:59:30.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.990+0000 7ff647fff700 1 --2- 172.21.15.121:0/2632479751 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff62c05a510 0x7ff62c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:30.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:30.994+0000 7ff647fff700 1 --2- 172.21.15.121:0/2632479751 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff62c05a510 0x7ff62c05c9d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7ff63400b340 tx=0x7ff634017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:31.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.002+0000 7ff644ff9700 1 -- 172.21.15.121:0/2632479751 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff63c0478c0 con 0x7ff6480bf000 2023-11-15T02:59:31.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.162+0000 7ff64d839700 1 -- 172.21.15.121:0/2632479751 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff648150c50 con 0x7ff6480bf000 2023-11-15T02:59:31.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.170+0000 7ff644ff9700 1 -- 172.21.15.121:0/2632479751 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+13760 (secure 0 0 0) 0x7ff63c04aee0 con 0x7ff6480bf000 2023-11-15T02:59:31.173 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:31.173 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":57,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:59:30.320390+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:59:31.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.170+0000 7ff64d839700 1 -- 172.21.15.121:0/2632479751 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff62c05a510 msgr2=0x7ff62c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:31.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.170+0000 7ff64d839700 1 --2- 172.21.15.121:0/2632479751 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff62c05a510 0x7ff62c05c9d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7ff63400b340 tx=0x7ff634017000).stop 2023-11-15T02:59:31.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.170+0000 7ff64d839700 1 -- 172.21.15.121:0/2632479751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6480bf000 msgr2=0x7ff648152b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:31.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.170+0000 7ff64d839700 1 --2- 172.21.15.121:0/2632479751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6480bf000 0x7ff648152b00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7ff63c00c3f0 tx=0x7ff63c007ce0).stop 2023-11-15T02:59:31.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.170+0000 7ff64d839700 1 -- 172.21.15.121:0/2632479751 shutdown_connections 2023-11-15T02:59:31.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.170+0000 7ff64d839700 1 --2- 172.21.15.121:0/2632479751 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff62c05a510 0x7ff62c05c9d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.170+0000 7ff64d839700 1 --2- 172.21.15.121:0/2632479751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff648151cf0 0x7ff6480bf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.170+0000 7ff64d839700 1 --2- 172.21.15.121:0/2632479751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff648151910 0x7ff648143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.170+0000 7ff64d839700 1 --2- 172.21.15.121:0/2632479751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6480bf000 0x7ff648152b00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.170+0000 7ff64d839700 1 -- 172.21.15.121:0/2632479751 >> 172.21.15.121:0/2632479751 conn(0x7ff6480643a0 msgr2=0x7ff6480aaa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:31.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.170+0000 7ff64d839700 1 -- 172.21.15.121:0/2632479751 shutdown_connections 2023-11-15T02:59:31.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.170+0000 7ff64d839700 1 -- 172.21.15.121:0/2632479751 wait complete. 2023-11-15T02:59:31.199 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-15T02:59:31.201 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T02:59:31.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.362+0000 7f7a820f1700 1 Processor -- start 2023-11-15T02:59:31.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.362+0000 7f7a820f1700 1 -- start start 2023-11-15T02:59:31.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.362+0000 7f7a820f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a7c0bc9b0 0x7f7a7c0670a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:31.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.362+0000 7f7a820f1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7a7c0bcd90 0x7f7a7c153d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:31.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.362+0000 7f7a820f1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a7c0bd170 con 0x7f7a7c062190 2023-11-15T02:59:31.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.362+0000 7f7a820f1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a7c154530 con 0x7f7a7c0bcd90 2023-11-15T02:59:31.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.362+0000 7f7a820f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c0c1dd0 con 0x7f7a7c0bc9b0 2023-11-15T02:59:31.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.362+0000 7f7a7affd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7a7c0bcd90 0x7f7a7c153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:31.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a7affd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7a7c0bcd90 0x7f7a7c153d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43426/0 (socket says 172.21.15.121:43426) 2023-11-15T02:59:31.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a7affd700 1 -- 172.21.15.121:0/3010347896 learned_addr learned my addr 172.21.15.121:0/3010347896 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:31.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a7bfff700 1 --1- 172.21.15.121:0/3010347896 >> v1:172.21.15.121:6789/0 conn(0x7f7a7c062190 0x7f7a7c0c2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58730/0 (socket says 172.21.15.121:58730) 2023-11-15T02:59:31.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a7b7fe700 1 --2- 172.21.15.121:0/3010347896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a7c0bc9b0 0x7f7a7c0670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:31.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a7affd700 1 -- 172.21.15.121:0/3010347896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a7c0bc9b0 msgr2=0x7f7a7c0670a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:31.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a7affd700 1 --2- 172.21.15.121:0/3010347896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a7c0bc9b0 0x7f7a7c0670a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a7affd700 1 -- 172.21.15.121:0/3010347896 >> v1:172.21.15.121:6789/0 conn(0x7f7a7c062190 legacy=0x7f7a7c0c2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:31.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a7affd700 1 -- 172.21.15.121:0/3010347896 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7c1547d0 con 0x7f7a7c0bcd90 2023-11-15T02:59:31.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a7b7fe700 1 --2- 172.21.15.121:0/3010347896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a7c0bc9b0 0x7f7a7c0670a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:31.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a7affd700 1 --2- 172.21.15.121:0/3010347896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a7c0bcd90 0x7f7a7c153d40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f7a70006fd0 tx=0x7f7a700090f0).ready entity=mon.0 client_cookie=d764d01259a10a47 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:31.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a7a7fc700 1 -- 172.21.15.121:0/3010347896 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7000b600 con 0x7f7a7c0bcd90 2023-11-15T02:59:31.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a7a7fc700 1 -- 172.21.15.121:0/3010347896 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7000b760 con 0x7f7a7c0bcd90 2023-11-15T02:59:31.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a7a7fc700 1 -- 172.21.15.121:0/3010347896 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7000bb00 con 0x7f7a7c0bcd90 2023-11-15T02:59:31.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a820f1700 1 -- 172.21.15.121:0/3010347896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a7c0bcd90 msgr2=0x7f7a7c153d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:31.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a820f1700 1 --2- 172.21.15.121:0/3010347896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a7c0bcd90 0x7f7a7c153d40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f7a70006fd0 tx=0x7f7a700090f0).stop 2023-11-15T02:59:31.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a820f1700 1 -- 172.21.15.121:0/3010347896 shutdown_connections 2023-11-15T02:59:31.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a820f1700 1 --2- 172.21.15.121:0/3010347896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a7c0bcd90 0x7f7a7c153d40 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a820f1700 1 --2- 172.21.15.121:0/3010347896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a7c0bc9b0 0x7f7a7c0670a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a820f1700 1 -- 172.21.15.121:0/3010347896 >> 172.21.15.121:0/3010347896 conn(0x7f7a7c0591c0 msgr2=0x7f7a7c0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:31.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a820f1700 1 -- 172.21.15.121:0/3010347896 shutdown_connections 2023-11-15T02:59:31.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.366+0000 7f7a820f1700 1 -- 172.21.15.121:0/3010347896 wait complete. 2023-11-15T02:59:31.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.370+0000 7f7a820f1700 1 Processor -- start 2023-11-15T02:59:31.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.370+0000 7f7a820f1700 1 -- start start 2023-11-15T02:59:31.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.370+0000 7f7a820f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a7c062190 0x7f7a7c0be800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:31.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.370+0000 7f7a820f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a7c0bc9b0 0x7f7a7c0c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:31.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.370+0000 7f7a820f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a7c0bcd90 0x7f7a7c145100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:31.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.370+0000 7f7a820f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c157af0 con 0x7f7a7c0bcd90 2023-11-15T02:59:31.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.370+0000 7f7a820f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c0becf0 con 0x7f7a7c0bc9b0 2023-11-15T02:59:31.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.370+0000 7f7a820f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c0bef70 con 0x7f7a7c062190 2023-11-15T02:59:31.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.370+0000 7f7a7bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a7c0bcd90 0x7f7a7c145100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:31.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.370+0000 7f7a7bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a7c0bcd90 0x7f7a7c145100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43430/0 (socket says 172.21.15.121:43430) 2023-11-15T02:59:31.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.370+0000 7f7a7bfff700 1 -- 172.21.15.121:0/1260536661 learned_addr learned my addr 172.21.15.121:0/1260536661 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:31.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.370+0000 7f7a7affd700 1 --2- 172.21.15.121:0/1260536661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a7c0bc9b0 0x7f7a7c0c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:31.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.370+0000 7f7a7b7fe700 1 --2- 172.21.15.121:0/1260536661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a7c062190 0x7f7a7c0be800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:31.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.370+0000 7f7a7bfff700 1 -- 172.21.15.121:0/1260536661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a7c062190 msgr2=0x7f7a7c0be800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:31.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.370+0000 7f7a7bfff700 1 --2- 172.21.15.121:0/1260536661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a7c062190 0x7f7a7c0be800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.370+0000 7f7a7bfff700 1 -- 172.21.15.121:0/1260536661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a7c0bc9b0 msgr2=0x7f7a7c0c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:31.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.370+0000 7f7a7bfff700 1 --2- 172.21.15.121:0/1260536661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a7c0bc9b0 0x7f7a7c0c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.370+0000 7f7a7bfff700 1 -- 172.21.15.121:0/1260536661 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a70009370 con 0x7f7a7c0bcd90 2023-11-15T02:59:31.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.370+0000 7f7a7bfff700 1 --2- 172.21.15.121:0/1260536661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a7c0bcd90 0x7f7a7c145100 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f7a7400c790 tx=0x7f7a740015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:31.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.374+0000 7f7a637fe700 1 -- 172.21.15.121:0/1260536661 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74005c70 con 0x7f7a7c0bcd90 2023-11-15T02:59:31.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.374+0000 7f7a637fe700 1 -- 172.21.15.121:0/1260536661 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a74005dd0 con 0x7f7a7c0bcd90 2023-11-15T02:59:31.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.374+0000 7f7a637fe700 1 -- 172.21.15.121:0/1260536661 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74007320 con 0x7f7a7c0bcd90 2023-11-15T02:59:31.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.374+0000 7f7a820f1700 1 -- 172.21.15.121:0/1260536661 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7c0bf250 con 0x7f7a7c0bcd90 2023-11-15T02:59:31.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.374+0000 7f7a820f1700 1 -- 172.21.15.121:0/1260536661 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a7c159380 con 0x7f7a7c0bcd90 2023-11-15T02:59:31.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.374+0000 7f7a820f1700 1 -- 172.21.15.121:0/1260536661 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a7c159510 con 0x7f7a7c0bcd90 2023-11-15T02:59:31.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.374+0000 7f7a637fe700 1 -- 172.21.15.121:0/1260536661 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7a740076d0 con 0x7f7a7c0bcd90 2023-11-15T02:59:31.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.374+0000 7f7a637fe700 1 --2- 172.21.15.121:0/1260536661 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7a6805a510 0x7f7a6805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:31.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.374+0000 7f7a637fe700 1 -- 172.21.15.121:0/1260536661 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f7a74074550 con 0x7f7a7c0bcd90 2023-11-15T02:59:31.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.374+0000 7f7a7b7fe700 1 --2- 172.21.15.121:0/1260536661 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7a6805a510 0x7f7a6805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:31.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.378+0000 7f7a7b7fe700 1 --2- 172.21.15.121:0/1260536661 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7a6805a510 0x7f7a6805c9d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f7a6c006fd0 tx=0x7f7a6c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:31.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.382+0000 7f7a637fe700 1 -- 172.21.15.121:0/1260536661 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7a74047fc0 con 0x7f7a7c0bcd90 2023-11-15T02:59:31.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.538+0000 7f7a820f1700 1 -- 172.21.15.121:0/1260536661 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7a7c154ab0 con 0x7f7a7c0bcd90 2023-11-15T02:59:31.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.538+0000 7f7a637fe700 1 -- 172.21.15.121:0/1260536661 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v57) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7a7404b5e0 con 0x7f7a7c0bcd90 2023-11-15T02:59:31.546 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T02:59:31.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.542+0000 7f7a820f1700 1 -- 172.21.15.121:0/1260536661 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7a6805a510 msgr2=0x7f7a6805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:31.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.542+0000 7f7a820f1700 1 --2- 172.21.15.121:0/1260536661 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7a6805a510 0x7f7a6805c9d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f7a6c006fd0 tx=0x7f7a6c008040).stop 2023-11-15T02:59:31.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.542+0000 7f7a820f1700 1 -- 172.21.15.121:0/1260536661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a7c0bcd90 msgr2=0x7f7a7c145100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:31.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.542+0000 7f7a820f1700 1 --2- 172.21.15.121:0/1260536661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a7c0bcd90 0x7f7a7c145100 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f7a7400c790 tx=0x7f7a740015e0).stop 2023-11-15T02:59:31.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.542+0000 7f7a820f1700 1 -- 172.21.15.121:0/1260536661 shutdown_connections 2023-11-15T02:59:31.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.542+0000 7f7a820f1700 1 --2- 172.21.15.121:0/1260536661 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7a6805a510 0x7f7a6805c9d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.542+0000 7f7a820f1700 1 --2- 172.21.15.121:0/1260536661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a7c0bcd90 0x7f7a7c145100 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.542+0000 7f7a820f1700 1 --2- 172.21.15.121:0/1260536661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a7c0bc9b0 0x7f7a7c0c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.542+0000 7f7a820f1700 1 --2- 172.21.15.121:0/1260536661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a7c062190 0x7f7a7c0be800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.542+0000 7f7a820f1700 1 -- 172.21.15.121:0/1260536661 >> 172.21.15.121:0/1260536661 conn(0x7f7a7c0591c0 msgr2=0x7f7a7c0a9760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:31.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.542+0000 7f7a820f1700 1 -- 172.21.15.121:0/1260536661 shutdown_connections 2023-11-15T02:59:31.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.542+0000 7f7a820f1700 1 -- 172.21.15.121:0/1260536661 wait complete. 2023-11-15T02:59:31.569 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T02:59:31.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.730+0000 7fb815a5b700 1 Processor -- start 2023-11-15T02:59:31.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.734+0000 7fb815a5b700 1 -- start start 2023-11-15T02:59:31.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.734+0000 7fb815a5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb810153e60 0x7fb810158420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:31.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.734+0000 7fb815a5b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb810154240 0x7fb8100657f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:31.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.734+0000 7fb815a5b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb810065ce0 0x7fb8100c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:31.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.734+0000 7fb815a5b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8101553c0 con 0x7fb810154240 2023-11-15T02:59:31.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.734+0000 7fb815a5b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb810158910 con 0x7fb810065ce0 2023-11-15T02:59:31.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.734+0000 7fb815a5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8100c7280 con 0x7fb810153e60 2023-11-15T02:59:31.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.734+0000 7fb80e7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb810154240 0x7fb8100657f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:31.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.734+0000 7fb80e7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb810154240 0x7fb8100657f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43438/0 (socket says 172.21.15.121:43438) 2023-11-15T02:59:31.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.734+0000 7fb80e7fc700 1 -- 172.21.15.121:0/4163819922 learned_addr learned my addr 172.21.15.121:0/4163819922 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:31.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.734+0000 7fb80f7fe700 1 --2- 172.21.15.121:0/4163819922 >> v2:172.21.15.190:3300/0 conn(0x7fb810065ce0 0x7fb8100c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:31.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.734+0000 7fb80effd700 1 --2- 172.21.15.121:0/4163819922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb810153e60 0x7fb810158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:31.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.734+0000 7fb80e7fc700 1 -- 172.21.15.121:0/4163819922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb810153e60 msgr2=0x7fb810158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:31.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.734+0000 7fb80e7fc700 1 --2- 172.21.15.121:0/4163819922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb810153e60 0x7fb810158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.734+0000 7fb80e7fc700 1 -- 172.21.15.121:0/4163819922 >> v2:172.21.15.190:3300/0 conn(0x7fb810065ce0 msgr2=0x7fb8100c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:31.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.734+0000 7fb80e7fc700 1 --2- 172.21.15.121:0/4163819922 >> v2:172.21.15.190:3300/0 conn(0x7fb810065ce0 0x7fb8100c5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.734+0000 7fb80e7fc700 1 -- 172.21.15.121:0/4163819922 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb810154b20 con 0x7fb810154240 2023-11-15T02:59:31.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.738+0000 7fb80e7fc700 1 --2- 172.21.15.121:0/4163819922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb810154240 0x7fb8100657f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7fb810064360 tx=0x7fb8040090f0).ready entity=mon.0 client_cookie=1b7efa0a6ca80464 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:31.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.738+0000 7fb80dffb700 1 -- 172.21.15.121:0/4163819922 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80400b520 con 0x7fb810154240 2023-11-15T02:59:31.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.738+0000 7fb80dffb700 1 -- 172.21.15.121:0/4163819922 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80400b680 con 0x7fb810154240 2023-11-15T02:59:31.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.738+0000 7fb80dffb700 1 -- 172.21.15.121:0/4163819922 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80400ba00 con 0x7fb810154240 2023-11-15T02:59:31.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.738+0000 7fb815a5b700 1 -- 172.21.15.121:0/4163819922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb810154240 msgr2=0x7fb8100657f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:31.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.738+0000 7fb815a5b700 1 --2- 172.21.15.121:0/4163819922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb810154240 0x7fb8100657f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7fb810064360 tx=0x7fb8040090f0).stop 2023-11-15T02:59:31.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.738+0000 7fb815a5b700 1 -- 172.21.15.121:0/4163819922 shutdown_connections 2023-11-15T02:59:31.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.738+0000 7fb815a5b700 1 --2- 172.21.15.121:0/4163819922 >> v2:172.21.15.190:3300/0 conn(0x7fb810065ce0 0x7fb8100c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.738+0000 7fb815a5b700 1 --2- 172.21.15.121:0/4163819922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb810154240 0x7fb8100657f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.738+0000 7fb815a5b700 1 --2- 172.21.15.121:0/4163819922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb810153e60 0x7fb810158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.738+0000 7fb815a5b700 1 -- 172.21.15.121:0/4163819922 >> 172.21.15.121:0/4163819922 conn(0x7fb810067a60 msgr2=0x7fb810067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:31.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.738+0000 7fb815a5b700 1 -- 172.21.15.121:0/4163819922 shutdown_connections 2023-11-15T02:59:31.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.738+0000 7fb815a5b700 1 -- 172.21.15.121:0/4163819922 wait complete. 2023-11-15T02:59:31.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb815a5b700 1 Processor -- start 2023-11-15T02:59:31.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb815a5b700 1 -- start start 2023-11-15T02:59:31.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb815a5b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb810065ce0 0x7fb8100ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:31.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb815a5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb810153e60 0x7fb8100acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:31.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb815a5b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb810154240 0x7fb810167a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:31.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb815a5b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8100abe80 con 0x7fb810065ce0 2023-11-15T02:59:31.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb815a5b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8100ac160 con 0x7fb810154240 2023-11-15T02:59:31.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb815a5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8100ac440 con 0x7fb810153e60 2023-11-15T02:59:31.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb80effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb810065ce0 0x7fb8100ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:31.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb80effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb810065ce0 0x7fb8100ab870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43440/0 (socket says 172.21.15.121:43440) 2023-11-15T02:59:31.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb80effd700 1 -- 172.21.15.121:0/4153631453 learned_addr learned my addr 172.21.15.121:0/4153631453 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:31.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb80e7fc700 1 --2- 172.21.15.121:0/4153631453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb810153e60 0x7fb8100acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:31.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb80f7fe700 1 --2- 172.21.15.121:0/4153631453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb810154240 0x7fb810167a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:31.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb80effd700 1 -- 172.21.15.121:0/4153631453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb810153e60 msgr2=0x7fb8100acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:31.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb80effd700 1 --2- 172.21.15.121:0/4153631453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb810153e60 0x7fb8100acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb80effd700 1 -- 172.21.15.121:0/4153631453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb810154240 msgr2=0x7fb810167a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:31.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb80effd700 1 --2- 172.21.15.121:0/4153631453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb810154240 0x7fb810167a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb80effd700 1 -- 172.21.15.121:0/4153631453 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb804009370 con 0x7fb810065ce0 2023-11-15T02:59:31.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb80f7fe700 1 --2- 172.21.15.121:0/4153631453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb810154240 0x7fb810167a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:31.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb80effd700 1 --2- 172.21.15.121:0/4153631453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb810065ce0 0x7fb8100ab870 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fb80000d890 tx=0x7fb800008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:31.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb7f6ffd700 1 -- 172.21.15.121:0/4153631453 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8000125f0 con 0x7fb810065ce0 2023-11-15T02:59:31.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb7f6ffd700 1 -- 172.21.15.121:0/4153631453 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb800012750 con 0x7fb810065ce0 2023-11-15T02:59:31.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb815a5b700 1 -- 172.21.15.121:0/4153631453 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8100ac720 con 0x7fb810065ce0 2023-11-15T02:59:31.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb7f6ffd700 1 -- 172.21.15.121:0/4153631453 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb800012ad0 con 0x7fb810065ce0 2023-11-15T02:59:31.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.742+0000 7fb815a5b700 1 -- 172.21.15.121:0/4153631453 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8100ad410 con 0x7fb810065ce0 2023-11-15T02:59:31.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.746+0000 7fb815a5b700 1 -- 172.21.15.121:0/4153631453 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81006bcf0 con 0x7fb810065ce0 2023-11-15T02:59:31.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.746+0000 7fb7f6ffd700 1 -- 172.21.15.121:0/4153631453 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb800013480 con 0x7fb810065ce0 2023-11-15T02:59:31.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.746+0000 7fb7f6ffd700 1 --2- 172.21.15.121:0/4153631453 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb7fc05a510 0x7fb7fc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:31.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.746+0000 7fb7f6ffd700 1 -- 172.21.15.121:0/4153631453 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fb8000742e0 con 0x7fb810065ce0 2023-11-15T02:59:31.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.746+0000 7fb80e7fc700 1 --2- 172.21.15.121:0/4153631453 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb7fc05a510 0x7fb7fc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:31.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.750+0000 7fb80e7fc700 1 --2- 172.21.15.121:0/4153631453 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb7fc05a510 0x7fb7fc05c9d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7fb810064360 tx=0x7fb804017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:31.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.754+0000 7fb7f6ffd700 1 -- 172.21.15.121:0/4153631453 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb800047cf0 con 0x7fb810065ce0 2023-11-15T02:59:31.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.914+0000 7fb815a5b700 1 -- 172.21.15.121:0/4153631453 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb810154e00 con 0x7fb810065ce0 2023-11-15T02:59:31.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.914+0000 7fb7f6ffd700 1 -- 172.21.15.121:0/4153631453 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v57) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb80004b310 con 0x7fb810065ce0 2023-11-15T02:59:31.922 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T02:59:31.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.918+0000 7fb815a5b700 1 -- 172.21.15.121:0/4153631453 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb7fc05a510 msgr2=0x7fb7fc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:31.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.918+0000 7fb815a5b700 1 --2- 172.21.15.121:0/4153631453 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb7fc05a510 0x7fb7fc05c9d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7fb810064360 tx=0x7fb804017000).stop 2023-11-15T02:59:31.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.918+0000 7fb815a5b700 1 -- 172.21.15.121:0/4153631453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb810065ce0 msgr2=0x7fb8100ab870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:31.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.918+0000 7fb815a5b700 1 --2- 172.21.15.121:0/4153631453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb810065ce0 0x7fb8100ab870 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fb80000d890 tx=0x7fb800008040).stop 2023-11-15T02:59:31.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.918+0000 7fb815a5b700 1 -- 172.21.15.121:0/4153631453 shutdown_connections 2023-11-15T02:59:31.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.918+0000 7fb815a5b700 1 --2- 172.21.15.121:0/4153631453 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb7fc05a510 0x7fb7fc05c9d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.918+0000 7fb815a5b700 1 --2- 172.21.15.121:0/4153631453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb810154240 0x7fb810167a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.918+0000 7fb815a5b700 1 --2- 172.21.15.121:0/4153631453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb810153e60 0x7fb8100acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.918+0000 7fb815a5b700 1 --2- 172.21.15.121:0/4153631453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb810065ce0 0x7fb8100ab870 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:31.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.918+0000 7fb815a5b700 1 -- 172.21.15.121:0/4153631453 >> 172.21.15.121:0/4153631453 conn(0x7fb810067a60 msgr2=0x7fb8100c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:31.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.918+0000 7fb815a5b700 1 -- 172.21.15.121:0/4153631453 shutdown_connections 2023-11-15T02:59:31.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:31.918+0000 7fb815a5b700 1 -- 172.21.15.121:0/4153631453 wait complete. 2023-11-15T02:59:31.946 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-15T02:59:32.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.090+0000 7f25a9569700 1 Processor -- start 2023-11-15T02:59:32.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.090+0000 7f25a9569700 1 -- start start 2023-11-15T02:59:32.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a9569700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25a40c4e30 0x7f25a415c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:32.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a9569700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a4068430 con 0x7f25a4151d00 2023-11-15T02:59:32.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a9569700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a40a5b10 con 0x7f25a4151920 2023-11-15T02:59:32.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a9569700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a4064b50 con 0x7f25a40c4e30 2023-11-15T02:59:32.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f259a7fc700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f25a4151d00 0x7f25a40c4720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58746/0 (socket says 172.21.15.121:58746) 2023-11-15T02:59:32.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f259a7fc700 1 -- 172.21.15.121:0/2589272290 learned_addr learned my addr 172.21.15.121:0/2589272290 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:32.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a27fc700 1 -- 172.21.15.121:0/2589272290 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712377459 0 0) 0x7f25a4068430 con 0x7f25a4151d00 2023-11-15T02:59:32.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a3fff700 1 --2- 172.21.15.121:0/2589272290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25a40c4e30 0x7f25a415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:32.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a27fc700 1 -- 172.21.15.121:0/2589272290 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25900033f0 con 0x7f25a4151d00 2023-11-15T02:59:32.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a27fc700 1 -- 172.21.15.121:0/2589272290 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1913778193 0 0) 0x7f25900033f0 con 0x7f25a4151d00 2023-11-15T02:59:32.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a27fc700 1 -- 172.21.15.121:0/2589272290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25a40c4e30 msgr2=0x7f25a415c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:32.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a27fc700 1 --2- 172.21.15.121:0/2589272290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25a40c4e30 0x7f25a415c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:32.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a27fc700 1 -- 172.21.15.121:0/2589272290 >> v1:172.21.15.190:6789/0 conn(0x7f25a4151920 legacy=0x7f25a4063fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:32.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a27fc700 1 -- 172.21.15.121:0/2589272290 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a40bec30 con 0x7f25a4151d00 2023-11-15T02:59:32.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a27fc700 1 -- 172.21.15.121:0/2589272290 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f258c001470 con 0x7f25a4151d00 2023-11-15T02:59:32.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a27fc700 1 -- 172.21.15.121:0/2589272290 >> v1:172.21.15.121:6789/0 conn(0x7f25a4151d00 legacy=0x7f25a40c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:32.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a27fc700 1 --2- 172.21.15.121:0/2589272290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2590003910 0x7f2590003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:32.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a27fc700 1 --2- 172.21.15.121:0/2589272290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2590004690 0x7f2590004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:32.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a27fc700 1 --2- 172.21.15.121:0/2589272290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2590005000 0x7f259000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:32.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a27fc700 1 -- 172.21.15.121:0/2589272290 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a4064b50 con 0x7f2590004690 2023-11-15T02:59:32.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a27fc700 1 -- 172.21.15.121:0/2589272290 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259000bf30 con 0x7f2590005000 2023-11-15T02:59:32.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a27fc700 1 -- 172.21.15.121:0/2589272290 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259000c0f0 con 0x7f2590003910 2023-11-15T02:59:32.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f259a7fc700 1 --2- 172.21.15.121:0/2589272290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2590004690 0x7f2590004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:32.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a2ffd700 1 --2- 172.21.15.121:0/2589272290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2590003910 0x7f2590003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:32.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f259a7fc700 1 -- 172.21.15.121:0/2589272290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2590003910 msgr2=0x7f2590003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:32.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a3fff700 1 --2- 172.21.15.121:0/2589272290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2590005000 0x7f259000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:32.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f259a7fc700 1 --2- 172.21.15.121:0/2589272290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2590003910 0x7f2590003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:32.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f259a7fc700 1 -- 172.21.15.121:0/2589272290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2590005000 msgr2=0x7f259000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:32.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f259a7fc700 1 --2- 172.21.15.121:0/2589272290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2590005000 0x7f259000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:32.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f259a7fc700 1 -- 172.21.15.121:0/2589272290 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25900033f0 con 0x7f2590004690 2023-11-15T02:59:32.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a3fff700 1 --2- 172.21.15.121:0/2589272290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2590005000 0x7f259000b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:32.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a2ffd700 1 --2- 172.21.15.121:0/2589272290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2590003910 0x7f2590003d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:32.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f259a7fc700 1 --2- 172.21.15.121:0/2589272290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2590004690 0x7f2590004b10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f258c00c190 tx=0x7f258c001e50).ready entity=mon.0 client_cookie=23afda8ccdabba04 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:32.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.094+0000 7f25a27fc700 1 -- 172.21.15.121:0/2589272290 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c007a20 con 0x7f2590004690 2023-11-15T02:59:32.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.098+0000 7f25a27fc700 1 -- 172.21.15.121:0/2589272290 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c00cac0 con 0x7f2590004690 2023-11-15T02:59:32.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.098+0000 7f25a27fc700 1 -- 172.21.15.121:0/2589272290 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c00ce40 con 0x7f2590004690 2023-11-15T02:59:32.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.098+0000 7f25a9569700 1 -- 172.21.15.121:0/2589272290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2590004690 msgr2=0x7f2590004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:32.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.098+0000 7f25a9569700 1 --2- 172.21.15.121:0/2589272290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2590004690 0x7f2590004b10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f258c00c190 tx=0x7f258c001e50).stop 2023-11-15T02:59:32.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.098+0000 7f25a3fff700 1 -- 172.21.15.121:0/2589272290 reap_dead start 2023-11-15T02:59:32.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.098+0000 7f25a9569700 1 -- 172.21.15.121:0/2589272290 shutdown_connections 2023-11-15T02:59:32.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.098+0000 7f25a9569700 1 -- 172.21.15.121:0/2589272290 >> 172.21.15.121:0/2589272290 conn(0x7f25a4069ac0 msgr2=0x7f25a4090a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:32.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.098+0000 7f25a9569700 1 -- 172.21.15.121:0/2589272290 shutdown_connections 2023-11-15T02:59:32.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.098+0000 7f25a9569700 1 -- 172.21.15.121:0/2589272290 wait complete. 2023-11-15T02:59:32.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.098+0000 7f25a9569700 1 Processor -- start 2023-11-15T02:59:32.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.098+0000 7f25a9569700 1 -- start start 2023-11-15T02:59:32.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f25a9569700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2590004690 0x7f25a40aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:32.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f25a9569700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25a414a880 0x7f25a414a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:32.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f25a9569700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25a4148360 0x7f25a41487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:32.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f25a9569700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a4149bb0 con 0x7f25a414a880 2023-11-15T02:59:32.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f25a9569700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a41492c0 con 0x7f25a4148360 2023-11-15T02:59:32.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f25a9569700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a4149560 con 0x7f2590004690 2023-11-15T02:59:32.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f259a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25a414a880 0x7f25a414a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:32.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f259a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25a414a880 0x7f25a414a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43454/0 (socket says 172.21.15.121:43454) 2023-11-15T02:59:32.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f259a7fc700 1 -- 172.21.15.121:0/2963674096 learned_addr learned my addr 172.21.15.121:0/2963674096 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:32.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f25a2ffd700 1 --2- 172.21.15.121:0/2963674096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2590004690 0x7f25a40aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:32.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f259a7fc700 1 -- 172.21.15.121:0/2963674096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2590004690 msgr2=0x7f25a40aa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:32.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f259a7fc700 1 --2- 172.21.15.121:0/2963674096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2590004690 0x7f25a40aa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:32.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f25a3fff700 1 --2- 172.21.15.121:0/2963674096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25a4148360 0x7f25a41487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:32.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f259a7fc700 1 -- 172.21.15.121:0/2963674096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25a4148360 msgr2=0x7f25a41487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:32.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f259a7fc700 1 --2- 172.21.15.121:0/2963674096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25a4148360 0x7f25a41487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:32.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f259a7fc700 1 -- 172.21.15.121:0/2963674096 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a4149840 con 0x7f25a414a880 2023-11-15T02:59:32.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f25a3fff700 1 --2- 172.21.15.121:0/2963674096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25a4148360 0x7f25a41487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:32.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f259a7fc700 1 --2- 172.21.15.121:0/2963674096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25a414a880 0x7f25a414a040 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f259000bf00 tx=0x7f258c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:32.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f259b7fe700 1 -- 172.21.15.121:0/2963674096 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c0046b0 con 0x7f25a414a880 2023-11-15T02:59:32.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f259b7fe700 1 -- 172.21.15.121:0/2963674096 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c004810 con 0x7f25a414a880 2023-11-15T02:59:32.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f25a9569700 1 -- 172.21.15.121:0/2963674096 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a414ad00 con 0x7f25a414a880 2023-11-15T02:59:32.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f259b7fe700 1 -- 172.21.15.121:0/2963674096 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c004bb0 con 0x7f25a414a880 2023-11-15T02:59:32.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.102+0000 7f25a9569700 1 -- 172.21.15.121:0/2963674096 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a414a530 con 0x7f25a414a880 2023-11-15T02:59:32.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.106+0000 7f25a9569700 1 -- 172.21.15.121:0/2963674096 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a4047060 con 0x7f25a414a880 2023-11-15T02:59:32.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.106+0000 7f259b7fe700 1 -- 172.21.15.121:0/2963674096 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f258c020480 con 0x7f25a414a880 2023-11-15T02:59:32.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.110+0000 7f259b7fe700 1 --2- 172.21.15.121:0/2963674096 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f258405a510 0x7f258405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:32.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.110+0000 7f259b7fe700 1 -- 172.21.15.121:0/2963674096 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f258c075510 con 0x7f25a414a880 2023-11-15T02:59:32.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.110+0000 7f25a2ffd700 1 --2- 172.21.15.121:0/2963674096 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f258405a510 0x7f258405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:32.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.110+0000 7f25a2ffd700 1 --2- 172.21.15.121:0/2963674096 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f258405a510 0x7f258405c9d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f2594007fd0 tx=0x7f2594009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:32.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.114+0000 7f259b7fe700 1 -- 172.21.15.121:0/2963674096 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f258c048f20 con 0x7f25a414a880 2023-11-15T02:59:32.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.278+0000 7f25a9569700 1 -- 172.21.15.121:0/2963674096 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f25a4067b00 con 0x7f25a414a880 2023-11-15T02:59:32.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.282+0000 7f259b7fe700 1 -- 172.21.15.121:0/2963674096 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v57) v1 ==== 84+0+143 (secure 0 0 0) 0x7f258c04c540 con 0x7f25a414a880 2023-11-15T02:59:32.287 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:32.287 INFO:teuthology.orchestra.run.smithi121.stdout:[ 2023-11-15T02:59:32.288 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:32.288 INFO:teuthology.orchestra.run.smithi121.stdout: "poolnum": 1, 2023-11-15T02:59:32.288 INFO:teuthology.orchestra.run.smithi121.stdout: "poolname": "device_health_metrics" 2023-11-15T02:59:32.288 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T02:59:32.289 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T02:59:32.289 INFO:teuthology.orchestra.run.smithi121.stdout: "poolnum": 2, 2023-11-15T02:59:32.289 INFO:teuthology.orchestra.run.smithi121.stdout: "poolname": "rbd" 2023-11-15T02:59:32.289 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T02:59:32.289 INFO:teuthology.orchestra.run.smithi121.stdout:] 2023-11-15T02:59:32.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.286+0000 7f25a9569700 1 -- 172.21.15.121:0/2963674096 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f258405a510 msgr2=0x7f258405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:32.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.286+0000 7f25a9569700 1 --2- 172.21.15.121:0/2963674096 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f258405a510 0x7f258405c9d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f2594007fd0 tx=0x7f2594009040).stop 2023-11-15T02:59:32.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.286+0000 7f25a9569700 1 -- 172.21.15.121:0/2963674096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25a414a880 msgr2=0x7f25a414a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:32.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.286+0000 7f25a9569700 1 --2- 172.21.15.121:0/2963674096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25a414a880 0x7f25a414a040 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f259000bf00 tx=0x7f258c015000).stop 2023-11-15T02:59:32.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.286+0000 7f25a9569700 1 -- 172.21.15.121:0/2963674096 shutdown_connections 2023-11-15T02:59:32.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.286+0000 7f25a9569700 1 --2- 172.21.15.121:0/2963674096 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f258405a510 0x7f258405c9d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:32.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.286+0000 7f25a9569700 1 --2- 172.21.15.121:0/2963674096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25a4148360 0x7f25a41487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:32.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.286+0000 7f25a9569700 1 --2- 172.21.15.121:0/2963674096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25a414a880 0x7f25a414a040 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:32.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.286+0000 7f25a9569700 1 --2- 172.21.15.121:0/2963674096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2590004690 0x7f25a40aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:32.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.286+0000 7f25a9569700 1 -- 172.21.15.121:0/2963674096 >> 172.21.15.121:0/2963674096 conn(0x7f25a4069ac0 msgr2=0x7f25a414bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:32.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.286+0000 7f25a9569700 1 -- 172.21.15.121:0/2963674096 shutdown_connections 2023-11-15T02:59:32.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.286+0000 7f25a9569700 1 -- 172.21.15.121:0/2963674096 wait complete. 2023-11-15T02:59:32.312 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-15T02:59:32.312 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-15T02:59:32.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.470+0000 7fa0a1b07700 1 Processor -- start 2023-11-15T02:59:32.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.470+0000 7fa0a1b07700 1 -- start start 2023-11-15T02:59:32.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.470+0000 7fa0a1b07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa09c0ad1b0 0x7fa09c065710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:32.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.470+0000 7fa0a1b07700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa09c066c90 con 0x7fa09c0ab0d0 2023-11-15T02:59:32.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.470+0000 7fa0a1b07700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa09c066080 con 0x7fa09c065c00 2023-11-15T02:59:32.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.470+0000 7fa0a1b07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c066360 con 0x7fa09c0ad1b0 2023-11-15T02:59:32.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa0a0b05700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa09c0ab0d0 0x7fa09c0ab4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58760/0 (socket says 172.21.15.121:58760) 2023-11-15T02:59:32.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa0a0b05700 1 -- 172.21.15.121:0/3872158713 learned_addr learned my addr 172.21.15.121:0/3872158713 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:32.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa09bfff700 1 --2- 172.21.15.121:0/3872158713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa09c0ad1b0 0x7fa09c065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:32.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa09b7fe700 1 -- 172.21.15.121:0/3872158713 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995380982 0 0) 0x7fa09c066080 con 0x7fa09c065c00 2023-11-15T02:59:32.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa09b7fe700 1 -- 172.21.15.121:0/3872158713 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa08c0033f0 con 0x7fa09c065c00 2023-11-15T02:59:32.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa09b7fe700 1 -- 172.21.15.121:0/3872158713 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828612780 0 0) 0x7fa09c066c90 con 0x7fa09c0ab0d0 2023-11-15T02:59:32.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa09b7fe700 1 -- 172.21.15.121:0/3872158713 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa09c066080 con 0x7fa09c0ab0d0 2023-11-15T02:59:32.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa09bfff700 1 -- 172.21.15.121:0/3872158713 >> v1:172.21.15.190:6789/0 conn(0x7fa09c065c00 legacy=0x7fa09c0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:32.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa09bfff700 1 -- 172.21.15.121:0/3872158713 >> v1:172.21.15.121:6789/0 conn(0x7fa09c0ab0d0 legacy=0x7fa09c0ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:32.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa09bfff700 1 -- 172.21.15.121:0/3872158713 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c066640 con 0x7fa09c0ad1b0 2023-11-15T02:59:32.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa09bfff700 1 --2- 172.21.15.121:0/3872158713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa09c0ad1b0 0x7fa09c065710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fa088009790 tx=0x7fa088009060).ready entity=mon.2 client_cookie=6c570d6fdd0ebbf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:32.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa09b7fe700 1 -- 172.21.15.121:0/3872158713 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08800f510 con 0x7fa09c0ad1b0 2023-11-15T02:59:32.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa09b7fe700 1 -- 172.21.15.121:0/3872158713 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08800f670 con 0x7fa09c0ad1b0 2023-11-15T02:59:32.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa09b7fe700 1 -- 172.21.15.121:0/3872158713 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08800f9f0 con 0x7fa09c0ad1b0 2023-11-15T02:59:32.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa0a1b07700 1 -- 172.21.15.121:0/3872158713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa09c0ad1b0 msgr2=0x7fa09c065710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:32.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa0a1b07700 1 --2- 172.21.15.121:0/3872158713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa09c0ad1b0 0x7fa09c065710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fa088009790 tx=0x7fa088009060).stop 2023-11-15T02:59:32.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa0a1b07700 1 -- 172.21.15.121:0/3872158713 shutdown_connections 2023-11-15T02:59:32.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa0a1b07700 1 --2- 172.21.15.121:0/3872158713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa09c0ad1b0 0x7fa09c065710 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:32.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa0a1b07700 1 -- 172.21.15.121:0/3872158713 >> 172.21.15.121:0/3872158713 conn(0x7fa09c067a60 msgr2=0x7fa09c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:32.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.474+0000 7fa0a1b07700 1 -- 172.21.15.121:0/3872158713 shutdown_connections 2023-11-15T02:59:32.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.478+0000 7fa0a1b07700 1 -- 172.21.15.121:0/3872158713 wait complete. 2023-11-15T02:59:32.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.478+0000 7fa0a1b07700 1 Processor -- start 2023-11-15T02:59:32.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.478+0000 7fa0a1b07700 1 -- start start 2023-11-15T02:59:32.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa0a1b07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa09c065c00 0x7fa09c14d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:32.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa0a1b07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa09c0ab0d0 0x7fa09c0a4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:32.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa0a1b07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa09c0ad1b0 0x7fa09c0a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:32.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa0a1b07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0a4b10 con 0x7fa09c065c00 2023-11-15T02:59:32.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa0a1b07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0c8ea0 con 0x7fa09c0ab0d0 2023-11-15T02:59:32.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa0a1b07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0c8290 con 0x7fa09c0ad1b0 2023-11-15T02:59:32.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa0a0b05700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa09c065c00 0x7fa09c14d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:32.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa0a0b05700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa09c065c00 0x7fa09c14d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43464/0 (socket says 172.21.15.121:43464) 2023-11-15T02:59:32.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa0a0b05700 1 -- 172.21.15.121:0/3944895341 learned_addr learned my addr 172.21.15.121:0/3944895341 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:32.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa09bfff700 1 --2- 172.21.15.121:0/3944895341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa09c0ab0d0 0x7fa09c0a4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:32.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa0a0b05700 1 -- 172.21.15.121:0/3944895341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa09c0ad1b0 msgr2=0x7fa09c0a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:32.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa0a0b05700 1 --2- 172.21.15.121:0/3944895341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa09c0ad1b0 0x7fa09c0a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T02:59:32.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa0a0b05700 1 -- 172.21.15.121:0/3944895341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa09c0ab0d0 msgr2=0x7fa09c0a4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:32.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa0a0b05700 1 --2- 172.21.15.121:0/3944895341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa09c0ab0d0 0x7fa09c0a4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:32.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa0a0b05700 1 -- 172.21.15.121:0/3944895341 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0880092a0 con 0x7fa09c065c00 2023-11-15T02:59:32.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa0a0b05700 1 --2- 172.21.15.121:0/3944895341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa09c065c00 0x7fa09c14d560 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fa0900049d0 tx=0x7fa090007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:32.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa098ff9700 1 -- 172.21.15.121:0/3944895341 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0900072e0 con 0x7fa09c065c00 2023-11-15T02:59:32.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa098ff9700 1 -- 172.21.15.121:0/3944895341 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa090007440 con 0x7fa09c065c00 2023-11-15T02:59:32.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa0a1b07700 1 -- 172.21.15.121:0/3944895341 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09c0c8570 con 0x7fa09c065c00 2023-11-15T02:59:32.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa098ff9700 1 -- 172.21.15.121:0/3944895341 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0900057a0 con 0x7fa09c065c00 2023-11-15T02:59:32.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa0a1b07700 1 -- 172.21.15.121:0/3944895341 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09c0c8b30 con 0x7fa09c065c00 2023-11-15T02:59:32.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.482+0000 7fa0a1b07700 1 -- 172.21.15.121:0/3944895341 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09c06bcf0 con 0x7fa09c065c00 2023-11-15T02:59:32.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.486+0000 7fa098ff9700 1 -- 172.21.15.121:0/3944895341 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa090005bd0 con 0x7fa09c065c00 2023-11-15T02:59:32.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.486+0000 7fa098ff9700 1 --2- 172.21.15.121:0/3944895341 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa08005a510 0x7fa08005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:32.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.486+0000 7fa098ff9700 1 -- 172.21.15.121:0/3944895341 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fa090073d90 con 0x7fa09c065c00 2023-11-15T02:59:32.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.486+0000 7fa09bfff700 1 --2- 172.21.15.121:0/3944895341 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa08005a510 0x7fa08005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:32.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.486+0000 7fa09bfff700 1 --2- 172.21.15.121:0/3944895341 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa08005a510 0x7fa08005c9d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fa09c0ad5d0 tx=0x7fa088017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:32.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.494+0000 7fa098ff9700 1 -- 172.21.15.121:0/3944895341 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa090047800 con 0x7fa09c065c00 2023-11-15T02:59:32.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:32.658+0000 7fa0a1b07700 1 -- 172.21.15.121:0/3944895341 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fa09c0c7160 con 0x7fa09c065c00 2023-11-15T02:59:33.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.218+0000 7fa098ff9700 1 -- 172.21.15.121:0/3944895341 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v58) v1 ==== 152+0+0 (secure 0 0 0) 0x7fa090007d90 con 0x7fa09c065c00 2023-11-15T02:59:33.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.222+0000 7fa0a1b07700 1 -- 172.21.15.121:0/3944895341 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa08005a510 msgr2=0x7fa08005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:33.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.222+0000 7fa0a1b07700 1 --2- 172.21.15.121:0/3944895341 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa08005a510 0x7fa08005c9d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fa09c0ad5d0 tx=0x7fa088017000).stop 2023-11-15T02:59:33.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.222+0000 7fa0a1b07700 1 -- 172.21.15.121:0/3944895341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa09c065c00 msgr2=0x7fa09c14d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:33.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.222+0000 7fa0a1b07700 1 --2- 172.21.15.121:0/3944895341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa09c065c00 0x7fa09c14d560 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fa0900049d0 tx=0x7fa090007c50).stop 2023-11-15T02:59:33.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.222+0000 7fa0a1b07700 1 -- 172.21.15.121:0/3944895341 shutdown_connections 2023-11-15T02:59:33.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.222+0000 7fa0a1b07700 1 --2- 172.21.15.121:0/3944895341 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa08005a510 0x7fa08005c9d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:33.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.222+0000 7fa0a1b07700 1 --2- 172.21.15.121:0/3944895341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa09c0ad1b0 0x7fa09c0a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T02:59:33.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.222+0000 7fa0a1b07700 1 --2- 172.21.15.121:0/3944895341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa09c0ab0d0 0x7fa09c0a4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:33.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.222+0000 7fa0a1b07700 1 --2- 172.21.15.121:0/3944895341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa09c065c00 0x7fa09c14d560 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:33.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.222+0000 7fa0a1b07700 1 -- 172.21.15.121:0/3944895341 >> 172.21.15.121:0/3944895341 conn(0x7fa09c067a60 msgr2=0x7fa09c0aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:33.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.226+0000 7fa0a1b07700 1 -- 172.21.15.121:0/3944895341 shutdown_connections 2023-11-15T02:59:33.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.226+0000 7fa0a1b07700 1 -- 172.21.15.121:0/3944895341 wait complete. 2023-11-15T02:59:33.241 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_metadata' created 2023-11-15T02:59:33.255 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-15T02:59:33.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.406+0000 7fec5d347700 1 Processor -- start 2023-11-15T02:59:33.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.406+0000 7fec5d347700 1 -- start start 2023-11-15T02:59:33.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.410+0000 7fec5d347700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec580a9f80 0x7fec58158260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:33.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.410+0000 7fec5d347700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fec580c7a90 0x7fec58154b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:33.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.410+0000 7fec5d347700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec580c6170 con 0x7fec580c5a00 2023-11-15T02:59:33.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.410+0000 7fec5d347700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec58154000 con 0x7fec580c7a90 2023-11-15T02:59:33.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.410+0000 7fec5d347700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec58157dd0 con 0x7fec580a9f80 2023-11-15T02:59:33.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.410+0000 7fec5cb46700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fec580c7a90 0x7fec58154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:33.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.410+0000 7fec57fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec580a9f80 0x7fec58158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:33.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.410+0000 7fec577fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fec580c5a00 0x7fec580c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:41280/0 (socket says 172.21.15.121:41280) 2023-11-15T02:59:33.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.410+0000 7fec577fe700 1 -- 172.21.15.121:0/1148653217 learned_addr learned my addr 172.21.15.121:0/1148653217 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:33.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.410+0000 7fec5cb46700 1 --2- 172.21.15.121:0/1148653217 >> v2:172.21.15.121:3300/0 conn(0x7fec580c7a90 0x7fec58154b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43468/0 (socket says 172.21.15.121:43468) 2023-11-15T02:59:33.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.410+0000 7fec5cb46700 1 -- 172.21.15.121:0/1148653217 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec580a9f80 msgr2=0x7fec58158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:33.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.410+0000 7fec5cb46700 1 --2- 172.21.15.121:0/1148653217 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec580a9f80 0x7fec58158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:33.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.410+0000 7fec5cb46700 1 -- 172.21.15.121:0/1148653217 >> v1:172.21.15.190:6789/0 conn(0x7fec580c5a00 legacy=0x7fec580c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:33.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.410+0000 7fec5cb46700 1 -- 172.21.15.121:0/1148653217 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec58155080 con 0x7fec580c7a90 2023-11-15T02:59:33.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.410+0000 7fec57fff700 1 --2- 172.21.15.121:0/1148653217 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec580a9f80 0x7fec58158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:33.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.410+0000 7fec5cb46700 1 --2- 172.21.15.121:0/1148653217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec580c7a90 0x7fec58154b90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7fec50009e60 tx=0x7fec50009780).ready entity=mon.0 client_cookie=d3a3cf5a00786798 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:33.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.410+0000 7fec56ffd700 1 -- 172.21.15.121:0/1148653217 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5000e070 con 0x7fec580c7a90 2023-11-15T02:59:33.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.410+0000 7fec56ffd700 1 -- 172.21.15.121:0/1148653217 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5000bc30 con 0x7fec580c7a90 2023-11-15T02:59:33.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.410+0000 7fec56ffd700 1 -- 172.21.15.121:0/1148653217 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec50012410 con 0x7fec580c7a90 2023-11-15T02:59:33.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.414+0000 7fec5d347700 1 -- 172.21.15.121:0/1148653217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec580c7a90 msgr2=0x7fec58154b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:33.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.414+0000 7fec5d347700 1 --2- 172.21.15.121:0/1148653217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec580c7a90 0x7fec58154b90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7fec50009e60 tx=0x7fec50009780).stop 2023-11-15T02:59:33.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.414+0000 7fec5d347700 1 -- 172.21.15.121:0/1148653217 shutdown_connections 2023-11-15T02:59:33.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.414+0000 7fec5d347700 1 --2- 172.21.15.121:0/1148653217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec580c7a90 0x7fec58154b90 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:33.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.414+0000 7fec5d347700 1 --2- 172.21.15.121:0/1148653217 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec580a9f80 0x7fec58158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:33.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.414+0000 7fec5d347700 1 -- 172.21.15.121:0/1148653217 >> 172.21.15.121:0/1148653217 conn(0x7fec58067a60 msgr2=0x7fec58067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:33.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.414+0000 7fec5d347700 1 -- 172.21.15.121:0/1148653217 shutdown_connections 2023-11-15T02:59:33.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.414+0000 7fec5d347700 1 -- 172.21.15.121:0/1148653217 wait complete. 2023-11-15T02:59:33.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.418+0000 7fec5d347700 1 Processor -- start 2023-11-15T02:59:33.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.418+0000 7fec5d347700 1 -- start start 2023-11-15T02:59:33.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.418+0000 7fec5d347700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec580a9f80 0x7fec580b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:33.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.418+0000 7fec5d347700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec580c7a90 0x7fec580caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:33.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.418+0000 7fec5d347700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec580cafc0 0x7fec581669a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:33.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.418+0000 7fec5d347700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec580afca0 con 0x7fec580a9f80 2023-11-15T02:59:33.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.418+0000 7fec5d347700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec580af090 con 0x7fec580cafc0 2023-11-15T02:59:33.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.418+0000 7fec5d347700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec580af370 con 0x7fec580c7a90 2023-11-15T02:59:33.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.418+0000 7fec57fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec580a9f80 0x7fec580b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:33.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.418+0000 7fec57fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec580a9f80 0x7fec580b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43480/0 (socket says 172.21.15.121:43480) 2023-11-15T02:59:33.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.418+0000 7fec57fff700 1 -- 172.21.15.121:0/1310900856 learned_addr learned my addr 172.21.15.121:0/1310900856 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:33.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.418+0000 7fec577fe700 1 --2- 172.21.15.121:0/1310900856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec580c7a90 0x7fec580caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:33.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.422+0000 7fec5cb46700 1 --2- 172.21.15.121:0/1310900856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec580cafc0 0x7fec581669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:33.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.422+0000 7fec57fff700 1 -- 172.21.15.121:0/1310900856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec580c7a90 msgr2=0x7fec580caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:33.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.422+0000 7fec57fff700 1 --2- 172.21.15.121:0/1310900856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec580c7a90 0x7fec580caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:33.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.422+0000 7fec57fff700 1 -- 172.21.15.121:0/1310900856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec580cafc0 msgr2=0x7fec581669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:33.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.422+0000 7fec57fff700 1 --2- 172.21.15.121:0/1310900856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec580cafc0 0x7fec581669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:33.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.422+0000 7fec57fff700 1 -- 172.21.15.121:0/1310900856 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec50009a20 con 0x7fec580a9f80 2023-11-15T02:59:33.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.422+0000 7fec5cb46700 1 --2- 172.21.15.121:0/1310900856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec580cafc0 0x7fec581669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:33.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.422+0000 7fec57fff700 1 --2- 172.21.15.121:0/1310900856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec580a9f80 0x7fec580b0610 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7fec4800d860 tx=0x7fec48008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:33.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.422+0000 7fec3ffff700 1 -- 172.21.15.121:0/1310900856 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec48011590 con 0x7fec580a9f80 2023-11-15T02:59:33.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.422+0000 7fec3ffff700 1 -- 172.21.15.121:0/1310900856 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec480116f0 con 0x7fec580a9f80 2023-11-15T02:59:33.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.422+0000 7fec3ffff700 1 -- 172.21.15.121:0/1310900856 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec48011a70 con 0x7fec580a9f80 2023-11-15T02:59:33.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.422+0000 7fec5d347700 1 -- 172.21.15.121:0/1310900856 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec580af650 con 0x7fec580a9f80 2023-11-15T02:59:33.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.422+0000 7fec5d347700 1 -- 172.21.15.121:0/1310900856 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec580b0b00 con 0x7fec580a9f80 2023-11-15T02:59:33.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.422+0000 7fec5d347700 1 -- 172.21.15.121:0/1310900856 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec5806bcf0 con 0x7fec580a9f80 2023-11-15T02:59:33.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.426+0000 7fec3ffff700 1 -- 172.21.15.121:0/1310900856 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fec4801c460 con 0x7fec580a9f80 2023-11-15T02:59:33.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.426+0000 7fec3ffff700 1 --2- 172.21.15.121:0/1310900856 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fec4405a510 0x7fec4405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:33.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.426+0000 7fec3ffff700 1 -- 172.21.15.121:0/1310900856 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5639+0+0 (secure 0 0 0) 0x7fec48074f10 con 0x7fec580a9f80 2023-11-15T02:59:33.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.426+0000 7fec577fe700 1 --2- 172.21.15.121:0/1310900856 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fec4405a510 0x7fec4405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:33.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.426+0000 7fec577fe700 1 --2- 172.21.15.121:0/1310900856 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fec4405a510 0x7fec4405c9d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fec580b0220 tx=0x7fec4c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:33.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.430+0000 7fec3ffff700 1 -- 172.21.15.121:0/1310900856 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fec48048870 con 0x7fec580a9f80 2023-11-15T02:59:33.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:33.590+0000 7fec5d347700 1 -- 172.21.15.121:0/1310900856 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fec580c5e90 con 0x7fec580a9f80 2023-11-15T02:59:34.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.233+0000 7fec3ffff700 1 -- 172.21.15.121:0/1310900856 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v59) v1 ==== 170+0+0 (secure 0 0 0) 0x7fec4804be90 con 0x7fec580a9f80 2023-11-15T02:59:34.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.241+0000 7fec5d347700 1 -- 172.21.15.121:0/1310900856 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fec4405a510 msgr2=0x7fec4405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:34.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.241+0000 7fec5d347700 1 --2- 172.21.15.121:0/1310900856 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fec4405a510 0x7fec4405c9d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fec580b0220 tx=0x7fec4c009040).stop 2023-11-15T02:59:34.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.241+0000 7fec5d347700 1 -- 172.21.15.121:0/1310900856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec580a9f80 msgr2=0x7fec580b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:34.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.241+0000 7fec5d347700 1 --2- 172.21.15.121:0/1310900856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec580a9f80 0x7fec580b0610 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7fec4800d860 tx=0x7fec48008040).stop 2023-11-15T02:59:34.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.241+0000 7fec5d347700 1 -- 172.21.15.121:0/1310900856 shutdown_connections 2023-11-15T02:59:34.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.241+0000 7fec5d347700 1 --2- 172.21.15.121:0/1310900856 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fec4405a510 0x7fec4405c9d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:34.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.241+0000 7fec5d347700 1 --2- 172.21.15.121:0/1310900856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec580cafc0 0x7fec581669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:34.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.241+0000 7fec5d347700 1 --2- 172.21.15.121:0/1310900856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec580c7a90 0x7fec580caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:34.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.241+0000 7fec5d347700 1 --2- 172.21.15.121:0/1310900856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec580a9f80 0x7fec580b0610 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:34.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.241+0000 7fec5d347700 1 -- 172.21.15.121:0/1310900856 >> 172.21.15.121:0/1310900856 conn(0x7fec58067a60 msgr2=0x7fec58158810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:34.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.245+0000 7fec5d347700 1 -- 172.21.15.121:0/1310900856 shutdown_connections 2023-11-15T02:59:34.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.245+0000 7fec5d347700 1 -- 172.21.15.121:0/1310900856 wait complete. 2023-11-15T02:59:34.256 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_data' created 2023-11-15T02:59:34.282 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-15T02:59:34.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.437+0000 7fbd1fddb700 1 Processor -- start 2023-11-15T02:59:34.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.437+0000 7fbd1fddb700 1 -- start start 2023-11-15T02:59:34.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.437+0000 7fbd1fddb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbd180670a0 0x7fbd180bcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:34.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.437+0000 7fbd1fddb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd18067480 con 0x7fbd18062190 2023-11-15T02:59:34.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.437+0000 7fbd1fddb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd18154530 con 0x7fbd180bc9b0 2023-11-15T02:59:34.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.437+0000 7fbd1fddb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd180c1dd0 con 0x7fbd180670a0 2023-11-15T02:59:34.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.437+0000 7fbd1e378700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fbd18062190 0x7fbd180c2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58776/0 (socket says 172.21.15.121:58776) 2023-11-15T02:59:34.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1e378700 1 -- 172.21.15.121:0/1929962546 learned_addr learned my addr 172.21.15.121:0/1929962546 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:34.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1d376700 1 --2- 172.21.15.121:0/1929962546 >> v2:172.21.15.190:3300/0 conn(0x7fbd180670a0 0x7fbd180bcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:34.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 -- 172.21.15.121:0/1929962546 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2382404080 0 0) 0x7fbd18067480 con 0x7fbd18062190 2023-11-15T02:59:34.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 -- 172.21.15.121:0/1929962546 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd04004500 con 0x7fbd18062190 2023-11-15T02:59:34.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 -- 172.21.15.121:0/1929962546 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373511306 0 0) 0x7fbd18154530 con 0x7fbd180bc9b0 2023-11-15T02:59:34.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 -- 172.21.15.121:0/1929962546 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd18067480 con 0x7fbd180bc9b0 2023-11-15T02:59:34.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 -- 172.21.15.121:0/1929962546 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2732511100 0 0) 0x7fbd04004500 con 0x7fbd18062190 2023-11-15T02:59:34.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 -- 172.21.15.121:0/1929962546 >> v2:172.21.15.190:3300/0 conn(0x7fbd180670a0 msgr2=0x7fbd180bcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:34.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 --2- 172.21.15.121:0/1929962546 >> v2:172.21.15.190:3300/0 conn(0x7fbd180670a0 0x7fbd180bcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:34.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 -- 172.21.15.121:0/1929962546 >> v1:172.21.15.190:6789/0 conn(0x7fbd180bc9b0 legacy=0x7fbd18153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:34.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 -- 172.21.15.121:0/1929962546 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd181547d0 con 0x7fbd18062190 2023-11-15T02:59:34.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 -- 172.21.15.121:0/1929962546 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fbd100017f0 con 0x7fbd18062190 2023-11-15T02:59:34.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 -- 172.21.15.121:0/1929962546 >> v1:172.21.15.121:6789/0 conn(0x7fbd18062190 legacy=0x7fbd180c2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:34.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 --2- 172.21.15.121:0/1929962546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd04004b80 0x7fbd04004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:34.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 --2- 172.21.15.121:0/1929962546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd040057a0 0x7fbd04005c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:34.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 --2- 172.21.15.121:0/1929962546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd04006160 0x7fbd0400c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:34.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 -- 172.21.15.121:0/1929962546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd180c1dd0 con 0x7fbd04006160 2023-11-15T02:59:34.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 -- 172.21.15.121:0/1929962546 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0400d0d0 con 0x7fbd040057a0 2023-11-15T02:59:34.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 -- 172.21.15.121:0/1929962546 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0400d290 con 0x7fbd04004b80 2023-11-15T02:59:34.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1e378700 1 --2- 172.21.15.121:0/1929962546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd04006160 0x7fbd0400c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:34.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1d376700 1 --2- 172.21.15.121:0/1929962546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd040057a0 0x7fbd04005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:34.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1db77700 1 --2- 172.21.15.121:0/1929962546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd04004b80 0x7fbd04004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:34.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1e378700 1 -- 172.21.15.121:0/1929962546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd04004b80 msgr2=0x7fbd04004f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:34.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1e378700 1 --2- 172.21.15.121:0/1929962546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd04004b80 0x7fbd04004f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:34.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1e378700 1 -- 172.21.15.121:0/1929962546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd040057a0 msgr2=0x7fbd04005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:34.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1e378700 1 --2- 172.21.15.121:0/1929962546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd040057a0 0x7fbd04005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:34.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1e378700 1 -- 172.21.15.121:0/1929962546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd04004500 con 0x7fbd04006160 2023-11-15T02:59:34.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1e378700 1 -- 172.21.15.121:0/1929962546 reap_dead start 2023-11-15T02:59:34.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1e378700 1 --2- 172.21.15.121:0/1929962546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd04006160 0x7fbd0400c9f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fbd1000c5c0 tx=0x7fbd10007f40).ready entity=mon.0 client_cookie=9ab55ffc088e1d14 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:34.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 -- 172.21.15.121:0/1929962546 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1000cdc0 con 0x7fbd04006160 2023-11-15T02:59:34.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 -- 172.21.15.121:0/1929962546 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd10005b20 con 0x7fbd04006160 2023-11-15T02:59:34.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.441+0000 7fbd1cb75700 1 -- 172.21.15.121:0/1929962546 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd100070d0 con 0x7fbd04006160 2023-11-15T02:59:34.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.445+0000 7fbd1fddb700 1 -- 172.21.15.121:0/1929962546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd04006160 msgr2=0x7fbd0400c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:34.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.445+0000 7fbd1fddb700 1 --2- 172.21.15.121:0/1929962546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd04006160 0x7fbd0400c9f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fbd1000c5c0 tx=0x7fbd10007f40).stop 2023-11-15T02:59:34.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.445+0000 7fbd1fddb700 1 -- 172.21.15.121:0/1929962546 shutdown_connections 2023-11-15T02:59:34.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.445+0000 7fbd1fddb700 1 --2- 172.21.15.121:0/1929962546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd04006160 0x7fbd0400c9f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:34.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.445+0000 7fbd1fddb700 1 -- 172.21.15.121:0/1929962546 >> 172.21.15.121:0/1929962546 conn(0x7fbd180591c0 msgr2=0x7fbd180c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:34.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.445+0000 7fbd1fddb700 1 -- 172.21.15.121:0/1929962546 shutdown_connections 2023-11-15T02:59:34.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.445+0000 7fbd1fddb700 1 -- 172.21.15.121:0/1929962546 wait complete. 2023-11-15T02:59:34.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1fddb700 1 Processor -- start 2023-11-15T02:59:34.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1fddb700 1 -- start start 2023-11-15T02:59:34.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1fddb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd04006160 0x7fbd18141390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:34.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1fddb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd1814ba50 0x7fbd1814beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:34.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1fddb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd1814dcb0 0x7fbd1814e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:34.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1fddb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1814d070 con 0x7fbd1814dcb0 2023-11-15T02:59:34.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1fddb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1814e620 con 0x7fbd04006160 2023-11-15T02:59:34.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1fddb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1814c460 con 0x7fbd1814ba50 2023-11-15T02:59:34.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1e378700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd1814dcb0 0x7fbd1814e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:34.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1e378700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd1814dcb0 0x7fbd1814e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43498/0 (socket says 172.21.15.121:43498) 2023-11-15T02:59:34.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1e378700 1 -- 172.21.15.121:0/4044712424 learned_addr learned my addr 172.21.15.121:0/4044712424 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:34.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1db77700 1 --2- 172.21.15.121:0/4044712424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd04006160 0x7fbd18141390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:34.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1d376700 1 --2- 172.21.15.121:0/4044712424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd1814ba50 0x7fbd1814beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1e378700 1 -- 172.21.15.121:0/4044712424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd1814ba50 msgr2=0x7fbd1814beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1e378700 1 --2- 172.21.15.121:0/4044712424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd1814ba50 0x7fbd1814beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1e378700 1 -- 172.21.15.121:0/4044712424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd04006160 msgr2=0x7fbd18141390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1e378700 1 --2- 172.21.15.121:0/4044712424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd04006160 0x7fbd18141390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1e378700 1 -- 172.21.15.121:0/4044712424 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1814c740 con 0x7fbd1814dcb0 2023-11-15T02:59:34.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1e378700 1 --2- 172.21.15.121:0/4044712424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd1814dcb0 0x7fbd1814e130 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7fbd1000c5c0 tx=0x7fbd10015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:34.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd0dffb700 1 -- 172.21.15.121:0/4044712424 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd10001400 con 0x7fbd1814dcb0 2023-11-15T02:59:34.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd0dffb700 1 -- 172.21.15.121:0/4044712424 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd100016f0 con 0x7fbd1814dcb0 2023-11-15T02:59:34.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd0dffb700 1 -- 172.21.15.121:0/4044712424 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd100136a0 con 0x7fbd1814dcb0 2023-11-15T02:59:34.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1fddb700 1 -- 172.21.15.121:0/4044712424 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd1814ca20 con 0x7fbd1814dcb0 2023-11-15T02:59:34.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.449+0000 7fbd1fddb700 1 -- 172.21.15.121:0/4044712424 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd180c05a0 con 0x7fbd1814dcb0 2023-11-15T02:59:34.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.453+0000 7fbd1fddb700 1 -- 172.21.15.121:0/4044712424 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd1814cea0 con 0x7fbd1814dcb0 2023-11-15T02:59:34.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.453+0000 7fbd0dffb700 1 -- 172.21.15.121:0/4044712424 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fbd10013ad0 con 0x7fbd1814dcb0 2023-11-15T02:59:34.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.453+0000 7fbd0dffb700 1 --2- 172.21.15.121:0/4044712424 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbcfc05a4e0 0x7fbcfc05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:34.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.453+0000 7fbd0dffb700 1 -- 172.21.15.121:0/4044712424 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6021+0+0 (secure 0 0 0) 0x7fbd10075cd0 con 0x7fbd1814dcb0 2023-11-15T02:59:34.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.457+0000 7fbd1db77700 1 --2- 172.21.15.121:0/4044712424 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbcfc05a4e0 0x7fbcfc05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:34.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.457+0000 7fbd1db77700 1 --2- 172.21.15.121:0/4044712424 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbcfc05a4e0 0x7fbcfc05c9a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7fbd140010e0 tx=0x7fbd14009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:34.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.465+0000 7fbd0dffb700 1 -- 172.21.15.121:0/4044712424 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbd10049450 con 0x7fbd1814dcb0 2023-11-15T02:59:34.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.661+0000 7fbd1fddb700 1 -- 172.21.15.121:0/4044712424 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbd18047060 con 0x7fbd1814dcb0 2023-11-15T02:59:34.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.673+0000 7fbd0dffb700 1 -- 172.21.15.121:0/4044712424 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v41) v1 ==== 173+0+0 (secure 0 0 0) 0x7fbd10025070 con 0x7fbd1814dcb0 2023-11-15T02:59:34.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.677+0000 7fbd1fddb700 1 -- 172.21.15.121:0/4044712424 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbcfc05a4e0 msgr2=0x7fbcfc05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:34.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.681+0000 7fbd1fddb700 1 --2- 172.21.15.121:0/4044712424 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbcfc05a4e0 0x7fbcfc05c9a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7fbd140010e0 tx=0x7fbd14009040).stop 2023-11-15T02:59:34.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.681+0000 7fbd1fddb700 1 -- 172.21.15.121:0/4044712424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd1814dcb0 msgr2=0x7fbd1814e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:34.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.681+0000 7fbd1fddb700 1 --2- 172.21.15.121:0/4044712424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd1814dcb0 0x7fbd1814e130 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7fbd1000c5c0 tx=0x7fbd10015000).stop 2023-11-15T02:59:34.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.681+0000 7fbd1fddb700 1 -- 172.21.15.121:0/4044712424 shutdown_connections 2023-11-15T02:59:34.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.681+0000 7fbd1fddb700 1 --2- 172.21.15.121:0/4044712424 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbcfc05a4e0 0x7fbcfc05c9a0 secure :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0x7fbd140010e0 tx=0x7fbd14009040).stop 2023-11-15T02:59:34.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.681+0000 7fbd1fddb700 1 --2- 172.21.15.121:0/4044712424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd1814dcb0 0x7fbd1814e130 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:34.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.681+0000 7fbd1fddb700 1 --2- 172.21.15.121:0/4044712424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd1814ba50 0x7fbd1814beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:34.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.681+0000 7fbd1fddb700 1 --2- 172.21.15.121:0/4044712424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd04006160 0x7fbd18141390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:34.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.681+0000 7fbd1fddb700 1 -- 172.21.15.121:0/4044712424 >> 172.21.15.121:0/4044712424 conn(0x7fbd180591c0 msgr2=0x7fbd180c36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:34.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.681+0000 7fbd1fddb700 1 -- 172.21.15.121:0/4044712424 shutdown_connections 2023-11-15T02:59:34.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.681+0000 7fbd1fddb700 1 -- 172.21.15.121:0/4044712424 wait complete. 2023-11-15T02:59:34.698 INFO:teuthology.orchestra.run.smithi121.stderr:new fs with metadata pool 9 and data pool 10 2023-11-15T02:59:34.721 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:59:34.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.865+0000 7fd29a307700 1 Processor -- start 2023-11-15T02:59:34.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.865+0000 7fd29a307700 1 -- start start 2023-11-15T02:59:34.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.865+0000 7fd29a307700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd2940c3280 0x7fd2940c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:34.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.865+0000 7fd29a307700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd294152140 0x7fd2941525c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:34.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd29a307700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd29415be30 0x7fd29415c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:34.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd29a307700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2940c1860 con 0x7fd2940c3280 2023-11-15T02:59:34.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd29a307700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2940c1d50 con 0x7fd29415be30 2023-11-15T02:59:34.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd29a307700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29415cb80 con 0x7fd294152140 2023-11-15T02:59:34.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd299305700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd2940c3280 0x7fd2940c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:34.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd299305700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd2940c3280 0x7fd2940c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39188/0 (socket says 172.21.15.121:39188) 2023-11-15T02:59:34.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd299305700 1 -- 172.21.15.121:0/2596644118 learned_addr learned my addr 172.21.15.121:0/2596644118 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:34.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd299305700 1 -- 172.21.15.121:0/2596644118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd294152140 msgr2=0x7fd2941525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:34.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd299305700 1 --2- 172.21.15.121:0/2596644118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd294152140 0x7fd2941525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T02:59:34.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd299305700 1 -- 172.21.15.121:0/2596644118 >> v2:172.21.15.190:3300/0 conn(0x7fd29415be30 msgr2=0x7fd29415c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:34.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd299305700 1 --2- 172.21.15.121:0/2596644118 >> v2:172.21.15.190:3300/0 conn(0x7fd29415be30 0x7fd29415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T02:59:34.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd299305700 1 -- 172.21.15.121:0/2596644118 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2940c1f80 con 0x7fd2940c3280 2023-11-15T02:59:34.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd299305700 1 --2- 172.21.15.121:0/2596644118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2940c3280 0x7fd2940c3660 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fd294156230 tx=0x7fd2900090f0).ready entity=mon.0 client_cookie=a8ee0f14268a5240 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:34.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd28bfff700 1 -- 172.21.15.121:0/2596644118 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29000b600 con 0x7fd2940c3280 2023-11-15T02:59:34.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd28bfff700 1 -- 172.21.15.121:0/2596644118 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29000b760 con 0x7fd2940c3280 2023-11-15T02:59:34.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd28bfff700 1 -- 172.21.15.121:0/2596644118 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29000bae0 con 0x7fd2940c3280 2023-11-15T02:59:34.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd29a307700 1 -- 172.21.15.121:0/2596644118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2940c3280 msgr2=0x7fd2940c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:34.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd29a307700 1 --2- 172.21.15.121:0/2596644118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2940c3280 0x7fd2940c3660 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fd294156230 tx=0x7fd2900090f0).stop 2023-11-15T02:59:34.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd29a307700 1 -- 172.21.15.121:0/2596644118 shutdown_connections 2023-11-15T02:59:34.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd29a307700 1 --2- 172.21.15.121:0/2596644118 >> v2:172.21.15.190:3300/0 conn(0x7fd29415be30 0x7fd29415c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T02:59:34.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd29a307700 1 --2- 172.21.15.121:0/2596644118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd294152140 0x7fd2941525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T02:59:34.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd29a307700 1 --2- 172.21.15.121:0/2596644118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2940c3280 0x7fd2940c3660 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:34.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd29a307700 1 -- 172.21.15.121:0/2596644118 >> 172.21.15.121:0/2596644118 conn(0x7fd294062190 msgr2=0x7fd294062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:34.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd29a307700 1 -- 172.21.15.121:0/2596644118 shutdown_connections 2023-11-15T02:59:34.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.869+0000 7fd29a307700 1 -- 172.21.15.121:0/2596644118 wait complete. 2023-11-15T02:59:34.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.873+0000 7fd29a307700 1 Processor -- start 2023-11-15T02:59:34.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.873+0000 7fd29a307700 1 -- start start 2023-11-15T02:59:34.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd29a307700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2940c3280 0x7fd294157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:34.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd29a307700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd294152140 0x7fd29414f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:34.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd29a307700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd29415be30 0x7fd2940a9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:34.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd29a307700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29416bed0 con 0x7fd29415be30 2023-11-15T02:59:34.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd29a307700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29416c0e0 con 0x7fd2940c3280 2023-11-15T02:59:34.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd29a307700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29416c3c0 con 0x7fd294152140 2023-11-15T02:59:34.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd299b06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd29415be30 0x7fd2940a9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:34.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd299b06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd29415be30 0x7fd2940a9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39190/0 (socket says 172.21.15.121:39190) 2023-11-15T02:59:34.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd299b06700 1 -- 172.21.15.121:0/205233596 learned_addr learned my addr 172.21.15.121:0/205233596 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:34.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd299305700 1 --2- 172.21.15.121:0/205233596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2940c3280 0x7fd294157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:34.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd298b04700 1 --2- 172.21.15.121:0/205233596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd294152140 0x7fd29414f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:34.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd299b06700 1 -- 172.21.15.121:0/205233596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd294152140 msgr2=0x7fd29414f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:34.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd299b06700 1 --2- 172.21.15.121:0/205233596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd294152140 0x7fd29414f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:34.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd299b06700 1 -- 172.21.15.121:0/205233596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2940c3280 msgr2=0x7fd294157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:34.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd299b06700 1 --2- 172.21.15.121:0/205233596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2940c3280 0x7fd294157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:34.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd299b06700 1 -- 172.21.15.121:0/205233596 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd290009370 con 0x7fd29415be30 2023-11-15T02:59:34.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd299b06700 1 --2- 172.21.15.121:0/205233596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd29415be30 0x7fd2940a9ae0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fd28c00b810 tx=0x7fd28c0092b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:34.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd2897fa700 1 -- 172.21.15.121:0/205233596 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c00d4a0 con 0x7fd29415be30 2023-11-15T02:59:34.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd2897fa700 1 -- 172.21.15.121:0/205233596 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c00d600 con 0x7fd29415be30 2023-11-15T02:59:34.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd29a307700 1 -- 172.21.15.121:0/205233596 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd29416c6a0 con 0x7fd29415be30 2023-11-15T02:59:34.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd2897fa700 1 -- 172.21.15.121:0/205233596 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c00d9c0 con 0x7fd29415be30 2023-11-15T02:59:34.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.877+0000 7fd29a307700 1 -- 172.21.15.121:0/205233596 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29416cc60 con 0x7fd29415be30 2023-11-15T02:59:34.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.881+0000 7fd29a307700 1 -- 172.21.15.121:0/205233596 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd29406bcf0 con 0x7fd29415be30 2023-11-15T02:59:34.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.881+0000 7fd2897fa700 1 -- 172.21.15.121:0/205233596 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd28c011400 con 0x7fd29415be30 2023-11-15T02:59:34.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.885+0000 7fd2897fa700 1 --2- 172.21.15.121:0/205233596 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd27805a510 0x7fd27805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:34.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.885+0000 7fd2897fa700 1 -- 172.21.15.121:0/205233596 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd28c0740a0 con 0x7fd29415be30 2023-11-15T02:59:34.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.885+0000 7fd299305700 1 --2- 172.21.15.121:0/205233596 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd27805a510 0x7fd27805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:34.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.885+0000 7fd299305700 1 --2- 172.21.15.121:0/205233596 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd27805a510 0x7fd27805c9d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fd290009130 tx=0x7fd290017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:34.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:34.889+0000 7fd2897fa700 1 -- 172.21.15.121:0/205233596 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd28c047810 con 0x7fd29415be30 2023-11-15T02:59:35.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.053+0000 7fd29a307700 1 -- 172.21.15.121:0/205233596 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2940c2260 con 0x7fd29415be30 2023-11-15T02:59:35.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.053+0000 7fd2897fa700 1 -- 172.21.15.121:0/205233596 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16991 (secure 0 0 0) 0x7fd28c04ae30 con 0x7fd29415be30 2023-11-15T02:59:35.063 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:35.063 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":60,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:59:34.667093+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:59:32.734725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-15T02:59:33.665552+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:59:35.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.057+0000 7fd29a307700 1 -- 172.21.15.121:0/205233596 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd27805a510 msgr2=0x7fd27805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:35.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.057+0000 7fd29a307700 1 --2- 172.21.15.121:0/205233596 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd27805a510 0x7fd27805c9d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fd290009130 tx=0x7fd290017000).stop 2023-11-15T02:59:35.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.057+0000 7fd29a307700 1 -- 172.21.15.121:0/205233596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd29415be30 msgr2=0x7fd2940a9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:35.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.057+0000 7fd29a307700 1 --2- 172.21.15.121:0/205233596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd29415be30 0x7fd2940a9ae0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fd28c00b810 tx=0x7fd28c0092b0).stop 2023-11-15T02:59:35.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.057+0000 7fd29a307700 1 -- 172.21.15.121:0/205233596 shutdown_connections 2023-11-15T02:59:35.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.057+0000 7fd29a307700 1 --2- 172.21.15.121:0/205233596 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd27805a510 0x7fd27805c9d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:35.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.057+0000 7fd29a307700 1 --2- 172.21.15.121:0/205233596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd29415be30 0x7fd2940a9ae0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:35.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.057+0000 7fd29a307700 1 --2- 172.21.15.121:0/205233596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd294152140 0x7fd29414f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:35.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.057+0000 7fd29a307700 1 --2- 172.21.15.121:0/205233596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2940c3280 0x7fd294157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:35.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.057+0000 7fd29a307700 1 -- 172.21.15.121:0/205233596 >> 172.21.15.121:0/205233596 conn(0x7fd294062190 msgr2=0x7fd294062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:35.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.057+0000 7fd29a307700 1 -- 172.21.15.121:0/205233596 shutdown_connections 2023-11-15T02:59:35.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.057+0000 7fd29a307700 1 -- 172.21.15.121:0/205233596 wait complete. 2023-11-15T02:59:35.086 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:59:35.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.233+0000 7f548b039700 1 Processor -- start 2023-11-15T02:59:35.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.233+0000 7f548b039700 1 -- start start 2023-11-15T02:59:35.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f548b039700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f54841392d0 0x7f5484066b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:35.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f548b039700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5484063730 con 0x7f54841396b0 2023-11-15T02:59:35.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f548b039700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5484063960 con 0x7f5484063280 2023-11-15T02:59:35.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f548b039700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5484063c40 con 0x7f54841392d0 2023-11-15T02:59:35.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489836700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f54841396b0 0x7f5484062b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51774/0 (socket says 172.21.15.121:51774) 2023-11-15T02:59:35.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489836700 1 -- 172.21.15.121:0/2406664493 learned_addr learned my addr 172.21.15.121:0/2406664493 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:35.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f548a037700 1 --2- 172.21.15.121:0/2406664493 >> v2:172.21.15.121:3300/0 conn(0x7f54841392d0 0x7f5484066b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:35.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 -- 172.21.15.121:0/2406664493 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1523989585 0 0) 0x7f5484063730 con 0x7f54841396b0 2023-11-15T02:59:35.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 -- 172.21.15.121:0/2406664493 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54700033f0 con 0x7f54841396b0 2023-11-15T02:59:35.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 -- 172.21.15.121:0/2406664493 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2465852491 0 0) 0x7f5484063960 con 0x7f5484063280 2023-11-15T02:59:35.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 -- 172.21.15.121:0/2406664493 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5484063730 con 0x7f5484063280 2023-11-15T02:59:35.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 -- 172.21.15.121:0/2406664493 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2929726627 0 0) 0x7f54700033f0 con 0x7f54841396b0 2023-11-15T02:59:35.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 -- 172.21.15.121:0/2406664493 >> v2:172.21.15.121:3300/0 conn(0x7f54841392d0 msgr2=0x7f5484066b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:35.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 --2- 172.21.15.121:0/2406664493 >> v2:172.21.15.121:3300/0 conn(0x7f54841392d0 0x7f5484066b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:35.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 -- 172.21.15.121:0/2406664493 >> v1:172.21.15.190:6789/0 conn(0x7f5484063280 legacy=0x7f54840c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:35.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 -- 172.21.15.121:0/2406664493 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5484063f20 con 0x7f54841396b0 2023-11-15T02:59:35.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 -- 172.21.15.121:0/2406664493 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f5474001470 con 0x7f54841396b0 2023-11-15T02:59:35.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 -- 172.21.15.121:0/2406664493 >> v1:172.21.15.121:6789/0 conn(0x7f54841396b0 legacy=0x7f5484062b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:35.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 --2- 172.21.15.121:0/2406664493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5470003a70 0x7f5470003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:35.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 --2- 172.21.15.121:0/2406664493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5470004690 0x7f5470004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:35.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 --2- 172.21.15.121:0/2406664493 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5470005000 0x7f547000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:35.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 -- 172.21.15.121:0/2406664493 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5484063c40 con 0x7f5470005000 2023-11-15T02:59:35.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 -- 172.21.15.121:0/2406664493 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547000bf60 con 0x7f5470004690 2023-11-15T02:59:35.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 -- 172.21.15.121:0/2406664493 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547000c0d0 con 0x7f5470003a70 2023-11-15T02:59:35.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f548a838700 1 --2- 172.21.15.121:0/2406664493 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5470005000 0x7f547000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:35.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f548a037700 1 --2- 172.21.15.121:0/2406664493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5470003a70 0x7f5470003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:35.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489836700 1 --2- 172.21.15.121:0/2406664493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5470004690 0x7f5470004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:35.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f548a838700 1 -- 172.21.15.121:0/2406664493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5470003a70 msgr2=0x7f5470003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:35.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f548a838700 1 --2- 172.21.15.121:0/2406664493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5470003a70 0x7f5470003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:35.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f548a838700 1 -- 172.21.15.121:0/2406664493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5470004690 msgr2=0x7f5470004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:35.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f548a838700 1 --2- 172.21.15.121:0/2406664493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5470004690 0x7f5470004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:35.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f548a838700 1 -- 172.21.15.121:0/2406664493 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54700033f0 con 0x7f5470005000 2023-11-15T02:59:35.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f548a838700 1 -- 172.21.15.121:0/2406664493 reap_dead start 2023-11-15T02:59:35.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f548a838700 1 --2- 172.21.15.121:0/2406664493 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5470005000 0x7f547000b8e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f547c001f90 tx=0x7f547c007f30).ready entity=mon.0 client_cookie=93646c6cfe85813d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:35.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 -- 172.21.15.121:0/2406664493 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c006a20 con 0x7f5470005000 2023-11-15T02:59:35.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 -- 172.21.15.121:0/2406664493 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c006b80 con 0x7f5470005000 2023-11-15T02:59:35.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.237+0000 7f5489035700 1 -- 172.21.15.121:0/2406664493 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c00c8b0 con 0x7f5470005000 2023-11-15T02:59:35.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.241+0000 7f548b039700 1 -- 172.21.15.121:0/2406664493 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5470005000 msgr2=0x7f547000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:35.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.241+0000 7f548b039700 1 --2- 172.21.15.121:0/2406664493 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5470005000 0x7f547000b8e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f547c001f90 tx=0x7f547c007f30).stop 2023-11-15T02:59:35.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.241+0000 7f548b039700 1 -- 172.21.15.121:0/2406664493 shutdown_connections 2023-11-15T02:59:35.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.241+0000 7f548b039700 1 --2- 172.21.15.121:0/2406664493 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5470005000 0x7f547000b8e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:35.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.241+0000 7f548b039700 1 -- 172.21.15.121:0/2406664493 >> 172.21.15.121:0/2406664493 conn(0x7f54840c7490 msgr2=0x7f54840c3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:35.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.241+0000 7f548b039700 1 -- 172.21.15.121:0/2406664493 shutdown_connections 2023-11-15T02:59:35.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.241+0000 7f548b039700 1 -- 172.21.15.121:0/2406664493 wait complete. 2023-11-15T02:59:35.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.241+0000 7f548b039700 1 Processor -- start 2023-11-15T02:59:35.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.241+0000 7f548b039700 1 -- start start 2023-11-15T02:59:35.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f548b039700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5470005000 0x7f5484138020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:35.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f548b039700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54840a42d0 0x7f54840a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:35.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f548b039700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54840bfbb0 0x7f54840c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:35.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f548b039700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54840c0500 con 0x7f54840bfbb0 2023-11-15T02:59:35.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f548b039700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54840a71e0 con 0x7f5470005000 2023-11-15T02:59:35.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f548b039700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54840a74c0 con 0x7f54840a42d0 2023-11-15T02:59:35.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f548a838700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54840bfbb0 0x7f54840c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:35.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f548a838700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54840bfbb0 0x7f54840c0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39206/0 (socket says 172.21.15.121:39206) 2023-11-15T02:59:35.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f548a838700 1 -- 172.21.15.121:0/198766941 learned_addr learned my addr 172.21.15.121:0/198766941 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:35.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f5489836700 1 --2- 172.21.15.121:0/198766941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54840a42d0 0x7f54840a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:35.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f548a037700 1 --2- 172.21.15.121:0/198766941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5470005000 0x7f5484138020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:35.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f548a838700 1 -- 172.21.15.121:0/198766941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54840a42d0 msgr2=0x7f54840a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:35.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f548a838700 1 --2- 172.21.15.121:0/198766941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54840a42d0 0x7f54840a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:35.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f548a838700 1 -- 172.21.15.121:0/198766941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5470005000 msgr2=0x7f5484138020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:35.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f548a838700 1 --2- 172.21.15.121:0/198766941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5470005000 0x7f5484138020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:35.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f548a838700 1 -- 172.21.15.121:0/198766941 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54840a77a0 con 0x7f54840bfbb0 2023-11-15T02:59:35.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f548a838700 1 --2- 172.21.15.121:0/198766941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54840bfbb0 0x7f54840c0010 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f5484065680 tx=0x7f547c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:35.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f547a7fc700 1 -- 172.21.15.121:0/198766941 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c006a20 con 0x7f54840bfbb0 2023-11-15T02:59:35.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f547a7fc700 1 -- 172.21.15.121:0/198766941 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c00c8b0 con 0x7f54840bfbb0 2023-11-15T02:59:35.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f548b039700 1 -- 172.21.15.121:0/198766941 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54840a7a80 con 0x7f54840bfbb0 2023-11-15T02:59:35.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f547a7fc700 1 -- 172.21.15.121:0/198766941 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c01f440 con 0x7f54840bfbb0 2023-11-15T02:59:35.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.245+0000 7f548b039700 1 -- 172.21.15.121:0/198766941 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54841645b0 con 0x7f54840bfbb0 2023-11-15T02:59:35.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.249+0000 7f548b039700 1 -- 172.21.15.121:0/198766941 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f548406bcf0 con 0x7f54840bfbb0 2023-11-15T02:59:35.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.249+0000 7f547a7fc700 1 -- 172.21.15.121:0/198766941 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f547c01e3e0 con 0x7f54840bfbb0 2023-11-15T02:59:35.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.249+0000 7f547a7fc700 1 --2- 172.21.15.121:0/198766941 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f546805a540 0x7f546805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:35.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.249+0000 7f547a7fc700 1 -- 172.21.15.121:0/198766941 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f547c075a30 con 0x7f54840bfbb0 2023-11-15T02:59:35.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.253+0000 7f548a037700 1 --2- 172.21.15.121:0/198766941 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f546805a540 0x7f546805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:35.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.253+0000 7f548a037700 1 --2- 172.21.15.121:0/198766941 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f546805a540 0x7f546805ca00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f5480009590 tx=0x7f5480008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:35.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.253+0000 7f547a7fc700 1 -- 172.21.15.121:0/198766941 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f547c0491b0 con 0x7f54840bfbb0 2023-11-15T02:59:35.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.417+0000 7f548b039700 1 -- 172.21.15.121:0/198766941 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54840c2880 con 0x7f54840bfbb0 2023-11-15T02:59:35.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.421+0000 7f547a7fc700 1 -- 172.21.15.121:0/198766941 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16991 (secure 0 0 0) 0x7f547c04c7d0 con 0x7f54840bfbb0 2023-11-15T02:59:35.428 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:35.429 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":60,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:59:34.667093+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:59:32.734725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-15T02:59:33.665552+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:59:35.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.421+0000 7f548b039700 1 -- 172.21.15.121:0/198766941 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f546805a540 msgr2=0x7f546805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:35.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.421+0000 7f548b039700 1 --2- 172.21.15.121:0/198766941 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f546805a540 0x7f546805ca00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f5480009590 tx=0x7f5480008040).stop 2023-11-15T02:59:35.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.421+0000 7f548b039700 1 -- 172.21.15.121:0/198766941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54840bfbb0 msgr2=0x7f54840c0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:35.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.421+0000 7f548b039700 1 --2- 172.21.15.121:0/198766941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54840bfbb0 0x7f54840c0010 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f5484065680 tx=0x7f547c015000).stop 2023-11-15T02:59:35.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.421+0000 7f548b039700 1 -- 172.21.15.121:0/198766941 shutdown_connections 2023-11-15T02:59:35.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.421+0000 7f548b039700 1 --2- 172.21.15.121:0/198766941 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f546805a540 0x7f546805ca00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:35.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.421+0000 7f548b039700 1 --2- 172.21.15.121:0/198766941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54840bfbb0 0x7f54840c0010 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:35.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.421+0000 7f548b039700 1 --2- 172.21.15.121:0/198766941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54840a42d0 0x7f54840a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:35.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.421+0000 7f548b039700 1 --2- 172.21.15.121:0/198766941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5470005000 0x7f5484138020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:35.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.421+0000 7f548b039700 1 -- 172.21.15.121:0/198766941 >> 172.21.15.121:0/198766941 conn(0x7f54840c7490 msgr2=0x7f54840bcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:35.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.421+0000 7f548b039700 1 -- 172.21.15.121:0/198766941 shutdown_connections 2023-11-15T02:59:35.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.421+0000 7f548b039700 1 -- 172.21.15.121:0/198766941 wait complete. 2023-11-15T02:59:35.453 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-15T02:59:35.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.609+0000 7f24919b9700 1 Processor -- start 2023-11-15T02:59:35.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.609+0000 7f24919b9700 1 -- start start 2023-11-15T02:59:35.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.609+0000 7f24919b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f248c153ac0 0x7f248c0c7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:35.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.609+0000 7f24919b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f248c0c7680 0x7f248c0ca860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:35.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.609+0000 7f24919b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248c065c90 con 0x7f248c0cad50 2023-11-15T02:59:35.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.609+0000 7f24919b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248c0c8210 con 0x7f248c153ac0 2023-11-15T02:59:35.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.609+0000 7f24919b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248c0cb1e0 con 0x7f248c0c7680 2023-11-15T02:59:35.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.609+0000 7f24909b7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f248c153ac0 0x7f248c0c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:35.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.609+0000 7f24909b7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f248c153ac0 0x7f248c0c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39218/0 (socket says 172.21.15.121:39218) 2023-11-15T02:59:35.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.609+0000 7f24909b7700 1 -- 172.21.15.121:0/484806249 learned_addr learned my addr 172.21.15.121:0/484806249 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:35.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.609+0000 7f248bfff700 1 --2- 172.21.15.121:0/484806249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f248c0c7680 0x7f248c0ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:35.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.609+0000 7f24909b7700 1 -- 172.21.15.121:0/484806249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f248c0c7680 msgr2=0x7f248c0ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:35.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.609+0000 7f24909b7700 1 --2- 172.21.15.121:0/484806249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f248c0c7680 0x7f248c0ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:35.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.609+0000 7f24909b7700 1 -- 172.21.15.121:0/484806249 >> v1:172.21.15.190:6789/0 conn(0x7f248c0cad50 legacy=0x7f248c066120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:35.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.609+0000 7f24909b7700 1 -- 172.21.15.121:0/484806249 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c0c7c50 con 0x7f248c153ac0 2023-11-15T02:59:35.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.609+0000 7f24909b7700 1 --2- 172.21.15.121:0/484806249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f248c153ac0 0x7f248c0c7190 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f2480006fd0 tx=0x7f24800090f0).ready entity=mon.0 client_cookie=e97b67f8a3e3f8f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:35.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.613+0000 7f248b7fe700 1 -- 172.21.15.121:0/484806249 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248000b580 con 0x7f248c153ac0 2023-11-15T02:59:35.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.613+0000 7f248b7fe700 1 -- 172.21.15.121:0/484806249 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248000b6e0 con 0x7f248c153ac0 2023-11-15T02:59:35.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.613+0000 7f248b7fe700 1 -- 172.21.15.121:0/484806249 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248000ba80 con 0x7f248c153ac0 2023-11-15T02:59:35.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.613+0000 7f24919b9700 1 -- 172.21.15.121:0/484806249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f248c153ac0 msgr2=0x7f248c0c7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:35.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.613+0000 7f24919b9700 1 --2- 172.21.15.121:0/484806249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f248c153ac0 0x7f248c0c7190 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f2480006fd0 tx=0x7f24800090f0).stop 2023-11-15T02:59:35.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.613+0000 7f24919b9700 1 -- 172.21.15.121:0/484806249 shutdown_connections 2023-11-15T02:59:35.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.613+0000 7f24919b9700 1 --2- 172.21.15.121:0/484806249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f248c0c7680 0x7f248c0ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:35.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.613+0000 7f24919b9700 1 --2- 172.21.15.121:0/484806249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f248c153ac0 0x7f248c0c7190 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:35.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.613+0000 7f24919b9700 1 -- 172.21.15.121:0/484806249 >> 172.21.15.121:0/484806249 conn(0x7f248c067a60 msgr2=0x7f248c067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:35.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.613+0000 7f24919b9700 1 -- 172.21.15.121:0/484806249 shutdown_connections 2023-11-15T02:59:35.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.613+0000 7f24919b9700 1 -- 172.21.15.121:0/484806249 wait complete. 2023-11-15T02:59:35.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.613+0000 7f24919b9700 1 Processor -- start 2023-11-15T02:59:35.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.613+0000 7f24919b9700 1 -- start start 2023-11-15T02:59:35.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f24919b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f248c0c7680 0x7f248c155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:35.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f24919b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f248c0cad50 0x7f248c14c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:35.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f24919b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f248c153ac0 0x7f248c158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:35.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f24919b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c156120 con 0x7f248c153ac0 2023-11-15T02:59:35.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f24919b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c154b90 con 0x7f248c0c7680 2023-11-15T02:59:35.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f24919b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248c154e70 con 0x7f248c0cad50 2023-11-15T02:59:35.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f24911b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f248c153ac0 0x7f248c158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:35.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f24911b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f248c153ac0 0x7f248c158480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39230/0 (socket says 172.21.15.121:39230) 2023-11-15T02:59:35.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f24911b8700 1 -- 172.21.15.121:0/1430550448 learned_addr learned my addr 172.21.15.121:0/1430550448 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:35.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f24909b7700 1 --2- 172.21.15.121:0/1430550448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f248c0c7680 0x7f248c155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:35.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f248bfff700 1 --2- 172.21.15.121:0/1430550448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f248c0cad50 0x7f248c14c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:35.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f24911b8700 1 -- 172.21.15.121:0/1430550448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f248c0cad50 msgr2=0x7f248c14c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:35.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f24911b8700 1 --2- 172.21.15.121:0/1430550448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f248c0cad50 0x7f248c14c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:35.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f24911b8700 1 -- 172.21.15.121:0/1430550448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f248c0c7680 msgr2=0x7f248c155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:35.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f24911b8700 1 --2- 172.21.15.121:0/1430550448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f248c0c7680 0x7f248c155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:35.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f24911b8700 1 -- 172.21.15.121:0/1430550448 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2480009370 con 0x7f248c153ac0 2023-11-15T02:59:35.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f24911b8700 1 --2- 172.21.15.121:0/1430550448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f248c153ac0 0x7f248c158480 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f2484002230 tx=0x7f24840015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:35.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f2488ff9700 1 -- 172.21.15.121:0/1430550448 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484010510 con 0x7f248c153ac0 2023-11-15T02:59:35.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f2488ff9700 1 -- 172.21.15.121:0/1430550448 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2484010670 con 0x7f248c153ac0 2023-11-15T02:59:35.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f2488ff9700 1 -- 172.21.15.121:0/1430550448 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24840109f0 con 0x7f248c153ac0 2023-11-15T02:59:35.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f24919b9700 1 -- 172.21.15.121:0/1430550448 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248c155150 con 0x7f248c153ac0 2023-11-15T02:59:35.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.617+0000 7f24919b9700 1 -- 172.21.15.121:0/1430550448 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248c155710 con 0x7f248c153ac0 2023-11-15T02:59:35.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.621+0000 7f24919b9700 1 -- 172.21.15.121:0/1430550448 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f248c155960 con 0x7f248c153ac0 2023-11-15T02:59:35.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.621+0000 7f2488ff9700 1 -- 172.21.15.121:0/1430550448 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f248401b400 con 0x7f248c153ac0 2023-11-15T02:59:35.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.621+0000 7f2488ff9700 1 --2- 172.21.15.121:0/1430550448 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f247005a510 0x7f247005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:35.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.621+0000 7f2488ff9700 1 -- 172.21.15.121:0/1430550448 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2484074600 con 0x7f248c153ac0 2023-11-15T02:59:35.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.621+0000 7f24909b7700 1 --2- 172.21.15.121:0/1430550448 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f247005a510 0x7f247005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:35.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.625+0000 7f24909b7700 1 --2- 172.21.15.121:0/1430550448 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f247005a510 0x7f247005c9d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f248000b340 tx=0x7f2480017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:35.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.629+0000 7f2488ff9700 1 -- 172.21.15.121:0/1430550448 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2484047cc0 con 0x7f248c153ac0 2023-11-15T02:59:35.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:35.825+0000 7f24919b9700 1 -- 172.21.15.121:0/1430550448 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f248c06bcf0 con 0x7f248c153ac0 2023-11-15T02:59:36.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.689+0000 7f2488ff9700 1 -- 172.21.15.121:0/1430550448 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v44) v1 ==== 118+0+0 (secure 0 0 0) 0x7f248404b2e0 con 0x7f248c153ac0 2023-11-15T02:59:36.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.693+0000 7f24919b9700 1 -- 172.21.15.121:0/1430550448 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f247005a510 msgr2=0x7f247005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:36.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.693+0000 7f24919b9700 1 --2- 172.21.15.121:0/1430550448 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f247005a510 0x7f247005c9d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f248000b340 tx=0x7f2480017000).stop 2023-11-15T02:59:36.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.693+0000 7f24919b9700 1 -- 172.21.15.121:0/1430550448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f248c153ac0 msgr2=0x7f248c158480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:36.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.693+0000 7f24919b9700 1 --2- 172.21.15.121:0/1430550448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f248c153ac0 0x7f248c158480 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f2484002230 tx=0x7f24840015e0).stop 2023-11-15T02:59:36.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.693+0000 7f24919b9700 1 -- 172.21.15.121:0/1430550448 shutdown_connections 2023-11-15T02:59:36.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.693+0000 7f24919b9700 1 --2- 172.21.15.121:0/1430550448 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f247005a510 0x7f247005c9d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:36.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.693+0000 7f24919b9700 1 --2- 172.21.15.121:0/1430550448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f248c153ac0 0x7f248c158480 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:36.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.693+0000 7f24919b9700 1 --2- 172.21.15.121:0/1430550448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f248c0cad50 0x7f248c14c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:36.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.693+0000 7f24919b9700 1 --2- 172.21.15.121:0/1430550448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f248c0c7680 0x7f248c155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:36.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.693+0000 7f24919b9700 1 -- 172.21.15.121:0/1430550448 >> 172.21.15.121:0/1430550448 conn(0x7f248c067a60 msgr2=0x7f248c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:36.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.693+0000 7f24919b9700 1 -- 172.21.15.121:0/1430550448 shutdown_connections 2023-11-15T02:59:36.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.693+0000 7f24919b9700 1 -- 172.21.15.121:0/1430550448 wait complete. 2023-11-15T02:59:36.724 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-15T02:59:36.724 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-15T02:59:36.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.893+0000 7fde96d07700 1 Processor -- start 2023-11-15T02:59:36.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.893+0000 7fde96d07700 1 -- start start 2023-11-15T02:59:36.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.897+0000 7fde96d07700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fde900c5c70 0x7fde90153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:36.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.897+0000 7fde96d07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde90066330 0x7fde900ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:36.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.897+0000 7fde96d07700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde90065ea0 con 0x7fde900acd00 2023-11-15T02:59:36.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.897+0000 7fde96d07700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde90066bb0 con 0x7fde900c5c70 2023-11-15T02:59:36.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.897+0000 7fde96d07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde90065890 con 0x7fde90066330 2023-11-15T02:59:36.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.897+0000 7fde95504700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fde900c5c70 0x7fde90153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:36.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.897+0000 7fde95d05700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fde900acd00 0x7fde901536c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51780/0 (socket says 172.21.15.121:51780) 2023-11-15T02:59:36.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.897+0000 7fde95d05700 1 -- 172.21.15.121:0/1335291158 learned_addr learned my addr 172.21.15.121:0/1335291158 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:36.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.897+0000 7fde96506700 1 --2- 172.21.15.121:0/1335291158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde90066330 0x7fde900ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:36.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.897+0000 7fde94d03700 1 -- 172.21.15.121:0/1335291158 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1312588322 0 0) 0x7fde90065ea0 con 0x7fde900acd00 2023-11-15T02:59:36.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.897+0000 7fde94d03700 1 -- 172.21.15.121:0/1335291158 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde7c0033f0 con 0x7fde900acd00 2023-11-15T02:59:36.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.897+0000 7fde95504700 1 -- 172.21.15.121:0/1335291158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde90066330 msgr2=0x7fde900ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:36.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.897+0000 7fde95504700 1 --2- 172.21.15.121:0/1335291158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde90066330 0x7fde900ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:36.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.897+0000 7fde95504700 1 -- 172.21.15.121:0/1335291158 >> v1:172.21.15.121:6789/0 conn(0x7fde900acd00 legacy=0x7fde901536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:36.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.897+0000 7fde95504700 1 -- 172.21.15.121:0/1335291158 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde900c64a0 con 0x7fde900c5c70 2023-11-15T02:59:36.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.897+0000 7fde95504700 1 --2- 172.21.15.121:0/1335291158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde900c5c70 0x7fde90153dd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7fde80009760 tx=0x7fde80009060).ready entity=mon.0 client_cookie=174cdc0a10d11271 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:36.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.897+0000 7fde94d03700 1 -- 172.21.15.121:0/1335291158 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8000d4a0 con 0x7fde900c5c70 2023-11-15T02:59:36.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.897+0000 7fde94d03700 1 -- 172.21.15.121:0/1335291158 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8000d600 con 0x7fde900c5c70 2023-11-15T02:59:36.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.897+0000 7fde94d03700 1 -- 172.21.15.121:0/1335291158 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8000d980 con 0x7fde900c5c70 2023-11-15T02:59:36.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.901+0000 7fde96d07700 1 -- 172.21.15.121:0/1335291158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde900c5c70 msgr2=0x7fde90153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:36.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.901+0000 7fde96d07700 1 --2- 172.21.15.121:0/1335291158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde900c5c70 0x7fde90153dd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7fde80009760 tx=0x7fde80009060).stop 2023-11-15T02:59:36.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.901+0000 7fde96d07700 1 -- 172.21.15.121:0/1335291158 shutdown_connections 2023-11-15T02:59:36.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.901+0000 7fde96d07700 1 --2- 172.21.15.121:0/1335291158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde90066330 0x7fde900ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:36.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.901+0000 7fde96d07700 1 --2- 172.21.15.121:0/1335291158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde900c5c70 0x7fde90153dd0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:36.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.901+0000 7fde96d07700 1 -- 172.21.15.121:0/1335291158 >> 172.21.15.121:0/1335291158 conn(0x7fde90063fc0 msgr2=0x7fde90067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:36.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.901+0000 7fde96d07700 1 -- 172.21.15.121:0/1335291158 shutdown_connections 2023-11-15T02:59:36.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.901+0000 7fde96d07700 1 -- 172.21.15.121:0/1335291158 wait complete. 2023-11-15T02:59:36.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.901+0000 7fde96d07700 1 Processor -- start 2023-11-15T02:59:36.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.901+0000 7fde96d07700 1 -- start start 2023-11-15T02:59:36.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.901+0000 7fde96d07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde90066330 0x7fde901422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:36.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.901+0000 7fde96d07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde900acd00 0x7fde90158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:36.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.901+0000 7fde96d07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde900c5c70 0x7fde90158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:36.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.901+0000 7fde96d07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde90159470 con 0x7fde90066330 2023-11-15T02:59:36.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.901+0000 7fde96d07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde901652b0 con 0x7fde900acd00 2023-11-15T02:59:36.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.901+0000 7fde96d07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde900cac10 con 0x7fde900c5c70 2023-11-15T02:59:36.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.905+0000 7fde95d05700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde90066330 0x7fde901422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:36.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.905+0000 7fde95d05700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde90066330 0x7fde901422f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39258/0 (socket says 172.21.15.121:39258) 2023-11-15T02:59:36.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.905+0000 7fde95d05700 1 -- 172.21.15.121:0/854597680 learned_addr learned my addr 172.21.15.121:0/854597680 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:36.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.905+0000 7fde95504700 1 --2- 172.21.15.121:0/854597680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde900acd00 0x7fde90158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:36.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.905+0000 7fde96506700 1 --2- 172.21.15.121:0/854597680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde900c5c70 0x7fde90158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:36.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.905+0000 7fde95d05700 1 -- 172.21.15.121:0/854597680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde900c5c70 msgr2=0x7fde90158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:36.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.905+0000 7fde95d05700 1 --2- 172.21.15.121:0/854597680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde900c5c70 0x7fde90158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:36.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.905+0000 7fde95d05700 1 -- 172.21.15.121:0/854597680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde900acd00 msgr2=0x7fde90158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:36.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.905+0000 7fde95d05700 1 --2- 172.21.15.121:0/854597680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde900acd00 0x7fde90158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:36.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.905+0000 7fde95d05700 1 -- 172.21.15.121:0/854597680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde800092a0 con 0x7fde90066330 2023-11-15T02:59:36.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.905+0000 7fde95d05700 1 --2- 172.21.15.121:0/854597680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde90066330 0x7fde901422f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7fde8c00c4e0 tx=0x7fde8c007b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:36.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.905+0000 7fde85ffb700 1 -- 172.21.15.121:0/854597680 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c00ce50 con 0x7fde90066330 2023-11-15T02:59:36.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.905+0000 7fde85ffb700 1 -- 172.21.15.121:0/854597680 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8c0072e0 con 0x7fde90066330 2023-11-15T02:59:36.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.905+0000 7fde85ffb700 1 -- 172.21.15.121:0/854597680 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c005670 con 0x7fde90066330 2023-11-15T02:59:36.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.905+0000 7fde96d07700 1 -- 172.21.15.121:0/854597680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde900ca930 con 0x7fde90066330 2023-11-15T02:59:36.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.905+0000 7fde96d07700 1 -- 172.21.15.121:0/854597680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde901546f0 con 0x7fde90066330 2023-11-15T02:59:36.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.909+0000 7fde96d07700 1 -- 172.21.15.121:0/854597680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9006bcf0 con 0x7fde90066330 2023-11-15T02:59:36.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.909+0000 7fde85ffb700 1 -- 172.21.15.121:0/854597680 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fde8c005a60 con 0x7fde90066330 2023-11-15T02:59:36.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.909+0000 7fde85ffb700 1 --2- 172.21.15.121:0/854597680 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fde7405a510 0x7fde7405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:36.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.909+0000 7fde85ffb700 1 -- 172.21.15.121:0/854597680 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fde8c073fa0 con 0x7fde90066330 2023-11-15T02:59:36.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.909+0000 7fde95504700 1 --2- 172.21.15.121:0/854597680 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fde7405a510 0x7fde7405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:36.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.913+0000 7fde95504700 1 --2- 172.21.15.121:0/854597680 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fde7405a510 0x7fde7405c9d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7fde90165260 tx=0x7fde80016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:36.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:36.917+0000 7fde85ffb700 1 -- 172.21.15.121:0/854597680 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fde8c0476c0 con 0x7fde90066330 2023-11-15T02:59:37.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.113+0000 7fde96d07700 1 -- 172.21.15.121:0/854597680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fde900c61c0 con 0x7fde90066330 2023-11-15T02:59:37.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.709+0000 7fde85ffb700 1 -- 172.21.15.121:0/854597680 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v45) v1 ==== 122+0+0 (secure 0 0 0) 0x7fde8c04ace0 con 0x7fde90066330 2023-11-15T02:59:37.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.713+0000 7fde96d07700 1 -- 172.21.15.121:0/854597680 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fde7405a510 msgr2=0x7fde7405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:37.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.713+0000 7fde96d07700 1 --2- 172.21.15.121:0/854597680 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fde7405a510 0x7fde7405c9d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7fde90165260 tx=0x7fde80016000).stop 2023-11-15T02:59:37.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.713+0000 7fde96d07700 1 -- 172.21.15.121:0/854597680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde90066330 msgr2=0x7fde901422f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:37.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.713+0000 7fde96d07700 1 --2- 172.21.15.121:0/854597680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde90066330 0x7fde901422f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7fde8c00c4e0 tx=0x7fde8c007b00).stop 2023-11-15T02:59:37.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.713+0000 7fde96d07700 1 -- 172.21.15.121:0/854597680 shutdown_connections 2023-11-15T02:59:37.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.713+0000 7fde96d07700 1 --2- 172.21.15.121:0/854597680 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fde7405a510 0x7fde7405c9d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:37.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.713+0000 7fde96d07700 1 --2- 172.21.15.121:0/854597680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde900c5c70 0x7fde90158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:37.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.713+0000 7fde96d07700 1 --2- 172.21.15.121:0/854597680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde900acd00 0x7fde90158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:37.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.713+0000 7fde96d07700 1 --2- 172.21.15.121:0/854597680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde90066330 0x7fde901422f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:37.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.713+0000 7fde96d07700 1 -- 172.21.15.121:0/854597680 >> 172.21.15.121:0/854597680 conn(0x7fde90063fc0 msgr2=0x7fde901528e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:37.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.713+0000 7fde96d07700 1 -- 172.21.15.121:0/854597680 shutdown_connections 2023-11-15T02:59:37.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.713+0000 7fde96d07700 1 -- 172.21.15.121:0/854597680 wait complete. 2023-11-15T02:59:37.742 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-15T02:59:37.743 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:59:37.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.897+0000 7f7489e1b700 1 Processor -- start 2023-11-15T02:59:37.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.897+0000 7f7489e1b700 1 -- start start 2023-11-15T02:59:37.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.897+0000 7f7489e1b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f74840aa240 0x7f7484067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:37.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.897+0000 7f7489e1b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7484067f50 0x7f7484065d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:37.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.897+0000 7f7489e1b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74840cb5c0 con 0x7f7484153bb0 2023-11-15T02:59:37.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.897+0000 7f7489e1b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74840aa6f0 con 0x7f7484067f50 2023-11-15T02:59:37.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.897+0000 7f7489e1b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74840c5900 con 0x7f74840aa240 2023-11-15T02:59:37.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.897+0000 7f74837fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f7484153bb0 0x7f74840a9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51794/0 (socket says 172.21.15.121:51794) 2023-11-15T02:59:37.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.901+0000 7f74837fe700 1 -- 172.21.15.121:0/3344953279 learned_addr learned my addr 172.21.15.121:0/3344953279 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:37.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.901+0000 7f7483fff700 1 --2- 172.21.15.121:0/3344953279 >> v2:172.21.15.121:3300/0 conn(0x7f7484067f50 0x7f7484065d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:37.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.901+0000 7f7482ffd700 1 --2- 172.21.15.121:0/3344953279 >> v2:172.21.15.190:3300/0 conn(0x7f74840aa240 0x7f7484067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:37.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.901+0000 7f7483fff700 1 -- 172.21.15.121:0/3344953279 >> v2:172.21.15.190:3300/0 conn(0x7f74840aa240 msgr2=0x7f7484067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:37.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.901+0000 7f7483fff700 1 --2- 172.21.15.121:0/3344953279 >> v2:172.21.15.190:3300/0 conn(0x7f74840aa240 0x7f7484067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:37.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.901+0000 7f7483fff700 1 -- 172.21.15.121:0/3344953279 >> v1:172.21.15.121:6789/0 conn(0x7f7484153bb0 legacy=0x7f74840a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:37.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.901+0000 7f7483fff700 1 -- 172.21.15.121:0/3344953279 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74840c9ca0 con 0x7f7484067f50 2023-11-15T02:59:37.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.901+0000 7f7483fff700 1 --2- 172.21.15.121:0/3344953279 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7484067f50 0x7f7484065d30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f74840c77f0 tx=0x7f747c009780).ready entity=mon.0 client_cookie=de5b1b1b65385d05 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:37.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.901+0000 7f74827fc700 1 -- 172.21.15.121:0/3344953279 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c00e070 con 0x7f7484067f50 2023-11-15T02:59:37.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.901+0000 7f74827fc700 1 -- 172.21.15.121:0/3344953279 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c00bb60 con 0x7f7484067f50 2023-11-15T02:59:37.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.901+0000 7f74827fc700 1 -- 172.21.15.121:0/3344953279 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c012410 con 0x7f7484067f50 2023-11-15T02:59:37.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.901+0000 7f7489e1b700 1 -- 172.21.15.121:0/3344953279 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7484067f50 msgr2=0x7f7484065d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:37.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.901+0000 7f7489e1b700 1 --2- 172.21.15.121:0/3344953279 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7484067f50 0x7f7484065d30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f74840c77f0 tx=0x7f747c009780).stop 2023-11-15T02:59:37.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.901+0000 7f7489e1b700 1 -- 172.21.15.121:0/3344953279 shutdown_connections 2023-11-15T02:59:37.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.901+0000 7f7489e1b700 1 --2- 172.21.15.121:0/3344953279 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7484067f50 0x7f7484065d30 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:37.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.901+0000 7f7489e1b700 1 --2- 172.21.15.121:0/3344953279 >> v2:172.21.15.190:3300/0 conn(0x7f74840aa240 0x7f7484067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:37.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.901+0000 7f7489e1b700 1 -- 172.21.15.121:0/3344953279 >> 172.21.15.121:0/3344953279 conn(0x7f7484058c50 msgr2=0x7f7484059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:37.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.901+0000 7f7489e1b700 1 -- 172.21.15.121:0/3344953279 shutdown_connections 2023-11-15T02:59:37.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.901+0000 7f7489e1b700 1 -- 172.21.15.121:0/3344953279 wait complete. 2023-11-15T02:59:37.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.905+0000 7f7489e1b700 1 Processor -- start 2023-11-15T02:59:37.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.905+0000 7f7489e1b700 1 -- start start 2023-11-15T02:59:37.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.905+0000 7f7489e1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7484067f50 0x7f74841564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:37.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.905+0000 7f7489e1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74840aa240 0x7f74840abec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:37.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.905+0000 7f7489e1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7484153bb0 0x7f74840ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:37.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.905+0000 7f7489e1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74840ac8a0 con 0x7f7484067f50 2023-11-15T02:59:37.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.905+0000 7f7489e1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7484154d00 con 0x7f74840aa240 2023-11-15T02:59:37.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.905+0000 7f7489e1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7484154130 con 0x7f7484153bb0 2023-11-15T02:59:37.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.905+0000 7f74837fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7484067f50 0x7f74841564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:37.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.905+0000 7f74837fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7484067f50 0x7f74841564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39278/0 (socket says 172.21.15.121:39278) 2023-11-15T02:59:37.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.905+0000 7f74837fe700 1 -- 172.21.15.121:0/3949892239 learned_addr learned my addr 172.21.15.121:0/3949892239 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:37.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.909+0000 7f74837fe700 1 -- 172.21.15.121:0/3949892239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7484153bb0 msgr2=0x7f74840ac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:37.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.909+0000 7f7483fff700 1 --2- 172.21.15.121:0/3949892239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7484153bb0 0x7f74840ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:37.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.909+0000 7f7482ffd700 1 --2- 172.21.15.121:0/3949892239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74840aa240 0x7f74840abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:37.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.909+0000 7f74837fe700 1 --2- 172.21.15.121:0/3949892239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7484153bb0 0x7f74840ac3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:37.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.909+0000 7f74837fe700 1 -- 172.21.15.121:0/3949892239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74840aa240 msgr2=0x7f74840abec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:37.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.909+0000 7f74837fe700 1 --2- 172.21.15.121:0/3949892239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74840aa240 0x7f74840abec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:37.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.909+0000 7f74837fe700 1 -- 172.21.15.121:0/3949892239 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c009a20 con 0x7f7484067f50 2023-11-15T02:59:37.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.909+0000 7f7483fff700 1 --2- 172.21.15.121:0/3949892239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7484153bb0 0x7f74840ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:37.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.909+0000 7f7482ffd700 1 --2- 172.21.15.121:0/3949892239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74840aa240 0x7f74840abec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:37.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.909+0000 7f74837fe700 1 --2- 172.21.15.121:0/3949892239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7484067f50 0x7f74841564f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f7474001eb0 tx=0x7f7474007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:37.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.909+0000 7f746b7fe700 1 -- 172.21.15.121:0/3949892239 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74740072e0 con 0x7f7484067f50 2023-11-15T02:59:37.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.909+0000 7f746b7fe700 1 -- 172.21.15.121:0/3949892239 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7474007440 con 0x7f7484067f50 2023-11-15T02:59:37.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.909+0000 7f746b7fe700 1 -- 172.21.15.121:0/3949892239 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74740057a0 con 0x7f7484067f50 2023-11-15T02:59:37.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.909+0000 7f7489e1b700 1 -- 172.21.15.121:0/3949892239 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74841543d0 con 0x7f7484067f50 2023-11-15T02:59:37.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.909+0000 7f7489e1b700 1 -- 172.21.15.121:0/3949892239 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7484154990 con 0x7f7484067f50 2023-11-15T02:59:37.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.909+0000 7f7489e1b700 1 -- 172.21.15.121:0/3949892239 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748406c150 con 0x7f7484067f50 2023-11-15T02:59:37.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.909+0000 7f746b7fe700 1 -- 172.21.15.121:0/3949892239 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7474005b10 con 0x7f7484067f50 2023-11-15T02:59:37.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.913+0000 7f746b7fe700 1 --2- 172.21.15.121:0/3949892239 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f747005a560 0x7f747005ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:37.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.913+0000 7f746b7fe700 1 -- 172.21.15.121:0/3949892239 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7474073f40 con 0x7f7484067f50 2023-11-15T02:59:37.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.913+0000 7f7482ffd700 1 --2- 172.21.15.121:0/3949892239 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f747005a560 0x7f747005ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:37.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.913+0000 7f7482ffd700 1 --2- 172.21.15.121:0/3949892239 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f747005a560 0x7f747005ca20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f7478006d80 tx=0x7f7478008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:37.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:37.917+0000 7f746b7fe700 1 -- 172.21.15.121:0/3949892239 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f74740477c0 con 0x7f7484067f50 2023-11-15T02:59:38.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.113+0000 7f7489e1b700 1 -- 172.21.15.121:0/3949892239 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74840c83c0 con 0x7f7484067f50 2023-11-15T02:59:38.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.117+0000 7f746b7fe700 1 -- 172.21.15.121:0/3949892239 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4502 (secure 0 0 0) 0x7f747404ade0 con 0x7f7484067f50 2023-11-15T02:59:38.123 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:38.123 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":45,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3462858472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3462858472},{"type":"v1","addr":"172.21.15.190:6833","nonce":3462858472}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4042390087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4042390087},{"type":"v1","addr":"172.21.15.190:6835","nonce":4042390087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5907,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2239114529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2239114529},{"type":"v1","addr":"172.21.15.121:6837","nonce":2239114529}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:59:34.667864+0000","modified":"2023-11-15T02:59:37.695764+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5910":{"gid":5910,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2394175138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":2394175138},{"type":"v1","addr":"172.21.15.121:6836","nonce":2394175138}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-15T02:59:38.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.121+0000 7f7489e1b700 1 -- 172.21.15.121:0/3949892239 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f747005a560 msgr2=0x7f747005ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:38.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.121+0000 7f7489e1b700 1 --2- 172.21.15.121:0/3949892239 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f747005a560 0x7f747005ca20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f7478006d80 tx=0x7f7478008040).stop 2023-11-15T02:59:38.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.121+0000 7f7489e1b700 1 -- 172.21.15.121:0/3949892239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7484067f50 msgr2=0x7f74841564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:38.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.121+0000 7f7489e1b700 1 --2- 172.21.15.121:0/3949892239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7484067f50 0x7f74841564f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f7474001eb0 tx=0x7f7474007e50).stop 2023-11-15T02:59:38.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.121+0000 7f7489e1b700 1 -- 172.21.15.121:0/3949892239 shutdown_connections 2023-11-15T02:59:38.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.121+0000 7f7489e1b700 1 --2- 172.21.15.121:0/3949892239 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f747005a560 0x7f747005ca20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:38.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.121+0000 7f7489e1b700 1 --2- 172.21.15.121:0/3949892239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7484153bb0 0x7f74840ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:38.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.121+0000 7f7489e1b700 1 --2- 172.21.15.121:0/3949892239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74840aa240 0x7f74840abec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:38.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.121+0000 7f7489e1b700 1 --2- 172.21.15.121:0/3949892239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7484067f50 0x7f74841564f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:38.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.121+0000 7f7489e1b700 1 -- 172.21.15.121:0/3949892239 >> 172.21.15.121:0/3949892239 conn(0x7f7484058c50 msgr2=0x7f74840c6870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:38.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.121+0000 7f7489e1b700 1 -- 172.21.15.121:0/3949892239 shutdown_connections 2023-11-15T02:59:38.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.121+0000 7f7489e1b700 1 -- 172.21.15.121:0/3949892239 wait complete. 2023-11-15T02:59:38.131 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 45 2023-11-15T02:59:38.149 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T02:59:38.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.297+0000 7f535def1700 1 Processor -- start 2023-11-15T02:59:38.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.297+0000 7f535def1700 1 -- start start 2023-11-15T02:59:38.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.301+0000 7f535def1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5358154ca0 0x7f5358155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:38.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.301+0000 7f535def1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53580c4c30 con 0x7f53581562f0 2023-11-15T02:59:38.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.301+0000 7f535def1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53580a6f50 con 0x7f53580a57c0 2023-11-15T02:59:38.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.301+0000 7f535def1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53580a8280 con 0x7f5358154ca0 2023-11-15T02:59:38.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.301+0000 7f534effd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f53581562f0 0x7f53580a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51806/0 (socket says 172.21.15.121:51806) 2023-11-15T02:59:38.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.301+0000 7f534effd700 1 -- 172.21.15.121:0/1883460151 learned_addr learned my addr 172.21.15.121:0/1883460151 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:38.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.301+0000 7f53577fe700 1 --2- 172.21.15.121:0/1883460151 >> v2:172.21.15.190:3300/0 conn(0x7f5358154ca0 0x7f5358155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:38.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.301+0000 7f5356ffd700 1 -- 172.21.15.121:0/1883460151 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 676802697 0 0) 0x7f53580c4c30 con 0x7f53581562f0 2023-11-15T02:59:38.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.301+0000 7f5356ffd700 1 -- 172.21.15.121:0/1883460151 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53440033f0 con 0x7f53581562f0 2023-11-15T02:59:38.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.301+0000 7f5356ffd700 1 -- 172.21.15.121:0/1883460151 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871978666 0 0) 0x7f53580a6f50 con 0x7f53580a57c0 2023-11-15T02:59:38.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.301+0000 7f5356ffd700 1 -- 172.21.15.121:0/1883460151 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53580c4c30 con 0x7f53580a57c0 2023-11-15T02:59:38.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.301+0000 7f53577fe700 1 -- 172.21.15.121:0/1883460151 >> v1:172.21.15.190:6789/0 conn(0x7f53580a57c0 legacy=0x7f53580c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:38.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.301+0000 7f53577fe700 1 -- 172.21.15.121:0/1883460151 >> v1:172.21.15.121:6789/0 conn(0x7f53581562f0 legacy=0x7f53580a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:38.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.301+0000 7f53577fe700 1 -- 172.21.15.121:0/1883460151 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53580a7230 con 0x7f5358154ca0 2023-11-15T02:59:38.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.305+0000 7f53577fe700 1 --2- 172.21.15.121:0/1883460151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5358154ca0 0x7f5358155080 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f5348009760 tx=0x7f5348009060).ready entity=mon.1 client_cookie=34327fbaedf655e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:38.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.305+0000 7f5356ffd700 1 -- 172.21.15.121:0/1883460151 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534800d4a0 con 0x7f5358154ca0 2023-11-15T02:59:38.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.305+0000 7f5356ffd700 1 -- 172.21.15.121:0/1883460151 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534800d600 con 0x7f5358154ca0 2023-11-15T02:59:38.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.305+0000 7f5356ffd700 1 -- 172.21.15.121:0/1883460151 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534800d980 con 0x7f5358154ca0 2023-11-15T02:59:38.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.305+0000 7f535def1700 1 -- 172.21.15.121:0/1883460151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5358154ca0 msgr2=0x7f5358155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:38.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.305+0000 7f535def1700 1 --2- 172.21.15.121:0/1883460151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5358154ca0 0x7f5358155080 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f5348009760 tx=0x7f5348009060).stop 2023-11-15T02:59:38.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.305+0000 7f535def1700 1 -- 172.21.15.121:0/1883460151 shutdown_connections 2023-11-15T02:59:38.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.305+0000 7f535def1700 1 --2- 172.21.15.121:0/1883460151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5358154ca0 0x7f5358155080 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:38.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.305+0000 7f535def1700 1 -- 172.21.15.121:0/1883460151 >> 172.21.15.121:0/1883460151 conn(0x7f5358062190 msgr2=0x7f53580625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:38.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.305+0000 7f535def1700 1 -- 172.21.15.121:0/1883460151 shutdown_connections 2023-11-15T02:59:38.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.305+0000 7f535def1700 1 -- 172.21.15.121:0/1883460151 wait complete. 2023-11-15T02:59:38.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f535def1700 1 Processor -- start 2023-11-15T02:59:38.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f535def1700 1 -- start start 2023-11-15T02:59:38.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f535def1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53580a57c0 0x7f535814bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:38.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f535def1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5358154ca0 0x7f53581478a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:38.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f535def1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53581562f0 0x7f53580aa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:38.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f535def1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5358145a30 con 0x7f53581562f0 2023-11-15T02:59:38.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f535def1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53580c1770 con 0x7f5358154ca0 2023-11-15T02:59:38.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f535def1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53580c2510 con 0x7f53580a57c0 2023-11-15T02:59:38.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f5357fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53581562f0 0x7f53580aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:38.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f5357fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53581562f0 0x7f53580aa800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39290/0 (socket says 172.21.15.121:39290) 2023-11-15T02:59:38.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f5357fff700 1 -- 172.21.15.121:0/1512320273 learned_addr learned my addr 172.21.15.121:0/1512320273 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:38.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f534effd700 1 --2- 172.21.15.121:0/1512320273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5358154ca0 0x7f53581478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:38.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f53577fe700 1 --2- 172.21.15.121:0/1512320273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53580a57c0 0x7f535814bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:38.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f5357fff700 1 -- 172.21.15.121:0/1512320273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53580a57c0 msgr2=0x7f535814bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:38.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f5357fff700 1 --2- 172.21.15.121:0/1512320273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53580a57c0 0x7f535814bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:38.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f5357fff700 1 -- 172.21.15.121:0/1512320273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5358154ca0 msgr2=0x7f53581478a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:38.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f5357fff700 1 --2- 172.21.15.121:0/1512320273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5358154ca0 0x7f53581478a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:38.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f5357fff700 1 -- 172.21.15.121:0/1512320273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53480092a0 con 0x7f53581562f0 2023-11-15T02:59:38.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f534effd700 1 --2- 172.21.15.121:0/1512320273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5358154ca0 0x7f53581478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:38.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f5357fff700 1 --2- 172.21.15.121:0/1512320273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53581562f0 0x7f53580aa800 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f535000c710 tx=0x7f535000cad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:38.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f534ffff700 1 -- 172.21.15.121:0/1512320273 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350010510 con 0x7f53581562f0 2023-11-15T02:59:38.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f534ffff700 1 -- 172.21.15.121:0/1512320273 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5350010670 con 0x7f53581562f0 2023-11-15T02:59:38.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f535def1700 1 -- 172.21.15.121:0/1512320273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535814c350 con 0x7f53581562f0 2023-11-15T02:59:38.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f534ffff700 1 -- 172.21.15.121:0/1512320273 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53500109f0 con 0x7f53581562f0 2023-11-15T02:59:38.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.309+0000 7f535def1700 1 -- 172.21.15.121:0/1512320273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53580c1d10 con 0x7f53581562f0 2023-11-15T02:59:38.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.313+0000 7f535def1700 1 -- 172.21.15.121:0/1512320273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535806c150 con 0x7f53581562f0 2023-11-15T02:59:38.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.313+0000 7f534ffff700 1 -- 172.21.15.121:0/1512320273 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f535001b400 con 0x7f53581562f0 2023-11-15T02:59:38.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.317+0000 7f534ffff700 1 --2- 172.21.15.121:0/1512320273 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f533805a510 0x7f533805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:38.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.317+0000 7f534ffff700 1 -- 172.21.15.121:0/1512320273 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5350074680 con 0x7f53581562f0 2023-11-15T02:59:38.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.317+0000 7f53577fe700 1 --2- 172.21.15.121:0/1512320273 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f533805a510 0x7f533805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:38.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.317+0000 7f53577fe700 1 --2- 172.21.15.121:0/1512320273 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f533805a510 0x7f533805c9d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f5348006d60 tx=0x7f5348016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:38.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.321+0000 7f534ffff700 1 -- 172.21.15.121:0/1512320273 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5350047e00 con 0x7f53581562f0 2023-11-15T02:59:38.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.489+0000 7f535def1700 1 -- 172.21.15.121:0/1512320273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53580c4500 con 0x7f53581562f0 2023-11-15T02:59:38.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.493+0000 7f534ffff700 1 -- 172.21.15.121:0/1512320273 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16978 (secure 0 0 0) 0x7f535004b420 con 0x7f53581562f0 2023-11-15T02:59:38.499 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:38.499 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":61,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T02:59:35.673421+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:59:32.734725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-15T02:59:33.665552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T02:59:38.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.493+0000 7f535def1700 1 -- 172.21.15.121:0/1512320273 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f533805a510 msgr2=0x7f533805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:38.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.493+0000 7f535def1700 1 --2- 172.21.15.121:0/1512320273 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f533805a510 0x7f533805c9d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f5348006d60 tx=0x7f5348016000).stop 2023-11-15T02:59:38.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.493+0000 7f535def1700 1 -- 172.21.15.121:0/1512320273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53581562f0 msgr2=0x7f53580aa800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:38.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.493+0000 7f535def1700 1 --2- 172.21.15.121:0/1512320273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53581562f0 0x7f53580aa800 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f535000c710 tx=0x7f535000cad0).stop 2023-11-15T02:59:38.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.493+0000 7f535def1700 1 -- 172.21.15.121:0/1512320273 shutdown_connections 2023-11-15T02:59:38.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.493+0000 7f535def1700 1 --2- 172.21.15.121:0/1512320273 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f533805a510 0x7f533805c9d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:38.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.493+0000 7f535def1700 1 --2- 172.21.15.121:0/1512320273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53581562f0 0x7f53580aa800 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:38.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.493+0000 7f535def1700 1 --2- 172.21.15.121:0/1512320273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5358154ca0 0x7f53581478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:38.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.493+0000 7f535def1700 1 --2- 172.21.15.121:0/1512320273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53580a57c0 0x7f535814bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:38.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.493+0000 7f535def1700 1 -- 172.21.15.121:0/1512320273 >> 172.21.15.121:0/1512320273 conn(0x7f5358062190 msgr2=0x7f535814f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:38.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.493+0000 7f535def1700 1 -- 172.21.15.121:0/1512320273 shutdown_connections 2023-11-15T02:59:38.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.493+0000 7f535def1700 1 -- 172.21.15.121:0/1512320273 wait complete. 2023-11-15T02:59:38.526 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-15T02:59:38.527 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T02:59:38.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.681+0000 7fab73f69700 1 Processor -- start 2023-11-15T02:59:38.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.681+0000 7fab73f69700 1 -- start start 2023-11-15T02:59:38.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.685+0000 7fab73f69700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fab6c0c6ce0 0x7fab6c159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:38.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.685+0000 7fab73f69700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6c0c3fa0 con 0x7fab6c0a5510 2023-11-15T02:59:38.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.685+0000 7fab73f69700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6c159630 con 0x7fab6c152780 2023-11-15T02:59:38.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.685+0000 7fab73f69700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6c0c30b0 con 0x7fab6c0c6ce0 2023-11-15T02:59:38.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.685+0000 7fab71504700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fab6c0c6ce0 0x7fab6c159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:38.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.685+0000 7fab71504700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fab6c0c6ce0 0x7fab6c159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39302/0 (socket says 172.21.15.121:39302) 2023-11-15T02:59:38.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.685+0000 7fab71504700 1 -- 172.21.15.121:0/1112822258 learned_addr learned my addr 172.21.15.121:0/1112822258 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:38.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.685+0000 7fab71504700 1 -- 172.21.15.121:0/1112822258 >> v1:172.21.15.190:6789/0 conn(0x7fab6c152780 legacy=0x7fab6c0c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:38.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.685+0000 7fab71504700 1 -- 172.21.15.121:0/1112822258 >> v1:172.21.15.121:6789/0 conn(0x7fab6c0a5510 legacy=0x7fab6c0a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:38.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.685+0000 7fab71504700 1 -- 172.21.15.121:0/1112822258 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c0c3390 con 0x7fab6c0c6ce0 2023-11-15T02:59:38.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.685+0000 7fab70d03700 1 -- 172.21.15.121:0/1112822258 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1826455341 0 0) 0x7fab6c0c3fa0 con 0x7fab6c0a5510 2023-11-15T02:59:38.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.685+0000 7fab71504700 1 --2- 172.21.15.121:0/1112822258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6c0c6ce0 0x7fab6c159140 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fab5c006fd0 tx=0x7fab5c0090f0).ready entity=mon.0 client_cookie=83ea3e9f7fc40b76 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:38.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.685+0000 7fab70d03700 1 -- 172.21.15.121:0/1112822258 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c00b600 con 0x7fab6c0c6ce0 2023-11-15T02:59:38.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.685+0000 7fab70d03700 1 -- 172.21.15.121:0/1112822258 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c00b760 con 0x7fab6c0c6ce0 2023-11-15T02:59:38.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.685+0000 7fab70d03700 1 -- 172.21.15.121:0/1112822258 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c00bae0 con 0x7fab6c0c6ce0 2023-11-15T02:59:38.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.685+0000 7fab73f69700 1 -- 172.21.15.121:0/1112822258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6c0c6ce0 msgr2=0x7fab6c159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:38.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.685+0000 7fab73f69700 1 --2- 172.21.15.121:0/1112822258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6c0c6ce0 0x7fab6c159140 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fab5c006fd0 tx=0x7fab5c0090f0).stop 2023-11-15T02:59:38.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.689+0000 7fab73f69700 1 -- 172.21.15.121:0/1112822258 shutdown_connections 2023-11-15T02:59:38.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.689+0000 7fab73f69700 1 --2- 172.21.15.121:0/1112822258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6c0c6ce0 0x7fab6c159140 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:38.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.689+0000 7fab73f69700 1 -- 172.21.15.121:0/1112822258 >> 172.21.15.121:0/1112822258 conn(0x7fab6c062190 msgr2=0x7fab6c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:38.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.689+0000 7fab73f69700 1 -- 172.21.15.121:0/1112822258 shutdown_connections 2023-11-15T02:59:38.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.689+0000 7fab73f69700 1 -- 172.21.15.121:0/1112822258 wait complete. 2023-11-15T02:59:38.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.689+0000 7fab73f69700 1 Processor -- start 2023-11-15T02:59:38.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.689+0000 7fab73f69700 1 -- start start 2023-11-15T02:59:38.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab73f69700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab6c0a5510 0x7fab6c1507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:38.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab73f69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab6c0c6ce0 0x7fab6c156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:38.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab73f69700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6c152780 0x7fab6c154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:38.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab73f69700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c1549c0 con 0x7fab6c152780 2023-11-15T02:59:38.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab73f69700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c153db0 con 0x7fab6c0a5510 2023-11-15T02:59:38.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab73f69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c154090 con 0x7fab6c0c6ce0 2023-11-15T02:59:38.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab72506700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6c152780 0x7fab6c154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:38.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab72506700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6c152780 0x7fab6c154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39312/0 (socket says 172.21.15.121:39312) 2023-11-15T02:59:38.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab72506700 1 -- 172.21.15.121:0/2198739328 learned_addr learned my addr 172.21.15.121:0/2198739328 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:38.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab71d05700 1 --2- 172.21.15.121:0/2198739328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab6c0a5510 0x7fab6c1507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:38.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab71504700 1 --2- 172.21.15.121:0/2198739328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab6c0c6ce0 0x7fab6c156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:38.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab72506700 1 -- 172.21.15.121:0/2198739328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab6c0c6ce0 msgr2=0x7fab6c156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:38.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab72506700 1 --2- 172.21.15.121:0/2198739328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab6c0c6ce0 0x7fab6c156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:38.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab72506700 1 -- 172.21.15.121:0/2198739328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab6c0a5510 msgr2=0x7fab6c1507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:38.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab72506700 1 --2- 172.21.15.121:0/2198739328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab6c0a5510 0x7fab6c1507b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:38.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab72506700 1 -- 172.21.15.121:0/2198739328 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c009370 con 0x7fab6c152780 2023-11-15T02:59:38.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab72506700 1 --2- 172.21.15.121:0/2198739328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6c152780 0x7fab6c154e50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fab6400c490 tx=0x7fab64007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:38.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab61ffb700 1 -- 172.21.15.121:0/2198739328 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6400ce90 con 0x7fab6c152780 2023-11-15T02:59:38.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab61ffb700 1 -- 172.21.15.121:0/2198739328 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6400f470 con 0x7fab6c152780 2023-11-15T02:59:38.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab73f69700 1 -- 172.21.15.121:0/2198739328 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6c154370 con 0x7fab6c152780 2023-11-15T02:59:38.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab61ffb700 1 -- 172.21.15.121:0/2198739328 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6400f810 con 0x7fab6c152780 2023-11-15T02:59:38.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.693+0000 7fab73f69700 1 -- 172.21.15.121:0/2198739328 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c150ca0 con 0x7fab6c152780 2023-11-15T02:59:38.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.697+0000 7fab73f69700 1 -- 172.21.15.121:0/2198739328 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6c154b30 con 0x7fab6c152780 2023-11-15T02:59:38.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.697+0000 7fab61ffb700 1 -- 172.21.15.121:0/2198739328 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fab6400fc00 con 0x7fab6c152780 2023-11-15T02:59:38.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.697+0000 7fab61ffb700 1 --2- 172.21.15.121:0/2198739328 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fab5005a510 0x7fab5005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:38.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.697+0000 7fab61ffb700 1 -- 172.21.15.121:0/2198739328 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fab64073c80 con 0x7fab6c152780 2023-11-15T02:59:38.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.697+0000 7fab71d05700 1 --2- 172.21.15.121:0/2198739328 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fab5005a510 0x7fab5005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:38.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.697+0000 7fab71d05700 1 --2- 172.21.15.121:0/2198739328 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fab5005a510 0x7fab5005c9d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fab68004290 tx=0x7fab680014c0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:38.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.701+0000 7fab61ffb700 1 -- 172.21.15.121:0/2198739328 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fab64047450 con 0x7fab6c152780 2023-11-15T02:59:38.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.861+0000 7fab73f69700 1 -- 172.21.15.121:0/2198739328 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fab6c0c3670 con 0x7fab5005a510 2023-11-15T02:59:38.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.873+0000 7fab61ffb700 1 -- 172.21.15.121:0/2198739328 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+355564 (secure 0 0 0) 0x7fab6c0c3670 con 0x7fab5005a510 2023-11-15T02:59:38.886 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:38.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.881+0000 7fab73f69700 1 -- 172.21.15.121:0/2198739328 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fab5005a510 msgr2=0x7fab5005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:38.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.881+0000 7fab73f69700 1 --2- 172.21.15.121:0/2198739328 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fab5005a510 0x7fab5005c9d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fab68004290 tx=0x7fab680014c0).stop 2023-11-15T02:59:38.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.881+0000 7fab73f69700 1 -- 172.21.15.121:0/2198739328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6c152780 msgr2=0x7fab6c154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:38.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.881+0000 7fab73f69700 1 --2- 172.21.15.121:0/2198739328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6c152780 0x7fab6c154e50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fab6400c490 tx=0x7fab64007bf0).stop 2023-11-15T02:59:38.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.881+0000 7fab73f69700 1 -- 172.21.15.121:0/2198739328 shutdown_connections 2023-11-15T02:59:38.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.881+0000 7fab73f69700 1 --2- 172.21.15.121:0/2198739328 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fab5005a510 0x7fab5005c9d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:38.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.881+0000 7fab73f69700 1 --2- 172.21.15.121:0/2198739328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6c152780 0x7fab6c154e50 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:38.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.881+0000 7fab73f69700 1 --2- 172.21.15.121:0/2198739328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab6c0c6ce0 0x7fab6c156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:38.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.881+0000 7fab73f69700 1 --2- 172.21.15.121:0/2198739328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab6c0a5510 0x7fab6c1507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:38.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.881+0000 7fab73f69700 1 -- 172.21.15.121:0/2198739328 >> 172.21.15.121:0/2198739328 conn(0x7fab6c062190 msgr2=0x7fab6c0a8980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:38.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.881+0000 7fab73f69700 1 -- 172.21.15.121:0/2198739328 shutdown_connections 2023-11-15T02:59:38.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:38.881+0000 7fab73f69700 1 -- 172.21.15.121:0/2198739328 wait complete. 2023-11-15T02:59:38.891 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T02:59:38.909 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2023-11-15T02:59:37.971314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":748,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":226,"acting":226,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382264,"kb_used_data":5496,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752592456,"statfs":{"total":773094113280,"available":770654674944,"internally_reserved":0,"allocated":5627904,"data_stored":2493914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12861,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-20,"num_read_kb":-23,"num_write":-72,"num_write_kb":-35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.466488"},"pg_stats":[{"pgid":"10.3c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.702569+0000","last_change":"2023-11-15T02:59:34.702569+0000","last_active":"2023-11-15T02:59:34.702569+0000","last_peered":"2023-11-15T02:59:34.702569+0000","last_clean":"2023-11-15T02:59:34.702569+0000","last_became_active":"2023-11-15T02:59:34.702272+0000","last_became_peered":"2023-11-15T02:59:34.702272+0000","last_unstale":"2023-11-15T02:59:34.702569+0000","last_undegraded":"2023-11-15T02:59:34.702569+0000","last_fullsized":"2023-11-15T02:59:34.702569+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.3f","version":"60'3","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.702378+0000","last_change":"2023-11-15T02:59:34.265036+0000","last_active":"2023-11-15T02:59:34.702378+0000","last_peered":"2023-11-15T02:59:34.702378+0000","last_clean":"2023-11-15T02:59:34.702378+0000","last_became_active":"2023-11-15T02:59:34.264631+0000","last_became_peered":"2023-11-15T02:59:34.264631+0000","last_unstale":"2023-11-15T02:59:34.702378+0000","last_undegraded":"2023-11-15T02:59:34.702378+0000","last_fullsized":"2023-11-15T02:59:34.702378+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.715077+0000","last_change":"2023-11-15T02:59:34.715077+0000","last_active":"2023-11-15T02:59:34.715077+0000","last_peered":"2023-11-15T02:59:34.715077+0000","last_clean":"2023-11-15T02:59:34.715077+0000","last_became_active":"2023-11-15T02:59:34.710308+0000","last_became_peered":"2023-11-15T02:59:34.710308+0000","last_unstale":"2023-11-15T02:59:34.715077+0000","last_undegraded":"2023-11-15T02:59:34.715077+0000","last_fullsized":"2023-11-15T02:59:34.715077+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.280938+0000","last_change":"2023-11-15T02:59:34.274806+0000","last_active":"2023-11-15T02:59:34.280938+0000","last_peered":"2023-11-15T02:59:34.280938+0000","last_clean":"2023-11-15T02:59:34.280938+0000","last_became_active":"2023-11-15T02:59:34.274536+0000","last_became_peered":"2023-11-15T02:59:34.274536+0000","last_unstale":"2023-11-15T02:59:34.280938+0000","last_undegraded":"2023-11-15T02:59:34.280938+0000","last_fullsized":"2023-11-15T02:59:34.280938+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.276406+0000","last_change":"2023-11-15T02:59:34.272252+0000","last_active":"2023-11-15T02:59:34.276406+0000","last_peered":"2023-11-15T02:59:34.276406+0000","last_clean":"2023-11-15T02:59:34.276406+0000","last_became_active":"2023-11-15T02:59:34.271979+0000","last_became_peered":"2023-11-15T02:59:34.271979+0000","last_unstale":"2023-11-15T02:59:34.276406+0000","last_undegraded":"2023-11-15T02:59:34.276406+0000","last_fullsized":"2023-11-15T02:59:34.276406+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.231018+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.694905+0000","last_change":"2023-11-15T02:59:34.263921+0000","last_active":"2023-11-15T02:59:34.694905+0000","last_peered":"2023-11-15T02:59:34.694905+0000","last_clean":"2023-11-15T02:59:34.694905+0000","last_became_active":"2023-11-15T02:59:34.261031+0000","last_became_peered":"2023-11-15T02:59:34.261031+0000","last_unstale":"2023-11-15T02:59:34.694905+0000","last_undegraded":"2023-11-15T02:59:34.694905+0000","last_fullsized":"2023-11-15T02:59:34.694905+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716845+0000","last_change":"2023-11-15T02:59:34.716845+0000","last_active":"2023-11-15T02:59:34.716845+0000","last_peered":"2023-11-15T02:59:34.716845+0000","last_clean":"2023-11-15T02:59:34.716845+0000","last_became_active":"2023-11-15T02:59:34.716558+0000","last_became_peered":"2023-11-15T02:59:34.716558+0000","last_unstale":"2023-11-15T02:59:34.716845+0000","last_undegraded":"2023-11-15T02:59:34.716845+0000","last_fullsized":"2023-11-15T02:59:34.716845+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.690603+0000","last_change":"2023-11-15T02:59:34.266240+0000","last_active":"2023-11-15T02:59:34.690603+0000","last_peered":"2023-11-15T02:59:34.690603+0000","last_clean":"2023-11-15T02:59:34.690603+0000","last_became_active":"2023-11-15T02:59:34.265188+0000","last_became_peered":"2023-11-15T02:59:34.265188+0000","last_unstale":"2023-11-15T02:59:34.690603+0000","last_undegraded":"2023-11-15T02:59:34.690603+0000","last_fullsized":"2023-11-15T02:59:34.690603+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.719027+0000","last_change":"2023-11-15T02:59:34.719027+0000","last_active":"2023-11-15T02:59:34.719027+0000","last_peered":"2023-11-15T02:59:34.719027+0000","last_clean":"2023-11-15T02:59:34.719027+0000","last_became_active":"2023-11-15T02:59:34.718717+0000","last_became_peered":"2023-11-15T02:59:34.718717+0000","last_unstale":"2023-11-15T02:59:34.719027+0000","last_undegraded":"2023-11-15T02:59:34.719027+0000","last_fullsized":"2023-11-15T02:59:34.719027+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.273500+0000","last_change":"2023-11-15T02:59:34.263881+0000","last_active":"2023-11-15T02:59:34.273500+0000","last_peered":"2023-11-15T02:59:34.273500+0000","last_clean":"2023-11-15T02:59:34.273500+0000","last_became_active":"2023-11-15T02:59:34.263571+0000","last_became_peered":"2023-11-15T02:59:34.263571+0000","last_unstale":"2023-11-15T02:59:34.273500+0000","last_undegraded":"2023-11-15T02:59:34.273500+0000","last_fullsized":"2023-11-15T02:59:34.273500+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.715365+0000","last_change":"2023-11-15T02:59:34.715365+0000","last_active":"2023-11-15T02:59:34.715365+0000","last_peered":"2023-11-15T02:59:34.715365+0000","last_clean":"2023-11-15T02:59:34.715365+0000","last_became_active":"2023-11-15T02:59:34.715043+0000","last_became_peered":"2023-11-15T02:59:34.715043+0000","last_unstale":"2023-11-15T02:59:34.715365+0000","last_undegraded":"2023-11-15T02:59:34.715365+0000","last_fullsized":"2023-11-15T02:59:34.715365+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.271621+0000","last_change":"2023-11-15T02:59:34.264832+0000","last_active":"2023-11-15T02:59:34.271621+0000","last_peered":"2023-11-15T02:59:34.271621+0000","last_clean":"2023-11-15T02:59:34.271621+0000","last_became_active":"2023-11-15T02:59:34.264571+0000","last_became_peered":"2023-11-15T02:59:34.264571+0000","last_unstale":"2023-11-15T02:59:34.271621+0000","last_undegraded":"2023-11-15T02:59:34.271621+0000","last_fullsized":"2023-11-15T02:59:34.271621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.270817+0000","last_change":"2023-11-15T02:59:34.262030+0000","last_active":"2023-11-15T02:59:34.270817+0000","last_peered":"2023-11-15T02:59:34.270817+0000","last_clean":"2023-11-15T02:59:34.270817+0000","last_became_active":"2023-11-15T02:59:34.261689+0000","last_became_peered":"2023-11-15T02:59:34.261689+0000","last_unstale":"2023-11-15T02:59:34.270817+0000","last_undegraded":"2023-11-15T02:59:34.270817+0000","last_fullsized":"2023-11-15T02:59:34.270817+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.701348+0000","last_change":"2023-11-15T02:59:34.701348+0000","last_active":"2023-11-15T02:59:34.701348+0000","last_peered":"2023-11-15T02:59:34.701348+0000","last_clean":"2023-11-15T02:59:34.701348+0000","last_became_active":"2023-11-15T02:59:34.701079+0000","last_became_peered":"2023-11-15T02:59:34.701079+0000","last_unstale":"2023-11-15T02:59:34.701348+0000","last_undegraded":"2023-11-15T02:59:34.701348+0000","last_fullsized":"2023-11-15T02:59:34.701348+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.207077+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.707253+0000","last_change":"2023-11-15T02:59:34.707253+0000","last_active":"2023-11-15T02:59:34.707253+0000","last_peered":"2023-11-15T02:59:34.707253+0000","last_clean":"2023-11-15T02:59:34.707253+0000","last_became_active":"2023-11-15T02:59:34.706969+0000","last_became_peered":"2023-11-15T02:59:34.706969+0000","last_unstale":"2023-11-15T02:59:34.707253+0000","last_undegraded":"2023-11-15T02:59:34.707253+0000","last_fullsized":"2023-11-15T02:59:34.707253+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.207077+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.717917+0000","last_change":"2023-11-15T02:59:34.717917+0000","last_active":"2023-11-15T02:59:34.717917+0000","last_peered":"2023-11-15T02:59:34.717917+0000","last_clean":"2023-11-15T02:59:34.717917+0000","last_became_active":"2023-11-15T02:59:34.717573+0000","last_became_peered":"2023-11-15T02:59:34.717573+0000","last_unstale":"2023-11-15T02:59:34.717917+0000","last_undegraded":"2023-11-15T02:59:34.717917+0000","last_fullsized":"2023-11-15T02:59:34.717917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.229134+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.709579+0000","last_change":"2023-11-15T02:59:34.709579+0000","last_active":"2023-11-15T02:59:34.709579+0000","last_peered":"2023-11-15T02:59:34.709579+0000","last_clean":"2023-11-15T02:59:34.709579+0000","last_became_active":"2023-11-15T02:59:34.709324+0000","last_became_peered":"2023-11-15T02:59:34.709324+0000","last_unstale":"2023-11-15T02:59:34.709579+0000","last_undegraded":"2023-11-15T02:59:34.709579+0000","last_fullsized":"2023-11-15T02:59:34.709579+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.33","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.689591+0000","last_change":"2023-11-15T02:59:34.261668+0000","last_active":"2023-11-15T02:59:34.689591+0000","last_peered":"2023-11-15T02:59:34.689591+0000","last_clean":"2023-11-15T02:59:34.689591+0000","last_became_active":"2023-11-15T02:59:34.261392+0000","last_became_peered":"2023-11-15T02:59:34.261392+0000","last_unstale":"2023-11-15T02:59:34.689591+0000","last_undegraded":"2023-11-15T02:59:34.689591+0000","last_fullsized":"2023-11-15T02:59:34.689591+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.717458+0000","last_change":"2023-11-15T02:59:34.717458+0000","last_active":"2023-11-15T02:59:34.717458+0000","last_peered":"2023-11-15T02:59:34.717458+0000","last_clean":"2023-11-15T02:59:34.717458+0000","last_became_active":"2023-11-15T02:59:34.717124+0000","last_became_peered":"2023-11-15T02:59:34.717124+0000","last_unstale":"2023-11-15T02:59:34.717458+0000","last_undegraded":"2023-11-15T02:59:34.717458+0000","last_fullsized":"2023-11-15T02:59:34.717458+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.274573+0000","last_change":"2023-11-15T02:59:34.267337+0000","last_active":"2023-11-15T02:59:34.274573+0000","last_peered":"2023-11-15T02:59:34.274573+0000","last_clean":"2023-11-15T02:59:34.274573+0000","last_became_active":"2023-11-15T02:59:34.267054+0000","last_became_peered":"2023-11-15T02:59:34.267054+0000","last_unstale":"2023-11-15T02:59:34.274573+0000","last_undegraded":"2023-11-15T02:59:34.274573+0000","last_fullsized":"2023-11-15T02:59:34.274573+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.709174+0000","last_change":"2023-11-15T02:59:34.709174+0000","last_active":"2023-11-15T02:59:34.709174+0000","last_peered":"2023-11-15T02:59:34.709174+0000","last_clean":"2023-11-15T02:59:34.709174+0000","last_became_active":"2023-11-15T02:59:34.708872+0000","last_became_peered":"2023-11-15T02:59:34.708872+0000","last_unstale":"2023-11-15T02:59:34.709174+0000","last_undegraded":"2023-11-15T02:59:34.709174+0000","last_fullsized":"2023-11-15T02:59:34.709174+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.270777+0000","last_change":"2023-11-15T02:59:34.261461+0000","last_active":"2023-11-15T02:59:34.270777+0000","last_peered":"2023-11-15T02:59:34.270777+0000","last_clean":"2023-11-15T02:59:34.270777+0000","last_became_active":"2023-11-15T02:59:34.261153+0000","last_became_peered":"2023-11-15T02:59:34.261153+0000","last_unstale":"2023-11-15T02:59:34.270777+0000","last_undegraded":"2023-11-15T02:59:34.270777+0000","last_fullsized":"2023-11-15T02:59:34.270777+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"60'4","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.721503+0000","last_change":"2023-11-15T02:59:34.264405+0000","last_active":"2023-11-15T02:59:34.721503+0000","last_peered":"2023-11-15T02:59:34.721503+0000","last_clean":"2023-11-15T02:59:34.721503+0000","last_became_active":"2023-11-15T02:59:34.264095+0000","last_became_peered":"2023-11-15T02:59:34.264095+0000","last_unstale":"2023-11-15T02:59:34.721503+0000","last_undegraded":"2023-11-15T02:59:34.721503+0000","last_fullsized":"2023-11-15T02:59:34.721503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.694211+0000","last_change":"2023-11-15T02:59:34.694211+0000","last_active":"2023-11-15T02:59:34.694211+0000","last_peered":"2023-11-15T02:59:34.694211+0000","last_clean":"2023-11-15T02:59:34.694211+0000","last_became_active":"2023-11-15T02:59:34.693708+0000","last_became_peered":"2023-11-15T02:59:34.693708+0000","last_unstale":"2023-11-15T02:59:34.694211+0000","last_undegraded":"2023-11-15T02:59:34.694211+0000","last_fullsized":"2023-11-15T02:59:34.694211+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278070+0000","last_change":"2023-11-15T02:59:34.271015+0000","last_active":"2023-11-15T02:59:34.278070+0000","last_peered":"2023-11-15T02:59:34.278070+0000","last_clean":"2023-11-15T02:59:34.278070+0000","last_became_active":"2023-11-15T02:59:34.270758+0000","last_became_peered":"2023-11-15T02:59:34.270758+0000","last_unstale":"2023-11-15T02:59:34.278070+0000","last_undegraded":"2023-11-15T02:59:34.278070+0000","last_fullsized":"2023-11-15T02:59:34.278070+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.721266+0000","last_change":"2023-11-15T02:59:34.721266+0000","last_active":"2023-11-15T02:59:34.721266+0000","last_peered":"2023-11-15T02:59:34.721266+0000","last_clean":"2023-11-15T02:59:34.721266+0000","last_became_active":"2023-11-15T02:59:34.720978+0000","last_became_peered":"2023-11-15T02:59:34.720978+0000","last_unstale":"2023-11-15T02:59:34.721266+0000","last_undegraded":"2023-11-15T02:59:34.721266+0000","last_fullsized":"2023-11-15T02:59:34.721266+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.276457+0000","last_change":"2023-11-15T02:59:34.269316+0000","last_active":"2023-11-15T02:59:34.276457+0000","last_peered":"2023-11-15T02:59:34.276457+0000","last_clean":"2023-11-15T02:59:34.276457+0000","last_became_active":"2023-11-15T02:59:34.268953+0000","last_became_peered":"2023-11-15T02:59:34.268953+0000","last_unstale":"2023-11-15T02:59:34.276457+0000","last_undegraded":"2023-11-15T02:59:34.276457+0000","last_fullsized":"2023-11-15T02:59:34.276457+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.703052+0000","last_change":"2023-11-15T02:59:34.703052+0000","last_active":"2023-11-15T02:59:34.703052+0000","last_peered":"2023-11-15T02:59:34.703052+0000","last_clean":"2023-11-15T02:59:34.703052+0000","last_became_active":"2023-11-15T02:59:34.702791+0000","last_became_peered":"2023-11-15T02:59:34.702791+0000","last_unstale":"2023-11-15T02:59:34.703052+0000","last_undegraded":"2023-11-15T02:59:34.703052+0000","last_fullsized":"2023-11-15T02:59:34.703052+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2d","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.696289+0000","last_change":"2023-11-15T02:59:34.257033+0000","last_active":"2023-11-15T02:59:34.696289+0000","last_peered":"2023-11-15T02:59:34.696289+0000","last_clean":"2023-11-15T02:59:34.696289+0000","last_became_active":"2023-11-15T02:59:34.256744+0000","last_became_peered":"2023-11-15T02:59:34.256744+0000","last_unstale":"2023-11-15T02:59:34.696289+0000","last_undegraded":"2023-11-15T02:59:34.696289+0000","last_fullsized":"2023-11-15T02:59:34.696289+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.280852+0000","last_change":"2023-11-15T02:59:34.275824+0000","last_active":"2023-11-15T02:59:34.280852+0000","last_peered":"2023-11-15T02:59:34.280852+0000","last_clean":"2023-11-15T02:59:34.280852+0000","last_became_active":"2023-11-15T02:59:34.275088+0000","last_became_peered":"2023-11-15T02:59:34.275088+0000","last_unstale":"2023-11-15T02:59:34.280852+0000","last_undegraded":"2023-11-15T02:59:34.280852+0000","last_fullsized":"2023-11-15T02:59:34.280852+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.718638+0000","last_change":"2023-11-15T02:59:34.718638+0000","last_active":"2023-11-15T02:59:34.718638+0000","last_peered":"2023-11-15T02:59:34.718638+0000","last_clean":"2023-11-15T02:59:34.718638+0000","last_became_active":"2023-11-15T02:59:34.718018+0000","last_became_peered":"2023-11-15T02:59:34.718018+0000","last_unstale":"2023-11-15T02:59:34.718638+0000","last_undegraded":"2023-11-15T02:59:34.718638+0000","last_fullsized":"2023-11-15T02:59:34.718638+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2b","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.688086+0000","last_change":"2023-11-15T02:59:34.271365+0000","last_active":"2023-11-15T02:59:34.688086+0000","last_peered":"2023-11-15T02:59:34.688086+0000","last_clean":"2023-11-15T02:59:34.688086+0000","last_became_active":"2023-11-15T02:59:34.270414+0000","last_became_peered":"2023-11-15T02:59:34.270414+0000","last_unstale":"2023-11-15T02:59:34.688086+0000","last_undegraded":"2023-11-15T02:59:34.688086+0000","last_fullsized":"2023-11-15T02:59:34.688086+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.705469+0000","last_change":"2023-11-15T02:59:34.705469+0000","last_active":"2023-11-15T02:59:34.705469+0000","last_peered":"2023-11-15T02:59:34.705469+0000","last_clean":"2023-11-15T02:59:34.705469+0000","last_became_active":"2023-11-15T02:59:34.705188+0000","last_became_peered":"2023-11-15T02:59:34.705188+0000","last_unstale":"2023-11-15T02:59:34.705469+0000","last_undegraded":"2023-11-15T02:59:34.705469+0000","last_fullsized":"2023-11-15T02:59:34.705469+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.274106+0000","last_change":"2023-11-15T02:59:34.269772+0000","last_active":"2023-11-15T02:59:34.274106+0000","last_peered":"2023-11-15T02:59:34.274106+0000","last_clean":"2023-11-15T02:59:34.274106+0000","last_became_active":"2023-11-15T02:59:34.269150+0000","last_became_peered":"2023-11-15T02:59:34.269150+0000","last_unstale":"2023-11-15T02:59:34.274106+0000","last_undegraded":"2023-11-15T02:59:34.274106+0000","last_fullsized":"2023-11-15T02:59:34.274106+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.706266+0000","last_change":"2023-11-15T02:59:34.706266+0000","last_active":"2023-11-15T02:59:34.706266+0000","last_peered":"2023-11-15T02:59:34.706266+0000","last_clean":"2023-11-15T02:59:34.706266+0000","last_became_active":"2023-11-15T02:59:34.705045+0000","last_became_peered":"2023-11-15T02:59:34.705045+0000","last_unstale":"2023-11-15T02:59:34.706266+0000","last_undegraded":"2023-11-15T02:59:34.706266+0000","last_fullsized":"2023-11-15T02:59:34.706266+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.275956+0000","last_change":"2023-11-15T02:59:34.271476+0000","last_active":"2023-11-15T02:59:34.275956+0000","last_peered":"2023-11-15T02:59:34.275956+0000","last_clean":"2023-11-15T02:59:34.275956+0000","last_became_active":"2023-11-15T02:59:34.271135+0000","last_became_peered":"2023-11-15T02:59:34.271135+0000","last_unstale":"2023-11-15T02:59:34.275956+0000","last_undegraded":"2023-11-15T02:59:34.275956+0000","last_fullsized":"2023-11-15T02:59:34.275956+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.268460+0000","last_change":"2023-11-15T02:59:34.264047+0000","last_active":"2023-11-15T02:59:34.268460+0000","last_peered":"2023-11-15T02:59:34.268460+0000","last_clean":"2023-11-15T02:59:34.268460+0000","last_became_active":"2023-11-15T02:59:34.263525+0000","last_became_peered":"2023-11-15T02:59:34.263525+0000","last_unstale":"2023-11-15T02:59:34.268460+0000","last_undegraded":"2023-11-15T02:59:34.268460+0000","last_fullsized":"2023-11-15T02:59:34.268460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.264845+0000","last_change":"2023-11-15T02:59:34.253441+0000","last_active":"2023-11-15T02:59:34.264845+0000","last_peered":"2023-11-15T02:59:34.264845+0000","last_clean":"2023-11-15T02:59:34.264845+0000","last_became_active":"2023-11-15T02:59:34.253172+0000","last_became_peered":"2023-11-15T02:59:34.253172+0000","last_unstale":"2023-11-15T02:59:34.264845+0000","last_undegraded":"2023-11-15T02:59:34.264845+0000","last_fullsized":"2023-11-15T02:59:34.264845+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.706426+0000","last_change":"2023-11-15T02:59:34.706426+0000","last_active":"2023-11-15T02:59:34.706426+0000","last_peered":"2023-11-15T02:59:34.706426+0000","last_clean":"2023-11-15T02:59:34.706426+0000","last_became_active":"2023-11-15T02:59:34.705567+0000","last_became_peered":"2023-11-15T02:59:34.705567+0000","last_unstale":"2023-11-15T02:59:34.706426+0000","last_undegraded":"2023-11-15T02:59:34.706426+0000","last_fullsized":"2023-11-15T02:59:34.706426+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.229642+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.275473+0000","last_change":"2023-11-15T02:59:34.264728+0000","last_active":"2023-11-15T02:59:34.275473+0000","last_peered":"2023-11-15T02:59:34.275473+0000","last_clean":"2023-11-15T02:59:34.275473+0000","last_became_active":"2023-11-15T02:59:34.264416+0000","last_became_peered":"2023-11-15T02:59:34.264416+0000","last_unstale":"2023-11-15T02:59:34.275473+0000","last_undegraded":"2023-11-15T02:59:34.275473+0000","last_fullsized":"2023-11-15T02:59:34.275473+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.270389+0000","last_change":"2023-11-15T02:59:34.261680+0000","last_active":"2023-11-15T02:59:34.270389+0000","last_peered":"2023-11-15T02:59:34.270389+0000","last_clean":"2023-11-15T02:59:34.270389+0000","last_became_active":"2023-11-15T02:59:34.260538+0000","last_became_peered":"2023-11-15T02:59:34.260538+0000","last_unstale":"2023-11-15T02:59:34.270389+0000","last_undegraded":"2023-11-15T02:59:34.270389+0000","last_fullsized":"2023-11-15T02:59:34.270389+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.707619+0000","last_change":"2023-11-15T02:59:34.707619+0000","last_active":"2023-11-15T02:59:34.707619+0000","last_peered":"2023-11-15T02:59:34.707619+0000","last_clean":"2023-11-15T02:59:34.707619+0000","last_became_active":"2023-11-15T02:59:34.707297+0000","last_became_peered":"2023-11-15T02:59:34.707297+0000","last_unstale":"2023-11-15T02:59:34.707619+0000","last_undegraded":"2023-11-15T02:59:34.707619+0000","last_fullsized":"2023-11-15T02:59:34.707619+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.272192+0000","last_change":"2023-11-15T02:59:34.266701+0000","last_active":"2023-11-15T02:59:34.272192+0000","last_peered":"2023-11-15T02:59:34.272192+0000","last_clean":"2023-11-15T02:59:34.272192+0000","last_became_active":"2023-11-15T02:59:34.266347+0000","last_became_peered":"2023-11-15T02:59:34.266347+0000","last_unstale":"2023-11-15T02:59:34.272192+0000","last_undegraded":"2023-11-15T02:59:34.272192+0000","last_fullsized":"2023-11-15T02:59:34.272192+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.705873+0000","last_change":"2023-11-15T02:59:34.705873+0000","last_active":"2023-11-15T02:59:34.705873+0000","last_peered":"2023-11-15T02:59:34.705873+0000","last_clean":"2023-11-15T02:59:34.705873+0000","last_became_active":"2023-11-15T02:59:34.705522+0000","last_became_peered":"2023-11-15T02:59:34.705522+0000","last_unstale":"2023-11-15T02:59:34.705873+0000","last_undegraded":"2023-11-15T02:59:34.705873+0000","last_fullsized":"2023-11-15T02:59:34.705873+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278740+0000","last_change":"2023-11-15T02:59:34.274845+0000","last_active":"2023-11-15T02:59:34.278740+0000","last_peered":"2023-11-15T02:59:34.278740+0000","last_clean":"2023-11-15T02:59:34.278740+0000","last_became_active":"2023-11-15T02:59:34.274495+0000","last_became_peered":"2023-11-15T02:59:34.274495+0000","last_unstale":"2023-11-15T02:59:34.278740+0000","last_undegraded":"2023-11-15T02:59:34.278740+0000","last_fullsized":"2023-11-15T02:59:34.278740+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.271334+0000","last_change":"2023-11-15T02:59:34.262355+0000","last_active":"2023-11-15T02:59:34.271334+0000","last_peered":"2023-11-15T02:59:34.271334+0000","last_clean":"2023-11-15T02:59:34.271334+0000","last_became_active":"2023-11-15T02:59:34.260844+0000","last_became_peered":"2023-11-15T02:59:34.260844+0000","last_unstale":"2023-11-15T02:59:34.271334+0000","last_undegraded":"2023-11-15T02:59:34.271334+0000","last_fullsized":"2023-11-15T02:59:34.271334+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.711149+0000","last_change":"2023-11-15T02:59:34.711149+0000","last_active":"2023-11-15T02:59:34.711149+0000","last_peered":"2023-11-15T02:59:34.711149+0000","last_clean":"2023-11-15T02:59:34.711149+0000","last_became_active":"2023-11-15T02:59:34.710867+0000","last_became_peered":"2023-11-15T02:59:34.710867+0000","last_unstale":"2023-11-15T02:59:34.711149+0000","last_undegraded":"2023-11-15T02:59:34.711149+0000","last_fullsized":"2023-11-15T02:59:34.711149+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278038+0000","last_change":"2023-11-15T02:59:34.276394+0000","last_active":"2023-11-15T02:59:34.278038+0000","last_peered":"2023-11-15T02:59:34.278038+0000","last_clean":"2023-11-15T02:59:34.278038+0000","last_became_active":"2023-11-15T02:59:34.276134+0000","last_became_peered":"2023-11-15T02:59:34.276134+0000","last_unstale":"2023-11-15T02:59:34.278038+0000","last_undegraded":"2023-11-15T02:59:34.278038+0000","last_fullsized":"2023-11-15T02:59:34.278038+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.697057+0000","last_change":"2023-11-15T02:59:34.697057+0000","last_active":"2023-11-15T02:59:34.697057+0000","last_peered":"2023-11-15T02:59:34.697057+0000","last_clean":"2023-11-15T02:59:34.697057+0000","last_became_active":"2023-11-15T02:59:34.696211+0000","last_became_peered":"2023-11-15T02:59:34.696211+0000","last_unstale":"2023-11-15T02:59:34.697057+0000","last_undegraded":"2023-11-15T02:59:34.697057+0000","last_fullsized":"2023-11-15T02:59:34.697057+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.207077+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.695873+0000","last_change":"2023-11-15T02:59:34.695873+0000","last_active":"2023-11-15T02:59:34.695873+0000","last_peered":"2023-11-15T02:59:34.695873+0000","last_clean":"2023-11-15T02:59:34.695873+0000","last_became_active":"2023-11-15T02:59:34.695590+0000","last_became_peered":"2023-11-15T02:59:34.695590+0000","last_unstale":"2023-11-15T02:59:34.695873+0000","last_undegraded":"2023-11-15T02:59:34.695873+0000","last_fullsized":"2023-11-15T02:59:34.695873+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.693584+0000","last_change":"2023-11-15T02:59:34.272647+0000","last_active":"2023-11-15T02:59:34.693584+0000","last_peered":"2023-11-15T02:59:34.693584+0000","last_clean":"2023-11-15T02:59:34.693584+0000","last_became_active":"2023-11-15T02:59:34.272331+0000","last_became_peered":"2023-11-15T02:59:34.272331+0000","last_unstale":"2023-11-15T02:59:34.693584+0000","last_undegraded":"2023-11-15T02:59:34.693584+0000","last_fullsized":"2023-11-15T02:59:34.693584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.710364+0000","last_change":"2023-11-15T02:59:34.710364+0000","last_active":"2023-11-15T02:59:34.710364+0000","last_peered":"2023-11-15T02:59:34.710364+0000","last_clean":"2023-11-15T02:59:34.710364+0000","last_became_active":"2023-11-15T02:59:34.706803+0000","last_became_peered":"2023-11-15T02:59:34.706803+0000","last_unstale":"2023-11-15T02:59:34.710364+0000","last_undegraded":"2023-11-15T02:59:34.710364+0000","last_fullsized":"2023-11-15T02:59:34.710364+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.710145+0000","last_change":"2023-11-15T02:59:34.263720+0000","last_active":"2023-11-15T02:59:34.710145+0000","last_peered":"2023-11-15T02:59:34.710145+0000","last_clean":"2023-11-15T02:59:34.710145+0000","last_became_active":"2023-11-15T02:59:34.262994+0000","last_became_peered":"2023-11-15T02:59:34.262994+0000","last_unstale":"2023-11-15T02:59:34.710145+0000","last_undegraded":"2023-11-15T02:59:34.710145+0000","last_fullsized":"2023-11-15T02:59:34.710145+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.702473+0000","last_change":"2023-11-15T02:59:34.702473+0000","last_active":"2023-11-15T02:59:34.702473+0000","last_peered":"2023-11-15T02:59:34.702473+0000","last_clean":"2023-11-15T02:59:34.702473+0000","last_became_active":"2023-11-15T02:59:34.702094+0000","last_became_peered":"2023-11-15T02:59:34.702094+0000","last_unstale":"2023-11-15T02:59:34.702473+0000","last_undegraded":"2023-11-15T02:59:34.702473+0000","last_fullsized":"2023-11-15T02:59:34.702473+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.271846+0000","last_change":"2023-11-15T02:59:34.260019+0000","last_active":"2023-11-15T02:59:34.271846+0000","last_peered":"2023-11-15T02:59:34.271846+0000","last_clean":"2023-11-15T02:59:34.271846+0000","last_became_active":"2023-11-15T02:59:34.259470+0000","last_became_peered":"2023-11-15T02:59:34.259470+0000","last_unstale":"2023-11-15T02:59:34.271846+0000","last_undegraded":"2023-11-15T02:59:34.271846+0000","last_fullsized":"2023-11-15T02:59:34.271846+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.707408+0000","last_change":"2023-11-15T02:59:34.707408+0000","last_active":"2023-11-15T02:59:34.707408+0000","last_peered":"2023-11-15T02:59:34.707408+0000","last_clean":"2023-11-15T02:59:34.707408+0000","last_became_active":"2023-11-15T02:59:34.706951+0000","last_became_peered":"2023-11-15T02:59:34.706951+0000","last_unstale":"2023-11-15T02:59:34.707408+0000","last_undegraded":"2023-11-15T02:59:34.707408+0000","last_fullsized":"2023-11-15T02:59:34.707408+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.280951+0000","last_change":"2023-11-15T02:59:34.276164+0000","last_active":"2023-11-15T02:59:34.280951+0000","last_peered":"2023-11-15T02:59:34.280951+0000","last_clean":"2023-11-15T02:59:34.280951+0000","last_became_active":"2023-11-15T02:59:34.275881+0000","last_became_peered":"2023-11-15T02:59:34.275881+0000","last_unstale":"2023-11-15T02:59:34.280951+0000","last_undegraded":"2023-11-15T02:59:34.280951+0000","last_fullsized":"2023-11-15T02:59:34.280951+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.274124+0000","last_change":"2023-11-15T02:59:34.269967+0000","last_active":"2023-11-15T02:59:34.274124+0000","last_peered":"2023-11-15T02:59:34.274124+0000","last_clean":"2023-11-15T02:59:34.274124+0000","last_became_active":"2023-11-15T02:59:34.269516+0000","last_became_peered":"2023-11-15T02:59:34.269516+0000","last_unstale":"2023-11-15T02:59:34.274124+0000","last_undegraded":"2023-11-15T02:59:34.274124+0000","last_fullsized":"2023-11-15T02:59:34.274124+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.713129+0000","last_change":"2023-11-15T02:59:34.713129+0000","last_active":"2023-11-15T02:59:34.713129+0000","last_peered":"2023-11-15T02:59:34.713129+0000","last_clean":"2023-11-15T02:59:34.713129+0000","last_became_active":"2023-11-15T02:59:34.704067+0000","last_became_peered":"2023-11-15T02:59:34.704067+0000","last_unstale":"2023-11-15T02:59:34.713129+0000","last_undegraded":"2023-11-15T02:59:34.713129+0000","last_fullsized":"2023-11-15T02:59:34.713129+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278113+0000","last_change":"2023-11-15T02:59:34.272334+0000","last_active":"2023-11-15T02:59:34.278113+0000","last_peered":"2023-11-15T02:59:34.278113+0000","last_clean":"2023-11-15T02:59:34.278113+0000","last_became_active":"2023-11-15T02:59:34.272096+0000","last_became_peered":"2023-11-15T02:59:34.272096+0000","last_unstale":"2023-11-15T02:59:34.278113+0000","last_undegraded":"2023-11-15T02:59:34.278113+0000","last_fullsized":"2023-11-15T02:59:34.278113+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.686879+0000","last_change":"2023-11-15T02:59:34.686879+0000","last_active":"2023-11-15T02:59:34.686879+0000","last_peered":"2023-11-15T02:59:34.686879+0000","last_clean":"2023-11-15T02:59:34.686879+0000","last_became_active":"2023-11-15T02:59:34.685697+0000","last_became_peered":"2023-11-15T02:59:34.685697+0000","last_unstale":"2023-11-15T02:59:34.686879+0000","last_undegraded":"2023-11-15T02:59:34.686879+0000","last_fullsized":"2023-11-15T02:59:34.686879+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.275520+0000","last_change":"2023-11-15T02:59:34.270391+0000","last_active":"2023-11-15T02:59:34.275520+0000","last_peered":"2023-11-15T02:59:34.275520+0000","last_clean":"2023-11-15T02:59:34.275520+0000","last_became_active":"2023-11-15T02:59:34.270049+0000","last_became_peered":"2023-11-15T02:59:34.270049+0000","last_unstale":"2023-11-15T02:59:34.275520+0000","last_undegraded":"2023-11-15T02:59:34.275520+0000","last_fullsized":"2023-11-15T02:59:34.275520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.282770+0000","last_change":"2023-11-15T02:59:34.280324+0000","last_active":"2023-11-15T02:59:34.282770+0000","last_peered":"2023-11-15T02:59:34.282770+0000","last_clean":"2023-11-15T02:59:34.282770+0000","last_became_active":"2023-11-15T02:59:34.280003+0000","last_became_peered":"2023-11-15T02:59:34.280003+0000","last_unstale":"2023-11-15T02:59:34.282770+0000","last_undegraded":"2023-11-15T02:59:34.282770+0000","last_fullsized":"2023-11-15T02:59:34.282770+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.706837+0000","last_change":"2023-11-15T02:59:34.706837+0000","last_active":"2023-11-15T02:59:34.706837+0000","last_peered":"2023-11-15T02:59:34.706837+0000","last_clean":"2023-11-15T02:59:34.706837+0000","last_became_active":"2023-11-15T02:59:34.706595+0000","last_became_peered":"2023-11-15T02:59:34.706595+0000","last_unstale":"2023-11-15T02:59:34.706837+0000","last_undegraded":"2023-11-15T02:59:34.706837+0000","last_fullsized":"2023-11-15T02:59:34.706837+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278312+0000","last_change":"2023-11-15T02:59:34.274116+0000","last_active":"2023-11-15T02:59:34.278312+0000","last_peered":"2023-11-15T02:59:34.278312+0000","last_clean":"2023-11-15T02:59:34.278312+0000","last_became_active":"2023-11-15T02:59:34.273108+0000","last_became_peered":"2023-11-15T02:59:34.273108+0000","last_unstale":"2023-11-15T02:59:34.278312+0000","last_undegraded":"2023-11-15T02:59:34.278312+0000","last_fullsized":"2023-11-15T02:59:34.278312+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.713931+0000","last_change":"2023-11-15T02:59:34.713931+0000","last_active":"2023-11-15T02:59:34.713931+0000","last_peered":"2023-11-15T02:59:34.713931+0000","last_clean":"2023-11-15T02:59:34.713931+0000","last_became_active":"2023-11-15T02:59:34.713640+0000","last_became_peered":"2023-11-15T02:59:34.713640+0000","last_unstale":"2023-11-15T02:59:34.713931+0000","last_undegraded":"2023-11-15T02:59:34.713931+0000","last_fullsized":"2023-11-15T02:59:34.713931+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"60'3","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.704814+0000","last_change":"2023-11-15T02:59:34.273818+0000","last_active":"2023-11-15T02:59:34.704814+0000","last_peered":"2023-11-15T02:59:34.704814+0000","last_clean":"2023-11-15T02:59:34.704814+0000","last_became_active":"2023-11-15T02:59:34.273463+0000","last_became_peered":"2023-11-15T02:59:34.273463+0000","last_unstale":"2023-11-15T02:59:34.704814+0000","last_undegraded":"2023-11-15T02:59:34.704814+0000","last_fullsized":"2023-11-15T02:59:34.704814+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.691994+0000","last_change":"2023-11-15T02:59:34.691994+0000","last_active":"2023-11-15T02:59:34.691994+0000","last_peered":"2023-11-15T02:59:34.691994+0000","last_clean":"2023-11-15T02:59:34.691994+0000","last_became_active":"2023-11-15T02:59:34.691573+0000","last_became_peered":"2023-11-15T02:59:34.691573+0000","last_unstale":"2023-11-15T02:59:34.691994+0000","last_undegraded":"2023-11-15T02:59:34.691994+0000","last_fullsized":"2023-11-15T02:59:34.691994+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.207077+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.708893+0000","last_change":"2023-11-15T02:59:34.708893+0000","last_active":"2023-11-15T02:59:34.708893+0000","last_peered":"2023-11-15T02:59:34.708893+0000","last_clean":"2023-11-15T02:59:34.708893+0000","last_became_active":"2023-11-15T02:59:34.708623+0000","last_became_peered":"2023-11-15T02:59:34.708623+0000","last_unstale":"2023-11-15T02:59:34.708893+0000","last_undegraded":"2023-11-15T02:59:34.708893+0000","last_fullsized":"2023-11-15T02:59:34.708893+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.688236+0000","last_change":"2023-11-15T02:59:34.270907+0000","last_active":"2023-11-15T02:59:34.688236+0000","last_peered":"2023-11-15T02:59:34.688236+0000","last_clean":"2023-11-15T02:59:34.688236+0000","last_became_active":"2023-11-15T02:59:34.270561+0000","last_became_peered":"2023-11-15T02:59:34.270561+0000","last_unstale":"2023-11-15T02:59:34.688236+0000","last_undegraded":"2023-11-15T02:59:34.688236+0000","last_fullsized":"2023-11-15T02:59:34.688236+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.230568+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.719857+0000","last_change":"2023-11-15T02:59:34.719857+0000","last_active":"2023-11-15T02:59:34.719857+0000","last_peered":"2023-11-15T02:59:34.719857+0000","last_clean":"2023-11-15T02:59:34.719857+0000","last_became_active":"2023-11-15T02:59:34.719591+0000","last_became_peered":"2023-11-15T02:59:34.719591+0000","last_unstale":"2023-11-15T02:59:34.719857+0000","last_undegraded":"2023-11-15T02:59:34.719857+0000","last_fullsized":"2023-11-15T02:59:34.719857+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.279642+0000","last_change":"2023-11-15T02:59:34.276260+0000","last_active":"2023-11-15T02:59:34.279642+0000","last_peered":"2023-11-15T02:59:34.279642+0000","last_clean":"2023-11-15T02:59:34.279642+0000","last_became_active":"2023-11-15T02:59:34.275758+0000","last_became_peered":"2023-11-15T02:59:34.275758+0000","last_unstale":"2023-11-15T02:59:34.279642+0000","last_undegraded":"2023-11-15T02:59:34.279642+0000","last_fullsized":"2023-11-15T02:59:34.279642+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"60'4","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.722402+0000","last_change":"2023-11-15T02:59:34.269037+0000","last_active":"2023-11-15T02:59:34.722402+0000","last_peered":"2023-11-15T02:59:34.722402+0000","last_clean":"2023-11-15T02:59:34.722402+0000","last_became_active":"2023-11-15T02:59:34.268123+0000","last_became_peered":"2023-11-15T02:59:34.268123+0000","last_unstale":"2023-11-15T02:59:34.722402+0000","last_undegraded":"2023-11-15T02:59:34.722402+0000","last_fullsized":"2023-11-15T02:59:34.722402+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.715434+0000","last_change":"2023-11-15T02:59:34.715434+0000","last_active":"2023-11-15T02:59:34.715434+0000","last_peered":"2023-11-15T02:59:34.715434+0000","last_clean":"2023-11-15T02:59:34.715434+0000","last_became_active":"2023-11-15T02:59:34.715198+0000","last_became_peered":"2023-11-15T02:59:34.715198+0000","last_unstale":"2023-11-15T02:59:34.715434+0000","last_undegraded":"2023-11-15T02:59:34.715434+0000","last_fullsized":"2023-11-15T02:59:34.715434+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.228982+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.705591+0000","last_change":"2023-11-15T02:59:34.705591+0000","last_active":"2023-11-15T02:59:34.705591+0000","last_peered":"2023-11-15T02:59:34.705591+0000","last_clean":"2023-11-15T02:59:34.705591+0000","last_became_active":"2023-11-15T02:59:34.703826+0000","last_became_peered":"2023-11-15T02:59:34.703826+0000","last_unstale":"2023-11-15T02:59:34.705591+0000","last_undegraded":"2023-11-15T02:59:34.705591+0000","last_fullsized":"2023-11-15T02:59:34.705591+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.228971+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.273908+0000","last_change":"2023-11-15T02:59:34.263759+0000","last_active":"2023-11-15T02:59:34.273908+0000","last_peered":"2023-11-15T02:59:34.273908+0000","last_clean":"2023-11-15T02:59:34.273908+0000","last_became_active":"2023-11-15T02:59:34.263333+0000","last_became_peered":"2023-11-15T02:59:34.263333+0000","last_unstale":"2023-11-15T02:59:34.273908+0000","last_undegraded":"2023-11-15T02:59:34.273908+0000","last_fullsized":"2023-11-15T02:59:34.273908+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.700382+0000","last_change":"2023-11-15T02:59:34.700382+0000","last_active":"2023-11-15T02:59:34.700382+0000","last_peered":"2023-11-15T02:59:34.700382+0000","last_clean":"2023-11-15T02:59:34.700382+0000","last_became_active":"2023-11-15T02:59:34.700091+0000","last_became_peered":"2023-11-15T02:59:34.700091+0000","last_unstale":"2023-11-15T02:59:34.700382+0000","last_undegraded":"2023-11-15T02:59:34.700382+0000","last_fullsized":"2023-11-15T02:59:34.700382+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.207077+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.697102+0000","last_change":"2023-11-15T02:59:34.697102+0000","last_active":"2023-11-15T02:59:34.697102+0000","last_peered":"2023-11-15T02:59:34.697102+0000","last_clean":"2023-11-15T02:59:34.697102+0000","last_became_active":"2023-11-15T02:59:34.696095+0000","last_became_peered":"2023-11-15T02:59:34.696095+0000","last_unstale":"2023-11-15T02:59:34.697102+0000","last_undegraded":"2023-11-15T02:59:34.697102+0000","last_fullsized":"2023-11-15T02:59:34.697102+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.279613+0000","last_change":"2023-11-15T02:59:34.275904+0000","last_active":"2023-11-15T02:59:34.279613+0000","last_peered":"2023-11-15T02:59:34.279613+0000","last_clean":"2023-11-15T02:59:34.279613+0000","last_became_active":"2023-11-15T02:59:34.275561+0000","last_became_peered":"2023-11-15T02:59:34.275561+0000","last_unstale":"2023-11-15T02:59:34.279613+0000","last_undegraded":"2023-11-15T02:59:34.279613+0000","last_fullsized":"2023-11-15T02:59:34.279613+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.710944+0000","last_change":"2023-11-15T02:59:34.710944+0000","last_active":"2023-11-15T02:59:34.710944+0000","last_peered":"2023-11-15T02:59:34.710944+0000","last_clean":"2023-11-15T02:59:34.710944+0000","last_became_active":"2023-11-15T02:59:34.710640+0000","last_became_peered":"2023-11-15T02:59:34.710640+0000","last_unstale":"2023-11-15T02:59:34.710944+0000","last_undegraded":"2023-11-15T02:59:34.710944+0000","last_fullsized":"2023-11-15T02:59:34.710944+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.280896+0000","last_change":"2023-11-15T02:59:34.275717+0000","last_active":"2023-11-15T02:59:34.280896+0000","last_peered":"2023-11-15T02:59:34.280896+0000","last_clean":"2023-11-15T02:59:34.280896+0000","last_became_active":"2023-11-15T02:59:34.274314+0000","last_became_peered":"2023-11-15T02:59:34.274314+0000","last_unstale":"2023-11-15T02:59:34.280896+0000","last_undegraded":"2023-11-15T02:59:34.280896+0000","last_fullsized":"2023-11-15T02:59:34.280896+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278096+0000","last_change":"2023-11-15T02:59:34.271643+0000","last_active":"2023-11-15T02:59:34.278096+0000","last_peered":"2023-11-15T02:59:34.278096+0000","last_clean":"2023-11-15T02:59:34.278096+0000","last_became_active":"2023-11-15T02:59:34.271375+0000","last_became_peered":"2023-11-15T02:59:34.271375+0000","last_unstale":"2023-11-15T02:59:34.278096+0000","last_undegraded":"2023-11-15T02:59:34.278096+0000","last_fullsized":"2023-11-15T02:59:34.278096+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.711599+0000","last_change":"2023-11-15T02:59:34.711599+0000","last_active":"2023-11-15T02:59:34.711599+0000","last_peered":"2023-11-15T02:59:34.711599+0000","last_clean":"2023-11-15T02:59:34.711599+0000","last_became_active":"2023-11-15T02:59:34.711325+0000","last_became_peered":"2023-11-15T02:59:34.711325+0000","last_unstale":"2023-11-15T02:59:34.711599+0000","last_undegraded":"2023-11-15T02:59:34.711599+0000","last_fullsized":"2023-11-15T02:59:34.711599+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.207077+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.699657+0000","last_change":"2023-11-15T02:59:34.699657+0000","last_active":"2023-11-15T02:59:34.699657+0000","last_peered":"2023-11-15T02:59:34.699657+0000","last_clean":"2023-11-15T02:59:34.699657+0000","last_became_active":"2023-11-15T02:59:34.699371+0000","last_became_peered":"2023-11-15T02:59:34.699371+0000","last_unstale":"2023-11-15T02:59:34.699657+0000","last_undegraded":"2023-11-15T02:59:34.699657+0000","last_fullsized":"2023-11-15T02:59:34.699657+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.272131+0000","last_change":"2023-11-15T02:59:34.264863+0000","last_active":"2023-11-15T02:59:34.272131+0000","last_peered":"2023-11-15T02:59:34.272131+0000","last_clean":"2023-11-15T02:59:34.272131+0000","last_became_active":"2023-11-15T02:59:34.264228+0000","last_became_peered":"2023-11-15T02:59:34.264228+0000","last_unstale":"2023-11-15T02:59:34.272131+0000","last_undegraded":"2023-11-15T02:59:34.272131+0000","last_fullsized":"2023-11-15T02:59:34.272131+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.252284+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T02:59:34.252284+0000","last_peered":"2023-11-15T02:59:34.252284+0000","last_clean":"2023-11-15T02:59:34.252284+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T02:59:34.252284+0000","last_undegraded":"2023-11-15T02:59:34.252284+0000","last_fullsized":"2023-11-15T02:59:34.252284+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.694895+0000","last_change":"2023-11-15T02:59:34.694895+0000","last_active":"2023-11-15T02:59:34.694895+0000","last_peered":"2023-11-15T02:59:34.694895+0000","last_clean":"2023-11-15T02:59:34.694895+0000","last_became_active":"2023-11-15T02:59:34.694592+0000","last_became_peered":"2023-11-15T02:59:34.694592+0000","last_unstale":"2023-11-15T02:59:34.694895+0000","last_undegraded":"2023-11-15T02:59:34.694895+0000","last_fullsized":"2023-11-15T02:59:34.694895+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.282767+0000","last_change":"2023-11-15T02:59:34.280250+0000","last_active":"2023-11-15T02:59:34.282767+0000","last_peered":"2023-11-15T02:59:34.282767+0000","last_clean":"2023-11-15T02:59:34.282767+0000","last_became_active":"2023-11-15T02:59:34.279961+0000","last_became_peered":"2023-11-15T02:59:34.279961+0000","last_unstale":"2023-11-15T02:59:34.282767+0000","last_undegraded":"2023-11-15T02:59:34.282767+0000","last_fullsized":"2023-11-15T02:59:34.282767+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-15T02:59:29.318446+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T02:59:29.318446+0000","last_peered":"2023-11-15T02:59:29.318446+0000","last_clean":"2023-11-15T02:59:29.318446+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T02:59:29.318446+0000","last_undegraded":"2023-11-15T02:59:29.318446+0000","last_fullsized":"2023-11-15T02:59:29.318446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.720662+0000","last_change":"2023-11-15T02:59:34.720662+0000","last_active":"2023-11-15T02:59:34.720662+0000","last_peered":"2023-11-15T02:59:34.720662+0000","last_clean":"2023-11-15T02:59:34.720662+0000","last_became_active":"2023-11-15T02:59:34.713450+0000","last_became_peered":"2023-11-15T02:59:34.713450+0000","last_unstale":"2023-11-15T02:59:34.720662+0000","last_undegraded":"2023-11-15T02:59:34.720662+0000","last_fullsized":"2023-11-15T02:59:34.720662+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716637+0000","last_change":"2023-11-15T02:59:34.272609+0000","last_active":"2023-11-15T02:59:34.716637+0000","last_peered":"2023-11-15T02:59:34.716637+0000","last_clean":"2023-11-15T02:59:34.716637+0000","last_became_active":"2023-11-15T02:59:34.272212+0000","last_became_peered":"2023-11-15T02:59:34.272212+0000","last_unstale":"2023-11-15T02:59:34.716637+0000","last_undegraded":"2023-11-15T02:59:34.716637+0000","last_fullsized":"2023-11-15T02:59:34.716637+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.257610+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T02:59:34.257610+0000","last_peered":"2023-11-15T02:59:34.257610+0000","last_clean":"2023-11-15T02:59:34.257610+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T02:59:34.257610+0000","last_undegraded":"2023-11-15T02:59:34.257610+0000","last_fullsized":"2023-11-15T02:59:34.257610+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.711825+0000","last_change":"2023-11-15T02:59:34.711825+0000","last_active":"2023-11-15T02:59:34.711825+0000","last_peered":"2023-11-15T02:59:34.711825+0000","last_clean":"2023-11-15T02:59:34.711825+0000","last_became_active":"2023-11-15T02:59:34.711551+0000","last_became_peered":"2023-11-15T02:59:34.711551+0000","last_unstale":"2023-11-15T02:59:34.711825+0000","last_undegraded":"2023-11-15T02:59:34.711825+0000","last_fullsized":"2023-11-15T02:59:34.711825+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.229993+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-15T02:59:29.318457+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T02:59:29.318457+0000","last_peered":"2023-11-15T02:59:29.318457+0000","last_clean":"2023-11-15T02:59:29.318457+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T02:59:29.318457+0000","last_undegraded":"2023-11-15T02:59:29.318457+0000","last_fullsized":"2023-11-15T02:59:29.318457+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.712699+0000","last_change":"2023-11-15T02:59:34.712699+0000","last_active":"2023-11-15T02:59:34.712699+0000","last_peered":"2023-11-15T02:59:34.712699+0000","last_clean":"2023-11-15T02:59:34.712699+0000","last_became_active":"2023-11-15T02:59:34.703686+0000","last_became_peered":"2023-11-15T02:59:34.703686+0000","last_unstale":"2023-11-15T02:59:34.712699+0000","last_undegraded":"2023-11-15T02:59:34.712699+0000","last_fullsized":"2023-11-15T02:59:34.712699+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.277673+0000","last_change":"2023-11-15T02:59:34.265955+0000","last_active":"2023-11-15T02:59:34.277673+0000","last_peered":"2023-11-15T02:59:34.277673+0000","last_clean":"2023-11-15T02:59:34.277673+0000","last_became_active":"2023-11-15T02:59:34.265681+0000","last_became_peered":"2023-11-15T02:59:34.265681+0000","last_unstale":"2023-11-15T02:59:34.277673+0000","last_undegraded":"2023-11-15T02:59:34.277673+0000","last_fullsized":"2023-11-15T02:59:34.277673+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":50,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-15T02:59:30.343893+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T02:59:30.343893+0000","last_peered":"2023-11-15T02:59:30.343893+0000","last_clean":"2023-11-15T02:59:30.343893+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T02:59:30.343893+0000","last_undegraded":"2023-11-15T02:59:30.343893+0000","last_fullsized":"2023-11-15T02:59:30.343893+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.229084+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.253156+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T02:59:34.253156+0000","last_peered":"2023-11-15T02:59:34.253156+0000","last_clean":"2023-11-15T02:59:34.253156+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T02:59:34.253156+0000","last_undegraded":"2023-11-15T02:59:34.253156+0000","last_fullsized":"2023-11-15T02:59:34.253156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.704325+0000","last_change":"2023-11-15T02:59:34.704325+0000","last_active":"2023-11-15T02:59:34.704325+0000","last_peered":"2023-11-15T02:59:34.704325+0000","last_clean":"2023-11-15T02:59:34.704325+0000","last_became_active":"2023-11-15T02:59:34.703365+0000","last_became_peered":"2023-11-15T02:59:34.703365+0000","last_unstale":"2023-11-15T02:59:34.704325+0000","last_undegraded":"2023-11-15T02:59:34.704325+0000","last_fullsized":"2023-11-15T02:59:34.704325+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.691558+0000","last_change":"2023-11-15T02:59:34.265022+0000","last_active":"2023-11-15T02:59:34.691558+0000","last_peered":"2023-11-15T02:59:34.691558+0000","last_clean":"2023-11-15T02:59:34.691558+0000","last_became_active":"2023-11-15T02:59:34.264125+0000","last_became_peered":"2023-11-15T02:59:34.264125+0000","last_unstale":"2023-11-15T02:59:34.691558+0000","last_undegraded":"2023-11-15T02:59:34.691558+0000","last_fullsized":"2023-11-15T02:59:34.691558+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.254819+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T02:59:34.254819+0000","last_peered":"2023-11-15T02:59:34.254819+0000","last_clean":"2023-11-15T02:59:34.254819+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T02:59:34.254819+0000","last_undegraded":"2023-11-15T02:59:34.254819+0000","last_fullsized":"2023-11-15T02:59:34.254819+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.697027+0000","last_change":"2023-11-15T02:59:34.697027+0000","last_active":"2023-11-15T02:59:34.697027+0000","last_peered":"2023-11-15T02:59:34.697027+0000","last_clean":"2023-11-15T02:59:34.697027+0000","last_became_active":"2023-11-15T02:59:34.696199+0000","last_became_peered":"2023-11-15T02:59:34.696199+0000","last_unstale":"2023-11-15T02:59:34.697027+0000","last_undegraded":"2023-11-15T02:59:34.697027+0000","last_fullsized":"2023-11-15T02:59:34.697027+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.283700+0000","last_change":"2023-11-15T02:59:34.279512+0000","last_active":"2023-11-15T02:59:34.283700+0000","last_peered":"2023-11-15T02:59:34.283700+0000","last_clean":"2023-11-15T02:59:34.283700+0000","last_became_active":"2023-11-15T02:59:34.278967+0000","last_became_peered":"2023-11-15T02:59:34.278967+0000","last_unstale":"2023-11-15T02:59:34.283700+0000","last_undegraded":"2023-11-15T02:59:34.283700+0000","last_fullsized":"2023-11-15T02:59:34.283700+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":50,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-15T02:59:30.343807+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T02:59:30.343807+0000","last_peered":"2023-11-15T02:59:30.343807+0000","last_clean":"2023-11-15T02:59:30.343807+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T02:59:30.343807+0000","last_undegraded":"2023-11-15T02:59:30.343807+0000","last_fullsized":"2023-11-15T02:59:30.343807+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.254906+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T02:59:34.254906+0000","last_peered":"2023-11-15T02:59:34.254906+0000","last_clean":"2023-11-15T02:59:34.254906+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T02:59:34.254906+0000","last_undegraded":"2023-11-15T02:59:34.254906+0000","last_fullsized":"2023-11-15T02:59:34.254906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":6},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":748,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":116,"acting":116,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542180,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":704512,"data_stored":312535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.1679999999999999,"15min":1.1859999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.458,"5min":3.952,"15min":3.952},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.046,"5min":1.1799999999999999,"15min":1.202},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4419999999999999,"5min":2.5,"15min":2.5},"last":1.016}]},{"osd":1,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.137,"15min":1.145},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.3740000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1739999999999999,"15min":1.1779999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.415,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.3049999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2150000000000001,"15min":1.2330000000000001},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.458,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2949999999999999,"15min":1.3069999999999999},"min":{"1min":0.97899999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.5149999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":0.92900000000000005}]},{"osd":3,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.3400000000000001,"15min":1.3740000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.381,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.175,"5min":1.2669999999999999,"15min":1.282},"min":{"1min":0.90600000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.351,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.093}]},{"osd":4,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.2689999999999999,"15min":1.3},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.4359999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":1.117},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.448,"15min":1.4990000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.5029999999999999,"5min":4.226,"15min":4.226},"last":0.87}]},{"osd":5,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.329,"15min":1.343},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.556,"5min":2.024,"15min":2.024},"last":1.28},{"interface":"front","average":{"1min":1.234,"5min":1.4399999999999999,"15min":1.474},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.653,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.216}]},{"osd":6,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.5620000000000001,"15min":1.601},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":1.5660000000000001,"5min":3.2789999999999999,"15min":3.2789999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.591,"15min":1.645},"min":{"1min":1.123,"5min":1.123,"15min":1.123},"max":{"1min":1.4399999999999999,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.98099999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542180,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297800,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074040,"statfs":{"total":96636764160,"available":96331816960,"internally_reserved":0,"allocated":720896,"data_stored":319748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.125,"15min":1.133},"min":{"1min":0.69799999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.353,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.0469999999999999,"15min":1.0409999999999999},"min":{"1min":0.72499999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.484,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.93899999999999995}]},{"osd":1,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0920000000000001,"15min":1.0840000000000001},"min":{"1min":0.876,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.415,"5min":1.623,"15min":1.623},"last":1.03},{"interface":"front","average":{"1min":1.173,"5min":1.169,"15min":1.169},"min":{"1min":0.80800000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.609,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.97199999999999998}]},{"osd":2,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.1870000000000001,"15min":1.1779999999999999},"min":{"1min":0.84399999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.496,"5min":2.621,"15min":2.621},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.0960000000000001,"15min":1.0840000000000001},"min":{"1min":0.91700000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.425,"5min":1.5,"15min":1.5},"last":1.2130000000000001}]},{"osd":3,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.167,"15min":1.157},"min":{"1min":1.0029999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.492,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.26,"15min":1.2629999999999999},"min":{"1min":0.99299999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.6850000000000001,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.3740000000000001}]},{"osd":4,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.173,"15min":1.1910000000000001},"min":{"1min":0.81399999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.4099999999999999,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":1.391},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.95599999999999996,"5min":0.625,"15min":0.625},"max":{"1min":1.4850000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.536}]},{"osd":5,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.417,"15min":1.411},"min":{"1min":1.155,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.772,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.3109999999999999,"15min":1.319},"min":{"1min":0.877,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6599999999999999,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":1.169}]},{"osd":7,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2809999999999999,"15min":1.3009999999999999},"min":{"1min":0.82199999999999995,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.395,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.522},{"interface":"front","average":{"1min":1.401,"5min":1.448,"15min":1.456},"min":{"1min":1.046,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8540000000000001,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":1.637}]}]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297800,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074040,"statfs":{"total":96636764160,"available":96331816960,"internally_reserved":0,"allocated":720896,"data_stored":319804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.107,"15min":1.1120000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.516,"5min":1.532,"15min":1.532},"last":1.125},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0069999999999999,"15min":1.0029999999999999},"min":{"1min":0.80700000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":1.28}]},{"osd":1,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2250000000000001,"15min":1.2370000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4770000000000001,"5min":2.617,"15min":2.617},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.115,"5min":1.1559999999999999,"15min":1.163},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.425,"5min":1.5,"15min":1.5},"last":1.2410000000000001}]},{"osd":2,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.1519999999999999,"15min":1.141},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.462,"5min":1.909,"15min":1.909},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1599999999999999,"15min":1.1639999999999999},"min":{"1min":0.88800000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.399,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.0700000000000001}]},{"osd":3,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1519999999999999,"15min":1.1599999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.462,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.345},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.294,"15min":1.3},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.5840000000000001,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":1.169}]},{"osd":5,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2430000000000001,"15min":1.2529999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5780000000000001,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.2290000000000001,"15min":1.2350000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.5049999999999999,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.6639999999999999}]},{"osd":6,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.333,"15min":1.343},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6619999999999999,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":1.456},{"interface":"front","average":{"1min":1.296,"5min":1.3979999999999999,"15min":1.415},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.7450000000000001,"5min":2.173,"15min":2.173},"last":1.4670000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.369,"15min":1.381},"min":{"1min":1.069,"5min":0.87,"15min":0.87},"max":{"1min":1.5820000000000001,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.4350000000000001,"15min":1.4450000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.587,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":1.415}]}]},{"osd":3,"up_from":14,"seq":60129542181,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":704512,"data_stored":312535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.77400000000000002,"15min":0.75900000000000001},"min":{"1min":0.64900000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.629},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.84199999999999997,"15min":0.82899999999999996},"min":{"1min":0.73599999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1779999999999999,"5min":1.26,"15min":1.26},"last":0.70699999999999996}]},{"osd":1,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.90000000000000002,"15min":0.90100000000000002},"min":{"1min":0.621,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.97999999999999998,"15min":0.96999999999999997},"min":{"1min":0.83299999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2749999999999999,"5min":1.375,"15min":1.375},"last":0.55800000000000005}]},{"osd":2,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97699999999999998,"15min":0.96799999999999997},"min":{"1min":0.76700000000000002,"5min":0.63,"15min":0.63},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0649999999999999,"15min":1.052},"min":{"1min":0.85899999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.96699999999999997}]},{"osd":4,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1859999999999999,"15min":1.1799999999999999},"min":{"1min":0.97899999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.456,"5min":1.837,"15min":1.837},"last":1.202},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.22,"15min":1.226},"min":{"1min":1.0249999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.492,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.0189999999999999}]},{"osd":5,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.27,"15min":1.2669999999999999},"min":{"1min":0.996,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.6459999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.05},{"interface":"front","average":{"1min":1.345,"5min":1.2789999999999999,"15min":1.268},"min":{"1min":1.01,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.89900000000000002}]},{"osd":6,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3120000000000001,"15min":1.306},"min":{"1min":1.1060000000000001,"5min":1.048,"15min":1.048},"max":{"1min":1.552,"5min":1.794,"15min":1.794},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.3580000000000001,"15min":1.3520000000000001},"min":{"1min":1.129,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":1.647,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.163}]},{"osd":7,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.3919999999999999,"15min":1.3899999999999999},"min":{"1min":1.083,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.365,"15min":1.357},"min":{"1min":1.1819999999999999,"5min":1.085,"15min":1.085},"max":{"1min":1.665,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.1850000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542181,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297780,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074060,"statfs":{"total":96636764160,"available":96331837440,"internally_reserved":0,"allocated":700416,"data_stored":311266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76700000000000002,"15min":0.76400000000000001},"min":{"1min":0.61699999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88700000000000001,"15min":0.88200000000000001},"min":{"1min":0.68300000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.67400000000000004}]},{"osd":1,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89600000000000002,"15min":0.89200000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3120000000000001,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.96099999999999997,"15min":0.94899999999999995},"min":{"1min":0.83499999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.93600000000000005}]},{"osd":3,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.002,"15min":1},"min":{"1min":0.79200000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.387,"5min":1.472,"15min":1.472},"last":0.629},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1259999999999999,"15min":1.1200000000000001},"min":{"1min":0.93000000000000005,"5min":0.622,"15min":0.622},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.81899999999999995}]},{"osd":4,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.208,"15min":1.21},"min":{"1min":1.0149999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.48,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.111},{"interface":"front","average":{"1min":1.286,"5min":1.238,"15min":1.23},"min":{"1min":1.0760000000000001,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.5429999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.071}]},{"osd":5,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.262,"15min":1.264},"min":{"1min":1.0149999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":1.613,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2989999999999999,"15min":1.2969999999999999},"min":{"1min":1.0700000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.6890000000000001,"5min":1.857,"15min":1.857},"last":1}]},{"osd":6,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.367,"15min":1.373},"min":{"1min":1.056,"5min":1.056,"15min":1.056},"max":{"1min":1.72,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.347,"15min":1.353},"min":{"1min":1.046,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":1.573,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.1599999999999999}]},{"osd":7,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3939999999999999,"15min":1.391},"min":{"1min":1.1719999999999999,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":1.8069999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.3799999999999999,"15min":1.3919999999999999},"min":{"1min":1.097,"5min":1.097,"15min":1.097},"max":{"1min":1.5529999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.96799999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542181,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":704512,"data_stored":312535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.84899999999999998,"15min":0.86899999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1990000000000001,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":4.3630000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.89700000000000002,"15min":0.89600000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1619999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":6.157}]},{"osd":2,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.99399999999999999,"15min":1.0129999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.147,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":4.1699999999999999},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.99199999999999999,"15min":0.98799999999999999},"min":{"1min":0.68000000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.2529999999999999,"5min":1.847,"15min":1.847},"last":6.8369999999999997}]},{"osd":3,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.96499999999999997,"15min":0.96199999999999997},"min":{"1min":0.76600000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.401,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":3.5070000000000001},{"interface":"front","average":{"1min":1.052,"5min":1.157,"15min":1.175},"min":{"1min":0.68899999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2210000000000001,"5min":3.54,"15min":3.54},"last":5.2149999999999999}]},{"osd":4,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2490000000000001,"15min":1.2470000000000001},"min":{"1min":1.0720000000000001,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.6319999999999999,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":6.2000000000000002},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.175,"15min":1.175},"min":{"1min":1.006,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.4850000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":3.577}]},{"osd":5,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2170000000000001,"15min":1.218},"min":{"1min":0.98899999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.393,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":3.548},{"interface":"front","average":{"1min":1.24,"5min":1.3420000000000001,"15min":1.359},"min":{"1min":1.083,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.375,"5min":3.6760000000000002,"15min":3.6760000000000002},"last":4.4080000000000004}]},{"osd":6,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.411,"15min":1.429},"min":{"1min":1.113,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":1.4730000000000001,"5min":3.71,"15min":3.71},"last":4.4489999999999998},{"interface":"front","average":{"1min":1.282,"5min":1.3340000000000001,"15min":1.343},"min":{"1min":1.05,"5min":1.024,"15min":1.024},"max":{"1min":1.4970000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":6.9690000000000003}]},{"osd":7,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.3640000000000001,"15min":1.379},"min":{"1min":1.018,"5min":1.018,"15min":1.018},"max":{"1min":1.554,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":18.253},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2649999999999999,"15min":1.2609999999999999},"min":{"1min":1.1040000000000001,"5min":0.876,"15min":0.876},"max":{"1min":1.5449999999999999,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":11.006}]}]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297772,"kb_used_data":676,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074068,"statfs":{"total":96636764160,"available":96331845632,"internally_reserved":0,"allocated":692224,"data_stored":305856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.05,"15min":1.046},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4359999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.216,"15min":1.2370000000000001},"min":{"1min":0.69699999999999995,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.4630000000000001,"5min":3.46,"15min":3.46},"last":0.83499999999999996}]},{"osd":1,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.2310000000000001,"15min":1.2589999999999999},"min":{"1min":0.82699999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.47,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.1739999999999999,"15min":1.1879999999999999},"min":{"1min":0.80300000000000005,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.4570000000000001,"5min":3.516,"15min":3.516},"last":1.0329999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.2070000000000001,"15min":1.2130000000000001},"min":{"1min":0.91600000000000004,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.3640000000000001,"5min":5.4180000000000001,"15min":5.4180000000000001},"last":1.163},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3069999999999999,"15min":1.321},"min":{"1min":0.94799999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.5089999999999999,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":0.69699999999999995}]},{"osd":3,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.29,"15min":1.3080000000000001},"min":{"1min":1.022,"5min":0.879,"15min":0.879},"max":{"1min":1.4850000000000001,"5min":2.141,"15min":2.141},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.448,"15min":1.464},"min":{"1min":1.1839999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":1.583,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":0.81599999999999995}]},{"osd":4,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3680000000000001,"15min":1.3879999999999999},"min":{"1min":0.84199999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.5309999999999999,"5min":3.706,"15min":3.706},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.597,"15min":1.6379999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6479999999999999,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":1.212}]},{"osd":6,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.327,"15min":1.3420000000000001},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.6439999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":1.25},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.546,"15min":1.5860000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.613,"5min":3.641,"15min":3.641},"last":1.2330000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4259999999999999,"15min":1.4370000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.762,"5min":3.516,"15min":3.516},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.46,"15min":1.4650000000000001},"min":{"1min":1.139,"5min":1.022,"15min":1.022},"max":{"1min":1.722,"5min":3.754,"15min":3.754},"last":1.343}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297760,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074080,"statfs":{"total":96636764160,"available":96331857920,"internally_reserved":0,"allocated":679936,"data_stored":299635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.84999999999999998,"15min":0.84599999999999997},"min":{"1min":0.63600000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2150000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.72199999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.155,"5min":1.419,"15min":1.419},"last":0.67800000000000005}]},{"osd":2,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.85799999999999998,"15min":0.84799999999999998},"min":{"1min":0.495,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.90700000000000003,"15min":0.91000000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.161,"5min":1.341,"15min":1.341},"last":0.76500000000000001}]},{"osd":3,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.0620000000000001,"15min":1.054},"min":{"1min":0.54100000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.25},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.98899999999999999,"15min":0.97699999999999998},"min":{"1min":0.67800000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":1.23}]},{"osd":4,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.175,"15min":1.165},"min":{"1min":0.75700000000000001,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.49,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.248,"15min":1.2589999999999999},"min":{"1min":0.92000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.5609999999999999,"5min":2.206,"15min":2.206},"last":1.1799999999999999}]},{"osd":5,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.26,"15min":1.2649999999999999},"min":{"1min":1.0089999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.4299999999999999,"5min":1.899,"15min":1.899},"last":1.048},{"interface":"front","average":{"1min":1.236,"5min":1.2829999999999999,"15min":1.2909999999999999},"min":{"1min":0.92800000000000005,"5min":0.871,"15min":0.871},"max":{"1min":1.5029999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.3029999999999999}]},{"osd":6,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.323,"15min":1.325},"min":{"1min":0.98299999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.5229999999999999,"5min":1.825,"15min":1.825},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.4099999999999999,"15min":1.4119999999999999},"min":{"1min":1.1870000000000001,"5min":0.87,"15min":0.87},"max":{"1min":1.5529999999999999,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.3740000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3200000000000001,"15min":1.3169999999999999},"min":{"1min":0.93999999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.6839999999999999,"5min":1.772,"15min":1.772},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3540000000000001,"15min":1.3600000000000001},"min":{"1min":0.95499999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.6120000000000001,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.1990000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T02:59:38.917 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T02:59:39.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.101+0000 7fbc8e05b700 1 Processor -- start 2023-11-15T02:59:39.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.105+0000 7fbc8e05b700 1 -- start start 2023-11-15T02:59:39.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.105+0000 7fbc8e05b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbc881539d0 0x7fbc880c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:39.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.105+0000 7fbc8e05b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc88153db0 0x7fbc880c2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:39.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.105+0000 7fbc8e05b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc880c1530 con 0x7fbc880c19c0 2023-11-15T02:59:39.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.105+0000 7fbc8e05b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc881541f0 con 0x7fbc881539d0 2023-11-15T02:59:39.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.105+0000 7fbc8e05b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880c2b60 con 0x7fbc88153db0 2023-11-15T02:59:39.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.105+0000 7fbc877fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbc881539d0 0x7fbc880c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:39.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.105+0000 7fbc877fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbc881539d0 0x7fbc880c06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39318/0 (socket says 172.21.15.121:39318) 2023-11-15T02:59:39.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.105+0000 7fbc877fe700 1 -- 172.21.15.121:0/819554333 learned_addr learned my addr 172.21.15.121:0/819554333 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:39.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.105+0000 7fbc86ffd700 1 --2- 172.21.15.121:0/819554333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc88153db0 0x7fbc880c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:39.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.105+0000 7fbc877fe700 1 -- 172.21.15.121:0/819554333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc88153db0 msgr2=0x7fbc880c2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:39.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.105+0000 7fbc877fe700 1 --2- 172.21.15.121:0/819554333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc88153db0 0x7fbc880c2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.105+0000 7fbc877fe700 1 -- 172.21.15.121:0/819554333 >> v1:172.21.15.190:6789/0 conn(0x7fbc880c19c0 legacy=0x7fbc880c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:39.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.105+0000 7fbc877fe700 1 -- 172.21.15.121:0/819554333 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc880c2d80 con 0x7fbc881539d0 2023-11-15T02:59:39.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.105+0000 7fbc86ffd700 1 --2- 172.21.15.121:0/819554333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc88153db0 0x7fbc880c2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:39.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.109+0000 7fbc877fe700 1 --2- 172.21.15.121:0/819554333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc881539d0 0x7fbc880c06e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7fbc78006fd0 tx=0x7fbc780090f0).ready entity=mon.0 client_cookie=ead8b7d3c8ba20aa server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:39.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.109+0000 7fbc867fc700 1 -- 172.21.15.121:0/819554333 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7800b600 con 0x7fbc881539d0 2023-11-15T02:59:39.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.109+0000 7fbc867fc700 1 -- 172.21.15.121:0/819554333 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7800b760 con 0x7fbc881539d0 2023-11-15T02:59:39.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.109+0000 7fbc867fc700 1 -- 172.21.15.121:0/819554333 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7800bae0 con 0x7fbc881539d0 2023-11-15T02:59:39.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.109+0000 7fbc8e05b700 1 -- 172.21.15.121:0/819554333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc881539d0 msgr2=0x7fbc880c06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:39.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.109+0000 7fbc8e05b700 1 --2- 172.21.15.121:0/819554333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc881539d0 0x7fbc880c06e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7fbc78006fd0 tx=0x7fbc780090f0).stop 2023-11-15T02:59:39.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.109+0000 7fbc8e05b700 1 -- 172.21.15.121:0/819554333 shutdown_connections 2023-11-15T02:59:39.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.109+0000 7fbc8e05b700 1 --2- 172.21.15.121:0/819554333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc88153db0 0x7fbc880c2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.109+0000 7fbc8e05b700 1 --2- 172.21.15.121:0/819554333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc881539d0 0x7fbc880c06e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.109+0000 7fbc8e05b700 1 -- 172.21.15.121:0/819554333 >> 172.21.15.121:0/819554333 conn(0x7fbc88062190 msgr2=0x7fbc880625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:39.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.109+0000 7fbc8e05b700 1 -- 172.21.15.121:0/819554333 shutdown_connections 2023-11-15T02:59:39.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.109+0000 7fbc8e05b700 1 -- 172.21.15.121:0/819554333 wait complete. 2023-11-15T02:59:39.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc8e05b700 1 Processor -- start 2023-11-15T02:59:39.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc8e05b700 1 -- start start 2023-11-15T02:59:39.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc8e05b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc880c19c0 0x7fbc88164ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:39.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc8e05b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc881539d0 0x7fbc8814d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:39.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc8e05b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc88153db0 0x7fbc88154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:39.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc8e05b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc88157540 con 0x7fbc880c19c0 2023-11-15T02:59:39.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc8e05b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc881653e0 con 0x7fbc88153db0 2023-11-15T02:59:39.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc8e05b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc88156930 con 0x7fbc881539d0 2023-11-15T02:59:39.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc877fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc880c19c0 0x7fbc88164ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:39.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc877fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc880c19c0 0x7fbc88164ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39326/0 (socket says 172.21.15.121:39326) 2023-11-15T02:59:39.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc877fe700 1 -- 172.21.15.121:0/3336251791 learned_addr learned my addr 172.21.15.121:0/3336251791 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:39.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc86ffd700 1 --2- 172.21.15.121:0/3336251791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc881539d0 0x7fbc8814d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:39.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc87fff700 1 --2- 172.21.15.121:0/3336251791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc88153db0 0x7fbc88154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:39.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc877fe700 1 -- 172.21.15.121:0/3336251791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc881539d0 msgr2=0x7fbc8814d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:39.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc877fe700 1 --2- 172.21.15.121:0/3336251791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc881539d0 0x7fbc8814d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc877fe700 1 -- 172.21.15.121:0/3336251791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc88153db0 msgr2=0x7fbc88154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:39.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc877fe700 1 --2- 172.21.15.121:0/3336251791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc88153db0 0x7fbc88154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc877fe700 1 -- 172.21.15.121:0/3336251791 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc78009370 con 0x7fbc880c19c0 2023-11-15T02:59:39.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc87fff700 1 --2- 172.21.15.121:0/3336251791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc88153db0 0x7fbc88154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:39.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc877fe700 1 --2- 172.21.15.121:0/3336251791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc880c19c0 0x7fbc88164ef0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7fbc78011fb0 tx=0x7fbc78017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:39.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc6f7fe700 1 -- 172.21.15.121:0/3336251791 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7800b8d0 con 0x7fbc880c19c0 2023-11-15T02:59:39.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc6f7fe700 1 -- 172.21.15.121:0/3336251791 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7800ba30 con 0x7fbc880c19c0 2023-11-15T02:59:39.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc6f7fe700 1 -- 172.21.15.121:0/3336251791 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78024680 con 0x7fbc880c19c0 2023-11-15T02:59:39.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc8e05b700 1 -- 172.21.15.121:0/3336251791 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc88156c10 con 0x7fbc880c19c0 2023-11-15T02:59:39.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.113+0000 7fbc8e05b700 1 -- 172.21.15.121:0/3336251791 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc881571d0 con 0x7fbc880c19c0 2023-11-15T02:59:39.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.117+0000 7fbc8e05b700 1 -- 172.21.15.121:0/3336251791 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc8806c150 con 0x7fbc880c19c0 2023-11-15T02:59:39.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.117+0000 7fbc6f7fe700 1 -- 172.21.15.121:0/3336251791 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fbc78021070 con 0x7fbc880c19c0 2023-11-15T02:59:39.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.117+0000 7fbc6f7fe700 1 --2- 172.21.15.121:0/3336251791 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbc7405a540 0x7fbc7405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:39.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.117+0000 7fbc6f7fe700 1 -- 172.21.15.121:0/3336251791 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fbc78076170 con 0x7fbc880c19c0 2023-11-15T02:59:39.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.121+0000 7fbc86ffd700 1 --2- 172.21.15.121:0/3336251791 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbc7405a540 0x7fbc7405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:39.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.121+0000 7fbc86ffd700 1 --2- 172.21.15.121:0/3336251791 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbc7405a540 0x7fbc7405ca00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fbc881589d0 tx=0x7fbc7c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:39.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.125+0000 7fbc6f7fe700 1 -- 172.21.15.121:0/3336251791 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbc78049890 con 0x7fbc880c19c0 2023-11-15T02:59:39.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.285+0000 7fbc8e05b700 1 -- 172.21.15.121:0/3336251791 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbc880c0ee0 con 0x7fbc7405a540 2023-11-15T02:59:39.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.293+0000 7fbc6f7fe700 1 -- 172.21.15.121:0/3336251791 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+355564 (secure 0 0 0) 0x7fbc880c0ee0 con 0x7fbc7405a540 2023-11-15T02:59:39.309 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:39.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.301+0000 7fbc8e05b700 1 -- 172.21.15.121:0/3336251791 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbc7405a540 msgr2=0x7fbc7405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:39.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.301+0000 7fbc8e05b700 1 --2- 172.21.15.121:0/3336251791 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbc7405a540 0x7fbc7405ca00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fbc881589d0 tx=0x7fbc7c008040).stop 2023-11-15T02:59:39.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.301+0000 7fbc8e05b700 1 -- 172.21.15.121:0/3336251791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc880c19c0 msgr2=0x7fbc88164ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:39.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.301+0000 7fbc8e05b700 1 --2- 172.21.15.121:0/3336251791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc880c19c0 0x7fbc88164ef0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7fbc78011fb0 tx=0x7fbc78017000).stop 2023-11-15T02:59:39.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.301+0000 7fbc8e05b700 1 -- 172.21.15.121:0/3336251791 shutdown_connections 2023-11-15T02:59:39.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.301+0000 7fbc8e05b700 1 --2- 172.21.15.121:0/3336251791 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbc7405a540 0x7fbc7405ca00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.301+0000 7fbc8e05b700 1 --2- 172.21.15.121:0/3336251791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc88153db0 0x7fbc88154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.301+0000 7fbc8e05b700 1 --2- 172.21.15.121:0/3336251791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc881539d0 0x7fbc8814d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.301+0000 7fbc8e05b700 1 --2- 172.21.15.121:0/3336251791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc880c19c0 0x7fbc88164ef0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.301+0000 7fbc8e05b700 1 -- 172.21.15.121:0/3336251791 >> 172.21.15.121:0/3336251791 conn(0x7fbc88062190 msgr2=0x7fbc88152c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:39.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.301+0000 7fbc8e05b700 1 -- 172.21.15.121:0/3336251791 shutdown_connections 2023-11-15T02:59:39.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.301+0000 7fbc8e05b700 1 -- 172.21.15.121:0/3336251791 wait complete. 2023-11-15T02:59:39.315 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T02:59:39.332 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2023-11-15T02:59:37.971314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":748,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":226,"acting":226,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382264,"kb_used_data":5496,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752592456,"statfs":{"total":773094113280,"available":770654674944,"internally_reserved":0,"allocated":5627904,"data_stored":2493914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12861,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-20,"num_read_kb":-23,"num_write":-72,"num_write_kb":-35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.466488"},"pg_stats":[{"pgid":"10.3c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.702569+0000","last_change":"2023-11-15T02:59:34.702569+0000","last_active":"2023-11-15T02:59:34.702569+0000","last_peered":"2023-11-15T02:59:34.702569+0000","last_clean":"2023-11-15T02:59:34.702569+0000","last_became_active":"2023-11-15T02:59:34.702272+0000","last_became_peered":"2023-11-15T02:59:34.702272+0000","last_unstale":"2023-11-15T02:59:34.702569+0000","last_undegraded":"2023-11-15T02:59:34.702569+0000","last_fullsized":"2023-11-15T02:59:34.702569+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.3f","version":"60'3","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.702378+0000","last_change":"2023-11-15T02:59:34.265036+0000","last_active":"2023-11-15T02:59:34.702378+0000","last_peered":"2023-11-15T02:59:34.702378+0000","last_clean":"2023-11-15T02:59:34.702378+0000","last_became_active":"2023-11-15T02:59:34.264631+0000","last_became_peered":"2023-11-15T02:59:34.264631+0000","last_unstale":"2023-11-15T02:59:34.702378+0000","last_undegraded":"2023-11-15T02:59:34.702378+0000","last_fullsized":"2023-11-15T02:59:34.702378+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.715077+0000","last_change":"2023-11-15T02:59:34.715077+0000","last_active":"2023-11-15T02:59:34.715077+0000","last_peered":"2023-11-15T02:59:34.715077+0000","last_clean":"2023-11-15T02:59:34.715077+0000","last_became_active":"2023-11-15T02:59:34.710308+0000","last_became_peered":"2023-11-15T02:59:34.710308+0000","last_unstale":"2023-11-15T02:59:34.715077+0000","last_undegraded":"2023-11-15T02:59:34.715077+0000","last_fullsized":"2023-11-15T02:59:34.715077+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.280938+0000","last_change":"2023-11-15T02:59:34.274806+0000","last_active":"2023-11-15T02:59:34.280938+0000","last_peered":"2023-11-15T02:59:34.280938+0000","last_clean":"2023-11-15T02:59:34.280938+0000","last_became_active":"2023-11-15T02:59:34.274536+0000","last_became_peered":"2023-11-15T02:59:34.274536+0000","last_unstale":"2023-11-15T02:59:34.280938+0000","last_undegraded":"2023-11-15T02:59:34.280938+0000","last_fullsized":"2023-11-15T02:59:34.280938+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.276406+0000","last_change":"2023-11-15T02:59:34.272252+0000","last_active":"2023-11-15T02:59:34.276406+0000","last_peered":"2023-11-15T02:59:34.276406+0000","last_clean":"2023-11-15T02:59:34.276406+0000","last_became_active":"2023-11-15T02:59:34.271979+0000","last_became_peered":"2023-11-15T02:59:34.271979+0000","last_unstale":"2023-11-15T02:59:34.276406+0000","last_undegraded":"2023-11-15T02:59:34.276406+0000","last_fullsized":"2023-11-15T02:59:34.276406+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.231018+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.694905+0000","last_change":"2023-11-15T02:59:34.263921+0000","last_active":"2023-11-15T02:59:34.694905+0000","last_peered":"2023-11-15T02:59:34.694905+0000","last_clean":"2023-11-15T02:59:34.694905+0000","last_became_active":"2023-11-15T02:59:34.261031+0000","last_became_peered":"2023-11-15T02:59:34.261031+0000","last_unstale":"2023-11-15T02:59:34.694905+0000","last_undegraded":"2023-11-15T02:59:34.694905+0000","last_fullsized":"2023-11-15T02:59:34.694905+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716845+0000","last_change":"2023-11-15T02:59:34.716845+0000","last_active":"2023-11-15T02:59:34.716845+0000","last_peered":"2023-11-15T02:59:34.716845+0000","last_clean":"2023-11-15T02:59:34.716845+0000","last_became_active":"2023-11-15T02:59:34.716558+0000","last_became_peered":"2023-11-15T02:59:34.716558+0000","last_unstale":"2023-11-15T02:59:34.716845+0000","last_undegraded":"2023-11-15T02:59:34.716845+0000","last_fullsized":"2023-11-15T02:59:34.716845+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.690603+0000","last_change":"2023-11-15T02:59:34.266240+0000","last_active":"2023-11-15T02:59:34.690603+0000","last_peered":"2023-11-15T02:59:34.690603+0000","last_clean":"2023-11-15T02:59:34.690603+0000","last_became_active":"2023-11-15T02:59:34.265188+0000","last_became_peered":"2023-11-15T02:59:34.265188+0000","last_unstale":"2023-11-15T02:59:34.690603+0000","last_undegraded":"2023-11-15T02:59:34.690603+0000","last_fullsized":"2023-11-15T02:59:34.690603+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.719027+0000","last_change":"2023-11-15T02:59:34.719027+0000","last_active":"2023-11-15T02:59:34.719027+0000","last_peered":"2023-11-15T02:59:34.719027+0000","last_clean":"2023-11-15T02:59:34.719027+0000","last_became_active":"2023-11-15T02:59:34.718717+0000","last_became_peered":"2023-11-15T02:59:34.718717+0000","last_unstale":"2023-11-15T02:59:34.719027+0000","last_undegraded":"2023-11-15T02:59:34.719027+0000","last_fullsized":"2023-11-15T02:59:34.719027+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.273500+0000","last_change":"2023-11-15T02:59:34.263881+0000","last_active":"2023-11-15T02:59:34.273500+0000","last_peered":"2023-11-15T02:59:34.273500+0000","last_clean":"2023-11-15T02:59:34.273500+0000","last_became_active":"2023-11-15T02:59:34.263571+0000","last_became_peered":"2023-11-15T02:59:34.263571+0000","last_unstale":"2023-11-15T02:59:34.273500+0000","last_undegraded":"2023-11-15T02:59:34.273500+0000","last_fullsized":"2023-11-15T02:59:34.273500+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.715365+0000","last_change":"2023-11-15T02:59:34.715365+0000","last_active":"2023-11-15T02:59:34.715365+0000","last_peered":"2023-11-15T02:59:34.715365+0000","last_clean":"2023-11-15T02:59:34.715365+0000","last_became_active":"2023-11-15T02:59:34.715043+0000","last_became_peered":"2023-11-15T02:59:34.715043+0000","last_unstale":"2023-11-15T02:59:34.715365+0000","last_undegraded":"2023-11-15T02:59:34.715365+0000","last_fullsized":"2023-11-15T02:59:34.715365+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.271621+0000","last_change":"2023-11-15T02:59:34.264832+0000","last_active":"2023-11-15T02:59:34.271621+0000","last_peered":"2023-11-15T02:59:34.271621+0000","last_clean":"2023-11-15T02:59:34.271621+0000","last_became_active":"2023-11-15T02:59:34.264571+0000","last_became_peered":"2023-11-15T02:59:34.264571+0000","last_unstale":"2023-11-15T02:59:34.271621+0000","last_undegraded":"2023-11-15T02:59:34.271621+0000","last_fullsized":"2023-11-15T02:59:34.271621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.270817+0000","last_change":"2023-11-15T02:59:34.262030+0000","last_active":"2023-11-15T02:59:34.270817+0000","last_peered":"2023-11-15T02:59:34.270817+0000","last_clean":"2023-11-15T02:59:34.270817+0000","last_became_active":"2023-11-15T02:59:34.261689+0000","last_became_peered":"2023-11-15T02:59:34.261689+0000","last_unstale":"2023-11-15T02:59:34.270817+0000","last_undegraded":"2023-11-15T02:59:34.270817+0000","last_fullsized":"2023-11-15T02:59:34.270817+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.701348+0000","last_change":"2023-11-15T02:59:34.701348+0000","last_active":"2023-11-15T02:59:34.701348+0000","last_peered":"2023-11-15T02:59:34.701348+0000","last_clean":"2023-11-15T02:59:34.701348+0000","last_became_active":"2023-11-15T02:59:34.701079+0000","last_became_peered":"2023-11-15T02:59:34.701079+0000","last_unstale":"2023-11-15T02:59:34.701348+0000","last_undegraded":"2023-11-15T02:59:34.701348+0000","last_fullsized":"2023-11-15T02:59:34.701348+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.207077+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.707253+0000","last_change":"2023-11-15T02:59:34.707253+0000","last_active":"2023-11-15T02:59:34.707253+0000","last_peered":"2023-11-15T02:59:34.707253+0000","last_clean":"2023-11-15T02:59:34.707253+0000","last_became_active":"2023-11-15T02:59:34.706969+0000","last_became_peered":"2023-11-15T02:59:34.706969+0000","last_unstale":"2023-11-15T02:59:34.707253+0000","last_undegraded":"2023-11-15T02:59:34.707253+0000","last_fullsized":"2023-11-15T02:59:34.707253+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.207077+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.717917+0000","last_change":"2023-11-15T02:59:34.717917+0000","last_active":"2023-11-15T02:59:34.717917+0000","last_peered":"2023-11-15T02:59:34.717917+0000","last_clean":"2023-11-15T02:59:34.717917+0000","last_became_active":"2023-11-15T02:59:34.717573+0000","last_became_peered":"2023-11-15T02:59:34.717573+0000","last_unstale":"2023-11-15T02:59:34.717917+0000","last_undegraded":"2023-11-15T02:59:34.717917+0000","last_fullsized":"2023-11-15T02:59:34.717917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.229134+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.709579+0000","last_change":"2023-11-15T02:59:34.709579+0000","last_active":"2023-11-15T02:59:34.709579+0000","last_peered":"2023-11-15T02:59:34.709579+0000","last_clean":"2023-11-15T02:59:34.709579+0000","last_became_active":"2023-11-15T02:59:34.709324+0000","last_became_peered":"2023-11-15T02:59:34.709324+0000","last_unstale":"2023-11-15T02:59:34.709579+0000","last_undegraded":"2023-11-15T02:59:34.709579+0000","last_fullsized":"2023-11-15T02:59:34.709579+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.33","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.689591+0000","last_change":"2023-11-15T02:59:34.261668+0000","last_active":"2023-11-15T02:59:34.689591+0000","last_peered":"2023-11-15T02:59:34.689591+0000","last_clean":"2023-11-15T02:59:34.689591+0000","last_became_active":"2023-11-15T02:59:34.261392+0000","last_became_peered":"2023-11-15T02:59:34.261392+0000","last_unstale":"2023-11-15T02:59:34.689591+0000","last_undegraded":"2023-11-15T02:59:34.689591+0000","last_fullsized":"2023-11-15T02:59:34.689591+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.717458+0000","last_change":"2023-11-15T02:59:34.717458+0000","last_active":"2023-11-15T02:59:34.717458+0000","last_peered":"2023-11-15T02:59:34.717458+0000","last_clean":"2023-11-15T02:59:34.717458+0000","last_became_active":"2023-11-15T02:59:34.717124+0000","last_became_peered":"2023-11-15T02:59:34.717124+0000","last_unstale":"2023-11-15T02:59:34.717458+0000","last_undegraded":"2023-11-15T02:59:34.717458+0000","last_fullsized":"2023-11-15T02:59:34.717458+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.274573+0000","last_change":"2023-11-15T02:59:34.267337+0000","last_active":"2023-11-15T02:59:34.274573+0000","last_peered":"2023-11-15T02:59:34.274573+0000","last_clean":"2023-11-15T02:59:34.274573+0000","last_became_active":"2023-11-15T02:59:34.267054+0000","last_became_peered":"2023-11-15T02:59:34.267054+0000","last_unstale":"2023-11-15T02:59:34.274573+0000","last_undegraded":"2023-11-15T02:59:34.274573+0000","last_fullsized":"2023-11-15T02:59:34.274573+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.709174+0000","last_change":"2023-11-15T02:59:34.709174+0000","last_active":"2023-11-15T02:59:34.709174+0000","last_peered":"2023-11-15T02:59:34.709174+0000","last_clean":"2023-11-15T02:59:34.709174+0000","last_became_active":"2023-11-15T02:59:34.708872+0000","last_became_peered":"2023-11-15T02:59:34.708872+0000","last_unstale":"2023-11-15T02:59:34.709174+0000","last_undegraded":"2023-11-15T02:59:34.709174+0000","last_fullsized":"2023-11-15T02:59:34.709174+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.270777+0000","last_change":"2023-11-15T02:59:34.261461+0000","last_active":"2023-11-15T02:59:34.270777+0000","last_peered":"2023-11-15T02:59:34.270777+0000","last_clean":"2023-11-15T02:59:34.270777+0000","last_became_active":"2023-11-15T02:59:34.261153+0000","last_became_peered":"2023-11-15T02:59:34.261153+0000","last_unstale":"2023-11-15T02:59:34.270777+0000","last_undegraded":"2023-11-15T02:59:34.270777+0000","last_fullsized":"2023-11-15T02:59:34.270777+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"60'4","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.721503+0000","last_change":"2023-11-15T02:59:34.264405+0000","last_active":"2023-11-15T02:59:34.721503+0000","last_peered":"2023-11-15T02:59:34.721503+0000","last_clean":"2023-11-15T02:59:34.721503+0000","last_became_active":"2023-11-15T02:59:34.264095+0000","last_became_peered":"2023-11-15T02:59:34.264095+0000","last_unstale":"2023-11-15T02:59:34.721503+0000","last_undegraded":"2023-11-15T02:59:34.721503+0000","last_fullsized":"2023-11-15T02:59:34.721503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.694211+0000","last_change":"2023-11-15T02:59:34.694211+0000","last_active":"2023-11-15T02:59:34.694211+0000","last_peered":"2023-11-15T02:59:34.694211+0000","last_clean":"2023-11-15T02:59:34.694211+0000","last_became_active":"2023-11-15T02:59:34.693708+0000","last_became_peered":"2023-11-15T02:59:34.693708+0000","last_unstale":"2023-11-15T02:59:34.694211+0000","last_undegraded":"2023-11-15T02:59:34.694211+0000","last_fullsized":"2023-11-15T02:59:34.694211+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278070+0000","last_change":"2023-11-15T02:59:34.271015+0000","last_active":"2023-11-15T02:59:34.278070+0000","last_peered":"2023-11-15T02:59:34.278070+0000","last_clean":"2023-11-15T02:59:34.278070+0000","last_became_active":"2023-11-15T02:59:34.270758+0000","last_became_peered":"2023-11-15T02:59:34.270758+0000","last_unstale":"2023-11-15T02:59:34.278070+0000","last_undegraded":"2023-11-15T02:59:34.278070+0000","last_fullsized":"2023-11-15T02:59:34.278070+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.721266+0000","last_change":"2023-11-15T02:59:34.721266+0000","last_active":"2023-11-15T02:59:34.721266+0000","last_peered":"2023-11-15T02:59:34.721266+0000","last_clean":"2023-11-15T02:59:34.721266+0000","last_became_active":"2023-11-15T02:59:34.720978+0000","last_became_peered":"2023-11-15T02:59:34.720978+0000","last_unstale":"2023-11-15T02:59:34.721266+0000","last_undegraded":"2023-11-15T02:59:34.721266+0000","last_fullsized":"2023-11-15T02:59:34.721266+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.276457+0000","last_change":"2023-11-15T02:59:34.269316+0000","last_active":"2023-11-15T02:59:34.276457+0000","last_peered":"2023-11-15T02:59:34.276457+0000","last_clean":"2023-11-15T02:59:34.276457+0000","last_became_active":"2023-11-15T02:59:34.268953+0000","last_became_peered":"2023-11-15T02:59:34.268953+0000","last_unstale":"2023-11-15T02:59:34.276457+0000","last_undegraded":"2023-11-15T02:59:34.276457+0000","last_fullsized":"2023-11-15T02:59:34.276457+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.703052+0000","last_change":"2023-11-15T02:59:34.703052+0000","last_active":"2023-11-15T02:59:34.703052+0000","last_peered":"2023-11-15T02:59:34.703052+0000","last_clean":"2023-11-15T02:59:34.703052+0000","last_became_active":"2023-11-15T02:59:34.702791+0000","last_became_peered":"2023-11-15T02:59:34.702791+0000","last_unstale":"2023-11-15T02:59:34.703052+0000","last_undegraded":"2023-11-15T02:59:34.703052+0000","last_fullsized":"2023-11-15T02:59:34.703052+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2d","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.696289+0000","last_change":"2023-11-15T02:59:34.257033+0000","last_active":"2023-11-15T02:59:34.696289+0000","last_peered":"2023-11-15T02:59:34.696289+0000","last_clean":"2023-11-15T02:59:34.696289+0000","last_became_active":"2023-11-15T02:59:34.256744+0000","last_became_peered":"2023-11-15T02:59:34.256744+0000","last_unstale":"2023-11-15T02:59:34.696289+0000","last_undegraded":"2023-11-15T02:59:34.696289+0000","last_fullsized":"2023-11-15T02:59:34.696289+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.280852+0000","last_change":"2023-11-15T02:59:34.275824+0000","last_active":"2023-11-15T02:59:34.280852+0000","last_peered":"2023-11-15T02:59:34.280852+0000","last_clean":"2023-11-15T02:59:34.280852+0000","last_became_active":"2023-11-15T02:59:34.275088+0000","last_became_peered":"2023-11-15T02:59:34.275088+0000","last_unstale":"2023-11-15T02:59:34.280852+0000","last_undegraded":"2023-11-15T02:59:34.280852+0000","last_fullsized":"2023-11-15T02:59:34.280852+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.718638+0000","last_change":"2023-11-15T02:59:34.718638+0000","last_active":"2023-11-15T02:59:34.718638+0000","last_peered":"2023-11-15T02:59:34.718638+0000","last_clean":"2023-11-15T02:59:34.718638+0000","last_became_active":"2023-11-15T02:59:34.718018+0000","last_became_peered":"2023-11-15T02:59:34.718018+0000","last_unstale":"2023-11-15T02:59:34.718638+0000","last_undegraded":"2023-11-15T02:59:34.718638+0000","last_fullsized":"2023-11-15T02:59:34.718638+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2b","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.688086+0000","last_change":"2023-11-15T02:59:34.271365+0000","last_active":"2023-11-15T02:59:34.688086+0000","last_peered":"2023-11-15T02:59:34.688086+0000","last_clean":"2023-11-15T02:59:34.688086+0000","last_became_active":"2023-11-15T02:59:34.270414+0000","last_became_peered":"2023-11-15T02:59:34.270414+0000","last_unstale":"2023-11-15T02:59:34.688086+0000","last_undegraded":"2023-11-15T02:59:34.688086+0000","last_fullsized":"2023-11-15T02:59:34.688086+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.705469+0000","last_change":"2023-11-15T02:59:34.705469+0000","last_active":"2023-11-15T02:59:34.705469+0000","last_peered":"2023-11-15T02:59:34.705469+0000","last_clean":"2023-11-15T02:59:34.705469+0000","last_became_active":"2023-11-15T02:59:34.705188+0000","last_became_peered":"2023-11-15T02:59:34.705188+0000","last_unstale":"2023-11-15T02:59:34.705469+0000","last_undegraded":"2023-11-15T02:59:34.705469+0000","last_fullsized":"2023-11-15T02:59:34.705469+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.274106+0000","last_change":"2023-11-15T02:59:34.269772+0000","last_active":"2023-11-15T02:59:34.274106+0000","last_peered":"2023-11-15T02:59:34.274106+0000","last_clean":"2023-11-15T02:59:34.274106+0000","last_became_active":"2023-11-15T02:59:34.269150+0000","last_became_peered":"2023-11-15T02:59:34.269150+0000","last_unstale":"2023-11-15T02:59:34.274106+0000","last_undegraded":"2023-11-15T02:59:34.274106+0000","last_fullsized":"2023-11-15T02:59:34.274106+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.706266+0000","last_change":"2023-11-15T02:59:34.706266+0000","last_active":"2023-11-15T02:59:34.706266+0000","last_peered":"2023-11-15T02:59:34.706266+0000","last_clean":"2023-11-15T02:59:34.706266+0000","last_became_active":"2023-11-15T02:59:34.705045+0000","last_became_peered":"2023-11-15T02:59:34.705045+0000","last_unstale":"2023-11-15T02:59:34.706266+0000","last_undegraded":"2023-11-15T02:59:34.706266+0000","last_fullsized":"2023-11-15T02:59:34.706266+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.275956+0000","last_change":"2023-11-15T02:59:34.271476+0000","last_active":"2023-11-15T02:59:34.275956+0000","last_peered":"2023-11-15T02:59:34.275956+0000","last_clean":"2023-11-15T02:59:34.275956+0000","last_became_active":"2023-11-15T02:59:34.271135+0000","last_became_peered":"2023-11-15T02:59:34.271135+0000","last_unstale":"2023-11-15T02:59:34.275956+0000","last_undegraded":"2023-11-15T02:59:34.275956+0000","last_fullsized":"2023-11-15T02:59:34.275956+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.268460+0000","last_change":"2023-11-15T02:59:34.264047+0000","last_active":"2023-11-15T02:59:34.268460+0000","last_peered":"2023-11-15T02:59:34.268460+0000","last_clean":"2023-11-15T02:59:34.268460+0000","last_became_active":"2023-11-15T02:59:34.263525+0000","last_became_peered":"2023-11-15T02:59:34.263525+0000","last_unstale":"2023-11-15T02:59:34.268460+0000","last_undegraded":"2023-11-15T02:59:34.268460+0000","last_fullsized":"2023-11-15T02:59:34.268460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.264845+0000","last_change":"2023-11-15T02:59:34.253441+0000","last_active":"2023-11-15T02:59:34.264845+0000","last_peered":"2023-11-15T02:59:34.264845+0000","last_clean":"2023-11-15T02:59:34.264845+0000","last_became_active":"2023-11-15T02:59:34.253172+0000","last_became_peered":"2023-11-15T02:59:34.253172+0000","last_unstale":"2023-11-15T02:59:34.264845+0000","last_undegraded":"2023-11-15T02:59:34.264845+0000","last_fullsized":"2023-11-15T02:59:34.264845+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.706426+0000","last_change":"2023-11-15T02:59:34.706426+0000","last_active":"2023-11-15T02:59:34.706426+0000","last_peered":"2023-11-15T02:59:34.706426+0000","last_clean":"2023-11-15T02:59:34.706426+0000","last_became_active":"2023-11-15T02:59:34.705567+0000","last_became_peered":"2023-11-15T02:59:34.705567+0000","last_unstale":"2023-11-15T02:59:34.706426+0000","last_undegraded":"2023-11-15T02:59:34.706426+0000","last_fullsized":"2023-11-15T02:59:34.706426+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.229642+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.275473+0000","last_change":"2023-11-15T02:59:34.264728+0000","last_active":"2023-11-15T02:59:34.275473+0000","last_peered":"2023-11-15T02:59:34.275473+0000","last_clean":"2023-11-15T02:59:34.275473+0000","last_became_active":"2023-11-15T02:59:34.264416+0000","last_became_peered":"2023-11-15T02:59:34.264416+0000","last_unstale":"2023-11-15T02:59:34.275473+0000","last_undegraded":"2023-11-15T02:59:34.275473+0000","last_fullsized":"2023-11-15T02:59:34.275473+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.270389+0000","last_change":"2023-11-15T02:59:34.261680+0000","last_active":"2023-11-15T02:59:34.270389+0000","last_peered":"2023-11-15T02:59:34.270389+0000","last_clean":"2023-11-15T02:59:34.270389+0000","last_became_active":"2023-11-15T02:59:34.260538+0000","last_became_peered":"2023-11-15T02:59:34.260538+0000","last_unstale":"2023-11-15T02:59:34.270389+0000","last_undegraded":"2023-11-15T02:59:34.270389+0000","last_fullsized":"2023-11-15T02:59:34.270389+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.707619+0000","last_change":"2023-11-15T02:59:34.707619+0000","last_active":"2023-11-15T02:59:34.707619+0000","last_peered":"2023-11-15T02:59:34.707619+0000","last_clean":"2023-11-15T02:59:34.707619+0000","last_became_active":"2023-11-15T02:59:34.707297+0000","last_became_peered":"2023-11-15T02:59:34.707297+0000","last_unstale":"2023-11-15T02:59:34.707619+0000","last_undegraded":"2023-11-15T02:59:34.707619+0000","last_fullsized":"2023-11-15T02:59:34.707619+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.272192+0000","last_change":"2023-11-15T02:59:34.266701+0000","last_active":"2023-11-15T02:59:34.272192+0000","last_peered":"2023-11-15T02:59:34.272192+0000","last_clean":"2023-11-15T02:59:34.272192+0000","last_became_active":"2023-11-15T02:59:34.266347+0000","last_became_peered":"2023-11-15T02:59:34.266347+0000","last_unstale":"2023-11-15T02:59:34.272192+0000","last_undegraded":"2023-11-15T02:59:34.272192+0000","last_fullsized":"2023-11-15T02:59:34.272192+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.705873+0000","last_change":"2023-11-15T02:59:34.705873+0000","last_active":"2023-11-15T02:59:34.705873+0000","last_peered":"2023-11-15T02:59:34.705873+0000","last_clean":"2023-11-15T02:59:34.705873+0000","last_became_active":"2023-11-15T02:59:34.705522+0000","last_became_peered":"2023-11-15T02:59:34.705522+0000","last_unstale":"2023-11-15T02:59:34.705873+0000","last_undegraded":"2023-11-15T02:59:34.705873+0000","last_fullsized":"2023-11-15T02:59:34.705873+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278740+0000","last_change":"2023-11-15T02:59:34.274845+0000","last_active":"2023-11-15T02:59:34.278740+0000","last_peered":"2023-11-15T02:59:34.278740+0000","last_clean":"2023-11-15T02:59:34.278740+0000","last_became_active":"2023-11-15T02:59:34.274495+0000","last_became_peered":"2023-11-15T02:59:34.274495+0000","last_unstale":"2023-11-15T02:59:34.278740+0000","last_undegraded":"2023-11-15T02:59:34.278740+0000","last_fullsized":"2023-11-15T02:59:34.278740+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.271334+0000","last_change":"2023-11-15T02:59:34.262355+0000","last_active":"2023-11-15T02:59:34.271334+0000","last_peered":"2023-11-15T02:59:34.271334+0000","last_clean":"2023-11-15T02:59:34.271334+0000","last_became_active":"2023-11-15T02:59:34.260844+0000","last_became_peered":"2023-11-15T02:59:34.260844+0000","last_unstale":"2023-11-15T02:59:34.271334+0000","last_undegraded":"2023-11-15T02:59:34.271334+0000","last_fullsized":"2023-11-15T02:59:34.271334+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.711149+0000","last_change":"2023-11-15T02:59:34.711149+0000","last_active":"2023-11-15T02:59:34.711149+0000","last_peered":"2023-11-15T02:59:34.711149+0000","last_clean":"2023-11-15T02:59:34.711149+0000","last_became_active":"2023-11-15T02:59:34.710867+0000","last_became_peered":"2023-11-15T02:59:34.710867+0000","last_unstale":"2023-11-15T02:59:34.711149+0000","last_undegraded":"2023-11-15T02:59:34.711149+0000","last_fullsized":"2023-11-15T02:59:34.711149+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278038+0000","last_change":"2023-11-15T02:59:34.276394+0000","last_active":"2023-11-15T02:59:34.278038+0000","last_peered":"2023-11-15T02:59:34.278038+0000","last_clean":"2023-11-15T02:59:34.278038+0000","last_became_active":"2023-11-15T02:59:34.276134+0000","last_became_peered":"2023-11-15T02:59:34.276134+0000","last_unstale":"2023-11-15T02:59:34.278038+0000","last_undegraded":"2023-11-15T02:59:34.278038+0000","last_fullsized":"2023-11-15T02:59:34.278038+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.697057+0000","last_change":"2023-11-15T02:59:34.697057+0000","last_active":"2023-11-15T02:59:34.697057+0000","last_peered":"2023-11-15T02:59:34.697057+0000","last_clean":"2023-11-15T02:59:34.697057+0000","last_became_active":"2023-11-15T02:59:34.696211+0000","last_became_peered":"2023-11-15T02:59:34.696211+0000","last_unstale":"2023-11-15T02:59:34.697057+0000","last_undegraded":"2023-11-15T02:59:34.697057+0000","last_fullsized":"2023-11-15T02:59:34.697057+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.207077+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.695873+0000","last_change":"2023-11-15T02:59:34.695873+0000","last_active":"2023-11-15T02:59:34.695873+0000","last_peered":"2023-11-15T02:59:34.695873+0000","last_clean":"2023-11-15T02:59:34.695873+0000","last_became_active":"2023-11-15T02:59:34.695590+0000","last_became_peered":"2023-11-15T02:59:34.695590+0000","last_unstale":"2023-11-15T02:59:34.695873+0000","last_undegraded":"2023-11-15T02:59:34.695873+0000","last_fullsized":"2023-11-15T02:59:34.695873+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.693584+0000","last_change":"2023-11-15T02:59:34.272647+0000","last_active":"2023-11-15T02:59:34.693584+0000","last_peered":"2023-11-15T02:59:34.693584+0000","last_clean":"2023-11-15T02:59:34.693584+0000","last_became_active":"2023-11-15T02:59:34.272331+0000","last_became_peered":"2023-11-15T02:59:34.272331+0000","last_unstale":"2023-11-15T02:59:34.693584+0000","last_undegraded":"2023-11-15T02:59:34.693584+0000","last_fullsized":"2023-11-15T02:59:34.693584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.710364+0000","last_change":"2023-11-15T02:59:34.710364+0000","last_active":"2023-11-15T02:59:34.710364+0000","last_peered":"2023-11-15T02:59:34.710364+0000","last_clean":"2023-11-15T02:59:34.710364+0000","last_became_active":"2023-11-15T02:59:34.706803+0000","last_became_peered":"2023-11-15T02:59:34.706803+0000","last_unstale":"2023-11-15T02:59:34.710364+0000","last_undegraded":"2023-11-15T02:59:34.710364+0000","last_fullsized":"2023-11-15T02:59:34.710364+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.710145+0000","last_change":"2023-11-15T02:59:34.263720+0000","last_active":"2023-11-15T02:59:34.710145+0000","last_peered":"2023-11-15T02:59:34.710145+0000","last_clean":"2023-11-15T02:59:34.710145+0000","last_became_active":"2023-11-15T02:59:34.262994+0000","last_became_peered":"2023-11-15T02:59:34.262994+0000","last_unstale":"2023-11-15T02:59:34.710145+0000","last_undegraded":"2023-11-15T02:59:34.710145+0000","last_fullsized":"2023-11-15T02:59:34.710145+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.702473+0000","last_change":"2023-11-15T02:59:34.702473+0000","last_active":"2023-11-15T02:59:34.702473+0000","last_peered":"2023-11-15T02:59:34.702473+0000","last_clean":"2023-11-15T02:59:34.702473+0000","last_became_active":"2023-11-15T02:59:34.702094+0000","last_became_peered":"2023-11-15T02:59:34.702094+0000","last_unstale":"2023-11-15T02:59:34.702473+0000","last_undegraded":"2023-11-15T02:59:34.702473+0000","last_fullsized":"2023-11-15T02:59:34.702473+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.271846+0000","last_change":"2023-11-15T02:59:34.260019+0000","last_active":"2023-11-15T02:59:34.271846+0000","last_peered":"2023-11-15T02:59:34.271846+0000","last_clean":"2023-11-15T02:59:34.271846+0000","last_became_active":"2023-11-15T02:59:34.259470+0000","last_became_peered":"2023-11-15T02:59:34.259470+0000","last_unstale":"2023-11-15T02:59:34.271846+0000","last_undegraded":"2023-11-15T02:59:34.271846+0000","last_fullsized":"2023-11-15T02:59:34.271846+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.707408+0000","last_change":"2023-11-15T02:59:34.707408+0000","last_active":"2023-11-15T02:59:34.707408+0000","last_peered":"2023-11-15T02:59:34.707408+0000","last_clean":"2023-11-15T02:59:34.707408+0000","last_became_active":"2023-11-15T02:59:34.706951+0000","last_became_peered":"2023-11-15T02:59:34.706951+0000","last_unstale":"2023-11-15T02:59:34.707408+0000","last_undegraded":"2023-11-15T02:59:34.707408+0000","last_fullsized":"2023-11-15T02:59:34.707408+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.280951+0000","last_change":"2023-11-15T02:59:34.276164+0000","last_active":"2023-11-15T02:59:34.280951+0000","last_peered":"2023-11-15T02:59:34.280951+0000","last_clean":"2023-11-15T02:59:34.280951+0000","last_became_active":"2023-11-15T02:59:34.275881+0000","last_became_peered":"2023-11-15T02:59:34.275881+0000","last_unstale":"2023-11-15T02:59:34.280951+0000","last_undegraded":"2023-11-15T02:59:34.280951+0000","last_fullsized":"2023-11-15T02:59:34.280951+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.274124+0000","last_change":"2023-11-15T02:59:34.269967+0000","last_active":"2023-11-15T02:59:34.274124+0000","last_peered":"2023-11-15T02:59:34.274124+0000","last_clean":"2023-11-15T02:59:34.274124+0000","last_became_active":"2023-11-15T02:59:34.269516+0000","last_became_peered":"2023-11-15T02:59:34.269516+0000","last_unstale":"2023-11-15T02:59:34.274124+0000","last_undegraded":"2023-11-15T02:59:34.274124+0000","last_fullsized":"2023-11-15T02:59:34.274124+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.713129+0000","last_change":"2023-11-15T02:59:34.713129+0000","last_active":"2023-11-15T02:59:34.713129+0000","last_peered":"2023-11-15T02:59:34.713129+0000","last_clean":"2023-11-15T02:59:34.713129+0000","last_became_active":"2023-11-15T02:59:34.704067+0000","last_became_peered":"2023-11-15T02:59:34.704067+0000","last_unstale":"2023-11-15T02:59:34.713129+0000","last_undegraded":"2023-11-15T02:59:34.713129+0000","last_fullsized":"2023-11-15T02:59:34.713129+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278113+0000","last_change":"2023-11-15T02:59:34.272334+0000","last_active":"2023-11-15T02:59:34.278113+0000","last_peered":"2023-11-15T02:59:34.278113+0000","last_clean":"2023-11-15T02:59:34.278113+0000","last_became_active":"2023-11-15T02:59:34.272096+0000","last_became_peered":"2023-11-15T02:59:34.272096+0000","last_unstale":"2023-11-15T02:59:34.278113+0000","last_undegraded":"2023-11-15T02:59:34.278113+0000","last_fullsized":"2023-11-15T02:59:34.278113+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.686879+0000","last_change":"2023-11-15T02:59:34.686879+0000","last_active":"2023-11-15T02:59:34.686879+0000","last_peered":"2023-11-15T02:59:34.686879+0000","last_clean":"2023-11-15T02:59:34.686879+0000","last_became_active":"2023-11-15T02:59:34.685697+0000","last_became_peered":"2023-11-15T02:59:34.685697+0000","last_unstale":"2023-11-15T02:59:34.686879+0000","last_undegraded":"2023-11-15T02:59:34.686879+0000","last_fullsized":"2023-11-15T02:59:34.686879+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.275520+0000","last_change":"2023-11-15T02:59:34.270391+0000","last_active":"2023-11-15T02:59:34.275520+0000","last_peered":"2023-11-15T02:59:34.275520+0000","last_clean":"2023-11-15T02:59:34.275520+0000","last_became_active":"2023-11-15T02:59:34.270049+0000","last_became_peered":"2023-11-15T02:59:34.270049+0000","last_unstale":"2023-11-15T02:59:34.275520+0000","last_undegraded":"2023-11-15T02:59:34.275520+0000","last_fullsized":"2023-11-15T02:59:34.275520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.282770+0000","last_change":"2023-11-15T02:59:34.280324+0000","last_active":"2023-11-15T02:59:34.282770+0000","last_peered":"2023-11-15T02:59:34.282770+0000","last_clean":"2023-11-15T02:59:34.282770+0000","last_became_active":"2023-11-15T02:59:34.280003+0000","last_became_peered":"2023-11-15T02:59:34.280003+0000","last_unstale":"2023-11-15T02:59:34.282770+0000","last_undegraded":"2023-11-15T02:59:34.282770+0000","last_fullsized":"2023-11-15T02:59:34.282770+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.706837+0000","last_change":"2023-11-15T02:59:34.706837+0000","last_active":"2023-11-15T02:59:34.706837+0000","last_peered":"2023-11-15T02:59:34.706837+0000","last_clean":"2023-11-15T02:59:34.706837+0000","last_became_active":"2023-11-15T02:59:34.706595+0000","last_became_peered":"2023-11-15T02:59:34.706595+0000","last_unstale":"2023-11-15T02:59:34.706837+0000","last_undegraded":"2023-11-15T02:59:34.706837+0000","last_fullsized":"2023-11-15T02:59:34.706837+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278312+0000","last_change":"2023-11-15T02:59:34.274116+0000","last_active":"2023-11-15T02:59:34.278312+0000","last_peered":"2023-11-15T02:59:34.278312+0000","last_clean":"2023-11-15T02:59:34.278312+0000","last_became_active":"2023-11-15T02:59:34.273108+0000","last_became_peered":"2023-11-15T02:59:34.273108+0000","last_unstale":"2023-11-15T02:59:34.278312+0000","last_undegraded":"2023-11-15T02:59:34.278312+0000","last_fullsized":"2023-11-15T02:59:34.278312+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.713931+0000","last_change":"2023-11-15T02:59:34.713931+0000","last_active":"2023-11-15T02:59:34.713931+0000","last_peered":"2023-11-15T02:59:34.713931+0000","last_clean":"2023-11-15T02:59:34.713931+0000","last_became_active":"2023-11-15T02:59:34.713640+0000","last_became_peered":"2023-11-15T02:59:34.713640+0000","last_unstale":"2023-11-15T02:59:34.713931+0000","last_undegraded":"2023-11-15T02:59:34.713931+0000","last_fullsized":"2023-11-15T02:59:34.713931+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"60'3","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.704814+0000","last_change":"2023-11-15T02:59:34.273818+0000","last_active":"2023-11-15T02:59:34.704814+0000","last_peered":"2023-11-15T02:59:34.704814+0000","last_clean":"2023-11-15T02:59:34.704814+0000","last_became_active":"2023-11-15T02:59:34.273463+0000","last_became_peered":"2023-11-15T02:59:34.273463+0000","last_unstale":"2023-11-15T02:59:34.704814+0000","last_undegraded":"2023-11-15T02:59:34.704814+0000","last_fullsized":"2023-11-15T02:59:34.704814+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.691994+0000","last_change":"2023-11-15T02:59:34.691994+0000","last_active":"2023-11-15T02:59:34.691994+0000","last_peered":"2023-11-15T02:59:34.691994+0000","last_clean":"2023-11-15T02:59:34.691994+0000","last_became_active":"2023-11-15T02:59:34.691573+0000","last_became_peered":"2023-11-15T02:59:34.691573+0000","last_unstale":"2023-11-15T02:59:34.691994+0000","last_undegraded":"2023-11-15T02:59:34.691994+0000","last_fullsized":"2023-11-15T02:59:34.691994+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.207077+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.708893+0000","last_change":"2023-11-15T02:59:34.708893+0000","last_active":"2023-11-15T02:59:34.708893+0000","last_peered":"2023-11-15T02:59:34.708893+0000","last_clean":"2023-11-15T02:59:34.708893+0000","last_became_active":"2023-11-15T02:59:34.708623+0000","last_became_peered":"2023-11-15T02:59:34.708623+0000","last_unstale":"2023-11-15T02:59:34.708893+0000","last_undegraded":"2023-11-15T02:59:34.708893+0000","last_fullsized":"2023-11-15T02:59:34.708893+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.688236+0000","last_change":"2023-11-15T02:59:34.270907+0000","last_active":"2023-11-15T02:59:34.688236+0000","last_peered":"2023-11-15T02:59:34.688236+0000","last_clean":"2023-11-15T02:59:34.688236+0000","last_became_active":"2023-11-15T02:59:34.270561+0000","last_became_peered":"2023-11-15T02:59:34.270561+0000","last_unstale":"2023-11-15T02:59:34.688236+0000","last_undegraded":"2023-11-15T02:59:34.688236+0000","last_fullsized":"2023-11-15T02:59:34.688236+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.230568+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.719857+0000","last_change":"2023-11-15T02:59:34.719857+0000","last_active":"2023-11-15T02:59:34.719857+0000","last_peered":"2023-11-15T02:59:34.719857+0000","last_clean":"2023-11-15T02:59:34.719857+0000","last_became_active":"2023-11-15T02:59:34.719591+0000","last_became_peered":"2023-11-15T02:59:34.719591+0000","last_unstale":"2023-11-15T02:59:34.719857+0000","last_undegraded":"2023-11-15T02:59:34.719857+0000","last_fullsized":"2023-11-15T02:59:34.719857+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.279642+0000","last_change":"2023-11-15T02:59:34.276260+0000","last_active":"2023-11-15T02:59:34.279642+0000","last_peered":"2023-11-15T02:59:34.279642+0000","last_clean":"2023-11-15T02:59:34.279642+0000","last_became_active":"2023-11-15T02:59:34.275758+0000","last_became_peered":"2023-11-15T02:59:34.275758+0000","last_unstale":"2023-11-15T02:59:34.279642+0000","last_undegraded":"2023-11-15T02:59:34.279642+0000","last_fullsized":"2023-11-15T02:59:34.279642+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"60'4","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.722402+0000","last_change":"2023-11-15T02:59:34.269037+0000","last_active":"2023-11-15T02:59:34.722402+0000","last_peered":"2023-11-15T02:59:34.722402+0000","last_clean":"2023-11-15T02:59:34.722402+0000","last_became_active":"2023-11-15T02:59:34.268123+0000","last_became_peered":"2023-11-15T02:59:34.268123+0000","last_unstale":"2023-11-15T02:59:34.722402+0000","last_undegraded":"2023-11-15T02:59:34.722402+0000","last_fullsized":"2023-11-15T02:59:34.722402+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.715434+0000","last_change":"2023-11-15T02:59:34.715434+0000","last_active":"2023-11-15T02:59:34.715434+0000","last_peered":"2023-11-15T02:59:34.715434+0000","last_clean":"2023-11-15T02:59:34.715434+0000","last_became_active":"2023-11-15T02:59:34.715198+0000","last_became_peered":"2023-11-15T02:59:34.715198+0000","last_unstale":"2023-11-15T02:59:34.715434+0000","last_undegraded":"2023-11-15T02:59:34.715434+0000","last_fullsized":"2023-11-15T02:59:34.715434+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.228982+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.705591+0000","last_change":"2023-11-15T02:59:34.705591+0000","last_active":"2023-11-15T02:59:34.705591+0000","last_peered":"2023-11-15T02:59:34.705591+0000","last_clean":"2023-11-15T02:59:34.705591+0000","last_became_active":"2023-11-15T02:59:34.703826+0000","last_became_peered":"2023-11-15T02:59:34.703826+0000","last_unstale":"2023-11-15T02:59:34.705591+0000","last_undegraded":"2023-11-15T02:59:34.705591+0000","last_fullsized":"2023-11-15T02:59:34.705591+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.228971+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.273908+0000","last_change":"2023-11-15T02:59:34.263759+0000","last_active":"2023-11-15T02:59:34.273908+0000","last_peered":"2023-11-15T02:59:34.273908+0000","last_clean":"2023-11-15T02:59:34.273908+0000","last_became_active":"2023-11-15T02:59:34.263333+0000","last_became_peered":"2023-11-15T02:59:34.263333+0000","last_unstale":"2023-11-15T02:59:34.273908+0000","last_undegraded":"2023-11-15T02:59:34.273908+0000","last_fullsized":"2023-11-15T02:59:34.273908+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.700382+0000","last_change":"2023-11-15T02:59:34.700382+0000","last_active":"2023-11-15T02:59:34.700382+0000","last_peered":"2023-11-15T02:59:34.700382+0000","last_clean":"2023-11-15T02:59:34.700382+0000","last_became_active":"2023-11-15T02:59:34.700091+0000","last_became_peered":"2023-11-15T02:59:34.700091+0000","last_unstale":"2023-11-15T02:59:34.700382+0000","last_undegraded":"2023-11-15T02:59:34.700382+0000","last_fullsized":"2023-11-15T02:59:34.700382+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.207077+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.697102+0000","last_change":"2023-11-15T02:59:34.697102+0000","last_active":"2023-11-15T02:59:34.697102+0000","last_peered":"2023-11-15T02:59:34.697102+0000","last_clean":"2023-11-15T02:59:34.697102+0000","last_became_active":"2023-11-15T02:59:34.696095+0000","last_became_peered":"2023-11-15T02:59:34.696095+0000","last_unstale":"2023-11-15T02:59:34.697102+0000","last_undegraded":"2023-11-15T02:59:34.697102+0000","last_fullsized":"2023-11-15T02:59:34.697102+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.279613+0000","last_change":"2023-11-15T02:59:34.275904+0000","last_active":"2023-11-15T02:59:34.279613+0000","last_peered":"2023-11-15T02:59:34.279613+0000","last_clean":"2023-11-15T02:59:34.279613+0000","last_became_active":"2023-11-15T02:59:34.275561+0000","last_became_peered":"2023-11-15T02:59:34.275561+0000","last_unstale":"2023-11-15T02:59:34.279613+0000","last_undegraded":"2023-11-15T02:59:34.279613+0000","last_fullsized":"2023-11-15T02:59:34.279613+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.710944+0000","last_change":"2023-11-15T02:59:34.710944+0000","last_active":"2023-11-15T02:59:34.710944+0000","last_peered":"2023-11-15T02:59:34.710944+0000","last_clean":"2023-11-15T02:59:34.710944+0000","last_became_active":"2023-11-15T02:59:34.710640+0000","last_became_peered":"2023-11-15T02:59:34.710640+0000","last_unstale":"2023-11-15T02:59:34.710944+0000","last_undegraded":"2023-11-15T02:59:34.710944+0000","last_fullsized":"2023-11-15T02:59:34.710944+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.280896+0000","last_change":"2023-11-15T02:59:34.275717+0000","last_active":"2023-11-15T02:59:34.280896+0000","last_peered":"2023-11-15T02:59:34.280896+0000","last_clean":"2023-11-15T02:59:34.280896+0000","last_became_active":"2023-11-15T02:59:34.274314+0000","last_became_peered":"2023-11-15T02:59:34.274314+0000","last_unstale":"2023-11-15T02:59:34.280896+0000","last_undegraded":"2023-11-15T02:59:34.280896+0000","last_fullsized":"2023-11-15T02:59:34.280896+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278096+0000","last_change":"2023-11-15T02:59:34.271643+0000","last_active":"2023-11-15T02:59:34.278096+0000","last_peered":"2023-11-15T02:59:34.278096+0000","last_clean":"2023-11-15T02:59:34.278096+0000","last_became_active":"2023-11-15T02:59:34.271375+0000","last_became_peered":"2023-11-15T02:59:34.271375+0000","last_unstale":"2023-11-15T02:59:34.278096+0000","last_undegraded":"2023-11-15T02:59:34.278096+0000","last_fullsized":"2023-11-15T02:59:34.278096+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.711599+0000","last_change":"2023-11-15T02:59:34.711599+0000","last_active":"2023-11-15T02:59:34.711599+0000","last_peered":"2023-11-15T02:59:34.711599+0000","last_clean":"2023-11-15T02:59:34.711599+0000","last_became_active":"2023-11-15T02:59:34.711325+0000","last_became_peered":"2023-11-15T02:59:34.711325+0000","last_unstale":"2023-11-15T02:59:34.711599+0000","last_undegraded":"2023-11-15T02:59:34.711599+0000","last_fullsized":"2023-11-15T02:59:34.711599+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.207077+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.699657+0000","last_change":"2023-11-15T02:59:34.699657+0000","last_active":"2023-11-15T02:59:34.699657+0000","last_peered":"2023-11-15T02:59:34.699657+0000","last_clean":"2023-11-15T02:59:34.699657+0000","last_became_active":"2023-11-15T02:59:34.699371+0000","last_became_peered":"2023-11-15T02:59:34.699371+0000","last_unstale":"2023-11-15T02:59:34.699657+0000","last_undegraded":"2023-11-15T02:59:34.699657+0000","last_fullsized":"2023-11-15T02:59:34.699657+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.272131+0000","last_change":"2023-11-15T02:59:34.264863+0000","last_active":"2023-11-15T02:59:34.272131+0000","last_peered":"2023-11-15T02:59:34.272131+0000","last_clean":"2023-11-15T02:59:34.272131+0000","last_became_active":"2023-11-15T02:59:34.264228+0000","last_became_peered":"2023-11-15T02:59:34.264228+0000","last_unstale":"2023-11-15T02:59:34.272131+0000","last_undegraded":"2023-11-15T02:59:34.272131+0000","last_fullsized":"2023-11-15T02:59:34.272131+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.252284+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T02:59:34.252284+0000","last_peered":"2023-11-15T02:59:34.252284+0000","last_clean":"2023-11-15T02:59:34.252284+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T02:59:34.252284+0000","last_undegraded":"2023-11-15T02:59:34.252284+0000","last_fullsized":"2023-11-15T02:59:34.252284+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.694895+0000","last_change":"2023-11-15T02:59:34.694895+0000","last_active":"2023-11-15T02:59:34.694895+0000","last_peered":"2023-11-15T02:59:34.694895+0000","last_clean":"2023-11-15T02:59:34.694895+0000","last_became_active":"2023-11-15T02:59:34.694592+0000","last_became_peered":"2023-11-15T02:59:34.694592+0000","last_unstale":"2023-11-15T02:59:34.694895+0000","last_undegraded":"2023-11-15T02:59:34.694895+0000","last_fullsized":"2023-11-15T02:59:34.694895+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.282767+0000","last_change":"2023-11-15T02:59:34.280250+0000","last_active":"2023-11-15T02:59:34.282767+0000","last_peered":"2023-11-15T02:59:34.282767+0000","last_clean":"2023-11-15T02:59:34.282767+0000","last_became_active":"2023-11-15T02:59:34.279961+0000","last_became_peered":"2023-11-15T02:59:34.279961+0000","last_unstale":"2023-11-15T02:59:34.282767+0000","last_undegraded":"2023-11-15T02:59:34.282767+0000","last_fullsized":"2023-11-15T02:59:34.282767+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-15T02:59:29.318446+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T02:59:29.318446+0000","last_peered":"2023-11-15T02:59:29.318446+0000","last_clean":"2023-11-15T02:59:29.318446+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T02:59:29.318446+0000","last_undegraded":"2023-11-15T02:59:29.318446+0000","last_fullsized":"2023-11-15T02:59:29.318446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.720662+0000","last_change":"2023-11-15T02:59:34.720662+0000","last_active":"2023-11-15T02:59:34.720662+0000","last_peered":"2023-11-15T02:59:34.720662+0000","last_clean":"2023-11-15T02:59:34.720662+0000","last_became_active":"2023-11-15T02:59:34.713450+0000","last_became_peered":"2023-11-15T02:59:34.713450+0000","last_unstale":"2023-11-15T02:59:34.720662+0000","last_undegraded":"2023-11-15T02:59:34.720662+0000","last_fullsized":"2023-11-15T02:59:34.720662+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716637+0000","last_change":"2023-11-15T02:59:34.272609+0000","last_active":"2023-11-15T02:59:34.716637+0000","last_peered":"2023-11-15T02:59:34.716637+0000","last_clean":"2023-11-15T02:59:34.716637+0000","last_became_active":"2023-11-15T02:59:34.272212+0000","last_became_peered":"2023-11-15T02:59:34.272212+0000","last_unstale":"2023-11-15T02:59:34.716637+0000","last_undegraded":"2023-11-15T02:59:34.716637+0000","last_fullsized":"2023-11-15T02:59:34.716637+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.257610+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T02:59:34.257610+0000","last_peered":"2023-11-15T02:59:34.257610+0000","last_clean":"2023-11-15T02:59:34.257610+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T02:59:34.257610+0000","last_undegraded":"2023-11-15T02:59:34.257610+0000","last_fullsized":"2023-11-15T02:59:34.257610+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.711825+0000","last_change":"2023-11-15T02:59:34.711825+0000","last_active":"2023-11-15T02:59:34.711825+0000","last_peered":"2023-11-15T02:59:34.711825+0000","last_clean":"2023-11-15T02:59:34.711825+0000","last_became_active":"2023-11-15T02:59:34.711551+0000","last_became_peered":"2023-11-15T02:59:34.711551+0000","last_unstale":"2023-11-15T02:59:34.711825+0000","last_undegraded":"2023-11-15T02:59:34.711825+0000","last_fullsized":"2023-11-15T02:59:34.711825+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.229993+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-15T02:59:29.318457+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T02:59:29.318457+0000","last_peered":"2023-11-15T02:59:29.318457+0000","last_clean":"2023-11-15T02:59:29.318457+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T02:59:29.318457+0000","last_undegraded":"2023-11-15T02:59:29.318457+0000","last_fullsized":"2023-11-15T02:59:29.318457+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.712699+0000","last_change":"2023-11-15T02:59:34.712699+0000","last_active":"2023-11-15T02:59:34.712699+0000","last_peered":"2023-11-15T02:59:34.712699+0000","last_clean":"2023-11-15T02:59:34.712699+0000","last_became_active":"2023-11-15T02:59:34.703686+0000","last_became_peered":"2023-11-15T02:59:34.703686+0000","last_unstale":"2023-11-15T02:59:34.712699+0000","last_undegraded":"2023-11-15T02:59:34.712699+0000","last_fullsized":"2023-11-15T02:59:34.712699+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.277673+0000","last_change":"2023-11-15T02:59:34.265955+0000","last_active":"2023-11-15T02:59:34.277673+0000","last_peered":"2023-11-15T02:59:34.277673+0000","last_clean":"2023-11-15T02:59:34.277673+0000","last_became_active":"2023-11-15T02:59:34.265681+0000","last_became_peered":"2023-11-15T02:59:34.265681+0000","last_unstale":"2023-11-15T02:59:34.277673+0000","last_undegraded":"2023-11-15T02:59:34.277673+0000","last_fullsized":"2023-11-15T02:59:34.277673+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":50,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-15T02:59:30.343893+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T02:59:30.343893+0000","last_peered":"2023-11-15T02:59:30.343893+0000","last_clean":"2023-11-15T02:59:30.343893+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T02:59:30.343893+0000","last_undegraded":"2023-11-15T02:59:30.343893+0000","last_fullsized":"2023-11-15T02:59:30.343893+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.229084+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.253156+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T02:59:34.253156+0000","last_peered":"2023-11-15T02:59:34.253156+0000","last_clean":"2023-11-15T02:59:34.253156+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T02:59:34.253156+0000","last_undegraded":"2023-11-15T02:59:34.253156+0000","last_fullsized":"2023-11-15T02:59:34.253156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.704325+0000","last_change":"2023-11-15T02:59:34.704325+0000","last_active":"2023-11-15T02:59:34.704325+0000","last_peered":"2023-11-15T02:59:34.704325+0000","last_clean":"2023-11-15T02:59:34.704325+0000","last_became_active":"2023-11-15T02:59:34.703365+0000","last_became_peered":"2023-11-15T02:59:34.703365+0000","last_unstale":"2023-11-15T02:59:34.704325+0000","last_undegraded":"2023-11-15T02:59:34.704325+0000","last_fullsized":"2023-11-15T02:59:34.704325+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.691558+0000","last_change":"2023-11-15T02:59:34.265022+0000","last_active":"2023-11-15T02:59:34.691558+0000","last_peered":"2023-11-15T02:59:34.691558+0000","last_clean":"2023-11-15T02:59:34.691558+0000","last_became_active":"2023-11-15T02:59:34.264125+0000","last_became_peered":"2023-11-15T02:59:34.264125+0000","last_unstale":"2023-11-15T02:59:34.691558+0000","last_undegraded":"2023-11-15T02:59:34.691558+0000","last_fullsized":"2023-11-15T02:59:34.691558+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.254819+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T02:59:34.254819+0000","last_peered":"2023-11-15T02:59:34.254819+0000","last_clean":"2023-11-15T02:59:34.254819+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T02:59:34.254819+0000","last_undegraded":"2023-11-15T02:59:34.254819+0000","last_fullsized":"2023-11-15T02:59:34.254819+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.697027+0000","last_change":"2023-11-15T02:59:34.697027+0000","last_active":"2023-11-15T02:59:34.697027+0000","last_peered":"2023-11-15T02:59:34.697027+0000","last_clean":"2023-11-15T02:59:34.697027+0000","last_became_active":"2023-11-15T02:59:34.696199+0000","last_became_peered":"2023-11-15T02:59:34.696199+0000","last_unstale":"2023-11-15T02:59:34.697027+0000","last_undegraded":"2023-11-15T02:59:34.697027+0000","last_fullsized":"2023-11-15T02:59:34.697027+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.283700+0000","last_change":"2023-11-15T02:59:34.279512+0000","last_active":"2023-11-15T02:59:34.283700+0000","last_peered":"2023-11-15T02:59:34.283700+0000","last_clean":"2023-11-15T02:59:34.283700+0000","last_became_active":"2023-11-15T02:59:34.278967+0000","last_became_peered":"2023-11-15T02:59:34.278967+0000","last_unstale":"2023-11-15T02:59:34.283700+0000","last_undegraded":"2023-11-15T02:59:34.283700+0000","last_fullsized":"2023-11-15T02:59:34.283700+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":50,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-15T02:59:30.343807+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T02:59:30.343807+0000","last_peered":"2023-11-15T02:59:30.343807+0000","last_clean":"2023-11-15T02:59:30.343807+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T02:59:30.343807+0000","last_undegraded":"2023-11-15T02:59:30.343807+0000","last_fullsized":"2023-11-15T02:59:30.343807+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.254906+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T02:59:34.254906+0000","last_peered":"2023-11-15T02:59:34.254906+0000","last_clean":"2023-11-15T02:59:34.254906+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T02:59:34.254906+0000","last_undegraded":"2023-11-15T02:59:34.254906+0000","last_fullsized":"2023-11-15T02:59:34.254906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":6},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":748,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":116,"acting":116,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542180,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":704512,"data_stored":312535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.1679999999999999,"15min":1.1859999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.458,"5min":3.952,"15min":3.952},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.046,"5min":1.1799999999999999,"15min":1.202},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4419999999999999,"5min":2.5,"15min":2.5},"last":1.016}]},{"osd":1,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.137,"15min":1.145},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.3740000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1739999999999999,"15min":1.1779999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.415,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.3049999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2150000000000001,"15min":1.2330000000000001},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.458,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2949999999999999,"15min":1.3069999999999999},"min":{"1min":0.97899999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.5149999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":0.92900000000000005}]},{"osd":3,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.3400000000000001,"15min":1.3740000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.381,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.175,"5min":1.2669999999999999,"15min":1.282},"min":{"1min":0.90600000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.351,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.093}]},{"osd":4,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.2689999999999999,"15min":1.3},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.4359999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":1.117},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.448,"15min":1.4990000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.5029999999999999,"5min":4.226,"15min":4.226},"last":0.87}]},{"osd":5,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.329,"15min":1.343},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.556,"5min":2.024,"15min":2.024},"last":1.28},{"interface":"front","average":{"1min":1.234,"5min":1.4399999999999999,"15min":1.474},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.653,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.216}]},{"osd":6,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.5620000000000001,"15min":1.601},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":1.5660000000000001,"5min":3.2789999999999999,"15min":3.2789999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.591,"15min":1.645},"min":{"1min":1.123,"5min":1.123,"15min":1.123},"max":{"1min":1.4399999999999999,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.98099999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542180,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297800,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074040,"statfs":{"total":96636764160,"available":96331816960,"internally_reserved":0,"allocated":720896,"data_stored":319748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.125,"15min":1.133},"min":{"1min":0.69799999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.353,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.0469999999999999,"15min":1.0409999999999999},"min":{"1min":0.72499999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.484,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.93899999999999995}]},{"osd":1,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0920000000000001,"15min":1.0840000000000001},"min":{"1min":0.876,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.415,"5min":1.623,"15min":1.623},"last":1.03},{"interface":"front","average":{"1min":1.173,"5min":1.169,"15min":1.169},"min":{"1min":0.80800000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.609,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.97199999999999998}]},{"osd":2,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.1870000000000001,"15min":1.1779999999999999},"min":{"1min":0.84399999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.496,"5min":2.621,"15min":2.621},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.0960000000000001,"15min":1.0840000000000001},"min":{"1min":0.91700000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.425,"5min":1.5,"15min":1.5},"last":1.2130000000000001}]},{"osd":3,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.167,"15min":1.157},"min":{"1min":1.0029999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.492,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.26,"15min":1.2629999999999999},"min":{"1min":0.99299999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.6850000000000001,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.3740000000000001}]},{"osd":4,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.173,"15min":1.1910000000000001},"min":{"1min":0.81399999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.4099999999999999,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":1.391},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.95599999999999996,"5min":0.625,"15min":0.625},"max":{"1min":1.4850000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.536}]},{"osd":5,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.417,"15min":1.411},"min":{"1min":1.155,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.772,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.3109999999999999,"15min":1.319},"min":{"1min":0.877,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6599999999999999,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":1.169}]},{"osd":7,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2809999999999999,"15min":1.3009999999999999},"min":{"1min":0.82199999999999995,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.395,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.522},{"interface":"front","average":{"1min":1.401,"5min":1.448,"15min":1.456},"min":{"1min":1.046,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8540000000000001,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":1.637}]}]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297800,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074040,"statfs":{"total":96636764160,"available":96331816960,"internally_reserved":0,"allocated":720896,"data_stored":319804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.107,"15min":1.1120000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.516,"5min":1.532,"15min":1.532},"last":1.125},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0069999999999999,"15min":1.0029999999999999},"min":{"1min":0.80700000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":1.28}]},{"osd":1,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2250000000000001,"15min":1.2370000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4770000000000001,"5min":2.617,"15min":2.617},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.115,"5min":1.1559999999999999,"15min":1.163},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.425,"5min":1.5,"15min":1.5},"last":1.2410000000000001}]},{"osd":2,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.1519999999999999,"15min":1.141},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.462,"5min":1.909,"15min":1.909},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1599999999999999,"15min":1.1639999999999999},"min":{"1min":0.88800000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.399,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.0700000000000001}]},{"osd":3,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1519999999999999,"15min":1.1599999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.462,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.345},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.294,"15min":1.3},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.5840000000000001,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":1.169}]},{"osd":5,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2430000000000001,"15min":1.2529999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5780000000000001,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.2290000000000001,"15min":1.2350000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.5049999999999999,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.6639999999999999}]},{"osd":6,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.333,"15min":1.343},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6619999999999999,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":1.456},{"interface":"front","average":{"1min":1.296,"5min":1.3979999999999999,"15min":1.415},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.7450000000000001,"5min":2.173,"15min":2.173},"last":1.4670000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.369,"15min":1.381},"min":{"1min":1.069,"5min":0.87,"15min":0.87},"max":{"1min":1.5820000000000001,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.4350000000000001,"15min":1.4450000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.587,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":1.415}]}]},{"osd":3,"up_from":14,"seq":60129542181,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":704512,"data_stored":312535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.77400000000000002,"15min":0.75900000000000001},"min":{"1min":0.64900000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.629},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.84199999999999997,"15min":0.82899999999999996},"min":{"1min":0.73599999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1779999999999999,"5min":1.26,"15min":1.26},"last":0.70699999999999996}]},{"osd":1,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.90000000000000002,"15min":0.90100000000000002},"min":{"1min":0.621,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.97999999999999998,"15min":0.96999999999999997},"min":{"1min":0.83299999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2749999999999999,"5min":1.375,"15min":1.375},"last":0.55800000000000005}]},{"osd":2,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97699999999999998,"15min":0.96799999999999997},"min":{"1min":0.76700000000000002,"5min":0.63,"15min":0.63},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0649999999999999,"15min":1.052},"min":{"1min":0.85899999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.96699999999999997}]},{"osd":4,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1859999999999999,"15min":1.1799999999999999},"min":{"1min":0.97899999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.456,"5min":1.837,"15min":1.837},"last":1.202},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.22,"15min":1.226},"min":{"1min":1.0249999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.492,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.0189999999999999}]},{"osd":5,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.27,"15min":1.2669999999999999},"min":{"1min":0.996,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.6459999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.05},{"interface":"front","average":{"1min":1.345,"5min":1.2789999999999999,"15min":1.268},"min":{"1min":1.01,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.89900000000000002}]},{"osd":6,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3120000000000001,"15min":1.306},"min":{"1min":1.1060000000000001,"5min":1.048,"15min":1.048},"max":{"1min":1.552,"5min":1.794,"15min":1.794},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.3580000000000001,"15min":1.3520000000000001},"min":{"1min":1.129,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":1.647,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.163}]},{"osd":7,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.3919999999999999,"15min":1.3899999999999999},"min":{"1min":1.083,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.365,"15min":1.357},"min":{"1min":1.1819999999999999,"5min":1.085,"15min":1.085},"max":{"1min":1.665,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.1850000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542181,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297780,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074060,"statfs":{"total":96636764160,"available":96331837440,"internally_reserved":0,"allocated":700416,"data_stored":311266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76700000000000002,"15min":0.76400000000000001},"min":{"1min":0.61699999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88700000000000001,"15min":0.88200000000000001},"min":{"1min":0.68300000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.67400000000000004}]},{"osd":1,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89600000000000002,"15min":0.89200000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3120000000000001,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.96099999999999997,"15min":0.94899999999999995},"min":{"1min":0.83499999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.93600000000000005}]},{"osd":3,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.002,"15min":1},"min":{"1min":0.79200000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.387,"5min":1.472,"15min":1.472},"last":0.629},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1259999999999999,"15min":1.1200000000000001},"min":{"1min":0.93000000000000005,"5min":0.622,"15min":0.622},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.81899999999999995}]},{"osd":4,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.208,"15min":1.21},"min":{"1min":1.0149999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.48,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.111},{"interface":"front","average":{"1min":1.286,"5min":1.238,"15min":1.23},"min":{"1min":1.0760000000000001,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.5429999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.071}]},{"osd":5,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.262,"15min":1.264},"min":{"1min":1.0149999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":1.613,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2989999999999999,"15min":1.2969999999999999},"min":{"1min":1.0700000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.6890000000000001,"5min":1.857,"15min":1.857},"last":1}]},{"osd":6,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.367,"15min":1.373},"min":{"1min":1.056,"5min":1.056,"15min":1.056},"max":{"1min":1.72,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.347,"15min":1.353},"min":{"1min":1.046,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":1.573,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.1599999999999999}]},{"osd":7,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3939999999999999,"15min":1.391},"min":{"1min":1.1719999999999999,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":1.8069999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.3799999999999999,"15min":1.3919999999999999},"min":{"1min":1.097,"5min":1.097,"15min":1.097},"max":{"1min":1.5529999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.96799999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542181,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":704512,"data_stored":312535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.84899999999999998,"15min":0.86899999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1990000000000001,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":4.3630000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.89700000000000002,"15min":0.89600000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1619999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":6.157}]},{"osd":2,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.99399999999999999,"15min":1.0129999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.147,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":4.1699999999999999},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.99199999999999999,"15min":0.98799999999999999},"min":{"1min":0.68000000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.2529999999999999,"5min":1.847,"15min":1.847},"last":6.8369999999999997}]},{"osd":3,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.96499999999999997,"15min":0.96199999999999997},"min":{"1min":0.76600000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.401,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":3.5070000000000001},{"interface":"front","average":{"1min":1.052,"5min":1.157,"15min":1.175},"min":{"1min":0.68899999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2210000000000001,"5min":3.54,"15min":3.54},"last":5.2149999999999999}]},{"osd":4,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2490000000000001,"15min":1.2470000000000001},"min":{"1min":1.0720000000000001,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.6319999999999999,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":6.2000000000000002},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.175,"15min":1.175},"min":{"1min":1.006,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.4850000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":3.577}]},{"osd":5,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2170000000000001,"15min":1.218},"min":{"1min":0.98899999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.393,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":3.548},{"interface":"front","average":{"1min":1.24,"5min":1.3420000000000001,"15min":1.359},"min":{"1min":1.083,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.375,"5min":3.6760000000000002,"15min":3.6760000000000002},"last":4.4080000000000004}]},{"osd":6,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.411,"15min":1.429},"min":{"1min":1.113,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":1.4730000000000001,"5min":3.71,"15min":3.71},"last":4.4489999999999998},{"interface":"front","average":{"1min":1.282,"5min":1.3340000000000001,"15min":1.343},"min":{"1min":1.05,"5min":1.024,"15min":1.024},"max":{"1min":1.4970000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":6.9690000000000003}]},{"osd":7,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.3640000000000001,"15min":1.379},"min":{"1min":1.018,"5min":1.018,"15min":1.018},"max":{"1min":1.554,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":18.253},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2649999999999999,"15min":1.2609999999999999},"min":{"1min":1.1040000000000001,"5min":0.876,"15min":0.876},"max":{"1min":1.5449999999999999,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":11.006}]}]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297772,"kb_used_data":676,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074068,"statfs":{"total":96636764160,"available":96331845632,"internally_reserved":0,"allocated":692224,"data_stored":305856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.05,"15min":1.046},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4359999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.216,"15min":1.2370000000000001},"min":{"1min":0.69699999999999995,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.4630000000000001,"5min":3.46,"15min":3.46},"last":0.83499999999999996}]},{"osd":1,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.2310000000000001,"15min":1.2589999999999999},"min":{"1min":0.82699999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.47,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.1739999999999999,"15min":1.1879999999999999},"min":{"1min":0.80300000000000005,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.4570000000000001,"5min":3.516,"15min":3.516},"last":1.0329999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.2070000000000001,"15min":1.2130000000000001},"min":{"1min":0.91600000000000004,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.3640000000000001,"5min":5.4180000000000001,"15min":5.4180000000000001},"last":1.163},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3069999999999999,"15min":1.321},"min":{"1min":0.94799999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.5089999999999999,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":0.69699999999999995}]},{"osd":3,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.29,"15min":1.3080000000000001},"min":{"1min":1.022,"5min":0.879,"15min":0.879},"max":{"1min":1.4850000000000001,"5min":2.141,"15min":2.141},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.448,"15min":1.464},"min":{"1min":1.1839999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":1.583,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":0.81599999999999995}]},{"osd":4,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3680000000000001,"15min":1.3879999999999999},"min":{"1min":0.84199999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.5309999999999999,"5min":3.706,"15min":3.706},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.597,"15min":1.6379999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6479999999999999,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":1.212}]},{"osd":6,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.327,"15min":1.3420000000000001},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.6439999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":1.25},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.546,"15min":1.5860000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.613,"5min":3.641,"15min":3.641},"last":1.2330000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4259999999999999,"15min":1.4370000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.762,"5min":3.516,"15min":3.516},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.46,"15min":1.4650000000000001},"min":{"1min":1.139,"5min":1.022,"15min":1.022},"max":{"1min":1.722,"5min":3.754,"15min":3.754},"last":1.343}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297760,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074080,"statfs":{"total":96636764160,"available":96331857920,"internally_reserved":0,"allocated":679936,"data_stored":299635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.84999999999999998,"15min":0.84599999999999997},"min":{"1min":0.63600000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2150000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.72199999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.155,"5min":1.419,"15min":1.419},"last":0.67800000000000005}]},{"osd":2,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.85799999999999998,"15min":0.84799999999999998},"min":{"1min":0.495,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.90700000000000003,"15min":0.91000000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.161,"5min":1.341,"15min":1.341},"last":0.76500000000000001}]},{"osd":3,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.0620000000000001,"15min":1.054},"min":{"1min":0.54100000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.25},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.98899999999999999,"15min":0.97699999999999998},"min":{"1min":0.67800000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":1.23}]},{"osd":4,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.175,"15min":1.165},"min":{"1min":0.75700000000000001,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.49,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.248,"15min":1.2589999999999999},"min":{"1min":0.92000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.5609999999999999,"5min":2.206,"15min":2.206},"last":1.1799999999999999}]},{"osd":5,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.26,"15min":1.2649999999999999},"min":{"1min":1.0089999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.4299999999999999,"5min":1.899,"15min":1.899},"last":1.048},{"interface":"front","average":{"1min":1.236,"5min":1.2829999999999999,"15min":1.2909999999999999},"min":{"1min":0.92800000000000005,"5min":0.871,"15min":0.871},"max":{"1min":1.5029999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.3029999999999999}]},{"osd":6,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.323,"15min":1.325},"min":{"1min":0.98299999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.5229999999999999,"5min":1.825,"15min":1.825},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.4099999999999999,"15min":1.4119999999999999},"min":{"1min":1.1870000000000001,"5min":0.87,"15min":0.87},"max":{"1min":1.5529999999999999,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.3740000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3200000000000001,"15min":1.3169999999999999},"min":{"1min":0.93999999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.6839999999999999,"5min":1.772,"15min":1.772},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3540000000000001,"15min":1.3600000000000001},"min":{"1min":0.95499999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.6120000000000001,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.1990000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T02:59:39.340 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T02:59:39.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.529+0000 7f7e61b46700 1 Processor -- start 2023-11-15T02:59:39.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.529+0000 7f7e61b46700 1 -- start start 2023-11-15T02:59:39.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.533+0000 7f7e61b46700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7e5c154ca0 0x7f7e5c155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:39.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.533+0000 7f7e61b46700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7e5c0a57c0 0x7f7e5c0a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:39.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.533+0000 7f7e61b46700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5c0c4c30 con 0x7f7e5c1562f0 2023-11-15T02:59:39.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.533+0000 7f7e61b46700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e5c0a8280 con 0x7f7e5c0a57c0 2023-11-15T02:59:39.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.533+0000 7f7e61b46700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e5c0a6d40 con 0x7f7e5c154ca0 2023-11-15T02:59:39.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.533+0000 7f7e5bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7e5c0a57c0 0x7f7e5c0a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:39.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.533+0000 7f7e5bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7e5c0a57c0 0x7f7e5c0a5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39328/0 (socket says 172.21.15.121:39328) 2023-11-15T02:59:39.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.533+0000 7f7e5bfff700 1 -- 172.21.15.121:0/2950145911 learned_addr learned my addr 172.21.15.121:0/2950145911 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:39.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.533+0000 7f7e5b7fe700 1 --2- 172.21.15.121:0/2950145911 >> v2:172.21.15.190:3300/0 conn(0x7f7e5c154ca0 0x7f7e5c155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:39.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.537+0000 7f7e5bfff700 1 -- 172.21.15.121:0/2950145911 >> v2:172.21.15.190:3300/0 conn(0x7f7e5c154ca0 msgr2=0x7f7e5c155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:39.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.537+0000 7f7e5bfff700 1 --2- 172.21.15.121:0/2950145911 >> v2:172.21.15.190:3300/0 conn(0x7f7e5c154ca0 0x7f7e5c155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.537+0000 7f7e5bfff700 1 -- 172.21.15.121:0/2950145911 >> v1:172.21.15.190:6789/0 conn(0x7f7e5c1562f0 legacy=0x7f7e5c0a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:39.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.537+0000 7f7e5bfff700 1 -- 172.21.15.121:0/2950145911 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c0a6100 con 0x7f7e5c0a57c0 2023-11-15T02:59:39.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.537+0000 7f7e5bfff700 1 --2- 172.21.15.121:0/2950145911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e5c0a57c0 0x7f7e5c0a5c10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f7e5c064f10 tx=0x7f7e54009840).ready entity=mon.0 client_cookie=a9ee63739bcf9562 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:39.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.537+0000 7f7e5affd700 1 -- 172.21.15.121:0/2950145911 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5400e070 con 0x7f7e5c0a57c0 2023-11-15T02:59:39.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.537+0000 7f7e5affd700 1 -- 172.21.15.121:0/2950145911 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5400ba20 con 0x7f7e5c0a57c0 2023-11-15T02:59:39.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.537+0000 7f7e5affd700 1 -- 172.21.15.121:0/2950145911 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5400bda0 con 0x7f7e5c0a57c0 2023-11-15T02:59:39.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.537+0000 7f7e61b46700 1 -- 172.21.15.121:0/2950145911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e5c0a57c0 msgr2=0x7f7e5c0a5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:39.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.537+0000 7f7e61b46700 1 --2- 172.21.15.121:0/2950145911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e5c0a57c0 0x7f7e5c0a5c10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f7e5c064f10 tx=0x7f7e54009840).stop 2023-11-15T02:59:39.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.537+0000 7f7e61b46700 1 -- 172.21.15.121:0/2950145911 shutdown_connections 2023-11-15T02:59:39.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.537+0000 7f7e61b46700 1 --2- 172.21.15.121:0/2950145911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e5c0a57c0 0x7f7e5c0a5c10 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.537+0000 7f7e61b46700 1 --2- 172.21.15.121:0/2950145911 >> v2:172.21.15.190:3300/0 conn(0x7f7e5c154ca0 0x7f7e5c155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.537+0000 7f7e61b46700 1 -- 172.21.15.121:0/2950145911 >> 172.21.15.121:0/2950145911 conn(0x7f7e5c062190 msgr2=0x7f7e5c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:39.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.537+0000 7f7e61b46700 1 -- 172.21.15.121:0/2950145911 shutdown_connections 2023-11-15T02:59:39.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.537+0000 7f7e61b46700 1 -- 172.21.15.121:0/2950145911 wait complete. 2023-11-15T02:59:39.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.541+0000 7f7e61b46700 1 Processor -- start 2023-11-15T02:59:39.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.541+0000 7f7e61b46700 1 -- start start 2023-11-15T02:59:39.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.541+0000 7f7e61b46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e5c0a57c0 0x7f7e5c0abae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:39.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.541+0000 7f7e61b46700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e5c154ca0 0x7f7e5c1447d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:39.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.541+0000 7f7e61b46700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e5c1562f0 0x7f7e5c066aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:39.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.541+0000 7f7e61b46700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c0abfd0 con 0x7f7e5c0a57c0 2023-11-15T02:59:39.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.541+0000 7f7e61b46700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c163ef0 con 0x7f7e5c1562f0 2023-11-15T02:59:39.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.541+0000 7f7e61b46700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c14ad50 con 0x7f7e5c154ca0 2023-11-15T02:59:39.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.541+0000 7f7e5b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e5c0a57c0 0x7f7e5c0abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:39.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.541+0000 7f7e5b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e5c0a57c0 0x7f7e5c0abae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39334/0 (socket says 172.21.15.121:39334) 2023-11-15T02:59:39.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.541+0000 7f7e5b7fe700 1 -- 172.21.15.121:0/2870538931 learned_addr learned my addr 172.21.15.121:0/2870538931 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:39.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.545+0000 7f7e5bfff700 1 --2- 172.21.15.121:0/2870538931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e5c1562f0 0x7f7e5c066aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:39.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.545+0000 7f7e53fff700 1 --2- 172.21.15.121:0/2870538931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e5c154ca0 0x7f7e5c1447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:39.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.545+0000 7f7e5b7fe700 1 -- 172.21.15.121:0/2870538931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e5c154ca0 msgr2=0x7f7e5c1447d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:39.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.545+0000 7f7e5b7fe700 1 --2- 172.21.15.121:0/2870538931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e5c154ca0 0x7f7e5c1447d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.545+0000 7f7e5b7fe700 1 -- 172.21.15.121:0/2870538931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e5c1562f0 msgr2=0x7f7e5c066aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:39.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.545+0000 7f7e5b7fe700 1 --2- 172.21.15.121:0/2870538931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e5c1562f0 0x7f7e5c066aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.545+0000 7f7e5b7fe700 1 -- 172.21.15.121:0/2870538931 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e54009ae0 con 0x7f7e5c0a57c0 2023-11-15T02:59:39.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.545+0000 7f7e5b7fe700 1 --2- 172.21.15.121:0/2870538931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e5c0a57c0 0x7f7e5c0abae0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f7e4c00e960 tx=0x7f7e4c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:39.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.545+0000 7f7e537fe700 1 -- 172.21.15.121:0/2870538931 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c012600 con 0x7f7e5c0a57c0 2023-11-15T02:59:39.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.545+0000 7f7e537fe700 1 -- 172.21.15.121:0/2870538931 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4c012760 con 0x7f7e5c0a57c0 2023-11-15T02:59:39.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.545+0000 7f7e537fe700 1 -- 172.21.15.121:0/2870538931 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c012ae0 con 0x7f7e5c0a57c0 2023-11-15T02:59:39.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.545+0000 7f7e61b46700 1 -- 172.21.15.121:0/2870538931 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5c0aae20 con 0x7f7e5c0a57c0 2023-11-15T02:59:39.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.545+0000 7f7e61b46700 1 -- 172.21.15.121:0/2870538931 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e5c0ab3e0 con 0x7f7e5c0a57c0 2023-11-15T02:59:39.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.545+0000 7f7e61b46700 1 -- 172.21.15.121:0/2870538931 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e5c06c150 con 0x7f7e5c0a57c0 2023-11-15T02:59:39.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.545+0000 7f7e537fe700 1 -- 172.21.15.121:0/2870538931 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7e4c01b400 con 0x7f7e5c0a57c0 2023-11-15T02:59:39.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.549+0000 7f7e537fe700 1 --2- 172.21.15.121:0/2870538931 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7e3c05a510 0x7f7e3c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:39.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.549+0000 7f7e537fe700 1 -- 172.21.15.121:0/2870538931 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7e4c074c30 con 0x7f7e5c0a57c0 2023-11-15T02:59:39.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.549+0000 7f7e53fff700 1 --2- 172.21.15.121:0/2870538931 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7e3c05a510 0x7f7e3c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:39.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.549+0000 7f7e53fff700 1 --2- 172.21.15.121:0/2870538931 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7e3c05a510 0x7f7e3c05c9d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f7e5c14c620 tx=0x7f7e44007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:39.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.553+0000 7f7e537fe700 1 -- 172.21.15.121:0/2870538931 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7e4c0483b0 con 0x7f7e5c0a57c0 2023-11-15T02:59:39.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.709+0000 7f7e61b46700 1 -- 172.21.15.121:0/2870538931 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7e5c0a6f50 con 0x7f7e3c05a510 2023-11-15T02:59:39.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.721+0000 7f7e537fe700 1 -- 172.21.15.121:0/2870538931 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+355564 (secure 0 0 0) 0x7f7e5c0a6f50 con 0x7f7e3c05a510 2023-11-15T02:59:39.734 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:39.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.729+0000 7f7e61b46700 1 -- 172.21.15.121:0/2870538931 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7e3c05a510 msgr2=0x7f7e3c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:39.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.729+0000 7f7e61b46700 1 --2- 172.21.15.121:0/2870538931 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7e3c05a510 0x7f7e3c05c9d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f7e5c14c620 tx=0x7f7e44007d40).stop 2023-11-15T02:59:39.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.729+0000 7f7e61b46700 1 -- 172.21.15.121:0/2870538931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e5c0a57c0 msgr2=0x7f7e5c0abae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:39.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.729+0000 7f7e61b46700 1 --2- 172.21.15.121:0/2870538931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e5c0a57c0 0x7f7e5c0abae0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f7e4c00e960 tx=0x7f7e4c009060).stop 2023-11-15T02:59:39.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.729+0000 7f7e61b46700 1 -- 172.21.15.121:0/2870538931 shutdown_connections 2023-11-15T02:59:39.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.729+0000 7f7e61b46700 1 --2- 172.21.15.121:0/2870538931 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7e3c05a510 0x7f7e3c05c9d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.729+0000 7f7e61b46700 1 --2- 172.21.15.121:0/2870538931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e5c1562f0 0x7f7e5c066aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.729+0000 7f7e61b46700 1 --2- 172.21.15.121:0/2870538931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e5c154ca0 0x7f7e5c1447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.729+0000 7f7e61b46700 1 --2- 172.21.15.121:0/2870538931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e5c0a57c0 0x7f7e5c0abae0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.729+0000 7f7e61b46700 1 -- 172.21.15.121:0/2870538931 >> 172.21.15.121:0/2870538931 conn(0x7f7e5c062190 msgr2=0x7f7e5c1456f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:39.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.733+0000 7f7e61b46700 1 -- 172.21.15.121:0/2870538931 shutdown_connections 2023-11-15T02:59:39.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.733+0000 7f7e61b46700 1 -- 172.21.15.121:0/2870538931 wait complete. 2023-11-15T02:59:39.741 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T02:59:39.758 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2023-11-15T02:59:37.971314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":748,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":226,"acting":226,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382264,"kb_used_data":5496,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752592456,"statfs":{"total":773094113280,"available":770654674944,"internally_reserved":0,"allocated":5627904,"data_stored":2493914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12861,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-20,"num_read_kb":-23,"num_write":-72,"num_write_kb":-35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.466488"},"pg_stats":[{"pgid":"10.3c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.702569+0000","last_change":"2023-11-15T02:59:34.702569+0000","last_active":"2023-11-15T02:59:34.702569+0000","last_peered":"2023-11-15T02:59:34.702569+0000","last_clean":"2023-11-15T02:59:34.702569+0000","last_became_active":"2023-11-15T02:59:34.702272+0000","last_became_peered":"2023-11-15T02:59:34.702272+0000","last_unstale":"2023-11-15T02:59:34.702569+0000","last_undegraded":"2023-11-15T02:59:34.702569+0000","last_fullsized":"2023-11-15T02:59:34.702569+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.3f","version":"60'3","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.702378+0000","last_change":"2023-11-15T02:59:34.265036+0000","last_active":"2023-11-15T02:59:34.702378+0000","last_peered":"2023-11-15T02:59:34.702378+0000","last_clean":"2023-11-15T02:59:34.702378+0000","last_became_active":"2023-11-15T02:59:34.264631+0000","last_became_peered":"2023-11-15T02:59:34.264631+0000","last_unstale":"2023-11-15T02:59:34.702378+0000","last_undegraded":"2023-11-15T02:59:34.702378+0000","last_fullsized":"2023-11-15T02:59:34.702378+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.715077+0000","last_change":"2023-11-15T02:59:34.715077+0000","last_active":"2023-11-15T02:59:34.715077+0000","last_peered":"2023-11-15T02:59:34.715077+0000","last_clean":"2023-11-15T02:59:34.715077+0000","last_became_active":"2023-11-15T02:59:34.710308+0000","last_became_peered":"2023-11-15T02:59:34.710308+0000","last_unstale":"2023-11-15T02:59:34.715077+0000","last_undegraded":"2023-11-15T02:59:34.715077+0000","last_fullsized":"2023-11-15T02:59:34.715077+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.280938+0000","last_change":"2023-11-15T02:59:34.274806+0000","last_active":"2023-11-15T02:59:34.280938+0000","last_peered":"2023-11-15T02:59:34.280938+0000","last_clean":"2023-11-15T02:59:34.280938+0000","last_became_active":"2023-11-15T02:59:34.274536+0000","last_became_peered":"2023-11-15T02:59:34.274536+0000","last_unstale":"2023-11-15T02:59:34.280938+0000","last_undegraded":"2023-11-15T02:59:34.280938+0000","last_fullsized":"2023-11-15T02:59:34.280938+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.276406+0000","last_change":"2023-11-15T02:59:34.272252+0000","last_active":"2023-11-15T02:59:34.276406+0000","last_peered":"2023-11-15T02:59:34.276406+0000","last_clean":"2023-11-15T02:59:34.276406+0000","last_became_active":"2023-11-15T02:59:34.271979+0000","last_became_peered":"2023-11-15T02:59:34.271979+0000","last_unstale":"2023-11-15T02:59:34.276406+0000","last_undegraded":"2023-11-15T02:59:34.276406+0000","last_fullsized":"2023-11-15T02:59:34.276406+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.231018+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.694905+0000","last_change":"2023-11-15T02:59:34.263921+0000","last_active":"2023-11-15T02:59:34.694905+0000","last_peered":"2023-11-15T02:59:34.694905+0000","last_clean":"2023-11-15T02:59:34.694905+0000","last_became_active":"2023-11-15T02:59:34.261031+0000","last_became_peered":"2023-11-15T02:59:34.261031+0000","last_unstale":"2023-11-15T02:59:34.694905+0000","last_undegraded":"2023-11-15T02:59:34.694905+0000","last_fullsized":"2023-11-15T02:59:34.694905+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716845+0000","last_change":"2023-11-15T02:59:34.716845+0000","last_active":"2023-11-15T02:59:34.716845+0000","last_peered":"2023-11-15T02:59:34.716845+0000","last_clean":"2023-11-15T02:59:34.716845+0000","last_became_active":"2023-11-15T02:59:34.716558+0000","last_became_peered":"2023-11-15T02:59:34.716558+0000","last_unstale":"2023-11-15T02:59:34.716845+0000","last_undegraded":"2023-11-15T02:59:34.716845+0000","last_fullsized":"2023-11-15T02:59:34.716845+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.690603+0000","last_change":"2023-11-15T02:59:34.266240+0000","last_active":"2023-11-15T02:59:34.690603+0000","last_peered":"2023-11-15T02:59:34.690603+0000","last_clean":"2023-11-15T02:59:34.690603+0000","last_became_active":"2023-11-15T02:59:34.265188+0000","last_became_peered":"2023-11-15T02:59:34.265188+0000","last_unstale":"2023-11-15T02:59:34.690603+0000","last_undegraded":"2023-11-15T02:59:34.690603+0000","last_fullsized":"2023-11-15T02:59:34.690603+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.719027+0000","last_change":"2023-11-15T02:59:34.719027+0000","last_active":"2023-11-15T02:59:34.719027+0000","last_peered":"2023-11-15T02:59:34.719027+0000","last_clean":"2023-11-15T02:59:34.719027+0000","last_became_active":"2023-11-15T02:59:34.718717+0000","last_became_peered":"2023-11-15T02:59:34.718717+0000","last_unstale":"2023-11-15T02:59:34.719027+0000","last_undegraded":"2023-11-15T02:59:34.719027+0000","last_fullsized":"2023-11-15T02:59:34.719027+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.273500+0000","last_change":"2023-11-15T02:59:34.263881+0000","last_active":"2023-11-15T02:59:34.273500+0000","last_peered":"2023-11-15T02:59:34.273500+0000","last_clean":"2023-11-15T02:59:34.273500+0000","last_became_active":"2023-11-15T02:59:34.263571+0000","last_became_peered":"2023-11-15T02:59:34.263571+0000","last_unstale":"2023-11-15T02:59:34.273500+0000","last_undegraded":"2023-11-15T02:59:34.273500+0000","last_fullsized":"2023-11-15T02:59:34.273500+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.715365+0000","last_change":"2023-11-15T02:59:34.715365+0000","last_active":"2023-11-15T02:59:34.715365+0000","last_peered":"2023-11-15T02:59:34.715365+0000","last_clean":"2023-11-15T02:59:34.715365+0000","last_became_active":"2023-11-15T02:59:34.715043+0000","last_became_peered":"2023-11-15T02:59:34.715043+0000","last_unstale":"2023-11-15T02:59:34.715365+0000","last_undegraded":"2023-11-15T02:59:34.715365+0000","last_fullsized":"2023-11-15T02:59:34.715365+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.271621+0000","last_change":"2023-11-15T02:59:34.264832+0000","last_active":"2023-11-15T02:59:34.271621+0000","last_peered":"2023-11-15T02:59:34.271621+0000","last_clean":"2023-11-15T02:59:34.271621+0000","last_became_active":"2023-11-15T02:59:34.264571+0000","last_became_peered":"2023-11-15T02:59:34.264571+0000","last_unstale":"2023-11-15T02:59:34.271621+0000","last_undegraded":"2023-11-15T02:59:34.271621+0000","last_fullsized":"2023-11-15T02:59:34.271621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.270817+0000","last_change":"2023-11-15T02:59:34.262030+0000","last_active":"2023-11-15T02:59:34.270817+0000","last_peered":"2023-11-15T02:59:34.270817+0000","last_clean":"2023-11-15T02:59:34.270817+0000","last_became_active":"2023-11-15T02:59:34.261689+0000","last_became_peered":"2023-11-15T02:59:34.261689+0000","last_unstale":"2023-11-15T02:59:34.270817+0000","last_undegraded":"2023-11-15T02:59:34.270817+0000","last_fullsized":"2023-11-15T02:59:34.270817+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.701348+0000","last_change":"2023-11-15T02:59:34.701348+0000","last_active":"2023-11-15T02:59:34.701348+0000","last_peered":"2023-11-15T02:59:34.701348+0000","last_clean":"2023-11-15T02:59:34.701348+0000","last_became_active":"2023-11-15T02:59:34.701079+0000","last_became_peered":"2023-11-15T02:59:34.701079+0000","last_unstale":"2023-11-15T02:59:34.701348+0000","last_undegraded":"2023-11-15T02:59:34.701348+0000","last_fullsized":"2023-11-15T02:59:34.701348+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.207077+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.707253+0000","last_change":"2023-11-15T02:59:34.707253+0000","last_active":"2023-11-15T02:59:34.707253+0000","last_peered":"2023-11-15T02:59:34.707253+0000","last_clean":"2023-11-15T02:59:34.707253+0000","last_became_active":"2023-11-15T02:59:34.706969+0000","last_became_peered":"2023-11-15T02:59:34.706969+0000","last_unstale":"2023-11-15T02:59:34.707253+0000","last_undegraded":"2023-11-15T02:59:34.707253+0000","last_fullsized":"2023-11-15T02:59:34.707253+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.207077+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.717917+0000","last_change":"2023-11-15T02:59:34.717917+0000","last_active":"2023-11-15T02:59:34.717917+0000","last_peered":"2023-11-15T02:59:34.717917+0000","last_clean":"2023-11-15T02:59:34.717917+0000","last_became_active":"2023-11-15T02:59:34.717573+0000","last_became_peered":"2023-11-15T02:59:34.717573+0000","last_unstale":"2023-11-15T02:59:34.717917+0000","last_undegraded":"2023-11-15T02:59:34.717917+0000","last_fullsized":"2023-11-15T02:59:34.717917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.229134+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.709579+0000","last_change":"2023-11-15T02:59:34.709579+0000","last_active":"2023-11-15T02:59:34.709579+0000","last_peered":"2023-11-15T02:59:34.709579+0000","last_clean":"2023-11-15T02:59:34.709579+0000","last_became_active":"2023-11-15T02:59:34.709324+0000","last_became_peered":"2023-11-15T02:59:34.709324+0000","last_unstale":"2023-11-15T02:59:34.709579+0000","last_undegraded":"2023-11-15T02:59:34.709579+0000","last_fullsized":"2023-11-15T02:59:34.709579+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.33","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.689591+0000","last_change":"2023-11-15T02:59:34.261668+0000","last_active":"2023-11-15T02:59:34.689591+0000","last_peered":"2023-11-15T02:59:34.689591+0000","last_clean":"2023-11-15T02:59:34.689591+0000","last_became_active":"2023-11-15T02:59:34.261392+0000","last_became_peered":"2023-11-15T02:59:34.261392+0000","last_unstale":"2023-11-15T02:59:34.689591+0000","last_undegraded":"2023-11-15T02:59:34.689591+0000","last_fullsized":"2023-11-15T02:59:34.689591+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.717458+0000","last_change":"2023-11-15T02:59:34.717458+0000","last_active":"2023-11-15T02:59:34.717458+0000","last_peered":"2023-11-15T02:59:34.717458+0000","last_clean":"2023-11-15T02:59:34.717458+0000","last_became_active":"2023-11-15T02:59:34.717124+0000","last_became_peered":"2023-11-15T02:59:34.717124+0000","last_unstale":"2023-11-15T02:59:34.717458+0000","last_undegraded":"2023-11-15T02:59:34.717458+0000","last_fullsized":"2023-11-15T02:59:34.717458+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.274573+0000","last_change":"2023-11-15T02:59:34.267337+0000","last_active":"2023-11-15T02:59:34.274573+0000","last_peered":"2023-11-15T02:59:34.274573+0000","last_clean":"2023-11-15T02:59:34.274573+0000","last_became_active":"2023-11-15T02:59:34.267054+0000","last_became_peered":"2023-11-15T02:59:34.267054+0000","last_unstale":"2023-11-15T02:59:34.274573+0000","last_undegraded":"2023-11-15T02:59:34.274573+0000","last_fullsized":"2023-11-15T02:59:34.274573+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.709174+0000","last_change":"2023-11-15T02:59:34.709174+0000","last_active":"2023-11-15T02:59:34.709174+0000","last_peered":"2023-11-15T02:59:34.709174+0000","last_clean":"2023-11-15T02:59:34.709174+0000","last_became_active":"2023-11-15T02:59:34.708872+0000","last_became_peered":"2023-11-15T02:59:34.708872+0000","last_unstale":"2023-11-15T02:59:34.709174+0000","last_undegraded":"2023-11-15T02:59:34.709174+0000","last_fullsized":"2023-11-15T02:59:34.709174+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.270777+0000","last_change":"2023-11-15T02:59:34.261461+0000","last_active":"2023-11-15T02:59:34.270777+0000","last_peered":"2023-11-15T02:59:34.270777+0000","last_clean":"2023-11-15T02:59:34.270777+0000","last_became_active":"2023-11-15T02:59:34.261153+0000","last_became_peered":"2023-11-15T02:59:34.261153+0000","last_unstale":"2023-11-15T02:59:34.270777+0000","last_undegraded":"2023-11-15T02:59:34.270777+0000","last_fullsized":"2023-11-15T02:59:34.270777+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"60'4","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.721503+0000","last_change":"2023-11-15T02:59:34.264405+0000","last_active":"2023-11-15T02:59:34.721503+0000","last_peered":"2023-11-15T02:59:34.721503+0000","last_clean":"2023-11-15T02:59:34.721503+0000","last_became_active":"2023-11-15T02:59:34.264095+0000","last_became_peered":"2023-11-15T02:59:34.264095+0000","last_unstale":"2023-11-15T02:59:34.721503+0000","last_undegraded":"2023-11-15T02:59:34.721503+0000","last_fullsized":"2023-11-15T02:59:34.721503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.694211+0000","last_change":"2023-11-15T02:59:34.694211+0000","last_active":"2023-11-15T02:59:34.694211+0000","last_peered":"2023-11-15T02:59:34.694211+0000","last_clean":"2023-11-15T02:59:34.694211+0000","last_became_active":"2023-11-15T02:59:34.693708+0000","last_became_peered":"2023-11-15T02:59:34.693708+0000","last_unstale":"2023-11-15T02:59:34.694211+0000","last_undegraded":"2023-11-15T02:59:34.694211+0000","last_fullsized":"2023-11-15T02:59:34.694211+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278070+0000","last_change":"2023-11-15T02:59:34.271015+0000","last_active":"2023-11-15T02:59:34.278070+0000","last_peered":"2023-11-15T02:59:34.278070+0000","last_clean":"2023-11-15T02:59:34.278070+0000","last_became_active":"2023-11-15T02:59:34.270758+0000","last_became_peered":"2023-11-15T02:59:34.270758+0000","last_unstale":"2023-11-15T02:59:34.278070+0000","last_undegraded":"2023-11-15T02:59:34.278070+0000","last_fullsized":"2023-11-15T02:59:34.278070+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.721266+0000","last_change":"2023-11-15T02:59:34.721266+0000","last_active":"2023-11-15T02:59:34.721266+0000","last_peered":"2023-11-15T02:59:34.721266+0000","last_clean":"2023-11-15T02:59:34.721266+0000","last_became_active":"2023-11-15T02:59:34.720978+0000","last_became_peered":"2023-11-15T02:59:34.720978+0000","last_unstale":"2023-11-15T02:59:34.721266+0000","last_undegraded":"2023-11-15T02:59:34.721266+0000","last_fullsized":"2023-11-15T02:59:34.721266+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.276457+0000","last_change":"2023-11-15T02:59:34.269316+0000","last_active":"2023-11-15T02:59:34.276457+0000","last_peered":"2023-11-15T02:59:34.276457+0000","last_clean":"2023-11-15T02:59:34.276457+0000","last_became_active":"2023-11-15T02:59:34.268953+0000","last_became_peered":"2023-11-15T02:59:34.268953+0000","last_unstale":"2023-11-15T02:59:34.276457+0000","last_undegraded":"2023-11-15T02:59:34.276457+0000","last_fullsized":"2023-11-15T02:59:34.276457+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.703052+0000","last_change":"2023-11-15T02:59:34.703052+0000","last_active":"2023-11-15T02:59:34.703052+0000","last_peered":"2023-11-15T02:59:34.703052+0000","last_clean":"2023-11-15T02:59:34.703052+0000","last_became_active":"2023-11-15T02:59:34.702791+0000","last_became_peered":"2023-11-15T02:59:34.702791+0000","last_unstale":"2023-11-15T02:59:34.703052+0000","last_undegraded":"2023-11-15T02:59:34.703052+0000","last_fullsized":"2023-11-15T02:59:34.703052+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2d","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.696289+0000","last_change":"2023-11-15T02:59:34.257033+0000","last_active":"2023-11-15T02:59:34.696289+0000","last_peered":"2023-11-15T02:59:34.696289+0000","last_clean":"2023-11-15T02:59:34.696289+0000","last_became_active":"2023-11-15T02:59:34.256744+0000","last_became_peered":"2023-11-15T02:59:34.256744+0000","last_unstale":"2023-11-15T02:59:34.696289+0000","last_undegraded":"2023-11-15T02:59:34.696289+0000","last_fullsized":"2023-11-15T02:59:34.696289+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.280852+0000","last_change":"2023-11-15T02:59:34.275824+0000","last_active":"2023-11-15T02:59:34.280852+0000","last_peered":"2023-11-15T02:59:34.280852+0000","last_clean":"2023-11-15T02:59:34.280852+0000","last_became_active":"2023-11-15T02:59:34.275088+0000","last_became_peered":"2023-11-15T02:59:34.275088+0000","last_unstale":"2023-11-15T02:59:34.280852+0000","last_undegraded":"2023-11-15T02:59:34.280852+0000","last_fullsized":"2023-11-15T02:59:34.280852+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.718638+0000","last_change":"2023-11-15T02:59:34.718638+0000","last_active":"2023-11-15T02:59:34.718638+0000","last_peered":"2023-11-15T02:59:34.718638+0000","last_clean":"2023-11-15T02:59:34.718638+0000","last_became_active":"2023-11-15T02:59:34.718018+0000","last_became_peered":"2023-11-15T02:59:34.718018+0000","last_unstale":"2023-11-15T02:59:34.718638+0000","last_undegraded":"2023-11-15T02:59:34.718638+0000","last_fullsized":"2023-11-15T02:59:34.718638+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2b","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.688086+0000","last_change":"2023-11-15T02:59:34.271365+0000","last_active":"2023-11-15T02:59:34.688086+0000","last_peered":"2023-11-15T02:59:34.688086+0000","last_clean":"2023-11-15T02:59:34.688086+0000","last_became_active":"2023-11-15T02:59:34.270414+0000","last_became_peered":"2023-11-15T02:59:34.270414+0000","last_unstale":"2023-11-15T02:59:34.688086+0000","last_undegraded":"2023-11-15T02:59:34.688086+0000","last_fullsized":"2023-11-15T02:59:34.688086+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.705469+0000","last_change":"2023-11-15T02:59:34.705469+0000","last_active":"2023-11-15T02:59:34.705469+0000","last_peered":"2023-11-15T02:59:34.705469+0000","last_clean":"2023-11-15T02:59:34.705469+0000","last_became_active":"2023-11-15T02:59:34.705188+0000","last_became_peered":"2023-11-15T02:59:34.705188+0000","last_unstale":"2023-11-15T02:59:34.705469+0000","last_undegraded":"2023-11-15T02:59:34.705469+0000","last_fullsized":"2023-11-15T02:59:34.705469+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.274106+0000","last_change":"2023-11-15T02:59:34.269772+0000","last_active":"2023-11-15T02:59:34.274106+0000","last_peered":"2023-11-15T02:59:34.274106+0000","last_clean":"2023-11-15T02:59:34.274106+0000","last_became_active":"2023-11-15T02:59:34.269150+0000","last_became_peered":"2023-11-15T02:59:34.269150+0000","last_unstale":"2023-11-15T02:59:34.274106+0000","last_undegraded":"2023-11-15T02:59:34.274106+0000","last_fullsized":"2023-11-15T02:59:34.274106+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.706266+0000","last_change":"2023-11-15T02:59:34.706266+0000","last_active":"2023-11-15T02:59:34.706266+0000","last_peered":"2023-11-15T02:59:34.706266+0000","last_clean":"2023-11-15T02:59:34.706266+0000","last_became_active":"2023-11-15T02:59:34.705045+0000","last_became_peered":"2023-11-15T02:59:34.705045+0000","last_unstale":"2023-11-15T02:59:34.706266+0000","last_undegraded":"2023-11-15T02:59:34.706266+0000","last_fullsized":"2023-11-15T02:59:34.706266+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.275956+0000","last_change":"2023-11-15T02:59:34.271476+0000","last_active":"2023-11-15T02:59:34.275956+0000","last_peered":"2023-11-15T02:59:34.275956+0000","last_clean":"2023-11-15T02:59:34.275956+0000","last_became_active":"2023-11-15T02:59:34.271135+0000","last_became_peered":"2023-11-15T02:59:34.271135+0000","last_unstale":"2023-11-15T02:59:34.275956+0000","last_undegraded":"2023-11-15T02:59:34.275956+0000","last_fullsized":"2023-11-15T02:59:34.275956+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.268460+0000","last_change":"2023-11-15T02:59:34.264047+0000","last_active":"2023-11-15T02:59:34.268460+0000","last_peered":"2023-11-15T02:59:34.268460+0000","last_clean":"2023-11-15T02:59:34.268460+0000","last_became_active":"2023-11-15T02:59:34.263525+0000","last_became_peered":"2023-11-15T02:59:34.263525+0000","last_unstale":"2023-11-15T02:59:34.268460+0000","last_undegraded":"2023-11-15T02:59:34.268460+0000","last_fullsized":"2023-11-15T02:59:34.268460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.264845+0000","last_change":"2023-11-15T02:59:34.253441+0000","last_active":"2023-11-15T02:59:34.264845+0000","last_peered":"2023-11-15T02:59:34.264845+0000","last_clean":"2023-11-15T02:59:34.264845+0000","last_became_active":"2023-11-15T02:59:34.253172+0000","last_became_peered":"2023-11-15T02:59:34.253172+0000","last_unstale":"2023-11-15T02:59:34.264845+0000","last_undegraded":"2023-11-15T02:59:34.264845+0000","last_fullsized":"2023-11-15T02:59:34.264845+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.706426+0000","last_change":"2023-11-15T02:59:34.706426+0000","last_active":"2023-11-15T02:59:34.706426+0000","last_peered":"2023-11-15T02:59:34.706426+0000","last_clean":"2023-11-15T02:59:34.706426+0000","last_became_active":"2023-11-15T02:59:34.705567+0000","last_became_peered":"2023-11-15T02:59:34.705567+0000","last_unstale":"2023-11-15T02:59:34.706426+0000","last_undegraded":"2023-11-15T02:59:34.706426+0000","last_fullsized":"2023-11-15T02:59:34.706426+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.229642+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.275473+0000","last_change":"2023-11-15T02:59:34.264728+0000","last_active":"2023-11-15T02:59:34.275473+0000","last_peered":"2023-11-15T02:59:34.275473+0000","last_clean":"2023-11-15T02:59:34.275473+0000","last_became_active":"2023-11-15T02:59:34.264416+0000","last_became_peered":"2023-11-15T02:59:34.264416+0000","last_unstale":"2023-11-15T02:59:34.275473+0000","last_undegraded":"2023-11-15T02:59:34.275473+0000","last_fullsized":"2023-11-15T02:59:34.275473+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.270389+0000","last_change":"2023-11-15T02:59:34.261680+0000","last_active":"2023-11-15T02:59:34.270389+0000","last_peered":"2023-11-15T02:59:34.270389+0000","last_clean":"2023-11-15T02:59:34.270389+0000","last_became_active":"2023-11-15T02:59:34.260538+0000","last_became_peered":"2023-11-15T02:59:34.260538+0000","last_unstale":"2023-11-15T02:59:34.270389+0000","last_undegraded":"2023-11-15T02:59:34.270389+0000","last_fullsized":"2023-11-15T02:59:34.270389+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.707619+0000","last_change":"2023-11-15T02:59:34.707619+0000","last_active":"2023-11-15T02:59:34.707619+0000","last_peered":"2023-11-15T02:59:34.707619+0000","last_clean":"2023-11-15T02:59:34.707619+0000","last_became_active":"2023-11-15T02:59:34.707297+0000","last_became_peered":"2023-11-15T02:59:34.707297+0000","last_unstale":"2023-11-15T02:59:34.707619+0000","last_undegraded":"2023-11-15T02:59:34.707619+0000","last_fullsized":"2023-11-15T02:59:34.707619+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.272192+0000","last_change":"2023-11-15T02:59:34.266701+0000","last_active":"2023-11-15T02:59:34.272192+0000","last_peered":"2023-11-15T02:59:34.272192+0000","last_clean":"2023-11-15T02:59:34.272192+0000","last_became_active":"2023-11-15T02:59:34.266347+0000","last_became_peered":"2023-11-15T02:59:34.266347+0000","last_unstale":"2023-11-15T02:59:34.272192+0000","last_undegraded":"2023-11-15T02:59:34.272192+0000","last_fullsized":"2023-11-15T02:59:34.272192+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.705873+0000","last_change":"2023-11-15T02:59:34.705873+0000","last_active":"2023-11-15T02:59:34.705873+0000","last_peered":"2023-11-15T02:59:34.705873+0000","last_clean":"2023-11-15T02:59:34.705873+0000","last_became_active":"2023-11-15T02:59:34.705522+0000","last_became_peered":"2023-11-15T02:59:34.705522+0000","last_unstale":"2023-11-15T02:59:34.705873+0000","last_undegraded":"2023-11-15T02:59:34.705873+0000","last_fullsized":"2023-11-15T02:59:34.705873+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278740+0000","last_change":"2023-11-15T02:59:34.274845+0000","last_active":"2023-11-15T02:59:34.278740+0000","last_peered":"2023-11-15T02:59:34.278740+0000","last_clean":"2023-11-15T02:59:34.278740+0000","last_became_active":"2023-11-15T02:59:34.274495+0000","last_became_peered":"2023-11-15T02:59:34.274495+0000","last_unstale":"2023-11-15T02:59:34.278740+0000","last_undegraded":"2023-11-15T02:59:34.278740+0000","last_fullsized":"2023-11-15T02:59:34.278740+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.271334+0000","last_change":"2023-11-15T02:59:34.262355+0000","last_active":"2023-11-15T02:59:34.271334+0000","last_peered":"2023-11-15T02:59:34.271334+0000","last_clean":"2023-11-15T02:59:34.271334+0000","last_became_active":"2023-11-15T02:59:34.260844+0000","last_became_peered":"2023-11-15T02:59:34.260844+0000","last_unstale":"2023-11-15T02:59:34.271334+0000","last_undegraded":"2023-11-15T02:59:34.271334+0000","last_fullsized":"2023-11-15T02:59:34.271334+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.711149+0000","last_change":"2023-11-15T02:59:34.711149+0000","last_active":"2023-11-15T02:59:34.711149+0000","last_peered":"2023-11-15T02:59:34.711149+0000","last_clean":"2023-11-15T02:59:34.711149+0000","last_became_active":"2023-11-15T02:59:34.710867+0000","last_became_peered":"2023-11-15T02:59:34.710867+0000","last_unstale":"2023-11-15T02:59:34.711149+0000","last_undegraded":"2023-11-15T02:59:34.711149+0000","last_fullsized":"2023-11-15T02:59:34.711149+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278038+0000","last_change":"2023-11-15T02:59:34.276394+0000","last_active":"2023-11-15T02:59:34.278038+0000","last_peered":"2023-11-15T02:59:34.278038+0000","last_clean":"2023-11-15T02:59:34.278038+0000","last_became_active":"2023-11-15T02:59:34.276134+0000","last_became_peered":"2023-11-15T02:59:34.276134+0000","last_unstale":"2023-11-15T02:59:34.278038+0000","last_undegraded":"2023-11-15T02:59:34.278038+0000","last_fullsized":"2023-11-15T02:59:34.278038+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.697057+0000","last_change":"2023-11-15T02:59:34.697057+0000","last_active":"2023-11-15T02:59:34.697057+0000","last_peered":"2023-11-15T02:59:34.697057+0000","last_clean":"2023-11-15T02:59:34.697057+0000","last_became_active":"2023-11-15T02:59:34.696211+0000","last_became_peered":"2023-11-15T02:59:34.696211+0000","last_unstale":"2023-11-15T02:59:34.697057+0000","last_undegraded":"2023-11-15T02:59:34.697057+0000","last_fullsized":"2023-11-15T02:59:34.697057+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.207077+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.695873+0000","last_change":"2023-11-15T02:59:34.695873+0000","last_active":"2023-11-15T02:59:34.695873+0000","last_peered":"2023-11-15T02:59:34.695873+0000","last_clean":"2023-11-15T02:59:34.695873+0000","last_became_active":"2023-11-15T02:59:34.695590+0000","last_became_peered":"2023-11-15T02:59:34.695590+0000","last_unstale":"2023-11-15T02:59:34.695873+0000","last_undegraded":"2023-11-15T02:59:34.695873+0000","last_fullsized":"2023-11-15T02:59:34.695873+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.693584+0000","last_change":"2023-11-15T02:59:34.272647+0000","last_active":"2023-11-15T02:59:34.693584+0000","last_peered":"2023-11-15T02:59:34.693584+0000","last_clean":"2023-11-15T02:59:34.693584+0000","last_became_active":"2023-11-15T02:59:34.272331+0000","last_became_peered":"2023-11-15T02:59:34.272331+0000","last_unstale":"2023-11-15T02:59:34.693584+0000","last_undegraded":"2023-11-15T02:59:34.693584+0000","last_fullsized":"2023-11-15T02:59:34.693584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.710364+0000","last_change":"2023-11-15T02:59:34.710364+0000","last_active":"2023-11-15T02:59:34.710364+0000","last_peered":"2023-11-15T02:59:34.710364+0000","last_clean":"2023-11-15T02:59:34.710364+0000","last_became_active":"2023-11-15T02:59:34.706803+0000","last_became_peered":"2023-11-15T02:59:34.706803+0000","last_unstale":"2023-11-15T02:59:34.710364+0000","last_undegraded":"2023-11-15T02:59:34.710364+0000","last_fullsized":"2023-11-15T02:59:34.710364+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.710145+0000","last_change":"2023-11-15T02:59:34.263720+0000","last_active":"2023-11-15T02:59:34.710145+0000","last_peered":"2023-11-15T02:59:34.710145+0000","last_clean":"2023-11-15T02:59:34.710145+0000","last_became_active":"2023-11-15T02:59:34.262994+0000","last_became_peered":"2023-11-15T02:59:34.262994+0000","last_unstale":"2023-11-15T02:59:34.710145+0000","last_undegraded":"2023-11-15T02:59:34.710145+0000","last_fullsized":"2023-11-15T02:59:34.710145+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.702473+0000","last_change":"2023-11-15T02:59:34.702473+0000","last_active":"2023-11-15T02:59:34.702473+0000","last_peered":"2023-11-15T02:59:34.702473+0000","last_clean":"2023-11-15T02:59:34.702473+0000","last_became_active":"2023-11-15T02:59:34.702094+0000","last_became_peered":"2023-11-15T02:59:34.702094+0000","last_unstale":"2023-11-15T02:59:34.702473+0000","last_undegraded":"2023-11-15T02:59:34.702473+0000","last_fullsized":"2023-11-15T02:59:34.702473+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.271846+0000","last_change":"2023-11-15T02:59:34.260019+0000","last_active":"2023-11-15T02:59:34.271846+0000","last_peered":"2023-11-15T02:59:34.271846+0000","last_clean":"2023-11-15T02:59:34.271846+0000","last_became_active":"2023-11-15T02:59:34.259470+0000","last_became_peered":"2023-11-15T02:59:34.259470+0000","last_unstale":"2023-11-15T02:59:34.271846+0000","last_undegraded":"2023-11-15T02:59:34.271846+0000","last_fullsized":"2023-11-15T02:59:34.271846+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.707408+0000","last_change":"2023-11-15T02:59:34.707408+0000","last_active":"2023-11-15T02:59:34.707408+0000","last_peered":"2023-11-15T02:59:34.707408+0000","last_clean":"2023-11-15T02:59:34.707408+0000","last_became_active":"2023-11-15T02:59:34.706951+0000","last_became_peered":"2023-11-15T02:59:34.706951+0000","last_unstale":"2023-11-15T02:59:34.707408+0000","last_undegraded":"2023-11-15T02:59:34.707408+0000","last_fullsized":"2023-11-15T02:59:34.707408+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.280951+0000","last_change":"2023-11-15T02:59:34.276164+0000","last_active":"2023-11-15T02:59:34.280951+0000","last_peered":"2023-11-15T02:59:34.280951+0000","last_clean":"2023-11-15T02:59:34.280951+0000","last_became_active":"2023-11-15T02:59:34.275881+0000","last_became_peered":"2023-11-15T02:59:34.275881+0000","last_unstale":"2023-11-15T02:59:34.280951+0000","last_undegraded":"2023-11-15T02:59:34.280951+0000","last_fullsized":"2023-11-15T02:59:34.280951+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.274124+0000","last_change":"2023-11-15T02:59:34.269967+0000","last_active":"2023-11-15T02:59:34.274124+0000","last_peered":"2023-11-15T02:59:34.274124+0000","last_clean":"2023-11-15T02:59:34.274124+0000","last_became_active":"2023-11-15T02:59:34.269516+0000","last_became_peered":"2023-11-15T02:59:34.269516+0000","last_unstale":"2023-11-15T02:59:34.274124+0000","last_undegraded":"2023-11-15T02:59:34.274124+0000","last_fullsized":"2023-11-15T02:59:34.274124+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.713129+0000","last_change":"2023-11-15T02:59:34.713129+0000","last_active":"2023-11-15T02:59:34.713129+0000","last_peered":"2023-11-15T02:59:34.713129+0000","last_clean":"2023-11-15T02:59:34.713129+0000","last_became_active":"2023-11-15T02:59:34.704067+0000","last_became_peered":"2023-11-15T02:59:34.704067+0000","last_unstale":"2023-11-15T02:59:34.713129+0000","last_undegraded":"2023-11-15T02:59:34.713129+0000","last_fullsized":"2023-11-15T02:59:34.713129+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278113+0000","last_change":"2023-11-15T02:59:34.272334+0000","last_active":"2023-11-15T02:59:34.278113+0000","last_peered":"2023-11-15T02:59:34.278113+0000","last_clean":"2023-11-15T02:59:34.278113+0000","last_became_active":"2023-11-15T02:59:34.272096+0000","last_became_peered":"2023-11-15T02:59:34.272096+0000","last_unstale":"2023-11-15T02:59:34.278113+0000","last_undegraded":"2023-11-15T02:59:34.278113+0000","last_fullsized":"2023-11-15T02:59:34.278113+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.686879+0000","last_change":"2023-11-15T02:59:34.686879+0000","last_active":"2023-11-15T02:59:34.686879+0000","last_peered":"2023-11-15T02:59:34.686879+0000","last_clean":"2023-11-15T02:59:34.686879+0000","last_became_active":"2023-11-15T02:59:34.685697+0000","last_became_peered":"2023-11-15T02:59:34.685697+0000","last_unstale":"2023-11-15T02:59:34.686879+0000","last_undegraded":"2023-11-15T02:59:34.686879+0000","last_fullsized":"2023-11-15T02:59:34.686879+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.275520+0000","last_change":"2023-11-15T02:59:34.270391+0000","last_active":"2023-11-15T02:59:34.275520+0000","last_peered":"2023-11-15T02:59:34.275520+0000","last_clean":"2023-11-15T02:59:34.275520+0000","last_became_active":"2023-11-15T02:59:34.270049+0000","last_became_peered":"2023-11-15T02:59:34.270049+0000","last_unstale":"2023-11-15T02:59:34.275520+0000","last_undegraded":"2023-11-15T02:59:34.275520+0000","last_fullsized":"2023-11-15T02:59:34.275520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.282770+0000","last_change":"2023-11-15T02:59:34.280324+0000","last_active":"2023-11-15T02:59:34.282770+0000","last_peered":"2023-11-15T02:59:34.282770+0000","last_clean":"2023-11-15T02:59:34.282770+0000","last_became_active":"2023-11-15T02:59:34.280003+0000","last_became_peered":"2023-11-15T02:59:34.280003+0000","last_unstale":"2023-11-15T02:59:34.282770+0000","last_undegraded":"2023-11-15T02:59:34.282770+0000","last_fullsized":"2023-11-15T02:59:34.282770+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.706837+0000","last_change":"2023-11-15T02:59:34.706837+0000","last_active":"2023-11-15T02:59:34.706837+0000","last_peered":"2023-11-15T02:59:34.706837+0000","last_clean":"2023-11-15T02:59:34.706837+0000","last_became_active":"2023-11-15T02:59:34.706595+0000","last_became_peered":"2023-11-15T02:59:34.706595+0000","last_unstale":"2023-11-15T02:59:34.706837+0000","last_undegraded":"2023-11-15T02:59:34.706837+0000","last_fullsized":"2023-11-15T02:59:34.706837+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278312+0000","last_change":"2023-11-15T02:59:34.274116+0000","last_active":"2023-11-15T02:59:34.278312+0000","last_peered":"2023-11-15T02:59:34.278312+0000","last_clean":"2023-11-15T02:59:34.278312+0000","last_became_active":"2023-11-15T02:59:34.273108+0000","last_became_peered":"2023-11-15T02:59:34.273108+0000","last_unstale":"2023-11-15T02:59:34.278312+0000","last_undegraded":"2023-11-15T02:59:34.278312+0000","last_fullsized":"2023-11-15T02:59:34.278312+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.713931+0000","last_change":"2023-11-15T02:59:34.713931+0000","last_active":"2023-11-15T02:59:34.713931+0000","last_peered":"2023-11-15T02:59:34.713931+0000","last_clean":"2023-11-15T02:59:34.713931+0000","last_became_active":"2023-11-15T02:59:34.713640+0000","last_became_peered":"2023-11-15T02:59:34.713640+0000","last_unstale":"2023-11-15T02:59:34.713931+0000","last_undegraded":"2023-11-15T02:59:34.713931+0000","last_fullsized":"2023-11-15T02:59:34.713931+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"60'3","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.704814+0000","last_change":"2023-11-15T02:59:34.273818+0000","last_active":"2023-11-15T02:59:34.704814+0000","last_peered":"2023-11-15T02:59:34.704814+0000","last_clean":"2023-11-15T02:59:34.704814+0000","last_became_active":"2023-11-15T02:59:34.273463+0000","last_became_peered":"2023-11-15T02:59:34.273463+0000","last_unstale":"2023-11-15T02:59:34.704814+0000","last_undegraded":"2023-11-15T02:59:34.704814+0000","last_fullsized":"2023-11-15T02:59:34.704814+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.691994+0000","last_change":"2023-11-15T02:59:34.691994+0000","last_active":"2023-11-15T02:59:34.691994+0000","last_peered":"2023-11-15T02:59:34.691994+0000","last_clean":"2023-11-15T02:59:34.691994+0000","last_became_active":"2023-11-15T02:59:34.691573+0000","last_became_peered":"2023-11-15T02:59:34.691573+0000","last_unstale":"2023-11-15T02:59:34.691994+0000","last_undegraded":"2023-11-15T02:59:34.691994+0000","last_fullsized":"2023-11-15T02:59:34.691994+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.207077+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.708893+0000","last_change":"2023-11-15T02:59:34.708893+0000","last_active":"2023-11-15T02:59:34.708893+0000","last_peered":"2023-11-15T02:59:34.708893+0000","last_clean":"2023-11-15T02:59:34.708893+0000","last_became_active":"2023-11-15T02:59:34.708623+0000","last_became_peered":"2023-11-15T02:59:34.708623+0000","last_unstale":"2023-11-15T02:59:34.708893+0000","last_undegraded":"2023-11-15T02:59:34.708893+0000","last_fullsized":"2023-11-15T02:59:34.708893+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.688236+0000","last_change":"2023-11-15T02:59:34.270907+0000","last_active":"2023-11-15T02:59:34.688236+0000","last_peered":"2023-11-15T02:59:34.688236+0000","last_clean":"2023-11-15T02:59:34.688236+0000","last_became_active":"2023-11-15T02:59:34.270561+0000","last_became_peered":"2023-11-15T02:59:34.270561+0000","last_unstale":"2023-11-15T02:59:34.688236+0000","last_undegraded":"2023-11-15T02:59:34.688236+0000","last_fullsized":"2023-11-15T02:59:34.688236+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.230568+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.719857+0000","last_change":"2023-11-15T02:59:34.719857+0000","last_active":"2023-11-15T02:59:34.719857+0000","last_peered":"2023-11-15T02:59:34.719857+0000","last_clean":"2023-11-15T02:59:34.719857+0000","last_became_active":"2023-11-15T02:59:34.719591+0000","last_became_peered":"2023-11-15T02:59:34.719591+0000","last_unstale":"2023-11-15T02:59:34.719857+0000","last_undegraded":"2023-11-15T02:59:34.719857+0000","last_fullsized":"2023-11-15T02:59:34.719857+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.279642+0000","last_change":"2023-11-15T02:59:34.276260+0000","last_active":"2023-11-15T02:59:34.279642+0000","last_peered":"2023-11-15T02:59:34.279642+0000","last_clean":"2023-11-15T02:59:34.279642+0000","last_became_active":"2023-11-15T02:59:34.275758+0000","last_became_peered":"2023-11-15T02:59:34.275758+0000","last_unstale":"2023-11-15T02:59:34.279642+0000","last_undegraded":"2023-11-15T02:59:34.279642+0000","last_fullsized":"2023-11-15T02:59:34.279642+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"60'4","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.722402+0000","last_change":"2023-11-15T02:59:34.269037+0000","last_active":"2023-11-15T02:59:34.722402+0000","last_peered":"2023-11-15T02:59:34.722402+0000","last_clean":"2023-11-15T02:59:34.722402+0000","last_became_active":"2023-11-15T02:59:34.268123+0000","last_became_peered":"2023-11-15T02:59:34.268123+0000","last_unstale":"2023-11-15T02:59:34.722402+0000","last_undegraded":"2023-11-15T02:59:34.722402+0000","last_fullsized":"2023-11-15T02:59:34.722402+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.715434+0000","last_change":"2023-11-15T02:59:34.715434+0000","last_active":"2023-11-15T02:59:34.715434+0000","last_peered":"2023-11-15T02:59:34.715434+0000","last_clean":"2023-11-15T02:59:34.715434+0000","last_became_active":"2023-11-15T02:59:34.715198+0000","last_became_peered":"2023-11-15T02:59:34.715198+0000","last_unstale":"2023-11-15T02:59:34.715434+0000","last_undegraded":"2023-11-15T02:59:34.715434+0000","last_fullsized":"2023-11-15T02:59:34.715434+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.228982+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.705591+0000","last_change":"2023-11-15T02:59:34.705591+0000","last_active":"2023-11-15T02:59:34.705591+0000","last_peered":"2023-11-15T02:59:34.705591+0000","last_clean":"2023-11-15T02:59:34.705591+0000","last_became_active":"2023-11-15T02:59:34.703826+0000","last_became_peered":"2023-11-15T02:59:34.703826+0000","last_unstale":"2023-11-15T02:59:34.705591+0000","last_undegraded":"2023-11-15T02:59:34.705591+0000","last_fullsized":"2023-11-15T02:59:34.705591+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.228971+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.273908+0000","last_change":"2023-11-15T02:59:34.263759+0000","last_active":"2023-11-15T02:59:34.273908+0000","last_peered":"2023-11-15T02:59:34.273908+0000","last_clean":"2023-11-15T02:59:34.273908+0000","last_became_active":"2023-11-15T02:59:34.263333+0000","last_became_peered":"2023-11-15T02:59:34.263333+0000","last_unstale":"2023-11-15T02:59:34.273908+0000","last_undegraded":"2023-11-15T02:59:34.273908+0000","last_fullsized":"2023-11-15T02:59:34.273908+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.700382+0000","last_change":"2023-11-15T02:59:34.700382+0000","last_active":"2023-11-15T02:59:34.700382+0000","last_peered":"2023-11-15T02:59:34.700382+0000","last_clean":"2023-11-15T02:59:34.700382+0000","last_became_active":"2023-11-15T02:59:34.700091+0000","last_became_peered":"2023-11-15T02:59:34.700091+0000","last_unstale":"2023-11-15T02:59:34.700382+0000","last_undegraded":"2023-11-15T02:59:34.700382+0000","last_fullsized":"2023-11-15T02:59:34.700382+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.207077+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.697102+0000","last_change":"2023-11-15T02:59:34.697102+0000","last_active":"2023-11-15T02:59:34.697102+0000","last_peered":"2023-11-15T02:59:34.697102+0000","last_clean":"2023-11-15T02:59:34.697102+0000","last_became_active":"2023-11-15T02:59:34.696095+0000","last_became_peered":"2023-11-15T02:59:34.696095+0000","last_unstale":"2023-11-15T02:59:34.697102+0000","last_undegraded":"2023-11-15T02:59:34.697102+0000","last_fullsized":"2023-11-15T02:59:34.697102+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.279613+0000","last_change":"2023-11-15T02:59:34.275904+0000","last_active":"2023-11-15T02:59:34.279613+0000","last_peered":"2023-11-15T02:59:34.279613+0000","last_clean":"2023-11-15T02:59:34.279613+0000","last_became_active":"2023-11-15T02:59:34.275561+0000","last_became_peered":"2023-11-15T02:59:34.275561+0000","last_unstale":"2023-11-15T02:59:34.279613+0000","last_undegraded":"2023-11-15T02:59:34.279613+0000","last_fullsized":"2023-11-15T02:59:34.279613+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.710944+0000","last_change":"2023-11-15T02:59:34.710944+0000","last_active":"2023-11-15T02:59:34.710944+0000","last_peered":"2023-11-15T02:59:34.710944+0000","last_clean":"2023-11-15T02:59:34.710944+0000","last_became_active":"2023-11-15T02:59:34.710640+0000","last_became_peered":"2023-11-15T02:59:34.710640+0000","last_unstale":"2023-11-15T02:59:34.710944+0000","last_undegraded":"2023-11-15T02:59:34.710944+0000","last_fullsized":"2023-11-15T02:59:34.710944+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.280896+0000","last_change":"2023-11-15T02:59:34.275717+0000","last_active":"2023-11-15T02:59:34.280896+0000","last_peered":"2023-11-15T02:59:34.280896+0000","last_clean":"2023-11-15T02:59:34.280896+0000","last_became_active":"2023-11-15T02:59:34.274314+0000","last_became_peered":"2023-11-15T02:59:34.274314+0000","last_unstale":"2023-11-15T02:59:34.280896+0000","last_undegraded":"2023-11-15T02:59:34.280896+0000","last_fullsized":"2023-11-15T02:59:34.280896+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278096+0000","last_change":"2023-11-15T02:59:34.271643+0000","last_active":"2023-11-15T02:59:34.278096+0000","last_peered":"2023-11-15T02:59:34.278096+0000","last_clean":"2023-11-15T02:59:34.278096+0000","last_became_active":"2023-11-15T02:59:34.271375+0000","last_became_peered":"2023-11-15T02:59:34.271375+0000","last_unstale":"2023-11-15T02:59:34.278096+0000","last_undegraded":"2023-11-15T02:59:34.278096+0000","last_fullsized":"2023-11-15T02:59:34.278096+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.711599+0000","last_change":"2023-11-15T02:59:34.711599+0000","last_active":"2023-11-15T02:59:34.711599+0000","last_peered":"2023-11-15T02:59:34.711599+0000","last_clean":"2023-11-15T02:59:34.711599+0000","last_became_active":"2023-11-15T02:59:34.711325+0000","last_became_peered":"2023-11-15T02:59:34.711325+0000","last_unstale":"2023-11-15T02:59:34.711599+0000","last_undegraded":"2023-11-15T02:59:34.711599+0000","last_fullsized":"2023-11-15T02:59:34.711599+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.207077+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.699657+0000","last_change":"2023-11-15T02:59:34.699657+0000","last_active":"2023-11-15T02:59:34.699657+0000","last_peered":"2023-11-15T02:59:34.699657+0000","last_clean":"2023-11-15T02:59:34.699657+0000","last_became_active":"2023-11-15T02:59:34.699371+0000","last_became_peered":"2023-11-15T02:59:34.699371+0000","last_unstale":"2023-11-15T02:59:34.699657+0000","last_undegraded":"2023-11-15T02:59:34.699657+0000","last_fullsized":"2023-11-15T02:59:34.699657+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.272131+0000","last_change":"2023-11-15T02:59:34.264863+0000","last_active":"2023-11-15T02:59:34.272131+0000","last_peered":"2023-11-15T02:59:34.272131+0000","last_clean":"2023-11-15T02:59:34.272131+0000","last_became_active":"2023-11-15T02:59:34.264228+0000","last_became_peered":"2023-11-15T02:59:34.264228+0000","last_unstale":"2023-11-15T02:59:34.272131+0000","last_undegraded":"2023-11-15T02:59:34.272131+0000","last_fullsized":"2023-11-15T02:59:34.272131+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.252284+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T02:59:34.252284+0000","last_peered":"2023-11-15T02:59:34.252284+0000","last_clean":"2023-11-15T02:59:34.252284+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T02:59:34.252284+0000","last_undegraded":"2023-11-15T02:59:34.252284+0000","last_fullsized":"2023-11-15T02:59:34.252284+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.694895+0000","last_change":"2023-11-15T02:59:34.694895+0000","last_active":"2023-11-15T02:59:34.694895+0000","last_peered":"2023-11-15T02:59:34.694895+0000","last_clean":"2023-11-15T02:59:34.694895+0000","last_became_active":"2023-11-15T02:59:34.694592+0000","last_became_peered":"2023-11-15T02:59:34.694592+0000","last_unstale":"2023-11-15T02:59:34.694895+0000","last_undegraded":"2023-11-15T02:59:34.694895+0000","last_fullsized":"2023-11-15T02:59:34.694895+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.282767+0000","last_change":"2023-11-15T02:59:34.280250+0000","last_active":"2023-11-15T02:59:34.282767+0000","last_peered":"2023-11-15T02:59:34.282767+0000","last_clean":"2023-11-15T02:59:34.282767+0000","last_became_active":"2023-11-15T02:59:34.279961+0000","last_became_peered":"2023-11-15T02:59:34.279961+0000","last_unstale":"2023-11-15T02:59:34.282767+0000","last_undegraded":"2023-11-15T02:59:34.282767+0000","last_fullsized":"2023-11-15T02:59:34.282767+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-15T02:59:29.318446+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T02:59:29.318446+0000","last_peered":"2023-11-15T02:59:29.318446+0000","last_clean":"2023-11-15T02:59:29.318446+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T02:59:29.318446+0000","last_undegraded":"2023-11-15T02:59:29.318446+0000","last_fullsized":"2023-11-15T02:59:29.318446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.720662+0000","last_change":"2023-11-15T02:59:34.720662+0000","last_active":"2023-11-15T02:59:34.720662+0000","last_peered":"2023-11-15T02:59:34.720662+0000","last_clean":"2023-11-15T02:59:34.720662+0000","last_became_active":"2023-11-15T02:59:34.713450+0000","last_became_peered":"2023-11-15T02:59:34.713450+0000","last_unstale":"2023-11-15T02:59:34.720662+0000","last_undegraded":"2023-11-15T02:59:34.720662+0000","last_fullsized":"2023-11-15T02:59:34.720662+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716637+0000","last_change":"2023-11-15T02:59:34.272609+0000","last_active":"2023-11-15T02:59:34.716637+0000","last_peered":"2023-11-15T02:59:34.716637+0000","last_clean":"2023-11-15T02:59:34.716637+0000","last_became_active":"2023-11-15T02:59:34.272212+0000","last_became_peered":"2023-11-15T02:59:34.272212+0000","last_unstale":"2023-11-15T02:59:34.716637+0000","last_undegraded":"2023-11-15T02:59:34.716637+0000","last_fullsized":"2023-11-15T02:59:34.716637+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.257610+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T02:59:34.257610+0000","last_peered":"2023-11-15T02:59:34.257610+0000","last_clean":"2023-11-15T02:59:34.257610+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T02:59:34.257610+0000","last_undegraded":"2023-11-15T02:59:34.257610+0000","last_fullsized":"2023-11-15T02:59:34.257610+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.711825+0000","last_change":"2023-11-15T02:59:34.711825+0000","last_active":"2023-11-15T02:59:34.711825+0000","last_peered":"2023-11-15T02:59:34.711825+0000","last_clean":"2023-11-15T02:59:34.711825+0000","last_became_active":"2023-11-15T02:59:34.711551+0000","last_became_peered":"2023-11-15T02:59:34.711551+0000","last_unstale":"2023-11-15T02:59:34.711825+0000","last_undegraded":"2023-11-15T02:59:34.711825+0000","last_fullsized":"2023-11-15T02:59:34.711825+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.229993+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-15T02:59:29.318457+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T02:59:29.318457+0000","last_peered":"2023-11-15T02:59:29.318457+0000","last_clean":"2023-11-15T02:59:29.318457+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T02:59:29.318457+0000","last_undegraded":"2023-11-15T02:59:29.318457+0000","last_fullsized":"2023-11-15T02:59:29.318457+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.712699+0000","last_change":"2023-11-15T02:59:34.712699+0000","last_active":"2023-11-15T02:59:34.712699+0000","last_peered":"2023-11-15T02:59:34.712699+0000","last_clean":"2023-11-15T02:59:34.712699+0000","last_became_active":"2023-11-15T02:59:34.703686+0000","last_became_peered":"2023-11-15T02:59:34.703686+0000","last_unstale":"2023-11-15T02:59:34.712699+0000","last_undegraded":"2023-11-15T02:59:34.712699+0000","last_fullsized":"2023-11-15T02:59:34.712699+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.277673+0000","last_change":"2023-11-15T02:59:34.265955+0000","last_active":"2023-11-15T02:59:34.277673+0000","last_peered":"2023-11-15T02:59:34.277673+0000","last_clean":"2023-11-15T02:59:34.277673+0000","last_became_active":"2023-11-15T02:59:34.265681+0000","last_became_peered":"2023-11-15T02:59:34.265681+0000","last_unstale":"2023-11-15T02:59:34.277673+0000","last_undegraded":"2023-11-15T02:59:34.277673+0000","last_fullsized":"2023-11-15T02:59:34.277673+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":50,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-15T02:59:30.343893+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T02:59:30.343893+0000","last_peered":"2023-11-15T02:59:30.343893+0000","last_clean":"2023-11-15T02:59:30.343893+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T02:59:30.343893+0000","last_undegraded":"2023-11-15T02:59:30.343893+0000","last_fullsized":"2023-11-15T02:59:30.343893+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T02:59:34.221558+0000","last_change":"2023-11-15T02:59:34.221558+0000","last_active":"2023-11-15T02:59:34.221558+0000","last_peered":"2023-11-15T02:59:34.221558+0000","last_clean":"2023-11-15T02:59:34.221558+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:34.221558+0000","last_undegraded":"2023-11-15T02:59:34.221558+0000","last_fullsized":"2023-11-15T02:59:34.221558+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T02:59:33.207077+0000","last_change":"2023-11-15T02:59:33.229084+0000","last_active":"2023-11-15T02:59:33.207077+0000","last_peered":"2023-11-15T02:59:33.207077+0000","last_clean":"2023-11-15T02:59:33.207077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T02:59:33.207077+0000","last_undegraded":"2023-11-15T02:59:33.207077+0000","last_fullsized":"2023-11-15T02:59:33.207077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.253156+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T02:59:34.253156+0000","last_peered":"2023-11-15T02:59:34.253156+0000","last_clean":"2023-11-15T02:59:34.253156+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T02:59:34.253156+0000","last_undegraded":"2023-11-15T02:59:34.253156+0000","last_fullsized":"2023-11-15T02:59:34.253156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.704325+0000","last_change":"2023-11-15T02:59:34.704325+0000","last_active":"2023-11-15T02:59:34.704325+0000","last_peered":"2023-11-15T02:59:34.704325+0000","last_clean":"2023-11-15T02:59:34.704325+0000","last_became_active":"2023-11-15T02:59:34.703365+0000","last_became_peered":"2023-11-15T02:59:34.703365+0000","last_unstale":"2023-11-15T02:59:34.704325+0000","last_undegraded":"2023-11-15T02:59:34.704325+0000","last_fullsized":"2023-11-15T02:59:34.704325+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.691558+0000","last_change":"2023-11-15T02:59:34.265022+0000","last_active":"2023-11-15T02:59:34.691558+0000","last_peered":"2023-11-15T02:59:34.691558+0000","last_clean":"2023-11-15T02:59:34.691558+0000","last_became_active":"2023-11-15T02:59:34.264125+0000","last_became_peered":"2023-11-15T02:59:34.264125+0000","last_unstale":"2023-11-15T02:59:34.691558+0000","last_undegraded":"2023-11-15T02:59:34.691558+0000","last_fullsized":"2023-11-15T02:59:34.691558+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.254819+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T02:59:34.254819+0000","last_peered":"2023-11-15T02:59:34.254819+0000","last_clean":"2023-11-15T02:59:34.254819+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T02:59:34.254819+0000","last_undegraded":"2023-11-15T02:59:34.254819+0000","last_fullsized":"2023-11-15T02:59:34.254819+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.697027+0000","last_change":"2023-11-15T02:59:34.697027+0000","last_active":"2023-11-15T02:59:34.697027+0000","last_peered":"2023-11-15T02:59:34.697027+0000","last_clean":"2023-11-15T02:59:34.697027+0000","last_became_active":"2023-11-15T02:59:34.696199+0000","last_became_peered":"2023-11-15T02:59:34.696199+0000","last_unstale":"2023-11-15T02:59:34.697027+0000","last_undegraded":"2023-11-15T02:59:34.697027+0000","last_fullsized":"2023-11-15T02:59:34.697027+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.283700+0000","last_change":"2023-11-15T02:59:34.279512+0000","last_active":"2023-11-15T02:59:34.283700+0000","last_peered":"2023-11-15T02:59:34.283700+0000","last_clean":"2023-11-15T02:59:34.283700+0000","last_became_active":"2023-11-15T02:59:34.278967+0000","last_became_peered":"2023-11-15T02:59:34.278967+0000","last_unstale":"2023-11-15T02:59:34.283700+0000","last_undegraded":"2023-11-15T02:59:34.283700+0000","last_fullsized":"2023-11-15T02:59:34.283700+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":50,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-15T02:59:30.343807+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T02:59:30.343807+0000","last_peered":"2023-11-15T02:59:30.343807+0000","last_clean":"2023-11-15T02:59:30.343807+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T02:59:30.343807+0000","last_undegraded":"2023-11-15T02:59:30.343807+0000","last_fullsized":"2023-11-15T02:59:30.343807+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.254906+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T02:59:34.254906+0000","last_peered":"2023-11-15T02:59:34.254906+0000","last_clean":"2023-11-15T02:59:34.254906+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T02:59:34.254906+0000","last_undegraded":"2023-11-15T02:59:34.254906+0000","last_fullsized":"2023-11-15T02:59:34.254906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":6},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":748,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":116,"acting":116,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542180,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":704512,"data_stored":312535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.1679999999999999,"15min":1.1859999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.458,"5min":3.952,"15min":3.952},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.046,"5min":1.1799999999999999,"15min":1.202},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4419999999999999,"5min":2.5,"15min":2.5},"last":1.016}]},{"osd":1,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.137,"15min":1.145},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.3740000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1739999999999999,"15min":1.1779999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.415,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.3049999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2150000000000001,"15min":1.2330000000000001},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.458,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2949999999999999,"15min":1.3069999999999999},"min":{"1min":0.97899999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.5149999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":0.92900000000000005}]},{"osd":3,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.3400000000000001,"15min":1.3740000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.381,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.175,"5min":1.2669999999999999,"15min":1.282},"min":{"1min":0.90600000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.351,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.093}]},{"osd":4,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.2689999999999999,"15min":1.3},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.4359999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":1.117},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.448,"15min":1.4990000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.5029999999999999,"5min":4.226,"15min":4.226},"last":0.87}]},{"osd":5,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.329,"15min":1.343},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.556,"5min":2.024,"15min":2.024},"last":1.28},{"interface":"front","average":{"1min":1.234,"5min":1.4399999999999999,"15min":1.474},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.653,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.216}]},{"osd":6,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.5620000000000001,"15min":1.601},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":1.5660000000000001,"5min":3.2789999999999999,"15min":3.2789999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.591,"15min":1.645},"min":{"1min":1.123,"5min":1.123,"15min":1.123},"max":{"1min":1.4399999999999999,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.98099999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542180,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297800,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074040,"statfs":{"total":96636764160,"available":96331816960,"internally_reserved":0,"allocated":720896,"data_stored":319748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.125,"15min":1.133},"min":{"1min":0.69799999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.353,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.0469999999999999,"15min":1.0409999999999999},"min":{"1min":0.72499999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.484,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.93899999999999995}]},{"osd":1,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0920000000000001,"15min":1.0840000000000001},"min":{"1min":0.876,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.415,"5min":1.623,"15min":1.623},"last":1.03},{"interface":"front","average":{"1min":1.173,"5min":1.169,"15min":1.169},"min":{"1min":0.80800000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.609,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.97199999999999998}]},{"osd":2,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.1870000000000001,"15min":1.1779999999999999},"min":{"1min":0.84399999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.496,"5min":2.621,"15min":2.621},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.0960000000000001,"15min":1.0840000000000001},"min":{"1min":0.91700000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.425,"5min":1.5,"15min":1.5},"last":1.2130000000000001}]},{"osd":3,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.167,"15min":1.157},"min":{"1min":1.0029999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.492,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.26,"15min":1.2629999999999999},"min":{"1min":0.99299999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.6850000000000001,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.3740000000000001}]},{"osd":4,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.173,"15min":1.1910000000000001},"min":{"1min":0.81399999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.4099999999999999,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":1.391},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.95599999999999996,"5min":0.625,"15min":0.625},"max":{"1min":1.4850000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.536}]},{"osd":5,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.417,"15min":1.411},"min":{"1min":1.155,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.772,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.3109999999999999,"15min":1.319},"min":{"1min":0.877,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6599999999999999,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":1.169}]},{"osd":7,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2809999999999999,"15min":1.3009999999999999},"min":{"1min":0.82199999999999995,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.395,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.522},{"interface":"front","average":{"1min":1.401,"5min":1.448,"15min":1.456},"min":{"1min":1.046,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8540000000000001,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":1.637}]}]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297800,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074040,"statfs":{"total":96636764160,"available":96331816960,"internally_reserved":0,"allocated":720896,"data_stored":319804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.107,"15min":1.1120000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.516,"5min":1.532,"15min":1.532},"last":1.125},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0069999999999999,"15min":1.0029999999999999},"min":{"1min":0.80700000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":1.28}]},{"osd":1,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2250000000000001,"15min":1.2370000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4770000000000001,"5min":2.617,"15min":2.617},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.115,"5min":1.1559999999999999,"15min":1.163},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.425,"5min":1.5,"15min":1.5},"last":1.2410000000000001}]},{"osd":2,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.1519999999999999,"15min":1.141},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.462,"5min":1.909,"15min":1.909},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1599999999999999,"15min":1.1639999999999999},"min":{"1min":0.88800000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.399,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.0700000000000001}]},{"osd":3,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1519999999999999,"15min":1.1599999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.462,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.345},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.294,"15min":1.3},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.5840000000000001,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":1.169}]},{"osd":5,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2430000000000001,"15min":1.2529999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5780000000000001,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.2290000000000001,"15min":1.2350000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.5049999999999999,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.6639999999999999}]},{"osd":6,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.333,"15min":1.343},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6619999999999999,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":1.456},{"interface":"front","average":{"1min":1.296,"5min":1.3979999999999999,"15min":1.415},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.7450000000000001,"5min":2.173,"15min":2.173},"last":1.4670000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.369,"15min":1.381},"min":{"1min":1.069,"5min":0.87,"15min":0.87},"max":{"1min":1.5820000000000001,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.4350000000000001,"15min":1.4450000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.587,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":1.415}]}]},{"osd":3,"up_from":14,"seq":60129542181,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":704512,"data_stored":312535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.77400000000000002,"15min":0.75900000000000001},"min":{"1min":0.64900000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.629},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.84199999999999997,"15min":0.82899999999999996},"min":{"1min":0.73599999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1779999999999999,"5min":1.26,"15min":1.26},"last":0.70699999999999996}]},{"osd":1,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.90000000000000002,"15min":0.90100000000000002},"min":{"1min":0.621,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.97999999999999998,"15min":0.96999999999999997},"min":{"1min":0.83299999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2749999999999999,"5min":1.375,"15min":1.375},"last":0.55800000000000005}]},{"osd":2,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97699999999999998,"15min":0.96799999999999997},"min":{"1min":0.76700000000000002,"5min":0.63,"15min":0.63},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0649999999999999,"15min":1.052},"min":{"1min":0.85899999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.96699999999999997}]},{"osd":4,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1859999999999999,"15min":1.1799999999999999},"min":{"1min":0.97899999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.456,"5min":1.837,"15min":1.837},"last":1.202},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.22,"15min":1.226},"min":{"1min":1.0249999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.492,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.0189999999999999}]},{"osd":5,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.27,"15min":1.2669999999999999},"min":{"1min":0.996,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.6459999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.05},{"interface":"front","average":{"1min":1.345,"5min":1.2789999999999999,"15min":1.268},"min":{"1min":1.01,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.89900000000000002}]},{"osd":6,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3120000000000001,"15min":1.306},"min":{"1min":1.1060000000000001,"5min":1.048,"15min":1.048},"max":{"1min":1.552,"5min":1.794,"15min":1.794},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.3580000000000001,"15min":1.3520000000000001},"min":{"1min":1.129,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":1.647,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.163}]},{"osd":7,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.3919999999999999,"15min":1.3899999999999999},"min":{"1min":1.083,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.365,"15min":1.357},"min":{"1min":1.1819999999999999,"5min":1.085,"15min":1.085},"max":{"1min":1.665,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.1850000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542181,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297780,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074060,"statfs":{"total":96636764160,"available":96331837440,"internally_reserved":0,"allocated":700416,"data_stored":311266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76700000000000002,"15min":0.76400000000000001},"min":{"1min":0.61699999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88700000000000001,"15min":0.88200000000000001},"min":{"1min":0.68300000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.67400000000000004}]},{"osd":1,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89600000000000002,"15min":0.89200000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3120000000000001,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.96099999999999997,"15min":0.94899999999999995},"min":{"1min":0.83499999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.93600000000000005}]},{"osd":3,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.002,"15min":1},"min":{"1min":0.79200000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.387,"5min":1.472,"15min":1.472},"last":0.629},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1259999999999999,"15min":1.1200000000000001},"min":{"1min":0.93000000000000005,"5min":0.622,"15min":0.622},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.81899999999999995}]},{"osd":4,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.208,"15min":1.21},"min":{"1min":1.0149999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.48,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.111},{"interface":"front","average":{"1min":1.286,"5min":1.238,"15min":1.23},"min":{"1min":1.0760000000000001,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.5429999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.071}]},{"osd":5,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.262,"15min":1.264},"min":{"1min":1.0149999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":1.613,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2989999999999999,"15min":1.2969999999999999},"min":{"1min":1.0700000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.6890000000000001,"5min":1.857,"15min":1.857},"last":1}]},{"osd":6,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.367,"15min":1.373},"min":{"1min":1.056,"5min":1.056,"15min":1.056},"max":{"1min":1.72,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.347,"15min":1.353},"min":{"1min":1.046,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":1.573,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.1599999999999999}]},{"osd":7,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3939999999999999,"15min":1.391},"min":{"1min":1.1719999999999999,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":1.8069999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.3799999999999999,"15min":1.3919999999999999},"min":{"1min":1.097,"5min":1.097,"15min":1.097},"max":{"1min":1.5529999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.96799999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542181,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":704512,"data_stored":312535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.84899999999999998,"15min":0.86899999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1990000000000001,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":4.3630000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.89700000000000002,"15min":0.89600000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1619999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":6.157}]},{"osd":2,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.99399999999999999,"15min":1.0129999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.147,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":4.1699999999999999},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.99199999999999999,"15min":0.98799999999999999},"min":{"1min":0.68000000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.2529999999999999,"5min":1.847,"15min":1.847},"last":6.8369999999999997}]},{"osd":3,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.96499999999999997,"15min":0.96199999999999997},"min":{"1min":0.76600000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.401,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":3.5070000000000001},{"interface":"front","average":{"1min":1.052,"5min":1.157,"15min":1.175},"min":{"1min":0.68899999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2210000000000001,"5min":3.54,"15min":3.54},"last":5.2149999999999999}]},{"osd":4,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2490000000000001,"15min":1.2470000000000001},"min":{"1min":1.0720000000000001,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.6319999999999999,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":6.2000000000000002},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.175,"15min":1.175},"min":{"1min":1.006,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.4850000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":3.577}]},{"osd":5,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2170000000000001,"15min":1.218},"min":{"1min":0.98899999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.393,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":3.548},{"interface":"front","average":{"1min":1.24,"5min":1.3420000000000001,"15min":1.359},"min":{"1min":1.083,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.375,"5min":3.6760000000000002,"15min":3.6760000000000002},"last":4.4080000000000004}]},{"osd":6,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.411,"15min":1.429},"min":{"1min":1.113,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":1.4730000000000001,"5min":3.71,"15min":3.71},"last":4.4489999999999998},{"interface":"front","average":{"1min":1.282,"5min":1.3340000000000001,"15min":1.343},"min":{"1min":1.05,"5min":1.024,"15min":1.024},"max":{"1min":1.4970000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":6.9690000000000003}]},{"osd":7,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.3640000000000001,"15min":1.379},"min":{"1min":1.018,"5min":1.018,"15min":1.018},"max":{"1min":1.554,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":18.253},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2649999999999999,"15min":1.2609999999999999},"min":{"1min":1.1040000000000001,"5min":0.876,"15min":0.876},"max":{"1min":1.5449999999999999,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":11.006}]}]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297772,"kb_used_data":676,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074068,"statfs":{"total":96636764160,"available":96331845632,"internally_reserved":0,"allocated":692224,"data_stored":305856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.05,"15min":1.046},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4359999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.216,"15min":1.2370000000000001},"min":{"1min":0.69699999999999995,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.4630000000000001,"5min":3.46,"15min":3.46},"last":0.83499999999999996}]},{"osd":1,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.2310000000000001,"15min":1.2589999999999999},"min":{"1min":0.82699999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.47,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.1739999999999999,"15min":1.1879999999999999},"min":{"1min":0.80300000000000005,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.4570000000000001,"5min":3.516,"15min":3.516},"last":1.0329999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.2070000000000001,"15min":1.2130000000000001},"min":{"1min":0.91600000000000004,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.3640000000000001,"5min":5.4180000000000001,"15min":5.4180000000000001},"last":1.163},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3069999999999999,"15min":1.321},"min":{"1min":0.94799999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.5089999999999999,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":0.69699999999999995}]},{"osd":3,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.29,"15min":1.3080000000000001},"min":{"1min":1.022,"5min":0.879,"15min":0.879},"max":{"1min":1.4850000000000001,"5min":2.141,"15min":2.141},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.448,"15min":1.464},"min":{"1min":1.1839999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":1.583,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":0.81599999999999995}]},{"osd":4,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3680000000000001,"15min":1.3879999999999999},"min":{"1min":0.84199999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.5309999999999999,"5min":3.706,"15min":3.706},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.597,"15min":1.6379999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6479999999999999,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":1.212}]},{"osd":6,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.327,"15min":1.3420000000000001},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.6439999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":1.25},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.546,"15min":1.5860000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.613,"5min":3.641,"15min":3.641},"last":1.2330000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4259999999999999,"15min":1.4370000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.762,"5min":3.516,"15min":3.516},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.46,"15min":1.4650000000000001},"min":{"1min":1.139,"5min":1.022,"15min":1.022},"max":{"1min":1.722,"5min":3.754,"15min":3.754},"last":1.343}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297760,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074080,"statfs":{"total":96636764160,"available":96331857920,"internally_reserved":0,"allocated":679936,"data_stored":299635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.84999999999999998,"15min":0.84599999999999997},"min":{"1min":0.63600000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2150000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.72199999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.155,"5min":1.419,"15min":1.419},"last":0.67800000000000005}]},{"osd":2,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.85799999999999998,"15min":0.84799999999999998},"min":{"1min":0.495,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.90700000000000003,"15min":0.91000000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.161,"5min":1.341,"15min":1.341},"last":0.76500000000000001}]},{"osd":3,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.0620000000000001,"15min":1.054},"min":{"1min":0.54100000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.25},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.98899999999999999,"15min":0.97699999999999998},"min":{"1min":0.67800000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":1.23}]},{"osd":4,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.175,"15min":1.165},"min":{"1min":0.75700000000000001,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.49,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.248,"15min":1.2589999999999999},"min":{"1min":0.92000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.5609999999999999,"5min":2.206,"15min":2.206},"last":1.1799999999999999}]},{"osd":5,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.26,"15min":1.2649999999999999},"min":{"1min":1.0089999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.4299999999999999,"5min":1.899,"15min":1.899},"last":1.048},{"interface":"front","average":{"1min":1.236,"5min":1.2829999999999999,"15min":1.2909999999999999},"min":{"1min":0.92800000000000005,"5min":0.871,"15min":0.871},"max":{"1min":1.5029999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.3029999999999999}]},{"osd":6,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.323,"15min":1.325},"min":{"1min":0.98299999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.5229999999999999,"5min":1.825,"15min":1.825},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.4099999999999999,"15min":1.4119999999999999},"min":{"1min":1.1870000000000001,"5min":0.87,"15min":0.87},"max":{"1min":1.5529999999999999,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.3740000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3200000000000001,"15min":1.3169999999999999},"min":{"1min":0.93999999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.6839999999999999,"5min":1.772,"15min":1.772},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3540000000000001,"15min":1.3600000000000001},"min":{"1min":0.95499999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.6120000000000001,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.1990000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T02:59:39.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3e is not active+clean 2023-11-15T02:59:39.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:34.221558+0000', 'last_change': '2023-11-15T02:59:34.221558+0000', 'last_active': '2023-11-15T02:59:34.221558+0000', 'last_peered': '2023-11-15T02:59:34.221558+0000', 'last_clean': '2023-11-15T02:59:34.221558+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:34.221558+0000', 'last_undegraded': '2023-11-15T02:59:34.221558+0000', 'last_fullsized': '2023-11-15T02:59:34.221558+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.768 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3f is not active+clean 2023-11-15T02:59:39.768 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:34.221558+0000', 'last_change': '2023-11-15T02:59:34.221558+0000', 'last_active': '2023-11-15T02:59:34.221558+0000', 'last_peered': '2023-11-15T02:59:34.221558+0000', 'last_clean': '2023-11-15T02:59:34.221558+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:34.221558+0000', 'last_undegraded': '2023-11-15T02:59:34.221558+0000', 'last_fullsized': '2023-11-15T02:59:34.221558+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.768 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3c is not active+clean 2023-11-15T02:59:39.769 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:59:33.207077+0000', 'last_change': '2023-11-15T02:59:33.231018+0000', 'last_active': '2023-11-15T02:59:33.207077+0000', 'last_peered': '2023-11-15T02:59:33.207077+0000', 'last_clean': '2023-11-15T02:59:33.207077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:33.207077+0000', 'last_undegraded': '2023-11-15T02:59:33.207077+0000', 'last_fullsized': '2023-11-15T02:59:33.207077+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-15T02:59:39.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.38 is not active+clean 2023-11-15T02:59:39.770 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:34.221558+0000', 'last_change': '2023-11-15T02:59:34.221558+0000', 'last_active': '2023-11-15T02:59:34.221558+0000', 'last_peered': '2023-11-15T02:59:34.221558+0000', 'last_clean': '2023-11-15T02:59:34.221558+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:34.221558+0000', 'last_undegraded': '2023-11-15T02:59:34.221558+0000', 'last_fullsized': '2023-11-15T02:59:34.221558+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.770 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.34 is not active+clean 2023-11-15T02:59:39.770 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:34.221558+0000', 'last_change': '2023-11-15T02:59:34.221558+0000', 'last_active': '2023-11-15T02:59:34.221558+0000', 'last_peered': '2023-11-15T02:59:34.221558+0000', 'last_clean': '2023-11-15T02:59:34.221558+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:34.221558+0000', 'last_undegraded': '2023-11-15T02:59:34.221558+0000', 'last_fullsized': '2023-11-15T02:59:34.221558+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.771 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.36 is not active+clean 2023-11-15T02:59:39.771 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:33.207077+0000', 'last_change': '2023-11-15T02:59:33.207077+0000', 'last_active': '2023-11-15T02:59:33.207077+0000', 'last_peered': '2023-11-15T02:59:33.207077+0000', 'last_clean': '2023-11-15T02:59:33.207077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:33.207077+0000', 'last_undegraded': '2023-11-15T02:59:33.207077+0000', 'last_fullsized': '2023-11-15T02:59:33.207077+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.772 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.35 is not active+clean 2023-11-15T02:59:39.772 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:33.207077+0000', 'last_change': '2023-11-15T02:59:33.207077+0000', 'last_active': '2023-11-15T02:59:33.207077+0000', 'last_peered': '2023-11-15T02:59:33.207077+0000', 'last_clean': '2023-11-15T02:59:33.207077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:33.207077+0000', 'last_undegraded': '2023-11-15T02:59:33.207077+0000', 'last_fullsized': '2023-11-15T02:59:33.207077+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.772 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.34 is not active+clean 2023-11-15T02:59:39.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:59:33.207077+0000', 'last_change': '2023-11-15T02:59:33.229134+0000', 'last_active': '2023-11-15T02:59:33.207077+0000', 'last_peered': '2023-11-15T02:59:33.207077+0000', 'last_clean': '2023-11-15T02:59:33.207077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:33.207077+0000', 'last_undegraded': '2023-11-15T02:59:33.207077+0000', 'last_fullsized': '2023-11-15T02:59:33.207077+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-15T02:59:39.773 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.33 is not active+clean 2023-11-15T02:59:39.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:34.221558+0000', 'last_change': '2023-11-15T02:59:34.221558+0000', 'last_active': '2023-11-15T02:59:34.221558+0000', 'last_peered': '2023-11-15T02:59:34.221558+0000', 'last_clean': '2023-11-15T02:59:34.221558+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:34.221558+0000', 'last_undegraded': '2023-11-15T02:59:34.221558+0000', 'last_fullsized': '2023-11-15T02:59:34.221558+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.774 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2f is not active+clean 2023-11-15T02:59:39.774 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:34.221558+0000', 'last_change': '2023-11-15T02:59:34.221558+0000', 'last_active': '2023-11-15T02:59:34.221558+0000', 'last_peered': '2023-11-15T02:59:34.221558+0000', 'last_clean': '2023-11-15T02:59:34.221558+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:34.221558+0000', 'last_undegraded': '2023-11-15T02:59:34.221558+0000', 'last_fullsized': '2023-11-15T02:59:34.221558+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.775 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2b is not active+clean 2023-11-15T02:59:39.775 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:34.221558+0000', 'last_change': '2023-11-15T02:59:34.221558+0000', 'last_active': '2023-11-15T02:59:34.221558+0000', 'last_peered': '2023-11-15T02:59:34.221558+0000', 'last_clean': '2023-11-15T02:59:34.221558+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:34.221558+0000', 'last_undegraded': '2023-11-15T02:59:34.221558+0000', 'last_fullsized': '2023-11-15T02:59:34.221558+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.776 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.24 is not active+clean 2023-11-15T02:59:39.776 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:34.221558+0000', 'last_change': '2023-11-15T02:59:34.221558+0000', 'last_active': '2023-11-15T02:59:34.221558+0000', 'last_peered': '2023-11-15T02:59:34.221558+0000', 'last_clean': '2023-11-15T02:59:34.221558+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:34.221558+0000', 'last_undegraded': '2023-11-15T02:59:34.221558+0000', 'last_fullsized': '2023-11-15T02:59:34.221558+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.776 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.26 is not active+clean 2023-11-15T02:59:39.777 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:59:33.207077+0000', 'last_change': '2023-11-15T02:59:33.229642+0000', 'last_active': '2023-11-15T02:59:33.207077+0000', 'last_peered': '2023-11-15T02:59:33.207077+0000', 'last_clean': '2023-11-15T02:59:33.207077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:33.207077+0000', 'last_undegraded': '2023-11-15T02:59:33.207077+0000', 'last_fullsized': '2023-11-15T02:59:33.207077+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-15T02:59:39.777 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.26 is not active+clean 2023-11-15T02:59:39.777 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:34.221558+0000', 'last_change': '2023-11-15T02:59:34.221558+0000', 'last_active': '2023-11-15T02:59:34.221558+0000', 'last_peered': '2023-11-15T02:59:34.221558+0000', 'last_clean': '2023-11-15T02:59:34.221558+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:34.221558+0000', 'last_undegraded': '2023-11-15T02:59:34.221558+0000', 'last_fullsized': '2023-11-15T02:59:34.221558+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.778 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.27 is not active+clean 2023-11-15T02:59:39.778 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:34.221558+0000', 'last_change': '2023-11-15T02:59:34.221558+0000', 'last_active': '2023-11-15T02:59:34.221558+0000', 'last_peered': '2023-11-15T02:59:34.221558+0000', 'last_clean': '2023-11-15T02:59:34.221558+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:34.221558+0000', 'last_undegraded': '2023-11-15T02:59:34.221558+0000', 'last_fullsized': '2023-11-15T02:59:34.221558+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.22 is not active+clean 2023-11-15T02:59:39.779 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:34.221558+0000', 'last_change': '2023-11-15T02:59:34.221558+0000', 'last_active': '2023-11-15T02:59:34.221558+0000', 'last_peered': '2023-11-15T02:59:34.221558+0000', 'last_clean': '2023-11-15T02:59:34.221558+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:34.221558+0000', 'last_undegraded': '2023-11-15T02:59:34.221558+0000', 'last_fullsized': '2023-11-15T02:59:34.221558+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2023-11-15T02:59:39.780 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:33.207077+0000', 'last_change': '2023-11-15T02:59:33.207077+0000', 'last_active': '2023-11-15T02:59:33.207077+0000', 'last_peered': '2023-11-15T02:59:33.207077+0000', 'last_clean': '2023-11-15T02:59:33.207077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:33.207077+0000', 'last_undegraded': '2023-11-15T02:59:33.207077+0000', 'last_fullsized': '2023-11-15T02:59:33.207077+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.780 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2023-11-15T02:59:39.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:34.221558+0000', 'last_change': '2023-11-15T02:59:34.221558+0000', 'last_active': '2023-11-15T02:59:34.221558+0000', 'last_peered': '2023-11-15T02:59:34.221558+0000', 'last_clean': '2023-11-15T02:59:34.221558+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:34.221558+0000', 'last_undegraded': '2023-11-15T02:59:34.221558+0000', 'last_fullsized': '2023-11-15T02:59:34.221558+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.781 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2023-11-15T02:59:39.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:34.221558+0000', 'last_change': '2023-11-15T02:59:34.221558+0000', 'last_active': '2023-11-15T02:59:34.221558+0000', 'last_peered': '2023-11-15T02:59:34.221558+0000', 'last_clean': '2023-11-15T02:59:34.221558+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:34.221558+0000', 'last_undegraded': '2023-11-15T02:59:34.221558+0000', 'last_fullsized': '2023-11-15T02:59:34.221558+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.782 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2023-11-15T02:59:39.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:33.207077+0000', 'last_change': '2023-11-15T02:59:33.207077+0000', 'last_active': '2023-11-15T02:59:33.207077+0000', 'last_peered': '2023-11-15T02:59:33.207077+0000', 'last_clean': '2023-11-15T02:59:33.207077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:33.207077+0000', 'last_undegraded': '2023-11-15T02:59:33.207077+0000', 'last_fullsized': '2023-11-15T02:59:33.207077+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2023-11-15T02:59:39.783 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:34.221558+0000', 'last_change': '2023-11-15T02:59:34.221558+0000', 'last_active': '2023-11-15T02:59:34.221558+0000', 'last_peered': '2023-11-15T02:59:34.221558+0000', 'last_clean': '2023-11-15T02:59:34.221558+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:34.221558+0000', 'last_undegraded': '2023-11-15T02:59:34.221558+0000', 'last_fullsized': '2023-11-15T02:59:34.221558+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2023-11-15T02:59:39.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:59:33.207077+0000', 'last_change': '2023-11-15T02:59:33.230568+0000', 'last_active': '2023-11-15T02:59:33.207077+0000', 'last_peered': '2023-11-15T02:59:33.207077+0000', 'last_clean': '2023-11-15T02:59:33.207077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:33.207077+0000', 'last_undegraded': '2023-11-15T02:59:33.207077+0000', 'last_fullsized': '2023-11-15T02:59:33.207077+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-15T02:59:39.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2023-11-15T02:59:39.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:34.221558+0000', 'last_change': '2023-11-15T02:59:34.221558+0000', 'last_active': '2023-11-15T02:59:34.221558+0000', 'last_peered': '2023-11-15T02:59:34.221558+0000', 'last_clean': '2023-11-15T02:59:34.221558+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:34.221558+0000', 'last_undegraded': '2023-11-15T02:59:34.221558+0000', 'last_fullsized': '2023-11-15T02:59:34.221558+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2023-11-15T02:59:39.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:59:33.207077+0000', 'last_change': '2023-11-15T02:59:33.228982+0000', 'last_active': '2023-11-15T02:59:33.207077+0000', 'last_peered': '2023-11-15T02:59:33.207077+0000', 'last_clean': '2023-11-15T02:59:33.207077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:33.207077+0000', 'last_undegraded': '2023-11-15T02:59:33.207077+0000', 'last_fullsized': '2023-11-15T02:59:33.207077+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-15T02:59:39.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2023-11-15T02:59:39.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:59:33.207077+0000', 'last_change': '2023-11-15T02:59:33.228971+0000', 'last_active': '2023-11-15T02:59:33.207077+0000', 'last_peered': '2023-11-15T02:59:33.207077+0000', 'last_clean': '2023-11-15T02:59:33.207077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:33.207077+0000', 'last_undegraded': '2023-11-15T02:59:33.207077+0000', 'last_fullsized': '2023-11-15T02:59:33.207077+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-15T02:59:39.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2023-11-15T02:59:39.787 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:34.221558+0000', 'last_change': '2023-11-15T02:59:34.221558+0000', 'last_active': '2023-11-15T02:59:34.221558+0000', 'last_peered': '2023-11-15T02:59:34.221558+0000', 'last_clean': '2023-11-15T02:59:34.221558+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:34.221558+0000', 'last_undegraded': '2023-11-15T02:59:34.221558+0000', 'last_fullsized': '2023-11-15T02:59:34.221558+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2023-11-15T02:59:39.788 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:33.207077+0000', 'last_change': '2023-11-15T02:59:33.207077+0000', 'last_active': '2023-11-15T02:59:33.207077+0000', 'last_peered': '2023-11-15T02:59:33.207077+0000', 'last_clean': '2023-11-15T02:59:33.207077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:33.207077+0000', 'last_undegraded': '2023-11-15T02:59:33.207077+0000', 'last_fullsized': '2023-11-15T02:59:33.207077+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.788 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2023-11-15T02:59:39.789 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:34.221558+0000', 'last_change': '2023-11-15T02:59:34.221558+0000', 'last_active': '2023-11-15T02:59:34.221558+0000', 'last_peered': '2023-11-15T02:59:34.221558+0000', 'last_clean': '2023-11-15T02:59:34.221558+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:34.221558+0000', 'last_undegraded': '2023-11-15T02:59:34.221558+0000', 'last_fullsized': '2023-11-15T02:59:34.221558+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.789 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2023-11-15T02:59:39.789 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:33.207077+0000', 'last_change': '2023-11-15T02:59:33.207077+0000', 'last_active': '2023-11-15T02:59:33.207077+0000', 'last_peered': '2023-11-15T02:59:33.207077+0000', 'last_clean': '2023-11-15T02:59:33.207077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:33.207077+0000', 'last_undegraded': '2023-11-15T02:59:33.207077+0000', 'last_fullsized': '2023-11-15T02:59:33.207077+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.790 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2023-11-15T02:59:39.790 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:59:33.207077+0000', 'last_change': '2023-11-15T02:59:33.229993+0000', 'last_active': '2023-11-15T02:59:33.207077+0000', 'last_peered': '2023-11-15T02:59:33.207077+0000', 'last_clean': '2023-11-15T02:59:33.207077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:33.207077+0000', 'last_undegraded': '2023-11-15T02:59:33.207077+0000', 'last_fullsized': '2023-11-15T02:59:33.207077+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-15T02:59:39.791 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2023-11-15T02:59:39.791 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T02:59:34.221558+0000', 'last_change': '2023-11-15T02:59:34.221558+0000', 'last_active': '2023-11-15T02:59:34.221558+0000', 'last_peered': '2023-11-15T02:59:34.221558+0000', 'last_clean': '2023-11-15T02:59:34.221558+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:34.221558+0000', 'last_undegraded': '2023-11-15T02:59:34.221558+0000', 'last_fullsized': '2023-11-15T02:59:34.221558+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:34.221558+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T02:59:39.791 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2023-11-15T02:59:39.792 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T02:59:33.207077+0000', 'last_change': '2023-11-15T02:59:33.229084+0000', 'last_active': '2023-11-15T02:59:33.207077+0000', 'last_peered': '2023-11-15T02:59:33.207077+0000', 'last_clean': '2023-11-15T02:59:33.207077+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T02:59:33.207077+0000', 'last_undegraded': '2023-11-15T02:59:33.207077+0000', 'last_fullsized': '2023-11-15T02:59:33.207077+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'last_clean_scrub_stamp': '2023-11-15T02:59:33.207077+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-15T02:59:39.793 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-15T02:59:39.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.945+0000 7f926b59e700 1 Processor -- start 2023-11-15T02:59:39.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.945+0000 7f926b59e700 1 -- start start 2023-11-15T02:59:39.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.949+0000 7f926b59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f926c153740 0x7f926c154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:39.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.949+0000 7f926b59e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926c153c90 con 0x7f926c0c2010 2023-11-15T02:59:39.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.949+0000 7f926b59e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926c0c1810 con 0x7f926c0c23f0 2023-11-15T02:59:39.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.949+0000 7f926b59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926c0c2830 con 0x7f926c153740 2023-11-15T02:59:39.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.949+0000 7f926ad9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f926c153740 0x7f926c154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:39.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.949+0000 7f926ad9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f926c153740 0x7f926c154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39350/0 (socket says 172.21.15.121:39350) 2023-11-15T02:59:39.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.949+0000 7f926ad9d700 1 -- 172.21.15.121:0/942361661 learned_addr learned my addr 172.21.15.121:0/942361661 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:39.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.949+0000 7f926a59c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f926c0c2010 0x7f926c0c0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51822/0 (socket says 172.21.15.121:51822) 2023-11-15T02:59:39.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.949+0000 7f926959a700 1 -- 172.21.15.121:0/942361661 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025566613 0 0) 0x7f926c0c1810 con 0x7f926c0c23f0 2023-11-15T02:59:39.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.949+0000 7f926ad9d700 1 -- 172.21.15.121:0/942361661 >> v1:172.21.15.190:6789/0 conn(0x7f926c0c23f0 legacy=0x7f926c153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:39.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.949+0000 7f926ad9d700 1 -- 172.21.15.121:0/942361661 >> v1:172.21.15.121:6789/0 conn(0x7f926c0c2010 legacy=0x7f926c0c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:39.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.949+0000 7f926ad9d700 1 -- 172.21.15.121:0/942361661 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c0c1af0 con 0x7f926c153740 2023-11-15T02:59:39.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.949+0000 7f926ad9d700 1 --2- 172.21.15.121:0/942361661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f926c153740 0x7f926c154750 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f926400d1a0 tx=0x7f926400ba50).ready entity=mon.0 client_cookie=aa9ac02e81161521 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:39.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.949+0000 7f926959a700 1 -- 172.21.15.121:0/942361661 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264010070 con 0x7f926c153740 2023-11-15T02:59:39.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.949+0000 7f926959a700 1 -- 172.21.15.121:0/942361661 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9264015470 con 0x7f926c153740 2023-11-15T02:59:39.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.949+0000 7f926959a700 1 -- 172.21.15.121:0/942361661 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92640157f0 con 0x7f926c153740 2023-11-15T02:59:39.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.953+0000 7f926b59e700 1 -- 172.21.15.121:0/942361661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f926c153740 msgr2=0x7f926c154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:39.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.953+0000 7f926b59e700 1 --2- 172.21.15.121:0/942361661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f926c153740 0x7f926c154750 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f926400d1a0 tx=0x7f926400ba50).stop 2023-11-15T02:59:39.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.953+0000 7f926b59e700 1 -- 172.21.15.121:0/942361661 shutdown_connections 2023-11-15T02:59:39.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.953+0000 7f926b59e700 1 --2- 172.21.15.121:0/942361661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f926c153740 0x7f926c154750 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.953+0000 7f926b59e700 1 -- 172.21.15.121:0/942361661 >> 172.21.15.121:0/942361661 conn(0x7f926c062190 msgr2=0x7f926c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:39.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.953+0000 7f926b59e700 1 -- 172.21.15.121:0/942361661 shutdown_connections 2023-11-15T02:59:39.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.953+0000 7f926b59e700 1 -- 172.21.15.121:0/942361661 wait complete. 2023-11-15T02:59:39.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.953+0000 7f926b59e700 1 Processor -- start 2023-11-15T02:59:39.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.953+0000 7f926b59e700 1 -- start start 2023-11-15T02:59:39.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f926b59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f926c0c2010 0x7f926c150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:39.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f926b59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f926c0c23f0 0x7f926c1678b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:39.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f926b59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f926c153740 0x7f926c0c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:39.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f926b59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0c5c80 con 0x7f926c0c2010 2023-11-15T02:59:39.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f926b59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c150f50 con 0x7f926c153740 2023-11-15T02:59:39.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f926b59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c14bb40 con 0x7f926c0c23f0 2023-11-15T02:59:39.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f926a59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f926c0c2010 0x7f926c150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:39.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f926a59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f926c0c2010 0x7f926c150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39366/0 (socket says 172.21.15.121:39366) 2023-11-15T02:59:39.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f926a59c700 1 -- 172.21.15.121:0/2089374965 learned_addr learned my addr 172.21.15.121:0/2089374965 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:39.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f926ad9d700 1 --2- 172.21.15.121:0/2089374965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f926c153740 0x7f926c0c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:39.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f9269d9b700 1 --2- 172.21.15.121:0/2089374965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f926c0c23f0 0x7f926c1678b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:39.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f926a59c700 1 -- 172.21.15.121:0/2089374965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f926c0c23f0 msgr2=0x7f926c1678b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:39.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f926a59c700 1 --2- 172.21.15.121:0/2089374965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f926c0c23f0 0x7f926c1678b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f926a59c700 1 -- 172.21.15.121:0/2089374965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f926c153740 msgr2=0x7f926c0c51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:39.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f926a59c700 1 --2- 172.21.15.121:0/2089374965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f926c153740 0x7f926c0c51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:39.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f926a59c700 1 -- 172.21.15.121:0/2089374965 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926400bcf0 con 0x7f926c0c2010 2023-11-15T02:59:39.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f926a59c700 1 --2- 172.21.15.121:0/2089374965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f926c0c2010 0x7f926c150a60 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f925c00c4c0 tx=0x7f925c007d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:39.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f925a7fc700 1 -- 172.21.15.121:0/2089374965 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c00ce10 con 0x7f926c0c2010 2023-11-15T02:59:39.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f925a7fc700 1 -- 172.21.15.121:0/2089374965 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925c0072e0 con 0x7f926c0c2010 2023-11-15T02:59:39.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f926b59e700 1 -- 172.21.15.121:0/2089374965 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926c14af70 con 0x7f926c0c2010 2023-11-15T02:59:39.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f925a7fc700 1 -- 172.21.15.121:0/2089374965 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c0056e0 con 0x7f926c0c2010 2023-11-15T02:59:39.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.957+0000 7f926b59e700 1 -- 172.21.15.121:0/2089374965 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926c14b7d0 con 0x7f926c0c2010 2023-11-15T02:59:39.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.961+0000 7f926b59e700 1 -- 172.21.15.121:0/2089374965 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926c06bcf0 con 0x7f926c0c2010 2023-11-15T02:59:39.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.961+0000 7f925a7fc700 1 -- 172.21.15.121:0/2089374965 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f925c005a90 con 0x7f926c0c2010 2023-11-15T02:59:39.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.961+0000 7f925a7fc700 1 --2- 172.21.15.121:0/2089374965 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f924c05a540 0x7f924c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:39.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.961+0000 7f925a7fc700 1 -- 172.21.15.121:0/2089374965 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f925c073da0 con 0x7f926c0c2010 2023-11-15T02:59:39.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.961+0000 7f9269d9b700 1 --2- 172.21.15.121:0/2089374965 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f924c05a540 0x7f924c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:39.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.965+0000 7f9269d9b700 1 --2- 172.21.15.121:0/2089374965 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f924c05a540 0x7f924c05ca00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f926c14eec0 tx=0x7f9260009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:39.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:39.969+0000 7f925a7fc700 1 -- 172.21.15.121:0/2089374965 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f925c0474c0 con 0x7f926c0c2010 2023-11-15T02:59:40.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.189+0000 7f926b59e700 1 -- 172.21.15.121:0/2089374965 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f926c0c64f0 con 0x7f926c0c2010 2023-11-15T02:59:40.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.193+0000 7f925a7fc700 1 -- 172.21.15.121:0/2089374965 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1533 (secure 0 0 0) 0x7f925c018610 con 0x7f926c0c2010 2023-11-15T02:59:40.199 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:40.199 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":187,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":61,"num_osds":8,"num_up_osds":8,"osd_up_since":1700017004,"num_in_osds":8,"osd_in_since":1700016996,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105},{"state_name":"unknown","count":24},{"state_name":"creating+peering","count":8}],"num_pgs":137,"num_pools":4,"num_objects":14,"data_bytes":748,"bytes_used":2439438336,"bytes_avail":770654674944,"bytes_total":773094113280,"unknown_pgs_ratio":0.17518247663974762,"inactive_pgs_ratio":0.058394160121679306},"fsmap":{"epoch":45,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":5910}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2023-11-15T02:59:13.963252+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-15T02:59:40.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.197+0000 7f926b59e700 1 -- 172.21.15.121:0/2089374965 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f924c05a540 msgr2=0x7f924c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:40.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.197+0000 7f926b59e700 1 --2- 172.21.15.121:0/2089374965 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f924c05a540 0x7f924c05ca00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f926c14eec0 tx=0x7f9260009040).stop 2023-11-15T02:59:40.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.197+0000 7f926b59e700 1 -- 172.21.15.121:0/2089374965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f926c0c2010 msgr2=0x7f926c150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.197+0000 7f926b59e700 1 --2- 172.21.15.121:0/2089374965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f926c0c2010 0x7f926c150a60 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f925c00c4c0 tx=0x7f925c007d80).stop 2023-11-15T02:59:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.197+0000 7f926b59e700 1 -- 172.21.15.121:0/2089374965 shutdown_connections 2023-11-15T02:59:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.197+0000 7f926b59e700 1 --2- 172.21.15.121:0/2089374965 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f924c05a540 0x7f924c05ca00 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.197+0000 7f926b59e700 1 --2- 172.21.15.121:0/2089374965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f926c153740 0x7f926c0c51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.197+0000 7f926b59e700 1 --2- 172.21.15.121:0/2089374965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f926c0c23f0 0x7f926c1678b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:40.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.197+0000 7f926b59e700 1 --2- 172.21.15.121:0/2089374965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f926c0c2010 0x7f926c150a60 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:40.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.197+0000 7f926b59e700 1 -- 172.21.15.121:0/2089374965 >> 172.21.15.121:0/2089374965 conn(0x7f926c062190 msgr2=0x7f926c14f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:40.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.197+0000 7f926b59e700 1 -- 172.21.15.121:0/2089374965 shutdown_connections 2023-11-15T02:59:40.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.197+0000 7f926b59e700 1 -- 172.21.15.121:0/2089374965 wait complete. 2023-11-15T02:59:40.207 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T02:59:40.197+0000 7fd03f594700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T02:59:40.225 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-15T02:59:40.226 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T02:59:40.382 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T02:59:40.389 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T02:59:40.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.389+0000 7fcfdd8db700 1 Processor -- start 2023-11-15T02:59:40.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.389+0000 7fcfdd8db700 1 -- start start 2023-11-15T02:59:40.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfdd8db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcfd81562f0 0x7fcfd8155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:40.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfdd8db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcfd80a57c0 0x7fcfd80a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:40.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfdd8db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd80c4c30 con 0x7fcfd8154ca0 2023-11-15T02:59:40.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfdd8db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd80a8280 con 0x7fcfd81562f0 2023-11-15T02:59:40.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfdd8db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd80a6d40 con 0x7fcfd80a57c0 2023-11-15T02:59:40.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfd6ffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fcfd8154ca0 0x7fcfd80a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51836/0 (socket says 172.21.15.121:51836) 2023-11-15T02:59:40.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfd6ffd700 1 -- 172.21.15.121:0/3837077705 learned_addr learned my addr 172.21.15.121:0/3837077705 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:40.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfd67fc700 1 --2- 172.21.15.121:0/3837077705 >> v2:172.21.15.121:3300/0 conn(0x7fcfd81562f0 0x7fcfd8155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:40.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfd77fe700 1 --2- 172.21.15.121:0/3837077705 >> v2:172.21.15.190:3300/0 conn(0x7fcfd80a57c0 0x7fcfd80a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:40.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfd5ffb700 1 -- 172.21.15.121:0/3837077705 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1139748179 0 0) 0x7fcfd80c4c30 con 0x7fcfd8154ca0 2023-11-15T02:59:40.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfd5ffb700 1 -- 172.21.15.121:0/3837077705 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfc00033f0 con 0x7fcfd8154ca0 2023-11-15T02:59:40.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfd67fc700 1 -- 172.21.15.121:0/3837077705 >> v2:172.21.15.190:3300/0 conn(0x7fcfd80a57c0 msgr2=0x7fcfd80a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:40.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfd67fc700 1 --2- 172.21.15.121:0/3837077705 >> v2:172.21.15.190:3300/0 conn(0x7fcfd80a57c0 0x7fcfd80a5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:40.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfd67fc700 1 -- 172.21.15.121:0/3837077705 >> v1:172.21.15.121:6789/0 conn(0x7fcfd8154ca0 legacy=0x7fcfd80a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:40.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfd67fc700 1 -- 172.21.15.121:0/3837077705 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd80a6100 con 0x7fcfd81562f0 2023-11-15T02:59:40.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfd67fc700 1 --2- 172.21.15.121:0/3837077705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfd81562f0 0x7fcfd8155080 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fcfd80a8820 tx=0x7fcfcc009060).ready entity=mon.0 client_cookie=cdc318464fd47b6e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:40.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfd5ffb700 1 -- 172.21.15.121:0/3837077705 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00d4a0 con 0x7fcfd81562f0 2023-11-15T02:59:40.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfd5ffb700 1 -- 172.21.15.121:0/3837077705 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc00d600 con 0x7fcfd81562f0 2023-11-15T02:59:40.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfd5ffb700 1 -- 172.21.15.121:0/3837077705 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00d980 con 0x7fcfd81562f0 2023-11-15T02:59:40.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfdd8db700 1 -- 172.21.15.121:0/3837077705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfd81562f0 msgr2=0x7fcfd8155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:40.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfdd8db700 1 --2- 172.21.15.121:0/3837077705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfd81562f0 0x7fcfd8155080 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fcfd80a8820 tx=0x7fcfcc009060).stop 2023-11-15T02:59:40.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfdd8db700 1 -- 172.21.15.121:0/3837077705 shutdown_connections 2023-11-15T02:59:40.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfdd8db700 1 --2- 172.21.15.121:0/3837077705 >> v2:172.21.15.190:3300/0 conn(0x7fcfd80a57c0 0x7fcfd80a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:40.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfdd8db700 1 --2- 172.21.15.121:0/3837077705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfd81562f0 0x7fcfd8155080 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:40.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfdd8db700 1 -- 172.21.15.121:0/3837077705 >> 172.21.15.121:0/3837077705 conn(0x7fcfd8062190 msgr2=0x7fcfd80625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:40.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfdd8db700 1 -- 172.21.15.121:0/3837077705 shutdown_connections 2023-11-15T02:59:40.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.393+0000 7fcfdd8db700 1 -- 172.21.15.121:0/3837077705 wait complete. 2023-11-15T02:59:40.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.397+0000 7fcfdd8db700 1 Processor -- start 2023-11-15T02:59:40.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.397+0000 7fcfdd8db700 1 -- start start 2023-11-15T02:59:40.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.397+0000 7fcfdd8db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfd80a57c0 0x7fcfd814ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:40.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.397+0000 7fcfdd8db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfd8154ca0 0x7fcfd80c0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:40.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.397+0000 7fcfdd8db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfd81562f0 0x7fcfd80bff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:40.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.397+0000 7fcfdd8db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd8152da0 con 0x7fcfd8154ca0 2023-11-15T02:59:40.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.397+0000 7fcfdd8db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd8164200 con 0x7fcfd81562f0 2023-11-15T02:59:40.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.397+0000 7fcfdd8db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd814e2e0 con 0x7fcfd80a57c0 2023-11-15T02:59:40.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.397+0000 7fcfd67fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfd8154ca0 0x7fcfd80c0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:40.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.397+0000 7fcfd67fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfd8154ca0 0x7fcfd80c0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39386/0 (socket says 172.21.15.121:39386) 2023-11-15T02:59:40.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.397+0000 7fcfd67fc700 1 -- 172.21.15.121:0/134632222 learned_addr learned my addr 172.21.15.121:0/134632222 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:40.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.397+0000 7fcfd6ffd700 1 --2- 172.21.15.121:0/134632222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfd80a57c0 0x7fcfd814ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:40.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.397+0000 7fcfd77fe700 1 --2- 172.21.15.121:0/134632222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfd81562f0 0x7fcfd80bff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:40.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.397+0000 7fcfd67fc700 1 -- 172.21.15.121:0/134632222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfd80a57c0 msgr2=0x7fcfd814ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:40.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.397+0000 7fcfd67fc700 1 --2- 172.21.15.121:0/134632222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfd80a57c0 0x7fcfd814ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:40.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.397+0000 7fcfd67fc700 1 -- 172.21.15.121:0/134632222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfd81562f0 msgr2=0x7fcfd80bff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:40.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.397+0000 7fcfd67fc700 1 --2- 172.21.15.121:0/134632222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfd81562f0 0x7fcfd80bff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:40.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.397+0000 7fcfd67fc700 1 -- 172.21.15.121:0/134632222 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc0092a0 con 0x7fcfd8154ca0 2023-11-15T02:59:40.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.397+0000 7fcfd67fc700 1 --2- 172.21.15.121:0/134632222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfd8154ca0 0x7fcfd80c0d60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fcfd8063d40 tx=0x7fcfcc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:40.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.401+0000 7fcfbeffd700 1 -- 172.21.15.121:0/134632222 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00dd30 con 0x7fcfd8154ca0 2023-11-15T02:59:40.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.401+0000 7fcfbeffd700 1 -- 172.21.15.121:0/134632222 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc00de90 con 0x7fcfd8154ca0 2023-11-15T02:59:40.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.401+0000 7fcfbeffd700 1 -- 172.21.15.121:0/134632222 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00d9c0 con 0x7fcfd8154ca0 2023-11-15T02:59:40.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.401+0000 7fcfdd8db700 1 -- 172.21.15.121:0/134632222 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd8151330 con 0x7fcfd8154ca0 2023-11-15T02:59:40.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.401+0000 7fcfdd8db700 1 -- 172.21.15.121:0/134632222 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd81518f0 con 0x7fcfd8154ca0 2023-11-15T02:59:40.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.401+0000 7fcfdd8db700 1 -- 172.21.15.121:0/134632222 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd8164390 con 0x7fcfd8154ca0 2023-11-15T02:59:40.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.401+0000 7fcfbeffd700 1 -- 172.21.15.121:0/134632222 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fcfcc0263b0 con 0x7fcfd8154ca0 2023-11-15T02:59:40.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.405+0000 7fcfbeffd700 1 --2- 172.21.15.121:0/134632222 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcfc405a510 0x7fcfc405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:40.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.405+0000 7fcfbeffd700 1 -- 172.21.15.121:0/134632222 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fcfcc075710 con 0x7fcfd8154ca0 2023-11-15T02:59:40.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.405+0000 7fcfd6ffd700 1 --2- 172.21.15.121:0/134632222 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcfc405a510 0x7fcfc405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:40.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.405+0000 7fcfd6ffd700 1 --2- 172.21.15.121:0/134632222 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcfc405a510 0x7fcfc405c9d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fcfc8005750 tx=0x7fcfc8009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:40.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.409+0000 7fcfbeffd700 1 -- 172.21.15.121:0/134632222 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcfcc048dd0 con 0x7fcfd8154ca0 2023-11-15T02:59:40.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.561+0000 7fcfdd8db700 1 -- 172.21.15.121:0/134632222 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcfd80a6f50 con 0x7fcfc405a510 2023-11-15T02:59:40.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.573+0000 7fcfbeffd700 1 -- 172.21.15.121:0/134632222 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+358044 (secure 0 0 0) 0x7fcfd80a6f50 con 0x7fcfc405a510 2023-11-15T02:59:40.589 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:40.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.581+0000 7fcfdd8db700 1 -- 172.21.15.121:0/134632222 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcfc405a510 msgr2=0x7fcfc405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:40.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.581+0000 7fcfdd8db700 1 --2- 172.21.15.121:0/134632222 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcfc405a510 0x7fcfc405c9d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fcfc8005750 tx=0x7fcfc8009040).stop 2023-11-15T02:59:40.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.581+0000 7fcfdd8db700 1 -- 172.21.15.121:0/134632222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfd8154ca0 msgr2=0x7fcfd80c0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:40.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.581+0000 7fcfdd8db700 1 --2- 172.21.15.121:0/134632222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfd8154ca0 0x7fcfd80c0d60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fcfd8063d40 tx=0x7fcfcc016000).stop 2023-11-15T02:59:40.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.581+0000 7fcfdd8db700 1 -- 172.21.15.121:0/134632222 shutdown_connections 2023-11-15T02:59:40.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.581+0000 7fcfdd8db700 1 --2- 172.21.15.121:0/134632222 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcfc405a510 0x7fcfc405c9d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:40.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.581+0000 7fcfdd8db700 1 --2- 172.21.15.121:0/134632222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfd81562f0 0x7fcfd80bff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:40.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.581+0000 7fcfdd8db700 1 --2- 172.21.15.121:0/134632222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfd8154ca0 0x7fcfd80c0d60 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:40.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.581+0000 7fcfdd8db700 1 --2- 172.21.15.121:0/134632222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfd80a57c0 0x7fcfd814ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:40.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.581+0000 7fcfdd8db700 1 -- 172.21.15.121:0/134632222 >> 172.21.15.121:0/134632222 conn(0x7fcfd8062190 msgr2=0x7fcfd80c6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:40.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.581+0000 7fcfdd8db700 1 -- 172.21.15.121:0/134632222 shutdown_connections 2023-11-15T02:59:40.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:40.581+0000 7fcfdd8db700 1 -- 172.21.15.121:0/134632222 wait complete. 2023-11-15T02:59:40.595 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T02:59:40.605 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2023-11-15T02:59:39.971723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":151,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382372,"kb_used_data":5604,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752592348,"statfs":{"total":773094113280,"available":770654564352,"internally_reserved":0,"allocated":5738496,"data_stored":2542929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11303,"num_objects":-17,"num_object_clones":0,"num_object_copies":-34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-17,"num_whiteouts":0,"num_read":-50,"num_read_kb":-56,"num_write":-60,"num_write_kb":-31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.903222"},"pg_stats":[{"pgid":"10.3c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.702569+0000","last_change":"2023-11-15T02:59:34.702569+0000","last_active":"2023-11-15T02:59:34.702569+0000","last_peered":"2023-11-15T02:59:34.702569+0000","last_clean":"2023-11-15T02:59:34.702569+0000","last_became_active":"2023-11-15T02:59:34.702272+0000","last_became_peered":"2023-11-15T02:59:34.702272+0000","last_unstale":"2023-11-15T02:59:34.702569+0000","last_undegraded":"2023-11-15T02:59:34.702569+0000","last_fullsized":"2023-11-15T02:59:34.702569+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.3f","version":"60'3","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.702378+0000","last_change":"2023-11-15T02:59:34.265036+0000","last_active":"2023-11-15T02:59:34.702378+0000","last_peered":"2023-11-15T02:59:34.702378+0000","last_clean":"2023-11-15T02:59:34.702378+0000","last_became_active":"2023-11-15T02:59:34.264631+0000","last_became_peered":"2023-11-15T02:59:34.264631+0000","last_unstale":"2023-11-15T02:59:34.702378+0000","last_undegraded":"2023-11-15T02:59:34.702378+0000","last_fullsized":"2023-11-15T02:59:34.702378+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.715077+0000","last_change":"2023-11-15T02:59:34.715077+0000","last_active":"2023-11-15T02:59:34.715077+0000","last_peered":"2023-11-15T02:59:34.715077+0000","last_clean":"2023-11-15T02:59:34.715077+0000","last_became_active":"2023-11-15T02:59:34.710308+0000","last_became_peered":"2023-11-15T02:59:34.710308+0000","last_unstale":"2023-11-15T02:59:34.715077+0000","last_undegraded":"2023-11-15T02:59:34.715077+0000","last_fullsized":"2023-11-15T02:59:34.715077+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.280938+0000","last_change":"2023-11-15T02:59:34.274806+0000","last_active":"2023-11-15T02:59:34.280938+0000","last_peered":"2023-11-15T02:59:34.280938+0000","last_clean":"2023-11-15T02:59:34.280938+0000","last_became_active":"2023-11-15T02:59:34.274536+0000","last_became_peered":"2023-11-15T02:59:34.274536+0000","last_unstale":"2023-11-15T02:59:34.280938+0000","last_undegraded":"2023-11-15T02:59:34.280938+0000","last_fullsized":"2023-11-15T02:59:34.280938+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.713626+0000","last_change":"2023-11-15T02:59:34.702688+0000","last_active":"2023-11-15T02:59:34.713626+0000","last_peered":"2023-11-15T02:59:34.713626+0000","last_clean":"2023-11-15T02:59:34.713626+0000","last_became_active":"2023-11-15T02:59:34.702267+0000","last_became_peered":"2023-11-15T02:59:34.702267+0000","last_unstale":"2023-11-15T02:59:34.713626+0000","last_undegraded":"2023-11-15T02:59:34.713626+0000","last_fullsized":"2023-11-15T02:59:34.713626+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.276406+0000","last_change":"2023-11-15T02:59:34.272252+0000","last_active":"2023-11-15T02:59:34.276406+0000","last_peered":"2023-11-15T02:59:34.276406+0000","last_clean":"2023-11-15T02:59:34.276406+0000","last_became_active":"2023-11-15T02:59:34.271979+0000","last_became_peered":"2023-11-15T02:59:34.271979+0000","last_unstale":"2023-11-15T02:59:34.276406+0000","last_undegraded":"2023-11-15T02:59:34.276406+0000","last_fullsized":"2023-11-15T02:59:34.276406+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.714332+0000","last_change":"2023-11-15T02:59:34.708004+0000","last_active":"2023-11-15T02:59:34.714332+0000","last_peered":"2023-11-15T02:59:34.714332+0000","last_clean":"2023-11-15T02:59:34.714332+0000","last_became_active":"2023-11-15T02:59:34.707733+0000","last_became_peered":"2023-11-15T02:59:34.707733+0000","last_unstale":"2023-11-15T02:59:34.714332+0000","last_undegraded":"2023-11-15T02:59:34.714332+0000","last_fullsized":"2023-11-15T02:59:34.714332+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3c","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.710045+0000","last_change":"2023-11-15T02:59:34.267850+0000","last_active":"2023-11-15T02:59:34.710045+0000","last_peered":"2023-11-15T02:59:34.710045+0000","last_clean":"2023-11-15T02:59:34.710045+0000","last_became_active":"2023-11-15T02:59:34.267579+0000","last_became_peered":"2023-11-15T02:59:34.267579+0000","last_unstale":"2023-11-15T02:59:34.710045+0000","last_undegraded":"2023-11-15T02:59:34.710045+0000","last_fullsized":"2023-11-15T02:59:34.710045+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.718958+0000","last_change":"2023-11-15T02:59:34.714724+0000","last_active":"2023-11-15T02:59:34.718958+0000","last_peered":"2023-11-15T02:59:34.718958+0000","last_clean":"2023-11-15T02:59:34.718958+0000","last_became_active":"2023-11-15T02:59:34.714460+0000","last_became_peered":"2023-11-15T02:59:34.714460+0000","last_unstale":"2023-11-15T02:59:34.718958+0000","last_undegraded":"2023-11-15T02:59:34.718958+0000","last_fullsized":"2023-11-15T02:59:34.718958+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.694905+0000","last_change":"2023-11-15T02:59:34.263921+0000","last_active":"2023-11-15T02:59:34.694905+0000","last_peered":"2023-11-15T02:59:34.694905+0000","last_clean":"2023-11-15T02:59:34.694905+0000","last_became_active":"2023-11-15T02:59:34.261031+0000","last_became_peered":"2023-11-15T02:59:34.261031+0000","last_unstale":"2023-11-15T02:59:34.694905+0000","last_undegraded":"2023-11-15T02:59:34.694905+0000","last_fullsized":"2023-11-15T02:59:34.694905+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716845+0000","last_change":"2023-11-15T02:59:34.716845+0000","last_active":"2023-11-15T02:59:34.716845+0000","last_peered":"2023-11-15T02:59:34.716845+0000","last_clean":"2023-11-15T02:59:34.716845+0000","last_became_active":"2023-11-15T02:59:34.716558+0000","last_became_peered":"2023-11-15T02:59:34.716558+0000","last_unstale":"2023-11-15T02:59:34.716845+0000","last_undegraded":"2023-11-15T02:59:34.716845+0000","last_fullsized":"2023-11-15T02:59:34.716845+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.690603+0000","last_change":"2023-11-15T02:59:34.266240+0000","last_active":"2023-11-15T02:59:34.690603+0000","last_peered":"2023-11-15T02:59:34.690603+0000","last_clean":"2023-11-15T02:59:34.690603+0000","last_became_active":"2023-11-15T02:59:34.265188+0000","last_became_peered":"2023-11-15T02:59:34.265188+0000","last_unstale":"2023-11-15T02:59:34.690603+0000","last_undegraded":"2023-11-15T02:59:34.690603+0000","last_fullsized":"2023-11-15T02:59:34.690603+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.719027+0000","last_change":"2023-11-15T02:59:34.719027+0000","last_active":"2023-11-15T02:59:34.719027+0000","last_peered":"2023-11-15T02:59:34.719027+0000","last_clean":"2023-11-15T02:59:34.719027+0000","last_became_active":"2023-11-15T02:59:34.718717+0000","last_became_peered":"2023-11-15T02:59:34.718717+0000","last_unstale":"2023-11-15T02:59:34.719027+0000","last_undegraded":"2023-11-15T02:59:34.719027+0000","last_fullsized":"2023-11-15T02:59:34.719027+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.273500+0000","last_change":"2023-11-15T02:59:34.263881+0000","last_active":"2023-11-15T02:59:34.273500+0000","last_peered":"2023-11-15T02:59:34.273500+0000","last_clean":"2023-11-15T02:59:34.273500+0000","last_became_active":"2023-11-15T02:59:34.263571+0000","last_became_peered":"2023-11-15T02:59:34.263571+0000","last_unstale":"2023-11-15T02:59:34.273500+0000","last_undegraded":"2023-11-15T02:59:34.273500+0000","last_fullsized":"2023-11-15T02:59:34.273500+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.715365+0000","last_change":"2023-11-15T02:59:34.715365+0000","last_active":"2023-11-15T02:59:34.715365+0000","last_peered":"2023-11-15T02:59:34.715365+0000","last_clean":"2023-11-15T02:59:34.715365+0000","last_became_active":"2023-11-15T02:59:34.715043+0000","last_became_peered":"2023-11-15T02:59:34.715043+0000","last_unstale":"2023-11-15T02:59:34.715365+0000","last_undegraded":"2023-11-15T02:59:34.715365+0000","last_fullsized":"2023-11-15T02:59:34.715365+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.271621+0000","last_change":"2023-11-15T02:59:34.264832+0000","last_active":"2023-11-15T02:59:34.271621+0000","last_peered":"2023-11-15T02:59:34.271621+0000","last_clean":"2023-11-15T02:59:34.271621+0000","last_became_active":"2023-11-15T02:59:34.264571+0000","last_became_peered":"2023-11-15T02:59:34.264571+0000","last_unstale":"2023-11-15T02:59:34.271621+0000","last_undegraded":"2023-11-15T02:59:34.271621+0000","last_fullsized":"2023-11-15T02:59:34.271621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.719012+0000","last_change":"2023-11-15T02:59:34.715353+0000","last_active":"2023-11-15T02:59:34.719012+0000","last_peered":"2023-11-15T02:59:34.719012+0000","last_clean":"2023-11-15T02:59:34.719012+0000","last_became_active":"2023-11-15T02:59:34.715023+0000","last_became_peered":"2023-11-15T02:59:34.715023+0000","last_unstale":"2023-11-15T02:59:34.719012+0000","last_undegraded":"2023-11-15T02:59:34.719012+0000","last_fullsized":"2023-11-15T02:59:34.719012+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.270817+0000","last_change":"2023-11-15T02:59:34.262030+0000","last_active":"2023-11-15T02:59:34.270817+0000","last_peered":"2023-11-15T02:59:34.270817+0000","last_clean":"2023-11-15T02:59:34.270817+0000","last_became_active":"2023-11-15T02:59:34.261689+0000","last_became_peered":"2023-11-15T02:59:34.261689+0000","last_unstale":"2023-11-15T02:59:34.270817+0000","last_undegraded":"2023-11-15T02:59:34.270817+0000","last_fullsized":"2023-11-15T02:59:34.270817+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.701348+0000","last_change":"2023-11-15T02:59:34.701348+0000","last_active":"2023-11-15T02:59:34.701348+0000","last_peered":"2023-11-15T02:59:34.701348+0000","last_clean":"2023-11-15T02:59:34.701348+0000","last_became_active":"2023-11-15T02:59:34.701079+0000","last_became_peered":"2023-11-15T02:59:34.701079+0000","last_unstale":"2023-11-15T02:59:34.701348+0000","last_undegraded":"2023-11-15T02:59:34.701348+0000","last_fullsized":"2023-11-15T02:59:34.701348+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.688465+0000","last_change":"2023-11-15T02:59:34.269872+0000","last_active":"2023-11-15T02:59:34.688465+0000","last_peered":"2023-11-15T02:59:34.688465+0000","last_clean":"2023-11-15T02:59:34.688465+0000","last_became_active":"2023-11-15T02:59:34.269585+0000","last_became_peered":"2023-11-15T02:59:34.269585+0000","last_unstale":"2023-11-15T02:59:34.688465+0000","last_undegraded":"2023-11-15T02:59:34.688465+0000","last_fullsized":"2023-11-15T02:59:34.688465+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.707253+0000","last_change":"2023-11-15T02:59:34.707253+0000","last_active":"2023-11-15T02:59:34.707253+0000","last_peered":"2023-11-15T02:59:34.707253+0000","last_clean":"2023-11-15T02:59:34.707253+0000","last_became_active":"2023-11-15T02:59:34.706969+0000","last_became_peered":"2023-11-15T02:59:34.706969+0000","last_unstale":"2023-11-15T02:59:34.707253+0000","last_undegraded":"2023-11-15T02:59:34.707253+0000","last_fullsized":"2023-11-15T02:59:34.707253+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.688078+0000","last_change":"2023-11-15T02:59:34.269836+0000","last_active":"2023-11-15T02:59:34.688078+0000","last_peered":"2023-11-15T02:59:34.688078+0000","last_clean":"2023-11-15T02:59:34.688078+0000","last_became_active":"2023-11-15T02:59:34.269512+0000","last_became_peered":"2023-11-15T02:59:34.269512+0000","last_unstale":"2023-11-15T02:59:34.688078+0000","last_undegraded":"2023-11-15T02:59:34.688078+0000","last_fullsized":"2023-11-15T02:59:34.688078+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.717917+0000","last_change":"2023-11-15T02:59:34.717917+0000","last_active":"2023-11-15T02:59:34.717917+0000","last_peered":"2023-11-15T02:59:34.717917+0000","last_clean":"2023-11-15T02:59:34.717917+0000","last_became_active":"2023-11-15T02:59:34.717573+0000","last_became_peered":"2023-11-15T02:59:34.717573+0000","last_unstale":"2023-11-15T02:59:34.717917+0000","last_undegraded":"2023-11-15T02:59:34.717917+0000","last_fullsized":"2023-11-15T02:59:34.717917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.34","version":"60'4","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.724147+0000","last_change":"2023-11-15T02:59:34.272591+0000","last_active":"2023-11-15T02:59:34.724147+0000","last_peered":"2023-11-15T02:59:34.724147+0000","last_clean":"2023-11-15T02:59:34.724147+0000","last_became_active":"2023-11-15T02:59:34.270963+0000","last_became_peered":"2023-11-15T02:59:34.270963+0000","last_unstale":"2023-11-15T02:59:34.724147+0000","last_undegraded":"2023-11-15T02:59:34.724147+0000","last_fullsized":"2023-11-15T02:59:34.724147+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.709579+0000","last_change":"2023-11-15T02:59:34.709579+0000","last_active":"2023-11-15T02:59:34.709579+0000","last_peered":"2023-11-15T02:59:34.709579+0000","last_clean":"2023-11-15T02:59:34.709579+0000","last_became_active":"2023-11-15T02:59:34.709324+0000","last_became_peered":"2023-11-15T02:59:34.709324+0000","last_unstale":"2023-11-15T02:59:34.709579+0000","last_undegraded":"2023-11-15T02:59:34.709579+0000","last_fullsized":"2023-11-15T02:59:34.709579+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.33","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.689591+0000","last_change":"2023-11-15T02:59:34.261668+0000","last_active":"2023-11-15T02:59:34.689591+0000","last_peered":"2023-11-15T02:59:34.689591+0000","last_clean":"2023-11-15T02:59:34.689591+0000","last_became_active":"2023-11-15T02:59:34.261392+0000","last_became_peered":"2023-11-15T02:59:34.261392+0000","last_unstale":"2023-11-15T02:59:34.689591+0000","last_undegraded":"2023-11-15T02:59:34.689591+0000","last_fullsized":"2023-11-15T02:59:34.689591+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.717458+0000","last_change":"2023-11-15T02:59:34.717458+0000","last_active":"2023-11-15T02:59:34.717458+0000","last_peered":"2023-11-15T02:59:34.717458+0000","last_clean":"2023-11-15T02:59:34.717458+0000","last_became_active":"2023-11-15T02:59:34.717124+0000","last_became_peered":"2023-11-15T02:59:34.717124+0000","last_unstale":"2023-11-15T02:59:34.717458+0000","last_undegraded":"2023-11-15T02:59:34.717458+0000","last_fullsized":"2023-11-15T02:59:34.717458+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.274573+0000","last_change":"2023-11-15T02:59:34.267337+0000","last_active":"2023-11-15T02:59:34.274573+0000","last_peered":"2023-11-15T02:59:34.274573+0000","last_clean":"2023-11-15T02:59:34.274573+0000","last_became_active":"2023-11-15T02:59:34.267054+0000","last_became_peered":"2023-11-15T02:59:34.267054+0000","last_unstale":"2023-11-15T02:59:34.274573+0000","last_undegraded":"2023-11-15T02:59:34.274573+0000","last_fullsized":"2023-11-15T02:59:34.274573+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.709174+0000","last_change":"2023-11-15T02:59:34.709174+0000","last_active":"2023-11-15T02:59:34.709174+0000","last_peered":"2023-11-15T02:59:34.709174+0000","last_clean":"2023-11-15T02:59:34.709174+0000","last_became_active":"2023-11-15T02:59:34.708872+0000","last_became_peered":"2023-11-15T02:59:34.708872+0000","last_unstale":"2023-11-15T02:59:34.709174+0000","last_undegraded":"2023-11-15T02:59:34.709174+0000","last_fullsized":"2023-11-15T02:59:34.709174+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.270777+0000","last_change":"2023-11-15T02:59:34.261461+0000","last_active":"2023-11-15T02:59:34.270777+0000","last_peered":"2023-11-15T02:59:34.270777+0000","last_clean":"2023-11-15T02:59:34.270777+0000","last_became_active":"2023-11-15T02:59:34.261153+0000","last_became_peered":"2023-11-15T02:59:34.261153+0000","last_unstale":"2023-11-15T02:59:34.270777+0000","last_undegraded":"2023-11-15T02:59:34.270777+0000","last_fullsized":"2023-11-15T02:59:34.270777+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.718989+0000","last_change":"2023-11-15T02:59:34.713836+0000","last_active":"2023-11-15T02:59:34.718989+0000","last_peered":"2023-11-15T02:59:34.718989+0000","last_clean":"2023-11-15T02:59:34.718989+0000","last_became_active":"2023-11-15T02:59:34.713342+0000","last_became_peered":"2023-11-15T02:59:34.713342+0000","last_unstale":"2023-11-15T02:59:34.718989+0000","last_undegraded":"2023-11-15T02:59:34.718989+0000","last_fullsized":"2023-11-15T02:59:34.718989+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.30","version":"60'4","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.721503+0000","last_change":"2023-11-15T02:59:34.264405+0000","last_active":"2023-11-15T02:59:34.721503+0000","last_peered":"2023-11-15T02:59:34.721503+0000","last_clean":"2023-11-15T02:59:34.721503+0000","last_became_active":"2023-11-15T02:59:34.264095+0000","last_became_peered":"2023-11-15T02:59:34.264095+0000","last_unstale":"2023-11-15T02:59:34.721503+0000","last_undegraded":"2023-11-15T02:59:34.721503+0000","last_fullsized":"2023-11-15T02:59:34.721503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.694211+0000","last_change":"2023-11-15T02:59:34.694211+0000","last_active":"2023-11-15T02:59:34.694211+0000","last_peered":"2023-11-15T02:59:34.694211+0000","last_clean":"2023-11-15T02:59:34.694211+0000","last_became_active":"2023-11-15T02:59:34.693708+0000","last_became_peered":"2023-11-15T02:59:34.693708+0000","last_unstale":"2023-11-15T02:59:34.694211+0000","last_undegraded":"2023-11-15T02:59:34.694211+0000","last_fullsized":"2023-11-15T02:59:34.694211+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278070+0000","last_change":"2023-11-15T02:59:34.271015+0000","last_active":"2023-11-15T02:59:34.278070+0000","last_peered":"2023-11-15T02:59:34.278070+0000","last_clean":"2023-11-15T02:59:34.278070+0000","last_became_active":"2023-11-15T02:59:34.270758+0000","last_became_peered":"2023-11-15T02:59:34.270758+0000","last_unstale":"2023-11-15T02:59:34.278070+0000","last_undegraded":"2023-11-15T02:59:34.278070+0000","last_fullsized":"2023-11-15T02:59:34.278070+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.721266+0000","last_change":"2023-11-15T02:59:34.721266+0000","last_active":"2023-11-15T02:59:34.721266+0000","last_peered":"2023-11-15T02:59:34.721266+0000","last_clean":"2023-11-15T02:59:34.721266+0000","last_became_active":"2023-11-15T02:59:34.720978+0000","last_became_peered":"2023-11-15T02:59:34.720978+0000","last_unstale":"2023-11-15T02:59:34.721266+0000","last_undegraded":"2023-11-15T02:59:34.721266+0000","last_fullsized":"2023-11-15T02:59:34.721266+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.276457+0000","last_change":"2023-11-15T02:59:34.269316+0000","last_active":"2023-11-15T02:59:34.276457+0000","last_peered":"2023-11-15T02:59:34.276457+0000","last_clean":"2023-11-15T02:59:34.276457+0000","last_became_active":"2023-11-15T02:59:34.268953+0000","last_became_peered":"2023-11-15T02:59:34.268953+0000","last_unstale":"2023-11-15T02:59:34.276457+0000","last_undegraded":"2023-11-15T02:59:34.276457+0000","last_fullsized":"2023-11-15T02:59:34.276457+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.703052+0000","last_change":"2023-11-15T02:59:34.703052+0000","last_active":"2023-11-15T02:59:34.703052+0000","last_peered":"2023-11-15T02:59:34.703052+0000","last_clean":"2023-11-15T02:59:34.703052+0000","last_became_active":"2023-11-15T02:59:34.702791+0000","last_became_peered":"2023-11-15T02:59:34.702791+0000","last_unstale":"2023-11-15T02:59:34.703052+0000","last_undegraded":"2023-11-15T02:59:34.703052+0000","last_fullsized":"2023-11-15T02:59:34.703052+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2d","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.696289+0000","last_change":"2023-11-15T02:59:34.257033+0000","last_active":"2023-11-15T02:59:34.696289+0000","last_peered":"2023-11-15T02:59:34.696289+0000","last_clean":"2023-11-15T02:59:34.696289+0000","last_became_active":"2023-11-15T02:59:34.256744+0000","last_became_peered":"2023-11-15T02:59:34.256744+0000","last_unstale":"2023-11-15T02:59:34.696289+0000","last_undegraded":"2023-11-15T02:59:34.696289+0000","last_fullsized":"2023-11-15T02:59:34.696289+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.714601+0000","last_change":"2023-11-15T02:59:34.709810+0000","last_active":"2023-11-15T02:59:34.714601+0000","last_peered":"2023-11-15T02:59:34.714601+0000","last_clean":"2023-11-15T02:59:34.714601+0000","last_became_active":"2023-11-15T02:59:34.709481+0000","last_became_peered":"2023-11-15T02:59:34.709481+0000","last_unstale":"2023-11-15T02:59:34.714601+0000","last_undegraded":"2023-11-15T02:59:34.714601+0000","last_fullsized":"2023-11-15T02:59:34.714601+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.280852+0000","last_change":"2023-11-15T02:59:34.275824+0000","last_active":"2023-11-15T02:59:34.280852+0000","last_peered":"2023-11-15T02:59:34.280852+0000","last_clean":"2023-11-15T02:59:34.280852+0000","last_became_active":"2023-11-15T02:59:34.275088+0000","last_became_peered":"2023-11-15T02:59:34.275088+0000","last_unstale":"2023-11-15T02:59:34.280852+0000","last_undegraded":"2023-11-15T02:59:34.280852+0000","last_fullsized":"2023-11-15T02:59:34.280852+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.718638+0000","last_change":"2023-11-15T02:59:34.718638+0000","last_active":"2023-11-15T02:59:34.718638+0000","last_peered":"2023-11-15T02:59:34.718638+0000","last_clean":"2023-11-15T02:59:34.718638+0000","last_became_active":"2023-11-15T02:59:34.718018+0000","last_became_peered":"2023-11-15T02:59:34.718018+0000","last_unstale":"2023-11-15T02:59:34.718638+0000","last_undegraded":"2023-11-15T02:59:34.718638+0000","last_fullsized":"2023-11-15T02:59:34.718638+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2b","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.688086+0000","last_change":"2023-11-15T02:59:34.271365+0000","last_active":"2023-11-15T02:59:34.688086+0000","last_peered":"2023-11-15T02:59:34.688086+0000","last_clean":"2023-11-15T02:59:34.688086+0000","last_became_active":"2023-11-15T02:59:34.270414+0000","last_became_peered":"2023-11-15T02:59:34.270414+0000","last_unstale":"2023-11-15T02:59:34.688086+0000","last_undegraded":"2023-11-15T02:59:34.688086+0000","last_fullsized":"2023-11-15T02:59:34.688086+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.705469+0000","last_change":"2023-11-15T02:59:34.705469+0000","last_active":"2023-11-15T02:59:34.705469+0000","last_peered":"2023-11-15T02:59:34.705469+0000","last_clean":"2023-11-15T02:59:34.705469+0000","last_became_active":"2023-11-15T02:59:34.705188+0000","last_became_peered":"2023-11-15T02:59:34.705188+0000","last_unstale":"2023-11-15T02:59:34.705469+0000","last_undegraded":"2023-11-15T02:59:34.705469+0000","last_fullsized":"2023-11-15T02:59:34.705469+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.274106+0000","last_change":"2023-11-15T02:59:34.269772+0000","last_active":"2023-11-15T02:59:34.274106+0000","last_peered":"2023-11-15T02:59:34.274106+0000","last_clean":"2023-11-15T02:59:34.274106+0000","last_became_active":"2023-11-15T02:59:34.269150+0000","last_became_peered":"2023-11-15T02:59:34.269150+0000","last_unstale":"2023-11-15T02:59:34.274106+0000","last_undegraded":"2023-11-15T02:59:34.274106+0000","last_fullsized":"2023-11-15T02:59:34.274106+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.706266+0000","last_change":"2023-11-15T02:59:34.706266+0000","last_active":"2023-11-15T02:59:34.706266+0000","last_peered":"2023-11-15T02:59:34.706266+0000","last_clean":"2023-11-15T02:59:34.706266+0000","last_became_active":"2023-11-15T02:59:34.705045+0000","last_became_peered":"2023-11-15T02:59:34.705045+0000","last_unstale":"2023-11-15T02:59:34.706266+0000","last_undegraded":"2023-11-15T02:59:34.706266+0000","last_fullsized":"2023-11-15T02:59:34.706266+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.275956+0000","last_change":"2023-11-15T02:59:34.271476+0000","last_active":"2023-11-15T02:59:34.275956+0000","last_peered":"2023-11-15T02:59:34.275956+0000","last_clean":"2023-11-15T02:59:34.275956+0000","last_became_active":"2023-11-15T02:59:34.271135+0000","last_became_peered":"2023-11-15T02:59:34.271135+0000","last_unstale":"2023-11-15T02:59:34.275956+0000","last_undegraded":"2023-11-15T02:59:34.275956+0000","last_fullsized":"2023-11-15T02:59:34.275956+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716085+0000","last_change":"2023-11-15T02:59:34.709466+0000","last_active":"2023-11-15T02:59:34.716085+0000","last_peered":"2023-11-15T02:59:34.716085+0000","last_clean":"2023-11-15T02:59:34.716085+0000","last_became_active":"2023-11-15T02:59:34.709166+0000","last_became_peered":"2023-11-15T02:59:34.709166+0000","last_unstale":"2023-11-15T02:59:34.716085+0000","last_undegraded":"2023-11-15T02:59:34.716085+0000","last_fullsized":"2023-11-15T02:59:34.716085+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.268460+0000","last_change":"2023-11-15T02:59:34.264047+0000","last_active":"2023-11-15T02:59:34.268460+0000","last_peered":"2023-11-15T02:59:34.268460+0000","last_clean":"2023-11-15T02:59:34.268460+0000","last_became_active":"2023-11-15T02:59:34.263525+0000","last_became_peered":"2023-11-15T02:59:34.263525+0000","last_unstale":"2023-11-15T02:59:34.268460+0000","last_undegraded":"2023-11-15T02:59:34.268460+0000","last_fullsized":"2023-11-15T02:59:34.268460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.714415+0000","last_change":"2023-11-15T02:59:34.707630+0000","last_active":"2023-11-15T02:59:34.714415+0000","last_peered":"2023-11-15T02:59:34.714415+0000","last_clean":"2023-11-15T02:59:34.714415+0000","last_became_active":"2023-11-15T02:59:34.707321+0000","last_became_peered":"2023-11-15T02:59:34.707321+0000","last_unstale":"2023-11-15T02:59:34.714415+0000","last_undegraded":"2023-11-15T02:59:34.714415+0000","last_fullsized":"2023-11-15T02:59:34.714415+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.264845+0000","last_change":"2023-11-15T02:59:34.253441+0000","last_active":"2023-11-15T02:59:34.264845+0000","last_peered":"2023-11-15T02:59:34.264845+0000","last_clean":"2023-11-15T02:59:34.264845+0000","last_became_active":"2023-11-15T02:59:34.253172+0000","last_became_peered":"2023-11-15T02:59:34.253172+0000","last_unstale":"2023-11-15T02:59:34.264845+0000","last_undegraded":"2023-11-15T02:59:34.264845+0000","last_fullsized":"2023-11-15T02:59:34.264845+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.706426+0000","last_change":"2023-11-15T02:59:34.706426+0000","last_active":"2023-11-15T02:59:34.706426+0000","last_peered":"2023-11-15T02:59:34.706426+0000","last_clean":"2023-11-15T02:59:34.706426+0000","last_became_active":"2023-11-15T02:59:34.705567+0000","last_became_peered":"2023-11-15T02:59:34.705567+0000","last_unstale":"2023-11-15T02:59:34.706426+0000","last_undegraded":"2023-11-15T02:59:34.706426+0000","last_fullsized":"2023-11-15T02:59:34.706426+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.702168+0000","last_change":"2023-11-15T02:59:34.264111+0000","last_active":"2023-11-15T02:59:34.702168+0000","last_peered":"2023-11-15T02:59:34.702168+0000","last_clean":"2023-11-15T02:59:34.702168+0000","last_became_active":"2023-11-15T02:59:34.263570+0000","last_became_peered":"2023-11-15T02:59:34.263570+0000","last_unstale":"2023-11-15T02:59:34.702168+0000","last_undegraded":"2023-11-15T02:59:34.702168+0000","last_fullsized":"2023-11-15T02:59:34.702168+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716225+0000","last_change":"2023-11-15T02:59:34.709971+0000","last_active":"2023-11-15T02:59:34.716225+0000","last_peered":"2023-11-15T02:59:34.716225+0000","last_clean":"2023-11-15T02:59:34.716225+0000","last_became_active":"2023-11-15T02:59:34.709667+0000","last_became_peered":"2023-11-15T02:59:34.709667+0000","last_unstale":"2023-11-15T02:59:34.716225+0000","last_undegraded":"2023-11-15T02:59:34.716225+0000","last_fullsized":"2023-11-15T02:59:34.716225+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.275473+0000","last_change":"2023-11-15T02:59:34.264728+0000","last_active":"2023-11-15T02:59:34.275473+0000","last_peered":"2023-11-15T02:59:34.275473+0000","last_clean":"2023-11-15T02:59:34.275473+0000","last_became_active":"2023-11-15T02:59:34.264416+0000","last_became_peered":"2023-11-15T02:59:34.264416+0000","last_unstale":"2023-11-15T02:59:34.275473+0000","last_undegraded":"2023-11-15T02:59:34.275473+0000","last_fullsized":"2023-11-15T02:59:34.275473+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716101+0000","last_change":"2023-11-15T02:59:34.709219+0000","last_active":"2023-11-15T02:59:34.716101+0000","last_peered":"2023-11-15T02:59:34.716101+0000","last_clean":"2023-11-15T02:59:34.716101+0000","last_became_active":"2023-11-15T02:59:34.708939+0000","last_became_peered":"2023-11-15T02:59:34.708939+0000","last_unstale":"2023-11-15T02:59:34.716101+0000","last_undegraded":"2023-11-15T02:59:34.716101+0000","last_fullsized":"2023-11-15T02:59:34.716101+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.270389+0000","last_change":"2023-11-15T02:59:34.261680+0000","last_active":"2023-11-15T02:59:34.270389+0000","last_peered":"2023-11-15T02:59:34.270389+0000","last_clean":"2023-11-15T02:59:34.270389+0000","last_became_active":"2023-11-15T02:59:34.260538+0000","last_became_peered":"2023-11-15T02:59:34.260538+0000","last_unstale":"2023-11-15T02:59:34.270389+0000","last_undegraded":"2023-11-15T02:59:34.270389+0000","last_fullsized":"2023-11-15T02:59:34.270389+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.707619+0000","last_change":"2023-11-15T02:59:34.707619+0000","last_active":"2023-11-15T02:59:34.707619+0000","last_peered":"2023-11-15T02:59:34.707619+0000","last_clean":"2023-11-15T02:59:34.707619+0000","last_became_active":"2023-11-15T02:59:34.707297+0000","last_became_peered":"2023-11-15T02:59:34.707297+0000","last_unstale":"2023-11-15T02:59:34.707619+0000","last_undegraded":"2023-11-15T02:59:34.707619+0000","last_fullsized":"2023-11-15T02:59:34.707619+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.272192+0000","last_change":"2023-11-15T02:59:34.266701+0000","last_active":"2023-11-15T02:59:34.272192+0000","last_peered":"2023-11-15T02:59:34.272192+0000","last_clean":"2023-11-15T02:59:34.272192+0000","last_became_active":"2023-11-15T02:59:34.266347+0000","last_became_peered":"2023-11-15T02:59:34.266347+0000","last_unstale":"2023-11-15T02:59:34.272192+0000","last_undegraded":"2023-11-15T02:59:34.272192+0000","last_fullsized":"2023-11-15T02:59:34.272192+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.705873+0000","last_change":"2023-11-15T02:59:34.705873+0000","last_active":"2023-11-15T02:59:34.705873+0000","last_peered":"2023-11-15T02:59:34.705873+0000","last_clean":"2023-11-15T02:59:34.705873+0000","last_became_active":"2023-11-15T02:59:34.705522+0000","last_became_peered":"2023-11-15T02:59:34.705522+0000","last_unstale":"2023-11-15T02:59:34.705873+0000","last_undegraded":"2023-11-15T02:59:34.705873+0000","last_fullsized":"2023-11-15T02:59:34.705873+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278740+0000","last_change":"2023-11-15T02:59:34.274845+0000","last_active":"2023-11-15T02:59:34.278740+0000","last_peered":"2023-11-15T02:59:34.278740+0000","last_clean":"2023-11-15T02:59:34.278740+0000","last_became_active":"2023-11-15T02:59:34.274495+0000","last_became_peered":"2023-11-15T02:59:34.274495+0000","last_unstale":"2023-11-15T02:59:34.278740+0000","last_undegraded":"2023-11-15T02:59:34.278740+0000","last_fullsized":"2023-11-15T02:59:34.278740+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.723692+0000","last_change":"2023-11-15T02:59:34.720105+0000","last_active":"2023-11-15T02:59:34.723692+0000","last_peered":"2023-11-15T02:59:34.723692+0000","last_clean":"2023-11-15T02:59:34.723692+0000","last_became_active":"2023-11-15T02:59:34.719807+0000","last_became_peered":"2023-11-15T02:59:34.719807+0000","last_unstale":"2023-11-15T02:59:34.723692+0000","last_undegraded":"2023-11-15T02:59:34.723692+0000","last_fullsized":"2023-11-15T02:59:34.723692+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.271334+0000","last_change":"2023-11-15T02:59:34.262355+0000","last_active":"2023-11-15T02:59:34.271334+0000","last_peered":"2023-11-15T02:59:34.271334+0000","last_clean":"2023-11-15T02:59:34.271334+0000","last_became_active":"2023-11-15T02:59:34.260844+0000","last_became_peered":"2023-11-15T02:59:34.260844+0000","last_unstale":"2023-11-15T02:59:34.271334+0000","last_undegraded":"2023-11-15T02:59:34.271334+0000","last_fullsized":"2023-11-15T02:59:34.271334+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.711149+0000","last_change":"2023-11-15T02:59:34.711149+0000","last_active":"2023-11-15T02:59:34.711149+0000","last_peered":"2023-11-15T02:59:34.711149+0000","last_clean":"2023-11-15T02:59:34.711149+0000","last_became_active":"2023-11-15T02:59:34.710867+0000","last_became_peered":"2023-11-15T02:59:34.710867+0000","last_unstale":"2023-11-15T02:59:34.711149+0000","last_undegraded":"2023-11-15T02:59:34.711149+0000","last_fullsized":"2023-11-15T02:59:34.711149+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278038+0000","last_change":"2023-11-15T02:59:34.276394+0000","last_active":"2023-11-15T02:59:34.278038+0000","last_peered":"2023-11-15T02:59:34.278038+0000","last_clean":"2023-11-15T02:59:34.278038+0000","last_became_active":"2023-11-15T02:59:34.276134+0000","last_became_peered":"2023-11-15T02:59:34.276134+0000","last_unstale":"2023-11-15T02:59:34.278038+0000","last_undegraded":"2023-11-15T02:59:34.278038+0000","last_fullsized":"2023-11-15T02:59:34.278038+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.697057+0000","last_change":"2023-11-15T02:59:34.697057+0000","last_active":"2023-11-15T02:59:34.697057+0000","last_peered":"2023-11-15T02:59:34.697057+0000","last_clean":"2023-11-15T02:59:34.697057+0000","last_became_active":"2023-11-15T02:59:34.696211+0000","last_became_peered":"2023-11-15T02:59:34.696211+0000","last_unstale":"2023-11-15T02:59:34.697057+0000","last_undegraded":"2023-11-15T02:59:34.697057+0000","last_fullsized":"2023-11-15T02:59:34.697057+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.703503+0000","last_change":"2023-11-15T02:59:34.272228+0000","last_active":"2023-11-15T02:59:34.703503+0000","last_peered":"2023-11-15T02:59:34.703503+0000","last_clean":"2023-11-15T02:59:34.703503+0000","last_became_active":"2023-11-15T02:59:34.271741+0000","last_became_peered":"2023-11-15T02:59:34.271741+0000","last_unstale":"2023-11-15T02:59:34.703503+0000","last_undegraded":"2023-11-15T02:59:34.703503+0000","last_fullsized":"2023-11-15T02:59:34.703503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.695873+0000","last_change":"2023-11-15T02:59:34.695873+0000","last_active":"2023-11-15T02:59:34.695873+0000","last_peered":"2023-11-15T02:59:34.695873+0000","last_clean":"2023-11-15T02:59:34.695873+0000","last_became_active":"2023-11-15T02:59:34.695590+0000","last_became_peered":"2023-11-15T02:59:34.695590+0000","last_unstale":"2023-11-15T02:59:34.695873+0000","last_undegraded":"2023-11-15T02:59:34.695873+0000","last_fullsized":"2023-11-15T02:59:34.695873+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.693584+0000","last_change":"2023-11-15T02:59:34.272647+0000","last_active":"2023-11-15T02:59:34.693584+0000","last_peered":"2023-11-15T02:59:34.693584+0000","last_clean":"2023-11-15T02:59:34.693584+0000","last_became_active":"2023-11-15T02:59:34.272331+0000","last_became_peered":"2023-11-15T02:59:34.272331+0000","last_unstale":"2023-11-15T02:59:34.693584+0000","last_undegraded":"2023-11-15T02:59:34.693584+0000","last_fullsized":"2023-11-15T02:59:34.693584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.710364+0000","last_change":"2023-11-15T02:59:34.710364+0000","last_active":"2023-11-15T02:59:34.710364+0000","last_peered":"2023-11-15T02:59:34.710364+0000","last_clean":"2023-11-15T02:59:34.710364+0000","last_became_active":"2023-11-15T02:59:34.706803+0000","last_became_peered":"2023-11-15T02:59:34.706803+0000","last_unstale":"2023-11-15T02:59:34.710364+0000","last_undegraded":"2023-11-15T02:59:34.710364+0000","last_fullsized":"2023-11-15T02:59:34.710364+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.710145+0000","last_change":"2023-11-15T02:59:34.263720+0000","last_active":"2023-11-15T02:59:34.710145+0000","last_peered":"2023-11-15T02:59:34.710145+0000","last_clean":"2023-11-15T02:59:34.710145+0000","last_became_active":"2023-11-15T02:59:34.262994+0000","last_became_peered":"2023-11-15T02:59:34.262994+0000","last_unstale":"2023-11-15T02:59:34.710145+0000","last_undegraded":"2023-11-15T02:59:34.710145+0000","last_fullsized":"2023-11-15T02:59:34.710145+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.702473+0000","last_change":"2023-11-15T02:59:34.702473+0000","last_active":"2023-11-15T02:59:34.702473+0000","last_peered":"2023-11-15T02:59:34.702473+0000","last_clean":"2023-11-15T02:59:34.702473+0000","last_became_active":"2023-11-15T02:59:34.702094+0000","last_became_peered":"2023-11-15T02:59:34.702094+0000","last_unstale":"2023-11-15T02:59:34.702473+0000","last_undegraded":"2023-11-15T02:59:34.702473+0000","last_fullsized":"2023-11-15T02:59:34.702473+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.271846+0000","last_change":"2023-11-15T02:59:34.260019+0000","last_active":"2023-11-15T02:59:34.271846+0000","last_peered":"2023-11-15T02:59:34.271846+0000","last_clean":"2023-11-15T02:59:34.271846+0000","last_became_active":"2023-11-15T02:59:34.259470+0000","last_became_peered":"2023-11-15T02:59:34.259470+0000","last_unstale":"2023-11-15T02:59:34.271846+0000","last_undegraded":"2023-11-15T02:59:34.271846+0000","last_fullsized":"2023-11-15T02:59:34.271846+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.707408+0000","last_change":"2023-11-15T02:59:34.707408+0000","last_active":"2023-11-15T02:59:34.707408+0000","last_peered":"2023-11-15T02:59:34.707408+0000","last_clean":"2023-11-15T02:59:34.707408+0000","last_became_active":"2023-11-15T02:59:34.706951+0000","last_became_peered":"2023-11-15T02:59:34.706951+0000","last_unstale":"2023-11-15T02:59:34.707408+0000","last_undegraded":"2023-11-15T02:59:34.707408+0000","last_fullsized":"2023-11-15T02:59:34.707408+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.280951+0000","last_change":"2023-11-15T02:59:34.276164+0000","last_active":"2023-11-15T02:59:34.280951+0000","last_peered":"2023-11-15T02:59:34.280951+0000","last_clean":"2023-11-15T02:59:34.280951+0000","last_became_active":"2023-11-15T02:59:34.275881+0000","last_became_peered":"2023-11-15T02:59:34.275881+0000","last_unstale":"2023-11-15T02:59:34.280951+0000","last_undegraded":"2023-11-15T02:59:34.280951+0000","last_fullsized":"2023-11-15T02:59:34.280951+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.714689+0000","last_change":"2023-11-15T02:59:34.711382+0000","last_active":"2023-11-15T02:59:34.714689+0000","last_peered":"2023-11-15T02:59:34.714689+0000","last_clean":"2023-11-15T02:59:34.714689+0000","last_became_active":"2023-11-15T02:59:34.710832+0000","last_became_peered":"2023-11-15T02:59:34.710832+0000","last_unstale":"2023-11-15T02:59:34.714689+0000","last_undegraded":"2023-11-15T02:59:34.714689+0000","last_fullsized":"2023-11-15T02:59:34.714689+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.274124+0000","last_change":"2023-11-15T02:59:34.269967+0000","last_active":"2023-11-15T02:59:34.274124+0000","last_peered":"2023-11-15T02:59:34.274124+0000","last_clean":"2023-11-15T02:59:34.274124+0000","last_became_active":"2023-11-15T02:59:34.269516+0000","last_became_peered":"2023-11-15T02:59:34.269516+0000","last_unstale":"2023-11-15T02:59:34.274124+0000","last_undegraded":"2023-11-15T02:59:34.274124+0000","last_fullsized":"2023-11-15T02:59:34.274124+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.713129+0000","last_change":"2023-11-15T02:59:34.713129+0000","last_active":"2023-11-15T02:59:34.713129+0000","last_peered":"2023-11-15T02:59:34.713129+0000","last_clean":"2023-11-15T02:59:34.713129+0000","last_became_active":"2023-11-15T02:59:34.704067+0000","last_became_peered":"2023-11-15T02:59:34.704067+0000","last_unstale":"2023-11-15T02:59:34.713129+0000","last_undegraded":"2023-11-15T02:59:34.713129+0000","last_fullsized":"2023-11-15T02:59:34.713129+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278113+0000","last_change":"2023-11-15T02:59:34.272334+0000","last_active":"2023-11-15T02:59:34.278113+0000","last_peered":"2023-11-15T02:59:34.278113+0000","last_clean":"2023-11-15T02:59:34.278113+0000","last_became_active":"2023-11-15T02:59:34.272096+0000","last_became_peered":"2023-11-15T02:59:34.272096+0000","last_unstale":"2023-11-15T02:59:34.278113+0000","last_undegraded":"2023-11-15T02:59:34.278113+0000","last_fullsized":"2023-11-15T02:59:34.278113+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.686879+0000","last_change":"2023-11-15T02:59:34.686879+0000","last_active":"2023-11-15T02:59:34.686879+0000","last_peered":"2023-11-15T02:59:34.686879+0000","last_clean":"2023-11-15T02:59:34.686879+0000","last_became_active":"2023-11-15T02:59:34.685697+0000","last_became_peered":"2023-11-15T02:59:34.685697+0000","last_unstale":"2023-11-15T02:59:34.686879+0000","last_undegraded":"2023-11-15T02:59:34.686879+0000","last_fullsized":"2023-11-15T02:59:34.686879+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.275520+0000","last_change":"2023-11-15T02:59:34.270391+0000","last_active":"2023-11-15T02:59:34.275520+0000","last_peered":"2023-11-15T02:59:34.275520+0000","last_clean":"2023-11-15T02:59:34.275520+0000","last_became_active":"2023-11-15T02:59:34.270049+0000","last_became_peered":"2023-11-15T02:59:34.270049+0000","last_unstale":"2023-11-15T02:59:34.275520+0000","last_undegraded":"2023-11-15T02:59:34.275520+0000","last_fullsized":"2023-11-15T02:59:34.275520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.712401+0000","last_change":"2023-11-15T02:59:34.700113+0000","last_active":"2023-11-15T02:59:34.712401+0000","last_peered":"2023-11-15T02:59:34.712401+0000","last_clean":"2023-11-15T02:59:34.712401+0000","last_became_active":"2023-11-15T02:59:34.699860+0000","last_became_peered":"2023-11-15T02:59:34.699860+0000","last_unstale":"2023-11-15T02:59:34.712401+0000","last_undegraded":"2023-11-15T02:59:34.712401+0000","last_fullsized":"2023-11-15T02:59:34.712401+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.282770+0000","last_change":"2023-11-15T02:59:34.280324+0000","last_active":"2023-11-15T02:59:34.282770+0000","last_peered":"2023-11-15T02:59:34.282770+0000","last_clean":"2023-11-15T02:59:34.282770+0000","last_became_active":"2023-11-15T02:59:34.280003+0000","last_became_peered":"2023-11-15T02:59:34.280003+0000","last_unstale":"2023-11-15T02:59:34.282770+0000","last_undegraded":"2023-11-15T02:59:34.282770+0000","last_fullsized":"2023-11-15T02:59:34.282770+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.706837+0000","last_change":"2023-11-15T02:59:34.706837+0000","last_active":"2023-11-15T02:59:34.706837+0000","last_peered":"2023-11-15T02:59:34.706837+0000","last_clean":"2023-11-15T02:59:34.706837+0000","last_became_active":"2023-11-15T02:59:34.706595+0000","last_became_peered":"2023-11-15T02:59:34.706595+0000","last_unstale":"2023-11-15T02:59:34.706837+0000","last_undegraded":"2023-11-15T02:59:34.706837+0000","last_fullsized":"2023-11-15T02:59:34.706837+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278312+0000","last_change":"2023-11-15T02:59:34.274116+0000","last_active":"2023-11-15T02:59:34.278312+0000","last_peered":"2023-11-15T02:59:34.278312+0000","last_clean":"2023-11-15T02:59:34.278312+0000","last_became_active":"2023-11-15T02:59:34.273108+0000","last_became_peered":"2023-11-15T02:59:34.273108+0000","last_unstale":"2023-11-15T02:59:34.278312+0000","last_undegraded":"2023-11-15T02:59:34.278312+0000","last_fullsized":"2023-11-15T02:59:34.278312+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.713931+0000","last_change":"2023-11-15T02:59:34.713931+0000","last_active":"2023-11-15T02:59:34.713931+0000","last_peered":"2023-11-15T02:59:34.713931+0000","last_clean":"2023-11-15T02:59:34.713931+0000","last_became_active":"2023-11-15T02:59:34.713640+0000","last_became_peered":"2023-11-15T02:59:34.713640+0000","last_unstale":"2023-11-15T02:59:34.713931+0000","last_undegraded":"2023-11-15T02:59:34.713931+0000","last_fullsized":"2023-11-15T02:59:34.713931+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"60'3","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.704814+0000","last_change":"2023-11-15T02:59:34.273818+0000","last_active":"2023-11-15T02:59:34.704814+0000","last_peered":"2023-11-15T02:59:34.704814+0000","last_clean":"2023-11-15T02:59:34.704814+0000","last_became_active":"2023-11-15T02:59:34.273463+0000","last_became_peered":"2023-11-15T02:59:34.273463+0000","last_unstale":"2023-11-15T02:59:34.704814+0000","last_undegraded":"2023-11-15T02:59:34.704814+0000","last_fullsized":"2023-11-15T02:59:34.704814+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.691994+0000","last_change":"2023-11-15T02:59:34.691994+0000","last_active":"2023-11-15T02:59:34.691994+0000","last_peered":"2023-11-15T02:59:34.691994+0000","last_clean":"2023-11-15T02:59:34.691994+0000","last_became_active":"2023-11-15T02:59:34.691573+0000","last_became_peered":"2023-11-15T02:59:34.691573+0000","last_unstale":"2023-11-15T02:59:34.691994+0000","last_undegraded":"2023-11-15T02:59:34.691994+0000","last_fullsized":"2023-11-15T02:59:34.691994+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.714362+0000","last_change":"2023-11-15T02:59:34.273189+0000","last_active":"2023-11-15T02:59:34.714362+0000","last_peered":"2023-11-15T02:59:34.714362+0000","last_clean":"2023-11-15T02:59:34.714362+0000","last_became_active":"2023-11-15T02:59:34.272734+0000","last_became_peered":"2023-11-15T02:59:34.272734+0000","last_unstale":"2023-11-15T02:59:34.714362+0000","last_undegraded":"2023-11-15T02:59:34.714362+0000","last_fullsized":"2023-11-15T02:59:34.714362+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.708893+0000","last_change":"2023-11-15T02:59:34.708893+0000","last_active":"2023-11-15T02:59:34.708893+0000","last_peered":"2023-11-15T02:59:34.708893+0000","last_clean":"2023-11-15T02:59:34.708893+0000","last_became_active":"2023-11-15T02:59:34.708623+0000","last_became_peered":"2023-11-15T02:59:34.708623+0000","last_unstale":"2023-11-15T02:59:34.708893+0000","last_undegraded":"2023-11-15T02:59:34.708893+0000","last_fullsized":"2023-11-15T02:59:34.708893+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.688236+0000","last_change":"2023-11-15T02:59:34.270907+0000","last_active":"2023-11-15T02:59:34.688236+0000","last_peered":"2023-11-15T02:59:34.688236+0000","last_clean":"2023-11-15T02:59:34.688236+0000","last_became_active":"2023-11-15T02:59:34.270561+0000","last_became_peered":"2023-11-15T02:59:34.270561+0000","last_unstale":"2023-11-15T02:59:34.688236+0000","last_undegraded":"2023-11-15T02:59:34.688236+0000","last_fullsized":"2023-11-15T02:59:34.688236+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.712809+0000","last_change":"2023-11-15T02:59:34.705948+0000","last_active":"2023-11-15T02:59:34.712809+0000","last_peered":"2023-11-15T02:59:34.712809+0000","last_clean":"2023-11-15T02:59:34.712809+0000","last_became_active":"2023-11-15T02:59:34.705666+0000","last_became_peered":"2023-11-15T02:59:34.705666+0000","last_unstale":"2023-11-15T02:59:34.712809+0000","last_undegraded":"2023-11-15T02:59:34.712809+0000","last_fullsized":"2023-11-15T02:59:34.712809+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.697233+0000","last_change":"2023-11-15T02:59:34.277797+0000","last_active":"2023-11-15T02:59:34.697233+0000","last_peered":"2023-11-15T02:59:34.697233+0000","last_clean":"2023-11-15T02:59:34.697233+0000","last_became_active":"2023-11-15T02:59:34.277536+0000","last_became_peered":"2023-11-15T02:59:34.277536+0000","last_unstale":"2023-11-15T02:59:34.697233+0000","last_undegraded":"2023-11-15T02:59:34.697233+0000","last_fullsized":"2023-11-15T02:59:34.697233+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.719857+0000","last_change":"2023-11-15T02:59:34.719857+0000","last_active":"2023-11-15T02:59:34.719857+0000","last_peered":"2023-11-15T02:59:34.719857+0000","last_clean":"2023-11-15T02:59:34.719857+0000","last_became_active":"2023-11-15T02:59:34.719591+0000","last_became_peered":"2023-11-15T02:59:34.719591+0000","last_unstale":"2023-11-15T02:59:34.719857+0000","last_undegraded":"2023-11-15T02:59:34.719857+0000","last_fullsized":"2023-11-15T02:59:34.719857+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.279642+0000","last_change":"2023-11-15T02:59:34.276260+0000","last_active":"2023-11-15T02:59:34.279642+0000","last_peered":"2023-11-15T02:59:34.279642+0000","last_clean":"2023-11-15T02:59:34.279642+0000","last_became_active":"2023-11-15T02:59:34.275758+0000","last_became_peered":"2023-11-15T02:59:34.275758+0000","last_unstale":"2023-11-15T02:59:34.279642+0000","last_undegraded":"2023-11-15T02:59:34.279642+0000","last_fullsized":"2023-11-15T02:59:34.279642+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.714570+0000","last_change":"2023-11-15T02:59:34.709342+0000","last_active":"2023-11-15T02:59:34.714570+0000","last_peered":"2023-11-15T02:59:34.714570+0000","last_clean":"2023-11-15T02:59:34.714570+0000","last_became_active":"2023-11-15T02:59:34.708937+0000","last_became_peered":"2023-11-15T02:59:34.708937+0000","last_unstale":"2023-11-15T02:59:34.714570+0000","last_undegraded":"2023-11-15T02:59:34.714570+0000","last_fullsized":"2023-11-15T02:59:34.714570+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"60'4","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.722402+0000","last_change":"2023-11-15T02:59:34.269037+0000","last_active":"2023-11-15T02:59:34.722402+0000","last_peered":"2023-11-15T02:59:34.722402+0000","last_clean":"2023-11-15T02:59:34.722402+0000","last_became_active":"2023-11-15T02:59:34.268123+0000","last_became_peered":"2023-11-15T02:59:34.268123+0000","last_unstale":"2023-11-15T02:59:34.722402+0000","last_undegraded":"2023-11-15T02:59:34.722402+0000","last_fullsized":"2023-11-15T02:59:34.722402+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.715434+0000","last_change":"2023-11-15T02:59:34.715434+0000","last_active":"2023-11-15T02:59:34.715434+0000","last_peered":"2023-11-15T02:59:34.715434+0000","last_clean":"2023-11-15T02:59:34.715434+0000","last_became_active":"2023-11-15T02:59:34.715198+0000","last_became_peered":"2023-11-15T02:59:34.715198+0000","last_unstale":"2023-11-15T02:59:34.715434+0000","last_undegraded":"2023-11-15T02:59:34.715434+0000","last_fullsized":"2023-11-15T02:59:34.715434+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.722345+0000","last_change":"2023-11-15T02:59:34.271705+0000","last_active":"2023-11-15T02:59:34.722345+0000","last_peered":"2023-11-15T02:59:34.722345+0000","last_clean":"2023-11-15T02:59:34.722345+0000","last_became_active":"2023-11-15T02:59:34.271393+0000","last_became_peered":"2023-11-15T02:59:34.271393+0000","last_unstale":"2023-11-15T02:59:34.722345+0000","last_undegraded":"2023-11-15T02:59:34.722345+0000","last_fullsized":"2023-11-15T02:59:34.722345+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.705591+0000","last_change":"2023-11-15T02:59:34.705591+0000","last_active":"2023-11-15T02:59:34.705591+0000","last_peered":"2023-11-15T02:59:34.705591+0000","last_clean":"2023-11-15T02:59:34.705591+0000","last_became_active":"2023-11-15T02:59:34.703826+0000","last_became_peered":"2023-11-15T02:59:34.703826+0000","last_unstale":"2023-11-15T02:59:34.705591+0000","last_undegraded":"2023-11-15T02:59:34.705591+0000","last_fullsized":"2023-11-15T02:59:34.705591+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.702284+0000","last_change":"2023-11-15T02:59:34.266622+0000","last_active":"2023-11-15T02:59:34.702284+0000","last_peered":"2023-11-15T02:59:34.702284+0000","last_clean":"2023-11-15T02:59:34.702284+0000","last_became_active":"2023-11-15T02:59:34.266349+0000","last_became_peered":"2023-11-15T02:59:34.266349+0000","last_unstale":"2023-11-15T02:59:34.702284+0000","last_undegraded":"2023-11-15T02:59:34.702284+0000","last_fullsized":"2023-11-15T02:59:34.702284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716077+0000","last_change":"2023-11-15T02:59:34.709700+0000","last_active":"2023-11-15T02:59:34.716077+0000","last_peered":"2023-11-15T02:59:34.716077+0000","last_clean":"2023-11-15T02:59:34.716077+0000","last_became_active":"2023-11-15T02:59:34.708353+0000","last_became_peered":"2023-11-15T02:59:34.708353+0000","last_unstale":"2023-11-15T02:59:34.716077+0000","last_undegraded":"2023-11-15T02:59:34.716077+0000","last_fullsized":"2023-11-15T02:59:34.716077+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.273908+0000","last_change":"2023-11-15T02:59:34.263759+0000","last_active":"2023-11-15T02:59:34.273908+0000","last_peered":"2023-11-15T02:59:34.273908+0000","last_clean":"2023-11-15T02:59:34.273908+0000","last_became_active":"2023-11-15T02:59:34.263333+0000","last_became_peered":"2023-11-15T02:59:34.263333+0000","last_unstale":"2023-11-15T02:59:34.273908+0000","last_undegraded":"2023-11-15T02:59:34.273908+0000","last_fullsized":"2023-11-15T02:59:34.273908+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.700382+0000","last_change":"2023-11-15T02:59:34.700382+0000","last_active":"2023-11-15T02:59:34.700382+0000","last_peered":"2023-11-15T02:59:34.700382+0000","last_clean":"2023-11-15T02:59:34.700382+0000","last_became_active":"2023-11-15T02:59:34.700091+0000","last_became_peered":"2023-11-15T02:59:34.700091+0000","last_unstale":"2023-11-15T02:59:34.700382+0000","last_undegraded":"2023-11-15T02:59:34.700382+0000","last_fullsized":"2023-11-15T02:59:34.700382+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.687840+0000","last_change":"2023-11-15T02:59:34.273003+0000","last_active":"2023-11-15T02:59:34.687840+0000","last_peered":"2023-11-15T02:59:34.687840+0000","last_clean":"2023-11-15T02:59:34.687840+0000","last_became_active":"2023-11-15T02:59:34.272361+0000","last_became_peered":"2023-11-15T02:59:34.272361+0000","last_unstale":"2023-11-15T02:59:34.687840+0000","last_undegraded":"2023-11-15T02:59:34.687840+0000","last_fullsized":"2023-11-15T02:59:34.687840+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.697102+0000","last_change":"2023-11-15T02:59:34.697102+0000","last_active":"2023-11-15T02:59:34.697102+0000","last_peered":"2023-11-15T02:59:34.697102+0000","last_clean":"2023-11-15T02:59:34.697102+0000","last_became_active":"2023-11-15T02:59:34.696095+0000","last_became_peered":"2023-11-15T02:59:34.696095+0000","last_unstale":"2023-11-15T02:59:34.697102+0000","last_undegraded":"2023-11-15T02:59:34.697102+0000","last_fullsized":"2023-11-15T02:59:34.697102+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.279613+0000","last_change":"2023-11-15T02:59:34.275904+0000","last_active":"2023-11-15T02:59:34.279613+0000","last_peered":"2023-11-15T02:59:34.279613+0000","last_clean":"2023-11-15T02:59:34.279613+0000","last_became_active":"2023-11-15T02:59:34.275561+0000","last_became_peered":"2023-11-15T02:59:34.275561+0000","last_unstale":"2023-11-15T02:59:34.279613+0000","last_undegraded":"2023-11-15T02:59:34.279613+0000","last_fullsized":"2023-11-15T02:59:34.279613+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.710944+0000","last_change":"2023-11-15T02:59:34.710944+0000","last_active":"2023-11-15T02:59:34.710944+0000","last_peered":"2023-11-15T02:59:34.710944+0000","last_clean":"2023-11-15T02:59:34.710944+0000","last_became_active":"2023-11-15T02:59:34.710640+0000","last_became_peered":"2023-11-15T02:59:34.710640+0000","last_unstale":"2023-11-15T02:59:34.710944+0000","last_undegraded":"2023-11-15T02:59:34.710944+0000","last_fullsized":"2023-11-15T02:59:34.710944+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.280896+0000","last_change":"2023-11-15T02:59:34.275717+0000","last_active":"2023-11-15T02:59:34.280896+0000","last_peered":"2023-11-15T02:59:34.280896+0000","last_clean":"2023-11-15T02:59:34.280896+0000","last_became_active":"2023-11-15T02:59:34.274314+0000","last_became_peered":"2023-11-15T02:59:34.274314+0000","last_unstale":"2023-11-15T02:59:34.280896+0000","last_undegraded":"2023-11-15T02:59:34.280896+0000","last_fullsized":"2023-11-15T02:59:34.280896+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.714395+0000","last_change":"2023-11-15T02:59:34.708647+0000","last_active":"2023-11-15T02:59:34.714395+0000","last_peered":"2023-11-15T02:59:34.714395+0000","last_clean":"2023-11-15T02:59:34.714395+0000","last_became_active":"2023-11-15T02:59:34.708182+0000","last_became_peered":"2023-11-15T02:59:34.708182+0000","last_unstale":"2023-11-15T02:59:34.714395+0000","last_undegraded":"2023-11-15T02:59:34.714395+0000","last_fullsized":"2023-11-15T02:59:34.714395+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.278096+0000","last_change":"2023-11-15T02:59:34.271643+0000","last_active":"2023-11-15T02:59:34.278096+0000","last_peered":"2023-11-15T02:59:34.278096+0000","last_clean":"2023-11-15T02:59:34.278096+0000","last_became_active":"2023-11-15T02:59:34.271375+0000","last_became_peered":"2023-11-15T02:59:34.271375+0000","last_unstale":"2023-11-15T02:59:34.278096+0000","last_undegraded":"2023-11-15T02:59:34.278096+0000","last_fullsized":"2023-11-15T02:59:34.278096+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.711599+0000","last_change":"2023-11-15T02:59:34.711599+0000","last_active":"2023-11-15T02:59:34.711599+0000","last_peered":"2023-11-15T02:59:34.711599+0000","last_clean":"2023-11-15T02:59:34.711599+0000","last_became_active":"2023-11-15T02:59:34.711325+0000","last_became_peered":"2023-11-15T02:59:34.711325+0000","last_unstale":"2023-11-15T02:59:34.711599+0000","last_undegraded":"2023-11-15T02:59:34.711599+0000","last_fullsized":"2023-11-15T02:59:34.711599+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.685117+0000","last_change":"2023-11-15T02:59:34.269929+0000","last_active":"2023-11-15T02:59:34.685117+0000","last_peered":"2023-11-15T02:59:34.685117+0000","last_clean":"2023-11-15T02:59:34.685117+0000","last_became_active":"2023-11-15T02:59:34.269560+0000","last_became_peered":"2023-11-15T02:59:34.269560+0000","last_unstale":"2023-11-15T02:59:34.685117+0000","last_undegraded":"2023-11-15T02:59:34.685117+0000","last_fullsized":"2023-11-15T02:59:34.685117+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.699657+0000","last_change":"2023-11-15T02:59:34.699657+0000","last_active":"2023-11-15T02:59:34.699657+0000","last_peered":"2023-11-15T02:59:34.699657+0000","last_clean":"2023-11-15T02:59:34.699657+0000","last_became_active":"2023-11-15T02:59:34.699371+0000","last_became_peered":"2023-11-15T02:59:34.699371+0000","last_unstale":"2023-11-15T02:59:34.699657+0000","last_undegraded":"2023-11-15T02:59:34.699657+0000","last_fullsized":"2023-11-15T02:59:34.699657+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.272131+0000","last_change":"2023-11-15T02:59:34.264863+0000","last_active":"2023-11-15T02:59:34.272131+0000","last_peered":"2023-11-15T02:59:34.272131+0000","last_clean":"2023-11-15T02:59:34.272131+0000","last_became_active":"2023-11-15T02:59:34.264228+0000","last_became_peered":"2023-11-15T02:59:34.264228+0000","last_unstale":"2023-11-15T02:59:34.272131+0000","last_undegraded":"2023-11-15T02:59:34.272131+0000","last_fullsized":"2023-11-15T02:59:34.272131+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.252284+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T02:59:34.252284+0000","last_peered":"2023-11-15T02:59:34.252284+0000","last_clean":"2023-11-15T02:59:34.252284+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T02:59:34.252284+0000","last_undegraded":"2023-11-15T02:59:34.252284+0000","last_fullsized":"2023-11-15T02:59:34.252284+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.694895+0000","last_change":"2023-11-15T02:59:34.694895+0000","last_active":"2023-11-15T02:59:34.694895+0000","last_peered":"2023-11-15T02:59:34.694895+0000","last_clean":"2023-11-15T02:59:34.694895+0000","last_became_active":"2023-11-15T02:59:34.694592+0000","last_became_peered":"2023-11-15T02:59:34.694592+0000","last_unstale":"2023-11-15T02:59:34.694895+0000","last_undegraded":"2023-11-15T02:59:34.694895+0000","last_fullsized":"2023-11-15T02:59:34.694895+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.282767+0000","last_change":"2023-11-15T02:59:34.280250+0000","last_active":"2023-11-15T02:59:34.282767+0000","last_peered":"2023-11-15T02:59:34.282767+0000","last_clean":"2023-11-15T02:59:34.282767+0000","last_became_active":"2023-11-15T02:59:34.279961+0000","last_became_peered":"2023-11-15T02:59:34.279961+0000","last_unstale":"2023-11-15T02:59:34.282767+0000","last_undegraded":"2023-11-15T02:59:34.282767+0000","last_fullsized":"2023-11-15T02:59:34.282767+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.688454+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T02:59:34.688454+0000","last_peered":"2023-11-15T02:59:34.688454+0000","last_clean":"2023-11-15T02:59:34.688454+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T02:59:34.688454+0000","last_undegraded":"2023-11-15T02:59:34.688454+0000","last_fullsized":"2023-11-15T02:59:34.688454+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.720662+0000","last_change":"2023-11-15T02:59:34.720662+0000","last_active":"2023-11-15T02:59:34.720662+0000","last_peered":"2023-11-15T02:59:34.720662+0000","last_clean":"2023-11-15T02:59:34.720662+0000","last_became_active":"2023-11-15T02:59:34.713450+0000","last_became_peered":"2023-11-15T02:59:34.713450+0000","last_unstale":"2023-11-15T02:59:34.720662+0000","last_undegraded":"2023-11-15T02:59:34.720662+0000","last_fullsized":"2023-11-15T02:59:34.720662+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"60'2","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716637+0000","last_change":"2023-11-15T02:59:34.272609+0000","last_active":"2023-11-15T02:59:34.716637+0000","last_peered":"2023-11-15T02:59:34.716637+0000","last_clean":"2023-11-15T02:59:34.716637+0000","last_became_active":"2023-11-15T02:59:34.272212+0000","last_became_peered":"2023-11-15T02:59:34.272212+0000","last_unstale":"2023-11-15T02:59:34.716637+0000","last_undegraded":"2023-11-15T02:59:34.716637+0000","last_fullsized":"2023-11-15T02:59:34.716637+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.257610+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T02:59:34.257610+0000","last_peered":"2023-11-15T02:59:34.257610+0000","last_clean":"2023-11-15T02:59:34.257610+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T02:59:34.257610+0000","last_undegraded":"2023-11-15T02:59:34.257610+0000","last_fullsized":"2023-11-15T02:59:34.257610+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.711825+0000","last_change":"2023-11-15T02:59:34.711825+0000","last_active":"2023-11-15T02:59:34.711825+0000","last_peered":"2023-11-15T02:59:34.711825+0000","last_clean":"2023-11-15T02:59:34.711825+0000","last_became_active":"2023-11-15T02:59:34.711551+0000","last_became_peered":"2023-11-15T02:59:34.711551+0000","last_unstale":"2023-11-15T02:59:34.711825+0000","last_undegraded":"2023-11-15T02:59:34.711825+0000","last_fullsized":"2023-11-15T02:59:34.711825+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716350+0000","last_change":"2023-11-15T02:59:34.273403+0000","last_active":"2023-11-15T02:59:34.716350+0000","last_peered":"2023-11-15T02:59:34.716350+0000","last_clean":"2023-11-15T02:59:34.716350+0000","last_became_active":"2023-11-15T02:59:34.271771+0000","last_became_peered":"2023-11-15T02:59:34.271771+0000","last_unstale":"2023-11-15T02:59:34.716350+0000","last_undegraded":"2023-11-15T02:59:34.716350+0000","last_fullsized":"2023-11-15T02:59:34.716350+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.687917+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T02:59:34.687917+0000","last_peered":"2023-11-15T02:59:34.687917+0000","last_clean":"2023-11-15T02:59:34.687917+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T02:59:34.687917+0000","last_undegraded":"2023-11-15T02:59:34.687917+0000","last_fullsized":"2023-11-15T02:59:34.687917+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.712699+0000","last_change":"2023-11-15T02:59:34.712699+0000","last_active":"2023-11-15T02:59:34.712699+0000","last_peered":"2023-11-15T02:59:34.712699+0000","last_clean":"2023-11-15T02:59:34.712699+0000","last_became_active":"2023-11-15T02:59:34.703686+0000","last_became_peered":"2023-11-15T02:59:34.703686+0000","last_unstale":"2023-11-15T02:59:34.712699+0000","last_undegraded":"2023-11-15T02:59:34.712699+0000","last_fullsized":"2023-11-15T02:59:34.712699+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.277673+0000","last_change":"2023-11-15T02:59:34.265955+0000","last_active":"2023-11-15T02:59:34.277673+0000","last_peered":"2023-11-15T02:59:34.277673+0000","last_clean":"2023-11-15T02:59:34.277673+0000","last_became_active":"2023-11-15T02:59:34.265681+0000","last_became_peered":"2023-11-15T02:59:34.265681+0000","last_unstale":"2023-11-15T02:59:34.277673+0000","last_undegraded":"2023-11-15T02:59:34.277673+0000","last_fullsized":"2023-11-15T02:59:34.277673+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":53,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.697416+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T02:59:34.697416+0000","last_peered":"2023-11-15T02:59:34.697416+0000","last_clean":"2023-11-15T02:59:34.697416+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T02:59:34.697416+0000","last_undegraded":"2023-11-15T02:59:34.697416+0000","last_fullsized":"2023-11-15T02:59:34.697416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.712838+0000","last_change":"2023-11-15T02:59:34.707313+0000","last_active":"2023-11-15T02:59:34.712838+0000","last_peered":"2023-11-15T02:59:34.712838+0000","last_clean":"2023-11-15T02:59:34.712838+0000","last_became_active":"2023-11-15T02:59:34.701575+0000","last_became_peered":"2023-11-15T02:59:34.701575+0000","last_unstale":"2023-11-15T02:59:34.712838+0000","last_undegraded":"2023-11-15T02:59:34.712838+0000","last_fullsized":"2023-11-15T02:59:34.712838+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.697334+0000","last_change":"2023-11-15T02:59:34.274214+0000","last_active":"2023-11-15T02:59:34.697334+0000","last_peered":"2023-11-15T02:59:34.697334+0000","last_clean":"2023-11-15T02:59:34.697334+0000","last_became_active":"2023-11-15T02:59:34.273920+0000","last_became_peered":"2023-11-15T02:59:34.273920+0000","last_unstale":"2023-11-15T02:59:34.697334+0000","last_undegraded":"2023-11-15T02:59:34.697334+0000","last_fullsized":"2023-11-15T02:59:34.697334+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.253156+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T02:59:34.253156+0000","last_peered":"2023-11-15T02:59:34.253156+0000","last_clean":"2023-11-15T02:59:34.253156+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T02:59:34.253156+0000","last_undegraded":"2023-11-15T02:59:34.253156+0000","last_fullsized":"2023-11-15T02:59:34.253156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.704325+0000","last_change":"2023-11-15T02:59:34.704325+0000","last_active":"2023-11-15T02:59:34.704325+0000","last_peered":"2023-11-15T02:59:34.704325+0000","last_clean":"2023-11-15T02:59:34.704325+0000","last_became_active":"2023-11-15T02:59:34.703365+0000","last_became_peered":"2023-11-15T02:59:34.703365+0000","last_unstale":"2023-11-15T02:59:34.704325+0000","last_undegraded":"2023-11-15T02:59:34.704325+0000","last_fullsized":"2023-11-15T02:59:34.704325+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"60'1","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.691558+0000","last_change":"2023-11-15T02:59:34.265022+0000","last_active":"2023-11-15T02:59:34.691558+0000","last_peered":"2023-11-15T02:59:34.691558+0000","last_clean":"2023-11-15T02:59:34.691558+0000","last_became_active":"2023-11-15T02:59:34.264125+0000","last_became_peered":"2023-11-15T02:59:34.264125+0000","last_unstale":"2023-11-15T02:59:34.691558+0000","last_undegraded":"2023-11-15T02:59:34.691558+0000","last_fullsized":"2023-11-15T02:59:34.691558+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.254819+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T02:59:34.254819+0000","last_peered":"2023-11-15T02:59:34.254819+0000","last_clean":"2023-11-15T02:59:34.254819+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T02:59:34.254819+0000","last_undegraded":"2023-11-15T02:59:34.254819+0000","last_fullsized":"2023-11-15T02:59:34.254819+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":8,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.697027+0000","last_change":"2023-11-15T02:59:34.697027+0000","last_active":"2023-11-15T02:59:34.697027+0000","last_peered":"2023-11-15T02:59:34.697027+0000","last_clean":"2023-11-15T02:59:34.697027+0000","last_became_active":"2023-11-15T02:59:34.696199+0000","last_became_peered":"2023-11-15T02:59:34.696199+0000","last_unstale":"2023-11-15T02:59:34.697027+0000","last_undegraded":"2023-11-15T02:59:34.697027+0000","last_fullsized":"2023-11-15T02:59:34.697027+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.283700+0000","last_change":"2023-11-15T02:59:34.279512+0000","last_active":"2023-11-15T02:59:34.283700+0000","last_peered":"2023-11-15T02:59:34.283700+0000","last_clean":"2023-11-15T02:59:34.283700+0000","last_became_active":"2023-11-15T02:59:34.278967+0000","last_became_peered":"2023-11-15T02:59:34.278967+0000","last_unstale":"2023-11-15T02:59:34.283700+0000","last_undegraded":"2023-11-15T02:59:34.283700+0000","last_fullsized":"2023-11-15T02:59:34.283700+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":53,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.700072+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T02:59:34.700072+0000","last_peered":"2023-11-15T02:59:34.700072+0000","last_clean":"2023-11-15T02:59:34.700072+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T02:59:34.700072+0000","last_undegraded":"2023-11-15T02:59:34.700072+0000","last_fullsized":"2023-11-15T02:59:34.700072+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.254906+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T02:59:34.254906+0000","last_peered":"2023-11-15T02:59:34.254906+0000","last_clean":"2023-11-15T02:59:34.254906+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T02:59:34.254906+0000","last_undegraded":"2023-11-15T02:59:34.254906+0000","last_fullsized":"2023-11-15T02:59:34.254906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542180,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":704512,"data_stored":312535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.1679999999999999,"15min":1.1859999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.458,"5min":3.952,"15min":3.952},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.046,"5min":1.1799999999999999,"15min":1.202},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4419999999999999,"5min":2.5,"15min":2.5},"last":1.016}]},{"osd":1,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.137,"15min":1.145},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.3740000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1739999999999999,"15min":1.1779999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.415,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.3049999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2150000000000001,"15min":1.2330000000000001},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.458,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2949999999999999,"15min":1.3069999999999999},"min":{"1min":0.97899999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.5149999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":0.92900000000000005}]},{"osd":3,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.3400000000000001,"15min":1.3740000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.381,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.175,"5min":1.2669999999999999,"15min":1.282},"min":{"1min":0.90600000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.351,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.093}]},{"osd":4,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.2689999999999999,"15min":1.3},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.4359999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":1.117},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.448,"15min":1.4990000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.5029999999999999,"5min":4.226,"15min":4.226},"last":0.87}]},{"osd":5,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.329,"15min":1.343},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.556,"5min":2.024,"15min":2.024},"last":1.28},{"interface":"front","average":{"1min":1.234,"5min":1.4399999999999999,"15min":1.474},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.653,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.216}]},{"osd":6,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.5620000000000001,"15min":1.601},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":1.5660000000000001,"5min":3.2789999999999999,"15min":3.2789999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.591,"15min":1.645},"min":{"1min":1.123,"5min":1.123,"15min":1.123},"max":{"1min":1.4399999999999999,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.98099999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542180,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297800,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074040,"statfs":{"total":96636764160,"available":96331816960,"internally_reserved":0,"allocated":720896,"data_stored":319748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.125,"15min":1.133},"min":{"1min":0.69799999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.353,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.0469999999999999,"15min":1.0409999999999999},"min":{"1min":0.72499999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.484,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.93899999999999995}]},{"osd":1,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0920000000000001,"15min":1.0840000000000001},"min":{"1min":0.876,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.415,"5min":1.623,"15min":1.623},"last":1.03},{"interface":"front","average":{"1min":1.173,"5min":1.169,"15min":1.169},"min":{"1min":0.80800000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.609,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.97199999999999998}]},{"osd":2,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.1870000000000001,"15min":1.1779999999999999},"min":{"1min":0.84399999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.496,"5min":2.621,"15min":2.621},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.0960000000000001,"15min":1.0840000000000001},"min":{"1min":0.91700000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.425,"5min":1.5,"15min":1.5},"last":1.2130000000000001}]},{"osd":3,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.167,"15min":1.157},"min":{"1min":1.0029999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.492,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.26,"15min":1.2629999999999999},"min":{"1min":0.99299999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.6850000000000001,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.3740000000000001}]},{"osd":4,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.173,"15min":1.1910000000000001},"min":{"1min":0.81399999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.4099999999999999,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":1.391},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.95599999999999996,"5min":0.625,"15min":0.625},"max":{"1min":1.4850000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.536}]},{"osd":5,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.417,"15min":1.411},"min":{"1min":1.155,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.772,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.3109999999999999,"15min":1.319},"min":{"1min":0.877,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6599999999999999,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":1.169}]},{"osd":7,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2809999999999999,"15min":1.3009999999999999},"min":{"1min":0.82199999999999995,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.395,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.522},{"interface":"front","average":{"1min":1.401,"5min":1.448,"15min":1.456},"min":{"1min":1.046,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8540000000000001,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":1.637}]}]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297800,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074040,"statfs":{"total":96636764160,"available":96331816960,"internally_reserved":0,"allocated":720896,"data_stored":319804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.107,"15min":1.1120000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.516,"5min":1.532,"15min":1.532},"last":1.125},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0069999999999999,"15min":1.0029999999999999},"min":{"1min":0.80700000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":1.28}]},{"osd":1,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2250000000000001,"15min":1.2370000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4770000000000001,"5min":2.617,"15min":2.617},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.115,"5min":1.1559999999999999,"15min":1.163},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.425,"5min":1.5,"15min":1.5},"last":1.2410000000000001}]},{"osd":2,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.1519999999999999,"15min":1.141},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.462,"5min":1.909,"15min":1.909},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1599999999999999,"15min":1.1639999999999999},"min":{"1min":0.88800000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.399,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.0700000000000001}]},{"osd":3,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1519999999999999,"15min":1.1599999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.462,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.345},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.294,"15min":1.3},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.5840000000000001,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":1.169}]},{"osd":5,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2430000000000001,"15min":1.2529999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5780000000000001,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.2290000000000001,"15min":1.2350000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.5049999999999999,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.6639999999999999}]},{"osd":6,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.333,"15min":1.343},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6619999999999999,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":1.456},{"interface":"front","average":{"1min":1.296,"5min":1.3979999999999999,"15min":1.415},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.7450000000000001,"5min":2.173,"15min":2.173},"last":1.4670000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.369,"15min":1.381},"min":{"1min":1.069,"5min":0.87,"15min":0.87},"max":{"1min":1.5820000000000001,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.4350000000000001,"15min":1.4450000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.587,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":1.415}]}]},{"osd":3,"up_from":14,"seq":60129542181,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":704512,"data_stored":312535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.77400000000000002,"15min":0.75900000000000001},"min":{"1min":0.64900000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.629},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.84199999999999997,"15min":0.82899999999999996},"min":{"1min":0.73599999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1779999999999999,"5min":1.26,"15min":1.26},"last":0.70699999999999996}]},{"osd":1,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.90000000000000002,"15min":0.90100000000000002},"min":{"1min":0.621,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.97999999999999998,"15min":0.96999999999999997},"min":{"1min":0.83299999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2749999999999999,"5min":1.375,"15min":1.375},"last":0.55800000000000005}]},{"osd":2,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97699999999999998,"15min":0.96799999999999997},"min":{"1min":0.76700000000000002,"5min":0.63,"15min":0.63},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0649999999999999,"15min":1.052},"min":{"1min":0.85899999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.96699999999999997}]},{"osd":4,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1859999999999999,"15min":1.1799999999999999},"min":{"1min":0.97899999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.456,"5min":1.837,"15min":1.837},"last":1.202},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.22,"15min":1.226},"min":{"1min":1.0249999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.492,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.0189999999999999}]},{"osd":5,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.27,"15min":1.2669999999999999},"min":{"1min":0.996,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.6459999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.05},{"interface":"front","average":{"1min":1.345,"5min":1.2789999999999999,"15min":1.268},"min":{"1min":1.01,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.89900000000000002}]},{"osd":6,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3120000000000001,"15min":1.306},"min":{"1min":1.1060000000000001,"5min":1.048,"15min":1.048},"max":{"1min":1.552,"5min":1.794,"15min":1.794},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.3580000000000001,"15min":1.3520000000000001},"min":{"1min":1.129,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":1.647,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.163}]},{"osd":7,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.3919999999999999,"15min":1.3899999999999999},"min":{"1min":1.083,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.365,"15min":1.357},"min":{"1min":1.1819999999999999,"5min":1.085,"15min":1.085},"max":{"1min":1.665,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.1850000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542181,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297780,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074060,"statfs":{"total":96636764160,"available":96331837440,"internally_reserved":0,"allocated":700416,"data_stored":311266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76700000000000002,"15min":0.76400000000000001},"min":{"1min":0.61699999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88700000000000001,"15min":0.88200000000000001},"min":{"1min":0.68300000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.67400000000000004}]},{"osd":1,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89600000000000002,"15min":0.89200000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3120000000000001,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.96099999999999997,"15min":0.94899999999999995},"min":{"1min":0.83499999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.93600000000000005}]},{"osd":3,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.002,"15min":1},"min":{"1min":0.79200000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.387,"5min":1.472,"15min":1.472},"last":0.629},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1259999999999999,"15min":1.1200000000000001},"min":{"1min":0.93000000000000005,"5min":0.622,"15min":0.622},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.81899999999999995}]},{"osd":4,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.208,"15min":1.21},"min":{"1min":1.0149999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.48,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.111},{"interface":"front","average":{"1min":1.286,"5min":1.238,"15min":1.23},"min":{"1min":1.0760000000000001,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.5429999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.071}]},{"osd":5,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.262,"15min":1.264},"min":{"1min":1.0149999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":1.613,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2989999999999999,"15min":1.2969999999999999},"min":{"1min":1.0700000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.6890000000000001,"5min":1.857,"15min":1.857},"last":1}]},{"osd":6,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.367,"15min":1.373},"min":{"1min":1.056,"5min":1.056,"15min":1.056},"max":{"1min":1.72,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.347,"15min":1.353},"min":{"1min":1.046,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":1.573,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.1599999999999999}]},{"osd":7,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3939999999999999,"15min":1.391},"min":{"1min":1.1719999999999999,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":1.8069999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.3799999999999999,"15min":1.3919999999999999},"min":{"1min":1.097,"5min":1.097,"15min":1.097},"max":{"1min":1.5529999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.96799999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542181,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":704512,"data_stored":312535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.84899999999999998,"15min":0.86899999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1990000000000001,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":4.3630000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.89700000000000002,"15min":0.89600000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1619999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":6.157}]},{"osd":2,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.99399999999999999,"15min":1.0129999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.147,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":4.1699999999999999},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.99199999999999999,"15min":0.98799999999999999},"min":{"1min":0.68000000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.2529999999999999,"5min":1.847,"15min":1.847},"last":6.8369999999999997}]},{"osd":3,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.96499999999999997,"15min":0.96199999999999997},"min":{"1min":0.76600000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.401,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":3.5070000000000001},{"interface":"front","average":{"1min":1.052,"5min":1.157,"15min":1.175},"min":{"1min":0.68899999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2210000000000001,"5min":3.54,"15min":3.54},"last":5.2149999999999999}]},{"osd":4,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2490000000000001,"15min":1.2470000000000001},"min":{"1min":1.0720000000000001,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.6319999999999999,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":6.2000000000000002},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.175,"15min":1.175},"min":{"1min":1.006,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.4850000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":3.577}]},{"osd":5,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2170000000000001,"15min":1.218},"min":{"1min":0.98899999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.393,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":3.548},{"interface":"front","average":{"1min":1.24,"5min":1.3420000000000001,"15min":1.359},"min":{"1min":1.083,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.375,"5min":3.6760000000000002,"15min":3.6760000000000002},"last":4.4080000000000004}]},{"osd":6,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.411,"15min":1.429},"min":{"1min":1.113,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":1.4730000000000001,"5min":3.71,"15min":3.71},"last":4.4489999999999998},{"interface":"front","average":{"1min":1.282,"5min":1.3340000000000001,"15min":1.343},"min":{"1min":1.05,"5min":1.024,"15min":1.024},"max":{"1min":1.4970000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":6.9690000000000003}]},{"osd":7,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.3640000000000001,"15min":1.379},"min":{"1min":1.018,"5min":1.018,"15min":1.018},"max":{"1min":1.554,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":18.253},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2649999999999999,"15min":1.2609999999999999},"min":{"1min":1.1040000000000001,"5min":0.876,"15min":0.876},"max":{"1min":1.5449999999999999,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":11.006}]}]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":737280,"data_stored":327332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.05,"15min":1.046},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4359999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.216,"15min":1.2370000000000001},"min":{"1min":0.69699999999999995,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.4630000000000001,"5min":3.46,"15min":3.46},"last":0.85399999999999998}]},{"osd":1,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.2310000000000001,"15min":1.2589999999999999},"min":{"1min":0.82699999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.47,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.091,"5min":1.1739999999999999,"15min":1.1879999999999999},"min":{"1min":0.80300000000000005,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.4570000000000001,"5min":3.516,"15min":3.516},"last":1.0009999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.2070000000000001,"15min":1.2130000000000001},"min":{"1min":0.91600000000000004,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.3640000000000001,"5min":5.4180000000000001,"15min":5.4180000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3069999999999999,"15min":1.321},"min":{"1min":0.94799999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.5089999999999999,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":1.0640000000000001}]},{"osd":3,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.29,"15min":1.3080000000000001},"min":{"1min":1.022,"5min":0.879,"15min":0.879},"max":{"1min":1.4850000000000001,"5min":2.141,"15min":2.141},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.448,"15min":1.464},"min":{"1min":1.1839999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":1.583,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":1.083}]},{"osd":4,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3680000000000001,"15min":1.3879999999999999},"min":{"1min":0.84199999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.5309999999999999,"5min":3.706,"15min":3.706},"last":1.167},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.597,"15min":1.6379999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6479999999999999,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":0.753}]},{"osd":6,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.327,"15min":1.3420000000000001},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.6439999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.546,"15min":1.5860000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.613,"5min":3.641,"15min":3.641},"last":1.2490000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4259999999999999,"15min":1.4370000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.762,"5min":3.516,"15min":3.516},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.46,"15min":1.4650000000000001},"min":{"1min":1.139,"5min":1.022,"15min":1.022},"max":{"1min":1.722,"5min":3.754,"15min":3.754},"last":1.2290000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297824,"kb_used_data":728,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":745472,"data_stored":327174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.84999999999999998,"15min":0.84599999999999997},"min":{"1min":0.63600000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2150000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.72199999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.155,"5min":1.419,"15min":1.419},"last":0.99299999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.85799999999999998,"15min":0.84799999999999998},"min":{"1min":0.495,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.90700000000000003,"15min":0.91000000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.161,"5min":1.341,"15min":1.341},"last":0.94799999999999995}]},{"osd":3,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.0620000000000001,"15min":1.054},"min":{"1min":0.54100000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.194},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.98899999999999999,"15min":0.97699999999999998},"min":{"1min":0.67800000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":1.1020000000000001}]},{"osd":4,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.175,"15min":1.165},"min":{"1min":0.75700000000000001,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.49,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.248,"15min":1.2589999999999999},"min":{"1min":0.92000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.5609999999999999,"5min":2.206,"15min":2.206},"last":1.0800000000000001}]},{"osd":5,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.26,"15min":1.2649999999999999},"min":{"1min":1.0089999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.4299999999999999,"5min":1.899,"15min":1.899},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.2829999999999999,"15min":1.2909999999999999},"min":{"1min":0.92800000000000005,"5min":0.871,"15min":0.871},"max":{"1min":1.5029999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.151}]},{"osd":6,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.323,"15min":1.325},"min":{"1min":0.98299999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.5229999999999999,"5min":1.825,"15min":1.825},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.4099999999999999,"15min":1.4119999999999999},"min":{"1min":1.1870000000000001,"5min":0.87,"15min":0.87},"max":{"1min":1.5529999999999999,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.204}]},{"osd":7,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3200000000000001,"15min":1.3169999999999999},"min":{"1min":0.93999999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.6839999999999999,"5min":1.772,"15min":1.772},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3540000000000001,"15min":1.3600000000000001},"min":{"1min":0.95499999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.6120000000000001,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.2869999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T02:59:43.613 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T02:59:43.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.769+0000 7fa00945b700 1 Processor -- start 2023-11-15T02:59:43.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.773+0000 7fa00945b700 1 -- start start 2023-11-15T02:59:43.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.773+0000 7fa00945b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa004154ca0 0x7fa004155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:43.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.773+0000 7fa00945b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0041562f0 0x7fa0040a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:43.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.773+0000 7fa00945b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0040c4c30 con 0x7fa0040a8f40 2023-11-15T02:59:43.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.773+0000 7fa00945b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0040a8280 con 0x7fa004154ca0 2023-11-15T02:59:43.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.773+0000 7fa00945b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0040a6d40 con 0x7fa0041562f0 2023-11-15T02:59:43.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.773+0000 7fa002ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa004154ca0 0x7fa004155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:43.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.773+0000 7fa002ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa004154ca0 0x7fa004155080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39392/0 (socket says 172.21.15.121:39392) 2023-11-15T02:59:43.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.773+0000 7fa002ffd700 1 -- 172.21.15.121:0/3882321161 learned_addr learned my addr 172.21.15.121:0/3882321161 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:43.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.773+0000 7f9ffbfff700 1 --2- 172.21.15.121:0/3882321161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0041562f0 0x7fa0040a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:43.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.773+0000 7fa002ffd700 1 -- 172.21.15.121:0/3882321161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0041562f0 msgr2=0x7fa0040a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:43.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.773+0000 7fa002ffd700 1 --2- 172.21.15.121:0/3882321161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0041562f0 0x7fa0040a8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:43.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.773+0000 7fa002ffd700 1 -- 172.21.15.121:0/3882321161 >> v1:172.21.15.190:6789/0 conn(0x7fa0040a8f40 legacy=0x7fa0040a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:43.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.773+0000 7fa002ffd700 1 -- 172.21.15.121:0/3882321161 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0040a6f50 con 0x7fa004154ca0 2023-11-15T02:59:43.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.773+0000 7f9ffbfff700 1 --2- 172.21.15.121:0/3882321161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0041562f0 0x7fa0040a8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:43.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.777+0000 7fa002ffd700 1 --2- 172.21.15.121:0/3882321161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa004154ca0 0x7fa004155080 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f9ff4006fd0 tx=0x7f9ff40090f0).ready entity=mon.0 client_cookie=acf891bbb26fd150 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:43.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.777+0000 7fa0027fc700 1 -- 172.21.15.121:0/3882321161 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff400b600 con 0x7fa004154ca0 2023-11-15T02:59:43.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.777+0000 7fa0027fc700 1 -- 172.21.15.121:0/3882321161 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff400b760 con 0x7fa004154ca0 2023-11-15T02:59:43.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.777+0000 7fa0027fc700 1 -- 172.21.15.121:0/3882321161 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff400bae0 con 0x7fa004154ca0 2023-11-15T02:59:43.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.777+0000 7fa00945b700 1 -- 172.21.15.121:0/3882321161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa004154ca0 msgr2=0x7fa004155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:43.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.777+0000 7fa00945b700 1 --2- 172.21.15.121:0/3882321161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa004154ca0 0x7fa004155080 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f9ff4006fd0 tx=0x7f9ff40090f0).stop 2023-11-15T02:59:43.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.777+0000 7fa00945b700 1 -- 172.21.15.121:0/3882321161 shutdown_connections 2023-11-15T02:59:43.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.777+0000 7fa00945b700 1 --2- 172.21.15.121:0/3882321161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0041562f0 0x7fa0040a8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:43.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.777+0000 7fa00945b700 1 --2- 172.21.15.121:0/3882321161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa004154ca0 0x7fa004155080 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:43.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.777+0000 7fa00945b700 1 -- 172.21.15.121:0/3882321161 >> 172.21.15.121:0/3882321161 conn(0x7fa004062190 msgr2=0x7fa0040625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:43.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.777+0000 7fa00945b700 1 -- 172.21.15.121:0/3882321161 shutdown_connections 2023-11-15T02:59:43.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.777+0000 7fa00945b700 1 -- 172.21.15.121:0/3882321161 wait complete. 2023-11-15T02:59:43.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa00945b700 1 Processor -- start 2023-11-15T02:59:43.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa00945b700 1 -- start start 2023-11-15T02:59:43.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa00945b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0040a8f40 0x7fa004149b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:43.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa00945b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa004154ca0 0x7fa0040a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:43.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa00945b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0041562f0 0x7fa0040a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:43.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa00945b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0040aa470 con 0x7fa0040a8f40 2023-11-15T02:59:43.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa00945b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0040ac670 con 0x7fa0041562f0 2023-11-15T02:59:43.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa00945b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00414a790 con 0x7fa004154ca0 2023-11-15T02:59:43.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa002ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0040a8f40 0x7fa004149b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:43.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa002ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0040a8f40 0x7fa004149b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39408/0 (socket says 172.21.15.121:39408) 2023-11-15T02:59:43.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa002ffd700 1 -- 172.21.15.121:0/1454185509 learned_addr learned my addr 172.21.15.121:0/1454185509 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:43.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7f9ffbfff700 1 --2- 172.21.15.121:0/1454185509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa004154ca0 0x7fa0040a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:43.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa0037fe700 1 --2- 172.21.15.121:0/1454185509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0041562f0 0x7fa0040a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:43.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa002ffd700 1 -- 172.21.15.121:0/1454185509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa004154ca0 msgr2=0x7fa0040a9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:43.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa002ffd700 1 --2- 172.21.15.121:0/1454185509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa004154ca0 0x7fa0040a9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:43.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa002ffd700 1 -- 172.21.15.121:0/1454185509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0041562f0 msgr2=0x7fa0040a9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:43.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa002ffd700 1 --2- 172.21.15.121:0/1454185509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0041562f0 0x7fa0040a9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:43.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa002ffd700 1 -- 172.21.15.121:0/1454185509 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff4009370 con 0x7fa0040a8f40 2023-11-15T02:59:43.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7f9ffbfff700 1 --2- 172.21.15.121:0/1454185509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa004154ca0 0x7fa0040a9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:43.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa002ffd700 1 --2- 172.21.15.121:0/1454185509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0040a8f40 0x7fa004149b80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f9ff4009f50 tx=0x7f9ff4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:43.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7f9ffaffd700 1 -- 172.21.15.121:0/1454185509 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff400b8d0 con 0x7fa0040a8f40 2023-11-15T02:59:43.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7f9ffaffd700 1 -- 172.21.15.121:0/1454185509 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff400ba30 con 0x7fa0040a8f40 2023-11-15T02:59:43.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa00945b700 1 -- 172.21.15.121:0/1454185509 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0040abac0 con 0x7fa0040a8f40 2023-11-15T02:59:43.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7f9ffaffd700 1 -- 172.21.15.121:0/1454185509 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff401b660 con 0x7fa0040a8f40 2023-11-15T02:59:43.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.781+0000 7fa00945b700 1 -- 172.21.15.121:0/1454185509 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0040ac020 con 0x7fa0040a8f40 2023-11-15T02:59:43.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.785+0000 7fa00945b700 1 -- 172.21.15.121:0/1454185509 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa00406c150 con 0x7fa0040a8f40 2023-11-15T02:59:43.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.785+0000 7f9ffaffd700 1 -- 172.21.15.121:0/1454185509 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9ff401b9d0 con 0x7fa0040a8f40 2023-11-15T02:59:43.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.785+0000 7f9ffaffd700 1 --2- 172.21.15.121:0/1454185509 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9fe405a4e0 0x7f9fe405c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:43.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.785+0000 7f9ffaffd700 1 -- 172.21.15.121:0/1454185509 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9ff4076240 con 0x7fa0040a8f40 2023-11-15T02:59:43.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.789+0000 7f9ffbfff700 1 --2- 172.21.15.121:0/1454185509 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9fe405a4e0 0x7f9fe405c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:43.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.789+0000 7f9ffbfff700 1 --2- 172.21.15.121:0/1454185509 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9fe405a4e0 0x7f9fe405c9a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f9fec006d80 tx=0x7f9fec008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:43.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.793+0000 7f9ffaffd700 1 -- 172.21.15.121:0/1454185509 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9ff4049a20 con 0x7fa0040a8f40 2023-11-15T02:59:43.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.953+0000 7fa00945b700 1 -- 172.21.15.121:0/1454185509 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0040a7230 con 0x7f9fe405a4e0 2023-11-15T02:59:43.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.969+0000 7f9ffaffd700 1 -- 172.21.15.121:0/1454185509 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+357958 (secure 0 0 0) 0x7fa0040a7230 con 0x7f9fe405a4e0 2023-11-15T02:59:43.984 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:43.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.981+0000 7fa00945b700 1 -- 172.21.15.121:0/1454185509 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9fe405a4e0 msgr2=0x7f9fe405c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:43.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.981+0000 7fa00945b700 1 --2- 172.21.15.121:0/1454185509 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9fe405a4e0 0x7f9fe405c9a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f9fec006d80 tx=0x7f9fec008040).stop 2023-11-15T02:59:43.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.981+0000 7fa00945b700 1 -- 172.21.15.121:0/1454185509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0040a8f40 msgr2=0x7fa004149b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:43.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.981+0000 7fa00945b700 1 --2- 172.21.15.121:0/1454185509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0040a8f40 0x7fa004149b80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f9ff4009f50 tx=0x7f9ff4017000).stop 2023-11-15T02:59:43.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.981+0000 7fa00945b700 1 -- 172.21.15.121:0/1454185509 shutdown_connections 2023-11-15T02:59:43.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.981+0000 7fa00945b700 1 --2- 172.21.15.121:0/1454185509 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9fe405a4e0 0x7f9fe405c9a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:43.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.981+0000 7fa00945b700 1 --2- 172.21.15.121:0/1454185509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0041562f0 0x7fa0040a9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:43.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.981+0000 7fa00945b700 1 --2- 172.21.15.121:0/1454185509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa004154ca0 0x7fa0040a9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:43.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.981+0000 7fa00945b700 1 --2- 172.21.15.121:0/1454185509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0040a8f40 0x7fa004149b80 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:43.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.981+0000 7fa00945b700 1 -- 172.21.15.121:0/1454185509 >> 172.21.15.121:0/1454185509 conn(0x7fa004062190 msgr2=0x7fa0040c59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:43.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.981+0000 7fa00945b700 1 -- 172.21.15.121:0/1454185509 shutdown_connections 2023-11-15T02:59:43.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:43.981+0000 7fa00945b700 1 -- 172.21.15.121:0/1454185509 wait complete. 2023-11-15T02:59:43.990 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T02:59:44.007 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2023-11-15T02:59:41.972195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382524,"kb_used_data":5756,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752592196,"statfs":{"total":773094113280,"available":770654408704,"internally_reserved":0,"allocated":5894144,"data_stored":2614687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11303,"num_objects":-17,"num_object_clones":0,"num_object_copies":-34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-17,"num_whiteouts":0,"num_read":-50,"num_read_kb":-56,"num_write":-60,"num_write_kb":-31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.466566"},"pg_stats":[{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.715495+0000","last_change":"2023-11-15T02:59:34.702569+0000","last_active":"2023-11-15T02:59:34.715495+0000","last_peered":"2023-11-15T02:59:34.715495+0000","last_clean":"2023-11-15T02:59:34.715495+0000","last_became_active":"2023-11-15T02:59:34.702272+0000","last_became_peered":"2023-11-15T02:59:34.702272+0000","last_unstale":"2023-11-15T02:59:34.715495+0000","last_undegraded":"2023-11-15T02:59:34.715495+0000","last_fullsized":"2023-11-15T02:59:34.715495+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.3f","version":"60'3","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.702515+0000","last_change":"2023-11-15T02:59:34.265036+0000","last_active":"2023-11-15T02:59:34.702515+0000","last_peered":"2023-11-15T02:59:34.702515+0000","last_clean":"2023-11-15T02:59:34.702515+0000","last_became_active":"2023-11-15T02:59:34.264631+0000","last_became_peered":"2023-11-15T02:59:34.264631+0000","last_unstale":"2023-11-15T02:59:34.702515+0000","last_undegraded":"2023-11-15T02:59:34.702515+0000","last_fullsized":"2023-11-15T02:59:34.702515+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.719722+0000","last_change":"2023-11-15T02:59:34.715077+0000","last_active":"2023-11-15T02:59:34.719722+0000","last_peered":"2023-11-15T02:59:34.719722+0000","last_clean":"2023-11-15T02:59:34.719722+0000","last_became_active":"2023-11-15T02:59:34.710308+0000","last_became_peered":"2023-11-15T02:59:34.710308+0000","last_unstale":"2023-11-15T02:59:34.719722+0000","last_undegraded":"2023-11-15T02:59:34.719722+0000","last_fullsized":"2023-11-15T02:59:34.719722+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.679013+0000","last_change":"2023-11-15T02:59:34.274806+0000","last_active":"2023-11-15T02:59:34.679013+0000","last_peered":"2023-11-15T02:59:34.679013+0000","last_clean":"2023-11-15T02:59:34.679013+0000","last_became_active":"2023-11-15T02:59:34.274536+0000","last_became_peered":"2023-11-15T02:59:34.274536+0000","last_unstale":"2023-11-15T02:59:34.679013+0000","last_undegraded":"2023-11-15T02:59:34.679013+0000","last_fullsized":"2023-11-15T02:59:34.679013+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.713626+0000","last_change":"2023-11-15T02:59:34.702688+0000","last_active":"2023-11-15T02:59:34.713626+0000","last_peered":"2023-11-15T02:59:34.713626+0000","last_clean":"2023-11-15T02:59:34.713626+0000","last_became_active":"2023-11-15T02:59:34.702267+0000","last_became_peered":"2023-11-15T02:59:34.702267+0000","last_unstale":"2023-11-15T02:59:34.713626+0000","last_undegraded":"2023-11-15T02:59:34.713626+0000","last_fullsized":"2023-11-15T02:59:34.713626+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.689775+0000","last_change":"2023-11-15T02:59:34.272252+0000","last_active":"2023-11-15T02:59:34.689775+0000","last_peered":"2023-11-15T02:59:34.689775+0000","last_clean":"2023-11-15T02:59:34.689775+0000","last_became_active":"2023-11-15T02:59:34.271979+0000","last_became_peered":"2023-11-15T02:59:34.271979+0000","last_unstale":"2023-11-15T02:59:34.689775+0000","last_undegraded":"2023-11-15T02:59:34.689775+0000","last_fullsized":"2023-11-15T02:59:34.689775+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.714332+0000","last_change":"2023-11-15T02:59:34.708004+0000","last_active":"2023-11-15T02:59:34.714332+0000","last_peered":"2023-11-15T02:59:34.714332+0000","last_clean":"2023-11-15T02:59:34.714332+0000","last_became_active":"2023-11-15T02:59:34.707733+0000","last_became_peered":"2023-11-15T02:59:34.707733+0000","last_unstale":"2023-11-15T02:59:34.714332+0000","last_undegraded":"2023-11-15T02:59:34.714332+0000","last_fullsized":"2023-11-15T02:59:34.714332+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3c","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.710045+0000","last_change":"2023-11-15T02:59:34.267850+0000","last_active":"2023-11-15T02:59:34.710045+0000","last_peered":"2023-11-15T02:59:34.710045+0000","last_clean":"2023-11-15T02:59:34.710045+0000","last_became_active":"2023-11-15T02:59:34.267579+0000","last_became_peered":"2023-11-15T02:59:34.267579+0000","last_unstale":"2023-11-15T02:59:34.710045+0000","last_undegraded":"2023-11-15T02:59:34.710045+0000","last_fullsized":"2023-11-15T02:59:34.710045+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.718958+0000","last_change":"2023-11-15T02:59:34.714724+0000","last_active":"2023-11-15T02:59:34.718958+0000","last_peered":"2023-11-15T02:59:34.718958+0000","last_clean":"2023-11-15T02:59:34.718958+0000","last_became_active":"2023-11-15T02:59:34.714460+0000","last_became_peered":"2023-11-15T02:59:34.714460+0000","last_unstale":"2023-11-15T02:59:34.718958+0000","last_undegraded":"2023-11-15T02:59:34.718958+0000","last_fullsized":"2023-11-15T02:59:34.718958+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.711839+0000","last_change":"2023-11-15T02:59:34.263921+0000","last_active":"2023-11-15T02:59:34.711839+0000","last_peered":"2023-11-15T02:59:34.711839+0000","last_clean":"2023-11-15T02:59:34.711839+0000","last_became_active":"2023-11-15T02:59:34.261031+0000","last_became_peered":"2023-11-15T02:59:34.261031+0000","last_unstale":"2023-11-15T02:59:34.711839+0000","last_undegraded":"2023-11-15T02:59:34.711839+0000","last_fullsized":"2023-11-15T02:59:34.711839+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.721937+0000","last_change":"2023-11-15T02:59:34.716845+0000","last_active":"2023-11-15T02:59:34.721937+0000","last_peered":"2023-11-15T02:59:34.721937+0000","last_clean":"2023-11-15T02:59:34.721937+0000","last_became_active":"2023-11-15T02:59:34.716558+0000","last_became_peered":"2023-11-15T02:59:34.716558+0000","last_unstale":"2023-11-15T02:59:34.721937+0000","last_undegraded":"2023-11-15T02:59:34.721937+0000","last_fullsized":"2023-11-15T02:59:34.721937+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.706213+0000","last_change":"2023-11-15T02:59:34.266240+0000","last_active":"2023-11-15T02:59:34.706213+0000","last_peered":"2023-11-15T02:59:34.706213+0000","last_clean":"2023-11-15T02:59:34.706213+0000","last_became_active":"2023-11-15T02:59:34.265188+0000","last_became_peered":"2023-11-15T02:59:34.265188+0000","last_unstale":"2023-11-15T02:59:34.706213+0000","last_undegraded":"2023-11-15T02:59:34.706213+0000","last_fullsized":"2023-11-15T02:59:34.706213+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.721951+0000","last_change":"2023-11-15T02:59:34.719027+0000","last_active":"2023-11-15T02:59:34.721951+0000","last_peered":"2023-11-15T02:59:34.721951+0000","last_clean":"2023-11-15T02:59:34.721951+0000","last_became_active":"2023-11-15T02:59:34.718717+0000","last_became_peered":"2023-11-15T02:59:34.718717+0000","last_unstale":"2023-11-15T02:59:34.721951+0000","last_undegraded":"2023-11-15T02:59:34.721951+0000","last_fullsized":"2023-11-15T02:59:34.721951+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.685525+0000","last_change":"2023-11-15T02:59:34.263881+0000","last_active":"2023-11-15T02:59:34.685525+0000","last_peered":"2023-11-15T02:59:34.685525+0000","last_clean":"2023-11-15T02:59:34.685525+0000","last_became_active":"2023-11-15T02:59:34.263571+0000","last_became_peered":"2023-11-15T02:59:34.263571+0000","last_unstale":"2023-11-15T02:59:34.685525+0000","last_undegraded":"2023-11-15T02:59:34.685525+0000","last_fullsized":"2023-11-15T02:59:34.685525+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.723676+0000","last_change":"2023-11-15T02:59:34.715365+0000","last_active":"2023-11-15T02:59:34.723676+0000","last_peered":"2023-11-15T02:59:34.723676+0000","last_clean":"2023-11-15T02:59:34.723676+0000","last_became_active":"2023-11-15T02:59:34.715043+0000","last_became_peered":"2023-11-15T02:59:34.715043+0000","last_unstale":"2023-11-15T02:59:34.723676+0000","last_undegraded":"2023-11-15T02:59:34.723676+0000","last_fullsized":"2023-11-15T02:59:34.723676+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.677675+0000","last_change":"2023-11-15T02:59:34.264832+0000","last_active":"2023-11-15T02:59:34.677675+0000","last_peered":"2023-11-15T02:59:34.677675+0000","last_clean":"2023-11-15T02:59:34.677675+0000","last_became_active":"2023-11-15T02:59:34.264571+0000","last_became_peered":"2023-11-15T02:59:34.264571+0000","last_unstale":"2023-11-15T02:59:34.677675+0000","last_undegraded":"2023-11-15T02:59:34.677675+0000","last_fullsized":"2023-11-15T02:59:34.677675+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.719012+0000","last_change":"2023-11-15T02:59:34.715353+0000","last_active":"2023-11-15T02:59:34.719012+0000","last_peered":"2023-11-15T02:59:34.719012+0000","last_clean":"2023-11-15T02:59:34.719012+0000","last_became_active":"2023-11-15T02:59:34.715023+0000","last_became_peered":"2023-11-15T02:59:34.715023+0000","last_unstale":"2023-11-15T02:59:34.719012+0000","last_undegraded":"2023-11-15T02:59:34.719012+0000","last_fullsized":"2023-11-15T02:59:34.719012+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.684404+0000","last_change":"2023-11-15T02:59:34.262030+0000","last_active":"2023-11-15T02:59:34.684404+0000","last_peered":"2023-11-15T02:59:34.684404+0000","last_clean":"2023-11-15T02:59:34.684404+0000","last_became_active":"2023-11-15T02:59:34.261689+0000","last_became_peered":"2023-11-15T02:59:34.261689+0000","last_unstale":"2023-11-15T02:59:34.684404+0000","last_undegraded":"2023-11-15T02:59:34.684404+0000","last_fullsized":"2023-11-15T02:59:34.684404+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.713282+0000","last_change":"2023-11-15T02:59:34.701348+0000","last_active":"2023-11-15T02:59:34.713282+0000","last_peered":"2023-11-15T02:59:34.713282+0000","last_clean":"2023-11-15T02:59:34.713282+0000","last_became_active":"2023-11-15T02:59:34.701079+0000","last_became_peered":"2023-11-15T02:59:34.701079+0000","last_unstale":"2023-11-15T02:59:34.713282+0000","last_undegraded":"2023-11-15T02:59:34.713282+0000","last_fullsized":"2023-11-15T02:59:34.713282+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.688465+0000","last_change":"2023-11-15T02:59:34.269872+0000","last_active":"2023-11-15T02:59:34.688465+0000","last_peered":"2023-11-15T02:59:34.688465+0000","last_clean":"2023-11-15T02:59:34.688465+0000","last_became_active":"2023-11-15T02:59:34.269585+0000","last_became_peered":"2023-11-15T02:59:34.269585+0000","last_unstale":"2023-11-15T02:59:34.688465+0000","last_undegraded":"2023-11-15T02:59:34.688465+0000","last_fullsized":"2023-11-15T02:59:34.688465+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.715982+0000","last_change":"2023-11-15T02:59:34.707253+0000","last_active":"2023-11-15T02:59:34.715982+0000","last_peered":"2023-11-15T02:59:34.715982+0000","last_clean":"2023-11-15T02:59:34.715982+0000","last_became_active":"2023-11-15T02:59:34.706969+0000","last_became_peered":"2023-11-15T02:59:34.706969+0000","last_unstale":"2023-11-15T02:59:34.715982+0000","last_undegraded":"2023-11-15T02:59:34.715982+0000","last_fullsized":"2023-11-15T02:59:34.715982+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.688078+0000","last_change":"2023-11-15T02:59:34.269836+0000","last_active":"2023-11-15T02:59:34.688078+0000","last_peered":"2023-11-15T02:59:34.688078+0000","last_clean":"2023-11-15T02:59:34.688078+0000","last_became_active":"2023-11-15T02:59:34.269512+0000","last_became_peered":"2023-11-15T02:59:34.269512+0000","last_unstale":"2023-11-15T02:59:34.688078+0000","last_undegraded":"2023-11-15T02:59:34.688078+0000","last_fullsized":"2023-11-15T02:59:34.688078+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.722440+0000","last_change":"2023-11-15T02:59:34.717917+0000","last_active":"2023-11-15T02:59:34.722440+0000","last_peered":"2023-11-15T02:59:34.722440+0000","last_clean":"2023-11-15T02:59:34.722440+0000","last_became_active":"2023-11-15T02:59:34.717573+0000","last_became_peered":"2023-11-15T02:59:34.717573+0000","last_unstale":"2023-11-15T02:59:34.722440+0000","last_undegraded":"2023-11-15T02:59:34.722440+0000","last_fullsized":"2023-11-15T02:59:34.722440+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.34","version":"60'4","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.724147+0000","last_change":"2023-11-15T02:59:34.272591+0000","last_active":"2023-11-15T02:59:34.724147+0000","last_peered":"2023-11-15T02:59:34.724147+0000","last_clean":"2023-11-15T02:59:34.724147+0000","last_became_active":"2023-11-15T02:59:34.270963+0000","last_became_peered":"2023-11-15T02:59:34.270963+0000","last_unstale":"2023-11-15T02:59:34.724147+0000","last_undegraded":"2023-11-15T02:59:34.724147+0000","last_fullsized":"2023-11-15T02:59:34.724147+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.713052+0000","last_change":"2023-11-15T02:59:34.709579+0000","last_active":"2023-11-15T02:59:34.713052+0000","last_peered":"2023-11-15T02:59:34.713052+0000","last_clean":"2023-11-15T02:59:34.713052+0000","last_became_active":"2023-11-15T02:59:34.709324+0000","last_became_peered":"2023-11-15T02:59:34.709324+0000","last_unstale":"2023-11-15T02:59:34.713052+0000","last_undegraded":"2023-11-15T02:59:34.713052+0000","last_fullsized":"2023-11-15T02:59:34.713052+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.33","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.715396+0000","last_change":"2023-11-15T02:59:34.261668+0000","last_active":"2023-11-15T02:59:34.715396+0000","last_peered":"2023-11-15T02:59:34.715396+0000","last_clean":"2023-11-15T02:59:34.715396+0000","last_became_active":"2023-11-15T02:59:34.261392+0000","last_became_peered":"2023-11-15T02:59:34.261392+0000","last_unstale":"2023-11-15T02:59:34.715396+0000","last_undegraded":"2023-11-15T02:59:34.715396+0000","last_fullsized":"2023-11-15T02:59:34.715396+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.721115+0000","last_change":"2023-11-15T02:59:34.717458+0000","last_active":"2023-11-15T02:59:34.721115+0000","last_peered":"2023-11-15T02:59:34.721115+0000","last_clean":"2023-11-15T02:59:34.721115+0000","last_became_active":"2023-11-15T02:59:34.717124+0000","last_became_peered":"2023-11-15T02:59:34.717124+0000","last_unstale":"2023-11-15T02:59:34.721115+0000","last_undegraded":"2023-11-15T02:59:34.721115+0000","last_fullsized":"2023-11-15T02:59:34.721115+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.683929+0000","last_change":"2023-11-15T02:59:34.267337+0000","last_active":"2023-11-15T02:59:34.683929+0000","last_peered":"2023-11-15T02:59:34.683929+0000","last_clean":"2023-11-15T02:59:34.683929+0000","last_became_active":"2023-11-15T02:59:34.267054+0000","last_became_peered":"2023-11-15T02:59:34.267054+0000","last_unstale":"2023-11-15T02:59:34.683929+0000","last_undegraded":"2023-11-15T02:59:34.683929+0000","last_fullsized":"2023-11-15T02:59:34.683929+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.719802+0000","last_change":"2023-11-15T02:59:34.709174+0000","last_active":"2023-11-15T02:59:34.719802+0000","last_peered":"2023-11-15T02:59:34.719802+0000","last_clean":"2023-11-15T02:59:34.719802+0000","last_became_active":"2023-11-15T02:59:34.708872+0000","last_became_peered":"2023-11-15T02:59:34.708872+0000","last_unstale":"2023-11-15T02:59:34.719802+0000","last_undegraded":"2023-11-15T02:59:34.719802+0000","last_fullsized":"2023-11-15T02:59:34.719802+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.685690+0000","last_change":"2023-11-15T02:59:34.261461+0000","last_active":"2023-11-15T02:59:34.685690+0000","last_peered":"2023-11-15T02:59:34.685690+0000","last_clean":"2023-11-15T02:59:34.685690+0000","last_became_active":"2023-11-15T02:59:34.261153+0000","last_became_peered":"2023-11-15T02:59:34.261153+0000","last_unstale":"2023-11-15T02:59:34.685690+0000","last_undegraded":"2023-11-15T02:59:34.685690+0000","last_fullsized":"2023-11-15T02:59:34.685690+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.718989+0000","last_change":"2023-11-15T02:59:34.713836+0000","last_active":"2023-11-15T02:59:34.718989+0000","last_peered":"2023-11-15T02:59:34.718989+0000","last_clean":"2023-11-15T02:59:34.718989+0000","last_became_active":"2023-11-15T02:59:34.713342+0000","last_became_peered":"2023-11-15T02:59:34.713342+0000","last_unstale":"2023-11-15T02:59:34.718989+0000","last_undegraded":"2023-11-15T02:59:34.718989+0000","last_fullsized":"2023-11-15T02:59:34.718989+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.30","version":"60'4","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.722402+0000","last_change":"2023-11-15T02:59:34.264405+0000","last_active":"2023-11-15T02:59:34.722402+0000","last_peered":"2023-11-15T02:59:34.722402+0000","last_clean":"2023-11-15T02:59:34.722402+0000","last_became_active":"2023-11-15T02:59:34.264095+0000","last_became_peered":"2023-11-15T02:59:34.264095+0000","last_unstale":"2023-11-15T02:59:34.722402+0000","last_undegraded":"2023-11-15T02:59:34.722402+0000","last_fullsized":"2023-11-15T02:59:34.722402+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.699684+0000","last_change":"2023-11-15T02:59:34.694211+0000","last_active":"2023-11-15T02:59:34.699684+0000","last_peered":"2023-11-15T02:59:34.699684+0000","last_clean":"2023-11-15T02:59:34.699684+0000","last_became_active":"2023-11-15T02:59:34.693708+0000","last_became_peered":"2023-11-15T02:59:34.693708+0000","last_unstale":"2023-11-15T02:59:34.699684+0000","last_undegraded":"2023-11-15T02:59:34.699684+0000","last_fullsized":"2023-11-15T02:59:34.699684+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.679621+0000","last_change":"2023-11-15T02:59:34.271015+0000","last_active":"2023-11-15T02:59:34.679621+0000","last_peered":"2023-11-15T02:59:34.679621+0000","last_clean":"2023-11-15T02:59:34.679621+0000","last_became_active":"2023-11-15T02:59:34.270758+0000","last_became_peered":"2023-11-15T02:59:34.270758+0000","last_unstale":"2023-11-15T02:59:34.679621+0000","last_undegraded":"2023-11-15T02:59:34.679621+0000","last_fullsized":"2023-11-15T02:59:34.679621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.724390+0000","last_change":"2023-11-15T02:59:34.721266+0000","last_active":"2023-11-15T02:59:34.724390+0000","last_peered":"2023-11-15T02:59:34.724390+0000","last_clean":"2023-11-15T02:59:34.724390+0000","last_became_active":"2023-11-15T02:59:34.720978+0000","last_became_peered":"2023-11-15T02:59:34.720978+0000","last_unstale":"2023-11-15T02:59:34.724390+0000","last_undegraded":"2023-11-15T02:59:34.724390+0000","last_fullsized":"2023-11-15T02:59:34.724390+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.694039+0000","last_change":"2023-11-15T02:59:34.269316+0000","last_active":"2023-11-15T02:59:34.694039+0000","last_peered":"2023-11-15T02:59:34.694039+0000","last_clean":"2023-11-15T02:59:34.694039+0000","last_became_active":"2023-11-15T02:59:34.268953+0000","last_became_peered":"2023-11-15T02:59:34.268953+0000","last_unstale":"2023-11-15T02:59:34.694039+0000","last_undegraded":"2023-11-15T02:59:34.694039+0000","last_fullsized":"2023-11-15T02:59:34.694039+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.713549+0000","last_change":"2023-11-15T02:59:34.703052+0000","last_active":"2023-11-15T02:59:34.713549+0000","last_peered":"2023-11-15T02:59:34.713549+0000","last_clean":"2023-11-15T02:59:34.713549+0000","last_became_active":"2023-11-15T02:59:34.702791+0000","last_became_peered":"2023-11-15T02:59:34.702791+0000","last_unstale":"2023-11-15T02:59:34.713549+0000","last_undegraded":"2023-11-15T02:59:34.713549+0000","last_fullsized":"2023-11-15T02:59:34.713549+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2d","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.708369+0000","last_change":"2023-11-15T02:59:34.257033+0000","last_active":"2023-11-15T02:59:34.708369+0000","last_peered":"2023-11-15T02:59:34.708369+0000","last_clean":"2023-11-15T02:59:34.708369+0000","last_became_active":"2023-11-15T02:59:34.256744+0000","last_became_peered":"2023-11-15T02:59:34.256744+0000","last_unstale":"2023-11-15T02:59:34.708369+0000","last_undegraded":"2023-11-15T02:59:34.708369+0000","last_fullsized":"2023-11-15T02:59:34.708369+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.714601+0000","last_change":"2023-11-15T02:59:34.709810+0000","last_active":"2023-11-15T02:59:34.714601+0000","last_peered":"2023-11-15T02:59:34.714601+0000","last_clean":"2023-11-15T02:59:34.714601+0000","last_became_active":"2023-11-15T02:59:34.709481+0000","last_became_peered":"2023-11-15T02:59:34.709481+0000","last_unstale":"2023-11-15T02:59:34.714601+0000","last_undegraded":"2023-11-15T02:59:34.714601+0000","last_fullsized":"2023-11-15T02:59:34.714601+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.679214+0000","last_change":"2023-11-15T02:59:34.275824+0000","last_active":"2023-11-15T02:59:34.679214+0000","last_peered":"2023-11-15T02:59:34.679214+0000","last_clean":"2023-11-15T02:59:34.679214+0000","last_became_active":"2023-11-15T02:59:34.275088+0000","last_became_peered":"2023-11-15T02:59:34.275088+0000","last_unstale":"2023-11-15T02:59:34.679214+0000","last_undegraded":"2023-11-15T02:59:34.679214+0000","last_fullsized":"2023-11-15T02:59:34.679214+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.723857+0000","last_change":"2023-11-15T02:59:34.718638+0000","last_active":"2023-11-15T02:59:34.723857+0000","last_peered":"2023-11-15T02:59:34.723857+0000","last_clean":"2023-11-15T02:59:34.723857+0000","last_became_active":"2023-11-15T02:59:34.718018+0000","last_became_peered":"2023-11-15T02:59:34.718018+0000","last_unstale":"2023-11-15T02:59:34.723857+0000","last_undegraded":"2023-11-15T02:59:34.723857+0000","last_fullsized":"2023-11-15T02:59:34.723857+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2b","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.714425+0000","last_change":"2023-11-15T02:59:34.271365+0000","last_active":"2023-11-15T02:59:34.714425+0000","last_peered":"2023-11-15T02:59:34.714425+0000","last_clean":"2023-11-15T02:59:34.714425+0000","last_became_active":"2023-11-15T02:59:34.270414+0000","last_became_peered":"2023-11-15T02:59:34.270414+0000","last_unstale":"2023-11-15T02:59:34.714425+0000","last_undegraded":"2023-11-15T02:59:34.714425+0000","last_fullsized":"2023-11-15T02:59:34.714425+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.710410+0000","last_change":"2023-11-15T02:59:34.705469+0000","last_active":"2023-11-15T02:59:34.710410+0000","last_peered":"2023-11-15T02:59:34.710410+0000","last_clean":"2023-11-15T02:59:34.710410+0000","last_became_active":"2023-11-15T02:59:34.705188+0000","last_became_peered":"2023-11-15T02:59:34.705188+0000","last_unstale":"2023-11-15T02:59:34.710410+0000","last_undegraded":"2023-11-15T02:59:34.710410+0000","last_fullsized":"2023-11-15T02:59:34.710410+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.678614+0000","last_change":"2023-11-15T02:59:34.269772+0000","last_active":"2023-11-15T02:59:34.678614+0000","last_peered":"2023-11-15T02:59:34.678614+0000","last_clean":"2023-11-15T02:59:34.678614+0000","last_became_active":"2023-11-15T02:59:34.269150+0000","last_became_peered":"2023-11-15T02:59:34.269150+0000","last_unstale":"2023-11-15T02:59:34.678614+0000","last_undegraded":"2023-11-15T02:59:34.678614+0000","last_fullsized":"2023-11-15T02:59:34.678614+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.710397+0000","last_change":"2023-11-15T02:59:34.706266+0000","last_active":"2023-11-15T02:59:34.710397+0000","last_peered":"2023-11-15T02:59:34.710397+0000","last_clean":"2023-11-15T02:59:34.710397+0000","last_became_active":"2023-11-15T02:59:34.705045+0000","last_became_peered":"2023-11-15T02:59:34.705045+0000","last_unstale":"2023-11-15T02:59:34.710397+0000","last_undegraded":"2023-11-15T02:59:34.710397+0000","last_fullsized":"2023-11-15T02:59:34.710397+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.699279+0000","last_change":"2023-11-15T02:59:34.271476+0000","last_active":"2023-11-15T02:59:34.699279+0000","last_peered":"2023-11-15T02:59:34.699279+0000","last_clean":"2023-11-15T02:59:34.699279+0000","last_became_active":"2023-11-15T02:59:34.271135+0000","last_became_peered":"2023-11-15T02:59:34.271135+0000","last_unstale":"2023-11-15T02:59:34.699279+0000","last_undegraded":"2023-11-15T02:59:34.699279+0000","last_fullsized":"2023-11-15T02:59:34.699279+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716085+0000","last_change":"2023-11-15T02:59:34.709466+0000","last_active":"2023-11-15T02:59:34.716085+0000","last_peered":"2023-11-15T02:59:34.716085+0000","last_clean":"2023-11-15T02:59:34.716085+0000","last_became_active":"2023-11-15T02:59:34.709166+0000","last_became_peered":"2023-11-15T02:59:34.709166+0000","last_unstale":"2023-11-15T02:59:34.716085+0000","last_undegraded":"2023-11-15T02:59:34.716085+0000","last_fullsized":"2023-11-15T02:59:34.716085+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.689536+0000","last_change":"2023-11-15T02:59:34.264047+0000","last_active":"2023-11-15T02:59:34.689536+0000","last_peered":"2023-11-15T02:59:34.689536+0000","last_clean":"2023-11-15T02:59:34.689536+0000","last_became_active":"2023-11-15T02:59:34.263525+0000","last_became_peered":"2023-11-15T02:59:34.263525+0000","last_unstale":"2023-11-15T02:59:34.689536+0000","last_undegraded":"2023-11-15T02:59:34.689536+0000","last_fullsized":"2023-11-15T02:59:34.689536+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.714415+0000","last_change":"2023-11-15T02:59:34.707630+0000","last_active":"2023-11-15T02:59:34.714415+0000","last_peered":"2023-11-15T02:59:34.714415+0000","last_clean":"2023-11-15T02:59:34.714415+0000","last_became_active":"2023-11-15T02:59:34.707321+0000","last_became_peered":"2023-11-15T02:59:34.707321+0000","last_unstale":"2023-11-15T02:59:34.714415+0000","last_undegraded":"2023-11-15T02:59:34.714415+0000","last_fullsized":"2023-11-15T02:59:34.714415+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.692196+0000","last_change":"2023-11-15T02:59:34.253441+0000","last_active":"2023-11-15T02:59:34.692196+0000","last_peered":"2023-11-15T02:59:34.692196+0000","last_clean":"2023-11-15T02:59:34.692196+0000","last_became_active":"2023-11-15T02:59:34.253172+0000","last_became_peered":"2023-11-15T02:59:34.253172+0000","last_unstale":"2023-11-15T02:59:34.692196+0000","last_undegraded":"2023-11-15T02:59:34.692196+0000","last_fullsized":"2023-11-15T02:59:34.692196+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.715622+0000","last_change":"2023-11-15T02:59:34.706426+0000","last_active":"2023-11-15T02:59:34.715622+0000","last_peered":"2023-11-15T02:59:34.715622+0000","last_clean":"2023-11-15T02:59:34.715622+0000","last_became_active":"2023-11-15T02:59:34.705567+0000","last_became_peered":"2023-11-15T02:59:34.705567+0000","last_unstale":"2023-11-15T02:59:34.715622+0000","last_undegraded":"2023-11-15T02:59:34.715622+0000","last_fullsized":"2023-11-15T02:59:34.715622+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.702168+0000","last_change":"2023-11-15T02:59:34.264111+0000","last_active":"2023-11-15T02:59:34.702168+0000","last_peered":"2023-11-15T02:59:34.702168+0000","last_clean":"2023-11-15T02:59:34.702168+0000","last_became_active":"2023-11-15T02:59:34.263570+0000","last_became_peered":"2023-11-15T02:59:34.263570+0000","last_unstale":"2023-11-15T02:59:34.702168+0000","last_undegraded":"2023-11-15T02:59:34.702168+0000","last_fullsized":"2023-11-15T02:59:34.702168+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716225+0000","last_change":"2023-11-15T02:59:34.709971+0000","last_active":"2023-11-15T02:59:34.716225+0000","last_peered":"2023-11-15T02:59:34.716225+0000","last_clean":"2023-11-15T02:59:34.716225+0000","last_became_active":"2023-11-15T02:59:34.709667+0000","last_became_peered":"2023-11-15T02:59:34.709667+0000","last_unstale":"2023-11-15T02:59:34.716225+0000","last_undegraded":"2023-11-15T02:59:34.716225+0000","last_fullsized":"2023-11-15T02:59:34.716225+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.690028+0000","last_change":"2023-11-15T02:59:34.264728+0000","last_active":"2023-11-15T02:59:34.690028+0000","last_peered":"2023-11-15T02:59:34.690028+0000","last_clean":"2023-11-15T02:59:34.690028+0000","last_became_active":"2023-11-15T02:59:34.264416+0000","last_became_peered":"2023-11-15T02:59:34.264416+0000","last_unstale":"2023-11-15T02:59:34.690028+0000","last_undegraded":"2023-11-15T02:59:34.690028+0000","last_fullsized":"2023-11-15T02:59:34.690028+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716101+0000","last_change":"2023-11-15T02:59:34.709219+0000","last_active":"2023-11-15T02:59:34.716101+0000","last_peered":"2023-11-15T02:59:34.716101+0000","last_clean":"2023-11-15T02:59:34.716101+0000","last_became_active":"2023-11-15T02:59:34.708939+0000","last_became_peered":"2023-11-15T02:59:34.708939+0000","last_unstale":"2023-11-15T02:59:34.716101+0000","last_undegraded":"2023-11-15T02:59:34.716101+0000","last_fullsized":"2023-11-15T02:59:34.716101+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.690938+0000","last_change":"2023-11-15T02:59:34.261680+0000","last_active":"2023-11-15T02:59:34.690938+0000","last_peered":"2023-11-15T02:59:34.690938+0000","last_clean":"2023-11-15T02:59:34.690938+0000","last_became_active":"2023-11-15T02:59:34.260538+0000","last_became_peered":"2023-11-15T02:59:34.260538+0000","last_unstale":"2023-11-15T02:59:34.690938+0000","last_undegraded":"2023-11-15T02:59:34.690938+0000","last_fullsized":"2023-11-15T02:59:34.690938+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.714556+0000","last_change":"2023-11-15T02:59:34.707619+0000","last_active":"2023-11-15T02:59:34.714556+0000","last_peered":"2023-11-15T02:59:34.714556+0000","last_clean":"2023-11-15T02:59:34.714556+0000","last_became_active":"2023-11-15T02:59:34.707297+0000","last_became_peered":"2023-11-15T02:59:34.707297+0000","last_unstale":"2023-11-15T02:59:34.714556+0000","last_undegraded":"2023-11-15T02:59:34.714556+0000","last_fullsized":"2023-11-15T02:59:34.714556+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.677962+0000","last_change":"2023-11-15T02:59:34.266701+0000","last_active":"2023-11-15T02:59:34.677962+0000","last_peered":"2023-11-15T02:59:34.677962+0000","last_clean":"2023-11-15T02:59:34.677962+0000","last_became_active":"2023-11-15T02:59:34.266347+0000","last_became_peered":"2023-11-15T02:59:34.266347+0000","last_unstale":"2023-11-15T02:59:34.677962+0000","last_undegraded":"2023-11-15T02:59:34.677962+0000","last_fullsized":"2023-11-15T02:59:34.677962+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.713662+0000","last_change":"2023-11-15T02:59:34.705873+0000","last_active":"2023-11-15T02:59:34.713662+0000","last_peered":"2023-11-15T02:59:34.713662+0000","last_clean":"2023-11-15T02:59:34.713662+0000","last_became_active":"2023-11-15T02:59:34.705522+0000","last_became_peered":"2023-11-15T02:59:34.705522+0000","last_unstale":"2023-11-15T02:59:34.713662+0000","last_undegraded":"2023-11-15T02:59:34.713662+0000","last_fullsized":"2023-11-15T02:59:34.713662+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.689660+0000","last_change":"2023-11-15T02:59:34.274845+0000","last_active":"2023-11-15T02:59:34.689660+0000","last_peered":"2023-11-15T02:59:34.689660+0000","last_clean":"2023-11-15T02:59:34.689660+0000","last_became_active":"2023-11-15T02:59:34.274495+0000","last_became_peered":"2023-11-15T02:59:34.274495+0000","last_unstale":"2023-11-15T02:59:34.689660+0000","last_undegraded":"2023-11-15T02:59:34.689660+0000","last_fullsized":"2023-11-15T02:59:34.689660+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.723692+0000","last_change":"2023-11-15T02:59:34.720105+0000","last_active":"2023-11-15T02:59:34.723692+0000","last_peered":"2023-11-15T02:59:34.723692+0000","last_clean":"2023-11-15T02:59:34.723692+0000","last_became_active":"2023-11-15T02:59:34.719807+0000","last_became_peered":"2023-11-15T02:59:34.719807+0000","last_unstale":"2023-11-15T02:59:34.723692+0000","last_undegraded":"2023-11-15T02:59:34.723692+0000","last_fullsized":"2023-11-15T02:59:34.723692+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.692197+0000","last_change":"2023-11-15T02:59:34.262355+0000","last_active":"2023-11-15T02:59:34.692197+0000","last_peered":"2023-11-15T02:59:34.692197+0000","last_clean":"2023-11-15T02:59:34.692197+0000","last_became_active":"2023-11-15T02:59:34.260844+0000","last_became_peered":"2023-11-15T02:59:34.260844+0000","last_unstale":"2023-11-15T02:59:34.692197+0000","last_undegraded":"2023-11-15T02:59:34.692197+0000","last_fullsized":"2023-11-15T02:59:34.692197+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.721789+0000","last_change":"2023-11-15T02:59:34.711149+0000","last_active":"2023-11-15T02:59:34.721789+0000","last_peered":"2023-11-15T02:59:34.721789+0000","last_clean":"2023-11-15T02:59:34.721789+0000","last_became_active":"2023-11-15T02:59:34.710867+0000","last_became_peered":"2023-11-15T02:59:34.710867+0000","last_unstale":"2023-11-15T02:59:34.721789+0000","last_undegraded":"2023-11-15T02:59:34.721789+0000","last_fullsized":"2023-11-15T02:59:34.721789+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.690031+0000","last_change":"2023-11-15T02:59:34.276394+0000","last_active":"2023-11-15T02:59:34.690031+0000","last_peered":"2023-11-15T02:59:34.690031+0000","last_clean":"2023-11-15T02:59:34.690031+0000","last_became_active":"2023-11-15T02:59:34.276134+0000","last_became_peered":"2023-11-15T02:59:34.276134+0000","last_unstale":"2023-11-15T02:59:34.690031+0000","last_undegraded":"2023-11-15T02:59:34.690031+0000","last_fullsized":"2023-11-15T02:59:34.690031+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.701902+0000","last_change":"2023-11-15T02:59:34.697057+0000","last_active":"2023-11-15T02:59:34.701902+0000","last_peered":"2023-11-15T02:59:34.701902+0000","last_clean":"2023-11-15T02:59:34.701902+0000","last_became_active":"2023-11-15T02:59:34.696211+0000","last_became_peered":"2023-11-15T02:59:34.696211+0000","last_unstale":"2023-11-15T02:59:34.701902+0000","last_undegraded":"2023-11-15T02:59:34.701902+0000","last_fullsized":"2023-11-15T02:59:34.701902+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.703503+0000","last_change":"2023-11-15T02:59:34.272228+0000","last_active":"2023-11-15T02:59:34.703503+0000","last_peered":"2023-11-15T02:59:34.703503+0000","last_clean":"2023-11-15T02:59:34.703503+0000","last_became_active":"2023-11-15T02:59:34.271741+0000","last_became_peered":"2023-11-15T02:59:34.271741+0000","last_unstale":"2023-11-15T02:59:34.703503+0000","last_undegraded":"2023-11-15T02:59:34.703503+0000","last_fullsized":"2023-11-15T02:59:34.703503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.710553+0000","last_change":"2023-11-15T02:59:34.695873+0000","last_active":"2023-11-15T02:59:34.710553+0000","last_peered":"2023-11-15T02:59:34.710553+0000","last_clean":"2023-11-15T02:59:34.710553+0000","last_became_active":"2023-11-15T02:59:34.695590+0000","last_became_peered":"2023-11-15T02:59:34.695590+0000","last_unstale":"2023-11-15T02:59:34.710553+0000","last_undegraded":"2023-11-15T02:59:34.710553+0000","last_fullsized":"2023-11-15T02:59:34.710553+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.718448+0000","last_change":"2023-11-15T02:59:34.272647+0000","last_active":"2023-11-15T02:59:34.718448+0000","last_peered":"2023-11-15T02:59:34.718448+0000","last_clean":"2023-11-15T02:59:34.718448+0000","last_became_active":"2023-11-15T02:59:34.272331+0000","last_became_peered":"2023-11-15T02:59:34.272331+0000","last_unstale":"2023-11-15T02:59:34.718448+0000","last_undegraded":"2023-11-15T02:59:34.718448+0000","last_fullsized":"2023-11-15T02:59:34.718448+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.714668+0000","last_change":"2023-11-15T02:59:34.710364+0000","last_active":"2023-11-15T02:59:34.714668+0000","last_peered":"2023-11-15T02:59:34.714668+0000","last_clean":"2023-11-15T02:59:34.714668+0000","last_became_active":"2023-11-15T02:59:34.706803+0000","last_became_peered":"2023-11-15T02:59:34.706803+0000","last_unstale":"2023-11-15T02:59:34.714668+0000","last_undegraded":"2023-11-15T02:59:34.714668+0000","last_fullsized":"2023-11-15T02:59:34.714668+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.711709+0000","last_change":"2023-11-15T02:59:34.263720+0000","last_active":"2023-11-15T02:59:34.711709+0000","last_peered":"2023-11-15T02:59:34.711709+0000","last_clean":"2023-11-15T02:59:34.711709+0000","last_became_active":"2023-11-15T02:59:34.262994+0000","last_became_peered":"2023-11-15T02:59:34.262994+0000","last_unstale":"2023-11-15T02:59:34.711709+0000","last_undegraded":"2023-11-15T02:59:34.711709+0000","last_fullsized":"2023-11-15T02:59:34.711709+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.708903+0000","last_change":"2023-11-15T02:59:34.702473+0000","last_active":"2023-11-15T02:59:34.708903+0000","last_peered":"2023-11-15T02:59:34.708903+0000","last_clean":"2023-11-15T02:59:34.708903+0000","last_became_active":"2023-11-15T02:59:34.702094+0000","last_became_peered":"2023-11-15T02:59:34.702094+0000","last_unstale":"2023-11-15T02:59:34.708903+0000","last_undegraded":"2023-11-15T02:59:34.708903+0000","last_fullsized":"2023-11-15T02:59:34.708903+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.691931+0000","last_change":"2023-11-15T02:59:34.260019+0000","last_active":"2023-11-15T02:59:34.691931+0000","last_peered":"2023-11-15T02:59:34.691931+0000","last_clean":"2023-11-15T02:59:34.691931+0000","last_became_active":"2023-11-15T02:59:34.259470+0000","last_became_peered":"2023-11-15T02:59:34.259470+0000","last_unstale":"2023-11-15T02:59:34.691931+0000","last_undegraded":"2023-11-15T02:59:34.691931+0000","last_fullsized":"2023-11-15T02:59:34.691931+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.711523+0000","last_change":"2023-11-15T02:59:34.707408+0000","last_active":"2023-11-15T02:59:34.711523+0000","last_peered":"2023-11-15T02:59:34.711523+0000","last_clean":"2023-11-15T02:59:34.711523+0000","last_became_active":"2023-11-15T02:59:34.706951+0000","last_became_peered":"2023-11-15T02:59:34.706951+0000","last_unstale":"2023-11-15T02:59:34.711523+0000","last_undegraded":"2023-11-15T02:59:34.711523+0000","last_fullsized":"2023-11-15T02:59:34.711523+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.678094+0000","last_change":"2023-11-15T02:59:34.276164+0000","last_active":"2023-11-15T02:59:34.678094+0000","last_peered":"2023-11-15T02:59:34.678094+0000","last_clean":"2023-11-15T02:59:34.678094+0000","last_became_active":"2023-11-15T02:59:34.275881+0000","last_became_peered":"2023-11-15T02:59:34.275881+0000","last_unstale":"2023-11-15T02:59:34.678094+0000","last_undegraded":"2023-11-15T02:59:34.678094+0000","last_fullsized":"2023-11-15T02:59:34.678094+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.714689+0000","last_change":"2023-11-15T02:59:34.711382+0000","last_active":"2023-11-15T02:59:34.714689+0000","last_peered":"2023-11-15T02:59:34.714689+0000","last_clean":"2023-11-15T02:59:34.714689+0000","last_became_active":"2023-11-15T02:59:34.710832+0000","last_became_peered":"2023-11-15T02:59:34.710832+0000","last_unstale":"2023-11-15T02:59:34.714689+0000","last_undegraded":"2023-11-15T02:59:34.714689+0000","last_fullsized":"2023-11-15T02:59:34.714689+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.679125+0000","last_change":"2023-11-15T02:59:34.269967+0000","last_active":"2023-11-15T02:59:34.679125+0000","last_peered":"2023-11-15T02:59:34.679125+0000","last_clean":"2023-11-15T02:59:34.679125+0000","last_became_active":"2023-11-15T02:59:34.269516+0000","last_became_peered":"2023-11-15T02:59:34.269516+0000","last_unstale":"2023-11-15T02:59:34.679125+0000","last_undegraded":"2023-11-15T02:59:34.679125+0000","last_fullsized":"2023-11-15T02:59:34.679125+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.722294+0000","last_change":"2023-11-15T02:59:34.713129+0000","last_active":"2023-11-15T02:59:34.722294+0000","last_peered":"2023-11-15T02:59:34.722294+0000","last_clean":"2023-11-15T02:59:34.722294+0000","last_became_active":"2023-11-15T02:59:34.704067+0000","last_became_peered":"2023-11-15T02:59:34.704067+0000","last_unstale":"2023-11-15T02:59:34.722294+0000","last_undegraded":"2023-11-15T02:59:34.722294+0000","last_fullsized":"2023-11-15T02:59:34.722294+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.678099+0000","last_change":"2023-11-15T02:59:34.272334+0000","last_active":"2023-11-15T02:59:34.678099+0000","last_peered":"2023-11-15T02:59:34.678099+0000","last_clean":"2023-11-15T02:59:34.678099+0000","last_became_active":"2023-11-15T02:59:34.272096+0000","last_became_peered":"2023-11-15T02:59:34.272096+0000","last_unstale":"2023-11-15T02:59:34.678099+0000","last_undegraded":"2023-11-15T02:59:34.678099+0000","last_fullsized":"2023-11-15T02:59:34.678099+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.689414+0000","last_change":"2023-11-15T02:59:34.686879+0000","last_active":"2023-11-15T02:59:34.689414+0000","last_peered":"2023-11-15T02:59:34.689414+0000","last_clean":"2023-11-15T02:59:34.689414+0000","last_became_active":"2023-11-15T02:59:34.685697+0000","last_became_peered":"2023-11-15T02:59:34.685697+0000","last_unstale":"2023-11-15T02:59:34.689414+0000","last_undegraded":"2023-11-15T02:59:34.689414+0000","last_fullsized":"2023-11-15T02:59:34.689414+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.689660+0000","last_change":"2023-11-15T02:59:34.270391+0000","last_active":"2023-11-15T02:59:34.689660+0000","last_peered":"2023-11-15T02:59:34.689660+0000","last_clean":"2023-11-15T02:59:34.689660+0000","last_became_active":"2023-11-15T02:59:34.270049+0000","last_became_peered":"2023-11-15T02:59:34.270049+0000","last_unstale":"2023-11-15T02:59:34.689660+0000","last_undegraded":"2023-11-15T02:59:34.689660+0000","last_fullsized":"2023-11-15T02:59:34.689660+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.712401+0000","last_change":"2023-11-15T02:59:34.700113+0000","last_active":"2023-11-15T02:59:34.712401+0000","last_peered":"2023-11-15T02:59:34.712401+0000","last_clean":"2023-11-15T02:59:34.712401+0000","last_became_active":"2023-11-15T02:59:34.699860+0000","last_became_peered":"2023-11-15T02:59:34.699860+0000","last_unstale":"2023-11-15T02:59:34.712401+0000","last_undegraded":"2023-11-15T02:59:34.712401+0000","last_fullsized":"2023-11-15T02:59:34.712401+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.688565+0000","last_change":"2023-11-15T02:59:34.280324+0000","last_active":"2023-11-15T02:59:34.688565+0000","last_peered":"2023-11-15T02:59:34.688565+0000","last_clean":"2023-11-15T02:59:34.688565+0000","last_became_active":"2023-11-15T02:59:34.280003+0000","last_became_peered":"2023-11-15T02:59:34.280003+0000","last_unstale":"2023-11-15T02:59:34.688565+0000","last_undegraded":"2023-11-15T02:59:34.688565+0000","last_fullsized":"2023-11-15T02:59:34.688565+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.710454+0000","last_change":"2023-11-15T02:59:34.706837+0000","last_active":"2023-11-15T02:59:34.710454+0000","last_peered":"2023-11-15T02:59:34.710454+0000","last_clean":"2023-11-15T02:59:34.710454+0000","last_became_active":"2023-11-15T02:59:34.706595+0000","last_became_peered":"2023-11-15T02:59:34.706595+0000","last_unstale":"2023-11-15T02:59:34.710454+0000","last_undegraded":"2023-11-15T02:59:34.710454+0000","last_fullsized":"2023-11-15T02:59:34.710454+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.685001+0000","last_change":"2023-11-15T02:59:34.274116+0000","last_active":"2023-11-15T02:59:34.685001+0000","last_peered":"2023-11-15T02:59:34.685001+0000","last_clean":"2023-11-15T02:59:34.685001+0000","last_became_active":"2023-11-15T02:59:34.273108+0000","last_became_peered":"2023-11-15T02:59:34.273108+0000","last_unstale":"2023-11-15T02:59:34.685001+0000","last_undegraded":"2023-11-15T02:59:34.685001+0000","last_fullsized":"2023-11-15T02:59:34.685001+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.722675+0000","last_change":"2023-11-15T02:59:34.713931+0000","last_active":"2023-11-15T02:59:34.722675+0000","last_peered":"2023-11-15T02:59:34.722675+0000","last_clean":"2023-11-15T02:59:34.722675+0000","last_became_active":"2023-11-15T02:59:34.713640+0000","last_became_peered":"2023-11-15T02:59:34.713640+0000","last_unstale":"2023-11-15T02:59:34.722675+0000","last_undegraded":"2023-11-15T02:59:34.722675+0000","last_fullsized":"2023-11-15T02:59:34.722675+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"60'3","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716863+0000","last_change":"2023-11-15T02:59:34.273818+0000","last_active":"2023-11-15T02:59:34.716863+0000","last_peered":"2023-11-15T02:59:34.716863+0000","last_clean":"2023-11-15T02:59:34.716863+0000","last_became_active":"2023-11-15T02:59:34.273463+0000","last_became_peered":"2023-11-15T02:59:34.273463+0000","last_unstale":"2023-11-15T02:59:34.716863+0000","last_undegraded":"2023-11-15T02:59:34.716863+0000","last_fullsized":"2023-11-15T02:59:34.716863+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.708636+0000","last_change":"2023-11-15T02:59:34.691994+0000","last_active":"2023-11-15T02:59:34.708636+0000","last_peered":"2023-11-15T02:59:34.708636+0000","last_clean":"2023-11-15T02:59:34.708636+0000","last_became_active":"2023-11-15T02:59:34.691573+0000","last_became_peered":"2023-11-15T02:59:34.691573+0000","last_unstale":"2023-11-15T02:59:34.708636+0000","last_undegraded":"2023-11-15T02:59:34.708636+0000","last_fullsized":"2023-11-15T02:59:34.708636+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.714362+0000","last_change":"2023-11-15T02:59:34.273189+0000","last_active":"2023-11-15T02:59:34.714362+0000","last_peered":"2023-11-15T02:59:34.714362+0000","last_clean":"2023-11-15T02:59:34.714362+0000","last_became_active":"2023-11-15T02:59:34.272734+0000","last_became_peered":"2023-11-15T02:59:34.272734+0000","last_unstale":"2023-11-15T02:59:34.714362+0000","last_undegraded":"2023-11-15T02:59:34.714362+0000","last_fullsized":"2023-11-15T02:59:34.714362+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.715557+0000","last_change":"2023-11-15T02:59:34.708893+0000","last_active":"2023-11-15T02:59:34.715557+0000","last_peered":"2023-11-15T02:59:34.715557+0000","last_clean":"2023-11-15T02:59:34.715557+0000","last_became_active":"2023-11-15T02:59:34.708623+0000","last_became_peered":"2023-11-15T02:59:34.708623+0000","last_unstale":"2023-11-15T02:59:34.715557+0000","last_undegraded":"2023-11-15T02:59:34.715557+0000","last_fullsized":"2023-11-15T02:59:34.715557+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.724605+0000","last_change":"2023-11-15T02:59:34.270907+0000","last_active":"2023-11-15T02:59:34.724605+0000","last_peered":"2023-11-15T02:59:34.724605+0000","last_clean":"2023-11-15T02:59:34.724605+0000","last_became_active":"2023-11-15T02:59:34.270561+0000","last_became_peered":"2023-11-15T02:59:34.270561+0000","last_unstale":"2023-11-15T02:59:34.724605+0000","last_undegraded":"2023-11-15T02:59:34.724605+0000","last_fullsized":"2023-11-15T02:59:34.724605+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.712809+0000","last_change":"2023-11-15T02:59:34.705948+0000","last_active":"2023-11-15T02:59:34.712809+0000","last_peered":"2023-11-15T02:59:34.712809+0000","last_clean":"2023-11-15T02:59:34.712809+0000","last_became_active":"2023-11-15T02:59:34.705666+0000","last_became_peered":"2023-11-15T02:59:34.705666+0000","last_unstale":"2023-11-15T02:59:34.712809+0000","last_undegraded":"2023-11-15T02:59:34.712809+0000","last_fullsized":"2023-11-15T02:59:34.712809+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.697233+0000","last_change":"2023-11-15T02:59:34.277797+0000","last_active":"2023-11-15T02:59:34.697233+0000","last_peered":"2023-11-15T02:59:34.697233+0000","last_clean":"2023-11-15T02:59:34.697233+0000","last_became_active":"2023-11-15T02:59:34.277536+0000","last_became_peered":"2023-11-15T02:59:34.277536+0000","last_unstale":"2023-11-15T02:59:34.697233+0000","last_undegraded":"2023-11-15T02:59:34.697233+0000","last_fullsized":"2023-11-15T02:59:34.697233+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.723770+0000","last_change":"2023-11-15T02:59:34.719857+0000","last_active":"2023-11-15T02:59:34.723770+0000","last_peered":"2023-11-15T02:59:34.723770+0000","last_clean":"2023-11-15T02:59:34.723770+0000","last_became_active":"2023-11-15T02:59:34.719591+0000","last_became_peered":"2023-11-15T02:59:34.719591+0000","last_unstale":"2023-11-15T02:59:34.723770+0000","last_undegraded":"2023-11-15T02:59:34.723770+0000","last_fullsized":"2023-11-15T02:59:34.723770+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.699375+0000","last_change":"2023-11-15T02:59:34.276260+0000","last_active":"2023-11-15T02:59:34.699375+0000","last_peered":"2023-11-15T02:59:34.699375+0000","last_clean":"2023-11-15T02:59:34.699375+0000","last_became_active":"2023-11-15T02:59:34.275758+0000","last_became_peered":"2023-11-15T02:59:34.275758+0000","last_unstale":"2023-11-15T02:59:34.699375+0000","last_undegraded":"2023-11-15T02:59:34.699375+0000","last_fullsized":"2023-11-15T02:59:34.699375+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.714570+0000","last_change":"2023-11-15T02:59:34.709342+0000","last_active":"2023-11-15T02:59:34.714570+0000","last_peered":"2023-11-15T02:59:34.714570+0000","last_clean":"2023-11-15T02:59:34.714570+0000","last_became_active":"2023-11-15T02:59:34.708937+0000","last_became_peered":"2023-11-15T02:59:34.708937+0000","last_unstale":"2023-11-15T02:59:34.714570+0000","last_undegraded":"2023-11-15T02:59:34.714570+0000","last_fullsized":"2023-11-15T02:59:34.714570+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"60'4","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.722505+0000","last_change":"2023-11-15T02:59:34.269037+0000","last_active":"2023-11-15T02:59:34.722505+0000","last_peered":"2023-11-15T02:59:34.722505+0000","last_clean":"2023-11-15T02:59:34.722505+0000","last_became_active":"2023-11-15T02:59:34.268123+0000","last_became_peered":"2023-11-15T02:59:34.268123+0000","last_unstale":"2023-11-15T02:59:34.722505+0000","last_undegraded":"2023-11-15T02:59:34.722505+0000","last_fullsized":"2023-11-15T02:59:34.722505+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.719846+0000","last_change":"2023-11-15T02:59:34.715434+0000","last_active":"2023-11-15T02:59:34.719846+0000","last_peered":"2023-11-15T02:59:34.719846+0000","last_clean":"2023-11-15T02:59:34.719846+0000","last_became_active":"2023-11-15T02:59:34.715198+0000","last_became_peered":"2023-11-15T02:59:34.715198+0000","last_unstale":"2023-11-15T02:59:34.719846+0000","last_undegraded":"2023-11-15T02:59:34.719846+0000","last_fullsized":"2023-11-15T02:59:34.719846+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.722345+0000","last_change":"2023-11-15T02:59:34.271705+0000","last_active":"2023-11-15T02:59:34.722345+0000","last_peered":"2023-11-15T02:59:34.722345+0000","last_clean":"2023-11-15T02:59:34.722345+0000","last_became_active":"2023-11-15T02:59:34.271393+0000","last_became_peered":"2023-11-15T02:59:34.271393+0000","last_unstale":"2023-11-15T02:59:34.722345+0000","last_undegraded":"2023-11-15T02:59:34.722345+0000","last_fullsized":"2023-11-15T02:59:34.722345+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.711505+0000","last_change":"2023-11-15T02:59:34.705591+0000","last_active":"2023-11-15T02:59:34.711505+0000","last_peered":"2023-11-15T02:59:34.711505+0000","last_clean":"2023-11-15T02:59:34.711505+0000","last_became_active":"2023-11-15T02:59:34.703826+0000","last_became_peered":"2023-11-15T02:59:34.703826+0000","last_unstale":"2023-11-15T02:59:34.711505+0000","last_undegraded":"2023-11-15T02:59:34.711505+0000","last_fullsized":"2023-11-15T02:59:34.711505+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.702284+0000","last_change":"2023-11-15T02:59:34.266622+0000","last_active":"2023-11-15T02:59:34.702284+0000","last_peered":"2023-11-15T02:59:34.702284+0000","last_clean":"2023-11-15T02:59:34.702284+0000","last_became_active":"2023-11-15T02:59:34.266349+0000","last_became_peered":"2023-11-15T02:59:34.266349+0000","last_unstale":"2023-11-15T02:59:34.702284+0000","last_undegraded":"2023-11-15T02:59:34.702284+0000","last_fullsized":"2023-11-15T02:59:34.702284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716077+0000","last_change":"2023-11-15T02:59:34.709700+0000","last_active":"2023-11-15T02:59:34.716077+0000","last_peered":"2023-11-15T02:59:34.716077+0000","last_clean":"2023-11-15T02:59:34.716077+0000","last_became_active":"2023-11-15T02:59:34.708353+0000","last_became_peered":"2023-11-15T02:59:34.708353+0000","last_unstale":"2023-11-15T02:59:34.716077+0000","last_undegraded":"2023-11-15T02:59:34.716077+0000","last_fullsized":"2023-11-15T02:59:34.716077+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.690128+0000","last_change":"2023-11-15T02:59:34.263759+0000","last_active":"2023-11-15T02:59:34.690128+0000","last_peered":"2023-11-15T02:59:34.690128+0000","last_clean":"2023-11-15T02:59:34.690128+0000","last_became_active":"2023-11-15T02:59:34.263333+0000","last_became_peered":"2023-11-15T02:59:34.263333+0000","last_unstale":"2023-11-15T02:59:34.690128+0000","last_undegraded":"2023-11-15T02:59:34.690128+0000","last_fullsized":"2023-11-15T02:59:34.690128+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.709615+0000","last_change":"2023-11-15T02:59:34.700382+0000","last_active":"2023-11-15T02:59:34.709615+0000","last_peered":"2023-11-15T02:59:34.709615+0000","last_clean":"2023-11-15T02:59:34.709615+0000","last_became_active":"2023-11-15T02:59:34.700091+0000","last_became_peered":"2023-11-15T02:59:34.700091+0000","last_unstale":"2023-11-15T02:59:34.709615+0000","last_undegraded":"2023-11-15T02:59:34.709615+0000","last_fullsized":"2023-11-15T02:59:34.709615+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.687840+0000","last_change":"2023-11-15T02:59:34.273003+0000","last_active":"2023-11-15T02:59:34.687840+0000","last_peered":"2023-11-15T02:59:34.687840+0000","last_clean":"2023-11-15T02:59:34.687840+0000","last_became_active":"2023-11-15T02:59:34.272361+0000","last_became_peered":"2023-11-15T02:59:34.272361+0000","last_unstale":"2023-11-15T02:59:34.687840+0000","last_undegraded":"2023-11-15T02:59:34.687840+0000","last_fullsized":"2023-11-15T02:59:34.687840+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.701945+0000","last_change":"2023-11-15T02:59:34.697102+0000","last_active":"2023-11-15T02:59:34.701945+0000","last_peered":"2023-11-15T02:59:34.701945+0000","last_clean":"2023-11-15T02:59:34.701945+0000","last_became_active":"2023-11-15T02:59:34.696095+0000","last_became_peered":"2023-11-15T02:59:34.696095+0000","last_unstale":"2023-11-15T02:59:34.701945+0000","last_undegraded":"2023-11-15T02:59:34.701945+0000","last_fullsized":"2023-11-15T02:59:34.701945+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.688346+0000","last_change":"2023-11-15T02:59:34.275904+0000","last_active":"2023-11-15T02:59:34.688346+0000","last_peered":"2023-11-15T02:59:34.688346+0000","last_clean":"2023-11-15T02:59:34.688346+0000","last_became_active":"2023-11-15T02:59:34.275561+0000","last_became_peered":"2023-11-15T02:59:34.275561+0000","last_unstale":"2023-11-15T02:59:34.688346+0000","last_undegraded":"2023-11-15T02:59:34.688346+0000","last_fullsized":"2023-11-15T02:59:34.688346+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.721871+0000","last_change":"2023-11-15T02:59:34.710944+0000","last_active":"2023-11-15T02:59:34.721871+0000","last_peered":"2023-11-15T02:59:34.721871+0000","last_clean":"2023-11-15T02:59:34.721871+0000","last_became_active":"2023-11-15T02:59:34.710640+0000","last_became_peered":"2023-11-15T02:59:34.710640+0000","last_unstale":"2023-11-15T02:59:34.721871+0000","last_undegraded":"2023-11-15T02:59:34.721871+0000","last_fullsized":"2023-11-15T02:59:34.721871+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.679251+0000","last_change":"2023-11-15T02:59:34.275717+0000","last_active":"2023-11-15T02:59:34.679251+0000","last_peered":"2023-11-15T02:59:34.679251+0000","last_clean":"2023-11-15T02:59:34.679251+0000","last_became_active":"2023-11-15T02:59:34.274314+0000","last_became_peered":"2023-11-15T02:59:34.274314+0000","last_unstale":"2023-11-15T02:59:34.679251+0000","last_undegraded":"2023-11-15T02:59:34.679251+0000","last_fullsized":"2023-11-15T02:59:34.679251+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.714395+0000","last_change":"2023-11-15T02:59:34.708647+0000","last_active":"2023-11-15T02:59:34.714395+0000","last_peered":"2023-11-15T02:59:34.714395+0000","last_clean":"2023-11-15T02:59:34.714395+0000","last_became_active":"2023-11-15T02:59:34.708182+0000","last_became_peered":"2023-11-15T02:59:34.708182+0000","last_unstale":"2023-11-15T02:59:34.714395+0000","last_undegraded":"2023-11-15T02:59:34.714395+0000","last_fullsized":"2023-11-15T02:59:34.714395+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.677885+0000","last_change":"2023-11-15T02:59:34.271643+0000","last_active":"2023-11-15T02:59:34.677885+0000","last_peered":"2023-11-15T02:59:34.677885+0000","last_clean":"2023-11-15T02:59:34.677885+0000","last_became_active":"2023-11-15T02:59:34.271375+0000","last_became_peered":"2023-11-15T02:59:34.271375+0000","last_unstale":"2023-11-15T02:59:34.677885+0000","last_undegraded":"2023-11-15T02:59:34.677885+0000","last_fullsized":"2023-11-15T02:59:34.677885+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.714024+0000","last_change":"2023-11-15T02:59:34.711599+0000","last_active":"2023-11-15T02:59:34.714024+0000","last_peered":"2023-11-15T02:59:34.714024+0000","last_clean":"2023-11-15T02:59:34.714024+0000","last_became_active":"2023-11-15T02:59:34.711325+0000","last_became_peered":"2023-11-15T02:59:34.711325+0000","last_unstale":"2023-11-15T02:59:34.714024+0000","last_undegraded":"2023-11-15T02:59:34.714024+0000","last_fullsized":"2023-11-15T02:59:34.714024+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.685117+0000","last_change":"2023-11-15T02:59:34.269929+0000","last_active":"2023-11-15T02:59:34.685117+0000","last_peered":"2023-11-15T02:59:34.685117+0000","last_clean":"2023-11-15T02:59:34.685117+0000","last_became_active":"2023-11-15T02:59:34.269560+0000","last_became_peered":"2023-11-15T02:59:34.269560+0000","last_unstale":"2023-11-15T02:59:34.685117+0000","last_undegraded":"2023-11-15T02:59:34.685117+0000","last_fullsized":"2023-11-15T02:59:34.685117+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.709598+0000","last_change":"2023-11-15T02:59:34.699657+0000","last_active":"2023-11-15T02:59:34.709598+0000","last_peered":"2023-11-15T02:59:34.709598+0000","last_clean":"2023-11-15T02:59:34.709598+0000","last_became_active":"2023-11-15T02:59:34.699371+0000","last_became_peered":"2023-11-15T02:59:34.699371+0000","last_unstale":"2023-11-15T02:59:34.709598+0000","last_undegraded":"2023-11-15T02:59:34.709598+0000","last_fullsized":"2023-11-15T02:59:34.709598+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.679356+0000","last_change":"2023-11-15T02:59:34.264863+0000","last_active":"2023-11-15T02:59:34.679356+0000","last_peered":"2023-11-15T02:59:34.679356+0000","last_clean":"2023-11-15T02:59:34.679356+0000","last_became_active":"2023-11-15T02:59:34.264228+0000","last_became_peered":"2023-11-15T02:59:34.264228+0000","last_unstale":"2023-11-15T02:59:34.679356+0000","last_undegraded":"2023-11-15T02:59:34.679356+0000","last_fullsized":"2023-11-15T02:59:34.679356+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":45,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.690231+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T02:59:34.690231+0000","last_peered":"2023-11-15T02:59:34.690231+0000","last_clean":"2023-11-15T02:59:34.690231+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T02:59:34.690231+0000","last_undegraded":"2023-11-15T02:59:34.690231+0000","last_fullsized":"2023-11-15T02:59:34.690231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.709996+0000","last_change":"2023-11-15T02:59:34.694895+0000","last_active":"2023-11-15T02:59:34.709996+0000","last_peered":"2023-11-15T02:59:34.709996+0000","last_clean":"2023-11-15T02:59:34.709996+0000","last_became_active":"2023-11-15T02:59:34.694592+0000","last_became_peered":"2023-11-15T02:59:34.694592+0000","last_unstale":"2023-11-15T02:59:34.709996+0000","last_undegraded":"2023-11-15T02:59:34.709996+0000","last_fullsized":"2023-11-15T02:59:34.709996+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.690944+0000","last_change":"2023-11-15T02:59:34.280250+0000","last_active":"2023-11-15T02:59:34.690944+0000","last_peered":"2023-11-15T02:59:34.690944+0000","last_clean":"2023-11-15T02:59:34.690944+0000","last_became_active":"2023-11-15T02:59:34.279961+0000","last_became_peered":"2023-11-15T02:59:34.279961+0000","last_unstale":"2023-11-15T02:59:34.690944+0000","last_undegraded":"2023-11-15T02:59:34.690944+0000","last_fullsized":"2023-11-15T02:59:34.690944+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.688454+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T02:59:34.688454+0000","last_peered":"2023-11-15T02:59:34.688454+0000","last_clean":"2023-11-15T02:59:34.688454+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T02:59:34.688454+0000","last_undegraded":"2023-11-15T02:59:34.688454+0000","last_fullsized":"2023-11-15T02:59:34.688454+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.723835+0000","last_change":"2023-11-15T02:59:34.720662+0000","last_active":"2023-11-15T02:59:34.723835+0000","last_peered":"2023-11-15T02:59:34.723835+0000","last_clean":"2023-11-15T02:59:34.723835+0000","last_became_active":"2023-11-15T02:59:34.713450+0000","last_became_peered":"2023-11-15T02:59:34.713450+0000","last_unstale":"2023-11-15T02:59:34.723835+0000","last_undegraded":"2023-11-15T02:59:34.723835+0000","last_fullsized":"2023-11-15T02:59:34.723835+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716981+0000","last_change":"2023-11-15T02:59:34.272609+0000","last_active":"2023-11-15T02:59:34.716981+0000","last_peered":"2023-11-15T02:59:34.716981+0000","last_clean":"2023-11-15T02:59:34.716981+0000","last_became_active":"2023-11-15T02:59:34.272212+0000","last_became_peered":"2023-11-15T02:59:34.272212+0000","last_unstale":"2023-11-15T02:59:34.716981+0000","last_undegraded":"2023-11-15T02:59:34.716981+0000","last_fullsized":"2023-11-15T02:59:34.716981+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.691030+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T02:59:34.691030+0000","last_peered":"2023-11-15T02:59:34.691030+0000","last_clean":"2023-11-15T02:59:34.691030+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T02:59:34.691030+0000","last_undegraded":"2023-11-15T02:59:34.691030+0000","last_fullsized":"2023-11-15T02:59:34.691030+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.719526+0000","last_change":"2023-11-15T02:59:34.711825+0000","last_active":"2023-11-15T02:59:34.719526+0000","last_peered":"2023-11-15T02:59:34.719526+0000","last_clean":"2023-11-15T02:59:34.719526+0000","last_became_active":"2023-11-15T02:59:34.711551+0000","last_became_peered":"2023-11-15T02:59:34.711551+0000","last_unstale":"2023-11-15T02:59:34.719526+0000","last_undegraded":"2023-11-15T02:59:34.719526+0000","last_fullsized":"2023-11-15T02:59:34.719526+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.716350+0000","last_change":"2023-11-15T02:59:34.273403+0000","last_active":"2023-11-15T02:59:34.716350+0000","last_peered":"2023-11-15T02:59:34.716350+0000","last_clean":"2023-11-15T02:59:34.716350+0000","last_became_active":"2023-11-15T02:59:34.271771+0000","last_became_peered":"2023-11-15T02:59:34.271771+0000","last_unstale":"2023-11-15T02:59:34.716350+0000","last_undegraded":"2023-11-15T02:59:34.716350+0000","last_fullsized":"2023-11-15T02:59:34.716350+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.687917+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T02:59:34.687917+0000","last_peered":"2023-11-15T02:59:34.687917+0000","last_clean":"2023-11-15T02:59:34.687917+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T02:59:34.687917+0000","last_undegraded":"2023-11-15T02:59:34.687917+0000","last_fullsized":"2023-11-15T02:59:34.687917+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.722258+0000","last_change":"2023-11-15T02:59:34.712699+0000","last_active":"2023-11-15T02:59:34.722258+0000","last_peered":"2023-11-15T02:59:34.722258+0000","last_clean":"2023-11-15T02:59:34.722258+0000","last_became_active":"2023-11-15T02:59:34.703686+0000","last_became_peered":"2023-11-15T02:59:34.703686+0000","last_unstale":"2023-11-15T02:59:34.722258+0000","last_undegraded":"2023-11-15T02:59:34.722258+0000","last_fullsized":"2023-11-15T02:59:34.722258+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.689049+0000","last_change":"2023-11-15T02:59:34.265955+0000","last_active":"2023-11-15T02:59:34.689049+0000","last_peered":"2023-11-15T02:59:34.689049+0000","last_clean":"2023-11-15T02:59:34.689049+0000","last_became_active":"2023-11-15T02:59:34.265681+0000","last_became_peered":"2023-11-15T02:59:34.265681+0000","last_unstale":"2023-11-15T02:59:34.689049+0000","last_undegraded":"2023-11-15T02:59:34.689049+0000","last_fullsized":"2023-11-15T02:59:34.689049+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":53,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.697416+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T02:59:34.697416+0000","last_peered":"2023-11-15T02:59:34.697416+0000","last_clean":"2023-11-15T02:59:34.697416+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T02:59:34.697416+0000","last_undegraded":"2023-11-15T02:59:34.697416+0000","last_fullsized":"2023-11-15T02:59:34.697416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.712838+0000","last_change":"2023-11-15T02:59:34.707313+0000","last_active":"2023-11-15T02:59:34.712838+0000","last_peered":"2023-11-15T02:59:34.712838+0000","last_clean":"2023-11-15T02:59:34.712838+0000","last_became_active":"2023-11-15T02:59:34.701575+0000","last_became_peered":"2023-11-15T02:59:34.701575+0000","last_unstale":"2023-11-15T02:59:34.712838+0000","last_undegraded":"2023-11-15T02:59:34.712838+0000","last_fullsized":"2023-11-15T02:59:34.712838+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.697334+0000","last_change":"2023-11-15T02:59:34.274214+0000","last_active":"2023-11-15T02:59:34.697334+0000","last_peered":"2023-11-15T02:59:34.697334+0000","last_clean":"2023-11-15T02:59:34.697334+0000","last_became_active":"2023-11-15T02:59:34.273920+0000","last_became_peered":"2023-11-15T02:59:34.273920+0000","last_unstale":"2023-11-15T02:59:34.697334+0000","last_undegraded":"2023-11-15T02:59:34.697334+0000","last_fullsized":"2023-11-15T02:59:34.697334+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.692293+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T02:59:34.692293+0000","last_peered":"2023-11-15T02:59:34.692293+0000","last_clean":"2023-11-15T02:59:34.692293+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T02:59:34.692293+0000","last_undegraded":"2023-11-15T02:59:34.692293+0000","last_fullsized":"2023-11-15T02:59:34.692293+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.712733+0000","last_change":"2023-11-15T02:59:34.704325+0000","last_active":"2023-11-15T02:59:34.712733+0000","last_peered":"2023-11-15T02:59:34.712733+0000","last_clean":"2023-11-15T02:59:34.712733+0000","last_became_active":"2023-11-15T02:59:34.703365+0000","last_became_peered":"2023-11-15T02:59:34.703365+0000","last_unstale":"2023-11-15T02:59:34.712733+0000","last_undegraded":"2023-11-15T02:59:34.712733+0000","last_fullsized":"2023-11-15T02:59:34.712733+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.715001+0000","last_change":"2023-11-15T02:59:34.265022+0000","last_active":"2023-11-15T02:59:34.715001+0000","last_peered":"2023-11-15T02:59:34.715001+0000","last_clean":"2023-11-15T02:59:34.715001+0000","last_became_active":"2023-11-15T02:59:34.264125+0000","last_became_peered":"2023-11-15T02:59:34.264125+0000","last_unstale":"2023-11-15T02:59:34.715001+0000","last_undegraded":"2023-11-15T02:59:34.715001+0000","last_fullsized":"2023-11-15T02:59:34.715001+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.696497+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T02:59:34.696497+0000","last_peered":"2023-11-15T02:59:34.696497+0000","last_clean":"2023-11-15T02:59:34.696497+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T02:59:34.696497+0000","last_undegraded":"2023-11-15T02:59:34.696497+0000","last_fullsized":"2023-11-15T02:59:34.696497+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.714194+0000","last_change":"2023-11-15T02:59:34.697027+0000","last_active":"2023-11-15T02:59:34.714194+0000","last_peered":"2023-11-15T02:59:34.714194+0000","last_clean":"2023-11-15T02:59:34.714194+0000","last_became_active":"2023-11-15T02:59:34.696199+0000","last_became_peered":"2023-11-15T02:59:34.696199+0000","last_unstale":"2023-11-15T02:59:34.714194+0000","last_undegraded":"2023-11-15T02:59:34.714194+0000","last_fullsized":"2023-11-15T02:59:34.714194+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:34.221558+0000","last_clean_scrub_stamp":"2023-11-15T02:59:34.221558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.677918+0000","last_change":"2023-11-15T02:59:34.279512+0000","last_active":"2023-11-15T02:59:34.677918+0000","last_peered":"2023-11-15T02:59:34.677918+0000","last_clean":"2023-11-15T02:59:34.677918+0000","last_became_active":"2023-11-15T02:59:34.278967+0000","last_became_peered":"2023-11-15T02:59:34.278967+0000","last_unstale":"2023-11-15T02:59:34.677918+0000","last_undegraded":"2023-11-15T02:59:34.677918+0000","last_fullsized":"2023-11-15T02:59:34.677918+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:59:33.207077+0000","last_clean_scrub_stamp":"2023-11-15T02:59:33.207077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":53,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.700072+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T02:59:34.700072+0000","last_peered":"2023-11-15T02:59:34.700072+0000","last_clean":"2023-11-15T02:59:34.700072+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T02:59:34.700072+0000","last_undegraded":"2023-11-15T02:59:34.700072+0000","last_fullsized":"2023-11-15T02:59:34.700072+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":45,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-15T02:59:34.696576+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T02:59:34.696576+0000","last_peered":"2023-11-15T02:59:34.696576+0000","last_clean":"2023-11-15T02:59:34.696576+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T02:59:34.696576+0000","last_undegraded":"2023-11-15T02:59:34.696576+0000","last_fullsized":"2023-11-15T02:59:34.696576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542181,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":741376,"data_stored":327434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.1679999999999999,"15min":1.1859999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.458,"5min":3.952,"15min":3.952},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.046,"5min":1.1799999999999999,"15min":1.202},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4419999999999999,"5min":2.5,"15min":2.5},"last":1.016}]},{"osd":1,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.137,"15min":1.145},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.3740000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1739999999999999,"15min":1.1779999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.415,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.3049999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2150000000000001,"15min":1.2330000000000001},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.458,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2949999999999999,"15min":1.3069999999999999},"min":{"1min":0.97899999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.5149999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":0.92900000000000005}]},{"osd":3,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.3400000000000001,"15min":1.3740000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.381,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.175,"5min":1.2669999999999999,"15min":1.282},"min":{"1min":0.90600000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.351,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.093}]},{"osd":4,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.2689999999999999,"15min":1.3},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.4359999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":1.117},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.448,"15min":1.4990000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.5029999999999999,"5min":4.226,"15min":4.226},"last":0.87}]},{"osd":5,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.329,"15min":1.343},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.556,"5min":2.024,"15min":2.024},"last":1.28},{"interface":"front","average":{"1min":1.234,"5min":1.4399999999999999,"15min":1.474},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.653,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.216}]},{"osd":6,"last update":"Wed Nov 15 02:58:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.5620000000000001,"15min":1.601},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":1.5660000000000001,"5min":3.2789999999999999,"15min":3.2789999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.591,"15min":1.645},"min":{"1min":1.123,"5min":1.123,"15min":1.123},"max":{"1min":1.4399999999999999,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.98099999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542181,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297812,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074028,"statfs":{"total":96636764160,"available":96331804672,"internally_reserved":0,"allocated":733184,"data_stored":326452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.125,"15min":1.133},"min":{"1min":0.69799999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.353,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.0469999999999999,"15min":1.0409999999999999},"min":{"1min":0.72499999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.484,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.077}]},{"osd":1,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0920000000000001,"15min":1.0840000000000001},"min":{"1min":0.876,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.415,"5min":1.623,"15min":1.623},"last":1.353},{"interface":"front","average":{"1min":1.173,"5min":1.169,"15min":1.169},"min":{"1min":0.80800000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.609,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.96299999999999997}]},{"osd":2,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.1870000000000001,"15min":1.1779999999999999},"min":{"1min":0.84399999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.496,"5min":2.621,"15min":2.621},"last":1.224},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.0960000000000001,"15min":1.0840000000000001},"min":{"1min":0.91700000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.425,"5min":1.5,"15min":1.5},"last":0.99199999999999999}]},{"osd":3,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.167,"15min":1.157},"min":{"1min":1.0029999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.492,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.24,"5min":1.26,"15min":1.2629999999999999},"min":{"1min":0.99299999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.6850000000000001,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.3}]},{"osd":4,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.173,"15min":1.1910000000000001},"min":{"1min":0.81399999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.4099999999999999,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.95599999999999996,"5min":0.625,"15min":0.625},"max":{"1min":1.4850000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.089}]},{"osd":5,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.417,"15min":1.411},"min":{"1min":1.155,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.772,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.3109999999999999,"15min":1.319},"min":{"1min":0.877,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6599999999999999,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":1.0289999999999999}]},{"osd":7,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2809999999999999,"15min":1.3009999999999999},"min":{"1min":0.82199999999999995,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.395,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.401,"5min":1.448,"15min":1.456},"min":{"1min":1.046,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8540000000000001,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":1.399}]}]},{"osd":4,"up_from":14,"seq":60129542182,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":737280,"data_stored":327062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.107,"15min":1.1120000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.516,"5min":1.532,"15min":1.532},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0069999999999999,"15min":1.0029999999999999},"min":{"1min":0.80700000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":1.3240000000000001}]},{"osd":1,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2250000000000001,"15min":1.2370000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4770000000000001,"5min":2.617,"15min":2.617},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.115,"5min":1.1559999999999999,"15min":1.163},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.425,"5min":1.5,"15min":1.5},"last":1.3939999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.1519999999999999,"15min":1.141},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.462,"5min":1.909,"15min":1.909},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1599999999999999,"15min":1.1639999999999999},"min":{"1min":0.88800000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.399,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.552}]},{"osd":3,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1519999999999999,"15min":1.1599999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.462,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.294,"15min":1.3},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.5840000000000001,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":1.169}]},{"osd":5,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2430000000000001,"15min":1.2529999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5780000000000001,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.52},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.2290000000000001,"15min":1.2350000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.5049999999999999,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.429}]},{"osd":6,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.333,"15min":1.343},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6619999999999999,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":1.28},{"interface":"front","average":{"1min":1.296,"5min":1.3979999999999999,"15min":1.415},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.7450000000000001,"5min":2.173,"15min":2.173},"last":1.2170000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.369,"15min":1.381},"min":{"1min":1.069,"5min":0.87,"15min":0.87},"max":{"1min":1.5820000000000001,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":1.617},{"interface":"front","average":{"1min":1.371,"5min":1.4350000000000001,"15min":1.4450000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.587,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":1.3660000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542182,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":737280,"data_stored":327042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.77400000000000002,"15min":0.75900000000000001},"min":{"1min":0.64900000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.84199999999999997,"15min":0.82899999999999996},"min":{"1min":0.73599999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1779999999999999,"5min":1.26,"15min":1.26},"last":0.752}]},{"osd":1,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.90000000000000002,"15min":0.90100000000000002},"min":{"1min":0.621,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.97999999999999998,"15min":0.96999999999999997},"min":{"1min":0.83299999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2749999999999999,"5min":1.375,"15min":1.375},"last":1.1299999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97699999999999998,"15min":0.96799999999999997},"min":{"1min":0.76700000000000002,"5min":0.63,"15min":0.63},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0649999999999999,"15min":1.052},"min":{"1min":0.85899999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.246}]},{"osd":4,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1859999999999999,"15min":1.1799999999999999},"min":{"1min":0.97899999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.456,"5min":1.837,"15min":1.837},"last":1.226},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.22,"15min":1.226},"min":{"1min":1.0249999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.492,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.038}]},{"osd":5,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.27,"15min":1.2669999999999999},"min":{"1min":0.996,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.6459999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.054},{"interface":"front","average":{"1min":1.345,"5min":1.2789999999999999,"15min":1.268},"min":{"1min":1.01,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":1.304}]},{"osd":6,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3120000000000001,"15min":1.306},"min":{"1min":1.1060000000000001,"5min":1.048,"15min":1.048},"max":{"1min":1.552,"5min":1.794,"15min":1.794},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.3580000000000001,"15min":1.3520000000000001},"min":{"1min":1.129,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":1.647,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.405}]},{"osd":7,"last update":"Wed Nov 15 02:58:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.3919999999999999,"15min":1.3899999999999999},"min":{"1min":1.083,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.365,"15min":1.357},"min":{"1min":1.1819999999999999,"5min":1.085,"15min":1.085},"max":{"1min":1.665,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.171}]}]},{"osd":2,"up_from":14,"seq":60129542182,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297812,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074028,"statfs":{"total":96636764160,"available":96331804672,"internally_reserved":0,"allocated":733184,"data_stored":325773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76700000000000002,"15min":0.76400000000000001},"min":{"1min":0.61699999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88700000000000001,"15min":0.88200000000000001},"min":{"1min":0.68300000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.61899999999999999}]},{"osd":1,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89600000000000002,"15min":0.89200000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3120000000000001,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.96099999999999997,"15min":0.94899999999999995},"min":{"1min":0.83499999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.84299999999999997}]},{"osd":3,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.002,"15min":1},"min":{"1min":0.79200000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.387,"5min":1.472,"15min":1.472},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1259999999999999,"15min":1.1200000000000001},"min":{"1min":0.93000000000000005,"5min":0.622,"15min":0.622},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.181}]},{"osd":4,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.208,"15min":1.21},"min":{"1min":1.0149999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.48,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.274},{"interface":"front","average":{"1min":1.286,"5min":1.238,"15min":1.23},"min":{"1min":1.0760000000000001,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.5429999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.169}]},{"osd":5,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.262,"15min":1.264},"min":{"1min":1.0149999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":1.613,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2989999999999999,"15min":1.2969999999999999},"min":{"1min":1.0700000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.6890000000000001,"5min":1.857,"15min":1.857},"last":1.377}]},{"osd":6,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.367,"15min":1.373},"min":{"1min":1.056,"5min":1.056,"15min":1.056},"max":{"1min":1.72,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.347,"15min":1.353},"min":{"1min":1.046,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":1.573,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.3480000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:45 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3939999999999999,"15min":1.391},"min":{"1min":1.1719999999999999,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":1.8069999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.321},{"interface":"front","average":{"1min":1.306,"5min":1.3799999999999999,"15min":1.3919999999999999},"min":{"1min":1.097,"5min":1.097,"15min":1.097},"max":{"1min":1.5529999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.214}]}]},{"osd":0,"up_from":14,"seq":60129542182,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297808,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074032,"statfs":{"total":96636764160,"available":96331808768,"internally_reserved":0,"allocated":729088,"data_stored":326418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.84899999999999998,"15min":0.86899999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1990000000000001,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.167},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.89700000000000002,"15min":0.89600000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1619999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":1.02}]},{"osd":2,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.99399999999999999,"15min":1.0129999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.147,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":1.054},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.99199999999999999,"15min":0.98799999999999999},"min":{"1min":0.68000000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.2529999999999999,"5min":1.847,"15min":1.847},"last":0.93799999999999994}]},{"osd":3,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.96499999999999997,"15min":0.96199999999999997},"min":{"1min":0.76600000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.401,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.052,"5min":1.157,"15min":1.175},"min":{"1min":0.68899999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2210000000000001,"5min":3.54,"15min":3.54},"last":0.87}]},{"osd":4,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2490000000000001,"15min":1.2470000000000001},"min":{"1min":1.0720000000000001,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.6319999999999999,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.417},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.175,"15min":1.175},"min":{"1min":1.006,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.4850000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.2150000000000001}]},{"osd":5,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2170000000000001,"15min":1.218},"min":{"1min":0.98899999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.393,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.286},{"interface":"front","average":{"1min":1.24,"5min":1.3420000000000001,"15min":1.359},"min":{"1min":1.083,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.375,"5min":3.6760000000000002,"15min":3.6760000000000002},"last":1.262}]},{"osd":6,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.411,"15min":1.429},"min":{"1min":1.113,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":1.4730000000000001,"5min":3.71,"15min":3.71},"last":1.9770000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.3340000000000001,"15min":1.343},"min":{"1min":1.05,"5min":1.024,"15min":1.024},"max":{"1min":1.4970000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.8899999999999999}]},{"osd":7,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.3640000000000001,"15min":1.379},"min":{"1min":1.018,"5min":1.018,"15min":1.018},"max":{"1min":1.554,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2649999999999999,"15min":1.2609999999999999},"min":{"1min":1.1040000000000001,"5min":0.876,"15min":0.876},"max":{"1min":1.5449999999999999,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.4399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":737280,"data_stored":327332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.05,"15min":1.046},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4359999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.216,"15min":1.2370000000000001},"min":{"1min":0.69699999999999995,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.4630000000000001,"5min":3.46,"15min":3.46},"last":0.85399999999999998}]},{"osd":1,"last update":"Wed Nov 15 02:58:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.2310000000000001,"15min":1.2589999999999999},"min":{"1min":0.82699999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.47,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.091,"5min":1.1739999999999999,"15min":1.1879999999999999},"min":{"1min":0.80300000000000005,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.4570000000000001,"5min":3.516,"15min":3.516},"last":1.0009999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.2070000000000001,"15min":1.2130000000000001},"min":{"1min":0.91600000000000004,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.3640000000000001,"5min":5.4180000000000001,"15min":5.4180000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3069999999999999,"15min":1.321},"min":{"1min":0.94799999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.5089999999999999,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":1.0640000000000001}]},{"osd":3,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.29,"15min":1.3080000000000001},"min":{"1min":1.022,"5min":0.879,"15min":0.879},"max":{"1min":1.4850000000000001,"5min":2.141,"15min":2.141},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.448,"15min":1.464},"min":{"1min":1.1839999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":1.583,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":1.083}]},{"osd":4,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3680000000000001,"15min":1.3879999999999999},"min":{"1min":0.84199999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.5309999999999999,"5min":3.706,"15min":3.706},"last":1.167},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.597,"15min":1.6379999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6479999999999999,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":0.753}]},{"osd":6,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.327,"15min":1.3420000000000001},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.6439999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.546,"15min":1.5860000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.613,"5min":3.641,"15min":3.641},"last":1.2490000000000001}]},{"osd":7,"last update":"Wed Nov 15 02:58:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4259999999999999,"15min":1.4370000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.762,"5min":3.516,"15min":3.516},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.46,"15min":1.4650000000000001},"min":{"1min":1.139,"5min":1.022,"15min":1.022},"max":{"1min":1.722,"5min":3.754,"15min":3.754},"last":1.2290000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297824,"kb_used_data":728,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":745472,"data_stored":327174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.84999999999999998,"15min":0.84599999999999997},"min":{"1min":0.63600000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2150000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.72199999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.155,"5min":1.419,"15min":1.419},"last":0.99299999999999999}]},{"osd":2,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.85799999999999998,"15min":0.84799999999999998},"min":{"1min":0.495,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.90700000000000003,"15min":0.91000000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.161,"5min":1.341,"15min":1.341},"last":0.94799999999999995}]},{"osd":3,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.0620000000000001,"15min":1.054},"min":{"1min":0.54100000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.194},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.98899999999999999,"15min":0.97699999999999998},"min":{"1min":0.67800000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":1.1020000000000001}]},{"osd":4,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.175,"15min":1.165},"min":{"1min":0.75700000000000001,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.49,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.248,"15min":1.2589999999999999},"min":{"1min":0.92000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.5609999999999999,"5min":2.206,"15min":2.206},"last":1.0800000000000001}]},{"osd":5,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.26,"15min":1.2649999999999999},"min":{"1min":1.0089999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.4299999999999999,"5min":1.899,"15min":1.899},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.2829999999999999,"15min":1.2909999999999999},"min":{"1min":0.92800000000000005,"5min":0.871,"15min":0.871},"max":{"1min":1.5029999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.151}]},{"osd":6,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.323,"15min":1.325},"min":{"1min":0.98299999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.5229999999999999,"5min":1.825,"15min":1.825},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.4099999999999999,"15min":1.4119999999999999},"min":{"1min":1.1870000000000001,"5min":0.87,"15min":0.87},"max":{"1min":1.5529999999999999,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.204}]},{"osd":7,"last update":"Wed Nov 15 02:58:48 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.3200000000000001,"15min":1.3169999999999999},"min":{"1min":0.93999999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.6839999999999999,"5min":1.772,"15min":1.772},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3540000000000001,"15min":1.3600000000000001},"min":{"1min":0.95499999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.6120000000000001,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.2869999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T02:59:44.015 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-15T02:59:44.016 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-15T02:59:44.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.217+0000 7f72e9cb9700 1 Processor -- start 2023-11-15T02:59:44.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.217+0000 7f72e9cb9700 1 -- start start 2023-11-15T02:59:44.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.221+0000 7f72e9cb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72e40ab0d0 0x7f72e40ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:44.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.221+0000 7f72e9cb9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f72e40ad1b0 0x7f72e4065710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:44.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e9cb9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e4066c90 con 0x7f72e4065c00 2023-11-15T02:59:44.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e9cb9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e4066080 con 0x7f72e40ad1b0 2023-11-15T02:59:44.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e9cb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e4066360 con 0x7f72e40ab0d0 2023-11-15T02:59:44.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e3fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f72e40ad1b0 0x7f72e4065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:44.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e8cb7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72e40ab0d0 0x7f72e40ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:44.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e3fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f72e40ad1b0 0x7f72e4065710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39418/0 (socket says 172.21.15.121:39418) 2023-11-15T02:59:44.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e3fff700 1 -- 172.21.15.121:0/58902700 learned_addr learned my addr 172.21.15.121:0/58902700 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:44.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e3fff700 1 -- 172.21.15.121:0/58902700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72e40ab0d0 msgr2=0x7f72e40ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:44.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e3fff700 1 --2- 172.21.15.121:0/58902700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72e40ab0d0 0x7f72e40ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:44.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e3fff700 1 -- 172.21.15.121:0/58902700 >> v1:172.21.15.121:6789/0 conn(0x7f72e4065c00 legacy=0x7f72e40c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:44.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e3fff700 1 -- 172.21.15.121:0/58902700 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e4066640 con 0x7f72e40ad1b0 2023-11-15T02:59:44.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e3fff700 1 --2- 172.21.15.121:0/58902700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72e40ad1b0 0x7f72e4065710 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f72d0006fd0 tx=0x7f72d00090f0).ready entity=mon.0 client_cookie=9dd660f2539791b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:44.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e37fe700 1 -- 172.21.15.121:0/58902700 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d000b580 con 0x7f72e40ad1b0 2023-11-15T02:59:44.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e37fe700 1 -- 172.21.15.121:0/58902700 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d000b6e0 con 0x7f72e40ad1b0 2023-11-15T02:59:44.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e37fe700 1 -- 172.21.15.121:0/58902700 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d000ba60 con 0x7f72e40ad1b0 2023-11-15T02:59:44.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e9cb9700 1 -- 172.21.15.121:0/58902700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72e40ad1b0 msgr2=0x7f72e4065710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:44.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e9cb9700 1 --2- 172.21.15.121:0/58902700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72e40ad1b0 0x7f72e4065710 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f72d0006fd0 tx=0x7f72d00090f0).stop 2023-11-15T02:59:44.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e9cb9700 1 -- 172.21.15.121:0/58902700 shutdown_connections 2023-11-15T02:59:44.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e9cb9700 1 --2- 172.21.15.121:0/58902700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72e40ad1b0 0x7f72e4065710 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:44.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e9cb9700 1 --2- 172.21.15.121:0/58902700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72e40ab0d0 0x7f72e40ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:44.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e9cb9700 1 -- 172.21.15.121:0/58902700 >> 172.21.15.121:0/58902700 conn(0x7f72e4067a60 msgr2=0x7f72e4067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:44.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e9cb9700 1 -- 172.21.15.121:0/58902700 shutdown_connections 2023-11-15T02:59:44.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.225+0000 7f72e9cb9700 1 -- 172.21.15.121:0/58902700 wait complete. 2023-11-15T02:59:44.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e9cb9700 1 Processor -- start 2023-11-15T02:59:44.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e9cb9700 1 -- start start 2023-11-15T02:59:44.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e9cb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72e4065c00 0x7f72e415a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:44.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e9cb9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72e40ab0d0 0x7f72e4167fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:44.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e9cb9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72e40ad1b0 0x7f72e4154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:44.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e9cb9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e4168490 con 0x7f72e40ab0d0 2023-11-15T02:59:44.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e9cb9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e415aed0 con 0x7f72e40ad1b0 2023-11-15T02:59:44.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e9cb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e40a4d50 con 0x7f72e4065c00 2023-11-15T02:59:44.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72e40ab0d0 0x7f72e4167fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:44.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72e40ab0d0 0x7f72e4167fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39424/0 (socket says 172.21.15.121:39424) 2023-11-15T02:59:44.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e3fff700 1 -- 172.21.15.121:0/3889542755 learned_addr learned my addr 172.21.15.121:0/3889542755 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:44.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e8cb7700 1 --2- 172.21.15.121:0/3889542755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72e4065c00 0x7f72e415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:44.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e3fff700 1 -- 172.21.15.121:0/3889542755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72e4065c00 msgr2=0x7f72e415a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:44.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e3fff700 1 --2- 172.21.15.121:0/3889542755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72e4065c00 0x7f72e415a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:44.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e3fff700 1 -- 172.21.15.121:0/3889542755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72e40ad1b0 msgr2=0x7f72e4154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:44.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e94b8700 1 --2- 172.21.15.121:0/3889542755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72e40ad1b0 0x7f72e4154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:44.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e3fff700 1 --2- 172.21.15.121:0/3889542755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72e40ad1b0 0x7f72e4154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:44.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e3fff700 1 -- 172.21.15.121:0/3889542755 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d0009370 con 0x7f72e40ab0d0 2023-11-15T02:59:44.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e8cb7700 1 --2- 172.21.15.121:0/3889542755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72e4065c00 0x7f72e415a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:44.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e94b8700 1 --2- 172.21.15.121:0/3889542755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72e40ad1b0 0x7f72e4154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:44.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.229+0000 7f72e3fff700 1 --2- 172.21.15.121:0/3889542755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72e40ab0d0 0x7f72e4167fa0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f72d0009130 tx=0x7f72d0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:44.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.233+0000 7f72e0ff9700 1 -- 172.21.15.121:0/3889542755 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d000b850 con 0x7f72e40ab0d0 2023-11-15T02:59:44.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.233+0000 7f72e0ff9700 1 -- 172.21.15.121:0/3889542755 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d000b9b0 con 0x7f72e40ab0d0 2023-11-15T02:59:44.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.233+0000 7f72e9cb9700 1 -- 172.21.15.121:0/3889542755 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e40a5030 con 0x7f72e40ab0d0 2023-11-15T02:59:44.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.233+0000 7f72e0ff9700 1 -- 172.21.15.121:0/3889542755 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d001d680 con 0x7f72e40ab0d0 2023-11-15T02:59:44.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.233+0000 7f72e9cb9700 1 -- 172.21.15.121:0/3889542755 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e40a55f0 con 0x7f72e40ab0d0 2023-11-15T02:59:44.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.233+0000 7f72e9cb9700 1 -- 172.21.15.121:0/3889542755 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e406bcf0 con 0x7f72e40ab0d0 2023-11-15T02:59:44.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.233+0000 7f72e0ff9700 1 -- 172.21.15.121:0/3889542755 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f72d001da30 con 0x7f72e40ab0d0 2023-11-15T02:59:44.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.237+0000 7f72e0ff9700 1 --2- 172.21.15.121:0/3889542755 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f72c805a510 0x7f72c805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:44.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.237+0000 7f72e0ff9700 1 -- 172.21.15.121:0/3889542755 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f72d0076430 con 0x7f72e40ab0d0 2023-11-15T02:59:44.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.237+0000 7f72e8cb7700 1 --2- 172.21.15.121:0/3889542755 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f72c805a510 0x7f72c805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:44.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.237+0000 7f72e8cb7700 1 --2- 172.21.15.121:0/3889542755 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f72c805a510 0x7f72c805c9d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f72d8006d20 tx=0x7f72d8008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:44.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.241+0000 7f72e0ff9700 1 -- 172.21.15.121:0/3889542755 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f72d0049c10 con 0x7f72e40ab0d0 2023-11-15T02:59:44.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.457+0000 7f72e9cb9700 1 -- 172.21.15.121:0/3889542755 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f72e40c7160 con 0x7f72e40ab0d0 2023-11-15T02:59:44.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.461+0000 7f72e0ff9700 1 -- 172.21.15.121:0/3889542755 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f72d004d230 con 0x7f72e40ab0d0 2023-11-15T02:59:44.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.465+0000 7f72e9cb9700 1 -- 172.21.15.121:0/3889542755 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f72c805a510 msgr2=0x7f72c805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:44.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.465+0000 7f72e9cb9700 1 --2- 172.21.15.121:0/3889542755 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f72c805a510 0x7f72c805c9d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f72d8006d20 tx=0x7f72d8008040).stop 2023-11-15T02:59:44.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.465+0000 7f72e9cb9700 1 -- 172.21.15.121:0/3889542755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72e40ab0d0 msgr2=0x7f72e4167fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:44.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.465+0000 7f72e9cb9700 1 --2- 172.21.15.121:0/3889542755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72e40ab0d0 0x7f72e4167fa0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f72d0009130 tx=0x7f72d0017000).stop 2023-11-15T02:59:44.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.465+0000 7f72e9cb9700 1 -- 172.21.15.121:0/3889542755 shutdown_connections 2023-11-15T02:59:44.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.465+0000 7f72e9cb9700 1 --2- 172.21.15.121:0/3889542755 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f72c805a510 0x7f72c805c9d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:44.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.465+0000 7f72e9cb9700 1 --2- 172.21.15.121:0/3889542755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72e40ad1b0 0x7f72e4154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:44.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.465+0000 7f72e9cb9700 1 --2- 172.21.15.121:0/3889542755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72e40ab0d0 0x7f72e4167fa0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:44.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.465+0000 7f72e9cb9700 1 --2- 172.21.15.121:0/3889542755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72e4065c00 0x7f72e415a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:44.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.465+0000 7f72e9cb9700 1 -- 172.21.15.121:0/3889542755 >> 172.21.15.121:0/3889542755 conn(0x7f72e4067a60 msgr2=0x7f72e41541a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:44.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.465+0000 7f72e9cb9700 1 -- 172.21.15.121:0/3889542755 shutdown_connections 2023-11-15T02:59:44.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.465+0000 7f72e9cb9700 1 -- 172.21.15.121:0/3889542755 wait complete. 2023-11-15T02:59:44.477 INFO:teuthology.orchestra.run.smithi121.stderr:updated caps for client.0 2023-11-15T02:59:44.492 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:59:44.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.645+0000 7f5422369700 1 Processor -- start 2023-11-15T02:59:44.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.645+0000 7f5422369700 1 -- start start 2023-11-15T02:59:44.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.645+0000 7f5422369700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f541c0c06e0 0x7f541c153db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:44.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.645+0000 7f5422369700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541c0c1530 con 0x7f541c1539d0 2023-11-15T02:59:44.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.645+0000 7f5422369700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541c0c2ac0 con 0x7f541c0c2670 2023-11-15T02:59:44.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.645+0000 7f5422369700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f541c0c2d80 con 0x7f541c0c06e0 2023-11-15T02:59:44.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.645+0000 7f5413fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f541c0c06e0 0x7f541c153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:44.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.645+0000 7f5413fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f541c0c06e0 0x7f541c153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39430/0 (socket says 172.21.15.121:39430) 2023-11-15T02:59:44.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.645+0000 7f541bfff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f541c1539d0 0x7f541c0c19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51842/0 (socket says 172.21.15.121:51842) 2023-11-15T02:59:44.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.645+0000 7f5413fff700 1 -- 172.21.15.121:0/4129447042 learned_addr learned my addr 172.21.15.121:0/4129447042 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:44.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.645+0000 7f541b7fe700 1 -- 172.21.15.121:0/4129447042 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 9394935 0 0) 0x7f541c0c1530 con 0x7f541c1539d0 2023-11-15T02:59:44.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.645+0000 7f5413fff700 1 -- 172.21.15.121:0/4129447042 >> v1:172.21.15.190:6789/0 conn(0x7f541c0c2670 legacy=0x7f541c0c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:44.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.645+0000 7f5413fff700 1 -- 172.21.15.121:0/4129447042 >> v1:172.21.15.121:6789/0 conn(0x7f541c1539d0 legacy=0x7f541c0c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:44.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.645+0000 7f5413fff700 1 -- 172.21.15.121:0/4129447042 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c0c0ee0 con 0x7f541c0c06e0 2023-11-15T02:59:44.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.649+0000 7f5413fff700 1 --2- 172.21.15.121:0/4129447042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f541c0c06e0 0x7f541c153db0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f5404006fd0 tx=0x7f54040090f0).ready entity=mon.0 client_cookie=2df2908fe8182c0e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:44.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.649+0000 7f541b7fe700 1 -- 172.21.15.121:0/4129447042 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540400b600 con 0x7f541c0c06e0 2023-11-15T02:59:44.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.649+0000 7f541b7fe700 1 -- 172.21.15.121:0/4129447042 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540400b760 con 0x7f541c0c06e0 2023-11-15T02:59:44.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.649+0000 7f541b7fe700 1 -- 172.21.15.121:0/4129447042 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540400bae0 con 0x7f541c0c06e0 2023-11-15T02:59:44.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.649+0000 7f5422369700 1 -- 172.21.15.121:0/4129447042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f541c0c06e0 msgr2=0x7f541c153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:44.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.649+0000 7f5422369700 1 --2- 172.21.15.121:0/4129447042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f541c0c06e0 0x7f541c153db0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f5404006fd0 tx=0x7f54040090f0).stop 2023-11-15T02:59:44.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.649+0000 7f5422369700 1 -- 172.21.15.121:0/4129447042 shutdown_connections 2023-11-15T02:59:44.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.649+0000 7f5422369700 1 --2- 172.21.15.121:0/4129447042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f541c0c06e0 0x7f541c153db0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:44.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.649+0000 7f5422369700 1 -- 172.21.15.121:0/4129447042 >> 172.21.15.121:0/4129447042 conn(0x7f541c062190 msgr2=0x7f541c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:44.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.649+0000 7f5422369700 1 -- 172.21.15.121:0/4129447042 shutdown_connections 2023-11-15T02:59:44.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.649+0000 7f5422369700 1 -- 172.21.15.121:0/4129447042 wait complete. 2023-11-15T02:59:44.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.649+0000 7f5422369700 1 Processor -- start 2023-11-15T02:59:44.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f5422369700 1 -- start start 2023-11-15T02:59:44.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f5422369700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f541c0c06e0 0x7f541c13bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:44.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f5422369700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f541c0c2670 0x7f541c14fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:44.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f5422369700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f541c1539d0 0x7f541c0aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:44.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f5422369700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c13c1c0 con 0x7f541c0c06e0 2023-11-15T02:59:44.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f5422369700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c151940 con 0x7f541c1539d0 2023-11-15T02:59:44.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f5422369700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541c156020 con 0x7f541c0c2670 2023-11-15T02:59:44.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f541bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f541c0c06e0 0x7f541c13bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:44.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f541bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f541c0c06e0 0x7f541c13bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39432/0 (socket says 172.21.15.121:39432) 2023-11-15T02:59:44.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f541bfff700 1 -- 172.21.15.121:0/3367365510 learned_addr learned my addr 172.21.15.121:0/3367365510 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:44.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f541bfff700 1 -- 172.21.15.121:0/3367365510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f541c0c2670 msgr2=0x7f541c14fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:44.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f5413fff700 1 --2- 172.21.15.121:0/3367365510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f541c0c2670 0x7f541c14fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:44.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f5420906700 1 --2- 172.21.15.121:0/3367365510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f541c1539d0 0x7f541c0aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:44.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f541bfff700 1 --2- 172.21.15.121:0/3367365510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f541c0c2670 0x7f541c14fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:44.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f541bfff700 1 -- 172.21.15.121:0/3367365510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f541c1539d0 msgr2=0x7f541c0aad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:44.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f541bfff700 1 --2- 172.21.15.121:0/3367365510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f541c1539d0 0x7f541c0aad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:44.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f541bfff700 1 -- 172.21.15.121:0/3367365510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5404009370 con 0x7f541c0c06e0 2023-11-15T02:59:44.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f5413fff700 1 --2- 172.21.15.121:0/3367365510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f541c0c2670 0x7f541c14fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:44.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f5420906700 1 --2- 172.21.15.121:0/3367365510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f541c1539d0 0x7f541c0aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:44.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f541bfff700 1 --2- 172.21.15.121:0/3367365510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f541c0c06e0 0x7f541c13bcd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f540c00c490 tx=0x7f540c007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:44.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f5418ff9700 1 -- 172.21.15.121:0/3367365510 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c0072e0 con 0x7f541c0c06e0 2023-11-15T02:59:44.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f5418ff9700 1 -- 172.21.15.121:0/3367365510 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540c007440 con 0x7f541c0c06e0 2023-11-15T02:59:44.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f5418ff9700 1 -- 172.21.15.121:0/3367365510 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c005930 con 0x7f541c0c06e0 2023-11-15T02:59:44.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f5422369700 1 -- 172.21.15.121:0/3367365510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541c150d30 con 0x7f541c0c06e0 2023-11-15T02:59:44.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.653+0000 7f5422369700 1 -- 172.21.15.121:0/3367365510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541c1512f0 con 0x7f541c0c06e0 2023-11-15T02:59:44.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.657+0000 7f5422369700 1 -- 172.21.15.121:0/3367365510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541c0c20d0 con 0x7f541c0c06e0 2023-11-15T02:59:44.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.657+0000 7f5418ff9700 1 -- 172.21.15.121:0/3367365510 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f540c005ce0 con 0x7f541c0c06e0 2023-11-15T02:59:44.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.657+0000 7f5418ff9700 1 --2- 172.21.15.121:0/3367365510 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f53fc05a590 0x7f53fc05ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:44.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.657+0000 7f5418ff9700 1 -- 172.21.15.121:0/3367365510 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f540c073ff0 con 0x7f541c0c06e0 2023-11-15T02:59:44.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.657+0000 7f5413fff700 1 --2- 172.21.15.121:0/3367365510 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f53fc05a590 0x7f53fc05ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:44.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.661+0000 7f5413fff700 1 --2- 172.21.15.121:0/3367365510 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f53fc05a590 0x7f53fc05ca50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f541c0aba90 tx=0x7f5404017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:44.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.669+0000 7f5418ff9700 1 -- 172.21.15.121:0/3367365510 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f540c047790 con 0x7f541c0c06e0 2023-11-15T02:59:44.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.857+0000 7f5422369700 1 -- 172.21.15.121:0/3367365510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f541c0c4480 con 0x7f541c0c06e0 2023-11-15T02:59:44.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.857+0000 7f5418ff9700 1 -- 172.21.15.121:0/3367365510 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4502 (secure 0 0 0) 0x7f541c0c4480 con 0x7f541c0c06e0 2023-11-15T02:59:44.864 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:44.864 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":45,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3462858472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3462858472},{"type":"v1","addr":"172.21.15.190:6833","nonce":3462858472}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4042390087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4042390087},{"type":"v1","addr":"172.21.15.190:6835","nonce":4042390087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5907,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2239114529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2239114529},{"type":"v1","addr":"172.21.15.121:6837","nonce":2239114529}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:59:34.667864+0000","modified":"2023-11-15T02:59:37.695764+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5910":{"gid":5910,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2394175138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":2394175138},{"type":"v1","addr":"172.21.15.121:6836","nonce":2394175138}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-15T02:59:44.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.857+0000 7f5422369700 1 -- 172.21.15.121:0/3367365510 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f53fc05a590 msgr2=0x7f53fc05ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:44.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.857+0000 7f5422369700 1 --2- 172.21.15.121:0/3367365510 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f53fc05a590 0x7f53fc05ca50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f541c0aba90 tx=0x7f5404017000).stop 2023-11-15T02:59:44.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.857+0000 7f5422369700 1 -- 172.21.15.121:0/3367365510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f541c0c06e0 msgr2=0x7f541c13bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:44.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.857+0000 7f5422369700 1 --2- 172.21.15.121:0/3367365510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f541c0c06e0 0x7f541c13bcd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f540c00c490 tx=0x7f540c007de0).stop 2023-11-15T02:59:44.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.861+0000 7f5422369700 1 -- 172.21.15.121:0/3367365510 shutdown_connections 2023-11-15T02:59:44.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.861+0000 7f5422369700 1 --2- 172.21.15.121:0/3367365510 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f53fc05a590 0x7f53fc05ca50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:44.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.861+0000 7f5422369700 1 --2- 172.21.15.121:0/3367365510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f541c1539d0 0x7f541c0aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:44.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.861+0000 7f5422369700 1 --2- 172.21.15.121:0/3367365510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f541c0c2670 0x7f541c14fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:44.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.861+0000 7f5422369700 1 --2- 172.21.15.121:0/3367365510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f541c0c06e0 0x7f541c13bcd0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:44.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.861+0000 7f5422369700 1 -- 172.21.15.121:0/3367365510 >> 172.21.15.121:0/3367365510 conn(0x7f541c062190 msgr2=0x7f541c066be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:44.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.861+0000 7f5422369700 1 -- 172.21.15.121:0/3367365510 shutdown_connections 2023-11-15T02:59:44.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:44.861+0000 7f5422369700 1 -- 172.21.15.121:0/3367365510 wait complete. 2023-11-15T02:59:44.870 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 45 2023-11-15T02:59:45.290 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T02:59:34.667864+0000', 'modified': '2023-11-15T02:59:37.695764+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5910}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5910': {'gid': 5910, 'name': 'a', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.121:6836/2394175138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6835', 'nonce': 2394175138}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 2394175138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T02:59:45.291 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T02:59:45.291 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-15T02:59:45.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.445+0000 7f85f99f9700 1 Processor -- start 2023-11-15T02:59:45.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.445+0000 7f85f99f9700 1 -- start start 2023-11-15T02:59:45.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.445+0000 7f85f99f9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f85f4062330 0x7f85f4062710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.445+0000 7f85f99f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85f40bd140 0x7f85f4150330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.445+0000 7f85f99f9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f40c14d0 con 0x7f85f40bcce0 2023-11-15T02:59:45.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.445+0000 7f85f99f9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f40c2330 con 0x7f85f4062330 2023-11-15T02:59:45.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.445+0000 7f85f99f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f4063800 con 0x7f85f40bd140 2023-11-15T02:59:45.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f89f7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f85f4062330 0x7f85f4062710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f89f7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f85f4062330 0x7f85f4062710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:60844/0 (socket says 172.21.15.121:60844) 2023-11-15T02:59:45.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f89f7700 1 -- 172.21.15.121:0/631027836 learned_addr learned my addr 172.21.15.121:0/631027836 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:45.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f91f8700 1 --2- 172.21.15.121:0/631027836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85f40bd140 0x7f85f4150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f89f7700 1 -- 172.21.15.121:0/631027836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85f40bd140 msgr2=0x7f85f4150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:45.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f89f7700 1 --2- 172.21.15.121:0/631027836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85f40bd140 0x7f85f4150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f89f7700 1 -- 172.21.15.121:0/631027836 >> v1:172.21.15.190:6789/0 conn(0x7f85f40bcce0 legacy=0x7f85f414fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:45.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f89f7700 1 -- 172.21.15.121:0/631027836 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f40c1720 con 0x7f85f4062330 2023-11-15T02:59:45.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f89f7700 1 --2- 172.21.15.121:0/631027836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85f4062330 0x7f85f4062710 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f85e8006fd0 tx=0x7f85e80090f0).ready entity=mon.1 client_cookie=750dca8ee4a57aa6 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:45.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f37fe700 1 -- 172.21.15.121:0/631027836 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e800b580 con 0x7f85f4062330 2023-11-15T02:59:45.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f37fe700 1 -- 172.21.15.121:0/631027836 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e800b6e0 con 0x7f85f4062330 2023-11-15T02:59:45.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f37fe700 1 -- 172.21.15.121:0/631027836 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e800ba60 con 0x7f85f4062330 2023-11-15T02:59:45.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f99f9700 1 -- 172.21.15.121:0/631027836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85f4062330 msgr2=0x7f85f4062710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:45.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f99f9700 1 --2- 172.21.15.121:0/631027836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85f4062330 0x7f85f4062710 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f85e8006fd0 tx=0x7f85e80090f0).stop 2023-11-15T02:59:45.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f99f9700 1 -- 172.21.15.121:0/631027836 shutdown_connections 2023-11-15T02:59:45.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f99f9700 1 --2- 172.21.15.121:0/631027836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85f40bd140 0x7f85f4150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f99f9700 1 --2- 172.21.15.121:0/631027836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85f4062330 0x7f85f4062710 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f99f9700 1 -- 172.21.15.121:0/631027836 >> 172.21.15.121:0/631027836 conn(0x7f85f4067a60 msgr2=0x7f85f4067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:45.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f99f9700 1 -- 172.21.15.121:0/631027836 shutdown_connections 2023-11-15T02:59:45.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.449+0000 7f85f99f9700 1 -- 172.21.15.121:0/631027836 wait complete. 2023-11-15T02:59:45.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.453+0000 7f85f99f9700 1 Processor -- start 2023-11-15T02:59:45.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.453+0000 7f85f99f9700 1 -- start start 2023-11-15T02:59:45.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.453+0000 7f85f99f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85f4062330 0x7f85f4153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.453+0000 7f85f99f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85f40bcce0 0x7f85f4150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.453+0000 7f85f99f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85f40bd140 0x7f85f40aa730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.453+0000 7f85f99f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f4154740 con 0x7f85f40bcce0 2023-11-15T02:59:45.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.453+0000 7f85f99f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f4153bc0 con 0x7f85f4062330 2023-11-15T02:59:45.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.453+0000 7f85f99f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f4153ea0 con 0x7f85f40bd140 2023-11-15T02:59:45.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.453+0000 7f85f3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85f40bcce0 0x7f85f4150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.457+0000 7f85f3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85f40bcce0 0x7f85f4150b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33424/0 (socket says 172.21.15.121:33424) 2023-11-15T02:59:45.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.457+0000 7f85f3fff700 1 -- 172.21.15.121:0/3819860958 learned_addr learned my addr 172.21.15.121:0/3819860958 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:45.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.457+0000 7f85f89f7700 1 --2- 172.21.15.121:0/3819860958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85f4062330 0x7f85f4153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.457+0000 7f85f91f8700 1 --2- 172.21.15.121:0/3819860958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85f40bd140 0x7f85f40aa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.457+0000 7f85f3fff700 1 -- 172.21.15.121:0/3819860958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85f40bd140 msgr2=0x7f85f40aa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:45.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.457+0000 7f85f3fff700 1 --2- 172.21.15.121:0/3819860958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85f40bd140 0x7f85f40aa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.457+0000 7f85f3fff700 1 -- 172.21.15.121:0/3819860958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85f4062330 msgr2=0x7f85f4153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:45.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.457+0000 7f85f3fff700 1 --2- 172.21.15.121:0/3819860958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85f4062330 0x7f85f4153550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.457+0000 7f85f3fff700 1 -- 172.21.15.121:0/3819860958 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e8009370 con 0x7f85f40bcce0 2023-11-15T02:59:45.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.457+0000 7f85f3fff700 1 --2- 172.21.15.121:0/3819860958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85f40bcce0 0x7f85f4150b20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f85e000c4c0 tx=0x7f85e0007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:45.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.457+0000 7f85f0ff9700 1 -- 172.21.15.121:0/3819860958 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e000ce70 con 0x7f85f40bcce0 2023-11-15T02:59:45.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.457+0000 7f85f0ff9700 1 -- 172.21.15.121:0/3819860958 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e0007020 con 0x7f85f40bcce0 2023-11-15T02:59:45.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.457+0000 7f85f99f9700 1 -- 172.21.15.121:0/3819860958 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f4154180 con 0x7f85f40bcce0 2023-11-15T02:59:45.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.457+0000 7f85f0ff9700 1 -- 172.21.15.121:0/3819860958 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e00073c0 con 0x7f85f40bcce0 2023-11-15T02:59:45.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.457+0000 7f85f99f9700 1 -- 172.21.15.121:0/3819860958 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f4151010 con 0x7f85f40bcce0 2023-11-15T02:59:45.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.461+0000 7f85f0ff9700 1 -- 172.21.15.121:0/3819860958 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f85e0006900 con 0x7f85f40bcce0 2023-11-15T02:59:45.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.461+0000 7f8604fe3740 1 Processor -- start 2023-11-15T02:59:45.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.461+0000 7f85f0ff9700 1 --2- 172.21.15.121:0/3819860958 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f85d805a4e0 0x7f85d805c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.461+0000 7f8604fe3740 1 -- start start 2023-11-15T02:59:45.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.461+0000 7f85f0ff9700 1 -- 172.21.15.121:0/3819860958 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f85e00741d0 con 0x7f85f40bcce0 2023-11-15T02:59:45.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.461+0000 7f85f91f8700 1 --2- 172.21.15.121:0/3819860958 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f85d805a4e0 0x7f85d805c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.461+0000 7f8604fe3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 0x1246790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.461+0000 7f8604fe3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x123fbe0 0x12878a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.461+0000 7f8604fe3740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x128d920 0x128dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.461+0000 7f8604fe3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x11c6410 con 0x12463b0 2023-11-15T02:59:45.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.461+0000 7f8604fe3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x11c6550 con 0x123fbe0 2023-11-15T02:59:45.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.461+0000 7f8604fe3740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x11c6a00 con 0x128d920 2023-11-15T02:59:45.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.461+0000 7f85f91f8700 1 --2- 172.21.15.121:0/3819860958 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f85d805a4e0 0x7f85d805c9a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f85f4160200 tx=0x7f85ec00c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:45.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.461+0000 7f85f89f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 0x1246790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.461+0000 7f85f89f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 0x1246790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33440/0 (socket says 172.21.15.121:33440) 2023-11-15T02:59:45.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.461+0000 7f85f89f7700 1 -- 172.21.15.121:0/1747485700 learned_addr learned my addr 172.21.15.121:0/1747485700 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:45.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f85f91f8700 1 --2- 172.21.15.121:0/1747485700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x128d920 0x128dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f85f3fff700 1 --2- 172.21.15.121:0/1747485700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x123fbe0 0x12878a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f85f89f7700 1 -- 172.21.15.121:0/1747485700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x128d920 msgr2=0x128dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:45.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f85f89f7700 1 --2- 172.21.15.121:0/1747485700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x128d920 0x128dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f85f89f7700 1 -- 172.21.15.121:0/1747485700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x123fbe0 msgr2=0x12878a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:45.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f85f89f7700 1 --2- 172.21.15.121:0/1747485700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x123fbe0 0x12878a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f85f89f7700 1 -- 172.21.15.121:0/1747485700 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x11bf730 con 0x12463b0 2023-11-15T02:59:45.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f85f89f7700 1 --2- 172.21.15.121:0/1747485700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 0x1246790 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f85e8009ea0 tx=0x7f85e8017000).ready entity=mon.0 client_cookie=1f5b9bd9713bbb30 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:45.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f85ddffb700 1 -- 172.21.15.121:0/1747485700 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e800b580 con 0x12463b0 2023-11-15T02:59:45.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f85ddffb700 1 -- 172.21.15.121:0/1747485700 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e8000bc0 con 0x12463b0 2023-11-15T02:59:45.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f85ddffb700 1 -- 172.21.15.121:0/1747485700 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e800ba60 con 0x12463b0 2023-11-15T02:59:45.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f8604fe3740 1 -- 172.21.15.121:0/1747485700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 msgr2=0x1246790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:45.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f8604fe3740 1 --2- 172.21.15.121:0/1747485700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 0x1246790 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f85e8009ea0 tx=0x7f85e8017000).stop 2023-11-15T02:59:45.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f8604fe3740 1 -- 172.21.15.121:0/1747485700 shutdown_connections 2023-11-15T02:59:45.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f8604fe3740 1 --2- 172.21.15.121:0/1747485700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x128d920 0x128dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f8604fe3740 1 --2- 172.21.15.121:0/1747485700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x123fbe0 0x12878a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f8604fe3740 1 --2- 172.21.15.121:0/1747485700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 0x1246790 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f8604fe3740 1 -- 172.21.15.121:0/1747485700 >> 172.21.15.121:0/1747485700 conn(0x1024550 msgr2=0x1249120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:45.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f8604fe3740 1 -- 172.21.15.121:0/1747485700 shutdown_connections 2023-11-15T02:59:45.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f8604fe3740 1 -- 172.21.15.121:0/1747485700 wait complete. 2023-11-15T02:59:45.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f8604fe3740 1 -- start start 2023-11-15T02:59:45.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.465+0000 7f8604fe3740 1 Processor -- start 2023-11-15T02:59:45.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f8604fe3740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x123fbe0 0x128ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f8604fe3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 0x126f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f8604fe3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x128d920 0x129ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f8604fe3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1191650 con 0x12463b0 2023-11-15T02:59:45.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f8604fe3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1275c30 con 0x128d920 2023-11-15T02:59:45.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f8604fe3740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1289ab0 con 0x123fbe0 2023-11-15T02:59:45.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f8604fe3740 10 client.0 ll_register_callbacks cb 0x11fe020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:59:45.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f85f3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 0x126f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f85f89f7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x123fbe0 0x128ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f85f3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 0x126f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33454/0 (socket says 172.21.15.121:33454) 2023-11-15T02:59:45.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f85f3fff700 1 -- 172.21.15.121:0/3916279932 learned_addr learned my addr 172.21.15.121:0/3916279932 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:45.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f85f91f8700 1 --2- 172.21.15.121:0/3916279932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x128d920 0x129ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f85f3fff700 1 -- 172.21.15.121:0/3916279932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x123fbe0 msgr2=0x128ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:45.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f85f3fff700 1 --2- 172.21.15.121:0/3916279932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x123fbe0 0x128ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f85f3fff700 1 -- 172.21.15.121:0/3916279932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x128d920 msgr2=0x129ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:45.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f85f3fff700 1 --2- 172.21.15.121:0/3916279932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x128d920 0x129ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f85f3fff700 1 -- 172.21.15.121:0/3916279932 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f40c1720 con 0x12463b0 2023-11-15T02:59:45.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f85f3fff700 1 --2- 172.21.15.121:0/3916279932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 0x126f190 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f85e0072650 tx=0x7f85e0003040).ready entity=mon.0 client_cookie=2900ec9d535969bd server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:45.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f85d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:59:45.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f85d2ffd700 1 -- 172.21.15.121:0/3916279932 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0047cc0 con 0x12463b0 2023-11-15T02:59:45.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f85d2ffd700 1 -- 172.21.15.121:0/3916279932 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f85e0047e20 con 0x12463b0 2023-11-15T02:59:45.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f8604fe3740 1 -- 172.21.15.121:0/3916279932 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x11bc1a0 con 0x12463b0 2023-11-15T02:59:45.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f85d2ffd700 1 -- 172.21.15.121:0/3916279932 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85e004acb0 con 0x12463b0 2023-11-15T02:59:45.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f8604fe3740 10 client.6093 fetch_fsmap learned FSMap version 45 2023-11-15T02:59:45.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f8604fe3740 1 -- 172.21.15.121:0/3916279932 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) v3 -- 0x11ba960 con 0x12463b0 2023-11-15T02:59:45.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f85d2ffd700 1 -- 172.21.15.121:0/3916279932 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e005d070 con 0x12463b0 2023-11-15T02:59:45.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f85d2ffd700 1 -- 172.21.15.121:0/3916279932 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 45) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f85e0006bb0 con 0x12463b0 2023-11-15T02:59:45.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f8604fe3740 10 client.6093 fetch_fsmap finished waiting for FSMap version 45 2023-11-15T02:59:45.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f8604fe3740 10 client.6093 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T02:59:45.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f8604fe3740 20 client.6093 populate_metadata read hostname 'smithi121' 2023-11-15T02:59:45.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f8604fe3740 1 --2- 172.21.15.121:0/3916279932 >> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] conn(0x1272cd0 0x129c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f8604fe3740 4 client.6093 mds_command: new command op to 5910 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T02:59:45.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f8604fe3740 1 -- 172.21.15.121:0/3916279932 --> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x11b26f0 con 0x1272cd0 2023-11-15T02:59:45.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.469+0000 7f85f89f7700 1 --2- 172.21.15.121:0/3916279932 >> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] conn(0x1272cd0 0x129c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.473+0000 7f85d2ffd700 10 client.6093 ms_handle_connect on v2:172.21.15.121:6835/2394175138 2023-11-15T02:59:45.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.473+0000 7f85f89f7700 1 --2- 172.21.15.121:0/3916279932 >> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] conn(0x1272cd0 0x129c8d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=d0aaf5534cf80b2b server_cookie=779bb479405e1a8e in_seq=0 out_seq=0 2023-11-15T02:59:45.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.473+0000 7f85d2ffd700 1 -- 172.21.15.121:0/3916279932 <== mds.0 v2:172.21.15.121:6835/2394175138 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x11b26f0 con 0x1272cd0 2023-11-15T02:59:45.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.473+0000 7f85d2ffd700 10 client.6093 handle_command_reply: tid=0 2023-11-15T02:59:45.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.473+0000 7f8604fe3740 1 client.6093 shutdown 2023-11-15T02:59:45.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.473+0000 7f8604fe3740 1 -- 172.21.15.121:0/3916279932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 msgr2=0x126f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:45.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.473+0000 7f8604fe3740 1 --2- 172.21.15.121:0/3916279932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 0x126f190 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f85e0072650 tx=0x7f85e0003040).stop 2023-11-15T02:59:45.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.473+0000 7f8604fe3740 1 -- 172.21.15.121:0/3916279932 shutdown_connections 2023-11-15T02:59:45.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.473+0000 7f8604fe3740 1 --2- 172.21.15.121:0/3916279932 >> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] conn(0x1272cd0 0x129c8d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.473+0000 7f8604fe3740 1 --2- 172.21.15.121:0/3916279932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x128d920 0x129ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.473+0000 7f8604fe3740 1 --2- 172.21.15.121:0/3916279932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 0x126f190 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.473+0000 7f8604fe3740 1 --2- 172.21.15.121:0/3916279932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x123fbe0 0x128ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.473+0000 7f85d2ffd700 0 client.6093 ms_handle_reset on v2:172.21.15.121:6835/2394175138 2023-11-15T02:59:45.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.473+0000 7f8604fe3740 1 -- 172.21.15.121:0/3916279932 >> 172.21.15.121:0/3916279932 conn(0x1024550 msgr2=0x1290390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:45.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.473+0000 7f8604fe3740 1 -- 172.21.15.121:0/3916279932 shutdown_connections 2023-11-15T02:59:45.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.473+0000 7f8604fe3740 1 -- 172.21.15.121:0/3916279932 wait complete. 2023-11-15T02:59:45.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.473+0000 7f8604fe3740 20 client.6093 trim_cache size 0 max 16384 2023-11-15T02:59:45.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.493+0000 7f8604fe3740 1 Processor -- start 2023-11-15T02:59:45.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.493+0000 7f8604fe3740 1 -- start start 2023-11-15T02:59:45.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.493+0000 7f8604fe3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x123fbe0 0x1249e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.493+0000 7f8604fe3740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x12463b0 0x12492d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.493+0000 7f8604fe3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x128d920 0x129e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.493+0000 7f8604fe3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x11b26f0 con 0x123fbe0 2023-11-15T02:59:45.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.493+0000 7f8604fe3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x119f9c0 con 0x128d920 2023-11-15T02:59:45.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.493+0000 7f8604fe3740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1277210 con 0x12463b0 2023-11-15T02:59:45.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f85f89f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x123fbe0 0x1249e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f85f89f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x123fbe0 0x1249e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33456/0 (socket says 172.21.15.121:33456) 2023-11-15T02:59:45.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f85f89f7700 1 -- 172.21.15.121:0/1723101218 learned_addr learned my addr 172.21.15.121:0/1723101218 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:45.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f85f3fff700 1 --2- 172.21.15.121:0/1723101218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x12463b0 0x12492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f85f91f8700 1 --2- 172.21.15.121:0/1723101218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x128d920 0x129e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f85f89f7700 1 -- 172.21.15.121:0/1723101218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x12463b0 msgr2=0x12492d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:45.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f85f89f7700 1 --2- 172.21.15.121:0/1723101218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x12463b0 0x12492d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f85f89f7700 1 -- 172.21.15.121:0/1723101218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x128d920 msgr2=0x129e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:45.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f85f89f7700 1 --2- 172.21.15.121:0/1723101218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x128d920 0x129e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f85f89f7700 1 -- 172.21.15.121:0/1723101218 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f40a8cc0 con 0x123fbe0 2023-11-15T02:59:45.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f85f89f7700 1 --2- 172.21.15.121:0/1723101218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x123fbe0 0x1249e50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f85e801d040 tx=0x7f85e8027040).ready entity=mon.0 client_cookie=76409c78bec0e785 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:45.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f85de7fc700 1 -- 172.21.15.121:0/1723101218 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e802a030 con 0x123fbe0 2023-11-15T02:59:45.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f85de7fc700 1 -- 172.21.15.121:0/1723101218 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e800b850 con 0x123fbe0 2023-11-15T02:59:45.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f85de7fc700 1 -- 172.21.15.121:0/1723101218 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e802c040 con 0x123fbe0 2023-11-15T02:59:45.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f8604fe3740 1 -- 172.21.15.121:0/1723101218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x123fbe0 msgr2=0x1249e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:45.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f8604fe3740 1 --2- 172.21.15.121:0/1723101218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x123fbe0 0x1249e50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f85e801d040 tx=0x7f85e8027040).stop 2023-11-15T02:59:45.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f8604fe3740 1 -- 172.21.15.121:0/1723101218 shutdown_connections 2023-11-15T02:59:45.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f8604fe3740 1 --2- 172.21.15.121:0/1723101218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x128d920 0x129e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f8604fe3740 1 --2- 172.21.15.121:0/1723101218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x12463b0 0x12492d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f8604fe3740 1 --2- 172.21.15.121:0/1723101218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x123fbe0 0x1249e50 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f8604fe3740 1 -- 172.21.15.121:0/1723101218 >> 172.21.15.121:0/1723101218 conn(0x1024550 msgr2=0x12730b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:45.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f8604fe3740 1 -- 172.21.15.121:0/1723101218 shutdown_connections 2023-11-15T02:59:45.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.497+0000 7f8604fe3740 1 -- 172.21.15.121:0/1723101218 wait complete. 2023-11-15T02:59:45.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.501+0000 7f8604fe3740 1 -- start start 2023-11-15T02:59:45.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.501+0000 7f8604fe3740 1 Processor -- start 2023-11-15T02:59:45.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.501+0000 7f8604fe3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x123fbe0 0x1276610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.501+0000 7f8604fe3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 0x12a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.501+0000 7f8604fe3740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x128d920 0x12a1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.501+0000 7f8604fe3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x12a1f20 con 0x12463b0 2023-11-15T02:59:45.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.501+0000 7f8604fe3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x128cba0 con 0x123fbe0 2023-11-15T02:59:45.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.501+0000 7f8604fe3740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x12741b0 con 0x128d920 2023-11-15T02:59:45.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f8604fe3740 10 client.0 ll_register_callbacks cb 0x7f85e004b0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:59:45.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85f3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 0x12a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85f3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 0x12a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33460/0 (socket says 172.21.15.121:33460) 2023-11-15T02:59:45.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85f3fff700 1 -- 172.21.15.121:0/1004833916 learned_addr learned my addr 172.21.15.121:0/1004833916 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:45.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85f89f7700 1 --2- 172.21.15.121:0/1004833916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x123fbe0 0x1276610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85f91f8700 1 --2- 172.21.15.121:0/1004833916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x128d920 0x12a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85f3fff700 1 -- 172.21.15.121:0/1004833916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x128d920 msgr2=0x12a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:45.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85f3fff700 1 --2- 172.21.15.121:0/1004833916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x128d920 0x12a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85f3fff700 1 -- 172.21.15.121:0/1004833916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x123fbe0 msgr2=0x1276610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:45.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85f3fff700 1 --2- 172.21.15.121:0/1004833916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x123fbe0 0x1276610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85f3fff700 1 -- 172.21.15.121:0/1004833916 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x11bf730 con 0x12463b0 2023-11-15T02:59:45.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85f89f7700 1 --2- 172.21.15.121:0/1004833916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x123fbe0 0x1276610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:45.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85f3fff700 1 --2- 172.21.15.121:0/1004833916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 0x12a0e10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f85e0007fd0 tx=0x7f85e0053000).ready entity=mon.0 client_cookie=70837535cafc3f44 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:45.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:59:45.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85d2ffd700 1 -- 172.21.15.121:0/1004833916 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0047cc0 con 0x12463b0 2023-11-15T02:59:45.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85d2ffd700 1 -- 172.21.15.121:0/1004833916 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f85e0047e20 con 0x12463b0 2023-11-15T02:59:45.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f8604fe3740 1 -- 172.21.15.121:0/1004833916 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85e8000bc0 con 0x12463b0 2023-11-15T02:59:45.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85d2ffd700 1 -- 172.21.15.121:0/1004833916 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85e00475b0 con 0x12463b0 2023-11-15T02:59:45.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f8604fe3740 10 client.6099 fetch_fsmap learned FSMap version 45 2023-11-15T02:59:45.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f8604fe3740 1 -- 172.21.15.121:0/1004833916 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) v3 -- 0x11ba6b0 con 0x12463b0 2023-11-15T02:59:45.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85d2ffd700 1 -- 172.21.15.121:0/1004833916 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e8000bc0 con 0x12463b0 2023-11-15T02:59:45.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85d2ffd700 1 -- 172.21.15.121:0/1004833916 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 45) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f85e0069030 con 0x12463b0 2023-11-15T02:59:45.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f8604fe3740 10 client.6099 fetch_fsmap finished waiting for FSMap version 45 2023-11-15T02:59:45.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f8604fe3740 10 client.6099 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T02:59:45.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f8604fe3740 20 client.6099 populate_metadata read hostname 'smithi121' 2023-11-15T02:59:45.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f8604fe3740 1 --2- 172.21.15.121:0/1004833916 >> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] conn(0x129e400 0x1247c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f8604fe3740 4 client.6099 mds_command: new command op to 5910 tid=0[{"prefix": "status"}] 2023-11-15T02:59:45.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f8604fe3740 1 -- 172.21.15.121:0/1004833916 --> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] -- command(tid 0: {"prefix": "status"}) v1 -- 0x107b4e0 con 0x129e400 2023-11-15T02:59:45.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85f89f7700 1 --2- 172.21.15.121:0/1004833916 >> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] conn(0x129e400 0x1247c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85f89f7700 1 --2- 172.21.15.121:0/1004833916 >> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] conn(0x129e400 0x1247c20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=6b8a441856d1ef23 server_cookie=5f43b696b00374f7 in_seq=0 out_seq=0 2023-11-15T02:59:45.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.505+0000 7f85d2ffd700 10 client.6099 ms_handle_connect on v2:172.21.15.121:6835/2394175138 2023-11-15T02:59:45.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f85d2ffd700 1 -- 172.21.15.121:0/1004833916 <== mds.0 v2:172.21.15.121:6835/2394175138 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x107b4e0 con 0x129e400 2023-11-15T02:59:45.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f85d2ffd700 10 client.6099 handle_command_reply: tid=0 2023-11-15T02:59:45.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f8604fe3740 1 client.6099 shutdown 2023-11-15T02:59:45.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f8604fe3740 1 -- 172.21.15.121:0/1004833916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 msgr2=0x12a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:45.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f8604fe3740 1 --2- 172.21.15.121:0/1004833916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 0x12a0e10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f85e0007fd0 tx=0x7f85e0053000).stop 2023-11-15T02:59:45.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f8604fe3740 1 -- 172.21.15.121:0/1004833916 shutdown_connections 2023-11-15T02:59:45.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f8604fe3740 1 --2- 172.21.15.121:0/1004833916 >> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] conn(0x129e400 0x1247c20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f8604fe3740 1 --2- 172.21.15.121:0/1004833916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x128d920 0x12a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f8604fe3740 1 --2- 172.21.15.121:0/1004833916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x12463b0 0x12a0e10 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f85d2ffd700 0 client.6099 ms_handle_reset on v2:172.21.15.121:6835/2394175138 2023-11-15T02:59:45.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f8604fe3740 1 --2- 172.21.15.121:0/1004833916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x123fbe0 0x1276610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f8604fe3740 1 -- 172.21.15.121:0/1004833916 >> 172.21.15.121:0/1004833916 conn(0x1024550 msgr2=0x1249e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:45.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f8604fe3740 1 -- 172.21.15.121:0/1004833916 shutdown_connections 2023-11-15T02:59:45.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f8604fe3740 1 -- 172.21.15.121:0/1004833916 wait complete. 2023-11-15T02:59:45.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f8604fe3740 20 client.6099 trim_cache size 0 max 16384 2023-11-15T02:59:45.528 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T02:59:45.528 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T02:59:45.529 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T02:59:45.529 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 5910, 2023-11-15T02:59:45.529 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T02:59:45.529 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T02:59:45.529 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T02:59:45.530 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 10.839740811, 2023-11-15T02:59:45.530 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 45, 2023-11-15T02:59:45.530 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 61, 2023-11-15T02:59:45.530 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 61, 2023-11-15T02:59:45.530 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 16.240210322999999 2023-11-15T02:59:45.531 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T02:59:45.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f85f99f9700 1 -- 172.21.15.121:0/3819860958 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f85d805a4e0 msgr2=0x7f85d805c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:45.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f85f99f9700 1 --2- 172.21.15.121:0/3819860958 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f85d805a4e0 0x7f85d805c9a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f85f4160200 tx=0x7f85ec00c040).stop 2023-11-15T02:59:45.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f85f99f9700 1 -- 172.21.15.121:0/3819860958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85f40bcce0 msgr2=0x7f85f4150b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:45.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f85f99f9700 1 --2- 172.21.15.121:0/3819860958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85f40bcce0 0x7f85f4150b20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f85e000c4c0 tx=0x7f85e0007df0).stop 2023-11-15T02:59:45.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f85f99f9700 1 -- 172.21.15.121:0/3819860958 shutdown_connections 2023-11-15T02:59:45.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f85f99f9700 1 --2- 172.21.15.121:0/3819860958 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f85d805a4e0 0x7f85d805c9a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f85f99f9700 1 --2- 172.21.15.121:0/3819860958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85f40bd140 0x7f85f40aa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f85f99f9700 1 --2- 172.21.15.121:0/3819860958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85f40bcce0 0x7f85f4150b20 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f85f99f9700 1 --2- 172.21.15.121:0/3819860958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85f4062330 0x7f85f4153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f85f99f9700 1 -- 172.21.15.121:0/3819860958 >> 172.21.15.121:0/3819860958 conn(0x7f85f4067a60 msgr2=0x7f85f40c5660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:45.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f85f99f9700 1 -- 172.21.15.121:0/3819860958 shutdown_connections 2023-11-15T02:59:45.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.513+0000 7f85f99f9700 1 -- 172.21.15.121:0/3819860958 wait complete. 2023-11-15T02:59:45.537 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T02:59:45.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.665+0000 7f218701b700 1 Processor -- start 2023-11-15T02:59:45.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.665+0000 7f218701b700 1 -- start start 2023-11-15T02:59:45.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.669+0000 7f218701b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f21800c7900 0x7f21800c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.669+0000 7f218701b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21800c0f70 con 0x7f21800aa980 2023-11-15T02:59:45.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.669+0000 7f218701b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21800c5db0 con 0x7f21800c7520 2023-11-15T02:59:45.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.669+0000 7f218701b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21800c6090 con 0x7f21800c7900 2023-11-15T02:59:45.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.669+0000 7f217ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f21800c7900 0x7f21800c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.669+0000 7f21855b8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f21800aa980 0x7f21800aadb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:36972/0 (socket says 172.21.15.121:36972) 2023-11-15T02:59:45.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.669+0000 7f217ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f21800c7900 0x7f21800c7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33468/0 (socket says 172.21.15.121:33468) 2023-11-15T02:59:45.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.669+0000 7f21855b8700 1 -- 172.21.15.121:0/3029081341 learned_addr learned my addr 172.21.15.121:0/3029081341 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:45.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.669+0000 7f217ffff700 1 -- 172.21.15.121:0/3029081341 >> v1:172.21.15.190:6789/0 conn(0x7f21800c7520 legacy=0x7f21800c1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:45.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.669+0000 7f217ffff700 1 -- 172.21.15.121:0/3029081341 >> v1:172.21.15.121:6789/0 conn(0x7f21800aa980 legacy=0x7f21800aadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:45.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.669+0000 7f217ffff700 1 -- 172.21.15.121:0/3029081341 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21800c0c00 con 0x7f21800c7900 2023-11-15T02:59:45.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.669+0000 7f217ffff700 1 --2- 172.21.15.121:0/3029081341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21800c7900 0x7f21800c7d90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f2170009760 tx=0x7f2170009060).ready entity=mon.0 client_cookie=84e2f56d8d344e01 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:45.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.669+0000 7f217f7fe700 1 -- 172.21.15.121:0/3029081341 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217000b580 con 0x7f21800c7900 2023-11-15T02:59:45.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.673+0000 7f217f7fe700 1 -- 172.21.15.121:0/3029081341 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217000b6e0 con 0x7f21800c7900 2023-11-15T02:59:45.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.673+0000 7f217f7fe700 1 -- 172.21.15.121:0/3029081341 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217000ba60 con 0x7f21800c7900 2023-11-15T02:59:45.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.673+0000 7f218701b700 1 -- 172.21.15.121:0/3029081341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21800c7900 msgr2=0x7f21800c7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:45.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.673+0000 7f218701b700 1 --2- 172.21.15.121:0/3029081341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21800c7900 0x7f21800c7d90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f2170009760 tx=0x7f2170009060).stop 2023-11-15T02:59:45.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.673+0000 7f218701b700 1 -- 172.21.15.121:0/3029081341 shutdown_connections 2023-11-15T02:59:45.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.673+0000 7f218701b700 1 --2- 172.21.15.121:0/3029081341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21800c7900 0x7f21800c7d90 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.673+0000 7f218701b700 1 -- 172.21.15.121:0/3029081341 >> 172.21.15.121:0/3029081341 conn(0x7f2180067780 msgr2=0x7f21800681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:45.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.673+0000 7f218701b700 1 -- 172.21.15.121:0/3029081341 shutdown_connections 2023-11-15T02:59:45.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.673+0000 7f218701b700 1 -- 172.21.15.121:0/3029081341 wait complete. 2023-11-15T02:59:45.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.673+0000 7f218701b700 1 Processor -- start 2023-11-15T02:59:45.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f218701b700 1 -- start start 2023-11-15T02:59:45.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f218701b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21800aa980 0x7f21800a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f218701b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21800c7520 0x7f21800a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f218701b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21800c7900 0x7f21800a2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f218701b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21800a3230 con 0x7f21800aa980 2023-11-15T02:59:45.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f218701b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21800a3510 con 0x7f21800c7520 2023-11-15T02:59:45.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f218701b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21800a37f0 con 0x7f21800c7900 2023-11-15T02:59:45.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f2184db7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21800aa980 0x7f21800a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f2184db7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21800aa980 0x7f21800a42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33478/0 (socket says 172.21.15.121:33478) 2023-11-15T02:59:45.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f2184db7700 1 -- 172.21.15.121:0/173686986 learned_addr learned my addr 172.21.15.121:0/173686986 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:45.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f21855b8700 1 --2- 172.21.15.121:0/173686986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21800c7900 0x7f21800a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f217ffff700 1 --2- 172.21.15.121:0/173686986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21800c7520 0x7f21800a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f2184db7700 1 -- 172.21.15.121:0/173686986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21800c7900 msgr2=0x7f21800a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:45.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f2184db7700 1 --2- 172.21.15.121:0/173686986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21800c7900 0x7f21800a2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f2184db7700 1 -- 172.21.15.121:0/173686986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21800c7520 msgr2=0x7f21800a5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:45.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f2184db7700 1 --2- 172.21.15.121:0/173686986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21800c7520 0x7f21800a5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f2184db7700 1 -- 172.21.15.121:0/173686986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21700092a0 con 0x7f21800aa980 2023-11-15T02:59:45.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f2184db7700 1 --2- 172.21.15.121:0/173686986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21800aa980 0x7f21800a42d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f217400c4c0 tx=0x7f2174007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:45.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f217cff9700 1 -- 172.21.15.121:0/173686986 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2174007020 con 0x7f21800aa980 2023-11-15T02:59:45.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f217cff9700 1 -- 172.21.15.121:0/173686986 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2174001e90 con 0x7f21800aa980 2023-11-15T02:59:45.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f218701b700 1 -- 172.21.15.121:0/173686986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21800a3ad0 con 0x7f21800aa980 2023-11-15T02:59:45.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f217cff9700 1 -- 172.21.15.121:0/173686986 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21740073b0 con 0x7f21800aa980 2023-11-15T02:59:45.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.677+0000 7f218701b700 1 -- 172.21.15.121:0/173686986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21800a47c0 con 0x7f21800aa980 2023-11-15T02:59:45.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.681+0000 7f218701b700 1 -- 172.21.15.121:0/173686986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21800a3fd0 con 0x7f21800aa980 2023-11-15T02:59:45.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.681+0000 7f217cff9700 1 -- 172.21.15.121:0/173686986 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f21740068b0 con 0x7f21800aa980 2023-11-15T02:59:45.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.685+0000 7f217cff9700 1 --2- 172.21.15.121:0/173686986 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f216c05a540 0x7f216c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:45.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.685+0000 7f217cff9700 1 -- 172.21.15.121:0/173686986 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f21740742b0 con 0x7f21800aa980 2023-11-15T02:59:45.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.685+0000 7f217ffff700 1 --2- 172.21.15.121:0/173686986 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f216c05a540 0x7f216c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:45.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.685+0000 7f217ffff700 1 --2- 172.21.15.121:0/173686986 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f216c05a540 0x7f216c05ca00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f21800a1ed0 tx=0x7f2170017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:45.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.689+0000 7f217cff9700 1 -- 172.21.15.121:0/173686986 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f21740479d0 con 0x7f21800aa980 2023-11-15T02:59:45.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.885+0000 7f218701b700 1 -- 172.21.15.121:0/173686986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f218005eeb0 con 0x7f21800aa980 2023-11-15T02:59:45.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.885+0000 7f217cff9700 1 -- 172.21.15.121:0/173686986 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) v1 ==== 53+0+73 (secure 0 0 0) 0x7f217404aff0 con 0x7f21800aa980 2023-11-15T02:59:45.889 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T02:59:45.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.889+0000 7f218701b700 1 -- 172.21.15.121:0/173686986 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f216c05a540 msgr2=0x7f216c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:45.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.889+0000 7f218701b700 1 --2- 172.21.15.121:0/173686986 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f216c05a540 0x7f216c05ca00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f21800a1ed0 tx=0x7f2170017000).stop 2023-11-15T02:59:45.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.889+0000 7f218701b700 1 -- 172.21.15.121:0/173686986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21800aa980 msgr2=0x7f21800a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:45.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.889+0000 7f218701b700 1 --2- 172.21.15.121:0/173686986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21800aa980 0x7f21800a42d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f217400c4c0 tx=0x7f2174007e00).stop 2023-11-15T02:59:45.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.889+0000 7f218701b700 1 -- 172.21.15.121:0/173686986 shutdown_connections 2023-11-15T02:59:45.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.889+0000 7f218701b700 1 --2- 172.21.15.121:0/173686986 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f216c05a540 0x7f216c05ca00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.889+0000 7f218701b700 1 --2- 172.21.15.121:0/173686986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21800c7900 0x7f21800a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.889+0000 7f218701b700 1 --2- 172.21.15.121:0/173686986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21800c7520 0x7f21800a5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.889+0000 7f218701b700 1 --2- 172.21.15.121:0/173686986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21800aa980 0x7f21800a42d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:45.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.889+0000 7f218701b700 1 -- 172.21.15.121:0/173686986 >> 172.21.15.121:0/173686986 conn(0x7f2180067780 msgr2=0x7f21800ac040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:45.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.889+0000 7f218701b700 1 -- 172.21.15.121:0/173686986 shutdown_connections 2023-11-15T02:59:45.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:45.889+0000 7f218701b700 1 -- 172.21.15.121:0/173686986 wait complete. 2023-11-15T02:59:45.912 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T02:59:45.913 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T02:59:45.913 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T02:59:45.913 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T02:59:45.914 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T02:59:45.914 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T02:59:45.914 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T02:59:45.915 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T02:59:45.915 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T02:59:45.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:45.916 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T02:59:45.935 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T02:59:45.935 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T02:59:45.935 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T02:59:45.936 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:45.936 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T02:59:45.936 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:45.936 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:59:45.936 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:59:45.937 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T02:59:45.937 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:59:45.937 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:59:45.937 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T02:59:45.937 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:59:45.938 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:59:45.938 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T02:59:45.938 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:59:45.938 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T02:59:45.938 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42254sec preferred_lft 42254sec 2023-11-15T02:59:45.939 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T02:59:45.939 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:45.939 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T02:59:45.939 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:59:45.939 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T02:59:45.940 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:45.940 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T02:59:45.940 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T02:59:45.940 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T02:59:45.940 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:45.941 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T02:59:45.941 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:45.941 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T02:59:45.941 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T02:59:45.941 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T02:59:45.942 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:45.942 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T02:59:45.942 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T02:59:45.942 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T02:59:45.942 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:45.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:45.944 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T02:59:46.020 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T02:59:46.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:46.024 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T02:59:46.088 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T02:59:46.088 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T02:59:46.089 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T02:59:46.089 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T02:59:46.089 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T02:59:46.089 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T02:59:46.089 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T02:59:46.090 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T02:59:46.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:46.091 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T02:59:46.091 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T02:59:46.091 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T02:59:46.092 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T02:59:46.092 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T02:59:46.092 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T02:59:46.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:46.223 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T02:59:46.241 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T02:59:46.241 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T02:59:46.242 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T02:59:46.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:46.243 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T02:59:46.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:46.327 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T02:59:46.389 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T02:59:46.390 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:59:46.507 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:59:46.508 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:59:46.532 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:59:46.532 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:59:46.538 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T02:59:46.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:46.539 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T02:59:46.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:59:46.569 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'mkdir -p a/{b,c}/d; sync .') 2023-11-15T02:59:46.665 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T02:59:46.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:59:46.666 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-15T02:59:46.722 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T02:59:46.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:46.723 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T02:59:46.743 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2023-11-15T02:59:46.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.953+0000 7f3f13f69700 1 Processor -- start 2023-11-15T02:59:46.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.957+0000 7f3f13f69700 1 -- start start 2023-11-15T02:59:46.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.957+0000 7f3f13f69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f0c0627f0 0x7f3f0c0c1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:46.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f13f69700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c0694c0 con 0x7f3f0c068ca0 2023-11-15T02:59:46.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f13f69700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c1547e0 con 0x7f3f0c069080 2023-11-15T02:59:46.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f13f69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c154540 con 0x7f3f0c0627f0 2023-11-15T02:59:46.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f11d05700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3f0c068ca0 0x7f3f0c150f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:36982/0 (socket says 172.21.15.121:36982) 2023-11-15T02:59:46.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f11d05700 1 -- 172.21.15.121:0/948987955 learned_addr learned my addr 172.21.15.121:0/948987955 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:46.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f12506700 1 --2- 172.21.15.121:0/948987955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f0c0627f0 0x7f3f0c0c1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:46.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f10d03700 1 -- 172.21.15.121:0/948987955 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000837029 0 0) 0x7f3f08001110 con 0x7f3f0c068ca0 2023-11-15T02:59:46.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f10d03700 1 -- 172.21.15.121:0/948987955 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ef80033f0 con 0x7f3f0c068ca0 2023-11-15T02:59:46.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f10d03700 1 -- 172.21.15.121:0/948987955 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281613380 0 0) 0x7f3f0c1547e0 con 0x7f3f0c069080 2023-11-15T02:59:46.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f10d03700 1 -- 172.21.15.121:0/948987955 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f08001110 con 0x7f3f0c069080 2023-11-15T02:59:46.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f10d03700 1 -- 172.21.15.121:0/948987955 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 472359446 0 0) 0x7f3ef80033f0 con 0x7f3f0c068ca0 2023-11-15T02:59:46.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f10d03700 1 -- 172.21.15.121:0/948987955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f0c0627f0 msgr2=0x7f3f0c0c1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:46.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f10d03700 1 --2- 172.21.15.121:0/948987955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f0c0627f0 0x7f3f0c0c1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:46.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f10d03700 1 -- 172.21.15.121:0/948987955 >> v1:172.21.15.190:6789/0 conn(0x7f3f0c069080 legacy=0x7f3f0c154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:46.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f10d03700 1 -- 172.21.15.121:0/948987955 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c155380 con 0x7f3f0c068ca0 2023-11-15T02:59:46.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f10d03700 1 -- 172.21.15.121:0/948987955 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f3f08001600 con 0x7f3f0c068ca0 2023-11-15T02:59:46.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f10d03700 1 -- 172.21.15.121:0/948987955 >> v1:172.21.15.121:6789/0 conn(0x7f3f0c068ca0 legacy=0x7f3f0c150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:46.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f10d03700 1 --2- 172.21.15.121:0/948987955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f0c0627f0 0x7f3ef8003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:46.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f10d03700 1 --2- 172.21.15.121:0/948987955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ef8003fb0 0x7f3ef8004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:46.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f10d03700 1 --2- 172.21.15.121:0/948987955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ef8004b80 0x7f3ef8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:46.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f10d03700 1 -- 172.21.15.121:0/948987955 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c154540 con 0x7f3ef8004b80 2023-11-15T02:59:46.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f10d03700 1 -- 172.21.15.121:0/948987955 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef800b990 con 0x7f3ef8003fb0 2023-11-15T02:59:46.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f10d03700 1 -- 172.21.15.121:0/948987955 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef800bb50 con 0x7f3f0c0627f0 2023-11-15T02:59:46.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f12506700 1 --2- 172.21.15.121:0/948987955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ef8004b80 0x7f3ef8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:46.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f11d05700 1 --2- 172.21.15.121:0/948987955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f0c0627f0 0x7f3ef8003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:46.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f11504700 1 --2- 172.21.15.121:0/948987955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ef8003fb0 0x7f3ef8004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:46.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f12506700 1 -- 172.21.15.121:0/948987955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f0c0627f0 msgr2=0x7f3ef8003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:46.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.961+0000 7f3f12506700 1 --2- 172.21.15.121:0/948987955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f0c0627f0 0x7f3ef8003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:46.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.965+0000 7f3f12506700 1 -- 172.21.15.121:0/948987955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ef8003fb0 msgr2=0x7f3ef8004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:46.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.965+0000 7f3f12506700 1 --2- 172.21.15.121:0/948987955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ef8003fb0 0x7f3ef8004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:46.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.965+0000 7f3f12506700 1 -- 172.21.15.121:0/948987955 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef80033f0 con 0x7f3ef8004b80 2023-11-15T02:59:46.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.965+0000 7f3f11504700 1 --2- 172.21.15.121:0/948987955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ef8003fb0 0x7f3ef8004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:46.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.965+0000 7f3f12506700 1 --2- 172.21.15.121:0/948987955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ef8004b80 0x7f3ef8005000 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f3f0400e210 tx=0x7f3f0400b040).ready entity=mon.0 client_cookie=b59c51697cdb4c79 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:46.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.965+0000 7f3f10d03700 1 -- 172.21.15.121:0/948987955 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04009ac0 con 0x7f3ef8004b80 2023-11-15T02:59:46.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.965+0000 7f3f10d03700 1 -- 172.21.15.121:0/948987955 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0400f040 con 0x7f3ef8004b80 2023-11-15T02:59:46.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.965+0000 7f3f10d03700 1 -- 172.21.15.121:0/948987955 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0400ecd0 con 0x7f3ef8004b80 2023-11-15T02:59:46.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.965+0000 7f3f13f69700 1 -- 172.21.15.121:0/948987955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ef8004b80 msgr2=0x7f3ef8005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:46.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.965+0000 7f3f13f69700 1 --2- 172.21.15.121:0/948987955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ef8004b80 0x7f3ef8005000 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f3f0400e210 tx=0x7f3f0400b040).stop 2023-11-15T02:59:46.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.965+0000 7f3f12506700 1 -- 172.21.15.121:0/948987955 reap_dead start 2023-11-15T02:59:46.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.965+0000 7f3f13f69700 1 -- 172.21.15.121:0/948987955 shutdown_connections 2023-11-15T02:59:46.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.965+0000 7f3f13f69700 1 -- 172.21.15.121:0/948987955 >> 172.21.15.121:0/948987955 conn(0x7f3f0c0c4580 msgr2=0x7f3f0c066180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:46.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.965+0000 7f3f13f69700 1 -- 172.21.15.121:0/948987955 shutdown_connections 2023-11-15T02:59:46.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.965+0000 7f3f13f69700 1 -- 172.21.15.121:0/948987955 wait complete. 2023-11-15T02:59:46.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.969+0000 7f3f13f69700 1 Processor -- start 2023-11-15T02:59:46.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.969+0000 7f3f13f69700 1 -- start start 2023-11-15T02:59:46.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.969+0000 7f3f13f69700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ef8004b80 0x7f3f0c0bf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:46.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.969+0000 7f3f13f69700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f0c153c80 0x7f3f0c1688a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:46.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.969+0000 7f3f13f69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f0c135980 0x7f3f0c135e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:46.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.969+0000 7f3f13f69700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c154380 con 0x7f3ef8004b80 2023-11-15T02:59:46.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.969+0000 7f3f13f69700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0bfed0 con 0x7f3f0c153c80 2023-11-15T02:59:46.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.969+0000 7f3f13f69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c168d90 con 0x7f3f0c135980 2023-11-15T02:59:46.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.969+0000 7f3f11d05700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ef8004b80 0x7f3f0c0bf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:46.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.969+0000 7f3f11d05700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ef8004b80 0x7f3f0c0bf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33504/0 (socket says 172.21.15.121:33504) 2023-11-15T02:59:46.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.969+0000 7f3f11d05700 1 -- 172.21.15.121:0/1543942138 learned_addr learned my addr 172.21.15.121:0/1543942138 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:46.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.969+0000 7f3f12506700 1 --2- 172.21.15.121:0/1543942138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f0c135980 0x7f3f0c135e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:46.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.969+0000 7f3f11504700 1 --2- 172.21.15.121:0/1543942138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f0c153c80 0x7f3f0c1688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:46.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.969+0000 7f3f11d05700 1 -- 172.21.15.121:0/1543942138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f0c135980 msgr2=0x7f3f0c135e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:46.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.969+0000 7f3f11d05700 1 --2- 172.21.15.121:0/1543942138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f0c135980 0x7f3f0c135e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:46.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.969+0000 7f3f11d05700 1 -- 172.21.15.121:0/1543942138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f0c153c80 msgr2=0x7f3f0c1688a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:46.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.969+0000 7f3f11d05700 1 --2- 172.21.15.121:0/1543942138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f0c153c80 0x7f3f0c1688a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:46.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.973+0000 7f3f11d05700 1 -- 172.21.15.121:0/1543942138 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c0a6c00 con 0x7f3ef8004b80 2023-11-15T02:59:46.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.973+0000 7f3f11d05700 1 --2- 172.21.15.121:0/1543942138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ef8004b80 0x7f3f0c0bf9e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f3f08004a10 tx=0x7f3f08007880).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:46.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.973+0000 7f3f01ffb700 1 -- 172.21.15.121:0/1543942138 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0800f870 con 0x7f3ef8004b80 2023-11-15T02:59:46.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.973+0000 7f3f01ffb700 1 -- 172.21.15.121:0/1543942138 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0800f9d0 con 0x7f3ef8004b80 2023-11-15T02:59:46.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.973+0000 7f3f13f69700 1 -- 172.21.15.121:0/1543942138 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0c1362f0 con 0x7f3ef8004b80 2023-11-15T02:59:46.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.973+0000 7f3f01ffb700 1 -- 172.21.15.121:0/1543942138 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0800fd70 con 0x7f3ef8004b80 2023-11-15T02:59:46.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.973+0000 7f3f13f69700 1 -- 172.21.15.121:0/1543942138 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c16c250 con 0x7f3ef8004b80 2023-11-15T02:59:46.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.977+0000 7f3f01ffb700 1 -- 172.21.15.121:0/1543942138 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3f08015620 con 0x7f3ef8004b80 2023-11-15T02:59:46.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.977+0000 7f3f01ffb700 1 --2- 172.21.15.121:0/1543942138 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ef005a510 0x7f3ef005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:46.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.977+0000 7f3f01ffb700 1 -- 172.21.15.121:0/1543942138 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3f08074b80 con 0x7f3ef8004b80 2023-11-15T02:59:46.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.977+0000 7f3f11504700 1 --2- 172.21.15.121:0/1543942138 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ef005a510 0x7f3ef005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:46.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.977+0000 7f3f1e330740 1 Processor -- start 2023-11-15T02:59:46.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.977+0000 7f3f1e330740 1 -- start start 2023-11-15T02:59:46.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.977+0000 7f3f11504700 1 --2- 172.21.15.121:0/1543942138 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ef005a510 0x7f3ef005c9d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f3efc007a40 tx=0x7f3efc009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:46.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f1e330740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15e9a70 0x15a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:46.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f1e330740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15a6990 0x15e9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:46.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f1e330740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15a0150 0x15a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:46.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f1e330740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1503540 con 0x15e9a70 2023-11-15T02:59:46.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f1e330740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x14c6e90 con 0x15a0150 2023-11-15T02:59:46.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f1e330740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x14ccd90 con 0x15a6990 2023-11-15T02:59:46.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f11d05700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15e9a70 0x15a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:46.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f11d05700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15e9a70 0x15a6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33508/0 (socket says 172.21.15.121:33508) 2023-11-15T02:59:46.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f11d05700 1 -- 172.21.15.121:0/2354601338 learned_addr learned my addr 172.21.15.121:0/2354601338 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:46.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f11504700 1 --2- 172.21.15.121:0/2354601338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15a6990 0x15e9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:46.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f12506700 1 --2- 172.21.15.121:0/2354601338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15a0150 0x15a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:46.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f11d05700 1 -- 172.21.15.121:0/2354601338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15a6990 msgr2=0x15e9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:46.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f11d05700 1 --2- 172.21.15.121:0/2354601338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15a6990 0x15e9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:46.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f11d05700 1 -- 172.21.15.121:0/2354601338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15a0150 msgr2=0x15a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:46.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f11d05700 1 --2- 172.21.15.121:0/2354601338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15a0150 0x15a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:46.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f11d05700 1 -- 172.21.15.121:0/2354601338 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x15314c0 con 0x15e9a70 2023-11-15T02:59:46.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f11d05700 1 --2- 172.21.15.121:0/2354601338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15e9a70 0x15a6180 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f3f08000e10 tx=0x7f3f08012000).ready entity=mon.0 client_cookie=bec3e3b8e22a4f58 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:46.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3ef6ffd700 1 -- 172.21.15.121:0/2354601338 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0804bf00 con 0x15e9a70 2023-11-15T02:59:46.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3ef6ffd700 1 -- 172.21.15.121:0/2354601338 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f08056030 con 0x15e9a70 2023-11-15T02:59:46.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3ef6ffd700 1 -- 172.21.15.121:0/2354601338 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f08011d20 con 0x15e9a70 2023-11-15T02:59:46.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f1e330740 1 -- 172.21.15.121:0/2354601338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15e9a70 msgr2=0x15a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:46.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f1e330740 1 --2- 172.21.15.121:0/2354601338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15e9a70 0x15a6180 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f3f08000e10 tx=0x7f3f08012000).stop 2023-11-15T02:59:46.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f1e330740 1 -- 172.21.15.121:0/2354601338 shutdown_connections 2023-11-15T02:59:46.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f1e330740 1 --2- 172.21.15.121:0/2354601338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15a0150 0x15a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:46.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f1e330740 1 --2- 172.21.15.121:0/2354601338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15a6990 0x15e9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:46.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f1e330740 1 --2- 172.21.15.121:0/2354601338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15e9a70 0x15a6180 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:46.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f1e330740 1 -- 172.21.15.121:0/2354601338 >> 172.21.15.121:0/2354601338 conn(0x1385680 msgr2=0x15a4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:47.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f1e330740 1 -- 172.21.15.121:0/2354601338 shutdown_connections 2023-11-15T02:59:47.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.981+0000 7f3f1e330740 1 -- 172.21.15.121:0/2354601338 wait complete. 2023-11-15T02:59:47.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f1e330740 1 -- start start 2023-11-15T02:59:47.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f1e330740 1 Processor -- start 2023-11-15T02:59:47.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f1e330740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15a0150 0x15e4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:47.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f1e330740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 0x15e68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:47.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f1e330740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15e9a70 0x15cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:47.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f1e330740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x15ce630 con 0x15a6990 2023-11-15T02:59:47.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f1e330740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x15e5680 con 0x15a0150 2023-11-15T02:59:47.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f1e330740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x15d1670 con 0x15e9a70 2023-11-15T02:59:47.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f1e330740 10 client.0 ll_register_callbacks cb 0x155d5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:59:47.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f11504700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 0x15e68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:47.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f11d05700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15a0150 0x15e4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:47.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f11504700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 0x15e68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33520/0 (socket says 172.21.15.121:33520) 2023-11-15T02:59:47.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f11504700 1 -- 172.21.15.121:0/2805185952 learned_addr learned my addr 172.21.15.121:0/2805185952 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:47.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f12506700 1 --2- 172.21.15.121:0/2805185952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15e9a70 0x15cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:47.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f11504700 1 -- 172.21.15.121:0/2805185952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15e9a70 msgr2=0x15cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:47.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f11504700 1 --2- 172.21.15.121:0/2805185952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15e9a70 0x15cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f11504700 1 -- 172.21.15.121:0/2805185952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15a0150 msgr2=0x15e4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:47.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f11504700 1 --2- 172.21.15.121:0/2805185952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15a0150 0x15e4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f11504700 1 -- 172.21.15.121:0/2805185952 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c16c250 con 0x15a6990 2023-11-15T02:59:47.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.985+0000 7f3f11d05700 1 --2- 172.21.15.121:0/2805185952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15a0150 0x15e4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T02:59:47.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3f11504700 1 --2- 172.21.15.121:0/2805185952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 0x15e68c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f3efc000f60 tx=0x7f3efc01c040).ready entity=mon.0 client_cookie=f90ffe732dd87980 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:47.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3eebfff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:59:47.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3eebfff700 1 -- 172.21.15.121:0/2805185952 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc01f040 con 0x15a6990 2023-11-15T02:59:47.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3eebfff700 1 -- 172.21.15.121:0/2805185952 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3efc015410 con 0x15a6990 2023-11-15T02:59:47.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3f1e330740 1 -- 172.21.15.121:0/2805185952 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x151d230 con 0x15a6990 2023-11-15T02:59:47.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3eebfff700 1 -- 172.21.15.121:0/2805185952 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3efc0157e0 con 0x15a6990 2023-11-15T02:59:47.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3f1e330740 10 client.6120 fetch_fsmap learned FSMap version 45 2023-11-15T02:59:47.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3f1e330740 1 -- 172.21.15.121:0/2805185952 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) v3 -- 0x15207c0 con 0x15a6990 2023-11-15T02:59:47.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3eebfff700 1 -- 172.21.15.121:0/2805185952 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x151d230 con 0x15a6990 2023-11-15T02:59:47.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3eebfff700 1 -- 172.21.15.121:0/2805185952 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 45) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3efc015940 con 0x15a6990 2023-11-15T02:59:47.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3f1e330740 10 client.6120 fetch_fsmap finished waiting for FSMap version 45 2023-11-15T02:59:47.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3f1e330740 10 client.6120 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-15T02:59:47.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3f1e330740 20 client.6120 populate_metadata read hostname 'smithi121' 2023-11-15T02:59:47.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3f1e330740 1 --2- 172.21.15.121:0/2805185952 >> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] conn(0x15d1ff0 0x15d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:47.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3f1e330740 4 client.6120 mds_command: new command op to 5910 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T02:59:47.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3f1e330740 1 -- 172.21.15.121:0/2805185952 --> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1513780 con 0x15d1ff0 2023-11-15T02:59:47.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3f11d05700 1 --2- 172.21.15.121:0/2805185952 >> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] conn(0x15d1ff0 0x15d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:47.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3f11d05700 1 --2- 172.21.15.121:0/2805185952 >> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] conn(0x15d1ff0 0x15d2400 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=452beeb579db2e84 server_cookie=2e6f1016c156dcee in_seq=0 out_seq=0 2023-11-15T02:59:47.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3eebfff700 10 client.6120 ms_handle_connect on v2:172.21.15.121:6835/2394175138 2023-11-15T02:59:47.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3eebfff700 1 -- 172.21.15.121:0/2805185952 <== mds.0 v2:172.21.15.121:6835/2394175138 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x1513780 con 0x15d1ff0 2023-11-15T02:59:47.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3eebfff700 10 client.6120 handle_command_reply: tid=0 2023-11-15T02:59:47.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3f1e330740 1 client.6120 shutdown 2023-11-15T02:59:47.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.989+0000 7f3f1e330740 1 -- 172.21.15.121:0/2805185952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 msgr2=0x15e68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:47.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.993+0000 7f3f1e330740 1 --2- 172.21.15.121:0/2805185952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 0x15e68c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f3efc000f60 tx=0x7f3efc01c040).stop 2023-11-15T02:59:47.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.993+0000 7f3f1e330740 1 -- 172.21.15.121:0/2805185952 shutdown_connections 2023-11-15T02:59:47.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.993+0000 7f3f1e330740 1 --2- 172.21.15.121:0/2805185952 >> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] conn(0x15d1ff0 0x15d2400 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.993+0000 7f3f1e330740 1 --2- 172.21.15.121:0/2805185952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15e9a70 0x15cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.993+0000 7f3f1e330740 1 --2- 172.21.15.121:0/2805185952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 0x15e68c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.993+0000 7f3f1e330740 1 --2- 172.21.15.121:0/2805185952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15a0150 0x15e4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.993+0000 7f3eebfff700 0 client.6120 ms_handle_reset on v2:172.21.15.121:6835/2394175138 2023-11-15T02:59:47.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.993+0000 7f3f1e330740 1 -- 172.21.15.121:0/2805185952 >> 172.21.15.121:0/2805185952 conn(0x1385680 msgr2=0x15a4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:47.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.993+0000 7f3f1e330740 1 -- 172.21.15.121:0/2805185952 shutdown_connections 2023-11-15T02:59:47.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.993+0000 7f3f1e330740 1 -- 172.21.15.121:0/2805185952 wait complete. 2023-11-15T02:59:47.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:46.993+0000 7f3f1e330740 20 client.6120 trim_cache size 0 max 16384 2023-11-15T02:59:47.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f1e330740 1 Processor -- start 2023-11-15T02:59:47.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f1e330740 1 -- start start 2023-11-15T02:59:47.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f1e330740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15a0150 0x15eab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:47.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f1e330740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 0x15d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:47.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f1e330740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15e9a70 0x15eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:47.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f1e330740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x15e3a50 con 0x15a6990 2023-11-15T02:59:47.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f1e330740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x15cb700 con 0x15e9a70 2023-11-15T02:59:47.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f1e330740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x15cb920 con 0x15a0150 2023-11-15T02:59:47.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f11504700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 0x15d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:47.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f11504700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 0x15d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33522/0 (socket says 172.21.15.121:33522) 2023-11-15T02:59:47.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f11504700 1 -- 172.21.15.121:0/3889361432 learned_addr learned my addr 172.21.15.121:0/3889361432 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:47.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f11d05700 1 --2- 172.21.15.121:0/3889361432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15a0150 0x15eab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:47.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f11504700 1 -- 172.21.15.121:0/3889361432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15a0150 msgr2=0x15eab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:47.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f12506700 1 --2- 172.21.15.121:0/3889361432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15e9a70 0x15eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:47.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f11504700 1 --2- 172.21.15.121:0/3889361432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15a0150 0x15eab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f11504700 1 -- 172.21.15.121:0/3889361432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15e9a70 msgr2=0x15eb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:47.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f11504700 1 --2- 172.21.15.121:0/3889361432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15e9a70 0x15eb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f11504700 1 -- 172.21.15.121:0/3889361432 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x15e6c20 con 0x15a6990 2023-11-15T02:59:47.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f11d05700 1 --2- 172.21.15.121:0/3889361432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15a0150 0x15eab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:47.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f12506700 1 --2- 172.21.15.121:0/3889361432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15e9a70 0x15eb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:47.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3f11504700 1 --2- 172.21.15.121:0/3889361432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 0x15d24d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f3efc001180 tx=0x7f3efc014000).ready entity=mon.0 client_cookie=1c90aea64b320e95 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:47.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.009+0000 7f3ef77fe700 1 -- 172.21.15.121:0/3889361432 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc01f030 con 0x15a6990 2023-11-15T02:59:47.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.013+0000 7f3ef77fe700 1 -- 172.21.15.121:0/3889361432 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3efc015410 con 0x15a6990 2023-11-15T02:59:47.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.013+0000 7f3ef77fe700 1 -- 172.21.15.121:0/3889361432 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc0159f0 con 0x15a6990 2023-11-15T02:59:47.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.013+0000 7f3f1e330740 1 -- 172.21.15.121:0/3889361432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 msgr2=0x15d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:47.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.013+0000 7f3f1e330740 1 --2- 172.21.15.121:0/3889361432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 0x15d24d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f3efc001180 tx=0x7f3efc014000).stop 2023-11-15T02:59:47.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.013+0000 7f3f1e330740 1 -- 172.21.15.121:0/3889361432 shutdown_connections 2023-11-15T02:59:47.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.013+0000 7f3f1e330740 1 --2- 172.21.15.121:0/3889361432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15e9a70 0x15eb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.013+0000 7f3f1e330740 1 --2- 172.21.15.121:0/3889361432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 0x15d24d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.013+0000 7f3f1e330740 1 --2- 172.21.15.121:0/3889361432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15a0150 0x15eab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.013+0000 7f3f1e330740 1 -- 172.21.15.121:0/3889361432 >> 172.21.15.121:0/3889361432 conn(0x1385680 msgr2=0x15a4820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:47.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.013+0000 7f3f1e330740 1 -- 172.21.15.121:0/3889361432 shutdown_connections 2023-11-15T02:59:47.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.013+0000 7f3f1e330740 1 -- 172.21.15.121:0/3889361432 wait complete. 2023-11-15T02:59:47.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.013+0000 7f3f1e330740 1 -- start start 2023-11-15T02:59:47.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.013+0000 7f3f1e330740 1 Processor -- start 2023-11-15T02:59:47.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f1e330740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15a0150 0x1609670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:47.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f1e330740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 0x1602950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:47.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f1e330740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15e9a70 0x1602e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:47.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f1e330740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1609f00 con 0x15a6990 2023-11-15T02:59:47.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f1e330740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x15eadd0 con 0x15a0150 2023-11-15T02:59:47.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f1e330740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x16024c0 con 0x15e9a70 2023-11-15T02:59:47.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f1e330740 10 client.0 ll_register_callbacks cb 0x7f3efc001d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:59:47.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f11504700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 0x1602950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:47.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f11504700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 0x1602950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33532/0 (socket says 172.21.15.121:33532) 2023-11-15T02:59:47.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f11504700 1 -- 172.21.15.121:0/4063107675 learned_addr learned my addr 172.21.15.121:0/4063107675 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:47.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f11d05700 1 --2- 172.21.15.121:0/4063107675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15a0150 0x1609670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:47.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f12506700 1 --2- 172.21.15.121:0/4063107675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15e9a70 0x1602e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:47.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f11504700 1 -- 172.21.15.121:0/4063107675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15e9a70 msgr2=0x1602e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:47.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f11504700 1 --2- 172.21.15.121:0/4063107675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15e9a70 0x1602e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f11504700 1 -- 172.21.15.121:0/4063107675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15a0150 msgr2=0x1609670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:47.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f11504700 1 --2- 172.21.15.121:0/4063107675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15a0150 0x1609670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f11504700 1 -- 172.21.15.121:0/4063107675 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x15207c0 con 0x15a6990 2023-11-15T02:59:47.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f11504700 1 --2- 172.21.15.121:0/4063107675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 0x1602950 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f3efc001ac0 tx=0x7f3efc032040).ready entity=mon.0 client_cookie=27edd646bf66b022 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:47.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3eebfff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:59:47.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3eebfff700 1 -- 172.21.15.121:0/4063107675 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc01f070 con 0x15a6990 2023-11-15T02:59:47.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3eebfff700 1 -- 172.21.15.121:0/4063107675 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3efc007230 con 0x15a6990 2023-11-15T02:59:47.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f1e330740 1 -- 172.21.15.121:0/4063107675 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f08056030 con 0x15a6990 2023-11-15T02:59:47.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3eebfff700 1 -- 172.21.15.121:0/4063107675 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3efc0066a0 con 0x15a6990 2023-11-15T02:59:47.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f1e330740 10 client.6126 fetch_fsmap learned FSMap version 45 2023-11-15T02:59:47.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.017+0000 7f3f1e330740 1 -- 172.21.15.121:0/4063107675 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) v3 -- 0x7f3f0c136e40 con 0x15a6990 2023-11-15T02:59:47.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.021+0000 7f3eebfff700 1 -- 172.21.15.121:0/4063107675 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f08056030 con 0x15a6990 2023-11-15T02:59:47.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.021+0000 7f3eebfff700 1 -- 172.21.15.121:0/4063107675 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 45) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3efc037040 con 0x15a6990 2023-11-15T02:59:47.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.021+0000 7f3f1e330740 10 client.6126 fetch_fsmap finished waiting for FSMap version 45 2023-11-15T02:59:47.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.021+0000 7f3f1e330740 10 client.6126 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-15T02:59:47.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.021+0000 7f3f1e330740 20 client.6126 populate_metadata read hostname 'smithi121' 2023-11-15T02:59:47.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.021+0000 7f3f1e330740 1 --2- 172.21.15.121:0/4063107675 >> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] conn(0x15e71f0 0x15e7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:47.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.021+0000 7f3f1e330740 4 client.6126 mds_command: new command op to 5910 tid=0[{"prefix": "flush journal"}] 2023-11-15T02:59:47.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.021+0000 7f3f1e330740 1 -- 172.21.15.121:0/4063107675 --> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x13dc610 con 0x15e71f0 2023-11-15T02:59:47.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.021+0000 7f3f11d05700 1 --2- 172.21.15.121:0/4063107675 >> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] conn(0x15e71f0 0x15e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:47.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.021+0000 7f3f11d05700 1 --2- 172.21.15.121:0/4063107675 >> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] conn(0x15e71f0 0x15e7600 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ad0d12d9366f3c6c server_cookie=6ed6db76fc6b0f56 in_seq=0 out_seq=0 2023-11-15T02:59:47.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.021+0000 7f3eebfff700 10 client.6126 ms_handle_connect on v2:172.21.15.121:6835/2394175138 2023-11-15T02:59:47.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.037+0000 7f3eebfff700 1 -- 172.21.15.121:0/4063107675 <== mds.0 v2:172.21.15.121:6835/2394175138 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x13dc610 con 0x15e71f0 2023-11-15T02:59:47.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.037+0000 7f3eebfff700 10 client.6126 handle_command_reply: tid=0 2023-11-15T02:59:47.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.037+0000 7f3f1e330740 1 client.6126 shutdown 2023-11-15T02:59:47.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.037+0000 7f3f1e330740 1 -- 172.21.15.121:0/4063107675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 msgr2=0x1602950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:47.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.037+0000 7f3f1e330740 1 --2- 172.21.15.121:0/4063107675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 0x1602950 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f3efc001ac0 tx=0x7f3efc032040).stop 2023-11-15T02:59:47.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.037+0000 7f3f1e330740 1 -- 172.21.15.121:0/4063107675 shutdown_connections 2023-11-15T02:59:47.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.037+0000 7f3f1e330740 1 --2- 172.21.15.121:0/4063107675 >> [v2:172.21.15.121:6835/2394175138,v1:172.21.15.121:6836/2394175138] conn(0x15e71f0 0x15e7600 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.037+0000 7f3f1e330740 1 --2- 172.21.15.121:0/4063107675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15e9a70 0x1602e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.037+0000 7f3f1e330740 1 --2- 172.21.15.121:0/4063107675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15a6990 0x1602950 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.037+0000 7f3f1e330740 1 --2- 172.21.15.121:0/4063107675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15a0150 0x1609670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.037+0000 7f3eebfff700 0 client.6126 ms_handle_reset on v2:172.21.15.121:6835/2394175138 2023-11-15T02:59:47.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.037+0000 7f3f1e330740 1 -- 172.21.15.121:0/4063107675 >> 172.21.15.121:0/4063107675 conn(0x1385680 msgr2=0x15cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:47.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.041+0000 7f3f1e330740 1 -- 172.21.15.121:0/4063107675 shutdown_connections 2023-11-15T02:59:47.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.041+0000 7f3f1e330740 1 -- 172.21.15.121:0/4063107675 wait complete. 2023-11-15T02:59:47.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.041+0000 7f3f1e330740 20 client.6126 trim_cache size 0 max 16384 2023-11-15T02:59:47.046 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T02:59:47.046 INFO:teuthology.orchestra.run.smithi121.stdout: "message": "", 2023-11-15T02:59:47.046 INFO:teuthology.orchestra.run.smithi121.stdout: "return_code": 0 2023-11-15T02:59:47.047 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T02:59:47.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.041+0000 7f3f13f69700 1 -- 172.21.15.121:0/1543942138 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ef005a510 msgr2=0x7f3ef005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:47.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.041+0000 7f3f13f69700 1 --2- 172.21.15.121:0/1543942138 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ef005a510 0x7f3ef005c9d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f3efc007a40 tx=0x7f3efc009040).stop 2023-11-15T02:59:47.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.041+0000 7f3f13f69700 1 -- 172.21.15.121:0/1543942138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ef8004b80 msgr2=0x7f3f0c0bf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:47.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.041+0000 7f3f13f69700 1 --2- 172.21.15.121:0/1543942138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ef8004b80 0x7f3f0c0bf9e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f3f08004a10 tx=0x7f3f08007880).stop 2023-11-15T02:59:47.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.041+0000 7f3f13f69700 1 -- 172.21.15.121:0/1543942138 shutdown_connections 2023-11-15T02:59:47.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.041+0000 7f3f13f69700 1 --2- 172.21.15.121:0/1543942138 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ef005a510 0x7f3ef005c9d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.041+0000 7f3f13f69700 1 --2- 172.21.15.121:0/1543942138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f0c135980 0x7f3f0c135e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.041+0000 7f3f13f69700 1 --2- 172.21.15.121:0/1543942138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f0c153c80 0x7f3f0c1688a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.041+0000 7f3f13f69700 1 --2- 172.21.15.121:0/1543942138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ef8004b80 0x7f3f0c0bf9e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.041+0000 7f3f13f69700 1 -- 172.21.15.121:0/1543942138 >> 172.21.15.121:0/1543942138 conn(0x7f3f0c0c4580 msgr2=0x7f3f0c0c2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:47.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.041+0000 7f3f13f69700 1 -- 172.21.15.121:0/1543942138 shutdown_connections 2023-11-15T02:59:47.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.041+0000 7f3f13f69700 1 -- 172.21.15.121:0/1543942138 wait complete. 2023-11-15T02:59:47.063 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T02:59:47.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.205+0000 7fa5f14ff700 1 Processor -- start 2023-11-15T02:59:47.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.205+0000 7fa5f14ff700 1 -- start start 2023-11-15T02:59:47.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.205+0000 7fa5f14ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5ec066180 0x7fa5ec158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:47.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.205+0000 7fa5f14ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa5ec0c58f0 0x7fa5ec0c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:47.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.205+0000 7fa5f14ff700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ec0c2390 con 0x7fa5ec0c7810 2023-11-15T02:59:47.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.205+0000 7fa5f14ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ec0c4650 con 0x7fa5ec0c58f0 2023-11-15T02:59:47.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.205+0000 7fa5f14ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec0c6260 con 0x7fa5ec066180 2023-11-15T02:59:47.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.205+0000 7fa5ea7fc700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa5ec0c7810 0x7fa5ec0c4ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:36992/0 (socket says 172.21.15.121:36992) 2023-11-15T02:59:47.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.205+0000 7fa5ea7fc700 1 -- 172.21.15.121:0/2940580799 learned_addr learned my addr 172.21.15.121:0/2940580799 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:47.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5eaffd700 1 --2- 172.21.15.121:0/2940580799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5ec066180 0x7fa5ec158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:47.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5ebfff700 1 --2- 172.21.15.121:0/2940580799 >> v2:172.21.15.190:3300/0 conn(0x7fa5ec0c58f0 0x7fa5ec0c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:47.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5e9ffb700 1 -- 172.21.15.121:0/2940580799 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441575643 0 0) 0x7fa5ec0c2390 con 0x7fa5ec0c7810 2023-11-15T02:59:47.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5e9ffb700 1 -- 172.21.15.121:0/2940580799 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5d40033f0 con 0x7fa5ec0c7810 2023-11-15T02:59:47.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5eaffd700 1 -- 172.21.15.121:0/2940580799 >> v2:172.21.15.190:3300/0 conn(0x7fa5ec0c58f0 msgr2=0x7fa5ec0c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:47.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5eaffd700 1 --2- 172.21.15.121:0/2940580799 >> v2:172.21.15.190:3300/0 conn(0x7fa5ec0c58f0 0x7fa5ec0c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5eaffd700 1 -- 172.21.15.121:0/2940580799 >> v1:172.21.15.121:6789/0 conn(0x7fa5ec0c7810 legacy=0x7fa5ec0c4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:47.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5eaffd700 1 -- 172.21.15.121:0/2940580799 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ec1593e0 con 0x7fa5ec066180 2023-11-15T02:59:47.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5eaffd700 1 --2- 172.21.15.121:0/2940580799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5ec066180 0x7fa5ec158ef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fa5dc009760 tx=0x7fa5dc009060).ready entity=mon.2 client_cookie=cbdb5d2e2e1adae4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:47.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5e9ffb700 1 -- 172.21.15.121:0/2940580799 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc00d4a0 con 0x7fa5ec066180 2023-11-15T02:59:47.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5e9ffb700 1 -- 172.21.15.121:0/2940580799 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5dc00d600 con 0x7fa5ec066180 2023-11-15T02:59:47.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5e9ffb700 1 -- 172.21.15.121:0/2940580799 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc00d980 con 0x7fa5ec066180 2023-11-15T02:59:47.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5f14ff700 1 -- 172.21.15.121:0/2940580799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5ec066180 msgr2=0x7fa5ec158ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:47.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5f14ff700 1 --2- 172.21.15.121:0/2940580799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5ec066180 0x7fa5ec158ef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fa5dc009760 tx=0x7fa5dc009060).stop 2023-11-15T02:59:47.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5f14ff700 1 -- 172.21.15.121:0/2940580799 shutdown_connections 2023-11-15T02:59:47.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5f14ff700 1 --2- 172.21.15.121:0/2940580799 >> v2:172.21.15.190:3300/0 conn(0x7fa5ec0c58f0 0x7fa5ec0c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5f14ff700 1 --2- 172.21.15.121:0/2940580799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5ec066180 0x7fa5ec158ef0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5f14ff700 1 -- 172.21.15.121:0/2940580799 >> 172.21.15.121:0/2940580799 conn(0x7fa5ec069960 msgr2=0x7fa5ec069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:47.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5f14ff700 1 -- 172.21.15.121:0/2940580799 shutdown_connections 2023-11-15T02:59:47.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.209+0000 7fa5f14ff700 1 -- 172.21.15.121:0/2940580799 wait complete. 2023-11-15T02:59:47.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.213+0000 7fa5f14ff700 1 Processor -- start 2023-11-15T02:59:47.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.213+0000 7fa5f14ff700 1 -- start start 2023-11-15T02:59:47.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5f14ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5ec066180 0x7fa5ec15ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:47.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5f14ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5ec0c58f0 0x7fa5ec0ad250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:47.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5f14ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5ec0c7810 0x7fa5ec150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:47.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5f14ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec0acdc0 con 0x7fa5ec0c7810 2023-11-15T02:59:47.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5f14ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec0c1cf0 con 0x7fa5ec0c58f0 2023-11-15T02:59:47.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5f14ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec0ac1b0 con 0x7fa5ec066180 2023-11-15T02:59:47.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5ebfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5ec0c7810 0x7fa5ec150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:47.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5eaffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5ec066180 0x7fa5ec15ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:47.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5ebfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5ec0c7810 0x7fa5ec150340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33548/0 (socket says 172.21.15.121:33548) 2023-11-15T02:59:47.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5ebfff700 1 -- 172.21.15.121:0/3725547860 learned_addr learned my addr 172.21.15.121:0/3725547860 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:47.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5ea7fc700 1 --2- 172.21.15.121:0/3725547860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5ec0c58f0 0x7fa5ec0ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:47.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5ebfff700 1 -- 172.21.15.121:0/3725547860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5ec066180 msgr2=0x7fa5ec15ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:47.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5ebfff700 1 --2- 172.21.15.121:0/3725547860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5ec066180 0x7fa5ec15ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5ebfff700 1 -- 172.21.15.121:0/3725547860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5ec0c58f0 msgr2=0x7fa5ec0ad250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:47.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5ebfff700 1 --2- 172.21.15.121:0/3725547860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5ec0c58f0 0x7fa5ec0ad250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5ebfff700 1 -- 172.21.15.121:0/3725547860 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5dc0092a0 con 0x7fa5ec0c7810 2023-11-15T02:59:47.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5ebfff700 1 --2- 172.21.15.121:0/3725547860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5ec0c7810 0x7fa5ec150340 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fa5e4009ec0 tx=0x7fa5e400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:47.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5d2ffd700 1 -- 172.21.15.121:0/3725547860 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e400cb70 con 0x7fa5ec0c7810 2023-11-15T02:59:47.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5d2ffd700 1 -- 172.21.15.121:0/3725547860 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e400f040 con 0x7fa5ec0c7810 2023-11-15T02:59:47.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5d2ffd700 1 -- 172.21.15.121:0/3725547860 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e400cee0 con 0x7fa5ec0c7810 2023-11-15T02:59:47.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5f14ff700 1 -- 172.21.15.121:0/3725547860 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ec0ac490 con 0x7fa5ec0c7810 2023-11-15T02:59:47.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.217+0000 7fa5f14ff700 1 -- 172.21.15.121:0/3725547860 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ec0aca50 con 0x7fa5ec0c7810 2023-11-15T02:59:47.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.221+0000 7fa5f14ff700 1 -- 172.21.15.121:0/3725547860 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ec0acf70 con 0x7fa5ec0c7810 2023-11-15T02:59:47.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.221+0000 7fa5d2ffd700 1 -- 172.21.15.121:0/3725547860 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa5e4013610 con 0x7fa5ec0c7810 2023-11-15T02:59:47.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.221+0000 7fa5d2ffd700 1 --2- 172.21.15.121:0/3725547860 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa5d805a510 0x7fa5d805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:47.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.221+0000 7fa5d2ffd700 1 -- 172.21.15.121:0/3725547860 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fa5e4074f90 con 0x7fa5ec0c7810 2023-11-15T02:59:47.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.221+0000 7fa5eaffd700 1 --2- 172.21.15.121:0/3725547860 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa5d805a510 0x7fa5d805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:47.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.225+0000 7fa5eaffd700 1 --2- 172.21.15.121:0/3725547860 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa5d805a510 0x7fa5d805c9d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7fa5dc00d260 tx=0x7fa5dc016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:47.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.225+0000 7fa5d2ffd700 1 -- 172.21.15.121:0/3725547860 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa5e40a3af0 con 0x7fa5ec0c7810 2023-11-15T02:59:47.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.421+0000 7fa5f14ff700 1 -- 172.21.15.121:0/3725547860 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa5ec0c51f0 con 0x7fa5ec0c7810 2023-11-15T02:59:47.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.421+0000 7fa5d2ffd700 1 -- 172.21.15.121:0/3725547860 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa5ec0acf70 con 0x7fa5ec0c7810 2023-11-15T02:59:47.425 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T02:59:47.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.425+0000 7fa5f14ff700 1 -- 172.21.15.121:0/3725547860 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa5d805a510 msgr2=0x7fa5d805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:47.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.425+0000 7fa5f14ff700 1 --2- 172.21.15.121:0/3725547860 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa5d805a510 0x7fa5d805c9d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7fa5dc00d260 tx=0x7fa5dc016000).stop 2023-11-15T02:59:47.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.425+0000 7fa5f14ff700 1 -- 172.21.15.121:0/3725547860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5ec0c7810 msgr2=0x7fa5ec150340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:47.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.425+0000 7fa5f14ff700 1 --2- 172.21.15.121:0/3725547860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5ec0c7810 0x7fa5ec150340 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fa5e4009ec0 tx=0x7fa5e400b040).stop 2023-11-15T02:59:47.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.425+0000 7fa5f14ff700 1 -- 172.21.15.121:0/3725547860 shutdown_connections 2023-11-15T02:59:47.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.425+0000 7fa5f14ff700 1 --2- 172.21.15.121:0/3725547860 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa5d805a510 0x7fa5d805c9d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.425+0000 7fa5f14ff700 1 --2- 172.21.15.121:0/3725547860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5ec0c7810 0x7fa5ec150340 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.425+0000 7fa5f14ff700 1 --2- 172.21.15.121:0/3725547860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5ec0c58f0 0x7fa5ec0ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.425+0000 7fa5f14ff700 1 --2- 172.21.15.121:0/3725547860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5ec066180 0x7fa5ec15ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:47.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.425+0000 7fa5f14ff700 1 -- 172.21.15.121:0/3725547860 >> 172.21.15.121:0/3725547860 conn(0x7fa5ec069960 msgr2=0x7fa5ec05e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:47.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.425+0000 7fa5f14ff700 1 -- 172.21.15.121:0/3725547860 shutdown_connections 2023-11-15T02:59:47.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:47.425+0000 7fa5f14ff700 1 -- 172.21.15.121:0/3725547860 wait complete. 2023-11-15T02:59:47.448 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T02:59:47.449 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T02:59:47.449 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T02:59:47.449 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T02:59:47.450 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T02:59:47.450 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T02:59:47.450 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T02:59:47.451 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T02:59:47.451 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T02:59:47.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:47.452 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T02:59:47.470 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T02:59:47.471 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T02:59:47.471 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T02:59:47.471 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:47.471 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T02:59:47.471 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:47.472 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:59:47.472 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:59:47.472 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T02:59:47.472 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:59:47.472 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:59:47.473 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T02:59:47.473 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T02:59:47.473 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:59:47.473 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T02:59:47.473 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:59:47.474 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T02:59:47.474 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42252sec preferred_lft 42252sec 2023-11-15T02:59:47.474 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T02:59:47.474 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:47.474 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T02:59:47.475 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T02:59:47.475 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T02:59:47.475 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:47.475 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T02:59:47.475 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T02:59:47.476 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T02:59:47.476 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:47.476 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T02:59:47.476 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:47.476 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T02:59:47.476 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T02:59:47.477 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T02:59:47.477 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:47.477 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T02:59:47.477 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T02:59:47.478 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T02:59:47.478 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T02:59:47.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:47.479 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T02:59:47.562 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T02:59:47.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:47.565 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T02:59:47.631 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T02:59:47.631 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T02:59:47.631 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T02:59:47.632 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T02:59:47.632 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T02:59:47.632 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T02:59:47.632 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T02:59:47.632 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T02:59:47.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:47.633 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T02:59:47.634 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T02:59:47.634 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T02:59:47.634 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T02:59:47.635 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T02:59:47.635 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T02:59:47.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:47.775 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T02:59:47.790 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T02:59:47.790 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T02:59:47.791 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T02:59:47.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:47.792 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T02:59:47.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:47.870 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T02:59:47.930 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T02:59:47.930 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:59:48.055 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:59:48.056 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:59:48.077 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T02:59:48.078 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T02:59:48.083 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T02:59:48.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:48.084 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T02:59:48.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:59:48.115 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'chmod 711 a/b/d; sync .') 2023-11-15T02:59:48.192 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T02:59:48.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T02:59:48.193 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-15T02:59:48.291 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T02:59:48.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T02:59:48.292 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T02:59:48.308 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T02:59:48.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.505+0000 7effe2ddb700 1 Processor -- start 2023-11-15T02:59:48.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.505+0000 7effe2ddb700 1 -- start start 2023-11-15T02:59:48.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.509+0000 7effe2ddb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effdc0c4740 0x7effdc094e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:48.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.509+0000 7effe2ddb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7effdc0a6e70 0x7effdc0bc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:48.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.509+0000 7effe2ddb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effdc0be5a0 con 0x7effdc153b80 2023-11-15T02:59:48.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.509+0000 7effe2ddb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effdc0a6a20 con 0x7effdc0a6e70 2023-11-15T02:59:48.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.509+0000 7effe2ddb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effdc095350 con 0x7effdc0c4740 2023-11-15T02:59:48.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.509+0000 7effdbfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effdc0c4740 0x7effdc094e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:48.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.509+0000 7effe1378700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7effdc0a6e70 0x7effdc0bc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:48.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.509+0000 7effdbfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effdc0c4740 0x7effdc094e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:48796/0 (socket says 172.21.15.121:48796) 2023-11-15T02:59:48.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.509+0000 7effdbfff700 1 -- 172.21.15.121:0/1680111961 learned_addr learned my addr 172.21.15.121:0/1680111961 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:48.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.509+0000 7effdbfff700 1 -- 172.21.15.121:0/1680111961 >> v2:172.21.15.190:3300/0 conn(0x7effdc0a6e70 msgr2=0x7effdc0bc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:48.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.509+0000 7effdbfff700 1 --2- 172.21.15.121:0/1680111961 >> v2:172.21.15.190:3300/0 conn(0x7effdc0a6e70 0x7effdc0bc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:48.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.509+0000 7effdbfff700 1 -- 172.21.15.121:0/1680111961 >> v1:172.21.15.190:6789/0 conn(0x7effdc153b80 legacy=0x7effdc0c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:48.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.509+0000 7effdbfff700 1 -- 172.21.15.121:0/1680111961 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc155150 con 0x7effdc0c4740 2023-11-15T02:59:48.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.509+0000 7effdbfff700 1 --2- 172.21.15.121:0/1680111961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effdc0c4740 0x7effdc094e60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7effd0009f20 tx=0x7effd0009840).ready entity=mon.2 client_cookie=243611b5a2b1a768 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:48.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.509+0000 7effdb7fe700 1 -- 172.21.15.121:0/1680111961 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd000e070 con 0x7effdc0c4740 2023-11-15T02:59:48.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.509+0000 7effdb7fe700 1 -- 172.21.15.121:0/1680111961 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd000b9d0 con 0x7effdc0c4740 2023-11-15T02:59:48.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.509+0000 7effdb7fe700 1 -- 172.21.15.121:0/1680111961 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd000bd50 con 0x7effdc0c4740 2023-11-15T02:59:48.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.513+0000 7effe2ddb700 1 -- 172.21.15.121:0/1680111961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effdc0c4740 msgr2=0x7effdc094e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:48.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.513+0000 7effe2ddb700 1 --2- 172.21.15.121:0/1680111961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effdc0c4740 0x7effdc094e60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7effd0009f20 tx=0x7effd0009840).stop 2023-11-15T02:59:48.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.513+0000 7effe2ddb700 1 -- 172.21.15.121:0/1680111961 shutdown_connections 2023-11-15T02:59:48.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.513+0000 7effe2ddb700 1 --2- 172.21.15.121:0/1680111961 >> v2:172.21.15.190:3300/0 conn(0x7effdc0a6e70 0x7effdc0bc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:48.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.513+0000 7effe2ddb700 1 --2- 172.21.15.121:0/1680111961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effdc0c4740 0x7effdc094e60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:48.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.513+0000 7effe2ddb700 1 -- 172.21.15.121:0/1680111961 >> 172.21.15.121:0/1680111961 conn(0x7effdc067480 msgr2=0x7effdc068b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:48.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.513+0000 7effe2ddb700 1 -- 172.21.15.121:0/1680111961 shutdown_connections 2023-11-15T02:59:48.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.513+0000 7effe2ddb700 1 -- 172.21.15.121:0/1680111961 wait complete. 2023-11-15T02:59:48.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effe2ddb700 1 Processor -- start 2023-11-15T02:59:48.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effe2ddb700 1 -- start start 2023-11-15T02:59:48.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effe2ddb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effdc0a6e70 0x7effdc145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:48.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effe2ddb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effdc0c4740 0x7effdc0be870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:48.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effe2ddb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effdc153b80 0x7effdc0a7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:48.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effe2ddb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc0bfe70 con 0x7effdc0c4740 2023-11-15T02:59:48.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effe2ddb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc0bfad0 con 0x7effdc0a6e70 2023-11-15T02:59:48.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effe2ddb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effdc0bf230 con 0x7effdc153b80 2023-11-15T02:59:48.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effdbfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effdc0c4740 0x7effdc0be870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:48.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effdbfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effdc0c4740 0x7effdc0be870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33554/0 (socket says 172.21.15.121:33554) 2023-11-15T02:59:48.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effdbfff700 1 -- 172.21.15.121:0/2274358880 learned_addr learned my addr 172.21.15.121:0/2274358880 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:48.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effe1378700 1 --2- 172.21.15.121:0/2274358880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effdc153b80 0x7effdc0a7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:48.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effe0b77700 1 --2- 172.21.15.121:0/2274358880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effdc0a6e70 0x7effdc145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:48.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effdbfff700 1 -- 172.21.15.121:0/2274358880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effdc153b80 msgr2=0x7effdc0a7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:48.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effdbfff700 1 --2- 172.21.15.121:0/2274358880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effdc153b80 0x7effdc0a7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:48.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effdbfff700 1 -- 172.21.15.121:0/2274358880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effdc0a6e70 msgr2=0x7effdc145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:48.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effdbfff700 1 --2- 172.21.15.121:0/2274358880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effdc0a6e70 0x7effdc145270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:48.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effdbfff700 1 -- 172.21.15.121:0/2274358880 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd0009ae0 con 0x7effdc0c4740 2023-11-15T02:59:48.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effdbfff700 1 --2- 172.21.15.121:0/2274358880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effdc0c4740 0x7effdc0be870 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7effdc0a6b60 tx=0x7effd0009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:48.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effd8ff9700 1 -- 172.21.15.121:0/2274358880 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd00092e0 con 0x7effdc0c4740 2023-11-15T02:59:48.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effd8ff9700 1 -- 172.21.15.121:0/2274358880 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd000bb40 con 0x7effdc0c4740 2023-11-15T02:59:48.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effe2ddb700 1 -- 172.21.15.121:0/2274358880 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effdc0bf510 con 0x7effdc0c4740 2023-11-15T02:59:48.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effd8ff9700 1 -- 172.21.15.121:0/2274358880 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd001e470 con 0x7effdc0c4740 2023-11-15T02:59:48.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.517+0000 7effe2ddb700 1 -- 172.21.15.121:0/2274358880 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effdc145760 con 0x7effdc0c4740 2023-11-15T02:59:48.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.521+0000 7effe2ddb700 1 -- 172.21.15.121:0/2274358880 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effdc047060 con 0x7effdc0c4740 2023-11-15T02:59:48.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.521+0000 7effd8ff9700 1 -- 172.21.15.121:0/2274358880 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7effd0003040 con 0x7effdc0c4740 2023-11-15T02:59:48.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.521+0000 7effd8ff9700 1 --2- 172.21.15.121:0/2274358880 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effc805a510 0x7effc805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:48.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.521+0000 7effd8ff9700 1 -- 172.21.15.121:0/2274358880 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6121+0+0 (secure 0 0 0) 0x7effd0075c60 con 0x7effdc0c4740 2023-11-15T02:59:48.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.521+0000 7effe0b77700 1 --2- 172.21.15.121:0/2274358880 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effc805a510 0x7effc805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:48.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.525+0000 7effe0b77700 1 --2- 172.21.15.121:0/2274358880 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effc805a510 0x7effc805c9d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7effcc003630 tx=0x7effcc009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:48.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.529+0000 7effd8ff9700 1 -- 172.21.15.121:0/2274358880 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7effd0049430 con 0x7effdc0c4740 2023-11-15T02:59:48.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:48.729+0000 7effe2ddb700 1 -- 172.21.15.121:0/2274358880 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7effdc069c30 con 0x7effdc0c4740 2023-11-15T02:59:49.053 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T02:59:49.049+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T02:59:49.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.057+0000 7effd8ff9700 1 -- 172.21.15.121:0/2274358880 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) v1 ==== 161+0+0 (secure 0 0 0) 0x7effd0023030 con 0x7effdc0c4740 2023-11-15T02:59:49.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.065+0000 7effe2ddb700 1 -- 172.21.15.121:0/2274358880 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effc805a510 msgr2=0x7effc805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:49.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.065+0000 7effe2ddb700 1 --2- 172.21.15.121:0/2274358880 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effc805a510 0x7effc805c9d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7effcc003630 tx=0x7effcc009040).stop 2023-11-15T02:59:49.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.065+0000 7effe2ddb700 1 -- 172.21.15.121:0/2274358880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effdc0c4740 msgr2=0x7effdc0be870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:49.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.065+0000 7effe2ddb700 1 --2- 172.21.15.121:0/2274358880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effdc0c4740 0x7effdc0be870 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7effdc0a6b60 tx=0x7effd0009840).stop 2023-11-15T02:59:49.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.065+0000 7effe2ddb700 1 -- 172.21.15.121:0/2274358880 shutdown_connections 2023-11-15T02:59:49.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.065+0000 7effe2ddb700 1 --2- 172.21.15.121:0/2274358880 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effc805a510 0x7effc805c9d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:49.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.065+0000 7effe2ddb700 1 --2- 172.21.15.121:0/2274358880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effdc153b80 0x7effdc0a7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:49.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.065+0000 7effe2ddb700 1 --2- 172.21.15.121:0/2274358880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effdc0c4740 0x7effdc0be870 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:49.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.065+0000 7effe2ddb700 1 --2- 172.21.15.121:0/2274358880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effdc0a6e70 0x7effdc145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:49.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.065+0000 7effe2ddb700 1 -- 172.21.15.121:0/2274358880 >> 172.21.15.121:0/2274358880 conn(0x7effdc067480 msgr2=0x7effdc0c28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:49.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.065+0000 7effe2ddb700 1 -- 172.21.15.121:0/2274358880 shutdown_connections 2023-11-15T02:59:49.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.065+0000 7effe2ddb700 1 -- 172.21.15.121:0/2274358880 wait complete. 2023-11-15T02:59:49.071 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T02:59:49.075 INFO:tasks.ceph.mds.a.smithi121.stderr: -3692> 2023-11-15T02:59:40.197+0000 7fd03f594700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T02:59:49.090 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_journal_corrupt_dentry_first 1.0 2023-11-15T02:59:49.123 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T02:59:49.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.249+0000 7f1bd2779700 1 Processor -- start 2023-11-15T02:59:49.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.249+0000 7f1bd2779700 1 -- start start 2023-11-15T02:59:49.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.253+0000 7f1bd2779700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1bcc0acb60 0x7f1bcc0acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:49.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.253+0000 7f1bd2779700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bcc065680 0x7f1bcc065b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:49.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.253+0000 7f1bd2779700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bcc0c7e90 con 0x7f1bcc14e380 2023-11-15T02:59:49.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.253+0000 7f1bd2779700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bcc0c8320 con 0x7f1bcc0acb60 2023-11-15T02:59:49.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.253+0000 7f1bd2779700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc0c7080 con 0x7f1bcc065680 2023-11-15T02:59:49.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.253+0000 7f1bd0f76700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1bcc0acb60 0x7f1bcc0acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:49.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.253+0000 7f1bd0f76700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1bcc0acb60 0x7f1bcc0acfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33592/0 (socket says 172.21.15.121:33592) 2023-11-15T02:59:49.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.253+0000 7f1bd0f76700 1 -- 172.21.15.121:0/3575466818 learned_addr learned my addr 172.21.15.121:0/3575466818 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:49.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.253+0000 7f1bd1f78700 1 --2- 172.21.15.121:0/3575466818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bcc065680 0x7f1bcc065b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:49.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.253+0000 7f1bd0f76700 1 -- 172.21.15.121:0/3575466818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bcc065680 msgr2=0x7f1bcc065b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:49.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.253+0000 7f1bd0f76700 1 --2- 172.21.15.121:0/3575466818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bcc065680 0x7f1bcc065b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:49.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.253+0000 7f1bd0f76700 1 -- 172.21.15.121:0/3575466818 >> v1:172.21.15.190:6789/0 conn(0x7f1bcc14e380 legacy=0x7f1bcc0c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:49.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.253+0000 7f1bd0f76700 1 -- 172.21.15.121:0/3575466818 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc0ad4d0 con 0x7f1bcc0acb60 2023-11-15T02:59:49.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.253+0000 7f1bd0f76700 1 --2- 172.21.15.121:0/3575466818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bcc0acb60 0x7f1bcc0acfe0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f1bbc006fd0 tx=0x7f1bbc0090f0).ready entity=mon.0 client_cookie=4ca4951edfe34d2d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:49.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.253+0000 7f1bc3fff700 1 -- 172.21.15.121:0/3575466818 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00b600 con 0x7f1bcc0acb60 2023-11-15T02:59:49.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.253+0000 7f1bc3fff700 1 -- 172.21.15.121:0/3575466818 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bbc00b760 con 0x7f1bcc0acb60 2023-11-15T02:59:49.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.253+0000 7f1bc3fff700 1 -- 172.21.15.121:0/3575466818 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00bae0 con 0x7f1bcc0acb60 2023-11-15T02:59:49.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.253+0000 7f1bd2779700 1 -- 172.21.15.121:0/3575466818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bcc0acb60 msgr2=0x7f1bcc0acfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:49.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.253+0000 7f1bd2779700 1 --2- 172.21.15.121:0/3575466818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bcc0acb60 0x7f1bcc0acfe0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f1bbc006fd0 tx=0x7f1bbc0090f0).stop 2023-11-15T02:59:49.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.257+0000 7f1bd2779700 1 -- 172.21.15.121:0/3575466818 shutdown_connections 2023-11-15T02:59:49.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.257+0000 7f1bd2779700 1 --2- 172.21.15.121:0/3575466818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bcc065680 0x7f1bcc065b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:49.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.257+0000 7f1bd2779700 1 --2- 172.21.15.121:0/3575466818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bcc0acb60 0x7f1bcc0acfe0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:49.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.257+0000 7f1bd2779700 1 -- 172.21.15.121:0/3575466818 >> 172.21.15.121:0/3575466818 conn(0x7f1bcc067a60 msgr2=0x7f1bcc067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:49.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.257+0000 7f1bd2779700 1 -- 172.21.15.121:0/3575466818 shutdown_connections 2023-11-15T02:59:49.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.257+0000 7f1bd2779700 1 -- 172.21.15.121:0/3575466818 wait complete. 2023-11-15T02:59:49.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd2779700 1 Processor -- start 2023-11-15T02:59:49.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd2779700 1 -- start start 2023-11-15T02:59:49.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd2779700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bcc065680 0x7f1bcc0b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:49.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd2779700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bcc0acb60 0x7f1bcc14aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:49.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd2779700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bcc14e380 0x7f1bcc0bfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:49.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd2779700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc14b280 con 0x7f1bcc14e380 2023-11-15T02:59:49.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd2779700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc0b0880 con 0x7f1bcc065680 2023-11-15T02:59:49.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd2779700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc168090 con 0x7f1bcc0acb60 2023-11-15T02:59:49.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd1f78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bcc14e380 0x7f1bcc0bfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:49.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd1f78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bcc14e380 0x7f1bcc0bfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33596/0 (socket says 172.21.15.121:33596) 2023-11-15T02:59:49.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd1f78700 1 -- 172.21.15.121:0/200211354 learned_addr learned my addr 172.21.15.121:0/200211354 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:49.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd0f76700 1 --2- 172.21.15.121:0/200211354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bcc0acb60 0x7f1bcc14aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:49.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd1777700 1 --2- 172.21.15.121:0/200211354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bcc065680 0x7f1bcc0b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:49.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd1f78700 1 -- 172.21.15.121:0/200211354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bcc0acb60 msgr2=0x7f1bcc14aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:49.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd1f78700 1 --2- 172.21.15.121:0/200211354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bcc0acb60 0x7f1bcc14aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:49.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd1f78700 1 -- 172.21.15.121:0/200211354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bcc065680 msgr2=0x7f1bcc0b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:49.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd1f78700 1 --2- 172.21.15.121:0/200211354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bcc065680 0x7f1bcc0b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:49.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd1f78700 1 -- 172.21.15.121:0/200211354 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bbc009370 con 0x7f1bcc14e380 2023-11-15T02:59:49.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd1f78700 1 --2- 172.21.15.121:0/200211354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bcc14e380 0x7f1bcc0bfdf0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f1bc4009fa0 tx=0x7f1bc4009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:49.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bc17fa700 1 -- 172.21.15.121:0/200211354 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc400ec70 con 0x7f1bcc14e380 2023-11-15T02:59:49.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bc17fa700 1 -- 172.21.15.121:0/200211354 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc400edd0 con 0x7f1bcc14e380 2023-11-15T02:59:49.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd2779700 1 -- 172.21.15.121:0/200211354 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bcc168370 con 0x7f1bcc14e380 2023-11-15T02:59:49.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bc17fa700 1 -- 172.21.15.121:0/200211354 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4014610 con 0x7f1bcc14e380 2023-11-15T02:59:49.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.261+0000 7f1bd2779700 1 -- 172.21.15.121:0/200211354 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bcc168930 con 0x7f1bcc14e380 2023-11-15T02:59:49.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.265+0000 7f1bd2779700 1 -- 172.21.15.121:0/200211354 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bcc14b450 con 0x7f1bcc14e380 2023-11-15T02:59:49.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.265+0000 7f1bc17fa700 1 -- 172.21.15.121:0/200211354 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1bc401d3b0 con 0x7f1bcc14e380 2023-11-15T02:59:49.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.269+0000 7f1bc17fa700 1 --2- 172.21.15.121:0/200211354 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1bb005a540 0x7f1bb005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:49.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.269+0000 7f1bc17fa700 1 -- 172.21.15.121:0/200211354 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f1bc4074c90 con 0x7f1bcc14e380 2023-11-15T02:59:49.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.269+0000 7f1bd1777700 1 --2- 172.21.15.121:0/200211354 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1bb005a540 0x7f1bb005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:49.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.269+0000 7f1bd1777700 1 --2- 172.21.15.121:0/200211354 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1bb005a540 0x7f1bb005ca00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f1bcc14a990 tx=0x7f1bc8009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:49.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.273+0000 7f1bc17fa700 1 -- 172.21.15.121:0/200211354 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1bcc14b450 con 0x7f1bcc14e380 2023-11-15T02:59:49.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.437+0000 7f1bd2779700 1 -- 172.21.15.121:0/200211354 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command([{prefix=config set, name=mds_inject_journal_corrupt_dentry_first}] v 0) v1 -- 0x7f1bcc14b450 con 0x7f1bcc14e380 2023-11-15T02:59:49.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.445+0000 7f1bc17fa700 1 -- 172.21.15.121:0/200211354 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_journal_corrupt_dentry_first}]=0 v11)=0 v11) v1 ==== 139+0+0 (secure 0 0 0) 0x7f1bcc14b450 con 0x7f1bcc14e380 2023-11-15T02:59:49.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.449+0000 7f1bd2779700 1 -- 172.21.15.121:0/200211354 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1bb005a540 msgr2=0x7f1bb005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:49.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.449+0000 7f1bd2779700 1 --2- 172.21.15.121:0/200211354 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1bb005a540 0x7f1bb005ca00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f1bcc14a990 tx=0x7f1bc8009040).stop 2023-11-15T02:59:49.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.449+0000 7f1bd2779700 1 -- 172.21.15.121:0/200211354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bcc14e380 msgr2=0x7f1bcc0bfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:49.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.449+0000 7f1bd2779700 1 --2- 172.21.15.121:0/200211354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bcc14e380 0x7f1bcc0bfdf0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f1bc4009fa0 tx=0x7f1bc4009900).stop 2023-11-15T02:59:49.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.449+0000 7f1bd2779700 1 -- 172.21.15.121:0/200211354 shutdown_connections 2023-11-15T02:59:49.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.449+0000 7f1bd2779700 1 --2- 172.21.15.121:0/200211354 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1bb005a540 0x7f1bb005ca00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:49.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.449+0000 7f1bd2779700 1 --2- 172.21.15.121:0/200211354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bcc14e380 0x7f1bcc0bfdf0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:49.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.449+0000 7f1bd2779700 1 --2- 172.21.15.121:0/200211354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bcc0acb60 0x7f1bcc14aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:49.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.449+0000 7f1bd2779700 1 --2- 172.21.15.121:0/200211354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bcc065680 0x7f1bcc0b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:49.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.449+0000 7f1bd2779700 1 -- 172.21.15.121:0/200211354 >> 172.21.15.121:0/200211354 conn(0x7f1bcc067a60 msgr2=0x7f1bcc0c11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:49.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.449+0000 7f1bd2779700 1 -- 172.21.15.121:0/200211354 shutdown_connections 2023-11-15T02:59:49.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:49.449+0000 7f1bd2779700 1 -- 172.21.15.121:0/200211354 wait complete. 2023-11-15T02:59:54.473 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T02:59:54.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.645+0000 7f61baf47700 1 Processor -- start 2023-11-15T02:59:54.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.645+0000 7f61baf47700 1 -- start start 2023-11-15T02:59:54.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.649+0000 7f61baf47700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f61b40acd00 0x7f61b40ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:54.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.649+0000 7f61baf47700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f61b4066330 0x7f61b4153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:54.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.649+0000 7f61baf47700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b4065ea0 con 0x7f61b40c5c70 2023-11-15T02:59:54.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.649+0000 7f61baf47700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b4066bb0 con 0x7f61b40acd00 2023-11-15T02:59:54.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.649+0000 7f61baf47700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b4065890 con 0x7f61b4066330 2023-11-15T02:59:54.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.649+0000 7f61b9744700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f61b40c5c70 0x7f61b41536c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:37008/0 (socket says 172.21.15.121:37008) 2023-11-15T02:59:54.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.649+0000 7f61b9744700 1 -- 172.21.15.121:0/1160491352 learned_addr learned my addr 172.21.15.121:0/1160491352 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:54.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.649+0000 7f61b9f45700 1 --2- 172.21.15.121:0/1160491352 >> v2:172.21.15.121:3300/0 conn(0x7f61b40acd00 0x7f61b40ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:54.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.649+0000 7f61ba746700 1 --2- 172.21.15.121:0/1160491352 >> v2:172.21.15.190:3300/0 conn(0x7f61b4066330 0x7f61b4153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:54.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.649+0000 7f61b8f43700 1 -- 172.21.15.121:0/1160491352 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 461999129 0 0) 0x7f61b4065ea0 con 0x7f61b40c5c70 2023-11-15T02:59:54.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.649+0000 7f61b9f45700 1 -- 172.21.15.121:0/1160491352 >> v2:172.21.15.190:3300/0 conn(0x7f61b4066330 msgr2=0x7f61b4153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:54.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.649+0000 7f61b9f45700 1 --2- 172.21.15.121:0/1160491352 >> v2:172.21.15.190:3300/0 conn(0x7f61b4066330 0x7f61b4153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:54.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.649+0000 7f61b9f45700 1 -- 172.21.15.121:0/1160491352 >> v1:172.21.15.121:6789/0 conn(0x7f61b40c5c70 legacy=0x7f61b41536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:54.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.649+0000 7f61b9f45700 1 -- 172.21.15.121:0/1160491352 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b40c64a0 con 0x7f61b40acd00 2023-11-15T02:59:54.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.649+0000 7f61b9f45700 1 --2- 172.21.15.121:0/1160491352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61b40acd00 0x7f61b40ad0e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f61b40c60d0 tx=0x7f61b0006f40).ready entity=mon.0 client_cookie=6b07f6cf039afade server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:54.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.649+0000 7f61b8f43700 1 -- 172.21.15.121:0/1160491352 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0005880 con 0x7f61b40acd00 2023-11-15T02:59:54.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.649+0000 7f61b8f43700 1 -- 172.21.15.121:0/1160491352 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b00059e0 con 0x7f61b40acd00 2023-11-15T02:59:54.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.649+0000 7f61b8f43700 1 -- 172.21.15.121:0/1160491352 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0005d60 con 0x7f61b40acd00 2023-11-15T02:59:54.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.653+0000 7f61baf47700 1 -- 172.21.15.121:0/1160491352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61b40acd00 msgr2=0x7f61b40ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:54.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.653+0000 7f61baf47700 1 --2- 172.21.15.121:0/1160491352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61b40acd00 0x7f61b40ad0e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f61b40c60d0 tx=0x7f61b0006f40).stop 2023-11-15T02:59:54.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.653+0000 7f61baf47700 1 -- 172.21.15.121:0/1160491352 shutdown_connections 2023-11-15T02:59:54.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.653+0000 7f61baf47700 1 --2- 172.21.15.121:0/1160491352 >> v2:172.21.15.190:3300/0 conn(0x7f61b4066330 0x7f61b4153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:54.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.653+0000 7f61baf47700 1 --2- 172.21.15.121:0/1160491352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61b40acd00 0x7f61b40ad0e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:54.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.653+0000 7f61baf47700 1 -- 172.21.15.121:0/1160491352 >> 172.21.15.121:0/1160491352 conn(0x7f61b4063fc0 msgr2=0x7f61b4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:54.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.653+0000 7f61baf47700 1 -- 172.21.15.121:0/1160491352 shutdown_connections 2023-11-15T02:59:54.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.653+0000 7f61baf47700 1 -- 172.21.15.121:0/1160491352 wait complete. 2023-11-15T02:59:54.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61baf47700 1 Processor -- start 2023-11-15T02:59:54.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61baf47700 1 -- start start 2023-11-15T02:59:54.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61baf47700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61b4066330 0x7f61b4169370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:54.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61baf47700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61b40acd00 0x7f61b40cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:54.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61baf47700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61b40c5c70 0x7f61b40c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:54.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61baf47700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b4169860 con 0x7f61b40acd00 2023-11-15T02:59:54.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61baf47700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b40c3c50 con 0x7f61b4066330 2023-11-15T02:59:54.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61baf47700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b40c3f30 con 0x7f61b40c5c70 2023-11-15T02:59:54.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61b9744700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61b40acd00 0x7f61b40cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:54.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61b9744700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61b40acd00 0x7f61b40cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33614/0 (socket says 172.21.15.121:33614) 2023-11-15T02:59:54.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61b9744700 1 -- 172.21.15.121:0/904732884 learned_addr learned my addr 172.21.15.121:0/904732884 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:54.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61ba746700 1 --2- 172.21.15.121:0/904732884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61b40c5c70 0x7f61b40c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:54.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61b9f45700 1 --2- 172.21.15.121:0/904732884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61b4066330 0x7f61b4169370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:54.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61b9744700 1 -- 172.21.15.121:0/904732884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61b40c5c70 msgr2=0x7f61b40c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:54.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61b9744700 1 --2- 172.21.15.121:0/904732884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61b40c5c70 0x7f61b40c1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:54.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61b9744700 1 -- 172.21.15.121:0/904732884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61b4066330 msgr2=0x7f61b4169370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:54.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61b9744700 1 --2- 172.21.15.121:0/904732884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61b4066330 0x7f61b4169370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:54.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61b9744700 1 -- 172.21.15.121:0/904732884 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b0009250 con 0x7f61b40acd00 2023-11-15T02:59:54.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61b9744700 1 --2- 172.21.15.121:0/904732884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61b40acd00 0x7f61b40cae20 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f61a4001eb0 tx=0x7f61a4007e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:54.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.657+0000 7f61a9ffb700 1 -- 172.21.15.121:0/904732884 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a40070d0 con 0x7f61b40acd00 2023-11-15T02:59:54.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.661+0000 7f61a9ffb700 1 -- 172.21.15.121:0/904732884 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a4007230 con 0x7f61b40acd00 2023-11-15T02:59:54.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.661+0000 7f61baf47700 1 -- 172.21.15.121:0/904732884 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b40c4210 con 0x7f61b40acd00 2023-11-15T02:59:54.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.661+0000 7f61a9ffb700 1 -- 172.21.15.121:0/904732884 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a40075d0 con 0x7f61b40acd00 2023-11-15T02:59:54.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.661+0000 7f61baf47700 1 -- 172.21.15.121:0/904732884 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b40c47d0 con 0x7f61b40acd00 2023-11-15T02:59:54.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.661+0000 7f61baf47700 1 -- 172.21.15.121:0/904732884 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b40c49f0 con 0x7f61b40acd00 2023-11-15T02:59:54.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.661+0000 7f61a9ffb700 1 -- 172.21.15.121:0/904732884 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f61a4005bc0 con 0x7f61b40acd00 2023-11-15T02:59:54.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.665+0000 7f61a9ffb700 1 --2- 172.21.15.121:0/904732884 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f619805a510 0x7f619805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:54.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.665+0000 7f61a9ffb700 1 -- 172.21.15.121:0/904732884 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f61a4074130 con 0x7f61b40acd00 2023-11-15T02:59:54.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.665+0000 7f61b9f45700 1 --2- 172.21.15.121:0/904732884 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f619805a510 0x7f619805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:54.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.665+0000 7f61b9f45700 1 --2- 172.21.15.121:0/904732884 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f619805a510 0x7f619805c9d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f61b40c35e0 tx=0x7f61b0016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:54.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.669+0000 7f61a9ffb700 1 -- 172.21.15.121:0/904732884 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f61a4047800 con 0x7f61b40acd00 2023-11-15T02:59:54.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:54.865+0000 7f61baf47700 1 -- 172.21.15.121:0/904732884 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f61b40c61c0 con 0x7f61b40acd00 2023-11-15T02:59:55.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.105+0000 7f61a9ffb700 1 -- 172.21.15.121:0/904732884 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v48) v1 ==== 162+0+0 (secure 0 0 0) 0x7f61a404ae20 con 0x7f61b40acd00 2023-11-15T02:59:55.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.109+0000 7f61baf47700 1 -- 172.21.15.121:0/904732884 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f619805a510 msgr2=0x7f619805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:55.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.109+0000 7f61baf47700 1 --2- 172.21.15.121:0/904732884 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f619805a510 0x7f619805c9d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f61b40c35e0 tx=0x7f61b0016000).stop 2023-11-15T02:59:55.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.109+0000 7f61baf47700 1 -- 172.21.15.121:0/904732884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61b40acd00 msgr2=0x7f61b40cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:55.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.109+0000 7f61baf47700 1 --2- 172.21.15.121:0/904732884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61b40acd00 0x7f61b40cae20 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f61a4001eb0 tx=0x7f61a4007e90).stop 2023-11-15T02:59:55.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.109+0000 7f61baf47700 1 -- 172.21.15.121:0/904732884 shutdown_connections 2023-11-15T02:59:55.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.109+0000 7f61baf47700 1 --2- 172.21.15.121:0/904732884 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f619805a510 0x7f619805c9d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:55.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.109+0000 7f61baf47700 1 --2- 172.21.15.121:0/904732884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61b40c5c70 0x7f61b40c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:55.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.109+0000 7f61baf47700 1 --2- 172.21.15.121:0/904732884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61b40acd00 0x7f61b40cae20 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:55.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.109+0000 7f61baf47700 1 --2- 172.21.15.121:0/904732884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61b4066330 0x7f61b4169370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:55.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.109+0000 7f61baf47700 1 -- 172.21.15.121:0/904732884 >> 172.21.15.121:0/904732884 conn(0x7f61b4063fc0 msgr2=0x7f61b414e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:55.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.109+0000 7f61baf47700 1 -- 172.21.15.121:0/904732884 shutdown_connections 2023-11-15T02:59:55.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.109+0000 7f61baf47700 1 -- 172.21.15.121:0/904732884 wait complete. 2023-11-15T02:59:55.117 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T02:59:55.138 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:59:55.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.293+0000 7ff7d9369700 1 Processor -- start 2023-11-15T02:59:55.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.293+0000 7ff7d9369700 1 -- start start 2023-11-15T02:59:55.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7d9369700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff7d40ac5d0 0x7ff7d40c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:55.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7d9369700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d40c39b0 0x7ff7d40c1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:55.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7d9369700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d4066950 con 0x7ff7d40ac1f0 2023-11-15T02:59:55.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7d9369700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d4158790 con 0x7ff7d40ac5d0 2023-11-15T02:59:55.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7d9369700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d40674f0 con 0x7ff7d40c39b0 2023-11-15T02:59:55.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7ca7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff7d40ac5d0 0x7ff7d40c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:55.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7d2ffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff7d40ac1f0 0x7ff7d4066de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45568/0 (socket says 172.21.15.121:45568) 2023-11-15T02:59:55.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7d2ffd700 1 -- 172.21.15.121:0/2999458662 learned_addr learned my addr 172.21.15.121:0/2999458662 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:55.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7d37fe700 1 --2- 172.21.15.121:0/2999458662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d40c39b0 0x7ff7d40c1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:55.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7ca7fc700 1 -- 172.21.15.121:0/2999458662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d40c39b0 msgr2=0x7ff7d40c1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:55.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7ca7fc700 1 --2- 172.21.15.121:0/2999458662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d40c39b0 0x7ff7d40c1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:55.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7ca7fc700 1 -- 172.21.15.121:0/2999458662 >> v1:172.21.15.121:6789/0 conn(0x7ff7d40ac1f0 legacy=0x7ff7d4066de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:55.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7ca7fc700 1 -- 172.21.15.121:0/2999458662 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d4065d40 con 0x7ff7d40ac5d0 2023-11-15T02:59:55.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7ca7fc700 1 --2- 172.21.15.121:0/2999458662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7d40ac5d0 0x7ff7d40c34c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7ff7bc009760 tx=0x7ff7bc009060).ready entity=mon.0 client_cookie=bd5452cae0c09f13 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:55.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7d27fc700 1 -- 172.21.15.121:0/2999458662 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc00f510 con 0x7ff7d40ac5d0 2023-11-15T02:59:55.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7d27fc700 1 -- 172.21.15.121:0/2999458662 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7bc00f670 con 0x7ff7d40ac5d0 2023-11-15T02:59:55.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7d27fc700 1 -- 172.21.15.121:0/2999458662 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc00f9f0 con 0x7ff7d40ac5d0 2023-11-15T02:59:55.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7d9369700 1 -- 172.21.15.121:0/2999458662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7d40ac5d0 msgr2=0x7ff7d40c34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:55.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7d9369700 1 --2- 172.21.15.121:0/2999458662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7d40ac5d0 0x7ff7d40c34c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7ff7bc009760 tx=0x7ff7bc009060).stop 2023-11-15T02:59:55.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7d9369700 1 -- 172.21.15.121:0/2999458662 shutdown_connections 2023-11-15T02:59:55.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7d9369700 1 --2- 172.21.15.121:0/2999458662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d40c39b0 0x7ff7d40c1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:55.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7d9369700 1 --2- 172.21.15.121:0/2999458662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7d40ac5d0 0x7ff7d40c34c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:55.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.297+0000 7ff7d9369700 1 -- 172.21.15.121:0/2999458662 >> 172.21.15.121:0/2999458662 conn(0x7ff7d4062190 msgr2=0x7ff7d4062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:55.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.301+0000 7ff7d9369700 1 -- 172.21.15.121:0/2999458662 shutdown_connections 2023-11-15T02:59:55.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.301+0000 7ff7d9369700 1 -- 172.21.15.121:0/2999458662 wait complete. 2023-11-15T02:59:55.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.301+0000 7ff7d9369700 1 Processor -- start 2023-11-15T02:59:55.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.301+0000 7ff7d9369700 1 -- start start 2023-11-15T02:59:55.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7d9369700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d40ac1f0 0x7ff7d4156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:55.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7d9369700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d40ac5d0 0x7ff7d40b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:55.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7d9369700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7d40c39b0 0x7ff7d40b7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:55.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7d9369700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d41572c0 con 0x7ff7d40c39b0 2023-11-15T02:59:55.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7d9369700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d4168ad0 con 0x7ff7d40ac1f0 2023-11-15T02:59:55.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7d9369700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d40b7700 con 0x7ff7d40ac5d0 2023-11-15T02:59:55.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7d37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7d40c39b0 0x7ff7d40b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:55.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7d37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7d40c39b0 0x7ff7d40b7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50046/0 (socket says 172.21.15.121:50046) 2023-11-15T02:59:55.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7d37fe700 1 -- 172.21.15.121:0/3101776636 learned_addr learned my addr 172.21.15.121:0/3101776636 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:55.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7ca7fc700 1 --2- 172.21.15.121:0/3101776636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d40ac5d0 0x7ff7d40b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:55.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7d2ffd700 1 --2- 172.21.15.121:0/3101776636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d40ac1f0 0x7ff7d4156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:55.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7d37fe700 1 -- 172.21.15.121:0/3101776636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d40ac5d0 msgr2=0x7ff7d40b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:55.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7d37fe700 1 --2- 172.21.15.121:0/3101776636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d40ac5d0 0x7ff7d40b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:55.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7d37fe700 1 -- 172.21.15.121:0/3101776636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d40ac1f0 msgr2=0x7ff7d4156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:55.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7d37fe700 1 --2- 172.21.15.121:0/3101776636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d40ac1f0 0x7ff7d4156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:55.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7d37fe700 1 -- 172.21.15.121:0/3101776636 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc0092a0 con 0x7ff7d40c39b0 2023-11-15T02:59:55.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7d2ffd700 1 --2- 172.21.15.121:0/3101776636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d40ac1f0 0x7ff7d4156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:55.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7d37fe700 1 --2- 172.21.15.121:0/3101776636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7d40c39b0 0x7ff7d40b7130 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7ff7cc009d70 tx=0x7ff7cc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:55.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7cb7fe700 1 -- 172.21.15.121:0/3101776636 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc00cb50 con 0x7ff7d40c39b0 2023-11-15T02:59:55.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7cb7fe700 1 -- 172.21.15.121:0/3101776636 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7cc00f040 con 0x7ff7d40c39b0 2023-11-15T02:59:55.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7d9369700 1 -- 172.21.15.121:0/3101776636 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d40b79e0 con 0x7ff7d40c39b0 2023-11-15T02:59:55.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7cb7fe700 1 -- 172.21.15.121:0/3101776636 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc013400 con 0x7ff7d40c39b0 2023-11-15T02:59:55.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.305+0000 7ff7d9369700 1 -- 172.21.15.121:0/3101776636 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d40b7fa0 con 0x7ff7d40c39b0 2023-11-15T02:59:55.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.309+0000 7ff7d9369700 1 -- 172.21.15.121:0/3101776636 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d406bcf0 con 0x7ff7d40c39b0 2023-11-15T02:59:55.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.309+0000 7ff7cb7fe700 1 -- 172.21.15.121:0/3101776636 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff7cc0137b0 con 0x7ff7d40c39b0 2023-11-15T02:59:55.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.309+0000 7ff7cb7fe700 1 --2- 172.21.15.121:0/3101776636 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff7b405a510 0x7ff7b405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:55.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.309+0000 7ff7cb7fe700 1 -- 172.21.15.121:0/3101776636 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ff7cc074fe0 con 0x7ff7d40c39b0 2023-11-15T02:59:55.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.313+0000 7ff7d2ffd700 1 --2- 172.21.15.121:0/3101776636 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff7b405a510 0x7ff7b405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:55.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.313+0000 7ff7d2ffd700 1 --2- 172.21.15.121:0/3101776636 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff7b405a510 0x7ff7b405c9d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7ff7c4001740 tx=0x7ff7c4009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:55.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.317+0000 7ff7cb7fe700 1 -- 172.21.15.121:0/3101776636 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff7cc0a3af0 con 0x7ff7d40c39b0 2023-11-15T02:59:55.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.513+0000 7ff7d9369700 1 -- 172.21.15.121:0/3101776636 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7d4066020 con 0x7ff7d40c39b0 2023-11-15T02:59:55.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.513+0000 7ff7cb7fe700 1 -- 172.21.15.121:0/3101776636 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4503 (secure 0 0 0) 0x7ff7cc048710 con 0x7ff7d40c39b0 2023-11-15T02:59:55.517 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:55.517 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":49,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3462858472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3462858472},{"type":"v1","addr":"172.21.15.190:6833","nonce":3462858472}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4042390087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4042390087},{"type":"v1","addr":"172.21.15.190:6835","nonce":4042390087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5907,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2239114529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2239114529},{"type":"v1","addr":"172.21.15.121:6837","nonce":2239114529}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:59:34.667864+0000","modified":"2023-11-15T02:59:55.108880+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":49,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/3429169582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3429169582},{"type":"v1","addr":"172.21.15.121:6836","nonce":3429169582}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-15T02:59:55.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.517+0000 7ff7d9369700 1 -- 172.21.15.121:0/3101776636 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff7b405a510 msgr2=0x7ff7b405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:55.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.517+0000 7ff7d9369700 1 --2- 172.21.15.121:0/3101776636 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff7b405a510 0x7ff7b405c9d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7ff7c4001740 tx=0x7ff7c4009040).stop 2023-11-15T02:59:55.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.517+0000 7ff7d9369700 1 -- 172.21.15.121:0/3101776636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7d40c39b0 msgr2=0x7ff7d40b7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:55.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.517+0000 7ff7d9369700 1 --2- 172.21.15.121:0/3101776636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7d40c39b0 0x7ff7d40b7130 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7ff7cc009d70 tx=0x7ff7cc00b040).stop 2023-11-15T02:59:55.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.517+0000 7ff7d9369700 1 -- 172.21.15.121:0/3101776636 shutdown_connections 2023-11-15T02:59:55.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.517+0000 7ff7d9369700 1 --2- 172.21.15.121:0/3101776636 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff7b405a510 0x7ff7b405c9d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:55.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.517+0000 7ff7d9369700 1 --2- 172.21.15.121:0/3101776636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7d40c39b0 0x7ff7d40b7130 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:55.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.517+0000 7ff7d9369700 1 --2- 172.21.15.121:0/3101776636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d40ac5d0 0x7ff7d40b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:55.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.517+0000 7ff7d9369700 1 --2- 172.21.15.121:0/3101776636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d40ac1f0 0x7ff7d4156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:55.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.517+0000 7ff7d9369700 1 -- 172.21.15.121:0/3101776636 >> 172.21.15.121:0/3101776636 conn(0x7ff7d4062190 msgr2=0x7ff7d41588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:55.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.517+0000 7ff7d9369700 1 -- 172.21.15.121:0/3101776636 shutdown_connections 2023-11-15T02:59:55.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:55.517+0000 7ff7d9369700 1 -- 172.21.15.121:0/3101776636 wait complete. 2023-11-15T02:59:55.522 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 49 2023-11-15T02:59:55.941 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T02:59:34.667864+0000', 'modified': '2023-11-15T02:59:55.108880+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 62, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6144}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6144': {'gid': 6144, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.121:6836/3429169582', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6835', 'nonce': 3429169582}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 3429169582}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T02:59:55.941 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6144:up:replay 2023-11-15T02:59:56.943 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:59:57.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.109+0000 7fc160a1a700 1 Processor -- start 2023-11-15T02:59:57.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.109+0000 7fc160a1a700 1 -- start start 2023-11-15T02:59:57.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.113+0000 7fc160a1a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc15c067150 0x7fc15c14e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:57.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.113+0000 7fc160a1a700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc15c0a9350 con 0x7fc15c0a87f0 2023-11-15T02:59:57.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.113+0000 7fc160a1a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc15c151420 con 0x7fc15c066d70 2023-11-15T02:59:57.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.113+0000 7fc160a1a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc15c150ed0 con 0x7fc15c067150 2023-11-15T02:59:57.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.113+0000 7fc15259c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc15c0a87f0 0x7fc15c0c29a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45578/0 (socket says 172.21.15.121:45578) 2023-11-15T02:59:57.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.113+0000 7fc15259c700 1 -- 172.21.15.121:0/2793022592 learned_addr learned my addr 172.21.15.121:0/2793022592 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:57.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.113+0000 7fc15b59e700 1 --2- 172.21.15.121:0/2793022592 >> v2:172.21.15.190:3300/0 conn(0x7fc15c067150 0x7fc15c14e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:57.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.113+0000 7fc15a59c700 1 -- 172.21.15.121:0/2793022592 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1326473454 0 0) 0x7fc15c0a9350 con 0x7fc15c0a87f0 2023-11-15T02:59:57.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.113+0000 7fc15a59c700 1 -- 172.21.15.121:0/2793022592 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1480033f0 con 0x7fc15c0a87f0 2023-11-15T02:59:57.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.113+0000 7fc15b59e700 1 -- 172.21.15.121:0/2793022592 >> v1:172.21.15.190:6789/0 conn(0x7fc15c066d70 legacy=0x7fc15c1518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:57.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.113+0000 7fc15b59e700 1 -- 172.21.15.121:0/2793022592 >> v1:172.21.15.121:6789/0 conn(0x7fc15c0a87f0 legacy=0x7fc15c0c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:57.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.113+0000 7fc15b59e700 1 -- 172.21.15.121:0/2793022592 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c0a8d90 con 0x7fc15c067150 2023-11-15T02:59:57.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.113+0000 7fc15b59e700 1 --2- 172.21.15.121:0/2793022592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc15c067150 0x7fc15c14e000 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fc154009e90 tx=0x7fc154009780).ready entity=mon.1 client_cookie=8593ce793f65ba2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:57.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.113+0000 7fc15a59c700 1 -- 172.21.15.121:0/2793022592 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15400bb20 con 0x7fc15c067150 2023-11-15T02:59:57.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.117+0000 7fc15a59c700 1 -- 172.21.15.121:0/2793022592 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15400bc80 con 0x7fc15c067150 2023-11-15T02:59:57.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.117+0000 7fc15a59c700 1 -- 172.21.15.121:0/2793022592 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc154011480 con 0x7fc15c067150 2023-11-15T02:59:57.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.117+0000 7fc160a1a700 1 -- 172.21.15.121:0/2793022592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc15c067150 msgr2=0x7fc15c14e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:57.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.117+0000 7fc160a1a700 1 --2- 172.21.15.121:0/2793022592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc15c067150 0x7fc15c14e000 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fc154009e90 tx=0x7fc154009780).stop 2023-11-15T02:59:57.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.117+0000 7fc160a1a700 1 -- 172.21.15.121:0/2793022592 shutdown_connections 2023-11-15T02:59:57.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.117+0000 7fc160a1a700 1 --2- 172.21.15.121:0/2793022592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc15c067150 0x7fc15c14e000 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:57.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.117+0000 7fc160a1a700 1 -- 172.21.15.121:0/2793022592 >> 172.21.15.121:0/2793022592 conn(0x7fc15c062190 msgr2=0x7fc15c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:57.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.117+0000 7fc160a1a700 1 -- 172.21.15.121:0/2793022592 shutdown_connections 2023-11-15T02:59:57.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.117+0000 7fc160a1a700 1 -- 172.21.15.121:0/2793022592 wait complete. 2023-11-15T02:59:57.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc160a1a700 1 Processor -- start 2023-11-15T02:59:57.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc160a1a700 1 -- start start 2023-11-15T02:59:57.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc160a1a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc15c066d70 0x7fc15c0c0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:57.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc160a1a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc15c067150 0x7fc15c14fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:57.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc160a1a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc15c0a87f0 0x7fc15c150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:57.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc160a1a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c14e0e0 con 0x7fc15c067150 2023-11-15T02:59:57.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc160a1a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c0bff50 con 0x7fc15c066d70 2023-11-15T02:59:57.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc160a1a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c0c0230 con 0x7fc15c0a87f0 2023-11-15T02:59:57.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc15259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc15c067150 0x7fc15c14fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:57.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc15259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc15c067150 0x7fc15c14fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50052/0 (socket says 172.21.15.121:50052) 2023-11-15T02:59:57.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc15259c700 1 -- 172.21.15.121:0/412503694 learned_addr learned my addr 172.21.15.121:0/412503694 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:57.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc15b59e700 1 --2- 172.21.15.121:0/412503694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc15c0a87f0 0x7fc15c150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:57.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc15ad9d700 1 --2- 172.21.15.121:0/412503694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc15c066d70 0x7fc15c0c0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:57.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc15259c700 1 -- 172.21.15.121:0/412503694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc15c0a87f0 msgr2=0x7fc15c150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:57.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc15259c700 1 --2- 172.21.15.121:0/412503694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc15c0a87f0 0x7fc15c150360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:57.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc15259c700 1 -- 172.21.15.121:0/412503694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc15c066d70 msgr2=0x7fc15c0c0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:57.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc15259c700 1 --2- 172.21.15.121:0/412503694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc15c066d70 0x7fc15c0c0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:57.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc15259c700 1 -- 172.21.15.121:0/412503694 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc154009a20 con 0x7fc15c067150 2023-11-15T02:59:57.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc15259c700 1 --2- 172.21.15.121:0/412503694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc15c067150 0x7fc15c14fe70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fc14400c420 tx=0x7fc144007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:57.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc1537fe700 1 -- 172.21.15.121:0/412503694 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1440070d0 con 0x7fc15c067150 2023-11-15T02:59:57.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.121+0000 7fc1537fe700 1 -- 172.21.15.121:0/412503694 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc144007230 con 0x7fc15c067150 2023-11-15T02:59:57.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.125+0000 7fc160a1a700 1 -- 172.21.15.121:0/412503694 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc15c0c0510 con 0x7fc15c067150 2023-11-15T02:59:57.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.125+0000 7fc1537fe700 1 -- 172.21.15.121:0/412503694 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1440075b0 con 0x7fc15c067150 2023-11-15T02:59:57.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.125+0000 7fc160a1a700 1 -- 172.21.15.121:0/412503694 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15c0c0ad0 con 0x7fc15c067150 2023-11-15T02:59:57.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.125+0000 7fc160a1a700 1 -- 172.21.15.121:0/412503694 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc15c06bcf0 con 0x7fc15c067150 2023-11-15T02:59:57.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.125+0000 7fc1537fe700 1 -- 172.21.15.121:0/412503694 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc1440059b0 con 0x7fc15c067150 2023-11-15T02:59:57.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.129+0000 7fc1537fe700 1 --2- 172.21.15.121:0/412503694 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc13c05a510 0x7fc13c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:57.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.129+0000 7fc1537fe700 1 -- 172.21.15.121:0/412503694 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fc14404b320 con 0x7fc15c067150 2023-11-15T02:59:57.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.129+0000 7fc15ad9d700 1 --2- 172.21.15.121:0/412503694 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc13c05a510 0x7fc13c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:57.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.129+0000 7fc15ad9d700 1 --2- 172.21.15.121:0/412503694 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc13c05a510 0x7fc13c05c9d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fc14c003900 tx=0x7fc14c007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:57.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.133+0000 7fc1537fe700 1 -- 172.21.15.121:0/412503694 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc1440476c0 con 0x7fc15c067150 2023-11-15T02:59:57.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.325+0000 7fc160a1a700 1 -- 172.21.15.121:0/412503694 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc15c0669b0 con 0x7fc15c067150 2023-11-15T02:59:57.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.329+0000 7fc1537fe700 1 -- 172.21.15.121:0/412503694 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4503 (secure 0 0 0) 0x7fc144018020 con 0x7fc15c067150 2023-11-15T02:59:57.331 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:57.332 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":51,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3462858472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3462858472},{"type":"v1","addr":"172.21.15.190:6833","nonce":3462858472}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4042390087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4042390087},{"type":"v1","addr":"172.21.15.190:6835","nonce":4042390087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5907,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2239114529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2239114529},{"type":"v1","addr":"172.21.15.121:6837","nonce":2239114529}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:59:34.667864+0000","modified":"2023-11-15T02:59:57.124107+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":49,"state":"up:rejoin","state_seq":4,"addr":"172.21.15.121:6836/3429169582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3429169582},{"type":"v1","addr":"172.21.15.121:6836","nonce":3429169582}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-15T02:59:57.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.329+0000 7fc160a1a700 1 -- 172.21.15.121:0/412503694 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc13c05a510 msgr2=0x7fc13c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:57.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.329+0000 7fc160a1a700 1 --2- 172.21.15.121:0/412503694 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc13c05a510 0x7fc13c05c9d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fc14c003900 tx=0x7fc14c007d40).stop 2023-11-15T02:59:57.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.329+0000 7fc160a1a700 1 -- 172.21.15.121:0/412503694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc15c067150 msgr2=0x7fc15c14fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:57.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.329+0000 7fc160a1a700 1 --2- 172.21.15.121:0/412503694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc15c067150 0x7fc15c14fe70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fc14400c420 tx=0x7fc144007c80).stop 2023-11-15T02:59:57.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.329+0000 7fc160a1a700 1 -- 172.21.15.121:0/412503694 shutdown_connections 2023-11-15T02:59:57.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.329+0000 7fc160a1a700 1 --2- 172.21.15.121:0/412503694 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc13c05a510 0x7fc13c05c9d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:57.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.329+0000 7fc160a1a700 1 --2- 172.21.15.121:0/412503694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc15c0a87f0 0x7fc15c150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:57.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.329+0000 7fc160a1a700 1 --2- 172.21.15.121:0/412503694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc15c067150 0x7fc15c14fe70 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:57.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.329+0000 7fc160a1a700 1 --2- 172.21.15.121:0/412503694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc15c066d70 0x7fc15c0c0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:57.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.329+0000 7fc160a1a700 1 -- 172.21.15.121:0/412503694 >> 172.21.15.121:0/412503694 conn(0x7fc15c062190 msgr2=0x7fc15c14bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:57.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.329+0000 7fc160a1a700 1 -- 172.21.15.121:0/412503694 shutdown_connections 2023-11-15T02:59:57.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:57.329+0000 7fc160a1a700 1 -- 172.21.15.121:0/412503694 wait complete. 2023-11-15T02:59:57.336 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 51 2023-11-15T02:59:57.760 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T02:59:34.667864+0000', 'modified': '2023-11-15T02:59:57.124107+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 62, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6144}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6144': {'gid': 6144, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:rejoin', 'state_seq': 4, 'addr': '172.21.15.121:6836/3429169582', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6835', 'nonce': 3429169582}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 3429169582}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T02:59:57.760 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6144:up:rejoin 2023-11-15T02:59:58.762 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T02:59:58.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.925+0000 7f9a5e85b700 1 Processor -- start 2023-11-15T02:59:58.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.929+0000 7f9a5e85b700 1 -- start start 2023-11-15T02:59:58.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.929+0000 7f9a5e85b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9a58153bb0 0x7f9a580a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:58.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.929+0000 7f9a5e85b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a580cb5c0 con 0x7f9a58068170 2023-11-15T02:59:58.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.929+0000 7f9a5e85b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a580c9ca0 con 0x7f9a580aa020 2023-11-15T02:59:58.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.929+0000 7f9a5e85b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a580aa6f0 con 0x7f9a58153bb0 2023-11-15T02:59:58.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.929+0000 7f9a5cdf8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9a58068170 0x7f9a58065d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45586/0 (socket says 172.21.15.121:45586) 2023-11-15T02:59:58.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.929+0000 7f9a5cdf8700 1 -- 172.21.15.121:0/1159755195 learned_addr learned my addr 172.21.15.121:0/1159755195 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T02:59:58.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.929+0000 7f9a57fff700 1 --2- 172.21.15.121:0/1159755195 >> v2:172.21.15.190:3300/0 conn(0x7f9a58153bb0 0x7f9a580a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:58.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.929+0000 7f9a577fe700 1 -- 172.21.15.121:0/1159755195 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257034390 0 0) 0x7f9a580cb5c0 con 0x7f9a58068170 2023-11-15T02:59:58.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.929+0000 7f9a577fe700 1 -- 172.21.15.121:0/1159755195 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a440033f0 con 0x7f9a58068170 2023-11-15T02:59:58.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.929+0000 7f9a57fff700 1 -- 172.21.15.121:0/1159755195 >> v1:172.21.15.190:6789/0 conn(0x7f9a580aa020 legacy=0x7f9a58067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:58.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.929+0000 7f9a57fff700 1 -- 172.21.15.121:0/1159755195 >> v1:172.21.15.121:6789/0 conn(0x7f9a58068170 legacy=0x7f9a58065d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:58.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.929+0000 7f9a57fff700 1 -- 172.21.15.121:0/1159755195 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a580caa20 con 0x7f9a58153bb0 2023-11-15T02:59:58.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.933+0000 7f9a57fff700 1 --2- 172.21.15.121:0/1159755195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a58153bb0 0x7f9a580a9b30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f9a48009e60 tx=0x7f9a48009780).ready entity=mon.1 client_cookie=2d26b4b6dbaa9383 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:58.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.933+0000 7f9a577fe700 1 -- 172.21.15.121:0/1159755195 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4800e070 con 0x7f9a58153bb0 2023-11-15T02:59:58.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.933+0000 7f9a577fe700 1 -- 172.21.15.121:0/1159755195 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4800bb60 con 0x7f9a58153bb0 2023-11-15T02:59:58.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.933+0000 7f9a577fe700 1 -- 172.21.15.121:0/1159755195 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a48012410 con 0x7f9a58153bb0 2023-11-15T02:59:58.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.933+0000 7f9a5e85b700 1 -- 172.21.15.121:0/1159755195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a58153bb0 msgr2=0x7f9a580a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:58.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.933+0000 7f9a5e85b700 1 --2- 172.21.15.121:0/1159755195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a58153bb0 0x7f9a580a9b30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f9a48009e60 tx=0x7f9a48009780).stop 2023-11-15T02:59:58.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.933+0000 7f9a5e85b700 1 -- 172.21.15.121:0/1159755195 shutdown_connections 2023-11-15T02:59:58.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.933+0000 7f9a5e85b700 1 --2- 172.21.15.121:0/1159755195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a58153bb0 0x7f9a580a9b30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:58.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.933+0000 7f9a5e85b700 1 -- 172.21.15.121:0/1159755195 >> 172.21.15.121:0/1159755195 conn(0x7f9a58058c50 msgr2=0x7f9a58059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:58.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.933+0000 7f9a5e85b700 1 -- 172.21.15.121:0/1159755195 shutdown_connections 2023-11-15T02:59:58.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.933+0000 7f9a5e85b700 1 -- 172.21.15.121:0/1159755195 wait complete. 2023-11-15T02:59:58.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.937+0000 7f9a5e85b700 1 Processor -- start 2023-11-15T02:59:58.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.937+0000 7f9a5e85b700 1 -- start start 2023-11-15T02:59:58.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a5e85b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a58068170 0x7f9a581657a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:58.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a5e85b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a580aa020 0x7f9a580c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:58.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a5e85b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a58153bb0 0x7f9a581529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:58.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a5e85b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a58166990 con 0x7f9a58153bb0 2023-11-15T02:59:58.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a5e85b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a58165d80 con 0x7f9a58068170 2023-11-15T02:59:58.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a5e85b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a58166060 con 0x7f9a580aa020 2023-11-15T02:59:58.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a5cdf8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a58153bb0 0x7f9a581529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:58.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a5cdf8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a58153bb0 0x7f9a581529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50068/0 (socket says 172.21.15.121:50068) 2023-11-15T02:59:58.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a5cdf8700 1 -- 172.21.15.121:0/3212853155 learned_addr learned my addr 172.21.15.121:0/3212853155 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:58.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a57fff700 1 --2- 172.21.15.121:0/3212853155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a58068170 0x7f9a581657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:58.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a4f7fe700 1 --2- 172.21.15.121:0/3212853155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a580aa020 0x7f9a580c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:58.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a5cdf8700 1 -- 172.21.15.121:0/3212853155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a580aa020 msgr2=0x7f9a580c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:58.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a5cdf8700 1 --2- 172.21.15.121:0/3212853155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a580aa020 0x7f9a580c3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:58.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a5cdf8700 1 -- 172.21.15.121:0/3212853155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a58068170 msgr2=0x7f9a581657a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:58.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a5cdf8700 1 --2- 172.21.15.121:0/3212853155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a58068170 0x7f9a581657a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:58.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a5cdf8700 1 -- 172.21.15.121:0/3212853155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a48009a20 con 0x7f9a58153bb0 2023-11-15T02:59:58.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a57fff700 1 --2- 172.21.15.121:0/3212853155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a58068170 0x7f9a581657a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:58.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a5cdf8700 1 --2- 172.21.15.121:0/3212853155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a58153bb0 0x7f9a581529c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f9a500010e0 tx=0x7f9a50007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:58.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a54ff9700 1 -- 172.21.15.121:0/3212853155 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a500072e0 con 0x7f9a58153bb0 2023-11-15T02:59:58.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a54ff9700 1 -- 172.21.15.121:0/3212853155 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a50007440 con 0x7f9a58153bb0 2023-11-15T02:59:58.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a5e85b700 1 -- 172.21.15.121:0/3212853155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a58166340 con 0x7f9a58153bb0 2023-11-15T02:59:58.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a54ff9700 1 -- 172.21.15.121:0/3212853155 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a500057a0 con 0x7f9a58153bb0 2023-11-15T02:59:58.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.941+0000 7f9a5e85b700 1 -- 172.21.15.121:0/3212853155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a580c3f20 con 0x7f9a58153bb0 2023-11-15T02:59:58.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.945+0000 7f9a5e85b700 1 -- 172.21.15.121:0/3212853155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a580cb590 con 0x7f9a58153bb0 2023-11-15T02:59:58.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.945+0000 7f9a54ff9700 1 -- 172.21.15.121:0/3212853155 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9a50005bd0 con 0x7f9a58153bb0 2023-11-15T02:59:58.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.949+0000 7f9a54ff9700 1 --2- 172.21.15.121:0/3212853155 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9a3805a540 0x7f9a3805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:58.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.949+0000 7f9a54ff9700 1 -- 172.21.15.121:0/3212853155 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f9a50073ff0 con 0x7f9a58153bb0 2023-11-15T02:59:58.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.949+0000 7f9a57fff700 1 --2- 172.21.15.121:0/3212853155 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9a3805a540 0x7f9a3805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:58.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.949+0000 7f9a57fff700 1 --2- 172.21.15.121:0/3212853155 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9a3805a540 0x7f9a3805ca00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f9a480094d0 tx=0x7f9a48009780).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:58.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:58.953+0000 7f9a54ff9700 1 -- 172.21.15.121:0/3212853155 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9a580cb590 con 0x7f9a58153bb0 2023-11-15T02:59:59.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.153+0000 7f9a5e85b700 1 -- 172.21.15.121:0/3212853155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a580cb590 con 0x7f9a58153bb0 2023-11-15T02:59:59.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.153+0000 7f9a54ff9700 1 -- 172.21.15.121:0/3212853155 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4503 (secure 0 0 0) 0x7f9a580cb590 con 0x7f9a58153bb0 2023-11-15T02:59:59.157 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T02:59:59.157 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":52,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3462858472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3462858472},{"type":"v1","addr":"172.21.15.190:6833","nonce":3462858472}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4042390087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4042390087},{"type":"v1","addr":"172.21.15.190:6835","nonce":4042390087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5907,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2239114529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2239114529},{"type":"v1","addr":"172.21.15.121:6837","nonce":2239114529}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:59:34.667864+0000","modified":"2023-11-15T02:59:58.138367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":5,"addr":"172.21.15.121:6836/3429169582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3429169582},{"type":"v1","addr":"172.21.15.121:6836","nonce":3429169582}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-15T02:59:59.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.157+0000 7f9a5e85b700 1 -- 172.21.15.121:0/3212853155 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9a3805a540 msgr2=0x7f9a3805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:59.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.157+0000 7f9a5e85b700 1 --2- 172.21.15.121:0/3212853155 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9a3805a540 0x7f9a3805ca00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f9a480094d0 tx=0x7f9a48009780).stop 2023-11-15T02:59:59.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.157+0000 7f9a5e85b700 1 -- 172.21.15.121:0/3212853155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a58153bb0 msgr2=0x7f9a581529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:59.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.157+0000 7f9a5e85b700 1 --2- 172.21.15.121:0/3212853155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a58153bb0 0x7f9a581529c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f9a500010e0 tx=0x7f9a50007c50).stop 2023-11-15T02:59:59.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.157+0000 7f9a5e85b700 1 -- 172.21.15.121:0/3212853155 shutdown_connections 2023-11-15T02:59:59.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.157+0000 7f9a5e85b700 1 --2- 172.21.15.121:0/3212853155 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9a3805a540 0x7f9a3805ca00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.157+0000 7f9a5e85b700 1 --2- 172.21.15.121:0/3212853155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a58153bb0 0x7f9a581529c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.157+0000 7f9a5e85b700 1 --2- 172.21.15.121:0/3212853155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a580aa020 0x7f9a580c3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.157+0000 7f9a5e85b700 1 --2- 172.21.15.121:0/3212853155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a58068170 0x7f9a581657a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.157+0000 7f9a5e85b700 1 -- 172.21.15.121:0/3212853155 >> 172.21.15.121:0/3212853155 conn(0x7f9a58058c50 msgr2=0x7f9a581581e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:59.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.157+0000 7f9a5e85b700 1 -- 172.21.15.121:0/3212853155 shutdown_connections 2023-11-15T02:59:59.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.157+0000 7f9a5e85b700 1 -- 172.21.15.121:0/3212853155 wait complete. 2023-11-15T02:59:59.163 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 52 2023-11-15T02:59:59.584 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T02:59:34.667864+0000', 'modified': '2023-11-15T02:59:58.138367+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 62, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6144}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6144': {'gid': 6144, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.121:6836/3429169582', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6835', 'nonce': 3429169582}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 3429169582}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T02:59:59.584 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T02:59:59.585 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-15T02:59:59.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.745+0000 7f6e69239700 1 Processor -- start 2023-11-15T02:59:59.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.745+0000 7f6e69239700 1 -- start start 2023-11-15T02:59:59.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.745+0000 7f6e69239700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6e64157d90 0x7f6e64158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.745+0000 7f6e69239700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6e640669b0 0x7f6e640c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.745+0000 7f6e69239700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e640a8750 0x7f6e640a8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.745+0000 7f6e69239700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e640c1fb0 con 0x7f6e64157d90 2023-11-15T02:59:59.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.745+0000 7f6e69239700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6415bbd0 con 0x7f6e640669b0 2023-11-15T02:59:59.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.745+0000 7f6e69239700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e640a90d0 con 0x7f6e640a8750 2023-11-15T02:59:59.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e63fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6e64157d90 0x7f6e64158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e63fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6e64157d90 0x7f6e64158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50070/0 (socket says 172.21.15.121:50070) 2023-11-15T02:59:59.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e63fff700 1 -- 172.21.15.121:0/85390870 learned_addr learned my addr 172.21.15.121:0/85390870 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:59.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e5b7fe700 1 --2- 172.21.15.121:0/85390870 >> v2:172.21.15.190:3300/0 conn(0x7f6e640669b0 0x7f6e640c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e68a38700 1 --2- 172.21.15.121:0/85390870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e640a8750 0x7f6e640a8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e63fff700 1 -- 172.21.15.121:0/85390870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e640a8750 msgr2=0x7f6e640a8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:59.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e63fff700 1 --2- 172.21.15.121:0/85390870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e640a8750 0x7f6e640a8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e63fff700 1 -- 172.21.15.121:0/85390870 >> v2:172.21.15.190:3300/0 conn(0x7f6e640669b0 msgr2=0x7f6e640c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:59.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e63fff700 1 --2- 172.21.15.121:0/85390870 >> v2:172.21.15.190:3300/0 conn(0x7f6e640669b0 0x7f6e640c1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e63fff700 1 -- 172.21.15.121:0/85390870 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e640bb910 con 0x7f6e64157d90 2023-11-15T02:59:59.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e5b7fe700 1 --2- 172.21.15.121:0/85390870 >> v2:172.21.15.190:3300/0 conn(0x7f6e640669b0 0x7f6e640c1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:59.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e63fff700 1 --2- 172.21.15.121:0/85390870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e64157d90 0x7f6e64158170 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f6e64059700 tx=0x7f6e540090f0).ready entity=mon.0 client_cookie=71c9a8d815b31366 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:59.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e637fe700 1 -- 172.21.15.121:0/85390870 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5400b600 con 0x7f6e64157d90 2023-11-15T02:59:59.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e637fe700 1 -- 172.21.15.121:0/85390870 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5400b760 con 0x7f6e64157d90 2023-11-15T02:59:59.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e637fe700 1 -- 172.21.15.121:0/85390870 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5400bae0 con 0x7f6e64157d90 2023-11-15T02:59:59.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e69239700 1 -- 172.21.15.121:0/85390870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e64157d90 msgr2=0x7f6e64158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:59.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e69239700 1 --2- 172.21.15.121:0/85390870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e64157d90 0x7f6e64158170 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f6e64059700 tx=0x7f6e540090f0).stop 2023-11-15T02:59:59.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e69239700 1 -- 172.21.15.121:0/85390870 shutdown_connections 2023-11-15T02:59:59.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e69239700 1 --2- 172.21.15.121:0/85390870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e640a8750 0x7f6e640a8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e69239700 1 --2- 172.21.15.121:0/85390870 >> v2:172.21.15.190:3300/0 conn(0x7f6e640669b0 0x7f6e640c1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e69239700 1 --2- 172.21.15.121:0/85390870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e64157d90 0x7f6e64158170 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e69239700 1 -- 172.21.15.121:0/85390870 >> 172.21.15.121:0/85390870 conn(0x7f6e64067a60 msgr2=0x7f6e64067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:59.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e69239700 1 -- 172.21.15.121:0/85390870 shutdown_connections 2023-11-15T02:59:59.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.749+0000 7f6e69239700 1 -- 172.21.15.121:0/85390870 wait complete. 2023-11-15T02:59:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.753+0000 7f6e69239700 1 Processor -- start 2023-11-15T02:59:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.753+0000 7f6e69239700 1 -- start start 2023-11-15T02:59:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.753+0000 7f6e69239700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e640669b0 0x7f6e640a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.753+0000 7f6e69239700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e640a8750 0x7f6e640c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.753+0000 7f6e69239700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e64157d90 0x7f6e64148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.753+0000 7f6e69239700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e640aa990 con 0x7f6e640a8750 2023-11-15T02:59:59.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.753+0000 7f6e69239700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e640a9e10 con 0x7f6e64157d90 2023-11-15T02:59:59.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.753+0000 7f6e69239700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e640aa0f0 con 0x7f6e640669b0 2023-11-15T02:59:59.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.757+0000 7f6e5b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e640a8750 0x7f6e640c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.757+0000 7f6e5b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e640a8750 0x7f6e640c30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50078/0 (socket says 172.21.15.121:50078) 2023-11-15T02:59:59.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.757+0000 7f6e5b7fe700 1 -- 172.21.15.121:0/178797086 learned_addr learned my addr 172.21.15.121:0/178797086 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:59.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.757+0000 7f6e63fff700 1 --2- 172.21.15.121:0/178797086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e640669b0 0x7f6e640a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.757+0000 7f6e68a38700 1 --2- 172.21.15.121:0/178797086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e64157d90 0x7f6e64148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.757+0000 7f6e5b7fe700 1 -- 172.21.15.121:0/178797086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e640669b0 msgr2=0x7f6e640a95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:59.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.757+0000 7f6e5b7fe700 1 --2- 172.21.15.121:0/178797086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e640669b0 0x7f6e640a95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.757+0000 7f6e5b7fe700 1 -- 172.21.15.121:0/178797086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e64157d90 msgr2=0x7f6e64148380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:59.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.757+0000 7f6e5b7fe700 1 --2- 172.21.15.121:0/178797086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e64157d90 0x7f6e64148380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.757+0000 7f6e5b7fe700 1 -- 172.21.15.121:0/178797086 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e54009370 con 0x7f6e640a8750 2023-11-15T02:59:59.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.757+0000 7f6e68a38700 1 --2- 172.21.15.121:0/178797086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e64157d90 0x7f6e64148380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:59.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.757+0000 7f6e5b7fe700 1 --2- 172.21.15.121:0/178797086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e640a8750 0x7f6e640c30c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f6e4c00d890 tx=0x7f6e4c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:59.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.757+0000 7f6e60ff9700 1 -- 172.21.15.121:0/178797086 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c017070 con 0x7f6e640a8750 2023-11-15T02:59:59.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.757+0000 7f6e60ff9700 1 -- 172.21.15.121:0/178797086 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c005880 con 0x7f6e640a8750 2023-11-15T02:59:59.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.757+0000 7f6e60ff9700 1 -- 172.21.15.121:0/178797086 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c005c00 con 0x7f6e640a8750 2023-11-15T02:59:59.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.757+0000 7f6e69239700 1 -- 172.21.15.121:0/178797086 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e640aa3d0 con 0x7f6e640a8750 2023-11-15T02:59:59.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.757+0000 7f6e69239700 1 -- 172.21.15.121:0/178797086 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e640a9ab0 con 0x7f6e640a8750 2023-11-15T02:59:59.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.761+0000 7f6e60ff9700 1 -- 172.21.15.121:0/178797086 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6e4c006020 con 0x7f6e640a8750 2023-11-15T02:59:59.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.761+0000 7f6e60ff9700 1 --2- 172.21.15.121:0/178797086 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e4405a510 0x7f6e4405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.761+0000 7f6e60ff9700 1 -- 172.21.15.121:0/178797086 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f6e4c074910 con 0x7f6e640a8750 2023-11-15T02:59:59.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.761+0000 7f6e74832740 1 Processor -- start 2023-11-15T02:59:59.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.761+0000 7f6e63fff700 1 --2- 172.21.15.121:0/178797086 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e4405a510 0x7f6e4405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.761+0000 7f6e74832740 1 -- start start 2023-11-15T02:59:59.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.761+0000 7f6e63fff700 1 --2- 172.21.15.121:0/178797086 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e4405a510 0x7f6e4405c9d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f6e54009130 tx=0x7f6e54017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:59.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e74832740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888ad0 0x28af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e74832740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2888eb0 0x28d5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e74832740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x28d5630 0x28d5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e74832740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2809410 con 0x2888ad0 2023-11-15T02:59:59.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e74832740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x2809550 con 0x2888eb0 2023-11-15T02:59:59.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e74832740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2809a00 con 0x28d5630 2023-11-15T02:59:59.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e63fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888ad0 0x28af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e63fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888ad0 0x28af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50084/0 (socket says 172.21.15.121:50084) 2023-11-15T02:59:59.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e63fff700 1 -- 172.21.15.121:0/3133505083 learned_addr learned my addr 172.21.15.121:0/3133505083 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:59.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e5b7fe700 1 --2- 172.21.15.121:0/3133505083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2888eb0 0x28d5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e68a38700 1 --2- 172.21.15.121:0/3133505083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x28d5630 0x28d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e63fff700 1 -- 172.21.15.121:0/3133505083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x28d5630 msgr2=0x28d5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:59.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e63fff700 1 --2- 172.21.15.121:0/3133505083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x28d5630 0x28d5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e63fff700 1 -- 172.21.15.121:0/3133505083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2888eb0 msgr2=0x28d5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:59.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e63fff700 1 --2- 172.21.15.121:0/3133505083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2888eb0 0x28d5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e63fff700 1 -- 172.21.15.121:0/3133505083 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2802730 con 0x2888ad0 2023-11-15T02:59:59.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e63fff700 1 --2- 172.21.15.121:0/3133505083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888ad0 0x28af090 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f6e54006fd0 tx=0x7f6e5401d000).ready entity=mon.0 client_cookie=135f30944f605df3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:59.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e597fa700 1 -- 172.21.15.121:0/3133505083 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5401c670 con 0x2888ad0 2023-11-15T02:59:59.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e597fa700 1 -- 172.21.15.121:0/3133505083 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5401c7d0 con 0x2888ad0 2023-11-15T02:59:59.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e597fa700 1 -- 172.21.15.121:0/3133505083 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5401cb50 con 0x2888ad0 2023-11-15T02:59:59.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e74832740 1 -- 172.21.15.121:0/3133505083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888ad0 msgr2=0x28af090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:59.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e74832740 1 --2- 172.21.15.121:0/3133505083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888ad0 0x28af090 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f6e54006fd0 tx=0x7f6e5401d000).stop 2023-11-15T02:59:59.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e74832740 1 -- 172.21.15.121:0/3133505083 shutdown_connections 2023-11-15T02:59:59.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e74832740 1 --2- 172.21.15.121:0/3133505083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x28d5630 0x28d5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e74832740 1 --2- 172.21.15.121:0/3133505083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2888eb0 0x28d5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e74832740 1 --2- 172.21.15.121:0/3133505083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888ad0 0x28af090 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e74832740 1 -- 172.21.15.121:0/3133505083 >> 172.21.15.121:0/3133505083 conn(0x2667550 msgr2=0x28877e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:59.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e74832740 1 -- 172.21.15.121:0/3133505083 shutdown_connections 2023-11-15T02:59:59.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.765+0000 7f6e74832740 1 -- 172.21.15.121:0/3133505083 wait complete. 2023-11-15T02:59:59.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e74832740 1 -- start start 2023-11-15T02:59:59.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e74832740 1 Processor -- start 2023-11-15T02:59:59.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e74832740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888ad0 0x28d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e74832740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2888eb0 0x28b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e74832740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x28d5630 0x28ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e74832740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x27d4650 con 0x2888ad0 2023-11-15T02:59:59.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e74832740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x28d2b40 con 0x28d5630 2023-11-15T02:59:59.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e74832740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x28b4320 con 0x2888eb0 2023-11-15T02:59:59.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e74832740 10 client.0 ll_register_callbacks cb 0x2841020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:59:59.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e63fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888ad0 0x28d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e63fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888ad0 0x28d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50090/0 (socket says 172.21.15.121:50090) 2023-11-15T02:59:59.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e63fff700 1 -- 172.21.15.121:0/1659415317 learned_addr learned my addr 172.21.15.121:0/1659415317 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:59.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e5b7fe700 1 --2- 172.21.15.121:0/1659415317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2888eb0 0x28b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e68a38700 1 --2- 172.21.15.121:0/1659415317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x28d5630 0x28ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e63fff700 1 -- 172.21.15.121:0/1659415317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2888eb0 msgr2=0x28b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:59.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e63fff700 1 --2- 172.21.15.121:0/1659415317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2888eb0 0x28b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e63fff700 1 -- 172.21.15.121:0/1659415317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x28d5630 msgr2=0x28ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:59.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e63fff700 1 --2- 172.21.15.121:0/1659415317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x28d5630 0x28ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e63fff700 1 -- 172.21.15.121:0/1659415317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e640bb910 con 0x2888ad0 2023-11-15T02:59:59.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e68a38700 1 --2- 172.21.15.121:0/1659415317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x28d5630 0x28ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T02:59:59.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e63fff700 1 --2- 172.21.15.121:0/1659415317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888ad0 0x28d0f50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f6e54006bc0 tx=0x7f6e54027000).ready entity=mon.0 client_cookie=61970f47027d6bdf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:59.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e427fc700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:59:59.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e427fc700 1 -- 172.21.15.121:0/1659415317 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e54027a80 con 0x2888ad0 2023-11-15T02:59:59.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e427fc700 1 -- 172.21.15.121:0/1659415317 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f6e54027be0 con 0x2888ad0 2023-11-15T02:59:59.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.769+0000 7f6e74832740 1 -- 172.21.15.121:0/1659415317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x27ff1a0 con 0x2888ad0 2023-11-15T02:59:59.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e427fc700 1 -- 172.21.15.121:0/1659415317 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e54039470 con 0x2888ad0 2023-11-15T02:59:59.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 10 client.6192 fetch_fsmap learned FSMap version 52 2023-11-15T02:59:59.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 1 -- 172.21.15.121:0/1659415317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x27fd960 con 0x2888ad0 2023-11-15T02:59:59.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e427fc700 1 -- 172.21.15.121:0/1659415317 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x27ff1a0 con 0x2888ad0 2023-11-15T02:59:59.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e427fc700 1 -- 172.21.15.121:0/1659415317 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 52) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6e54024030 con 0x2888ad0 2023-11-15T02:59:59.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 10 client.6192 fetch_fsmap finished waiting for FSMap version 52 2023-11-15T02:59:59.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 10 client.6192 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T02:59:59.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 20 client.6192 populate_metadata read hostname 'smithi121' 2023-11-15T02:59:59.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 1 --2- 172.21.15.121:0/1659415317 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x28b5350 0x28b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 4 client.6192 mds_command: new command op to 6144 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T02:59:59.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 1 -- 172.21.15.121:0/1659415317 --> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x27f56f0 con 0x28b5350 2023-11-15T02:59:59.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e5b7fe700 1 --2- 172.21.15.121:0/1659415317 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x28b5350 0x28b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e5b7fe700 1 --2- 172.21.15.121:0/1659415317 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x28b5350 0x28b13c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=9752475ad84f5d1 server_cookie=aa87bb5a401df220 in_seq=0 out_seq=0 2023-11-15T02:59:59.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e427fc700 10 client.6192 ms_handle_connect on v2:172.21.15.121:6835/3429169582 2023-11-15T02:59:59.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e427fc700 1 -- 172.21.15.121:0/1659415317 <== mds.0 v2:172.21.15.121:6835/3429169582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x27f56f0 con 0x28b5350 2023-11-15T02:59:59.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e427fc700 10 client.6192 handle_command_reply: tid=0 2023-11-15T02:59:59.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 1 client.6192 shutdown 2023-11-15T02:59:59.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 1 -- 172.21.15.121:0/1659415317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888ad0 msgr2=0x28d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:59.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 1 --2- 172.21.15.121:0/1659415317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888ad0 0x28d0f50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f6e54006bc0 tx=0x7f6e54027000).stop 2023-11-15T02:59:59.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 1 -- 172.21.15.121:0/1659415317 shutdown_connections 2023-11-15T02:59:59.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 1 --2- 172.21.15.121:0/1659415317 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x28b5350 0x28b13c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 1 --2- 172.21.15.121:0/1659415317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x28d5630 0x28ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 1 --2- 172.21.15.121:0/1659415317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2888eb0 0x28b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 1 --2- 172.21.15.121:0/1659415317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888ad0 0x28d0f50 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 1 -- 172.21.15.121:0/1659415317 >> 172.21.15.121:0/1659415317 conn(0x2667550 msgr2=0x2883d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:59.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e427fc700 0 client.6192 ms_handle_reset on v2:172.21.15.121:6835/3429169582 2023-11-15T02:59:59.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 1 -- 172.21.15.121:0/1659415317 shutdown_connections 2023-11-15T02:59:59.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 1 -- 172.21.15.121:0/1659415317 wait complete. 2023-11-15T02:59:59.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.773+0000 7f6e74832740 20 client.6192 trim_cache size 0 max 16384 2023-11-15T02:59:59.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.793+0000 7f6e74832740 1 Processor -- start 2023-11-15T02:59:59.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.793+0000 7f6e74832740 1 -- start start 2023-11-15T02:59:59.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e74832740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2888ad0 0x29273e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e74832740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888eb0 0x28b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e74832740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x28d5630 0x28af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e74832740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x28ae9f0 con 0x2888eb0 2023-11-15T02:59:59.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e74832740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x28b2840 con 0x2888ad0 2023-11-15T02:59:59.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e74832740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x27d8c50 con 0x28d5630 2023-11-15T02:59:59.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e5b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888eb0 0x28b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e5b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888eb0 0x28b58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50104/0 (socket says 172.21.15.121:50104) 2023-11-15T02:59:59.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e5b7fe700 1 -- 172.21.15.121:0/3929587297 learned_addr learned my addr 172.21.15.121:0/3929587297 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:59.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e68a38700 1 --2- 172.21.15.121:0/3929587297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x28d5630 0x28af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e63fff700 1 --2- 172.21.15.121:0/3929587297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2888ad0 0x29273e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e5b7fe700 1 -- 172.21.15.121:0/3929587297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x28d5630 msgr2=0x28af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:59.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e5b7fe700 1 --2- 172.21.15.121:0/3929587297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x28d5630 0x28af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e5b7fe700 1 -- 172.21.15.121:0/3929587297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2888ad0 msgr2=0x29273e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:59.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e5b7fe700 1 --2- 172.21.15.121:0/3929587297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2888ad0 0x29273e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e5b7fe700 1 -- 172.21.15.121:0/3929587297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x27fd6b0 con 0x2888eb0 2023-11-15T02:59:59.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e68a38700 1 --2- 172.21.15.121:0/3929587297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x28d5630 0x28af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:59.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e63fff700 1 --2- 172.21.15.121:0/3929587297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2888ad0 0x29273e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T02:59:59.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e5b7fe700 1 --2- 172.21.15.121:0/3929587297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888eb0 0x28b58c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f6e4c04b9a0 tx=0x7f6e4c04d040).ready entity=mon.0 client_cookie=a96719ccba03d591 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:59.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e59ffb700 1 -- 172.21.15.121:0/3929587297 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c050040 con 0x2888eb0 2023-11-15T02:59:59.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e59ffb700 1 -- 172.21.15.121:0/3929587297 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c051040 con 0x2888eb0 2023-11-15T02:59:59.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.797+0000 7f6e59ffb700 1 -- 172.21.15.121:0/3929587297 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c04a110 con 0x2888eb0 2023-11-15T02:59:59.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.801+0000 7f6e74832740 1 -- 172.21.15.121:0/3929587297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888eb0 msgr2=0x28b58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:59.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.801+0000 7f6e74832740 1 --2- 172.21.15.121:0/3929587297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888eb0 0x28b58c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f6e4c04b9a0 tx=0x7f6e4c04d040).stop 2023-11-15T02:59:59.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.801+0000 7f6e74832740 1 -- 172.21.15.121:0/3929587297 shutdown_connections 2023-11-15T02:59:59.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.801+0000 7f6e74832740 1 --2- 172.21.15.121:0/3929587297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x28d5630 0x28af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.801+0000 7f6e74832740 1 --2- 172.21.15.121:0/3929587297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888eb0 0x28b58c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.801+0000 7f6e74832740 1 --2- 172.21.15.121:0/3929587297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2888ad0 0x29273e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.801+0000 7f6e74832740 1 -- 172.21.15.121:0/3929587297 >> 172.21.15.121:0/3929587297 conn(0x2667550 msgr2=0x28e3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:59.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.801+0000 7f6e74832740 1 -- 172.21.15.121:0/3929587297 shutdown_connections 2023-11-15T02:59:59.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.801+0000 7f6e74832740 1 -- 172.21.15.121:0/3929587297 wait complete. 2023-11-15T02:59:59.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.801+0000 7f6e74832740 1 -- start start 2023-11-15T02:59:59.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.801+0000 7f6e74832740 1 Processor -- start 2023-11-15T02:59:59.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e74832740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2888ad0 0x2884960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e74832740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888eb0 0x2926310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e74832740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x28d5630 0x28cc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e74832740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x28ccf40 con 0x2888eb0 2023-11-15T02:59:59.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e74832740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x28b1bb0 con 0x28d5630 2023-11-15T02:59:59.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e74832740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x28844d0 con 0x2888ad0 2023-11-15T02:59:59.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e74832740 10 client.0 ll_register_callbacks cb 0x7f6e5401c670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T02:59:59.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e5b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888eb0 0x2926310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e5b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888eb0 0x2926310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50112/0 (socket says 172.21.15.121:50112) 2023-11-15T02:59:59.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e5b7fe700 1 -- 172.21.15.121:0/3336136407 learned_addr learned my addr 172.21.15.121:0/3336136407 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T02:59:59.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e68a38700 1 --2- 172.21.15.121:0/3336136407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x28d5630 0x28cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e63fff700 1 --2- 172.21.15.121:0/3336136407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2888ad0 0x2884960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e5b7fe700 1 -- 172.21.15.121:0/3336136407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2888ad0 msgr2=0x2884960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:59.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e5b7fe700 1 --2- 172.21.15.121:0/3336136407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2888ad0 0x2884960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e5b7fe700 1 -- 172.21.15.121:0/3336136407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x28d5630 msgr2=0x28cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T02:59:59.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e5b7fe700 1 --2- 172.21.15.121:0/3336136407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x28d5630 0x28cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e5b7fe700 1 -- 172.21.15.121:0/3336136407 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e640a9ab0 con 0x2888eb0 2023-11-15T02:59:59.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e5b7fe700 1 --2- 172.21.15.121:0/3336136407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888eb0 0x2926310 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f6e4c04b890 tx=0x7f6e4c054000).ready entity=mon.0 client_cookie=f28257e3ccaadecc server_cookie=0 in_seq=0 out_seq=0 2023-11-15T02:59:59.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e427fc700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T02:59:59.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e427fc700 1 -- 172.21.15.121:0/3336136407 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c050040 con 0x2888eb0 2023-11-15T02:59:59.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e427fc700 1 -- 172.21.15.121:0/3336136407 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f6e4c05bcc0 con 0x2888eb0 2023-11-15T02:59:59.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e74832740 1 -- 172.21.15.121:0/3336136407 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e5401c7d0 con 0x2888eb0 2023-11-15T02:59:59.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e427fc700 1 -- 172.21.15.121:0/3336136407 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e4c0513c0 con 0x2888eb0 2023-11-15T02:59:59.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e74832740 10 client.6198 fetch_fsmap learned FSMap version 52 2023-11-15T02:59:59.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e74832740 1 -- 172.21.15.121:0/3336136407 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x2705850 con 0x2888eb0 2023-11-15T02:59:59.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e427fc700 1 -- 172.21.15.121:0/3336136407 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5401c7d0 con 0x2888eb0 2023-11-15T02:59:59.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e427fc700 1 -- 172.21.15.121:0/3336136407 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 52) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6e4c011340 con 0x2888eb0 2023-11-15T02:59:59.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e74832740 10 client.6198 fetch_fsmap finished waiting for FSMap version 52 2023-11-15T02:59:59.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e74832740 10 client.6198 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T02:59:59.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e74832740 20 client.6198 populate_metadata read hostname 'smithi121' 2023-11-15T02:59:59.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e74832740 1 --2- 172.21.15.121:0/3336136407 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x28d0990 0x28d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T02:59:59.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e74832740 4 client.6198 mds_command: new command op to 6144 tid=0[{"prefix": "status"}] 2023-11-15T02:59:59.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e74832740 1 -- 172.21.15.121:0/3336136407 --> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] -- command(tid 0: {"prefix": "status"}) v1 -- 0x26be4e0 con 0x28d0990 2023-11-15T02:59:59.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e63fff700 1 --2- 172.21.15.121:0/3336136407 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x28d0990 0x28d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T02:59:59.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e63fff700 1 --2- 172.21.15.121:0/3336136407 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x28d0990 0x28d0d70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=3e992c83fa0f2daa server_cookie=affd7b85957dc6cd in_seq=0 out_seq=0 2023-11-15T02:59:59.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e427fc700 10 client.6198 ms_handle_connect on v2:172.21.15.121:6835/3429169582 2023-11-15T02:59:59.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e427fc700 1 -- 172.21.15.121:0/3336136407 <== mds.0 v2:172.21.15.121:6835/3429169582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x26be4e0 con 0x28d0990 2023-11-15T02:59:59.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.805+0000 7f6e427fc700 10 client.6198 handle_command_reply: tid=0 2023-11-15T02:59:59.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e74832740 1 client.6198 shutdown 2023-11-15T02:59:59.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e74832740 1 -- 172.21.15.121:0/3336136407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888eb0 msgr2=0x2926310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:59.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e74832740 1 --2- 172.21.15.121:0/3336136407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888eb0 0x2926310 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f6e4c04b890 tx=0x7f6e4c054000).stop 2023-11-15T02:59:59.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e74832740 1 -- 172.21.15.121:0/3336136407 shutdown_connections 2023-11-15T02:59:59.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e74832740 1 --2- 172.21.15.121:0/3336136407 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x28d0990 0x28d0d70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e74832740 1 --2- 172.21.15.121:0/3336136407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x28d5630 0x28cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e74832740 1 --2- 172.21.15.121:0/3336136407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2888eb0 0x2926310 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e74832740 1 --2- 172.21.15.121:0/3336136407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2888ad0 0x2884960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e427fc700 0 client.6198 ms_handle_reset on v2:172.21.15.121:6835/3429169582 2023-11-15T02:59:59.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e74832740 1 -- 172.21.15.121:0/3336136407 >> 172.21.15.121:0/3336136407 conn(0x2667550 msgr2=0x28877e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:59.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e74832740 1 -- 172.21.15.121:0/3336136407 shutdown_connections 2023-11-15T02:59:59.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e74832740 1 -- 172.21.15.121:0/3336136407 wait complete. 2023-11-15T02:59:59.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e74832740 20 client.6198 trim_cache size 0 max 16384 2023-11-15T02:59:59.822 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T02:59:59.822 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T02:59:59.822 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T02:59:59.822 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 6144, 2023-11-15T02:59:59.822 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T02:59:59.823 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T02:59:59.823 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T02:59:59.823 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 4.6959341080000003, 2023-11-15T02:59:59.823 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 52, 2023-11-15T02:59:59.823 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 62, 2023-11-15T02:59:59.824 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 62, 2023-11-15T02:59:59.824 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 10.683250361000001 2023-11-15T02:59:59.824 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T02:59:59.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e69239700 1 -- 172.21.15.121:0/178797086 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e4405a510 msgr2=0x7f6e4405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:59.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e69239700 1 --2- 172.21.15.121:0/178797086 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e4405a510 0x7f6e4405c9d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f6e54009130 tx=0x7f6e54017000).stop 2023-11-15T02:59:59.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e69239700 1 -- 172.21.15.121:0/178797086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e640a8750 msgr2=0x7f6e640c30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T02:59:59.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e69239700 1 --2- 172.21.15.121:0/178797086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e640a8750 0x7f6e640c30c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f6e4c00d890 tx=0x7f6e4c008040).stop 2023-11-15T02:59:59.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e69239700 1 -- 172.21.15.121:0/178797086 shutdown_connections 2023-11-15T02:59:59.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e69239700 1 --2- 172.21.15.121:0/178797086 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e4405a510 0x7f6e4405c9d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e69239700 1 --2- 172.21.15.121:0/178797086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e64157d90 0x7f6e64148380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e69239700 1 --2- 172.21.15.121:0/178797086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e640a8750 0x7f6e640c30c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e69239700 1 --2- 172.21.15.121:0/178797086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e640669b0 0x7f6e640a95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T02:59:59.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e69239700 1 -- 172.21.15.121:0/178797086 >> 172.21.15.121:0/178797086 conn(0x7f6e64067a60 msgr2=0x7f6e64156ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T02:59:59.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e69239700 1 -- 172.21.15.121:0/178797086 shutdown_connections 2023-11-15T02:59:59.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T02:59:59.809+0000 7f6e69239700 1 -- 172.21.15.121:0/178797086 wait complete. 2023-11-15T02:59:59.832 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:00:00.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.001+0000 7f8fca769700 1 Processor -- start 2023-11-15T03:00:00.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.001+0000 7f8fca769700 1 -- start start 2023-11-15T03:00:00.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.005+0000 7f8fca769700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8fc40ae1f0 0x7f8fc4157840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:00.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.005+0000 7f8fca769700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8fc4157d30 0x7f8fc40c3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:00.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.005+0000 7f8fca769700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc40c34d0 con 0x7f8fc40ade10 2023-11-15T03:00:00.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.005+0000 7f8fca769700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc40c7530 con 0x7f8fc40ae1f0 2023-11-15T03:00:00.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.005+0000 7f8fca769700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc41581c0 con 0x7f8fc4157d30 2023-11-15T03:00:00.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.005+0000 7f8fc3fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f8fc40ade10 0x7f8fc40a8d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45590/0 (socket says 172.21.15.121:45590) 2023-11-15T03:00:00.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.005+0000 7f8fc3fff700 1 -- 172.21.15.121:0/1956845463 learned_addr learned my addr 172.21.15.121:0/1956845463 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:00:00.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.005+0000 7f8fbb7fe700 1 --2- 172.21.15.121:0/1956845463 >> v2:172.21.15.190:3300/0 conn(0x7f8fc40ae1f0 0x7f8fc4157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:00.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.005+0000 7f8fc8d06700 1 --2- 172.21.15.121:0/1956845463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8fc4157d30 0x7f8fc40c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:00.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.009+0000 7f8fbb7fe700 1 -- 172.21.15.121:0/1956845463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8fc4157d30 msgr2=0x7f8fc40c3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:00.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.009+0000 7f8fbb7fe700 1 --2- 172.21.15.121:0/1956845463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8fc4157d30 0x7f8fc40c3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:00.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.009+0000 7f8fc37fe700 1 -- 172.21.15.121:0/1956845463 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2767085517 0 0) 0x7f8fc40c34d0 con 0x7f8fc40ade10 2023-11-15T03:00:00.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.009+0000 7f8fbb7fe700 1 -- 172.21.15.121:0/1956845463 >> v1:172.21.15.121:6789/0 conn(0x7f8fc40ade10 legacy=0x7f8fc40a8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:00.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.009+0000 7f8fbb7fe700 1 -- 172.21.15.121:0/1956845463 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc40a9460 con 0x7f8fc40ae1f0 2023-11-15T03:00:00.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.009+0000 7f8fbb7fe700 1 --2- 172.21.15.121:0/1956845463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc40ae1f0 0x7f8fc4157840 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f8fac009e60 tx=0x7f8fac009780).ready entity=mon.1 client_cookie=a97e5f9029710aac server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:00.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.009+0000 7f8fc37fe700 1 -- 172.21.15.121:0/1956845463 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac00e070 con 0x7f8fc40ae1f0 2023-11-15T03:00:00.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.009+0000 7f8fc37fe700 1 -- 172.21.15.121:0/1956845463 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fac00bb60 con 0x7f8fc40ae1f0 2023-11-15T03:00:00.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.009+0000 7f8fc37fe700 1 -- 172.21.15.121:0/1956845463 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac012410 con 0x7f8fc40ae1f0 2023-11-15T03:00:00.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.009+0000 7f8fca769700 1 -- 172.21.15.121:0/1956845463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc40ae1f0 msgr2=0x7f8fc4157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:00.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.009+0000 7f8fca769700 1 --2- 172.21.15.121:0/1956845463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc40ae1f0 0x7f8fc4157840 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f8fac009e60 tx=0x7f8fac009780).stop 2023-11-15T03:00:00.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.009+0000 7f8fca769700 1 -- 172.21.15.121:0/1956845463 shutdown_connections 2023-11-15T03:00:00.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.009+0000 7f8fca769700 1 --2- 172.21.15.121:0/1956845463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8fc4157d30 0x7f8fc40c3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:00.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.009+0000 7f8fca769700 1 --2- 172.21.15.121:0/1956845463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc40ae1f0 0x7f8fc4157840 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:00.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.009+0000 7f8fca769700 1 -- 172.21.15.121:0/1956845463 >> 172.21.15.121:0/1956845463 conn(0x7f8fc40c89c0 msgr2=0x7f8fc40c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:00.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.009+0000 7f8fca769700 1 -- 172.21.15.121:0/1956845463 shutdown_connections 2023-11-15T03:00:00.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.009+0000 7f8fca769700 1 -- 172.21.15.121:0/1956845463 wait complete. 2023-11-15T03:00:00.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.013+0000 7f8fca769700 1 Processor -- start 2023-11-15T03:00:00.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.013+0000 7f8fca769700 1 -- start start 2023-11-15T03:00:00.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.013+0000 7f8fca769700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc40ade10 0x7f8fc4147c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:00.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.013+0000 7f8fca769700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc40ae1f0 0x7f8fc4155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:00.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.013+0000 7f8fca769700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8fc4157d30 0x7f8fc41563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:00.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.013+0000 7f8fca769700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc41568c0 con 0x7f8fc40ae1f0 2023-11-15T03:00:00.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.013+0000 7f8fca769700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc4065d60 con 0x7f8fc40ade10 2023-11-15T03:00:00.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.013+0000 7f8fca769700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc4148170 con 0x7f8fc4157d30 2023-11-15T03:00:00.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.013+0000 7f8fbb7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc40ae1f0 0x7f8fc4155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:00.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.013+0000 7f8fbb7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc40ae1f0 0x7f8fc4155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50126/0 (socket says 172.21.15.121:50126) 2023-11-15T03:00:00.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.013+0000 7f8fbb7fe700 1 -- 172.21.15.121:0/2147377889 learned_addr learned my addr 172.21.15.121:0/2147377889 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:00.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.013+0000 7f8fc3fff700 1 --2- 172.21.15.121:0/2147377889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc40ade10 0x7f8fc4147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:00.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.013+0000 7f8fc8d06700 1 --2- 172.21.15.121:0/2147377889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8fc4157d30 0x7f8fc41563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:00.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.013+0000 7f8fbb7fe700 1 -- 172.21.15.121:0/2147377889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8fc4157d30 msgr2=0x7f8fc41563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:00.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.013+0000 7f8fbb7fe700 1 --2- 172.21.15.121:0/2147377889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8fc4157d30 0x7f8fc41563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:00.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.013+0000 7f8fbb7fe700 1 -- 172.21.15.121:0/2147377889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc40ade10 msgr2=0x7f8fc4147c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:00.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.013+0000 7f8fbb7fe700 1 --2- 172.21.15.121:0/2147377889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc40ade10 0x7f8fc4147c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:00.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.013+0000 7f8fbb7fe700 1 -- 172.21.15.121:0/2147377889 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac009a20 con 0x7f8fc40ae1f0 2023-11-15T03:00:00.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.017+0000 7f8fbb7fe700 1 --2- 172.21.15.121:0/2147377889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc40ae1f0 0x7f8fc4155ee0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f8fac00b6c0 tx=0x7f8fac009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:00.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.017+0000 7f8fc0ff9700 1 -- 172.21.15.121:0/2147377889 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac01d420 con 0x7f8fc40ae1f0 2023-11-15T03:00:00.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.017+0000 7f8fc0ff9700 1 -- 172.21.15.121:0/2147377889 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fac01d580 con 0x7f8fc40ae1f0 2023-11-15T03:00:00.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.017+0000 7f8fca769700 1 -- 172.21.15.121:0/2147377889 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc4065150 con 0x7f8fc40ae1f0 2023-11-15T03:00:00.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.017+0000 7f8fc0ff9700 1 -- 172.21.15.121:0/2147377889 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac01d920 con 0x7f8fc40ae1f0 2023-11-15T03:00:00.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.017+0000 7f8fca769700 1 -- 172.21.15.121:0/2147377889 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc4065710 con 0x7f8fc40ae1f0 2023-11-15T03:00:00.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.017+0000 7f8fca769700 1 -- 172.21.15.121:0/2147377889 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc406c150 con 0x7f8fc40ae1f0 2023-11-15T03:00:00.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.021+0000 7f8fc0ff9700 1 -- 172.21.15.121:0/2147377889 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8fac003040 con 0x7f8fc40ae1f0 2023-11-15T03:00:00.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.021+0000 7f8fc0ff9700 1 --2- 172.21.15.121:0/2147377889 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8fa405a4e0 0x7f8fa405c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:00.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.021+0000 7f8fc0ff9700 1 -- 172.21.15.121:0/2147377889 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f8fac0766e0 con 0x7f8fc40ae1f0 2023-11-15T03:00:00.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.021+0000 7f8fc3fff700 1 --2- 172.21.15.121:0/2147377889 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8fa405a4e0 0x7f8fa405c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:00.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.021+0000 7f8fc3fff700 1 --2- 172.21.15.121:0/2147377889 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8fa405a4e0 0x7f8fa405c9a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f8fc40ad170 tx=0x7f8fb4007bb0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:00.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.025+0000 7f8fc0ff9700 1 -- 172.21.15.121:0/2147377889 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8fac049eb0 con 0x7f8fc40ae1f0 2023-11-15T03:00:00.090 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:00:00.085+0000 7f13603f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:00:00.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.225+0000 7f8fca769700 1 -- 172.21.15.121:0/2147377889 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fc4065ea0 con 0x7f8fc40ae1f0 2023-11-15T03:00:00.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.225+0000 7f8fc0ff9700 1 -- 172.21.15.121:0/2147377889 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4503 (secure 0 0 0) 0x7f8fac04d4d0 con 0x7f8fc40ae1f0 2023-11-15T03:00:00.231 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:00:00.231 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":52,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3462858472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3462858472},{"type":"v1","addr":"172.21.15.190:6833","nonce":3462858472}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4042390087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4042390087},{"type":"v1","addr":"172.21.15.190:6835","nonce":4042390087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5907,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2239114529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2239114529},{"type":"v1","addr":"172.21.15.121:6837","nonce":2239114529}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:59:34.667864+0000","modified":"2023-11-15T02:59:58.138367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":5,"addr":"172.21.15.121:6836/3429169582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3429169582},{"type":"v1","addr":"172.21.15.121:6836","nonce":3429169582}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-15T03:00:00.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.229+0000 7f8fca769700 1 -- 172.21.15.121:0/2147377889 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8fa405a4e0 msgr2=0x7f8fa405c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:00.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.229+0000 7f8fca769700 1 --2- 172.21.15.121:0/2147377889 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8fa405a4e0 0x7f8fa405c9a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f8fc40ad170 tx=0x7f8fb4007bb0).stop 2023-11-15T03:00:00.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.229+0000 7f8fca769700 1 -- 172.21.15.121:0/2147377889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc40ae1f0 msgr2=0x7f8fc4155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:00.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.229+0000 7f8fca769700 1 --2- 172.21.15.121:0/2147377889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc40ae1f0 0x7f8fc4155ee0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f8fac00b6c0 tx=0x7f8fac009780).stop 2023-11-15T03:00:00.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.229+0000 7f8fca769700 1 -- 172.21.15.121:0/2147377889 shutdown_connections 2023-11-15T03:00:00.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.229+0000 7f8fca769700 1 --2- 172.21.15.121:0/2147377889 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8fa405a4e0 0x7f8fa405c9a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:00.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.229+0000 7f8fca769700 1 --2- 172.21.15.121:0/2147377889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8fc4157d30 0x7f8fc41563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:00.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.229+0000 7f8fca769700 1 --2- 172.21.15.121:0/2147377889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc40ae1f0 0x7f8fc4155ee0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:00.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.229+0000 7f8fca769700 1 --2- 172.21.15.121:0/2147377889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc40ade10 0x7f8fc4147c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:00.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.229+0000 7f8fca769700 1 -- 172.21.15.121:0/2147377889 >> 172.21.15.121:0/2147377889 conn(0x7f8fc40c89c0 msgr2=0x7f8fc4151e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:00.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.229+0000 7f8fca769700 1 -- 172.21.15.121:0/2147377889 shutdown_connections 2023-11-15T03:00:00.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.229+0000 7f8fca769700 1 -- 172.21.15.121:0/2147377889 wait complete. 2023-11-15T03:00:00.237 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 52 2023-11-15T03:00:00.254 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 4:0 true 2023-11-15T03:00:00.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.409+0000 7f3591969700 1 Processor -- start 2023-11-15T03:00:00.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.409+0000 7f3591969700 1 -- start start 2023-11-15T03:00:00.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.409+0000 7f3591969700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f358c0a86a0 0x7f358c0c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:00.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.409+0000 7f3591969700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f358c152c00 0x7f358c0a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:00.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f3591969700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358c152770 con 0x7f358c0670e0 2023-11-15T03:00:00.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f3591969700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f358c0c72a0 con 0x7f358c152c00 2023-11-15T03:00:00.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f3591969700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358c152120 con 0x7f358c0a86a0 2023-11-15T03:00:00.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f358a7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f358c152c00 0x7f358c0a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:00.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f358a7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f358c152c00 0x7f358c0a8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50142/0 (socket says 172.21.15.121:50142) 2023-11-15T03:00:00.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f358a7fc700 1 -- 172.21.15.121:0/912105028 learned_addr learned my addr 172.21.15.121:0/912105028 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:00.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f358affd700 1 --2- 172.21.15.121:0/912105028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f358c0a86a0 0x7f358c0c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:00.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f358a7fc700 1 -- 172.21.15.121:0/912105028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f358c0a86a0 msgr2=0x7f358c0c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:00.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f358a7fc700 1 --2- 172.21.15.121:0/912105028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f358c0a86a0 0x7f358c0c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:00.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f358a7fc700 1 -- 172.21.15.121:0/912105028 >> v1:172.21.15.190:6789/0 conn(0x7f358c0670e0 legacy=0x7f358c151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:00.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f358a7fc700 1 -- 172.21.15.121:0/912105028 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c152400 con 0x7f358c152c00 2023-11-15T03:00:00.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f358affd700 1 --2- 172.21.15.121:0/912105028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f358c0a86a0 0x7f358c0c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:00:00.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f358a7fc700 1 --2- 172.21.15.121:0/912105028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f358c152c00 0x7f358c0a8a80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f3580006fd0 tx=0x7f35800090f0).ready entity=mon.0 client_cookie=fcb8c3a5998a555a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:00.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f3589ffb700 1 -- 172.21.15.121:0/912105028 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358000b600 con 0x7f358c152c00 2023-11-15T03:00:00.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f3589ffb700 1 -- 172.21.15.121:0/912105028 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358000b760 con 0x7f358c152c00 2023-11-15T03:00:00.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f3589ffb700 1 -- 172.21.15.121:0/912105028 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358000bae0 con 0x7f358c152c00 2023-11-15T03:00:00.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f3591969700 1 -- 172.21.15.121:0/912105028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f358c152c00 msgr2=0x7f358c0a8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:00.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f3591969700 1 --2- 172.21.15.121:0/912105028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f358c152c00 0x7f358c0a8a80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f3580006fd0 tx=0x7f35800090f0).stop 2023-11-15T03:00:00.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f3591969700 1 -- 172.21.15.121:0/912105028 shutdown_connections 2023-11-15T03:00:00.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f3591969700 1 --2- 172.21.15.121:0/912105028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f358c152c00 0x7f358c0a8a80 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:00.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f3591969700 1 --2- 172.21.15.121:0/912105028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f358c0a86a0 0x7f358c0c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:00.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f3591969700 1 -- 172.21.15.121:0/912105028 >> 172.21.15.121:0/912105028 conn(0x7f358c068530 msgr2=0x7f358c0c6560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:00.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f3591969700 1 -- 172.21.15.121:0/912105028 shutdown_connections 2023-11-15T03:00:00.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.413+0000 7f3591969700 1 -- 172.21.15.121:0/912105028 wait complete. 2023-11-15T03:00:00.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.417+0000 7f3591969700 1 Processor -- start 2023-11-15T03:00:00.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.417+0000 7f3591969700 1 -- start start 2023-11-15T03:00:00.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f3591969700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f358c0670e0 0x7f358c0c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:00.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f3591969700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f358c0a86a0 0x7f358c14ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:00.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f3591969700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f358c152c00 0x7f358c0a6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:00.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f3591969700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c1514e0 con 0x7f358c0670e0 2023-11-15T03:00:00.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f3591969700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c0c2470 con 0x7f358c0a86a0 2023-11-15T03:00:00.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f3591969700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358c1508d0 con 0x7f358c152c00 2023-11-15T03:00:00.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f358affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f358c0670e0 0x7f358c0c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:00.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f358affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f358c0670e0 0x7f358c0c1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50152/0 (socket says 172.21.15.121:50152) 2023-11-15T03:00:00.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f358affd700 1 -- 172.21.15.121:0/575393752 learned_addr learned my addr 172.21.15.121:0/575393752 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:00.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f358b7fe700 1 --2- 172.21.15.121:0/575393752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f358c152c00 0x7f358c0a6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:00.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f358a7fc700 1 --2- 172.21.15.121:0/575393752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f358c0a86a0 0x7f358c14ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:00.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f358affd700 1 -- 172.21.15.121:0/575393752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f358c152c00 msgr2=0x7f358c0a6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:00.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f358affd700 1 --2- 172.21.15.121:0/575393752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f358c152c00 0x7f358c0a6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:00.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f358affd700 1 -- 172.21.15.121:0/575393752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f358c0a86a0 msgr2=0x7f358c14ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:00.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f358affd700 1 --2- 172.21.15.121:0/575393752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f358c0a86a0 0x7f358c14ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:00.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f358affd700 1 -- 172.21.15.121:0/575393752 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3580009370 con 0x7f358c0670e0 2023-11-15T03:00:00.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f358a7fc700 1 --2- 172.21.15.121:0/575393752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f358c0a86a0 0x7f358c14ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:00:00.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f358affd700 1 --2- 172.21.15.121:0/575393752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f358c0670e0 0x7f358c0c1f80 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f357c00c8b0 tx=0x7f357c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:00.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f3572ffd700 1 -- 172.21.15.121:0/575393752 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c010580 con 0x7f358c0670e0 2023-11-15T03:00:00.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f3572ffd700 1 -- 172.21.15.121:0/575393752 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357c0106e0 con 0x7f358c0670e0 2023-11-15T03:00:00.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f3572ffd700 1 -- 172.21.15.121:0/575393752 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c010a60 con 0x7f358c0670e0 2023-11-15T03:00:00.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f3591969700 1 -- 172.21.15.121:0/575393752 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358c150bb0 con 0x7f358c0670e0 2023-11-15T03:00:00.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.421+0000 7f3591969700 1 -- 172.21.15.121:0/575393752 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358c151170 con 0x7f358c0670e0 2023-11-15T03:00:00.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.425+0000 7f3591969700 1 -- 172.21.15.121:0/575393752 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f358c06bcf0 con 0x7f358c0670e0 2023-11-15T03:00:00.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.425+0000 7f3572ffd700 1 -- 172.21.15.121:0/575393752 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f357c01b460 con 0x7f358c0670e0 2023-11-15T03:00:00.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.425+0000 7f3572ffd700 1 --2- 172.21.15.121:0/575393752 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f357805a510 0x7f357805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:00.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.425+0000 7f3572ffd700 1 -- 172.21.15.121:0/575393752 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f357c074960 con 0x7f358c0670e0 2023-11-15T03:00:00.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.425+0000 7f358a7fc700 1 --2- 172.21.15.121:0/575393752 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f357805a510 0x7f357805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:00.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.429+0000 7f358a7fc700 1 --2- 172.21.15.121:0/575393752 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f357805a510 0x7f357805c9d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f3580006fd0 tx=0x7f3580017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:00.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.433+0000 7f3572ffd700 1 -- 172.21.15.121:0/575393752 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f357c047fd0 con 0x7f358c0670e0 2023-11-15T03:00:00.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:00.649+0000 7f3591969700 1 -- 172.21.15.121:0/575393752 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "4:0", "val": "true"} v 0) v1 -- 0x7f358c063840 con 0x7f358c0670e0 2023-11-15T03:00:01.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.169+0000 7f3572ffd700 1 -- 172.21.15.121:0/575393752 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "4:0", "val": "true"}]=0 freezing mds.6144 v53) v1 ==== 112+0+0 (secure 0 0 0) 0x7f357c04b5f0 con 0x7f358c0670e0 2023-11-15T03:00:01.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.173+0000 7f3591969700 1 -- 172.21.15.121:0/575393752 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f357805a510 msgr2=0x7f357805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:01.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.173+0000 7f3591969700 1 --2- 172.21.15.121:0/575393752 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f357805a510 0x7f357805c9d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f3580006fd0 tx=0x7f3580017000).stop 2023-11-15T03:00:01.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.173+0000 7f3591969700 1 -- 172.21.15.121:0/575393752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f358c0670e0 msgr2=0x7f358c0c1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:01.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.173+0000 7f3591969700 1 --2- 172.21.15.121:0/575393752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f358c0670e0 0x7f358c0c1f80 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f357c00c8b0 tx=0x7f357c008040).stop 2023-11-15T03:00:01.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.173+0000 7f3591969700 1 -- 172.21.15.121:0/575393752 shutdown_connections 2023-11-15T03:00:01.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.173+0000 7f3591969700 1 --2- 172.21.15.121:0/575393752 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f357805a510 0x7f357805c9d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.173+0000 7f3591969700 1 --2- 172.21.15.121:0/575393752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f358c152c00 0x7f358c0a6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.173+0000 7f3591969700 1 --2- 172.21.15.121:0/575393752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f358c0a86a0 0x7f358c14ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.173+0000 7f3591969700 1 --2- 172.21.15.121:0/575393752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f358c0670e0 0x7f358c0c1f80 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.173+0000 7f3591969700 1 -- 172.21.15.121:0/575393752 >> 172.21.15.121:0/575393752 conn(0x7f358c068530 msgr2=0x7f358c0aaf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:01.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.173+0000 7f3591969700 1 -- 172.21.15.121:0/575393752 shutdown_connections 2023-11-15T03:00:01.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.173+0000 7f3591969700 1 -- 172.21.15.121:0/575393752 wait complete. 2023-11-15T03:00:01.178 INFO:teuthology.orchestra.run.smithi121.stderr:freezing mds.6144 2023-11-15T03:00:01.197 DEBUG:teuthology.orchestra.run.smithi121:> timeout 30 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --connect-timeout=60 tell mds.4:0 flush journal 2023-11-15T03:00:01.204 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-15T03:00:01.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.365+0000 7f71e759e700 1 Processor -- start 2023-11-15T03:00:01.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.365+0000 7f71e759e700 1 -- start start 2023-11-15T03:00:01.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e759e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f71e814ac10 0x7f71e80c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e759e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f71e814aff0 0x7f71e8066c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71e8067160 0x7f71e8154370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e759e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e80657f0 con 0x7f71e814aff0 2023-11-15T03:00:01.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e759e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e8066650 con 0x7f71e814ac10 2023-11-15T03:00:01.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e80675b0 con 0x7f71e8067160 2023-11-15T03:00:01.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71dffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f71e814aff0 0x7f71e8066c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71dffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f71e814aff0 0x7f71e8066c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50164/0 (socket says 172.21.15.121:50164) 2023-11-15T03:00:01.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71dffff700 1 -- 172.21.15.121:0/304936814 learned_addr learned my addr 172.21.15.121:0/304936814 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:01.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e6d9d700 1 --2- 172.21.15.121:0/304936814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71e8067160 0x7f71e8154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e659c700 1 --2- 172.21.15.121:0/304936814 >> v2:172.21.15.190:3300/0 conn(0x7f71e814ac10 0x7f71e80c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71dffff700 1 -- 172.21.15.121:0/304936814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71e8067160 msgr2=0x7f71e8154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:01.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71dffff700 1 --2- 172.21.15.121:0/304936814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71e8067160 0x7f71e8154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71dffff700 1 -- 172.21.15.121:0/304936814 >> v2:172.21.15.190:3300/0 conn(0x7f71e814ac10 msgr2=0x7f71e80c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:01.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71dffff700 1 --2- 172.21.15.121:0/304936814 >> v2:172.21.15.190:3300/0 conn(0x7f71e814ac10 0x7f71e80c6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71dffff700 1 -- 172.21.15.121:0/304936814 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e8065a40 con 0x7f71e814aff0 2023-11-15T03:00:01.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71dffff700 1 --2- 172.21.15.121:0/304936814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71e814aff0 0x7f71e8066c70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f71e8064d60 tx=0x7f71d00090f0).ready entity=mon.0 client_cookie=3b1fe682ea2e6ad0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:01.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e5d9b700 1 -- 172.21.15.121:0/304936814 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d000b600 con 0x7f71e814aff0 2023-11-15T03:00:01.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e5d9b700 1 -- 172.21.15.121:0/304936814 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d000b760 con 0x7f71e814aff0 2023-11-15T03:00:01.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e5d9b700 1 -- 172.21.15.121:0/304936814 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d000bae0 con 0x7f71e814aff0 2023-11-15T03:00:01.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e759e700 1 -- 172.21.15.121:0/304936814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71e814aff0 msgr2=0x7f71e8066c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:01.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e759e700 1 --2- 172.21.15.121:0/304936814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71e814aff0 0x7f71e8066c70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f71e8064d60 tx=0x7f71d00090f0).stop 2023-11-15T03:00:01.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e759e700 1 -- 172.21.15.121:0/304936814 shutdown_connections 2023-11-15T03:00:01.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e759e700 1 --2- 172.21.15.121:0/304936814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71e8067160 0x7f71e8154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e759e700 1 --2- 172.21.15.121:0/304936814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71e814aff0 0x7f71e8066c70 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e759e700 1 --2- 172.21.15.121:0/304936814 >> v2:172.21.15.190:3300/0 conn(0x7f71e814ac10 0x7f71e80c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e759e700 1 -- 172.21.15.121:0/304936814 >> 172.21.15.121:0/304936814 conn(0x7f71e8067a60 msgr2=0x7f71e8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:01.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e759e700 1 -- 172.21.15.121:0/304936814 shutdown_connections 2023-11-15T03:00:01.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.369+0000 7f71e759e700 1 -- 172.21.15.121:0/304936814 wait complete. 2023-11-15T03:00:01.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.373+0000 7f71e759e700 1 Processor -- start 2023-11-15T03:00:01.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.373+0000 7f71e759e700 1 -- start start 2023-11-15T03:00:01.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71e759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71e8067160 0x7f71e8166d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71e759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71e814ac10 0x7f71e8163ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71e759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71e814aff0 0x7f71e80c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71e759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e80ac340 con 0x7f71e8067160 2023-11-15T03:00:01.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71e759e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e8147650 con 0x7f71e814aff0 2023-11-15T03:00:01.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71e759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e80abd80 con 0x7f71e814ac10 2023-11-15T03:00:01.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71e659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71e8067160 0x7f71e8166d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71e659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71e8067160 0x7f71e8166d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50166/0 (socket says 172.21.15.121:50166) 2023-11-15T03:00:01.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71e659c700 1 -- 172.21.15.121:0/2263607540 learned_addr learned my addr 172.21.15.121:0/2263607540 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:01.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71dffff700 1 --2- 172.21.15.121:0/2263607540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71e814ac10 0x7f71e8163ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71e6d9d700 1 --2- 172.21.15.121:0/2263607540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71e814aff0 0x7f71e80c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71e659c700 1 -- 172.21.15.121:0/2263607540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71e814ac10 msgr2=0x7f71e8163ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:01.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71e659c700 1 --2- 172.21.15.121:0/2263607540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71e814ac10 0x7f71e8163ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71e659c700 1 -- 172.21.15.121:0/2263607540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71e814aff0 msgr2=0x7f71e80c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:01.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71e659c700 1 --2- 172.21.15.121:0/2263607540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71e814aff0 0x7f71e80c15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71e659c700 1 -- 172.21.15.121:0/2263607540 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d0009370 con 0x7f71e8067160 2023-11-15T03:00:01.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71e6d9d700 1 --2- 172.21.15.121:0/2263607540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71e814aff0 0x7f71e80c15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:00:01.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71e659c700 1 --2- 172.21.15.121:0/2263607540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71e8067160 0x7f71e8166d60 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f71d800d890 tx=0x7f71d8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:01.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71de7fc700 1 -- 172.21.15.121:0/2263607540 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d80125f0 con 0x7f71e8067160 2023-11-15T03:00:01.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71e759e700 1 -- 172.21.15.121:0/2263607540 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e80ac060 con 0x7f71e8067160 2023-11-15T03:00:01.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71de7fc700 1 -- 172.21.15.121:0/2263607540 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d8012750 con 0x7f71e8067160 2023-11-15T03:00:01.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71de7fc700 1 -- 172.21.15.121:0/2263607540 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8012ad0 con 0x7f71e8067160 2023-11-15T03:00:01.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.377+0000 7f71e759e700 1 -- 172.21.15.121:0/2263607540 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e8167250 con 0x7f71e8067160 2023-11-15T03:00:01.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.381+0000 7f71de7fc700 1 -- 172.21.15.121:0/2263607540 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f71d8013480 con 0x7f71e8067160 2023-11-15T03:00:01.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.381+0000 7f71f70e4740 1 Processor -- start 2023-11-15T03:00:01.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.381+0000 7f71f70e4740 1 -- start start 2023-11-15T03:00:01.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.381+0000 7f71de7fc700 1 --2- 172.21.15.121:0/2263607540 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71cc05a4e0 0x7f71cc05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.381+0000 7f71de7fc700 1 -- 172.21.15.121:0/2263607540 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f71d8074790 con 0x7f71e8067160 2023-11-15T03:00:01.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.381+0000 7f71e6d9d700 1 --2- 172.21.15.121:0/2263607540 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71cc05a4e0 0x7f71cc05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.381+0000 7f71f70e4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x186df90 0x186e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.381+0000 7f71f70e4740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x184faf0 0x18239c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.381+0000 7f71f70e4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1823eb0 0x186bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.381+0000 7f71f70e4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x176f650 con 0x1823eb0 2023-11-15T03:00:01.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.381+0000 7f71f70e4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1764f30 con 0x186df90 2023-11-15T03:00:01.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.381+0000 7f71f70e4740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x17523b0 con 0x184faf0 2023-11-15T03:00:01.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71e6d9d700 1 --2- 172.21.15.121:0/2263607540 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71cc05a4e0 0x7f71cc05c9a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f71e0007540 tx=0x7f71e000b040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:01.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71dffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x184faf0 0x18239c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71e659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x186df90 0x186e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71e6d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1823eb0 0x186bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71dffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x184faf0 0x18239c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:40888/0 (socket says 172.21.15.121:40888) 2023-11-15T03:00:01.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71dffff700 1 -- 172.21.15.121:0/2435079368 learned_addr learned my addr 172.21.15.121:0/2435079368 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:01.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71e6d9d700 1 -- 172.21.15.121:0/2435079368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x184faf0 msgr2=0x18239c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:01.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71e6d9d700 1 --2- 172.21.15.121:0/2435079368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x184faf0 0x18239c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71e6d9d700 1 -- 172.21.15.121:0/2435079368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x186df90 msgr2=0x186e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:01.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71e6d9d700 1 --2- 172.21.15.121:0/2435079368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x186df90 0x186e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71e6d9d700 1 -- 172.21.15.121:0/2435079368 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x17a5840 con 0x1823eb0 2023-11-15T03:00:01.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71e659c700 1 --2- 172.21.15.121:0/2435079368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x186df90 0x186e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:00:01.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71e6d9d700 1 --2- 172.21.15.121:0/2435079368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1823eb0 0x186bbc0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f71e00168a0 tx=0x7f71e0014000).ready entity=mon.0 client_cookie=f9b43d75e990f861 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:01.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71d77fe700 1 -- 172.21.15.121:0/2435079368 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e001b070 con 0x1823eb0 2023-11-15T03:00:01.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71d77fe700 1 -- 172.21.15.121:0/2435079368 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e00160d0 con 0x1823eb0 2023-11-15T03:00:01.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71d77fe700 1 -- 172.21.15.121:0/2435079368 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e00136d0 con 0x1823eb0 2023-11-15T03:00:01.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71f70e4740 1 -- 172.21.15.121:0/2435079368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1823eb0 msgr2=0x186bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:01.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71f70e4740 1 --2- 172.21.15.121:0/2435079368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1823eb0 0x186bbc0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f71e00168a0 tx=0x7f71e0014000).stop 2023-11-15T03:00:01.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71f70e4740 1 -- 172.21.15.121:0/2435079368 shutdown_connections 2023-11-15T03:00:01.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71f70e4740 1 --2- 172.21.15.121:0/2435079368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1823eb0 0x186bbc0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71f70e4740 1 --2- 172.21.15.121:0/2435079368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x184faf0 0x18239c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71f70e4740 1 --2- 172.21.15.121:0/2435079368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x186df90 0x186e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71f70e4740 1 -- 172.21.15.121:0/2435079368 >> 172.21.15.121:0/2435079368 conn(0x160a7d0 msgr2=0x182ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:01.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71f70e4740 1 -- 172.21.15.121:0/2435079368 shutdown_connections 2023-11-15T03:00:01.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.385+0000 7f71f70e4740 1 -- 172.21.15.121:0/2435079368 wait complete. 2023-11-15T03:00:01.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71f70e4740 1 -- start start 2023-11-15T03:00:01.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71f70e4740 1 Processor -- start 2023-11-15T03:00:01.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71f70e4740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1823eb0 0x186aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71f70e4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x184faf0 0x186b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71f70e4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x186df90 0x1868aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71f70e4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1854900 con 0x184faf0 2023-11-15T03:00:01.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71f70e4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1854be0 con 0x186df90 2023-11-15T03:00:01.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71f70e4740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1854ec0 con 0x1823eb0 2023-11-15T03:00:01.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71f70e4740 10 client.0 ll_register_callbacks cb 0x17e21b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:00:01.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71dffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x184faf0 0x186b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71dffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x184faf0 0x186b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50184/0 (socket says 172.21.15.121:50184) 2023-11-15T03:00:01.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71dffff700 1 -- 172.21.15.121:0/806431936 learned_addr learned my addr 172.21.15.121:0/806431936 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:01.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71e659c700 1 --2- 172.21.15.121:0/806431936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1823eb0 0x186aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71e6d9d700 1 --2- 172.21.15.121:0/806431936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x186df90 0x1868aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71dffff700 1 -- 172.21.15.121:0/806431936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1823eb0 msgr2=0x186aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:01.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71dffff700 1 --2- 172.21.15.121:0/806431936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1823eb0 0x186aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71dffff700 1 -- 172.21.15.121:0/806431936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x186df90 msgr2=0x1868aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:01.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71dffff700 1 --2- 172.21.15.121:0/806431936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x186df90 0x1868aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71dffff700 1 -- 172.21.15.121:0/806431936 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e8167250 con 0x184faf0 2023-11-15T03:00:01.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71dffff700 1 --2- 172.21.15.121:0/806431936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x184faf0 0x186b190 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f71e8063d00 tx=0x7f71d0017000).ready entity=mon.0 client_cookie=35d3e494fd390c95 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:01.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:00:01.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71d4ff9700 1 -- 172.21.15.121:0/806431936 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d0022070 con 0x184faf0 2023-11-15T03:00:01.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71d4ff9700 1 -- 172.21.15.121:0/806431936 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f71d001d600 con 0x184faf0 2023-11-15T03:00:01.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.389+0000 7f71f70e4740 1 -- 172.21.15.121:0/806431936 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x17a22b0 con 0x184faf0 2023-11-15T03:00:01.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71d4ff9700 1 -- 172.21.15.121:0/806431936 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71d000b600 con 0x184faf0 2023-11-15T03:00:01.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 10 client.6222 fetch_fsmap learned FSMap version 53 2023-11-15T03:00:01.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 1 -- 172.21.15.121:0/806431936 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x7f71e0016570 con 0x184faf0 2023-11-15T03:00:01.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71d4ff9700 1 -- 172.21.15.121:0/806431936 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x17a22b0 con 0x184faf0 2023-11-15T03:00:01.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71d4ff9700 1 -- 172.21.15.121:0/806431936 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 53) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f71d00098e0 con 0x184faf0 2023-11-15T03:00:01.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 10 client.6222 fetch_fsmap finished waiting for FSMap version 53 2023-11-15T03:00:01.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 10 client.6222 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-15T03:00:01.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 20 client.6222 populate_metadata read hostname 'smithi121' 2023-11-15T03:00:01.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 1 --2- 172.21.15.121:0/806431936 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x184cf20 0x184d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 4 client.6222 mds_command: new command op to 6144 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:00:01.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 1 -- 172.21.15.121:0/806431936 --> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1798800 con 0x184cf20 2023-11-15T03:00:01.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71e659c700 1 --2- 172.21.15.121:0/806431936 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x184cf20 0x184d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71d4ff9700 10 client.6222 ms_handle_connect on v2:172.21.15.121:6835/3429169582 2023-11-15T03:00:01.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71e659c700 1 --2- 172.21.15.121:0/806431936 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x184cf20 0x184d330 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=a211eb37048dd655 server_cookie=612340401e12fe62 in_seq=0 out_seq=0 2023-11-15T03:00:01.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71d4ff9700 1 -- 172.21.15.121:0/806431936 <== mds.0 v2:172.21.15.121:6835/3429169582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x1798800 con 0x184cf20 2023-11-15T03:00:01.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71d4ff9700 10 client.6222 handle_command_reply: tid=0 2023-11-15T03:00:01.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 1 client.6222 shutdown 2023-11-15T03:00:01.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 1 -- 172.21.15.121:0/806431936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x184faf0 msgr2=0x186b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:01.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 1 --2- 172.21.15.121:0/806431936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x184faf0 0x186b190 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f71e8063d00 tx=0x7f71d0017000).stop 2023-11-15T03:00:01.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 1 -- 172.21.15.121:0/806431936 shutdown_connections 2023-11-15T03:00:01.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 1 --2- 172.21.15.121:0/806431936 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x184cf20 0x184d330 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 1 --2- 172.21.15.121:0/806431936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x186df90 0x1868aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 1 --2- 172.21.15.121:0/806431936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x184faf0 0x186b190 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71d4ff9700 0 client.6222 ms_handle_reset on v2:172.21.15.121:6835/3429169582 2023-11-15T03:00:01.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 1 --2- 172.21.15.121:0/806431936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1823eb0 0x186aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 1 -- 172.21.15.121:0/806431936 >> 172.21.15.121:0/806431936 conn(0x160a7d0 msgr2=0x18293d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:01.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 1 -- 172.21.15.121:0/806431936 shutdown_connections 2023-11-15T03:00:01.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 1 -- 172.21.15.121:0/806431936 wait complete. 2023-11-15T03:00:01.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.393+0000 7f71f70e4740 20 client.6222 trim_cache size 0 max 16384 2023-11-15T03:00:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-15T03:00:01.411 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-15T03:00:01.412 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:00:01.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.409+0000 7f71f70e4740 1 Processor -- start 2023-11-15T03:00:01.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.409+0000 7f71f70e4740 1 -- start start 2023-11-15T03:00:01.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.409+0000 7f71f70e4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1823eb0 0x187ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.409+0000 7f71f70e4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x184faf0 0x186bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.409+0000 7f71f70e4740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x186df90 0x186a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.409+0000 7f71f70e4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x184f070 con 0x184faf0 2023-11-15T03:00:01.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.409+0000 7f71f70e4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1825170 con 0x1823eb0 2023-11-15T03:00:01.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.409+0000 7f71f70e4740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1798800 con 0x186df90 2023-11-15T03:00:01.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.409+0000 7f71dffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x184faf0 0x186bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.409+0000 7f71dffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x184faf0 0x186bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50186/0 (socket says 172.21.15.121:50186) 2023-11-15T03:00:01.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.409+0000 7f71dffff700 1 -- 172.21.15.121:0/3356148551 learned_addr learned my addr 172.21.15.121:0/3356148551 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:01.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.409+0000 7f71e659c700 1 --2- 172.21.15.121:0/3356148551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1823eb0 0x187ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.409+0000 7f71e6d9d700 1 --2- 172.21.15.121:0/3356148551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x186df90 0x186a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71dffff700 1 -- 172.21.15.121:0/3356148551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x186df90 msgr2=0x186a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:01.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71dffff700 1 --2- 172.21.15.121:0/3356148551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x186df90 0x186a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71dffff700 1 -- 172.21.15.121:0/3356148551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1823eb0 msgr2=0x187ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:01.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71dffff700 1 --2- 172.21.15.121:0/3356148551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1823eb0 0x187ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71dffff700 1 -- 172.21.15.121:0/3356148551 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e80c1df0 con 0x184faf0 2023-11-15T03:00:01.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71e6d9d700 1 --2- 172.21.15.121:0/3356148551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x186df90 0x186a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:00:01.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71dffff700 1 --2- 172.21.15.121:0/3356148551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x184faf0 0x186bbd0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f71d0009ff0 tx=0x7f71d0022000).ready entity=mon.0 client_cookie=d7f8d202c167abf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:01.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71d7fff700 1 -- 172.21.15.121:0/3356148551 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d0036070 con 0x184faf0 2023-11-15T03:00:01.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71d7fff700 1 -- 172.21.15.121:0/3356148551 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d001d600 con 0x184faf0 2023-11-15T03:00:01.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71d7fff700 1 -- 172.21.15.121:0/3356148551 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d000b600 con 0x184faf0 2023-11-15T03:00:01.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71f70e4740 1 -- 172.21.15.121:0/3356148551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x184faf0 msgr2=0x186bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:01.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71f70e4740 1 --2- 172.21.15.121:0/3356148551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x184faf0 0x186bbd0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f71d0009ff0 tx=0x7f71d0022000).stop 2023-11-15T03:00:01.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71f70e4740 1 -- 172.21.15.121:0/3356148551 shutdown_connections 2023-11-15T03:00:01.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71f70e4740 1 --2- 172.21.15.121:0/3356148551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x186df90 0x186a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71f70e4740 1 --2- 172.21.15.121:0/3356148551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x184faf0 0x186bbd0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71f70e4740 1 --2- 172.21.15.121:0/3356148551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1823eb0 0x187ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71f70e4740 1 -- 172.21.15.121:0/3356148551 >> 172.21.15.121:0/3356148551 conn(0x160a7d0 msgr2=0x184da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:01.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71f70e4740 1 -- 172.21.15.121:0/3356148551 shutdown_connections 2023-11-15T03:00:01.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71f70e4740 1 -- 172.21.15.121:0/3356148551 wait complete. 2023-11-15T03:00:01.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71f70e4740 1 -- start start 2023-11-15T03:00:01.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.413+0000 7f71f70e4740 1 Processor -- start 2023-11-15T03:00:01.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71f70e4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1823eb0 0x186ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71f70e4740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x186df90 0x186b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71f70e4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x18cd650 0x18cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71f70e4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x182b2e0 con 0x18cd650 2023-11-15T03:00:01.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71f70e4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x188b730 con 0x1823eb0 2023-11-15T03:00:01.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71f70e4740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x182b090 con 0x186df90 2023-11-15T03:00:01.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71f70e4740 10 client.0 ll_register_callbacks cb 0x7f71d0006ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:00:01.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71e6d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x18cd650 0x18cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71e6d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x18cd650 0x18cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50188/0 (socket says 172.21.15.121:50188) 2023-11-15T03:00:01.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71e6d9d700 1 -- 172.21.15.121:0/4034411113 learned_addr learned my addr 172.21.15.121:0/4034411113 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:01.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71dffff700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x186df90 0x186b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1823eb0 0x186ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71dffff700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1823eb0 msgr2=0x186ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:01.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71dffff700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1823eb0 0x186ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71dffff700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x18cd650 msgr2=0x18cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:01.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71dffff700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x18cd650 0x18cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71dffff700 1 -- 172.21.15.121:0/4034411113 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e0016570 con 0x186df90 2023-11-15T03:00:01.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71e6d9d700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x18cd650 0x18cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:00:01.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71dffff700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x186df90 0x186b010 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f71d0006ec0 tx=0x7f71d0013000).ready entity=mon.2 client_cookie=d2c084ca6f9d6ab4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:01.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.190:3301/0 2023-11-15T03:00:01.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71d4ff9700 1 -- 172.21.15.121:0/4034411113 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d0022040 con 0x186df90 2023-11-15T03:00:01.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71f70e4740 1 -- 172.21.15.121:0/4034411113 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71e00160d0 con 0x186df90 2023-11-15T03:00:01.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71d4ff9700 1 -- 172.21.15.121:0/4034411113 <== mon.2 v2:172.21.15.190:3301/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f71d0032db0 con 0x186df90 2023-11-15T03:00:01.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71d4ff9700 1 -- 172.21.15.121:0/4034411113 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71d0036040 con 0x186df90 2023-11-15T03:00:01.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71f70e4740 10 client.5633 fetch_fsmap learned FSMap version 53 2023-11-15T03:00:01.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71f70e4740 1 -- 172.21.15.121:0/4034411113 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x1868660 con 0x186df90 2023-11-15T03:00:01.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71d4ff9700 1 -- 172.21.15.121:0/4034411113 <== mon.2 v2:172.21.15.190:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e00160d0 con 0x186df90 2023-11-15T03:00:01.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71d4ff9700 1 -- 172.21.15.121:0/4034411113 <== mon.2 v2:172.21.15.190:3301/0 5 ==== fsmap(e 53) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f71d0043070 con 0x186df90 2023-11-15T03:00:01.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71f70e4740 10 client.5633 fetch_fsmap finished waiting for FSMap version 53 2023-11-15T03:00:01.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71f70e4740 10 client.5633 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-15T03:00:01.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71f70e4740 20 client.5633 populate_metadata read hostname 'smithi121' 2023-11-15T03:00:01.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71f70e4740 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71f70e4740 4 client.5633 mds_command: new command op to 6144 tid=0[{"prefix": "flush journal"}] 2023-11-15T03:00:01.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.417+0000 7f71f70e4740 1 -- 172.21.15.121:0/4034411113 --> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x1661760 con 0x188a500 2023-11-15T03:00:01.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.421+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.421+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=bdf9595402cc9e56 server_cookie=aefd7a1d4ac525b7 in_seq=0 out_seq=0 2023-11-15T03:00:01.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.421+0000 7f71d4ff9700 10 client.5633 ms_handle_connect on v2:172.21.15.121:6835/3429169582 2023-11-15T03:00:01.435 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:00:01.425+0000 7f135cbea700 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x56060dd9b680] 2023-11-15T03:00:01.436 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:00:01.425+0000 7f135cbea700 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x56060dd9b680] 2023-11-15T03:00:01.436 INFO:tasks.ceph.mds.a.smithi121.stderr:/build/ceph-16.2.14-170-g620f25c1/src/mds/CDentry.cc: In function 'bool CDentry::check_corruption(bool)' thread 7f135cbea700 time 2023-11-15T03:00:01.427855+0000 2023-11-15T03:00:01.436 INFO:tasks.ceph.mds.a.smithi121.stderr:/build/ceph-16.2.14-170-g620f25c1/src/mds/CDentry.cc: 699: ceph_abort_msg("abort() called") 2023-11-15T03:00:01.436 INFO:tasks.ceph.mds.a.smithi121.stderr: ceph version 16.2.14-170-g620f25c1 (620f25c1794f62207424fc9319427943fb8d1fae) pacific (stable) 2023-11-15T03:00:01.436 INFO:tasks.ceph.mds.a.smithi121.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe0) [0x7f13680684b5] 2023-11-15T03:00:01.437 INFO:tasks.ceph.mds.a.smithi121.stderr: 2: (CDentry::check_corruption(bool)+0x86b) [0x56060b9a0cbb] 2023-11-15T03:00:01.437 INFO:tasks.ceph.mds.a.smithi121.stderr: 3: ceph-mds(+0x48daaa) [0x56060b9c4aaa] 2023-11-15T03:00:01.437 INFO:tasks.ceph.mds.a.smithi121.stderr: 4: (CDir::_omap_commit(int)+0x318) [0x56060b9c51e8] 2023-11-15T03:00:01.437 INFO:tasks.ceph.mds.a.smithi121.stderr: 5: (CDir::_commit(unsigned long, int)+0x289) [0x56060b9c6839] 2023-11-15T03:00:01.437 INFO:tasks.ceph.mds.a.smithi121.stderr: 6: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x138) [0x56060b9c6eb8] 2023-11-15T03:00:01.438 INFO:tasks.ceph.mds.a.smithi121.stderr: 7: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x170f) [0x56060bb0ff9f] 2023-11-15T03:00:01.438 INFO:tasks.ceph.mds.a.smithi121.stderr: 8: (MDLog::try_expire(LogSegment*, int)+0x6e) [0x56060ba9901e] 2023-11-15T03:00:01.438 INFO:tasks.ceph.mds.a.smithi121.stderr: 9: (MDLog::trim_all()+0xa18) [0x56060ba9af48] 2023-11-15T03:00:01.438 INFO:tasks.ceph.mds.a.smithi121.stderr: 10: (LambdaContext::finish(int)+0x231) [0x56060b74d3c1] 2023-11-15T03:00:01.438 INFO:tasks.ceph.mds.a.smithi121.stderr: 11: (Context::complete(int)+0xd) [0x56060b70317d] 2023-11-15T03:00:01.439 INFO:tasks.ceph.mds.a.smithi121.stderr: 12: (MDSContext::complete(int)+0x61) [0x56060ba7a4b1] 2023-11-15T03:00:01.439 INFO:tasks.ceph.mds.a.smithi121.stderr: 13: (C_IO_Wrapper::finish(int)+0x12) [0x56060ba7b212] 2023-11-15T03:00:01.439 INFO:tasks.ceph.mds.a.smithi121.stderr: 14: (MDSContext::complete(int)+0x61) [0x56060ba7a4b1] 2023-11-15T03:00:01.439 INFO:tasks.ceph.mds.a.smithi121.stderr: 15: (MDSIOContextBase::complete(int)+0x4fc) [0x56060ba7abdc] 2023-11-15T03:00:01.439 INFO:tasks.ceph.mds.a.smithi121.stderr: 16: (C_IO_Wrapper::complete(int)+0x105) [0x56060ba7b195] 2023-11-15T03:00:01.440 INFO:tasks.ceph.mds.a.smithi121.stderr: 17: (Finisher::finisher_thread_entry()+0x19d) [0x7f136811ca9d] 2023-11-15T03:00:01.440 INFO:tasks.ceph.mds.a.smithi121.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f1370bc1609] 2023-11-15T03:00:01.440 INFO:tasks.ceph.mds.a.smithi121.stderr: 19: clone() 2023-11-15T03:00:01.440 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:00:01.429+0000 7f135cbea700 -1 /build/ceph-16.2.14-170-g620f25c1/src/mds/CDentry.cc: In function 'bool CDentry::check_corruption(bool)' thread 7f135cbea700 time 2023-11-15T03:00:01.427855+0000 2023-11-15T03:00:01.440 INFO:tasks.ceph.mds.a.smithi121.stderr:/build/ceph-16.2.14-170-g620f25c1/src/mds/CDentry.cc: 699: ceph_abort_msg("abort() called") 2023-11-15T03:00:01.441 INFO:tasks.ceph.mds.a.smithi121.stderr: 2023-11-15T03:00:01.441 INFO:tasks.ceph.mds.a.smithi121.stderr: ceph version 16.2.14-170-g620f25c1 (620f25c1794f62207424fc9319427943fb8d1fae) pacific (stable) 2023-11-15T03:00:01.441 INFO:tasks.ceph.mds.a.smithi121.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe0) [0x7f13680684b5] 2023-11-15T03:00:01.441 INFO:tasks.ceph.mds.a.smithi121.stderr: 2: (CDentry::check_corruption(bool)+0x86b) [0x56060b9a0cbb] 2023-11-15T03:00:01.441 INFO:tasks.ceph.mds.a.smithi121.stderr: 3: ceph-mds(+0x48daaa) [0x56060b9c4aaa] 2023-11-15T03:00:01.442 INFO:tasks.ceph.mds.a.smithi121.stderr: 4: (CDir::_omap_commit(int)+0x318) [0x56060b9c51e8] 2023-11-15T03:00:01.442 INFO:tasks.ceph.mds.a.smithi121.stderr: 5: (CDir::_commit(unsigned long, int)+0x289) [0x56060b9c6839] 2023-11-15T03:00:01.442 INFO:tasks.ceph.mds.a.smithi121.stderr: 6: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x138) [0x56060b9c6eb8] 2023-11-15T03:00:01.442 INFO:tasks.ceph.mds.a.smithi121.stderr: 7: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x170f) [0x56060bb0ff9f] 2023-11-15T03:00:01.442 INFO:tasks.ceph.mds.a.smithi121.stderr: 8: (MDLog::try_expire(LogSegment*, int)+0x6e) [0x56060ba9901e] 2023-11-15T03:00:01.442 INFO:tasks.ceph.mds.a.smithi121.stderr: 9: (MDLog::trim_all()+0xa18) [0x56060ba9af48] 2023-11-15T03:00:01.443 INFO:tasks.ceph.mds.a.smithi121.stderr: 10: (LambdaContext::finish(int)+0x231) [0x56060b74d3c1] 2023-11-15T03:00:01.443 INFO:tasks.ceph.mds.a.smithi121.stderr: 11: (Context::complete(int)+0xd) [0x56060b70317d] 2023-11-15T03:00:01.443 INFO:tasks.ceph.mds.a.smithi121.stderr: 12: (MDSContext::complete(int)+0x61) [0x56060ba7a4b1] 2023-11-15T03:00:01.443 INFO:tasks.ceph.mds.a.smithi121.stderr: 13: (C_IO_Wrapper::finish(int)+0x12) [0x56060ba7b212] 2023-11-15T03:00:01.443 INFO:tasks.ceph.mds.a.smithi121.stderr: 14: (MDSContext::complete(int)+0x61) [0x56060ba7a4b1] 2023-11-15T03:00:01.444 INFO:tasks.ceph.mds.a.smithi121.stderr: 15: (MDSIOContextBase::complete(int)+0x4fc) [0x56060ba7abdc] 2023-11-15T03:00:01.444 INFO:tasks.ceph.mds.a.smithi121.stderr: 16: (C_IO_Wrapper::complete(int)+0x105) [0x56060ba7b195] 2023-11-15T03:00:01.444 INFO:tasks.ceph.mds.a.smithi121.stderr: 17: (Finisher::finisher_thread_entry()+0x19d) [0x7f136811ca9d] 2023-11-15T03:00:01.444 INFO:tasks.ceph.mds.a.smithi121.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f1370bc1609] 2023-11-15T03:00:01.444 INFO:tasks.ceph.mds.a.smithi121.stderr: 19: clone() 2023-11-15T03:00:01.445 INFO:tasks.ceph.mds.a.smithi121.stderr: 2023-11-15T03:00:01.445 INFO:tasks.ceph.mds.a.smithi121.stderr:*** Caught signal (Aborted) ** 2023-11-15T03:00:01.445 INFO:tasks.ceph.mds.a.smithi121.stderr: in thread 7f135cbea700 thread_name:MR_Finisher 2023-11-15T03:00:01.445 INFO:tasks.ceph.mds.a.smithi121.stderr: ceph version 16.2.14-170-g620f25c1 (620f25c1794f62207424fc9319427943fb8d1fae) pacific (stable) 2023-11-15T03:00:01.445 INFO:tasks.ceph.mds.a.smithi121.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f1370bcd420] 2023-11-15T03:00:01.446 INFO:tasks.ceph.mds.a.smithi121.stderr: 2: gsignal() 2023-11-15T03:00:01.446 INFO:tasks.ceph.mds.a.smithi121.stderr: 3: abort() 2023-11-15T03:00:01.446 INFO:tasks.ceph.mds.a.smithi121.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x1af) [0x7f1368068584] 2023-11-15T03:00:01.446 INFO:tasks.ceph.mds.a.smithi121.stderr: 5: (CDentry::check_corruption(bool)+0x86b) [0x56060b9a0cbb] 2023-11-15T03:00:01.447 INFO:tasks.ceph.mds.a.smithi121.stderr: 6: ceph-mds(+0x48daaa) [0x56060b9c4aaa] 2023-11-15T03:00:01.447 INFO:tasks.ceph.mds.a.smithi121.stderr: 7: (CDir::_omap_commit(int)+0x318) [0x56060b9c51e8] 2023-11-15T03:00:01.447 INFO:tasks.ceph.mds.a.smithi121.stderr: 8: (CDir::_commit(unsigned long, int)+0x289) [0x56060b9c6839] 2023-11-15T03:00:01.447 INFO:tasks.ceph.mds.a.smithi121.stderr: 9: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x138) [0x56060b9c6eb8] 2023-11-15T03:00:01.448 INFO:tasks.ceph.mds.a.smithi121.stderr: 10: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x170f) [0x56060bb0ff9f] 2023-11-15T03:00:01.448 INFO:tasks.ceph.mds.a.smithi121.stderr: 11: (MDLog::try_expire(LogSegment*, int)+0x6e) [0x56060ba9901e] 2023-11-15T03:00:01.448 INFO:tasks.ceph.mds.a.smithi121.stderr: 12: (MDLog::trim_all()+0xa18) [0x56060ba9af48] 2023-11-15T03:00:01.448 INFO:tasks.ceph.mds.a.smithi121.stderr: 13: (LambdaContext::finish(int)+0x231) [0x56060b74d3c1] 2023-11-15T03:00:01.448 INFO:tasks.ceph.mds.a.smithi121.stderr: 14: (Context::complete(int)+0xd) [0x56060b70317d] 2023-11-15T03:00:01.449 INFO:tasks.ceph.mds.a.smithi121.stderr: 15: (MDSContext::complete(int)+0x61) [0x56060ba7a4b1] 2023-11-15T03:00:01.449 INFO:tasks.ceph.mds.a.smithi121.stderr: 16: (C_IO_Wrapper::finish(int)+0x12) [0x56060ba7b212] 2023-11-15T03:00:01.449 INFO:tasks.ceph.mds.a.smithi121.stderr: 17: (MDSContext::complete(int)+0x61) [0x56060ba7a4b1] 2023-11-15T03:00:01.449 INFO:tasks.ceph.mds.a.smithi121.stderr: 18: (MDSIOContextBase::complete(int)+0x4fc) [0x56060ba7abdc] 2023-11-15T03:00:01.449 INFO:tasks.ceph.mds.a.smithi121.stderr: 19: (C_IO_Wrapper::complete(int)+0x105) [0x56060ba7b195] 2023-11-15T03:00:01.450 INFO:tasks.ceph.mds.a.smithi121.stderr: 20: (Finisher::finisher_thread_entry()+0x19d) [0x7f136811ca9d] 2023-11-15T03:00:01.450 INFO:tasks.ceph.mds.a.smithi121.stderr: 21: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f1370bc1609] 2023-11-15T03:00:01.450 INFO:tasks.ceph.mds.a.smithi121.stderr: 22: clone() 2023-11-15T03:00:01.450 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:00:01.429+0000 7f135cbea700 -1 *** Caught signal (Aborted) ** 2023-11-15T03:00:01.450 INFO:tasks.ceph.mds.a.smithi121.stderr: in thread 7f135cbea700 thread_name:MR_Finisher 2023-11-15T03:00:01.451 INFO:tasks.ceph.mds.a.smithi121.stderr: 2023-11-15T03:00:01.451 INFO:tasks.ceph.mds.a.smithi121.stderr: ceph version 16.2.14-170-g620f25c1 (620f25c1794f62207424fc9319427943fb8d1fae) pacific (stable) 2023-11-15T03:00:01.451 INFO:tasks.ceph.mds.a.smithi121.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f1370bcd420] 2023-11-15T03:00:01.451 INFO:tasks.ceph.mds.a.smithi121.stderr: 2: gsignal() 2023-11-15T03:00:01.451 INFO:tasks.ceph.mds.a.smithi121.stderr: 3: abort() 2023-11-15T03:00:01.452 INFO:tasks.ceph.mds.a.smithi121.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x1af) [0x7f1368068584] 2023-11-15T03:00:01.452 INFO:tasks.ceph.mds.a.smithi121.stderr: 5: (CDentry::check_corruption(bool)+0x86b) [0x56060b9a0cbb] 2023-11-15T03:00:01.452 INFO:tasks.ceph.mds.a.smithi121.stderr: 6: ceph-mds(+0x48daaa) [0x56060b9c4aaa] 2023-11-15T03:00:01.452 INFO:tasks.ceph.mds.a.smithi121.stderr: 7: (CDir::_omap_commit(int)+0x318) [0x56060b9c51e8] 2023-11-15T03:00:01.452 INFO:tasks.ceph.mds.a.smithi121.stderr: 8: (CDir::_commit(unsigned long, int)+0x289) [0x56060b9c6839] 2023-11-15T03:00:01.453 INFO:tasks.ceph.mds.a.smithi121.stderr: 9: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x138) [0x56060b9c6eb8] 2023-11-15T03:00:01.453 INFO:tasks.ceph.mds.a.smithi121.stderr: 10: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x170f) [0x56060bb0ff9f] 2023-11-15T03:00:01.453 INFO:tasks.ceph.mds.a.smithi121.stderr: 11: (MDLog::try_expire(LogSegment*, int)+0x6e) [0x56060ba9901e] 2023-11-15T03:00:01.453 INFO:tasks.ceph.mds.a.smithi121.stderr: 12: (MDLog::trim_all()+0xa18) [0x56060ba9af48] 2023-11-15T03:00:01.453 INFO:tasks.ceph.mds.a.smithi121.stderr: 13: (LambdaContext::finish(int)+0x231) [0x56060b74d3c1] 2023-11-15T03:00:01.454 INFO:tasks.ceph.mds.a.smithi121.stderr: 14: (Context::complete(int)+0xd) [0x56060b70317d] 2023-11-15T03:00:01.454 INFO:tasks.ceph.mds.a.smithi121.stderr: 15: (MDSContext::complete(int)+0x61) [0x56060ba7a4b1] 2023-11-15T03:00:01.454 INFO:tasks.ceph.mds.a.smithi121.stderr: 16: (C_IO_Wrapper::finish(int)+0x12) [0x56060ba7b212] 2023-11-15T03:00:01.454 INFO:tasks.ceph.mds.a.smithi121.stderr: 17: (MDSContext::complete(int)+0x61) [0x56060ba7a4b1] 2023-11-15T03:00:01.454 INFO:tasks.ceph.mds.a.smithi121.stderr: 18: (MDSIOContextBase::complete(int)+0x4fc) [0x56060ba7abdc] 2023-11-15T03:00:01.455 INFO:tasks.ceph.mds.a.smithi121.stderr: 19: (C_IO_Wrapper::complete(int)+0x105) [0x56060ba7b195] 2023-11-15T03:00:01.455 INFO:tasks.ceph.mds.a.smithi121.stderr: 20: (Finisher::finisher_thread_entry()+0x19d) [0x7f136811ca9d] 2023-11-15T03:00:01.455 INFO:tasks.ceph.mds.a.smithi121.stderr: 21: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f1370bc1609] 2023-11-15T03:00:01.455 INFO:tasks.ceph.mds.a.smithi121.stderr: 22: clone() 2023-11-15T03:00:01.455 INFO:tasks.ceph.mds.a.smithi121.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-15T03:00:01.455 INFO:tasks.ceph.mds.a.smithi121.stderr: 2023-11-15T03:00:01.457 INFO:tasks.ceph.mds.a.smithi121.stderr: -160> 2023-11-15T03:00:00.085+0000 7f13603f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:00:01.457 INFO:tasks.ceph.mds.a.smithi121.stderr: -6> 2023-11-15T03:00:01.425+0000 7f135cbea700 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x56060dd9b680] 2023-11-15T03:00:01.457 INFO:tasks.ceph.mds.a.smithi121.stderr: -4> 2023-11-15T03:00:01.425+0000 7f135cbea700 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x56060dd9b680] 2023-11-15T03:00:01.457 INFO:tasks.ceph.mds.a.smithi121.stderr: -2> 2023-11-15T03:00:01.429+0000 7f135cbea700 -1 /build/ceph-16.2.14-170-g620f25c1/src/mds/CDentry.cc: In function 'bool CDentry::check_corruption(bool)' thread 7f135cbea700 time 2023-11-15T03:00:01.427855+0000 2023-11-15T03:00:01.457 INFO:tasks.ceph.mds.a.smithi121.stderr:/build/ceph-16.2.14-170-g620f25c1/src/mds/CDentry.cc: 699: ceph_abort_msg("abort() called") 2023-11-15T03:00:01.458 INFO:tasks.ceph.mds.a.smithi121.stderr: 2023-11-15T03:00:01.458 INFO:tasks.ceph.mds.a.smithi121.stderr: ceph version 16.2.14-170-g620f25c1 (620f25c1794f62207424fc9319427943fb8d1fae) pacific (stable) 2023-11-15T03:00:01.458 INFO:tasks.ceph.mds.a.smithi121.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe0) [0x7f13680684b5] 2023-11-15T03:00:01.458 INFO:tasks.ceph.mds.a.smithi121.stderr: 2: (CDentry::check_corruption(bool)+0x86b) [0x56060b9a0cbb] 2023-11-15T03:00:01.458 INFO:tasks.ceph.mds.a.smithi121.stderr: 3: ceph-mds(+0x48daaa) [0x56060b9c4aaa] 2023-11-15T03:00:01.459 INFO:tasks.ceph.mds.a.smithi121.stderr: 4: (CDir::_omap_commit(int)+0x318) [0x56060b9c51e8] 2023-11-15T03:00:01.459 INFO:tasks.ceph.mds.a.smithi121.stderr: 5: (CDir::_commit(unsigned long, int)+0x289) [0x56060b9c6839] 2023-11-15T03:00:01.459 INFO:tasks.ceph.mds.a.smithi121.stderr: 6: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x138) [0x56060b9c6eb8] 2023-11-15T03:00:01.459 INFO:tasks.ceph.mds.a.smithi121.stderr: 7: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x170f) [0x56060bb0ff9f] 2023-11-15T03:00:01.459 INFO:tasks.ceph.mds.a.smithi121.stderr: 8: (MDLog::try_expire(LogSegment*, int)+0x6e) [0x56060ba9901e] 2023-11-15T03:00:01.460 INFO:tasks.ceph.mds.a.smithi121.stderr: 9: (MDLog::trim_all()+0xa18) [0x56060ba9af48] 2023-11-15T03:00:01.460 INFO:tasks.ceph.mds.a.smithi121.stderr: 10: (LambdaContext::finish(int)+0x231) [0x56060b74d3c1] 2023-11-15T03:00:01.460 INFO:tasks.ceph.mds.a.smithi121.stderr: 11: (Context::complete(int)+0xd) [0x56060b70317d] 2023-11-15T03:00:01.460 INFO:tasks.ceph.mds.a.smithi121.stderr: 12: (MDSContext::complete(int)+0x61) [0x56060ba7a4b1] 2023-11-15T03:00:01.460 INFO:tasks.ceph.mds.a.smithi121.stderr: 13: (C_IO_Wrapper::finish(int)+0x12) [0x56060ba7b212] 2023-11-15T03:00:01.461 INFO:tasks.ceph.mds.a.smithi121.stderr: 14: (MDSContext::complete(int)+0x61) [0x56060ba7a4b1] 2023-11-15T03:00:01.461 INFO:tasks.ceph.mds.a.smithi121.stderr: 15: (MDSIOContextBase::complete(int)+0x4fc) [0x56060ba7abdc] 2023-11-15T03:00:01.461 INFO:tasks.ceph.mds.a.smithi121.stderr: 16: (C_IO_Wrapper::complete(int)+0x105) [0x56060ba7b195] 2023-11-15T03:00:01.461 INFO:tasks.ceph.mds.a.smithi121.stderr: 17: (Finisher::finisher_thread_entry()+0x19d) [0x7f136811ca9d] 2023-11-15T03:00:01.461 INFO:tasks.ceph.mds.a.smithi121.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f1370bc1609] 2023-11-15T03:00:01.462 INFO:tasks.ceph.mds.a.smithi121.stderr: 19: clone() 2023-11-15T03:00:01.462 INFO:tasks.ceph.mds.a.smithi121.stderr: 2023-11-15T03:00:01.462 INFO:tasks.ceph.mds.a.smithi121.stderr: 0> 2023-11-15T03:00:01.429+0000 7f135cbea700 -1 *** Caught signal (Aborted) ** 2023-11-15T03:00:01.462 INFO:tasks.ceph.mds.a.smithi121.stderr: in thread 7f135cbea700 thread_name:MR_Finisher 2023-11-15T03:00:01.462 INFO:tasks.ceph.mds.a.smithi121.stderr: 2023-11-15T03:00:01.462 INFO:tasks.ceph.mds.a.smithi121.stderr: ceph version 16.2.14-170-g620f25c1 (620f25c1794f62207424fc9319427943fb8d1fae) pacific (stable) 2023-11-15T03:00:01.463 INFO:tasks.ceph.mds.a.smithi121.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f1370bcd420] 2023-11-15T03:00:01.463 INFO:tasks.ceph.mds.a.smithi121.stderr: 2: gsignal() 2023-11-15T03:00:01.463 INFO:tasks.ceph.mds.a.smithi121.stderr: 3: abort() 2023-11-15T03:00:01.463 INFO:tasks.ceph.mds.a.smithi121.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x1af) [0x7f1368068584] 2023-11-15T03:00:01.463 INFO:tasks.ceph.mds.a.smithi121.stderr: 5: (CDentry::check_corruption(bool)+0x86b) [0x56060b9a0cbb] 2023-11-15T03:00:01.464 INFO:tasks.ceph.mds.a.smithi121.stderr: 6: ceph-mds(+0x48daaa) [0x56060b9c4aaa] 2023-11-15T03:00:01.464 INFO:tasks.ceph.mds.a.smithi121.stderr: 7: (CDir::_omap_commit(int)+0x318) [0x56060b9c51e8] 2023-11-15T03:00:01.464 INFO:tasks.ceph.mds.a.smithi121.stderr: 8: (CDir::_commit(unsigned long, int)+0x289) [0x56060b9c6839] 2023-11-15T03:00:01.464 INFO:tasks.ceph.mds.a.smithi121.stderr: 9: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x138) [0x56060b9c6eb8] 2023-11-15T03:00:01.464 INFO:tasks.ceph.mds.a.smithi121.stderr: 10: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x170f) [0x56060bb0ff9f] 2023-11-15T03:00:01.465 INFO:tasks.ceph.mds.a.smithi121.stderr: 11: (MDLog::try_expire(LogSegment*, int)+0x6e) [0x56060ba9901e] 2023-11-15T03:00:01.465 INFO:tasks.ceph.mds.a.smithi121.stderr: 12: (MDLog::trim_all()+0xa18) [0x56060ba9af48] 2023-11-15T03:00:01.465 INFO:tasks.ceph.mds.a.smithi121.stderr: 13: (LambdaContext::finish(int)+0x231) [0x56060b74d3c1] 2023-11-15T03:00:01.465 INFO:tasks.ceph.mds.a.smithi121.stderr: 14: (Context::complete(int)+0xd) [0x56060b70317d] 2023-11-15T03:00:01.465 INFO:tasks.ceph.mds.a.smithi121.stderr: 15: (MDSContext::complete(int)+0x61) [0x56060ba7a4b1] 2023-11-15T03:00:01.466 INFO:tasks.ceph.mds.a.smithi121.stderr: 16: (C_IO_Wrapper::finish(int)+0x12) [0x56060ba7b212] 2023-11-15T03:00:01.466 INFO:tasks.ceph.mds.a.smithi121.stderr: 17: (MDSContext::complete(int)+0x61) [0x56060ba7a4b1] 2023-11-15T03:00:01.466 INFO:tasks.ceph.mds.a.smithi121.stderr: 18: (MDSIOContextBase::complete(int)+0x4fc) [0x56060ba7abdc] 2023-11-15T03:00:01.466 INFO:tasks.ceph.mds.a.smithi121.stderr: 19: (C_IO_Wrapper::complete(int)+0x105) [0x56060ba7b195] 2023-11-15T03:00:01.466 INFO:tasks.ceph.mds.a.smithi121.stderr: 20: (Finisher::finisher_thread_entry()+0x19d) [0x7f136811ca9d] 2023-11-15T03:00:01.466 INFO:tasks.ceph.mds.a.smithi121.stderr: 21: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f1370bc1609] 2023-11-15T03:00:01.467 INFO:tasks.ceph.mds.a.smithi121.stderr: 22: clone() 2023-11-15T03:00:01.467 INFO:tasks.ceph.mds.a.smithi121.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-15T03:00:01.467 INFO:tasks.ceph.mds.a.smithi121.stderr: 2023-11-15T03:00:01.467 INFO:tasks.ceph.mds.a.smithi121.stderr: -160> 2023-11-15T03:00:00.085+0000 7f13603f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:00:01.467 INFO:tasks.ceph.mds.a.smithi121.stderr: -6> 2023-11-15T03:00:01.425+0000 7f135cbea700 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x56060dd9b680] 2023-11-15T03:00:01.468 INFO:tasks.ceph.mds.a.smithi121.stderr: -4> 2023-11-15T03:00:01.425+0000 7f135cbea700 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x56060dd9b680] 2023-11-15T03:00:01.468 INFO:tasks.ceph.mds.a.smithi121.stderr: -2> 2023-11-15T03:00:01.429+0000 7f135cbea700 -1 /build/ceph-16.2.14-170-g620f25c1/src/mds/CDentry.cc: In function 'bool CDentry::check_corruption(bool)' thread 7f135cbea700 time 2023-11-15T03:00:01.427855+0000 2023-11-15T03:00:01.468 INFO:tasks.ceph.mds.a.smithi121.stderr:/build/ceph-16.2.14-170-g620f25c1/src/mds/CDentry.cc: 699: ceph_abort_msg("abort() called") 2023-11-15T03:00:01.468 INFO:tasks.ceph.mds.a.smithi121.stderr: 2023-11-15T03:00:01.468 INFO:tasks.ceph.mds.a.smithi121.stderr: ceph version 16.2.14-170-g620f25c1 (620f25c1794f62207424fc9319427943fb8d1fae) pacific (stable) 2023-11-15T03:00:01.469 INFO:tasks.ceph.mds.a.smithi121.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe0) [0x7f13680684b5] 2023-11-15T03:00:01.469 INFO:tasks.ceph.mds.a.smithi121.stderr: 2: (CDentry::check_corruption(bool)+0x86b) [0x56060b9a0cbb] 2023-11-15T03:00:01.469 INFO:tasks.ceph.mds.a.smithi121.stderr: 3: ceph-mds(+0x48daaa) [0x56060b9c4aaa] 2023-11-15T03:00:01.469 INFO:tasks.ceph.mds.a.smithi121.stderr: 4: (CDir::_omap_commit(int)+0x318) [0x56060b9c51e8] 2023-11-15T03:00:01.469 INFO:tasks.ceph.mds.a.smithi121.stderr: 5: (CDir::_commit(unsigned long, int)+0x289) [0x56060b9c6839] 2023-11-15T03:00:01.470 INFO:tasks.ceph.mds.a.smithi121.stderr: 6: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x138) [0x56060b9c6eb8] 2023-11-15T03:00:01.470 INFO:tasks.ceph.mds.a.smithi121.stderr: 7: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x170f) [0x56060bb0ff9f] 2023-11-15T03:00:01.470 INFO:tasks.ceph.mds.a.smithi121.stderr: 8: (MDLog::try_expire(LogSegment*, int)+0x6e) [0x56060ba9901e] 2023-11-15T03:00:01.470 INFO:tasks.ceph.mds.a.smithi121.stderr: 9: (MDLog::trim_all()+0xa18) [0x56060ba9af48] 2023-11-15T03:00:01.470 INFO:tasks.ceph.mds.a.smithi121.stderr: 10: (LambdaContext::finish(int)+0x231) [0x56060b74d3c1] 2023-11-15T03:00:01.471 INFO:tasks.ceph.mds.a.smithi121.stderr: 11: (Context::complete(int)+0xd) [0x56060b70317d] 2023-11-15T03:00:01.471 INFO:tasks.ceph.mds.a.smithi121.stderr: 12: (MDSContext::complete(int)+0x61) [0x56060ba7a4b1] 2023-11-15T03:00:01.471 INFO:tasks.ceph.mds.a.smithi121.stderr: 13: (C_IO_Wrapper::finish(int)+0x12) [0x56060ba7b212] 2023-11-15T03:00:01.471 INFO:tasks.ceph.mds.a.smithi121.stderr: 14: (MDSContext::complete(int)+0x61) [0x56060ba7a4b1] 2023-11-15T03:00:01.471 INFO:tasks.ceph.mds.a.smithi121.stderr: 15: (MDSIOContextBase::complete(int)+0x4fc) [0x56060ba7abdc] 2023-11-15T03:00:01.471 INFO:tasks.ceph.mds.a.smithi121.stderr: 16: (C_IO_Wrapper::complete(int)+0x105) [0x56060ba7b195] 2023-11-15T03:00:01.472 INFO:tasks.ceph.mds.a.smithi121.stderr: 17: (Finisher::finisher_thread_entry()+0x19d) [0x7f136811ca9d] 2023-11-15T03:00:01.472 INFO:tasks.ceph.mds.a.smithi121.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f1370bc1609] 2023-11-15T03:00:01.472 INFO:tasks.ceph.mds.a.smithi121.stderr: 19: clone() 2023-11-15T03:00:01.472 INFO:tasks.ceph.mds.a.smithi121.stderr: 2023-11-15T03:00:01.472 INFO:tasks.ceph.mds.a.smithi121.stderr: 0> 2023-11-15T03:00:01.429+0000 7f135cbea700 -1 *** Caught signal (Aborted) ** 2023-11-15T03:00:01.473 INFO:tasks.ceph.mds.a.smithi121.stderr: in thread 7f135cbea700 thread_name:MR_Finisher 2023-11-15T03:00:01.473 INFO:tasks.ceph.mds.a.smithi121.stderr: 2023-11-15T03:00:01.473 INFO:tasks.ceph.mds.a.smithi121.stderr: ceph version 16.2.14-170-g620f25c1 (620f25c1794f62207424fc9319427943fb8d1fae) pacific (stable) 2023-11-15T03:00:01.473 INFO:tasks.ceph.mds.a.smithi121.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f1370bcd420] 2023-11-15T03:00:01.473 INFO:tasks.ceph.mds.a.smithi121.stderr: 2: gsignal() 2023-11-15T03:00:01.474 INFO:tasks.ceph.mds.a.smithi121.stderr: 3: abort() 2023-11-15T03:00:01.474 INFO:tasks.ceph.mds.a.smithi121.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x1af) [0x7f1368068584] 2023-11-15T03:00:01.474 INFO:tasks.ceph.mds.a.smithi121.stderr: 5: (CDentry::check_corruption(bool)+0x86b) [0x56060b9a0cbb] 2023-11-15T03:00:01.474 INFO:tasks.ceph.mds.a.smithi121.stderr: 6: ceph-mds(+0x48daaa) [0x56060b9c4aaa] 2023-11-15T03:00:01.474 INFO:tasks.ceph.mds.a.smithi121.stderr: 7: (CDir::_omap_commit(int)+0x318) [0x56060b9c51e8] 2023-11-15T03:00:01.475 INFO:tasks.ceph.mds.a.smithi121.stderr: 8: (CDir::_commit(unsigned long, int)+0x289) [0x56060b9c6839] 2023-11-15T03:00:01.475 INFO:tasks.ceph.mds.a.smithi121.stderr: 9: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x138) [0x56060b9c6eb8] 2023-11-15T03:00:01.475 INFO:tasks.ceph.mds.a.smithi121.stderr: 10: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x170f) [0x56060bb0ff9f] 2023-11-15T03:00:01.475 INFO:tasks.ceph.mds.a.smithi121.stderr: 11: (MDLog::try_expire(LogSegment*, int)+0x6e) [0x56060ba9901e] 2023-11-15T03:00:01.475 INFO:tasks.ceph.mds.a.smithi121.stderr: 12: (MDLog::trim_all()+0xa18) [0x56060ba9af48] 2023-11-15T03:00:01.476 INFO:tasks.ceph.mds.a.smithi121.stderr: 13: (LambdaContext::finish(int)+0x231) [0x56060b74d3c1] 2023-11-15T03:00:01.476 INFO:tasks.ceph.mds.a.smithi121.stderr: 14: (Context::complete(int)+0xd) [0x56060b70317d] 2023-11-15T03:00:01.476 INFO:tasks.ceph.mds.a.smithi121.stderr: 15: (MDSContext::complete(int)+0x61) [0x56060ba7a4b1] 2023-11-15T03:00:01.476 INFO:tasks.ceph.mds.a.smithi121.stderr: 16: (C_IO_Wrapper::finish(int)+0x12) [0x56060ba7b212] 2023-11-15T03:00:01.476 INFO:tasks.ceph.mds.a.smithi121.stderr: 17: (MDSContext::complete(int)+0x61) [0x56060ba7a4b1] 2023-11-15T03:00:01.476 INFO:tasks.ceph.mds.a.smithi121.stderr: 18: (MDSIOContextBase::complete(int)+0x4fc) [0x56060ba7abdc] 2023-11-15T03:00:01.477 INFO:tasks.ceph.mds.a.smithi121.stderr: 19: (C_IO_Wrapper::complete(int)+0x105) [0x56060ba7b195] 2023-11-15T03:00:01.477 INFO:tasks.ceph.mds.a.smithi121.stderr: 20: (Finisher::finisher_thread_entry()+0x19d) [0x7f136811ca9d] 2023-11-15T03:00:01.477 INFO:tasks.ceph.mds.a.smithi121.stderr: 21: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f1370bc1609] 2023-11-15T03:00:01.477 INFO:tasks.ceph.mds.a.smithi121.stderr: 22: clone() 2023-11-15T03:00:01.477 INFO:tasks.ceph.mds.a.smithi121.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-15T03:00:01.478 INFO:tasks.ceph.mds.a.smithi121.stderr: 2023-11-15T03:00:01.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.533+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2023-11-15T03:00:01.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.533+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T03:00:01.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.533+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T03:00:01.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.533+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0)._fault initiating reconnect 2023-11-15T03:00:01.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.533+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk reading from fd=18 : -104 Unknown error -104 2023-11-15T03:00:01.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.533+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T03:00:01.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.533+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=1 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner read peer banner failed r=-1 ((1) Operation not permitted) 2023-11-15T03:00:01.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.533+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=START_CONNECT pgs=15 cs=2 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-11-15T03:00:01.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.569+0000 7f505b59e700 1 Processor -- start 2023-11-15T03:00:01.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.569+0000 7f505b59e700 1 -- start start 2023-11-15T03:00:01.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f505b59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f505c0cabd0 0x7f505c0cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f505b59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f505c0acc90 0x7f505c066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f505b59e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505c0ad650 con 0x7f505c0ad0f0 2023-11-15T03:00:01.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f505b59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c065fc0 con 0x7f505c0acc90 2023-11-15T03:00:01.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f505b59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0662a0 con 0x7f505c0cabd0 2023-11-15T03:00:01.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f505ad9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f505c0ad0f0 0x7f505c0ae190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45596/0 (socket says 172.21.15.121:45596) 2023-11-15T03:00:01.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f505ad9d700 1 -- 172.21.15.121:0/2693219684 learned_addr learned my addr 172.21.15.121:0/2693219684 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:00:01.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f5059d9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f505c0acc90 0x7f505c066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f505a59c700 1 --2- 172.21.15.121:0/2693219684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f505c0cabd0 0x7f505c0cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f505959a700 1 -- 172.21.15.121:0/2693219684 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807946241 0 0) 0x7f505c0ad650 con 0x7f505c0ad0f0 2023-11-15T03:00:01.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f5059d9b700 1 -- 172.21.15.121:0/2693219684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f505c0cabd0 msgr2=0x7f505c0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:01.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f5059d9b700 1 --2- 172.21.15.121:0/2693219684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f505c0cabd0 0x7f505c0cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f5059d9b700 1 -- 172.21.15.121:0/2693219684 >> v1:172.21.15.121:6789/0 conn(0x7f505c0ad0f0 legacy=0x7f505c0ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:01.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f5059d9b700 1 -- 172.21.15.121:0/2693219684 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c066580 con 0x7f505c0acc90 2023-11-15T03:00:01.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f5059d9b700 1 --2- 172.21.15.121:0/2693219684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f505c0acc90 0x7f505c066d80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f5050009790 tx=0x7f5050009060).ready entity=mon.0 client_cookie=e7b9996bf52c5d0c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:01.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f505959a700 1 -- 172.21.15.121:0/2693219684 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505000f510 con 0x7f505c0acc90 2023-11-15T03:00:01.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f505959a700 1 -- 172.21.15.121:0/2693219684 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505000f670 con 0x7f505c0acc90 2023-11-15T03:00:01.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f505959a700 1 -- 172.21.15.121:0/2693219684 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505000f9f0 con 0x7f505c0acc90 2023-11-15T03:00:01.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f505b59e700 1 -- 172.21.15.121:0/2693219684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f505c0acc90 msgr2=0x7f505c066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:01.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.573+0000 7f505b59e700 1 --2- 172.21.15.121:0/2693219684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f505c0acc90 0x7f505c066d80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f5050009790 tx=0x7f5050009060).stop 2023-11-15T03:00:01.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.577+0000 7f505b59e700 1 -- 172.21.15.121:0/2693219684 shutdown_connections 2023-11-15T03:00:01.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.577+0000 7f505b59e700 1 --2- 172.21.15.121:0/2693219684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f505c0acc90 0x7f505c066d80 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.577+0000 7f505b59e700 1 --2- 172.21.15.121:0/2693219684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f505c0cabd0 0x7f505c0cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.577+0000 7f505b59e700 1 -- 172.21.15.121:0/2693219684 >> 172.21.15.121:0/2693219684 conn(0x7f505c062190 msgr2=0x7f505c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:01.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.577+0000 7f505b59e700 1 -- 172.21.15.121:0/2693219684 shutdown_connections 2023-11-15T03:00:01.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.577+0000 7f505b59e700 1 -- 172.21.15.121:0/2693219684 wait complete. 2023-11-15T03:00:01.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.577+0000 7f505b59e700 1 Processor -- start 2023-11-15T03:00:01.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.577+0000 7f505b59e700 1 -- start start 2023-11-15T03:00:01.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.577+0000 7f505b59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f505c0acc90 0x7f505c0c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.577+0000 7f505b59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f505c0ad0f0 0x7f505c14f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.577+0000 7f505b59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f505c0cabd0 0x7f505c0bfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.577+0000 7f505b59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c168510 con 0x7f505c0ad0f0 2023-11-15T03:00:01.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.577+0000 7f505b59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c167970 con 0x7f505c0cabd0 2023-11-15T03:00:01.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.577+0000 7f505b59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c167be0 con 0x7f505c0acc90 2023-11-15T03:00:01.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.577+0000 7f5059d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f505c0ad0f0 0x7f505c14f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.577+0000 7f5059d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f505c0ad0f0 0x7f505c14f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50192/0 (socket says 172.21.15.121:50192) 2023-11-15T03:00:01.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.577+0000 7f5059d9b700 1 -- 172.21.15.121:0/3252948870 learned_addr learned my addr 172.21.15.121:0/3252948870 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:01.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.577+0000 7f505a59c700 1 --2- 172.21.15.121:0/3252948870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f505c0acc90 0x7f505c0c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.577+0000 7f505ad9d700 1 --2- 172.21.15.121:0/3252948870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f505c0cabd0 0x7f505c0bfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.581+0000 7f5059d9b700 1 -- 172.21.15.121:0/3252948870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f505c0acc90 msgr2=0x7f505c0c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:01.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.581+0000 7f5059d9b700 1 --2- 172.21.15.121:0/3252948870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f505c0acc90 0x7f505c0c26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.581+0000 7f5059d9b700 1 -- 172.21.15.121:0/3252948870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f505c0cabd0 msgr2=0x7f505c0bfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:01.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.581+0000 7f5059d9b700 1 --2- 172.21.15.121:0/3252948870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f505c0cabd0 0x7f505c0bfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.581+0000 7f5059d9b700 1 -- 172.21.15.121:0/3252948870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50500092a0 con 0x7f505c0ad0f0 2023-11-15T03:00:01.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.581+0000 7f505a59c700 1 --2- 172.21.15.121:0/3252948870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f505c0acc90 0x7f505c0c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:00:01.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.581+0000 7f5059d9b700 1 --2- 172.21.15.121:0/3252948870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f505c0ad0f0 0x7f505c14f350 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f5050009e30 tx=0x7f5050017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:01.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.581+0000 7f504a7fc700 1 -- 172.21.15.121:0/3252948870 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505000f9f0 con 0x7f505c0ad0f0 2023-11-15T03:00:01.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.581+0000 7f504a7fc700 1 -- 172.21.15.121:0/3252948870 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5050000c90 con 0x7f505c0ad0f0 2023-11-15T03:00:01.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.581+0000 7f505b59e700 1 -- 172.21.15.121:0/3252948870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505c167ec0 con 0x7f505c0ad0f0 2023-11-15T03:00:01.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.581+0000 7f504a7fc700 1 -- 172.21.15.121:0/3252948870 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505001b660 con 0x7f505c0ad0f0 2023-11-15T03:00:01.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.581+0000 7f505b59e700 1 -- 172.21.15.121:0/3252948870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c14f840 con 0x7f505c0ad0f0 2023-11-15T03:00:01.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.581+0000 7f505b59e700 1 -- 172.21.15.121:0/3252948870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505c168680 con 0x7f505c0ad0f0 2023-11-15T03:00:01.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.581+0000 7f504a7fc700 1 -- 172.21.15.121:0/3252948870 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f505001ba10 con 0x7f505c0ad0f0 2023-11-15T03:00:01.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.585+0000 7f504a7fc700 1 --2- 172.21.15.121:0/3252948870 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f503c05a540 0x7f503c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:01.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.585+0000 7f504a7fc700 1 -- 172.21.15.121:0/3252948870 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f5050076470 con 0x7f505c0ad0f0 2023-11-15T03:00:01.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.585+0000 7f505a59c700 1 --2- 172.21.15.121:0/3252948870 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f503c05a540 0x7f503c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:01.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.585+0000 7f505a59c700 1 --2- 172.21.15.121:0/3252948870 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f503c05a540 0x7f503c05ca00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f504c009b60 tx=0x7f504c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:01.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.589+0000 7f504a7fc700 1 -- 172.21.15.121:0/3252948870 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5050049b40 con 0x7f505c0ad0f0 2023-11-15T03:00:01.693 INFO:tasks.ceph.mds.a.smithi121.stderr:daemon-helper: command crashed with signal 6 2023-11-15T03:00:01.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.737+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.121:6835/3429169582 2023-11-15T03:00:01.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.737+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=START_CONNECT pgs=15 cs=3 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2023-11-15T03:00:01.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.737+0000 7f71d4ff9700 1 client.5633 ms_handle_refused on v2:172.21.15.121:6835/3429169582 2023-11-15T03:00:01.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.793+0000 7f505b59e700 1 -- 172.21.15.121:0/3252948870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f505c066860 con 0x7f505c0ad0f0 2023-11-15T03:00:01.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.793+0000 7f504a7fc700 1 -- 172.21.15.121:0/3252948870 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4503 (secure 0 0 0) 0x7f505004d160 con 0x7f505c0ad0f0 2023-11-15T03:00:01.798 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:00:01.798 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":53,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3462858472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3462858472},{"type":"v1","addr":"172.21.15.190:6833","nonce":3462858472}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4042390087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4042390087},{"type":"v1","addr":"172.21.15.190:6835","nonce":4042390087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5907,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2239114529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2239114529},{"type":"v1","addr":"172.21.15.121:6837","nonce":2239114529}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:59:34.667864+0000","modified":"2023-11-15T03:00:01.157656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":5,"addr":"172.21.15.121:6836/3429169582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3429169582},{"type":"v1","addr":"172.21.15.121:6836","nonce":3429169582}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-15T03:00:01.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.797+0000 7f505b59e700 1 -- 172.21.15.121:0/3252948870 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f503c05a540 msgr2=0x7f503c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:01.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.797+0000 7f505b59e700 1 --2- 172.21.15.121:0/3252948870 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f503c05a540 0x7f503c05ca00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f504c009b60 tx=0x7f504c008040).stop 2023-11-15T03:00:01.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.797+0000 7f505b59e700 1 -- 172.21.15.121:0/3252948870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f505c0ad0f0 msgr2=0x7f505c14f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:01.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.797+0000 7f505b59e700 1 --2- 172.21.15.121:0/3252948870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f505c0ad0f0 0x7f505c14f350 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f5050009e30 tx=0x7f5050017000).stop 2023-11-15T03:00:01.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.797+0000 7f505b59e700 1 -- 172.21.15.121:0/3252948870 shutdown_connections 2023-11-15T03:00:01.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.797+0000 7f505b59e700 1 --2- 172.21.15.121:0/3252948870 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f503c05a540 0x7f503c05ca00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.797+0000 7f505b59e700 1 --2- 172.21.15.121:0/3252948870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f505c0cabd0 0x7f505c0bfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.797+0000 7f505b59e700 1 --2- 172.21.15.121:0/3252948870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f505c0ad0f0 0x7f505c14f350 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.797+0000 7f505b59e700 1 --2- 172.21.15.121:0/3252948870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f505c0acc90 0x7f505c0c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:01.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.797+0000 7f505b59e700 1 -- 172.21.15.121:0/3252948870 >> 172.21.15.121:0/3252948870 conn(0x7f505c062190 msgr2=0x7f505c151700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:01.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.797+0000 7f505b59e700 1 -- 172.21.15.121:0/3252948870 shutdown_connections 2023-11-15T03:00:01.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:01.797+0000 7f505b59e700 1 -- 172.21.15.121:0/3252948870 wait complete. 2023-11-15T03:00:01.804 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 53 2023-11-15T03:00:01.820 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-15T03:00:02.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:02.141+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.121:6835/3429169582 2023-11-15T03:00:02.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:02.141+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=START_CONNECT pgs=15 cs=4 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2023-11-15T03:00:02.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:02.141+0000 7f71d4ff9700 1 client.5633 ms_handle_refused on v2:172.21.15.121:6835/3429169582 2023-11-15T03:00:02.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:02.945+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.121:6835/3429169582 2023-11-15T03:00:02.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:02.945+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=START_CONNECT pgs=15 cs=5 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2023-11-15T03:00:02.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:02.945+0000 7f71d4ff9700 1 client.5633 ms_handle_refused on v2:172.21.15.121:6835/3429169582 2023-11-15T03:00:04.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:04.549+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.121:6835/3429169582 2023-11-15T03:00:04.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:04.549+0000 7f71d4ff9700 1 client.5633 ms_handle_refused on v2:172.21.15.121:6835/3429169582 2023-11-15T03:00:04.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:04.549+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=START_CONNECT pgs=15 cs=6 l=0 rev1=1 rx=0 tx=0)._fault waiting 3.200000 2023-11-15T03:00:05.884 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-11-15T03:00:06.822 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:00:06.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.981+0000 7f861bddb700 1 Processor -- start 2023-11-15T03:00:06.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.981+0000 7f861bddb700 1 -- start start 2023-11-15T03:00:06.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.985+0000 7f861bddb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8614153bb0 0x7f86140a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:06.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.985+0000 7f861bddb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86140aa020 0x7f8614067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:06.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.985+0000 7f861bddb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86140cb5c0 con 0x7f8614067f50 2023-11-15T03:00:06.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.985+0000 7f861bddb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86140aa6f0 con 0x7f8614153bb0 2023-11-15T03:00:06.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.985+0000 7f861bddb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86140c5900 con 0x7f86140aa020 2023-11-15T03:00:06.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.985+0000 7f8619b77700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8614153bb0 0x7f86140a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:06.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.985+0000 7f8619b77700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8614153bb0 0x7f86140a9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:43832/0 (socket says 172.21.15.121:43832) 2023-11-15T03:00:06.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.985+0000 7f8619b77700 1 -- 172.21.15.121:0/2430789984 learned_addr learned my addr 172.21.15.121:0/2430789984 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:06.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.985+0000 7f8619376700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86140aa020 0x7f8614067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:06.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.985+0000 7f861a378700 1 --1- 172.21.15.121:0/2430789984 >> v1:172.21.15.190:6789/0 conn(0x7f8614067f50 0x7f86141578a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:40842/0 (socket says 172.21.15.121:40842) 2023-11-15T03:00:06.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.985+0000 7f8619b77700 1 -- 172.21.15.121:0/2430789984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86140aa020 msgr2=0x7f8614067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:06.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.985+0000 7f8619b77700 1 --2- 172.21.15.121:0/2430789984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86140aa020 0x7f8614067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:06.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.985+0000 7f8619b77700 1 -- 172.21.15.121:0/2430789984 >> v1:172.21.15.190:6789/0 conn(0x7f8614067f50 legacy=0x7f86141578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:06.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.985+0000 7f8619b77700 1 -- 172.21.15.121:0/2430789984 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86140683e0 con 0x7f8614153bb0 2023-11-15T03:00:06.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.985+0000 7f8619b77700 1 --2- 172.21.15.121:0/2430789984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8614153bb0 0x7f86140a9b30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f8610006fd0 tx=0x7f86100090f0).ready entity=mon.1 client_cookie=1f9519e56d5a4ee3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:06.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.985+0000 7f8618b75700 1 -- 172.21.15.121:0/2430789984 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861000b580 con 0x7f8614153bb0 2023-11-15T03:00:06.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.985+0000 7f8618b75700 1 -- 172.21.15.121:0/2430789984 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861000b6e0 con 0x7f8614153bb0 2023-11-15T03:00:06.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.985+0000 7f8618b75700 1 -- 172.21.15.121:0/2430789984 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861000ba60 con 0x7f8614153bb0 2023-11-15T03:00:06.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.989+0000 7f861bddb700 1 -- 172.21.15.121:0/2430789984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8614153bb0 msgr2=0x7f86140a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:06.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.989+0000 7f861bddb700 1 --2- 172.21.15.121:0/2430789984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8614153bb0 0x7f86140a9b30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f8610006fd0 tx=0x7f86100090f0).stop 2023-11-15T03:00:06.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.989+0000 7f861bddb700 1 -- 172.21.15.121:0/2430789984 shutdown_connections 2023-11-15T03:00:06.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.989+0000 7f861bddb700 1 --2- 172.21.15.121:0/2430789984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86140aa020 0x7f8614067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:06.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.989+0000 7f861bddb700 1 --2- 172.21.15.121:0/2430789984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8614153bb0 0x7f86140a9b30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:06.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.989+0000 7f861bddb700 1 -- 172.21.15.121:0/2430789984 >> 172.21.15.121:0/2430789984 conn(0x7f8614058c50 msgr2=0x7f8614059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:06.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.989+0000 7f861bddb700 1 -- 172.21.15.121:0/2430789984 shutdown_connections 2023-11-15T03:00:06.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.989+0000 7f861bddb700 1 -- 172.21.15.121:0/2430789984 wait complete. 2023-11-15T03:00:06.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.989+0000 7f861bddb700 1 Processor -- start 2023-11-15T03:00:06.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.989+0000 7f861bddb700 1 -- start start 2023-11-15T03:00:06.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f861bddb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8614067f50 0x7f86140aa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:06.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f861bddb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86140aa020 0x7f861414e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:06.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f861bddb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8614153bb0 0x7f86140abf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:06.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f861bddb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8614154770 con 0x7f86140aa020 2023-11-15T03:00:06.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f861bddb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86140ab120 con 0x7f8614067f50 2023-11-15T03:00:06.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f861bddb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86140ab400 con 0x7f8614153bb0 2023-11-15T03:00:06.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f8619376700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86140aa020 0x7f861414e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:06.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f8619376700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86140aa020 0x7f861414e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48932/0 (socket says 172.21.15.121:48932) 2023-11-15T03:00:06.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f8619376700 1 -- 172.21.15.121:0/4255272777 learned_addr learned my addr 172.21.15.121:0/4255272777 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:06.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f8619b77700 1 --2- 172.21.15.121:0/4255272777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8614067f50 0x7f86140aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:06.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f861a378700 1 --2- 172.21.15.121:0/4255272777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8614153bb0 0x7f86140abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:06.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f8619376700 1 -- 172.21.15.121:0/4255272777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8614153bb0 msgr2=0x7f86140abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:06.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f8619376700 1 --2- 172.21.15.121:0/4255272777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8614153bb0 0x7f86140abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:07.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f8619376700 1 -- 172.21.15.121:0/4255272777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8614067f50 msgr2=0x7f86140aa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:07.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f8619376700 1 --2- 172.21.15.121:0/4255272777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8614067f50 0x7f86140aa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:07.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f8619376700 1 -- 172.21.15.121:0/4255272777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8610009370 con 0x7f86140aa020 2023-11-15T03:00:07.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f861a378700 1 --2- 172.21.15.121:0/4255272777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8614153bb0 0x7f86140abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:00:07.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f8619b77700 1 --2- 172.21.15.121:0/4255272777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8614067f50 0x7f86140aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:00:07.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f8619376700 1 --2- 172.21.15.121:0/4255272777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86140aa020 0x7f861414e610 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f860400e930 tx=0x7f8604009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:07.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f8609ffb700 1 -- 172.21.15.121:0/4255272777 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860400eed0 con 0x7f86140aa020 2023-11-15T03:00:07.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f8609ffb700 1 -- 172.21.15.121:0/4255272777 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860400c850 con 0x7f86140aa020 2023-11-15T03:00:07.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f861bddb700 1 -- 172.21.15.121:0/4255272777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86140ab6e0 con 0x7f86140aa020 2023-11-15T03:00:07.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f8609ffb700 1 -- 172.21.15.121:0/4255272777 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860400cbd0 con 0x7f86140aa020 2023-11-15T03:00:07.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.993+0000 7f861bddb700 1 -- 172.21.15.121:0/4255272777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86140aad30 con 0x7f86140aa020 2023-11-15T03:00:07.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.997+0000 7f861bddb700 1 -- 172.21.15.121:0/4255272777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f861406c150 con 0x7f86140aa020 2023-11-15T03:00:07.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.997+0000 7f8609ffb700 1 -- 172.21.15.121:0/4255272777 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f860401b400 con 0x7f86140aa020 2023-11-15T03:00:07.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.997+0000 7f8609ffb700 1 --2- 172.21.15.121:0/4255272777 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f85f805a510 0x7f85f805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:07.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:06.997+0000 7f8609ffb700 1 -- 172.21.15.121:0/4255272777 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f8604074df0 con 0x7f86140aa020 2023-11-15T03:00:07.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.001+0000 7f8619b77700 1 --2- 172.21.15.121:0/4255272777 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f85f805a510 0x7f85f805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:07.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.001+0000 7f8619b77700 1 --2- 172.21.15.121:0/4255272777 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f85f805a510 0x7f85f805c9d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f8610009130 tx=0x7f8610017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:07.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.005+0000 7f8609ffb700 1 -- 172.21.15.121:0/4255272777 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8604048550 con 0x7f86140aa020 2023-11-15T03:00:07.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.205+0000 7f861bddb700 1 -- 172.21.15.121:0/4255272777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86140c83c0 con 0x7f86140aa020 2023-11-15T03:00:07.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.209+0000 7f8609ffb700 1 -- 172.21.15.121:0/4255272777 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4503 (secure 0 0 0) 0x7f860404e1b0 con 0x7f86140aa020 2023-11-15T03:00:07.214 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:00:07.214 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":53,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3462858472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3462858472},{"type":"v1","addr":"172.21.15.190:6833","nonce":3462858472}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4042390087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4042390087},{"type":"v1","addr":"172.21.15.190:6835","nonce":4042390087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5907,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2239114529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2239114529},{"type":"v1","addr":"172.21.15.121:6837","nonce":2239114529}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:59:34.667864+0000","modified":"2023-11-15T03:00:01.157656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":5,"addr":"172.21.15.121:6836/3429169582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3429169582},{"type":"v1","addr":"172.21.15.121:6836","nonce":3429169582}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-15T03:00:07.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.213+0000 7f861bddb700 1 -- 172.21.15.121:0/4255272777 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f85f805a510 msgr2=0x7f85f805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:07.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.213+0000 7f861bddb700 1 --2- 172.21.15.121:0/4255272777 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f85f805a510 0x7f85f805c9d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f8610009130 tx=0x7f8610017000).stop 2023-11-15T03:00:07.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.213+0000 7f861bddb700 1 -- 172.21.15.121:0/4255272777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86140aa020 msgr2=0x7f861414e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:07.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.213+0000 7f861bddb700 1 --2- 172.21.15.121:0/4255272777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86140aa020 0x7f861414e610 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f860400e930 tx=0x7f8604009060).stop 2023-11-15T03:00:07.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.213+0000 7f861bddb700 1 -- 172.21.15.121:0/4255272777 shutdown_connections 2023-11-15T03:00:07.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.213+0000 7f861bddb700 1 --2- 172.21.15.121:0/4255272777 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f85f805a510 0x7f85f805c9d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:07.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.213+0000 7f861bddb700 1 --2- 172.21.15.121:0/4255272777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8614153bb0 0x7f86140abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:07.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.213+0000 7f861bddb700 1 --2- 172.21.15.121:0/4255272777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86140aa020 0x7f861414e610 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:07.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.213+0000 7f861bddb700 1 --2- 172.21.15.121:0/4255272777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8614067f50 0x7f86140aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:07.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.213+0000 7f861bddb700 1 -- 172.21.15.121:0/4255272777 >> 172.21.15.121:0/4255272777 conn(0x7f8614058c50 msgr2=0x7f86140c5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:07.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.213+0000 7f861bddb700 1 -- 172.21.15.121:0/4255272777 shutdown_connections 2023-11-15T03:00:07.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.213+0000 7f861bddb700 1 -- 172.21.15.121:0/4255272777 wait complete. 2023-11-15T03:00:07.218 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 53 2023-11-15T03:00:07.237 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-15T03:00:07.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.753+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.121:6835/3429169582 2023-11-15T03:00:07.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.753+0000 7f71d4ff9700 1 client.5633 ms_handle_refused on v2:172.21.15.121:6835/3429169582 2023-11-15T03:00:07.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:07.753+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=START_CONNECT pgs=15 cs=7 l=0 rev1=1 rx=0 tx=0)._fault waiting 6.400000 2023-11-15T03:00:10.602 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:00:10.610 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:00:12.238 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:00:12.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.401+0000 7f0b4f769700 1 Processor -- start 2023-11-15T03:00:12.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.401+0000 7f0b4f769700 1 -- start start 2023-11-15T03:00:12.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4f769700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0b48153bb0 0x7f0b480a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:12.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4f769700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b480aa020 0x7f0b48067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:12.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4f769700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0b48067f50 0x7f0b481578a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:12.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4f769700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b480aa6f0 con 0x7f0b48067f50 2023-11-15T03:00:12.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4f769700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b480c5900 con 0x7f0b48153bb0 2023-11-15T03:00:12.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4f769700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b480cb5c0 con 0x7f0b480aa020 2023-11-15T03:00:12.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4dd06700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0b48067f50 0x7f0b481578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:12.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4cd04700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b480aa020 0x7f0b48067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:12.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4d505700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0b48153bb0 0x7f0b480a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:12.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4cd04700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b480aa020 0x7f0b48067a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:57442/0 (socket says 172.21.15.121:57442) 2023-11-15T03:00:12.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4d505700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0b48153bb0 0x7f0b480a9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:43844/0 (socket says 172.21.15.121:43844) 2023-11-15T03:00:12.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4cd04700 1 -- 172.21.15.121:0/3937572789 learned_addr learned my addr 172.21.15.121:0/3937572789 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:12.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4dd06700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0b48067f50 0x7f0b481578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48940/0 (socket says 172.21.15.121:48940) 2023-11-15T03:00:12.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4cd04700 1 -- 172.21.15.121:0/3937572789 >> v2:172.21.15.190:3300/0 conn(0x7f0b48153bb0 msgr2=0x7f0b480a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:12.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4cd04700 1 --2- 172.21.15.121:0/3937572789 >> v2:172.21.15.190:3300/0 conn(0x7f0b48153bb0 0x7f0b480a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:12.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4cd04700 1 -- 172.21.15.121:0/3937572789 >> v2:172.21.15.121:3300/0 conn(0x7f0b48067f50 msgr2=0x7f0b481578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:12.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4cd04700 1 --2- 172.21.15.121:0/3937572789 >> v2:172.21.15.121:3300/0 conn(0x7f0b48067f50 0x7f0b481578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:12.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4cd04700 1 -- 172.21.15.121:0/3937572789 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b480683e0 con 0x7f0b480aa020 2023-11-15T03:00:12.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4dd06700 1 --2- 172.21.15.121:0/3937572789 >> v2:172.21.15.121:3300/0 conn(0x7f0b48067f50 0x7f0b481578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:00:12.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.405+0000 7f0b4d505700 1 --2- 172.21.15.121:0/3937572789 >> v2:172.21.15.190:3300/0 conn(0x7f0b48153bb0 0x7f0b480a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:00:12.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.409+0000 7f0b4cd04700 1 --2- 172.21.15.121:0/3937572789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b480aa020 0x7f0b48067a60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f0b48069f00 tx=0x7f0b380090f0).ready entity=mon.2 client_cookie=e7d9060192d240e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:12.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.409+0000 7f0b3ffff700 1 -- 172.21.15.121:0/3937572789 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3800f510 con 0x7f0b480aa020 2023-11-15T03:00:12.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.409+0000 7f0b3ffff700 1 -- 172.21.15.121:0/3937572789 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3800f670 con 0x7f0b480aa020 2023-11-15T03:00:12.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.409+0000 7f0b3ffff700 1 -- 172.21.15.121:0/3937572789 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3800f9f0 con 0x7f0b480aa020 2023-11-15T03:00:12.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.409+0000 7f0b4f769700 1 -- 172.21.15.121:0/3937572789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b480aa020 msgr2=0x7f0b48067a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:12.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.409+0000 7f0b4f769700 1 --2- 172.21.15.121:0/3937572789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b480aa020 0x7f0b48067a60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f0b48069f00 tx=0x7f0b380090f0).stop 2023-11-15T03:00:12.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.409+0000 7f0b4f769700 1 -- 172.21.15.121:0/3937572789 shutdown_connections 2023-11-15T03:00:12.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.409+0000 7f0b4f769700 1 --2- 172.21.15.121:0/3937572789 >> v2:172.21.15.121:3300/0 conn(0x7f0b48067f50 0x7f0b481578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:12.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.409+0000 7f0b4f769700 1 --2- 172.21.15.121:0/3937572789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b480aa020 0x7f0b48067a60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:12.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.409+0000 7f0b4f769700 1 --2- 172.21.15.121:0/3937572789 >> v2:172.21.15.190:3300/0 conn(0x7f0b48153bb0 0x7f0b480a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:12.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.409+0000 7f0b4f769700 1 -- 172.21.15.121:0/3937572789 >> 172.21.15.121:0/3937572789 conn(0x7f0b48058c50 msgr2=0x7f0b48059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:12.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.409+0000 7f0b4f769700 1 -- 172.21.15.121:0/3937572789 shutdown_connections 2023-11-15T03:00:12.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.409+0000 7f0b4f769700 1 -- 172.21.15.121:0/3937572789 wait complete. 2023-11-15T03:00:12.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.413+0000 7f0b4f769700 1 Processor -- start 2023-11-15T03:00:12.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.413+0000 7f0b4f769700 1 -- start start 2023-11-15T03:00:12.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.413+0000 7f0b4f769700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b48067f50 0x7f0b4814e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:12.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.413+0000 7f0b4f769700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b480aa020 0x7f0b480c0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:12.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b4f769700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b48153bb0 0x7f0b480c1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:12.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b4f769700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b480c1600 con 0x7f0b48153bb0 2023-11-15T03:00:12.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b4f769700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b48165300 con 0x7f0b48067f50 2023-11-15T03:00:12.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b4f769700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b48164710 con 0x7f0b480aa020 2023-11-15T03:00:12.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b4dd06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b48153bb0 0x7f0b480c1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:12.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b4dd06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b48153bb0 0x7f0b480c1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48946/0 (socket says 172.21.15.121:48946) 2023-11-15T03:00:12.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b4dd06700 1 -- 172.21.15.121:0/1260749087 learned_addr learned my addr 172.21.15.121:0/1260749087 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:12.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b4d505700 1 --2- 172.21.15.121:0/1260749087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b48067f50 0x7f0b4814e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:12.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b4cd04700 1 --2- 172.21.15.121:0/1260749087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b480aa020 0x7f0b480c0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:12.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b4dd06700 1 -- 172.21.15.121:0/1260749087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b480aa020 msgr2=0x7f0b480c0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:12.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b4dd06700 1 --2- 172.21.15.121:0/1260749087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b480aa020 0x7f0b480c0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:12.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b4dd06700 1 -- 172.21.15.121:0/1260749087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b48067f50 msgr2=0x7f0b4814e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:12.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b4dd06700 1 --2- 172.21.15.121:0/1260749087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b48067f50 0x7f0b4814e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:12.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b4dd06700 1 -- 172.21.15.121:0/1260749087 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b40009a20 con 0x7f0b48153bb0 2023-11-15T03:00:12.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b4dd06700 1 --2- 172.21.15.121:0/1260749087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b48153bb0 0x7f0b480c1110 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f0b40009fc0 tx=0x7f0b4000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:12.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b3d7fa700 1 -- 172.21.15.121:0/1260749087 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4000cc80 con 0x7f0b48153bb0 2023-11-15T03:00:12.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b3d7fa700 1 -- 172.21.15.121:0/1260749087 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4000f040 con 0x7f0b48153bb0 2023-11-15T03:00:12.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b4f769700 1 -- 172.21.15.121:0/1260749087 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b440092a0 con 0x7f0b48153bb0 2023-11-15T03:00:12.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b3d7fa700 1 -- 172.21.15.121:0/1260749087 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40013400 con 0x7f0b48153bb0 2023-11-15T03:00:12.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b4f769700 1 -- 172.21.15.121:0/1260749087 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b38009370 con 0x7f0b48153bb0 2023-11-15T03:00:12.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.417+0000 7f0b4f769700 1 -- 172.21.15.121:0/1260749087 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b48165470 con 0x7f0b48153bb0 2023-11-15T03:00:12.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.421+0000 7f0b3d7fa700 1 -- 172.21.15.121:0/1260749087 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0b400137b0 con 0x7f0b48153bb0 2023-11-15T03:00:12.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.421+0000 7f0b3d7fa700 1 --2- 172.21.15.121:0/1260749087 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0b2c05a4e0 0x7f0b2c05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:12.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.421+0000 7f0b3d7fa700 1 -- 172.21.15.121:0/1260749087 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f0b40075240 con 0x7f0b48153bb0 2023-11-15T03:00:12.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.421+0000 7f0b4d505700 1 --2- 172.21.15.121:0/1260749087 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0b2c05a4e0 0x7f0b2c05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:12.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.421+0000 7f0b4d505700 1 --2- 172.21.15.121:0/1260749087 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0b2c05a4e0 0x7f0b2c05c9a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f0b44009d20 tx=0x7f0b44009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:12.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.429+0000 7f0b3d7fa700 1 -- 172.21.15.121:0/1260749087 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0b40048910 con 0x7f0b48153bb0 2023-11-15T03:00:12.496 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2023-11-15T03:00:12.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.625+0000 7f0b4f769700 1 -- 172.21.15.121:0/1260749087 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b4806c150 con 0x7f0b48153bb0 2023-11-15T03:00:12.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.629+0000 7f0b3d7fa700 1 -- 172.21.15.121:0/1260749087 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4503 (secure 0 0 0) 0x7f0b4004bf30 con 0x7f0b48153bb0 2023-11-15T03:00:12.632 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:00:12.632 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":53,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3462858472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3462858472},{"type":"v1","addr":"172.21.15.190:6833","nonce":3462858472}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4042390087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4042390087},{"type":"v1","addr":"172.21.15.190:6835","nonce":4042390087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5907,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2239114529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2239114529},{"type":"v1","addr":"172.21.15.121:6837","nonce":2239114529}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:59:34.667864+0000","modified":"2023-11-15T03:00:01.157656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":5,"addr":"172.21.15.121:6836/3429169582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3429169582},{"type":"v1","addr":"172.21.15.121:6836","nonce":3429169582}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-15T03:00:12.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.629+0000 7f0b4f769700 1 -- 172.21.15.121:0/1260749087 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0b2c05a4e0 msgr2=0x7f0b2c05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:12.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.629+0000 7f0b4f769700 1 --2- 172.21.15.121:0/1260749087 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0b2c05a4e0 0x7f0b2c05c9a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f0b44009d20 tx=0x7f0b44009060).stop 2023-11-15T03:00:12.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.629+0000 7f0b4f769700 1 -- 172.21.15.121:0/1260749087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b48153bb0 msgr2=0x7f0b480c1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:12.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.629+0000 7f0b4f769700 1 --2- 172.21.15.121:0/1260749087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b48153bb0 0x7f0b480c1110 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f0b40009fc0 tx=0x7f0b4000b040).stop 2023-11-15T03:00:12.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.629+0000 7f0b4f769700 1 -- 172.21.15.121:0/1260749087 shutdown_connections 2023-11-15T03:00:12.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.629+0000 7f0b4f769700 1 --2- 172.21.15.121:0/1260749087 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0b2c05a4e0 0x7f0b2c05c9a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:12.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.629+0000 7f0b4f769700 1 --2- 172.21.15.121:0/1260749087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b48153bb0 0x7f0b480c1110 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:12.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.629+0000 7f0b4f769700 1 --2- 172.21.15.121:0/1260749087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b480aa020 0x7f0b480c0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:12.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.629+0000 7f0b4f769700 1 --2- 172.21.15.121:0/1260749087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b48067f50 0x7f0b4814e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:12.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.629+0000 7f0b4f769700 1 -- 172.21.15.121:0/1260749087 >> 172.21.15.121:0/1260749087 conn(0x7f0b48058c50 msgr2=0x7f0b480b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:12.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.629+0000 7f0b4f769700 1 -- 172.21.15.121:0/1260749087 shutdown_connections 2023-11-15T03:00:12.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:12.629+0000 7f0b4f769700 1 -- 172.21.15.121:0/1260749087 wait complete. 2023-11-15T03:00:12.638 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 53 2023-11-15T03:00:12.652 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-15T03:00:14.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:14.157+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.121:6835/3429169582 2023-11-15T03:00:14.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:14.157+0000 7f71d4ff9700 1 client.5633 ms_handle_refused on v2:172.21.15.121:6835/3429169582 2023-11-15T03:00:14.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:14.157+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=START_CONNECT pgs=15 cs=8 l=0 rev1=1 rx=0 tx=0)._fault waiting 12.800000 2023-11-15T03:00:17.653 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:00:17.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.817+0000 7fd2c0d1b700 1 Processor -- start 2023-11-15T03:00:17.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.817+0000 7fd2c0d1b700 1 -- start start 2023-11-15T03:00:17.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.821+0000 7fd2c0d1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2bc153d20 0x7fd2bc150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:17.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.821+0000 7fd2c0d1b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd2bc0c38a0 0x7fd2bc151140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:17.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.821+0000 7fd2c0d1b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2bc0c6350 con 0x7fd2bc150870 2023-11-15T03:00:17.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.821+0000 7fd2c0d1b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2bc1541b0 con 0x7fd2bc0c38a0 2023-11-15T03:00:17.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.821+0000 7fd2c0d1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc154490 con 0x7fd2bc153d20 2023-11-15T03:00:17.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.821+0000 7fd2bad9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd2bc0c38a0 0x7fd2bc151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:17.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.821+0000 7fd2bad9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd2bc0c38a0 0x7fd2bc151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:54430/0 (socket says 172.21.15.121:54430) 2023-11-15T03:00:17.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.821+0000 7fd2bad9d700 1 -- 172.21.15.121:0/1341656622 learned_addr learned my addr 172.21.15.121:0/1341656622 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:17.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.821+0000 7fd2b9d9b700 1 --2- 172.21.15.121:0/1341656622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2bc153d20 0x7fd2bc150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:17.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.825+0000 7fd2bad9d700 1 -- 172.21.15.121:0/1341656622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2bc153d20 msgr2=0x7fd2bc150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:17.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.825+0000 7fd2bad9d700 1 --2- 172.21.15.121:0/1341656622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2bc153d20 0x7fd2bc150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:17.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.825+0000 7fd2bad9d700 1 -- 172.21.15.121:0/1341656622 >> v1:172.21.15.190:6789/0 conn(0x7fd2bc150870 legacy=0x7fd2bc0c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:17.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.825+0000 7fd2bad9d700 1 -- 172.21.15.121:0/1341656622 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2bc154770 con 0x7fd2bc0c38a0 2023-11-15T03:00:17.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.825+0000 7fd2bad9d700 1 --2- 172.21.15.121:0/1341656622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2bc0c38a0 0x7fd2bc151140 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fd2b4009f20 tx=0x7fd2b4009840).ready entity=mon.1 client_cookie=4d8672a1b5871d4d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:17.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.825+0000 7fd2b959a700 1 -- 172.21.15.121:0/1341656622 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b400e070 con 0x7fd2bc0c38a0 2023-11-15T03:00:17.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.825+0000 7fd2b959a700 1 -- 172.21.15.121:0/1341656622 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b400b9d0 con 0x7fd2bc0c38a0 2023-11-15T03:00:17.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.825+0000 7fd2b959a700 1 -- 172.21.15.121:0/1341656622 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b400bd50 con 0x7fd2bc0c38a0 2023-11-15T03:00:17.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.825+0000 7fd2c0d1b700 1 -- 172.21.15.121:0/1341656622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2bc0c38a0 msgr2=0x7fd2bc151140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:17.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.825+0000 7fd2c0d1b700 1 --2- 172.21.15.121:0/1341656622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2bc0c38a0 0x7fd2bc151140 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fd2b4009f20 tx=0x7fd2b4009840).stop 2023-11-15T03:00:17.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.825+0000 7fd2c0d1b700 1 -- 172.21.15.121:0/1341656622 shutdown_connections 2023-11-15T03:00:17.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.825+0000 7fd2c0d1b700 1 --2- 172.21.15.121:0/1341656622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2bc0c38a0 0x7fd2bc151140 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:17.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.825+0000 7fd2c0d1b700 1 --2- 172.21.15.121:0/1341656622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2bc153d20 0x7fd2bc150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:17.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.825+0000 7fd2c0d1b700 1 -- 172.21.15.121:0/1341656622 >> 172.21.15.121:0/1341656622 conn(0x7fd2bc062190 msgr2=0x7fd2bc0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:17.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.825+0000 7fd2c0d1b700 1 -- 172.21.15.121:0/1341656622 shutdown_connections 2023-11-15T03:00:17.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.825+0000 7fd2c0d1b700 1 -- 172.21.15.121:0/1341656622 wait complete. 2023-11-15T03:00:17.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.829+0000 7fd2c0d1b700 1 Processor -- start 2023-11-15T03:00:17.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.829+0000 7fd2c0d1b700 1 -- start start 2023-11-15T03:00:17.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.829+0000 7fd2c0d1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2bc0c38a0 0x7fd2bc149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:17.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.829+0000 7fd2c0d1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2bc150870 0x7fd2bc1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:17.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.829+0000 7fd2c0d1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2bc151c20 0x7fd2bc0a8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:17.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.829+0000 7fd2c0d1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc152c30 con 0x7fd2bc0c38a0 2023-11-15T03:00:17.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.829+0000 7fd2c0d1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc1520b0 con 0x7fd2bc150870 2023-11-15T03:00:17.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.829+0000 7fd2c0d1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc152390 con 0x7fd2bc151c20 2023-11-15T03:00:17.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.829+0000 7fd2ba59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2bc0c38a0 0x7fd2bc149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:17.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.829+0000 7fd2ba59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2bc0c38a0 0x7fd2bc149610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33842/0 (socket says 172.21.15.121:33842) 2023-11-15T03:00:17.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.829+0000 7fd2ba59c700 1 -- 172.21.15.121:0/2975992682 learned_addr learned my addr 172.21.15.121:0/2975992682 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:17.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.833+0000 7fd2b9d9b700 1 --2- 172.21.15.121:0/2975992682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2bc150870 0x7fd2bc1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:17.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.833+0000 7fd2bad9d700 1 --2- 172.21.15.121:0/2975992682 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2bc151c20 0x7fd2bc0a8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:17.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.833+0000 7fd2ba59c700 1 -- 172.21.15.121:0/2975992682 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2bc151c20 msgr2=0x7fd2bc0a8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:17.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.833+0000 7fd2ba59c700 1 --2- 172.21.15.121:0/2975992682 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2bc151c20 0x7fd2bc0a8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:17.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.833+0000 7fd2ba59c700 1 -- 172.21.15.121:0/2975992682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2bc150870 msgr2=0x7fd2bc1533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:17.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.833+0000 7fd2ba59c700 1 --2- 172.21.15.121:0/2975992682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2bc150870 0x7fd2bc1533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:17.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.833+0000 7fd2ba59c700 1 -- 172.21.15.121:0/2975992682 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b4009ae0 con 0x7fd2bc0c38a0 2023-11-15T03:00:17.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.833+0000 7fd2b9d9b700 1 --2- 172.21.15.121:0/2975992682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2bc150870 0x7fd2bc1533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:00:17.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.833+0000 7fd2ba59c700 1 --2- 172.21.15.121:0/2975992682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2bc0c38a0 0x7fd2bc149610 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fd2ac00c4c0 tx=0x7fd2ac007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:17.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.833+0000 7fd2aa7fc700 1 -- 172.21.15.121:0/2975992682 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac007020 con 0x7fd2bc0c38a0 2023-11-15T03:00:17.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.833+0000 7fd2aa7fc700 1 -- 172.21.15.121:0/2975992682 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ac001e90 con 0x7fd2bc0c38a0 2023-11-15T03:00:17.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.833+0000 7fd2c0d1b700 1 -- 172.21.15.121:0/2975992682 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2bc152670 con 0x7fd2bc0c38a0 2023-11-15T03:00:17.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.833+0000 7fd2aa7fc700 1 -- 172.21.15.121:0/2975992682 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac0073b0 con 0x7fd2bc0c38a0 2023-11-15T03:00:17.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.833+0000 7fd2c0d1b700 1 -- 172.21.15.121:0/2975992682 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2bc1538d0 con 0x7fd2bc0c38a0 2023-11-15T03:00:17.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.833+0000 7fd2c0d1b700 1 -- 172.21.15.121:0/2975992682 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2bc06c150 con 0x7fd2bc0c38a0 2023-11-15T03:00:17.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.833+0000 7fd2aa7fc700 1 -- 172.21.15.121:0/2975992682 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd2ac0068b0 con 0x7fd2bc0c38a0 2023-11-15T03:00:17.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.837+0000 7fd2aa7fc700 1 --2- 172.21.15.121:0/2975992682 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd29c05a510 0x7fd29c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:17.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.837+0000 7fd2aa7fc700 1 -- 172.21.15.121:0/2975992682 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fd2ac074200 con 0x7fd2bc0c38a0 2023-11-15T03:00:17.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.837+0000 7fd2b9d9b700 1 --2- 172.21.15.121:0/2975992682 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd29c05a510 0x7fd29c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:17.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.837+0000 7fd2b9d9b700 1 --2- 172.21.15.121:0/2975992682 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd29c05a510 0x7fd29c05c9d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fd2bc14c9e0 tx=0x7fd2b0008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:17.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:17.845+0000 7fd2aa7fc700 1 -- 172.21.15.121:0/2975992682 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd2ac047990 con 0x7fd2bc0c38a0 2023-11-15T03:00:18.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.045+0000 7fd2c0d1b700 1 -- 172.21.15.121:0/2975992682 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2bc0c7440 con 0x7fd2bc0c38a0 2023-11-15T03:00:18.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.045+0000 7fd2aa7fc700 1 -- 172.21.15.121:0/2975992682 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4551 (secure 0 0 0) 0x7fd2ac04afb0 con 0x7fd2bc0c38a0 2023-11-15T03:00:18.050 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:00:18.050 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":54,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3462858472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3462858472},{"type":"v1","addr":"172.21.15.190:6833","nonce":3462858472}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4042390087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4042390087},{"type":"v1","addr":"172.21.15.190:6835","nonce":4042390087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5907,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2239114529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2239114529},{"type":"v1","addr":"172.21.15.121:6837","nonce":2239114529}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:59:34.667864+0000","modified":"2023-11-15T03:00:17.916898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":5,"addr":"172.21.15.121:6836/3429169582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3429169582},{"type":"v1","addr":"172.21.15.121:6836","nonce":3429169582}]},"join_fscid":-1,"laggy_since":"2023-11-15T03:00:17.916781+0000","export_targets":[],"features":4540138314316775423,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-15T03:00:18.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.049+0000 7fd2c0d1b700 1 -- 172.21.15.121:0/2975992682 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd29c05a510 msgr2=0x7fd29c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:18.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.049+0000 7fd2c0d1b700 1 --2- 172.21.15.121:0/2975992682 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd29c05a510 0x7fd29c05c9d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fd2bc14c9e0 tx=0x7fd2b0008040).stop 2023-11-15T03:00:18.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.049+0000 7fd2c0d1b700 1 -- 172.21.15.121:0/2975992682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2bc0c38a0 msgr2=0x7fd2bc149610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:18.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.049+0000 7fd2c0d1b700 1 --2- 172.21.15.121:0/2975992682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2bc0c38a0 0x7fd2bc149610 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fd2ac00c4c0 tx=0x7fd2ac007e00).stop 2023-11-15T03:00:18.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.049+0000 7fd2c0d1b700 1 -- 172.21.15.121:0/2975992682 shutdown_connections 2023-11-15T03:00:18.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.049+0000 7fd2c0d1b700 1 --2- 172.21.15.121:0/2975992682 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd29c05a510 0x7fd29c05c9d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:18.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.049+0000 7fd2c0d1b700 1 --2- 172.21.15.121:0/2975992682 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2bc151c20 0x7fd2bc0a8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:18.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.049+0000 7fd2c0d1b700 1 --2- 172.21.15.121:0/2975992682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2bc150870 0x7fd2bc1533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:18.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.049+0000 7fd2c0d1b700 1 --2- 172.21.15.121:0/2975992682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2bc0c38a0 0x7fd2bc149610 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:18.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.049+0000 7fd2c0d1b700 1 -- 172.21.15.121:0/2975992682 >> 172.21.15.121:0/2975992682 conn(0x7fd2bc062190 msgr2=0x7fd2bc0bfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:18.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.049+0000 7fd2c0d1b700 1 -- 172.21.15.121:0/2975992682 shutdown_connections 2023-11-15T03:00:18.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.049+0000 7fd2c0d1b700 1 -- 172.21.15.121:0/2975992682 wait complete. 2023-11-15T03:00:18.056 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 54 2023-11-15T03:00:18.071 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2023-11-15T03:00:18.071 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_journal_corrupt_dentry_first 2023-11-15T03:00:18.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.233+0000 7ff2da9b9700 1 Processor -- start 2023-11-15T03:00:18.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.233+0000 7ff2da9b9700 1 -- start start 2023-11-15T03:00:18.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.237+0000 7ff2da9b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2d414e380 0x7ff2d40c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:18.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.237+0000 7ff2da9b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff2d40c6040 0x7ff2d40acb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:18.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.237+0000 7ff2da9b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d40ad500 con 0x7ff2d40ad050 2023-11-15T03:00:18.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.237+0000 7ff2da9b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d40c8320 con 0x7ff2d40c6040 2023-11-15T03:00:18.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.237+0000 7ff2da9b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d40c7080 con 0x7ff2d414e380 2023-11-15T03:00:18.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.237+0000 7ff2d99b7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2d414e380 0x7ff2d40c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:18.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.237+0000 7ff2d91b6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff2d40c6040 0x7ff2d40acb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:18.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.237+0000 7ff2d91b6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff2d40c6040 0x7ff2d40acb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:54450/0 (socket says 172.21.15.121:54450) 2023-11-15T03:00:18.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.237+0000 7ff2d91b6700 1 -- 172.21.15.121:0/2904966945 learned_addr learned my addr 172.21.15.121:0/2904966945 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:18.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.237+0000 7ff2d99b7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2d414e380 0x7ff2d40c5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:59006/0 (socket says 172.21.15.121:59006) 2023-11-15T03:00:18.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.237+0000 7ff2da1b8700 1 --1- 172.21.15.121:0/2904966945 >> v1:172.21.15.190:6789/0 conn(0x7ff2d40ad050 0x7ff2d4062190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:38140/0 (socket says 172.21.15.121:38140) 2023-11-15T03:00:18.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.237+0000 7ff2d91b6700 1 -- 172.21.15.121:0/2904966945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2d414e380 msgr2=0x7ff2d40c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:18.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.237+0000 7ff2d91b6700 1 --2- 172.21.15.121:0/2904966945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2d414e380 0x7ff2d40c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:18.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.237+0000 7ff2d91b6700 1 -- 172.21.15.121:0/2904966945 >> v1:172.21.15.190:6789/0 conn(0x7ff2d40ad050 legacy=0x7ff2d4062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:18.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.237+0000 7ff2d91b6700 1 -- 172.21.15.121:0/2904966945 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d40c7e90 con 0x7ff2d40c6040 2023-11-15T03:00:18.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.237+0000 7ff2d91b6700 1 --2- 172.21.15.121:0/2904966945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d40c6040 0x7ff2d40acb60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7ff2c4006fd0 tx=0x7ff2c40090f0).ready entity=mon.1 client_cookie=d1b68d858933a0b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:18.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.237+0000 7ff2d89b5700 1 -- 172.21.15.121:0/2904966945 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c400b580 con 0x7ff2d40c6040 2023-11-15T03:00:18.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.241+0000 7ff2d89b5700 1 -- 172.21.15.121:0/2904966945 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c400b6e0 con 0x7ff2d40c6040 2023-11-15T03:00:18.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.241+0000 7ff2d89b5700 1 -- 172.21.15.121:0/2904966945 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c400ba60 con 0x7ff2d40c6040 2023-11-15T03:00:18.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.241+0000 7ff2da9b9700 1 -- 172.21.15.121:0/2904966945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d40c6040 msgr2=0x7ff2d40acb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:18.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.241+0000 7ff2da9b9700 1 --2- 172.21.15.121:0/2904966945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d40c6040 0x7ff2d40acb60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7ff2c4006fd0 tx=0x7ff2c40090f0).stop 2023-11-15T03:00:18.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.241+0000 7ff2da9b9700 1 -- 172.21.15.121:0/2904966945 shutdown_connections 2023-11-15T03:00:18.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.241+0000 7ff2da9b9700 1 --2- 172.21.15.121:0/2904966945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d40c6040 0x7ff2d40acb60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:18.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.241+0000 7ff2da9b9700 1 --2- 172.21.15.121:0/2904966945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2d414e380 0x7ff2d40c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:18.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.241+0000 7ff2da9b9700 1 -- 172.21.15.121:0/2904966945 >> 172.21.15.121:0/2904966945 conn(0x7ff2d4067a60 msgr2=0x7ff2d4067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:18.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.241+0000 7ff2da9b9700 1 -- 172.21.15.121:0/2904966945 shutdown_connections 2023-11-15T03:00:18.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.241+0000 7ff2da9b9700 1 -- 172.21.15.121:0/2904966945 wait complete. 2023-11-15T03:00:18.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.241+0000 7ff2da9b9700 1 Processor -- start 2023-11-15T03:00:18.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.241+0000 7ff2da9b9700 1 -- start start 2023-11-15T03:00:18.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2da9b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d40ad050 0x7ff2d414caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:18.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2da9b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2d40c6040 0x7ff2d4164de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:18.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2da9b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2d414e380 0x7ff2d4155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:18.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2da9b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d40c2370 con 0x7ff2d414e380 2023-11-15T03:00:18.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2da9b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d4156290 con 0x7ff2d40ad050 2023-11-15T03:00:18.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2da9b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d40c1a80 con 0x7ff2d40c6040 2023-11-15T03:00:18.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2da1b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2d414e380 0x7ff2d4155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:18.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2da1b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2d414e380 0x7ff2d4155da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33848/0 (socket says 172.21.15.121:33848) 2023-11-15T03:00:18.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2da1b8700 1 -- 172.21.15.121:0/757978447 learned_addr learned my addr 172.21.15.121:0/757978447 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:18.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2d99b7700 1 --2- 172.21.15.121:0/757978447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d40ad050 0x7ff2d414caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:18.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2d91b6700 1 --2- 172.21.15.121:0/757978447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2d40c6040 0x7ff2d4164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:18.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2da1b8700 1 -- 172.21.15.121:0/757978447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2d40c6040 msgr2=0x7ff2d4164de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:18.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2da1b8700 1 --2- 172.21.15.121:0/757978447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2d40c6040 0x7ff2d4164de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:18.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2da1b8700 1 -- 172.21.15.121:0/757978447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d40ad050 msgr2=0x7ff2d414caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:18.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2da1b8700 1 --2- 172.21.15.121:0/757978447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d40ad050 0x7ff2d414caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:18.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2da1b8700 1 -- 172.21.15.121:0/757978447 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c4009370 con 0x7ff2d414e380 2023-11-15T03:00:18.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2d91b6700 1 --2- 172.21.15.121:0/757978447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2d40c6040 0x7ff2d4164de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:00:18.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2da1b8700 1 --2- 172.21.15.121:0/757978447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2d414e380 0x7ff2d4155da0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7ff2cc00c7c0 tx=0x7ff2cc0015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:18.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2c9ffb700 1 -- 172.21.15.121:0/757978447 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc005c70 con 0x7ff2d414e380 2023-11-15T03:00:18.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2c9ffb700 1 -- 172.21.15.121:0/757978447 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2cc005dd0 con 0x7ff2d414e380 2023-11-15T03:00:18.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2da9b9700 1 -- 172.21.15.121:0/757978447 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d40c1d20 con 0x7ff2d414e380 2023-11-15T03:00:18.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2c9ffb700 1 -- 172.21.15.121:0/757978447 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc007320 con 0x7ff2d414e380 2023-11-15T03:00:18.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.245+0000 7ff2da9b9700 1 -- 172.21.15.121:0/757978447 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d4156510 con 0x7ff2d414e380 2023-11-15T03:00:18.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.249+0000 7ff2da9b9700 1 -- 172.21.15.121:0/757978447 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d40c21a0 con 0x7ff2d414e380 2023-11-15T03:00:18.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.249+0000 7ff2c9ffb700 1 -- 172.21.15.121:0/757978447 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff2cc0076d0 con 0x7ff2d414e380 2023-11-15T03:00:18.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.249+0000 7ff2c9ffb700 1 --2- 172.21.15.121:0/757978447 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff2b805a510 0x7ff2b805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:18.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.249+0000 7ff2c9ffb700 1 -- 172.21.15.121:0/757978447 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ff2cc074670 con 0x7ff2d414e380 2023-11-15T03:00:18.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.253+0000 7ff2d99b7700 1 --2- 172.21.15.121:0/757978447 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff2b805a510 0x7ff2b805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:18.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.253+0000 7ff2d99b7700 1 --2- 172.21.15.121:0/757978447 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff2b805a510 0x7ff2b805c9d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7ff2d414d2d0 tx=0x7ff2d0009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:18.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.257+0000 7ff2c9ffb700 1 -- 172.21.15.121:0/757978447 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff2cc047d40 con 0x7ff2d414e380 2023-11-15T03:00:18.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.425+0000 7ff2da9b9700 1 -- 172.21.15.121:0/757978447 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"} v 0) v1 -- 0x7ff2d40ad500 con 0x7ff2d414e380 2023-11-15T03:00:18.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.433+0000 7ff2c9ffb700 1 -- 172.21.15.121:0/757978447 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"}]=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff2d40ad500 con 0x7ff2d414e380 2023-11-15T03:00:18.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.437+0000 7ff2da9b9700 1 -- 172.21.15.121:0/757978447 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff2b805a510 msgr2=0x7ff2b805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:18.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.437+0000 7ff2da9b9700 1 --2- 172.21.15.121:0/757978447 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff2b805a510 0x7ff2b805c9d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7ff2d414d2d0 tx=0x7ff2d0009060).stop 2023-11-15T03:00:18.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.437+0000 7ff2da9b9700 1 -- 172.21.15.121:0/757978447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2d414e380 msgr2=0x7ff2d4155da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:18.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.437+0000 7ff2da9b9700 1 --2- 172.21.15.121:0/757978447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2d414e380 0x7ff2d4155da0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7ff2cc00c7c0 tx=0x7ff2cc0015e0).stop 2023-11-15T03:00:18.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.437+0000 7ff2da9b9700 1 -- 172.21.15.121:0/757978447 shutdown_connections 2023-11-15T03:00:18.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.437+0000 7ff2da9b9700 1 --2- 172.21.15.121:0/757978447 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff2b805a510 0x7ff2b805c9d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:18.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.437+0000 7ff2da9b9700 1 --2- 172.21.15.121:0/757978447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2d414e380 0x7ff2d4155da0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:18.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.437+0000 7ff2da9b9700 1 --2- 172.21.15.121:0/757978447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2d40c6040 0x7ff2d4164de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:18.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.437+0000 7ff2da9b9700 1 --2- 172.21.15.121:0/757978447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d40ad050 0x7ff2d414caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:18.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.437+0000 7ff2da9b9700 1 -- 172.21.15.121:0/757978447 >> 172.21.15.121:0/757978447 conn(0x7ff2d4067a60 msgr2=0x7ff2d40c0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:18.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.437+0000 7ff2da9b9700 1 -- 172.21.15.121:0/757978447 shutdown_connections 2023-11-15T03:00:18.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.437+0000 7ff2da9b9700 1 -- 172.21.15.121:0/757978447 wait complete. 2023-11-15T03:00:18.462 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 4:0 false 2023-11-15T03:00:18.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.625+0000 7fc3cad07700 1 Processor -- start 2023-11-15T03:00:18.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.629+0000 7fc3cad07700 1 -- start start 2023-11-15T03:00:18.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.629+0000 7fc3cad07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3c40a8960 0x7fc3c40c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:18.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.629+0000 7fc3cad07700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc3c4066520 0x7fc3c40669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:18.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.629+0000 7fc3cad07700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3c40657f0 con 0x7fc3c40a8d40 2023-11-15T03:00:18.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.629+0000 7fc3cad07700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c41583d0 con 0x7fc3c4066520 2023-11-15T03:00:18.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.629+0000 7fc3cad07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c40c1b60 con 0x7fc3c40a8960 2023-11-15T03:00:18.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3c9d05700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3c40a8960 0x7fc3c40c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:18.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3ca506700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc3c4066520 0x7fc3c40669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:18.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3c9d05700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3c40a8960 0x7fc3c40c15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:59032/0 (socket says 172.21.15.121:59032) 2023-11-15T03:00:18.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3c9d05700 1 -- 172.21.15.121:0/1331138482 learned_addr learned my addr 172.21.15.121:0/1331138482 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:18.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3c9504700 1 --1- 172.21.15.121:0/1331138482 >> v1:172.21.15.121:6789/0 conn(0x7fc3c40a8d40 0x7fc3c415c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:46816/0 (socket says 172.21.15.121:46816) 2023-11-15T03:00:18.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3c9d05700 1 -- 172.21.15.121:0/1331138482 >> v2:172.21.15.121:3300/0 conn(0x7fc3c4066520 msgr2=0x7fc3c40669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:18.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3c9d05700 1 --2- 172.21.15.121:0/1331138482 >> v2:172.21.15.121:3300/0 conn(0x7fc3c4066520 0x7fc3c40669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:18.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3c9d05700 1 -- 172.21.15.121:0/1331138482 >> v1:172.21.15.121:6789/0 conn(0x7fc3c40a8d40 legacy=0x7fc3c415c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:18.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3c9d05700 1 -- 172.21.15.121:0/1331138482 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c4065a40 con 0x7fc3c40a8960 2023-11-15T03:00:18.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3c9d05700 1 --2- 172.21.15.121:0/1331138482 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3c40a8960 0x7fc3c40c15c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fc3c0006fd0 tx=0x7fc3c00090f0).ready entity=mon.2 client_cookie=57d89c80bcb4a0ff server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:18.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3c8d03700 1 -- 172.21.15.121:0/1331138482 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c000b580 con 0x7fc3c40a8960 2023-11-15T03:00:18.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3c8d03700 1 -- 172.21.15.121:0/1331138482 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c000b6e0 con 0x7fc3c40a8960 2023-11-15T03:00:18.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3c8d03700 1 -- 172.21.15.121:0/1331138482 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c000ba60 con 0x7fc3c40a8960 2023-11-15T03:00:18.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3cad07700 1 -- 172.21.15.121:0/1331138482 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3c40a8960 msgr2=0x7fc3c40c15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:18.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3cad07700 1 --2- 172.21.15.121:0/1331138482 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3c40a8960 0x7fc3c40c15c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fc3c0006fd0 tx=0x7fc3c00090f0).stop 2023-11-15T03:00:18.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3cad07700 1 -- 172.21.15.121:0/1331138482 shutdown_connections 2023-11-15T03:00:18.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3cad07700 1 --2- 172.21.15.121:0/1331138482 >> v2:172.21.15.121:3300/0 conn(0x7fc3c4066520 0x7fc3c40669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:18.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3cad07700 1 --2- 172.21.15.121:0/1331138482 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3c40a8960 0x7fc3c40c15c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:18.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3cad07700 1 -- 172.21.15.121:0/1331138482 >> 172.21.15.121:0/1331138482 conn(0x7fc3c4067a60 msgr2=0x7fc3c4067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:18.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3cad07700 1 -- 172.21.15.121:0/1331138482 shutdown_connections 2023-11-15T03:00:18.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.633+0000 7fc3cad07700 1 -- 172.21.15.121:0/1331138482 wait complete. 2023-11-15T03:00:18.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3cad07700 1 Processor -- start 2023-11-15T03:00:18.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3cad07700 1 -- start start 2023-11-15T03:00:18.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3cad07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3c4066520 0x7fc3c416c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:18.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3cad07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3c40a8960 0x7fc3c40c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:18.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3cad07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3c40a8d40 0x7fc3c416b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:18.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3cad07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c41553c0 con 0x7fc3c4066520 2023-11-15T03:00:18.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3cad07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c40c73c0 con 0x7fc3c40a8960 2023-11-15T03:00:18.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3cad07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c40c6b20 con 0x7fc3c40a8d40 2023-11-15T03:00:18.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3c9d05700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3c4066520 0x7fc3c416c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:18.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3c9d05700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3c4066520 0x7fc3c416c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33868/0 (socket says 172.21.15.121:33868) 2023-11-15T03:00:18.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3c9d05700 1 -- 172.21.15.121:0/94887650 learned_addr learned my addr 172.21.15.121:0/94887650 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:18.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3c9d05700 1 -- 172.21.15.121:0/94887650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3c40a8d40 msgr2=0x7fc3c416b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:18.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3c9d05700 1 --2- 172.21.15.121:0/94887650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3c40a8d40 0x7fc3c416b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:00:18.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3c9d05700 1 -- 172.21.15.121:0/94887650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3c40a8960 msgr2=0x7fc3c40c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:18.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3c9d05700 1 --2- 172.21.15.121:0/94887650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3c40a8960 0x7fc3c40c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:00:18.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3c9d05700 1 -- 172.21.15.121:0/94887650 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c0009370 con 0x7fc3c4066520 2023-11-15T03:00:18.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3c9d05700 1 --2- 172.21.15.121:0/94887650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3c4066520 0x7fc3c416c3d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7fc3c0009fd0 tx=0x7fc3c0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:18.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3b9ffb700 1 -- 172.21.15.121:0/94887650 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c000b850 con 0x7fc3c4066520 2023-11-15T03:00:18.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3b9ffb700 1 -- 172.21.15.121:0/94887650 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c000b9b0 con 0x7fc3c4066520 2023-11-15T03:00:18.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3cad07700 1 -- 172.21.15.121:0/94887650 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c40c6e00 con 0x7fc3c4066520 2023-11-15T03:00:18.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.637+0000 7fc3b9ffb700 1 -- 172.21.15.121:0/94887650 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c001c670 con 0x7fc3c4066520 2023-11-15T03:00:18.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.641+0000 7fc3cad07700 1 -- 172.21.15.121:0/94887650 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c40c4ae0 con 0x7fc3c4066520 2023-11-15T03:00:18.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.641+0000 7fc3cad07700 1 -- 172.21.15.121:0/94887650 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c40c4940 con 0x7fc3c4066520 2023-11-15T03:00:18.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.641+0000 7fc3b9ffb700 1 -- 172.21.15.121:0/94887650 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc3c001ca60 con 0x7fc3c4066520 2023-11-15T03:00:18.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.645+0000 7fc3b9ffb700 1 --2- 172.21.15.121:0/94887650 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc3a805a510 0x7fc3a805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:18.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.645+0000 7fc3b9ffb700 1 -- 172.21.15.121:0/94887650 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fc3c0076590 con 0x7fc3c4066520 2023-11-15T03:00:18.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.645+0000 7fc3c9504700 1 --2- 172.21.15.121:0/94887650 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc3a805a510 0x7fc3a805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:18.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.645+0000 7fc3c9504700 1 --2- 172.21.15.121:0/94887650 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc3a805a510 0x7fc3a805c9d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7fc3c416c980 tx=0x7fc3b40079b0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:18.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.649+0000 7fc3b9ffb700 1 -- 172.21.15.121:0/94887650 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc3c40c4940 con 0x7fc3c4066520 2023-11-15T03:00:18.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:18.861+0000 7fc3cad07700 1 -- 172.21.15.121:0/94887650 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "4:0", "val": "false"} v 0) v1 -- 0x7fc3c40c4940 con 0x7fc3c4066520 2023-11-15T03:00:19.109 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2023-11-15T03:00:19.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.437+0000 7fc3b9ffb700 1 -- 172.21.15.121:0/94887650 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "4:0", "val": "false"}]=0 unfreezing mds.6144 v55) v1 ==== 115+0+0 (secure 0 0 0) 0x7fc3c40c4940 con 0x7fc3c4066520 2023-11-15T03:00:19.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.441+0000 7fc3cad07700 1 -- 172.21.15.121:0/94887650 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc3a805a510 msgr2=0x7fc3a805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:19.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.441+0000 7fc3cad07700 1 --2- 172.21.15.121:0/94887650 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc3a805a510 0x7fc3a805c9d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7fc3c416c980 tx=0x7fc3b40079b0).stop 2023-11-15T03:00:19.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.441+0000 7fc3cad07700 1 -- 172.21.15.121:0/94887650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3c4066520 msgr2=0x7fc3c416c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:19.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.441+0000 7fc3cad07700 1 --2- 172.21.15.121:0/94887650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3c4066520 0x7fc3c416c3d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7fc3c0009fd0 tx=0x7fc3c0017000).stop 2023-11-15T03:00:19.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.441+0000 7fc3cad07700 1 -- 172.21.15.121:0/94887650 shutdown_connections 2023-11-15T03:00:19.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.441+0000 7fc3cad07700 1 --2- 172.21.15.121:0/94887650 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc3a805a510 0x7fc3a805c9d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:19.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.441+0000 7fc3cad07700 1 --2- 172.21.15.121:0/94887650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3c40a8d40 0x7fc3c416b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:00:19.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.441+0000 7fc3cad07700 1 --2- 172.21.15.121:0/94887650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3c40a8960 0x7fc3c40c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:00:19.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.441+0000 7fc3cad07700 1 --2- 172.21.15.121:0/94887650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3c4066520 0x7fc3c416c3d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:19.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.441+0000 7fc3cad07700 1 -- 172.21.15.121:0/94887650 >> 172.21.15.121:0/94887650 conn(0x7fc3c4067a60 msgr2=0x7fc3c4067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:19.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.445+0000 7fc3cad07700 1 -- 172.21.15.121:0/94887650 shutdown_connections 2023-11-15T03:00:19.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.445+0000 7fc3cad07700 1 -- 172.21.15.121:0/94887650 wait complete. 2023-11-15T03:00:19.449 INFO:teuthology.orchestra.run.smithi121.stderr:unfreezing mds.6144 2023-11-15T03:00:19.469 DEBUG:teuthology.orchestra.run.smithi121:> sudo sysctl -n kernel.core_pattern 2023-11-15T03:00:19.498 INFO:teuthology.orchestra.run.smithi121.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-15T03:00:19.500 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-15T03:00:19.500 DEBUG:teuthology.orchestra.run.smithi121:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-15T03:00:19.598 INFO:teuthology.orchestra.run.smithi121.stdout:1700017201.43532.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/proc/self/exe', platform: 'x86_64' 2023-11-15T03:00:19.604 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1700017201.43532.core'] 2023-11-15T03:00:19.604 INFO:tasks.cephfs.cephfs_test_case:Found core file 1700017201.43532.core, deleting it 2023-11-15T03:00:19.605 DEBUG:teuthology.orchestra.run.smithi121:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1700017201.43532.core 2023-11-15T03:00:19.648 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-15T03:00:19.649 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-15T03:00:19.649 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-15T03:00:19.650 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:00:19.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:00:19.650 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi121 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2023-11-15T03:00:19.651 INFO:tasks.ceph.mds.a:Stopped 2023-11-15T03:00:19.651 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-15T03:00:19.697 INFO:tasks.ceph.mds.a:Started 2023-11-15T03:00:19.698 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:00:19.808 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:00:19.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.857+0000 7ff7617f9700 1 Processor -- start 2023-11-15T03:00:19.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.857+0000 7ff7617f9700 1 -- start start 2023-11-15T03:00:19.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.861+0000 7ff7617f9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff75c066700 0x7ff75c154110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:19.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.861+0000 7ff7617f9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff75c0acf10 0x7ff75c062190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:19.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.861+0000 7ff7617f9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c0c73e0 con 0x7ff75c066320 2023-11-15T03:00:19.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.861+0000 7ff7617f9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75c1546c0 con 0x7ff75c0acf10 2023-11-15T03:00:19.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.861+0000 7ff7617f9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75c1549a0 con 0x7ff75c066700 2023-11-15T03:00:19.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.861+0000 7ff75bfff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff75c066320 0x7ff75c0656a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:46844/0 (socket says 172.21.15.121:46844) 2023-11-15T03:00:19.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.861+0000 7ff75bfff700 1 -- 172.21.15.121:0/3552146730 learned_addr learned my addr 172.21.15.121:0/3552146730 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:00:19.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.861+0000 7ff760ff8700 1 --2- 172.21.15.121:0/3552146730 >> v2:172.21.15.121:3300/0 conn(0x7ff75c0acf10 0x7ff75c062190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:19.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.861+0000 7ff75b7fe700 1 --2- 172.21.15.121:0/3552146730 >> v2:172.21.15.190:3300/0 conn(0x7ff75c066700 0x7ff75c154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:19.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.861+0000 7ff75affd700 1 -- 172.21.15.121:0/3552146730 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2831185892 0 0) 0x7ff75c0c73e0 con 0x7ff75c066320 2023-11-15T03:00:19.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.861+0000 7ff75affd700 1 -- 172.21.15.121:0/3552146730 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7440033f0 con 0x7ff75c066320 2023-11-15T03:00:19.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.861+0000 7ff760ff8700 1 -- 172.21.15.121:0/3552146730 >> v2:172.21.15.190:3300/0 conn(0x7ff75c066700 msgr2=0x7ff75c154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:19.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.861+0000 7ff760ff8700 1 --2- 172.21.15.121:0/3552146730 >> v2:172.21.15.190:3300/0 conn(0x7ff75c066700 0x7ff75c154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:19.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.861+0000 7ff760ff8700 1 -- 172.21.15.121:0/3552146730 >> v1:172.21.15.121:6789/0 conn(0x7ff75c066320 legacy=0x7ff75c0656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:19.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.861+0000 7ff760ff8700 1 -- 172.21.15.121:0/3552146730 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c154c80 con 0x7ff75c0acf10 2023-11-15T03:00:19.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.865+0000 7ff760ff8700 1 --2- 172.21.15.121:0/3552146730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff75c0acf10 0x7ff75c062190 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7ff75c068cc0 tx=0x7ff754009780).ready entity=mon.0 client_cookie=557f51ebddbad084 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:19.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.865+0000 7ff75affd700 1 -- 172.21.15.121:0/3552146730 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75400e070 con 0x7ff75c0acf10 2023-11-15T03:00:19.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.865+0000 7ff75affd700 1 -- 172.21.15.121:0/3552146730 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75400bab0 con 0x7ff75c0acf10 2023-11-15T03:00:19.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.865+0000 7ff75affd700 1 -- 172.21.15.121:0/3552146730 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75400be30 con 0x7ff75c0acf10 2023-11-15T03:00:19.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.865+0000 7ff7617f9700 1 -- 172.21.15.121:0/3552146730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff75c0acf10 msgr2=0x7ff75c062190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:19.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.865+0000 7ff7617f9700 1 --2- 172.21.15.121:0/3552146730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff75c0acf10 0x7ff75c062190 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7ff75c068cc0 tx=0x7ff754009780).stop 2023-11-15T03:00:19.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.865+0000 7ff7617f9700 1 -- 172.21.15.121:0/3552146730 shutdown_connections 2023-11-15T03:00:19.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.865+0000 7ff7617f9700 1 --2- 172.21.15.121:0/3552146730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff75c0acf10 0x7ff75c062190 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:19.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.865+0000 7ff7617f9700 1 --2- 172.21.15.121:0/3552146730 >> v2:172.21.15.190:3300/0 conn(0x7ff75c066700 0x7ff75c154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:19.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.865+0000 7ff7617f9700 1 -- 172.21.15.121:0/3552146730 >> 172.21.15.121:0/3552146730 conn(0x7ff75c067a60 msgr2=0x7ff75c067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:19.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.865+0000 7ff7617f9700 1 -- 172.21.15.121:0/3552146730 shutdown_connections 2023-11-15T03:00:19.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.865+0000 7ff7617f9700 1 -- 172.21.15.121:0/3552146730 wait complete. 2023-11-15T03:00:19.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.865+0000 7ff7617f9700 1 Processor -- start 2023-11-15T03:00:19.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff7617f9700 1 -- start start 2023-11-15T03:00:19.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff7617f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff75c066320 0x7ff75c164440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:19.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff7617f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff75c066700 0x7ff75c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:19.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff7617f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff75c0acf10 0x7ff75c150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:19.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff7617f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c157a20 con 0x7ff75c066320 2023-11-15T03:00:19.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff7617f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0a4b20 con 0x7ff75c066700 2023-11-15T03:00:19.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff7617f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0a4e00 con 0x7ff75c0acf10 2023-11-15T03:00:19.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff75bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff75c066320 0x7ff75c164440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:19.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff75bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff75c066320 0x7ff75c164440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33894/0 (socket says 172.21.15.121:33894) 2023-11-15T03:00:19.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff75bfff700 1 -- 172.21.15.121:0/3923901862 learned_addr learned my addr 172.21.15.121:0/3923901862 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:19.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff75b7fe700 1 --2- 172.21.15.121:0/3923901862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff75c066700 0x7ff75c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:19.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff760ff8700 1 --2- 172.21.15.121:0/3923901862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff75c0acf10 0x7ff75c150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:19.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff75bfff700 1 -- 172.21.15.121:0/3923901862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff75c0acf10 msgr2=0x7ff75c150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:19.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff75bfff700 1 --2- 172.21.15.121:0/3923901862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff75c0acf10 0x7ff75c150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:19.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff75bfff700 1 -- 172.21.15.121:0/3923901862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff75c066700 msgr2=0x7ff75c0a42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:19.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff75bfff700 1 --2- 172.21.15.121:0/3923901862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff75c066700 0x7ff75c0a42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:19.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff75bfff700 1 -- 172.21.15.121:0/3923901862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff754009a20 con 0x7ff75c066320 2023-11-15T03:00:19.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff760ff8700 1 --2- 172.21.15.121:0/3923901862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff75c0acf10 0x7ff75c150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:00:19.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff75bfff700 1 --2- 172.21.15.121:0/3923901862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff75c066320 0x7ff75c164440 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7ff74c00c590 tx=0x7ff74c007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:19.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff743fff700 1 -- 172.21.15.121:0/3923901862 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c007e40 con 0x7ff75c066320 2023-11-15T03:00:19.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff743fff700 1 -- 172.21.15.121:0/3923901862 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74c0072e0 con 0x7ff75c066320 2023-11-15T03:00:19.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff743fff700 1 -- 172.21.15.121:0/3923901862 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c0057a0 con 0x7ff75c066320 2023-11-15T03:00:19.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff7617f9700 1 -- 172.21.15.121:0/3923901862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c0a50e0 con 0x7ff75c066320 2023-11-15T03:00:19.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.869+0000 7ff7617f9700 1 -- 172.21.15.121:0/3923901862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c0a47c0 con 0x7ff75c066320 2023-11-15T03:00:19.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.873+0000 7ff7617f9700 1 -- 172.21.15.121:0/3923901862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75c157850 con 0x7ff75c066320 2023-11-15T03:00:19.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.873+0000 7ff743fff700 1 -- 172.21.15.121:0/3923901862 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff74c005bd0 con 0x7ff75c066320 2023-11-15T03:00:19.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.873+0000 7ff743fff700 1 --2- 172.21.15.121:0/3923901862 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff74805a510 0x7ff74805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:19.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.873+0000 7ff743fff700 1 -- 172.21.15.121:0/3923901862 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6293+0+0 (secure 0 0 0) 0x7ff74c074150 con 0x7ff75c066320 2023-11-15T03:00:19.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.877+0000 7ff75b7fe700 1 --2- 172.21.15.121:0/3923901862 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff74805a510 0x7ff74805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:19.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.877+0000 7ff75b7fe700 1 --2- 172.21.15.121:0/3923901862 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff74805a510 0x7ff74805c9d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7ff75c164930 tx=0x7ff750008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:19.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:19.881+0000 7ff743fff700 1 -- 172.21.15.121:0/3923901862 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff74c047880 con 0x7ff75c066320 2023-11-15T03:00:20.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:20.081+0000 7ff7617f9700 1 -- 172.21.15.121:0/3923901862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff75c06bcf0 con 0x7ff75c066320 2023-11-15T03:00:20.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:20.081+0000 7ff743fff700 1 -- 172.21.15.121:0/3923901862 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3743 (secure 0 0 0) 0x7ff74c04aea0 con 0x7ff75c066320 2023-11-15T03:00:20.084 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:00:20.085 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":56,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3462858472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3462858472},{"type":"v1","addr":"172.21.15.190:6833","nonce":3462858472}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4042390087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4042390087},{"type":"v1","addr":"172.21.15.190:6835","nonce":4042390087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:59:34.667864+0000","modified":"2023-11-15T03:00:19.441101+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5907},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5907":{"gid":5907,"name":"c","rank":0,"incarnation":56,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6837/2239114529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2239114529},{"type":"v1","addr":"172.21.15.121:6837","nonce":2239114529}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-15T03:00:20.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:20.081+0000 7ff7617f9700 1 -- 172.21.15.121:0/3923901862 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff74805a510 msgr2=0x7ff74805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:20.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:20.081+0000 7ff7617f9700 1 --2- 172.21.15.121:0/3923901862 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff74805a510 0x7ff74805c9d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7ff75c164930 tx=0x7ff750008040).stop 2023-11-15T03:00:20.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:20.081+0000 7ff7617f9700 1 -- 172.21.15.121:0/3923901862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff75c066320 msgr2=0x7ff75c164440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:20.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:20.081+0000 7ff7617f9700 1 --2- 172.21.15.121:0/3923901862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff75c066320 0x7ff75c164440 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7ff74c00c590 tx=0x7ff74c007c80).stop 2023-11-15T03:00:20.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:20.081+0000 7ff7617f9700 1 -- 172.21.15.121:0/3923901862 shutdown_connections 2023-11-15T03:00:20.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:20.081+0000 7ff7617f9700 1 --2- 172.21.15.121:0/3923901862 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff74805a510 0x7ff74805c9d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:20.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:20.081+0000 7ff7617f9700 1 --2- 172.21.15.121:0/3923901862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff75c0acf10 0x7ff75c150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:20.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:20.081+0000 7ff7617f9700 1 --2- 172.21.15.121:0/3923901862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff75c066700 0x7ff75c0a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:20.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:20.081+0000 7ff7617f9700 1 --2- 172.21.15.121:0/3923901862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff75c066320 0x7ff75c164440 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:20.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:20.081+0000 7ff7617f9700 1 -- 172.21.15.121:0/3923901862 >> 172.21.15.121:0/3923901862 conn(0x7ff75c067a60 msgr2=0x7ff75c151270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:20.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:20.085+0000 7ff7617f9700 1 -- 172.21.15.121:0/3923901862 shutdown_connections 2023-11-15T03:00:20.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:20.085+0000 7ff7617f9700 1 -- 172.21.15.121:0/3923901862 wait complete. 2023-11-15T03:00:20.090 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 56 2023-11-15T03:00:20.511 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T02:59:34.667864+0000', 'modified': '2023-11-15T03:00:19.441101+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 63, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5907}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5907': {'gid': 5907, 'name': 'c', 'rank': 0, 'incarnation': 56, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.121:6837/2239114529', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 2239114529}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 2239114529}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:00:20.511 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5907:up:replay 2023-11-15T03:00:21.513 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:00:21.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.677+0000 7fe75d369700 1 Processor -- start 2023-11-15T03:00:21.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.681+0000 7fe75d369700 1 -- start start 2023-11-15T03:00:21.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.681+0000 7fe75d369700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe7580c06e0 0x7fe758153db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:21.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.681+0000 7fe75d369700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7580c2670 0x7fe7580c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:21.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.681+0000 7fe75d369700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7580c1530 con 0x7fe7581539d0 2023-11-15T03:00:21.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.681+0000 7fe75d369700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7580c2b00 con 0x7fe7580c06e0 2023-11-15T03:00:21.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.681+0000 7fe75d369700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7580c2d80 con 0x7fe7580c2670 2023-11-15T03:00:21.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.681+0000 7fe74e7fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe7580c06e0 0x7fe758153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:21.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.681+0000 7fe7577fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7580c2670 0x7fe7580c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:21.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.681+0000 7fe756ffd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe7581539d0 0x7fe7580c19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:38160/0 (socket says 172.21.15.121:38160) 2023-11-15T03:00:21.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.681+0000 7fe756ffd700 1 -- 172.21.15.121:0/2829816475 learned_addr learned my addr 172.21.15.121:0/2829816475 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:00:21.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.681+0000 7fe74e7fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe7580c06e0 0x7fe758153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:54514/0 (socket says 172.21.15.121:54514) 2023-11-15T03:00:21.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.681+0000 7fe7567fc700 1 -- 172.21.15.121:0/2829816475 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408762040 0 0) 0x7fe7580c1530 con 0x7fe7581539d0 2023-11-15T03:00:21.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.681+0000 7fe7567fc700 1 -- 172.21.15.121:0/2829816475 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7380033f0 con 0x7fe7581539d0 2023-11-15T03:00:21.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.681+0000 7fe74e7fc700 1 -- 172.21.15.121:0/2829816475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7580c2670 msgr2=0x7fe7580c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:21.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.681+0000 7fe74e7fc700 1 --2- 172.21.15.121:0/2829816475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7580c2670 0x7fe7580c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:21.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.681+0000 7fe74e7fc700 1 -- 172.21.15.121:0/2829816475 >> v1:172.21.15.190:6789/0 conn(0x7fe7581539d0 legacy=0x7fe7580c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:21.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.681+0000 7fe74e7fc700 1 -- 172.21.15.121:0/2829816475 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7580c0ee0 con 0x7fe7580c06e0 2023-11-15T03:00:21.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.681+0000 7fe7577fe700 1 --2- 172.21.15.121:0/2829816475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7580c2670 0x7fe7580c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:00:21.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.685+0000 7fe74e7fc700 1 --2- 172.21.15.121:0/2829816475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7580c06e0 0x7fe758153db0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fe740009760 tx=0x7fe740009060).ready entity=mon.1 client_cookie=a5e8318dc67c72c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:21.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.685+0000 7fe7567fc700 1 -- 172.21.15.121:0/2829816475 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74000d4a0 con 0x7fe7580c06e0 2023-11-15T03:00:21.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.685+0000 7fe7567fc700 1 -- 172.21.15.121:0/2829816475 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74000d600 con 0x7fe7580c06e0 2023-11-15T03:00:21.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.685+0000 7fe7567fc700 1 -- 172.21.15.121:0/2829816475 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74000d980 con 0x7fe7580c06e0 2023-11-15T03:00:21.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.685+0000 7fe75d369700 1 -- 172.21.15.121:0/2829816475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7580c06e0 msgr2=0x7fe758153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:21.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.685+0000 7fe75d369700 1 --2- 172.21.15.121:0/2829816475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7580c06e0 0x7fe758153db0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fe740009760 tx=0x7fe740009060).stop 2023-11-15T03:00:21.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.685+0000 7fe75d369700 1 -- 172.21.15.121:0/2829816475 shutdown_connections 2023-11-15T03:00:21.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.685+0000 7fe75d369700 1 --2- 172.21.15.121:0/2829816475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7580c2670 0x7fe7580c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:21.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.685+0000 7fe75d369700 1 --2- 172.21.15.121:0/2829816475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7580c06e0 0x7fe758153db0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:21.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.685+0000 7fe75d369700 1 -- 172.21.15.121:0/2829816475 >> 172.21.15.121:0/2829816475 conn(0x7fe758062190 msgr2=0x7fe7580625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:21.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.685+0000 7fe75d369700 1 -- 172.21.15.121:0/2829816475 shutdown_connections 2023-11-15T03:00:21.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.685+0000 7fe75d369700 1 -- 172.21.15.121:0/2829816475 wait complete. 2023-11-15T03:00:21.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.689+0000 7fe75d369700 1 Processor -- start 2023-11-15T03:00:21.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.689+0000 7fe75d369700 1 -- start start 2023-11-15T03:00:21.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.689+0000 7fe75d369700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7580c06e0 0x7fe758142d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:21.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.689+0000 7fe75d369700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7580c2670 0x7fe758146d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:21.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.689+0000 7fe75d369700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7581539d0 0x7fe758147db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:21.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.689+0000 7fe75d369700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe758147220 con 0x7fe7581539d0 2023-11-15T03:00:21.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.689+0000 7fe75d369700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe758149a50 con 0x7fe7580c06e0 2023-11-15T03:00:21.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.689+0000 7fe75d369700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75814a5b0 con 0x7fe7580c2670 2023-11-15T03:00:21.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.689+0000 7fe7577fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7581539d0 0x7fe758147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:21.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.689+0000 7fe7577fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7581539d0 0x7fe758147db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33900/0 (socket says 172.21.15.121:33900) 2023-11-15T03:00:21.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.689+0000 7fe7577fe700 1 -- 172.21.15.121:0/1694654444 learned_addr learned my addr 172.21.15.121:0/1694654444 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:21.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.689+0000 7fe756ffd700 1 --2- 172.21.15.121:0/1694654444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7580c06e0 0x7fe758142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:21.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.689+0000 7fe74e7fc700 1 --2- 172.21.15.121:0/1694654444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7580c2670 0x7fe758146d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:21.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.693+0000 7fe7577fe700 1 -- 172.21.15.121:0/1694654444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7580c2670 msgr2=0x7fe758146d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:21.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.693+0000 7fe7577fe700 1 --2- 172.21.15.121:0/1694654444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7580c2670 0x7fe758146d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:21.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.693+0000 7fe7577fe700 1 -- 172.21.15.121:0/1694654444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7580c06e0 msgr2=0x7fe758142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:21.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.693+0000 7fe7577fe700 1 --2- 172.21.15.121:0/1694654444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7580c06e0 0x7fe758142d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:21.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.693+0000 7fe7577fe700 1 -- 172.21.15.121:0/1694654444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe750009a20 con 0x7fe7581539d0 2023-11-15T03:00:21.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.693+0000 7fe756ffd700 1 --2- 172.21.15.121:0/1694654444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7580c06e0 0x7fe758142d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:00:21.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.693+0000 7fe7577fe700 1 --2- 172.21.15.121:0/1694654444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7581539d0 0x7fe758147db0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7fe750009fc0 tx=0x7fe75000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:21.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.693+0000 7fe74f7fe700 1 -- 172.21.15.121:0/1694654444 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75000cbd0 con 0x7fe7581539d0 2023-11-15T03:00:21.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.693+0000 7fe74f7fe700 1 -- 172.21.15.121:0/1694654444 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75000f040 con 0x7fe7581539d0 2023-11-15T03:00:21.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.693+0000 7fe75d369700 1 -- 172.21.15.121:0/1694654444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7400092a0 con 0x7fe7581539d0 2023-11-15T03:00:21.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.693+0000 7fe74f7fe700 1 -- 172.21.15.121:0/1694654444 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe750013400 con 0x7fe7581539d0 2023-11-15T03:00:21.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.693+0000 7fe75d369700 1 -- 172.21.15.121:0/1694654444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe758149c80 con 0x7fe7581539d0 2023-11-15T03:00:21.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.693+0000 7fe75d369700 1 -- 172.21.15.121:0/1694654444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75806c150 con 0x7fe7581539d0 2023-11-15T03:00:21.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.697+0000 7fe74f7fe700 1 -- 172.21.15.121:0/1694654444 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe7500137f0 con 0x7fe7581539d0 2023-11-15T03:00:21.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.697+0000 7fe74f7fe700 1 --2- 172.21.15.121:0/1694654444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe73c05a510 0x7fe73c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:21.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.697+0000 7fe74f7fe700 1 -- 172.21.15.121:0/1694654444 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fe750075270 con 0x7fe7581539d0 2023-11-15T03:00:21.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.697+0000 7fe756ffd700 1 --2- 172.21.15.121:0/1694654444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe73c05a510 0x7fe73c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:21.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.697+0000 7fe756ffd700 1 --2- 172.21.15.121:0/1694654444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe73c05a510 0x7fe73c05c9d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7fe748005750 tx=0x7fe748009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:21.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.701+0000 7fe74f7fe700 1 -- 172.21.15.121:0/1694654444 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe7500488e0 con 0x7fe7581539d0 2023-11-15T03:00:21.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.901+0000 7fe75d369700 1 -- 172.21.15.121:0/1694654444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe75814a6f0 con 0x7fe7581539d0 2023-11-15T03:00:21.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.905+0000 7fe74f7fe700 1 -- 172.21.15.121:0/1694654444 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4504 (secure 0 0 0) 0x7fe75814a6f0 con 0x7fe7581539d0 2023-11-15T03:00:21.908 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:00:21.908 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":58,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3462858472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3462858472},{"type":"v1","addr":"172.21.15.190:6833","nonce":3462858472}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4042390087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4042390087},{"type":"v1","addr":"172.21.15.190:6835","nonce":4042390087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3908033272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3908033272},{"type":"v1","addr":"172.21.15.121:6836","nonce":3908033272}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:59:34.667864+0000","modified":"2023-11-15T03:00:21.456216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5907},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5907":{"gid":5907,"name":"c","rank":0,"incarnation":56,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.121:6837/2239114529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2239114529},{"type":"v1","addr":"172.21.15.121:6837","nonce":2239114529}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-15T03:00:21.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.905+0000 7fe75d369700 1 -- 172.21.15.121:0/1694654444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe73c05a510 msgr2=0x7fe73c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:21.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.905+0000 7fe75d369700 1 --2- 172.21.15.121:0/1694654444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe73c05a510 0x7fe73c05c9d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7fe748005750 tx=0x7fe748009040).stop 2023-11-15T03:00:21.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.905+0000 7fe75d369700 1 -- 172.21.15.121:0/1694654444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7581539d0 msgr2=0x7fe758147db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:21.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.905+0000 7fe75d369700 1 --2- 172.21.15.121:0/1694654444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7581539d0 0x7fe758147db0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7fe750009fc0 tx=0x7fe75000b040).stop 2023-11-15T03:00:21.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.905+0000 7fe75d369700 1 -- 172.21.15.121:0/1694654444 shutdown_connections 2023-11-15T03:00:21.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.905+0000 7fe75d369700 1 --2- 172.21.15.121:0/1694654444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe73c05a510 0x7fe73c05c9d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:21.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.905+0000 7fe75d369700 1 --2- 172.21.15.121:0/1694654444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7581539d0 0x7fe758147db0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:21.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.905+0000 7fe75d369700 1 --2- 172.21.15.121:0/1694654444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7580c2670 0x7fe758146d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:21.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.905+0000 7fe75d369700 1 --2- 172.21.15.121:0/1694654444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7580c06e0 0x7fe758142d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:21.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.905+0000 7fe75d369700 1 -- 172.21.15.121:0/1694654444 >> 172.21.15.121:0/1694654444 conn(0x7fe758062190 msgr2=0x7fe75814d380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:21.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.909+0000 7fe75d369700 1 -- 172.21.15.121:0/1694654444 shutdown_connections 2023-11-15T03:00:21.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:21.909+0000 7fe75d369700 1 -- 172.21.15.121:0/1694654444 wait complete. 2023-11-15T03:00:21.914 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 58 2023-11-15T03:00:22.335 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T02:59:34.667864+0000', 'modified': '2023-11-15T03:00:21.456216+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 63, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5907}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5907': {'gid': 5907, 'name': 'c', 'rank': 0, 'incarnation': 56, 'state': 'up:rejoin', 'state_seq': 15, 'addr': '172.21.15.121:6837/2239114529', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 2239114529}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 2239114529}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:00:22.336 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5907:up:rejoin 2023-11-15T03:00:23.337 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:00:23.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.489+0000 7f48b92b9700 1 Processor -- start 2023-11-15T03:00:23.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.489+0000 7f48b92b9700 1 -- start start 2023-11-15T03:00:23.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.493+0000 7f48b92b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b40cafb0 0x7f48b4066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:23.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.493+0000 7f48b92b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b40ad3a0 con 0x7f48b4067270 2023-11-15T03:00:23.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.493+0000 7f48b92b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b40ad650 con 0x7f48b40cabd0 2023-11-15T03:00:23.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.493+0000 7f48b92b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b40cb410 con 0x7f48b40cafb0 2023-11-15T03:00:23.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.493+0000 7f48b37fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b40cafb0 0x7f48b4066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:23.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.493+0000 7f48b3fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f48b40cabd0 0x7f48b40acc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:38172/0 (socket says 172.21.15.121:38172) 2023-11-15T03:00:23.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.493+0000 7f48b3fff700 1 -- 172.21.15.121:0/3434272426 learned_addr learned my addr 172.21.15.121:0/3434272426 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:00:23.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.493+0000 7f48b2ffd700 1 -- 172.21.15.121:0/3434272426 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812146812 0 0) 0x7f48b40ad650 con 0x7f48b40cabd0 2023-11-15T03:00:23.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.493+0000 7f48b2ffd700 1 -- 172.21.15.121:0/3434272426 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f489c0033f0 con 0x7f48b40cabd0 2023-11-15T03:00:23.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.493+0000 7f48b2ffd700 1 -- 172.21.15.121:0/3434272426 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 563673459 0 0) 0x7f48b40ad3a0 con 0x7f48b4067270 2023-11-15T03:00:23.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.493+0000 7f48b37fe700 1 -- 172.21.15.121:0/3434272426 >> v1:172.21.15.190:6789/0 conn(0x7f48b40cabd0 legacy=0x7f48b40acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:23.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.493+0000 7f48b37fe700 1 -- 172.21.15.121:0/3434272426 >> v1:172.21.15.121:6789/0 conn(0x7f48b4067270 legacy=0x7f48b40ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:23.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.493+0000 7f48b37fe700 1 -- 172.21.15.121:0/3434272426 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b4065fc0 con 0x7f48b40cafb0 2023-11-15T03:00:23.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.497+0000 7f48b37fe700 1 --2- 172.21.15.121:0/3434272426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b40cafb0 0x7f48b4066d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f48a8009790 tx=0x7f48a8009060).ready entity=mon.2 client_cookie=2c6b3ab2b608fff5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:23.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.497+0000 7f48b2ffd700 1 -- 172.21.15.121:0/3434272426 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a800f510 con 0x7f48b40cafb0 2023-11-15T03:00:23.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.497+0000 7f48b2ffd700 1 -- 172.21.15.121:0/3434272426 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a800f670 con 0x7f48b40cafb0 2023-11-15T03:00:23.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.497+0000 7f48b2ffd700 1 -- 172.21.15.121:0/3434272426 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a800f9f0 con 0x7f48b40cafb0 2023-11-15T03:00:23.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.497+0000 7f48b92b9700 1 -- 172.21.15.121:0/3434272426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b40cafb0 msgr2=0x7f48b4066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:23.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.497+0000 7f48b92b9700 1 --2- 172.21.15.121:0/3434272426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b40cafb0 0x7f48b4066d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f48a8009790 tx=0x7f48a8009060).stop 2023-11-15T03:00:23.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.497+0000 7f48b92b9700 1 -- 172.21.15.121:0/3434272426 shutdown_connections 2023-11-15T03:00:23.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.497+0000 7f48b92b9700 1 --2- 172.21.15.121:0/3434272426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b40cafb0 0x7f48b4066d80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:23.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.497+0000 7f48b92b9700 1 -- 172.21.15.121:0/3434272426 >> 172.21.15.121:0/3434272426 conn(0x7f48b4062190 msgr2=0x7f48b40625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:23.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.497+0000 7f48b92b9700 1 -- 172.21.15.121:0/3434272426 shutdown_connections 2023-11-15T03:00:23.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.497+0000 7f48b92b9700 1 -- 172.21.15.121:0/3434272426 wait complete. 2023-11-15T03:00:23.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.497+0000 7f48b92b9700 1 Processor -- start 2023-11-15T03:00:23.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f48b92b9700 1 -- start start 2023-11-15T03:00:23.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f48b92b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b4067270 0x7f48b40bfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:23.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f48b92b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b40cabd0 0x7f48b414fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:23.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f48b92b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48b40cafb0 0x7f48b40c2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:23.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f48b92b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b4150110 con 0x7f48b40cabd0 2023-11-15T03:00:23.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f48b92b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b40c2ef0 con 0x7f48b40cafb0 2023-11-15T03:00:23.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f48b92b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b40c31d0 con 0x7f48b4067270 2023-11-15T03:00:23.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f48b37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b40cabd0 0x7f48b414fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:23.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f48b37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b40cabd0 0x7f48b414fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33916/0 (socket says 172.21.15.121:33916) 2023-11-15T03:00:23.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f48b37fe700 1 -- 172.21.15.121:0/3621586102 learned_addr learned my addr 172.21.15.121:0/3621586102 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:23.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f48b8ab8700 1 --2- 172.21.15.121:0/3621586102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48b40cafb0 0x7f48b40c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:23.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f48b3fff700 1 --2- 172.21.15.121:0/3621586102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b4067270 0x7f48b40bfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:23.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f48b37fe700 1 -- 172.21.15.121:0/3621586102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b4067270 msgr2=0x7f48b40bfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:23.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f48b37fe700 1 --2- 172.21.15.121:0/3621586102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b4067270 0x7f48b40bfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:23.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f48b37fe700 1 -- 172.21.15.121:0/3621586102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48b40cafb0 msgr2=0x7f48b40c2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:23.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f48b37fe700 1 --2- 172.21.15.121:0/3621586102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48b40cafb0 0x7f48b40c2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:23.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f48b37fe700 1 -- 172.21.15.121:0/3621586102 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a80092a0 con 0x7f48b40cabd0 2023-11-15T03:00:23.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f48b37fe700 1 --2- 172.21.15.121:0/3621586102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b40cabd0 0x7f48b414fc20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f48a8009e30 tx=0x7f48a8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:23.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f489bfff700 1 -- 172.21.15.121:0/3621586102 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a800fe80 con 0x7f48b40cabd0 2023-11-15T03:00:23.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f489bfff700 1 -- 172.21.15.121:0/3621586102 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a800f7e0 con 0x7f48b40cabd0 2023-11-15T03:00:23.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f489bfff700 1 -- 172.21.15.121:0/3621586102 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a801b620 con 0x7f48b40cabd0 2023-11-15T03:00:23.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.501+0000 7f48b92b9700 1 -- 172.21.15.121:0/3621586102 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b40c34b0 con 0x7f48b40cabd0 2023-11-15T03:00:23.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.505+0000 7f48b92b9700 1 -- 172.21.15.121:0/3621586102 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b40c2b70 con 0x7f48b40cabd0 2023-11-15T03:00:23.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.505+0000 7f48b92b9700 1 -- 172.21.15.121:0/3621586102 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b4155550 con 0x7f48b40cabd0 2023-11-15T03:00:23.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.505+0000 7f489bfff700 1 -- 172.21.15.121:0/3621586102 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f48a801b990 con 0x7f48b40cabd0 2023-11-15T03:00:23.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.509+0000 7f489bfff700 1 --2- 172.21.15.121:0/3621586102 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f48a005a510 0x7f48a005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:23.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.509+0000 7f489bfff700 1 -- 172.21.15.121:0/3621586102 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f48a8075d50 con 0x7f48b40cabd0 2023-11-15T03:00:23.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.509+0000 7f48b3fff700 1 --2- 172.21.15.121:0/3621586102 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f48a005a510 0x7f48a005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:23.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.509+0000 7f48b3fff700 1 --2- 172.21.15.121:0/3621586102 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f48a005a510 0x7f48a005c9d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f48a4005750 tx=0x7f48a4009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:23.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.513+0000 7f489bfff700 1 -- 172.21.15.121:0/3621586102 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f48b4155550 con 0x7f48b40cabd0 2023-11-15T03:00:23.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.713+0000 7f48b92b9700 1 -- 172.21.15.121:0/3621586102 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48b4155550 con 0x7f48b40cabd0 2023-11-15T03:00:23.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.717+0000 7f489bfff700 1 -- 172.21.15.121:0/3621586102 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4504 (secure 0 0 0) 0x7f48b4155550 con 0x7f48b40cabd0 2023-11-15T03:00:23.719 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:00:23.720 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3462858472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3462858472},{"type":"v1","addr":"172.21.15.190:6833","nonce":3462858472}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4042390087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4042390087},{"type":"v1","addr":"172.21.15.190:6835","nonce":4042390087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3908033272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3908033272},{"type":"v1","addr":"172.21.15.121:6836","nonce":3908033272}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:59:34.667864+0000","modified":"2023-11-15T03:00:22.468286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5907},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5907":{"gid":5907,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":16,"addr":"172.21.15.121:6837/2239114529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2239114529},{"type":"v1","addr":"172.21.15.121:6837","nonce":2239114529}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-15T03:00:23.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.717+0000 7f48b92b9700 1 -- 172.21.15.121:0/3621586102 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f48a005a510 msgr2=0x7f48a005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:23.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.717+0000 7f48b92b9700 1 --2- 172.21.15.121:0/3621586102 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f48a005a510 0x7f48a005c9d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f48a4005750 tx=0x7f48a4009040).stop 2023-11-15T03:00:23.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.717+0000 7f48b92b9700 1 -- 172.21.15.121:0/3621586102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b40cabd0 msgr2=0x7f48b414fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:23.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.717+0000 7f48b92b9700 1 --2- 172.21.15.121:0/3621586102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b40cabd0 0x7f48b414fc20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f48a8009e30 tx=0x7f48a8017000).stop 2023-11-15T03:00:23.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.717+0000 7f48b92b9700 1 -- 172.21.15.121:0/3621586102 shutdown_connections 2023-11-15T03:00:23.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.717+0000 7f48b92b9700 1 --2- 172.21.15.121:0/3621586102 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f48a005a510 0x7f48a005c9d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:23.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.717+0000 7f48b92b9700 1 --2- 172.21.15.121:0/3621586102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48b40cafb0 0x7f48b40c2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:23.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.717+0000 7f48b92b9700 1 --2- 172.21.15.121:0/3621586102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b40cabd0 0x7f48b414fc20 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:23.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.717+0000 7f48b92b9700 1 --2- 172.21.15.121:0/3621586102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b4067270 0x7f48b40bfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:23.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.717+0000 7f48b92b9700 1 -- 172.21.15.121:0/3621586102 >> 172.21.15.121:0/3621586102 conn(0x7f48b4062190 msgr2=0x7f48b40af920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:23.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.717+0000 7f48b92b9700 1 -- 172.21.15.121:0/3621586102 shutdown_connections 2023-11-15T03:00:23.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:23.717+0000 7f48b92b9700 1 -- 172.21.15.121:0/3621586102 wait complete. 2023-11-15T03:00:23.724 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 59 2023-11-15T03:00:24.142 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T02:59:34.667864+0000', 'modified': '2023-11-15T03:00:22.468286+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 63, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5907}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5907': {'gid': 5907, 'name': 'c', 'rank': 0, 'incarnation': 56, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.121:6837/2239114529', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 2239114529}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 2239114529}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:00:24.143 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T03:00:24.143 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-15T03:00:24.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.309+0000 7f14146db700 1 Processor -- start 2023-11-15T03:00:24.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.309+0000 7f14146db700 1 -- start start 2023-11-15T03:00:24.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.313+0000 7f14146db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f140c151920 0x7f140c0bb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:24.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.313+0000 7f14146db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f140c155de0 con 0x7f140c0ac150 2023-11-15T03:00:24.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.313+0000 7f14146db700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f140c0c3aa0 con 0x7f140c067150 2023-11-15T03:00:24.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.313+0000 7f14146db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f140c0c4060 con 0x7f140c151920 2023-11-15T03:00:24.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.313+0000 7f1412c78700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f140c0ac150 0x7f140c062190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:46872/0 (socket says 172.21.15.121:46872) 2023-11-15T03:00:24.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.313+0000 7f1412c78700 1 -- 172.21.15.121:0/3551690611 learned_addr learned my addr 172.21.15.121:0/3551690611 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:00:24.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.313+0000 7f1412477700 1 --2- 172.21.15.121:0/3551690611 >> v2:172.21.15.190:3300/0 conn(0x7f140c151920 0x7f140c0bb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:24.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.313+0000 7f1411475700 1 -- 172.21.15.121:0/3551690611 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462314280 0 0) 0x7f140c155de0 con 0x7f140c0ac150 2023-11-15T03:00:24.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.313+0000 7f1411475700 1 -- 172.21.15.121:0/3551690611 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13f80033f0 con 0x7f140c0ac150 2023-11-15T03:00:24.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.313+0000 7f1412477700 1 -- 172.21.15.121:0/3551690611 >> v1:172.21.15.190:6789/0 conn(0x7f140c067150 legacy=0x7f140c0aba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:24.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.313+0000 7f1412477700 1 -- 172.21.15.121:0/3551690611 >> v1:172.21.15.121:6789/0 conn(0x7f140c0ac150 legacy=0x7f140c062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:24.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.313+0000 7f1412477700 1 -- 172.21.15.121:0/3551690611 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140c0c3d80 con 0x7f140c151920 2023-11-15T03:00:24.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.313+0000 7f1412477700 1 --2- 172.21.15.121:0/3551690611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f140c151920 0x7f140c0bb6b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f1408009760 tx=0x7f1408009060).ready entity=mon.1 client_cookie=ed895706f13f82d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:24.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.317+0000 7f1411475700 1 -- 172.21.15.121:0/3551690611 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140800f510 con 0x7f140c151920 2023-11-15T03:00:24.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.317+0000 7f1411475700 1 -- 172.21.15.121:0/3551690611 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140800f670 con 0x7f140c151920 2023-11-15T03:00:24.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.317+0000 7f1411475700 1 -- 172.21.15.121:0/3551690611 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140800f9f0 con 0x7f140c151920 2023-11-15T03:00:24.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.317+0000 7f14146db700 1 -- 172.21.15.121:0/3551690611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f140c151920 msgr2=0x7f140c0bb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:24.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.317+0000 7f14146db700 1 --2- 172.21.15.121:0/3551690611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f140c151920 0x7f140c0bb6b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f1408009760 tx=0x7f1408009060).stop 2023-11-15T03:00:24.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.317+0000 7f14146db700 1 -- 172.21.15.121:0/3551690611 shutdown_connections 2023-11-15T03:00:24.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.317+0000 7f14146db700 1 --2- 172.21.15.121:0/3551690611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f140c151920 0x7f140c0bb6b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.317+0000 7f14146db700 1 -- 172.21.15.121:0/3551690611 >> 172.21.15.121:0/3551690611 conn(0x7f140c067a60 msgr2=0x7f140c067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:24.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.317+0000 7f14146db700 1 -- 172.21.15.121:0/3551690611 shutdown_connections 2023-11-15T03:00:24.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.317+0000 7f14146db700 1 -- 172.21.15.121:0/3551690611 wait complete. 2023-11-15T03:00:24.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.317+0000 7f14146db700 1 Processor -- start 2023-11-15T03:00:24.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.317+0000 7f14146db700 1 -- start start 2023-11-15T03:00:24.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f14146db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f140c067150 0x7f140c144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:24.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f14146db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f140c0ac150 0x7f140c0c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:24.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f14146db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f140c151920 0x7f140c0c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:24.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f14146db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c0c5cc0 con 0x7f140c151920 2023-11-15T03:00:24.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f14146db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c144780 con 0x7f140c0ac150 2023-11-15T03:00:24.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f14146db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f140c14e630 con 0x7f140c067150 2023-11-15T03:00:24.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f1412c78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f140c151920 0x7f140c0c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:24.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f1412c78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f140c151920 0x7f140c0c57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33926/0 (socket says 172.21.15.121:33926) 2023-11-15T03:00:24.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f1412c78700 1 -- 172.21.15.121:0/946897614 learned_addr learned my addr 172.21.15.121:0/946897614 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:24.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f1412477700 1 --2- 172.21.15.121:0/946897614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f140c067150 0x7f140c144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:24.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f1411c76700 1 --2- 172.21.15.121:0/946897614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f140c0ac150 0x7f140c0c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:24.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f1412c78700 1 -- 172.21.15.121:0/946897614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f140c067150 msgr2=0x7f140c144290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:24.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f1412c78700 1 --2- 172.21.15.121:0/946897614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f140c067150 0x7f140c144290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f1412c78700 1 -- 172.21.15.121:0/946897614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f140c0ac150 msgr2=0x7f140c0c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:24.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f1412c78700 1 --2- 172.21.15.121:0/946897614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f140c0ac150 0x7f140c0c52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f1412c78700 1 -- 172.21.15.121:0/946897614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14080092a0 con 0x7f140c151920 2023-11-15T03:00:24.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f1411c76700 1 --2- 172.21.15.121:0/946897614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f140c0ac150 0x7f140c0c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:00:24.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f1412c78700 1 --2- 172.21.15.121:0/946897614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f140c151920 0x7f140c0c57d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f1404001460 tx=0x7f1404001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:24.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f14027fc700 1 -- 172.21.15.121:0/946897614 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1404005d30 con 0x7f140c151920 2023-11-15T03:00:24.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f14027fc700 1 -- 172.21.15.121:0/946897614 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1404005e90 con 0x7f140c151920 2023-11-15T03:00:24.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f14146db700 1 -- 172.21.15.121:0/946897614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140c14da20 con 0x7f140c151920 2023-11-15T03:00:24.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f14027fc700 1 -- 172.21.15.121:0/946897614 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1404007320 con 0x7f140c151920 2023-11-15T03:00:24.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.321+0000 7f14146db700 1 -- 172.21.15.121:0/946897614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140c14dfe0 con 0x7f140c151920 2023-11-15T03:00:24.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.325+0000 7f14027fc700 1 -- 172.21.15.121:0/946897614 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1404007710 con 0x7f140c151920 2023-11-15T03:00:24.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.329+0000 7f14027fc700 1 --2- 172.21.15.121:0/946897614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f13f005a510 0x7f13f005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:24.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.329+0000 7f14027fc700 1 -- 172.21.15.121:0/946897614 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f14040749d0 con 0x7f140c151920 2023-11-15T03:00:24.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.329+0000 7f1412477700 1 --2- 172.21.15.121:0/946897614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f13f005a510 0x7f13f005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:24.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.329+0000 7f1412477700 1 --2- 172.21.15.121:0/946897614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f13f005a510 0x7f13f005c9d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f140c0c6160 tx=0x7f1408017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:24.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.329+0000 7f141ea68740 1 Processor -- start 2023-11-15T03:00:24.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.329+0000 7f141ea68740 1 -- start start 2023-11-15T03:00:24.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f141ea68740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2410300 0x24575b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:24.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f141ea68740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x240f510 0x24106e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:24.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f141ea68740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2455ab0 0x2455f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:24.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f141ea68740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2390410 con 0x240f510 2023-11-15T03:00:24.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f141ea68740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x2390550 con 0x2410300 2023-11-15T03:00:24.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f141ea68740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2390a00 con 0x2455ab0 2023-11-15T03:00:24.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f1411c76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x240f510 0x24106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:24.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f1411c76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x240f510 0x24106e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33934/0 (socket says 172.21.15.121:33934) 2023-11-15T03:00:24.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f1411c76700 1 -- 172.21.15.121:0/237208149 learned_addr learned my addr 172.21.15.121:0/237208149 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:24.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f1412477700 1 --2- 172.21.15.121:0/237208149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2410300 0x24575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:24.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f1412c78700 1 --2- 172.21.15.121:0/237208149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2455ab0 0x2455f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:24.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f1411c76700 1 -- 172.21.15.121:0/237208149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2455ab0 msgr2=0x2455f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:24.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f1411c76700 1 --2- 172.21.15.121:0/237208149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2455ab0 0x2455f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f1411c76700 1 -- 172.21.15.121:0/237208149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2410300 msgr2=0x24575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:24.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f1411c76700 1 --2- 172.21.15.121:0/237208149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2410300 0x24575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f1411c76700 1 -- 172.21.15.121:0/237208149 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2389730 con 0x240f510 2023-11-15T03:00:24.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f1412c78700 1 --2- 172.21.15.121:0/237208149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2455ab0 0x2455f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:00:24.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f1412477700 1 --2- 172.21.15.121:0/237208149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2410300 0x24575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:00:24.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f1411c76700 1 --2- 172.21.15.121:0/237208149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x240f510 0x24106e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f13fc007b20 tx=0x7f13fc009040).ready entity=mon.0 client_cookie=3c2594361db57c40 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:24.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f13ef7fe700 1 -- 172.21.15.121:0/237208149 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc010520 con 0x240f510 2023-11-15T03:00:24.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f13ef7fe700 1 -- 172.21.15.121:0/237208149 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13fc010680 con 0x240f510 2023-11-15T03:00:24.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f13ef7fe700 1 -- 172.21.15.121:0/237208149 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc010a00 con 0x240f510 2023-11-15T03:00:24.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f141ea68740 1 -- 172.21.15.121:0/237208149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x240f510 msgr2=0x24106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:24.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f141ea68740 1 --2- 172.21.15.121:0/237208149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x240f510 0x24106e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f13fc007b20 tx=0x7f13fc009040).stop 2023-11-15T03:00:24.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f141ea68740 1 -- 172.21.15.121:0/237208149 shutdown_connections 2023-11-15T03:00:24.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f141ea68740 1 --2- 172.21.15.121:0/237208149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2455ab0 0x2455f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f141ea68740 1 --2- 172.21.15.121:0/237208149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x240f510 0x24106e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.333+0000 7f141ea68740 1 --2- 172.21.15.121:0/237208149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2410300 0x24575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.337+0000 7f141ea68740 1 -- 172.21.15.121:0/237208149 >> 172.21.15.121:0/237208149 conn(0x21ee550 msgr2=0x2456a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:24.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.337+0000 7f141ea68740 1 -- 172.21.15.121:0/237208149 shutdown_connections 2023-11-15T03:00:24.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.337+0000 7f141ea68740 1 -- 172.21.15.121:0/237208149 wait complete. 2023-11-15T03:00:24.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.337+0000 7f141ea68740 1 -- start start 2023-11-15T03:00:24.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.337+0000 7f141ea68740 1 Processor -- start 2023-11-15T03:00:24.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f141ea68740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x240f510 0x2433840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:24.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f141ea68740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2410300 0x24383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:24.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f141ea68740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2455ab0 0x24396e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:24.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f141ea68740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x235b650 con 0x2410300 2023-11-15T03:00:24.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f141ea68740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x243a320 con 0x2455ab0 2023-11-15T03:00:24.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f141ea68740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2437f30 con 0x240f510 2023-11-15T03:00:24.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f141ea68740 10 client.0 ll_register_callbacks cb 0x23c8020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:00:24.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f1411c76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2410300 0x24383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:24.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f1411c76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2410300 0x24383c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33944/0 (socket says 172.21.15.121:33944) 2023-11-15T03:00:24.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f1411c76700 1 -- 172.21.15.121:0/684618904 learned_addr learned my addr 172.21.15.121:0/684618904 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:24.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f1412477700 1 --2- 172.21.15.121:0/684618904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x240f510 0x2433840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:24.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f1412c78700 1 --2- 172.21.15.121:0/684618904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2455ab0 0x24396e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:24.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f1411c76700 1 -- 172.21.15.121:0/684618904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x240f510 msgr2=0x2433840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:24.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f1411c76700 1 --2- 172.21.15.121:0/684618904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x240f510 0x2433840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f1411c76700 1 -- 172.21.15.121:0/684618904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2455ab0 msgr2=0x24396e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:24.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f1411c76700 1 --2- 172.21.15.121:0/684618904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2455ab0 0x24396e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f1411c76700 1 -- 172.21.15.121:0/684618904 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13fc00f040 con 0x2410300 2023-11-15T03:00:24.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f1412c78700 1 --2- 172.21.15.121:0/684618904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2455ab0 0x24396e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:00:24.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f1411c76700 1 --2- 172.21.15.121:0/684618904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2410300 0x24383c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f13fc002000 tx=0x7f13fc009040).ready entity=mon.0 client_cookie=960b7f54417d7161 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:24.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f13ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:00:24.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f13ecff9700 1 -- 172.21.15.121:0/684618904 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc007240 con 0x2410300 2023-11-15T03:00:24.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f13ecff9700 1 -- 172.21.15.121:0/684618904 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f13fc020c10 con 0x2410300 2023-11-15T03:00:24.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f141ea68740 1 -- 172.21.15.121:0/684618904 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x23861a0 con 0x2410300 2023-11-15T03:00:24.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f13ecff9700 1 -- 172.21.15.121:0/684618904 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13fc0073a0 con 0x2410300 2023-11-15T03:00:24.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f141ea68740 10 client.6297 fetch_fsmap learned FSMap version 59 2023-11-15T03:00:24.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f141ea68740 1 -- 172.21.15.121:0/684618904 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x2384960 con 0x2410300 2023-11-15T03:00:24.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f13ecff9700 1 -- 172.21.15.121:0/684618904 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x23861a0 con 0x2410300 2023-11-15T03:00:24.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f13ecff9700 1 -- 172.21.15.121:0/684618904 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 59) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f13fc007500 con 0x2410300 2023-11-15T03:00:24.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f141ea68740 10 client.6297 fetch_fsmap finished waiting for FSMap version 59 2023-11-15T03:00:24.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f141ea68740 10 client.6297 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-15T03:00:24.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f141ea68740 20 client.6297 populate_metadata read hostname 'smithi121' 2023-11-15T03:00:24.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f141ea68740 1 --2- 172.21.15.121:0/684618904 >> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] conn(0x2435f60 0x2436340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:24.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f141ea68740 4 client.6297 mds_command: new command op to 5907 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:00:24.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.341+0000 7f141ea68740 1 -- 172.21.15.121:0/684618904 --> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x237c6f0 con 0x2435f60 2023-11-15T03:00:24.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.345+0000 7f1412477700 1 --2- 172.21.15.121:0/684618904 >> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] conn(0x2435f60 0x2436340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:24.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.345+0000 7f1412477700 1 --2- 172.21.15.121:0/684618904 >> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] conn(0x2435f60 0x2436340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=23ba6a3f38419b28 server_cookie=3f38f8fa6de0ec0f in_seq=0 out_seq=0 2023-11-15T03:00:24.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.345+0000 7f13ecff9700 10 client.6297 ms_handle_connect on v2:172.21.15.121:6834/2239114529 2023-11-15T03:00:24.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.345+0000 7f13ecff9700 1 -- 172.21.15.121:0/684618904 <== mds.0 v2:172.21.15.121:6834/2239114529 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x237c6f0 con 0x2435f60 2023-11-15T03:00:24.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.345+0000 7f13ecff9700 10 client.6297 handle_command_reply: tid=0 2023-11-15T03:00:24.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.345+0000 7f141ea68740 1 client.6297 shutdown 2023-11-15T03:00:24.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.345+0000 7f141ea68740 1 -- 172.21.15.121:0/684618904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2410300 msgr2=0x24383c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:24.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.345+0000 7f141ea68740 1 --2- 172.21.15.121:0/684618904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2410300 0x24383c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f13fc002000 tx=0x7f13fc009040).stop 2023-11-15T03:00:24.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.345+0000 7f141ea68740 1 -- 172.21.15.121:0/684618904 shutdown_connections 2023-11-15T03:00:24.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.345+0000 7f141ea68740 1 --2- 172.21.15.121:0/684618904 >> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] conn(0x2435f60 0x2436340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.345+0000 7f141ea68740 1 --2- 172.21.15.121:0/684618904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2455ab0 0x24396e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.345+0000 7f141ea68740 1 --2- 172.21.15.121:0/684618904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2410300 0x24383c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.345+0000 7f141ea68740 1 --2- 172.21.15.121:0/684618904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x240f510 0x2433840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.345+0000 7f13ecff9700 0 client.6297 ms_handle_reset on v2:172.21.15.121:6834/2239114529 2023-11-15T03:00:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.345+0000 7f141ea68740 1 -- 172.21.15.121:0/684618904 >> 172.21.15.121:0/684618904 conn(0x21ee550 msgr2=0x2409800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:24.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.345+0000 7f141ea68740 1 -- 172.21.15.121:0/684618904 shutdown_connections 2023-11-15T03:00:24.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.345+0000 7f141ea68740 1 -- 172.21.15.121:0/684618904 wait complete. 2023-11-15T03:00:24.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.345+0000 7f141ea68740 20 client.6297 trim_cache size 0 max 16384 2023-11-15T03:00:24.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.365+0000 7f141ea68740 1 Processor -- start 2023-11-15T03:00:24.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.365+0000 7f141ea68740 1 -- start start 2023-11-15T03:00:24.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.365+0000 7f141ea68740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x240f510 0x2433830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:24.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f141ea68740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2410300 0x2464c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:24.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f141ea68740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2455ab0 0x24364f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:24.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f141ea68740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2452ca0 con 0x240f510 2023-11-15T03:00:24.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f141ea68740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x237c6f0 con 0x2410300 2023-11-15T03:00:24.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f141ea68740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2454e70 con 0x2455ab0 2023-11-15T03:00:24.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f1412477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x240f510 0x2433830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:24.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f1412477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x240f510 0x2433830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33950/0 (socket says 172.21.15.121:33950) 2023-11-15T03:00:24.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f1412477700 1 -- 172.21.15.121:0/2108135960 learned_addr learned my addr 172.21.15.121:0/2108135960 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:24.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f1411c76700 1 --2- 172.21.15.121:0/2108135960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2410300 0x2464c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:24.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f1412c78700 1 --2- 172.21.15.121:0/2108135960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2455ab0 0x24364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:24.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f1412477700 1 -- 172.21.15.121:0/2108135960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2455ab0 msgr2=0x24364f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:24.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f1412477700 1 --2- 172.21.15.121:0/2108135960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2455ab0 0x24364f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f1412477700 1 -- 172.21.15.121:0/2108135960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2410300 msgr2=0x2464c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:24.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f1412477700 1 --2- 172.21.15.121:0/2108135960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2410300 0x2464c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f1412477700 1 -- 172.21.15.121:0/2108135960 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140c065c00 con 0x240f510 2023-11-15T03:00:24.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f1412477700 1 --2- 172.21.15.121:0/2108135960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x240f510 0x2433830 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f140c063390 tx=0x7f140802f280).ready entity=mon.0 client_cookie=a432d929fbd5f9c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:24.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f13effff700 1 -- 172.21.15.121:0/2108135960 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140802c070 con 0x240f510 2023-11-15T03:00:24.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f13effff700 1 -- 172.21.15.121:0/2108135960 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140801c5f0 con 0x240f510 2023-11-15T03:00:24.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f13effff700 1 -- 172.21.15.121:0/2108135960 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140801c760 con 0x240f510 2023-11-15T03:00:24.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f141ea68740 1 -- 172.21.15.121:0/2108135960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x240f510 msgr2=0x2433830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:24.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f141ea68740 1 --2- 172.21.15.121:0/2108135960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x240f510 0x2433830 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f140c063390 tx=0x7f140802f280).stop 2023-11-15T03:00:24.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f141ea68740 1 -- 172.21.15.121:0/2108135960 shutdown_connections 2023-11-15T03:00:24.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f141ea68740 1 --2- 172.21.15.121:0/2108135960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2455ab0 0x24364f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f141ea68740 1 --2- 172.21.15.121:0/2108135960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2410300 0x2464c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f141ea68740 1 --2- 172.21.15.121:0/2108135960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x240f510 0x2433830 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f141ea68740 1 -- 172.21.15.121:0/2108135960 >> 172.21.15.121:0/2108135960 conn(0x21ee550 msgr2=0x2409820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:24.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f141ea68740 1 -- 172.21.15.121:0/2108135960 shutdown_connections 2023-11-15T03:00:24.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.369+0000 7f141ea68740 1 -- 172.21.15.121:0/2108135960 wait complete. 2023-11-15T03:00:24.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.373+0000 7f141ea68740 1 -- start start 2023-11-15T03:00:24.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.373+0000 7f141ea68740 1 Processor -- start 2023-11-15T03:00:24.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.373+0000 7f141ea68740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x240f510 0x244ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:24.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.373+0000 7f141ea68740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2410300 0x2453120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:24.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f141ea68740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2455ab0 0x2453610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:24.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f141ea68740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2453b00 con 0x2410300 2023-11-15T03:00:24.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f141ea68740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x2464280 con 0x240f510 2023-11-15T03:00:24.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f141ea68740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2456750 con 0x2455ab0 2023-11-15T03:00:24.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f141ea68740 10 client.0 ll_register_callbacks cb 0x7f13fc0011b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:00:24.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f1411c76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2410300 0x2453120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:24.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f1411c76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2410300 0x2453120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33964/0 (socket says 172.21.15.121:33964) 2023-11-15T03:00:24.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f1411c76700 1 -- 172.21.15.121:0/3517603272 learned_addr learned my addr 172.21.15.121:0/3517603272 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:00:24.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f1412c78700 1 --2- 172.21.15.121:0/3517603272 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2455ab0 0x2453610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:24.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f1412477700 1 --2- 172.21.15.121:0/3517603272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x240f510 0x244ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:24.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f1411c76700 1 -- 172.21.15.121:0/3517603272 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2455ab0 msgr2=0x2453610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:24.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f1411c76700 1 --2- 172.21.15.121:0/3517603272 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2455ab0 0x2453610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f1411c76700 1 -- 172.21.15.121:0/3517603272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x240f510 msgr2=0x244ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:00:24.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f1411c76700 1 --2- 172.21.15.121:0/3517603272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x240f510 0x244ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f1411c76700 1 -- 172.21.15.121:0/3517603272 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140c0c3d80 con 0x2410300 2023-11-15T03:00:24.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f1412477700 1 --2- 172.21.15.121:0/3517603272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x240f510 0x244ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:00:24.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f1411c76700 1 --2- 172.21.15.121:0/3517603272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2410300 0x2453120 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f13fc001ed0 tx=0x7f13fc01f000).ready entity=mon.0 client_cookie=6d8abfd6aa47f01c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:00:24.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f13ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:00:24.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f13ecff9700 1 -- 172.21.15.121:0/3517603272 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc035070 con 0x2410300 2023-11-15T03:00:24.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f13ecff9700 1 -- 172.21.15.121:0/3517603272 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f13fc020c10 con 0x2410300 2023-11-15T03:00:24.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f141ea68740 1 -- 172.21.15.121:0/3517603272 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13fc010680 con 0x2410300 2023-11-15T03:00:24.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f13ecff9700 1 -- 172.21.15.121:0/3517603272 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13fc006ad0 con 0x2410300 2023-11-15T03:00:24.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f141ea68740 10 client.6303 fetch_fsmap learned FSMap version 59 2023-11-15T03:00:24.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f141ea68740 1 -- 172.21.15.121:0/3517603272 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x23846b0 con 0x2410300 2023-11-15T03:00:24.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f13ecff9700 1 -- 172.21.15.121:0/3517603272 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13fc010680 con 0x2410300 2023-11-15T03:00:24.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f13ecff9700 1 -- 172.21.15.121:0/3517603272 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 59) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f13fc038030 con 0x2410300 2023-11-15T03:00:24.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f141ea68740 10 client.6303 fetch_fsmap finished waiting for FSMap version 59 2023-11-15T03:00:24.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f141ea68740 10 client.6303 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-15T03:00:24.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f141ea68740 20 client.6303 populate_metadata read hostname 'smithi121' 2023-11-15T03:00:24.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f141ea68740 1 --2- 172.21.15.121:0/3517603272 >> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] conn(0x2432380 0x2432760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:00:24.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f141ea68740 4 client.6303 mds_command: new command op to 5907 tid=0[{"prefix": "status"}] 2023-11-15T03:00:24.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f141ea68740 1 -- 172.21.15.121:0/3517603272 --> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] -- command(tid 0: {"prefix": "status"}) v1 -- 0x22454e0 con 0x2432380 2023-11-15T03:00:24.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.377+0000 7f1412477700 1 --2- 172.21.15.121:0/3517603272 >> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] conn(0x2432380 0x2432760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:24.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f1412477700 1 --2- 172.21.15.121:0/3517603272 >> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] conn(0x2432380 0x2432760 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=94f8c149dad321be server_cookie=5a21fabd7122d654 in_seq=0 out_seq=0 2023-11-15T03:00:24.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f13ecff9700 10 client.6303 ms_handle_connect on v2:172.21.15.121:6834/2239114529 2023-11-15T03:00:24.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f13ecff9700 1 -- 172.21.15.121:0/3517603272 <== mds.0 v2:172.21.15.121:6834/2239114529 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x22454e0 con 0x2432380 2023-11-15T03:00:24.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f13ecff9700 10 client.6303 handle_command_reply: tid=0 2023-11-15T03:00:24.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f141ea68740 1 client.6303 shutdown 2023-11-15T03:00:24.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f141ea68740 1 -- 172.21.15.121:0/3517603272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2410300 msgr2=0x2453120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:24.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f141ea68740 1 --2- 172.21.15.121:0/3517603272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2410300 0x2453120 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f13fc001ed0 tx=0x7f13fc01f000).stop 2023-11-15T03:00:24.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f141ea68740 1 -- 172.21.15.121:0/3517603272 shutdown_connections 2023-11-15T03:00:24.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f141ea68740 1 --2- 172.21.15.121:0/3517603272 >> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] conn(0x2432380 0x2432760 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f141ea68740 1 --2- 172.21.15.121:0/3517603272 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2455ab0 0x2453610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f13ecff9700 0 client.6303 ms_handle_reset on v2:172.21.15.121:6834/2239114529 2023-11-15T03:00:24.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f141ea68740 1 --2- 172.21.15.121:0/3517603272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2410300 0x2453120 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f141ea68740 1 --2- 172.21.15.121:0/3517603272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x240f510 0x244ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f141ea68740 1 -- 172.21.15.121:0/3517603272 >> 172.21.15.121:0/3517603272 conn(0x21ee550 msgr2=0x24364f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:24.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f141ea68740 1 -- 172.21.15.121:0/3517603272 shutdown_connections 2023-11-15T03:00:24.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f141ea68740 1 -- 172.21.15.121:0/3517603272 wait complete. 2023-11-15T03:00:24.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f141ea68740 20 client.6303 trim_cache size 0 max 16384 2023-11-15T03:00:24.395 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:00:24.396 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T03:00:24.396 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T03:00:24.396 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 5907, 2023-11-15T03:00:24.396 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T03:00:24.396 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T03:00:24.397 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T03:00:24.397 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 4.9326904599999999, 2023-11-15T03:00:24.397 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 59, 2023-11-15T03:00:24.397 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 63, 2023-11-15T03:00:24.397 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 63, 2023-11-15T03:00:24.398 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 55.108793001000002 2023-11-15T03:00:24.398 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:00:24.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f14146db700 1 -- 172.21.15.121:0/946897614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f13f005a510 msgr2=0x7f13f005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:24.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f14146db700 1 --2- 172.21.15.121:0/946897614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f13f005a510 0x7f13f005c9d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f140c0c6160 tx=0x7f1408017000).stop 2023-11-15T03:00:24.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f14146db700 1 -- 172.21.15.121:0/946897614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f140c151920 msgr2=0x7f140c0c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:00:24.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f14146db700 1 --2- 172.21.15.121:0/946897614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f140c151920 0x7f140c0c57d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f1404001460 tx=0x7f1404001650).stop 2023-11-15T03:00:24.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f14146db700 1 -- 172.21.15.121:0/946897614 shutdown_connections 2023-11-15T03:00:24.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f14146db700 1 --2- 172.21.15.121:0/946897614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f13f005a510 0x7f13f005c9d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f14146db700 1 --2- 172.21.15.121:0/946897614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f140c151920 0x7f140c0c57d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f14146db700 1 --2- 172.21.15.121:0/946897614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f140c0ac150 0x7f140c0c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f14146db700 1 --2- 172.21.15.121:0/946897614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f140c067150 0x7f140c144290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:00:24.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f14146db700 1 -- 172.21.15.121:0/946897614 >> 172.21.15.121:0/946897614 conn(0x7f140c067a60 msgr2=0x7f140c0a8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:00:24.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f14146db700 1 -- 172.21.15.121:0/946897614 shutdown_connections 2023-11-15T03:00:24.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:24.381+0000 7f14146db700 1 -- 172.21.15.121:0/946897614 wait complete. 2023-11-15T03:00:25.205 INFO:tasks.ceph.mds.c.smithi121.stderr:2023-11-15T03:00:25.201+0000 7f258eedc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:00:25.825 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-11-15T03:00:26.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:26.961+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=8 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:26.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:26.965+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 crc :-1 s=SESSION_RECONNECTING pgs=15 cs=8 l=0 rev1=1 rx=0 tx=0).handle_session_reset received session reset full=1 2023-11-15T03:00:26.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:26.965+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 crc :-1 s=SESSION_RECONNECTING pgs=15 cs=8 l=0 rev1=1 rx=0 tx=0).reset_session 2023-11-15T03:00:26.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:26.965+0000 7f71d4ff9700 0 client.5633 ms_handle_remote_reset on v2:172.21.15.121:6835/3429169582 2023-11-15T03:00:26.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:26.965+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2023-11-15T03:00:26.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:26.965+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T03:00:26.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:26.965+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T03:00:26.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:26.965+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 15.000000 2023-11-15T03:00:40.682 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:00:40.690 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:00:41.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:41.969+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:41.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:41.969+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2023-11-15T03:00:41.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:41.969+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T03:00:41.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:41.969+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T03:00:41.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:41.969+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 15.000000 2023-11-15T03:00:56.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:56.972+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:00:56.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:56.972+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2023-11-15T03:00:56.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:56.972+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T03:00:56.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:56.972+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T03:00:56.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:00:56.972+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 15.000000 2023-11-15T03:01:10.767 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:01:10.775 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:01:11.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:11.980+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:01:11.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:11.984+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2023-11-15T03:01:11.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:11.984+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T03:01:11.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:11.984+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T03:01:11.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:11.984+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 15.000000 2023-11-15T03:01:26.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:26.988+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:01:26.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:26.988+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2023-11-15T03:01:26.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:26.988+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T03:01:26.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:26.988+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T03:01:26.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:26.988+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 15.000000 2023-11-15T03:01:40.850 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:01:40.858 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:01:41.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:41.991+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:01:41.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:41.991+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2023-11-15T03:01:41.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:41.991+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T03:01:41.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:41.991+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T03:01:41.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:41.991+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 15.000000 2023-11-15T03:01:56.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:56.995+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:01:56.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:56.995+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2023-11-15T03:01:56.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:56.995+0000 7f71e659c700 1 -- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 msgr2=0x188a8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-15T03:01:57.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:56.995+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-15T03:01:57.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:01:56.995+0000 7f71e659c700 1 --2- 172.21.15.121:0/4034411113 >> [v2:172.21.15.121:6835/3429169582,v1:172.21.15.121:6836/3429169582] conn(0x188a500 0x188a8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 15.000000 2023-11-15T03:02:01.254 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-11-15T03:02:01.254 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T03:02:01.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.431+0000 7f23b51b9700 1 Processor -- start 2023-11-15T03:02:01.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.431+0000 7f23b51b9700 1 -- start start 2023-11-15T03:02:01.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.435+0000 7f23b51b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f23b00a9a30 0x7f23b00657a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:01.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.435+0000 7f23b51b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f23b0065c90 0x7f23b00bc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:01.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.435+0000 7f23b51b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b0064080 con 0x7f23b00a5d80 2023-11-15T03:02:01.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.435+0000 7f23b51b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b00c2f30 con 0x7f23b00a9a30 2023-11-15T03:02:01.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.435+0000 7f23b51b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b00c3210 con 0x7f23b0065c90 2023-11-15T03:02:01.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.435+0000 7f23af7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f23b00a9a30 0x7f23b00657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:01.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.435+0000 7f23af7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f23b00a9a30 0x7f23b00657a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55134/0 (socket says 172.21.15.121:55134) 2023-11-15T03:02:01.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.435+0000 7f23af7fe700 1 -- 172.21.15.121:0/2696070512 learned_addr learned my addr 172.21.15.121:0/2696070512 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:01.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.435+0000 7f23b49b8700 1 --2- 172.21.15.121:0/2696070512 >> v2:172.21.15.190:3300/0 conn(0x7f23b0065c90 0x7f23b00bc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:01.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.435+0000 7f23af7fe700 1 -- 172.21.15.121:0/2696070512 >> v2:172.21.15.190:3300/0 conn(0x7f23b0065c90 msgr2=0x7f23b00bc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:01.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.435+0000 7f23af7fe700 1 --2- 172.21.15.121:0/2696070512 >> v2:172.21.15.190:3300/0 conn(0x7f23b0065c90 0x7f23b00bc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:01.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.435+0000 7f23af7fe700 1 -- 172.21.15.121:0/2696070512 >> v1:172.21.15.121:6789/0 conn(0x7f23b00a5d80 legacy=0x7f23b00c1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:01.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.435+0000 7f23af7fe700 1 -- 172.21.15.121:0/2696070512 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b00bb9b0 con 0x7f23b00a9a30 2023-11-15T03:02:01.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.435+0000 7f23af7fe700 1 --2- 172.21.15.121:0/2696070512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23b00a9a30 0x7f23b00657a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7f23b00bd9d0 tx=0x7f23a40090f0).ready entity=mon.0 client_cookie=72310659986d5fbf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:01.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.435+0000 7f23aeffd700 1 -- 172.21.15.121:0/2696070512 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a400b600 con 0x7f23b00a9a30 2023-11-15T03:02:01.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.435+0000 7f23aeffd700 1 -- 172.21.15.121:0/2696070512 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a400b760 con 0x7f23b00a9a30 2023-11-15T03:02:01.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.435+0000 7f23aeffd700 1 -- 172.21.15.121:0/2696070512 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a400bae0 con 0x7f23b00a9a30 2023-11-15T03:02:01.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.435+0000 7f23b51b9700 1 -- 172.21.15.121:0/2696070512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23b00a9a30 msgr2=0x7f23b00657a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:01.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.435+0000 7f23b51b9700 1 --2- 172.21.15.121:0/2696070512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23b00a9a30 0x7f23b00657a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7f23b00bd9d0 tx=0x7f23a40090f0).stop 2023-11-15T03:02:01.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.439+0000 7f23b51b9700 1 -- 172.21.15.121:0/2696070512 shutdown_connections 2023-11-15T03:02:01.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.439+0000 7f23b51b9700 1 --2- 172.21.15.121:0/2696070512 >> v2:172.21.15.190:3300/0 conn(0x7f23b0065c90 0x7f23b00bc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:01.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.439+0000 7f23b51b9700 1 --2- 172.21.15.121:0/2696070512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23b00a9a30 0x7f23b00657a0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:01.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.439+0000 7f23b51b9700 1 -- 172.21.15.121:0/2696070512 >> 172.21.15.121:0/2696070512 conn(0x7f23b00677c0 msgr2=0x7f23b00c69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:01.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.439+0000 7f23b51b9700 1 -- 172.21.15.121:0/2696070512 shutdown_connections 2023-11-15T03:02:01.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.439+0000 7f23b51b9700 1 -- 172.21.15.121:0/2696070512 wait complete. 2023-11-15T03:02:01.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.439+0000 7f23b51b9700 1 Processor -- start 2023-11-15T03:02:01.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.439+0000 7f23b51b9700 1 -- start start 2023-11-15T03:02:01.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f23b51b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23b0065c90 0x7f23b0149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:01.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f23b51b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23b00a5d80 0x7f23b00bc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:01.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f23b51b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23b00a9a30 0x7f23b00bc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:01.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f23b51b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b00bcd70 con 0x7f23b00a9a30 2023-11-15T03:02:01.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f23b51b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b0153f40 con 0x7f23b0065c90 2023-11-15T03:02:01.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f23b51b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b01694b0 con 0x7f23b00a5d80 2023-11-15T03:02:01.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f23b49b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23b00a9a30 0x7f23b00bc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:01.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f23b49b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23b00a9a30 0x7f23b00bc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55136/0 (socket says 172.21.15.121:55136) 2023-11-15T03:02:01.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f23b49b8700 1 -- 172.21.15.121:0/3464905233 learned_addr learned my addr 172.21.15.121:0/3464905233 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:01.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f23af7fe700 1 --2- 172.21.15.121:0/3464905233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23b00a5d80 0x7f23b00bc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:01.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f23affff700 1 --2- 172.21.15.121:0/3464905233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23b0065c90 0x7f23b0149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:01.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f23b49b8700 1 -- 172.21.15.121:0/3464905233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23b00a5d80 msgr2=0x7f23b00bc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:01.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f23b49b8700 1 --2- 172.21.15.121:0/3464905233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23b00a5d80 0x7f23b00bc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:01.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f23b49b8700 1 -- 172.21.15.121:0/3464905233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23b0065c90 msgr2=0x7f23b0149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:01.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f23b49b8700 1 --2- 172.21.15.121:0/3464905233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23b0065c90 0x7f23b0149e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:01.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f23b49b8700 1 -- 172.21.15.121:0/3464905233 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a4009370 con 0x7f23b00a9a30 2023-11-15T03:02:01.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f23b49b8700 1 --2- 172.21.15.121:0/3464905233 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23b00a9a30 0x7f23b00bc880 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f23a8007f20 tx=0x7f23a8009860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:01.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f2397fff700 1 -- 172.21.15.121:0/3464905233 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a800eca0 con 0x7f23b00a9a30 2023-11-15T03:02:01.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f2397fff700 1 -- 172.21.15.121:0/3464905233 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a800ee00 con 0x7f23b00a9a30 2023-11-15T03:02:01.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f23b51b9700 1 -- 172.21.15.121:0/3464905233 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b0153610 con 0x7f23b00a9a30 2023-11-15T03:02:01.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f2397fff700 1 -- 172.21.15.121:0/3464905233 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8014610 con 0x7f23b00a9a30 2023-11-15T03:02:01.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.443+0000 7f23b51b9700 1 -- 172.21.15.121:0/3464905233 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b0153bd0 con 0x7f23b00a9a30 2023-11-15T03:02:01.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.447+0000 7f23b51b9700 1 -- 172.21.15.121:0/3464905233 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b01540b0 con 0x7f23b00a9a30 2023-11-15T03:02:01.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.447+0000 7f2397fff700 1 -- 172.21.15.121:0/3464905233 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f23a8014980 con 0x7f23b00a9a30 2023-11-15T03:02:01.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.451+0000 7f2397fff700 1 --2- 172.21.15.121:0/3464905233 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f239c05a510 0x7f239c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:01.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.451+0000 7f2397fff700 1 -- 172.21.15.121:0/3464905233 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6380+0+0 (secure 0 0 0) 0x7f23a8074e40 con 0x7f23b00a9a30 2023-11-15T03:02:01.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.451+0000 7f23affff700 1 --2- 172.21.15.121:0/3464905233 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f239c05a510 0x7f239c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:01.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.451+0000 7f23affff700 1 --2- 172.21.15.121:0/3464905233 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f239c05a510 0x7f239c05c9d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f23b01661b0 tx=0x7f23a0009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:01.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.455+0000 7f2397fff700 1 -- 172.21.15.121:0/3464905233 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f23a8048520 con 0x7f23b00a9a30 2023-11-15T03:02:01.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.651+0000 7f23b51b9700 1 -- 172.21.15.121:0/3464905233 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f23b0046800 con 0x7f23b00a9a30 2023-11-15T03:02:01.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.651+0000 7f2397fff700 1 -- 172.21.15.121:0/3464905233 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v59) v1 ==== 53+0+73 (secure 0 0 0) 0x7f23a804bb40 con 0x7f23b00a9a30 2023-11-15T03:02:01.658 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T03:02:01.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.655+0000 7f23b51b9700 1 -- 172.21.15.121:0/3464905233 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f239c05a510 msgr2=0x7f239c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:01.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.655+0000 7f23b51b9700 1 --2- 172.21.15.121:0/3464905233 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f239c05a510 0x7f239c05c9d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f23b01661b0 tx=0x7f23a0009040).stop 2023-11-15T03:02:01.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.655+0000 7f23b51b9700 1 -- 172.21.15.121:0/3464905233 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23b00a9a30 msgr2=0x7f23b00bc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:01.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.655+0000 7f23b51b9700 1 --2- 172.21.15.121:0/3464905233 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23b00a9a30 0x7f23b00bc880 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f23a8007f20 tx=0x7f23a8009860).stop 2023-11-15T03:02:01.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.655+0000 7f23b51b9700 1 -- 172.21.15.121:0/3464905233 shutdown_connections 2023-11-15T03:02:01.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.655+0000 7f23b51b9700 1 --2- 172.21.15.121:0/3464905233 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f239c05a510 0x7f239c05c9d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:01.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.655+0000 7f23b51b9700 1 --2- 172.21.15.121:0/3464905233 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23b00a9a30 0x7f23b00bc880 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:01.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.655+0000 7f23b51b9700 1 --2- 172.21.15.121:0/3464905233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23b00a5d80 0x7f23b00bc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:01.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.655+0000 7f23b51b9700 1 --2- 172.21.15.121:0/3464905233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23b0065c90 0x7f23b0149e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:01.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.655+0000 7f23b51b9700 1 -- 172.21.15.121:0/3464905233 >> 172.21.15.121:0/3464905233 conn(0x7f23b00677c0 msgr2=0x7f23b00c2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:01.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.655+0000 7f23b51b9700 1 -- 172.21.15.121:0/3464905233 shutdown_connections 2023-11-15T03:02:01.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:01.655+0000 7f23b51b9700 1 -- 172.21.15.121:0/3464905233 wait complete. 2023-11-15T03:02:01.681 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T03:02:01.681 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T03:02:01.682 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T03:02:01.682 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T03:02:01.683 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T03:02:01.683 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T03:02:01.683 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T03:02:01.684 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T03:02:01.684 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:02:01.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:02:01.685 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T03:02:01.707 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T03:02:01.708 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T03:02:01.708 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T03:02:01.708 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:02:01.708 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T03:02:01.708 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:02:01.709 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:02:01.709 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:02:01.709 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T03:02:01.710 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:02:01.710 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:02:01.710 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T03:02:01.710 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:02:01.710 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:02:01.711 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T03:02:01.712 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:02:01.712 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T03:02:01.712 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42118sec preferred_lft 42118sec 2023-11-15T03:02:01.712 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T03:02:01.713 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:02:01.713 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T03:02:01.716 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:02:01.716 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T03:02:01.717 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:02:01.717 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T03:02:01.717 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T03:02:01.717 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T03:02:01.717 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:02:01.718 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T03:02:01.718 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:02:01.718 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:02:01.718 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T03:02:01.718 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T03:02:01.719 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:02:01.719 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:02:01.719 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T03:02:01.719 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T03:02:01.719 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:02:01.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:02:01.720 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T03:02:01.789 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T03:02:01.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:02:01.794 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T03:02:01.862 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T03:02:01.863 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T03:02:01.863 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T03:02:01.863 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T03:02:01.864 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T03:02:01.864 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T03:02:01.864 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T03:02:01.864 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T03:02:01.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:02:01.865 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T03:02:01.866 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T03:02:01.866 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T03:02:01.867 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T03:02:01.867 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T03:02:01.868 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T03:02:02.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:02:02.001 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T03:02:02.020 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T03:02:02.020 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T03:02:02.020 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:02:02.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:02:02.021 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T03:02:02.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:02:02.100 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T03:02:02.163 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T03:02:02.164 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:02:02.283 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:02:02.284 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:02:02.307 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:02:02.308 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:02:02.313 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T03:02:02.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:02:02.314 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T03:02:02.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:02:02.347 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat a/ && find a/') 2023-11-15T03:02:02.417 INFO:teuthology.orchestra.run.smithi121.stdout: File: a/ 2023-11-15T03:02:02.417 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 2 Blocks: 0 IO Block: 65536 directory 2023-11-15T03:02:02.418 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 31h/49d Inode: 1099511627776 Links: 4 2023-11-15T03:02:02.418 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0775/drwxrwxr-x) Uid: ( 1000/ ubuntu) Gid: ( 1267/ ubuntu) 2023-11-15T03:02:02.418 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-11-15 02:59:46.641832696 +0000 2023-11-15T03:02:02.419 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-11-15 02:59:46.653832515 +0000 2023-11-15T03:02:02.419 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-11-15 02:59:46.653832515 +0000 2023-11-15T03:02:02.419 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-11-15T03:02:02.430 INFO:teuthology.orchestra.run.smithi121.stdout:a/ 2023-11-15T03:02:02.431 INFO:teuthology.orchestra.run.smithi121.stdout:a/c 2023-11-15T03:02:02.431 INFO:teuthology.orchestra.run.smithi121.stdout:a/c/d 2023-11-15T03:02:02.431 INFO:teuthology.orchestra.run.smithi121.stdout:a/b 2023-11-15T03:02:02.431 INFO:teuthology.orchestra.run.smithi121.stdout:a/b/d 2023-11-15T03:02:02.433 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2023-11-15T03:02:02.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.639+0000 7f52e748f700 1 Processor -- start 2023-11-15T03:02:02.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.639+0000 7f52e748f700 1 -- start start 2023-11-15T03:02:02.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.643+0000 7f52e748f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e00c5560 0x7f52e00c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.643+0000 7f52e748f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f52e0150ea0 0x7f52e00c7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.643+0000 7f52e748f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e00c5a90 con 0x7f52e00a6a00 2023-11-15T03:02:02.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.643+0000 7f52e748f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e00bdee0 con 0x7f52e0150ea0 2023-11-15T03:02:02.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.643+0000 7f52e748f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e00c5d70 con 0x7f52e00c5560 2023-11-15T03:02:02.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.643+0000 7f52e6c8e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f52e0150ea0 0x7f52e00c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.643+0000 7f52e6c8e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f52e0150ea0 0x7f52e00c7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55152/0 (socket says 172.21.15.121:55152) 2023-11-15T03:02:02.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.643+0000 7f52e6c8e700 1 -- 172.21.15.121:0/1697256376 learned_addr learned my addr 172.21.15.121:0/1697256376 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:02.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.643+0000 7f52e648d700 1 --1- 172.21.15.121:0/1697256376 >> v1:172.21.15.121:6789/0 conn(0x7f52e00a6a00 0x7f52e0152210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59656/0 (socket says 172.21.15.121:59656) 2023-11-15T03:02:02.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.643+0000 7f52e5c8c700 1 --2- 172.21.15.121:0/1697256376 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e00c5560 0x7f52e00c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.647+0000 7f52e6c8e700 1 -- 172.21.15.121:0/1697256376 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e00c5560 msgr2=0x7f52e00c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:02.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.647+0000 7f52e6c8e700 1 --2- 172.21.15.121:0/1697256376 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e00c5560 0x7f52e00c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.647+0000 7f52e6c8e700 1 -- 172.21.15.121:0/1697256376 >> v1:172.21.15.121:6789/0 conn(0x7f52e00a6a00 legacy=0x7f52e0152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:02.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.647+0000 7f52e6c8e700 1 -- 172.21.15.121:0/1697256376 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e00c6050 con 0x7f52e0150ea0 2023-11-15T03:02:02.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.647+0000 7f52e6c8e700 1 --2- 172.21.15.121:0/1697256376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e0150ea0 0x7f52e00c7020 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f52d8009f20 tx=0x7f52d8009840).ready entity=mon.0 client_cookie=53160211476ce962 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:02.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.647+0000 7f52e548b700 1 -- 172.21.15.121:0/1697256376 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d800ba80 con 0x7f52e0150ea0 2023-11-15T03:02:02.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.647+0000 7f52e548b700 1 -- 172.21.15.121:0/1697256376 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d800bbe0 con 0x7f52e0150ea0 2023-11-15T03:02:02.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.647+0000 7f52e548b700 1 -- 172.21.15.121:0/1697256376 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d8011480 con 0x7f52e0150ea0 2023-11-15T03:02:02.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.647+0000 7f52e748f700 1 -- 172.21.15.121:0/1697256376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e0150ea0 msgr2=0x7f52e00c7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:02.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.647+0000 7f52e748f700 1 --2- 172.21.15.121:0/1697256376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e0150ea0 0x7f52e00c7020 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f52d8009f20 tx=0x7f52d8009840).stop 2023-11-15T03:02:02.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.647+0000 7f52e748f700 1 -- 172.21.15.121:0/1697256376 shutdown_connections 2023-11-15T03:02:02.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.647+0000 7f52e748f700 1 --2- 172.21.15.121:0/1697256376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e0150ea0 0x7f52e00c7020 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.647+0000 7f52e748f700 1 --2- 172.21.15.121:0/1697256376 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e00c5560 0x7f52e00c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.647+0000 7f52e748f700 1 -- 172.21.15.121:0/1697256376 >> 172.21.15.121:0/1697256376 conn(0x7f52e0066e00 msgr2=0x7f52e0090dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:02.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.647+0000 7f52e748f700 1 -- 172.21.15.121:0/1697256376 shutdown_connections 2023-11-15T03:02:02.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.647+0000 7f52e748f700 1 -- 172.21.15.121:0/1697256376 wait complete. 2023-11-15T03:02:02.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.651+0000 7f52e748f700 1 Processor -- start 2023-11-15T03:02:02.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.651+0000 7f52e748f700 1 -- start start 2023-11-15T03:02:02.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.651+0000 7f52e748f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e00a6a00 0x7f52e00a7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.651+0000 7f52e748f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52e00c5560 0x7f52e0164ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.651+0000 7f52e748f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e0150ea0 0x7f52e0150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.651+0000 7f52e748f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e00a8ea0 con 0x7f52e0150ea0 2023-11-15T03:02:02.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.651+0000 7f52e748f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e0164fc0 con 0x7f52e00c5560 2023-11-15T03:02:02.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.651+0000 7f52e748f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e0158970 con 0x7f52e00a6a00 2023-11-15T03:02:02.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.651+0000 7f52e6c8e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e0150ea0 0x7f52e0150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.651+0000 7f52e6c8e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e0150ea0 0x7f52e0150240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55154/0 (socket says 172.21.15.121:55154) 2023-11-15T03:02:02.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.651+0000 7f52e6c8e700 1 -- 172.21.15.121:0/3215206611 learned_addr learned my addr 172.21.15.121:0/3215206611 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:02.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.651+0000 7f52e5c8c700 1 --2- 172.21.15.121:0/3215206611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52e00c5560 0x7f52e0164ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.651+0000 7f52e648d700 1 --2- 172.21.15.121:0/3215206611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e00a6a00 0x7f52e00a7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.651+0000 7f52e6c8e700 1 -- 172.21.15.121:0/3215206611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e00a6a00 msgr2=0x7f52e00a7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:02.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.651+0000 7f52e6c8e700 1 --2- 172.21.15.121:0/3215206611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e00a6a00 0x7f52e00a7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.651+0000 7f52e6c8e700 1 -- 172.21.15.121:0/3215206611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52e00c5560 msgr2=0x7f52e0164ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:02.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.651+0000 7f52e6c8e700 1 --2- 172.21.15.121:0/3215206611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52e00c5560 0x7f52e0164ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.651+0000 7f52e6c8e700 1 -- 172.21.15.121:0/3215206611 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d8009ae0 con 0x7f52e0150ea0 2023-11-15T03:02:02.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.651+0000 7f52e6c8e700 1 --2- 172.21.15.121:0/3215206611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e0150ea0 0x7f52e0150240 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f52d80070b0 tx=0x7f52d8009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:02.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.655+0000 7f52d67fc700 1 -- 172.21.15.121:0/3215206611 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d8007400 con 0x7f52e0150ea0 2023-11-15T03:02:02.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.655+0000 7f52d67fc700 1 -- 172.21.15.121:0/3215206611 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d8007560 con 0x7f52e0150ea0 2023-11-15T03:02:02.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.655+0000 7f52e748f700 1 -- 172.21.15.121:0/3215206611 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e00a8290 con 0x7f52e0150ea0 2023-11-15T03:02:02.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.655+0000 7f52d67fc700 1 -- 172.21.15.121:0/3215206611 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d8007920 con 0x7f52e0150ea0 2023-11-15T03:02:02.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.655+0000 7f52e748f700 1 -- 172.21.15.121:0/3215206611 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e00a8850 con 0x7f52e0150ea0 2023-11-15T03:02:02.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.655+0000 7f52d67fc700 1 -- 172.21.15.121:0/3215206611 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f52d8003040 con 0x7f52e0150ea0 2023-11-15T03:02:02.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.659+0000 7f52d67fc700 1 --2- 172.21.15.121:0/3215206611 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f52c405a540 0x7f52c405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.659+0000 7f52d67fc700 1 -- 172.21.15.121:0/3215206611 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6380+0+0 (secure 0 0 0) 0x7f52d80767d0 con 0x7f52e0150ea0 2023-11-15T03:02:02.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.659+0000 7f52e648d700 1 --2- 172.21.15.121:0/3215206611 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f52c405a540 0x7f52c405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.659+0000 7f52e648d700 1 --2- 172.21.15.121:0/3215206611 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f52c405a540 0x7f52c405ca00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f52e015a9b0 tx=0x7f52dc009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:02.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.659+0000 7f52f2aa3740 1 Processor -- start 2023-11-15T03:02:02.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.659+0000 7f52f2aa3740 1 -- start start 2023-11-15T03:02:02.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52f2aa3740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd2ba60 0xd2be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52f2aa3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd30e20 0xd365c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52f2aa3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd31f20 0xd36ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52f2aa3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0xc47540 con 0xd30e20 2023-11-15T03:02:02.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52f2aa3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0xc0ae90 con 0xd31f20 2023-11-15T03:02:02.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52f2aa3740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0xc10d90 con 0xd2ba60 2023-11-15T03:02:02.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52e5c8c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd30e20 0xd365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52e5c8c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd30e20 0xd365c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55164/0 (socket says 172.21.15.121:55164) 2023-11-15T03:02:02.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52e5c8c700 1 -- 172.21.15.121:0/2792389250 learned_addr learned my addr 172.21.15.121:0/2792389250 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:02.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52e648d700 1 --2- 172.21.15.121:0/2792389250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd2ba60 0xd2be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52e6c8e700 1 --2- 172.21.15.121:0/2792389250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd31f20 0xd36ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52e5c8c700 1 -- 172.21.15.121:0/2792389250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd2ba60 msgr2=0xd2be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:02.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52e5c8c700 1 --2- 172.21.15.121:0/2792389250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd2ba60 0xd2be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52e5c8c700 1 -- 172.21.15.121:0/2792389250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd31f20 msgr2=0xd36ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:02.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52e5c8c700 1 --2- 172.21.15.121:0/2792389250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd31f20 0xd36ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52e5c8c700 1 -- 172.21.15.121:0/2792389250 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xc754c0 con 0xd30e20 2023-11-15T03:02:02.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52e5c8c700 1 --2- 172.21.15.121:0/2792389250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd30e20 0xd365c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f52d0005dc0 tx=0x7f52d0008040).ready entity=mon.0 client_cookie=84f16df1790eb038 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:02.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52c37fe700 1 -- 172.21.15.121:0/2792389250 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d0005a10 con 0xd30e20 2023-11-15T03:02:02.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52c37fe700 1 -- 172.21.15.121:0/2792389250 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d0014430 con 0xd30e20 2023-11-15T03:02:02.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52c37fe700 1 -- 172.21.15.121:0/2792389250 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d00147b0 con 0xd30e20 2023-11-15T03:02:02.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52f2aa3740 1 -- 172.21.15.121:0/2792389250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd30e20 msgr2=0xd365c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:02.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/2792389250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd30e20 0xd365c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f52d0005dc0 tx=0x7f52d0008040).stop 2023-11-15T03:02:02.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52f2aa3740 1 -- 172.21.15.121:0/2792389250 shutdown_connections 2023-11-15T03:02:02.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/2792389250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd31f20 0xd36ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/2792389250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd30e20 0xd365c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/2792389250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd2ba60 0xd2be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52f2aa3740 1 -- 172.21.15.121:0/2792389250 >> 172.21.15.121:0/2792389250 conn(0xac9680 msgr2=0xd31210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:02.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52f2aa3740 1 -- 172.21.15.121:0/2792389250 shutdown_connections 2023-11-15T03:02:02.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.663+0000 7f52f2aa3740 1 -- 172.21.15.121:0/2792389250 wait complete. 2023-11-15T03:02:02.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52f2aa3740 1 -- start start 2023-11-15T03:02:02.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52f2aa3740 1 Processor -- start 2023-11-15T03:02:02.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52f2aa3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd2ba60 0xd0d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52f2aa3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd31f20 0xd296c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52f2aa3740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd29bb0 0xd2e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52f2aa3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0xd2a3a0 con 0xd31f20 2023-11-15T03:02:02.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52f2aa3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0xd2a680 con 0xd2ba60 2023-11-15T03:02:02.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52f2aa3740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0xd2a960 con 0xd29bb0 2023-11-15T03:02:02.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52f2aa3740 10 client.0 ll_register_callbacks cb 0xca15d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:02:02.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52e5c8c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd31f20 0xd296c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52e5c8c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd31f20 0xd296c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55172/0 (socket says 172.21.15.121:55172) 2023-11-15T03:02:02.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52e5c8c700 1 -- 172.21.15.121:0/3243624998 learned_addr learned my addr 172.21.15.121:0/3243624998 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:02.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52e6c8e700 1 --2- 172.21.15.121:0/3243624998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd29bb0 0xd2e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52e648d700 1 --2- 172.21.15.121:0/3243624998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd2ba60 0xd0d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52e5c8c700 1 -- 172.21.15.121:0/3243624998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd29bb0 msgr2=0xd2e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:02.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52e5c8c700 1 --2- 172.21.15.121:0/3243624998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd29bb0 0xd2e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52e5c8c700 1 -- 172.21.15.121:0/3243624998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd2ba60 msgr2=0xd0d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:02.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52e5c8c700 1 --2- 172.21.15.121:0/3243624998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd2ba60 0xd0d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52e5c8c700 1 -- 172.21.15.121:0/3243624998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d000e040 con 0xd31f20 2023-11-15T03:02:02.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52e5c8c700 1 --2- 172.21.15.121:0/3243624998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd31f20 0xd296c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7f52d0014000 tx=0x7f52d0008040).ready entity=mon.0 client_cookie=28d1ce45a933ddc2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:02.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:02:02.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52c0ff9700 1 -- 172.21.15.121:0/3243624998 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d00147f0 con 0xd31f20 2023-11-15T03:02:02.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.667+0000 7f52c0ff9700 1 -- 172.21.15.121:0/3243624998 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6380+0+0 (secure 0 0 0) 0x7f52d001f630 con 0xd31f20 2023-11-15T03:02:02.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52f2aa3740 1 -- 172.21.15.121:0/3243624998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xc61230 con 0xd31f20 2023-11-15T03:02:02.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52c0ff9700 1 -- 172.21.15.121:0/3243624998 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52d001fa00 con 0xd31f20 2023-11-15T03:02:02.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52f2aa3740 10 client.6327 fetch_fsmap learned FSMap version 59 2023-11-15T03:02:02.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52f2aa3740 1 -- 172.21.15.121:0/3243624998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0xc647c0 con 0xd31f20 2023-11-15T03:02:02.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52c0ff9700 1 -- 172.21.15.121:0/3243624998 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xc61230 con 0xd31f20 2023-11-15T03:02:02.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52c0ff9700 1 -- 172.21.15.121:0/3243624998 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 59) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f52d001b070 con 0xd31f20 2023-11-15T03:02:02.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52f2aa3740 10 client.6327 fetch_fsmap finished waiting for FSMap version 59 2023-11-15T03:02:02.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52f2aa3740 10 client.6327 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-15T03:02:02.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52f2aa3740 20 client.6327 populate_metadata read hostname 'smithi121' 2023-11-15T03:02:02.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/3243624998 >> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] conn(0xd3d800 0xd3dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52f2aa3740 4 client.6327 mds_command: new command op to 5907 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:02:02.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52f2aa3740 1 -- 172.21.15.121:0/3243624998 --> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xc57780 con 0xd3d800 2023-11-15T03:02:02.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52e648d700 1 --2- 172.21.15.121:0/3243624998 >> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] conn(0xd3d800 0xd3dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52e648d700 1 --2- 172.21.15.121:0/3243624998 >> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] conn(0xd3d800 0xd3dbe0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=427583cfd86c933f server_cookie=665f7cf874918b2 in_seq=0 out_seq=0 2023-11-15T03:02:02.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52c0ff9700 10 client.6327 ms_handle_connect on v2:172.21.15.121:6834/2239114529 2023-11-15T03:02:02.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52c0ff9700 1 -- 172.21.15.121:0/3243624998 <== mds.0 v2:172.21.15.121:6834/2239114529 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0xc57780 con 0xd3d800 2023-11-15T03:02:02.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52c0ff9700 10 client.6327 handle_command_reply: tid=0 2023-11-15T03:02:02.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52f2aa3740 1 client.6327 shutdown 2023-11-15T03:02:02.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52f2aa3740 1 -- 172.21.15.121:0/3243624998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd31f20 msgr2=0xd296c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:02.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/3243624998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd31f20 0xd296c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7f52d0014000 tx=0x7f52d0008040).stop 2023-11-15T03:02:02.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52f2aa3740 1 -- 172.21.15.121:0/3243624998 shutdown_connections 2023-11-15T03:02:02.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/3243624998 >> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] conn(0xd3d800 0xd3dbe0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/3243624998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd29bb0 0xd2e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/3243624998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd31f20 0xd296c0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/3243624998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd2ba60 0xd0d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.671+0000 7f52c0ff9700 0 client.6327 ms_handle_reset on v2:172.21.15.121:6834/2239114529 2023-11-15T03:02:02.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.675+0000 7f52f2aa3740 1 -- 172.21.15.121:0/3243624998 >> 172.21.15.121:0/3243624998 conn(0xac9680 msgr2=0xd0f050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:02.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.675+0000 7f52f2aa3740 1 -- 172.21.15.121:0/3243624998 shutdown_connections 2023-11-15T03:02:02.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.675+0000 7f52f2aa3740 1 -- 172.21.15.121:0/3243624998 wait complete. 2023-11-15T03:02:02.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.675+0000 7f52f2aa3740 20 client.6327 trim_cache size 0 max 16384 2023-11-15T03:02:02.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.691+0000 7f52f2aa3740 1 Processor -- start 2023-11-15T03:02:02.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.691+0000 7f52f2aa3740 1 -- start start 2023-11-15T03:02:02.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.691+0000 7f52f2aa3740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd29bb0 0xd45520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.691+0000 7f52f2aa3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd2ba60 0xd0cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.691+0000 7f52f2aa3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd31f20 0xd31500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.691+0000 7f52f2aa3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0xcb5df0 con 0xd2ba60 2023-11-15T03:02:02.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.691+0000 7f52f2aa3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0xcad040 con 0xd31f20 2023-11-15T03:02:02.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.691+0000 7f52f2aa3740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0xce4fc0 con 0xd29bb0 2023-11-15T03:02:02.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.691+0000 7f52e648d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd29bb0 0xd45520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.691+0000 7f52e5c8c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd2ba60 0xd0cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.691+0000 7f52e648d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd29bb0 0xd45520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:46376/0 (socket says 172.21.15.121:46376) 2023-11-15T03:02:02.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.691+0000 7f52e648d700 1 -- 172.21.15.121:0/3015962010 learned_addr learned my addr 172.21.15.121:0/3015962010 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:02.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.691+0000 7f52e6c8e700 1 --2- 172.21.15.121:0/3015962010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd31f20 0xd31500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.691+0000 7f52e648d700 1 -- 172.21.15.121:0/3015962010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd31f20 msgr2=0xd31500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:02.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.691+0000 7f52e648d700 1 --2- 172.21.15.121:0/3015962010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd31f20 0xd31500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.691+0000 7f52e648d700 1 -- 172.21.15.121:0/3015962010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd2ba60 msgr2=0xd0cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:02.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.691+0000 7f52e648d700 1 --2- 172.21.15.121:0/3015962010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd2ba60 0xd0cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.691+0000 7f52e648d700 1 -- 172.21.15.121:0/3015962010 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e0067dc0 con 0xd29bb0 2023-11-15T03:02:02.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.695+0000 7f52e648d700 1 --2- 172.21.15.121:0/3015962010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd29bb0 0xd45520 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f52dc001eb0 tx=0x7f52dc016040).ready entity=mon.2 client_cookie=d85a69b4ce40af9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:02.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.695+0000 7f52c3fff700 1 -- 172.21.15.121:0/3015962010 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc0250d0 con 0xd29bb0 2023-11-15T03:02:02.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.695+0000 7f52c3fff700 1 -- 172.21.15.121:0/3015962010 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc0256b0 con 0xd29bb0 2023-11-15T03:02:02.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.695+0000 7f52c3fff700 1 -- 172.21.15.121:0/3015962010 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc025a70 con 0xd29bb0 2023-11-15T03:02:02.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.695+0000 7f52f2aa3740 1 -- 172.21.15.121:0/3015962010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd29bb0 msgr2=0xd45520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:02.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.695+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/3015962010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd29bb0 0xd45520 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f52dc001eb0 tx=0x7f52dc016040).stop 2023-11-15T03:02:02.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.695+0000 7f52f2aa3740 1 -- 172.21.15.121:0/3015962010 shutdown_connections 2023-11-15T03:02:02.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.695+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/3015962010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd31f20 0xd31500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.695+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/3015962010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd2ba60 0xd0cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.695+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/3015962010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd29bb0 0xd45520 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.695+0000 7f52f2aa3740 1 -- 172.21.15.121:0/3015962010 >> 172.21.15.121:0/3015962010 conn(0xac9680 msgr2=0xd3c350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:02.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.695+0000 7f52f2aa3740 1 -- 172.21.15.121:0/3015962010 shutdown_connections 2023-11-15T03:02:02.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.695+0000 7f52f2aa3740 1 -- 172.21.15.121:0/3015962010 wait complete. 2023-11-15T03:02:02.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.695+0000 7f52f2aa3740 1 -- start start 2023-11-15T03:02:02.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.695+0000 7f52f2aa3740 1 Processor -- start 2023-11-15T03:02:02.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52f2aa3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd29bb0 0xcea200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52f2aa3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd2ba60 0xd91c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52f2aa3740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd31f20 0xd92150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52f2aa3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0xd9c070 con 0xd29bb0 2023-11-15T03:02:02.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52f2aa3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0xd9d800 con 0xd2ba60 2023-11-15T03:02:02.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52f2aa3740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0xd948e0 con 0xd31f20 2023-11-15T03:02:02.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52f2aa3740 10 client.0 ll_register_callbacks cb 0x7f52d0014f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:02:02.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52e648d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd29bb0 0xcea200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52e6c8e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd31f20 0xd92150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52e5c8c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd2ba60 0xd91c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52e648d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd29bb0 0xcea200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55186/0 (socket says 172.21.15.121:55186) 2023-11-15T03:02:02.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52e648d700 1 -- 172.21.15.121:0/2452895792 learned_addr learned my addr 172.21.15.121:0/2452895792 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:02.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52e648d700 1 -- 172.21.15.121:0/2452895792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd31f20 msgr2=0xd92150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:02.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52e648d700 1 --2- 172.21.15.121:0/2452895792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd31f20 0xd92150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52e648d700 1 -- 172.21.15.121:0/2452895792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd2ba60 msgr2=0xd91c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:02.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52e648d700 1 --2- 172.21.15.121:0/2452895792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd2ba60 0xd91c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52e648d700 1 -- 172.21.15.121:0/2452895792 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52dc01c040 con 0xd29bb0 2023-11-15T03:02:02.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52e6c8e700 1 --2- 172.21.15.121:0/2452895792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd31f20 0xd92150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:02:02.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52e648d700 1 --2- 172.21.15.121:0/2452895792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd29bb0 0xcea200 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7f52dc028040 tx=0x7f52dc01e000).ready entity=mon.0 client_cookie=1ae8f7b1fe180cb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:02.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:02:02.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52c0ff9700 1 -- 172.21.15.121:0/2452895792 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc023630 con 0xd29bb0 2023-11-15T03:02:02.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52c0ff9700 1 -- 172.21.15.121:0/2452895792 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6380+0+0 (secure 0 0 0) 0x7f52dc023790 con 0xd29bb0 2023-11-15T03:02:02.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52f2aa3740 1 -- 172.21.15.121:0/2452895792 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52d0014430 con 0xd29bb0 2023-11-15T03:02:02.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52c0ff9700 1 -- 172.21.15.121:0/2452895792 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52dc023b60 con 0xd29bb0 2023-11-15T03:02:02.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52f2aa3740 10 client.6333 fetch_fsmap learned FSMap version 59 2023-11-15T03:02:02.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52f2aa3740 1 -- 172.21.15.121:0/2452895792 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0xc5f9f0 con 0xd29bb0 2023-11-15T03:02:02.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52c0ff9700 1 -- 172.21.15.121:0/2452895792 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d0014430 con 0xd29bb0 2023-11-15T03:02:02.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52c0ff9700 1 -- 172.21.15.121:0/2452895792 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 59) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f52dc023cc0 con 0xd29bb0 2023-11-15T03:02:02.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52f2aa3740 10 client.6333 fetch_fsmap finished waiting for FSMap version 59 2023-11-15T03:02:02.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52f2aa3740 10 client.6333 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-15T03:02:02.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52f2aa3740 20 client.6333 populate_metadata read hostname 'smithi121' 2023-11-15T03:02:02.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.699+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/2452895792 >> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] conn(0xce9550 0xd0b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:02.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.703+0000 7f52f2aa3740 4 client.6333 mds_command: new command op to 5907 tid=0[{"prefix": "flush journal"}] 2023-11-15T03:02:02.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.703+0000 7f52f2aa3740 1 -- 172.21.15.121:0/2452895792 --> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0xb20610 con 0xce9550 2023-11-15T03:02:02.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.703+0000 7f52e5c8c700 1 --2- 172.21.15.121:0/2452895792 >> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] conn(0xce9550 0xd0b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:02.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.703+0000 7f52e5c8c700 1 --2- 172.21.15.121:0/2452895792 >> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] conn(0xce9550 0xd0b850 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=cb6df1e490933ab5 server_cookie=1974779774dd740e in_seq=0 out_seq=0 2023-11-15T03:02:02.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.703+0000 7f52c0ff9700 10 client.6333 ms_handle_connect on v2:172.21.15.121:6834/2239114529 2023-11-15T03:02:02.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52c0ff9700 1 -- 172.21.15.121:0/2452895792 <== mds.0 v2:172.21.15.121:6834/2239114529 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0xb20610 con 0xce9550 2023-11-15T03:02:02.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52c0ff9700 10 client.6333 handle_command_reply: tid=0 2023-11-15T03:02:02.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52f2aa3740 1 client.6333 shutdown 2023-11-15T03:02:02.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52f2aa3740 1 -- 172.21.15.121:0/2452895792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd29bb0 msgr2=0xcea200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:02.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/2452895792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd29bb0 0xcea200 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7f52dc028040 tx=0x7f52dc01e000).stop 2023-11-15T03:02:02.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52f2aa3740 1 -- 172.21.15.121:0/2452895792 shutdown_connections 2023-11-15T03:02:02.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/2452895792 >> [v2:172.21.15.121:6834/2239114529,v1:172.21.15.121:6837/2239114529] conn(0xce9550 0xd0b850 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/2452895792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd31f20 0xd92150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/2452895792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd2ba60 0xd91c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52f2aa3740 1 --2- 172.21.15.121:0/2452895792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd29bb0 0xcea200 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52c0ff9700 0 client.6333 ms_handle_reset on v2:172.21.15.121:6834/2239114529 2023-11-15T03:02:02.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52f2aa3740 1 -- 172.21.15.121:0/2452895792 >> 172.21.15.121:0/2452895792 conn(0xac9680 msgr2=0xd0f200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:02.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52f2aa3740 1 -- 172.21.15.121:0/2452895792 shutdown_connections 2023-11-15T03:02:02.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52f2aa3740 1 -- 172.21.15.121:0/2452895792 wait complete. 2023-11-15T03:02:02.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52f2aa3740 20 client.6333 trim_cache size 0 max 16384 2023-11-15T03:02:02.736 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:02:02.736 INFO:teuthology.orchestra.run.smithi121.stdout: "message": "", 2023-11-15T03:02:02.737 INFO:teuthology.orchestra.run.smithi121.stdout: "return_code": 0 2023-11-15T03:02:02.737 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:02:02.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52e748f700 1 -- 172.21.15.121:0/3215206611 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f52c405a540 msgr2=0x7f52c405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:02.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52e748f700 1 --2- 172.21.15.121:0/3215206611 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f52c405a540 0x7f52c405ca00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f52e015a9b0 tx=0x7f52dc009040).stop 2023-11-15T03:02:02.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52e748f700 1 -- 172.21.15.121:0/3215206611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e0150ea0 msgr2=0x7f52e0150240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:02.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52e748f700 1 --2- 172.21.15.121:0/3215206611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e0150ea0 0x7f52e0150240 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f52d80070b0 tx=0x7f52d8009840).stop 2023-11-15T03:02:02.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52e748f700 1 -- 172.21.15.121:0/3215206611 shutdown_connections 2023-11-15T03:02:02.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52e748f700 1 --2- 172.21.15.121:0/3215206611 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f52c405a540 0x7f52c405ca00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52e748f700 1 --2- 172.21.15.121:0/3215206611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52e0150ea0 0x7f52e0150240 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52e748f700 1 --2- 172.21.15.121:0/3215206611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52e00c5560 0x7f52e0164ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52e748f700 1 --2- 172.21.15.121:0/3215206611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e00a6a00 0x7f52e00a7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:02.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52e748f700 1 -- 172.21.15.121:0/3215206611 >> 172.21.15.121:0/3215206611 conn(0x7f52e0066e00 msgr2=0x7f52e0062fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:02.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52e748f700 1 -- 172.21.15.121:0/3215206611 shutdown_connections 2023-11-15T03:02:02.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:02.723+0000 7f52e748f700 1 -- 172.21.15.121:0/3215206611 wait complete. 2023-11-15T03:02:02.754 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-15T03:02:02.755 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T03:02:02.798 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T03:02:02.845 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-15T03:02:02.846 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T03:02:02.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:02:02.847 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-15T03:02:02.895 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:02:02.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:02:02.896 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:02:02.961 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:02:03.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.159+0000 7fa4fdcf1700 1 Processor -- start 2023-11-15T03:02:03.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.159+0000 7fa4fdcf1700 1 -- start start 2023-11-15T03:02:03.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.163+0000 7fa4fdcf1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f8151830 0x7fa4f80a87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:03.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.163+0000 7fa4fdcf1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa4f80ab260 0x7fa4f8155640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:03.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.163+0000 7fa4fdcf1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f8065d00 con 0x7fa4f8151c10 2023-11-15T03:02:03.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.163+0000 7fa4fdcf1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f80c6b90 con 0x7fa4f80ab260 2023-11-15T03:02:03.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.163+0000 7fa4fdcf1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f80c6e30 con 0x7fa4f8151830 2023-11-15T03:02:03.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.163+0000 7fa4f6ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa4f80ab260 0x7fa4f8155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:03.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.163+0000 7fa4f77fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f8151830 0x7fa4f80a87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:03.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.163+0000 7fa4f7fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fa4f8151c10 0x7fa4f80c7910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:58030/0 (socket says 172.21.15.121:58030) 2023-11-15T03:02:03.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.163+0000 7fa4f7fff700 1 -- 172.21.15.121:0/4022458671 learned_addr learned my addr 172.21.15.121:0/4022458671 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:03.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.163+0000 7fa4f6ffd700 1 -- 172.21.15.121:0/4022458671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f8151830 msgr2=0x7fa4f80a87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:03.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.163+0000 7fa4f6ffd700 1 --2- 172.21.15.121:0/4022458671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f8151830 0x7fa4f80a87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.163+0000 7fa4f67fc700 1 -- 172.21.15.121:0/4022458671 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 859497931 0 0) 0x7fa4f8065d00 con 0x7fa4f8151c10 2023-11-15T03:02:03.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.163+0000 7fa4f6ffd700 1 -- 172.21.15.121:0/4022458671 >> v1:172.21.15.190:6789/0 conn(0x7fa4f8151c10 legacy=0x7fa4f80c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:03.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.163+0000 7fa4f6ffd700 1 -- 172.21.15.121:0/4022458671 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f80c7110 con 0x7fa4f80ab260 2023-11-15T03:02:03.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.167+0000 7fa4f6ffd700 1 --2- 172.21.15.121:0/4022458671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f80ab260 0x7fa4f8155640 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7fa4ec009760 tx=0x7fa4ec009060).ready entity=mon.0 client_cookie=237aed3228ec5eb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:03.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.167+0000 7fa4f67fc700 1 -- 172.21.15.121:0/4022458671 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec00f510 con 0x7fa4f80ab260 2023-11-15T03:02:03.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.167+0000 7fa4f67fc700 1 -- 172.21.15.121:0/4022458671 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ec00f670 con 0x7fa4f80ab260 2023-11-15T03:02:03.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.167+0000 7fa4f67fc700 1 -- 172.21.15.121:0/4022458671 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec00f9f0 con 0x7fa4f80ab260 2023-11-15T03:02:03.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.167+0000 7fa4fdcf1700 1 -- 172.21.15.121:0/4022458671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f80ab260 msgr2=0x7fa4f8155640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:03.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.167+0000 7fa4fdcf1700 1 --2- 172.21.15.121:0/4022458671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f80ab260 0x7fa4f8155640 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7fa4ec009760 tx=0x7fa4ec009060).stop 2023-11-15T03:02:03.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.167+0000 7fa4fdcf1700 1 -- 172.21.15.121:0/4022458671 shutdown_connections 2023-11-15T03:02:03.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.167+0000 7fa4fdcf1700 1 --2- 172.21.15.121:0/4022458671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f80ab260 0x7fa4f8155640 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.167+0000 7fa4fdcf1700 1 --2- 172.21.15.121:0/4022458671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f8151830 0x7fa4f80a87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.167+0000 7fa4fdcf1700 1 -- 172.21.15.121:0/4022458671 >> 172.21.15.121:0/4022458671 conn(0x7fa4f80c8960 msgr2=0x7fa4f80c8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:03.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.167+0000 7fa4fdcf1700 1 -- 172.21.15.121:0/4022458671 shutdown_connections 2023-11-15T03:02:03.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.167+0000 7fa4fdcf1700 1 -- 172.21.15.121:0/4022458671 wait complete. 2023-11-15T03:02:03.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.171+0000 7fa4fdcf1700 1 Processor -- start 2023-11-15T03:02:03.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.171+0000 7fa4fdcf1700 1 -- start start 2023-11-15T03:02:03.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.171+0000 7fa4fdcf1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4f80ab260 0x7fa4f80c1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:03.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.171+0000 7fa4fdcf1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f8151830 0x7fa4f8063b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:03.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.171+0000 7fa4fdcf1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8151c10 0x7fa4f80c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:03.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.171+0000 7fa4fdcf1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f814a950 con 0x7fa4f8151c10 2023-11-15T03:02:03.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.171+0000 7fa4fdcf1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f80bfd50 con 0x7fa4f80ab260 2023-11-15T03:02:03.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.171+0000 7fa4fdcf1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f816cfa0 con 0x7fa4f8151830 2023-11-15T03:02:03.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.171+0000 7fa4f7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8151c10 0x7fa4f80c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:03.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.171+0000 7fa4f7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8151c10 0x7fa4f80c2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55200/0 (socket says 172.21.15.121:55200) 2023-11-15T03:02:03.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.171+0000 7fa4f7fff700 1 -- 172.21.15.121:0/2302784479 learned_addr learned my addr 172.21.15.121:0/2302784479 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:03.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.171+0000 7fa4f6ffd700 1 --2- 172.21.15.121:0/2302784479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f8151830 0x7fa4f8063b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:03.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.171+0000 7fa4f77fe700 1 --2- 172.21.15.121:0/2302784479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4f80ab260 0x7fa4f80c1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:03.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.175+0000 7fa4f7fff700 1 -- 172.21.15.121:0/2302784479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f8151830 msgr2=0x7fa4f8063b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:03.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.175+0000 7fa4f7fff700 1 --2- 172.21.15.121:0/2302784479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f8151830 0x7fa4f8063b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.175+0000 7fa4f7fff700 1 -- 172.21.15.121:0/2302784479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4f80ab260 msgr2=0x7fa4f80c1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:03.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.175+0000 7fa4f7fff700 1 --2- 172.21.15.121:0/2302784479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4f80ab260 0x7fa4f80c1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.175+0000 7fa4f7fff700 1 -- 172.21.15.121:0/2302784479 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ec0092a0 con 0x7fa4f8151c10 2023-11-15T03:02:03.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.175+0000 7fa4f6ffd700 1 --2- 172.21.15.121:0/2302784479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f8151830 0x7fa4f8063b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:02:03.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.175+0000 7fa4f7fff700 1 --2- 172.21.15.121:0/2302784479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8151c10 0x7fa4f80c2180 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7fa4f000c730 tx=0x7fa4f0001520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:03.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.175+0000 7fa4df7fe700 1 -- 172.21.15.121:0/2302784479 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0005d30 con 0x7fa4f8151c10 2023-11-15T03:02:03.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.175+0000 7fa4df7fe700 1 -- 172.21.15.121:0/2302784479 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f0005e90 con 0x7fa4f8151c10 2023-11-15T03:02:03.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.175+0000 7fa4fdcf1700 1 -- 172.21.15.121:0/2302784479 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f816c390 con 0x7fa4f8151c10 2023-11-15T03:02:03.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.175+0000 7fa4df7fe700 1 -- 172.21.15.121:0/2302784479 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0007300 con 0x7fa4f8151c10 2023-11-15T03:02:03.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.175+0000 7fa4fdcf1700 1 -- 172.21.15.121:0/2302784479 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f816c950 con 0x7fa4f8151c10 2023-11-15T03:02:03.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.175+0000 7fa4fdcf1700 1 -- 172.21.15.121:0/2302784479 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f8168aa0 con 0x7fa4f8151c10 2023-11-15T03:02:03.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.179+0000 7fa4df7fe700 1 -- 172.21.15.121:0/2302784479 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa4f00076b0 con 0x7fa4f8151c10 2023-11-15T03:02:03.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.179+0000 7fa4df7fe700 1 --2- 172.21.15.121:0/2302784479 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa4e405a4e0 0x7fa4e405c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:03.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.179+0000 7fa4df7fe700 1 -- 172.21.15.121:0/2302784479 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6380+0+0 (secure 0 0 0) 0x7fa4f0074300 con 0x7fa4f8151c10 2023-11-15T03:02:03.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.179+0000 7fa4f77fe700 1 --2- 172.21.15.121:0/2302784479 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa4e405a4e0 0x7fa4e405c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:03.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.179+0000 7fa4f77fe700 1 --2- 172.21.15.121:0/2302784479 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa4e405a4e0 0x7fa4e405c9a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7fa4e8009c30 tx=0x7fa4e8009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:03.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.183+0000 7fa4df7fe700 1 -- 172.21.15.121:0/2302784479 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa4f00479e0 con 0x7fa4f8151c10 2023-11-15T03:02:03.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.387+0000 7fa4fdcf1700 1 -- 172.21.15.121:0/2302784479 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4f806c150 con 0x7fa4f8151c10 2023-11-15T03:02:03.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.387+0000 7fa4df7fe700 1 -- 172.21.15.121:0/2302784479 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4504 (secure 0 0 0) 0x7fa4f0018070 con 0x7fa4f8151c10 2023-11-15T03:02:03.393 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:03.394 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3462858472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3462858472},{"type":"v1","addr":"172.21.15.190:6833","nonce":3462858472}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4042390087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4042390087},{"type":"v1","addr":"172.21.15.190:6835","nonce":4042390087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3908033272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3908033272},{"type":"v1","addr":"172.21.15.121:6836","nonce":3908033272}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:59:34.667864+0000","modified":"2023-11-15T03:00:22.468286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5907},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5907":{"gid":5907,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":16,"addr":"172.21.15.121:6837/2239114529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2239114529},{"type":"v1","addr":"172.21.15.121:6837","nonce":2239114529}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-15T03:02:03.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.391+0000 7fa4fdcf1700 1 -- 172.21.15.121:0/2302784479 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa4e405a4e0 msgr2=0x7fa4e405c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:03.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.391+0000 7fa4fdcf1700 1 --2- 172.21.15.121:0/2302784479 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa4e405a4e0 0x7fa4e405c9a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7fa4e8009c30 tx=0x7fa4e8009060).stop 2023-11-15T03:02:03.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.391+0000 7fa4fdcf1700 1 -- 172.21.15.121:0/2302784479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8151c10 msgr2=0x7fa4f80c2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:03.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.391+0000 7fa4fdcf1700 1 --2- 172.21.15.121:0/2302784479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8151c10 0x7fa4f80c2180 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7fa4f000c730 tx=0x7fa4f0001520).stop 2023-11-15T03:02:03.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.391+0000 7fa4fdcf1700 1 -- 172.21.15.121:0/2302784479 shutdown_connections 2023-11-15T03:02:03.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.391+0000 7fa4fdcf1700 1 --2- 172.21.15.121:0/2302784479 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa4e405a4e0 0x7fa4e405c9a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.391+0000 7fa4fdcf1700 1 --2- 172.21.15.121:0/2302784479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8151c10 0x7fa4f80c2180 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.391+0000 7fa4fdcf1700 1 --2- 172.21.15.121:0/2302784479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f8151830 0x7fa4f8063b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.391+0000 7fa4fdcf1700 1 --2- 172.21.15.121:0/2302784479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4f80ab260 0x7fa4f80c1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.391+0000 7fa4fdcf1700 1 -- 172.21.15.121:0/2302784479 >> 172.21.15.121:0/2302784479 conn(0x7fa4f80c8960 msgr2=0x7fa4f814b120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:03.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.391+0000 7fa4fdcf1700 1 -- 172.21.15.121:0/2302784479 shutdown_connections 2023-11-15T03:02:03.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.391+0000 7fa4fdcf1700 1 -- 172.21.15.121:0/2302784479 wait complete. 2023-11-15T03:02:03.399 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 59 2023-11-15T03:02:03.414 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:02:03.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.563+0000 7ff9265b9700 1 Processor -- start 2023-11-15T03:02:03.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.563+0000 7ff9265b9700 1 -- start start 2023-11-15T03:02:03.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.563+0000 7ff9265b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff9200c3280 0x7ff9200c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:03.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.563+0000 7ff9265b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff920152140 0x7ff9201525c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:03.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.563+0000 7ff9265b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9200c1d50 con 0x7ff92015be30 2023-11-15T03:02:03.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.563+0000 7ff9265b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9200c1860 con 0x7ff9200c3280 2023-11-15T03:02:03.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.563+0000 7ff9265b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92015c990 con 0x7ff920152140 2023-11-15T03:02:03.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.567+0000 7ff9255b7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff9200c3280 0x7ff9200c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:03.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.567+0000 7ff924db6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff920152140 0x7ff9201525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:03.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.567+0000 7ff9255b7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff9200c3280 0x7ff9200c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55208/0 (socket says 172.21.15.121:55208) 2023-11-15T03:02:03.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.567+0000 7ff9255b7700 1 -- 172.21.15.121:0/4163344254 learned_addr learned my addr 172.21.15.121:0/4163344254 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:03.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.567+0000 7ff9255b7700 1 -- 172.21.15.121:0/4163344254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff920152140 msgr2=0x7ff9201525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:03.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.567+0000 7ff9255b7700 1 --2- 172.21.15.121:0/4163344254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff920152140 0x7ff9201525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.567+0000 7ff9255b7700 1 -- 172.21.15.121:0/4163344254 >> v1:172.21.15.190:6789/0 conn(0x7ff92015be30 legacy=0x7ff92015c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:03.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.567+0000 7ff9255b7700 1 -- 172.21.15.121:0/4163344254 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92015cb80 con 0x7ff9200c3280 2023-11-15T03:02:03.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.567+0000 7ff924db6700 1 --2- 172.21.15.121:0/4163344254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff920152140 0x7ff9201525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:02:03.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.567+0000 7ff9255b7700 1 --2- 172.21.15.121:0/4163344254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9200c3280 0x7ff9200c3660 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7ff91c009f20 tx=0x7ff91c009840).ready entity=mon.0 client_cookie=b469f30bde022bb8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:03.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.567+0000 7ff917fff700 1 -- 172.21.15.121:0/4163344254 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c00e070 con 0x7ff9200c3280 2023-11-15T03:02:03.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.567+0000 7ff917fff700 1 -- 172.21.15.121:0/4163344254 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91c00b9d0 con 0x7ff9200c3280 2023-11-15T03:02:03.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.567+0000 7ff917fff700 1 -- 172.21.15.121:0/4163344254 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c00bd50 con 0x7ff9200c3280 2023-11-15T03:02:03.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.567+0000 7ff9265b9700 1 -- 172.21.15.121:0/4163344254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9200c3280 msgr2=0x7ff9200c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:03.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.567+0000 7ff9265b9700 1 --2- 172.21.15.121:0/4163344254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9200c3280 0x7ff9200c3660 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7ff91c009f20 tx=0x7ff91c009840).stop 2023-11-15T03:02:03.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.567+0000 7ff9265b9700 1 -- 172.21.15.121:0/4163344254 shutdown_connections 2023-11-15T03:02:03.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.567+0000 7ff9265b9700 1 --2- 172.21.15.121:0/4163344254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff920152140 0x7ff9201525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.567+0000 7ff9265b9700 1 --2- 172.21.15.121:0/4163344254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9200c3280 0x7ff9200c3660 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.567+0000 7ff9265b9700 1 -- 172.21.15.121:0/4163344254 >> 172.21.15.121:0/4163344254 conn(0x7ff920062190 msgr2=0x7ff920062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:03.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.571+0000 7ff9265b9700 1 -- 172.21.15.121:0/4163344254 shutdown_connections 2023-11-15T03:02:03.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.571+0000 7ff9265b9700 1 -- 172.21.15.121:0/4163344254 wait complete. 2023-11-15T03:02:03.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.571+0000 7ff9265b9700 1 Processor -- start 2023-11-15T03:02:03.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.571+0000 7ff9265b9700 1 -- start start 2023-11-15T03:02:03.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9265b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9200c3280 0x7ff92014d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:03.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9265b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff920152140 0x7ff920156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:03.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9265b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff92015be30 0x7ff9200c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:03.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9265b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9200c44e0 con 0x7ff9200c3280 2023-11-15T03:02:03.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9265b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9200c47c0 con 0x7ff920152140 2023-11-15T03:02:03.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9265b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9200c4aa0 con 0x7ff92015be30 2023-11-15T03:02:03.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9255b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9200c3280 0x7ff92014d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:03.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9255b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9200c3280 0x7ff92014d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55218/0 (socket says 172.21.15.121:55218) 2023-11-15T03:02:03.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9255b7700 1 -- 172.21.15.121:0/2498109756 learned_addr learned my addr 172.21.15.121:0/2498109756 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:03.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff925db8700 1 --2- 172.21.15.121:0/2498109756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff92015be30 0x7ff9200c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:03.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff924db6700 1 --2- 172.21.15.121:0/2498109756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff920152140 0x7ff920156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:03.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9255b7700 1 -- 172.21.15.121:0/2498109756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff92015be30 msgr2=0x7ff9200c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:03.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9255b7700 1 --2- 172.21.15.121:0/2498109756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff92015be30 0x7ff9200c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9255b7700 1 -- 172.21.15.121:0/2498109756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff920152140 msgr2=0x7ff920156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:03.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9255b7700 1 --2- 172.21.15.121:0/2498109756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff920152140 0x7ff920156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9255b7700 1 -- 172.21.15.121:0/2498109756 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9100092a0 con 0x7ff9200c3280 2023-11-15T03:02:03.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff924db6700 1 --2- 172.21.15.121:0/2498109756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff920152140 0x7ff920156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:03.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff925db8700 1 --2- 172.21.15.121:0/2498109756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff92015be30 0x7ff9200c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:03.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9255b7700 1 --2- 172.21.15.121:0/2498109756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9200c3280 0x7ff92014d830 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7ff91c00b610 tx=0x7ff91c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:03.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9157fa700 1 -- 172.21.15.121:0/2498109756 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c00bb40 con 0x7ff9200c3280 2023-11-15T03:02:03.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9157fa700 1 -- 172.21.15.121:0/2498109756 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91c00bca0 con 0x7ff9200c3280 2023-11-15T03:02:03.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9265b9700 1 -- 172.21.15.121:0/2498109756 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91c009ae0 con 0x7ff9200c3280 2023-11-15T03:02:03.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9157fa700 1 -- 172.21.15.121:0/2498109756 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c01d670 con 0x7ff9200c3280 2023-11-15T03:02:03.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.575+0000 7ff9265b9700 1 -- 172.21.15.121:0/2498109756 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9200c5060 con 0x7ff9200c3280 2023-11-15T03:02:03.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.579+0000 7ff9265b9700 1 -- 172.21.15.121:0/2498109756 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92006bcf0 con 0x7ff9200c3280 2023-11-15T03:02:03.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.579+0000 7ff9157fa700 1 -- 172.21.15.121:0/2498109756 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff91c003040 con 0x7ff9200c3280 2023-11-15T03:02:03.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.583+0000 7ff9157fa700 1 --2- 172.21.15.121:0/2498109756 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff90405a510 0x7ff90405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:03.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.583+0000 7ff9157fa700 1 -- 172.21.15.121:0/2498109756 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6380+0+0 (secure 0 0 0) 0x7ff91c075b30 con 0x7ff9200c3280 2023-11-15T03:02:03.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.583+0000 7ff924db6700 1 --2- 172.21.15.121:0/2498109756 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff90405a510 0x7ff90405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:03.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.583+0000 7ff924db6700 1 --2- 172.21.15.121:0/2498109756 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff90405a510 0x7ff90405c9d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7ff910009270 tx=0x7ff910008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:03.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.587+0000 7ff9157fa700 1 -- 172.21.15.121:0/2498109756 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff91c049200 con 0x7ff9200c3280 2023-11-15T03:02:03.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.755+0000 7ff9265b9700 1 -- 172.21.15.121:0/2498109756 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff920068a70 con 0x7ff9200c3280 2023-11-15T03:02:03.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.755+0000 7ff9157fa700 1 -- 172.21.15.121:0/2498109756 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+17387 (secure 0 0 0) 0x7ff91c04ce60 con 0x7ff9200c3280 2023-11-15T03:02:03.764 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:03.764 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":65,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:00:39.942021+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:59:32.734725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-15T02:59:33.665552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.1d","mappings":[{"from":4,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]},{"pgid":"10.3c","mappings":[{"from":4,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6836/3429169582":"2023-11-16T03:00:19.439898+0000","172.21.15.121:6835/3429169582":"2023-11-16T03:00:19.439898+0000","172.21.15.121:6836/2394175138":"2023-11-16T02:59:48.732462+0000","172.21.15.121:6835/2394175138":"2023-11-16T02:59:48.732462+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:02:03.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.759+0000 7ff9265b9700 1 -- 172.21.15.121:0/2498109756 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff90405a510 msgr2=0x7ff90405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:03.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.759+0000 7ff9265b9700 1 --2- 172.21.15.121:0/2498109756 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff90405a510 0x7ff90405c9d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7ff910009270 tx=0x7ff910008040).stop 2023-11-15T03:02:03.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.759+0000 7ff9265b9700 1 -- 172.21.15.121:0/2498109756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9200c3280 msgr2=0x7ff92014d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:03.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.759+0000 7ff9265b9700 1 --2- 172.21.15.121:0/2498109756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9200c3280 0x7ff92014d830 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7ff91c00b610 tx=0x7ff91c009840).stop 2023-11-15T03:02:03.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.759+0000 7ff9265b9700 1 -- 172.21.15.121:0/2498109756 shutdown_connections 2023-11-15T03:02:03.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.759+0000 7ff9265b9700 1 --2- 172.21.15.121:0/2498109756 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff90405a510 0x7ff90405c9d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.759+0000 7ff9265b9700 1 --2- 172.21.15.121:0/2498109756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff92015be30 0x7ff9200c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.759+0000 7ff9265b9700 1 --2- 172.21.15.121:0/2498109756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff920152140 0x7ff920156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.759+0000 7ff9265b9700 1 --2- 172.21.15.121:0/2498109756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9200c3280 0x7ff92014d830 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.759+0000 7ff9265b9700 1 -- 172.21.15.121:0/2498109756 >> 172.21.15.121:0/2498109756 conn(0x7ff920062190 msgr2=0x7ff92015a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:03.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.759+0000 7ff9265b9700 1 -- 172.21.15.121:0/2498109756 shutdown_connections 2023-11-15T03:02:03.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.759+0000 7ff9265b9700 1 -- 172.21.15.121:0/2498109756 wait complete. 2023-11-15T03:02:03.790 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-15T02:59:32.734725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-15T02:59:33.665552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-15T03:02:03.791 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T03:02:03.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.951+0000 7fc0c7107700 1 Processor -- start 2023-11-15T03:02:03.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.951+0000 7fc0c7107700 1 -- start start 2023-11-15T03:02:03.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.951+0000 7fc0c7107700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc0c00a8960 0x7fc0c00c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:03.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.951+0000 7fc0c7107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c00a8d40 0x7fc0c015c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:03.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.951+0000 7fc0c7107700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c00657f0 con 0x7fc0c0066520 2023-11-15T03:02:03.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.951+0000 7fc0c7107700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c01583d0 con 0x7fc0c00a8960 2023-11-15T03:02:03.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.951+0000 7fc0c7107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c00c1b60 con 0x7fc0c00a8d40 2023-11-15T03:02:03.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.955+0000 7fc0c5904700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c00a8d40 0x7fc0c015c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:03.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.955+0000 7fc0c6105700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc0c00a8960 0x7fc0c00c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:03.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.955+0000 7fc0c5904700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c00a8d40 0x7fc0c015c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:46436/0 (socket says 172.21.15.121:46436) 2023-11-15T03:02:03.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.955+0000 7fc0c6105700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc0c00a8960 0x7fc0c00c15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:43790/0 (socket says 172.21.15.121:43790) 2023-11-15T03:02:03.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.955+0000 7fc0c5904700 1 -- 172.21.15.121:0/2606615802 learned_addr learned my addr 172.21.15.121:0/2606615802 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:03.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.955+0000 7fc0c5904700 1 -- 172.21.15.121:0/2606615802 >> v2:172.21.15.190:3300/0 conn(0x7fc0c00a8960 msgr2=0x7fc0c00c15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:03.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.955+0000 7fc0c5904700 1 --2- 172.21.15.121:0/2606615802 >> v2:172.21.15.190:3300/0 conn(0x7fc0c00a8960 0x7fc0c00c15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.955+0000 7fc0c5904700 1 -- 172.21.15.121:0/2606615802 >> v1:172.21.15.190:6789/0 conn(0x7fc0c0066520 legacy=0x7fc0c00c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:03.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.955+0000 7fc0c5904700 1 -- 172.21.15.121:0/2606615802 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c0065a40 con 0x7fc0c00a8d40 2023-11-15T03:02:03.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.955+0000 7fc0c5904700 1 --2- 172.21.15.121:0/2606615802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c00a8d40 0x7fc0c015c4c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fc0b0006fd0 tx=0x7fc0b00090f0).ready entity=mon.2 client_cookie=119c1c5679a0c838 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:03.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.955+0000 7fc0c5103700 1 -- 172.21.15.121:0/2606615802 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b000b580 con 0x7fc0c00a8d40 2023-11-15T03:02:03.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.955+0000 7fc0c5103700 1 -- 172.21.15.121:0/2606615802 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b000b6e0 con 0x7fc0c00a8d40 2023-11-15T03:02:03.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.955+0000 7fc0c5103700 1 -- 172.21.15.121:0/2606615802 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b000ba60 con 0x7fc0c00a8d40 2023-11-15T03:02:03.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.955+0000 7fc0c7107700 1 -- 172.21.15.121:0/2606615802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c00a8d40 msgr2=0x7fc0c015c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:03.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.955+0000 7fc0c7107700 1 --2- 172.21.15.121:0/2606615802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c00a8d40 0x7fc0c015c4c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fc0b0006fd0 tx=0x7fc0b00090f0).stop 2023-11-15T03:02:03.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.959+0000 7fc0c7107700 1 -- 172.21.15.121:0/2606615802 shutdown_connections 2023-11-15T03:02:03.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.959+0000 7fc0c7107700 1 --2- 172.21.15.121:0/2606615802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c00a8d40 0x7fc0c015c4c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.959+0000 7fc0c7107700 1 --2- 172.21.15.121:0/2606615802 >> v2:172.21.15.190:3300/0 conn(0x7fc0c00a8960 0x7fc0c00c15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.959+0000 7fc0c7107700 1 -- 172.21.15.121:0/2606615802 >> 172.21.15.121:0/2606615802 conn(0x7fc0c0067a60 msgr2=0x7fc0c0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:03.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.959+0000 7fc0c7107700 1 -- 172.21.15.121:0/2606615802 shutdown_connections 2023-11-15T03:02:03.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.959+0000 7fc0c7107700 1 -- 172.21.15.121:0/2606615802 wait complete. 2023-11-15T03:02:03.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.959+0000 7fc0c7107700 1 Processor -- start 2023-11-15T03:02:03.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.959+0000 7fc0c7107700 1 -- start start 2023-11-15T03:02:03.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0c7107700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c0066520 0x7fc0c00a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:03.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0c7107700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c00a8960 0x7fc0c015cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:03.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0c7107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c00a8d40 0x7fc0c00c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:03.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0c7107700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c01579f0 con 0x7fc0c0066520 2023-11-15T03:02:03.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0c7107700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c00bcd00 con 0x7fc0c00a8960 2023-11-15T03:02:03.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0c7107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c00bc0f0 con 0x7fc0c00a8d40 2023-11-15T03:02:03.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0c6105700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c0066520 0x7fc0c00a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:03.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0c6105700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c0066520 0x7fc0c00a5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55228/0 (socket says 172.21.15.121:55228) 2023-11-15T03:02:03.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0c6105700 1 -- 172.21.15.121:0/3429996096 learned_addr learned my addr 172.21.15.121:0/3429996096 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:03.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0c5904700 1 --2- 172.21.15.121:0/3429996096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c00a8960 0x7fc0c015cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:03.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0c6906700 1 --2- 172.21.15.121:0/3429996096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c00a8d40 0x7fc0c00c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:03.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0c6105700 1 -- 172.21.15.121:0/3429996096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c00a8d40 msgr2=0x7fc0c00c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:03.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0c6105700 1 --2- 172.21.15.121:0/3429996096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c00a8d40 0x7fc0c00c6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0c6105700 1 -- 172.21.15.121:0/3429996096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c00a8960 msgr2=0x7fc0c015cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:03.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0c6105700 1 --2- 172.21.15.121:0/3429996096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c00a8960 0x7fc0c015cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:03.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0c6105700 1 -- 172.21.15.121:0/3429996096 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b0009370 con 0x7fc0c0066520 2023-11-15T03:02:03.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0c6105700 1 --2- 172.21.15.121:0/3429996096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c0066520 0x7fc0c00a5c70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7fc0bc009ef0 tx=0x7fc0bc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:03.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0b67fc700 1 -- 172.21.15.121:0/3429996096 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc00cc50 con 0x7fc0c0066520 2023-11-15T03:02:03.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0b67fc700 1 -- 172.21.15.121:0/3429996096 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc00f040 con 0x7fc0c0066520 2023-11-15T03:02:03.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0b67fc700 1 -- 172.21.15.121:0/3429996096 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc013400 con 0x7fc0c0066520 2023-11-15T03:02:03.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0c7107700 1 -- 172.21.15.121:0/3429996096 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c00bc3d0 con 0x7fc0c0066520 2023-11-15T03:02:03.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.963+0000 7fc0c7107700 1 -- 172.21.15.121:0/3429996096 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c00bc990 con 0x7fc0c0066520 2023-11-15T03:02:03.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.967+0000 7fc0c7107700 1 -- 172.21.15.121:0/3429996096 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c00be330 con 0x7fc0c0066520 2023-11-15T03:02:03.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.967+0000 7fc0b67fc700 1 -- 172.21.15.121:0/3429996096 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc0bc013770 con 0x7fc0c0066520 2023-11-15T03:02:03.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.971+0000 7fc0b67fc700 1 --2- 172.21.15.121:0/3429996096 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc0a405a510 0x7fc0a405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:03.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.971+0000 7fc0b67fc700 1 -- 172.21.15.121:0/3429996096 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6380+0+0 (secure 0 0 0) 0x7fc0bc0752c0 con 0x7fc0c0066520 2023-11-15T03:02:03.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.971+0000 7fc0c5904700 1 --2- 172.21.15.121:0/3429996096 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc0a405a510 0x7fc0a405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:03.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.971+0000 7fc0c5904700 1 --2- 172.21.15.121:0/3429996096 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc0a405a510 0x7fc0a405c9d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fc0c00a9160 tx=0x7fc0b0017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:03.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:03.975+0000 7fc0b67fc700 1 -- 172.21.15.121:0/3429996096 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc0bc0488e0 con 0x7fc0c0066520 2023-11-15T03:02:04.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.151+0000 7fc0c7107700 1 -- 172.21.15.121:0/3429996096 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc0c006bcf0 con 0x7fc0c0066520 2023-11-15T03:02:04.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.151+0000 7fc0b67fc700 1 -- 172.21.15.121:0/3429996096 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v65) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc0bc04bf00 con 0x7fc0c0066520 2023-11-15T03:02:04.157 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T03:02:04.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.151+0000 7fc0c7107700 1 -- 172.21.15.121:0/3429996096 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc0a405a510 msgr2=0x7fc0a405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:04.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.151+0000 7fc0c7107700 1 --2- 172.21.15.121:0/3429996096 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc0a405a510 0x7fc0a405c9d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fc0c00a9160 tx=0x7fc0b0017000).stop 2023-11-15T03:02:04.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.151+0000 7fc0c7107700 1 -- 172.21.15.121:0/3429996096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c0066520 msgr2=0x7fc0c00a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:04.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.151+0000 7fc0c7107700 1 --2- 172.21.15.121:0/3429996096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c0066520 0x7fc0c00a5c70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7fc0bc009ef0 tx=0x7fc0bc00b040).stop 2023-11-15T03:02:04.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.151+0000 7fc0c7107700 1 -- 172.21.15.121:0/3429996096 shutdown_connections 2023-11-15T03:02:04.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.151+0000 7fc0c7107700 1 --2- 172.21.15.121:0/3429996096 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc0a405a510 0x7fc0a405c9d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.151+0000 7fc0c7107700 1 --2- 172.21.15.121:0/3429996096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c00a8d40 0x7fc0c00c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.151+0000 7fc0c7107700 1 --2- 172.21.15.121:0/3429996096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c00a8960 0x7fc0c015cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.151+0000 7fc0c7107700 1 --2- 172.21.15.121:0/3429996096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c0066520 0x7fc0c00a5c70 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.151+0000 7fc0c7107700 1 -- 172.21.15.121:0/3429996096 >> 172.21.15.121:0/3429996096 conn(0x7fc0c0067a60 msgr2=0x7fc0c015a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:04.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.151+0000 7fc0c7107700 1 -- 172.21.15.121:0/3429996096 shutdown_connections 2023-11-15T03:02:04.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.155+0000 7fc0c7107700 1 -- 172.21.15.121:0/3429996096 wait complete. 2023-11-15T03:02:04.179 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T03:02:04.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.343+0000 7f5aa34db700 1 Processor -- start 2023-11-15T03:02:04.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.347+0000 7f5aa34db700 1 -- start start 2023-11-15T03:02:04.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.347+0000 7f5aa34db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5a9c0a8b80 0x7f5a9c0a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:04.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.347+0000 7f5aa34db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5a9c065850 0x7f5a9c067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:04.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.347+0000 7f5aa34db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a9c067f50 0x7f5a9c0bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:04.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.347+0000 7f5aa34db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a9c068400 con 0x7f5a9c0a8b80 2023-11-15T03:02:04.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.347+0000 7f5aa34db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a9c152150 con 0x7f5a9c065850 2023-11-15T03:02:04.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.347+0000 7f5aa34db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c148f70 con 0x7f5a9c067f50 2023-11-15T03:02:04.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.347+0000 7f5aa1277700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5a9c0a8b80 0x7f5a9c0a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:04.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.347+0000 7f5aa1277700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5a9c0a8b80 0x7f5a9c0a8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55242/0 (socket says 172.21.15.121:55242) 2023-11-15T03:02:04.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.347+0000 7f5aa1277700 1 -- 172.21.15.121:0/2148107954 learned_addr learned my addr 172.21.15.121:0/2148107954 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:04.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.347+0000 7f5aa1a78700 1 --2- 172.21.15.121:0/2148107954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a9c067f50 0x7f5a9c0bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:04.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.347+0000 7f5aa0a76700 1 --2- 172.21.15.121:0/2148107954 >> v2:172.21.15.190:3300/0 conn(0x7f5a9c065850 0x7f5a9c067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:04.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.351+0000 7f5aa1277700 1 -- 172.21.15.121:0/2148107954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a9c067f50 msgr2=0x7f5a9c0bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:04.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.351+0000 7f5aa1277700 1 --2- 172.21.15.121:0/2148107954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a9c067f50 0x7f5a9c0bbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.351+0000 7f5aa1277700 1 -- 172.21.15.121:0/2148107954 >> v2:172.21.15.190:3300/0 conn(0x7f5a9c065850 msgr2=0x7f5a9c067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:04.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.351+0000 7f5aa1277700 1 --2- 172.21.15.121:0/2148107954 >> v2:172.21.15.190:3300/0 conn(0x7f5a9c065850 0x7f5a9c067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.351+0000 7f5aa1277700 1 -- 172.21.15.121:0/2148107954 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c1523a0 con 0x7f5a9c0a8b80 2023-11-15T03:02:04.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.351+0000 7f5aa1277700 1 --2- 172.21.15.121:0/2148107954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9c0a8b80 0x7f5a9c0a8f60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f5a9c0676c0 tx=0x7f5a980090f0).ready entity=mon.0 client_cookie=80c079ccb5f92a39 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:04.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.351+0000 7f5a93fff700 1 -- 172.21.15.121:0/2148107954 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9800f510 con 0x7f5a9c0a8b80 2023-11-15T03:02:04.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.351+0000 7f5a93fff700 1 -- 172.21.15.121:0/2148107954 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9800f670 con 0x7f5a9c0a8b80 2023-11-15T03:02:04.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.351+0000 7f5a93fff700 1 -- 172.21.15.121:0/2148107954 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9800fa10 con 0x7f5a9c0a8b80 2023-11-15T03:02:04.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.351+0000 7f5aa34db700 1 -- 172.21.15.121:0/2148107954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9c0a8b80 msgr2=0x7f5a9c0a8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:04.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.351+0000 7f5aa34db700 1 --2- 172.21.15.121:0/2148107954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9c0a8b80 0x7f5a9c0a8f60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f5a9c0676c0 tx=0x7f5a980090f0).stop 2023-11-15T03:02:04.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.351+0000 7f5aa34db700 1 -- 172.21.15.121:0/2148107954 shutdown_connections 2023-11-15T03:02:04.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.351+0000 7f5aa34db700 1 --2- 172.21.15.121:0/2148107954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a9c067f50 0x7f5a9c0bbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.351+0000 7f5aa34db700 1 --2- 172.21.15.121:0/2148107954 >> v2:172.21.15.190:3300/0 conn(0x7f5a9c065850 0x7f5a9c067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.351+0000 7f5aa34db700 1 --2- 172.21.15.121:0/2148107954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9c0a8b80 0x7f5a9c0a8f60 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.351+0000 7f5aa34db700 1 -- 172.21.15.121:0/2148107954 >> 172.21.15.121:0/2148107954 conn(0x7f5a9c0591c0 msgr2=0x7f5a9c0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:04.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.351+0000 7f5aa34db700 1 -- 172.21.15.121:0/2148107954 shutdown_connections 2023-11-15T03:02:04.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.351+0000 7f5aa34db700 1 -- 172.21.15.121:0/2148107954 wait complete. 2023-11-15T03:02:04.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.355+0000 7f5aa34db700 1 Processor -- start 2023-11-15T03:02:04.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.355+0000 7f5aa34db700 1 -- start start 2023-11-15T03:02:04.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.355+0000 7f5aa34db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9c065850 0x7f5a9c063160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:04.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.355+0000 7f5aa34db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a9c067f50 0x7f5a9c145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:04.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.355+0000 7f5aa34db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a9c0a8b80 0x7f5a9c14b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:04.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.355+0000 7f5aa34db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c0c0640 con 0x7f5a9c065850 2023-11-15T03:02:04.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.355+0000 7f5aa34db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c155ee0 con 0x7f5a9c067f50 2023-11-15T03:02:04.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.355+0000 7f5aa34db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c145a20 con 0x7f5a9c0a8b80 2023-11-15T03:02:04.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.355+0000 7f5aa1277700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9c065850 0x7f5a9c063160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:04.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.355+0000 7f5aa1277700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9c065850 0x7f5a9c063160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55246/0 (socket says 172.21.15.121:55246) 2023-11-15T03:02:04.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.355+0000 7f5aa1277700 1 -- 172.21.15.121:0/339842907 learned_addr learned my addr 172.21.15.121:0/339842907 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:04.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.355+0000 7f5aa1a78700 1 --2- 172.21.15.121:0/339842907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a9c0a8b80 0x7f5a9c14b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:04.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.355+0000 7f5aa0a76700 1 --2- 172.21.15.121:0/339842907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a9c067f50 0x7f5a9c145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:04.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.359+0000 7f5aa1277700 1 -- 172.21.15.121:0/339842907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a9c0a8b80 msgr2=0x7f5a9c14b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:04.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.359+0000 7f5aa1277700 1 --2- 172.21.15.121:0/339842907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a9c0a8b80 0x7f5a9c14b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.359+0000 7f5aa1277700 1 -- 172.21.15.121:0/339842907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a9c067f50 msgr2=0x7f5a9c145530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:04.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.359+0000 7f5aa1277700 1 --2- 172.21.15.121:0/339842907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a9c067f50 0x7f5a9c145530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.359+0000 7f5aa1277700 1 -- 172.21.15.121:0/339842907 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a98009370 con 0x7f5a9c065850 2023-11-15T03:02:04.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.359+0000 7f5aa0a76700 1 --2- 172.21.15.121:0/339842907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a9c067f50 0x7f5a9c145530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:04.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.359+0000 7f5aa1277700 1 --2- 172.21.15.121:0/339842907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9c065850 0x7f5a9c063160 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f5a9c059730 tx=0x7f5a98017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:04.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.359+0000 7f5a917fa700 1 -- 172.21.15.121:0/339842907 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9800fe50 con 0x7f5a9c065850 2023-11-15T03:02:04.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.359+0000 7f5a917fa700 1 -- 172.21.15.121:0/339842907 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a980119a0 con 0x7f5a9c065850 2023-11-15T03:02:04.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.359+0000 7f5a917fa700 1 -- 172.21.15.121:0/339842907 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98011d40 con 0x7f5a9c065850 2023-11-15T03:02:04.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.359+0000 7f5aa34db700 1 -- 172.21.15.121:0/339842907 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9c0bd960 con 0x7f5a9c065850 2023-11-15T03:02:04.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.359+0000 7f5aa34db700 1 -- 172.21.15.121:0/339842907 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9c155b70 con 0x7f5a9c065850 2023-11-15T03:02:04.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.359+0000 7f5aa34db700 1 -- 172.21.15.121:0/339842907 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a9c047060 con 0x7f5a9c065850 2023-11-15T03:02:04.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.359+0000 7f5a917fa700 1 -- 172.21.15.121:0/339842907 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5a98021070 con 0x7f5a9c065850 2023-11-15T03:02:04.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.363+0000 7f5a917fa700 1 --2- 172.21.15.121:0/339842907 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5a8005a540 0x7f5a8005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:04.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.363+0000 7f5a917fa700 1 -- 172.21.15.121:0/339842907 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6380+0+0 (secure 0 0 0) 0x7f5a98076250 con 0x7f5a9c065850 2023-11-15T03:02:04.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.363+0000 7f5aa0a76700 1 --2- 172.21.15.121:0/339842907 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5a8005a540 0x7f5a8005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:04.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.363+0000 7f5aa0a76700 1 --2- 172.21.15.121:0/339842907 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5a8005a540 0x7f5a8005ca00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f5a9c063650 tx=0x7f5a8c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:04.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.367+0000 7f5a917fa700 1 -- 172.21.15.121:0/339842907 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5a98049870 con 0x7f5a9c065850 2023-11-15T03:02:04.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.531+0000 7f5aa34db700 1 -- 172.21.15.121:0/339842907 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5a9c0ab7d0 con 0x7f5a9c065850 2023-11-15T03:02:04.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.531+0000 7f5a917fa700 1 -- 172.21.15.121:0/339842907 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5a98029020 con 0x7f5a9c065850 2023-11-15T03:02:04.539 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T03:02:04.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.535+0000 7f5aa34db700 1 -- 172.21.15.121:0/339842907 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5a8005a540 msgr2=0x7f5a8005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:04.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.535+0000 7f5aa34db700 1 --2- 172.21.15.121:0/339842907 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5a8005a540 0x7f5a8005ca00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f5a9c063650 tx=0x7f5a8c008040).stop 2023-11-15T03:02:04.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.535+0000 7f5aa34db700 1 -- 172.21.15.121:0/339842907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9c065850 msgr2=0x7f5a9c063160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:04.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.535+0000 7f5aa34db700 1 --2- 172.21.15.121:0/339842907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9c065850 0x7f5a9c063160 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f5a9c059730 tx=0x7f5a98017000).stop 2023-11-15T03:02:04.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.535+0000 7f5aa34db700 1 -- 172.21.15.121:0/339842907 shutdown_connections 2023-11-15T03:02:04.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.535+0000 7f5aa34db700 1 --2- 172.21.15.121:0/339842907 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5a8005a540 0x7f5a8005ca00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.535+0000 7f5aa34db700 1 --2- 172.21.15.121:0/339842907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a9c0a8b80 0x7f5a9c14b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.535+0000 7f5aa34db700 1 --2- 172.21.15.121:0/339842907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a9c067f50 0x7f5a9c145530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.535+0000 7f5aa34db700 1 --2- 172.21.15.121:0/339842907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9c065850 0x7f5a9c063160 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.535+0000 7f5aa34db700 1 -- 172.21.15.121:0/339842907 >> 172.21.15.121:0/339842907 conn(0x7f5a9c0591c0 msgr2=0x7f5a9c0c4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:04.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.535+0000 7f5aa34db700 1 -- 172.21.15.121:0/339842907 shutdown_connections 2023-11-15T03:02:04.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.535+0000 7f5aa34db700 1 -- 172.21.15.121:0/339842907 wait complete. 2023-11-15T03:02:04.560 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-15T03:02:04.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.719+0000 7f4dc73b9700 1 Processor -- start 2023-11-15T03:02:04.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.719+0000 7f4dc73b9700 1 -- start start 2023-11-15T03:02:04.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.723+0000 7f4dc73b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4dc00acb60 0x7f4dc00acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:04.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.723+0000 7f4dc73b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc00c7e90 con 0x7f4dc0065680 2023-11-15T03:02:04.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.723+0000 7f4dc73b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc00ad4d0 con 0x7f4dc014e380 2023-11-15T03:02:04.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.723+0000 7f4dc73b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc00c8320 con 0x7f4dc00acb60 2023-11-15T03:02:04.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.723+0000 7f4dc6bb8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4dc0065680 0x7f4dc0065af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34364/0 (socket says 172.21.15.121:34364) 2023-11-15T03:02:04.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.723+0000 7f4dc6bb8700 1 -- 172.21.15.121:0/3112704286 learned_addr learned my addr 172.21.15.121:0/3112704286 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:04.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.723+0000 7f4dc5bb6700 1 --2- 172.21.15.121:0/3112704286 >> v2:172.21.15.190:3300/0 conn(0x7f4dc00acb60 0x7f4dc00acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:04.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.723+0000 7f4dc53b5700 1 -- 172.21.15.121:0/3112704286 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3151941891 0 0) 0x7f4dc00c7e90 con 0x7f4dc0065680 2023-11-15T03:02:04.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.723+0000 7f4dc53b5700 1 -- 172.21.15.121:0/3112704286 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dac0033f0 con 0x7f4dc0065680 2023-11-15T03:02:04.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.723+0000 7f4dc53b5700 1 -- 172.21.15.121:0/3112704286 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2813976826 0 0) 0x7f4dac0033f0 con 0x7f4dc0065680 2023-11-15T03:02:04.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.723+0000 7f4dc5bb6700 1 -- 172.21.15.121:0/3112704286 >> v1:172.21.15.190:6789/0 conn(0x7f4dc014e380 legacy=0x7f4dc00c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:04.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.723+0000 7f4dc5bb6700 1 -- 172.21.15.121:0/3112704286 >> v1:172.21.15.121:6789/0 conn(0x7f4dc0065680 legacy=0x7f4dc0065af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:04.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.723+0000 7f4dc5bb6700 1 -- 172.21.15.121:0/3112704286 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc014e760 con 0x7f4dc00acb60 2023-11-15T03:02:04.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.723+0000 7f4dc5bb6700 1 --2- 172.21.15.121:0/3112704286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dc00acb60 0x7f4dc00acfe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f4db0009760 tx=0x7f4db0009060).ready entity=mon.1 client_cookie=89668600ae375620 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:04.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.723+0000 7f4dc53b5700 1 -- 172.21.15.121:0/3112704286 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db80017f0 con 0x7f4dc00acb60 2023-11-15T03:02:04.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.723+0000 7f4dc53b5700 1 -- 172.21.15.121:0/3112704286 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db0011400 con 0x7f4dc00acb60 2023-11-15T03:02:04.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.723+0000 7f4dc53b5700 1 -- 172.21.15.121:0/3112704286 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db0011780 con 0x7f4dc00acb60 2023-11-15T03:02:04.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.727+0000 7f4dc73b9700 1 -- 172.21.15.121:0/3112704286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dc00acb60 msgr2=0x7f4dc00acfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:04.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.727+0000 7f4dc73b9700 1 --2- 172.21.15.121:0/3112704286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dc00acb60 0x7f4dc00acfe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f4db0009760 tx=0x7f4db0009060).stop 2023-11-15T03:02:04.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.727+0000 7f4dc73b9700 1 -- 172.21.15.121:0/3112704286 shutdown_connections 2023-11-15T03:02:04.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.727+0000 7f4dc73b9700 1 --2- 172.21.15.121:0/3112704286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dc00acb60 0x7f4dc00acfe0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.727+0000 7f4dc73b9700 1 -- 172.21.15.121:0/3112704286 >> 172.21.15.121:0/3112704286 conn(0x7f4dc0067a60 msgr2=0x7f4dc0067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:04.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.727+0000 7f4dc73b9700 1 -- 172.21.15.121:0/3112704286 shutdown_connections 2023-11-15T03:02:04.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.727+0000 7f4dc73b9700 1 -- 172.21.15.121:0/3112704286 wait complete. 2023-11-15T03:02:04.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.727+0000 7f4dc73b9700 1 Processor -- start 2023-11-15T03:02:04.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.727+0000 7f4dc73b9700 1 -- start start 2023-11-15T03:02:04.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc73b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4dc0065680 0x7f4dc013e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:04.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc73b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dc00acb60 0x7f4dc014b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:04.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc73b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dc014e380 0x7f4dc014bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:04.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc73b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc014c2e0 con 0x7f4dc014e380 2023-11-15T03:02:04.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc73b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc0168610 con 0x7f4dc00acb60 2023-11-15T03:02:04.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc73b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc0142260 con 0x7f4dc0065680 2023-11-15T03:02:04.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc6bb8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dc014e380 0x7f4dc014bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:04.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc6bb8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dc014e380 0x7f4dc014bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53006/0 (socket says 172.21.15.121:53006) 2023-11-15T03:02:04.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc6bb8700 1 -- 172.21.15.121:0/1578265129 learned_addr learned my addr 172.21.15.121:0/1578265129 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:04.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc5bb6700 1 --2- 172.21.15.121:0/1578265129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dc00acb60 0x7f4dc014b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:04.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc63b7700 1 --2- 172.21.15.121:0/1578265129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4dc0065680 0x7f4dc013e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:04.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc6bb8700 1 -- 172.21.15.121:0/1578265129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4dc0065680 msgr2=0x7f4dc013e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:04.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc6bb8700 1 --2- 172.21.15.121:0/1578265129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4dc0065680 0x7f4dc013e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc6bb8700 1 -- 172.21.15.121:0/1578265129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dc00acb60 msgr2=0x7f4dc014b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:04.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc6bb8700 1 --2- 172.21.15.121:0/1578265129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dc00acb60 0x7f4dc014b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc6bb8700 1 -- 172.21.15.121:0/1578265129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db00092a0 con 0x7f4dc014e380 2023-11-15T03:02:04.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc5bb6700 1 --2- 172.21.15.121:0/1578265129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dc00acb60 0x7f4dc014b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:02:04.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc63b7700 1 --2- 172.21.15.121:0/1578265129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4dc0065680 0x7f4dc013e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:02:04.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc6bb8700 1 --2- 172.21.15.121:0/1578265129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dc014e380 0x7f4dc014bdf0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f4db800cbc0 tx=0x7f4db8001d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:04.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4db67fc700 1 -- 172.21.15.121:0/1578265129 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8004c40 con 0x7f4dc014e380 2023-11-15T03:02:04.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4db67fc700 1 -- 172.21.15.121:0/1578265129 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db8004da0 con 0x7f4dc014e380 2023-11-15T03:02:04.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc73b9700 1 -- 172.21.15.121:0/1578265129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc0167a00 con 0x7f4dc014e380 2023-11-15T03:02:04.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4db67fc700 1 -- 172.21.15.121:0/1578265129 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8007380 con 0x7f4dc014e380 2023-11-15T03:02:04.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc73b9700 1 -- 172.21.15.121:0/1578265129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc0167fc0 con 0x7f4dc014e380 2023-11-15T03:02:04.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.731+0000 7f4dc73b9700 1 -- 172.21.15.121:0/1578265129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc01687a0 con 0x7f4dc014e380 2023-11-15T03:02:04.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.735+0000 7f4db67fc700 1 -- 172.21.15.121:0/1578265129 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4db8007770 con 0x7f4dc014e380 2023-11-15T03:02:04.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.735+0000 7f4db67fc700 1 --2- 172.21.15.121:0/1578265129 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4da405a510 0x7f4da405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:04.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.735+0000 7f4db67fc700 1 -- 172.21.15.121:0/1578265129 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6380+0+0 (secure 0 0 0) 0x7f4db80748c0 con 0x7f4dc014e380 2023-11-15T03:02:04.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.735+0000 7f4dc63b7700 1 --2- 172.21.15.121:0/1578265129 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4da405a510 0x7f4da405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:04.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.735+0000 7f4dc63b7700 1 --2- 172.21.15.121:0/1578265129 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4da405a510 0x7f4da405c9d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f4dbc001710 tx=0x7f4dbc009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:04.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.743+0000 7f4db67fc700 1 -- 172.21.15.121:0/1578265129 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4db8047ee0 con 0x7f4dc014e380 2023-11-15T03:02:04.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.903+0000 7f4dc73b9700 1 -- 172.21.15.121:0/1578265129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4dc0068a70 con 0x7f4dc014e380 2023-11-15T03:02:04.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.903+0000 7f4db67fc700 1 -- 172.21.15.121:0/1578265129 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4db8019070 con 0x7f4dc014e380 2023-11-15T03:02:04.910 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T03:02:04.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.907+0000 7f4dc73b9700 1 -- 172.21.15.121:0/1578265129 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4da405a510 msgr2=0x7f4da405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:04.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.907+0000 7f4dc73b9700 1 --2- 172.21.15.121:0/1578265129 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4da405a510 0x7f4da405c9d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f4dbc001710 tx=0x7f4dbc009040).stop 2023-11-15T03:02:04.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.907+0000 7f4dc73b9700 1 -- 172.21.15.121:0/1578265129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dc014e380 msgr2=0x7f4dc014bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:04.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.907+0000 7f4dc73b9700 1 --2- 172.21.15.121:0/1578265129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dc014e380 0x7f4dc014bdf0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f4db800cbc0 tx=0x7f4db8001d80).stop 2023-11-15T03:02:04.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.907+0000 7f4dc73b9700 1 -- 172.21.15.121:0/1578265129 shutdown_connections 2023-11-15T03:02:04.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.907+0000 7f4dc73b9700 1 --2- 172.21.15.121:0/1578265129 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4da405a510 0x7f4da405c9d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.907+0000 7f4dc73b9700 1 --2- 172.21.15.121:0/1578265129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dc014e380 0x7f4dc014bdf0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.907+0000 7f4dc73b9700 1 --2- 172.21.15.121:0/1578265129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dc00acb60 0x7f4dc014b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.907+0000 7f4dc73b9700 1 --2- 172.21.15.121:0/1578265129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4dc0065680 0x7f4dc013e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:04.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.907+0000 7f4dc73b9700 1 -- 172.21.15.121:0/1578265129 >> 172.21.15.121:0/1578265129 conn(0x7f4dc0067a60 msgr2=0x7f4dc00ca140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:04.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.907+0000 7f4dc73b9700 1 -- 172.21.15.121:0/1578265129 shutdown_connections 2023-11-15T03:02:04.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:04.907+0000 7f4dc73b9700 1 -- 172.21.15.121:0/1578265129 wait complete. 2023-11-15T03:02:04.930 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-15T03:02:05.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.087+0000 7f6f80e39700 1 Processor -- start 2023-11-15T03:02:05.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.087+0000 7f6f80e39700 1 -- start start 2023-11-15T03:02:05.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.091+0000 7f6f80e39700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6f7c0acd00 0x7f6f7c0ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:05.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.091+0000 7f6f80e39700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f7c0c5c70 0x7f6f7c1536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:05.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.091+0000 7f6f80e39700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f7c065ea0 con 0x7f6f7c066330 2023-11-15T03:02:05.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.091+0000 7f6f80e39700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f7c066bb0 con 0x7f6f7c0acd00 2023-11-15T03:02:05.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.091+0000 7f6f80e39700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c065890 con 0x7f6f7c0c5c70 2023-11-15T03:02:05.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.091+0000 7f6f7b7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6f7c0acd00 0x7f6f7c0ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:05.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.091+0000 7f6f7bfff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6f7c066330 0x7f6f7c153bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34368/0 (socket says 172.21.15.121:34368) 2023-11-15T03:02:05.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.091+0000 7f6f7bfff700 1 -- 172.21.15.121:0/3640960400 learned_addr learned my addr 172.21.15.121:0/3640960400 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:05.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.091+0000 7f6f7affd700 1 --2- 172.21.15.121:0/3640960400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f7c0c5c70 0x7f6f7c1536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:05.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.091+0000 7f6f7b7fe700 1 -- 172.21.15.121:0/3640960400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f7c0c5c70 msgr2=0x7f6f7c1536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:05.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.091+0000 7f6f7b7fe700 1 --2- 172.21.15.121:0/3640960400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f7c0c5c70 0x7f6f7c1536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.091+0000 7f6f7b7fe700 1 -- 172.21.15.121:0/3640960400 >> v1:172.21.15.121:6789/0 conn(0x7f6f7c066330 legacy=0x7f6f7c153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:05.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.091+0000 7f6f7a7fc700 1 -- 172.21.15.121:0/3640960400 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2435997258 0 0) 0x7f6f7c065ea0 con 0x7f6f7c066330 2023-11-15T03:02:05.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.091+0000 7f6f7b7fe700 1 -- 172.21.15.121:0/3640960400 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c0c64a0 con 0x7f6f7c0acd00 2023-11-15T03:02:05.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.095+0000 7f6f7b7fe700 1 --2- 172.21.15.121:0/3640960400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f7c0acd00 0x7f6f7c0ad0e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f6f6c009750 tx=0x7f6f6c009060).ready entity=mon.0 client_cookie=1a838e71fbebdfcd server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:05.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.095+0000 7f6f7a7fc700 1 -- 172.21.15.121:0/3640960400 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c00f510 con 0x7f6f7c0acd00 2023-11-15T03:02:05.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.095+0000 7f6f7a7fc700 1 -- 172.21.15.121:0/3640960400 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c00f670 con 0x7f6f7c0acd00 2023-11-15T03:02:05.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.095+0000 7f6f7a7fc700 1 -- 172.21.15.121:0/3640960400 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c00f9f0 con 0x7f6f7c0acd00 2023-11-15T03:02:05.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.095+0000 7f6f80e39700 1 -- 172.21.15.121:0/3640960400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f7c0acd00 msgr2=0x7f6f7c0ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:05.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.095+0000 7f6f80e39700 1 --2- 172.21.15.121:0/3640960400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f7c0acd00 0x7f6f7c0ad0e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f6f6c009750 tx=0x7f6f6c009060).stop 2023-11-15T03:02:05.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.095+0000 7f6f80e39700 1 -- 172.21.15.121:0/3640960400 shutdown_connections 2023-11-15T03:02:05.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.095+0000 7f6f80e39700 1 --2- 172.21.15.121:0/3640960400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f7c0c5c70 0x7f6f7c1536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.095+0000 7f6f80e39700 1 --2- 172.21.15.121:0/3640960400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f7c0acd00 0x7f6f7c0ad0e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.095+0000 7f6f80e39700 1 -- 172.21.15.121:0/3640960400 >> 172.21.15.121:0/3640960400 conn(0x7f6f7c063fc0 msgr2=0x7f6f7c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:05.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.095+0000 7f6f80e39700 1 -- 172.21.15.121:0/3640960400 shutdown_connections 2023-11-15T03:02:05.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.095+0000 7f6f80e39700 1 -- 172.21.15.121:0/3640960400 wait complete. 2023-11-15T03:02:05.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f80e39700 1 Processor -- start 2023-11-15T03:02:05.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f80e39700 1 -- start start 2023-11-15T03:02:05.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f80e39700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f7c066330 0x7f6f7c0aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:05.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f80e39700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f7c0acd00 0x7f6f7c158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:05.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f80e39700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f7c0c5c70 0x7f6f7c0c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:05.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f80e39700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c0add70 con 0x7f6f7c0acd00 2023-11-15T03:02:05.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f80e39700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c0c54d0 con 0x7f6f7c0c5c70 2023-11-15T03:02:05.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f80e39700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c0c4f10 con 0x7f6f7c066330 2023-11-15T03:02:05.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f7affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f7c0acd00 0x7f6f7c158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:05.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f7affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f7c0acd00 0x7f6f7c158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53036/0 (socket says 172.21.15.121:53036) 2023-11-15T03:02:05.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f7affd700 1 -- 172.21.15.121:0/1781763991 learned_addr learned my addr 172.21.15.121:0/1781763991 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:05.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f7b7fe700 1 --2- 172.21.15.121:0/1781763991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f7c066330 0x7f6f7c0aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:05.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f7bfff700 1 --2- 172.21.15.121:0/1781763991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f7c0c5c70 0x7f6f7c0c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:05.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f7affd700 1 -- 172.21.15.121:0/1781763991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f7c066330 msgr2=0x7f6f7c0aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:05.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f7affd700 1 --2- 172.21.15.121:0/1781763991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f7c066330 0x7f6f7c0aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f7affd700 1 -- 172.21.15.121:0/1781763991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f7c0c5c70 msgr2=0x7f6f7c0c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:05.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f7affd700 1 --2- 172.21.15.121:0/1781763991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f7c0c5c70 0x7f6f7c0c23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f7affd700 1 -- 172.21.15.121:0/1781763991 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6c009240 con 0x7f6f7c0acd00 2023-11-15T03:02:05.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f7affd700 1 --2- 172.21.15.121:0/1781763991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f7c0acd00 0x7f6f7c158240 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f6f7000e900 tx=0x7f6f70009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:05.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.099+0000 7f6f637fe700 1 -- 172.21.15.121:0/1781763991 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70012600 con 0x7f6f7c0acd00 2023-11-15T03:02:05.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.103+0000 7f6f637fe700 1 -- 172.21.15.121:0/1781763991 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f70012760 con 0x7f6f7c0acd00 2023-11-15T03:02:05.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.103+0000 7f6f637fe700 1 -- 172.21.15.121:0/1781763991 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70012ae0 con 0x7f6f7c0acd00 2023-11-15T03:02:05.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.103+0000 7f6f80e39700 1 -- 172.21.15.121:0/1781763991 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f7c0c51f0 con 0x7f6f7c0acd00 2023-11-15T03:02:05.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.103+0000 7f6f80e39700 1 -- 172.21.15.121:0/1781763991 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f7c0af4b0 con 0x7f6f7c0acd00 2023-11-15T03:02:05.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.103+0000 7f6f80e39700 1 -- 172.21.15.121:0/1781763991 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7c0adf20 con 0x7f6f7c0acd00 2023-11-15T03:02:05.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.103+0000 7f6f637fe700 1 -- 172.21.15.121:0/1781763991 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6f7001b400 con 0x7f6f7c0acd00 2023-11-15T03:02:05.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.103+0000 7f6f637fe700 1 --2- 172.21.15.121:0/1781763991 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6f6805a510 0x7f6f6805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:05.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.103+0000 7f6f637fe700 1 -- 172.21.15.121:0/1781763991 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6380+0+0 (secure 0 0 0) 0x7f6f70074cf0 con 0x7f6f7c0acd00 2023-11-15T03:02:05.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.107+0000 7f6f7b7fe700 1 --2- 172.21.15.121:0/1781763991 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6f6805a510 0x7f6f6805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:05.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.107+0000 7f6f7b7fe700 1 --2- 172.21.15.121:0/1781763991 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6f6805a510 0x7f6f6805c9d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f6f6c009750 tx=0x7f6f6c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:05.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.111+0000 7f6f637fe700 1 -- 172.21.15.121:0/1781763991 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6f7c0adf20 con 0x7f6f7c0acd00 2023-11-15T03:02:05.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.275+0000 7f6f80e39700 1 -- 172.21.15.121:0/1781763991 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6f7c0adeb0 con 0x7f6f7c0acd00 2023-11-15T03:02:05.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.275+0000 7f6f637fe700 1 -- 172.21.15.121:0/1781763991 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6f700482b0 con 0x7f6f7c0acd00 2023-11-15T03:02:05.282 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T03:02:05.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.279+0000 7f6f80e39700 1 -- 172.21.15.121:0/1781763991 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6f6805a510 msgr2=0x7f6f6805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:05.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.279+0000 7f6f80e39700 1 --2- 172.21.15.121:0/1781763991 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6f6805a510 0x7f6f6805c9d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f6f6c009750 tx=0x7f6f6c017000).stop 2023-11-15T03:02:05.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.279+0000 7f6f80e39700 1 -- 172.21.15.121:0/1781763991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f7c0acd00 msgr2=0x7f6f7c158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:05.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.279+0000 7f6f80e39700 1 --2- 172.21.15.121:0/1781763991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f7c0acd00 0x7f6f7c158240 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f6f7000e900 tx=0x7f6f70009060).stop 2023-11-15T03:02:05.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.279+0000 7f6f80e39700 1 -- 172.21.15.121:0/1781763991 shutdown_connections 2023-11-15T03:02:05.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.279+0000 7f6f80e39700 1 --2- 172.21.15.121:0/1781763991 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6f6805a510 0x7f6f6805c9d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.279+0000 7f6f80e39700 1 --2- 172.21.15.121:0/1781763991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f7c0c5c70 0x7f6f7c0c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.279+0000 7f6f80e39700 1 --2- 172.21.15.121:0/1781763991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f7c0acd00 0x7f6f7c158240 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.279+0000 7f6f80e39700 1 --2- 172.21.15.121:0/1781763991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f7c066330 0x7f6f7c0aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.279+0000 7f6f80e39700 1 -- 172.21.15.121:0/1781763991 >> 172.21.15.121:0/1781763991 conn(0x7f6f7c063fc0 msgr2=0x7f6f7c0ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:05.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.279+0000 7f6f80e39700 1 -- 172.21.15.121:0/1781763991 shutdown_connections 2023-11-15T03:02:05.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.279+0000 7f6f80e39700 1 -- 172.21.15.121:0/1781763991 wait complete. 2023-11-15T03:02:05.308 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:02:05.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.463+0000 7f01f4c1b700 1 Processor -- start 2023-11-15T03:02:05.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.463+0000 7f01f4c1b700 1 -- start start 2023-11-15T03:02:05.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.467+0000 7f01f4c1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01f0154ca0 0x7f01f0155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:05.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.467+0000 7f01f4c1b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f01f00a57c0 0x7f01f00a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:05.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.467+0000 7f01f4c1b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f00a6160 con 0x7f01f01562f0 2023-11-15T03:02:05.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.467+0000 7f01f4c1b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f00a8280 con 0x7f01f00a57c0 2023-11-15T03:02:05.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.467+0000 7f01f4c1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f00a6d40 con 0x7f01f0154ca0 2023-11-15T03:02:05.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.467+0000 7f01eed9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f01f00a57c0 0x7f01f00a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:05.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.467+0000 7f01ee59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01f0154ca0 0x7f01f0155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:05.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.467+0000 7f01eed9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f01f00a57c0 0x7f01f00a5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53050/0 (socket says 172.21.15.121:53050) 2023-11-15T03:02:05.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.467+0000 7f01eed9d700 1 -- 172.21.15.121:0/245936061 learned_addr learned my addr 172.21.15.121:0/245936061 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:05.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.467+0000 7f01edd9b700 1 --1- 172.21.15.121:0/245936061 >> v1:172.21.15.190:6789/0 conn(0x7f01f01562f0 0x7f01f00a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:34524/0 (socket says 172.21.15.121:34524) 2023-11-15T03:02:05.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.467+0000 7f01eed9d700 1 -- 172.21.15.121:0/245936061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01f0154ca0 msgr2=0x7f01f0155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:05.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.467+0000 7f01eed9d700 1 --2- 172.21.15.121:0/245936061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01f0154ca0 0x7f01f0155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.467+0000 7f01eed9d700 1 -- 172.21.15.121:0/245936061 >> v1:172.21.15.190:6789/0 conn(0x7f01f01562f0 legacy=0x7f01f00a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:05.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.467+0000 7f01eed9d700 1 -- 172.21.15.121:0/245936061 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f00c4c30 con 0x7f01f00a57c0 2023-11-15T03:02:05.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.467+0000 7f01ed59a700 1 -- 172.21.15.121:0/245936061 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218380749 0 0) 0x7f01e4001110 con 0x7f01f01562f0 2023-11-15T03:02:05.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.471+0000 7f01eed9d700 1 --2- 172.21.15.121:0/245936061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01f00a57c0 0x7f01f00a5c70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f01e8009f20 tx=0x7f01e8009840).ready entity=mon.0 client_cookie=545cf4a3f411073 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:05.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.471+0000 7f01ed59a700 1 -- 172.21.15.121:0/245936061 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e800e070 con 0x7f01f00a57c0 2023-11-15T03:02:05.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.471+0000 7f01ed59a700 1 -- 172.21.15.121:0/245936061 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e800b9d0 con 0x7f01f00a57c0 2023-11-15T03:02:05.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.471+0000 7f01ed59a700 1 -- 172.21.15.121:0/245936061 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e800bd50 con 0x7f01f00a57c0 2023-11-15T03:02:05.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.471+0000 7f01f4c1b700 1 -- 172.21.15.121:0/245936061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01f00a57c0 msgr2=0x7f01f00a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:05.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.471+0000 7f01f4c1b700 1 --2- 172.21.15.121:0/245936061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01f00a57c0 0x7f01f00a5c70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f01e8009f20 tx=0x7f01e8009840).stop 2023-11-15T03:02:05.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.471+0000 7f01f4c1b700 1 -- 172.21.15.121:0/245936061 shutdown_connections 2023-11-15T03:02:05.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.471+0000 7f01f4c1b700 1 --2- 172.21.15.121:0/245936061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01f00a57c0 0x7f01f00a5c70 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.471+0000 7f01f4c1b700 1 --2- 172.21.15.121:0/245936061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01f0154ca0 0x7f01f0155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.471+0000 7f01f4c1b700 1 -- 172.21.15.121:0/245936061 >> 172.21.15.121:0/245936061 conn(0x7f01f0062190 msgr2=0x7f01f00625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:05.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.471+0000 7f01f4c1b700 1 -- 172.21.15.121:0/245936061 shutdown_connections 2023-11-15T03:02:05.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.471+0000 7f01f4c1b700 1 -- 172.21.15.121:0/245936061 wait complete. 2023-11-15T03:02:05.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01f4c1b700 1 Processor -- start 2023-11-15T03:02:05.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01f4c1b700 1 -- start start 2023-11-15T03:02:05.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01f4c1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01f00a57c0 0x7f01f0162510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:05.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01f4c1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01f0154ca0 0x7f01f00ac260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:05.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01f4c1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01f01562f0 0x7f01f0165120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:05.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01f4c1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f0166990 con 0x7f01f00a57c0 2023-11-15T03:02:05.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01f4c1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f0165ae0 con 0x7f01f0154ca0 2023-11-15T03:02:05.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01f4c1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f0165d80 con 0x7f01f01562f0 2023-11-15T03:02:05.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01ee59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01f00a57c0 0x7f01f0162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:05.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01ee59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01f00a57c0 0x7f01f0162510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53066/0 (socket says 172.21.15.121:53066) 2023-11-15T03:02:05.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01ee59c700 1 -- 172.21.15.121:0/2266087297 learned_addr learned my addr 172.21.15.121:0/2266087297 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:05.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01edd9b700 1 --2- 172.21.15.121:0/2266087297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01f0154ca0 0x7f01f00ac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:05.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01eed9d700 1 --2- 172.21.15.121:0/2266087297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01f01562f0 0x7f01f0165120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:05.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01ee59c700 1 -- 172.21.15.121:0/2266087297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01f01562f0 msgr2=0x7f01f0165120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:05.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01ee59c700 1 --2- 172.21.15.121:0/2266087297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01f01562f0 0x7f01f0165120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01ee59c700 1 -- 172.21.15.121:0/2266087297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01f0154ca0 msgr2=0x7f01f00ac260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:05.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01ee59c700 1 --2- 172.21.15.121:0/2266087297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01f0154ca0 0x7f01f00ac260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01ee59c700 1 -- 172.21.15.121:0/2266087297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e8009ae0 con 0x7f01f00a57c0 2023-11-15T03:02:05.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01eed9d700 1 --2- 172.21.15.121:0/2266087297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01f01562f0 0x7f01f0165120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:02:05.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01ee59c700 1 --2- 172.21.15.121:0/2266087297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01f00a57c0 0x7f01f0162510 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f01e000e930 tx=0x7f01e0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:05.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.475+0000 7f01de7fc700 1 -- 172.21.15.121:0/2266087297 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e000eed0 con 0x7f01f00a57c0 2023-11-15T03:02:05.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.479+0000 7f01de7fc700 1 -- 172.21.15.121:0/2266087297 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e000c850 con 0x7f01f00a57c0 2023-11-15T03:02:05.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.479+0000 7f01f4c1b700 1 -- 172.21.15.121:0/2266087297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f0166060 con 0x7f01f00a57c0 2023-11-15T03:02:05.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.479+0000 7f01de7fc700 1 -- 172.21.15.121:0/2266087297 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e000cbd0 con 0x7f01f00a57c0 2023-11-15T03:02:05.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.479+0000 7f01f4c1b700 1 -- 172.21.15.121:0/2266087297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f0166620 con 0x7f01f00a57c0 2023-11-15T03:02:05.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.479+0000 7f01f4c1b700 1 -- 172.21.15.121:0/2266087297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f006c150 con 0x7f01f00a57c0 2023-11-15T03:02:05.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.479+0000 7f01de7fc700 1 -- 172.21.15.121:0/2266087297 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f01e001b400 con 0x7f01f00a57c0 2023-11-15T03:02:05.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.483+0000 7f01de7fc700 1 --2- 172.21.15.121:0/2266087297 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f01d005a510 0x7f01d005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:05.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.483+0000 7f01de7fc700 1 -- 172.21.15.121:0/2266087297 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6380+0+0 (secure 0 0 0) 0x7f01e0074e50 con 0x7f01f00a57c0 2023-11-15T03:02:05.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.483+0000 7f01edd9b700 1 --2- 172.21.15.121:0/2266087297 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f01d005a510 0x7f01d005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:05.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.483+0000 7f01edd9b700 1 --2- 172.21.15.121:0/2266087297 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f01d005a510 0x7f01d005c9d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f01f00a5790 tx=0x7f01e4007180).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:05.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.487+0000 7f01de7fc700 1 -- 172.21.15.121:0/2266087297 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f01e0048410 con 0x7f01f00a57c0 2023-11-15T03:02:05.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.679+0000 7f01f4c1b700 1 -- 172.21.15.121:0/2266087297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01f00c4500 con 0x7f01f00a57c0 2023-11-15T03:02:05.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.679+0000 7f01de7fc700 1 -- 172.21.15.121:0/2266087297 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4504 (secure 0 0 0) 0x7f01e004ba30 con 0x7f01f00a57c0 2023-11-15T03:02:05.687 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:05.687 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3462858472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3462858472},{"type":"v1","addr":"172.21.15.190:6833","nonce":3462858472}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4042390087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4042390087},{"type":"v1","addr":"172.21.15.190:6835","nonce":4042390087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3908033272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3908033272},{"type":"v1","addr":"172.21.15.121:6836","nonce":3908033272}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:59:34.667864+0000","modified":"2023-11-15T03:00:22.468286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5907},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5907":{"gid":5907,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":16,"addr":"172.21.15.121:6837/2239114529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2239114529},{"type":"v1","addr":"172.21.15.121:6837","nonce":2239114529}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-15T03:02:05.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.683+0000 7f01f4c1b700 1 -- 172.21.15.121:0/2266087297 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f01d005a510 msgr2=0x7f01d005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:05.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.683+0000 7f01f4c1b700 1 --2- 172.21.15.121:0/2266087297 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f01d005a510 0x7f01d005c9d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f01f00a5790 tx=0x7f01e4007180).stop 2023-11-15T03:02:05.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.683+0000 7f01f4c1b700 1 -- 172.21.15.121:0/2266087297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01f00a57c0 msgr2=0x7f01f0162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:05.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.683+0000 7f01f4c1b700 1 --2- 172.21.15.121:0/2266087297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01f00a57c0 0x7f01f0162510 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f01e000e930 tx=0x7f01e0009060).stop 2023-11-15T03:02:05.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.683+0000 7f01f4c1b700 1 -- 172.21.15.121:0/2266087297 shutdown_connections 2023-11-15T03:02:05.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.683+0000 7f01f4c1b700 1 --2- 172.21.15.121:0/2266087297 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f01d005a510 0x7f01d005c9d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.683+0000 7f01f4c1b700 1 --2- 172.21.15.121:0/2266087297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01f01562f0 0x7f01f0165120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.683+0000 7f01f4c1b700 1 --2- 172.21.15.121:0/2266087297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01f0154ca0 0x7f01f00ac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.683+0000 7f01f4c1b700 1 --2- 172.21.15.121:0/2266087297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01f00a57c0 0x7f01f0162510 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.683+0000 7f01f4c1b700 1 -- 172.21.15.121:0/2266087297 >> 172.21.15.121:0/2266087297 conn(0x7f01f0062190 msgr2=0x7f01f00a8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:05.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.683+0000 7f01f4c1b700 1 -- 172.21.15.121:0/2266087297 shutdown_connections 2023-11-15T03:02:05.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.683+0000 7f01f4c1b700 1 -- 172.21.15.121:0/2266087297 wait complete. 2023-11-15T03:02:05.692 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 59 2023-11-15T03:02:05.709 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:02:05.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.875+0000 7fa6b9969700 1 Processor -- start 2023-11-15T03:02:05.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.875+0000 7fa6b9969700 1 -- start start 2023-11-15T03:02:05.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.879+0000 7fa6b9969700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6b40a90b0 0x7fa6b4066c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:05.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.879+0000 7fa6b9969700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b40c0d80 con 0x7fa6b40a8cd0 2023-11-15T03:02:05.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.879+0000 7fa6b9969700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b40c1060 con 0x7fa6b4067170 2023-11-15T03:02:05.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.879+0000 7fa6b9969700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b4067600 con 0x7fa6b40a90b0 2023-11-15T03:02:05.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.879+0000 7fa6b2ffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa6b40a8cd0 0x7fa6b40aa550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34376/0 (socket says 172.21.15.121:34376) 2023-11-15T03:02:05.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.879+0000 7fa6b2ffd700 1 -- 172.21.15.121:0/3988902197 learned_addr learned my addr 172.21.15.121:0/3988902197 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:05.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.879+0000 7fa6aa7fc700 1 --2- 172.21.15.121:0/3988902197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6b40a90b0 0x7fa6b4066c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:05.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.879+0000 7fa6b27fc700 1 -- 172.21.15.121:0/3988902197 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605224392 0 0) 0x7fa6b40c0d80 con 0x7fa6b40a8cd0 2023-11-15T03:02:05.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.879+0000 7fa6b27fc700 1 -- 172.21.15.121:0/3988902197 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6940033f0 con 0x7fa6b40a8cd0 2023-11-15T03:02:05.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.879+0000 7fa6b27fc700 1 -- 172.21.15.121:0/3988902197 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3659581400 0 0) 0x7fa6b40c1060 con 0x7fa6b4067170 2023-11-15T03:02:05.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.879+0000 7fa6b27fc700 1 -- 172.21.15.121:0/3988902197 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6b40c0d80 con 0x7fa6b4067170 2023-11-15T03:02:05.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.879+0000 7fa6aa7fc700 1 -- 172.21.15.121:0/3988902197 >> v1:172.21.15.190:6789/0 conn(0x7fa6b4067170 legacy=0x7fa6b40c1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:05.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.879+0000 7fa6aa7fc700 1 -- 172.21.15.121:0/3988902197 >> v1:172.21.15.121:6789/0 conn(0x7fa6b40a8cd0 legacy=0x7fa6b40aa550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:05.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.879+0000 7fa6aa7fc700 1 -- 172.21.15.121:0/3988902197 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b40c1340 con 0x7fa6b40a90b0 2023-11-15T03:02:05.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.879+0000 7fa6aa7fc700 1 --2- 172.21.15.121:0/3988902197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6b40a90b0 0x7fa6b4066c80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fa69c009790 tx=0x7fa69c009060).ready entity=mon.2 client_cookie=4dbc81af6edba191 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:05.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.879+0000 7fa6b27fc700 1 -- 172.21.15.121:0/3988902197 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c010620 con 0x7fa6b40a90b0 2023-11-15T03:02:05.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.879+0000 7fa6b27fc700 1 -- 172.21.15.121:0/3988902197 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c010780 con 0x7fa6b40a90b0 2023-11-15T03:02:05.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.879+0000 7fa6b27fc700 1 -- 172.21.15.121:0/3988902197 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c010b00 con 0x7fa6b40a90b0 2023-11-15T03:02:05.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.883+0000 7fa6b9969700 1 -- 172.21.15.121:0/3988902197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6b40a90b0 msgr2=0x7fa6b4066c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:05.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.883+0000 7fa6b9969700 1 --2- 172.21.15.121:0/3988902197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6b40a90b0 0x7fa6b4066c80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fa69c009790 tx=0x7fa69c009060).stop 2023-11-15T03:02:05.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.883+0000 7fa6b9969700 1 -- 172.21.15.121:0/3988902197 shutdown_connections 2023-11-15T03:02:05.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.883+0000 7fa6b9969700 1 --2- 172.21.15.121:0/3988902197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6b40a90b0 0x7fa6b4066c80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.883+0000 7fa6b9969700 1 -- 172.21.15.121:0/3988902197 >> 172.21.15.121:0/3988902197 conn(0x7fa6b4062190 msgr2=0x7fa6b4062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:05.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.883+0000 7fa6b9969700 1 -- 172.21.15.121:0/3988902197 shutdown_connections 2023-11-15T03:02:05.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.883+0000 7fa6b9969700 1 -- 172.21.15.121:0/3988902197 wait complete. 2023-11-15T03:02:05.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.883+0000 7fa6b9969700 1 Processor -- start 2023-11-15T03:02:05.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.883+0000 7fa6b9969700 1 -- start start 2023-11-15T03:02:05.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6b9969700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6b4067170 0x7fa6b4146cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:05.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6b9969700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6b40a8cd0 0x7fa6b415a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:05.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6b9969700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6b40a90b0 0x7fa6b415a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:05.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6b9969700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b415abd0 con 0x7fa6b4067170 2023-11-15T03:02:05.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6b9969700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b414c720 con 0x7fa6b40a90b0 2023-11-15T03:02:05.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6b9969700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b40c01f0 con 0x7fa6b40a8cd0 2023-11-15T03:02:05.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6b2ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6b4067170 0x7fa6b4146cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:05.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6b2ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6b4067170 0x7fa6b4146cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53070/0 (socket says 172.21.15.121:53070) 2023-11-15T03:02:05.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6b2ffd700 1 -- 172.21.15.121:0/2024181124 learned_addr learned my addr 172.21.15.121:0/2024181124 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:05.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6b37fe700 1 --2- 172.21.15.121:0/2024181124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6b40a90b0 0x7fa6b415a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:05.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6aa7fc700 1 --2- 172.21.15.121:0/2024181124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6b40a8cd0 0x7fa6b415a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:05.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6b2ffd700 1 -- 172.21.15.121:0/2024181124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6b40a8cd0 msgr2=0x7fa6b415a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:05.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6b2ffd700 1 --2- 172.21.15.121:0/2024181124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6b40a8cd0 0x7fa6b415a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6b2ffd700 1 -- 172.21.15.121:0/2024181124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6b40a90b0 msgr2=0x7fa6b415a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:05.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6b2ffd700 1 --2- 172.21.15.121:0/2024181124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6b40a90b0 0x7fa6b415a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:05.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6b2ffd700 1 -- 172.21.15.121:0/2024181124 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69c0092a0 con 0x7fa6b4067170 2023-11-15T03:02:05.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6b2ffd700 1 --2- 172.21.15.121:0/2024181124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6b4067170 0x7fa6b4146cd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7fa6a400c490 tx=0x7fa6a4007d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:05.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6ab7fe700 1 -- 172.21.15.121:0/2024181124 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a40070d0 con 0x7fa6b4067170 2023-11-15T03:02:05.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6ab7fe700 1 -- 172.21.15.121:0/2024181124 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a4007230 con 0x7fa6b4067170 2023-11-15T03:02:05.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6b9969700 1 -- 172.21.15.121:0/2024181124 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b414c0d0 con 0x7fa6b4067170 2023-11-15T03:02:05.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6ab7fe700 1 -- 172.21.15.121:0/2024181124 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a40075b0 con 0x7fa6b4067170 2023-11-15T03:02:05.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.887+0000 7fa6b9969700 1 -- 172.21.15.121:0/2024181124 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b414bcf0 con 0x7fa6b4067170 2023-11-15T03:02:05.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.891+0000 7fa6b9969700 1 -- 172.21.15.121:0/2024181124 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b414bee0 con 0x7fa6b4067170 2023-11-15T03:02:05.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.891+0000 7fa6ab7fe700 1 -- 172.21.15.121:0/2024181124 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa6a40059b0 con 0x7fa6b4067170 2023-11-15T03:02:05.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.895+0000 7fa6ab7fe700 1 --2- 172.21.15.121:0/2024181124 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa69805a540 0x7fa69805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:05.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.895+0000 7fa6ab7fe700 1 -- 172.21.15.121:0/2024181124 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6380+0+0 (secure 0 0 0) 0x7fa6a4073ec0 con 0x7fa6b4067170 2023-11-15T03:02:05.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.895+0000 7fa6aa7fc700 1 --2- 172.21.15.121:0/2024181124 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa69805a540 0x7fa69805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:05.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.895+0000 7fa6aa7fc700 1 --2- 172.21.15.121:0/2024181124 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa69805a540 0x7fa69805ca00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7fa69c009fd0 tx=0x7fa69c018000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:05.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:05.899+0000 7fa6ab7fe700 1 -- 172.21.15.121:0/2024181124 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa6a40475a0 con 0x7fa6b4067170 2023-11-15T03:02:06.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.067+0000 7fa6b9969700 1 -- 172.21.15.121:0/2024181124 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6b40c1620 con 0x7fa6b4067170 2023-11-15T03:02:06.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.071+0000 7fa6ab7fe700 1 -- 172.21.15.121:0/2024181124 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+17387 (secure 0 0 0) 0x7fa6a4007e60 con 0x7fa6b4067170 2023-11-15T03:02:06.077 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:06.077 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":65,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:00:39.942021+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:59:32.734725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-15T02:59:33.665552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.1d","mappings":[{"from":4,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]},{"pgid":"10.3c","mappings":[{"from":4,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6836/3429169582":"2023-11-16T03:00:19.439898+0000","172.21.15.121:6835/3429169582":"2023-11-16T03:00:19.439898+0000","172.21.15.121:6836/2394175138":"2023-11-16T02:59:48.732462+0000","172.21.15.121:6835/2394175138":"2023-11-16T02:59:48.732462+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:02:06.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.071+0000 7fa6b9969700 1 -- 172.21.15.121:0/2024181124 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa69805a540 msgr2=0x7fa69805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:06.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.071+0000 7fa6b9969700 1 --2- 172.21.15.121:0/2024181124 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa69805a540 0x7fa69805ca00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7fa69c009fd0 tx=0x7fa69c018000).stop 2023-11-15T03:02:06.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.071+0000 7fa6b9969700 1 -- 172.21.15.121:0/2024181124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6b4067170 msgr2=0x7fa6b4146cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:06.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.071+0000 7fa6b9969700 1 --2- 172.21.15.121:0/2024181124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6b4067170 0x7fa6b4146cd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7fa6a400c490 tx=0x7fa6a4007d20).stop 2023-11-15T03:02:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.071+0000 7fa6b9969700 1 -- 172.21.15.121:0/2024181124 shutdown_connections 2023-11-15T03:02:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.071+0000 7fa6b9969700 1 --2- 172.21.15.121:0/2024181124 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa69805a540 0x7fa69805ca00 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.071+0000 7fa6b9969700 1 --2- 172.21.15.121:0/2024181124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6b40a90b0 0x7fa6b415a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.071+0000 7fa6b9969700 1 --2- 172.21.15.121:0/2024181124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6b40a8cd0 0x7fa6b415a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.071+0000 7fa6b9969700 1 --2- 172.21.15.121:0/2024181124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6b4067170 0x7fa6b4146cd0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.071+0000 7fa6b9969700 1 -- 172.21.15.121:0/2024181124 >> 172.21.15.121:0/2024181124 conn(0x7fa6b4062190 msgr2=0x7fa6b40aac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:06.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.075+0000 7fa6b9969700 1 -- 172.21.15.121:0/2024181124 shutdown_connections 2023-11-15T03:02:06.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.075+0000 7fa6b9969700 1 -- 172.21.15.121:0/2024181124 wait complete. 2023-11-15T03:02:06.097 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-15T03:02:06.098 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:02:06.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.247+0000 7ff71745b700 1 Processor -- start 2023-11-15T03:02:06.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.247+0000 7ff71745b700 1 -- start start 2023-11-15T03:02:06.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.251+0000 7ff71745b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff710154ca0 0x7ff710155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:06.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.251+0000 7ff71745b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7100a57c0 0x7ff7100a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:06.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.251+0000 7ff71745b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7100c4c30 con 0x7ff7101562f0 2023-11-15T03:02:06.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.251+0000 7ff71745b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7100a8280 con 0x7ff710154ca0 2023-11-15T03:02:06.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.251+0000 7ff71745b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7100a6d40 con 0x7ff7100a57c0 2023-11-15T03:02:06.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.251+0000 7ff7149f6700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff7101562f0 0x7ff7100a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34390/0 (socket says 172.21.15.121:34390) 2023-11-15T03:02:06.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.251+0000 7ff7149f6700 1 -- 172.21.15.121:0/1550678314 learned_addr learned my addr 172.21.15.121:0/1550678314 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:06.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.251+0000 7ff7151f7700 1 --2- 172.21.15.121:0/1550678314 >> v2:172.21.15.121:3300/0 conn(0x7ff710154ca0 0x7ff710155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:06.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.251+0000 7ff7159f8700 1 --2- 172.21.15.121:0/1550678314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7100a57c0 0x7ff7100a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:06.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.251+0000 7ff707fff700 1 -- 172.21.15.121:0/1550678314 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1685020196 0 0) 0x7ff7100c4c30 con 0x7ff7101562f0 2023-11-15T03:02:06.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.251+0000 7ff707fff700 1 -- 172.21.15.121:0/1550678314 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6fc0033f0 con 0x7ff7101562f0 2023-11-15T03:02:06.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.251+0000 7ff7151f7700 1 -- 172.21.15.121:0/1550678314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7100a57c0 msgr2=0x7ff7100a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:06.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.251+0000 7ff7151f7700 1 --2- 172.21.15.121:0/1550678314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7100a57c0 0x7ff7100a5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.251+0000 7ff7151f7700 1 -- 172.21.15.121:0/1550678314 >> v1:172.21.15.121:6789/0 conn(0x7ff7101562f0 legacy=0x7ff7100a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:06.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.251+0000 7ff7151f7700 1 -- 172.21.15.121:0/1550678314 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7100a6140 con 0x7ff710154ca0 2023-11-15T03:02:06.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.255+0000 7ff7151f7700 1 --2- 172.21.15.121:0/1550678314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff710154ca0 0x7ff710155080 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7ff70c009760 tx=0x7ff70c009060).ready entity=mon.0 client_cookie=f707d37a697ea070 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:06.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.255+0000 7ff707fff700 1 -- 172.21.15.121:0/1550678314 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c00f510 con 0x7ff710154ca0 2023-11-15T03:02:06.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.255+0000 7ff707fff700 1 -- 172.21.15.121:0/1550678314 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70c00f670 con 0x7ff710154ca0 2023-11-15T03:02:06.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.255+0000 7ff707fff700 1 -- 172.21.15.121:0/1550678314 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c00f9f0 con 0x7ff710154ca0 2023-11-15T03:02:06.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.255+0000 7ff71745b700 1 -- 172.21.15.121:0/1550678314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff710154ca0 msgr2=0x7ff710155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:06.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.255+0000 7ff71745b700 1 --2- 172.21.15.121:0/1550678314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff710154ca0 0x7ff710155080 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7ff70c009760 tx=0x7ff70c009060).stop 2023-11-15T03:02:06.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.255+0000 7ff71745b700 1 -- 172.21.15.121:0/1550678314 shutdown_connections 2023-11-15T03:02:06.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.255+0000 7ff71745b700 1 --2- 172.21.15.121:0/1550678314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7100a57c0 0x7ff7100a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.255+0000 7ff71745b700 1 --2- 172.21.15.121:0/1550678314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff710154ca0 0x7ff710155080 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.255+0000 7ff71745b700 1 -- 172.21.15.121:0/1550678314 >> 172.21.15.121:0/1550678314 conn(0x7ff710062190 msgr2=0x7ff7100625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:06.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.255+0000 7ff71745b700 1 -- 172.21.15.121:0/1550678314 shutdown_connections 2023-11-15T03:02:06.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.255+0000 7ff71745b700 1 -- 172.21.15.121:0/1550678314 wait complete. 2023-11-15T03:02:06.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.259+0000 7ff71745b700 1 Processor -- start 2023-11-15T03:02:06.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.259+0000 7ff71745b700 1 -- start start 2023-11-15T03:02:06.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.259+0000 7ff71745b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7100a57c0 0x7ff7100aa960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:06.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.259+0000 7ff71745b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff710154ca0 0x7ff7100ac180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:06.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.259+0000 7ff71745b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7101562f0 0x7ff7101660c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:06.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.259+0000 7ff71745b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff710165c30 con 0x7ff7101562f0 2023-11-15T03:02:06.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.259+0000 7ff71745b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71014c7b0 con 0x7ff7100a57c0 2023-11-15T03:02:06.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.259+0000 7ff71745b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7100aae50 con 0x7ff710154ca0 2023-11-15T03:02:06.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.259+0000 7ff7159f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7101562f0 0x7ff7101660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:06.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.259+0000 7ff7159f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7101562f0 0x7ff7101660c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53088/0 (socket says 172.21.15.121:53088) 2023-11-15T03:02:06.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.259+0000 7ff7159f8700 1 -- 172.21.15.121:0/3252938841 learned_addr learned my addr 172.21.15.121:0/3252938841 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:06.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.259+0000 7ff7151f7700 1 --2- 172.21.15.121:0/3252938841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7100a57c0 0x7ff7100aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:06.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.259+0000 7ff7149f6700 1 --2- 172.21.15.121:0/3252938841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff710154ca0 0x7ff7100ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:06.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.263+0000 7ff7159f8700 1 -- 172.21.15.121:0/3252938841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff710154ca0 msgr2=0x7ff7100ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:06.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.263+0000 7ff7159f8700 1 --2- 172.21.15.121:0/3252938841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff710154ca0 0x7ff7100ac180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.263+0000 7ff7159f8700 1 -- 172.21.15.121:0/3252938841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7100a57c0 msgr2=0x7ff7100aa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:06.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.263+0000 7ff7159f8700 1 --2- 172.21.15.121:0/3252938841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7100a57c0 0x7ff7100aa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.263+0000 7ff7159f8700 1 -- 172.21.15.121:0/3252938841 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70c0092a0 con 0x7ff7101562f0 2023-11-15T03:02:06.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.263+0000 7ff7159f8700 1 --2- 172.21.15.121:0/3252938841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7101562f0 0x7ff7101660c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7ff708009fa0 tx=0x7ff708009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:06.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.263+0000 7ff7057fa700 1 -- 172.21.15.121:0/3252938841 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70800ebc0 con 0x7ff7101562f0 2023-11-15T03:02:06.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.263+0000 7ff7057fa700 1 -- 172.21.15.121:0/3252938841 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70800ed20 con 0x7ff7101562f0 2023-11-15T03:02:06.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.263+0000 7ff71745b700 1 -- 172.21.15.121:0/3252938841 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff710146d10 con 0x7ff7101562f0 2023-11-15T03:02:06.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.263+0000 7ff7057fa700 1 -- 172.21.15.121:0/3252938841 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708014610 con 0x7ff7101562f0 2023-11-15T03:02:06.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.263+0000 7ff71745b700 1 -- 172.21.15.121:0/3252938841 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7101658c0 con 0x7ff7101562f0 2023-11-15T03:02:06.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.263+0000 7ff71745b700 1 -- 172.21.15.121:0/3252938841 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff710165de0 con 0x7ff7101562f0 2023-11-15T03:02:06.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.267+0000 7ff7057fa700 1 -- 172.21.15.121:0/3252938841 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff70801d3b0 con 0x7ff7101562f0 2023-11-15T03:02:06.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.267+0000 7ff7057fa700 1 --2- 172.21.15.121:0/3252938841 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff6f405a510 0x7ff6f405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:06.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.267+0000 7ff7057fa700 1 -- 172.21.15.121:0/3252938841 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6380+0+0 (secure 0 0 0) 0x7ff708074800 con 0x7ff7101562f0 2023-11-15T03:02:06.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.267+0000 7ff7151f7700 1 --2- 172.21.15.121:0/3252938841 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff6f405a510 0x7ff6f405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:06.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.267+0000 7ff7151f7700 1 --2- 172.21.15.121:0/3252938841 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff6f405a510 0x7ff6f405c9d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7ff70c009fd0 tx=0x7ff70c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:06.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.275+0000 7ff7057fa700 1 -- 172.21.15.121:0/3252938841 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff710165de0 con 0x7ff7101562f0 2023-11-15T03:02:06.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.467+0000 7ff71745b700 1 -- 172.21.15.121:0/3252938841 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff71006c150 con 0x7ff7101562f0 2023-11-15T03:02:06.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.467+0000 7ff7057fa700 1 -- 172.21.15.121:0/3252938841 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4504 (secure 0 0 0) 0x7ff708047f30 con 0x7ff7101562f0 2023-11-15T03:02:06.473 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:06.473 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3462858472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3462858472},{"type":"v1","addr":"172.21.15.190:6833","nonce":3462858472}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4042390087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4042390087},{"type":"v1","addr":"172.21.15.190:6835","nonce":4042390087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3908033272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3908033272},{"type":"v1","addr":"172.21.15.121:6836","nonce":3908033272}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:59:34.667864+0000","modified":"2023-11-15T03:00:22.468286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5907},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5907":{"gid":5907,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":16,"addr":"172.21.15.121:6837/2239114529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2239114529},{"type":"v1","addr":"172.21.15.121:6837","nonce":2239114529}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-15T03:02:06.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.471+0000 7ff71745b700 1 -- 172.21.15.121:0/3252938841 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff6f405a510 msgr2=0x7ff6f405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:06.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.471+0000 7ff71745b700 1 --2- 172.21.15.121:0/3252938841 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff6f405a510 0x7ff6f405c9d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7ff70c009fd0 tx=0x7ff70c017000).stop 2023-11-15T03:02:06.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.471+0000 7ff71745b700 1 -- 172.21.15.121:0/3252938841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7101562f0 msgr2=0x7ff7101660c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:06.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.471+0000 7ff71745b700 1 --2- 172.21.15.121:0/3252938841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7101562f0 0x7ff7101660c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7ff708009fa0 tx=0x7ff708009900).stop 2023-11-15T03:02:06.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.471+0000 7ff71745b700 1 -- 172.21.15.121:0/3252938841 shutdown_connections 2023-11-15T03:02:06.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.471+0000 7ff71745b700 1 --2- 172.21.15.121:0/3252938841 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff6f405a510 0x7ff6f405c9d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.471+0000 7ff71745b700 1 --2- 172.21.15.121:0/3252938841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7101562f0 0x7ff7101660c0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.471+0000 7ff71745b700 1 --2- 172.21.15.121:0/3252938841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff710154ca0 0x7ff7100ac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.471+0000 7ff71745b700 1 --2- 172.21.15.121:0/3252938841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7100a57c0 0x7ff7100aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.471+0000 7ff71745b700 1 -- 172.21.15.121:0/3252938841 >> 172.21.15.121:0/3252938841 conn(0x7ff710062190 msgr2=0x7ff7100c5510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:06.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.471+0000 7ff71745b700 1 -- 172.21.15.121:0/3252938841 shutdown_connections 2023-11-15T03:02:06.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.471+0000 7ff71745b700 1 -- 172.21.15.121:0/3252938841 wait complete. 2023-11-15T03:02:06.479 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 59 2023-11-15T03:02:06.498 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:02:06.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.647+0000 7fd2382db700 1 Processor -- start 2023-11-15T03:02:06.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.647+0000 7fd2382db700 1 -- start start 2023-11-15T03:02:06.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.651+0000 7fd2382db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd230154ca0 0x7fd230155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:06.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.651+0000 7fd2382db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2301562f0 0x7fd2300a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:06.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.651+0000 7fd2382db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2300c4c30 con 0x7fd2300a8f40 2023-11-15T03:02:06.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.651+0000 7fd2382db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2300a8280 con 0x7fd230154ca0 2023-11-15T03:02:06.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.651+0000 7fd2382db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2300a6d40 con 0x7fd2301562f0 2023-11-15T03:02:06.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.651+0000 7fd236878700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fd2300a8f40 0x7fd2300a57c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34398/0 (socket says 172.21.15.121:34398) 2023-11-15T03:02:06.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd236878700 1 -- 172.21.15.121:0/3386920610 learned_addr learned my addr 172.21.15.121:0/3386920610 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:06.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd236077700 1 --2- 172.21.15.121:0/3386920610 >> v2:172.21.15.190:3300/0 conn(0x7fd230154ca0 0x7fd230155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:06.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd235876700 1 --2- 172.21.15.121:0/3386920610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2301562f0 0x7fd2300a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:06.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd235075700 1 -- 172.21.15.121:0/3386920610 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 501922652 0 0) 0x7fd2300c4c30 con 0x7fd2300a8f40 2023-11-15T03:02:06.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd235075700 1 -- 172.21.15.121:0/3386920610 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd21c0033f0 con 0x7fd2300a8f40 2023-11-15T03:02:06.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd236077700 1 -- 172.21.15.121:0/3386920610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2301562f0 msgr2=0x7fd2300a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:06.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd236077700 1 --2- 172.21.15.121:0/3386920610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2301562f0 0x7fd2300a8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd236077700 1 -- 172.21.15.121:0/3386920610 >> v1:172.21.15.121:6789/0 conn(0x7fd2300a8f40 legacy=0x7fd2300a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:06.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd236077700 1 -- 172.21.15.121:0/3386920610 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2300a6f50 con 0x7fd230154ca0 2023-11-15T03:02:06.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd236077700 1 --2- 172.21.15.121:0/3386920610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd230154ca0 0x7fd230155080 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fd22c009e60 tx=0x7fd22c009780).ready entity=mon.1 client_cookie=47db58e703e73814 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:06.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd235075700 1 -- 172.21.15.121:0/3386920610 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c00e070 con 0x7fd230154ca0 2023-11-15T03:02:06.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd235075700 1 -- 172.21.15.121:0/3386920610 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22c00bab0 con 0x7fd230154ca0 2023-11-15T03:02:06.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd235075700 1 -- 172.21.15.121:0/3386920610 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c00be30 con 0x7fd230154ca0 2023-11-15T03:02:06.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd2382db700 1 -- 172.21.15.121:0/3386920610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd230154ca0 msgr2=0x7fd230155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:06.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd2382db700 1 --2- 172.21.15.121:0/3386920610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd230154ca0 0x7fd230155080 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fd22c009e60 tx=0x7fd22c009780).stop 2023-11-15T03:02:06.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd2382db700 1 -- 172.21.15.121:0/3386920610 shutdown_connections 2023-11-15T03:02:06.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd2382db700 1 --2- 172.21.15.121:0/3386920610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2301562f0 0x7fd2300a8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd2382db700 1 --2- 172.21.15.121:0/3386920610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd230154ca0 0x7fd230155080 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd2382db700 1 -- 172.21.15.121:0/3386920610 >> 172.21.15.121:0/3386920610 conn(0x7fd230062190 msgr2=0x7fd2300625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:06.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd2382db700 1 -- 172.21.15.121:0/3386920610 shutdown_connections 2023-11-15T03:02:06.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.655+0000 7fd2382db700 1 -- 172.21.15.121:0/3386920610 wait complete. 2023-11-15T03:02:06.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.659+0000 7fd2382db700 1 Processor -- start 2023-11-15T03:02:06.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.659+0000 7fd2382db700 1 -- start start 2023-11-15T03:02:06.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.659+0000 7fd2382db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2300a8f40 0x7fd230156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:06.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.659+0000 7fd2382db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd230154ca0 0x7fd2300c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:06.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.659+0000 7fd2382db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2301562f0 0x7fd23014b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:06.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.659+0000 7fd2382db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2300c72b0 con 0x7fd2300a8f40 2023-11-15T03:02:06.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.659+0000 7fd2382db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2301687f0 con 0x7fd2301562f0 2023-11-15T03:02:06.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.659+0000 7fd2382db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd230167be0 con 0x7fd230154ca0 2023-11-15T03:02:06.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.659+0000 7fd236077700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2300a8f40 0x7fd230156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:06.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.659+0000 7fd236077700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2300a8f40 0x7fd230156d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53092/0 (socket says 172.21.15.121:53092) 2023-11-15T03:02:06.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.659+0000 7fd236077700 1 -- 172.21.15.121:0/1601222207 learned_addr learned my addr 172.21.15.121:0/1601222207 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:06.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.659+0000 7fd236878700 1 --2- 172.21.15.121:0/1601222207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2301562f0 0x7fd23014b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:06.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.659+0000 7fd235876700 1 --2- 172.21.15.121:0/1601222207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd230154ca0 0x7fd2300c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:06.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.663+0000 7fd236077700 1 -- 172.21.15.121:0/1601222207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd230154ca0 msgr2=0x7fd2300c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:06.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.663+0000 7fd236077700 1 --2- 172.21.15.121:0/1601222207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd230154ca0 0x7fd2300c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.663+0000 7fd236077700 1 -- 172.21.15.121:0/1601222207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2301562f0 msgr2=0x7fd23014b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:06.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.663+0000 7fd236077700 1 --2- 172.21.15.121:0/1601222207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2301562f0 0x7fd23014b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.663+0000 7fd236077700 1 -- 172.21.15.121:0/1601222207 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c009a20 con 0x7fd2300a8f40 2023-11-15T03:02:06.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.663+0000 7fd236077700 1 --2- 172.21.15.121:0/1601222207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2300a8f40 0x7fd230156d50 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fd22c013960 tx=0x7fd2300c4ba0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:06.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.663+0000 7fd2267fc700 1 -- 172.21.15.121:0/1601222207 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c01d420 con 0x7fd2300a8f40 2023-11-15T03:02:06.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.663+0000 7fd2267fc700 1 -- 172.21.15.121:0/1601222207 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22c01d580 con 0x7fd2300a8f40 2023-11-15T03:02:06.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.663+0000 7fd2267fc700 1 -- 172.21.15.121:0/1601222207 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c01d900 con 0x7fd2300a8f40 2023-11-15T03:02:06.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.663+0000 7fd2382db700 1 -- 172.21.15.121:0/1601222207 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd230167ec0 con 0x7fd2300a8f40 2023-11-15T03:02:06.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.663+0000 7fd2382db700 1 -- 172.21.15.121:0/1601222207 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd230168480 con 0x7fd2300a8f40 2023-11-15T03:02:06.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.663+0000 7fd2382db700 1 -- 172.21.15.121:0/1601222207 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd23006c150 con 0x7fd2300a8f40 2023-11-15T03:02:06.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.667+0000 7fd2267fc700 1 -- 172.21.15.121:0/1601222207 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd22c003040 con 0x7fd2300a8f40 2023-11-15T03:02:06.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.667+0000 7fd2267fc700 1 --2- 172.21.15.121:0/1601222207 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd21405a510 0x7fd21405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:06.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.667+0000 7fd2267fc700 1 -- 172.21.15.121:0/1601222207 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6380+0+0 (secure 0 0 0) 0x7fd22c076660 con 0x7fd2300a8f40 2023-11-15T03:02:06.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.667+0000 7fd235876700 1 --2- 172.21.15.121:0/1601222207 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd21405a510 0x7fd21405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:06.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.667+0000 7fd235876700 1 --2- 172.21.15.121:0/1601222207 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd21405a510 0x7fd21405c9d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7fd2300c3810 tx=0x7fd220008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:06.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.671+0000 7fd2267fc700 1 -- 172.21.15.121:0/1601222207 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd22c049c80 con 0x7fd2300a8f40 2023-11-15T03:02:06.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.875+0000 7fd2382db700 1 -- 172.21.15.121:0/1601222207 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2300c4500 con 0x7fd2300a8f40 2023-11-15T03:02:06.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.875+0000 7fd2267fc700 1 -- 172.21.15.121:0/1601222207 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4504 (secure 0 0 0) 0x7fd22c04d2a0 con 0x7fd2300a8f40 2023-11-15T03:02:06.883 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:06.883 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3462858472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3462858472},{"type":"v1","addr":"172.21.15.190:6833","nonce":3462858472}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4042390087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4042390087},{"type":"v1","addr":"172.21.15.190:6835","nonce":4042390087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3908033272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3908033272},{"type":"v1","addr":"172.21.15.121:6836","nonce":3908033272}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T02:59:34.667864+0000","modified":"2023-11-15T03:00:22.468286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5907},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5907":{"gid":5907,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":16,"addr":"172.21.15.121:6837/2239114529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2239114529},{"type":"v1","addr":"172.21.15.121:6837","nonce":2239114529}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-15T03:02:06.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.879+0000 7fd2382db700 1 -- 172.21.15.121:0/1601222207 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd21405a510 msgr2=0x7fd21405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:06.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.879+0000 7fd2382db700 1 --2- 172.21.15.121:0/1601222207 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd21405a510 0x7fd21405c9d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7fd2300c3810 tx=0x7fd220008040).stop 2023-11-15T03:02:06.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.879+0000 7fd2382db700 1 -- 172.21.15.121:0/1601222207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2300a8f40 msgr2=0x7fd230156d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:06.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.879+0000 7fd2382db700 1 --2- 172.21.15.121:0/1601222207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2300a8f40 0x7fd230156d50 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fd22c013960 tx=0x7fd2300c4ba0).stop 2023-11-15T03:02:06.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.879+0000 7fd2382db700 1 -- 172.21.15.121:0/1601222207 shutdown_connections 2023-11-15T03:02:06.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.879+0000 7fd2382db700 1 --2- 172.21.15.121:0/1601222207 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd21405a510 0x7fd21405c9d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.879+0000 7fd2382db700 1 --2- 172.21.15.121:0/1601222207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2301562f0 0x7fd23014b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.879+0000 7fd2382db700 1 --2- 172.21.15.121:0/1601222207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd230154ca0 0x7fd2300c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.879+0000 7fd2382db700 1 --2- 172.21.15.121:0/1601222207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2300a8f40 0x7fd230156d50 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:06.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.879+0000 7fd2382db700 1 -- 172.21.15.121:0/1601222207 >> 172.21.15.121:0/1601222207 conn(0x7fd230062190 msgr2=0x7fd23014e340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:06.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.883+0000 7fd2382db700 1 -- 172.21.15.121:0/1601222207 shutdown_connections 2023-11-15T03:02:06.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:06.883+0000 7fd2382db700 1 -- 172.21.15.121:0/1601222207 wait complete. 2023-11-15T03:02:06.891 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 59 2023-11-15T03:02:06.907 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:02:07.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.051+0000 7f1c138ff700 1 Processor -- start 2023-11-15T03:02:07.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.051+0000 7f1c138ff700 1 -- start start 2023-11-15T03:02:07.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.055+0000 7f1c138ff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1c0c0aa240 0x7f1c0c067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:07.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.055+0000 7f1c138ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c0c067f50 0x7f1c0c065d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:07.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.055+0000 7f1c138ff700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c0c0cb5c0 con 0x7f1c0c153bb0 2023-11-15T03:02:07.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.055+0000 7f1c138ff700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c0c0aa6f0 con 0x7f1c0c0aa240 2023-11-15T03:02:07.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.055+0000 7f1c138ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c0c5900 con 0x7f1c0c067f50 2023-11-15T03:02:07.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.055+0000 7f1c1169b700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f1c0c153bb0 0x7f1c0c0a9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34406/0 (socket says 172.21.15.121:34406) 2023-11-15T03:02:07.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.055+0000 7f1c1169b700 1 -- 172.21.15.121:0/4018102667 learned_addr learned my addr 172.21.15.121:0/4018102667 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:07.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.055+0000 7f1c10e9a700 1 --2- 172.21.15.121:0/4018102667 >> v2:172.21.15.121:3300/0 conn(0x7f1c0c0aa240 0x7f1c0c067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:07.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.055+0000 7f1c11e9c700 1 --2- 172.21.15.121:0/4018102667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c0c067f50 0x7f1c0c065d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:07.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.055+0000 7f1c03fff700 1 -- 172.21.15.121:0/4018102667 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3602600905 0 0) 0x7f1c0c0cb5c0 con 0x7f1c0c153bb0 2023-11-15T03:02:07.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.055+0000 7f1c03fff700 1 -- 172.21.15.121:0/4018102667 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bf80033f0 con 0x7f1c0c153bb0 2023-11-15T03:02:07.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.059+0000 7f1c10e9a700 1 -- 172.21.15.121:0/4018102667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c0c067f50 msgr2=0x7f1c0c065d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:07.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.059+0000 7f1c10e9a700 1 --2- 172.21.15.121:0/4018102667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c0c067f50 0x7f1c0c065d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:07.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.059+0000 7f1c10e9a700 1 -- 172.21.15.121:0/4018102667 >> v1:172.21.15.121:6789/0 conn(0x7f1c0c153bb0 legacy=0x7f1c0c0a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:07.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.059+0000 7f1c10e9a700 1 -- 172.21.15.121:0/4018102667 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0c0c9ca0 con 0x7f1c0c0aa240 2023-11-15T03:02:07.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.059+0000 7f1c10e9a700 1 --2- 172.21.15.121:0/4018102667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c0aa240 0x7f1c0c067a60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f1c0c0c5a40 tx=0x7f1bfc009060).ready entity=mon.0 client_cookie=c6a18874138829c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:07.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.059+0000 7f1c03fff700 1 -- 172.21.15.121:0/4018102667 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc00d4a0 con 0x7f1c0c0aa240 2023-11-15T03:02:07.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.059+0000 7f1c03fff700 1 -- 172.21.15.121:0/4018102667 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bfc00d600 con 0x7f1c0c0aa240 2023-11-15T03:02:07.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.059+0000 7f1c03fff700 1 -- 172.21.15.121:0/4018102667 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc00d980 con 0x7f1c0c0aa240 2023-11-15T03:02:07.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.059+0000 7f1c138ff700 1 -- 172.21.15.121:0/4018102667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c0aa240 msgr2=0x7f1c0c067a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:07.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.059+0000 7f1c138ff700 1 --2- 172.21.15.121:0/4018102667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c0aa240 0x7f1c0c067a60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f1c0c0c5a40 tx=0x7f1bfc009060).stop 2023-11-15T03:02:07.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.059+0000 7f1c138ff700 1 -- 172.21.15.121:0/4018102667 shutdown_connections 2023-11-15T03:02:07.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.059+0000 7f1c138ff700 1 --2- 172.21.15.121:0/4018102667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c0c067f50 0x7f1c0c065d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:07.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.059+0000 7f1c138ff700 1 --2- 172.21.15.121:0/4018102667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c0aa240 0x7f1c0c067a60 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:07.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.059+0000 7f1c138ff700 1 -- 172.21.15.121:0/4018102667 >> 172.21.15.121:0/4018102667 conn(0x7f1c0c058c50 msgr2=0x7f1c0c059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:07.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.059+0000 7f1c138ff700 1 -- 172.21.15.121:0/4018102667 shutdown_connections 2023-11-15T03:02:07.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.059+0000 7f1c138ff700 1 -- 172.21.15.121:0/4018102667 wait complete. 2023-11-15T03:02:07.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.063+0000 7f1c138ff700 1 Processor -- start 2023-11-15T03:02:07.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.063+0000 7f1c138ff700 1 -- start start 2023-11-15T03:02:07.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.063+0000 7f1c138ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c0c067f50 0x7f1c0c14e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:07.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.063+0000 7f1c138ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c0c0aa240 0x7f1c0c150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:07.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.063+0000 7f1c138ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c153bb0 0x7f1c0c1511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:07.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.063+0000 7f1c138ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c1516c0 con 0x7f1c0c153bb0 2023-11-15T03:02:07.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.063+0000 7f1c138ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c0ad8f0 con 0x7f1c0c0aa240 2023-11-15T03:02:07.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.063+0000 7f1c138ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c1524d0 con 0x7f1c0c067f50 2023-11-15T03:02:07.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.063+0000 7f1c11e9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c153bb0 0x7f1c0c1511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:07.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.063+0000 7f1c11e9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c153bb0 0x7f1c0c1511d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53108/0 (socket says 172.21.15.121:53108) 2023-11-15T03:02:07.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.063+0000 7f1c11e9c700 1 -- 172.21.15.121:0/2098472643 learned_addr learned my addr 172.21.15.121:0/2098472643 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:07.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.063+0000 7f1c10e9a700 1 --2- 172.21.15.121:0/2098472643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c0c0aa240 0x7f1c0c150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:07.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.063+0000 7f1c1169b700 1 --2- 172.21.15.121:0/2098472643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c0c067f50 0x7f1c0c14e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:07.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.063+0000 7f1c11e9c700 1 -- 172.21.15.121:0/2098472643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c0c067f50 msgr2=0x7f1c0c14e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:07.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.063+0000 7f1c11e9c700 1 --2- 172.21.15.121:0/2098472643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c0c067f50 0x7f1c0c14e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:07.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.063+0000 7f1c11e9c700 1 -- 172.21.15.121:0/2098472643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c0c0aa240 msgr2=0x7f1c0c150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:07.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.063+0000 7f1c11e9c700 1 --2- 172.21.15.121:0/2098472643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c0c0aa240 0x7f1c0c150ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:07.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.063+0000 7f1c11e9c700 1 -- 172.21.15.121:0/2098472643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc0092a0 con 0x7f1c0c153bb0 2023-11-15T03:02:07.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.067+0000 7f1c11e9c700 1 --2- 172.21.15.121:0/2098472643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c153bb0 0x7f1c0c1511d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f1c04009e90 tx=0x7f1c0400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:07.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.067+0000 7f1c017fa700 1 -- 172.21.15.121:0/2098472643 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0400cc20 con 0x7f1c0c153bb0 2023-11-15T03:02:07.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.067+0000 7f1c017fa700 1 -- 172.21.15.121:0/2098472643 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0400f040 con 0x7f1c0c153bb0 2023-11-15T03:02:07.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.067+0000 7f1c017fa700 1 -- 172.21.15.121:0/2098472643 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c04013400 con 0x7f1c0c153bb0 2023-11-15T03:02:07.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.067+0000 7f1c138ff700 1 -- 172.21.15.121:0/2098472643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c0c1518c0 con 0x7f1c0c153bb0 2023-11-15T03:02:07.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.067+0000 7f1c138ff700 1 -- 172.21.15.121:0/2098472643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c0c151e80 con 0x7f1c0c153bb0 2023-11-15T03:02:07.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.067+0000 7f1c138ff700 1 -- 172.21.15.121:0/2098472643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c0c152640 con 0x7f1c0c153bb0 2023-11-15T03:02:07.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.067+0000 7f1c017fa700 1 -- 172.21.15.121:0/2098472643 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1c040137b0 con 0x7f1c0c153bb0 2023-11-15T03:02:07.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.071+0000 7f1c017fa700 1 --2- 172.21.15.121:0/2098472643 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1bf005a510 0x7f1bf005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:07.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.071+0000 7f1c017fa700 1 -- 172.21.15.121:0/2098472643 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6380+0+0 (secure 0 0 0) 0x7f1c04075200 con 0x7f1c0c153bb0 2023-11-15T03:02:07.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.071+0000 7f1c1169b700 1 --2- 172.21.15.121:0/2098472643 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1bf005a510 0x7f1bf005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:07.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.071+0000 7f1c1169b700 1 --2- 172.21.15.121:0/2098472643 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1bf005a510 0x7f1bf005c9d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f1c0c0aad40 tx=0x7f1c08009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:07.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.075+0000 7f1c017fa700 1 -- 172.21.15.121:0/2098472643 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1c040488e0 con 0x7f1c0c153bb0 2023-11-15T03:02:07.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.239+0000 7f1c138ff700 1 -- 172.21.15.121:0/2098472643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c0c06c150 con 0x7f1c0c153bb0 2023-11-15T03:02:07.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.243+0000 7f1c017fa700 1 -- 172.21.15.121:0/2098472643 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+17387 (secure 0 0 0) 0x7f1c0404bf00 con 0x7f1c0c153bb0 2023-11-15T03:02:07.249 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:07.249 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":65,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:00:39.942021+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-15T02:59:32.734725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-15T02:59:33.665552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.1d","mappings":[{"from":4,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]},{"pgid":"10.3c","mappings":[{"from":4,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6836/3429169582":"2023-11-16T03:00:19.439898+0000","172.21.15.121:6835/3429169582":"2023-11-16T03:00:19.439898+0000","172.21.15.121:6836/2394175138":"2023-11-16T02:59:48.732462+0000","172.21.15.121:6835/2394175138":"2023-11-16T02:59:48.732462+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:02:07.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.243+0000 7f1c138ff700 1 -- 172.21.15.121:0/2098472643 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1bf005a510 msgr2=0x7f1bf005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:07.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.243+0000 7f1c138ff700 1 --2- 172.21.15.121:0/2098472643 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1bf005a510 0x7f1bf005c9d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f1c0c0aad40 tx=0x7f1c08009040).stop 2023-11-15T03:02:07.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.243+0000 7f1c138ff700 1 -- 172.21.15.121:0/2098472643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c153bb0 msgr2=0x7f1c0c1511d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:07.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.243+0000 7f1c138ff700 1 --2- 172.21.15.121:0/2098472643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c153bb0 0x7f1c0c1511d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f1c04009e90 tx=0x7f1c0400b040).stop 2023-11-15T03:02:07.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.243+0000 7f1c138ff700 1 -- 172.21.15.121:0/2098472643 shutdown_connections 2023-11-15T03:02:07.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.243+0000 7f1c138ff700 1 --2- 172.21.15.121:0/2098472643 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1bf005a510 0x7f1bf005c9d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:07.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.243+0000 7f1c138ff700 1 --2- 172.21.15.121:0/2098472643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c153bb0 0x7f1c0c1511d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:07.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.243+0000 7f1c138ff700 1 --2- 172.21.15.121:0/2098472643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c0c0aa240 0x7f1c0c150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:07.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.243+0000 7f1c138ff700 1 --2- 172.21.15.121:0/2098472643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c0c067f50 0x7f1c0c14e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:07.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.243+0000 7f1c138ff700 1 -- 172.21.15.121:0/2098472643 >> 172.21.15.121:0/2098472643 conn(0x7f1c0c058c50 msgr2=0x7f1c0c0bfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:07.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.247+0000 7f1c138ff700 1 -- 172.21.15.121:0/2098472643 shutdown_connections 2023-11-15T03:02:07.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.247+0000 7f1c138ff700 1 -- 172.21.15.121:0/2098472643 wait complete. 2023-11-15T03:02:07.272 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:02:07.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.427+0000 7f94acf07700 1 Processor -- start 2023-11-15T03:02:07.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.427+0000 7f94acf07700 1 -- start start 2023-11-15T03:02:07.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.427+0000 7f94acf07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94a80669b0 0x7f94a8158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:07.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.427+0000 7f94acf07700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f94a80a8750 0x7f94a80a8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:07.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.427+0000 7f94acf07700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a80a90d0 con 0x7f94a8157d90 2023-11-15T03:02:07.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.427+0000 7f94acf07700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a80c1fb0 con 0x7f94a80a8750 2023-11-15T03:02:07.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.427+0000 7f94acf07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a815bbd0 con 0x7f94a80669b0 2023-11-15T03:02:07.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.427+0000 7f94a7fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f94a80a8750 0x7f94a80a8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:07.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.427+0000 7f94a7fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f94a80a8750 0x7f94a80a8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53110/0 (socket says 172.21.15.121:53110) 2023-11-15T03:02:07.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.427+0000 7f94a7fff700 1 -- 172.21.15.121:0/1043965772 learned_addr learned my addr 172.21.15.121:0/1043965772 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:07.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.427+0000 7f94a6ffd700 1 --2- 172.21.15.121:0/1043965772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94a80669b0 0x7f94a8158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:07.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.431+0000 7f94a7fff700 1 -- 172.21.15.121:0/1043965772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94a80669b0 msgr2=0x7f94a8158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:07.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.431+0000 7f94a7fff700 1 --2- 172.21.15.121:0/1043965772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94a80669b0 0x7f94a8158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:07.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.431+0000 7f94a7fff700 1 -- 172.21.15.121:0/1043965772 >> v1:172.21.15.190:6789/0 conn(0x7f94a8157d90 legacy=0x7f94a80c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:07.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.431+0000 7f94a7fff700 1 -- 172.21.15.121:0/1043965772 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a80bb910 con 0x7f94a80a8750 2023-11-15T03:02:07.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.431+0000 7f94a67fc700 1 -- 172.21.15.121:0/1043965772 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1300481833 0 0) 0x7f94a80a90d0 con 0x7f94a8157d90 2023-11-15T03:02:07.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.431+0000 7f94a7fff700 1 --2- 172.21.15.121:0/1043965772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a80a8750 0x7f94a80a8be0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f94a0009f20 tx=0x7f94a0009840).ready entity=mon.0 client_cookie=f25699d3ddc3bdff server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:07.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.431+0000 7f94a67fc700 1 -- 172.21.15.121:0/1043965772 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a000e070 con 0x7f94a80a8750 2023-11-15T03:02:07.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.431+0000 7f94a67fc700 1 -- 172.21.15.121:0/1043965772 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a000b9d0 con 0x7f94a80a8750 2023-11-15T03:02:07.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.431+0000 7f94a67fc700 1 -- 172.21.15.121:0/1043965772 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a000bd50 con 0x7f94a80a8750 2023-11-15T03:02:07.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.431+0000 7f94acf07700 1 -- 172.21.15.121:0/1043965772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a80a8750 msgr2=0x7f94a80a8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:07.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.431+0000 7f94acf07700 1 --2- 172.21.15.121:0/1043965772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a80a8750 0x7f94a80a8be0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f94a0009f20 tx=0x7f94a0009840).stop 2023-11-15T03:02:07.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.431+0000 7f94acf07700 1 -- 172.21.15.121:0/1043965772 shutdown_connections 2023-11-15T03:02:07.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.431+0000 7f94acf07700 1 --2- 172.21.15.121:0/1043965772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a80a8750 0x7f94a80a8be0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:07.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.431+0000 7f94acf07700 1 --2- 172.21.15.121:0/1043965772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94a80669b0 0x7f94a8158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:07.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.431+0000 7f94acf07700 1 -- 172.21.15.121:0/1043965772 >> 172.21.15.121:0/1043965772 conn(0x7f94a8067a60 msgr2=0x7f94a8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:07.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.431+0000 7f94acf07700 1 -- 172.21.15.121:0/1043965772 shutdown_connections 2023-11-15T03:02:07.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.431+0000 7f94acf07700 1 -- 172.21.15.121:0/1043965772 wait complete. 2023-11-15T03:02:07.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.435+0000 7f94acf07700 1 Processor -- start 2023-11-15T03:02:07.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.435+0000 7f94acf07700 1 -- start start 2023-11-15T03:02:07.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.435+0000 7f94acf07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94a80669b0 0x7f94a80ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:07.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.435+0000 7f94acf07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a80a8750 0x7f94a8169c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:07.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.435+0000 7f94acf07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94a8157d90 0x7f94a816a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:07.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.435+0000 7f94acf07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a80a9c80 con 0x7f94a80a8750 2023-11-15T03:02:07.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.435+0000 7f94acf07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a8157670 con 0x7f94a80669b0 2023-11-15T03:02:07.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.435+0000 7f94acf07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a8169640 con 0x7f94a8157d90 2023-11-15T03:02:07.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.439+0000 7f94a77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94a80669b0 0x7f94a80ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:07.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.439+0000 7f94a6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a80a8750 0x7f94a8169c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:07.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.439+0000 7f94a77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94a80669b0 0x7f94a80ab5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:51956/0 (socket says 172.21.15.121:51956) 2023-11-15T03:02:07.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.439+0000 7f94a77fe700 1 -- 172.21.15.121:0/2102417988 learned_addr learned my addr 172.21.15.121:0/2102417988 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:07.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.439+0000 7f94a7fff700 1 --2- 172.21.15.121:0/2102417988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94a8157d90 0x7f94a816a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:07.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.439+0000 7f94a6ffd700 1 -- 172.21.15.121:0/2102417988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94a8157d90 msgr2=0x7f94a816a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:07.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.439+0000 7f94a6ffd700 1 --2- 172.21.15.121:0/2102417988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94a8157d90 0x7f94a816a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:07.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.439+0000 7f94a6ffd700 1 -- 172.21.15.121:0/2102417988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94a80669b0 msgr2=0x7f94a80ab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:07.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.439+0000 7f94a6ffd700 1 --2- 172.21.15.121:0/2102417988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94a80669b0 0x7f94a80ab5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:07.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.439+0000 7f94a6ffd700 1 -- 172.21.15.121:0/2102417988 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a0009ae0 con 0x7f94a80a8750 2023-11-15T03:02:07.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.439+0000 7f94a77fe700 1 --2- 172.21.15.121:0/2102417988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94a80669b0 0x7f94a80ab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:02:07.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.439+0000 7f94a6ffd700 1 --2- 172.21.15.121:0/2102417988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a80a8750 0x7f94a8169c50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7f949c00e8d0 tx=0x7f949c006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:07.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.439+0000 7f948f7fe700 1 -- 172.21.15.121:0/2102417988 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c009350 con 0x7f94a80a8750 2023-11-15T03:02:07.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.439+0000 7f948f7fe700 1 -- 172.21.15.121:0/2102417988 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c012600 con 0x7f94a80a8750 2023-11-15T03:02:07.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.439+0000 7f94acf07700 1 -- 172.21.15.121:0/2102417988 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a8169920 con 0x7f94a80a8750 2023-11-15T03:02:07.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.439+0000 7f948f7fe700 1 -- 172.21.15.121:0/2102417988 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c012980 con 0x7f94a80a8750 2023-11-15T03:02:07.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.439+0000 7f94acf07700 1 -- 172.21.15.121:0/2102417988 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a80a98a0 con 0x7f94a80a8750 2023-11-15T03:02:07.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.443+0000 7f94acf07700 1 -- 172.21.15.121:0/2102417988 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a80a9ef0 con 0x7f94a80a8750 2023-11-15T03:02:07.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.443+0000 7f948f7fe700 1 -- 172.21.15.121:0/2102417988 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f949c012d30 con 0x7f94a80a8750 2023-11-15T03:02:07.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.443+0000 7f948f7fe700 1 --2- 172.21.15.121:0/2102417988 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f949405a510 0x7f949405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:07.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.443+0000 7f948f7fe700 1 -- 172.21.15.121:0/2102417988 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6380+0+0 (secure 0 0 0) 0x7f949c074ac0 con 0x7f94a80a8750 2023-11-15T03:02:07.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.443+0000 7f94a77fe700 1 --2- 172.21.15.121:0/2102417988 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f949405a510 0x7f949405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:07.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.447+0000 7f94a77fe700 1 --2- 172.21.15.121:0/2102417988 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f949405a510 0x7f949405c9d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f94980011e0 tx=0x7f9498009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:07.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.451+0000 7f948f7fe700 1 -- 172.21.15.121:0/2102417988 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f949c0481a0 con 0x7f94a80a8750 2023-11-15T03:02:07.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:07.647+0000 7f94acf07700 1 -- 172.21.15.121:0/2102417988 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f94a806bcf0 con 0x7f94a80a8750 2023-11-15T03:02:08.140 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:02:08.135+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:02:08.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.147+0000 7f948f7fe700 1 -- 172.21.15.121:0/2102417988 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) v1 ==== 161+0+0 (secure 0 0 0) 0x7f949c074e60 con 0x7f94a80a8750 2023-11-15T03:02:08.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.151+0000 7f94acf07700 1 -- 172.21.15.121:0/2102417988 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f949405a510 msgr2=0x7f949405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:08.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.151+0000 7f94acf07700 1 --2- 172.21.15.121:0/2102417988 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f949405a510 0x7f949405c9d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f94980011e0 tx=0x7f9498009040).stop 2023-11-15T03:02:08.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.151+0000 7f94acf07700 1 -- 172.21.15.121:0/2102417988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a80a8750 msgr2=0x7f94a8169c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:08.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.151+0000 7f94acf07700 1 --2- 172.21.15.121:0/2102417988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a80a8750 0x7f94a8169c50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7f949c00e8d0 tx=0x7f949c006f40).stop 2023-11-15T03:02:08.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.151+0000 7f94acf07700 1 -- 172.21.15.121:0/2102417988 shutdown_connections 2023-11-15T03:02:08.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.151+0000 7f94acf07700 1 --2- 172.21.15.121:0/2102417988 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f949405a510 0x7f949405c9d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:08.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.151+0000 7f94acf07700 1 --2- 172.21.15.121:0/2102417988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94a8157d90 0x7f94a816a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:08.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.151+0000 7f94acf07700 1 --2- 172.21.15.121:0/2102417988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a80a8750 0x7f94a8169c50 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:08.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.151+0000 7f94acf07700 1 --2- 172.21.15.121:0/2102417988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94a80669b0 0x7f94a80ab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:08.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.151+0000 7f94acf07700 1 -- 172.21.15.121:0/2102417988 >> 172.21.15.121:0/2102417988 conn(0x7f94a8067a60 msgr2=0x7f94a80c5860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:08.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.151+0000 7f94acf07700 1 -- 172.21.15.121:0/2102417988 shutdown_connections 2023-11-15T03:02:08.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.151+0000 7f94acf07700 1 -- 172.21.15.121:0/2102417988 wait complete. 2023-11-15T03:02:08.161 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:02:08.169 INFO:tasks.ceph.mds.c.smithi121.stderr: -3326> 2023-11-15T03:00:25.201+0000 7f258eedc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:02:08.180 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-15T03:02:08.224 INFO:tasks.ceph.mds.c.smithi121.stdout:starting mds.c at 2023-11-15T03:02:08.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.347+0000 7f981e107700 1 Processor -- start 2023-11-15T03:02:08.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.347+0000 7f981e107700 1 -- start start 2023-11-15T03:02:08.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981e107700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f98180acd00 0x7f98180ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:08.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981e107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98180c5c70 0x7f98181536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:08.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981e107700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9818066330 0x7f9818153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:08.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981e107700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9818065ea0 con 0x7f9818066330 2023-11-15T03:02:08.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981e107700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9818066bb0 con 0x7f98180acd00 2023-11-15T03:02:08.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981e107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9818065890 con 0x7f98180c5c70 2023-11-15T03:02:08.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981c904700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98180c5c70 0x7f98181536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:08.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981d105700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f98180acd00 0x7f98180ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:08.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981c904700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98180c5c70 0x7f98181536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:52490/0 (socket says 172.21.15.121:52490) 2023-11-15T03:02:08.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981c904700 1 -- 172.21.15.121:0/549897004 learned_addr learned my addr 172.21.15.121:0/549897004 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:08.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981d105700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f98180acd00 0x7f98180ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:51994/0 (socket says 172.21.15.121:51994) 2023-11-15T03:02:08.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981d906700 1 --2- 172.21.15.121:0/549897004 >> v2:172.21.15.121:3300/0 conn(0x7f9818066330 0x7f9818153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:08.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981c904700 1 -- 172.21.15.121:0/549897004 >> v2:172.21.15.190:3300/0 conn(0x7f98180acd00 msgr2=0x7f98180ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:08.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981c904700 1 --2- 172.21.15.121:0/549897004 >> v2:172.21.15.190:3300/0 conn(0x7f98180acd00 0x7f98180ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:08.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981c904700 1 -- 172.21.15.121:0/549897004 >> v2:172.21.15.121:3300/0 conn(0x7f9818066330 msgr2=0x7f9818153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:08.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981c904700 1 --2- 172.21.15.121:0/549897004 >> v2:172.21.15.121:3300/0 conn(0x7f9818066330 0x7f9818153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:08.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981c904700 1 -- 172.21.15.121:0/549897004 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98180c64a0 con 0x7f98180c5c70 2023-11-15T03:02:08.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981c904700 1 --2- 172.21.15.121:0/549897004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98180c5c70 0x7f98181536c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f981806c280 tx=0x7f98080090f0).ready entity=mon.2 client_cookie=a3552a7acb5daba3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:08.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f980ffff700 1 -- 172.21.15.121:0/549897004 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980800f510 con 0x7f98180c5c70 2023-11-15T03:02:08.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f980ffff700 1 -- 172.21.15.121:0/549897004 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980800f670 con 0x7f98180c5c70 2023-11-15T03:02:08.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f980ffff700 1 -- 172.21.15.121:0/549897004 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980800f9f0 con 0x7f98180c5c70 2023-11-15T03:02:08.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981e107700 1 -- 172.21.15.121:0/549897004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98180c5c70 msgr2=0x7f98181536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:08.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.351+0000 7f981e107700 1 --2- 172.21.15.121:0/549897004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98180c5c70 0x7f98181536c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f981806c280 tx=0x7f98080090f0).stop 2023-11-15T03:02:08.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.355+0000 7f981e107700 1 -- 172.21.15.121:0/549897004 shutdown_connections 2023-11-15T03:02:08.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.355+0000 7f981e107700 1 --2- 172.21.15.121:0/549897004 >> v2:172.21.15.121:3300/0 conn(0x7f9818066330 0x7f9818153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:08.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.355+0000 7f981e107700 1 --2- 172.21.15.121:0/549897004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98180c5c70 0x7f98181536c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:08.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.355+0000 7f981e107700 1 --2- 172.21.15.121:0/549897004 >> v2:172.21.15.190:3300/0 conn(0x7f98180acd00 0x7f98180ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:08.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.355+0000 7f981e107700 1 -- 172.21.15.121:0/549897004 >> 172.21.15.121:0/549897004 conn(0x7f9818063fc0 msgr2=0x7f9818067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:08.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.355+0000 7f981e107700 1 -- 172.21.15.121:0/549897004 shutdown_connections 2023-11-15T03:02:08.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.355+0000 7f981e107700 1 -- 172.21.15.121:0/549897004 wait complete. 2023-11-15T03:02:08.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.355+0000 7f981e107700 1 Processor -- start 2023-11-15T03:02:08.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.355+0000 7f981e107700 1 -- start start 2023-11-15T03:02:08.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.355+0000 7f981e107700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9818066330 0x7f9818165460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:08.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.355+0000 7f981e107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98180acd00 0x7f98181670d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:08.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.355+0000 7f981e107700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98180c5c70 0x7f9818153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:08.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.355+0000 7f981e107700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9818168830 con 0x7f98180c5c70 2023-11-15T03:02:08.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.355+0000 7f981e107700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9818165950 con 0x7f9818066330 2023-11-15T03:02:08.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.355+0000 7f981e107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98180c31f0 con 0x7f98180acd00 2023-11-15T03:02:08.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.359+0000 7f981d906700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98180c5c70 0x7f9818153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:08.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.359+0000 7f981d906700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98180c5c70 0x7f9818153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53144/0 (socket says 172.21.15.121:53144) 2023-11-15T03:02:08.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.359+0000 7f981d105700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9818066330 0x7f9818165460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:08.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.359+0000 7f981d906700 1 -- 172.21.15.121:0/480107014 learned_addr learned my addr 172.21.15.121:0/480107014 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:08.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.359+0000 7f981c904700 1 --2- 172.21.15.121:0/480107014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98180acd00 0x7f98181670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:08.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.359+0000 7f981d906700 1 -- 172.21.15.121:0/480107014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98180acd00 msgr2=0x7f98181670d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:08.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.359+0000 7f981d906700 1 --2- 172.21.15.121:0/480107014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98180acd00 0x7f98181670d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:08.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.359+0000 7f981d906700 1 -- 172.21.15.121:0/480107014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9818066330 msgr2=0x7f9818165460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:08.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.359+0000 7f981d906700 1 --2- 172.21.15.121:0/480107014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9818066330 0x7f9818165460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:08.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.359+0000 7f981d906700 1 -- 172.21.15.121:0/480107014 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9808009370 con 0x7f98180c5c70 2023-11-15T03:02:08.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.359+0000 7f981d105700 1 --2- 172.21.15.121:0/480107014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9818066330 0x7f9818165460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:02:08.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.359+0000 7f981d906700 1 --2- 172.21.15.121:0/480107014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98180c5c70 0x7f9818153490 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f9810009e90 tx=0x7f981000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:08.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.359+0000 7f980d7fa700 1 -- 172.21.15.121:0/480107014 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981000caa0 con 0x7f98180c5c70 2023-11-15T03:02:08.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.359+0000 7f980d7fa700 1 -- 172.21.15.121:0/480107014 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981000f040 con 0x7f98180c5c70 2023-11-15T03:02:08.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.359+0000 7f981e107700 1 -- 172.21.15.121:0/480107014 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98181675c0 con 0x7f98180c5c70 2023-11-15T03:02:08.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.359+0000 7f980d7fa700 1 -- 172.21.15.121:0/480107014 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981000ce10 con 0x7f98180c5c70 2023-11-15T03:02:08.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.359+0000 7f981e107700 1 -- 172.21.15.121:0/480107014 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98180c2ba0 con 0x7f98180c5c70 2023-11-15T03:02:08.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.359+0000 7f981e107700 1 -- 172.21.15.121:0/480107014 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98180c2a00 con 0x7f98180c5c70 2023-11-15T03:02:08.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.363+0000 7f980d7fa700 1 -- 172.21.15.121:0/480107014 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9810013610 con 0x7f98180c5c70 2023-11-15T03:02:08.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.363+0000 7f980d7fa700 1 --2- 172.21.15.121:0/480107014 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f97fc05a510 0x7f97fc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:08.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.363+0000 7f980d7fa700 1 -- 172.21.15.121:0/480107014 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f98100751c0 con 0x7f98180c5c70 2023-11-15T03:02:08.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.363+0000 7f981d105700 1 --2- 172.21.15.121:0/480107014 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f97fc05a510 0x7f97fc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:08.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.363+0000 7f981d105700 1 --2- 172.21.15.121:0/480107014 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f97fc05a510 0x7f97fc05c9d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f9814006fd0 tx=0x7f9814006f40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:08.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.371+0000 7f980d7fa700 1 -- 172.21.15.121:0/480107014 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f98180c2a00 con 0x7f98180c5c70 2023-11-15T03:02:08.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:08.571+0000 7f981e107700 1 -- 172.21.15.121:0/480107014 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f98180c2a00 con 0x7f98180c5c70 2023-11-15T03:02:09.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.151+0000 7f980d7fa700 1 -- 172.21.15.121:0/480107014 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7f98180c2a00 con 0x7f98180c5c70 2023-11-15T03:02:09.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.155+0000 7f981e107700 1 -- 172.21.15.121:0/480107014 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f97fc05a510 msgr2=0x7f97fc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:09.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.155+0000 7f981e107700 1 --2- 172.21.15.121:0/480107014 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f97fc05a510 0x7f97fc05c9d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f9814006fd0 tx=0x7f9814006f40).stop 2023-11-15T03:02:09.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.155+0000 7f981e107700 1 -- 172.21.15.121:0/480107014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98180c5c70 msgr2=0x7f9818153490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:09.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.155+0000 7f981e107700 1 --2- 172.21.15.121:0/480107014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98180c5c70 0x7f9818153490 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f9810009e90 tx=0x7f981000b040).stop 2023-11-15T03:02:09.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.155+0000 7f981e107700 1 -- 172.21.15.121:0/480107014 shutdown_connections 2023-11-15T03:02:09.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.155+0000 7f981e107700 1 --2- 172.21.15.121:0/480107014 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f97fc05a510 0x7f97fc05c9d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:09.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.155+0000 7f981e107700 1 --2- 172.21.15.121:0/480107014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98180c5c70 0x7f9818153490 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:09.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.155+0000 7f981e107700 1 --2- 172.21.15.121:0/480107014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98180acd00 0x7f98181670d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:09.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.155+0000 7f981e107700 1 --2- 172.21.15.121:0/480107014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9818066330 0x7f9818165460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:09.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.155+0000 7f981e107700 1 -- 172.21.15.121:0/480107014 >> 172.21.15.121:0/480107014 conn(0x7f9818063fc0 msgr2=0x7f9818152a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:09.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.155+0000 7f981e107700 1 -- 172.21.15.121:0/480107014 shutdown_connections 2023-11-15T03:02:09.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.155+0000 7f981e107700 1 -- 172.21.15.121:0/480107014 wait complete. 2023-11-15T03:02:09.184 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-15T03:02:09.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.395+0000 7fa674e39700 1 Processor -- start 2023-11-15T03:02:09.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.395+0000 7fa674e39700 1 -- start start 2023-11-15T03:02:09.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa674e39700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa6700a8880 0x7fa670151b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:09.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa674e39700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6700c2c00 con 0x7fa6700abaa0 2023-11-15T03:02:09.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa674e39700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6700657b0 con 0x7fa6700bb510 2023-11-15T03:02:09.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa674e39700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa670065a90 con 0x7fa6700a8880 2023-11-15T03:02:09.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa66ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa6700a8880 0x7fa670151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:09.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa66f7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa6700abaa0 0x7fa670063570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34416/0 (socket says 172.21.15.121:34416) 2023-11-15T03:02:09.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa66f7fe700 1 -- 172.21.15.121:0/1454339608 learned_addr learned my addr 172.21.15.121:0/1454339608 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:09.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa66e7fc700 1 -- 172.21.15.121:0/1454339608 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4195287645 0 0) 0x7fa6700c2c00 con 0x7fa6700abaa0 2023-11-15T03:02:09.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa66e7fc700 1 -- 172.21.15.121:0/1454339608 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6580033f0 con 0x7fa6700abaa0 2023-11-15T03:02:09.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa66ffff700 1 -- 172.21.15.121:0/1454339608 >> v1:172.21.15.190:6789/0 conn(0x7fa6700bb510 legacy=0x7fa670151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:09.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa66ffff700 1 -- 172.21.15.121:0/1454339608 >> v1:172.21.15.121:6789/0 conn(0x7fa6700abaa0 legacy=0x7fa670063570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:09.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa66ffff700 1 -- 172.21.15.121:0/1454339608 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6700c2890 con 0x7fa6700a8880 2023-11-15T03:02:09.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa66ffff700 1 --2- 172.21.15.121:0/1454339608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6700a8880 0x7fa670151b10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7fa668009e60 tx=0x7fa668009780).ready entity=mon.0 client_cookie=9f696564035c2b9d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:09.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa66e7fc700 1 -- 172.21.15.121:0/1454339608 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66800e070 con 0x7fa6700a8880 2023-11-15T03:02:09.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa66e7fc700 1 -- 172.21.15.121:0/1454339608 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66800bb60 con 0x7fa6700a8880 2023-11-15T03:02:09.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa66e7fc700 1 -- 172.21.15.121:0/1454339608 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668012410 con 0x7fa6700a8880 2023-11-15T03:02:09.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa674e39700 1 -- 172.21.15.121:0/1454339608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6700a8880 msgr2=0x7fa670151b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:09.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa674e39700 1 --2- 172.21.15.121:0/1454339608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6700a8880 0x7fa670151b10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7fa668009e60 tx=0x7fa668009780).stop 2023-11-15T03:02:09.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa674e39700 1 -- 172.21.15.121:0/1454339608 shutdown_connections 2023-11-15T03:02:09.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa674e39700 1 --2- 172.21.15.121:0/1454339608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6700a8880 0x7fa670151b10 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:09.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa674e39700 1 -- 172.21.15.121:0/1454339608 >> 172.21.15.121:0/1454339608 conn(0x7fa670067a60 msgr2=0x7fa670067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:09.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa674e39700 1 -- 172.21.15.121:0/1454339608 shutdown_connections 2023-11-15T03:02:09.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.399+0000 7fa674e39700 1 -- 172.21.15.121:0/1454339608 wait complete. 2023-11-15T03:02:09.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.403+0000 7fa674e39700 1 Processor -- start 2023-11-15T03:02:09.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.403+0000 7fa674e39700 1 -- start start 2023-11-15T03:02:09.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.403+0000 7fa674e39700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6700a8880 0x7fa67014aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:09.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.403+0000 7fa674e39700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6700abaa0 0x7fa6701443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:09.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.403+0000 7fa674e39700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6700bb510 0x7fa670156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:09.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.403+0000 7fa674e39700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6701448c0 con 0x7fa6700a8880 2023-11-15T03:02:09.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.403+0000 7fa674e39700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700a98d0 con 0x7fa6700bb510 2023-11-15T03:02:09.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.403+0000 7fa674e39700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700a9bb0 con 0x7fa6700abaa0 2023-11-15T03:02:09.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.407+0000 7fa66f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6700a8880 0x7fa67014aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:09.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.407+0000 7fa66f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6700a8880 0x7fa67014aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53170/0 (socket says 172.21.15.121:53170) 2023-11-15T03:02:09.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.407+0000 7fa66f7fe700 1 -- 172.21.15.121:0/1133409887 learned_addr learned my addr 172.21.15.121:0/1133409887 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:09.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.407+0000 7fa66ffff700 1 --2- 172.21.15.121:0/1133409887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6700bb510 0x7fa670156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:09.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.407+0000 7fa66effd700 1 --2- 172.21.15.121:0/1133409887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6700abaa0 0x7fa6701443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:09.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.407+0000 7fa66f7fe700 1 -- 172.21.15.121:0/1133409887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6700abaa0 msgr2=0x7fa6701443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:09.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.407+0000 7fa66f7fe700 1 --2- 172.21.15.121:0/1133409887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6700abaa0 0x7fa6701443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:09.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.407+0000 7fa66f7fe700 1 -- 172.21.15.121:0/1133409887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6700bb510 msgr2=0x7fa670156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:09.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.407+0000 7fa66f7fe700 1 --2- 172.21.15.121:0/1133409887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6700bb510 0x7fa670156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:09.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.407+0000 7fa66f7fe700 1 -- 172.21.15.121:0/1133409887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa668009a20 con 0x7fa6700a8880 2023-11-15T03:02:09.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.407+0000 7fa66f7fe700 1 --2- 172.21.15.121:0/1133409887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6700a8880 0x7fa67014aee0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7fa66000c590 tx=0x7fa660007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:09.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.407+0000 7fa6577fe700 1 -- 172.21.15.121:0/1133409887 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6600072e0 con 0x7fa6700a8880 2023-11-15T03:02:09.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.407+0000 7fa6577fe700 1 -- 172.21.15.121:0/1133409887 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa660007440 con 0x7fa6700a8880 2023-11-15T03:02:09.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.407+0000 7fa6577fe700 1 -- 172.21.15.121:0/1133409887 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6600057a0 con 0x7fa6700a8880 2023-11-15T03:02:09.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.407+0000 7fa674e39700 1 -- 172.21.15.121:0/1133409887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6700a9e90 con 0x7fa6700a8880 2023-11-15T03:02:09.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.407+0000 7fa674e39700 1 -- 172.21.15.121:0/1133409887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6700aa450 con 0x7fa6700a8880 2023-11-15T03:02:09.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.407+0000 7fa674e39700 1 -- 172.21.15.121:0/1133409887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67006bcf0 con 0x7fa6700a8880 2023-11-15T03:02:09.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.411+0000 7fa6577fe700 1 -- 172.21.15.121:0/1133409887 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa660005bd0 con 0x7fa6700a8880 2023-11-15T03:02:09.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.411+0000 7fa6577fe700 1 --2- 172.21.15.121:0/1133409887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa65c05a510 0x7fa65c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:09.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.411+0000 7fa6577fe700 1 -- 172.21.15.121:0/1133409887 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6466+0+0 (secure 0 0 0) 0x7fa660074220 con 0x7fa6700a8880 2023-11-15T03:02:09.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.411+0000 7fa66effd700 1 --2- 172.21.15.121:0/1133409887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa65c05a510 0x7fa65c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:09.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.411+0000 7fa66effd700 1 --2- 172.21.15.121:0/1133409887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa65c05a510 0x7fa65c05c9d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fa670167270 tx=0x7fa664009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:09.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.419+0000 7fa6577fe700 1 -- 172.21.15.121:0/1133409887 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa6600477e0 con 0x7fa6700a8880 2023-11-15T03:02:09.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:09.579+0000 7fa674e39700 1 -- 172.21.15.121:0/1133409887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa670068a70 con 0x7fa6700a8880 2023-11-15T03:02:10.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.159+0000 7fa6577fe700 1 -- 172.21.15.121:0/1133409887 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v67) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa66004ae00 con 0x7fa6700a8880 2023-11-15T03:02:10.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.167+0000 7fa674e39700 1 -- 172.21.15.121:0/1133409887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa65c05a510 msgr2=0x7fa65c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:10.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.167+0000 7fa674e39700 1 --2- 172.21.15.121:0/1133409887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa65c05a510 0x7fa65c05c9d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fa670167270 tx=0x7fa664009040).stop 2023-11-15T03:02:10.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.167+0000 7fa674e39700 1 -- 172.21.15.121:0/1133409887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6700a8880 msgr2=0x7fa67014aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:10.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.167+0000 7fa674e39700 1 --2- 172.21.15.121:0/1133409887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6700a8880 0x7fa67014aee0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7fa66000c590 tx=0x7fa660007c80).stop 2023-11-15T03:02:10.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.167+0000 7fa674e39700 1 -- 172.21.15.121:0/1133409887 shutdown_connections 2023-11-15T03:02:10.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.167+0000 7fa674e39700 1 --2- 172.21.15.121:0/1133409887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa65c05a510 0x7fa65c05c9d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:10.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.167+0000 7fa674e39700 1 --2- 172.21.15.121:0/1133409887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6700bb510 0x7fa670156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:10.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.167+0000 7fa674e39700 1 --2- 172.21.15.121:0/1133409887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6700abaa0 0x7fa6701443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:10.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.167+0000 7fa674e39700 1 --2- 172.21.15.121:0/1133409887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6700a8880 0x7fa67014aee0 secure :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 rx=0x7fa66000c590 tx=0x7fa660007c80).stop 2023-11-15T03:02:10.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.167+0000 7fa674e39700 1 -- 172.21.15.121:0/1133409887 >> 172.21.15.121:0/1133409887 conn(0x7fa670067a60 msgr2=0x7fa670147130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:10.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.167+0000 7fa674e39700 1 -- 172.21.15.121:0/1133409887 shutdown_connections 2023-11-15T03:02:10.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.167+0000 7fa674e39700 1 -- 172.21.15.121:0/1133409887 wait complete. 2023-11-15T03:02:10.177 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_metadata' removed 2023-11-15T03:02:10.196 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-15T03:02:10.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.347+0000 7f6c330b9700 1 Processor -- start 2023-11-15T03:02:10.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.347+0000 7f6c330b9700 1 -- start start 2023-11-15T03:02:10.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.351+0000 7f6c330b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c2c066570 0x7f6c2c1515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:10.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.351+0000 7f6c330b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6c2c151ac0 0x7f6c2c0a8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:10.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.351+0000 7f6c330b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2c0660e0 con 0x7f6c2c1511f0 2023-11-15T03:02:10.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.351+0000 7f6c330b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2c0669f0 con 0x7f6c2c151ac0 2023-11-15T03:02:10.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.351+0000 7f6c330b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0bd390 con 0x7f6c2c066570 2023-11-15T03:02:10.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.351+0000 7f6c318b6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c2c066570 0x7f6c2c1515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:10.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.351+0000 7f6c318b6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c2c066570 0x7f6c2c1515d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:52514/0 (socket says 172.21.15.121:52514) 2023-11-15T03:02:10.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.351+0000 7f6c318b6700 1 -- 172.21.15.121:0/1686009230 learned_addr learned my addr 172.21.15.121:0/1686009230 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:10.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.355+0000 7f6c328b8700 1 --2- 172.21.15.121:0/1686009230 >> v2:172.21.15.190:3300/0 conn(0x7f6c2c151ac0 0x7f6c2c0a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:10.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.355+0000 7f6c318b6700 1 -- 172.21.15.121:0/1686009230 >> v2:172.21.15.190:3300/0 conn(0x7f6c2c151ac0 msgr2=0x7f6c2c0a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:10.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.355+0000 7f6c318b6700 1 --2- 172.21.15.121:0/1686009230 >> v2:172.21.15.190:3300/0 conn(0x7f6c2c151ac0 0x7f6c2c0a8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:10.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.355+0000 7f6c318b6700 1 -- 172.21.15.121:0/1686009230 >> v1:172.21.15.190:6789/0 conn(0x7f6c2c1511f0 legacy=0x7f6c2c0bcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:10.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.355+0000 7f6c318b6700 1 -- 172.21.15.121:0/1686009230 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c0657b0 con 0x7f6c2c066570 2023-11-15T03:02:10.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.355+0000 7f6c318b6700 1 --2- 172.21.15.121:0/1686009230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c2c066570 0x7f6c2c1515d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f6c1c00bab0 tx=0x7f6c1c00b300).ready entity=mon.2 client_cookie=acdd0f3596b18478 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:10.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.355+0000 7f6c310b5700 1 -- 172.21.15.121:0/1686009230 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c00da70 con 0x7f6c2c066570 2023-11-15T03:02:10.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.355+0000 7f6c310b5700 1 -- 172.21.15.121:0/1686009230 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c00dbd0 con 0x7f6c2c066570 2023-11-15T03:02:10.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.355+0000 7f6c310b5700 1 -- 172.21.15.121:0/1686009230 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c005880 con 0x7f6c2c066570 2023-11-15T03:02:10.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.355+0000 7f6c330b9700 1 -- 172.21.15.121:0/1686009230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c2c066570 msgr2=0x7f6c2c1515d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:10.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.355+0000 7f6c330b9700 1 --2- 172.21.15.121:0/1686009230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c2c066570 0x7f6c2c1515d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f6c1c00bab0 tx=0x7f6c1c00b300).stop 2023-11-15T03:02:10.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.355+0000 7f6c330b9700 1 -- 172.21.15.121:0/1686009230 shutdown_connections 2023-11-15T03:02:10.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.355+0000 7f6c330b9700 1 --2- 172.21.15.121:0/1686009230 >> v2:172.21.15.190:3300/0 conn(0x7f6c2c151ac0 0x7f6c2c0a8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:10.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.355+0000 7f6c330b9700 1 --2- 172.21.15.121:0/1686009230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c2c066570 0x7f6c2c1515d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:10.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.355+0000 7f6c330b9700 1 -- 172.21.15.121:0/1686009230 >> 172.21.15.121:0/1686009230 conn(0x7f6c2c067a60 msgr2=0x7f6c2c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:10.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.355+0000 7f6c330b9700 1 -- 172.21.15.121:0/1686009230 shutdown_connections 2023-11-15T03:02:10.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.355+0000 7f6c330b9700 1 -- 172.21.15.121:0/1686009230 wait complete. 2023-11-15T03:02:10.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.359+0000 7f6c330b9700 1 Processor -- start 2023-11-15T03:02:10.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.359+0000 7f6c330b9700 1 -- start start 2023-11-15T03:02:10.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.359+0000 7f6c330b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c2c066570 0x7f6c2c0bfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:10.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.359+0000 7f6c330b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c2c1511f0 0x7f6c2c149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:10.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.359+0000 7f6c330b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c2c151ac0 0x7f6c2c1676e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:10.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.359+0000 7f6c330b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c152a10 con 0x7f6c2c1511f0 2023-11-15T03:02:10.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.359+0000 7f6c330b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c153590 con 0x7f6c2c151ac0 2023-11-15T03:02:10.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.359+0000 7f6c330b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c152c60 con 0x7f6c2c066570 2023-11-15T03:02:10.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.359+0000 7f6c318b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c2c1511f0 0x7f6c2c149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:10.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.359+0000 7f6c318b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c2c1511f0 0x7f6c2c149030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53176/0 (socket says 172.21.15.121:53176) 2023-11-15T03:02:10.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.359+0000 7f6c318b6700 1 -- 172.21.15.121:0/1476531902 learned_addr learned my addr 172.21.15.121:0/1476531902 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:10.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.359+0000 7f6c320b7700 1 --2- 172.21.15.121:0/1476531902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c2c066570 0x7f6c2c0bfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:10.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.359+0000 7f6c328b8700 1 --2- 172.21.15.121:0/1476531902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c2c151ac0 0x7f6c2c1676e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:10.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.363+0000 7f6c318b6700 1 -- 172.21.15.121:0/1476531902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c2c066570 msgr2=0x7f6c2c0bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:10.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.363+0000 7f6c318b6700 1 --2- 172.21.15.121:0/1476531902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c2c066570 0x7f6c2c0bfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:10.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.363+0000 7f6c318b6700 1 -- 172.21.15.121:0/1476531902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c2c151ac0 msgr2=0x7f6c2c1676e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:10.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.363+0000 7f6c318b6700 1 --2- 172.21.15.121:0/1476531902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c2c151ac0 0x7f6c2c1676e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:10.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.363+0000 7f6c318b6700 1 -- 172.21.15.121:0/1476531902 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c00b5a0 con 0x7f6c2c1511f0 2023-11-15T03:02:10.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.363+0000 7f6c318b6700 1 --2- 172.21.15.121:0/1476531902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c2c1511f0 0x7f6c2c149030 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f6c1c00b560 tx=0x7f6c1c00b300).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:10.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.363+0000 7f6c227fc700 1 -- 172.21.15.121:0/1476531902 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c01d070 con 0x7f6c2c1511f0 2023-11-15T03:02:10.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.363+0000 7f6c227fc700 1 -- 172.21.15.121:0/1476531902 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c0069e0 con 0x7f6c2c1511f0 2023-11-15T03:02:10.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.363+0000 7f6c330b9700 1 -- 172.21.15.121:0/1476531902 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2c152f40 con 0x7f6c2c1511f0 2023-11-15T03:02:10.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.363+0000 7f6c227fc700 1 -- 172.21.15.121:0/1476531902 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c005880 con 0x7f6c2c1511f0 2023-11-15T03:02:10.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.363+0000 7f6c330b9700 1 -- 172.21.15.121:0/1476531902 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2c0c0210 con 0x7f6c2c1511f0 2023-11-15T03:02:10.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.363+0000 7f6c330b9700 1 -- 172.21.15.121:0/1476531902 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2c06bcf0 con 0x7f6c2c1511f0 2023-11-15T03:02:10.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.367+0000 7f6c227fc700 1 -- 172.21.15.121:0/1476531902 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6c1c010680 con 0x7f6c2c1511f0 2023-11-15T03:02:10.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.367+0000 7f6c227fc700 1 --2- 172.21.15.121:0/1476531902 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6c1005a510 0x7f6c1005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:10.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.367+0000 7f6c227fc700 1 -- 172.21.15.121:0/1476531902 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6028+0+0 (secure 0 0 0) 0x7f6c1c075aa0 con 0x7f6c2c1511f0 2023-11-15T03:02:10.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.367+0000 7f6c320b7700 1 --2- 172.21.15.121:0/1476531902 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6c1005a510 0x7f6c1005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:10.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.367+0000 7f6c320b7700 1 --2- 172.21.15.121:0/1476531902 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6c1005a510 0x7f6c1005c9d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f6c28003870 tx=0x7f6c28007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:10.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.371+0000 7f6c227fc700 1 -- 172.21.15.121:0/1476531902 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6c1c0491c0 con 0x7f6c2c1511f0 2023-11-15T03:02:10.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:10.535+0000 7f6c330b9700 1 -- 172.21.15.121:0/1476531902 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6c2c067e40 con 0x7f6c2c1511f0 2023-11-15T03:02:10.929 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:02:10.937 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:02:11.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.171+0000 7f6c227fc700 1 -- 172.21.15.121:0/1476531902 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v68) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6c1c027030 con 0x7f6c2c1511f0 2023-11-15T03:02:11.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.179+0000 7f6c330b9700 1 -- 172.21.15.121:0/1476531902 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6c1005a510 msgr2=0x7f6c1005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:11.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.179+0000 7f6c330b9700 1 --2- 172.21.15.121:0/1476531902 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6c1005a510 0x7f6c1005c9d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f6c28003870 tx=0x7f6c28007d40).stop 2023-11-15T03:02:11.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.179+0000 7f6c330b9700 1 -- 172.21.15.121:0/1476531902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c2c1511f0 msgr2=0x7f6c2c149030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:11.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.179+0000 7f6c330b9700 1 --2- 172.21.15.121:0/1476531902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c2c1511f0 0x7f6c2c149030 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f6c1c00b560 tx=0x7f6c1c00b300).stop 2023-11-15T03:02:11.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.179+0000 7f6c330b9700 1 -- 172.21.15.121:0/1476531902 shutdown_connections 2023-11-15T03:02:11.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.179+0000 7f6c330b9700 1 --2- 172.21.15.121:0/1476531902 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6c1005a510 0x7f6c1005c9d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.179+0000 7f6c330b9700 1 --2- 172.21.15.121:0/1476531902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c2c151ac0 0x7f6c2c1676e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.179+0000 7f6c330b9700 1 --2- 172.21.15.121:0/1476531902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c2c1511f0 0x7f6c2c149030 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.179+0000 7f6c330b9700 1 --2- 172.21.15.121:0/1476531902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c2c066570 0x7f6c2c0bfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.179+0000 7f6c330b9700 1 -- 172.21.15.121:0/1476531902 >> 172.21.15.121:0/1476531902 conn(0x7f6c2c067a60 msgr2=0x7f6c2c0c1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:11.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.179+0000 7f6c330b9700 1 -- 172.21.15.121:0/1476531902 shutdown_connections 2023-11-15T03:02:11.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.179+0000 7f6c330b9700 1 -- 172.21.15.121:0/1476531902 wait complete. 2023-11-15T03:02:11.194 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_data' removed 2023-11-15T03:02:11.207 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-15T03:02:11.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.375+0000 7fb726707700 1 Processor -- start 2023-11-15T03:02:11.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.375+0000 7fb726707700 1 -- start start 2023-11-15T03:02:11.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.379+0000 7fb726707700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7200acd00 0x7fb7200ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:11.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.379+0000 7fb726707700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb7200c5c70 0x7fb7201536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:11.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.379+0000 7fb726707700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb720065ea0 con 0x7fb720066330 2023-11-15T03:02:11.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.379+0000 7fb726707700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb720066bb0 con 0x7fb7200c5c70 2023-11-15T03:02:11.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.379+0000 7fb726707700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb720065890 con 0x7fb7200acd00 2023-11-15T03:02:11.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.379+0000 7fb724f04700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb7200c5c70 0x7fb7201536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:11.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.379+0000 7fb725705700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7200acd00 0x7fb7200ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:11.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.379+0000 7fb725f06700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fb720066330 0x7fb720153bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:34566/0 (socket says 172.21.15.121:34566) 2023-11-15T03:02:11.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.379+0000 7fb725f06700 1 -- 172.21.15.121:0/3336566463 learned_addr learned my addr 172.21.15.121:0/3336566463 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:11.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.379+0000 7fb724f04700 1 -- 172.21.15.121:0/3336566463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7200acd00 msgr2=0x7fb7200ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:11.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.379+0000 7fb724f04700 1 --2- 172.21.15.121:0/3336566463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7200acd00 0x7fb7200ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.379+0000 7fb724f04700 1 -- 172.21.15.121:0/3336566463 >> v1:172.21.15.190:6789/0 conn(0x7fb720066330 legacy=0x7fb720153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:11.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.379+0000 7fb724f04700 1 -- 172.21.15.121:0/3336566463 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7200c64a0 con 0x7fb7200c5c70 2023-11-15T03:02:11.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.379+0000 7fb724f04700 1 --2- 172.21.15.121:0/3336566463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7200c5c70 0x7fb7201536c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fb710009760 tx=0x7fb710009060).ready entity=mon.1 client_cookie=ba7b70499226de38 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:11.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.379+0000 7fb717fff700 1 -- 172.21.15.121:0/3336566463 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71000f510 con 0x7fb7200c5c70 2023-11-15T03:02:11.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.379+0000 7fb717fff700 1 -- 172.21.15.121:0/3336566463 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71000f670 con 0x7fb7200c5c70 2023-11-15T03:02:11.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.383+0000 7fb717fff700 1 -- 172.21.15.121:0/3336566463 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71000f9f0 con 0x7fb7200c5c70 2023-11-15T03:02:11.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.383+0000 7fb726707700 1 -- 172.21.15.121:0/3336566463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7200c5c70 msgr2=0x7fb7201536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:11.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.383+0000 7fb726707700 1 --2- 172.21.15.121:0/3336566463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7200c5c70 0x7fb7201536c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fb710009760 tx=0x7fb710009060).stop 2023-11-15T03:02:11.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.383+0000 7fb726707700 1 -- 172.21.15.121:0/3336566463 shutdown_connections 2023-11-15T03:02:11.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.383+0000 7fb726707700 1 --2- 172.21.15.121:0/3336566463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7200c5c70 0x7fb7201536c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.383+0000 7fb726707700 1 --2- 172.21.15.121:0/3336566463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7200acd00 0x7fb7200ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.383+0000 7fb726707700 1 -- 172.21.15.121:0/3336566463 >> 172.21.15.121:0/3336566463 conn(0x7fb720063fc0 msgr2=0x7fb720067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:11.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.383+0000 7fb726707700 1 -- 172.21.15.121:0/3336566463 shutdown_connections 2023-11-15T03:02:11.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.383+0000 7fb726707700 1 -- 172.21.15.121:0/3336566463 wait complete. 2023-11-15T03:02:11.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.387+0000 7fb726707700 1 Processor -- start 2023-11-15T03:02:11.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.387+0000 7fb726707700 1 -- start start 2023-11-15T03:02:11.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.387+0000 7fb726707700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb720066330 0x7fb72014fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:11.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.387+0000 7fb726707700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7200acd00 0x7fb7200ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:11.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.387+0000 7fb726707700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7200c5c70 0x7fb720141230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:11.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.387+0000 7fb726707700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7201688c0 con 0x7fb7200c5c70 2023-11-15T03:02:11.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.387+0000 7fb726707700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb720167cb0 con 0x7fb7200acd00 2023-11-15T03:02:11.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.387+0000 7fb726707700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb720167f90 con 0x7fb720066330 2023-11-15T03:02:11.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.387+0000 7fb725f06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7200c5c70 0x7fb720141230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:11.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.387+0000 7fb725f06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7200c5c70 0x7fb720141230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53178/0 (socket says 172.21.15.121:53178) 2023-11-15T03:02:11.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.387+0000 7fb725f06700 1 -- 172.21.15.121:0/2171149254 learned_addr learned my addr 172.21.15.121:0/2171149254 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:11.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.387+0000 7fb724f04700 1 --2- 172.21.15.121:0/2171149254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7200acd00 0x7fb7200ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:11.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.387+0000 7fb725705700 1 --2- 172.21.15.121:0/2171149254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb720066330 0x7fb72014fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:11.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.391+0000 7fb725f06700 1 -- 172.21.15.121:0/2171149254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb720066330 msgr2=0x7fb72014fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:11.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.391+0000 7fb725f06700 1 --2- 172.21.15.121:0/2171149254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb720066330 0x7fb72014fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.391+0000 7fb725f06700 1 -- 172.21.15.121:0/2171149254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7200acd00 msgr2=0x7fb7200ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:11.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.391+0000 7fb725f06700 1 --2- 172.21.15.121:0/2171149254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7200acd00 0x7fb7200ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.391+0000 7fb725f06700 1 -- 172.21.15.121:0/2171149254 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7100092a0 con 0x7fb7200c5c70 2023-11-15T03:02:11.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.391+0000 7fb725f06700 1 --2- 172.21.15.121:0/2171149254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7200c5c70 0x7fb720141230 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7fb71800c790 tx=0x7fb7180015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:11.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.391+0000 7fb7157fa700 1 -- 172.21.15.121:0/2171149254 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718005d30 con 0x7fb7200c5c70 2023-11-15T03:02:11.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.391+0000 7fb7157fa700 1 -- 172.21.15.121:0/2171149254 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb718005e90 con 0x7fb7200c5c70 2023-11-15T03:02:11.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.391+0000 7fb726707700 1 -- 172.21.15.121:0/2171149254 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb720168270 con 0x7fb7200c5c70 2023-11-15T03:02:11.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.391+0000 7fb7157fa700 1 -- 172.21.15.121:0/2171149254 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718007320 con 0x7fb7200c5c70 2023-11-15T03:02:11.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.391+0000 7fb726707700 1 -- 172.21.15.121:0/2171149254 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb720150060 con 0x7fb7200c5c70 2023-11-15T03:02:11.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.391+0000 7fb726707700 1 -- 172.21.15.121:0/2171149254 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb720168a30 con 0x7fb7200c5c70 2023-11-15T03:02:11.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.395+0000 7fb7157fa700 1 -- 172.21.15.121:0/2171149254 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb7180076d0 con 0x7fb7200c5c70 2023-11-15T03:02:11.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.395+0000 7fb7157fa700 1 --2- 172.21.15.121:0/2171149254 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb70405a510 0x7fb70405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:11.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.395+0000 7fb7157fa700 1 -- 172.21.15.121:0/2171149254 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5527+0+0 (secure 0 0 0) 0x7fb718074420 con 0x7fb7200c5c70 2023-11-15T03:02:11.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.395+0000 7fb725705700 1 --2- 172.21.15.121:0/2171149254 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb70405a510 0x7fb70405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:11.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.399+0000 7fb725705700 1 --2- 172.21.15.121:0/2171149254 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb70405a510 0x7fb70405c9d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fb71c006fd0 tx=0x7fb71c006f40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:11.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.403+0000 7fb7157fa700 1 -- 172.21.15.121:0/2171149254 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb718047e50 con 0x7fb7200c5c70 2023-11-15T03:02:11.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.579+0000 7fb726707700 1 -- 172.21.15.121:0/2171149254 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb72006bcf0 con 0x7fb7200c5c70 2023-11-15T03:02:11.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.587+0000 7fb7157fa700 1 -- 172.21.15.121:0/2171149254 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb71804b470 con 0x7fb7200c5c70 2023-11-15T03:02:11.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.591+0000 7fb726707700 1 -- 172.21.15.121:0/2171149254 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb70405a510 msgr2=0x7fb70405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:11.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.591+0000 7fb726707700 1 --2- 172.21.15.121:0/2171149254 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb70405a510 0x7fb70405c9d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fb71c006fd0 tx=0x7fb71c006f40).stop 2023-11-15T03:02:11.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.591+0000 7fb726707700 1 -- 172.21.15.121:0/2171149254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7200c5c70 msgr2=0x7fb720141230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:11.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.591+0000 7fb726707700 1 --2- 172.21.15.121:0/2171149254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7200c5c70 0x7fb720141230 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7fb71800c790 tx=0x7fb7180015e0).stop 2023-11-15T03:02:11.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.591+0000 7fb726707700 1 -- 172.21.15.121:0/2171149254 shutdown_connections 2023-11-15T03:02:11.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.591+0000 7fb726707700 1 --2- 172.21.15.121:0/2171149254 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb70405a510 0x7fb70405c9d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.591+0000 7fb726707700 1 --2- 172.21.15.121:0/2171149254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7200c5c70 0x7fb720141230 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.591+0000 7fb726707700 1 --2- 172.21.15.121:0/2171149254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7200acd00 0x7fb7200ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.591+0000 7fb726707700 1 --2- 172.21.15.121:0/2171149254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb720066330 0x7fb72014fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.591+0000 7fb726707700 1 -- 172.21.15.121:0/2171149254 >> 172.21.15.121:0/2171149254 conn(0x7fb720063fc0 msgr2=0x7fb7200c0720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:11.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.591+0000 7fb726707700 1 -- 172.21.15.121:0/2171149254 shutdown_connections 2023-11-15T03:02:11.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.591+0000 7fb726707700 1 -- 172.21.15.121:0/2171149254 wait complete. 2023-11-15T03:02:11.616 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_journal_corrupt_dentry_first 2023-11-15T03:02:11.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.783+0000 7f2c2b5a9700 1 Processor -- start 2023-11-15T03:02:11.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.783+0000 7f2c2b5a9700 1 -- start start 2023-11-15T03:02:11.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.787+0000 7f2c2b5a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c24068170 0x7f2c241582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:11.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.787+0000 7f2c2b5a9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c24062da0 con 0x7f2c24157ed0 2023-11-15T03:02:11.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.787+0000 7f2c2b5a9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c24062190 con 0x7f2c24067020 2023-11-15T03:02:11.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.787+0000 7f2c2b5a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c240a93a0 con 0x7f2c24068170 2023-11-15T03:02:11.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.787+0000 7f2c29345700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f2c24157ed0 0x7f2c24067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34428/0 (socket says 172.21.15.121:34428) 2023-11-15T03:02:11.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.787+0000 7f2c29345700 1 -- 172.21.15.121:0/4035800857 learned_addr learned my addr 172.21.15.121:0/4035800857 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:11.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.787+0000 7f2c29b46700 1 --2- 172.21.15.121:0/4035800857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c24068170 0x7f2c241582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:11.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.787+0000 7f2c1bfff700 1 -- 172.21.15.121:0/4035800857 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623997771 0 0) 0x7f2c24062da0 con 0x7f2c24157ed0 2023-11-15T03:02:11.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.787+0000 7f2c1bfff700 1 -- 172.21.15.121:0/4035800857 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c100033f0 con 0x7f2c24157ed0 2023-11-15T03:02:11.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.787+0000 7f2c29b46700 1 -- 172.21.15.121:0/4035800857 >> v1:172.21.15.190:6789/0 conn(0x7f2c24067020 legacy=0x7f2c240bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:11.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.787+0000 7f2c29b46700 1 -- 172.21.15.121:0/4035800857 >> v1:172.21.15.121:6789/0 conn(0x7f2c24157ed0 legacy=0x7f2c24067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:11.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.787+0000 7f2c29b46700 1 -- 172.21.15.121:0/4035800857 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c24062470 con 0x7f2c24068170 2023-11-15T03:02:11.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.787+0000 7f2c29b46700 1 --2- 172.21.15.121:0/4035800857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c24068170 0x7f2c241582b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f2c1c009e90 tx=0x7f2c1c009780).ready entity=mon.2 client_cookie=75183e80c0ffd4a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:11.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.787+0000 7f2c1bfff700 1 -- 172.21.15.121:0/4035800857 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c00e070 con 0x7f2c24068170 2023-11-15T03:02:11.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.787+0000 7f2c1bfff700 1 -- 172.21.15.121:0/4035800857 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1c00bb60 con 0x7f2c24068170 2023-11-15T03:02:11.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.787+0000 7f2c1bfff700 1 -- 172.21.15.121:0/4035800857 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c012410 con 0x7f2c24068170 2023-11-15T03:02:11.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.791+0000 7f2c2b5a9700 1 -- 172.21.15.121:0/4035800857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c24068170 msgr2=0x7f2c241582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:11.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.791+0000 7f2c2b5a9700 1 --2- 172.21.15.121:0/4035800857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c24068170 0x7f2c241582b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f2c1c009e90 tx=0x7f2c1c009780).stop 2023-11-15T03:02:11.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.791+0000 7f2c2b5a9700 1 -- 172.21.15.121:0/4035800857 shutdown_connections 2023-11-15T03:02:11.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.791+0000 7f2c2b5a9700 1 --2- 172.21.15.121:0/4035800857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c24068170 0x7f2c241582b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.791+0000 7f2c2b5a9700 1 -- 172.21.15.121:0/4035800857 >> 172.21.15.121:0/4035800857 conn(0x7f2c240591c0 msgr2=0x7f2c240c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:11.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.791+0000 7f2c2b5a9700 1 -- 172.21.15.121:0/4035800857 shutdown_connections 2023-11-15T03:02:11.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.791+0000 7f2c2b5a9700 1 -- 172.21.15.121:0/4035800857 wait complete. 2023-11-15T03:02:11.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.795+0000 7f2c2b5a9700 1 Processor -- start 2023-11-15T03:02:11.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.795+0000 7f2c2b5a9700 1 -- start start 2023-11-15T03:02:11.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.795+0000 7f2c2b5a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c24067020 0x7f2c240a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:11.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.795+0000 7f2c2b5a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c24068170 0x7f2c2416bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:11.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.795+0000 7f2c2b5a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c24157ed0 0x7f2c24149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:11.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.795+0000 7f2c2b5a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c24149c10 con 0x7f2c24067020 2023-11-15T03:02:11.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.795+0000 7f2c2b5a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c240a0d20 con 0x7f2c24157ed0 2023-11-15T03:02:11.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.795+0000 7f2c2b5a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c240a1000 con 0x7f2c24068170 2023-11-15T03:02:11.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.795+0000 7f2c29345700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c24067020 0x7f2c240a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:11.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.795+0000 7f2c29345700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c24067020 0x7f2c240a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53190/0 (socket says 172.21.15.121:53190) 2023-11-15T03:02:11.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.795+0000 7f2c29345700 1 -- 172.21.15.121:0/767221991 learned_addr learned my addr 172.21.15.121:0/767221991 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:11.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.795+0000 7f2c29b46700 1 --2- 172.21.15.121:0/767221991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c24157ed0 0x7f2c24149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:11.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.795+0000 7f2c28b44700 1 --2- 172.21.15.121:0/767221991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c24068170 0x7f2c2416bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:11.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.795+0000 7f2c29345700 1 -- 172.21.15.121:0/767221991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c24068170 msgr2=0x7f2c2416bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:11.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.795+0000 7f2c29345700 1 --2- 172.21.15.121:0/767221991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c24068170 0x7f2c2416bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.795+0000 7f2c29345700 1 -- 172.21.15.121:0/767221991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c24157ed0 msgr2=0x7f2c24149190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:11.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.795+0000 7f2c29345700 1 --2- 172.21.15.121:0/767221991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c24157ed0 0x7f2c24149190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.795+0000 7f2c29345700 1 -- 172.21.15.121:0/767221991 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1c009a20 con 0x7f2c24067020 2023-11-15T03:02:11.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.799+0000 7f2c29345700 1 --2- 172.21.15.121:0/767221991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c24067020 0x7f2c240a0740 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f2c2000c420 tx=0x7f2c20007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:11.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.799+0000 7f2c197fa700 1 -- 172.21.15.121:0/767221991 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c200072e0 con 0x7f2c24067020 2023-11-15T03:02:11.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.799+0000 7f2c197fa700 1 -- 172.21.15.121:0/767221991 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c20007440 con 0x7f2c24067020 2023-11-15T03:02:11.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.799+0000 7f2c197fa700 1 -- 172.21.15.121:0/767221991 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c200057a0 con 0x7f2c24067020 2023-11-15T03:02:11.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.799+0000 7f2c2b5a9700 1 -- 172.21.15.121:0/767221991 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c240a12e0 con 0x7f2c24067020 2023-11-15T03:02:11.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.799+0000 7f2c2b5a9700 1 -- 172.21.15.121:0/767221991 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c240a18a0 con 0x7f2c24067020 2023-11-15T03:02:11.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.799+0000 7f2c2b5a9700 1 -- 172.21.15.121:0/767221991 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c24047060 con 0x7f2c24067020 2023-11-15T03:02:11.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.799+0000 7f2c197fa700 1 -- 172.21.15.121:0/767221991 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2c20005bd0 con 0x7f2c24067020 2023-11-15T03:02:11.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.803+0000 7f2c197fa700 1 --2- 172.21.15.121:0/767221991 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c0805a510 0x7f2c0805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:11.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.803+0000 7f2c197fa700 1 -- 172.21.15.121:0/767221991 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5527+0+0 (secure 0 0 0) 0x7f2c20073d90 con 0x7f2c24067020 2023-11-15T03:02:11.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.803+0000 7f2c28b44700 1 --2- 172.21.15.121:0/767221991 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c0805a510 0x7f2c0805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:11.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.803+0000 7f2c28b44700 1 --2- 172.21.15.121:0/767221991 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c0805a510 0x7f2c0805c9d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f2c240c40a0 tx=0x7f2c14009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:11.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.807+0000 7f2c197fa700 1 -- 172.21.15.121:0/767221991 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2c200477c0 con 0x7f2c24067020 2023-11-15T03:02:11.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.975+0000 7f2c2b5a9700 1 -- 172.21.15.121:0/767221991 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"} v 0) v1 -- 0x7f2c24068b50 con 0x7f2c24067020 2023-11-15T03:02:11.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.975+0000 7f2c197fa700 1 -- 172.21.15.121:0/767221991 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"}]=0 v13) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2c2004ade0 con 0x7f2c24067020 2023-11-15T03:02:11.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.979+0000 7f2c2b5a9700 1 -- 172.21.15.121:0/767221991 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c0805a510 msgr2=0x7f2c0805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:11.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.979+0000 7f2c2b5a9700 1 --2- 172.21.15.121:0/767221991 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c0805a510 0x7f2c0805c9d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f2c240c40a0 tx=0x7f2c14009040).stop 2023-11-15T03:02:11.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.979+0000 7f2c2b5a9700 1 -- 172.21.15.121:0/767221991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c24067020 msgr2=0x7f2c240a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:11.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.979+0000 7f2c2b5a9700 1 --2- 172.21.15.121:0/767221991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c24067020 0x7f2c240a0740 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f2c2000c420 tx=0x7f2c20007c80).stop 2023-11-15T03:02:11.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.979+0000 7f2c2b5a9700 1 -- 172.21.15.121:0/767221991 shutdown_connections 2023-11-15T03:02:11.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.979+0000 7f2c2b5a9700 1 --2- 172.21.15.121:0/767221991 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c0805a510 0x7f2c0805c9d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.979+0000 7f2c2b5a9700 1 --2- 172.21.15.121:0/767221991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c24157ed0 0x7f2c24149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.979+0000 7f2c2b5a9700 1 --2- 172.21.15.121:0/767221991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c24068170 0x7f2c2416bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.979+0000 7f2c2b5a9700 1 --2- 172.21.15.121:0/767221991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c24067020 0x7f2c240a0740 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:11.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.979+0000 7f2c2b5a9700 1 -- 172.21.15.121:0/767221991 >> 172.21.15.121:0/767221991 conn(0x7f2c240591c0 msgr2=0x7f2c240abbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:11.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.979+0000 7f2c2b5a9700 1 -- 172.21.15.121:0/767221991 shutdown_connections 2023-11-15T03:02:11.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:11.979+0000 7f2c2b5a9700 1 -- 172.21.15.121:0/767221991 wait complete. 2023-11-15T03:02:12.004 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit' 2023-11-15T03:02:12.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.155+0000 7fac0e3f9700 1 Processor -- start 2023-11-15T03:02:12.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.155+0000 7fac0e3f9700 1 -- start start 2023-11-15T03:02:12.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.159+0000 7fac0e3f9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fac080c3280 0x7fac080c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:12.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.159+0000 7fac0e3f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac08152140 0x7fac081525c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:12.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.159+0000 7fac0e3f9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac080c1d50 con 0x7fac0815be30 2023-11-15T03:02:12.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.159+0000 7fac0e3f9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac080c1860 con 0x7fac080c3280 2023-11-15T03:02:12.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.159+0000 7fac0e3f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0815c990 con 0x7fac08152140 2023-11-15T03:02:12.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.159+0000 7fac0d3f7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fac080c3280 0x7fac080c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:12.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.159+0000 7fac0d3f7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fac080c3280 0x7fac080c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:52084/0 (socket says 172.21.15.121:52084) 2023-11-15T03:02:12.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.159+0000 7fac0d3f7700 1 -- 172.21.15.121:0/1178496893 learned_addr learned my addr 172.21.15.121:0/1178496893 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:12.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.159+0000 7fac0cbf6700 1 --2- 172.21.15.121:0/1178496893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac08152140 0x7fac081525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:12.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.159+0000 7fac0d3f7700 1 -- 172.21.15.121:0/1178496893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac08152140 msgr2=0x7fac081525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:12.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.159+0000 7fac0d3f7700 1 --2- 172.21.15.121:0/1178496893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac08152140 0x7fac081525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:12.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.159+0000 7fac0d3f7700 1 -- 172.21.15.121:0/1178496893 >> v1:172.21.15.121:6789/0 conn(0x7fac0815be30 legacy=0x7fac0815c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:12.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.159+0000 7fac0d3f7700 1 -- 172.21.15.121:0/1178496893 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0815cb80 con 0x7fac080c3280 2023-11-15T03:02:12.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.159+0000 7fabfffff700 1 -- 172.21.15.121:0/1178496893 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503526974 0 0) 0x7fac080c1d50 con 0x7fac0815be30 2023-11-15T03:02:12.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.159+0000 7fac0d3f7700 1 --2- 172.21.15.121:0/1178496893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac080c3280 0x7fac080c3660 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fac04006fd0 tx=0x7fac040090f0).ready entity=mon.1 client_cookie=e0a3225169bb16f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:12.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.159+0000 7fabfffff700 1 -- 172.21.15.121:0/1178496893 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0400b580 con 0x7fac080c3280 2023-11-15T03:02:12.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.159+0000 7fabfffff700 1 -- 172.21.15.121:0/1178496893 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0400b6e0 con 0x7fac080c3280 2023-11-15T03:02:12.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.163+0000 7fabfffff700 1 -- 172.21.15.121:0/1178496893 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0400ba60 con 0x7fac080c3280 2023-11-15T03:02:12.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.163+0000 7fac0e3f9700 1 -- 172.21.15.121:0/1178496893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac080c3280 msgr2=0x7fac080c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:12.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.163+0000 7fac0e3f9700 1 --2- 172.21.15.121:0/1178496893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac080c3280 0x7fac080c3660 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fac04006fd0 tx=0x7fac040090f0).stop 2023-11-15T03:02:12.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.163+0000 7fac0e3f9700 1 -- 172.21.15.121:0/1178496893 shutdown_connections 2023-11-15T03:02:12.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.163+0000 7fac0e3f9700 1 --2- 172.21.15.121:0/1178496893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac08152140 0x7fac081525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:12.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.163+0000 7fac0e3f9700 1 --2- 172.21.15.121:0/1178496893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac080c3280 0x7fac080c3660 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:12.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.163+0000 7fac0e3f9700 1 -- 172.21.15.121:0/1178496893 >> 172.21.15.121:0/1178496893 conn(0x7fac08062190 msgr2=0x7fac08062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:12.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.163+0000 7fac0e3f9700 1 -- 172.21.15.121:0/1178496893 shutdown_connections 2023-11-15T03:02:12.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.163+0000 7fac0e3f9700 1 -- 172.21.15.121:0/1178496893 wait complete. 2023-11-15T03:02:12.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.163+0000 7fac0e3f9700 1 Processor -- start 2023-11-15T03:02:12.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.163+0000 7fac0e3f9700 1 -- start start 2023-11-15T03:02:12.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fac0e3f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac080c3280 0x7fac08153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:12.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fac0e3f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac08152140 0x7fac08142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:12.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fac0e3f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac0815be30 0x7fac0816c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:12.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fac0e3f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac080c4c50 con 0x7fac0815be30 2023-11-15T03:02:12.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fac0e3f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac08158d20 con 0x7fac080c3280 2023-11-15T03:02:12.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fac0e3f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0816cf10 con 0x7fac08152140 2023-11-15T03:02:12.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fac0dbf8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac0815be30 0x7fac0816c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:12.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fac0dbf8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac0815be30 0x7fac0816c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53204/0 (socket says 172.21.15.121:53204) 2023-11-15T03:02:12.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fac0dbf8700 1 -- 172.21.15.121:0/2163543972 learned_addr learned my addr 172.21.15.121:0/2163543972 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:12.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fac0d3f7700 1 --2- 172.21.15.121:0/2163543972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac080c3280 0x7fac08153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:12.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fac0cbf6700 1 --2- 172.21.15.121:0/2163543972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac08152140 0x7fac08142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:12.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fac0dbf8700 1 -- 172.21.15.121:0/2163543972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac08152140 msgr2=0x7fac08142f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:12.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fac0dbf8700 1 --2- 172.21.15.121:0/2163543972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac08152140 0x7fac08142f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:12.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fac0dbf8700 1 -- 172.21.15.121:0/2163543972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac080c3280 msgr2=0x7fac08153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:12.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fac0dbf8700 1 --2- 172.21.15.121:0/2163543972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac080c3280 0x7fac08153600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:12.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fac0dbf8700 1 -- 172.21.15.121:0/2163543972 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac04009370 con 0x7fac0815be30 2023-11-15T03:02:12.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fac0dbf8700 1 --2- 172.21.15.121:0/2163543972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac0815be30 0x7fac0816c3a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7fac00002230 tx=0x7fac00007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:12.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fabfd7fa700 1 -- 172.21.15.121:0/2163543972 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00010510 con 0x7fac0815be30 2023-11-15T03:02:12.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fabfd7fa700 1 -- 172.21.15.121:0/2163543972 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac00010670 con 0x7fac0815be30 2023-11-15T03:02:12.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fac0e3f9700 1 -- 172.21.15.121:0/2163543972 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac0816d1f0 con 0x7fac0815be30 2023-11-15T03:02:12.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fabfd7fa700 1 -- 172.21.15.121:0/2163543972 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00010a10 con 0x7fac0815be30 2023-11-15T03:02:12.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.167+0000 7fac0e3f9700 1 -- 172.21.15.121:0/2163543972 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac080c48e0 con 0x7fac0815be30 2023-11-15T03:02:12.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.171+0000 7fac0e3f9700 1 -- 172.21.15.121:0/2163543972 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac080c4e00 con 0x7fac0815be30 2023-11-15T03:02:12.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.171+0000 7fabfd7fa700 1 -- 172.21.15.121:0/2163543972 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fac0001b400 con 0x7fac0815be30 2023-11-15T03:02:12.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.175+0000 7fabfd7fa700 1 --2- 172.21.15.121:0/2163543972 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fabec05a510 0x7fabec05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:12.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.175+0000 7fabfd7fa700 1 -- 172.21.15.121:0/2163543972 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5527+0+0 (secure 0 0 0) 0x7fac000743a0 con 0x7fac0815be30 2023-11-15T03:02:12.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.175+0000 7fac0d3f7700 1 --2- 172.21.15.121:0/2163543972 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fabec05a510 0x7fabec05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:12.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.175+0000 7fac0d3f7700 1 --2- 172.21.15.121:0/2163543972 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fabec05a510 0x7fabec05c9d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7fac04009fd0 tx=0x7fac04017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:12.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.179+0000 7fabfd7fa700 1 -- 172.21.15.121:0/2163543972 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fac080c4e00 con 0x7fac0815be30 2023-11-15T03:02:12.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:12.391+0000 7fac0e3f9700 1 -- 172.21.15.121:0/2163543972 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]} v 0) v1 -- 0x7fac0806bcf0 con 0x7fac0815be30 2023-11-15T03:02:13.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.187+0000 7fabfd7fa700 1 -- 172.21.15.121:0/2163543972 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]}]=0 v259) v1 ==== 142+0+0 (secure 0 0 0) 0x7fac00047cb0 con 0x7fac0815be30 2023-11-15T03:02:13.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.191+0000 7fac0e3f9700 1 -- 172.21.15.121:0/2163543972 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fabec05a510 msgr2=0x7fabec05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:13.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.191+0000 7fac0e3f9700 1 --2- 172.21.15.121:0/2163543972 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fabec05a510 0x7fabec05c9d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7fac04009fd0 tx=0x7fac04017000).stop 2023-11-15T03:02:13.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.191+0000 7fac0e3f9700 1 -- 172.21.15.121:0/2163543972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac0815be30 msgr2=0x7fac0816c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:13.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.191+0000 7fac0e3f9700 1 --2- 172.21.15.121:0/2163543972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac0815be30 0x7fac0816c3a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7fac00002230 tx=0x7fac00007f70).stop 2023-11-15T03:02:13.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.191+0000 7fac0e3f9700 1 -- 172.21.15.121:0/2163543972 shutdown_connections 2023-11-15T03:02:13.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.191+0000 7fac0e3f9700 1 --2- 172.21.15.121:0/2163543972 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fabec05a510 0x7fabec05c9d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:13.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.191+0000 7fac0e3f9700 1 --2- 172.21.15.121:0/2163543972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac0815be30 0x7fac0816c3a0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:13.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.191+0000 7fac0e3f9700 1 --2- 172.21.15.121:0/2163543972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac08152140 0x7fac08142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:13.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.191+0000 7fac0e3f9700 1 --2- 172.21.15.121:0/2163543972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac080c3280 0x7fac08153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:13.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.191+0000 7fac0e3f9700 1 -- 172.21.15.121:0/2163543972 >> 172.21.15.121:0/2163543972 conn(0x7fac08062190 msgr2=0x7fac0815a810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:13.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.191+0000 7fac0e3f9700 1 -- 172.21.15.121:0/2163543972 shutdown_connections 2023-11-15T03:02:13.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.191+0000 7fac0e3f9700 1 -- 172.21.15.121:0/2163543972 wait complete. 2023-11-15T03:02:13.223 INFO:tasks.cephfs_test_runner:That the MDS won't write a dentry with new damage to CDentry::first ... ok 2023-11-15T03:02:13.224 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_preflush (tasks.cephfs.test_damage.TestDamage) That the MDS won't write a dentry with new damage to CDentry::first 2023-11-15T03:02:13.224 INFO:tasks.cephfs_test_runner:test_dentry_first_preflush (tasks.cephfs.test_damage.TestDamage) 2023-11-15T03:02:13.224 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush' 2023-11-15T03:02:13.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.383+0000 7fa651d07700 1 Processor -- start 2023-11-15T03:02:13.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.383+0000 7fa651d07700 1 -- start start 2023-11-15T03:02:13.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.383+0000 7fa651d07700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa64c151910 0x7fa64c0bd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:13.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.383+0000 7fa651d07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa64c151cf0 0x7fa64c0c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:13.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.383+0000 7fa651d07700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64c0beb70 con 0x7fa64c0bf000 2023-11-15T03:02:13.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.383+0000 7fa651d07700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64c0bdf60 con 0x7fa64c151910 2023-11-15T03:02:13.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.383+0000 7fa651d07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64c0be240 con 0x7fa64c151cf0 2023-11-15T03:02:13.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.383+0000 7fa650d05700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa64c151910 0x7fa64c0bd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:13.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.383+0000 7fa650d05700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa64c151910 0x7fa64c0bd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53216/0 (socket says 172.21.15.121:53216) 2023-11-15T03:02:13.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.383+0000 7fa650d05700 1 -- 172.21.15.121:0/3871877596 learned_addr learned my addr 172.21.15.121:0/3871877596 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:13.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.383+0000 7fa643fff700 1 --2- 172.21.15.121:0/3871877596 >> v2:172.21.15.190:3300/0 conn(0x7fa64c151cf0 0x7fa64c0c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:13.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.383+0000 7fa650d05700 1 -- 172.21.15.121:0/3871877596 >> v2:172.21.15.190:3300/0 conn(0x7fa64c151cf0 msgr2=0x7fa64c0c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:13.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.383+0000 7fa650d05700 1 --2- 172.21.15.121:0/3871877596 >> v2:172.21.15.190:3300/0 conn(0x7fa64c151cf0 0x7fa64c0c5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:13.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.383+0000 7fa650d05700 1 -- 172.21.15.121:0/3871877596 >> v1:172.21.15.190:6789/0 conn(0x7fa64c0bf000 legacy=0x7fa64c0c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:13.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.383+0000 7fa650d05700 1 -- 172.21.15.121:0/3871877596 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c0be520 con 0x7fa64c151910 2023-11-15T03:02:13.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.387+0000 7fa650d05700 1 --2- 172.21.15.121:0/3871877596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa64c151910 0x7fa64c0bd8d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7fa64c05fb40 tx=0x7fa648009840).ready entity=mon.0 client_cookie=8b6d7d7654dc4251 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:13.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.387+0000 7fa6437fe700 1 -- 172.21.15.121:0/3871877596 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64800e070 con 0x7fa64c151910 2023-11-15T03:02:13.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.387+0000 7fa6437fe700 1 -- 172.21.15.121:0/3871877596 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64800bad0 con 0x7fa64c151910 2023-11-15T03:02:13.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.387+0000 7fa6437fe700 1 -- 172.21.15.121:0/3871877596 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64800be50 con 0x7fa64c151910 2023-11-15T03:02:13.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.387+0000 7fa651d07700 1 -- 172.21.15.121:0/3871877596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa64c151910 msgr2=0x7fa64c0bd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:13.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.387+0000 7fa651d07700 1 --2- 172.21.15.121:0/3871877596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa64c151910 0x7fa64c0bd8d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7fa64c05fb40 tx=0x7fa648009840).stop 2023-11-15T03:02:13.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.387+0000 7fa651d07700 1 -- 172.21.15.121:0/3871877596 shutdown_connections 2023-11-15T03:02:13.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.387+0000 7fa651d07700 1 --2- 172.21.15.121:0/3871877596 >> v2:172.21.15.190:3300/0 conn(0x7fa64c151cf0 0x7fa64c0c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:13.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.387+0000 7fa651d07700 1 --2- 172.21.15.121:0/3871877596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa64c151910 0x7fa64c0bd8d0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:13.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.387+0000 7fa651d07700 1 -- 172.21.15.121:0/3871877596 >> 172.21.15.121:0/3871877596 conn(0x7fa64c0643a0 msgr2=0x7fa64c0647b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:13.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.387+0000 7fa651d07700 1 -- 172.21.15.121:0/3871877596 shutdown_connections 2023-11-15T03:02:13.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.387+0000 7fa651d07700 1 -- 172.21.15.121:0/3871877596 wait complete. 2023-11-15T03:02:13.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.391+0000 7fa651d07700 1 Processor -- start 2023-11-15T03:02:13.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.391+0000 7fa651d07700 1 -- start start 2023-11-15T03:02:13.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.391+0000 7fa651d07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa64c0bf000 0x7fa64c1507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:13.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.391+0000 7fa651d07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64c151910 0x7fa64c14de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:13.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.391+0000 7fa651d07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa64c151cf0 0x7fa64c14e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:13.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.391+0000 7fa651d07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c14e9e0 con 0x7fa64c151cf0 2023-11-15T03:02:13.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.391+0000 7fa651d07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0c03e0 con 0x7fa64c0bf000 2023-11-15T03:02:13.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.391+0000 7fa651d07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0bfab0 con 0x7fa64c151910 2023-11-15T03:02:13.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.391+0000 7fa651506700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa64c151cf0 0x7fa64c14e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:13.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.391+0000 7fa651506700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa64c151cf0 0x7fa64c14e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53230/0 (socket says 172.21.15.121:53230) 2023-11-15T03:02:13.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.391+0000 7fa651506700 1 -- 172.21.15.121:0/817049063 learned_addr learned my addr 172.21.15.121:0/817049063 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:13.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.395+0000 7fa650d05700 1 --2- 172.21.15.121:0/817049063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa64c0bf000 0x7fa64c1507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:13.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.395+0000 7fa643fff700 1 --2- 172.21.15.121:0/817049063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64c151910 0x7fa64c14de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:13.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.395+0000 7fa651506700 1 -- 172.21.15.121:0/817049063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64c151910 msgr2=0x7fa64c14de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:13.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.395+0000 7fa651506700 1 --2- 172.21.15.121:0/817049063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64c151910 0x7fa64c14de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:13.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.395+0000 7fa651506700 1 -- 172.21.15.121:0/817049063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa64c0bf000 msgr2=0x7fa64c1507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:13.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.395+0000 7fa651506700 1 --2- 172.21.15.121:0/817049063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa64c0bf000 0x7fa64c1507e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:13.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.395+0000 7fa651506700 1 -- 172.21.15.121:0/817049063 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa648009ae0 con 0x7fa64c151cf0 2023-11-15T03:02:13.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.395+0000 7fa643fff700 1 --2- 172.21.15.121:0/817049063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64c151910 0x7fa64c14de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:13.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.395+0000 7fa651506700 1 --2- 172.21.15.121:0/817049063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa64c151cf0 0x7fa64c14e330 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fa64400c440 tx=0x7fa644007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:13.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.395+0000 7fa640ff9700 1 -- 172.21.15.121:0/817049063 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64400cea0 con 0x7fa64c151cf0 2023-11-15T03:02:13.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.395+0000 7fa640ff9700 1 -- 172.21.15.121:0/817049063 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa644001e90 con 0x7fa64c151cf0 2023-11-15T03:02:13.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.395+0000 7fa651d07700 1 -- 172.21.15.121:0/817049063 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64c0bfd90 con 0x7fa64c151cf0 2023-11-15T03:02:13.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.395+0000 7fa640ff9700 1 -- 172.21.15.121:0/817049063 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64400f680 con 0x7fa64c151cf0 2023-11-15T03:02:13.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.395+0000 7fa651d07700 1 -- 172.21.15.121:0/817049063 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64c150cd0 con 0x7fa64c151cf0 2023-11-15T03:02:13.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.395+0000 7fa651d07700 1 -- 172.21.15.121:0/817049063 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64c06bcf0 con 0x7fa64c151cf0 2023-11-15T03:02:13.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.399+0000 7fa640ff9700 1 -- 172.21.15.121:0/817049063 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa64400fa70 con 0x7fa64c151cf0 2023-11-15T03:02:13.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.399+0000 7fa640ff9700 1 --2- 172.21.15.121:0/817049063 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa63805a540 0x7fa63805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:13.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.399+0000 7fa640ff9700 1 -- 172.21.15.121:0/817049063 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5527+0+0 (secure 0 0 0) 0x7fa644073990 con 0x7fa64c151cf0 2023-11-15T03:02:13.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.399+0000 7fa650d05700 1 --2- 172.21.15.121:0/817049063 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa63805a540 0x7fa63805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:13.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.399+0000 7fa650d05700 1 --2- 172.21.15.121:0/817049063 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa63805a540 0x7fa63805ca00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fa648009fd0 tx=0x7fa648009840).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:13.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.403+0000 7fa640ff9700 1 -- 172.21.15.121:0/817049063 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa644047300 con 0x7fa64c151cf0 2023-11-15T03:02:13.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:13.627+0000 7fa651d07700 1 -- 172.21.15.121:0/817049063 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]} v 0) v1 -- 0x7fa64c0be800 con 0x7fa64c151cf0 2023-11-15T03:02:14.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.195+0000 7fa640ff9700 1 -- 172.21.15.121:0/817049063 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]}]=0 v260) v1 ==== 144+0+0 (secure 0 0 0) 0x7fa64404a920 con 0x7fa64c151cf0 2023-11-15T03:02:14.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.199+0000 7fa651d07700 1 -- 172.21.15.121:0/817049063 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa63805a540 msgr2=0x7fa63805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:14.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.199+0000 7fa651d07700 1 --2- 172.21.15.121:0/817049063 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa63805a540 0x7fa63805ca00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fa648009fd0 tx=0x7fa648009840).stop 2023-11-15T03:02:14.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.199+0000 7fa651d07700 1 -- 172.21.15.121:0/817049063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa64c151cf0 msgr2=0x7fa64c14e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:14.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.199+0000 7fa651d07700 1 --2- 172.21.15.121:0/817049063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa64c151cf0 0x7fa64c14e330 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fa64400c440 tx=0x7fa644007d70).stop 2023-11-15T03:02:14.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.199+0000 7fa651d07700 1 -- 172.21.15.121:0/817049063 shutdown_connections 2023-11-15T03:02:14.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.199+0000 7fa651d07700 1 --2- 172.21.15.121:0/817049063 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa63805a540 0x7fa63805ca00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:14.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.199+0000 7fa651d07700 1 --2- 172.21.15.121:0/817049063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa64c151cf0 0x7fa64c14e330 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:14.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.199+0000 7fa651d07700 1 --2- 172.21.15.121:0/817049063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64c151910 0x7fa64c14de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:14.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.199+0000 7fa651d07700 1 --2- 172.21.15.121:0/817049063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa64c0bf000 0x7fa64c1507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:14.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.199+0000 7fa651d07700 1 -- 172.21.15.121:0/817049063 >> 172.21.15.121:0/817049063 conn(0x7fa64c0643a0 msgr2=0x7fa64c14d320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:14.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.199+0000 7fa651d07700 1 -- 172.21.15.121:0/817049063 shutdown_connections 2023-11-15T03:02:14.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.199+0000 7fa651d07700 1 -- 172.21.15.121:0/817049063 wait complete. 2023-11-15T03:02:14.225 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-15T03:02:14.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.379+0000 7f4b1e6db700 1 Processor -- start 2023-11-15T03:02:14.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.379+0000 7f4b1e6db700 1 -- start start 2023-11-15T03:02:14.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b1e6db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4b18065850 0x7f4b180a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:14.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b1e6db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b18068170 0x7f4b180bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:14.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b1e6db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b18152150 con 0x7f4b180a8b80 2023-11-15T03:02:14.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b1e6db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b18148f70 con 0x7f4b18065850 2023-11-15T03:02:14.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b1e6db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b181523a0 con 0x7f4b18068170 2023-11-15T03:02:14.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b177fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4b18065850 0x7f4b180a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:14.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b17fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4b180a8b80 0x7f4b18067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34438/0 (socket says 172.21.15.121:34438) 2023-11-15T03:02:14.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b177fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4b18065850 0x7f4b180a8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53238/0 (socket says 172.21.15.121:53238) 2023-11-15T03:02:14.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b17fff700 1 -- 172.21.15.121:0/4266017306 learned_addr learned my addr 172.21.15.121:0/4266017306 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:14.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b1cc78700 1 --2- 172.21.15.121:0/4266017306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b18068170 0x7f4b180bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:14.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b177fe700 1 -- 172.21.15.121:0/4266017306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b18068170 msgr2=0x7f4b180bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:14.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b177fe700 1 --2- 172.21.15.121:0/4266017306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b18068170 0x7f4b180bbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:14.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b177fe700 1 -- 172.21.15.121:0/4266017306 >> v1:172.21.15.121:6789/0 conn(0x7f4b180a8b80 legacy=0x7f4b18067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:14.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b177fe700 1 -- 172.21.15.121:0/4266017306 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b18152680 con 0x7f4b18065850 2023-11-15T03:02:14.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b177fe700 1 --2- 172.21.15.121:0/4266017306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18065850 0x7f4b180a8f60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f4b0c009760 tx=0x7f4b0c009060).ready entity=mon.0 client_cookie=e8384c3aef4bfa64 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:14.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b16ffd700 1 -- 172.21.15.121:0/4266017306 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c00f510 con 0x7f4b18065850 2023-11-15T03:02:14.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b16ffd700 1 -- 172.21.15.121:0/4266017306 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c00f670 con 0x7f4b18065850 2023-11-15T03:02:14.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b16ffd700 1 -- 172.21.15.121:0/4266017306 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c00f9f0 con 0x7f4b18065850 2023-11-15T03:02:14.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b1e6db700 1 -- 172.21.15.121:0/4266017306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18065850 msgr2=0x7f4b180a8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:14.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.383+0000 7f4b1e6db700 1 --2- 172.21.15.121:0/4266017306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18065850 0x7f4b180a8f60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f4b0c009760 tx=0x7f4b0c009060).stop 2023-11-15T03:02:14.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.387+0000 7f4b1e6db700 1 -- 172.21.15.121:0/4266017306 shutdown_connections 2023-11-15T03:02:14.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.387+0000 7f4b1e6db700 1 --2- 172.21.15.121:0/4266017306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b18068170 0x7f4b180bbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:14.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.387+0000 7f4b1e6db700 1 --2- 172.21.15.121:0/4266017306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18065850 0x7f4b180a8f60 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:14.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.387+0000 7f4b1e6db700 1 -- 172.21.15.121:0/4266017306 >> 172.21.15.121:0/4266017306 conn(0x7f4b180591c0 msgr2=0x7f4b180c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:14.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.387+0000 7f4b1e6db700 1 -- 172.21.15.121:0/4266017306 shutdown_connections 2023-11-15T03:02:14.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.387+0000 7f4b1e6db700 1 -- 172.21.15.121:0/4266017306 wait complete. 2023-11-15T03:02:14.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.387+0000 7f4b1e6db700 1 Processor -- start 2023-11-15T03:02:14.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.391+0000 7f4b1e6db700 1 -- start start 2023-11-15T03:02:14.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.391+0000 7f4b1e6db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18065850 0x7f4b181481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:14.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.391+0000 7f4b1e6db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b18068170 0x7f4b180c1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:14.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.391+0000 7f4b1e6db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b180a8b80 0x7f4b180c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:14.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.391+0000 7f4b1e6db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b180c2b70 con 0x7f4b18065850 2023-11-15T03:02:14.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.391+0000 7f4b1e6db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b180c1f60 con 0x7f4b180a8b80 2023-11-15T03:02:14.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.391+0000 7f4b1e6db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b180c2240 con 0x7f4b18068170 2023-11-15T03:02:14.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.391+0000 7f4b17fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18065850 0x7f4b181481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:14.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.391+0000 7f4b17fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18065850 0x7f4b181481a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53242/0 (socket says 172.21.15.121:53242) 2023-11-15T03:02:14.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.391+0000 7f4b17fff700 1 -- 172.21.15.121:0/1552419563 learned_addr learned my addr 172.21.15.121:0/1552419563 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:14.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.391+0000 7f4b177fe700 1 --2- 172.21.15.121:0/1552419563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b18068170 0x7f4b180c1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:14.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.391+0000 7f4b1cc78700 1 --2- 172.21.15.121:0/1552419563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b180a8b80 0x7f4b180c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:14.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.391+0000 7f4b17fff700 1 -- 172.21.15.121:0/1552419563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b18068170 msgr2=0x7f4b180c1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:14.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.391+0000 7f4b17fff700 1 --2- 172.21.15.121:0/1552419563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b18068170 0x7f4b180c1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:14.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.391+0000 7f4b17fff700 1 -- 172.21.15.121:0/1552419563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b180a8b80 msgr2=0x7f4b180c31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:14.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.391+0000 7f4b17fff700 1 --2- 172.21.15.121:0/1552419563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b180a8b80 0x7f4b180c31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:14.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.391+0000 7f4b17fff700 1 -- 172.21.15.121:0/1552419563 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0c0092a0 con 0x7f4b18065850 2023-11-15T03:02:14.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.391+0000 7f4b17fff700 1 --2- 172.21.15.121:0/1552419563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18065850 0x7f4b181481a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f4b08001eb0 tx=0x7f4b08007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:14.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.395+0000 7f4afffff700 1 -- 172.21.15.121:0/1552419563 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0800ce40 con 0x7f4b18065850 2023-11-15T03:02:14.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.395+0000 7f4afffff700 1 -- 172.21.15.121:0/1552419563 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b080072e0 con 0x7f4b18065850 2023-11-15T03:02:14.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.395+0000 7f4b1e6db700 1 -- 172.21.15.121:0/1552419563 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b180c2520 con 0x7f4b18065850 2023-11-15T03:02:14.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.395+0000 7f4afffff700 1 -- 172.21.15.121:0/1552419563 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b080057a0 con 0x7f4b18065850 2023-11-15T03:02:14.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.395+0000 7f4b1e6db700 1 -- 172.21.15.121:0/1552419563 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b18148690 con 0x7f4b18065850 2023-11-15T03:02:14.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.395+0000 7f4b1e6db700 1 -- 172.21.15.121:0/1552419563 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b18047060 con 0x7f4b18065850 2023-11-15T03:02:14.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.395+0000 7f4afffff700 1 -- 172.21.15.121:0/1552419563 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4b08005b50 con 0x7f4b18065850 2023-11-15T03:02:14.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.399+0000 7f4afffff700 1 --2- 172.21.15.121:0/1552419563 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4b0405a540 0x7f4b0405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:14.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.399+0000 7f4afffff700 1 -- 172.21.15.121:0/1552419563 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5527+0+0 (secure 0 0 0) 0x7f4b08073db0 con 0x7f4b18065850 2023-11-15T03:02:14.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.399+0000 7f4b177fe700 1 --2- 172.21.15.121:0/1552419563 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4b0405a540 0x7f4b0405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:14.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.399+0000 7f4b177fe700 1 --2- 172.21.15.121:0/1552419563 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4b0405a540 0x7f4b0405ca00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f4b0c009fd0 tx=0x7f4b0c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:14.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.403+0000 7f4afffff700 1 -- 172.21.15.121:0/1552419563 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4b08078070 con 0x7f4b18065850 2023-11-15T03:02:14.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.563+0000 7f4b1e6db700 1 -- 172.21.15.121:0/1552419563 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4b18063b60 con 0x7f4b18065850 2023-11-15T03:02:14.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.571+0000 7f4afffff700 1 -- 172.21.15.121:0/1552419563 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4b08047720 con 0x7f4b18065850 2023-11-15T03:02:14.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.575+0000 7f4b1e6db700 1 -- 172.21.15.121:0/1552419563 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4b0405a540 msgr2=0x7f4b0405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:14.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.575+0000 7f4b1e6db700 1 --2- 172.21.15.121:0/1552419563 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4b0405a540 0x7f4b0405ca00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f4b0c009fd0 tx=0x7f4b0c017000).stop 2023-11-15T03:02:14.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.575+0000 7f4b1e6db700 1 -- 172.21.15.121:0/1552419563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18065850 msgr2=0x7f4b181481a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:14.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.575+0000 7f4b1e6db700 1 --2- 172.21.15.121:0/1552419563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18065850 0x7f4b181481a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f4b08001eb0 tx=0x7f4b08007e50).stop 2023-11-15T03:02:14.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.575+0000 7f4b1e6db700 1 -- 172.21.15.121:0/1552419563 shutdown_connections 2023-11-15T03:02:14.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.575+0000 7f4b1e6db700 1 --2- 172.21.15.121:0/1552419563 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4b0405a540 0x7f4b0405ca00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:14.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.575+0000 7f4b1e6db700 1 --2- 172.21.15.121:0/1552419563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b180a8b80 0x7f4b180c31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:14.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.575+0000 7f4b1e6db700 1 --2- 172.21.15.121:0/1552419563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b18068170 0x7f4b180c1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:14.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.575+0000 7f4b1e6db700 1 --2- 172.21.15.121:0/1552419563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18065850 0x7f4b181481a0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:14.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.575+0000 7f4b1e6db700 1 -- 172.21.15.121:0/1552419563 >> 172.21.15.121:0/1552419563 conn(0x7f4b180591c0 msgr2=0x7f4b180669c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:14.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.575+0000 7f4b1e6db700 1 -- 172.21.15.121:0/1552419563 shutdown_connections 2023-11-15T03:02:14.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.575+0000 7f4b1e6db700 1 -- 172.21.15.121:0/1552419563 wait complete. 2023-11-15T03:02:14.602 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-15T03:02:14.603 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T03:02:14.646 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T03:02:14.692 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-15T03:02:14.692 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-15T03:02:14.693 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:02:14.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.843+0000 7fc626f07700 1 Processor -- start 2023-11-15T03:02:14.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.847+0000 7fc626f07700 1 -- start start 2023-11-15T03:02:14.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.847+0000 7fc626f07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc6200c3280 0x7fc6200c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:14.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.847+0000 7fc626f07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc62015be30 0x7fc62015c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:14.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.847+0000 7fc626f07700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6200c1d50 con 0x7fc620152140 2023-11-15T03:02:14.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.847+0000 7fc626f07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6200c1860 con 0x7fc6200c3280 2023-11-15T03:02:14.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.847+0000 7fc626f07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62015cb80 con 0x7fc62015be30 2023-11-15T03:02:14.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc625704700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc620152140 0x7fc6201525f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:54984/0 (socket says 172.21.15.121:54984) 2023-11-15T03:02:14.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc625704700 1 -- 172.21.15.121:0/3536967331 learned_addr learned my addr 172.21.15.121:0/3536967331 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:14.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc626706700 1 --2- 172.21.15.121:0/3536967331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc62015be30 0x7fc62015c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:14.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc625f05700 1 --2- 172.21.15.121:0/3536967331 >> v2:172.21.15.190:3300/0 conn(0x7fc6200c3280 0x7fc6200c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:14.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc624f03700 1 -- 172.21.15.121:0/3536967331 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473215970 0 0) 0x7fc6200c1d50 con 0x7fc620152140 2023-11-15T03:02:14.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc624f03700 1 -- 172.21.15.121:0/3536967331 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc60c0033f0 con 0x7fc620152140 2023-11-15T03:02:14.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc624f03700 1 -- 172.21.15.121:0/3536967331 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1933475595 0 0) 0x7fc60c0033f0 con 0x7fc620152140 2023-11-15T03:02:14.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc624f03700 1 -- 172.21.15.121:0/3536967331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc62015be30 msgr2=0x7fc62015c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:14.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc624f03700 1 --2- 172.21.15.121:0/3536967331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc62015be30 0x7fc62015c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:14.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc624f03700 1 -- 172.21.15.121:0/3536967331 >> v2:172.21.15.190:3300/0 conn(0x7fc6200c3280 msgr2=0x7fc6200c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:14.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc624f03700 1 --2- 172.21.15.121:0/3536967331 >> v2:172.21.15.190:3300/0 conn(0x7fc6200c3280 0x7fc6200c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:14.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc624f03700 1 -- 172.21.15.121:0/3536967331 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6200a9240 con 0x7fc620152140 2023-11-15T03:02:14.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc624f03700 1 -- 172.21.15.121:0/3536967331 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fc610001470 con 0x7fc620152140 2023-11-15T03:02:14.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc625f05700 1 --2- 172.21.15.121:0/3536967331 >> v2:172.21.15.190:3300/0 conn(0x7fc6200c3280 0x7fc6200c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:02:14.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc624f03700 1 -- 172.21.15.121:0/3536967331 >> v1:172.21.15.121:6789/0 conn(0x7fc620152140 legacy=0x7fc6201525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:14.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc624f03700 1 --2- 172.21.15.121:0/3536967331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc60c0038e0 0x7fc60c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:14.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc624f03700 1 --2- 172.21.15.121:0/3536967331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc62015be30 0x7fc60c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:14.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc624f03700 1 --2- 172.21.15.121:0/3536967331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc60c004b80 0x7fc60c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:14.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc624f03700 1 -- 172.21.15.121:0/3536967331 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6200c1860 con 0x7fc60c004b80 2023-11-15T03:02:14.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc624f03700 1 -- 172.21.15.121:0/3536967331 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62015cb80 con 0x7fc60c0038e0 2023-11-15T03:02:14.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc624f03700 1 -- 172.21.15.121:0/3536967331 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c00b8e0 con 0x7fc62015be30 2023-11-15T03:02:14.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc626706700 1 --2- 172.21.15.121:0/3536967331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc60c004b80 0x7fc60c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:14.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc625704700 1 --2- 172.21.15.121:0/3536967331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc62015be30 0x7fc60c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:14.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc625f05700 1 --2- 172.21.15.121:0/3536967331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc60c0038e0 0x7fc60c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:14.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc626706700 1 -- 172.21.15.121:0/3536967331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc62015be30 msgr2=0x7fc60c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:14.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc626706700 1 --2- 172.21.15.121:0/3536967331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc62015be30 0x7fc60c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:14.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc626706700 1 -- 172.21.15.121:0/3536967331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc60c0038e0 msgr2=0x7fc60c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:14.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc626706700 1 --2- 172.21.15.121:0/3536967331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc60c0038e0 0x7fc60c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:14.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc626706700 1 -- 172.21.15.121:0/3536967331 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c0033f0 con 0x7fc60c004b80 2023-11-15T03:02:14.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.851+0000 7fc626706700 1 --2- 172.21.15.121:0/3536967331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc60c004b80 0x7fc60c005000 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7fc61800e210 tx=0x7fc61800b040).ready entity=mon.0 client_cookie=f2ab040ea88662be server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:14.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.855+0000 7fc624f03700 1 -- 172.21.15.121:0/3536967331 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc618009ac0 con 0x7fc60c004b80 2023-11-15T03:02:14.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.855+0000 7fc624f03700 1 -- 172.21.15.121:0/3536967331 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61800f040 con 0x7fc60c004b80 2023-11-15T03:02:14.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.855+0000 7fc624f03700 1 -- 172.21.15.121:0/3536967331 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61800ecd0 con 0x7fc60c004b80 2023-11-15T03:02:14.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.855+0000 7fc626f07700 1 -- 172.21.15.121:0/3536967331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc60c004b80 msgr2=0x7fc60c005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:14.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.855+0000 7fc626f07700 1 --2- 172.21.15.121:0/3536967331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc60c004b80 0x7fc60c005000 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7fc61800e210 tx=0x7fc61800b040).stop 2023-11-15T03:02:14.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.855+0000 7fc626706700 1 -- 172.21.15.121:0/3536967331 reap_dead start 2023-11-15T03:02:14.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.855+0000 7fc626f07700 1 -- 172.21.15.121:0/3536967331 shutdown_connections 2023-11-15T03:02:14.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.855+0000 7fc626f07700 1 -- 172.21.15.121:0/3536967331 >> 172.21.15.121:0/3536967331 conn(0x7fc620062190 msgr2=0x7fc620062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:14.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.855+0000 7fc626f07700 1 -- 172.21.15.121:0/3536967331 shutdown_connections 2023-11-15T03:02:14.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.855+0000 7fc626f07700 1 -- 172.21.15.121:0/3536967331 wait complete. 2023-11-15T03:02:14.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.855+0000 7fc626f07700 1 Processor -- start 2023-11-15T03:02:14.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.855+0000 7fc626f07700 1 -- start start 2023-11-15T03:02:14.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc626f07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc60c004b80 0x7fc62015ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:14.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc626f07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc62015b040 0x7fc6200abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:14.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc626f07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6200abff0 0x7fc62013fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:14.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc626f07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6200aa4f0 con 0x7fc62015b040 2023-11-15T03:02:14.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc626f07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6200a98e0 con 0x7fc60c004b80 2023-11-15T03:02:14.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc626f07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6200a9bc0 con 0x7fc6200abff0 2023-11-15T03:02:14.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc625704700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc62015b040 0x7fc6200abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:14.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc625704700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc62015b040 0x7fc6200abb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54502/0 (socket says 172.21.15.121:54502) 2023-11-15T03:02:14.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc625704700 1 -- 172.21.15.121:0/3530198088 learned_addr learned my addr 172.21.15.121:0/3530198088 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:14.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc626706700 1 --2- 172.21.15.121:0/3530198088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6200abff0 0x7fc62013fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:14.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc625f05700 1 --2- 172.21.15.121:0/3530198088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc60c004b80 0x7fc62015ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:14.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc625704700 1 -- 172.21.15.121:0/3530198088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6200abff0 msgr2=0x7fc62013fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:14.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc625704700 1 --2- 172.21.15.121:0/3530198088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6200abff0 0x7fc62013fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:14.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc625704700 1 -- 172.21.15.121:0/3530198088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc60c004b80 msgr2=0x7fc62015ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:14.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc625704700 1 --2- 172.21.15.121:0/3530198088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc60c004b80 0x7fc62015ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:14.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc625704700 1 -- 172.21.15.121:0/3530198088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6200a9ea0 con 0x7fc62015b040 2023-11-15T03:02:14.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc625f05700 1 --2- 172.21.15.121:0/3530198088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc60c004b80 0x7fc62015ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:14.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc625704700 1 --2- 172.21.15.121:0/3530198088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc62015b040 0x7fc6200abb00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7fc6100049e0 tx=0x7fc6200c1020).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:14.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc615ffb700 1 -- 172.21.15.121:0/3530198088 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61000f660 con 0x7fc62015b040 2023-11-15T03:02:14.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc615ffb700 1 -- 172.21.15.121:0/3530198088 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61000f7c0 con 0x7fc62015b040 2023-11-15T03:02:14.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc626f07700 1 -- 172.21.15.121:0/3530198088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6200aa180 con 0x7fc62015b040 2023-11-15T03:02:14.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc615ffb700 1 -- 172.21.15.121:0/3530198088 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61000fb60 con 0x7fc62015b040 2023-11-15T03:02:14.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.859+0000 7fc626f07700 1 -- 172.21.15.121:0/3530198088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc620140380 con 0x7fc62015b040 2023-11-15T03:02:14.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.863+0000 7fc626f07700 1 -- 172.21.15.121:0/3530198088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62006bcf0 con 0x7fc62015b040 2023-11-15T03:02:14.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.863+0000 7fc615ffb700 1 -- 172.21.15.121:0/3530198088 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc61001d3b0 con 0x7fc62015b040 2023-11-15T03:02:14.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.863+0000 7fc615ffb700 1 --2- 172.21.15.121:0/3530198088 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc60405a510 0x7fc60405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:14.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.863+0000 7fc615ffb700 1 -- 172.21.15.121:0/3530198088 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5527+0+0 (secure 0 0 0) 0x7fc610073e10 con 0x7fc62015b040 2023-11-15T03:02:14.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.863+0000 7fc625f05700 1 --2- 172.21.15.121:0/3530198088 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc60405a510 0x7fc60405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:14.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.867+0000 7fc625f05700 1 --2- 172.21.15.121:0/3530198088 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc60405a510 0x7fc60405c9d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fc61c006d80 tx=0x7fc61c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:14.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:14.871+0000 7fc615ffb700 1 -- 172.21.15.121:0/3530198088 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc610047890 con 0x7fc62015b040 2023-11-15T03:02:15.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.063+0000 7fc626f07700 1 -- 172.21.15.121:0/3530198088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6200c1f80 con 0x7fc62015b040 2023-11-15T03:02:15.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.067+0000 7fc615ffb700 1 -- 172.21.15.121:0/3530198088 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+3541 (secure 0 0 0) 0x7fc610019020 con 0x7fc62015b040 2023-11-15T03:02:15.075 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:15.075 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":61,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3462858472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3462858472},{"type":"v1","addr":"172.21.15.190:6833","nonce":3462858472}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4042390087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4042390087},{"type":"v1","addr":"172.21.15.190:6835","nonce":4042390087}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3908033272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3908033272},{"type":"v1","addr":"172.21.15.121:6836","nonce":3908033272}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6417,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3102645719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3102645719},{"type":"v1","addr":"172.21.15.121:6837","nonce":3102645719}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[]} 2023-11-15T03:02:15.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.067+0000 7fc626f07700 1 -- 172.21.15.121:0/3530198088 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc60405a510 msgr2=0x7fc60405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:15.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.067+0000 7fc626f07700 1 --2- 172.21.15.121:0/3530198088 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc60405a510 0x7fc60405c9d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fc61c006d80 tx=0x7fc61c008040).stop 2023-11-15T03:02:15.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.067+0000 7fc626f07700 1 -- 172.21.15.121:0/3530198088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc62015b040 msgr2=0x7fc6200abb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:15.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.067+0000 7fc626f07700 1 --2- 172.21.15.121:0/3530198088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc62015b040 0x7fc6200abb00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7fc6100049e0 tx=0x7fc6200c1020).stop 2023-11-15T03:02:15.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.067+0000 7fc626f07700 1 -- 172.21.15.121:0/3530198088 shutdown_connections 2023-11-15T03:02:15.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.067+0000 7fc626f07700 1 --2- 172.21.15.121:0/3530198088 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc60405a510 0x7fc60405c9d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:15.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.067+0000 7fc626f07700 1 --2- 172.21.15.121:0/3530198088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6200abff0 0x7fc62013fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:15.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.067+0000 7fc626f07700 1 --2- 172.21.15.121:0/3530198088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc62015b040 0x7fc6200abb00 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:15.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.067+0000 7fc626f07700 1 --2- 172.21.15.121:0/3530198088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc60c004b80 0x7fc62015ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:15.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.067+0000 7fc626f07700 1 -- 172.21.15.121:0/3530198088 >> 172.21.15.121:0/3530198088 conn(0x7fc620062190 msgr2=0x7fc6201515c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:15.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.067+0000 7fc626f07700 1 -- 172.21.15.121:0/3530198088 shutdown_connections 2023-11-15T03:02:15.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.071+0000 7fc626f07700 1 -- 172.21.15.121:0/3530198088 wait complete. 2023-11-15T03:02:15.080 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 61 2023-11-15T03:02:15.094 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-15T03:02:15.094 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-15T03:02:15.095 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-15T03:02:15.095 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:02:15.095 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-15T03:02:15.096 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-15T03:02:15.096 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-15T03:02:15.096 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:02:15.096 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-15T03:02:15.097 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-15T03:02:15.097 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-15T03:02:15.097 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:02:15.098 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-15T03:02:15.098 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-15T03:02:15.098 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-15T03:02:15.098 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:02:15.113 INFO:tasks.ceph.mds.a:Stopped 2023-11-15T03:02:15.114 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-15T03:02:15.115 INFO:tasks.ceph.mds.c:Stopped 2023-11-15T03:02:15.115 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-15T03:02:15.122 INFO:tasks.ceph.mds.d:Stopped 2023-11-15T03:02:15.123 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-15T03:02:15.124 INFO:tasks.ceph.mds.b:Stopped 2023-11-15T03:02:15.124 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-15T03:02:15.163 INFO:tasks.ceph.mds.a:Started 2023-11-15T03:02:15.163 DEBUG:teuthology.parallel:result is None 2023-11-15T03:02:15.168 INFO:tasks.ceph.mds.c:Started 2023-11-15T03:02:15.168 DEBUG:teuthology.parallel:result is None 2023-11-15T03:02:15.174 INFO:tasks.ceph.mds.d:Started 2023-11-15T03:02:15.174 DEBUG:teuthology.parallel:result is None 2023-11-15T03:02:15.177 INFO:tasks.ceph.mds.b:Started 2023-11-15T03:02:15.177 DEBUG:teuthology.parallel:result is None 2023-11-15T03:02:15.179 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-15T03:02:15.273 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:02:15.274 INFO:tasks.ceph.mds.c.smithi121.stdout:starting mds.c at 2023-11-15T03:02:15.279 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-11-15T03:02:15.288 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-11-15T03:02:15.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.319+0000 7fa575779700 1 Processor -- start 2023-11-15T03:02:15.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.319+0000 7fa575779700 1 -- start start 2023-11-15T03:02:15.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.319+0000 7fa575779700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa5700ad1b0 0x7fa570065710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:15.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.319+0000 7fa575779700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa570065c00 0x7fa5700add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:15.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.319+0000 7fa575779700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa570066c90 con 0x7fa5700ab0d0 2023-11-15T03:02:15.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.319+0000 7fa575779700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa570066080 con 0x7fa5700ad1b0 2023-11-15T03:02:15.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.319+0000 7fa575779700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa570066360 con 0x7fa570065c00 2023-11-15T03:02:15.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.319+0000 7fa574f78700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa570065c00 0x7fa5700add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:15.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.319+0000 7fa56ffff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa5700ab0d0 0x7fa5700ab4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:55008/0 (socket says 172.21.15.121:55008) 2023-11-15T03:02:15.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.319+0000 7fa56ffff700 1 -- 172.21.15.121:0/1124770554 learned_addr learned my addr 172.21.15.121:0/1124770554 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:15.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.319+0000 7fa5677fe700 1 --2- 172.21.15.121:0/1124770554 >> v2:172.21.15.190:3300/0 conn(0x7fa5700ad1b0 0x7fa570065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:15.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.323+0000 7fa5677fe700 1 -- 172.21.15.121:0/1124770554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa570065c00 msgr2=0x7fa5700add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:15.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.323+0000 7fa56f7fe700 1 -- 172.21.15.121:0/1124770554 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331009551 0 0) 0x7fa570066c90 con 0x7fa5700ab0d0 2023-11-15T03:02:15.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.323+0000 7fa5677fe700 1 --2- 172.21.15.121:0/1124770554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa570065c00 0x7fa5700add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:15.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.323+0000 7fa5677fe700 1 -- 172.21.15.121:0/1124770554 >> v1:172.21.15.121:6789/0 conn(0x7fa5700ab0d0 legacy=0x7fa5700ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:15.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.323+0000 7fa5677fe700 1 -- 172.21.15.121:0/1124770554 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa570066640 con 0x7fa5700ad1b0 2023-11-15T03:02:15.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.323+0000 7fa5677fe700 1 --2- 172.21.15.121:0/1124770554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5700ad1b0 0x7fa570065710 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fa558009e60 tx=0x7fa558009780).ready entity=mon.1 client_cookie=b764fb13afae9917 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:15.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.323+0000 7fa56f7fe700 1 -- 172.21.15.121:0/1124770554 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55800e070 con 0x7fa5700ad1b0 2023-11-15T03:02:15.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.323+0000 7fa56f7fe700 1 -- 172.21.15.121:0/1124770554 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55800bb60 con 0x7fa5700ad1b0 2023-11-15T03:02:15.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.323+0000 7fa56f7fe700 1 -- 172.21.15.121:0/1124770554 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa558012410 con 0x7fa5700ad1b0 2023-11-15T03:02:15.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.323+0000 7fa575779700 1 -- 172.21.15.121:0/1124770554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5700ad1b0 msgr2=0x7fa570065710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:15.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.323+0000 7fa575779700 1 --2- 172.21.15.121:0/1124770554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5700ad1b0 0x7fa570065710 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fa558009e60 tx=0x7fa558009780).stop 2023-11-15T03:02:15.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.323+0000 7fa575779700 1 -- 172.21.15.121:0/1124770554 shutdown_connections 2023-11-15T03:02:15.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.323+0000 7fa575779700 1 --2- 172.21.15.121:0/1124770554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa570065c00 0x7fa5700add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:15.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.323+0000 7fa575779700 1 --2- 172.21.15.121:0/1124770554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5700ad1b0 0x7fa570065710 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:15.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.323+0000 7fa575779700 1 -- 172.21.15.121:0/1124770554 >> 172.21.15.121:0/1124770554 conn(0x7fa570067a60 msgr2=0x7fa570067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:15.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.323+0000 7fa575779700 1 -- 172.21.15.121:0/1124770554 shutdown_connections 2023-11-15T03:02:15.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.323+0000 7fa575779700 1 -- 172.21.15.121:0/1124770554 wait complete. 2023-11-15T03:02:15.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.327+0000 7fa575779700 1 Processor -- start 2023-11-15T03:02:15.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.327+0000 7fa575779700 1 -- start start 2023-11-15T03:02:15.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.327+0000 7fa575779700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa570065c00 0x7fa570164970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:15.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.327+0000 7fa575779700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5700ab0d0 0x7fa570145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:15.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.327+0000 7fa575779700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5700ad1b0 0x7fa5700a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:15.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.327+0000 7fa575779700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5700a9170 con 0x7fa570065c00 2023-11-15T03:02:15.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.327+0000 7fa575779700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5701661d0 con 0x7fa5700ad1b0 2023-11-15T03:02:15.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.327+0000 7fa575779700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa570164e60 con 0x7fa5700ab0d0 2023-11-15T03:02:15.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.327+0000 7fa56ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa570065c00 0x7fa570164970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:15.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.331+0000 7fa56ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa570065c00 0x7fa570164970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54550/0 (socket says 172.21.15.121:54550) 2023-11-15T03:02:15.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.331+0000 7fa56ffff700 1 -- 172.21.15.121:0/728953396 learned_addr learned my addr 172.21.15.121:0/728953396 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:15.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.331+0000 7fa5677fe700 1 --2- 172.21.15.121:0/728953396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5700ab0d0 0x7fa570145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:15.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.331+0000 7fa574f78700 1 --2- 172.21.15.121:0/728953396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5700ad1b0 0x7fa5700a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:15.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.331+0000 7fa56ffff700 1 -- 172.21.15.121:0/728953396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5700ab0d0 msgr2=0x7fa570145460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:15.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.331+0000 7fa56ffff700 1 --2- 172.21.15.121:0/728953396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5700ab0d0 0x7fa570145460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:15.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.331+0000 7fa56ffff700 1 -- 172.21.15.121:0/728953396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5700ad1b0 msgr2=0x7fa5700a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:15.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.331+0000 7fa56ffff700 1 --2- 172.21.15.121:0/728953396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5700ad1b0 0x7fa5700a68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:15.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.331+0000 7fa56ffff700 1 -- 172.21.15.121:0/728953396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa558009a20 con 0x7fa570065c00 2023-11-15T03:02:15.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.331+0000 7fa56ffff700 1 --2- 172.21.15.121:0/728953396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa570065c00 0x7fa570164970 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7fa56000c490 tx=0x7fa560007d00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:15.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.331+0000 7fa56cff9700 1 -- 172.21.15.121:0/728953396 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5600070d0 con 0x7fa570065c00 2023-11-15T03:02:15.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.331+0000 7fa56cff9700 1 -- 172.21.15.121:0/728953396 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa560007230 con 0x7fa570065c00 2023-11-15T03:02:15.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.331+0000 7fa56cff9700 1 -- 172.21.15.121:0/728953396 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa560005670 con 0x7fa570065c00 2023-11-15T03:02:15.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.331+0000 7fa575779700 1 -- 172.21.15.121:0/728953396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa570145950 con 0x7fa570065c00 2023-11-15T03:02:15.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.331+0000 7fa575779700 1 -- 172.21.15.121:0/728953396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5701655c0 con 0x7fa570065c00 2023-11-15T03:02:15.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.331+0000 7fa575779700 1 -- 172.21.15.121:0/728953396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57006bcf0 con 0x7fa570065c00 2023-11-15T03:02:15.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.331+0000 7fa56cff9700 1 -- 172.21.15.121:0/728953396 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa560005c70 con 0x7fa570065c00 2023-11-15T03:02:15.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.335+0000 7fa56cff9700 1 --2- 172.21.15.121:0/728953396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa55005a4e0 0x7fa55005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:15.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.335+0000 7fa56cff9700 1 -- 172.21.15.121:0/728953396 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5527+0+0 (secure 0 0 0) 0x7fa560073cf0 con 0x7fa570065c00 2023-11-15T03:02:15.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.335+0000 7fa5677fe700 1 --2- 172.21.15.121:0/728953396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa55005a4e0 0x7fa55005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:15.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.335+0000 7fa5677fe700 1 --2- 172.21.15.121:0/728953396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa55005a4e0 0x7fa55005c9a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fa5700ad5d0 tx=0x7fa558009780).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:15.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.339+0000 7fa56cff9700 1 -- 172.21.15.121:0/728953396 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa560078070 con 0x7fa570065c00 2023-11-15T03:02:15.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:15.519+0000 7fa575779700 1 -- 172.21.15.121:0/728953396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa5700c7160 con 0x7fa570065c00 2023-11-15T03:02:16.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.331+0000 7fa56cff9700 1 -- 172.21.15.121:0/728953396 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v73) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa560018030 con 0x7fa570065c00 2023-11-15T03:02:16.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.335+0000 7fa575779700 1 -- 172.21.15.121:0/728953396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa55005a4e0 msgr2=0x7fa55005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:16.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.335+0000 7fa575779700 1 --2- 172.21.15.121:0/728953396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa55005a4e0 0x7fa55005c9a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fa5700ad5d0 tx=0x7fa558009780).stop 2023-11-15T03:02:16.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.335+0000 7fa575779700 1 -- 172.21.15.121:0/728953396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa570065c00 msgr2=0x7fa570164970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:16.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.335+0000 7fa575779700 1 --2- 172.21.15.121:0/728953396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa570065c00 0x7fa570164970 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7fa56000c490 tx=0x7fa560007d00).stop 2023-11-15T03:02:16.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.335+0000 7fa575779700 1 -- 172.21.15.121:0/728953396 shutdown_connections 2023-11-15T03:02:16.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.335+0000 7fa575779700 1 --2- 172.21.15.121:0/728953396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa55005a4e0 0x7fa55005c9a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:16.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.335+0000 7fa575779700 1 --2- 172.21.15.121:0/728953396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5700ad1b0 0x7fa5700a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:16.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.335+0000 7fa575779700 1 --2- 172.21.15.121:0/728953396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5700ab0d0 0x7fa570145460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:16.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.335+0000 7fa575779700 1 --2- 172.21.15.121:0/728953396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa570065c00 0x7fa570164970 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:16.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.335+0000 7fa575779700 1 -- 172.21.15.121:0/728953396 >> 172.21.15.121:0/728953396 conn(0x7fa570067a60 msgr2=0x7fa570154d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:16.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.335+0000 7fa575779700 1 -- 172.21.15.121:0/728953396 shutdown_connections 2023-11-15T03:02:16.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.335+0000 7fa575779700 1 -- 172.21.15.121:0/728953396 wait complete. 2023-11-15T03:02:16.343 INFO:teuthology.orchestra.run.smithi121.stderr: removed all blocklist entries 2023-11-15T03:02:16.361 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-15T03:02:16.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.514+0000 7f7c4ffff700 1 Processor -- start 2023-11-15T03:02:16.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.514+0000 7f7c4ffff700 1 -- start start 2023-11-15T03:02:16.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c4ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c500a9830 0x7f7c500a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:16.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c4ffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c500c30e0 0x7f7c500c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:16.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c4ffff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c500657f0 con 0x7f7c50151870 2023-11-15T03:02:16.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c4ffff700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c50065ad0 con 0x7f7c500a9830 2023-11-15T03:02:16.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c4ffff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c500c3a60 con 0x7f7c500c30e0 2023-11-15T03:02:16.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c467fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c500a9830 0x7f7c500a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:16.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c467fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c500a9830 0x7f7c500a9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54566/0 (socket says 172.21.15.121:54566) 2023-11-15T03:02:16.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c467fc700 1 -- 172.21.15.121:0/978838063 learned_addr learned my addr 172.21.15.121:0/978838063 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:16.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c4f7fe700 1 --2- 172.21.15.121:0/978838063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c500c30e0 0x7f7c500c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:16.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c467fc700 1 -- 172.21.15.121:0/978838063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c500c30e0 msgr2=0x7f7c500c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:16.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c467fc700 1 --2- 172.21.15.121:0/978838063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c500c30e0 0x7f7c500c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:16.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c467fc700 1 -- 172.21.15.121:0/978838063 >> v1:172.21.15.190:6789/0 conn(0x7f7c50151870 legacy=0x7f7c50151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:16.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c467fc700 1 -- 172.21.15.121:0/978838063 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c500aa260 con 0x7f7c500a9830 2023-11-15T03:02:16.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c4f7fe700 1 --2- 172.21.15.121:0/978838063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c500c30e0 0x7f7c500c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:16.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c467fc700 1 --2- 172.21.15.121:0/978838063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c500a9830 0x7f7c500a9c70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f7c38006fd0 tx=0x7f7c380090f0).ready entity=mon.0 client_cookie=3db19fc8ba1ccdaf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:16.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c4e7fc700 1 -- 172.21.15.121:0/978838063 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3800b600 con 0x7f7c500a9830 2023-11-15T03:02:16.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c4e7fc700 1 -- 172.21.15.121:0/978838063 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3800b760 con 0x7f7c500a9830 2023-11-15T03:02:16.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c4e7fc700 1 -- 172.21.15.121:0/978838063 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3800bae0 con 0x7f7c500a9830 2023-11-15T03:02:16.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c4ffff700 1 -- 172.21.15.121:0/978838063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c500a9830 msgr2=0x7f7c500a9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:16.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c4ffff700 1 --2- 172.21.15.121:0/978838063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c500a9830 0x7f7c500a9c70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f7c38006fd0 tx=0x7f7c380090f0).stop 2023-11-15T03:02:16.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c4ffff700 1 -- 172.21.15.121:0/978838063 shutdown_connections 2023-11-15T03:02:16.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c4ffff700 1 --2- 172.21.15.121:0/978838063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c500c30e0 0x7f7c500c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:16.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c4ffff700 1 --2- 172.21.15.121:0/978838063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c500a9830 0x7f7c500a9c70 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:16.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c4ffff700 1 -- 172.21.15.121:0/978838063 >> 172.21.15.121:0/978838063 conn(0x7f7c50062190 msgr2=0x7f7c50062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:16.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c4ffff700 1 -- 172.21.15.121:0/978838063 shutdown_connections 2023-11-15T03:02:16.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.518+0000 7f7c4ffff700 1 -- 172.21.15.121:0/978838063 wait complete. 2023-11-15T03:02:16.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.522+0000 7f7c4ffff700 1 Processor -- start 2023-11-15T03:02:16.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.522+0000 7f7c4ffff700 1 -- start start 2023-11-15T03:02:16.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c4ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c500a9830 0x7f7c5014da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:16.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c4ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c500c30e0 0x7f7c501583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:16.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c4ffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c50151870 0x7f7c50156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:16.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c4ffff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5014df80 con 0x7f7c500c30e0 2023-11-15T03:02:16.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c4ffff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c50154a90 con 0x7f7c500a9830 2023-11-15T03:02:16.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c4ffff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c501541a0 con 0x7f7c50151870 2023-11-15T03:02:16.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c467fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c500c30e0 0x7f7c501583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:16.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c467fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c500c30e0 0x7f7c501583a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54574/0 (socket says 172.21.15.121:54574) 2023-11-15T03:02:16.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c467fc700 1 -- 172.21.15.121:0/2559479729 learned_addr learned my addr 172.21.15.121:0/2559479729 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:16.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c4f7fe700 1 --2- 172.21.15.121:0/2559479729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c50151870 0x7f7c50156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:16.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c4effd700 1 --2- 172.21.15.121:0/2559479729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c500a9830 0x7f7c5014da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:16.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c467fc700 1 -- 172.21.15.121:0/2559479729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c50151870 msgr2=0x7f7c50156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:16.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c467fc700 1 --2- 172.21.15.121:0/2559479729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c50151870 0x7f7c50156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:16.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c467fc700 1 -- 172.21.15.121:0/2559479729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c500a9830 msgr2=0x7f7c5014da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:16.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c467fc700 1 --2- 172.21.15.121:0/2559479729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c500a9830 0x7f7c5014da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:16.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c467fc700 1 -- 172.21.15.121:0/2559479729 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c38009370 con 0x7f7c500c30e0 2023-11-15T03:02:16.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c467fc700 1 --2- 172.21.15.121:0/2559479729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c500c30e0 0x7f7c501583a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f7c38009f50 tx=0x7f7c38017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:16.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c477fe700 1 -- 172.21.15.121:0/2559479729 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3800bea0 con 0x7f7c500c30e0 2023-11-15T03:02:16.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c477fe700 1 -- 172.21.15.121:0/2559479729 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3800b8d0 con 0x7f7c500c30e0 2023-11-15T03:02:16.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c4ffff700 1 -- 172.21.15.121:0/2559479729 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c50154440 con 0x7f7c500c30e0 2023-11-15T03:02:16.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c477fe700 1 -- 172.21.15.121:0/2559479729 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3801b690 con 0x7f7c500c30e0 2023-11-15T03:02:16.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.526+0000 7f7c4ffff700 1 -- 172.21.15.121:0/2559479729 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c50158c10 con 0x7f7c500c30e0 2023-11-15T03:02:16.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.530+0000 7f7c4ffff700 1 -- 172.21.15.121:0/2559479729 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c50154c00 con 0x7f7c500c30e0 2023-11-15T03:02:16.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.530+0000 7f7c477fe700 1 -- 172.21.15.121:0/2559479729 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7c3801ba00 con 0x7f7c500c30e0 2023-11-15T03:02:16.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.530+0000 7f7c477fe700 1 --2- 172.21.15.121:0/2559479729 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c3005a510 0x7f7c3005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:16.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.530+0000 7f7c477fe700 1 -- 172.21.15.121:0/2559479729 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f7c38075b40 con 0x7f7c500c30e0 2023-11-15T03:02:16.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.530+0000 7f7c4effd700 1 --2- 172.21.15.121:0/2559479729 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c3005a510 0x7f7c3005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:16.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.534+0000 7f7c4effd700 1 --2- 172.21.15.121:0/2559479729 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c3005a510 0x7f7c3005c9d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f7c40003900 tx=0x7f7c40007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:16.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.538+0000 7f7c477fe700 1 -- 172.21.15.121:0/2559479729 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7c38049630 con 0x7f7c500c30e0 2023-11-15T03:02:16.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.754+0000 7f7c4ffff700 1 -- 172.21.15.121:0/2559479729 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7c5006bcf0 con 0x7f7c500c30e0 2023-11-15T03:02:16.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.754+0000 7f7c477fe700 1 -- 172.21.15.121:0/2559479729 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7c3801bcb0 con 0x7f7c500c30e0 2023-11-15T03:02:16.762 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:16.763 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:02:16.763 INFO:teuthology.orchestra.run.smithi121.stdout: "auth_dump": [ 2023-11-15T03:02:16.763 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:16.763 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.a", 2023-11-15T03:02:16.764 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1Rlzy3zOhAAeJ3COoWQP/Kw+P/0AwHbwA==", 2023-11-15T03:02:16.764 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:02:16.764 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:02:16.764 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T03:02:16.764 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T03:02:16.765 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:02:16.765 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.765 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:02:16.765 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:16.765 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.b", 2023-11-15T03:02:16.766 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlUglaERAAC6WUS9ftCfr+KbPpiEIyqw==", 2023-11-15T03:02:16.766 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:02:16.766 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:02:16.766 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T03:02:16.767 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T03:02:16.767 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:02:16.767 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.768 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:02:16.768 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:16.768 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.c", 2023-11-15T03:02:16.769 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RldFcvCRAATIouBNyHGip5hFrH+z4v5A==", 2023-11-15T03:02:16.769 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:02:16.769 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:02:16.769 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T03:02:16.770 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T03:02:16.770 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:02:16.770 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.771 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:02:16.771 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:16.771 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.d", 2023-11-15T03:02:16.772 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1Rlg5iZGxAADwdL2RrtU0kDeb9EmNJU6A==", 2023-11-15T03:02:16.773 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:02:16.773 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:02:16.773 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T03:02:16.773 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T03:02:16.773 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:02:16.774 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.774 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:02:16.774 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:16.774 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.0", 2023-11-15T03:02:16.774 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAzM1RldPH9CBAAUCF/a47aFbbNsWV1r+u3Tw==", 2023-11-15T03:02:16.775 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:02:16.776 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:02:16.777 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:02:16.777 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:02:16.777 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.777 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:02:16.778 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:16.778 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.1", 2023-11-15T03:02:16.778 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQA2M1RltoxwMhAAZN3bgOjSaoK3ZxE5ezZ3IA==", 2023-11-15T03:02:16.778 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:02:16.781 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:02:16.782 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:02:16.782 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:02:16.782 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.782 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:02:16.783 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:16.783 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.2", 2023-11-15T03:02:16.783 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQA6M1RlgltjHxAAx/m5gm+qs8LiEFyZL+uLzQ==", 2023-11-15T03:02:16.783 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:02:16.783 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:02:16.784 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:02:16.784 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:02:16.784 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.784 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:02:16.784 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:16.785 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.3", 2023-11-15T03:02:16.785 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQA+M1RlhtQcDRAA+g11PmSNKazpk+VoK17EEQ==", 2023-11-15T03:02:16.785 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:02:16.786 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:02:16.787 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:02:16.787 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:02:16.787 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.787 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:02:16.787 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:16.788 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.4", 2023-11-15T03:02:16.788 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBFM1RlJ7hnBBAAjskKfD63N/61T4GSPqOiRg==", 2023-11-15T03:02:16.788 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:02:16.788 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:02:16.788 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:02:16.789 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:02:16.789 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.790 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:02:16.790 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:16.792 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.5", 2023-11-15T03:02:16.792 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBIM1RlI1HPLRAAgVbiNq1tmgK6PaWP+c2JFw==", 2023-11-15T03:02:16.792 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:02:16.792 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:02:16.792 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:02:16.793 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:02:16.793 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.793 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:02:16.793 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:16.793 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.6", 2023-11-15T03:02:16.794 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBMM1RlPDshDRAAVgIqxklHSQcFb5sM6GUfyQ==", 2023-11-15T03:02:16.794 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:02:16.794 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:02:16.794 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:02:16.794 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:02:16.795 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.795 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:02:16.795 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:16.795 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.7", 2023-11-15T03:02:16.795 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBPM1RlN8IuJxAA2rN2848kaB2eOM6zDb+eJg==", 2023-11-15T03:02:16.796 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:02:16.796 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:02:16.796 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:02:16.797 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:02:16.797 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.797 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:02:16.797 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:16.797 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.0", 2023-11-15T03:02:16.798 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlfHPEIRAAJ4YV46zzBQ7v7y0kD4P2ZA==", 2023-11-15T03:02:16.798 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:02:16.798 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:02:16.798 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow r", 2023-11-15T03:02:16.798 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-15T03:02:16.799 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.799 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:02:16.799 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:16.799 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.1", 2023-11-15T03:02:16.799 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlibaULBAA+y1EyhVOt+wXSk2uRajJ3g==", 2023-11-15T03:02:16.800 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:02:16.800 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:02:16.800 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2023-11-15T03:02:16.801 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2023-11-15T03:02:16.801 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2023-11-15T03:02:16.801 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.802 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:02:16.802 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:16.802 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.2", 2023-11-15T03:02:16.802 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlHHbvJhAAX5TE6UvcpBeJPSCZkfqD8w==", 2023-11-15T03:02:16.803 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:02:16.803 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:02:16.803 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2023-11-15T03:02:16.803 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2023-11-15T03:02:16.803 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2023-11-15T03:02:16.804 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.804 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:02:16.804 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:16.804 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.3", 2023-11-15T03:02:16.804 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlQtQXMhAAGAvtnvJLSoUKuEiZDsSxww==", 2023-11-15T03:02:16.806 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:02:16.806 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:02:16.806 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2023-11-15T03:02:16.807 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2023-11-15T03:02:16.807 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2023-11-15T03:02:16.807 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.807 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:02:16.807 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:16.808 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.admin", 2023-11-15T03:02:16.810 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1Rl67qqEhAAI48kZ9D7T807bTVoUmwajg==", 2023-11-15T03:02:16.810 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:02:16.810 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2023-11-15T03:02:16.810 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T03:02:16.811 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T03:02:16.811 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:02:16.811 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.811 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:02:16.812 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:16.812 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mgr.x", 2023-11-15T03:02:16.812 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1RlJQFPNRAAy7E5mrn8Xh98339fWSkX9Q==", 2023-11-15T03:02:16.813 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:02:16.813 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2023-11-15T03:02:16.813 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile mgr", 2023-11-15T03:02:16.813 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:02:16.813 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.814 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:02:16.814 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:16.814 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mgr.y", 2023-11-15T03:02:16.814 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1RldQJlLhAAbc46liEP03yBNeVhZCb7IQ==", 2023-11-15T03:02:16.814 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:02:16.815 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2023-11-15T03:02:16.815 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile mgr", 2023-11-15T03:02:16.816 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:02:16.816 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.816 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:16.816 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2023-11-15T03:02:16.816 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:02:16.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.758+0000 7f7c4ffff700 1 -- 172.21.15.121:0/2559479729 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c3005a510 msgr2=0x7f7c3005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:16.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.758+0000 7f7c4ffff700 1 --2- 172.21.15.121:0/2559479729 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c3005a510 0x7f7c3005c9d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f7c40003900 tx=0x7f7c40007d40).stop 2023-11-15T03:02:16.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.758+0000 7f7c4ffff700 1 -- 172.21.15.121:0/2559479729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c500c30e0 msgr2=0x7f7c501583a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:16.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.758+0000 7f7c4ffff700 1 --2- 172.21.15.121:0/2559479729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c500c30e0 0x7f7c501583a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f7c38009f50 tx=0x7f7c38017000).stop 2023-11-15T03:02:16.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.758+0000 7f7c4ffff700 1 -- 172.21.15.121:0/2559479729 shutdown_connections 2023-11-15T03:02:16.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.758+0000 7f7c4ffff700 1 --2- 172.21.15.121:0/2559479729 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c3005a510 0x7f7c3005c9d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:16.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.758+0000 7f7c4ffff700 1 --2- 172.21.15.121:0/2559479729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c50151870 0x7f7c50156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:16.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.758+0000 7f7c4ffff700 1 --2- 172.21.15.121:0/2559479729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c500c30e0 0x7f7c501583a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:16.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.758+0000 7f7c4ffff700 1 --2- 172.21.15.121:0/2559479729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c500a9830 0x7f7c5014da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:16.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.758+0000 7f7c4ffff700 1 -- 172.21.15.121:0/2559479729 >> 172.21.15.121:0/2559479729 conn(0x7f7c50062190 msgr2=0x7f7c50067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:16.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.758+0000 7f7c4ffff700 1 -- 172.21.15.121:0/2559479729 shutdown_connections 2023-11-15T03:02:16.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.758+0000 7f7c4ffff700 1 -- 172.21.15.121:0/2559479729 wait complete. 2023-11-15T03:02:16.821 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:02:16.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.974+0000 7f5335369700 1 Processor -- start 2023-11-15T03:02:16.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.974+0000 7f5335369700 1 -- start start 2023-11-15T03:02:16.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.978+0000 7f5335369700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5330150870 0x7f53300c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:16.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.978+0000 7f5335369700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53300c6350 con 0x7f5330153d20 2023-11-15T03:02:16.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.978+0000 7f5335369700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53301541b0 con 0x7f5330151360 2023-11-15T03:02:16.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.978+0000 7f5335369700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5330154490 con 0x7f5330150870 2023-11-15T03:02:16.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.978+0000 7f532effd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5330150870 0x7f53300c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:16.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.978+0000 7f532e7fc700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f5330153d20 0x7f5330150c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:55018/0 (socket says 172.21.15.121:55018) 2023-11-15T03:02:16.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.978+0000 7f532e7fc700 1 -- 172.21.15.121:0/3049243020 learned_addr learned my addr 172.21.15.121:0/3049243020 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:16.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.982+0000 7f532dffb700 1 -- 172.21.15.121:0/3049243020 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941748395 0 0) 0x7f53300c6350 con 0x7f5330153d20 2023-11-15T03:02:16.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.982+0000 7f532dffb700 1 -- 172.21.15.121:0/3049243020 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5318004500 con 0x7f5330153d20 2023-11-15T03:02:16.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.982+0000 7f532dffb700 1 -- 172.21.15.121:0/3049243020 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1838967954 0 0) 0x7f53301541b0 con 0x7f5330151360 2023-11-15T03:02:16.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.982+0000 7f532dffb700 1 -- 172.21.15.121:0/3049243020 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53300c6350 con 0x7f5330151360 2023-11-15T03:02:16.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.982+0000 7f532effd700 1 -- 172.21.15.121:0/3049243020 >> v1:172.21.15.190:6789/0 conn(0x7f5330151360 legacy=0x7f53300c1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:16.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.982+0000 7f532effd700 1 -- 172.21.15.121:0/3049243020 >> v1:172.21.15.121:6789/0 conn(0x7f5330153d20 legacy=0x7f5330150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:16.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.982+0000 7f532effd700 1 -- 172.21.15.121:0/3049243020 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5330154770 con 0x7f5330150870 2023-11-15T03:02:16.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.982+0000 7f532effd700 1 --2- 172.21.15.121:0/3049243020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5330150870 0x7f53300c3190 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f5320009760 tx=0x7f5320009060).ready entity=mon.0 client_cookie=6bc990a8cd4bc103 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:16.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.982+0000 7f532dffb700 1 -- 172.21.15.121:0/3049243020 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532000d4a0 con 0x7f5330150870 2023-11-15T03:02:16.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.982+0000 7f532dffb700 1 -- 172.21.15.121:0/3049243020 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532000d600 con 0x7f5330150870 2023-11-15T03:02:16.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.982+0000 7f532dffb700 1 -- 172.21.15.121:0/3049243020 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532000d980 con 0x7f5330150870 2023-11-15T03:02:16.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.982+0000 7f5335369700 1 -- 172.21.15.121:0/3049243020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5330150870 msgr2=0x7f53300c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:16.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.982+0000 7f5335369700 1 --2- 172.21.15.121:0/3049243020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5330150870 0x7f53300c3190 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f5320009760 tx=0x7f5320009060).stop 2023-11-15T03:02:16.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.982+0000 7f5335369700 1 -- 172.21.15.121:0/3049243020 shutdown_connections 2023-11-15T03:02:16.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.982+0000 7f5335369700 1 --2- 172.21.15.121:0/3049243020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5330150870 0x7f53300c3190 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:16.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.982+0000 7f5335369700 1 -- 172.21.15.121:0/3049243020 >> 172.21.15.121:0/3049243020 conn(0x7f5330062190 msgr2=0x7f53300625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:16.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.982+0000 7f5335369700 1 -- 172.21.15.121:0/3049243020 shutdown_connections 2023-11-15T03:02:16.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.982+0000 7f5335369700 1 -- 172.21.15.121:0/3049243020 wait complete. 2023-11-15T03:02:16.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.986+0000 7f5335369700 1 Processor -- start 2023-11-15T03:02:16.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.986+0000 7f5335369700 1 -- start start 2023-11-15T03:02:16.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.986+0000 7f5335369700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5330150870 0x7f5330142190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:16.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.986+0000 7f5335369700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5330151360 0x7f5330144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:16.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.986+0000 7f5335369700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5330148d80 0x7f5330144780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:16.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.986+0000 7f5335369700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5330143e00 con 0x7f5330150870 2023-11-15T03:02:16.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.986+0000 7f5335369700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5330149280 con 0x7f5330151360 2023-11-15T03:02:16.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.986+0000 7f5335369700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5330149560 con 0x7f5330148d80 2023-11-15T03:02:16.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.990+0000 7f532effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5330150870 0x7f5330142190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:16.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.990+0000 7f532effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5330150870 0x7f5330142190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54580/0 (socket says 172.21.15.121:54580) 2023-11-15T03:02:16.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.990+0000 7f532effd700 1 -- 172.21.15.121:0/572084498 learned_addr learned my addr 172.21.15.121:0/572084498 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:16.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.990+0000 7f532e7fc700 1 --2- 172.21.15.121:0/572084498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5330151360 0x7f5330144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:16.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.990+0000 7f532f7fe700 1 --2- 172.21.15.121:0/572084498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5330148d80 0x7f5330144780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:16.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.990+0000 7f532effd700 1 -- 172.21.15.121:0/572084498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5330148d80 msgr2=0x7f5330144780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:17.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.990+0000 7f532effd700 1 --2- 172.21.15.121:0/572084498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5330148d80 0x7f5330144780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:17.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.990+0000 7f532effd700 1 -- 172.21.15.121:0/572084498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5330151360 msgr2=0x7f5330144290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:17.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.990+0000 7f532effd700 1 --2- 172.21.15.121:0/572084498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5330151360 0x7f5330144290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:17.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.990+0000 7f532effd700 1 -- 172.21.15.121:0/572084498 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53200092a0 con 0x7f5330150870 2023-11-15T03:02:17.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.990+0000 7f532f7fe700 1 --2- 172.21.15.121:0/572084498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5330148d80 0x7f5330144780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:17.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.990+0000 7f532effd700 1 --2- 172.21.15.121:0/572084498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5330150870 0x7f5330142190 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f5330154e00 tx=0x7f5320016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:17.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.990+0000 7f5316ffd700 1 -- 172.21.15.121:0/572084498 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532000dcf0 con 0x7f5330150870 2023-11-15T03:02:17.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.990+0000 7f5316ffd700 1 -- 172.21.15.121:0/572084498 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532000de50 con 0x7f5330150870 2023-11-15T03:02:17.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.990+0000 7f5335369700 1 -- 172.21.15.121:0/572084498 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5330143a90 con 0x7f5330150870 2023-11-15T03:02:17.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.990+0000 7f5316ffd700 1 -- 172.21.15.121:0/572084498 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532001d430 con 0x7f5330150870 2023-11-15T03:02:17.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.990+0000 7f5335369700 1 -- 172.21.15.121:0/572084498 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5330142680 con 0x7f5330150870 2023-11-15T03:02:17.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.990+0000 7f5335369700 1 -- 172.21.15.121:0/572084498 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53301438f0 con 0x7f5330150870 2023-11-15T03:02:17.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.990+0000 7f5316ffd700 1 -- 172.21.15.121:0/572084498 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f532001d7a0 con 0x7f5330150870 2023-11-15T03:02:17.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.994+0000 7f5316ffd700 1 --2- 172.21.15.121:0/572084498 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f531c05a510 0x7f531c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:17.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.994+0000 7f5316ffd700 1 -- 172.21.15.121:0/572084498 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f5320075cb0 con 0x7f5330150870 2023-11-15T03:02:17.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.994+0000 7f532e7fc700 1 --2- 172.21.15.121:0/572084498 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f531c05a510 0x7f531c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:17.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.994+0000 7f532e7fc700 1 --2- 172.21.15.121:0/572084498 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f531c05a510 0x7f531c05c9d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f53300c1d40 tx=0x7f5324009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:17.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:16.998+0000 7f5316ffd700 1 -- 172.21.15.121:0/572084498 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f53301438f0 con 0x7f5330150870 2023-11-15T03:02:17.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.166+0000 7f5335369700 1 -- 172.21.15.121:0/572084498 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53301438f0 con 0x7f5330150870 2023-11-15T03:02:17.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.166+0000 7f5316ffd700 1 -- 172.21.15.121:0/572084498 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+13761 (secure 0 0 0) 0x7f53301438f0 con 0x7f5330150870 2023-11-15T03:02:17.175 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:17.175 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":73,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:02:16.322248+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:02:17.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.170+0000 7f5335369700 1 -- 172.21.15.121:0/572084498 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f531c05a510 msgr2=0x7f531c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:17.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.170+0000 7f5335369700 1 --2- 172.21.15.121:0/572084498 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f531c05a510 0x7f531c05c9d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f53300c1d40 tx=0x7f5324009040).stop 2023-11-15T03:02:17.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.170+0000 7f5335369700 1 -- 172.21.15.121:0/572084498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5330150870 msgr2=0x7f5330142190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:17.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.170+0000 7f5335369700 1 --2- 172.21.15.121:0/572084498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5330150870 0x7f5330142190 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f5330154e00 tx=0x7f5320016000).stop 2023-11-15T03:02:17.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.170+0000 7f5335369700 1 -- 172.21.15.121:0/572084498 shutdown_connections 2023-11-15T03:02:17.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.170+0000 7f5335369700 1 --2- 172.21.15.121:0/572084498 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f531c05a510 0x7f531c05c9d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:17.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.170+0000 7f5335369700 1 --2- 172.21.15.121:0/572084498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5330148d80 0x7f5330144780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:17.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.170+0000 7f5335369700 1 --2- 172.21.15.121:0/572084498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5330151360 0x7f5330144290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:17.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.170+0000 7f5335369700 1 --2- 172.21.15.121:0/572084498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5330150870 0x7f5330142190 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:17.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.170+0000 7f5335369700 1 -- 172.21.15.121:0/572084498 >> 172.21.15.121:0/572084498 conn(0x7f5330062190 msgr2=0x7f5330142a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:17.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.170+0000 7f5335369700 1 -- 172.21.15.121:0/572084498 shutdown_connections 2023-11-15T03:02:17.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.170+0000 7f5335369700 1 -- 172.21.15.121:0/572084498 wait complete. 2023-11-15T03:02:17.200 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-15T03:02:17.201 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T03:02:17.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.354+0000 7f039e6e4700 1 Processor -- start 2023-11-15T03:02:17.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.354+0000 7f039e6e4700 1 -- start start 2023-11-15T03:02:17.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.358+0000 7f039e6e4700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f03980c3650 0x7f039815c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:17.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.358+0000 7f039e6e4700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03981528b0 con 0x7f03980a7070 2023-11-15T03:02:17.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.358+0000 7f039e6e4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03980a74d0 con 0x7f0398151af0 2023-11-15T03:02:17.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.358+0000 7f039e6e4700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0398066d80 con 0x7f03980c3650 2023-11-15T03:02:17.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.358+0000 7f039dee3700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f03980c3650 0x7f039815c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:17.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.358+0000 7f039dee3700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f03980c3650 0x7f039815c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54582/0 (socket says 172.21.15.121:54582) 2023-11-15T03:02:17.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.358+0000 7f039dee3700 1 -- 172.21.15.121:0/817122491 learned_addr learned my addr 172.21.15.121:0/817122491 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:17.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.358+0000 7f039d6e2700 1 --1- 172.21.15.121:0/817122491 >> v1:172.21.15.190:6789/0 conn(0x7f0398151af0 0x7f03980a5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:39236/0 (socket says 172.21.15.121:39236) 2023-11-15T03:02:17.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.358+0000 7f039dee3700 1 -- 172.21.15.121:0/817122491 >> v1:172.21.15.190:6789/0 conn(0x7f0398151af0 legacy=0x7f03980a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:17.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.358+0000 7f039dee3700 1 -- 172.21.15.121:0/817122491 >> v1:172.21.15.121:6789/0 conn(0x7f03980a7070 legacy=0x7f03980c2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:17.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.358+0000 7f039dee3700 1 -- 172.21.15.121:0/817122491 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0398151ff0 con 0x7f03980c3650 2023-11-15T03:02:17.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.358+0000 7f039dee3700 1 --2- 172.21.15.121:0/817122491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03980c3650 0x7f039815c110 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f0390009f50 tx=0x7f0390009840).ready entity=mon.0 client_cookie=f3e09a676df364b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:17.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.358+0000 7f038ffff700 1 -- 172.21.15.121:0/817122491 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039000e070 con 0x7f03980c3650 2023-11-15T03:02:17.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.358+0000 7f038ffff700 1 -- 172.21.15.121:0/817122491 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039000ba70 con 0x7f03980c3650 2023-11-15T03:02:17.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.358+0000 7f038ffff700 1 -- 172.21.15.121:0/817122491 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039000bdf0 con 0x7f03980c3650 2023-11-15T03:02:17.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.362+0000 7f039e6e4700 1 -- 172.21.15.121:0/817122491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03980c3650 msgr2=0x7f039815c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:17.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.362+0000 7f039e6e4700 1 --2- 172.21.15.121:0/817122491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03980c3650 0x7f039815c110 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f0390009f50 tx=0x7f0390009840).stop 2023-11-15T03:02:17.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.362+0000 7f039e6e4700 1 -- 172.21.15.121:0/817122491 shutdown_connections 2023-11-15T03:02:17.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.362+0000 7f039e6e4700 1 --2- 172.21.15.121:0/817122491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03980c3650 0x7f039815c110 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:17.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.362+0000 7f039e6e4700 1 -- 172.21.15.121:0/817122491 >> 172.21.15.121:0/817122491 conn(0x7f0398067a60 msgr2=0x7f0398067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:17.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.362+0000 7f039e6e4700 1 -- 172.21.15.121:0/817122491 shutdown_connections 2023-11-15T03:02:17.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.362+0000 7f039e6e4700 1 -- 172.21.15.121:0/817122491 wait complete. 2023-11-15T03:02:17.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f039e6e4700 1 Processor -- start 2023-11-15T03:02:17.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f039e6e4700 1 -- start start 2023-11-15T03:02:17.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f039e6e4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03980a7070 0x7f039814e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:17.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f039e6e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03980c3650 0x7f03980c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:17.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f039e6e4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03981641d0 0x7f0398164650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:17.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f039e6e4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0398158510 con 0x7f03980a7070 2023-11-15T03:02:17.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f039e6e4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0398158720 con 0x7f03981641d0 2023-11-15T03:02:17.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f039e6e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0398158a00 con 0x7f03980c3650 2023-11-15T03:02:17.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f039d6e2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03980a7070 0x7f039814e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:17.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f039d6e2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03980a7070 0x7f039814e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54592/0 (socket says 172.21.15.121:54592) 2023-11-15T03:02:17.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f039d6e2700 1 -- 172.21.15.121:0/1215300403 learned_addr learned my addr 172.21.15.121:0/1215300403 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:17.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f039d6e2700 1 -- 172.21.15.121:0/1215300403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03980c3650 msgr2=0x7f03980c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:17.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f039d6e2700 1 --2- 172.21.15.121:0/1215300403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03980c3650 0x7f03980c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:02:17.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f039d6e2700 1 -- 172.21.15.121:0/1215300403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03981641d0 msgr2=0x7f0398164650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:17.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f039d6e2700 1 --2- 172.21.15.121:0/1215300403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03981641d0 0x7f0398164650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:02:17.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f039d6e2700 1 -- 172.21.15.121:0/1215300403 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0390009ae0 con 0x7f03980a7070 2023-11-15T03:02:17.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f039d6e2700 1 --2- 172.21.15.121:0/1215300403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03980a7070 0x7f039814e0f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f0394001eb0 tx=0x7f0394007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:17.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f038d7fa700 1 -- 172.21.15.121:0/1215300403 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03940072e0 con 0x7f03980a7070 2023-11-15T03:02:17.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f038d7fa700 1 -- 172.21.15.121:0/1215300403 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0394007440 con 0x7f03980a7070 2023-11-15T03:02:17.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f038d7fa700 1 -- 172.21.15.121:0/1215300403 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03940058f0 con 0x7f03980a7070 2023-11-15T03:02:17.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f039e6e4700 1 -- 172.21.15.121:0/1215300403 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0398158ce0 con 0x7f03980a7070 2023-11-15T03:02:17.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.366+0000 7f039e6e4700 1 -- 172.21.15.121:0/1215300403 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03980c45e0 con 0x7f03980a7070 2023-11-15T03:02:17.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.370+0000 7f039e6e4700 1 -- 172.21.15.121:0/1215300403 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03980c47d0 con 0x7f03980a7070 2023-11-15T03:02:17.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.370+0000 7f038d7fa700 1 -- 172.21.15.121:0/1215300403 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0394005ce0 con 0x7f03980a7070 2023-11-15T03:02:17.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.374+0000 7f038d7fa700 1 --2- 172.21.15.121:0/1215300403 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f037c05a4e0 0x7f037c05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:17.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.374+0000 7f038d7fa700 1 -- 172.21.15.121:0/1215300403 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f0394073c60 con 0x7f03980a7070 2023-11-15T03:02:17.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.374+0000 7f039cee1700 1 --2- 172.21.15.121:0/1215300403 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f037c05a4e0 0x7f037c05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:17.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.374+0000 7f039cee1700 1 --2- 172.21.15.121:0/1215300403 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f037c05a4e0 0x7f037c05c9a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f039814f010 tx=0x7f03880079c0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:17.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.378+0000 7f038d7fa700 1 -- 172.21.15.121:0/1215300403 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f03940476d0 con 0x7f03980a7070 2023-11-15T03:02:17.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.546+0000 7f039e6e4700 1 -- 172.21.15.121:0/1215300403 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f039806bcf0 con 0x7f03980a7070 2023-11-15T03:02:17.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.546+0000 7f038d7fa700 1 -- 172.21.15.121:0/1215300403 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v73) v1 ==== 110+0+10 (secure 0 0 0) 0x7f039404acf0 con 0x7f03980a7070 2023-11-15T03:02:17.554 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T03:02:17.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.550+0000 7f039e6e4700 1 -- 172.21.15.121:0/1215300403 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f037c05a4e0 msgr2=0x7f037c05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:17.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.550+0000 7f039e6e4700 1 --2- 172.21.15.121:0/1215300403 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f037c05a4e0 0x7f037c05c9a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f039814f010 tx=0x7f03880079c0).stop 2023-11-15T03:02:17.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.550+0000 7f039e6e4700 1 -- 172.21.15.121:0/1215300403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03980a7070 msgr2=0x7f039814e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:17.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.550+0000 7f039e6e4700 1 --2- 172.21.15.121:0/1215300403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03980a7070 0x7f039814e0f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f0394001eb0 tx=0x7f0394007df0).stop 2023-11-15T03:02:17.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.550+0000 7f039e6e4700 1 -- 172.21.15.121:0/1215300403 shutdown_connections 2023-11-15T03:02:17.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.550+0000 7f039e6e4700 1 --2- 172.21.15.121:0/1215300403 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f037c05a4e0 0x7f037c05c9a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:17.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.550+0000 7f039e6e4700 1 --2- 172.21.15.121:0/1215300403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03981641d0 0x7f0398164650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:02:17.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.550+0000 7f039e6e4700 1 --2- 172.21.15.121:0/1215300403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03980c3650 0x7f03980c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:02:17.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.550+0000 7f039e6e4700 1 --2- 172.21.15.121:0/1215300403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03980a7070 0x7f039814e0f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:17.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.550+0000 7f039e6e4700 1 -- 172.21.15.121:0/1215300403 >> 172.21.15.121:0/1215300403 conn(0x7f0398067a60 msgr2=0x7f03980c0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:17.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.550+0000 7f039e6e4700 1 -- 172.21.15.121:0/1215300403 shutdown_connections 2023-11-15T03:02:17.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.550+0000 7f039e6e4700 1 -- 172.21.15.121:0/1215300403 wait complete. 2023-11-15T03:02:17.579 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T03:02:17.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.738+0000 7fdc7fc1b700 1 Processor -- start 2023-11-15T03:02:17.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.738+0000 7fdc7fc1b700 1 -- start start 2023-11-15T03:02:17.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.742+0000 7fdc7fc1b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdc78155e00 0x7fdc7814c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:17.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.742+0000 7fdc7fc1b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc780bf220 con 0x7fdc7814be20 2023-11-15T03:02:17.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.742+0000 7fdc7fc1b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc78064bd0 con 0x7fdc780bdc50 2023-11-15T03:02:17.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.742+0000 7fdc7fc1b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7814c6f0 con 0x7fdc78155e00 2023-11-15T03:02:17.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.742+0000 7fdc7d1b6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdc78155e00 0x7fdc7814c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:17.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.742+0000 7fdc7d9b7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fdc7814be20 0x7fdc781556f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:55038/0 (socket says 172.21.15.121:55038) 2023-11-15T03:02:17.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.742+0000 7fdc7d9b7700 1 -- 172.21.15.121:0/3097729614 learned_addr learned my addr 172.21.15.121:0/3097729614 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:17.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.742+0000 7fdc7d1b6700 1 -- 172.21.15.121:0/3097729614 >> v1:172.21.15.190:6789/0 conn(0x7fdc780bdc50 legacy=0x7fdc780be0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:17.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.742+0000 7fdc7d1b6700 1 -- 172.21.15.121:0/3097729614 >> v1:172.21.15.121:6789/0 conn(0x7fdc7814be20 legacy=0x7fdc781556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:17.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.742+0000 7fdc7d1b6700 1 -- 172.21.15.121:0/3097729614 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc780be980 con 0x7fdc78155e00 2023-11-15T03:02:17.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.742+0000 7fdc7d1b6700 1 --2- 172.21.15.121:0/3097729614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc78155e00 0x7fdc7814c200 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7fdc68009760 tx=0x7fdc68009060).ready entity=mon.0 client_cookie=bb2f9dd4f98a6bc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:17.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.742+0000 7fdc7c9b5700 1 -- 172.21.15.121:0/3097729614 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc68009120 con 0x7fdc78155e00 2023-11-15T03:02:17.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.742+0000 7fdc7c9b5700 1 -- 172.21.15.121:0/3097729614 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6800b580 con 0x7fdc78155e00 2023-11-15T03:02:17.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.742+0000 7fdc7c9b5700 1 -- 172.21.15.121:0/3097729614 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6800b900 con 0x7fdc78155e00 2023-11-15T03:02:17.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.746+0000 7fdc7fc1b700 1 -- 172.21.15.121:0/3097729614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc78155e00 msgr2=0x7fdc7814c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:17.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.746+0000 7fdc7fc1b700 1 --2- 172.21.15.121:0/3097729614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc78155e00 0x7fdc7814c200 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7fdc68009760 tx=0x7fdc68009060).stop 2023-11-15T03:02:17.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.746+0000 7fdc7fc1b700 1 -- 172.21.15.121:0/3097729614 shutdown_connections 2023-11-15T03:02:17.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.746+0000 7fdc7fc1b700 1 --2- 172.21.15.121:0/3097729614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc78155e00 0x7fdc7814c200 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:17.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.746+0000 7fdc7fc1b700 1 -- 172.21.15.121:0/3097729614 >> 172.21.15.121:0/3097729614 conn(0x7fdc780c9440 msgr2=0x7fdc780c5410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:17.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.746+0000 7fdc7fc1b700 1 -- 172.21.15.121:0/3097729614 shutdown_connections 2023-11-15T03:02:17.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.746+0000 7fdc7fc1b700 1 -- 172.21.15.121:0/3097729614 wait complete. 2023-11-15T03:02:17.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.746+0000 7fdc7fc1b700 1 Processor -- start 2023-11-15T03:02:17.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.746+0000 7fdc7fc1b700 1 -- start start 2023-11-15T03:02:17.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc7fc1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc780bdc50 0x7fdc7814dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:17.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc7fc1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc7814be20 0x7fdc7814e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:17.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc7fc1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc78155e00 0x7fdc78167c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:17.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc7fc1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc780c0690 con 0x7fdc780bdc50 2023-11-15T03:02:17.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc7fc1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc780c0970 con 0x7fdc7814be20 2023-11-15T03:02:17.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc7fc1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc780c0c50 con 0x7fdc78155e00 2023-11-15T03:02:17.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc7d9b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc780bdc50 0x7fdc7814dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:17.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc7d9b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc780bdc50 0x7fdc7814dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54600/0 (socket says 172.21.15.121:54600) 2023-11-15T03:02:17.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc7d9b7700 1 -- 172.21.15.121:0/3276918276 learned_addr learned my addr 172.21.15.121:0/3276918276 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:17.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc7d1b6700 1 --2- 172.21.15.121:0/3276918276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc7814be20 0x7fdc7814e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:17.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc7e1b8700 1 --2- 172.21.15.121:0/3276918276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc78155e00 0x7fdc78167c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:17.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc7d9b7700 1 -- 172.21.15.121:0/3276918276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc78155e00 msgr2=0x7fdc78167c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:17.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc7d9b7700 1 --2- 172.21.15.121:0/3276918276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc78155e00 0x7fdc78167c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:17.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc7d9b7700 1 -- 172.21.15.121:0/3276918276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc7814be20 msgr2=0x7fdc7814e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:17.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc7d9b7700 1 --2- 172.21.15.121:0/3276918276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc7814be20 0x7fdc7814e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:17.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc7d9b7700 1 -- 172.21.15.121:0/3276918276 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc680092a0 con 0x7fdc780bdc50 2023-11-15T03:02:17.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc7d9b7700 1 --2- 172.21.15.121:0/3276918276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc780bdc50 0x7fdc7814dca0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7fdc7400c4c0 tx=0x7fdc74007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:17.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc6dffb700 1 -- 172.21.15.121:0/3276918276 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc740072e0 con 0x7fdc780bdc50 2023-11-15T03:02:17.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc6dffb700 1 -- 172.21.15.121:0/3276918276 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc74001e90 con 0x7fdc780bdc50 2023-11-15T03:02:17.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc7fc1b700 1 -- 172.21.15.121:0/3276918276 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc780c0f30 con 0x7fdc780bdc50 2023-11-15T03:02:17.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc6dffb700 1 -- 172.21.15.121:0/3276918276 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc74005670 con 0x7fdc780bdc50 2023-11-15T03:02:17.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.750+0000 7fdc7fc1b700 1 -- 172.21.15.121:0/3276918276 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc780aba60 con 0x7fdc780bdc50 2023-11-15T03:02:17.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.754+0000 7fdc7fc1b700 1 -- 172.21.15.121:0/3276918276 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc780c5ff0 con 0x7fdc780bdc50 2023-11-15T03:02:17.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.754+0000 7fdc6dffb700 1 -- 172.21.15.121:0/3276918276 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fdc74005a60 con 0x7fdc780bdc50 2023-11-15T03:02:17.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.754+0000 7fdc6dffb700 1 --2- 172.21.15.121:0/3276918276 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdc5c05a540 0x7fdc5c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:17.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.754+0000 7fdc6dffb700 1 -- 172.21.15.121:0/3276918276 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fdc74073ca0 con 0x7fdc780bdc50 2023-11-15T03:02:17.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.754+0000 7fdc7d1b6700 1 --2- 172.21.15.121:0/3276918276 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdc5c05a540 0x7fdc5c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:17.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.758+0000 7fdc7d1b6700 1 --2- 172.21.15.121:0/3276918276 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdc5c05a540 0x7fdc5c05ca00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fdc78156200 tx=0x7fdc68017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:17.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.758+0000 7fdc6dffb700 1 -- 172.21.15.121:0/3276918276 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdc780c5ff0 con 0x7fdc780bdc50 2023-11-15T03:02:17.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.918+0000 7fdc7fc1b700 1 -- 172.21.15.121:0/3276918276 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdc780c5ff0 con 0x7fdc780bdc50 2023-11-15T03:02:17.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.918+0000 7fdc6dffb700 1 -- 172.21.15.121:0/3276918276 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v73) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdc780c5ff0 con 0x7fdc780bdc50 2023-11-15T03:02:17.927 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T03:02:17.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.922+0000 7fdc7fc1b700 1 -- 172.21.15.121:0/3276918276 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdc5c05a540 msgr2=0x7fdc5c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:17.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.922+0000 7fdc7fc1b700 1 --2- 172.21.15.121:0/3276918276 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdc5c05a540 0x7fdc5c05ca00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fdc78156200 tx=0x7fdc68017000).stop 2023-11-15T03:02:17.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.922+0000 7fdc7fc1b700 1 -- 172.21.15.121:0/3276918276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc780bdc50 msgr2=0x7fdc7814dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:17.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.922+0000 7fdc7fc1b700 1 --2- 172.21.15.121:0/3276918276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc780bdc50 0x7fdc7814dca0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7fdc7400c4c0 tx=0x7fdc74007c60).stop 2023-11-15T03:02:17.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.922+0000 7fdc7fc1b700 1 -- 172.21.15.121:0/3276918276 shutdown_connections 2023-11-15T03:02:17.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.922+0000 7fdc7fc1b700 1 --2- 172.21.15.121:0/3276918276 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdc5c05a540 0x7fdc5c05ca00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:17.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.922+0000 7fdc7fc1b700 1 --2- 172.21.15.121:0/3276918276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc78155e00 0x7fdc78167c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:17.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.922+0000 7fdc7fc1b700 1 --2- 172.21.15.121:0/3276918276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc7814be20 0x7fdc7814e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:17.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.922+0000 7fdc7fc1b700 1 --2- 172.21.15.121:0/3276918276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc780bdc50 0x7fdc7814dca0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:17.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.922+0000 7fdc7fc1b700 1 -- 172.21.15.121:0/3276918276 >> 172.21.15.121:0/3276918276 conn(0x7fdc780c9440 msgr2=0x7fdc78065060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:17.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.922+0000 7fdc7fc1b700 1 -- 172.21.15.121:0/3276918276 shutdown_connections 2023-11-15T03:02:17.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:17.922+0000 7fdc7fc1b700 1 -- 172.21.15.121:0/3276918276 wait complete. 2023-11-15T03:02:17.950 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-15T03:02:18.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.098+0000 7fb6a1b47700 1 Processor -- start 2023-11-15T03:02:18.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.098+0000 7fb6a1b47700 1 -- start start 2023-11-15T03:02:18.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.098+0000 7fb6a1b47700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb69c151cf0 0x7fb69c0bd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:18.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.098+0000 7fb6a1b47700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb69c0bf000 0x7fb69c0c8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:18.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.098+0000 7fb6a1b47700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb69c0beb70 con 0x7fb69c151910 2023-11-15T03:02:18.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.098+0000 7fb6a1b47700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69c0bdf60 con 0x7fb69c151cf0 2023-11-15T03:02:18.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.098+0000 7fb6a1b47700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69c0be240 con 0x7fb69c0bf000 2023-11-15T03:02:18.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.098+0000 7fb6a0b45700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb69c151910 0x7fb69c0c5c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:55054/0 (socket says 172.21.15.121:55054) 2023-11-15T03:02:18.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.098+0000 7fb6a0b45700 1 -- 172.21.15.121:0/1712252543 learned_addr learned my addr 172.21.15.121:0/1712252543 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:18.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.098+0000 7fb69bfff700 1 --2- 172.21.15.121:0/1712252543 >> v2:172.21.15.121:3300/0 conn(0x7fb69c151cf0 0x7fb69c0bd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:18.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.098+0000 7fb6a1346700 1 --2- 172.21.15.121:0/1712252543 >> v2:172.21.15.190:3300/0 conn(0x7fb69c0bf000 0x7fb69c0c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:18.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.098+0000 7fb69bfff700 1 -- 172.21.15.121:0/1712252543 >> v2:172.21.15.190:3300/0 conn(0x7fb69c0bf000 msgr2=0x7fb69c0c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:18.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.098+0000 7fb69bfff700 1 --2- 172.21.15.121:0/1712252543 >> v2:172.21.15.190:3300/0 conn(0x7fb69c0bf000 0x7fb69c0c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:18.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.098+0000 7fb69bfff700 1 -- 172.21.15.121:0/1712252543 >> v1:172.21.15.121:6789/0 conn(0x7fb69c151910 legacy=0x7fb69c0c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:18.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.098+0000 7fb69bfff700 1 -- 172.21.15.121:0/1712252543 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c0be520 con 0x7fb69c151cf0 2023-11-15T03:02:18.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.102+0000 7fb69bfff700 1 --2- 172.21.15.121:0/1712252543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb69c151cf0 0x7fb69c0bd8d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7fb69c065540 tx=0x7fb688009060).ready entity=mon.0 client_cookie=70389e61e96ba9af server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:18.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.102+0000 7fb69b7fe700 1 -- 172.21.15.121:0/1712252543 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68800b580 con 0x7fb69c151cf0 2023-11-15T03:02:18.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.102+0000 7fb69b7fe700 1 -- 172.21.15.121:0/1712252543 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68800b6e0 con 0x7fb69c151cf0 2023-11-15T03:02:18.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.102+0000 7fb69b7fe700 1 -- 172.21.15.121:0/1712252543 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68800ba60 con 0x7fb69c151cf0 2023-11-15T03:02:18.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.102+0000 7fb6a1b47700 1 -- 172.21.15.121:0/1712252543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb69c151cf0 msgr2=0x7fb69c0bd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:18.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.102+0000 7fb6a1b47700 1 --2- 172.21.15.121:0/1712252543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb69c151cf0 0x7fb69c0bd8d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7fb69c065540 tx=0x7fb688009060).stop 2023-11-15T03:02:18.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.102+0000 7fb6a1b47700 1 -- 172.21.15.121:0/1712252543 shutdown_connections 2023-11-15T03:02:18.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.102+0000 7fb6a1b47700 1 --2- 172.21.15.121:0/1712252543 >> v2:172.21.15.190:3300/0 conn(0x7fb69c0bf000 0x7fb69c0c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:18.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.102+0000 7fb6a1b47700 1 --2- 172.21.15.121:0/1712252543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb69c151cf0 0x7fb69c0bd8d0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:18.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.102+0000 7fb6a1b47700 1 -- 172.21.15.121:0/1712252543 >> 172.21.15.121:0/1712252543 conn(0x7fb69c0643a0 msgr2=0x7fb69c0647b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:18.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.102+0000 7fb6a1b47700 1 -- 172.21.15.121:0/1712252543 shutdown_connections 2023-11-15T03:02:18.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.102+0000 7fb6a1b47700 1 -- 172.21.15.121:0/1712252543 wait complete. 2023-11-15T03:02:18.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb6a1b47700 1 Processor -- start 2023-11-15T03:02:18.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb6a1b47700 1 -- start start 2023-11-15T03:02:18.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb6a1b47700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb69c0bf000 0x7fb69c14b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:18.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb6a1b47700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb69c151910 0x7fb69c165710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:18.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb6a1b47700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb69c151cf0 0x7fb69c165c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:18.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb6a1b47700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c0c06a0 con 0x7fb69c151910 2023-11-15T03:02:18.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb6a1b47700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c0bfa90 con 0x7fb69c151cf0 2023-11-15T03:02:18.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb6a1b47700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c0bfd70 con 0x7fb69c0bf000 2023-11-15T03:02:18.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb69bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb69c151910 0x7fb69c165710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:18.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb69bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb69c151910 0x7fb69c165710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54626/0 (socket says 172.21.15.121:54626) 2023-11-15T03:02:18.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb69bfff700 1 -- 172.21.15.121:0/1910118121 learned_addr learned my addr 172.21.15.121:0/1910118121 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:18.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb69bfff700 1 -- 172.21.15.121:0/1910118121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb69c0bf000 msgr2=0x7fb69c14b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:18.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb69bfff700 1 --2- 172.21.15.121:0/1910118121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb69c0bf000 0x7fb69c14b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:02:18.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb6a1346700 1 --2- 172.21.15.121:0/1910118121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb69c151cf0 0x7fb69c165c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:18.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb69bfff700 1 -- 172.21.15.121:0/1910118121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb69c151cf0 msgr2=0x7fb69c165c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:18.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb69bfff700 1 --2- 172.21.15.121:0/1910118121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb69c151cf0 0x7fb69c165c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:18.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb69bfff700 1 -- 172.21.15.121:0/1910118121 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6880092a0 con 0x7fb69c151910 2023-11-15T03:02:18.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb6a1346700 1 --2- 172.21.15.121:0/1910118121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb69c151cf0 0x7fb69c165c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:02:18.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb69bfff700 1 --2- 172.21.15.121:0/1910118121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb69c151910 0x7fb69c165710 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7fb688011000 tx=0x7fb688017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:18.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb698ff9700 1 -- 172.21.15.121:0/1910118121 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68800ba60 con 0x7fb69c151910 2023-11-15T03:02:18.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb698ff9700 1 -- 172.21.15.121:0/1910118121 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb688000c90 con 0x7fb69c151910 2023-11-15T03:02:18.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb6a1b47700 1 -- 172.21.15.121:0/1910118121 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69c0c0050 con 0x7fb69c151910 2023-11-15T03:02:18.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb698ff9700 1 -- 172.21.15.121:0/1910118121 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68801b660 con 0x7fb69c151910 2023-11-15T03:02:18.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.106+0000 7fb6a1b47700 1 -- 172.21.15.121:0/1910118121 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69c0c0b30 con 0x7fb69c151910 2023-11-15T03:02:18.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.110+0000 7fb6a1b47700 1 -- 172.21.15.121:0/1910118121 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69c06c150 con 0x7fb69c151910 2023-11-15T03:02:18.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.110+0000 7fb698ff9700 1 -- 172.21.15.121:0/1910118121 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb68801ba50 con 0x7fb69c151910 2023-11-15T03:02:18.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.110+0000 7fb698ff9700 1 --2- 172.21.15.121:0/1910118121 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb68005a540 0x7fb68005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:18.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.110+0000 7fb698ff9700 1 -- 172.21.15.121:0/1910118121 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fb6880761c0 con 0x7fb69c151910 2023-11-15T03:02:18.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.110+0000 7fb6a0b45700 1 --2- 172.21.15.121:0/1910118121 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb68005a540 0x7fb68005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:18.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.114+0000 7fb6a0b45700 1 --2- 172.21.15.121:0/1910118121 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb68005a540 0x7fb68005ca00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fb6900010e0 tx=0x7fb690007830).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:18.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.118+0000 7fb698ff9700 1 -- 172.21.15.121:0/1910118121 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb69c06c150 con 0x7fb69c151910 2023-11-15T03:02:18.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.282+0000 7fb6a1b47700 1 -- 172.21.15.121:0/1910118121 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb69c06c150 con 0x7fb69c151910 2023-11-15T03:02:18.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.282+0000 7fb698ff9700 1 -- 172.21.15.121:0/1910118121 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) v1 ==== 84+0+143 (secure 0 0 0) 0x7fb69c06c150 con 0x7fb69c151910 2023-11-15T03:02:18.292 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:18.292 INFO:teuthology.orchestra.run.smithi121.stdout:[ 2023-11-15T03:02:18.292 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:18.292 INFO:teuthology.orchestra.run.smithi121.stdout: "poolnum": 1, 2023-11-15T03:02:18.293 INFO:teuthology.orchestra.run.smithi121.stdout: "poolname": "device_health_metrics" 2023-11-15T03:02:18.293 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:02:18.293 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:02:18.293 INFO:teuthology.orchestra.run.smithi121.stdout: "poolnum": 2, 2023-11-15T03:02:18.293 INFO:teuthology.orchestra.run.smithi121.stdout: "poolname": "rbd" 2023-11-15T03:02:18.294 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:02:18.294 INFO:teuthology.orchestra.run.smithi121.stdout:] 2023-11-15T03:02:18.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.286+0000 7fb6a1b47700 1 -- 172.21.15.121:0/1910118121 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb68005a540 msgr2=0x7fb68005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:18.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.286+0000 7fb6a1b47700 1 --2- 172.21.15.121:0/1910118121 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb68005a540 0x7fb68005ca00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fb6900010e0 tx=0x7fb690007830).stop 2023-11-15T03:02:18.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.286+0000 7fb6a1b47700 1 -- 172.21.15.121:0/1910118121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb69c151910 msgr2=0x7fb69c165710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:18.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.286+0000 7fb6a1b47700 1 --2- 172.21.15.121:0/1910118121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb69c151910 0x7fb69c165710 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7fb688011000 tx=0x7fb688017000).stop 2023-11-15T03:02:18.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.286+0000 7fb6a1b47700 1 -- 172.21.15.121:0/1910118121 shutdown_connections 2023-11-15T03:02:18.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.286+0000 7fb6a1b47700 1 --2- 172.21.15.121:0/1910118121 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb68005a540 0x7fb68005ca00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:18.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.286+0000 7fb6a1b47700 1 --2- 172.21.15.121:0/1910118121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb69c151cf0 0x7fb69c165c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:18.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.286+0000 7fb6a1b47700 1 --2- 172.21.15.121:0/1910118121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb69c151910 0x7fb69c165710 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:18.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.286+0000 7fb6a1b47700 1 --2- 172.21.15.121:0/1910118121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb69c0bf000 0x7fb69c14b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:02:18.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.290+0000 7fb6a1b47700 1 -- 172.21.15.121:0/1910118121 >> 172.21.15.121:0/1910118121 conn(0x7fb69c0643a0 msgr2=0x7fb69c0c83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:18.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.290+0000 7fb6a1b47700 1 -- 172.21.15.121:0/1910118121 shutdown_connections 2023-11-15T03:02:18.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.290+0000 7fb6a1b47700 1 -- 172.21.15.121:0/1910118121 wait complete. 2023-11-15T03:02:18.316 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-15T03:02:18.317 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-15T03:02:18.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.474+0000 7f4d2288f700 1 Processor -- start 2023-11-15T03:02:18.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.474+0000 7f4d2288f700 1 -- start start 2023-11-15T03:02:18.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.478+0000 7f4d2288f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4d1c0a8e60 0x7f4d1c158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:18.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.478+0000 7f4d2288f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1c0bb910 con 0x7f4d1c157d90 2023-11-15T03:02:18.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.478+0000 7f4d2288f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1c066690 con 0x7f4d1c0669b0 2023-11-15T03:02:18.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.478+0000 7f4d2288f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1c0c1fb0 con 0x7f4d1c0a8e60 2023-11-15T03:02:18.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.478+0000 7f4d2188d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4d1c157d90 0x7f4d1c0c1760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:55070/0 (socket says 172.21.15.121:55070) 2023-11-15T03:02:18.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.478+0000 7f4d2188d700 1 -- 172.21.15.121:0/2035869750 learned_addr learned my addr 172.21.15.121:0/2035869750 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:18.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.478+0000 7f4d2208e700 1 --2- 172.21.15.121:0/2035869750 >> v2:172.21.15.190:3300/0 conn(0x7f4d1c0a8e60 0x7f4d1c158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:18.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.478+0000 7f4d2088b700 1 -- 172.21.15.121:0/2035869750 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276721462 0 0) 0x7f4d1c0bb910 con 0x7f4d1c157d90 2023-11-15T03:02:18.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.478+0000 7f4d2088b700 1 -- 172.21.15.121:0/2035869750 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d080033f0 con 0x7f4d1c157d90 2023-11-15T03:02:18.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.478+0000 7f4d2208e700 1 -- 172.21.15.121:0/2035869750 >> v1:172.21.15.190:6789/0 conn(0x7f4d1c0669b0 legacy=0x7f4d1c0a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:18.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.478+0000 7f4d2208e700 1 -- 172.21.15.121:0/2035869750 >> v1:172.21.15.121:6789/0 conn(0x7f4d1c157d90 legacy=0x7f4d1c0c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:18.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.478+0000 7f4d2208e700 1 -- 172.21.15.121:0/2035869750 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c15bde0 con 0x7f4d1c0a8e60 2023-11-15T03:02:18.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.478+0000 7f4d2208e700 1 --2- 172.21.15.121:0/2035869750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c0a8e60 0x7f4d1c158170 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f4d14009e60 tx=0x7f4d14009780).ready entity=mon.1 client_cookie=5eae580b1574bd8b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:18.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.478+0000 7f4d2088b700 1 -- 172.21.15.121:0/2035869750 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1400e070 con 0x7f4d1c0a8e60 2023-11-15T03:02:18.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.478+0000 7f4d2088b700 1 -- 172.21.15.121:0/2035869750 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1400bb60 con 0x7f4d1c0a8e60 2023-11-15T03:02:18.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.478+0000 7f4d2088b700 1 -- 172.21.15.121:0/2035869750 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14012410 con 0x7f4d1c0a8e60 2023-11-15T03:02:18.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.478+0000 7f4d2288f700 1 -- 172.21.15.121:0/2035869750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c0a8e60 msgr2=0x7f4d1c158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:18.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.478+0000 7f4d2288f700 1 --2- 172.21.15.121:0/2035869750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c0a8e60 0x7f4d1c158170 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f4d14009e60 tx=0x7f4d14009780).stop 2023-11-15T03:02:18.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.482+0000 7f4d2288f700 1 -- 172.21.15.121:0/2035869750 shutdown_connections 2023-11-15T03:02:18.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.482+0000 7f4d2288f700 1 --2- 172.21.15.121:0/2035869750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c0a8e60 0x7f4d1c158170 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:18.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.482+0000 7f4d2288f700 1 -- 172.21.15.121:0/2035869750 >> 172.21.15.121:0/2035869750 conn(0x7f4d1c067a60 msgr2=0x7f4d1c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:18.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.482+0000 7f4d2288f700 1 -- 172.21.15.121:0/2035869750 shutdown_connections 2023-11-15T03:02:18.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.482+0000 7f4d2288f700 1 -- 172.21.15.121:0/2035869750 wait complete. 2023-11-15T03:02:18.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.482+0000 7f4d2288f700 1 Processor -- start 2023-11-15T03:02:18.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.482+0000 7f4d2288f700 1 -- start start 2023-11-15T03:02:18.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.486+0000 7f4d2288f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c0669b0 0x7f4d1c14b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:18.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.486+0000 7f4d2288f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d1c0a8e60 0x7f4d1c1513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:18.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.486+0000 7f4d2288f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c157d90 0x7f4d1c156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:18.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.486+0000 7f4d2288f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c169870 con 0x7f4d1c0a8e60 2023-11-15T03:02:18.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.486+0000 7f4d2288f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c169b10 con 0x7f4d1c157d90 2023-11-15T03:02:18.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.486+0000 7f4d2288f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c169df0 con 0x7f4d1c0669b0 2023-11-15T03:02:18.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.486+0000 7f4d2108c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d1c0a8e60 0x7f4d1c1513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:18.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.486+0000 7f4d2108c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d1c0a8e60 0x7f4d1c1513e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54638/0 (socket says 172.21.15.121:54638) 2023-11-15T03:02:18.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.486+0000 7f4d2108c700 1 -- 172.21.15.121:0/180846491 learned_addr learned my addr 172.21.15.121:0/180846491 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:18.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.486+0000 7f4d2188d700 1 --2- 172.21.15.121:0/180846491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c0669b0 0x7f4d1c14b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:18.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.486+0000 7f4d2208e700 1 --2- 172.21.15.121:0/180846491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c157d90 0x7f4d1c156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:18.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.486+0000 7f4d2108c700 1 -- 172.21.15.121:0/180846491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c0669b0 msgr2=0x7f4d1c14b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:18.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.486+0000 7f4d2108c700 1 --2- 172.21.15.121:0/180846491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c0669b0 0x7f4d1c14b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:18.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.486+0000 7f4d2108c700 1 -- 172.21.15.121:0/180846491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c157d90 msgr2=0x7f4d1c156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:18.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.486+0000 7f4d2108c700 1 --2- 172.21.15.121:0/180846491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c157d90 0x7f4d1c156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:18.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.486+0000 7f4d2108c700 1 -- 172.21.15.121:0/180846491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d14009a20 con 0x7f4d1c0a8e60 2023-11-15T03:02:18.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.486+0000 7f4d2208e700 1 --2- 172.21.15.121:0/180846491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c157d90 0x7f4d1c156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:02:18.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.486+0000 7f4d2108c700 1 --2- 172.21.15.121:0/180846491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d1c0a8e60 0x7f4d1c1513e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f4d0c00c4c0 tx=0x7f4d0c007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:18.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.486+0000 7f4d11ffb700 1 -- 172.21.15.121:0/180846491 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c00ce70 con 0x7f4d1c0a8e60 2023-11-15T03:02:18.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.490+0000 7f4d11ffb700 1 -- 172.21.15.121:0/180846491 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c007020 con 0x7f4d1c0a8e60 2023-11-15T03:02:18.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.490+0000 7f4d11ffb700 1 -- 172.21.15.121:0/180846491 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c0073c0 con 0x7f4d1c0a8e60 2023-11-15T03:02:18.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.490+0000 7f4d2288f700 1 -- 172.21.15.121:0/180846491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c16a0d0 con 0x7f4d1c0a8e60 2023-11-15T03:02:18.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.490+0000 7f4d2288f700 1 -- 172.21.15.121:0/180846491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c14bcf0 con 0x7f4d1c0a8e60 2023-11-15T03:02:18.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.490+0000 7f4d2288f700 1 -- 172.21.15.121:0/180846491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c06bcf0 con 0x7f4d1c0a8e60 2023-11-15T03:02:18.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.490+0000 7f4d11ffb700 1 -- 172.21.15.121:0/180846491 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4d0c006900 con 0x7f4d1c0a8e60 2023-11-15T03:02:18.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.494+0000 7f4d11ffb700 1 --2- 172.21.15.121:0/180846491 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4d0005a4e0 0x7f4d0005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:18.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.494+0000 7f4d11ffb700 1 -- 172.21.15.121:0/180846491 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f4d0c073db0 con 0x7f4d1c0a8e60 2023-11-15T03:02:18.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.494+0000 7f4d2188d700 1 --2- 172.21.15.121:0/180846491 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4d0005a4e0 0x7f4d0005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:18.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.494+0000 7f4d2188d700 1 --2- 172.21.15.121:0/180846491 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4d0005a4e0 0x7f4d0005c9a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f4d18007fd0 tx=0x7f4d18009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:18.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.498+0000 7f4d11ffb700 1 -- 172.21.15.121:0/180846491 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4d0c0478e0 con 0x7f4d1c0a8e60 2023-11-15T03:02:18.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:18.662+0000 7f4d2288f700 1 -- 172.21.15.121:0/180846491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f4d1c0c3c70 con 0x7f4d1c0a8e60 2023-11-15T03:02:19.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.234+0000 7f4d11ffb700 1 -- 172.21.15.121:0/180846491 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) v1 ==== 152+0+0 (secure 0 0 0) 0x7f4d0c04af00 con 0x7f4d1c0a8e60 2023-11-15T03:02:19.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.238+0000 7f4d2288f700 1 -- 172.21.15.121:0/180846491 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4d0005a4e0 msgr2=0x7f4d0005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:19.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.238+0000 7f4d2288f700 1 --2- 172.21.15.121:0/180846491 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4d0005a4e0 0x7f4d0005c9a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f4d18007fd0 tx=0x7f4d18009040).stop 2023-11-15T03:02:19.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.238+0000 7f4d2288f700 1 -- 172.21.15.121:0/180846491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d1c0a8e60 msgr2=0x7f4d1c1513e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:19.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.238+0000 7f4d2288f700 1 --2- 172.21.15.121:0/180846491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d1c0a8e60 0x7f4d1c1513e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f4d0c00c4c0 tx=0x7f4d0c007df0).stop 2023-11-15T03:02:19.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.238+0000 7f4d2288f700 1 -- 172.21.15.121:0/180846491 shutdown_connections 2023-11-15T03:02:19.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.238+0000 7f4d2288f700 1 --2- 172.21.15.121:0/180846491 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4d0005a4e0 0x7f4d0005c9a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:19.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.238+0000 7f4d2288f700 1 --2- 172.21.15.121:0/180846491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c157d90 0x7f4d1c156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:19.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.238+0000 7f4d2288f700 1 --2- 172.21.15.121:0/180846491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d1c0a8e60 0x7f4d1c1513e0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:19.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.238+0000 7f4d2288f700 1 --2- 172.21.15.121:0/180846491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c0669b0 0x7f4d1c14b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:19.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.238+0000 7f4d2288f700 1 -- 172.21.15.121:0/180846491 >> 172.21.15.121:0/180846491 conn(0x7f4d1c067a60 msgr2=0x7f4d1c0c64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:19.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.238+0000 7f4d2288f700 1 -- 172.21.15.121:0/180846491 shutdown_connections 2023-11-15T03:02:19.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.242+0000 7f4d2288f700 1 -- 172.21.15.121:0/180846491 wait complete. 2023-11-15T03:02:19.257 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_metadata' created 2023-11-15T03:02:19.273 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-15T03:02:19.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.442+0000 7f869ac5b700 1 Processor -- start 2023-11-15T03:02:19.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.442+0000 7f869ac5b700 1 -- start start 2023-11-15T03:02:19.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.446+0000 7f869ac5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86940c5c80 0x7f869415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:19.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.446+0000 7f869ac5b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8694062da0 con 0x7f86941570b0 2023-11-15T03:02:19.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.446+0000 7f869ac5b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86940bfb90 con 0x7f8694157490 2023-11-15T03:02:19.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.446+0000 7f869ac5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86940c6140 con 0x7f86940c5c80 2023-11-15T03:02:19.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.446+0000 7f86989f7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f86941570b0 0x7f8694066860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:55084/0 (socket says 172.21.15.121:55084) 2023-11-15T03:02:19.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.446+0000 7f86989f7700 1 -- 172.21.15.121:0/1976208063 learned_addr learned my addr 172.21.15.121:0/1976208063 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:19.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.446+0000 7f86991f8700 1 --2- 172.21.15.121:0/1976208063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86940c5c80 0x7f869415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:19.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.446+0000 7f86937fe700 1 -- 172.21.15.121:0/1976208063 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405220209 0 0) 0x7f8694062da0 con 0x7f86941570b0 2023-11-15T03:02:19.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86937fe700 1 -- 172.21.15.121:0/1976208063 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86840033f0 con 0x7f86941570b0 2023-11-15T03:02:19.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86937fe700 1 -- 172.21.15.121:0/1976208063 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1043337713 0 0) 0x7f86940bfb90 con 0x7f8694157490 2023-11-15T03:02:19.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86937fe700 1 -- 172.21.15.121:0/1976208063 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8694062da0 con 0x7f8694157490 2023-11-15T03:02:19.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86937fe700 1 -- 172.21.15.121:0/1976208063 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4057641882 0 0) 0x7f86840033f0 con 0x7f86941570b0 2023-11-15T03:02:19.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86937fe700 1 -- 172.21.15.121:0/1976208063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86940c5c80 msgr2=0x7f869415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:19.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86937fe700 1 --2- 172.21.15.121:0/1976208063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86940c5c80 0x7f869415ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:19.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86937fe700 1 -- 172.21.15.121:0/1976208063 >> v1:172.21.15.190:6789/0 conn(0x7f8694157490 legacy=0x7f8694065950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:19.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86937fe700 1 -- 172.21.15.121:0/1976208063 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86940c6420 con 0x7f86941570b0 2023-11-15T03:02:19.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86937fe700 1 -- 172.21.15.121:0/1976208063 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f8688001470 con 0x7f86941570b0 2023-11-15T03:02:19.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86937fe700 1 -- 172.21.15.121:0/1976208063 >> v1:172.21.15.121:6789/0 conn(0x7f86941570b0 legacy=0x7f8694066860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:19.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86937fe700 1 --2- 172.21.15.121:0/1976208063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8684003aa0 0x7f8684003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:19.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86937fe700 1 --2- 172.21.15.121:0/1976208063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86940c5c80 0x7f8684004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:19.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86937fe700 1 --2- 172.21.15.121:0/1976208063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8684004b80 0x7f8684005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:19.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86937fe700 1 -- 172.21.15.121:0/1976208063 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86940c6140 con 0x7f8684003aa0 2023-11-15T03:02:19.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86937fe700 1 -- 172.21.15.121:0/1976208063 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868400b940 con 0x7f8684004b80 2023-11-15T03:02:19.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86937fe700 1 -- 172.21.15.121:0/1976208063 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868400bb00 con 0x7f86940c5c80 2023-11-15T03:02:19.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86989f7700 1 --2- 172.21.15.121:0/1976208063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8684003aa0 0x7f8684003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:19.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f8693fff700 1 --2- 172.21.15.121:0/1976208063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86940c5c80 0x7f8684004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:19.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86991f8700 1 --2- 172.21.15.121:0/1976208063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8684004b80 0x7f8684005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:19.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86989f7700 1 -- 172.21.15.121:0/1976208063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86940c5c80 msgr2=0x7f8684004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:19.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86989f7700 1 --2- 172.21.15.121:0/1976208063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86940c5c80 0x7f8684004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:19.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86989f7700 1 -- 172.21.15.121:0/1976208063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8684004b80 msgr2=0x7f8684005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:19.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86989f7700 1 --2- 172.21.15.121:0/1976208063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8684004b80 0x7f8684005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:19.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86989f7700 1 -- 172.21.15.121:0/1976208063 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86840033f0 con 0x7f8684003aa0 2023-11-15T03:02:19.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86991f8700 1 --2- 172.21.15.121:0/1976208063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8684004b80 0x7f8684005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:02:19.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86989f7700 1 --2- 172.21.15.121:0/1976208063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8684003aa0 0x7f8684003ea0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f868800c190 tx=0x7f8688001e10).ready entity=mon.0 client_cookie=7c0c2f11179d570c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:19.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86937fe700 1 -- 172.21.15.121:0/1976208063 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868800ca00 con 0x7f8684003aa0 2023-11-15T03:02:19.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86937fe700 1 -- 172.21.15.121:0/1976208063 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868800cb60 con 0x7f8684003aa0 2023-11-15T03:02:19.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86937fe700 1 -- 172.21.15.121:0/1976208063 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86880070d0 con 0x7f8684003aa0 2023-11-15T03:02:19.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f869ac5b700 1 -- 172.21.15.121:0/1976208063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8684003aa0 msgr2=0x7f8684003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:19.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f869ac5b700 1 --2- 172.21.15.121:0/1976208063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8684003aa0 0x7f8684003ea0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f868800c190 tx=0x7f8688001e10).stop 2023-11-15T03:02:19.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f86991f8700 1 -- 172.21.15.121:0/1976208063 reap_dead start 2023-11-15T03:02:19.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f869ac5b700 1 -- 172.21.15.121:0/1976208063 shutdown_connections 2023-11-15T03:02:19.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f869ac5b700 1 -- 172.21.15.121:0/1976208063 >> 172.21.15.121:0/1976208063 conn(0x7f86940684f0 msgr2=0x7f86940cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:19.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f869ac5b700 1 -- 172.21.15.121:0/1976208063 shutdown_connections 2023-11-15T03:02:19.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.450+0000 7f869ac5b700 1 -- 172.21.15.121:0/1976208063 wait complete. 2023-11-15T03:02:19.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.454+0000 7f869ac5b700 1 Processor -- start 2023-11-15T03:02:19.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.454+0000 7f869ac5b700 1 -- start start 2023-11-15T03:02:19.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f869ac5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8684003aa0 0x7f8694164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:19.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f869ac5b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8694135e30 0x7f86940c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:19.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f869ac5b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86941362b0 0x7f86940c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:19.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f869ac5b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86941678b0 con 0x7f86941362b0 2023-11-15T03:02:19.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f869ac5b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86940c9310 con 0x7f8694135e30 2023-11-15T03:02:19.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f869ac5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8694148cd0 con 0x7f8684003aa0 2023-11-15T03:02:19.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f86991f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86941362b0 0x7f86940c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:19.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f86991f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86941362b0 0x7f86940c8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54652/0 (socket says 172.21.15.121:54652) 2023-11-15T03:02:19.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f86991f8700 1 -- 172.21.15.121:0/2318338843 learned_addr learned my addr 172.21.15.121:0/2318338843 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:19.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f8693fff700 1 --2- 172.21.15.121:0/2318338843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8694135e30 0x7f86940c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:19.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f86989f7700 1 --2- 172.21.15.121:0/2318338843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8684003aa0 0x7f8694164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:19.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f86991f8700 1 -- 172.21.15.121:0/2318338843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8684003aa0 msgr2=0x7f8694164ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:19.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f86991f8700 1 --2- 172.21.15.121:0/2318338843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8684003aa0 0x7f8694164ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:19.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f86991f8700 1 -- 172.21.15.121:0/2318338843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8694135e30 msgr2=0x7f86940c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:19.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f86991f8700 1 --2- 172.21.15.121:0/2318338843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8694135e30 0x7f86940c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:19.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f86991f8700 1 -- 172.21.15.121:0/2318338843 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869414fdc0 con 0x7f86941362b0 2023-11-15T03:02:19.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f86989f7700 1 --2- 172.21.15.121:0/2318338843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8684003aa0 0x7f8694164ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:19.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f86991f8700 1 --2- 172.21.15.121:0/2318338843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86941362b0 0x7f86940c8e20 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f8694069b30 tx=0x7f868c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:19.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f8690ff9700 1 -- 172.21.15.121:0/2318338843 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c009bf0 con 0x7f86941362b0 2023-11-15T03:02:19.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f869ac5b700 1 -- 172.21.15.121:0/2318338843 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86941653c0 con 0x7f86941362b0 2023-11-15T03:02:19.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f8690ff9700 1 -- 172.21.15.121:0/2318338843 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c00fcb0 con 0x7f86941362b0 2023-11-15T03:02:19.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f8690ff9700 1 -- 172.21.15.121:0/2318338843 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c014410 con 0x7f86941362b0 2023-11-15T03:02:19.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.458+0000 7f869ac5b700 1 -- 172.21.15.121:0/2318338843 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869414f490 con 0x7f86941362b0 2023-11-15T03:02:19.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.462+0000 7f869ac5b700 1 -- 172.21.15.121:0/2318338843 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f869406c150 con 0x7f86941362b0 2023-11-15T03:02:19.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.462+0000 7f8690ff9700 1 -- 172.21.15.121:0/2318338843 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f868c014800 con 0x7f86941362b0 2023-11-15T03:02:19.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.462+0000 7f8690ff9700 1 --2- 172.21.15.121:0/2318338843 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f867805a510 0x7f867805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:19.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.462+0000 7f8690ff9700 1 -- 172.21.15.121:0/2318338843 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f868c074f30 con 0x7f86941362b0 2023-11-15T03:02:19.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.462+0000 7f86989f7700 1 --2- 172.21.15.121:0/2318338843 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f867805a510 0x7f867805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:19.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.466+0000 7f86989f7700 1 --2- 172.21.15.121:0/2318338843 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f867805a510 0x7f867805c9d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f8688001e70 tx=0x7f8688015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:19.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.470+0000 7f8690ff9700 1 -- 172.21.15.121:0/2318338843 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f868c0488b0 con 0x7f86941362b0 2023-11-15T03:02:19.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:19.638+0000 7f869ac5b700 1 -- 172.21.15.121:0/2318338843 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8694165660 con 0x7f86941362b0 2023-11-15T03:02:20.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.246+0000 7f8690ff9700 1 -- 172.21.15.121:0/2318338843 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) v1 ==== 170+0+0 (secure 0 0 0) 0x7f868c04bed0 con 0x7f86941362b0 2023-11-15T03:02:20.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.258+0000 7f869ac5b700 1 -- 172.21.15.121:0/2318338843 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f867805a510 msgr2=0x7f867805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:20.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.258+0000 7f869ac5b700 1 --2- 172.21.15.121:0/2318338843 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f867805a510 0x7f867805c9d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f8688001e70 tx=0x7f8688015000).stop 2023-11-15T03:02:20.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.258+0000 7f869ac5b700 1 -- 172.21.15.121:0/2318338843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86941362b0 msgr2=0x7f86940c8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:20.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.258+0000 7f869ac5b700 1 --2- 172.21.15.121:0/2318338843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86941362b0 0x7f86940c8e20 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f8694069b30 tx=0x7f868c00b040).stop 2023-11-15T03:02:20.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.262+0000 7f869ac5b700 1 -- 172.21.15.121:0/2318338843 shutdown_connections 2023-11-15T03:02:20.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.262+0000 7f869ac5b700 1 --2- 172.21.15.121:0/2318338843 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f867805a510 0x7f867805c9d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:20.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.262+0000 7f869ac5b700 1 --2- 172.21.15.121:0/2318338843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86941362b0 0x7f86940c8e20 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:20.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.262+0000 7f869ac5b700 1 --2- 172.21.15.121:0/2318338843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8694135e30 0x7f86940c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:20.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.262+0000 7f869ac5b700 1 --2- 172.21.15.121:0/2318338843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8684003aa0 0x7f8694164ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:20.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.262+0000 7f869ac5b700 1 -- 172.21.15.121:0/2318338843 >> 172.21.15.121:0/2318338843 conn(0x7f86940684f0 msgr2=0x7f86940b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:20.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.266+0000 7f869ac5b700 1 -- 172.21.15.121:0/2318338843 shutdown_connections 2023-11-15T03:02:20.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.266+0000 7f869ac5b700 1 -- 172.21.15.121:0/2318338843 wait complete. 2023-11-15T03:02:20.283 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_data' created 2023-11-15T03:02:20.307 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-15T03:02:20.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.470+0000 7f2378ee4700 1 Processor -- start 2023-11-15T03:02:20.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.470+0000 7f2378ee4700 1 -- start start 2023-11-15T03:02:20.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.474+0000 7f2378ee4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f23740c5c70 0x7f2374153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:20.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.474+0000 7f2378ee4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2374066330 0x7f23740ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:20.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.474+0000 7f2378ee4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2374065ea0 con 0x7f23740acd00 2023-11-15T03:02:20.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.474+0000 7f2378ee4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2374066bb0 con 0x7f23740c5c70 2023-11-15T03:02:20.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.474+0000 7f2378ee4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2374065890 con 0x7f2374066330 2023-11-15T03:02:20.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.474+0000 7f2372ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f23740c5c70 0x7f2374153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:20.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.474+0000 7f2372ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f23740c5c70 0x7f2374153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:33392/0 (socket says 172.21.15.121:33392) 2023-11-15T03:02:20.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.474+0000 7f2372ffd700 1 -- 172.21.15.121:0/850588061 learned_addr learned my addr 172.21.15.121:0/850588061 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:20.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.474+0000 7f2373fff700 1 --2- 172.21.15.121:0/850588061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2374066330 0x7f23740ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:20.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.478+0000 7f2372ffd700 1 -- 172.21.15.121:0/850588061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2374066330 msgr2=0x7f23740ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:20.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.478+0000 7f2372ffd700 1 --2- 172.21.15.121:0/850588061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2374066330 0x7f23740ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:20.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.478+0000 7f2372ffd700 1 -- 172.21.15.121:0/850588061 >> v1:172.21.15.190:6789/0 conn(0x7f23740acd00 legacy=0x7f23741536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:20.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.478+0000 7f2372ffd700 1 -- 172.21.15.121:0/850588061 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23740c64a0 con 0x7f23740c5c70 2023-11-15T03:02:20.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.478+0000 7f2373fff700 1 --2- 172.21.15.121:0/850588061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2374066330 0x7f23740ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:20.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.478+0000 7f2372ffd700 1 --2- 172.21.15.121:0/850588061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23740c5c70 0x7f2374153dd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f235c006fd0 tx=0x7f235c0090f0).ready entity=mon.1 client_cookie=521f23c02c4cf055 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:20.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.478+0000 7f23727fc700 1 -- 172.21.15.121:0/850588061 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c00b600 con 0x7f23740c5c70 2023-11-15T03:02:20.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.478+0000 7f23727fc700 1 -- 172.21.15.121:0/850588061 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c00b760 con 0x7f23740c5c70 2023-11-15T03:02:20.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.478+0000 7f23727fc700 1 -- 172.21.15.121:0/850588061 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c00bae0 con 0x7f23740c5c70 2023-11-15T03:02:20.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.478+0000 7f2378ee4700 1 -- 172.21.15.121:0/850588061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23740c5c70 msgr2=0x7f2374153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:20.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.478+0000 7f2378ee4700 1 --2- 172.21.15.121:0/850588061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23740c5c70 0x7f2374153dd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f235c006fd0 tx=0x7f235c0090f0).stop 2023-11-15T03:02:20.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.478+0000 7f2378ee4700 1 -- 172.21.15.121:0/850588061 shutdown_connections 2023-11-15T03:02:20.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.478+0000 7f2378ee4700 1 --2- 172.21.15.121:0/850588061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2374066330 0x7f23740ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:20.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.478+0000 7f2378ee4700 1 --2- 172.21.15.121:0/850588061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23740c5c70 0x7f2374153dd0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:20.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.478+0000 7f2378ee4700 1 -- 172.21.15.121:0/850588061 >> 172.21.15.121:0/850588061 conn(0x7f2374063fc0 msgr2=0x7f2374067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:20.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.478+0000 7f2378ee4700 1 -- 172.21.15.121:0/850588061 shutdown_connections 2023-11-15T03:02:20.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.478+0000 7f2378ee4700 1 -- 172.21.15.121:0/850588061 wait complete. 2023-11-15T03:02:20.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.482+0000 7f2378ee4700 1 Processor -- start 2023-11-15T03:02:20.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.482+0000 7f2378ee4700 1 -- start start 2023-11-15T03:02:20.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.482+0000 7f2378ee4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2374066330 0x7f2374154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:20.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.482+0000 7f2378ee4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23740acd00 0x7f23740aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:20.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.482+0000 7f2378ee4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23740c5c70 0x7f23740af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:20.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.482+0000 7f2378ee4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740af8a0 con 0x7f23740acd00 2023-11-15T03:02:20.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.482+0000 7f2378ee4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2374156c40 con 0x7f23740c5c70 2023-11-15T03:02:20.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.482+0000 7f2378ee4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23741542e0 con 0x7f2374066330 2023-11-15T03:02:20.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.482+0000 7f2372ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23740acd00 0x7f23740aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:20.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.482+0000 7f2372ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23740acd00 0x7f23740aeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54660/0 (socket says 172.21.15.121:54660) 2023-11-15T03:02:20.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.482+0000 7f2372ffd700 1 -- 172.21.15.121:0/2858516256 learned_addr learned my addr 172.21.15.121:0/2858516256 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:20.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.482+0000 7f2373fff700 1 --2- 172.21.15.121:0/2858516256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23740c5c70 0x7f23740af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:20.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.482+0000 7f23737fe700 1 --2- 172.21.15.121:0/2858516256 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2374066330 0x7f2374154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:20.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.482+0000 7f2372ffd700 1 -- 172.21.15.121:0/2858516256 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2374066330 msgr2=0x7f2374154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:20.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.482+0000 7f2372ffd700 1 --2- 172.21.15.121:0/2858516256 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2374066330 0x7f2374154bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:20.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.482+0000 7f2372ffd700 1 -- 172.21.15.121:0/2858516256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23740c5c70 msgr2=0x7f23740af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:20.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.482+0000 7f2372ffd700 1 --2- 172.21.15.121:0/2858516256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23740c5c70 0x7f23740af3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:20.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.482+0000 7f2372ffd700 1 -- 172.21.15.121:0/2858516256 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c009370 con 0x7f23740acd00 2023-11-15T03:02:20.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.486+0000 7f2372ffd700 1 --2- 172.21.15.121:0/2858516256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23740acd00 0x7f23740aeec0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f235c009f50 tx=0x7f235c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:20.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.486+0000 7f236b7fe700 1 -- 172.21.15.121:0/2858516256 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c00b8d0 con 0x7f23740acd00 2023-11-15T03:02:20.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.486+0000 7f236b7fe700 1 -- 172.21.15.121:0/2858516256 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c00ba30 con 0x7f23740acd00 2023-11-15T03:02:20.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.486+0000 7f2378ee4700 1 -- 172.21.15.121:0/2858516256 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23741570d0 con 0x7f23740acd00 2023-11-15T03:02:20.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.486+0000 7f236b7fe700 1 -- 172.21.15.121:0/2858516256 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c01b660 con 0x7f23740acd00 2023-11-15T03:02:20.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.486+0000 7f2378ee4700 1 -- 172.21.15.121:0/2858516256 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23741565f0 con 0x7f23740acd00 2023-11-15T03:02:20.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.486+0000 7f2378ee4700 1 -- 172.21.15.121:0/2858516256 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f237406bcf0 con 0x7f23740acd00 2023-11-15T03:02:20.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.490+0000 7f236b7fe700 1 -- 172.21.15.121:0/2858516256 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f235c01ba10 con 0x7f23740acd00 2023-11-15T03:02:20.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.490+0000 7f236b7fe700 1 --2- 172.21.15.121:0/2858516256 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f235805a510 0x7f235805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:20.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.490+0000 7f236b7fe700 1 -- 172.21.15.121:0/2858516256 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6021+0+0 (secure 0 0 0) 0x7f235c0760e0 con 0x7f23740acd00 2023-11-15T03:02:20.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.490+0000 7f23737fe700 1 --2- 172.21.15.121:0/2858516256 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f235805a510 0x7f235805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:20.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.490+0000 7f23737fe700 1 --2- 172.21.15.121:0/2858516256 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f235805a510 0x7f235805c9d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f2364003e10 tx=0x7f2364009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:20.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.498+0000 7f236b7fe700 1 -- 172.21.15.121:0/2858516256 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f235c049860 con 0x7f23740acd00 2023-11-15T03:02:20.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.694+0000 7f2378ee4700 1 -- 172.21.15.121:0/2858516256 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f23740b0a50 con 0x7f23740acd00 2023-11-15T03:02:20.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.710+0000 7f236b7fe700 1 -- 172.21.15.121:0/2858516256 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v63) v1 ==== 174+0+0 (secure 0 0 0) 0x7f235c04ce80 con 0x7f23740acd00 2023-11-15T03:02:20.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.714+0000 7f2378ee4700 1 -- 172.21.15.121:0/2858516256 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f235805a510 msgr2=0x7f235805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:20.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.714+0000 7f2378ee4700 1 --2- 172.21.15.121:0/2858516256 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f235805a510 0x7f235805c9d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f2364003e10 tx=0x7f2364009040).stop 2023-11-15T03:02:20.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.714+0000 7f2378ee4700 1 -- 172.21.15.121:0/2858516256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23740acd00 msgr2=0x7f23740aeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:20.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.714+0000 7f2378ee4700 1 --2- 172.21.15.121:0/2858516256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23740acd00 0x7f23740aeec0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f235c009f50 tx=0x7f235c017000).stop 2023-11-15T03:02:20.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.718+0000 7f2378ee4700 1 -- 172.21.15.121:0/2858516256 shutdown_connections 2023-11-15T03:02:20.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.718+0000 7f2378ee4700 1 --2- 172.21.15.121:0/2858516256 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f235805a510 0x7f235805c9d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:20.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.718+0000 7f2378ee4700 1 --2- 172.21.15.121:0/2858516256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23740c5c70 0x7f23740af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:20.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.718+0000 7f2378ee4700 1 --2- 172.21.15.121:0/2858516256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23740acd00 0x7f23740aeec0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:20.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.718+0000 7f2378ee4700 1 --2- 172.21.15.121:0/2858516256 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2374066330 0x7f2374154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:20.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.718+0000 7f2378ee4700 1 -- 172.21.15.121:0/2858516256 >> 172.21.15.121:0/2858516256 conn(0x7f2374063fc0 msgr2=0x7f23741528c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:20.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.722+0000 7f2378ee4700 1 -- 172.21.15.121:0/2858516256 shutdown_connections 2023-11-15T03:02:20.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.722+0000 7f2378ee4700 1 -- 172.21.15.121:0/2858516256 wait complete. 2023-11-15T03:02:20.740 INFO:teuthology.orchestra.run.smithi121.stderr:new fs with metadata pool 11 and data pool 12 2023-11-15T03:02:20.761 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:02:20.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.910+0000 7f4a3b08f700 1 Processor -- start 2023-11-15T03:02:20.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.910+0000 7f4a3b08f700 1 -- start start 2023-11-15T03:02:20.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.914+0000 7f4a3b08f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4a34151730 0x7f4a34151b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:20.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.914+0000 7f4a3b08f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4a340bbe00 0x7f4a340c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:20.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.914+0000 7f4a3b08f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a340c3190 con 0x7f4a340bc290 2023-11-15T03:02:20.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.914+0000 7f4a3b08f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a340c3470 con 0x7f4a34151730 2023-11-15T03:02:20.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.914+0000 7f4a3b08f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a340c3750 con 0x7f4a340bbe00 2023-11-15T03:02:20.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.914+0000 7f4a3a08d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4a34151730 0x7f4a34151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:20.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.914+0000 7f4a3a08d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4a34151730 0x7f4a34151b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54662/0 (socket says 172.21.15.121:54662) 2023-11-15T03:02:20.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.914+0000 7f4a3a08d700 1 -- 172.21.15.121:0/2688997471 learned_addr learned my addr 172.21.15.121:0/2688997471 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:20.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.914+0000 7f4a3988c700 1 --2- 172.21.15.121:0/2688997471 >> v2:172.21.15.190:3300/0 conn(0x7f4a340bbe00 0x7f4a340c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:20.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.914+0000 7f4a3a08d700 1 -- 172.21.15.121:0/2688997471 >> v2:172.21.15.190:3300/0 conn(0x7f4a340bbe00 msgr2=0x7f4a340c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:20.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.914+0000 7f4a3a08d700 1 --2- 172.21.15.121:0/2688997471 >> v2:172.21.15.190:3300/0 conn(0x7f4a340bbe00 0x7f4a340c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:20.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.914+0000 7f4a3a08d700 1 -- 172.21.15.121:0/2688997471 >> v1:172.21.15.190:6789/0 conn(0x7f4a340bc290 legacy=0x7f4a34155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:20.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.914+0000 7f4a3a08d700 1 -- 172.21.15.121:0/2688997471 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a340c3a30 con 0x7f4a34151730 2023-11-15T03:02:20.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.914+0000 7f4a3a08d700 1 --2- 172.21.15.121:0/2688997471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a34151730 0x7f4a34151b10 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f4a340a64c0 tx=0x7f4a300090f0).ready entity=mon.0 client_cookie=bc33d314ec010e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:20.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.918+0000 7f4a3908b700 1 -- 172.21.15.121:0/2688997471 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3000b5a0 con 0x7f4a34151730 2023-11-15T03:02:20.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.918+0000 7f4a3908b700 1 -- 172.21.15.121:0/2688997471 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3000b700 con 0x7f4a34151730 2023-11-15T03:02:20.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.918+0000 7f4a3908b700 1 -- 172.21.15.121:0/2688997471 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3000ba80 con 0x7f4a34151730 2023-11-15T03:02:20.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.918+0000 7f4a3b08f700 1 -- 172.21.15.121:0/2688997471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a34151730 msgr2=0x7f4a34151b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:20.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.918+0000 7f4a3b08f700 1 --2- 172.21.15.121:0/2688997471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a34151730 0x7f4a34151b10 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f4a340a64c0 tx=0x7f4a300090f0).stop 2023-11-15T03:02:20.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.918+0000 7f4a3b08f700 1 -- 172.21.15.121:0/2688997471 shutdown_connections 2023-11-15T03:02:20.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.918+0000 7f4a3b08f700 1 --2- 172.21.15.121:0/2688997471 >> v2:172.21.15.190:3300/0 conn(0x7f4a340bbe00 0x7f4a340c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:20.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.918+0000 7f4a3b08f700 1 --2- 172.21.15.121:0/2688997471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a34151730 0x7f4a34151b10 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:20.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.918+0000 7f4a3b08f700 1 -- 172.21.15.121:0/2688997471 >> 172.21.15.121:0/2688997471 conn(0x7f4a34058c50 msgr2=0x7f4a34059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:20.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.918+0000 7f4a3b08f700 1 -- 172.21.15.121:0/2688997471 shutdown_connections 2023-11-15T03:02:20.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.918+0000 7f4a3b08f700 1 -- 172.21.15.121:0/2688997471 wait complete. 2023-11-15T03:02:20.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.918+0000 7f4a3b08f700 1 Processor -- start 2023-11-15T03:02:20.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.918+0000 7f4a3b08f700 1 -- start start 2023-11-15T03:02:20.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3b08f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a340bbe00 0x7f4a3414d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:20.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3b08f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a340bc290 0x7f4a340aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:20.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3b08f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a34151730 0x7f4a340ab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:20.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3b08f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a340ab8e0 con 0x7f4a340bc290 2023-11-15T03:02:20.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3b08f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a34152240 con 0x7f4a34151730 2023-11-15T03:02:20.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3b08f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a34152520 con 0x7f4a340bbe00 2023-11-15T03:02:20.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3988c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a340bc290 0x7f4a340aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:20.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3988c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a340bc290 0x7f4a340aaf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54666/0 (socket says 172.21.15.121:54666) 2023-11-15T03:02:20.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3988c700 1 -- 172.21.15.121:0/1885726892 learned_addr learned my addr 172.21.15.121:0/1885726892 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:20.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3a88e700 1 --2- 172.21.15.121:0/1885726892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a34151730 0x7f4a340ab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:20.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3a08d700 1 --2- 172.21.15.121:0/1885726892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a340bbe00 0x7f4a3414d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:20.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3988c700 1 -- 172.21.15.121:0/1885726892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a340bbe00 msgr2=0x7f4a3414d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:20.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3988c700 1 --2- 172.21.15.121:0/1885726892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a340bbe00 0x7f4a3414d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:20.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3988c700 1 -- 172.21.15.121:0/1885726892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a34151730 msgr2=0x7f4a340ab3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:20.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3988c700 1 --2- 172.21.15.121:0/1885726892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a34151730 0x7f4a340ab3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:20.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3988c700 1 -- 172.21.15.121:0/1885726892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a30009370 con 0x7f4a340bc290 2023-11-15T03:02:20.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3a88e700 1 --2- 172.21.15.121:0/1885726892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a34151730 0x7f4a340ab3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:20.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3988c700 1 --2- 172.21.15.121:0/1885726892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a340bc290 0x7f4a340aaf00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f4a2400d890 tx=0x7f4a24008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:20.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a2a7fc700 1 -- 172.21.15.121:0/1885726892 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a240125f0 con 0x7f4a340bc290 2023-11-15T03:02:20.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a2a7fc700 1 -- 172.21.15.121:0/1885726892 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a24012750 con 0x7f4a340bc290 2023-11-15T03:02:20.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3b08f700 1 -- 172.21.15.121:0/1885726892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a34152800 con 0x7f4a340bc290 2023-11-15T03:02:20.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a2a7fc700 1 -- 172.21.15.121:0/1885726892 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a24012ad0 con 0x7f4a340bc290 2023-11-15T03:02:20.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.922+0000 7f4a3b08f700 1 -- 172.21.15.121:0/1885726892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a34152dc0 con 0x7f4a340bc290 2023-11-15T03:02:20.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.926+0000 7f4a3b08f700 1 -- 172.21.15.121:0/1885726892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a34152fe0 con 0x7f4a340bc290 2023-11-15T03:02:20.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.926+0000 7f4a2a7fc700 1 -- 172.21.15.121:0/1885726892 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4a24013480 con 0x7f4a340bc290 2023-11-15T03:02:20.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.926+0000 7f4a2a7fc700 1 --2- 172.21.15.121:0/1885726892 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a1805a510 0x7f4a1805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:20.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.926+0000 7f4a2a7fc700 1 -- 172.21.15.121:0/1885726892 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4a24074a70 con 0x7f4a340bc290 2023-11-15T03:02:20.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.926+0000 7f4a3a08d700 1 --2- 172.21.15.121:0/1885726892 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a1805a510 0x7f4a1805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:20.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.930+0000 7f4a3a08d700 1 --2- 172.21.15.121:0/1885726892 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a1805a510 0x7f4a1805c9d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f4a30009130 tx=0x7f4a30015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:20.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:20.930+0000 7f4a2a7fc700 1 -- 172.21.15.121:0/1885726892 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4a24048130 con 0x7f4a340bc290 2023-11-15T03:02:21.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.098+0000 7f4a3b08f700 1 -- 172.21.15.121:0/1885726892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a3406bcf0 con 0x7f4a340bc290 2023-11-15T03:02:21.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.102+0000 7f4a2a7fc700 1 -- 172.21.15.121:0/1885726892 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16992 (secure 0 0 0) 0x7f4a2404b750 con 0x7f4a340bc290 2023-11-15T03:02:21.110 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:21.111 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":76,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:02:20.702341+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-15T03:02:18.737496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-15T03:02:19.718114+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:02:21.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.106+0000 7f4a3b08f700 1 -- 172.21.15.121:0/1885726892 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a1805a510 msgr2=0x7f4a1805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:21.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.106+0000 7f4a3b08f700 1 --2- 172.21.15.121:0/1885726892 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a1805a510 0x7f4a1805c9d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f4a30009130 tx=0x7f4a30015000).stop 2023-11-15T03:02:21.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.106+0000 7f4a3b08f700 1 -- 172.21.15.121:0/1885726892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a340bc290 msgr2=0x7f4a340aaf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:21.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.106+0000 7f4a3b08f700 1 --2- 172.21.15.121:0/1885726892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a340bc290 0x7f4a340aaf00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f4a2400d890 tx=0x7f4a24008040).stop 2023-11-15T03:02:21.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.106+0000 7f4a3b08f700 1 -- 172.21.15.121:0/1885726892 shutdown_connections 2023-11-15T03:02:21.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.106+0000 7f4a3b08f700 1 --2- 172.21.15.121:0/1885726892 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a1805a510 0x7f4a1805c9d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.106+0000 7f4a3b08f700 1 --2- 172.21.15.121:0/1885726892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a34151730 0x7f4a340ab3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.106+0000 7f4a3b08f700 1 --2- 172.21.15.121:0/1885726892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a340bc290 0x7f4a340aaf00 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.106+0000 7f4a3b08f700 1 --2- 172.21.15.121:0/1885726892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a340bbe00 0x7f4a3414d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.106+0000 7f4a3b08f700 1 -- 172.21.15.121:0/1885726892 >> 172.21.15.121:0/1885726892 conn(0x7f4a34058c50 msgr2=0x7f4a34157a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:21.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.106+0000 7f4a3b08f700 1 -- 172.21.15.121:0/1885726892 shutdown_connections 2023-11-15T03:02:21.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.106+0000 7f4a3b08f700 1 -- 172.21.15.121:0/1885726892 wait complete. 2023-11-15T03:02:21.137 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:02:21.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.290+0000 7fea75879700 1 Processor -- start 2023-11-15T03:02:21.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.290+0000 7fea75879700 1 -- start start 2023-11-15T03:02:21.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.294+0000 7fea75879700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea700c2010 0x7fea700c23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:21.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.294+0000 7fea75879700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fea70154750 0x7fea700c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:21.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.294+0000 7fea75879700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fea700c1240 0x7fea70153030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:21.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.294+0000 7fea75879700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea70154b90 con 0x7fea700c1240 2023-11-15T03:02:21.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.294+0000 7fea75879700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea70153c90 con 0x7fea70154750 2023-11-15T03:02:21.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.294+0000 7fea75879700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea700c1810 con 0x7fea700c2010 2023-11-15T03:02:21.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea75078700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fea700c1240 0x7fea70153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:21.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea75078700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fea700c1240 0x7fea70153030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54680/0 (socket says 172.21.15.121:54680) 2023-11-15T03:02:21.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea75078700 1 -- 172.21.15.121:0/3898937436 learned_addr learned my addr 172.21.15.121:0/3898937436 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:21.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea6ffff700 1 --2- 172.21.15.121:0/3898937436 >> v2:172.21.15.190:3300/0 conn(0x7fea70154750 0x7fea700c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:21.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea74877700 1 --2- 172.21.15.121:0/3898937436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea700c2010 0x7fea700c23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:21.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea75078700 1 -- 172.21.15.121:0/3898937436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea700c2010 msgr2=0x7fea700c23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:21.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea75078700 1 --2- 172.21.15.121:0/3898937436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea700c2010 0x7fea700c23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea75078700 1 -- 172.21.15.121:0/3898937436 >> v2:172.21.15.190:3300/0 conn(0x7fea70154750 msgr2=0x7fea700c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:21.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea75078700 1 --2- 172.21.15.121:0/3898937436 >> v2:172.21.15.190:3300/0 conn(0x7fea70154750 0x7fea700c0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea75078700 1 -- 172.21.15.121:0/3898937436 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea700c1af0 con 0x7fea700c1240 2023-11-15T03:02:21.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea75078700 1 --2- 172.21.15.121:0/3898937436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea700c1240 0x7fea70153030 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7fea70064c40 tx=0x7fea6800ba50).ready entity=mon.0 client_cookie=b02a59f018469144 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:21.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea6f7fe700 1 -- 172.21.15.121:0/3898937436 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68010070 con 0x7fea700c1240 2023-11-15T03:02:21.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea6f7fe700 1 -- 172.21.15.121:0/3898937436 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6800dd30 con 0x7fea700c1240 2023-11-15T03:02:21.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea6f7fe700 1 -- 172.21.15.121:0/3898937436 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6800dea0 con 0x7fea700c1240 2023-11-15T03:02:21.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea75879700 1 -- 172.21.15.121:0/3898937436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea700c1240 msgr2=0x7fea70153030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:21.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea75879700 1 --2- 172.21.15.121:0/3898937436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea700c1240 0x7fea70153030 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7fea70064c40 tx=0x7fea6800ba50).stop 2023-11-15T03:02:21.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea75879700 1 -- 172.21.15.121:0/3898937436 shutdown_connections 2023-11-15T03:02:21.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea75879700 1 --2- 172.21.15.121:0/3898937436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea700c1240 0x7fea70153030 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea75879700 1 --2- 172.21.15.121:0/3898937436 >> v2:172.21.15.190:3300/0 conn(0x7fea70154750 0x7fea700c0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea75879700 1 --2- 172.21.15.121:0/3898937436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea700c2010 0x7fea700c23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea75879700 1 -- 172.21.15.121:0/3898937436 >> 172.21.15.121:0/3898937436 conn(0x7fea70062190 msgr2=0x7fea70062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:21.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea75879700 1 -- 172.21.15.121:0/3898937436 shutdown_connections 2023-11-15T03:02:21.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.298+0000 7fea75879700 1 -- 172.21.15.121:0/3898937436 wait complete. 2023-11-15T03:02:21.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.302+0000 7fea75879700 1 Processor -- start 2023-11-15T03:02:21.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.302+0000 7fea75879700 1 -- start start 2023-11-15T03:02:21.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.302+0000 7fea75879700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea700c1240 0x7fea70148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:21.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.302+0000 7fea75879700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea700c2010 0x7fea700be210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:21.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.302+0000 7fea75879700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea70154750 0x7fea700be700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:21.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.302+0000 7fea75879700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea700bebf0 con 0x7fea700c2010 2023-11-15T03:02:21.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.302+0000 7fea75879700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea700a9270 con 0x7fea70154750 2023-11-15T03:02:21.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.302+0000 7fea75879700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea70152840 con 0x7fea700c1240 2023-11-15T03:02:21.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.302+0000 7fea6ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea700c2010 0x7fea700be210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:21.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.302+0000 7fea6ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea700c2010 0x7fea700be210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54688/0 (socket says 172.21.15.121:54688) 2023-11-15T03:02:21.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.302+0000 7fea6ffff700 1 -- 172.21.15.121:0/76978919 learned_addr learned my addr 172.21.15.121:0/76978919 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:21.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.302+0000 7fea74877700 1 --2- 172.21.15.121:0/76978919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea700c1240 0x7fea70148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:21.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.302+0000 7fea75078700 1 --2- 172.21.15.121:0/76978919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea70154750 0x7fea700be700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:21.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.306+0000 7fea6ffff700 1 -- 172.21.15.121:0/76978919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea700c1240 msgr2=0x7fea70148230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:21.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.306+0000 7fea6ffff700 1 --2- 172.21.15.121:0/76978919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea700c1240 0x7fea70148230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.306+0000 7fea6ffff700 1 -- 172.21.15.121:0/76978919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea70154750 msgr2=0x7fea700be700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:21.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.306+0000 7fea6ffff700 1 --2- 172.21.15.121:0/76978919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea70154750 0x7fea700be700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.306+0000 7fea6ffff700 1 -- 172.21.15.121:0/76978919 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6800bcf0 con 0x7fea700c2010 2023-11-15T03:02:21.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.306+0000 7fea75078700 1 --2- 172.21.15.121:0/76978919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea70154750 0x7fea700be700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:21.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.306+0000 7fea6ffff700 1 --2- 172.21.15.121:0/76978919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea700c2010 0x7fea700be210 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7fea5c00e960 tx=0x7fea5c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:21.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.306+0000 7fea6cff9700 1 -- 172.21.15.121:0/76978919 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c00c6a0 con 0x7fea700c2010 2023-11-15T03:02:21.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.306+0000 7fea6cff9700 1 -- 172.21.15.121:0/76978919 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5c00c800 con 0x7fea700c2010 2023-11-15T03:02:21.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.306+0000 7fea75879700 1 -- 172.21.15.121:0/76978919 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea700bc140 con 0x7fea700c2010 2023-11-15T03:02:21.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.306+0000 7fea6cff9700 1 -- 172.21.15.121:0/76978919 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c00cb80 con 0x7fea700c2010 2023-11-15T03:02:21.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.306+0000 7fea75879700 1 -- 172.21.15.121:0/76978919 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea700bbb70 con 0x7fea700c2010 2023-11-15T03:02:21.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.306+0000 7fea75879700 1 -- 172.21.15.121:0/76978919 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea700bc2d0 con 0x7fea700c2010 2023-11-15T03:02:21.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.306+0000 7fea6cff9700 1 -- 172.21.15.121:0/76978919 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fea5c01b400 con 0x7fea700c2010 2023-11-15T03:02:21.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.310+0000 7fea6cff9700 1 --2- 172.21.15.121:0/76978919 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fea5405a540 0x7fea5405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:21.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.310+0000 7fea6cff9700 1 -- 172.21.15.121:0/76978919 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fea5c074d30 con 0x7fea700c2010 2023-11-15T03:02:21.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.310+0000 7fea74877700 1 --2- 172.21.15.121:0/76978919 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fea5405a540 0x7fea5405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:21.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.310+0000 7fea74877700 1 --2- 172.21.15.121:0/76978919 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fea5405a540 0x7fea5405ca00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fea64009c30 tx=0x7fea64009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:21.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.318+0000 7fea6cff9700 1 -- 172.21.15.121:0/76978919 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fea5c0484a0 con 0x7fea700c2010 2023-11-15T03:02:21.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.486+0000 7fea75879700 1 -- 172.21.15.121:0/76978919 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea700c7080 con 0x7fea700c2010 2023-11-15T03:02:21.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.486+0000 7fea6cff9700 1 -- 172.21.15.121:0/76978919 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16992 (secure 0 0 0) 0x7fea5c04bac0 con 0x7fea700c2010 2023-11-15T03:02:21.496 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:21.496 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":76,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:02:20.702341+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-15T03:02:18.737496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-15T03:02:19.718114+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:02:21.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.490+0000 7fea75879700 1 -- 172.21.15.121:0/76978919 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fea5405a540 msgr2=0x7fea5405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:21.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.490+0000 7fea75879700 1 --2- 172.21.15.121:0/76978919 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fea5405a540 0x7fea5405ca00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fea64009c30 tx=0x7fea64009060).stop 2023-11-15T03:02:21.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.490+0000 7fea75879700 1 -- 172.21.15.121:0/76978919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea700c2010 msgr2=0x7fea700be210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:21.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.490+0000 7fea75879700 1 --2- 172.21.15.121:0/76978919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea700c2010 0x7fea700be210 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7fea5c00e960 tx=0x7fea5c009060).stop 2023-11-15T03:02:21.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.490+0000 7fea75879700 1 -- 172.21.15.121:0/76978919 shutdown_connections 2023-11-15T03:02:21.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.490+0000 7fea75879700 1 --2- 172.21.15.121:0/76978919 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fea5405a540 0x7fea5405ca00 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.490+0000 7fea75879700 1 --2- 172.21.15.121:0/76978919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea70154750 0x7fea700be700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.490+0000 7fea75879700 1 --2- 172.21.15.121:0/76978919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea700c2010 0x7fea700be210 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.490+0000 7fea75879700 1 --2- 172.21.15.121:0/76978919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea700c1240 0x7fea70148230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.490+0000 7fea75879700 1 -- 172.21.15.121:0/76978919 >> 172.21.15.121:0/76978919 conn(0x7fea70062190 msgr2=0x7fea700657f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:21.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.490+0000 7fea75879700 1 -- 172.21.15.121:0/76978919 shutdown_connections 2023-11-15T03:02:21.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.490+0000 7fea75879700 1 -- 172.21.15.121:0/76978919 wait complete. 2023-11-15T03:02:21.519 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-15T03:02:21.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.678+0000 7f8024a5a700 1 Processor -- start 2023-11-15T03:02:21.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.678+0000 7f8024a5a700 1 -- start start 2023-11-15T03:02:21.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f8024a5a700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8020153a40 0x7f80200bf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:21.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f8024a5a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80200bd6e0 0x7f80200bdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:21.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f8024a5a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f80200c51b0 0x7f80200c5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:21.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f8024a5a700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80200bed40 con 0x7f8020153a40 2023-11-15T03:02:21.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f8024a5a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80200c5b30 con 0x7f80200c51b0 2023-11-15T03:02:21.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f8024a5a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80200be130 con 0x7f80200bd6e0 2023-11-15T03:02:21.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f801ed9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8020153a40 0x7f80200bf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:21.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f801f59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f80200c51b0 0x7f80200c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:21.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f801ed9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8020153a40 0x7f80200bf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54702/0 (socket says 172.21.15.121:54702) 2023-11-15T03:02:21.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f801ed9d700 1 -- 172.21.15.121:0/3411048406 learned_addr learned my addr 172.21.15.121:0/3411048406 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:21.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f8017fff700 1 --2- 172.21.15.121:0/3411048406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80200bd6e0 0x7f80200bdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:21.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f801f59e700 1 -- 172.21.15.121:0/3411048406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80200bd6e0 msgr2=0x7f80200bdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:21.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f801f59e700 1 --2- 172.21.15.121:0/3411048406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80200bd6e0 0x7f80200bdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f801f59e700 1 -- 172.21.15.121:0/3411048406 >> v2:172.21.15.121:3300/0 conn(0x7f8020153a40 msgr2=0x7f80200bf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:21.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f801f59e700 1 --2- 172.21.15.121:0/3411048406 >> v2:172.21.15.121:3300/0 conn(0x7f8020153a40 0x7f80200bf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f801f59e700 1 -- 172.21.15.121:0/3411048406 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80200be410 con 0x7f80200c51b0 2023-11-15T03:02:21.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f801ed9d700 1 --2- 172.21.15.121:0/3411048406 >> v2:172.21.15.121:3300/0 conn(0x7f8020153a40 0x7f80200bf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:02:21.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f801f59e700 1 --2- 172.21.15.121:0/3411048406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80200c51b0 0x7f80200c5640 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f8020069c10 tx=0x7f8018009780).ready entity=mon.1 client_cookie=5f6a3326ece126be server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:21.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f801e59c700 1 -- 172.21.15.121:0/3411048406 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801800e070 con 0x7f80200c51b0 2023-11-15T03:02:21.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f801e59c700 1 -- 172.21.15.121:0/3411048406 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801800bab0 con 0x7f80200c51b0 2023-11-15T03:02:21.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f801e59c700 1 -- 172.21.15.121:0/3411048406 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801800be30 con 0x7f80200c51b0 2023-11-15T03:02:21.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f8024a5a700 1 -- 172.21.15.121:0/3411048406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80200c51b0 msgr2=0x7f80200c5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:21.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.682+0000 7f8024a5a700 1 --2- 172.21.15.121:0/3411048406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80200c51b0 0x7f80200c5640 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f8020069c10 tx=0x7f8018009780).stop 2023-11-15T03:02:21.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.686+0000 7f8024a5a700 1 -- 172.21.15.121:0/3411048406 shutdown_connections 2023-11-15T03:02:21.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.686+0000 7f8024a5a700 1 --2- 172.21.15.121:0/3411048406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80200c51b0 0x7f80200c5640 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.686+0000 7f8024a5a700 1 --2- 172.21.15.121:0/3411048406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80200bd6e0 0x7f80200bdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.686+0000 7f8024a5a700 1 --2- 172.21.15.121:0/3411048406 >> v2:172.21.15.121:3300/0 conn(0x7f8020153a40 0x7f80200bf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.686+0000 7f8024a5a700 1 -- 172.21.15.121:0/3411048406 >> 172.21.15.121:0/3411048406 conn(0x7f8020067a60 msgr2=0x7f8020067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:21.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.686+0000 7f8024a5a700 1 -- 172.21.15.121:0/3411048406 shutdown_connections 2023-11-15T03:02:21.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.686+0000 7f8024a5a700 1 -- 172.21.15.121:0/3411048406 wait complete. 2023-11-15T03:02:21.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.686+0000 7f8024a5a700 1 Processor -- start 2023-11-15T03:02:21.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.686+0000 7f8024a5a700 1 -- start start 2023-11-15T03:02:21.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8024a5a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80200bd6e0 0x7f80201643f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:21.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8024a5a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80200c51b0 0x7f8020145d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:21.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8024a5a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8020153a40 0x7f8020062190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:21.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8024a5a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8020065080 con 0x7f80200c51b0 2023-11-15T03:02:21.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8024a5a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8020163f60 con 0x7f80200bd6e0 2023-11-15T03:02:21.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8024a5a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8020146260 con 0x7f8020153a40 2023-11-15T03:02:21.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8017fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80200c51b0 0x7f8020145d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:21.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8017fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80200c51b0 0x7f8020145d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54704/0 (socket says 172.21.15.121:54704) 2023-11-15T03:02:21.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8017fff700 1 -- 172.21.15.121:0/1083200807 learned_addr learned my addr 172.21.15.121:0/1083200807 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:21.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f801f59e700 1 --2- 172.21.15.121:0/1083200807 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8020153a40 0x7f8020062190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:21.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f801ed9d700 1 --2- 172.21.15.121:0/1083200807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80200bd6e0 0x7f80201643f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:21.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8017fff700 1 -- 172.21.15.121:0/1083200807 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8020153a40 msgr2=0x7f8020062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:21.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8017fff700 1 --2- 172.21.15.121:0/1083200807 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8020153a40 0x7f8020062190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8017fff700 1 -- 172.21.15.121:0/1083200807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80200bd6e0 msgr2=0x7f80201643f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:21.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8017fff700 1 --2- 172.21.15.121:0/1083200807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80200bd6e0 0x7f80201643f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:21.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8017fff700 1 -- 172.21.15.121:0/1083200807 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8010009370 con 0x7f80200c51b0 2023-11-15T03:02:21.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f801ed9d700 1 --2- 172.21.15.121:0/1083200807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80200bd6e0 0x7f80201643f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:21.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8017fff700 1 --2- 172.21.15.121:0/1083200807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80200c51b0 0x7f8020145d70 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7f800800e980 tx=0x7f8008009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:21.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8016ffd700 1 -- 172.21.15.121:0/1083200807 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800800ee70 con 0x7f80200c51b0 2023-11-15T03:02:21.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8016ffd700 1 -- 172.21.15.121:0/1083200807 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800800c850 con 0x7f80200c51b0 2023-11-15T03:02:21.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8024a5a700 1 -- 172.21.15.121:0/1083200807 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8018009a20 con 0x7f80200c51b0 2023-11-15T03:02:21.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8016ffd700 1 -- 172.21.15.121:0/1083200807 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800800cbd0 con 0x7f80200c51b0 2023-11-15T03:02:21.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.690+0000 7f8024a5a700 1 -- 172.21.15.121:0/1083200807 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8020062f30 con 0x7f80200c51b0 2023-11-15T03:02:21.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.694+0000 7f8024a5a700 1 -- 172.21.15.121:0/1083200807 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80201640f0 con 0x7f80200c51b0 2023-11-15T03:02:21.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.694+0000 7f8016ffd700 1 -- 172.21.15.121:0/1083200807 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f800801b400 con 0x7f80200c51b0 2023-11-15T03:02:21.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.698+0000 7f8016ffd700 1 --2- 172.21.15.121:0/1083200807 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f800005a510 0x7f800005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:21.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.698+0000 7f8016ffd700 1 -- 172.21.15.121:0/1083200807 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8008074cf0 con 0x7f80200c51b0 2023-11-15T03:02:21.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.698+0000 7f801ed9d700 1 --2- 172.21.15.121:0/1083200807 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f800005a510 0x7f800005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:21.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.698+0000 7f801ed9d700 1 --2- 172.21.15.121:0/1083200807 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f800005a510 0x7f800005c9d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f8020063250 tx=0x7f8010008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:21.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.702+0000 7f8016ffd700 1 -- 172.21.15.121:0/1083200807 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8008079070 con 0x7f80200c51b0 2023-11-15T03:02:21.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:21.902+0000 7f8024a5a700 1 -- 172.21.15.121:0/1083200807 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f80200be6f0 con 0x7f80200c51b0 2023-11-15T03:02:22.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.738+0000 7f8016ffd700 1 -- 172.21.15.121:0/1083200807 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8008016070 con 0x7f80200c51b0 2023-11-15T03:02:22.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.742+0000 7f8024a5a700 1 -- 172.21.15.121:0/1083200807 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f800005a510 msgr2=0x7f800005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:22.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.742+0000 7f8024a5a700 1 --2- 172.21.15.121:0/1083200807 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f800005a510 0x7f800005c9d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f8020063250 tx=0x7f8010008040).stop 2023-11-15T03:02:22.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.742+0000 7f8024a5a700 1 -- 172.21.15.121:0/1083200807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80200c51b0 msgr2=0x7f8020145d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:22.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.742+0000 7f8024a5a700 1 --2- 172.21.15.121:0/1083200807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80200c51b0 0x7f8020145d70 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7f800800e980 tx=0x7f8008009060).stop 2023-11-15T03:02:22.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.742+0000 7f8024a5a700 1 -- 172.21.15.121:0/1083200807 shutdown_connections 2023-11-15T03:02:22.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.742+0000 7f8024a5a700 1 --2- 172.21.15.121:0/1083200807 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f800005a510 0x7f800005c9d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:22.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.742+0000 7f8024a5a700 1 --2- 172.21.15.121:0/1083200807 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8020153a40 0x7f8020062190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:22.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.742+0000 7f8024a5a700 1 --2- 172.21.15.121:0/1083200807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80200c51b0 0x7f8020145d70 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:22.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.742+0000 7f8024a5a700 1 --2- 172.21.15.121:0/1083200807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80200bd6e0 0x7f80201643f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:22.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.742+0000 7f8024a5a700 1 -- 172.21.15.121:0/1083200807 >> 172.21.15.121:0/1083200807 conn(0x7f8020067a60 msgr2=0x7f8020150e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:22.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.742+0000 7f8024a5a700 1 -- 172.21.15.121:0/1083200807 shutdown_connections 2023-11-15T03:02:22.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.742+0000 7f8024a5a700 1 -- 172.21.15.121:0/1083200807 wait complete. 2023-11-15T03:02:22.773 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-15T03:02:22.774 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-15T03:02:22.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.934+0000 7f24af0e4700 1 Processor -- start 2023-11-15T03:02:22.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.934+0000 7f24af0e4700 1 -- start start 2023-11-15T03:02:22.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.938+0000 7f24af0e4700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f24a80a8d40 0x7f24a80c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:22.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.938+0000 7f24af0e4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f24a8066520 0x7f24a8066990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:22.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.938+0000 7f24af0e4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a80657f0 con 0x7f24a80a8960 2023-11-15T03:02:22.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.938+0000 7f24af0e4700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a81583d0 con 0x7f24a80a8d40 2023-11-15T03:02:22.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.938+0000 7f24af0e4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a80c1b60 con 0x7f24a8066520 2023-11-15T03:02:22.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.938+0000 7f24ad8e1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f24a80a8d40 0x7f24a80c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:22.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.938+0000 7f24ad8e1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f24a80a8d40 0x7f24a80c15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54706/0 (socket says 172.21.15.121:54706) 2023-11-15T03:02:22.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.938+0000 7f24ad8e1700 1 -- 172.21.15.121:0/3521962767 learned_addr learned my addr 172.21.15.121:0/3521962767 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:22.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.938+0000 7f24ae8e3700 1 --2- 172.21.15.121:0/3521962767 >> v2:172.21.15.190:3300/0 conn(0x7f24a8066520 0x7f24a8066990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:22.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.938+0000 7f24ad8e1700 1 -- 172.21.15.121:0/3521962767 >> v2:172.21.15.190:3300/0 conn(0x7f24a8066520 msgr2=0x7f24a8066990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:22.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.938+0000 7f24ad8e1700 1 --2- 172.21.15.121:0/3521962767 >> v2:172.21.15.190:3300/0 conn(0x7f24a8066520 0x7f24a8066990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:22.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.938+0000 7f24ad8e1700 1 -- 172.21.15.121:0/3521962767 >> v1:172.21.15.190:6789/0 conn(0x7f24a80a8960 legacy=0x7f24a815c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:22.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.938+0000 7f24ad8e1700 1 -- 172.21.15.121:0/3521962767 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a8065a40 con 0x7f24a80a8d40 2023-11-15T03:02:22.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.942+0000 7f24ad8e1700 1 --2- 172.21.15.121:0/3521962767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24a80a8d40 0x7f24a80c15c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7f24a805f8f0 tx=0x7f24980090f0).ready entity=mon.0 client_cookie=3f782296d26e090d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:22.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.942+0000 7f24ad0e0700 1 -- 172.21.15.121:0/3521962767 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249800b520 con 0x7f24a80a8d40 2023-11-15T03:02:22.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.942+0000 7f24ad0e0700 1 -- 172.21.15.121:0/3521962767 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249800b680 con 0x7f24a80a8d40 2023-11-15T03:02:22.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.942+0000 7f24ad0e0700 1 -- 172.21.15.121:0/3521962767 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249800ba00 con 0x7f24a80a8d40 2023-11-15T03:02:22.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.942+0000 7f24af0e4700 1 -- 172.21.15.121:0/3521962767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24a80a8d40 msgr2=0x7f24a80c15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:22.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.942+0000 7f24af0e4700 1 --2- 172.21.15.121:0/3521962767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24a80a8d40 0x7f24a80c15c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7f24a805f8f0 tx=0x7f24980090f0).stop 2023-11-15T03:02:22.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.942+0000 7f24af0e4700 1 -- 172.21.15.121:0/3521962767 shutdown_connections 2023-11-15T03:02:22.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.942+0000 7f24af0e4700 1 --2- 172.21.15.121:0/3521962767 >> v2:172.21.15.190:3300/0 conn(0x7f24a8066520 0x7f24a8066990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:22.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.942+0000 7f24af0e4700 1 --2- 172.21.15.121:0/3521962767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24a80a8d40 0x7f24a80c15c0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:22.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.942+0000 7f24af0e4700 1 -- 172.21.15.121:0/3521962767 >> 172.21.15.121:0/3521962767 conn(0x7f24a8067a60 msgr2=0x7f24a8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:22.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.942+0000 7f24af0e4700 1 -- 172.21.15.121:0/3521962767 shutdown_connections 2023-11-15T03:02:22.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.942+0000 7f24af0e4700 1 -- 172.21.15.121:0/3521962767 wait complete. 2023-11-15T03:02:22.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24af0e4700 1 Processor -- start 2023-11-15T03:02:22.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24af0e4700 1 -- start start 2023-11-15T03:02:22.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24af0e4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24a8066520 0x7f24a8168d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:22.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24af0e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24a80a8960 0x7f24a8149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:22.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24af0e4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24a80a8d40 0x7f24a80a65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:22.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24af0e4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a8156960 con 0x7f24a80a8d40 2023-11-15T03:02:22.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24af0e4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a8169200 con 0x7f24a8066520 2023-11-15T03:02:22.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24af0e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a8155ce0 con 0x7f24a80a8960 2023-11-15T03:02:22.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24ae8e3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24a80a8d40 0x7f24a80a65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:22.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24ae8e3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24a80a8d40 0x7f24a80a65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54716/0 (socket says 172.21.15.121:54716) 2023-11-15T03:02:22.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24ae8e3700 1 -- 172.21.15.121:0/2209134655 learned_addr learned my addr 172.21.15.121:0/2209134655 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:22.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24ae0e2700 1 --2- 172.21.15.121:0/2209134655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24a8066520 0x7f24a8168d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:22.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24ad8e1700 1 --2- 172.21.15.121:0/2209134655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24a80a8960 0x7f24a8149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:22.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24ae8e3700 1 -- 172.21.15.121:0/2209134655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24a80a8960 msgr2=0x7f24a8149640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:22.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24ae8e3700 1 --2- 172.21.15.121:0/2209134655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24a80a8960 0x7f24a8149640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:22.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24ae8e3700 1 -- 172.21.15.121:0/2209134655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24a8066520 msgr2=0x7f24a8168d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:22.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24ae8e3700 1 --2- 172.21.15.121:0/2209134655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24a8066520 0x7f24a8168d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:22.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24ae8e3700 1 -- 172.21.15.121:0/2209134655 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2498009370 con 0x7f24a80a8d40 2023-11-15T03:02:22.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24ae8e3700 1 --2- 172.21.15.121:0/2209134655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24a80a8d40 0x7f24a80a65d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f24a000e900 tx=0x7f24a0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:22.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f249e7fc700 1 -- 172.21.15.121:0/2209134655 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0012600 con 0x7f24a80a8d40 2023-11-15T03:02:22.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f249e7fc700 1 -- 172.21.15.121:0/2209134655 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a0012760 con 0x7f24a80a8d40 2023-11-15T03:02:22.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24af0e4700 1 -- 172.21.15.121:0/2209134655 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a8155fc0 con 0x7f24a80a8d40 2023-11-15T03:02:22.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f249e7fc700 1 -- 172.21.15.121:0/2209134655 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0012ae0 con 0x7f24a80a8d40 2023-11-15T03:02:22.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.946+0000 7f24af0e4700 1 -- 172.21.15.121:0/2209134655 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a8156580 con 0x7f24a80a8d40 2023-11-15T03:02:22.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.950+0000 7f24af0e4700 1 -- 172.21.15.121:0/2209134655 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a8156aa0 con 0x7f24a80a8d40 2023-11-15T03:02:22.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.950+0000 7f249e7fc700 1 -- 172.21.15.121:0/2209134655 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f24a001b400 con 0x7f24a80a8d40 2023-11-15T03:02:22.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.950+0000 7f249e7fc700 1 --2- 172.21.15.121:0/2209134655 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f248c05a510 0x7f248c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:22.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.954+0000 7f249e7fc700 1 -- 172.21.15.121:0/2209134655 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f24a0074c70 con 0x7f24a80a8d40 2023-11-15T03:02:22.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.954+0000 7f24ae0e2700 1 --2- 172.21.15.121:0/2209134655 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f248c05a510 0x7f248c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:22.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.954+0000 7f24ae0e2700 1 --2- 172.21.15.121:0/2209134655 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f248c05a510 0x7f248c05c9d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f24a4004290 tx=0x7f24a4000f40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:22.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:22.958+0000 7f249e7fc700 1 -- 172.21.15.121:0/2209134655 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f24a8156aa0 con 0x7f24a80a8d40 2023-11-15T03:02:23.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:23.158+0000 7f24af0e4700 1 -- 172.21.15.121:0/2209134655 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f24a8156aa0 con 0x7f24a80a8d40 2023-11-15T03:02:23.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:23.970+0000 7f249e7fc700 1 -- 172.21.15.121:0/2209134655 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v67) v1 ==== 122+0+0 (secure 0 0 0) 0x7f24a8156aa0 con 0x7f24a80a8d40 2023-11-15T03:02:23.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:23.974+0000 7f24af0e4700 1 -- 172.21.15.121:0/2209134655 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f248c05a510 msgr2=0x7f248c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:23.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:23.974+0000 7f24af0e4700 1 --2- 172.21.15.121:0/2209134655 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f248c05a510 0x7f248c05c9d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f24a4004290 tx=0x7f24a4000f40).stop 2023-11-15T03:02:23.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:23.974+0000 7f24af0e4700 1 -- 172.21.15.121:0/2209134655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24a80a8d40 msgr2=0x7f24a80a65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:23.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:23.974+0000 7f24af0e4700 1 --2- 172.21.15.121:0/2209134655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24a80a8d40 0x7f24a80a65d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f24a000e900 tx=0x7f24a0009060).stop 2023-11-15T03:02:23.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:23.974+0000 7f24af0e4700 1 -- 172.21.15.121:0/2209134655 shutdown_connections 2023-11-15T03:02:23.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:23.974+0000 7f24af0e4700 1 --2- 172.21.15.121:0/2209134655 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f248c05a510 0x7f248c05c9d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:23.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:23.974+0000 7f24af0e4700 1 --2- 172.21.15.121:0/2209134655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24a80a8d40 0x7f24a80a65d0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:23.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:23.974+0000 7f24af0e4700 1 --2- 172.21.15.121:0/2209134655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24a80a8960 0x7f24a8149640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:23.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:23.974+0000 7f24af0e4700 1 --2- 172.21.15.121:0/2209134655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24a8066520 0x7f24a8168d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:23.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:23.974+0000 7f24af0e4700 1 -- 172.21.15.121:0/2209134655 >> 172.21.15.121:0/2209134655 conn(0x7f24a8067a60 msgr2=0x7f24a80a9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:23.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:23.974+0000 7f24af0e4700 1 -- 172.21.15.121:0/2209134655 shutdown_connections 2023-11-15T03:02:23.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:23.974+0000 7f24af0e4700 1 -- 172.21.15.121:0/2209134655 wait complete. 2023-11-15T03:02:24.004 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-15T03:02:24.005 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:02:24.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.158+0000 7f0d6e4b9700 1 Processor -- start 2023-11-15T03:02:24.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.158+0000 7f0d6e4b9700 1 -- start start 2023-11-15T03:02:24.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d6e4b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0d6815c540 0x7f0d680c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:24.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d6e4b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d680c1d50 con 0x7f0d68152850 2023-11-15T03:02:24.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d6e4b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d6815cb80 con 0x7f0d680c3280 2023-11-15T03:02:24.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d6e4b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d680c1860 con 0x7f0d6815c540 2023-11-15T03:02:24.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d6ccb6700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f0d68152850 0x7f0d6815be30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:55100/0 (socket says 172.21.15.121:55100) 2023-11-15T03:02:24.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d6ccb6700 1 -- 172.21.15.121:0/3284987395 learned_addr learned my addr 172.21.15.121:0/3284987395 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:24.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d6dcb8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0d6815c540 0x7f0d680c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:24.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d5ffff700 1 -- 172.21.15.121:0/3284987395 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714286675 0 0) 0x7f0d680c1d50 con 0x7f0d68152850 2023-11-15T03:02:24.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d5ffff700 1 -- 172.21.15.121:0/3284987395 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d540033f0 con 0x7f0d68152850 2023-11-15T03:02:24.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d6dcb8700 1 -- 172.21.15.121:0/3284987395 >> v1:172.21.15.190:6789/0 conn(0x7f0d680c3280 legacy=0x7f0d68152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:24.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d6dcb8700 1 -- 172.21.15.121:0/3284987395 >> v1:172.21.15.121:6789/0 conn(0x7f0d68152850 legacy=0x7f0d6815be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:24.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d6dcb8700 1 -- 172.21.15.121:0/3284987395 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d680a9240 con 0x7f0d6815c540 2023-11-15T03:02:24.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d6dcb8700 1 --2- 172.21.15.121:0/3284987395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d6815c540 0x7f0d680c3660 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f0d60009e60 tx=0x7f0d60009780).ready entity=mon.0 client_cookie=20f7e0eddfc13244 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:24.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d5ffff700 1 -- 172.21.15.121:0/3284987395 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6000e070 con 0x7f0d6815c540 2023-11-15T03:02:24.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d5ffff700 1 -- 172.21.15.121:0/3284987395 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6000bb60 con 0x7f0d6815c540 2023-11-15T03:02:24.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d5ffff700 1 -- 172.21.15.121:0/3284987395 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60012410 con 0x7f0d6815c540 2023-11-15T03:02:24.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d6e4b9700 1 -- 172.21.15.121:0/3284987395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d6815c540 msgr2=0x7f0d680c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:24.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d6e4b9700 1 --2- 172.21.15.121:0/3284987395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d6815c540 0x7f0d680c3660 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f0d60009e60 tx=0x7f0d60009780).stop 2023-11-15T03:02:24.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d6e4b9700 1 -- 172.21.15.121:0/3284987395 shutdown_connections 2023-11-15T03:02:24.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d6e4b9700 1 --2- 172.21.15.121:0/3284987395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d6815c540 0x7f0d680c3660 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:24.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.162+0000 7f0d6e4b9700 1 -- 172.21.15.121:0/3284987395 >> 172.21.15.121:0/3284987395 conn(0x7f0d68062190 msgr2=0x7f0d68062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:24.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.166+0000 7f0d6e4b9700 1 -- 172.21.15.121:0/3284987395 shutdown_connections 2023-11-15T03:02:24.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.166+0000 7f0d6e4b9700 1 -- 172.21.15.121:0/3284987395 wait complete. 2023-11-15T03:02:24.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.166+0000 7f0d6e4b9700 1 Processor -- start 2023-11-15T03:02:24.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.166+0000 7f0d6e4b9700 1 -- start start 2023-11-15T03:02:24.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.166+0000 7f0d6e4b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d680c3280 0x7f0d68159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:24.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.166+0000 7f0d6e4b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d68152850 0x7f0d6814e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:24.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.166+0000 7f0d6e4b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d6815c540 0x7f0d680c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:24.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.166+0000 7f0d6e4b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6816c6d0 con 0x7f0d68152850 2023-11-15T03:02:24.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.166+0000 7f0d6e4b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6816bac0 con 0x7f0d680c3280 2023-11-15T03:02:24.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.166+0000 7f0d6e4b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6816bda0 con 0x7f0d6815c540 2023-11-15T03:02:24.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.166+0000 7f0d6ccb6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d68152850 0x7f0d6814e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:24.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.170+0000 7f0d6ccb6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d68152850 0x7f0d6814e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54726/0 (socket says 172.21.15.121:54726) 2023-11-15T03:02:24.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.170+0000 7f0d6ccb6700 1 -- 172.21.15.121:0/4258839309 learned_addr learned my addr 172.21.15.121:0/4258839309 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:24.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.170+0000 7f0d6dcb8700 1 --2- 172.21.15.121:0/4258839309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d6815c540 0x7f0d680c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:24.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.170+0000 7f0d6d4b7700 1 --2- 172.21.15.121:0/4258839309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d680c3280 0x7f0d68159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:24.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.170+0000 7f0d6ccb6700 1 -- 172.21.15.121:0/4258839309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d6815c540 msgr2=0x7f0d680c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:24.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.170+0000 7f0d6ccb6700 1 --2- 172.21.15.121:0/4258839309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d6815c540 0x7f0d680c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:24.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.170+0000 7f0d6ccb6700 1 -- 172.21.15.121:0/4258839309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d680c3280 msgr2=0x7f0d68159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:24.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.170+0000 7f0d6ccb6700 1 --2- 172.21.15.121:0/4258839309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d680c3280 0x7f0d68159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:24.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.170+0000 7f0d6ccb6700 1 -- 172.21.15.121:0/4258839309 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d60009a20 con 0x7f0d68152850 2023-11-15T03:02:24.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.170+0000 7f0d6ccb6700 1 --2- 172.21.15.121:0/4258839309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d68152850 0x7f0d6814e410 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f0d5800c590 tx=0x7f0d58007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:24.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.170+0000 7f0d5d7fa700 1 -- 172.21.15.121:0/4258839309 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d580072e0 con 0x7f0d68152850 2023-11-15T03:02:24.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.170+0000 7f0d5d7fa700 1 -- 172.21.15.121:0/4258839309 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d58007440 con 0x7f0d68152850 2023-11-15T03:02:24.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.170+0000 7f0d6e4b9700 1 -- 172.21.15.121:0/4258839309 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d6816c080 con 0x7f0d68152850 2023-11-15T03:02:24.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.170+0000 7f0d5d7fa700 1 -- 172.21.15.121:0/4258839309 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d580057a0 con 0x7f0d68152850 2023-11-15T03:02:24.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.170+0000 7f0d6e4b9700 1 -- 172.21.15.121:0/4258839309 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d6814e900 con 0x7f0d68152850 2023-11-15T03:02:24.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.170+0000 7f0d6e4b9700 1 -- 172.21.15.121:0/4258839309 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d6816c880 con 0x7f0d68152850 2023-11-15T03:02:24.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.174+0000 7f0d5d7fa700 1 -- 172.21.15.121:0/4258839309 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0d58005bd0 con 0x7f0d68152850 2023-11-15T03:02:24.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.174+0000 7f0d5d7fa700 1 --2- 172.21.15.121:0/4258839309 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0d4c05a510 0x7f0d4c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:24.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.174+0000 7f0d5d7fa700 1 -- 172.21.15.121:0/4258839309 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0d58073c60 con 0x7f0d68152850 2023-11-15T03:02:24.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.174+0000 7f0d6d4b7700 1 --2- 172.21.15.121:0/4258839309 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0d4c05a510 0x7f0d4c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:24.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.174+0000 7f0d6d4b7700 1 --2- 172.21.15.121:0/4258839309 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0d4c05a510 0x7f0d4c05c9d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f0d64003900 tx=0x7f0d64007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:24.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.182+0000 7f0d5d7fa700 1 -- 172.21.15.121:0/4258839309 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0d6816c880 con 0x7f0d68152850 2023-11-15T03:02:24.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.374+0000 7f0d6e4b9700 1 -- 172.21.15.121:0/4258839309 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d6806bcf0 con 0x7f0d68152850 2023-11-15T03:02:24.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.374+0000 7f0d5d7fa700 1 -- 172.21.15.121:0/4258839309 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f0d58018070 con 0x7f0d68152850 2023-11-15T03:02:24.383 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:24.384 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":67,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1508681155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1508681155},{"type":"v1","addr":"172.21.15.121:6837","nonce":1508681155}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6385,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3559276534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3559276534},{"type":"v1","addr":"172.21.15.190:6835","nonce":3559276534}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6486,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/834528709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":834528709},{"type":"v1","addr":"172.21.15.121:6836","nonce":834528709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:02:20.703078+0000","modified":"2023-11-15T03:02:23.967255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6492},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6492":{"gid":6492,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6833/914796342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":914796342},{"type":"v1","addr":"172.21.15.190:6833","nonce":914796342}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-15T03:02:24.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.378+0000 7f0d6e4b9700 1 -- 172.21.15.121:0/4258839309 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0d4c05a510 msgr2=0x7f0d4c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:24.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.378+0000 7f0d6e4b9700 1 --2- 172.21.15.121:0/4258839309 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0d4c05a510 0x7f0d4c05c9d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f0d64003900 tx=0x7f0d64007d40).stop 2023-11-15T03:02:24.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.378+0000 7f0d6e4b9700 1 -- 172.21.15.121:0/4258839309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d68152850 msgr2=0x7f0d6814e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:24.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.378+0000 7f0d6e4b9700 1 --2- 172.21.15.121:0/4258839309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d68152850 0x7f0d6814e410 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f0d5800c590 tx=0x7f0d58007c80).stop 2023-11-15T03:02:24.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.378+0000 7f0d6e4b9700 1 -- 172.21.15.121:0/4258839309 shutdown_connections 2023-11-15T03:02:24.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.378+0000 7f0d6e4b9700 1 --2- 172.21.15.121:0/4258839309 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0d4c05a510 0x7f0d4c05c9d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:24.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.378+0000 7f0d6e4b9700 1 --2- 172.21.15.121:0/4258839309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d6815c540 0x7f0d680c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:24.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.378+0000 7f0d6e4b9700 1 --2- 172.21.15.121:0/4258839309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d68152850 0x7f0d6814e410 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:24.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.378+0000 7f0d6e4b9700 1 --2- 172.21.15.121:0/4258839309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d680c3280 0x7f0d68159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:24.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.378+0000 7f0d6e4b9700 1 -- 172.21.15.121:0/4258839309 >> 172.21.15.121:0/4258839309 conn(0x7f0d68062190 msgr2=0x7f0d6814f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:24.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.378+0000 7f0d6e4b9700 1 -- 172.21.15.121:0/4258839309 shutdown_connections 2023-11-15T03:02:24.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.378+0000 7f0d6e4b9700 1 -- 172.21.15.121:0/4258839309 wait complete. 2023-11-15T03:02:24.389 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 67 2023-11-15T03:02:24.405 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:02:24.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.550+0000 7fb644a79700 1 Processor -- start 2023-11-15T03:02:24.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.550+0000 7fb644a79700 1 -- start start 2023-11-15T03:02:24.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.554+0000 7fb644a79700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb6400c3280 0x7fb6400c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:24.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.554+0000 7fb644a79700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb640152140 0x7fb6401525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:24.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.554+0000 7fb644a79700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6400c1d50 con 0x7fb64015be30 2023-11-15T03:02:24.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.554+0000 7fb644a79700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6400c1860 con 0x7fb640152140 2023-11-15T03:02:24.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.554+0000 7fb644a79700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb64015c990 con 0x7fb6400c3280 2023-11-15T03:02:24.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.554+0000 7fb636ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb640152140 0x7fb6401525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:24.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.554+0000 7fb636ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb640152140 0x7fb6401525f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54730/0 (socket says 172.21.15.121:54730) 2023-11-15T03:02:24.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.554+0000 7fb636ffd700 1 -- 172.21.15.121:0/2586713897 learned_addr learned my addr 172.21.15.121:0/2586713897 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:24.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.554+0000 7fb63f7fe700 1 --2- 172.21.15.121:0/2586713897 >> v2:172.21.15.190:3300/0 conn(0x7fb6400c3280 0x7fb6400c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:24.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.554+0000 7fb63ffff700 1 --1- 172.21.15.121:0/2586713897 >> v1:172.21.15.121:6789/0 conn(0x7fb64015be30 0x7fb64015c280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:55104/0 (socket says 172.21.15.121:55104) 2023-11-15T03:02:24.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.554+0000 7fb636ffd700 1 -- 172.21.15.121:0/2586713897 >> v2:172.21.15.190:3300/0 conn(0x7fb6400c3280 msgr2=0x7fb6400c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:24.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.554+0000 7fb636ffd700 1 --2- 172.21.15.121:0/2586713897 >> v2:172.21.15.190:3300/0 conn(0x7fb6400c3280 0x7fb6400c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:24.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.554+0000 7fb636ffd700 1 -- 172.21.15.121:0/2586713897 >> v1:172.21.15.121:6789/0 conn(0x7fb64015be30 legacy=0x7fb64015c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:24.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.554+0000 7fb636ffd700 1 -- 172.21.15.121:0/2586713897 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64015cb80 con 0x7fb640152140 2023-11-15T03:02:24.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.554+0000 7fb636ffd700 1 --2- 172.21.15.121:0/2586713897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb640152140 0x7fb6401525f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7fb6400c7720 tx=0x7fb6280090f0).ready entity=mon.0 client_cookie=2ddbc5143760e61e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:24.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.554+0000 7fb63effd700 1 -- 172.21.15.121:0/2586713897 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62800b520 con 0x7fb640152140 2023-11-15T03:02:24.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.554+0000 7fb63effd700 1 -- 172.21.15.121:0/2586713897 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62800b680 con 0x7fb640152140 2023-11-15T03:02:24.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.554+0000 7fb63effd700 1 -- 172.21.15.121:0/2586713897 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62800ba00 con 0x7fb640152140 2023-11-15T03:02:24.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.558+0000 7fb644a79700 1 -- 172.21.15.121:0/2586713897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb640152140 msgr2=0x7fb6401525f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:24.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.558+0000 7fb644a79700 1 --2- 172.21.15.121:0/2586713897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb640152140 0x7fb6401525f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7fb6400c7720 tx=0x7fb6280090f0).stop 2023-11-15T03:02:24.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.558+0000 7fb644a79700 1 -- 172.21.15.121:0/2586713897 shutdown_connections 2023-11-15T03:02:24.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.558+0000 7fb644a79700 1 --2- 172.21.15.121:0/2586713897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb640152140 0x7fb6401525f0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:24.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.558+0000 7fb644a79700 1 --2- 172.21.15.121:0/2586713897 >> v2:172.21.15.190:3300/0 conn(0x7fb6400c3280 0x7fb6400c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:24.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.558+0000 7fb644a79700 1 -- 172.21.15.121:0/2586713897 >> 172.21.15.121:0/2586713897 conn(0x7fb640062190 msgr2=0x7fb640062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:24.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.558+0000 7fb644a79700 1 -- 172.21.15.121:0/2586713897 shutdown_connections 2023-11-15T03:02:24.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.558+0000 7fb644a79700 1 -- 172.21.15.121:0/2586713897 wait complete. 2023-11-15T03:02:24.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.558+0000 7fb644a79700 1 Processor -- start 2023-11-15T03:02:24.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.558+0000 7fb644a79700 1 -- start start 2023-11-15T03:02:24.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb644a79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6400c3280 0x7fb6401503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:24.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb644a79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb640152140 0x7fb640168db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:24.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb644a79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb64015be30 0x7fb640155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:24.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb644a79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6401692a0 con 0x7fb640152140 2023-11-15T03:02:24.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb644a79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6400c5fb0 con 0x7fb6400c3280 2023-11-15T03:02:24.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb644a79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6400c6290 con 0x7fb64015be30 2023-11-15T03:02:24.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb636ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb640152140 0x7fb640168db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:24.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb636ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb640152140 0x7fb640168db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54746/0 (socket says 172.21.15.121:54746) 2023-11-15T03:02:24.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb636ffd700 1 -- 172.21.15.121:0/2796537016 learned_addr learned my addr 172.21.15.121:0/2796537016 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:24.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb63f7fe700 1 --2- 172.21.15.121:0/2796537016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6400c3280 0x7fb6401503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:24.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb63ffff700 1 --2- 172.21.15.121:0/2796537016 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb64015be30 0x7fb640155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:24.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb636ffd700 1 -- 172.21.15.121:0/2796537016 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb64015be30 msgr2=0x7fb640155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:24.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb636ffd700 1 --2- 172.21.15.121:0/2796537016 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb64015be30 0x7fb640155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:24.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb636ffd700 1 -- 172.21.15.121:0/2796537016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6400c3280 msgr2=0x7fb6401503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:24.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb636ffd700 1 --2- 172.21.15.121:0/2796537016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6400c3280 0x7fb6401503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:24.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb636ffd700 1 -- 172.21.15.121:0/2796537016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb628009370 con 0x7fb640152140 2023-11-15T03:02:24.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb63f7fe700 1 --2- 172.21.15.121:0/2796537016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6400c3280 0x7fb6401503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:24.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb636ffd700 1 --2- 172.21.15.121:0/2796537016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb640152140 0x7fb640168db0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7fb628009130 tx=0x7fb628017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:24.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb637fff700 1 -- 172.21.15.121:0/2796537016 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62800be90 con 0x7fb640152140 2023-11-15T03:02:24.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb637fff700 1 -- 172.21.15.121:0/2796537016 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62800b7f0 con 0x7fb640152140 2023-11-15T03:02:24.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb644a79700 1 -- 172.21.15.121:0/2796537016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6400c6570 con 0x7fb640152140 2023-11-15T03:02:24.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb637fff700 1 -- 172.21.15.121:0/2796537016 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62801c670 con 0x7fb640152140 2023-11-15T03:02:24.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.562+0000 7fb644a79700 1 -- 172.21.15.121:0/2796537016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb640156600 con 0x7fb640152140 2023-11-15T03:02:24.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.566+0000 7fb644a79700 1 -- 172.21.15.121:0/2796537016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb640156820 con 0x7fb640152140 2023-11-15T03:02:24.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.566+0000 7fb637fff700 1 -- 172.21.15.121:0/2796537016 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb62801ca20 con 0x7fb640152140 2023-11-15T03:02:24.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.566+0000 7fb637fff700 1 --2- 172.21.15.121:0/2796537016 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb62005a510 0x7fb62005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:24.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.566+0000 7fb637fff700 1 -- 172.21.15.121:0/2796537016 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb628076460 con 0x7fb640152140 2023-11-15T03:02:24.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.566+0000 7fb63f7fe700 1 --2- 172.21.15.121:0/2796537016 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb62005a510 0x7fb62005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:24.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.566+0000 7fb63f7fe700 1 --2- 172.21.15.121:0/2796537016 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb62005a510 0x7fb62005c9d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fb640159720 tx=0x7fb63000b040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:24.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.570+0000 7fb637fff700 1 -- 172.21.15.121:0/2796537016 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb628049b80 con 0x7fb640152140 2023-11-15T03:02:24.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.726+0000 7fb644a79700 1 -- 172.21.15.121:0/2796537016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb64006bcf0 con 0x7fb640152140 2023-11-15T03:02:24.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.730+0000 7fb637fff700 1 -- 172.21.15.121:0/2796537016 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16979 (secure 0 0 0) 0x7fb628020020 con 0x7fb640152140 2023-11-15T03:02:24.738 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:24.739 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":77,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:02:21.707926+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-15T03:02:18.737496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-15T03:02:19.718114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:02:24.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.734+0000 7fb644a79700 1 -- 172.21.15.121:0/2796537016 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb62005a510 msgr2=0x7fb62005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:24.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.734+0000 7fb644a79700 1 --2- 172.21.15.121:0/2796537016 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb62005a510 0x7fb62005c9d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fb640159720 tx=0x7fb63000b040).stop 2023-11-15T03:02:24.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.734+0000 7fb644a79700 1 -- 172.21.15.121:0/2796537016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb640152140 msgr2=0x7fb640168db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:24.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.734+0000 7fb644a79700 1 --2- 172.21.15.121:0/2796537016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb640152140 0x7fb640168db0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7fb628009130 tx=0x7fb628017000).stop 2023-11-15T03:02:24.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.734+0000 7fb644a79700 1 -- 172.21.15.121:0/2796537016 shutdown_connections 2023-11-15T03:02:24.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.734+0000 7fb644a79700 1 --2- 172.21.15.121:0/2796537016 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb62005a510 0x7fb62005c9d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:24.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.734+0000 7fb644a79700 1 --2- 172.21.15.121:0/2796537016 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb64015be30 0x7fb640155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:24.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.734+0000 7fb644a79700 1 --2- 172.21.15.121:0/2796537016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb640152140 0x7fb640168db0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:24.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.734+0000 7fb644a79700 1 --2- 172.21.15.121:0/2796537016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6400c3280 0x7fb6401503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:24.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.734+0000 7fb644a79700 1 -- 172.21.15.121:0/2796537016 >> 172.21.15.121:0/2796537016 conn(0x7fb640062190 msgr2=0x7fb64015abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:24.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.734+0000 7fb644a79700 1 -- 172.21.15.121:0/2796537016 shutdown_connections 2023-11-15T03:02:24.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.734+0000 7fb644a79700 1 -- 172.21.15.121:0/2796537016 wait complete. 2023-11-15T03:02:24.759 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-15T03:02:24.760 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T03:02:24.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.922+0000 7f07d041b700 1 Processor -- start 2023-11-15T03:02:24.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.922+0000 7f07d041b700 1 -- start start 2023-11-15T03:02:24.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.922+0000 7f07d041b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f07c80aa240 0x7f07c8067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:24.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.922+0000 7f07d041b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07c80cb5c0 con 0x7f07c8067f50 2023-11-15T03:02:24.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.922+0000 7f07d041b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07c80c9ca0 con 0x7f07c8153bb0 2023-11-15T03:02:24.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.922+0000 7f07d041b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07c80aa6f0 con 0x7f07c80aa240 2023-11-15T03:02:24.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.926+0000 7f07ce9b8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f07c8067f50 0x7f07c8065d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:41672/0 (socket says 172.21.15.121:41672) 2023-11-15T03:02:24.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.926+0000 7f07ce9b8700 1 -- 172.21.15.121:0/947198853 learned_addr learned my addr 172.21.15.121:0/947198853 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:24.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.926+0000 7f07cd9b6700 1 --2- 172.21.15.121:0/947198853 >> v2:172.21.15.190:3300/0 conn(0x7f07c80aa240 0x7f07c8067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:24.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.926+0000 7f07cd1b5700 1 -- 172.21.15.121:0/947198853 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227113431 0 0) 0x7f07c80c9ca0 con 0x7f07c8153bb0 2023-11-15T03:02:24.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.926+0000 7f07cd1b5700 1 -- 172.21.15.121:0/947198853 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07b40033f0 con 0x7f07c8153bb0 2023-11-15T03:02:24.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.926+0000 7f07cd1b5700 1 -- 172.21.15.121:0/947198853 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4117256748 0 0) 0x7f07c80cb5c0 con 0x7f07c8067f50 2023-11-15T03:02:24.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.926+0000 7f07cd1b5700 1 -- 172.21.15.121:0/947198853 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07c80c9ca0 con 0x7f07c8067f50 2023-11-15T03:02:24.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.926+0000 7f07cd9b6700 1 -- 172.21.15.121:0/947198853 >> v1:172.21.15.190:6789/0 conn(0x7f07c8153bb0 legacy=0x7f07c80a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:24.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.926+0000 7f07cd9b6700 1 -- 172.21.15.121:0/947198853 >> v1:172.21.15.121:6789/0 conn(0x7f07c8067f50 legacy=0x7f07c8065d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:24.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.926+0000 7f07cd9b6700 1 -- 172.21.15.121:0/947198853 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c80caa20 con 0x7f07c80aa240 2023-11-15T03:02:24.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.926+0000 7f07cd9b6700 1 --2- 172.21.15.121:0/947198853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07c80aa240 0x7f07c8067a60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f07b8009760 tx=0x7f07b8009060).ready entity=mon.1 client_cookie=ac457a09598c297 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:24.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.926+0000 7f07cd1b5700 1 -- 172.21.15.121:0/947198853 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b800d4a0 con 0x7f07c80aa240 2023-11-15T03:02:24.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.926+0000 7f07cd1b5700 1 -- 172.21.15.121:0/947198853 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b800d600 con 0x7f07c80aa240 2023-11-15T03:02:24.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.926+0000 7f07cd1b5700 1 -- 172.21.15.121:0/947198853 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b800d980 con 0x7f07c80aa240 2023-11-15T03:02:24.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.926+0000 7f07d041b700 1 -- 172.21.15.121:0/947198853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07c80aa240 msgr2=0x7f07c8067a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:24.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.926+0000 7f07d041b700 1 --2- 172.21.15.121:0/947198853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07c80aa240 0x7f07c8067a60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f07b8009760 tx=0x7f07b8009060).stop 2023-11-15T03:02:24.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.926+0000 7f07d041b700 1 -- 172.21.15.121:0/947198853 shutdown_connections 2023-11-15T03:02:24.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.926+0000 7f07d041b700 1 --2- 172.21.15.121:0/947198853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07c80aa240 0x7f07c8067a60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:24.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.926+0000 7f07d041b700 1 -- 172.21.15.121:0/947198853 >> 172.21.15.121:0/947198853 conn(0x7f07c8058c50 msgr2=0x7f07c8059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:24.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.930+0000 7f07d041b700 1 -- 172.21.15.121:0/947198853 shutdown_connections 2023-11-15T03:02:24.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.930+0000 7f07d041b700 1 -- 172.21.15.121:0/947198853 wait complete. 2023-11-15T03:02:24.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.930+0000 7f07d041b700 1 Processor -- start 2023-11-15T03:02:24.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.930+0000 7f07d041b700 1 -- start start 2023-11-15T03:02:24.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.930+0000 7f07d041b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07c8067f50 0x7f07c814ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:24.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.930+0000 7f07d041b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07c80aa240 0x7f07c80c1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:24.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.930+0000 7f07d041b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07c8153bb0 0x7f07c80c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:24.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.930+0000 7f07d041b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c80c3890 con 0x7f07c8153bb0 2023-11-15T03:02:24.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.930+0000 7f07d041b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c80c2f00 con 0x7f07c80aa240 2023-11-15T03:02:24.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.930+0000 7f07d041b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c80c22f0 con 0x7f07c8067f50 2023-11-15T03:02:24.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.934+0000 7f07ce9b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07c8153bb0 0x7f07c80c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:24.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.934+0000 7f07ce9b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07c8153bb0 0x7f07c80c3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40388/0 (socket says 172.21.15.121:40388) 2023-11-15T03:02:24.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.934+0000 7f07ce9b8700 1 -- 172.21.15.121:0/364537679 learned_addr learned my addr 172.21.15.121:0/364537679 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:24.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.934+0000 7f07cd9b6700 1 --2- 172.21.15.121:0/364537679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07c80aa240 0x7f07c80c1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:24.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.934+0000 7f07ce1b7700 1 --2- 172.21.15.121:0/364537679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07c8067f50 0x7f07c814ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:24.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.934+0000 7f07ce9b8700 1 -- 172.21.15.121:0/364537679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07c8067f50 msgr2=0x7f07c814ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:24.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.934+0000 7f07ce9b8700 1 --2- 172.21.15.121:0/364537679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07c8067f50 0x7f07c814ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:24.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.934+0000 7f07ce9b8700 1 -- 172.21.15.121:0/364537679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07c80aa240 msgr2=0x7f07c80c1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:24.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.934+0000 7f07ce9b8700 1 --2- 172.21.15.121:0/364537679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07c80aa240 0x7f07c80c1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:24.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.934+0000 7f07ce9b8700 1 -- 172.21.15.121:0/364537679 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b80092a0 con 0x7f07c8153bb0 2023-11-15T03:02:24.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.934+0000 7f07ce9b8700 1 --2- 172.21.15.121:0/364537679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07c8153bb0 0x7f07c80c3bf0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f07c0005ad0 tx=0x7f07c0001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:24.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.934+0000 7f07be7fc700 1 -- 172.21.15.121:0/364537679 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c0005d30 con 0x7f07c8153bb0 2023-11-15T03:02:24.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.934+0000 7f07be7fc700 1 -- 172.21.15.121:0/364537679 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c0005e90 con 0x7f07c8153bb0 2023-11-15T03:02:24.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.934+0000 7f07d041b700 1 -- 172.21.15.121:0/364537679 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c80c25d0 con 0x7f07c8153bb0 2023-11-15T03:02:24.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.934+0000 7f07be7fc700 1 -- 172.21.15.121:0/364537679 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c0007320 con 0x7f07c8153bb0 2023-11-15T03:02:24.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.934+0000 7f07d041b700 1 -- 172.21.15.121:0/364537679 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c80c2b90 con 0x7f07c8153bb0 2023-11-15T03:02:24.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.934+0000 7f07d041b700 1 -- 172.21.15.121:0/364537679 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c80c3090 con 0x7f07c8153bb0 2023-11-15T03:02:24.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.938+0000 7f07be7fc700 1 -- 172.21.15.121:0/364537679 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f07c0007710 con 0x7f07c8153bb0 2023-11-15T03:02:24.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.938+0000 7f07be7fc700 1 --2- 172.21.15.121:0/364537679 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f07ac05a510 0x7f07ac05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:24.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.938+0000 7f07be7fc700 1 -- 172.21.15.121:0/364537679 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f07c00747b0 con 0x7f07c8153bb0 2023-11-15T03:02:24.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.938+0000 7f07ce1b7700 1 --2- 172.21.15.121:0/364537679 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f07ac05a510 0x7f07ac05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:24.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.938+0000 7f07ce1b7700 1 --2- 172.21.15.121:0/364537679 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f07ac05a510 0x7f07ac05c9d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f07c4003660 tx=0x7f07c4009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:24.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:24.942+0000 7f07be7fc700 1 -- 172.21.15.121:0/364537679 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f07c0047ef0 con 0x7f07c8153bb0 2023-11-15T03:02:25.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.114+0000 7f07d041b700 1 -- 172.21.15.121:0/364537679 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f07c806c150 con 0x7f07ac05a510 2023-11-15T03:02:25.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.130+0000 7f07be7fc700 1 -- 172.21.15.121:0/364537679 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+350407 (secure 0 0 0) 0x7f07c806c150 con 0x7f07ac05a510 2023-11-15T03:02:25.148 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:25.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.142+0000 7f07d041b700 1 -- 172.21.15.121:0/364537679 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f07ac05a510 msgr2=0x7f07ac05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:25.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.142+0000 7f07d041b700 1 --2- 172.21.15.121:0/364537679 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f07ac05a510 0x7f07ac05c9d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f07c4003660 tx=0x7f07c4009040).stop 2023-11-15T03:02:25.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.142+0000 7f07d041b700 1 -- 172.21.15.121:0/364537679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07c8153bb0 msgr2=0x7f07c80c3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:25.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.142+0000 7f07d041b700 1 --2- 172.21.15.121:0/364537679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07c8153bb0 0x7f07c80c3bf0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f07c0005ad0 tx=0x7f07c0001650).stop 2023-11-15T03:02:25.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.142+0000 7f07d041b700 1 -- 172.21.15.121:0/364537679 shutdown_connections 2023-11-15T03:02:25.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.142+0000 7f07d041b700 1 --2- 172.21.15.121:0/364537679 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f07ac05a510 0x7f07ac05c9d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.142+0000 7f07d041b700 1 --2- 172.21.15.121:0/364537679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07c8153bb0 0x7f07c80c3bf0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.142+0000 7f07d041b700 1 --2- 172.21.15.121:0/364537679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07c80aa240 0x7f07c80c1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.142+0000 7f07d041b700 1 --2- 172.21.15.121:0/364537679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07c8067f50 0x7f07c814ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.142+0000 7f07d041b700 1 -- 172.21.15.121:0/364537679 >> 172.21.15.121:0/364537679 conn(0x7f07c8058c50 msgr2=0x7f07c80c5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:25.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.142+0000 7f07d041b700 1 -- 172.21.15.121:0/364537679 shutdown_connections 2023-11-15T03:02:25.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.142+0000 7f07d041b700 1 -- 172.21.15.121:0/364537679 wait complete. 2023-11-15T03:02:25.157 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T03:02:25.170 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2023-11-15T03:02:24.040452+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":892,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":144,"acting":144,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2383800,"kb_used_data":7032,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752590920,"statfs":{"total":773094113280,"available":770653102080,"internally_reserved":0,"allocated":7200768,"data_stored":3264589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25972,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-99,"num_read_kb":-123,"num_write":-91,"num_write_kb":-53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.130555"},"pg_stats":[{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247222+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250336+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251487+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745613+0000","last_change":"2023-11-15T03:02:20.282870+0000","last_active":"2023-11-15T03:02:20.745613+0000","last_peered":"2023-11-15T03:02:20.745613+0000","last_clean":"2023-11-15T03:02:20.745613+0000","last_became_active":"2023-11-15T03:02:20.282610+0000","last_became_peered":"2023-11-15T03:02:20.282610+0000","last_unstale":"2023-11-15T03:02:20.745613+0000","last_undegraded":"2023-11-15T03:02:20.745613+0000","last_fullsized":"2023-11-15T03:02:20.745613+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.246707+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745400+0000","last_change":"2023-11-15T03:02:20.730582+0000","last_active":"2023-11-15T03:02:20.745400+0000","last_peered":"2023-11-15T03:02:20.745400+0000","last_clean":"2023-11-15T03:02:20.745400+0000","last_became_active":"2023-11-15T03:02:20.730317+0000","last_became_peered":"2023-11-15T03:02:20.730317+0000","last_unstale":"2023-11-15T03:02:20.745400+0000","last_undegraded":"2023-11-15T03:02:20.745400+0000","last_fullsized":"2023-11-15T03:02:20.745400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250208+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250406+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.248379+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251917+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247687+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"76'4","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.754390+0000","last_change":"2023-11-15T03:02:20.280107+0000","last_active":"2023-11-15T03:02:20.754390+0000","last_peered":"2023-11-15T03:02:20.754390+0000","last_clean":"2023-11-15T03:02:20.754390+0000","last_became_active":"2023-11-15T03:02:20.279672+0000","last_became_peered":"2023-11-15T03:02:20.279672+0000","last_unstale":"2023-11-15T03:02:20.754390+0000","last_undegraded":"2023-11-15T03:02:20.754390+0000","last_fullsized":"2023-11-15T03:02:20.754390+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250882+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247684+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250911+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251733+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247901+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.714183+0000","last_change":"2023-11-15T03:02:20.282164+0000","last_active":"2023-11-15T03:02:20.714183+0000","last_peered":"2023-11-15T03:02:20.714183+0000","last_clean":"2023-11-15T03:02:20.714183+0000","last_became_active":"2023-11-15T03:02:20.281841+0000","last_became_peered":"2023-11-15T03:02:20.281841+0000","last_unstale":"2023-11-15T03:02:20.714183+0000","last_undegraded":"2023-11-15T03:02:20.714183+0000","last_fullsized":"2023-11-15T03:02:20.714183+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250066+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250037+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251700+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251055+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.748655+0000","last_change":"2023-11-15T03:02:20.740354+0000","last_active":"2023-11-15T03:02:20.748655+0000","last_peered":"2023-11-15T03:02:20.748655+0000","last_clean":"2023-11-15T03:02:20.748655+0000","last_became_active":"2023-11-15T03:02:20.739907+0000","last_became_peered":"2023-11-15T03:02:20.739907+0000","last_unstale":"2023-11-15T03:02:20.748655+0000","last_undegraded":"2023-11-15T03:02:20.748655+0000","last_fullsized":"2023-11-15T03:02:20.748655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.244098+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.244535+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250911+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.749278+0000","last_change":"2023-11-15T03:02:20.745921+0000","last_active":"2023-11-15T03:02:20.749278+0000","last_peered":"2023-11-15T03:02:20.749278+0000","last_clean":"2023-11-15T03:02:20.749278+0000","last_became_active":"2023-11-15T03:02:20.745541+0000","last_became_peered":"2023-11-15T03:02:20.745541+0000","last_unstale":"2023-11-15T03:02:20.749278+0000","last_undegraded":"2023-11-15T03:02:20.749278+0000","last_fullsized":"2023-11-15T03:02:20.749278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.249601+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250911+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251299+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.249643+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.747882+0000","last_change":"2023-11-15T03:02:20.736251+0000","last_active":"2023-11-15T03:02:20.747882+0000","last_peered":"2023-11-15T03:02:20.747882+0000","last_clean":"2023-11-15T03:02:20.747882+0000","last_became_active":"2023-11-15T03:02:20.735947+0000","last_became_peered":"2023-11-15T03:02:20.735947+0000","last_unstale":"2023-11-15T03:02:20.747882+0000","last_undegraded":"2023-11-15T03:02:20.747882+0000","last_fullsized":"2023-11-15T03:02:20.747882+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247315+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.245886+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247139+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251558+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251224+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250451+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.246576+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.246709+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247598+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.748847+0000","last_change":"2023-11-15T03:02:20.740969+0000","last_active":"2023-11-15T03:02:20.748847+0000","last_peered":"2023-11-15T03:02:20.748847+0000","last_clean":"2023-11-15T03:02:20.748847+0000","last_became_active":"2023-11-15T03:02:20.740724+0000","last_became_peered":"2023-11-15T03:02:20.740724+0000","last_unstale":"2023-11-15T03:02:20.748847+0000","last_undegraded":"2023-11-15T03:02:20.748847+0000","last_fullsized":"2023-11-15T03:02:20.748847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250474+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.244374+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247757+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.249509+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.249500+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.246525+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.245327+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250012+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.246591+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745673+0000","last_change":"2023-11-15T03:02:20.730485+0000","last_active":"2023-11-15T03:02:20.745673+0000","last_peered":"2023-11-15T03:02:20.745673+0000","last_clean":"2023-11-15T03:02:20.745673+0000","last_became_active":"2023-11-15T03:02:20.730219+0000","last_became_peered":"2023-11-15T03:02:20.730219+0000","last_unstale":"2023-11-15T03:02:20.745673+0000","last_undegraded":"2023-11-15T03:02:20.745673+0000","last_fullsized":"2023-11-15T03:02:20.745673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.249729+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.748129+0000","last_change":"2023-11-15T03:02:20.737437+0000","last_active":"2023-11-15T03:02:20.748129+0000","last_peered":"2023-11-15T03:02:20.748129+0000","last_clean":"2023-11-15T03:02:20.748129+0000","last_became_active":"2023-11-15T03:02:20.737189+0000","last_became_peered":"2023-11-15T03:02:20.737189+0000","last_unstale":"2023-11-15T03:02:20.748129+0000","last_undegraded":"2023-11-15T03:02:20.748129+0000","last_fullsized":"2023-11-15T03:02:20.748129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.712554+0000","last_change":"2023-11-15T03:02:20.277917+0000","last_active":"2023-11-15T03:02:20.712554+0000","last_peered":"2023-11-15T03:02:20.712554+0000","last_clean":"2023-11-15T03:02:20.712554+0000","last_became_active":"2023-11-15T03:02:20.275072+0000","last_became_peered":"2023-11-15T03:02:20.275072+0000","last_unstale":"2023-11-15T03:02:20.712554+0000","last_undegraded":"2023-11-15T03:02:20.712554+0000","last_fullsized":"2023-11-15T03:02:20.712554+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251321+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-15T03:02:17.441603+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T03:02:17.441603+0000","last_peered":"2023-11-15T03:02:17.441603+0000","last_clean":"2023-11-15T03:02:17.441603+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T03:02:17.441603+0000","last_undegraded":"2023-11-15T03:02:17.441603+0000","last_fullsized":"2023-11-15T03:02:17.441603+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.246191+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.714328+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T03:02:20.714328+0000","last_peered":"2023-11-15T03:02:20.714328+0000","last_clean":"2023-11-15T03:02:20.714328+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T03:02:20.714328+0000","last_undegraded":"2023-11-15T03:02:20.714328+0000","last_fullsized":"2023-11-15T03:02:20.714328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.712972+0000","last_change":"2023-11-15T03:02:20.279939+0000","last_active":"2023-11-15T03:02:20.712972+0000","last_peered":"2023-11-15T03:02:20.712972+0000","last_clean":"2023-11-15T03:02:20.712972+0000","last_became_active":"2023-11-15T03:02:20.279319+0000","last_became_peered":"2023-11-15T03:02:20.279319+0000","last_unstale":"2023-11-15T03:02:20.712972+0000","last_undegraded":"2023-11-15T03:02:20.712972+0000","last_fullsized":"2023-11-15T03:02:20.712972+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":55,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-15T03:02:18.435968+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T03:02:18.435968+0000","last_peered":"2023-11-15T03:02:18.435968+0000","last_clean":"2023-11-15T03:02:18.435968+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T03:02:18.435968+0000","last_undegraded":"2023-11-15T03:02:18.435968+0000","last_fullsized":"2023-11-15T03:02:18.435968+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247085+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.713011+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T03:02:20.713011+0000","last_peered":"2023-11-15T03:02:20.713011+0000","last_clean":"2023-11-15T03:02:20.713011+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T03:02:20.713011+0000","last_undegraded":"2023-11-15T03:02:20.713011+0000","last_fullsized":"2023-11-15T03:02:20.713011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250745+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":65,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-15T03:02:15.330804+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T03:02:15.330804+0000","last_peered":"2023-11-15T03:02:15.330804+0000","last_clean":"2023-11-15T03:02:15.330804+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T03:02:15.330804+0000","last_undegraded":"2023-11-15T03:02:15.330804+0000","last_fullsized":"2023-11-15T03:02:15.330804+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.249036+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-15T03:02:16.993728+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T03:02:16.993728+0000","last_peered":"2023-11-15T03:02:16.993728+0000","last_clean":"2023-11-15T03:02:16.993728+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T03:02:16.993728+0000","last_undegraded":"2023-11-15T03:02:16.993728+0000","last_fullsized":"2023-11-15T03:02:16.993728+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.248909+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":55,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-15T03:02:18.435964+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T03:02:18.435964+0000","last_peered":"2023-11-15T03:02:18.435964+0000","last_clean":"2023-11-15T03:02:18.435964+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T03:02:18.435964+0000","last_undegraded":"2023-11-15T03:02:18.435964+0000","last_fullsized":"2023-11-15T03:02:18.435964+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.248786+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":65,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-15T03:02:15.330789+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T03:02:15.330789+0000","last_peered":"2023-11-15T03:02:15.330789+0000","last_clean":"2023-11-15T03:02:15.330789+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T03:02:15.330789+0000","last_undegraded":"2023-11-15T03:02:15.330789+0000","last_fullsized":"2023-11-15T03:02:15.330789+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":56,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-15T03:02:18.435958+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T03:02:18.435958+0000","last_peered":"2023-11-15T03:02:18.435958+0000","last_clean":"2023-11-15T03:02:18.435958+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T03:02:18.435958+0000","last_undegraded":"2023-11-15T03:02:18.435958+0000","last_fullsized":"2023-11-15T03:02:18.435958+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":892,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":112,"acting":112,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542213,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297972,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073868,"statfs":{"total":96636764160,"available":96331640832,"internally_reserved":0,"allocated":897024,"data_stored":407083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.181,"15min":1.1910000000000001},"min":{"1min":0.72399999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3360000000000001,"5min":10.961,"15min":10.961},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.141,"15min":1.1890000000000001},"min":{"1min":0.876,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.331,"5min":3.4910000000000001,"15min":3.4910000000000001},"last":1.5589999999999999}]},{"osd":1,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.161,"15min":1.153},"min":{"1min":0.83099999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.4219999999999999,"5min":5.4930000000000003,"15min":5.4930000000000003},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2050000000000001,"15min":1.1879999999999999},"min":{"1min":0.79400000000000004,"5min":0.747,"15min":0.747},"max":{"1min":1.4830000000000001,"5min":6.8239999999999998,"15min":6.8239999999999998},"last":1.6970000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.1679999999999999,"15min":1.218},"min":{"1min":0.76300000000000001,"5min":0.751,"15min":0.751},"max":{"1min":1.49,"5min":5.7000000000000002,"15min":5.7000000000000002},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2430000000000001,"15min":1.2889999999999999},"min":{"1min":0.93600000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.597,"5min":2.75,"15min":2.75},"last":1.5449999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.335,"15min":1.3720000000000001},"min":{"1min":1.014,"5min":0.751,"15min":0.751},"max":{"1min":1.6220000000000001,"5min":12.907,"15min":12.907},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.3160000000000001,"15min":1.298},"min":{"1min":0.995,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6679999999999999,"5min":9.9909999999999997,"15min":9.9909999999999997},"last":1.4950000000000001}]},{"osd":4,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.139,"15min":1.2569999999999999},"min":{"1min":0.86599999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.3360000000000001,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2230000000000001,"15min":1.4239999999999999},"min":{"1min":0.85399999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5640000000000001,"5min":4.226,"15min":4.226},"last":1.6080000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.91000000000000003,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.5580000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.298,"15min":1.427},"min":{"1min":0.96899999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.609,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":2.1299999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.403,"15min":1.548},"min":{"1min":1.115,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.619,"5min":3.2789999999999999,"15min":3.2789999999999999},"last":1.365},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.359,"15min":1.5669999999999999},"min":{"1min":0.93400000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6719999999999999,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":1.3520000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542213,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297972,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073868,"statfs":{"total":96636764160,"available":96331640832,"internally_reserved":0,"allocated":897024,"data_stored":407083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.0980000000000001,"15min":1.1240000000000001},"min":{"1min":0.80700000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.74,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.073,"15min":1.05},"min":{"1min":0.874,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.802,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.3380000000000001}]},{"osd":1,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.125,"15min":1.095},"min":{"1min":0.86899999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.512,"5min":1.623,"15min":1.623},"last":1.851},{"interface":"front","average":{"1min":1.123,"5min":1.1439999999999999,"15min":1.1599999999999999},"min":{"1min":0.83899999999999997,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.407,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.7390000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2030000000000001,"15min":1.1830000000000001},"min":{"1min":0.96499999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.7190000000000001,"5min":2.621,"15min":2.621},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.1399999999999999,"15min":1.099},"min":{"1min":0.93899999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4690000000000001,"5min":1.5,"15min":1.5},"last":2.0059999999999998}]},{"osd":3,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2070000000000001,"15min":1.1699999999999999},"min":{"1min":1.0469999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.452,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.004},{"interface":"front","average":{"1min":1.373,"5min":1.286,"15min":1.272},"min":{"1min":0.96199999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7569999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.3009999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.097,"15min":1.1659999999999999},"min":{"1min":0.63500000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.3879999999999999,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":1.9610000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.181,"15min":1.1850000000000001},"min":{"1min":0.91300000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.831,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":2.0739999999999998}]},{"osd":5,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4279999999999999,"15min":1.4139999999999999},"min":{"1min":1.1919999999999999,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.621,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":2.157},{"interface":"front","average":{"1min":1.175,"5min":1.262,"15min":1.302},"min":{"1min":0.83599999999999997,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.492,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":2.1749999999999998}]},{"osd":7,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2170000000000001,"15min":1.28},"min":{"1min":0.76200000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4570000000000001,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.796},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.4219999999999999,"15min":1.4470000000000001},"min":{"1min":0.98099999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.681,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":1.4870000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542214,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297972,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073868,"statfs":{"total":96636764160,"available":96331640832,"internally_reserved":0,"allocated":897024,"data_stored":407083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0940000000000001,"15min":1.1080000000000001},"min":{"1min":0.68200000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.355,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.345},{"interface":"front","average":{"1min":1.125,"5min":1.077,"15min":1.026},"min":{"1min":0.85799999999999998,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.3819999999999999,"5min":1.677,"15min":1.677},"last":1.149}]},{"osd":1,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.1479999999999999,"15min":1.212},"min":{"1min":0.83999999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3089999999999999,"5min":2.617,"15min":2.617},"last":1.105},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1639999999999999,"15min":1.1659999999999999},"min":{"1min":0.88100000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.431,"5min":1.657,"15min":1.657},"last":1.262}]},{"osd":2,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.204,"15min":1.1579999999999999},"min":{"1min":0.98899999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4730000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.171,"5min":1.1519999999999999,"15min":1.161},"min":{"1min":0.82899999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.4590000000000001,"5min":1.667,"15min":1.667},"last":1.2310000000000001}]},{"osd":3,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.141,"15min":1.1559999999999999},"min":{"1min":0.91100000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.401,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.22},{"interface":"front","average":{"1min":1.214,"5min":1.276,"15min":1.294},"min":{"1min":1.04,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.4319999999999999,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":1.3720000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1819999999999999,"15min":1.232},"min":{"1min":0.752,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.4399999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.105,"5min":1.194,"15min":1.224},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.5149999999999999,"5min":1.994,"15min":1.994},"last":1.284}]},{"osd":6,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.28,"15min":1.325},"min":{"1min":0.877,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.494,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":1.296},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.341,"15min":1.3959999999999999},"min":{"1min":0.96999999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.52,"5min":2.173,"15min":2.173},"last":1.1879999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3160000000000001,"15min":1.3640000000000001},"min":{"1min":0.96499999999999997,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.486,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":1.331},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3819999999999999,"15min":1.4279999999999999},"min":{"1min":0.96499999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.512,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":1.427}]}]},{"osd":3,"up_from":14,"seq":60129542214,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297972,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073868,"statfs":{"total":96636764160,"available":96331640832,"internally_reserved":0,"allocated":897024,"data_stored":407083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.81999999999999995,"15min":0.77400000000000002},"min":{"1min":0.68300000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.02,"5min":1.169,"15min":1.169},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.87,"15min":0.83799999999999997},"min":{"1min":0.64000000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.244,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.39800000000000002}]},{"osd":1,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89100000000000001,"15min":0.89800000000000002},"min":{"1min":0.72899999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.163,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.006,"15min":0.97799999999999998},"min":{"1min":0.89600000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2549999999999999,"5min":1.375,"15min":1.375},"last":0.69699999999999995}]},{"osd":2,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.012,"15min":0.97999999999999998},"min":{"1min":0.75600000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4259999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.099,"5min":1.0920000000000001,"15min":1.0609999999999999},"min":{"1min":0.83099999999999996,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3009999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2310000000000001,"15min":1.1950000000000001},"min":{"1min":1.089,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.542,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":1.21,"5min":1.206,"15min":1.222},"min":{"1min":1.0049999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.4370000000000001,"5min":1.659,"15min":1.659},"last":0.621}]},{"osd":5,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":1.0269999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.4790000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.288,"5min":1.2689999999999999,"15min":1.2649999999999999},"min":{"1min":1.0509999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.512,"5min":1.829,"15min":1.829},"last":0.63500000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3340000000000001,"15min":1.3129999999999999},"min":{"1min":1.143,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":1.5620000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.383,"5min":1.3779999999999999,"15min":1.359},"min":{"1min":1.1399999999999999,"5min":1.004,"15min":1.004},"max":{"1min":1.581,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.77700000000000002}]},{"osd":7,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.357,"15min":1.3779999999999999},"min":{"1min":1.173,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.5800000000000001,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.361,"5min":1.3640000000000001,"15min":1.357},"min":{"1min":1.1339999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"max":{"1min":1.514,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.66600000000000004}]}]},{"osd":2,"up_from":14,"seq":60129542214,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297968,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073872,"statfs":{"total":96636764160,"available":96331644928,"internally_reserved":0,"allocated":892928,"data_stored":405814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78100000000000003,"15min":0.76900000000000002},"min":{"1min":0.63700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.1479999999999999,"5min":1.409,"15min":1.409},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.85299999999999998,"15min":0.871},"min":{"1min":0.70199999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.339,"5min":1.619,"15min":1.619},"last":0.47399999999999998}]},{"osd":1,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.92400000000000004,"15min":0.90200000000000002},"min":{"1min":0.73399999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.014,"5min":0.97499999999999998,"15min":0.95299999999999996},"min":{"1min":0.81699999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.238,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.70199999999999996}]},{"osd":3,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.98999999999999999,"15min":0.996},"min":{"1min":0.626,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.347,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":1.2,"5min":1.1379999999999999,"15min":1.1240000000000001},"min":{"1min":0.92600000000000005,"5min":0.622,"15min":0.622},"max":{"1min":1.5580000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.044}]},{"osd":4,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.218,"15min":1.2130000000000001},"min":{"1min":1.133,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6719999999999999,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.673},{"interface":"front","average":{"1min":1.228,"5min":1.2270000000000001,"15min":1.226},"min":{"1min":1.03,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5029999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.7589999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2070000000000001,"15min":1.246},"min":{"1min":0.93999999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.554,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.276,"15min":1.29},"min":{"1min":1.0780000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.702,"5min":1.857,"15min":1.857},"last":1.3839999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.288,"15min":1.347},"min":{"1min":1.0109999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.679,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.147},{"interface":"front","average":{"1min":1.272,"5min":1.2789999999999999,"15min":1.331},"min":{"1min":0.92400000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.524,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.127}]},{"osd":7,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3460000000000001,"15min":1.375},"min":{"1min":1.081,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.821,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3069999999999999,"15min":1.3680000000000001},"min":{"1min":1.1140000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.635,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.80200000000000005}]}]},{"osd":0,"up_from":14,"seq":60129542214,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297960,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073880,"statfs":{"total":96636764160,"available":96331653120,"internally_reserved":0,"allocated":884736,"data_stored":400862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.85699999999999998,"15min":0.871},"min":{"1min":0.61499999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4390000000000001,"5min":4.3630000000000004,"15min":4.3630000000000004},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.97399999999999998,"15min":0.92200000000000004},"min":{"1min":0.67600000000000005,"5min":0.441,"15min":0.441},"max":{"1min":1.3520000000000001,"5min":6.157,"15min":6.157},"last":0.71699999999999997}]},{"osd":2,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0109999999999999,"15min":1.0189999999999999},"min":{"1min":0.76800000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.571,"5min":4.1699999999999999,"15min":4.1699999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.046,"15min":1.006},"min":{"1min":0.75600000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4099999999999999,"5min":6.8369999999999997,"15min":6.8369999999999997},"last":1.087}]},{"osd":3,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.97599999999999998,"15min":0.96599999999999997},"min":{"1min":0.69599999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7330000000000001,"5min":3.5070000000000001,"15min":3.5070000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.143,"15min":1.1699999999999999},"min":{"1min":0.90400000000000003,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.55,"5min":5.2149999999999999,"15min":5.2149999999999999},"last":1.286}]},{"osd":4,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.2949999999999999,"15min":1.2629999999999999},"min":{"1min":1.0369999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6479999999999999,"5min":6.2000000000000002,"15min":6.2000000000000002},"last":1.216},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2270000000000001,"15min":1.1930000000000001},"min":{"1min":0.874,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.927,"5min":3.577,"15min":3.577},"last":1.006}]},{"osd":5,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.262,"15min":1.2330000000000001},"min":{"1min":0.97499999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9850000000000001,"5min":3.548,"15min":3.548},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.3220000000000001,"15min":1.3520000000000001},"min":{"1min":1.0840000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.6899999999999999,"5min":4.4080000000000004,"15min":4.4080000000000004},"last":1.675}]},{"osd":6,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.365,"15min":1.4139999999999999},"min":{"1min":1.1910000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.528,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3660000000000001,"15min":1.3540000000000001},"min":{"1min":1.0369999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9970000000000001,"5min":6.9690000000000003,"15min":6.9690000000000003},"last":1.151}]},{"osd":7,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.506,"15min":1.4259999999999999},"min":{"1min":0.94599999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.9390000000000001,"5min":18.253,"15min":18.253},"last":1.377},{"interface":"front","average":{"1min":1.343,"5min":1.4099999999999999,"15min":1.3100000000000001},"min":{"1min":1.129,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8280000000000001,"5min":11.006,"15min":11.006},"last":1.238}]}]},{"osd":5,"up_from":13,"seq":55834574918,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297960,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073880,"statfs":{"total":96636764160,"available":96331653120,"internally_reserved":0,"allocated":884736,"data_stored":400862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.081,"15min":1.056},"min":{"1min":0.81299999999999994,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4690000000000001,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.012,"5min":1.091,"15min":1.1950000000000001},"min":{"1min":0.86799999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3360000000000001,"5min":3.46,"15min":3.46},"last":0.89500000000000002}]},{"osd":1,"last update":"Wed Nov 15 03:01:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.127,"15min":1.224},"min":{"1min":0.75800000000000001,"5min":0.626,"15min":0.626},"max":{"1min":1.3460000000000001,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1499999999999999,"15min":1.1799999999999999},"min":{"1min":0.85399999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.407,"5min":3.516,"15min":3.516},"last":1.02}]},{"osd":2,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.181,"15min":1.2050000000000001},"min":{"1min":0.84799999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.7370000000000001,"5min":5.4180000000000001,"15min":5.4180000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.234,"5min":1.218,"15min":1.2909999999999999},"min":{"1min":0.877,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.746,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":1.077}]},{"osd":3,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1990000000000001,"15min":1.2769999999999999},"min":{"1min":0.90600000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.5049999999999999,"5min":2.141,"15min":2.141},"last":0.999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.339,"15min":1.427},"min":{"1min":1.0069999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.786,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":1.224}]},{"osd":4,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2749999999999999,"15min":1.357},"min":{"1min":0.86599999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.554,"5min":3.706,"15min":3.706},"last":1.173},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3380000000000001,"15min":1.552},"min":{"1min":0.84599999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5920000000000001,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":1.3420000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.262,"15min":1.3200000000000001},"min":{"1min":0.94299999999999995,"5min":0.749,"15min":0.749},"max":{"1min":1.5189999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.365,"15min":1.526},"min":{"1min":1.038,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.6659999999999999,"5min":3.641,"15min":3.641},"last":1.2869999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3109999999999999,"15min":1.399},"min":{"1min":0.995,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.635,"5min":3.516,"15min":3.516},"last":1.25},{"interface":"front","average":{"1min":1.417,"5min":1.4079999999999999,"15min":1.448},"min":{"1min":0.94899999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.7909999999999999,"5min":3.754,"15min":3.754},"last":1.377}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298024,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073816,"statfs":{"total":96636764160,"available":96331587584,"internally_reserved":0,"allocated":950272,"data_stored":428719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.82399999999999995,"15min":0.83799999999999997},"min":{"1min":0.63700000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.103,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.89400000000000002,"15min":0.88200000000000001},"min":{"1min":0.63,"5min":0.495,"15min":0.495},"max":{"1min":1.1950000000000001,"5min":1.419,"15min":1.419},"last":1.123}]},{"osd":2,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.91400000000000003,"15min":0.86599999999999999},"min":{"1min":0.70999999999999996,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94299999999999995,"15min":0.92200000000000004},"min":{"1min":0.66100000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.653,"5min":1.887,"15min":1.887},"last":0.99199999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.137,"15min":1.079},"min":{"1min":0.66600000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0629999999999999,"15min":1.0009999999999999},"min":{"1min":0.80800000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.444,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.92300000000000004}]},{"osd":4,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.1850000000000001,"15min":1.169},"min":{"1min":0.94299999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.341,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.2},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2370000000000001,"15min":1.2549999999999999},"min":{"1min":1.024,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.522,"5min":2.206,"15min":2.206},"last":1.254}]},{"osd":5,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2410000000000001,"15min":1.2589999999999999},"min":{"1min":1.0329999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.5049999999999999,"5min":1.899,"15min":1.899},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2669999999999999,"15min":1.2849999999999999},"min":{"1min":0.99399999999999999,"5min":0.871,"15min":0.871},"max":{"1min":1.4039999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.329}]},{"osd":6,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.335},"min":{"1min":1.127,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.726,"5min":1.948,"15min":1.948},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.401,"15min":1.409},"min":{"1min":1.054,"5min":0.87,"15min":0.87},"max":{"1min":1.637,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.3979999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3400000000000001,"15min":1.3240000000000001},"min":{"1min":1.0800000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.659,"5min":2.161,"15min":2.161},"last":1.427},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3520000000000001,"15min":1.359},"min":{"1min":1.0700000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.573,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.4810000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T03:02:25.182 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T03:02:25.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.362+0000 7f1055747700 1 Processor -- start 2023-11-15T03:02:25.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.362+0000 7f1055747700 1 -- start start 2023-11-15T03:02:25.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f1055747700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1050066d70 0x7f10500a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:25.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f1055747700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1050067150 0x7f10501518b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:25.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f1055747700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f10500c29a0 0x7f105014e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:25.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f1055747700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1050150ed0 con 0x7f10500c29a0 2023-11-15T03:02:25.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f1055747700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10500675d0 con 0x7f1050066d70 2023-11-15T03:02:25.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f1055747700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10500a9350 con 0x7f1050067150 2023-11-15T03:02:25.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f1054f46700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f10500c29a0 0x7f105014e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:25.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f1054f46700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f10500c29a0 0x7f105014e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40398/0 (socket says 172.21.15.121:40398) 2023-11-15T03:02:25.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f1054f46700 1 -- 172.21.15.121:0/2821445356 learned_addr learned my addr 172.21.15.121:0/2821445356 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:25.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f104ffff700 1 --2- 172.21.15.121:0/2821445356 >> v2:172.21.15.190:3300/0 conn(0x7f1050066d70 0x7f10500a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:25.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f104f7fe700 1 --2- 172.21.15.121:0/2821445356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1050067150 0x7f10501518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:25.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f1054f46700 1 -- 172.21.15.121:0/2821445356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1050067150 msgr2=0x7f10501518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:25.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f1054f46700 1 --2- 172.21.15.121:0/2821445356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1050067150 0x7f10501518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f1054f46700 1 -- 172.21.15.121:0/2821445356 >> v2:172.21.15.190:3300/0 conn(0x7f1050066d70 msgr2=0x7f10500a87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:25.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f1054f46700 1 --2- 172.21.15.121:0/2821445356 >> v2:172.21.15.190:3300/0 conn(0x7f1050066d70 0x7f10500a87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f1054f46700 1 -- 172.21.15.121:0/2821445356 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1050151420 con 0x7f10500c29a0 2023-11-15T03:02:25.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f104ffff700 1 --2- 172.21.15.121:0/2821445356 >> v2:172.21.15.190:3300/0 conn(0x7f1050066d70 0x7f10500a87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:25.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f1054f46700 1 --2- 172.21.15.121:0/2821445356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10500c29a0 0x7f105014e000 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f1050064da0 tx=0x7f1048009840).ready entity=mon.0 client_cookie=60b4927ddb2a3c37 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:25.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f104effd700 1 -- 172.21.15.121:0/2821445356 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104800e070 con 0x7f10500c29a0 2023-11-15T03:02:25.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f104effd700 1 -- 172.21.15.121:0/2821445356 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104800ba50 con 0x7f10500c29a0 2023-11-15T03:02:25.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.366+0000 7f104effd700 1 -- 172.21.15.121:0/2821445356 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104800bdd0 con 0x7f10500c29a0 2023-11-15T03:02:25.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.370+0000 7f1055747700 1 -- 172.21.15.121:0/2821445356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10500c29a0 msgr2=0x7f105014e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:25.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.370+0000 7f1055747700 1 --2- 172.21.15.121:0/2821445356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10500c29a0 0x7f105014e000 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f1050064da0 tx=0x7f1048009840).stop 2023-11-15T03:02:25.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.370+0000 7f1055747700 1 -- 172.21.15.121:0/2821445356 shutdown_connections 2023-11-15T03:02:25.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.370+0000 7f1055747700 1 --2- 172.21.15.121:0/2821445356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10500c29a0 0x7f105014e000 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.370+0000 7f1055747700 1 --2- 172.21.15.121:0/2821445356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1050067150 0x7f10501518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.370+0000 7f1055747700 1 --2- 172.21.15.121:0/2821445356 >> v2:172.21.15.190:3300/0 conn(0x7f1050066d70 0x7f10500a87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.370+0000 7f1055747700 1 -- 172.21.15.121:0/2821445356 >> 172.21.15.121:0/2821445356 conn(0x7f1050062190 msgr2=0x7f1050062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:25.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.370+0000 7f1055747700 1 -- 172.21.15.121:0/2821445356 shutdown_connections 2023-11-15T03:02:25.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.370+0000 7f1055747700 1 -- 172.21.15.121:0/2821445356 wait complete. 2023-11-15T03:02:25.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.374+0000 7f1055747700 1 Processor -- start 2023-11-15T03:02:25.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.374+0000 7f1055747700 1 -- start start 2023-11-15T03:02:25.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.374+0000 7f1055747700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1050066d70 0x7f105014ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:25.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.374+0000 7f1055747700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1050067150 0x7f105014f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:25.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.374+0000 7f1055747700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10500c29a0 0x7f105014fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:25.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.374+0000 7f1055747700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105014f0c0 con 0x7f10500c29a0 2023-11-15T03:02:25.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.374+0000 7f1055747700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105014e4b0 con 0x7f1050066d70 2023-11-15T03:02:25.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.374+0000 7f1055747700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105014e790 con 0x7f1050067150 2023-11-15T03:02:25.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.374+0000 7f1054f46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10500c29a0 0x7f105014fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:25.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.374+0000 7f1054f46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10500c29a0 0x7f105014fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40406/0 (socket says 172.21.15.121:40406) 2023-11-15T03:02:25.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.374+0000 7f1054f46700 1 -- 172.21.15.121:0/3789985617 learned_addr learned my addr 172.21.15.121:0/3789985617 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:25.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.374+0000 7f104ffff700 1 --2- 172.21.15.121:0/3789985617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1050066d70 0x7f105014ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:25.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.374+0000 7f104f7fe700 1 --2- 172.21.15.121:0/3789985617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1050067150 0x7f105014f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:25.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.374+0000 7f1054f46700 1 -- 172.21.15.121:0/3789985617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1050067150 msgr2=0x7f105014f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:25.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.374+0000 7f1054f46700 1 --2- 172.21.15.121:0/3789985617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1050067150 0x7f105014f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.374+0000 7f1054f46700 1 -- 172.21.15.121:0/3789985617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1050066d70 msgr2=0x7f105014ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:25.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.374+0000 7f1054f46700 1 --2- 172.21.15.121:0/3789985617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1050066d70 0x7f105014ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.374+0000 7f1054f46700 1 -- 172.21.15.121:0/3789985617 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1048009ae0 con 0x7f10500c29a0 2023-11-15T03:02:25.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.378+0000 7f1054f46700 1 --2- 172.21.15.121:0/3789985617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10500c29a0 0x7f105014fa40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f1050065200 tx=0x7f1050151390).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:25.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.378+0000 7f1047fff700 1 -- 172.21.15.121:0/3789985617 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048013ea0 con 0x7f10500c29a0 2023-11-15T03:02:25.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.378+0000 7f1047fff700 1 -- 172.21.15.121:0/3789985617 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10480032f0 con 0x7f10500c29a0 2023-11-15T03:02:25.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.378+0000 7f1047fff700 1 -- 172.21.15.121:0/3789985617 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048007860 con 0x7f10500c29a0 2023-11-15T03:02:25.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.378+0000 7f1055747700 1 -- 172.21.15.121:0/3789985617 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105014ea70 con 0x7f10500c29a0 2023-11-15T03:02:25.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.378+0000 7f1055747700 1 -- 172.21.15.121:0/3789985617 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105014b200 con 0x7f10500c29a0 2023-11-15T03:02:25.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.378+0000 7f1055747700 1 -- 172.21.15.121:0/3789985617 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105014f270 con 0x7f10500c29a0 2023-11-15T03:02:25.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.378+0000 7f1047fff700 1 -- 172.21.15.121:0/3789985617 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1048003040 con 0x7f10500c29a0 2023-11-15T03:02:25.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.382+0000 7f1047fff700 1 --2- 172.21.15.121:0/3789985617 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f103005a510 0x7f103005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:25.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.382+0000 7f1047fff700 1 -- 172.21.15.121:0/3789985617 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1048076300 con 0x7f10500c29a0 2023-11-15T03:02:25.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.382+0000 7f104ffff700 1 --2- 172.21.15.121:0/3789985617 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f103005a510 0x7f103005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:25.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.382+0000 7f104ffff700 1 --2- 172.21.15.121:0/3789985617 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f103005a510 0x7f103005c9d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f1040006fd0 tx=0x7f1040008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:25.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.390+0000 7f1047fff700 1 -- 172.21.15.121:0/3789985617 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f105014f270 con 0x7f10500c29a0 2023-11-15T03:02:25.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.550+0000 7f1055747700 1 -- 172.21.15.121:0/3789985617 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f105006bcf0 con 0x7f103005a510 2023-11-15T03:02:25.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.562+0000 7f1047fff700 1 -- 172.21.15.121:0/3789985617 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+350407 (secure 0 0 0) 0x7f105006bcf0 con 0x7f103005a510 2023-11-15T03:02:25.576 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:25.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.570+0000 7f1055747700 1 -- 172.21.15.121:0/3789985617 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f103005a510 msgr2=0x7f103005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:25.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.570+0000 7f1055747700 1 --2- 172.21.15.121:0/3789985617 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f103005a510 0x7f103005c9d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f1040006fd0 tx=0x7f1040008040).stop 2023-11-15T03:02:25.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.570+0000 7f1055747700 1 -- 172.21.15.121:0/3789985617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10500c29a0 msgr2=0x7f105014fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:25.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.570+0000 7f1055747700 1 --2- 172.21.15.121:0/3789985617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10500c29a0 0x7f105014fa40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f1050065200 tx=0x7f1050151390).stop 2023-11-15T03:02:25.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.570+0000 7f1055747700 1 -- 172.21.15.121:0/3789985617 shutdown_connections 2023-11-15T03:02:25.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.570+0000 7f1055747700 1 --2- 172.21.15.121:0/3789985617 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f103005a510 0x7f103005c9d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.570+0000 7f1055747700 1 --2- 172.21.15.121:0/3789985617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10500c29a0 0x7f105014fa40 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.570+0000 7f1055747700 1 --2- 172.21.15.121:0/3789985617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1050067150 0x7f105014f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.570+0000 7f1055747700 1 --2- 172.21.15.121:0/3789985617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1050066d70 0x7f105014ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.570+0000 7f1055747700 1 -- 172.21.15.121:0/3789985617 >> 172.21.15.121:0/3789985617 conn(0x7f1050062190 msgr2=0x7f10500ab0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:25.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.570+0000 7f1055747700 1 -- 172.21.15.121:0/3789985617 shutdown_connections 2023-11-15T03:02:25.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.570+0000 7f1055747700 1 -- 172.21.15.121:0/3789985617 wait complete. 2023-11-15T03:02:25.583 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T03:02:25.598 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2023-11-15T03:02:24.040452+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":892,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":144,"acting":144,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2383800,"kb_used_data":7032,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752590920,"statfs":{"total":773094113280,"available":770653102080,"internally_reserved":0,"allocated":7200768,"data_stored":3264589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25972,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-99,"num_read_kb":-123,"num_write":-91,"num_write_kb":-53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.130555"},"pg_stats":[{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247222+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250336+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251487+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745613+0000","last_change":"2023-11-15T03:02:20.282870+0000","last_active":"2023-11-15T03:02:20.745613+0000","last_peered":"2023-11-15T03:02:20.745613+0000","last_clean":"2023-11-15T03:02:20.745613+0000","last_became_active":"2023-11-15T03:02:20.282610+0000","last_became_peered":"2023-11-15T03:02:20.282610+0000","last_unstale":"2023-11-15T03:02:20.745613+0000","last_undegraded":"2023-11-15T03:02:20.745613+0000","last_fullsized":"2023-11-15T03:02:20.745613+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.246707+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745400+0000","last_change":"2023-11-15T03:02:20.730582+0000","last_active":"2023-11-15T03:02:20.745400+0000","last_peered":"2023-11-15T03:02:20.745400+0000","last_clean":"2023-11-15T03:02:20.745400+0000","last_became_active":"2023-11-15T03:02:20.730317+0000","last_became_peered":"2023-11-15T03:02:20.730317+0000","last_unstale":"2023-11-15T03:02:20.745400+0000","last_undegraded":"2023-11-15T03:02:20.745400+0000","last_fullsized":"2023-11-15T03:02:20.745400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250208+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250406+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.248379+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251917+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247687+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"76'4","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.754390+0000","last_change":"2023-11-15T03:02:20.280107+0000","last_active":"2023-11-15T03:02:20.754390+0000","last_peered":"2023-11-15T03:02:20.754390+0000","last_clean":"2023-11-15T03:02:20.754390+0000","last_became_active":"2023-11-15T03:02:20.279672+0000","last_became_peered":"2023-11-15T03:02:20.279672+0000","last_unstale":"2023-11-15T03:02:20.754390+0000","last_undegraded":"2023-11-15T03:02:20.754390+0000","last_fullsized":"2023-11-15T03:02:20.754390+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250882+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247684+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250911+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251733+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247901+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.714183+0000","last_change":"2023-11-15T03:02:20.282164+0000","last_active":"2023-11-15T03:02:20.714183+0000","last_peered":"2023-11-15T03:02:20.714183+0000","last_clean":"2023-11-15T03:02:20.714183+0000","last_became_active":"2023-11-15T03:02:20.281841+0000","last_became_peered":"2023-11-15T03:02:20.281841+0000","last_unstale":"2023-11-15T03:02:20.714183+0000","last_undegraded":"2023-11-15T03:02:20.714183+0000","last_fullsized":"2023-11-15T03:02:20.714183+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250066+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250037+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251700+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251055+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.748655+0000","last_change":"2023-11-15T03:02:20.740354+0000","last_active":"2023-11-15T03:02:20.748655+0000","last_peered":"2023-11-15T03:02:20.748655+0000","last_clean":"2023-11-15T03:02:20.748655+0000","last_became_active":"2023-11-15T03:02:20.739907+0000","last_became_peered":"2023-11-15T03:02:20.739907+0000","last_unstale":"2023-11-15T03:02:20.748655+0000","last_undegraded":"2023-11-15T03:02:20.748655+0000","last_fullsized":"2023-11-15T03:02:20.748655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.244098+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.244535+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250911+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.749278+0000","last_change":"2023-11-15T03:02:20.745921+0000","last_active":"2023-11-15T03:02:20.749278+0000","last_peered":"2023-11-15T03:02:20.749278+0000","last_clean":"2023-11-15T03:02:20.749278+0000","last_became_active":"2023-11-15T03:02:20.745541+0000","last_became_peered":"2023-11-15T03:02:20.745541+0000","last_unstale":"2023-11-15T03:02:20.749278+0000","last_undegraded":"2023-11-15T03:02:20.749278+0000","last_fullsized":"2023-11-15T03:02:20.749278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.249601+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250911+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251299+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.249643+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.747882+0000","last_change":"2023-11-15T03:02:20.736251+0000","last_active":"2023-11-15T03:02:20.747882+0000","last_peered":"2023-11-15T03:02:20.747882+0000","last_clean":"2023-11-15T03:02:20.747882+0000","last_became_active":"2023-11-15T03:02:20.735947+0000","last_became_peered":"2023-11-15T03:02:20.735947+0000","last_unstale":"2023-11-15T03:02:20.747882+0000","last_undegraded":"2023-11-15T03:02:20.747882+0000","last_fullsized":"2023-11-15T03:02:20.747882+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247315+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.245886+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247139+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251558+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251224+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250451+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.246576+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.246709+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247598+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.748847+0000","last_change":"2023-11-15T03:02:20.740969+0000","last_active":"2023-11-15T03:02:20.748847+0000","last_peered":"2023-11-15T03:02:20.748847+0000","last_clean":"2023-11-15T03:02:20.748847+0000","last_became_active":"2023-11-15T03:02:20.740724+0000","last_became_peered":"2023-11-15T03:02:20.740724+0000","last_unstale":"2023-11-15T03:02:20.748847+0000","last_undegraded":"2023-11-15T03:02:20.748847+0000","last_fullsized":"2023-11-15T03:02:20.748847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250474+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.244374+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247757+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.249509+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.249500+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.246525+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.245327+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250012+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.246591+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745673+0000","last_change":"2023-11-15T03:02:20.730485+0000","last_active":"2023-11-15T03:02:20.745673+0000","last_peered":"2023-11-15T03:02:20.745673+0000","last_clean":"2023-11-15T03:02:20.745673+0000","last_became_active":"2023-11-15T03:02:20.730219+0000","last_became_peered":"2023-11-15T03:02:20.730219+0000","last_unstale":"2023-11-15T03:02:20.745673+0000","last_undegraded":"2023-11-15T03:02:20.745673+0000","last_fullsized":"2023-11-15T03:02:20.745673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.249729+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.748129+0000","last_change":"2023-11-15T03:02:20.737437+0000","last_active":"2023-11-15T03:02:20.748129+0000","last_peered":"2023-11-15T03:02:20.748129+0000","last_clean":"2023-11-15T03:02:20.748129+0000","last_became_active":"2023-11-15T03:02:20.737189+0000","last_became_peered":"2023-11-15T03:02:20.737189+0000","last_unstale":"2023-11-15T03:02:20.748129+0000","last_undegraded":"2023-11-15T03:02:20.748129+0000","last_fullsized":"2023-11-15T03:02:20.748129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.712554+0000","last_change":"2023-11-15T03:02:20.277917+0000","last_active":"2023-11-15T03:02:20.712554+0000","last_peered":"2023-11-15T03:02:20.712554+0000","last_clean":"2023-11-15T03:02:20.712554+0000","last_became_active":"2023-11-15T03:02:20.275072+0000","last_became_peered":"2023-11-15T03:02:20.275072+0000","last_unstale":"2023-11-15T03:02:20.712554+0000","last_undegraded":"2023-11-15T03:02:20.712554+0000","last_fullsized":"2023-11-15T03:02:20.712554+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251321+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-15T03:02:17.441603+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T03:02:17.441603+0000","last_peered":"2023-11-15T03:02:17.441603+0000","last_clean":"2023-11-15T03:02:17.441603+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T03:02:17.441603+0000","last_undegraded":"2023-11-15T03:02:17.441603+0000","last_fullsized":"2023-11-15T03:02:17.441603+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.246191+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.714328+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T03:02:20.714328+0000","last_peered":"2023-11-15T03:02:20.714328+0000","last_clean":"2023-11-15T03:02:20.714328+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T03:02:20.714328+0000","last_undegraded":"2023-11-15T03:02:20.714328+0000","last_fullsized":"2023-11-15T03:02:20.714328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.712972+0000","last_change":"2023-11-15T03:02:20.279939+0000","last_active":"2023-11-15T03:02:20.712972+0000","last_peered":"2023-11-15T03:02:20.712972+0000","last_clean":"2023-11-15T03:02:20.712972+0000","last_became_active":"2023-11-15T03:02:20.279319+0000","last_became_peered":"2023-11-15T03:02:20.279319+0000","last_unstale":"2023-11-15T03:02:20.712972+0000","last_undegraded":"2023-11-15T03:02:20.712972+0000","last_fullsized":"2023-11-15T03:02:20.712972+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":55,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-15T03:02:18.435968+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T03:02:18.435968+0000","last_peered":"2023-11-15T03:02:18.435968+0000","last_clean":"2023-11-15T03:02:18.435968+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T03:02:18.435968+0000","last_undegraded":"2023-11-15T03:02:18.435968+0000","last_fullsized":"2023-11-15T03:02:18.435968+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247085+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.713011+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T03:02:20.713011+0000","last_peered":"2023-11-15T03:02:20.713011+0000","last_clean":"2023-11-15T03:02:20.713011+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T03:02:20.713011+0000","last_undegraded":"2023-11-15T03:02:20.713011+0000","last_fullsized":"2023-11-15T03:02:20.713011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250745+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":65,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-15T03:02:15.330804+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T03:02:15.330804+0000","last_peered":"2023-11-15T03:02:15.330804+0000","last_clean":"2023-11-15T03:02:15.330804+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T03:02:15.330804+0000","last_undegraded":"2023-11-15T03:02:15.330804+0000","last_fullsized":"2023-11-15T03:02:15.330804+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.249036+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-15T03:02:16.993728+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T03:02:16.993728+0000","last_peered":"2023-11-15T03:02:16.993728+0000","last_clean":"2023-11-15T03:02:16.993728+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T03:02:16.993728+0000","last_undegraded":"2023-11-15T03:02:16.993728+0000","last_fullsized":"2023-11-15T03:02:16.993728+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.248909+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":55,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-15T03:02:18.435964+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T03:02:18.435964+0000","last_peered":"2023-11-15T03:02:18.435964+0000","last_clean":"2023-11-15T03:02:18.435964+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T03:02:18.435964+0000","last_undegraded":"2023-11-15T03:02:18.435964+0000","last_fullsized":"2023-11-15T03:02:18.435964+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.248786+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":65,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-15T03:02:15.330789+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T03:02:15.330789+0000","last_peered":"2023-11-15T03:02:15.330789+0000","last_clean":"2023-11-15T03:02:15.330789+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T03:02:15.330789+0000","last_undegraded":"2023-11-15T03:02:15.330789+0000","last_fullsized":"2023-11-15T03:02:15.330789+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":56,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-15T03:02:18.435958+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T03:02:18.435958+0000","last_peered":"2023-11-15T03:02:18.435958+0000","last_clean":"2023-11-15T03:02:18.435958+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T03:02:18.435958+0000","last_undegraded":"2023-11-15T03:02:18.435958+0000","last_fullsized":"2023-11-15T03:02:18.435958+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":892,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":112,"acting":112,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542213,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297972,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073868,"statfs":{"total":96636764160,"available":96331640832,"internally_reserved":0,"allocated":897024,"data_stored":407083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.181,"15min":1.1910000000000001},"min":{"1min":0.72399999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3360000000000001,"5min":10.961,"15min":10.961},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.141,"15min":1.1890000000000001},"min":{"1min":0.876,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.331,"5min":3.4910000000000001,"15min":3.4910000000000001},"last":1.5589999999999999}]},{"osd":1,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.161,"15min":1.153},"min":{"1min":0.83099999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.4219999999999999,"5min":5.4930000000000003,"15min":5.4930000000000003},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2050000000000001,"15min":1.1879999999999999},"min":{"1min":0.79400000000000004,"5min":0.747,"15min":0.747},"max":{"1min":1.4830000000000001,"5min":6.8239999999999998,"15min":6.8239999999999998},"last":1.6970000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.1679999999999999,"15min":1.218},"min":{"1min":0.76300000000000001,"5min":0.751,"15min":0.751},"max":{"1min":1.49,"5min":5.7000000000000002,"15min":5.7000000000000002},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2430000000000001,"15min":1.2889999999999999},"min":{"1min":0.93600000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.597,"5min":2.75,"15min":2.75},"last":1.5449999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.335,"15min":1.3720000000000001},"min":{"1min":1.014,"5min":0.751,"15min":0.751},"max":{"1min":1.6220000000000001,"5min":12.907,"15min":12.907},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.3160000000000001,"15min":1.298},"min":{"1min":0.995,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6679999999999999,"5min":9.9909999999999997,"15min":9.9909999999999997},"last":1.4950000000000001}]},{"osd":4,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.139,"15min":1.2569999999999999},"min":{"1min":0.86599999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.3360000000000001,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2230000000000001,"15min":1.4239999999999999},"min":{"1min":0.85399999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5640000000000001,"5min":4.226,"15min":4.226},"last":1.6080000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.91000000000000003,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.5580000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.298,"15min":1.427},"min":{"1min":0.96899999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.609,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":2.1299999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.403,"15min":1.548},"min":{"1min":1.115,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.619,"5min":3.2789999999999999,"15min":3.2789999999999999},"last":1.365},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.359,"15min":1.5669999999999999},"min":{"1min":0.93400000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6719999999999999,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":1.3520000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542213,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297972,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073868,"statfs":{"total":96636764160,"available":96331640832,"internally_reserved":0,"allocated":897024,"data_stored":407083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.0980000000000001,"15min":1.1240000000000001},"min":{"1min":0.80700000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.74,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.073,"15min":1.05},"min":{"1min":0.874,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.802,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.3380000000000001}]},{"osd":1,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.125,"15min":1.095},"min":{"1min":0.86899999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.512,"5min":1.623,"15min":1.623},"last":1.851},{"interface":"front","average":{"1min":1.123,"5min":1.1439999999999999,"15min":1.1599999999999999},"min":{"1min":0.83899999999999997,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.407,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.7390000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2030000000000001,"15min":1.1830000000000001},"min":{"1min":0.96499999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.7190000000000001,"5min":2.621,"15min":2.621},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.1399999999999999,"15min":1.099},"min":{"1min":0.93899999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4690000000000001,"5min":1.5,"15min":1.5},"last":2.0059999999999998}]},{"osd":3,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2070000000000001,"15min":1.1699999999999999},"min":{"1min":1.0469999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.452,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.004},{"interface":"front","average":{"1min":1.373,"5min":1.286,"15min":1.272},"min":{"1min":0.96199999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7569999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.3009999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.097,"15min":1.1659999999999999},"min":{"1min":0.63500000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.3879999999999999,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":1.9610000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.181,"15min":1.1850000000000001},"min":{"1min":0.91300000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.831,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":2.0739999999999998}]},{"osd":5,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4279999999999999,"15min":1.4139999999999999},"min":{"1min":1.1919999999999999,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.621,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":2.157},{"interface":"front","average":{"1min":1.175,"5min":1.262,"15min":1.302},"min":{"1min":0.83599999999999997,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.492,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":2.1749999999999998}]},{"osd":7,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2170000000000001,"15min":1.28},"min":{"1min":0.76200000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4570000000000001,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.796},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.4219999999999999,"15min":1.4470000000000001},"min":{"1min":0.98099999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.681,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":1.4870000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542214,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297972,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073868,"statfs":{"total":96636764160,"available":96331640832,"internally_reserved":0,"allocated":897024,"data_stored":407083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0940000000000001,"15min":1.1080000000000001},"min":{"1min":0.68200000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.355,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.345},{"interface":"front","average":{"1min":1.125,"5min":1.077,"15min":1.026},"min":{"1min":0.85799999999999998,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.3819999999999999,"5min":1.677,"15min":1.677},"last":1.149}]},{"osd":1,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.1479999999999999,"15min":1.212},"min":{"1min":0.83999999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3089999999999999,"5min":2.617,"15min":2.617},"last":1.105},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1639999999999999,"15min":1.1659999999999999},"min":{"1min":0.88100000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.431,"5min":1.657,"15min":1.657},"last":1.262}]},{"osd":2,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.204,"15min":1.1579999999999999},"min":{"1min":0.98899999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4730000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.171,"5min":1.1519999999999999,"15min":1.161},"min":{"1min":0.82899999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.4590000000000001,"5min":1.667,"15min":1.667},"last":1.2310000000000001}]},{"osd":3,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.141,"15min":1.1559999999999999},"min":{"1min":0.91100000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.401,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.22},{"interface":"front","average":{"1min":1.214,"5min":1.276,"15min":1.294},"min":{"1min":1.04,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.4319999999999999,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":1.3720000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1819999999999999,"15min":1.232},"min":{"1min":0.752,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.4399999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.105,"5min":1.194,"15min":1.224},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.5149999999999999,"5min":1.994,"15min":1.994},"last":1.284}]},{"osd":6,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.28,"15min":1.325},"min":{"1min":0.877,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.494,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":1.296},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.341,"15min":1.3959999999999999},"min":{"1min":0.96999999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.52,"5min":2.173,"15min":2.173},"last":1.1879999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3160000000000001,"15min":1.3640000000000001},"min":{"1min":0.96499999999999997,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.486,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":1.331},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3819999999999999,"15min":1.4279999999999999},"min":{"1min":0.96499999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.512,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":1.427}]}]},{"osd":3,"up_from":14,"seq":60129542214,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297972,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073868,"statfs":{"total":96636764160,"available":96331640832,"internally_reserved":0,"allocated":897024,"data_stored":407083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.81999999999999995,"15min":0.77400000000000002},"min":{"1min":0.68300000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.02,"5min":1.169,"15min":1.169},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.87,"15min":0.83799999999999997},"min":{"1min":0.64000000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.244,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.39800000000000002}]},{"osd":1,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89100000000000001,"15min":0.89800000000000002},"min":{"1min":0.72899999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.163,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.006,"15min":0.97799999999999998},"min":{"1min":0.89600000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2549999999999999,"5min":1.375,"15min":1.375},"last":0.69699999999999995}]},{"osd":2,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.012,"15min":0.97999999999999998},"min":{"1min":0.75600000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4259999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.099,"5min":1.0920000000000001,"15min":1.0609999999999999},"min":{"1min":0.83099999999999996,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3009999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2310000000000001,"15min":1.1950000000000001},"min":{"1min":1.089,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.542,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":1.21,"5min":1.206,"15min":1.222},"min":{"1min":1.0049999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.4370000000000001,"5min":1.659,"15min":1.659},"last":0.621}]},{"osd":5,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":1.0269999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.4790000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.288,"5min":1.2689999999999999,"15min":1.2649999999999999},"min":{"1min":1.0509999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.512,"5min":1.829,"15min":1.829},"last":0.63500000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3340000000000001,"15min":1.3129999999999999},"min":{"1min":1.143,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":1.5620000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.383,"5min":1.3779999999999999,"15min":1.359},"min":{"1min":1.1399999999999999,"5min":1.004,"15min":1.004},"max":{"1min":1.581,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.77700000000000002}]},{"osd":7,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.357,"15min":1.3779999999999999},"min":{"1min":1.173,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.5800000000000001,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.361,"5min":1.3640000000000001,"15min":1.357},"min":{"1min":1.1339999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"max":{"1min":1.514,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.66600000000000004}]}]},{"osd":2,"up_from":14,"seq":60129542214,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297968,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073872,"statfs":{"total":96636764160,"available":96331644928,"internally_reserved":0,"allocated":892928,"data_stored":405814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78100000000000003,"15min":0.76900000000000002},"min":{"1min":0.63700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.1479999999999999,"5min":1.409,"15min":1.409},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.85299999999999998,"15min":0.871},"min":{"1min":0.70199999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.339,"5min":1.619,"15min":1.619},"last":0.47399999999999998}]},{"osd":1,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.92400000000000004,"15min":0.90200000000000002},"min":{"1min":0.73399999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.014,"5min":0.97499999999999998,"15min":0.95299999999999996},"min":{"1min":0.81699999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.238,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.70199999999999996}]},{"osd":3,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.98999999999999999,"15min":0.996},"min":{"1min":0.626,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.347,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":1.2,"5min":1.1379999999999999,"15min":1.1240000000000001},"min":{"1min":0.92600000000000005,"5min":0.622,"15min":0.622},"max":{"1min":1.5580000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.044}]},{"osd":4,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.218,"15min":1.2130000000000001},"min":{"1min":1.133,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6719999999999999,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.673},{"interface":"front","average":{"1min":1.228,"5min":1.2270000000000001,"15min":1.226},"min":{"1min":1.03,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5029999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.7589999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2070000000000001,"15min":1.246},"min":{"1min":0.93999999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.554,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.276,"15min":1.29},"min":{"1min":1.0780000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.702,"5min":1.857,"15min":1.857},"last":1.3839999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.288,"15min":1.347},"min":{"1min":1.0109999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.679,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.147},{"interface":"front","average":{"1min":1.272,"5min":1.2789999999999999,"15min":1.331},"min":{"1min":0.92400000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.524,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.127}]},{"osd":7,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3460000000000001,"15min":1.375},"min":{"1min":1.081,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.821,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3069999999999999,"15min":1.3680000000000001},"min":{"1min":1.1140000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.635,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.80200000000000005}]}]},{"osd":0,"up_from":14,"seq":60129542214,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297960,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073880,"statfs":{"total":96636764160,"available":96331653120,"internally_reserved":0,"allocated":884736,"data_stored":400862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.85699999999999998,"15min":0.871},"min":{"1min":0.61499999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4390000000000001,"5min":4.3630000000000004,"15min":4.3630000000000004},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.97399999999999998,"15min":0.92200000000000004},"min":{"1min":0.67600000000000005,"5min":0.441,"15min":0.441},"max":{"1min":1.3520000000000001,"5min":6.157,"15min":6.157},"last":0.71699999999999997}]},{"osd":2,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0109999999999999,"15min":1.0189999999999999},"min":{"1min":0.76800000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.571,"5min":4.1699999999999999,"15min":4.1699999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.046,"15min":1.006},"min":{"1min":0.75600000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4099999999999999,"5min":6.8369999999999997,"15min":6.8369999999999997},"last":1.087}]},{"osd":3,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.97599999999999998,"15min":0.96599999999999997},"min":{"1min":0.69599999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7330000000000001,"5min":3.5070000000000001,"15min":3.5070000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.143,"15min":1.1699999999999999},"min":{"1min":0.90400000000000003,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.55,"5min":5.2149999999999999,"15min":5.2149999999999999},"last":1.286}]},{"osd":4,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.2949999999999999,"15min":1.2629999999999999},"min":{"1min":1.0369999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6479999999999999,"5min":6.2000000000000002,"15min":6.2000000000000002},"last":1.216},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2270000000000001,"15min":1.1930000000000001},"min":{"1min":0.874,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.927,"5min":3.577,"15min":3.577},"last":1.006}]},{"osd":5,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.262,"15min":1.2330000000000001},"min":{"1min":0.97499999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9850000000000001,"5min":3.548,"15min":3.548},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.3220000000000001,"15min":1.3520000000000001},"min":{"1min":1.0840000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.6899999999999999,"5min":4.4080000000000004,"15min":4.4080000000000004},"last":1.675}]},{"osd":6,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.365,"15min":1.4139999999999999},"min":{"1min":1.1910000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.528,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3660000000000001,"15min":1.3540000000000001},"min":{"1min":1.0369999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9970000000000001,"5min":6.9690000000000003,"15min":6.9690000000000003},"last":1.151}]},{"osd":7,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.506,"15min":1.4259999999999999},"min":{"1min":0.94599999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.9390000000000001,"5min":18.253,"15min":18.253},"last":1.377},{"interface":"front","average":{"1min":1.343,"5min":1.4099999999999999,"15min":1.3100000000000001},"min":{"1min":1.129,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8280000000000001,"5min":11.006,"15min":11.006},"last":1.238}]}]},{"osd":5,"up_from":13,"seq":55834574918,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297960,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073880,"statfs":{"total":96636764160,"available":96331653120,"internally_reserved":0,"allocated":884736,"data_stored":400862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.081,"15min":1.056},"min":{"1min":0.81299999999999994,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4690000000000001,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.012,"5min":1.091,"15min":1.1950000000000001},"min":{"1min":0.86799999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3360000000000001,"5min":3.46,"15min":3.46},"last":0.89500000000000002}]},{"osd":1,"last update":"Wed Nov 15 03:01:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.127,"15min":1.224},"min":{"1min":0.75800000000000001,"5min":0.626,"15min":0.626},"max":{"1min":1.3460000000000001,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1499999999999999,"15min":1.1799999999999999},"min":{"1min":0.85399999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.407,"5min":3.516,"15min":3.516},"last":1.02}]},{"osd":2,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.181,"15min":1.2050000000000001},"min":{"1min":0.84799999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.7370000000000001,"5min":5.4180000000000001,"15min":5.4180000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.234,"5min":1.218,"15min":1.2909999999999999},"min":{"1min":0.877,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.746,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":1.077}]},{"osd":3,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1990000000000001,"15min":1.2769999999999999},"min":{"1min":0.90600000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.5049999999999999,"5min":2.141,"15min":2.141},"last":0.999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.339,"15min":1.427},"min":{"1min":1.0069999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.786,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":1.224}]},{"osd":4,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2749999999999999,"15min":1.357},"min":{"1min":0.86599999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.554,"5min":3.706,"15min":3.706},"last":1.173},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3380000000000001,"15min":1.552},"min":{"1min":0.84599999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5920000000000001,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":1.3420000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.262,"15min":1.3200000000000001},"min":{"1min":0.94299999999999995,"5min":0.749,"15min":0.749},"max":{"1min":1.5189999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.365,"15min":1.526},"min":{"1min":1.038,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.6659999999999999,"5min":3.641,"15min":3.641},"last":1.2869999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3109999999999999,"15min":1.399},"min":{"1min":0.995,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.635,"5min":3.516,"15min":3.516},"last":1.25},{"interface":"front","average":{"1min":1.417,"5min":1.4079999999999999,"15min":1.448},"min":{"1min":0.94899999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.7909999999999999,"5min":3.754,"15min":3.754},"last":1.377}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298024,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073816,"statfs":{"total":96636764160,"available":96331587584,"internally_reserved":0,"allocated":950272,"data_stored":428719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.82399999999999995,"15min":0.83799999999999997},"min":{"1min":0.63700000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.103,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.89400000000000002,"15min":0.88200000000000001},"min":{"1min":0.63,"5min":0.495,"15min":0.495},"max":{"1min":1.1950000000000001,"5min":1.419,"15min":1.419},"last":1.123}]},{"osd":2,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.91400000000000003,"15min":0.86599999999999999},"min":{"1min":0.70999999999999996,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94299999999999995,"15min":0.92200000000000004},"min":{"1min":0.66100000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.653,"5min":1.887,"15min":1.887},"last":0.99199999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.137,"15min":1.079},"min":{"1min":0.66600000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0629999999999999,"15min":1.0009999999999999},"min":{"1min":0.80800000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.444,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.92300000000000004}]},{"osd":4,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.1850000000000001,"15min":1.169},"min":{"1min":0.94299999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.341,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.2},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2370000000000001,"15min":1.2549999999999999},"min":{"1min":1.024,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.522,"5min":2.206,"15min":2.206},"last":1.254}]},{"osd":5,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2410000000000001,"15min":1.2589999999999999},"min":{"1min":1.0329999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.5049999999999999,"5min":1.899,"15min":1.899},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2669999999999999,"15min":1.2849999999999999},"min":{"1min":0.99399999999999999,"5min":0.871,"15min":0.871},"max":{"1min":1.4039999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.329}]},{"osd":6,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.335},"min":{"1min":1.127,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.726,"5min":1.948,"15min":1.948},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.401,"15min":1.409},"min":{"1min":1.054,"5min":0.87,"15min":0.87},"max":{"1min":1.637,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.3979999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3400000000000001,"15min":1.3240000000000001},"min":{"1min":1.0800000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.659,"5min":2.161,"15min":2.161},"last":1.427},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3520000000000001,"15min":1.359},"min":{"1min":1.0700000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.573,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.4810000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T03:02:25.606 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T03:02:25.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.790+0000 7f71c985b700 1 Processor -- start 2023-11-15T03:02:25.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.790+0000 7f71c985b700 1 -- start start 2023-11-15T03:02:25.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.794+0000 7f71c985b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c41562f0 0x7f71c4155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:25.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.794+0000 7f71c985b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f71c40a57c0 0x7f71c40a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:25.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.794+0000 7f71c985b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c40c4c30 con 0x7f71c4154ca0 2023-11-15T03:02:25.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.794+0000 7f71c985b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c40a8280 con 0x7f71c40a57c0 2023-11-15T03:02:25.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.794+0000 7f71c985b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c40a6d40 con 0x7f71c41562f0 2023-11-15T03:02:25.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.794+0000 7f71c37fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f71c40a57c0 0x7f71c40a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:25.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.794+0000 7f71c37fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f71c40a57c0 0x7f71c40a5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40422/0 (socket says 172.21.15.121:40422) 2023-11-15T03:02:25.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.794+0000 7f71c37fe700 1 -- 172.21.15.121:0/3990787919 learned_addr learned my addr 172.21.15.121:0/3990787919 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:25.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.794+0000 7f71ba7fc700 1 --2- 172.21.15.121:0/3990787919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c41562f0 0x7f71c4155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:25.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.794+0000 7f71c37fe700 1 -- 172.21.15.121:0/3990787919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c41562f0 msgr2=0x7f71c4155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:25.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.794+0000 7f71c37fe700 1 --2- 172.21.15.121:0/3990787919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c41562f0 0x7f71c4155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.794+0000 7f71c37fe700 1 -- 172.21.15.121:0/3990787919 >> v1:172.21.15.190:6789/0 conn(0x7f71c4154ca0 legacy=0x7f71c40a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:25.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.794+0000 7f71c37fe700 1 -- 172.21.15.121:0/3990787919 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c40a6140 con 0x7f71c40a57c0 2023-11-15T03:02:25.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.794+0000 7f71c37fe700 1 --2- 172.21.15.121:0/3990787919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c40a57c0 0x7f71c40a5c50 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f71bc006fd0 tx=0x7f71bc0090f0).ready entity=mon.0 client_cookie=c54997eb300685c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:25.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.794+0000 7f71c27fc700 1 -- 172.21.15.121:0/3990787919 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc00b580 con 0x7f71c40a57c0 2023-11-15T03:02:25.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.794+0000 7f71c27fc700 1 -- 172.21.15.121:0/3990787919 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc00b6e0 con 0x7f71c40a57c0 2023-11-15T03:02:25.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.794+0000 7f71c27fc700 1 -- 172.21.15.121:0/3990787919 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc00ba60 con 0x7f71c40a57c0 2023-11-15T03:02:25.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.798+0000 7f71c985b700 1 -- 172.21.15.121:0/3990787919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c40a57c0 msgr2=0x7f71c40a5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:25.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.798+0000 7f71c985b700 1 --2- 172.21.15.121:0/3990787919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c40a57c0 0x7f71c40a5c50 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f71bc006fd0 tx=0x7f71bc0090f0).stop 2023-11-15T03:02:25.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.798+0000 7f71c985b700 1 -- 172.21.15.121:0/3990787919 shutdown_connections 2023-11-15T03:02:25.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.798+0000 7f71c985b700 1 --2- 172.21.15.121:0/3990787919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c40a57c0 0x7f71c40a5c50 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.798+0000 7f71c985b700 1 --2- 172.21.15.121:0/3990787919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c41562f0 0x7f71c4155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.798+0000 7f71c985b700 1 -- 172.21.15.121:0/3990787919 >> 172.21.15.121:0/3990787919 conn(0x7f71c4062190 msgr2=0x7f71c40625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:25.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.798+0000 7f71c985b700 1 -- 172.21.15.121:0/3990787919 shutdown_connections 2023-11-15T03:02:25.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.798+0000 7f71c985b700 1 -- 172.21.15.121:0/3990787919 wait complete. 2023-11-15T03:02:25.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.798+0000 7f71c985b700 1 Processor -- start 2023-11-15T03:02:25.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.798+0000 7f71c985b700 1 -- start start 2023-11-15T03:02:25.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71c985b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c40a57c0 0x7f71c4143b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:25.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71c985b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71c4154ca0 0x7f71c40ab980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:25.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71c985b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c41562f0 0x7f71c40abe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:25.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71c985b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c40ac360 con 0x7f71c40a57c0 2023-11-15T03:02:25.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71c985b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c4165f60 con 0x7f71c4154ca0 2023-11-15T03:02:25.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71c985b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c4165350 con 0x7f71c41562f0 2023-11-15T03:02:25.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71c2ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c40a57c0 0x7f71c4143b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:25.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71c2ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c40a57c0 0x7f71c4143b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40436/0 (socket says 172.21.15.121:40436) 2023-11-15T03:02:25.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71c2ffd700 1 -- 172.21.15.121:0/78131085 learned_addr learned my addr 172.21.15.121:0/78131085 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:25.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71ba7fc700 1 --2- 172.21.15.121:0/78131085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71c4154ca0 0x7f71c40ab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:25.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71c37fe700 1 --2- 172.21.15.121:0/78131085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c41562f0 0x7f71c40abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:25.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71c2ffd700 1 -- 172.21.15.121:0/78131085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c41562f0 msgr2=0x7f71c40abe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:25.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71c2ffd700 1 --2- 172.21.15.121:0/78131085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c41562f0 0x7f71c40abe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71c2ffd700 1 -- 172.21.15.121:0/78131085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71c4154ca0 msgr2=0x7f71c40ab980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:25.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71c2ffd700 1 --2- 172.21.15.121:0/78131085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71c4154ca0 0x7f71c40ab980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:25.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71c2ffd700 1 -- 172.21.15.121:0/78131085 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc009370 con 0x7f71c40a57c0 2023-11-15T03:02:25.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71ba7fc700 1 --2- 172.21.15.121:0/78131085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71c4154ca0 0x7f71c40ab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:25.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71c2ffd700 1 --2- 172.21.15.121:0/78131085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c40a57c0 0x7f71c4143b20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f71b400c840 tx=0x7f71b4001660).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:25.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71bb7fe700 1 -- 172.21.15.121:0/78131085 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4006a20 con 0x7f71c40a57c0 2023-11-15T03:02:25.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71bb7fe700 1 -- 172.21.15.121:0/78131085 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b4006b80 con 0x7f71c40a57c0 2023-11-15T03:02:25.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71bb7fe700 1 -- 172.21.15.121:0/78131085 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4007020 con 0x7f71c40a57c0 2023-11-15T03:02:25.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71c985b700 1 -- 172.21.15.121:0/78131085 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c4165630 con 0x7f71c40a57c0 2023-11-15T03:02:25.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.802+0000 7f71c985b700 1 -- 172.21.15.121:0/78131085 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c4165bf0 con 0x7f71c40a57c0 2023-11-15T03:02:25.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.806+0000 7f71c985b700 1 -- 172.21.15.121:0/78131085 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c406c150 con 0x7f71c40a57c0 2023-11-15T03:02:25.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.806+0000 7f71bb7fe700 1 -- 172.21.15.121:0/78131085 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f71b4007410 con 0x7f71c40a57c0 2023-11-15T03:02:25.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.806+0000 7f71bb7fe700 1 --2- 172.21.15.121:0/78131085 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71a405a510 0x7f71a405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:25.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.806+0000 7f71bb7fe700 1 -- 172.21.15.121:0/78131085 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f71b4074480 con 0x7f71c40a57c0 2023-11-15T03:02:25.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.810+0000 7f71ba7fc700 1 --2- 172.21.15.121:0/78131085 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71a405a510 0x7f71a405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:25.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.810+0000 7f71ba7fc700 1 --2- 172.21.15.121:0/78131085 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71a405a510 0x7f71a405c9d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f71c4066150 tx=0x7f71ac008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:25.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.814+0000 7f71bb7fe700 1 -- 172.21.15.121:0/78131085 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f71b4047c60 con 0x7f71c40a57c0 2023-11-15T03:02:25.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.974+0000 7f71c985b700 1 -- 172.21.15.121:0/78131085 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f71c40a6f50 con 0x7f71a405a510 2023-11-15T03:02:25.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.990+0000 7f71bb7fe700 1 -- 172.21.15.121:0/78131085 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+350407 (secure 0 0 0) 0x7f71c40a6f50 con 0x7f71a405a510 2023-11-15T03:02:26.011 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:26.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.998+0000 7f71c985b700 1 -- 172.21.15.121:0/78131085 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71a405a510 msgr2=0x7f71a405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:26.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.998+0000 7f71c985b700 1 --2- 172.21.15.121:0/78131085 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71a405a510 0x7f71a405c9d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f71c4066150 tx=0x7f71ac008040).stop 2023-11-15T03:02:26.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.998+0000 7f71c985b700 1 -- 172.21.15.121:0/78131085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c40a57c0 msgr2=0x7f71c4143b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:26.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.998+0000 7f71c985b700 1 --2- 172.21.15.121:0/78131085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c40a57c0 0x7f71c4143b20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f71b400c840 tx=0x7f71b4001660).stop 2023-11-15T03:02:26.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.998+0000 7f71c985b700 1 -- 172.21.15.121:0/78131085 shutdown_connections 2023-11-15T03:02:26.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.998+0000 7f71c985b700 1 --2- 172.21.15.121:0/78131085 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71a405a510 0x7f71a405c9d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.998+0000 7f71c985b700 1 --2- 172.21.15.121:0/78131085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c41562f0 0x7f71c40abe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.998+0000 7f71c985b700 1 --2- 172.21.15.121:0/78131085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71c4154ca0 0x7f71c40ab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.998+0000 7f71c985b700 1 --2- 172.21.15.121:0/78131085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71c40a57c0 0x7f71c4143b20 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.998+0000 7f71c985b700 1 -- 172.21.15.121:0/78131085 >> 172.21.15.121:0/78131085 conn(0x7f71c4062190 msgr2=0x7f71c414e200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:26.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.998+0000 7f71c985b700 1 -- 172.21.15.121:0/78131085 shutdown_connections 2023-11-15T03:02:26.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:25.998+0000 7f71c985b700 1 -- 172.21.15.121:0/78131085 wait complete. 2023-11-15T03:02:26.021 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T03:02:26.029 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2023-11-15T03:02:24.040452+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":892,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":144,"acting":144,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2383800,"kb_used_data":7032,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752590920,"statfs":{"total":773094113280,"available":770653102080,"internally_reserved":0,"allocated":7200768,"data_stored":3264589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25972,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-99,"num_read_kb":-123,"num_write":-91,"num_write_kb":-53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.130555"},"pg_stats":[{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247222+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250336+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251487+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745613+0000","last_change":"2023-11-15T03:02:20.282870+0000","last_active":"2023-11-15T03:02:20.745613+0000","last_peered":"2023-11-15T03:02:20.745613+0000","last_clean":"2023-11-15T03:02:20.745613+0000","last_became_active":"2023-11-15T03:02:20.282610+0000","last_became_peered":"2023-11-15T03:02:20.282610+0000","last_unstale":"2023-11-15T03:02:20.745613+0000","last_undegraded":"2023-11-15T03:02:20.745613+0000","last_fullsized":"2023-11-15T03:02:20.745613+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.246707+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745400+0000","last_change":"2023-11-15T03:02:20.730582+0000","last_active":"2023-11-15T03:02:20.745400+0000","last_peered":"2023-11-15T03:02:20.745400+0000","last_clean":"2023-11-15T03:02:20.745400+0000","last_became_active":"2023-11-15T03:02:20.730317+0000","last_became_peered":"2023-11-15T03:02:20.730317+0000","last_unstale":"2023-11-15T03:02:20.745400+0000","last_undegraded":"2023-11-15T03:02:20.745400+0000","last_fullsized":"2023-11-15T03:02:20.745400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250208+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250406+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.248379+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251917+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247687+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"76'4","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.754390+0000","last_change":"2023-11-15T03:02:20.280107+0000","last_active":"2023-11-15T03:02:20.754390+0000","last_peered":"2023-11-15T03:02:20.754390+0000","last_clean":"2023-11-15T03:02:20.754390+0000","last_became_active":"2023-11-15T03:02:20.279672+0000","last_became_peered":"2023-11-15T03:02:20.279672+0000","last_unstale":"2023-11-15T03:02:20.754390+0000","last_undegraded":"2023-11-15T03:02:20.754390+0000","last_fullsized":"2023-11-15T03:02:20.754390+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250882+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247684+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250911+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251733+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247901+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.714183+0000","last_change":"2023-11-15T03:02:20.282164+0000","last_active":"2023-11-15T03:02:20.714183+0000","last_peered":"2023-11-15T03:02:20.714183+0000","last_clean":"2023-11-15T03:02:20.714183+0000","last_became_active":"2023-11-15T03:02:20.281841+0000","last_became_peered":"2023-11-15T03:02:20.281841+0000","last_unstale":"2023-11-15T03:02:20.714183+0000","last_undegraded":"2023-11-15T03:02:20.714183+0000","last_fullsized":"2023-11-15T03:02:20.714183+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250066+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250037+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251700+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251055+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.748655+0000","last_change":"2023-11-15T03:02:20.740354+0000","last_active":"2023-11-15T03:02:20.748655+0000","last_peered":"2023-11-15T03:02:20.748655+0000","last_clean":"2023-11-15T03:02:20.748655+0000","last_became_active":"2023-11-15T03:02:20.739907+0000","last_became_peered":"2023-11-15T03:02:20.739907+0000","last_unstale":"2023-11-15T03:02:20.748655+0000","last_undegraded":"2023-11-15T03:02:20.748655+0000","last_fullsized":"2023-11-15T03:02:20.748655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.244098+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.244535+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250911+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.749278+0000","last_change":"2023-11-15T03:02:20.745921+0000","last_active":"2023-11-15T03:02:20.749278+0000","last_peered":"2023-11-15T03:02:20.749278+0000","last_clean":"2023-11-15T03:02:20.749278+0000","last_became_active":"2023-11-15T03:02:20.745541+0000","last_became_peered":"2023-11-15T03:02:20.745541+0000","last_unstale":"2023-11-15T03:02:20.749278+0000","last_undegraded":"2023-11-15T03:02:20.749278+0000","last_fullsized":"2023-11-15T03:02:20.749278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.249601+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250911+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251299+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.249643+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.747882+0000","last_change":"2023-11-15T03:02:20.736251+0000","last_active":"2023-11-15T03:02:20.747882+0000","last_peered":"2023-11-15T03:02:20.747882+0000","last_clean":"2023-11-15T03:02:20.747882+0000","last_became_active":"2023-11-15T03:02:20.735947+0000","last_became_peered":"2023-11-15T03:02:20.735947+0000","last_unstale":"2023-11-15T03:02:20.747882+0000","last_undegraded":"2023-11-15T03:02:20.747882+0000","last_fullsized":"2023-11-15T03:02:20.747882+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247315+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.245886+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247139+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251558+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251224+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250451+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.246576+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.246709+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247598+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.748847+0000","last_change":"2023-11-15T03:02:20.740969+0000","last_active":"2023-11-15T03:02:20.748847+0000","last_peered":"2023-11-15T03:02:20.748847+0000","last_clean":"2023-11-15T03:02:20.748847+0000","last_became_active":"2023-11-15T03:02:20.740724+0000","last_became_peered":"2023-11-15T03:02:20.740724+0000","last_unstale":"2023-11-15T03:02:20.748847+0000","last_undegraded":"2023-11-15T03:02:20.748847+0000","last_fullsized":"2023-11-15T03:02:20.748847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250474+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.244374+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247757+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.249509+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.222469+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.249500+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.246525+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.245327+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250012+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.246591+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745673+0000","last_change":"2023-11-15T03:02:20.730485+0000","last_active":"2023-11-15T03:02:20.745673+0000","last_peered":"2023-11-15T03:02:20.745673+0000","last_clean":"2023-11-15T03:02:20.745673+0000","last_became_active":"2023-11-15T03:02:20.730219+0000","last_became_peered":"2023-11-15T03:02:20.730219+0000","last_unstale":"2023-11-15T03:02:20.745673+0000","last_undegraded":"2023-11-15T03:02:20.745673+0000","last_fullsized":"2023-11-15T03:02:20.745673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.249729+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.748129+0000","last_change":"2023-11-15T03:02:20.737437+0000","last_active":"2023-11-15T03:02:20.748129+0000","last_peered":"2023-11-15T03:02:20.748129+0000","last_clean":"2023-11-15T03:02:20.748129+0000","last_became_active":"2023-11-15T03:02:20.737189+0000","last_became_peered":"2023-11-15T03:02:20.737189+0000","last_unstale":"2023-11-15T03:02:20.748129+0000","last_undegraded":"2023-11-15T03:02:20.748129+0000","last_fullsized":"2023-11-15T03:02:20.748129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.712554+0000","last_change":"2023-11-15T03:02:20.277917+0000","last_active":"2023-11-15T03:02:20.712554+0000","last_peered":"2023-11-15T03:02:20.712554+0000","last_clean":"2023-11-15T03:02:20.712554+0000","last_became_active":"2023-11-15T03:02:20.275072+0000","last_became_peered":"2023-11-15T03:02:20.275072+0000","last_unstale":"2023-11-15T03:02:20.712554+0000","last_undegraded":"2023-11-15T03:02:20.712554+0000","last_fullsized":"2023-11-15T03:02:20.712554+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.251321+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-15T03:02:17.441603+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T03:02:17.441603+0000","last_peered":"2023-11-15T03:02:17.441603+0000","last_clean":"2023-11-15T03:02:17.441603+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T03:02:17.441603+0000","last_undegraded":"2023-11-15T03:02:17.441603+0000","last_fullsized":"2023-11-15T03:02:17.441603+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.246191+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.714328+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T03:02:20.714328+0000","last_peered":"2023-11-15T03:02:20.714328+0000","last_clean":"2023-11-15T03:02:20.714328+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T03:02:20.714328+0000","last_undegraded":"2023-11-15T03:02:20.714328+0000","last_fullsized":"2023-11-15T03:02:20.714328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.712972+0000","last_change":"2023-11-15T03:02:20.279939+0000","last_active":"2023-11-15T03:02:20.712972+0000","last_peered":"2023-11-15T03:02:20.712972+0000","last_clean":"2023-11-15T03:02:20.712972+0000","last_became_active":"2023-11-15T03:02:20.279319+0000","last_became_peered":"2023-11-15T03:02:20.279319+0000","last_unstale":"2023-11-15T03:02:20.712972+0000","last_undegraded":"2023-11-15T03:02:20.712972+0000","last_fullsized":"2023-11-15T03:02:20.712972+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":55,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-15T03:02:18.435968+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T03:02:18.435968+0000","last_peered":"2023-11-15T03:02:18.435968+0000","last_clean":"2023-11-15T03:02:18.435968+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T03:02:18.435968+0000","last_undegraded":"2023-11-15T03:02:18.435968+0000","last_fullsized":"2023-11-15T03:02:18.435968+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.247085+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.713011+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T03:02:20.713011+0000","last_peered":"2023-11-15T03:02:20.713011+0000","last_clean":"2023-11-15T03:02:20.713011+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T03:02:20.713011+0000","last_undegraded":"2023-11-15T03:02:20.713011+0000","last_fullsized":"2023-11-15T03:02:20.713011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.250745+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":65,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-15T03:02:15.330804+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T03:02:15.330804+0000","last_peered":"2023-11-15T03:02:15.330804+0000","last_clean":"2023-11-15T03:02:15.330804+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T03:02:15.330804+0000","last_undegraded":"2023-11-15T03:02:15.330804+0000","last_fullsized":"2023-11-15T03:02:15.330804+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.249036+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-15T03:02:16.993728+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T03:02:16.993728+0000","last_peered":"2023-11-15T03:02:16.993728+0000","last_clean":"2023-11-15T03:02:16.993728+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T03:02:16.993728+0000","last_undegraded":"2023-11-15T03:02:16.993728+0000","last_fullsized":"2023-11-15T03:02:16.993728+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.248909+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":55,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-15T03:02:18.435964+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T03:02:18.435964+0000","last_peered":"2023-11-15T03:02:18.435964+0000","last_clean":"2023-11-15T03:02:18.435964+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T03:02:18.435964+0000","last_undegraded":"2023-11-15T03:02:18.435964+0000","last_fullsized":"2023-11-15T03:02:18.435964+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:02:20.235488+0000","last_change":"2023-11-15T03:02:20.235488+0000","last_active":"2023-11-15T03:02:20.235488+0000","last_peered":"2023-11-15T03:02:20.235488+0000","last_clean":"2023-11-15T03:02:20.235488+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:20.235488+0000","last_undegraded":"2023-11-15T03:02:20.235488+0000","last_fullsized":"2023-11-15T03:02:20.235488+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-15T03:02:19.222469+0000","last_change":"2023-11-15T03:02:19.248786+0000","last_active":"2023-11-15T03:02:19.222469+0000","last_peered":"2023-11-15T03:02:19.222469+0000","last_clean":"2023-11-15T03:02:19.222469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:02:19.222469+0000","last_undegraded":"2023-11-15T03:02:19.222469+0000","last_fullsized":"2023-11-15T03:02:19.222469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":65,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-15T03:02:15.330789+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T03:02:15.330789+0000","last_peered":"2023-11-15T03:02:15.330789+0000","last_clean":"2023-11-15T03:02:15.330789+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T03:02:15.330789+0000","last_undegraded":"2023-11-15T03:02:15.330789+0000","last_fullsized":"2023-11-15T03:02:15.330789+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":56,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-15T03:02:18.435958+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T03:02:18.435958+0000","last_peered":"2023-11-15T03:02:18.435958+0000","last_clean":"2023-11-15T03:02:18.435958+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T03:02:18.435958+0000","last_undegraded":"2023-11-15T03:02:18.435958+0000","last_fullsized":"2023-11-15T03:02:18.435958+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":892,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":112,"acting":112,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542213,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297972,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073868,"statfs":{"total":96636764160,"available":96331640832,"internally_reserved":0,"allocated":897024,"data_stored":407083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.181,"15min":1.1910000000000001},"min":{"1min":0.72399999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3360000000000001,"5min":10.961,"15min":10.961},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.141,"15min":1.1890000000000001},"min":{"1min":0.876,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.331,"5min":3.4910000000000001,"15min":3.4910000000000001},"last":1.5589999999999999}]},{"osd":1,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.161,"15min":1.153},"min":{"1min":0.83099999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.4219999999999999,"5min":5.4930000000000003,"15min":5.4930000000000003},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2050000000000001,"15min":1.1879999999999999},"min":{"1min":0.79400000000000004,"5min":0.747,"15min":0.747},"max":{"1min":1.4830000000000001,"5min":6.8239999999999998,"15min":6.8239999999999998},"last":1.6970000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.1679999999999999,"15min":1.218},"min":{"1min":0.76300000000000001,"5min":0.751,"15min":0.751},"max":{"1min":1.49,"5min":5.7000000000000002,"15min":5.7000000000000002},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2430000000000001,"15min":1.2889999999999999},"min":{"1min":0.93600000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.597,"5min":2.75,"15min":2.75},"last":1.5449999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.335,"15min":1.3720000000000001},"min":{"1min":1.014,"5min":0.751,"15min":0.751},"max":{"1min":1.6220000000000001,"5min":12.907,"15min":12.907},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.3160000000000001,"15min":1.298},"min":{"1min":0.995,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6679999999999999,"5min":9.9909999999999997,"15min":9.9909999999999997},"last":1.4950000000000001}]},{"osd":4,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.139,"15min":1.2569999999999999},"min":{"1min":0.86599999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.3360000000000001,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2230000000000001,"15min":1.4239999999999999},"min":{"1min":0.85399999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5640000000000001,"5min":4.226,"15min":4.226},"last":1.6080000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.91000000000000003,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.5580000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.298,"15min":1.427},"min":{"1min":0.96899999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.609,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":2.1299999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.403,"15min":1.548},"min":{"1min":1.115,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.619,"5min":3.2789999999999999,"15min":3.2789999999999999},"last":1.365},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.359,"15min":1.5669999999999999},"min":{"1min":0.93400000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6719999999999999,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":1.3520000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542213,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297972,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073868,"statfs":{"total":96636764160,"available":96331640832,"internally_reserved":0,"allocated":897024,"data_stored":407083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.0980000000000001,"15min":1.1240000000000001},"min":{"1min":0.80700000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.74,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.073,"15min":1.05},"min":{"1min":0.874,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.802,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.3380000000000001}]},{"osd":1,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.125,"15min":1.095},"min":{"1min":0.86899999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.512,"5min":1.623,"15min":1.623},"last":1.851},{"interface":"front","average":{"1min":1.123,"5min":1.1439999999999999,"15min":1.1599999999999999},"min":{"1min":0.83899999999999997,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.407,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.7390000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2030000000000001,"15min":1.1830000000000001},"min":{"1min":0.96499999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.7190000000000001,"5min":2.621,"15min":2.621},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.1399999999999999,"15min":1.099},"min":{"1min":0.93899999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4690000000000001,"5min":1.5,"15min":1.5},"last":2.0059999999999998}]},{"osd":3,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2070000000000001,"15min":1.1699999999999999},"min":{"1min":1.0469999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.452,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.004},{"interface":"front","average":{"1min":1.373,"5min":1.286,"15min":1.272},"min":{"1min":0.96199999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7569999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.3009999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.097,"15min":1.1659999999999999},"min":{"1min":0.63500000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.3879999999999999,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":1.9610000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.181,"15min":1.1850000000000001},"min":{"1min":0.91300000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.831,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":2.0739999999999998}]},{"osd":5,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4279999999999999,"15min":1.4139999999999999},"min":{"1min":1.1919999999999999,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.621,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":2.157},{"interface":"front","average":{"1min":1.175,"5min":1.262,"15min":1.302},"min":{"1min":0.83599999999999997,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.492,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":2.1749999999999998}]},{"osd":7,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2170000000000001,"15min":1.28},"min":{"1min":0.76200000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4570000000000001,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.796},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.4219999999999999,"15min":1.4470000000000001},"min":{"1min":0.98099999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.681,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":1.4870000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542214,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297972,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073868,"statfs":{"total":96636764160,"available":96331640832,"internally_reserved":0,"allocated":897024,"data_stored":407083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0940000000000001,"15min":1.1080000000000001},"min":{"1min":0.68200000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.355,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.345},{"interface":"front","average":{"1min":1.125,"5min":1.077,"15min":1.026},"min":{"1min":0.85799999999999998,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.3819999999999999,"5min":1.677,"15min":1.677},"last":1.149}]},{"osd":1,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.1479999999999999,"15min":1.212},"min":{"1min":0.83999999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3089999999999999,"5min":2.617,"15min":2.617},"last":1.105},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1639999999999999,"15min":1.1659999999999999},"min":{"1min":0.88100000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.431,"5min":1.657,"15min":1.657},"last":1.262}]},{"osd":2,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.204,"15min":1.1579999999999999},"min":{"1min":0.98899999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4730000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.171,"5min":1.1519999999999999,"15min":1.161},"min":{"1min":0.82899999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.4590000000000001,"5min":1.667,"15min":1.667},"last":1.2310000000000001}]},{"osd":3,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.141,"15min":1.1559999999999999},"min":{"1min":0.91100000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.401,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.22},{"interface":"front","average":{"1min":1.214,"5min":1.276,"15min":1.294},"min":{"1min":1.04,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.4319999999999999,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":1.3720000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1819999999999999,"15min":1.232},"min":{"1min":0.752,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.4399999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.105,"5min":1.194,"15min":1.224},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.5149999999999999,"5min":1.994,"15min":1.994},"last":1.284}]},{"osd":6,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.28,"15min":1.325},"min":{"1min":0.877,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.494,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":1.296},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.341,"15min":1.3959999999999999},"min":{"1min":0.96999999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.52,"5min":2.173,"15min":2.173},"last":1.1879999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3160000000000001,"15min":1.3640000000000001},"min":{"1min":0.96499999999999997,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.486,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":1.331},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3819999999999999,"15min":1.4279999999999999},"min":{"1min":0.96499999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.512,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":1.427}]}]},{"osd":3,"up_from":14,"seq":60129542214,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297972,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073868,"statfs":{"total":96636764160,"available":96331640832,"internally_reserved":0,"allocated":897024,"data_stored":407083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.81999999999999995,"15min":0.77400000000000002},"min":{"1min":0.68300000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.02,"5min":1.169,"15min":1.169},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.87,"15min":0.83799999999999997},"min":{"1min":0.64000000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.244,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.39800000000000002}]},{"osd":1,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89100000000000001,"15min":0.89800000000000002},"min":{"1min":0.72899999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.163,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.006,"15min":0.97799999999999998},"min":{"1min":0.89600000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2549999999999999,"5min":1.375,"15min":1.375},"last":0.69699999999999995}]},{"osd":2,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.012,"15min":0.97999999999999998},"min":{"1min":0.75600000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4259999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.099,"5min":1.0920000000000001,"15min":1.0609999999999999},"min":{"1min":0.83099999999999996,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3009999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2310000000000001,"15min":1.1950000000000001},"min":{"1min":1.089,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.542,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":1.21,"5min":1.206,"15min":1.222},"min":{"1min":1.0049999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.4370000000000001,"5min":1.659,"15min":1.659},"last":0.621}]},{"osd":5,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":1.0269999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.4790000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.288,"5min":1.2689999999999999,"15min":1.2649999999999999},"min":{"1min":1.0509999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.512,"5min":1.829,"15min":1.829},"last":0.63500000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3340000000000001,"15min":1.3129999999999999},"min":{"1min":1.143,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":1.5620000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.383,"5min":1.3779999999999999,"15min":1.359},"min":{"1min":1.1399999999999999,"5min":1.004,"15min":1.004},"max":{"1min":1.581,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.77700000000000002}]},{"osd":7,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.357,"15min":1.3779999999999999},"min":{"1min":1.173,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.5800000000000001,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.361,"5min":1.3640000000000001,"15min":1.357},"min":{"1min":1.1339999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"max":{"1min":1.514,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.66600000000000004}]}]},{"osd":2,"up_from":14,"seq":60129542214,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297968,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073872,"statfs":{"total":96636764160,"available":96331644928,"internally_reserved":0,"allocated":892928,"data_stored":405814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78100000000000003,"15min":0.76900000000000002},"min":{"1min":0.63700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.1479999999999999,"5min":1.409,"15min":1.409},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.85299999999999998,"15min":0.871},"min":{"1min":0.70199999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.339,"5min":1.619,"15min":1.619},"last":0.47399999999999998}]},{"osd":1,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.92400000000000004,"15min":0.90200000000000002},"min":{"1min":0.73399999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.014,"5min":0.97499999999999998,"15min":0.95299999999999996},"min":{"1min":0.81699999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.238,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.70199999999999996}]},{"osd":3,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.98999999999999999,"15min":0.996},"min":{"1min":0.626,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.347,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":1.2,"5min":1.1379999999999999,"15min":1.1240000000000001},"min":{"1min":0.92600000000000005,"5min":0.622,"15min":0.622},"max":{"1min":1.5580000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.044}]},{"osd":4,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.218,"15min":1.2130000000000001},"min":{"1min":1.133,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6719999999999999,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.673},{"interface":"front","average":{"1min":1.228,"5min":1.2270000000000001,"15min":1.226},"min":{"1min":1.03,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5029999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.7589999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2070000000000001,"15min":1.246},"min":{"1min":0.93999999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.554,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.276,"15min":1.29},"min":{"1min":1.0780000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.702,"5min":1.857,"15min":1.857},"last":1.3839999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.288,"15min":1.347},"min":{"1min":1.0109999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.679,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.147},{"interface":"front","average":{"1min":1.272,"5min":1.2789999999999999,"15min":1.331},"min":{"1min":0.92400000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.524,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.127}]},{"osd":7,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3460000000000001,"15min":1.375},"min":{"1min":1.081,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.821,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3069999999999999,"15min":1.3680000000000001},"min":{"1min":1.1140000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.635,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.80200000000000005}]}]},{"osd":0,"up_from":14,"seq":60129542214,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297960,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073880,"statfs":{"total":96636764160,"available":96331653120,"internally_reserved":0,"allocated":884736,"data_stored":400862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.85699999999999998,"15min":0.871},"min":{"1min":0.61499999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4390000000000001,"5min":4.3630000000000004,"15min":4.3630000000000004},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.97399999999999998,"15min":0.92200000000000004},"min":{"1min":0.67600000000000005,"5min":0.441,"15min":0.441},"max":{"1min":1.3520000000000001,"5min":6.157,"15min":6.157},"last":0.71699999999999997}]},{"osd":2,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0109999999999999,"15min":1.0189999999999999},"min":{"1min":0.76800000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.571,"5min":4.1699999999999999,"15min":4.1699999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.046,"15min":1.006},"min":{"1min":0.75600000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4099999999999999,"5min":6.8369999999999997,"15min":6.8369999999999997},"last":1.087}]},{"osd":3,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.97599999999999998,"15min":0.96599999999999997},"min":{"1min":0.69599999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7330000000000001,"5min":3.5070000000000001,"15min":3.5070000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.143,"15min":1.1699999999999999},"min":{"1min":0.90400000000000003,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.55,"5min":5.2149999999999999,"15min":5.2149999999999999},"last":1.286}]},{"osd":4,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.2949999999999999,"15min":1.2629999999999999},"min":{"1min":1.0369999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6479999999999999,"5min":6.2000000000000002,"15min":6.2000000000000002},"last":1.216},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2270000000000001,"15min":1.1930000000000001},"min":{"1min":0.874,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.927,"5min":3.577,"15min":3.577},"last":1.006}]},{"osd":5,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.262,"15min":1.2330000000000001},"min":{"1min":0.97499999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9850000000000001,"5min":3.548,"15min":3.548},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.3220000000000001,"15min":1.3520000000000001},"min":{"1min":1.0840000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.6899999999999999,"5min":4.4080000000000004,"15min":4.4080000000000004},"last":1.675}]},{"osd":6,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.365,"15min":1.4139999999999999},"min":{"1min":1.1910000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.528,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3660000000000001,"15min":1.3540000000000001},"min":{"1min":1.0369999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9970000000000001,"5min":6.9690000000000003,"15min":6.9690000000000003},"last":1.151}]},{"osd":7,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.506,"15min":1.4259999999999999},"min":{"1min":0.94599999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.9390000000000001,"5min":18.253,"15min":18.253},"last":1.377},{"interface":"front","average":{"1min":1.343,"5min":1.4099999999999999,"15min":1.3100000000000001},"min":{"1min":1.129,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8280000000000001,"5min":11.006,"15min":11.006},"last":1.238}]}]},{"osd":5,"up_from":13,"seq":55834574918,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297960,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073880,"statfs":{"total":96636764160,"available":96331653120,"internally_reserved":0,"allocated":884736,"data_stored":400862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.081,"15min":1.056},"min":{"1min":0.81299999999999994,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4690000000000001,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.012,"5min":1.091,"15min":1.1950000000000001},"min":{"1min":0.86799999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3360000000000001,"5min":3.46,"15min":3.46},"last":0.89500000000000002}]},{"osd":1,"last update":"Wed Nov 15 03:01:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.127,"15min":1.224},"min":{"1min":0.75800000000000001,"5min":0.626,"15min":0.626},"max":{"1min":1.3460000000000001,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1499999999999999,"15min":1.1799999999999999},"min":{"1min":0.85399999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.407,"5min":3.516,"15min":3.516},"last":1.02}]},{"osd":2,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.181,"15min":1.2050000000000001},"min":{"1min":0.84799999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.7370000000000001,"5min":5.4180000000000001,"15min":5.4180000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.234,"5min":1.218,"15min":1.2909999999999999},"min":{"1min":0.877,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.746,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":1.077}]},{"osd":3,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1990000000000001,"15min":1.2769999999999999},"min":{"1min":0.90600000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.5049999999999999,"5min":2.141,"15min":2.141},"last":0.999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.339,"15min":1.427},"min":{"1min":1.0069999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.786,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":1.224}]},{"osd":4,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2749999999999999,"15min":1.357},"min":{"1min":0.86599999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.554,"5min":3.706,"15min":3.706},"last":1.173},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3380000000000001,"15min":1.552},"min":{"1min":0.84599999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5920000000000001,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":1.3420000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.262,"15min":1.3200000000000001},"min":{"1min":0.94299999999999995,"5min":0.749,"15min":0.749},"max":{"1min":1.5189999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.365,"15min":1.526},"min":{"1min":1.038,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.6659999999999999,"5min":3.641,"15min":3.641},"last":1.2869999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3109999999999999,"15min":1.399},"min":{"1min":0.995,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.635,"5min":3.516,"15min":3.516},"last":1.25},{"interface":"front","average":{"1min":1.417,"5min":1.4079999999999999,"15min":1.448},"min":{"1min":0.94899999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.7909999999999999,"5min":3.754,"15min":3.754},"last":1.377}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298024,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073816,"statfs":{"total":96636764160,"available":96331587584,"internally_reserved":0,"allocated":950272,"data_stored":428719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.82399999999999995,"15min":0.83799999999999997},"min":{"1min":0.63700000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.103,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.89400000000000002,"15min":0.88200000000000001},"min":{"1min":0.63,"5min":0.495,"15min":0.495},"max":{"1min":1.1950000000000001,"5min":1.419,"15min":1.419},"last":1.123}]},{"osd":2,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.91400000000000003,"15min":0.86599999999999999},"min":{"1min":0.70999999999999996,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94299999999999995,"15min":0.92200000000000004},"min":{"1min":0.66100000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.653,"5min":1.887,"15min":1.887},"last":0.99199999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.137,"15min":1.079},"min":{"1min":0.66600000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0629999999999999,"15min":1.0009999999999999},"min":{"1min":0.80800000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.444,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.92300000000000004}]},{"osd":4,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.1850000000000001,"15min":1.169},"min":{"1min":0.94299999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.341,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.2},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2370000000000001,"15min":1.2549999999999999},"min":{"1min":1.024,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.522,"5min":2.206,"15min":2.206},"last":1.254}]},{"osd":5,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2410000000000001,"15min":1.2589999999999999},"min":{"1min":1.0329999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.5049999999999999,"5min":1.899,"15min":1.899},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2669999999999999,"15min":1.2849999999999999},"min":{"1min":0.99399999999999999,"5min":0.871,"15min":0.871},"max":{"1min":1.4039999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.329}]},{"osd":6,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.335},"min":{"1min":1.127,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.726,"5min":1.948,"15min":1.948},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.401,"15min":1.409},"min":{"1min":1.054,"5min":0.87,"15min":0.87},"max":{"1min":1.637,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.3979999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3400000000000001,"15min":1.3240000000000001},"min":{"1min":1.0800000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.659,"5min":2.161,"15min":2.161},"last":1.427},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3520000000000001,"15min":1.359},"min":{"1min":1.0700000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.573,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.4810000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T03:02:26.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.38 is not active+clean 2023-11-15T03:02:26.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.037 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2023-11-15T03:02:26.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.247222+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-15T03:02:26.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.39 is not active+clean 2023-11-15T03:02:26.038 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2023-11-15T03:02:26.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.250336+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-15T03:02:26.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3a is not active+clean 2023-11-15T03:02:26.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2023-11-15T03:02:26.041 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.251487+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-15T03:02:26.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3b is not active+clean 2023-11-15T03:02:26.041 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3c is not active+clean 2023-11-15T03:02:26.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.043 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2023-11-15T03:02:26.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.246707+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-15T03:02:26.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2023-11-15T03:02:26.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.250208+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-15T03:02:26.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3e is not active+clean 2023-11-15T03:02:26.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2023-11-15T03:02:26.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.222469+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3f is not active+clean 2023-11-15T03:02:26.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2023-11-15T03:02:26.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.250406+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-15T03:02:26.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.30 is not active+clean 2023-11-15T03:02:26.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2023-11-15T03:02:26.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.248379+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-15T03:02:26.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.31 is not active+clean 2023-11-15T03:02:26.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2023-11-15T03:02:26.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.251917+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-15T03:02:26.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.32 is not active+clean 2023-11-15T03:02:26.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2023-11-15T03:02:26.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.247687+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-15T03:02:26.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.33 is not active+clean 2023-11-15T03:02:26.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.34 is not active+clean 2023-11-15T03:02:26.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2023-11-15T03:02:26.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.250882+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-15T03:02:26.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.35 is not active+clean 2023-11-15T03:02:26.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2023-11-15T03:02:26.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.247684+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-15T03:02:26.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.36 is not active+clean 2023-11-15T03:02:26.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2023-11-15T03:02:26.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.250911+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-15T03:02:26.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.37 is not active+clean 2023-11-15T03:02:26.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2023-11-15T03:02:26.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.251733+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-15T03:02:26.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.28 is not active+clean 2023-11-15T03:02:26.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2023-11-15T03:02:26.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.247901+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-15T03:02:26.073 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.29 is not active+clean 2023-11-15T03:02:26.073 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.074 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2a is not active+clean 2023-11-15T03:02:26.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2023-11-15T03:02:26.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.250066+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-15T03:02:26.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2b is not active+clean 2023-11-15T03:02:26.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2023-11-15T03:02:26.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.250037+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-15T03:02:26.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2c is not active+clean 2023-11-15T03:02:26.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2023-11-15T03:02:26.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.251700+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-15T03:02:26.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2d is not active+clean 2023-11-15T03:02:26.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2023-11-15T03:02:26.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.251055+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-15T03:02:26.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2023-11-15T03:02:26.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.244098+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-15T03:02:26.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2f is not active+clean 2023-11-15T03:02:26.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2023-11-15T03:02:26.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.244535+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-15T03:02:26.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.20 is not active+clean 2023-11-15T03:02:26.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2023-11-15T03:02:26.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.250911+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-15T03:02:26.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2023-11-15T03:02:26.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.222469+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.22 is not active+clean 2023-11-15T03:02:26.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2023-11-15T03:02:26.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.249601+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-15T03:02:26.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.23 is not active+clean 2023-11-15T03:02:26.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2023-11-15T03:02:26.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.250911+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-15T03:02:26.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.24 is not active+clean 2023-11-15T03:02:26.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2023-11-15T03:02:26.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.251299+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-15T03:02:26.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.25 is not active+clean 2023-11-15T03:02:26.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2023-11-15T03:02:26.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.222469+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.26 is not active+clean 2023-11-15T03:02:26.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2023-11-15T03:02:26.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.222469+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.27 is not active+clean 2023-11-15T03:02:26.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2023-11-15T03:02:26.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.249643+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-15T03:02:26.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2023-11-15T03:02:26.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.247315+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-15T03:02:26.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.19 is not active+clean 2023-11-15T03:02:26.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2023-11-15T03:02:26.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.245886+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-15T03:02:26.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1a is not active+clean 2023-11-15T03:02:26.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2023-11-15T03:02:26.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.247139+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-15T03:02:26.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1b is not active+clean 2023-11-15T03:02:26.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2023-11-15T03:02:26.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.251558+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-15T03:02:26.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1c is not active+clean 2023-11-15T03:02:26.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2023-11-15T03:02:26.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.251224+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-15T03:02:26.108 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1d is not active+clean 2023-11-15T03:02:26.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.108 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2023-11-15T03:02:26.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.222469+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.109 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1e is not active+clean 2023-11-15T03:02:26.110 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2023-11-15T03:02:26.110 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.250451+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-15T03:02:26.111 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1f is not active+clean 2023-11-15T03:02:26.112 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.112 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2023-11-15T03:02:26.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.246576+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-15T03:02:26.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.10 is not active+clean 2023-11-15T03:02:26.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2023-11-15T03:02:26.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.246709+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-15T03:02:26.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.11 is not active+clean 2023-11-15T03:02:26.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2023-11-15T03:02:26.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.222469+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.12 is not active+clean 2023-11-15T03:02:26.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2023-11-15T03:02:26.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.247598+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-15T03:02:26.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2023-11-15T03:02:26.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.250474+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-15T03:02:26.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.14 is not active+clean 2023-11-15T03:02:26.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2023-11-15T03:02:26.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.222469+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.15 is not active+clean 2023-11-15T03:02:26.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2023-11-15T03:02:26.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.244374+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-15T03:02:26.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.16 is not active+clean 2023-11-15T03:02:26.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2023-11-15T03:02:26.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.247757+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-15T03:02:26.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.17 is not active+clean 2023-11-15T03:02:26.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2023-11-15T03:02:26.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.249509+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-15T03:02:26.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.0 is not active+clean 2023-11-15T03:02:26.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2023-11-15T03:02:26.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.222469+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1 is not active+clean 2023-11-15T03:02:26.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.128 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2023-11-15T03:02:26.128 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.249500+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-15T03:02:26.129 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2 is not active+clean 2023-11-15T03:02:26.129 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.129 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2023-11-15T03:02:26.130 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.246525+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-15T03:02:26.130 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3 is not active+clean 2023-11-15T03:02:26.131 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.131 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2023-11-15T03:02:26.132 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.245327+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-15T03:02:26.132 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.4 is not active+clean 2023-11-15T03:02:26.133 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.133 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2023-11-15T03:02:26.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.250012+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-15T03:02:26.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.5 is not active+clean 2023-11-15T03:02:26.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2023-11-15T03:02:26.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.246591+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-15T03:02:26.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2023-11-15T03:02:26.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.249729+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-15T03:02:26.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.9 is not active+clean 2023-11-15T03:02:26.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2023-11-15T03:02:26.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.251321+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-15T03:02:26.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.8 is not active+clean 2023-11-15T03:02:26.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2023-11-15T03:02:26.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.246191+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-15T03:02:26.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.b is not active+clean 2023-11-15T03:02:26.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.a is not active+clean 2023-11-15T03:02:26.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2023-11-15T03:02:26.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.247085+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-15T03:02:26.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.c is not active+clean 2023-11-15T03:02:26.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2023-11-15T03:02:26.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.250745+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-15T03:02:26.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.f is not active+clean 2023-11-15T03:02:26.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.147 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2023-11-15T03:02:26.147 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.249036+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-15T03:02:26.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.e is not active+clean 2023-11-15T03:02:26.148 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2023-11-15T03:02:26.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.248909+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-15T03:02:26.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.d is not active+clean 2023-11-15T03:02:26.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-15T03:02:20.235488+0000', 'last_change': '2023-11-15T03:02:20.235488+0000', 'last_active': '2023-11-15T03:02:20.235488+0000', 'last_peered': '2023-11-15T03:02:20.235488+0000', 'last_clean': '2023-11-15T03:02:20.235488+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:20.235488+0000', 'last_undegraded': '2023-11-15T03:02:20.235488+0000', 'last_fullsized': '2023-11-15T03:02:20.235488+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:20.235488+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-15T03:02:26.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2023-11-15T03:02:26.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-15T03:02:19.222469+0000', 'last_change': '2023-11-15T03:02:19.248786+0000', 'last_active': '2023-11-15T03:02:19.222469+0000', 'last_peered': '2023-11-15T03:02:19.222469+0000', 'last_clean': '2023-11-15T03:02:19.222469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-15T03:02:19.222469+0000', 'last_undegraded': '2023-11-15T03:02:19.222469+0000', 'last_fullsized': '2023-11-15T03:02:19.222469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'last_clean_scrub_stamp': '2023-11-15T03:02:19.222469+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-15T03:02:26.151 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-15T03:02:26.228 INFO:tasks.ceph.mds.d.smithi190.stderr:2023-11-15T03:02:26.228+0000 7f9de85f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:02:26.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.302+0000 7f067e21b700 1 Processor -- start 2023-11-15T03:02:26.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.302+0000 7f067e21b700 1 -- start start 2023-11-15T03:02:26.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.306+0000 7f067e21b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f06780a5ed0 0x7f0678155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:26.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.306+0000 7f067e21b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06780c4c30 con 0x7f06781562f0 2023-11-15T03:02:26.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.306+0000 7f067e21b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06780a6f50 con 0x7f0678154ca0 2023-11-15T03:02:26.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.306+0000 7f067e21b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06780a8280 con 0x7f06780a5ed0 2023-11-15T03:02:26.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.306+0000 7f0677fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f06780a5ed0 0x7f0678155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:26.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.306+0000 7f066effd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f06781562f0 0x7f06780a57c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:41676/0 (socket says 172.21.15.121:41676) 2023-11-15T03:02:26.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.306+0000 7f066effd700 1 -- 172.21.15.121:0/1662488367 learned_addr learned my addr 172.21.15.121:0/1662488367 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:26.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.306+0000 7f0676ffd700 1 -- 172.21.15.121:0/1662488367 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724388113 0 0) 0x7f06780c4c30 con 0x7f06781562f0 2023-11-15T03:02:26.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.306+0000 7f0676ffd700 1 -- 172.21.15.121:0/1662488367 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06640033f0 con 0x7f06781562f0 2023-11-15T03:02:26.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.306+0000 7f0676ffd700 1 -- 172.21.15.121:0/1662488367 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092570009 0 0) 0x7f06780a6f50 con 0x7f0678154ca0 2023-11-15T03:02:26.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.306+0000 7f0676ffd700 1 -- 172.21.15.121:0/1662488367 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06780c4c30 con 0x7f0678154ca0 2023-11-15T03:02:26.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.306+0000 7f0677fff700 1 -- 172.21.15.121:0/1662488367 >> v1:172.21.15.190:6789/0 conn(0x7f0678154ca0 legacy=0x7f06780a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:26.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.306+0000 7f0677fff700 1 -- 172.21.15.121:0/1662488367 >> v1:172.21.15.121:6789/0 conn(0x7f06781562f0 legacy=0x7f06780a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:26.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.306+0000 7f0677fff700 1 -- 172.21.15.121:0/1662488367 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06780a7230 con 0x7f06780a5ed0 2023-11-15T03:02:26.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.306+0000 7f0677fff700 1 --2- 172.21.15.121:0/1662488367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06780a5ed0 0x7f0678155080 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f0670009e60 tx=0x7f0670009780).ready entity=mon.0 client_cookie=22e71e075b2cea1f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:26.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.306+0000 7f0676ffd700 1 -- 172.21.15.121:0/1662488367 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067000e070 con 0x7f06780a5ed0 2023-11-15T03:02:26.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.310+0000 7f0676ffd700 1 -- 172.21.15.121:0/1662488367 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067000bab0 con 0x7f06780a5ed0 2023-11-15T03:02:26.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.310+0000 7f0676ffd700 1 -- 172.21.15.121:0/1662488367 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067000be30 con 0x7f06780a5ed0 2023-11-15T03:02:26.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.310+0000 7f067e21b700 1 -- 172.21.15.121:0/1662488367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06780a5ed0 msgr2=0x7f0678155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:26.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.310+0000 7f067e21b700 1 --2- 172.21.15.121:0/1662488367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06780a5ed0 0x7f0678155080 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f0670009e60 tx=0x7f0670009780).stop 2023-11-15T03:02:26.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.310+0000 7f067e21b700 1 -- 172.21.15.121:0/1662488367 shutdown_connections 2023-11-15T03:02:26.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.310+0000 7f067e21b700 1 --2- 172.21.15.121:0/1662488367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06780a5ed0 0x7f0678155080 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.310+0000 7f067e21b700 1 -- 172.21.15.121:0/1662488367 >> 172.21.15.121:0/1662488367 conn(0x7f0678062190 msgr2=0x7f06780625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:26.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.310+0000 7f067e21b700 1 -- 172.21.15.121:0/1662488367 shutdown_connections 2023-11-15T03:02:26.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.310+0000 7f067e21b700 1 -- 172.21.15.121:0/1662488367 wait complete. 2023-11-15T03:02:26.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.310+0000 7f067e21b700 1 Processor -- start 2023-11-15T03:02:26.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.310+0000 7f067e21b700 1 -- start start 2023-11-15T03:02:26.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f067e21b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06780a5ed0 0x7f0678150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:26.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f067e21b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0678154ca0 0x7f0678151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:26.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f067e21b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06781562f0 0x7f06781531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:26.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f067e21b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0678151850 con 0x7f06781562f0 2023-11-15T03:02:26.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f067e21b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0678157b70 con 0x7f0678154ca0 2023-11-15T03:02:26.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f067e21b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06781699b0 con 0x7f06780a5ed0 2023-11-15T03:02:26.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f0677fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06781562f0 0x7f06781531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:26.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f0677fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06781562f0 0x7f06781531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40458/0 (socket says 172.21.15.121:40458) 2023-11-15T03:02:26.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f0677fff700 1 -- 172.21.15.121:0/1318673373 learned_addr learned my addr 172.21.15.121:0/1318673373 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:26.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f06777fe700 1 --2- 172.21.15.121:0/1318673373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06780a5ed0 0x7f0678150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:26.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f066effd700 1 --2- 172.21.15.121:0/1318673373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0678154ca0 0x7f0678151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:26.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f0677fff700 1 -- 172.21.15.121:0/1318673373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06780a5ed0 msgr2=0x7f0678150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:26.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f0677fff700 1 --2- 172.21.15.121:0/1318673373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06780a5ed0 0x7f0678150e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f0677fff700 1 -- 172.21.15.121:0/1318673373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0678154ca0 msgr2=0x7f0678151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:26.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f0677fff700 1 --2- 172.21.15.121:0/1318673373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0678154ca0 0x7f0678151360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f0677fff700 1 -- 172.21.15.121:0/1318673373 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0670009a20 con 0x7f06781562f0 2023-11-15T03:02:26.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f066effd700 1 --2- 172.21.15.121:0/1318673373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0678154ca0 0x7f0678151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:26.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f0677fff700 1 --2- 172.21.15.121:0/1318673373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06781562f0 0x7f06781531e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7f067000b610 tx=0x7f0670009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:26.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f066ffff700 1 -- 172.21.15.121:0/1318673373 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067001d630 con 0x7f06781562f0 2023-11-15T03:02:26.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f066ffff700 1 -- 172.21.15.121:0/1318673373 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067001d790 con 0x7f06781562f0 2023-11-15T03:02:26.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f066ffff700 1 -- 172.21.15.121:0/1318673373 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067001db10 con 0x7f06781562f0 2023-11-15T03:02:26.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f067e21b700 1 -- 172.21.15.121:0/1318673373 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0678153f50 con 0x7f06781562f0 2023-11-15T03:02:26.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.314+0000 7f067e21b700 1 -- 172.21.15.121:0/1318673373 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0678154510 con 0x7f06781562f0 2023-11-15T03:02:26.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.318+0000 7f067e21b700 1 -- 172.21.15.121:0/1318673373 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0678157ce0 con 0x7f06781562f0 2023-11-15T03:02:26.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.318+0000 7f066ffff700 1 -- 172.21.15.121:0/1318673373 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0670026410 con 0x7f06781562f0 2023-11-15T03:02:26.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.318+0000 7f066ffff700 1 --2- 172.21.15.121:0/1318673373 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f065805a540 0x7f065805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:26.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.318+0000 7f066ffff700 1 -- 172.21.15.121:0/1318673373 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0670076620 con 0x7f06781562f0 2023-11-15T03:02:26.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.318+0000 7f06777fe700 1 --2- 172.21.15.121:0/1318673373 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f065805a540 0x7f065805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:26.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.322+0000 7f06777fe700 1 --2- 172.21.15.121:0/1318673373 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f065805a540 0x7f065805ca00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f0668003930 tx=0x7f0668007bd0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:26.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.322+0000 7f066ffff700 1 -- 172.21.15.121:0/1318673373 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0670049ce0 con 0x7f06781562f0 2023-11-15T03:02:26.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.538+0000 7f067e21b700 1 -- 172.21.15.121:0/1318673373 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f067806c150 con 0x7f06781562f0 2023-11-15T03:02:26.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.538+0000 7f066ffff700 1 -- 172.21.15.121:0/1318673373 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1112 (secure 0 0 0) 0x7f0670009880 con 0x7f06781562f0 2023-11-15T03:02:26.547 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:26.547 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":353,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":77,"num_osds":8,"num_up_osds":8,"osd_up_since":1700017004,"num_in_osds":8,"osd_in_since":1700016996,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":65},{"state_name":"creating+peering","count":51},{"state_name":"active+clean","count":21}],"num_pgs":137,"num_pools":4,"num_objects":3,"data_bytes":892,"bytes_used":2441011200,"bytes_avail":770653102080,"bytes_total":773094113280,"unknown_pgs_ratio":0.47445255517959595,"inactive_pgs_ratio":0.37226277589797974},"fsmap":{"epoch":67,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"d","status":"up:active","gid":6492}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2023-11-15T03:01:26.018620+0000","services":{}},"progress_events":{}} 2023-11-15T03:02:26.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.542+0000 7f067e21b700 1 -- 172.21.15.121:0/1318673373 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f065805a540 msgr2=0x7f065805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:26.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.542+0000 7f067e21b700 1 --2- 172.21.15.121:0/1318673373 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f065805a540 0x7f065805ca00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f0668003930 tx=0x7f0668007bd0).stop 2023-11-15T03:02:26.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.542+0000 7f067e21b700 1 -- 172.21.15.121:0/1318673373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06781562f0 msgr2=0x7f06781531e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:26.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.542+0000 7f067e21b700 1 --2- 172.21.15.121:0/1318673373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06781562f0 0x7f06781531e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7f067000b610 tx=0x7f0670009780).stop 2023-11-15T03:02:26.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.542+0000 7f067e21b700 1 -- 172.21.15.121:0/1318673373 shutdown_connections 2023-11-15T03:02:26.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.542+0000 7f067e21b700 1 --2- 172.21.15.121:0/1318673373 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f065805a540 0x7f065805ca00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.542+0000 7f067e21b700 1 --2- 172.21.15.121:0/1318673373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06781562f0 0x7f06781531e0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.542+0000 7f067e21b700 1 --2- 172.21.15.121:0/1318673373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0678154ca0 0x7f0678151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.542+0000 7f067e21b700 1 --2- 172.21.15.121:0/1318673373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06780a5ed0 0x7f0678150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.542+0000 7f067e21b700 1 -- 172.21.15.121:0/1318673373 >> 172.21.15.121:0/1318673373 conn(0x7f0678062190 msgr2=0x7f0678150540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:26.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.546+0000 7f067e21b700 1 -- 172.21.15.121:0/1318673373 shutdown_connections 2023-11-15T03:02:26.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.546+0000 7f067e21b700 1 -- 172.21.15.121:0/1318673373 wait complete. 2023-11-15T03:02:26.573 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-15T03:02:26.574 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T03:02:26.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.726+0000 7fe867039700 1 Processor -- start 2023-11-15T03:02:26.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.730+0000 7fe867039700 1 -- start start 2023-11-15T03:02:26.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.730+0000 7fe867039700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8600c23f0 0x7fe860154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:26.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.730+0000 7fe867039700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe860153030 0x7fe8601534a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:26.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.730+0000 7fe867039700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe860153c90 con 0x7fe8600c2010 2023-11-15T03:02:26.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.730+0000 7fe867039700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8600c2830 con 0x7fe860153030 2023-11-15T03:02:26.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.730+0000 7fe867039700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8600c1810 con 0x7fe8600c23f0 2023-11-15T03:02:26.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.734+0000 7fe866037700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe8600c2010 0x7fe8600c0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:41690/0 (socket says 172.21.15.121:41690) 2023-11-15T03:02:26.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.734+0000 7fe866037700 1 -- 172.21.15.121:0/562074498 learned_addr learned my addr 172.21.15.121:0/562074498 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:26.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.734+0000 7fe866838700 1 --2- 172.21.15.121:0/562074498 >> v2:172.21.15.121:3300/0 conn(0x7fe860153030 0x7fe8601534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:26.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.734+0000 7fe865836700 1 --2- 172.21.15.121:0/562074498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8600c23f0 0x7fe860154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:26.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.734+0000 7fe865035700 1 -- 172.21.15.121:0/562074498 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 687080404 0 0) 0x7fe860153c90 con 0x7fe8600c2010 2023-11-15T03:02:26.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.734+0000 7fe865035700 1 -- 172.21.15.121:0/562074498 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe84c0033f0 con 0x7fe8600c2010 2023-11-15T03:02:26.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.734+0000 7fe865836700 1 -- 172.21.15.121:0/562074498 >> v2:172.21.15.121:3300/0 conn(0x7fe860153030 msgr2=0x7fe8601534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:26.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.734+0000 7fe865836700 1 --2- 172.21.15.121:0/562074498 >> v2:172.21.15.121:3300/0 conn(0x7fe860153030 0x7fe8601534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.734+0000 7fe865836700 1 -- 172.21.15.121:0/562074498 >> v1:172.21.15.121:6789/0 conn(0x7fe8600c2010 legacy=0x7fe8600c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:26.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.734+0000 7fe865836700 1 -- 172.21.15.121:0/562074498 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8600c1af0 con 0x7fe8600c23f0 2023-11-15T03:02:26.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.734+0000 7fe865836700 1 --2- 172.21.15.121:0/562074498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8600c23f0 0x7fe860154750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fe850009760 tx=0x7fe850009060).ready entity=mon.2 client_cookie=9494d4131b8d6a71 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:26.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.734+0000 7fe865035700 1 -- 172.21.15.121:0/562074498 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85000f6b0 con 0x7fe8600c23f0 2023-11-15T03:02:26.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.734+0000 7fe865035700 1 -- 172.21.15.121:0/562074498 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85000f810 con 0x7fe8600c23f0 2023-11-15T03:02:26.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.734+0000 7fe865035700 1 -- 172.21.15.121:0/562074498 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85000fb90 con 0x7fe8600c23f0 2023-11-15T03:02:26.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.734+0000 7fe867039700 1 -- 172.21.15.121:0/562074498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8600c23f0 msgr2=0x7fe860154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:26.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.734+0000 7fe867039700 1 --2- 172.21.15.121:0/562074498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8600c23f0 0x7fe860154750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fe850009760 tx=0x7fe850009060).stop 2023-11-15T03:02:26.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.738+0000 7fe867039700 1 -- 172.21.15.121:0/562074498 shutdown_connections 2023-11-15T03:02:26.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.738+0000 7fe867039700 1 --2- 172.21.15.121:0/562074498 >> v2:172.21.15.121:3300/0 conn(0x7fe860153030 0x7fe8601534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.738+0000 7fe867039700 1 --2- 172.21.15.121:0/562074498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8600c23f0 0x7fe860154750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.738+0000 7fe867039700 1 -- 172.21.15.121:0/562074498 >> 172.21.15.121:0/562074498 conn(0x7fe860062190 msgr2=0x7fe860062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:26.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.738+0000 7fe867039700 1 -- 172.21.15.121:0/562074498 shutdown_connections 2023-11-15T03:02:26.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.738+0000 7fe867039700 1 -- 172.21.15.121:0/562074498 wait complete. 2023-11-15T03:02:26.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.738+0000 7fe867039700 1 Processor -- start 2023-11-15T03:02:26.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.738+0000 7fe867039700 1 -- start start 2023-11-15T03:02:26.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe867039700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8600c2010 0x7fe8600bf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:26.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe867039700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8600c23f0 0x7fe860147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:26.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe867039700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe860153030 0x7fe8600c0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:26.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe867039700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8600bef60 con 0x7fe8600c2010 2023-11-15T03:02:26.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe867039700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8600bf8e0 con 0x7fe860153030 2023-11-15T03:02:26.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe867039700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8600be350 con 0x7fe8600c23f0 2023-11-15T03:02:26.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe866037700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8600c2010 0x7fe8600bf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:26.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe866037700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8600c2010 0x7fe8600bf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40470/0 (socket says 172.21.15.121:40470) 2023-11-15T03:02:26.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe866037700 1 -- 172.21.15.121:0/3442090596 learned_addr learned my addr 172.21.15.121:0/3442090596 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:26.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe865836700 1 --2- 172.21.15.121:0/3442090596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8600c23f0 0x7fe860147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:26.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe866838700 1 --2- 172.21.15.121:0/3442090596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe860153030 0x7fe8600c0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:26.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe866037700 1 -- 172.21.15.121:0/3442090596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8600c23f0 msgr2=0x7fe860147fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:26.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe866037700 1 --2- 172.21.15.121:0/3442090596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8600c23f0 0x7fe860147fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe866037700 1 -- 172.21.15.121:0/3442090596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe860153030 msgr2=0x7fe8600c0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:26.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe866037700 1 --2- 172.21.15.121:0/3442090596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe860153030 0x7fe8600c0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe866037700 1 -- 172.21.15.121:0/3442090596 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8500092a0 con 0x7fe8600c2010 2023-11-15T03:02:26.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe865836700 1 --2- 172.21.15.121:0/3442090596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8600c23f0 0x7fe860147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:02:26.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe866037700 1 --2- 172.21.15.121:0/3442090596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8600c2010 0x7fe8600bf3f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7fe85c00c490 tx=0x7fe85c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:26.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe8567fc700 1 -- 172.21.15.121:0/3442090596 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c0072e0 con 0x7fe8600c2010 2023-11-15T03:02:26.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe8567fc700 1 -- 172.21.15.121:0/3442090596 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85c007440 con 0x7fe8600c2010 2023-11-15T03:02:26.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe8567fc700 1 -- 172.21.15.121:0/3442090596 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c0057a0 con 0x7fe8600c2010 2023-11-15T03:02:26.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe867039700 1 -- 172.21.15.121:0/3442090596 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8600be630 con 0x7fe8600c2010 2023-11-15T03:02:26.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.742+0000 7fe867039700 1 -- 172.21.15.121:0/3442090596 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8600bebf0 con 0x7fe8600c2010 2023-11-15T03:02:26.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.746+0000 7fe867039700 1 -- 172.21.15.121:0/3442090596 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe86006bcf0 con 0x7fe8600c2010 2023-11-15T03:02:26.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.746+0000 7fe8567fc700 1 -- 172.21.15.121:0/3442090596 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe85c005bd0 con 0x7fe8600c2010 2023-11-15T03:02:26.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.746+0000 7fe8567fc700 1 --2- 172.21.15.121:0/3442090596 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe84405a510 0x7fe84405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:26.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.750+0000 7fe8567fc700 1 -- 172.21.15.121:0/3442090596 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe85c074120 con 0x7fe8600c2010 2023-11-15T03:02:26.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.750+0000 7fe865836700 1 --2- 172.21.15.121:0/3442090596 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe84405a510 0x7fe84405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:26.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.750+0000 7fe865836700 1 --2- 172.21.15.121:0/3442090596 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe84405a510 0x7fe84405c9d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7fe860152300 tx=0x7fe850018000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:26.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.754+0000 7fe8567fc700 1 -- 172.21.15.121:0/3442090596 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe85c047840 con 0x7fe8600c2010 2023-11-15T03:02:26.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.926+0000 7fe867039700 1 -- 172.21.15.121:0/3442090596 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe8600bf0a0 con 0x7fe84405a510 2023-11-15T03:02:26.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.938+0000 7fe8567fc700 1 -- 172.21.15.121:0/3442090596 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+358281 (secure 0 0 0) 0x7fe85000f980 con 0x7fe84405a510 2023-11-15T03:02:26.949 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:26.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.946+0000 7fe867039700 1 -- 172.21.15.121:0/3442090596 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe84405a510 msgr2=0x7fe84405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:26.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.946+0000 7fe867039700 1 --2- 172.21.15.121:0/3442090596 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe84405a510 0x7fe84405c9d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7fe860152300 tx=0x7fe850018000).stop 2023-11-15T03:02:26.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.946+0000 7fe867039700 1 -- 172.21.15.121:0/3442090596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8600c2010 msgr2=0x7fe8600bf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:26.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.946+0000 7fe867039700 1 --2- 172.21.15.121:0/3442090596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8600c2010 0x7fe8600bf3f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7fe85c00c490 tx=0x7fe85c007c50).stop 2023-11-15T03:02:26.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.946+0000 7fe867039700 1 -- 172.21.15.121:0/3442090596 shutdown_connections 2023-11-15T03:02:26.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.946+0000 7fe867039700 1 --2- 172.21.15.121:0/3442090596 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe84405a510 0x7fe84405c9d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.946+0000 7fe867039700 1 --2- 172.21.15.121:0/3442090596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe860153030 0x7fe8600c0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.946+0000 7fe867039700 1 --2- 172.21.15.121:0/3442090596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8600c23f0 0x7fe860147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.946+0000 7fe867039700 1 --2- 172.21.15.121:0/3442090596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8600c2010 0x7fe8600bf3f0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:26.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.946+0000 7fe867039700 1 -- 172.21.15.121:0/3442090596 >> 172.21.15.121:0/3442090596 conn(0x7fe860062190 msgr2=0x7fe860062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:26.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.946+0000 7fe867039700 1 -- 172.21.15.121:0/3442090596 shutdown_connections 2023-11-15T03:02:26.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:26.946+0000 7fe867039700 1 -- 172.21.15.121:0/3442090596 wait complete. 2023-11-15T03:02:26.961 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T03:02:26.982 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2023-11-15T03:02:26.041327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":247,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2384124,"kb_used_data":7356,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752590596,"statfs":{"total":773094113280,"available":770652770304,"internally_reserved":0,"allocated":7532544,"data_stored":3424503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24558,"num_objects":-9,"num_object_clones":0,"num_object_copies":-18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-9,"num_whiteouts":0,"num_read":-99,"num_read_kb":-123,"num_write":-58,"num_write_kb":-45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.130424"},"pg_stats":[{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.750047+0000","last_change":"2023-11-15T03:02:20.737262+0000","last_active":"2023-11-15T03:02:20.750047+0000","last_peered":"2023-11-15T03:02:20.750047+0000","last_clean":"2023-11-15T03:02:20.750047+0000","last_became_active":"2023-11-15T03:02:20.735691+0000","last_became_peered":"2023-11-15T03:02:20.735691+0000","last_unstale":"2023-11-15T03:02:20.750047+0000","last_undegraded":"2023-11-15T03:02:20.750047+0000","last_fullsized":"2023-11-15T03:02:20.750047+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3f","version":"76'3","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.753765+0000","last_change":"2023-11-15T03:02:20.291047+0000","last_active":"2023-11-15T03:02:20.753765+0000","last_peered":"2023-11-15T03:02:20.753765+0000","last_clean":"2023-11-15T03:02:20.753765+0000","last_became_active":"2023-11-15T03:02:20.290541+0000","last_became_peered":"2023-11-15T03:02:20.290541+0000","last_unstale":"2023-11-15T03:02:20.753765+0000","last_undegraded":"2023-11-15T03:02:20.753765+0000","last_fullsized":"2023-11-15T03:02:20.753765+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.758807+0000","last_change":"2023-11-15T03:02:20.756957+0000","last_active":"2023-11-15T03:02:20.758807+0000","last_peered":"2023-11-15T03:02:20.758807+0000","last_clean":"2023-11-15T03:02:20.758807+0000","last_became_active":"2023-11-15T03:02:20.756690+0000","last_became_peered":"2023-11-15T03:02:20.756690+0000","last_unstale":"2023-11-15T03:02:20.758807+0000","last_undegraded":"2023-11-15T03:02:20.758807+0000","last_fullsized":"2023-11-15T03:02:20.758807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.728997+0000","last_change":"2023-11-15T03:02:20.292484+0000","last_active":"2023-11-15T03:02:20.728997+0000","last_peered":"2023-11-15T03:02:20.728997+0000","last_clean":"2023-11-15T03:02:20.728997+0000","last_became_active":"2023-11-15T03:02:20.292028+0000","last_became_peered":"2023-11-15T03:02:20.292028+0000","last_unstale":"2023-11-15T03:02:20.728997+0000","last_undegraded":"2023-11-15T03:02:20.728997+0000","last_fullsized":"2023-11-15T03:02:20.728997+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.743824+0000","last_change":"2023-11-15T03:02:20.738357+0000","last_active":"2023-11-15T03:02:20.743824+0000","last_peered":"2023-11-15T03:02:20.743824+0000","last_clean":"2023-11-15T03:02:20.743824+0000","last_became_active":"2023-11-15T03:02:20.737691+0000","last_became_peered":"2023-11-15T03:02:20.737691+0000","last_unstale":"2023-11-15T03:02:20.743824+0000","last_undegraded":"2023-11-15T03:02:20.743824+0000","last_fullsized":"2023-11-15T03:02:20.743824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.719561+0000","last_change":"2023-11-15T03:02:20.280633+0000","last_active":"2023-11-15T03:02:20.719561+0000","last_peered":"2023-11-15T03:02:20.719561+0000","last_clean":"2023-11-15T03:02:20.719561+0000","last_became_active":"2023-11-15T03:02:20.280353+0000","last_became_peered":"2023-11-15T03:02:20.280353+0000","last_unstale":"2023-11-15T03:02:20.719561+0000","last_undegraded":"2023-11-15T03:02:20.719561+0000","last_fullsized":"2023-11-15T03:02:20.719561+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.750369+0000","last_change":"2023-11-15T03:02:20.740898+0000","last_active":"2023-11-15T03:02:20.750369+0000","last_peered":"2023-11-15T03:02:20.750369+0000","last_clean":"2023-11-15T03:02:20.750369+0000","last_became_active":"2023-11-15T03:02:20.740611+0000","last_became_peered":"2023-11-15T03:02:20.740611+0000","last_unstale":"2023-11-15T03:02:20.750369+0000","last_undegraded":"2023-11-15T03:02:20.750369+0000","last_fullsized":"2023-11-15T03:02:20.750369+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3c","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745613+0000","last_change":"2023-11-15T03:02:20.282870+0000","last_active":"2023-11-15T03:02:20.745613+0000","last_peered":"2023-11-15T03:02:20.745613+0000","last_clean":"2023-11-15T03:02:20.745613+0000","last_became_active":"2023-11-15T03:02:20.282610+0000","last_became_peered":"2023-11-15T03:02:20.282610+0000","last_unstale":"2023-11-15T03:02:20.745613+0000","last_undegraded":"2023-11-15T03:02:20.745613+0000","last_fullsized":"2023-11-15T03:02:20.745613+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.749217+0000","last_change":"2023-11-15T03:02:20.744670+0000","last_active":"2023-11-15T03:02:20.749217+0000","last_peered":"2023-11-15T03:02:20.749217+0000","last_clean":"2023-11-15T03:02:20.749217+0000","last_became_active":"2023-11-15T03:02:20.743282+0000","last_became_peered":"2023-11-15T03:02:20.743282+0000","last_unstale":"2023-11-15T03:02:20.749217+0000","last_undegraded":"2023-11-15T03:02:20.749217+0000","last_fullsized":"2023-11-15T03:02:20.749217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3b","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.743864+0000","last_change":"2023-11-15T03:02:20.280906+0000","last_active":"2023-11-15T03:02:20.743864+0000","last_peered":"2023-11-15T03:02:20.743864+0000","last_clean":"2023-11-15T03:02:20.743864+0000","last_became_active":"2023-11-15T03:02:20.280418+0000","last_became_peered":"2023-11-15T03:02:20.280418+0000","last_unstale":"2023-11-15T03:02:20.743864+0000","last_undegraded":"2023-11-15T03:02:20.743864+0000","last_fullsized":"2023-11-15T03:02:20.743864+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745400+0000","last_change":"2023-11-15T03:02:20.730582+0000","last_active":"2023-11-15T03:02:20.745400+0000","last_peered":"2023-11-15T03:02:20.745400+0000","last_clean":"2023-11-15T03:02:20.745400+0000","last_became_active":"2023-11-15T03:02:20.730317+0000","last_became_peered":"2023-11-15T03:02:20.730317+0000","last_unstale":"2023-11-15T03:02:20.745400+0000","last_undegraded":"2023-11-15T03:02:20.745400+0000","last_fullsized":"2023-11-15T03:02:20.745400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3a","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.750200+0000","last_change":"2023-11-15T03:02:20.287727+0000","last_active":"2023-11-15T03:02:20.750200+0000","last_peered":"2023-11-15T03:02:20.750200+0000","last_clean":"2023-11-15T03:02:20.750200+0000","last_became_active":"2023-11-15T03:02:20.287442+0000","last_became_peered":"2023-11-15T03:02:20.287442+0000","last_unstale":"2023-11-15T03:02:20.750200+0000","last_undegraded":"2023-11-15T03:02:20.750200+0000","last_fullsized":"2023-11-15T03:02:20.750200+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745341+0000","last_change":"2023-11-15T03:02:20.740774+0000","last_active":"2023-11-15T03:02:20.745341+0000","last_peered":"2023-11-15T03:02:20.745341+0000","last_clean":"2023-11-15T03:02:20.745341+0000","last_became_active":"2023-11-15T03:02:20.740489+0000","last_became_peered":"2023-11-15T03:02:20.740489+0000","last_unstale":"2023-11-15T03:02:20.745341+0000","last_undegraded":"2023-11-15T03:02:20.745341+0000","last_fullsized":"2023-11-15T03:02:20.745341+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.727360+0000","last_change":"2023-11-15T03:02:20.274052+0000","last_active":"2023-11-15T03:02:20.727360+0000","last_peered":"2023-11-15T03:02:20.727360+0000","last_clean":"2023-11-15T03:02:20.727360+0000","last_became_active":"2023-11-15T03:02:20.273717+0000","last_became_peered":"2023-11-15T03:02:20.273717+0000","last_unstale":"2023-11-15T03:02:20.727360+0000","last_undegraded":"2023-11-15T03:02:20.727360+0000","last_fullsized":"2023-11-15T03:02:20.727360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.755857+0000","last_change":"2023-11-15T03:02:20.751318+0000","last_active":"2023-11-15T03:02:20.755857+0000","last_peered":"2023-11-15T03:02:20.755857+0000","last_clean":"2023-11-15T03:02:20.755857+0000","last_became_active":"2023-11-15T03:02:20.749439+0000","last_became_peered":"2023-11-15T03:02:20.749439+0000","last_unstale":"2023-11-15T03:02:20.755857+0000","last_undegraded":"2023-11-15T03:02:20.755857+0000","last_fullsized":"2023-11-15T03:02:20.755857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.728213+0000","last_change":"2023-11-15T03:02:20.280607+0000","last_active":"2023-11-15T03:02:20.728213+0000","last_peered":"2023-11-15T03:02:20.728213+0000","last_clean":"2023-11-15T03:02:20.728213+0000","last_became_active":"2023-11-15T03:02:20.280047+0000","last_became_peered":"2023-11-15T03:02:20.280047+0000","last_unstale":"2023-11-15T03:02:20.728213+0000","last_undegraded":"2023-11-15T03:02:20.728213+0000","last_fullsized":"2023-11-15T03:02:20.728213+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745749+0000","last_change":"2023-11-15T03:02:20.734411+0000","last_active":"2023-11-15T03:02:20.745749+0000","last_peered":"2023-11-15T03:02:20.745749+0000","last_clean":"2023-11-15T03:02:20.745749+0000","last_became_active":"2023-11-15T03:02:20.733875+0000","last_became_peered":"2023-11-15T03:02:20.733875+0000","last_unstale":"2023-11-15T03:02:20.745749+0000","last_undegraded":"2023-11-15T03:02:20.745749+0000","last_fullsized":"2023-11-15T03:02:20.745749+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.723328+0000","last_change":"2023-11-15T03:02:20.277343+0000","last_active":"2023-11-15T03:02:20.723328+0000","last_peered":"2023-11-15T03:02:20.723328+0000","last_clean":"2023-11-15T03:02:20.723328+0000","last_became_active":"2023-11-15T03:02:20.276105+0000","last_became_peered":"2023-11-15T03:02:20.276105+0000","last_unstale":"2023-11-15T03:02:20.723328+0000","last_undegraded":"2023-11-15T03:02:20.723328+0000","last_fullsized":"2023-11-15T03:02:20.723328+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.737783+0000","last_change":"2023-11-15T03:02:20.729124+0000","last_active":"2023-11-15T03:02:20.737783+0000","last_peered":"2023-11-15T03:02:20.737783+0000","last_clean":"2023-11-15T03:02:20.737783+0000","last_became_active":"2023-11-15T03:02:20.728781+0000","last_became_peered":"2023-11-15T03:02:20.728781+0000","last_unstale":"2023-11-15T03:02:20.737783+0000","last_undegraded":"2023-11-15T03:02:20.737783+0000","last_fullsized":"2023-11-15T03:02:20.737783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.722395+0000","last_change":"2023-11-15T03:02:20.288666+0000","last_active":"2023-11-15T03:02:20.722395+0000","last_peered":"2023-11-15T03:02:20.722395+0000","last_clean":"2023-11-15T03:02:20.722395+0000","last_became_active":"2023-11-15T03:02:20.288312+0000","last_became_peered":"2023-11-15T03:02:20.288312+0000","last_unstale":"2023-11-15T03:02:20.722395+0000","last_undegraded":"2023-11-15T03:02:20.722395+0000","last_fullsized":"2023-11-15T03:02:20.722395+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.743428+0000","last_change":"2023-11-15T03:02:20.737683+0000","last_active":"2023-11-15T03:02:20.743428+0000","last_peered":"2023-11-15T03:02:20.743428+0000","last_clean":"2023-11-15T03:02:20.743428+0000","last_became_active":"2023-11-15T03:02:20.737348+0000","last_became_peered":"2023-11-15T03:02:20.737348+0000","last_unstale":"2023-11-15T03:02:20.743428+0000","last_undegraded":"2023-11-15T03:02:20.743428+0000","last_fullsized":"2023-11-15T03:02:20.743428+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.729554+0000","last_change":"2023-11-15T03:02:20.282482+0000","last_active":"2023-11-15T03:02:20.729554+0000","last_peered":"2023-11-15T03:02:20.729554+0000","last_clean":"2023-11-15T03:02:20.729554+0000","last_became_active":"2023-11-15T03:02:20.281117+0000","last_became_peered":"2023-11-15T03:02:20.281117+0000","last_unstale":"2023-11-15T03:02:20.729554+0000","last_undegraded":"2023-11-15T03:02:20.729554+0000","last_fullsized":"2023-11-15T03:02:20.729554+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.749668+0000","last_change":"2023-11-15T03:02:20.734878+0000","last_active":"2023-11-15T03:02:20.749668+0000","last_peered":"2023-11-15T03:02:20.749668+0000","last_clean":"2023-11-15T03:02:20.749668+0000","last_became_active":"2023-11-15T03:02:20.734624+0000","last_became_peered":"2023-11-15T03:02:20.734624+0000","last_unstale":"2023-11-15T03:02:20.749668+0000","last_undegraded":"2023-11-15T03:02:20.749668+0000","last_fullsized":"2023-11-15T03:02:20.749668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.34","version":"76'4","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.754390+0000","last_change":"2023-11-15T03:02:20.280107+0000","last_active":"2023-11-15T03:02:20.754390+0000","last_peered":"2023-11-15T03:02:20.754390+0000","last_clean":"2023-11-15T03:02:20.754390+0000","last_became_active":"2023-11-15T03:02:20.279672+0000","last_became_peered":"2023-11-15T03:02:20.279672+0000","last_unstale":"2023-11-15T03:02:20.754390+0000","last_undegraded":"2023-11-15T03:02:20.754390+0000","last_fullsized":"2023-11-15T03:02:20.754390+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.749185+0000","last_change":"2023-11-15T03:02:20.744053+0000","last_active":"2023-11-15T03:02:20.749185+0000","last_peered":"2023-11-15T03:02:20.749185+0000","last_clean":"2023-11-15T03:02:20.749185+0000","last_became_active":"2023-11-15T03:02:20.742348+0000","last_became_peered":"2023-11-15T03:02:20.742348+0000","last_unstale":"2023-11-15T03:02:20.749185+0000","last_undegraded":"2023-11-15T03:02:20.749185+0000","last_fullsized":"2023-11-15T03:02:20.749185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.33","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.746597+0000","last_change":"2023-11-15T03:02:20.285168+0000","last_active":"2023-11-15T03:02:20.746597+0000","last_peered":"2023-11-15T03:02:20.746597+0000","last_clean":"2023-11-15T03:02:20.746597+0000","last_became_active":"2023-11-15T03:02:20.284913+0000","last_became_peered":"2023-11-15T03:02:20.284913+0000","last_unstale":"2023-11-15T03:02:20.746597+0000","last_undegraded":"2023-11-15T03:02:20.746597+0000","last_fullsized":"2023-11-15T03:02:20.746597+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.753410+0000","last_change":"2023-11-15T03:02:20.751877+0000","last_active":"2023-11-15T03:02:20.753410+0000","last_peered":"2023-11-15T03:02:20.753410+0000","last_clean":"2023-11-15T03:02:20.753410+0000","last_became_active":"2023-11-15T03:02:20.751590+0000","last_became_peered":"2023-11-15T03:02:20.751590+0000","last_unstale":"2023-11-15T03:02:20.753410+0000","last_undegraded":"2023-11-15T03:02:20.753410+0000","last_fullsized":"2023-11-15T03:02:20.753410+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.723105+0000","last_change":"2023-11-15T03:02:20.276550+0000","last_active":"2023-11-15T03:02:20.723105+0000","last_peered":"2023-11-15T03:02:20.723105+0000","last_clean":"2023-11-15T03:02:20.723105+0000","last_became_active":"2023-11-15T03:02:20.275732+0000","last_became_peered":"2023-11-15T03:02:20.275732+0000","last_unstale":"2023-11-15T03:02:20.723105+0000","last_undegraded":"2023-11-15T03:02:20.723105+0000","last_fullsized":"2023-11-15T03:02:20.723105+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.749154+0000","last_change":"2023-11-15T03:02:20.744718+0000","last_active":"2023-11-15T03:02:20.749154+0000","last_peered":"2023-11-15T03:02:20.749154+0000","last_clean":"2023-11-15T03:02:20.749154+0000","last_became_active":"2023-11-15T03:02:20.744096+0000","last_became_peered":"2023-11-15T03:02:20.744096+0000","last_unstale":"2023-11-15T03:02:20.749154+0000","last_undegraded":"2023-11-15T03:02:20.749154+0000","last_fullsized":"2023-11-15T03:02:20.749154+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.725954+0000","last_change":"2023-11-15T03:02:20.285069+0000","last_active":"2023-11-15T03:02:20.725954+0000","last_peered":"2023-11-15T03:02:20.725954+0000","last_clean":"2023-11-15T03:02:20.725954+0000","last_became_active":"2023-11-15T03:02:20.284784+0000","last_became_peered":"2023-11-15T03:02:20.284784+0000","last_unstale":"2023-11-15T03:02:20.725954+0000","last_undegraded":"2023-11-15T03:02:20.725954+0000","last_fullsized":"2023-11-15T03:02:20.725954+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.746498+0000","last_change":"2023-11-15T03:02:20.736058+0000","last_active":"2023-11-15T03:02:20.746498+0000","last_peered":"2023-11-15T03:02:20.746498+0000","last_clean":"2023-11-15T03:02:20.746498+0000","last_became_active":"2023-11-15T03:02:20.735765+0000","last_became_peered":"2023-11-15T03:02:20.735765+0000","last_unstale":"2023-11-15T03:02:20.746498+0000","last_undegraded":"2023-11-15T03:02:20.746498+0000","last_fullsized":"2023-11-15T03:02:20.746498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"76'4","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.756392+0000","last_change":"2023-11-15T03:02:20.284214+0000","last_active":"2023-11-15T03:02:20.756392+0000","last_peered":"2023-11-15T03:02:20.756392+0000","last_clean":"2023-11-15T03:02:20.756392+0000","last_became_active":"2023-11-15T03:02:20.283823+0000","last_became_peered":"2023-11-15T03:02:20.283823+0000","last_unstale":"2023-11-15T03:02:20.756392+0000","last_undegraded":"2023-11-15T03:02:20.756392+0000","last_fullsized":"2023-11-15T03:02:20.756392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.742010+0000","last_change":"2023-11-15T03:02:20.734950+0000","last_active":"2023-11-15T03:02:20.742010+0000","last_peered":"2023-11-15T03:02:20.742010+0000","last_clean":"2023-11-15T03:02:20.742010+0000","last_became_active":"2023-11-15T03:02:20.734605+0000","last_became_peered":"2023-11-15T03:02:20.734605+0000","last_unstale":"2023-11-15T03:02:20.742010+0000","last_undegraded":"2023-11-15T03:02:20.742010+0000","last_fullsized":"2023-11-15T03:02:20.742010+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.723429+0000","last_change":"2023-11-15T03:02:20.275502+0000","last_active":"2023-11-15T03:02:20.723429+0000","last_peered":"2023-11-15T03:02:20.723429+0000","last_clean":"2023-11-15T03:02:20.723429+0000","last_became_active":"2023-11-15T03:02:20.275217+0000","last_became_peered":"2023-11-15T03:02:20.275217+0000","last_unstale":"2023-11-15T03:02:20.723429+0000","last_undegraded":"2023-11-15T03:02:20.723429+0000","last_fullsized":"2023-11-15T03:02:20.723429+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.744291+0000","last_change":"2023-11-15T03:02:20.737075+0000","last_active":"2023-11-15T03:02:20.744291+0000","last_peered":"2023-11-15T03:02:20.744291+0000","last_clean":"2023-11-15T03:02:20.744291+0000","last_became_active":"2023-11-15T03:02:20.736702+0000","last_became_peered":"2023-11-15T03:02:20.736702+0000","last_unstale":"2023-11-15T03:02:20.744291+0000","last_undegraded":"2023-11-15T03:02:20.744291+0000","last_fullsized":"2023-11-15T03:02:20.744291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.714183+0000","last_change":"2023-11-15T03:02:20.282164+0000","last_active":"2023-11-15T03:02:20.714183+0000","last_peered":"2023-11-15T03:02:20.714183+0000","last_clean":"2023-11-15T03:02:20.714183+0000","last_became_active":"2023-11-15T03:02:20.281841+0000","last_became_peered":"2023-11-15T03:02:20.281841+0000","last_unstale":"2023-11-15T03:02:20.714183+0000","last_undegraded":"2023-11-15T03:02:20.714183+0000","last_fullsized":"2023-11-15T03:02:20.714183+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.750388+0000","last_change":"2023-11-15T03:02:20.740936+0000","last_active":"2023-11-15T03:02:20.750388+0000","last_peered":"2023-11-15T03:02:20.750388+0000","last_clean":"2023-11-15T03:02:20.750388+0000","last_became_active":"2023-11-15T03:02:20.740697+0000","last_became_peered":"2023-11-15T03:02:20.740697+0000","last_unstale":"2023-11-15T03:02:20.750388+0000","last_undegraded":"2023-11-15T03:02:20.750388+0000","last_fullsized":"2023-11-15T03:02:20.750388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2d","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.751975+0000","last_change":"2023-11-15T03:02:20.278505+0000","last_active":"2023-11-15T03:02:20.751975+0000","last_peered":"2023-11-15T03:02:20.751975+0000","last_clean":"2023-11-15T03:02:20.751975+0000","last_became_active":"2023-11-15T03:02:20.278225+0000","last_became_peered":"2023-11-15T03:02:20.278225+0000","last_unstale":"2023-11-15T03:02:20.751975+0000","last_undegraded":"2023-11-15T03:02:20.751975+0000","last_fullsized":"2023-11-15T03:02:20.751975+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.738650+0000","last_change":"2023-11-15T03:02:20.726564+0000","last_active":"2023-11-15T03:02:20.738650+0000","last_peered":"2023-11-15T03:02:20.738650+0000","last_clean":"2023-11-15T03:02:20.738650+0000","last_became_active":"2023-11-15T03:02:20.726104+0000","last_became_peered":"2023-11-15T03:02:20.726104+0000","last_unstale":"2023-11-15T03:02:20.738650+0000","last_undegraded":"2023-11-15T03:02:20.738650+0000","last_fullsized":"2023-11-15T03:02:20.738650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.727457+0000","last_change":"2023-11-15T03:02:20.276336+0000","last_active":"2023-11-15T03:02:20.727457+0000","last_peered":"2023-11-15T03:02:20.727457+0000","last_clean":"2023-11-15T03:02:20.727457+0000","last_became_active":"2023-11-15T03:02:20.276000+0000","last_became_peered":"2023-11-15T03:02:20.276000+0000","last_unstale":"2023-11-15T03:02:20.727457+0000","last_undegraded":"2023-11-15T03:02:20.727457+0000","last_fullsized":"2023-11-15T03:02:20.727457+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.752513+0000","last_change":"2023-11-15T03:02:20.743490+0000","last_active":"2023-11-15T03:02:20.752513+0000","last_peered":"2023-11-15T03:02:20.752513+0000","last_clean":"2023-11-15T03:02:20.752513+0000","last_became_active":"2023-11-15T03:02:20.743192+0000","last_became_peered":"2023-11-15T03:02:20.743192+0000","last_unstale":"2023-11-15T03:02:20.752513+0000","last_undegraded":"2023-11-15T03:02:20.752513+0000","last_fullsized":"2023-11-15T03:02:20.752513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2b","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.751383+0000","last_change":"2023-11-15T03:02:20.293538+0000","last_active":"2023-11-15T03:02:20.751383+0000","last_peered":"2023-11-15T03:02:20.751383+0000","last_clean":"2023-11-15T03:02:20.751383+0000","last_became_active":"2023-11-15T03:02:20.290652+0000","last_became_peered":"2023-11-15T03:02:20.290652+0000","last_unstale":"2023-11-15T03:02:20.751383+0000","last_undegraded":"2023-11-15T03:02:20.751383+0000","last_fullsized":"2023-11-15T03:02:20.751383+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.743404+0000","last_change":"2023-11-15T03:02:20.737535+0000","last_active":"2023-11-15T03:02:20.743404+0000","last_peered":"2023-11-15T03:02:20.743404+0000","last_clean":"2023-11-15T03:02:20.743404+0000","last_became_active":"2023-11-15T03:02:20.736675+0000","last_became_peered":"2023-11-15T03:02:20.736675+0000","last_unstale":"2023-11-15T03:02:20.743404+0000","last_undegraded":"2023-11-15T03:02:20.743404+0000","last_fullsized":"2023-11-15T03:02:20.743404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.724426+0000","last_change":"2023-11-15T03:02:20.292132+0000","last_active":"2023-11-15T03:02:20.724426+0000","last_peered":"2023-11-15T03:02:20.724426+0000","last_clean":"2023-11-15T03:02:20.724426+0000","last_became_active":"2023-11-15T03:02:20.288549+0000","last_became_peered":"2023-11-15T03:02:20.288549+0000","last_unstale":"2023-11-15T03:02:20.724426+0000","last_undegraded":"2023-11-15T03:02:20.724426+0000","last_fullsized":"2023-11-15T03:02:20.724426+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.748655+0000","last_change":"2023-11-15T03:02:20.740354+0000","last_active":"2023-11-15T03:02:20.748655+0000","last_peered":"2023-11-15T03:02:20.748655+0000","last_clean":"2023-11-15T03:02:20.748655+0000","last_became_active":"2023-11-15T03:02:20.739907+0000","last_became_peered":"2023-11-15T03:02:20.739907+0000","last_unstale":"2023-11-15T03:02:20.748655+0000","last_undegraded":"2023-11-15T03:02:20.748655+0000","last_fullsized":"2023-11-15T03:02:20.748655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.730651+0000","last_change":"2023-11-15T03:02:20.276701+0000","last_active":"2023-11-15T03:02:20.730651+0000","last_peered":"2023-11-15T03:02:20.730651+0000","last_clean":"2023-11-15T03:02:20.730651+0000","last_became_active":"2023-11-15T03:02:20.275656+0000","last_became_peered":"2023-11-15T03:02:20.275656+0000","last_unstale":"2023-11-15T03:02:20.730651+0000","last_undegraded":"2023-11-15T03:02:20.730651+0000","last_fullsized":"2023-11-15T03:02:20.730651+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.753097+0000","last_change":"2023-11-15T03:02:20.746260+0000","last_active":"2023-11-15T03:02:20.753097+0000","last_peered":"2023-11-15T03:02:20.753097+0000","last_clean":"2023-11-15T03:02:20.753097+0000","last_became_active":"2023-11-15T03:02:20.744833+0000","last_became_peered":"2023-11-15T03:02:20.744833+0000","last_unstale":"2023-11-15T03:02:20.753097+0000","last_undegraded":"2023-11-15T03:02:20.753097+0000","last_fullsized":"2023-11-15T03:02:20.753097+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.724594+0000","last_change":"2023-11-15T03:02:20.286122+0000","last_active":"2023-11-15T03:02:20.724594+0000","last_peered":"2023-11-15T03:02:20.724594+0000","last_clean":"2023-11-15T03:02:20.724594+0000","last_became_active":"2023-11-15T03:02:20.285859+0000","last_became_peered":"2023-11-15T03:02:20.285859+0000","last_unstale":"2023-11-15T03:02:20.724594+0000","last_undegraded":"2023-11-15T03:02:20.724594+0000","last_fullsized":"2023-11-15T03:02:20.724594+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.753858+0000","last_change":"2023-11-15T03:02:20.747332+0000","last_active":"2023-11-15T03:02:20.753858+0000","last_peered":"2023-11-15T03:02:20.753858+0000","last_clean":"2023-11-15T03:02:20.753858+0000","last_became_active":"2023-11-15T03:02:20.745865+0000","last_became_peered":"2023-11-15T03:02:20.745865+0000","last_unstale":"2023-11-15T03:02:20.753858+0000","last_undegraded":"2023-11-15T03:02:20.753858+0000","last_fullsized":"2023-11-15T03:02:20.753858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.723996+0000","last_change":"2023-11-15T03:02:20.286821+0000","last_active":"2023-11-15T03:02:20.723996+0000","last_peered":"2023-11-15T03:02:20.723996+0000","last_clean":"2023-11-15T03:02:20.723996+0000","last_became_active":"2023-11-15T03:02:20.286544+0000","last_became_peered":"2023-11-15T03:02:20.286544+0000","last_unstale":"2023-11-15T03:02:20.723996+0000","last_undegraded":"2023-11-15T03:02:20.723996+0000","last_fullsized":"2023-11-15T03:02:20.723996+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.749278+0000","last_change":"2023-11-15T03:02:20.745921+0000","last_active":"2023-11-15T03:02:20.749278+0000","last_peered":"2023-11-15T03:02:20.749278+0000","last_clean":"2023-11-15T03:02:20.749278+0000","last_became_active":"2023-11-15T03:02:20.745541+0000","last_became_peered":"2023-11-15T03:02:20.745541+0000","last_unstale":"2023-11-15T03:02:20.749278+0000","last_undegraded":"2023-11-15T03:02:20.749278+0000","last_fullsized":"2023-11-15T03:02:20.749278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.727988+0000","last_change":"2023-11-15T03:02:20.282482+0000","last_active":"2023-11-15T03:02:20.727988+0000","last_peered":"2023-11-15T03:02:20.727988+0000","last_clean":"2023-11-15T03:02:20.727988+0000","last_became_active":"2023-11-15T03:02:20.280516+0000","last_became_peered":"2023-11-15T03:02:20.280516+0000","last_unstale":"2023-11-15T03:02:20.727988+0000","last_undegraded":"2023-11-15T03:02:20.727988+0000","last_fullsized":"2023-11-15T03:02:20.727988+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.752870+0000","last_change":"2023-11-15T03:02:20.746541+0000","last_active":"2023-11-15T03:02:20.752870+0000","last_peered":"2023-11-15T03:02:20.752870+0000","last_clean":"2023-11-15T03:02:20.752870+0000","last_became_active":"2023-11-15T03:02:20.746114+0000","last_became_peered":"2023-11-15T03:02:20.746114+0000","last_unstale":"2023-11-15T03:02:20.752870+0000","last_undegraded":"2023-11-15T03:02:20.752870+0000","last_fullsized":"2023-11-15T03:02:20.752870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.719708+0000","last_change":"2023-11-15T03:02:20.274474+0000","last_active":"2023-11-15T03:02:20.719708+0000","last_peered":"2023-11-15T03:02:20.719708+0000","last_clean":"2023-11-15T03:02:20.719708+0000","last_became_active":"2023-11-15T03:02:20.274126+0000","last_became_peered":"2023-11-15T03:02:20.274126+0000","last_unstale":"2023-11-15T03:02:20.719708+0000","last_undegraded":"2023-11-15T03:02:20.719708+0000","last_fullsized":"2023-11-15T03:02:20.719708+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.743138+0000","last_change":"2023-11-15T03:02:20.739107+0000","last_active":"2023-11-15T03:02:20.743138+0000","last_peered":"2023-11-15T03:02:20.743138+0000","last_clean":"2023-11-15T03:02:20.743138+0000","last_became_active":"2023-11-15T03:02:20.738715+0000","last_became_peered":"2023-11-15T03:02:20.738715+0000","last_unstale":"2023-11-15T03:02:20.743138+0000","last_undegraded":"2023-11-15T03:02:20.743138+0000","last_fullsized":"2023-11-15T03:02:20.743138+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.721653+0000","last_change":"2023-11-15T03:02:20.288868+0000","last_active":"2023-11-15T03:02:20.721653+0000","last_peered":"2023-11-15T03:02:20.721653+0000","last_clean":"2023-11-15T03:02:20.721653+0000","last_became_active":"2023-11-15T03:02:20.288475+0000","last_became_peered":"2023-11-15T03:02:20.288475+0000","last_unstale":"2023-11-15T03:02:20.721653+0000","last_undegraded":"2023-11-15T03:02:20.721653+0000","last_fullsized":"2023-11-15T03:02:20.721653+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.753207+0000","last_change":"2023-11-15T03:02:20.744976+0000","last_active":"2023-11-15T03:02:20.753207+0000","last_peered":"2023-11-15T03:02:20.753207+0000","last_clean":"2023-11-15T03:02:20.753207+0000","last_became_active":"2023-11-15T03:02:20.743529+0000","last_became_peered":"2023-11-15T03:02:20.743529+0000","last_unstale":"2023-11-15T03:02:20.753207+0000","last_undegraded":"2023-11-15T03:02:20.753207+0000","last_fullsized":"2023-11-15T03:02:20.753207+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.720047+0000","last_change":"2023-11-15T03:02:20.273095+0000","last_active":"2023-11-15T03:02:20.720047+0000","last_peered":"2023-11-15T03:02:20.720047+0000","last_clean":"2023-11-15T03:02:20.720047+0000","last_became_active":"2023-11-15T03:02:20.272789+0000","last_became_peered":"2023-11-15T03:02:20.272789+0000","last_unstale":"2023-11-15T03:02:20.720047+0000","last_undegraded":"2023-11-15T03:02:20.720047+0000","last_fullsized":"2023-11-15T03:02:20.720047+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.747000+0000","last_change":"2023-11-15T03:02:20.739196+0000","last_active":"2023-11-15T03:02:20.747000+0000","last_peered":"2023-11-15T03:02:20.747000+0000","last_clean":"2023-11-15T03:02:20.747000+0000","last_became_active":"2023-11-15T03:02:20.738539+0000","last_became_peered":"2023-11-15T03:02:20.738539+0000","last_unstale":"2023-11-15T03:02:20.747000+0000","last_undegraded":"2023-11-15T03:02:20.747000+0000","last_fullsized":"2023-11-15T03:02:20.747000+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.726540+0000","last_change":"2023-11-15T03:02:20.279761+0000","last_active":"2023-11-15T03:02:20.726540+0000","last_peered":"2023-11-15T03:02:20.726540+0000","last_clean":"2023-11-15T03:02:20.726540+0000","last_became_active":"2023-11-15T03:02:20.279488+0000","last_became_peered":"2023-11-15T03:02:20.279488+0000","last_unstale":"2023-11-15T03:02:20.726540+0000","last_undegraded":"2023-11-15T03:02:20.726540+0000","last_fullsized":"2023-11-15T03:02:20.726540+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.750750+0000","last_change":"2023-11-15T03:02:20.749026+0000","last_active":"2023-11-15T03:02:20.750750+0000","last_peered":"2023-11-15T03:02:20.750750+0000","last_clean":"2023-11-15T03:02:20.750750+0000","last_became_active":"2023-11-15T03:02:20.748607+0000","last_became_peered":"2023-11-15T03:02:20.748607+0000","last_unstale":"2023-11-15T03:02:20.750750+0000","last_undegraded":"2023-11-15T03:02:20.750750+0000","last_fullsized":"2023-11-15T03:02:20.750750+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.727533+0000","last_change":"2023-11-15T03:02:20.281300+0000","last_active":"2023-11-15T03:02:20.727533+0000","last_peered":"2023-11-15T03:02:20.727533+0000","last_clean":"2023-11-15T03:02:20.727533+0000","last_became_active":"2023-11-15T03:02:20.280992+0000","last_became_peered":"2023-11-15T03:02:20.280992+0000","last_unstale":"2023-11-15T03:02:20.727533+0000","last_undegraded":"2023-11-15T03:02:20.727533+0000","last_fullsized":"2023-11-15T03:02:20.727533+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.741421+0000","last_change":"2023-11-15T03:02:20.735782+0000","last_active":"2023-11-15T03:02:20.741421+0000","last_peered":"2023-11-15T03:02:20.741421+0000","last_clean":"2023-11-15T03:02:20.741421+0000","last_became_active":"2023-11-15T03:02:20.735491+0000","last_became_peered":"2023-11-15T03:02:20.735491+0000","last_unstale":"2023-11-15T03:02:20.741421+0000","last_undegraded":"2023-11-15T03:02:20.741421+0000","last_fullsized":"2023-11-15T03:02:20.741421+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.728298+0000","last_change":"2023-11-15T03:02:20.280351+0000","last_active":"2023-11-15T03:02:20.728298+0000","last_peered":"2023-11-15T03:02:20.728298+0000","last_clean":"2023-11-15T03:02:20.728298+0000","last_became_active":"2023-11-15T03:02:20.279706+0000","last_became_peered":"2023-11-15T03:02:20.279706+0000","last_unstale":"2023-11-15T03:02:20.728298+0000","last_undegraded":"2023-11-15T03:02:20.728298+0000","last_fullsized":"2023-11-15T03:02:20.728298+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.747882+0000","last_change":"2023-11-15T03:02:20.736251+0000","last_active":"2023-11-15T03:02:20.747882+0000","last_peered":"2023-11-15T03:02:20.747882+0000","last_clean":"2023-11-15T03:02:20.747882+0000","last_became_active":"2023-11-15T03:02:20.735947+0000","last_became_peered":"2023-11-15T03:02:20.735947+0000","last_unstale":"2023-11-15T03:02:20.747882+0000","last_undegraded":"2023-11-15T03:02:20.747882+0000","last_fullsized":"2023-11-15T03:02:20.747882+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.741640+0000","last_change":"2023-11-15T03:02:20.275451+0000","last_active":"2023-11-15T03:02:20.741640+0000","last_peered":"2023-11-15T03:02:20.741640+0000","last_clean":"2023-11-15T03:02:20.741640+0000","last_became_active":"2023-11-15T03:02:20.275177+0000","last_became_peered":"2023-11-15T03:02:20.275177+0000","last_unstale":"2023-11-15T03:02:20.741640+0000","last_undegraded":"2023-11-15T03:02:20.741640+0000","last_fullsized":"2023-11-15T03:02:20.741640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.740852+0000","last_change":"2023-11-15T03:02:20.734278+0000","last_active":"2023-11-15T03:02:20.740852+0000","last_peered":"2023-11-15T03:02:20.740852+0000","last_clean":"2023-11-15T03:02:20.740852+0000","last_became_active":"2023-11-15T03:02:20.734000+0000","last_became_peered":"2023-11-15T03:02:20.734000+0000","last_unstale":"2023-11-15T03:02:20.740852+0000","last_undegraded":"2023-11-15T03:02:20.740852+0000","last_fullsized":"2023-11-15T03:02:20.740852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.752283+0000","last_change":"2023-11-15T03:02:20.286904+0000","last_active":"2023-11-15T03:02:20.752283+0000","last_peered":"2023-11-15T03:02:20.752283+0000","last_clean":"2023-11-15T03:02:20.752283+0000","last_became_active":"2023-11-15T03:02:20.286652+0000","last_became_peered":"2023-11-15T03:02:20.286652+0000","last_unstale":"2023-11-15T03:02:20.752283+0000","last_undegraded":"2023-11-15T03:02:20.752283+0000","last_fullsized":"2023-11-15T03:02:20.752283+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.750510+0000","last_change":"2023-11-15T03:02:20.745856+0000","last_active":"2023-11-15T03:02:20.750510+0000","last_peered":"2023-11-15T03:02:20.750510+0000","last_clean":"2023-11-15T03:02:20.750510+0000","last_became_active":"2023-11-15T03:02:20.745318+0000","last_became_peered":"2023-11-15T03:02:20.745318+0000","last_unstale":"2023-11-15T03:02:20.750510+0000","last_undegraded":"2023-11-15T03:02:20.750510+0000","last_fullsized":"2023-11-15T03:02:20.750510+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.746199+0000","last_change":"2023-11-15T03:02:20.287943+0000","last_active":"2023-11-15T03:02:20.746199+0000","last_peered":"2023-11-15T03:02:20.746199+0000","last_clean":"2023-11-15T03:02:20.746199+0000","last_became_active":"2023-11-15T03:02:20.287644+0000","last_became_peered":"2023-11-15T03:02:20.287644+0000","last_unstale":"2023-11-15T03:02:20.746199+0000","last_undegraded":"2023-11-15T03:02:20.746199+0000","last_fullsized":"2023-11-15T03:02:20.746199+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.738630+0000","last_change":"2023-11-15T03:02:20.726378+0000","last_active":"2023-11-15T03:02:20.738630+0000","last_peered":"2023-11-15T03:02:20.738630+0000","last_clean":"2023-11-15T03:02:20.738630+0000","last_became_active":"2023-11-15T03:02:20.725792+0000","last_became_peered":"2023-11-15T03:02:20.725792+0000","last_unstale":"2023-11-15T03:02:20.738630+0000","last_undegraded":"2023-11-15T03:02:20.738630+0000","last_fullsized":"2023-11-15T03:02:20.738630+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.731116+0000","last_change":"2023-11-15T03:02:20.287833+0000","last_active":"2023-11-15T03:02:20.731116+0000","last_peered":"2023-11-15T03:02:20.731116+0000","last_clean":"2023-11-15T03:02:20.731116+0000","last_became_active":"2023-11-15T03:02:20.287294+0000","last_became_peered":"2023-11-15T03:02:20.287294+0000","last_unstale":"2023-11-15T03:02:20.731116+0000","last_undegraded":"2023-11-15T03:02:20.731116+0000","last_fullsized":"2023-11-15T03:02:20.731116+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.751643+0000","last_change":"2023-11-15T03:02:20.749149+0000","last_active":"2023-11-15T03:02:20.751643+0000","last_peered":"2023-11-15T03:02:20.751643+0000","last_clean":"2023-11-15T03:02:20.751643+0000","last_became_active":"2023-11-15T03:02:20.747720+0000","last_became_peered":"2023-11-15T03:02:20.747720+0000","last_unstale":"2023-11-15T03:02:20.751643+0000","last_undegraded":"2023-11-15T03:02:20.751643+0000","last_fullsized":"2023-11-15T03:02:20.751643+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.722622+0000","last_change":"2023-11-15T03:02:20.290914+0000","last_active":"2023-11-15T03:02:20.722622+0000","last_peered":"2023-11-15T03:02:20.722622+0000","last_clean":"2023-11-15T03:02:20.722622+0000","last_became_active":"2023-11-15T03:02:20.290227+0000","last_became_peered":"2023-11-15T03:02:20.290227+0000","last_unstale":"2023-11-15T03:02:20.722622+0000","last_undegraded":"2023-11-15T03:02:20.722622+0000","last_fullsized":"2023-11-15T03:02:20.722622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.751461+0000","last_change":"2023-11-15T03:02:20.748167+0000","last_active":"2023-11-15T03:02:20.751461+0000","last_peered":"2023-11-15T03:02:20.751461+0000","last_clean":"2023-11-15T03:02:20.751461+0000","last_became_active":"2023-11-15T03:02:20.747887+0000","last_became_peered":"2023-11-15T03:02:20.747887+0000","last_unstale":"2023-11-15T03:02:20.751461+0000","last_undegraded":"2023-11-15T03:02:20.751461+0000","last_fullsized":"2023-11-15T03:02:20.751461+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.726662+0000","last_change":"2023-11-15T03:02:20.272187+0000","last_active":"2023-11-15T03:02:20.726662+0000","last_peered":"2023-11-15T03:02:20.726662+0000","last_clean":"2023-11-15T03:02:20.726662+0000","last_became_active":"2023-11-15T03:02:20.271834+0000","last_became_peered":"2023-11-15T03:02:20.271834+0000","last_unstale":"2023-11-15T03:02:20.726662+0000","last_undegraded":"2023-11-15T03:02:20.726662+0000","last_fullsized":"2023-11-15T03:02:20.726662+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.744559+0000","last_change":"2023-11-15T03:02:20.739189+0000","last_active":"2023-11-15T03:02:20.744559+0000","last_peered":"2023-11-15T03:02:20.744559+0000","last_clean":"2023-11-15T03:02:20.744559+0000","last_became_active":"2023-11-15T03:02:20.738107+0000","last_became_peered":"2023-11-15T03:02:20.738107+0000","last_unstale":"2023-11-15T03:02:20.744559+0000","last_undegraded":"2023-11-15T03:02:20.744559+0000","last_fullsized":"2023-11-15T03:02:20.744559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.726041+0000","last_change":"2023-11-15T03:02:20.284564+0000","last_active":"2023-11-15T03:02:20.726041+0000","last_peered":"2023-11-15T03:02:20.726041+0000","last_clean":"2023-11-15T03:02:20.726041+0000","last_became_active":"2023-11-15T03:02:20.284292+0000","last_became_peered":"2023-11-15T03:02:20.284292+0000","last_unstale":"2023-11-15T03:02:20.726041+0000","last_undegraded":"2023-11-15T03:02:20.726041+0000","last_fullsized":"2023-11-15T03:02:20.726041+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.758163+0000","last_change":"2023-11-15T03:02:20.754792+0000","last_active":"2023-11-15T03:02:20.758163+0000","last_peered":"2023-11-15T03:02:20.758163+0000","last_clean":"2023-11-15T03:02:20.758163+0000","last_became_active":"2023-11-15T03:02:20.754442+0000","last_became_peered":"2023-11-15T03:02:20.754442+0000","last_unstale":"2023-11-15T03:02:20.758163+0000","last_undegraded":"2023-11-15T03:02:20.758163+0000","last_fullsized":"2023-11-15T03:02:20.758163+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.723436+0000","last_change":"2023-11-15T03:02:20.278296+0000","last_active":"2023-11-15T03:02:20.723436+0000","last_peered":"2023-11-15T03:02:20.723436+0000","last_clean":"2023-11-15T03:02:20.723436+0000","last_became_active":"2023-11-15T03:02:20.277760+0000","last_became_peered":"2023-11-15T03:02:20.277760+0000","last_unstale":"2023-11-15T03:02:20.723436+0000","last_undegraded":"2023-11-15T03:02:20.723436+0000","last_fullsized":"2023-11-15T03:02:20.723436+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745485+0000","last_change":"2023-11-15T03:02:20.741587+0000","last_active":"2023-11-15T03:02:20.745485+0000","last_peered":"2023-11-15T03:02:20.745485+0000","last_clean":"2023-11-15T03:02:20.745485+0000","last_became_active":"2023-11-15T03:02:20.739815+0000","last_became_peered":"2023-11-15T03:02:20.739815+0000","last_unstale":"2023-11-15T03:02:20.745485+0000","last_undegraded":"2023-11-15T03:02:20.745485+0000","last_fullsized":"2023-11-15T03:02:20.745485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.730598+0000","last_change":"2023-11-15T03:02:20.289756+0000","last_active":"2023-11-15T03:02:20.730598+0000","last_peered":"2023-11-15T03:02:20.730598+0000","last_clean":"2023-11-15T03:02:20.730598+0000","last_became_active":"2023-11-15T03:02:20.289374+0000","last_became_peered":"2023-11-15T03:02:20.289374+0000","last_unstale":"2023-11-15T03:02:20.730598+0000","last_undegraded":"2023-11-15T03:02:20.730598+0000","last_fullsized":"2023-11-15T03:02:20.730598+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745148+0000","last_change":"2023-11-15T03:02:20.739098+0000","last_active":"2023-11-15T03:02:20.745148+0000","last_peered":"2023-11-15T03:02:20.745148+0000","last_clean":"2023-11-15T03:02:20.745148+0000","last_became_active":"2023-11-15T03:02:20.738546+0000","last_became_peered":"2023-11-15T03:02:20.738546+0000","last_unstale":"2023-11-15T03:02:20.745148+0000","last_undegraded":"2023-11-15T03:02:20.745148+0000","last_fullsized":"2023-11-15T03:02:20.745148+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.728101+0000","last_change":"2023-11-15T03:02:20.278051+0000","last_active":"2023-11-15T03:02:20.728101+0000","last_peered":"2023-11-15T03:02:20.728101+0000","last_clean":"2023-11-15T03:02:20.728101+0000","last_became_active":"2023-11-15T03:02:20.277758+0000","last_became_peered":"2023-11-15T03:02:20.277758+0000","last_unstale":"2023-11-15T03:02:20.728101+0000","last_undegraded":"2023-11-15T03:02:20.728101+0000","last_fullsized":"2023-11-15T03:02:20.728101+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.737566+0000","last_change":"2023-11-15T03:02:20.725085+0000","last_active":"2023-11-15T03:02:20.737566+0000","last_peered":"2023-11-15T03:02:20.737566+0000","last_clean":"2023-11-15T03:02:20.737566+0000","last_became_active":"2023-11-15T03:02:20.724827+0000","last_became_peered":"2023-11-15T03:02:20.724827+0000","last_unstale":"2023-11-15T03:02:20.737566+0000","last_undegraded":"2023-11-15T03:02:20.737566+0000","last_fullsized":"2023-11-15T03:02:20.737566+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"76'3","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.751205+0000","last_change":"2023-11-15T03:02:20.272400+0000","last_active":"2023-11-15T03:02:20.751205+0000","last_peered":"2023-11-15T03:02:20.751205+0000","last_clean":"2023-11-15T03:02:20.751205+0000","last_became_active":"2023-11-15T03:02:20.272107+0000","last_became_peered":"2023-11-15T03:02:20.272107+0000","last_unstale":"2023-11-15T03:02:20.751205+0000","last_undegraded":"2023-11-15T03:02:20.751205+0000","last_fullsized":"2023-11-15T03:02:20.751205+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.748847+0000","last_change":"2023-11-15T03:02:20.740969+0000","last_active":"2023-11-15T03:02:20.748847+0000","last_peered":"2023-11-15T03:02:20.748847+0000","last_clean":"2023-11-15T03:02:20.748847+0000","last_became_active":"2023-11-15T03:02:20.740724+0000","last_became_peered":"2023-11-15T03:02:20.740724+0000","last_unstale":"2023-11-15T03:02:20.748847+0000","last_undegraded":"2023-11-15T03:02:20.748847+0000","last_fullsized":"2023-11-15T03:02:20.748847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.751582+0000","last_change":"2023-11-15T03:02:20.291581+0000","last_active":"2023-11-15T03:02:20.751582+0000","last_peered":"2023-11-15T03:02:20.751582+0000","last_clean":"2023-11-15T03:02:20.751582+0000","last_became_active":"2023-11-15T03:02:20.291122+0000","last_became_peered":"2023-11-15T03:02:20.291122+0000","last_unstale":"2023-11-15T03:02:20.751582+0000","last_undegraded":"2023-11-15T03:02:20.751582+0000","last_fullsized":"2023-11-15T03:02:20.751582+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.749203+0000","last_change":"2023-11-15T03:02:20.744457+0000","last_active":"2023-11-15T03:02:20.749203+0000","last_peered":"2023-11-15T03:02:20.749203+0000","last_clean":"2023-11-15T03:02:20.749203+0000","last_became_active":"2023-11-15T03:02:20.743512+0000","last_became_peered":"2023-11-15T03:02:20.743512+0000","last_unstale":"2023-11-15T03:02:20.749203+0000","last_undegraded":"2023-11-15T03:02:20.749203+0000","last_fullsized":"2023-11-15T03:02:20.749203+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.748201+0000","last_change":"2023-11-15T03:02:20.285924+0000","last_active":"2023-11-15T03:02:20.748201+0000","last_peered":"2023-11-15T03:02:20.748201+0000","last_clean":"2023-11-15T03:02:20.748201+0000","last_became_active":"2023-11-15T03:02:20.285590+0000","last_became_peered":"2023-11-15T03:02:20.285590+0000","last_unstale":"2023-11-15T03:02:20.748201+0000","last_undegraded":"2023-11-15T03:02:20.748201+0000","last_fullsized":"2023-11-15T03:02:20.748201+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.756505+0000","last_change":"2023-11-15T03:02:20.752529+0000","last_active":"2023-11-15T03:02:20.756505+0000","last_peered":"2023-11-15T03:02:20.756505+0000","last_clean":"2023-11-15T03:02:20.756505+0000","last_became_active":"2023-11-15T03:02:20.752153+0000","last_became_peered":"2023-11-15T03:02:20.752153+0000","last_unstale":"2023-11-15T03:02:20.756505+0000","last_undegraded":"2023-11-15T03:02:20.756505+0000","last_fullsized":"2023-11-15T03:02:20.756505+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.729140+0000","last_change":"2023-11-15T03:02:20.290468+0000","last_active":"2023-11-15T03:02:20.729140+0000","last_peered":"2023-11-15T03:02:20.729140+0000","last_clean":"2023-11-15T03:02:20.729140+0000","last_became_active":"2023-11-15T03:02:20.290104+0000","last_became_peered":"2023-11-15T03:02:20.290104+0000","last_unstale":"2023-11-15T03:02:20.729140+0000","last_undegraded":"2023-11-15T03:02:20.729140+0000","last_fullsized":"2023-11-15T03:02:20.729140+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.753109+0000","last_change":"2023-11-15T03:02:20.748616+0000","last_active":"2023-11-15T03:02:20.753109+0000","last_peered":"2023-11-15T03:02:20.753109+0000","last_clean":"2023-11-15T03:02:20.753109+0000","last_became_active":"2023-11-15T03:02:20.748246+0000","last_became_peered":"2023-11-15T03:02:20.748246+0000","last_unstale":"2023-11-15T03:02:20.753109+0000","last_undegraded":"2023-11-15T03:02:20.753109+0000","last_fullsized":"2023-11-15T03:02:20.753109+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.723347+0000","last_change":"2023-11-15T03:02:20.276925+0000","last_active":"2023-11-15T03:02:20.723347+0000","last_peered":"2023-11-15T03:02:20.723347+0000","last_clean":"2023-11-15T03:02:20.723347+0000","last_became_active":"2023-11-15T03:02:20.276660+0000","last_became_peered":"2023-11-15T03:02:20.276660+0000","last_unstale":"2023-11-15T03:02:20.723347+0000","last_undegraded":"2023-11-15T03:02:20.723347+0000","last_fullsized":"2023-11-15T03:02:20.723347+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.756108+0000","last_change":"2023-11-15T03:02:20.752836+0000","last_active":"2023-11-15T03:02:20.756108+0000","last_peered":"2023-11-15T03:02:20.756108+0000","last_clean":"2023-11-15T03:02:20.756108+0000","last_became_active":"2023-11-15T03:02:20.752532+0000","last_became_peered":"2023-11-15T03:02:20.752532+0000","last_unstale":"2023-11-15T03:02:20.756108+0000","last_undegraded":"2023-11-15T03:02:20.756108+0000","last_fullsized":"2023-11-15T03:02:20.756108+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"76'4","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.748653+0000","last_change":"2023-11-15T03:02:20.283810+0000","last_active":"2023-11-15T03:02:20.748653+0000","last_peered":"2023-11-15T03:02:20.748653+0000","last_clean":"2023-11-15T03:02:20.748653+0000","last_became_active":"2023-11-15T03:02:20.283572+0000","last_became_peered":"2023-11-15T03:02:20.283572+0000","last_unstale":"2023-11-15T03:02:20.748653+0000","last_undegraded":"2023-11-15T03:02:20.748653+0000","last_fullsized":"2023-11-15T03:02:20.748653+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.747796+0000","last_change":"2023-11-15T03:02:20.742850+0000","last_active":"2023-11-15T03:02:20.747796+0000","last_peered":"2023-11-15T03:02:20.747796+0000","last_clean":"2023-11-15T03:02:20.747796+0000","last_became_active":"2023-11-15T03:02:20.738256+0000","last_became_peered":"2023-11-15T03:02:20.738256+0000","last_unstale":"2023-11-15T03:02:20.747796+0000","last_undegraded":"2023-11-15T03:02:20.747796+0000","last_fullsized":"2023-11-15T03:02:20.747796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.749942+0000","last_change":"2023-11-15T03:02:20.274513+0000","last_active":"2023-11-15T03:02:20.749942+0000","last_peered":"2023-11-15T03:02:20.749942+0000","last_clean":"2023-11-15T03:02:20.749942+0000","last_became_active":"2023-11-15T03:02:20.274169+0000","last_became_peered":"2023-11-15T03:02:20.274169+0000","last_unstale":"2023-11-15T03:02:20.749942+0000","last_undegraded":"2023-11-15T03:02:20.749942+0000","last_fullsized":"2023-11-15T03:02:20.749942+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.747110+0000","last_change":"2023-11-15T03:02:20.740694+0000","last_active":"2023-11-15T03:02:20.747110+0000","last_peered":"2023-11-15T03:02:20.747110+0000","last_clean":"2023-11-15T03:02:20.747110+0000","last_became_active":"2023-11-15T03:02:20.740377+0000","last_became_peered":"2023-11-15T03:02:20.740377+0000","last_unstale":"2023-11-15T03:02:20.747110+0000","last_undegraded":"2023-11-15T03:02:20.747110+0000","last_fullsized":"2023-11-15T03:02:20.747110+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.729268+0000","last_change":"2023-11-15T03:02:20.278754+0000","last_active":"2023-11-15T03:02:20.729268+0000","last_peered":"2023-11-15T03:02:20.729268+0000","last_clean":"2023-11-15T03:02:20.729268+0000","last_became_active":"2023-11-15T03:02:20.278460+0000","last_became_peered":"2023-11-15T03:02:20.278460+0000","last_unstale":"2023-11-15T03:02:20.729268+0000","last_undegraded":"2023-11-15T03:02:20.729268+0000","last_fullsized":"2023-11-15T03:02:20.729268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.750488+0000","last_change":"2023-11-15T03:02:20.745675+0000","last_active":"2023-11-15T03:02:20.750488+0000","last_peered":"2023-11-15T03:02:20.750488+0000","last_clean":"2023-11-15T03:02:20.750488+0000","last_became_active":"2023-11-15T03:02:20.745053+0000","last_became_peered":"2023-11-15T03:02:20.745053+0000","last_unstale":"2023-11-15T03:02:20.750488+0000","last_undegraded":"2023-11-15T03:02:20.750488+0000","last_fullsized":"2023-11-15T03:02:20.750488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.729824+0000","last_change":"2023-11-15T03:02:20.283476+0000","last_active":"2023-11-15T03:02:20.729824+0000","last_peered":"2023-11-15T03:02:20.729824+0000","last_clean":"2023-11-15T03:02:20.729824+0000","last_became_active":"2023-11-15T03:02:20.283183+0000","last_became_peered":"2023-11-15T03:02:20.283183+0000","last_unstale":"2023-11-15T03:02:20.729824+0000","last_undegraded":"2023-11-15T03:02:20.729824+0000","last_fullsized":"2023-11-15T03:02:20.729824+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.746980+0000","last_change":"2023-11-15T03:02:20.742607+0000","last_active":"2023-11-15T03:02:20.746980+0000","last_peered":"2023-11-15T03:02:20.746980+0000","last_clean":"2023-11-15T03:02:20.746980+0000","last_became_active":"2023-11-15T03:02:20.742287+0000","last_became_peered":"2023-11-15T03:02:20.742287+0000","last_unstale":"2023-11-15T03:02:20.746980+0000","last_undegraded":"2023-11-15T03:02:20.746980+0000","last_fullsized":"2023-11-15T03:02:20.746980+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.728808+0000","last_change":"2023-11-15T03:02:20.285244+0000","last_active":"2023-11-15T03:02:20.728808+0000","last_peered":"2023-11-15T03:02:20.728808+0000","last_clean":"2023-11-15T03:02:20.728808+0000","last_became_active":"2023-11-15T03:02:20.284983+0000","last_became_peered":"2023-11-15T03:02:20.284983+0000","last_unstale":"2023-11-15T03:02:20.728808+0000","last_undegraded":"2023-11-15T03:02:20.728808+0000","last_fullsized":"2023-11-15T03:02:20.728808+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.758135+0000","last_change":"2023-11-15T03:02:20.754340+0000","last_active":"2023-11-15T03:02:20.758135+0000","last_peered":"2023-11-15T03:02:20.758135+0000","last_clean":"2023-11-15T03:02:20.758135+0000","last_became_active":"2023-11-15T03:02:20.753930+0000","last_became_peered":"2023-11-15T03:02:20.753930+0000","last_unstale":"2023-11-15T03:02:20.758135+0000","last_undegraded":"2023-11-15T03:02:20.758135+0000","last_fullsized":"2023-11-15T03:02:20.758135+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.729857+0000","last_change":"2023-11-15T03:02:20.287975+0000","last_active":"2023-11-15T03:02:20.729857+0000","last_peered":"2023-11-15T03:02:20.729857+0000","last_clean":"2023-11-15T03:02:20.729857+0000","last_became_active":"2023-11-15T03:02:20.287650+0000","last_became_peered":"2023-11-15T03:02:20.287650+0000","last_unstale":"2023-11-15T03:02:20.729857+0000","last_undegraded":"2023-11-15T03:02:20.729857+0000","last_fullsized":"2023-11-15T03:02:20.729857+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.758779+0000","last_change":"2023-11-15T03:02:20.756575+0000","last_active":"2023-11-15T03:02:20.758779+0000","last_peered":"2023-11-15T03:02:20.758779+0000","last_clean":"2023-11-15T03:02:20.758779+0000","last_became_active":"2023-11-15T03:02:20.756255+0000","last_became_peered":"2023-11-15T03:02:20.756255+0000","last_unstale":"2023-11-15T03:02:20.758779+0000","last_undegraded":"2023-11-15T03:02:20.758779+0000","last_fullsized":"2023-11-15T03:02:20.758779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.723210+0000","last_change":"2023-11-15T03:02:20.277034+0000","last_active":"2023-11-15T03:02:20.723210+0000","last_peered":"2023-11-15T03:02:20.723210+0000","last_clean":"2023-11-15T03:02:20.723210+0000","last_became_active":"2023-11-15T03:02:20.276787+0000","last_became_peered":"2023-11-15T03:02:20.276787+0000","last_unstale":"2023-11-15T03:02:20.723210+0000","last_undegraded":"2023-11-15T03:02:20.723210+0000","last_fullsized":"2023-11-15T03:02:20.723210+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745673+0000","last_change":"2023-11-15T03:02:20.730485+0000","last_active":"2023-11-15T03:02:20.745673+0000","last_peered":"2023-11-15T03:02:20.745673+0000","last_clean":"2023-11-15T03:02:20.745673+0000","last_became_active":"2023-11-15T03:02:20.730219+0000","last_became_peered":"2023-11-15T03:02:20.730219+0000","last_unstale":"2023-11-15T03:02:20.745673+0000","last_undegraded":"2023-11-15T03:02:20.745673+0000","last_fullsized":"2023-11-15T03:02:20.745673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.735339+0000","last_change":"2023-11-15T03:02:20.288684+0000","last_active":"2023-11-15T03:02:20.735339+0000","last_peered":"2023-11-15T03:02:20.735339+0000","last_clean":"2023-11-15T03:02:20.735339+0000","last_became_active":"2023-11-15T03:02:20.288348+0000","last_became_peered":"2023-11-15T03:02:20.288348+0000","last_unstale":"2023-11-15T03:02:20.735339+0000","last_undegraded":"2023-11-15T03:02:20.735339+0000","last_fullsized":"2023-11-15T03:02:20.735339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.748129+0000","last_change":"2023-11-15T03:02:20.737437+0000","last_active":"2023-11-15T03:02:20.748129+0000","last_peered":"2023-11-15T03:02:20.748129+0000","last_clean":"2023-11-15T03:02:20.748129+0000","last_became_active":"2023-11-15T03:02:20.737189+0000","last_became_peered":"2023-11-15T03:02:20.737189+0000","last_unstale":"2023-11-15T03:02:20.748129+0000","last_undegraded":"2023-11-15T03:02:20.748129+0000","last_fullsized":"2023-11-15T03:02:20.748129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.712554+0000","last_change":"2023-11-15T03:02:20.277917+0000","last_active":"2023-11-15T03:02:20.712554+0000","last_peered":"2023-11-15T03:02:20.712554+0000","last_clean":"2023-11-15T03:02:20.712554+0000","last_became_active":"2023-11-15T03:02:20.275072+0000","last_became_peered":"2023-11-15T03:02:20.275072+0000","last_unstale":"2023-11-15T03:02:20.712554+0000","last_undegraded":"2023-11-15T03:02:20.712554+0000","last_fullsized":"2023-11-15T03:02:20.712554+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.753312+0000","last_change":"2023-11-15T03:02:20.745019+0000","last_active":"2023-11-15T03:02:20.753312+0000","last_peered":"2023-11-15T03:02:20.753312+0000","last_clean":"2023-11-15T03:02:20.753312+0000","last_became_active":"2023-11-15T03:02:20.744563+0000","last_became_peered":"2023-11-15T03:02:20.744563+0000","last_unstale":"2023-11-15T03:02:20.753312+0000","last_undegraded":"2023-11-15T03:02:20.753312+0000","last_fullsized":"2023-11-15T03:02:20.753312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.752327+0000","last_change":"2023-11-15T03:02:20.289275+0000","last_active":"2023-11-15T03:02:20.752327+0000","last_peered":"2023-11-15T03:02:20.752327+0000","last_clean":"2023-11-15T03:02:20.752327+0000","last_became_active":"2023-11-15T03:02:20.288989+0000","last_became_peered":"2023-11-15T03:02:20.288989+0000","last_unstale":"2023-11-15T03:02:20.752327+0000","last_undegraded":"2023-11-15T03:02:20.752327+0000","last_fullsized":"2023-11-15T03:02:20.752327+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":58,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.734004+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T03:02:20.734004+0000","last_peered":"2023-11-15T03:02:20.734004+0000","last_clean":"2023-11-15T03:02:20.734004+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T03:02:20.734004+0000","last_undegraded":"2023-11-15T03:02:20.734004+0000","last_fullsized":"2023-11-15T03:02:20.734004+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.738673+0000","last_change":"2023-11-15T03:02:20.732039+0000","last_active":"2023-11-15T03:02:20.738673+0000","last_peered":"2023-11-15T03:02:20.738673+0000","last_clean":"2023-11-15T03:02:20.738673+0000","last_became_active":"2023-11-15T03:02:20.731756+0000","last_became_peered":"2023-11-15T03:02:20.731756+0000","last_unstale":"2023-11-15T03:02:20.738673+0000","last_undegraded":"2023-11-15T03:02:20.738673+0000","last_fullsized":"2023-11-15T03:02:20.738673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.752627+0000","last_change":"2023-11-15T03:02:20.290797+0000","last_active":"2023-11-15T03:02:20.752627+0000","last_peered":"2023-11-15T03:02:20.752627+0000","last_clean":"2023-11-15T03:02:20.752627+0000","last_became_active":"2023-11-15T03:02:20.290141+0000","last_became_peered":"2023-11-15T03:02:20.290141+0000","last_unstale":"2023-11-15T03:02:20.752627+0000","last_undegraded":"2023-11-15T03:02:20.752627+0000","last_fullsized":"2023-11-15T03:02:20.752627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.714328+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T03:02:20.714328+0000","last_peered":"2023-11-15T03:02:20.714328+0000","last_clean":"2023-11-15T03:02:20.714328+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T03:02:20.714328+0000","last_undegraded":"2023-11-15T03:02:20.714328+0000","last_fullsized":"2023-11-15T03:02:20.714328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.747006+0000","last_change":"2023-11-15T03:02:20.739333+0000","last_active":"2023-11-15T03:02:20.747006+0000","last_peered":"2023-11-15T03:02:20.747006+0000","last_clean":"2023-11-15T03:02:20.747006+0000","last_became_active":"2023-11-15T03:02:20.735473+0000","last_became_peered":"2023-11-15T03:02:20.735473+0000","last_unstale":"2023-11-15T03:02:20.747006+0000","last_undegraded":"2023-11-15T03:02:20.747006+0000","last_fullsized":"2023-11-15T03:02:20.747006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.712972+0000","last_change":"2023-11-15T03:02:20.279939+0000","last_active":"2023-11-15T03:02:20.712972+0000","last_peered":"2023-11-15T03:02:20.712972+0000","last_clean":"2023-11-15T03:02:20.712972+0000","last_became_active":"2023-11-15T03:02:20.279319+0000","last_became_peered":"2023-11-15T03:02:20.279319+0000","last_unstale":"2023-11-15T03:02:20.712972+0000","last_undegraded":"2023-11-15T03:02:20.712972+0000","last_fullsized":"2023-11-15T03:02:20.712972+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":58,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.729992+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T03:02:20.729992+0000","last_peered":"2023-11-15T03:02:20.729992+0000","last_clean":"2023-11-15T03:02:20.729992+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T03:02:20.729992+0000","last_undegraded":"2023-11-15T03:02:20.729992+0000","last_fullsized":"2023-11-15T03:02:20.729992+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.753051+0000","last_change":"2023-11-15T03:02:20.743899+0000","last_active":"2023-11-15T03:02:20.753051+0000","last_peered":"2023-11-15T03:02:20.753051+0000","last_clean":"2023-11-15T03:02:20.753051+0000","last_became_active":"2023-11-15T03:02:20.743634+0000","last_became_peered":"2023-11-15T03:02:20.743634+0000","last_unstale":"2023-11-15T03:02:20.753051+0000","last_undegraded":"2023-11-15T03:02:20.753051+0000","last_fullsized":"2023-11-15T03:02:20.753051+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.723281+0000","last_change":"2023-11-15T03:02:20.286870+0000","last_active":"2023-11-15T03:02:20.723281+0000","last_peered":"2023-11-15T03:02:20.723281+0000","last_clean":"2023-11-15T03:02:20.723281+0000","last_became_active":"2023-11-15T03:02:20.285961+0000","last_became_peered":"2023-11-15T03:02:20.285961+0000","last_unstale":"2023-11-15T03:02:20.723281+0000","last_undegraded":"2023-11-15T03:02:20.723281+0000","last_fullsized":"2023-11-15T03:02:20.723281+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.713011+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T03:02:20.713011+0000","last_peered":"2023-11-15T03:02:20.713011+0000","last_clean":"2023-11-15T03:02:20.713011+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T03:02:20.713011+0000","last_undegraded":"2023-11-15T03:02:20.713011+0000","last_fullsized":"2023-11-15T03:02:20.713011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.743036+0000","last_change":"2023-11-15T03:02:20.735712+0000","last_active":"2023-11-15T03:02:20.743036+0000","last_peered":"2023-11-15T03:02:20.743036+0000","last_clean":"2023-11-15T03:02:20.743036+0000","last_became_active":"2023-11-15T03:02:20.735426+0000","last_became_peered":"2023-11-15T03:02:20.735426+0000","last_unstale":"2023-11-15T03:02:20.743036+0000","last_undegraded":"2023-11-15T03:02:20.743036+0000","last_fullsized":"2023-11-15T03:02:20.743036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.747304+0000","last_change":"2023-11-15T03:02:20.268524+0000","last_active":"2023-11-15T03:02:20.747304+0000","last_peered":"2023-11-15T03:02:20.747304+0000","last_clean":"2023-11-15T03:02:20.747304+0000","last_became_active":"2023-11-15T03:02:20.267955+0000","last_became_peered":"2023-11-15T03:02:20.267955+0000","last_unstale":"2023-11-15T03:02:20.747304+0000","last_undegraded":"2023-11-15T03:02:20.747304+0000","last_fullsized":"2023-11-15T03:02:20.747304+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":69,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.726764+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T03:02:20.726764+0000","last_peered":"2023-11-15T03:02:20.726764+0000","last_clean":"2023-11-15T03:02:20.726764+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T03:02:20.726764+0000","last_undegraded":"2023-11-15T03:02:20.726764+0000","last_fullsized":"2023-11-15T03:02:20.726764+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.752737+0000","last_change":"2023-11-15T03:02:20.741987+0000","last_active":"2023-11-15T03:02:20.752737+0000","last_peered":"2023-11-15T03:02:20.752737+0000","last_clean":"2023-11-15T03:02:20.752737+0000","last_became_active":"2023-11-15T03:02:20.741662+0000","last_became_peered":"2023-11-15T03:02:20.741662+0000","last_unstale":"2023-11-15T03:02:20.752737+0000","last_undegraded":"2023-11-15T03:02:20.752737+0000","last_fullsized":"2023-11-15T03:02:20.752737+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.719624+0000","last_change":"2023-11-15T03:02:20.282979+0000","last_active":"2023-11-15T03:02:20.719624+0000","last_peered":"2023-11-15T03:02:20.719624+0000","last_clean":"2023-11-15T03:02:20.719624+0000","last_became_active":"2023-11-15T03:02:20.282672+0000","last_became_peered":"2023-11-15T03:02:20.282672+0000","last_unstale":"2023-11-15T03:02:20.719624+0000","last_undegraded":"2023-11-15T03:02:20.719624+0000","last_fullsized":"2023-11-15T03:02:20.719624+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.723312+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T03:02:20.723312+0000","last_peered":"2023-11-15T03:02:20.723312+0000","last_clean":"2023-11-15T03:02:20.723312+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T03:02:20.723312+0000","last_undegraded":"2023-11-15T03:02:20.723312+0000","last_fullsized":"2023-11-15T03:02:20.723312+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.751486+0000","last_change":"2023-11-15T03:02:20.748293+0000","last_active":"2023-11-15T03:02:20.751486+0000","last_peered":"2023-11-15T03:02:20.751486+0000","last_clean":"2023-11-15T03:02:20.751486+0000","last_became_active":"2023-11-15T03:02:20.748008+0000","last_became_peered":"2023-11-15T03:02:20.748008+0000","last_unstale":"2023-11-15T03:02:20.751486+0000","last_undegraded":"2023-11-15T03:02:20.751486+0000","last_fullsized":"2023-11-15T03:02:20.751486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.719594+0000","last_change":"2023-11-15T03:02:20.283130+0000","last_active":"2023-11-15T03:02:20.719594+0000","last_peered":"2023-11-15T03:02:20.719594+0000","last_clean":"2023-11-15T03:02:20.719594+0000","last_became_active":"2023-11-15T03:02:20.282849+0000","last_became_peered":"2023-11-15T03:02:20.282849+0000","last_unstale":"2023-11-15T03:02:20.719594+0000","last_undegraded":"2023-11-15T03:02:20.719594+0000","last_fullsized":"2023-11-15T03:02:20.719594+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":58,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.724729+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T03:02:20.724729+0000","last_peered":"2023-11-15T03:02:20.724729+0000","last_clean":"2023-11-15T03:02:20.724729+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T03:02:20.724729+0000","last_undegraded":"2023-11-15T03:02:20.724729+0000","last_fullsized":"2023-11-15T03:02:20.724729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.747020+0000","last_change":"2023-11-15T03:02:20.739863+0000","last_active":"2023-11-15T03:02:20.747020+0000","last_peered":"2023-11-15T03:02:20.747020+0000","last_clean":"2023-11-15T03:02:20.747020+0000","last_became_active":"2023-11-15T03:02:20.739574+0000","last_became_peered":"2023-11-15T03:02:20.739574+0000","last_unstale":"2023-11-15T03:02:20.747020+0000","last_undegraded":"2023-11-15T03:02:20.747020+0000","last_fullsized":"2023-11-15T03:02:20.747020+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.721133+0000","last_change":"2023-11-15T03:02:20.278159+0000","last_active":"2023-11-15T03:02:20.721133+0000","last_peered":"2023-11-15T03:02:20.721133+0000","last_clean":"2023-11-15T03:02:20.721133+0000","last_became_active":"2023-11-15T03:02:20.277707+0000","last_became_peered":"2023-11-15T03:02:20.277707+0000","last_unstale":"2023-11-15T03:02:20.721133+0000","last_undegraded":"2023-11-15T03:02:20.721133+0000","last_fullsized":"2023-11-15T03:02:20.721133+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":69,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.725650+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T03:02:20.725650+0000","last_peered":"2023-11-15T03:02:20.725650+0000","last_clean":"2023-11-15T03:02:20.725650+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T03:02:20.725650+0000","last_undegraded":"2023-11-15T03:02:20.725650+0000","last_fullsized":"2023-11-15T03:02:20.725650+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.729942+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T03:02:20.729942+0000","last_peered":"2023-11-15T03:02:20.729942+0000","last_clean":"2023-11-15T03:02:20.729942+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T03:02:20.729942+0000","last_undegraded":"2023-11-15T03:02:20.729942+0000","last_fullsized":"2023-11-15T03:02:20.729942+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542214,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298016,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073824,"statfs":{"total":96636764160,"available":96331595776,"internally_reserved":0,"allocated":942080,"data_stored":428221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.181,"15min":1.1910000000000001},"min":{"1min":0.72399999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3360000000000001,"5min":10.961,"15min":10.961},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.141,"15min":1.1890000000000001},"min":{"1min":0.876,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.331,"5min":3.4910000000000001,"15min":3.4910000000000001},"last":1.101}]},{"osd":1,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.161,"15min":1.153},"min":{"1min":0.83099999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.4219999999999999,"5min":5.4930000000000003,"15min":5.4930000000000003},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2050000000000001,"15min":1.1879999999999999},"min":{"1min":0.79400000000000004,"5min":0.747,"15min":0.747},"max":{"1min":1.4830000000000001,"5min":6.8239999999999998,"15min":6.8239999999999998},"last":1.343}]},{"osd":2,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.1679999999999999,"15min":1.218},"min":{"1min":0.76300000000000001,"5min":0.751,"15min":0.751},"max":{"1min":1.49,"5min":5.7000000000000002,"15min":5.7000000000000002},"last":1.51},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2430000000000001,"15min":1.2889999999999999},"min":{"1min":0.93600000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.597,"5min":2.75,"15min":2.75},"last":1.119}]},{"osd":3,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.335,"15min":1.3720000000000001},"min":{"1min":1.014,"5min":0.751,"15min":0.751},"max":{"1min":1.6220000000000001,"5min":12.907,"15min":12.907},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.3160000000000001,"15min":1.298},"min":{"1min":0.995,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6679999999999999,"5min":9.9909999999999997,"15min":9.9909999999999997},"last":1.214}]},{"osd":4,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.139,"15min":1.2569999999999999},"min":{"1min":0.86599999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.3360000000000001,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":1.004},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2230000000000001,"15min":1.4239999999999999},"min":{"1min":0.85399999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5640000000000001,"5min":4.226,"15min":4.226},"last":1.1559999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.91000000000000003,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.5580000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.298,"15min":1.427},"min":{"1min":0.96899999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.609,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.8859999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.403,"15min":1.548},"min":{"1min":1.115,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.619,"5min":3.2789999999999999,"15min":3.2789999999999999},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.359,"15min":1.5669999999999999},"min":{"1min":0.93400000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6719999999999999,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":1.75}]}]},{"osd":6,"up_from":14,"seq":60129542214,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298020,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073820,"statfs":{"total":96636764160,"available":96331591680,"internally_reserved":0,"allocated":946176,"data_stored":428359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.0980000000000001,"15min":1.1240000000000001},"min":{"1min":0.80700000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.74,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.121,"5min":1.073,"15min":1.05},"min":{"1min":0.874,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.802,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.1739999999999999}]},{"osd":1,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.125,"15min":1.095},"min":{"1min":0.86899999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.512,"5min":1.623,"15min":1.623},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.123,"5min":1.1439999999999999,"15min":1.1599999999999999},"min":{"1min":0.83899999999999997,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.407,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.0329999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2030000000000001,"15min":1.1830000000000001},"min":{"1min":0.96499999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.7190000000000001,"5min":2.621,"15min":2.621},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.1399999999999999,"15min":1.099},"min":{"1min":0.93899999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4690000000000001,"5min":1.5,"15min":1.5},"last":0.98599999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2070000000000001,"15min":1.1699999999999999},"min":{"1min":1.0469999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.452,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":0.877},{"interface":"front","average":{"1min":1.373,"5min":1.286,"15min":1.272},"min":{"1min":0.96199999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7569999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.212}]},{"osd":4,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.097,"15min":1.1659999999999999},"min":{"1min":0.63500000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.3879999999999999,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.181,"15min":1.1850000000000001},"min":{"1min":0.91300000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.831,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.2789999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4279999999999999,"15min":1.4139999999999999},"min":{"1min":1.1919999999999999,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.621,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.329},{"interface":"front","average":{"1min":1.175,"5min":1.262,"15min":1.302},"min":{"1min":0.83599999999999997,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.492,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":1.143}]},{"osd":7,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2170000000000001,"15min":1.28},"min":{"1min":0.76200000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4570000000000001,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.071},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.4219999999999999,"15min":1.4470000000000001},"min":{"1min":0.98099999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.681,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":1.3779999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542215,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298020,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073820,"statfs":{"total":96636764160,"available":96331591680,"internally_reserved":0,"allocated":946176,"data_stored":428323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0940000000000001,"15min":1.1080000000000001},"min":{"1min":0.68200000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.355,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.175},{"interface":"front","average":{"1min":1.125,"5min":1.077,"15min":1.026},"min":{"1min":0.85799999999999998,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.3819999999999999,"5min":1.677,"15min":1.677},"last":0.85599999999999998}]},{"osd":1,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.1479999999999999,"15min":1.212},"min":{"1min":0.83999999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3089999999999999,"5min":2.617,"15min":2.617},"last":1.264},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1639999999999999,"15min":1.1659999999999999},"min":{"1min":0.88100000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.431,"5min":1.657,"15min":1.657},"last":1.0629999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.204,"15min":1.1579999999999999},"min":{"1min":0.98899999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4730000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.171,"5min":1.1519999999999999,"15min":1.161},"min":{"1min":0.82899999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.4590000000000001,"5min":1.667,"15min":1.667},"last":1.1359999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.141,"15min":1.1559999999999999},"min":{"1min":0.91100000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.401,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.276,"15min":1.294},"min":{"1min":1.04,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.4319999999999999,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":1}]},{"osd":5,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1819999999999999,"15min":1.232},"min":{"1min":0.752,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.4399999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.105,"5min":1.194,"15min":1.224},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.5149999999999999,"5min":1.994,"15min":1.994},"last":1.401}]},{"osd":6,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.28,"15min":1.325},"min":{"1min":0.877,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.494,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.341,"15min":1.3959999999999999},"min":{"1min":0.96999999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.52,"5min":2.173,"15min":2.173},"last":1.224}]},{"osd":7,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3160000000000001,"15min":1.3640000000000001},"min":{"1min":0.96499999999999997,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.486,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":1.046},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3819999999999999,"15min":1.4279999999999999},"min":{"1min":0.96499999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.512,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":1.3720000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542215,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298008,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073832,"statfs":{"total":96636764160,"available":96331603968,"internally_reserved":0,"allocated":933888,"data_stored":427645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.81999999999999995,"15min":0.77400000000000002},"min":{"1min":0.68300000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.02,"5min":1.169,"15min":1.169},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.87,"15min":0.83799999999999997},"min":{"1min":0.64000000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.244,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.71699999999999997}]},{"osd":1,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89100000000000001,"15min":0.89800000000000002},"min":{"1min":0.72899999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.163,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.006,"15min":0.97799999999999998},"min":{"1min":0.89600000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2549999999999999,"5min":1.375,"15min":1.375},"last":0.60099999999999998}]},{"osd":2,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.012,"15min":0.97999999999999998},"min":{"1min":0.75600000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4259999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.016},{"interface":"front","average":{"1min":1.099,"5min":1.0920000000000001,"15min":1.0609999999999999},"min":{"1min":0.83099999999999996,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3009999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.109}]},{"osd":4,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2310000000000001,"15min":1.1950000000000001},"min":{"1min":1.089,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.542,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.127},{"interface":"front","average":{"1min":1.21,"5min":1.206,"15min":1.222},"min":{"1min":1.0049999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.4370000000000001,"5min":1.659,"15min":1.659},"last":1.056}]},{"osd":5,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":1.0269999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.4790000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.288,"5min":1.2689999999999999,"15min":1.2649999999999999},"min":{"1min":1.0509999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.512,"5min":1.829,"15min":1.829},"last":1.0840000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3340000000000001,"15min":1.3129999999999999},"min":{"1min":1.143,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":1.5620000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.246},{"interface":"front","average":{"1min":1.383,"5min":1.3779999999999999,"15min":1.359},"min":{"1min":1.1399999999999999,"5min":1.004,"15min":1.004},"max":{"1min":1.581,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.96099999999999997}]},{"osd":7,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.357,"15min":1.3779999999999999},"min":{"1min":1.173,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.5800000000000001,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.3640000000000001,"15min":1.357},"min":{"1min":1.1339999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"max":{"1min":1.514,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.3620000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542215,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298016,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073824,"statfs":{"total":96636764160,"available":96331595776,"internally_reserved":0,"allocated":942080,"data_stored":427912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78100000000000003,"15min":0.76900000000000002},"min":{"1min":0.63700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.1479999999999999,"5min":1.409,"15min":1.409},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.878,"5min":0.85299999999999998,"15min":0.871},"min":{"1min":0.70199999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.339,"5min":1.619,"15min":1.619},"last":0.86099999999999999}]},{"osd":1,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.92400000000000004,"15min":0.90200000000000002},"min":{"1min":0.73399999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.014,"5min":0.97499999999999998,"15min":0.95299999999999996},"min":{"1min":0.81699999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.238,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.73399999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.98999999999999999,"15min":0.996},"min":{"1min":0.626,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.347,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.2,"5min":1.1379999999999999,"15min":1.1240000000000001},"min":{"1min":0.92600000000000005,"5min":0.622,"15min":0.622},"max":{"1min":1.5580000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.006}]},{"osd":4,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.218,"15min":1.2130000000000001},"min":{"1min":1.133,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6719999999999999,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.2270000000000001,"15min":1.226},"min":{"1min":1.03,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5029999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.266}]},{"osd":5,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2070000000000001,"15min":1.246},"min":{"1min":0.93999999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.554,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.276,"15min":1.29},"min":{"1min":1.0780000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.702,"5min":1.857,"15min":1.857},"last":1.2010000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.288,"15min":1.347},"min":{"1min":1.0109999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.679,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.2789999999999999,"15min":1.331},"min":{"1min":0.92400000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.524,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.252}]},{"osd":7,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3460000000000001,"15min":1.375},"min":{"1min":1.081,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.821,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3069999999999999,"15min":1.3680000000000001},"min":{"1min":1.1140000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.635,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.1759999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542215,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298008,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073832,"statfs":{"total":96636764160,"available":96331603968,"internally_reserved":0,"allocated":933888,"data_stored":427645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.85699999999999998,"15min":0.871},"min":{"1min":0.61499999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4390000000000001,"5min":4.3630000000000004,"15min":4.3630000000000004},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.97399999999999998,"15min":0.92200000000000004},"min":{"1min":0.67600000000000005,"5min":0.441,"15min":0.441},"max":{"1min":1.3520000000000001,"5min":6.157,"15min":6.157},"last":1.2450000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0109999999999999,"15min":1.0189999999999999},"min":{"1min":0.76800000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.571,"5min":4.1699999999999999,"15min":4.1699999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.046,"15min":1.006},"min":{"1min":0.75600000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4099999999999999,"5min":6.8369999999999997,"15min":6.8369999999999997},"last":1.482}]},{"osd":3,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.97599999999999998,"15min":0.96599999999999997},"min":{"1min":0.69599999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7330000000000001,"5min":3.5070000000000001,"15min":3.5070000000000001},"last":1.7050000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.143,"15min":1.1699999999999999},"min":{"1min":0.90400000000000003,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.55,"5min":5.2149999999999999,"15min":5.2149999999999999},"last":0.995}]},{"osd":4,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.2949999999999999,"15min":1.2629999999999999},"min":{"1min":1.0369999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6479999999999999,"5min":6.2000000000000002,"15min":6.2000000000000002},"last":1.54},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2270000000000001,"15min":1.1930000000000001},"min":{"1min":0.874,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.927,"5min":3.577,"15min":3.577},"last":1.357}]},{"osd":5,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.262,"15min":1.2330000000000001},"min":{"1min":0.97499999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9850000000000001,"5min":3.548,"15min":3.548},"last":1.8839999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.3220000000000001,"15min":1.3520000000000001},"min":{"1min":1.0840000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.6899999999999999,"5min":4.4080000000000004,"15min":4.4080000000000004},"last":1.456}]},{"osd":6,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.365,"15min":1.4139999999999999},"min":{"1min":1.1910000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.528,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3660000000000001,"15min":1.3540000000000001},"min":{"1min":1.0369999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9970000000000001,"5min":6.9690000000000003,"15min":6.9690000000000003},"last":1.585}]},{"osd":7,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.506,"15min":1.4259999999999999},"min":{"1min":0.94599999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.9390000000000001,"5min":18.253,"15min":18.253},"last":1.633},{"interface":"front","average":{"1min":1.343,"5min":1.4099999999999999,"15min":1.3100000000000001},"min":{"1min":1.129,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8280000000000001,"5min":11.006,"15min":11.006},"last":1.792}]}]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298012,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073828,"statfs":{"total":96636764160,"available":96331599872,"internally_reserved":0,"allocated":937984,"data_stored":427679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.081,"15min":1.056},"min":{"1min":0.81299999999999994,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4690000000000001,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.012,"5min":1.091,"15min":1.1950000000000001},"min":{"1min":0.86799999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3360000000000001,"5min":3.46,"15min":3.46},"last":0.78400000000000003}]},{"osd":1,"last update":"Wed Nov 15 03:01:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.127,"15min":1.224},"min":{"1min":0.75800000000000001,"5min":0.626,"15min":0.626},"max":{"1min":1.3460000000000001,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1499999999999999,"15min":1.1799999999999999},"min":{"1min":0.85399999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.407,"5min":3.516,"15min":3.516},"last":1.1080000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.181,"15min":1.2050000000000001},"min":{"1min":0.84799999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.7370000000000001,"5min":5.4180000000000001,"15min":5.4180000000000001},"last":0.876},{"interface":"front","average":{"1min":1.234,"5min":1.218,"15min":1.2909999999999999},"min":{"1min":0.877,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.746,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":0.83699999999999997}]},{"osd":3,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1990000000000001,"15min":1.2769999999999999},"min":{"1min":0.90600000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.5049999999999999,"5min":2.141,"15min":2.141},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.339,"15min":1.427},"min":{"1min":1.0069999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.786,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":1.3600000000000001}]},{"osd":4,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2749999999999999,"15min":1.357},"min":{"1min":0.86599999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.554,"5min":3.706,"15min":3.706},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3380000000000001,"15min":1.552},"min":{"1min":0.84599999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5920000000000001,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":1.1659999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.262,"15min":1.3200000000000001},"min":{"1min":0.94299999999999995,"5min":0.749,"15min":0.749},"max":{"1min":1.5189999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":1.417},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.365,"15min":1.526},"min":{"1min":1.038,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.6659999999999999,"5min":3.641,"15min":3.641},"last":1.45}]},{"osd":7,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3109999999999999,"15min":1.399},"min":{"1min":0.995,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.635,"5min":3.516,"15min":3.516},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.417,"5min":1.4079999999999999,"15min":1.448},"min":{"1min":0.94899999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.7909999999999999,"5min":3.754,"15min":3.754},"last":1.4850000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298024,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073816,"statfs":{"total":96636764160,"available":96331587584,"internally_reserved":0,"allocated":950272,"data_stored":428719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.82399999999999995,"15min":0.83799999999999997},"min":{"1min":0.63700000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.103,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.89400000000000002,"15min":0.88200000000000001},"min":{"1min":0.63,"5min":0.495,"15min":0.495},"max":{"1min":1.1950000000000001,"5min":1.419,"15min":1.419},"last":1.123}]},{"osd":2,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.91400000000000003,"15min":0.86599999999999999},"min":{"1min":0.70999999999999996,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94299999999999995,"15min":0.92200000000000004},"min":{"1min":0.66100000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.653,"5min":1.887,"15min":1.887},"last":0.99199999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.137,"15min":1.079},"min":{"1min":0.66600000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0629999999999999,"15min":1.0009999999999999},"min":{"1min":0.80800000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.444,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.92300000000000004}]},{"osd":4,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.1850000000000001,"15min":1.169},"min":{"1min":0.94299999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.341,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.2},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2370000000000001,"15min":1.2549999999999999},"min":{"1min":1.024,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.522,"5min":2.206,"15min":2.206},"last":1.254}]},{"osd":5,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2410000000000001,"15min":1.2589999999999999},"min":{"1min":1.0329999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.5049999999999999,"5min":1.899,"15min":1.899},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2669999999999999,"15min":1.2849999999999999},"min":{"1min":0.99399999999999999,"5min":0.871,"15min":0.871},"max":{"1min":1.4039999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.329}]},{"osd":6,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.335},"min":{"1min":1.127,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.726,"5min":1.948,"15min":1.948},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.401,"15min":1.409},"min":{"1min":1.054,"5min":0.87,"15min":0.87},"max":{"1min":1.637,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.3979999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3400000000000001,"15min":1.3240000000000001},"min":{"1min":1.0800000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.659,"5min":2.161,"15min":2.161},"last":1.427},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3520000000000001,"15min":1.359},"min":{"1min":1.0700000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.573,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.4810000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T03:02:29.991 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T03:02:30.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.146+0000 7f4f309f9700 1 Processor -- start 2023-11-15T03:02:30.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.146+0000 7f4f309f9700 1 -- start start 2023-11-15T03:02:30.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.150+0000 7f4f309f9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4f2c0cabd0 0x7f4f2c0cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:30.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.150+0000 7f4f309f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f2c0ad0f0 0x7f4f2c0ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:30.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.150+0000 7f4f309f9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f2c0ad650 con 0x7f4f2c0acc90 2023-11-15T03:02:30.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.150+0000 7f4f309f9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f2c065fc0 con 0x7f4f2c0cabd0 2023-11-15T03:02:30.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.150+0000 7f4f309f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c0662a0 con 0x7f4f2c0ad0f0 2023-11-15T03:02:30.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.150+0000 7f4f2b7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4f2c0cabd0 0x7f4f2c0cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:30.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.150+0000 7f4f2b7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4f2c0cabd0 0x7f4f2c0cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40478/0 (socket says 172.21.15.121:40478) 2023-11-15T03:02:30.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.150+0000 7f4f2b7fe700 1 -- 172.21.15.121:0/3621887255 learned_addr learned my addr 172.21.15.121:0/3621887255 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:30.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.150+0000 7f4f2bfff700 1 --2- 172.21.15.121:0/3621887255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f2c0ad0f0 0x7f4f2c0ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:30.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.150+0000 7f4f2b7fe700 1 -- 172.21.15.121:0/3621887255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f2c0ad0f0 msgr2=0x7f4f2c0ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:30.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.150+0000 7f4f2b7fe700 1 --2- 172.21.15.121:0/3621887255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f2c0ad0f0 0x7f4f2c0ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:30.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.150+0000 7f4f2b7fe700 1 -- 172.21.15.121:0/3621887255 >> v1:172.21.15.121:6789/0 conn(0x7f4f2c0acc90 legacy=0x7f4f2c066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:30.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.150+0000 7f4f2b7fe700 1 -- 172.21.15.121:0/3621887255 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2c066580 con 0x7f4f2c0cabd0 2023-11-15T03:02:30.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.154+0000 7f4f2b7fe700 1 --2- 172.21.15.121:0/3621887255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f2c0cabd0 0x7f4f2c0cafb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f4f1c006fd0 tx=0x7f4f1c0090f0).ready entity=mon.0 client_cookie=9b570d07f2d74732 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:30.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.154+0000 7f4f2a7fc700 1 -- 172.21.15.121:0/3621887255 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c00b580 con 0x7f4f2c0cabd0 2023-11-15T03:02:30.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.154+0000 7f4f2a7fc700 1 -- 172.21.15.121:0/3621887255 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1c00b6e0 con 0x7f4f2c0cabd0 2023-11-15T03:02:30.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.154+0000 7f4f2a7fc700 1 -- 172.21.15.121:0/3621887255 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c00ba60 con 0x7f4f2c0cabd0 2023-11-15T03:02:30.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.154+0000 7f4f309f9700 1 -- 172.21.15.121:0/3621887255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f2c0cabd0 msgr2=0x7f4f2c0cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:30.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.154+0000 7f4f309f9700 1 --2- 172.21.15.121:0/3621887255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f2c0cabd0 0x7f4f2c0cafb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f4f1c006fd0 tx=0x7f4f1c0090f0).stop 2023-11-15T03:02:30.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.154+0000 7f4f309f9700 1 -- 172.21.15.121:0/3621887255 shutdown_connections 2023-11-15T03:02:30.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.154+0000 7f4f309f9700 1 --2- 172.21.15.121:0/3621887255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f2c0ad0f0 0x7f4f2c0ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:30.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.154+0000 7f4f309f9700 1 --2- 172.21.15.121:0/3621887255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f2c0cabd0 0x7f4f2c0cafb0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:30.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.154+0000 7f4f309f9700 1 -- 172.21.15.121:0/3621887255 >> 172.21.15.121:0/3621887255 conn(0x7f4f2c062190 msgr2=0x7f4f2c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:30.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.154+0000 7f4f309f9700 1 -- 172.21.15.121:0/3621887255 shutdown_connections 2023-11-15T03:02:30.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.154+0000 7f4f309f9700 1 -- 172.21.15.121:0/3621887255 wait complete. 2023-11-15T03:02:30.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.158+0000 7f4f309f9700 1 Processor -- start 2023-11-15T03:02:30.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.158+0000 7f4f309f9700 1 -- start start 2023-11-15T03:02:30.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.158+0000 7f4f309f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f2c0acc90 0x7f4f2c14be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:30.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.158+0000 7f4f309f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f2c0ad0f0 0x7f4f2c14a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:30.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.158+0000 7f4f309f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f2c1493d0 0x7f4f2c146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:30.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.158+0000 7f4f309f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c14ba10 con 0x7f4f2c0ad0f0 2023-11-15T03:02:30.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.158+0000 7f4f309f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c14ad90 con 0x7f4f2c0acc90 2023-11-15T03:02:30.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.158+0000 7f4f309f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c14b070 con 0x7f4f2c1493d0 2023-11-15T03:02:30.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.158+0000 7f4f2affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f2c0ad0f0 0x7f4f2c14a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:30.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.158+0000 7f4f2affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f2c0ad0f0 0x7f4f2c14a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40488/0 (socket says 172.21.15.121:40488) 2023-11-15T03:02:30.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.158+0000 7f4f2affd700 1 -- 172.21.15.121:0/4201983243 learned_addr learned my addr 172.21.15.121:0/4201983243 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:30.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.162+0000 7f4f2bfff700 1 --2- 172.21.15.121:0/4201983243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f2c1493d0 0x7f4f2c146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:30.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.162+0000 7f4f2b7fe700 1 --2- 172.21.15.121:0/4201983243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f2c0acc90 0x7f4f2c14be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:30.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.162+0000 7f4f2affd700 1 -- 172.21.15.121:0/4201983243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f2c1493d0 msgr2=0x7f4f2c146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:30.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.162+0000 7f4f2affd700 1 --2- 172.21.15.121:0/4201983243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f2c1493d0 0x7f4f2c146f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:30.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.162+0000 7f4f2affd700 1 -- 172.21.15.121:0/4201983243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f2c0acc90 msgr2=0x7f4f2c14be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:30.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.162+0000 7f4f2affd700 1 --2- 172.21.15.121:0/4201983243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f2c0acc90 0x7f4f2c14be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:30.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.162+0000 7f4f2affd700 1 -- 172.21.15.121:0/4201983243 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c009370 con 0x7f4f2c0ad0f0 2023-11-15T03:02:30.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.162+0000 7f4f2affd700 1 --2- 172.21.15.121:0/4201983243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f2c0ad0f0 0x7f4f2c14a4b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7f4f2000c4c0 tx=0x7f4f20007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:30.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.162+0000 7f4f137fe700 1 -- 172.21.15.121:0/4201983243 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2000ce70 con 0x7f4f2c0ad0f0 2023-11-15T03:02:30.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.162+0000 7f4f137fe700 1 -- 172.21.15.121:0/4201983243 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f20001e90 con 0x7f4f2c0ad0f0 2023-11-15T03:02:30.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.162+0000 7f4f309f9700 1 -- 172.21.15.121:0/4201983243 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2c14b350 con 0x7f4f2c0ad0f0 2023-11-15T03:02:30.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.162+0000 7f4f137fe700 1 -- 172.21.15.121:0/4201983243 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f20007250 con 0x7f4f2c0ad0f0 2023-11-15T03:02:30.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.162+0000 7f4f309f9700 1 -- 172.21.15.121:0/4201983243 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f2c14c320 con 0x7f4f2c0ad0f0 2023-11-15T03:02:30.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.162+0000 7f4f309f9700 1 -- 172.21.15.121:0/4201983243 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f2c14bb50 con 0x7f4f2c0ad0f0 2023-11-15T03:02:30.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.166+0000 7f4f137fe700 1 -- 172.21.15.121:0/4201983243 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4f200066a0 con 0x7f4f2c0ad0f0 2023-11-15T03:02:30.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.166+0000 7f4f137fe700 1 --2- 172.21.15.121:0/4201983243 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4f1805a510 0x7f4f1805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:30.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.166+0000 7f4f137fe700 1 -- 172.21.15.121:0/4201983243 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4f20074190 con 0x7f4f2c0ad0f0 2023-11-15T03:02:30.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.166+0000 7f4f2b7fe700 1 --2- 172.21.15.121:0/4201983243 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4f1805a510 0x7f4f1805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:30.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.166+0000 7f4f2b7fe700 1 --2- 172.21.15.121:0/4201983243 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4f1805a510 0x7f4f1805c9d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f4f2c0c8600 tx=0x7f4f1c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:30.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.170+0000 7f4f137fe700 1 -- 172.21.15.121:0/4201983243 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4f2c14bb50 con 0x7f4f2c0ad0f0 2023-11-15T03:02:30.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.334+0000 7f4f309f9700 1 -- 172.21.15.121:0/4201983243 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4f2c14b7d0 con 0x7f4f1805a510 2023-11-15T03:02:30.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.354+0000 7f4f137fe700 1 -- 172.21.15.121:0/4201983243 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+358236 (secure 0 0 0) 0x7f4f2c14b7d0 con 0x7f4f1805a510 2023-11-15T03:02:30.364 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:30.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.358+0000 7f4f309f9700 1 -- 172.21.15.121:0/4201983243 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4f1805a510 msgr2=0x7f4f1805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:30.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.358+0000 7f4f309f9700 1 --2- 172.21.15.121:0/4201983243 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4f1805a510 0x7f4f1805c9d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f4f2c0c8600 tx=0x7f4f1c017000).stop 2023-11-15T03:02:30.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.358+0000 7f4f309f9700 1 -- 172.21.15.121:0/4201983243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f2c0ad0f0 msgr2=0x7f4f2c14a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:30.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.358+0000 7f4f309f9700 1 --2- 172.21.15.121:0/4201983243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f2c0ad0f0 0x7f4f2c14a4b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7f4f2000c4c0 tx=0x7f4f20007e00).stop 2023-11-15T03:02:30.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.358+0000 7f4f309f9700 1 -- 172.21.15.121:0/4201983243 shutdown_connections 2023-11-15T03:02:30.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.358+0000 7f4f309f9700 1 --2- 172.21.15.121:0/4201983243 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4f1805a510 0x7f4f1805c9d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:30.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.358+0000 7f4f309f9700 1 --2- 172.21.15.121:0/4201983243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f2c1493d0 0x7f4f2c146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:30.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.358+0000 7f4f309f9700 1 --2- 172.21.15.121:0/4201983243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f2c0ad0f0 0x7f4f2c14a4b0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:30.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.358+0000 7f4f309f9700 1 --2- 172.21.15.121:0/4201983243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f2c0acc90 0x7f4f2c14be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:30.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.362+0000 7f4f309f9700 1 -- 172.21.15.121:0/4201983243 >> 172.21.15.121:0/4201983243 conn(0x7f4f2c062190 msgr2=0x7f4f2c0ae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:30.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.362+0000 7f4f309f9700 1 -- 172.21.15.121:0/4201983243 shutdown_connections 2023-11-15T03:02:30.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.362+0000 7f4f309f9700 1 -- 172.21.15.121:0/4201983243 wait complete. 2023-11-15T03:02:30.371 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T03:02:30.389 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2023-11-15T03:02:30.042900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2384124,"kb_used_data":7356,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752590596,"statfs":{"total":773094113280,"available":770652770304,"internally_reserved":0,"allocated":7532544,"data_stored":3424503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25849,"num_objects":-8,"num_object_clones":0,"num_object_copies":-16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8,"num_whiteouts":0,"num_read":-119,"num_read_kb":-132,"num_write":-83,"num_write_kb":-53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.509126"},"pg_stats":[{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.750047+0000","last_change":"2023-11-15T03:02:20.737262+0000","last_active":"2023-11-15T03:02:20.750047+0000","last_peered":"2023-11-15T03:02:20.750047+0000","last_clean":"2023-11-15T03:02:20.750047+0000","last_became_active":"2023-11-15T03:02:20.735691+0000","last_became_peered":"2023-11-15T03:02:20.735691+0000","last_unstale":"2023-11-15T03:02:20.750047+0000","last_undegraded":"2023-11-15T03:02:20.750047+0000","last_fullsized":"2023-11-15T03:02:20.750047+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3f","version":"76'3","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.753765+0000","last_change":"2023-11-15T03:02:20.291047+0000","last_active":"2023-11-15T03:02:20.753765+0000","last_peered":"2023-11-15T03:02:20.753765+0000","last_clean":"2023-11-15T03:02:20.753765+0000","last_became_active":"2023-11-15T03:02:20.290541+0000","last_became_peered":"2023-11-15T03:02:20.290541+0000","last_unstale":"2023-11-15T03:02:20.753765+0000","last_undegraded":"2023-11-15T03:02:20.753765+0000","last_fullsized":"2023-11-15T03:02:20.753765+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.758807+0000","last_change":"2023-11-15T03:02:20.756957+0000","last_active":"2023-11-15T03:02:20.758807+0000","last_peered":"2023-11-15T03:02:20.758807+0000","last_clean":"2023-11-15T03:02:20.758807+0000","last_became_active":"2023-11-15T03:02:20.756690+0000","last_became_peered":"2023-11-15T03:02:20.756690+0000","last_unstale":"2023-11-15T03:02:20.758807+0000","last_undegraded":"2023-11-15T03:02:20.758807+0000","last_fullsized":"2023-11-15T03:02:20.758807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.728997+0000","last_change":"2023-11-15T03:02:20.292484+0000","last_active":"2023-11-15T03:02:20.728997+0000","last_peered":"2023-11-15T03:02:20.728997+0000","last_clean":"2023-11-15T03:02:20.728997+0000","last_became_active":"2023-11-15T03:02:20.292028+0000","last_became_peered":"2023-11-15T03:02:20.292028+0000","last_unstale":"2023-11-15T03:02:20.728997+0000","last_undegraded":"2023-11-15T03:02:20.728997+0000","last_fullsized":"2023-11-15T03:02:20.728997+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.743824+0000","last_change":"2023-11-15T03:02:20.738357+0000","last_active":"2023-11-15T03:02:20.743824+0000","last_peered":"2023-11-15T03:02:20.743824+0000","last_clean":"2023-11-15T03:02:20.743824+0000","last_became_active":"2023-11-15T03:02:20.737691+0000","last_became_peered":"2023-11-15T03:02:20.737691+0000","last_unstale":"2023-11-15T03:02:20.743824+0000","last_undegraded":"2023-11-15T03:02:20.743824+0000","last_fullsized":"2023-11-15T03:02:20.743824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.719561+0000","last_change":"2023-11-15T03:02:20.280633+0000","last_active":"2023-11-15T03:02:20.719561+0000","last_peered":"2023-11-15T03:02:20.719561+0000","last_clean":"2023-11-15T03:02:20.719561+0000","last_became_active":"2023-11-15T03:02:20.280353+0000","last_became_peered":"2023-11-15T03:02:20.280353+0000","last_unstale":"2023-11-15T03:02:20.719561+0000","last_undegraded":"2023-11-15T03:02:20.719561+0000","last_fullsized":"2023-11-15T03:02:20.719561+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.750369+0000","last_change":"2023-11-15T03:02:20.740898+0000","last_active":"2023-11-15T03:02:20.750369+0000","last_peered":"2023-11-15T03:02:20.750369+0000","last_clean":"2023-11-15T03:02:20.750369+0000","last_became_active":"2023-11-15T03:02:20.740611+0000","last_became_peered":"2023-11-15T03:02:20.740611+0000","last_unstale":"2023-11-15T03:02:20.750369+0000","last_undegraded":"2023-11-15T03:02:20.750369+0000","last_fullsized":"2023-11-15T03:02:20.750369+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3c","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745613+0000","last_change":"2023-11-15T03:02:20.282870+0000","last_active":"2023-11-15T03:02:20.745613+0000","last_peered":"2023-11-15T03:02:20.745613+0000","last_clean":"2023-11-15T03:02:20.745613+0000","last_became_active":"2023-11-15T03:02:20.282610+0000","last_became_peered":"2023-11-15T03:02:20.282610+0000","last_unstale":"2023-11-15T03:02:20.745613+0000","last_undegraded":"2023-11-15T03:02:20.745613+0000","last_fullsized":"2023-11-15T03:02:20.745613+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.749217+0000","last_change":"2023-11-15T03:02:20.744670+0000","last_active":"2023-11-15T03:02:20.749217+0000","last_peered":"2023-11-15T03:02:20.749217+0000","last_clean":"2023-11-15T03:02:20.749217+0000","last_became_active":"2023-11-15T03:02:20.743282+0000","last_became_peered":"2023-11-15T03:02:20.743282+0000","last_unstale":"2023-11-15T03:02:20.749217+0000","last_undegraded":"2023-11-15T03:02:20.749217+0000","last_fullsized":"2023-11-15T03:02:20.749217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3b","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.743864+0000","last_change":"2023-11-15T03:02:20.280906+0000","last_active":"2023-11-15T03:02:20.743864+0000","last_peered":"2023-11-15T03:02:20.743864+0000","last_clean":"2023-11-15T03:02:20.743864+0000","last_became_active":"2023-11-15T03:02:20.280418+0000","last_became_peered":"2023-11-15T03:02:20.280418+0000","last_unstale":"2023-11-15T03:02:20.743864+0000","last_undegraded":"2023-11-15T03:02:20.743864+0000","last_fullsized":"2023-11-15T03:02:20.743864+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745400+0000","last_change":"2023-11-15T03:02:20.730582+0000","last_active":"2023-11-15T03:02:20.745400+0000","last_peered":"2023-11-15T03:02:20.745400+0000","last_clean":"2023-11-15T03:02:20.745400+0000","last_became_active":"2023-11-15T03:02:20.730317+0000","last_became_peered":"2023-11-15T03:02:20.730317+0000","last_unstale":"2023-11-15T03:02:20.745400+0000","last_undegraded":"2023-11-15T03:02:20.745400+0000","last_fullsized":"2023-11-15T03:02:20.745400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3a","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.750200+0000","last_change":"2023-11-15T03:02:20.287727+0000","last_active":"2023-11-15T03:02:20.750200+0000","last_peered":"2023-11-15T03:02:20.750200+0000","last_clean":"2023-11-15T03:02:20.750200+0000","last_became_active":"2023-11-15T03:02:20.287442+0000","last_became_peered":"2023-11-15T03:02:20.287442+0000","last_unstale":"2023-11-15T03:02:20.750200+0000","last_undegraded":"2023-11-15T03:02:20.750200+0000","last_fullsized":"2023-11-15T03:02:20.750200+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745341+0000","last_change":"2023-11-15T03:02:20.740774+0000","last_active":"2023-11-15T03:02:20.745341+0000","last_peered":"2023-11-15T03:02:20.745341+0000","last_clean":"2023-11-15T03:02:20.745341+0000","last_became_active":"2023-11-15T03:02:20.740489+0000","last_became_peered":"2023-11-15T03:02:20.740489+0000","last_unstale":"2023-11-15T03:02:20.745341+0000","last_undegraded":"2023-11-15T03:02:20.745341+0000","last_fullsized":"2023-11-15T03:02:20.745341+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.727360+0000","last_change":"2023-11-15T03:02:20.274052+0000","last_active":"2023-11-15T03:02:20.727360+0000","last_peered":"2023-11-15T03:02:20.727360+0000","last_clean":"2023-11-15T03:02:20.727360+0000","last_became_active":"2023-11-15T03:02:20.273717+0000","last_became_peered":"2023-11-15T03:02:20.273717+0000","last_unstale":"2023-11-15T03:02:20.727360+0000","last_undegraded":"2023-11-15T03:02:20.727360+0000","last_fullsized":"2023-11-15T03:02:20.727360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.755857+0000","last_change":"2023-11-15T03:02:20.751318+0000","last_active":"2023-11-15T03:02:20.755857+0000","last_peered":"2023-11-15T03:02:20.755857+0000","last_clean":"2023-11-15T03:02:20.755857+0000","last_became_active":"2023-11-15T03:02:20.749439+0000","last_became_peered":"2023-11-15T03:02:20.749439+0000","last_unstale":"2023-11-15T03:02:20.755857+0000","last_undegraded":"2023-11-15T03:02:20.755857+0000","last_fullsized":"2023-11-15T03:02:20.755857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.728213+0000","last_change":"2023-11-15T03:02:20.280607+0000","last_active":"2023-11-15T03:02:20.728213+0000","last_peered":"2023-11-15T03:02:20.728213+0000","last_clean":"2023-11-15T03:02:20.728213+0000","last_became_active":"2023-11-15T03:02:20.280047+0000","last_became_peered":"2023-11-15T03:02:20.280047+0000","last_unstale":"2023-11-15T03:02:20.728213+0000","last_undegraded":"2023-11-15T03:02:20.728213+0000","last_fullsized":"2023-11-15T03:02:20.728213+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745749+0000","last_change":"2023-11-15T03:02:20.734411+0000","last_active":"2023-11-15T03:02:20.745749+0000","last_peered":"2023-11-15T03:02:20.745749+0000","last_clean":"2023-11-15T03:02:20.745749+0000","last_became_active":"2023-11-15T03:02:20.733875+0000","last_became_peered":"2023-11-15T03:02:20.733875+0000","last_unstale":"2023-11-15T03:02:20.745749+0000","last_undegraded":"2023-11-15T03:02:20.745749+0000","last_fullsized":"2023-11-15T03:02:20.745749+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.723328+0000","last_change":"2023-11-15T03:02:20.277343+0000","last_active":"2023-11-15T03:02:20.723328+0000","last_peered":"2023-11-15T03:02:20.723328+0000","last_clean":"2023-11-15T03:02:20.723328+0000","last_became_active":"2023-11-15T03:02:20.276105+0000","last_became_peered":"2023-11-15T03:02:20.276105+0000","last_unstale":"2023-11-15T03:02:20.723328+0000","last_undegraded":"2023-11-15T03:02:20.723328+0000","last_fullsized":"2023-11-15T03:02:20.723328+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.737783+0000","last_change":"2023-11-15T03:02:20.729124+0000","last_active":"2023-11-15T03:02:20.737783+0000","last_peered":"2023-11-15T03:02:20.737783+0000","last_clean":"2023-11-15T03:02:20.737783+0000","last_became_active":"2023-11-15T03:02:20.728781+0000","last_became_peered":"2023-11-15T03:02:20.728781+0000","last_unstale":"2023-11-15T03:02:20.737783+0000","last_undegraded":"2023-11-15T03:02:20.737783+0000","last_fullsized":"2023-11-15T03:02:20.737783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.722395+0000","last_change":"2023-11-15T03:02:20.288666+0000","last_active":"2023-11-15T03:02:20.722395+0000","last_peered":"2023-11-15T03:02:20.722395+0000","last_clean":"2023-11-15T03:02:20.722395+0000","last_became_active":"2023-11-15T03:02:20.288312+0000","last_became_peered":"2023-11-15T03:02:20.288312+0000","last_unstale":"2023-11-15T03:02:20.722395+0000","last_undegraded":"2023-11-15T03:02:20.722395+0000","last_fullsized":"2023-11-15T03:02:20.722395+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.743428+0000","last_change":"2023-11-15T03:02:20.737683+0000","last_active":"2023-11-15T03:02:20.743428+0000","last_peered":"2023-11-15T03:02:20.743428+0000","last_clean":"2023-11-15T03:02:20.743428+0000","last_became_active":"2023-11-15T03:02:20.737348+0000","last_became_peered":"2023-11-15T03:02:20.737348+0000","last_unstale":"2023-11-15T03:02:20.743428+0000","last_undegraded":"2023-11-15T03:02:20.743428+0000","last_fullsized":"2023-11-15T03:02:20.743428+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.729554+0000","last_change":"2023-11-15T03:02:20.282482+0000","last_active":"2023-11-15T03:02:20.729554+0000","last_peered":"2023-11-15T03:02:20.729554+0000","last_clean":"2023-11-15T03:02:20.729554+0000","last_became_active":"2023-11-15T03:02:20.281117+0000","last_became_peered":"2023-11-15T03:02:20.281117+0000","last_unstale":"2023-11-15T03:02:20.729554+0000","last_undegraded":"2023-11-15T03:02:20.729554+0000","last_fullsized":"2023-11-15T03:02:20.729554+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.749668+0000","last_change":"2023-11-15T03:02:20.734878+0000","last_active":"2023-11-15T03:02:20.749668+0000","last_peered":"2023-11-15T03:02:20.749668+0000","last_clean":"2023-11-15T03:02:20.749668+0000","last_became_active":"2023-11-15T03:02:20.734624+0000","last_became_peered":"2023-11-15T03:02:20.734624+0000","last_unstale":"2023-11-15T03:02:20.749668+0000","last_undegraded":"2023-11-15T03:02:20.749668+0000","last_fullsized":"2023-11-15T03:02:20.749668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.34","version":"76'4","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.754390+0000","last_change":"2023-11-15T03:02:20.280107+0000","last_active":"2023-11-15T03:02:20.754390+0000","last_peered":"2023-11-15T03:02:20.754390+0000","last_clean":"2023-11-15T03:02:20.754390+0000","last_became_active":"2023-11-15T03:02:20.279672+0000","last_became_peered":"2023-11-15T03:02:20.279672+0000","last_unstale":"2023-11-15T03:02:20.754390+0000","last_undegraded":"2023-11-15T03:02:20.754390+0000","last_fullsized":"2023-11-15T03:02:20.754390+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.749185+0000","last_change":"2023-11-15T03:02:20.744053+0000","last_active":"2023-11-15T03:02:20.749185+0000","last_peered":"2023-11-15T03:02:20.749185+0000","last_clean":"2023-11-15T03:02:20.749185+0000","last_became_active":"2023-11-15T03:02:20.742348+0000","last_became_peered":"2023-11-15T03:02:20.742348+0000","last_unstale":"2023-11-15T03:02:20.749185+0000","last_undegraded":"2023-11-15T03:02:20.749185+0000","last_fullsized":"2023-11-15T03:02:20.749185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.33","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.746597+0000","last_change":"2023-11-15T03:02:20.285168+0000","last_active":"2023-11-15T03:02:20.746597+0000","last_peered":"2023-11-15T03:02:20.746597+0000","last_clean":"2023-11-15T03:02:20.746597+0000","last_became_active":"2023-11-15T03:02:20.284913+0000","last_became_peered":"2023-11-15T03:02:20.284913+0000","last_unstale":"2023-11-15T03:02:20.746597+0000","last_undegraded":"2023-11-15T03:02:20.746597+0000","last_fullsized":"2023-11-15T03:02:20.746597+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.753410+0000","last_change":"2023-11-15T03:02:20.751877+0000","last_active":"2023-11-15T03:02:20.753410+0000","last_peered":"2023-11-15T03:02:20.753410+0000","last_clean":"2023-11-15T03:02:20.753410+0000","last_became_active":"2023-11-15T03:02:20.751590+0000","last_became_peered":"2023-11-15T03:02:20.751590+0000","last_unstale":"2023-11-15T03:02:20.753410+0000","last_undegraded":"2023-11-15T03:02:20.753410+0000","last_fullsized":"2023-11-15T03:02:20.753410+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.723105+0000","last_change":"2023-11-15T03:02:20.276550+0000","last_active":"2023-11-15T03:02:20.723105+0000","last_peered":"2023-11-15T03:02:20.723105+0000","last_clean":"2023-11-15T03:02:20.723105+0000","last_became_active":"2023-11-15T03:02:20.275732+0000","last_became_peered":"2023-11-15T03:02:20.275732+0000","last_unstale":"2023-11-15T03:02:20.723105+0000","last_undegraded":"2023-11-15T03:02:20.723105+0000","last_fullsized":"2023-11-15T03:02:20.723105+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.749154+0000","last_change":"2023-11-15T03:02:20.744718+0000","last_active":"2023-11-15T03:02:20.749154+0000","last_peered":"2023-11-15T03:02:20.749154+0000","last_clean":"2023-11-15T03:02:20.749154+0000","last_became_active":"2023-11-15T03:02:20.744096+0000","last_became_peered":"2023-11-15T03:02:20.744096+0000","last_unstale":"2023-11-15T03:02:20.749154+0000","last_undegraded":"2023-11-15T03:02:20.749154+0000","last_fullsized":"2023-11-15T03:02:20.749154+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.725954+0000","last_change":"2023-11-15T03:02:20.285069+0000","last_active":"2023-11-15T03:02:20.725954+0000","last_peered":"2023-11-15T03:02:20.725954+0000","last_clean":"2023-11-15T03:02:20.725954+0000","last_became_active":"2023-11-15T03:02:20.284784+0000","last_became_peered":"2023-11-15T03:02:20.284784+0000","last_unstale":"2023-11-15T03:02:20.725954+0000","last_undegraded":"2023-11-15T03:02:20.725954+0000","last_fullsized":"2023-11-15T03:02:20.725954+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.746498+0000","last_change":"2023-11-15T03:02:20.736058+0000","last_active":"2023-11-15T03:02:20.746498+0000","last_peered":"2023-11-15T03:02:20.746498+0000","last_clean":"2023-11-15T03:02:20.746498+0000","last_became_active":"2023-11-15T03:02:20.735765+0000","last_became_peered":"2023-11-15T03:02:20.735765+0000","last_unstale":"2023-11-15T03:02:20.746498+0000","last_undegraded":"2023-11-15T03:02:20.746498+0000","last_fullsized":"2023-11-15T03:02:20.746498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"76'4","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.756392+0000","last_change":"2023-11-15T03:02:20.284214+0000","last_active":"2023-11-15T03:02:20.756392+0000","last_peered":"2023-11-15T03:02:20.756392+0000","last_clean":"2023-11-15T03:02:20.756392+0000","last_became_active":"2023-11-15T03:02:20.283823+0000","last_became_peered":"2023-11-15T03:02:20.283823+0000","last_unstale":"2023-11-15T03:02:20.756392+0000","last_undegraded":"2023-11-15T03:02:20.756392+0000","last_fullsized":"2023-11-15T03:02:20.756392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.742010+0000","last_change":"2023-11-15T03:02:20.734950+0000","last_active":"2023-11-15T03:02:20.742010+0000","last_peered":"2023-11-15T03:02:20.742010+0000","last_clean":"2023-11-15T03:02:20.742010+0000","last_became_active":"2023-11-15T03:02:20.734605+0000","last_became_peered":"2023-11-15T03:02:20.734605+0000","last_unstale":"2023-11-15T03:02:20.742010+0000","last_undegraded":"2023-11-15T03:02:20.742010+0000","last_fullsized":"2023-11-15T03:02:20.742010+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.723429+0000","last_change":"2023-11-15T03:02:20.275502+0000","last_active":"2023-11-15T03:02:20.723429+0000","last_peered":"2023-11-15T03:02:20.723429+0000","last_clean":"2023-11-15T03:02:20.723429+0000","last_became_active":"2023-11-15T03:02:20.275217+0000","last_became_peered":"2023-11-15T03:02:20.275217+0000","last_unstale":"2023-11-15T03:02:20.723429+0000","last_undegraded":"2023-11-15T03:02:20.723429+0000","last_fullsized":"2023-11-15T03:02:20.723429+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.744291+0000","last_change":"2023-11-15T03:02:20.737075+0000","last_active":"2023-11-15T03:02:20.744291+0000","last_peered":"2023-11-15T03:02:20.744291+0000","last_clean":"2023-11-15T03:02:20.744291+0000","last_became_active":"2023-11-15T03:02:20.736702+0000","last_became_peered":"2023-11-15T03:02:20.736702+0000","last_unstale":"2023-11-15T03:02:20.744291+0000","last_undegraded":"2023-11-15T03:02:20.744291+0000","last_fullsized":"2023-11-15T03:02:20.744291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.714183+0000","last_change":"2023-11-15T03:02:20.282164+0000","last_active":"2023-11-15T03:02:20.714183+0000","last_peered":"2023-11-15T03:02:20.714183+0000","last_clean":"2023-11-15T03:02:20.714183+0000","last_became_active":"2023-11-15T03:02:20.281841+0000","last_became_peered":"2023-11-15T03:02:20.281841+0000","last_unstale":"2023-11-15T03:02:20.714183+0000","last_undegraded":"2023-11-15T03:02:20.714183+0000","last_fullsized":"2023-11-15T03:02:20.714183+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.750388+0000","last_change":"2023-11-15T03:02:20.740936+0000","last_active":"2023-11-15T03:02:20.750388+0000","last_peered":"2023-11-15T03:02:20.750388+0000","last_clean":"2023-11-15T03:02:20.750388+0000","last_became_active":"2023-11-15T03:02:20.740697+0000","last_became_peered":"2023-11-15T03:02:20.740697+0000","last_unstale":"2023-11-15T03:02:20.750388+0000","last_undegraded":"2023-11-15T03:02:20.750388+0000","last_fullsized":"2023-11-15T03:02:20.750388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2d","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.751975+0000","last_change":"2023-11-15T03:02:20.278505+0000","last_active":"2023-11-15T03:02:20.751975+0000","last_peered":"2023-11-15T03:02:20.751975+0000","last_clean":"2023-11-15T03:02:20.751975+0000","last_became_active":"2023-11-15T03:02:20.278225+0000","last_became_peered":"2023-11-15T03:02:20.278225+0000","last_unstale":"2023-11-15T03:02:20.751975+0000","last_undegraded":"2023-11-15T03:02:20.751975+0000","last_fullsized":"2023-11-15T03:02:20.751975+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.738650+0000","last_change":"2023-11-15T03:02:20.726564+0000","last_active":"2023-11-15T03:02:20.738650+0000","last_peered":"2023-11-15T03:02:20.738650+0000","last_clean":"2023-11-15T03:02:20.738650+0000","last_became_active":"2023-11-15T03:02:20.726104+0000","last_became_peered":"2023-11-15T03:02:20.726104+0000","last_unstale":"2023-11-15T03:02:20.738650+0000","last_undegraded":"2023-11-15T03:02:20.738650+0000","last_fullsized":"2023-11-15T03:02:20.738650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.727457+0000","last_change":"2023-11-15T03:02:20.276336+0000","last_active":"2023-11-15T03:02:20.727457+0000","last_peered":"2023-11-15T03:02:20.727457+0000","last_clean":"2023-11-15T03:02:20.727457+0000","last_became_active":"2023-11-15T03:02:20.276000+0000","last_became_peered":"2023-11-15T03:02:20.276000+0000","last_unstale":"2023-11-15T03:02:20.727457+0000","last_undegraded":"2023-11-15T03:02:20.727457+0000","last_fullsized":"2023-11-15T03:02:20.727457+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.752513+0000","last_change":"2023-11-15T03:02:20.743490+0000","last_active":"2023-11-15T03:02:20.752513+0000","last_peered":"2023-11-15T03:02:20.752513+0000","last_clean":"2023-11-15T03:02:20.752513+0000","last_became_active":"2023-11-15T03:02:20.743192+0000","last_became_peered":"2023-11-15T03:02:20.743192+0000","last_unstale":"2023-11-15T03:02:20.752513+0000","last_undegraded":"2023-11-15T03:02:20.752513+0000","last_fullsized":"2023-11-15T03:02:20.752513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2b","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.751383+0000","last_change":"2023-11-15T03:02:20.293538+0000","last_active":"2023-11-15T03:02:20.751383+0000","last_peered":"2023-11-15T03:02:20.751383+0000","last_clean":"2023-11-15T03:02:20.751383+0000","last_became_active":"2023-11-15T03:02:20.290652+0000","last_became_peered":"2023-11-15T03:02:20.290652+0000","last_unstale":"2023-11-15T03:02:20.751383+0000","last_undegraded":"2023-11-15T03:02:20.751383+0000","last_fullsized":"2023-11-15T03:02:20.751383+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.743404+0000","last_change":"2023-11-15T03:02:20.737535+0000","last_active":"2023-11-15T03:02:20.743404+0000","last_peered":"2023-11-15T03:02:20.743404+0000","last_clean":"2023-11-15T03:02:20.743404+0000","last_became_active":"2023-11-15T03:02:20.736675+0000","last_became_peered":"2023-11-15T03:02:20.736675+0000","last_unstale":"2023-11-15T03:02:20.743404+0000","last_undegraded":"2023-11-15T03:02:20.743404+0000","last_fullsized":"2023-11-15T03:02:20.743404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.724426+0000","last_change":"2023-11-15T03:02:20.292132+0000","last_active":"2023-11-15T03:02:20.724426+0000","last_peered":"2023-11-15T03:02:20.724426+0000","last_clean":"2023-11-15T03:02:20.724426+0000","last_became_active":"2023-11-15T03:02:20.288549+0000","last_became_peered":"2023-11-15T03:02:20.288549+0000","last_unstale":"2023-11-15T03:02:20.724426+0000","last_undegraded":"2023-11-15T03:02:20.724426+0000","last_fullsized":"2023-11-15T03:02:20.724426+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.748655+0000","last_change":"2023-11-15T03:02:20.740354+0000","last_active":"2023-11-15T03:02:20.748655+0000","last_peered":"2023-11-15T03:02:20.748655+0000","last_clean":"2023-11-15T03:02:20.748655+0000","last_became_active":"2023-11-15T03:02:20.739907+0000","last_became_peered":"2023-11-15T03:02:20.739907+0000","last_unstale":"2023-11-15T03:02:20.748655+0000","last_undegraded":"2023-11-15T03:02:20.748655+0000","last_fullsized":"2023-11-15T03:02:20.748655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.730651+0000","last_change":"2023-11-15T03:02:20.276701+0000","last_active":"2023-11-15T03:02:20.730651+0000","last_peered":"2023-11-15T03:02:20.730651+0000","last_clean":"2023-11-15T03:02:20.730651+0000","last_became_active":"2023-11-15T03:02:20.275656+0000","last_became_peered":"2023-11-15T03:02:20.275656+0000","last_unstale":"2023-11-15T03:02:20.730651+0000","last_undegraded":"2023-11-15T03:02:20.730651+0000","last_fullsized":"2023-11-15T03:02:20.730651+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.753097+0000","last_change":"2023-11-15T03:02:20.746260+0000","last_active":"2023-11-15T03:02:20.753097+0000","last_peered":"2023-11-15T03:02:20.753097+0000","last_clean":"2023-11-15T03:02:20.753097+0000","last_became_active":"2023-11-15T03:02:20.744833+0000","last_became_peered":"2023-11-15T03:02:20.744833+0000","last_unstale":"2023-11-15T03:02:20.753097+0000","last_undegraded":"2023-11-15T03:02:20.753097+0000","last_fullsized":"2023-11-15T03:02:20.753097+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.724594+0000","last_change":"2023-11-15T03:02:20.286122+0000","last_active":"2023-11-15T03:02:20.724594+0000","last_peered":"2023-11-15T03:02:20.724594+0000","last_clean":"2023-11-15T03:02:20.724594+0000","last_became_active":"2023-11-15T03:02:20.285859+0000","last_became_peered":"2023-11-15T03:02:20.285859+0000","last_unstale":"2023-11-15T03:02:20.724594+0000","last_undegraded":"2023-11-15T03:02:20.724594+0000","last_fullsized":"2023-11-15T03:02:20.724594+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.753858+0000","last_change":"2023-11-15T03:02:20.747332+0000","last_active":"2023-11-15T03:02:20.753858+0000","last_peered":"2023-11-15T03:02:20.753858+0000","last_clean":"2023-11-15T03:02:20.753858+0000","last_became_active":"2023-11-15T03:02:20.745865+0000","last_became_peered":"2023-11-15T03:02:20.745865+0000","last_unstale":"2023-11-15T03:02:20.753858+0000","last_undegraded":"2023-11-15T03:02:20.753858+0000","last_fullsized":"2023-11-15T03:02:20.753858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.723996+0000","last_change":"2023-11-15T03:02:20.286821+0000","last_active":"2023-11-15T03:02:20.723996+0000","last_peered":"2023-11-15T03:02:20.723996+0000","last_clean":"2023-11-15T03:02:20.723996+0000","last_became_active":"2023-11-15T03:02:20.286544+0000","last_became_peered":"2023-11-15T03:02:20.286544+0000","last_unstale":"2023-11-15T03:02:20.723996+0000","last_undegraded":"2023-11-15T03:02:20.723996+0000","last_fullsized":"2023-11-15T03:02:20.723996+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.749278+0000","last_change":"2023-11-15T03:02:20.745921+0000","last_active":"2023-11-15T03:02:20.749278+0000","last_peered":"2023-11-15T03:02:20.749278+0000","last_clean":"2023-11-15T03:02:20.749278+0000","last_became_active":"2023-11-15T03:02:20.745541+0000","last_became_peered":"2023-11-15T03:02:20.745541+0000","last_unstale":"2023-11-15T03:02:20.749278+0000","last_undegraded":"2023-11-15T03:02:20.749278+0000","last_fullsized":"2023-11-15T03:02:20.749278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"77'1","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-15T03:02:21.723778+0000","last_change":"2023-11-15T03:02:20.282482+0000","last_active":"2023-11-15T03:02:21.723778+0000","last_peered":"2023-11-15T03:02:21.723778+0000","last_clean":"2023-11-15T03:02:21.723778+0000","last_became_active":"2023-11-15T03:02:20.280516+0000","last_became_peered":"2023-11-15T03:02:20.280516+0000","last_unstale":"2023-11-15T03:02:21.723778+0000","last_undegraded":"2023-11-15T03:02:21.723778+0000","last_fullsized":"2023-11-15T03:02:21.723778+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.752870+0000","last_change":"2023-11-15T03:02:20.746541+0000","last_active":"2023-11-15T03:02:20.752870+0000","last_peered":"2023-11-15T03:02:20.752870+0000","last_clean":"2023-11-15T03:02:20.752870+0000","last_became_active":"2023-11-15T03:02:20.746114+0000","last_became_peered":"2023-11-15T03:02:20.746114+0000","last_unstale":"2023-11-15T03:02:20.752870+0000","last_undegraded":"2023-11-15T03:02:20.752870+0000","last_fullsized":"2023-11-15T03:02:20.752870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.719708+0000","last_change":"2023-11-15T03:02:20.274474+0000","last_active":"2023-11-15T03:02:20.719708+0000","last_peered":"2023-11-15T03:02:20.719708+0000","last_clean":"2023-11-15T03:02:20.719708+0000","last_became_active":"2023-11-15T03:02:20.274126+0000","last_became_peered":"2023-11-15T03:02:20.274126+0000","last_unstale":"2023-11-15T03:02:20.719708+0000","last_undegraded":"2023-11-15T03:02:20.719708+0000","last_fullsized":"2023-11-15T03:02:20.719708+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.743138+0000","last_change":"2023-11-15T03:02:20.739107+0000","last_active":"2023-11-15T03:02:20.743138+0000","last_peered":"2023-11-15T03:02:20.743138+0000","last_clean":"2023-11-15T03:02:20.743138+0000","last_became_active":"2023-11-15T03:02:20.738715+0000","last_became_peered":"2023-11-15T03:02:20.738715+0000","last_unstale":"2023-11-15T03:02:20.743138+0000","last_undegraded":"2023-11-15T03:02:20.743138+0000","last_fullsized":"2023-11-15T03:02:20.743138+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.721653+0000","last_change":"2023-11-15T03:02:20.288868+0000","last_active":"2023-11-15T03:02:20.721653+0000","last_peered":"2023-11-15T03:02:20.721653+0000","last_clean":"2023-11-15T03:02:20.721653+0000","last_became_active":"2023-11-15T03:02:20.288475+0000","last_became_peered":"2023-11-15T03:02:20.288475+0000","last_unstale":"2023-11-15T03:02:20.721653+0000","last_undegraded":"2023-11-15T03:02:20.721653+0000","last_fullsized":"2023-11-15T03:02:20.721653+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.753207+0000","last_change":"2023-11-15T03:02:20.744976+0000","last_active":"2023-11-15T03:02:20.753207+0000","last_peered":"2023-11-15T03:02:20.753207+0000","last_clean":"2023-11-15T03:02:20.753207+0000","last_became_active":"2023-11-15T03:02:20.743529+0000","last_became_peered":"2023-11-15T03:02:20.743529+0000","last_unstale":"2023-11-15T03:02:20.753207+0000","last_undegraded":"2023-11-15T03:02:20.753207+0000","last_fullsized":"2023-11-15T03:02:20.753207+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.720047+0000","last_change":"2023-11-15T03:02:20.273095+0000","last_active":"2023-11-15T03:02:20.720047+0000","last_peered":"2023-11-15T03:02:20.720047+0000","last_clean":"2023-11-15T03:02:20.720047+0000","last_became_active":"2023-11-15T03:02:20.272789+0000","last_became_peered":"2023-11-15T03:02:20.272789+0000","last_unstale":"2023-11-15T03:02:20.720047+0000","last_undegraded":"2023-11-15T03:02:20.720047+0000","last_fullsized":"2023-11-15T03:02:20.720047+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.747000+0000","last_change":"2023-11-15T03:02:20.739196+0000","last_active":"2023-11-15T03:02:20.747000+0000","last_peered":"2023-11-15T03:02:20.747000+0000","last_clean":"2023-11-15T03:02:20.747000+0000","last_became_active":"2023-11-15T03:02:20.738539+0000","last_became_peered":"2023-11-15T03:02:20.738539+0000","last_unstale":"2023-11-15T03:02:20.747000+0000","last_undegraded":"2023-11-15T03:02:20.747000+0000","last_fullsized":"2023-11-15T03:02:20.747000+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.726540+0000","last_change":"2023-11-15T03:02:20.279761+0000","last_active":"2023-11-15T03:02:20.726540+0000","last_peered":"2023-11-15T03:02:20.726540+0000","last_clean":"2023-11-15T03:02:20.726540+0000","last_became_active":"2023-11-15T03:02:20.279488+0000","last_became_peered":"2023-11-15T03:02:20.279488+0000","last_unstale":"2023-11-15T03:02:20.726540+0000","last_undegraded":"2023-11-15T03:02:20.726540+0000","last_fullsized":"2023-11-15T03:02:20.726540+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.750750+0000","last_change":"2023-11-15T03:02:20.749026+0000","last_active":"2023-11-15T03:02:20.750750+0000","last_peered":"2023-11-15T03:02:20.750750+0000","last_clean":"2023-11-15T03:02:20.750750+0000","last_became_active":"2023-11-15T03:02:20.748607+0000","last_became_peered":"2023-11-15T03:02:20.748607+0000","last_unstale":"2023-11-15T03:02:20.750750+0000","last_undegraded":"2023-11-15T03:02:20.750750+0000","last_fullsized":"2023-11-15T03:02:20.750750+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.727533+0000","last_change":"2023-11-15T03:02:20.281300+0000","last_active":"2023-11-15T03:02:20.727533+0000","last_peered":"2023-11-15T03:02:20.727533+0000","last_clean":"2023-11-15T03:02:20.727533+0000","last_became_active":"2023-11-15T03:02:20.280992+0000","last_became_peered":"2023-11-15T03:02:20.280992+0000","last_unstale":"2023-11-15T03:02:20.727533+0000","last_undegraded":"2023-11-15T03:02:20.727533+0000","last_fullsized":"2023-11-15T03:02:20.727533+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.741421+0000","last_change":"2023-11-15T03:02:20.735782+0000","last_active":"2023-11-15T03:02:20.741421+0000","last_peered":"2023-11-15T03:02:20.741421+0000","last_clean":"2023-11-15T03:02:20.741421+0000","last_became_active":"2023-11-15T03:02:20.735491+0000","last_became_peered":"2023-11-15T03:02:20.735491+0000","last_unstale":"2023-11-15T03:02:20.741421+0000","last_undegraded":"2023-11-15T03:02:20.741421+0000","last_fullsized":"2023-11-15T03:02:20.741421+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.728298+0000","last_change":"2023-11-15T03:02:20.280351+0000","last_active":"2023-11-15T03:02:20.728298+0000","last_peered":"2023-11-15T03:02:20.728298+0000","last_clean":"2023-11-15T03:02:20.728298+0000","last_became_active":"2023-11-15T03:02:20.279706+0000","last_became_peered":"2023-11-15T03:02:20.279706+0000","last_unstale":"2023-11-15T03:02:20.728298+0000","last_undegraded":"2023-11-15T03:02:20.728298+0000","last_fullsized":"2023-11-15T03:02:20.728298+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.747882+0000","last_change":"2023-11-15T03:02:20.736251+0000","last_active":"2023-11-15T03:02:20.747882+0000","last_peered":"2023-11-15T03:02:20.747882+0000","last_clean":"2023-11-15T03:02:20.747882+0000","last_became_active":"2023-11-15T03:02:20.735947+0000","last_became_peered":"2023-11-15T03:02:20.735947+0000","last_unstale":"2023-11-15T03:02:20.747882+0000","last_undegraded":"2023-11-15T03:02:20.747882+0000","last_fullsized":"2023-11-15T03:02:20.747882+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.741640+0000","last_change":"2023-11-15T03:02:20.275451+0000","last_active":"2023-11-15T03:02:20.741640+0000","last_peered":"2023-11-15T03:02:20.741640+0000","last_clean":"2023-11-15T03:02:20.741640+0000","last_became_active":"2023-11-15T03:02:20.275177+0000","last_became_peered":"2023-11-15T03:02:20.275177+0000","last_unstale":"2023-11-15T03:02:20.741640+0000","last_undegraded":"2023-11-15T03:02:20.741640+0000","last_fullsized":"2023-11-15T03:02:20.741640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.740852+0000","last_change":"2023-11-15T03:02:20.734278+0000","last_active":"2023-11-15T03:02:20.740852+0000","last_peered":"2023-11-15T03:02:20.740852+0000","last_clean":"2023-11-15T03:02:20.740852+0000","last_became_active":"2023-11-15T03:02:20.734000+0000","last_became_peered":"2023-11-15T03:02:20.734000+0000","last_unstale":"2023-11-15T03:02:20.740852+0000","last_undegraded":"2023-11-15T03:02:20.740852+0000","last_fullsized":"2023-11-15T03:02:20.740852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.752283+0000","last_change":"2023-11-15T03:02:20.286904+0000","last_active":"2023-11-15T03:02:20.752283+0000","last_peered":"2023-11-15T03:02:20.752283+0000","last_clean":"2023-11-15T03:02:20.752283+0000","last_became_active":"2023-11-15T03:02:20.286652+0000","last_became_peered":"2023-11-15T03:02:20.286652+0000","last_unstale":"2023-11-15T03:02:20.752283+0000","last_undegraded":"2023-11-15T03:02:20.752283+0000","last_fullsized":"2023-11-15T03:02:20.752283+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.750510+0000","last_change":"2023-11-15T03:02:20.745856+0000","last_active":"2023-11-15T03:02:20.750510+0000","last_peered":"2023-11-15T03:02:20.750510+0000","last_clean":"2023-11-15T03:02:20.750510+0000","last_became_active":"2023-11-15T03:02:20.745318+0000","last_became_peered":"2023-11-15T03:02:20.745318+0000","last_unstale":"2023-11-15T03:02:20.750510+0000","last_undegraded":"2023-11-15T03:02:20.750510+0000","last_fullsized":"2023-11-15T03:02:20.750510+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.746199+0000","last_change":"2023-11-15T03:02:20.287943+0000","last_active":"2023-11-15T03:02:20.746199+0000","last_peered":"2023-11-15T03:02:20.746199+0000","last_clean":"2023-11-15T03:02:20.746199+0000","last_became_active":"2023-11-15T03:02:20.287644+0000","last_became_peered":"2023-11-15T03:02:20.287644+0000","last_unstale":"2023-11-15T03:02:20.746199+0000","last_undegraded":"2023-11-15T03:02:20.746199+0000","last_fullsized":"2023-11-15T03:02:20.746199+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.738630+0000","last_change":"2023-11-15T03:02:20.726378+0000","last_active":"2023-11-15T03:02:20.738630+0000","last_peered":"2023-11-15T03:02:20.738630+0000","last_clean":"2023-11-15T03:02:20.738630+0000","last_became_active":"2023-11-15T03:02:20.725792+0000","last_became_peered":"2023-11-15T03:02:20.725792+0000","last_unstale":"2023-11-15T03:02:20.738630+0000","last_undegraded":"2023-11-15T03:02:20.738630+0000","last_fullsized":"2023-11-15T03:02:20.738630+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.731116+0000","last_change":"2023-11-15T03:02:20.287833+0000","last_active":"2023-11-15T03:02:20.731116+0000","last_peered":"2023-11-15T03:02:20.731116+0000","last_clean":"2023-11-15T03:02:20.731116+0000","last_became_active":"2023-11-15T03:02:20.287294+0000","last_became_peered":"2023-11-15T03:02:20.287294+0000","last_unstale":"2023-11-15T03:02:20.731116+0000","last_undegraded":"2023-11-15T03:02:20.731116+0000","last_fullsized":"2023-11-15T03:02:20.731116+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.751643+0000","last_change":"2023-11-15T03:02:20.749149+0000","last_active":"2023-11-15T03:02:20.751643+0000","last_peered":"2023-11-15T03:02:20.751643+0000","last_clean":"2023-11-15T03:02:20.751643+0000","last_became_active":"2023-11-15T03:02:20.747720+0000","last_became_peered":"2023-11-15T03:02:20.747720+0000","last_unstale":"2023-11-15T03:02:20.751643+0000","last_undegraded":"2023-11-15T03:02:20.751643+0000","last_fullsized":"2023-11-15T03:02:20.751643+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.722622+0000","last_change":"2023-11-15T03:02:20.290914+0000","last_active":"2023-11-15T03:02:20.722622+0000","last_peered":"2023-11-15T03:02:20.722622+0000","last_clean":"2023-11-15T03:02:20.722622+0000","last_became_active":"2023-11-15T03:02:20.290227+0000","last_became_peered":"2023-11-15T03:02:20.290227+0000","last_unstale":"2023-11-15T03:02:20.722622+0000","last_undegraded":"2023-11-15T03:02:20.722622+0000","last_fullsized":"2023-11-15T03:02:20.722622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.751461+0000","last_change":"2023-11-15T03:02:20.748167+0000","last_active":"2023-11-15T03:02:20.751461+0000","last_peered":"2023-11-15T03:02:20.751461+0000","last_clean":"2023-11-15T03:02:20.751461+0000","last_became_active":"2023-11-15T03:02:20.747887+0000","last_became_peered":"2023-11-15T03:02:20.747887+0000","last_unstale":"2023-11-15T03:02:20.751461+0000","last_undegraded":"2023-11-15T03:02:20.751461+0000","last_fullsized":"2023-11-15T03:02:20.751461+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.726662+0000","last_change":"2023-11-15T03:02:20.272187+0000","last_active":"2023-11-15T03:02:20.726662+0000","last_peered":"2023-11-15T03:02:20.726662+0000","last_clean":"2023-11-15T03:02:20.726662+0000","last_became_active":"2023-11-15T03:02:20.271834+0000","last_became_peered":"2023-11-15T03:02:20.271834+0000","last_unstale":"2023-11-15T03:02:20.726662+0000","last_undegraded":"2023-11-15T03:02:20.726662+0000","last_fullsized":"2023-11-15T03:02:20.726662+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.744559+0000","last_change":"2023-11-15T03:02:20.739189+0000","last_active":"2023-11-15T03:02:20.744559+0000","last_peered":"2023-11-15T03:02:20.744559+0000","last_clean":"2023-11-15T03:02:20.744559+0000","last_became_active":"2023-11-15T03:02:20.738107+0000","last_became_peered":"2023-11-15T03:02:20.738107+0000","last_unstale":"2023-11-15T03:02:20.744559+0000","last_undegraded":"2023-11-15T03:02:20.744559+0000","last_fullsized":"2023-11-15T03:02:20.744559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.726041+0000","last_change":"2023-11-15T03:02:20.284564+0000","last_active":"2023-11-15T03:02:20.726041+0000","last_peered":"2023-11-15T03:02:20.726041+0000","last_clean":"2023-11-15T03:02:20.726041+0000","last_became_active":"2023-11-15T03:02:20.284292+0000","last_became_peered":"2023-11-15T03:02:20.284292+0000","last_unstale":"2023-11-15T03:02:20.726041+0000","last_undegraded":"2023-11-15T03:02:20.726041+0000","last_fullsized":"2023-11-15T03:02:20.726041+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.758163+0000","last_change":"2023-11-15T03:02:20.754792+0000","last_active":"2023-11-15T03:02:20.758163+0000","last_peered":"2023-11-15T03:02:20.758163+0000","last_clean":"2023-11-15T03:02:20.758163+0000","last_became_active":"2023-11-15T03:02:20.754442+0000","last_became_peered":"2023-11-15T03:02:20.754442+0000","last_unstale":"2023-11-15T03:02:20.758163+0000","last_undegraded":"2023-11-15T03:02:20.758163+0000","last_fullsized":"2023-11-15T03:02:20.758163+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.723436+0000","last_change":"2023-11-15T03:02:20.278296+0000","last_active":"2023-11-15T03:02:20.723436+0000","last_peered":"2023-11-15T03:02:20.723436+0000","last_clean":"2023-11-15T03:02:20.723436+0000","last_became_active":"2023-11-15T03:02:20.277760+0000","last_became_peered":"2023-11-15T03:02:20.277760+0000","last_unstale":"2023-11-15T03:02:20.723436+0000","last_undegraded":"2023-11-15T03:02:20.723436+0000","last_fullsized":"2023-11-15T03:02:20.723436+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745485+0000","last_change":"2023-11-15T03:02:20.741587+0000","last_active":"2023-11-15T03:02:20.745485+0000","last_peered":"2023-11-15T03:02:20.745485+0000","last_clean":"2023-11-15T03:02:20.745485+0000","last_became_active":"2023-11-15T03:02:20.739815+0000","last_became_peered":"2023-11-15T03:02:20.739815+0000","last_unstale":"2023-11-15T03:02:20.745485+0000","last_undegraded":"2023-11-15T03:02:20.745485+0000","last_fullsized":"2023-11-15T03:02:20.745485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.730598+0000","last_change":"2023-11-15T03:02:20.289756+0000","last_active":"2023-11-15T03:02:20.730598+0000","last_peered":"2023-11-15T03:02:20.730598+0000","last_clean":"2023-11-15T03:02:20.730598+0000","last_became_active":"2023-11-15T03:02:20.289374+0000","last_became_peered":"2023-11-15T03:02:20.289374+0000","last_unstale":"2023-11-15T03:02:20.730598+0000","last_undegraded":"2023-11-15T03:02:20.730598+0000","last_fullsized":"2023-11-15T03:02:20.730598+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745148+0000","last_change":"2023-11-15T03:02:20.739098+0000","last_active":"2023-11-15T03:02:20.745148+0000","last_peered":"2023-11-15T03:02:20.745148+0000","last_clean":"2023-11-15T03:02:20.745148+0000","last_became_active":"2023-11-15T03:02:20.738546+0000","last_became_peered":"2023-11-15T03:02:20.738546+0000","last_unstale":"2023-11-15T03:02:20.745148+0000","last_undegraded":"2023-11-15T03:02:20.745148+0000","last_fullsized":"2023-11-15T03:02:20.745148+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.728101+0000","last_change":"2023-11-15T03:02:20.278051+0000","last_active":"2023-11-15T03:02:20.728101+0000","last_peered":"2023-11-15T03:02:20.728101+0000","last_clean":"2023-11-15T03:02:20.728101+0000","last_became_active":"2023-11-15T03:02:20.277758+0000","last_became_peered":"2023-11-15T03:02:20.277758+0000","last_unstale":"2023-11-15T03:02:20.728101+0000","last_undegraded":"2023-11-15T03:02:20.728101+0000","last_fullsized":"2023-11-15T03:02:20.728101+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.737566+0000","last_change":"2023-11-15T03:02:20.725085+0000","last_active":"2023-11-15T03:02:20.737566+0000","last_peered":"2023-11-15T03:02:20.737566+0000","last_clean":"2023-11-15T03:02:20.737566+0000","last_became_active":"2023-11-15T03:02:20.724827+0000","last_became_peered":"2023-11-15T03:02:20.724827+0000","last_unstale":"2023-11-15T03:02:20.737566+0000","last_undegraded":"2023-11-15T03:02:20.737566+0000","last_fullsized":"2023-11-15T03:02:20.737566+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"76'3","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.751205+0000","last_change":"2023-11-15T03:02:20.272400+0000","last_active":"2023-11-15T03:02:20.751205+0000","last_peered":"2023-11-15T03:02:20.751205+0000","last_clean":"2023-11-15T03:02:20.751205+0000","last_became_active":"2023-11-15T03:02:20.272107+0000","last_became_peered":"2023-11-15T03:02:20.272107+0000","last_unstale":"2023-11-15T03:02:20.751205+0000","last_undegraded":"2023-11-15T03:02:20.751205+0000","last_fullsized":"2023-11-15T03:02:20.751205+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.748847+0000","last_change":"2023-11-15T03:02:20.740969+0000","last_active":"2023-11-15T03:02:20.748847+0000","last_peered":"2023-11-15T03:02:20.748847+0000","last_clean":"2023-11-15T03:02:20.748847+0000","last_became_active":"2023-11-15T03:02:20.740724+0000","last_became_peered":"2023-11-15T03:02:20.740724+0000","last_unstale":"2023-11-15T03:02:20.748847+0000","last_undegraded":"2023-11-15T03:02:20.748847+0000","last_fullsized":"2023-11-15T03:02:20.748847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.751582+0000","last_change":"2023-11-15T03:02:20.291581+0000","last_active":"2023-11-15T03:02:20.751582+0000","last_peered":"2023-11-15T03:02:20.751582+0000","last_clean":"2023-11-15T03:02:20.751582+0000","last_became_active":"2023-11-15T03:02:20.291122+0000","last_became_peered":"2023-11-15T03:02:20.291122+0000","last_unstale":"2023-11-15T03:02:20.751582+0000","last_undegraded":"2023-11-15T03:02:20.751582+0000","last_fullsized":"2023-11-15T03:02:20.751582+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.749203+0000","last_change":"2023-11-15T03:02:20.744457+0000","last_active":"2023-11-15T03:02:20.749203+0000","last_peered":"2023-11-15T03:02:20.749203+0000","last_clean":"2023-11-15T03:02:20.749203+0000","last_became_active":"2023-11-15T03:02:20.743512+0000","last_became_peered":"2023-11-15T03:02:20.743512+0000","last_unstale":"2023-11-15T03:02:20.749203+0000","last_undegraded":"2023-11-15T03:02:20.749203+0000","last_fullsized":"2023-11-15T03:02:20.749203+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.748201+0000","last_change":"2023-11-15T03:02:20.285924+0000","last_active":"2023-11-15T03:02:20.748201+0000","last_peered":"2023-11-15T03:02:20.748201+0000","last_clean":"2023-11-15T03:02:20.748201+0000","last_became_active":"2023-11-15T03:02:20.285590+0000","last_became_peered":"2023-11-15T03:02:20.285590+0000","last_unstale":"2023-11-15T03:02:20.748201+0000","last_undegraded":"2023-11-15T03:02:20.748201+0000","last_fullsized":"2023-11-15T03:02:20.748201+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.756505+0000","last_change":"2023-11-15T03:02:20.752529+0000","last_active":"2023-11-15T03:02:20.756505+0000","last_peered":"2023-11-15T03:02:20.756505+0000","last_clean":"2023-11-15T03:02:20.756505+0000","last_became_active":"2023-11-15T03:02:20.752153+0000","last_became_peered":"2023-11-15T03:02:20.752153+0000","last_unstale":"2023-11-15T03:02:20.756505+0000","last_undegraded":"2023-11-15T03:02:20.756505+0000","last_fullsized":"2023-11-15T03:02:20.756505+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.729140+0000","last_change":"2023-11-15T03:02:20.290468+0000","last_active":"2023-11-15T03:02:20.729140+0000","last_peered":"2023-11-15T03:02:20.729140+0000","last_clean":"2023-11-15T03:02:20.729140+0000","last_became_active":"2023-11-15T03:02:20.290104+0000","last_became_peered":"2023-11-15T03:02:20.290104+0000","last_unstale":"2023-11-15T03:02:20.729140+0000","last_undegraded":"2023-11-15T03:02:20.729140+0000","last_fullsized":"2023-11-15T03:02:20.729140+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.753109+0000","last_change":"2023-11-15T03:02:20.748616+0000","last_active":"2023-11-15T03:02:20.753109+0000","last_peered":"2023-11-15T03:02:20.753109+0000","last_clean":"2023-11-15T03:02:20.753109+0000","last_became_active":"2023-11-15T03:02:20.748246+0000","last_became_peered":"2023-11-15T03:02:20.748246+0000","last_unstale":"2023-11-15T03:02:20.753109+0000","last_undegraded":"2023-11-15T03:02:20.753109+0000","last_fullsized":"2023-11-15T03:02:20.753109+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.723347+0000","last_change":"2023-11-15T03:02:20.276925+0000","last_active":"2023-11-15T03:02:20.723347+0000","last_peered":"2023-11-15T03:02:20.723347+0000","last_clean":"2023-11-15T03:02:20.723347+0000","last_became_active":"2023-11-15T03:02:20.276660+0000","last_became_peered":"2023-11-15T03:02:20.276660+0000","last_unstale":"2023-11-15T03:02:20.723347+0000","last_undegraded":"2023-11-15T03:02:20.723347+0000","last_fullsized":"2023-11-15T03:02:20.723347+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.756108+0000","last_change":"2023-11-15T03:02:20.752836+0000","last_active":"2023-11-15T03:02:20.756108+0000","last_peered":"2023-11-15T03:02:20.756108+0000","last_clean":"2023-11-15T03:02:20.756108+0000","last_became_active":"2023-11-15T03:02:20.752532+0000","last_became_peered":"2023-11-15T03:02:20.752532+0000","last_unstale":"2023-11-15T03:02:20.756108+0000","last_undegraded":"2023-11-15T03:02:20.756108+0000","last_fullsized":"2023-11-15T03:02:20.756108+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"76'4","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.748653+0000","last_change":"2023-11-15T03:02:20.283810+0000","last_active":"2023-11-15T03:02:20.748653+0000","last_peered":"2023-11-15T03:02:20.748653+0000","last_clean":"2023-11-15T03:02:20.748653+0000","last_became_active":"2023-11-15T03:02:20.283572+0000","last_became_peered":"2023-11-15T03:02:20.283572+0000","last_unstale":"2023-11-15T03:02:20.748653+0000","last_undegraded":"2023-11-15T03:02:20.748653+0000","last_fullsized":"2023-11-15T03:02:20.748653+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.747796+0000","last_change":"2023-11-15T03:02:20.742850+0000","last_active":"2023-11-15T03:02:20.747796+0000","last_peered":"2023-11-15T03:02:20.747796+0000","last_clean":"2023-11-15T03:02:20.747796+0000","last_became_active":"2023-11-15T03:02:20.738256+0000","last_became_peered":"2023-11-15T03:02:20.738256+0000","last_unstale":"2023-11-15T03:02:20.747796+0000","last_undegraded":"2023-11-15T03:02:20.747796+0000","last_fullsized":"2023-11-15T03:02:20.747796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.749942+0000","last_change":"2023-11-15T03:02:20.274513+0000","last_active":"2023-11-15T03:02:20.749942+0000","last_peered":"2023-11-15T03:02:20.749942+0000","last_clean":"2023-11-15T03:02:20.749942+0000","last_became_active":"2023-11-15T03:02:20.274169+0000","last_became_peered":"2023-11-15T03:02:20.274169+0000","last_unstale":"2023-11-15T03:02:20.749942+0000","last_undegraded":"2023-11-15T03:02:20.749942+0000","last_fullsized":"2023-11-15T03:02:20.749942+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.747110+0000","last_change":"2023-11-15T03:02:20.740694+0000","last_active":"2023-11-15T03:02:20.747110+0000","last_peered":"2023-11-15T03:02:20.747110+0000","last_clean":"2023-11-15T03:02:20.747110+0000","last_became_active":"2023-11-15T03:02:20.740377+0000","last_became_peered":"2023-11-15T03:02:20.740377+0000","last_unstale":"2023-11-15T03:02:20.747110+0000","last_undegraded":"2023-11-15T03:02:20.747110+0000","last_fullsized":"2023-11-15T03:02:20.747110+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.729268+0000","last_change":"2023-11-15T03:02:20.278754+0000","last_active":"2023-11-15T03:02:20.729268+0000","last_peered":"2023-11-15T03:02:20.729268+0000","last_clean":"2023-11-15T03:02:20.729268+0000","last_became_active":"2023-11-15T03:02:20.278460+0000","last_became_peered":"2023-11-15T03:02:20.278460+0000","last_unstale":"2023-11-15T03:02:20.729268+0000","last_undegraded":"2023-11-15T03:02:20.729268+0000","last_fullsized":"2023-11-15T03:02:20.729268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.750488+0000","last_change":"2023-11-15T03:02:20.745675+0000","last_active":"2023-11-15T03:02:20.750488+0000","last_peered":"2023-11-15T03:02:20.750488+0000","last_clean":"2023-11-15T03:02:20.750488+0000","last_became_active":"2023-11-15T03:02:20.745053+0000","last_became_peered":"2023-11-15T03:02:20.745053+0000","last_unstale":"2023-11-15T03:02:20.750488+0000","last_undegraded":"2023-11-15T03:02:20.750488+0000","last_fullsized":"2023-11-15T03:02:20.750488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.729824+0000","last_change":"2023-11-15T03:02:20.283476+0000","last_active":"2023-11-15T03:02:20.729824+0000","last_peered":"2023-11-15T03:02:20.729824+0000","last_clean":"2023-11-15T03:02:20.729824+0000","last_became_active":"2023-11-15T03:02:20.283183+0000","last_became_peered":"2023-11-15T03:02:20.283183+0000","last_unstale":"2023-11-15T03:02:20.729824+0000","last_undegraded":"2023-11-15T03:02:20.729824+0000","last_fullsized":"2023-11-15T03:02:20.729824+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.746980+0000","last_change":"2023-11-15T03:02:20.742607+0000","last_active":"2023-11-15T03:02:20.746980+0000","last_peered":"2023-11-15T03:02:20.746980+0000","last_clean":"2023-11-15T03:02:20.746980+0000","last_became_active":"2023-11-15T03:02:20.742287+0000","last_became_peered":"2023-11-15T03:02:20.742287+0000","last_unstale":"2023-11-15T03:02:20.746980+0000","last_undegraded":"2023-11-15T03:02:20.746980+0000","last_fullsized":"2023-11-15T03:02:20.746980+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.728808+0000","last_change":"2023-11-15T03:02:20.285244+0000","last_active":"2023-11-15T03:02:20.728808+0000","last_peered":"2023-11-15T03:02:20.728808+0000","last_clean":"2023-11-15T03:02:20.728808+0000","last_became_active":"2023-11-15T03:02:20.284983+0000","last_became_peered":"2023-11-15T03:02:20.284983+0000","last_unstale":"2023-11-15T03:02:20.728808+0000","last_undegraded":"2023-11-15T03:02:20.728808+0000","last_fullsized":"2023-11-15T03:02:20.728808+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.758135+0000","last_change":"2023-11-15T03:02:20.754340+0000","last_active":"2023-11-15T03:02:20.758135+0000","last_peered":"2023-11-15T03:02:20.758135+0000","last_clean":"2023-11-15T03:02:20.758135+0000","last_became_active":"2023-11-15T03:02:20.753930+0000","last_became_peered":"2023-11-15T03:02:20.753930+0000","last_unstale":"2023-11-15T03:02:20.758135+0000","last_undegraded":"2023-11-15T03:02:20.758135+0000","last_fullsized":"2023-11-15T03:02:20.758135+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.729857+0000","last_change":"2023-11-15T03:02:20.287975+0000","last_active":"2023-11-15T03:02:20.729857+0000","last_peered":"2023-11-15T03:02:20.729857+0000","last_clean":"2023-11-15T03:02:20.729857+0000","last_became_active":"2023-11-15T03:02:20.287650+0000","last_became_peered":"2023-11-15T03:02:20.287650+0000","last_unstale":"2023-11-15T03:02:20.729857+0000","last_undegraded":"2023-11-15T03:02:20.729857+0000","last_fullsized":"2023-11-15T03:02:20.729857+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.758779+0000","last_change":"2023-11-15T03:02:20.756575+0000","last_active":"2023-11-15T03:02:20.758779+0000","last_peered":"2023-11-15T03:02:20.758779+0000","last_clean":"2023-11-15T03:02:20.758779+0000","last_became_active":"2023-11-15T03:02:20.756255+0000","last_became_peered":"2023-11-15T03:02:20.756255+0000","last_unstale":"2023-11-15T03:02:20.758779+0000","last_undegraded":"2023-11-15T03:02:20.758779+0000","last_fullsized":"2023-11-15T03:02:20.758779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.723210+0000","last_change":"2023-11-15T03:02:20.277034+0000","last_active":"2023-11-15T03:02:20.723210+0000","last_peered":"2023-11-15T03:02:20.723210+0000","last_clean":"2023-11-15T03:02:20.723210+0000","last_became_active":"2023-11-15T03:02:20.276787+0000","last_became_peered":"2023-11-15T03:02:20.276787+0000","last_unstale":"2023-11-15T03:02:20.723210+0000","last_undegraded":"2023-11-15T03:02:20.723210+0000","last_fullsized":"2023-11-15T03:02:20.723210+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.745673+0000","last_change":"2023-11-15T03:02:20.730485+0000","last_active":"2023-11-15T03:02:20.745673+0000","last_peered":"2023-11-15T03:02:20.745673+0000","last_clean":"2023-11-15T03:02:20.745673+0000","last_became_active":"2023-11-15T03:02:20.730219+0000","last_became_peered":"2023-11-15T03:02:20.730219+0000","last_unstale":"2023-11-15T03:02:20.745673+0000","last_undegraded":"2023-11-15T03:02:20.745673+0000","last_fullsized":"2023-11-15T03:02:20.745673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.735339+0000","last_change":"2023-11-15T03:02:20.288684+0000","last_active":"2023-11-15T03:02:20.735339+0000","last_peered":"2023-11-15T03:02:20.735339+0000","last_clean":"2023-11-15T03:02:20.735339+0000","last_became_active":"2023-11-15T03:02:20.288348+0000","last_became_peered":"2023-11-15T03:02:20.288348+0000","last_unstale":"2023-11-15T03:02:20.735339+0000","last_undegraded":"2023-11-15T03:02:20.735339+0000","last_fullsized":"2023-11-15T03:02:20.735339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.748129+0000","last_change":"2023-11-15T03:02:20.737437+0000","last_active":"2023-11-15T03:02:20.748129+0000","last_peered":"2023-11-15T03:02:20.748129+0000","last_clean":"2023-11-15T03:02:20.748129+0000","last_became_active":"2023-11-15T03:02:20.737189+0000","last_became_peered":"2023-11-15T03:02:20.737189+0000","last_unstale":"2023-11-15T03:02:20.748129+0000","last_undegraded":"2023-11-15T03:02:20.748129+0000","last_fullsized":"2023-11-15T03:02:20.748129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.712554+0000","last_change":"2023-11-15T03:02:20.277917+0000","last_active":"2023-11-15T03:02:20.712554+0000","last_peered":"2023-11-15T03:02:20.712554+0000","last_clean":"2023-11-15T03:02:20.712554+0000","last_became_active":"2023-11-15T03:02:20.275072+0000","last_became_peered":"2023-11-15T03:02:20.275072+0000","last_unstale":"2023-11-15T03:02:20.712554+0000","last_undegraded":"2023-11-15T03:02:20.712554+0000","last_fullsized":"2023-11-15T03:02:20.712554+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.753312+0000","last_change":"2023-11-15T03:02:20.745019+0000","last_active":"2023-11-15T03:02:20.753312+0000","last_peered":"2023-11-15T03:02:20.753312+0000","last_clean":"2023-11-15T03:02:20.753312+0000","last_became_active":"2023-11-15T03:02:20.744563+0000","last_became_peered":"2023-11-15T03:02:20.744563+0000","last_unstale":"2023-11-15T03:02:20.753312+0000","last_undegraded":"2023-11-15T03:02:20.753312+0000","last_fullsized":"2023-11-15T03:02:20.753312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.752327+0000","last_change":"2023-11-15T03:02:20.289275+0000","last_active":"2023-11-15T03:02:20.752327+0000","last_peered":"2023-11-15T03:02:20.752327+0000","last_clean":"2023-11-15T03:02:20.752327+0000","last_became_active":"2023-11-15T03:02:20.288989+0000","last_became_peered":"2023-11-15T03:02:20.288989+0000","last_unstale":"2023-11-15T03:02:20.752327+0000","last_undegraded":"2023-11-15T03:02:20.752327+0000","last_fullsized":"2023-11-15T03:02:20.752327+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":58,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.734004+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T03:02:20.734004+0000","last_peered":"2023-11-15T03:02:20.734004+0000","last_clean":"2023-11-15T03:02:20.734004+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T03:02:20.734004+0000","last_undegraded":"2023-11-15T03:02:20.734004+0000","last_fullsized":"2023-11-15T03:02:20.734004+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.738673+0000","last_change":"2023-11-15T03:02:20.732039+0000","last_active":"2023-11-15T03:02:20.738673+0000","last_peered":"2023-11-15T03:02:20.738673+0000","last_clean":"2023-11-15T03:02:20.738673+0000","last_became_active":"2023-11-15T03:02:20.731756+0000","last_became_peered":"2023-11-15T03:02:20.731756+0000","last_unstale":"2023-11-15T03:02:20.738673+0000","last_undegraded":"2023-11-15T03:02:20.738673+0000","last_fullsized":"2023-11-15T03:02:20.738673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.752627+0000","last_change":"2023-11-15T03:02:20.290797+0000","last_active":"2023-11-15T03:02:20.752627+0000","last_peered":"2023-11-15T03:02:20.752627+0000","last_clean":"2023-11-15T03:02:20.752627+0000","last_became_active":"2023-11-15T03:02:20.290141+0000","last_became_peered":"2023-11-15T03:02:20.290141+0000","last_unstale":"2023-11-15T03:02:20.752627+0000","last_undegraded":"2023-11-15T03:02:20.752627+0000","last_fullsized":"2023-11-15T03:02:20.752627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.714328+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T03:02:20.714328+0000","last_peered":"2023-11-15T03:02:20.714328+0000","last_clean":"2023-11-15T03:02:20.714328+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T03:02:20.714328+0000","last_undegraded":"2023-11-15T03:02:20.714328+0000","last_fullsized":"2023-11-15T03:02:20.714328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.747006+0000","last_change":"2023-11-15T03:02:20.739333+0000","last_active":"2023-11-15T03:02:20.747006+0000","last_peered":"2023-11-15T03:02:20.747006+0000","last_clean":"2023-11-15T03:02:20.747006+0000","last_became_active":"2023-11-15T03:02:20.735473+0000","last_became_peered":"2023-11-15T03:02:20.735473+0000","last_unstale":"2023-11-15T03:02:20.747006+0000","last_undegraded":"2023-11-15T03:02:20.747006+0000","last_fullsized":"2023-11-15T03:02:20.747006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.712972+0000","last_change":"2023-11-15T03:02:20.279939+0000","last_active":"2023-11-15T03:02:20.712972+0000","last_peered":"2023-11-15T03:02:20.712972+0000","last_clean":"2023-11-15T03:02:20.712972+0000","last_became_active":"2023-11-15T03:02:20.279319+0000","last_became_peered":"2023-11-15T03:02:20.279319+0000","last_unstale":"2023-11-15T03:02:20.712972+0000","last_undegraded":"2023-11-15T03:02:20.712972+0000","last_fullsized":"2023-11-15T03:02:20.712972+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":58,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.729992+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T03:02:20.729992+0000","last_peered":"2023-11-15T03:02:20.729992+0000","last_clean":"2023-11-15T03:02:20.729992+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T03:02:20.729992+0000","last_undegraded":"2023-11-15T03:02:20.729992+0000","last_fullsized":"2023-11-15T03:02:20.729992+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.753051+0000","last_change":"2023-11-15T03:02:20.743899+0000","last_active":"2023-11-15T03:02:20.753051+0000","last_peered":"2023-11-15T03:02:20.753051+0000","last_clean":"2023-11-15T03:02:20.753051+0000","last_became_active":"2023-11-15T03:02:20.743634+0000","last_became_peered":"2023-11-15T03:02:20.743634+0000","last_unstale":"2023-11-15T03:02:20.753051+0000","last_undegraded":"2023-11-15T03:02:20.753051+0000","last_fullsized":"2023-11-15T03:02:20.753051+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.723281+0000","last_change":"2023-11-15T03:02:20.286870+0000","last_active":"2023-11-15T03:02:20.723281+0000","last_peered":"2023-11-15T03:02:20.723281+0000","last_clean":"2023-11-15T03:02:20.723281+0000","last_became_active":"2023-11-15T03:02:20.285961+0000","last_became_peered":"2023-11-15T03:02:20.285961+0000","last_unstale":"2023-11-15T03:02:20.723281+0000","last_undegraded":"2023-11-15T03:02:20.723281+0000","last_fullsized":"2023-11-15T03:02:20.723281+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.713011+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T03:02:20.713011+0000","last_peered":"2023-11-15T03:02:20.713011+0000","last_clean":"2023-11-15T03:02:20.713011+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T03:02:20.713011+0000","last_undegraded":"2023-11-15T03:02:20.713011+0000","last_fullsized":"2023-11-15T03:02:20.713011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.743036+0000","last_change":"2023-11-15T03:02:20.735712+0000","last_active":"2023-11-15T03:02:20.743036+0000","last_peered":"2023-11-15T03:02:20.743036+0000","last_clean":"2023-11-15T03:02:20.743036+0000","last_became_active":"2023-11-15T03:02:20.735426+0000","last_became_peered":"2023-11-15T03:02:20.735426+0000","last_unstale":"2023-11-15T03:02:20.743036+0000","last_undegraded":"2023-11-15T03:02:20.743036+0000","last_fullsized":"2023-11-15T03:02:20.743036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.747304+0000","last_change":"2023-11-15T03:02:20.268524+0000","last_active":"2023-11-15T03:02:20.747304+0000","last_peered":"2023-11-15T03:02:20.747304+0000","last_clean":"2023-11-15T03:02:20.747304+0000","last_became_active":"2023-11-15T03:02:20.267955+0000","last_became_peered":"2023-11-15T03:02:20.267955+0000","last_unstale":"2023-11-15T03:02:20.747304+0000","last_undegraded":"2023-11-15T03:02:20.747304+0000","last_fullsized":"2023-11-15T03:02:20.747304+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":69,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.726764+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T03:02:20.726764+0000","last_peered":"2023-11-15T03:02:20.726764+0000","last_clean":"2023-11-15T03:02:20.726764+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T03:02:20.726764+0000","last_undegraded":"2023-11-15T03:02:20.726764+0000","last_fullsized":"2023-11-15T03:02:20.726764+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.752737+0000","last_change":"2023-11-15T03:02:20.741987+0000","last_active":"2023-11-15T03:02:20.752737+0000","last_peered":"2023-11-15T03:02:20.752737+0000","last_clean":"2023-11-15T03:02:20.752737+0000","last_became_active":"2023-11-15T03:02:20.741662+0000","last_became_peered":"2023-11-15T03:02:20.741662+0000","last_unstale":"2023-11-15T03:02:20.752737+0000","last_undegraded":"2023-11-15T03:02:20.752737+0000","last_fullsized":"2023-11-15T03:02:20.752737+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.719624+0000","last_change":"2023-11-15T03:02:20.282979+0000","last_active":"2023-11-15T03:02:20.719624+0000","last_peered":"2023-11-15T03:02:20.719624+0000","last_clean":"2023-11-15T03:02:20.719624+0000","last_became_active":"2023-11-15T03:02:20.282672+0000","last_became_peered":"2023-11-15T03:02:20.282672+0000","last_unstale":"2023-11-15T03:02:20.719624+0000","last_undegraded":"2023-11-15T03:02:20.719624+0000","last_fullsized":"2023-11-15T03:02:20.719624+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.723312+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T03:02:20.723312+0000","last_peered":"2023-11-15T03:02:20.723312+0000","last_clean":"2023-11-15T03:02:20.723312+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T03:02:20.723312+0000","last_undegraded":"2023-11-15T03:02:20.723312+0000","last_fullsized":"2023-11-15T03:02:20.723312+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.751486+0000","last_change":"2023-11-15T03:02:20.748293+0000","last_active":"2023-11-15T03:02:20.751486+0000","last_peered":"2023-11-15T03:02:20.751486+0000","last_clean":"2023-11-15T03:02:20.751486+0000","last_became_active":"2023-11-15T03:02:20.748008+0000","last_became_peered":"2023-11-15T03:02:20.748008+0000","last_unstale":"2023-11-15T03:02:20.751486+0000","last_undegraded":"2023-11-15T03:02:20.751486+0000","last_fullsized":"2023-11-15T03:02:20.751486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.719594+0000","last_change":"2023-11-15T03:02:20.283130+0000","last_active":"2023-11-15T03:02:20.719594+0000","last_peered":"2023-11-15T03:02:20.719594+0000","last_clean":"2023-11-15T03:02:20.719594+0000","last_became_active":"2023-11-15T03:02:20.282849+0000","last_became_peered":"2023-11-15T03:02:20.282849+0000","last_unstale":"2023-11-15T03:02:20.719594+0000","last_undegraded":"2023-11-15T03:02:20.719594+0000","last_fullsized":"2023-11-15T03:02:20.719594+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":58,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.724729+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T03:02:20.724729+0000","last_peered":"2023-11-15T03:02:20.724729+0000","last_clean":"2023-11-15T03:02:20.724729+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T03:02:20.724729+0000","last_undegraded":"2023-11-15T03:02:20.724729+0000","last_fullsized":"2023-11-15T03:02:20.724729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.747020+0000","last_change":"2023-11-15T03:02:20.739863+0000","last_active":"2023-11-15T03:02:20.747020+0000","last_peered":"2023-11-15T03:02:20.747020+0000","last_clean":"2023-11-15T03:02:20.747020+0000","last_became_active":"2023-11-15T03:02:20.739574+0000","last_became_peered":"2023-11-15T03:02:20.739574+0000","last_unstale":"2023-11-15T03:02:20.747020+0000","last_undegraded":"2023-11-15T03:02:20.747020+0000","last_fullsized":"2023-11-15T03:02:20.747020+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:20.235488+0000","last_clean_scrub_stamp":"2023-11-15T03:02:20.235488+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.721133+0000","last_change":"2023-11-15T03:02:20.278159+0000","last_active":"2023-11-15T03:02:20.721133+0000","last_peered":"2023-11-15T03:02:20.721133+0000","last_clean":"2023-11-15T03:02:20.721133+0000","last_became_active":"2023-11-15T03:02:20.277707+0000","last_became_peered":"2023-11-15T03:02:20.277707+0000","last_unstale":"2023-11-15T03:02:20.721133+0000","last_undegraded":"2023-11-15T03:02:20.721133+0000","last_fullsized":"2023-11-15T03:02:20.721133+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:02:19.222469+0000","last_clean_scrub_stamp":"2023-11-15T03:02:19.222469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":69,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.725650+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T03:02:20.725650+0000","last_peered":"2023-11-15T03:02:20.725650+0000","last_clean":"2023-11-15T03:02:20.725650+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T03:02:20.725650+0000","last_undegraded":"2023-11-15T03:02:20.725650+0000","last_fullsized":"2023-11-15T03:02:20.725650+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-15T03:02:20.729942+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T03:02:20.729942+0000","last_peered":"2023-11-15T03:02:20.729942+0000","last_clean":"2023-11-15T03:02:20.729942+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T03:02:20.729942+0000","last_undegraded":"2023-11-15T03:02:20.729942+0000","last_fullsized":"2023-11-15T03:02:20.729942+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542214,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298016,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073824,"statfs":{"total":96636764160,"available":96331595776,"internally_reserved":0,"allocated":942080,"data_stored":428221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.181,"15min":1.1910000000000001},"min":{"1min":0.72399999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3360000000000001,"5min":10.961,"15min":10.961},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.141,"15min":1.1890000000000001},"min":{"1min":0.876,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.331,"5min":3.4910000000000001,"15min":3.4910000000000001},"last":1.101}]},{"osd":1,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.161,"15min":1.153},"min":{"1min":0.83099999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.4219999999999999,"5min":5.4930000000000003,"15min":5.4930000000000003},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2050000000000001,"15min":1.1879999999999999},"min":{"1min":0.79400000000000004,"5min":0.747,"15min":0.747},"max":{"1min":1.4830000000000001,"5min":6.8239999999999998,"15min":6.8239999999999998},"last":1.343}]},{"osd":2,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.1679999999999999,"15min":1.218},"min":{"1min":0.76300000000000001,"5min":0.751,"15min":0.751},"max":{"1min":1.49,"5min":5.7000000000000002,"15min":5.7000000000000002},"last":1.51},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2430000000000001,"15min":1.2889999999999999},"min":{"1min":0.93600000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.597,"5min":2.75,"15min":2.75},"last":1.119}]},{"osd":3,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.335,"15min":1.3720000000000001},"min":{"1min":1.014,"5min":0.751,"15min":0.751},"max":{"1min":1.6220000000000001,"5min":12.907,"15min":12.907},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.3160000000000001,"15min":1.298},"min":{"1min":0.995,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6679999999999999,"5min":9.9909999999999997,"15min":9.9909999999999997},"last":1.214}]},{"osd":4,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.139,"15min":1.2569999999999999},"min":{"1min":0.86599999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.3360000000000001,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":1.004},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2230000000000001,"15min":1.4239999999999999},"min":{"1min":0.85399999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5640000000000001,"5min":4.226,"15min":4.226},"last":1.1559999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.91000000000000003,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.5580000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.298,"15min":1.427},"min":{"1min":0.96899999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.609,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.8859999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:02:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.403,"15min":1.548},"min":{"1min":1.115,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.619,"5min":3.2789999999999999,"15min":3.2789999999999999},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.359,"15min":1.5669999999999999},"min":{"1min":0.93400000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6719999999999999,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":1.75}]}]},{"osd":6,"up_from":14,"seq":60129542214,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298020,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073820,"statfs":{"total":96636764160,"available":96331591680,"internally_reserved":0,"allocated":946176,"data_stored":428359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.0980000000000001,"15min":1.1240000000000001},"min":{"1min":0.80700000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.74,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.121,"5min":1.073,"15min":1.05},"min":{"1min":0.874,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.802,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.1739999999999999}]},{"osd":1,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.125,"15min":1.095},"min":{"1min":0.86899999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.512,"5min":1.623,"15min":1.623},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.123,"5min":1.1439999999999999,"15min":1.1599999999999999},"min":{"1min":0.83899999999999997,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.407,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.0329999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2030000000000001,"15min":1.1830000000000001},"min":{"1min":0.96499999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.7190000000000001,"5min":2.621,"15min":2.621},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.1399999999999999,"15min":1.099},"min":{"1min":0.93899999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4690000000000001,"5min":1.5,"15min":1.5},"last":0.98599999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2070000000000001,"15min":1.1699999999999999},"min":{"1min":1.0469999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.452,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":0.877},{"interface":"front","average":{"1min":1.373,"5min":1.286,"15min":1.272},"min":{"1min":0.96199999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7569999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.212}]},{"osd":4,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.097,"15min":1.1659999999999999},"min":{"1min":0.63500000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.3879999999999999,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.181,"15min":1.1850000000000001},"min":{"1min":0.91300000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.831,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.2789999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4279999999999999,"15min":1.4139999999999999},"min":{"1min":1.1919999999999999,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.621,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.329},{"interface":"front","average":{"1min":1.175,"5min":1.262,"15min":1.302},"min":{"1min":0.83599999999999997,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.492,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":1.143}]},{"osd":7,"last update":"Wed Nov 15 03:01:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2170000000000001,"15min":1.28},"min":{"1min":0.76200000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4570000000000001,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.071},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.4219999999999999,"15min":1.4470000000000001},"min":{"1min":0.98099999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.681,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":1.3779999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542215,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298020,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073820,"statfs":{"total":96636764160,"available":96331591680,"internally_reserved":0,"allocated":946176,"data_stored":428323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0940000000000001,"15min":1.1080000000000001},"min":{"1min":0.68200000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.355,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.175},{"interface":"front","average":{"1min":1.125,"5min":1.077,"15min":1.026},"min":{"1min":0.85799999999999998,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.3819999999999999,"5min":1.677,"15min":1.677},"last":0.85599999999999998}]},{"osd":1,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.1479999999999999,"15min":1.212},"min":{"1min":0.83999999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3089999999999999,"5min":2.617,"15min":2.617},"last":1.264},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1639999999999999,"15min":1.1659999999999999},"min":{"1min":0.88100000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.431,"5min":1.657,"15min":1.657},"last":1.0629999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.204,"15min":1.1579999999999999},"min":{"1min":0.98899999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4730000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.171,"5min":1.1519999999999999,"15min":1.161},"min":{"1min":0.82899999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.4590000000000001,"5min":1.667,"15min":1.667},"last":1.1359999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.141,"15min":1.1559999999999999},"min":{"1min":0.91100000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.401,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.276,"15min":1.294},"min":{"1min":1.04,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.4319999999999999,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":1}]},{"osd":5,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1819999999999999,"15min":1.232},"min":{"1min":0.752,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.4399999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.105,"5min":1.194,"15min":1.224},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.5149999999999999,"5min":1.994,"15min":1.994},"last":1.401}]},{"osd":6,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.28,"15min":1.325},"min":{"1min":0.877,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.494,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.341,"15min":1.3959999999999999},"min":{"1min":0.96999999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.52,"5min":2.173,"15min":2.173},"last":1.224}]},{"osd":7,"last update":"Wed Nov 15 03:01:56 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3160000000000001,"15min":1.3640000000000001},"min":{"1min":0.96499999999999997,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.486,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":1.046},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3819999999999999,"15min":1.4279999999999999},"min":{"1min":0.96499999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.512,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":1.3720000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542215,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298008,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073832,"statfs":{"total":96636764160,"available":96331603968,"internally_reserved":0,"allocated":933888,"data_stored":427645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.81999999999999995,"15min":0.77400000000000002},"min":{"1min":0.68300000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.02,"5min":1.169,"15min":1.169},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.87,"15min":0.83799999999999997},"min":{"1min":0.64000000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.244,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.71699999999999997}]},{"osd":1,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89100000000000001,"15min":0.89800000000000002},"min":{"1min":0.72899999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.163,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.006,"15min":0.97799999999999998},"min":{"1min":0.89600000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2549999999999999,"5min":1.375,"15min":1.375},"last":0.60099999999999998}]},{"osd":2,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.012,"15min":0.97999999999999998},"min":{"1min":0.75600000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4259999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.016},{"interface":"front","average":{"1min":1.099,"5min":1.0920000000000001,"15min":1.0609999999999999},"min":{"1min":0.83099999999999996,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3009999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.109}]},{"osd":4,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2310000000000001,"15min":1.1950000000000001},"min":{"1min":1.089,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.542,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.127},{"interface":"front","average":{"1min":1.21,"5min":1.206,"15min":1.222},"min":{"1min":1.0049999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.4370000000000001,"5min":1.659,"15min":1.659},"last":1.056}]},{"osd":5,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":1.0269999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.4790000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.288,"5min":1.2689999999999999,"15min":1.2649999999999999},"min":{"1min":1.0509999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.512,"5min":1.829,"15min":1.829},"last":1.0840000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3340000000000001,"15min":1.3129999999999999},"min":{"1min":1.143,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":1.5620000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.246},{"interface":"front","average":{"1min":1.383,"5min":1.3779999999999999,"15min":1.359},"min":{"1min":1.1399999999999999,"5min":1.004,"15min":1.004},"max":{"1min":1.581,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.96099999999999997}]},{"osd":7,"last update":"Wed Nov 15 03:01:51 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.357,"15min":1.3779999999999999},"min":{"1min":1.173,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.5800000000000001,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.3640000000000001,"15min":1.357},"min":{"1min":1.1339999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"max":{"1min":1.514,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.3620000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542215,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298016,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073824,"statfs":{"total":96636764160,"available":96331595776,"internally_reserved":0,"allocated":942080,"data_stored":427912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78100000000000003,"15min":0.76900000000000002},"min":{"1min":0.63700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.1479999999999999,"5min":1.409,"15min":1.409},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.878,"5min":0.85299999999999998,"15min":0.871},"min":{"1min":0.70199999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.339,"5min":1.619,"15min":1.619},"last":0.86099999999999999}]},{"osd":1,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.92400000000000004,"15min":0.90200000000000002},"min":{"1min":0.73399999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.014,"5min":0.97499999999999998,"15min":0.95299999999999996},"min":{"1min":0.81699999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.238,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.73399999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.98999999999999999,"15min":0.996},"min":{"1min":0.626,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.347,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.2,"5min":1.1379999999999999,"15min":1.1240000000000001},"min":{"1min":0.92600000000000005,"5min":0.622,"15min":0.622},"max":{"1min":1.5580000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.006}]},{"osd":4,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.218,"15min":1.2130000000000001},"min":{"1min":1.133,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6719999999999999,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.2270000000000001,"15min":1.226},"min":{"1min":1.03,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5029999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.266}]},{"osd":5,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2070000000000001,"15min":1.246},"min":{"1min":0.93999999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.554,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.276,"15min":1.29},"min":{"1min":1.0780000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.702,"5min":1.857,"15min":1.857},"last":1.2010000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.288,"15min":1.347},"min":{"1min":1.0109999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.679,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.2789999999999999,"15min":1.331},"min":{"1min":0.92400000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.524,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.252}]},{"osd":7,"last update":"Wed Nov 15 03:01:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3460000000000001,"15min":1.375},"min":{"1min":1.081,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.821,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3069999999999999,"15min":1.3680000000000001},"min":{"1min":1.1140000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.635,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.1759999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542215,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298008,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073832,"statfs":{"total":96636764160,"available":96331603968,"internally_reserved":0,"allocated":933888,"data_stored":427645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.85699999999999998,"15min":0.871},"min":{"1min":0.61499999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4390000000000001,"5min":4.3630000000000004,"15min":4.3630000000000004},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.97399999999999998,"15min":0.92200000000000004},"min":{"1min":0.67600000000000005,"5min":0.441,"15min":0.441},"max":{"1min":1.3520000000000001,"5min":6.157,"15min":6.157},"last":1.2450000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0109999999999999,"15min":1.0189999999999999},"min":{"1min":0.76800000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.571,"5min":4.1699999999999999,"15min":4.1699999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.046,"15min":1.006},"min":{"1min":0.75600000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4099999999999999,"5min":6.8369999999999997,"15min":6.8369999999999997},"last":1.482}]},{"osd":3,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.97599999999999998,"15min":0.96599999999999997},"min":{"1min":0.69599999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7330000000000001,"5min":3.5070000000000001,"15min":3.5070000000000001},"last":1.7050000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.143,"15min":1.1699999999999999},"min":{"1min":0.90400000000000003,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.55,"5min":5.2149999999999999,"15min":5.2149999999999999},"last":0.995}]},{"osd":4,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.2949999999999999,"15min":1.2629999999999999},"min":{"1min":1.0369999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6479999999999999,"5min":6.2000000000000002,"15min":6.2000000000000002},"last":1.54},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2270000000000001,"15min":1.1930000000000001},"min":{"1min":0.874,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.927,"5min":3.577,"15min":3.577},"last":1.357}]},{"osd":5,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.262,"15min":1.2330000000000001},"min":{"1min":0.97499999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9850000000000001,"5min":3.548,"15min":3.548},"last":1.8839999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.3220000000000001,"15min":1.3520000000000001},"min":{"1min":1.0840000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.6899999999999999,"5min":4.4080000000000004,"15min":4.4080000000000004},"last":1.456}]},{"osd":6,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.365,"15min":1.4139999999999999},"min":{"1min":1.1910000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.528,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3660000000000001,"15min":1.3540000000000001},"min":{"1min":1.0369999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9970000000000001,"5min":6.9690000000000003,"15min":6.9690000000000003},"last":1.585}]},{"osd":7,"last update":"Wed Nov 15 03:01:52 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.506,"15min":1.4259999999999999},"min":{"1min":0.94599999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.9390000000000001,"5min":18.253,"15min":18.253},"last":1.633},{"interface":"front","average":{"1min":1.343,"5min":1.4099999999999999,"15min":1.3100000000000001},"min":{"1min":1.129,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8280000000000001,"5min":11.006,"15min":11.006},"last":1.792}]}]},{"osd":5,"up_from":13,"seq":55834574920,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298012,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073828,"statfs":{"total":96636764160,"available":96331599872,"internally_reserved":0,"allocated":937984,"data_stored":427679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.081,"15min":1.056},"min":{"1min":0.81299999999999994,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4690000000000001,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.091,"15min":1.1950000000000001},"min":{"1min":0.86799999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3360000000000001,"5min":3.46,"15min":3.46},"last":1.1759999999999999}]},{"osd":1,"last update":"Wed Nov 15 03:01:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.127,"15min":1.224},"min":{"1min":0.75800000000000001,"5min":0.626,"15min":0.626},"max":{"1min":1.3460000000000001,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1499999999999999,"15min":1.1799999999999999},"min":{"1min":0.85399999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.407,"5min":3.516,"15min":3.516},"last":0.90100000000000002}]},{"osd":2,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.181,"15min":1.2050000000000001},"min":{"1min":0.84799999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.7370000000000001,"5min":5.4180000000000001,"15min":5.4180000000000001},"last":1.111},{"interface":"front","average":{"1min":1.234,"5min":1.218,"15min":1.2909999999999999},"min":{"1min":0.877,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.746,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":0.72699999999999998}]},{"osd":3,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1990000000000001,"15min":1.2769999999999999},"min":{"1min":0.90600000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.5049999999999999,"5min":2.141,"15min":2.141},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.339,"15min":1.427},"min":{"1min":1.0069999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.786,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":1.214}]},{"osd":4,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2749999999999999,"15min":1.357},"min":{"1min":0.86599999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.554,"5min":3.706,"15min":3.706},"last":1.159},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3380000000000001,"15min":1.552},"min":{"1min":0.84599999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5920000000000001,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":1.375}]},{"osd":6,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.262,"15min":1.3200000000000001},"min":{"1min":0.94299999999999995,"5min":0.749,"15min":0.749},"max":{"1min":1.5189999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":1.284},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.365,"15min":1.526},"min":{"1min":1.038,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.6659999999999999,"5min":3.641,"15min":3.641},"last":1.3320000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:01:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3109999999999999,"15min":1.399},"min":{"1min":0.995,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.635,"5min":3.516,"15min":3.516},"last":1.502},{"interface":"front","average":{"1min":1.417,"5min":1.4079999999999999,"15min":1.448},"min":{"1min":0.94899999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.7909999999999999,"5min":3.754,"15min":3.754},"last":1.355}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298024,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073816,"statfs":{"total":96636764160,"available":96331587584,"internally_reserved":0,"allocated":950272,"data_stored":428719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.82399999999999995,"15min":0.83799999999999997},"min":{"1min":0.63700000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.103,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.89400000000000002,"15min":0.88200000000000001},"min":{"1min":0.63,"5min":0.495,"15min":0.495},"max":{"1min":1.1950000000000001,"5min":1.419,"15min":1.419},"last":0.73999999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.91400000000000003,"15min":0.86599999999999999},"min":{"1min":0.70999999999999996,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94299999999999995,"15min":0.92200000000000004},"min":{"1min":0.66100000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.653,"5min":1.887,"15min":1.887},"last":0.88}]},{"osd":3,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.137,"15min":1.079},"min":{"1min":0.66600000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.494},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0629999999999999,"15min":1.0009999999999999},"min":{"1min":0.80800000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.444,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.3799999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.1850000000000001,"15min":1.169},"min":{"1min":0.94299999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.341,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2370000000000001,"15min":1.2549999999999999},"min":{"1min":1.024,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.522,"5min":2.206,"15min":2.206},"last":1.0609999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2410000000000001,"15min":1.2589999999999999},"min":{"1min":1.0329999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.5049999999999999,"5min":1.899,"15min":1.899},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2669999999999999,"15min":1.2849999999999999},"min":{"1min":0.99399999999999999,"5min":0.871,"15min":0.871},"max":{"1min":1.4039999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.2030000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.335},"min":{"1min":1.127,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.726,"5min":1.948,"15min":1.948},"last":1.325},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.401,"15min":1.409},"min":{"1min":1.054,"5min":0.87,"15min":0.87},"max":{"1min":1.637,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.3500000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:01:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3400000000000001,"15min":1.3240000000000001},"min":{"1min":1.0800000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.659,"5min":2.161,"15min":2.161},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3520000000000001,"15min":1.359},"min":{"1min":1.0700000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.573,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.266}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T03:02:30.397 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-15T03:02:30.398 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-15T03:02:30.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.598+0000 7f862f3f9700 1 Processor -- start 2023-11-15T03:02:30.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.598+0000 7f862f3f9700 1 -- start start 2023-11-15T03:02:30.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.602+0000 7f862f3f9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8628066350 0x7f8628151b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:30.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.602+0000 7f862f3f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8628156010 0x7f86280ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:30.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.602+0000 7f862f3f9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86281555c0 con 0x7f8628151780 2023-11-15T03:02:30.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.602+0000 7f862f3f9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8628155b80 con 0x7f8628066350 2023-11-15T03:02:30.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.602+0000 7f862f3f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86281564a0 con 0x7f8628156010 2023-11-15T03:02:30.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.602+0000 7f862dbf6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8628066350 0x7f8628151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:30.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.602+0000 7f862dbf6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8628066350 0x7f8628151b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40500/0 (socket says 172.21.15.121:40500) 2023-11-15T03:02:30.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.602+0000 7f862dbf6700 1 -- 172.21.15.121:0/2203637849 learned_addr learned my addr 172.21.15.121:0/2203637849 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:30.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.602+0000 7f862e3f7700 1 --1- 172.21.15.121:0/2203637849 >> v1:172.21.15.190:6789/0 conn(0x7f8628151780 0x7f8628065680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:53568/0 (socket says 172.21.15.121:53568) 2023-11-15T03:02:30.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.602+0000 7f862ebf8700 1 --2- 172.21.15.121:0/2203637849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8628156010 0x7f86280ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:30.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.602+0000 7f862dbf6700 1 -- 172.21.15.121:0/2203637849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8628156010 msgr2=0x7f86280ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:30.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.602+0000 7f862dbf6700 1 --2- 172.21.15.121:0/2203637849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8628156010 0x7f86280ab8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:30.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.602+0000 7f862dbf6700 1 -- 172.21.15.121:0/2203637849 >> v1:172.21.15.190:6789/0 conn(0x7f8628151780 legacy=0x7f8628065680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:30.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.602+0000 7f862dbf6700 1 -- 172.21.15.121:0/2203637849 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8628066850 con 0x7f8628066350 2023-11-15T03:02:30.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.602+0000 7f862dbf6700 1 --2- 172.21.15.121:0/2203637849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8628066350 0x7f8628151b60 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f8618009810 tx=0x7f8618009060).ready entity=mon.0 client_cookie=3fb98e465f71f88b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:30.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.602+0000 7f862d3f5700 1 -- 172.21.15.121:0/2203637849 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861800b640 con 0x7f8628066350 2023-11-15T03:02:30.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.602+0000 7f862d3f5700 1 -- 172.21.15.121:0/2203637849 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861800b7a0 con 0x7f8628066350 2023-11-15T03:02:30.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.602+0000 7f862d3f5700 1 -- 172.21.15.121:0/2203637849 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861800bb20 con 0x7f8628066350 2023-11-15T03:02:30.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.606+0000 7f862f3f9700 1 -- 172.21.15.121:0/2203637849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8628066350 msgr2=0x7f8628151b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:30.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.606+0000 7f862f3f9700 1 --2- 172.21.15.121:0/2203637849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8628066350 0x7f8628151b60 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f8618009810 tx=0x7f8618009060).stop 2023-11-15T03:02:30.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.606+0000 7f862f3f9700 1 -- 172.21.15.121:0/2203637849 shutdown_connections 2023-11-15T03:02:30.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.606+0000 7f862f3f9700 1 --2- 172.21.15.121:0/2203637849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8628156010 0x7f86280ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:30.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.606+0000 7f862f3f9700 1 --2- 172.21.15.121:0/2203637849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8628066350 0x7f8628151b60 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:30.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.606+0000 7f862f3f9700 1 -- 172.21.15.121:0/2203637849 >> 172.21.15.121:0/2203637849 conn(0x7f8628067a60 msgr2=0x7f8628067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:30.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.606+0000 7f862f3f9700 1 -- 172.21.15.121:0/2203637849 shutdown_connections 2023-11-15T03:02:30.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.606+0000 7f862f3f9700 1 -- 172.21.15.121:0/2203637849 wait complete. 2023-11-15T03:02:30.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.606+0000 7f862f3f9700 1 Processor -- start 2023-11-15T03:02:30.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.606+0000 7f862f3f9700 1 -- start start 2023-11-15T03:02:30.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f862f3f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8628066350 0x7f862814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:30.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f862f3f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8628151780 0x7f862814fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:30.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f862f3f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8628156010 0x7f8628153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:30.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f862f3f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86280c6aa0 con 0x7f8628066350 2023-11-15T03:02:30.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f862f3f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86280c5760 con 0x7f8628156010 2023-11-15T03:02:30.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f862f3f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86280c5a00 con 0x7f8628151780 2023-11-15T03:02:30.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f862e3f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8628066350 0x7f862814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:30.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f862e3f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8628066350 0x7f862814afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40512/0 (socket says 172.21.15.121:40512) 2023-11-15T03:02:30.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f862e3f7700 1 -- 172.21.15.121:0/3447951922 learned_addr learned my addr 172.21.15.121:0/3447951922 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:30.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f862dbf6700 1 --2- 172.21.15.121:0/3447951922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8628151780 0x7f862814fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:30.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f862ebf8700 1 --2- 172.21.15.121:0/3447951922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8628156010 0x7f8628153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:30.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f862e3f7700 1 -- 172.21.15.121:0/3447951922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8628151780 msgr2=0x7f862814fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:30.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f862e3f7700 1 --2- 172.21.15.121:0/3447951922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8628151780 0x7f862814fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:30.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f862e3f7700 1 -- 172.21.15.121:0/3447951922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8628156010 msgr2=0x7f8628153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:30.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f862e3f7700 1 --2- 172.21.15.121:0/3447951922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8628156010 0x7f8628153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:30.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f862e3f7700 1 -- 172.21.15.121:0/3447951922 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8618009300 con 0x7f8628066350 2023-11-15T03:02:30.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f862e3f7700 1 --2- 172.21.15.121:0/3447951922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8628066350 0x7f862814afd0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f8624007fd0 tx=0x7f8624007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:30.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f861e7fc700 1 -- 172.21.15.121:0/3447951922 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86240072e0 con 0x7f8628066350 2023-11-15T03:02:30.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f861e7fc700 1 -- 172.21.15.121:0/3447951922 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8624007440 con 0x7f8628066350 2023-11-15T03:02:30.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f862f3f9700 1 -- 172.21.15.121:0/3447951922 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86280c5ce0 con 0x7f8628066350 2023-11-15T03:02:30.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f861e7fc700 1 -- 172.21.15.121:0/3447951922 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86240058f0 con 0x7f8628066350 2023-11-15T03:02:30.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.610+0000 7f862f3f9700 1 -- 172.21.15.121:0/3447951922 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86280c6580 con 0x7f8628066350 2023-11-15T03:02:30.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.614+0000 7f862f3f9700 1 -- 172.21.15.121:0/3447951922 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86280c5ed0 con 0x7f8628066350 2023-11-15T03:02:30.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.614+0000 7f861e7fc700 1 -- 172.21.15.121:0/3447951922 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8624005ca0 con 0x7f8628066350 2023-11-15T03:02:30.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.614+0000 7f861e7fc700 1 --2- 172.21.15.121:0/3447951922 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f860c05a510 0x7f860c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:30.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.618+0000 7f861e7fc700 1 -- 172.21.15.121:0/3447951922 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8624074060 con 0x7f8628066350 2023-11-15T03:02:30.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.618+0000 7f862dbf6700 1 --2- 172.21.15.121:0/3447951922 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f860c05a510 0x7f860c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:30.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.618+0000 7f862dbf6700 1 --2- 172.21.15.121:0/3447951922 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f860c05a510 0x7f860c05c9d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f8618009fd0 tx=0x7f8618017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:30.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.622+0000 7f861e7fc700 1 -- 172.21.15.121:0/3447951922 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8624047840 con 0x7f8628066350 2023-11-15T03:02:30.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.838+0000 7f862f3f9700 1 -- 172.21.15.121:0/3447951922 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f86280c60d0 con 0x7f8628066350 2023-11-15T03:02:30.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.846+0000 7f861e7fc700 1 -- 172.21.15.121:0/3447951922 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7f8624018030 con 0x7f8628066350 2023-11-15T03:02:30.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.850+0000 7f862f3f9700 1 -- 172.21.15.121:0/3447951922 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f860c05a510 msgr2=0x7f860c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:30.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.850+0000 7f862f3f9700 1 --2- 172.21.15.121:0/3447951922 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f860c05a510 0x7f860c05c9d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f8618009fd0 tx=0x7f8618017000).stop 2023-11-15T03:02:30.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.850+0000 7f862f3f9700 1 -- 172.21.15.121:0/3447951922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8628066350 msgr2=0x7f862814afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:30.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.850+0000 7f862f3f9700 1 --2- 172.21.15.121:0/3447951922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8628066350 0x7f862814afd0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f8624007fd0 tx=0x7f8624007df0).stop 2023-11-15T03:02:30.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.850+0000 7f862f3f9700 1 -- 172.21.15.121:0/3447951922 shutdown_connections 2023-11-15T03:02:30.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.850+0000 7f862f3f9700 1 --2- 172.21.15.121:0/3447951922 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f860c05a510 0x7f860c05c9d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:30.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.850+0000 7f862f3f9700 1 --2- 172.21.15.121:0/3447951922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8628156010 0x7f8628153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:30.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.850+0000 7f862f3f9700 1 --2- 172.21.15.121:0/3447951922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8628151780 0x7f862814fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:30.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.850+0000 7f862f3f9700 1 --2- 172.21.15.121:0/3447951922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8628066350 0x7f862814afd0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:30.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.850+0000 7f862f3f9700 1 -- 172.21.15.121:0/3447951922 >> 172.21.15.121:0/3447951922 conn(0x7f8628067a60 msgr2=0x7f8628154da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:30.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.850+0000 7f862f3f9700 1 -- 172.21.15.121:0/3447951922 shutdown_connections 2023-11-15T03:02:30.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:30.850+0000 7f862f3f9700 1 -- 172.21.15.121:0/3447951922 wait complete. 2023-11-15T03:02:30.867 INFO:teuthology.orchestra.run.smithi121.stderr:updated caps for client.0 2023-11-15T03:02:30.878 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:02:31.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.042+0000 7fda0759e700 1 Processor -- start 2023-11-15T03:02:31.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.042+0000 7fda0759e700 1 -- start start 2023-11-15T03:02:31.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda0759e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fda080a9830 0x7fda080a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda0759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda080c30e0 0x7fda080c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda0759e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda080657f0 con 0x7fda08151870 2023-11-15T03:02:31.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda0759e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda08065ad0 con 0x7fda080a9830 2023-11-15T03:02:31.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda0759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda080c3a60 con 0x7fda080c30e0 2023-11-15T03:02:31.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda05d9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fda080a9830 0x7fda080a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda05d9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fda080a9830 0x7fda080a9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40524/0 (socket says 172.21.15.121:40524) 2023-11-15T03:02:31.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda05d9b700 1 -- 172.21.15.121:0/498213216 learned_addr learned my addr 172.21.15.121:0/498213216 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:31.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda05d9b700 1 -- 172.21.15.121:0/498213216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda080c30e0 msgr2=0x7fda080c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:31.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda06d9d700 1 --2- 172.21.15.121:0/498213216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda080c30e0 0x7fda080c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda05d9b700 1 --2- 172.21.15.121:0/498213216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda080c30e0 0x7fda080c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda05d9b700 1 -- 172.21.15.121:0/498213216 >> v1:172.21.15.190:6789/0 conn(0x7fda08151870 legacy=0x7fda08151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:31.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda05d9b700 1 -- 172.21.15.121:0/498213216 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda080aa260 con 0x7fda080a9830 2023-11-15T03:02:31.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda06d9d700 1 --2- 172.21.15.121:0/498213216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda080c30e0 0x7fda080c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:02:31.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda05d9b700 1 --2- 172.21.15.121:0/498213216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda080a9830 0x7fda080a9c70 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7fd9fc006fd0 tx=0x7fd9fc0090f0).ready entity=mon.0 client_cookie=fed8276646a8aeac server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:31.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda0559a700 1 -- 172.21.15.121:0/498213216 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc00b600 con 0x7fda080a9830 2023-11-15T03:02:31.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda0559a700 1 -- 172.21.15.121:0/498213216 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9fc00b760 con 0x7fda080a9830 2023-11-15T03:02:31.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda0559a700 1 -- 172.21.15.121:0/498213216 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc00bae0 con 0x7fda080a9830 2023-11-15T03:02:31.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda0759e700 1 -- 172.21.15.121:0/498213216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda080a9830 msgr2=0x7fda080a9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:31.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda0759e700 1 --2- 172.21.15.121:0/498213216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda080a9830 0x7fda080a9c70 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7fd9fc006fd0 tx=0x7fd9fc0090f0).stop 2023-11-15T03:02:31.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda0759e700 1 -- 172.21.15.121:0/498213216 shutdown_connections 2023-11-15T03:02:31.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda0759e700 1 --2- 172.21.15.121:0/498213216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda080c30e0 0x7fda080c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda0759e700 1 --2- 172.21.15.121:0/498213216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda080a9830 0x7fda080a9c70 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda0759e700 1 -- 172.21.15.121:0/498213216 >> 172.21.15.121:0/498213216 conn(0x7fda08062190 msgr2=0x7fda08062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:31.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda0759e700 1 -- 172.21.15.121:0/498213216 shutdown_connections 2023-11-15T03:02:31.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.046+0000 7fda0759e700 1 -- 172.21.15.121:0/498213216 wait complete. 2023-11-15T03:02:31.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda0759e700 1 Processor -- start 2023-11-15T03:02:31.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda0759e700 1 -- start start 2023-11-15T03:02:31.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda0759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda080a9830 0x7fda08164130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda0759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda080c30e0 0x7fda0814d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda0759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda08151870 0x7fda08156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda0759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda08159a00 con 0x7fda08151870 2023-11-15T03:02:31.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda0759e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda08157230 con 0x7fda080c30e0 2023-11-15T03:02:31.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda0759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda08152c20 con 0x7fda080a9830 2023-11-15T03:02:31.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda06d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda08151870 0x7fda08156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda05d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda080c30e0 0x7fda0814d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda06d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda08151870 0x7fda08156850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40536/0 (socket says 172.21.15.121:40536) 2023-11-15T03:02:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda06d9d700 1 -- 172.21.15.121:0/1146646590 learned_addr learned my addr 172.21.15.121:0/1146646590 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda0659c700 1 --2- 172.21.15.121:0/1146646590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda080a9830 0x7fda08164130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda06d9d700 1 -- 172.21.15.121:0/1146646590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda080a9830 msgr2=0x7fda08164130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda06d9d700 1 --2- 172.21.15.121:0/1146646590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda080a9830 0x7fda08164130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda06d9d700 1 -- 172.21.15.121:0/1146646590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda080c30e0 msgr2=0x7fda0814d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:31.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda06d9d700 1 --2- 172.21.15.121:0/1146646590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda080c30e0 0x7fda0814d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda06d9d700 1 -- 172.21.15.121:0/1146646590 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc009370 con 0x7fda08151870 2023-11-15T03:02:31.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda06d9d700 1 --2- 172.21.15.121:0/1146646590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda08151870 0x7fda08156850 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7fda0000cef0 tx=0x7fda00007860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:31.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fd9f67fc700 1 -- 172.21.15.121:0/1146646590 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0000add0 con 0x7fda08151870 2023-11-15T03:02:31.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fd9f67fc700 1 -- 172.21.15.121:0/1146646590 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda00016400 con 0x7fda08151870 2023-11-15T03:02:31.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda0759e700 1 -- 172.21.15.121:0/1146646590 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda08152f00 con 0x7fda08151870 2023-11-15T03:02:31.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fd9f67fc700 1 -- 172.21.15.121:0/1146646590 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00016780 con 0x7fda08151870 2023-11-15T03:02:31.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.050+0000 7fda0759e700 1 -- 172.21.15.121:0/1146646590 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda081534c0 con 0x7fda08151870 2023-11-15T03:02:31.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.054+0000 7fda0759e700 1 -- 172.21.15.121:0/1146646590 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda081646e0 con 0x7fda08151870 2023-11-15T03:02:31.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.054+0000 7fd9f67fc700 1 -- 172.21.15.121:0/1146646590 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fda000203f0 con 0x7fda08151870 2023-11-15T03:02:31.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.058+0000 7fd9f67fc700 1 --2- 172.21.15.121:0/1146646590 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd9e805a510 0x7fd9e805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.058+0000 7fd9f67fc700 1 -- 172.21.15.121:0/1146646590 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fda00075260 con 0x7fda08151870 2023-11-15T03:02:31.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.058+0000 7fda0659c700 1 --2- 172.21.15.121:0/1146646590 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd9e805a510 0x7fd9e805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.058+0000 7fda0659c700 1 --2- 172.21.15.121:0/1146646590 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd9e805a510 0x7fd9e805c9d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fd9f80036e0 tx=0x7fd9f8009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:31.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.062+0000 7fd9f67fc700 1 -- 172.21.15.121:0/1146646590 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fda00048920 con 0x7fda08151870 2023-11-15T03:02:31.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.262+0000 7fda0759e700 1 -- 172.21.15.121:0/1146646590 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda080aa540 con 0x7fda08151870 2023-11-15T03:02:31.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.262+0000 7fd9f67fc700 1 -- 172.21.15.121:0/1146646590 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4497 (secure 0 0 0) 0x7fda0004bf40 con 0x7fda08151870 2023-11-15T03:02:31.273 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:31.273 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":67,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1508681155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1508681155},{"type":"v1","addr":"172.21.15.121:6837","nonce":1508681155}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6385,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3559276534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3559276534},{"type":"v1","addr":"172.21.15.190:6835","nonce":3559276534}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6486,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/834528709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":834528709},{"type":"v1","addr":"172.21.15.121:6836","nonce":834528709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:02:20.703078+0000","modified":"2023-11-15T03:02:23.967255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6492},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6492":{"gid":6492,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6833/914796342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":914796342},{"type":"v1","addr":"172.21.15.190:6833","nonce":914796342}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-15T03:02:31.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.266+0000 7fda0759e700 1 -- 172.21.15.121:0/1146646590 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd9e805a510 msgr2=0x7fd9e805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:31.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.266+0000 7fda0759e700 1 --2- 172.21.15.121:0/1146646590 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd9e805a510 0x7fd9e805c9d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fd9f80036e0 tx=0x7fd9f8009040).stop 2023-11-15T03:02:31.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.266+0000 7fda0759e700 1 -- 172.21.15.121:0/1146646590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda08151870 msgr2=0x7fda08156850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:31.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.266+0000 7fda0759e700 1 --2- 172.21.15.121:0/1146646590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda08151870 0x7fda08156850 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7fda0000cef0 tx=0x7fda00007860).stop 2023-11-15T03:02:31.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.266+0000 7fda0759e700 1 -- 172.21.15.121:0/1146646590 shutdown_connections 2023-11-15T03:02:31.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.266+0000 7fda0759e700 1 --2- 172.21.15.121:0/1146646590 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd9e805a510 0x7fd9e805c9d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.266+0000 7fda0759e700 1 --2- 172.21.15.121:0/1146646590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda08151870 0x7fda08156850 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.266+0000 7fda0759e700 1 --2- 172.21.15.121:0/1146646590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda080c30e0 0x7fda0814d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.266+0000 7fda0759e700 1 --2- 172.21.15.121:0/1146646590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda080a9830 0x7fda08164130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.266+0000 7fda0759e700 1 -- 172.21.15.121:0/1146646590 >> 172.21.15.121:0/1146646590 conn(0x7fda08062190 msgr2=0x7fda080c26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:31.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.266+0000 7fda0759e700 1 -- 172.21.15.121:0/1146646590 shutdown_connections 2023-11-15T03:02:31.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.266+0000 7fda0759e700 1 -- 172.21.15.121:0/1146646590 wait complete. 2023-11-15T03:02:31.278 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 67 2023-11-15T03:02:31.696 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:02:20.703078+0000', 'modified': '2023-11-15T03:02:23.967255+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6492}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6492': {'gid': 6492, 'name': 'd', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6833/914796342', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 914796342}, {'type': 'v1', 'addr': '172.21.15.190:6833', 'nonce': 914796342}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:02:31.696 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T03:02:31.697 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-15T03:02:31.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.854+0000 7f02815b9700 1 Processor -- start 2023-11-15T03:02:31.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.854+0000 7f02815b9700 1 -- start start 2023-11-15T03:02:31.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.858+0000 7f02815b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f027c0669b0 0x7f027c158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.858+0000 7f02815b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f027c0a8750 0x7f027c0a8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.858+0000 7f02815b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f027c0bb910 con 0x7f027c157d90 2023-11-15T03:02:31.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.858+0000 7f02815b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f027c0c1fb0 con 0x7f027c0669b0 2023-11-15T03:02:31.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.858+0000 7f02815b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f027c15bbd0 con 0x7f027c0a8750 2023-11-15T03:02:31.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.858+0000 7f02737fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f027c0669b0 0x7f027c158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.858+0000 7f02737fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f027c0669b0 0x7f027c158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40552/0 (socket says 172.21.15.121:40552) 2023-11-15T03:02:31.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.858+0000 7f02737fe700 1 -- 172.21.15.121:0/1303127070 learned_addr learned my addr 172.21.15.121:0/1303127070 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:31.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.858+0000 7f027bfff700 1 --1- 172.21.15.121:0/1303127070 >> v1:172.21.15.190:6789/0 conn(0x7f027c157d90 0x7f027c0c1760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:53584/0 (socket says 172.21.15.121:53584) 2023-11-15T03:02:31.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.858+0000 7f0280db8700 1 --2- 172.21.15.121:0/1303127070 >> v2:172.21.15.190:3300/0 conn(0x7f027c0a8750 0x7f027c0a8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.858+0000 7f02737fe700 1 -- 172.21.15.121:0/1303127070 >> v2:172.21.15.190:3300/0 conn(0x7f027c0a8750 msgr2=0x7f027c0a8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:31.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.858+0000 7f02737fe700 1 --2- 172.21.15.121:0/1303127070 >> v2:172.21.15.190:3300/0 conn(0x7f027c0a8750 0x7f027c0a8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.858+0000 7f02737fe700 1 -- 172.21.15.121:0/1303127070 >> v1:172.21.15.190:6789/0 conn(0x7f027c157d90 legacy=0x7f027c0c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:31.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.858+0000 7f02737fe700 1 -- 172.21.15.121:0/1303127070 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027c0a9090 con 0x7f027c0669b0 2023-11-15T03:02:31.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.862+0000 7f02737fe700 1 --2- 172.21.15.121:0/1303127070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f027c0669b0 0x7f027c158170 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f027c15bd10 tx=0x7f0264009060).ready entity=mon.0 client_cookie=268d788a099420bc server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:31.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.862+0000 7f027b7fe700 1 -- 172.21.15.121:0/1303127070 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264005880 con 0x7f027c0669b0 2023-11-15T03:02:31.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.862+0000 7f027b7fe700 1 -- 172.21.15.121:0/1303127070 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02640059e0 con 0x7f027c0669b0 2023-11-15T03:02:31.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.862+0000 7f027b7fe700 1 -- 172.21.15.121:0/1303127070 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264005d60 con 0x7f027c0669b0 2023-11-15T03:02:31.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.862+0000 7f02815b9700 1 -- 172.21.15.121:0/1303127070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f027c0669b0 msgr2=0x7f027c158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:31.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.862+0000 7f02815b9700 1 --2- 172.21.15.121:0/1303127070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f027c0669b0 0x7f027c158170 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f027c15bd10 tx=0x7f0264009060).stop 2023-11-15T03:02:31.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.862+0000 7f02815b9700 1 -- 172.21.15.121:0/1303127070 shutdown_connections 2023-11-15T03:02:31.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.862+0000 7f02815b9700 1 --2- 172.21.15.121:0/1303127070 >> v2:172.21.15.190:3300/0 conn(0x7f027c0a8750 0x7f027c0a8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.862+0000 7f02815b9700 1 --2- 172.21.15.121:0/1303127070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f027c0669b0 0x7f027c158170 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.862+0000 7f02815b9700 1 -- 172.21.15.121:0/1303127070 >> 172.21.15.121:0/1303127070 conn(0x7f027c067a60 msgr2=0x7f027c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:31.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.862+0000 7f02815b9700 1 -- 172.21.15.121:0/1303127070 shutdown_connections 2023-11-15T03:02:31.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.862+0000 7f02815b9700 1 -- 172.21.15.121:0/1303127070 wait complete. 2023-11-15T03:02:31.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.866+0000 7f02815b9700 1 Processor -- start 2023-11-15T03:02:31.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.866+0000 7f02815b9700 1 -- start start 2023-11-15T03:02:31.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.866+0000 7f02815b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f027c0669b0 0x7f027c14ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.866+0000 7f02815b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f027c0a8750 0x7f027c156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.866+0000 7f02815b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f027c157d90 0x7f027c156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.866+0000 7f02815b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c156e20 con 0x7f027c157d90 2023-11-15T03:02:31.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.866+0000 7f02815b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c14f340 con 0x7f027c0669b0 2023-11-15T03:02:31.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.866+0000 7f02815b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027c150a10 con 0x7f027c0a8750 2023-11-15T03:02:31.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.866+0000 7f0280db8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f027c157d90 0x7f027c156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.866+0000 7f0280db8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f027c157d90 0x7f027c156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40568/0 (socket says 172.21.15.121:40568) 2023-11-15T03:02:31.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.866+0000 7f027bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f027c0669b0 0x7f027c14ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.866+0000 7f0280db8700 1 -- 172.21.15.121:0/191997716 learned_addr learned my addr 172.21.15.121:0/191997716 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:31.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.866+0000 7f02737fe700 1 --2- 172.21.15.121:0/191997716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f027c0a8750 0x7f027c156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.866+0000 7f0280db8700 1 -- 172.21.15.121:0/191997716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f027c0a8750 msgr2=0x7f027c156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:31.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.866+0000 7f0280db8700 1 --2- 172.21.15.121:0/191997716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f027c0a8750 0x7f027c156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.866+0000 7f0280db8700 1 -- 172.21.15.121:0/191997716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f027c0669b0 msgr2=0x7f027c14ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:31.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.866+0000 7f0280db8700 1 --2- 172.21.15.121:0/191997716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f027c0669b0 0x7f027c14ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.866+0000 7f0280db8700 1 -- 172.21.15.121:0/191997716 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0264009300 con 0x7f027c157d90 2023-11-15T03:02:31.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.870+0000 7f0280db8700 1 --2- 172.21.15.121:0/191997716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f027c157d90 0x7f027c156930 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7f0274009e90 tx=0x7f027400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:31.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.870+0000 7f0278ff9700 1 -- 172.21.15.121:0/191997716 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027400caa0 con 0x7f027c157d90 2023-11-15T03:02:31.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.870+0000 7f0278ff9700 1 -- 172.21.15.121:0/191997716 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027400f040 con 0x7f027c157d90 2023-11-15T03:02:31.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.870+0000 7f0278ff9700 1 -- 172.21.15.121:0/191997716 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027400ce10 con 0x7f027c157d90 2023-11-15T03:02:31.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.870+0000 7f02815b9700 1 -- 172.21.15.121:0/191997716 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027c150cf0 con 0x7f027c157d90 2023-11-15T03:02:31.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.870+0000 7f02815b9700 1 -- 172.21.15.121:0/191997716 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f027c158970 con 0x7f027c157d90 2023-11-15T03:02:31.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.874+0000 7f0278ff9700 1 -- 172.21.15.121:0/191997716 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0274013610 con 0x7f027c157d90 2023-11-15T03:02:31.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.874+0000 7f0278ff9700 1 --2- 172.21.15.121:0/191997716 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f026005a510 0x7f026005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.874+0000 7f0278ff9700 1 -- 172.21.15.121:0/191997716 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f02740750a0 con 0x7f027c157d90 2023-11-15T03:02:31.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.874+0000 7f028cbc1740 1 Processor -- start 2023-11-15T03:02:31.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.874+0000 7f027bfff700 1 --2- 172.21.15.121:0/191997716 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f026005a510 0x7f026005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.874+0000 7f028cbc1740 1 -- start start 2023-11-15T03:02:31.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.874+0000 7f027bfff700 1 --2- 172.21.15.121:0/191997716 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f026005a510 0x7f026005c9d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f026c007fd0 tx=0x7f026c007d30).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:31.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f028cbc1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2035ad0 0x205c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f028cbc1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035eb0 0x2082140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f028cbc1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2082630 0x2082ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f028cbc1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fb6410 con 0x2082630 2023-11-15T03:02:31.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f028cbc1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fb6550 con 0x2035eb0 2023-11-15T03:02:31.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f028cbc1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1fb6a00 con 0x2035ad0 2023-11-15T03:02:31.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f0280db8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2082630 0x2082ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f0280db8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2082630 0x2082ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40570/0 (socket says 172.21.15.121:40570) 2023-11-15T03:02:31.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f0280db8700 1 -- 172.21.15.121:0/981450552 learned_addr learned my addr 172.21.15.121:0/981450552 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:31.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f027bfff700 1 --2- 172.21.15.121:0/981450552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2035ad0 0x205c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f02737fe700 1 --2- 172.21.15.121:0/981450552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035eb0 0x2082140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f0280db8700 1 -- 172.21.15.121:0/981450552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2035ad0 msgr2=0x205c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:31.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f0280db8700 1 --2- 172.21.15.121:0/981450552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2035ad0 0x205c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f0280db8700 1 -- 172.21.15.121:0/981450552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035eb0 msgr2=0x2082140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:31.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f0280db8700 1 --2- 172.21.15.121:0/981450552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035eb0 0x2082140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f0280db8700 1 -- 172.21.15.121:0/981450552 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1faf730 con 0x2082630 2023-11-15T03:02:31.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f027bfff700 1 --2- 172.21.15.121:0/981450552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2035ad0 0x205c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:31.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f0280db8700 1 --2- 172.21.15.121:0/981450552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2082630 0x2082ab0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f027404c180 tx=0x7f0274009780).ready entity=mon.0 client_cookie=133ea5c91ef21c7e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:31.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f02717fa700 1 -- 172.21.15.121:0/981450552 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0274051070 con 0x2082630 2023-11-15T03:02:31.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f02717fa700 1 -- 172.21.15.121:0/981450552 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0274048b90 con 0x2082630 2023-11-15T03:02:31.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f02717fa700 1 -- 172.21.15.121:0/981450552 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027404bb80 con 0x2082630 2023-11-15T03:02:31.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f028cbc1740 1 -- 172.21.15.121:0/981450552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2082630 msgr2=0x2082ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:31.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f028cbc1740 1 --2- 172.21.15.121:0/981450552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2082630 0x2082ab0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f027404c180 tx=0x7f0274009780).stop 2023-11-15T03:02:31.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f028cbc1740 1 -- 172.21.15.121:0/981450552 shutdown_connections 2023-11-15T03:02:31.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f028cbc1740 1 --2- 172.21.15.121:0/981450552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2082630 0x2082ab0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f028cbc1740 1 --2- 172.21.15.121:0/981450552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035eb0 0x2082140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f028cbc1740 1 --2- 172.21.15.121:0/981450552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2035ad0 0x205c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f028cbc1740 1 -- 172.21.15.121:0/981450552 >> 172.21.15.121:0/981450552 conn(0x1e14550 msgr2=0x20347e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:31.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f028cbc1740 1 -- 172.21.15.121:0/981450552 shutdown_connections 2023-11-15T03:02:31.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.878+0000 7f028cbc1740 1 -- 172.21.15.121:0/981450552 wait complete. 2023-11-15T03:02:31.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f028cbc1740 1 -- start start 2023-11-15T03:02:31.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f028cbc1740 1 Processor -- start 2023-11-15T03:02:31.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f028cbc1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2035ad0 0x20735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f028cbc1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035eb0 0x205b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f028cbc1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2082630 0x2076290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f028cbc1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f81650 con 0x2035ad0 2023-11-15T03:02:31.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f028cbc1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x2081920 con 0x2035eb0 2023-11-15T03:02:31.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f028cbc1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x205e5a0 con 0x2082630 2023-11-15T03:02:31.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f028cbc1740 10 client.0 ll_register_callbacks cb 0x1fee020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:02:31.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f027bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2035ad0 0x20735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f027bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2035ad0 0x20735c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40578/0 (socket says 172.21.15.121:40578) 2023-11-15T03:02:31.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f027bfff700 1 -- 172.21.15.121:0/4118908340 learned_addr learned my addr 172.21.15.121:0/4118908340 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:31.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f0280db8700 1 --2- 172.21.15.121:0/4118908340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2082630 0x2076290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f02737fe700 1 --2- 172.21.15.121:0/4118908340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035eb0 0x205b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f027bfff700 1 -- 172.21.15.121:0/4118908340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2082630 msgr2=0x2076290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:31.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f027bfff700 1 --2- 172.21.15.121:0/4118908340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2082630 0x2076290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f027bfff700 1 -- 172.21.15.121:0/4118908340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035eb0 msgr2=0x205b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:31.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f027bfff700 1 --2- 172.21.15.121:0/4118908340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035eb0 0x205b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f027bfff700 1 -- 172.21.15.121:0/4118908340 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f027c158970 con 0x2035ad0 2023-11-15T03:02:31.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f02737fe700 1 --2- 172.21.15.121:0/4118908340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035eb0 0x205b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:02:31.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f0280db8700 1 --2- 172.21.15.121:0/4118908340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2082630 0x2076290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:02:31.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f027bfff700 1 --2- 172.21.15.121:0/4118908340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2035ad0 0x20735c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f026c0143d0 tx=0x7f026c00a000).ready entity=mon.0 client_cookie=996cb8bf7bf9cec6 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:31.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f026a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:02:31.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f026a7fc700 1 -- 172.21.15.121:0/4118908340 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c014cd0 con 0x2035ad0 2023-11-15T03:02:31.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f026a7fc700 1 -- 172.21.15.121:0/4118908340 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f026c014e30 con 0x2035ad0 2023-11-15T03:02:31.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f028cbc1740 1 -- 172.21.15.121:0/4118908340 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1fac1a0 con 0x2035ad0 2023-11-15T03:02:31.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f026a7fc700 1 -- 172.21.15.121:0/4118908340 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f026c023e70 con 0x2035ad0 2023-11-15T03:02:31.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f028cbc1740 10 client.6672 fetch_fsmap learned FSMap version 67 2023-11-15T03:02:31.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.882+0000 7f028cbc1740 1 -- 172.21.15.121:0/4118908340 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x1faa960 con 0x2035ad0 2023-11-15T03:02:31.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.886+0000 7f026a7fc700 1 -- 172.21.15.121:0/4118908340 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1fac1a0 con 0x2035ad0 2023-11-15T03:02:31.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.886+0000 7f026a7fc700 1 -- 172.21.15.121:0/4118908340 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 67) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f026c020070 con 0x2035ad0 2023-11-15T03:02:31.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.886+0000 7f028cbc1740 10 client.6672 fetch_fsmap finished waiting for FSMap version 67 2023-11-15T03:02:31.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.886+0000 7f028cbc1740 10 client.6672 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-15T03:02:31.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.886+0000 7f028cbc1740 20 client.6672 populate_metadata read hostname 'smithi121' 2023-11-15T03:02:31.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.886+0000 7f028cbc1740 1 --2- 172.21.15.121:0/4118908340 >> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] conn(0x205da20 0x205de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.886+0000 7f028cbc1740 4 client.6672 mds_command: new command op to 6492 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:02:31.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.886+0000 7f028cbc1740 1 -- 172.21.15.121:0/4118908340 --> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1fa26f0 con 0x205da20 2023-11-15T03:02:31.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.886+0000 7f02737fe700 1 --2- 172.21.15.121:0/4118908340 >> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] conn(0x205da20 0x205de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.886+0000 7f026a7fc700 10 client.6672 ms_handle_connect on v2:172.21.15.190:6832/914796342 2023-11-15T03:02:31.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.886+0000 7f02737fe700 1 --2- 172.21.15.121:0/4118908340 >> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] conn(0x205da20 0x205de00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=26df8d7c22060504 server_cookie=18151831417aad2a in_seq=0 out_seq=0 2023-11-15T03:02:31.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.890+0000 7f026a7fc700 1 -- 172.21.15.121:0/4118908340 <== mds.0 v2:172.21.15.190:6832/914796342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x1fa26f0 con 0x205da20 2023-11-15T03:02:31.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.890+0000 7f026a7fc700 10 client.6672 handle_command_reply: tid=0 2023-11-15T03:02:31.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.890+0000 7f028cbc1740 1 client.6672 shutdown 2023-11-15T03:02:31.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.890+0000 7f028cbc1740 1 -- 172.21.15.121:0/4118908340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2035ad0 msgr2=0x20735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:31.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.890+0000 7f028cbc1740 1 --2- 172.21.15.121:0/4118908340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2035ad0 0x20735c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f026c0143d0 tx=0x7f026c00a000).stop 2023-11-15T03:02:31.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.890+0000 7f028cbc1740 1 -- 172.21.15.121:0/4118908340 shutdown_connections 2023-11-15T03:02:31.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.890+0000 7f028cbc1740 1 --2- 172.21.15.121:0/4118908340 >> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] conn(0x205da20 0x205de00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.890+0000 7f028cbc1740 1 --2- 172.21.15.121:0/4118908340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2082630 0x2076290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.890+0000 7f026a7fc700 0 client.6672 ms_handle_reset on v2:172.21.15.190:6832/914796342 2023-11-15T03:02:31.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.890+0000 7f028cbc1740 1 --2- 172.21.15.121:0/4118908340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035eb0 0x205b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.890+0000 7f028cbc1740 1 --2- 172.21.15.121:0/4118908340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2035ad0 0x20735c0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.890+0000 7f028cbc1740 1 -- 172.21.15.121:0/4118908340 >> 172.21.15.121:0/4118908340 conn(0x1e14550 msgr2=0x207d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:31.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.890+0000 7f028cbc1740 1 -- 172.21.15.121:0/4118908340 shutdown_connections 2023-11-15T03:02:31.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.890+0000 7f028cbc1740 1 -- 172.21.15.121:0/4118908340 wait complete. 2023-11-15T03:02:31.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.890+0000 7f028cbc1740 20 client.6672 trim_cache size 0 max 16384 2023-11-15T03:02:31.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.910+0000 7f028cbc1740 1 Processor -- start 2023-11-15T03:02:31.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.910+0000 7f028cbc1740 1 -- start start 2023-11-15T03:02:31.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.910+0000 7f028cbc1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2035ad0 0x207ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.910+0000 7f028cbc1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035eb0 0x2075d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.910+0000 7f028cbc1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2082630 0x2030f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.910+0000 7f028cbc1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f8ef80 con 0x2082630 2023-11-15T03:02:31.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.910+0000 7f028cbc1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x2011e20 con 0x2035eb0 2023-11-15T03:02:31.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.910+0000 7f028cbc1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x207ccf0 con 0x2035ad0 2023-11-15T03:02:31.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.910+0000 7f0280db8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2082630 0x2030f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.910+0000 7f0280db8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2082630 0x2030f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40594/0 (socket says 172.21.15.121:40594) 2023-11-15T03:02:31.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.910+0000 7f0280db8700 1 -- 172.21.15.121:0/676646717 learned_addr learned my addr 172.21.15.121:0/676646717 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:31.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.910+0000 7f027bfff700 1 --2- 172.21.15.121:0/676646717 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2035ad0 0x207ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.910+0000 7f02737fe700 1 --2- 172.21.15.121:0/676646717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035eb0 0x2075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f0280db8700 1 -- 172.21.15.121:0/676646717 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2035ad0 msgr2=0x207ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:31.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f0280db8700 1 --2- 172.21.15.121:0/676646717 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2035ad0 0x207ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f0280db8700 1 -- 172.21.15.121:0/676646717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035eb0 msgr2=0x2075d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:31.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f0280db8700 1 --2- 172.21.15.121:0/676646717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035eb0 0x2075d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f0280db8700 1 -- 172.21.15.121:0/676646717 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x205bb70 con 0x2082630 2023-11-15T03:02:31.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f0280db8700 1 --2- 172.21.15.121:0/676646717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2082630 0x2030f10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f027404c180 tx=0x7f0274055000).ready entity=mon.0 client_cookie=930ed8801b167763 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:31.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f0271ffb700 1 -- 172.21.15.121:0/676646717 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0274048d00 con 0x2082630 2023-11-15T03:02:31.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f0271ffb700 1 -- 172.21.15.121:0/676646717 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027400c890 con 0x2082630 2023-11-15T03:02:31.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f0271ffb700 1 -- 172.21.15.121:0/676646717 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027404b970 con 0x2082630 2023-11-15T03:02:31.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f028cbc1740 1 -- 172.21.15.121:0/676646717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2082630 msgr2=0x2030f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:31.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f028cbc1740 1 --2- 172.21.15.121:0/676646717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2082630 0x2030f10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f027404c180 tx=0x7f0274055000).stop 2023-11-15T03:02:31.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f028cbc1740 1 -- 172.21.15.121:0/676646717 shutdown_connections 2023-11-15T03:02:31.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f028cbc1740 1 --2- 172.21.15.121:0/676646717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2082630 0x2030f10 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f028cbc1740 1 --2- 172.21.15.121:0/676646717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035eb0 0x2075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f028cbc1740 1 --2- 172.21.15.121:0/676646717 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2035ad0 0x207ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f028cbc1740 1 -- 172.21.15.121:0/676646717 >> 172.21.15.121:0/676646717 conn(0x1e14550 msgr2=0x205de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:31.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f028cbc1740 1 -- 172.21.15.121:0/676646717 shutdown_connections 2023-11-15T03:02:31.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f028cbc1740 1 -- 172.21.15.121:0/676646717 wait complete. 2023-11-15T03:02:31.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f028cbc1740 1 -- start start 2023-11-15T03:02:31.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.914+0000 7f028cbc1740 1 Processor -- start 2023-11-15T03:02:31.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f028cbc1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035ad0 0x205dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f028cbc1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2035eb0 0x20d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f028cbc1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2082630 0x20d8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f028cbc1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x20d78f0 con 0x2035eb0 2023-11-15T03:02:31.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f028cbc1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x205f310 con 0x2035ad0 2023-11-15T03:02:31.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f028cbc1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x208b2e0 con 0x2082630 2023-11-15T03:02:31.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f028cbc1740 10 client.0 ll_register_callbacks cb 0x7f026c017b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:02:31.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f02737fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2035eb0 0x20d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f02737fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2035eb0 0x20d36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40604/0 (socket says 172.21.15.121:40604) 2023-11-15T03:02:31.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f02737fe700 1 -- 172.21.15.121:0/3888123456 learned_addr learned my addr 172.21.15.121:0/3888123456 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:31.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f027bfff700 1 --2- 172.21.15.121:0/3888123456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035ad0 0x205dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f0280db8700 1 --2- 172.21.15.121:0/3888123456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2082630 0x20d8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f02737fe700 1 -- 172.21.15.121:0/3888123456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2082630 msgr2=0x20d8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:31.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f02737fe700 1 --2- 172.21.15.121:0/3888123456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2082630 0x20d8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f02737fe700 1 -- 172.21.15.121:0/3888123456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035ad0 msgr2=0x205dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:31.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f02737fe700 1 --2- 172.21.15.121:0/3888123456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035ad0 0x205dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f02737fe700 1 -- 172.21.15.121:0/3888123456 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1faf730 con 0x2035eb0 2023-11-15T03:02:31.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f02737fe700 1 --2- 172.21.15.121:0/3888123456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2035eb0 0x20d36e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f027c0689c0 tx=0x7f026401e040).ready entity=mon.0 client_cookie=c293f780f66fe65e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:31.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f026a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:02:31.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f026a7fc700 1 -- 172.21.15.121:0/3888123456 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264021040 con 0x2035eb0 2023-11-15T03:02:31.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f028cbc1740 1 -- 172.21.15.121:0/3888123456 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0274048b90 con 0x2035eb0 2023-11-15T03:02:31.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f026a7fc700 1 -- 172.21.15.121:0/3888123456 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0264006240 con 0x2035eb0 2023-11-15T03:02:31.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f026a7fc700 1 -- 172.21.15.121:0/3888123456 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0264006610 con 0x2035eb0 2023-11-15T03:02:31.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f028cbc1740 10 client.6678 fetch_fsmap learned FSMap version 67 2023-11-15T03:02:31.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f028cbc1740 1 -- 172.21.15.121:0/3888123456 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x7f027c0c3280 con 0x2035eb0 2023-11-15T03:02:31.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f026a7fc700 1 -- 172.21.15.121:0/3888123456 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0274048b90 con 0x2035eb0 2023-11-15T03:02:31.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f026a7fc700 1 -- 172.21.15.121:0/3888123456 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 67) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f026400f510 con 0x2035eb0 2023-11-15T03:02:31.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f028cbc1740 10 client.6678 fetch_fsmap finished waiting for FSMap version 67 2023-11-15T03:02:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f028cbc1740 10 client.6678 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-15T03:02:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f028cbc1740 20 client.6678 populate_metadata read hostname 'smithi121' 2023-11-15T03:02:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f028cbc1740 1 --2- 172.21.15.121:0/3888123456 >> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] conn(0x20921f0 0x20925d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f028cbc1740 4 client.6678 mds_command: new command op to 6492 tid=0[{"prefix": "status"}] 2023-11-15T03:02:31.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f028cbc1740 1 -- 172.21.15.121:0/3888123456 --> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1e6b4e0 con 0x20921f0 2023-11-15T03:02:31.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.918+0000 7f027bfff700 1 --2- 172.21.15.121:0/3888123456 >> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] conn(0x20921f0 0x20925d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:31.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f027bfff700 1 --2- 172.21.15.121:0/3888123456 >> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] conn(0x20921f0 0x20925d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=bc706612fcbc08f9 server_cookie=74137b3205adf1c3 in_seq=0 out_seq=0 2023-11-15T03:02:31.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f026a7fc700 10 client.6678 ms_handle_connect on v2:172.21.15.190:6832/914796342 2023-11-15T03:02:31.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f026a7fc700 1 -- 172.21.15.121:0/3888123456 <== mds.0 v2:172.21.15.190:6832/914796342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x1e6b4e0 con 0x20921f0 2023-11-15T03:02:31.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f026a7fc700 10 client.6678 handle_command_reply: tid=0 2023-11-15T03:02:31.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f028cbc1740 1 client.6678 shutdown 2023-11-15T03:02:31.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f028cbc1740 1 -- 172.21.15.121:0/3888123456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2035eb0 msgr2=0x20d36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:31.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f028cbc1740 1 --2- 172.21.15.121:0/3888123456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2035eb0 0x20d36e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f027c0689c0 tx=0x7f026401e040).stop 2023-11-15T03:02:31.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f028cbc1740 1 -- 172.21.15.121:0/3888123456 shutdown_connections 2023-11-15T03:02:31.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f028cbc1740 1 --2- 172.21.15.121:0/3888123456 >> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] conn(0x20921f0 0x20925d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f028cbc1740 1 --2- 172.21.15.121:0/3888123456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2082630 0x20d8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f028cbc1740 1 --2- 172.21.15.121:0/3888123456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2035eb0 0x20d36e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f026a7fc700 0 client.6678 ms_handle_reset on v2:172.21.15.190:6832/914796342 2023-11-15T03:02:31.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f028cbc1740 1 --2- 172.21.15.121:0/3888123456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2035ad0 0x205dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f028cbc1740 1 -- 172.21.15.121:0/3888123456 >> 172.21.15.121:0/3888123456 conn(0x1e14550 msgr2=0x2030f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:31.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f028cbc1740 1 -- 172.21.15.121:0/3888123456 shutdown_connections 2023-11-15T03:02:31.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f028cbc1740 1 -- 172.21.15.121:0/3888123456 wait complete. 2023-11-15T03:02:31.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f028cbc1740 20 client.6678 trim_cache size 0 max 16384 2023-11-15T03:02:31.946 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:02:31.946 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T03:02:31.946 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T03:02:31.947 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 6492, 2023-11-15T03:02:31.947 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T03:02:31.947 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T03:02:31.947 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T03:02:31.948 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 11.217019346000001, 2023-11-15T03:02:31.948 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 67, 2023-11-15T03:02:31.948 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 77, 2023-11-15T03:02:31.948 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 77, 2023-11-15T03:02:31.948 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 16.645213881 2023-11-15T03:02:31.949 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:02:31.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f02815b9700 1 -- 172.21.15.121:0/191997716 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f026005a510 msgr2=0x7f026005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:31.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f02815b9700 1 --2- 172.21.15.121:0/191997716 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f026005a510 0x7f026005c9d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f026c007fd0 tx=0x7f026c007d30).stop 2023-11-15T03:02:31.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f02815b9700 1 -- 172.21.15.121:0/191997716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f027c157d90 msgr2=0x7f027c156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:31.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f02815b9700 1 --2- 172.21.15.121:0/191997716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f027c157d90 0x7f027c156930 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7f0274009e90 tx=0x7f027400b040).stop 2023-11-15T03:02:31.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f02815b9700 1 -- 172.21.15.121:0/191997716 shutdown_connections 2023-11-15T03:02:31.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f02815b9700 1 --2- 172.21.15.121:0/191997716 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f026005a510 0x7f026005c9d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f02815b9700 1 --2- 172.21.15.121:0/191997716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f027c157d90 0x7f027c156930 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f02815b9700 1 --2- 172.21.15.121:0/191997716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f027c0a8750 0x7f027c156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f02815b9700 1 --2- 172.21.15.121:0/191997716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f027c0669b0 0x7f027c14ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:31.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f02815b9700 1 -- 172.21.15.121:0/191997716 >> 172.21.15.121:0/191997716 conn(0x7f027c067a60 msgr2=0x7f027c0bc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:31.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f02815b9700 1 -- 172.21.15.121:0/191997716 shutdown_connections 2023-11-15T03:02:31.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:31.922+0000 7f02815b9700 1 -- 172.21.15.121:0/191997716 wait complete. 2023-11-15T03:02:31.954 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T03:02:32.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.078+0000 7fc485907700 1 Processor -- start 2023-11-15T03:02:32.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.078+0000 7fc485907700 1 -- start start 2023-11-15T03:02:32.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.078+0000 7fc485907700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc48015b450 0x7fc4800657a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:32.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.078+0000 7fc485907700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc4800c2380 0x7fc4800c2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:32.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.078+0000 7fc485907700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4800c2d20 con 0x7fc48015b830 2023-11-15T03:02:32.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.078+0000 7fc485907700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc480064a10 con 0x7fc48015b450 2023-11-15T03:02:32.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.078+0000 7fc485907700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc480064280 con 0x7fc4800c2380 2023-11-15T03:02:32.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.078+0000 7fc484905700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc48015b450 0x7fc4800657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:32.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.078+0000 7fc484905700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc48015b450 0x7fc4800657a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40612/0 (socket says 172.21.15.121:40612) 2023-11-15T03:02:32.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.078+0000 7fc484905700 1 -- 172.21.15.121:0/3340690703 learned_addr learned my addr 172.21.15.121:0/3340690703 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:32.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.078+0000 7fc485106700 1 --2- 172.21.15.121:0/3340690703 >> v2:172.21.15.190:3300/0 conn(0x7fc4800c2380 0x7fc4800c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:32.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.078+0000 7fc484905700 1 -- 172.21.15.121:0/3340690703 >> v2:172.21.15.190:3300/0 conn(0x7fc4800c2380 msgr2=0x7fc4800c2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:32.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.078+0000 7fc484905700 1 --2- 172.21.15.121:0/3340690703 >> v2:172.21.15.190:3300/0 conn(0x7fc4800c2380 0x7fc4800c2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:32.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.078+0000 7fc484905700 1 -- 172.21.15.121:0/3340690703 >> v1:172.21.15.190:6789/0 conn(0x7fc48015b830 legacy=0x7fc480063aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:32.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.078+0000 7fc484905700 1 -- 172.21.15.121:0/3340690703 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4800c1790 con 0x7fc48015b450 2023-11-15T03:02:32.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.082+0000 7fc484905700 1 --2- 172.21.15.121:0/3340690703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc48015b450 0x7fc4800657a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7fc480063870 tx=0x7fc4740090f0).ready entity=mon.0 client_cookie=ed52ce4879828007 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:32.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.082+0000 7fc47f7fe700 1 -- 172.21.15.121:0/3340690703 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47400b600 con 0x7fc48015b450 2023-11-15T03:02:32.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.082+0000 7fc47f7fe700 1 -- 172.21.15.121:0/3340690703 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47400b760 con 0x7fc48015b450 2023-11-15T03:02:32.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.082+0000 7fc47f7fe700 1 -- 172.21.15.121:0/3340690703 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47400bae0 con 0x7fc48015b450 2023-11-15T03:02:32.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.082+0000 7fc485907700 1 -- 172.21.15.121:0/3340690703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc48015b450 msgr2=0x7fc4800657a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:32.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.082+0000 7fc485907700 1 --2- 172.21.15.121:0/3340690703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc48015b450 0x7fc4800657a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7fc480063870 tx=0x7fc4740090f0).stop 2023-11-15T03:02:32.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.082+0000 7fc485907700 1 -- 172.21.15.121:0/3340690703 shutdown_connections 2023-11-15T03:02:32.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.082+0000 7fc485907700 1 --2- 172.21.15.121:0/3340690703 >> v2:172.21.15.190:3300/0 conn(0x7fc4800c2380 0x7fc4800c2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:32.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.082+0000 7fc485907700 1 --2- 172.21.15.121:0/3340690703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc48015b450 0x7fc4800657a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:32.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.082+0000 7fc485907700 1 -- 172.21.15.121:0/3340690703 >> 172.21.15.121:0/3340690703 conn(0x7fc4800677c0 msgr2=0x7fc4800c65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:32.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.082+0000 7fc485907700 1 -- 172.21.15.121:0/3340690703 shutdown_connections 2023-11-15T03:02:32.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.082+0000 7fc485907700 1 -- 172.21.15.121:0/3340690703 wait complete. 2023-11-15T03:02:32.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.082+0000 7fc485907700 1 Processor -- start 2023-11-15T03:02:32.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc485907700 1 -- start start 2023-11-15T03:02:32.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc485907700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4800c2380 0x7fc480150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:32.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc485907700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc48015b450 0x7fc4800bd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:32.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc485907700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc48015b830 0x7fc4800bdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:32.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc485907700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4801510c0 con 0x7fc48015b450 2023-11-15T03:02:32.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc485907700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4800a3c10 con 0x7fc4800c2380 2023-11-15T03:02:32.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc485907700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4800a3ef0 con 0x7fc48015b830 2023-11-15T03:02:32.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc47ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc48015b450 0x7fc4800bd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:32.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc47ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc48015b450 0x7fc4800bd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40614/0 (socket says 172.21.15.121:40614) 2023-11-15T03:02:32.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc47ffff700 1 -- 172.21.15.121:0/3112565950 learned_addr learned my addr 172.21.15.121:0/3112565950 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:32.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc484905700 1 --2- 172.21.15.121:0/3112565950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4800c2380 0x7fc480150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:32.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc485106700 1 --2- 172.21.15.121:0/3112565950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc48015b830 0x7fc4800bdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:32.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc47ffff700 1 -- 172.21.15.121:0/3112565950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc48015b830 msgr2=0x7fc4800bdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:32.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc47ffff700 1 --2- 172.21.15.121:0/3112565950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc48015b830 0x7fc4800bdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:32.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc47ffff700 1 -- 172.21.15.121:0/3112565950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4800c2380 msgr2=0x7fc480150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:32.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc47ffff700 1 --2- 172.21.15.121:0/3112565950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4800c2380 0x7fc480150bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:32.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc47ffff700 1 -- 172.21.15.121:0/3112565950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc474009370 con 0x7fc48015b450 2023-11-15T03:02:32.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc47ffff700 1 --2- 172.21.15.121:0/3112565950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc48015b450 0x7fc4800bd6d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7fc46c00c440 tx=0x7fc46c007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:32.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc47cff9700 1 -- 172.21.15.121:0/3112565950 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c007020 con 0x7fc48015b450 2023-11-15T03:02:32.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc47cff9700 1 -- 172.21.15.121:0/3112565950 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46c007180 con 0x7fc48015b450 2023-11-15T03:02:32.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc485907700 1 -- 172.21.15.121:0/3112565950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4800a41d0 con 0x7fc48015b450 2023-11-15T03:02:32.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc47cff9700 1 -- 172.21.15.121:0/3112565950 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c007520 con 0x7fc48015b450 2023-11-15T03:02:32.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.086+0000 7fc485907700 1 -- 172.21.15.121:0/3112565950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4800a4790 con 0x7fc48015b450 2023-11-15T03:02:32.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.090+0000 7fc485907700 1 -- 172.21.15.121:0/3112565950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc480062720 con 0x7fc48015b450 2023-11-15T03:02:32.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.090+0000 7fc47cff9700 1 -- 172.21.15.121:0/3112565950 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc46c0068b0 con 0x7fc48015b450 2023-11-15T03:02:32.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.090+0000 7fc47cff9700 1 --2- 172.21.15.121:0/3112565950 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc46405a510 0x7fc46405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:32.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.090+0000 7fc47cff9700 1 -- 172.21.15.121:0/3112565950 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc46c0740d0 con 0x7fc48015b450 2023-11-15T03:02:32.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.090+0000 7fc484905700 1 --2- 172.21.15.121:0/3112565950 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc46405a510 0x7fc46405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:32.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.094+0000 7fc484905700 1 --2- 172.21.15.121:0/3112565950 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc46405a510 0x7fc46405c9d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fc474009130 tx=0x7fc474017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:32.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.098+0000 7fc47cff9700 1 -- 172.21.15.121:0/3112565950 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc46c047830 con 0x7fc48015b450 2023-11-15T03:02:32.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.298+0000 7fc485907700 1 -- 172.21.15.121:0/3112565950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc4800c2d20 con 0x7fc48015b450 2023-11-15T03:02:32.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.298+0000 7fc47cff9700 1 -- 172.21.15.121:0/3112565950 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc4800c2d20 con 0x7fc48015b450 2023-11-15T03:02:32.306 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T03:02:32.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.302+0000 7fc485907700 1 -- 172.21.15.121:0/3112565950 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc46405a510 msgr2=0x7fc46405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:32.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.302+0000 7fc485907700 1 --2- 172.21.15.121:0/3112565950 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc46405a510 0x7fc46405c9d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fc474009130 tx=0x7fc474017000).stop 2023-11-15T03:02:32.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.302+0000 7fc485907700 1 -- 172.21.15.121:0/3112565950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc48015b450 msgr2=0x7fc4800bd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:32.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.302+0000 7fc485907700 1 --2- 172.21.15.121:0/3112565950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc48015b450 0x7fc4800bd6d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7fc46c00c440 tx=0x7fc46c007da0).stop 2023-11-15T03:02:32.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.302+0000 7fc485907700 1 -- 172.21.15.121:0/3112565950 shutdown_connections 2023-11-15T03:02:32.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.302+0000 7fc485907700 1 --2- 172.21.15.121:0/3112565950 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc46405a510 0x7fc46405c9d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:32.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.302+0000 7fc485907700 1 --2- 172.21.15.121:0/3112565950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc48015b830 0x7fc4800bdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:32.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.302+0000 7fc485907700 1 --2- 172.21.15.121:0/3112565950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc48015b450 0x7fc4800bd6d0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:32.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.302+0000 7fc485907700 1 --2- 172.21.15.121:0/3112565950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4800c2380 0x7fc480150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:32.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.302+0000 7fc485907700 1 -- 172.21.15.121:0/3112565950 >> 172.21.15.121:0/3112565950 conn(0x7fc4800677c0 msgr2=0x7fc480063aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:32.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.302+0000 7fc485907700 1 -- 172.21.15.121:0/3112565950 shutdown_connections 2023-11-15T03:02:32.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:32.302+0000 7fc485907700 1 -- 172.21.15.121:0/3112565950 wait complete. 2023-11-15T03:02:32.328 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T03:02:32.329 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T03:02:32.329 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T03:02:32.330 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T03:02:32.330 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T03:02:32.331 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T03:02:32.331 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T03:02:32.332 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T03:02:32.332 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:02:32.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:02:32.333 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T03:02:32.353 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T03:02:32.353 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T03:02:32.353 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T03:02:32.354 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:02:32.354 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T03:02:32.354 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:02:32.354 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:02:32.355 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:02:32.355 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T03:02:32.355 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:02:32.355 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:02:32.355 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T03:02:32.355 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:02:32.356 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:02:32.356 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T03:02:32.356 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:02:32.356 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T03:02:32.356 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42087sec preferred_lft 42087sec 2023-11-15T03:02:32.357 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T03:02:32.357 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:02:32.357 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T03:02:32.357 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:02:32.357 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T03:02:32.358 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:02:32.358 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T03:02:32.358 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T03:02:32.358 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T03:02:32.358 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:02:32.359 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T03:02:32.359 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:02:32.359 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:02:32.365 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T03:02:32.365 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T03:02:32.365 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:02:32.365 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:02:32.366 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T03:02:32.366 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T03:02:32.366 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:02:32.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:02:32.368 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T03:02:32.449 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T03:02:32.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:02:32.452 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T03:02:32.516 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T03:02:32.517 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T03:02:32.517 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T03:02:32.517 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T03:02:32.517 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T03:02:32.518 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T03:02:32.518 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T03:02:32.518 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T03:02:32.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:02:32.519 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T03:02:32.520 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T03:02:32.520 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T03:02:32.520 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T03:02:32.521 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T03:02:32.521 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T03:02:32.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:02:32.652 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T03:02:32.670 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T03:02:32.670 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T03:02:32.671 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:02:32.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:02:32.672 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T03:02:32.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:02:32.749 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T03:02:32.809 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T03:02:32.809 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:02:32.928 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:02:32.929 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:02:32.953 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:02:32.953 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:02:32.958 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T03:02:32.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:02:32.959 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T03:02:32.990 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:02:33.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.190+0000 7fb252fa9700 1 Processor -- start 2023-11-15T03:02:33.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.190+0000 7fb252fa9700 1 -- start start 2023-11-15T03:02:33.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb252fa9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb24c153bb0 0x7fb24c0a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:33.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb252fa9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb24c0aa020 0x7fb24c067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:33.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb252fa9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb24c0cb5c0 con 0x7fb24c067f50 2023-11-15T03:02:33.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb252fa9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb24c0aa6f0 con 0x7fb24c153bb0 2023-11-15T03:02:33.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb252fa9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c0c5900 con 0x7fb24c0aa020 2023-11-15T03:02:33.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb250d45700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb24c153bb0 0x7fb24c0a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:33.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb250d45700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb24c153bb0 0x7fb24c0a9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40618/0 (socket says 172.21.15.121:40618) 2023-11-15T03:02:33.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb250d45700 1 -- 172.21.15.121:0/1235764080 learned_addr learned my addr 172.21.15.121:0/1235764080 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:33.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb251546700 1 --1- 172.21.15.121:0/1235764080 >> v1:172.21.15.121:6789/0 conn(0x7fb24c067f50 0x7fb24c1578a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:41698/0 (socket says 172.21.15.121:41698) 2023-11-15T03:02:33.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb24bfff700 1 --2- 172.21.15.121:0/1235764080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb24c0aa020 0x7fb24c067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:33.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb250d45700 1 -- 172.21.15.121:0/1235764080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb24c0aa020 msgr2=0x7fb24c067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:33.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb250d45700 1 --2- 172.21.15.121:0/1235764080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb24c0aa020 0x7fb24c067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:33.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb250d45700 1 -- 172.21.15.121:0/1235764080 >> v1:172.21.15.121:6789/0 conn(0x7fb24c067f50 legacy=0x7fb24c1578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:33.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb250d45700 1 -- 172.21.15.121:0/1235764080 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c0683e0 con 0x7fb24c153bb0 2023-11-15T03:02:33.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb250d45700 1 --2- 172.21.15.121:0/1235764080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb24c153bb0 0x7fb24c0a9b30 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7fb240006fd0 tx=0x7fb2400090f0).ready entity=mon.0 client_cookie=716911560c3fc16f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:33.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb24b7fe700 1 -- 172.21.15.121:0/1235764080 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24000b580 con 0x7fb24c153bb0 2023-11-15T03:02:33.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb24b7fe700 1 -- 172.21.15.121:0/1235764080 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24000b6e0 con 0x7fb24c153bb0 2023-11-15T03:02:33.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb24b7fe700 1 -- 172.21.15.121:0/1235764080 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24000ba60 con 0x7fb24c153bb0 2023-11-15T03:02:33.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb252fa9700 1 -- 172.21.15.121:0/1235764080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb24c153bb0 msgr2=0x7fb24c0a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:33.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb252fa9700 1 --2- 172.21.15.121:0/1235764080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb24c153bb0 0x7fb24c0a9b30 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7fb240006fd0 tx=0x7fb2400090f0).stop 2023-11-15T03:02:33.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb252fa9700 1 -- 172.21.15.121:0/1235764080 shutdown_connections 2023-11-15T03:02:33.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb252fa9700 1 --2- 172.21.15.121:0/1235764080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb24c0aa020 0x7fb24c067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:33.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb252fa9700 1 --2- 172.21.15.121:0/1235764080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb24c153bb0 0x7fb24c0a9b30 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:33.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb252fa9700 1 -- 172.21.15.121:0/1235764080 >> 172.21.15.121:0/1235764080 conn(0x7fb24c058c50 msgr2=0x7fb24c059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:33.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb252fa9700 1 -- 172.21.15.121:0/1235764080 shutdown_connections 2023-11-15T03:02:33.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.194+0000 7fb252fa9700 1 -- 172.21.15.121:0/1235764080 wait complete. 2023-11-15T03:02:33.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.198+0000 7fb252fa9700 1 Processor -- start 2023-11-15T03:02:33.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.198+0000 7fb252fa9700 1 -- start start 2023-11-15T03:02:33.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.198+0000 7fb252fa9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb24c067f50 0x7fb24c14e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:33.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.198+0000 7fb252fa9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb24c0aa020 0x7fb24c0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:33.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.198+0000 7fb252fa9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb24c153bb0 0x7fb24c14f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:33.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.198+0000 7fb252fa9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c153540 con 0x7fb24c0aa020 2023-11-15T03:02:33.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.198+0000 7fb252fa9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c155e30 con 0x7fb24c153bb0 2023-11-15T03:02:33.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.198+0000 7fb252fa9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c154030 con 0x7fb24c067f50 2023-11-15T03:02:33.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.198+0000 7fb24bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb24c0aa020 0x7fb24c0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:33.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.198+0000 7fb24bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb24c0aa020 0x7fb24c0ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40624/0 (socket says 172.21.15.121:40624) 2023-11-15T03:02:33.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.198+0000 7fb24bfff700 1 -- 172.21.15.121:0/1151505568 learned_addr learned my addr 172.21.15.121:0/1151505568 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:33.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.198+0000 7fb250d45700 1 --2- 172.21.15.121:0/1151505568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb24c067f50 0x7fb24c14e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:33.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.198+0000 7fb251546700 1 --2- 172.21.15.121:0/1151505568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb24c153bb0 0x7fb24c14f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:33.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.202+0000 7fb24bfff700 1 -- 172.21.15.121:0/1151505568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb24c067f50 msgr2=0x7fb24c14e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:33.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.202+0000 7fb24bfff700 1 --2- 172.21.15.121:0/1151505568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb24c067f50 0x7fb24c14e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:33.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.202+0000 7fb24bfff700 1 -- 172.21.15.121:0/1151505568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb24c153bb0 msgr2=0x7fb24c14f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:33.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.202+0000 7fb24bfff700 1 --2- 172.21.15.121:0/1151505568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb24c153bb0 0x7fb24c14f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:33.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.202+0000 7fb24bfff700 1 -- 172.21.15.121:0/1151505568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb240009370 con 0x7fb24c0aa020 2023-11-15T03:02:33.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.202+0000 7fb24bfff700 1 --2- 172.21.15.121:0/1151505568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb24c0aa020 0x7fb24c0ac8e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7fb23800e930 tx=0x7fb238009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:33.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.202+0000 7fb248ff9700 1 -- 172.21.15.121:0/1151505568 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb238012600 con 0x7fb24c0aa020 2023-11-15T03:02:33.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.202+0000 7fb248ff9700 1 -- 172.21.15.121:0/1151505568 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb238012760 con 0x7fb24c0aa020 2023-11-15T03:02:33.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.202+0000 7fb248ff9700 1 -- 172.21.15.121:0/1151505568 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb238012ae0 con 0x7fb24c0aa020 2023-11-15T03:02:33.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.202+0000 7fb252fa9700 1 -- 172.21.15.121:0/1151505568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb24c155870 con 0x7fb24c0aa020 2023-11-15T03:02:33.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.202+0000 7fb252fa9700 1 -- 172.21.15.121:0/1151505568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb24c1542d0 con 0x7fb24c0aa020 2023-11-15T03:02:33.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.202+0000 7fb252fa9700 1 -- 172.21.15.121:0/1151505568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb24c06c150 con 0x7fb24c0aa020 2023-11-15T03:02:33.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.206+0000 7fb248ff9700 1 -- 172.21.15.121:0/1151505568 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb23801b400 con 0x7fb24c0aa020 2023-11-15T03:02:33.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.206+0000 7fb248ff9700 1 --2- 172.21.15.121:0/1151505568 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb23005a4e0 0x7fb23005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:33.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.206+0000 7fb248ff9700 1 -- 172.21.15.121:0/1151505568 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb238074ca0 con 0x7fb24c0aa020 2023-11-15T03:02:33.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.206+0000 7fb250d45700 1 --2- 172.21.15.121:0/1151505568 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb23005a4e0 0x7fb23005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:33.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.206+0000 7fb250d45700 1 --2- 172.21.15.121:0/1151505568 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb23005a4e0 0x7fb23005c9a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7fb240009130 tx=0x7fb240017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:33.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.214+0000 7fb248ff9700 1 -- 172.21.15.121:0/1151505568 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb238048360 con 0x7fb24c0aa020 2023-11-15T03:02:33.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.402+0000 7fb252fa9700 1 -- 172.21.15.121:0/1151505568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb24c068ae0 con 0x7fb24c0aa020 2023-11-15T03:02:33.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.402+0000 7fb248ff9700 1 -- 172.21.15.121:0/1151505568 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4497 (secure 0 0 0) 0x7fb23804b980 con 0x7fb24c0aa020 2023-11-15T03:02:33.412 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:33.412 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":67,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1508681155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1508681155},{"type":"v1","addr":"172.21.15.121:6837","nonce":1508681155}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6385,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3559276534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3559276534},{"type":"v1","addr":"172.21.15.190:6835","nonce":3559276534}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6486,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/834528709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":834528709},{"type":"v1","addr":"172.21.15.121:6836","nonce":834528709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:02:20.703078+0000","modified":"2023-11-15T03:02:23.967255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6492},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6492":{"gid":6492,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6833/914796342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":914796342},{"type":"v1","addr":"172.21.15.190:6833","nonce":914796342}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-15T03:02:33.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.406+0000 7fb252fa9700 1 -- 172.21.15.121:0/1151505568 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb23005a4e0 msgr2=0x7fb23005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:33.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.406+0000 7fb252fa9700 1 --2- 172.21.15.121:0/1151505568 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb23005a4e0 0x7fb23005c9a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7fb240009130 tx=0x7fb240017000).stop 2023-11-15T03:02:33.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.406+0000 7fb252fa9700 1 -- 172.21.15.121:0/1151505568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb24c0aa020 msgr2=0x7fb24c0ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:33.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.406+0000 7fb252fa9700 1 --2- 172.21.15.121:0/1151505568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb24c0aa020 0x7fb24c0ac8e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7fb23800e930 tx=0x7fb238009060).stop 2023-11-15T03:02:33.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.410+0000 7fb252fa9700 1 -- 172.21.15.121:0/1151505568 shutdown_connections 2023-11-15T03:02:33.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.410+0000 7fb252fa9700 1 --2- 172.21.15.121:0/1151505568 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb23005a4e0 0x7fb23005c9a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:33.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.410+0000 7fb252fa9700 1 --2- 172.21.15.121:0/1151505568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb24c153bb0 0x7fb24c14f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:33.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.410+0000 7fb252fa9700 1 --2- 172.21.15.121:0/1151505568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb24c0aa020 0x7fb24c0ac8e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:33.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.410+0000 7fb252fa9700 1 --2- 172.21.15.121:0/1151505568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb24c067f50 0x7fb24c14e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:33.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.410+0000 7fb252fa9700 1 -- 172.21.15.121:0/1151505568 >> 172.21.15.121:0/1151505568 conn(0x7fb24c058c50 msgr2=0x7fb24c0c0670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:33.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.410+0000 7fb252fa9700 1 -- 172.21.15.121:0/1151505568 shutdown_connections 2023-11-15T03:02:33.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.410+0000 7fb252fa9700 1 -- 172.21.15.121:0/1151505568 wait complete. 2023-11-15T03:02:33.419 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 67 2023-11-15T03:02:33.438 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 5:0 true 2023-11-15T03:02:33.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.594+0000 7fd2e8d47700 1 Processor -- start 2023-11-15T03:02:33.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.594+0000 7fd2e8d47700 1 -- start start 2023-11-15T03:02:33.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.598+0000 7fd2e8d47700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2e40acd00 0x7fd2e40ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:33.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.598+0000 7fd2e8d47700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd2e4066330 0x7fd2e4153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:33.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.598+0000 7fd2e8d47700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e4065ea0 con 0x7fd2e40c5c70 2023-11-15T03:02:33.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.598+0000 7fd2e8d47700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e4066bb0 con 0x7fd2e4066330 2023-11-15T03:02:33.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.598+0000 7fd2e8d47700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e4065890 con 0x7fd2e40acd00 2023-11-15T03:02:33.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.598+0000 7fd2e3fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd2e4066330 0x7fd2e4153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:33.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.598+0000 7fd2e37fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2e40acd00 0x7fd2e40ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:33.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.598+0000 7fd2e3fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd2e4066330 0x7fd2e4153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:36834/0 (socket says 172.21.15.121:36834) 2023-11-15T03:02:33.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.598+0000 7fd2e3fff700 1 -- 172.21.15.121:0/1886346925 learned_addr learned my addr 172.21.15.121:0/1886346925 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:33.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.598+0000 7fd2e3fff700 1 -- 172.21.15.121:0/1886346925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2e40acd00 msgr2=0x7fd2e40ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:33.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.598+0000 7fd2e3fff700 1 --2- 172.21.15.121:0/1886346925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2e40acd00 0x7fd2e40ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:33.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.598+0000 7fd2e3fff700 1 -- 172.21.15.121:0/1886346925 >> v1:172.21.15.190:6789/0 conn(0x7fd2e40c5c70 legacy=0x7fd2e41536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:33.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.598+0000 7fd2e3fff700 1 -- 172.21.15.121:0/1886346925 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e40c64a0 con 0x7fd2e4066330 2023-11-15T03:02:33.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.598+0000 7fd2e3fff700 1 --2- 172.21.15.121:0/1886346925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2e4066330 0x7fd2e4153dd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fd2dc009f20 tx=0x7fd2dc009840).ready entity=mon.1 client_cookie=a6ac8ab178dba3f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:33.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.598+0000 7fd2e27fc700 1 -- 172.21.15.121:0/1886346925 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc00e070 con 0x7fd2e4066330 2023-11-15T03:02:33.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.598+0000 7fd2e27fc700 1 -- 172.21.15.121:0/1886346925 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2dc00b9d0 con 0x7fd2e4066330 2023-11-15T03:02:33.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.598+0000 7fd2e27fc700 1 -- 172.21.15.121:0/1886346925 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc00bd50 con 0x7fd2e4066330 2023-11-15T03:02:33.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.602+0000 7fd2e8d47700 1 -- 172.21.15.121:0/1886346925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2e4066330 msgr2=0x7fd2e4153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:33.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.602+0000 7fd2e8d47700 1 --2- 172.21.15.121:0/1886346925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2e4066330 0x7fd2e4153dd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fd2dc009f20 tx=0x7fd2dc009840).stop 2023-11-15T03:02:33.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.602+0000 7fd2e8d47700 1 -- 172.21.15.121:0/1886346925 shutdown_connections 2023-11-15T03:02:33.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.602+0000 7fd2e8d47700 1 --2- 172.21.15.121:0/1886346925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2e4066330 0x7fd2e4153dd0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:33.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.602+0000 7fd2e8d47700 1 --2- 172.21.15.121:0/1886346925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2e40acd00 0x7fd2e40ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:33.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.602+0000 7fd2e8d47700 1 -- 172.21.15.121:0/1886346925 >> 172.21.15.121:0/1886346925 conn(0x7fd2e4063fc0 msgr2=0x7fd2e4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:33.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.602+0000 7fd2e8d47700 1 -- 172.21.15.121:0/1886346925 shutdown_connections 2023-11-15T03:02:33.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.602+0000 7fd2e8d47700 1 -- 172.21.15.121:0/1886346925 wait complete. 2023-11-15T03:02:33.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.602+0000 7fd2e8d47700 1 Processor -- start 2023-11-15T03:02:33.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.602+0000 7fd2e8d47700 1 -- start start 2023-11-15T03:02:33.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e8d47700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2e4066330 0x7fd2e4166e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:33.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e8d47700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2e40acd00 0x7fd2e4164b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:33.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e8d47700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2e40c5c70 0x7fd2e4165030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:33.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e8d47700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e4165520 con 0x7fd2e4066330 2023-11-15T03:02:33.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e8d47700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e40aeca0 con 0x7fd2e40acd00 2023-11-15T03:02:33.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e8d47700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e4154ed0 con 0x7fd2e40c5c70 2023-11-15T03:02:33.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2e4066330 0x7fd2e4166e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:33.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2e4066330 0x7fd2e4166e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40638/0 (socket says 172.21.15.121:40638) 2023-11-15T03:02:33.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e37fe700 1 -- 172.21.15.121:0/1507867124 learned_addr learned my addr 172.21.15.121:0/1507867124 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:33.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e3fff700 1 --2- 172.21.15.121:0/1507867124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2e40c5c70 0x7fd2e4165030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:33.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e2ffd700 1 --2- 172.21.15.121:0/1507867124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2e40acd00 0x7fd2e4164b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:33.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e37fe700 1 -- 172.21.15.121:0/1507867124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2e40c5c70 msgr2=0x7fd2e4165030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:33.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e37fe700 1 --2- 172.21.15.121:0/1507867124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2e40c5c70 0x7fd2e4165030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:33.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e37fe700 1 -- 172.21.15.121:0/1507867124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2e40acd00 msgr2=0x7fd2e4164b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:33.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e37fe700 1 --2- 172.21.15.121:0/1507867124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2e40acd00 0x7fd2e4164b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:33.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e37fe700 1 -- 172.21.15.121:0/1507867124 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2dc009ae0 con 0x7fd2e4066330 2023-11-15T03:02:33.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e2ffd700 1 --2- 172.21.15.121:0/1507867124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2e40acd00 0x7fd2e4164b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:33.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e37fe700 1 --2- 172.21.15.121:0/1507867124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2e4066330 0x7fd2e4166e30 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7fd2d400e8d0 tx=0x7fd2d4006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:33.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2cb7fe700 1 -- 172.21.15.121:0/1507867124 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4009450 con 0x7fd2e4066330 2023-11-15T03:02:33.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2cb7fe700 1 -- 172.21.15.121:0/1507867124 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d4012600 con 0x7fd2e4066330 2023-11-15T03:02:33.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e8d47700 1 -- 172.21.15.121:0/1507867124 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e40aef40 con 0x7fd2e4066330 2023-11-15T03:02:33.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2cb7fe700 1 -- 172.21.15.121:0/1507867124 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4012980 con 0x7fd2e4066330 2023-11-15T03:02:33.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.606+0000 7fd2e8d47700 1 -- 172.21.15.121:0/1507867124 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e40af2d0 con 0x7fd2e4066330 2023-11-15T03:02:33.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.610+0000 7fd2e8d47700 1 -- 172.21.15.121:0/1507867124 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e40af460 con 0x7fd2e4066330 2023-11-15T03:02:33.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.610+0000 7fd2cb7fe700 1 -- 172.21.15.121:0/1507867124 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd2d4012cf0 con 0x7fd2e4066330 2023-11-15T03:02:33.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.610+0000 7fd2cb7fe700 1 --2- 172.21.15.121:0/1507867124 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd2d005a510 0x7fd2d005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:33.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.610+0000 7fd2cb7fe700 1 -- 172.21.15.121:0/1507867124 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd2d4074c80 con 0x7fd2e4066330 2023-11-15T03:02:33.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.614+0000 7fd2e2ffd700 1 --2- 172.21.15.121:0/1507867124 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd2d005a510 0x7fd2d005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:33.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.614+0000 7fd2e2ffd700 1 --2- 172.21.15.121:0/1507867124 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd2d005a510 0x7fd2d005c9d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fd2e41507e0 tx=0x7fd2d8009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:33.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.618+0000 7fd2cb7fe700 1 -- 172.21.15.121:0/1507867124 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd2e40af460 con 0x7fd2e4066330 2023-11-15T03:02:33.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:33.834+0000 7fd2e8d47700 1 -- 172.21.15.121:0/1507867124 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "5:0", "val": "true"} v 0) v1 -- 0x7fd2e40af460 con 0x7fd2e4066330 2023-11-15T03:02:34.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.126+0000 7fd2cb7fe700 1 -- 172.21.15.121:0/1507867124 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "5:0", "val": "true"}]=0 freezing mds.6492 v68) v1 ==== 112+0+0 (secure 0 0 0) 0x7fd2d4016070 con 0x7fd2e4066330 2023-11-15T03:02:34.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.130+0000 7fd2e8d47700 1 -- 172.21.15.121:0/1507867124 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd2d005a510 msgr2=0x7fd2d005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:34.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.130+0000 7fd2e8d47700 1 --2- 172.21.15.121:0/1507867124 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd2d005a510 0x7fd2d005c9d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fd2e41507e0 tx=0x7fd2d8009040).stop 2023-11-15T03:02:34.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.130+0000 7fd2e8d47700 1 -- 172.21.15.121:0/1507867124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2e4066330 msgr2=0x7fd2e4166e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:34.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.130+0000 7fd2e8d47700 1 --2- 172.21.15.121:0/1507867124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2e4066330 0x7fd2e4166e30 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7fd2d400e8d0 tx=0x7fd2d4006f40).stop 2023-11-15T03:02:34.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.130+0000 7fd2e8d47700 1 -- 172.21.15.121:0/1507867124 shutdown_connections 2023-11-15T03:02:34.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.130+0000 7fd2e8d47700 1 --2- 172.21.15.121:0/1507867124 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd2d005a510 0x7fd2d005c9d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.130+0000 7fd2e8d47700 1 --2- 172.21.15.121:0/1507867124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2e40c5c70 0x7fd2e4165030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.130+0000 7fd2e8d47700 1 --2- 172.21.15.121:0/1507867124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2e40acd00 0x7fd2e4164b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.130+0000 7fd2e8d47700 1 --2- 172.21.15.121:0/1507867124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2e4066330 0x7fd2e4166e30 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.130+0000 7fd2e8d47700 1 -- 172.21.15.121:0/1507867124 >> 172.21.15.121:0/1507867124 conn(0x7fd2e4063fc0 msgr2=0x7fd2e40c9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:34.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.130+0000 7fd2e8d47700 1 -- 172.21.15.121:0/1507867124 shutdown_connections 2023-11-15T03:02:34.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.130+0000 7fd2e8d47700 1 -- 172.21.15.121:0/1507867124 wait complete. 2023-11-15T03:02:34.142 INFO:teuthology.orchestra.run.smithi121.stderr:freezing mds.6492 2023-11-15T03:02:34.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:02:34.159 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'mkdir -p a/{b,c}/d') 2023-11-15T03:02:34.212 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2023-11-15T03:02:34.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.370+0000 7f07b41a9700 1 Processor -- start 2023-11-15T03:02:34.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.370+0000 7f07b41a9700 1 -- start start 2023-11-15T03:02:34.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.370+0000 7f07b41a9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f07ac067020 0x7f07ac1582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.370+0000 7f07b41a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ac068170 0x7f07ac0bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.370+0000 7f07b41a9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07ac062da0 con 0x7f07ac157ed0 2023-11-15T03:02:34.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.370+0000 7f07b41a9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07ac0a93a0 con 0x7f07ac067020 2023-11-15T03:02:34.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.370+0000 7f07b41a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac062190 con 0x7f07ac068170 2023-11-15T03:02:34.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.370+0000 7f07b1f45700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f07ac157ed0 0x7f07ac067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:41702/0 (socket says 172.21.15.121:41702) 2023-11-15T03:02:34.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.370+0000 7f07b1f45700 1 -- 172.21.15.121:0/3103049856 learned_addr learned my addr 172.21.15.121:0/3103049856 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:34.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.370+0000 7f07b1744700 1 --2- 172.21.15.121:0/3103049856 >> v2:172.21.15.190:3300/0 conn(0x7f07ac067020 0x7f07ac1582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.370+0000 7f07b2746700 1 --2- 172.21.15.121:0/3103049856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ac068170 0x7f07ac0bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.370+0000 7f07b0f43700 1 -- 172.21.15.121:0/3103049856 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801546871 0 0) 0x7f07ac062da0 con 0x7f07ac157ed0 2023-11-15T03:02:34.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.370+0000 7f07b0f43700 1 -- 172.21.15.121:0/3103049856 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07980033f0 con 0x7f07ac157ed0 2023-11-15T03:02:34.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.370+0000 7f07b2746700 1 -- 172.21.15.121:0/3103049856 >> v2:172.21.15.190:3300/0 conn(0x7f07ac067020 msgr2=0x7f07ac1582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:34.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.370+0000 7f07b2746700 1 --2- 172.21.15.121:0/3103049856 >> v2:172.21.15.190:3300/0 conn(0x7f07ac067020 0x7f07ac1582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.370+0000 7f07b2746700 1 -- 172.21.15.121:0/3103049856 >> v1:172.21.15.121:6789/0 conn(0x7f07ac157ed0 legacy=0x7f07ac067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:34.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.370+0000 7f07b2746700 1 -- 172.21.15.121:0/3103049856 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ac062470 con 0x7f07ac068170 2023-11-15T03:02:34.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.374+0000 7f07b2746700 1 --2- 172.21.15.121:0/3103049856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ac068170 0x7f07ac0bc0d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f07a4009e60 tx=0x7f07a4009780).ready entity=mon.2 client_cookie=8c245efb9f20827e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:34.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.374+0000 7f07b0f43700 1 -- 172.21.15.121:0/3103049856 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a400e070 con 0x7f07ac068170 2023-11-15T03:02:34.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.374+0000 7f07b0f43700 1 -- 172.21.15.121:0/3103049856 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a400bab0 con 0x7f07ac068170 2023-11-15T03:02:34.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.374+0000 7f07b0f43700 1 -- 172.21.15.121:0/3103049856 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a400be30 con 0x7f07ac068170 2023-11-15T03:02:34.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.374+0000 7f07b41a9700 1 -- 172.21.15.121:0/3103049856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ac068170 msgr2=0x7f07ac0bc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:34.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.374+0000 7f07b41a9700 1 --2- 172.21.15.121:0/3103049856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ac068170 0x7f07ac0bc0d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f07a4009e60 tx=0x7f07a4009780).stop 2023-11-15T03:02:34.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.374+0000 7f07b41a9700 1 -- 172.21.15.121:0/3103049856 shutdown_connections 2023-11-15T03:02:34.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.374+0000 7f07b41a9700 1 --2- 172.21.15.121:0/3103049856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ac068170 0x7f07ac0bc0d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.374+0000 7f07b41a9700 1 --2- 172.21.15.121:0/3103049856 >> v2:172.21.15.190:3300/0 conn(0x7f07ac067020 0x7f07ac1582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.374+0000 7f07b41a9700 1 -- 172.21.15.121:0/3103049856 >> 172.21.15.121:0/3103049856 conn(0x7f07ac0591c0 msgr2=0x7f07ac0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:34.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.374+0000 7f07b41a9700 1 -- 172.21.15.121:0/3103049856 shutdown_connections 2023-11-15T03:02:34.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.374+0000 7f07b41a9700 1 -- 172.21.15.121:0/3103049856 wait complete. 2023-11-15T03:02:34.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.378+0000 7f07b41a9700 1 Processor -- start 2023-11-15T03:02:34.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.378+0000 7f07b41a9700 1 -- start start 2023-11-15T03:02:34.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07b41a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07ac067020 0x7f07ac148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07b41a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07ac068170 0x7f07ac0a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07b41a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ac157ed0 0x7f07ac0a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07b41a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac155310 con 0x7f07ac067020 2023-11-15T03:02:34.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07b41a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac0a1510 con 0x7f07ac068170 2023-11-15T03:02:34.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07b41a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac0a0900 con 0x7f07ac157ed0 2023-11-15T03:02:34.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07b1f45700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07ac067020 0x7f07ac148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07b1f45700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07ac067020 0x7f07ac148550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40644/0 (socket says 172.21.15.121:40644) 2023-11-15T03:02:34.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07b1f45700 1 -- 172.21.15.121:0/349755504 learned_addr learned my addr 172.21.15.121:0/349755504 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:34.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07b2746700 1 --2- 172.21.15.121:0/349755504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ac157ed0 0x7f07ac0a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07b1744700 1 --2- 172.21.15.121:0/349755504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07ac068170 0x7f07ac0a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07b1f45700 1 -- 172.21.15.121:0/349755504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ac157ed0 msgr2=0x7f07ac0a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:34.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07b1f45700 1 --2- 172.21.15.121:0/349755504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ac157ed0 0x7f07ac0a2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07b1f45700 1 -- 172.21.15.121:0/349755504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07ac068170 msgr2=0x7f07ac0a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:34.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07b1f45700 1 --2- 172.21.15.121:0/349755504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07ac068170 0x7f07ac0a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07b1f45700 1 -- 172.21.15.121:0/349755504 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a4009a20 con 0x7f07ac067020 2023-11-15T03:02:34.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07b1f45700 1 --2- 172.21.15.121:0/349755504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07ac067020 0x7f07ac148550 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f07a800c420 tx=0x7f07a8007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:34.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07a1ffb700 1 -- 172.21.15.121:0/349755504 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a80072e0 con 0x7f07ac067020 2023-11-15T03:02:34.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07a1ffb700 1 -- 172.21.15.121:0/349755504 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a8007440 con 0x7f07ac067020 2023-11-15T03:02:34.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07a1ffb700 1 -- 172.21.15.121:0/349755504 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a80057a0 con 0x7f07ac067020 2023-11-15T03:02:34.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07b41a9700 1 -- 172.21.15.121:0/349755504 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ac0a0be0 con 0x7f07ac067020 2023-11-15T03:02:34.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.382+0000 7f07b41a9700 1 -- 172.21.15.121:0/349755504 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ac0a11a0 con 0x7f07ac067020 2023-11-15T03:02:34.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.386+0000 7f07a1ffb700 1 -- 172.21.15.121:0/349755504 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f07a8005bd0 con 0x7f07ac067020 2023-11-15T03:02:34.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.386+0000 7f07a1ffb700 1 --2- 172.21.15.121:0/349755504 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f079005a540 0x7f079005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.386+0000 7f07a1ffb700 1 -- 172.21.15.121:0/349755504 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f07a8073fb0 con 0x7f07ac067020 2023-11-15T03:02:34.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.386+0000 7f07b1744700 1 --2- 172.21.15.121:0/349755504 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f079005a540 0x7f079005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.386+0000 7f07b1744700 1 --2- 172.21.15.121:0/349755504 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f079005a540 0x7f079005ca00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f07ac0bfa80 tx=0x7f079c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:34.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.390+0000 7f07be54e740 1 Processor -- start 2023-11-15T03:02:34.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.390+0000 7f07be54e740 1 -- start start 2023-11-15T03:02:34.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.390+0000 7f07be54e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1441a70 0x13fe180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.390+0000 7f07be54e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13fe990 0x1441e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.390+0000 7f07be54e740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x13f8150 0x13f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.390+0000 7f07be54e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x135b540 con 0x13fe990 2023-11-15T03:02:34.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.390+0000 7f07be54e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x131ee90 con 0x1441a70 2023-11-15T03:02:34.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.390+0000 7f07be54e740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1324d90 con 0x13f8150 2023-11-15T03:02:34.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07b1744700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13fe990 0x1441e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07b1744700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13fe990 0x1441e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40646/0 (socket says 172.21.15.121:40646) 2023-11-15T03:02:34.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07b1744700 1 -- 172.21.15.121:0/3099778573 learned_addr learned my addr 172.21.15.121:0/3099778573 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:34.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07b1f45700 1 --2- 172.21.15.121:0/3099778573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1441a70 0x13fe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07b2746700 1 --2- 172.21.15.121:0/3099778573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x13f8150 0x13f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07b1744700 1 -- 172.21.15.121:0/3099778573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x13f8150 msgr2=0x13f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:34.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07b1744700 1 --2- 172.21.15.121:0/3099778573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x13f8150 0x13f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07b1744700 1 -- 172.21.15.121:0/3099778573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1441a70 msgr2=0x13fe180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:34.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07b1744700 1 --2- 172.21.15.121:0/3099778573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1441a70 0x13fe180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07b1744700 1 -- 172.21.15.121:0/3099778573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x13894c0 con 0x13fe990 2023-11-15T03:02:34.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07b1744700 1 --2- 172.21.15.121:0/3099778573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13fe990 0x1441e50 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f079c011b30 tx=0x7f079c016000).ready entity=mon.0 client_cookie=5a57a83ef870427a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:34.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f078effd700 1 -- 172.21.15.121:0/3099778573 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c015910 con 0x13fe990 2023-11-15T03:02:34.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f078effd700 1 -- 172.21.15.121:0/3099778573 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c015a70 con 0x13fe990 2023-11-15T03:02:34.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f078effd700 1 -- 172.21.15.121:0/3099778573 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c015df0 con 0x13fe990 2023-11-15T03:02:34.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07be54e740 1 -- 172.21.15.121:0/3099778573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13fe990 msgr2=0x1441e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:34.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07be54e740 1 --2- 172.21.15.121:0/3099778573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13fe990 0x1441e50 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f079c011b30 tx=0x7f079c016000).stop 2023-11-15T03:02:34.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07be54e740 1 -- 172.21.15.121:0/3099778573 shutdown_connections 2023-11-15T03:02:34.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07be54e740 1 --2- 172.21.15.121:0/3099778573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x13f8150 0x13f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07be54e740 1 --2- 172.21.15.121:0/3099778573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13fe990 0x1441e50 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07be54e740 1 --2- 172.21.15.121:0/3099778573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1441a70 0x13fe180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07be54e740 1 -- 172.21.15.121:0/3099778573 >> 172.21.15.121:0/3099778573 conn(0x11dd680 msgr2=0x13fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:34.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07be54e740 1 -- 172.21.15.121:0/3099778573 shutdown_connections 2023-11-15T03:02:34.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07be54e740 1 -- 172.21.15.121:0/3099778573 wait complete. 2023-11-15T03:02:34.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.394+0000 7f07be54e740 1 -- start start 2023-11-15T03:02:34.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.398+0000 7f07be54e740 1 Processor -- start 2023-11-15T03:02:34.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.398+0000 7f07be54e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13f8150 0x1428ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.398+0000 7f07be54e740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x13fe990 0x141af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.398+0000 7f07be54e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1441a70 0x141b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.398+0000 7f07be54e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1429720 con 0x13f8150 2023-11-15T03:02:34.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.398+0000 7f07be54e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1429a00 con 0x1441a70 2023-11-15T03:02:34.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.398+0000 7f07be54e740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1429ce0 con 0x13fe990 2023-11-15T03:02:34.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.398+0000 7f07be54e740 10 client.0 ll_register_callbacks cb 0x13b55d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:02:34.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.398+0000 7f07b1f45700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13f8150 0x1428ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.398+0000 7f07b1f45700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13f8150 0x1428ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40662/0 (socket says 172.21.15.121:40662) 2023-11-15T03:02:34.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.398+0000 7f07b1f45700 1 -- 172.21.15.121:0/1779989297 learned_addr learned my addr 172.21.15.121:0/1779989297 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:34.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.398+0000 7f07b2746700 1 --2- 172.21.15.121:0/1779989297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1441a70 0x141b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.398+0000 7f07b1744700 1 --2- 172.21.15.121:0/1779989297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x13fe990 0x141af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.398+0000 7f07b1f45700 1 -- 172.21.15.121:0/1779989297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x13fe990 msgr2=0x141af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:34.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.398+0000 7f07b1f45700 1 --2- 172.21.15.121:0/1779989297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x13fe990 0x141af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.398+0000 7f07b1f45700 1 -- 172.21.15.121:0/1779989297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1441a70 msgr2=0x141b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:34.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.398+0000 7f07b1f45700 1 --2- 172.21.15.121:0/1779989297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1441a70 0x141b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.398+0000 7f07b1f45700 1 -- 172.21.15.121:0/1779989297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079c011850 con 0x13f8150 2023-11-15T03:02:34.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.398+0000 7f07b1f45700 1 --2- 172.21.15.121:0/1779989297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13f8150 0x1428ed0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f07a800cfd0 tx=0x7f07a800f000).ready entity=mon.0 client_cookie=1d7d0b1682d217e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:34.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.402+0000 7f0787fff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:02:34.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.402+0000 7f0787fff700 1 -- 172.21.15.121:0/1779989297 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8055070 con 0x13f8150 2023-11-15T03:02:34.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.402+0000 7f0787fff700 1 -- 172.21.15.121:0/1779989297 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f07a804b3f0 con 0x13f8150 2023-11-15T03:02:34.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.402+0000 7f07be54e740 1 -- 172.21.15.121:0/1779989297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1375230 con 0x13f8150 2023-11-15T03:02:34.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.402+0000 7f0787fff700 1 -- 172.21.15.121:0/1779989297 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07a804a940 con 0x13f8150 2023-11-15T03:02:34.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.402+0000 7f07be54e740 10 client.6711 fetch_fsmap learned FSMap version 68 2023-11-15T03:02:34.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.402+0000 7f07be54e740 1 -- 172.21.15.121:0/1779989297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x13787c0 con 0x13f8150 2023-11-15T03:02:34.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.402+0000 7f0787fff700 1 -- 172.21.15.121:0/1779989297 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1375230 con 0x13f8150 2023-11-15T03:02:34.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.402+0000 7f0787fff700 1 -- 172.21.15.121:0/1779989297 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 68) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f07a8047240 con 0x13f8150 2023-11-15T03:02:34.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.402+0000 7f07be54e740 10 client.6711 fetch_fsmap finished waiting for FSMap version 68 2023-11-15T03:02:34.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.402+0000 7f07be54e740 10 client.6711 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2023-11-15T03:02:34.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.402+0000 7f07be54e740 20 client.6711 populate_metadata read hostname 'smithi121' 2023-11-15T03:02:34.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.402+0000 7f07be54e740 1 --2- 172.21.15.121:0/1779989297 >> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] conn(0x145b5d0 0x145d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.402+0000 7f07be54e740 4 client.6711 mds_command: new command op to 6492 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:02:34.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.402+0000 7f07be54e740 1 -- 172.21.15.121:0/1779989297 --> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x136b780 con 0x145b5d0 2023-11-15T03:02:34.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.402+0000 7f07b1744700 1 --2- 172.21.15.121:0/1779989297 >> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] conn(0x145b5d0 0x145d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.402+0000 7f0787fff700 10 client.6711 ms_handle_connect on v2:172.21.15.190:6832/914796342 2023-11-15T03:02:34.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.402+0000 7f07b1744700 1 --2- 172.21.15.121:0/1779989297 >> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] conn(0x145b5d0 0x145d9c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=db10d015ed5ad98e server_cookie=48d368ffbbd869a7 in_seq=0 out_seq=0 2023-11-15T03:02:34.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.406+0000 7f0787fff700 1 -- 172.21.15.121:0/1779989297 <== mds.0 v2:172.21.15.190:6832/914796342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x136b780 con 0x145b5d0 2023-11-15T03:02:34.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.406+0000 7f0787fff700 10 client.6711 handle_command_reply: tid=0 2023-11-15T03:02:34.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.406+0000 7f07be54e740 1 client.6711 shutdown 2023-11-15T03:02:34.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.406+0000 7f07be54e740 1 -- 172.21.15.121:0/1779989297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13f8150 msgr2=0x1428ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:34.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.406+0000 7f07be54e740 1 --2- 172.21.15.121:0/1779989297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13f8150 0x1428ed0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f07a800cfd0 tx=0x7f07a800f000).stop 2023-11-15T03:02:34.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.406+0000 7f07be54e740 1 -- 172.21.15.121:0/1779989297 shutdown_connections 2023-11-15T03:02:34.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.406+0000 7f07be54e740 1 --2- 172.21.15.121:0/1779989297 >> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] conn(0x145b5d0 0x145d9c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.406+0000 7f07be54e740 1 --2- 172.21.15.121:0/1779989297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1441a70 0x141b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.406+0000 7f07be54e740 1 --2- 172.21.15.121:0/1779989297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x13fe990 0x141af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.406+0000 7f0787fff700 0 client.6711 ms_handle_reset on v2:172.21.15.190:6832/914796342 2023-11-15T03:02:34.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.406+0000 7f07be54e740 1 --2- 172.21.15.121:0/1779989297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13f8150 0x1428ed0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.406+0000 7f07be54e740 1 -- 172.21.15.121:0/1779989297 >> 172.21.15.121:0/1779989297 conn(0x11dd680 msgr2=0x13fc820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:34.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.406+0000 7f07be54e740 1 -- 172.21.15.121:0/1779989297 shutdown_connections 2023-11-15T03:02:34.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.406+0000 7f07be54e740 1 -- 172.21.15.121:0/1779989297 wait complete. 2023-11-15T03:02:34.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.406+0000 7f07be54e740 20 client.6711 trim_cache size 0 max 16384 2023-11-15T03:02:34.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.422+0000 7f07be54e740 1 Processor -- start 2023-11-15T03:02:34.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.422+0000 7f07be54e740 1 -- start start 2023-11-15T03:02:34.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.422+0000 7f07be54e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13f8150 0x1427160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.422+0000 7f07be54e740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x13fe990 0x1443850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.422+0000 7f07be54e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1441a70 0x1443d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.422+0000 7f07be54e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x142b8a0 con 0x13f8150 2023-11-15T03:02:34.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.422+0000 7f07be54e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x13f3540 con 0x1441a70 2023-11-15T03:02:34.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.422+0000 7f07be54e740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1441770 con 0x13fe990 2023-11-15T03:02:34.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.422+0000 7f07b1f45700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13f8150 0x1427160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.422+0000 7f07b1744700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x13fe990 0x1443850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.422+0000 7f07b2746700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1441a70 0x1443d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.422+0000 7f07b1744700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x13fe990 0x1443850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:33044/0 (socket says 172.21.15.121:33044) 2023-11-15T03:02:34.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.422+0000 7f07b2746700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1441a70 0x1443d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:36906/0 (socket says 172.21.15.121:36906) 2023-11-15T03:02:34.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.422+0000 7f07b1744700 1 -- 172.21.15.121:0/2043839965 learned_addr learned my addr 172.21.15.121:0/2043839965 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:34.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f07b1f45700 1 -- 172.21.15.121:0/2043839965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x13fe990 msgr2=0x1443850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:34.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f07b1f45700 1 --2- 172.21.15.121:0/2043839965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x13fe990 0x1443850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f07b1f45700 1 -- 172.21.15.121:0/2043839965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1441a70 msgr2=0x1443d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:34.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f07b1f45700 1 --2- 172.21.15.121:0/2043839965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1441a70 0x1443d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f07b1f45700 1 -- 172.21.15.121:0/2043839965 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ac0a29c0 con 0x13f8150 2023-11-15T03:02:34.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f07b2746700 1 --2- 172.21.15.121:0/2043839965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1441a70 0x1443d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:02:34.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f07b1f45700 1 --2- 172.21.15.121:0/2043839965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13f8150 0x1427160 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7f07a804b380 tx=0x7f07a8059000).ready entity=mon.0 client_cookie=d2212ea1b877c8dc server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:34.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f078f7fe700 1 -- 172.21.15.121:0/2043839965 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8069030 con 0x13f8150 2023-11-15T03:02:34.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f078f7fe700 1 -- 172.21.15.121:0/2043839965 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a804b1c0 con 0x13f8150 2023-11-15T03:02:34.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f078f7fe700 1 -- 172.21.15.121:0/2043839965 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a804b3f0 con 0x13f8150 2023-11-15T03:02:34.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f07be54e740 1 -- 172.21.15.121:0/2043839965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13f8150 msgr2=0x1427160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:34.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f07be54e740 1 --2- 172.21.15.121:0/2043839965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13f8150 0x1427160 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7f07a804b380 tx=0x7f07a8059000).stop 2023-11-15T03:02:34.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f07be54e740 1 -- 172.21.15.121:0/2043839965 shutdown_connections 2023-11-15T03:02:34.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f07be54e740 1 --2- 172.21.15.121:0/2043839965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1441a70 0x1443d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f07be54e740 1 --2- 172.21.15.121:0/2043839965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x13fe990 0x1443850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f07be54e740 1 --2- 172.21.15.121:0/2043839965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13f8150 0x1427160 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f07be54e740 1 -- 172.21.15.121:0/2043839965 >> 172.21.15.121:0/2043839965 conn(0x11dd680 msgr2=0x1442b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:34.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f07be54e740 1 -- 172.21.15.121:0/2043839965 shutdown_connections 2023-11-15T03:02:34.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f07be54e740 1 -- 172.21.15.121:0/2043839965 wait complete. 2023-11-15T03:02:34.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.426+0000 7f07be54e740 1 -- start start 2023-11-15T03:02:34.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.430+0000 7f07be54e740 1 Processor -- start 2023-11-15T03:02:34.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.430+0000 7f07be54e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x13f8150 0x144ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.430+0000 7f07be54e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13fe990 0x1428c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.430+0000 7f07be54e740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1441a70 0x143a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.430+0000 7f07be54e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1427090 con 0x13fe990 2023-11-15T03:02:34.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.430+0000 7f07be54e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1453ab0 con 0x13f8150 2023-11-15T03:02:34.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.430+0000 7f07be54e740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x144f720 con 0x1441a70 2023-11-15T03:02:34.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.430+0000 7f07be54e740 10 client.0 ll_register_callbacks cb 0x7f07a80477d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:02:34.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.430+0000 7f07b1744700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13fe990 0x1428c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.430+0000 7f07b1744700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13fe990 0x1428c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40684/0 (socket says 172.21.15.121:40684) 2023-11-15T03:02:34.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.430+0000 7f07b1744700 1 -- 172.21.15.121:0/3726482208 learned_addr learned my addr 172.21.15.121:0/3726482208 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:34.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.430+0000 7f07b2746700 1 --2- 172.21.15.121:0/3726482208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1441a70 0x143a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.430+0000 7f07b1f45700 1 --2- 172.21.15.121:0/3726482208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x13f8150 0x144ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.430+0000 7f07b1744700 1 -- 172.21.15.121:0/3726482208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1441a70 msgr2=0x143a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:34.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.430+0000 7f07b1744700 1 --2- 172.21.15.121:0/3726482208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1441a70 0x143a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.430+0000 7f07b1744700 1 -- 172.21.15.121:0/3726482208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x13f8150 msgr2=0x144ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:34.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.430+0000 7f07b1744700 1 --2- 172.21.15.121:0/3726482208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x13f8150 0x144ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.430+0000 7f07b1744700 1 -- 172.21.15.121:0/3726482208 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ac062470 con 0x13fe990 2023-11-15T03:02:34.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f07b1f45700 1 --2- 172.21.15.121:0/3726482208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x13f8150 0x144ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:34.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f07b1744700 1 --2- 172.21.15.121:0/3726482208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13fe990 0x1428c40 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f079c006980 tx=0x7f079c030280).ready entity=mon.0 client_cookie=6ee0b3bb88eea0ad server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:34.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f0787fff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:02:34.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f0787fff700 1 -- 172.21.15.121:0/3726482208 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c020820 con 0x13fe990 2023-11-15T03:02:34.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f07be54e740 1 -- 172.21.15.121:0/3726482208 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f079c015a70 con 0x13fe990 2023-11-15T03:02:34.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f0787fff700 1 -- 172.21.15.121:0/3726482208 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f079c020980 con 0x13fe990 2023-11-15T03:02:34.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f0787fff700 1 -- 172.21.15.121:0/3726482208 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f079c020d50 con 0x13fe990 2023-11-15T03:02:34.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f07be54e740 10 client.6717 fetch_fsmap learned FSMap version 68 2023-11-15T03:02:34.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f07be54e740 1 -- 172.21.15.121:0/3726482208 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x13787c0 con 0x13fe990 2023-11-15T03:02:34.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f0787fff700 1 -- 172.21.15.121:0/3726482208 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c015a70 con 0x13fe990 2023-11-15T03:02:34.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f0787fff700 1 -- 172.21.15.121:0/3726482208 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 68) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f079c02a440 con 0x13fe990 2023-11-15T03:02:34.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f07be54e740 10 client.6717 fetch_fsmap finished waiting for FSMap version 68 2023-11-15T03:02:34.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f07be54e740 10 client.6717 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2023-11-15T03:02:34.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f07be54e740 20 client.6717 populate_metadata read hostname 'smithi121' 2023-11-15T03:02:34.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f07be54e740 1 --2- 172.21.15.121:0/3726482208 >> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] conn(0x1429e20 0x143ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f07be54e740 4 client.6717 mds_command: new command op to 6492 tid=0[{"prefix": "flush journal"}] 2023-11-15T03:02:34.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f07be54e740 1 -- 172.21.15.121:0/3726482208 --> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x1234610 con 0x1429e20 2023-11-15T03:02:34.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f07b1f45700 1 --2- 172.21.15.121:0/3726482208 >> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] conn(0x1429e20 0x143ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f0787fff700 10 client.6717 ms_handle_connect on v2:172.21.15.190:6832/914796342 2023-11-15T03:02:34.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.434+0000 7f07b1f45700 1 --2- 172.21.15.121:0/3726482208 >> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] conn(0x1429e20 0x143ad40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=81a029baeda8cfea server_cookie=9cda9399b0b8e4f4 in_seq=0 out_seq=0 2023-11-15T03:02:34.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.458+0000 7f0787fff700 1 -- 172.21.15.121:0/3726482208 <== mds.0 v2:172.21.15.190:6832/914796342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x1234610 con 0x1429e20 2023-11-15T03:02:34.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.458+0000 7f0787fff700 10 client.6717 handle_command_reply: tid=0 2023-11-15T03:02:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.458+0000 7f07be54e740 1 client.6717 shutdown 2023-11-15T03:02:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.458+0000 7f07be54e740 1 -- 172.21.15.121:0/3726482208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13fe990 msgr2=0x1428c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.458+0000 7f07be54e740 1 --2- 172.21.15.121:0/3726482208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13fe990 0x1428c40 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f079c006980 tx=0x7f079c030280).stop 2023-11-15T03:02:34.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.458+0000 7f07be54e740 1 -- 172.21.15.121:0/3726482208 shutdown_connections 2023-11-15T03:02:34.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.458+0000 7f07be54e740 1 --2- 172.21.15.121:0/3726482208 >> [v2:172.21.15.190:6832/914796342,v1:172.21.15.190:6833/914796342] conn(0x1429e20 0x143ad40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.458+0000 7f07be54e740 1 --2- 172.21.15.121:0/3726482208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1441a70 0x143a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.458+0000 7f07be54e740 1 --2- 172.21.15.121:0/3726482208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x13fe990 0x1428c40 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.458+0000 7f0787fff700 0 client.6717 ms_handle_reset on v2:172.21.15.190:6832/914796342 2023-11-15T03:02:34.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.458+0000 7f07be54e740 1 --2- 172.21.15.121:0/3726482208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x13f8150 0x144ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.458+0000 7f07be54e740 1 -- 172.21.15.121:0/3726482208 >> 172.21.15.121:0/3726482208 conn(0x11dd680 msgr2=0x1443850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:34.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.462+0000 7f07be54e740 1 -- 172.21.15.121:0/3726482208 shutdown_connections 2023-11-15T03:02:34.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.462+0000 7f07be54e740 1 -- 172.21.15.121:0/3726482208 wait complete. 2023-11-15T03:02:34.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.462+0000 7f07be54e740 20 client.6717 trim_cache size 0 max 16384 2023-11-15T03:02:34.472 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:02:34.472 INFO:teuthology.orchestra.run.smithi121.stdout: "message": "", 2023-11-15T03:02:34.472 INFO:teuthology.orchestra.run.smithi121.stdout: "return_code": 0 2023-11-15T03:02:34.473 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:02:34.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.462+0000 7f07b41a9700 1 -- 172.21.15.121:0/349755504 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f079005a540 msgr2=0x7f079005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:34.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.462+0000 7f07b41a9700 1 --2- 172.21.15.121:0/349755504 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f079005a540 0x7f079005ca00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f07ac0bfa80 tx=0x7f079c008040).stop 2023-11-15T03:02:34.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.462+0000 7f07b41a9700 1 -- 172.21.15.121:0/349755504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07ac067020 msgr2=0x7f07ac148550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:34.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.462+0000 7f07b41a9700 1 --2- 172.21.15.121:0/349755504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07ac067020 0x7f07ac148550 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f07a800c420 tx=0x7f07a8007c80).stop 2023-11-15T03:02:34.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.462+0000 7f07b41a9700 1 -- 172.21.15.121:0/349755504 shutdown_connections 2023-11-15T03:02:34.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.462+0000 7f07b41a9700 1 --2- 172.21.15.121:0/349755504 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f079005a540 0x7f079005ca00 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.462+0000 7f07b41a9700 1 --2- 172.21.15.121:0/349755504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ac157ed0 0x7f07ac0a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.462+0000 7f07b41a9700 1 --2- 172.21.15.121:0/349755504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07ac068170 0x7f07ac0a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.462+0000 7f07b41a9700 1 --2- 172.21.15.121:0/349755504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07ac067020 0x7f07ac148550 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.462+0000 7f07b41a9700 1 -- 172.21.15.121:0/349755504 >> 172.21.15.121:0/349755504 conn(0x7f07ac0591c0 msgr2=0x7f07ac0a9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.462+0000 7f07b41a9700 1 -- 172.21.15.121:0/349755504 shutdown_connections 2023-11-15T03:02:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.462+0000 7f07b41a9700 1 -- 172.21.15.121:0/349755504 wait complete. 2023-11-15T03:02:34.493 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_rename_corrupt_dentry_first 1.0 2023-11-15T03:02:34.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.650+0000 7f678e039700 1 Processor -- start 2023-11-15T03:02:34.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.650+0000 7f678e039700 1 -- start start 2023-11-15T03:02:34.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.654+0000 7f678e039700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67880acd00 0x7f67880ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.654+0000 7f678e039700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f67880c5c70 0x7f67881536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.654+0000 7f678e039700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6788065ea0 con 0x7f6788066330 2023-11-15T03:02:34.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.654+0000 7f678e039700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6788066bb0 con 0x7f67880c5c70 2023-11-15T03:02:34.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.654+0000 7f678e039700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6788065890 con 0x7f67880acd00 2023-11-15T03:02:34.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.658+0000 7f678d037700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67880acd00 0x7f67880ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.658+0000 7f678d037700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67880acd00 0x7f67880ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:33060/0 (socket says 172.21.15.121:33060) 2023-11-15T03:02:34.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.658+0000 7f678d037700 1 -- 172.21.15.121:0/2419030000 learned_addr learned my addr 172.21.15.121:0/2419030000 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:34.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.658+0000 7f678c836700 1 --2- 172.21.15.121:0/2419030000 >> v2:172.21.15.190:3300/0 conn(0x7f67880c5c70 0x7f67881536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.658+0000 7f678d037700 1 -- 172.21.15.121:0/2419030000 >> v2:172.21.15.190:3300/0 conn(0x7f67880c5c70 msgr2=0x7f67881536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:34.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.658+0000 7f678d037700 1 --2- 172.21.15.121:0/2419030000 >> v2:172.21.15.190:3300/0 conn(0x7f67880c5c70 0x7f67881536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.658+0000 7f678d037700 1 -- 172.21.15.121:0/2419030000 >> v1:172.21.15.190:6789/0 conn(0x7f6788066330 legacy=0x7f6788153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:34.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.658+0000 7f678d037700 1 -- 172.21.15.121:0/2419030000 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67880c64a0 con 0x7f67880acd00 2023-11-15T03:02:34.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.658+0000 7f678d037700 1 --2- 172.21.15.121:0/2419030000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67880acd00 0x7f67880ad0e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f6784009810 tx=0x7f6784009060).ready entity=mon.2 client_cookie=7c45ac09932ea09c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:34.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.658+0000 7f677ffff700 1 -- 172.21.15.121:0/2419030000 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6784005880 con 0x7f67880acd00 2023-11-15T03:02:34.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.658+0000 7f677ffff700 1 -- 172.21.15.121:0/2419030000 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67840059e0 con 0x7f67880acd00 2023-11-15T03:02:34.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.658+0000 7f677ffff700 1 -- 172.21.15.121:0/2419030000 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6784005d60 con 0x7f67880acd00 2023-11-15T03:02:34.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.658+0000 7f678e039700 1 -- 172.21.15.121:0/2419030000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67880acd00 msgr2=0x7f67880ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:34.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.658+0000 7f678e039700 1 --2- 172.21.15.121:0/2419030000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67880acd00 0x7f67880ad0e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f6784009810 tx=0x7f6784009060).stop 2023-11-15T03:02:34.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.658+0000 7f678e039700 1 -- 172.21.15.121:0/2419030000 shutdown_connections 2023-11-15T03:02:34.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.658+0000 7f678e039700 1 --2- 172.21.15.121:0/2419030000 >> v2:172.21.15.190:3300/0 conn(0x7f67880c5c70 0x7f67881536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.658+0000 7f678e039700 1 --2- 172.21.15.121:0/2419030000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67880acd00 0x7f67880ad0e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.658+0000 7f678e039700 1 -- 172.21.15.121:0/2419030000 >> 172.21.15.121:0/2419030000 conn(0x7f6788063fc0 msgr2=0x7f6788067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:34.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.662+0000 7f678e039700 1 -- 172.21.15.121:0/2419030000 shutdown_connections 2023-11-15T03:02:34.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.662+0000 7f678e039700 1 -- 172.21.15.121:0/2419030000 wait complete. 2023-11-15T03:02:34.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.662+0000 7f678e039700 1 Processor -- start 2023-11-15T03:02:34.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.662+0000 7f678e039700 1 -- start start 2023-11-15T03:02:34.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678e039700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6788066330 0x7f6788156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678e039700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67880acd00 0x7f678814e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678e039700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67880c5c70 0x7f67880c1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678e039700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6788168580 con 0x7f6788066330 2023-11-15T03:02:34.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678e039700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6788167970 con 0x7f67880acd00 2023-11-15T03:02:34.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678e039700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6788167c50 con 0x7f67880c5c70 2023-11-15T03:02:34.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678d037700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6788066330 0x7f6788156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678d037700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6788066330 0x7f6788156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53196/0 (socket says 172.21.15.121:53196) 2023-11-15T03:02:34.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678d037700 1 -- 172.21.15.121:0/699330526 learned_addr learned my addr 172.21.15.121:0/699330526 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:34.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678c836700 1 --2- 172.21.15.121:0/699330526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67880acd00 0x7f678814e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678d838700 1 --2- 172.21.15.121:0/699330526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67880c5c70 0x7f67880c1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678d037700 1 -- 172.21.15.121:0/699330526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67880c5c70 msgr2=0x7f67880c1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:34.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678d037700 1 --2- 172.21.15.121:0/699330526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67880c5c70 0x7f67880c1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678d037700 1 -- 172.21.15.121:0/699330526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67880acd00 msgr2=0x7f678814e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:34.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678d037700 1 --2- 172.21.15.121:0/699330526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67880acd00 0x7f678814e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678d037700 1 -- 172.21.15.121:0/699330526 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6784009300 con 0x7f6788066330 2023-11-15T03:02:34.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678c836700 1 --2- 172.21.15.121:0/699330526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67880acd00 0x7f678814e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:34.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678d037700 1 --2- 172.21.15.121:0/699330526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6788066330 0x7f6788156c00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f6784009eb0 tx=0x7f6784016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:34.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f677d7fa700 1 -- 172.21.15.121:0/699330526 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678400b790 con 0x7f6788066330 2023-11-15T03:02:34.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f677d7fa700 1 -- 172.21.15.121:0/699330526 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678400b8f0 con 0x7f6788066330 2023-11-15T03:02:34.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678e039700 1 -- 172.21.15.121:0/699330526 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6788167f30 con 0x7f6788066330 2023-11-15T03:02:34.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f677d7fa700 1 -- 172.21.15.121:0/699330526 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678400bc90 con 0x7f6788066330 2023-11-15T03:02:34.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.666+0000 7f678e039700 1 -- 172.21.15.121:0/699330526 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6788153490 con 0x7f6788066330 2023-11-15T03:02:34.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.670+0000 7f678e039700 1 -- 172.21.15.121:0/699330526 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678806bcf0 con 0x7f6788066330 2023-11-15T03:02:34.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.670+0000 7f677d7fa700 1 -- 172.21.15.121:0/699330526 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6784018070 con 0x7f6788066330 2023-11-15T03:02:34.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.674+0000 7f677d7fa700 1 --2- 172.21.15.121:0/699330526 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f676c05a540 0x7f676c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:34.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.674+0000 7f677d7fa700 1 -- 172.21.15.121:0/699330526 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6784075960 con 0x7f6788066330 2023-11-15T03:02:34.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.674+0000 7f678c836700 1 --2- 172.21.15.121:0/699330526 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f676c05a540 0x7f676c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:34.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.674+0000 7f678c836700 1 --2- 172.21.15.121:0/699330526 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f676c05a540 0x7f676c05ca00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f67880c3040 tx=0x7f6778008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:34.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.678+0000 7f677d7fa700 1 -- 172.21.15.121:0/699330526 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f67840490c0 con 0x7f6788066330 2023-11-15T03:02:34.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.842+0000 7f678e039700 1 -- 172.21.15.121:0/699330526 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}] v 0) v1 -- 0x7f67880c61c0 con 0x7f6788066330 2023-11-15T03:02:34.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.850+0000 7f677d7fa700 1 -- 172.21.15.121:0/699330526 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}]=0 v15)=0 v15) v1 ==== 138+0+0 (secure 0 0 0) 0x7f678404c6a0 con 0x7f6788066330 2023-11-15T03:02:34.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.854+0000 7f678e039700 1 -- 172.21.15.121:0/699330526 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f676c05a540 msgr2=0x7f676c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:34.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.854+0000 7f678e039700 1 --2- 172.21.15.121:0/699330526 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f676c05a540 0x7f676c05ca00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f67880c3040 tx=0x7f6778008040).stop 2023-11-15T03:02:34.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.854+0000 7f678e039700 1 -- 172.21.15.121:0/699330526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6788066330 msgr2=0x7f6788156c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:34.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.854+0000 7f678e039700 1 --2- 172.21.15.121:0/699330526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6788066330 0x7f6788156c00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f6784009eb0 tx=0x7f6784016000).stop 2023-11-15T03:02:34.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.854+0000 7f678e039700 1 -- 172.21.15.121:0/699330526 shutdown_connections 2023-11-15T03:02:34.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.854+0000 7f678e039700 1 --2- 172.21.15.121:0/699330526 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f676c05a540 0x7f676c05ca00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.854+0000 7f678e039700 1 --2- 172.21.15.121:0/699330526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67880c5c70 0x7f67880c1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.854+0000 7f678e039700 1 --2- 172.21.15.121:0/699330526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67880acd00 0x7f678814e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.854+0000 7f678e039700 1 --2- 172.21.15.121:0/699330526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6788066330 0x7f6788156c00 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:34.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.854+0000 7f678e039700 1 -- 172.21.15.121:0/699330526 >> 172.21.15.121:0/699330526 conn(0x7f6788063fc0 msgr2=0x7f678814f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:34.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.854+0000 7f678e039700 1 -- 172.21.15.121:0/699330526 shutdown_connections 2023-11-15T03:02:34.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:34.854+0000 7f678e039700 1 -- 172.21.15.121:0/699330526 wait complete. 2023-11-15T03:02:39.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:02:39.882 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'timeout 60 mv a/b a/z') 2023-11-15T03:02:39.891 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-15T03:02:39.918 INFO:tasks.ceph.mds.d.smithi190.stderr:2023-11-15T03:02:39.916+0000 7f9deadf6700 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by 0x563a6a3f0800) (dversion lock w=1 last_client=6687) pv=11 v=10 ap=2 ino=(nil) state=1073741888|bottomlru | request=1 lock=2 authpin=1 0x563a6a4a4c80] 2023-11-15T03:02:39.918 INFO:tasks.ceph.mds.d.smithi190.stderr:2023-11-15T03:02:39.916+0000 7f9deadf6700 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by 0x563a6a3f0800) (dversion lock w=1 last_client=6687) pv=11 v=10 ap=2 ino=(nil) state=1073741888|bottomlru | request=1 lock=2 authpin=1 0x563a6a4a4c80] 2023-11-15T03:02:39.922 INFO:tasks.ceph.mds.d.smithi190.stderr:/build/ceph-16.2.14-170-g620f25c1/src/mds/CDentry.cc: In function 'bool CDentry::check_corruption(bool)' thread 7f9deadf6700 time 2023-11-15T03:02:39.919174+0000 2023-11-15T03:02:39.922 INFO:tasks.ceph.mds.d.smithi190.stderr:/build/ceph-16.2.14-170-g620f25c1/src/mds/CDentry.cc: 699: ceph_abort_msg("abort() called") 2023-11-15T03:02:39.922 INFO:tasks.ceph.mds.d.smithi190.stderr: ceph version 16.2.14-170-g620f25c1 (620f25c1794f62207424fc9319427943fb8d1fae) pacific (stable) 2023-11-15T03:02:39.922 INFO:tasks.ceph.mds.d.smithi190.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe0) [0x7f9df02684b5] 2023-11-15T03:02:39.923 INFO:tasks.ceph.mds.d.smithi190.stderr: 2: (CDentry::check_corruption(bool)+0x86b) [0x563a67c49cbb] 2023-11-15T03:02:39.923 INFO:tasks.ceph.mds.d.smithi190.stderr: 3: (EMetaBlob::add_primary_dentry(EMetaBlob::dirlump&, CDentry*, CInode*, unsigned char)+0x47) [0x563a67a97e37] 2023-11-15T03:02:39.923 INFO:tasks.ceph.mds.d.smithi190.stderr: 4: (EMetaBlob::add_primary_dentry(CDentry*, CInode*, bool, bool, bool, bool)+0x1b5) [0x563a67a99425] 2023-11-15T03:02:39.923 INFO:tasks.ceph.mds.d.smithi190.stderr: 5: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x122e) [0x563a67a5e80e] 2023-11-15T03:02:39.924 INFO:tasks.ceph.mds.d.smithi190.stderr: 6: (Server::handle_client_rename(boost::intrusive_ptr&)+0x2a94) [0x563a67a63b74] 2023-11-15T03:02:39.924 INFO:tasks.ceph.mds.d.smithi190.stderr: 7: (Server::dispatch_client_request(boost::intrusive_ptr&)+0x9d0) [0x563a67a75c40] 2023-11-15T03:02:39.924 INFO:tasks.ceph.mds.d.smithi190.stderr: 8: (Server::handle_client_request(boost::intrusive_ptr const&)+0x605) [0x563a67a769e5] 2023-11-15T03:02:39.924 INFO:tasks.ceph.mds.d.smithi190.stderr: 9: (Server::dispatch(boost::intrusive_ptr const&)+0x12c) [0x563a67a7b88c] 2023-11-15T03:02:39.924 INFO:tasks.ceph.mds.d.smithi190.stderr: 10: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x3eb) [0x563a679b78ab] 2023-11-15T03:02:39.925 INFO:tasks.ceph.mds.d.smithi190.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x643) [0x563a679d27e3] 2023-11-15T03:02:39.925 INFO:tasks.ceph.mds.d.smithi190.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5e) [0x563a679d2e7e] 2023-11-15T03:02:39.925 INFO:tasks.ceph.mds.d.smithi190.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1d6) [0x563a679a93c6] 2023-11-15T03:02:39.925 INFO:tasks.ceph.mds.d.smithi190.stderr: 14: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f9df04ba640] 2023-11-15T03:02:39.925 INFO:tasks.ceph.mds.d.smithi190.stderr: 15: (DispatchQueue::entry()+0x58f) [0x7f9df04b7edf] 2023-11-15T03:02:39.926 INFO:tasks.ceph.mds.d.smithi190.stderr: 16: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9df0586c91] 2023-11-15T03:02:39.926 INFO:tasks.ceph.mds.d.smithi190.stderr: 17: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f9df8d81609] 2023-11-15T03:02:39.926 INFO:tasks.ceph.mds.d.smithi190.stderr: 18: clone() 2023-11-15T03:02:39.926 INFO:tasks.ceph.mds.d.smithi190.stderr:*** Caught signal (Aborted) ** 2023-11-15T03:02:39.927 INFO:tasks.ceph.mds.d.smithi190.stderr: in thread 7f9deadf6700 thread_name:ms_dispatch 2023-11-15T03:02:39.927 INFO:tasks.ceph.mds.d.smithi190.stderr:2023-11-15T03:02:39.920+0000 7f9deadf6700 -1 /build/ceph-16.2.14-170-g620f25c1/src/mds/CDentry.cc: In function 'bool CDentry::check_corruption(bool)' thread 7f9deadf6700 time 2023-11-15T03:02:39.919174+0000 2023-11-15T03:02:39.927 INFO:tasks.ceph.mds.d.smithi190.stderr:/build/ceph-16.2.14-170-g620f25c1/src/mds/CDentry.cc: 699: ceph_abort_msg("abort() called") 2023-11-15T03:02:39.927 INFO:tasks.ceph.mds.d.smithi190.stderr: 2023-11-15T03:02:39.928 INFO:tasks.ceph.mds.d.smithi190.stderr: ceph version 16.2.14-170-g620f25c1 (620f25c1794f62207424fc9319427943fb8d1fae) pacific (stable) 2023-11-15T03:02:39.929 INFO:tasks.ceph.mds.d.smithi190.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe0) [0x7f9df02684b5] 2023-11-15T03:02:39.929 INFO:tasks.ceph.mds.d.smithi190.stderr: 2: (CDentry::check_corruption(bool)+0x86b) [0x563a67c49cbb] 2023-11-15T03:02:39.929 INFO:tasks.ceph.mds.d.smithi190.stderr: 3: (EMetaBlob::add_primary_dentry(EMetaBlob::dirlump&, CDentry*, CInode*, unsigned char)+0x47) [0x563a67a97e37] 2023-11-15T03:02:39.929 INFO:tasks.ceph.mds.d.smithi190.stderr: 4: (EMetaBlob::add_primary_dentry(CDentry*, CInode*, bool, bool, bool, bool)+0x1b5) [0x563a67a99425] 2023-11-15T03:02:39.929 INFO:tasks.ceph.mds.d.smithi190.stderr: 5: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x122e) [0x563a67a5e80e] 2023-11-15T03:02:39.930 INFO:tasks.ceph.mds.d.smithi190.stderr: 6: (Server::handle_client_rename(boost::intrusive_ptr&)+0x2a94) [0x563a67a63b74] 2023-11-15T03:02:39.931 INFO:tasks.ceph.mds.d.smithi190.stderr: 7: (Server::dispatch_client_request(boost::intrusive_ptr&)+0x9d0) [0x563a67a75c40] 2023-11-15T03:02:39.932 INFO:tasks.ceph.mds.d.smithi190.stderr: 8: (Server::handle_client_request(boost::intrusive_ptr const&)+0x605) [0x563a67a769e5] 2023-11-15T03:02:39.932 INFO:tasks.ceph.mds.d.smithi190.stderr: 9: (Server::dispatch(boost::intrusive_ptr const&)+0x12c) [0x563a67a7b88c] 2023-11-15T03:02:39.932 INFO:tasks.ceph.mds.d.smithi190.stderr: 10: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x3eb) [0x563a679b78ab] 2023-11-15T03:02:39.932 INFO:tasks.ceph.mds.d.smithi190.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x643) [0x563a679d27e3] 2023-11-15T03:02:39.932 INFO:tasks.ceph.mds.d.smithi190.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5e) [0x563a679d2e7e] 2023-11-15T03:02:39.933 INFO:tasks.ceph.mds.d.smithi190.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1d6) [0x563a679a93c6] 2023-11-15T03:02:39.933 INFO:tasks.ceph.mds.d.smithi190.stderr: 14: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f9df04ba640] 2023-11-15T03:02:39.933 INFO:tasks.ceph.mds.d.smithi190.stderr: 15: (DispatchQueue::entry()+0x58f) [0x7f9df04b7edf] 2023-11-15T03:02:39.934 INFO:tasks.ceph.mds.d.smithi190.stderr: 16: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9df0586c91] 2023-11-15T03:02:39.934 INFO:tasks.ceph.mds.d.smithi190.stderr: 17: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f9df8d81609] 2023-11-15T03:02:39.934 INFO:tasks.ceph.mds.d.smithi190.stderr: 18: clone() 2023-11-15T03:02:39.935 INFO:tasks.ceph.mds.d.smithi190.stderr: 2023-11-15T03:02:39.935 INFO:tasks.ceph.mds.d.smithi190.stderr: ceph version 16.2.14-170-g620f25c1 (620f25c1794f62207424fc9319427943fb8d1fae) pacific (stable) 2023-11-15T03:02:39.936 INFO:tasks.ceph.mds.d.smithi190.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f9df8d8d420] 2023-11-15T03:02:39.936 INFO:tasks.ceph.mds.d.smithi190.stderr: 2: gsignal() 2023-11-15T03:02:39.936 INFO:tasks.ceph.mds.d.smithi190.stderr: 3: abort() 2023-11-15T03:02:39.936 INFO:tasks.ceph.mds.d.smithi190.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x1af) [0x7f9df0268584] 2023-11-15T03:02:39.936 INFO:tasks.ceph.mds.d.smithi190.stderr: 5: (CDentry::check_corruption(bool)+0x86b) [0x563a67c49cbb] 2023-11-15T03:02:39.937 INFO:tasks.ceph.mds.d.smithi190.stderr: 6: (EMetaBlob::add_primary_dentry(EMetaBlob::dirlump&, CDentry*, CInode*, unsigned char)+0x47) [0x563a67a97e37] 2023-11-15T03:02:39.937 INFO:tasks.ceph.mds.d.smithi190.stderr: 7: (EMetaBlob::add_primary_dentry(CDentry*, CInode*, bool, bool, bool, bool)+0x1b5) [0x563a67a99425] 2023-11-15T03:02:39.937 INFO:tasks.ceph.mds.d.smithi190.stderr: 8: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x122e) [0x563a67a5e80e] 2023-11-15T03:02:39.937 INFO:tasks.ceph.mds.d.smithi190.stderr: 9: (Server::handle_client_rename(boost::intrusive_ptr&)+0x2a94) [0x563a67a63b74] 2023-11-15T03:02:39.937 INFO:tasks.ceph.mds.d.smithi190.stderr: 10: (Server::dispatch_client_request(boost::intrusive_ptr&)+0x9d0) [0x563a67a75c40] 2023-11-15T03:02:39.938 INFO:tasks.ceph.mds.d.smithi190.stderr: 11: (Server::handle_client_request(boost::intrusive_ptr const&)+0x605) [0x563a67a769e5] 2023-11-15T03:02:39.938 INFO:tasks.ceph.mds.d.smithi190.stderr: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x12c) [0x563a67a7b88c] 2023-11-15T03:02:39.938 INFO:tasks.ceph.mds.d.smithi190.stderr: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x3eb) [0x563a679b78ab] 2023-11-15T03:02:39.938 INFO:tasks.ceph.mds.d.smithi190.stderr: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x643) [0x563a679d27e3] 2023-11-15T03:02:39.939 INFO:tasks.ceph.mds.d.smithi190.stderr: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5e) [0x563a679d2e7e] 2023-11-15T03:02:39.939 INFO:tasks.ceph.mds.d.smithi190.stderr: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1d6) [0x563a679a93c6] 2023-11-15T03:02:39.939 INFO:tasks.ceph.mds.d.smithi190.stderr: 17: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f9df04ba640] 2023-11-15T03:02:39.939 INFO:tasks.ceph.mds.d.smithi190.stderr: 18: (DispatchQueue::entry()+0x58f) [0x7f9df04b7edf] 2023-11-15T03:02:39.939 INFO:tasks.ceph.mds.d.smithi190.stderr: 19: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9df0586c91] 2023-11-15T03:02:39.940 INFO:tasks.ceph.mds.d.smithi190.stderr: 20: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f9df8d81609] 2023-11-15T03:02:39.940 INFO:tasks.ceph.mds.d.smithi190.stderr: 21: clone() 2023-11-15T03:02:39.940 INFO:tasks.ceph.mds.d.smithi190.stderr:2023-11-15T03:02:39.924+0000 7f9deadf6700 -1 *** Caught signal (Aborted) ** 2023-11-15T03:02:39.940 INFO:tasks.ceph.mds.d.smithi190.stderr: in thread 7f9deadf6700 thread_name:ms_dispatch 2023-11-15T03:02:39.940 INFO:tasks.ceph.mds.d.smithi190.stderr: 2023-11-15T03:02:39.941 INFO:tasks.ceph.mds.d.smithi190.stderr: ceph version 16.2.14-170-g620f25c1 (620f25c1794f62207424fc9319427943fb8d1fae) pacific (stable) 2023-11-15T03:02:39.941 INFO:tasks.ceph.mds.d.smithi190.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f9df8d8d420] 2023-11-15T03:02:39.942 INFO:tasks.ceph.mds.d.smithi190.stderr: 2: gsignal() 2023-11-15T03:02:39.943 INFO:tasks.ceph.mds.d.smithi190.stderr: 3: abort() 2023-11-15T03:02:39.943 INFO:tasks.ceph.mds.d.smithi190.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x1af) [0x7f9df0268584] 2023-11-15T03:02:39.943 INFO:tasks.ceph.mds.d.smithi190.stderr: 5: (CDentry::check_corruption(bool)+0x86b) [0x563a67c49cbb] 2023-11-15T03:02:39.943 INFO:tasks.ceph.mds.d.smithi190.stderr: 6: (EMetaBlob::add_primary_dentry(EMetaBlob::dirlump&, CDentry*, CInode*, unsigned char)+0x47) [0x563a67a97e37] 2023-11-15T03:02:39.944 INFO:tasks.ceph.mds.d.smithi190.stderr: 7: (EMetaBlob::add_primary_dentry(CDentry*, CInode*, bool, bool, bool, bool)+0x1b5) [0x563a67a99425] 2023-11-15T03:02:39.944 INFO:tasks.ceph.mds.d.smithi190.stderr: 8: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x122e) [0x563a67a5e80e] 2023-11-15T03:02:39.944 INFO:tasks.ceph.mds.d.smithi190.stderr: 9: (Server::handle_client_rename(boost::intrusive_ptr&)+0x2a94) [0x563a67a63b74] 2023-11-15T03:02:39.944 INFO:tasks.ceph.mds.d.smithi190.stderr: 10: (Server::dispatch_client_request(boost::intrusive_ptr&)+0x9d0) [0x563a67a75c40] 2023-11-15T03:02:39.944 INFO:tasks.ceph.mds.d.smithi190.stderr: 11: (Server::handle_client_request(boost::intrusive_ptr const&)+0x605) [0x563a67a769e5] 2023-11-15T03:02:39.944 INFO:tasks.ceph.mds.d.smithi190.stderr: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x12c) [0x563a67a7b88c] 2023-11-15T03:02:39.945 INFO:tasks.ceph.mds.d.smithi190.stderr: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x3eb) [0x563a679b78ab] 2023-11-15T03:02:39.945 INFO:tasks.ceph.mds.d.smithi190.stderr: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x643) [0x563a679d27e3] 2023-11-15T03:02:39.945 INFO:tasks.ceph.mds.d.smithi190.stderr: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5e) [0x563a679d2e7e] 2023-11-15T03:02:39.945 INFO:tasks.ceph.mds.d.smithi190.stderr: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1d6) [0x563a679a93c6] 2023-11-15T03:02:39.946 INFO:tasks.ceph.mds.d.smithi190.stderr: 17: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f9df04ba640] 2023-11-15T03:02:39.946 INFO:tasks.ceph.mds.d.smithi190.stderr: 18: (DispatchQueue::entry()+0x58f) [0x7f9df04b7edf] 2023-11-15T03:02:39.946 INFO:tasks.ceph.mds.d.smithi190.stderr: 19: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9df0586c91] 2023-11-15T03:02:39.946 INFO:tasks.ceph.mds.d.smithi190.stderr: 20: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f9df8d81609] 2023-11-15T03:02:39.947 INFO:tasks.ceph.mds.d.smithi190.stderr: 21: clone() 2023-11-15T03:02:39.947 INFO:tasks.ceph.mds.d.smithi190.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-15T03:02:39.947 INFO:tasks.ceph.mds.d.smithi190.stderr: 2023-11-15T03:02:39.949 INFO:tasks.ceph.mds.d.smithi190.stderr: -3386> 2023-11-15T03:02:26.228+0000 7f9de85f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:02:39.949 INFO:tasks.ceph.mds.d.smithi190.stderr: -4> 2023-11-15T03:02:39.916+0000 7f9deadf6700 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by 0x563a6a3f0800) (dversion lock w=1 last_client=6687) pv=11 v=10 ap=2 ino=(nil) state=1073741888|bottomlru | request=1 lock=2 authpin=1 0x563a6a4a4c80] 2023-11-15T03:02:39.949 INFO:tasks.ceph.mds.d.smithi190.stderr: -2> 2023-11-15T03:02:39.916+0000 7f9deadf6700 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by 0x563a6a3f0800) (dversion lock w=1 last_client=6687) pv=11 v=10 ap=2 ino=(nil) state=1073741888|bottomlru | request=1 lock=2 authpin=1 0x563a6a4a4c80] 2023-11-15T03:02:39.950 INFO:tasks.ceph.mds.d.smithi190.stderr: -1> 2023-11-15T03:02:39.920+0000 7f9deadf6700 -1 /build/ceph-16.2.14-170-g620f25c1/src/mds/CDentry.cc: In function 'bool CDentry::check_corruption(bool)' thread 7f9deadf6700 time 2023-11-15T03:02:39.919174+0000 2023-11-15T03:02:39.950 INFO:tasks.ceph.mds.d.smithi190.stderr:/build/ceph-16.2.14-170-g620f25c1/src/mds/CDentry.cc: 699: ceph_abort_msg("abort() called") 2023-11-15T03:02:39.950 INFO:tasks.ceph.mds.d.smithi190.stderr: 2023-11-15T03:02:39.950 INFO:tasks.ceph.mds.d.smithi190.stderr: ceph version 16.2.14-170-g620f25c1 (620f25c1794f62207424fc9319427943fb8d1fae) pacific (stable) 2023-11-15T03:02:39.951 INFO:tasks.ceph.mds.d.smithi190.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe0) [0x7f9df02684b5] 2023-11-15T03:02:39.951 INFO:tasks.ceph.mds.d.smithi190.stderr: 2: (CDentry::check_corruption(bool)+0x86b) [0x563a67c49cbb] 2023-11-15T03:02:39.951 INFO:tasks.ceph.mds.d.smithi190.stderr: 3: (EMetaBlob::add_primary_dentry(EMetaBlob::dirlump&, CDentry*, CInode*, unsigned char)+0x47) [0x563a67a97e37] 2023-11-15T03:02:39.951 INFO:tasks.ceph.mds.d.smithi190.stderr: 4: (EMetaBlob::add_primary_dentry(CDentry*, CInode*, bool, bool, bool, bool)+0x1b5) [0x563a67a99425] 2023-11-15T03:02:39.951 INFO:tasks.ceph.mds.d.smithi190.stderr: 5: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x122e) [0x563a67a5e80e] 2023-11-15T03:02:39.952 INFO:tasks.ceph.mds.d.smithi190.stderr: 6: (Server::handle_client_rename(boost::intrusive_ptr&)+0x2a94) [0x563a67a63b74] 2023-11-15T03:02:39.952 INFO:tasks.ceph.mds.d.smithi190.stderr: 7: (Server::dispatch_client_request(boost::intrusive_ptr&)+0x9d0) [0x563a67a75c40] 2023-11-15T03:02:39.952 INFO:tasks.ceph.mds.d.smithi190.stderr: 8: (Server::handle_client_request(boost::intrusive_ptr const&)+0x605) [0x563a67a769e5] 2023-11-15T03:02:39.953 INFO:tasks.ceph.mds.d.smithi190.stderr: 9: (Server::dispatch(boost::intrusive_ptr const&)+0x12c) [0x563a67a7b88c] 2023-11-15T03:02:39.953 INFO:tasks.ceph.mds.d.smithi190.stderr: 10: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x3eb) [0x563a679b78ab] 2023-11-15T03:02:39.953 INFO:tasks.ceph.mds.d.smithi190.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x643) [0x563a679d27e3] 2023-11-15T03:02:39.953 INFO:tasks.ceph.mds.d.smithi190.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5e) [0x563a679d2e7e] 2023-11-15T03:02:39.953 INFO:tasks.ceph.mds.d.smithi190.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1d6) [0x563a679a93c6] 2023-11-15T03:02:39.954 INFO:tasks.ceph.mds.d.smithi190.stderr: 14: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f9df04ba640] 2023-11-15T03:02:39.954 INFO:tasks.ceph.mds.d.smithi190.stderr: 15: (DispatchQueue::entry()+0x58f) [0x7f9df04b7edf] 2023-11-15T03:02:39.954 INFO:tasks.ceph.mds.d.smithi190.stderr: 16: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9df0586c91] 2023-11-15T03:02:39.954 INFO:tasks.ceph.mds.d.smithi190.stderr: 17: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f9df8d81609] 2023-11-15T03:02:39.954 INFO:tasks.ceph.mds.d.smithi190.stderr: 18: clone() 2023-11-15T03:02:39.955 INFO:tasks.ceph.mds.d.smithi190.stderr: 2023-11-15T03:02:39.955 INFO:tasks.ceph.mds.d.smithi190.stderr: 0> 2023-11-15T03:02:39.924+0000 7f9deadf6700 -1 *** Caught signal (Aborted) ** 2023-11-15T03:02:39.955 INFO:tasks.ceph.mds.d.smithi190.stderr: in thread 7f9deadf6700 thread_name:ms_dispatch 2023-11-15T03:02:39.955 INFO:tasks.ceph.mds.d.smithi190.stderr: 2023-11-15T03:02:39.955 INFO:tasks.ceph.mds.d.smithi190.stderr: ceph version 16.2.14-170-g620f25c1 (620f25c1794f62207424fc9319427943fb8d1fae) pacific (stable) 2023-11-15T03:02:39.956 INFO:tasks.ceph.mds.d.smithi190.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f9df8d8d420] 2023-11-15T03:02:39.956 INFO:tasks.ceph.mds.d.smithi190.stderr: 2: gsignal() 2023-11-15T03:02:39.956 INFO:tasks.ceph.mds.d.smithi190.stderr: 3: abort() 2023-11-15T03:02:39.956 INFO:tasks.ceph.mds.d.smithi190.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x1af) [0x7f9df0268584] 2023-11-15T03:02:39.956 INFO:tasks.ceph.mds.d.smithi190.stderr: 5: (CDentry::check_corruption(bool)+0x86b) [0x563a67c49cbb] 2023-11-15T03:02:39.957 INFO:tasks.ceph.mds.d.smithi190.stderr: 6: (EMetaBlob::add_primary_dentry(EMetaBlob::dirlump&, CDentry*, CInode*, unsigned char)+0x47) [0x563a67a97e37] 2023-11-15T03:02:39.957 INFO:tasks.ceph.mds.d.smithi190.stderr: 7: (EMetaBlob::add_primary_dentry(CDentry*, CInode*, bool, bool, bool, bool)+0x1b5) [0x563a67a99425] 2023-11-15T03:02:39.957 INFO:tasks.ceph.mds.d.smithi190.stderr: 8: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x122e) [0x563a67a5e80e] 2023-11-15T03:02:39.957 INFO:tasks.ceph.mds.d.smithi190.stderr: 9: (Server::handle_client_rename(boost::intrusive_ptr&)+0x2a94) [0x563a67a63b74] 2023-11-15T03:02:39.957 INFO:tasks.ceph.mds.d.smithi190.stderr: 10: (Server::dispatch_client_request(boost::intrusive_ptr&)+0x9d0) [0x563a67a75c40] 2023-11-15T03:02:39.958 INFO:tasks.ceph.mds.d.smithi190.stderr: 11: (Server::handle_client_request(boost::intrusive_ptr const&)+0x605) [0x563a67a769e5] 2023-11-15T03:02:39.958 INFO:tasks.ceph.mds.d.smithi190.stderr: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x12c) [0x563a67a7b88c] 2023-11-15T03:02:39.958 INFO:tasks.ceph.mds.d.smithi190.stderr: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x3eb) [0x563a679b78ab] 2023-11-15T03:02:39.958 INFO:tasks.ceph.mds.d.smithi190.stderr: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x643) [0x563a679d27e3] 2023-11-15T03:02:39.959 INFO:tasks.ceph.mds.d.smithi190.stderr: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5e) [0x563a679d2e7e] 2023-11-15T03:02:39.959 INFO:tasks.ceph.mds.d.smithi190.stderr: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1d6) [0x563a679a93c6] 2023-11-15T03:02:39.959 INFO:tasks.ceph.mds.d.smithi190.stderr: 17: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f9df04ba640] 2023-11-15T03:02:39.959 INFO:tasks.ceph.mds.d.smithi190.stderr: 18: (DispatchQueue::entry()+0x58f) [0x7f9df04b7edf] 2023-11-15T03:02:39.959 INFO:tasks.ceph.mds.d.smithi190.stderr: 19: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9df0586c91] 2023-11-15T03:02:39.959 INFO:tasks.ceph.mds.d.smithi190.stderr: 20: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f9df8d81609] 2023-11-15T03:02:39.960 INFO:tasks.ceph.mds.d.smithi190.stderr: 21: clone() 2023-11-15T03:02:39.960 INFO:tasks.ceph.mds.d.smithi190.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-15T03:02:39.960 INFO:tasks.ceph.mds.d.smithi190.stderr: 2023-11-15T03:02:39.961 INFO:tasks.ceph.mds.d.smithi190.stderr: -3386> 2023-11-15T03:02:26.228+0000 7f9de85f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:02:39.961 INFO:tasks.ceph.mds.d.smithi190.stderr: -4> 2023-11-15T03:02:39.916+0000 7f9deadf6700 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by 0x563a6a3f0800) (dversion lock w=1 last_client=6687) pv=11 v=10 ap=2 ino=(nil) state=1073741888|bottomlru | request=1 lock=2 authpin=1 0x563a6a4a4c80] 2023-11-15T03:02:39.961 INFO:tasks.ceph.mds.d.smithi190.stderr: -2> 2023-11-15T03:02:39.916+0000 7f9deadf6700 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by 0x563a6a3f0800) (dversion lock w=1 last_client=6687) pv=11 v=10 ap=2 ino=(nil) state=1073741888|bottomlru | request=1 lock=2 authpin=1 0x563a6a4a4c80] 2023-11-15T03:02:39.962 INFO:tasks.ceph.mds.d.smithi190.stderr: -1> 2023-11-15T03:02:39.920+0000 7f9deadf6700 -1 /build/ceph-16.2.14-170-g620f25c1/src/mds/CDentry.cc: In function 'bool CDentry::check_corruption(bool)' thread 7f9deadf6700 time 2023-11-15T03:02:39.919174+0000 2023-11-15T03:02:39.962 INFO:tasks.ceph.mds.d.smithi190.stderr:/build/ceph-16.2.14-170-g620f25c1/src/mds/CDentry.cc: 699: ceph_abort_msg("abort() called") 2023-11-15T03:02:39.962 INFO:tasks.ceph.mds.d.smithi190.stderr: 2023-11-15T03:02:39.963 INFO:tasks.ceph.mds.d.smithi190.stderr: ceph version 16.2.14-170-g620f25c1 (620f25c1794f62207424fc9319427943fb8d1fae) pacific (stable) 2023-11-15T03:02:39.963 INFO:tasks.ceph.mds.d.smithi190.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe0) [0x7f9df02684b5] 2023-11-15T03:02:39.963 INFO:tasks.ceph.mds.d.smithi190.stderr: 2: (CDentry::check_corruption(bool)+0x86b) [0x563a67c49cbb] 2023-11-15T03:02:39.963 INFO:tasks.ceph.mds.d.smithi190.stderr: 3: (EMetaBlob::add_primary_dentry(EMetaBlob::dirlump&, CDentry*, CInode*, unsigned char)+0x47) [0x563a67a97e37] 2023-11-15T03:02:39.963 INFO:tasks.ceph.mds.d.smithi190.stderr: 4: (EMetaBlob::add_primary_dentry(CDentry*, CInode*, bool, bool, bool, bool)+0x1b5) [0x563a67a99425] 2023-11-15T03:02:39.964 INFO:tasks.ceph.mds.d.smithi190.stderr: 5: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x122e) [0x563a67a5e80e] 2023-11-15T03:02:39.964 INFO:tasks.ceph.mds.d.smithi190.stderr: 6: (Server::handle_client_rename(boost::intrusive_ptr&)+0x2a94) [0x563a67a63b74] 2023-11-15T03:02:39.964 INFO:tasks.ceph.mds.d.smithi190.stderr: 7: (Server::dispatch_client_request(boost::intrusive_ptr&)+0x9d0) [0x563a67a75c40] 2023-11-15T03:02:39.964 INFO:tasks.ceph.mds.d.smithi190.stderr: 8: (Server::handle_client_request(boost::intrusive_ptr const&)+0x605) [0x563a67a769e5] 2023-11-15T03:02:39.964 INFO:tasks.ceph.mds.d.smithi190.stderr: 9: (Server::dispatch(boost::intrusive_ptr const&)+0x12c) [0x563a67a7b88c] 2023-11-15T03:02:39.965 INFO:tasks.ceph.mds.d.smithi190.stderr: 10: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x3eb) [0x563a679b78ab] 2023-11-15T03:02:39.965 INFO:tasks.ceph.mds.d.smithi190.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x643) [0x563a679d27e3] 2023-11-15T03:02:39.965 INFO:tasks.ceph.mds.d.smithi190.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5e) [0x563a679d2e7e] 2023-11-15T03:02:39.965 INFO:tasks.ceph.mds.d.smithi190.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1d6) [0x563a679a93c6] 2023-11-15T03:02:39.965 INFO:tasks.ceph.mds.d.smithi190.stderr: 14: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f9df04ba640] 2023-11-15T03:02:39.965 INFO:tasks.ceph.mds.d.smithi190.stderr: 15: (DispatchQueue::entry()+0x58f) [0x7f9df04b7edf] 2023-11-15T03:02:39.966 INFO:tasks.ceph.mds.d.smithi190.stderr: 16: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9df0586c91] 2023-11-15T03:02:39.966 INFO:tasks.ceph.mds.d.smithi190.stderr: 17: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f9df8d81609] 2023-11-15T03:02:39.966 INFO:tasks.ceph.mds.d.smithi190.stderr: 18: clone() 2023-11-15T03:02:39.966 INFO:tasks.ceph.mds.d.smithi190.stderr: 2023-11-15T03:02:39.966 INFO:tasks.ceph.mds.d.smithi190.stderr: 0> 2023-11-15T03:02:39.924+0000 7f9deadf6700 -1 *** Caught signal (Aborted) ** 2023-11-15T03:02:39.967 INFO:tasks.ceph.mds.d.smithi190.stderr: in thread 7f9deadf6700 thread_name:ms_dispatch 2023-11-15T03:02:39.967 INFO:tasks.ceph.mds.d.smithi190.stderr: 2023-11-15T03:02:39.967 INFO:tasks.ceph.mds.d.smithi190.stderr: ceph version 16.2.14-170-g620f25c1 (620f25c1794f62207424fc9319427943fb8d1fae) pacific (stable) 2023-11-15T03:02:39.967 INFO:tasks.ceph.mds.d.smithi190.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f9df8d8d420] 2023-11-15T03:02:39.967 INFO:tasks.ceph.mds.d.smithi190.stderr: 2: gsignal() 2023-11-15T03:02:39.968 INFO:tasks.ceph.mds.d.smithi190.stderr: 3: abort() 2023-11-15T03:02:39.968 INFO:tasks.ceph.mds.d.smithi190.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x1af) [0x7f9df0268584] 2023-11-15T03:02:39.968 INFO:tasks.ceph.mds.d.smithi190.stderr: 5: (CDentry::check_corruption(bool)+0x86b) [0x563a67c49cbb] 2023-11-15T03:02:39.968 INFO:tasks.ceph.mds.d.smithi190.stderr: 6: (EMetaBlob::add_primary_dentry(EMetaBlob::dirlump&, CDentry*, CInode*, unsigned char)+0x47) [0x563a67a97e37] 2023-11-15T03:02:39.968 INFO:tasks.ceph.mds.d.smithi190.stderr: 7: (EMetaBlob::add_primary_dentry(CDentry*, CInode*, bool, bool, bool, bool)+0x1b5) [0x563a67a99425] 2023-11-15T03:02:39.969 INFO:tasks.ceph.mds.d.smithi190.stderr: 8: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x122e) [0x563a67a5e80e] 2023-11-15T03:02:39.969 INFO:tasks.ceph.mds.d.smithi190.stderr: 9: (Server::handle_client_rename(boost::intrusive_ptr&)+0x2a94) [0x563a67a63b74] 2023-11-15T03:02:39.969 INFO:tasks.ceph.mds.d.smithi190.stderr: 10: (Server::dispatch_client_request(boost::intrusive_ptr&)+0x9d0) [0x563a67a75c40] 2023-11-15T03:02:39.969 INFO:tasks.ceph.mds.d.smithi190.stderr: 11: (Server::handle_client_request(boost::intrusive_ptr const&)+0x605) [0x563a67a769e5] 2023-11-15T03:02:39.969 INFO:tasks.ceph.mds.d.smithi190.stderr: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x12c) [0x563a67a7b88c] 2023-11-15T03:02:39.969 INFO:tasks.ceph.mds.d.smithi190.stderr: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x3eb) [0x563a679b78ab] 2023-11-15T03:02:39.970 INFO:tasks.ceph.mds.d.smithi190.stderr: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x643) [0x563a679d27e3] 2023-11-15T03:02:39.970 INFO:tasks.ceph.mds.d.smithi190.stderr: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5e) [0x563a679d2e7e] 2023-11-15T03:02:39.970 INFO:tasks.ceph.mds.d.smithi190.stderr: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1d6) [0x563a679a93c6] 2023-11-15T03:02:39.971 INFO:tasks.ceph.mds.d.smithi190.stderr: 17: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f9df04ba640] 2023-11-15T03:02:39.971 INFO:tasks.ceph.mds.d.smithi190.stderr: 18: (DispatchQueue::entry()+0x58f) [0x7f9df04b7edf] 2023-11-15T03:02:39.971 INFO:tasks.ceph.mds.d.smithi190.stderr: 19: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9df0586c91] 2023-11-15T03:02:39.971 INFO:tasks.ceph.mds.d.smithi190.stderr: 20: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f9df8d81609] 2023-11-15T03:02:39.971 INFO:tasks.ceph.mds.d.smithi190.stderr: 21: clone() 2023-11-15T03:02:39.972 INFO:tasks.ceph.mds.d.smithi190.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-15T03:02:39.972 INFO:tasks.ceph.mds.d.smithi190.stderr: 2023-11-15T03:02:40.105 INFO:teuthology.orchestra.run.smithi121.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-15T03:02:40.106 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-15T03:02:40.107 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:02:40.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.258+0000 7eff4d946700 1 Processor -- start 2023-11-15T03:02:40.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.258+0000 7eff4d946700 1 -- start start 2023-11-15T03:02:40.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.262+0000 7eff4d946700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7eff480c28d0 0x7eff48159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:40.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.262+0000 7eff4d946700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff480c3fa0 con 0x7eff48152780 2023-11-15T03:02:40.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.262+0000 7eff4d946700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff48159630 con 0x7eff480c6ce0 2023-11-15T03:02:40.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.262+0000 7eff4d946700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff480c30b0 con 0x7eff480c28d0 2023-11-15T03:02:40.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.262+0000 7eff477fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7eff480c28d0 0x7eff48159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:40.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.262+0000 7eff477fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7eff480c28d0 0x7eff48159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53206/0 (socket says 172.21.15.121:53206) 2023-11-15T03:02:40.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.262+0000 7eff477fe700 1 -- 172.21.15.121:0/1003081516 learned_addr learned my addr 172.21.15.121:0/1003081516 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:40.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.262+0000 7eff46ffd700 1 --1- 172.21.15.121:0/1003081516 >> v1:172.21.15.121:6789/0 conn(0x7eff48152780 0x7eff480c65d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39118/0 (socket says 172.21.15.121:39118) 2023-11-15T03:02:40.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.262+0000 7eff477fe700 1 -- 172.21.15.121:0/1003081516 >> v1:172.21.15.190:6789/0 conn(0x7eff480c6ce0 legacy=0x7eff480a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:40.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.262+0000 7eff477fe700 1 -- 172.21.15.121:0/1003081516 >> v1:172.21.15.121:6789/0 conn(0x7eff48152780 legacy=0x7eff480c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:40.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.262+0000 7eff477fe700 1 -- 172.21.15.121:0/1003081516 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff480c3390 con 0x7eff480c28d0 2023-11-15T03:02:40.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.262+0000 7eff45ffb700 1 -- 172.21.15.121:0/1003081516 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167237404 0 0) 0x7eff480c3fa0 con 0x7eff48152780 2023-11-15T03:02:40.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.262+0000 7eff477fe700 1 --2- 172.21.15.121:0/1003081516 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff480c28d0 0x7eff48159140 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7eff40006fd0 tx=0x7eff400090f0).ready entity=mon.0 client_cookie=6e376f6dba0cf773 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:40.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.262+0000 7eff45ffb700 1 -- 172.21.15.121:0/1003081516 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4000b600 con 0x7eff480c28d0 2023-11-15T03:02:40.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.262+0000 7eff45ffb700 1 -- 172.21.15.121:0/1003081516 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4000b760 con 0x7eff480c28d0 2023-11-15T03:02:40.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.262+0000 7eff45ffb700 1 -- 172.21.15.121:0/1003081516 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4000bae0 con 0x7eff480c28d0 2023-11-15T03:02:40.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.266+0000 7eff4d946700 1 -- 172.21.15.121:0/1003081516 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff480c28d0 msgr2=0x7eff48159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:40.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.266+0000 7eff4d946700 1 --2- 172.21.15.121:0/1003081516 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff480c28d0 0x7eff48159140 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7eff40006fd0 tx=0x7eff400090f0).stop 2023-11-15T03:02:40.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.266+0000 7eff4d946700 1 -- 172.21.15.121:0/1003081516 shutdown_connections 2023-11-15T03:02:40.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.266+0000 7eff4d946700 1 --2- 172.21.15.121:0/1003081516 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff480c28d0 0x7eff48159140 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:40.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.266+0000 7eff4d946700 1 -- 172.21.15.121:0/1003081516 >> 172.21.15.121:0/1003081516 conn(0x7eff48062190 msgr2=0x7eff480625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:40.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.266+0000 7eff4d946700 1 -- 172.21.15.121:0/1003081516 shutdown_connections 2023-11-15T03:02:40.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.266+0000 7eff4d946700 1 -- 172.21.15.121:0/1003081516 wait complete. 2023-11-15T03:02:40.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.266+0000 7eff4d946700 1 Processor -- start 2023-11-15T03:02:40.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.266+0000 7eff4d946700 1 -- start start 2023-11-15T03:02:40.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff4d946700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff480c28d0 0x7eff4814b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:40.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff4d946700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff480c6ce0 0x7eff4814b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:40.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff4d946700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff48152780 0x7eff48168500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:40.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff4d946700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4814c0c0 con 0x7eff48152780 2023-11-15T03:02:40.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff4d946700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4814bde0 con 0x7eff480c28d0 2023-11-15T03:02:40.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff4d946700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff48169840 con 0x7eff480c6ce0 2023-11-15T03:02:40.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff477fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff48152780 0x7eff48168500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:40.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff477fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff48152780 0x7eff48168500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53214/0 (socket says 172.21.15.121:53214) 2023-11-15T03:02:40.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff477fe700 1 -- 172.21.15.121:0/563055599 learned_addr learned my addr 172.21.15.121:0/563055599 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:40.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff46ffd700 1 --2- 172.21.15.121:0/563055599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff480c28d0 0x7eff4814b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:40.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff467fc700 1 --2- 172.21.15.121:0/563055599 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff480c6ce0 0x7eff4814b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:40.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff477fe700 1 -- 172.21.15.121:0/563055599 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff480c6ce0 msgr2=0x7eff4814b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:40.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff477fe700 1 --2- 172.21.15.121:0/563055599 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff480c6ce0 0x7eff4814b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:40.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff477fe700 1 -- 172.21.15.121:0/563055599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff480c28d0 msgr2=0x7eff4814b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:40.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff477fe700 1 --2- 172.21.15.121:0/563055599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff480c28d0 0x7eff4814b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:40.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff477fe700 1 -- 172.21.15.121:0/563055599 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff40009370 con 0x7eff48152780 2023-11-15T03:02:40.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff467fc700 1 --2- 172.21.15.121:0/563055599 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff480c6ce0 0x7eff4814b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:02:40.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff477fe700 1 --2- 172.21.15.121:0/563055599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff48152780 0x7eff48168500 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7eff48064f10 tx=0x7eff40017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:40.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff2effd700 1 -- 172.21.15.121:0/563055599 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4000bae0 con 0x7eff48152780 2023-11-15T03:02:40.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff2effd700 1 -- 172.21.15.121:0/563055599 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4001c420 con 0x7eff48152780 2023-11-15T03:02:40.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff4d946700 1 -- 172.21.15.121:0/563055599 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff48169b20 con 0x7eff48152780 2023-11-15T03:02:40.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff2effd700 1 -- 172.21.15.121:0/563055599 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4001c7a0 con 0x7eff48152780 2023-11-15T03:02:40.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.270+0000 7eff4d946700 1 -- 172.21.15.121:0/563055599 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff4814ba40 con 0x7eff48152780 2023-11-15T03:02:40.290 INFO:tasks.ceph.mds.d.smithi190.stderr:daemon-helper: command crashed with signal 6 2023-11-15T03:02:40.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.274+0000 7eff4d946700 1 -- 172.21.15.121:0/563055599 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff4806c150 con 0x7eff48152780 2023-11-15T03:02:40.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.274+0000 7eff2effd700 1 -- 172.21.15.121:0/563055599 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7eff4001cb90 con 0x7eff48152780 2023-11-15T03:02:40.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.274+0000 7eff2effd700 1 --2- 172.21.15.121:0/563055599 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7eff3405a510 0x7eff3405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:40.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.274+0000 7eff2effd700 1 -- 172.21.15.121:0/563055599 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6179+0+0 (secure 0 0 0) 0x7eff40076430 con 0x7eff48152780 2023-11-15T03:02:40.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.274+0000 7eff46ffd700 1 --2- 172.21.15.121:0/563055599 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7eff3405a510 0x7eff3405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:40.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.278+0000 7eff46ffd700 1 --2- 172.21.15.121:0/563055599 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7eff3405a510 0x7eff3405c9d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7eff4816ba70 tx=0x7eff38007c30).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:40.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.278+0000 7eff2effd700 1 -- 172.21.15.121:0/563055599 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7eff40049b10 con 0x7eff48152780 2023-11-15T03:02:40.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.478+0000 7eff4d946700 1 -- 172.21.15.121:0/563055599 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff480c11b0 con 0x7eff48152780 2023-11-15T03:02:40.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.478+0000 7eff2effd700 1 -- 172.21.15.121:0/563055599 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4497 (secure 0 0 0) 0x7eff40006bf0 con 0x7eff48152780 2023-11-15T03:02:40.490 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:40.490 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":68,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1508681155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1508681155},{"type":"v1","addr":"172.21.15.121:6837","nonce":1508681155}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6385,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3559276534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3559276534},{"type":"v1","addr":"172.21.15.190:6835","nonce":3559276534}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6486,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/834528709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":834528709},{"type":"v1","addr":"172.21.15.121:6836","nonce":834528709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:02:20.703078+0000","modified":"2023-11-15T03:02:34.118521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6492},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6492":{"gid":6492,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6833/914796342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":914796342},{"type":"v1","addr":"172.21.15.190:6833","nonce":914796342}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-15T03:02:40.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.482+0000 7eff4d946700 1 -- 172.21.15.121:0/563055599 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7eff3405a510 msgr2=0x7eff3405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:40.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.482+0000 7eff4d946700 1 --2- 172.21.15.121:0/563055599 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7eff3405a510 0x7eff3405c9d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7eff4816ba70 tx=0x7eff38007c30).stop 2023-11-15T03:02:40.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.482+0000 7eff4d946700 1 -- 172.21.15.121:0/563055599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff48152780 msgr2=0x7eff48168500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:40.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.482+0000 7eff4d946700 1 --2- 172.21.15.121:0/563055599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff48152780 0x7eff48168500 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7eff48064f10 tx=0x7eff40017000).stop 2023-11-15T03:02:40.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.482+0000 7eff4d946700 1 -- 172.21.15.121:0/563055599 shutdown_connections 2023-11-15T03:02:40.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.482+0000 7eff4d946700 1 --2- 172.21.15.121:0/563055599 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7eff3405a510 0x7eff3405c9d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:40.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.482+0000 7eff4d946700 1 --2- 172.21.15.121:0/563055599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff48152780 0x7eff48168500 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:40.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.482+0000 7eff4d946700 1 --2- 172.21.15.121:0/563055599 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff480c6ce0 0x7eff4814b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:40.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.482+0000 7eff4d946700 1 --2- 172.21.15.121:0/563055599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff480c28d0 0x7eff4814b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:40.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.482+0000 7eff4d946700 1 -- 172.21.15.121:0/563055599 >> 172.21.15.121:0/563055599 conn(0x7eff48062190 msgr2=0x7eff48067030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:40.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.482+0000 7eff4d946700 1 -- 172.21.15.121:0/563055599 shutdown_connections 2023-11-15T03:02:40.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:40.482+0000 7eff4d946700 1 -- 172.21.15.121:0/563055599 wait complete. 2023-11-15T03:02:40.494 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 68 2023-11-15T03:02:40.510 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-15T03:02:41.046 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:02:41.053 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:02:45.512 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:02:45.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.674+0000 7f54daadb700 1 Processor -- start 2023-11-15T03:02:45.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.674+0000 7f54daadb700 1 -- start start 2023-11-15T03:02:45.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.678+0000 7f54daadb700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f54d414af80 0x7f54d40c1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:45.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.678+0000 7f54daadb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f54d40c1bf0 0x7f54d4150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:45.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.678+0000 7f54daadb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54d41514e0 0x7f54d40c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:45.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.678+0000 7f54daadb700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d4152000 con 0x7f54d414af80 2023-11-15T03:02:45.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.678+0000 7f54daadb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d40c1270 con 0x7f54d40c1bf0 2023-11-15T03:02:45.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.678+0000 7f54daadb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d4151a40 con 0x7f54d41514e0 2023-11-15T03:02:45.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.678+0000 7f54d8877700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f54d414af80 0x7f54d40c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:45.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.678+0000 7f54d8877700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f54d414af80 0x7f54d40c1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41558/0 (socket says 172.21.15.121:41558) 2023-11-15T03:02:45.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.678+0000 7f54d8877700 1 -- 172.21.15.121:0/39720579 learned_addr learned my addr 172.21.15.121:0/39720579 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:45.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.678+0000 7f54d9078700 1 --2- 172.21.15.121:0/39720579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54d41514e0 0x7f54d40c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:45.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.678+0000 7f54d3fff700 1 --2- 172.21.15.121:0/39720579 >> v2:172.21.15.190:3300/0 conn(0x7f54d40c1bf0 0x7f54d4150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:45.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54d8877700 1 -- 172.21.15.121:0/39720579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54d41514e0 msgr2=0x7f54d40c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:45.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54d8877700 1 --2- 172.21.15.121:0/39720579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54d41514e0 0x7f54d40c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:45.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54d8877700 1 -- 172.21.15.121:0/39720579 >> v2:172.21.15.190:3300/0 conn(0x7f54d40c1bf0 msgr2=0x7f54d4150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:45.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54d8877700 1 --2- 172.21.15.121:0/39720579 >> v2:172.21.15.190:3300/0 conn(0x7f54d40c1bf0 0x7f54d4150ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:45.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54d8877700 1 -- 172.21.15.121:0/39720579 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d4151d20 con 0x7f54d414af80 2023-11-15T03:02:45.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54d9078700 1 --2- 172.21.15.121:0/39720579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54d41514e0 0x7f54d40c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:45.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54d3fff700 1 --2- 172.21.15.121:0/39720579 >> v2:172.21.15.190:3300/0 conn(0x7f54d40c1bf0 0x7f54d4150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:45.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54d8877700 1 --2- 172.21.15.121:0/39720579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54d414af80 0x7f54d40c1700 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f54d40c7590 tx=0x7f54c800b3b0).ready entity=mon.0 client_cookie=968a02b7234ad3db server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:45.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54d37fe700 1 -- 172.21.15.121:0/39720579 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c800db10 con 0x7f54d414af80 2023-11-15T03:02:45.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54d37fe700 1 -- 172.21.15.121:0/39720579 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c800dc70 con 0x7f54d414af80 2023-11-15T03:02:45.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54d37fe700 1 -- 172.21.15.121:0/39720579 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c8010470 con 0x7f54d414af80 2023-11-15T03:02:45.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54daadb700 1 -- 172.21.15.121:0/39720579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54d414af80 msgr2=0x7f54d40c1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:45.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54daadb700 1 --2- 172.21.15.121:0/39720579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54d414af80 0x7f54d40c1700 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f54d40c7590 tx=0x7f54c800b3b0).stop 2023-11-15T03:02:45.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54daadb700 1 -- 172.21.15.121:0/39720579 shutdown_connections 2023-11-15T03:02:45.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54daadb700 1 --2- 172.21.15.121:0/39720579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54d41514e0 0x7f54d40c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:45.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54daadb700 1 --2- 172.21.15.121:0/39720579 >> v2:172.21.15.190:3300/0 conn(0x7f54d40c1bf0 0x7f54d4150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:45.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54daadb700 1 --2- 172.21.15.121:0/39720579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54d414af80 0x7f54d40c1700 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:45.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54daadb700 1 -- 172.21.15.121:0/39720579 >> 172.21.15.121:0/39720579 conn(0x7f54d4062190 msgr2=0x7f54d40625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:45.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54daadb700 1 -- 172.21.15.121:0/39720579 shutdown_connections 2023-11-15T03:02:45.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.682+0000 7f54daadb700 1 -- 172.21.15.121:0/39720579 wait complete. 2023-11-15T03:02:45.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.686+0000 7f54daadb700 1 Processor -- start 2023-11-15T03:02:45.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.686+0000 7f54daadb700 1 -- start start 2023-11-15T03:02:45.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54daadb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54d40c1bf0 0x7f54d414cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:45.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54daadb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54d414af80 0x7f54d414b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:45.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54daadb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54d41514e0 0x7f54d414b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:45.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54daadb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d414ca30 con 0x7f54d41514e0 2023-11-15T03:02:45.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54daadb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d414be20 con 0x7f54d414af80 2023-11-15T03:02:45.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54daadb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d414c100 con 0x7f54d40c1bf0 2023-11-15T03:02:45.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54d9078700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54d41514e0 0x7f54d414b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:45.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54d9078700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54d41514e0 0x7f54d414b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41572/0 (socket says 172.21.15.121:41572) 2023-11-15T03:02:45.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54d9078700 1 -- 172.21.15.121:0/1467806147 learned_addr learned my addr 172.21.15.121:0/1467806147 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:45.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54d8877700 1 --2- 172.21.15.121:0/1467806147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54d40c1bf0 0x7f54d414cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:45.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54d3fff700 1 --2- 172.21.15.121:0/1467806147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54d414af80 0x7f54d414b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:45.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54d9078700 1 -- 172.21.15.121:0/1467806147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54d40c1bf0 msgr2=0x7f54d414cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:45.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54d9078700 1 --2- 172.21.15.121:0/1467806147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54d40c1bf0 0x7f54d414cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:45.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54d9078700 1 -- 172.21.15.121:0/1467806147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54d414af80 msgr2=0x7f54d414b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:45.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54d9078700 1 --2- 172.21.15.121:0/1467806147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54d414af80 0x7f54d414b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:45.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54d9078700 1 -- 172.21.15.121:0/1467806147 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c800b610 con 0x7f54d41514e0 2023-11-15T03:02:45.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54d9078700 1 --2- 172.21.15.121:0/1467806147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54d41514e0 0x7f54d414b850 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f54cc007f10 tx=0x7f54cc009930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:45.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54d0ff9700 1 -- 172.21.15.121:0/1467806147 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc00db80 con 0x7f54d41514e0 2023-11-15T03:02:45.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54d0ff9700 1 -- 172.21.15.121:0/1467806147 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54cc00f040 con 0x7f54d41514e0 2023-11-15T03:02:45.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54daadb700 1 -- 172.21.15.121:0/1467806147 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d414c3e0 con 0x7f54d41514e0 2023-11-15T03:02:45.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54d0ff9700 1 -- 172.21.15.121:0/1467806147 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc0133f0 con 0x7f54d41514e0 2023-11-15T03:02:45.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.690+0000 7f54daadb700 1 -- 172.21.15.121:0/1467806147 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d41533f0 con 0x7f54d41514e0 2023-11-15T03:02:45.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.694+0000 7f54daadb700 1 -- 172.21.15.121:0/1467806147 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d406c150 con 0x7f54d41514e0 2023-11-15T03:02:45.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.694+0000 7f54d0ff9700 1 -- 172.21.15.121:0/1467806147 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f54cc0137e0 con 0x7f54d41514e0 2023-11-15T03:02:45.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.698+0000 7f54d0ff9700 1 --2- 172.21.15.121:0/1467806147 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f54b805a510 0x7f54b805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:45.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.698+0000 7f54d0ff9700 1 -- 172.21.15.121:0/1467806147 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6179+0+0 (secure 0 0 0) 0x7f54cc074e10 con 0x7f54d41514e0 2023-11-15T03:02:45.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.698+0000 7f54d8877700 1 --2- 172.21.15.121:0/1467806147 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f54b805a510 0x7f54b805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:45.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.698+0000 7f54d8877700 1 --2- 172.21.15.121:0/1467806147 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f54b805a510 0x7f54b805c9d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f54c800bfd0 tx=0x7f54c800b3b0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:45.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.702+0000 7f54d0ff9700 1 -- 172.21.15.121:0/1467806147 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f54cc078030 con 0x7f54d41514e0 2023-11-15T03:02:45.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.906+0000 7f54daadb700 1 -- 172.21.15.121:0/1467806147 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54d414cb70 con 0x7f54d41514e0 2023-11-15T03:02:45.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.906+0000 7f54d0ff9700 1 -- 172.21.15.121:0/1467806147 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f54cc0484f0 con 0x7f54d41514e0 2023-11-15T03:02:45.915 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:45.916 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":68,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1508681155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1508681155},{"type":"v1","addr":"172.21.15.121:6837","nonce":1508681155}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6385,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3559276534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3559276534},{"type":"v1","addr":"172.21.15.190:6835","nonce":3559276534}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6486,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/834528709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":834528709},{"type":"v1","addr":"172.21.15.121:6836","nonce":834528709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:02:20.703078+0000","modified":"2023-11-15T03:02:34.118521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6492},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6492":{"gid":6492,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6833/914796342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":914796342},{"type":"v1","addr":"172.21.15.190:6833","nonce":914796342}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-15T03:02:45.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.910+0000 7f54daadb700 1 -- 172.21.15.121:0/1467806147 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f54b805a510 msgr2=0x7f54b805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:45.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.910+0000 7f54daadb700 1 --2- 172.21.15.121:0/1467806147 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f54b805a510 0x7f54b805c9d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f54c800bfd0 tx=0x7f54c800b3b0).stop 2023-11-15T03:02:45.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.910+0000 7f54daadb700 1 -- 172.21.15.121:0/1467806147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54d41514e0 msgr2=0x7f54d414b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:45.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.910+0000 7f54daadb700 1 --2- 172.21.15.121:0/1467806147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54d41514e0 0x7f54d414b850 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f54cc007f10 tx=0x7f54cc009930).stop 2023-11-15T03:02:45.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.910+0000 7f54daadb700 1 -- 172.21.15.121:0/1467806147 shutdown_connections 2023-11-15T03:02:45.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.910+0000 7f54daadb700 1 --2- 172.21.15.121:0/1467806147 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f54b805a510 0x7f54b805c9d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:45.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.910+0000 7f54daadb700 1 --2- 172.21.15.121:0/1467806147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54d41514e0 0x7f54d414b850 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:45.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.910+0000 7f54daadb700 1 --2- 172.21.15.121:0/1467806147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54d414af80 0x7f54d414b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:45.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.910+0000 7f54daadb700 1 --2- 172.21.15.121:0/1467806147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54d40c1bf0 0x7f54d414cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:45.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.910+0000 7f54daadb700 1 -- 172.21.15.121:0/1467806147 >> 172.21.15.121:0/1467806147 conn(0x7f54d4062190 msgr2=0x7f54d40c23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:45.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.910+0000 7f54daadb700 1 -- 172.21.15.121:0/1467806147 shutdown_connections 2023-11-15T03:02:45.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:45.910+0000 7f54daadb700 1 -- 172.21.15.121:0/1467806147 wait complete. 2023-11-15T03:02:45.921 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 68 2023-11-15T03:02:45.938 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-15T03:02:46.812 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-11-15T03:02:50.939 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:02:51.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.102+0000 7fe3dd28f700 1 Processor -- start 2023-11-15T03:02:51.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.102+0000 7fe3dd28f700 1 -- start start 2023-11-15T03:02:51.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.102+0000 7fe3dd28f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe3d8151870 0x7fe3d8151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:51.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.102+0000 7fe3dd28f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3d8152140 0x7fe3d80a9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:51.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.102+0000 7fe3dd28f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3d80657f0 con 0x7fe3d80a9d20 2023-11-15T03:02:51.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.102+0000 7fe3dd28f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3d8065ad0 con 0x7fe3d8151870 2023-11-15T03:02:51.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.102+0000 7fe3dd28f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d80aa260 con 0x7fe3d8152140 2023-11-15T03:02:51.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.106+0000 7fe3dca8e700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe3d80a9d20 0x7fe3d80c30e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:46910/0 (socket says 172.21.15.121:46910) 2023-11-15T03:02:51.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.106+0000 7fe3dca8e700 1 -- 172.21.15.121:0/52742573 learned_addr learned my addr 172.21.15.121:0/52742573 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:51.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.106+0000 7fe3cf7fe700 1 --2- 172.21.15.121:0/52742573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3d8152140 0x7fe3d80a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:51.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.106+0000 7fe3d7fff700 1 --2- 172.21.15.121:0/52742573 >> v2:172.21.15.190:3300/0 conn(0x7fe3d8151870 0x7fe3d8151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:51.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.106+0000 7fe3d77fe700 1 -- 172.21.15.121:0/52742573 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1086109846 0 0) 0x7fe3d80657f0 con 0x7fe3d80a9d20 2023-11-15T03:02:51.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.106+0000 7fe3d77fe700 1 -- 172.21.15.121:0/52742573 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3c40033f0 con 0x7fe3d80a9d20 2023-11-15T03:02:51.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.106+0000 7fe3cf7fe700 1 -- 172.21.15.121:0/52742573 >> v2:172.21.15.190:3300/0 conn(0x7fe3d8151870 msgr2=0x7fe3d8151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:51.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.106+0000 7fe3cf7fe700 1 --2- 172.21.15.121:0/52742573 >> v2:172.21.15.190:3300/0 conn(0x7fe3d8151870 0x7fe3d8151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:51.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.106+0000 7fe3cf7fe700 1 -- 172.21.15.121:0/52742573 >> v1:172.21.15.121:6789/0 conn(0x7fe3d80a9d20 legacy=0x7fe3d80c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:51.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.106+0000 7fe3cf7fe700 1 -- 172.21.15.121:0/52742573 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d80aa540 con 0x7fe3d8152140 2023-11-15T03:02:51.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.106+0000 7fe3cf7fe700 1 --2- 172.21.15.121:0/52742573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3d8152140 0x7fe3d80a9830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fe3c0009760 tx=0x7fe3c0009060).ready entity=mon.2 client_cookie=473a6b44fbc8a09b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:51.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.110+0000 7fe3d77fe700 1 -- 172.21.15.121:0/52742573 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c000d4a0 con 0x7fe3d8152140 2023-11-15T03:02:51.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.110+0000 7fe3d77fe700 1 -- 172.21.15.121:0/52742573 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c000d600 con 0x7fe3d8152140 2023-11-15T03:02:51.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.110+0000 7fe3d77fe700 1 -- 172.21.15.121:0/52742573 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c000d980 con 0x7fe3d8152140 2023-11-15T03:02:51.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.110+0000 7fe3dd28f700 1 -- 172.21.15.121:0/52742573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3d8152140 msgr2=0x7fe3d80a9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:51.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.110+0000 7fe3dd28f700 1 --2- 172.21.15.121:0/52742573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3d8152140 0x7fe3d80a9830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fe3c0009760 tx=0x7fe3c0009060).stop 2023-11-15T03:02:51.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.110+0000 7fe3dd28f700 1 -- 172.21.15.121:0/52742573 shutdown_connections 2023-11-15T03:02:51.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.110+0000 7fe3dd28f700 1 --2- 172.21.15.121:0/52742573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3d8152140 0x7fe3d80a9830 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:51.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.110+0000 7fe3dd28f700 1 --2- 172.21.15.121:0/52742573 >> v2:172.21.15.190:3300/0 conn(0x7fe3d8151870 0x7fe3d8151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:51.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.110+0000 7fe3dd28f700 1 -- 172.21.15.121:0/52742573 >> 172.21.15.121:0/52742573 conn(0x7fe3d8062190 msgr2=0x7fe3d8062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:51.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.110+0000 7fe3dd28f700 1 -- 172.21.15.121:0/52742573 shutdown_connections 2023-11-15T03:02:51.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.110+0000 7fe3dd28f700 1 -- 172.21.15.121:0/52742573 wait complete. 2023-11-15T03:02:51.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.114+0000 7fe3dd28f700 1 Processor -- start 2023-11-15T03:02:51.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.114+0000 7fe3dd28f700 1 -- start start 2023-11-15T03:02:51.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3dd28f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3d80a9d20 0x7fe3d80bfc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:51.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3dd28f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3d8151870 0x7fe3d80c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:51.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3dd28f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3d8152140 0x7fe3d80c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:51.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3dd28f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d80c44e0 con 0x7fe3d8151870 2023-11-15T03:02:51.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3dd28f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d80c0d50 con 0x7fe3d8152140 2023-11-15T03:02:51.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3dd28f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d80c72a0 con 0x7fe3d80a9d20 2023-11-15T03:02:51.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3cf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3d8151870 0x7fe3d80c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:51.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3cf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3d8151870 0x7fe3d80c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41584/0 (socket says 172.21.15.121:41584) 2023-11-15T03:02:51.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3cf7fe700 1 -- 172.21.15.121:0/615437783 learned_addr learned my addr 172.21.15.121:0/615437783 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:51.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3dca8e700 1 --2- 172.21.15.121:0/615437783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3d8152140 0x7fe3d80c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:51.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3d7fff700 1 --2- 172.21.15.121:0/615437783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3d80a9d20 0x7fe3d80bfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:51.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3cf7fe700 1 -- 172.21.15.121:0/615437783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3d80a9d20 msgr2=0x7fe3d80bfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:51.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3cf7fe700 1 --2- 172.21.15.121:0/615437783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3d80a9d20 0x7fe3d80bfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:51.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3cf7fe700 1 -- 172.21.15.121:0/615437783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3d8152140 msgr2=0x7fe3d80c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:51.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3cf7fe700 1 --2- 172.21.15.121:0/615437783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3d8152140 0x7fe3d80c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:51.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3cf7fe700 1 -- 172.21.15.121:0/615437783 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c00092a0 con 0x7fe3d8151870 2023-11-15T03:02:51.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3dca8e700 1 --2- 172.21.15.121:0/615437783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3d8152140 0x7fe3d80c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:02:51.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3cf7fe700 1 --2- 172.21.15.121:0/615437783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3d8151870 0x7fe3d80c6700 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fe3d8065200 tx=0x7fe3c0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:51.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3d4ff9700 1 -- 172.21.15.121:0/615437783 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c000dd30 con 0x7fe3d8151870 2023-11-15T03:02:51.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3d4ff9700 1 -- 172.21.15.121:0/615437783 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c000de90 con 0x7fe3d8151870 2023-11-15T03:02:51.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3dd28f700 1 -- 172.21.15.121:0/615437783 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d80c46f0 con 0x7fe3d8151870 2023-11-15T03:02:51.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3d4ff9700 1 -- 172.21.15.121:0/615437783 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c000d980 con 0x7fe3d8151870 2023-11-15T03:02:51.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3dd28f700 1 -- 172.21.15.121:0/615437783 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d80c4cb0 con 0x7fe3d8151870 2023-11-15T03:02:51.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.118+0000 7fe3dd28f700 1 -- 172.21.15.121:0/615437783 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d80c4ed0 con 0x7fe3d8151870 2023-11-15T03:02:51.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.122+0000 7fe3d4ff9700 1 -- 172.21.15.121:0/615437783 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe3c001d640 con 0x7fe3d8151870 2023-11-15T03:02:51.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.122+0000 7fe3d4ff9700 1 --2- 172.21.15.121:0/615437783 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe3b805a4e0 0x7fe3b805c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:51.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.122+0000 7fe3d4ff9700 1 -- 172.21.15.121:0/615437783 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6179+0+0 (secure 0 0 0) 0x7fe3c0075f40 con 0x7fe3d8151870 2023-11-15T03:02:51.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.122+0000 7fe3d7fff700 1 --2- 172.21.15.121:0/615437783 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe3b805a4e0 0x7fe3b805c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:51.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.122+0000 7fe3d7fff700 1 --2- 172.21.15.121:0/615437783 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe3b805a4e0 0x7fe3b805c9a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7fe3c8009c10 tx=0x7fe3c8009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:51.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.130+0000 7fe3d4ff9700 1 -- 172.21.15.121:0/615437783 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe3c0049620 con 0x7fe3d8151870 2023-11-15T03:02:51.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.338+0000 7fe3dd28f700 1 -- 172.21.15.121:0/615437783 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3d80aa820 con 0x7fe3d8151870 2023-11-15T03:02:51.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.338+0000 7fe3d4ff9700 1 -- 172.21.15.121:0/615437783 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4497 (secure 0 0 0) 0x7fe3c004cc40 con 0x7fe3d8151870 2023-11-15T03:02:51.349 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:51.349 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":68,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1508681155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1508681155},{"type":"v1","addr":"172.21.15.121:6837","nonce":1508681155}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6385,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3559276534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3559276534},{"type":"v1","addr":"172.21.15.190:6835","nonce":3559276534}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6486,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/834528709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":834528709},{"type":"v1","addr":"172.21.15.121:6836","nonce":834528709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:02:20.703078+0000","modified":"2023-11-15T03:02:34.118521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6492},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6492":{"gid":6492,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6833/914796342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":914796342},{"type":"v1","addr":"172.21.15.190:6833","nonce":914796342}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-15T03:02:51.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.342+0000 7fe3dd28f700 1 -- 172.21.15.121:0/615437783 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe3b805a4e0 msgr2=0x7fe3b805c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:51.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.342+0000 7fe3dd28f700 1 --2- 172.21.15.121:0/615437783 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe3b805a4e0 0x7fe3b805c9a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7fe3c8009c10 tx=0x7fe3c8009060).stop 2023-11-15T03:02:51.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.342+0000 7fe3dd28f700 1 -- 172.21.15.121:0/615437783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3d8151870 msgr2=0x7fe3d80c6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:51.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.342+0000 7fe3dd28f700 1 --2- 172.21.15.121:0/615437783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3d8151870 0x7fe3d80c6700 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fe3d8065200 tx=0x7fe3c0016000).stop 2023-11-15T03:02:51.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.342+0000 7fe3dd28f700 1 -- 172.21.15.121:0/615437783 shutdown_connections 2023-11-15T03:02:51.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.342+0000 7fe3dd28f700 1 --2- 172.21.15.121:0/615437783 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe3b805a4e0 0x7fe3b805c9a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:51.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.342+0000 7fe3dd28f700 1 --2- 172.21.15.121:0/615437783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3d8152140 0x7fe3d80c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:51.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.342+0000 7fe3dd28f700 1 --2- 172.21.15.121:0/615437783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3d8151870 0x7fe3d80c6700 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:51.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.342+0000 7fe3dd28f700 1 --2- 172.21.15.121:0/615437783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3d80a9d20 0x7fe3d80bfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:51.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.342+0000 7fe3dd28f700 1 -- 172.21.15.121:0/615437783 >> 172.21.15.121:0/615437783 conn(0x7fe3d8062190 msgr2=0x7fe3d80c3080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:51.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.342+0000 7fe3dd28f700 1 -- 172.21.15.121:0/615437783 shutdown_connections 2023-11-15T03:02:51.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:51.342+0000 7fe3dd28f700 1 -- 172.21.15.121:0/615437783 wait complete. 2023-11-15T03:02:51.354 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 68 2023-11-15T03:02:51.373 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-15T03:02:53.426 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~7s 2023-11-15T03:02:56.374 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:02:56.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.534+0000 7fd343107700 1 Processor -- start 2023-11-15T03:02:56.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.534+0000 7fd343107700 1 -- start start 2023-11-15T03:02:56.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.534+0000 7fd343107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c151870 0x7fd33c151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:56.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.534+0000 7fd343107700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd33c0c30e0 0x7fd33c0c3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:56.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.534+0000 7fd343107700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd33c0c3a80 con 0x7fd33c152140 2023-11-15T03:02:56.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.534+0000 7fd343107700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd33c0657f0 con 0x7fd33c0c30e0 2023-11-15T03:02:56.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.534+0000 7fd343107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c065ad0 con 0x7fd33c151870 2023-11-15T03:02:56.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.534+0000 7fd342906700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd33c0c30e0 0x7fd33c0c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:56.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.534+0000 7fd342105700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c151870 0x7fd33c151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:56.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.534+0000 7fd341904700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd33c152140 0x7fd33c0a9830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:34984/0 (socket says 172.21.15.121:34984) 2023-11-15T03:02:56.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.534+0000 7fd341904700 1 -- 172.21.15.121:0/1676958310 learned_addr learned my addr 172.21.15.121:0/1676958310 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:56.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.534+0000 7fd342105700 1 --2- 172.21.15.121:0/1676958310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c151870 0x7fd33c151c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:42762/0 (socket says 172.21.15.121:42762) 2023-11-15T03:02:56.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.534+0000 7fd342906700 1 --2- 172.21.15.121:0/1676958310 >> v2:172.21.15.190:3300/0 conn(0x7fd33c0c30e0 0x7fd33c0c3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:33320/0 (socket says 172.21.15.121:33320) 2023-11-15T03:02:56.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.534+0000 7fd341103700 1 -- 172.21.15.121:0/1676958310 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287784190 0 0) 0x7fd32c001110 con 0x7fd33c152140 2023-11-15T03:02:56.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.538+0000 7fd341103700 1 -- 172.21.15.121:0/1676958310 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3280033f0 con 0x7fd33c152140 2023-11-15T03:02:56.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.538+0000 7fd342105700 1 -- 172.21.15.121:0/1676958310 >> v2:172.21.15.190:3300/0 conn(0x7fd33c0c30e0 msgr2=0x7fd33c0c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:56.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.538+0000 7fd342105700 1 --2- 172.21.15.121:0/1676958310 >> v2:172.21.15.190:3300/0 conn(0x7fd33c0c30e0 0x7fd33c0c3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:56.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.538+0000 7fd342105700 1 -- 172.21.15.121:0/1676958310 >> v1:172.21.15.190:6789/0 conn(0x7fd33c152140 legacy=0x7fd33c0a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:56.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.538+0000 7fd342105700 1 -- 172.21.15.121:0/1676958310 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33c0aa260 con 0x7fd33c151870 2023-11-15T03:02:56.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.538+0000 7fd342105700 1 --2- 172.21.15.121:0/1676958310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c151870 0x7fd33c151c50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fd338009760 tx=0x7fd338009060).ready entity=mon.2 client_cookie=320360e4af8fa626 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:56.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.538+0000 7fd341103700 1 -- 172.21.15.121:0/1676958310 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33800d4a0 con 0x7fd33c151870 2023-11-15T03:02:56.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.538+0000 7fd341103700 1 -- 172.21.15.121:0/1676958310 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33800d600 con 0x7fd33c151870 2023-11-15T03:02:56.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.538+0000 7fd341103700 1 -- 172.21.15.121:0/1676958310 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33800d980 con 0x7fd33c151870 2023-11-15T03:02:56.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.538+0000 7fd343107700 1 -- 172.21.15.121:0/1676958310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c151870 msgr2=0x7fd33c151c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:56.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.538+0000 7fd343107700 1 --2- 172.21.15.121:0/1676958310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c151870 0x7fd33c151c50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fd338009760 tx=0x7fd338009060).stop 2023-11-15T03:02:56.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.538+0000 7fd343107700 1 -- 172.21.15.121:0/1676958310 shutdown_connections 2023-11-15T03:02:56.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.538+0000 7fd343107700 1 --2- 172.21.15.121:0/1676958310 >> v2:172.21.15.190:3300/0 conn(0x7fd33c0c30e0 0x7fd33c0c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:56.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.538+0000 7fd343107700 1 --2- 172.21.15.121:0/1676958310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c151870 0x7fd33c151c50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:56.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.538+0000 7fd343107700 1 -- 172.21.15.121:0/1676958310 >> 172.21.15.121:0/1676958310 conn(0x7fd33c062190 msgr2=0x7fd33c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:56.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.538+0000 7fd343107700 1 -- 172.21.15.121:0/1676958310 shutdown_connections 2023-11-15T03:02:56.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.538+0000 7fd343107700 1 -- 172.21.15.121:0/1676958310 wait complete. 2023-11-15T03:02:56.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.542+0000 7fd343107700 1 Processor -- start 2023-11-15T03:02:56.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.542+0000 7fd343107700 1 -- start start 2023-11-15T03:02:56.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.542+0000 7fd343107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c0c30e0 0x7fd33c158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:56.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.542+0000 7fd343107700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd33c151870 0x7fd33c167560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:56.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.542+0000 7fd343107700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd33c152140 0x7fd33c167a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:56.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.542+0000 7fd343107700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c1511f0 con 0x7fd33c151870 2023-11-15T03:02:56.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.542+0000 7fd343107700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c153dd0 con 0x7fd33c152140 2023-11-15T03:02:56.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.542+0000 7fd343107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c0c4f30 con 0x7fd33c0c30e0 2023-11-15T03:02:56.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.542+0000 7fd341904700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd33c151870 0x7fd33c167560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:56.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.542+0000 7fd341904700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd33c151870 0x7fd33c167560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33654/0 (socket says 172.21.15.121:33654) 2023-11-15T03:02:56.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.542+0000 7fd341904700 1 -- 172.21.15.121:0/4210161890 learned_addr learned my addr 172.21.15.121:0/4210161890 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:56.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.542+0000 7fd342105700 1 --2- 172.21.15.121:0/4210161890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c0c30e0 0x7fd33c158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:56.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.542+0000 7fd342906700 1 --2- 172.21.15.121:0/4210161890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd33c152140 0x7fd33c167a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:56.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.546+0000 7fd341904700 1 -- 172.21.15.121:0/4210161890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c0c30e0 msgr2=0x7fd33c158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:56.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.546+0000 7fd341904700 1 --2- 172.21.15.121:0/4210161890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c0c30e0 0x7fd33c158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:56.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.546+0000 7fd341904700 1 -- 172.21.15.121:0/4210161890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd33c152140 msgr2=0x7fd33c167a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:56.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.546+0000 7fd341904700 1 --2- 172.21.15.121:0/4210161890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd33c152140 0x7fd33c167a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:56.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.546+0000 7fd341904700 1 -- 172.21.15.121:0/4210161890 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3380092a0 con 0x7fd33c151870 2023-11-15T03:02:56.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.546+0000 7fd341904700 1 --2- 172.21.15.121:0/4210161890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd33c151870 0x7fd33c167560 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7fd32c00c4a0 tx=0x7fd32c007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:56.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.546+0000 7fd3327fc700 1 -- 172.21.15.121:0/4210161890 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c0070d0 con 0x7fd33c151870 2023-11-15T03:02:56.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.546+0000 7fd3327fc700 1 -- 172.21.15.121:0/4210161890 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32c007250 con 0x7fd33c151870 2023-11-15T03:02:56.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.546+0000 7fd343107700 1 -- 172.21.15.121:0/4210161890 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd33c14edb0 con 0x7fd33c151870 2023-11-15T03:02:56.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.546+0000 7fd3327fc700 1 -- 172.21.15.121:0/4210161890 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c007610 con 0x7fd33c151870 2023-11-15T03:02:56.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.546+0000 7fd343107700 1 -- 172.21.15.121:0/4210161890 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33c0c54b0 con 0x7fd33c151870 2023-11-15T03:02:56.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.546+0000 7fd343107700 1 -- 172.21.15.121:0/4210161890 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd33c0c0460 con 0x7fd33c151870 2023-11-15T03:02:56.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.550+0000 7fd3327fc700 1 -- 172.21.15.121:0/4210161890 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd32c0059b0 con 0x7fd33c151870 2023-11-15T03:02:56.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.550+0000 7fd3327fc700 1 --2- 172.21.15.121:0/4210161890 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd32005a4e0 0x7fd32005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:56.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.550+0000 7fd3327fc700 1 -- 172.21.15.121:0/4210161890 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6179+0+0 (secure 0 0 0) 0x7fd32c073be0 con 0x7fd33c151870 2023-11-15T03:02:56.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.550+0000 7fd342105700 1 --2- 172.21.15.121:0/4210161890 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd32005a4e0 0x7fd32005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:56.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.550+0000 7fd342105700 1 --2- 172.21.15.121:0/4210161890 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd32005a4e0 0x7fd32005c9a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7fd33800d260 tx=0x7fd338016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:56.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.554+0000 7fd3327fc700 1 -- 172.21.15.121:0/4210161890 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd33c0c0460 con 0x7fd33c151870 2023-11-15T03:02:56.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.750+0000 7fd343107700 1 -- 172.21.15.121:0/4210161890 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd33c0c3a80 con 0x7fd33c151870 2023-11-15T03:02:56.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.750+0000 7fd3327fc700 1 -- 172.21.15.121:0/4210161890 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4545 (secure 0 0 0) 0x7fd33c0c3a80 con 0x7fd33c151870 2023-11-15T03:02:56.762 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:56.762 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":69,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1508681155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1508681155},{"type":"v1","addr":"172.21.15.121:6837","nonce":1508681155}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6385,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3559276534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3559276534},{"type":"v1","addr":"172.21.15.190:6835","nonce":3559276534}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6486,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/834528709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":834528709},{"type":"v1","addr":"172.21.15.121:6836","nonce":834528709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:02:20.703078+0000","modified":"2023-11-15T03:02:52.975054+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6492},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6492":{"gid":6492,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6833/914796342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":914796342},{"type":"v1","addr":"172.21.15.190:6833","nonce":914796342}]},"join_fscid":-1,"laggy_since":"2023-11-15T03:02:52.974926+0000","export_targets":[],"features":4540138314316775423,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-15T03:02:56.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.758+0000 7fd343107700 1 -- 172.21.15.121:0/4210161890 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd32005a4e0 msgr2=0x7fd32005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:56.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.758+0000 7fd343107700 1 --2- 172.21.15.121:0/4210161890 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd32005a4e0 0x7fd32005c9a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7fd33800d260 tx=0x7fd338016000).stop 2023-11-15T03:02:56.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.758+0000 7fd343107700 1 -- 172.21.15.121:0/4210161890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd33c151870 msgr2=0x7fd33c167560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:56.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.758+0000 7fd343107700 1 --2- 172.21.15.121:0/4210161890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd33c151870 0x7fd33c167560 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7fd32c00c4a0 tx=0x7fd32c007a90).stop 2023-11-15T03:02:56.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.758+0000 7fd343107700 1 -- 172.21.15.121:0/4210161890 shutdown_connections 2023-11-15T03:02:56.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.758+0000 7fd343107700 1 --2- 172.21.15.121:0/4210161890 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd32005a4e0 0x7fd32005c9a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:56.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.758+0000 7fd343107700 1 --2- 172.21.15.121:0/4210161890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd33c152140 0x7fd33c167a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:56.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.758+0000 7fd343107700 1 --2- 172.21.15.121:0/4210161890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd33c151870 0x7fd33c167560 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:56.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.758+0000 7fd343107700 1 --2- 172.21.15.121:0/4210161890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c0c30e0 0x7fd33c158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:56.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.758+0000 7fd343107700 1 -- 172.21.15.121:0/4210161890 >> 172.21.15.121:0/4210161890 conn(0x7fd33c062190 msgr2=0x7fd33c14e470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:56.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.758+0000 7fd343107700 1 -- 172.21.15.121:0/4210161890 shutdown_connections 2023-11-15T03:02:56.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.758+0000 7fd343107700 1 -- 172.21.15.121:0/4210161890 wait complete. 2023-11-15T03:02:56.768 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 69 2023-11-15T03:02:56.784 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2023-11-15T03:02:56.784 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2023-11-15T03:02:56.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.946+0000 7f7c73fff700 1 Processor -- start 2023-11-15T03:02:56.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.946+0000 7f7c73fff700 1 -- start start 2023-11-15T03:02:56.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.950+0000 7f7c73fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c74062330 0x7f7c74062710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:56.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.950+0000 7f7c73fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c74062db0 0x7f7c740c11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:56.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.950+0000 7f7c73fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7c74157b10 0x7f7c74157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:56.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.950+0000 7f7c73fff700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c740acb80 con 0x7f7c74062db0 2023-11-15T03:02:56.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.950+0000 7f7c73fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c740662a0 con 0x7f7c74157b10 2023-11-15T03:02:56.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.950+0000 7f7c73fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c74066580 con 0x7f7c74062330 2023-11-15T03:02:56.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.950+0000 7f7c727fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c74062db0 0x7f7c740c11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:56.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.950+0000 7f7c727fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c74062db0 0x7f7c740c11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33660/0 (socket says 172.21.15.121:33660) 2023-11-15T03:02:56.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.950+0000 7f7c727fc700 1 -- 172.21.15.121:0/4206972226 learned_addr learned my addr 172.21.15.121:0/4206972226 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:56.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.950+0000 7f7c737fe700 1 --2- 172.21.15.121:0/4206972226 >> v2:172.21.15.190:3300/0 conn(0x7f7c74157b10 0x7f7c74157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:56.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.950+0000 7f7c72ffd700 1 --2- 172.21.15.121:0/4206972226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c74062330 0x7f7c74062710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:56.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.950+0000 7f7c727fc700 1 -- 172.21.15.121:0/4206972226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c74062330 msgr2=0x7f7c74062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:56.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.950+0000 7f7c727fc700 1 --2- 172.21.15.121:0/4206972226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c74062330 0x7f7c74062710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:56.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.950+0000 7f7c727fc700 1 -- 172.21.15.121:0/4206972226 >> v2:172.21.15.190:3300/0 conn(0x7f7c74157b10 msgr2=0x7f7c74157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:56.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.950+0000 7f7c727fc700 1 --2- 172.21.15.121:0/4206972226 >> v2:172.21.15.190:3300/0 conn(0x7f7c74157b10 0x7f7c74157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:56.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.950+0000 7f7c727fc700 1 -- 172.21.15.121:0/4206972226 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c740ace20 con 0x7f7c74062db0 2023-11-15T03:02:56.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.950+0000 7f7c737fe700 1 --2- 172.21.15.121:0/4206972226 >> v2:172.21.15.190:3300/0 conn(0x7f7c74157b10 0x7f7c74157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:02:56.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.950+0000 7f7c72ffd700 1 --2- 172.21.15.121:0/4206972226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c74062330 0x7f7c74062710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:56.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.950+0000 7f7c727fc700 1 --2- 172.21.15.121:0/4206972226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c74062db0 0x7f7c740c11c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f7c740c5b70 tx=0x7f7c680090f0).ready entity=mon.0 client_cookie=6ab5c3d65bc31535 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:56.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.954+0000 7f7c71ffb700 1 -- 172.21.15.121:0/4206972226 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6800b520 con 0x7f7c74062db0 2023-11-15T03:02:56.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.954+0000 7f7c71ffb700 1 -- 172.21.15.121:0/4206972226 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6800b680 con 0x7f7c74062db0 2023-11-15T03:02:56.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.954+0000 7f7c71ffb700 1 -- 172.21.15.121:0/4206972226 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6800ba00 con 0x7f7c74062db0 2023-11-15T03:02:56.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.954+0000 7f7c73fff700 1 -- 172.21.15.121:0/4206972226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c74062db0 msgr2=0x7f7c740c11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:56.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.954+0000 7f7c73fff700 1 --2- 172.21.15.121:0/4206972226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c74062db0 0x7f7c740c11c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f7c740c5b70 tx=0x7f7c680090f0).stop 2023-11-15T03:02:56.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.954+0000 7f7c73fff700 1 -- 172.21.15.121:0/4206972226 shutdown_connections 2023-11-15T03:02:56.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.954+0000 7f7c73fff700 1 --2- 172.21.15.121:0/4206972226 >> v2:172.21.15.190:3300/0 conn(0x7f7c74157b10 0x7f7c74157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:56.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.954+0000 7f7c73fff700 1 --2- 172.21.15.121:0/4206972226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c74062db0 0x7f7c740c11c0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:56.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.954+0000 7f7c73fff700 1 --2- 172.21.15.121:0/4206972226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c74062330 0x7f7c74062710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:56.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.954+0000 7f7c73fff700 1 -- 172.21.15.121:0/4206972226 >> 172.21.15.121:0/4206972226 conn(0x7f7c74067a60 msgr2=0x7f7c74067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:56.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.954+0000 7f7c73fff700 1 -- 172.21.15.121:0/4206972226 shutdown_connections 2023-11-15T03:02:56.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.954+0000 7f7c73fff700 1 -- 172.21.15.121:0/4206972226 wait complete. 2023-11-15T03:02:56.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.958+0000 7f7c73fff700 1 Processor -- start 2023-11-15T03:02:56.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.958+0000 7f7c73fff700 1 -- start start 2023-11-15T03:02:56.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.958+0000 7f7c73fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c74062330 0x7f7c74140c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:56.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.958+0000 7f7c73fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c74062db0 0x7f7c740c19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:56.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.958+0000 7f7c73fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c74157b10 0x7f7c7414eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:56.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.958+0000 7f7c73fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c74151e30 con 0x7f7c74062db0 2023-11-15T03:02:56.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.958+0000 7f7c73fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741520d0 con 0x7f7c74157b10 2023-11-15T03:02:56.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.958+0000 7f7c73fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c74150940 con 0x7f7c74062330 2023-11-15T03:02:56.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.958+0000 7f7c727fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c74062db0 0x7f7c740c19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:56.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.958+0000 7f7c727fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c74062db0 0x7f7c740c19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33670/0 (socket says 172.21.15.121:33670) 2023-11-15T03:02:56.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.958+0000 7f7c727fc700 1 -- 172.21.15.121:0/54696798 learned_addr learned my addr 172.21.15.121:0/54696798 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:56.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.958+0000 7f7c72ffd700 1 --2- 172.21.15.121:0/54696798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c74062330 0x7f7c74140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:56.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.958+0000 7f7c737fe700 1 --2- 172.21.15.121:0/54696798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c74157b10 0x7f7c7414eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:56.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.962+0000 7f7c727fc700 1 -- 172.21.15.121:0/54696798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c74062330 msgr2=0x7f7c74140c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:56.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.962+0000 7f7c727fc700 1 --2- 172.21.15.121:0/54696798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c74062330 0x7f7c74140c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:56.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.962+0000 7f7c727fc700 1 -- 172.21.15.121:0/54696798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c74157b10 msgr2=0x7f7c7414eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:56.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.962+0000 7f7c727fc700 1 --2- 172.21.15.121:0/54696798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c74157b10 0x7f7c7414eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:56.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.962+0000 7f7c727fc700 1 -- 172.21.15.121:0/54696798 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c0092a0 con 0x7f7c74062db0 2023-11-15T03:02:56.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.962+0000 7f7c737fe700 1 --2- 172.21.15.121:0/54696798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c74157b10 0x7f7c7414eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:02:56.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.962+0000 7f7c727fc700 1 --2- 172.21.15.121:0/54696798 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c74062db0 0x7f7c740c19e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f7c68011000 tx=0x7f7c68017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:56.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.962+0000 7f7c5affd700 1 -- 172.21.15.121:0/54696798 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c68000bc0 con 0x7f7c74062db0 2023-11-15T03:02:56.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.962+0000 7f7c5affd700 1 -- 172.21.15.121:0/54696798 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6800b7f0 con 0x7f7c74062db0 2023-11-15T03:02:56.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.962+0000 7f7c5affd700 1 -- 172.21.15.121:0/54696798 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c68000d20 con 0x7f7c74062db0 2023-11-15T03:02:56.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.962+0000 7f7c73fff700 1 -- 172.21.15.121:0/54696798 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c68009370 con 0x7f7c74062db0 2023-11-15T03:02:56.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.962+0000 7f7c73fff700 1 -- 172.21.15.121:0/54696798 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c74150e90 con 0x7f7c74062db0 2023-11-15T03:02:56.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.962+0000 7f7c73fff700 1 -- 172.21.15.121:0/54696798 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c74152260 con 0x7f7c74062db0 2023-11-15T03:02:56.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.966+0000 7f7c5affd700 1 -- 172.21.15.121:0/54696798 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7c6801c900 con 0x7f7c74062db0 2023-11-15T03:02:56.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.966+0000 7f7c5affd700 1 --2- 172.21.15.121:0/54696798 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c6005a510 0x7f7c6005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:56.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.966+0000 7f7c5affd700 1 -- 172.21.15.121:0/54696798 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6179+0+0 (secure 0 0 0) 0x7f7c680763c0 con 0x7f7c74062db0 2023-11-15T03:02:56.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.966+0000 7f7c72ffd700 1 --2- 172.21.15.121:0/54696798 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c6005a510 0x7f7c6005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:56.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.966+0000 7f7c72ffd700 1 --2- 172.21.15.121:0/54696798 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c6005a510 0x7f7c6005c9d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f7c74150590 tx=0x7f7c640099d0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:56.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:56.974+0000 7f7c5affd700 1 -- 172.21.15.121:0/54696798 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7c68049aa0 con 0x7f7c74062db0 2023-11-15T03:02:57.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.142+0000 7f7c73fff700 1 -- 172.21.15.121:0/54696798 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) v1 -- 0x7f7c7406bcf0 con 0x7f7c74062db0 2023-11-15T03:02:57.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.150+0000 7f7c5affd700 1 -- 172.21.15.121:0/54696798 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v16) v1 ==== 121+0+0 (secure 0 0 0) 0x7f7c68020020 con 0x7f7c74062db0 2023-11-15T03:02:57.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.150+0000 7f7c73fff700 1 -- 172.21.15.121:0/54696798 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c6005a510 msgr2=0x7f7c6005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:57.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.150+0000 7f7c73fff700 1 --2- 172.21.15.121:0/54696798 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c6005a510 0x7f7c6005c9d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f7c74150590 tx=0x7f7c640099d0).stop 2023-11-15T03:02:57.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.150+0000 7f7c73fff700 1 -- 172.21.15.121:0/54696798 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c74062db0 msgr2=0x7f7c740c19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:57.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.150+0000 7f7c73fff700 1 --2- 172.21.15.121:0/54696798 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c74062db0 0x7f7c740c19e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f7c68011000 tx=0x7f7c68017000).stop 2023-11-15T03:02:57.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.154+0000 7f7c73fff700 1 -- 172.21.15.121:0/54696798 shutdown_connections 2023-11-15T03:02:57.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.154+0000 7f7c73fff700 1 --2- 172.21.15.121:0/54696798 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c6005a510 0x7f7c6005c9d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:57.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.154+0000 7f7c73fff700 1 --2- 172.21.15.121:0/54696798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c74157b10 0x7f7c7414eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:57.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.154+0000 7f7c73fff700 1 --2- 172.21.15.121:0/54696798 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c74062db0 0x7f7c740c19e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:57.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.154+0000 7f7c73fff700 1 --2- 172.21.15.121:0/54696798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c74062330 0x7f7c74140c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:57.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.154+0000 7f7c73fff700 1 -- 172.21.15.121:0/54696798 >> 172.21.15.121:0/54696798 conn(0x7f7c74067a60 msgr2=0x7f7c74151350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:57.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.154+0000 7f7c73fff700 1 -- 172.21.15.121:0/54696798 shutdown_connections 2023-11-15T03:02:57.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.154+0000 7f7c73fff700 1 -- 172.21.15.121:0/54696798 wait complete. 2023-11-15T03:02:57.182 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 5:0 false 2023-11-15T03:02:57.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.338+0000 7f333da1b700 1 Processor -- start 2023-11-15T03:02:57.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.338+0000 7f333da1b700 1 -- start start 2023-11-15T03:02:57.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.342+0000 7f333da1b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3338153e60 0x7f3338158420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:57.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.342+0000 7f333da1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3338154240 0x7f33380657f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:57.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.342+0000 7f333da1b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3338158910 con 0x7f3338065ce0 2023-11-15T03:02:57.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.342+0000 7f333da1b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33381553c0 con 0x7f3338153e60 2023-11-15T03:02:57.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.342+0000 7f333da1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33380c7280 con 0x7f3338154240 2023-11-15T03:02:57.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.342+0000 7f33377fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3338065ce0 0x7f33380c5ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47080/0 (socket says 172.21.15.121:47080) 2023-11-15T03:02:57.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.342+0000 7f33377fe700 1 -- 172.21.15.121:0/1085798000 learned_addr learned my addr 172.21.15.121:0/1085798000 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:02:57.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.342+0000 7f3336ffd700 1 --2- 172.21.15.121:0/1085798000 >> v2:172.21.15.190:3300/0 conn(0x7f3338153e60 0x7f3338158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:57.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.342+0000 7f33367fc700 1 --2- 172.21.15.121:0/1085798000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3338154240 0x7f33380657f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:57.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.342+0000 7f3336ffd700 1 -- 172.21.15.121:0/1085798000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3338154240 msgr2=0x7f33380657f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:57.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.342+0000 7f3336ffd700 1 --2- 172.21.15.121:0/1085798000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3338154240 0x7f33380657f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:57.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.342+0000 7f3335ffb700 1 -- 172.21.15.121:0/1085798000 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1463581273 0 0) 0x7f3330001490 con 0x7f3338065ce0 2023-11-15T03:02:57.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.342+0000 7f3336ffd700 1 -- 172.21.15.121:0/1085798000 >> v1:172.21.15.121:6789/0 conn(0x7f3338065ce0 legacy=0x7f33380c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:57.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.342+0000 7f3336ffd700 1 -- 172.21.15.121:0/1085798000 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3338154b20 con 0x7f3338153e60 2023-11-15T03:02:57.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.342+0000 7f3336ffd700 1 --2- 172.21.15.121:0/1085798000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3338153e60 0x7f3338158420 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f3328009760 tx=0x7f3328009060).ready entity=mon.1 client_cookie=ba7c665790572845 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:57.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.342+0000 7f3335ffb700 1 -- 172.21.15.121:0/1085798000 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332800f510 con 0x7f3338153e60 2023-11-15T03:02:57.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.342+0000 7f3335ffb700 1 -- 172.21.15.121:0/1085798000 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332800f670 con 0x7f3338153e60 2023-11-15T03:02:57.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.342+0000 7f3335ffb700 1 -- 172.21.15.121:0/1085798000 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332800f9f0 con 0x7f3338153e60 2023-11-15T03:02:57.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.346+0000 7f333da1b700 1 -- 172.21.15.121:0/1085798000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3338153e60 msgr2=0x7f3338158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:57.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.346+0000 7f333da1b700 1 --2- 172.21.15.121:0/1085798000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3338153e60 0x7f3338158420 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f3328009760 tx=0x7f3328009060).stop 2023-11-15T03:02:57.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.346+0000 7f333da1b700 1 -- 172.21.15.121:0/1085798000 shutdown_connections 2023-11-15T03:02:57.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.346+0000 7f333da1b700 1 --2- 172.21.15.121:0/1085798000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3338154240 0x7f33380657f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:57.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.346+0000 7f333da1b700 1 --2- 172.21.15.121:0/1085798000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3338153e60 0x7f3338158420 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:57.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.346+0000 7f333da1b700 1 -- 172.21.15.121:0/1085798000 >> 172.21.15.121:0/1085798000 conn(0x7f3338067a60 msgr2=0x7f3338067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:57.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.346+0000 7f333da1b700 1 -- 172.21.15.121:0/1085798000 shutdown_connections 2023-11-15T03:02:57.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.346+0000 7f333da1b700 1 -- 172.21.15.121:0/1085798000 wait complete. 2023-11-15T03:02:57.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.346+0000 7f333da1b700 1 Processor -- start 2023-11-15T03:02:57.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.346+0000 7f333da1b700 1 -- start start 2023-11-15T03:02:57.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.346+0000 7f333da1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3338065ce0 0x7f33380ac530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:57.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.346+0000 7f333da1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3338153e60 0x7f3338169270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:57.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.346+0000 7f333da1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3338154240 0x7f3338169760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:57.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.346+0000 7f333da1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33380ace30 con 0x7f3338065ce0 2023-11-15T03:02:57.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.346+0000 7f333da1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33380aebb0 con 0x7f3338153e60 2023-11-15T03:02:57.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.346+0000 7f333da1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33380aee90 con 0x7f3338154240 2023-11-15T03:02:57.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.350+0000 7f3336ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3338065ce0 0x7f33380ac530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:57.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.350+0000 7f3336ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3338065ce0 0x7f33380ac530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33676/0 (socket says 172.21.15.121:33676) 2023-11-15T03:02:57.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.350+0000 7f3336ffd700 1 -- 172.21.15.121:0/3974339573 learned_addr learned my addr 172.21.15.121:0/3974339573 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:57.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.350+0000 7f33367fc700 1 --2- 172.21.15.121:0/3974339573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3338153e60 0x7f3338169270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:57.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.350+0000 7f3336ffd700 1 -- 172.21.15.121:0/3974339573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3338154240 msgr2=0x7f3338169760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:57.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.350+0000 7f33377fe700 1 --2- 172.21.15.121:0/3974339573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3338154240 0x7f3338169760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:57.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.350+0000 7f3336ffd700 1 --2- 172.21.15.121:0/3974339573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3338154240 0x7f3338169760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:57.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.350+0000 7f3336ffd700 1 -- 172.21.15.121:0/3974339573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3338153e60 msgr2=0x7f3338169270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:57.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.350+0000 7f3336ffd700 1 --2- 172.21.15.121:0/3974339573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3338153e60 0x7f3338169270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:57.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.350+0000 7f3336ffd700 1 -- 172.21.15.121:0/3974339573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33280092a0 con 0x7f3338065ce0 2023-11-15T03:02:57.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.350+0000 7f33367fc700 1 --2- 172.21.15.121:0/3974339573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3338153e60 0x7f3338169270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:02:57.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.350+0000 7f33377fe700 1 --2- 172.21.15.121:0/3974339573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3338154240 0x7f3338169760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:02:57.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.350+0000 7f3336ffd700 1 --2- 172.21.15.121:0/3974339573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3338065ce0 0x7f33380ac530 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f3338064960 tx=0x7f3328017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:57.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.350+0000 7f331effd700 1 -- 172.21.15.121:0/3974339573 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332800f7e0 con 0x7f3338065ce0 2023-11-15T03:02:57.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.350+0000 7f331effd700 1 -- 172.21.15.121:0/3974339573 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332800f940 con 0x7f3338065ce0 2023-11-15T03:02:57.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.350+0000 7f333da1b700 1 -- 172.21.15.121:0/3974339573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33380af170 con 0x7f3338065ce0 2023-11-15T03:02:57.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.350+0000 7f331effd700 1 -- 172.21.15.121:0/3974339573 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332801b660 con 0x7f3338065ce0 2023-11-15T03:02:57.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.350+0000 7f333da1b700 1 -- 172.21.15.121:0/3974339573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33380aca20 con 0x7f3338065ce0 2023-11-15T03:02:57.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.350+0000 7f333da1b700 1 -- 172.21.15.121:0/3974339573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33380af670 con 0x7f3338065ce0 2023-11-15T03:02:57.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.354+0000 7f331effd700 1 -- 172.21.15.121:0/3974339573 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f332801ba50 con 0x7f3338065ce0 2023-11-15T03:02:57.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.354+0000 7f331effd700 1 --2- 172.21.15.121:0/3974339573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f332405a540 0x7f332405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:57.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.354+0000 7f331effd700 1 -- 172.21.15.121:0/3974339573 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6179+0+0 (secure 0 0 0) 0x7f33280763b0 con 0x7f3338065ce0 2023-11-15T03:02:57.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.354+0000 7f33367fc700 1 --2- 172.21.15.121:0/3974339573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f332405a540 0x7f332405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:57.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.354+0000 7f33367fc700 1 --2- 172.21.15.121:0/3974339573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f332405a540 0x7f332405ca00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f332c006d80 tx=0x7f332c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:57.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.358+0000 7f331effd700 1 -- 172.21.15.121:0/3974339573 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3328049b10 con 0x7f3338065ce0 2023-11-15T03:02:57.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.578+0000 7f333da1b700 1 -- 172.21.15.121:0/3974339573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "5:0", "val": "false"} v 0) v1 -- 0x7f333806bcf0 con 0x7f3338065ce0 2023-11-15T03:02:57.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.990+0000 7f331effd700 1 -- 172.21.15.121:0/3974339573 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "5:0", "val": "false"}]=0 unfreezing mds.6492 v70) v1 ==== 115+0+0 (secure 0 0 0) 0x7f3328006b30 con 0x7f3338065ce0 2023-11-15T03:02:58.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.994+0000 7f333da1b700 1 -- 172.21.15.121:0/3974339573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f332405a540 msgr2=0x7f332405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:58.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.994+0000 7f333da1b700 1 --2- 172.21.15.121:0/3974339573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f332405a540 0x7f332405ca00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f332c006d80 tx=0x7f332c008040).stop 2023-11-15T03:02:58.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.994+0000 7f333da1b700 1 -- 172.21.15.121:0/3974339573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3338065ce0 msgr2=0x7f33380ac530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:58.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.994+0000 7f333da1b700 1 --2- 172.21.15.121:0/3974339573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3338065ce0 0x7f33380ac530 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f3338064960 tx=0x7f3328017000).stop 2023-11-15T03:02:58.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.994+0000 7f333da1b700 1 -- 172.21.15.121:0/3974339573 shutdown_connections 2023-11-15T03:02:58.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.994+0000 7f333da1b700 1 --2- 172.21.15.121:0/3974339573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f332405a540 0x7f332405ca00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:58.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.994+0000 7f333da1b700 1 --2- 172.21.15.121:0/3974339573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3338154240 0x7f3338169760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:58.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.994+0000 7f333da1b700 1 --2- 172.21.15.121:0/3974339573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3338153e60 0x7f3338169270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:58.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.994+0000 7f333da1b700 1 --2- 172.21.15.121:0/3974339573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3338065ce0 0x7f33380ac530 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:58.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.994+0000 7f333da1b700 1 -- 172.21.15.121:0/3974339573 >> 172.21.15.121:0/3974339573 conn(0x7f3338067a60 msgr2=0x7f3338155c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:58.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.994+0000 7f333da1b700 1 -- 172.21.15.121:0/3974339573 shutdown_connections 2023-11-15T03:02:58.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:57.994+0000 7f333da1b700 1 -- 172.21.15.121:0/3974339573 wait complete. 2023-11-15T03:02:58.005 INFO:teuthology.orchestra.run.smithi121.stderr:unfreezing mds.6492 2023-11-15T03:02:58.025 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -n kernel.core_pattern 2023-11-15T03:02:58.054 INFO:teuthology.orchestra.run.smithi190.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-15T03:02:58.056 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-15T03:02:58.057 DEBUG:teuthology.orchestra.run.smithi190:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-15T03:02:58.158 INFO:teuthology.orchestra.run.smithi190.stdout:1700017359.37191.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i d', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/usr/bin/ceph-mds', platform: 'x86_64' 2023-11-15T03:02:58.165 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1700017359.37191.core'] 2023-11-15T03:02:58.166 INFO:tasks.cephfs.cephfs_test_case:Found core file 1700017359.37191.core, deleting it 2023-11-15T03:02:58.166 DEBUG:teuthology.orchestra.run.smithi190:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1700017359.37191.core 2023-11-15T03:02:58.216 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-15T03:02:58.217 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-15T03:02:58.217 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-15T03:02:58.218 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:02:58.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:02:58.219 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi190 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' 2023-11-15T03:02:58.219 INFO:tasks.ceph.mds.d:Stopped 2023-11-15T03:02:58.220 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-15T03:02:58.265 INFO:tasks.ceph.mds.d:Started 2023-11-15T03:02:58.266 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:02:58.382 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-11-15T03:02:58.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.426+0000 7f8609307700 1 Processor -- start 2023-11-15T03:02:58.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.426+0000 7f8609307700 1 -- start start 2023-11-15T03:02:58.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.426+0000 7f8609307700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8604066d70 0x7f86040a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:58.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.426+0000 7f8609307700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8604067150 0x7f86041518b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:58.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.426+0000 7f8609307700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86040a9350 con 0x7f86040c29a0 2023-11-15T03:02:58.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.426+0000 7f8609307700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8604150ed0 con 0x7f8604067150 2023-11-15T03:02:58.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.426+0000 7f8609307700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86040675d0 con 0x7f8604066d70 2023-11-15T03:02:58.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f85fbfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8604067150 0x7f86041518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:58.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f8603fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8604066d70 0x7f86040a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:58.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f85fbfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8604067150 0x7f86041518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33680/0 (socket says 172.21.15.121:33680) 2023-11-15T03:02:58.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f85fbfff700 1 -- 172.21.15.121:0/4068033971 learned_addr learned my addr 172.21.15.121:0/4068033971 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:58.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f85fbfff700 1 -- 172.21.15.121:0/4068033971 >> v2:172.21.15.190:3300/0 conn(0x7f8604066d70 msgr2=0x7f86040a87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:02:58.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f85fbfff700 1 --2- 172.21.15.121:0/4068033971 >> v2:172.21.15.190:3300/0 conn(0x7f8604066d70 0x7f86040a87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:58.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f85fbfff700 1 -- 172.21.15.121:0/4068033971 >> v1:172.21.15.121:6789/0 conn(0x7f86040c29a0 legacy=0x7f860414e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:58.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f85fbfff700 1 -- 172.21.15.121:0/4068033971 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8604151420 con 0x7f8604067150 2023-11-15T03:02:58.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f85fbfff700 1 --2- 172.21.15.121:0/4068033971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8604067150 0x7f86041518b0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f85ec006fd0 tx=0x7f85ec0090f0).ready entity=mon.0 client_cookie=c6a61596e911dee7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:58.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f86037fe700 1 -- 172.21.15.121:0/4068033971 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec00b520 con 0x7f8604067150 2023-11-15T03:02:58.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f86037fe700 1 -- 172.21.15.121:0/4068033971 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ec00b680 con 0x7f8604067150 2023-11-15T03:02:58.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f86037fe700 1 -- 172.21.15.121:0/4068033971 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec00ba00 con 0x7f8604067150 2023-11-15T03:02:58.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f8609307700 1 -- 172.21.15.121:0/4068033971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8604067150 msgr2=0x7f86041518b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:58.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f8609307700 1 --2- 172.21.15.121:0/4068033971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8604067150 0x7f86041518b0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f85ec006fd0 tx=0x7f85ec0090f0).stop 2023-11-15T03:02:58.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f8609307700 1 -- 172.21.15.121:0/4068033971 shutdown_connections 2023-11-15T03:02:58.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f8609307700 1 --2- 172.21.15.121:0/4068033971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8604067150 0x7f86041518b0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:58.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f8609307700 1 --2- 172.21.15.121:0/4068033971 >> v2:172.21.15.190:3300/0 conn(0x7f8604066d70 0x7f86040a87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:58.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f8609307700 1 -- 172.21.15.121:0/4068033971 >> 172.21.15.121:0/4068033971 conn(0x7f8604062190 msgr2=0x7f8604062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:58.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f8609307700 1 -- 172.21.15.121:0/4068033971 shutdown_connections 2023-11-15T03:02:58.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.430+0000 7f8609307700 1 -- 172.21.15.121:0/4068033971 wait complete. 2023-11-15T03:02:58.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.434+0000 7f8609307700 1 Processor -- start 2023-11-15T03:02:58.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.434+0000 7f8609307700 1 -- start start 2023-11-15T03:02:58.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8609307700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8604066d70 0x7f86040a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:58.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8609307700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8604067150 0x7f860409fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:58.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8609307700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86040c29a0 0x7f8604163910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:58.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8609307700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8604152280 con 0x7f8604066d70 2023-11-15T03:02:58.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8609307700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86040a1520 con 0x7f8604067150 2023-11-15T03:02:58.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8609307700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f860413ebe0 con 0x7f86040c29a0 2023-11-15T03:02:58.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8603fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8604066d70 0x7f86040a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:58.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8603fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8604066d70 0x7f86040a1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33690/0 (socket says 172.21.15.121:33690) 2023-11-15T03:02:58.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8603fff700 1 -- 172.21.15.121:0/2198412573 learned_addr learned my addr 172.21.15.121:0/2198412573 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:02:58.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f85fbfff700 1 --2- 172.21.15.121:0/2198412573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8604067150 0x7f860409fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:58.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8608b06700 1 --2- 172.21.15.121:0/2198412573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86040c29a0 0x7f8604163910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:58.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8603fff700 1 -- 172.21.15.121:0/2198412573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86040c29a0 msgr2=0x7f8604163910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:58.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8603fff700 1 --2- 172.21.15.121:0/2198412573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86040c29a0 0x7f8604163910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:58.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8603fff700 1 -- 172.21.15.121:0/2198412573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8604067150 msgr2=0x7f860409fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:58.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8603fff700 1 --2- 172.21.15.121:0/2198412573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8604067150 0x7f860409fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:58.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8603fff700 1 -- 172.21.15.121:0/2198412573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ec009370 con 0x7f8604066d70 2023-11-15T03:02:58.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8603fff700 1 --2- 172.21.15.121:0/2198412573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8604066d70 0x7f86040a1030 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f85f400e960 tx=0x7f85f4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:58.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8600ff9700 1 -- 172.21.15.121:0/2198412573 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4009510 con 0x7f8604066d70 2023-11-15T03:02:58.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8600ff9700 1 -- 172.21.15.121:0/2198412573 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f400c850 con 0x7f8604066d70 2023-11-15T03:02:58.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8600ff9700 1 -- 172.21.15.121:0/2198412573 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f400cbd0 con 0x7f8604066d70 2023-11-15T03:02:58.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8609307700 1 -- 172.21.15.121:0/2198412573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8604142510 con 0x7f8604066d70 2023-11-15T03:02:58.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.438+0000 7f8609307700 1 -- 172.21.15.121:0/2198412573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86041518e0 con 0x7f8604066d70 2023-11-15T03:02:58.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.442+0000 7f8609307700 1 -- 172.21.15.121:0/2198412573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86041524f0 con 0x7f8604066d70 2023-11-15T03:02:58.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.442+0000 7f8600ff9700 1 -- 172.21.15.121:0/2198412573 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f85f401b400 con 0x7f8604066d70 2023-11-15T03:02:58.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.442+0000 7f8600ff9700 1 --2- 172.21.15.121:0/2198412573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f85e405a510 0x7f85e405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:02:58.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.442+0000 7f8600ff9700 1 -- 172.21.15.121:0/2198412573 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f85f4074e60 con 0x7f8604066d70 2023-11-15T03:02:58.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.442+0000 7f85fbfff700 1 --2- 172.21.15.121:0/2198412573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f85e405a510 0x7f85e405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:02:58.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.442+0000 7f85fbfff700 1 --2- 172.21.15.121:0/2198412573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f85e405a510 0x7f85e405c9d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f85ec009130 tx=0x7f85ec015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:02:58.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.446+0000 7f8600ff9700 1 -- 172.21.15.121:0/2198412573 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f85f4048490 con 0x7f8604066d70 2023-11-15T03:02:58.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.642+0000 7f8609307700 1 -- 172.21.15.121:0/2198412573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f860406bcf0 con 0x7f8604066d70 2023-11-15T03:02:58.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.646+0000 7f8600ff9700 1 -- 172.21.15.121:0/2198412573 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+3741 (secure 0 0 0) 0x7f85f404bab0 con 0x7f8604066d70 2023-11-15T03:02:58.652 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:02:58.652 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":70,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1508681155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1508681155},{"type":"v1","addr":"172.21.15.121:6837","nonce":1508681155}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6385,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3559276534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3559276534},{"type":"v1","addr":"172.21.15.190:6835","nonce":3559276534}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:02:20.703078+0000","modified":"2023-11-15T03:02:57.979061+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":70,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/834528709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":834528709},{"type":"v1","addr":"172.21.15.121:6836","nonce":834528709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-15T03:02:58.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.646+0000 7f8609307700 1 -- 172.21.15.121:0/2198412573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f85e405a510 msgr2=0x7f85e405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:58.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.646+0000 7f8609307700 1 --2- 172.21.15.121:0/2198412573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f85e405a510 0x7f85e405c9d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f85ec009130 tx=0x7f85ec015000).stop 2023-11-15T03:02:58.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.646+0000 7f8609307700 1 -- 172.21.15.121:0/2198412573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8604066d70 msgr2=0x7f86040a1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:02:58.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.646+0000 7f8609307700 1 --2- 172.21.15.121:0/2198412573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8604066d70 0x7f86040a1030 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f85f400e960 tx=0x7f85f4009060).stop 2023-11-15T03:02:58.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.646+0000 7f8609307700 1 -- 172.21.15.121:0/2198412573 shutdown_connections 2023-11-15T03:02:58.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.646+0000 7f8609307700 1 --2- 172.21.15.121:0/2198412573 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f85e405a510 0x7f85e405c9d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:58.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.646+0000 7f8609307700 1 --2- 172.21.15.121:0/2198412573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86040c29a0 0x7f8604163910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:58.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.646+0000 7f8609307700 1 --2- 172.21.15.121:0/2198412573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8604067150 0x7f860409fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:58.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.646+0000 7f8609307700 1 --2- 172.21.15.121:0/2198412573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8604066d70 0x7f86040a1030 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:02:58.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.646+0000 7f8609307700 1 -- 172.21.15.121:0/2198412573 >> 172.21.15.121:0/2198412573 conn(0x7f8604062190 msgr2=0x7f86040c37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:02:58.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.646+0000 7f8609307700 1 -- 172.21.15.121:0/2198412573 shutdown_connections 2023-11-15T03:02:58.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:02:58.646+0000 7f8609307700 1 -- 172.21.15.121:0/2198412573 wait complete. 2023-11-15T03:02:58.658 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 70 2023-11-15T03:02:59.078 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:02:20.703078+0000', 'modified': '2023-11-15T03:02:57.979061+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 80, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6486}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6486': {'gid': 6486, 'name': 'a', 'rank': 0, 'incarnation': 70, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.121:6836/834528709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 834528709}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 834528709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:02:59.078 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6486:up:replay 2023-11-15T03:03:00.079 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:03:00.149 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-11-15T03:03:00.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.237+0000 7fa523e5b700 1 Processor -- start 2023-11-15T03:03:00.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.237+0000 7fa523e5b700 1 -- start start 2023-11-15T03:03:00.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.241+0000 7fa523e5b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa51c152780 0x7fa51c159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:00.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.241+0000 7fa523e5b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa51c0a5510 0x7fa51c0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:00.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.241+0000 7fa523e5b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa51c0c3fa0 con 0x7fa51c0c65d0 2023-11-15T03:03:00.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.241+0000 7fa523e5b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa51c159630 con 0x7fa51c152780 2023-11-15T03:03:00.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.241+0000 7fa523e5b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa51c0c30b0 con 0x7fa51c0a5510 2023-11-15T03:03:00.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.241+0000 7fa521bf7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa51c152780 0x7fa51c159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:00.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.241+0000 7fa5213f6700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa51c0c65d0 0x7fa51c0c6a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47096/0 (socket says 172.21.15.121:47096) 2023-11-15T03:03:00.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.241+0000 7fa5213f6700 1 -- 172.21.15.121:0/2326222767 learned_addr learned my addr 172.21.15.121:0/2326222767 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:00.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.241+0000 7fa5223f8700 1 --2- 172.21.15.121:0/2326222767 >> v2:172.21.15.190:3300/0 conn(0x7fa51c0a5510 0x7fa51c0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:00.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.241+0000 7fa521bf7700 1 -- 172.21.15.121:0/2326222767 >> v2:172.21.15.190:3300/0 conn(0x7fa51c0a5510 msgr2=0x7fa51c0a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:00.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.241+0000 7fa521bf7700 1 --2- 172.21.15.121:0/2326222767 >> v2:172.21.15.190:3300/0 conn(0x7fa51c0a5510 0x7fa51c0a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:00.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.241+0000 7fa521bf7700 1 -- 172.21.15.121:0/2326222767 >> v1:172.21.15.121:6789/0 conn(0x7fa51c0c65d0 legacy=0x7fa51c0c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:00.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.241+0000 7fa521bf7700 1 -- 172.21.15.121:0/2326222767 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c0c3390 con 0x7fa51c152780 2023-11-15T03:03:00.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.241+0000 7fa520bf5700 1 -- 172.21.15.121:0/2326222767 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2410416143 0 0) 0x7fa51c0c3fa0 con 0x7fa51c0c65d0 2023-11-15T03:03:00.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.241+0000 7fa521bf7700 1 --2- 172.21.15.121:0/2326222767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa51c152780 0x7fa51c159140 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7fa51c0c0ec0 tx=0x7fa518009060).ready entity=mon.0 client_cookie=867e0f26e2336a20 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:00.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.241+0000 7fa520bf5700 1 -- 172.21.15.121:0/2326222767 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51800f510 con 0x7fa51c152780 2023-11-15T03:03:00.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.245+0000 7fa520bf5700 1 -- 172.21.15.121:0/2326222767 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51800f670 con 0x7fa51c152780 2023-11-15T03:03:00.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.245+0000 7fa520bf5700 1 -- 172.21.15.121:0/2326222767 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51800f9f0 con 0x7fa51c152780 2023-11-15T03:03:00.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.245+0000 7fa523e5b700 1 -- 172.21.15.121:0/2326222767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa51c152780 msgr2=0x7fa51c159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:00.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.245+0000 7fa523e5b700 1 --2- 172.21.15.121:0/2326222767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa51c152780 0x7fa51c159140 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7fa51c0c0ec0 tx=0x7fa518009060).stop 2023-11-15T03:03:00.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.245+0000 7fa523e5b700 1 -- 172.21.15.121:0/2326222767 shutdown_connections 2023-11-15T03:03:00.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.245+0000 7fa523e5b700 1 --2- 172.21.15.121:0/2326222767 >> v2:172.21.15.190:3300/0 conn(0x7fa51c0a5510 0x7fa51c0a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:00.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.245+0000 7fa523e5b700 1 --2- 172.21.15.121:0/2326222767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa51c152780 0x7fa51c159140 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:00.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.245+0000 7fa523e5b700 1 -- 172.21.15.121:0/2326222767 >> 172.21.15.121:0/2326222767 conn(0x7fa51c062190 msgr2=0x7fa51c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:00.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.245+0000 7fa523e5b700 1 -- 172.21.15.121:0/2326222767 shutdown_connections 2023-11-15T03:03:00.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.245+0000 7fa523e5b700 1 -- 172.21.15.121:0/2326222767 wait complete. 2023-11-15T03:03:00.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.249+0000 7fa523e5b700 1 Processor -- start 2023-11-15T03:03:00.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.249+0000 7fa523e5b700 1 -- start start 2023-11-15T03:03:00.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.249+0000 7fa523e5b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa51c0a5510 0x7fa51c166120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:00.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.249+0000 7fa523e5b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa51c0c65d0 0x7fa51c14dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:00.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.249+0000 7fa523e5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa51c152780 0x7fa51c150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:00.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.249+0000 7fa523e5b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c150e30 con 0x7fa51c0a5510 2023-11-15T03:03:00.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.249+0000 7fa523e5b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c0aaf80 con 0x7fa51c0c65d0 2023-11-15T03:03:00.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.249+0000 7fa523e5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c150b50 con 0x7fa51c152780 2023-11-15T03:03:00.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.249+0000 7fa521bf7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa51c0a5510 0x7fa51c166120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:00.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.249+0000 7fa521bf7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa51c0a5510 0x7fa51c166120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33704/0 (socket says 172.21.15.121:33704) 2023-11-15T03:03:00.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.249+0000 7fa521bf7700 1 -- 172.21.15.121:0/144352679 learned_addr learned my addr 172.21.15.121:0/144352679 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:00.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.249+0000 7fa5213f6700 1 --2- 172.21.15.121:0/144352679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa51c0c65d0 0x7fa51c14dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:00.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.249+0000 7fa5223f8700 1 --2- 172.21.15.121:0/144352679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa51c152780 0x7fa51c150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:00.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.249+0000 7fa521bf7700 1 -- 172.21.15.121:0/144352679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa51c152780 msgr2=0x7fa51c150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:00.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.249+0000 7fa521bf7700 1 --2- 172.21.15.121:0/144352679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa51c152780 0x7fa51c150370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:00.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.249+0000 7fa521bf7700 1 -- 172.21.15.121:0/144352679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa51c0c65d0 msgr2=0x7fa51c14dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:00.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.249+0000 7fa521bf7700 1 --2- 172.21.15.121:0/144352679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa51c0c65d0 0x7fa51c14dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:00.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.249+0000 7fa521bf7700 1 -- 172.21.15.121:0/144352679 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5180092a0 con 0x7fa51c0a5510 2023-11-15T03:03:00.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.249+0000 7fa5213f6700 1 --2- 172.21.15.121:0/144352679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa51c0c65d0 0x7fa51c14dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:00.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.253+0000 7fa521bf7700 1 --2- 172.21.15.121:0/144352679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa51c0a5510 0x7fa51c166120 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7fa518009dd0 tx=0x7fa518017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:00.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.253+0000 7fa511ffb700 1 -- 172.21.15.121:0/144352679 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51800f7e0 con 0x7fa51c0a5510 2023-11-15T03:03:00.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.253+0000 7fa511ffb700 1 -- 172.21.15.121:0/144352679 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51800f940 con 0x7fa51c0a5510 2023-11-15T03:03:00.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.253+0000 7fa511ffb700 1 -- 172.21.15.121:0/144352679 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa518022650 con 0x7fa51c0a5510 2023-11-15T03:03:00.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.253+0000 7fa523e5b700 1 -- 172.21.15.121:0/144352679 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51c0aa930 con 0x7fa51c0a5510 2023-11-15T03:03:00.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.253+0000 7fa523e5b700 1 -- 172.21.15.121:0/144352679 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51c150860 con 0x7fa51c0a5510 2023-11-15T03:03:00.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.253+0000 7fa523e5b700 1 -- 172.21.15.121:0/144352679 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa51c0aadb0 con 0x7fa51c0a5510 2023-11-15T03:03:00.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.257+0000 7fa511ffb700 1 -- 172.21.15.121:0/144352679 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa518022a00 con 0x7fa51c0a5510 2023-11-15T03:03:00.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.257+0000 7fa511ffb700 1 --2- 172.21.15.121:0/144352679 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa50005a510 0x7fa50005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:00.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.257+0000 7fa511ffb700 1 -- 172.21.15.121:0/144352679 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fa518075e70 con 0x7fa51c0a5510 2023-11-15T03:03:00.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.257+0000 7fa5213f6700 1 --2- 172.21.15.121:0/144352679 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa50005a510 0x7fa50005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:00.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.261+0000 7fa5213f6700 1 --2- 172.21.15.121:0/144352679 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa50005a510 0x7fa50005c9d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fa50c007fd0 tx=0x7fa50c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:00.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.265+0000 7fa511ffb700 1 -- 172.21.15.121:0/144352679 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa518049580 con 0x7fa51c0a5510 2023-11-15T03:03:00.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.465+0000 7fa523e5b700 1 -- 172.21.15.121:0/144352679 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa51c06c150 con 0x7fa51c0a5510 2023-11-15T03:03:00.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.469+0000 7fa511ffb700 1 -- 172.21.15.121:0/144352679 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4502 (secure 0 0 0) 0x7fa518022cb0 con 0x7fa51c0a5510 2023-11-15T03:03:00.477 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:00.477 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":72,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1508681155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1508681155},{"type":"v1","addr":"172.21.15.121:6837","nonce":1508681155}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6385,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3559276534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3559276534},{"type":"v1","addr":"172.21.15.190:6835","nonce":3559276534}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1842535107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1842535107},{"type":"v1","addr":"172.21.15.190:6833","nonce":1842535107}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:02:20.703078+0000","modified":"2023-11-15T03:03:00.005171+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":70,"state":"up:rejoin","state_seq":13,"addr":"172.21.15.121:6836/834528709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":834528709},{"type":"v1","addr":"172.21.15.121:6836","nonce":834528709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-15T03:03:00.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.469+0000 7fa523e5b700 1 -- 172.21.15.121:0/144352679 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa50005a510 msgr2=0x7fa50005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:00.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.469+0000 7fa523e5b700 1 --2- 172.21.15.121:0/144352679 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa50005a510 0x7fa50005c9d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fa50c007fd0 tx=0x7fa50c009040).stop 2023-11-15T03:03:00.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.469+0000 7fa523e5b700 1 -- 172.21.15.121:0/144352679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa51c0a5510 msgr2=0x7fa51c166120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:00.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.469+0000 7fa523e5b700 1 --2- 172.21.15.121:0/144352679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa51c0a5510 0x7fa51c166120 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7fa518009dd0 tx=0x7fa518017000).stop 2023-11-15T03:03:00.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.469+0000 7fa523e5b700 1 -- 172.21.15.121:0/144352679 shutdown_connections 2023-11-15T03:03:00.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.469+0000 7fa523e5b700 1 --2- 172.21.15.121:0/144352679 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa50005a510 0x7fa50005c9d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:00.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.469+0000 7fa523e5b700 1 --2- 172.21.15.121:0/144352679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa51c152780 0x7fa51c150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:00.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.469+0000 7fa523e5b700 1 --2- 172.21.15.121:0/144352679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa51c0c65d0 0x7fa51c14dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:00.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.469+0000 7fa523e5b700 1 --2- 172.21.15.121:0/144352679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa51c0a5510 0x7fa51c166120 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:00.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.469+0000 7fa523e5b700 1 -- 172.21.15.121:0/144352679 >> 172.21.15.121:0/144352679 conn(0x7fa51c062190 msgr2=0x7fa51c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:00.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.469+0000 7fa523e5b700 1 -- 172.21.15.121:0/144352679 shutdown_connections 2023-11-15T03:03:00.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:00.469+0000 7fa523e5b700 1 -- 172.21.15.121:0/144352679 wait complete. 2023-11-15T03:03:00.482 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 72 2023-11-15T03:03:00.907 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:02:20.703078+0000', 'modified': '2023-11-15T03:03:00.005171+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 80, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6486}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6486': {'gid': 6486, 'name': 'a', 'rank': 0, 'incarnation': 70, 'state': 'up:rejoin', 'state_seq': 13, 'addr': '172.21.15.121:6836/834528709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 834528709}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 834528709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:03:00.907 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6486:up:rejoin 2023-11-15T03:03:01.228 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:03:01.217+0000 7f5f09bf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:03:01.910 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:03:02.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.065+0000 7f52116db700 1 Processor -- start 2023-11-15T03:03:02.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.069+0000 7f52116db700 1 -- start start 2023-11-15T03:03:02.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.069+0000 7f52116db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f520c0aa240 0x7f520c067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.069+0000 7f52116db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c067f50 0x7f520c065d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.069+0000 7f52116db700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f520c0cb5c0 con 0x7f520c153bb0 2023-11-15T03:03:02.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.069+0000 7f52116db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f520c0aa6f0 con 0x7f520c0aa240 2023-11-15T03:03:02.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.069+0000 7f52116db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520c0c5900 con 0x7f520c067f50 2023-11-15T03:03:02.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.069+0000 7f520a7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f520c0aa240 0x7f520c067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.069+0000 7f520affd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f520c153bb0 0x7f520c0a9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:34990/0 (socket says 172.21.15.121:34990) 2023-11-15T03:03:02.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.069+0000 7f520affd700 1 -- 172.21.15.121:0/4184919838 learned_addr learned my addr 172.21.15.121:0/4184919838 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:02.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.069+0000 7f520b7fe700 1 --2- 172.21.15.121:0/4184919838 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c067f50 0x7f520c065d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.073+0000 7f5209ffb700 1 -- 172.21.15.121:0/4184919838 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347105015 0 0) 0x7f520c0cb5c0 con 0x7f520c153bb0 2023-11-15T03:03:02.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.073+0000 7f5209ffb700 1 -- 172.21.15.121:0/4184919838 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f40033f0 con 0x7f520c153bb0 2023-11-15T03:03:02.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.073+0000 7f520a7fc700 1 -- 172.21.15.121:0/4184919838 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c067f50 msgr2=0x7f520c065d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:02.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.073+0000 7f520a7fc700 1 --2- 172.21.15.121:0/4184919838 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c067f50 0x7f520c065d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.073+0000 7f520a7fc700 1 -- 172.21.15.121:0/4184919838 >> v1:172.21.15.190:6789/0 conn(0x7f520c153bb0 legacy=0x7f520c0a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:02.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.073+0000 7f520a7fc700 1 -- 172.21.15.121:0/4184919838 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520c0c9ca0 con 0x7f520c0aa240 2023-11-15T03:03:02.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.073+0000 7f520a7fc700 1 --2- 172.21.15.121:0/4184919838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f520c0aa240 0x7f520c067a60 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f520c0c5a40 tx=0x7f5200009060).ready entity=mon.0 client_cookie=973db9304d9dc033 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:02.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.073+0000 7f5209ffb700 1 -- 172.21.15.121:0/4184919838 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520000d4a0 con 0x7f520c0aa240 2023-11-15T03:03:02.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.073+0000 7f5209ffb700 1 -- 172.21.15.121:0/4184919838 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520000d600 con 0x7f520c0aa240 2023-11-15T03:03:02.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.073+0000 7f5209ffb700 1 -- 172.21.15.121:0/4184919838 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520000d980 con 0x7f520c0aa240 2023-11-15T03:03:02.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.073+0000 7f52116db700 1 -- 172.21.15.121:0/4184919838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f520c0aa240 msgr2=0x7f520c067a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:02.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.073+0000 7f52116db700 1 --2- 172.21.15.121:0/4184919838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f520c0aa240 0x7f520c067a60 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f520c0c5a40 tx=0x7f5200009060).stop 2023-11-15T03:03:02.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.073+0000 7f52116db700 1 -- 172.21.15.121:0/4184919838 shutdown_connections 2023-11-15T03:03:02.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.073+0000 7f52116db700 1 --2- 172.21.15.121:0/4184919838 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c067f50 0x7f520c065d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.073+0000 7f52116db700 1 --2- 172.21.15.121:0/4184919838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f520c0aa240 0x7f520c067a60 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.073+0000 7f52116db700 1 -- 172.21.15.121:0/4184919838 >> 172.21.15.121:0/4184919838 conn(0x7f520c058c50 msgr2=0x7f520c059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:02.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.073+0000 7f52116db700 1 -- 172.21.15.121:0/4184919838 shutdown_connections 2023-11-15T03:03:02.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.073+0000 7f52116db700 1 -- 172.21.15.121:0/4184919838 wait complete. 2023-11-15T03:03:02.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.077+0000 7f52116db700 1 Processor -- start 2023-11-15T03:03:02.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.077+0000 7f52116db700 1 -- start start 2023-11-15T03:03:02.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.077+0000 7f52116db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c067f50 0x7f520c0aa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.077+0000 7f52116db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f520c0aa240 0x7f520c14e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.077+0000 7f52116db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f520c153bb0 0x7f520c0c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.077+0000 7f52116db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c0c22f0 con 0x7f520c153bb0 2023-11-15T03:03:02.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.077+0000 7f52116db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c0c1fc0 con 0x7f520c0aa240 2023-11-15T03:03:02.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.077+0000 7f52116db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520c0ab090 con 0x7f520c067f50 2023-11-15T03:03:02.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.077+0000 7f520b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f520c153bb0 0x7f520c0c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.077+0000 7f520b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f520c153bb0 0x7f520c0c0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33724/0 (socket says 172.21.15.121:33724) 2023-11-15T03:03:02.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.077+0000 7f520b7fe700 1 -- 172.21.15.121:0/3285553290 learned_addr learned my addr 172.21.15.121:0/3285553290 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:02.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.077+0000 7f520affd700 1 --2- 172.21.15.121:0/3285553290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c067f50 0x7f520c0aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.077+0000 7f520a7fc700 1 --2- 172.21.15.121:0/3285553290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f520c0aa240 0x7f520c14e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.077+0000 7f520b7fe700 1 -- 172.21.15.121:0/3285553290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c067f50 msgr2=0x7f520c0aa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:02.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.077+0000 7f520b7fe700 1 --2- 172.21.15.121:0/3285553290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c067f50 0x7f520c0aa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.077+0000 7f520b7fe700 1 -- 172.21.15.121:0/3285553290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f520c0aa240 msgr2=0x7f520c14e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:02.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.077+0000 7f520b7fe700 1 --2- 172.21.15.121:0/3285553290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f520c0aa240 0x7f520c14e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.077+0000 7f520b7fe700 1 -- 172.21.15.121:0/3285553290 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52000092a0 con 0x7f520c153bb0 2023-11-15T03:03:02.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.077+0000 7f520a7fc700 1 --2- 172.21.15.121:0/3285553290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f520c0aa240 0x7f520c14e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:02.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.081+0000 7f520b7fe700 1 --2- 172.21.15.121:0/3285553290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f520c153bb0 0x7f520c0c0c50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f5204009e90 tx=0x7f520400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:02.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.081+0000 7f51f2ffd700 1 -- 172.21.15.121:0/3285553290 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520400cb50 con 0x7f520c153bb0 2023-11-15T03:03:02.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.081+0000 7f51f2ffd700 1 -- 172.21.15.121:0/3285553290 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520400f040 con 0x7f520c153bb0 2023-11-15T03:03:02.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.081+0000 7f51f2ffd700 1 -- 172.21.15.121:0/3285553290 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520400cec0 con 0x7f520c153bb0 2023-11-15T03:03:02.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.081+0000 7f52116db700 1 -- 172.21.15.121:0/3285553290 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f520c0ab370 con 0x7f520c153bb0 2023-11-15T03:03:02.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.081+0000 7f52116db700 1 -- 172.21.15.121:0/3285553290 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520c0aad30 con 0x7f520c153bb0 2023-11-15T03:03:02.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.081+0000 7f52116db700 1 -- 172.21.15.121:0/3285553290 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f520c06c150 con 0x7f520c153bb0 2023-11-15T03:03:02.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.081+0000 7f51f2ffd700 1 -- 172.21.15.121:0/3285553290 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5204013610 con 0x7f520c153bb0 2023-11-15T03:03:02.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.085+0000 7f51f2ffd700 1 --2- 172.21.15.121:0/3285553290 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f51f805a540 0x7f51f805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.085+0000 7f51f2ffd700 1 -- 172.21.15.121:0/3285553290 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f5204075160 con 0x7f520c153bb0 2023-11-15T03:03:02.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.085+0000 7f520affd700 1 --2- 172.21.15.121:0/3285553290 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f51f805a540 0x7f51f805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.085+0000 7f520affd700 1 --2- 172.21.15.121:0/3285553290 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f51f805a540 0x7f51f805ca00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f51fc007fd0 tx=0x7f51fc009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:02.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.089+0000 7f51f2ffd700 1 -- 172.21.15.121:0/3285553290 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f52040488b0 con 0x7f520c153bb0 2023-11-15T03:03:02.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.297+0000 7f52116db700 1 -- 172.21.15.121:0/3285553290 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f520c0c83c0 con 0x7f520c153bb0 2023-11-15T03:03:02.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.297+0000 7f51f2ffd700 1 -- 172.21.15.121:0/3285553290 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4502 (secure 0 0 0) 0x7f520404bed0 con 0x7f520c153bb0 2023-11-15T03:03:02.306 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:02.307 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":73,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1508681155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1508681155},{"type":"v1","addr":"172.21.15.121:6837","nonce":1508681155}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6385,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3559276534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3559276534},{"type":"v1","addr":"172.21.15.190:6835","nonce":3559276534}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1842535107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1842535107},{"type":"v1","addr":"172.21.15.190:6833","nonce":1842535107}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:02:20.703078+0000","modified":"2023-11-15T03:03:01.009934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":14,"addr":"172.21.15.121:6836/834528709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":834528709},{"type":"v1","addr":"172.21.15.121:6836","nonce":834528709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-15T03:03:02.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.301+0000 7f52116db700 1 -- 172.21.15.121:0/3285553290 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f51f805a540 msgr2=0x7f51f805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:02.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.301+0000 7f52116db700 1 --2- 172.21.15.121:0/3285553290 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f51f805a540 0x7f51f805ca00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f51fc007fd0 tx=0x7f51fc009040).stop 2023-11-15T03:03:02.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.301+0000 7f52116db700 1 -- 172.21.15.121:0/3285553290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f520c153bb0 msgr2=0x7f520c0c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:02.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.301+0000 7f52116db700 1 --2- 172.21.15.121:0/3285553290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f520c153bb0 0x7f520c0c0c50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f5204009e90 tx=0x7f520400b040).stop 2023-11-15T03:03:02.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.301+0000 7f52116db700 1 -- 172.21.15.121:0/3285553290 shutdown_connections 2023-11-15T03:03:02.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.301+0000 7f52116db700 1 --2- 172.21.15.121:0/3285553290 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f51f805a540 0x7f51f805ca00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.301+0000 7f52116db700 1 --2- 172.21.15.121:0/3285553290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f520c153bb0 0x7f520c0c0c50 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.301+0000 7f52116db700 1 --2- 172.21.15.121:0/3285553290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f520c0aa240 0x7f520c14e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.301+0000 7f52116db700 1 --2- 172.21.15.121:0/3285553290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c067f50 0x7f520c0aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.301+0000 7f52116db700 1 -- 172.21.15.121:0/3285553290 >> 172.21.15.121:0/3285553290 conn(0x7f520c058c50 msgr2=0x7f520c065d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:02.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.301+0000 7f52116db700 1 -- 172.21.15.121:0/3285553290 shutdown_connections 2023-11-15T03:03:02.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.301+0000 7f52116db700 1 -- 172.21.15.121:0/3285553290 wait complete. 2023-11-15T03:03:02.313 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 73 2023-11-15T03:03:02.733 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:02:20.703078+0000', 'modified': '2023-11-15T03:03:01.009934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 80, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6486}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6486': {'gid': 6486, 'name': 'a', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.121:6836/834528709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 834528709}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 834528709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:03:02.734 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T03:03:02.734 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-15T03:03:02.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.893+0000 7f134759e700 1 Processor -- start 2023-11-15T03:03:02.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.893+0000 7f134759e700 1 -- start start 2023-11-15T03:03:02.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f134759e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1348157d90 0x7f1348158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f134759e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f13480669b0 0x7f13480c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f134759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13480a8750 0x7f13480a8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f134759e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13480c1fb0 con 0x7f13480669b0 2023-11-15T03:03:02.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f134759e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f134815bbd0 con 0x7f1348157d90 2023-11-15T03:03:02.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f134759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13480a90d0 con 0x7f13480a8750 2023-11-15T03:03:02.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f133dd9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f13480669b0 0x7f13480c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f133dd9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f13480669b0 0x7f13480c1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33734/0 (socket says 172.21.15.121:33734) 2023-11-15T03:03:02.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f133dd9b700 1 -- 172.21.15.121:0/3089785843 learned_addr learned my addr 172.21.15.121:0/3089785843 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:02.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f134659c700 1 --2- 172.21.15.121:0/3089785843 >> v2:172.21.15.190:3300/0 conn(0x7f1348157d90 0x7f1348158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f1346d9d700 1 --2- 172.21.15.121:0/3089785843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13480a8750 0x7f13480a8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f133dd9b700 1 -- 172.21.15.121:0/3089785843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13480a8750 msgr2=0x7f13480a8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:02.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f133dd9b700 1 --2- 172.21.15.121:0/3089785843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13480a8750 0x7f13480a8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f133dd9b700 1 -- 172.21.15.121:0/3089785843 >> v2:172.21.15.190:3300/0 conn(0x7f1348157d90 msgr2=0x7f1348158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:02.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f133dd9b700 1 --2- 172.21.15.121:0/3089785843 >> v2:172.21.15.190:3300/0 conn(0x7f1348157d90 0x7f1348158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f133dd9b700 1 -- 172.21.15.121:0/3089785843 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13480bb910 con 0x7f13480669b0 2023-11-15T03:03:02.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f133dd9b700 1 --2- 172.21.15.121:0/3089785843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13480669b0 0x7f13480c1760 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7f13480599c0 tx=0x7f1330009840).ready entity=mon.0 client_cookie=f64628a6171880bc server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:02.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f1345d9b700 1 -- 172.21.15.121:0/3089785843 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133000e070 con 0x7f13480669b0 2023-11-15T03:03:02.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f1345d9b700 1 -- 172.21.15.121:0/3089785843 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133000bad0 con 0x7f13480669b0 2023-11-15T03:03:02.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.897+0000 7f1345d9b700 1 -- 172.21.15.121:0/3089785843 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133000be50 con 0x7f13480669b0 2023-11-15T03:03:02.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.901+0000 7f134759e700 1 -- 172.21.15.121:0/3089785843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13480669b0 msgr2=0x7f13480c1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:02.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.901+0000 7f134759e700 1 --2- 172.21.15.121:0/3089785843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13480669b0 0x7f13480c1760 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7f13480599c0 tx=0x7f1330009840).stop 2023-11-15T03:03:02.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.901+0000 7f134759e700 1 -- 172.21.15.121:0/3089785843 shutdown_connections 2023-11-15T03:03:02.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.901+0000 7f134759e700 1 --2- 172.21.15.121:0/3089785843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13480a8750 0x7f13480a8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.901+0000 7f134759e700 1 --2- 172.21.15.121:0/3089785843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13480669b0 0x7f13480c1760 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.901+0000 7f134759e700 1 --2- 172.21.15.121:0/3089785843 >> v2:172.21.15.190:3300/0 conn(0x7f1348157d90 0x7f1348158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.901+0000 7f134759e700 1 -- 172.21.15.121:0/3089785843 >> 172.21.15.121:0/3089785843 conn(0x7f1348067a60 msgr2=0x7f1348067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:02.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.901+0000 7f134759e700 1 -- 172.21.15.121:0/3089785843 shutdown_connections 2023-11-15T03:03:02.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.901+0000 7f134759e700 1 -- 172.21.15.121:0/3089785843 wait complete. 2023-11-15T03:03:02.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.901+0000 7f134759e700 1 Processor -- start 2023-11-15T03:03:02.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.901+0000 7f134759e700 1 -- start start 2023-11-15T03:03:02.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f134759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13480669b0 0x7f13480bdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f134759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13480a8750 0x7f13480c0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f134759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1348157d90 0x7f13481535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f134759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13480c5000 con 0x7f1348157d90 2023-11-15T03:03:02.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f134759e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13480c4380 con 0x7f13480669b0 2023-11-15T03:03:02.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f134759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13480c4660 con 0x7f13480a8750 2023-11-15T03:03:02.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f1346d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1348157d90 0x7f13481535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f1346d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1348157d90 0x7f13481535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33748/0 (socket says 172.21.15.121:33748) 2023-11-15T03:03:02.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f1346d9d700 1 -- 172.21.15.121:0/2843230128 learned_addr learned my addr 172.21.15.121:0/2843230128 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:02.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f134659c700 1 --2- 172.21.15.121:0/2843230128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13480669b0 0x7f13480bdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f133dd9b700 1 --2- 172.21.15.121:0/2843230128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13480a8750 0x7f13480c0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f1346d9d700 1 -- 172.21.15.121:0/2843230128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13480a8750 msgr2=0x7f13480c0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:02.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f1346d9d700 1 --2- 172.21.15.121:0/2843230128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13480a8750 0x7f13480c0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f1346d9d700 1 -- 172.21.15.121:0/2843230128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13480669b0 msgr2=0x7f13480bdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:02.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f1346d9d700 1 --2- 172.21.15.121:0/2843230128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13480669b0 0x7f13480bdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f1346d9d700 1 -- 172.21.15.121:0/2843230128 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1330009ae0 con 0x7f1348157d90 2023-11-15T03:03:02.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f1346d9d700 1 --2- 172.21.15.121:0/2843230128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1348157d90 0x7f13481535d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7f134000d890 tx=0x7f1340008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:02.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f133effd700 1 -- 172.21.15.121:0/2843230128 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340011590 con 0x7f1348157d90 2023-11-15T03:03:02.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f133effd700 1 -- 172.21.15.121:0/2843230128 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13400116f0 con 0x7f1348157d90 2023-11-15T03:03:02.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f134759e700 1 -- 172.21.15.121:0/2843230128 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13480c4940 con 0x7f1348157d90 2023-11-15T03:03:02.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.905+0000 7f133effd700 1 -- 172.21.15.121:0/2843230128 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340011a70 con 0x7f1348157d90 2023-11-15T03:03:02.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.909+0000 7f134759e700 1 -- 172.21.15.121:0/2843230128 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13480be460 con 0x7f1348157d90 2023-11-15T03:03:02.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.909+0000 7f133effd700 1 -- 172.21.15.121:0/2843230128 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f134001c460 con 0x7f1348157d90 2023-11-15T03:03:02.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.913+0000 7f133effd700 1 --2- 172.21.15.121:0/2843230128 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f132805a4e0 0x7f132805c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.913+0000 7f133effd700 1 -- 172.21.15.121:0/2843230128 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f1340074580 con 0x7f1348157d90 2023-11-15T03:03:02.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.913+0000 7f1356d43740 1 Processor -- start 2023-11-15T03:03:02.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.913+0000 7f134659c700 1 --2- 172.21.15.121:0/2843230128 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f132805a4e0 0x7f132805c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.913+0000 7f1356d43740 1 -- start start 2023-11-15T03:03:02.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.913+0000 7f134659c700 1 --2- 172.21.15.121:0/2843230128 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f132805a4e0 0x7f132805c9a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f133800bb10 tx=0x7f1338008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:02.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.913+0000 7f1356d43740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21bd8d0 0x21bdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.913+0000 7f1356d43740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21beb20 0x2203bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.913+0000 7f1356d43740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22040a0 0x2204520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.913+0000 7f1356d43740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x213e410 con 0x22040a0 2023-11-15T03:03:02.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.913+0000 7f1356d43740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x213e550 con 0x21beb20 2023-11-15T03:03:02.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.913+0000 7f1356d43740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x213ea00 con 0x21bd8d0 2023-11-15T03:03:02.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.913+0000 7f1346d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22040a0 0x2204520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.913+0000 7f1346d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22040a0 0x2204520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33760/0 (socket says 172.21.15.121:33760) 2023-11-15T03:03:02.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.913+0000 7f1346d9d700 1 -- 172.21.15.121:0/1233609699 learned_addr learned my addr 172.21.15.121:0/1233609699 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:02.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.913+0000 7f133dd9b700 1 --2- 172.21.15.121:0/1233609699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21beb20 0x2203bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.913+0000 7f134659c700 1 --2- 172.21.15.121:0/1233609699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21bd8d0 0x21bdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f1346d9d700 1 -- 172.21.15.121:0/1233609699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21bd8d0 msgr2=0x21bdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:02.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f1346d9d700 1 --2- 172.21.15.121:0/1233609699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21bd8d0 0x21bdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f1346d9d700 1 -- 172.21.15.121:0/1233609699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21beb20 msgr2=0x2203bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:02.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f1346d9d700 1 --2- 172.21.15.121:0/1233609699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21beb20 0x2203bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f1346d9d700 1 -- 172.21.15.121:0/1233609699 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2137730 con 0x22040a0 2023-11-15T03:03:02.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f1346d9d700 1 --2- 172.21.15.121:0/1233609699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22040a0 0x2204520 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7f1340047d70 tx=0x7f1340010000).ready entity=mon.0 client_cookie=9ebc6334493af90b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:02.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f13277fe700 1 -- 172.21.15.121:0/1233609699 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340050fb0 con 0x22040a0 2023-11-15T03:03:02.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f13277fe700 1 -- 172.21.15.121:0/1233609699 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1340049c10 con 0x22040a0 2023-11-15T03:03:02.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f13277fe700 1 -- 172.21.15.121:0/1233609699 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13400740d0 con 0x22040a0 2023-11-15T03:03:02.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f1356d43740 1 -- 172.21.15.121:0/1233609699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22040a0 msgr2=0x2204520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:02.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f1356d43740 1 --2- 172.21.15.121:0/1233609699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22040a0 0x2204520 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7f1340047d70 tx=0x7f1340010000).stop 2023-11-15T03:03:02.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f1356d43740 1 -- 172.21.15.121:0/1233609699 shutdown_connections 2023-11-15T03:03:02.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f1356d43740 1 --2- 172.21.15.121:0/1233609699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22040a0 0x2204520 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f1356d43740 1 --2- 172.21.15.121:0/1233609699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21beb20 0x2203bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f1356d43740 1 --2- 172.21.15.121:0/1233609699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21bd8d0 0x21bdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f1356d43740 1 -- 172.21.15.121:0/1233609699 >> 172.21.15.121:0/1233609699 conn(0x1f9c550 msgr2=0x2204b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:02.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f1356d43740 1 -- 172.21.15.121:0/1233609699 shutdown_connections 2023-11-15T03:03:02.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f1356d43740 1 -- 172.21.15.121:0/1233609699 wait complete. 2023-11-15T03:03:02.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f1356d43740 1 -- start start 2023-11-15T03:03:02.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.917+0000 7f1356d43740 1 Processor -- start 2023-11-15T03:03:02.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1356d43740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21bd8d0 0x21fcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1356d43740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 0x21fff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1356d43740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22040a0 0x2200440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1356d43740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2109650 con 0x21beb20 2023-11-15T03:03:02.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1356d43740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x2200930 con 0x21bd8d0 2023-11-15T03:03:02.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1356d43740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x21fd010 con 0x22040a0 2023-11-15T03:03:02.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1356d43740 10 client.0 ll_register_callbacks cb 0x2176020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:03:02.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f133dd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 0x21fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f133dd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 0x21fff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33772/0 (socket says 172.21.15.121:33772) 2023-11-15T03:03:02.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f133dd9b700 1 -- 172.21.15.121:0/1164324893 learned_addr learned my addr 172.21.15.121:0/1164324893 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:02.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1346d9d700 1 --2- 172.21.15.121:0/1164324893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22040a0 0x2200440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f134659c700 1 --2- 172.21.15.121:0/1164324893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21bd8d0 0x21fcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f133dd9b700 1 -- 172.21.15.121:0/1164324893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22040a0 msgr2=0x2200440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:02.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f133dd9b700 1 --2- 172.21.15.121:0/1164324893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22040a0 0x2200440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f133dd9b700 1 -- 172.21.15.121:0/1164324893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21bd8d0 msgr2=0x21fcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:02.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f133dd9b700 1 --2- 172.21.15.121:0/1164324893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21bd8d0 0x21fcb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f133dd9b700 1 -- 172.21.15.121:0/1164324893 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13480be460 con 0x21beb20 2023-11-15T03:03:02.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f133dd9b700 1 --2- 172.21.15.121:0/1164324893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 0x21fff50 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7f1330010000 tx=0x7f1330009840).ready entity=mon.0 client_cookie=cd984166d77e0027 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:02.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1324ff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:03:02.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1324ff9700 1 -- 172.21.15.121:0/1164324893 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1330013700 con 0x21beb20 2023-11-15T03:03:02.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1324ff9700 1 -- 172.21.15.121:0/1164324893 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f1330013860 con 0x21beb20 2023-11-15T03:03:02.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1356d43740 1 -- 172.21.15.121:0/1164324893 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x21341a0 con 0x21beb20 2023-11-15T03:03:02.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1324ff9700 1 -- 172.21.15.121:0/1164324893 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1330013a20 con 0x21beb20 2023-11-15T03:03:02.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1356d43740 10 client.6801 fetch_fsmap learned FSMap version 73 2023-11-15T03:03:02.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1356d43740 1 -- 172.21.15.121:0/1164324893 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x2132960 con 0x21beb20 2023-11-15T03:03:02.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1324ff9700 1 -- 172.21.15.121:0/1164324893 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x21341a0 con 0x21beb20 2023-11-15T03:03:02.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1324ff9700 1 -- 172.21.15.121:0/1164324893 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 73) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1330013b80 con 0x21beb20 2023-11-15T03:03:02.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1356d43740 10 client.6801 fetch_fsmap finished waiting for FSMap version 73 2023-11-15T03:03:02.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1356d43740 10 client.6801 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:03:02.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1356d43740 20 client.6801 populate_metadata read hostname 'smithi121' 2023-11-15T03:03:02.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1356d43740 1 --2- 172.21.15.121:0/1164324893 >> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] conn(0x2214730 0x2214b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1356d43740 4 client.6801 mds_command: new command op to 6486 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:03:02.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.921+0000 7f1356d43740 1 -- 172.21.15.121:0/1164324893 --> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x212a6f0 con 0x2214730 2023-11-15T03:03:02.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.925+0000 7f134659c700 1 --2- 172.21.15.121:0/1164324893 >> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] conn(0x2214730 0x2214b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.925+0000 7f134659c700 1 --2- 172.21.15.121:0/1164324893 >> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] conn(0x2214730 0x2214b10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=5b22cd69272dc114 server_cookie=25f1c5672fe76efe in_seq=0 out_seq=0 2023-11-15T03:03:02.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.925+0000 7f1324ff9700 10 client.6801 ms_handle_connect on v2:172.21.15.121:6834/834528709 2023-11-15T03:03:02.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.925+0000 7f1324ff9700 1 -- 172.21.15.121:0/1164324893 <== mds.0 v2:172.21.15.121:6834/834528709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x212a6f0 con 0x2214730 2023-11-15T03:03:02.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.925+0000 7f1324ff9700 10 client.6801 handle_command_reply: tid=0 2023-11-15T03:03:02.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.925+0000 7f1356d43740 1 client.6801 shutdown 2023-11-15T03:03:02.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.925+0000 7f1356d43740 1 -- 172.21.15.121:0/1164324893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 msgr2=0x21fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:02.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.925+0000 7f1356d43740 1 --2- 172.21.15.121:0/1164324893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 0x21fff50 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7f1330010000 tx=0x7f1330009840).stop 2023-11-15T03:03:02.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.925+0000 7f1356d43740 1 -- 172.21.15.121:0/1164324893 shutdown_connections 2023-11-15T03:03:02.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.925+0000 7f1356d43740 1 --2- 172.21.15.121:0/1164324893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22040a0 0x2200440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.925+0000 7f1356d43740 1 --2- 172.21.15.121:0/1164324893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 0x21fff50 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.925+0000 7f1356d43740 1 --2- 172.21.15.121:0/1164324893 >> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] conn(0x2214730 0x2214b10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.925+0000 7f1356d43740 1 --2- 172.21.15.121:0/1164324893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21bd8d0 0x21fcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.925+0000 7f1324ff9700 0 client.6801 ms_handle_reset on v2:172.21.15.121:6834/834528709 2023-11-15T03:03:02.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.925+0000 7f1356d43740 1 -- 172.21.15.121:0/1164324893 >> 172.21.15.121:0/1164324893 conn(0x1f9c550 msgr2=0x22044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:02.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.925+0000 7f1356d43740 1 -- 172.21.15.121:0/1164324893 shutdown_connections 2023-11-15T03:03:02.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.925+0000 7f1356d43740 1 -- 172.21.15.121:0/1164324893 wait complete. 2023-11-15T03:03:02.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.925+0000 7f1356d43740 20 client.6801 trim_cache size 0 max 16384 2023-11-15T03:03:02.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.945+0000 7f1356d43740 1 Processor -- start 2023-11-15T03:03:02.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.945+0000 7f1356d43740 1 -- start start 2023-11-15T03:03:02.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.945+0000 7f1356d43740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21bd8d0 0x2200350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.945+0000 7f1356d43740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 0x220fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.945+0000 7f1356d43740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22040a0 0x22100f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.945+0000 7f1356d43740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x21b9b60 con 0x21beb20 2023-11-15T03:03:02.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.945+0000 7f1356d43740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x2172120 con 0x21bd8d0 2023-11-15T03:03:02.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.945+0000 7f1356d43740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x21fce00 con 0x22040a0 2023-11-15T03:03:02.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.945+0000 7f133dd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 0x220fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.945+0000 7f133dd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 0x220fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33776/0 (socket says 172.21.15.121:33776) 2023-11-15T03:03:02.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.945+0000 7f133dd9b700 1 -- 172.21.15.121:0/3491218573 learned_addr learned my addr 172.21.15.121:0/3491218573 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:02.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.945+0000 7f1346d9d700 1 --2- 172.21.15.121:0/3491218573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22040a0 0x22100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.945+0000 7f134659c700 1 --2- 172.21.15.121:0/3491218573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21bd8d0 0x2200350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.945+0000 7f133dd9b700 1 -- 172.21.15.121:0/3491218573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22040a0 msgr2=0x22100f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:02.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.945+0000 7f133dd9b700 1 --2- 172.21.15.121:0/3491218573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22040a0 0x22100f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.945+0000 7f133dd9b700 1 -- 172.21.15.121:0/3491218573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21bd8d0 msgr2=0x2200350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:02.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.945+0000 7f133dd9b700 1 --2- 172.21.15.121:0/3491218573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21bd8d0 0x2200350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.945+0000 7f133dd9b700 1 -- 172.21.15.121:0/3491218573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x21326b0 con 0x21beb20 2023-11-15T03:03:02.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.945+0000 7f134659c700 1 --2- 172.21.15.121:0/3491218573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21bd8d0 0x2200350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:02.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.949+0000 7f133dd9b700 1 --2- 172.21.15.121:0/3491218573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 0x220fc00 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f1330007030 tx=0x7f1330022070).ready entity=mon.0 client_cookie=8c356536da3cc243 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:02.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.949+0000 7f1327fff700 1 -- 172.21.15.121:0/3491218573 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1330029410 con 0x21beb20 2023-11-15T03:03:02.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.949+0000 7f1327fff700 1 -- 172.21.15.121:0/3491218573 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1330024070 con 0x21beb20 2023-11-15T03:03:02.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.949+0000 7f1327fff700 1 -- 172.21.15.121:0/3491218573 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1330029600 con 0x21beb20 2023-11-15T03:03:02.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.949+0000 7f1356d43740 1 -- 172.21.15.121:0/3491218573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 msgr2=0x220fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:02.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.949+0000 7f1356d43740 1 --2- 172.21.15.121:0/3491218573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 0x220fc00 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f1330007030 tx=0x7f1330022070).stop 2023-11-15T03:03:02.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.949+0000 7f1356d43740 1 -- 172.21.15.121:0/3491218573 shutdown_connections 2023-11-15T03:03:02.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.949+0000 7f1356d43740 1 --2- 172.21.15.121:0/3491218573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22040a0 0x22100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.949+0000 7f1356d43740 1 --2- 172.21.15.121:0/3491218573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 0x220fc00 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.949+0000 7f1356d43740 1 --2- 172.21.15.121:0/3491218573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21bd8d0 0x2200350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.949+0000 7f1356d43740 1 -- 172.21.15.121:0/3491218573 >> 172.21.15.121:0/3491218573 conn(0x1f9c550 msgr2=0x21bdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:02.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.949+0000 7f1356d43740 1 -- 172.21.15.121:0/3491218573 shutdown_connections 2023-11-15T03:03:02.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.949+0000 7f1356d43740 1 -- 172.21.15.121:0/3491218573 wait complete. 2023-11-15T03:03:02.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.949+0000 7f1356d43740 1 -- start start 2023-11-15T03:03:02.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.949+0000 7f1356d43740 1 Processor -- start 2023-11-15T03:03:02.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.953+0000 7f1356d43740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21bd8d0 0x21bbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.953+0000 7f1356d43740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 0x2266be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.953+0000 7f1356d43740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22040a0 0x2201ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.953+0000 7f1356d43740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x22015b0 con 0x21beb20 2023-11-15T03:03:02.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.953+0000 7f1356d43740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x2267690 con 0x22040a0 2023-11-15T03:03:02.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.953+0000 7f1356d43740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2201850 con 0x21bd8d0 2023-11-15T03:03:02.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.953+0000 7f1356d43740 10 client.0 ll_register_callbacks cb 0x7f13300093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:03:02.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.953+0000 7f133dd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 0x2266be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.953+0000 7f133dd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 0x2266be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33788/0 (socket says 172.21.15.121:33788) 2023-11-15T03:03:02.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.953+0000 7f133dd9b700 1 -- 172.21.15.121:0/3565051686 learned_addr learned my addr 172.21.15.121:0/3565051686 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:02.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.953+0000 7f134659c700 1 --2- 172.21.15.121:0/3565051686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21bd8d0 0x21bbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.953+0000 7f1346d9d700 1 --2- 172.21.15.121:0/3565051686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22040a0 0x2201ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.953+0000 7f133dd9b700 1 -- 172.21.15.121:0/3565051686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21bd8d0 msgr2=0x21bbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:02.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.953+0000 7f133dd9b700 1 --2- 172.21.15.121:0/3565051686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21bd8d0 0x21bbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.953+0000 7f133dd9b700 1 -- 172.21.15.121:0/3565051686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22040a0 msgr2=0x2201ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:02.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.953+0000 7f133dd9b700 1 --2- 172.21.15.121:0/3565051686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22040a0 0x2201ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.953+0000 7f133dd9b700 1 -- 172.21.15.121:0/3565051686 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2132960 con 0x21beb20 2023-11-15T03:03:02.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.953+0000 7f133dd9b700 1 --2- 172.21.15.121:0/3565051686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 0x2266be0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f1330037000 tx=0x7f1330034070).ready entity=mon.0 client_cookie=65fd16fc7ded8b59 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:02.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.953+0000 7f1324ff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:03:02.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1324ff9700 1 -- 172.21.15.121:0/3565051686 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1330029970 con 0x21beb20 2023-11-15T03:03:02.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1324ff9700 1 -- 172.21.15.121:0/3565051686 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f1330029ad0 con 0x21beb20 2023-11-15T03:03:02.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1356d43740 1 -- 172.21.15.121:0/3565051686 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1340049c10 con 0x21beb20 2023-11-15T03:03:02.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1324ff9700 1 -- 172.21.15.121:0/3565051686 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1330013630 con 0x21beb20 2023-11-15T03:03:02.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1356d43740 10 client.6807 fetch_fsmap learned FSMap version 73 2023-11-15T03:03:02.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1356d43740 1 -- 172.21.15.121:0/3565051686 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x203a850 con 0x21beb20 2023-11-15T03:03:02.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1324ff9700 1 -- 172.21.15.121:0/3565051686 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1340049c10 con 0x21beb20 2023-11-15T03:03:02.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1324ff9700 1 -- 172.21.15.121:0/3565051686 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 73) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1330013790 con 0x21beb20 2023-11-15T03:03:02.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1356d43740 10 client.6807 fetch_fsmap finished waiting for FSMap version 73 2023-11-15T03:03:02.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1356d43740 10 client.6807 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:03:02.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1356d43740 20 client.6807 populate_metadata read hostname 'smithi121' 2023-11-15T03:03:02.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1356d43740 1 --2- 172.21.15.121:0/3565051686 >> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] conn(0x2263ca0 0x2264080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:02.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1356d43740 4 client.6807 mds_command: new command op to 6486 tid=0[{"prefix": "status"}] 2023-11-15T03:03:02.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1356d43740 1 -- 172.21.15.121:0/3565051686 --> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1ff34e0 con 0x2263ca0 2023-11-15T03:03:02.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f134659c700 1 --2- 172.21.15.121:0/3565051686 >> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] conn(0x2263ca0 0x2264080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:02.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1324ff9700 10 client.6807 ms_handle_connect on v2:172.21.15.121:6834/834528709 2023-11-15T03:03:02.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f134659c700 1 --2- 172.21.15.121:0/3565051686 >> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] conn(0x2263ca0 0x2264080 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=5b3a99f45bba025a server_cookie=1495fc1ab35d0793 in_seq=0 out_seq=0 2023-11-15T03:03:02.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1324ff9700 1 -- 172.21.15.121:0/3565051686 <== mds.0 v2:172.21.15.121:6834/834528709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x1ff34e0 con 0x2263ca0 2023-11-15T03:03:02.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1324ff9700 10 client.6807 handle_command_reply: tid=0 2023-11-15T03:03:02.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1356d43740 1 client.6807 shutdown 2023-11-15T03:03:02.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1356d43740 1 -- 172.21.15.121:0/3565051686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 msgr2=0x2266be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:02.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1356d43740 1 --2- 172.21.15.121:0/3565051686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 0x2266be0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f1330037000 tx=0x7f1330034070).stop 2023-11-15T03:03:02.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1356d43740 1 -- 172.21.15.121:0/3565051686 shutdown_connections 2023-11-15T03:03:02.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1356d43740 1 --2- 172.21.15.121:0/3565051686 >> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] conn(0x2263ca0 0x2264080 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1356d43740 1 --2- 172.21.15.121:0/3565051686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x22040a0 0x2201ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1356d43740 1 --2- 172.21.15.121:0/3565051686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21beb20 0x2266be0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1356d43740 1 --2- 172.21.15.121:0/3565051686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21bd8d0 0x21bbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1324ff9700 0 client.6807 ms_handle_reset on v2:172.21.15.121:6834/834528709 2023-11-15T03:03:02.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.957+0000 7f1356d43740 1 -- 172.21.15.121:0/3565051686 >> 172.21.15.121:0/3565051686 conn(0x1f9c550 msgr2=0x220fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:02.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.961+0000 7f1356d43740 1 -- 172.21.15.121:0/3565051686 shutdown_connections 2023-11-15T03:03:02.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.961+0000 7f1356d43740 1 -- 172.21.15.121:0/3565051686 wait complete. 2023-11-15T03:03:02.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.961+0000 7f1356d43740 20 client.6807 trim_cache size 0 max 16384 2023-11-15T03:03:02.980 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:03:02.980 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T03:03:02.980 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T03:03:02.980 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 6486, 2023-11-15T03:03:02.981 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T03:03:02.982 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T03:03:02.982 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T03:03:02.983 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 4.9728502959999998, 2023-11-15T03:03:02.983 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 73, 2023-11-15T03:03:02.983 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 80, 2023-11-15T03:03:02.983 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 80, 2023-11-15T03:03:02.984 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 47.688729852999998 2023-11-15T03:03:02.984 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:03:02.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.961+0000 7f134759e700 1 -- 172.21.15.121:0/2843230128 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f132805a4e0 msgr2=0x7f132805c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:02.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.961+0000 7f134759e700 1 --2- 172.21.15.121:0/2843230128 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f132805a4e0 0x7f132805c9a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f133800bb10 tx=0x7f1338008040).stop 2023-11-15T03:03:02.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.961+0000 7f134759e700 1 -- 172.21.15.121:0/2843230128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1348157d90 msgr2=0x7f13481535d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:02.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.961+0000 7f134759e700 1 --2- 172.21.15.121:0/2843230128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1348157d90 0x7f13481535d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7f134000d890 tx=0x7f1340008040).stop 2023-11-15T03:03:02.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.961+0000 7f134759e700 1 -- 172.21.15.121:0/2843230128 shutdown_connections 2023-11-15T03:03:02.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.961+0000 7f134759e700 1 --2- 172.21.15.121:0/2843230128 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f132805a4e0 0x7f132805c9a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.961+0000 7f134759e700 1 --2- 172.21.15.121:0/2843230128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1348157d90 0x7f13481535d0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.961+0000 7f134759e700 1 --2- 172.21.15.121:0/2843230128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13480a8750 0x7f13480c0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.961+0000 7f134759e700 1 --2- 172.21.15.121:0/2843230128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13480669b0 0x7f13480bdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:02.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.961+0000 7f134759e700 1 -- 172.21.15.121:0/2843230128 >> 172.21.15.121:0/2843230128 conn(0x7f1348067a60 msgr2=0x7f1348155b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:02.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.961+0000 7f134759e700 1 -- 172.21.15.121:0/2843230128 shutdown_connections 2023-11-15T03:03:02.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:02.961+0000 7f134759e700 1 -- 172.21.15.121:0/2843230128 wait complete. 2023-11-15T03:03:02.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:03:02.994 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat a/ && find a/') 2023-11-15T03:03:03.022 INFO:teuthology.orchestra.run.smithi121.stdout: File: a/ 2023-11-15T03:03:03.022 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 2 Blocks: 0 IO Block: 65536 directory 2023-11-15T03:03:03.022 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 31h/49d Inode: 1099511627776 Links: 4 2023-11-15T03:03:03.023 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0775/drwxrwxr-x) Uid: ( 1000/ ubuntu) Gid: ( 1267/ ubuntu) 2023-11-15T03:03:03.023 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-11-15 03:02:34.190589639 +0000 2023-11-15T03:03:03.023 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-11-15 03:02:39.910457729 +0000 2023-11-15T03:03:03.023 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-11-15 03:02:39.910457729 +0000 2023-11-15T03:03:03.023 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-11-15T03:03:03.036 INFO:teuthology.orchestra.run.smithi121.stdout:a/ 2023-11-15T03:03:03.036 INFO:teuthology.orchestra.run.smithi121.stdout:a/z 2023-11-15T03:03:03.036 INFO:teuthology.orchestra.run.smithi121.stdout:a/z/d 2023-11-15T03:03:03.036 INFO:teuthology.orchestra.run.smithi121.stdout:a/c 2023-11-15T03:03:03.037 INFO:teuthology.orchestra.run.smithi121.stdout:a/c/d 2023-11-15T03:03:03.038 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2023-11-15T03:03:03.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.233+0000 7f52a2547700 1 Processor -- start 2023-11-15T03:03:03.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.233+0000 7f52a2547700 1 -- start start 2023-11-15T03:03:03.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.233+0000 7f52a2547700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f529c151af0 0x7f529c0a7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.233+0000 7f52a2547700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c0a5720 0x7f529c0c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.233+0000 7f52a2547700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529c1528b0 con 0x7f529c0c3430 2023-11-15T03:03:03.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.233+0000 7f52a2547700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529c066d80 con 0x7f529c151af0 2023-11-15T03:03:03.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.233+0000 7f52a2547700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529c151ff0 con 0x7f529c0a5720 2023-11-15T03:03:03.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.233+0000 7f52a1545700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f529c151af0 0x7f529c0a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.233+0000 7f52a1545700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f529c151af0 0x7f529c0a7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:33468/0 (socket says 172.21.15.121:33468) 2023-11-15T03:03:03.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.233+0000 7f52a1545700 1 -- 172.21.15.121:0/1254755285 learned_addr learned my addr 172.21.15.121:0/1254755285 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:03.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.233+0000 7f52a0d44700 1 --2- 172.21.15.121:0/1254755285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c0a5720 0x7f529c0c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.237+0000 7f52a0d44700 1 -- 172.21.15.121:0/1254755285 >> v2:172.21.15.190:3300/0 conn(0x7f529c151af0 msgr2=0x7f529c0a7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:03.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.237+0000 7f52a0d44700 1 --2- 172.21.15.121:0/1254755285 >> v2:172.21.15.190:3300/0 conn(0x7f529c151af0 0x7f529c0a7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.237+0000 7f52a0d44700 1 -- 172.21.15.121:0/1254755285 >> v1:172.21.15.190:6789/0 conn(0x7f529c0c3430 legacy=0x7f529c0c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:03.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.237+0000 7f52a0d44700 1 -- 172.21.15.121:0/1254755285 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c0c3890 con 0x7f529c0a5720 2023-11-15T03:03:03.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.237+0000 7f52a0d44700 1 --2- 172.21.15.121:0/1254755285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c0a5720 0x7f529c0c2f40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f528c009760 tx=0x7f528c009060).ready entity=mon.2 client_cookie=409e5148b15ce156 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:03.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.237+0000 7f5293fff700 1 -- 172.21.15.121:0/1254755285 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c00f510 con 0x7f529c0a5720 2023-11-15T03:03:03.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.237+0000 7f5293fff700 1 -- 172.21.15.121:0/1254755285 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528c00f670 con 0x7f529c0a5720 2023-11-15T03:03:03.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.237+0000 7f5293fff700 1 -- 172.21.15.121:0/1254755285 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c00f9f0 con 0x7f529c0a5720 2023-11-15T03:03:03.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.237+0000 7f52a2547700 1 -- 172.21.15.121:0/1254755285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c0a5720 msgr2=0x7f529c0c2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:03.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.237+0000 7f52a2547700 1 --2- 172.21.15.121:0/1254755285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c0a5720 0x7f529c0c2f40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f528c009760 tx=0x7f528c009060).stop 2023-11-15T03:03:03.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.237+0000 7f52a2547700 1 -- 172.21.15.121:0/1254755285 shutdown_connections 2023-11-15T03:03:03.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.237+0000 7f52a2547700 1 --2- 172.21.15.121:0/1254755285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c0a5720 0x7f529c0c2f40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.237+0000 7f52a2547700 1 --2- 172.21.15.121:0/1254755285 >> v2:172.21.15.190:3300/0 conn(0x7f529c151af0 0x7f529c0a7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.237+0000 7f52a2547700 1 -- 172.21.15.121:0/1254755285 >> 172.21.15.121:0/1254755285 conn(0x7f529c067a60 msgr2=0x7f529c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:03.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.237+0000 7f52a2547700 1 -- 172.21.15.121:0/1254755285 shutdown_connections 2023-11-15T03:03:03.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.237+0000 7f52a2547700 1 -- 172.21.15.121:0/1254755285 wait complete. 2023-11-15T03:03:03.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.241+0000 7f52a2547700 1 Processor -- start 2023-11-15T03:03:03.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.241+0000 7f52a2547700 1 -- start start 2023-11-15T03:03:03.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a2547700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f529c0a5720 0x7f529c14d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a2547700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f529c0c3430 0x7f529c0c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a2547700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c0c5df0 0x7f529c0a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a2547700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c14d7c0 con 0x7f529c0c3430 2023-11-15T03:03:03.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a2547700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c0c6f30 con 0x7f529c0a5720 2023-11-15T03:03:03.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a2547700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529c0c6320 con 0x7f529c0c5df0 2023-11-15T03:03:03.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a0d44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f529c0c3430 0x7f529c0c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a0d44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f529c0c3430 0x7f529c0c5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33802/0 (socket says 172.21.15.121:33802) 2023-11-15T03:03:03.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a0d44700 1 -- 172.21.15.121:0/2196583547 learned_addr learned my addr 172.21.15.121:0/2196583547 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:03.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a1545700 1 --2- 172.21.15.121:0/2196583547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f529c0a5720 0x7f529c14d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a1d46700 1 --2- 172.21.15.121:0/2196583547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c0c5df0 0x7f529c0a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a0d44700 1 -- 172.21.15.121:0/2196583547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c0c5df0 msgr2=0x7f529c0a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:03.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a0d44700 1 --2- 172.21.15.121:0/2196583547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c0c5df0 0x7f529c0a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a0d44700 1 -- 172.21.15.121:0/2196583547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f529c0a5720 msgr2=0x7f529c14d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:03.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a0d44700 1 --2- 172.21.15.121:0/2196583547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f529c0a5720 0x7f529c14d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a0d44700 1 -- 172.21.15.121:0/2196583547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c0092a0 con 0x7f529c0c3430 2023-11-15T03:03:03.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a1d46700 1 --2- 172.21.15.121:0/2196583547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c0c5df0 0x7f529c0a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:03.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a1545700 1 --2- 172.21.15.121:0/2196583547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f529c0a5720 0x7f529c14d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:03.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a0d44700 1 --2- 172.21.15.121:0/2196583547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f529c0c3430 0x7f529c0c5900 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7f528c009e00 tx=0x7f528c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:03.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52917fa700 1 -- 172.21.15.121:0/2196583547 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c00f7e0 con 0x7f529c0c3430 2023-11-15T03:03:03.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52917fa700 1 -- 172.21.15.121:0/2196583547 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528c00f940 con 0x7f529c0c3430 2023-11-15T03:03:03.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a2547700 1 -- 172.21.15.121:0/2196583547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529c0c6600 con 0x7f529c0c3430 2023-11-15T03:03:03.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52917fa700 1 -- 172.21.15.121:0/2196583547 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c01b690 con 0x7f529c0c3430 2023-11-15T03:03:03.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.245+0000 7f52a2547700 1 -- 172.21.15.121:0/2196583547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c0c6bc0 con 0x7f529c0c3430 2023-11-15T03:03:03.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.249+0000 7f52917fa700 1 -- 172.21.15.121:0/2196583547 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f528c01ba80 con 0x7f529c0c3430 2023-11-15T03:03:03.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.249+0000 7f52917fa700 1 --2- 172.21.15.121:0/2196583547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f528005a540 0x7f528005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.249+0000 7f52917fa700 1 -- 172.21.15.121:0/2196583547 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f528c076190 con 0x7f529c0c3430 2023-11-15T03:03:03.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.249+0000 7f52a1545700 1 --2- 172.21.15.121:0/2196583547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f528005a540 0x7f528005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52adb64740 1 Processor -- start 2023-11-15T03:03:03.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52adb64740 1 -- start start 2023-11-15T03:03:03.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52a1545700 1 --2- 172.21.15.121:0/2196583547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f528005a540 0x7f528005ca00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f5298006bc0 tx=0x7f5298008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:03.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52adb64740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21f6930 0x21f6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52adb64740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b4bd0 0x21af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52adb64740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 0x21af640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52adb64740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2112540 con 0x21b5ac0 2023-11-15T03:03:03.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52adb64740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x20d5e90 con 0x21b4bd0 2023-11-15T03:03:03.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52adb64740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x20dbd90 con 0x21f6930 2023-11-15T03:03:03.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52a1d46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 0x21af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52a1d46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 0x21af640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33818/0 (socket says 172.21.15.121:33818) 2023-11-15T03:03:03.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52a1d46700 1 -- 172.21.15.121:0/1312754953 learned_addr learned my addr 172.21.15.121:0/1312754953 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:03.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52a1545700 1 --2- 172.21.15.121:0/1312754953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21f6930 0x21f6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52a0d44700 1 --2- 172.21.15.121:0/1312754953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b4bd0 0x21af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52a1d46700 1 -- 172.21.15.121:0/1312754953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21f6930 msgr2=0x21f6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:03.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52a1d46700 1 --2- 172.21.15.121:0/1312754953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21f6930 0x21f6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52a1d46700 1 -- 172.21.15.121:0/1312754953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b4bd0 msgr2=0x21af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:03.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52a1d46700 1 --2- 172.21.15.121:0/1312754953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b4bd0 0x21af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52a1d46700 1 -- 172.21.15.121:0/1312754953 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x21404c0 con 0x21b5ac0 2023-11-15T03:03:03.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52a1d46700 1 --2- 172.21.15.121:0/1312754953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 0x21af640 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f5294001bc0 tx=0x7f5294009040).ready entity=mon.0 client_cookie=67e5f10f32d40e27 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:03.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52867fc700 1 -- 172.21.15.121:0/1312754953 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529400e860 con 0x21b5ac0 2023-11-15T03:03:03.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52867fc700 1 -- 172.21.15.121:0/1312754953 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529400e9c0 con 0x21b5ac0 2023-11-15T03:03:03.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.253+0000 7f52867fc700 1 -- 172.21.15.121:0/1312754953 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529400ed40 con 0x21b5ac0 2023-11-15T03:03:03.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.257+0000 7f52adb64740 1 -- 172.21.15.121:0/1312754953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 msgr2=0x21af640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:03.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.257+0000 7f52adb64740 1 --2- 172.21.15.121:0/1312754953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 0x21af640 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f5294001bc0 tx=0x7f5294009040).stop 2023-11-15T03:03:03.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.257+0000 7f52adb64740 1 -- 172.21.15.121:0/1312754953 shutdown_connections 2023-11-15T03:03:03.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.257+0000 7f52adb64740 1 --2- 172.21.15.121:0/1312754953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 0x21af640 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.257+0000 7f52adb64740 1 --2- 172.21.15.121:0/1312754953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b4bd0 0x21af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.257+0000 7f52adb64740 1 --2- 172.21.15.121:0/1312754953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21f6930 0x21f6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.257+0000 7f52adb64740 1 -- 172.21.15.121:0/1312754953 >> 172.21.15.121:0/1312754953 conn(0x1f94680 msgr2=0x21fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:03.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.257+0000 7f52adb64740 1 -- 172.21.15.121:0/1312754953 shutdown_connections 2023-11-15T03:03:03.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.257+0000 7f52adb64740 1 -- 172.21.15.121:0/1312754953 wait complete. 2023-11-15T03:03:03.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.257+0000 7f52adb64740 1 -- start start 2023-11-15T03:03:03.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.257+0000 7f52adb64740 1 Processor -- start 2023-11-15T03:03:03.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52adb64740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b4bd0 0x21cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52adb64740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 0x21e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52adb64740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21f6930 0x21afef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52adb64740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x21dcf90 con 0x21b5ac0 2023-11-15T03:03:03.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52adb64740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x2208db0 con 0x21b4bd0 2023-11-15T03:03:03.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52adb64740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x21dbca0 con 0x21f6930 2023-11-15T03:03:03.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52adb64740 10 client.0 ll_register_callbacks cb 0x216c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:03:03.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52a0d44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 0x21e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52a0d44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 0x21e1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33828/0 (socket says 172.21.15.121:33828) 2023-11-15T03:03:03.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52a0d44700 1 -- 172.21.15.121:0/2008870979 learned_addr learned my addr 172.21.15.121:0/2008870979 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:03.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52a1545700 1 --2- 172.21.15.121:0/2008870979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b4bd0 0x21cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52a1d46700 1 --2- 172.21.15.121:0/2008870979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21f6930 0x21afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52a0d44700 1 -- 172.21.15.121:0/2008870979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21f6930 msgr2=0x21afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:03.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52a0d44700 1 --2- 172.21.15.121:0/2008870979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21f6930 0x21afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52a0d44700 1 -- 172.21.15.121:0/2008870979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b4bd0 msgr2=0x21cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:03.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52a0d44700 1 --2- 172.21.15.121:0/2008870979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b4bd0 0x21cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52a0d44700 1 -- 172.21.15.121:0/2008870979 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529400f040 con 0x21b5ac0 2023-11-15T03:03:03.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52a0d44700 1 --2- 172.21.15.121:0/2008870979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 0x21e1b80 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7f528c000e50 tx=0x7f528c00e000).ready entity=mon.0 client_cookie=b5f38393b4bb8c9c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:03.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f527b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:03:03.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f527b7fe700 1 -- 172.21.15.121:0/2008870979 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c019330 con 0x21b5ac0 2023-11-15T03:03:03.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f527b7fe700 1 -- 172.21.15.121:0/2008870979 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f528c019490 con 0x21b5ac0 2023-11-15T03:03:03.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52adb64740 1 -- 172.21.15.121:0/2008870979 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x212c230 con 0x21b5ac0 2023-11-15T03:03:03.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f527b7fe700 1 -- 172.21.15.121:0/2008870979 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f528c04c9d0 con 0x21b5ac0 2023-11-15T03:03:03.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52adb64740 10 client.6816 fetch_fsmap learned FSMap version 73 2023-11-15T03:03:03.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.261+0000 7f52adb64740 1 -- 172.21.15.121:0/2008870979 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x212f7c0 con 0x21b5ac0 2023-11-15T03:03:03.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f527b7fe700 1 -- 172.21.15.121:0/2008870979 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528c038020 con 0x21b5ac0 2023-11-15T03:03:03.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f527b7fe700 1 -- 172.21.15.121:0/2008870979 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 73) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f528c0492d0 con 0x21b5ac0 2023-11-15T03:03:03.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f52adb64740 10 client.6816 fetch_fsmap finished waiting for FSMap version 73 2023-11-15T03:03:03.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f52adb64740 10 client.6816 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2023-11-15T03:03:03.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f52adb64740 20 client.6816 populate_metadata read hostname 'smithi121' 2023-11-15T03:03:03.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f52adb64740 1 --2- 172.21.15.121:0/2008870979 >> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] conn(0x2207b50 0x2207f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f52adb64740 4 client.6816 mds_command: new command op to 6486 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:03:03.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f52adb64740 1 -- 172.21.15.121:0/2008870979 --> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2122780 con 0x2207b50 2023-11-15T03:03:03.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f52a1545700 1 --2- 172.21.15.121:0/2008870979 >> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] conn(0x2207b50 0x2207f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f52a1545700 1 --2- 172.21.15.121:0/2008870979 >> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] conn(0x2207b50 0x2207f30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=3beccdcaffc31e72 server_cookie=4fbbb49c99afa31f in_seq=0 out_seq=0 2023-11-15T03:03:03.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f527b7fe700 10 client.6816 ms_handle_connect on v2:172.21.15.121:6834/834528709 2023-11-15T03:03:03.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f527b7fe700 1 -- 172.21.15.121:0/2008870979 <== mds.0 v2:172.21.15.121:6834/834528709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x2122780 con 0x2207b50 2023-11-15T03:03:03.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f527b7fe700 10 client.6816 handle_command_reply: tid=0 2023-11-15T03:03:03.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f52adb64740 1 client.6816 shutdown 2023-11-15T03:03:03.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f52adb64740 1 -- 172.21.15.121:0/2008870979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 msgr2=0x21e1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:03.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f52adb64740 1 --2- 172.21.15.121:0/2008870979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 0x21e1b80 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7f528c000e50 tx=0x7f528c00e000).stop 2023-11-15T03:03:03.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f52adb64740 1 -- 172.21.15.121:0/2008870979 shutdown_connections 2023-11-15T03:03:03.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f52adb64740 1 --2- 172.21.15.121:0/2008870979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21f6930 0x21afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f52adb64740 1 --2- 172.21.15.121:0/2008870979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 0x21e1b80 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f52adb64740 1 --2- 172.21.15.121:0/2008870979 >> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] conn(0x2207b50 0x2207f30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f52adb64740 1 --2- 172.21.15.121:0/2008870979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b4bd0 0x21cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f52adb64740 1 -- 172.21.15.121:0/2008870979 >> 172.21.15.121:0/2008870979 conn(0x1f94680 msgr2=0x21f6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:03.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f527b7fe700 0 client.6816 ms_handle_reset on v2:172.21.15.121:6834/834528709 2023-11-15T03:03:03.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.265+0000 7f52adb64740 1 -- 172.21.15.121:0/2008870979 shutdown_connections 2023-11-15T03:03:03.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.269+0000 7f52adb64740 1 -- 172.21.15.121:0/2008870979 wait complete. 2023-11-15T03:03:03.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.269+0000 7f52adb64740 20 client.6816 trim_cache size 0 max 16384 2023-11-15T03:03:03.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.285+0000 7f52adb64740 1 Processor -- start 2023-11-15T03:03:03.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.285+0000 7f52adb64740 1 -- start start 2023-11-15T03:03:03.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.285+0000 7f52adb64740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b4bd0 0x21af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.285+0000 7f52adb64740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b5ac0 0x22071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.285+0000 7f52adb64740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21f6930 0x21cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.285+0000 7f52adb64740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2159ff0 con 0x21f6930 2023-11-15T03:03:03.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.285+0000 7f52adb64740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x21dd460 con 0x21b4bd0 2023-11-15T03:03:03.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.285+0000 7f52adb64740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2208510 con 0x21b5ac0 2023-11-15T03:03:03.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.285+0000 7f52a1d46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21f6930 0x21cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.285+0000 7f52a1d46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21f6930 0x21cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33842/0 (socket says 172.21.15.121:33842) 2023-11-15T03:03:03.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.285+0000 7f52a1d46700 1 -- 172.21.15.121:0/3917732133 learned_addr learned my addr 172.21.15.121:0/3917732133 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:03.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.285+0000 7f52a0d44700 1 --2- 172.21.15.121:0/3917732133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b5ac0 0x22071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.285+0000 7f52a1545700 1 --2- 172.21.15.121:0/3917732133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b4bd0 0x21af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.285+0000 7f52a1d46700 1 -- 172.21.15.121:0/3917732133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b5ac0 msgr2=0x22071b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:03.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.285+0000 7f52a1d46700 1 --2- 172.21.15.121:0/3917732133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b5ac0 0x22071b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.285+0000 7f52a1d46700 1 -- 172.21.15.121:0/3917732133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b4bd0 msgr2=0x21af8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:03.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.285+0000 7f52a1d46700 1 --2- 172.21.15.121:0/3917732133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b4bd0 0x21af8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.289+0000 7f52a1d46700 1 -- 172.21.15.121:0/3917732133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c0624f0 con 0x21f6930 2023-11-15T03:03:03.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.289+0000 7f52a0d44700 1 --2- 172.21.15.121:0/3917732133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b5ac0 0x22071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:03.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.289+0000 7f52a1d46700 1 --2- 172.21.15.121:0/3917732133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21f6930 0x21cdba0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x21db880 tx=0x7f5294009040).ready entity=mon.0 client_cookie=a21619c0299364f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:03.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.289+0000 7f5286ffd700 1 -- 172.21.15.121:0/3917732133 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5294000f40 con 0x21f6930 2023-11-15T03:03:03.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.289+0000 7f5286ffd700 1 -- 172.21.15.121:0/3917732133 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529400eb30 con 0x21f6930 2023-11-15T03:03:03.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.289+0000 7f5286ffd700 1 -- 172.21.15.121:0/3917732133 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529400eeb0 con 0x21f6930 2023-11-15T03:03:03.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.289+0000 7f52adb64740 1 -- 172.21.15.121:0/3917732133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21f6930 msgr2=0x21cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:03.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.289+0000 7f52adb64740 1 --2- 172.21.15.121:0/3917732133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21f6930 0x21cdba0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x21db880 tx=0x7f5294009040).stop 2023-11-15T03:03:03.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.289+0000 7f52adb64740 1 -- 172.21.15.121:0/3917732133 shutdown_connections 2023-11-15T03:03:03.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.289+0000 7f52adb64740 1 --2- 172.21.15.121:0/3917732133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21f6930 0x21cdba0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.289+0000 7f52adb64740 1 --2- 172.21.15.121:0/3917732133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b5ac0 0x22071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.289+0000 7f52adb64740 1 --2- 172.21.15.121:0/3917732133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b4bd0 0x21af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.289+0000 7f52adb64740 1 -- 172.21.15.121:0/3917732133 >> 172.21.15.121:0/3917732133 conn(0x1f94680 msgr2=0x21b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:03.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.289+0000 7f52adb64740 1 -- 172.21.15.121:0/3917732133 shutdown_connections 2023-11-15T03:03:03.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.289+0000 7f52adb64740 1 -- 172.21.15.121:0/3917732133 wait complete. 2023-11-15T03:03:03.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.289+0000 7f52adb64740 1 -- start start 2023-11-15T03:03:03.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f52adb64740 1 Processor -- start 2023-11-15T03:03:03.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f52adb64740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b4bd0 0x2255580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f52adb64740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 0x2254f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f52adb64740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21f6930 0x21afc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f52adb64740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2255410 con 0x21b5ac0 2023-11-15T03:03:03.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f52adb64740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x225f190 con 0x21f6930 2023-11-15T03:03:03.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f52adb64740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2211c40 con 0x21b4bd0 2023-11-15T03:03:03.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f52adb64740 10 client.0 ll_register_callbacks cb 0x216c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:03:03.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f52a0d44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 0x2254f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f52a0d44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 0x2254f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33846/0 (socket says 172.21.15.121:33846) 2023-11-15T03:03:03.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f52a0d44700 1 -- 172.21.15.121:0/3354446567 learned_addr learned my addr 172.21.15.121:0/3354446567 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:03.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f52a1545700 1 --2- 172.21.15.121:0/3354446567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b4bd0 0x2255580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f52a1d46700 1 --2- 172.21.15.121:0/3354446567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21f6930 0x21afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f52a0d44700 1 -- 172.21.15.121:0/3354446567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b4bd0 msgr2=0x2255580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:03.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f52a0d44700 1 --2- 172.21.15.121:0/3354446567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b4bd0 0x2255580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f52a0d44700 1 -- 172.21.15.121:0/3354446567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21f6930 msgr2=0x21afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:03.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f52a0d44700 1 --2- 172.21.15.121:0/3354446567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21f6930 0x21afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f52a0d44700 1 -- 172.21.15.121:0/3354446567 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x21404c0 con 0x21b5ac0 2023-11-15T03:03:03.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f52a0d44700 1 --2- 172.21.15.121:0/3354446567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 0x2254f20 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f528c006a10 tx=0x7f528c030000).ready entity=mon.0 client_cookie=e1b541eb7f244a3b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:03.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.293+0000 7f527b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:03:03.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.297+0000 7f527b7fe700 1 -- 172.21.15.121:0/3354446567 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c00f510 con 0x21b5ac0 2023-11-15T03:03:03.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.297+0000 7f527b7fe700 1 -- 172.21.15.121:0/3354446567 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f528c039080 con 0x21b5ac0 2023-11-15T03:03:03.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.297+0000 7f52adb64740 1 -- 172.21.15.121:0/3354446567 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f529400e9c0 con 0x21b5ac0 2023-11-15T03:03:03.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.297+0000 7f527b7fe700 1 -- 172.21.15.121:0/3354446567 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f528c04d480 con 0x21b5ac0 2023-11-15T03:03:03.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.297+0000 7f52adb64740 10 client.6822 fetch_fsmap learned FSMap version 73 2023-11-15T03:03:03.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.297+0000 7f52adb64740 1 -- 172.21.15.121:0/3354446567 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x212a9f0 con 0x21b5ac0 2023-11-15T03:03:03.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.297+0000 7f527b7fe700 1 -- 172.21.15.121:0/3354446567 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529400e9c0 con 0x21b5ac0 2023-11-15T03:03:03.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.297+0000 7f527b7fe700 1 -- 172.21.15.121:0/3354446567 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 73) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f528c04c9d0 con 0x21b5ac0 2023-11-15T03:03:03.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.297+0000 7f52adb64740 10 client.6822 fetch_fsmap finished waiting for FSMap version 73 2023-11-15T03:03:03.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.297+0000 7f52adb64740 10 client.6822 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2023-11-15T03:03:03.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.297+0000 7f52adb64740 20 client.6822 populate_metadata read hostname 'smithi121' 2023-11-15T03:03:03.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.297+0000 7f52adb64740 1 --2- 172.21.15.121:0/3354446567 >> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] conn(0x225e550 0x225e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.297+0000 7f52adb64740 4 client.6822 mds_command: new command op to 6486 tid=0[{"prefix": "flush journal"}] 2023-11-15T03:03:03.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.297+0000 7f52adb64740 1 -- 172.21.15.121:0/3354446567 --> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x1feb610 con 0x225e550 2023-11-15T03:03:03.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.297+0000 7f52a1545700 1 --2- 172.21.15.121:0/3354446567 >> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] conn(0x225e550 0x225e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.297+0000 7f527b7fe700 10 client.6822 ms_handle_connect on v2:172.21.15.121:6834/834528709 2023-11-15T03:03:03.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.297+0000 7f52a1545700 1 --2- 172.21.15.121:0/3354446567 >> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] conn(0x225e550 0x225e930 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=e82781e452e6f82c server_cookie=3d0349d7d1bda252 in_seq=0 out_seq=0 2023-11-15T03:03:03.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.313+0000 7f527b7fe700 1 -- 172.21.15.121:0/3354446567 <== mds.0 v2:172.21.15.121:6834/834528709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x1feb610 con 0x225e550 2023-11-15T03:03:03.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.313+0000 7f527b7fe700 10 client.6822 handle_command_reply: tid=0 2023-11-15T03:03:03.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.313+0000 7f52adb64740 1 client.6822 shutdown 2023-11-15T03:03:03.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52adb64740 1 -- 172.21.15.121:0/3354446567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 msgr2=0x2254f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:03.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52adb64740 1 --2- 172.21.15.121:0/3354446567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 0x2254f20 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f528c006a10 tx=0x7f528c030000).stop 2023-11-15T03:03:03.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52adb64740 1 -- 172.21.15.121:0/3354446567 shutdown_connections 2023-11-15T03:03:03.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52adb64740 1 --2- 172.21.15.121:0/3354446567 >> [v2:172.21.15.121:6834/834528709,v1:172.21.15.121:6836/834528709] conn(0x225e550 0x225e930 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52adb64740 1 --2- 172.21.15.121:0/3354446567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21f6930 0x21afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52adb64740 1 --2- 172.21.15.121:0/3354446567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b5ac0 0x2254f20 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f527b7fe700 0 client.6822 ms_handle_reset on v2:172.21.15.121:6834/834528709 2023-11-15T03:03:03.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52adb64740 1 --2- 172.21.15.121:0/3354446567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b4bd0 0x2255580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52adb64740 1 -- 172.21.15.121:0/3354446567 >> 172.21.15.121:0/3354446567 conn(0x1f94680 msgr2=0x21ddf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:03.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52adb64740 1 -- 172.21.15.121:0/3354446567 shutdown_connections 2023-11-15T03:03:03.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52adb64740 1 -- 172.21.15.121:0/3354446567 wait complete. 2023-11-15T03:03:03.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52adb64740 20 client.6822 trim_cache size 0 max 16384 2023-11-15T03:03:03.333 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:03:03.334 INFO:teuthology.orchestra.run.smithi121.stdout: "message": "", 2023-11-15T03:03:03.334 INFO:teuthology.orchestra.run.smithi121.stdout: "return_code": 0 2023-11-15T03:03:03.334 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:03:03.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52a2547700 1 -- 172.21.15.121:0/2196583547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f528005a540 msgr2=0x7f528005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:03.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52a2547700 1 --2- 172.21.15.121:0/2196583547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f528005a540 0x7f528005ca00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f5298006bc0 tx=0x7f5298008040).stop 2023-11-15T03:03:03.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52a2547700 1 -- 172.21.15.121:0/2196583547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f529c0c3430 msgr2=0x7f529c0c5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:03.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52a2547700 1 --2- 172.21.15.121:0/2196583547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f529c0c3430 0x7f529c0c5900 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7f528c009e00 tx=0x7f528c017000).stop 2023-11-15T03:03:03.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52a2547700 1 -- 172.21.15.121:0/2196583547 shutdown_connections 2023-11-15T03:03:03.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52a2547700 1 --2- 172.21.15.121:0/2196583547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f528005a540 0x7f528005ca00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52a2547700 1 --2- 172.21.15.121:0/2196583547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c0c5df0 0x7f529c0a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52a2547700 1 --2- 172.21.15.121:0/2196583547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f529c0c3430 0x7f529c0c5900 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52a2547700 1 --2- 172.21.15.121:0/2196583547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f529c0a5720 0x7f529c14d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52a2547700 1 -- 172.21.15.121:0/2196583547 >> 172.21.15.121:0/2196583547 conn(0x7f529c067a60 msgr2=0x7f529c14db30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:03.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52a2547700 1 -- 172.21.15.121:0/2196583547 shutdown_connections 2023-11-15T03:03:03.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.317+0000 7f52a2547700 1 -- 172.21.15.121:0/2196583547 wait complete. 2023-11-15T03:03:03.345 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-15T03:03:03.345 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T03:03:03.388 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T03:03:03.437 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-15T03:03:03.437 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T03:03:03.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:03:03.438 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-15T03:03:03.483 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:03:03.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:03.484 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:03:03.551 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:03:03.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.749+0000 7fa49545b700 1 Processor -- start 2023-11-15T03:03:03.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.749+0000 7fa49545b700 1 -- start start 2023-11-15T03:03:03.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.749+0000 7fa49545b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa4900c3ed0 0x7fa4900c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.749+0000 7fa49545b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa4901546b0 0x7fa490154b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.749+0000 7fa49545b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa490068880 con 0x7fa490064980 2023-11-15T03:03:03.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.749+0000 7fa49545b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa490155010 con 0x7fa4900c3ed0 2023-11-15T03:03:03.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.749+0000 7fa49545b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4900bd270 con 0x7fa4901546b0 2023-11-15T03:03:03.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.749+0000 7fa48effd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa4900c3ed0 0x7fa4900c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.749+0000 7fa487fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa490064980 0x7fa4900bd590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47108/0 (socket says 172.21.15.121:47108) 2023-11-15T03:03:03.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.749+0000 7fa487fff700 1 -- 172.21.15.121:0/2919559752 learned_addr learned my addr 172.21.15.121:0/2919559752 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:03.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.749+0000 7fa48ffff700 1 --2- 172.21.15.121:0/2919559752 >> v2:172.21.15.190:3300/0 conn(0x7fa4901546b0 0x7fa490154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.753+0000 7fa48effd700 1 -- 172.21.15.121:0/2919559752 >> v2:172.21.15.190:3300/0 conn(0x7fa4901546b0 msgr2=0x7fa490154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:03.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.753+0000 7fa48effd700 1 --2- 172.21.15.121:0/2919559752 >> v2:172.21.15.190:3300/0 conn(0x7fa4901546b0 0x7fa490154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.753+0000 7fa48effd700 1 -- 172.21.15.121:0/2919559752 >> v1:172.21.15.121:6789/0 conn(0x7fa490064980 legacy=0x7fa4900bd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:03.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.753+0000 7fa48e7fc700 1 -- 172.21.15.121:0/2919559752 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2013429100 0 0) 0x7fa490068880 con 0x7fa490064980 2023-11-15T03:03:03.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.753+0000 7fa48effd700 1 -- 172.21.15.121:0/2919559752 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4900bccb0 con 0x7fa4900c3ed0 2023-11-15T03:03:03.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.753+0000 7fa48effd700 1 --2- 172.21.15.121:0/2919559752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4900c3ed0 0x7fa4900c42b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7fa490059740 tx=0x7fa480009060).ready entity=mon.0 client_cookie=91564346a75ee680 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:03.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.753+0000 7fa48e7fc700 1 -- 172.21.15.121:0/2919559752 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48000f510 con 0x7fa4900c3ed0 2023-11-15T03:03:03.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.753+0000 7fa48e7fc700 1 -- 172.21.15.121:0/2919559752 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48000f670 con 0x7fa4900c3ed0 2023-11-15T03:03:03.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.753+0000 7fa48e7fc700 1 -- 172.21.15.121:0/2919559752 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48000f9f0 con 0x7fa4900c3ed0 2023-11-15T03:03:03.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.753+0000 7fa49545b700 1 -- 172.21.15.121:0/2919559752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4900c3ed0 msgr2=0x7fa4900c42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:03.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.753+0000 7fa49545b700 1 --2- 172.21.15.121:0/2919559752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4900c3ed0 0x7fa4900c42b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7fa490059740 tx=0x7fa480009060).stop 2023-11-15T03:03:03.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.753+0000 7fa49545b700 1 -- 172.21.15.121:0/2919559752 shutdown_connections 2023-11-15T03:03:03.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.753+0000 7fa49545b700 1 --2- 172.21.15.121:0/2919559752 >> v2:172.21.15.190:3300/0 conn(0x7fa4901546b0 0x7fa490154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.753+0000 7fa49545b700 1 --2- 172.21.15.121:0/2919559752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4900c3ed0 0x7fa4900c42b0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.753+0000 7fa49545b700 1 -- 172.21.15.121:0/2919559752 >> 172.21.15.121:0/2919559752 conn(0x7fa490069ac0 msgr2=0x7fa490090a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:03.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.753+0000 7fa49545b700 1 -- 172.21.15.121:0/2919559752 shutdown_connections 2023-11-15T03:03:03.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.753+0000 7fa49545b700 1 -- 172.21.15.121:0/2919559752 wait complete. 2023-11-15T03:03:03.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.757+0000 7fa49545b700 1 Processor -- start 2023-11-15T03:03:03.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.757+0000 7fa49545b700 1 -- start start 2023-11-15T03:03:03.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.757+0000 7fa49545b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa490064980 0x7fa4900c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.757+0000 7fa49545b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4900c3ed0 0x7fa4900c4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.757+0000 7fa49545b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4901546b0 0x7fa490164170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.757+0000 7fa49545b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa490167ad0 con 0x7fa4900c3ed0 2023-11-15T03:03:03.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.757+0000 7fa49545b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4901650f0 con 0x7fa490064980 2023-11-15T03:03:03.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.757+0000 7fa49545b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4900c7f10 con 0x7fa4901546b0 2023-11-15T03:03:03.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.757+0000 7fa487fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4900c3ed0 0x7fa4900c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.757+0000 7fa487fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4900c3ed0 0x7fa4900c4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33860/0 (socket says 172.21.15.121:33860) 2023-11-15T03:03:03.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.757+0000 7fa487fff700 1 -- 172.21.15.121:0/3363819817 learned_addr learned my addr 172.21.15.121:0/3363819817 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:03.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.757+0000 7fa48effd700 1 --2- 172.21.15.121:0/3363819817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa490064980 0x7fa4900c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.757+0000 7fa48ffff700 1 --2- 172.21.15.121:0/3363819817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4901546b0 0x7fa490164170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.761+0000 7fa487fff700 1 -- 172.21.15.121:0/3363819817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4901546b0 msgr2=0x7fa490164170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:03.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.761+0000 7fa487fff700 1 --2- 172.21.15.121:0/3363819817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4901546b0 0x7fa490164170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.761+0000 7fa487fff700 1 -- 172.21.15.121:0/3363819817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa490064980 msgr2=0x7fa4900c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:03.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.761+0000 7fa487fff700 1 --2- 172.21.15.121:0/3363819817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa490064980 0x7fa4900c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.761+0000 7fa487fff700 1 -- 172.21.15.121:0/3363819817 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4800092a0 con 0x7fa4900c3ed0 2023-11-15T03:03:03.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.761+0000 7fa487fff700 1 --2- 172.21.15.121:0/3363819817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4900c3ed0 0x7fa4900c4990 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7fa47800c490 tx=0x7fa478007e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:03.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.761+0000 7fa486ffd700 1 -- 172.21.15.121:0/3363819817 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47800ce90 con 0x7fa4900c3ed0 2023-11-15T03:03:03.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.761+0000 7fa486ffd700 1 -- 172.21.15.121:0/3363819817 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4780072e0 con 0x7fa4900c3ed0 2023-11-15T03:03:03.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.761+0000 7fa486ffd700 1 -- 172.21.15.121:0/3363819817 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4780057a0 con 0x7fa4900c3ed0 2023-11-15T03:03:03.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.761+0000 7fa49545b700 1 -- 172.21.15.121:0/3363819817 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa490164e10 con 0x7fa4900c3ed0 2023-11-15T03:03:03.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.761+0000 7fa49545b700 1 -- 172.21.15.121:0/3363819817 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4900c7ba0 con 0x7fa4900c3ed0 2023-11-15T03:03:03.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.761+0000 7fa49545b700 1 -- 172.21.15.121:0/3363819817 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa490047060 con 0x7fa4900c3ed0 2023-11-15T03:03:03.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.765+0000 7fa486ffd700 1 -- 172.21.15.121:0/3363819817 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa478005bd0 con 0x7fa4900c3ed0 2023-11-15T03:03:03.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.765+0000 7fa486ffd700 1 --2- 172.21.15.121:0/3363819817 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa47005a510 0x7fa47005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:03.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.765+0000 7fa486ffd700 1 -- 172.21.15.121:0/3363819817 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fa478073ec0 con 0x7fa4900c3ed0 2023-11-15T03:03:03.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.765+0000 7fa48effd700 1 --2- 172.21.15.121:0/3363819817 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa47005a510 0x7fa47005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:03.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.765+0000 7fa48effd700 1 --2- 172.21.15.121:0/3363819817 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa47005a510 0x7fa47005c9d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7fa490059740 tx=0x7fa480017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:03.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.773+0000 7fa486ffd700 1 -- 172.21.15.121:0/3363819817 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa478047610 con 0x7fa4900c3ed0 2023-11-15T03:03:03.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.973+0000 7fa49545b700 1 -- 172.21.15.121:0/3363819817 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa490091230 con 0x7fa4900c3ed0 2023-11-15T03:03:03.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.973+0000 7fa486ffd700 1 -- 172.21.15.121:0/3363819817 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4502 (secure 0 0 0) 0x7fa47804ac30 con 0x7fa4900c3ed0 2023-11-15T03:03:03.983 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:03.984 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":73,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1508681155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1508681155},{"type":"v1","addr":"172.21.15.121:6837","nonce":1508681155}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6385,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3559276534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3559276534},{"type":"v1","addr":"172.21.15.190:6835","nonce":3559276534}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1842535107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1842535107},{"type":"v1","addr":"172.21.15.190:6833","nonce":1842535107}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:02:20.703078+0000","modified":"2023-11-15T03:03:01.009934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":14,"addr":"172.21.15.121:6836/834528709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":834528709},{"type":"v1","addr":"172.21.15.121:6836","nonce":834528709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-15T03:03:03.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.977+0000 7fa49545b700 1 -- 172.21.15.121:0/3363819817 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa47005a510 msgr2=0x7fa47005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:03.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.977+0000 7fa49545b700 1 --2- 172.21.15.121:0/3363819817 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa47005a510 0x7fa47005c9d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7fa490059740 tx=0x7fa480017000).stop 2023-11-15T03:03:03.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.977+0000 7fa49545b700 1 -- 172.21.15.121:0/3363819817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4900c3ed0 msgr2=0x7fa4900c4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:03.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.977+0000 7fa49545b700 1 --2- 172.21.15.121:0/3363819817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4900c3ed0 0x7fa4900c4990 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7fa47800c490 tx=0x7fa478007e60).stop 2023-11-15T03:03:03.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.977+0000 7fa49545b700 1 -- 172.21.15.121:0/3363819817 shutdown_connections 2023-11-15T03:03:03.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.977+0000 7fa49545b700 1 --2- 172.21.15.121:0/3363819817 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa47005a510 0x7fa47005c9d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.977+0000 7fa49545b700 1 --2- 172.21.15.121:0/3363819817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4901546b0 0x7fa490164170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.977+0000 7fa49545b700 1 --2- 172.21.15.121:0/3363819817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4900c3ed0 0x7fa4900c4990 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.977+0000 7fa49545b700 1 --2- 172.21.15.121:0/3363819817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa490064980 0x7fa4900c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:03.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.977+0000 7fa49545b700 1 -- 172.21.15.121:0/3363819817 >> 172.21.15.121:0/3363819817 conn(0x7fa490069ac0 msgr2=0x7fa4900a4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:03.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.977+0000 7fa49545b700 1 -- 172.21.15.121:0/3363819817 shutdown_connections 2023-11-15T03:03:03.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:03.977+0000 7fa49545b700 1 -- 172.21.15.121:0/3363819817 wait complete. 2023-11-15T03:03:03.988 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 73 2023-11-15T03:03:04.008 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:03:04.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.153+0000 7fd7c8a79700 1 Processor -- start 2023-11-15T03:03:04.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.153+0000 7fd7c8a79700 1 -- start start 2023-11-15T03:03:04.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.153+0000 7fd7c8a79700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd7c40c2010 0x7fd7c40c23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:04.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.153+0000 7fd7c8a79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7c4154750 0x7fd7c40c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:04.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.153+0000 7fd7c8a79700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c4154b90 con 0x7fd7c40c1240 2023-11-15T03:03:04.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.153+0000 7fd7c8a79700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c4153c90 con 0x7fd7c40c2010 2023-11-15T03:03:04.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.153+0000 7fd7c8a79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c40c1810 con 0x7fd7c4154750 2023-11-15T03:03:04.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c2ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7c4154750 0x7fd7c40c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:04.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c37fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd7c40c2010 0x7fd7c40c23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:04.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c3fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd7c40c1240 0x7fd7c4153030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:35004/0 (socket says 172.21.15.121:35004) 2023-11-15T03:03:04.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c3fff700 1 -- 172.21.15.121:0/2528862364 learned_addr learned my addr 172.21.15.121:0/2528862364 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:04.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c2ffd700 1 -- 172.21.15.121:0/2528862364 >> v2:172.21.15.190:3300/0 conn(0x7fd7c40c2010 msgr2=0x7fd7c40c23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:04.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c2ffd700 1 --2- 172.21.15.121:0/2528862364 >> v2:172.21.15.190:3300/0 conn(0x7fd7c40c2010 0x7fd7c40c23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c2ffd700 1 -- 172.21.15.121:0/2528862364 >> v1:172.21.15.190:6789/0 conn(0x7fd7c40c1240 legacy=0x7fd7c4153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:04.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c2ffd700 1 -- 172.21.15.121:0/2528862364 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c40c1af0 con 0x7fd7c4154750 2023-11-15T03:03:04.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c2ffd700 1 --2- 172.21.15.121:0/2528862364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7c4154750 0x7fd7c40c0d50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fd7b8009760 tx=0x7fd7b8009060).ready entity=mon.2 client_cookie=71795de1e82185f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:04.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c27fc700 1 -- 172.21.15.121:0/2528862364 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b800f510 con 0x7fd7c4154750 2023-11-15T03:03:04.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c27fc700 1 -- 172.21.15.121:0/2528862364 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b800f670 con 0x7fd7c4154750 2023-11-15T03:03:04.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c27fc700 1 -- 172.21.15.121:0/2528862364 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b800f9f0 con 0x7fd7c4154750 2023-11-15T03:03:04.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c8a79700 1 -- 172.21.15.121:0/2528862364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7c4154750 msgr2=0x7fd7c40c0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:04.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c8a79700 1 --2- 172.21.15.121:0/2528862364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7c4154750 0x7fd7c40c0d50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fd7b8009760 tx=0x7fd7b8009060).stop 2023-11-15T03:03:04.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c8a79700 1 -- 172.21.15.121:0/2528862364 shutdown_connections 2023-11-15T03:03:04.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c8a79700 1 --2- 172.21.15.121:0/2528862364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7c4154750 0x7fd7c40c0d50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c8a79700 1 --2- 172.21.15.121:0/2528862364 >> v2:172.21.15.190:3300/0 conn(0x7fd7c40c2010 0x7fd7c40c23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c8a79700 1 -- 172.21.15.121:0/2528862364 >> 172.21.15.121:0/2528862364 conn(0x7fd7c4062190 msgr2=0x7fd7c4062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:04.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c8a79700 1 -- 172.21.15.121:0/2528862364 shutdown_connections 2023-11-15T03:03:04.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.157+0000 7fd7c8a79700 1 -- 172.21.15.121:0/2528862364 wait complete. 2023-11-15T03:03:04.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.161+0000 7fd7c8a79700 1 Processor -- start 2023-11-15T03:03:04.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.161+0000 7fd7c8a79700 1 -- start start 2023-11-15T03:03:04.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.165+0000 7fd7c8a79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7c40c1240 0x7fd7c413a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:04.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.165+0000 7fd7c8a79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7c40c2010 0x7fd7c414b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:04.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.165+0000 7fd7c8a79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7c4154750 0x7fd7c4150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:04.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.165+0000 7fd7c8a79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c414ad70 con 0x7fd7c40c1240 2023-11-15T03:03:04.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.165+0000 7fd7c8a79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c414ba30 con 0x7fd7c40c2010 2023-11-15T03:03:04.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.165+0000 7fd7c8a79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c4165810 con 0x7fd7c4154750 2023-11-15T03:03:04.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.165+0000 7fd7c37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7c40c1240 0x7fd7c413a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:04.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.165+0000 7fd7c37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7c40c1240 0x7fd7c413a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33876/0 (socket says 172.21.15.121:33876) 2023-11-15T03:03:04.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.165+0000 7fd7c37fe700 1 -- 172.21.15.121:0/3776648255 learned_addr learned my addr 172.21.15.121:0/3776648255 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:04.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.165+0000 7fd7c3fff700 1 --2- 172.21.15.121:0/3776648255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7c4154750 0x7fd7c4150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:04.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.165+0000 7fd7c2ffd700 1 --2- 172.21.15.121:0/3776648255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7c40c2010 0x7fd7c414b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:04.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.165+0000 7fd7c37fe700 1 -- 172.21.15.121:0/3776648255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7c4154750 msgr2=0x7fd7c4150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:04.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.165+0000 7fd7c37fe700 1 --2- 172.21.15.121:0/3776648255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7c4154750 0x7fd7c4150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.165+0000 7fd7c37fe700 1 -- 172.21.15.121:0/3776648255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7c40c2010 msgr2=0x7fd7c414b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:04.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.165+0000 7fd7c37fe700 1 --2- 172.21.15.121:0/3776648255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7c40c2010 0x7fd7c414b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.165+0000 7fd7c37fe700 1 -- 172.21.15.121:0/3776648255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b80092a0 con 0x7fd7c40c1240 2023-11-15T03:03:04.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.169+0000 7fd7c37fe700 1 --2- 172.21.15.121:0/3776648255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7c40c1240 0x7fd7c413a270 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7fd7b400e9a0 tx=0x7fd7b400a200).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:04.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.169+0000 7fd7ab7fe700 1 -- 172.21.15.121:0/3776648255 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b400a610 con 0x7fd7c40c1240 2023-11-15T03:03:04.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.169+0000 7fd7ab7fe700 1 -- 172.21.15.121:0/3776648255 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b4010630 con 0x7fd7c40c1240 2023-11-15T03:03:04.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.169+0000 7fd7ab7fe700 1 -- 172.21.15.121:0/3776648255 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b40109b0 con 0x7fd7c40c1240 2023-11-15T03:03:04.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.169+0000 7fd7c8a79700 1 -- 172.21.15.121:0/3776648255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c414a160 con 0x7fd7c40c1240 2023-11-15T03:03:04.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.169+0000 7fd7c8a79700 1 -- 172.21.15.121:0/3776648255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c414a720 con 0x7fd7c40c1240 2023-11-15T03:03:04.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.173+0000 7fd7c8a79700 1 -- 172.21.15.121:0/3776648255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c406bcf0 con 0x7fd7c40c1240 2023-11-15T03:03:04.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.173+0000 7fd7ab7fe700 1 -- 172.21.15.121:0/3776648255 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd7b401a400 con 0x7fd7c40c1240 2023-11-15T03:03:04.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.173+0000 7fd7ab7fe700 1 --2- 172.21.15.121:0/3776648255 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd7b005a510 0x7fd7b005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:04.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.173+0000 7fd7ab7fe700 1 -- 172.21.15.121:0/3776648255 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fd7b4074b20 con 0x7fd7c40c1240 2023-11-15T03:03:04.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.173+0000 7fd7c2ffd700 1 --2- 172.21.15.121:0/3776648255 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd7b005a510 0x7fd7b005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:04.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.177+0000 7fd7c2ffd700 1 --2- 172.21.15.121:0/3776648255 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd7b005a510 0x7fd7b005c9d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fd7c414d120 tx=0x7fd7b8017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:04.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.181+0000 7fd7ab7fe700 1 -- 172.21.15.121:0/3776648255 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd7b4048150 con 0x7fd7c40c1240 2023-11-15T03:03:04.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.341+0000 7fd7c8a79700 1 -- 172.21.15.121:0/3776648255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7c4062570 con 0x7fd7c40c1240 2023-11-15T03:03:04.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.345+0000 7fd7ab7fe700 1 -- 172.21.15.121:0/3776648255 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17206 (secure 0 0 0) 0x7fd7b4015070 con 0x7fd7c40c1240 2023-11-15T03:03:04.353 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:04.354 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":80,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:02:57.977723+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-15T03:02:18.737496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-15T03:02:19.718114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.38","mappings":[{"from":6,"to":0}]},{"pgid":"11.3a","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:6833/914796342":"2023-11-16T03:02:57.977589+0000","172.21.15.190:6832/914796342":"2023-11-16T03:02:57.977589+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:03:04.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.345+0000 7fd7c8a79700 1 -- 172.21.15.121:0/3776648255 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd7b005a510 msgr2=0x7fd7b005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:04.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.345+0000 7fd7c8a79700 1 --2- 172.21.15.121:0/3776648255 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd7b005a510 0x7fd7b005c9d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fd7c414d120 tx=0x7fd7b8017000).stop 2023-11-15T03:03:04.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.345+0000 7fd7c8a79700 1 -- 172.21.15.121:0/3776648255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7c40c1240 msgr2=0x7fd7c413a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:04.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.345+0000 7fd7c8a79700 1 --2- 172.21.15.121:0/3776648255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7c40c1240 0x7fd7c413a270 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7fd7b400e9a0 tx=0x7fd7b400a200).stop 2023-11-15T03:03:04.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.345+0000 7fd7c8a79700 1 -- 172.21.15.121:0/3776648255 shutdown_connections 2023-11-15T03:03:04.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.345+0000 7fd7c8a79700 1 --2- 172.21.15.121:0/3776648255 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd7b005a510 0x7fd7b005c9d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.345+0000 7fd7c8a79700 1 --2- 172.21.15.121:0/3776648255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7c4154750 0x7fd7c4150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.345+0000 7fd7c8a79700 1 --2- 172.21.15.121:0/3776648255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7c40c2010 0x7fd7c414b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.345+0000 7fd7c8a79700 1 --2- 172.21.15.121:0/3776648255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7c40c1240 0x7fd7c413a270 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.345+0000 7fd7c8a79700 1 -- 172.21.15.121:0/3776648255 >> 172.21.15.121:0/3776648255 conn(0x7fd7c4062190 msgr2=0x7fd7c40bfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:04.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.345+0000 7fd7c8a79700 1 -- 172.21.15.121:0/3776648255 shutdown_connections 2023-11-15T03:03:04.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.345+0000 7fd7c8a79700 1 -- 172.21.15.121:0/3776648255 wait complete. 2023-11-15T03:03:04.376 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-15T03:02:18.737496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-11-15T03:02:19.718114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-15T03:03:04.377 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T03:03:04.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.537+0000 7fdc7ffff700 1 Processor -- start 2023-11-15T03:03:04.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.537+0000 7fdc7ffff700 1 -- start start 2023-11-15T03:03:04.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.541+0000 7fdc7ffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc8014e380 0x7fdc800c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:04.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.541+0000 7fdc7ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdc800ad270 0x7fdc80065680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:04.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.541+0000 7fdc7ffff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc800c7e90 con 0x7fdc800c6040 2023-11-15T03:03:04.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.541+0000 7fdc7ffff700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc800c8320 con 0x7fdc800ad270 2023-11-15T03:03:04.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.541+0000 7fdc7ffff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc800c7080 con 0x7fdc8014e380 2023-11-15T03:03:04.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.541+0000 7fdc7f7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdc800ad270 0x7fdc80065680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:04.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.541+0000 7fdc7f7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdc800ad270 0x7fdc80065680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33884/0 (socket says 172.21.15.121:33884) 2023-11-15T03:03:04.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.541+0000 7fdc7f7fe700 1 -- 172.21.15.121:0/2990907739 learned_addr learned my addr 172.21.15.121:0/2990907739 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:04.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.541+0000 7fdc7effd700 1 --2- 172.21.15.121:0/2990907739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc8014e380 0x7fdc800c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:04.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.541+0000 7fdc7f7fe700 1 -- 172.21.15.121:0/2990907739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc8014e380 msgr2=0x7fdc800c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:04.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.541+0000 7fdc7f7fe700 1 --2- 172.21.15.121:0/2990907739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc8014e380 0x7fdc800c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.541+0000 7fdc7f7fe700 1 -- 172.21.15.121:0/2990907739 >> v1:172.21.15.190:6789/0 conn(0x7fdc800c6040 legacy=0x7fdc800acb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:04.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.541+0000 7fdc7f7fe700 1 -- 172.21.15.121:0/2990907739 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8014e760 con 0x7fdc800ad270 2023-11-15T03:03:04.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.541+0000 7fdc7f7fe700 1 --2- 172.21.15.121:0/2990907739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc800ad270 0x7fdc80065680 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7fdc78009f20 tx=0x7fdc78009840).ready entity=mon.0 client_cookie=d6fcdfcb8ba14056 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:04.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.541+0000 7fdc7dffb700 1 -- 172.21.15.121:0/2990907739 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7800e070 con 0x7fdc800ad270 2023-11-15T03:03:04.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.541+0000 7fdc7dffb700 1 -- 172.21.15.121:0/2990907739 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7800b9d0 con 0x7fdc800ad270 2023-11-15T03:03:04.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.541+0000 7fdc7dffb700 1 -- 172.21.15.121:0/2990907739 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7800bd50 con 0x7fdc800ad270 2023-11-15T03:03:04.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.545+0000 7fdc7ffff700 1 -- 172.21.15.121:0/2990907739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc800ad270 msgr2=0x7fdc80065680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:04.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.545+0000 7fdc7ffff700 1 --2- 172.21.15.121:0/2990907739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc800ad270 0x7fdc80065680 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7fdc78009f20 tx=0x7fdc78009840).stop 2023-11-15T03:03:04.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.545+0000 7fdc7ffff700 1 -- 172.21.15.121:0/2990907739 shutdown_connections 2023-11-15T03:03:04.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.545+0000 7fdc7ffff700 1 --2- 172.21.15.121:0/2990907739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc800ad270 0x7fdc80065680 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.545+0000 7fdc7ffff700 1 --2- 172.21.15.121:0/2990907739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc8014e380 0x7fdc800c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.545+0000 7fdc7ffff700 1 -- 172.21.15.121:0/2990907739 >> 172.21.15.121:0/2990907739 conn(0x7fdc80067a60 msgr2=0x7fdc80067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:04.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.545+0000 7fdc7ffff700 1 -- 172.21.15.121:0/2990907739 shutdown_connections 2023-11-15T03:03:04.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.545+0000 7fdc7ffff700 1 -- 172.21.15.121:0/2990907739 wait complete. 2023-11-15T03:03:04.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.545+0000 7fdc7ffff700 1 Processor -- start 2023-11-15T03:03:04.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.545+0000 7fdc7ffff700 1 -- start start 2023-11-15T03:03:04.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.545+0000 7fdc7ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc800ad270 0x7fdc80149490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:04.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.545+0000 7fdc7ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc800c6040 0x7fdc80156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:04.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.545+0000 7fdc7ffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc8014e380 0x7fdc8014ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:04.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.545+0000 7fdc7ffff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc800c19e0 con 0x7fdc800c6040 2023-11-15T03:03:04.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.545+0000 7fdc7ffff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc800c2560 con 0x7fdc800ad270 2023-11-15T03:03:04.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.545+0000 7fdc7ffff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc800c1c30 con 0x7fdc8014e380 2023-11-15T03:03:04.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.549+0000 7fdc7e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc800c6040 0x7fdc80156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:04.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.549+0000 7fdc7e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc800c6040 0x7fdc80156000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33896/0 (socket says 172.21.15.121:33896) 2023-11-15T03:03:04.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.549+0000 7fdc7e7fc700 1 -- 172.21.15.121:0/1059731842 learned_addr learned my addr 172.21.15.121:0/1059731842 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:04.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.549+0000 7fdc7effd700 1 --2- 172.21.15.121:0/1059731842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc800ad270 0x7fdc80149490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:04.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.549+0000 7fdc7f7fe700 1 --2- 172.21.15.121:0/1059731842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc8014e380 0x7fdc8014ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:04.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.549+0000 7fdc7e7fc700 1 -- 172.21.15.121:0/1059731842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc8014e380 msgr2=0x7fdc8014ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:04.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.549+0000 7fdc7e7fc700 1 --2- 172.21.15.121:0/1059731842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc8014e380 0x7fdc8014ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.549+0000 7fdc7e7fc700 1 -- 172.21.15.121:0/1059731842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc800ad270 msgr2=0x7fdc80149490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:04.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.549+0000 7fdc7e7fc700 1 --2- 172.21.15.121:0/1059731842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc800ad270 0x7fdc80149490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.549+0000 7fdc7e7fc700 1 -- 172.21.15.121:0/1059731842 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc78009ae0 con 0x7fdc800c6040 2023-11-15T03:03:04.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.549+0000 7fdc7e7fc700 1 --2- 172.21.15.121:0/1059731842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc800c6040 0x7fdc80156000 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7fdc7400c440 tx=0x7fdc74007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:04.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.549+0000 7fdc66ffd700 1 -- 172.21.15.121:0/1059731842 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc74007020 con 0x7fdc800c6040 2023-11-15T03:03:04.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.549+0000 7fdc66ffd700 1 -- 172.21.15.121:0/1059731842 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc74007180 con 0x7fdc800c6040 2023-11-15T03:03:04.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.549+0000 7fdc7ffff700 1 -- 172.21.15.121:0/1059731842 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc800c1f10 con 0x7fdc800c6040 2023-11-15T03:03:04.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.549+0000 7fdc66ffd700 1 -- 172.21.15.121:0/1059731842 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc74007520 con 0x7fdc800c6040 2023-11-15T03:03:04.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.549+0000 7fdc7ffff700 1 -- 172.21.15.121:0/1059731842 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc80149980 con 0x7fdc800c6040 2023-11-15T03:03:04.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.549+0000 7fdc7ffff700 1 -- 172.21.15.121:0/1059731842 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc80153be0 con 0x7fdc800c6040 2023-11-15T03:03:04.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.553+0000 7fdc66ffd700 1 -- 172.21.15.121:0/1059731842 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fdc740068b0 con 0x7fdc800c6040 2023-11-15T03:03:04.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.553+0000 7fdc66ffd700 1 --2- 172.21.15.121:0/1059731842 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdc6c05a4e0 0x7fdc6c05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:04.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.553+0000 7fdc66ffd700 1 -- 172.21.15.121:0/1059731842 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fdc74073b70 con 0x7fdc800c6040 2023-11-15T03:03:04.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.553+0000 7fdc7effd700 1 --2- 172.21.15.121:0/1059731842 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdc6c05a4e0 0x7fdc6c05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:04.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.553+0000 7fdc7effd700 1 --2- 172.21.15.121:0/1059731842 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdc6c05a4e0 0x7fdc6c05c9a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fdc70009b90 tx=0x7fdc70009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:04.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.561+0000 7fdc66ffd700 1 -- 172.21.15.121:0/1059731842 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdc74047240 con 0x7fdc800c6040 2023-11-15T03:03:04.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.721+0000 7fdc7ffff700 1 -- 172.21.15.121:0/1059731842 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fdc800c0be0 con 0x7fdc800c6040 2023-11-15T03:03:04.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.721+0000 7fdc66ffd700 1 -- 172.21.15.121:0/1059731842 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v80) v1 ==== 110+0+10 (secure 0 0 0) 0x7fdc800c0be0 con 0x7fdc800c6040 2023-11-15T03:03:04.731 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T03:03:04.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.725+0000 7fdc7ffff700 1 -- 172.21.15.121:0/1059731842 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdc6c05a4e0 msgr2=0x7fdc6c05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:04.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.725+0000 7fdc7ffff700 1 --2- 172.21.15.121:0/1059731842 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdc6c05a4e0 0x7fdc6c05c9a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fdc70009b90 tx=0x7fdc70009060).stop 2023-11-15T03:03:04.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.725+0000 7fdc7ffff700 1 -- 172.21.15.121:0/1059731842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc800c6040 msgr2=0x7fdc80156000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:04.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.725+0000 7fdc7ffff700 1 --2- 172.21.15.121:0/1059731842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc800c6040 0x7fdc80156000 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7fdc7400c440 tx=0x7fdc74007da0).stop 2023-11-15T03:03:04.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.725+0000 7fdc7ffff700 1 -- 172.21.15.121:0/1059731842 shutdown_connections 2023-11-15T03:03:04.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.725+0000 7fdc7ffff700 1 --2- 172.21.15.121:0/1059731842 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdc6c05a4e0 0x7fdc6c05c9a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.725+0000 7fdc7ffff700 1 --2- 172.21.15.121:0/1059731842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc8014e380 0x7fdc8014ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.725+0000 7fdc7ffff700 1 --2- 172.21.15.121:0/1059731842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc800c6040 0x7fdc80156000 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.725+0000 7fdc7ffff700 1 --2- 172.21.15.121:0/1059731842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc800ad270 0x7fdc80149490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.725+0000 7fdc7ffff700 1 -- 172.21.15.121:0/1059731842 >> 172.21.15.121:0/1059731842 conn(0x7fdc80067a60 msgr2=0x7fdc8014d740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:04.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.725+0000 7fdc7ffff700 1 -- 172.21.15.121:0/1059731842 shutdown_connections 2023-11-15T03:03:04.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.725+0000 7fdc7ffff700 1 -- 172.21.15.121:0/1059731842 wait complete. 2023-11-15T03:03:04.756 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T03:03:04.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.929+0000 7fc044c79700 1 Processor -- start 2023-11-15T03:03:04.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.933+0000 7fc044c79700 1 -- start start 2023-11-15T03:03:04.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.933+0000 7fc044c79700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc0400c5c70 0x7fc040153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:04.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.933+0000 7fc044c79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc040066330 0x7fc0400ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:04.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.933+0000 7fc044c79700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc040065ea0 con 0x7fc0400acd00 2023-11-15T03:03:04.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.933+0000 7fc044c79700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc040066bb0 con 0x7fc0400c5c70 2023-11-15T03:03:04.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.933+0000 7fc044c79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc040065890 con 0x7fc040066330 2023-11-15T03:03:04.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.933+0000 7fc03f7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc0400acd00 0x7fc0401536c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45714/0 (socket says 172.21.15.121:45714) 2023-11-15T03:03:04.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.933+0000 7fc03f7fe700 1 -- 172.21.15.121:0/2946758168 learned_addr learned my addr 172.21.15.121:0/2946758168 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:04.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.933+0000 7fc03effd700 1 --2- 172.21.15.121:0/2946758168 >> v2:172.21.15.190:3300/0 conn(0x7fc0400c5c70 0x7fc040153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:04.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.933+0000 7fc03ffff700 1 --2- 172.21.15.121:0/2946758168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc040066330 0x7fc0400ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:04.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.933+0000 7fc03e7fc700 1 -- 172.21.15.121:0/2946758168 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767157443 0 0) 0x7fc040065ea0 con 0x7fc0400acd00 2023-11-15T03:03:04.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.933+0000 7fc03effd700 1 -- 172.21.15.121:0/2946758168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc040066330 msgr2=0x7fc0400ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:04.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.937+0000 7fc03effd700 1 --2- 172.21.15.121:0/2946758168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc040066330 0x7fc0400ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.937+0000 7fc03effd700 1 -- 172.21.15.121:0/2946758168 >> v1:172.21.15.121:6789/0 conn(0x7fc0400acd00 legacy=0x7fc0401536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:04.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.937+0000 7fc03effd700 1 -- 172.21.15.121:0/2946758168 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0400c64a0 con 0x7fc0400c5c70 2023-11-15T03:03:04.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.937+0000 7fc03effd700 1 --2- 172.21.15.121:0/2946758168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0400c5c70 0x7fc040153dd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fc034009760 tx=0x7fc034009060).ready entity=mon.1 client_cookie=a8dbc37cc491f1e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:04.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.937+0000 7fc03e7fc700 1 -- 172.21.15.121:0/2946758168 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03400f510 con 0x7fc0400c5c70 2023-11-15T03:03:04.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.937+0000 7fc03e7fc700 1 -- 172.21.15.121:0/2946758168 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03400f670 con 0x7fc0400c5c70 2023-11-15T03:03:04.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.937+0000 7fc03e7fc700 1 -- 172.21.15.121:0/2946758168 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03400f9f0 con 0x7fc0400c5c70 2023-11-15T03:03:04.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.937+0000 7fc044c79700 1 -- 172.21.15.121:0/2946758168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0400c5c70 msgr2=0x7fc040153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:04.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.937+0000 7fc044c79700 1 --2- 172.21.15.121:0/2946758168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0400c5c70 0x7fc040153dd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fc034009760 tx=0x7fc034009060).stop 2023-11-15T03:03:04.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.937+0000 7fc044c79700 1 -- 172.21.15.121:0/2946758168 shutdown_connections 2023-11-15T03:03:04.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.937+0000 7fc044c79700 1 --2- 172.21.15.121:0/2946758168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc040066330 0x7fc0400ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.937+0000 7fc044c79700 1 --2- 172.21.15.121:0/2946758168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0400c5c70 0x7fc040153dd0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.937+0000 7fc044c79700 1 -- 172.21.15.121:0/2946758168 >> 172.21.15.121:0/2946758168 conn(0x7fc040063fc0 msgr2=0x7fc040067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:04.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.937+0000 7fc044c79700 1 -- 172.21.15.121:0/2946758168 shutdown_connections 2023-11-15T03:03:04.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.937+0000 7fc044c79700 1 -- 172.21.15.121:0/2946758168 wait complete. 2023-11-15T03:03:04.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.941+0000 7fc044c79700 1 Processor -- start 2023-11-15T03:03:04.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.941+0000 7fc044c79700 1 -- start start 2023-11-15T03:03:04.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.941+0000 7fc044c79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc040066330 0x7fc040151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:04.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.941+0000 7fc044c79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0400acd00 0x7fc0400c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:04.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.941+0000 7fc044c79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0400c5c70 0x7fc04013f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:04.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.941+0000 7fc044c79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc040155090 con 0x7fc0400c5c70 2023-11-15T03:03:04.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.941+0000 7fc044c79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04014eb50 con 0x7fc040066330 2023-11-15T03:03:04.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.941+0000 7fc044c79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc040154410 con 0x7fc0400acd00 2023-11-15T03:03:04.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.941+0000 7fc03ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0400c5c70 0x7fc04013f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:04.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.941+0000 7fc03ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0400c5c70 0x7fc04013f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:60876/0 (socket says 172.21.15.121:60876) 2023-11-15T03:03:04.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.941+0000 7fc03ffff700 1 -- 172.21.15.121:0/2630596463 learned_addr learned my addr 172.21.15.121:0/2630596463 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:04.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.941+0000 7fc03f7fe700 1 --2- 172.21.15.121:0/2630596463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc040066330 0x7fc040151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:04.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.941+0000 7fc03effd700 1 --2- 172.21.15.121:0/2630596463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0400acd00 0x7fc0400c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:04.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.941+0000 7fc03ffff700 1 -- 172.21.15.121:0/2630596463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0400acd00 msgr2=0x7fc0400c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:04.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.941+0000 7fc03ffff700 1 --2- 172.21.15.121:0/2630596463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0400acd00 0x7fc0400c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.941+0000 7fc03ffff700 1 -- 172.21.15.121:0/2630596463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc040066330 msgr2=0x7fc040151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:04.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.941+0000 7fc03ffff700 1 --2- 172.21.15.121:0/2630596463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc040066330 0x7fc040151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:04.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.941+0000 7fc03ffff700 1 -- 172.21.15.121:0/2630596463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0340092a0 con 0x7fc0400c5c70 2023-11-15T03:03:04.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.945+0000 7fc03effd700 1 --2- 172.21.15.121:0/2630596463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0400acd00 0x7fc0400c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:04.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.945+0000 7fc03ffff700 1 --2- 172.21.15.121:0/2630596463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0400c5c70 0x7fc04013f250 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7fc038009ec0 tx=0x7fc03800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:04.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.945+0000 7fc0277fe700 1 -- 172.21.15.121:0/2630596463 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc038009bf0 con 0x7fc0400c5c70 2023-11-15T03:03:04.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.945+0000 7fc0277fe700 1 -- 172.21.15.121:0/2630596463 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03800f040 con 0x7fc0400c5c70 2023-11-15T03:03:04.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.945+0000 7fc044c79700 1 -- 172.21.15.121:0/2630596463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0401546f0 con 0x7fc0400c5c70 2023-11-15T03:03:04.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.945+0000 7fc0277fe700 1 -- 172.21.15.121:0/2630596463 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03800ce30 con 0x7fc0400c5c70 2023-11-15T03:03:04.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.945+0000 7fc044c79700 1 -- 172.21.15.121:0/2630596463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc040154cb0 con 0x7fc0400c5c70 2023-11-15T03:03:04.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.945+0000 7fc044c79700 1 -- 172.21.15.121:0/2630596463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0401551d0 con 0x7fc0400c5c70 2023-11-15T03:03:04.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.949+0000 7fc0277fe700 1 -- 172.21.15.121:0/2630596463 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc038013610 con 0x7fc0400c5c70 2023-11-15T03:03:04.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.949+0000 7fc0277fe700 1 --2- 172.21.15.121:0/2630596463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc02c05a510 0x7fc02c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:04.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.949+0000 7fc0277fe700 1 -- 172.21.15.121:0/2630596463 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fc038075040 con 0x7fc0400c5c70 2023-11-15T03:03:04.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.949+0000 7fc03f7fe700 1 --2- 172.21.15.121:0/2630596463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc02c05a510 0x7fc02c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:04.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.949+0000 7fc03f7fe700 1 --2- 172.21.15.121:0/2630596463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc02c05a510 0x7fc02c05c9d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7fc030001230 tx=0x7fc030007b50).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:04.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:04.957+0000 7fc0277fe700 1 -- 172.21.15.121:0/2630596463 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc0401551d0 con 0x7fc0400c5c70 2023-11-15T03:03:05.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.113+0000 7fc044c79700 1 -- 172.21.15.121:0/2630596463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc0401551d0 con 0x7fc0400c5c70 2023-11-15T03:03:05.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.113+0000 7fc0277fe700 1 -- 172.21.15.121:0/2630596463 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v80) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc0401551d0 con 0x7fc0400c5c70 2023-11-15T03:03:05.123 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T03:03:05.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.117+0000 7fc044c79700 1 -- 172.21.15.121:0/2630596463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc02c05a510 msgr2=0x7fc02c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:05.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.117+0000 7fc044c79700 1 --2- 172.21.15.121:0/2630596463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc02c05a510 0x7fc02c05c9d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7fc030001230 tx=0x7fc030007b50).stop 2023-11-15T03:03:05.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.117+0000 7fc044c79700 1 -- 172.21.15.121:0/2630596463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0400c5c70 msgr2=0x7fc04013f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:05.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.117+0000 7fc044c79700 1 --2- 172.21.15.121:0/2630596463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0400c5c70 0x7fc04013f250 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7fc038009ec0 tx=0x7fc03800b040).stop 2023-11-15T03:03:05.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.121+0000 7fc044c79700 1 -- 172.21.15.121:0/2630596463 shutdown_connections 2023-11-15T03:03:05.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.121+0000 7fc044c79700 1 --2- 172.21.15.121:0/2630596463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc02c05a510 0x7fc02c05c9d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.121+0000 7fc044c79700 1 --2- 172.21.15.121:0/2630596463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0400c5c70 0x7fc04013f250 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.121+0000 7fc044c79700 1 --2- 172.21.15.121:0/2630596463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0400acd00 0x7fc0400c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.121+0000 7fc044c79700 1 --2- 172.21.15.121:0/2630596463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc040066330 0x7fc040151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.121+0000 7fc044c79700 1 -- 172.21.15.121:0/2630596463 >> 172.21.15.121:0/2630596463 conn(0x7fc040063fc0 msgr2=0x7fc0401528e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:05.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.121+0000 7fc044c79700 1 -- 172.21.15.121:0/2630596463 shutdown_connections 2023-11-15T03:03:05.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.121+0000 7fc044c79700 1 -- 172.21.15.121:0/2630596463 wait complete. 2023-11-15T03:03:05.149 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-15T03:03:05.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.297+0000 7f2bc80f1700 1 Processor -- start 2023-11-15T03:03:05.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.301+0000 7f2bc80f1700 1 -- start start 2023-11-15T03:03:05.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.301+0000 7f2bc80f1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2bc00a8b80 0x7f2bc00a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:05.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.301+0000 7f2bc80f1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2bc0065850 0x7f2bc0067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:05.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.301+0000 7f2bc80f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bc0067f50 0x7f2bc00bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:05.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.301+0000 7f2bc80f1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc0068400 con 0x7f2bc0065850 2023-11-15T03:03:05.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.301+0000 7f2bc80f1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc0152150 con 0x7f2bc00a8b80 2023-11-15T03:03:05.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.301+0000 7f2bc80f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc0148f70 con 0x7f2bc0067f50 2023-11-15T03:03:05.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.301+0000 7f2bc568c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2bc0065850 0x7f2bc0067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:05.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.301+0000 7f2bc668e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bc0067f50 0x7f2bc00bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:05.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.301+0000 7f2bc5e8d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2bc00a8b80 0x7f2bc00a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:05.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.301+0000 7f2bc568c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2bc0065850 0x7f2bc0067a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:60884/0 (socket says 172.21.15.121:60884) 2023-11-15T03:03:05.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.301+0000 7f2bc668e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bc0067f50 0x7f2bc00bbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:39468/0 (socket says 172.21.15.121:39468) 2023-11-15T03:03:05.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.301+0000 7f2bc568c700 1 -- 172.21.15.121:0/1969559766 learned_addr learned my addr 172.21.15.121:0/1969559766 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:05.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.305+0000 7f2bc568c700 1 -- 172.21.15.121:0/1969559766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bc0067f50 msgr2=0x7f2bc00bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:05.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.305+0000 7f2bc568c700 1 --2- 172.21.15.121:0/1969559766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bc0067f50 0x7f2bc00bbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.305+0000 7f2bc568c700 1 -- 172.21.15.121:0/1969559766 >> v2:172.21.15.190:3300/0 conn(0x7f2bc00a8b80 msgr2=0x7f2bc00a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:05.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.305+0000 7f2bc568c700 1 --2- 172.21.15.121:0/1969559766 >> v2:172.21.15.190:3300/0 conn(0x7f2bc00a8b80 0x7f2bc00a8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.305+0000 7f2bc568c700 1 -- 172.21.15.121:0/1969559766 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc01523a0 con 0x7f2bc0065850 2023-11-15T03:03:05.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.305+0000 7f2bc568c700 1 --2- 172.21.15.121:0/1969559766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bc0065850 0x7f2bc0067a60 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f2bc006a400 tx=0x7f2bb00090f0).ready entity=mon.0 client_cookie=cbb81b60993dacd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:05.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.305+0000 7f2bc4e8b700 1 -- 172.21.15.121:0/1969559766 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb000f510 con 0x7f2bc0065850 2023-11-15T03:03:05.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.305+0000 7f2bc4e8b700 1 -- 172.21.15.121:0/1969559766 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb000f670 con 0x7f2bc0065850 2023-11-15T03:03:05.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.305+0000 7f2bc4e8b700 1 -- 172.21.15.121:0/1969559766 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb000fa10 con 0x7f2bc0065850 2023-11-15T03:03:05.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.305+0000 7f2bc80f1700 1 -- 172.21.15.121:0/1969559766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bc0065850 msgr2=0x7f2bc0067a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:05.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.305+0000 7f2bc80f1700 1 --2- 172.21.15.121:0/1969559766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bc0065850 0x7f2bc0067a60 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f2bc006a400 tx=0x7f2bb00090f0).stop 2023-11-15T03:03:05.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.305+0000 7f2bc80f1700 1 -- 172.21.15.121:0/1969559766 shutdown_connections 2023-11-15T03:03:05.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.305+0000 7f2bc80f1700 1 --2- 172.21.15.121:0/1969559766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bc0067f50 0x7f2bc00bbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.305+0000 7f2bc80f1700 1 --2- 172.21.15.121:0/1969559766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bc0065850 0x7f2bc0067a60 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.305+0000 7f2bc80f1700 1 --2- 172.21.15.121:0/1969559766 >> v2:172.21.15.190:3300/0 conn(0x7f2bc00a8b80 0x7f2bc00a8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.305+0000 7f2bc80f1700 1 -- 172.21.15.121:0/1969559766 >> 172.21.15.121:0/1969559766 conn(0x7f2bc00591c0 msgr2=0x7f2bc00c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:05.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.305+0000 7f2bc80f1700 1 -- 172.21.15.121:0/1969559766 shutdown_connections 2023-11-15T03:03:05.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.305+0000 7f2bc80f1700 1 -- 172.21.15.121:0/1969559766 wait complete. 2023-11-15T03:03:05.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.309+0000 7f2bc80f1700 1 Processor -- start 2023-11-15T03:03:05.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.309+0000 7f2bc80f1700 1 -- start start 2023-11-15T03:03:05.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.309+0000 7f2bc80f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bc0065850 0x7f2bc0145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:05.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.309+0000 7f2bc80f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bc0067f50 0x7f2bc01555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:05.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.309+0000 7f2bc80f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bc00a8b80 0x7f2bc0155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:05.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.309+0000 7f2bc80f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc0155f80 con 0x7f2bc00a8b80 2023-11-15T03:03:05.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.309+0000 7f2bc80f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc00aa650 con 0x7f2bc0067f50 2023-11-15T03:03:05.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.309+0000 7f2bc80f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc00aa930 con 0x7f2bc0065850 2023-11-15T03:03:05.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.309+0000 7f2bc668e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bc00a8b80 0x7f2bc0155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:05.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.309+0000 7f2bc668e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bc00a8b80 0x7f2bc0155a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:60894/0 (socket says 172.21.15.121:60894) 2023-11-15T03:03:05.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.309+0000 7f2bc668e700 1 -- 172.21.15.121:0/2655845475 learned_addr learned my addr 172.21.15.121:0/2655845475 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:05.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.309+0000 7f2bc5e8d700 1 --2- 172.21.15.121:0/2655845475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bc0065850 0x7f2bc0145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:05.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.309+0000 7f2bc568c700 1 --2- 172.21.15.121:0/2655845475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bc0067f50 0x7f2bc01555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:05.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.309+0000 7f2bc668e700 1 -- 172.21.15.121:0/2655845475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bc0065850 msgr2=0x7f2bc0145610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:05.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.309+0000 7f2bc668e700 1 --2- 172.21.15.121:0/2655845475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bc0065850 0x7f2bc0145610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.309+0000 7f2bc668e700 1 -- 172.21.15.121:0/2655845475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bc0067f50 msgr2=0x7f2bc01555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:05.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.309+0000 7f2bc668e700 1 --2- 172.21.15.121:0/2655845475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bc0067f50 0x7f2bc01555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.313+0000 7f2bc668e700 1 -- 172.21.15.121:0/2655845475 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb0009370 con 0x7f2bc00a8b80 2023-11-15T03:03:05.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.313+0000 7f2bc668e700 1 --2- 172.21.15.121:0/2655845475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bc00a8b80 0x7f2bc0155a90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f2bb8009df0 tx=0x7f2bb800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:05.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.313+0000 7f2bb5ffb700 1 -- 172.21.15.121:0/2655845475 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb800caa0 con 0x7f2bc00a8b80 2023-11-15T03:03:05.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.313+0000 7f2bb5ffb700 1 -- 172.21.15.121:0/2655845475 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb800f040 con 0x7f2bc00a8b80 2023-11-15T03:03:05.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.313+0000 7f2bb5ffb700 1 -- 172.21.15.121:0/2655845475 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb800ce10 con 0x7f2bc00a8b80 2023-11-15T03:03:05.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.313+0000 7f2bc80f1700 1 -- 172.21.15.121:0/2655845475 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc014a210 con 0x7f2bc00a8b80 2023-11-15T03:03:05.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.313+0000 7f2bc80f1700 1 -- 172.21.15.121:0/2655845475 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc014a7d0 con 0x7f2bc00a8b80 2023-11-15T03:03:05.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.313+0000 7f2bc80f1700 1 -- 172.21.15.121:0/2655845475 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc0047060 con 0x7f2bc00a8b80 2023-11-15T03:03:05.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.317+0000 7f2bb5ffb700 1 -- 172.21.15.121:0/2655845475 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2bb8013610 con 0x7f2bc00a8b80 2023-11-15T03:03:05.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.317+0000 7f2bb5ffb700 1 --2- 172.21.15.121:0/2655845475 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2ba405a510 0x7f2ba405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:05.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.317+0000 7f2bb5ffb700 1 -- 172.21.15.121:0/2655845475 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f2bb8074f00 con 0x7f2bc00a8b80 2023-11-15T03:03:05.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.317+0000 7f2bc5e8d700 1 --2- 172.21.15.121:0/2655845475 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2ba405a510 0x7f2ba405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:05.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.317+0000 7f2bc5e8d700 1 --2- 172.21.15.121:0/2655845475 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2ba405a510 0x7f2ba405c9d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f2bbc009b00 tx=0x7f2bbc009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:05.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.325+0000 7f2bb5ffb700 1 -- 172.21.15.121:0/2655845475 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2bb8048650 con 0x7f2bc00a8b80 2023-11-15T03:03:05.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.485+0000 7f2bc80f1700 1 -- 172.21.15.121:0/2655845475 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2bc014a9c0 con 0x7f2bc00a8b80 2023-11-15T03:03:05.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.485+0000 7f2bb5ffb700 1 -- 172.21.15.121:0/2655845475 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2bb804bc70 con 0x7f2bc00a8b80 2023-11-15T03:03:05.495 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T03:03:05.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.489+0000 7f2bc80f1700 1 -- 172.21.15.121:0/2655845475 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2ba405a510 msgr2=0x7f2ba405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:05.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.489+0000 7f2bc80f1700 1 --2- 172.21.15.121:0/2655845475 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2ba405a510 0x7f2ba405c9d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f2bbc009b00 tx=0x7f2bbc009060).stop 2023-11-15T03:03:05.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.489+0000 7f2bc80f1700 1 -- 172.21.15.121:0/2655845475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bc00a8b80 msgr2=0x7f2bc0155a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:05.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.489+0000 7f2bc80f1700 1 --2- 172.21.15.121:0/2655845475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bc00a8b80 0x7f2bc0155a90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f2bb8009df0 tx=0x7f2bb800b040).stop 2023-11-15T03:03:05.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.489+0000 7f2bc80f1700 1 -- 172.21.15.121:0/2655845475 shutdown_connections 2023-11-15T03:03:05.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.489+0000 7f2bc80f1700 1 --2- 172.21.15.121:0/2655845475 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2ba405a510 0x7f2ba405c9d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.489+0000 7f2bc80f1700 1 --2- 172.21.15.121:0/2655845475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bc00a8b80 0x7f2bc0155a90 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.489+0000 7f2bc80f1700 1 --2- 172.21.15.121:0/2655845475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bc0067f50 0x7f2bc01555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.489+0000 7f2bc80f1700 1 --2- 172.21.15.121:0/2655845475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bc0065850 0x7f2bc0145610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.489+0000 7f2bc80f1700 1 -- 172.21.15.121:0/2655845475 >> 172.21.15.121:0/2655845475 conn(0x7f2bc00591c0 msgr2=0x7f2bc00c4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:05.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.489+0000 7f2bc80f1700 1 -- 172.21.15.121:0/2655845475 shutdown_connections 2023-11-15T03:03:05.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.489+0000 7f2bc80f1700 1 -- 172.21.15.121:0/2655845475 wait complete. 2023-11-15T03:03:05.518 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-15T03:03:05.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.685+0000 7f6d54969700 1 Processor -- start 2023-11-15T03:03:05.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.685+0000 7f6d54969700 1 -- start start 2023-11-15T03:03:05.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.685+0000 7f6d54969700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d500639b0 0x7f6d500bcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:05.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.685+0000 7f6d54969700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6d500bb6f0 0x7f6d50153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:05.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.685+0000 7f6d54969700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d500a9250 con 0x7f6d50153c10 2023-11-15T03:03:05.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.685+0000 7f6d54969700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d500bd5a0 con 0x7f6d500bb6f0 2023-11-15T03:03:05.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.685+0000 7f6d54969700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d500a8920 con 0x7f6d500639b0 2023-11-15T03:03:05.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.685+0000 7f6d4ed9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6d500bb6f0 0x7f6d50153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:05.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.685+0000 7f6d4e59c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f6d50153c10 0x7f6d500632a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:60676/0 (socket says 172.21.15.121:60676) 2023-11-15T03:03:05.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.685+0000 7f6d4e59c700 1 -- 172.21.15.121:0/3539673180 learned_addr learned my addr 172.21.15.121:0/3539673180 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:05.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.685+0000 7f6d4dd9b700 1 --2- 172.21.15.121:0/3539673180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d500639b0 0x7f6d500bcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:05.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.685+0000 7f6d4d59a700 1 -- 172.21.15.121:0/3539673180 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318145569 0 0) 0x7f6d500a9250 con 0x7f6d50153c10 2023-11-15T03:03:05.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.685+0000 7f6d4d59a700 1 -- 172.21.15.121:0/3539673180 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d380033f0 con 0x7f6d50153c10 2023-11-15T03:03:05.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.685+0000 7f6d4ed9d700 1 -- 172.21.15.121:0/3539673180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d500639b0 msgr2=0x7f6d500bcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:05.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.685+0000 7f6d4ed9d700 1 --2- 172.21.15.121:0/3539673180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d500639b0 0x7f6d500bcfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.685+0000 7f6d4ed9d700 1 -- 172.21.15.121:0/3539673180 >> v1:172.21.15.190:6789/0 conn(0x7f6d50153c10 legacy=0x7f6d500632a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:05.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.685+0000 7f6d4ed9d700 1 -- 172.21.15.121:0/3539673180 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d500a8c00 con 0x7f6d500bb6f0 2023-11-15T03:03:05.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.685+0000 7f6d4dd9b700 1 --2- 172.21.15.121:0/3539673180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d500639b0 0x7f6d500bcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:03:05.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.689+0000 7f6d4ed9d700 1 --2- 172.21.15.121:0/3539673180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d500bb6f0 0x7f6d50153ff0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f6d48009e60 tx=0x7f6d48009780).ready entity=mon.0 client_cookie=29f0fe0871a87647 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:05.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.689+0000 7f6d4d59a700 1 -- 172.21.15.121:0/3539673180 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4800e070 con 0x7f6d500bb6f0 2023-11-15T03:03:05.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.689+0000 7f6d4d59a700 1 -- 172.21.15.121:0/3539673180 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4800bab0 con 0x7f6d500bb6f0 2023-11-15T03:03:05.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.689+0000 7f6d4d59a700 1 -- 172.21.15.121:0/3539673180 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4800be30 con 0x7f6d500bb6f0 2023-11-15T03:03:05.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.689+0000 7f6d54969700 1 -- 172.21.15.121:0/3539673180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d500bb6f0 msgr2=0x7f6d50153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:05.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.689+0000 7f6d54969700 1 --2- 172.21.15.121:0/3539673180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d500bb6f0 0x7f6d50153ff0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f6d48009e60 tx=0x7f6d48009780).stop 2023-11-15T03:03:05.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.689+0000 7f6d54969700 1 -- 172.21.15.121:0/3539673180 shutdown_connections 2023-11-15T03:03:05.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.689+0000 7f6d54969700 1 --2- 172.21.15.121:0/3539673180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d500bb6f0 0x7f6d50153ff0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.689+0000 7f6d54969700 1 --2- 172.21.15.121:0/3539673180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d500639b0 0x7f6d500bcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.689+0000 7f6d54969700 1 -- 172.21.15.121:0/3539673180 >> 172.21.15.121:0/3539673180 conn(0x7f6d500591c0 msgr2=0x7f6d500c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:05.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.689+0000 7f6d54969700 1 -- 172.21.15.121:0/3539673180 shutdown_connections 2023-11-15T03:03:05.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.689+0000 7f6d54969700 1 -- 172.21.15.121:0/3539673180 wait complete. 2023-11-15T03:03:05.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d54969700 1 Processor -- start 2023-11-15T03:03:05.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d54969700 1 -- start start 2023-11-15T03:03:05.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d54969700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d500639b0 0x7f6d50143780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:05.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d54969700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d500bb6f0 0x7f6d500bd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:05.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d54969700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d50153c10 0x7f6d500c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:05.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d54969700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d500bee70 con 0x7f6d500bb6f0 2023-11-15T03:03:05.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d54969700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d500be210 con 0x7f6d50153c10 2023-11-15T03:03:05.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d54969700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d500be4f0 con 0x7f6d500639b0 2023-11-15T03:03:05.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d4dd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d500bb6f0 0x7f6d500bd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:05.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d4dd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d500bb6f0 0x7f6d500bd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:60900/0 (socket says 172.21.15.121:60900) 2023-11-15T03:03:05.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d4dd9b700 1 -- 172.21.15.121:0/2373798181 learned_addr learned my addr 172.21.15.121:0/2373798181 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:05.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d4ed9d700 1 --2- 172.21.15.121:0/2373798181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d50153c10 0x7f6d500c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:05.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d4e59c700 1 --2- 172.21.15.121:0/2373798181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d500639b0 0x7f6d50143780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:05.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d4dd9b700 1 -- 172.21.15.121:0/2373798181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d500639b0 msgr2=0x7f6d50143780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:05.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d4dd9b700 1 --2- 172.21.15.121:0/2373798181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d500639b0 0x7f6d50143780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d4dd9b700 1 -- 172.21.15.121:0/2373798181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d50153c10 msgr2=0x7f6d500c5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:05.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d4dd9b700 1 --2- 172.21.15.121:0/2373798181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d50153c10 0x7f6d500c5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d4dd9b700 1 -- 172.21.15.121:0/2373798181 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d440092a0 con 0x7f6d500bb6f0 2023-11-15T03:03:05.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d4dd9b700 1 --2- 172.21.15.121:0/2373798181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d500bb6f0 0x7f6d500bd830 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f6d4400e9a0 tx=0x7f6d44009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:05.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d3e7fc700 1 -- 172.21.15.121:0/2373798181 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4400ee90 con 0x7f6d500bb6f0 2023-11-15T03:03:05.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d3e7fc700 1 -- 172.21.15.121:0/2373798181 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4400c850 con 0x7f6d500bb6f0 2023-11-15T03:03:05.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d54969700 1 -- 172.21.15.121:0/2373798181 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d48009a20 con 0x7f6d500bb6f0 2023-11-15T03:03:05.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d3e7fc700 1 -- 172.21.15.121:0/2373798181 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4400cbd0 con 0x7f6d500bb6f0 2023-11-15T03:03:05.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d54969700 1 -- 172.21.15.121:0/2373798181 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5014d1a0 con 0x7f6d500bb6f0 2023-11-15T03:03:05.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d54969700 1 -- 172.21.15.121:0/2373798181 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d50047060 con 0x7f6d500bb6f0 2023-11-15T03:03:05.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.697+0000 7f6d3e7fc700 1 -- 172.21.15.121:0/2373798181 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6d4401b400 con 0x7f6d500bb6f0 2023-11-15T03:03:05.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.701+0000 7f6d3e7fc700 1 --2- 172.21.15.121:0/2373798181 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6d3005a510 0x7f6d3005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:05.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.701+0000 7f6d3e7fc700 1 -- 172.21.15.121:0/2373798181 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f6d44074f10 con 0x7f6d500bb6f0 2023-11-15T03:03:05.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.701+0000 7f6d4e59c700 1 --2- 172.21.15.121:0/2373798181 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6d3005a510 0x7f6d3005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:05.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.701+0000 7f6d4e59c700 1 --2- 172.21.15.121:0/2373798181 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6d3005a510 0x7f6d3005c9d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f6d40005750 tx=0x7f6d40009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:05.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.705+0000 7f6d3e7fc700 1 -- 172.21.15.121:0/2373798181 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6d44048540 con 0x7f6d500bb6f0 2023-11-15T03:03:05.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.877+0000 7f6d54969700 1 -- 172.21.15.121:0/2373798181 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6d50066ad0 con 0x7f6d500bb6f0 2023-11-15T03:03:05.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.881+0000 7f6d3e7fc700 1 -- 172.21.15.121:0/2373798181 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6d44016020 con 0x7f6d500bb6f0 2023-11-15T03:03:05.889 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T03:03:05.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.885+0000 7f6d54969700 1 -- 172.21.15.121:0/2373798181 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6d3005a510 msgr2=0x7f6d3005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:05.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.885+0000 7f6d54969700 1 --2- 172.21.15.121:0/2373798181 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6d3005a510 0x7f6d3005c9d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f6d40005750 tx=0x7f6d40009040).stop 2023-11-15T03:03:05.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.885+0000 7f6d54969700 1 -- 172.21.15.121:0/2373798181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d500bb6f0 msgr2=0x7f6d500bd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:05.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.885+0000 7f6d54969700 1 --2- 172.21.15.121:0/2373798181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d500bb6f0 0x7f6d500bd830 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f6d4400e9a0 tx=0x7f6d44009060).stop 2023-11-15T03:03:05.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.885+0000 7f6d54969700 1 -- 172.21.15.121:0/2373798181 shutdown_connections 2023-11-15T03:03:05.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.885+0000 7f6d54969700 1 --2- 172.21.15.121:0/2373798181 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6d3005a510 0x7f6d3005c9d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.885+0000 7f6d54969700 1 --2- 172.21.15.121:0/2373798181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d50153c10 0x7f6d500c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.885+0000 7f6d54969700 1 --2- 172.21.15.121:0/2373798181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d500bb6f0 0x7f6d500bd830 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.885+0000 7f6d54969700 1 --2- 172.21.15.121:0/2373798181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d500639b0 0x7f6d50143780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:05.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.885+0000 7f6d54969700 1 -- 172.21.15.121:0/2373798181 >> 172.21.15.121:0/2373798181 conn(0x7f6d500591c0 msgr2=0x7f6d501463c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:05.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.885+0000 7f6d54969700 1 -- 172.21.15.121:0/2373798181 shutdown_connections 2023-11-15T03:03:05.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:05.885+0000 7f6d54969700 1 -- 172.21.15.121:0/2373798181 wait complete. 2023-11-15T03:03:05.918 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:03:06.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f8474db9700 1 Processor -- start 2023-11-15T03:03:06.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f8474db9700 1 -- start start 2023-11-15T03:03:06.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f8474db9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f84700cabd0 0x7f84700cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:06.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f8474db9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84700acc90 0x7f8470066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:06.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f8474db9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84700ad650 con 0x7f84700ad0f0 2023-11-15T03:03:06.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f8474db9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8470065fc0 con 0x7f84700cabd0 2023-11-15T03:03:06.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f8474db9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84700662a0 con 0x7f84700acc90 2023-11-15T03:03:06.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f846f7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f84700cabd0 0x7f84700cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:06.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f846ffff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f84700ad0f0 0x7f84700ae190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45730/0 (socket says 172.21.15.121:45730) 2023-11-15T03:03:06.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f846ffff700 1 -- 172.21.15.121:0/2261950258 learned_addr learned my addr 172.21.15.121:0/2261950258 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:06.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f846effd700 1 --2- 172.21.15.121:0/2261950258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84700acc90 0x7f8470066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:06.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f846f7fe700 1 -- 172.21.15.121:0/2261950258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84700acc90 msgr2=0x7f8470066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:06.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f846f7fe700 1 --2- 172.21.15.121:0/2261950258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84700acc90 0x7f8470066d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f846f7fe700 1 -- 172.21.15.121:0/2261950258 >> v1:172.21.15.121:6789/0 conn(0x7f84700ad0f0 legacy=0x7f84700ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:06.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f846f7fe700 1 -- 172.21.15.121:0/2261950258 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8470066580 con 0x7f84700cabd0 2023-11-15T03:03:06.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f846e7fc700 1 -- 172.21.15.121:0/2261950258 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638986446 0 0) 0x7f84700ad650 con 0x7f84700ad0f0 2023-11-15T03:03:06.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f846f7fe700 1 --2- 172.21.15.121:0/2261950258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84700cabd0 0x7f84700cafb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f8460009760 tx=0x7f8460009060).ready entity=mon.0 client_cookie=75d9d258079b9691 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:06.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f846e7fc700 1 -- 172.21.15.121:0/2261950258 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846000f510 con 0x7f84700cabd0 2023-11-15T03:03:06.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f846e7fc700 1 -- 172.21.15.121:0/2261950258 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846000f670 con 0x7f84700cabd0 2023-11-15T03:03:06.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.077+0000 7f846e7fc700 1 -- 172.21.15.121:0/2261950258 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846000f9f0 con 0x7f84700cabd0 2023-11-15T03:03:06.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.081+0000 7f8474db9700 1 -- 172.21.15.121:0/2261950258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84700cabd0 msgr2=0x7f84700cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:06.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.081+0000 7f8474db9700 1 --2- 172.21.15.121:0/2261950258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84700cabd0 0x7f84700cafb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f8460009760 tx=0x7f8460009060).stop 2023-11-15T03:03:06.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.081+0000 7f8474db9700 1 -- 172.21.15.121:0/2261950258 shutdown_connections 2023-11-15T03:03:06.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.081+0000 7f8474db9700 1 --2- 172.21.15.121:0/2261950258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84700acc90 0x7f8470066d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.081+0000 7f8474db9700 1 --2- 172.21.15.121:0/2261950258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84700cabd0 0x7f84700cafb0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.081+0000 7f8474db9700 1 -- 172.21.15.121:0/2261950258 >> 172.21.15.121:0/2261950258 conn(0x7f8470062190 msgr2=0x7f84700625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:06.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.081+0000 7f8474db9700 1 -- 172.21.15.121:0/2261950258 shutdown_connections 2023-11-15T03:03:06.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.081+0000 7f8474db9700 1 -- 172.21.15.121:0/2261950258 wait complete. 2023-11-15T03:03:06.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.081+0000 7f8474db9700 1 Processor -- start 2023-11-15T03:03:06.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.081+0000 7f8474db9700 1 -- start start 2023-11-15T03:03:06.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f8474db9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84700acc90 0x7f847014ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:06.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f8474db9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84700ad0f0 0x7f84700c1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:06.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f8474db9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84700cabd0 0x7f8470154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:06.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f8474db9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8470153a60 con 0x7f84700acc90 2023-11-15T03:03:06.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f8474db9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847014b200 con 0x7f84700ad0f0 2023-11-15T03:03:06.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f8474db9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8470155630 con 0x7f84700cabd0 2023-11-15T03:03:06.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f846f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84700acc90 0x7f847014ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:06.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f846f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84700acc90 0x7f847014ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:60926/0 (socket says 172.21.15.121:60926) 2023-11-15T03:03:06.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f846f7fe700 1 -- 172.21.15.121:0/1979550428 learned_addr learned my addr 172.21.15.121:0/1979550428 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:06.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f846ffff700 1 --2- 172.21.15.121:0/1979550428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84700cabd0 0x7f8470154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:06.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f846f7fe700 1 -- 172.21.15.121:0/1979550428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84700cabd0 msgr2=0x7f8470154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:06.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f846f7fe700 1 --2- 172.21.15.121:0/1979550428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84700cabd0 0x7f8470154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f846f7fe700 1 -- 172.21.15.121:0/1979550428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84700ad0f0 msgr2=0x7f84700c1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:06.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f846effd700 1 --2- 172.21.15.121:0/1979550428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84700ad0f0 0x7f84700c1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:06.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f846f7fe700 1 --2- 172.21.15.121:0/1979550428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84700ad0f0 0x7f84700c1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f846f7fe700 1 -- 172.21.15.121:0/1979550428 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84600092a0 con 0x7f84700acc90 2023-11-15T03:03:06.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f846effd700 1 --2- 172.21.15.121:0/1979550428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84700ad0f0 0x7f84700c1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:06.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f846f7fe700 1 --2- 172.21.15.121:0/1979550428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84700acc90 0x7f847014ad10 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f8460009e00 tx=0x7f8460017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:06.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f84577fe700 1 -- 172.21.15.121:0/1979550428 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846000f7e0 con 0x7f84700acc90 2023-11-15T03:03:06.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f84577fe700 1 -- 172.21.15.121:0/1979550428 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846000f960 con 0x7f84700acc90 2023-11-15T03:03:06.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f8474db9700 1 -- 172.21.15.121:0/1979550428 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8470155910 con 0x7f84700acc90 2023-11-15T03:03:06.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f84577fe700 1 -- 172.21.15.121:0/1979550428 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846001b690 con 0x7f84700acc90 2023-11-15T03:03:06.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.085+0000 7f8474db9700 1 -- 172.21.15.121:0/1979550428 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84700bfc50 con 0x7f84700acc90 2023-11-15T03:03:06.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.089+0000 7f8474db9700 1 -- 172.21.15.121:0/1979550428 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8470153550 con 0x7f84700acc90 2023-11-15T03:03:06.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.089+0000 7f84577fe700 1 -- 172.21.15.121:0/1979550428 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f846001ba00 con 0x7f84700acc90 2023-11-15T03:03:06.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.089+0000 7f84577fe700 1 --2- 172.21.15.121:0/1979550428 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f845c05a4e0 0x7f845c05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:06.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.089+0000 7f84577fe700 1 -- 172.21.15.121:0/1979550428 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f8460076030 con 0x7f84700acc90 2023-11-15T03:03:06.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.089+0000 7f846effd700 1 --2- 172.21.15.121:0/1979550428 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f845c05a4e0 0x7f845c05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:06.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.093+0000 7f846effd700 1 --2- 172.21.15.121:0/1979550428 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f845c05a4e0 0x7f845c05c9a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f8464006d80 tx=0x7f8464008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:06.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.101+0000 7f84577fe700 1 -- 172.21.15.121:0/1979550428 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8470153550 con 0x7f84700acc90 2023-11-15T03:03:06.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.293+0000 7f8474db9700 1 -- 172.21.15.121:0/1979550428 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8470153550 con 0x7f84700acc90 2023-11-15T03:03:06.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.293+0000 7f84577fe700 1 -- 172.21.15.121:0/1979550428 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4502 (secure 0 0 0) 0x7f8470153550 con 0x7f84700acc90 2023-11-15T03:03:06.306 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:06.306 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":73,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1508681155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1508681155},{"type":"v1","addr":"172.21.15.121:6837","nonce":1508681155}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6385,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3559276534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3559276534},{"type":"v1","addr":"172.21.15.190:6835","nonce":3559276534}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1842535107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1842535107},{"type":"v1","addr":"172.21.15.190:6833","nonce":1842535107}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:02:20.703078+0000","modified":"2023-11-15T03:03:01.009934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":14,"addr":"172.21.15.121:6836/834528709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":834528709},{"type":"v1","addr":"172.21.15.121:6836","nonce":834528709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-15T03:03:06.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.297+0000 7f8474db9700 1 -- 172.21.15.121:0/1979550428 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f845c05a4e0 msgr2=0x7f845c05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:06.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.297+0000 7f8474db9700 1 --2- 172.21.15.121:0/1979550428 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f845c05a4e0 0x7f845c05c9a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f8464006d80 tx=0x7f8464008040).stop 2023-11-15T03:03:06.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.297+0000 7f8474db9700 1 -- 172.21.15.121:0/1979550428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84700acc90 msgr2=0x7f847014ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:06.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.297+0000 7f8474db9700 1 --2- 172.21.15.121:0/1979550428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84700acc90 0x7f847014ad10 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f8460009e00 tx=0x7f8460017000).stop 2023-11-15T03:03:06.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.301+0000 7f8474db9700 1 -- 172.21.15.121:0/1979550428 shutdown_connections 2023-11-15T03:03:06.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.301+0000 7f8474db9700 1 --2- 172.21.15.121:0/1979550428 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f845c05a4e0 0x7f845c05c9a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.301+0000 7f8474db9700 1 --2- 172.21.15.121:0/1979550428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84700cabd0 0x7f8470154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.301+0000 7f8474db9700 1 --2- 172.21.15.121:0/1979550428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84700ad0f0 0x7f84700c1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.301+0000 7f8474db9700 1 --2- 172.21.15.121:0/1979550428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84700acc90 0x7f847014ad10 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.301+0000 7f8474db9700 1 -- 172.21.15.121:0/1979550428 >> 172.21.15.121:0/1979550428 conn(0x7f8470062190 msgr2=0x7f8470150590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:06.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.301+0000 7f8474db9700 1 -- 172.21.15.121:0/1979550428 shutdown_connections 2023-11-15T03:03:06.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.301+0000 7f8474db9700 1 -- 172.21.15.121:0/1979550428 wait complete. 2023-11-15T03:03:06.312 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 73 2023-11-15T03:03:06.330 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:03:06.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.485+0000 7fa922707700 1 Processor -- start 2023-11-15T03:03:06.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.489+0000 7fa922707700 1 -- start start 2023-11-15T03:03:06.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.489+0000 7fa922707700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa91c0cafb0 0x7fa91c066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:06.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.489+0000 7fa922707700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa91c067270 0x7fa91c0ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:06.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.489+0000 7fa922707700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91c0ad3a0 con 0x7fa91c0cabd0 2023-11-15T03:03:06.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.489+0000 7fa922707700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91c0cb410 con 0x7fa91c0cafb0 2023-11-15T03:03:06.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.489+0000 7fa922707700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91c0ad650 con 0x7fa91c067270 2023-11-15T03:03:06.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa920f04700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa91c0cafb0 0x7fa91c066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:06.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa920f04700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa91c0cafb0 0x7fa91c066d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:60936/0 (socket says 172.21.15.121:60936) 2023-11-15T03:03:06.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa920f04700 1 -- 172.21.15.121:0/1550382937 learned_addr learned my addr 172.21.15.121:0/1550382937 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:06.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa921f06700 1 --2- 172.21.15.121:0/1550382937 >> v2:172.21.15.190:3300/0 conn(0x7fa91c067270 0x7fa91c0ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:06.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa921705700 1 --1- 172.21.15.121:0/1550382937 >> v1:172.21.15.190:6789/0 conn(0x7fa91c0cabd0 0x7fa91c0acc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:60690/0 (socket says 172.21.15.121:60690) 2023-11-15T03:03:06.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa920f04700 1 -- 172.21.15.121:0/1550382937 >> v2:172.21.15.190:3300/0 conn(0x7fa91c067270 msgr2=0x7fa91c0ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:06.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa920f04700 1 --2- 172.21.15.121:0/1550382937 >> v2:172.21.15.190:3300/0 conn(0x7fa91c067270 0x7fa91c0ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa920f04700 1 -- 172.21.15.121:0/1550382937 >> v1:172.21.15.190:6789/0 conn(0x7fa91c0cabd0 legacy=0x7fa91c0acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:06.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa920f04700 1 -- 172.21.15.121:0/1550382937 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c065fc0 con 0x7fa91c0cafb0 2023-11-15T03:03:06.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa920f04700 1 --2- 172.21.15.121:0/1550382937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa91c0cafb0 0x7fa91c066d80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7fa91c0c7000 tx=0x7fa90c0090f0).ready entity=mon.0 client_cookie=842f001442a294e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:06.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa913fff700 1 -- 172.21.15.121:0/1550382937 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00b520 con 0x7fa91c0cafb0 2023-11-15T03:03:06.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa913fff700 1 -- 172.21.15.121:0/1550382937 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90c00b680 con 0x7fa91c0cafb0 2023-11-15T03:03:06.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa913fff700 1 -- 172.21.15.121:0/1550382937 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00ba00 con 0x7fa91c0cafb0 2023-11-15T03:03:06.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa922707700 1 -- 172.21.15.121:0/1550382937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa91c0cafb0 msgr2=0x7fa91c066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:06.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa922707700 1 --2- 172.21.15.121:0/1550382937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa91c0cafb0 0x7fa91c066d80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7fa91c0c7000 tx=0x7fa90c0090f0).stop 2023-11-15T03:03:06.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa922707700 1 -- 172.21.15.121:0/1550382937 shutdown_connections 2023-11-15T03:03:06.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa922707700 1 --2- 172.21.15.121:0/1550382937 >> v2:172.21.15.190:3300/0 conn(0x7fa91c067270 0x7fa91c0ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa922707700 1 --2- 172.21.15.121:0/1550382937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa91c0cafb0 0x7fa91c066d80 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa922707700 1 -- 172.21.15.121:0/1550382937 >> 172.21.15.121:0/1550382937 conn(0x7fa91c062190 msgr2=0x7fa91c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:06.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa922707700 1 -- 172.21.15.121:0/1550382937 shutdown_connections 2023-11-15T03:03:06.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.493+0000 7fa922707700 1 -- 172.21.15.121:0/1550382937 wait complete. 2023-11-15T03:03:06.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.497+0000 7fa922707700 1 Processor -- start 2023-11-15T03:03:06.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.497+0000 7fa922707700 1 -- start start 2023-11-15T03:03:06.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.497+0000 7fa922707700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa91c067270 0x7fa91c13edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:06.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.497+0000 7fa922707700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa91c0cabd0 0x7fa91c155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:06.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.497+0000 7fa922707700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa91c0cafb0 0x7fa91c155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:06.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.497+0000 7fa922707700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c156370 con 0x7fa91c067270 2023-11-15T03:03:06.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.497+0000 7fa922707700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c0bfcd0 con 0x7fa91c0cafb0 2023-11-15T03:03:06.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.497+0000 7fa922707700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c0c0290 con 0x7fa91c0cabd0 2023-11-15T03:03:06.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.501+0000 7fa921705700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa91c067270 0x7fa91c13edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:06.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.501+0000 7fa921705700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa91c067270 0x7fa91c13edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:60952/0 (socket says 172.21.15.121:60952) 2023-11-15T03:03:06.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.501+0000 7fa921705700 1 -- 172.21.15.121:0/2847366529 learned_addr learned my addr 172.21.15.121:0/2847366529 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:06.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.501+0000 7fa920f04700 1 --2- 172.21.15.121:0/2847366529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa91c0cabd0 0x7fa91c155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:06.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.501+0000 7fa921f06700 1 --2- 172.21.15.121:0/2847366529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa91c0cafb0 0x7fa91c155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:06.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.501+0000 7fa921705700 1 -- 172.21.15.121:0/2847366529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa91c0cabd0 msgr2=0x7fa91c155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:06.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.501+0000 7fa921705700 1 --2- 172.21.15.121:0/2847366529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa91c0cabd0 0x7fa91c155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.501+0000 7fa921705700 1 -- 172.21.15.121:0/2847366529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa91c0cafb0 msgr2=0x7fa91c155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:06.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.501+0000 7fa921705700 1 --2- 172.21.15.121:0/2847366529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa91c0cafb0 0x7fa91c155e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.501+0000 7fa921705700 1 -- 172.21.15.121:0/2847366529 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c009370 con 0x7fa91c067270 2023-11-15T03:03:06.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.501+0000 7fa921f06700 1 --2- 172.21.15.121:0/2847366529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa91c0cafb0 0x7fa91c155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:06.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.501+0000 7fa920f04700 1 --2- 172.21.15.121:0/2847366529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa91c0cabd0 0x7fa91c155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:06.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.501+0000 7fa921705700 1 --2- 172.21.15.121:0/2847366529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa91c067270 0x7fa91c13edc0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7fa91800c490 tx=0x7fa918007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:06.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.501+0000 7fa9117fa700 1 -- 172.21.15.121:0/2847366529 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9180072e0 con 0x7fa91c067270 2023-11-15T03:03:06.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.501+0000 7fa9117fa700 1 -- 172.21.15.121:0/2847366529 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa918007440 con 0x7fa91c067270 2023-11-15T03:03:06.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.501+0000 7fa922707700 1 -- 172.21.15.121:0/2847366529 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91c156570 con 0x7fa91c067270 2023-11-15T03:03:06.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.501+0000 7fa9117fa700 1 -- 172.21.15.121:0/2847366529 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9180058f0 con 0x7fa91c067270 2023-11-15T03:03:06.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.501+0000 7fa922707700 1 -- 172.21.15.121:0/2847366529 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c0bff20 con 0x7fa91c067270 2023-11-15T03:03:06.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.501+0000 7fa922707700 1 -- 172.21.15.121:0/2847366529 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91c1569f0 con 0x7fa91c067270 2023-11-15T03:03:06.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.505+0000 7fa9117fa700 1 -- 172.21.15.121:0/2847366529 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa918005ce0 con 0x7fa91c067270 2023-11-15T03:03:06.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.505+0000 7fa9117fa700 1 --2- 172.21.15.121:0/2847366529 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa90005a510 0x7fa90005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:06.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.505+0000 7fa9117fa700 1 -- 172.21.15.121:0/2847366529 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fa9180740d0 con 0x7fa91c067270 2023-11-15T03:03:06.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.505+0000 7fa920f04700 1 --2- 172.21.15.121:0/2847366529 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa90005a510 0x7fa90005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:06.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.505+0000 7fa920f04700 1 --2- 172.21.15.121:0/2847366529 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa90005a510 0x7fa90005c9d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fa90c009130 tx=0x7fa90c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:06.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.513+0000 7fa9117fa700 1 -- 172.21.15.121:0/2847366529 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa918047760 con 0x7fa91c067270 2023-11-15T03:03:06.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.673+0000 7fa922707700 1 -- 172.21.15.121:0/2847366529 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa91c06bcf0 con 0x7fa91c067270 2023-11-15T03:03:06.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.677+0000 7fa9117fa700 1 -- 172.21.15.121:0/2847366529 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17206 (secure 0 0 0) 0x7fa91804ad80 con 0x7fa91c067270 2023-11-15T03:03:06.683 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:06.683 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":80,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:02:57.977723+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-15T03:02:18.737496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-15T03:02:19.718114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.38","mappings":[{"from":6,"to":0}]},{"pgid":"11.3a","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:6833/914796342":"2023-11-16T03:02:57.977589+0000","172.21.15.190:6832/914796342":"2023-11-16T03:02:57.977589+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:03:06.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.681+0000 7fa922707700 1 -- 172.21.15.121:0/2847366529 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa90005a510 msgr2=0x7fa90005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:06.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.681+0000 7fa922707700 1 --2- 172.21.15.121:0/2847366529 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa90005a510 0x7fa90005c9d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fa90c009130 tx=0x7fa90c017000).stop 2023-11-15T03:03:06.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.681+0000 7fa922707700 1 -- 172.21.15.121:0/2847366529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa91c067270 msgr2=0x7fa91c13edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:06.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.681+0000 7fa922707700 1 --2- 172.21.15.121:0/2847366529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa91c067270 0x7fa91c13edc0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7fa91800c490 tx=0x7fa918007df0).stop 2023-11-15T03:03:06.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.681+0000 7fa922707700 1 -- 172.21.15.121:0/2847366529 shutdown_connections 2023-11-15T03:03:06.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.681+0000 7fa922707700 1 --2- 172.21.15.121:0/2847366529 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa90005a510 0x7fa90005c9d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.681+0000 7fa922707700 1 --2- 172.21.15.121:0/2847366529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa91c0cafb0 0x7fa91c155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.681+0000 7fa922707700 1 --2- 172.21.15.121:0/2847366529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa91c0cabd0 0x7fa91c155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.681+0000 7fa922707700 1 --2- 172.21.15.121:0/2847366529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa91c067270 0x7fa91c13edc0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.681+0000 7fa922707700 1 -- 172.21.15.121:0/2847366529 >> 172.21.15.121:0/2847366529 conn(0x7fa91c062190 msgr2=0x7fa91c0c5960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:06.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.681+0000 7fa922707700 1 -- 172.21.15.121:0/2847366529 shutdown_connections 2023-11-15T03:03:06.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.681+0000 7fa922707700 1 -- 172.21.15.121:0/2847366529 wait complete. 2023-11-15T03:03:06.705 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-15T03:03:06.706 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:03:06.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.869+0000 7f5e7e7f9700 1 Processor -- start 2023-11-15T03:03:06.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.869+0000 7f5e7e7f9700 1 -- start start 2023-11-15T03:03:06.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.873+0000 7f5e7e7f9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5e780cabd0 0x7f5e780cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:06.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.873+0000 7f5e7e7f9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e780ad650 con 0x7f5e780ad0f0 2023-11-15T03:03:06.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.873+0000 7f5e7e7f9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e78065fc0 con 0x7f5e780acc90 2023-11-15T03:03:06.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.873+0000 7f5e7e7f9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e780662a0 con 0x7f5e780cabd0 2023-11-15T03:03:06.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.873+0000 7f5e7dff8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f5e780ad0f0 0x7f5e780ae190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45744/0 (socket says 172.21.15.121:45744) 2023-11-15T03:03:06.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.873+0000 7f5e7dff8700 1 -- 172.21.15.121:0/2104911560 learned_addr learned my addr 172.21.15.121:0/2104911560 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:06.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.873+0000 7f5e7d7f7700 1 --2- 172.21.15.121:0/2104911560 >> v2:172.21.15.190:3300/0 conn(0x7f5e780cabd0 0x7f5e780cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:06.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.873+0000 7f5e6ffff700 1 -- 172.21.15.121:0/2104911560 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1587979067 0 0) 0x7f5e780ad650 con 0x7f5e780ad0f0 2023-11-15T03:03:06.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.873+0000 7f5e6ffff700 1 -- 172.21.15.121:0/2104911560 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e640033f0 con 0x7f5e780ad0f0 2023-11-15T03:03:06.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.873+0000 7f5e7d7f7700 1 -- 172.21.15.121:0/2104911560 >> v1:172.21.15.190:6789/0 conn(0x7f5e780acc90 legacy=0x7f5e78066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:06.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.873+0000 7f5e6ffff700 1 -- 172.21.15.121:0/2104911560 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2603453262 0 0) 0x7f5e78065fc0 con 0x7f5e780acc90 2023-11-15T03:03:06.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.873+0000 7f5e7d7f7700 1 -- 172.21.15.121:0/2104911560 >> v1:172.21.15.121:6789/0 conn(0x7f5e780ad0f0 legacy=0x7f5e780ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:06.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.873+0000 7f5e7d7f7700 1 -- 172.21.15.121:0/2104911560 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e78066580 con 0x7f5e780cabd0 2023-11-15T03:03:06.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.873+0000 7f5e7d7f7700 1 --2- 172.21.15.121:0/2104911560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e780cabd0 0x7f5e780cafb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f5e74009760 tx=0x7f5e74009060).ready entity=mon.1 client_cookie=9eb7d18b715f632c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:06.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.873+0000 7f5e6ffff700 1 -- 172.21.15.121:0/2104911560 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7400f510 con 0x7f5e780cabd0 2023-11-15T03:03:06.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.877+0000 7f5e6ffff700 1 -- 172.21.15.121:0/2104911560 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7400f670 con 0x7f5e780cabd0 2023-11-15T03:03:06.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.877+0000 7f5e6ffff700 1 -- 172.21.15.121:0/2104911560 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7400f9f0 con 0x7f5e780cabd0 2023-11-15T03:03:06.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.877+0000 7f5e7e7f9700 1 -- 172.21.15.121:0/2104911560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e780cabd0 msgr2=0x7f5e780cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:06.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.877+0000 7f5e7e7f9700 1 --2- 172.21.15.121:0/2104911560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e780cabd0 0x7f5e780cafb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f5e74009760 tx=0x7f5e74009060).stop 2023-11-15T03:03:06.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.877+0000 7f5e7e7f9700 1 -- 172.21.15.121:0/2104911560 shutdown_connections 2023-11-15T03:03:06.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.877+0000 7f5e7e7f9700 1 --2- 172.21.15.121:0/2104911560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e780cabd0 0x7f5e780cafb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.877+0000 7f5e7e7f9700 1 -- 172.21.15.121:0/2104911560 >> 172.21.15.121:0/2104911560 conn(0x7f5e78062190 msgr2=0x7f5e780625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:06.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.877+0000 7f5e7e7f9700 1 -- 172.21.15.121:0/2104911560 shutdown_connections 2023-11-15T03:03:06.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.877+0000 7f5e7e7f9700 1 -- 172.21.15.121:0/2104911560 wait complete. 2023-11-15T03:03:06.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.877+0000 7f5e7e7f9700 1 Processor -- start 2023-11-15T03:03:06.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.877+0000 7f5e7e7f9700 1 -- start start 2023-11-15T03:03:06.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e7e7f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780acc90 0x7f5e78144280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:06.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e7e7f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e780ad0f0 0x7f5e780bfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:06.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e7e7f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e780cabd0 0x7f5e780c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:06.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e7e7f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e78155300 con 0x7f5e780ad0f0 2023-11-15T03:03:06.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e7e7f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e781555e0 con 0x7f5e780cabd0 2023-11-15T03:03:06.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e7e7f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e780c3370 con 0x7f5e780acc90 2023-11-15T03:03:06.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e7cff6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e780ad0f0 0x7f5e780bfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:06.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e7cff6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e780ad0f0 0x7f5e780bfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:60956/0 (socket says 172.21.15.121:60956) 2023-11-15T03:03:06.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e7cff6700 1 -- 172.21.15.121:0/2569706343 learned_addr learned my addr 172.21.15.121:0/2569706343 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:06.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e7d7f7700 1 --2- 172.21.15.121:0/2569706343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780acc90 0x7f5e78144280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:06.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e7dff8700 1 --2- 172.21.15.121:0/2569706343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e780cabd0 0x7f5e780c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:06.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e7cff6700 1 -- 172.21.15.121:0/2569706343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780acc90 msgr2=0x7f5e78144280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:06.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e7cff6700 1 --2- 172.21.15.121:0/2569706343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780acc90 0x7f5e78144280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e7cff6700 1 -- 172.21.15.121:0/2569706343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e780cabd0 msgr2=0x7f5e780c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:06.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e7cff6700 1 --2- 172.21.15.121:0/2569706343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e780cabd0 0x7f5e780c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:06.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e7cff6700 1 -- 172.21.15.121:0/2569706343 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e740092a0 con 0x7f5e780ad0f0 2023-11-15T03:03:06.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e7cff6700 1 --2- 172.21.15.121:0/2569706343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e780ad0f0 0x7f5e780bfac0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7f5e68001eb0 tx=0x7f5e68007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:06.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e6d7fa700 1 -- 172.21.15.121:0/2569706343 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6800ce60 con 0x7f5e780ad0f0 2023-11-15T03:03:06.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e6d7fa700 1 -- 172.21.15.121:0/2569706343 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6800f470 con 0x7f5e780ad0f0 2023-11-15T03:03:06.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e7e7f9700 1 -- 172.21.15.121:0/2569706343 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e780c3650 con 0x7f5e780ad0f0 2023-11-15T03:03:06.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e6d7fa700 1 -- 172.21.15.121:0/2569706343 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6800f810 con 0x7f5e780ad0f0 2023-11-15T03:03:06.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.881+0000 7f5e7e7f9700 1 -- 172.21.15.121:0/2569706343 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e78144770 con 0x7f5e780ad0f0 2023-11-15T03:03:06.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.885+0000 7f5e7e7f9700 1 -- 172.21.15.121:0/2569706343 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e780c31d0 con 0x7f5e780ad0f0 2023-11-15T03:03:06.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.885+0000 7f5e6d7fa700 1 -- 172.21.15.121:0/2569706343 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5e6800fb80 con 0x7f5e780ad0f0 2023-11-15T03:03:06.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.885+0000 7f5e6d7fa700 1 --2- 172.21.15.121:0/2569706343 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5e5c05a510 0x7f5e5c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:06.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.885+0000 7f5e6d7fa700 1 -- 172.21.15.121:0/2569706343 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f5e68073ea0 con 0x7f5e780ad0f0 2023-11-15T03:03:06.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.885+0000 7f5e7d7f7700 1 --2- 172.21.15.121:0/2569706343 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5e5c05a510 0x7f5e5c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:06.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.889+0000 7f5e7d7f7700 1 --2- 172.21.15.121:0/2569706343 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5e5c05a510 0x7f5e5c05c9d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f5e74009760 tx=0x7f5e74017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:06.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:06.893+0000 7f5e6d7fa700 1 -- 172.21.15.121:0/2569706343 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5e780c31d0 con 0x7f5e780ad0f0 2023-11-15T03:03:07.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.093+0000 7f5e7e7f9700 1 -- 172.21.15.121:0/2569706343 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e780c31d0 con 0x7f5e780ad0f0 2023-11-15T03:03:07.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.097+0000 7f5e6d7fa700 1 -- 172.21.15.121:0/2569706343 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4502 (secure 0 0 0) 0x7f5e780c31d0 con 0x7f5e780ad0f0 2023-11-15T03:03:07.102 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:07.102 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":73,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1508681155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1508681155},{"type":"v1","addr":"172.21.15.121:6837","nonce":1508681155}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6385,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3559276534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3559276534},{"type":"v1","addr":"172.21.15.190:6835","nonce":3559276534}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1842535107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1842535107},{"type":"v1","addr":"172.21.15.190:6833","nonce":1842535107}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:02:20.703078+0000","modified":"2023-11-15T03:03:01.009934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":14,"addr":"172.21.15.121:6836/834528709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":834528709},{"type":"v1","addr":"172.21.15.121:6836","nonce":834528709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-15T03:03:07.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.097+0000 7f5e7e7f9700 1 -- 172.21.15.121:0/2569706343 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5e5c05a510 msgr2=0x7f5e5c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:07.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.097+0000 7f5e7e7f9700 1 --2- 172.21.15.121:0/2569706343 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5e5c05a510 0x7f5e5c05c9d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f5e74009760 tx=0x7f5e74017000).stop 2023-11-15T03:03:07.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.097+0000 7f5e7e7f9700 1 -- 172.21.15.121:0/2569706343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e780ad0f0 msgr2=0x7f5e780bfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:07.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.097+0000 7f5e7e7f9700 1 --2- 172.21.15.121:0/2569706343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e780ad0f0 0x7f5e780bfac0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7f5e68001eb0 tx=0x7f5e68007c70).stop 2023-11-15T03:03:07.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.097+0000 7f5e7e7f9700 1 -- 172.21.15.121:0/2569706343 shutdown_connections 2023-11-15T03:03:07.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.097+0000 7f5e7e7f9700 1 --2- 172.21.15.121:0/2569706343 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5e5c05a510 0x7f5e5c05c9d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:07.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.097+0000 7f5e7e7f9700 1 --2- 172.21.15.121:0/2569706343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e780cabd0 0x7f5e780c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:07.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.097+0000 7f5e7e7f9700 1 --2- 172.21.15.121:0/2569706343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e780ad0f0 0x7f5e780bfac0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:07.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.097+0000 7f5e7e7f9700 1 --2- 172.21.15.121:0/2569706343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780acc90 0x7f5e78144280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:07.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.097+0000 7f5e7e7f9700 1 -- 172.21.15.121:0/2569706343 >> 172.21.15.121:0/2569706343 conn(0x7f5e78062190 msgr2=0x7f5e78152710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:07.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.101+0000 7f5e7e7f9700 1 -- 172.21.15.121:0/2569706343 shutdown_connections 2023-11-15T03:03:07.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.101+0000 7f5e7e7f9700 1 -- 172.21.15.121:0/2569706343 wait complete. 2023-11-15T03:03:07.107 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 73 2023-11-15T03:03:07.124 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:03:07.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.277+0000 7f41f4df9700 1 Processor -- start 2023-11-15T03:03:07.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.277+0000 7f41f4df9700 1 -- start start 2023-11-15T03:03:07.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.277+0000 7f41f4df9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f41f00c37f0 0x7f41f00a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:07.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.277+0000 7f41f4df9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f00657f0 con 0x7f41f0151870 2023-11-15T03:03:07.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.277+0000 7f41f4df9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f0065ad0 con 0x7f41f00a9830 2023-11-15T03:03:07.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.277+0000 7f41f4df9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f00aa260 con 0x7f41f00c37f0 2023-11-15T03:03:07.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.277+0000 7f41ef7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f41f0151870 0x7f41f0151c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45750/0 (socket says 172.21.15.121:45750) 2023-11-15T03:03:07.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.277+0000 7f41ef7fe700 1 -- 172.21.15.121:0/3630321736 learned_addr learned my addr 172.21.15.121:0/3630321736 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:07.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.281+0000 7f41effff700 1 --2- 172.21.15.121:0/3630321736 >> v2:172.21.15.190:3300/0 conn(0x7f41f00c37f0 0x7f41f00a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:07.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.281+0000 7f41eeffd700 1 -- 172.21.15.121:0/3630321736 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000718113 0 0) 0x7f41f00657f0 con 0x7f41f0151870 2023-11-15T03:03:07.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.281+0000 7f41eeffd700 1 -- 172.21.15.121:0/3630321736 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41dc0033f0 con 0x7f41f0151870 2023-11-15T03:03:07.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.281+0000 7f41eeffd700 1 -- 172.21.15.121:0/3630321736 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1111103428 0 0) 0x7f41dc0033f0 con 0x7f41f0151870 2023-11-15T03:03:07.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.281+0000 7f41effff700 1 -- 172.21.15.121:0/3630321736 >> v1:172.21.15.190:6789/0 conn(0x7f41f00a9830 legacy=0x7f41f00c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:07.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.281+0000 7f41effff700 1 -- 172.21.15.121:0/3630321736 >> v1:172.21.15.121:6789/0 conn(0x7f41f0151870 legacy=0x7f41f0151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:07.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.281+0000 7f41effff700 1 -- 172.21.15.121:0/3630321736 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f00aa540 con 0x7f41f00c37f0 2023-11-15T03:03:07.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.281+0000 7f41effff700 1 --2- 172.21.15.121:0/3630321736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41f00c37f0 0x7f41f00a9c70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f41e8009760 tx=0x7f41e8009060).ready entity=mon.1 client_cookie=d32e594dc31eb43 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:07.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.281+0000 7f41eeffd700 1 -- 172.21.15.121:0/3630321736 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0001470 con 0x7f41f00c37f0 2023-11-15T03:03:07.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.281+0000 7f41eeffd700 1 -- 172.21.15.121:0/3630321736 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e8011400 con 0x7f41f00c37f0 2023-11-15T03:03:07.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.281+0000 7f41eeffd700 1 -- 172.21.15.121:0/3630321736 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e8011780 con 0x7f41f00c37f0 2023-11-15T03:03:07.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.281+0000 7f41f4df9700 1 -- 172.21.15.121:0/3630321736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41f00c37f0 msgr2=0x7f41f00a9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:07.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.281+0000 7f41f4df9700 1 --2- 172.21.15.121:0/3630321736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41f00c37f0 0x7f41f00a9c70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f41e8009760 tx=0x7f41e8009060).stop 2023-11-15T03:03:07.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.281+0000 7f41f4df9700 1 -- 172.21.15.121:0/3630321736 shutdown_connections 2023-11-15T03:03:07.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.281+0000 7f41f4df9700 1 --2- 172.21.15.121:0/3630321736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41f00c37f0 0x7f41f00a9c70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:07.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.281+0000 7f41f4df9700 1 -- 172.21.15.121:0/3630321736 >> 172.21.15.121:0/3630321736 conn(0x7f41f0062190 msgr2=0x7f41f0062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:07.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.281+0000 7f41f4df9700 1 -- 172.21.15.121:0/3630321736 shutdown_connections 2023-11-15T03:03:07.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.281+0000 7f41f4df9700 1 -- 172.21.15.121:0/3630321736 wait complete. 2023-11-15T03:03:07.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.285+0000 7f41f4df9700 1 Processor -- start 2023-11-15T03:03:07.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.285+0000 7f41f4df9700 1 -- start start 2023-11-15T03:03:07.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.285+0000 7f41f4df9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41f00a9830 0x7f41f014ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:07.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.285+0000 7f41f4df9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41f00c37f0 0x7f41f014ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:07.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.285+0000 7f41f4df9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41f0151870 0x7f41f014e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:07.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.285+0000 7f41f4df9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f014e7b0 con 0x7f41f00c37f0 2023-11-15T03:03:07.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.285+0000 7f41f4df9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f00c0320 con 0x7f41f00a9830 2023-11-15T03:03:07.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.285+0000 7f41f4df9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f00bf710 con 0x7f41f0151870 2023-11-15T03:03:07.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.285+0000 7f41e6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41f00c37f0 0x7f41f014ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:07.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.289+0000 7f41e6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41f00c37f0 0x7f41f014ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:60962/0 (socket says 172.21.15.121:60962) 2023-11-15T03:03:07.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.289+0000 7f41e6ffd700 1 -- 172.21.15.121:0/460767858 learned_addr learned my addr 172.21.15.121:0/460767858 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:07.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.289+0000 7f41ef7fe700 1 --2- 172.21.15.121:0/460767858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41f00a9830 0x7f41f014ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:07.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.289+0000 7f41effff700 1 --2- 172.21.15.121:0/460767858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41f0151870 0x7f41f014e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:07.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.289+0000 7f41e6ffd700 1 -- 172.21.15.121:0/460767858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41f0151870 msgr2=0x7f41f014e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:07.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.289+0000 7f41e6ffd700 1 --2- 172.21.15.121:0/460767858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41f0151870 0x7f41f014e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:07.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.289+0000 7f41e6ffd700 1 -- 172.21.15.121:0/460767858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41f00a9830 msgr2=0x7f41f014ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:07.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.289+0000 7f41e6ffd700 1 --2- 172.21.15.121:0/460767858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41f00a9830 0x7f41f014ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:07.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.289+0000 7f41e6ffd700 1 -- 172.21.15.121:0/460767858 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e80092a0 con 0x7f41f00c37f0 2023-11-15T03:03:07.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.289+0000 7f41effff700 1 --2- 172.21.15.121:0/460767858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41f0151870 0x7f41f014e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:07.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.289+0000 7f41e6ffd700 1 --2- 172.21.15.121:0/460767858 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41f00c37f0 0x7f41f014ddd0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7f41d800c840 tx=0x7f41d8001660).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:07.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.289+0000 7f41e7fff700 1 -- 172.21.15.121:0/460767858 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d8006a20 con 0x7f41f00c37f0 2023-11-15T03:03:07.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.289+0000 7f41e7fff700 1 -- 172.21.15.121:0/460767858 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d8006b80 con 0x7f41f00c37f0 2023-11-15T03:03:07.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.289+0000 7f41f4df9700 1 -- 172.21.15.121:0/460767858 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f00bf9f0 con 0x7f41f00c37f0 2023-11-15T03:03:07.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.289+0000 7f41e7fff700 1 -- 172.21.15.121:0/460767858 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d8007020 con 0x7f41f00c37f0 2023-11-15T03:03:07.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.289+0000 7f41f4df9700 1 -- 172.21.15.121:0/460767858 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f00bffb0 con 0x7f41f00c37f0 2023-11-15T03:03:07.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.289+0000 7f41f4df9700 1 -- 172.21.15.121:0/460767858 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f00c04d0 con 0x7f41f00c37f0 2023-11-15T03:03:07.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.293+0000 7f41e7fff700 1 -- 172.21.15.121:0/460767858 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f41d80073d0 con 0x7f41f00c37f0 2023-11-15T03:03:07.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.293+0000 7f41e7fff700 1 --2- 172.21.15.121:0/460767858 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41d005a510 0x7f41d005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:07.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.293+0000 7f41e7fff700 1 -- 172.21.15.121:0/460767858 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f41d80740d0 con 0x7f41f00c37f0 2023-11-15T03:03:07.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.293+0000 7f41ef7fe700 1 --2- 172.21.15.121:0/460767858 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41d005a510 0x7f41d005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:07.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.293+0000 7f41ef7fe700 1 --2- 172.21.15.121:0/460767858 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41d005a510 0x7f41d005c9d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f41e0000c00 tx=0x7f41e0001660).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:07.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.297+0000 7f41e7fff700 1 -- 172.21.15.121:0/460767858 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f41f00c04d0 con 0x7f41f00c37f0 2023-11-15T03:03:07.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.497+0000 7f41f4df9700 1 -- 172.21.15.121:0/460767858 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41f00c0490 con 0x7f41f00c37f0 2023-11-15T03:03:07.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.501+0000 7f41e7fff700 1 -- 172.21.15.121:0/460767858 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4502 (secure 0 0 0) 0x7f41d8047760 con 0x7f41f00c37f0 2023-11-15T03:03:07.505 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:07.505 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":73,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1508681155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1508681155},{"type":"v1","addr":"172.21.15.121:6837","nonce":1508681155}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6385,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3559276534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3559276534},{"type":"v1","addr":"172.21.15.190:6835","nonce":3559276534}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1842535107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1842535107},{"type":"v1","addr":"172.21.15.190:6833","nonce":1842535107}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:02:20.703078+0000","modified":"2023-11-15T03:03:01.009934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":14,"addr":"172.21.15.121:6836/834528709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":834528709},{"type":"v1","addr":"172.21.15.121:6836","nonce":834528709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-15T03:03:07.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.501+0000 7f41f4df9700 1 -- 172.21.15.121:0/460767858 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41d005a510 msgr2=0x7f41d005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:07.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.501+0000 7f41f4df9700 1 --2- 172.21.15.121:0/460767858 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41d005a510 0x7f41d005c9d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f41e0000c00 tx=0x7f41e0001660).stop 2023-11-15T03:03:07.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.501+0000 7f41f4df9700 1 -- 172.21.15.121:0/460767858 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41f00c37f0 msgr2=0x7f41f014ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:07.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.501+0000 7f41f4df9700 1 --2- 172.21.15.121:0/460767858 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41f00c37f0 0x7f41f014ddd0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7f41d800c840 tx=0x7f41d8001660).stop 2023-11-15T03:03:07.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.501+0000 7f41f4df9700 1 -- 172.21.15.121:0/460767858 shutdown_connections 2023-11-15T03:03:07.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.501+0000 7f41f4df9700 1 --2- 172.21.15.121:0/460767858 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41d005a510 0x7f41d005c9d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:07.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.501+0000 7f41f4df9700 1 --2- 172.21.15.121:0/460767858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41f0151870 0x7f41f014e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:07.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.501+0000 7f41f4df9700 1 --2- 172.21.15.121:0/460767858 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41f00c37f0 0x7f41f014ddd0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:07.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.501+0000 7f41f4df9700 1 --2- 172.21.15.121:0/460767858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41f00a9830 0x7f41f014ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:07.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.501+0000 7f41f4df9700 1 -- 172.21.15.121:0/460767858 >> 172.21.15.121:0/460767858 conn(0x7f41f0062190 msgr2=0x7f41f0150960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:07.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.501+0000 7f41f4df9700 1 -- 172.21.15.121:0/460767858 shutdown_connections 2023-11-15T03:03:07.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.501+0000 7f41f4df9700 1 -- 172.21.15.121:0/460767858 wait complete. 2023-11-15T03:03:07.513 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 73 2023-11-15T03:03:07.530 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:03:07.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.689+0000 7f771f0b9700 1 Processor -- start 2023-11-15T03:03:07.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.689+0000 7f771f0b9700 1 -- start start 2023-11-15T03:03:07.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.689+0000 7f771f0b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7718152850 0x7f77180c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:07.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.689+0000 7f771f0b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77180c1d50 con 0x7f771815be30 2023-11-15T03:03:07.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.689+0000 7f771f0b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f771815cb80 con 0x7f77180c3280 2023-11-15T03:03:07.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.689+0000 7f771f0b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77180c1860 con 0x7f7718152850 2023-11-15T03:03:07.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.689+0000 7f771e0b7700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f77180c3280 0x7f7718152140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:60726/0 (socket says 172.21.15.121:60726) 2023-11-15T03:03:07.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771e0b7700 1 -- 172.21.15.121:0/3895191186 learned_addr learned my addr 172.21.15.121:0/3895191186 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:07.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.689+0000 7f771e8b8700 1 --1- 172.21.15.121:0/3895191186 >> v1:172.21.15.121:6789/0 conn(0x7f771815be30 0x7f771815c280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45756/0 (socket says 172.21.15.121:45756) 2023-11-15T03:03:07.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771d8b6700 1 --2- 172.21.15.121:0/3895191186 >> v2:172.21.15.190:3300/0 conn(0x7f7718152850 0x7f77180c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:07.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771d0b5700 1 -- 172.21.15.121:0/3895191186 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144050632 0 0) 0x7f77180c1d50 con 0x7f771815be30 2023-11-15T03:03:07.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771d0b5700 1 -- 172.21.15.121:0/3895191186 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77040033f0 con 0x7f771815be30 2023-11-15T03:03:07.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771d0b5700 1 -- 172.21.15.121:0/3895191186 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22302209 0 0) 0x7f771815cb80 con 0x7f77180c3280 2023-11-15T03:03:07.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771d0b5700 1 -- 172.21.15.121:0/3895191186 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77180c1d50 con 0x7f77180c3280 2023-11-15T03:03:07.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771d8b6700 1 -- 172.21.15.121:0/3895191186 >> v1:172.21.15.190:6789/0 conn(0x7f77180c3280 legacy=0x7f7718152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:07.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771d8b6700 1 -- 172.21.15.121:0/3895191186 >> v1:172.21.15.121:6789/0 conn(0x7f771815be30 legacy=0x7f771815c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:07.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771d8b6700 1 -- 172.21.15.121:0/3895191186 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77180a9240 con 0x7f7718152850 2023-11-15T03:03:07.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771d8b6700 1 --2- 172.21.15.121:0/3895191186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7718152850 0x7f77180c3660 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f7708009760 tx=0x7f7708009060).ready entity=mon.1 client_cookie=2944f345b66b9038 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:07.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771d0b5700 1 -- 172.21.15.121:0/3895191186 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770800d4a0 con 0x7f7718152850 2023-11-15T03:03:07.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771d0b5700 1 -- 172.21.15.121:0/3895191186 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770800d600 con 0x7f7718152850 2023-11-15T03:03:07.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771d0b5700 1 -- 172.21.15.121:0/3895191186 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770800d980 con 0x7f7718152850 2023-11-15T03:03:07.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771f0b9700 1 -- 172.21.15.121:0/3895191186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7718152850 msgr2=0x7f77180c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:07.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771f0b9700 1 --2- 172.21.15.121:0/3895191186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7718152850 0x7f77180c3660 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f7708009760 tx=0x7f7708009060).stop 2023-11-15T03:03:07.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771f0b9700 1 -- 172.21.15.121:0/3895191186 shutdown_connections 2023-11-15T03:03:07.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771f0b9700 1 --2- 172.21.15.121:0/3895191186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7718152850 0x7f77180c3660 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:07.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771f0b9700 1 -- 172.21.15.121:0/3895191186 >> 172.21.15.121:0/3895191186 conn(0x7f7718062190 msgr2=0x7f7718062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:07.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771f0b9700 1 -- 172.21.15.121:0/3895191186 shutdown_connections 2023-11-15T03:03:07.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.693+0000 7f771f0b9700 1 -- 172.21.15.121:0/3895191186 wait complete. 2023-11-15T03:03:07.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.697+0000 7f771f0b9700 1 Processor -- start 2023-11-15T03:03:07.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.697+0000 7f771f0b9700 1 -- start start 2023-11-15T03:03:07.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.697+0000 7f771f0b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77180c3280 0x7f7718151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:07.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.697+0000 7f771f0b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7718152850 0x7f77180bc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:07.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.697+0000 7f771f0b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f771815be30 0x7f77180bcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:07.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.697+0000 7f771f0b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77180bcfe0 con 0x7f7718152850 2023-11-15T03:03:07.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.697+0000 7f771f0b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77180c44e0 con 0x7f771815be30 2023-11-15T03:03:07.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.697+0000 7f771f0b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77180c47c0 con 0x7f77180c3280 2023-11-15T03:03:07.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.697+0000 7f771d8b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7718152850 0x7f77180bc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:07.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.697+0000 7f771d8b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7718152850 0x7f77180bc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:60968/0 (socket says 172.21.15.121:60968) 2023-11-15T03:03:07.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.697+0000 7f771d8b6700 1 -- 172.21.15.121:0/3811147329 learned_addr learned my addr 172.21.15.121:0/3811147329 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:07.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.697+0000 7f771e8b8700 1 --2- 172.21.15.121:0/3811147329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f771815be30 0x7f77180bcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:07.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.697+0000 7f771e0b7700 1 --2- 172.21.15.121:0/3811147329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77180c3280 0x7f7718151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:07.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.697+0000 7f771d8b6700 1 -- 172.21.15.121:0/3811147329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77180c3280 msgr2=0x7f7718151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:07.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.697+0000 7f771d8b6700 1 --2- 172.21.15.121:0/3811147329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77180c3280 0x7f7718151230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:07.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.697+0000 7f771d8b6700 1 -- 172.21.15.121:0/3811147329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f771815be30 msgr2=0x7f77180bcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:07.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.697+0000 7f771d8b6700 1 --2- 172.21.15.121:0/3811147329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f771815be30 0x7f77180bcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:07.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.697+0000 7f771d8b6700 1 -- 172.21.15.121:0/3811147329 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77080092a0 con 0x7f7718152850 2023-11-15T03:03:07.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.697+0000 7f771e8b8700 1 --2- 172.21.15.121:0/3811147329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f771815be30 0x7f77180bcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:07.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.701+0000 7f771d8b6700 1 --2- 172.21.15.121:0/3811147329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7718152850 0x7f77180bc600 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7f77180654e0 tx=0x7f7708016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:07.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.701+0000 7f770e7fc700 1 -- 172.21.15.121:0/3811147329 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770800d770 con 0x7f7718152850 2023-11-15T03:03:07.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.701+0000 7f770e7fc700 1 -- 172.21.15.121:0/3811147329 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770800d8d0 con 0x7f7718152850 2023-11-15T03:03:07.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.701+0000 7f770e7fc700 1 -- 172.21.15.121:0/3811147329 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770801d660 con 0x7f7718152850 2023-11-15T03:03:07.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.701+0000 7f771f0b9700 1 -- 172.21.15.121:0/3811147329 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77180c4aa0 con 0x7f7718152850 2023-11-15T03:03:07.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.701+0000 7f771f0b9700 1 -- 172.21.15.121:0/3811147329 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77180c5060 con 0x7f7718152850 2023-11-15T03:03:07.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.701+0000 7f771f0b9700 1 -- 172.21.15.121:0/3811147329 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f771806bcf0 con 0x7f7718152850 2023-11-15T03:03:07.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.701+0000 7f770e7fc700 1 -- 172.21.15.121:0/3811147329 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f770801da50 con 0x7f7718152850 2023-11-15T03:03:07.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.705+0000 7f770e7fc700 1 --2- 172.21.15.121:0/3811147329 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f76fc05a560 0x7f76fc05ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:07.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.705+0000 7f770e7fc700 1 -- 172.21.15.121:0/3811147329 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f7708076060 con 0x7f7718152850 2023-11-15T03:03:07.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.705+0000 7f771e0b7700 1 --2- 172.21.15.121:0/3811147329 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f76fc05a560 0x7f76fc05ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:07.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.705+0000 7f771e0b7700 1 --2- 172.21.15.121:0/3811147329 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f76fc05a560 0x7f76fc05ca20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f7714005750 tx=0x7f7714009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:07.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.709+0000 7f770e7fc700 1 -- 172.21.15.121:0/3811147329 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f77080a4af0 con 0x7f7718152850 2023-11-15T03:03:07.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.873+0000 7f771f0b9700 1 -- 172.21.15.121:0/3811147329 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77180c1f80 con 0x7f7718152850 2023-11-15T03:03:07.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.873+0000 7f770e7fc700 1 -- 172.21.15.121:0/3811147329 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17206 (secure 0 0 0) 0x7f7708049770 con 0x7f7718152850 2023-11-15T03:03:07.880 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:07.881 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":80,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:02:57.977723+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-15T03:02:18.737496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-15T03:02:19.718114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.38","mappings":[{"from":6,"to":0}]},{"pgid":"11.3a","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:6833/914796342":"2023-11-16T03:02:57.977589+0000","172.21.15.190:6832/914796342":"2023-11-16T03:02:57.977589+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:03:07.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.877+0000 7f771f0b9700 1 -- 172.21.15.121:0/3811147329 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f76fc05a560 msgr2=0x7f76fc05ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:07.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.877+0000 7f771f0b9700 1 --2- 172.21.15.121:0/3811147329 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f76fc05a560 0x7f76fc05ca20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f7714005750 tx=0x7f7714009040).stop 2023-11-15T03:03:07.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.877+0000 7f771f0b9700 1 -- 172.21.15.121:0/3811147329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7718152850 msgr2=0x7f77180bc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:07.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.877+0000 7f771f0b9700 1 --2- 172.21.15.121:0/3811147329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7718152850 0x7f77180bc600 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7f77180654e0 tx=0x7f7708016000).stop 2023-11-15T03:03:07.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.877+0000 7f771f0b9700 1 -- 172.21.15.121:0/3811147329 shutdown_connections 2023-11-15T03:03:07.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.877+0000 7f771f0b9700 1 --2- 172.21.15.121:0/3811147329 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f76fc05a560 0x7f76fc05ca20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:07.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.877+0000 7f771f0b9700 1 --2- 172.21.15.121:0/3811147329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f771815be30 0x7f77180bcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:07.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.877+0000 7f771f0b9700 1 --2- 172.21.15.121:0/3811147329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7718152850 0x7f77180bc600 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:07.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.877+0000 7f771f0b9700 1 --2- 172.21.15.121:0/3811147329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77180c3280 0x7f7718151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:07.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.877+0000 7f771f0b9700 1 -- 172.21.15.121:0/3811147329 >> 172.21.15.121:0/3811147329 conn(0x7f7718062190 msgr2=0x7f7718062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:07.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.877+0000 7f771f0b9700 1 -- 172.21.15.121:0/3811147329 shutdown_connections 2023-11-15T03:03:07.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:07.877+0000 7f771f0b9700 1 -- 172.21.15.121:0/3811147329 wait complete. 2023-11-15T03:03:07.904 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:03:08.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.065+0000 7f9199ba9700 1 Processor -- start 2023-11-15T03:03:08.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.065+0000 7f9199ba9700 1 -- start start 2023-11-15T03:03:08.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.069+0000 7f9199ba9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f91940c3b30 0x7f91940bdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:08.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.069+0000 7f9199ba9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91940c4990 0x7f919415a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:08.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.069+0000 7f9199ba9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9194150e80 con 0x7f91940be1f0 2023-11-15T03:03:08.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.069+0000 7f9199ba9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91940be630 con 0x7f91940c3b30 2023-11-15T03:03:08.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.069+0000 7f9199ba9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9194062a80 con 0x7f91940c4990 2023-11-15T03:03:08.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.069+0000 7f91937fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f91940c3b30 0x7f91940bdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:08.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.069+0000 7f91937fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f91940c3b30 0x7f91940bdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:60972/0 (socket says 172.21.15.121:60972) 2023-11-15T03:03:08.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.069+0000 7f91937fe700 1 -- 172.21.15.121:0/3420132900 learned_addr learned my addr 172.21.15.121:0/3420132900 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:08.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.069+0000 7f9193fff700 1 --2- 172.21.15.121:0/3420132900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91940c4990 0x7f919415a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:08.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.069+0000 7f91937fe700 1 -- 172.21.15.121:0/3420132900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91940c4990 msgr2=0x7f919415a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:08.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.069+0000 7f91937fe700 1 --2- 172.21.15.121:0/3420132900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91940c4990 0x7f919415a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:08.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.069+0000 7f91937fe700 1 -- 172.21.15.121:0/3420132900 >> v1:172.21.15.121:6789/0 conn(0x7f91940be1f0 legacy=0x7f91940c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:08.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.069+0000 7f91937fe700 1 -- 172.21.15.121:0/3420132900 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9194062d60 con 0x7f91940c3b30 2023-11-15T03:03:08.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.069+0000 7f91927fc700 1 -- 172.21.15.121:0/3420132900 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003134237 0 0) 0x7f9188001110 con 0x7f91940be1f0 2023-11-15T03:03:08.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.069+0000 7f91937fe700 1 --2- 172.21.15.121:0/3420132900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91940c3b30 0x7f91940bdd00 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f9184006fd0 tx=0x7f918400b3b0).ready entity=mon.0 client_cookie=8f07ab7a44faa5d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:08.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.069+0000 7f91927fc700 1 -- 172.21.15.121:0/3420132900 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918400db40 con 0x7f91940c3b30 2023-11-15T03:03:08.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.069+0000 7f91927fc700 1 -- 172.21.15.121:0/3420132900 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918400dca0 con 0x7f91940c3b30 2023-11-15T03:03:08.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.069+0000 7f91927fc700 1 -- 172.21.15.121:0/3420132900 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9184010680 con 0x7f91940c3b30 2023-11-15T03:03:08.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.073+0000 7f9199ba9700 1 -- 172.21.15.121:0/3420132900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91940c3b30 msgr2=0x7f91940bdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:08.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.073+0000 7f9199ba9700 1 --2- 172.21.15.121:0/3420132900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91940c3b30 0x7f91940bdd00 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f9184006fd0 tx=0x7f918400b3b0).stop 2023-11-15T03:03:08.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.073+0000 7f9199ba9700 1 -- 172.21.15.121:0/3420132900 shutdown_connections 2023-11-15T03:03:08.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.073+0000 7f9199ba9700 1 --2- 172.21.15.121:0/3420132900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91940c4990 0x7f919415a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:08.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.073+0000 7f9199ba9700 1 --2- 172.21.15.121:0/3420132900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91940c3b30 0x7f91940bdd00 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:08.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.073+0000 7f9199ba9700 1 -- 172.21.15.121:0/3420132900 >> 172.21.15.121:0/3420132900 conn(0x7f9194068b70 msgr2=0x7f9194068f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:08.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.073+0000 7f9199ba9700 1 -- 172.21.15.121:0/3420132900 shutdown_connections 2023-11-15T03:03:08.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.073+0000 7f9199ba9700 1 -- 172.21.15.121:0/3420132900 wait complete. 2023-11-15T03:03:08.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.073+0000 7f9199ba9700 1 Processor -- start 2023-11-15T03:03:08.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.073+0000 7f9199ba9700 1 -- start start 2023-11-15T03:03:08.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f9199ba9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91940be1f0 0x7f9194164e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:08.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f9199ba9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91940c3b30 0x7f9194165540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:08.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f9199ba9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91940c4990 0x7f91940c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:08.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f9199ba9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91940a4f40 con 0x7f91940be1f0 2023-11-15T03:03:08.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f9199ba9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91940a5500 con 0x7f91940c4990 2023-11-15T03:03:08.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f9199ba9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9194165350 con 0x7f91940c3b30 2023-11-15T03:03:08.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f91937fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91940be1f0 0x7f9194164e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:08.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f91937fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91940be1f0 0x7f9194164e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:60986/0 (socket says 172.21.15.121:60986) 2023-11-15T03:03:08.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f91937fe700 1 -- 172.21.15.121:0/592402362 learned_addr learned my addr 172.21.15.121:0/592402362 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:08.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f91937fe700 1 -- 172.21.15.121:0/592402362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91940c3b30 msgr2=0x7f9194165540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:08.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f91937fe700 1 --2- 172.21.15.121:0/592402362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91940c3b30 0x7f9194165540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:03:08.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f91937fe700 1 -- 172.21.15.121:0/592402362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91940c4990 msgr2=0x7f91940c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:08.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f91937fe700 1 --2- 172.21.15.121:0/592402362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91940c4990 0x7f91940c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:03:08.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f91937fe700 1 -- 172.21.15.121:0/592402362 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918400b610 con 0x7f91940be1f0 2023-11-15T03:03:08.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f91937fe700 1 --2- 172.21.15.121:0/592402362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91940be1f0 0x7f9194164e60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7f9184006e10 tx=0x7f918400b3b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:08.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f917b7fe700 1 -- 172.21.15.121:0/592402362 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9184010640 con 0x7f91940be1f0 2023-11-15T03:03:08.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f917b7fe700 1 -- 172.21.15.121:0/592402362 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9184019660 con 0x7f91940be1f0 2023-11-15T03:03:08.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f9199ba9700 1 -- 172.21.15.121:0/592402362 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9194153890 con 0x7f91940be1f0 2023-11-15T03:03:08.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f917b7fe700 1 -- 172.21.15.121:0/592402362 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9184019a00 con 0x7f91940be1f0 2023-11-15T03:03:08.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.077+0000 7f9199ba9700 1 -- 172.21.15.121:0/592402362 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91940a5190 con 0x7f91940be1f0 2023-11-15T03:03:08.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.081+0000 7f9199ba9700 1 -- 172.21.15.121:0/592402362 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9194047060 con 0x7f91940be1f0 2023-11-15T03:03:08.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.081+0000 7f917b7fe700 1 -- 172.21.15.121:0/592402362 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f918400f3d0 con 0x7f91940be1f0 2023-11-15T03:03:08.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.081+0000 7f917b7fe700 1 --2- 172.21.15.121:0/592402362 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f918005a510 0x7f918005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:08.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.085+0000 7f917b7fe700 1 -- 172.21.15.121:0/592402362 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f9184076440 con 0x7f91940be1f0 2023-11-15T03:03:08.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.085+0000 7f9192ffd700 1 --2- 172.21.15.121:0/592402362 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f918005a510 0x7f918005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:08.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.085+0000 7f9192ffd700 1 --2- 172.21.15.121:0/592402362 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f918005a510 0x7f918005c9d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f9188007fd0 tx=0x7f9188007930).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:08.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.089+0000 7f917b7fe700 1 -- 172.21.15.121:0/592402362 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9184049b20 con 0x7f91940be1f0 2023-11-15T03:03:08.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:08.285+0000 7f9199ba9700 1 -- 172.21.15.121:0/592402362 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f91940c1dd0 con 0x7f91940be1f0 2023-11-15T03:03:09.205 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:03:09.201+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:03:09.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.217+0000 7f917b7fe700 1 -- 172.21.15.121:0/592402362 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9184019db0 con 0x7f91940be1f0 2023-11-15T03:03:09.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.221+0000 7f9199ba9700 1 -- 172.21.15.121:0/592402362 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f918005a510 msgr2=0x7f918005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:09.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.221+0000 7f9199ba9700 1 --2- 172.21.15.121:0/592402362 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f918005a510 0x7f918005c9d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f9188007fd0 tx=0x7f9188007930).stop 2023-11-15T03:03:09.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.221+0000 7f9199ba9700 1 -- 172.21.15.121:0/592402362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91940be1f0 msgr2=0x7f9194164e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:09.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.221+0000 7f9199ba9700 1 --2- 172.21.15.121:0/592402362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91940be1f0 0x7f9194164e60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7f9184006e10 tx=0x7f918400b3b0).stop 2023-11-15T03:03:09.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.221+0000 7f9199ba9700 1 -- 172.21.15.121:0/592402362 shutdown_connections 2023-11-15T03:03:09.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.221+0000 7f9199ba9700 1 --2- 172.21.15.121:0/592402362 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f918005a510 0x7f918005c9d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:09.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.221+0000 7f9199ba9700 1 --2- 172.21.15.121:0/592402362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91940c4990 0x7f91940c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:03:09.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.221+0000 7f9199ba9700 1 --2- 172.21.15.121:0/592402362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91940c3b30 0x7f9194165540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:03:09.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.221+0000 7f9199ba9700 1 --2- 172.21.15.121:0/592402362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91940be1f0 0x7f9194164e60 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:09.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.221+0000 7f9199ba9700 1 -- 172.21.15.121:0/592402362 >> 172.21.15.121:0/592402362 conn(0x7f9194068b70 msgr2=0x7f91940acd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:09.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.221+0000 7f9199ba9700 1 -- 172.21.15.121:0/592402362 shutdown_connections 2023-11-15T03:03:09.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.221+0000 7f9199ba9700 1 -- 172.21.15.121:0/592402362 wait complete. 2023-11-15T03:03:09.231 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:03:09.239 INFO:tasks.ceph.mds.a.smithi121.stderr: -1376> 2023-11-15T03:03:01.217+0000 7f5f09bf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:03:09.258 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-15T03:03:09.275 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:03:09.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.413+0000 7f3d752db700 1 Processor -- start 2023-11-15T03:03:09.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.413+0000 7f3d752db700 1 -- start start 2023-11-15T03:03:09.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.417+0000 7f3d752db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3d70157ed0 0x7f3d701582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:09.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.417+0000 7f3d752db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d70067020 0x7f3d700bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:09.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.417+0000 7f3d752db700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d70062da0 con 0x7f3d70067a60 2023-11-15T03:03:09.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.417+0000 7f3d752db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d700a93a0 con 0x7f3d70157ed0 2023-11-15T03:03:09.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.417+0000 7f3d752db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d70062190 con 0x7f3d70067020 2023-11-15T03:03:09.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.417+0000 7f3d6f7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d70067020 0x7f3d700bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:09.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.417+0000 7f3d6e7fc700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f3d70067a60 0x7f3d70067e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:60734/0 (socket says 172.21.15.121:60734) 2023-11-15T03:03:09.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.417+0000 7f3d6e7fc700 1 -- 172.21.15.121:0/2110665769 learned_addr learned my addr 172.21.15.121:0/2110665769 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:09.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.417+0000 7f3d6effd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3d70157ed0 0x7f3d701582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:09.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.417+0000 7f3d6dffb700 1 -- 172.21.15.121:0/2110665769 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 729763523 0 0) 0x7f3d70062da0 con 0x7f3d70067a60 2023-11-15T03:03:09.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.417+0000 7f3d6dffb700 1 -- 172.21.15.121:0/2110665769 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d580033f0 con 0x7f3d70067a60 2023-11-15T03:03:09.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.417+0000 7f3d6effd700 1 -- 172.21.15.121:0/2110665769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d70067020 msgr2=0x7f3d700bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:09.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.417+0000 7f3d6effd700 1 --2- 172.21.15.121:0/2110665769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d70067020 0x7f3d700bc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:09.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.417+0000 7f3d6effd700 1 -- 172.21.15.121:0/2110665769 >> v1:172.21.15.190:6789/0 conn(0x7f3d70067a60 legacy=0x7f3d70067e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:09.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.417+0000 7f3d6effd700 1 -- 172.21.15.121:0/2110665769 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d70062470 con 0x7f3d70157ed0 2023-11-15T03:03:09.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.417+0000 7f3d6effd700 1 --2- 172.21.15.121:0/2110665769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d70157ed0 0x7f3d701582b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f3d60009760 tx=0x7f3d60009060).ready entity=mon.1 client_cookie=62214389632e4d80 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:09.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.421+0000 7f3d6dffb700 1 -- 172.21.15.121:0/2110665769 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6000d4a0 con 0x7f3d70157ed0 2023-11-15T03:03:09.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.421+0000 7f3d6dffb700 1 -- 172.21.15.121:0/2110665769 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6000d600 con 0x7f3d70157ed0 2023-11-15T03:03:09.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.421+0000 7f3d6dffb700 1 -- 172.21.15.121:0/2110665769 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6000d980 con 0x7f3d70157ed0 2023-11-15T03:03:09.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.421+0000 7f3d752db700 1 -- 172.21.15.121:0/2110665769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d70157ed0 msgr2=0x7f3d701582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:09.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.421+0000 7f3d752db700 1 --2- 172.21.15.121:0/2110665769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d70157ed0 0x7f3d701582b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f3d60009760 tx=0x7f3d60009060).stop 2023-11-15T03:03:09.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.421+0000 7f3d752db700 1 -- 172.21.15.121:0/2110665769 shutdown_connections 2023-11-15T03:03:09.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.421+0000 7f3d752db700 1 --2- 172.21.15.121:0/2110665769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d70067020 0x7f3d700bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:09.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.421+0000 7f3d752db700 1 --2- 172.21.15.121:0/2110665769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d70157ed0 0x7f3d701582b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:09.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.421+0000 7f3d752db700 1 -- 172.21.15.121:0/2110665769 >> 172.21.15.121:0/2110665769 conn(0x7f3d700591c0 msgr2=0x7f3d700c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:09.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.421+0000 7f3d752db700 1 -- 172.21.15.121:0/2110665769 shutdown_connections 2023-11-15T03:03:09.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.421+0000 7f3d752db700 1 -- 172.21.15.121:0/2110665769 wait complete. 2023-11-15T03:03:09.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.425+0000 7f3d752db700 1 Processor -- start 2023-11-15T03:03:09.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.425+0000 7f3d752db700 1 -- start start 2023-11-15T03:03:09.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.425+0000 7f3d752db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d70067020 0x7f3d700abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:09.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.425+0000 7f3d752db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d70067a60 0x7f3d70153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.425+0000 7f3d752db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d70157ed0 0x7f3d70159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.425+0000 7f3d752db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d700aa920 con 0x7f3d70067020 2023-11-15T03:03:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.425+0000 7f3d752db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d700aac00 con 0x7f3d70157ed0 2023-11-15T03:03:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.425+0000 7f3d752db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d700aaee0 con 0x7f3d70067a60 2023-11-15T03:03:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.425+0000 7f3d6effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d70067020 0x7f3d700abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.425+0000 7f3d6effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d70067020 0x7f3d700abb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32778/0 (socket says 172.21.15.121:32778) 2023-11-15T03:03:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.425+0000 7f3d6effd700 1 -- 172.21.15.121:0/3992286257 learned_addr learned my addr 172.21.15.121:0/3992286257 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.429+0000 7f3d6e7fc700 1 --2- 172.21.15.121:0/3992286257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d70067a60 0x7f3d70153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:09.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.429+0000 7f3d6f7fe700 1 --2- 172.21.15.121:0/3992286257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d70157ed0 0x7f3d70159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:09.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.429+0000 7f3d6effd700 1 -- 172.21.15.121:0/3992286257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d70067a60 msgr2=0x7f3d70153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:09.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.429+0000 7f3d6effd700 1 --2- 172.21.15.121:0/3992286257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d70067a60 0x7f3d70153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:09.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.429+0000 7f3d6effd700 1 -- 172.21.15.121:0/3992286257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d70157ed0 msgr2=0x7f3d70159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:09.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.429+0000 7f3d6effd700 1 --2- 172.21.15.121:0/3992286257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d70157ed0 0x7f3d70159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:09.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.429+0000 7f3d6effd700 1 -- 172.21.15.121:0/3992286257 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d600092a0 con 0x7f3d70067020 2023-11-15T03:03:09.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.429+0000 7f3d6effd700 1 --2- 172.21.15.121:0/3992286257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d70067020 0x7f3d700abb00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f3d70059730 tx=0x7f3d60016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:09.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.429+0000 7f3d56ffd700 1 -- 172.21.15.121:0/3992286257 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6000dd30 con 0x7f3d70067020 2023-11-15T03:03:09.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.429+0000 7f3d56ffd700 1 -- 172.21.15.121:0/3992286257 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6000de90 con 0x7f3d70067020 2023-11-15T03:03:09.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.429+0000 7f3d752db700 1 -- 172.21.15.121:0/3992286257 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d700ab1c0 con 0x7f3d70067020 2023-11-15T03:03:09.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.429+0000 7f3d56ffd700 1 -- 172.21.15.121:0/3992286257 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6000d980 con 0x7f3d70067020 2023-11-15T03:03:09.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.429+0000 7f3d752db700 1 -- 172.21.15.121:0/3992286257 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d700abff0 con 0x7f3d70067020 2023-11-15T03:03:09.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.429+0000 7f3d752db700 1 -- 172.21.15.121:0/3992286257 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d70047060 con 0x7f3d70067020 2023-11-15T03:03:09.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.433+0000 7f3d56ffd700 1 -- 172.21.15.121:0/3992286257 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3d6001d640 con 0x7f3d70067020 2023-11-15T03:03:09.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.433+0000 7f3d56ffd700 1 --2- 172.21.15.121:0/3992286257 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3d5c05a4e0 0x7f3d5c05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:09.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.433+0000 7f3d56ffd700 1 -- 172.21.15.121:0/3992286257 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6351+0+0 (secure 0 0 0) 0x7f3d600761f0 con 0x7f3d70067020 2023-11-15T03:03:09.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.433+0000 7f3d6e7fc700 1 --2- 172.21.15.121:0/3992286257 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3d5c05a4e0 0x7f3d5c05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:09.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.437+0000 7f3d6e7fc700 1 --2- 172.21.15.121:0/3992286257 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3d5c05a4e0 0x7f3d5c05c9a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f3d700a8cc0 tx=0x7f3d64009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:09.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.441+0000 7f3d56ffd700 1 -- 172.21.15.121:0/3992286257 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3d60049830 con 0x7f3d70067020 2023-11-15T03:03:09.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:09.641+0000 7f3d752db700 1 -- 172.21.15.121:0/3992286257 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3d700ac6d0 con 0x7f3d70067020 2023-11-15T03:03:10.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.229+0000 7f3d56ffd700 1 -- 172.21.15.121:0/3992286257 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3d6004ce50 con 0x7f3d70067020 2023-11-15T03:03:10.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.233+0000 7f3d752db700 1 -- 172.21.15.121:0/3992286257 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3d5c05a4e0 msgr2=0x7f3d5c05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:10.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.233+0000 7f3d752db700 1 --2- 172.21.15.121:0/3992286257 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3d5c05a4e0 0x7f3d5c05c9a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f3d700a8cc0 tx=0x7f3d64009040).stop 2023-11-15T03:03:10.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.233+0000 7f3d752db700 1 -- 172.21.15.121:0/3992286257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d70067020 msgr2=0x7f3d700abb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:10.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.233+0000 7f3d752db700 1 --2- 172.21.15.121:0/3992286257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d70067020 0x7f3d700abb00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f3d70059730 tx=0x7f3d60016000).stop 2023-11-15T03:03:10.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.233+0000 7f3d752db700 1 -- 172.21.15.121:0/3992286257 shutdown_connections 2023-11-15T03:03:10.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.233+0000 7f3d752db700 1 --2- 172.21.15.121:0/3992286257 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3d5c05a4e0 0x7f3d5c05c9a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:10.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.233+0000 7f3d752db700 1 --2- 172.21.15.121:0/3992286257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d70157ed0 0x7f3d70159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:10.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.233+0000 7f3d752db700 1 --2- 172.21.15.121:0/3992286257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d70067a60 0x7f3d70153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:10.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.233+0000 7f3d752db700 1 --2- 172.21.15.121:0/3992286257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d70067020 0x7f3d700abb00 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:10.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.233+0000 7f3d752db700 1 -- 172.21.15.121:0/3992286257 >> 172.21.15.121:0/3992286257 conn(0x7f3d700591c0 msgr2=0x7f3d70155a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:10.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.237+0000 7f3d752db700 1 -- 172.21.15.121:0/3992286257 shutdown_connections 2023-11-15T03:03:10.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.237+0000 7f3d752db700 1 -- 172.21.15.121:0/3992286257 wait complete. 2023-11-15T03:03:10.264 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-15T03:03:10.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.477+0000 7fb3d2747700 1 Processor -- start 2023-11-15T03:03:10.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.477+0000 7fb3d2747700 1 -- start start 2023-11-15T03:03:10.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3d2747700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb3cc066570 0x7fb3cc1515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:10.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3d2747700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3cc0660e0 con 0x7fb3cc151ac0 2023-11-15T03:03:10.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3d2747700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3cc066a20 con 0x7fb3cc1511f0 2023-11-15T03:03:10.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3d2747700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3cc151ef0 con 0x7fb3cc066570 2023-11-15T03:03:10.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3d1f46700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb3cc151ac0 0x7fb3cc0a8750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45776/0 (socket says 172.21.15.121:45776) 2023-11-15T03:03:10.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3d1f46700 1 -- 172.21.15.121:0/986676649 learned_addr learned my addr 172.21.15.121:0/986676649 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:10.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3d0f44700 1 --2- 172.21.15.121:0/986676649 >> v2:172.21.15.190:3300/0 conn(0x7fb3cc066570 0x7fb3cc1515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:10.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3c3fff700 1 -- 172.21.15.121:0/986676649 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901705386 0 0) 0x7fb3cc0660e0 con 0x7fb3cc151ac0 2023-11-15T03:03:10.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3c3fff700 1 -- 172.21.15.121:0/986676649 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3b80033f0 con 0x7fb3cc151ac0 2023-11-15T03:03:10.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3d0f44700 1 -- 172.21.15.121:0/986676649 >> v1:172.21.15.190:6789/0 conn(0x7fb3cc1511f0 legacy=0x7fb3cc0bcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:10.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3d0f44700 1 -- 172.21.15.121:0/986676649 >> v1:172.21.15.121:6789/0 conn(0x7fb3cc151ac0 legacy=0x7fb3cc0a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:10.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3d0f44700 1 -- 172.21.15.121:0/986676649 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc0bd390 con 0x7fb3cc066570 2023-11-15T03:03:10.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3d0f44700 1 --2- 172.21.15.121:0/986676649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3cc066570 0x7fb3cc1515d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7fb3bc006fd0 tx=0x7fb3bc006f40).ready entity=mon.1 client_cookie=353210c7ff1382d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:10.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3c3fff700 1 -- 172.21.15.121:0/986676649 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc005880 con 0x7fb3cc066570 2023-11-15T03:03:10.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3c3fff700 1 -- 172.21.15.121:0/986676649 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3bc0059e0 con 0x7fb3cc066570 2023-11-15T03:03:10.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3c3fff700 1 -- 172.21.15.121:0/986676649 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc005d60 con 0x7fb3cc066570 2023-11-15T03:03:10.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3d2747700 1 -- 172.21.15.121:0/986676649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3cc066570 msgr2=0x7fb3cc1515d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:10.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3d2747700 1 --2- 172.21.15.121:0/986676649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3cc066570 0x7fb3cc1515d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7fb3bc006fd0 tx=0x7fb3bc006f40).stop 2023-11-15T03:03:10.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3d2747700 1 -- 172.21.15.121:0/986676649 shutdown_connections 2023-11-15T03:03:10.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.481+0000 7fb3d2747700 1 --2- 172.21.15.121:0/986676649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3cc066570 0x7fb3cc1515d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:10.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.485+0000 7fb3d2747700 1 -- 172.21.15.121:0/986676649 >> 172.21.15.121:0/986676649 conn(0x7fb3cc067a60 msgr2=0x7fb3cc067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:10.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.485+0000 7fb3d2747700 1 -- 172.21.15.121:0/986676649 shutdown_connections 2023-11-15T03:03:10.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.485+0000 7fb3d2747700 1 -- 172.21.15.121:0/986676649 wait complete. 2023-11-15T03:03:10.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.485+0000 7fb3d2747700 1 Processor -- start 2023-11-15T03:03:10.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.485+0000 7fb3d2747700 1 -- start start 2023-11-15T03:03:10.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3d2747700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc066570 0x7fb3cc0a7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:10.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3d2747700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3cc1511f0 0x7fb3cc164490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:10.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3d2747700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3cc151ac0 0x7fb3cc0a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:10.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3d2747700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc1653a0 con 0x7fb3cc066570 2023-11-15T03:03:10.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3d2747700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc0a9a60 con 0x7fb3cc1511f0 2023-11-15T03:03:10.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3d2747700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc164de0 con 0x7fb3cc151ac0 2023-11-15T03:03:10.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3d1745700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc066570 0x7fb3cc0a7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:10.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3d1745700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc066570 0x7fb3cc0a7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32786/0 (socket says 172.21.15.121:32786) 2023-11-15T03:03:10.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3d1745700 1 -- 172.21.15.121:0/960121731 learned_addr learned my addr 172.21.15.121:0/960121731 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:10.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3d1f46700 1 --2- 172.21.15.121:0/960121731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3cc151ac0 0x7fb3cc0a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:10.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3d0f44700 1 --2- 172.21.15.121:0/960121731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3cc1511f0 0x7fb3cc164490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:10.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3d1745700 1 -- 172.21.15.121:0/960121731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3cc151ac0 msgr2=0x7fb3cc0a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:10.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3d1745700 1 --2- 172.21.15.121:0/960121731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3cc151ac0 0x7fb3cc0a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:10.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3d1745700 1 -- 172.21.15.121:0/960121731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3cc1511f0 msgr2=0x7fb3cc164490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:10.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3d1745700 1 --2- 172.21.15.121:0/960121731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3cc1511f0 0x7fb3cc164490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:10.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3d1745700 1 -- 172.21.15.121:0/960121731 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3bc009250 con 0x7fb3cc066570 2023-11-15T03:03:10.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3d1745700 1 --2- 172.21.15.121:0/960121731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc066570 0x7fb3cc0a7e90 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7fb3c800c4f0 tx=0x7fb3c8007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:10.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3c17fa700 1 -- 172.21.15.121:0/960121731 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c800ce60 con 0x7fb3cc066570 2023-11-15T03:03:10.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3c17fa700 1 -- 172.21.15.121:0/960121731 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c8001e90 con 0x7fb3cc066570 2023-11-15T03:03:10.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3d2747700 1 -- 172.21.15.121:0/960121731 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3cc1650c0 con 0x7fb3cc066570 2023-11-15T03:03:10.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3c17fa700 1 -- 172.21.15.121:0/960121731 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8007250 con 0x7fb3cc066570 2023-11-15T03:03:10.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.489+0000 7fb3d2747700 1 -- 172.21.15.121:0/960121731 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3cc0a8380 con 0x7fb3cc066570 2023-11-15T03:03:10.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.493+0000 7fb3d2747700 1 -- 172.21.15.121:0/960121731 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3cc06bcf0 con 0x7fb3cc066570 2023-11-15T03:03:10.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.493+0000 7fb3c17fa700 1 -- 172.21.15.121:0/960121731 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb3c80066a0 con 0x7fb3cc066570 2023-11-15T03:03:10.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.493+0000 7fb3c17fa700 1 --2- 172.21.15.121:0/960121731 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3b005a510 0x7fb3b005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:10.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.493+0000 7fb3c17fa700 1 -- 172.21.15.121:0/960121731 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6351+0+0 (secure 0 0 0) 0x7fb3c80741f0 con 0x7fb3cc066570 2023-11-15T03:03:10.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.497+0000 7fb3d0f44700 1 --2- 172.21.15.121:0/960121731 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3b005a510 0x7fb3b005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:10.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.497+0000 7fb3d0f44700 1 --2- 172.21.15.121:0/960121731 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3b005a510 0x7fb3b005c9d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fb3bc006d00 tx=0x7fb3bc016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:10.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.501+0000 7fb3c17fa700 1 -- 172.21.15.121:0/960121731 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb3c8047830 con 0x7fb3cc066570 2023-11-15T03:03:10.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:10.665+0000 7fb3d2747700 1 -- 172.21.15.121:0/960121731 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb3cc0c3ce0 con 0x7fb3cc066570 2023-11-15T03:03:11.131 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:03:11.140 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:03:11.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.233+0000 7fb3c17fa700 1 -- 172.21.15.121:0/960121731 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v82) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb3c804ae50 con 0x7fb3cc066570 2023-11-15T03:03:11.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.237+0000 7fb3d2747700 1 -- 172.21.15.121:0/960121731 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3b005a510 msgr2=0x7fb3b005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:11.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.237+0000 7fb3d2747700 1 --2- 172.21.15.121:0/960121731 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3b005a510 0x7fb3b005c9d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fb3bc006d00 tx=0x7fb3bc016000).stop 2023-11-15T03:03:11.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.237+0000 7fb3d2747700 1 -- 172.21.15.121:0/960121731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc066570 msgr2=0x7fb3cc0a7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:11.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.237+0000 7fb3d2747700 1 --2- 172.21.15.121:0/960121731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc066570 0x7fb3cc0a7e90 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7fb3c800c4f0 tx=0x7fb3c8007e00).stop 2023-11-15T03:03:11.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.241+0000 7fb3d2747700 1 -- 172.21.15.121:0/960121731 shutdown_connections 2023-11-15T03:03:11.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.241+0000 7fb3d2747700 1 --2- 172.21.15.121:0/960121731 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3b005a510 0x7fb3b005c9d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:11.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.241+0000 7fb3d2747700 1 --2- 172.21.15.121:0/960121731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3cc151ac0 0x7fb3cc0a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:11.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.241+0000 7fb3d2747700 1 --2- 172.21.15.121:0/960121731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3cc1511f0 0x7fb3cc164490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:11.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.241+0000 7fb3d2747700 1 --2- 172.21.15.121:0/960121731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc066570 0x7fb3cc0a7e90 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:11.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.241+0000 7fb3d2747700 1 -- 172.21.15.121:0/960121731 >> 172.21.15.121:0/960121731 conn(0x7fb3cc067a60 msgr2=0x7fb3cc14bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:11.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.241+0000 7fb3d2747700 1 -- 172.21.15.121:0/960121731 shutdown_connections 2023-11-15T03:03:11.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.241+0000 7fb3d2747700 1 -- 172.21.15.121:0/960121731 wait complete. 2023-11-15T03:03:11.247 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_metadata' removed 2023-11-15T03:03:11.266 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-15T03:03:11.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.481+0000 7f4bb2d69700 1 Processor -- start 2023-11-15T03:03:11.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.481+0000 7f4bb2d69700 1 -- start start 2023-11-15T03:03:11.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.485+0000 7f4bb2d69700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4bac0a5950 0x7f4bac0669d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:11.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.485+0000 7f4bb2d69700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bac065ca0 con 0x7f4bac151ad0 2023-11-15T03:03:11.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.485+0000 7f4bb2d69700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bac0a5e00 con 0x7f4bac066ec0 2023-11-15T03:03:11.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.485+0000 7f4bb2d69700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bac0a60c0 con 0x7f4bac0a5950 2023-11-15T03:03:11.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.485+0000 7f4bb0b05700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4bac151ad0 0x7f4bac062190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45790/0 (socket says 172.21.15.121:45790) 2023-11-15T03:03:11.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.485+0000 7f4babfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4bac0a5950 0x7f4bac0669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:11.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.485+0000 7f4bb0b05700 1 -- 172.21.15.121:0/2928370325 learned_addr learned my addr 172.21.15.121:0/2928370325 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:11.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.485+0000 7f4bab7fe700 1 -- 172.21.15.121:0/2928370325 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855994125 0 0) 0x7f4bac065ca0 con 0x7f4bac151ad0 2023-11-15T03:03:11.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.485+0000 7f4babfff700 1 -- 172.21.15.121:0/2928370325 >> v1:172.21.15.190:6789/0 conn(0x7f4bac066ec0 legacy=0x7f4bac15b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:11.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.485+0000 7f4babfff700 1 -- 172.21.15.121:0/2928370325 >> v1:172.21.15.121:6789/0 conn(0x7f4bac151ad0 legacy=0x7f4bac062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:11.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.485+0000 7f4babfff700 1 -- 172.21.15.121:0/2928370325 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac065ef0 con 0x7f4bac0a5950 2023-11-15T03:03:11.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.485+0000 7f4babfff700 1 --2- 172.21.15.121:0/2928370325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bac0a5950 0x7f4bac0669d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f4b98009760 tx=0x7f4b98009060).ready entity=mon.1 client_cookie=501c5d992c19b1ff server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:11.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.485+0000 7f4bab7fe700 1 -- 172.21.15.121:0/2928370325 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9800b580 con 0x7f4bac0a5950 2023-11-15T03:03:11.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.485+0000 7f4bab7fe700 1 -- 172.21.15.121:0/2928370325 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9800b6e0 con 0x7f4bac0a5950 2023-11-15T03:03:11.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.485+0000 7f4bab7fe700 1 -- 172.21.15.121:0/2928370325 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9800ba60 con 0x7f4bac0a5950 2023-11-15T03:03:11.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.489+0000 7f4bb2d69700 1 -- 172.21.15.121:0/2928370325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bac0a5950 msgr2=0x7f4bac0669d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:11.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.489+0000 7f4bb2d69700 1 --2- 172.21.15.121:0/2928370325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bac0a5950 0x7f4bac0669d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f4b98009760 tx=0x7f4b98009060).stop 2023-11-15T03:03:11.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.489+0000 7f4bb2d69700 1 -- 172.21.15.121:0/2928370325 shutdown_connections 2023-11-15T03:03:11.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.489+0000 7f4bb2d69700 1 --2- 172.21.15.121:0/2928370325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bac0a5950 0x7f4bac0669d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:11.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.489+0000 7f4bb2d69700 1 -- 172.21.15.121:0/2928370325 >> 172.21.15.121:0/2928370325 conn(0x7f4bac068500 msgr2=0x7f4bac0c6780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:11.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.489+0000 7f4bb2d69700 1 -- 172.21.15.121:0/2928370325 shutdown_connections 2023-11-15T03:03:11.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.489+0000 7f4bb2d69700 1 -- 172.21.15.121:0/2928370325 wait complete. 2023-11-15T03:03:11.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.489+0000 7f4bb2d69700 1 Processor -- start 2023-11-15T03:03:11.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.493+0000 7f4bb2d69700 1 -- start start 2023-11-15T03:03:11.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.493+0000 7f4bb2d69700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bac066ec0 0x7f4bac0bf000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:11.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.493+0000 7f4bb2d69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bac0a5950 0x7f4bac0be8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:11.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.493+0000 7f4bb2d69700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bac151ad0 0x7f4bac13e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:11.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.493+0000 7f4bb2d69700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac0bedd0 con 0x7f4bac151ad0 2023-11-15T03:03:11.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.493+0000 7f4bb2d69700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac0bf5c0 con 0x7f4bac066ec0 2023-11-15T03:03:11.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.493+0000 7f4bb2d69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac0bf8a0 con 0x7f4bac0a5950 2023-11-15T03:03:11.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.493+0000 7f4bb1306700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bac151ad0 0x7f4bac13e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:11.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.493+0000 7f4bb1306700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bac151ad0 0x7f4bac13e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32788/0 (socket says 172.21.15.121:32788) 2023-11-15T03:03:11.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.493+0000 7f4bb0b05700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bac066ec0 0x7f4bac0bf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:11.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.493+0000 7f4bb1306700 1 -- 172.21.15.121:0/1323581413 learned_addr learned my addr 172.21.15.121:0/1323581413 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:11.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.493+0000 7f4babfff700 1 --2- 172.21.15.121:0/1323581413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bac0a5950 0x7f4bac0be8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:11.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.493+0000 7f4bb1306700 1 -- 172.21.15.121:0/1323581413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bac0a5950 msgr2=0x7f4bac0be8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:11.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.493+0000 7f4bb1306700 1 --2- 172.21.15.121:0/1323581413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bac0a5950 0x7f4bac0be8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:11.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.493+0000 7f4bb1306700 1 -- 172.21.15.121:0/1323581413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bac066ec0 msgr2=0x7f4bac0bf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:11.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.493+0000 7f4bb1306700 1 --2- 172.21.15.121:0/1323581413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bac066ec0 0x7f4bac0bf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:11.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.493+0000 7f4bb1306700 1 -- 172.21.15.121:0/1323581413 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b980092a0 con 0x7f4bac151ad0 2023-11-15T03:03:11.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.497+0000 7f4bb1306700 1 --2- 172.21.15.121:0/1323581413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bac151ad0 0x7f4bac13e080 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7f4ba4002230 tx=0x7f4ba40015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:11.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.497+0000 7f4ba8ff9700 1 -- 172.21.15.121:0/1323581413 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba400e4a0 con 0x7f4bac151ad0 2023-11-15T03:03:11.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.497+0000 7f4ba8ff9700 1 -- 172.21.15.121:0/1323581413 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba400e600 con 0x7f4bac151ad0 2023-11-15T03:03:11.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.497+0000 7f4ba8ff9700 1 -- 172.21.15.121:0/1323581413 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba400e980 con 0x7f4bac151ad0 2023-11-15T03:03:11.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.497+0000 7f4bb2d69700 1 -- 172.21.15.121:0/1323581413 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bac0bfb80 con 0x7f4bac151ad0 2023-11-15T03:03:11.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.497+0000 7f4bb2d69700 1 -- 172.21.15.121:0/1323581413 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bac0c0140 con 0x7f4bac151ad0 2023-11-15T03:03:11.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.497+0000 7f4bb2d69700 1 -- 172.21.15.121:0/1323581413 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bac0c0360 con 0x7f4bac151ad0 2023-11-15T03:03:11.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.501+0000 7f4ba8ff9700 1 -- 172.21.15.121:0/1323581413 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4ba401b3b0 con 0x7f4bac151ad0 2023-11-15T03:03:11.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.501+0000 7f4ba8ff9700 1 --2- 172.21.15.121:0/1323581413 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4b9005a510 0x7f4b9005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:11.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.501+0000 7f4ba8ff9700 1 -- 172.21.15.121:0/1323581413 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5855+0+0 (secure 0 0 0) 0x7f4ba4073dc0 con 0x7f4bac151ad0 2023-11-15T03:03:11.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.501+0000 7f4bb0b05700 1 --2- 172.21.15.121:0/1323581413 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4b9005a510 0x7f4b9005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:11.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.501+0000 7f4bb0b05700 1 --2- 172.21.15.121:0/1323581413 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4b9005a510 0x7f4b9005c9d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f4ba0007fd0 tx=0x7f4ba0007db0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:11.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.505+0000 7f4ba8ff9700 1 -- 172.21.15.121:0/1323581413 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4ba4047700 con 0x7f4bac151ad0 2023-11-15T03:03:11.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:11.681+0000 7f4bb2d69700 1 -- 172.21.15.121:0/1323581413 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4bac068b50 con 0x7f4bac151ad0 2023-11-15T03:03:12.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.245+0000 7f4ba8ff9700 1 -- 172.21.15.121:0/1323581413 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v83) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4ba404ad20 con 0x7f4bac151ad0 2023-11-15T03:03:12.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.253+0000 7f4bb2d69700 1 -- 172.21.15.121:0/1323581413 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4b9005a510 msgr2=0x7f4b9005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:12.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.253+0000 7f4bb2d69700 1 --2- 172.21.15.121:0/1323581413 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4b9005a510 0x7f4b9005c9d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f4ba0007fd0 tx=0x7f4ba0007db0).stop 2023-11-15T03:03:12.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.253+0000 7f4bb2d69700 1 -- 172.21.15.121:0/1323581413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bac151ad0 msgr2=0x7f4bac13e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:12.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.253+0000 7f4bb2d69700 1 --2- 172.21.15.121:0/1323581413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bac151ad0 0x7f4bac13e080 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7f4ba4002230 tx=0x7f4ba40015e0).stop 2023-11-15T03:03:12.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.253+0000 7f4bb2d69700 1 -- 172.21.15.121:0/1323581413 shutdown_connections 2023-11-15T03:03:12.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.253+0000 7f4bb2d69700 1 --2- 172.21.15.121:0/1323581413 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4b9005a510 0x7f4b9005c9d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:12.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.253+0000 7f4bb2d69700 1 --2- 172.21.15.121:0/1323581413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bac151ad0 0x7f4bac13e080 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:12.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.253+0000 7f4bb2d69700 1 --2- 172.21.15.121:0/1323581413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bac0a5950 0x7f4bac0be8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:12.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.253+0000 7f4bb2d69700 1 --2- 172.21.15.121:0/1323581413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bac066ec0 0x7f4bac0bf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:12.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.253+0000 7f4bb2d69700 1 -- 172.21.15.121:0/1323581413 >> 172.21.15.121:0/1323581413 conn(0x7f4bac068500 msgr2=0x7f4bac0a39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:12.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.253+0000 7f4bb2d69700 1 -- 172.21.15.121:0/1323581413 shutdown_connections 2023-11-15T03:03:12.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.253+0000 7f4bb2d69700 1 -- 172.21.15.121:0/1323581413 wait complete. 2023-11-15T03:03:12.261 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_data' removed 2023-11-15T03:03:12.279 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-15T03:03:12.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.433+0000 7f103f59e700 1 Processor -- start 2023-11-15T03:03:12.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.433+0000 7f103f59e700 1 -- start start 2023-11-15T03:03:12.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.437+0000 7f103f59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1040156db0 0x7f1040157190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:12.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.437+0000 7f103f59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f10400c2140 0x7f1040064820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:12.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.437+0000 7f103f59e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1040158660 con 0x7f10400c3fe0 2023-11-15T03:03:12.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.437+0000 7f103f59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10400c25a0 con 0x7f10400c2140 2023-11-15T03:03:12.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.437+0000 7f103f59e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1040157a50 con 0x7f1040156db0 2023-11-15T03:03:12.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.437+0000 7f103ed9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f10400c3fe0 0x7f1040067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45800/0 (socket says 172.21.15.121:45800) 2023-11-15T03:03:12.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.437+0000 7f103ed9d700 1 -- 172.21.15.121:0/3354311509 learned_addr learned my addr 172.21.15.121:0/3354311509 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:12.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.437+0000 7f103dd9b700 1 --2- 172.21.15.121:0/3354311509 >> v2:172.21.15.121:3300/0 conn(0x7f10400c2140 0x7f1040064820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:12.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.437+0000 7f103e59c700 1 --2- 172.21.15.121:0/3354311509 >> v2:172.21.15.190:3300/0 conn(0x7f1040156db0 0x7f1040157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:12.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.441+0000 7f103d59a700 1 -- 172.21.15.121:0/3354311509 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051014327 0 0) 0x7f1040158660 con 0x7f10400c3fe0 2023-11-15T03:03:12.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.441+0000 7f103d59a700 1 -- 172.21.15.121:0/3354311509 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10280033f0 con 0x7f10400c3fe0 2023-11-15T03:03:12.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.441+0000 7f103dd9b700 1 -- 172.21.15.121:0/3354311509 >> v2:172.21.15.190:3300/0 conn(0x7f1040156db0 msgr2=0x7f1040157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:12.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.441+0000 7f103dd9b700 1 --2- 172.21.15.121:0/3354311509 >> v2:172.21.15.190:3300/0 conn(0x7f1040156db0 0x7f1040157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:12.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.441+0000 7f103dd9b700 1 -- 172.21.15.121:0/3354311509 >> v1:172.21.15.121:6789/0 conn(0x7f10400c3fe0 legacy=0x7f1040067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:12.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.441+0000 7f103dd9b700 1 -- 172.21.15.121:0/3354311509 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1040157d30 con 0x7f10400c2140 2023-11-15T03:03:12.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.441+0000 7f103dd9b700 1 --2- 172.21.15.121:0/3354311509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10400c2140 0x7f1040064820 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f1040062c90 tx=0x7f1034009060).ready entity=mon.0 client_cookie=393714f139a4604c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:12.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.441+0000 7f103d59a700 1 -- 172.21.15.121:0/3354311509 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103400d4a0 con 0x7f10400c2140 2023-11-15T03:03:12.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.441+0000 7f103d59a700 1 -- 172.21.15.121:0/3354311509 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103400d600 con 0x7f10400c2140 2023-11-15T03:03:12.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.441+0000 7f103d59a700 1 -- 172.21.15.121:0/3354311509 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103400d980 con 0x7f10400c2140 2023-11-15T03:03:12.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.441+0000 7f103f59e700 1 -- 172.21.15.121:0/3354311509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10400c2140 msgr2=0x7f1040064820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:12.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.441+0000 7f103f59e700 1 --2- 172.21.15.121:0/3354311509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10400c2140 0x7f1040064820 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f1040062c90 tx=0x7f1034009060).stop 2023-11-15T03:03:12.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.441+0000 7f103f59e700 1 -- 172.21.15.121:0/3354311509 shutdown_connections 2023-11-15T03:03:12.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.441+0000 7f103f59e700 1 --2- 172.21.15.121:0/3354311509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10400c2140 0x7f1040064820 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:12.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.441+0000 7f103f59e700 1 --2- 172.21.15.121:0/3354311509 >> v2:172.21.15.190:3300/0 conn(0x7f1040156db0 0x7f1040157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:12.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.441+0000 7f103f59e700 1 -- 172.21.15.121:0/3354311509 >> 172.21.15.121:0/3354311509 conn(0x7f1040066830 msgr2=0x7f1040066c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:12.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.441+0000 7f103f59e700 1 -- 172.21.15.121:0/3354311509 shutdown_connections 2023-11-15T03:03:12.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.441+0000 7f103f59e700 1 -- 172.21.15.121:0/3354311509 wait complete. 2023-11-15T03:03:12.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.445+0000 7f103f59e700 1 Processor -- start 2023-11-15T03:03:12.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.445+0000 7f103f59e700 1 -- start start 2023-11-15T03:03:12.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.445+0000 7f103f59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10400c2140 0x7f10400bf860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:12.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.445+0000 7f103f59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10400c3fe0 0x7f104013a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:12.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.445+0000 7f103f59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1040156db0 0x7f10400a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:12.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.445+0000 7f103f59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10400bf3d0 con 0x7f10400c2140 2023-11-15T03:03:12.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.445+0000 7f103f59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10400ac7e0 con 0x7f1040156db0 2023-11-15T03:03:12.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.445+0000 7f103f59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10400be7c0 con 0x7f10400c3fe0 2023-11-15T03:03:12.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.449+0000 7f103e59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10400c2140 0x7f10400bf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:12.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.449+0000 7f103e59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10400c2140 0x7f10400bf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32814/0 (socket says 172.21.15.121:32814) 2023-11-15T03:03:12.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.449+0000 7f103e59c700 1 -- 172.21.15.121:0/731011877 learned_addr learned my addr 172.21.15.121:0/731011877 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:12.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.449+0000 7f103ed9d700 1 --2- 172.21.15.121:0/731011877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1040156db0 0x7f10400a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:12.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.449+0000 7f103dd9b700 1 --2- 172.21.15.121:0/731011877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10400c3fe0 0x7f104013a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:12.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.449+0000 7f103e59c700 1 -- 172.21.15.121:0/731011877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10400c3fe0 msgr2=0x7f104013a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:12.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.449+0000 7f103e59c700 1 --2- 172.21.15.121:0/731011877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10400c3fe0 0x7f104013a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:12.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.449+0000 7f103e59c700 1 -- 172.21.15.121:0/731011877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1040156db0 msgr2=0x7f10400a5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:12.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.449+0000 7f103e59c700 1 --2- 172.21.15.121:0/731011877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1040156db0 0x7f10400a5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:12.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.449+0000 7f103e59c700 1 -- 172.21.15.121:0/731011877 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10340092a0 con 0x7f10400c2140 2023-11-15T03:03:12.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.449+0000 7f103e59c700 1 --2- 172.21.15.121:0/731011877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10400c2140 0x7f10400bf860 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7f103000e900 tx=0x7f1030009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:12.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.449+0000 7f102e7fc700 1 -- 172.21.15.121:0/731011877 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103000c640 con 0x7f10400c2140 2023-11-15T03:03:12.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.449+0000 7f102e7fc700 1 -- 172.21.15.121:0/731011877 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103000c7a0 con 0x7f10400c2140 2023-11-15T03:03:12.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.449+0000 7f102e7fc700 1 -- 172.21.15.121:0/731011877 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103000cb20 con 0x7f10400c2140 2023-11-15T03:03:12.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.449+0000 7f103f59e700 1 -- 172.21.15.121:0/731011877 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10400beaa0 con 0x7f10400c2140 2023-11-15T03:03:12.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.449+0000 7f103f59e700 1 -- 172.21.15.121:0/731011877 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10400bf060 con 0x7f10400c2140 2023-11-15T03:03:12.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.449+0000 7f103f59e700 1 -- 172.21.15.121:0/731011877 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f104006bcf0 con 0x7f10400c2140 2023-11-15T03:03:12.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.449+0000 7f102e7fc700 1 -- 172.21.15.121:0/731011877 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f103001b400 con 0x7f10400c2140 2023-11-15T03:03:12.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.453+0000 7f102e7fc700 1 --2- 172.21.15.121:0/731011877 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f102005a510 0x7f102005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:12.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.453+0000 7f102e7fc700 1 -- 172.21.15.121:0/731011877 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f1030074a30 con 0x7f10400c2140 2023-11-15T03:03:12.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.453+0000 7f103dd9b700 1 --2- 172.21.15.121:0/731011877 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f102005a510 0x7f102005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:12.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.453+0000 7f103dd9b700 1 --2- 172.21.15.121:0/731011877 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f102005a510 0x7f102005c9d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f1040148020 tx=0x7f1034016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:12.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.457+0000 7f102e7fc700 1 -- 172.21.15.121:0/731011877 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1030048390 con 0x7f10400c2140 2023-11-15T03:03:12.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.617+0000 7f103f59e700 1 -- 172.21.15.121:0/731011877 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1040158010 con 0x7f10400c2140 2023-11-15T03:03:12.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.625+0000 7f102e7fc700 1 -- 172.21.15.121:0/731011877 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1030016070 con 0x7f10400c2140 2023-11-15T03:03:12.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.629+0000 7f103f59e700 1 -- 172.21.15.121:0/731011877 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f102005a510 msgr2=0x7f102005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:12.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.629+0000 7f103f59e700 1 --2- 172.21.15.121:0/731011877 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f102005a510 0x7f102005c9d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f1040148020 tx=0x7f1034016000).stop 2023-11-15T03:03:12.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.629+0000 7f103f59e700 1 -- 172.21.15.121:0/731011877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10400c2140 msgr2=0x7f10400bf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:12.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.629+0000 7f103f59e700 1 --2- 172.21.15.121:0/731011877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10400c2140 0x7f10400bf860 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7f103000e900 tx=0x7f1030009060).stop 2023-11-15T03:03:12.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.629+0000 7f103f59e700 1 -- 172.21.15.121:0/731011877 shutdown_connections 2023-11-15T03:03:12.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.629+0000 7f103f59e700 1 --2- 172.21.15.121:0/731011877 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f102005a510 0x7f102005c9d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:12.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.629+0000 7f103f59e700 1 --2- 172.21.15.121:0/731011877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1040156db0 0x7f10400a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:12.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.629+0000 7f103f59e700 1 --2- 172.21.15.121:0/731011877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10400c3fe0 0x7f104013a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:12.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.629+0000 7f103f59e700 1 --2- 172.21.15.121:0/731011877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10400c2140 0x7f10400bf860 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:12.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.629+0000 7f103f59e700 1 -- 172.21.15.121:0/731011877 >> 172.21.15.121:0/731011877 conn(0x7f1040066830 msgr2=0x7f10400bc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:12.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.629+0000 7f103f59e700 1 -- 172.21.15.121:0/731011877 shutdown_connections 2023-11-15T03:03:12.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.629+0000 7f103f59e700 1 -- 172.21.15.121:0/731011877 wait complete. 2023-11-15T03:03:12.652 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2023-11-15T03:03:12.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.821+0000 7fccea707700 1 Processor -- start 2023-11-15T03:03:12.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.821+0000 7fccea707700 1 -- start start 2023-11-15T03:03:12.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.825+0000 7fccea707700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcce4151af0 0x7fcce40a7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:12.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.825+0000 7fccea707700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcce40a5720 0x7fcce40c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:12.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.825+0000 7fccea707700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce41528b0 con 0x7fcce40c3430 2023-11-15T03:03:12.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.825+0000 7fccea707700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcce4066d80 con 0x7fcce4151af0 2023-11-15T03:03:12.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.825+0000 7fccea707700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce4151ff0 con 0x7fcce40a5720 2023-11-15T03:03:12.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.825+0000 7fcce9705700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcce4151af0 0x7fcce40a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:12.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.825+0000 7fcce9705700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcce4151af0 0x7fcce40a7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32822/0 (socket says 172.21.15.121:32822) 2023-11-15T03:03:12.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.825+0000 7fcce9705700 1 -- 172.21.15.121:0/2091532565 learned_addr learned my addr 172.21.15.121:0/2091532565 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:12.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.825+0000 7fcce8f04700 1 --2- 172.21.15.121:0/2091532565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcce40a5720 0x7fcce40c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:12.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.825+0000 7fcce9705700 1 -- 172.21.15.121:0/2091532565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcce40a5720 msgr2=0x7fcce40c2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:12.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.825+0000 7fcce9705700 1 --2- 172.21.15.121:0/2091532565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcce40a5720 0x7fcce40c2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:12.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.825+0000 7fcce9705700 1 -- 172.21.15.121:0/2091532565 >> v1:172.21.15.190:6789/0 conn(0x7fcce40c3430 legacy=0x7fcce40c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:12.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.829+0000 7fcce9705700 1 -- 172.21.15.121:0/2091532565 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce40c3890 con 0x7fcce4151af0 2023-11-15T03:03:12.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.829+0000 7fcce9705700 1 --2- 172.21.15.121:0/2091532565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcce4151af0 0x7fcce40a7070 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7fcce0006fd0 tx=0x7fcce00090f0).ready entity=mon.0 client_cookie=1574eead09efdcab server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:12.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.829+0000 7fccdbfff700 1 -- 172.21.15.121:0/2091532565 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce000b600 con 0x7fcce4151af0 2023-11-15T03:03:12.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.829+0000 7fccdbfff700 1 -- 172.21.15.121:0/2091532565 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce000b760 con 0x7fcce4151af0 2023-11-15T03:03:12.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.829+0000 7fccdbfff700 1 -- 172.21.15.121:0/2091532565 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce000bae0 con 0x7fcce4151af0 2023-11-15T03:03:12.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.829+0000 7fccea707700 1 -- 172.21.15.121:0/2091532565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcce4151af0 msgr2=0x7fcce40a7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:12.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.829+0000 7fccea707700 1 --2- 172.21.15.121:0/2091532565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcce4151af0 0x7fcce40a7070 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7fcce0006fd0 tx=0x7fcce00090f0).stop 2023-11-15T03:03:12.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.829+0000 7fccea707700 1 -- 172.21.15.121:0/2091532565 shutdown_connections 2023-11-15T03:03:12.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.829+0000 7fccea707700 1 --2- 172.21.15.121:0/2091532565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcce40a5720 0x7fcce40c2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:12.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.829+0000 7fccea707700 1 --2- 172.21.15.121:0/2091532565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcce4151af0 0x7fcce40a7070 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:12.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.829+0000 7fccea707700 1 -- 172.21.15.121:0/2091532565 >> 172.21.15.121:0/2091532565 conn(0x7fcce4067a60 msgr2=0x7fcce4067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:12.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.829+0000 7fccea707700 1 -- 172.21.15.121:0/2091532565 shutdown_connections 2023-11-15T03:03:12.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.829+0000 7fccea707700 1 -- 172.21.15.121:0/2091532565 wait complete. 2023-11-15T03:03:12.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.833+0000 7fccea707700 1 Processor -- start 2023-11-15T03:03:12.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.833+0000 7fccea707700 1 -- start start 2023-11-15T03:03:12.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.833+0000 7fccea707700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcce40a5720 0x7fcce414dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:12.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.833+0000 7fccea707700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcce40c3430 0x7fcce40a3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:12.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.833+0000 7fccea707700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcce40c1fa0 0x7fcce40a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:12.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.833+0000 7fccea707700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce409f5e0 con 0x7fcce40a5720 2023-11-15T03:03:12.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.833+0000 7fccea707700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce414e390 con 0x7fcce40c3430 2023-11-15T03:03:12.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.833+0000 7fccea707700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce40a3bd0 con 0x7fcce40c1fa0 2023-11-15T03:03:12.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.837+0000 7fcce9705700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcce40a5720 0x7fcce414dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:12.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.837+0000 7fcce9705700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcce40a5720 0x7fcce414dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32830/0 (socket says 172.21.15.121:32830) 2023-11-15T03:03:12.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.837+0000 7fcce9705700 1 -- 172.21.15.121:0/2988037338 learned_addr learned my addr 172.21.15.121:0/2988037338 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:12.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.837+0000 7fcce8f04700 1 --2- 172.21.15.121:0/2988037338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcce40c3430 0x7fcce40a3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:12.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.837+0000 7fcce9f06700 1 --2- 172.21.15.121:0/2988037338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcce40c1fa0 0x7fcce40a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:12.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.837+0000 7fcce9705700 1 -- 172.21.15.121:0/2988037338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcce40c1fa0 msgr2=0x7fcce40a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:12.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.837+0000 7fcce9705700 1 --2- 172.21.15.121:0/2988037338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcce40c1fa0 0x7fcce40a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:12.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.837+0000 7fcce9705700 1 -- 172.21.15.121:0/2988037338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcce40c3430 msgr2=0x7fcce40a3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:12.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.837+0000 7fcce9705700 1 --2- 172.21.15.121:0/2988037338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcce40c3430 0x7fcce40a3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:12.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.837+0000 7fcce9705700 1 -- 172.21.15.121:0/2988037338 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce0009370 con 0x7fcce40a5720 2023-11-15T03:03:12.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.837+0000 7fcce8f04700 1 --2- 172.21.15.121:0/2988037338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcce40c3430 0x7fcce40a3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:12.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.837+0000 7fcce9705700 1 --2- 172.21.15.121:0/2988037338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcce40a5720 0x7fcce414dea0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7fcce40c75a0 tx=0x7fcce0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:12.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.837+0000 7fccd97fa700 1 -- 172.21.15.121:0/2988037338 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce000b8d0 con 0x7fcce40a5720 2023-11-15T03:03:12.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.837+0000 7fccd97fa700 1 -- 172.21.15.121:0/2988037338 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce000ba30 con 0x7fcce40a5720 2023-11-15T03:03:12.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.837+0000 7fccea707700 1 -- 172.21.15.121:0/2988037338 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce40a3eb0 con 0x7fcce40a5720 2023-11-15T03:03:12.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.837+0000 7fccd97fa700 1 -- 172.21.15.121:0/2988037338 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce001c670 con 0x7fcce40a5720 2023-11-15T03:03:12.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.837+0000 7fccea707700 1 -- 172.21.15.121:0/2988037338 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce40a4470 con 0x7fcce40a5720 2023-11-15T03:03:12.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.837+0000 7fccea707700 1 -- 172.21.15.121:0/2988037338 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce406bcf0 con 0x7fcce40a5720 2023-11-15T03:03:12.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.841+0000 7fccd97fa700 1 -- 172.21.15.121:0/2988037338 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fcce001ca20 con 0x7fcce40a5720 2023-11-15T03:03:12.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.841+0000 7fccd97fa700 1 --2- 172.21.15.121:0/2988037338 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fccc805a510 0x7fccc805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:12.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.841+0000 7fccd97fa700 1 -- 172.21.15.121:0/2988037338 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5441+0+0 (secure 0 0 0) 0x7fcce0076370 con 0x7fcce40a5720 2023-11-15T03:03:12.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.841+0000 7fcce8f04700 1 --2- 172.21.15.121:0/2988037338 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fccc805a510 0x7fccc805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:12.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.841+0000 7fcce8f04700 1 --2- 172.21.15.121:0/2988037338 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fccc805a510 0x7fccc805c9d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fccd4006fd0 tx=0x7fccd4008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:12.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:12.845+0000 7fccd97fa700 1 -- 172.21.15.121:0/2988037338 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcce0049db0 con 0x7fcce40a5720 2023-11-15T03:03:13.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.013+0000 7fccea707700 1 -- 172.21.15.121:0/2988037338 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) v1 -- 0x7fcce41509e0 con 0x7fcce40a5720 2023-11-15T03:03:13.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.013+0000 7fccd97fa700 1 -- 172.21.15.121:0/2988037338 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v17) v1 ==== 121+0+0 (secure 0 0 0) 0x7fcce002a070 con 0x7fcce40a5720 2023-11-15T03:03:13.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.013+0000 7fccea707700 1 -- 172.21.15.121:0/2988037338 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fccc805a510 msgr2=0x7fccc805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:13.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.013+0000 7fccea707700 1 --2- 172.21.15.121:0/2988037338 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fccc805a510 0x7fccc805c9d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fccd4006fd0 tx=0x7fccd4008040).stop 2023-11-15T03:03:13.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.013+0000 7fccea707700 1 -- 172.21.15.121:0/2988037338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcce40a5720 msgr2=0x7fcce414dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:13.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.013+0000 7fccea707700 1 --2- 172.21.15.121:0/2988037338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcce40a5720 0x7fcce414dea0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7fcce40c75a0 tx=0x7fcce0017000).stop 2023-11-15T03:03:13.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.017+0000 7fccea707700 1 -- 172.21.15.121:0/2988037338 shutdown_connections 2023-11-15T03:03:13.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.017+0000 7fccea707700 1 --2- 172.21.15.121:0/2988037338 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fccc805a510 0x7fccc805c9d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:13.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.017+0000 7fccea707700 1 --2- 172.21.15.121:0/2988037338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcce40c1fa0 0x7fcce40a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:13.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.017+0000 7fccea707700 1 --2- 172.21.15.121:0/2988037338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcce40c3430 0x7fcce40a3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:13.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.017+0000 7fccea707700 1 --2- 172.21.15.121:0/2988037338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcce40a5720 0x7fcce414dea0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:13.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.017+0000 7fccea707700 1 -- 172.21.15.121:0/2988037338 >> 172.21.15.121:0/2988037338 conn(0x7fcce4067a60 msgr2=0x7fcce4150cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:13.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.017+0000 7fccea707700 1 -- 172.21.15.121:0/2988037338 shutdown_connections 2023-11-15T03:03:13.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.017+0000 7fccea707700 1 -- 172.21.15.121:0/2988037338 wait complete. 2023-11-15T03:03:13.044 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush' 2023-11-15T03:03:13.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.213+0000 7f2c393f9700 1 Processor -- start 2023-11-15T03:03:13.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.213+0000 7f2c393f9700 1 -- start start 2023-11-15T03:03:13.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.213+0000 7f2c393f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c34152850 0x7f2c340c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:13.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.213+0000 7f2c393f9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c340c1d50 con 0x7f2c3415be30 2023-11-15T03:03:13.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.213+0000 7f2c393f9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c3415cb80 con 0x7f2c340c3280 2023-11-15T03:03:13.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.213+0000 7f2c393f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c340c1860 con 0x7f2c34152850 2023-11-15T03:03:13.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.213+0000 7f2c38bf8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f2c3415be30 0x7f2c3415c280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45814/0 (socket says 172.21.15.121:45814) 2023-11-15T03:03:13.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.213+0000 7f2c38bf8700 1 -- 172.21.15.121:0/1760347227 learned_addr learned my addr 172.21.15.121:0/1760347227 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:13.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.213+0000 7f2c337fe700 1 --2- 172.21.15.121:0/1760347227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c34152850 0x7f2c340c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:13.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.217+0000 7f2c32ffd700 1 -- 172.21.15.121:0/1760347227 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935986317 0 0) 0x7f2c3415cb80 con 0x7f2c340c3280 2023-11-15T03:03:13.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.217+0000 7f2c32ffd700 1 -- 172.21.15.121:0/1760347227 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c1c0033f0 con 0x7f2c340c3280 2023-11-15T03:03:13.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.217+0000 7f2c32ffd700 1 -- 172.21.15.121:0/1760347227 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140183743 0 0) 0x7f2c340c1d50 con 0x7f2c3415be30 2023-11-15T03:03:13.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.217+0000 7f2c32ffd700 1 -- 172.21.15.121:0/1760347227 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c3415cb80 con 0x7f2c3415be30 2023-11-15T03:03:13.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.217+0000 7f2c337fe700 1 -- 172.21.15.121:0/1760347227 >> v1:172.21.15.190:6789/0 conn(0x7f2c340c3280 legacy=0x7f2c34152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:13.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.217+0000 7f2c337fe700 1 -- 172.21.15.121:0/1760347227 >> v1:172.21.15.121:6789/0 conn(0x7f2c3415be30 legacy=0x7f2c3415c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:13.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.217+0000 7f2c337fe700 1 -- 172.21.15.121:0/1760347227 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c340a9240 con 0x7f2c34152850 2023-11-15T03:03:13.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.217+0000 7f2c337fe700 1 --2- 172.21.15.121:0/1760347227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c34152850 0x7f2c340c3660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f2c28009790 tx=0x7f2c28009060).ready entity=mon.2 client_cookie=fef3337fa5ab3aa7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:13.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.217+0000 7f2c32ffd700 1 -- 172.21.15.121:0/1760347227 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2800f510 con 0x7f2c34152850 2023-11-15T03:03:13.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.217+0000 7f2c32ffd700 1 -- 172.21.15.121:0/1760347227 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2800f670 con 0x7f2c34152850 2023-11-15T03:03:13.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.217+0000 7f2c32ffd700 1 -- 172.21.15.121:0/1760347227 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2800f9f0 con 0x7f2c34152850 2023-11-15T03:03:13.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.217+0000 7f2c393f9700 1 -- 172.21.15.121:0/1760347227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c34152850 msgr2=0x7f2c340c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:13.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.217+0000 7f2c393f9700 1 --2- 172.21.15.121:0/1760347227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c34152850 0x7f2c340c3660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f2c28009790 tx=0x7f2c28009060).stop 2023-11-15T03:03:13.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.217+0000 7f2c393f9700 1 -- 172.21.15.121:0/1760347227 shutdown_connections 2023-11-15T03:03:13.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.217+0000 7f2c393f9700 1 --2- 172.21.15.121:0/1760347227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c34152850 0x7f2c340c3660 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:13.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.217+0000 7f2c393f9700 1 -- 172.21.15.121:0/1760347227 >> 172.21.15.121:0/1760347227 conn(0x7f2c34062190 msgr2=0x7f2c34062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:13.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.217+0000 7f2c393f9700 1 -- 172.21.15.121:0/1760347227 shutdown_connections 2023-11-15T03:03:13.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.217+0000 7f2c393f9700 1 -- 172.21.15.121:0/1760347227 wait complete. 2023-11-15T03:03:13.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.221+0000 7f2c393f9700 1 Processor -- start 2023-11-15T03:03:13.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.221+0000 7f2c393f9700 1 -- start start 2023-11-15T03:03:13.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.221+0000 7f2c393f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c340c3280 0x7f2c34156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:13.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.221+0000 7f2c393f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c34152850 0x7f2c34155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:13.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.221+0000 7f2c393f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c3415be30 0x7f2c34065d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:13.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.221+0000 7f2c393f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c34153850 con 0x7f2c34152850 2023-11-15T03:03:13.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.221+0000 7f2c393f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c34157370 con 0x7f2c340c3280 2023-11-15T03:03:13.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.221+0000 7f2c393f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c34153a80 con 0x7f2c3415be30 2023-11-15T03:03:13.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.221+0000 7f2c337fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c34152850 0x7f2c34155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:13.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.221+0000 7f2c337fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c34152850 0x7f2c34155960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32838/0 (socket says 172.21.15.121:32838) 2023-11-15T03:03:13.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.221+0000 7f2c337fe700 1 -- 172.21.15.121:0/2884566694 learned_addr learned my addr 172.21.15.121:0/2884566694 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:13.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.221+0000 7f2c33fff700 1 --2- 172.21.15.121:0/2884566694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c340c3280 0x7f2c34156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:13.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.221+0000 7f2c38bf8700 1 --2- 172.21.15.121:0/2884566694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c3415be30 0x7f2c34065d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:13.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.221+0000 7f2c337fe700 1 -- 172.21.15.121:0/2884566694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c3415be30 msgr2=0x7f2c34065d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:13.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.221+0000 7f2c337fe700 1 --2- 172.21.15.121:0/2884566694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c3415be30 0x7f2c34065d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:13.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.221+0000 7f2c337fe700 1 -- 172.21.15.121:0/2884566694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c340c3280 msgr2=0x7f2c34156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:13.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.221+0000 7f2c337fe700 1 --2- 172.21.15.121:0/2884566694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c340c3280 0x7f2c34156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:13.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.221+0000 7f2c337fe700 1 -- 172.21.15.121:0/2884566694 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c280092a0 con 0x7f2c34152850 2023-11-15T03:03:13.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.225+0000 7f2c337fe700 1 --2- 172.21.15.121:0/2884566694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c34152850 0x7f2c34155960 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f2c340654e0 tx=0x7f2c28017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:13.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.225+0000 7f2c1bfff700 1 -- 172.21.15.121:0/2884566694 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2800f7e0 con 0x7f2c34152850 2023-11-15T03:03:13.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.225+0000 7f2c1bfff700 1 -- 172.21.15.121:0/2884566694 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2800f940 con 0x7f2c34152850 2023-11-15T03:03:13.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.225+0000 7f2c393f9700 1 -- 172.21.15.121:0/2884566694 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c34156ad0 con 0x7f2c34152850 2023-11-15T03:03:13.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.225+0000 7f2c1bfff700 1 -- 172.21.15.121:0/2884566694 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2801b660 con 0x7f2c34152850 2023-11-15T03:03:13.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.225+0000 7f2c393f9700 1 -- 172.21.15.121:0/2884566694 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c34157090 con 0x7f2c34152850 2023-11-15T03:03:13.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.225+0000 7f2c393f9700 1 -- 172.21.15.121:0/2884566694 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c34153bc0 con 0x7f2c34152850 2023-11-15T03:03:13.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.225+0000 7f2c1bfff700 1 -- 172.21.15.121:0/2884566694 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2c2801b9d0 con 0x7f2c34152850 2023-11-15T03:03:13.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.229+0000 7f2c1bfff700 1 --2- 172.21.15.121:0/2884566694 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c2005a560 0x7f2c2005ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:13.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.229+0000 7f2c1bfff700 1 -- 172.21.15.121:0/2884566694 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f2c280760c0 con 0x7f2c34152850 2023-11-15T03:03:13.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.229+0000 7f2c33fff700 1 --2- 172.21.15.121:0/2884566694 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c2005a560 0x7f2c2005ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:13.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.229+0000 7f2c33fff700 1 --2- 172.21.15.121:0/2884566694 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c2005a560 0x7f2c2005ca20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f2c24003700 tx=0x7f2c24009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:13.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.233+0000 7f2c1bfff700 1 -- 172.21.15.121:0/2884566694 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2c28049b00 con 0x7f2c34152850 2023-11-15T03:03:13.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:13.449+0000 7f2c393f9700 1 -- 172.21.15.121:0/2884566694 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]} v 0) v1 -- 0x7f2c341597e0 con 0x7f2c34152850 2023-11-15T03:03:14.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.261+0000 7f2c1bfff700 1 -- 172.21.15.121:0/2884566694 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]}]=0 v309) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2c2804d120 con 0x7f2c34152850 2023-11-15T03:03:14.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.265+0000 7f2c393f9700 1 -- 172.21.15.121:0/2884566694 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c2005a560 msgr2=0x7f2c2005ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:14.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.265+0000 7f2c393f9700 1 --2- 172.21.15.121:0/2884566694 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c2005a560 0x7f2c2005ca20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f2c24003700 tx=0x7f2c24009040).stop 2023-11-15T03:03:14.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.265+0000 7f2c393f9700 1 -- 172.21.15.121:0/2884566694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c34152850 msgr2=0x7f2c34155960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:14.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.265+0000 7f2c393f9700 1 --2- 172.21.15.121:0/2884566694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c34152850 0x7f2c34155960 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f2c340654e0 tx=0x7f2c28017000).stop 2023-11-15T03:03:14.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.265+0000 7f2c393f9700 1 -- 172.21.15.121:0/2884566694 shutdown_connections 2023-11-15T03:03:14.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.265+0000 7f2c393f9700 1 --2- 172.21.15.121:0/2884566694 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c2005a560 0x7f2c2005ca20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:14.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.265+0000 7f2c393f9700 1 --2- 172.21.15.121:0/2884566694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c3415be30 0x7f2c34065d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:14.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.265+0000 7f2c393f9700 1 --2- 172.21.15.121:0/2884566694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c34152850 0x7f2c34155960 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:14.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.265+0000 7f2c393f9700 1 --2- 172.21.15.121:0/2884566694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c340c3280 0x7f2c34156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:14.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.265+0000 7f2c393f9700 1 -- 172.21.15.121:0/2884566694 >> 172.21.15.121:0/2884566694 conn(0x7f2c34062190 msgr2=0x7f2c34159a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:14.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.265+0000 7f2c393f9700 1 -- 172.21.15.121:0/2884566694 shutdown_connections 2023-11-15T03:03:14.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.265+0000 7f2c393f9700 1 -- 172.21.15.121:0/2884566694 wait complete. 2023-11-15T03:03:14.298 INFO:tasks.cephfs_test_runner:That the MDS won't write a dentry with new damage to CDentry::first ... ok 2023-11-15T03:03:14.299 INFO:tasks.cephfs_test_runner:Starting test: test_object_deletion (tasks.cephfs.test_damage.TestDamage) That the MDS has a clean 'damaged' response to loss of any single metadata object 2023-11-15T03:03:14.299 INFO:tasks.cephfs_test_runner:test_object_deletion (tasks.cephfs.test_damage.TestDamage) 2023-11-15T03:03:14.301 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion' 2023-11-15T03:03:14.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.469+0000 7f3ad6bb9700 1 Processor -- start 2023-11-15T03:03:14.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.469+0000 7f3ad6bb9700 1 -- start start 2023-11-15T03:03:14.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.469+0000 7f3ad6bb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad00c3280 0x7f3ad00c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:14.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.469+0000 7f3ad6bb9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3ad0152140 0x7f3ad01525c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:14.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.469+0000 7f3ad6bb9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3ad015be30 0x7f3ad015c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:14.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.469+0000 7f3ad6bb9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad00c1860 con 0x7f3ad015be30 2023-11-15T03:03:14.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.469+0000 7f3ad6bb9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad00c1d50 con 0x7f3ad0152140 2023-11-15T03:03:14.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.469+0000 7f3ad6bb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad015cb80 con 0x7f3ad00c3280 2023-11-15T03:03:14.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad63b8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3ad015be30 0x7f3ad015c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:14.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad63b8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3ad015be30 0x7f3ad015c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32852/0 (socket says 172.21.15.121:32852) 2023-11-15T03:03:14.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad63b8700 1 -- 172.21.15.121:0/1111848438 learned_addr learned my addr 172.21.15.121:0/1111848438 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:14.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad5bb7700 1 --2- 172.21.15.121:0/1111848438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad00c3280 0x7f3ad00c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:14.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad53b6700 1 --2- 172.21.15.121:0/1111848438 >> v2:172.21.15.190:3300/0 conn(0x7f3ad0152140 0x7f3ad01525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:14.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad63b8700 1 -- 172.21.15.121:0/1111848438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad00c3280 msgr2=0x7f3ad00c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:14.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad63b8700 1 --2- 172.21.15.121:0/1111848438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad00c3280 0x7f3ad00c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:14.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad63b8700 1 -- 172.21.15.121:0/1111848438 >> v2:172.21.15.190:3300/0 conn(0x7f3ad0152140 msgr2=0x7f3ad01525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:14.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad63b8700 1 --2- 172.21.15.121:0/1111848438 >> v2:172.21.15.190:3300/0 conn(0x7f3ad0152140 0x7f3ad01525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:14.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad63b8700 1 -- 172.21.15.121:0/1111848438 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad00c1f80 con 0x7f3ad015be30 2023-11-15T03:03:14.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad63b8700 1 --2- 172.21.15.121:0/1111848438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad015be30 0x7f3ad015c2a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f3ad00c7780 tx=0x7f3ac8009840).ready entity=mon.0 client_cookie=bca30dd1773701be server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:14.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad4bb5700 1 -- 172.21.15.121:0/1111848438 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac800e070 con 0x7f3ad015be30 2023-11-15T03:03:14.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad4bb5700 1 -- 172.21.15.121:0/1111848438 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac800b930 con 0x7f3ad015be30 2023-11-15T03:03:14.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad4bb5700 1 -- 172.21.15.121:0/1111848438 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac800bcb0 con 0x7f3ad015be30 2023-11-15T03:03:14.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad6bb9700 1 -- 172.21.15.121:0/1111848438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad015be30 msgr2=0x7f3ad015c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:14.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad6bb9700 1 --2- 172.21.15.121:0/1111848438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad015be30 0x7f3ad015c2a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f3ad00c7780 tx=0x7f3ac8009840).stop 2023-11-15T03:03:14.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad6bb9700 1 -- 172.21.15.121:0/1111848438 shutdown_connections 2023-11-15T03:03:14.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad6bb9700 1 --2- 172.21.15.121:0/1111848438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad015be30 0x7f3ad015c2a0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:14.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad6bb9700 1 --2- 172.21.15.121:0/1111848438 >> v2:172.21.15.190:3300/0 conn(0x7f3ad0152140 0x7f3ad01525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:14.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad6bb9700 1 --2- 172.21.15.121:0/1111848438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad00c3280 0x7f3ad00c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:14.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.473+0000 7f3ad6bb9700 1 -- 172.21.15.121:0/1111848438 >> 172.21.15.121:0/1111848438 conn(0x7f3ad0062190 msgr2=0x7f3ad0062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:14.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.477+0000 7f3ad6bb9700 1 -- 172.21.15.121:0/1111848438 shutdown_connections 2023-11-15T03:03:14.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.477+0000 7f3ad6bb9700 1 -- 172.21.15.121:0/1111848438 wait complete. 2023-11-15T03:03:14.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.477+0000 7f3ad6bb9700 1 Processor -- start 2023-11-15T03:03:14.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.477+0000 7f3ad6bb9700 1 -- start start 2023-11-15T03:03:14.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.477+0000 7f3ad6bb9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad00c3280 0x7f3ad00c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:14.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.477+0000 7f3ad6bb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad0152140 0x7f3ad0156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:14.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.477+0000 7f3ad6bb9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad015be30 0x7f3ad01430f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:14.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.477+0000 7f3ad6bb9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad0157660 con 0x7f3ad00c3280 2023-11-15T03:03:14.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.477+0000 7f3ad6bb9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad0156a50 con 0x7f3ad015be30 2023-11-15T03:03:14.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.477+0000 7f3ad6bb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad0156d30 con 0x7f3ad0152140 2023-11-15T03:03:14.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.477+0000 7f3ad5bb7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad00c3280 0x7f3ad00c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:14.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.481+0000 7f3ad5bb7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad00c3280 0x7f3ad00c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:32856/0 (socket says 172.21.15.121:32856) 2023-11-15T03:03:14.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.481+0000 7f3ad5bb7700 1 -- 172.21.15.121:0/4161387744 learned_addr learned my addr 172.21.15.121:0/4161387744 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:14.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.481+0000 7f3ad63b8700 1 --2- 172.21.15.121:0/4161387744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad015be30 0x7f3ad01430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:14.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.481+0000 7f3ad53b6700 1 --2- 172.21.15.121:0/4161387744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad0152140 0x7f3ad0156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:14.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.481+0000 7f3ad5bb7700 1 -- 172.21.15.121:0/4161387744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad0152140 msgr2=0x7f3ad0156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:14.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.481+0000 7f3ad5bb7700 1 --2- 172.21.15.121:0/4161387744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad0152140 0x7f3ad0156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:14.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.481+0000 7f3ad5bb7700 1 -- 172.21.15.121:0/4161387744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad015be30 msgr2=0x7f3ad01430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:14.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.481+0000 7f3ad5bb7700 1 --2- 172.21.15.121:0/4161387744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad015be30 0x7f3ad01430f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:14.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.481+0000 7f3ad5bb7700 1 -- 172.21.15.121:0/4161387744 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac8009ae0 con 0x7f3ad00c3280 2023-11-15T03:03:14.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.481+0000 7f3ad5bb7700 1 --2- 172.21.15.121:0/4161387744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad00c3280 0x7f3ad00c71c0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f3acc00e930 tx=0x7f3acc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:14.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.481+0000 7f3ac5ffb700 1 -- 172.21.15.121:0/4161387744 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc00eed0 con 0x7f3ad00c3280 2023-11-15T03:03:14.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.481+0000 7f3ac5ffb700 1 -- 172.21.15.121:0/4161387744 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3acc00c850 con 0x7f3ad00c3280 2023-11-15T03:03:14.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.481+0000 7f3ac5ffb700 1 -- 172.21.15.121:0/4161387744 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc00cbd0 con 0x7f3ad00c3280 2023-11-15T03:03:14.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.481+0000 7f3ad6bb9700 1 -- 172.21.15.121:0/4161387744 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad0157010 con 0x7f3ad00c3280 2023-11-15T03:03:14.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.481+0000 7f3ad6bb9700 1 -- 172.21.15.121:0/4161387744 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad0156750 con 0x7f3ad00c3280 2023-11-15T03:03:14.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.481+0000 7f3ad6bb9700 1 -- 172.21.15.121:0/4161387744 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad00c2a60 con 0x7f3ad00c3280 2023-11-15T03:03:14.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.485+0000 7f3ac5ffb700 1 -- 172.21.15.121:0/4161387744 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3acc01b400 con 0x7f3ad00c3280 2023-11-15T03:03:14.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.485+0000 7f3ac5ffb700 1 --2- 172.21.15.121:0/4161387744 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ab405a510 0x7f3ab405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:14.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.485+0000 7f3ac5ffb700 1 -- 172.21.15.121:0/4161387744 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f3acc074a40 con 0x7f3ad00c3280 2023-11-15T03:03:14.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.485+0000 7f3ad53b6700 1 --2- 172.21.15.121:0/4161387744 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ab405a510 0x7f3ab405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:14.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.485+0000 7f3ad53b6700 1 --2- 172.21.15.121:0/4161387744 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ab405a510 0x7f3ab405c9d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f3ad00c6a90 tx=0x7f3ac0008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:14.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.493+0000 7f3ac5ffb700 1 -- 172.21.15.121:0/4161387744 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3ad00c2a60 con 0x7f3ad00c3280 2023-11-15T03:03:14.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:14.709+0000 7f3ad6bb9700 1 -- 172.21.15.121:0/4161387744 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]} v 0) v1 -- 0x7f3ad00c2a60 con 0x7f3ad00c3280 2023-11-15T03:03:15.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.265+0000 7f3ac5ffb700 1 -- 172.21.15.121:0/4161387744 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]}]=0 v310) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3ad00c2a60 con 0x7f3ad00c3280 2023-11-15T03:03:15.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.269+0000 7f3ad6bb9700 1 -- 172.21.15.121:0/4161387744 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ab405a510 msgr2=0x7f3ab405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:15.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.269+0000 7f3ad6bb9700 1 --2- 172.21.15.121:0/4161387744 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ab405a510 0x7f3ab405c9d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f3ad00c6a90 tx=0x7f3ac0008040).stop 2023-11-15T03:03:15.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.269+0000 7f3ad6bb9700 1 -- 172.21.15.121:0/4161387744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad00c3280 msgr2=0x7f3ad00c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:15.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.269+0000 7f3ad6bb9700 1 --2- 172.21.15.121:0/4161387744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad00c3280 0x7f3ad00c71c0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f3acc00e930 tx=0x7f3acc009060).stop 2023-11-15T03:03:15.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.269+0000 7f3ad6bb9700 1 -- 172.21.15.121:0/4161387744 shutdown_connections 2023-11-15T03:03:15.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.269+0000 7f3ad6bb9700 1 --2- 172.21.15.121:0/4161387744 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ab405a510 0x7f3ab405c9d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:15.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.269+0000 7f3ad6bb9700 1 --2- 172.21.15.121:0/4161387744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad015be30 0x7f3ad01430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:15.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.269+0000 7f3ad6bb9700 1 --2- 172.21.15.121:0/4161387744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad0152140 0x7f3ad0156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:15.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.269+0000 7f3ad6bb9700 1 --2- 172.21.15.121:0/4161387744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ad00c3280 0x7f3ad00c71c0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:15.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.269+0000 7f3ad6bb9700 1 -- 172.21.15.121:0/4161387744 >> 172.21.15.121:0/4161387744 conn(0x7f3ad0062190 msgr2=0x7f3ad00a8230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:15.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.273+0000 7f3ad6bb9700 1 -- 172.21.15.121:0/4161387744 shutdown_connections 2023-11-15T03:03:15.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.273+0000 7f3ad6bb9700 1 -- 172.21.15.121:0/4161387744 wait complete. 2023-11-15T03:03:15.300 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-15T03:03:15.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.465+0000 7f1154df9700 1 Processor -- start 2023-11-15T03:03:15.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.465+0000 7f1154df9700 1 -- start start 2023-11-15T03:03:15.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.469+0000 7f1154df9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f11500a8d40 0x7f11500c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:15.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.469+0000 7f1154df9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1150066520 0x7f1150066990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:15.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.469+0000 7f1154df9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11500657f0 con 0x7f11500a8960 2023-11-15T03:03:15.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.469+0000 7f1154df9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11501583d0 con 0x7f11500a8d40 2023-11-15T03:03:15.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.469+0000 7f1154df9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11500c1b60 con 0x7f1150066520 2023-11-15T03:03:15.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.469+0000 7f114f7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f11500a8960 0x7f115015c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40860/0 (socket says 172.21.15.121:40860) 2023-11-15T03:03:15.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.469+0000 7f114f7fe700 1 -- 172.21.15.121:0/2907086881 learned_addr learned my addr 172.21.15.121:0/2907086881 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:15.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.469+0000 7f114ffff700 1 --2- 172.21.15.121:0/2907086881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1150066520 0x7f1150066990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:15.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.469+0000 7f114effd700 1 --2- 172.21.15.121:0/2907086881 >> v2:172.21.15.190:3300/0 conn(0x7f11500a8d40 0x7f11500c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:15.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.469+0000 7f114e7fc700 1 -- 172.21.15.121:0/2907086881 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978031138 0 0) 0x7f11500657f0 con 0x7f11500a8960 2023-11-15T03:03:15.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.469+0000 7f114e7fc700 1 -- 172.21.15.121:0/2907086881 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11380033f0 con 0x7f11500a8960 2023-11-15T03:03:15.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.469+0000 7f114effd700 1 -- 172.21.15.121:0/2907086881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1150066520 msgr2=0x7f1150066990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:15.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.469+0000 7f114effd700 1 --2- 172.21.15.121:0/2907086881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1150066520 0x7f1150066990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:15.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.469+0000 7f114e7fc700 1 -- 172.21.15.121:0/2907086881 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3826981280 0 0) 0x7f11380033f0 con 0x7f11500a8960 2023-11-15T03:03:15.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.469+0000 7f114effd700 1 -- 172.21.15.121:0/2907086881 >> v1:172.21.15.121:6789/0 conn(0x7f11500a8960 legacy=0x7f115015c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:15.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.469+0000 7f114effd700 1 -- 172.21.15.121:0/2907086881 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1150065a40 con 0x7f11500a8d40 2023-11-15T03:03:15.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.469+0000 7f114effd700 1 --2- 172.21.15.121:0/2907086881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11500a8d40 0x7f11500c15c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f1144009760 tx=0x7f1144009060).ready entity=mon.1 client_cookie=2ecac1f6c7de29b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:15.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.469+0000 7f114e7fc700 1 -- 172.21.15.121:0/2907086881 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140001470 con 0x7f11500a8d40 2023-11-15T03:03:15.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.473+0000 7f114e7fc700 1 -- 172.21.15.121:0/2907086881 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1144011400 con 0x7f11500a8d40 2023-11-15T03:03:15.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.473+0000 7f114e7fc700 1 -- 172.21.15.121:0/2907086881 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1144011780 con 0x7f11500a8d40 2023-11-15T03:03:15.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.473+0000 7f1154df9700 1 -- 172.21.15.121:0/2907086881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11500a8d40 msgr2=0x7f11500c15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:15.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.473+0000 7f1154df9700 1 --2- 172.21.15.121:0/2907086881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11500a8d40 0x7f11500c15c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f1144009760 tx=0x7f1144009060).stop 2023-11-15T03:03:15.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.473+0000 7f1154df9700 1 -- 172.21.15.121:0/2907086881 shutdown_connections 2023-11-15T03:03:15.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.473+0000 7f1154df9700 1 --2- 172.21.15.121:0/2907086881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1150066520 0x7f1150066990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:15.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.473+0000 7f1154df9700 1 --2- 172.21.15.121:0/2907086881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11500a8d40 0x7f11500c15c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:15.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.473+0000 7f1154df9700 1 -- 172.21.15.121:0/2907086881 >> 172.21.15.121:0/2907086881 conn(0x7f1150067a60 msgr2=0x7f1150067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:15.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.473+0000 7f1154df9700 1 -- 172.21.15.121:0/2907086881 shutdown_connections 2023-11-15T03:03:15.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.473+0000 7f1154df9700 1 -- 172.21.15.121:0/2907086881 wait complete. 2023-11-15T03:03:15.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.473+0000 7f1154df9700 1 Processor -- start 2023-11-15T03:03:15.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f1154df9700 1 -- start start 2023-11-15T03:03:15.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f1154df9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1150066520 0x7f11500bba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:15.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f1154df9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11500a8960 0x7f11500bd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:15.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f1154df9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11500a8d40 0x7f11500aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:15.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f1154df9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11500be800 con 0x7f11500a8960 2023-11-15T03:03:15.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f1154df9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11500ac2f0 con 0x7f11500a8d40 2023-11-15T03:03:15.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f1154df9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11500bdbf0 con 0x7f1150066520 2023-11-15T03:03:15.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f114effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11500a8960 0x7f11500bd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:15.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f114effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11500a8960 0x7f11500bd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41172/0 (socket says 172.21.15.121:41172) 2023-11-15T03:03:15.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f114effd700 1 -- 172.21.15.121:0/2266240951 learned_addr learned my addr 172.21.15.121:0/2266240951 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:15.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f114ffff700 1 --2- 172.21.15.121:0/2266240951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11500a8d40 0x7f11500aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:15.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f114f7fe700 1 --2- 172.21.15.121:0/2266240951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1150066520 0x7f11500bba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:15.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f114effd700 1 -- 172.21.15.121:0/2266240951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1150066520 msgr2=0x7f11500bba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:15.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f114effd700 1 --2- 172.21.15.121:0/2266240951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1150066520 0x7f11500bba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:15.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f114effd700 1 -- 172.21.15.121:0/2266240951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11500a8d40 msgr2=0x7f11500aa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:15.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f114effd700 1 --2- 172.21.15.121:0/2266240951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11500a8d40 0x7f11500aa9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:15.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f114effd700 1 -- 172.21.15.121:0/2266240951 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11440092a0 con 0x7f11500a8960 2023-11-15T03:03:15.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f114ffff700 1 --2- 172.21.15.121:0/2266240951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11500a8d40 0x7f11500aa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:15.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f114effd700 1 --2- 172.21.15.121:0/2266240951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11500a8960 0x7f11500bd2e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7f1144009d50 tx=0x7f1144016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:15.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f11377fe700 1 -- 172.21.15.121:0/2266240951 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1144010870 con 0x7f11500a8960 2023-11-15T03:03:15.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f11377fe700 1 -- 172.21.15.121:0/2266240951 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11440109d0 con 0x7f11500a8960 2023-11-15T03:03:15.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f1154df9700 1 -- 172.21.15.121:0/2266240951 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11500bded0 con 0x7f11500a8960 2023-11-15T03:03:15.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f11377fe700 1 -- 172.21.15.121:0/2266240951 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1144010d90 con 0x7f11500a8960 2023-11-15T03:03:15.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.477+0000 7f1154df9700 1 -- 172.21.15.121:0/2266240951 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11500be490 con 0x7f11500a8960 2023-11-15T03:03:15.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.481+0000 7f1154df9700 1 -- 172.21.15.121:0/2266240951 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115006bcf0 con 0x7f11500a8960 2023-11-15T03:03:15.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.481+0000 7f11377fe700 1 -- 172.21.15.121:0/2266240951 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f11440263b0 con 0x7f11500a8960 2023-11-15T03:03:15.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.481+0000 7f11377fe700 1 --2- 172.21.15.121:0/2266240951 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f113c05a510 0x7f113c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:15.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.481+0000 7f11377fe700 1 -- 172.21.15.121:0/2266240951 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f1144075300 con 0x7f11500a8960 2023-11-15T03:03:15.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.485+0000 7f114f7fe700 1 --2- 172.21.15.121:0/2266240951 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f113c05a510 0x7f113c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:15.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.485+0000 7f114f7fe700 1 --2- 172.21.15.121:0/2266240951 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f113c05a510 0x7f113c05c9d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f11400011e0 tx=0x7f1140001520).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:15.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.489+0000 7f11377fe700 1 -- 172.21.15.121:0/2266240951 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1144048d20 con 0x7f11500a8960 2023-11-15T03:03:15.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.657+0000 7f1154df9700 1 -- 172.21.15.121:0/2266240951 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f11500a7740 con 0x7f11500a8960 2023-11-15T03:03:15.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.665+0000 7f11377fe700 1 -- 172.21.15.121:0/2266240951 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1144022070 con 0x7f11500a8960 2023-11-15T03:03:15.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.673+0000 7f1154df9700 1 -- 172.21.15.121:0/2266240951 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f113c05a510 msgr2=0x7f113c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:15.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.673+0000 7f1154df9700 1 --2- 172.21.15.121:0/2266240951 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f113c05a510 0x7f113c05c9d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f11400011e0 tx=0x7f1140001520).stop 2023-11-15T03:03:15.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.673+0000 7f1154df9700 1 -- 172.21.15.121:0/2266240951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11500a8960 msgr2=0x7f11500bd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:15.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.673+0000 7f1154df9700 1 --2- 172.21.15.121:0/2266240951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11500a8960 0x7f11500bd2e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7f1144009d50 tx=0x7f1144016000).stop 2023-11-15T03:03:15.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.673+0000 7f1154df9700 1 -- 172.21.15.121:0/2266240951 shutdown_connections 2023-11-15T03:03:15.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.673+0000 7f1154df9700 1 --2- 172.21.15.121:0/2266240951 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f113c05a510 0x7f113c05c9d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:15.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.673+0000 7f1154df9700 1 --2- 172.21.15.121:0/2266240951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11500a8d40 0x7f11500aa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:15.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.673+0000 7f1154df9700 1 --2- 172.21.15.121:0/2266240951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11500a8960 0x7f11500bd2e0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:15.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.673+0000 7f1154df9700 1 --2- 172.21.15.121:0/2266240951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1150066520 0x7f11500bba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:15.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.673+0000 7f1154df9700 1 -- 172.21.15.121:0/2266240951 >> 172.21.15.121:0/2266240951 conn(0x7f1150067a60 msgr2=0x7f11500a98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:15.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.673+0000 7f1154df9700 1 -- 172.21.15.121:0/2266240951 shutdown_connections 2023-11-15T03:03:15.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.673+0000 7f1154df9700 1 -- 172.21.15.121:0/2266240951 wait complete. 2023-11-15T03:03:15.698 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-15T03:03:15.698 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T03:03:15.741 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T03:03:15.788 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-15T03:03:15.789 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-15T03:03:15.789 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:03:15.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.945+0000 7fd38d379700 1 Processor -- start 2023-11-15T03:03:15.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.945+0000 7fd38d379700 1 -- start start 2023-11-15T03:03:15.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.945+0000 7fd38d379700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd388066320 0x7fd388154110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:15.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.945+0000 7fd38d379700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd388066700 0x7fd3880656a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:15.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.945+0000 7fd38d379700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3880c73e0 con 0x7fd3880acf10 2023-11-15T03:03:15.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.945+0000 7fd38d379700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3881546c0 con 0x7fd388066320 2023-11-15T03:03:15.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.945+0000 7fd38d379700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3881549a0 con 0x7fd388066700 2023-11-15T03:03:15.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd387fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd388066320 0x7fd388154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:15.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd37f7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd388066700 0x7fd3880656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:15.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd387fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd388066320 0x7fd388154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:34486/0 (socket says 172.21.15.121:34486) 2023-11-15T03:03:15.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd387fff700 1 -- 172.21.15.121:0/3964257792 learned_addr learned my addr 172.21.15.121:0/3964257792 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:15.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd37f7fe700 1 -- 172.21.15.121:0/3964257792 >> v2:172.21.15.190:3300/0 conn(0x7fd388066320 msgr2=0x7fd388154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:15.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd37f7fe700 1 --2- 172.21.15.121:0/3964257792 >> v2:172.21.15.190:3300/0 conn(0x7fd388066320 0x7fd388154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:15.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd37f7fe700 1 -- 172.21.15.121:0/3964257792 >> v1:172.21.15.190:6789/0 conn(0x7fd3880acf10 legacy=0x7fd388062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:15.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd37f7fe700 1 -- 172.21.15.121:0/3964257792 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd388154c80 con 0x7fd388066700 2023-11-15T03:03:15.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd3877fe700 1 -- 172.21.15.121:0/3964257792 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931061555 0 0) 0x7fd3880c73e0 con 0x7fd3880acf10 2023-11-15T03:03:15.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd37f7fe700 1 --2- 172.21.15.121:0/3964257792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd388066700 0x7fd3880656a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fd370009760 tx=0x7fd370009060).ready entity=mon.2 client_cookie=df8a66e2f0ab5c67 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:15.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd3877fe700 1 -- 172.21.15.121:0/3964257792 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37000f510 con 0x7fd388066700 2023-11-15T03:03:15.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd3877fe700 1 -- 172.21.15.121:0/3964257792 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37000f670 con 0x7fd388066700 2023-11-15T03:03:15.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd3877fe700 1 -- 172.21.15.121:0/3964257792 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37000f9f0 con 0x7fd388066700 2023-11-15T03:03:15.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd38d379700 1 -- 172.21.15.121:0/3964257792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd388066700 msgr2=0x7fd3880656a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:15.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd38d379700 1 --2- 172.21.15.121:0/3964257792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd388066700 0x7fd3880656a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fd370009760 tx=0x7fd370009060).stop 2023-11-15T03:03:15.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd38d379700 1 -- 172.21.15.121:0/3964257792 shutdown_connections 2023-11-15T03:03:15.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd38d379700 1 --2- 172.21.15.121:0/3964257792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd388066700 0x7fd3880656a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:15.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd38d379700 1 --2- 172.21.15.121:0/3964257792 >> v2:172.21.15.190:3300/0 conn(0x7fd388066320 0x7fd388154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:15.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd38d379700 1 -- 172.21.15.121:0/3964257792 >> 172.21.15.121:0/3964257792 conn(0x7fd388067a60 msgr2=0x7fd388067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:15.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.949+0000 7fd38d379700 1 -- 172.21.15.121:0/3964257792 shutdown_connections 2023-11-15T03:03:15.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.953+0000 7fd38d379700 1 -- 172.21.15.121:0/3964257792 wait complete. 2023-11-15T03:03:15.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.953+0000 7fd38d379700 1 Processor -- start 2023-11-15T03:03:15.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.953+0000 7fd38d379700 1 -- start start 2023-11-15T03:03:15.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.953+0000 7fd38d379700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd388066320 0x7fd38814aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:15.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.953+0000 7fd38d379700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd388066700 0x7fd3881482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:15.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.953+0000 7fd38d379700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3880acf10 0x7fd38814cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:15.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.953+0000 7fd38d379700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38814e610 con 0x7fd388066320 2023-11-15T03:03:15.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.953+0000 7fd38d379700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38814da00 con 0x7fd388066700 2023-11-15T03:03:15.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.953+0000 7fd38d379700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38814dce0 con 0x7fd3880acf10 2023-11-15T03:03:15.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.957+0000 7fd387fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd388066320 0x7fd38814aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:15.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.957+0000 7fd387fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd388066320 0x7fd38814aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41178/0 (socket says 172.21.15.121:41178) 2023-11-15T03:03:15.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.957+0000 7fd387fff700 1 -- 172.21.15.121:0/4030123444 learned_addr learned my addr 172.21.15.121:0/4030123444 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:15.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.957+0000 7fd37f7fe700 1 --2- 172.21.15.121:0/4030123444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd388066700 0x7fd3881482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:15.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.957+0000 7fd38cb78700 1 --2- 172.21.15.121:0/4030123444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3880acf10 0x7fd38814cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:15.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.957+0000 7fd387fff700 1 -- 172.21.15.121:0/4030123444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3880acf10 msgr2=0x7fd38814cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:15.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.957+0000 7fd387fff700 1 --2- 172.21.15.121:0/4030123444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3880acf10 0x7fd38814cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:15.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.957+0000 7fd387fff700 1 -- 172.21.15.121:0/4030123444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd388066700 msgr2=0x7fd3881482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:15.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.957+0000 7fd387fff700 1 --2- 172.21.15.121:0/4030123444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd388066700 0x7fd3881482c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:15.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.957+0000 7fd387fff700 1 -- 172.21.15.121:0/4030123444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3700092a0 con 0x7fd388066320 2023-11-15T03:03:15.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.957+0000 7fd37f7fe700 1 --2- 172.21.15.121:0/4030123444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd388066700 0x7fd3881482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:15.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.957+0000 7fd38cb78700 1 --2- 172.21.15.121:0/4030123444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3880acf10 0x7fd38814cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:15.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.957+0000 7fd387fff700 1 --2- 172.21.15.121:0/4030123444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd388066320 0x7fd38814aab0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7fd37800e900 tx=0x7fd3780090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:15.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.957+0000 7fd384ff9700 1 -- 172.21.15.121:0/4030123444 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37800ee40 con 0x7fd388066320 2023-11-15T03:03:15.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.957+0000 7fd384ff9700 1 -- 172.21.15.121:0/4030123444 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37800c850 con 0x7fd388066320 2023-11-15T03:03:15.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.957+0000 7fd38d379700 1 -- 172.21.15.121:0/4030123444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38814dfc0 con 0x7fd388066320 2023-11-15T03:03:15.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.957+0000 7fd384ff9700 1 -- 172.21.15.121:0/4030123444 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37800cbd0 con 0x7fd388066320 2023-11-15T03:03:15.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.957+0000 7fd38d379700 1 -- 172.21.15.121:0/4030123444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38814afa0 con 0x7fd388066320 2023-11-15T03:03:15.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.957+0000 7fd38d379700 1 -- 172.21.15.121:0/4030123444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38806bcf0 con 0x7fd388066320 2023-11-15T03:03:15.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.961+0000 7fd384ff9700 1 -- 172.21.15.121:0/4030123444 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd37801b400 con 0x7fd388066320 2023-11-15T03:03:15.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.961+0000 7fd384ff9700 1 --2- 172.21.15.121:0/4030123444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd36805a510 0x7fd36805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:15.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.961+0000 7fd384ff9700 1 -- 172.21.15.121:0/4030123444 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5441+0+0 (secure 0 0 0) 0x7fd378074a80 con 0x7fd388066320 2023-11-15T03:03:15.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.961+0000 7fd37f7fe700 1 --2- 172.21.15.121:0/4030123444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd36805a510 0x7fd36805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:15.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.965+0000 7fd37f7fe700 1 --2- 172.21.15.121:0/4030123444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd36805a510 0x7fd36805c9d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fd370009760 tx=0x7fd370017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:15.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:15.969+0000 7fd384ff9700 1 -- 172.21.15.121:0/4030123444 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd3780483e0 con 0x7fd388066320 2023-11-15T03:03:16.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.165+0000 7fd38d379700 1 -- 172.21.15.121:0/4030123444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd388154f60 con 0x7fd388066320 2023-11-15T03:03:16.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.165+0000 7fd384ff9700 1 -- 172.21.15.121:0/4030123444 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+3541 (secure 0 0 0) 0x7fd378016030 con 0x7fd388066320 2023-11-15T03:03:16.171 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:16.172 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":75,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1508681155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1508681155},{"type":"v1","addr":"172.21.15.121:6837","nonce":1508681155}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6385,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3559276534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3559276534},{"type":"v1","addr":"172.21.15.190:6835","nonce":3559276534}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1842535107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1842535107},{"type":"v1","addr":"172.21.15.190:6833","nonce":1842535107}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":6906,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3563484616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3563484616},{"type":"v1","addr":"172.21.15.121:6836","nonce":3563484616}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[]} 2023-11-15T03:03:16.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.169+0000 7fd38d379700 1 -- 172.21.15.121:0/4030123444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd36805a510 msgr2=0x7fd36805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:16.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.169+0000 7fd38d379700 1 --2- 172.21.15.121:0/4030123444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd36805a510 0x7fd36805c9d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fd370009760 tx=0x7fd370017000).stop 2023-11-15T03:03:16.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.169+0000 7fd38d379700 1 -- 172.21.15.121:0/4030123444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd388066320 msgr2=0x7fd38814aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:16.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.169+0000 7fd38d379700 1 --2- 172.21.15.121:0/4030123444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd388066320 0x7fd38814aab0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7fd37800e900 tx=0x7fd3780090f0).stop 2023-11-15T03:03:16.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.169+0000 7fd38d379700 1 -- 172.21.15.121:0/4030123444 shutdown_connections 2023-11-15T03:03:16.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.169+0000 7fd38d379700 1 --2- 172.21.15.121:0/4030123444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd36805a510 0x7fd36805c9d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:16.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.169+0000 7fd38d379700 1 --2- 172.21.15.121:0/4030123444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3880acf10 0x7fd38814cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:16.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.169+0000 7fd38d379700 1 --2- 172.21.15.121:0/4030123444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd388066700 0x7fd3881482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:16.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.169+0000 7fd38d379700 1 --2- 172.21.15.121:0/4030123444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd388066320 0x7fd38814aab0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:16.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.169+0000 7fd38d379700 1 -- 172.21.15.121:0/4030123444 >> 172.21.15.121:0/4030123444 conn(0x7fd388067a60 msgr2=0x7fd3881529d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:16.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.169+0000 7fd38d379700 1 -- 172.21.15.121:0/4030123444 shutdown_connections 2023-11-15T03:03:16.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.169+0000 7fd38d379700 1 -- 172.21.15.121:0/4030123444 wait complete. 2023-11-15T03:03:16.177 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 75 2023-11-15T03:03:16.191 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-15T03:03:16.191 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-15T03:03:16.192 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-15T03:03:16.192 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:03:16.192 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-15T03:03:16.193 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-15T03:03:16.193 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-15T03:03:16.193 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:03:16.194 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-15T03:03:16.194 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-15T03:03:16.194 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-15T03:03:16.195 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:03:16.195 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-15T03:03:16.195 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-15T03:03:16.196 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-15T03:03:16.196 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:03:16.211 INFO:tasks.ceph.mds.c:Stopped 2023-11-15T03:03:16.211 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-15T03:03:16.219 INFO:tasks.ceph.mds.d:Stopped 2023-11-15T03:03:16.219 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-15T03:03:16.220 INFO:tasks.ceph.mds.b:Stopped 2023-11-15T03:03:16.221 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-15T03:03:16.222 INFO:tasks.ceph.mds.a:Stopped 2023-11-15T03:03:16.222 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-15T03:03:16.270 INFO:tasks.ceph.mds.d:Started 2023-11-15T03:03:16.270 DEBUG:teuthology.parallel:result is None 2023-11-15T03:03:16.271 INFO:tasks.ceph.mds.c:Started 2023-11-15T03:03:16.272 DEBUG:teuthology.parallel:result is None 2023-11-15T03:03:16.275 INFO:tasks.ceph.mds.b:Started 2023-11-15T03:03:16.276 DEBUG:teuthology.parallel:result is None 2023-11-15T03:03:16.276 INFO:tasks.ceph.mds.a:Started 2023-11-15T03:03:16.277 DEBUG:teuthology.parallel:result is None 2023-11-15T03:03:16.277 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-15T03:03:16.370 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:03:16.371 INFO:tasks.ceph.mds.c.smithi121.stdout:starting mds.c at 2023-11-15T03:03:16.390 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-11-15T03:03:16.393 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-11-15T03:03:16.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.425+0000 7fa2de1f9700 1 Processor -- start 2023-11-15T03:03:16.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.425+0000 7fa2de1f9700 1 -- start start 2023-11-15T03:03:16.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.425+0000 7fa2de1f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2d8063f20 0x7fa2d8065b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:16.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.425+0000 7fa2de1f9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa2d8064300 0x7fa2d80671b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:16.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.425+0000 7fa2de1f9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d8066d20 con 0x7fa2d80bcf90 2023-11-15T03:03:16.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.425+0000 7fa2de1f9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d80676a0 con 0x7fa2d8064300 2023-11-15T03:03:16.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.425+0000 7fa2de1f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d81540a0 con 0x7fa2d8063f20 2023-11-15T03:03:16.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.425+0000 7fa2dd9f8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa2d80bcf90 0x7fa2d80c1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40882/0 (socket says 172.21.15.121:40882) 2023-11-15T03:03:16.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.425+0000 7fa2dd9f8700 1 -- 172.21.15.121:0/2358135372 learned_addr learned my addr 172.21.15.121:0/2358135372 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:16.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.425+0000 7fa2dd1f7700 1 --2- 172.21.15.121:0/2358135372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2d8063f20 0x7fa2d8065b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:16.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.425+0000 7fa2dc9f6700 1 --2- 172.21.15.121:0/2358135372 >> v2:172.21.15.121:3300/0 conn(0x7fa2d8064300 0x7fa2d80671b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:16.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.429+0000 7fa2cffff700 1 -- 172.21.15.121:0/2358135372 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2710163369 0 0) 0x7fa2d8066d20 con 0x7fa2d80bcf90 2023-11-15T03:03:16.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.429+0000 7fa2dd1f7700 1 -- 172.21.15.121:0/2358135372 >> v2:172.21.15.121:3300/0 conn(0x7fa2d8064300 msgr2=0x7fa2d80671b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:16.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.429+0000 7fa2dd1f7700 1 --2- 172.21.15.121:0/2358135372 >> v2:172.21.15.121:3300/0 conn(0x7fa2d8064300 0x7fa2d80671b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:16.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.429+0000 7fa2dd1f7700 1 -- 172.21.15.121:0/2358135372 >> v1:172.21.15.121:6789/0 conn(0x7fa2d80bcf90 legacy=0x7fa2d80c1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:16.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.429+0000 7fa2dd1f7700 1 -- 172.21.15.121:0/2358135372 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d80bd440 con 0x7fa2d8063f20 2023-11-15T03:03:16.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.429+0000 7fa2dd1f7700 1 --2- 172.21.15.121:0/2358135372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2d8063f20 0x7fa2d8065b70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fa2d4009790 tx=0x7fa2d4009060).ready entity=mon.2 client_cookie=648c7ae3737f9fa5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:16.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.429+0000 7fa2cffff700 1 -- 172.21.15.121:0/2358135372 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d400f510 con 0x7fa2d8063f20 2023-11-15T03:03:16.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.429+0000 7fa2cffff700 1 -- 172.21.15.121:0/2358135372 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d400f670 con 0x7fa2d8063f20 2023-11-15T03:03:16.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.429+0000 7fa2cffff700 1 -- 172.21.15.121:0/2358135372 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d400f9f0 con 0x7fa2d8063f20 2023-11-15T03:03:16.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.429+0000 7fa2de1f9700 1 -- 172.21.15.121:0/2358135372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2d8063f20 msgr2=0x7fa2d8065b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:16.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.429+0000 7fa2de1f9700 1 --2- 172.21.15.121:0/2358135372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2d8063f20 0x7fa2d8065b70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fa2d4009790 tx=0x7fa2d4009060).stop 2023-11-15T03:03:16.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.429+0000 7fa2de1f9700 1 -- 172.21.15.121:0/2358135372 shutdown_connections 2023-11-15T03:03:16.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.429+0000 7fa2de1f9700 1 --2- 172.21.15.121:0/2358135372 >> v2:172.21.15.121:3300/0 conn(0x7fa2d8064300 0x7fa2d80671b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:16.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.429+0000 7fa2de1f9700 1 --2- 172.21.15.121:0/2358135372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2d8063f20 0x7fa2d8065b70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:16.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.429+0000 7fa2de1f9700 1 -- 172.21.15.121:0/2358135372 >> 172.21.15.121:0/2358135372 conn(0x7fa2d80c69b0 msgr2=0x7fa2d80c6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:16.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.429+0000 7fa2de1f9700 1 -- 172.21.15.121:0/2358135372 shutdown_connections 2023-11-15T03:03:16.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.429+0000 7fa2de1f9700 1 -- 172.21.15.121:0/2358135372 wait complete. 2023-11-15T03:03:16.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.433+0000 7fa2de1f9700 1 Processor -- start 2023-11-15T03:03:16.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.433+0000 7fa2de1f9700 1 -- start start 2023-11-15T03:03:16.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.433+0000 7fa2de1f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2d8063f20 0x7fa2d8142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:16.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.433+0000 7fa2de1f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2d8064300 0x7fa2d80abe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:16.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.433+0000 7fa2de1f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2d80bcf90 0x7fa2d80ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:16.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.433+0000 7fa2de1f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d80aca40 con 0x7fa2d8063f20 2023-11-15T03:03:16.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.433+0000 7fa2de1f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d80ac840 con 0x7fa2d8064300 2023-11-15T03:03:16.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.433+0000 7fa2de1f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d81681e0 con 0x7fa2d80bcf90 2023-11-15T03:03:16.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.433+0000 7fa2dd1f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2d8063f20 0x7fa2d8142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:16.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.433+0000 7fa2dd1f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2d8063f20 0x7fa2d8142350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41244/0 (socket says 172.21.15.121:41244) 2023-11-15T03:03:16.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.433+0000 7fa2dc9f6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2d8064300 0x7fa2d80abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:16.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.433+0000 7fa2dd1f7700 1 -- 172.21.15.121:0/2636908045 learned_addr learned my addr 172.21.15.121:0/2636908045 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:16.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.433+0000 7fa2dd9f8700 1 --2- 172.21.15.121:0/2636908045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2d80bcf90 0x7fa2d80ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:16.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.433+0000 7fa2dd1f7700 1 -- 172.21.15.121:0/2636908045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2d80bcf90 msgr2=0x7fa2d80ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:16.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.433+0000 7fa2dd1f7700 1 --2- 172.21.15.121:0/2636908045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2d80bcf90 0x7fa2d80ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:16.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.433+0000 7fa2dd1f7700 1 -- 172.21.15.121:0/2636908045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2d8064300 msgr2=0x7fa2d80abe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:16.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.433+0000 7fa2dd1f7700 1 --2- 172.21.15.121:0/2636908045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2d8064300 0x7fa2d80abe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:16.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.433+0000 7fa2dd1f7700 1 -- 172.21.15.121:0/2636908045 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d40092a0 con 0x7fa2d8063f20 2023-11-15T03:03:16.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.433+0000 7fa2dd1f7700 1 --2- 172.21.15.121:0/2636908045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2d8063f20 0x7fa2d8142350 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7fa2d4009e30 tx=0x7fa2d4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:16.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.437+0000 7fa2cd7fa700 1 -- 172.21.15.121:0/2636908045 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d400f7e0 con 0x7fa2d8063f20 2023-11-15T03:03:16.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.437+0000 7fa2cd7fa700 1 -- 172.21.15.121:0/2636908045 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d400f960 con 0x7fa2d8063f20 2023-11-15T03:03:16.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.437+0000 7fa2cd7fa700 1 -- 172.21.15.121:0/2636908045 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d401b690 con 0x7fa2d8063f20 2023-11-15T03:03:16.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.437+0000 7fa2de1f9700 1 -- 172.21.15.121:0/2636908045 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d81678b0 con 0x7fa2d8063f20 2023-11-15T03:03:16.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.437+0000 7fa2de1f9700 1 -- 172.21.15.121:0/2636908045 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d8167e70 con 0x7fa2d8063f20 2023-11-15T03:03:16.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.437+0000 7fa2de1f9700 1 -- 172.21.15.121:0/2636908045 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d806bcf0 con 0x7fa2d8063f20 2023-11-15T03:03:16.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.437+0000 7fa2cd7fa700 1 -- 172.21.15.121:0/2636908045 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa2d401ba40 con 0x7fa2d8063f20 2023-11-15T03:03:16.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.441+0000 7fa2cd7fa700 1 --2- 172.21.15.121:0/2636908045 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa2bc05a510 0x7fa2bc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:16.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.441+0000 7fa2cd7fa700 1 -- 172.21.15.121:0/2636908045 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5441+0+0 (secure 0 0 0) 0x7fa2d4075b70 con 0x7fa2d8063f20 2023-11-15T03:03:16.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.441+0000 7fa2dc9f6700 1 --2- 172.21.15.121:0/2636908045 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa2bc05a510 0x7fa2bc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:16.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.441+0000 7fa2dc9f6700 1 --2- 172.21.15.121:0/2636908045 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa2bc05a510 0x7fa2bc05c9d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7fa2d81518d0 tx=0x7fa2c8009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:16.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.449+0000 7fa2cd7fa700 1 -- 172.21.15.121:0/2636908045 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa2d40496a0 con 0x7fa2d8063f20 2023-11-15T03:03:16.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:16.621+0000 7fa2de1f9700 1 -- 172.21.15.121:0/2636908045 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa2d80bbe50 con 0x7fa2d8063f20 2023-11-15T03:03:17.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.429+0000 7fa2cd7fa700 1 -- 172.21.15.121:0/2636908045 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v88) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa2d404ccc0 con 0x7fa2d8063f20 2023-11-15T03:03:17.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.433+0000 7fa2de1f9700 1 -- 172.21.15.121:0/2636908045 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa2bc05a510 msgr2=0x7fa2bc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:17.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.433+0000 7fa2de1f9700 1 --2- 172.21.15.121:0/2636908045 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa2bc05a510 0x7fa2bc05c9d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7fa2d81518d0 tx=0x7fa2c8009060).stop 2023-11-15T03:03:17.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.433+0000 7fa2de1f9700 1 -- 172.21.15.121:0/2636908045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2d8063f20 msgr2=0x7fa2d8142350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:17.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.433+0000 7fa2de1f9700 1 --2- 172.21.15.121:0/2636908045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2d8063f20 0x7fa2d8142350 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7fa2d4009e30 tx=0x7fa2d4017000).stop 2023-11-15T03:03:17.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.433+0000 7fa2de1f9700 1 -- 172.21.15.121:0/2636908045 shutdown_connections 2023-11-15T03:03:17.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.433+0000 7fa2de1f9700 1 --2- 172.21.15.121:0/2636908045 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa2bc05a510 0x7fa2bc05c9d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:17.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.433+0000 7fa2de1f9700 1 --2- 172.21.15.121:0/2636908045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2d80bcf90 0x7fa2d80ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:17.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.433+0000 7fa2de1f9700 1 --2- 172.21.15.121:0/2636908045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2d8064300 0x7fa2d80abe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:17.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.433+0000 7fa2de1f9700 1 --2- 172.21.15.121:0/2636908045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2d8063f20 0x7fa2d8142350 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:17.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.433+0000 7fa2de1f9700 1 -- 172.21.15.121:0/2636908045 >> 172.21.15.121:0/2636908045 conn(0x7fa2d80c69b0 msgr2=0x7fa2d80c3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:17.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.433+0000 7fa2de1f9700 1 -- 172.21.15.121:0/2636908045 shutdown_connections 2023-11-15T03:03:17.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.433+0000 7fa2de1f9700 1 -- 172.21.15.121:0/2636908045 wait complete. 2023-11-15T03:03:17.443 INFO:teuthology.orchestra.run.smithi121.stderr: removed all blocklist entries 2023-11-15T03:03:17.461 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-15T03:03:17.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.613+0000 7f1722907700 1 Processor -- start 2023-11-15T03:03:17.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.617+0000 7f1722907700 1 -- start start 2023-11-15T03:03:17.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.617+0000 7f1722907700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f171c0a9830 0x7f171c0a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:17.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.617+0000 7f1722907700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f171c0c30e0 0x7f171c0c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:17.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.617+0000 7f1722907700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f171c0657f0 con 0x7f171c151870 2023-11-15T03:03:17.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.617+0000 7f1722907700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f171c065ad0 con 0x7f171c0c30e0 2023-11-15T03:03:17.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.617+0000 7f1722907700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f171c0c3a60 con 0x7f171c0a9830 2023-11-15T03:03:17.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.617+0000 7f1722106700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f171c0c30e0 0x7f171c0c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:17.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.617+0000 7f1722106700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f171c0c30e0 0x7f171c0c3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41252/0 (socket says 172.21.15.121:41252) 2023-11-15T03:03:17.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.617+0000 7f1721104700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f171c0a9830 0x7f171c0a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:17.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.617+0000 7f1722106700 1 -- 172.21.15.121:0/3923622311 learned_addr learned my addr 172.21.15.121:0/3923622311 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:17.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.617+0000 7f1722106700 1 -- 172.21.15.121:0/3923622311 >> v2:172.21.15.190:3300/0 conn(0x7f171c0a9830 msgr2=0x7f171c0a9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:17.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.617+0000 7f1722106700 1 --2- 172.21.15.121:0/3923622311 >> v2:172.21.15.190:3300/0 conn(0x7f171c0a9830 0x7f171c0a9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:17.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.617+0000 7f1722106700 1 -- 172.21.15.121:0/3923622311 >> v1:172.21.15.190:6789/0 conn(0x7f171c151870 legacy=0x7f171c151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:17.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.617+0000 7f1722106700 1 -- 172.21.15.121:0/3923622311 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c0aa260 con 0x7f171c0c30e0 2023-11-15T03:03:17.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.617+0000 7f1722106700 1 --2- 172.21.15.121:0/3923622311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f171c0c30e0 0x7f171c0c3570 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f171c063d00 tx=0x7f1714009840).ready entity=mon.0 client_cookie=255b9b902d16513e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:17.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.621+0000 7f1720903700 1 -- 172.21.15.121:0/3923622311 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171400e070 con 0x7f171c0c30e0 2023-11-15T03:03:17.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.621+0000 7f1720903700 1 -- 172.21.15.121:0/3923622311 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171400ba20 con 0x7f171c0c30e0 2023-11-15T03:03:17.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.621+0000 7f1720903700 1 -- 172.21.15.121:0/3923622311 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171400bda0 con 0x7f171c0c30e0 2023-11-15T03:03:17.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.621+0000 7f1722907700 1 -- 172.21.15.121:0/3923622311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f171c0c30e0 msgr2=0x7f171c0c3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:17.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.621+0000 7f1722907700 1 --2- 172.21.15.121:0/3923622311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f171c0c30e0 0x7f171c0c3570 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f171c063d00 tx=0x7f1714009840).stop 2023-11-15T03:03:17.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.621+0000 7f1722907700 1 -- 172.21.15.121:0/3923622311 shutdown_connections 2023-11-15T03:03:17.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.621+0000 7f1722907700 1 --2- 172.21.15.121:0/3923622311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f171c0c30e0 0x7f171c0c3570 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:17.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.621+0000 7f1722907700 1 --2- 172.21.15.121:0/3923622311 >> v2:172.21.15.190:3300/0 conn(0x7f171c0a9830 0x7f171c0a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:17.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.621+0000 7f1722907700 1 -- 172.21.15.121:0/3923622311 >> 172.21.15.121:0/3923622311 conn(0x7f171c062190 msgr2=0x7f171c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:17.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.621+0000 7f1722907700 1 -- 172.21.15.121:0/3923622311 shutdown_connections 2023-11-15T03:03:17.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.621+0000 7f1722907700 1 -- 172.21.15.121:0/3923622311 wait complete. 2023-11-15T03:03:17.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1722907700 1 Processor -- start 2023-11-15T03:03:17.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1722907700 1 -- start start 2023-11-15T03:03:17.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1722907700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f171c0a9830 0x7f171c146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:17.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1722907700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f171c0c30e0 0x7f171c0c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:17.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1722907700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f171c151870 0x7f171c0c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:17.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1722907700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c0c5ef0 con 0x7f171c0a9830 2023-11-15T03:03:17.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1722907700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c1465f0 con 0x7f171c151870 2023-11-15T03:03:17.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1722907700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171c1476f0 con 0x7f171c0c30e0 2023-11-15T03:03:17.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1721905700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f171c0a9830 0x7f171c146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:17.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1721905700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f171c0a9830 0x7f171c146100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41264/0 (socket says 172.21.15.121:41264) 2023-11-15T03:03:17.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1721905700 1 -- 172.21.15.121:0/3022609000 learned_addr learned my addr 172.21.15.121:0/3022609000 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:17.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1722106700 1 --2- 172.21.15.121:0/3022609000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f171c151870 0x7f171c0c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:17.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1721104700 1 --2- 172.21.15.121:0/3022609000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f171c0c30e0 0x7f171c0c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:17.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1721905700 1 -- 172.21.15.121:0/3022609000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f171c0c30e0 msgr2=0x7f171c0c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:17.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1721905700 1 --2- 172.21.15.121:0/3022609000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f171c0c30e0 0x7f171c0c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:17.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1721905700 1 -- 172.21.15.121:0/3022609000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f171c151870 msgr2=0x7f171c0c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:17.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1721905700 1 --2- 172.21.15.121:0/3022609000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f171c151870 0x7f171c0c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:17.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1721905700 1 -- 172.21.15.121:0/3022609000 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1714009ae0 con 0x7f171c0a9830 2023-11-15T03:03:17.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1721104700 1 --2- 172.21.15.121:0/3022609000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f171c0c30e0 0x7f171c0c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:17.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1722106700 1 --2- 172.21.15.121:0/3022609000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f171c151870 0x7f171c0c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:17.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.625+0000 7f1721905700 1 --2- 172.21.15.121:0/3022609000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f171c0a9830 0x7f171c146100 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7f171800c4c0 tx=0x7f1718007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:17.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.629+0000 7f1711ffb700 1 -- 172.21.15.121:0/3022609000 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718007020 con 0x7f171c0a9830 2023-11-15T03:03:17.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.629+0000 7f1711ffb700 1 -- 172.21.15.121:0/3022609000 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1718001e90 con 0x7f171c0a9830 2023-11-15T03:03:17.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.629+0000 7f1711ffb700 1 -- 172.21.15.121:0/3022609000 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17180073d0 con 0x7f171c0a9830 2023-11-15T03:03:17.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.629+0000 7f1722907700 1 -- 172.21.15.121:0/3022609000 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171c0c58a0 con 0x7f171c0a9830 2023-11-15T03:03:17.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.629+0000 7f1722907700 1 -- 172.21.15.121:0/3022609000 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171c147180 con 0x7f171c0a9830 2023-11-15T03:03:17.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.629+0000 7f1711ffb700 1 -- 172.21.15.121:0/3022609000 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f17180068b0 con 0x7f171c0a9830 2023-11-15T03:03:17.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.629+0000 7f1722907700 1 -- 172.21.15.121:0/3022609000 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171c0c5700 con 0x7f171c0a9830 2023-11-15T03:03:17.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.633+0000 7f1711ffb700 1 --2- 172.21.15.121:0/3022609000 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f170005a510 0x7f170005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:17.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.633+0000 7f1711ffb700 1 -- 172.21.15.121:0/3022609000 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f1718073d80 con 0x7f171c0a9830 2023-11-15T03:03:17.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.633+0000 7f1721104700 1 --2- 172.21.15.121:0/3022609000 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f170005a510 0x7f170005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:17.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.633+0000 7f1721104700 1 --2- 172.21.15.121:0/3022609000 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f170005a510 0x7f170005c9d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f170c006d80 tx=0x7f170c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:17.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.637+0000 7f1711ffb700 1 -- 172.21.15.121:0/3022609000 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f171c0c5700 con 0x7f171c0a9830 2023-11-15T03:03:17.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.861+0000 7f1722907700 1 -- 172.21.15.121:0/3022609000 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f171c0c5700 con 0x7f171c0a9830 2023-11-15T03:03:17.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.861+0000 7f1711ffb700 1 -- 172.21.15.121:0/3022609000 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f171c0c5700 con 0x7f171c0a9830 2023-11-15T03:03:17.866 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:17.866 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:03:17.867 INFO:teuthology.orchestra.run.smithi121.stdout: "auth_dump": [ 2023-11-15T03:03:17.867 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:17.867 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.a", 2023-11-15T03:03:17.867 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1Rlzy3zOhAAeJ3COoWQP/Kw+P/0AwHbwA==", 2023-11-15T03:03:17.867 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:03:17.868 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:03:17.868 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T03:03:17.868 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T03:03:17.868 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:03:17.869 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.869 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:03:17.869 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:17.869 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.b", 2023-11-15T03:03:17.870 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlUglaERAAC6WUS9ftCfr+KbPpiEIyqw==", 2023-11-15T03:03:17.870 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:03:17.870 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:03:17.870 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T03:03:17.871 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T03:03:17.871 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:03:17.871 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.871 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:03:17.872 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:17.872 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.c", 2023-11-15T03:03:17.872 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RldFcvCRAATIouBNyHGip5hFrH+z4v5A==", 2023-11-15T03:03:17.873 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:03:17.873 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:03:17.873 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T03:03:17.875 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T03:03:17.875 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:03:17.876 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.878 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:03:17.878 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:17.879 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.d", 2023-11-15T03:03:17.879 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1Rlg5iZGxAADwdL2RrtU0kDeb9EmNJU6A==", 2023-11-15T03:03:17.879 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:03:17.879 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:03:17.881 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T03:03:17.881 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T03:03:17.881 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:03:17.881 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.882 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:03:17.882 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:17.882 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.0", 2023-11-15T03:03:17.883 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAzM1RldPH9CBAAUCF/a47aFbbNsWV1r+u3Tw==", 2023-11-15T03:03:17.883 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:03:17.883 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:03:17.883 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:03:17.884 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:03:17.884 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.884 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:03:17.885 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:17.885 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.1", 2023-11-15T03:03:17.886 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQA2M1RltoxwMhAAZN3bgOjSaoK3ZxE5ezZ3IA==", 2023-11-15T03:03:17.886 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:03:17.886 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:03:17.886 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:03:17.886 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:03:17.887 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.887 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:03:17.887 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:17.887 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.2", 2023-11-15T03:03:17.887 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQA6M1RlgltjHxAAx/m5gm+qs8LiEFyZL+uLzQ==", 2023-11-15T03:03:17.888 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:03:17.888 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:03:17.888 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:03:17.888 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:03:17.888 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.889 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:03:17.889 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:17.889 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.3", 2023-11-15T03:03:17.890 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQA+M1RlhtQcDRAA+g11PmSNKazpk+VoK17EEQ==", 2023-11-15T03:03:17.890 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:03:17.890 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:03:17.890 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:03:17.890 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:03:17.891 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.891 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:03:17.891 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:17.891 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.4", 2023-11-15T03:03:17.892 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBFM1RlJ7hnBBAAjskKfD63N/61T4GSPqOiRg==", 2023-11-15T03:03:17.892 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:03:17.892 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:03:17.892 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:03:17.892 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:03:17.893 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.893 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:03:17.893 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:17.893 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.5", 2023-11-15T03:03:17.893 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBIM1RlI1HPLRAAgVbiNq1tmgK6PaWP+c2JFw==", 2023-11-15T03:03:17.894 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:03:17.894 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:03:17.894 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:03:17.894 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:03:17.894 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.895 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:03:17.895 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:17.895 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.6", 2023-11-15T03:03:17.895 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBMM1RlPDshDRAAVgIqxklHSQcFb5sM6GUfyQ==", 2023-11-15T03:03:17.896 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:03:17.896 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:03:17.896 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:03:17.897 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:03:17.897 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.897 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:03:17.897 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:17.897 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.7", 2023-11-15T03:03:17.898 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBPM1RlN8IuJxAA2rN2848kaB2eOM6zDb+eJg==", 2023-11-15T03:03:17.898 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:03:17.898 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:03:17.898 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:03:17.899 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:03:17.899 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.899 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:03:17.899 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:17.899 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.0", 2023-11-15T03:03:17.900 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlfHPEIRAAJ4YV46zzBQ7v7y0kD4P2ZA==", 2023-11-15T03:03:17.900 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:03:17.900 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:03:17.900 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow r", 2023-11-15T03:03:17.900 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-15T03:03:17.901 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.901 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:03:17.901 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:17.901 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.1", 2023-11-15T03:03:17.901 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlibaULBAA+y1EyhVOt+wXSk2uRajJ3g==", 2023-11-15T03:03:17.903 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:03:17.903 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:03:17.903 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2023-11-15T03:03:17.903 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2023-11-15T03:03:17.903 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2023-11-15T03:03:17.904 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.904 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:03:17.904 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:17.904 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.2", 2023-11-15T03:03:17.904 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlHHbvJhAAX5TE6UvcpBeJPSCZkfqD8w==", 2023-11-15T03:03:17.905 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:03:17.905 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:03:17.905 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2023-11-15T03:03:17.905 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2023-11-15T03:03:17.905 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2023-11-15T03:03:17.906 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.907 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:03:17.907 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:17.907 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.3", 2023-11-15T03:03:17.907 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlQtQXMhAAGAvtnvJLSoUKuEiZDsSxww==", 2023-11-15T03:03:17.907 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:03:17.908 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:03:17.908 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2023-11-15T03:03:17.908 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2023-11-15T03:03:17.908 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2023-11-15T03:03:17.909 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.909 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:03:17.909 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:17.909 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.admin", 2023-11-15T03:03:17.909 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1Rl67qqEhAAI48kZ9D7T807bTVoUmwajg==", 2023-11-15T03:03:17.910 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:03:17.910 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2023-11-15T03:03:17.910 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T03:03:17.911 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T03:03:17.911 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:03:17.912 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.912 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:03:17.913 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:17.913 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mgr.x", 2023-11-15T03:03:17.913 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1RlJQFPNRAAy7E5mrn8Xh98339fWSkX9Q==", 2023-11-15T03:03:17.928 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:03:17.928 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2023-11-15T03:03:17.928 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile mgr", 2023-11-15T03:03:17.929 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:03:17.929 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.929 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:03:17.929 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:17.929 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mgr.y", 2023-11-15T03:03:17.930 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1RldQJlLhAAbc46liEP03yBNeVhZCb7IQ==", 2023-11-15T03:03:17.930 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:03:17.930 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2023-11-15T03:03:17.930 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile mgr", 2023-11-15T03:03:17.930 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:03:17.931 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.931 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:17.931 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2023-11-15T03:03:17.931 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:03:17.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.865+0000 7f1722907700 1 -- 172.21.15.121:0/3022609000 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f170005a510 msgr2=0x7f170005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:17.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.865+0000 7f1722907700 1 --2- 172.21.15.121:0/3022609000 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f170005a510 0x7f170005c9d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f170c006d80 tx=0x7f170c008040).stop 2023-11-15T03:03:17.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.865+0000 7f1722907700 1 -- 172.21.15.121:0/3022609000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f171c0a9830 msgr2=0x7f171c146100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:17.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.865+0000 7f1722907700 1 --2- 172.21.15.121:0/3022609000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f171c0a9830 0x7f171c146100 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7f171800c4c0 tx=0x7f1718007e00).stop 2023-11-15T03:03:17.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.865+0000 7f1722907700 1 -- 172.21.15.121:0/3022609000 shutdown_connections 2023-11-15T03:03:17.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.865+0000 7f1722907700 1 --2- 172.21.15.121:0/3022609000 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f170005a510 0x7f170005c9d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:17.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.865+0000 7f1722907700 1 --2- 172.21.15.121:0/3022609000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f171c151870 0x7f171c0c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:17.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.865+0000 7f1722907700 1 --2- 172.21.15.121:0/3022609000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f171c0c30e0 0x7f171c0c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:17.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.865+0000 7f1722907700 1 --2- 172.21.15.121:0/3022609000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f171c0a9830 0x7f171c146100 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:17.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.865+0000 7f1722907700 1 -- 172.21.15.121:0/3022609000 >> 172.21.15.121:0/3022609000 conn(0x7f171c062190 msgr2=0x7f171c14f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:17.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.865+0000 7f1722907700 1 -- 172.21.15.121:0/3022609000 shutdown_connections 2023-11-15T03:03:17.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:17.865+0000 7f1722907700 1 -- 172.21.15.121:0/3022609000 wait complete. 2023-11-15T03:03:17.936 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:03:18.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.105+0000 7f8dc209b700 1 Processor -- start 2023-11-15T03:03:18.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.105+0000 7f8dc209b700 1 -- start start 2023-11-15T03:03:18.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.109+0000 7f8dc209b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8dbc154ca0 0x7f8dbc155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:18.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.109+0000 7f8dc209b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dbc0c4c30 con 0x7f8dbc1562f0 2023-11-15T03:03:18.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.109+0000 7f8dc209b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dbc0a6f50 con 0x7f8dbc0a57c0 2023-11-15T03:03:18.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.109+0000 7f8dc209b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dbc0a8280 con 0x7f8dbc154ca0 2023-11-15T03:03:18.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.109+0000 7f8db2ffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f8dbc1562f0 0x7f8dbc0a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40894/0 (socket says 172.21.15.121:40894) 2023-11-15T03:03:18.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.109+0000 7f8db2ffd700 1 -- 172.21.15.121:0/543671121 learned_addr learned my addr 172.21.15.121:0/543671121 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:18.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.109+0000 7f8dbb7fe700 1 --2- 172.21.15.121:0/543671121 >> v2:172.21.15.190:3300/0 conn(0x7f8dbc154ca0 0x7f8dbc155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:18.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.109+0000 7f8dbaffd700 1 -- 172.21.15.121:0/543671121 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269378173 0 0) 0x7f8dbc0c4c30 con 0x7f8dbc1562f0 2023-11-15T03:03:18.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.109+0000 7f8dbaffd700 1 -- 172.21.15.121:0/543671121 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8da80033f0 con 0x7f8dbc1562f0 2023-11-15T03:03:18.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.109+0000 7f8dbb7fe700 1 -- 172.21.15.121:0/543671121 >> v1:172.21.15.190:6789/0 conn(0x7f8dbc0a57c0 legacy=0x7f8dbc0c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:18.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.109+0000 7f8dbb7fe700 1 -- 172.21.15.121:0/543671121 >> v1:172.21.15.121:6789/0 conn(0x7f8dbc1562f0 legacy=0x7f8dbc0a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:18.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.109+0000 7f8dbaffd700 1 -- 172.21.15.121:0/543671121 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 184060148 0 0) 0x7f8da80033f0 con 0x7f8dbc1562f0 2023-11-15T03:03:18.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.109+0000 7f8dbb7fe700 1 -- 172.21.15.121:0/543671121 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc0a7230 con 0x7f8dbc154ca0 2023-11-15T03:03:18.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.109+0000 7f8dbb7fe700 1 --2- 172.21.15.121:0/543671121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dbc154ca0 0x7f8dbc155080 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f8dac009e60 tx=0x7f8dac009780).ready entity=mon.1 client_cookie=ce4754bd20fe99b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:18.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.109+0000 7f8dbaffd700 1 -- 172.21.15.121:0/543671121 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac00e070 con 0x7f8dbc154ca0 2023-11-15T03:03:18.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.113+0000 7f8dbaffd700 1 -- 172.21.15.121:0/543671121 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac00b850 con 0x7f8dbc154ca0 2023-11-15T03:03:18.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.113+0000 7f8dbaffd700 1 -- 172.21.15.121:0/543671121 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4001470 con 0x7f8dbc154ca0 2023-11-15T03:03:18.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.113+0000 7f8dc209b700 1 -- 172.21.15.121:0/543671121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dbc154ca0 msgr2=0x7f8dbc155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:18.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.113+0000 7f8dc209b700 1 --2- 172.21.15.121:0/543671121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dbc154ca0 0x7f8dbc155080 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f8dac009e60 tx=0x7f8dac009780).stop 2023-11-15T03:03:18.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.113+0000 7f8dc209b700 1 -- 172.21.15.121:0/543671121 shutdown_connections 2023-11-15T03:03:18.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.113+0000 7f8dc209b700 1 --2- 172.21.15.121:0/543671121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dbc154ca0 0x7f8dbc155080 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.113+0000 7f8dc209b700 1 -- 172.21.15.121:0/543671121 >> 172.21.15.121:0/543671121 conn(0x7f8dbc062190 msgr2=0x7f8dbc0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:18.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.113+0000 7f8dc209b700 1 -- 172.21.15.121:0/543671121 shutdown_connections 2023-11-15T03:03:18.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.113+0000 7f8dc209b700 1 -- 172.21.15.121:0/543671121 wait complete. 2023-11-15T03:03:18.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.117+0000 7f8dc209b700 1 Processor -- start 2023-11-15T03:03:18.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.117+0000 7f8dc209b700 1 -- start start 2023-11-15T03:03:18.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.117+0000 7f8dc209b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dbc0a57c0 0x7f8dbc145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:18.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.117+0000 7f8dc209b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dbc154ca0 0x7f8dbc0c1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:18.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.117+0000 7f8dc209b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dbc1562f0 0x7f8dbc0aa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:18.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.117+0000 7f8dc209b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc0c0d60 con 0x7f8dbc0a57c0 2023-11-15T03:03:18.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.117+0000 7f8dc209b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc0c0fb0 con 0x7f8dbc1562f0 2023-11-15T03:03:18.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.117+0000 7f8dc209b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc0c1290 con 0x7f8dbc154ca0 2023-11-15T03:03:18.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.117+0000 7f8dbb7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dbc0a57c0 0x7f8dbc145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:18.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.117+0000 7f8dbb7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dbc0a57c0 0x7f8dbc145220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41270/0 (socket says 172.21.15.121:41270) 2023-11-15T03:03:18.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.117+0000 7f8dbb7fe700 1 -- 172.21.15.121:0/3760011940 learned_addr learned my addr 172.21.15.121:0/3760011940 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:18.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.117+0000 7f8dbbfff700 1 --2- 172.21.15.121:0/3760011940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dbc1562f0 0x7f8dbc0aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:18.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.117+0000 7f8db2ffd700 1 --2- 172.21.15.121:0/3760011940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dbc154ca0 0x7f8dbc0c1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:18.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.121+0000 7f8dbb7fe700 1 -- 172.21.15.121:0/3760011940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dbc154ca0 msgr2=0x7f8dbc0c1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:18.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.121+0000 7f8dbb7fe700 1 --2- 172.21.15.121:0/3760011940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dbc154ca0 0x7f8dbc0c1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.121+0000 7f8dbb7fe700 1 -- 172.21.15.121:0/3760011940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dbc1562f0 msgr2=0x7f8dbc0aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:18.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.121+0000 7f8dbb7fe700 1 --2- 172.21.15.121:0/3760011940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dbc1562f0 0x7f8dbc0aa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.121+0000 7f8dbb7fe700 1 -- 172.21.15.121:0/3760011940 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dac009a20 con 0x7f8dbc0a57c0 2023-11-15T03:03:18.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.121+0000 7f8dbb7fe700 1 --2- 172.21.15.121:0/3760011940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dbc0a57c0 0x7f8dbc145220 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f8dac00b6c0 tx=0x7f8dac009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:18.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.121+0000 7f8db3fff700 1 -- 172.21.15.121:0/3760011940 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac012e70 con 0x7f8dbc0a57c0 2023-11-15T03:03:18.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.121+0000 7f8db3fff700 1 -- 172.21.15.121:0/3760011940 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac009460 con 0x7f8dbc0a57c0 2023-11-15T03:03:18.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.121+0000 7f8dc209b700 1 -- 172.21.15.121:0/3760011940 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dbc0c1570 con 0x7f8dbc0a57c0 2023-11-15T03:03:18.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.121+0000 7f8db3fff700 1 -- 172.21.15.121:0/3760011940 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac01d670 con 0x7f8dbc0a57c0 2023-11-15T03:03:18.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.121+0000 7f8dc209b700 1 -- 172.21.15.121:0/3760011940 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dbc145710 con 0x7f8dbc0a57c0 2023-11-15T03:03:18.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.121+0000 7f8dc209b700 1 -- 172.21.15.121:0/3760011940 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dbc06c150 con 0x7f8dbc0a57c0 2023-11-15T03:03:18.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.125+0000 7f8db3fff700 1 -- 172.21.15.121:0/3760011940 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8dac01d9e0 con 0x7f8dbc0a57c0 2023-11-15T03:03:18.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.125+0000 7f8db3fff700 1 --2- 172.21.15.121:0/3760011940 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8d9c05a510 0x7f8d9c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:18.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.125+0000 7f8db3fff700 1 -- 172.21.15.121:0/3760011940 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f8dac076270 con 0x7f8dbc0a57c0 2023-11-15T03:03:18.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.125+0000 7f8db2ffd700 1 --2- 172.21.15.121:0/3760011940 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8d9c05a510 0x7f8d9c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:18.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.125+0000 7f8db2ffd700 1 --2- 172.21.15.121:0/3760011940 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8d9c05a510 0x7f8d9c05c9d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f8dbc154770 tx=0x7f8da4000bc0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:18.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.133+0000 7f8db3fff700 1 -- 172.21.15.121:0/3760011940 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8dac049ce0 con 0x7f8dbc0a57c0 2023-11-15T03:03:18.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.305+0000 7f8dc209b700 1 -- 172.21.15.121:0/3760011940 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8dbc068ae0 con 0x7f8dbc0a57c0 2023-11-15T03:03:18.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.305+0000 7f8db3fff700 1 -- 172.21.15.121:0/3760011940 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+13761 (secure 0 0 0) 0x7f8dac04d300 con 0x7f8dbc0a57c0 2023-11-15T03:03:18.311 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:18.311 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":88,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:03:17.422495+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:03:18.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.309+0000 7f8dc209b700 1 -- 172.21.15.121:0/3760011940 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8d9c05a510 msgr2=0x7f8d9c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:18.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.309+0000 7f8dc209b700 1 --2- 172.21.15.121:0/3760011940 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8d9c05a510 0x7f8d9c05c9d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f8dbc154770 tx=0x7f8da4000bc0).stop 2023-11-15T03:03:18.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.309+0000 7f8dc209b700 1 -- 172.21.15.121:0/3760011940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dbc0a57c0 msgr2=0x7f8dbc145220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:18.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.309+0000 7f8dc209b700 1 --2- 172.21.15.121:0/3760011940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dbc0a57c0 0x7f8dbc145220 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f8dac00b6c0 tx=0x7f8dac009780).stop 2023-11-15T03:03:18.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.309+0000 7f8dc209b700 1 -- 172.21.15.121:0/3760011940 shutdown_connections 2023-11-15T03:03:18.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.309+0000 7f8dc209b700 1 --2- 172.21.15.121:0/3760011940 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8d9c05a510 0x7f8d9c05c9d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.309+0000 7f8dc209b700 1 --2- 172.21.15.121:0/3760011940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dbc1562f0 0x7f8dbc0aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.309+0000 7f8dc209b700 1 --2- 172.21.15.121:0/3760011940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dbc154ca0 0x7f8dbc0c1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.309+0000 7f8dc209b700 1 --2- 172.21.15.121:0/3760011940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dbc0a57c0 0x7f8dbc145220 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.309+0000 7f8dc209b700 1 -- 172.21.15.121:0/3760011940 >> 172.21.15.121:0/3760011940 conn(0x7f8dbc062190 msgr2=0x7f8dbc14f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:18.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.309+0000 7f8dc209b700 1 -- 172.21.15.121:0/3760011940 shutdown_connections 2023-11-15T03:03:18.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.309+0000 7f8dc209b700 1 -- 172.21.15.121:0/3760011940 wait complete. 2023-11-15T03:03:18.336 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-15T03:03:18.337 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T03:03:18.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.489+0000 7fc67759e700 1 Processor -- start 2023-11-15T03:03:18.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.493+0000 7fc67759e700 1 -- start start 2023-11-15T03:03:18.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.493+0000 7fc67759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6780acd00 0x7fc6780ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:18.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.493+0000 7fc67759e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc6780c5c70 0x7fc6781536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:18.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.493+0000 7fc67759e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc678066330 0x7fc678153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:18.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.493+0000 7fc67759e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc678065ea0 con 0x7fc678066330 2023-11-15T03:03:18.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.493+0000 7fc67759e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc678066bb0 con 0x7fc6780c5c70 2023-11-15T03:03:18.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.493+0000 7fc67759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc678065890 con 0x7fc6780acd00 2023-11-15T03:03:18.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.493+0000 7fc67659c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6780acd00 0x7fc6780ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:18.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67659c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6780acd00 0x7fc6780ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:34346/0 (socket says 172.21.15.121:34346) 2023-11-15T03:03:18.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc676d9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc678066330 0x7fc678153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:18.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67659c700 1 -- 172.21.15.121:0/2034990849 learned_addr learned my addr 172.21.15.121:0/2034990849 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:18.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc675d9b700 1 --2- 172.21.15.121:0/2034990849 >> v2:172.21.15.190:3300/0 conn(0x7fc6780c5c70 0x7fc6781536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:18.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67659c700 1 -- 172.21.15.121:0/2034990849 >> v2:172.21.15.190:3300/0 conn(0x7fc6780c5c70 msgr2=0x7fc6781536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:18.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67659c700 1 --2- 172.21.15.121:0/2034990849 >> v2:172.21.15.190:3300/0 conn(0x7fc6780c5c70 0x7fc6781536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67659c700 1 -- 172.21.15.121:0/2034990849 >> v2:172.21.15.121:3300/0 conn(0x7fc678066330 msgr2=0x7fc678153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:18.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67659c700 1 --2- 172.21.15.121:0/2034990849 >> v2:172.21.15.121:3300/0 conn(0x7fc678066330 0x7fc678153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67659c700 1 -- 172.21.15.121:0/2034990849 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6780c64a0 con 0x7fc6780acd00 2023-11-15T03:03:18.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67659c700 1 --2- 172.21.15.121:0/2034990849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6780acd00 0x7fc6780ad0e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fc678065200 tx=0x7fc668009060).ready entity=mon.2 client_cookie=7304ad57a40fa004 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:18.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67559a700 1 -- 172.21.15.121:0/2034990849 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668005880 con 0x7fc6780acd00 2023-11-15T03:03:18.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67559a700 1 -- 172.21.15.121:0/2034990849 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6680059e0 con 0x7fc6780acd00 2023-11-15T03:03:18.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67559a700 1 -- 172.21.15.121:0/2034990849 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668005d60 con 0x7fc6780acd00 2023-11-15T03:03:18.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67759e700 1 -- 172.21.15.121:0/2034990849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6780acd00 msgr2=0x7fc6780ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:18.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67759e700 1 --2- 172.21.15.121:0/2034990849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6780acd00 0x7fc6780ad0e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fc678065200 tx=0x7fc668009060).stop 2023-11-15T03:03:18.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67759e700 1 -- 172.21.15.121:0/2034990849 shutdown_connections 2023-11-15T03:03:18.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67759e700 1 --2- 172.21.15.121:0/2034990849 >> v2:172.21.15.121:3300/0 conn(0x7fc678066330 0x7fc678153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67759e700 1 --2- 172.21.15.121:0/2034990849 >> v2:172.21.15.190:3300/0 conn(0x7fc6780c5c70 0x7fc6781536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67759e700 1 --2- 172.21.15.121:0/2034990849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6780acd00 0x7fc6780ad0e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67759e700 1 -- 172.21.15.121:0/2034990849 >> 172.21.15.121:0/2034990849 conn(0x7fc678063fc0 msgr2=0x7fc678067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:18.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67759e700 1 -- 172.21.15.121:0/2034990849 shutdown_connections 2023-11-15T03:03:18.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.497+0000 7fc67759e700 1 -- 172.21.15.121:0/2034990849 wait complete. 2023-11-15T03:03:18.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.501+0000 7fc67759e700 1 Processor -- start 2023-11-15T03:03:18.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.501+0000 7fc67759e700 1 -- start start 2023-11-15T03:03:18.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.501+0000 7fc67759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc678066330 0x7fc678141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:18.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.501+0000 7fc67759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6780acd00 0x7fc67814fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:18.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.501+0000 7fc67759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6780c5c70 0x7fc6780c2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:18.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.501+0000 7fc67759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6780c1da0 con 0x7fc6780acd00 2023-11-15T03:03:18.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.501+0000 7fc67759e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6780c30d0 con 0x7fc678066330 2023-11-15T03:03:18.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.501+0000 7fc67759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6780c33b0 con 0x7fc6780c5c70 2023-11-15T03:03:18.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.505+0000 7fc676d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6780c5c70 0x7fc6780c2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:18.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.505+0000 7fc675d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6780acd00 0x7fc67814fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:18.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.505+0000 7fc676d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6780c5c70 0x7fc6780c2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:34352/0 (socket says 172.21.15.121:34352) 2023-11-15T03:03:18.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.505+0000 7fc676d9d700 1 -- 172.21.15.121:0/2839045296 learned_addr learned my addr 172.21.15.121:0/2839045296 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:18.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.505+0000 7fc67659c700 1 --2- 172.21.15.121:0/2839045296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc678066330 0x7fc678141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:18.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.505+0000 7fc675d9b700 1 -- 172.21.15.121:0/2839045296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6780c5c70 msgr2=0x7fc6780c2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:18.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.505+0000 7fc675d9b700 1 --2- 172.21.15.121:0/2839045296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6780c5c70 0x7fc6780c2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.505+0000 7fc675d9b700 1 -- 172.21.15.121:0/2839045296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc678066330 msgr2=0x7fc678141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:18.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.505+0000 7fc675d9b700 1 --2- 172.21.15.121:0/2839045296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc678066330 0x7fc678141370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.505+0000 7fc675d9b700 1 -- 172.21.15.121:0/2839045296 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc668009300 con 0x7fc6780acd00 2023-11-15T03:03:18.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.505+0000 7fc675d9b700 1 --2- 172.21.15.121:0/2839045296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6780acd00 0x7fc67814fd40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7fc66c00e6a0 tx=0x7fc66c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:18.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.505+0000 7fc6667fc700 1 -- 172.21.15.121:0/2839045296 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c012600 con 0x7fc6780acd00 2023-11-15T03:03:18.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.505+0000 7fc6667fc700 1 -- 172.21.15.121:0/2839045296 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66c012760 con 0x7fc6780acd00 2023-11-15T03:03:18.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.505+0000 7fc67759e700 1 -- 172.21.15.121:0/2839045296 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6780c3690 con 0x7fc6780acd00 2023-11-15T03:03:18.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.505+0000 7fc6667fc700 1 -- 172.21.15.121:0/2839045296 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c012b00 con 0x7fc6780acd00 2023-11-15T03:03:18.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.505+0000 7fc67759e700 1 -- 172.21.15.121:0/2839045296 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc678152f50 con 0x7fc6780acd00 2023-11-15T03:03:18.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.509+0000 7fc67759e700 1 -- 172.21.15.121:0/2839045296 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6780c1f50 con 0x7fc6780acd00 2023-11-15T03:03:18.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.509+0000 7fc6667fc700 1 -- 172.21.15.121:0/2839045296 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc66c01a3b0 con 0x7fc6780acd00 2023-11-15T03:03:18.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.509+0000 7fc6667fc700 1 --2- 172.21.15.121:0/2839045296 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc65805a510 0x7fc65805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:18.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.509+0000 7fc6667fc700 1 -- 172.21.15.121:0/2839045296 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fc66c074340 con 0x7fc6780acd00 2023-11-15T03:03:18.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.509+0000 7fc67659c700 1 --2- 172.21.15.121:0/2839045296 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc65805a510 0x7fc65805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:18.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.513+0000 7fc67659c700 1 --2- 172.21.15.121:0/2839045296 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc65805a510 0x7fc65805c9d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fc678065200 tx=0x7fc668016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:18.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.517+0000 7fc6667fc700 1 -- 172.21.15.121:0/2839045296 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc6780c1f50 con 0x7fc6780acd00 2023-11-15T03:03:18.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.685+0000 7fc67759e700 1 -- 172.21.15.121:0/2839045296 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc6780c1bd0 con 0x7fc6780acd00 2023-11-15T03:03:18.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.685+0000 7fc6667fc700 1 -- 172.21.15.121:0/2839045296 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v88) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc66c016020 con 0x7fc6780acd00 2023-11-15T03:03:18.690 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T03:03:18.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.685+0000 7fc67759e700 1 -- 172.21.15.121:0/2839045296 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc65805a510 msgr2=0x7fc65805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:18.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.685+0000 7fc67759e700 1 --2- 172.21.15.121:0/2839045296 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc65805a510 0x7fc65805c9d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fc678065200 tx=0x7fc668016000).stop 2023-11-15T03:03:18.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.685+0000 7fc67759e700 1 -- 172.21.15.121:0/2839045296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6780acd00 msgr2=0x7fc67814fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:18.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.685+0000 7fc67759e700 1 --2- 172.21.15.121:0/2839045296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6780acd00 0x7fc67814fd40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7fc66c00e6a0 tx=0x7fc66c009060).stop 2023-11-15T03:03:18.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.689+0000 7fc67759e700 1 -- 172.21.15.121:0/2839045296 shutdown_connections 2023-11-15T03:03:18.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.689+0000 7fc67759e700 1 --2- 172.21.15.121:0/2839045296 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc65805a510 0x7fc65805c9d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.689+0000 7fc67759e700 1 --2- 172.21.15.121:0/2839045296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6780c5c70 0x7fc6780c2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.689+0000 7fc67759e700 1 --2- 172.21.15.121:0/2839045296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6780acd00 0x7fc67814fd40 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.689+0000 7fc67759e700 1 --2- 172.21.15.121:0/2839045296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc678066330 0x7fc678141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.689+0000 7fc67759e700 1 -- 172.21.15.121:0/2839045296 >> 172.21.15.121:0/2839045296 conn(0x7fc678063fc0 msgr2=0x7fc67814f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:18.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.689+0000 7fc67759e700 1 -- 172.21.15.121:0/2839045296 shutdown_connections 2023-11-15T03:03:18.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.689+0000 7fc67759e700 1 -- 172.21.15.121:0/2839045296 wait complete. 2023-11-15T03:03:18.714 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T03:03:18.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.873+0000 7fa27169b700 1 Processor -- start 2023-11-15T03:03:18.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.873+0000 7fa27169b700 1 -- start start 2023-11-15T03:03:18.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.877+0000 7fa27169b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa26c14abc0 0x7fa26c14afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:18.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.877+0000 7fa27169b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa26c0a7b10 0x7fa26c0a7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:18.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.877+0000 7fa27169b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa26c066790 con 0x7fa26c1543e0 2023-11-15T03:03:18.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.877+0000 7fa27169b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa26c0a60a0 con 0x7fa26c0a7b10 2023-11-15T03:03:18.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.877+0000 7fa27169b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa26c0a8480 con 0x7fa26c14abc0 2023-11-15T03:03:18.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.877+0000 7fa26a7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa26c0a7b10 0x7fa26c0a7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:18.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.877+0000 7fa26a7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa26c0a7b10 0x7fa26c0a7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41298/0 (socket says 172.21.15.121:41298) 2023-11-15T03:03:18.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.877+0000 7fa26a7fc700 1 -- 172.21.15.121:0/1903019438 learned_addr learned my addr 172.21.15.121:0/1903019438 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:18.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.877+0000 7fa26affd700 1 --2- 172.21.15.121:0/1903019438 >> v2:172.21.15.190:3300/0 conn(0x7fa26c14abc0 0x7fa26c14afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:18.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.877+0000 7fa26a7fc700 1 -- 172.21.15.121:0/1903019438 >> v2:172.21.15.190:3300/0 conn(0x7fa26c14abc0 msgr2=0x7fa26c14afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:18.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.877+0000 7fa26a7fc700 1 --2- 172.21.15.121:0/1903019438 >> v2:172.21.15.190:3300/0 conn(0x7fa26c14abc0 0x7fa26c14afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.877+0000 7fa26a7fc700 1 -- 172.21.15.121:0/1903019438 >> v1:172.21.15.121:6789/0 conn(0x7fa26c1543e0 legacy=0x7fa26c1547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:18.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.877+0000 7fa26a7fc700 1 -- 172.21.15.121:0/1903019438 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c0a6660 con 0x7fa26c0a7b10 2023-11-15T03:03:18.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.877+0000 7fa26affd700 1 --2- 172.21.15.121:0/1903019438 >> v2:172.21.15.190:3300/0 conn(0x7fa26c14abc0 0x7fa26c14afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:18.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.877+0000 7fa26a7fc700 1 --2- 172.21.15.121:0/1903019438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa26c0a7b10 0x7fa26c0a7f90 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7fa26c065190 tx=0x7fa2600090f0).ready entity=mon.0 client_cookie=b62fc5669bd54b32 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:18.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.877+0000 7fa269ffb700 1 -- 172.21.15.121:0/1903019438 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26000b600 con 0x7fa26c0a7b10 2023-11-15T03:03:18.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.877+0000 7fa269ffb700 1 -- 172.21.15.121:0/1903019438 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26000b760 con 0x7fa26c0a7b10 2023-11-15T03:03:18.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.877+0000 7fa269ffb700 1 -- 172.21.15.121:0/1903019438 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26000bae0 con 0x7fa26c0a7b10 2023-11-15T03:03:18.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.881+0000 7fa27169b700 1 -- 172.21.15.121:0/1903019438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa26c0a7b10 msgr2=0x7fa26c0a7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:18.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.881+0000 7fa27169b700 1 --2- 172.21.15.121:0/1903019438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa26c0a7b10 0x7fa26c0a7f90 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7fa26c065190 tx=0x7fa2600090f0).stop 2023-11-15T03:03:18.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.881+0000 7fa27169b700 1 -- 172.21.15.121:0/1903019438 shutdown_connections 2023-11-15T03:03:18.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.881+0000 7fa27169b700 1 --2- 172.21.15.121:0/1903019438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa26c0a7b10 0x7fa26c0a7f90 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.881+0000 7fa27169b700 1 --2- 172.21.15.121:0/1903019438 >> v2:172.21.15.190:3300/0 conn(0x7fa26c14abc0 0x7fa26c14afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.881+0000 7fa27169b700 1 -- 172.21.15.121:0/1903019438 >> 172.21.15.121:0/1903019438 conn(0x7fa26c068530 msgr2=0x7fa26c0c6560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:18.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.881+0000 7fa27169b700 1 -- 172.21.15.121:0/1903019438 shutdown_connections 2023-11-15T03:03:18.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.881+0000 7fa27169b700 1 -- 172.21.15.121:0/1903019438 wait complete. 2023-11-15T03:03:18.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.881+0000 7fa27169b700 1 Processor -- start 2023-11-15T03:03:18.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.881+0000 7fa27169b700 1 -- start start 2023-11-15T03:03:18.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa27169b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa26c0a7b10 0x7fa26c0a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:18.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa27169b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa26c14abc0 0x7fa26c147960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:18.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa27169b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa26c1543e0 0x7fa26c167920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:18.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa27169b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c147e50 con 0x7fa26c14abc0 2023-11-15T03:03:18.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa27169b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c147fd0 con 0x7fa26c0a7b10 2023-11-15T03:03:18.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa27169b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c1482b0 con 0x7fa26c1543e0 2023-11-15T03:03:18.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa26a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa26c14abc0 0x7fa26c147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:18.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa26a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa26c14abc0 0x7fa26c147960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41304/0 (socket says 172.21.15.121:41304) 2023-11-15T03:03:18.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa26a7fc700 1 -- 172.21.15.121:0/2275836870 learned_addr learned my addr 172.21.15.121:0/2275836870 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:18.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa26affd700 1 --2- 172.21.15.121:0/2275836870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa26c0a7b10 0x7fa26c0a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:18.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa26b7fe700 1 --2- 172.21.15.121:0/2275836870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa26c1543e0 0x7fa26c167920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:18.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa26a7fc700 1 -- 172.21.15.121:0/2275836870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa26c1543e0 msgr2=0x7fa26c167920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:18.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa26a7fc700 1 --2- 172.21.15.121:0/2275836870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa26c1543e0 0x7fa26c167920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa26a7fc700 1 -- 172.21.15.121:0/2275836870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa26c0a7b10 msgr2=0x7fa26c0a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:18.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa26a7fc700 1 --2- 172.21.15.121:0/2275836870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa26c0a7b10 0x7fa26c0a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:18.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa26a7fc700 1 -- 172.21.15.121:0/2275836870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa260009370 con 0x7fa26c14abc0 2023-11-15T03:03:18.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa26a7fc700 1 --2- 172.21.15.121:0/2275836870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa26c14abc0 0x7fa26c147960 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7fa26c0695a0 tx=0x7fa260017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:18.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa252ffd700 1 -- 172.21.15.121:0/2275836870 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26000b8d0 con 0x7fa26c14abc0 2023-11-15T03:03:18.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa252ffd700 1 -- 172.21.15.121:0/2275836870 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26000ba30 con 0x7fa26c14abc0 2023-11-15T03:03:18.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa252ffd700 1 -- 172.21.15.121:0/2275836870 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26001c650 con 0x7fa26c14abc0 2023-11-15T03:03:18.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa27169b700 1 -- 172.21.15.121:0/2275836870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa26c168100 con 0x7fa26c14abc0 2023-11-15T03:03:18.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.885+0000 7fa27169b700 1 -- 172.21.15.121:0/2275836870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa26c1686c0 con 0x7fa26c14abc0 2023-11-15T03:03:18.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.889+0000 7fa27169b700 1 -- 172.21.15.121:0/2275836870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa26c148420 con 0x7fa26c14abc0 2023-11-15T03:03:18.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.889+0000 7fa252ffd700 1 -- 172.21.15.121:0/2275836870 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa26001ca40 con 0x7fa26c14abc0 2023-11-15T03:03:18.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.889+0000 7fa252ffd700 1 --2- 172.21.15.121:0/2275836870 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa25805a540 0x7fa25805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:18.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.889+0000 7fa252ffd700 1 -- 172.21.15.121:0/2275836870 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fa260075d30 con 0x7fa26c14abc0 2023-11-15T03:03:18.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.889+0000 7fa26affd700 1 --2- 172.21.15.121:0/2275836870 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa25805a540 0x7fa25805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:18.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.893+0000 7fa26affd700 1 --2- 172.21.15.121:0/2275836870 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa25805a540 0x7fa25805ca00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fa25c00abd0 tx=0x7fa25c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:18.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:18.897+0000 7fa252ffd700 1 -- 172.21.15.121:0/2275836870 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa2600497a0 con 0x7fa26c14abc0 2023-11-15T03:03:19.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.049+0000 7fa27169b700 1 -- 172.21.15.121:0/2275836870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa26c063aa0 con 0x7fa26c14abc0 2023-11-15T03:03:19.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.053+0000 7fa252ffd700 1 -- 172.21.15.121:0/2275836870 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v88) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa260020020 con 0x7fa26c14abc0 2023-11-15T03:03:19.057 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T03:03:19.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.057+0000 7fa27169b700 1 -- 172.21.15.121:0/2275836870 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa25805a540 msgr2=0x7fa25805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:19.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.057+0000 7fa27169b700 1 --2- 172.21.15.121:0/2275836870 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa25805a540 0x7fa25805ca00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fa25c00abd0 tx=0x7fa25c008040).stop 2023-11-15T03:03:19.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.057+0000 7fa27169b700 1 -- 172.21.15.121:0/2275836870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa26c14abc0 msgr2=0x7fa26c147960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:19.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.057+0000 7fa27169b700 1 --2- 172.21.15.121:0/2275836870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa26c14abc0 0x7fa26c147960 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7fa26c0695a0 tx=0x7fa260017000).stop 2023-11-15T03:03:19.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.057+0000 7fa27169b700 1 -- 172.21.15.121:0/2275836870 shutdown_connections 2023-11-15T03:03:19.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.057+0000 7fa27169b700 1 --2- 172.21.15.121:0/2275836870 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa25805a540 0x7fa25805ca00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:19.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.057+0000 7fa27169b700 1 --2- 172.21.15.121:0/2275836870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa26c1543e0 0x7fa26c167920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:19.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.057+0000 7fa27169b700 1 --2- 172.21.15.121:0/2275836870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa26c14abc0 0x7fa26c147960 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:19.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.057+0000 7fa27169b700 1 --2- 172.21.15.121:0/2275836870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa26c0a7b10 0x7fa26c0a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:19.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.057+0000 7fa27169b700 1 -- 172.21.15.121:0/2275836870 >> 172.21.15.121:0/2275836870 conn(0x7fa26c068530 msgr2=0x7fa26c065780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:19.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.057+0000 7fa27169b700 1 -- 172.21.15.121:0/2275836870 shutdown_connections 2023-11-15T03:03:19.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.057+0000 7fa27169b700 1 -- 172.21.15.121:0/2275836870 wait complete. 2023-11-15T03:03:19.086 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-15T03:03:19.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.241+0000 7f232e639700 1 Processor -- start 2023-11-15T03:03:19.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.241+0000 7f232e639700 1 -- start start 2023-11-15T03:03:19.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.241+0000 7f232e639700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2328152850 0x7f23280c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:19.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.241+0000 7f232e639700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f232815be30 0x7f232815c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:19.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.241+0000 7f232e639700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23280c1d50 con 0x7f23280c3280 2023-11-15T03:03:19.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.241+0000 7f232e639700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23280c1860 con 0x7f232815be30 2023-11-15T03:03:19.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.241+0000 7f232e639700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232815cb80 con 0x7f2328152850 2023-11-15T03:03:19.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.241+0000 7f232de38700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f232815be30 0x7f232815c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:19.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.241+0000 7f232d637700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f23280c3280 0x7f2328152140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40918/0 (socket says 172.21.15.121:40918) 2023-11-15T03:03:19.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.241+0000 7f232de38700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f232815be30 0x7f232815c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41308/0 (socket says 172.21.15.121:41308) 2023-11-15T03:03:19.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.241+0000 7f232d637700 1 -- 172.21.15.121:0/2451858715 learned_addr learned my addr 172.21.15.121:0/2451858715 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:19.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.241+0000 7f232ce36700 1 --2- 172.21.15.121:0/2451858715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2328152850 0x7f23280c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:19.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.245+0000 7f232de38700 1 -- 172.21.15.121:0/2451858715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2328152850 msgr2=0x7f23280c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:19.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.245+0000 7f232de38700 1 --2- 172.21.15.121:0/2451858715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2328152850 0x7f23280c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:19.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.245+0000 7f232de38700 1 -- 172.21.15.121:0/2451858715 >> v1:172.21.15.121:6789/0 conn(0x7f23280c3280 legacy=0x7f2328152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:19.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.245+0000 7f231ffff700 1 -- 172.21.15.121:0/2451858715 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916857587 0 0) 0x7f23280c1d50 con 0x7f23280c3280 2023-11-15T03:03:19.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.245+0000 7f232de38700 1 -- 172.21.15.121:0/2451858715 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23280a9240 con 0x7f232815be30 2023-11-15T03:03:19.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.245+0000 7f232de38700 1 --2- 172.21.15.121:0/2451858715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232815be30 0x7f232815c280 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f2320009e60 tx=0x7f2320009780).ready entity=mon.0 client_cookie=e2da19d52e473166 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:19.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.245+0000 7f231ffff700 1 -- 172.21.15.121:0/2451858715 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232000e070 con 0x7f232815be30 2023-11-15T03:03:19.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.245+0000 7f231ffff700 1 -- 172.21.15.121:0/2451858715 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232000bb60 con 0x7f232815be30 2023-11-15T03:03:19.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.245+0000 7f231ffff700 1 -- 172.21.15.121:0/2451858715 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2320012410 con 0x7f232815be30 2023-11-15T03:03:19.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.245+0000 7f232e639700 1 -- 172.21.15.121:0/2451858715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232815be30 msgr2=0x7f232815c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:19.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.245+0000 7f232e639700 1 --2- 172.21.15.121:0/2451858715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232815be30 0x7f232815c280 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f2320009e60 tx=0x7f2320009780).stop 2023-11-15T03:03:19.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.245+0000 7f232e639700 1 -- 172.21.15.121:0/2451858715 shutdown_connections 2023-11-15T03:03:19.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.245+0000 7f232e639700 1 --2- 172.21.15.121:0/2451858715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232815be30 0x7f232815c280 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:19.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.245+0000 7f232e639700 1 --2- 172.21.15.121:0/2451858715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2328152850 0x7f23280c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:19.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.245+0000 7f232e639700 1 -- 172.21.15.121:0/2451858715 >> 172.21.15.121:0/2451858715 conn(0x7f2328062190 msgr2=0x7f2328062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:19.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.245+0000 7f232e639700 1 -- 172.21.15.121:0/2451858715 shutdown_connections 2023-11-15T03:03:19.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.245+0000 7f232e639700 1 -- 172.21.15.121:0/2451858715 wait complete. 2023-11-15T03:03:19.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232e639700 1 Processor -- start 2023-11-15T03:03:19.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232e639700 1 -- start start 2023-11-15T03:03:19.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232e639700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23280c3280 0x7f232814e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:19.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232e639700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2328152850 0x7f23280bf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:19.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232e639700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232815be30 0x7f23280bfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:19.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232e639700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23280c0430 con 0x7f232815be30 2023-11-15T03:03:19.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232e639700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23280bbea0 con 0x7f2328152850 2023-11-15T03:03:19.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232e639700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23280c6550 con 0x7f23280c3280 2023-11-15T03:03:19.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232de38700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232815be30 0x7f23280bfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:19.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232de38700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232815be30 0x7f23280bfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41312/0 (socket says 172.21.15.121:41312) 2023-11-15T03:03:19.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232de38700 1 -- 172.21.15.121:0/3893526375 learned_addr learned my addr 172.21.15.121:0/3893526375 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:19.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232d637700 1 --2- 172.21.15.121:0/3893526375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23280c3280 0x7f232814e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:19.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232de38700 1 -- 172.21.15.121:0/3893526375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23280c3280 msgr2=0x7f232814e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:19.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232de38700 1 --2- 172.21.15.121:0/3893526375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23280c3280 0x7f232814e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:19.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232de38700 1 -- 172.21.15.121:0/3893526375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2328152850 msgr2=0x7f23280bf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:19.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232de38700 1 --2- 172.21.15.121:0/3893526375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2328152850 0x7f23280bf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:03:19.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232de38700 1 -- 172.21.15.121:0/3893526375 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2320009a20 con 0x7f232815be30 2023-11-15T03:03:19.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232de38700 1 --2- 172.21.15.121:0/3893526375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232815be30 0x7f23280bfaa0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7f232000b6c0 tx=0x7f2320009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:19.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f231d7fa700 1 -- 172.21.15.121:0/3893526375 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232001d630 con 0x7f232815be30 2023-11-15T03:03:19.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f231d7fa700 1 -- 172.21.15.121:0/3893526375 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232001d790 con 0x7f232815be30 2023-11-15T03:03:19.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232e639700 1 -- 172.21.15.121:0/3893526375 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23280c0150 con 0x7f232815be30 2023-11-15T03:03:19.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f231d7fa700 1 -- 172.21.15.121:0/3893526375 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232001db10 con 0x7f232815be30 2023-11-15T03:03:19.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.249+0000 7f232e639700 1 -- 172.21.15.121:0/3893526375 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23280bbb30 con 0x7f232815be30 2023-11-15T03:03:19.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.253+0000 7f232e639700 1 -- 172.21.15.121:0/3893526375 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23280bb6b0 con 0x7f232815be30 2023-11-15T03:03:19.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.253+0000 7f231d7fa700 1 -- 172.21.15.121:0/3893526375 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2320026410 con 0x7f232815be30 2023-11-15T03:03:19.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.253+0000 7f231d7fa700 1 --2- 172.21.15.121:0/3893526375 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f230c05a510 0x7f230c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:19.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.253+0000 7f231d7fa700 1 -- 172.21.15.121:0/3893526375 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f23200763e0 con 0x7f232815be30 2023-11-15T03:03:19.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.253+0000 7f232d637700 1 --2- 172.21.15.121:0/3893526375 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f230c05a510 0x7f230c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:19.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.257+0000 7f232d637700 1 --2- 172.21.15.121:0/3893526375 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f230c05a510 0x7f230c05c9d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f2324001660 tx=0x7f2324009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:19.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.257+0000 7f231d7fa700 1 -- 172.21.15.121:0/3893526375 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f23280bb6b0 con 0x7f232815be30 2023-11-15T03:03:19.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.421+0000 7f232e639700 1 -- 172.21.15.121:0/3893526375 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f23280bb6b0 con 0x7f232815be30 2023-11-15T03:03:19.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.421+0000 7f231d7fa700 1 -- 172.21.15.121:0/3893526375 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v88) v1 ==== 84+0+143 (secure 0 0 0) 0x7f23280bb6b0 con 0x7f232815be30 2023-11-15T03:03:19.428 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:19.428 INFO:teuthology.orchestra.run.smithi121.stdout:[ 2023-11-15T03:03:19.428 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:19.428 INFO:teuthology.orchestra.run.smithi121.stdout: "poolnum": 1, 2023-11-15T03:03:19.428 INFO:teuthology.orchestra.run.smithi121.stdout: "poolname": "device_health_metrics" 2023-11-15T03:03:19.429 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:03:19.429 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:03:19.429 INFO:teuthology.orchestra.run.smithi121.stdout: "poolnum": 2, 2023-11-15T03:03:19.429 INFO:teuthology.orchestra.run.smithi121.stdout: "poolname": "rbd" 2023-11-15T03:03:19.429 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:03:19.430 INFO:teuthology.orchestra.run.smithi121.stdout:] 2023-11-15T03:03:19.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.425+0000 7f232e639700 1 -- 172.21.15.121:0/3893526375 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f230c05a510 msgr2=0x7f230c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:19.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.425+0000 7f232e639700 1 --2- 172.21.15.121:0/3893526375 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f230c05a510 0x7f230c05c9d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f2324001660 tx=0x7f2324009040).stop 2023-11-15T03:03:19.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.425+0000 7f232e639700 1 -- 172.21.15.121:0/3893526375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232815be30 msgr2=0x7f23280bfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:19.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.425+0000 7f232e639700 1 --2- 172.21.15.121:0/3893526375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232815be30 0x7f23280bfaa0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7f232000b6c0 tx=0x7f2320009780).stop 2023-11-15T03:03:19.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.425+0000 7f232e639700 1 -- 172.21.15.121:0/3893526375 shutdown_connections 2023-11-15T03:03:19.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.425+0000 7f232e639700 1 --2- 172.21.15.121:0/3893526375 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f230c05a510 0x7f230c05c9d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:19.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.425+0000 7f232e639700 1 --2- 172.21.15.121:0/3893526375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232815be30 0x7f23280bfaa0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:19.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.425+0000 7f232e639700 1 --2- 172.21.15.121:0/3893526375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2328152850 0x7f23280bf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:03:19.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.425+0000 7f232e639700 1 --2- 172.21.15.121:0/3893526375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23280c3280 0x7f232814e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:19.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.425+0000 7f232e639700 1 -- 172.21.15.121:0/3893526375 >> 172.21.15.121:0/3893526375 conn(0x7f2328062190 msgr2=0x7f232814f130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:19.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.425+0000 7f232e639700 1 -- 172.21.15.121:0/3893526375 shutdown_connections 2023-11-15T03:03:19.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.425+0000 7f232e639700 1 -- 172.21.15.121:0/3893526375 wait complete. 2023-11-15T03:03:19.449 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-15T03:03:19.450 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-15T03:03:19.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.609+0000 7f33499b9700 1 Processor -- start 2023-11-15T03:03:19.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.609+0000 7f33499b9700 1 -- start start 2023-11-15T03:03:19.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.613+0000 7f33499b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f33440abaa0 0x7f33440abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:19.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.613+0000 7f33499b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f33440a8880 0x7f3344151400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:19.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.613+0000 7f33499b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33440c2c00 con 0x7f33440bb510 2023-11-15T03:03:19.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.613+0000 7f33499b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33440657b0 con 0x7f33440a8880 2023-11-15T03:03:19.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.613+0000 7f33499b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3344065a90 con 0x7f33440abaa0 2023-11-15T03:03:19.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.613+0000 7f33491b8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f33440a8880 0x7f3344151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:19.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.613+0000 7f33489b7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f33440abaa0 0x7f33440abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:19.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.613+0000 7f33491b8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f33440a8880 0x7f3344151400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41322/0 (socket says 172.21.15.121:41322) 2023-11-15T03:03:19.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.613+0000 7f33491b8700 1 -- 172.21.15.121:0/4016728535 learned_addr learned my addr 172.21.15.121:0/4016728535 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:19.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.613+0000 7f33491b8700 1 -- 172.21.15.121:0/4016728535 >> v2:172.21.15.190:3300/0 conn(0x7f33440abaa0 msgr2=0x7f33440abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:19.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.613+0000 7f33491b8700 1 --2- 172.21.15.121:0/4016728535 >> v2:172.21.15.190:3300/0 conn(0x7f33440abaa0 0x7f33440abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:19.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.613+0000 7f33491b8700 1 -- 172.21.15.121:0/4016728535 >> v1:172.21.15.190:6789/0 conn(0x7f33440bb510 legacy=0x7f3344063570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:19.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.613+0000 7f33491b8700 1 -- 172.21.15.121:0/4016728535 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33440c2890 con 0x7f33440a8880 2023-11-15T03:03:19.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.613+0000 7f33491b8700 1 --2- 172.21.15.121:0/4016728535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33440a8880 0x7f3344151400 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7f33440599c0 tx=0x7f333c0090f0).ready entity=mon.0 client_cookie=72757178af0533fb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:19.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.613+0000 7f33437fe700 1 -- 172.21.15.121:0/4016728535 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c00b520 con 0x7f33440a8880 2023-11-15T03:03:19.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.613+0000 7f33437fe700 1 -- 172.21.15.121:0/4016728535 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333c00b680 con 0x7f33440a8880 2023-11-15T03:03:19.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.613+0000 7f33437fe700 1 -- 172.21.15.121:0/4016728535 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c00ba00 con 0x7f33440a8880 2023-11-15T03:03:19.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.617+0000 7f33499b9700 1 -- 172.21.15.121:0/4016728535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33440a8880 msgr2=0x7f3344151400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:19.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.617+0000 7f33499b9700 1 --2- 172.21.15.121:0/4016728535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33440a8880 0x7f3344151400 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7f33440599c0 tx=0x7f333c0090f0).stop 2023-11-15T03:03:19.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.617+0000 7f33499b9700 1 -- 172.21.15.121:0/4016728535 shutdown_connections 2023-11-15T03:03:19.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.617+0000 7f33499b9700 1 --2- 172.21.15.121:0/4016728535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33440a8880 0x7f3344151400 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:19.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.617+0000 7f33499b9700 1 --2- 172.21.15.121:0/4016728535 >> v2:172.21.15.190:3300/0 conn(0x7f33440abaa0 0x7f33440abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:19.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.617+0000 7f33499b9700 1 -- 172.21.15.121:0/4016728535 >> 172.21.15.121:0/4016728535 conn(0x7f3344067a60 msgr2=0x7f3344067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:19.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.617+0000 7f33499b9700 1 -- 172.21.15.121:0/4016728535 shutdown_connections 2023-11-15T03:03:19.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.617+0000 7f33499b9700 1 -- 172.21.15.121:0/4016728535 wait complete. 2023-11-15T03:03:19.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.617+0000 7f33499b9700 1 Processor -- start 2023-11-15T03:03:19.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.617+0000 7f33499b9700 1 -- start start 2023-11-15T03:03:19.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f33499b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33440a8880 0x7f33441647c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:19.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f33499b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33440abaa0 0x7f3344167930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:19.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f33499b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33440bb510 0x7f3344167e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:19.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f33499b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3344164330 con 0x7f33440abaa0 2023-11-15T03:03:19.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f33499b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33440c43b0 con 0x7f33440bb510 2023-11-15T03:03:19.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f33499b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33440c4690 con 0x7f33440a8880 2023-11-15T03:03:19.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f3343fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33440abaa0 0x7f3344167930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:19.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f3343fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33440abaa0 0x7f3344167930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41324/0 (socket says 172.21.15.121:41324) 2023-11-15T03:03:19.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f3343fff700 1 -- 172.21.15.121:0/919719913 learned_addr learned my addr 172.21.15.121:0/919719913 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:19.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f33489b7700 1 --2- 172.21.15.121:0/919719913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33440a8880 0x7f33441647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:19.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f33491b8700 1 --2- 172.21.15.121:0/919719913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33440bb510 0x7f3344167e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:19.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f3343fff700 1 -- 172.21.15.121:0/919719913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33440a8880 msgr2=0x7f33441647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:19.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f3343fff700 1 --2- 172.21.15.121:0/919719913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33440a8880 0x7f33441647c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:19.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f3343fff700 1 -- 172.21.15.121:0/919719913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33440bb510 msgr2=0x7f3344167e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:19.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f3343fff700 1 --2- 172.21.15.121:0/919719913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33440bb510 0x7f3344167e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:19.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f3343fff700 1 -- 172.21.15.121:0/919719913 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c009370 con 0x7f33440abaa0 2023-11-15T03:03:19.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f33489b7700 1 --2- 172.21.15.121:0/919719913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33440a8880 0x7f33441647c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:19.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f33491b8700 1 --2- 172.21.15.121:0/919719913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33440bb510 0x7f3344167e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:19.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f3343fff700 1 --2- 172.21.15.121:0/919719913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33440abaa0 0x7f3344167930 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7f333000c4c0 tx=0x7f3330007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:19.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f3340ff9700 1 -- 172.21.15.121:0/919719913 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3330007020 con 0x7f33440abaa0 2023-11-15T03:03:19.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f3340ff9700 1 -- 172.21.15.121:0/919719913 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3330007180 con 0x7f33440abaa0 2023-11-15T03:03:19.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f33499b9700 1 -- 172.21.15.121:0/919719913 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3344163fc0 con 0x7f33440abaa0 2023-11-15T03:03:19.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f3340ff9700 1 -- 172.21.15.121:0/919719913 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3330007520 con 0x7f33440abaa0 2023-11-15T03:03:19.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.621+0000 7f33499b9700 1 -- 172.21.15.121:0/919719913 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33440c4060 con 0x7f33440abaa0 2023-11-15T03:03:19.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.625+0000 7f33499b9700 1 -- 172.21.15.121:0/919719913 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33441644a0 con 0x7f33440abaa0 2023-11-15T03:03:19.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.625+0000 7f3340ff9700 1 -- 172.21.15.121:0/919719913 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3330006900 con 0x7f33440abaa0 2023-11-15T03:03:19.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.629+0000 7f3340ff9700 1 --2- 172.21.15.121:0/919719913 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f332805a510 0x7f332805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:19.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.629+0000 7f3340ff9700 1 -- 172.21.15.121:0/919719913 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f3330073ff0 con 0x7f33440abaa0 2023-11-15T03:03:19.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.629+0000 7f33489b7700 1 --2- 172.21.15.121:0/919719913 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f332805a510 0x7f332805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:19.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.629+0000 7f33489b7700 1 --2- 172.21.15.121:0/919719913 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f332805a510 0x7f332805c9d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f3338007620 tx=0x7f333800b040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:19.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.633+0000 7f3340ff9700 1 -- 172.21.15.121:0/919719913 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f33300a2af0 con 0x7f33440abaa0 2023-11-15T03:03:19.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:19.797+0000 7f33499b9700 1 -- 172.21.15.121:0/919719913 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f334406bcf0 con 0x7f33440abaa0 2023-11-15T03:03:20.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.309+0000 7f3340ff9700 1 -- 172.21.15.121:0/919719913 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v89) v1 ==== 152+0+0 (secure 0 0 0) 0x7f3330047a60 con 0x7f33440abaa0 2023-11-15T03:03:20.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.313+0000 7f33499b9700 1 -- 172.21.15.121:0/919719913 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f332805a510 msgr2=0x7f332805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:20.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.313+0000 7f33499b9700 1 --2- 172.21.15.121:0/919719913 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f332805a510 0x7f332805c9d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f3338007620 tx=0x7f333800b040).stop 2023-11-15T03:03:20.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.313+0000 7f33499b9700 1 -- 172.21.15.121:0/919719913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33440abaa0 msgr2=0x7f3344167930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:20.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.313+0000 7f33499b9700 1 --2- 172.21.15.121:0/919719913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33440abaa0 0x7f3344167930 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7f333000c4c0 tx=0x7f3330007df0).stop 2023-11-15T03:03:20.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.313+0000 7f33499b9700 1 -- 172.21.15.121:0/919719913 shutdown_connections 2023-11-15T03:03:20.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.313+0000 7f33499b9700 1 --2- 172.21.15.121:0/919719913 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f332805a510 0x7f332805c9d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:20.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.313+0000 7f33499b9700 1 --2- 172.21.15.121:0/919719913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33440bb510 0x7f3344167e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:20.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.313+0000 7f33499b9700 1 --2- 172.21.15.121:0/919719913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33440abaa0 0x7f3344167930 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:20.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.313+0000 7f33499b9700 1 --2- 172.21.15.121:0/919719913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33440a8880 0x7f33441647c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:20.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.313+0000 7f33499b9700 1 -- 172.21.15.121:0/919719913 >> 172.21.15.121:0/919719913 conn(0x7f3344067a60 msgr2=0x7f334414f440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:20.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.313+0000 7f33499b9700 1 -- 172.21.15.121:0/919719913 shutdown_connections 2023-11-15T03:03:20.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.317+0000 7f33499b9700 1 -- 172.21.15.121:0/919719913 wait complete. 2023-11-15T03:03:20.326 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_metadata' created 2023-11-15T03:03:20.344 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-15T03:03:20.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.509+0000 7fc419f07700 1 Processor -- start 2023-11-15T03:03:20.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.509+0000 7fc419f07700 1 -- start start 2023-11-15T03:03:20.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.513+0000 7fc419f07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc414153990 0x7fc414158100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:20.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.513+0000 7fc419f07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc414068170 0x7fc4140be090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:20.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.513+0000 7fc419f07700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4141586c0 con 0x7fc414153d70 2023-11-15T03:03:20.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.513+0000 7fc419f07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4140a6850 con 0x7fc414153990 2023-11-15T03:03:20.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.513+0000 7fc419f07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4140a6b30 con 0x7fc414068170 2023-11-15T03:03:20.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.513+0000 7fc418f05700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc414153990 0x7fc414158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:20.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.513+0000 7fc418f05700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc414153990 0x7fc414158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:34604/0 (socket says 172.21.15.121:34604) 2023-11-15T03:03:20.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.513+0000 7fc418f05700 1 -- 172.21.15.121:0/3508566292 learned_addr learned my addr 172.21.15.121:0/3508566292 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:20.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.513+0000 7fc419706700 1 --2- 172.21.15.121:0/3508566292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc414068170 0x7fc4140be090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:20.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.513+0000 7fc413fff700 1 --1- 172.21.15.121:0/3508566292 >> v1:172.21.15.190:6789/0 conn(0x7fc414153d70 0x7fc414067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:45688/0 (socket says 172.21.15.121:45688) 2023-11-15T03:03:20.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.517+0000 7fc418f05700 1 -- 172.21.15.121:0/3508566292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc414068170 msgr2=0x7fc4140be090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:20.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.517+0000 7fc418f05700 1 --2- 172.21.15.121:0/3508566292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc414068170 0x7fc4140be090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:20.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.517+0000 7fc418f05700 1 -- 172.21.15.121:0/3508566292 >> v1:172.21.15.190:6789/0 conn(0x7fc414153d70 legacy=0x7fc414067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:20.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.517+0000 7fc418f05700 1 -- 172.21.15.121:0/3508566292 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4140be580 con 0x7fc414153990 2023-11-15T03:03:20.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.517+0000 7fc418f05700 1 --2- 172.21.15.121:0/3508566292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc414153990 0x7fc414158100 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fc408006fd0 tx=0x7fc4080090f0).ready entity=mon.1 client_cookie=e4e9bdf847ca8ea1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:20.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.517+0000 7fc4137fe700 1 -- 172.21.15.121:0/3508566292 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40800b580 con 0x7fc414153990 2023-11-15T03:03:20.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.517+0000 7fc4137fe700 1 -- 172.21.15.121:0/3508566292 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40800b6e0 con 0x7fc414153990 2023-11-15T03:03:20.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.517+0000 7fc4137fe700 1 -- 172.21.15.121:0/3508566292 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40800ba80 con 0x7fc414153990 2023-11-15T03:03:20.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.517+0000 7fc419f07700 1 -- 172.21.15.121:0/3508566292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc414153990 msgr2=0x7fc414158100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:20.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.517+0000 7fc419f07700 1 --2- 172.21.15.121:0/3508566292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc414153990 0x7fc414158100 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fc408006fd0 tx=0x7fc4080090f0).stop 2023-11-15T03:03:20.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.517+0000 7fc419f07700 1 -- 172.21.15.121:0/3508566292 shutdown_connections 2023-11-15T03:03:20.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.517+0000 7fc419f07700 1 --2- 172.21.15.121:0/3508566292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc414068170 0x7fc4140be090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:20.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.517+0000 7fc419f07700 1 --2- 172.21.15.121:0/3508566292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc414153990 0x7fc414158100 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:20.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.517+0000 7fc419f07700 1 -- 172.21.15.121:0/3508566292 >> 172.21.15.121:0/3508566292 conn(0x7fc414069640 msgr2=0x7fc414066860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:20.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.517+0000 7fc419f07700 1 -- 172.21.15.121:0/3508566292 shutdown_connections 2023-11-15T03:03:20.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.517+0000 7fc419f07700 1 -- 172.21.15.121:0/3508566292 wait complete. 2023-11-15T03:03:20.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.521+0000 7fc419f07700 1 Processor -- start 2023-11-15T03:03:20.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.521+0000 7fc419f07700 1 -- start start 2023-11-15T03:03:20.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.521+0000 7fc419f07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc414068170 0x7fc41413d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:20.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.521+0000 7fc419f07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc414153990 0x7fc4140bf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:20.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.521+0000 7fc419f07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc414153d70 0x7fc414167b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:20.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.521+0000 7fc419f07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4140c0c90 con 0x7fc414068170 2023-11-15T03:03:20.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.521+0000 7fc419f07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4140c0080 con 0x7fc414153990 2023-11-15T03:03:20.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.521+0000 7fc419f07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4140c0360 con 0x7fc414153d70 2023-11-15T03:03:20.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.521+0000 7fc418f05700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc414068170 0x7fc41413d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:20.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.521+0000 7fc418f05700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc414068170 0x7fc41413d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41336/0 (socket says 172.21.15.121:41336) 2023-11-15T03:03:20.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.521+0000 7fc418f05700 1 -- 172.21.15.121:0/2143365606 learned_addr learned my addr 172.21.15.121:0/2143365606 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:20.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.521+0000 7fc419706700 1 --2- 172.21.15.121:0/2143365606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc414153d70 0x7fc414167b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:20.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.521+0000 7fc413fff700 1 --2- 172.21.15.121:0/2143365606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc414153990 0x7fc4140bf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:20.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.521+0000 7fc418f05700 1 -- 172.21.15.121:0/2143365606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc414153d70 msgr2=0x7fc414167b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:20.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.521+0000 7fc418f05700 1 --2- 172.21.15.121:0/2143365606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc414153d70 0x7fc414167b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:20.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.521+0000 7fc418f05700 1 -- 172.21.15.121:0/2143365606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc414153990 msgr2=0x7fc4140bf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:20.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.521+0000 7fc418f05700 1 --2- 172.21.15.121:0/2143365606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc414153990 0x7fc4140bf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:20.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.521+0000 7fc418f05700 1 -- 172.21.15.121:0/2143365606 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc408009370 con 0x7fc414068170 2023-11-15T03:03:20.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.525+0000 7fc418f05700 1 --2- 172.21.15.121:0/2143365606 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc414068170 0x7fc41413d7e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7fc408009ea0 tx=0x7fc408017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:20.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.525+0000 7fc410ff9700 1 -- 172.21.15.121:0/2143365606 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40800be40 con 0x7fc414068170 2023-11-15T03:03:20.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.525+0000 7fc410ff9700 1 -- 172.21.15.121:0/2143365606 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40800b850 con 0x7fc414068170 2023-11-15T03:03:20.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.525+0000 7fc419f07700 1 -- 172.21.15.121:0/2143365606 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4140c0640 con 0x7fc414068170 2023-11-15T03:03:20.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.525+0000 7fc410ff9700 1 -- 172.21.15.121:0/2143365606 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40801c420 con 0x7fc414068170 2023-11-15T03:03:20.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.525+0000 7fc419f07700 1 -- 172.21.15.121:0/2143365606 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41413dcd0 con 0x7fc414068170 2023-11-15T03:03:20.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.525+0000 7fc419f07700 1 -- 172.21.15.121:0/2143365606 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41406bcf0 con 0x7fc414068170 2023-11-15T03:03:20.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.525+0000 7fc410ff9700 1 -- 172.21.15.121:0/2143365606 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc40801c7d0 con 0x7fc414068170 2023-11-15T03:03:20.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.529+0000 7fc410ff9700 1 --2- 172.21.15.121:0/2143365606 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc3f805a540 0x7fc3f805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:20.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.529+0000 7fc410ff9700 1 -- 172.21.15.121:0/2143365606 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5639+0+0 (secure 0 0 0) 0x7fc408076490 con 0x7fc414068170 2023-11-15T03:03:20.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.529+0000 7fc413fff700 1 --2- 172.21.15.121:0/2143365606 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc3f805a540 0x7fc3f805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:20.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.529+0000 7fc413fff700 1 --2- 172.21.15.121:0/2143365606 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc3f805a540 0x7fc3f805ca00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fc41414f0f0 tx=0x7fc400007b90).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:20.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.533+0000 7fc410ff9700 1 -- 172.21.15.121:0/2143365606 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc408049d90 con 0x7fc414068170 2023-11-15T03:03:20.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:20.697+0000 7fc419f07700 1 -- 172.21.15.121:0/2143365606 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc414095420 con 0x7fc414068170 2023-11-15T03:03:21.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.321+0000 7fc410ff9700 1 -- 172.21.15.121:0/2143365606 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v90) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc408020020 con 0x7fc414068170 2023-11-15T03:03:21.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.329+0000 7fc419f07700 1 -- 172.21.15.121:0/2143365606 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc3f805a540 msgr2=0x7fc3f805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:21.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.329+0000 7fc419f07700 1 --2- 172.21.15.121:0/2143365606 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc3f805a540 0x7fc3f805ca00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fc41414f0f0 tx=0x7fc400007b90).stop 2023-11-15T03:03:21.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.329+0000 7fc419f07700 1 -- 172.21.15.121:0/2143365606 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc414068170 msgr2=0x7fc41413d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:21.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.329+0000 7fc419f07700 1 --2- 172.21.15.121:0/2143365606 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc414068170 0x7fc41413d7e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7fc408009ea0 tx=0x7fc408017000).stop 2023-11-15T03:03:21.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.329+0000 7fc419f07700 1 -- 172.21.15.121:0/2143365606 shutdown_connections 2023-11-15T03:03:21.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.329+0000 7fc419f07700 1 --2- 172.21.15.121:0/2143365606 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc3f805a540 0x7fc3f805ca00 secure :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0x7fc41414f0f0 tx=0x7fc400007b90).stop 2023-11-15T03:03:21.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.329+0000 7fc419f07700 1 --2- 172.21.15.121:0/2143365606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc414153d70 0x7fc414167b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:21.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.329+0000 7fc419f07700 1 --2- 172.21.15.121:0/2143365606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc414153990 0x7fc4140bf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:21.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.329+0000 7fc419f07700 1 --2- 172.21.15.121:0/2143365606 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc414068170 0x7fc41413d7e0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:21.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.329+0000 7fc419f07700 1 -- 172.21.15.121:0/2143365606 >> 172.21.15.121:0/2143365606 conn(0x7fc414069640 msgr2=0x7fc4140ac600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:21.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.333+0000 7fc419f07700 1 -- 172.21.15.121:0/2143365606 shutdown_connections 2023-11-15T03:03:21.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.333+0000 7fc419f07700 1 -- 172.21.15.121:0/2143365606 wait complete. 2023-11-15T03:03:21.357 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_data' created 2023-11-15T03:03:21.382 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-15T03:03:21.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.557+0000 7f68a79a9700 1 Processor -- start 2023-11-15T03:03:21.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.557+0000 7f68a79a9700 1 -- start start 2023-11-15T03:03:21.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f68a79a9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f68a0157ed0 0x7f68a01582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:21.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f68a79a9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68a0062da0 con 0x7f68a0067a60 2023-11-15T03:03:21.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f68a79a9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68a0062190 con 0x7f68a0067020 2023-11-15T03:03:21.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f68a79a9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a00a93a0 con 0x7f68a0157ed0 2023-11-15T03:03:21.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f68a4f44700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f68a0067a60 0x7f68a0067e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40922/0 (socket says 172.21.15.121:40922) 2023-11-15T03:03:21.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f68a4f44700 1 -- 172.21.15.121:0/2560841696 learned_addr learned my addr 172.21.15.121:0/2560841696 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:21.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f68a5745700 1 --2- 172.21.15.121:0/2560841696 >> v2:172.21.15.190:3300/0 conn(0x7f68a0157ed0 0x7f68a01582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:21.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f6897fff700 1 -- 172.21.15.121:0/2560841696 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765720569 0 0) 0x7f68a0062da0 con 0x7f68a0067a60 2023-11-15T03:03:21.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f6897fff700 1 -- 172.21.15.121:0/2560841696 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f688c0033f0 con 0x7f68a0067a60 2023-11-15T03:03:21.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f6897fff700 1 -- 172.21.15.121:0/2560841696 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3667290334 0 0) 0x7f688c0033f0 con 0x7f68a0067a60 2023-11-15T03:03:21.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f6897fff700 1 -- 172.21.15.121:0/2560841696 >> v2:172.21.15.190:3300/0 conn(0x7f68a0157ed0 msgr2=0x7f68a01582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:21.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f6897fff700 1 --2- 172.21.15.121:0/2560841696 >> v2:172.21.15.190:3300/0 conn(0x7f68a0157ed0 0x7f68a01582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:21.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f6897fff700 1 -- 172.21.15.121:0/2560841696 >> v1:172.21.15.190:6789/0 conn(0x7f68a0067020 legacy=0x7f68a00bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:21.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f6897fff700 1 -- 172.21.15.121:0/2560841696 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a0062470 con 0x7f68a0067a60 2023-11-15T03:03:21.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f6897fff700 1 -- 172.21.15.121:0/2560841696 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f6890001470 con 0x7f68a0067a60 2023-11-15T03:03:21.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f6897fff700 1 -- 172.21.15.121:0/2560841696 >> v1:172.21.15.121:6789/0 conn(0x7f68a0067a60 legacy=0x7f68a0067e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:21.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f6897fff700 1 --2- 172.21.15.121:0/2560841696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f688c0038e0 0x7f688c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:21.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f6897fff700 1 --2- 172.21.15.121:0/2560841696 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f688c004690 0x7f688c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:21.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f6897fff700 1 --2- 172.21.15.121:0/2560841696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f688c005000 0x7f688c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:21.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f6897fff700 1 -- 172.21.15.121:0/2560841696 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c00bf60 con 0x7f688c005000 2023-11-15T03:03:21.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f6897fff700 1 -- 172.21.15.121:0/2560841696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c00c0a0 con 0x7f688c0038e0 2023-11-15T03:03:21.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.561+0000 7f6897fff700 1 -- 172.21.15.121:0/2560841696 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688c00c230 con 0x7f688c004690 2023-11-15T03:03:21.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f68a5745700 1 --2- 172.21.15.121:0/2560841696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f688c0038e0 0x7f688c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:21.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f68a5f46700 1 --2- 172.21.15.121:0/2560841696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f688c005000 0x7f688c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:21.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f68a4f44700 1 --2- 172.21.15.121:0/2560841696 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f688c004690 0x7f688c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:21.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f68a5f46700 1 -- 172.21.15.121:0/2560841696 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f688c004690 msgr2=0x7f688c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:21.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f68a5f46700 1 --2- 172.21.15.121:0/2560841696 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f688c004690 0x7f688c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:21.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f68a5f46700 1 -- 172.21.15.121:0/2560841696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f688c0038e0 msgr2=0x7f688c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:21.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f68a5f46700 1 --2- 172.21.15.121:0/2560841696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f688c0038e0 0x7f688c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:21.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f68a5f46700 1 -- 172.21.15.121:0/2560841696 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688c0033f0 con 0x7f688c005000 2023-11-15T03:03:21.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f68a5745700 1 --2- 172.21.15.121:0/2560841696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f688c0038e0 0x7f688c003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:03:21.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f68a5f46700 1 -- 172.21.15.121:0/2560841696 reap_dead start 2023-11-15T03:03:21.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f68a5f46700 1 --2- 172.21.15.121:0/2560841696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f688c005000 0x7f688c00b8e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7f6898001e30 tx=0x7f6898007f30).ready entity=mon.0 client_cookie=693fce76931a05f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:21.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f6897fff700 1 -- 172.21.15.121:0/2560841696 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6898006a20 con 0x7f688c005000 2023-11-15T03:03:21.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f6897fff700 1 -- 172.21.15.121:0/2560841696 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6898006b80 con 0x7f688c005000 2023-11-15T03:03:21.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f6897fff700 1 -- 172.21.15.121:0/2560841696 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689800c5d0 con 0x7f688c005000 2023-11-15T03:03:21.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f68a79a9700 1 -- 172.21.15.121:0/2560841696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f688c005000 msgr2=0x7f688c00b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:21.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f68a79a9700 1 --2- 172.21.15.121:0/2560841696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f688c005000 0x7f688c00b8e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7f6898001e30 tx=0x7f6898007f30).stop 2023-11-15T03:03:21.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f68a79a9700 1 -- 172.21.15.121:0/2560841696 shutdown_connections 2023-11-15T03:03:21.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f68a79a9700 1 --2- 172.21.15.121:0/2560841696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f688c005000 0x7f688c00b8e0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:21.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f68a79a9700 1 -- 172.21.15.121:0/2560841696 >> 172.21.15.121:0/2560841696 conn(0x7f68a00591c0 msgr2=0x7f68a00c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:21.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f68a79a9700 1 -- 172.21.15.121:0/2560841696 shutdown_connections 2023-11-15T03:03:21.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.565+0000 7f68a79a9700 1 -- 172.21.15.121:0/2560841696 wait complete. 2023-11-15T03:03:21.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.569+0000 7f68a79a9700 1 Processor -- start 2023-11-15T03:03:21.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.569+0000 7f68a79a9700 1 -- start start 2023-11-15T03:03:21.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.569+0000 7f68a79a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f688c005000 0x7f68a01449d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:21.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.569+0000 7f68a79a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68a00c4fa0 0x7f68a00c5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:21.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.569+0000 7f68a79a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68a00a9930 0x7f68a00a9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:21.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.569+0000 7f68a79a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a0158900 con 0x7f68a00a9930 2023-11-15T03:03:21.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.569+0000 7f68a79a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a00c5910 con 0x7f688c005000 2023-11-15T03:03:21.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.569+0000 7f68a79a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a00aa5b0 con 0x7f68a00c4fa0 2023-11-15T03:03:21.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.569+0000 7f68a5f46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68a00a9930 0x7f68a00a9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:21.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.569+0000 7f68a5f46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68a00a9930 0x7f68a00a9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41356/0 (socket says 172.21.15.121:41356) 2023-11-15T03:03:21.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.569+0000 7f68a5f46700 1 -- 172.21.15.121:0/2615808656 learned_addr learned my addr 172.21.15.121:0/2615808656 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:21.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.569+0000 7f68a5745700 1 --2- 172.21.15.121:0/2615808656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f688c005000 0x7f68a01449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:21.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.569+0000 7f68a4f44700 1 --2- 172.21.15.121:0/2615808656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68a00c4fa0 0x7f68a00c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:21.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.569+0000 7f68a5f46700 1 -- 172.21.15.121:0/2615808656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68a00c4fa0 msgr2=0x7f68a00c5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:21.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.569+0000 7f68a5f46700 1 --2- 172.21.15.121:0/2615808656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68a00c4fa0 0x7f68a00c5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:21.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.569+0000 7f68a5f46700 1 -- 172.21.15.121:0/2615808656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f688c005000 msgr2=0x7f68a01449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:21.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.569+0000 7f68a5f46700 1 --2- 172.21.15.121:0/2615808656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f688c005000 0x7f68a01449d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:21.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.569+0000 7f68a5f46700 1 -- 172.21.15.121:0/2615808656 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a00aa890 con 0x7f68a00a9930 2023-11-15T03:03:21.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.569+0000 7f68a5f46700 1 --2- 172.21.15.121:0/2615808656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68a00a9930 0x7f68a00a9db0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f6898001600 tx=0x7f6898016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:21.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.573+0000 7f68957fa700 1 -- 172.21.15.121:0/2615808656 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689800c5d0 con 0x7f68a00a9930 2023-11-15T03:03:21.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.573+0000 7f68957fa700 1 -- 172.21.15.121:0/2615808656 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6898013c00 con 0x7f68a00a9930 2023-11-15T03:03:21.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.573+0000 7f68a79a9700 1 -- 172.21.15.121:0/2615808656 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a01582b0 con 0x7f68a00a9930 2023-11-15T03:03:21.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.573+0000 7f68957fa700 1 -- 172.21.15.121:0/2615808656 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6898006a20 con 0x7f68a00a9930 2023-11-15T03:03:21.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.573+0000 7f68a79a9700 1 -- 172.21.15.121:0/2615808656 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a00aa2a0 con 0x7f68a00a9930 2023-11-15T03:03:21.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.573+0000 7f68a79a9700 1 -- 172.21.15.121:0/2615808656 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a0158a70 con 0x7f68a00a9930 2023-11-15T03:03:21.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.573+0000 7f68957fa700 1 -- 172.21.15.121:0/2615808656 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f689801d680 con 0x7f68a00a9930 2023-11-15T03:03:21.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.577+0000 7f68957fa700 1 --2- 172.21.15.121:0/2615808656 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f688405a540 0x7f688405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:21.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.577+0000 7f68957fa700 1 -- 172.21.15.121:0/2615808656 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6021+0+0 (secure 0 0 0) 0x7f68980760f0 con 0x7f68a00a9930 2023-11-15T03:03:21.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.577+0000 7f68a5745700 1 --2- 172.21.15.121:0/2615808656 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f688405a540 0x7f688405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:21.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.577+0000 7f68a5745700 1 --2- 172.21.15.121:0/2615808656 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f688405a540 0x7f688405ca00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f689c0095a0 tx=0x7f689c009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:21.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.581+0000 7f68957fa700 1 -- 172.21.15.121:0/2615808656 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6898049870 con 0x7f68a00a9930 2023-11-15T03:03:21.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.781+0000 7f68a79a9700 1 -- 172.21.15.121:0/2615808656 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f68a0047060 con 0x7f68a00a9930 2023-11-15T03:03:21.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.797+0000 7f68957fa700 1 -- 172.21.15.121:0/2615808656 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v77) v1 ==== 174+0+0 (secure 0 0 0) 0x7f689804ce90 con 0x7f68a00a9930 2023-11-15T03:03:21.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.813+0000 7f68a79a9700 1 -- 172.21.15.121:0/2615808656 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f688405a540 msgr2=0x7f688405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:21.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.813+0000 7f68a79a9700 1 --2- 172.21.15.121:0/2615808656 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f688405a540 0x7f688405ca00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f689c0095a0 tx=0x7f689c009060).stop 2023-11-15T03:03:21.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.813+0000 7f68a79a9700 1 -- 172.21.15.121:0/2615808656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68a00a9930 msgr2=0x7f68a00a9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:21.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.813+0000 7f68a79a9700 1 --2- 172.21.15.121:0/2615808656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68a00a9930 0x7f68a00a9db0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f6898001600 tx=0x7f6898016000).stop 2023-11-15T03:03:21.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.817+0000 7f68a79a9700 1 -- 172.21.15.121:0/2615808656 shutdown_connections 2023-11-15T03:03:21.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.817+0000 7f68a79a9700 1 --2- 172.21.15.121:0/2615808656 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f688405a540 0x7f688405ca00 secure :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0x7f689c0095a0 tx=0x7f689c009060).stop 2023-11-15T03:03:21.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.817+0000 7f68a79a9700 1 --2- 172.21.15.121:0/2615808656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68a00a9930 0x7f68a00a9db0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:21.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.817+0000 7f68a79a9700 1 --2- 172.21.15.121:0/2615808656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68a00c4fa0 0x7f68a00c5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:21.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.817+0000 7f68a79a9700 1 --2- 172.21.15.121:0/2615808656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f688c005000 0x7f68a01449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:21.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.817+0000 7f68a79a9700 1 -- 172.21.15.121:0/2615808656 >> 172.21.15.121:0/2615808656 conn(0x7f68a00591c0 msgr2=0x7f68a0155b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:21.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.821+0000 7f68a79a9700 1 -- 172.21.15.121:0/2615808656 shutdown_connections 2023-11-15T03:03:21.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:21.821+0000 7f68a79a9700 1 -- 172.21.15.121:0/2615808656 wait complete. 2023-11-15T03:03:21.839 INFO:teuthology.orchestra.run.smithi121.stderr:new fs with metadata pool 13 and data pool 14 2023-11-15T03:03:21.852 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:03:22.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.009+0000 7f4749e9b700 1 Processor -- start 2023-11-15T03:03:22.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.009+0000 7f4749e9b700 1 -- start start 2023-11-15T03:03:22.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.009+0000 7f4749e9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f47441539d0 0x7f47440c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:22.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.009+0000 7f4749e9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f47440c2670 0x7f47440c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:22.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.009+0000 7f4749e9b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47440c1530 con 0x7f4744153db0 2023-11-15T03:03:22.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.009+0000 7f4749e9b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47441541f0 con 0x7f47440c2670 2023-11-15T03:03:22.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.009+0000 7f4749e9b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47440c2ac0 con 0x7f47441539d0 2023-11-15T03:03:22.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.009+0000 7f4743fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f47440c2670 0x7f47440c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:22.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.009+0000 7f473affd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4744153db0 0x7f47440c19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40934/0 (socket says 172.21.15.121:40934) 2023-11-15T03:03:22.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.009+0000 7f473affd700 1 -- 172.21.15.121:0/3295528660 learned_addr learned my addr 172.21.15.121:0/3295528660 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:22.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.009+0000 7f47437fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f47441539d0 0x7f47440c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:22.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.013+0000 7f47437fe700 1 -- 172.21.15.121:0/3295528660 >> v2:172.21.15.121:3300/0 conn(0x7f47440c2670 msgr2=0x7f47440c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:22.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.013+0000 7f47437fe700 1 --2- 172.21.15.121:0/3295528660 >> v2:172.21.15.121:3300/0 conn(0x7f47440c2670 0x7f47440c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.013+0000 7f47437fe700 1 -- 172.21.15.121:0/3295528660 >> v1:172.21.15.121:6789/0 conn(0x7f4744153db0 legacy=0x7f47440c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:22.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.013+0000 7f47437fe700 1 -- 172.21.15.121:0/3295528660 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47440c2d80 con 0x7f47441539d0 2023-11-15T03:03:22.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.013+0000 7f47437fe700 1 --2- 172.21.15.121:0/3295528660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47441539d0 0x7f47440c06e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f47440c4270 tx=0x7f4734009780).ready entity=mon.1 client_cookie=cac24f0a8aa09901 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:22.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.013+0000 7f4742ffd700 1 -- 172.21.15.121:0/3295528660 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473400e070 con 0x7f47441539d0 2023-11-15T03:03:22.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.013+0000 7f4742ffd700 1 -- 172.21.15.121:0/3295528660 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473400bb60 con 0x7f47441539d0 2023-11-15T03:03:22.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.013+0000 7f4742ffd700 1 -- 172.21.15.121:0/3295528660 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4734012410 con 0x7f47441539d0 2023-11-15T03:03:22.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.013+0000 7f4749e9b700 1 -- 172.21.15.121:0/3295528660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47441539d0 msgr2=0x7f47440c06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:22.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.013+0000 7f4749e9b700 1 --2- 172.21.15.121:0/3295528660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47441539d0 0x7f47440c06e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f47440c4270 tx=0x7f4734009780).stop 2023-11-15T03:03:22.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.013+0000 7f4749e9b700 1 -- 172.21.15.121:0/3295528660 shutdown_connections 2023-11-15T03:03:22.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.013+0000 7f4749e9b700 1 --2- 172.21.15.121:0/3295528660 >> v2:172.21.15.121:3300/0 conn(0x7f47440c2670 0x7f47440c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.013+0000 7f4749e9b700 1 --2- 172.21.15.121:0/3295528660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47441539d0 0x7f47440c06e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.013+0000 7f4749e9b700 1 -- 172.21.15.121:0/3295528660 >> 172.21.15.121:0/3295528660 conn(0x7f4744062190 msgr2=0x7f47440625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:22.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.013+0000 7f4749e9b700 1 -- 172.21.15.121:0/3295528660 shutdown_connections 2023-11-15T03:03:22.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.013+0000 7f4749e9b700 1 -- 172.21.15.121:0/3295528660 wait complete. 2023-11-15T03:03:22.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.017+0000 7f4749e9b700 1 Processor -- start 2023-11-15T03:03:22.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.017+0000 7f4749e9b700 1 -- start start 2023-11-15T03:03:22.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.017+0000 7f4749e9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47440c2670 0x7f47441580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:22.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.017+0000 7f4749e9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47441539d0 0x7f4744155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:22.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.017+0000 7f4749e9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4744153db0 0x7f47440ab980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:22.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.017+0000 7f4749e9b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4744150680 con 0x7f47441539d0 2023-11-15T03:03:22.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.017+0000 7f4749e9b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4744150240 con 0x7f4744153db0 2023-11-15T03:03:22.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.017+0000 7f4749e9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4744150920 con 0x7f47440c2670 2023-11-15T03:03:22.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.017+0000 7f473affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47441539d0 0x7f4744155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:22.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.017+0000 7f473affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47441539d0 0x7f4744155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41374/0 (socket says 172.21.15.121:41374) 2023-11-15T03:03:22.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.017+0000 7f473affd700 1 -- 172.21.15.121:0/3549852447 learned_addr learned my addr 172.21.15.121:0/3549852447 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:22.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.017+0000 7f4743fff700 1 --2- 172.21.15.121:0/3549852447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4744153db0 0x7f47440ab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:22.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.017+0000 7f47437fe700 1 --2- 172.21.15.121:0/3549852447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47440c2670 0x7f47441580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:22.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.017+0000 7f473affd700 1 -- 172.21.15.121:0/3549852447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47440c2670 msgr2=0x7f47441580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:22.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.017+0000 7f473affd700 1 --2- 172.21.15.121:0/3549852447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47440c2670 0x7f47441580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.017+0000 7f473affd700 1 -- 172.21.15.121:0/3549852447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4744153db0 msgr2=0x7f47440ab980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:22.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.017+0000 7f473affd700 1 --2- 172.21.15.121:0/3549852447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4744153db0 0x7f47440ab980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.017+0000 7f473affd700 1 -- 172.21.15.121:0/3549852447 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4734009a20 con 0x7f47441539d0 2023-11-15T03:03:22.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.021+0000 7f473affd700 1 --2- 172.21.15.121:0/3549852447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47441539d0 0x7f4744155f60 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7f472c007fd0 tx=0x7f472c007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:22.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.021+0000 7f473bfff700 1 -- 172.21.15.121:0/3549852447 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c00ce10 con 0x7f47441539d0 2023-11-15T03:03:22.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.021+0000 7f473bfff700 1 -- 172.21.15.121:0/3549852447 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472c0072e0 con 0x7f47441539d0 2023-11-15T03:03:22.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.021+0000 7f473bfff700 1 -- 172.21.15.121:0/3549852447 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c007660 con 0x7f47441539d0 2023-11-15T03:03:22.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.021+0000 7f4749e9b700 1 -- 172.21.15.121:0/3549852447 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4744150c00 con 0x7f47441539d0 2023-11-15T03:03:22.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.021+0000 7f4749e9b700 1 -- 172.21.15.121:0/3549852447 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47441585c0 con 0x7f47441539d0 2023-11-15T03:03:22.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.021+0000 7f4749e9b700 1 -- 172.21.15.121:0/3549852447 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47440674d0 con 0x7f47441539d0 2023-11-15T03:03:22.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.021+0000 7f473bfff700 1 -- 172.21.15.121:0/3549852447 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f472c0059b0 con 0x7f47441539d0 2023-11-15T03:03:22.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.025+0000 7f473bfff700 1 --2- 172.21.15.121:0/3549852447 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f472405a510 0x7f472405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:22.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.025+0000 7f473bfff700 1 -- 172.21.15.121:0/3549852447 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f472c073e20 con 0x7f47441539d0 2023-11-15T03:03:22.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.025+0000 7f47437fe700 1 --2- 172.21.15.121:0/3549852447 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f472405a510 0x7f472405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:22.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.025+0000 7f47437fe700 1 --2- 172.21.15.121:0/3549852447 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f472405a510 0x7f472405c9d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f47440c4270 tx=0x7f4734009780).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:22.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.029+0000 7f473bfff700 1 -- 172.21.15.121:0/3549852447 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f47440674d0 con 0x7f47441539d0 2023-11-15T03:03:22.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.193+0000 7f4749e9b700 1 -- 172.21.15.121:0/3549852447 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47440674d0 con 0x7f47441539d0 2023-11-15T03:03:22.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.193+0000 7f473bfff700 1 -- 172.21.15.121:0/3549852447 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16992 (secure 0 0 0) 0x7f47440674d0 con 0x7f47441539d0 2023-11-15T03:03:22.200 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:22.200 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":91,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:03:21.784031+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-15T03:03:19.872820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-15T03:03:20.771478+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:03:22.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.197+0000 7f4749e9b700 1 -- 172.21.15.121:0/3549852447 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f472405a510 msgr2=0x7f472405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:22.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.197+0000 7f4749e9b700 1 --2- 172.21.15.121:0/3549852447 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f472405a510 0x7f472405c9d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f47440c4270 tx=0x7f4734009780).stop 2023-11-15T03:03:22.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.197+0000 7f4749e9b700 1 -- 172.21.15.121:0/3549852447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47441539d0 msgr2=0x7f4744155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:22.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.197+0000 7f4749e9b700 1 --2- 172.21.15.121:0/3549852447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47441539d0 0x7f4744155f60 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7f472c007fd0 tx=0x7f472c007df0).stop 2023-11-15T03:03:22.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.197+0000 7f4749e9b700 1 -- 172.21.15.121:0/3549852447 shutdown_connections 2023-11-15T03:03:22.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.197+0000 7f4749e9b700 1 --2- 172.21.15.121:0/3549852447 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f472405a510 0x7f472405c9d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.197+0000 7f4749e9b700 1 --2- 172.21.15.121:0/3549852447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4744153db0 0x7f47440ab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.197+0000 7f4749e9b700 1 --2- 172.21.15.121:0/3549852447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47441539d0 0x7f4744155f60 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.197+0000 7f4749e9b700 1 --2- 172.21.15.121:0/3549852447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47440c2670 0x7f47441580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.197+0000 7f4749e9b700 1 -- 172.21.15.121:0/3549852447 >> 172.21.15.121:0/3549852447 conn(0x7f4744062190 msgr2=0x7f4744066580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:22.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.197+0000 7f4749e9b700 1 -- 172.21.15.121:0/3549852447 shutdown_connections 2023-11-15T03:03:22.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.197+0000 7f4749e9b700 1 -- 172.21.15.121:0/3549852447 wait complete. 2023-11-15T03:03:22.223 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:03:22.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.373+0000 7f47fd65b700 1 Processor -- start 2023-11-15T03:03:22.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.373+0000 7f47fd65b700 1 -- start start 2023-11-15T03:03:22.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.377+0000 7f47fd65b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f47f80c06e0 0x7f47f8153db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:22.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.377+0000 7f47fd65b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f80c2670 0x7f47f80c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:22.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.377+0000 7f47fd65b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f80c1530 con 0x7f47f81539d0 2023-11-15T03:03:22.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.377+0000 7f47fd65b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f80c2b00 con 0x7f47f80c06e0 2023-11-15T03:03:22.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.377+0000 7f47fd65b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80c2d80 con 0x7f47f80c2670 2023-11-15T03:03:22.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.377+0000 7f47effff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f47f80c06e0 0x7f47f8153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:22.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.377+0000 7f47effff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f47f80c06e0 0x7f47f8153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41390/0 (socket says 172.21.15.121:41390) 2023-11-15T03:03:22.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.377+0000 7f47effff700 1 -- 172.21.15.121:0/3719900718 learned_addr learned my addr 172.21.15.121:0/3719900718 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:22.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.377+0000 7f47f77fe700 1 --2- 172.21.15.121:0/3719900718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f80c2670 0x7f47f80c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:22.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.377+0000 7f47f67fc700 1 -- 172.21.15.121:0/3719900718 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 614062041 0 0) 0x7f47f80c1530 con 0x7f47f81539d0 2023-11-15T03:03:22.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.377+0000 7f47f67fc700 1 -- 172.21.15.121:0/3719900718 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47e40033f0 con 0x7f47f81539d0 2023-11-15T03:03:22.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.377+0000 7f47effff700 1 -- 172.21.15.121:0/3719900718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f80c2670 msgr2=0x7f47f80c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:22.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.377+0000 7f47effff700 1 --2- 172.21.15.121:0/3719900718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f80c2670 0x7f47f80c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.377+0000 7f47effff700 1 -- 172.21.15.121:0/3719900718 >> v1:172.21.15.190:6789/0 conn(0x7f47f81539d0 legacy=0x7f47f80c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:22.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.377+0000 7f47effff700 1 -- 172.21.15.121:0/3719900718 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f80c0ee0 con 0x7f47f80c06e0 2023-11-15T03:03:22.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.377+0000 7f47effff700 1 --2- 172.21.15.121:0/3719900718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47f80c06e0 0x7f47f8153db0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f47e0006fd0 tx=0x7f47e00090f0).ready entity=mon.0 client_cookie=e13022a539a82bd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:22.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.377+0000 7f47f67fc700 1 -- 172.21.15.121:0/3719900718 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e000f510 con 0x7f47f80c06e0 2023-11-15T03:03:22.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.377+0000 7f47f67fc700 1 -- 172.21.15.121:0/3719900718 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e000f670 con 0x7f47f80c06e0 2023-11-15T03:03:22.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.377+0000 7f47f67fc700 1 -- 172.21.15.121:0/3719900718 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e000f9f0 con 0x7f47f80c06e0 2023-11-15T03:03:22.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.381+0000 7f47fd65b700 1 -- 172.21.15.121:0/3719900718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47f80c06e0 msgr2=0x7f47f8153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:22.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.381+0000 7f47fd65b700 1 --2- 172.21.15.121:0/3719900718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47f80c06e0 0x7f47f8153db0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f47e0006fd0 tx=0x7f47e00090f0).stop 2023-11-15T03:03:22.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.381+0000 7f47fd65b700 1 -- 172.21.15.121:0/3719900718 shutdown_connections 2023-11-15T03:03:22.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.381+0000 7f47fd65b700 1 --2- 172.21.15.121:0/3719900718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f80c2670 0x7f47f80c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.381+0000 7f47fd65b700 1 --2- 172.21.15.121:0/3719900718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47f80c06e0 0x7f47f8153db0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.381+0000 7f47fd65b700 1 -- 172.21.15.121:0/3719900718 >> 172.21.15.121:0/3719900718 conn(0x7f47f8062190 msgr2=0x7f47f80625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:22.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.381+0000 7f47fd65b700 1 -- 172.21.15.121:0/3719900718 shutdown_connections 2023-11-15T03:03:22.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.381+0000 7f47fd65b700 1 -- 172.21.15.121:0/3719900718 wait complete. 2023-11-15T03:03:22.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.381+0000 7f47fd65b700 1 Processor -- start 2023-11-15T03:03:22.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.381+0000 7f47fd65b700 1 -- start start 2023-11-15T03:03:22.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47fd65b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47f80c06e0 0x7f47f814b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:22.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47fd65b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f80c2670 0x7f47f8157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:22.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47fd65b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47f81539d0 0x7f47f8158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:22.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47fd65b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80a9830 con 0x7f47f80c06e0 2023-11-15T03:03:22.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47fd65b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80aa060 con 0x7f47f81539d0 2023-11-15T03:03:22.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47fd65b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80ac590 con 0x7f47f80c2670 2023-11-15T03:03:22.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47f6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47f80c06e0 0x7f47f814b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:22.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47f6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47f80c06e0 0x7f47f814b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41402/0 (socket says 172.21.15.121:41402) 2023-11-15T03:03:22.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47f6ffd700 1 -- 172.21.15.121:0/2115048452 learned_addr learned my addr 172.21.15.121:0/2115048452 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:22.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47f77fe700 1 --2- 172.21.15.121:0/2115048452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47f81539d0 0x7f47f8158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:22.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47effff700 1 --2- 172.21.15.121:0/2115048452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f80c2670 0x7f47f8157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:22.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47f6ffd700 1 -- 172.21.15.121:0/2115048452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f80c2670 msgr2=0x7f47f8157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:22.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47f6ffd700 1 --2- 172.21.15.121:0/2115048452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f80c2670 0x7f47f8157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47f6ffd700 1 -- 172.21.15.121:0/2115048452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47f81539d0 msgr2=0x7f47f8158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:22.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47f6ffd700 1 --2- 172.21.15.121:0/2115048452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47f81539d0 0x7f47f8158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47f6ffd700 1 -- 172.21.15.121:0/2115048452 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e0009370 con 0x7f47f80c06e0 2023-11-15T03:03:22.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47f77fe700 1 --2- 172.21.15.121:0/2115048452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47f81539d0 0x7f47f8158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:22.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47effff700 1 --2- 172.21.15.121:0/2115048452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f80c2670 0x7f47f8157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:22.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47f6ffd700 1 --2- 172.21.15.121:0/2115048452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47f80c06e0 0x7f47f814b2f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7f47e800c470 tx=0x7f47e8007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:22.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47eeffd700 1 -- 172.21.15.121:0/2115048452 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e800f470 con 0x7f47f80c06e0 2023-11-15T03:03:22.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47eeffd700 1 -- 172.21.15.121:0/2115048452 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e800f5d0 con 0x7f47f80c06e0 2023-11-15T03:03:22.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47fd65b700 1 -- 172.21.15.121:0/2115048452 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f80a9a10 con 0x7f47f80c06e0 2023-11-15T03:03:22.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47eeffd700 1 -- 172.21.15.121:0/2115048452 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e800f990 con 0x7f47f80c06e0 2023-11-15T03:03:22.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.385+0000 7f47fd65b700 1 -- 172.21.15.121:0/2115048452 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f814b7e0 con 0x7f47f80c06e0 2023-11-15T03:03:22.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.389+0000 7f47fd65b700 1 -- 172.21.15.121:0/2115048452 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f806c150 con 0x7f47f80c06e0 2023-11-15T03:03:22.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.389+0000 7f47eeffd700 1 -- 172.21.15.121:0/2115048452 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f47e800fd40 con 0x7f47f80c06e0 2023-11-15T03:03:22.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.393+0000 7f47eeffd700 1 --2- 172.21.15.121:0/2115048452 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f47d805a510 0x7f47d805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:22.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.393+0000 7f47eeffd700 1 -- 172.21.15.121:0/2115048452 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f47e8073db0 con 0x7f47f80c06e0 2023-11-15T03:03:22.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.393+0000 7f47effff700 1 --2- 172.21.15.121:0/2115048452 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f47d805a510 0x7f47d805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:22.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.393+0000 7f47effff700 1 --2- 172.21.15.121:0/2115048452 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f47d805a510 0x7f47d805c9d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f47e0009130 tx=0x7f47e0017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:22.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.397+0000 7f47eeffd700 1 -- 172.21.15.121:0/2115048452 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f47e8047590 con 0x7f47f80c06e0 2023-11-15T03:03:22.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.561+0000 7f47fd65b700 1 -- 172.21.15.121:0/2115048452 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47f8068ae0 con 0x7f47f80c06e0 2023-11-15T03:03:22.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.565+0000 7f47eeffd700 1 -- 172.21.15.121:0/2115048452 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16992 (secure 0 0 0) 0x7f47e804abb0 con 0x7f47f80c06e0 2023-11-15T03:03:22.571 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:22.571 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":91,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:03:21.784031+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-15T03:03:19.872820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-15T03:03:20.771478+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:03:22.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.569+0000 7f47fd65b700 1 -- 172.21.15.121:0/2115048452 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f47d805a510 msgr2=0x7f47d805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:22.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.569+0000 7f47fd65b700 1 --2- 172.21.15.121:0/2115048452 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f47d805a510 0x7f47d805c9d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f47e0009130 tx=0x7f47e0017000).stop 2023-11-15T03:03:22.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.569+0000 7f47fd65b700 1 -- 172.21.15.121:0/2115048452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47f80c06e0 msgr2=0x7f47f814b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:22.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.569+0000 7f47fd65b700 1 --2- 172.21.15.121:0/2115048452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47f80c06e0 0x7f47f814b2f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7f47e800c470 tx=0x7f47e8007c90).stop 2023-11-15T03:03:22.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.569+0000 7f47fd65b700 1 -- 172.21.15.121:0/2115048452 shutdown_connections 2023-11-15T03:03:22.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.569+0000 7f47fd65b700 1 --2- 172.21.15.121:0/2115048452 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f47d805a510 0x7f47d805c9d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.569+0000 7f47fd65b700 1 --2- 172.21.15.121:0/2115048452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47f81539d0 0x7f47f8158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.569+0000 7f47fd65b700 1 --2- 172.21.15.121:0/2115048452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f80c2670 0x7f47f8157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.569+0000 7f47fd65b700 1 --2- 172.21.15.121:0/2115048452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47f80c06e0 0x7f47f814b2f0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.569+0000 7f47fd65b700 1 -- 172.21.15.121:0/2115048452 >> 172.21.15.121:0/2115048452 conn(0x7f47f8062190 msgr2=0x7f47f814f530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:22.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.569+0000 7f47fd65b700 1 -- 172.21.15.121:0/2115048452 shutdown_connections 2023-11-15T03:03:22.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.569+0000 7f47fd65b700 1 -- 172.21.15.121:0/2115048452 wait complete. 2023-11-15T03:03:22.595 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-15T03:03:22.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.753+0000 7fc7044f1700 1 Processor -- start 2023-11-15T03:03:22.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.753+0000 7fc7044f1700 1 -- start start 2023-11-15T03:03:22.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.757+0000 7fc7044f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6fc0682a0 0x7fc6fc0c10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:22.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.757+0000 7fc7044f1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc6fc153700 0x7fc6fc153b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:22.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.757+0000 7fc7044f1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6fc0c15a0 con 0x7fc6fc067ec0 2023-11-15T03:03:22.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.757+0000 7fc7044f1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6fc0686e0 con 0x7fc6fc153700 2023-11-15T03:03:22.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.757+0000 7fc7044f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc0c0c20 con 0x7fc6fc0682a0 2023-11-15T03:03:22.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.757+0000 7fc70228d700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fc6fc067ec0 0x7fc6fc062c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:45718/0 (socket says 172.21.15.121:45718) 2023-11-15T03:03:22.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.757+0000 7fc70228d700 1 -- 172.21.15.121:0/513485084 learned_addr learned my addr 172.21.15.121:0/513485084 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:22.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.757+0000 7fc701a8c700 1 --2- 172.21.15.121:0/513485084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6fc0682a0 0x7fc6fc0c10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:22.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.761+0000 7fc702a8e700 1 --2- 172.21.15.121:0/513485084 >> v2:172.21.15.190:3300/0 conn(0x7fc6fc153700 0x7fc6fc153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:22.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.761+0000 7fc70128b700 1 -- 172.21.15.121:0/513485084 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335851649 0 0) 0x7fc6f8001110 con 0x7fc6fc067ec0 2023-11-15T03:03:22.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.761+0000 7fc70128b700 1 -- 172.21.15.121:0/513485084 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6e80033f0 con 0x7fc6fc067ec0 2023-11-15T03:03:22.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.761+0000 7fc701a8c700 1 -- 172.21.15.121:0/513485084 >> v2:172.21.15.190:3300/0 conn(0x7fc6fc153700 msgr2=0x7fc6fc153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:22.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.761+0000 7fc701a8c700 1 --2- 172.21.15.121:0/513485084 >> v2:172.21.15.190:3300/0 conn(0x7fc6fc153700 0x7fc6fc153b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.761+0000 7fc701a8c700 1 -- 172.21.15.121:0/513485084 >> v1:172.21.15.190:6789/0 conn(0x7fc6fc067ec0 legacy=0x7fc6fc062c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:22.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.761+0000 7fc701a8c700 1 -- 172.21.15.121:0/513485084 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc158730 con 0x7fc6fc0682a0 2023-11-15T03:03:22.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.761+0000 7fc701a8c700 1 --2- 172.21.15.121:0/513485084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6fc0682a0 0x7fc6fc0c10b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fc6ec009760 tx=0x7fc6ec009060).ready entity=mon.2 client_cookie=2f8026364e99a347 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:22.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.761+0000 7fc70128b700 1 -- 172.21.15.121:0/513485084 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec00d4a0 con 0x7fc6fc0682a0 2023-11-15T03:03:22.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.761+0000 7fc70128b700 1 -- 172.21.15.121:0/513485084 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ec00d600 con 0x7fc6fc0682a0 2023-11-15T03:03:22.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.761+0000 7fc70128b700 1 -- 172.21.15.121:0/513485084 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec00d980 con 0x7fc6fc0682a0 2023-11-15T03:03:22.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.761+0000 7fc7044f1700 1 -- 172.21.15.121:0/513485084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6fc0682a0 msgr2=0x7fc6fc0c10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:22.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.761+0000 7fc7044f1700 1 --2- 172.21.15.121:0/513485084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6fc0682a0 0x7fc6fc0c10b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fc6ec009760 tx=0x7fc6ec009060).stop 2023-11-15T03:03:22.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.761+0000 7fc7044f1700 1 -- 172.21.15.121:0/513485084 shutdown_connections 2023-11-15T03:03:22.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.761+0000 7fc7044f1700 1 --2- 172.21.15.121:0/513485084 >> v2:172.21.15.190:3300/0 conn(0x7fc6fc153700 0x7fc6fc153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.761+0000 7fc7044f1700 1 --2- 172.21.15.121:0/513485084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6fc0682a0 0x7fc6fc0c10b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.761+0000 7fc7044f1700 1 -- 172.21.15.121:0/513485084 >> 172.21.15.121:0/513485084 conn(0x7fc6fc0cad40 msgr2=0x7fc6fc0cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:22.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.761+0000 7fc7044f1700 1 -- 172.21.15.121:0/513485084 shutdown_connections 2023-11-15T03:03:22.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.761+0000 7fc7044f1700 1 -- 172.21.15.121:0/513485084 wait complete. 2023-11-15T03:03:22.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.765+0000 7fc7044f1700 1 Processor -- start 2023-11-15T03:03:22.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.765+0000 7fc7044f1700 1 -- start start 2023-11-15T03:03:22.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.765+0000 7fc7044f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6fc067ec0 0x7fc6fc14fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:22.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.765+0000 7fc7044f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6fc0682a0 0x7fc6fc156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:22.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.765+0000 7fc7044f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6fc153700 0x7fc6fc156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:22.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.765+0000 7fc7044f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc156f70 con 0x7fc6fc153700 2023-11-15T03:03:22.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.765+0000 7fc7044f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc154e30 con 0x7fc6fc067ec0 2023-11-15T03:03:22.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.765+0000 7fc7044f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc155110 con 0x7fc6fc0682a0 2023-11-15T03:03:22.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.769+0000 7fc702a8e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6fc153700 0x7fc6fc156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:22.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.769+0000 7fc702a8e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6fc153700 0x7fc6fc156a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41406/0 (socket says 172.21.15.121:41406) 2023-11-15T03:03:22.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.769+0000 7fc702a8e700 1 -- 172.21.15.121:0/3350368892 learned_addr learned my addr 172.21.15.121:0/3350368892 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:22.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.769+0000 7fc70228d700 1 --2- 172.21.15.121:0/3350368892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6fc067ec0 0x7fc6fc14fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:22.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.769+0000 7fc701a8c700 1 --2- 172.21.15.121:0/3350368892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6fc0682a0 0x7fc6fc156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:22.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.769+0000 7fc702a8e700 1 -- 172.21.15.121:0/3350368892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6fc0682a0 msgr2=0x7fc6fc156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:22.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.769+0000 7fc702a8e700 1 --2- 172.21.15.121:0/3350368892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6fc0682a0 0x7fc6fc156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.769+0000 7fc702a8e700 1 -- 172.21.15.121:0/3350368892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6fc067ec0 msgr2=0x7fc6fc14fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:22.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.769+0000 7fc702a8e700 1 --2- 172.21.15.121:0/3350368892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6fc067ec0 0x7fc6fc14fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:22.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.769+0000 7fc702a8e700 1 -- 172.21.15.121:0/3350368892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec0092a0 con 0x7fc6fc153700 2023-11-15T03:03:22.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.769+0000 7fc702a8e700 1 --2- 172.21.15.121:0/3350368892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6fc153700 0x7fc6fc156a80 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7fc6f4009df0 tx=0x7fc6f400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:22.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.769+0000 7fc6f27fc700 1 -- 172.21.15.121:0/3350368892 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f400cb30 con 0x7fc6fc153700 2023-11-15T03:03:22.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.769+0000 7fc6f27fc700 1 -- 172.21.15.121:0/3350368892 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f400f040 con 0x7fc6fc153700 2023-11-15T03:03:22.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.769+0000 7fc6f27fc700 1 -- 172.21.15.121:0/3350368892 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f400cea0 con 0x7fc6fc153700 2023-11-15T03:03:22.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.769+0000 7fc7044f1700 1 -- 172.21.15.121:0/3350368892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6fc1553f0 con 0x7fc6fc153700 2023-11-15T03:03:22.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.769+0000 7fc7044f1700 1 -- 172.21.15.121:0/3350368892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6fc150330 con 0x7fc6fc153700 2023-11-15T03:03:22.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.769+0000 7fc7044f1700 1 -- 172.21.15.121:0/3350368892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6fc0665a0 con 0x7fc6fc153700 2023-11-15T03:03:22.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.773+0000 7fc6f27fc700 1 -- 172.21.15.121:0/3350368892 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc6f4013610 con 0x7fc6fc153700 2023-11-15T03:03:22.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.773+0000 7fc6f27fc700 1 --2- 172.21.15.121:0/3350368892 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc6e005a510 0x7fc6e005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:22.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.773+0000 7fc6f27fc700 1 -- 172.21.15.121:0/3350368892 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc6f4074fe0 con 0x7fc6fc153700 2023-11-15T03:03:22.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.773+0000 7fc70228d700 1 --2- 172.21.15.121:0/3350368892 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc6e005a510 0x7fc6e005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:22.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.773+0000 7fc70228d700 1 --2- 172.21.15.121:0/3350368892 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc6e005a510 0x7fc6e005c9d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fc6f8001270 tx=0x7fc6f8007a70).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:22.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.777+0000 7fc6f27fc700 1 -- 172.21.15.121:0/3350368892 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc6fc0665a0 con 0x7fc6fc153700 2023-11-15T03:03:22.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:22.977+0000 7fc7044f1700 1 -- 172.21.15.121:0/3350368892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc6fc0c2c10 con 0x7fc6fc153700 2023-11-15T03:03:23.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:23.805+0000 7fc6f27fc700 1 -- 172.21.15.121:0/3350368892 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v80) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc6fc0c2c10 con 0x7fc6fc153700 2023-11-15T03:03:23.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:23.809+0000 7fc7044f1700 1 -- 172.21.15.121:0/3350368892 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc6e005a510 msgr2=0x7fc6e005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:23.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:23.809+0000 7fc7044f1700 1 --2- 172.21.15.121:0/3350368892 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc6e005a510 0x7fc6e005c9d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fc6f8001270 tx=0x7fc6f8007a70).stop 2023-11-15T03:03:23.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:23.809+0000 7fc7044f1700 1 -- 172.21.15.121:0/3350368892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6fc153700 msgr2=0x7fc6fc156a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:23.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:23.809+0000 7fc7044f1700 1 --2- 172.21.15.121:0/3350368892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6fc153700 0x7fc6fc156a80 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7fc6f4009df0 tx=0x7fc6f400b040).stop 2023-11-15T03:03:23.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:23.809+0000 7fc7044f1700 1 -- 172.21.15.121:0/3350368892 shutdown_connections 2023-11-15T03:03:23.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:23.809+0000 7fc7044f1700 1 --2- 172.21.15.121:0/3350368892 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc6e005a510 0x7fc6e005c9d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:23.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:23.809+0000 7fc7044f1700 1 --2- 172.21.15.121:0/3350368892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6fc153700 0x7fc6fc156a80 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:23.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:23.809+0000 7fc7044f1700 1 --2- 172.21.15.121:0/3350368892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6fc0682a0 0x7fc6fc156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:23.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:23.809+0000 7fc7044f1700 1 --2- 172.21.15.121:0/3350368892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6fc067ec0 0x7fc6fc14fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:23.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:23.809+0000 7fc7044f1700 1 -- 172.21.15.121:0/3350368892 >> 172.21.15.121:0/3350368892 conn(0x7fc6fc0cad40 msgr2=0x7fc6fc065850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:23.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:23.809+0000 7fc7044f1700 1 -- 172.21.15.121:0/3350368892 shutdown_connections 2023-11-15T03:03:23.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:23.809+0000 7fc7044f1700 1 -- 172.21.15.121:0/3350368892 wait complete. 2023-11-15T03:03:23.841 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-15T03:03:23.842 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-15T03:03:24.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.001+0000 7fe026bdb700 1 Processor -- start 2023-11-15T03:03:24.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.001+0000 7fe026bdb700 1 -- start start 2023-11-15T03:03:24.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.005+0000 7fe026bdb700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe020062830 0x7fe020062c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:24.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.005+0000 7fe026bdb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe020154b10 con 0x7fe0200a8e00 2023-11-15T03:03:24.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.005+0000 7fe026bdb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe020154df0 con 0x7fe020067100 2023-11-15T03:03:24.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.005+0000 7fe026bdb700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe020063130 con 0x7fe020062830 2023-11-15T03:03:24.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.005+0000 7fe01ffff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe0200a8e00 0x7fe020066010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40946/0 (socket says 172.21.15.121:40946) 2023-11-15T03:03:24.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.005+0000 7fe01ffff700 1 -- 172.21.15.121:0/2458301187 learned_addr learned my addr 172.21.15.121:0/2458301187 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:24.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.005+0000 7fe025178700 1 --2- 172.21.15.121:0/2458301187 >> v2:172.21.15.121:3300/0 conn(0x7fe020062830 0x7fe020062c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:24.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.005+0000 7fe01f7fe700 1 -- 172.21.15.121:0/2458301187 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242535962 0 0) 0x7fe020154b10 con 0x7fe0200a8e00 2023-11-15T03:03:24.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.005+0000 7fe01f7fe700 1 -- 172.21.15.121:0/2458301187 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0100033f0 con 0x7fe0200a8e00 2023-11-15T03:03:24.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.005+0000 7fe01f7fe700 1 -- 172.21.15.121:0/2458301187 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2056574148 0 0) 0x7fe020154df0 con 0x7fe020067100 2023-11-15T03:03:24.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.005+0000 7fe01f7fe700 1 -- 172.21.15.121:0/2458301187 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe020154b10 con 0x7fe020067100 2023-11-15T03:03:24.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.005+0000 7fe025178700 1 -- 172.21.15.121:0/2458301187 >> v1:172.21.15.190:6789/0 conn(0x7fe020067100 legacy=0x7fe020154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:24.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.005+0000 7fe025178700 1 -- 172.21.15.121:0/2458301187 >> v1:172.21.15.121:6789/0 conn(0x7fe0200a8e00 legacy=0x7fe020066010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:24.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.005+0000 7fe025178700 1 -- 172.21.15.121:0/2458301187 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0201550d0 con 0x7fe020062830 2023-11-15T03:03:24.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.005+0000 7fe01f7fe700 1 -- 172.21.15.121:0/2458301187 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3576528291 0 0) 0x7fe0100033f0 con 0x7fe0200a8e00 2023-11-15T03:03:24.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.005+0000 7fe025178700 1 --2- 172.21.15.121:0/2458301187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe020062830 0x7fe020062c40 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7fe018009e60 tx=0x7fe018009780).ready entity=mon.0 client_cookie=ed5a61015bc79489 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:24.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.005+0000 7fe01f7fe700 1 -- 172.21.15.121:0/2458301187 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01800e070 con 0x7fe020062830 2023-11-15T03:03:24.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.005+0000 7fe01f7fe700 1 -- 172.21.15.121:0/2458301187 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01800bab0 con 0x7fe020062830 2023-11-15T03:03:24.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.005+0000 7fe01f7fe700 1 -- 172.21.15.121:0/2458301187 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01800be30 con 0x7fe020062830 2023-11-15T03:03:24.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.009+0000 7fe026bdb700 1 -- 172.21.15.121:0/2458301187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe020062830 msgr2=0x7fe020062c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:24.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.009+0000 7fe026bdb700 1 --2- 172.21.15.121:0/2458301187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe020062830 0x7fe020062c40 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7fe018009e60 tx=0x7fe018009780).stop 2023-11-15T03:03:24.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.009+0000 7fe026bdb700 1 -- 172.21.15.121:0/2458301187 shutdown_connections 2023-11-15T03:03:24.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.009+0000 7fe026bdb700 1 --2- 172.21.15.121:0/2458301187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe020062830 0x7fe020062c40 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:24.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.009+0000 7fe026bdb700 1 -- 172.21.15.121:0/2458301187 >> 172.21.15.121:0/2458301187 conn(0x7fe0200591c0 msgr2=0x7fe0200c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:24.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.009+0000 7fe026bdb700 1 -- 172.21.15.121:0/2458301187 shutdown_connections 2023-11-15T03:03:24.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.009+0000 7fe026bdb700 1 -- 172.21.15.121:0/2458301187 wait complete. 2023-11-15T03:03:24.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.009+0000 7fe026bdb700 1 Processor -- start 2023-11-15T03:03:24.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.009+0000 7fe026bdb700 1 -- start start 2023-11-15T03:03:24.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe026bdb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe020062830 0x7fe02013ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:24.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe026bdb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe020067100 0x7fe0200c0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:24.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe026bdb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0200a8e00 0x7fe020159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:24.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe026bdb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200c0990 con 0x7fe020062830 2023-11-15T03:03:24.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe026bdb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200aa940 con 0x7fe020067100 2023-11-15T03:03:24.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe026bdb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200aa010 con 0x7fe0200a8e00 2023-11-15T03:03:24.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe024977700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe020062830 0x7fe02013ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:24.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe024977700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe020062830 0x7fe02013ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41428/0 (socket says 172.21.15.121:41428) 2023-11-15T03:03:24.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe024977700 1 -- 172.21.15.121:0/2408376714 learned_addr learned my addr 172.21.15.121:0/2408376714 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:24.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe025178700 1 --2- 172.21.15.121:0/2408376714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0200a8e00 0x7fe020159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:24.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe01ffff700 1 --2- 172.21.15.121:0/2408376714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe020067100 0x7fe0200c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:24.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe024977700 1 -- 172.21.15.121:0/2408376714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0200a8e00 msgr2=0x7fe020159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:24.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe024977700 1 --2- 172.21.15.121:0/2408376714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0200a8e00 0x7fe020159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:24.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe024977700 1 -- 172.21.15.121:0/2408376714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe020067100 msgr2=0x7fe0200c0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:24.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe024977700 1 --2- 172.21.15.121:0/2408376714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe020067100 0x7fe0200c0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:24.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe024977700 1 -- 172.21.15.121:0/2408376714 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe018009a20 con 0x7fe020062830 2023-11-15T03:03:24.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe01ffff700 1 --2- 172.21.15.121:0/2408376714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe020067100 0x7fe0200c0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:24.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe024977700 1 --2- 172.21.15.121:0/2408376714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe020062830 0x7fe02013ab50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7fe01400c4c0 tx=0x7fe014007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:24.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe01cff9700 1 -- 172.21.15.121:0/2408376714 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01400f470 con 0x7fe020062830 2023-11-15T03:03:24.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe01cff9700 1 -- 172.21.15.121:0/2408376714 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01400f5d0 con 0x7fe020062830 2023-11-15T03:03:24.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe026bdb700 1 -- 172.21.15.121:0/2408376714 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0200aa2f0 con 0x7fe020062830 2023-11-15T03:03:24.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe01cff9700 1 -- 172.21.15.121:0/2408376714 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01400f990 con 0x7fe020062830 2023-11-15T03:03:24.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.013+0000 7fe026bdb700 1 -- 172.21.15.121:0/2408376714 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02013b040 con 0x7fe020062830 2023-11-15T03:03:24.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.017+0000 7fe026bdb700 1 -- 172.21.15.121:0/2408376714 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe020047060 con 0x7fe020062830 2023-11-15T03:03:24.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.017+0000 7fe01cff9700 1 -- 172.21.15.121:0/2408376714 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe01400fd40 con 0x7fe020062830 2023-11-15T03:03:24.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.017+0000 7fe01cff9700 1 --2- 172.21.15.121:0/2408376714 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe00405a510 0x7fe00405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:24.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.017+0000 7fe01cff9700 1 -- 172.21.15.121:0/2408376714 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe014073e30 con 0x7fe020062830 2023-11-15T03:03:24.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.017+0000 7fe01ffff700 1 --2- 172.21.15.121:0/2408376714 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe00405a510 0x7fe00405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:24.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.021+0000 7fe01ffff700 1 --2- 172.21.15.121:0/2408376714 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe00405a510 0x7fe00405c9d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7fe0200a9c80 tx=0x7fe00c007ad0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:24.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.025+0000 7fe01cff9700 1 -- 172.21.15.121:0/2408376714 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe0140474f0 con 0x7fe020062830 2023-11-15T03:03:24.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.229+0000 7fe026bdb700 1 -- 172.21.15.121:0/2408376714 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe0200c40d0 con 0x7fe020062830 2023-11-15T03:03:24.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.813+0000 7fe01cff9700 1 -- 172.21.15.121:0/2408376714 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v81) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe01404ab10 con 0x7fe020062830 2023-11-15T03:03:24.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.821+0000 7fe026bdb700 1 -- 172.21.15.121:0/2408376714 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe00405a510 msgr2=0x7fe00405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:24.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.821+0000 7fe026bdb700 1 --2- 172.21.15.121:0/2408376714 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe00405a510 0x7fe00405c9d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7fe0200a9c80 tx=0x7fe00c007ad0).stop 2023-11-15T03:03:24.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.821+0000 7fe026bdb700 1 -- 172.21.15.121:0/2408376714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe020062830 msgr2=0x7fe02013ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:24.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.821+0000 7fe026bdb700 1 --2- 172.21.15.121:0/2408376714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe020062830 0x7fe02013ab50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7fe01400c4c0 tx=0x7fe014007c90).stop 2023-11-15T03:03:24.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.821+0000 7fe026bdb700 1 -- 172.21.15.121:0/2408376714 shutdown_connections 2023-11-15T03:03:24.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.821+0000 7fe026bdb700 1 --2- 172.21.15.121:0/2408376714 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe00405a510 0x7fe00405c9d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:24.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.821+0000 7fe026bdb700 1 --2- 172.21.15.121:0/2408376714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0200a8e00 0x7fe020159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:24.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.821+0000 7fe026bdb700 1 --2- 172.21.15.121:0/2408376714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe020067100 0x7fe0200c0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:24.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.821+0000 7fe026bdb700 1 --2- 172.21.15.121:0/2408376714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe020062830 0x7fe02013ab50 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:24.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.821+0000 7fe026bdb700 1 -- 172.21.15.121:0/2408376714 >> 172.21.15.121:0/2408376714 conn(0x7fe0200591c0 msgr2=0x7fe0200c6770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:24.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.821+0000 7fe026bdb700 1 -- 172.21.15.121:0/2408376714 shutdown_connections 2023-11-15T03:03:24.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:24.821+0000 7fe026bdb700 1 -- 172.21.15.121:0/2408376714 wait complete. 2023-11-15T03:03:24.848 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-15T03:03:24.848 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:03:25.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.005+0000 7f65e27f9700 1 Processor -- start 2023-11-15T03:03:25.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.005+0000 7f65e27f9700 1 -- start start 2023-11-15T03:03:25.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.009+0000 7f65e27f9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f65dc0c37f0 0x7f65dc0a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:25.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.009+0000 7f65e27f9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65dc0657f0 con 0x7f65dc0a9830 2023-11-15T03:03:25.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.009+0000 7f65e27f9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65dc065ad0 con 0x7f65dc151870 2023-11-15T03:03:25.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.009+0000 7f65e27f9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65dc0aa260 con 0x7f65dc0c37f0 2023-11-15T03:03:25.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.009+0000 7f65e1ff8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f65dc0c37f0 0x7f65dc0a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:25.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.009+0000 7f65e0ff6700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f65dc0a9830 0x7f65dc0c30e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45306/0 (socket says 172.21.15.121:45306) 2023-11-15T03:03:25.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.009+0000 7f65e0ff6700 1 -- 172.21.15.121:0/1591446338 learned_addr learned my addr 172.21.15.121:0/1591446338 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:25.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.009+0000 7f65e1ff8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f65dc0c37f0 0x7f65dc0a9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49644/0 (socket says 172.21.15.121:49644) 2023-11-15T03:03:25.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.009+0000 7f65d3fff700 1 -- 172.21.15.121:0/1591446338 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913844339 0 0) 0x7f65dc0657f0 con 0x7f65dc0a9830 2023-11-15T03:03:25.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.009+0000 7f65d3fff700 1 -- 172.21.15.121:0/1591446338 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65c80033f0 con 0x7f65dc0a9830 2023-11-15T03:03:25.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.009+0000 7f65e1ff8700 1 -- 172.21.15.121:0/1591446338 >> v1:172.21.15.190:6789/0 conn(0x7f65dc151870 legacy=0x7f65dc151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:25.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.009+0000 7f65e1ff8700 1 -- 172.21.15.121:0/1591446338 >> v1:172.21.15.121:6789/0 conn(0x7f65dc0a9830 legacy=0x7f65dc0c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:25.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.009+0000 7f65e1ff8700 1 -- 172.21.15.121:0/1591446338 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc0aa540 con 0x7f65dc0c37f0 2023-11-15T03:03:25.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.009+0000 7f65e1ff8700 1 --2- 172.21.15.121:0/1591446338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65dc0c37f0 0x7f65dc0a9c70 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7f65d4009e60 tx=0x7f65d4009780).ready entity=mon.0 client_cookie=528fe25b7e6d9c3c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:25.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.013+0000 7f65d3fff700 1 -- 172.21.15.121:0/1591446338 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d400e070 con 0x7f65dc0c37f0 2023-11-15T03:03:25.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.013+0000 7f65d3fff700 1 -- 172.21.15.121:0/1591446338 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d400bb60 con 0x7f65dc0c37f0 2023-11-15T03:03:25.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.013+0000 7f65d3fff700 1 -- 172.21.15.121:0/1591446338 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4012410 con 0x7f65dc0c37f0 2023-11-15T03:03:25.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.013+0000 7f65e27f9700 1 -- 172.21.15.121:0/1591446338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65dc0c37f0 msgr2=0x7f65dc0a9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:25.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.013+0000 7f65e27f9700 1 --2- 172.21.15.121:0/1591446338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65dc0c37f0 0x7f65dc0a9c70 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7f65d4009e60 tx=0x7f65d4009780).stop 2023-11-15T03:03:25.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.013+0000 7f65e27f9700 1 -- 172.21.15.121:0/1591446338 shutdown_connections 2023-11-15T03:03:25.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.013+0000 7f65e27f9700 1 --2- 172.21.15.121:0/1591446338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65dc0c37f0 0x7f65dc0a9c70 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.013+0000 7f65e27f9700 1 -- 172.21.15.121:0/1591446338 >> 172.21.15.121:0/1591446338 conn(0x7f65dc062190 msgr2=0x7f65dc062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:25.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.013+0000 7f65e27f9700 1 -- 172.21.15.121:0/1591446338 shutdown_connections 2023-11-15T03:03:25.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.013+0000 7f65e27f9700 1 -- 172.21.15.121:0/1591446338 wait complete. 2023-11-15T03:03:25.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.017+0000 7f65e27f9700 1 Processor -- start 2023-11-15T03:03:25.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.017+0000 7f65e27f9700 1 -- start start 2023-11-15T03:03:25.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.017+0000 7f65e27f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65dc0a9830 0x7f65dc1450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:25.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.017+0000 7f65e27f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65dc0c37f0 0x7f65dc0c1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:25.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.017+0000 7f65e27f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65dc151870 0x7f65dc1553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:25.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.017+0000 7f65e27f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc163e10 con 0x7f65dc0c37f0 2023-11-15T03:03:25.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.017+0000 7f65e27f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc0c2090 con 0x7f65dc151870 2023-11-15T03:03:25.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.017+0000 7f65e27f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc0c2370 con 0x7f65dc0a9830 2023-11-15T03:03:25.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.017+0000 7f65e0ff6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65dc0c37f0 0x7f65dc0c1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:25.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.017+0000 7f65e0ff6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65dc0c37f0 0x7f65dc0c1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49658/0 (socket says 172.21.15.121:49658) 2023-11-15T03:03:25.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.017+0000 7f65e0ff6700 1 -- 172.21.15.121:0/4112198563 learned_addr learned my addr 172.21.15.121:0/4112198563 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:25.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.017+0000 7f65e1ff8700 1 --2- 172.21.15.121:0/4112198563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65dc151870 0x7f65dc1553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:25.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.017+0000 7f65e17f7700 1 --2- 172.21.15.121:0/4112198563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65dc0a9830 0x7f65dc1450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:25.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.017+0000 7f65e0ff6700 1 -- 172.21.15.121:0/4112198563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65dc0a9830 msgr2=0x7f65dc1450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:25.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.017+0000 7f65e0ff6700 1 --2- 172.21.15.121:0/4112198563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65dc0a9830 0x7f65dc1450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.017+0000 7f65e0ff6700 1 -- 172.21.15.121:0/4112198563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65dc151870 msgr2=0x7f65dc1553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:25.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.017+0000 7f65e0ff6700 1 --2- 172.21.15.121:0/4112198563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65dc151870 0x7f65dc1553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.017+0000 7f65e0ff6700 1 -- 172.21.15.121:0/4112198563 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d4009a20 con 0x7f65dc0c37f0 2023-11-15T03:03:25.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.017+0000 7f65e1ff8700 1 --2- 172.21.15.121:0/4112198563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65dc151870 0x7f65dc1553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:25.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.021+0000 7f65e0ff6700 1 --2- 172.21.15.121:0/4112198563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65dc0c37f0 0x7f65dc0c1720 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f65cc00c590 tx=0x7f65cc007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:25.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.021+0000 7f65d17fa700 1 -- 172.21.15.121:0/4112198563 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc007e40 con 0x7f65dc0c37f0 2023-11-15T03:03:25.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.021+0000 7f65d17fa700 1 -- 172.21.15.121:0/4112198563 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65cc0072e0 con 0x7f65dc0c37f0 2023-11-15T03:03:25.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.021+0000 7f65e27f9700 1 -- 172.21.15.121:0/4112198563 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65dc0c2650 con 0x7f65dc0c37f0 2023-11-15T03:03:25.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.021+0000 7f65d17fa700 1 -- 172.21.15.121:0/4112198563 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc0057a0 con 0x7f65dc0c37f0 2023-11-15T03:03:25.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.021+0000 7f65e27f9700 1 -- 172.21.15.121:0/4112198563 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65dc145590 con 0x7f65dc0c37f0 2023-11-15T03:03:25.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.021+0000 7f65e27f9700 1 -- 172.21.15.121:0/4112198563 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65dc0c2890 con 0x7f65dc0c37f0 2023-11-15T03:03:25.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.025+0000 7f65d17fa700 1 -- 172.21.15.121:0/4112198563 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f65cc005bd0 con 0x7f65dc0c37f0 2023-11-15T03:03:25.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.025+0000 7f65d17fa700 1 --2- 172.21.15.121:0/4112198563 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f65c005a560 0x7f65c005ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:25.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.025+0000 7f65d17fa700 1 -- 172.21.15.121:0/4112198563 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f65cc073df0 con 0x7f65dc0c37f0 2023-11-15T03:03:25.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.025+0000 7f65e17f7700 1 --2- 172.21.15.121:0/4112198563 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f65c005a560 0x7f65c005ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:25.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.025+0000 7f65e17f7700 1 --2- 172.21.15.121:0/4112198563 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f65c005a560 0x7f65c005ca20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f65d8003630 tx=0x7f65d8009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:25.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.029+0000 7f65d17fa700 1 -- 172.21.15.121:0/4112198563 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f65dc0c2890 con 0x7f65dc0c37f0 2023-11-15T03:03:25.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.229+0000 7f65e27f9700 1 -- 172.21.15.121:0/4112198563 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65dc06bcf0 con 0x7f65dc0c37f0 2023-11-15T03:03:25.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.233+0000 7f65d17fa700 1 -- 172.21.15.121:0/4112198563 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4500 (secure 0 0 0) 0x7f65cc018070 con 0x7f65dc0c37f0 2023-11-15T03:03:25.237 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:25.237 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":81,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:03:24.804563+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6966},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6966":{"gid":6966,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2646177140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2646177140},{"type":"v1","addr":"172.21.15.121:6836","nonce":2646177140}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:03:25.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.237+0000 7f65e27f9700 1 -- 172.21.15.121:0/4112198563 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f65c005a560 msgr2=0x7f65c005ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:25.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.237+0000 7f65e27f9700 1 --2- 172.21.15.121:0/4112198563 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f65c005a560 0x7f65c005ca20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f65d8003630 tx=0x7f65d8009040).stop 2023-11-15T03:03:25.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.237+0000 7f65e27f9700 1 -- 172.21.15.121:0/4112198563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65dc0c37f0 msgr2=0x7f65dc0c1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:25.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.237+0000 7f65e27f9700 1 --2- 172.21.15.121:0/4112198563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65dc0c37f0 0x7f65dc0c1720 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f65cc00c590 tx=0x7f65cc007c80).stop 2023-11-15T03:03:25.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.237+0000 7f65e27f9700 1 -- 172.21.15.121:0/4112198563 shutdown_connections 2023-11-15T03:03:25.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.237+0000 7f65e27f9700 1 --2- 172.21.15.121:0/4112198563 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f65c005a560 0x7f65c005ca20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.237+0000 7f65e27f9700 1 --2- 172.21.15.121:0/4112198563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65dc151870 0x7f65dc1553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.237+0000 7f65e27f9700 1 --2- 172.21.15.121:0/4112198563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65dc0c37f0 0x7f65dc0c1720 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.237+0000 7f65e27f9700 1 --2- 172.21.15.121:0/4112198563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65dc0a9830 0x7f65dc1450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.237+0000 7f65e27f9700 1 -- 172.21.15.121:0/4112198563 >> 172.21.15.121:0/4112198563 conn(0x7f65dc062190 msgr2=0x7f65dc0c44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:25.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.237+0000 7f65e27f9700 1 -- 172.21.15.121:0/4112198563 shutdown_connections 2023-11-15T03:03:25.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.237+0000 7f65e27f9700 1 -- 172.21.15.121:0/4112198563 wait complete. 2023-11-15T03:03:25.248 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 81 2023-11-15T03:03:25.264 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:03:25.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.421+0000 7f364161b700 1 Processor -- start 2023-11-15T03:03:25.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.421+0000 7f364161b700 1 -- start start 2023-11-15T03:03:25.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.421+0000 7f364161b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f363c0be230 0x7f363c0be610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:25.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.421+0000 7f364161b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f363c0c23f0 0x7f363c0c2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:25.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f364161b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f363c157be0 0x7f363c158070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:25.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f364161b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363c15bfa0 con 0x7f363c157be0 2023-11-15T03:03:25.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f364161b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363c0c2d20 con 0x7f363c0be230 2023-11-15T03:03:25.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f364161b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c0c3d90 con 0x7f363c0c23f0 2023-11-15T03:03:25.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f363bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f363c157be0 0x7f363c158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:25.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f363a7fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f363c0c23f0 0x7f363c0c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:25.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f363affd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f363c0be230 0x7f363c0be610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:25.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f363bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f363c157be0 0x7f363c158070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49666/0 (socket says 172.21.15.121:49666) 2023-11-15T03:03:25.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f363bfff700 1 -- 172.21.15.121:0/2696320574 learned_addr learned my addr 172.21.15.121:0/2696320574 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:25.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f363affd700 1 -- 172.21.15.121:0/2696320574 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f363c0c23f0 msgr2=0x7f363c0c2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:25.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f363affd700 1 --2- 172.21.15.121:0/2696320574 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f363c0c23f0 0x7f363c0c2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f363affd700 1 -- 172.21.15.121:0/2696320574 >> v2:172.21.15.121:3300/0 conn(0x7f363c157be0 msgr2=0x7f363c158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:25.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f363affd700 1 --2- 172.21.15.121:0/2696320574 >> v2:172.21.15.121:3300/0 conn(0x7f363c157be0 0x7f363c158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f363affd700 1 -- 172.21.15.121:0/2696320574 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c0c2f60 con 0x7f363c0be230 2023-11-15T03:03:25.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f363bfff700 1 --2- 172.21.15.121:0/2696320574 >> v2:172.21.15.121:3300/0 conn(0x7f363c157be0 0x7f363c158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:03:25.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f363affd700 1 --2- 172.21.15.121:0/2696320574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f363c0be230 0x7f363c0be610 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f363c0c8dd0 tx=0x7f362c009060).ready entity=mon.1 client_cookie=48c91ee2f4f267e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:25.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f3639ffb700 1 -- 172.21.15.121:0/2696320574 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c00f510 con 0x7f363c0be230 2023-11-15T03:03:25.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f3639ffb700 1 -- 172.21.15.121:0/2696320574 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c00f670 con 0x7f363c0be230 2023-11-15T03:03:25.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f3639ffb700 1 -- 172.21.15.121:0/2696320574 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c00f9f0 con 0x7f363c0be230 2023-11-15T03:03:25.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f364161b700 1 -- 172.21.15.121:0/2696320574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f363c0be230 msgr2=0x7f363c0be610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:25.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f364161b700 1 --2- 172.21.15.121:0/2696320574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f363c0be230 0x7f363c0be610 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f363c0c8dd0 tx=0x7f362c009060).stop 2023-11-15T03:03:25.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f364161b700 1 -- 172.21.15.121:0/2696320574 shutdown_connections 2023-11-15T03:03:25.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f364161b700 1 --2- 172.21.15.121:0/2696320574 >> v2:172.21.15.121:3300/0 conn(0x7f363c157be0 0x7f363c158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f364161b700 1 --2- 172.21.15.121:0/2696320574 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f363c0c23f0 0x7f363c0c2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f364161b700 1 --2- 172.21.15.121:0/2696320574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f363c0be230 0x7f363c0be610 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.425+0000 7f364161b700 1 -- 172.21.15.121:0/2696320574 >> 172.21.15.121:0/2696320574 conn(0x7f363c069ac0 msgr2=0x7f363c090a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:25.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.429+0000 7f364161b700 1 -- 172.21.15.121:0/2696320574 shutdown_connections 2023-11-15T03:03:25.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.429+0000 7f364161b700 1 -- 172.21.15.121:0/2696320574 wait complete. 2023-11-15T03:03:25.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.429+0000 7f364161b700 1 Processor -- start 2023-11-15T03:03:25.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.429+0000 7f364161b700 1 -- start start 2023-11-15T03:03:25.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f364161b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f363c0be230 0x7f363c1683c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:25.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f364161b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f363c0c23f0 0x7f363c16c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:25.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f364161b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f363c157be0 0x7f363c156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:25.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f364161b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c1688b0 con 0x7f363c0c23f0 2023-11-15T03:03:25.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f364161b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c0ad910 con 0x7f363c0be230 2023-11-15T03:03:25.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f364161b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c16ce20 con 0x7f363c157be0 2023-11-15T03:03:25.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f363a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f363c0c23f0 0x7f363c16c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:25.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f363a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f363c0c23f0 0x7f363c16c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49676/0 (socket says 172.21.15.121:49676) 2023-11-15T03:03:25.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f363a7fc700 1 -- 172.21.15.121:0/3847326406 learned_addr learned my addr 172.21.15.121:0/3847326406 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:25.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f363affd700 1 --2- 172.21.15.121:0/3847326406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f363c0be230 0x7f363c1683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:25.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f363bfff700 1 --2- 172.21.15.121:0/3847326406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f363c157be0 0x7f363c156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:25.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f363bfff700 1 -- 172.21.15.121:0/3847326406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f363c0be230 msgr2=0x7f363c1683c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:25.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f363bfff700 1 --2- 172.21.15.121:0/3847326406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f363c0be230 0x7f363c1683c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f363bfff700 1 -- 172.21.15.121:0/3847326406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f363c0c23f0 msgr2=0x7f363c16c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:25.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f363bfff700 1 --2- 172.21.15.121:0/3847326406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f363c0c23f0 0x7f363c16c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f363bfff700 1 -- 172.21.15.121:0/3847326406 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3634009ae0 con 0x7f363c157be0 2023-11-15T03:03:25.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f363a7fc700 1 --2- 172.21.15.121:0/3847326406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f363c0c23f0 0x7f363c16c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:03:25.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f363bfff700 1 --2- 172.21.15.121:0/3847326406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f363c157be0 0x7f363c156e40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f3634009e70 tx=0x7f363400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:25.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f3622ffd700 1 -- 172.21.15.121:0/3847326406 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36340092e0 con 0x7f363c157be0 2023-11-15T03:03:25.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f3622ffd700 1 -- 172.21.15.121:0/3847326406 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363400f040 con 0x7f363c157be0 2023-11-15T03:03:25.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f364161b700 1 -- 172.21.15.121:0/3847326406 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c0092a0 con 0x7f363c157be0 2023-11-15T03:03:25.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f3622ffd700 1 -- 172.21.15.121:0/3847326406 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363400caa0 con 0x7f363c157be0 2023-11-15T03:03:25.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.433+0000 7f364161b700 1 -- 172.21.15.121:0/3847326406 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363c0acd00 con 0x7f363c157be0 2023-11-15T03:03:25.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.437+0000 7f364161b700 1 -- 172.21.15.121:0/3847326406 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363c0adaa0 con 0x7f363c157be0 2023-11-15T03:03:25.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.437+0000 7f3622ffd700 1 -- 172.21.15.121:0/3847326406 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3634014460 con 0x7f363c157be0 2023-11-15T03:03:25.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.441+0000 7f3622ffd700 1 --2- 172.21.15.121:0/3847326406 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f362805a510 0x7f362805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:25.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.441+0000 7f3622ffd700 1 -- 172.21.15.121:0/3847326406 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3634074670 con 0x7f363c157be0 2023-11-15T03:03:25.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.441+0000 7f363affd700 1 --2- 172.21.15.121:0/3847326406 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f362805a510 0x7f362805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:25.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.441+0000 7f363affd700 1 --2- 172.21.15.121:0/3847326406 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f362805a510 0x7f362805c9d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f363c0c8dd0 tx=0x7f362c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:25.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.449+0000 7f3622ffd700 1 -- 172.21.15.121:0/3847326406 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3634047de0 con 0x7f363c157be0 2023-11-15T03:03:25.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.609+0000 7f364161b700 1 -- 172.21.15.121:0/3847326406 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f363c0c3240 con 0x7f363c157be0 2023-11-15T03:03:25.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.609+0000 7f3622ffd700 1 -- 172.21.15.121:0/3847326406 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16979 (secure 0 0 0) 0x7f363404b400 con 0x7f363c157be0 2023-11-15T03:03:25.617 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:25.617 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":92,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:03:22.790219+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-15T03:03:19.872820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-15T03:03:20.771478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:03:25.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.613+0000 7f364161b700 1 -- 172.21.15.121:0/3847326406 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f362805a510 msgr2=0x7f362805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:25.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.613+0000 7f364161b700 1 --2- 172.21.15.121:0/3847326406 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f362805a510 0x7f362805c9d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f363c0c8dd0 tx=0x7f362c017000).stop 2023-11-15T03:03:25.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.617+0000 7f364161b700 1 -- 172.21.15.121:0/3847326406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f363c157be0 msgr2=0x7f363c156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:25.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.617+0000 7f364161b700 1 --2- 172.21.15.121:0/3847326406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f363c157be0 0x7f363c156e40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f3634009e70 tx=0x7f363400b040).stop 2023-11-15T03:03:25.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.617+0000 7f364161b700 1 -- 172.21.15.121:0/3847326406 shutdown_connections 2023-11-15T03:03:25.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.617+0000 7f364161b700 1 --2- 172.21.15.121:0/3847326406 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f362805a510 0x7f362805c9d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.617+0000 7f364161b700 1 --2- 172.21.15.121:0/3847326406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f363c157be0 0x7f363c156e40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.617+0000 7f364161b700 1 --2- 172.21.15.121:0/3847326406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f363c0c23f0 0x7f363c16c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.617+0000 7f364161b700 1 --2- 172.21.15.121:0/3847326406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f363c0be230 0x7f363c1683c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.617+0000 7f364161b700 1 -- 172.21.15.121:0/3847326406 >> 172.21.15.121:0/3847326406 conn(0x7f363c069ac0 msgr2=0x7f363c0a6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:25.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.617+0000 7f364161b700 1 -- 172.21.15.121:0/3847326406 shutdown_connections 2023-11-15T03:03:25.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.617+0000 7f364161b700 1 -- 172.21.15.121:0/3847326406 wait complete. 2023-11-15T03:03:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-15T03:03:25.643 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T03:03:25.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.801+0000 7f6a6cc9b700 1 Processor -- start 2023-11-15T03:03:25.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.801+0000 7f6a6cc9b700 1 -- start start 2023-11-15T03:03:25.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.805+0000 7f6a6cc9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a68154ca0 0x7f6a68155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:25.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.805+0000 7f6a6cc9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6a681562f0 0x7f6a680a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:25.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.805+0000 7f6a6cc9b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a680c4c30 con 0x7f6a680a8f40 2023-11-15T03:03:25.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.805+0000 7f6a6cc9b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a680a8280 con 0x7f6a681562f0 2023-11-15T03:03:25.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.805+0000 7f6a6cc9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a680a6d40 con 0x7f6a68154ca0 2023-11-15T03:03:25.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.805+0000 7f6a6659c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a68154ca0 0x7f6a68155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:25.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.805+0000 7f6a6659c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a68154ca0 0x7f6a68155080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:52252/0 (socket says 172.21.15.121:52252) 2023-11-15T03:03:25.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.805+0000 7f6a6659c700 1 -- 172.21.15.121:0/821205626 learned_addr learned my addr 172.21.15.121:0/821205626 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:25.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.805+0000 7f6a65d9b700 1 --2- 172.21.15.121:0/821205626 >> v2:172.21.15.190:3300/0 conn(0x7f6a681562f0 0x7f6a680a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:25.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.809+0000 7f6a6659c700 1 -- 172.21.15.121:0/821205626 >> v2:172.21.15.190:3300/0 conn(0x7f6a681562f0 msgr2=0x7f6a680a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:25.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.809+0000 7f6a6659c700 1 --2- 172.21.15.121:0/821205626 >> v2:172.21.15.190:3300/0 conn(0x7f6a681562f0 0x7f6a680a8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.809+0000 7f6a6659c700 1 -- 172.21.15.121:0/821205626 >> v1:172.21.15.190:6789/0 conn(0x7f6a680a8f40 legacy=0x7f6a680a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:25.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.809+0000 7f6a6659c700 1 -- 172.21.15.121:0/821205626 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a680a6f50 con 0x7f6a68154ca0 2023-11-15T03:03:25.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.809+0000 7f6a6659c700 1 --2- 172.21.15.121:0/821205626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a68154ca0 0x7f6a68155080 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f6a58006fd0 tx=0x7f6a580090f0).ready entity=mon.2 client_cookie=a4588db0d21ec7e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:25.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.809+0000 7f6a6559a700 1 -- 172.21.15.121:0/821205626 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5800b580 con 0x7f6a68154ca0 2023-11-15T03:03:25.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.809+0000 7f6a6559a700 1 -- 172.21.15.121:0/821205626 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5800b6e0 con 0x7f6a68154ca0 2023-11-15T03:03:25.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.809+0000 7f6a6559a700 1 -- 172.21.15.121:0/821205626 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5800ba60 con 0x7f6a68154ca0 2023-11-15T03:03:25.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.809+0000 7f6a6cc9b700 1 -- 172.21.15.121:0/821205626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a68154ca0 msgr2=0x7f6a68155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:25.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.809+0000 7f6a6cc9b700 1 --2- 172.21.15.121:0/821205626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a68154ca0 0x7f6a68155080 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f6a58006fd0 tx=0x7f6a580090f0).stop 2023-11-15T03:03:25.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.809+0000 7f6a6cc9b700 1 -- 172.21.15.121:0/821205626 shutdown_connections 2023-11-15T03:03:25.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.809+0000 7f6a6cc9b700 1 --2- 172.21.15.121:0/821205626 >> v2:172.21.15.190:3300/0 conn(0x7f6a681562f0 0x7f6a680a8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.809+0000 7f6a6cc9b700 1 --2- 172.21.15.121:0/821205626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a68154ca0 0x7f6a68155080 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.809+0000 7f6a6cc9b700 1 -- 172.21.15.121:0/821205626 >> 172.21.15.121:0/821205626 conn(0x7f6a68062190 msgr2=0x7f6a680625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:25.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.809+0000 7f6a6cc9b700 1 -- 172.21.15.121:0/821205626 shutdown_connections 2023-11-15T03:03:25.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.809+0000 7f6a6cc9b700 1 -- 172.21.15.121:0/821205626 wait complete. 2023-11-15T03:03:25.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.813+0000 7f6a6cc9b700 1 Processor -- start 2023-11-15T03:03:25.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.813+0000 7f6a6cc9b700 1 -- start start 2023-11-15T03:03:25.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.817+0000 7f6a6cc9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a680a8f40 0x7f6a68066270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:25.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.817+0000 7f6a6cc9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a68154ca0 0x7f6a681516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:25.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.817+0000 7f6a6cc9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a681562f0 0x7f6a68151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:25.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.817+0000 7f6a6cc9b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a680ab1d0 con 0x7f6a68154ca0 2023-11-15T03:03:25.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.817+0000 7f6a6cc9b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a681521b0 con 0x7f6a680a8f40 2023-11-15T03:03:25.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.817+0000 7f6a6cc9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a680aa8a0 con 0x7f6a681562f0 2023-11-15T03:03:25.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.817+0000 7f6a65d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a68154ca0 0x7f6a681516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:25.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.817+0000 7f6a65d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a68154ca0 0x7f6a681516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49684/0 (socket says 172.21.15.121:49684) 2023-11-15T03:03:25.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.817+0000 7f6a65d9b700 1 -- 172.21.15.121:0/2337475878 learned_addr learned my addr 172.21.15.121:0/2337475878 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:25.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.817+0000 7f6a6659c700 1 --2- 172.21.15.121:0/2337475878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a680a8f40 0x7f6a68066270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:25.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.817+0000 7f6a66d9d700 1 --2- 172.21.15.121:0/2337475878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a681562f0 0x7f6a68151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:25.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.817+0000 7f6a65d9b700 1 -- 172.21.15.121:0/2337475878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a681562f0 msgr2=0x7f6a68151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:25.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.817+0000 7f6a65d9b700 1 --2- 172.21.15.121:0/2337475878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a681562f0 0x7f6a68151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.817+0000 7f6a65d9b700 1 -- 172.21.15.121:0/2337475878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a680a8f40 msgr2=0x7f6a68066270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:25.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.817+0000 7f6a65d9b700 1 --2- 172.21.15.121:0/2337475878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a680a8f40 0x7f6a68066270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:25.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.817+0000 7f6a65d9b700 1 -- 172.21.15.121:0/2337475878 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a58009370 con 0x7f6a68154ca0 2023-11-15T03:03:25.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.817+0000 7f6a66d9d700 1 --2- 172.21.15.121:0/2337475878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a681562f0 0x7f6a68151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:25.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.817+0000 7f6a65d9b700 1 --2- 172.21.15.121:0/2337475878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a68154ca0 0x7f6a681516f0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f6a5c00e8d0 tx=0x7f6a5c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:25.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.821+0000 7f6a567fc700 1 -- 172.21.15.121:0/2337475878 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c00c640 con 0x7f6a68154ca0 2023-11-15T03:03:25.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.821+0000 7f6a6cc9b700 1 -- 172.21.15.121:0/2337475878 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a680aab80 con 0x7f6a68154ca0 2023-11-15T03:03:25.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.821+0000 7f6a567fc700 1 -- 172.21.15.121:0/2337475878 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5c00c7a0 con 0x7f6a68154ca0 2023-11-15T03:03:25.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.821+0000 7f6a567fc700 1 -- 172.21.15.121:0/2337475878 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c00cb40 con 0x7f6a68154ca0 2023-11-15T03:03:25.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.821+0000 7f6a6cc9b700 1 -- 172.21.15.121:0/2337475878 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6813bb10 con 0x7f6a68154ca0 2023-11-15T03:03:25.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.821+0000 7f6a567fc700 1 -- 172.21.15.121:0/2337475878 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6a5c0123f0 con 0x7f6a68154ca0 2023-11-15T03:03:25.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.821+0000 7f6a6cc9b700 1 -- 172.21.15.121:0/2337475878 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a680ab380 con 0x7f6a68154ca0 2023-11-15T03:03:25.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.825+0000 7f6a567fc700 1 --2- 172.21.15.121:0/2337475878 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6a4805a510 0x7f6a4805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:25.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.825+0000 7f6a567fc700 1 -- 172.21.15.121:0/2337475878 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6a5c074870 con 0x7f6a68154ca0 2023-11-15T03:03:25.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.825+0000 7f6a6659c700 1 --2- 172.21.15.121:0/2337475878 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6a4805a510 0x7f6a4805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:25.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.825+0000 7f6a6659c700 1 --2- 172.21.15.121:0/2337475878 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6a4805a510 0x7f6a4805c9d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f6a58009fd0 tx=0x7f6a58017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:25.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.829+0000 7f6a567fc700 1 -- 172.21.15.121:0/2337475878 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6a5c047ff0 con 0x7f6a68154ca0 2023-11-15T03:03:25.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:25.989+0000 7f6a6cc9b700 1 -- 172.21.15.121:0/2337475878 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6a6806c150 con 0x7f6a4805a510 2023-11-15T03:03:26.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.001+0000 7f6a567fc700 1 -- 172.21.15.121:0/2337475878 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+348928 (secure 0 0 0) 0x7f6a6806c150 con 0x7f6a4805a510 2023-11-15T03:03:26.016 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:26.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.013+0000 7f6a6cc9b700 1 -- 172.21.15.121:0/2337475878 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6a4805a510 msgr2=0x7f6a4805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:26.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.013+0000 7f6a6cc9b700 1 --2- 172.21.15.121:0/2337475878 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6a4805a510 0x7f6a4805c9d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f6a58009fd0 tx=0x7f6a58017000).stop 2023-11-15T03:03:26.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.013+0000 7f6a6cc9b700 1 -- 172.21.15.121:0/2337475878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a68154ca0 msgr2=0x7f6a681516f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:26.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.013+0000 7f6a6cc9b700 1 --2- 172.21.15.121:0/2337475878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a68154ca0 0x7f6a681516f0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f6a5c00e8d0 tx=0x7f6a5c009060).stop 2023-11-15T03:03:26.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.013+0000 7f6a6cc9b700 1 -- 172.21.15.121:0/2337475878 shutdown_connections 2023-11-15T03:03:26.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.013+0000 7f6a6cc9b700 1 --2- 172.21.15.121:0/2337475878 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6a4805a510 0x7f6a4805c9d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:26.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.013+0000 7f6a6cc9b700 1 --2- 172.21.15.121:0/2337475878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a681562f0 0x7f6a68151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:26.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.013+0000 7f6a6cc9b700 1 --2- 172.21.15.121:0/2337475878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a68154ca0 0x7f6a681516f0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:26.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.013+0000 7f6a6cc9b700 1 --2- 172.21.15.121:0/2337475878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a680a8f40 0x7f6a68066270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:26.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.013+0000 7f6a6cc9b700 1 -- 172.21.15.121:0/2337475878 >> 172.21.15.121:0/2337475878 conn(0x7f6a68062190 msgr2=0x7f6a6814efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:26.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.013+0000 7f6a6cc9b700 1 -- 172.21.15.121:0/2337475878 shutdown_connections 2023-11-15T03:03:26.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.013+0000 7f6a6cc9b700 1 -- 172.21.15.121:0/2337475878 wait complete. 2023-11-15T03:03:26.024 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T03:03:26.047 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2023-11-15T03:03:24.063638+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2385216,"kb_used_data":8448,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752589504,"statfs":{"total":773094113280,"available":770651652096,"internally_reserved":0,"allocated":8650752,"data_stored":3973530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19397,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-17,"num_read_kb":-19,"num_write":-82,"num_write_kb":-44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.176614"},"pg_stats":[{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.830211+0000","last_change":"2023-11-15T03:03:21.819554+0000","last_active":"2023-11-15T03:03:21.830211+0000","last_peered":"2023-11-15T03:03:21.830211+0000","last_clean":"2023-11-15T03:03:21.830211+0000","last_became_active":"2023-11-15T03:03:21.818891+0000","last_became_peered":"2023-11-15T03:03:21.818891+0000","last_unstale":"2023-11-15T03:03:21.830211+0000","last_undegraded":"2023-11-15T03:03:21.830211+0000","last_fullsized":"2023-11-15T03:03:21.830211+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.818317+0000","last_change":"2023-11-15T03:03:21.351823+0000","last_active":"2023-11-15T03:03:21.818317+0000","last_peered":"2023-11-15T03:03:21.818317+0000","last_clean":"2023-11-15T03:03:21.818317+0000","last_became_active":"2023-11-15T03:03:21.351514+0000","last_became_peered":"2023-11-15T03:03:21.351514+0000","last_unstale":"2023-11-15T03:03:21.818317+0000","last_undegraded":"2023-11-15T03:03:21.818317+0000","last_fullsized":"2023-11-15T03:03:21.818317+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.824956+0000","last_change":"2023-11-15T03:03:21.814549+0000","last_active":"2023-11-15T03:03:21.824956+0000","last_peered":"2023-11-15T03:03:21.824956+0000","last_clean":"2023-11-15T03:03:21.824956+0000","last_became_active":"2023-11-15T03:03:21.814150+0000","last_became_peered":"2023-11-15T03:03:21.814150+0000","last_unstale":"2023-11-15T03:03:21.824956+0000","last_undegraded":"2023-11-15T03:03:21.824956+0000","last_fullsized":"2023-11-15T03:03:21.824956+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.816449+0000","last_change":"2023-11-15T03:03:21.351141+0000","last_active":"2023-11-15T03:03:21.816449+0000","last_peered":"2023-11-15T03:03:21.816449+0000","last_clean":"2023-11-15T03:03:21.816449+0000","last_became_active":"2023-11-15T03:03:21.350890+0000","last_became_peered":"2023-11-15T03:03:21.350890+0000","last_unstale":"2023-11-15T03:03:21.816449+0000","last_undegraded":"2023-11-15T03:03:21.816449+0000","last_fullsized":"2023-11-15T03:03:21.816449+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.830192+0000","last_change":"2023-11-15T03:03:21.819428+0000","last_active":"2023-11-15T03:03:21.830192+0000","last_peered":"2023-11-15T03:03:21.830192+0000","last_clean":"2023-11-15T03:03:21.830192+0000","last_became_active":"2023-11-15T03:03:21.819132+0000","last_became_peered":"2023-11-15T03:03:21.819132+0000","last_unstale":"2023-11-15T03:03:21.830192+0000","last_undegraded":"2023-11-15T03:03:21.830192+0000","last_fullsized":"2023-11-15T03:03:21.830192+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.830405+0000","last_change":"2023-11-15T03:03:21.820480+0000","last_active":"2023-11-15T03:03:21.830405+0000","last_peered":"2023-11-15T03:03:21.830405+0000","last_clean":"2023-11-15T03:03:21.830405+0000","last_became_active":"2023-11-15T03:03:21.820118+0000","last_became_peered":"2023-11-15T03:03:21.820118+0000","last_unstale":"2023-11-15T03:03:21.830405+0000","last_undegraded":"2023-11-15T03:03:21.830405+0000","last_fullsized":"2023-11-15T03:03:21.830405+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.830594+0000","last_change":"2023-11-15T03:03:21.823107+0000","last_active":"2023-11-15T03:03:21.830594+0000","last_peered":"2023-11-15T03:03:21.830594+0000","last_clean":"2023-11-15T03:03:21.830594+0000","last_became_active":"2023-11-15T03:03:21.820441+0000","last_became_peered":"2023-11-15T03:03:21.820441+0000","last_unstale":"2023-11-15T03:03:21.830594+0000","last_undegraded":"2023-11-15T03:03:21.830594+0000","last_fullsized":"2023-11-15T03:03:21.830594+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.817488+0000","last_change":"2023-11-15T03:03:21.351355+0000","last_active":"2023-11-15T03:03:21.817488+0000","last_peered":"2023-11-15T03:03:21.817488+0000","last_clean":"2023-11-15T03:03:21.817488+0000","last_became_active":"2023-11-15T03:03:21.351073+0000","last_became_peered":"2023-11-15T03:03:21.351073+0000","last_unstale":"2023-11-15T03:03:21.817488+0000","last_undegraded":"2023-11-15T03:03:21.817488+0000","last_fullsized":"2023-11-15T03:03:21.817488+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.836644+0000","last_change":"2023-11-15T03:03:21.832368+0000","last_active":"2023-11-15T03:03:21.836644+0000","last_peered":"2023-11-15T03:03:21.836644+0000","last_clean":"2023-11-15T03:03:21.836644+0000","last_became_active":"2023-11-15T03:03:21.832108+0000","last_became_peered":"2023-11-15T03:03:21.832108+0000","last_unstale":"2023-11-15T03:03:21.836644+0000","last_undegraded":"2023-11-15T03:03:21.836644+0000","last_fullsized":"2023-11-15T03:03:21.836644+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.816562+0000","last_change":"2023-11-15T03:03:21.353299+0000","last_active":"2023-11-15T03:03:21.816562+0000","last_peered":"2023-11-15T03:03:21.816562+0000","last_clean":"2023-11-15T03:03:21.816562+0000","last_became_active":"2023-11-15T03:03:21.353028+0000","last_became_peered":"2023-11-15T03:03:21.353028+0000","last_unstale":"2023-11-15T03:03:21.816562+0000","last_undegraded":"2023-11-15T03:03:21.816562+0000","last_fullsized":"2023-11-15T03:03:21.816562+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.816410+0000","last_change":"2023-11-15T03:03:21.349980+0000","last_active":"2023-11-15T03:03:21.816410+0000","last_peered":"2023-11-15T03:03:21.816410+0000","last_clean":"2023-11-15T03:03:21.816410+0000","last_became_active":"2023-11-15T03:03:21.349676+0000","last_became_peered":"2023-11-15T03:03:21.349676+0000","last_unstale":"2023-11-15T03:03:21.816410+0000","last_undegraded":"2023-11-15T03:03:21.816410+0000","last_fullsized":"2023-11-15T03:03:21.816410+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.817281+0000","last_change":"2023-11-15T03:03:21.359062+0000","last_active":"2023-11-15T03:03:21.817281+0000","last_peered":"2023-11-15T03:03:21.817281+0000","last_clean":"2023-11-15T03:03:21.817281+0000","last_became_active":"2023-11-15T03:03:21.358623+0000","last_became_peered":"2023-11-15T03:03:21.358623+0000","last_unstale":"2023-11-15T03:03:21.817281+0000","last_undegraded":"2023-11-15T03:03:21.817281+0000","last_fullsized":"2023-11-15T03:03:21.817281+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.830433+0000","last_change":"2023-11-15T03:03:21.817138+0000","last_active":"2023-11-15T03:03:21.830433+0000","last_peered":"2023-11-15T03:03:21.830433+0000","last_clean":"2023-11-15T03:03:21.830433+0000","last_became_active":"2023-11-15T03:03:21.816865+0000","last_became_peered":"2023-11-15T03:03:21.816865+0000","last_unstale":"2023-11-15T03:03:21.830433+0000","last_undegraded":"2023-11-15T03:03:21.830433+0000","last_fullsized":"2023-11-15T03:03:21.830433+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.817379+0000","last_change":"2023-11-15T03:03:21.359616+0000","last_active":"2023-11-15T03:03:21.817379+0000","last_peered":"2023-11-15T03:03:21.817379+0000","last_clean":"2023-11-15T03:03:21.817379+0000","last_became_active":"2023-11-15T03:03:21.359297+0000","last_became_peered":"2023-11-15T03:03:21.359297+0000","last_unstale":"2023-11-15T03:03:21.817379+0000","last_undegraded":"2023-11-15T03:03:21.817379+0000","last_fullsized":"2023-11-15T03:03:21.817379+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.830100+0000","last_change":"2023-11-15T03:03:21.816894+0000","last_active":"2023-11-15T03:03:21.830100+0000","last_peered":"2023-11-15T03:03:21.830100+0000","last_clean":"2023-11-15T03:03:21.830100+0000","last_became_active":"2023-11-15T03:03:21.816628+0000","last_became_peered":"2023-11-15T03:03:21.816628+0000","last_unstale":"2023-11-15T03:03:21.830100+0000","last_undegraded":"2023-11-15T03:03:21.830100+0000","last_fullsized":"2023-11-15T03:03:21.830100+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.830237+0000","last_change":"2023-11-15T03:03:21.817028+0000","last_active":"2023-11-15T03:03:21.830237+0000","last_peered":"2023-11-15T03:03:21.830237+0000","last_clean":"2023-11-15T03:03:21.830237+0000","last_became_active":"2023-11-15T03:03:21.816752+0000","last_became_peered":"2023-11-15T03:03:21.816752+0000","last_unstale":"2023-11-15T03:03:21.830237+0000","last_undegraded":"2023-11-15T03:03:21.830237+0000","last_fullsized":"2023-11-15T03:03:21.830237+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.825703+0000","last_change":"2023-11-15T03:03:21.815044+0000","last_active":"2023-11-15T03:03:21.825703+0000","last_peered":"2023-11-15T03:03:21.825703+0000","last_clean":"2023-11-15T03:03:21.825703+0000","last_became_active":"2023-11-15T03:03:21.814775+0000","last_became_peered":"2023-11-15T03:03:21.814775+0000","last_unstale":"2023-11-15T03:03:21.825703+0000","last_undegraded":"2023-11-15T03:03:21.825703+0000","last_fullsized":"2023-11-15T03:03:21.825703+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.817428+0000","last_change":"2023-11-15T03:03:21.351938+0000","last_active":"2023-11-15T03:03:21.817428+0000","last_peered":"2023-11-15T03:03:21.817428+0000","last_clean":"2023-11-15T03:03:21.817428+0000","last_became_active":"2023-11-15T03:03:21.351589+0000","last_became_peered":"2023-11-15T03:03:21.351589+0000","last_unstale":"2023-11-15T03:03:21.817428+0000","last_undegraded":"2023-11-15T03:03:21.817428+0000","last_fullsized":"2023-11-15T03:03:21.817428+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"91'4","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.833043+0000","last_change":"2023-11-15T03:03:21.359067+0000","last_active":"2023-11-15T03:03:21.833043+0000","last_peered":"2023-11-15T03:03:21.833043+0000","last_clean":"2023-11-15T03:03:21.833043+0000","last_became_active":"2023-11-15T03:03:21.358763+0000","last_became_peered":"2023-11-15T03:03:21.358763+0000","last_unstale":"2023-11-15T03:03:21.833043+0000","last_undegraded":"2023-11-15T03:03:21.833043+0000","last_fullsized":"2023-11-15T03:03:21.833043+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-15T03:03:17.180968+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T03:03:17.180968+0000","last_peered":"2023-11-15T03:03:17.180968+0000","last_clean":"2023-11-15T03:03:17.180968+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T03:03:17.180968+0000","last_undegraded":"2023-11-15T03:03:17.180968+0000","last_fullsized":"2023-11-15T03:03:17.180968+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":75,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.818847+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T03:03:21.818847+0000","last_peered":"2023-11-15T03:03:21.818847+0000","last_clean":"2023-11-15T03:03:21.818847+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T03:03:21.818847+0000","last_undegraded":"2023-11-15T03:03:21.818847+0000","last_fullsized":"2023-11-15T03:03:21.818847+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-15T03:03:17.180317+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T03:03:17.180317+0000","last_peered":"2023-11-15T03:03:17.180317+0000","last_clean":"2023-11-15T03:03:17.180317+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T03:03:17.180317+0000","last_undegraded":"2023-11-15T03:03:17.180317+0000","last_fullsized":"2023-11-15T03:03:17.180317+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":75,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.819444+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T03:03:21.819444+0000","last_peered":"2023-11-15T03:03:21.819444+0000","last_clean":"2023-11-15T03:03:21.819444+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T03:03:21.819444+0000","last_undegraded":"2023-11-15T03:03:21.819444+0000","last_fullsized":"2023-11-15T03:03:21.819444+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":80,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-15T03:03:16.422788+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T03:03:16.422788+0000","last_peered":"2023-11-15T03:03:16.422788+0000","last_clean":"2023-11-15T03:03:16.422788+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T03:03:16.422788+0000","last_undegraded":"2023-11-15T03:03:16.422788+0000","last_fullsized":"2023-11-15T03:03:16.422788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-15T03:03:17.174786+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T03:03:17.174786+0000","last_peered":"2023-11-15T03:03:17.174786+0000","last_clean":"2023-11-15T03:03:17.174786+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T03:03:17.174786+0000","last_undegraded":"2023-11-15T03:03:17.174786+0000","last_fullsized":"2023-11-15T03:03:17.174786+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:20.290946+0000","last_change":"2023-11-15T03:03:20.290946+0000","last_active":"2023-11-15T03:03:20.290946+0000","last_peered":"2023-11-15T03:03:20.290946+0000","last_clean":"2023-11-15T03:03:20.290946+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:20.290946+0000","last_undegraded":"2023-11-15T03:03:20.290946+0000","last_fullsized":"2023-11-15T03:03:20.290946+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-15T03:03:17.180276+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T03:03:17.180276+0000","last_peered":"2023-11-15T03:03:17.180276+0000","last_clean":"2023-11-15T03:03:17.180276+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T03:03:17.180276+0000","last_undegraded":"2023-11-15T03:03:17.180276+0000","last_fullsized":"2023-11-15T03:03:17.180276+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:03:21.306874+0000","last_change":"2023-11-15T03:03:21.306874+0000","last_active":"2023-11-15T03:03:21.306874+0000","last_peered":"2023-11-15T03:03:21.306874+0000","last_clean":"2023-11-15T03:03:21.306874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:03:21.306874+0000","last_undegraded":"2023-11-15T03:03:21.306874+0000","last_fullsized":"2023-11-15T03:03:21.306874+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.819365+0000","last_change":"2023-11-15T03:03:21.348694+0000","last_active":"2023-11-15T03:03:21.819365+0000","last_peered":"2023-11-15T03:03:21.819365+0000","last_clean":"2023-11-15T03:03:21.819365+0000","last_became_active":"2023-11-15T03:03:21.348422+0000","last_became_peered":"2023-11-15T03:03:21.348422+0000","last_unstale":"2023-11-15T03:03:21.819365+0000","last_undegraded":"2023-11-15T03:03:21.819365+0000","last_fullsized":"2023-11-15T03:03:21.819365+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":80,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-15T03:03:16.422838+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T03:03:16.422838+0000","last_peered":"2023-11-15T03:03:16.422838+0000","last_clean":"2023-11-15T03:03:16.422838+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T03:03:16.422838+0000","last_undegraded":"2023-11-15T03:03:16.422838+0000","last_fullsized":"2023-11-15T03:03:16.422838+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-15T03:03:17.180321+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T03:03:17.180321+0000","last_peered":"2023-11-15T03:03:17.180321+0000","last_clean":"2023-11-15T03:03:17.180321+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T03:03:17.180321+0000","last_undegraded":"2023-11-15T03:03:17.180321+0000","last_fullsized":"2023-11-15T03:03:17.180321+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":20,"acting":20,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542225,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298148,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073692,"statfs":{"total":96636764160,"available":96331460608,"internally_reserved":0,"allocated":1077248,"data_stored":494340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:03:06 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.1559999999999999,"15min":1.1819999999999999},"min":{"1min":0.56200000000000006,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4370000000000001,"5min":10.961,"15min":10.961},"last":1.159},{"interface":"front","average":{"1min":1.085,"5min":1.1160000000000001,"15min":1.181},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.5589999999999999,"5min":3.4910000000000001,"15min":3.4910000000000001},"last":1.0840000000000001}]},{"osd":1,"last update":"Wed Nov 15 03:03:06 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.157,"15min":1.151},"min":{"1min":0.84999999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.486,"5min":5.4930000000000003,"15min":5.4930000000000003},"last":1.107},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2050000000000001,"15min":1.1879999999999999},"min":{"1min":0.873,"5min":0.747,"15min":0.747},"max":{"1min":1.6970000000000001,"5min":6.8239999999999998,"15min":6.8239999999999998},"last":0.98899999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:03:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.145,"15min":1.21},"min":{"1min":0.82599999999999996,"5min":0.751,"15min":0.751},"max":{"1min":1.51,"5min":5.7000000000000002,"15min":5.7000000000000002},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.21,"15min":1.278},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5449999999999999,"5min":2.75,"15min":2.75},"last":0.94299999999999995}]},{"osd":3,"last update":"Wed Nov 15 03:03:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.3,"15min":1.3600000000000001},"min":{"1min":0.97999999999999998,"5min":0.751,"15min":0.751},"max":{"1min":1.6519999999999999,"5min":12.907,"15min":12.907},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.27,"5min":1.3120000000000001,"15min":1.2969999999999999},"min":{"1min":0.89400000000000002,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.581,"5min":9.9909999999999997,"15min":9.9909999999999997},"last":1.0089999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:03:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.101,"15min":1.244},"min":{"1min":0.72199999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.3420000000000001,"5min":1.5469999999999999,"15min":2.5649999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.1679999999999999,"15min":1.4059999999999999},"min":{"1min":0.98899999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.6080000000000001,"5min":1.6679999999999999,"15min":4.226},"last":1.131}]},{"osd":5,"last update":"Wed Nov 15 03:03:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3,"15min":1.3340000000000001},"min":{"1min":1.0620000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.6950000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.286,"15min":1.423},"min":{"1min":1.046,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.1299999999999999,"5min":2.1299999999999999,"15min":2.4550000000000001},"last":1.2749999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:03:06 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.355,"15min":1.532},"min":{"1min":1.151,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6060000000000001,"5min":1.7490000000000001,"15min":3.2789999999999999},"last":1.171},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.2949999999999999,"15min":1.546},"min":{"1min":1.0880000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.75,"5min":1.7969999999999999,"15min":4.7800000000000002},"last":1.248}]}]},{"osd":6,"up_from":14,"seq":60129542225,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298148,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073692,"statfs":{"total":96636764160,"available":96331460608,"internally_reserved":0,"allocated":1077248,"data_stored":494340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:02:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.0900000000000001,"15min":1.1220000000000001},"min":{"1min":0.73599999999999999,"5min":0.69799999999999995,"15min":0.44500000000000001},"max":{"1min":1.4379999999999999,"5min":1.7569999999999999,"15min":4.3159999999999998},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.087,"15min":1.0549999999999999},"min":{"1min":0.67700000000000005,"5min":0.61199999999999999,"15min":0.58599999999999997},"max":{"1min":1.5529999999999999,"5min":1.802,"15min":2.2480000000000002},"last":1.3180000000000001}]},{"osd":1,"last update":"Wed Nov 15 03:02:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1299999999999999,"15min":1.097},"min":{"1min":0.70699999999999996,"5min":0.63200000000000001,"15min":0.57599999999999996},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.145,"15min":1.161},"min":{"1min":0.83499999999999996,"5min":0.74199999999999999,"15min":0.60699999999999998},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":2.8570000000000002},"last":1.0640000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:02:57 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.21,"15min":1.1859999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.49099999999999999},"max":{"1min":1.4179999999999999,"5min":1.7190000000000001,"15min":2.621},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.1739999999999999,"15min":1.1100000000000001},"min":{"1min":0.72099999999999997,"5min":0.69199999999999995,"15min":0.68300000000000005},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.087}]},{"osd":3,"last update":"Wed Nov 15 03:02:57 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.212,"15min":1.1719999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.50900000000000001},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":2.0630000000000002},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.2849999999999999,"15min":1.2709999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.67100000000000004},"max":{"1min":1.633,"5min":1.7569999999999999,"15min":2.8359999999999999},"last":1.1299999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:02:57 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.0960000000000001,"15min":1.1659999999999999},"min":{"1min":0.68999999999999995,"5min":0.63500000000000001,"15min":0.56000000000000005},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":4.5350000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.298,"5min":1.2030000000000001,"15min":1.1930000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.02}]},{"osd":5,"last update":"Wed Nov 15 03:02:57 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.4359999999999999,"15min":1.417},"min":{"1min":0.75,"5min":0.75,"15min":0.72799999999999998},"max":{"1min":2.157,"5min":2.157,"15min":2.157},"last":1.413},{"interface":"front","average":{"1min":1.286,"5min":1.254,"15min":1.3},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.9460000000000002},"last":1.359}]},{"osd":7,"last update":"Wed Nov 15 03:02:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2010000000000001,"15min":1.2749999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.796,"5min":1.796,"15min":3.1989999999999998},"last":1.175},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4159999999999999,"15min":1.4450000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.7470000000000001,"5min":1.8540000000000001,"15min":2.9350000000000001},"last":1.2909999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542226,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":298148,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073692,"statfs":{"total":96636764160,"available":96331460608,"internally_reserved":0,"allocated":1077248,"data_stored":494340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.103,"15min":1.111},"min":{"1min":0.65400000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4530000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.089,"15min":1.03},"min":{"1min":0.78400000000000003,"5min":0.71099999999999997,"15min":0.66300000000000003},"max":{"1min":1.3540000000000001,"5min":1.677,"15min":1.677},"last":0.81899999999999995}]},{"osd":1,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.123,"15min":1.2030000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.575,"5min":2.073,"15min":2.617},"last":1.466},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5269999999999999,"5min":1.657,"15min":1.657},"last":1.2370000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.216,"15min":1.1619999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5209999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.1479999999999999,"15min":1.1599999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.403,"5min":1.667,"15min":1.667},"last":1.4039999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.143,"15min":1.157},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.8660000000000001},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2609999999999999,"15min":1.2889999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.591,"5min":2.1869999999999998,"15min":2.5630000000000002},"last":1.6080000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.171,"15min":1.2290000000000001},"min":{"1min":0.60399999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.4930000000000001,"5min":1.7889999999999999,"15min":2.0049999999999999},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.181,"15min":1.2190000000000001},"min":{"1min":0.84699999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.411,"5min":1.994,"15min":1.994},"last":1.1799999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.2669999999999999,"15min":1.321},"min":{"1min":0.86499999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.776,"5min":1.913,"15min":2.6640000000000001},"last":1.661},{"interface":"front","average":{"1min":1.337,"5min":1.3240000000000001,"15min":1.3899999999999999},"min":{"1min":0.95299999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.679,"5min":1.871,"15min":2.173},"last":1.571}]},{"osd":7,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2889999999999999,"15min":1.355},"min":{"1min":0.96599999999999997,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.508,"5min":2.097,"15min":2.6030000000000002},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.371,"15min":1.4239999999999999},"min":{"1min":0.94599999999999995,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.859,"5min":2.2719999999999998,"15min":2.7669999999999999},"last":1.369}]}]},{"osd":3,"up_from":14,"seq":60129542226,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298148,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073692,"statfs":{"total":96636764160,"available":96331460608,"internally_reserved":0,"allocated":1077248,"data_stored":494340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83699999999999997,"15min":0.78000000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.879,"15min":0.84099999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.1439999999999999}]},{"osd":1,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.875,"15min":0.89300000000000002},"min":{"1min":0.44500000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.0860000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.0029999999999999,"15min":0.97699999999999998},"min":{"1min":0.59499999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.2090000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0229999999999999,"15min":0.98299999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.53,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.093,"15min":1.0609999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.431,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.446}]},{"osd":4,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.228,"15min":1.194},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.484,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.53},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2,"15min":1.22},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.5409999999999999,"5min":1.659,"15min":1.659},"last":1.5840000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2589999999999999,"15min":1.2629999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.548,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":2.0459999999999998},{"interface":"front","average":{"1min":1.244,"5min":1.266,"15min":1.264},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.7390000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3380000000000001,"15min":1.3140000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.6599999999999999,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.379,"15min":1.359},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.0259999999999998,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":2.2669999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3340000000000001,"15min":1.3700000000000001},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7929999999999999,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.9219999999999999},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.3500000000000001,"15min":1.3520000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.587,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.8069999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542226,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":1073152,"data_stored":493071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.78100000000000003,"15min":0.76900000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.266,"5min":1.409,"15min":1.409},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.83599999999999997,"15min":0.86499999999999999},"min":{"1min":0.47399999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.137,"5min":1.619,"15min":1.619},"last":0.63400000000000001}]},{"osd":1,"last update":"Wed Nov 15 03:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.93000000000000005,"15min":0.90400000000000003},"min":{"1min":0.60299999999999998,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2809999999999999,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.046},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.95999999999999996,"15min":0.94799999999999995},"min":{"1min":0.70199999999999996,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.3720000000000001,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.78700000000000003}]},{"osd":3,"last update":"Wed Nov 15 03:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.96599999999999997,"15min":0.98799999999999999},"min":{"1min":0.56999999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2609999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.032,"5min":1.121,"15min":1.119},"min":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.622},"max":{"1min":1.2130000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.1160000000000001}]},{"osd":4,"last update":"Wed Nov 15 03:02:51 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.2150000000000001,"15min":1.212},"min":{"1min":0.88300000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.673,"5min":1.673,"15min":1.9410000000000001},"last":1.181},{"interface":"front","average":{"1min":1.264,"5min":1.234,"15min":1.228},"min":{"1min":1.024,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7829999999999999},"last":0.89500000000000002}]},{"osd":5,"last update":"Wed Nov 15 03:02:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1819999999999999,"15min":1.238},"min":{"1min":0.94699999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.351,"5min":1.613,"15min":1.8939999999999999},"last":1.073},{"interface":"front","average":{"1min":1.286,"5min":1.274,"15min":1.2889999999999999},"min":{"1min":1.056,"5min":0.92600000000000005,"15min":0.90400000000000003},"max":{"1min":1.5780000000000001,"5min":1.702,"15min":1.857},"last":1.2809999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:02:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.26,"15min":1.3380000000000001},"min":{"1min":1.0109999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.597,"5min":1.72,"15min":1.9379999999999999},"last":1.024},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.252,"15min":1.3220000000000001},"min":{"1min":1.008,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.486,"5min":1.573,"15min":2.0870000000000002},"last":0.88200000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:02:51 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3240000000000001,"15min":1.3680000000000001},"min":{"1min":0.96199999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.6519999999999999,"5min":1.821,"15min":1.9570000000000001},"last":1.147},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.2669999999999999,"15min":1.355},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.4430000000000001,"5min":1.635,"15min":2.0350000000000001},"last":0.95999999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542226,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298148,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073692,"statfs":{"total":96636764160,"available":96331460608,"internally_reserved":0,"allocated":1077248,"data_stored":494340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.83099999999999996,"15min":0.86299999999999999},"min":{"1min":0.49099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.98999999999999999,"5min":4.3630000000000004,"15min":4.3630000000000004},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.98799999999999999,"15min":0.92600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.2450000000000001,"5min":6.157,"15min":6.157},"last":0.88900000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.996,"15min":1.014},"min":{"1min":0.63,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3440000000000001,"5min":4.1699999999999999,"15min":4.1699999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.038,"5min":1.056,"15min":1.0089999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.482,"5min":6.8369999999999997,"15min":6.8369999999999997},"last":1.038}]},{"osd":3,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.99099999999999999,"15min":0.97099999999999997},"min":{"1min":0.58099999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7050000000000001,"5min":3.5070000000000001,"15min":3.5070000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1279999999999999,"15min":1.165},"min":{"1min":0.746,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.4950000000000001,"5min":5.2149999999999999,"15min":5.2149999999999999},"last":1.306}]},{"osd":4,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3160000000000001,"15min":1.27},"min":{"1min":0.95599999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.671,"5min":6.2000000000000002,"15min":6.2000000000000002},"last":1.339},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.22,"15min":1.1899999999999999},"min":{"1min":0.81699999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.357,"5min":3.577,"15min":3.577},"last":1.1120000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.286,"15min":1.2410000000000001},"min":{"1min":0.95799999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.8839999999999999,"5min":3.548,"15min":3.548},"last":1.071},{"interface":"front","average":{"1min":1.276,"5min":1.304,"15min":1.3460000000000001},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.675,"5min":4.4080000000000004,"15min":4.4080000000000004},"last":1.365}]},{"osd":6,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.3400000000000001,"15min":1.4059999999999999},"min":{"1min":0.97099999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.661,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.361,"15min":1.3520000000000001},"min":{"1min":0.82999999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.585,"5min":6.9690000000000003,"15min":6.9690000000000003},"last":1.127}]},{"osd":7,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.506,"15min":1.4259999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.738,"5min":18.253,"15min":18.253},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.4259999999999999,"15min":1.3149999999999999},"min":{"1min":0.96399999999999997,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.792,"5min":11.006,"15min":11.006},"last":1.3240000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298148,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073692,"statfs":{"total":96636764160,"available":96331460608,"internally_reserved":0,"allocated":1077248,"data_stored":494340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.099,"15min":1.0620000000000001},"min":{"1min":0.65600000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4319999999999999,"5min":1.5640000000000001,"15min":2.3370000000000002},"last":0.62},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.05,"15min":1.1819999999999999},"min":{"1min":0.78400000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.369,"5min":1.4630000000000001,"15min":3.46},"last":1.0700000000000001}]},{"osd":1,"last update":"Wed Nov 15 03:02:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0900000000000001,"15min":1.212},"min":{"1min":0.80600000000000005,"5min":0.69399999999999995,"15min":0.626},"max":{"1min":1.478,"5min":1.478,"15min":3.6349999999999998},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.135,"15min":1.175},"min":{"1min":0.89900000000000002,"5min":0.80300000000000005,"15min":0.65000000000000002},"max":{"1min":1.409,"5min":1.734,"15min":3.516},"last":0.97299999999999998}]},{"osd":2,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.159,"15min":1.1970000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.73399999999999999},"max":{"1min":1.5489999999999999,"5min":1.7370000000000001,"15min":5.4180000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.123,"5min":1.177,"15min":1.2769999999999999},"min":{"1min":0.72699999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.454,"5min":1.7789999999999999,"15min":3.5859999999999999},"last":0.73399999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1599999999999999,"15min":1.264},"min":{"1min":0.85799999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.417,"5min":1.5049999999999999,"15min":2.141},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.298,"15min":1.4139999999999999},"min":{"1min":1.083,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.5620000000000001,"5min":1.786,"15min":4.2729999999999997},"last":0.84499999999999997}]},{"osd":4,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.244,"15min":1.3460000000000001},"min":{"1min":0.998,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.419,"5min":1.825,"15min":3.706},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.296,"5min":1.2649999999999999,"15min":1.5269999999999999},"min":{"1min":1.079,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.4910000000000001,"5min":1.7250000000000001,"15min":6.1180000000000003},"last":1.198}]},{"osd":6,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2430000000000001,"15min":1.3140000000000001},"min":{"1min":0.91400000000000003,"5min":0.749,"15min":0.749},"max":{"1min":1.593,"5min":1.794,"15min":4.2309999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.3109999999999999,"15min":1.508},"min":{"1min":1.1100000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.5209999999999999,"5min":1.6910000000000001,"15min":3.641},"last":1.0489999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.286,"15min":1.3899999999999999},"min":{"1min":1.0780000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.55,"5min":1.8380000000000001,"15min":3.516},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.379,"15min":1.4379999999999999},"min":{"1min":1.0309999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.623,"5min":1.9339999999999999,"15min":3.754},"last":1.002}]}]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298184,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073656,"statfs":{"total":96636764160,"available":96331423744,"internally_reserved":0,"allocated":1114112,"data_stored":514419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:02:56 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.81100000000000005,"15min":0.83299999999999996},"min":{"1min":0.65200000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.94699999999999995,"5min":1.2150000000000001,"15min":1.2370000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.90600000000000003,"15min":0.88600000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.123,"5min":1.399,"15min":1.419},"last":0.72299999999999998}]},{"osd":2,"last update":"Wed Nov 15 03:02:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93300000000000005,"15min":0.873},"min":{"1min":0.71799999999999997,"5min":0.495,"15min":0.44600000000000001},"max":{"1min":1.2070000000000001,"5min":1.472,"15min":1.472},"last":1.054},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.94799999999999995,"15min":0.92400000000000004},"min":{"1min":0.629,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3740000000000001,"5min":1.887,"15min":1.887},"last":0.98999999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:02:56 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.161,"15min":1.087},"min":{"1min":0.77000000000000002,"5min":0.53300000000000003,"15min":0.503},"max":{"1min":1.5680000000000001,"5min":1.597,"15min":1.597},"last":1.24},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.081,"15min":1.0069999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.46000000000000002},"max":{"1min":1.3799999999999999,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.212}]},{"osd":4,"last update":"Wed Nov 15 03:02:56 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2070000000000001,"15min":1.1759999999999999},"min":{"1min":1.034,"5min":0.75700000000000001,"15min":0.65100000000000002},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.224,"15min":1.2509999999999999},"min":{"1min":1.0209999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.4630000000000001,"5min":1.5760000000000001,"15min":2.206},"last":1.3839999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:02:56 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2230000000000001,"15min":1.2529999999999999},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.746},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.899},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2589999999999999,"15min":1.2829999999999999},"min":{"1min":1.1000000000000001,"5min":0.92800000000000005,"15min":0.871},"max":{"1min":1.4850000000000001,"5min":1.764,"15min":2.4609999999999999},"last":1.1499999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:02:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3540000000000001,"15min":1.335},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.641,"5min":1.948,"15min":1.948},"last":1.609},{"interface":"front","average":{"1min":1.399,"5min":1.3979999999999999,"15min":1.4079999999999999},"min":{"1min":1.204,"5min":1.054,"15min":0.87},"max":{"1min":1.8100000000000001,"5min":1.9019999999999999,"15min":2.3290000000000002},"last":1.6000000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:02:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.3400000000000001,"15min":1.3240000000000001},"min":{"1min":0.95799999999999996,"5min":0.93999999999999995,"15min":0.84199999999999997},"max":{"1min":1.708,"5min":2.161,"15min":2.161},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.361,"15min":1.3620000000000001},"min":{"1min":1.1659999999999999,"5min":0.95499999999999996,"15min":0.78600000000000003},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":2.2349999999999999},"last":1.4099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T03:03:26.106 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T03:03:26.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.269+0000 7f3087c9b700 1 Processor -- start 2023-11-15T03:03:26.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.269+0000 7f3087c9b700 1 -- start start 2023-11-15T03:03:26.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.269+0000 7f3087c9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3080154ca0 0x7f3080155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:26.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.269+0000 7f3087c9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30800a57c0 0x7f30800a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:26.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.269+0000 7f3087c9b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30800c4c30 con 0x7f30801562f0 2023-11-15T03:03:26.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.269+0000 7f3087c9b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30800a8280 con 0x7f3080154ca0 2023-11-15T03:03:26.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.269+0000 7f3087c9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30800a6d40 con 0x7f30800a57c0 2023-11-15T03:03:26.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.269+0000 7f3085236700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f30801562f0 0x7f30800a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45314/0 (socket says 172.21.15.121:45314) 2023-11-15T03:03:26.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.269+0000 7f3085236700 1 -- 172.21.15.121:0/1521214198 learned_addr learned my addr 172.21.15.121:0/1521214198 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:26.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.269+0000 7f3086238700 1 --2- 172.21.15.121:0/1521214198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30800a57c0 0x7f30800a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:26.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.269+0000 7f3085a37700 1 --2- 172.21.15.121:0/1521214198 >> v2:172.21.15.121:3300/0 conn(0x7f3080154ca0 0x7f3080155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:26.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.273+0000 7f3086238700 1 -- 172.21.15.121:0/1521214198 >> v2:172.21.15.121:3300/0 conn(0x7f3080154ca0 msgr2=0x7f3080155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:26.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.273+0000 7f3084a35700 1 -- 172.21.15.121:0/1521214198 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1173869286 0 0) 0x7f30800c4c30 con 0x7f30801562f0 2023-11-15T03:03:26.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.273+0000 7f3086238700 1 --2- 172.21.15.121:0/1521214198 >> v2:172.21.15.121:3300/0 conn(0x7f3080154ca0 0x7f3080155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:26.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.273+0000 7f3086238700 1 -- 172.21.15.121:0/1521214198 >> v1:172.21.15.121:6789/0 conn(0x7f30801562f0 legacy=0x7f30800a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:26.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.273+0000 7f3086238700 1 -- 172.21.15.121:0/1521214198 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30800a6140 con 0x7f30800a57c0 2023-11-15T03:03:26.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.273+0000 7f3086238700 1 --2- 172.21.15.121:0/1521214198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30800a57c0 0x7f30800a5c50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f3078009e60 tx=0x7f3078009780).ready entity=mon.2 client_cookie=b79571d88a777b95 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:26.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.273+0000 7f3084a35700 1 -- 172.21.15.121:0/1521214198 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307800e070 con 0x7f30800a57c0 2023-11-15T03:03:26.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.273+0000 7f3084a35700 1 -- 172.21.15.121:0/1521214198 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307800bb60 con 0x7f30800a57c0 2023-11-15T03:03:26.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.273+0000 7f3084a35700 1 -- 172.21.15.121:0/1521214198 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078012410 con 0x7f30800a57c0 2023-11-15T03:03:26.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.273+0000 7f3087c9b700 1 -- 172.21.15.121:0/1521214198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30800a57c0 msgr2=0x7f30800a5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:26.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.273+0000 7f3087c9b700 1 --2- 172.21.15.121:0/1521214198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30800a57c0 0x7f30800a5c50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f3078009e60 tx=0x7f3078009780).stop 2023-11-15T03:03:26.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.273+0000 7f3087c9b700 1 -- 172.21.15.121:0/1521214198 shutdown_connections 2023-11-15T03:03:26.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.273+0000 7f3087c9b700 1 --2- 172.21.15.121:0/1521214198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30800a57c0 0x7f30800a5c50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:26.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.273+0000 7f3087c9b700 1 --2- 172.21.15.121:0/1521214198 >> v2:172.21.15.121:3300/0 conn(0x7f3080154ca0 0x7f3080155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:26.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.273+0000 7f3087c9b700 1 -- 172.21.15.121:0/1521214198 >> 172.21.15.121:0/1521214198 conn(0x7f3080062190 msgr2=0x7f30800625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:26.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.273+0000 7f3087c9b700 1 -- 172.21.15.121:0/1521214198 shutdown_connections 2023-11-15T03:03:26.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.273+0000 7f3087c9b700 1 -- 172.21.15.121:0/1521214198 wait complete. 2023-11-15T03:03:26.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.277+0000 7f3087c9b700 1 Processor -- start 2023-11-15T03:03:26.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.277+0000 7f3087c9b700 1 -- start start 2023-11-15T03:03:26.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.277+0000 7f3087c9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30800a57c0 0x7f30801482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:26.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.277+0000 7f3087c9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3080154ca0 0x7f3080152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:26.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.277+0000 7f3087c9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30801562f0 0x7f30800c0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:26.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.277+0000 7f3087c9b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30800ab380 con 0x7f30801562f0 2023-11-15T03:03:26.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.277+0000 7f3087c9b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30801487b0 con 0x7f3080154ca0 2023-11-15T03:03:26.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.277+0000 7f3087c9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30801536e0 con 0x7f30800a57c0 2023-11-15T03:03:26.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.277+0000 7f3086238700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30801562f0 0x7f30800c0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:26.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.277+0000 7f3086238700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30801562f0 0x7f30800c0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49698/0 (socket says 172.21.15.121:49698) 2023-11-15T03:03:26.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.277+0000 7f3086238700 1 -- 172.21.15.121:0/3594494945 learned_addr learned my addr 172.21.15.121:0/3594494945 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:26.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.281+0000 7f3085236700 1 --2- 172.21.15.121:0/3594494945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3080154ca0 0x7f3080152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:26.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.281+0000 7f3085a37700 1 --2- 172.21.15.121:0/3594494945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30800a57c0 0x7f30801482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:26.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.281+0000 7f3086238700 1 -- 172.21.15.121:0/3594494945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30800a57c0 msgr2=0x7f30801482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:26.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.281+0000 7f3086238700 1 --2- 172.21.15.121:0/3594494945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30800a57c0 0x7f30801482c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:26.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.281+0000 7f3086238700 1 -- 172.21.15.121:0/3594494945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3080154ca0 msgr2=0x7f3080152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:26.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.281+0000 7f3086238700 1 --2- 172.21.15.121:0/3594494945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3080154ca0 0x7f3080152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:26.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.281+0000 7f3086238700 1 -- 172.21.15.121:0/3594494945 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3078009a20 con 0x7f30801562f0 2023-11-15T03:03:26.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.281+0000 7f3085a37700 1 --2- 172.21.15.121:0/3594494945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30800a57c0 0x7f30801482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:26.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.281+0000 7f3086238700 1 --2- 172.21.15.121:0/3594494945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30801562f0 0x7f30800c0d60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7f307800b6c0 tx=0x7f3078009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:26.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.281+0000 7f307dffb700 1 -- 172.21.15.121:0/3594494945 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307801d630 con 0x7f30801562f0 2023-11-15T03:03:26.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.281+0000 7f307dffb700 1 -- 172.21.15.121:0/3594494945 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307801d790 con 0x7f30801562f0 2023-11-15T03:03:26.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.281+0000 7f3087c9b700 1 -- 172.21.15.121:0/3594494945 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30800aaa50 con 0x7f30801562f0 2023-11-15T03:03:26.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.281+0000 7f307dffb700 1 -- 172.21.15.121:0/3594494945 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307801db50 con 0x7f30801562f0 2023-11-15T03:03:26.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.281+0000 7f3087c9b700 1 -- 172.21.15.121:0/3594494945 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30800ab010 con 0x7f30801562f0 2023-11-15T03:03:26.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.281+0000 7f3087c9b700 1 -- 172.21.15.121:0/3594494945 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308006c150 con 0x7f30801562f0 2023-11-15T03:03:26.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.285+0000 7f307dffb700 1 -- 172.21.15.121:0/3594494945 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3078003040 con 0x7f30801562f0 2023-11-15T03:03:26.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.285+0000 7f307dffb700 1 --2- 172.21.15.121:0/3594494945 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f306c05a4e0 0x7f306c05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:26.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.285+0000 7f307dffb700 1 -- 172.21.15.121:0/3594494945 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f30780766b0 con 0x7f30801562f0 2023-11-15T03:03:26.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.285+0000 7f3085a37700 1 --2- 172.21.15.121:0/3594494945 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f306c05a4e0 0x7f306c05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:26.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.285+0000 7f3085a37700 1 --2- 172.21.15.121:0/3594494945 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f306c05a4e0 0x7f306c05c9a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f30800ac940 tx=0x7f3070008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:26.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.293+0000 7f307dffb700 1 -- 172.21.15.121:0/3594494945 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3078049dd0 con 0x7f30801562f0 2023-11-15T03:03:26.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.457+0000 7f3087c9b700 1 -- 172.21.15.121:0/3594494945 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3080154580 con 0x7f306c05a4e0 2023-11-15T03:03:26.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.473+0000 7f307dffb700 1 -- 172.21.15.121:0/3594494945 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+358510 (secure 0 0 0) 0x7f3080154580 con 0x7f306c05a4e0 2023-11-15T03:03:26.484 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:26.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.481+0000 7f3087c9b700 1 -- 172.21.15.121:0/3594494945 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f306c05a4e0 msgr2=0x7f306c05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:26.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.481+0000 7f3087c9b700 1 --2- 172.21.15.121:0/3594494945 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f306c05a4e0 0x7f306c05c9a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f30800ac940 tx=0x7f3070008040).stop 2023-11-15T03:03:26.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.481+0000 7f3087c9b700 1 -- 172.21.15.121:0/3594494945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30801562f0 msgr2=0x7f30800c0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:26.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.481+0000 7f3087c9b700 1 --2- 172.21.15.121:0/3594494945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30801562f0 0x7f30800c0d60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7f307800b6c0 tx=0x7f3078009780).stop 2023-11-15T03:03:26.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.481+0000 7f3087c9b700 1 -- 172.21.15.121:0/3594494945 shutdown_connections 2023-11-15T03:03:26.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.481+0000 7f3087c9b700 1 --2- 172.21.15.121:0/3594494945 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f306c05a4e0 0x7f306c05c9a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:26.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.481+0000 7f3087c9b700 1 --2- 172.21.15.121:0/3594494945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30801562f0 0x7f30800c0d60 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:26.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.481+0000 7f3087c9b700 1 --2- 172.21.15.121:0/3594494945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3080154ca0 0x7f3080152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:26.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.481+0000 7f3087c9b700 1 --2- 172.21.15.121:0/3594494945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30800a57c0 0x7f30801482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:26.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.481+0000 7f3087c9b700 1 -- 172.21.15.121:0/3594494945 >> 172.21.15.121:0/3594494945 conn(0x7f3080062190 msgr2=0x7f30800c30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:26.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.481+0000 7f3087c9b700 1 -- 172.21.15.121:0/3594494945 shutdown_connections 2023-11-15T03:03:26.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.481+0000 7f3087c9b700 1 -- 172.21.15.121:0/3594494945 wait complete. 2023-11-15T03:03:26.500 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T03:03:26.508 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2023-11-15T03:03:26.064282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":253,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385616,"kb_used_data":8848,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752589104,"statfs":{"total":773094113280,"available":770651242496,"internally_reserved":0,"allocated":9060352,"data_stored":4165623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19632,"num_objects":-8,"num_object_clones":0,"num_object_copies":-16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8,"num_whiteouts":0,"num_read":-23,"num_read_kb":-25,"num_write":-46,"num_write_kb":-35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.176306"},"pg_stats":[{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.830211+0000","last_change":"2023-11-15T03:03:21.819554+0000","last_active":"2023-11-15T03:03:21.830211+0000","last_peered":"2023-11-15T03:03:21.830211+0000","last_clean":"2023-11-15T03:03:21.830211+0000","last_became_active":"2023-11-15T03:03:21.818891+0000","last_became_peered":"2023-11-15T03:03:21.818891+0000","last_unstale":"2023-11-15T03:03:21.830211+0000","last_undegraded":"2023-11-15T03:03:21.830211+0000","last_fullsized":"2023-11-15T03:03:21.830211+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3f","version":"91'3","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.836797+0000","last_change":"2023-11-15T03:03:21.350048+0000","last_active":"2023-11-15T03:03:21.836797+0000","last_peered":"2023-11-15T03:03:21.836797+0000","last_clean":"2023-11-15T03:03:21.836797+0000","last_became_active":"2023-11-15T03:03:21.349782+0000","last_became_peered":"2023-11-15T03:03:21.349782+0000","last_unstale":"2023-11-15T03:03:21.836797+0000","last_undegraded":"2023-11-15T03:03:21.836797+0000","last_fullsized":"2023-11-15T03:03:21.836797+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.834025+0000","last_change":"2023-11-15T03:03:21.830691+0000","last_active":"2023-11-15T03:03:21.834025+0000","last_peered":"2023-11-15T03:03:21.834025+0000","last_clean":"2023-11-15T03:03:21.834025+0000","last_became_active":"2023-11-15T03:03:21.830431+0000","last_became_peered":"2023-11-15T03:03:21.830431+0000","last_unstale":"2023-11-15T03:03:21.834025+0000","last_undegraded":"2023-11-15T03:03:21.834025+0000","last_fullsized":"2023-11-15T03:03:21.834025+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.818317+0000","last_change":"2023-11-15T03:03:21.351823+0000","last_active":"2023-11-15T03:03:21.818317+0000","last_peered":"2023-11-15T03:03:21.818317+0000","last_clean":"2023-11-15T03:03:21.818317+0000","last_became_active":"2023-11-15T03:03:21.351514+0000","last_became_peered":"2023-11-15T03:03:21.351514+0000","last_unstale":"2023-11-15T03:03:21.818317+0000","last_undegraded":"2023-11-15T03:03:21.818317+0000","last_fullsized":"2023-11-15T03:03:21.818317+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.827160+0000","last_change":"2023-11-15T03:03:21.822725+0000","last_active":"2023-11-15T03:03:21.827160+0000","last_peered":"2023-11-15T03:03:21.827160+0000","last_clean":"2023-11-15T03:03:21.827160+0000","last_became_active":"2023-11-15T03:03:21.822417+0000","last_became_peered":"2023-11-15T03:03:21.822417+0000","last_unstale":"2023-11-15T03:03:21.827160+0000","last_undegraded":"2023-11-15T03:03:21.827160+0000","last_fullsized":"2023-11-15T03:03:21.827160+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.801755+0000","last_change":"2023-11-15T03:03:21.362490+0000","last_active":"2023-11-15T03:03:21.801755+0000","last_peered":"2023-11-15T03:03:21.801755+0000","last_clean":"2023-11-15T03:03:21.801755+0000","last_became_active":"2023-11-15T03:03:21.362168+0000","last_became_peered":"2023-11-15T03:03:21.362168+0000","last_unstale":"2023-11-15T03:03:21.801755+0000","last_undegraded":"2023-11-15T03:03:21.801755+0000","last_fullsized":"2023-11-15T03:03:21.801755+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.834075+0000","last_change":"2023-11-15T03:03:21.830268+0000","last_active":"2023-11-15T03:03:21.834075+0000","last_peered":"2023-11-15T03:03:21.834075+0000","last_clean":"2023-11-15T03:03:21.834075+0000","last_became_active":"2023-11-15T03:03:21.829990+0000","last_became_peered":"2023-11-15T03:03:21.829990+0000","last_unstale":"2023-11-15T03:03:21.834075+0000","last_undegraded":"2023-11-15T03:03:21.834075+0000","last_fullsized":"2023-11-15T03:03:21.834075+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3c","version":"91'2","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.815602+0000","last_change":"2023-11-15T03:03:21.360705+0000","last_active":"2023-11-15T03:03:21.815602+0000","last_peered":"2023-11-15T03:03:21.815602+0000","last_clean":"2023-11-15T03:03:21.815602+0000","last_became_active":"2023-11-15T03:03:21.359539+0000","last_became_peered":"2023-11-15T03:03:21.359539+0000","last_unstale":"2023-11-15T03:03:21.815602+0000","last_undegraded":"2023-11-15T03:03:21.815602+0000","last_fullsized":"2023-11-15T03:03:21.815602+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.824956+0000","last_change":"2023-11-15T03:03:21.814549+0000","last_active":"2023-11-15T03:03:21.824956+0000","last_peered":"2023-11-15T03:03:21.824956+0000","last_clean":"2023-11-15T03:03:21.824956+0000","last_became_active":"2023-11-15T03:03:21.814150+0000","last_became_peered":"2023-11-15T03:03:21.814150+0000","last_unstale":"2023-11-15T03:03:21.824956+0000","last_undegraded":"2023-11-15T03:03:21.824956+0000","last_fullsized":"2023-11-15T03:03:21.824956+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3b","version":"91'1","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.837211+0000","last_change":"2023-11-15T03:03:21.361365+0000","last_active":"2023-11-15T03:03:21.837211+0000","last_peered":"2023-11-15T03:03:21.837211+0000","last_clean":"2023-11-15T03:03:21.837211+0000","last_became_active":"2023-11-15T03:03:21.361054+0000","last_became_peered":"2023-11-15T03:03:21.361054+0000","last_unstale":"2023-11-15T03:03:21.837211+0000","last_undegraded":"2023-11-15T03:03:21.837211+0000","last_fullsized":"2023-11-15T03:03:21.837211+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.826234+0000","last_change":"2023-11-15T03:03:21.822855+0000","last_active":"2023-11-15T03:03:21.826234+0000","last_peered":"2023-11-15T03:03:21.826234+0000","last_clean":"2023-11-15T03:03:21.826234+0000","last_became_active":"2023-11-15T03:03:21.822580+0000","last_became_peered":"2023-11-15T03:03:21.822580+0000","last_unstale":"2023-11-15T03:03:21.826234+0000","last_undegraded":"2023-11-15T03:03:21.826234+0000","last_fullsized":"2023-11-15T03:03:21.826234+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3a","version":"91'1","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.818200+0000","last_change":"2023-11-15T03:03:21.350568+0000","last_active":"2023-11-15T03:03:21.818200+0000","last_peered":"2023-11-15T03:03:21.818200+0000","last_clean":"2023-11-15T03:03:21.818200+0000","last_became_active":"2023-11-15T03:03:21.350294+0000","last_became_peered":"2023-11-15T03:03:21.350294+0000","last_unstale":"2023-11-15T03:03:21.818200+0000","last_undegraded":"2023-11-15T03:03:21.818200+0000","last_fullsized":"2023-11-15T03:03:21.818200+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.833995+0000","last_change":"2023-11-15T03:03:21.828339+0000","last_active":"2023-11-15T03:03:21.833995+0000","last_peered":"2023-11-15T03:03:21.833995+0000","last_clean":"2023-11-15T03:03:21.833995+0000","last_became_active":"2023-11-15T03:03:21.827621+0000","last_became_peered":"2023-11-15T03:03:21.827621+0000","last_unstale":"2023-11-15T03:03:21.833995+0000","last_undegraded":"2023-11-15T03:03:21.833995+0000","last_fullsized":"2023-11-15T03:03:21.833995+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.796864+0000","last_change":"2023-11-15T03:03:21.356811+0000","last_active":"2023-11-15T03:03:21.796864+0000","last_peered":"2023-11-15T03:03:21.796864+0000","last_clean":"2023-11-15T03:03:21.796864+0000","last_became_active":"2023-11-15T03:03:21.356441+0000","last_became_peered":"2023-11-15T03:03:21.356441+0000","last_unstale":"2023-11-15T03:03:21.796864+0000","last_undegraded":"2023-11-15T03:03:21.796864+0000","last_fullsized":"2023-11-15T03:03:21.796864+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.839151+0000","last_change":"2023-11-15T03:03:21.829645+0000","last_active":"2023-11-15T03:03:21.839151+0000","last_peered":"2023-11-15T03:03:21.839151+0000","last_clean":"2023-11-15T03:03:21.839151+0000","last_became_active":"2023-11-15T03:03:21.829278+0000","last_became_peered":"2023-11-15T03:03:21.829278+0000","last_unstale":"2023-11-15T03:03:21.839151+0000","last_undegraded":"2023-11-15T03:03:21.839151+0000","last_fullsized":"2023-11-15T03:03:21.839151+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.803034+0000","last_change":"2023-11-15T03:03:21.359676+0000","last_active":"2023-11-15T03:03:21.803034+0000","last_peered":"2023-11-15T03:03:21.803034+0000","last_clean":"2023-11-15T03:03:21.803034+0000","last_became_active":"2023-11-15T03:03:21.359216+0000","last_became_peered":"2023-11-15T03:03:21.359216+0000","last_unstale":"2023-11-15T03:03:21.803034+0000","last_undegraded":"2023-11-15T03:03:21.803034+0000","last_fullsized":"2023-11-15T03:03:21.803034+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.823043+0000","last_change":"2023-11-15T03:03:21.816982+0000","last_active":"2023-11-15T03:03:21.823043+0000","last_peered":"2023-11-15T03:03:21.823043+0000","last_clean":"2023-11-15T03:03:21.823043+0000","last_became_active":"2023-11-15T03:03:21.815684+0000","last_became_peered":"2023-11-15T03:03:21.815684+0000","last_unstale":"2023-11-15T03:03:21.823043+0000","last_undegraded":"2023-11-15T03:03:21.823043+0000","last_fullsized":"2023-11-15T03:03:21.823043+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.804545+0000","last_change":"2023-11-15T03:03:21.354914+0000","last_active":"2023-11-15T03:03:21.804545+0000","last_peered":"2023-11-15T03:03:21.804545+0000","last_clean":"2023-11-15T03:03:21.804545+0000","last_became_active":"2023-11-15T03:03:21.354634+0000","last_became_peered":"2023-11-15T03:03:21.354634+0000","last_unstale":"2023-11-15T03:03:21.804545+0000","last_undegraded":"2023-11-15T03:03:21.804545+0000","last_fullsized":"2023-11-15T03:03:21.804545+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.839180+0000","last_change":"2023-11-15T03:03:21.830702+0000","last_active":"2023-11-15T03:03:21.839180+0000","last_peered":"2023-11-15T03:03:21.839180+0000","last_clean":"2023-11-15T03:03:21.839180+0000","last_became_active":"2023-11-15T03:03:21.830331+0000","last_became_peered":"2023-11-15T03:03:21.830331+0000","last_unstale":"2023-11-15T03:03:21.839180+0000","last_undegraded":"2023-11-15T03:03:21.839180+0000","last_fullsized":"2023-11-15T03:03:21.839180+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.810396+0000","last_change":"2023-11-15T03:03:21.351902+0000","last_active":"2023-11-15T03:03:21.810396+0000","last_peered":"2023-11-15T03:03:21.810396+0000","last_clean":"2023-11-15T03:03:21.810396+0000","last_became_active":"2023-11-15T03:03:21.351599+0000","last_became_peered":"2023-11-15T03:03:21.351599+0000","last_unstale":"2023-11-15T03:03:21.810396+0000","last_undegraded":"2023-11-15T03:03:21.810396+0000","last_fullsized":"2023-11-15T03:03:21.810396+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.832086+0000","last_change":"2023-11-15T03:03:21.827040+0000","last_active":"2023-11-15T03:03:21.832086+0000","last_peered":"2023-11-15T03:03:21.832086+0000","last_clean":"2023-11-15T03:03:21.832086+0000","last_became_active":"2023-11-15T03:03:21.823490+0000","last_became_peered":"2023-11-15T03:03:21.823490+0000","last_unstale":"2023-11-15T03:03:21.832086+0000","last_undegraded":"2023-11-15T03:03:21.832086+0000","last_fullsized":"2023-11-15T03:03:21.832086+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.816449+0000","last_change":"2023-11-15T03:03:21.351141+0000","last_active":"2023-11-15T03:03:21.816449+0000","last_peered":"2023-11-15T03:03:21.816449+0000","last_clean":"2023-11-15T03:03:21.816449+0000","last_became_active":"2023-11-15T03:03:21.350890+0000","last_became_peered":"2023-11-15T03:03:21.350890+0000","last_unstale":"2023-11-15T03:03:21.816449+0000","last_undegraded":"2023-11-15T03:03:21.816449+0000","last_fullsized":"2023-11-15T03:03:21.816449+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.839110+0000","last_change":"2023-11-15T03:03:21.829579+0000","last_active":"2023-11-15T03:03:21.839110+0000","last_peered":"2023-11-15T03:03:21.839110+0000","last_clean":"2023-11-15T03:03:21.839110+0000","last_became_active":"2023-11-15T03:03:21.829190+0000","last_became_peered":"2023-11-15T03:03:21.829190+0000","last_unstale":"2023-11-15T03:03:21.839110+0000","last_undegraded":"2023-11-15T03:03:21.839110+0000","last_fullsized":"2023-11-15T03:03:21.839110+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.34","version":"91'4","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.843678+0000","last_change":"2023-11-15T03:03:21.351748+0000","last_active":"2023-11-15T03:03:21.843678+0000","last_peered":"2023-11-15T03:03:21.843678+0000","last_clean":"2023-11-15T03:03:21.843678+0000","last_became_active":"2023-11-15T03:03:21.351229+0000","last_became_peered":"2023-11-15T03:03:21.351229+0000","last_unstale":"2023-11-15T03:03:21.843678+0000","last_undegraded":"2023-11-15T03:03:21.843678+0000","last_fullsized":"2023-11-15T03:03:21.843678+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.831402+0000","last_change":"2023-11-15T03:03:21.822904+0000","last_active":"2023-11-15T03:03:21.831402+0000","last_peered":"2023-11-15T03:03:21.831402+0000","last_clean":"2023-11-15T03:03:21.831402+0000","last_became_active":"2023-11-15T03:03:21.822416+0000","last_became_peered":"2023-11-15T03:03:21.822416+0000","last_unstale":"2023-11-15T03:03:21.831402+0000","last_undegraded":"2023-11-15T03:03:21.831402+0000","last_fullsized":"2023-11-15T03:03:21.831402+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"91'1","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.836916+0000","last_change":"2023-11-15T03:03:21.353943+0000","last_active":"2023-11-15T03:03:21.836916+0000","last_peered":"2023-11-15T03:03:21.836916+0000","last_clean":"2023-11-15T03:03:21.836916+0000","last_became_active":"2023-11-15T03:03:21.353612+0000","last_became_peered":"2023-11-15T03:03:21.353612+0000","last_unstale":"2023-11-15T03:03:21.836916+0000","last_undegraded":"2023-11-15T03:03:21.836916+0000","last_fullsized":"2023-11-15T03:03:21.836916+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.830192+0000","last_change":"2023-11-15T03:03:21.819428+0000","last_active":"2023-11-15T03:03:21.830192+0000","last_peered":"2023-11-15T03:03:21.830192+0000","last_clean":"2023-11-15T03:03:21.830192+0000","last_became_active":"2023-11-15T03:03:21.819132+0000","last_became_peered":"2023-11-15T03:03:21.819132+0000","last_unstale":"2023-11-15T03:03:21.830192+0000","last_undegraded":"2023-11-15T03:03:21.830192+0000","last_fullsized":"2023-11-15T03:03:21.830192+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.797359+0000","last_change":"2023-11-15T03:03:21.361407+0000","last_active":"2023-11-15T03:03:21.797359+0000","last_peered":"2023-11-15T03:03:21.797359+0000","last_clean":"2023-11-15T03:03:21.797359+0000","last_became_active":"2023-11-15T03:03:21.361132+0000","last_became_peered":"2023-11-15T03:03:21.361132+0000","last_unstale":"2023-11-15T03:03:21.797359+0000","last_undegraded":"2023-11-15T03:03:21.797359+0000","last_fullsized":"2023-11-15T03:03:21.797359+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.839715+0000","last_change":"2023-11-15T03:03:21.837832+0000","last_active":"2023-11-15T03:03:21.839715+0000","last_peered":"2023-11-15T03:03:21.839715+0000","last_clean":"2023-11-15T03:03:21.839715+0000","last_became_active":"2023-11-15T03:03:21.837509+0000","last_became_peered":"2023-11-15T03:03:21.837509+0000","last_unstale":"2023-11-15T03:03:21.839715+0000","last_undegraded":"2023-11-15T03:03:21.839715+0000","last_fullsized":"2023-11-15T03:03:21.839715+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.809153+0000","last_change":"2023-11-15T03:03:21.350809+0000","last_active":"2023-11-15T03:03:21.809153+0000","last_peered":"2023-11-15T03:03:21.809153+0000","last_clean":"2023-11-15T03:03:21.809153+0000","last_became_active":"2023-11-15T03:03:21.350547+0000","last_became_peered":"2023-11-15T03:03:21.350547+0000","last_unstale":"2023-11-15T03:03:21.809153+0000","last_undegraded":"2023-11-15T03:03:21.809153+0000","last_fullsized":"2023-11-15T03:03:21.809153+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.830405+0000","last_change":"2023-11-15T03:03:21.820480+0000","last_active":"2023-11-15T03:03:21.830405+0000","last_peered":"2023-11-15T03:03:21.830405+0000","last_clean":"2023-11-15T03:03:21.830405+0000","last_became_active":"2023-11-15T03:03:21.820118+0000","last_became_peered":"2023-11-15T03:03:21.820118+0000","last_unstale":"2023-11-15T03:03:21.830405+0000","last_undegraded":"2023-11-15T03:03:21.830405+0000","last_fullsized":"2023-11-15T03:03:21.830405+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.30","version":"91'4","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.836129+0000","last_change":"2023-11-15T03:03:21.355816+0000","last_active":"2023-11-15T03:03:21.836129+0000","last_peered":"2023-11-15T03:03:21.836129+0000","last_clean":"2023-11-15T03:03:21.836129+0000","last_became_active":"2023-11-15T03:03:21.355506+0000","last_became_peered":"2023-11-15T03:03:21.355506+0000","last_unstale":"2023-11-15T03:03:21.836129+0000","last_undegraded":"2023-11-15T03:03:21.836129+0000","last_fullsized":"2023-11-15T03:03:21.836129+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.830594+0000","last_change":"2023-11-15T03:03:21.823107+0000","last_active":"2023-11-15T03:03:21.830594+0000","last_peered":"2023-11-15T03:03:21.830594+0000","last_clean":"2023-11-15T03:03:21.830594+0000","last_became_active":"2023-11-15T03:03:21.820441+0000","last_became_peered":"2023-11-15T03:03:21.820441+0000","last_unstale":"2023-11-15T03:03:21.830594+0000","last_undegraded":"2023-11-15T03:03:21.830594+0000","last_fullsized":"2023-11-15T03:03:21.830594+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.823395+0000","last_change":"2023-11-15T03:03:21.357734+0000","last_active":"2023-11-15T03:03:21.823395+0000","last_peered":"2023-11-15T03:03:21.823395+0000","last_clean":"2023-11-15T03:03:21.823395+0000","last_became_active":"2023-11-15T03:03:21.357406+0000","last_became_peered":"2023-11-15T03:03:21.357406+0000","last_unstale":"2023-11-15T03:03:21.823395+0000","last_undegraded":"2023-11-15T03:03:21.823395+0000","last_fullsized":"2023-11-15T03:03:21.823395+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.839388+0000","last_change":"2023-11-15T03:03:21.834951+0000","last_active":"2023-11-15T03:03:21.839388+0000","last_peered":"2023-11-15T03:03:21.839388+0000","last_clean":"2023-11-15T03:03:21.839388+0000","last_became_active":"2023-11-15T03:03:21.834681+0000","last_became_peered":"2023-11-15T03:03:21.834681+0000","last_unstale":"2023-11-15T03:03:21.839388+0000","last_undegraded":"2023-11-15T03:03:21.839388+0000","last_fullsized":"2023-11-15T03:03:21.839388+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.798059+0000","last_change":"2023-11-15T03:03:21.350374+0000","last_active":"2023-11-15T03:03:21.798059+0000","last_peered":"2023-11-15T03:03:21.798059+0000","last_clean":"2023-11-15T03:03:21.798059+0000","last_became_active":"2023-11-15T03:03:21.346256+0000","last_became_peered":"2023-11-15T03:03:21.346256+0000","last_unstale":"2023-11-15T03:03:21.798059+0000","last_undegraded":"2023-11-15T03:03:21.798059+0000","last_fullsized":"2023-11-15T03:03:21.798059+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.824243+0000","last_change":"2023-11-15T03:03:21.818958+0000","last_active":"2023-11-15T03:03:21.824243+0000","last_peered":"2023-11-15T03:03:21.824243+0000","last_clean":"2023-11-15T03:03:21.824243+0000","last_became_active":"2023-11-15T03:03:21.818662+0000","last_became_peered":"2023-11-15T03:03:21.818662+0000","last_unstale":"2023-11-15T03:03:21.824243+0000","last_undegraded":"2023-11-15T03:03:21.824243+0000","last_fullsized":"2023-11-15T03:03:21.824243+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2d","version":"91'1","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.819079+0000","last_change":"2023-11-15T03:03:21.351063+0000","last_active":"2023-11-15T03:03:21.819079+0000","last_peered":"2023-11-15T03:03:21.819079+0000","last_clean":"2023-11-15T03:03:21.819079+0000","last_became_active":"2023-11-15T03:03:21.350657+0000","last_became_peered":"2023-11-15T03:03:21.350657+0000","last_unstale":"2023-11-15T03:03:21.819079+0000","last_undegraded":"2023-11-15T03:03:21.819079+0000","last_fullsized":"2023-11-15T03:03:21.819079+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.829438+0000","last_change":"2023-11-15T03:03:21.821894+0000","last_active":"2023-11-15T03:03:21.829438+0000","last_peered":"2023-11-15T03:03:21.829438+0000","last_clean":"2023-11-15T03:03:21.829438+0000","last_became_active":"2023-11-15T03:03:21.821636+0000","last_became_peered":"2023-11-15T03:03:21.821636+0000","last_unstale":"2023-11-15T03:03:21.829438+0000","last_undegraded":"2023-11-15T03:03:21.829438+0000","last_fullsized":"2023-11-15T03:03:21.829438+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.797582+0000","last_change":"2023-11-15T03:03:21.356871+0000","last_active":"2023-11-15T03:03:21.797582+0000","last_peered":"2023-11-15T03:03:21.797582+0000","last_clean":"2023-11-15T03:03:21.797582+0000","last_became_active":"2023-11-15T03:03:21.356509+0000","last_became_peered":"2023-11-15T03:03:21.356509+0000","last_unstale":"2023-11-15T03:03:21.797582+0000","last_undegraded":"2023-11-15T03:03:21.797582+0000","last_fullsized":"2023-11-15T03:03:21.797582+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.829670+0000","last_change":"2023-11-15T03:03:21.822798+0000","last_active":"2023-11-15T03:03:21.829670+0000","last_peered":"2023-11-15T03:03:21.829670+0000","last_clean":"2023-11-15T03:03:21.829670+0000","last_became_active":"2023-11-15T03:03:21.822530+0000","last_became_peered":"2023-11-15T03:03:21.822530+0000","last_unstale":"2023-11-15T03:03:21.829670+0000","last_undegraded":"2023-11-15T03:03:21.829670+0000","last_fullsized":"2023-11-15T03:03:21.829670+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2b","version":"91'1","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.824473+0000","last_change":"2023-11-15T03:03:21.355709+0000","last_active":"2023-11-15T03:03:21.824473+0000","last_peered":"2023-11-15T03:03:21.824473+0000","last_clean":"2023-11-15T03:03:21.824473+0000","last_became_active":"2023-11-15T03:03:21.355401+0000","last_became_peered":"2023-11-15T03:03:21.355401+0000","last_unstale":"2023-11-15T03:03:21.824473+0000","last_undegraded":"2023-11-15T03:03:21.824473+0000","last_fullsized":"2023-11-15T03:03:21.824473+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.841113+0000","last_change":"2023-11-15T03:03:21.836274+0000","last_active":"2023-11-15T03:03:21.841113+0000","last_peered":"2023-11-15T03:03:21.841113+0000","last_clean":"2023-11-15T03:03:21.841113+0000","last_became_active":"2023-11-15T03:03:21.835544+0000","last_became_peered":"2023-11-15T03:03:21.835544+0000","last_unstale":"2023-11-15T03:03:21.841113+0000","last_undegraded":"2023-11-15T03:03:21.841113+0000","last_fullsized":"2023-11-15T03:03:21.841113+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.817488+0000","last_change":"2023-11-15T03:03:21.351355+0000","last_active":"2023-11-15T03:03:21.817488+0000","last_peered":"2023-11-15T03:03:21.817488+0000","last_clean":"2023-11-15T03:03:21.817488+0000","last_became_active":"2023-11-15T03:03:21.351073+0000","last_became_peered":"2023-11-15T03:03:21.351073+0000","last_unstale":"2023-11-15T03:03:21.817488+0000","last_undegraded":"2023-11-15T03:03:21.817488+0000","last_fullsized":"2023-11-15T03:03:21.817488+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.839691+0000","last_change":"2023-11-15T03:03:21.837783+0000","last_active":"2023-11-15T03:03:21.839691+0000","last_peered":"2023-11-15T03:03:21.839691+0000","last_clean":"2023-11-15T03:03:21.839691+0000","last_became_active":"2023-11-15T03:03:21.837440+0000","last_became_peered":"2023-11-15T03:03:21.837440+0000","last_unstale":"2023-11-15T03:03:21.839691+0000","last_undegraded":"2023-11-15T03:03:21.839691+0000","last_fullsized":"2023-11-15T03:03:21.839691+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.809529+0000","last_change":"2023-11-15T03:03:21.354108+0000","last_active":"2023-11-15T03:03:21.809529+0000","last_peered":"2023-11-15T03:03:21.809529+0000","last_clean":"2023-11-15T03:03:21.809529+0000","last_became_active":"2023-11-15T03:03:21.351217+0000","last_became_peered":"2023-11-15T03:03:21.351217+0000","last_unstale":"2023-11-15T03:03:21.809529+0000","last_undegraded":"2023-11-15T03:03:21.809529+0000","last_fullsized":"2023-11-15T03:03:21.809529+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.839218+0000","last_change":"2023-11-15T03:03:21.831597+0000","last_active":"2023-11-15T03:03:21.839218+0000","last_peered":"2023-11-15T03:03:21.839218+0000","last_clean":"2023-11-15T03:03:21.839218+0000","last_became_active":"2023-11-15T03:03:21.831310+0000","last_became_peered":"2023-11-15T03:03:21.831310+0000","last_unstale":"2023-11-15T03:03:21.839218+0000","last_undegraded":"2023-11-15T03:03:21.839218+0000","last_fullsized":"2023-11-15T03:03:21.839218+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.796945+0000","last_change":"2023-11-15T03:03:21.359017+0000","last_active":"2023-11-15T03:03:21.796945+0000","last_peered":"2023-11-15T03:03:21.796945+0000","last_clean":"2023-11-15T03:03:21.796945+0000","last_became_active":"2023-11-15T03:03:21.358656+0000","last_became_peered":"2023-11-15T03:03:21.358656+0000","last_unstale":"2023-11-15T03:03:21.796945+0000","last_undegraded":"2023-11-15T03:03:21.796945+0000","last_fullsized":"2023-11-15T03:03:21.796945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.836644+0000","last_change":"2023-11-15T03:03:21.832368+0000","last_active":"2023-11-15T03:03:21.836644+0000","last_peered":"2023-11-15T03:03:21.836644+0000","last_clean":"2023-11-15T03:03:21.836644+0000","last_became_active":"2023-11-15T03:03:21.832108+0000","last_became_peered":"2023-11-15T03:03:21.832108+0000","last_unstale":"2023-11-15T03:03:21.836644+0000","last_undegraded":"2023-11-15T03:03:21.836644+0000","last_fullsized":"2023-11-15T03:03:21.836644+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.816562+0000","last_change":"2023-11-15T03:03:21.353299+0000","last_active":"2023-11-15T03:03:21.816562+0000","last_peered":"2023-11-15T03:03:21.816562+0000","last_clean":"2023-11-15T03:03:21.816562+0000","last_became_active":"2023-11-15T03:03:21.353028+0000","last_became_peered":"2023-11-15T03:03:21.353028+0000","last_unstale":"2023-11-15T03:03:21.816562+0000","last_undegraded":"2023-11-15T03:03:21.816562+0000","last_fullsized":"2023-11-15T03:03:21.816562+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.828939+0000","last_change":"2023-11-15T03:03:21.818353+0000","last_active":"2023-11-15T03:03:21.828939+0000","last_peered":"2023-11-15T03:03:21.828939+0000","last_clean":"2023-11-15T03:03:21.828939+0000","last_became_active":"2023-11-15T03:03:21.818050+0000","last_became_peered":"2023-11-15T03:03:21.818050+0000","last_unstale":"2023-11-15T03:03:21.828939+0000","last_undegraded":"2023-11-15T03:03:21.828939+0000","last_fullsized":"2023-11-15T03:03:21.828939+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.802126+0000","last_change":"2023-11-15T03:03:21.363634+0000","last_active":"2023-11-15T03:03:21.802126+0000","last_peered":"2023-11-15T03:03:21.802126+0000","last_clean":"2023-11-15T03:03:21.802126+0000","last_became_active":"2023-11-15T03:03:21.362698+0000","last_became_peered":"2023-11-15T03:03:21.362698+0000","last_unstale":"2023-11-15T03:03:21.802126+0000","last_undegraded":"2023-11-15T03:03:21.802126+0000","last_fullsized":"2023-11-15T03:03:21.802126+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.839402+0000","last_change":"2023-11-15T03:03:21.834996+0000","last_active":"2023-11-15T03:03:21.839402+0000","last_peered":"2023-11-15T03:03:21.839402+0000","last_clean":"2023-11-15T03:03:21.839402+0000","last_became_active":"2023-11-15T03:03:21.834714+0000","last_became_peered":"2023-11-15T03:03:21.834714+0000","last_unstale":"2023-11-15T03:03:21.839402+0000","last_undegraded":"2023-11-15T03:03:21.839402+0000","last_fullsized":"2023-11-15T03:03:21.839402+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.823076+0000","last_change":"2023-11-15T03:03:21.354231+0000","last_active":"2023-11-15T03:03:21.823076+0000","last_peered":"2023-11-15T03:03:21.823076+0000","last_clean":"2023-11-15T03:03:21.823076+0000","last_became_active":"2023-11-15T03:03:21.353674+0000","last_became_peered":"2023-11-15T03:03:21.353674+0000","last_unstale":"2023-11-15T03:03:21.823076+0000","last_undegraded":"2023-11-15T03:03:21.823076+0000","last_fullsized":"2023-11-15T03:03:21.823076+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.827443+0000","last_change":"2023-11-15T03:03:21.823112+0000","last_active":"2023-11-15T03:03:21.827443+0000","last_peered":"2023-11-15T03:03:21.827443+0000","last_clean":"2023-11-15T03:03:21.827443+0000","last_became_active":"2023-11-15T03:03:21.822842+0000","last_became_peered":"2023-11-15T03:03:21.822842+0000","last_unstale":"2023-11-15T03:03:21.827443+0000","last_undegraded":"2023-11-15T03:03:21.827443+0000","last_fullsized":"2023-11-15T03:03:21.827443+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.809349+0000","last_change":"2023-11-15T03:03:21.349255+0000","last_active":"2023-11-15T03:03:21.809349+0000","last_peered":"2023-11-15T03:03:21.809349+0000","last_clean":"2023-11-15T03:03:21.809349+0000","last_became_active":"2023-11-15T03:03:21.348954+0000","last_became_peered":"2023-11-15T03:03:21.348954+0000","last_unstale":"2023-11-15T03:03:21.809349+0000","last_undegraded":"2023-11-15T03:03:21.809349+0000","last_fullsized":"2023-11-15T03:03:21.809349+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.831385+0000","last_change":"2023-11-15T03:03:21.822744+0000","last_active":"2023-11-15T03:03:21.831385+0000","last_peered":"2023-11-15T03:03:21.831385+0000","last_clean":"2023-11-15T03:03:21.831385+0000","last_became_active":"2023-11-15T03:03:21.822090+0000","last_became_peered":"2023-11-15T03:03:21.822090+0000","last_unstale":"2023-11-15T03:03:21.831385+0000","last_undegraded":"2023-11-15T03:03:21.831385+0000","last_fullsized":"2023-11-15T03:03:21.831385+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.816410+0000","last_change":"2023-11-15T03:03:21.349980+0000","last_active":"2023-11-15T03:03:21.816410+0000","last_peered":"2023-11-15T03:03:21.816410+0000","last_clean":"2023-11-15T03:03:21.816410+0000","last_became_active":"2023-11-15T03:03:21.349676+0000","last_became_peered":"2023-11-15T03:03:21.349676+0000","last_unstale":"2023-11-15T03:03:21.816410+0000","last_undegraded":"2023-11-15T03:03:21.816410+0000","last_fullsized":"2023-11-15T03:03:21.816410+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.834008+0000","last_change":"2023-11-15T03:03:21.828654+0000","last_active":"2023-11-15T03:03:21.834008+0000","last_peered":"2023-11-15T03:03:21.834008+0000","last_clean":"2023-11-15T03:03:21.834008+0000","last_became_active":"2023-11-15T03:03:21.826956+0000","last_became_peered":"2023-11-15T03:03:21.826956+0000","last_unstale":"2023-11-15T03:03:21.834008+0000","last_undegraded":"2023-11-15T03:03:21.834008+0000","last_fullsized":"2023-11-15T03:03:21.834008+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.817281+0000","last_change":"2023-11-15T03:03:21.359062+0000","last_active":"2023-11-15T03:03:21.817281+0000","last_peered":"2023-11-15T03:03:21.817281+0000","last_clean":"2023-11-15T03:03:21.817281+0000","last_became_active":"2023-11-15T03:03:21.358623+0000","last_became_peered":"2023-11-15T03:03:21.358623+0000","last_unstale":"2023-11-15T03:03:21.817281+0000","last_undegraded":"2023-11-15T03:03:21.817281+0000","last_fullsized":"2023-11-15T03:03:21.817281+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.839206+0000","last_change":"2023-11-15T03:03:21.830939+0000","last_active":"2023-11-15T03:03:21.839206+0000","last_peered":"2023-11-15T03:03:21.839206+0000","last_clean":"2023-11-15T03:03:21.839206+0000","last_became_active":"2023-11-15T03:03:21.830347+0000","last_became_peered":"2023-11-15T03:03:21.830347+0000","last_unstale":"2023-11-15T03:03:21.839206+0000","last_undegraded":"2023-11-15T03:03:21.839206+0000","last_fullsized":"2023-11-15T03:03:21.839206+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.819158+0000","last_change":"2023-11-15T03:03:21.357036+0000","last_active":"2023-11-15T03:03:21.819158+0000","last_peered":"2023-11-15T03:03:21.819158+0000","last_clean":"2023-11-15T03:03:21.819158+0000","last_became_active":"2023-11-15T03:03:21.356699+0000","last_became_peered":"2023-11-15T03:03:21.356699+0000","last_unstale":"2023-11-15T03:03:21.819158+0000","last_undegraded":"2023-11-15T03:03:21.819158+0000","last_fullsized":"2023-11-15T03:03:21.819158+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.812604+0000","last_change":"2023-11-15T03:03:21.807931+0000","last_active":"2023-11-15T03:03:21.812604+0000","last_peered":"2023-11-15T03:03:21.812604+0000","last_clean":"2023-11-15T03:03:21.812604+0000","last_became_active":"2023-11-15T03:03:21.807495+0000","last_became_peered":"2023-11-15T03:03:21.807495+0000","last_unstale":"2023-11-15T03:03:21.812604+0000","last_undegraded":"2023-11-15T03:03:21.812604+0000","last_fullsized":"2023-11-15T03:03:21.812604+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.808735+0000","last_change":"2023-11-15T03:03:21.359085+0000","last_active":"2023-11-15T03:03:21.808735+0000","last_peered":"2023-11-15T03:03:21.808735+0000","last_clean":"2023-11-15T03:03:21.808735+0000","last_became_active":"2023-11-15T03:03:21.358801+0000","last_became_peered":"2023-11-15T03:03:21.358801+0000","last_unstale":"2023-11-15T03:03:21.808735+0000","last_undegraded":"2023-11-15T03:03:21.808735+0000","last_fullsized":"2023-11-15T03:03:21.808735+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.832891+0000","last_change":"2023-11-15T03:03:21.827890+0000","last_active":"2023-11-15T03:03:21.832891+0000","last_peered":"2023-11-15T03:03:21.832891+0000","last_clean":"2023-11-15T03:03:21.832891+0000","last_became_active":"2023-11-15T03:03:21.827588+0000","last_became_peered":"2023-11-15T03:03:21.827588+0000","last_unstale":"2023-11-15T03:03:21.832891+0000","last_undegraded":"2023-11-15T03:03:21.832891+0000","last_fullsized":"2023-11-15T03:03:21.832891+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"91'1","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.833615+0000","last_change":"2023-11-15T03:03:21.355296+0000","last_active":"2023-11-15T03:03:21.833615+0000","last_peered":"2023-11-15T03:03:21.833615+0000","last_clean":"2023-11-15T03:03:21.833615+0000","last_became_active":"2023-11-15T03:03:21.354957+0000","last_became_peered":"2023-11-15T03:03:21.354957+0000","last_unstale":"2023-11-15T03:03:21.833615+0000","last_undegraded":"2023-11-15T03:03:21.833615+0000","last_fullsized":"2023-11-15T03:03:21.833615+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.832689+0000","last_change":"2023-11-15T03:03:21.825918+0000","last_active":"2023-11-15T03:03:21.832689+0000","last_peered":"2023-11-15T03:03:21.832689+0000","last_clean":"2023-11-15T03:03:21.832689+0000","last_became_active":"2023-11-15T03:03:21.825356+0000","last_became_peered":"2023-11-15T03:03:21.825356+0000","last_unstale":"2023-11-15T03:03:21.832689+0000","last_undegraded":"2023-11-15T03:03:21.832689+0000","last_fullsized":"2023-11-15T03:03:21.832689+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"91'1","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.835738+0000","last_change":"2023-11-15T03:03:21.354700+0000","last_active":"2023-11-15T03:03:21.835738+0000","last_peered":"2023-11-15T03:03:21.835738+0000","last_clean":"2023-11-15T03:03:21.835738+0000","last_became_active":"2023-11-15T03:03:21.354353+0000","last_became_peered":"2023-11-15T03:03:21.354353+0000","last_unstale":"2023-11-15T03:03:21.835738+0000","last_undegraded":"2023-11-15T03:03:21.835738+0000","last_fullsized":"2023-11-15T03:03:21.835738+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.826049+0000","last_change":"2023-11-15T03:03:21.820582+0000","last_active":"2023-11-15T03:03:21.826049+0000","last_peered":"2023-11-15T03:03:21.826049+0000","last_clean":"2023-11-15T03:03:21.826049+0000","last_became_active":"2023-11-15T03:03:21.820289+0000","last_became_peered":"2023-11-15T03:03:21.820289+0000","last_unstale":"2023-11-15T03:03:21.826049+0000","last_undegraded":"2023-11-15T03:03:21.826049+0000","last_fullsized":"2023-11-15T03:03:21.826049+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"91'2","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.834914+0000","last_change":"2023-11-15T03:03:21.361229+0000","last_active":"2023-11-15T03:03:21.834914+0000","last_peered":"2023-11-15T03:03:21.834914+0000","last_clean":"2023-11-15T03:03:21.834914+0000","last_became_active":"2023-11-15T03:03:21.360944+0000","last_became_peered":"2023-11-15T03:03:21.360944+0000","last_unstale":"2023-11-15T03:03:21.834914+0000","last_undegraded":"2023-11-15T03:03:21.834914+0000","last_fullsized":"2023-11-15T03:03:21.834914+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.828889+0000","last_change":"2023-11-15T03:03:21.820504+0000","last_active":"2023-11-15T03:03:21.828889+0000","last_peered":"2023-11-15T03:03:21.828889+0000","last_clean":"2023-11-15T03:03:21.828889+0000","last_became_active":"2023-11-15T03:03:21.820083+0000","last_became_peered":"2023-11-15T03:03:21.820083+0000","last_unstale":"2023-11-15T03:03:21.828889+0000","last_undegraded":"2023-11-15T03:03:21.828889+0000","last_fullsized":"2023-11-15T03:03:21.828889+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.809466+0000","last_change":"2023-11-15T03:03:21.365563+0000","last_active":"2023-11-15T03:03:21.809466+0000","last_peered":"2023-11-15T03:03:21.809466+0000","last_clean":"2023-11-15T03:03:21.809466+0000","last_became_active":"2023-11-15T03:03:21.365275+0000","last_became_peered":"2023-11-15T03:03:21.365275+0000","last_unstale":"2023-11-15T03:03:21.809466+0000","last_undegraded":"2023-11-15T03:03:21.809466+0000","last_fullsized":"2023-11-15T03:03:21.809466+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.830433+0000","last_change":"2023-11-15T03:03:21.817138+0000","last_active":"2023-11-15T03:03:21.830433+0000","last_peered":"2023-11-15T03:03:21.830433+0000","last_clean":"2023-11-15T03:03:21.830433+0000","last_became_active":"2023-11-15T03:03:21.816865+0000","last_became_peered":"2023-11-15T03:03:21.816865+0000","last_unstale":"2023-11-15T03:03:21.830433+0000","last_undegraded":"2023-11-15T03:03:21.830433+0000","last_fullsized":"2023-11-15T03:03:21.830433+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.817456+0000","last_change":"2023-11-15T03:03:21.358938+0000","last_active":"2023-11-15T03:03:21.817456+0000","last_peered":"2023-11-15T03:03:21.817456+0000","last_clean":"2023-11-15T03:03:21.817456+0000","last_became_active":"2023-11-15T03:03:21.358671+0000","last_became_peered":"2023-11-15T03:03:21.358671+0000","last_unstale":"2023-11-15T03:03:21.817456+0000","last_undegraded":"2023-11-15T03:03:21.817456+0000","last_fullsized":"2023-11-15T03:03:21.817456+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.827890+0000","last_change":"2023-11-15T03:03:21.823361+0000","last_active":"2023-11-15T03:03:21.827890+0000","last_peered":"2023-11-15T03:03:21.827890+0000","last_clean":"2023-11-15T03:03:21.827890+0000","last_became_active":"2023-11-15T03:03:21.822943+0000","last_became_peered":"2023-11-15T03:03:21.822943+0000","last_unstale":"2023-11-15T03:03:21.827890+0000","last_undegraded":"2023-11-15T03:03:21.827890+0000","last_fullsized":"2023-11-15T03:03:21.827890+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.817379+0000","last_change":"2023-11-15T03:03:21.359616+0000","last_active":"2023-11-15T03:03:21.817379+0000","last_peered":"2023-11-15T03:03:21.817379+0000","last_clean":"2023-11-15T03:03:21.817379+0000","last_became_active":"2023-11-15T03:03:21.359297+0000","last_became_peered":"2023-11-15T03:03:21.359297+0000","last_unstale":"2023-11-15T03:03:21.817379+0000","last_undegraded":"2023-11-15T03:03:21.817379+0000","last_fullsized":"2023-11-15T03:03:21.817379+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.834016+0000","last_change":"2023-11-15T03:03:21.828522+0000","last_active":"2023-11-15T03:03:21.834016+0000","last_peered":"2023-11-15T03:03:21.834016+0000","last_clean":"2023-11-15T03:03:21.834016+0000","last_became_active":"2023-11-15T03:03:21.827976+0000","last_became_peered":"2023-11-15T03:03:21.827976+0000","last_unstale":"2023-11-15T03:03:21.834016+0000","last_undegraded":"2023-11-15T03:03:21.834016+0000","last_fullsized":"2023-11-15T03:03:21.834016+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.809392+0000","last_change":"2023-11-15T03:03:21.349964+0000","last_active":"2023-11-15T03:03:21.809392+0000","last_peered":"2023-11-15T03:03:21.809392+0000","last_clean":"2023-11-15T03:03:21.809392+0000","last_became_active":"2023-11-15T03:03:21.349689+0000","last_became_peered":"2023-11-15T03:03:21.349689+0000","last_unstale":"2023-11-15T03:03:21.809392+0000","last_undegraded":"2023-11-15T03:03:21.809392+0000","last_fullsized":"2023-11-15T03:03:21.809392+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.830100+0000","last_change":"2023-11-15T03:03:21.816894+0000","last_active":"2023-11-15T03:03:21.830100+0000","last_peered":"2023-11-15T03:03:21.830100+0000","last_clean":"2023-11-15T03:03:21.830100+0000","last_became_active":"2023-11-15T03:03:21.816628+0000","last_became_peered":"2023-11-15T03:03:21.816628+0000","last_unstale":"2023-11-15T03:03:21.830100+0000","last_undegraded":"2023-11-15T03:03:21.830100+0000","last_fullsized":"2023-11-15T03:03:21.830100+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.800671+0000","last_change":"2023-11-15T03:03:21.362349+0000","last_active":"2023-11-15T03:03:21.800671+0000","last_peered":"2023-11-15T03:03:21.800671+0000","last_clean":"2023-11-15T03:03:21.800671+0000","last_became_active":"2023-11-15T03:03:21.361595+0000","last_became_peered":"2023-11-15T03:03:21.361595+0000","last_unstale":"2023-11-15T03:03:21.800671+0000","last_undegraded":"2023-11-15T03:03:21.800671+0000","last_fullsized":"2023-11-15T03:03:21.800671+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.839131+0000","last_change":"2023-11-15T03:03:21.829607+0000","last_active":"2023-11-15T03:03:21.839131+0000","last_peered":"2023-11-15T03:03:21.839131+0000","last_clean":"2023-11-15T03:03:21.839131+0000","last_became_active":"2023-11-15T03:03:21.829218+0000","last_became_peered":"2023-11-15T03:03:21.829218+0000","last_unstale":"2023-11-15T03:03:21.839131+0000","last_undegraded":"2023-11-15T03:03:21.839131+0000","last_fullsized":"2023-11-15T03:03:21.839131+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.804610+0000","last_change":"2023-11-15T03:03:21.353113+0000","last_active":"2023-11-15T03:03:21.804610+0000","last_peered":"2023-11-15T03:03:21.804610+0000","last_clean":"2023-11-15T03:03:21.804610+0000","last_became_active":"2023-11-15T03:03:21.352810+0000","last_became_peered":"2023-11-15T03:03:21.352810+0000","last_unstale":"2023-11-15T03:03:21.804610+0000","last_undegraded":"2023-11-15T03:03:21.804610+0000","last_fullsized":"2023-11-15T03:03:21.804610+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.834545+0000","last_change":"2023-11-15T03:03:21.831892+0000","last_active":"2023-11-15T03:03:21.834545+0000","last_peered":"2023-11-15T03:03:21.834545+0000","last_clean":"2023-11-15T03:03:21.834545+0000","last_became_active":"2023-11-15T03:03:21.831266+0000","last_became_peered":"2023-11-15T03:03:21.831266+0000","last_unstale":"2023-11-15T03:03:21.834545+0000","last_undegraded":"2023-11-15T03:03:21.834545+0000","last_fullsized":"2023-11-15T03:03:21.834545+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.813350+0000","last_change":"2023-11-15T03:03:21.355937+0000","last_active":"2023-11-15T03:03:21.813350+0000","last_peered":"2023-11-15T03:03:21.813350+0000","last_clean":"2023-11-15T03:03:21.813350+0000","last_became_active":"2023-11-15T03:03:21.355592+0000","last_became_peered":"2023-11-15T03:03:21.355592+0000","last_unstale":"2023-11-15T03:03:21.813350+0000","last_undegraded":"2023-11-15T03:03:21.813350+0000","last_fullsized":"2023-11-15T03:03:21.813350+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.833455+0000","last_change":"2023-11-15T03:03:21.829097+0000","last_active":"2023-11-15T03:03:21.833455+0000","last_peered":"2023-11-15T03:03:21.833455+0000","last_clean":"2023-11-15T03:03:21.833455+0000","last_became_active":"2023-11-15T03:03:21.828844+0000","last_became_peered":"2023-11-15T03:03:21.828844+0000","last_unstale":"2023-11-15T03:03:21.833455+0000","last_undegraded":"2023-11-15T03:03:21.833455+0000","last_fullsized":"2023-11-15T03:03:21.833455+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"91'3","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.839091+0000","last_change":"2023-11-15T03:03:21.356723+0000","last_active":"2023-11-15T03:03:21.839091+0000","last_peered":"2023-11-15T03:03:21.839091+0000","last_clean":"2023-11-15T03:03:21.839091+0000","last_became_active":"2023-11-15T03:03:21.355197+0000","last_became_peered":"2023-11-15T03:03:21.355197+0000","last_unstale":"2023-11-15T03:03:21.839091+0000","last_undegraded":"2023-11-15T03:03:21.839091+0000","last_fullsized":"2023-11-15T03:03:21.839091+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.830237+0000","last_change":"2023-11-15T03:03:21.817028+0000","last_active":"2023-11-15T03:03:21.830237+0000","last_peered":"2023-11-15T03:03:21.830237+0000","last_clean":"2023-11-15T03:03:21.830237+0000","last_became_active":"2023-11-15T03:03:21.816752+0000","last_became_peered":"2023-11-15T03:03:21.816752+0000","last_unstale":"2023-11-15T03:03:21.830237+0000","last_undegraded":"2023-11-15T03:03:21.830237+0000","last_fullsized":"2023-11-15T03:03:21.830237+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"91'2","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.823393+0000","last_change":"2023-11-15T03:03:21.346978+0000","last_active":"2023-11-15T03:03:21.823393+0000","last_peered":"2023-11-15T03:03:21.823393+0000","last_clean":"2023-11-15T03:03:21.823393+0000","last_became_active":"2023-11-15T03:03:21.346152+0000","last_became_peered":"2023-11-15T03:03:21.346152+0000","last_unstale":"2023-11-15T03:03:21.823393+0000","last_undegraded":"2023-11-15T03:03:21.823393+0000","last_fullsized":"2023-11-15T03:03:21.823393+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.825703+0000","last_change":"2023-11-15T03:03:21.815044+0000","last_active":"2023-11-15T03:03:21.825703+0000","last_peered":"2023-11-15T03:03:21.825703+0000","last_clean":"2023-11-15T03:03:21.825703+0000","last_became_active":"2023-11-15T03:03:21.814775+0000","last_became_peered":"2023-11-15T03:03:21.814775+0000","last_unstale":"2023-11-15T03:03:21.825703+0000","last_undegraded":"2023-11-15T03:03:21.825703+0000","last_fullsized":"2023-11-15T03:03:21.825703+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"91'1","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.833989+0000","last_change":"2023-11-15T03:03:21.361703+0000","last_active":"2023-11-15T03:03:21.833989+0000","last_peered":"2023-11-15T03:03:21.833989+0000","last_clean":"2023-11-15T03:03:21.833989+0000","last_became_active":"2023-11-15T03:03:21.361136+0000","last_became_peered":"2023-11-15T03:03:21.361136+0000","last_unstale":"2023-11-15T03:03:21.833989+0000","last_undegraded":"2023-11-15T03:03:21.833989+0000","last_fullsized":"2023-11-15T03:03:21.833989+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.835731+0000","last_change":"2023-11-15T03:03:21.826000+0000","last_active":"2023-11-15T03:03:21.835731+0000","last_peered":"2023-11-15T03:03:21.835731+0000","last_clean":"2023-11-15T03:03:21.835731+0000","last_became_active":"2023-11-15T03:03:21.825726+0000","last_became_peered":"2023-11-15T03:03:21.825726+0000","last_unstale":"2023-11-15T03:03:21.835731+0000","last_undegraded":"2023-11-15T03:03:21.835731+0000","last_fullsized":"2023-11-15T03:03:21.835731+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.806916+0000","last_change":"2023-11-15T03:03:21.353335+0000","last_active":"2023-11-15T03:03:21.806916+0000","last_peered":"2023-11-15T03:03:21.806916+0000","last_clean":"2023-11-15T03:03:21.806916+0000","last_became_active":"2023-11-15T03:03:21.353046+0000","last_became_peered":"2023-11-15T03:03:21.353046+0000","last_unstale":"2023-11-15T03:03:21.806916+0000","last_undegraded":"2023-11-15T03:03:21.806916+0000","last_fullsized":"2023-11-15T03:03:21.806916+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.829556+0000","last_change":"2023-11-15T03:03:21.820794+0000","last_active":"2023-11-15T03:03:21.829556+0000","last_peered":"2023-11-15T03:03:21.829556+0000","last_clean":"2023-11-15T03:03:21.829556+0000","last_became_active":"2023-11-15T03:03:21.820544+0000","last_became_peered":"2023-11-15T03:03:21.820544+0000","last_unstale":"2023-11-15T03:03:21.829556+0000","last_undegraded":"2023-11-15T03:03:21.829556+0000","last_fullsized":"2023-11-15T03:03:21.829556+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.817428+0000","last_change":"2023-11-15T03:03:21.351938+0000","last_active":"2023-11-15T03:03:21.817428+0000","last_peered":"2023-11-15T03:03:21.817428+0000","last_clean":"2023-11-15T03:03:21.817428+0000","last_became_active":"2023-11-15T03:03:21.351589+0000","last_became_peered":"2023-11-15T03:03:21.351589+0000","last_unstale":"2023-11-15T03:03:21.817428+0000","last_undegraded":"2023-11-15T03:03:21.817428+0000","last_fullsized":"2023-11-15T03:03:21.817428+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.832133+0000","last_change":"2023-11-15T03:03:21.827203+0000","last_active":"2023-11-15T03:03:21.832133+0000","last_peered":"2023-11-15T03:03:21.832133+0000","last_clean":"2023-11-15T03:03:21.832133+0000","last_became_active":"2023-11-15T03:03:21.826912+0000","last_became_peered":"2023-11-15T03:03:21.826912+0000","last_unstale":"2023-11-15T03:03:21.832133+0000","last_undegraded":"2023-11-15T03:03:21.832133+0000","last_fullsized":"2023-11-15T03:03:21.832133+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"91'4","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.833043+0000","last_change":"2023-11-15T03:03:21.359067+0000","last_active":"2023-11-15T03:03:21.833043+0000","last_peered":"2023-11-15T03:03:21.833043+0000","last_clean":"2023-11-15T03:03:21.833043+0000","last_became_active":"2023-11-15T03:03:21.358763+0000","last_became_peered":"2023-11-15T03:03:21.358763+0000","last_unstale":"2023-11-15T03:03:21.833043+0000","last_undegraded":"2023-11-15T03:03:21.833043+0000","last_fullsized":"2023-11-15T03:03:21.833043+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.827938+0000","last_change":"2023-11-15T03:03:21.824029+0000","last_active":"2023-11-15T03:03:21.827938+0000","last_peered":"2023-11-15T03:03:21.827938+0000","last_clean":"2023-11-15T03:03:21.827938+0000","last_became_active":"2023-11-15T03:03:21.823703+0000","last_became_peered":"2023-11-15T03:03:21.823703+0000","last_unstale":"2023-11-15T03:03:21.827938+0000","last_undegraded":"2023-11-15T03:03:21.827938+0000","last_fullsized":"2023-11-15T03:03:21.827938+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"91'2","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.828367+0000","last_change":"2023-11-15T03:03:21.363345+0000","last_active":"2023-11-15T03:03:21.828367+0000","last_peered":"2023-11-15T03:03:21.828367+0000","last_clean":"2023-11-15T03:03:21.828367+0000","last_became_active":"2023-11-15T03:03:21.363044+0000","last_became_peered":"2023-11-15T03:03:21.363044+0000","last_unstale":"2023-11-15T03:03:21.828367+0000","last_undegraded":"2023-11-15T03:03:21.828367+0000","last_fullsized":"2023-11-15T03:03:21.828367+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.833983+0000","last_change":"2023-11-15T03:03:21.828299+0000","last_active":"2023-11-15T03:03:21.833983+0000","last_peered":"2023-11-15T03:03:21.833983+0000","last_clean":"2023-11-15T03:03:21.833983+0000","last_became_active":"2023-11-15T03:03:21.827762+0000","last_became_peered":"2023-11-15T03:03:21.827762+0000","last_unstale":"2023-11-15T03:03:21.833983+0000","last_undegraded":"2023-11-15T03:03:21.833983+0000","last_fullsized":"2023-11-15T03:03:21.833983+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.824892+0000","last_change":"2023-11-15T03:03:21.350580+0000","last_active":"2023-11-15T03:03:21.824892+0000","last_peered":"2023-11-15T03:03:21.824892+0000","last_clean":"2023-11-15T03:03:21.824892+0000","last_became_active":"2023-11-15T03:03:21.350262+0000","last_became_peered":"2023-11-15T03:03:21.350262+0000","last_unstale":"2023-11-15T03:03:21.824892+0000","last_undegraded":"2023-11-15T03:03:21.824892+0000","last_fullsized":"2023-11-15T03:03:21.824892+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.832850+0000","last_change":"2023-11-15T03:03:21.822530+0000","last_active":"2023-11-15T03:03:21.832850+0000","last_peered":"2023-11-15T03:03:21.832850+0000","last_clean":"2023-11-15T03:03:21.832850+0000","last_became_active":"2023-11-15T03:03:21.819683+0000","last_became_peered":"2023-11-15T03:03:21.819683+0000","last_unstale":"2023-11-15T03:03:21.832850+0000","last_undegraded":"2023-11-15T03:03:21.832850+0000","last_fullsized":"2023-11-15T03:03:21.832850+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.823462+0000","last_change":"2023-11-15T03:03:21.354020+0000","last_active":"2023-11-15T03:03:21.823462+0000","last_peered":"2023-11-15T03:03:21.823462+0000","last_clean":"2023-11-15T03:03:21.823462+0000","last_became_active":"2023-11-15T03:03:21.353299+0000","last_became_peered":"2023-11-15T03:03:21.353299+0000","last_unstale":"2023-11-15T03:03:21.823462+0000","last_undegraded":"2023-11-15T03:03:21.823462+0000","last_fullsized":"2023-11-15T03:03:21.823462+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.834239+0000","last_change":"2023-11-15T03:03:21.831226+0000","last_active":"2023-11-15T03:03:21.834239+0000","last_peered":"2023-11-15T03:03:21.834239+0000","last_clean":"2023-11-15T03:03:21.834239+0000","last_became_active":"2023-11-15T03:03:21.830886+0000","last_became_peered":"2023-11-15T03:03:21.830886+0000","last_unstale":"2023-11-15T03:03:21.834239+0000","last_undegraded":"2023-11-15T03:03:21.834239+0000","last_fullsized":"2023-11-15T03:03:21.834239+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.809576+0000","last_change":"2023-11-15T03:03:21.358593+0000","last_active":"2023-11-15T03:03:21.809576+0000","last_peered":"2023-11-15T03:03:21.809576+0000","last_clean":"2023-11-15T03:03:21.809576+0000","last_became_active":"2023-11-15T03:03:21.358264+0000","last_became_peered":"2023-11-15T03:03:21.358264+0000","last_unstale":"2023-11-15T03:03:21.809576+0000","last_undegraded":"2023-11-15T03:03:21.809576+0000","last_fullsized":"2023-11-15T03:03:21.809576+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.828956+0000","last_change":"2023-11-15T03:03:21.819405+0000","last_active":"2023-11-15T03:03:21.828956+0000","last_peered":"2023-11-15T03:03:21.828956+0000","last_clean":"2023-11-15T03:03:21.828956+0000","last_became_active":"2023-11-15T03:03:21.816309+0000","last_became_peered":"2023-11-15T03:03:21.816309+0000","last_unstale":"2023-11-15T03:03:21.828956+0000","last_undegraded":"2023-11-15T03:03:21.828956+0000","last_fullsized":"2023-11-15T03:03:21.828956+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.810980+0000","last_change":"2023-11-15T03:03:21.355413+0000","last_active":"2023-11-15T03:03:21.810980+0000","last_peered":"2023-11-15T03:03:21.810980+0000","last_clean":"2023-11-15T03:03:21.810980+0000","last_became_active":"2023-11-15T03:03:21.354050+0000","last_became_peered":"2023-11-15T03:03:21.354050+0000","last_unstale":"2023-11-15T03:03:21.810980+0000","last_undegraded":"2023-11-15T03:03:21.810980+0000","last_fullsized":"2023-11-15T03:03:21.810980+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.829635+0000","last_change":"2023-11-15T03:03:21.821485+0000","last_active":"2023-11-15T03:03:21.829635+0000","last_peered":"2023-11-15T03:03:21.829635+0000","last_clean":"2023-11-15T03:03:21.829635+0000","last_became_active":"2023-11-15T03:03:21.820781+0000","last_became_peered":"2023-11-15T03:03:21.820781+0000","last_unstale":"2023-11-15T03:03:21.829635+0000","last_undegraded":"2023-11-15T03:03:21.829635+0000","last_fullsized":"2023-11-15T03:03:21.829635+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.804021+0000","last_change":"2023-11-15T03:03:21.352379+0000","last_active":"2023-11-15T03:03:21.804021+0000","last_peered":"2023-11-15T03:03:21.804021+0000","last_clean":"2023-11-15T03:03:21.804021+0000","last_became_active":"2023-11-15T03:03:21.352070+0000","last_became_peered":"2023-11-15T03:03:21.352070+0000","last_unstale":"2023-11-15T03:03:21.804021+0000","last_undegraded":"2023-11-15T03:03:21.804021+0000","last_fullsized":"2023-11-15T03:03:21.804021+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.836412+0000","last_change":"2023-11-15T03:03:21.833254+0000","last_active":"2023-11-15T03:03:21.836412+0000","last_peered":"2023-11-15T03:03:21.836412+0000","last_clean":"2023-11-15T03:03:21.836412+0000","last_became_active":"2023-11-15T03:03:21.832967+0000","last_became_peered":"2023-11-15T03:03:21.832967+0000","last_unstale":"2023-11-15T03:03:21.836412+0000","last_undegraded":"2023-11-15T03:03:21.836412+0000","last_fullsized":"2023-11-15T03:03:21.836412+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.809642+0000","last_change":"2023-11-15T03:03:21.349252+0000","last_active":"2023-11-15T03:03:21.809642+0000","last_peered":"2023-11-15T03:03:21.809642+0000","last_clean":"2023-11-15T03:03:21.809642+0000","last_became_active":"2023-11-15T03:03:21.348936+0000","last_became_peered":"2023-11-15T03:03:21.348936+0000","last_unstale":"2023-11-15T03:03:21.809642+0000","last_undegraded":"2023-11-15T03:03:21.809642+0000","last_fullsized":"2023-11-15T03:03:21.809642+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.829017+0000","last_change":"2023-11-15T03:03:21.822737+0000","last_active":"2023-11-15T03:03:21.829017+0000","last_peered":"2023-11-15T03:03:21.829017+0000","last_clean":"2023-11-15T03:03:21.829017+0000","last_became_active":"2023-11-15T03:03:21.821235+0000","last_became_peered":"2023-11-15T03:03:21.821235+0000","last_unstale":"2023-11-15T03:03:21.829017+0000","last_undegraded":"2023-11-15T03:03:21.829017+0000","last_fullsized":"2023-11-15T03:03:21.829017+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.800678+0000","last_change":"2023-11-15T03:03:21.362987+0000","last_active":"2023-11-15T03:03:21.800678+0000","last_peered":"2023-11-15T03:03:21.800678+0000","last_clean":"2023-11-15T03:03:21.800678+0000","last_became_active":"2023-11-15T03:03:21.362620+0000","last_became_peered":"2023-11-15T03:03:21.362620+0000","last_unstale":"2023-11-15T03:03:21.800678+0000","last_undegraded":"2023-11-15T03:03:21.800678+0000","last_fullsized":"2023-11-15T03:03:21.800678+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.832767+0000","last_change":"2023-11-15T03:03:21.827865+0000","last_active":"2023-11-15T03:03:21.832767+0000","last_peered":"2023-11-15T03:03:21.832767+0000","last_clean":"2023-11-15T03:03:21.832767+0000","last_became_active":"2023-11-15T03:03:21.825508+0000","last_became_peered":"2023-11-15T03:03:21.825508+0000","last_unstale":"2023-11-15T03:03:21.832767+0000","last_undegraded":"2023-11-15T03:03:21.832767+0000","last_fullsized":"2023-11-15T03:03:21.832767+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.797821+0000","last_change":"2023-11-15T03:03:21.353514+0000","last_active":"2023-11-15T03:03:21.797821+0000","last_peered":"2023-11-15T03:03:21.797821+0000","last_clean":"2023-11-15T03:03:21.797821+0000","last_became_active":"2023-11-15T03:03:21.353262+0000","last_became_peered":"2023-11-15T03:03:21.353262+0000","last_unstale":"2023-11-15T03:03:21.797821+0000","last_undegraded":"2023-11-15T03:03:21.797821+0000","last_fullsized":"2023-11-15T03:03:21.797821+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.811388+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T03:03:21.811388+0000","last_peered":"2023-11-15T03:03:21.811388+0000","last_clean":"2023-11-15T03:03:21.811388+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T03:03:21.811388+0000","last_undegraded":"2023-11-15T03:03:21.811388+0000","last_fullsized":"2023-11-15T03:03:21.811388+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.841137+0000","last_change":"2023-11-15T03:03:21.836588+0000","last_active":"2023-11-15T03:03:21.841137+0000","last_peered":"2023-11-15T03:03:21.841137+0000","last_clean":"2023-11-15T03:03:21.841137+0000","last_became_active":"2023-11-15T03:03:21.836128+0000","last_became_peered":"2023-11-15T03:03:21.836128+0000","last_unstale":"2023-11-15T03:03:21.841137+0000","last_undegraded":"2023-11-15T03:03:21.841137+0000","last_fullsized":"2023-11-15T03:03:21.841137+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.803344+0000","last_change":"2023-11-15T03:03:21.348017+0000","last_active":"2023-11-15T03:03:21.803344+0000","last_peered":"2023-11-15T03:03:21.803344+0000","last_clean":"2023-11-15T03:03:21.803344+0000","last_became_active":"2023-11-15T03:03:21.347687+0000","last_became_peered":"2023-11-15T03:03:21.347687+0000","last_unstale":"2023-11-15T03:03:21.803344+0000","last_undegraded":"2023-11-15T03:03:21.803344+0000","last_fullsized":"2023-11-15T03:03:21.803344+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":75,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.818847+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T03:03:21.818847+0000","last_peered":"2023-11-15T03:03:21.818847+0000","last_clean":"2023-11-15T03:03:21.818847+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T03:03:21.818847+0000","last_undegraded":"2023-11-15T03:03:21.818847+0000","last_fullsized":"2023-11-15T03:03:21.818847+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.825002+0000","last_change":"2023-11-15T03:03:21.819880+0000","last_active":"2023-11-15T03:03:21.825002+0000","last_peered":"2023-11-15T03:03:21.825002+0000","last_clean":"2023-11-15T03:03:21.825002+0000","last_became_active":"2023-11-15T03:03:21.818111+0000","last_became_peered":"2023-11-15T03:03:21.818111+0000","last_unstale":"2023-11-15T03:03:21.825002+0000","last_undegraded":"2023-11-15T03:03:21.825002+0000","last_fullsized":"2023-11-15T03:03:21.825002+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.797503+0000","last_change":"2023-11-15T03:03:21.363981+0000","last_active":"2023-11-15T03:03:21.797503+0000","last_peered":"2023-11-15T03:03:21.797503+0000","last_clean":"2023-11-15T03:03:21.797503+0000","last_became_active":"2023-11-15T03:03:21.363379+0000","last_became_peered":"2023-11-15T03:03:21.363379+0000","last_unstale":"2023-11-15T03:03:21.797503+0000","last_undegraded":"2023-11-15T03:03:21.797503+0000","last_fullsized":"2023-11-15T03:03:21.797503+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.801974+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T03:03:21.801974+0000","last_peered":"2023-11-15T03:03:21.801974+0000","last_clean":"2023-11-15T03:03:21.801974+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T03:03:21.801974+0000","last_undegraded":"2023-11-15T03:03:21.801974+0000","last_fullsized":"2023-11-15T03:03:21.801974+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.834026+0000","last_change":"2023-11-15T03:03:21.830257+0000","last_active":"2023-11-15T03:03:21.834026+0000","last_peered":"2023-11-15T03:03:21.834026+0000","last_clean":"2023-11-15T03:03:21.834026+0000","last_became_active":"2023-11-15T03:03:21.828641+0000","last_became_peered":"2023-11-15T03:03:21.828641+0000","last_unstale":"2023-11-15T03:03:21.834026+0000","last_undegraded":"2023-11-15T03:03:21.834026+0000","last_fullsized":"2023-11-15T03:03:21.834026+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"91'1","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.822174+0000","last_change":"2023-11-15T03:03:21.349218+0000","last_active":"2023-11-15T03:03:21.822174+0000","last_peered":"2023-11-15T03:03:21.822174+0000","last_clean":"2023-11-15T03:03:21.822174+0000","last_became_active":"2023-11-15T03:03:21.348942+0000","last_became_peered":"2023-11-15T03:03:21.348942+0000","last_unstale":"2023-11-15T03:03:21.822174+0000","last_undegraded":"2023-11-15T03:03:21.822174+0000","last_fullsized":"2023-11-15T03:03:21.822174+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":75,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.819444+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T03:03:21.819444+0000","last_peered":"2023-11-15T03:03:21.819444+0000","last_clean":"2023-11-15T03:03:21.819444+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T03:03:21.819444+0000","last_undegraded":"2023-11-15T03:03:21.819444+0000","last_fullsized":"2023-11-15T03:03:21.819444+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.835831+0000","last_change":"2023-11-15T03:03:21.827107+0000","last_active":"2023-11-15T03:03:21.835831+0000","last_peered":"2023-11-15T03:03:21.835831+0000","last_clean":"2023-11-15T03:03:21.835831+0000","last_became_active":"2023-11-15T03:03:21.826821+0000","last_became_peered":"2023-11-15T03:03:21.826821+0000","last_unstale":"2023-11-15T03:03:21.835831+0000","last_undegraded":"2023-11-15T03:03:21.835831+0000","last_fullsized":"2023-11-15T03:03:21.835831+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.823343+0000","last_change":"2023-11-15T03:03:21.361290+0000","last_active":"2023-11-15T03:03:21.823343+0000","last_peered":"2023-11-15T03:03:21.823343+0000","last_clean":"2023-11-15T03:03:21.823343+0000","last_became_active":"2023-11-15T03:03:21.360944+0000","last_became_peered":"2023-11-15T03:03:21.360944+0000","last_unstale":"2023-11-15T03:03:21.823343+0000","last_undegraded":"2023-11-15T03:03:21.823343+0000","last_fullsized":"2023-11-15T03:03:21.823343+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":84,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.804030+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T03:03:21.804030+0000","last_peered":"2023-11-15T03:03:21.804030+0000","last_clean":"2023-11-15T03:03:21.804030+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T03:03:21.804030+0000","last_undegraded":"2023-11-15T03:03:21.804030+0000","last_fullsized":"2023-11-15T03:03:21.804030+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.826296+0000","last_change":"2023-11-15T03:03:21.817496+0000","last_active":"2023-11-15T03:03:21.826296+0000","last_peered":"2023-11-15T03:03:21.826296+0000","last_clean":"2023-11-15T03:03:21.826296+0000","last_became_active":"2023-11-15T03:03:21.817067+0000","last_became_peered":"2023-11-15T03:03:21.817067+0000","last_unstale":"2023-11-15T03:03:21.826296+0000","last_undegraded":"2023-11-15T03:03:21.826296+0000","last_fullsized":"2023-11-15T03:03:21.826296+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"91'2","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.833678+0000","last_change":"2023-11-15T03:03:21.363828+0000","last_active":"2023-11-15T03:03:21.833678+0000","last_peered":"2023-11-15T03:03:21.833678+0000","last_clean":"2023-11-15T03:03:21.833678+0000","last_became_active":"2023-11-15T03:03:21.360592+0000","last_became_peered":"2023-11-15T03:03:21.360592+0000","last_unstale":"2023-11-15T03:03:21.833678+0000","last_undegraded":"2023-11-15T03:03:21.833678+0000","last_fullsized":"2023-11-15T03:03:21.833678+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.797038+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T03:03:21.797038+0000","last_peered":"2023-11-15T03:03:21.797038+0000","last_clean":"2023-11-15T03:03:21.797038+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T03:03:21.797038+0000","last_undegraded":"2023-11-15T03:03:21.797038+0000","last_fullsized":"2023-11-15T03:03:21.797038+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.833993+0000","last_change":"2023-11-15T03:03:21.828261+0000","last_active":"2023-11-15T03:03:21.833993+0000","last_peered":"2023-11-15T03:03:21.833993+0000","last_clean":"2023-11-15T03:03:21.833993+0000","last_became_active":"2023-11-15T03:03:21.827967+0000","last_became_peered":"2023-11-15T03:03:21.827967+0000","last_unstale":"2023-11-15T03:03:21.833993+0000","last_undegraded":"2023-11-15T03:03:21.833993+0000","last_fullsized":"2023-11-15T03:03:21.833993+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"91'1","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.827887+0000","last_change":"2023-11-15T03:03:21.355048+0000","last_active":"2023-11-15T03:03:21.827887+0000","last_peered":"2023-11-15T03:03:21.827887+0000","last_clean":"2023-11-15T03:03:21.827887+0000","last_became_active":"2023-11-15T03:03:21.354588+0000","last_became_peered":"2023-11-15T03:03:21.354588+0000","last_unstale":"2023-11-15T03:03:21.827887+0000","last_undegraded":"2023-11-15T03:03:21.827887+0000","last_fullsized":"2023-11-15T03:03:21.827887+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.800813+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T03:03:21.800813+0000","last_peered":"2023-11-15T03:03:21.800813+0000","last_clean":"2023-11-15T03:03:21.800813+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T03:03:21.800813+0000","last_undegraded":"2023-11-15T03:03:21.800813+0000","last_fullsized":"2023-11-15T03:03:21.800813+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.829788+0000","last_change":"2023-11-15T03:03:21.824202+0000","last_active":"2023-11-15T03:03:21.829788+0000","last_peered":"2023-11-15T03:03:21.829788+0000","last_clean":"2023-11-15T03:03:21.829788+0000","last_became_active":"2023-11-15T03:03:21.823819+0000","last_became_peered":"2023-11-15T03:03:21.823819+0000","last_unstale":"2023-11-15T03:03:21.829788+0000","last_undegraded":"2023-11-15T03:03:21.829788+0000","last_fullsized":"2023-11-15T03:03:21.829788+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:21.306874+0000","last_clean_scrub_stamp":"2023-11-15T03:03:21.306874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.819365+0000","last_change":"2023-11-15T03:03:21.348694+0000","last_active":"2023-11-15T03:03:21.819365+0000","last_peered":"2023-11-15T03:03:21.819365+0000","last_clean":"2023-11-15T03:03:21.819365+0000","last_became_active":"2023-11-15T03:03:21.348422+0000","last_became_peered":"2023-11-15T03:03:21.348422+0000","last_unstale":"2023-11-15T03:03:21.819365+0000","last_undegraded":"2023-11-15T03:03:21.819365+0000","last_fullsized":"2023-11-15T03:03:21.819365+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:03:20.290946+0000","last_clean_scrub_stamp":"2023-11-15T03:03:20.290946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":84,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.804249+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T03:03:21.804249+0000","last_peered":"2023-11-15T03:03:21.804249+0000","last_clean":"2023-11-15T03:03:21.804249+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T03:03:21.804249+0000","last_undegraded":"2023-11-15T03:03:21.804249+0000","last_fullsized":"2023-11-15T03:03:21.804249+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-15T03:03:21.800828+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T03:03:21.800828+0000","last_peered":"2023-11-15T03:03:21.800828+0000","last_clean":"2023-11-15T03:03:21.800828+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T03:03:21.800828+0000","last_undegraded":"2023-11-15T03:03:21.800828+0000","last_fullsized":"2023-11-15T03:03:21.800828+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542226,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298204,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073636,"statfs":{"total":96636764160,"available":96331403264,"internally_reserved":0,"allocated":1134592,"data_stored":521179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:03:06 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.1559999999999999,"15min":1.1819999999999999},"min":{"1min":0.56200000000000006,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4370000000000001,"5min":10.961,"15min":10.961},"last":1.02},{"interface":"front","average":{"1min":1.085,"5min":1.1160000000000001,"15min":1.181},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.5589999999999999,"5min":3.4910000000000001,"15min":3.4910000000000001},"last":1.23}]},{"osd":1,"last update":"Wed Nov 15 03:03:06 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.157,"15min":1.151},"min":{"1min":0.84999999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.486,"5min":5.4930000000000003,"15min":5.4930000000000003},"last":1.286},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2050000000000001,"15min":1.1879999999999999},"min":{"1min":0.873,"5min":0.747,"15min":0.747},"max":{"1min":1.6970000000000001,"5min":6.8239999999999998,"15min":6.8239999999999998},"last":1.1479999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:03:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.145,"15min":1.21},"min":{"1min":0.82599999999999996,"5min":0.751,"15min":0.751},"max":{"1min":1.51,"5min":5.7000000000000002,"15min":5.7000000000000002},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.21,"15min":1.278},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5449999999999999,"5min":2.75,"15min":2.75},"last":1.4550000000000001}]},{"osd":3,"last update":"Wed Nov 15 03:03:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.3,"15min":1.3600000000000001},"min":{"1min":0.97999999999999998,"5min":0.751,"15min":0.751},"max":{"1min":1.6519999999999999,"5min":12.907,"15min":12.907},"last":1.111},{"interface":"front","average":{"1min":1.27,"5min":1.3120000000000001,"15min":1.2969999999999999},"min":{"1min":0.89400000000000002,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.581,"5min":9.9909999999999997,"15min":9.9909999999999997},"last":1.391}]},{"osd":4,"last update":"Wed Nov 15 03:03:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.101,"15min":1.244},"min":{"1min":0.72199999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.3420000000000001,"5min":1.5469999999999999,"15min":2.5649999999999999},"last":1.421},{"interface":"front","average":{"1min":1.252,"5min":1.1679999999999999,"15min":1.4059999999999999},"min":{"1min":0.98899999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.6080000000000001,"5min":1.6679999999999999,"15min":4.226},"last":1.262}]},{"osd":5,"last update":"Wed Nov 15 03:03:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3,"15min":1.3340000000000001},"min":{"1min":1.0620000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.6950000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.286,"15min":1.423},"min":{"1min":1.046,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.1299999999999999,"5min":2.1299999999999999,"15min":2.4550000000000001},"last":1.3520000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:03:06 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.355,"15min":1.532},"min":{"1min":1.151,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6060000000000001,"5min":1.7490000000000001,"15min":3.2789999999999999},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.2949999999999999,"15min":1.546},"min":{"1min":1.0880000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.75,"5min":1.7969999999999999,"15min":4.7800000000000002},"last":0.999}]}]},{"osd":6,"up_from":14,"seq":60129542226,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298216,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073624,"statfs":{"total":96636764160,"available":96331390976,"internally_reserved":0,"allocated":1146880,"data_stored":523235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:02:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.0900000000000001,"15min":1.1220000000000001},"min":{"1min":0.73599999999999999,"5min":0.69799999999999995,"15min":0.44500000000000001},"max":{"1min":1.4379999999999999,"5min":1.7569999999999999,"15min":4.3159999999999998},"last":1.091},{"interface":"front","average":{"1min":1.111,"5min":1.087,"15min":1.0549999999999999},"min":{"1min":0.67700000000000005,"5min":0.61199999999999999,"15min":0.58599999999999997},"max":{"1min":1.5529999999999999,"5min":1.802,"15min":2.2480000000000002},"last":1.109}]},{"osd":1,"last update":"Wed Nov 15 03:02:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1299999999999999,"15min":1.097},"min":{"1min":0.70699999999999996,"5min":0.63200000000000001,"15min":0.57599999999999996},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.145,"15min":1.161},"min":{"1min":0.83499999999999996,"5min":0.74199999999999999,"15min":0.60699999999999998},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":2.8570000000000002},"last":0.98899999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:02:57 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.21,"15min":1.1859999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.49099999999999999},"max":{"1min":1.4179999999999999,"5min":1.7190000000000001,"15min":2.621},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.1739999999999999,"15min":1.1100000000000001},"min":{"1min":0.72099999999999997,"5min":0.69199999999999995,"15min":0.68300000000000005},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.1919999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:02:57 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.212,"15min":1.1719999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.50900000000000001},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":2.0630000000000002},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.26,"5min":1.2849999999999999,"15min":1.2709999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.67100000000000004},"max":{"1min":1.633,"5min":1.7569999999999999,"15min":2.8359999999999999},"last":1.075}]},{"osd":4,"last update":"Wed Nov 15 03:02:57 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.0960000000000001,"15min":1.1659999999999999},"min":{"1min":0.68999999999999995,"5min":0.63500000000000001,"15min":0.56000000000000005},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":4.5350000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.298,"5min":1.2030000000000001,"15min":1.1930000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.95299999999999996}]},{"osd":5,"last update":"Wed Nov 15 03:02:57 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.4359999999999999,"15min":1.417},"min":{"1min":0.75,"5min":0.75,"15min":0.72799999999999998},"max":{"1min":2.157,"5min":2.157,"15min":2.157},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.254,"15min":1.3},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.9460000000000002},"last":1.5660000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:02:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2010000000000001,"15min":1.2749999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.796,"5min":1.796,"15min":3.1989999999999998},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4159999999999999,"15min":1.4450000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.7470000000000001,"5min":1.8540000000000001,"15min":2.9350000000000001},"last":1.4379999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542227,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298204,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073636,"statfs":{"total":96636764160,"available":96331403264,"internally_reserved":0,"allocated":1134592,"data_stored":521767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.103,"15min":1.111},"min":{"1min":0.65400000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4530000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.089,"15min":1.03},"min":{"1min":0.78400000000000003,"5min":0.71099999999999997,"15min":0.66300000000000003},"max":{"1min":1.3540000000000001,"5min":1.677,"15min":1.677},"last":0.83899999999999997}]},{"osd":1,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.123,"15min":1.2030000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.575,"5min":2.073,"15min":2.617},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5269999999999999,"5min":1.657,"15min":1.657},"last":0.74199999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.216,"15min":1.1619999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5209999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.048},{"interface":"front","average":{"1min":1.145,"5min":1.1479999999999999,"15min":1.1599999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.403,"5min":1.667,"15min":1.667},"last":0.96699999999999997}]},{"osd":3,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.143,"15min":1.157},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.8660000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2609999999999999,"15min":1.2889999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.591,"5min":2.1869999999999998,"15min":2.5630000000000002},"last":1.0740000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.171,"15min":1.2290000000000001},"min":{"1min":0.60399999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.4930000000000001,"5min":1.7889999999999999,"15min":2.0049999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.181,"15min":1.2190000000000001},"min":{"1min":0.84699999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.411,"5min":1.994,"15min":1.994},"last":0.88800000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.2669999999999999,"15min":1.321},"min":{"1min":0.86499999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.776,"5min":1.913,"15min":2.6640000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.337,"5min":1.3240000000000001,"15min":1.3899999999999999},"min":{"1min":0.95299999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.679,"5min":1.871,"15min":2.173},"last":1.1559999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2889999999999999,"15min":1.355},"min":{"1min":0.96599999999999997,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.508,"5min":2.097,"15min":2.6030000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.371,"15min":1.4239999999999999},"min":{"1min":0.94599999999999995,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.859,"5min":2.2719999999999998,"15min":2.7669999999999999},"last":1.2050000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542227,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298204,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073636,"statfs":{"total":96636764160,"available":96331403264,"internally_reserved":0,"allocated":1134592,"data_stored":522085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83699999999999997,"15min":0.78000000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.879,"15min":0.84099999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.80200000000000005}]},{"osd":1,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.875,"15min":0.89300000000000002},"min":{"1min":0.44500000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.0860000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.377},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.0029999999999999,"15min":0.97699999999999998},"min":{"1min":0.59499999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.97899999999999998}]},{"osd":2,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0229999999999999,"15min":0.98299999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.53,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.093,"15min":1.0609999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.431,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.0640000000000001}]},{"osd":4,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.228,"15min":1.194},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.484,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.21},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2,"15min":1.22},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.5409999999999999,"5min":1.659,"15min":1.659},"last":1.464}]},{"osd":5,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2589999999999999,"15min":1.2629999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.548,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.266,"15min":1.264},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.1399999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3380000000000001,"15min":1.3140000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.6599999999999999,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.379,"15min":1.359},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.0259999999999998,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.2909999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3340000000000001,"15min":1.3700000000000001},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7929999999999999,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.425},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.3500000000000001,"15min":1.3520000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.587,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.518}]}]},{"osd":2,"up_from":14,"seq":60129542227,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298192,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073648,"statfs":{"total":96636764160,"available":96331415552,"internally_reserved":0,"allocated":1122304,"data_stored":519854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.78100000000000003,"15min":0.76900000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.266,"5min":1.409,"15min":1.409},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.83599999999999997,"15min":0.86499999999999999},"min":{"1min":0.47399999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.137,"5min":1.619,"15min":1.619},"last":0.71999999999999997}]},{"osd":1,"last update":"Wed Nov 15 03:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.93000000000000005,"15min":0.90400000000000003},"min":{"1min":0.60299999999999998,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2809999999999999,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.95999999999999996,"15min":0.94799999999999995},"min":{"1min":0.70199999999999996,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.3720000000000001,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.58499999999999996}]},{"osd":3,"last update":"Wed Nov 15 03:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.96599999999999997,"15min":0.98799999999999999},"min":{"1min":0.56999999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2609999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.032,"5min":1.121,"15min":1.119},"min":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.622},"max":{"1min":1.2130000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.61199999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:02:51 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.2150000000000001,"15min":1.212},"min":{"1min":0.88300000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.673,"5min":1.673,"15min":1.9410000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.234,"15min":1.228},"min":{"1min":1.024,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7829999999999999},"last":0.86899999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:02:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1819999999999999,"15min":1.238},"min":{"1min":0.94699999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.351,"5min":1.613,"15min":1.8939999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.286,"5min":1.274,"15min":1.2889999999999999},"min":{"1min":1.056,"5min":0.92600000000000005,"15min":0.90400000000000003},"max":{"1min":1.5780000000000001,"5min":1.702,"15min":1.857},"last":0.90200000000000002}]},{"osd":6,"last update":"Wed Nov 15 03:02:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.26,"15min":1.3380000000000001},"min":{"1min":1.0109999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.597,"5min":1.72,"15min":1.9379999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.252,"15min":1.3220000000000001},"min":{"1min":1.008,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.486,"5min":1.573,"15min":2.0870000000000002},"last":1.0309999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:02:51 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3240000000000001,"15min":1.3680000000000001},"min":{"1min":0.96199999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.6519999999999999,"5min":1.821,"15min":1.9570000000000001},"last":1.048},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.2669999999999999,"15min":1.355},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.4430000000000001,"5min":1.635,"15min":2.0350000000000001},"last":1.087}]}]},{"osd":0,"up_from":14,"seq":60129542227,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298216,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073624,"statfs":{"total":96636764160,"available":96331390976,"internally_reserved":0,"allocated":1146880,"data_stored":521961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.83099999999999996,"15min":0.86299999999999999},"min":{"1min":0.49099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.98999999999999999,"5min":4.3630000000000004,"15min":4.3630000000000004},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.98799999999999999,"15min":0.92600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.2450000000000001,"5min":6.157,"15min":6.157},"last":0.872}]},{"osd":2,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.996,"15min":1.014},"min":{"1min":0.63,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3440000000000001,"5min":4.1699999999999999,"15min":4.1699999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.056,"15min":1.0089999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.482,"5min":6.8369999999999997,"15min":6.8369999999999997},"last":1.1659999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.99099999999999999,"15min":0.97099999999999997},"min":{"1min":0.58099999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7050000000000001,"5min":3.5070000000000001,"15min":3.5070000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1279999999999999,"15min":1.165},"min":{"1min":0.746,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.4950000000000001,"5min":5.2149999999999999,"15min":5.2149999999999999},"last":0.91000000000000003}]},{"osd":4,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3160000000000001,"15min":1.27},"min":{"1min":0.95599999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.671,"5min":6.2000000000000002,"15min":6.2000000000000002},"last":1.232},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.22,"15min":1.1899999999999999},"min":{"1min":0.81699999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.357,"5min":3.577,"15min":3.577},"last":1.1379999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.286,"15min":1.2410000000000001},"min":{"1min":0.95799999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.8839999999999999,"5min":3.548,"15min":3.548},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.304,"15min":1.3460000000000001},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.675,"5min":4.4080000000000004,"15min":4.4080000000000004},"last":1.2}]},{"osd":6,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.3400000000000001,"15min":1.4059999999999999},"min":{"1min":0.97099999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.661,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":1.292},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.361,"15min":1.3520000000000001},"min":{"1min":0.82999999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.585,"5min":6.9690000000000003,"15min":6.9690000000000003},"last":1.2809999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:02:55 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.506,"15min":1.4259999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.738,"5min":18.253,"15min":18.253},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.4259999999999999,"15min":1.3149999999999999},"min":{"1min":0.96399999999999997,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.792,"5min":11.006,"15min":11.006},"last":1.0940000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574931,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298196,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073644,"statfs":{"total":96636764160,"available":96331411456,"internally_reserved":0,"allocated":1126400,"data_stored":521123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.099,"15min":1.0620000000000001},"min":{"1min":0.65600000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4319999999999999,"5min":1.5640000000000001,"15min":2.3370000000000002},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.05,"15min":1.1819999999999999},"min":{"1min":0.78400000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.369,"5min":1.4630000000000001,"15min":3.46},"last":0.90000000000000002}]},{"osd":1,"last update":"Wed Nov 15 03:02:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0900000000000001,"15min":1.212},"min":{"1min":0.80600000000000005,"5min":0.69399999999999995,"15min":0.626},"max":{"1min":1.478,"5min":1.478,"15min":3.6349999999999998},"last":1.196},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.135,"15min":1.175},"min":{"1min":0.89900000000000002,"5min":0.80300000000000005,"15min":0.65000000000000002},"max":{"1min":1.409,"5min":1.734,"15min":3.516},"last":1.135}]},{"osd":2,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.159,"15min":1.1970000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.73399999999999999},"max":{"1min":1.5489999999999999,"5min":1.7370000000000001,"15min":5.4180000000000001},"last":1.486},{"interface":"front","average":{"1min":1.123,"5min":1.177,"15min":1.2769999999999999},"min":{"1min":0.72699999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.454,"5min":1.7789999999999999,"15min":3.5859999999999999},"last":1.0660000000000001}]},{"osd":3,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1599999999999999,"15min":1.264},"min":{"1min":0.85799999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.417,"5min":1.5049999999999999,"15min":2.141},"last":1.325},{"interface":"front","average":{"1min":1.268,"5min":1.298,"15min":1.4139999999999999},"min":{"1min":1.083,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.5620000000000001,"5min":1.786,"15min":4.2729999999999997},"last":1.423}]},{"osd":4,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.244,"15min":1.3460000000000001},"min":{"1min":0.998,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.419,"5min":1.825,"15min":3.706},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.296,"5min":1.2649999999999999,"15min":1.5269999999999999},"min":{"1min":1.079,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.4910000000000001,"5min":1.7250000000000001,"15min":6.1180000000000003},"last":1.228}]},{"osd":6,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2430000000000001,"15min":1.3140000000000001},"min":{"1min":0.91400000000000003,"5min":0.749,"15min":0.749},"max":{"1min":1.593,"5min":1.794,"15min":4.2309999999999999},"last":1.292},{"interface":"front","average":{"1min":1.333,"5min":1.3109999999999999,"15min":1.508},"min":{"1min":1.1100000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.5209999999999999,"5min":1.6910000000000001,"15min":3.641},"last":1.1830000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.286,"15min":1.3899999999999999},"min":{"1min":1.0780000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.55,"5min":1.8380000000000001,"15min":3.516},"last":1.099},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.379,"15min":1.4379999999999999},"min":{"1min":1.0309999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.623,"5min":1.9339999999999999,"15min":3.754},"last":1.357}]}]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298184,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073656,"statfs":{"total":96636764160,"available":96331423744,"internally_reserved":0,"allocated":1114112,"data_stored":514419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:02:56 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.81100000000000005,"15min":0.83299999999999996},"min":{"1min":0.65200000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.94699999999999995,"5min":1.2150000000000001,"15min":1.2370000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.90600000000000003,"15min":0.88600000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.123,"5min":1.399,"15min":1.419},"last":0.72299999999999998}]},{"osd":2,"last update":"Wed Nov 15 03:02:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93300000000000005,"15min":0.873},"min":{"1min":0.71799999999999997,"5min":0.495,"15min":0.44600000000000001},"max":{"1min":1.2070000000000001,"5min":1.472,"15min":1.472},"last":1.054},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.94799999999999995,"15min":0.92400000000000004},"min":{"1min":0.629,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3740000000000001,"5min":1.887,"15min":1.887},"last":0.98999999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:02:56 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.161,"15min":1.087},"min":{"1min":0.77000000000000002,"5min":0.53300000000000003,"15min":0.503},"max":{"1min":1.5680000000000001,"5min":1.597,"15min":1.597},"last":1.24},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.081,"15min":1.0069999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.46000000000000002},"max":{"1min":1.3799999999999999,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.212}]},{"osd":4,"last update":"Wed Nov 15 03:02:56 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2070000000000001,"15min":1.1759999999999999},"min":{"1min":1.034,"5min":0.75700000000000001,"15min":0.65100000000000002},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.224,"15min":1.2509999999999999},"min":{"1min":1.0209999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.4630000000000001,"5min":1.5760000000000001,"15min":2.206},"last":1.3839999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:02:56 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2230000000000001,"15min":1.2529999999999999},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.746},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.899},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2589999999999999,"15min":1.2829999999999999},"min":{"1min":1.1000000000000001,"5min":0.92800000000000005,"15min":0.871},"max":{"1min":1.4850000000000001,"5min":1.764,"15min":2.4609999999999999},"last":1.1499999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:02:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3540000000000001,"15min":1.335},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.641,"5min":1.948,"15min":1.948},"last":1.609},{"interface":"front","average":{"1min":1.399,"5min":1.3979999999999999,"15min":1.4079999999999999},"min":{"1min":1.204,"5min":1.054,"15min":0.87},"max":{"1min":1.8100000000000001,"5min":1.9019999999999999,"15min":2.3290000000000002},"last":1.6000000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:02:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.3400000000000001,"15min":1.3240000000000001},"min":{"1min":0.95799999999999996,"5min":0.93999999999999995,"15min":0.84199999999999997},"max":{"1min":1.708,"5min":2.161,"15min":2.161},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.361,"15min":1.3620000000000001},"min":{"1min":1.1659999999999999,"5min":0.95499999999999996,"15min":0.78600000000000003},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":2.2349999999999999},"last":1.4099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T03:03:26.520 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-15T03:03:26.521 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-15T03:03:26.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.785+0000 7fb46e65b700 1 Processor -- start 2023-11-15T03:03:26.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.785+0000 7fb46e65b700 1 -- start start 2023-11-15T03:03:26.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb46e65b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb4681570b0 0x7fb468066860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:26.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb46e65b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4680c5c80 0x7fb468063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:26.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb46e65b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb468062da0 con 0x7fb468157490 2023-11-15T03:03:26.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb46e65b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4680bfb90 con 0x7fb4681570b0 2023-11-15T03:03:26.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb46e65b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680c6140 con 0x7fb4680c5c80 2023-11-15T03:03:26.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb4677fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb468157490 0x7fb468065950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45320/0 (socket says 172.21.15.121:45320) 2023-11-15T03:03:26.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb4677fe700 1 -- 172.21.15.121:0/2523764026 learned_addr learned my addr 172.21.15.121:0/2523764026 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:26.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb467fff700 1 --2- 172.21.15.121:0/2523764026 >> v2:172.21.15.190:3300/0 conn(0x7fb4681570b0 0x7fb468066860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:26.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb46cbf8700 1 --2- 172.21.15.121:0/2523764026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4680c5c80 0x7fb468063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:26.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb466ffd700 1 -- 172.21.15.121:0/2523764026 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746285591 0 0) 0x7fb468062da0 con 0x7fb468157490 2023-11-15T03:03:26.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb466ffd700 1 -- 172.21.15.121:0/2523764026 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4500033f0 con 0x7fb468157490 2023-11-15T03:03:26.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb466ffd700 1 -- 172.21.15.121:0/2523764026 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 28147361 0 0) 0x7fb4500033f0 con 0x7fb468157490 2023-11-15T03:03:26.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb466ffd700 1 -- 172.21.15.121:0/2523764026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4680c5c80 msgr2=0x7fb468063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:26.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb466ffd700 1 --2- 172.21.15.121:0/2523764026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4680c5c80 0x7fb468063230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:26.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb466ffd700 1 -- 172.21.15.121:0/2523764026 >> v2:172.21.15.190:3300/0 conn(0x7fb4681570b0 msgr2=0x7fb468066860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:26.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb466ffd700 1 --2- 172.21.15.121:0/2523764026 >> v2:172.21.15.190:3300/0 conn(0x7fb4681570b0 0x7fb468066860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:26.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb466ffd700 1 -- 172.21.15.121:0/2523764026 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4680c6420 con 0x7fb468157490 2023-11-15T03:03:26.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb466ffd700 1 -- 172.21.15.121:0/2523764026 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fb45c001470 con 0x7fb468157490 2023-11-15T03:03:26.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb46cbf8700 1 --2- 172.21.15.121:0/2523764026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4680c5c80 0x7fb468063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:03:26.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb466ffd700 1 -- 172.21.15.121:0/2523764026 >> v1:172.21.15.121:6789/0 conn(0x7fb468157490 legacy=0x7fb468065950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:26.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb466ffd700 1 --2- 172.21.15.121:0/2523764026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4500038e0 0x7fb450003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:26.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb466ffd700 1 --2- 172.21.15.121:0/2523764026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb450004690 0x7fb450004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:26.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb466ffd700 1 --2- 172.21.15.121:0/2523764026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4680c5c80 0x7fb450005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:26.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb466ffd700 1 -- 172.21.15.121:0/2523764026 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680bfb90 con 0x7fb4500038e0 2023-11-15T03:03:26.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb466ffd700 1 -- 172.21.15.121:0/2523764026 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680c6140 con 0x7fb450004690 2023-11-15T03:03:26.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb466ffd700 1 -- 172.21.15.121:0/2523764026 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45000b8e0 con 0x7fb4680c5c80 2023-11-15T03:03:26.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.789+0000 7fb467fff700 1 --2- 172.21.15.121:0/2523764026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4500038e0 0x7fb450003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:26.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.793+0000 7fb4677fe700 1 --2- 172.21.15.121:0/2523764026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb450004690 0x7fb450004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:26.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.793+0000 7fb46cbf8700 1 --2- 172.21.15.121:0/2523764026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4680c5c80 0x7fb450005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:26.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.793+0000 7fb467fff700 1 -- 172.21.15.121:0/2523764026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4680c5c80 msgr2=0x7fb450005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:26.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.793+0000 7fb467fff700 1 --2- 172.21.15.121:0/2523764026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4680c5c80 0x7fb450005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:26.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.793+0000 7fb467fff700 1 -- 172.21.15.121:0/2523764026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb450004690 msgr2=0x7fb450004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:26.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.793+0000 7fb467fff700 1 --2- 172.21.15.121:0/2523764026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb450004690 0x7fb450004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:26.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.793+0000 7fb467fff700 1 -- 172.21.15.121:0/2523764026 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb460009a20 con 0x7fb4500038e0 2023-11-15T03:03:26.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.793+0000 7fb467fff700 1 --2- 172.21.15.121:0/2523764026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4500038e0 0x7fb450003d00 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7fb458009a80 tx=0x7fb458009060).ready entity=mon.0 client_cookie=c1fabc51e0db7b85 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:26.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.793+0000 7fb466ffd700 1 -- 172.21.15.121:0/2523764026 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458009430 con 0x7fb4500038e0 2023-11-15T03:03:26.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.793+0000 7fb466ffd700 1 -- 172.21.15.121:0/2523764026 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45800e640 con 0x7fb4500038e0 2023-11-15T03:03:26.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.793+0000 7fb466ffd700 1 -- 172.21.15.121:0/2523764026 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45800e9c0 con 0x7fb4500038e0 2023-11-15T03:03:26.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.793+0000 7fb46e65b700 1 -- 172.21.15.121:0/2523764026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4500038e0 msgr2=0x7fb450003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:26.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.793+0000 7fb46e65b700 1 --2- 172.21.15.121:0/2523764026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4500038e0 0x7fb450003d00 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7fb458009a80 tx=0x7fb458009060).stop 2023-11-15T03:03:26.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.793+0000 7fb46cbf8700 1 -- 172.21.15.121:0/2523764026 reap_dead start 2023-11-15T03:03:26.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.793+0000 7fb46e65b700 1 -- 172.21.15.121:0/2523764026 shutdown_connections 2023-11-15T03:03:26.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.793+0000 7fb46e65b700 1 -- 172.21.15.121:0/2523764026 >> 172.21.15.121:0/2523764026 conn(0x7fb4680684f0 msgr2=0x7fb4680cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:26.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.793+0000 7fb46e65b700 1 -- 172.21.15.121:0/2523764026 shutdown_connections 2023-11-15T03:03:26.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.793+0000 7fb46e65b700 1 -- 172.21.15.121:0/2523764026 wait complete. 2023-11-15T03:03:26.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.797+0000 7fb46e65b700 1 Processor -- start 2023-11-15T03:03:26.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.797+0000 7fb46e65b700 1 -- start start 2023-11-15T03:03:26.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.797+0000 7fb46e65b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb468151fd0 0x7fb4680c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:26.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.797+0000 7fb46e65b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb468164c50 0x7fb4680c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:26.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.797+0000 7fb46e65b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4681523b0 0x7fb468154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:26.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.797+0000 7fb46e65b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680c33c0 con 0x7fb468164c50 2023-11-15T03:03:26.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.797+0000 7fb46e65b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680c0140 con 0x7fb4681523b0 2023-11-15T03:03:26.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.797+0000 7fb46e65b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680c09e0 con 0x7fb468151fd0 2023-11-15T03:03:26.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.797+0000 7fb4677fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb468164c50 0x7fb4680c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:26.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.797+0000 7fb4677fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb468164c50 0x7fb4680c2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49710/0 (socket says 172.21.15.121:49710) 2023-11-15T03:03:26.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.797+0000 7fb4677fe700 1 -- 172.21.15.121:0/3074100732 learned_addr learned my addr 172.21.15.121:0/3074100732 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:26.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.801+0000 7fb467fff700 1 --2- 172.21.15.121:0/3074100732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb468151fd0 0x7fb4680c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:26.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.801+0000 7fb46cbf8700 1 --2- 172.21.15.121:0/3074100732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4681523b0 0x7fb468154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:26.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.801+0000 7fb4677fe700 1 -- 172.21.15.121:0/3074100732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb468151fd0 msgr2=0x7fb4680c29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:26.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.801+0000 7fb4677fe700 1 --2- 172.21.15.121:0/3074100732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb468151fd0 0x7fb4680c29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:26.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.801+0000 7fb4677fe700 1 -- 172.21.15.121:0/3074100732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4681523b0 msgr2=0x7fb468154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:26.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.801+0000 7fb4677fe700 1 --2- 172.21.15.121:0/3074100732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4681523b0 0x7fb468154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:26.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.801+0000 7fb4677fe700 1 -- 172.21.15.121:0/3074100732 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb468155cf0 con 0x7fb468164c50 2023-11-15T03:03:26.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.801+0000 7fb4677fe700 1 --2- 172.21.15.121:0/3074100732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb468164c50 0x7fb4680c2ed0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7fb45c007fa0 tx=0x7fb45c001e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:26.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.801+0000 7fb44ffff700 1 -- 172.21.15.121:0/3074100732 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0114c0 con 0x7fb468164c50 2023-11-15T03:03:26.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.801+0000 7fb44ffff700 1 -- 172.21.15.121:0/3074100732 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c011620 con 0x7fb468164c50 2023-11-15T03:03:26.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.801+0000 7fb46e65b700 1 -- 172.21.15.121:0/3074100732 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4680c0390 con 0x7fb468164c50 2023-11-15T03:03:26.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.801+0000 7fb44ffff700 1 -- 172.21.15.121:0/3074100732 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0119a0 con 0x7fb468164c50 2023-11-15T03:03:26.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.801+0000 7fb46e65b700 1 -- 172.21.15.121:0/3074100732 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb468155440 con 0x7fb468164c50 2023-11-15T03:03:26.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.801+0000 7fb46e65b700 1 -- 172.21.15.121:0/3074100732 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46806c150 con 0x7fb468164c50 2023-11-15T03:03:26.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.805+0000 7fb44ffff700 1 -- 172.21.15.121:0/3074100732 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb45c011d50 con 0x7fb468164c50 2023-11-15T03:03:26.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.805+0000 7fb44ffff700 1 --2- 172.21.15.121:0/3074100732 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb45405a510 0x7fb45405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:26.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.805+0000 7fb44ffff700 1 -- 172.21.15.121:0/3074100732 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb45c073f00 con 0x7fb468164c50 2023-11-15T03:03:26.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.805+0000 7fb467fff700 1 --2- 172.21.15.121:0/3074100732 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb45405a510 0x7fb45405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:26.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.805+0000 7fb467fff700 1 --2- 172.21.15.121:0/3074100732 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb45405a510 0x7fb45405c9d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7fb468064f60 tx=0x7fb45801a000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:26.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:26.809+0000 7fb44ffff700 1 -- 172.21.15.121:0/3074100732 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb45c047730 con 0x7fb468164c50 2023-11-15T03:03:27.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.029+0000 7fb46e65b700 1 -- 172.21.15.121:0/3074100732 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb4680a9ea0 con 0x7fb468164c50 2023-11-15T03:03:27.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.037+0000 7fb44ffff700 1 -- 172.21.15.121:0/3074100732 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7fb45c019020 con 0x7fb468164c50 2023-11-15T03:03:27.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.041+0000 7fb46e65b700 1 -- 172.21.15.121:0/3074100732 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb45405a510 msgr2=0x7fb45405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:27.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.041+0000 7fb46e65b700 1 --2- 172.21.15.121:0/3074100732 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb45405a510 0x7fb45405c9d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7fb468064f60 tx=0x7fb45801a000).stop 2023-11-15T03:03:27.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.041+0000 7fb46e65b700 1 -- 172.21.15.121:0/3074100732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb468164c50 msgr2=0x7fb4680c2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:27.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.041+0000 7fb46e65b700 1 --2- 172.21.15.121:0/3074100732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb468164c50 0x7fb4680c2ed0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7fb45c007fa0 tx=0x7fb45c001e90).stop 2023-11-15T03:03:27.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.041+0000 7fb46e65b700 1 -- 172.21.15.121:0/3074100732 shutdown_connections 2023-11-15T03:03:27.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.041+0000 7fb46e65b700 1 --2- 172.21.15.121:0/3074100732 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb45405a510 0x7fb45405c9d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:27.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.041+0000 7fb46e65b700 1 --2- 172.21.15.121:0/3074100732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4681523b0 0x7fb468154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:27.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.041+0000 7fb46e65b700 1 --2- 172.21.15.121:0/3074100732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb468164c50 0x7fb4680c2ed0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:27.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.041+0000 7fb46e65b700 1 --2- 172.21.15.121:0/3074100732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb468151fd0 0x7fb4680c29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:27.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.041+0000 7fb46e65b700 1 -- 172.21.15.121:0/3074100732 >> 172.21.15.121:0/3074100732 conn(0x7fb4680684f0 msgr2=0x7fb4680afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:27.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.041+0000 7fb46e65b700 1 -- 172.21.15.121:0/3074100732 shutdown_connections 2023-11-15T03:03:27.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.041+0000 7fb46e65b700 1 -- 172.21.15.121:0/3074100732 wait complete. 2023-11-15T03:03:27.051 INFO:teuthology.orchestra.run.smithi121.stderr:updated caps for client.0 2023-11-15T03:03:27.068 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:03:27.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.225+0000 7f4d16edb700 1 Processor -- start 2023-11-15T03:03:27.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.225+0000 7f4d16edb700 1 -- start start 2023-11-15T03:03:27.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.225+0000 7f4d16edb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d10154ca0 0x7f4d10155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:27.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.225+0000 7f4d16edb700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4d101562f0 0x7f4d100a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:27.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.225+0000 7f4d16edb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4d100a8f40 0x7f4d100a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:27.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.225+0000 7f4d16edb700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d100a8280 con 0x7f4d101562f0 2023-11-15T03:03:27.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.225+0000 7f4d16edb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d100a6d40 con 0x7f4d100a8f40 2023-11-15T03:03:27.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.225+0000 7f4d16edb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100c4c30 con 0x7f4d10154ca0 2023-11-15T03:03:27.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.225+0000 7f4d07fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4d101562f0 0x7f4d100a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:27.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.225+0000 7f4d07fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4d101562f0 0x7f4d100a8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49718/0 (socket says 172.21.15.121:49718) 2023-11-15T03:03:27.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.225+0000 7f4d07fff700 1 -- 172.21.15.121:0/2985294680 learned_addr learned my addr 172.21.15.121:0/2985294680 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:27.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.225+0000 7f4d14c77700 1 --2- 172.21.15.121:0/2985294680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d10154ca0 0x7f4d10155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:27.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.225+0000 7f4d15478700 1 --2- 172.21.15.121:0/2985294680 >> v2:172.21.15.190:3300/0 conn(0x7f4d100a8f40 0x7f4d100a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:27.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.229+0000 7f4d07fff700 1 -- 172.21.15.121:0/2985294680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d10154ca0 msgr2=0x7f4d10155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:27.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.229+0000 7f4d07fff700 1 --2- 172.21.15.121:0/2985294680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d10154ca0 0x7f4d10155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:27.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.229+0000 7f4d07fff700 1 -- 172.21.15.121:0/2985294680 >> v2:172.21.15.190:3300/0 conn(0x7f4d100a8f40 msgr2=0x7f4d100a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:27.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.229+0000 7f4d07fff700 1 --2- 172.21.15.121:0/2985294680 >> v2:172.21.15.190:3300/0 conn(0x7f4d100a8f40 0x7f4d100a57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:27.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.229+0000 7f4d07fff700 1 -- 172.21.15.121:0/2985294680 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d100a6f50 con 0x7f4d101562f0 2023-11-15T03:03:27.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.229+0000 7f4d07fff700 1 --2- 172.21.15.121:0/2985294680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d101562f0 0x7f4d100a8a50 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f4d100ac9f0 tx=0x7f4d000090f0).ready entity=mon.0 client_cookie=efb4a430ba400f88 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:27.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.229+0000 7f4d077fe700 1 -- 172.21.15.121:0/2985294680 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0000b520 con 0x7f4d101562f0 2023-11-15T03:03:27.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.229+0000 7f4d077fe700 1 -- 172.21.15.121:0/2985294680 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0000b680 con 0x7f4d101562f0 2023-11-15T03:03:27.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.229+0000 7f4d077fe700 1 -- 172.21.15.121:0/2985294680 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0000ba00 con 0x7f4d101562f0 2023-11-15T03:03:27.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.229+0000 7f4d16edb700 1 -- 172.21.15.121:0/2985294680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d101562f0 msgr2=0x7f4d100a8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:27.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.229+0000 7f4d16edb700 1 --2- 172.21.15.121:0/2985294680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d101562f0 0x7f4d100a8a50 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f4d100ac9f0 tx=0x7f4d000090f0).stop 2023-11-15T03:03:27.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.229+0000 7f4d16edb700 1 -- 172.21.15.121:0/2985294680 shutdown_connections 2023-11-15T03:03:27.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.229+0000 7f4d16edb700 1 --2- 172.21.15.121:0/2985294680 >> v2:172.21.15.190:3300/0 conn(0x7f4d100a8f40 0x7f4d100a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:27.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.229+0000 7f4d16edb700 1 --2- 172.21.15.121:0/2985294680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d101562f0 0x7f4d100a8a50 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:27.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.229+0000 7f4d16edb700 1 --2- 172.21.15.121:0/2985294680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d10154ca0 0x7f4d10155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:27.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.229+0000 7f4d16edb700 1 -- 172.21.15.121:0/2985294680 >> 172.21.15.121:0/2985294680 conn(0x7f4d10062190 msgr2=0x7f4d100625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:27.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.229+0000 7f4d16edb700 1 -- 172.21.15.121:0/2985294680 shutdown_connections 2023-11-15T03:03:27.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.229+0000 7f4d16edb700 1 -- 172.21.15.121:0/2985294680 wait complete. 2023-11-15T03:03:27.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d16edb700 1 Processor -- start 2023-11-15T03:03:27.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d16edb700 1 -- start start 2023-11-15T03:03:27.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d16edb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d100a8f40 0x7f4d1014c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:27.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d16edb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d10154ca0 0x7f4d100c1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:27.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d16edb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d101562f0 0x7f4d10151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:27.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d16edb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100aa880 con 0x7f4d100a8f40 2023-11-15T03:03:27.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d16edb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100aa580 con 0x7f4d101562f0 2023-11-15T03:03:27.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d16edb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100c2070 con 0x7f4d10154ca0 2023-11-15T03:03:27.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d14c77700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d100a8f40 0x7f4d1014c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:27.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d14c77700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d100a8f40 0x7f4d1014c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49728/0 (socket says 172.21.15.121:49728) 2023-11-15T03:03:27.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d14c77700 1 -- 172.21.15.121:0/513852815 learned_addr learned my addr 172.21.15.121:0/513852815 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:27.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d07fff700 1 --2- 172.21.15.121:0/513852815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d10154ca0 0x7f4d100c1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:27.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d15478700 1 --2- 172.21.15.121:0/513852815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d101562f0 0x7f4d10151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:27.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d14c77700 1 -- 172.21.15.121:0/513852815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d10154ca0 msgr2=0x7f4d100c1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:27.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d14c77700 1 --2- 172.21.15.121:0/513852815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d10154ca0 0x7f4d100c1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:27.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d14c77700 1 -- 172.21.15.121:0/513852815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d101562f0 msgr2=0x7f4d10151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:27.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d14c77700 1 --2- 172.21.15.121:0/513852815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d101562f0 0x7f4d10151630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:27.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d14c77700 1 -- 172.21.15.121:0/513852815 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d00009370 con 0x7f4d100a8f40 2023-11-15T03:03:27.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d15478700 1 --2- 172.21.15.121:0/513852815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d101562f0 0x7f4d10151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:27.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d14c77700 1 --2- 172.21.15.121:0/513852815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d100a8f40 0x7f4d1014c560 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f4d0c00e950 tx=0x7f4d0c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:27.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d04ff9700 1 -- 172.21.15.121:0/513852815 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c00c850 con 0x7f4d100a8f40 2023-11-15T03:03:27.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d04ff9700 1 -- 172.21.15.121:0/513852815 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c00c9b0 con 0x7f4d100a8f40 2023-11-15T03:03:27.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d04ff9700 1 -- 172.21.15.121:0/513852815 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c00cd30 con 0x7f4d100a8f40 2023-11-15T03:03:27.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.233+0000 7f4d16edb700 1 -- 172.21.15.121:0/513852815 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d100c2350 con 0x7f4d100a8f40 2023-11-15T03:03:27.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.237+0000 7f4d16edb700 1 -- 172.21.15.121:0/513852815 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1014ca50 con 0x7f4d100a8f40 2023-11-15T03:03:27.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.237+0000 7f4d16edb700 1 -- 172.21.15.121:0/513852815 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1006c150 con 0x7f4d100a8f40 2023-11-15T03:03:27.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.237+0000 7f4d04ff9700 1 -- 172.21.15.121:0/513852815 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4d0c012600 con 0x7f4d100a8f40 2023-11-15T03:03:27.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.241+0000 7f4d04ff9700 1 --2- 172.21.15.121:0/513852815 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4cfc05a4e0 0x7f4cfc05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:27.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.241+0000 7f4d04ff9700 1 -- 172.21.15.121:0/513852815 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4d0c074dc0 con 0x7f4d100a8f40 2023-11-15T03:03:27.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.241+0000 7f4d07fff700 1 --2- 172.21.15.121:0/513852815 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4cfc05a4e0 0x7f4cfc05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:27.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.241+0000 7f4d07fff700 1 --2- 172.21.15.121:0/513852815 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4cfc05a4e0 0x7f4cfc05c9a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f4d100ac9f0 tx=0x7f4d00017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:27.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.245+0000 7f4d04ff9700 1 -- 172.21.15.121:0/513852815 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4d0c0485a0 con 0x7f4d100a8f40 2023-11-15T03:03:27.298 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:03:27.293+0000 7f27100f8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:03:27.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.441+0000 7f4d16edb700 1 -- 172.21.15.121:0/513852815 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d100a7230 con 0x7f4d100a8f40 2023-11-15T03:03:27.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.441+0000 7f4d04ff9700 1 -- 172.21.15.121:0/513852815 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4500 (secure 0 0 0) 0x7f4d0c016020 con 0x7f4d100a8f40 2023-11-15T03:03:27.448 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:27.448 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":81,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:03:24.804563+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6966},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6966":{"gid":6966,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2646177140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2646177140},{"type":"v1","addr":"172.21.15.121:6836","nonce":2646177140}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:03:27.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.445+0000 7f4d16edb700 1 -- 172.21.15.121:0/513852815 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4cfc05a4e0 msgr2=0x7f4cfc05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:27.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.445+0000 7f4d16edb700 1 --2- 172.21.15.121:0/513852815 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4cfc05a4e0 0x7f4cfc05c9a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f4d100ac9f0 tx=0x7f4d00017000).stop 2023-11-15T03:03:27.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.445+0000 7f4d16edb700 1 -- 172.21.15.121:0/513852815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d100a8f40 msgr2=0x7f4d1014c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:27.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.445+0000 7f4d16edb700 1 --2- 172.21.15.121:0/513852815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d100a8f40 0x7f4d1014c560 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f4d0c00e950 tx=0x7f4d0c009060).stop 2023-11-15T03:03:27.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.445+0000 7f4d16edb700 1 -- 172.21.15.121:0/513852815 shutdown_connections 2023-11-15T03:03:27.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.445+0000 7f4d16edb700 1 --2- 172.21.15.121:0/513852815 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4cfc05a4e0 0x7f4cfc05c9a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:27.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.445+0000 7f4d16edb700 1 --2- 172.21.15.121:0/513852815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d101562f0 0x7f4d10151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:27.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.445+0000 7f4d16edb700 1 --2- 172.21.15.121:0/513852815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d10154ca0 0x7f4d100c1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:27.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.445+0000 7f4d16edb700 1 --2- 172.21.15.121:0/513852815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d100a8f40 0x7f4d1014c560 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:27.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.445+0000 7f4d16edb700 1 -- 172.21.15.121:0/513852815 >> 172.21.15.121:0/513852815 conn(0x7f4d10062190 msgr2=0x7f4d1014e390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:27.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.445+0000 7f4d16edb700 1 -- 172.21.15.121:0/513852815 shutdown_connections 2023-11-15T03:03:27.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:27.445+0000 7f4d16edb700 1 -- 172.21.15.121:0/513852815 wait complete. 2023-11-15T03:03:27.452 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 81 2023-11-15T03:03:27.872 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:03:24.804563+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6966}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6966': {'gid': 6966, 'name': 'a', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.121:6836/2646177140', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 2646177140}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 2646177140}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:03:27.873 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T03:03:27.874 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-15T03:03:28.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.041+0000 7f3a72907700 1 Processor -- start 2023-11-15T03:03:28.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.041+0000 7f3a72907700 1 -- start start 2023-11-15T03:03:28.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.041+0000 7f3a72907700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3a6c0bc790 0x7f3a6c0bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.041+0000 7f3a72907700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3a6c069730 0x7f3a6c0a4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.041+0000 7f3a72907700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a6c0c6060 con 0x7f3a6c0c64f0 2023-11-15T03:03:28.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.041+0000 7f3a72907700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6c0a4950 con 0x7f3a6c0bc790 2023-11-15T03:03:28.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.041+0000 7f3a72907700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6c0c5730 con 0x7f3a6c069730 2023-11-15T03:03:28.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.041+0000 7f3a71905700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3a6c0bc790 0x7f3a6c0bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.041+0000 7f3a71905700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3a6c0bc790 0x7f3a6c0bcb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49736/0 (socket says 172.21.15.121:49736) 2023-11-15T03:03:28.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.041+0000 7f3a71905700 1 -- 172.21.15.121:0/2321437979 learned_addr learned my addr 172.21.15.121:0/2321437979 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:28.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.041+0000 7f3a71104700 1 --2- 172.21.15.121:0/2321437979 >> v2:172.21.15.190:3300/0 conn(0x7f3a6c069730 0x7f3a6c0a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.041+0000 7f3a71905700 1 -- 172.21.15.121:0/2321437979 >> v2:172.21.15.190:3300/0 conn(0x7f3a6c069730 msgr2=0x7f3a6c0a4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:28.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.041+0000 7f3a71905700 1 --2- 172.21.15.121:0/2321437979 >> v2:172.21.15.190:3300/0 conn(0x7f3a6c069730 0x7f3a6c0a4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.041+0000 7f3a71905700 1 -- 172.21.15.121:0/2321437979 >> v1:172.21.15.190:6789/0 conn(0x7f3a6c0c64f0 legacy=0x7f3a6c0c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:28.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.041+0000 7f3a71905700 1 -- 172.21.15.121:0/2321437979 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c0c5a10 con 0x7f3a6c0bc790 2023-11-15T03:03:28.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.041+0000 7f3a71104700 1 --2- 172.21.15.121:0/2321437979 >> v2:172.21.15.190:3300/0 conn(0x7f3a6c069730 0x7f3a6c0a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:28.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.045+0000 7f3a71905700 1 --2- 172.21.15.121:0/2321437979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a6c0bc790 0x7f3a6c0bcb70 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7f3a6c066d10 tx=0x7f3a680090f0).ready entity=mon.0 client_cookie=c2c7fc99a414e928 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:28.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.045+0000 7f3a70903700 1 -- 172.21.15.121:0/2321437979 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6800b600 con 0x7f3a6c0bc790 2023-11-15T03:03:28.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.045+0000 7f3a70903700 1 -- 172.21.15.121:0/2321437979 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6800b760 con 0x7f3a6c0bc790 2023-11-15T03:03:28.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.045+0000 7f3a70903700 1 -- 172.21.15.121:0/2321437979 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6800bae0 con 0x7f3a6c0bc790 2023-11-15T03:03:28.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.045+0000 7f3a72907700 1 -- 172.21.15.121:0/2321437979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a6c0bc790 msgr2=0x7f3a6c0bcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:28.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.045+0000 7f3a72907700 1 --2- 172.21.15.121:0/2321437979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a6c0bc790 0x7f3a6c0bcb70 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7f3a6c066d10 tx=0x7f3a680090f0).stop 2023-11-15T03:03:28.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.045+0000 7f3a72907700 1 -- 172.21.15.121:0/2321437979 shutdown_connections 2023-11-15T03:03:28.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.045+0000 7f3a72907700 1 --2- 172.21.15.121:0/2321437979 >> v2:172.21.15.190:3300/0 conn(0x7f3a6c069730 0x7f3a6c0a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.045+0000 7f3a72907700 1 --2- 172.21.15.121:0/2321437979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a6c0bc790 0x7f3a6c0bcb70 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.045+0000 7f3a72907700 1 -- 172.21.15.121:0/2321437979 >> 172.21.15.121:0/2321437979 conn(0x7f3a6c069350 msgr2=0x7f3a6c0c7690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:28.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.045+0000 7f3a72907700 1 -- 172.21.15.121:0/2321437979 shutdown_connections 2023-11-15T03:03:28.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.045+0000 7f3a72907700 1 -- 172.21.15.121:0/2321437979 wait complete. 2023-11-15T03:03:28.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a72907700 1 Processor -- start 2023-11-15T03:03:28.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a72907700 1 -- start start 2023-11-15T03:03:28.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a72907700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a6c0bc790 0x7f3a6c0c0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a72907700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a6c0c64f0 0x7f3a6c0ac5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a72907700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a6c0be730 0x7f3a6c0bebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a72907700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0c0f30 con 0x7f3a6c0bc790 2023-11-15T03:03:28.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a72907700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0c1210 con 0x7f3a6c0c64f0 2023-11-15T03:03:28.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a72907700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0c14f0 con 0x7f3a6c0be730 2023-11-15T03:03:28.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a71905700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a6c0bc790 0x7f3a6c0c0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a71905700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a6c0bc790 0x7f3a6c0c0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49738/0 (socket says 172.21.15.121:49738) 2023-11-15T03:03:28.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a71905700 1 -- 172.21.15.121:0/1500910440 learned_addr learned my addr 172.21.15.121:0/1500910440 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:28.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a71104700 1 --2- 172.21.15.121:0/1500910440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a6c0c64f0 0x7f3a6c0ac5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a72106700 1 --2- 172.21.15.121:0/1500910440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a6c0be730 0x7f3a6c0bebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a71905700 1 -- 172.21.15.121:0/1500910440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a6c0be730 msgr2=0x7f3a6c0bebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:28.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a71905700 1 --2- 172.21.15.121:0/1500910440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a6c0be730 0x7f3a6c0bebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a71905700 1 -- 172.21.15.121:0/1500910440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a6c0c64f0 msgr2=0x7f3a6c0ac5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:28.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a71905700 1 --2- 172.21.15.121:0/1500910440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a6c0c64f0 0x7f3a6c0ac5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a71905700 1 -- 172.21.15.121:0/1500910440 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a68009370 con 0x7f3a6c0bc790 2023-11-15T03:03:28.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a71905700 1 --2- 172.21.15.121:0/1500910440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a6c0bc790 0x7f3a6c0c0870 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f3a6c06c3e0 tx=0x7f3a68017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:28.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a61ffb700 1 -- 172.21.15.121:0/1500910440 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6800b8d0 con 0x7f3a6c0bc790 2023-11-15T03:03:28.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a61ffb700 1 -- 172.21.15.121:0/1500910440 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6800ba50 con 0x7f3a6c0bc790 2023-11-15T03:03:28.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.049+0000 7f3a72907700 1 -- 172.21.15.121:0/1500910440 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6c0c17d0 con 0x7f3a6c0bc790 2023-11-15T03:03:28.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.053+0000 7f3a61ffb700 1 -- 172.21.15.121:0/1500910440 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6801b660 con 0x7f3a6c0bc790 2023-11-15T03:03:28.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.053+0000 7f3a72907700 1 -- 172.21.15.121:0/1500910440 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6c0acaa0 con 0x7f3a6c0bc790 2023-11-15T03:03:28.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.053+0000 7f3a61ffb700 1 -- 172.21.15.121:0/1500910440 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3a6801b9d0 con 0x7f3a6c0bc790 2023-11-15T03:03:28.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.057+0000 7f3a61ffb700 1 --2- 172.21.15.121:0/1500910440 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3a5005a540 0x7f3a5005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.057+0000 7f3a61ffb700 1 -- 172.21.15.121:0/1500910440 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3a680761e0 con 0x7f3a6c0bc790 2023-11-15T03:03:28.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.057+0000 7f3a71104700 1 --2- 172.21.15.121:0/1500910440 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3a5005a540 0x7f3a5005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.057+0000 7f3a71104700 1 --2- 172.21.15.121:0/1500910440 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3a5005a540 0x7f3a5005ca00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f3a6c14e3b0 tx=0x7f3a5c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:28.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.057+0000 7f3a7df05740 1 Processor -- start 2023-11-15T03:03:28.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.057+0000 7f3a7df05740 1 -- start start 2023-11-15T03:03:28.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a7df05740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 0x2662cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a7df05740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2663b20 0x26a8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a7df05740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x26a90a0 0x26a9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a7df05740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x25e3410 con 0x2663b20 2023-11-15T03:03:28.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a7df05740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x25e3550 con 0x26a90a0 2023-11-15T03:03:28.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a7df05740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x25e3a00 con 0x26628d0 2023-11-15T03:03:28.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a71104700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2663b20 0x26a8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a71104700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2663b20 0x26a8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49754/0 (socket says 172.21.15.121:49754) 2023-11-15T03:03:28.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a71104700 1 -- 172.21.15.121:0/2587972563 learned_addr learned my addr 172.21.15.121:0/2587972563 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:28.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a71905700 1 --2- 172.21.15.121:0/2587972563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 0x2662cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a72106700 1 --2- 172.21.15.121:0/2587972563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x26a90a0 0x26a9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a71104700 1 -- 172.21.15.121:0/2587972563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 msgr2=0x2662cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:28.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a71104700 1 --2- 172.21.15.121:0/2587972563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 0x2662cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a71104700 1 -- 172.21.15.121:0/2587972563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x26a90a0 msgr2=0x26a9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:28.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a71104700 1 --2- 172.21.15.121:0/2587972563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x26a90a0 0x26a9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a71104700 1 -- 172.21.15.121:0/2587972563 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x25dc730 con 0x2663b20 2023-11-15T03:03:28.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a71104700 1 --2- 172.21.15.121:0/2587972563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2663b20 0x26a8bb0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f3a5c010a30 tx=0x7f3a5c015000).ready entity=mon.0 client_cookie=62bc8650833bb24b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:28.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a56ffd700 1 -- 172.21.15.121:0/2587972563 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c014900 con 0x2663b20 2023-11-15T03:03:28.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a56ffd700 1 -- 172.21.15.121:0/2587972563 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5c014a60 con 0x2663b20 2023-11-15T03:03:28.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a56ffd700 1 -- 172.21.15.121:0/2587972563 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c014de0 con 0x2663b20 2023-11-15T03:03:28.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a7df05740 1 -- 172.21.15.121:0/2587972563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2663b20 msgr2=0x26a8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:28.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a7df05740 1 --2- 172.21.15.121:0/2587972563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2663b20 0x26a8bb0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f3a5c010a30 tx=0x7f3a5c015000).stop 2023-11-15T03:03:28.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a7df05740 1 -- 172.21.15.121:0/2587972563 shutdown_connections 2023-11-15T03:03:28.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a7df05740 1 --2- 172.21.15.121:0/2587972563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x26a90a0 0x26a9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a7df05740 1 --2- 172.21.15.121:0/2587972563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2663b20 0x26a8bb0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a7df05740 1 --2- 172.21.15.121:0/2587972563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 0x2662cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.061+0000 7f3a7df05740 1 -- 172.21.15.121:0/2587972563 >> 172.21.15.121:0/2587972563 conn(0x2441550 msgr2=0x26a9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:28.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a7df05740 1 -- 172.21.15.121:0/2587972563 shutdown_connections 2023-11-15T03:03:28.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a7df05740 1 -- 172.21.15.121:0/2587972563 wait complete. 2023-11-15T03:03:28.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a7df05740 1 -- start start 2023-11-15T03:03:28.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a7df05740 1 Processor -- start 2023-11-15T03:03:28.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a7df05740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 0x26a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a7df05740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2663b20 0x268bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a7df05740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 0x268d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a7df05740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x25ae650 con 0x26a90a0 2023-11-15T03:03:28.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a7df05740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x268f570 con 0x2663b20 2023-11-15T03:03:28.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a7df05740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x26a3a60 con 0x26628d0 2023-11-15T03:03:28.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a7df05740 10 client.0 ll_register_callbacks cb 0x261b020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:03:28.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a72106700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 0x268d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a72106700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 0x268d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49756/0 (socket says 172.21.15.121:49756) 2023-11-15T03:03:28.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a72106700 1 -- 172.21.15.121:0/2165272698 learned_addr learned my addr 172.21.15.121:0/2165272698 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:28.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a71104700 1 --2- 172.21.15.121:0/2165272698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2663b20 0x268bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a71905700 1 --2- 172.21.15.121:0/2165272698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 0x26a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a72106700 1 -- 172.21.15.121:0/2165272698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 msgr2=0x26a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:28.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a72106700 1 --2- 172.21.15.121:0/2165272698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 0x26a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a72106700 1 -- 172.21.15.121:0/2165272698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2663b20 msgr2=0x268bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:28.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a72106700 1 --2- 172.21.15.121:0/2165272698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2663b20 0x268bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.065+0000 7f3a72106700 1 -- 172.21.15.121:0/2165272698 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5c00e040 con 0x26a90a0 2023-11-15T03:03:28.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a72106700 1 --2- 172.21.15.121:0/2165272698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 0x268d5b0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7f3a640115b0 tx=0x7f3a64009040).ready entity=mon.0 client_cookie=7432ca5214383123 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:28.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a4bfff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:03:28.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a4bfff700 1 -- 172.21.15.121:0/2165272698 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64003840 con 0x26a90a0 2023-11-15T03:03:28.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a4bfff700 1 -- 172.21.15.121:0/2165272698 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3a64011e80 con 0x26a90a0 2023-11-15T03:03:28.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a7df05740 1 -- 172.21.15.121:0/2165272698 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x25d91a0 con 0x26a90a0 2023-11-15T03:03:28.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a4bfff700 1 -- 172.21.15.121:0/2165272698 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a64001780 con 0x26a90a0 2023-11-15T03:03:28.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a7df05740 10 client.7116 fetch_fsmap learned FSMap version 81 2023-11-15T03:03:28.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a7df05740 1 -- 172.21.15.121:0/2165272698 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x25d7960 con 0x26a90a0 2023-11-15T03:03:28.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a4bfff700 1 -- 172.21.15.121:0/2165272698 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x25d91a0 con 0x26a90a0 2023-11-15T03:03:28.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a4bfff700 1 -- 172.21.15.121:0/2165272698 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 81) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3a64006a20 con 0x26a90a0 2023-11-15T03:03:28.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a7df05740 10 client.7116 fetch_fsmap finished waiting for FSMap version 81 2023-11-15T03:03:28.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a7df05740 10 client.7116 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:03:28.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a7df05740 20 client.7116 populate_metadata read hostname 'smithi121' 2023-11-15T03:03:28.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a7df05740 1 --2- 172.21.15.121:0/2165272698 >> [v2:172.21.15.121:6834/2646177140,v1:172.21.15.121:6836/2646177140] conn(0x268e800 0x268ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a7df05740 4 client.7116 mds_command: new command op to 6966 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:03:28.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a7df05740 1 -- 172.21.15.121:0/2165272698 --> [v2:172.21.15.121:6834/2646177140,v1:172.21.15.121:6836/2646177140] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x25cf6f0 con 0x268e800 2023-11-15T03:03:28.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a71905700 1 --2- 172.21.15.121:0/2165272698 >> [v2:172.21.15.121:6834/2646177140,v1:172.21.15.121:6836/2646177140] conn(0x268e800 0x268ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a71905700 1 --2- 172.21.15.121:0/2165272698 >> [v2:172.21.15.121:6834/2646177140,v1:172.21.15.121:6836/2646177140] conn(0x268e800 0x268ebe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=91981bb96c2a5102 server_cookie=e238bf116e30a818 in_seq=0 out_seq=0 2023-11-15T03:03:28.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a4bfff700 10 client.7116 ms_handle_connect on v2:172.21.15.121:6834/2646177140 2023-11-15T03:03:28.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a4bfff700 1 -- 172.21.15.121:0/2165272698 <== mds.0 v2:172.21.15.121:6834/2646177140 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x25cf6f0 con 0x268e800 2023-11-15T03:03:28.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a4bfff700 10 client.7116 handle_command_reply: tid=0 2023-11-15T03:03:28.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.069+0000 7f3a7df05740 1 client.7116 shutdown 2023-11-15T03:03:28.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.073+0000 7f3a7df05740 1 -- 172.21.15.121:0/2165272698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 msgr2=0x268d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:28.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.073+0000 7f3a7df05740 1 --2- 172.21.15.121:0/2165272698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 0x268d5b0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7f3a640115b0 tx=0x7f3a64009040).stop 2023-11-15T03:03:28.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.073+0000 7f3a7df05740 1 -- 172.21.15.121:0/2165272698 shutdown_connections 2023-11-15T03:03:28.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.073+0000 7f3a7df05740 1 --2- 172.21.15.121:0/2165272698 >> [v2:172.21.15.121:6834/2646177140,v1:172.21.15.121:6836/2646177140] conn(0x268e800 0x268ebe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.073+0000 7f3a7df05740 1 --2- 172.21.15.121:0/2165272698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 0x268d5b0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.073+0000 7f3a7df05740 1 --2- 172.21.15.121:0/2165272698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2663b20 0x268bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.073+0000 7f3a7df05740 1 --2- 172.21.15.121:0/2165272698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 0x26a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.073+0000 7f3a4bfff700 0 client.7116 ms_handle_reset on v2:172.21.15.121:6834/2646177140 2023-11-15T03:03:28.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.073+0000 7f3a7df05740 1 -- 172.21.15.121:0/2165272698 >> 172.21.15.121:0/2165272698 conn(0x2441550 msgr2=0x26aa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:28.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.073+0000 7f3a7df05740 1 -- 172.21.15.121:0/2165272698 shutdown_connections 2023-11-15T03:03:28.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.073+0000 7f3a7df05740 1 -- 172.21.15.121:0/2165272698 wait complete. 2023-11-15T03:03:28.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.073+0000 7f3a7df05740 20 client.7116 trim_cache size 0 max 16384 2023-11-15T03:03:28.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.093+0000 7f3a7df05740 1 Processor -- start 2023-11-15T03:03:28.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.093+0000 7f3a7df05740 1 -- start start 2023-11-15T03:03:28.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.093+0000 7f3a7df05740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 0x2661260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.093+0000 7f3a7df05740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2663b20 0x2661750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.093+0000 7f3a7df05740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 0x26beeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.093+0000 7f3a7df05740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x26638f0 con 0x26a90a0 2023-11-15T03:03:28.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.093+0000 7f3a7df05740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x25b8c60 con 0x2663b20 2023-11-15T03:03:28.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.093+0000 7f3a7df05740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2617120 con 0x26628d0 2023-11-15T03:03:28.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.093+0000 7f3a72106700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 0x26beeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.093+0000 7f3a72106700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 0x26beeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49770/0 (socket says 172.21.15.121:49770) 2023-11-15T03:03:28.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.093+0000 7f3a72106700 1 -- 172.21.15.121:0/2364783734 learned_addr learned my addr 172.21.15.121:0/2364783734 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:28.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.093+0000 7f3a71104700 1 --2- 172.21.15.121:0/2364783734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2663b20 0x2661750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.093+0000 7f3a71905700 1 --2- 172.21.15.121:0/2364783734 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 0x2661260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.093+0000 7f3a72106700 1 -- 172.21.15.121:0/2364783734 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 msgr2=0x2661260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:28.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.093+0000 7f3a72106700 1 --2- 172.21.15.121:0/2364783734 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 0x2661260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.093+0000 7f3a72106700 1 -- 172.21.15.121:0/2364783734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2663b20 msgr2=0x2661750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:28.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.093+0000 7f3a72106700 1 --2- 172.21.15.121:0/2364783734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2663b20 0x2661750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.093+0000 7f3a72106700 1 -- 172.21.15.121:0/2364783734 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c0c3230 con 0x26a90a0 2023-11-15T03:03:28.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.093+0000 7f3a71104700 1 --2- 172.21.15.121:0/2364783734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2663b20 0x2661750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:03:28.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.097+0000 7f3a72106700 1 --2- 172.21.15.121:0/2364783734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 0x26beeb0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7f3a64007d60 tx=0x7f3a64022000).ready entity=mon.0 client_cookie=c2fa44326d40a4f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:28.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.097+0000 7f3a577fe700 1 -- 172.21.15.121:0/2364783734 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64011e80 con 0x26a90a0 2023-11-15T03:03:28.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.097+0000 7f3a577fe700 1 -- 172.21.15.121:0/2364783734 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a64001780 con 0x26a90a0 2023-11-15T03:03:28.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.097+0000 7f3a577fe700 1 -- 172.21.15.121:0/2364783734 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64001a00 con 0x26a90a0 2023-11-15T03:03:28.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.097+0000 7f3a7df05740 1 -- 172.21.15.121:0/2364783734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 msgr2=0x26beeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:28.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.097+0000 7f3a7df05740 1 --2- 172.21.15.121:0/2364783734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 0x26beeb0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7f3a64007d60 tx=0x7f3a64022000).stop 2023-11-15T03:03:28.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.097+0000 7f3a7df05740 1 -- 172.21.15.121:0/2364783734 shutdown_connections 2023-11-15T03:03:28.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.097+0000 7f3a7df05740 1 --2- 172.21.15.121:0/2364783734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 0x26beeb0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.097+0000 7f3a7df05740 1 --2- 172.21.15.121:0/2364783734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2663b20 0x2661750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.097+0000 7f3a7df05740 1 --2- 172.21.15.121:0/2364783734 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 0x2661260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.097+0000 7f3a7df05740 1 -- 172.21.15.121:0/2364783734 >> 172.21.15.121:0/2364783734 conn(0x2441550 msgr2=0x268f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:28.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.097+0000 7f3a7df05740 1 -- 172.21.15.121:0/2364783734 shutdown_connections 2023-11-15T03:03:28.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.097+0000 7f3a7df05740 1 -- 172.21.15.121:0/2364783734 wait complete. 2023-11-15T03:03:28.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.097+0000 7f3a7df05740 1 -- start start 2023-11-15T03:03:28.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.097+0000 7f3a7df05740 1 Processor -- start 2023-11-15T03:03:28.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.101+0000 7f3a7df05740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 0x270ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.101+0000 7f3a7df05740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2663b20 0x2661120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.101+0000 7f3a7df05740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 0x2661610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.101+0000 7f3a7df05740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2661b00 con 0x26a90a0 2023-11-15T03:03:28.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.101+0000 7f3a7df05740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x270ca00 con 0x2663b20 2023-11-15T03:03:28.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.101+0000 7f3a7df05740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x26a05d0 con 0x26628d0 2023-11-15T03:03:28.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.101+0000 7f3a7df05740 10 client.0 ll_register_callbacks cb 0x7f3a64003790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:03:28.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.101+0000 7f3a72106700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 0x2661610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.101+0000 7f3a72106700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 0x2661610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49778/0 (socket says 172.21.15.121:49778) 2023-11-15T03:03:28.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.101+0000 7f3a72106700 1 -- 172.21.15.121:0/3639105461 learned_addr learned my addr 172.21.15.121:0/3639105461 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:28.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.101+0000 7f3a71905700 1 --2- 172.21.15.121:0/3639105461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 0x270ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.101+0000 7f3a71104700 1 --2- 172.21.15.121:0/3639105461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2663b20 0x2661120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.101+0000 7f3a72106700 1 -- 172.21.15.121:0/3639105461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 msgr2=0x270ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:28.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.101+0000 7f3a72106700 1 --2- 172.21.15.121:0/3639105461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 0x270ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.101+0000 7f3a72106700 1 -- 172.21.15.121:0/3639105461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2663b20 msgr2=0x2661120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:28.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.101+0000 7f3a72106700 1 --2- 172.21.15.121:0/3639105461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2663b20 0x2661120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.101+0000 7f3a72106700 1 -- 172.21.15.121:0/3639105461 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x25dc730 con 0x26a90a0 2023-11-15T03:03:28.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.101+0000 7f3a72106700 1 --2- 172.21.15.121:0/3639105461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 0x2661610 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f3a640115b0 tx=0x7f3a64028070).ready entity=mon.0 client_cookie=2ffd3c8c753162f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:28.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.101+0000 7f3a4bfff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:03:28.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a4bfff700 1 -- 172.21.15.121:0/3639105461 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64001300 con 0x26a90a0 2023-11-15T03:03:28.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a4bfff700 1 -- 172.21.15.121:0/3639105461 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3a64011db0 con 0x26a90a0 2023-11-15T03:03:28.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a7df05740 1 -- 172.21.15.121:0/3639105461 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a5c014a60 con 0x26a90a0 2023-11-15T03:03:28.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a4bfff700 1 -- 172.21.15.121:0/3639105461 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a64007020 con 0x26a90a0 2023-11-15T03:03:28.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a7df05740 10 client.7122 fetch_fsmap learned FSMap version 81 2023-11-15T03:03:28.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a7df05740 1 -- 172.21.15.121:0/3639105461 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x25d7960 con 0x26a90a0 2023-11-15T03:03:28.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a4bfff700 1 -- 172.21.15.121:0/3639105461 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5c014a60 con 0x26a90a0 2023-11-15T03:03:28.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a4bfff700 1 -- 172.21.15.121:0/3639105461 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 81) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3a64007180 con 0x26a90a0 2023-11-15T03:03:28.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a7df05740 10 client.7122 fetch_fsmap finished waiting for FSMap version 81 2023-11-15T03:03:28.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a7df05740 10 client.7122 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:03:28.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a7df05740 20 client.7122 populate_metadata read hostname 'smithi121' 2023-11-15T03:03:28.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a7df05740 1 --2- 172.21.15.121:0/3639105461 >> [v2:172.21.15.121:6834/2646177140,v1:172.21.15.121:6836/2646177140] conn(0x26b8dd0 0x26b91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a7df05740 4 client.7122 mds_command: new command op to 6966 tid=0[{"prefix": "status"}] 2023-11-15T03:03:28.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a7df05740 1 -- 172.21.15.121:0/3639105461 --> [v2:172.21.15.121:6834/2646177140,v1:172.21.15.121:6836/2646177140] -- command(tid 0: {"prefix": "status"}) v1 -- 0x24984e0 con 0x26b8dd0 2023-11-15T03:03:28.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a71905700 1 --2- 172.21.15.121:0/3639105461 >> [v2:172.21.15.121:6834/2646177140,v1:172.21.15.121:6836/2646177140] conn(0x26b8dd0 0x26b91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a71905700 1 --2- 172.21.15.121:0/3639105461 >> [v2:172.21.15.121:6834/2646177140,v1:172.21.15.121:6836/2646177140] conn(0x26b8dd0 0x26b91b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=45d789df7bdc8758 server_cookie=5681c5fee9d6180b in_seq=0 out_seq=0 2023-11-15T03:03:28.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a4bfff700 10 client.7122 ms_handle_connect on v2:172.21.15.121:6834/2646177140 2023-11-15T03:03:28.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a4bfff700 1 -- 172.21.15.121:0/3639105461 <== mds.0 v2:172.21.15.121:6834/2646177140 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x24984e0 con 0x26b8dd0 2023-11-15T03:03:28.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a4bfff700 10 client.7122 handle_command_reply: tid=0 2023-11-15T03:03:28.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a7df05740 1 client.7122 shutdown 2023-11-15T03:03:28.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a7df05740 1 -- 172.21.15.121:0/3639105461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 msgr2=0x2661610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:28.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a7df05740 1 --2- 172.21.15.121:0/3639105461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 0x2661610 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f3a640115b0 tx=0x7f3a64028070).stop 2023-11-15T03:03:28.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a7df05740 1 -- 172.21.15.121:0/3639105461 shutdown_connections 2023-11-15T03:03:28.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a7df05740 1 --2- 172.21.15.121:0/3639105461 >> [v2:172.21.15.121:6834/2646177140,v1:172.21.15.121:6836/2646177140] conn(0x26b8dd0 0x26b91b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a7df05740 1 --2- 172.21.15.121:0/3639105461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x26a90a0 0x2661610 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a4bfff700 0 client.7122 ms_handle_reset on v2:172.21.15.121:6834/2646177140 2023-11-15T03:03:28.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a7df05740 1 --2- 172.21.15.121:0/3639105461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2663b20 0x2661120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a7df05740 1 --2- 172.21.15.121:0/3639105461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x26628d0 0x270ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a7df05740 1 -- 172.21.15.121:0/3639105461 >> 172.21.15.121:0/3639105461 conn(0x2441550 msgr2=0x265f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:28.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.105+0000 7f3a7df05740 1 -- 172.21.15.121:0/3639105461 shutdown_connections 2023-11-15T03:03:28.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.109+0000 7f3a7df05740 1 -- 172.21.15.121:0/3639105461 wait complete. 2023-11-15T03:03:28.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.109+0000 7f3a7df05740 20 client.7122 trim_cache size 0 max 16384 2023-11-15T03:03:28.143 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:03:28.143 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T03:03:28.144 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T03:03:28.144 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 6966, 2023-11-15T03:03:28.144 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T03:03:28.144 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T03:03:28.145 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T03:03:28.145 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 6.317904575, 2023-11-15T03:03:28.145 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 81, 2023-11-15T03:03:28.145 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 92, 2023-11-15T03:03:28.146 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 92, 2023-11-15T03:03:28.146 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 11.736589772 2023-11-15T03:03:28.146 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:03:28.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.109+0000 7f3a72907700 1 -- 172.21.15.121:0/1500910440 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3a5005a540 msgr2=0x7f3a5005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:28.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.109+0000 7f3a72907700 1 --2- 172.21.15.121:0/1500910440 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3a5005a540 0x7f3a5005ca00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f3a6c14e3b0 tx=0x7f3a5c008040).stop 2023-11-15T03:03:28.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.109+0000 7f3a72907700 1 -- 172.21.15.121:0/1500910440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a6c0bc790 msgr2=0x7f3a6c0c0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:28.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.109+0000 7f3a72907700 1 --2- 172.21.15.121:0/1500910440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a6c0bc790 0x7f3a6c0c0870 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f3a6c06c3e0 tx=0x7f3a68017000).stop 2023-11-15T03:03:28.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.109+0000 7f3a72907700 1 -- 172.21.15.121:0/1500910440 shutdown_connections 2023-11-15T03:03:28.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.109+0000 7f3a72907700 1 --2- 172.21.15.121:0/1500910440 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3a5005a540 0x7f3a5005ca00 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.109+0000 7f3a72907700 1 --2- 172.21.15.121:0/1500910440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a6c0be730 0x7f3a6c0bebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.109+0000 7f3a72907700 1 --2- 172.21.15.121:0/1500910440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a6c0c64f0 0x7f3a6c0ac5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.109+0000 7f3a72907700 1 --2- 172.21.15.121:0/1500910440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a6c0bc790 0x7f3a6c0c0870 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.109+0000 7f3a72907700 1 -- 172.21.15.121:0/1500910440 >> 172.21.15.121:0/1500910440 conn(0x7f3a6c069350 msgr2=0x7f3a6c0bd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:28.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.109+0000 7f3a72907700 1 -- 172.21.15.121:0/1500910440 shutdown_connections 2023-11-15T03:03:28.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.109+0000 7f3a72907700 1 -- 172.21.15.121:0/1500910440 wait complete. 2023-11-15T03:03:28.152 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T03:03:28.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.289+0000 7f865ca79700 1 Processor -- start 2023-11-15T03:03:28.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.289+0000 7f865ca79700 1 -- start start 2023-11-15T03:03:28.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.293+0000 7f865ca79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86581508d0 0x7f8658150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.293+0000 7f865ca79700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f86580c27a0 0x7f86580bc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.293+0000 7f865ca79700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86580c0e80 con 0x7f8658063aa0 2023-11-15T03:03:28.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.293+0000 7f865ca79700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8658063f60 con 0x7f86580c27a0 2023-11-15T03:03:28.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.293+0000 7f865ca79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8658064240 con 0x7f86581508d0 2023-11-15T03:03:28.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.293+0000 7f8657fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f86580c27a0 0x7f86580bc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.293+0000 7f864effd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f8658063aa0 0x7f8658068cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:58616/0 (socket says 172.21.15.121:58616) 2023-11-15T03:03:28.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.293+0000 7f864effd700 1 -- 172.21.15.121:0/3673295595 learned_addr learned my addr 172.21.15.121:0/3673295595 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:28.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.293+0000 7f86577fe700 1 --2- 172.21.15.121:0/3673295595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86581508d0 0x7f8658150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.293+0000 7f8656ffd700 1 -- 172.21.15.121:0/3673295595 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424506511 0 0) 0x7f86580c0e80 con 0x7f8658063aa0 2023-11-15T03:03:28.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.293+0000 7f8656ffd700 1 -- 172.21.15.121:0/3673295595 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86440033f0 con 0x7f8658063aa0 2023-11-15T03:03:28.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.297+0000 7f8657fff700 1 -- 172.21.15.121:0/3673295595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86581508d0 msgr2=0x7f8658150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:28.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.297+0000 7f8657fff700 1 --2- 172.21.15.121:0/3673295595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86581508d0 0x7f8658150cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.297+0000 7f8657fff700 1 -- 172.21.15.121:0/3673295595 >> v1:172.21.15.190:6789/0 conn(0x7f8658063aa0 legacy=0x7f8658068cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:28.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.297+0000 7f8657fff700 1 -- 172.21.15.121:0/3673295595 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8658064520 con 0x7f86580c27a0 2023-11-15T03:03:28.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.297+0000 7f8657fff700 1 --2- 172.21.15.121:0/3673295595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86580c27a0 0x7f86580bc620 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f8650009e60 tx=0x7f8650009780).ready entity=mon.1 client_cookie=10d1025e50a15db8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:28.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.297+0000 7f8656ffd700 1 -- 172.21.15.121:0/3673295595 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865000e070 con 0x7f86580c27a0 2023-11-15T03:03:28.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.297+0000 7f8656ffd700 1 -- 172.21.15.121:0/3673295595 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865000bab0 con 0x7f86580c27a0 2023-11-15T03:03:28.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.297+0000 7f8656ffd700 1 -- 172.21.15.121:0/3673295595 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865000be30 con 0x7f86580c27a0 2023-11-15T03:03:28.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.297+0000 7f865ca79700 1 -- 172.21.15.121:0/3673295595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86580c27a0 msgr2=0x7f86580bc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:28.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.297+0000 7f865ca79700 1 --2- 172.21.15.121:0/3673295595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86580c27a0 0x7f86580bc620 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f8650009e60 tx=0x7f8650009780).stop 2023-11-15T03:03:28.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.297+0000 7f865ca79700 1 -- 172.21.15.121:0/3673295595 shutdown_connections 2023-11-15T03:03:28.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.297+0000 7f865ca79700 1 --2- 172.21.15.121:0/3673295595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86580c27a0 0x7f86580bc620 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.297+0000 7f865ca79700 1 --2- 172.21.15.121:0/3673295595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86581508d0 0x7f8658150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.297+0000 7f865ca79700 1 -- 172.21.15.121:0/3673295595 >> 172.21.15.121:0/3673295595 conn(0x7f86580677c0 msgr2=0x7f86580c65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:28.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.297+0000 7f865ca79700 1 -- 172.21.15.121:0/3673295595 shutdown_connections 2023-11-15T03:03:28.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.297+0000 7f865ca79700 1 -- 172.21.15.121:0/3673295595 wait complete. 2023-11-15T03:03:28.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.301+0000 7f865ca79700 1 Processor -- start 2023-11-15T03:03:28.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.301+0000 7f865ca79700 1 -- start start 2023-11-15T03:03:28.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.301+0000 7f865ca79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8658063aa0 0x7f8658152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.301+0000 7f865ca79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86580c27a0 0x7f86580a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.301+0000 7f865ca79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86581508d0 0x7f86581372b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.301+0000 7f865ca79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86581536d0 con 0x7f8658063aa0 2023-11-15T03:03:28.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.301+0000 7f865ca79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8658138410 con 0x7f86581508d0 2023-11-15T03:03:28.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.301+0000 7f865ca79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86581420f0 con 0x7f86580c27a0 2023-11-15T03:03:28.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.301+0000 7f86577fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8658063aa0 0x7f8658152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.301+0000 7f86577fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8658063aa0 0x7f8658152d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49780/0 (socket says 172.21.15.121:49780) 2023-11-15T03:03:28.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.301+0000 7f86577fe700 1 -- 172.21.15.121:0/1047727489 learned_addr learned my addr 172.21.15.121:0/1047727489 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:28.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.301+0000 7f8657fff700 1 --2- 172.21.15.121:0/1047727489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86581508d0 0x7f86581372b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.301+0000 7f864effd700 1 --2- 172.21.15.121:0/1047727489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86580c27a0 0x7f86580a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.301+0000 7f86577fe700 1 -- 172.21.15.121:0/1047727489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86580c27a0 msgr2=0x7f86580a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:28.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.305+0000 7f86577fe700 1 --2- 172.21.15.121:0/1047727489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86580c27a0 0x7f86580a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.305+0000 7f86577fe700 1 -- 172.21.15.121:0/1047727489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86581508d0 msgr2=0x7f86581372b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:28.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.305+0000 7f86577fe700 1 --2- 172.21.15.121:0/1047727489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86581508d0 0x7f86581372b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.305+0000 7f86577fe700 1 -- 172.21.15.121:0/1047727489 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8650009a20 con 0x7f8658063aa0 2023-11-15T03:03:28.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.305+0000 7f86577fe700 1 --2- 172.21.15.121:0/1047727489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8658063aa0 0x7f8658152d30 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7f864800e900 tx=0x7f8648009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:28.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.305+0000 7f864ffff700 1 -- 172.21.15.121:0/1047727489 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648012600 con 0x7f8658063aa0 2023-11-15T03:03:28.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.305+0000 7f864ffff700 1 -- 172.21.15.121:0/1047727489 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8648012760 con 0x7f8658063aa0 2023-11-15T03:03:28.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.305+0000 7f864ffff700 1 -- 172.21.15.121:0/1047727489 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648012ae0 con 0x7f8658063aa0 2023-11-15T03:03:28.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.305+0000 7f865ca79700 1 -- 172.21.15.121:0/1047727489 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86581533f0 con 0x7f8658063aa0 2023-11-15T03:03:28.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.305+0000 7f865ca79700 1 -- 172.21.15.121:0/1047727489 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86581380a0 con 0x7f8658063aa0 2023-11-15T03:03:28.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.305+0000 7f865ca79700 1 -- 172.21.15.121:0/1047727489 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8658137c20 con 0x7f8658063aa0 2023-11-15T03:03:28.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.305+0000 7f864ffff700 1 -- 172.21.15.121:0/1047727489 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f864801b400 con 0x7f8658063aa0 2023-11-15T03:03:28.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.309+0000 7f864ffff700 1 --2- 172.21.15.121:0/1047727489 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f863805a540 0x7f863805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:28.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.309+0000 7f864ffff700 1 -- 172.21.15.121:0/1047727489 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8648074a30 con 0x7f8658063aa0 2023-11-15T03:03:28.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.309+0000 7f864effd700 1 --2- 172.21.15.121:0/1047727489 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f863805a540 0x7f863805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:28.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.309+0000 7f864effd700 1 --2- 172.21.15.121:0/1047727489 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f863805a540 0x7f863805ca00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f8658143a40 tx=0x7f8640009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:28.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.313+0000 7f864ffff700 1 -- 172.21.15.121:0/1047727489 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8658137c20 con 0x7f8658063aa0 2023-11-15T03:03:28.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.509+0000 7f865ca79700 1 -- 172.21.15.121:0/1047727489 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8658137c20 con 0x7f8658063aa0 2023-11-15T03:03:28.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.513+0000 7f864ffff700 1 -- 172.21.15.121:0/1047727489 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8658137c20 con 0x7f8658063aa0 2023-11-15T03:03:28.528 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T03:03:28.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.513+0000 7f865ca79700 1 -- 172.21.15.121:0/1047727489 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f863805a540 msgr2=0x7f863805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:28.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.513+0000 7f865ca79700 1 --2- 172.21.15.121:0/1047727489 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f863805a540 0x7f863805ca00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f8658143a40 tx=0x7f8640009040).stop 2023-11-15T03:03:28.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.513+0000 7f865ca79700 1 -- 172.21.15.121:0/1047727489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8658063aa0 msgr2=0x7f8658152d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:28.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.513+0000 7f865ca79700 1 --2- 172.21.15.121:0/1047727489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8658063aa0 0x7f8658152d30 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7f864800e900 tx=0x7f8648009060).stop 2023-11-15T03:03:28.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.513+0000 7f865ca79700 1 -- 172.21.15.121:0/1047727489 shutdown_connections 2023-11-15T03:03:28.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.513+0000 7f865ca79700 1 --2- 172.21.15.121:0/1047727489 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f863805a540 0x7f863805ca00 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.513+0000 7f865ca79700 1 --2- 172.21.15.121:0/1047727489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86581508d0 0x7f86581372b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.513+0000 7f865ca79700 1 --2- 172.21.15.121:0/1047727489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86580c27a0 0x7f86580a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.513+0000 7f865ca79700 1 --2- 172.21.15.121:0/1047727489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8658063aa0 0x7f8658152d30 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:28.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.513+0000 7f865ca79700 1 -- 172.21.15.121:0/1047727489 >> 172.21.15.121:0/1047727489 conn(0x7f86580677c0 msgr2=0x7f86581546c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:28.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.513+0000 7f865ca79700 1 -- 172.21.15.121:0/1047727489 shutdown_connections 2023-11-15T03:03:28.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:28.513+0000 7f865ca79700 1 -- 172.21.15.121:0/1047727489 wait complete. 2023-11-15T03:03:28.538 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T03:03:28.539 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T03:03:28.539 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T03:03:28.539 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T03:03:28.540 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T03:03:28.540 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T03:03:28.540 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T03:03:28.541 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T03:03:28.541 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:03:28.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:28.545 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T03:03:28.563 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T03:03:28.563 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T03:03:28.563 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T03:03:28.564 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:03:28.564 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T03:03:28.564 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:03:28.564 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:03:28.564 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:03:28.565 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T03:03:28.565 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:03:28.565 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:03:28.565 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T03:03:28.565 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:03:28.566 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:03:28.566 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T03:03:28.566 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:03:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T03:03:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42031sec preferred_lft 42031sec 2023-11-15T03:03:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T03:03:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:03:28.567 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T03:03:28.568 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:03:28.568 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T03:03:28.568 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:03:28.568 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T03:03:28.569 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T03:03:28.569 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T03:03:28.569 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:03:28.569 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T03:03:28.569 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:03:28.570 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:03:28.570 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T03:03:28.570 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T03:03:28.570 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:03:28.570 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:03:28.571 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T03:03:28.571 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T03:03:28.571 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:03:28.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:28.573 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T03:03:28.654 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T03:03:28.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:28.656 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T03:03:28.724 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T03:03:28.724 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T03:03:28.724 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T03:03:28.725 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T03:03:28.725 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T03:03:28.725 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T03:03:28.725 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T03:03:28.725 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T03:03:28.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:28.726 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T03:03:28.727 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T03:03:28.727 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T03:03:28.729 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T03:03:28.729 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T03:03:28.730 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T03:03:28.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:28.873 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T03:03:28.888 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T03:03:28.888 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T03:03:28.889 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:03:28.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:28.890 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T03:03:28.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:28.963 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T03:03:29.026 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T03:03:29.026 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:03:29.145 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:03:29.145 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:03:29.168 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:03:29.168 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:03:29.174 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T03:03:29.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:29.175 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T03:03:29.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:03:29.217 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-11-15T03:03:29.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:03:29.296 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2023-11-15T03:03:29.454 INFO:teuthology.orchestra.run.smithi121.stderr:6+0 records in 2023-11-15T03:03:29.455 INFO:teuthology.orchestra.run.smithi121.stderr:6+0 records out 2023-11-15T03:03:29.455 INFO:teuthology.orchestra.run.smithi121.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0917351 s, 68.6 MB/s 2023-11-15T03:03:29.456 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-15T03:03:29.457 DEBUG:teuthology.orchestra.run.smithi121:> import os 2023-11-15T03:03:29.457 DEBUG:teuthology.orchestra.run.smithi121:> import stat 2023-11-15T03:03:29.457 DEBUG:teuthology.orchestra.run.smithi121:> import json 2023-11-15T03:03:29.460 DEBUG:teuthology.orchestra.run.smithi121:> import sys 2023-11-15T03:03:29.462 DEBUG:teuthology.orchestra.run.smithi121:> 2023-11-15T03:03:29.462 DEBUG:teuthology.orchestra.run.smithi121:> try: 2023-11-15T03:03:29.463 DEBUG:teuthology.orchestra.run.smithi121:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2023-11-15T03:03:29.463 DEBUG:teuthology.orchestra.run.smithi121:> except OSError as e: 2023-11-15T03:03:29.463 DEBUG:teuthology.orchestra.run.smithi121:> sys.exit(e.errno) 2023-11-15T03:03:29.464 DEBUG:teuthology.orchestra.run.smithi121:> 2023-11-15T03:03:29.464 DEBUG:teuthology.orchestra.run.smithi121:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-11-15T03:03:29.465 DEBUG:teuthology.orchestra.run.smithi121:> print(json.dumps( 2023-11-15T03:03:29.465 DEBUG:teuthology.orchestra.run.smithi121:> dict([(a, getattr(s, a)) for a in attrs]), 2023-11-15T03:03:29.465 DEBUG:teuthology.orchestra.run.smithi121:> indent=2)) 2023-11-15T03:03:29.466 DEBUG:teuthology.orchestra.run.smithi121:> ' 2023-11-15T03:03:29.562 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:03:29.562 INFO:teuthology.orchestra.run.smithi121.stdout: "st_mode": 33204, 2023-11-15T03:03:29.563 INFO:teuthology.orchestra.run.smithi121.stdout: "st_ino": 1099511627777, 2023-11-15T03:03:29.563 INFO:teuthology.orchestra.run.smithi121.stdout: "st_dev": 49, 2023-11-15T03:03:29.563 INFO:teuthology.orchestra.run.smithi121.stdout: "st_nlink": 1, 2023-11-15T03:03:29.563 INFO:teuthology.orchestra.run.smithi121.stdout: "st_uid": 1000, 2023-11-15T03:03:29.563 INFO:teuthology.orchestra.run.smithi121.stdout: "st_gid": 1267, 2023-11-15T03:03:29.564 INFO:teuthology.orchestra.run.smithi121.stdout: "st_size": 6291456, 2023-11-15T03:03:29.564 INFO:teuthology.orchestra.run.smithi121.stdout: "st_atime": 1700017409.3573174, 2023-11-15T03:03:29.564 INFO:teuthology.orchestra.run.smithi121.stdout: "st_mtime": 1700017409.3973165, 2023-11-15T03:03:29.564 INFO:teuthology.orchestra.run.smithi121.stdout: "st_ctime": 1700017409.3973165 2023-11-15T03:03:29.565 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:03:29.747 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T03:03:29.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:03:29.748 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-15T03:03:29.883 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:03:29.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:29.884 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:03:29.908 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:03:30.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.113+0000 7fabcd4f1700 1 Processor -- start 2023-11-15T03:03:30.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.113+0000 7fabcd4f1700 1 -- start start 2023-11-15T03:03:30.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.113+0000 7fabcd4f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc8153a30 0x7fabc8153e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:30.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.113+0000 7fabcd4f1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fabc80ab620 0x7fabc80c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:30.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.113+0000 7fabcd4f1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fabc80a8f50 0x7fabc80cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:30.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.113+0000 7fabcd4f1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc8067a60 con 0x7fabc80ab620 2023-11-15T03:03:30.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.113+0000 7fabcd4f1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc80a93e0 con 0x7fabc80a8f50 2023-11-15T03:03:30.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.113+0000 7fabcd4f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80c2920 con 0x7fabc8153a30 2023-11-15T03:03:30.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.113+0000 7fabc67fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fabc80ab620 0x7fabc80c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:30.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.113+0000 7fabc6ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc8153a30 0x7fabc8153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:30.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.113+0000 7fabc67fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fabc80ab620 0x7fabc80c22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49796/0 (socket says 172.21.15.121:49796) 2023-11-15T03:03:30.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.113+0000 7fabc6ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc8153a30 0x7fabc8153e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:52398/0 (socket says 172.21.15.121:52398) 2023-11-15T03:03:30.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.113+0000 7fabc67fc700 1 -- 172.21.15.121:0/676479797 learned_addr learned my addr 172.21.15.121:0/676479797 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:30.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.113+0000 7fabc7fff700 1 --2- 172.21.15.121:0/676479797 >> v2:172.21.15.190:3300/0 conn(0x7fabc80a8f50 0x7fabc80cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:30.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.117+0000 7fabc67fc700 1 -- 172.21.15.121:0/676479797 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc8153a30 msgr2=0x7fabc8153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:30.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.117+0000 7fabc67fc700 1 --2- 172.21.15.121:0/676479797 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc8153a30 0x7fabc8153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:30.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.117+0000 7fabc67fc700 1 -- 172.21.15.121:0/676479797 >> v2:172.21.15.190:3300/0 conn(0x7fabc80a8f50 msgr2=0x7fabc80cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:30.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.117+0000 7fabc67fc700 1 --2- 172.21.15.121:0/676479797 >> v2:172.21.15.190:3300/0 conn(0x7fabc80a8f50 0x7fabc80cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:30.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.117+0000 7fabc67fc700 1 -- 172.21.15.121:0/676479797 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc80c2c00 con 0x7fabc80ab620 2023-11-15T03:03:30.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.117+0000 7fabc7fff700 1 --2- 172.21.15.121:0/676479797 >> v2:172.21.15.190:3300/0 conn(0x7fabc80a8f50 0x7fabc80cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:30.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.117+0000 7fabc67fc700 1 --2- 172.21.15.121:0/676479797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc80ab620 0x7fabc80c22b0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7fabc80c9bd0 tx=0x7fabbc0090f0).ready entity=mon.0 client_cookie=c7a2bf33fce63e4c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:30.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.117+0000 7fabc5ffb700 1 -- 172.21.15.121:0/676479797 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc00b520 con 0x7fabc80ab620 2023-11-15T03:03:30.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.117+0000 7fabc5ffb700 1 -- 172.21.15.121:0/676479797 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabbc00b680 con 0x7fabc80ab620 2023-11-15T03:03:30.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.117+0000 7fabc5ffb700 1 -- 172.21.15.121:0/676479797 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc00ba00 con 0x7fabc80ab620 2023-11-15T03:03:30.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.117+0000 7fabcd4f1700 1 -- 172.21.15.121:0/676479797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc80ab620 msgr2=0x7fabc80c22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:30.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.117+0000 7fabcd4f1700 1 --2- 172.21.15.121:0/676479797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc80ab620 0x7fabc80c22b0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7fabc80c9bd0 tx=0x7fabbc0090f0).stop 2023-11-15T03:03:30.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.117+0000 7fabcd4f1700 1 -- 172.21.15.121:0/676479797 shutdown_connections 2023-11-15T03:03:30.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.117+0000 7fabcd4f1700 1 --2- 172.21.15.121:0/676479797 >> v2:172.21.15.190:3300/0 conn(0x7fabc80a8f50 0x7fabc80cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:30.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.117+0000 7fabcd4f1700 1 --2- 172.21.15.121:0/676479797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc80ab620 0x7fabc80c22b0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:30.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.117+0000 7fabcd4f1700 1 --2- 172.21.15.121:0/676479797 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc8153a30 0x7fabc8153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:30.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.117+0000 7fabcd4f1700 1 -- 172.21.15.121:0/676479797 >> 172.21.15.121:0/676479797 conn(0x7fabc8095820 msgr2=0x7fabc8069820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:30.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.117+0000 7fabcd4f1700 1 -- 172.21.15.121:0/676479797 shutdown_connections 2023-11-15T03:03:30.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.117+0000 7fabcd4f1700 1 -- 172.21.15.121:0/676479797 wait complete. 2023-11-15T03:03:30.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.121+0000 7fabcd4f1700 1 Processor -- start 2023-11-15T03:03:30.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.121+0000 7fabcd4f1700 1 -- start start 2023-11-15T03:03:30.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.121+0000 7fabcd4f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc80a8f50 0x7fabc8084a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:30.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.121+0000 7fabcd4f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc80ab620 0x7fabc80a9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:30.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.121+0000 7fabcd4f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc8153a30 0x7fabc80aa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:30.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.121+0000 7fabcd4f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80aed30 con 0x7fabc8153a30 2023-11-15T03:03:30.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.121+0000 7fabcd4f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80aa8e0 con 0x7fabc80ab620 2023-11-15T03:03:30.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.121+0000 7fabcd4f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80ae8a0 con 0x7fabc80a8f50 2023-11-15T03:03:30.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.125+0000 7fabc7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc8153a30 0x7fabc80aa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:30.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.125+0000 7fabc7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc8153a30 0x7fabc80aa3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49802/0 (socket says 172.21.15.121:49802) 2023-11-15T03:03:30.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.125+0000 7fabc7fff700 1 -- 172.21.15.121:0/938415448 learned_addr learned my addr 172.21.15.121:0/938415448 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:30.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.125+0000 7fabc67fc700 1 --2- 172.21.15.121:0/938415448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc80ab620 0x7fabc80a9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:30.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.125+0000 7fabc6ffd700 1 --2- 172.21.15.121:0/938415448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc80a8f50 0x7fabc8084a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:30.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.125+0000 7fabc7fff700 1 -- 172.21.15.121:0/938415448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc80a8f50 msgr2=0x7fabc8084a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:30.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.125+0000 7fabc7fff700 1 --2- 172.21.15.121:0/938415448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc80a8f50 0x7fabc8084a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:30.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.125+0000 7fabc7fff700 1 -- 172.21.15.121:0/938415448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc80ab620 msgr2=0x7fabc80a9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:30.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.125+0000 7fabc7fff700 1 --2- 172.21.15.121:0/938415448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc80ab620 0x7fabc80a9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:30.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.125+0000 7fabc7fff700 1 -- 172.21.15.121:0/938415448 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabbc009370 con 0x7fabc8153a30 2023-11-15T03:03:30.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.125+0000 7fabc7fff700 1 --2- 172.21.15.121:0/938415448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc8153a30 0x7fabc80aa3f0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7fabc000d100 tx=0x7fabc00099b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:30.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.125+0000 7fabaeffd700 1 -- 172.21.15.121:0/938415448 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc000dc40 con 0x7fabc8153a30 2023-11-15T03:03:30.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.125+0000 7fabaeffd700 1 -- 172.21.15.121:0/938415448 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc000f040 con 0x7fabc8153a30 2023-11-15T03:03:30.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.125+0000 7fabaeffd700 1 -- 172.21.15.121:0/938415448 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc00133f0 con 0x7fabc8153a30 2023-11-15T03:03:30.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.125+0000 7fabcd4f1700 1 -- 172.21.15.121:0/938415448 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc80adf70 con 0x7fabc8153a30 2023-11-15T03:03:30.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.125+0000 7fabcd4f1700 1 -- 172.21.15.121:0/938415448 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc80ae530 con 0x7fabc8153a30 2023-11-15T03:03:30.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.125+0000 7fabcd4f1700 1 -- 172.21.15.121:0/938415448 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc80aea50 con 0x7fabc8153a30 2023-11-15T03:03:30.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.129+0000 7fabaeffd700 1 -- 172.21.15.121:0/938415448 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fabc00137e0 con 0x7fabc8153a30 2023-11-15T03:03:30.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.129+0000 7fabaeffd700 1 --2- 172.21.15.121:0/938415448 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fabb405a540 0x7fabb405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:30.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.129+0000 7fabaeffd700 1 -- 172.21.15.121:0/938415448 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fabc0074cd0 con 0x7fabc8153a30 2023-11-15T03:03:30.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.129+0000 7fabc6ffd700 1 --2- 172.21.15.121:0/938415448 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fabb405a540 0x7fabb405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:30.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.129+0000 7fabc6ffd700 1 --2- 172.21.15.121:0/938415448 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fabb405a540 0x7fabb405ca00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fabc8084470 tx=0x7fabb8009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:30.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.133+0000 7fabaeffd700 1 -- 172.21.15.121:0/938415448 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fabc80aea50 con 0x7fabc8153a30 2023-11-15T03:03:30.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.329+0000 7fabcd4f1700 1 -- 172.21.15.121:0/938415448 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabc80aea10 con 0x7fabc8153a30 2023-11-15T03:03:30.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.329+0000 7fabaeffd700 1 -- 172.21.15.121:0/938415448 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4500 (secure 0 0 0) 0x7fabc0048450 con 0x7fabc8153a30 2023-11-15T03:03:30.339 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:30.339 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":81,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:03:24.804563+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6966},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6966":{"gid":6966,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2646177140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2646177140},{"type":"v1","addr":"172.21.15.121:6836","nonce":2646177140}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:03:30.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.333+0000 7fabcd4f1700 1 -- 172.21.15.121:0/938415448 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fabb405a540 msgr2=0x7fabb405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:30.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.333+0000 7fabcd4f1700 1 --2- 172.21.15.121:0/938415448 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fabb405a540 0x7fabb405ca00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fabc8084470 tx=0x7fabb8009060).stop 2023-11-15T03:03:30.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.333+0000 7fabcd4f1700 1 -- 172.21.15.121:0/938415448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc8153a30 msgr2=0x7fabc80aa3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:30.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.333+0000 7fabcd4f1700 1 --2- 172.21.15.121:0/938415448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc8153a30 0x7fabc80aa3f0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7fabc000d100 tx=0x7fabc00099b0).stop 2023-11-15T03:03:30.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.333+0000 7fabcd4f1700 1 -- 172.21.15.121:0/938415448 shutdown_connections 2023-11-15T03:03:30.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.333+0000 7fabcd4f1700 1 --2- 172.21.15.121:0/938415448 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fabb405a540 0x7fabb405ca00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:30.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.333+0000 7fabcd4f1700 1 --2- 172.21.15.121:0/938415448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabc8153a30 0x7fabc80aa3f0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:30.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.333+0000 7fabcd4f1700 1 --2- 172.21.15.121:0/938415448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc80ab620 0x7fabc80a9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:30.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.333+0000 7fabcd4f1700 1 --2- 172.21.15.121:0/938415448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc80a8f50 0x7fabc8084a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:30.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.333+0000 7fabcd4f1700 1 -- 172.21.15.121:0/938415448 >> 172.21.15.121:0/938415448 conn(0x7fabc8095820 msgr2=0x7fabc80b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:30.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.333+0000 7fabcd4f1700 1 -- 172.21.15.121:0/938415448 shutdown_connections 2023-11-15T03:03:30.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.333+0000 7fabcd4f1700 1 -- 172.21.15.121:0/938415448 wait complete. 2023-11-15T03:03:30.344 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 81 2023-11-15T03:03:30.357 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-15T03:03:30.549 INFO:teuthology.orchestra.run.smithi121.stdout:{"message":"","return_code":0} 2023-11-15T03:03:30.549 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-15T03:03:30.550 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:03:30.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.713+0000 7f76deedb700 1 Processor -- start 2023-11-15T03:03:30.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.713+0000 7f76deedb700 1 -- start start 2023-11-15T03:03:30.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.717+0000 7f76deedb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d80c04d0 0x7f76d80c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:30.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.717+0000 7f76deedb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f76d80c8fb0 0x7f76d80c9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:30.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.717+0000 7f76deedb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d80c5720 con 0x7f76d8153670 2023-11-15T03:03:30.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.717+0000 7f76deedb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d80c0da0 con 0x7f76d80c8fb0 2023-11-15T03:03:30.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.717+0000 7f76deedb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d80ca8a0 con 0x7f76d80c04d0 2023-11-15T03:03:30.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.717+0000 7f76d7fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f76d8153670 0x7f76d8153ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:58624/0 (socket says 172.21.15.121:58624) 2023-11-15T03:03:30.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.717+0000 7f76d7fff700 1 -- 172.21.15.121:0/1606657029 learned_addr learned my addr 172.21.15.121:0/1606657029 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:30.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.717+0000 7f76dd478700 1 --2- 172.21.15.121:0/1606657029 >> v2:172.21.15.190:3300/0 conn(0x7f76d80c8fb0 0x7f76d80c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:30.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.717+0000 7f76dcc77700 1 --2- 172.21.15.121:0/1606657029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d80c04d0 0x7f76d80c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:30.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.717+0000 7f76d77fe700 1 -- 172.21.15.121:0/1606657029 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 864116100 0 0) 0x7f76d80c5720 con 0x7f76d8153670 2023-11-15T03:03:30.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.717+0000 7f76d77fe700 1 -- 172.21.15.121:0/1606657029 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76c80033f0 con 0x7f76d8153670 2023-11-15T03:03:30.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.721+0000 7f76dd478700 1 -- 172.21.15.121:0/1606657029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d80c04d0 msgr2=0x7f76d80c08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:30.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.721+0000 7f76dd478700 1 --2- 172.21.15.121:0/1606657029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d80c04d0 0x7f76d80c08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:30.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.721+0000 7f76dd478700 1 -- 172.21.15.121:0/1606657029 >> v1:172.21.15.190:6789/0 conn(0x7f76d8153670 legacy=0x7f76d8153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:30.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.721+0000 7f76d77fe700 1 -- 172.21.15.121:0/1606657029 <== mon.1 v1:172.21.15.190:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3996814242 0 0) 0x7f76c80033f0 con 0x7f76d8153670 2023-11-15T03:03:30.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.721+0000 7f76dd478700 1 -- 172.21.15.121:0/1606657029 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d80c9d20 con 0x7f76d80c8fb0 2023-11-15T03:03:30.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.721+0000 7f76dd478700 1 --2- 172.21.15.121:0/1606657029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d80c8fb0 0x7f76d80c9460 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f76d0009e60 tx=0x7f76d0009780).ready entity=mon.1 client_cookie=350480c0a133c85 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:30.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.721+0000 7f76d77fe700 1 -- 172.21.15.121:0/1606657029 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d000e070 con 0x7f76d80c8fb0 2023-11-15T03:03:30.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.721+0000 7f76d77fe700 1 -- 172.21.15.121:0/1606657029 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d000bab0 con 0x7f76d80c8fb0 2023-11-15T03:03:30.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.721+0000 7f76d77fe700 1 -- 172.21.15.121:0/1606657029 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d000be30 con 0x7f76d80c8fb0 2023-11-15T03:03:30.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.721+0000 7f76deedb700 1 -- 172.21.15.121:0/1606657029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d80c8fb0 msgr2=0x7f76d80c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:30.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.721+0000 7f76deedb700 1 --2- 172.21.15.121:0/1606657029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d80c8fb0 0x7f76d80c9460 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f76d0009e60 tx=0x7f76d0009780).stop 2023-11-15T03:03:30.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.721+0000 7f76deedb700 1 -- 172.21.15.121:0/1606657029 shutdown_connections 2023-11-15T03:03:30.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.721+0000 7f76deedb700 1 --2- 172.21.15.121:0/1606657029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d80c8fb0 0x7f76d80c9460 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:30.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.721+0000 7f76deedb700 1 --2- 172.21.15.121:0/1606657029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d80c04d0 0x7f76d80c08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:30.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.721+0000 7f76deedb700 1 -- 172.21.15.121:0/1606657029 >> 172.21.15.121:0/1606657029 conn(0x7f76d8067a60 msgr2=0x7f76d8067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:30.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.721+0000 7f76deedb700 1 -- 172.21.15.121:0/1606657029 shutdown_connections 2023-11-15T03:03:30.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.721+0000 7f76deedb700 1 -- 172.21.15.121:0/1606657029 wait complete. 2023-11-15T03:03:30.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.725+0000 7f76deedb700 1 Processor -- start 2023-11-15T03:03:30.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.725+0000 7f76deedb700 1 -- start start 2023-11-15T03:03:30.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.725+0000 7f76deedb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d80c04d0 0x7f76d8147060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:30.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.725+0000 7f76deedb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d80c8fb0 0x7f76d81439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:30.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.725+0000 7f76deedb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76d8153670 0x7f76d8147770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:30.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.725+0000 7f76deedb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d80c5e00 con 0x7f76d8153670 2023-11-15T03:03:30.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.725+0000 7f76deedb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d80c4210 con 0x7f76d80c8fb0 2023-11-15T03:03:30.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.725+0000 7f76deedb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d80c44f0 con 0x7f76d80c04d0 2023-11-15T03:03:30.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.725+0000 7f76dd478700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76d8153670 0x7f76d8147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:30.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.725+0000 7f76d7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d80c8fb0 0x7f76d81439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:30.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.725+0000 7f76dcc77700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d80c04d0 0x7f76d8147060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:30.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.725+0000 7f76d7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d80c8fb0 0x7f76d81439e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:50478/0 (socket says 172.21.15.121:50478) 2023-11-15T03:03:30.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.725+0000 7f76dd478700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76d8153670 0x7f76d8147770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49808/0 (socket says 172.21.15.121:49808) 2023-11-15T03:03:30.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.725+0000 7f76dcc77700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d80c04d0 0x7f76d8147060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:52432/0 (socket says 172.21.15.121:52432) 2023-11-15T03:03:30.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.725+0000 7f76d7fff700 1 -- 172.21.15.121:0/1118469422 learned_addr learned my addr 172.21.15.121:0/1118469422 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:30.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.729+0000 7f76d7fff700 1 -- 172.21.15.121:0/1118469422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d80c04d0 msgr2=0x7f76d8147060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:30.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.729+0000 7f76d7fff700 1 --2- 172.21.15.121:0/1118469422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d80c04d0 0x7f76d8147060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:30.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.729+0000 7f76d7fff700 1 -- 172.21.15.121:0/1118469422 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76d8153670 msgr2=0x7f76d8147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:30.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.729+0000 7f76d7fff700 1 --2- 172.21.15.121:0/1118469422 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76d8153670 0x7f76d8147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:30.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.729+0000 7f76d7fff700 1 -- 172.21.15.121:0/1118469422 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d0009a20 con 0x7f76d80c8fb0 2023-11-15T03:03:30.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.729+0000 7f76d7fff700 1 --2- 172.21.15.121:0/1118469422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d80c8fb0 0x7f76d81439e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f76c400c470 tx=0x7f76c4007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:30.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.729+0000 7f76d4ff9700 1 -- 172.21.15.121:0/1118469422 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c4007170 con 0x7f76d80c8fb0 2023-11-15T03:03:30.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.729+0000 7f76d4ff9700 1 -- 172.21.15.121:0/1118469422 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c40072d0 con 0x7f76d80c8fb0 2023-11-15T03:03:30.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.729+0000 7f76deedb700 1 -- 172.21.15.121:0/1118469422 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d80c47d0 con 0x7f76d80c8fb0 2023-11-15T03:03:30.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.729+0000 7f76d4ff9700 1 -- 172.21.15.121:0/1118469422 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c40046b0 con 0x7f76d80c8fb0 2023-11-15T03:03:30.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.729+0000 7f76deedb700 1 -- 172.21.15.121:0/1118469422 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d80c1d10 con 0x7f76d80c8fb0 2023-11-15T03:03:30.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.729+0000 7f76deedb700 1 -- 172.21.15.121:0/1118469422 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d80c5f70 con 0x7f76d80c8fb0 2023-11-15T03:03:30.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.733+0000 7f76d4ff9700 1 -- 172.21.15.121:0/1118469422 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f76c4004aa0 con 0x7f76d80c8fb0 2023-11-15T03:03:30.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.733+0000 7f76d4ff9700 1 --2- 172.21.15.121:0/1118469422 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f76bc05a510 0x7f76bc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:30.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.733+0000 7f76d4ff9700 1 -- 172.21.15.121:0/1118469422 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f76c4074000 con 0x7f76d80c8fb0 2023-11-15T03:03:30.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.733+0000 7f76dcc77700 1 --2- 172.21.15.121:0/1118469422 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f76bc05a510 0x7f76bc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:30.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.733+0000 7f76dcc77700 1 --2- 172.21.15.121:0/1118469422 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f76bc05a510 0x7f76bc05c9d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f76cc009c30 tx=0x7f76cc009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:30.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.741+0000 7f76d4ff9700 1 -- 172.21.15.121:0/1118469422 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f76c4047720 con 0x7f76d80c8fb0 2023-11-15T03:03:30.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:30.941+0000 7f76deedb700 1 -- 172.21.15.121:0/1118469422 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f76d8047060 con 0x7f76d80c8fb0 2023-11-15T03:03:31.123 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:03:31.117+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:03:31.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.129+0000 7f76d4ff9700 1 -- 172.21.15.121:0/1118469422 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v82) v1 ==== 161+0+0 (secure 0 0 0) 0x7f76c404ad40 con 0x7f76d80c8fb0 2023-11-15T03:03:31.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.133+0000 7f76deedb700 1 -- 172.21.15.121:0/1118469422 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f76bc05a510 msgr2=0x7f76bc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.133+0000 7f76deedb700 1 --2- 172.21.15.121:0/1118469422 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f76bc05a510 0x7f76bc05c9d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f76cc009c30 tx=0x7f76cc009060).stop 2023-11-15T03:03:31.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.133+0000 7f76deedb700 1 -- 172.21.15.121:0/1118469422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d80c8fb0 msgr2=0x7f76d81439e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.133+0000 7f76deedb700 1 --2- 172.21.15.121:0/1118469422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d80c8fb0 0x7f76d81439e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f76c400c470 tx=0x7f76c4007c40).stop 2023-11-15T03:03:31.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.133+0000 7f76deedb700 1 -- 172.21.15.121:0/1118469422 shutdown_connections 2023-11-15T03:03:31.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.133+0000 7f76deedb700 1 --2- 172.21.15.121:0/1118469422 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f76bc05a510 0x7f76bc05c9d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.133+0000 7f76deedb700 1 --2- 172.21.15.121:0/1118469422 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76d8153670 0x7f76d8147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.133+0000 7f76deedb700 1 --2- 172.21.15.121:0/1118469422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d80c8fb0 0x7f76d81439e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.133+0000 7f76deedb700 1 --2- 172.21.15.121:0/1118469422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d80c04d0 0x7f76d8147060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.133+0000 7f76deedb700 1 -- 172.21.15.121:0/1118469422 >> 172.21.15.121:0/1118469422 conn(0x7f76d8067a60 msgr2=0x7f76d8145630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:31.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.133+0000 7f76deedb700 1 -- 172.21.15.121:0/1118469422 shutdown_connections 2023-11-15T03:03:31.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.133+0000 7f76deedb700 1 -- 172.21.15.121:0/1118469422 wait complete. 2023-11-15T03:03:31.143 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:03:31.144 INFO:tasks.ceph.mds.a.smithi121.stderr: -2983> 2023-11-15T03:03:27.293+0000 7f27100f8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:03:31.166 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata export - 2023-11-15T03:03:31.200 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:03:31.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef1dd3ad00 1 Processor -- start 2023-11-15T03:03:31.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef1dd3ad00 1 -- start start 2023-11-15T03:03:31.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef1dd3ad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5631989167e0 0x563198956370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef1dd3ad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631989d4650 0x563198943350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef1dd3ad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631989d4a30 0x5631989d1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef1dd3ad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56319888b220 con 0x5631989d4a30 2023-11-15T03:03:31.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef1dd3ad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631989d34c0 con 0x5631989d4650 2023-11-15T03:03:31.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef1dd3ad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631989d2e80 con 0x5631989167e0 2023-11-15T03:03:31.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef12ab8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631989d4a30 0x5631989d1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef12ab8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631989d4a30 0x5631989d1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49850/0 (socket says 172.21.15.121:49850) 2023-11-15T03:03:31.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef12ab8700 1 -- 172.21.15.121:0/1960535289 learned_addr learned my addr 172.21.15.121:0/1960535289 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:31.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef122b7700 1 --2- 172.21.15.121:0/1960535289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5631989167e0 0x563198956370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef11ab6700 1 --2- 172.21.15.121:0/1960535289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631989d4650 0x563198943350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef12ab8700 1 -- 172.21.15.121:0/1960535289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5631989167e0 msgr2=0x563198956370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:31.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef12ab8700 1 --2- 172.21.15.121:0/1960535289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5631989167e0 0x563198956370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef12ab8700 1 -- 172.21.15.121:0/1960535289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631989d4650 msgr2=0x563198943350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:31.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef12ab8700 1 --2- 172.21.15.121:0/1960535289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631989d4650 0x563198943350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef12ab8700 1 -- 172.21.15.121:0/1960535289 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef04005db0 con 0x5631989d4a30 2023-11-15T03:03:31.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef11ab6700 1 --2- 172.21.15.121:0/1960535289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631989d4650 0x563198943350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:31.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef12ab8700 1 --2- 172.21.15.121:0/1960535289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631989d4a30 0x5631989d1ae0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7fef0c00b7e0 tx=0x7fef0c005d40).ready entity=mon.0 client_cookie=f3da8e6fd41448a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef10ab4700 1 -- 172.21.15.121:0/1960535289 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00d670 con 0x5631989d4a30 2023-11-15T03:03:31.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef10ab4700 1 -- 172.21.15.121:0/1960535289 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0c00d7d0 con 0x5631989d4a30 2023-11-15T03:03:31.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.377+0000 7fef10ab4700 1 -- 172.21.15.121:0/1960535289 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00db50 con 0x5631989d4a30 2023-11-15T03:03:31.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/1960535289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631989d4a30 msgr2=0x5631989d1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef1dd3ad00 1 --2- 172.21.15.121:0/1960535289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631989d4a30 0x5631989d1ae0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7fef0c00b7e0 tx=0x7fef0c005d40).stop 2023-11-15T03:03:31.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/1960535289 shutdown_connections 2023-11-15T03:03:31.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef1dd3ad00 1 --2- 172.21.15.121:0/1960535289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631989d4a30 0x5631989d1ae0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef1dd3ad00 1 --2- 172.21.15.121:0/1960535289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631989d4650 0x563198943350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef1dd3ad00 1 --2- 172.21.15.121:0/1960535289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5631989167e0 0x563198956370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/1960535289 >> 172.21.15.121:0/1960535289 conn(0x56319895c8f0 msgr2=0x56319895bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:31.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/1960535289 shutdown_connections 2023-11-15T03:03:31.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/1960535289 wait complete. 2023-11-15T03:03:31.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef1dd3ad00 1 Processor -- start 2023-11-15T03:03:31.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef1dd3ad00 1 -- start start 2023-11-15T03:03:31.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef1dd3ad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631989167e0 0x5631989cea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef1dd3ad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5631989d4650 0x56319895bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef1dd3ad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631989d4a30 0x56319895c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef1dd3ad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56319895c6e0 con 0x5631989167e0 2023-11-15T03:03:31.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef1dd3ad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563198913900 con 0x5631989d4a30 2023-11-15T03:03:31.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef1dd3ad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x563198941260 con 0x5631989d4650 2023-11-15T03:03:31.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef122b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631989167e0 0x5631989cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef122b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631989167e0 0x5631989cea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49858/0 (socket says 172.21.15.121:49858) 2023-11-15T03:03:31.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 learned_addr learned my addr 172.21.15.121:0/972298167 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:31.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.381+0000 7fef12ab8700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631989d4a30 0x56319895c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.385+0000 7fef11ab6700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5631989d4650 0x56319895bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.385+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5631989d4650 msgr2=0x56319895bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.385+0000 7fef122b7700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5631989d4650 0x56319895bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.385+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631989d4a30 msgr2=0x56319895c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.385+0000 7fef122b7700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631989d4a30 0x56319895c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.385+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0c00b430 con 0x5631989167e0 2023-11-15T03:03:31.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.385+0000 7fef12ab8700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631989d4a30 0x56319895c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:31.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.385+0000 7fef122b7700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631989167e0 0x5631989cea50 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7fef04010000 tx=0x7fef04016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.385+0000 7fef02ffd700 1 -- 172.21.15.121:0/972298167 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04013440 con 0x5631989167e0 2023-11-15T03:03:31.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.385+0000 7fef02ffd700 1 -- 172.21.15.121:0/972298167 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef040135a0 con 0x5631989167e0 2023-11-15T03:03:31.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.385+0000 7fef02ffd700 1 -- 172.21.15.121:0/972298167 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04013940 con 0x5631989167e0 2023-11-15T03:03:31.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.385+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5631988b4770 con 0x5631989167e0 2023-11-15T03:03:31.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.385+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563198883a70 con 0x5631989167e0 2023-11-15T03:03:31.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.385+0000 7fef02ffd700 1 -- 172.21.15.121:0/972298167 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fef04013cf0 con 0x5631989167e0 2023-11-15T03:03:31.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.389+0000 7fef02ffd700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feef4059020 0x7feef405b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.389+0000 7fef02ffd700 1 -- 172.21.15.121:0/972298167 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fef04074e10 con 0x5631989167e0 2023-11-15T03:03:31.408 INFO:teuthology.orchestra.run.smithi121.stdout:���� 2023-11-15T03:03:31.408 INFO:teuthology.orchestra.run.smithi121.stdout: �� 2023-11-15T03:03:31.409 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:31.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.389+0000 7fef11ab6700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feef4059020 0x7feef405b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.389+0000 7fef1dd3ad00 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x563198942700 0x5631989ee030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.389+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 13.0 13:00000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x563198942700 2023-11-15T03:03:31.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.389+0000 7fef11ab6700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feef4059020 0x7feef405b4e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x56319895a140 tx=0x7fef08000bc0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.389+0000 7fef12ab8700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x563198942700 0x5631989ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.389+0000 7fef12ab8700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x563198942700 0x5631989ee030 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.397+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.397+0000 7fef12ab8700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7fef0c0113f0 0x7fef0c0117d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.397+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:2 13.20 13:04000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0c0113f0 2023-11-15T03:03:31.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.397+0000 7fef122b7700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7fef0c0113f0 0x7fef0c0117d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.397+0000 7fef122b7700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7fef0c0113f0 0x7fef0c0117d0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.405+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(2 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.405+0000 7fef122b7700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7fef0404b690 0x7fef0404a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.405+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:08000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0404b690 2023-11-15T03:03:31.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.405+0000 7fef11ab6700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7fef0404b690 0x7fef0404a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.405+0000 7fef11ab6700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7fef0404b690 0x7fef0404a800 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.405+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(3 [pgnls start_epoch 93 out=97b] v91'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0404b690 2023-11-15T03:03:31.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.405+0000 7fef11ab6700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7fef0800f8a0 0x7fef0800a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.405+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:4 13.30 13:0c000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0800f8a0 2023-11-15T03:03:31.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.405+0000 7fef12ab8700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7fef0800f8a0 0x7fef0800a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.409+0000 7fef12ab8700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7fef0800f8a0 0x7fef0800a340 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.409+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(4 [pgnls start_epoch 93 out=129b] v92'6 uv6 ondisk = 1) v8 ==== 144+0+129 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.409+0000 7fef12ab8700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7fef0c01b910 0x7fef0c01a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.409+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:5 13.8 13:10000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0c01b910 2023-11-15T03:03:31.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.409+0000 7fef122b7700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7fef0c01b910 0x7fef0c01a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.409+0000 7fef122b7700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7fef0c01b910 0x7fef0c01a390 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.417+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(5 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c01b910 2023-11-15T03:03:31.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.417+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:6 13.28 13:14000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0c01b910 2023-11-15T03:03:31.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.417+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(6 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c01b910 2023-11-15T03:03:31.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.417+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:7 13.18 13:18000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x563198942700 2023-11-15T03:03:31.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.417+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(7 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.417+0000 7fef12ab8700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7fef0c02c3e0 0x7fef0c02c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.417+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:8 13.38 13:1c000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0c02c3e0 2023-11-15T03:03:31.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.417+0000 7fef11ab6700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7fef0c02c3e0 0x7fef0c02c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.417+0000 7fef11ab6700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7fef0c02c3e0 0x7fef0c02c800 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.417+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(8 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0c02c3e0 2023-11-15T03:03:31.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.417+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:9 13.4 13:20000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0c0113f0 2023-11-15T03:03:31.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.421+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(9 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.421+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:10 13.24 13:24000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0c0113f0 2023-11-15T03:03:31.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.421+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(10 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.421+0000 7fef122b7700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7fef0404acf0 0x7fef04049910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.421+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:11 13.14 13:28000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0404acf0 2023-11-15T03:03:31.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.421+0000 7fef12ab8700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7fef0404acf0 0x7fef04049910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.421+0000 7fef12ab8700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7fef0404acf0 0x7fef04049910 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.425+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(11 [pgnls start_epoch 93 out=101b] v92'5 uv5 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0404acf0 2023-11-15T03:03:31.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.425+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:12 13.34 13:2c000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0c0113f0 2023-11-15T03:03:31.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.429+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(12 [pgnls start_epoch 93 out=97b] v92'13 uv13 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.429+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:13 13.c 13:30000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0404b690 2023-11-15T03:03:31.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.429+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(13 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0404b690 2023-11-15T03:03:31.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.429+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:14 13.2c 13:34000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0c01b910 2023-11-15T03:03:31.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.429+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(14 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c01b910 2023-11-15T03:03:31.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.429+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:15 13.1c 13:38000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0c0113f0 2023-11-15T03:03:31.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.429+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(15 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.429+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:16 13.3c 13:3c000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0c01b910 2023-11-15T03:03:31.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.429+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(16 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c01b910 2023-11-15T03:03:31.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.429+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:17 13.2 13:40000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0c02c3e0 2023-11-15T03:03:31.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.433+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(17 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0c02c3e0 2023-11-15T03:03:31.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.433+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:18 13.22 13:44000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0404b690 2023-11-15T03:03:31.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.433+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(18 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0404b690 2023-11-15T03:03:31.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.433+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:19 13.12 13:48000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0404acf0 2023-11-15T03:03:31.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.433+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(19 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0404acf0 2023-11-15T03:03:31.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.433+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:20 13.32 13:4c000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0c01b910 2023-11-15T03:03:31.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.433+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(20 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c01b910 2023-11-15T03:03:31.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.433+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:21 13.a 13:50000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0c01b910 2023-11-15T03:03:31.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.433+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(21 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c01b910 2023-11-15T03:03:31.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.433+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:22 13.2a 13:54000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0404b690 2023-11-15T03:03:31.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.433+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(22 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0404b690 2023-11-15T03:03:31.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.433+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:23 13.1a 13:58000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0404b690 2023-11-15T03:03:31.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.433+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(23 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0404b690 2023-11-15T03:03:31.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.433+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:24 13.3a 13:5c000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0c02c3e0 2023-11-15T03:03:31.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.437+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(24 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0c02c3e0 2023-11-15T03:03:31.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.437+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.6 13:60000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0800f8a0 2023-11-15T03:03:31.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.437+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(25 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.437+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:26 13.26 13:64000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x563198942700 2023-11-15T03:03:31.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.437+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(26 [pgnls start_epoch 93 out=77b] v92'2 uv2 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.437+0000 7fef12ab8700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7fef0c03b040 0x7fef0c02f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.437+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] -- osd_op(unknown.0.0:27 13.16 13:68000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0c03b040 2023-11-15T03:03:31.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.437+0000 7fef122b7700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7fef0c03b040 0x7fef0c02f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.437+0000 7fef122b7700 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7fef0c03b040 0x7fef0c02f970 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.445+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.3 v2:172.21.15.121:6820/33135 1 ==== osd_op_reply(27 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c03b040 2023-11-15T03:03:31.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.445+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:28 13.36 13:6c000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x563198917c90 con 0x7fef0c0113f0 2023-11-15T03:03:31.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.445+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(28 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.445+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:29 13.e 13:70000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x563198942700 2023-11-15T03:03:31.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.445+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(29 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.445+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:30 13.2e 13:74000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0c01b910 2023-11-15T03:03:31.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.445+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.2 v2:172.21.15.121:6808/33134 7 ==== osd_op_reply(30 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c01b910 2023-11-15T03:03:31.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.445+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.1e 13:78000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0800f8a0 2023-11-15T03:03:31.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.445+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(31 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.445+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:32 13.3e 13:7c000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0404b690 2023-11-15T03:03:31.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.445+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(32 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0404b690 2023-11-15T03:03:31.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.449+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:33 13.1 13:80000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0c0113f0 2023-11-15T03:03:31.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.449+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(33 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.449+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:34 13.21 13:84000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0800f8a0 2023-11-15T03:03:31.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.449+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(34 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.449+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:35 13.11 13:88000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0404b690 2023-11-15T03:03:31.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.449+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(35 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0404b690 2023-11-15T03:03:31.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.449+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] -- osd_op(unknown.0.0:36 13.31 13:8c000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0c03b040 2023-11-15T03:03:31.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.449+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.3 v2:172.21.15.121:6820/33135 2 ==== osd_op_reply(36 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c03b040 2023-11-15T03:03:31.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.449+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:37 13.9 13:90000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0c02c3e0 2023-11-15T03:03:31.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.453+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(37 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0c02c3e0 2023-11-15T03:03:31.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.453+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:38 13.29 13:94000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0c0113f0 2023-11-15T03:03:31.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.453+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(38 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.453+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] -- osd_op(unknown.0.0:39 13.19 13:98000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0c03b040 2023-11-15T03:03:31.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.453+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.3 v2:172.21.15.121:6820/33135 3 ==== osd_op_reply(39 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c03b040 2023-11-15T03:03:31.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.453+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:40 13.39 13:9c000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0c01b910 2023-11-15T03:03:31.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.453+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.2 v2:172.21.15.121:6808/33134 8 ==== osd_op_reply(40 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c01b910 2023-11-15T03:03:31.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.453+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:41 13.5 13:a0000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0800f8a0 2023-11-15T03:03:31.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.453+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(41 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.453+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:42 13.25 13:a4000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0800f8a0 2023-11-15T03:03:31.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.453+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(42 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.453+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:43 13.15 13:a8000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0800f8a0 2023-11-15T03:03:31.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.457+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(43 [pgnls start_epoch 93 out=97b] v91'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.457+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:44 13.35 13:ac000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0404b690 2023-11-15T03:03:31.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.457+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(44 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0404b690 2023-11-15T03:03:31.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.457+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:45 13.d 13:b0000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0800f8a0 2023-11-15T03:03:31.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.457+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(45 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.457+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:46 13.2d 13:b4000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0404acf0 2023-11-15T03:03:31.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.457+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(46 [pgnls start_epoch 93 out=74b] v91'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0404acf0 2023-11-15T03:03:31.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.457+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:47 13.1d 13:b8000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x563198942700 2023-11-15T03:03:31.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.457+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(47 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.457+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:48 13.3d 13:bc000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x563198942700 2023-11-15T03:03:31.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.461+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(48 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.461+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:49 13.3 13:c0000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0404acf0 2023-11-15T03:03:31.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.461+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(49 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0404acf0 2023-11-15T03:03:31.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.461+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:50 13.23 13:c4000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0404b690 2023-11-15T03:03:31.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.461+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(50 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0404b690 2023-11-15T03:03:31.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.461+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:51 13.13 13:c8000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x563198942700 2023-11-15T03:03:31.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.461+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(51 [pgnls start_epoch 93 out=74b] v92'2 uv2 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.461+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:52 13.33 13:cc000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x563198942700 2023-11-15T03:03:31.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.461+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(52 [pgnls start_epoch 93 out=73b] v91'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.461+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:53 13.b 13:d0000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0c02c3e0 2023-11-15T03:03:31.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.465+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.5 v2:172.21.15.190:6800/33349 5 ==== osd_op_reply(53 [pgnls start_epoch 93 out=76b] v92'2 uv2 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0c02c3e0 2023-11-15T03:03:31.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.465+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.2b 13:d4000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0c01b910 2023-11-15T03:03:31.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.465+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.2 v2:172.21.15.121:6808/33134 9 ==== osd_op_reply(54 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c01b910 2023-11-15T03:03:31.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.465+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:55 13.1b 13:d8000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0800f8a0 2023-11-15T03:03:31.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.465+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(55 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.465+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:56 13.3b 13:dc000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0800f8a0 2023-11-15T03:03:31.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.465+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(56 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.465+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:57 13.7 13:e0000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x563198942700 2023-11-15T03:03:31.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.469+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(57 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.469+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:58 13.27 13:e4000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0404b690 2023-11-15T03:03:31.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.469+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(58 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0404b690 2023-11-15T03:03:31.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.469+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:59 13.17 13:e8000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0c02c3e0 2023-11-15T03:03:31.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.469+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.5 v2:172.21.15.190:6800/33349 6 ==== osd_op_reply(59 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0c02c3e0 2023-11-15T03:03:31.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.469+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:60 13.37 13:ec000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0c02c3e0 2023-11-15T03:03:31.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.469+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.5 v2:172.21.15.190:6800/33349 7 ==== osd_op_reply(60 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0c02c3e0 2023-11-15T03:03:31.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.469+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:61 13.f 13:f0000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0c0113f0 2023-11-15T03:03:31.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.469+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(61 [pgnls start_epoch 93 out=73b] v91'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.469+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:62 13.2f 13:f4000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0800f8a0 2023-11-15T03:03:31.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.473+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(62 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.473+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:63 13.1f 13:f8000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0c0113f0 2023-11-15T03:03:31.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.473+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(63 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.473+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:64 13.3f 13:fc000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7fef0405d040 con 0x7fef0c0113f0 2023-11-15T03:03:31.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.473+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(64 [pgnls start_epoch 93 out=101b] v92'5 uv5 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.473+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:65 13.10 13:09401a81:::601.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x5631989ee520 con 0x7fef0404b690 2023-11-15T03:03:31.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.473+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(65 601.00000000 [read 0~0] v0'0 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0404b690 2023-11-15T03:03:31.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.473+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:66 13.10 13:09401a81:::601.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x5631989416f0 con 0x7fef0404b690 2023-11-15T03:03:31.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.477+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(66 601.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0404b690 2023-11-15T03:03:31.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.477+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:67 13.10 13:09401a81:::601.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198941aa0 con 0x7fef0404b690 2023-11-15T03:03:31.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.477+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.1 v2:172.21.15.121:6800/33131 13 ==== osd_op_reply(67 601.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0404b690 2023-11-15T03:03:31.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.477+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:68 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198941ea0 con 0x7fef0404b690 2023-11-15T03:03:31.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.477+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.1 v2:172.21.15.121:6800/33131 14 ==== osd_op_reply(68 601.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0404b690 2023-11-15T03:03:31.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.477+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:69 13.10 13:0bd6d154:::602.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198942250 con 0x7fef0404b690 2023-11-15T03:03:31.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.477+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.1 v2:172.21.15.121:6800/33131 15 ==== osd_op_reply(69 602.00000000 [read 0~0] v0'0 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0404b690 2023-11-15T03:03:31.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.477+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:70 13.10 13:0bd6d154:::602.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a087e0 con 0x7fef0404b690 2023-11-15T03:03:31.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.477+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.1 v2:172.21.15.121:6800/33131 16 ==== osd_op_reply(70 602.00000000 [getxattrs out=124b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0404b690 2023-11-15T03:03:31.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.477+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:71 13.10 13:0bd6d154:::602.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a08b90 con 0x7fef0404b690 2023-11-15T03:03:31.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.481+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.1 v2:172.21.15.121:6800/33131 17 ==== osd_op_reply(71 602.00000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0404b690 2023-11-15T03:03:31.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.481+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:72 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a09f50 con 0x7fef0404b690 2023-11-15T03:03:31.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.481+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.1 v2:172.21.15.121:6800/33131 18 ==== osd_op_reply(72 602.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0404b690 2023-11-15T03:03:31.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.481+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.30 13:0cf6ad0f:::10000000000.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a08f40 con 0x7fef0800f8a0 2023-11-15T03:03:31.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.481+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(73 10000000000.00000000 [read 0~0] v0'0 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.481+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.30 13:0cf6ad0f:::10000000000.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a092f0 con 0x7fef0800f8a0 2023-11-15T03:03:31.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.481+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(74 10000000000.00000000 [getxattrs out=124b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+124 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.481+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a096a0 con 0x7fef0800f8a0 2023-11-15T03:03:31.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.481+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(75 10000000000.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.481+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0aa60 con 0x7fef0800f8a0 2023-11-15T03:03:31.491 INFO:teuthology.orchestra.run.smithi121.stdout: ��*$- 601.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray1 ���� �� ((88���� �� ��*$- 602.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray2 ���� �� ((88���� �� ��2,510000000000.00000000�������������������!�������������������������� ���~_layout��������_parent>8subdir ���� ��  (5Te;L(5Te;L8`5Tem��8`5Tem������ �� sixmegs_head�i���5Tem������@@`��������5Tem��5Te;L(8`5Tem��8`5Tem�� 2023-11-15T03:03:31.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.481+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(76 10000000000.00000000 [omap-get-vals out=513b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+513 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.485+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.30 13:0d82a743:::600.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a09a50 con 0x7fef0800f8a0 2023-11-15T03:03:31.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.485+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(77 600.00000000 [read 0~0] v0'0 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.485+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.30 13:0d82a743:::600.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a09e50 con 0x7fef0800f8a0 2023-11-15T03:03:31.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.485+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(78 600.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.485+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.30 13:0d82a743:::600.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0a250 con 0x7fef0800f8a0 2023-11-15T03:03:31.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.485+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(79 600.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.485+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:80 13.30 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0a600 con 0x7fef0800f8a0 2023-11-15T03:03:31.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.485+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(80 600.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.485+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:81 13.30 13:0d89b25e:::603.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0a600 con 0x7fef0800f8a0 2023-11-15T03:03:31.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.489+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(81 603.00000000 [read 0~0] v0'0 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.489+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:82 13.30 13:0d89b25e:::603.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0a600 con 0x7fef0800f8a0 2023-11-15T03:03:31.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.489+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(82 603.00000000 [getxattrs out=124b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.489+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:83 13.30 13:0d89b25e:::603.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0a600 con 0x7fef0800f8a0 2023-11-15T03:03:31.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.489+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(83 603.00000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.489+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:84 13.30 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0a600 con 0x7fef0800f8a0 2023-11-15T03:03:31.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.489+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.489+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:85 13.14 13:287034c4:::1.00000000.inode:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0a600 con 0x7fef0404acf0 2023-11-15T03:03:31.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.489+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(85 1.00000000.inode [read 0~554 out=554b] v0'0 uv3 ondisk = 0) v8 ==== 160+0+554 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0404acf0 2023-11-15T03:03:31.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.489+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:86 13.14 13:287034c4:::1.00000000.inode:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0a600 con 0x7fef0404acf0 2023-11-15T03:03:31.500 INFO:teuthology.orchestra.run.smithi121.stdout:��������5Te;L���������������� �� ��*$- 600.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray0 ���� �� ((88���� �� ��*$- 603.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray3 ���� �� ((88���� �� ��.(11.00000000.inode�������������������!�������������������������� ��D>**ceph fs volume v011 �5Te����C@@��������5Te���(5Te���85Te���8�4TeT<9/�4TeT<9/����>8������������ �� 2023-11-15T03:03:31.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.493+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(86 1.00000000.inode [getxattrs out=4b] v0'0 uv3 ondisk = 0) v8 ==== 160+0+4 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0404acf0 2023-11-15T03:03:31.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.493+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:87 13.14 13:287034c4:::1.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0a600 con 0x7fef0404acf0 2023-11-15T03:03:31.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.493+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(87 1.00000000.inode [omap-get-header] v0'0 uv3 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0404acf0 2023-11-15T03:03:31.502 INFO:teuthology.orchestra.run.smithi121.stdout:���� �� 2023-11-15T03:03:31.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.493+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:88 13.14 13:287034c4:::1.00000000.inode:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0a600 con 0x7fef0404acf0 2023-11-15T03:03:31.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.493+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(88 1.00000000.inode [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 160+0+5 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0404acf0 2023-11-15T03:03:31.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.493+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:89 13.14 13:292cf221:::200.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0a600 con 0x7fef0404acf0 2023-11-15T03:03:31.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.493+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(89 200.00000000 [read 0~90 out=90b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0404acf0 2023-11-15T03:03:31.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.493+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:90 13.14 13:292cf221:::200.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0a600 con 0x7fef0404acf0 2023-11-15T03:03:31.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.493+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(90 200.00000000 [getxattrs out=4b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+4 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0404acf0 2023-11-15T03:03:31.505 INFO:teuthology.orchestra.run.smithi121.stdout:���� �� ��*$- 200.00000000�������������������!�������������������������� ��tnZZTceph fs volume v011@�8@�8@d@@@@ ���� �� 2023-11-15T03:03:31.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.493+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:91 13.14 13:292cf221:::200.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0a600 con 0x7fef0404acf0 2023-11-15T03:03:31.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.493+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(91 200.00000000 [omap-get-header] v0'0 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0404acf0 2023-11-15T03:03:31.507 INFO:teuthology.orchestra.run.smithi121.stdout:���� �� 2023-11-15T03:03:31.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.493+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:92 13.14 13:292cf221:::200.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0a600 con 0x7fef0404acf0 2023-11-15T03:03:31.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.493+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(92 200.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0404acf0 2023-11-15T03:03:31.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.493+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:93 13.34 13:2e2fa760:::200.00000001:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0a600 con 0x7fef0c0113f0 2023-11-15T03:03:31.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.497+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(93 200.00000001 [read 0~16484 out=16484b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+16484 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.509 INFO:teuthology.orchestra.run.smithi121.stdout:���� �� ��*$- 200.00000001�������������������!�������������������������� ��~@x@d@d@��XS&YA1h^T�4TeVVq/2 ((88 2 ( 2023-11-15T03:03:31.509 INFO:teuthology.orchestra.run.smithi121.stdout:( 2023-11-15T03:03:31.510 INFO:teuthology.orchestra.run.smithi121.stdout:8 2023-11-15T03:03:31.510 INFO:teuthology.orchestra.run.smithi121.stdout:�4Te��d/8 2023-11-15T03:03:31.514 INFO:teuthology.orchestra.run.smithi121.stdout:�4Te��d/ ��������@@��XS&YA1�� 2023-11-15T03:03:31.514 INFO:teuthology.orchestra.run.smithi121.stdout:�5Te8�x �,3��y� entity_id0hostname smithi121kernel_version6.6.0-rc5-g3b796cfe2969root/�� ���|@��XS&YA1���5Te�xZ n��������~@��XS&YA1���5Te|�a setattr� '���������5Te��< �C@@���������4TeT<9/(85Te��< 8�4TeT<9/�4TeT<9/����>8�����������������*@��XS&YA1���5Te~�mkdirY' (5Te���(5Te���85Te���85Te��� �subdir���������5Te����A������������������5Te���5Te���(88��������5Te���������������2 ((88  '���������5Te����C@@��������5Te���(5Te���85Te���8�4TeT<9/�4TeT<9/����>8����������� ��������=@��XS&YA1}si5Te��jopenc4' (5Te���(5Te���85Te;L85Te��� �subdir���������5Te;L�A������������������5Te;L5Te���(5Te;L85Te;L85Te;L��������5Te���������������J (5Te;L(5Te;L85Te;L85Te;L$ sixmegs���������5Te;L����@@��������5Te;L5Te;L�@(88��������5Te;L�������������� ���������@��XS&YA1�xn5Te?^� 2023-11-15T03:03:31.515 INFO:teuthology.orchestra.run.smithi121.stdout:cap update4' (5Te���(5Te���8 5Te�]@85Te��� �subdir���������5Te;L�A������������������5Te;L5Te���(5Te;L8 5Te�]@8 5Te�]@��������5Te���������������J (5Te;L(5Te;L8 5Te�]@8 5Te�]@$ sixmegs���������5Te�]@����@@ ��������5Te�]@5Te;L��(8 5Te�]@8 5Te�]@��������5Te;L����������������������@��XS&YA1�xn5Te�� 2023-11-15T03:03:31.515 INFO:teuthology.orchestra.run.smithi121.stdout:cap update4'  (5Te���(5Te���8P5TeɈq85Te��� �subdir���������5Te;L�A������������������5Te;L5Te���(5Te;L8P5TeɈq8P5TeɈq��������5Te���������������J (5Te;L(5Te;L8P5TeɈq8P5TeɈq$ sixmegs���������5TeɈq����@@P��������5TeɈq5Te;L��(8P5TeɈq8P5TeɈq��������5Te;L���������������������@��XS&YA1�xn5Te�"11 2023-11-15T03:03:31.515 INFO:teuthology.orchestra.run.smithi121.stdout:cap update4'  (5Te���(5Te���8`5Tem��85Te��� �subdir�������� 2023-11-15T03:03:31.516 INFO:teuthology.orchestra.run.smithi121.stdout:�5Te;L�A������������������5Te;L5Te���(5Te;L8`5Tem��8`5Tem�� 2023-11-15T03:03:31.516 INFO:teuthology.orchestra.run.smithi121.stdout:��������5Te���������������J  (5Te;L(5Te;L8`5Tem��8`5Tem��$ sixmegs���������5Tem������@@`��������5Tem��5Te;L�(8`5Tem��8`5Tem����������5Te;L����������������������&@��XS&YA1�� 2023-11-15T03:03:31.516 INFO:teuthology.orchestra.run.smithi121.stdout:�5Te���1�,3��y���F.@��XS&YA1? 5 + 5Te�3check_inode_max_size�'  (5Te���(5Te���8`5Tem��85Te��� �subdir�������� �5Te;L�A������������������5Te;L5Te���(5Te;L8`5Tem��8`5Tem�� ��������5Te���������������  (5Te;L(5Te;L8`5Tem��8`5Tem��� �sixmegs�������� 2023-11-15T03:03:31.516 INFO:teuthology.orchestra.run.smithi121.stdout:�5Tem������@@`��������5Tem��5Te;L(8`5Tem��8`5Tem�� 2023-11-15T03:03:31.517 INFO:teuthology.orchestra.run.smithi121.stdout:��������5Te;L������������ �sixmegs�������� 2023-11-15T03:03:31.517 INFO:teuthology.orchestra.run.smithi121.stdout:�5Tem������@@`��������5Tem��5Te;L(8`5Tem��8`5Tem�� 2023-11-15T03:03:31.517 INFO:teuthology.orchestra.run.smithi121.stdout:��������5Te;L���������������������.@��XS&YA18.$�F8@��XS&YA1h^T5Te"�2  (5Te���(5Te���8`5Tem��85Te��� 2 ( 2023-11-15T03:03:31.517 INFO:teuthology.orchestra.run.smithi121.stdout:( 2023-11-15T03:03:31.517 INFO:teuthology.orchestra.run.smithi121.stdout:8 2023-11-15T03:03:31.518 INFO:teuthology.orchestra.run.smithi121.stdout:�4Te��d/8 2023-11-15T03:03:31.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.497+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:94 13.34 13:2e2fa760:::200.00000001:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0ae10 con 0x7fef0c0113f0 2023-11-15T03:03:31.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.497+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(94 200.00000001 [getxattrs out=4b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+4 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.522 INFO:teuthology.orchestra.run.smithi121.stdout:�4Te��d/ ��������@ �8@��XS&YA1B8.5Te� rscatter_writebehind�2 (5Te���(5Te���8`5Tem��8`5Tem��  '���������5Te����C@@��������5Te���(5Te���8`5Tem��8�4TeT<9/�4TeT<9/����>8����������������<@���� �� 2023-11-15T03:03:31.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.497+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:95 13.34 13:2e2fa760:::200.00000001:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b1c0 con 0x7fef0c0113f0 2023-11-15T03:03:31.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.497+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(95 200.00000001 [omap-get-header] v0'0 uv13 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.523 INFO:teuthology.orchestra.run.smithi121.stdout:���� �� 2023-11-15T03:03:31.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.497+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:96 13.34 13:2e2fa760:::200.00000001:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0c580 con 0x7fef0c0113f0 2023-11-15T03:03:31.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.497+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(96 200.00000001 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.497+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:97 13.34 13:2f55791f:::606.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b570 con 0x7fef0c0113f0 2023-11-15T03:03:31.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.497+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(97 606.00000000 [read 0~0] v0'0 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.497+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:98 13.34 13:2f55791f:::606.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c0113f0 2023-11-15T03:03:31.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.501+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(98 606.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.501+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:99 13.34 13:2f55791f:::606.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c0113f0 2023-11-15T03:03:31.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.501+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(99 606.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.501+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:100 13.34 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c0113f0 2023-11-15T03:03:31.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.501+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(100 606.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.501+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:101 13.3c 13:3ed09add:::607.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c01b910 2023-11-15T03:03:31.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.501+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.2 v2:172.21.15.121:6808/33134 10 ==== osd_op_reply(101 607.00000000 [read 0~0] v0'0 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c01b910 2023-11-15T03:03:31.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.501+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:102 13.3c 13:3ed09add:::607.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c01b910 2023-11-15T03:03:31.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.501+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.2 v2:172.21.15.121:6808/33134 11 ==== osd_op_reply(102 607.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c01b910 2023-11-15T03:03:31.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.501+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:103 13.3c 13:3ed09add:::607.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c01b910 2023-11-15T03:03:31.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.501+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.2 v2:172.21.15.121:6808/33134 12 ==== osd_op_reply(103 607.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c01b910 2023-11-15T03:03:31.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.501+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:104 13.3c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c01b910 2023-11-15T03:03:31.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.505+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.2 v2:172.21.15.121:6808/33134 13 ==== osd_op_reply(104 607.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c01b910 2023-11-15T03:03:31.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.505+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.26 13:654134d2:::mds0_openfiles.0:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.505+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(105 mds0_openfiles.0 [read 0~0] v0'0 uv2 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.505+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.26 13:654134d2:::mds0_openfiles.0:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.536 INFO:teuthology.orchestra.run.smithi121.stdout:���� �� ��*$- 606.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray6 ���� �� ((88���� �� ��*$- 607.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray7 ���� �� ((88���� �� ��.(1mds0_openfiles.0�������������������!�������������������������� �� 2023-11-15T03:03:31.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.505+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(106 mds0_openfiles.0 [getxattrs out=4b] v0'0 uv2 ondisk = 0) v8 ==== 160+0+4 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.505+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.26 13:654134d2:::mds0_openfiles.0:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.505+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(107 mds0_openfiles.0 [omap-get-header out=42b] v0'0 uv2 ondisk = 0) v8 ==== 160+0+42 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.505+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.26 13:654134d2:::mds0_openfiles.0:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.509+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(108 mds0_openfiles.0 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 160+0+5 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.509+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.e 13:717a0223:::608.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.509+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(109 608.00000000 [read 0~0] v0'0 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.509+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.e 13:717a0223:::608.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.509+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(110 608.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.509+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:111 13.e 13:717a0223:::608.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.509+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(111 608.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.509+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:112 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.509+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(112 608.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.509+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:113 13.15 13:a93a17c2:::604.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0800f8a0 2023-11-15T03:03:31.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.513+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(113 604.00000000 [read 0~0] v0'0 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.513+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:114 13.15 13:a93a17c2:::604.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0800f8a0 2023-11-15T03:03:31.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.513+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(114 604.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.513+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:115 13.15 13:a93a17c2:::604.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0800f8a0 2023-11-15T03:03:31.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.513+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(115 604.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.513+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:116 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0800f8a0 2023-11-15T03:03:31.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.513+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(116 604.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.513+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:117 13.15 13:aa448500:::500.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0800f8a0 2023-11-15T03:03:31.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.513+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(117 500.00000000 [read 0~90 out=90b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.517+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:118 13.15 13:aa448500:::500.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0800f8a0 2023-11-15T03:03:31.548 INFO:teuthology.orchestra.run.smithi121.stdout:���� ��4.*ceph fs volume v011 ���� �� ��*$- 608.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray8 ���� �� ((88���� �� ��*$- 604.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray4 ���� �� ((88���� �� ��*$- 500.00000000�������������������!�������������������������� ��tnZZTceph fs volume v011@@@@@@ ���� �� 2023-11-15T03:03:31.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.517+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(118 500.00000000 [getxattrs out=4b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+4 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.517+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:119 13.15 13:aa448500:::500.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0800f8a0 2023-11-15T03:03:31.550 INFO:teuthology.orchestra.run.smithi121.stdout:���� �� 2023-11-15T03:03:31.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.517+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(119 500.00000000 [omap-get-header] v0'0 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.517+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:120 13.15 13:aa448500:::500.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0800f8a0 2023-11-15T03:03:31.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.517+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.4 v2:172.21.15.190:6816/33348 31 ==== osd_op_reply(120 500.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0800f8a0 2023-11-15T03:03:31.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.517+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:121 13.2d 13:b50e409b:::mds_snaptable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0404acf0 2023-11-15T03:03:31.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.517+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(121 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0404acf0 2023-11-15T03:03:31.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.517+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:122 13.2d 13:b50e409b:::mds_snaptable:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0404acf0 2023-11-15T03:03:31.553 INFO:teuthology.orchestra.run.smithi121.stdout:���� �� ��+%. mds_snaptable�������������������!�������������������������� ��`ZFF4���� �� 2023-11-15T03:03:31.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.521+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(122 mds_snaptable [getxattrs out=4b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+4 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0404acf0 2023-11-15T03:03:31.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.521+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:123 13.2d 13:b50e409b:::mds_snaptable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0404acf0 2023-11-15T03:03:31.555 INFO:teuthology.orchestra.run.smithi121.stdout:���� �� 2023-11-15T03:03:31.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.521+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(123 mds_snaptable [omap-get-header] v0'0 uv1 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0404acf0 2023-11-15T03:03:31.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.521+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:124 13.2d 13:b50e409b:::mds_snaptable:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0404acf0 2023-11-15T03:03:31.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.521+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.0 v2:172.21.15.121:6816/33133 16 ==== osd_op_reply(124 mds_snaptable [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+5 (crc 0 0 0) 0x7fef0c017020 con 0x7fef0404acf0 2023-11-15T03:03:31.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.521+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:125 13.1d 13:b871830b:::605.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.521+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(125 605.00000000 [read 0~0] v0'0 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.521+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:126 13.1d 13:b871830b:::605.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.521+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(126 605.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.521+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:127 13.1d 13:b871830b:::605.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.521+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(127 605.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.521+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:128 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.525+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(128 605.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.525+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:129 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.525+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(129 mds0_inotable [read 0~34 out=34b] v0'0 uv2 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.525+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:130 13.13 13:c91d4a1d:::mds0_inotable:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.561 INFO:teuthology.orchestra.run.smithi121.stdout:���� �� ��*$- 605.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray5 ���� �� ((88���� �� ��+%. mds0_inotable�������������������!�������������������������� ��<6""��������� �� 2023-11-15T03:03:31.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.525+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(130 mds0_inotable [getxattrs out=4b] v0'0 uv2 ondisk = 0) v8 ==== 157+0+4 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.525+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:131 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.563 INFO:teuthology.orchestra.run.smithi121.stdout:���� �� 2023-11-15T03:03:31.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.525+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(131 mds0_inotable [omap-get-header] v0'0 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.525+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:132 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.525+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(132 mds0_inotable [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 157+0+5 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.525+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:133 13.33 13:cd5a64a3:::100.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.529+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(133 100.00000000 [read 0~0] v0'0 uv1 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.529+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:134 13.33 13:cd5a64a3:::100.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.566 INFO:teuthology.orchestra.run.smithi121.stdout:���� �� ��*$- 100.00000000�������������������!�������������������������� �� 2023-11-15T03:03:31.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.529+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(134 100.00000000 [getxattrs out=4b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+4 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.529+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:135 13.33 13:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.568 INFO:teuthology.orchestra.run.smithi121.stdout:���� �� ( 2023-11-15T03:03:31.568 INFO:teuthology.orchestra.run.smithi121.stdout:( 2023-11-15T03:03:31.568 INFO:teuthology.orchestra.run.smithi121.stdout:8 2023-11-15T03:03:31.568 INFO:teuthology.orchestra.run.smithi121.stdout:�4Te��d/8 2023-11-15T03:03:31.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.529+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(135 100.00000000 [omap-get-header out=274b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.529+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:136 13.33 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.529+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(136 100.00000000 [omap-get-vals out=5075b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+5075 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.570 INFO:teuthology.orchestra.run.smithi121.stdout:�4Te��d/���� ���� 2023-11-15T03:03:31.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.529+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:137 13.b 13:d0630e4c:::mds0_sessionmap:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0bd70 con 0x7fef0c02c3e0 2023-11-15T03:03:31.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.529+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.5 v2:172.21.15.190:6800/33349 8 ==== osd_op_reply(137 mds0_sessionmap [read 0~0] v0'0 uv2 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0c02c3e0 2023-11-15T03:03:31.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.529+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:138 13.b 13:d0630e4c:::mds0_sessionmap:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0bd70 con 0x7fef0c02c3e0 2023-11-15T03:03:31.571 INFO:teuthology.orchestra.run.smithi121.stdout: stray0_head�i����4Te%sA/@A�����������������4Te%sA/(8�4Te%sA/8�4Te%sA/�4Te%sA/������������ stray1_head�i����4Te��D/@A�����������������4Te��D/(8�4Te��D/8�4Te��D/�4Te��D/������������ stray2_head�i����4Te>�H/@A�����������������4Te>�H/(8�4Te>�H/8�4Te>�H/�4Te>�H/������������ stray3_head�i����4Te'VP/@A�����������������4Te'VP/(8�4Te'VP/8�4Te'VP/�4Te'VP/������������ stray4_head�i����4Te��T/@A�����������������4Te��T/(8�4Te��T/8�4Te��T/�4Te��T/������������ stray5_head�i����4Te��W/@A�����������������4Te��W/(8�4Te��W/8�4Te��W/�4Te��W/������������ stray6_head�i����4TeJ\/@A�����������������4TeJ\/(8�4TeJ\/8�4TeJ\/�4TeJ\/������������ stray7_head�i����4TeS�^/@A�����������������4TeS�^/(8�4TeS�^/8�4TeS�^/�4TeS�^/������������ stray8_head�i����4TeJ�a/@A�����������������4TeJ�a/(8�4TeJ�a/8�4TeJ�a/�4TeJ�a/������������ stray9_head�i��� �4Te��d/@A�����������������4Te��d/(8�4Te��d/8�4Te��d/�4Te��d/���������������� �� ��-'0mds0_sessionmap�������������������!�������������������������� �� 2023-11-15T03:03:31.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.533+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.5 v2:172.21.15.190:6800/33349 9 ==== osd_op_reply(138 mds0_sessionmap [getxattrs out=4b] v0'0 uv2 ondisk = 0) v8 ==== 159+0+4 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0c02c3e0 2023-11-15T03:03:31.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.533+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:139 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0bd70 con 0x7fef0c02c3e0 2023-11-15T03:03:31.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.533+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.5 v2:172.21.15.190:6800/33349 10 ==== osd_op_reply(139 mds0_sessionmap [omap-get-header out=14b] v0'0 uv2 ondisk = 0) v8 ==== 159+0+14 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0c02c3e0 2023-11-15T03:03:31.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.533+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:140 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c02c3e0 2023-11-15T03:03:31.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.533+0000 7fef11ab6700 1 -- 172.21.15.121:0/972298167 <== osd.5 v2:172.21.15.190:6800/33349 11 ==== osd_op_reply(140 mds0_sessionmap [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 159+0+5 (crc 0 0 0) 0x7fef0800b040 con 0x7fef0c02c3e0 2023-11-15T03:03:31.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.533+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:141 13.7 13:e0b41d06:::609.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.533+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(141 609.00000000 [read 0~0] v0'0 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.533+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:142 13.7 13:e0b41d06:::609.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.537+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(142 609.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.537+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:143 13.7 13:e0b41d06:::609.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.537+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(143 609.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.537+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:144 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x563198942700 2023-11-15T03:03:31.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.537+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 <== osd.7 v2:172.21.15.190:6808/33347 33 ==== osd_op_reply(144 609.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fef0c017020 con 0x563198942700 2023-11-15T03:03:31.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.537+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:145 13.f 13:f1f69726:::400.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c0113f0 2023-11-15T03:03:31.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.537+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(145 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.537+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:146 13.f 13:f1f69726:::400.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c0113f0 2023-11-15T03:03:31.585 INFO:teuthology.orchestra.run.smithi121.stdout:���� ������ �� ��*$- 609.00000000�������������������!�������������������������� ���~_layout��������_parent>8 stray9 ���� �� ((88���� �� ��*$- 400.00000000�������������������!�������������������������� ��0*���� �� 2023-11-15T03:03:31.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.541+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(146 400.00000000 [getxattrs out=4b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+4 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.541+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:147 13.f 13:f1f69726:::400.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c0113f0 2023-11-15T03:03:31.587 INFO:teuthology.orchestra.run.smithi121.stdout:���� �� 2023-11-15T03:03:31.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.541+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(147 400.00000000 [omap-get-header] v0'0 uv1 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.541+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:148 13.f 13:f1f69726:::400.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c0113f0 2023-11-15T03:03:31.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.541+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(148 400.00000000 [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.541+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:149 13.3f 13:fe07bba1:::100.00000000.inode:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c0113f0 2023-11-15T03:03:31.589 INFO:teuthology.orchestra.run.smithi121.stdout:���� �� ��0*3100.00000000.inode�������������������!�������������������������� ��D>**ceph fs volume v011 ��4Teħ@/@A�����������������4Teħ@/( 2023-11-15T03:03:31.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.541+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(149 100.00000000.inode [read 0~554 out=554b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+554 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.541+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:150 13.3f 13:fe07bba1:::100.00000000.inode:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c0113f0 2023-11-15T03:03:31.590 INFO:teuthology.orchestra.run.smithi121.stdout:8 �4Te��d/8 �4Te��d/�4Teħ@/����>8������������ �� 2023-11-15T03:03:31.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.545+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(150 100.00000000.inode [getxattrs out=4b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+4 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.545+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:151 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c0113f0 2023-11-15T03:03:31.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.545+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(151 100.00000000.inode [omap-get-header] v0'0 uv3 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.592 INFO:teuthology.orchestra.run.smithi121.stdout:���� �� 2023-11-15T03:03:31.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.545+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:152 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c0113f0 2023-11-15T03:03:31.593 INFO:teuthology.orchestra.run.smithi121.stdout:���� �� ��("+ 2023-11-15T03:03:31.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.545+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(152 100.00000000.inode [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+5 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.545+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:153 13.3f 13:ff5b34d6:::1.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c0113f0 2023-11-15T03:03:31.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.545+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(153 1.00000000 [read 0~0] v0'0 uv5 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.545+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:154 13.3f 13:ff5b34d6:::1.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c0113f0 2023-11-15T03:03:31.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.545+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 29 ==== osd_op_reply(154 1.00000000 [getxattrs out=92b] v0'0 uv5 ondisk = 0) v8 ==== 154+0+92 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:155 13.3f 13:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c0113f0 2023-11-15T03:03:31.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 30 ==== osd_op_reply(155 1.00000000 [omap-get-header out=274b] v0'0 uv5 ondisk = 0) v8 ==== 154+0+274 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:156 13.3f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563198a0b970 con 0x7fef0c0113f0 2023-11-15T03:03:31.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef122b7700 1 -- 172.21.15.121:0/972298167 <== osd.6 v2:172.21.15.190:6809/33346 31 ==== osd_op_reply(156 1.00000000 [omap-get-vals out=512b] v0'0 uv5 ondisk = 0) v8 ==== 154+0+512 (crc 0 0 0) 0x7fef04023020 con 0x7fef0c0113f0 2023-11-15T03:03:31.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7fef0404acf0 msgr2=0x7fef04049910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7fef0404acf0 0x7fef04049910 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7fef0404b690 msgr2=0x7fef0404a800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7fef0404b690 0x7fef0404a800 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7fef0c01b910 msgr2=0x7fef0c01a390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7fef0c01b910 0x7fef0c01a390 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7fef0c03b040 msgr2=0x7fef0c02f970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7fef0c03b040 0x7fef0c02f970 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7fef0800f8a0 msgr2=0x7fef0800a340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 reap_dead start 2023-11-15T03:03:31.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7fef0800f8a0 0x7fef0800a340 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7fef0c02c3e0 msgr2=0x7fef0c02c800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7fef0c02c3e0 0x7fef0c02c800 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7fef0c0113f0 msgr2=0x7fef0c0117d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7fef0c0113f0 0x7fef0c0117d0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x563198942700 msgr2=0x5631989ee030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x563198942700 0x5631989ee030 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feef4059020 msgr2=0x7feef405b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feef4059020 0x7feef405b4e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x56319895a140 tx=0x7fef08000bc0).stop 2023-11-15T03:03:31.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631989167e0 msgr2=0x5631989cea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef12ab8700 1 -- 172.21.15.121:0/972298167 reap_dead start 2023-11-15T03:03:31.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631989167e0 0x5631989cea50 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7fef04010000 tx=0x7fef04016000).stop 2023-11-15T03:03:31.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 shutdown_connections 2023-11-15T03:03:31.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 --2- 172.21.15.121:0/972298167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631989167e0 0x5631989cea50 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 >> 172.21.15.121:0/972298167 conn(0x56319895c8f0 msgr2=0x563198956370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:31.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 shutdown_connections 2023-11-15T03:03:31.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.549+0000 7fef1dd3ad00 1 -- 172.21.15.121:0/972298167 wait complete. 2023-11-15T03:03:31.606 INFO:teuthology.orchestra.run.smithi121.stdout:1.00000000�������������������!�������������������������� ��d^_layout@@_parent ���� ��  (5Te���(5Te���8`5Tem��85Te������� ��� subdir_head�i���5Te;L�A������������������5Te;L5Te���(5Te;L8`5Tem��8`5Tem�� ��������5Te������������������� �� �� 2023-11-15T03:03:31.606 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata ls 2023-11-15T03:03:31.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.697+0000 7f117f93ad00 1 Processor -- start 2023-11-15T03:03:31.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.697+0000 7f117f93ad00 1 -- start start 2023-11-15T03:03:31.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.697+0000 7f117f93ad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562f0ce58db0 0x562f0ce55ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.697+0000 7f117f93ad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562f0cece640 0x562f0ce10a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.697+0000 7f117f93ad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f0cecea20 0x562f0ce10f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.697+0000 7f117f93ad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0cd85220 con 0x562f0cecea20 2023-11-15T03:03:31.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.697+0000 7f117f93ad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0cecdee0 con 0x562f0cece640 2023-11-15T03:03:31.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.697+0000 7f117f93ad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f0ce3df80 con 0x562f0ce58db0 2023-11-15T03:03:31.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f1174786700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f0cecea20 0x562f0ce10f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f1174786700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f0cecea20 0x562f0ce10f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49888/0 (socket says 172.21.15.121:49888) 2023-11-15T03:03:31.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f1174786700 1 -- 172.21.15.121:0/750630549 learned_addr learned my addr 172.21.15.121:0/750630549 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:31.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f1173f85700 1 --2- 172.21.15.121:0/750630549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562f0ce58db0 0x562f0ce55ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f1173784700 1 --2- 172.21.15.121:0/750630549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562f0cece640 0x562f0ce10a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f1174786700 1 -- 172.21.15.121:0/750630549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562f0ce58db0 msgr2=0x562f0ce55ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:31.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f1174786700 1 --2- 172.21.15.121:0/750630549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562f0ce58db0 0x562f0ce55ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f1174786700 1 -- 172.21.15.121:0/750630549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562f0cece640 msgr2=0x562f0ce10a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:31.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f1174786700 1 --2- 172.21.15.121:0/750630549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562f0cece640 0x562f0ce10a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f1174786700 1 -- 172.21.15.121:0/750630549 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1168005cf0 con 0x562f0cecea20 2023-11-15T03:03:31.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f1173f85700 1 --2- 172.21.15.121:0/750630549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562f0ce58db0 0x562f0ce55ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:31.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f1174786700 1 --2- 172.21.15.121:0/750630549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f0cecea20 0x562f0ce10f30 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f116c00b780 tx=0x7f116c005e00).ready entity=mon.0 client_cookie=584c30d603050107 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f1172782700 1 -- 172.21.15.121:0/750630549 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c00d5f0 con 0x562f0cecea20 2023-11-15T03:03:31.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f1172782700 1 -- 172.21.15.121:0/750630549 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116c00d750 con 0x562f0cecea20 2023-11-15T03:03:31.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f1172782700 1 -- 172.21.15.121:0/750630549 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c00daf0 con 0x562f0cecea20 2023-11-15T03:03:31.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f117f93ad00 1 -- 172.21.15.121:0/750630549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f0cecea20 msgr2=0x562f0ce10f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f117f93ad00 1 --2- 172.21.15.121:0/750630549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f0cecea20 0x562f0ce10f30 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f116c00b780 tx=0x7f116c005e00).stop 2023-11-15T03:03:31.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f117f93ad00 1 -- 172.21.15.121:0/750630549 shutdown_connections 2023-11-15T03:03:31.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f117f93ad00 1 --2- 172.21.15.121:0/750630549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f0cecea20 0x562f0ce10f30 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f117f93ad00 1 --2- 172.21.15.121:0/750630549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562f0cece640 0x562f0ce10a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f117f93ad00 1 --2- 172.21.15.121:0/750630549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562f0ce58db0 0x562f0ce55ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f117f93ad00 1 -- 172.21.15.121:0/750630549 >> 172.21.15.121:0/750630549 conn(0x562f0ce568e0 msgr2=0x562f0ce11910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:31.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f117f93ad00 1 -- 172.21.15.121:0/750630549 shutdown_connections 2023-11-15T03:03:31.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.701+0000 7f117f93ad00 1 -- 172.21.15.121:0/750630549 wait complete. 2023-11-15T03:03:31.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.705+0000 7f117f93ad00 1 Processor -- start 2023-11-15T03:03:31.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.705+0000 7f117f93ad00 1 -- start start 2023-11-15T03:03:31.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.705+0000 7f117f93ad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562f0ce58db0 0x562f0ce51ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.705+0000 7f117f93ad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562f0cece640 0x562f0cee46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.705+0000 7f117f93ad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f0cecea20 0x562f0ceebbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.705+0000 7f117f93ad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0cec8930 con 0x562f0cecea20 2023-11-15T03:03:31.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.705+0000 7f117f93ad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f0ce3c880 con 0x562f0cece640 2023-11-15T03:03:31.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.705+0000 7f117f93ad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f0ce51040 con 0x562f0ce58db0 2023-11-15T03:03:31.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.705+0000 7f1174786700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f0cecea20 0x562f0ceebbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.705+0000 7f1174786700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f0cecea20 0x562f0ceebbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49892/0 (socket says 172.21.15.121:49892) 2023-11-15T03:03:31.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.705+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 learned_addr learned my addr 172.21.15.121:0/3205119015 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:31.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.705+0000 7f1173f85700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562f0ce58db0 0x562f0ce51ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.705+0000 7f1173784700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562f0cece640 0x562f0cee46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.705+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562f0ce58db0 msgr2=0x562f0ce51ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.705+0000 7f1174786700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562f0ce58db0 0x562f0ce51ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.705+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562f0cece640 msgr2=0x562f0cee46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.705+0000 7f1174786700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562f0cece640 0x562f0cee46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.705+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c00b430 con 0x562f0cecea20 2023-11-15T03:03:31.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.709+0000 7f1174786700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f0cecea20 0x562f0ceebbe0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x562f0ce105d0 tx=0x7f116c018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.709+0000 7f1170f7f700 1 -- 172.21.15.121:0/3205119015 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c00dad0 con 0x562f0cecea20 2023-11-15T03:03:31.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.709+0000 7f1170f7f700 1 -- 172.21.15.121:0/3205119015 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116c009030 con 0x562f0cecea20 2023-11-15T03:03:31.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.709+0000 7f1170f7f700 1 -- 172.21.15.121:0/3205119015 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c00de60 con 0x562f0cecea20 2023-11-15T03:03:31.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.709+0000 7f117f93ad00 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562f0cdae770 con 0x562f0cecea20 2023-11-15T03:03:31.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.709+0000 7f117f93ad00 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f0cd7da70 con 0x562f0cecea20 2023-11-15T03:03:31.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.709+0000 7f1170f7f700 1 -- 172.21.15.121:0/3205119015 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f116c003040 con 0x562f0cecea20 2023-11-15T03:03:31.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.713+0000 7f1170f7f700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1160058ff0 0x7f116005b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.713+0000 7f1170f7f700 1 -- 172.21.15.121:0/3205119015 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f116c0757c0 con 0x562f0cecea20 2023-11-15T03:03:31.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.713+0000 7f1173f85700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1160058ff0 0x7f116005b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.713+0000 7f117f93ad00 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562f0cee4bb0 0x562f0ce3afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.713+0000 7f117f93ad00 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x562f0cee4bb0 2023-11-15T03:03:31.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.713+0000 7f1173784700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562f0cee4bb0 0x562f0ce3afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.713+0000 7f1173f85700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1160058ff0 0x7f116005b4b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f1164000c00 tx=0x7f1164001910).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.713+0000 7f1173784700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562f0cee4bb0 0x562f0ce3afd0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.713+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1168000bc0 con 0x562f0cee4bb0 2023-11-15T03:03:31.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.713+0000 7f1173784700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7f116801e040 0x7f1168007400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.713+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:2 13.20 13:04000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f116801e040 2023-11-15T03:03:31.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.717+0000 7f1173f85700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7f116801e040 0x7f1168007400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.717+0000 7f1173f85700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7f116801e040 0x7f1168007400 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.717+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(2 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f11640128a0 con 0x7f116801e040 2023-11-15T03:03:31.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.717+0000 7f1173f85700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7f11640118d0 0x7f1164010350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.717+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:08000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f11640118d0 2023-11-15T03:03:31.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.717+0000 7f1173784700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7f11640118d0 0x7f1164010350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.717+0000 7f1173784700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7f11640118d0 0x7f1164010350 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.721+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(3 [pgnls start_epoch 93 out=97b] v91'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7f1168000bc0 con 0x7f11640118d0 2023-11-15T03:03:31.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.721+0000 7f1173784700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7f1168020430 0x7f1168020830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.721+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:4 13.30 13:0c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f1168020430 2023-11-15T03:03:31.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.721+0000 7f1174786700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7f1168020430 0x7f1168020830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.721+0000 7f1174786700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7f1168020430 0x7f1168020830 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.721+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(4 [pgnls start_epoch 93 out=129b] v92'6 uv6 ondisk = 1) v8 ==== 144+0+129 (crc 0 0 0) 0x7f116c017020 con 0x7f1168020430 2023-11-15T03:03:31.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.721+0000 7f1174786700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7f116c04c060 0x7f116c0750b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.721+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:5 13.8 13:10000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f116c04c060 2023-11-15T03:03:31.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.721+0000 7f1173f85700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7f116c04c060 0x7f116c0750b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.725+0000 7f1173f85700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7f116c04c060 0x7f116c0750b0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.725+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(5 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f11640128a0 con 0x7f116c04c060 2023-11-15T03:03:31.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.725+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:6 13.28 13:14000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f116c04c060 2023-11-15T03:03:31.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.725+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(6 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f11640128a0 con 0x7f116c04c060 2023-11-15T03:03:31.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.725+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:7 13.18 13:18000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x562f0cee4bb0 2023-11-15T03:03:31.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.725+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(7 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1168000bc0 con 0x562f0cee4bb0 2023-11-15T03:03:31.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.725+0000 7f1173784700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7f11680128f0 0x7f116801d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.725+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:8 13.38 13:1c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f11680128f0 2023-11-15T03:03:31.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.725+0000 7f1173784700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7f11680128f0 0x7f116801d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.725+0000 7f1173784700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7f11680128f0 0x7f116801d3b0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.729+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(8 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1168000bc0 con 0x7f11680128f0 2023-11-15T03:03:31.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.729+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:9 13.4 13:20000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f116801e040 2023-11-15T03:03:31.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.729+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(9 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f11640128a0 con 0x7f116801e040 2023-11-15T03:03:31.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.729+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:10 13.24 13:24000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f116801e040 2023-11-15T03:03:31.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.729+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(10 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f11640128a0 con 0x7f116801e040 2023-11-15T03:03:31.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.729+0000 7f1173f85700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7f1164026940 0x7f11640233a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.729+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:11 13.14 13:28000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f1164026940 2023-11-15T03:03:31.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.729+0000 7f1174786700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7f1164026940 0x7f11640233a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.729+0000 7f1174786700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7f1164026940 0x7f11640233a0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.733+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(11 [pgnls start_epoch 93 out=101b] v92'5 uv5 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7f116c017020 con 0x7f1164026940 2023-11-15T03:03:31.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.733+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:12 13.34 13:2c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f116801e040 2023-11-15T03:03:31.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.733+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(12 [pgnls start_epoch 93 out=97b] v92'13 uv13 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7f11640128a0 con 0x7f116801e040 2023-11-15T03:03:31.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.733+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:13 13.c 13:30000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f11640118d0 2023-11-15T03:03:31.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.733+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(13 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1168000bc0 con 0x7f11640118d0 2023-11-15T03:03:31.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.733+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:14 13.2c 13:34000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f116c04c060 2023-11-15T03:03:31.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.733+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(14 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f11640128a0 con 0x7f116c04c060 2023-11-15T03:03:31.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.733+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:15 13.1c 13:38000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f116801e040 2023-11-15T03:03:31.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.737+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(15 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f11640128a0 con 0x7f116801e040 2023-11-15T03:03:31.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.737+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:16 13.3c 13:3c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f116c04c060 2023-11-15T03:03:31.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.737+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(16 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f11640128a0 con 0x7f116c04c060 2023-11-15T03:03:31.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.737+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:17 13.2 13:40000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f11680128f0 2023-11-15T03:03:31.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.737+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(17 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1168000bc0 con 0x7f11680128f0 2023-11-15T03:03:31.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.737+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:18 13.22 13:44000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f11640118d0 2023-11-15T03:03:31.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.737+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(18 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1168000bc0 con 0x7f11640118d0 2023-11-15T03:03:31.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.737+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:19 13.12 13:48000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f1164026940 2023-11-15T03:03:31.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.741+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(19 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116c017020 con 0x7f1164026940 2023-11-15T03:03:31.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.741+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:20 13.32 13:4c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f116c04c060 2023-11-15T03:03:31.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.741+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(20 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f11640128a0 con 0x7f116c04c060 2023-11-15T03:03:31.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.741+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:21 13.a 13:50000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f116c04c060 2023-11-15T03:03:31.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.741+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(21 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f11640128a0 con 0x7f116c04c060 2023-11-15T03:03:31.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.741+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:22 13.2a 13:54000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f11640118d0 2023-11-15T03:03:31.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.741+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(22 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1168000bc0 con 0x7f11640118d0 2023-11-15T03:03:31.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.741+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:23 13.1a 13:58000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f11640118d0 2023-11-15T03:03:31.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.741+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(23 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1168000bc0 con 0x7f11640118d0 2023-11-15T03:03:31.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.741+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:24 13.3a 13:5c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f11680128f0 2023-11-15T03:03:31.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.745+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(24 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1168000bc0 con 0x7f11680128f0 2023-11-15T03:03:31.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.745+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.6 13:60000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f1168020430 2023-11-15T03:03:31.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.745+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(25 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116c017020 con 0x7f1168020430 2023-11-15T03:03:31.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.745+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:26 13.26 13:64000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x562f0cee4bb0 2023-11-15T03:03:31.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.745+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(26 [pgnls start_epoch 93 out=77b] v92'2 uv2 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7f1168000bc0 con 0x562f0cee4bb0 2023-11-15T03:03:31.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.745+0000 7f1173784700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7f1168032460 0x7f11680328a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.745+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] -- osd_op(unknown.0.0:27 13.16 13:68000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f1168032460 2023-11-15T03:03:31.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.745+0000 7f1173f85700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7f1168032460 0x7f11680328a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.745+0000 7f1173f85700 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7f1168032460 0x7f11680328a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.749+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.3 v2:172.21.15.121:6820/33135 1 ==== osd_op_reply(27 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f11640128a0 con 0x7f1168032460 2023-11-15T03:03:31.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.749+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:28 13.36 13:6c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f116801e040 2023-11-15T03:03:31.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.749+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(28 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f11640128a0 con 0x7f116801e040 2023-11-15T03:03:31.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.749+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:29 13.e 13:70000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x562f0cee4bb0 2023-11-15T03:03:31.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.749+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(29 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f1168000bc0 con 0x562f0cee4bb0 2023-11-15T03:03:31.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.749+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:30 13.2e 13:74000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f116c04c060 2023-11-15T03:03:31.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.749+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.2 v2:172.21.15.121:6808/33134 7 ==== osd_op_reply(30 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f11640128a0 con 0x7f116c04c060 2023-11-15T03:03:31.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.749+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.1e 13:78000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f1168020430 2023-11-15T03:03:31.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.749+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(31 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116c017020 con 0x7f1168020430 2023-11-15T03:03:31.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.749+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:32 13.3e 13:7c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f11640118d0 2023-11-15T03:03:31.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.753+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(32 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1168000bc0 con 0x7f11640118d0 2023-11-15T03:03:31.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.753+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:33 13.1 13:80000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f116801e040 2023-11-15T03:03:31.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.753+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(33 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f11640128a0 con 0x7f116801e040 2023-11-15T03:03:31.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.753+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:34 13.21 13:84000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f1168020430 2023-11-15T03:03:31.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.753+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(34 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116c017020 con 0x7f1168020430 2023-11-15T03:03:31.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.753+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:35 13.11 13:88000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f11640118d0 2023-11-15T03:03:31.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.753+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(35 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1168000bc0 con 0x7f11640118d0 2023-11-15T03:03:31.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.753+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] -- osd_op(unknown.0.0:36 13.31 13:8c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f1168032460 2023-11-15T03:03:31.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.753+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.3 v2:172.21.15.121:6820/33135 2 ==== osd_op_reply(36 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f11640128a0 con 0x7f1168032460 2023-11-15T03:03:31.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.753+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:37 13.9 13:90000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f11680128f0 2023-11-15T03:03:31.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.757+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(37 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1168000bc0 con 0x7f11680128f0 2023-11-15T03:03:31.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.757+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:38 13.29 13:94000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f116801e040 2023-11-15T03:03:31.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.757+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(38 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f11640128a0 con 0x7f116801e040 2023-11-15T03:03:31.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.757+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] -- osd_op(unknown.0.0:39 13.19 13:98000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f1168032460 2023-11-15T03:03:31.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.757+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.3 v2:172.21.15.121:6820/33135 3 ==== osd_op_reply(39 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f11640128a0 con 0x7f1168032460 2023-11-15T03:03:31.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.757+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:40 13.39 13:9c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f116c04c060 2023-11-15T03:03:31.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.757+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.2 v2:172.21.15.121:6808/33134 8 ==== osd_op_reply(40 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f11640128a0 con 0x7f116c04c060 2023-11-15T03:03:31.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.757+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:41 13.5 13:a0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f1168020430 2023-11-15T03:03:31.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.761+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(41 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116c017020 con 0x7f1168020430 2023-11-15T03:03:31.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.761+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:42 13.25 13:a4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f1168020430 2023-11-15T03:03:31.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.761+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(42 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116c017020 con 0x7f1168020430 2023-11-15T03:03:31.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.761+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:43 13.15 13:a8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f1168020430 2023-11-15T03:03:31.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.761+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(43 [pgnls start_epoch 93 out=97b] v91'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7f116c017020 con 0x7f1168020430 2023-11-15T03:03:31.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.761+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:44 13.35 13:ac000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x562f0ce3c260 con 0x7f11640118d0 2023-11-15T03:03:31.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.761+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(44 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1168000bc0 con 0x7f11640118d0 2023-11-15T03:03:31.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.761+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:45 13.d 13:b0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x7f1168020430 2023-11-15T03:03:31.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.761+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(45 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116c017020 con 0x7f1168020430 2023-11-15T03:03:31.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.761+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:46 13.2d 13:b4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x7f1164026940 2023-11-15T03:03:31.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.765+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(46 [pgnls start_epoch 93 out=74b] v91'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f116c017020 con 0x7f1164026940 2023-11-15T03:03:31.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.765+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:47 13.1d 13:b8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x562f0cee4bb0 2023-11-15T03:03:31.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.765+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(47 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f1168000bc0 con 0x562f0cee4bb0 2023-11-15T03:03:31.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.765+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:48 13.3d 13:bc000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x562f0cee4bb0 2023-11-15T03:03:31.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.765+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(48 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1168000bc0 con 0x562f0cee4bb0 2023-11-15T03:03:31.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.765+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:49 13.3 13:c0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x7f1164026940 2023-11-15T03:03:31.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.765+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(49 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116c017020 con 0x7f1164026940 2023-11-15T03:03:31.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.765+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:50 13.23 13:c4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x7f11640118d0 2023-11-15T03:03:31.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.765+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(50 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1168000bc0 con 0x7f11640118d0 2023-11-15T03:03:31.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.765+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:51 13.13 13:c8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x562f0cee4bb0 2023-11-15T03:03:31.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.769+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(51 [pgnls start_epoch 93 out=74b] v92'2 uv2 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f1168051040 con 0x562f0cee4bb0 2023-11-15T03:03:31.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.769+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:52 13.33 13:cc000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x562f0cee4bb0 2023-11-15T03:03:31.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.769+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(52 [pgnls start_epoch 93 out=73b] v91'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f1168051040 con 0x562f0cee4bb0 2023-11-15T03:03:31.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.769+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:53 13.b 13:d0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x7f11680128f0 2023-11-15T03:03:31.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.769+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.5 v2:172.21.15.190:6800/33349 5 ==== osd_op_reply(53 [pgnls start_epoch 93 out=76b] v92'2 uv2 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7f1168051040 con 0x7f11680128f0 2023-11-15T03:03:31.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.769+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.2b 13:d4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x7f116c04c060 2023-11-15T03:03:31.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.769+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.2 v2:172.21.15.121:6808/33134 9 ==== osd_op_reply(54 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f11640128a0 con 0x7f116c04c060 2023-11-15T03:03:31.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.769+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:55 13.1b 13:d8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x7f1168020430 2023-11-15T03:03:31.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.773+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(55 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116c017020 con 0x7f1168020430 2023-11-15T03:03:31.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.773+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:56 13.3b 13:dc000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x7f1168020430 2023-11-15T03:03:31.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.773+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(56 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116c017020 con 0x7f1168020430 2023-11-15T03:03:31.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.773+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:57 13.7 13:e0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x562f0cee4bb0 2023-11-15T03:03:31.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.773+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(57 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f1168051040 con 0x562f0cee4bb0 2023-11-15T03:03:31.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.773+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:58 13.27 13:e4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x7f11640118d0 2023-11-15T03:03:31.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.773+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(58 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1168051040 con 0x7f11640118d0 2023-11-15T03:03:31.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.773+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:59 13.17 13:e8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x7f11680128f0 2023-11-15T03:03:31.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.777+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.5 v2:172.21.15.190:6800/33349 6 ==== osd_op_reply(59 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1168051040 con 0x7f11680128f0 2023-11-15T03:03:31.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.777+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:60 13.37 13:ec000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x7f11680128f0 2023-11-15T03:03:31.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.777+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 <== osd.5 v2:172.21.15.190:6800/33349 7 ==== osd_op_reply(60 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1168051040 con 0x7f11680128f0 2023-11-15T03:03:31.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.777+0000 7f1173784700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:61 13.f 13:f0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x7f116801e040 2023-11-15T03:03:31.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.777+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(61 [pgnls start_epoch 93 out=73b] v91'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f11640128a0 con 0x7f116801e040 2023-11-15T03:03:31.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.777+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:62 13.2f 13:f4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x7f1168020430 2023-11-15T03:03:31.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.777+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(62 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116c017020 con 0x7f1168020430 2023-11-15T03:03:31.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.777+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:63 13.1f 13:f8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x7f116801e040 2023-11-15T03:03:31.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(63 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f11640128a0 con 0x7f116801e040 2023-11-15T03:03:31.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:64 13.3f 13:fc000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e93) v8 -- 0x7f116804b040 con 0x7f116801e040 2023-11-15T03:03:31.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f1173f85700 1 -- 172.21.15.121:0/3205119015 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(64 [pgnls start_epoch 93 out=101b] v92'5 uv5 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7f11640128a0 con 0x7f116801e040 2023-11-15T03:03:31.788 INFO:teuthology.orchestra.run.smithi121.stdout:601.00000000 2023-11-15T03:03:31.789 INFO:teuthology.orchestra.run.smithi121.stdout:602.00000000 2023-11-15T03:03:31.789 INFO:teuthology.orchestra.run.smithi121.stdout:10000000000.00000000 2023-11-15T03:03:31.789 INFO:teuthology.orchestra.run.smithi121.stdout:600.00000000 2023-11-15T03:03:31.789 INFO:teuthology.orchestra.run.smithi121.stdout:603.00000000 2023-11-15T03:03:31.790 INFO:teuthology.orchestra.run.smithi121.stdout:1.00000000.inode 2023-11-15T03:03:31.790 INFO:teuthology.orchestra.run.smithi121.stdout:200.00000000 2023-11-15T03:03:31.790 INFO:teuthology.orchestra.run.smithi121.stdout:200.00000001 2023-11-15T03:03:31.790 INFO:teuthology.orchestra.run.smithi121.stdout:606.00000000 2023-11-15T03:03:31.790 INFO:teuthology.orchestra.run.smithi121.stdout:607.00000000 2023-11-15T03:03:31.791 INFO:teuthology.orchestra.run.smithi121.stdout:mds0_openfiles.0 2023-11-15T03:03:31.791 INFO:teuthology.orchestra.run.smithi121.stdout:608.00000000 2023-11-15T03:03:31.791 INFO:teuthology.orchestra.run.smithi121.stdout:604.00000000 2023-11-15T03:03:31.791 INFO:teuthology.orchestra.run.smithi121.stdout:500.00000000 2023-11-15T03:03:31.791 INFO:teuthology.orchestra.run.smithi121.stdout:mds_snaptable 2023-11-15T03:03:31.792 INFO:teuthology.orchestra.run.smithi121.stdout:605.00000000 2023-11-15T03:03:31.792 INFO:teuthology.orchestra.run.smithi121.stdout:mds0_inotable 2023-11-15T03:03:31.792 INFO:teuthology.orchestra.run.smithi121.stdout:100.00000000 2023-11-15T03:03:31.792 INFO:teuthology.orchestra.run.smithi121.stdout:mds0_sessionmap 2023-11-15T03:03:31.792 INFO:teuthology.orchestra.run.smithi121.stdout:609.00000000 2023-11-15T03:03:31.792 INFO:teuthology.orchestra.run.smithi121.stdout:400.00000000 2023-11-15T03:03:31.793 INFO:teuthology.orchestra.run.smithi121.stdout:100.00000000.inode 2023-11-15T03:03:31.793 INFO:teuthology.orchestra.run.smithi121.stdout:1.00000000 2023-11-15T03:03:31.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 -- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7f1164026940 msgr2=0x7f11640233a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7f1164026940 0x7f11640233a0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 -- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7f11640118d0 msgr2=0x7f1164010350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7f11640118d0 0x7f1164010350 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 -- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7f116c04c060 msgr2=0x7f116c0750b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7f116c04c060 0x7f116c0750b0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 -- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7f1168032460 msgr2=0x7f11680328a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7f1168032460 0x7f11680328a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 reap_dead start 2023-11-15T03:03:31.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 -- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7f1168020430 msgr2=0x7f1168020830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7f1168020430 0x7f1168020830 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 reap_dead start 2023-11-15T03:03:31.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 -- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7f11680128f0 msgr2=0x7f116801d3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7f11680128f0 0x7f116801d3b0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 -- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7f116801e040 msgr2=0x7f1168007400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7f116801e040 0x7f1168007400 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 -- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562f0cee4bb0 msgr2=0x562f0ce3afd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562f0cee4bb0 0x562f0ce3afd0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 -- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1160058ff0 msgr2=0x7f116005b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1160058ff0 0x7f116005b4b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f1164000c00 tx=0x7f1164001910).stop 2023-11-15T03:03:31.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 -- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f0cecea20 msgr2=0x562f0ceebbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f0cecea20 0x562f0ceebbe0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x562f0ce105d0 tx=0x7f116c018000).stop 2023-11-15T03:03:31.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f1174786700 1 -- 172.21.15.121:0/3205119015 reap_dead start 2023-11-15T03:03:31.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 -- 172.21.15.121:0/3205119015 shutdown_connections 2023-11-15T03:03:31.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 --2- 172.21.15.121:0/3205119015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f0cecea20 0x562f0ceebbe0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 -- 172.21.15.121:0/3205119015 >> 172.21.15.121:0/3205119015 conn(0x562f0ce568e0 msgr2=0x562f0ce3e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:31.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 -- 172.21.15.121:0/3205119015 shutdown_connections 2023-11-15T03:03:31.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.781+0000 7f117f93ad00 1 -- 172.21.15.121:0/3205119015 wait complete. 2023-11-15T03:03:31.805 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 10000000000.00000000 2023-11-15T03:03:31.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.861+0000 7f4487936d00 1 Processor -- start 2023-11-15T03:03:31.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.861+0000 7f4487936d00 1 -- start start 2023-11-15T03:03:31.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f4487936d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad5f5210e0 0x55ad5f5e0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f4487936d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ad5f5e1650 0x55ad5f605170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f4487936d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ad5f5e1a30 0x55ad5f605660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f4487936d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f4cdcc0 con 0x55ad5f5e1650 2023-11-15T03:03:31.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f4487936d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f498220 con 0x55ad5f5210e0 2023-11-15T03:03:31.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f4487936d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f550620 con 0x55ad5f5e1a30 2023-11-15T03:03:31.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f447383b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ad5f5e1650 0x55ad5f605170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f447383b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ad5f5e1650 0x55ad5f605170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49908/0 (socket says 172.21.15.121:49908) 2023-11-15T03:03:31.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f447383b700 1 -- 172.21.15.121:0/1589273658 learned_addr learned my addr 172.21.15.121:0/1589273658 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:31.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f447c03c700 1 --2- 172.21.15.121:0/1589273658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad5f5210e0 0x55ad5f5e0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f447c83d700 1 --2- 172.21.15.121:0/1589273658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ad5f5e1a30 0x55ad5f605660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f447383b700 1 -- 172.21.15.121:0/1589273658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ad5f5e1a30 msgr2=0x55ad5f605660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:31.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f447383b700 1 --2- 172.21.15.121:0/1589273658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ad5f5e1a30 0x55ad5f605660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f447383b700 1 -- 172.21.15.121:0/1589273658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad5f5210e0 msgr2=0x55ad5f5e0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:31.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f447383b700 1 --2- 172.21.15.121:0/1589273658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad5f5210e0 0x55ad5f5e0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f447383b700 1 -- 172.21.15.121:0/1589273658 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447400a0e0 con 0x55ad5f5e1650 2023-11-15T03:03:31.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f447c03c700 1 --2- 172.21.15.121:0/1589273658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad5f5210e0 0x55ad5f5e0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:31.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f447383b700 1 --2- 172.21.15.121:0/1589273658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ad5f5e1650 0x55ad5f605170 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f4464002e00 tx=0x7f44640028d0).ready entity=mon.0 client_cookie=8cb0bf7f1d1a85c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f447b03a700 1 -- 172.21.15.121:0/1589273658 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446400bbe0 con 0x55ad5f5e1650 2023-11-15T03:03:31.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f447b03a700 1 -- 172.21.15.121:0/1589273658 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4464008070 con 0x55ad5f5e1650 2023-11-15T03:03:31.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f447b03a700 1 -- 172.21.15.121:0/1589273658 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446400bd60 con 0x55ad5f5e1650 2023-11-15T03:03:31.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f4487936d00 1 -- 172.21.15.121:0/1589273658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ad5f5e1650 msgr2=0x55ad5f605170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f4487936d00 1 --2- 172.21.15.121:0/1589273658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ad5f5e1650 0x55ad5f605170 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f4464002e00 tx=0x7f44640028d0).stop 2023-11-15T03:03:31.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f4487936d00 1 -- 172.21.15.121:0/1589273658 shutdown_connections 2023-11-15T03:03:31.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f4487936d00 1 --2- 172.21.15.121:0/1589273658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ad5f5e1a30 0x55ad5f605660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f4487936d00 1 --2- 172.21.15.121:0/1589273658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ad5f5e1650 0x55ad5f605170 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f4487936d00 1 --2- 172.21.15.121:0/1589273658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad5f5210e0 0x55ad5f5e0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f4487936d00 1 -- 172.21.15.121:0/1589273658 >> 172.21.15.121:0/1589273658 conn(0x55ad5f5698f0 msgr2=0x55ad5f51fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:31.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f4487936d00 1 -- 172.21.15.121:0/1589273658 shutdown_connections 2023-11-15T03:03:31.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.865+0000 7f4487936d00 1 -- 172.21.15.121:0/1589273658 wait complete. 2023-11-15T03:03:31.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f4487936d00 1 Processor -- start 2023-11-15T03:03:31.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f4487936d00 1 -- start start 2023-11-15T03:03:31.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f4487936d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad5f5210e0 0x55ad5f604510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f4487936d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ad5f5e1650 0x55ad5f5dea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f4487936d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ad5f5e1a30 0x55ad5f600f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f4487936d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f5de5b0 con 0x55ad5f5e1650 2023-11-15T03:03:31.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f4487936d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f5df650 con 0x55ad5f5210e0 2023-11-15T03:03:31.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f4487936d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f563650 con 0x55ad5f5e1a30 2023-11-15T03:03:31.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f447383b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ad5f5e1650 0x55ad5f5dea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f447c03c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad5f5210e0 0x55ad5f604510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f447c03c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad5f5210e0 0x55ad5f604510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:50556/0 (socket says 172.21.15.121:50556) 2023-11-15T03:03:31.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f447c03c700 1 -- 172.21.15.121:0/3878036155 learned_addr learned my addr 172.21.15.121:0/3878036155 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:31.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f447383b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ad5f5e1650 0x55ad5f5dea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49912/0 (socket says 172.21.15.121:49912) 2023-11-15T03:03:31.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f447c83d700 1 --2- 172.21.15.121:0/3878036155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ad5f5e1a30 0x55ad5f600f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f447c03c700 1 -- 172.21.15.121:0/3878036155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ad5f5e1a30 msgr2=0x55ad5f600f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f447c03c700 1 --2- 172.21.15.121:0/3878036155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ad5f5e1a30 0x55ad5f600f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f447c03c700 1 -- 172.21.15.121:0/3878036155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ad5f5e1650 msgr2=0x55ad5f5dea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f447c03c700 1 --2- 172.21.15.121:0/3878036155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ad5f5e1650 0x55ad5f5dea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f447c03c700 1 -- 172.21.15.121:0/3878036155 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446400a040 con 0x55ad5f5210e0 2023-11-15T03:03:31.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f447383b700 1 --2- 172.21.15.121:0/3878036155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ad5f5e1650 0x55ad5f5dea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:03:31.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.869+0000 7f447c03c700 1 --2- 172.21.15.121:0/3878036155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad5f5210e0 0x55ad5f604510 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f446c00b570 tx=0x7f446c001bf0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.873+0000 7f4479837700 1 -- 172.21.15.121:0/3878036155 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c00be20 con 0x55ad5f5210e0 2023-11-15T03:03:31.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.873+0000 7f4479837700 1 -- 172.21.15.121:0/3878036155 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c00f580 con 0x55ad5f5210e0 2023-11-15T03:03:31.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.873+0000 7f4479837700 1 -- 172.21.15.121:0/3878036155 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c00f920 con 0x55ad5f5210e0 2023-11-15T03:03:31.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.873+0000 7f4487936d00 1 -- 172.21.15.121:0/3878036155 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ad5f490a70 con 0x55ad5f5210e0 2023-11-15T03:03:31.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.873+0000 7f4487936d00 1 -- 172.21.15.121:0/3878036155 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad5f49c010 con 0x55ad5f5210e0 2023-11-15T03:03:31.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.877+0000 7f4479837700 1 -- 172.21.15.121:0/3878036155 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f446c00fcd0 con 0x55ad5f5210e0 2023-11-15T03:03:31.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.877+0000 7f4479837700 1 --2- 172.21.15.121:0/3878036155 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f445c058ff0 0x7f445c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.877+0000 7f4479837700 1 -- 172.21.15.121:0/3878036155 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f446c00a040 con 0x55ad5f5210e0 2023-11-15T03:03:31.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.877+0000 7f447383b700 1 --2- 172.21.15.121:0/3878036155 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f445c058ff0 0x7f445c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.877+0000 7f4487936d00 1 --2- 172.21.15.121:0/3878036155 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55ad5f604a00 0x55ad5f60c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.877+0000 7f4487936d00 1 -- 172.21.15.121:0/3878036155 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:1 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55ad5f5def30 con 0x55ad5f604a00 2023-11-15T03:03:31.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.877+0000 7f447383b700 1 --2- 172.21.15.121:0/3878036155 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f445c058ff0 0x7f445c05b4b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x55ad5f521650 tx=0x7f44640028d0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.877+0000 7f447c83d700 1 --2- 172.21.15.121:0/3878036155 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55ad5f604a00 0x55ad5f60c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.877+0000 7f447c83d700 1 --2- 172.21.15.121:0/3878036155 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55ad5f604a00 0x55ad5f60c7d0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.881+0000 7f447c83d700 1 -- 172.21.15.121:0/3878036155 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x7f4474017040 con 0x55ad5f604a00 2023-11-15T03:03:31.897 INFO:teuthology.orchestra.run.smithi121.stdout:header (274 bytes) : 2023-11-15T03:03:31.897 INFO:teuthology.orchestra.run.smithi121.stdout:00000000 04 03 0c 01 00 00 0b 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:31.897 INFO:teuthology.orchestra.run.smithi121.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2023-11-15T03:03:31.898 INFO:teuthology.orchestra.run.smithi121.stdout:00000020 00 00 00 00 01 35 54 65 07 3b 4c 15 01 00 00 00 |.....5Te.;L.....| 2023-11-15T03:03:31.898 INFO:teuthology.orchestra.run.smithi121.stdout:00000030 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 |................| 2023-11-15T03:03:31.898 INFO:teuthology.orchestra.run.smithi121.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2023-11-15T03:03:31.899 INFO:teuthology.orchestra.run.smithi121.stdout:00000050 00 00 01 35 54 65 07 3b 4c 15 01 00 00 00 00 00 |...5Te.;L.......| 2023-11-15T03:03:31.900 INFO:teuthology.orchestra.run.smithi121.stdout:00000060 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 |................| 2023-11-15T03:03:31.900 INFO:teuthology.orchestra.run.smithi121.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2023-11-15T03:03:31.900 INFO:teuthology.orchestra.run.smithi121.stdout:00000080 00 00 60 00 00 00 00 00 01 00 00 00 00 00 00 00 |..`.............| 2023-11-15T03:03:31.900 INFO:teuthology.orchestra.run.smithi121.stdout:00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:31.901 INFO:teuthology.orchestra.run.smithi121.stdout:000000a0 00 00 00 00 00 00 00 00 01 35 54 65 6d 91 ae 17 |.........5Tem...| 2023-11-15T03:03:31.902 INFO:teuthology.orchestra.run.smithi121.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2023-11-15T03:03:31.902 INFO:teuthology.orchestra.run.smithi121.stdout:000000c0 60 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 |`...............| 2023-11-15T03:03:31.902 INFO:teuthology.orchestra.run.smithi121.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:31.907 INFO:teuthology.orchestra.run.smithi121.stdout:000000e0 00 00 00 00 00 00 01 35 54 65 6d 91 ae 17 00 00 |.......5Tem.....| 2023-11-15T03:03:31.907 INFO:teuthology.orchestra.run.smithi121.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:31.907 INFO:teuthology.orchestra.run.smithi121.stdout:* 2023-11-15T03:03:31.907 INFO:teuthology.orchestra.run.smithi121.stdout:00000110 00 00 |..| 2023-11-15T03:03:31.908 INFO:teuthology.orchestra.run.smithi121.stdout:00000112 2023-11-15T03:03:31.908 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:31.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.881+0000 7f4487936d00 1 -- 172.21.15.121:0/3878036155 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55ad5f604a00 msgr2=0x55ad5f60c7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.881+0000 7f4487936d00 1 --2- 172.21.15.121:0/3878036155 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55ad5f604a00 0x55ad5f60c7d0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.881+0000 7f4487936d00 1 -- 172.21.15.121:0/3878036155 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f445c058ff0 msgr2=0x7f445c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.881+0000 7f4487936d00 1 --2- 172.21.15.121:0/3878036155 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f445c058ff0 0x7f445c05b4b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x55ad5f521650 tx=0x7f44640028d0).stop 2023-11-15T03:03:31.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.881+0000 7f4487936d00 1 -- 172.21.15.121:0/3878036155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad5f5210e0 msgr2=0x55ad5f604510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.881+0000 7f4487936d00 1 --2- 172.21.15.121:0/3878036155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ad5f5210e0 0x55ad5f604510 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f446c00b570 tx=0x7f446c001bf0).stop 2023-11-15T03:03:31.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.881+0000 7f447c83d700 1 -- 172.21.15.121:0/3878036155 reap_dead start 2023-11-15T03:03:31.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.881+0000 7f4487936d00 1 -- 172.21.15.121:0/3878036155 shutdown_connections 2023-11-15T03:03:31.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.881+0000 7f4487936d00 1 -- 172.21.15.121:0/3878036155 >> 172.21.15.121:0/3878036155 conn(0x55ad5f5698f0 msgr2=0x55ad5f54dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:31.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.881+0000 7f4487936d00 1 -- 172.21.15.121:0/3878036155 shutdown_connections 2023-11-15T03:03:31.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.881+0000 7f4487936d00 1 -- 172.21.15.121:0/3878036155 wait complete. 2023-11-15T03:03:31.914 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 600.00000000 2023-11-15T03:03:31.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.973+0000 7f28f4731d00 1 Processor -- start 2023-11-15T03:03:31.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.973+0000 7f28f4731d00 1 -- start start 2023-11-15T03:03:31.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.973+0000 7f28f4731d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dc8a91920 0x563dc8ad19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.973+0000 7f28f4731d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563dc8ad8be0 0x563dc8b5cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.973+0000 7f28f4731d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563dc8b67a20 0x563dc8ad6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.973+0000 7f28f4731d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc8a06220 con 0x563dc8a91920 2023-11-15T03:03:31.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.973+0000 7f28f4731d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc8b62110 con 0x563dc8ad8be0 2023-11-15T03:03:31.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.973+0000 7f28f4731d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x563dc8b66630 con 0x563dc8b67a20 2023-11-15T03:03:31.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.973+0000 7f28e8d34700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dc8a91920 0x563dc8ad19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28e8d34700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dc8a91920 0x563dc8ad19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49936/0 (socket says 172.21.15.121:49936) 2023-11-15T03:03:31.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28e8d34700 1 -- 172.21.15.121:0/303219348 learned_addr learned my addr 172.21.15.121:0/303219348 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:31.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28e9535700 1 --2- 172.21.15.121:0/303219348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563dc8b67a20 0x563dc8ad6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28e3fff700 1 --2- 172.21.15.121:0/303219348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563dc8ad8be0 0x563dc8b5cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28e8d34700 1 -- 172.21.15.121:0/303219348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563dc8b67a20 msgr2=0x563dc8ad6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:31.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28e8d34700 1 --2- 172.21.15.121:0/303219348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563dc8b67a20 0x563dc8ad6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28e8d34700 1 -- 172.21.15.121:0/303219348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563dc8ad8be0 msgr2=0x563dc8b5cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:31.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28e8d34700 1 --2- 172.21.15.121:0/303219348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563dc8ad8be0 0x563dc8b5cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28e8d34700 1 -- 172.21.15.121:0/303219348 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d4005cf0 con 0x563dc8a91920 2023-11-15T03:03:31.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28e8d34700 1 --2- 172.21.15.121:0/303219348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dc8a91920 0x563dc8ad19d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f28dc009950 tx=0x7f28dc00c040).ready entity=mon.0 client_cookie=51002d5b814448a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:31.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28e2ffd700 1 -- 172.21.15.121:0/303219348 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc00f040 con 0x563dc8a91920 2023-11-15T03:03:31.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28e2ffd700 1 -- 172.21.15.121:0/303219348 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc008ac0 con 0x563dc8a91920 2023-11-15T03:03:31.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28e2ffd700 1 -- 172.21.15.121:0/303219348 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc008e40 con 0x563dc8a91920 2023-11-15T03:03:31.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28f4731d00 1 -- 172.21.15.121:0/303219348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dc8a91920 msgr2=0x563dc8ad19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28f4731d00 1 --2- 172.21.15.121:0/303219348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dc8a91920 0x563dc8ad19d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f28dc009950 tx=0x7f28dc00c040).stop 2023-11-15T03:03:31.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28f4731d00 1 -- 172.21.15.121:0/303219348 shutdown_connections 2023-11-15T03:03:31.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28f4731d00 1 --2- 172.21.15.121:0/303219348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563dc8b67a20 0x563dc8ad6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28f4731d00 1 --2- 172.21.15.121:0/303219348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563dc8ad8be0 0x563dc8b5cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28f4731d00 1 --2- 172.21.15.121:0/303219348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dc8a91920 0x563dc8ad19d0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28f4731d00 1 -- 172.21.15.121:0/303219348 >> 172.21.15.121:0/303219348 conn(0x563dc8ad78f0 msgr2=0x563dc8ad12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:31.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28f4731d00 1 -- 172.21.15.121:0/303219348 shutdown_connections 2023-11-15T03:03:31.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28f4731d00 1 -- 172.21.15.121:0/303219348 wait complete. 2023-11-15T03:03:31.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28f4731d00 1 Processor -- start 2023-11-15T03:03:31.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.977+0000 7f28f4731d00 1 -- start start 2023-11-15T03:03:31.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28f4731d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563dc8a91920 0x563dc8a901f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28f4731d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dc8ad8be0 0x563dc8b5a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28f4731d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563dc8b67a20 0x563dc8b595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:31.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28f4731d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc8a8f1a0 con 0x563dc8ad8be0 2023-11-15T03:03:31.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28f4731d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc8a8f430 con 0x563dc8b67a20 2023-11-15T03:03:31.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28f4731d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x563dc8a8f710 con 0x563dc8a91920 2023-11-15T03:03:31.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28e3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dc8ad8be0 0x563dc8b5a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28e3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dc8ad8be0 0x563dc8b5a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49942/0 (socket says 172.21.15.121:49942) 2023-11-15T03:03:31.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28e3fff700 1 -- 172.21.15.121:0/3378419167 learned_addr learned my addr 172.21.15.121:0/3378419167 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:31.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28e8d34700 1 --2- 172.21.15.121:0/3378419167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563dc8a91920 0x563dc8a901f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28e9535700 1 --2- 172.21.15.121:0/3378419167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563dc8b67a20 0x563dc8b595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:31.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28e3fff700 1 -- 172.21.15.121:0/3378419167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563dc8a91920 msgr2=0x563dc8a901f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28e3fff700 1 --2- 172.21.15.121:0/3378419167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563dc8a91920 0x563dc8a901f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:31.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28e3fff700 1 -- 172.21.15.121:0/3378419167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563dc8b67a20 msgr2=0x563dc8b595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:31.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28e3fff700 1 --2- 172.21.15.121:0/3378419167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563dc8b67a20 0x563dc8b595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28e3fff700 1 -- 172.21.15.121:0/3378419167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc009580 con 0x563dc8ad8be0 2023-11-15T03:03:32.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28e8d34700 1 --2- 172.21.15.121:0/3378419167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563dc8a91920 0x563dc8a901f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:32.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28e9535700 1 --2- 172.21.15.121:0/3378419167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563dc8b67a20 0x563dc8b595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:32.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28e3fff700 1 --2- 172.21.15.121:0/3378419167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dc8ad8be0 0x563dc8b5a380 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x563dc8ad53e0 tx=0x7f28d4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28e17fa700 1 -- 172.21.15.121:0/3378419167 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d4007610 con 0x563dc8ad8be0 2023-11-15T03:03:32.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28e17fa700 1 -- 172.21.15.121:0/3378419167 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d401b040 con 0x563dc8ad8be0 2023-11-15T03:03:32.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28f4731d00 1 -- 172.21.15.121:0/3378419167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563dc8a2f770 con 0x563dc8ad8be0 2023-11-15T03:03:32.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28e17fa700 1 -- 172.21.15.121:0/3378419167 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d4005530 con 0x563dc8ad8be0 2023-11-15T03:03:32.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.981+0000 7f28f4731d00 1 -- 172.21.15.121:0/3378419167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563dc89fea70 con 0x563dc8ad8be0 2023-11-15T03:03:32.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.985+0000 7f28e17fa700 1 -- 172.21.15.121:0/3378419167 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f28d4029440 con 0x563dc8ad8be0 2023-11-15T03:03:32.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.985+0000 7f28e17fa700 1 --2- 172.21.15.121:0/3378419167 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f28cc059020 0x7f28cc05b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.985+0000 7f28e17fa700 1 -- 172.21.15.121:0/3378419167 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f28d4074810 con 0x563dc8ad8be0 2023-11-15T03:03:32.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.985+0000 7f28e8d34700 1 --2- 172.21.15.121:0/3378419167 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f28cc059020 0x7f28cc05b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.985+0000 7f28f4731d00 1 --2- 172.21.15.121:0/3378419167 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x563dc8b5a870 0x563dc8b6c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.985+0000 7f28f4731d00 1 -- 172.21.15.121:0/3378419167 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:1 13.30 13:0d82a743:::600.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x563dc8b62510 con 0x563dc8b5a870 2023-11-15T03:03:32.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.985+0000 7f28e9535700 1 --2- 172.21.15.121:0/3378419167 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x563dc8b5a870 0x563dc8b6c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.985+0000 7f28e8d34700 1 --2- 172.21.15.121:0/3378419167 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f28cc059020 0x7f28cc05b4e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f28dc0087a0 tx=0x7f28dc018000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.989+0000 7f28e9535700 1 --2- 172.21.15.121:0/3378419167 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x563dc8b5a870 0x563dc8b6c720 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.989+0000 7f28e9535700 1 -- 172.21.15.121:0/3378419167 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(1 600.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7f28e4004520 con 0x563dc8b5a870 2023-11-15T03:03:32.017 INFO:teuthology.orchestra.run.smithi121.stdout:header (274 bytes) : 2023-11-15T03:03:32.017 INFO:teuthology.orchestra.run.smithi121.stdout:00000000 04 03 0c 01 00 00 01 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:32.018 INFO:teuthology.orchestra.run.smithi121.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2023-11-15T03:03:32.018 INFO:teuthology.orchestra.run.smithi121.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:32.018 INFO:teuthology.orchestra.run.smithi121.stdout:* 2023-11-15T03:03:32.018 INFO:teuthology.orchestra.run.smithi121.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2023-11-15T03:03:32.020 INFO:teuthology.orchestra.run.smithi121.stdout:00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:32.020 INFO:teuthology.orchestra.run.smithi121.stdout:* 2023-11-15T03:03:32.020 INFO:teuthology.orchestra.run.smithi121.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2023-11-15T03:03:32.020 INFO:teuthology.orchestra.run.smithi121.stdout:00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:32.020 INFO:teuthology.orchestra.run.smithi121.stdout:* 2023-11-15T03:03:32.022 INFO:teuthology.orchestra.run.smithi121.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2023-11-15T03:03:32.022 INFO:teuthology.orchestra.run.smithi121.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:32.023 INFO:teuthology.orchestra.run.smithi121.stdout:* 2023-11-15T03:03:32.023 INFO:teuthology.orchestra.run.smithi121.stdout:00000110 00 00 |..| 2023-11-15T03:03:32.023 INFO:teuthology.orchestra.run.smithi121.stdout:00000112 2023-11-15T03:03:32.023 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:32.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.989+0000 7f28f4731d00 1 -- 172.21.15.121:0/3378419167 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x563dc8b5a870 msgr2=0x563dc8b6c720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.989+0000 7f28f4731d00 1 --2- 172.21.15.121:0/3378419167 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x563dc8b5a870 0x563dc8b6c720 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.989+0000 7f28f4731d00 1 -- 172.21.15.121:0/3378419167 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f28cc059020 msgr2=0x7f28cc05b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.989+0000 7f28f4731d00 1 --2- 172.21.15.121:0/3378419167 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f28cc059020 0x7f28cc05b4e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f28dc0087a0 tx=0x7f28dc018000).stop 2023-11-15T03:03:32.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.989+0000 7f28f4731d00 1 -- 172.21.15.121:0/3378419167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dc8ad8be0 msgr2=0x563dc8b5a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.989+0000 7f28f4731d00 1 --2- 172.21.15.121:0/3378419167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dc8ad8be0 0x563dc8b5a380 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x563dc8ad53e0 tx=0x7f28d4015000).stop 2023-11-15T03:03:32.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.989+0000 7f28e9535700 1 -- 172.21.15.121:0/3378419167 reap_dead start 2023-11-15T03:03:32.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.989+0000 7f28f4731d00 1 -- 172.21.15.121:0/3378419167 shutdown_connections 2023-11-15T03:03:32.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.989+0000 7f28f4731d00 1 -- 172.21.15.121:0/3378419167 >> 172.21.15.121:0/3378419167 conn(0x563dc8ad78f0 msgr2=0x563dc8ad1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.989+0000 7f28f4731d00 1 -- 172.21.15.121:0/3378419167 shutdown_connections 2023-11-15T03:03:32.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:31.989+0000 7f28f4731d00 1 -- 172.21.15.121:0/3378419167 wait complete. 2023-11-15T03:03:32.028 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 1.00000000.inode 2023-11-15T03:03:32.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.085+0000 7f2f4a931d00 1 Processor -- start 2023-11-15T03:03:32.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.085+0000 7f2f4a931d00 1 -- start start 2023-11-15T03:03:32.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f4a931d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d02cadd0 0x55c9d0314ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f4a931d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9d0314b00 0x55c9d03a2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f4a931d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9d03a3670 0x55c9d030fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f4a931d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9d0243220 con 0x55c9d02cadd0 2023-11-15T03:03:32.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f4a931d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9d02fd610 con 0x55c9d0314b00 2023-11-15T03:03:32.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f4a931d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c9d03a5f00 con 0x55c9d03a3670 2023-11-15T03:03:32.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f3ef9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d02cadd0 0x55c9d0314ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f3ef9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d02cadd0 0x55c9d0314ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49958/0 (socket says 172.21.15.121:49958) 2023-11-15T03:03:32.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f3ef9c700 1 -- 172.21.15.121:0/2825812000 learned_addr learned my addr 172.21.15.121:0/2825812000 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f3f79d700 1 --2- 172.21.15.121:0/2825812000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9d03a3670 0x55c9d030fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f3679b700 1 --2- 172.21.15.121:0/2825812000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9d0314b00 0x55c9d03a2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f3ef9c700 1 -- 172.21.15.121:0/2825812000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9d03a3670 msgr2=0x55c9d030fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f3ef9c700 1 --2- 172.21.15.121:0/2825812000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9d03a3670 0x55c9d030fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f3ef9c700 1 -- 172.21.15.121:0/2825812000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9d0314b00 msgr2=0x55c9d03a2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f3ef9c700 1 --2- 172.21.15.121:0/2825812000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9d0314b00 0x55c9d03a2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f3ef9c700 1 -- 172.21.15.121:0/2825812000 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f30005cf0 con 0x55c9d02cadd0 2023-11-15T03:03:32.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f3679b700 1 --2- 172.21.15.121:0/2825812000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9d0314b00 0x55c9d03a2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:32.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f3ef9c700 1 --2- 172.21.15.121:0/2825812000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d02cadd0 0x55c9d0314ee0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7f2f30005c50 tx=0x7f2f30015000).ready entity=mon.0 client_cookie=5b66c0f7007390c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f3df9a700 1 -- 172.21.15.121:0/2825812000 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f30005330 con 0x55c9d02cadd0 2023-11-15T03:03:32.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f3df9a700 1 -- 172.21.15.121:0/2825812000 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f30000bc0 con 0x55c9d02cadd0 2023-11-15T03:03:32.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f3df9a700 1 -- 172.21.15.121:0/2825812000 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f30007b10 con 0x55c9d02cadd0 2023-11-15T03:03:32.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f4a931d00 1 -- 172.21.15.121:0/2825812000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d02cadd0 msgr2=0x55c9d0314ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f4a931d00 1 --2- 172.21.15.121:0/2825812000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d02cadd0 0x55c9d0314ee0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7f2f30005c50 tx=0x7f2f30015000).stop 2023-11-15T03:03:32.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f4a931d00 1 -- 172.21.15.121:0/2825812000 shutdown_connections 2023-11-15T03:03:32.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f4a931d00 1 --2- 172.21.15.121:0/2825812000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9d03a3670 0x55c9d030fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f4a931d00 1 --2- 172.21.15.121:0/2825812000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9d0314b00 0x55c9d03a2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f4a931d00 1 --2- 172.21.15.121:0/2825812000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d02cadd0 0x55c9d0314ee0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f4a931d00 1 -- 172.21.15.121:0/2825812000 >> 172.21.15.121:0/2825812000 conn(0x55c9d031a080 msgr2=0x55c9d02fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f4a931d00 1 -- 172.21.15.121:0/2825812000 shutdown_connections 2023-11-15T03:03:32.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.089+0000 7f2f4a931d00 1 -- 172.21.15.121:0/2825812000 wait complete. 2023-11-15T03:03:32.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f4a931d00 1 Processor -- start 2023-11-15T03:03:32.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f4a931d00 1 -- start start 2023-11-15T03:03:32.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f4a931d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d02cadd0 0x55c9d03ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f4a931d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9d0314b00 0x55c9d03a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f4a931d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9d03a3670 0x55c9d02fe4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f4a931d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9d02fe050 con 0x55c9d02cadd0 2023-11-15T03:03:32.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f4a931d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9d03acf70 con 0x55c9d0314b00 2023-11-15T03:03:32.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f4a931d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c9d03ad250 con 0x55c9d03a3670 2023-11-15T03:03:32.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f3ef9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d02cadd0 0x55c9d03ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f3ef9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d02cadd0 0x55c9d03ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49964/0 (socket says 172.21.15.121:49964) 2023-11-15T03:03:32.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f3ef9c700 1 -- 172.21.15.121:0/3602937806 learned_addr learned my addr 172.21.15.121:0/3602937806 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f3f79d700 1 --2- 172.21.15.121:0/3602937806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9d03a3670 0x55c9d02fe4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f3679b700 1 --2- 172.21.15.121:0/3602937806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9d0314b00 0x55c9d03a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f3ef9c700 1 -- 172.21.15.121:0/3602937806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9d03a3670 msgr2=0x55c9d02fe4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f3ef9c700 1 --2- 172.21.15.121:0/3602937806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9d03a3670 0x55c9d02fe4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f3ef9c700 1 -- 172.21.15.121:0/3602937806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9d0314b00 msgr2=0x55c9d03a1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f3ef9c700 1 --2- 172.21.15.121:0/3602937806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9d0314b00 0x55c9d03a1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f3ef9c700 1 -- 172.21.15.121:0/3602937806 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9d026ca20 con 0x55c9d02cadd0 2023-11-15T03:03:32.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f3ef9c700 1 --2- 172.21.15.121:0/3602937806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d02cadd0 0x55c9d03ac8e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f2f30020040 tx=0x7f2f3001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f37fff700 1 -- 172.21.15.121:0/3602937806 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f30007af0 con 0x55c9d02cadd0 2023-11-15T03:03:32.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f37fff700 1 -- 172.21.15.121:0/3602937806 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f30026430 con 0x55c9d02cadd0 2023-11-15T03:03:32.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f37fff700 1 -- 172.21.15.121:0/3602937806 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f300267f0 con 0x55c9d02cadd0 2023-11-15T03:03:32.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.093+0000 7f2f4a931d00 1 -- 172.21.15.121:0/3602937806 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c9d026c770 con 0x55c9d02cadd0 2023-11-15T03:03:32.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.097+0000 7f2f4a931d00 1 -- 172.21.15.121:0/3602937806 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9d023ba70 con 0x55c9d02cadd0 2023-11-15T03:03:32.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.097+0000 7f2f37fff700 1 -- 172.21.15.121:0/3602937806 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2f30024040 con 0x55c9d02cadd0 2023-11-15T03:03:32.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.097+0000 7f2f37fff700 1 --2- 172.21.15.121:0/3602937806 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2f20059020 0x7f2f2005b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.097+0000 7f2f37fff700 1 -- 172.21.15.121:0/3602937806 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f2f300751d0 con 0x55c9d02cadd0 2023-11-15T03:03:32.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.101+0000 7f2f3679b700 1 --2- 172.21.15.121:0/3602937806 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2f20059020 0x7f2f2005b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.101+0000 7f2f4a931d00 1 --2- 172.21.15.121:0/3602937806 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c9d03ac1e0 0x55c9d03aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.101+0000 7f2f4a931d00 1 -- 172.21.15.121:0/3602937806 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55c9d03b7470 con 0x55c9d03ac1e0 2023-11-15T03:03:32.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.101+0000 7f2f3f79d700 1 --2- 172.21.15.121:0/3602937806 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c9d03ac1e0 0x55c9d03aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.101+0000 7f2f3679b700 1 --2- 172.21.15.121:0/3602937806 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2f20059020 0x7f2f2005b4e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f2f28001ed0 tx=0x7f2f280017e0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.101+0000 7f2f3f79d700 1 --2- 172.21.15.121:0/3602937806 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c9d03ac1e0 0x55c9d03aeec0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.101+0000 7f2f3f79d700 1 -- 172.21.15.121:0/3602937806 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(1 1.00000000.inode [omap-get-header] v0'0 uv3 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2f38002b70 con 0x55c9d03ac1e0 2023-11-15T03:03:32.122 INFO:teuthology.orchestra.run.smithi121.stdout:header (0 bytes) : 2023-11-15T03:03:32.122 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:32.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.101+0000 7f2f4a931d00 1 -- 172.21.15.121:0/3602937806 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c9d03ac1e0 msgr2=0x55c9d03aeec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.101+0000 7f2f4a931d00 1 --2- 172.21.15.121:0/3602937806 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c9d03ac1e0 0x55c9d03aeec0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.101+0000 7f2f4a931d00 1 -- 172.21.15.121:0/3602937806 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2f20059020 msgr2=0x7f2f2005b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.101+0000 7f2f4a931d00 1 --2- 172.21.15.121:0/3602937806 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2f20059020 0x7f2f2005b4e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f2f28001ed0 tx=0x7f2f280017e0).stop 2023-11-15T03:03:32.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.101+0000 7f2f4a931d00 1 -- 172.21.15.121:0/3602937806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d02cadd0 msgr2=0x55c9d03ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.101+0000 7f2f4a931d00 1 --2- 172.21.15.121:0/3602937806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d02cadd0 0x55c9d03ac8e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f2f30020040 tx=0x7f2f3001a000).stop 2023-11-15T03:03:32.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.101+0000 7f2f3f79d700 1 -- 172.21.15.121:0/3602937806 reap_dead start 2023-11-15T03:03:32.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.101+0000 7f2f4a931d00 1 -- 172.21.15.121:0/3602937806 shutdown_connections 2023-11-15T03:03:32.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.101+0000 7f2f4a931d00 1 -- 172.21.15.121:0/3602937806 >> 172.21.15.121:0/3602937806 conn(0x55c9d031a080 msgr2=0x55c9d0317ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.101+0000 7f2f4a931d00 1 -- 172.21.15.121:0/3602937806 shutdown_connections 2023-11-15T03:03:32.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.101+0000 7f2f4a931d00 1 -- 172.21.15.121:0/3602937806 wait complete. 2023-11-15T03:03:32.135 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 200.00000000 2023-11-15T03:03:32.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.185+0000 7f22f433cd00 1 Processor -- start 2023-11-15T03:03:32.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.185+0000 7f22f433cd00 1 -- start start 2023-11-15T03:03:32.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.185+0000 7f22f433cd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ea2b9b1bb0 0x55ea2ba7be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.185+0000 7f22f433cd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ea2b9fbb00 0x55ea2b9f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22f433cd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea2b9f74e0 0x55ea2b9e5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22f433cd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2b95fcc0 con 0x55ea2b9fbb00 2023-11-15T03:03:32.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22f433cd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2b92a220 con 0x55ea2b9f74e0 2023-11-15T03:03:32.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22f433cd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea2b9f7940 con 0x55ea2b9b1bb0 2023-11-15T03:03:32.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22e00b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ea2b9fbb00 0x55ea2b9f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22e00b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ea2b9fbb00 0x55ea2b9f6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49988/0 (socket says 172.21.15.121:49988) 2023-11-15T03:03:32.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22e00b6700 1 -- 172.21.15.121:0/3152759128 learned_addr learned my addr 172.21.15.121:0/3152759128 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22e88b7700 1 --2- 172.21.15.121:0/3152759128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ea2b9b1bb0 0x55ea2ba7be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22e90b8700 1 --2- 172.21.15.121:0/3152759128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea2b9f74e0 0x55ea2b9e5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22e00b6700 1 -- 172.21.15.121:0/3152759128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ea2b9b1bb0 msgr2=0x55ea2ba7be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22e00b6700 1 --2- 172.21.15.121:0/3152759128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ea2b9b1bb0 0x55ea2ba7be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22e00b6700 1 -- 172.21.15.121:0/3152759128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea2b9f74e0 msgr2=0x55ea2b9e5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22e00b6700 1 --2- 172.21.15.121:0/3152759128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea2b9f74e0 0x55ea2b9e5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22e00b6700 1 -- 172.21.15.121:0/3152759128 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d0005cf0 con 0x55ea2b9fbb00 2023-11-15T03:03:32.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22e90b8700 1 --2- 172.21.15.121:0/3152759128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea2b9f74e0 0x55ea2b9e5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:32.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22e88b7700 1 --2- 172.21.15.121:0/3152759128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ea2b9b1bb0 0x55ea2ba7be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:32.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22e00b6700 1 --2- 172.21.15.121:0/3152759128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ea2b9fbb00 0x55ea2b9f6ff0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f22d000a520 tx=0x7f22d0015000).ready entity=mon.0 client_cookie=d5e5a900d9e1c92f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22e37fe700 1 -- 172.21.15.121:0/3152759128 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0000bc0 con 0x55ea2b9fbb00 2023-11-15T03:03:32.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22e37fe700 1 -- 172.21.15.121:0/3152759128 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d00078e0 con 0x55ea2b9fbb00 2023-11-15T03:03:32.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22e37fe700 1 -- 172.21.15.121:0/3152759128 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0013690 con 0x55ea2b9fbb00 2023-11-15T03:03:32.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22f433cd00 1 -- 172.21.15.121:0/3152759128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ea2b9fbb00 msgr2=0x55ea2b9f6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22f433cd00 1 --2- 172.21.15.121:0/3152759128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ea2b9fbb00 0x55ea2b9f6ff0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f22d000a520 tx=0x7f22d0015000).stop 2023-11-15T03:03:32.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22f433cd00 1 -- 172.21.15.121:0/3152759128 shutdown_connections 2023-11-15T03:03:32.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22f433cd00 1 --2- 172.21.15.121:0/3152759128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea2b9f74e0 0x55ea2b9e5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22f433cd00 1 --2- 172.21.15.121:0/3152759128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ea2b9fbb00 0x55ea2b9f6ff0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22f433cd00 1 --2- 172.21.15.121:0/3152759128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ea2b9b1bb0 0x55ea2ba7be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22f433cd00 1 -- 172.21.15.121:0/3152759128 >> 172.21.15.121:0/3152759128 conn(0x55ea2ba01080 msgr2=0x55ea2ba82ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22f433cd00 1 -- 172.21.15.121:0/3152759128 shutdown_connections 2023-11-15T03:03:32.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.189+0000 7f22f433cd00 1 -- 172.21.15.121:0/3152759128 wait complete. 2023-11-15T03:03:32.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22f433cd00 1 Processor -- start 2023-11-15T03:03:32.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22f433cd00 1 -- start start 2023-11-15T03:03:32.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22f433cd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ea2b9b1bb0 0x55ea2b9fac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22f433cd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ea2b9f74e0 0x55ea2b9e15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22f433cd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea2b9fbb00 0x55ea2ba78ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22f433cd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2b9f9b90 con 0x55ea2b9b1bb0 2023-11-15T03:03:32.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22f433cd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2b9f9e70 con 0x55ea2b9fbb00 2023-11-15T03:03:32.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22f433cd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea2b9fa150 con 0x55ea2b9f74e0 2023-11-15T03:03:32.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22e88b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ea2b9b1bb0 0x55ea2b9fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22e88b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ea2b9b1bb0 0x55ea2b9fac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50002/0 (socket says 172.21.15.121:50002) 2023-11-15T03:03:32.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22e88b7700 1 -- 172.21.15.121:0/342993472 learned_addr learned my addr 172.21.15.121:0/342993472 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22e00b6700 1 --2- 172.21.15.121:0/342993472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ea2b9f74e0 0x55ea2b9e15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22e90b8700 1 --2- 172.21.15.121:0/342993472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea2b9fbb00 0x55ea2ba78ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22e88b7700 1 -- 172.21.15.121:0/342993472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ea2b9f74e0 msgr2=0x55ea2b9e15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22e88b7700 1 --2- 172.21.15.121:0/342993472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ea2b9f74e0 0x55ea2b9e15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22e88b7700 1 -- 172.21.15.121:0/342993472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea2b9fbb00 msgr2=0x55ea2ba78ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22e88b7700 1 --2- 172.21.15.121:0/342993472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea2b9fbb00 0x55ea2ba78ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22e88b7700 1 -- 172.21.15.121:0/342993472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea2b953770 con 0x55ea2b9b1bb0 2023-11-15T03:03:32.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22e88b7700 1 --2- 172.21.15.121:0/342993472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ea2b9b1bb0 0x55ea2b9fac30 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f22d800c4b0 tx=0x7f22d8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22e1ffb700 1 -- 172.21.15.121:0/342993472 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8006170 con 0x55ea2b9b1bb0 2023-11-15T03:03:32.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22e1ffb700 1 -- 172.21.15.121:0/342993472 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d80062d0 con 0x55ea2b9b1bb0 2023-11-15T03:03:32.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22e1ffb700 1 -- 172.21.15.121:0/342993472 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8003870 con 0x55ea2b9b1bb0 2023-11-15T03:03:32.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22f433cd00 1 -- 172.21.15.121:0/342993472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ea2b922a70 con 0x55ea2b9b1bb0 2023-11-15T03:03:32.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.193+0000 7f22f433cd00 1 -- 172.21.15.121:0/342993472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea2b92e010 con 0x55ea2b9b1bb0 2023-11-15T03:03:32.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.197+0000 7f22e1ffb700 1 -- 172.21.15.121:0/342993472 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f22d8003c60 con 0x55ea2b9b1bb0 2023-11-15T03:03:32.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.197+0000 7f22e1ffb700 1 --2- 172.21.15.121:0/342993472 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f22c8059020 0x7f22c805b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.197+0000 7f22e1ffb700 1 -- 172.21.15.121:0/342993472 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f22d8072fb0 con 0x55ea2b9b1bb0 2023-11-15T03:03:32.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.197+0000 7f22e00b6700 1 --2- 172.21.15.121:0/342993472 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f22c8059020 0x7f22c805b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.201+0000 7f22f433cd00 1 --2- 172.21.15.121:0/342993472 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55ea2ba79ee0 0x55ea2ba9b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.201+0000 7f22f433cd00 1 -- 172.21.15.121:0/342993472 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55ea2ba9b6a0 con 0x55ea2ba79ee0 2023-11-15T03:03:32.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.201+0000 7f22e90b8700 1 --2- 172.21.15.121:0/342993472 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55ea2ba79ee0 0x55ea2ba9b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.201+0000 7f22e00b6700 1 --2- 172.21.15.121:0/342993472 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f22c8059020 0x7f22c805b4e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f22d001f000 tx=0x7f22d001a000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.201+0000 7f22e90b8700 1 --2- 172.21.15.121:0/342993472 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55ea2ba79ee0 0x55ea2ba9b1b0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.201+0000 7f22e90b8700 1 -- 172.21.15.121:0/342993472 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(1 200.00000000 [omap-get-header] v0'0 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f22e400e6b0 con 0x55ea2ba79ee0 2023-11-15T03:03:32.215 INFO:teuthology.orchestra.run.smithi121.stdout:header (0 bytes) : 2023-11-15T03:03:32.215 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:32.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.201+0000 7f22f433cd00 1 -- 172.21.15.121:0/342993472 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55ea2ba79ee0 msgr2=0x55ea2ba9b1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.201+0000 7f22f433cd00 1 --2- 172.21.15.121:0/342993472 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55ea2ba79ee0 0x55ea2ba9b1b0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.201+0000 7f22f433cd00 1 -- 172.21.15.121:0/342993472 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f22c8059020 msgr2=0x7f22c805b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.201+0000 7f22f433cd00 1 --2- 172.21.15.121:0/342993472 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f22c8059020 0x7f22c805b4e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f22d001f000 tx=0x7f22d001a000).stop 2023-11-15T03:03:32.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.201+0000 7f22f433cd00 1 -- 172.21.15.121:0/342993472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ea2b9b1bb0 msgr2=0x55ea2b9fac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.201+0000 7f22f433cd00 1 --2- 172.21.15.121:0/342993472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ea2b9b1bb0 0x55ea2b9fac30 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f22d800c4b0 tx=0x7f22d8008040).stop 2023-11-15T03:03:32.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.201+0000 7f22e90b8700 1 -- 172.21.15.121:0/342993472 reap_dead start 2023-11-15T03:03:32.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.201+0000 7f22f433cd00 1 -- 172.21.15.121:0/342993472 shutdown_connections 2023-11-15T03:03:32.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.201+0000 7f22f433cd00 1 -- 172.21.15.121:0/342993472 >> 172.21.15.121:0/342993472 conn(0x55ea2ba01080 msgr2=0x55ea2ba79420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.201+0000 7f22f433cd00 1 -- 172.21.15.121:0/342993472 shutdown_connections 2023-11-15T03:03:32.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.201+0000 7f22f433cd00 1 -- 172.21.15.121:0/342993472 wait complete. 2023-11-15T03:03:32.220 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 200.00000001 2023-11-15T03:03:32.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.281+0000 7ff28e535d00 1 Processor -- start 2023-11-15T03:03:32.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.281+0000 7ff28e535d00 1 -- start start 2023-11-15T03:03:32.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.281+0000 7ff28e535d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653593dbde0 0x5653593de3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.281+0000 7ff28e535d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653594b19f0 0x56535940bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.281+0000 7ff28e535d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x565359409a60 0x56535940c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.281+0000 7ff28e535d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x565359354220 con 0x5653593dbde0 2023-11-15T03:03:32.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.281+0000 7ff28e535d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x565359407e10 con 0x5653594b19f0 2023-11-15T03:03:32.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.281+0000 7ff28e535d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x565359408360 con 0x565359409a60 2023-11-15T03:03:32.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.281+0000 7ff282bc4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653593dbde0 0x5653593de3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.281+0000 7ff282bc4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653593dbde0 0x5653593de3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50024/0 (socket says 172.21.15.121:50024) 2023-11-15T03:03:32.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff282bc4700 1 -- 172.21.15.121:0/1495019159 learned_addr learned my addr 172.21.15.121:0/1495019159 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff2833c5700 1 --2- 172.21.15.121:0/1495019159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x565359409a60 0x56535940c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff2823c3700 1 --2- 172.21.15.121:0/1495019159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653594b19f0 0x56535940bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff282bc4700 1 -- 172.21.15.121:0/1495019159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x565359409a60 msgr2=0x56535940c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff282bc4700 1 --2- 172.21.15.121:0/1495019159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x565359409a60 0x56535940c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff282bc4700 1 -- 172.21.15.121:0/1495019159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653594b19f0 msgr2=0x56535940bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff282bc4700 1 --2- 172.21.15.121:0/1495019159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653594b19f0 0x56535940bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff282bc4700 1 -- 172.21.15.121:0/1495019159 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff274005cf0 con 0x5653593dbde0 2023-11-15T03:03:32.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff282bc4700 1 --2- 172.21.15.121:0/1495019159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653593dbde0 0x5653593de3c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x5653593c8c90 tx=0x7ff274016000).ready entity=mon.0 client_cookie=f168d046eae45349 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff2813c1700 1 -- 172.21.15.121:0/1495019159 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2740086a0 con 0x5653593dbde0 2023-11-15T03:03:32.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff2813c1700 1 -- 172.21.15.121:0/1495019159 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff274005330 con 0x5653593dbde0 2023-11-15T03:03:32.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff2813c1700 1 -- 172.21.15.121:0/1495019159 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274000bc0 con 0x5653593dbde0 2023-11-15T03:03:32.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff28e535d00 1 -- 172.21.15.121:0/1495019159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653593dbde0 msgr2=0x5653593de3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff28e535d00 1 --2- 172.21.15.121:0/1495019159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653593dbde0 0x5653593de3c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x5653593c8c90 tx=0x7ff274016000).stop 2023-11-15T03:03:32.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff28e535d00 1 -- 172.21.15.121:0/1495019159 shutdown_connections 2023-11-15T03:03:32.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff28e535d00 1 --2- 172.21.15.121:0/1495019159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x565359409a60 0x56535940c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff28e535d00 1 --2- 172.21.15.121:0/1495019159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653594b19f0 0x56535940bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff28e535d00 1 --2- 172.21.15.121:0/1495019159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653593dbde0 0x5653593de3c0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff28e535d00 1 -- 172.21.15.121:0/1495019159 >> 172.21.15.121:0/1495019159 conn(0x5653593dc7d0 msgr2=0x565359408bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff28e535d00 1 -- 172.21.15.121:0/1495019159 shutdown_connections 2023-11-15T03:03:32.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff28e535d00 1 -- 172.21.15.121:0/1495019159 wait complete. 2023-11-15T03:03:32.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.285+0000 7ff28e535d00 1 Processor -- start 2023-11-15T03:03:32.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff28e535d00 1 -- start start 2023-11-15T03:03:32.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff28e535d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653593dbde0 0x5653594ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff28e535d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653594b19f0 0x565359423150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff28e535d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5653594af610 0x565359423640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff28e535d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x565359423b30 con 0x5653593dbde0 2023-11-15T03:03:32.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff28e535d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653593ff520 con 0x5653594b19f0 2023-11-15T03:03:32.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff28e535d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653594b09a0 con 0x5653594af610 2023-11-15T03:03:32.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff282bc4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653593dbde0 0x5653594ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff282bc4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653593dbde0 0x5653594ac850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50032/0 (socket says 172.21.15.121:50032) 2023-11-15T03:03:32.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff282bc4700 1 -- 172.21.15.121:0/1697179263 learned_addr learned my addr 172.21.15.121:0/1697179263 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff2833c5700 1 --2- 172.21.15.121:0/1697179263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5653594af610 0x565359423640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff2823c3700 1 --2- 172.21.15.121:0/1697179263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653594b19f0 0x565359423150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff282bc4700 1 -- 172.21.15.121:0/1697179263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5653594af610 msgr2=0x565359423640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff282bc4700 1 --2- 172.21.15.121:0/1697179263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5653594af610 0x565359423640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff282bc4700 1 -- 172.21.15.121:0/1697179263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653594b19f0 msgr2=0x565359423150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff282bc4700 1 --2- 172.21.15.121:0/1697179263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653594b19f0 0x565359423150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff282bc4700 1 -- 172.21.15.121:0/1697179263 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56535937da20 con 0x5653593dbde0 2023-11-15T03:03:32.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff282bc4700 1 --2- 172.21.15.121:0/1697179263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653593dbde0 0x5653594ac850 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7ff274021040 tx=0x7ff27401c000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff2737fe700 1 -- 172.21.15.121:0/1697179263 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27401e7d0 con 0x5653593dbde0 2023-11-15T03:03:32.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff28e535d00 1 -- 172.21.15.121:0/1697179263 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56535937d770 con 0x5653593dbde0 2023-11-15T03:03:32.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff2737fe700 1 -- 172.21.15.121:0/1697179263 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27401e930 con 0x5653593dbde0 2023-11-15T03:03:32.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff2737fe700 1 -- 172.21.15.121:0/1697179263 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27401ecf0 con 0x5653593dbde0 2023-11-15T03:03:32.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.289+0000 7ff28e535d00 1 -- 172.21.15.121:0/1697179263 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56535934ca70 con 0x5653593dbde0 2023-11-15T03:03:32.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.293+0000 7ff2737fe700 1 -- 172.21.15.121:0/1697179263 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff274025040 con 0x5653593dbde0 2023-11-15T03:03:32.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.293+0000 7ff2737fe700 1 --2- 172.21.15.121:0/1697179263 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff268058ff0 0x7ff26805b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.293+0000 7ff2737fe700 1 -- 172.21.15.121:0/1697179263 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ff274075e50 con 0x5653593dbde0 2023-11-15T03:03:32.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.297+0000 7ff2823c3700 1 --2- 172.21.15.121:0/1697179263 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff268058ff0 0x7ff26805b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.297+0000 7ff28e535d00 1 --2- 172.21.15.121:0/1697179263 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5653594221a0 0x5653593ff990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.297+0000 7ff28e535d00 1 -- 172.21.15.121:0/1697179263 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.34 13:2e2fa760:::200.00000001:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x565359421380 con 0x5653594221a0 2023-11-15T03:03:32.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.297+0000 7ff2823c3700 1 --2- 172.21.15.121:0/1697179263 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff268058ff0 0x7ff26805b4b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x565359409990 tx=0x7ff27800c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.297+0000 7ff2833c5700 1 --2- 172.21.15.121:0/1697179263 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5653594221a0 0x5653593ff990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.297+0000 7ff2833c5700 1 --2- 172.21.15.121:0/1697179263 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5653594221a0 0x5653593ff990 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.297+0000 7ff2833c5700 1 -- 172.21.15.121:0/1697179263 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 200.00000001 [omap-get-header] v0'0 uv13 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff27c003680 con 0x5653594221a0 2023-11-15T03:03:32.313 INFO:teuthology.orchestra.run.smithi121.stdout:header (0 bytes) : 2023-11-15T03:03:32.313 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:32.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.297+0000 7ff28e535d00 1 -- 172.21.15.121:0/1697179263 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5653594221a0 msgr2=0x5653593ff990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.297+0000 7ff28e535d00 1 --2- 172.21.15.121:0/1697179263 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5653594221a0 0x5653593ff990 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.297+0000 7ff28e535d00 1 -- 172.21.15.121:0/1697179263 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff268058ff0 msgr2=0x7ff26805b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.297+0000 7ff28e535d00 1 --2- 172.21.15.121:0/1697179263 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff268058ff0 0x7ff26805b4b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x565359409990 tx=0x7ff27800c040).stop 2023-11-15T03:03:32.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.297+0000 7ff28e535d00 1 -- 172.21.15.121:0/1697179263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653593dbde0 msgr2=0x5653594ac850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.297+0000 7ff28e535d00 1 --2- 172.21.15.121:0/1697179263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653593dbde0 0x5653594ac850 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7ff274021040 tx=0x7ff27401c000).stop 2023-11-15T03:03:32.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.297+0000 7ff2833c5700 1 -- 172.21.15.121:0/1697179263 reap_dead start 2023-11-15T03:03:32.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.297+0000 7ff28e535d00 1 -- 172.21.15.121:0/1697179263 shutdown_connections 2023-11-15T03:03:32.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.297+0000 7ff28e535d00 1 -- 172.21.15.121:0/1697179263 >> 172.21.15.121:0/1697179263 conn(0x5653593dc7d0 msgr2=0x565359408bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.297+0000 7ff28e535d00 1 -- 172.21.15.121:0/1697179263 shutdown_connections 2023-11-15T03:03:32.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.297+0000 7ff28e535d00 1 -- 172.21.15.121:0/1697179263 wait complete. 2023-11-15T03:03:32.318 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_openfiles.0 2023-11-15T03:03:32.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.373+0000 7f2e44b2fd00 1 Processor -- start 2023-11-15T03:03:32.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.373+0000 7f2e44b2fd00 1 -- start start 2023-11-15T03:03:32.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e44b2fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560783f6edd0 0x560784040e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e44b2fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560784047670 0x560783fa1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e44b2fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560784041830 0x560783fa2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e44b2fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560783ee7220 con 0x560784047670 2023-11-15T03:03:32.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e44b2fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560783fa2850 con 0x560783f6edd0 2023-11-15T03:03:32.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e44b2fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560784047b00 con 0x560784041830 2023-11-15T03:03:32.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e308b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560784047670 0x560783fa1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e308b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560784047670 0x560783fa1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50050/0 (socket says 172.21.15.121:50050) 2023-11-15T03:03:32.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e308b6700 1 -- 172.21.15.121:0/3091510124 learned_addr learned my addr 172.21.15.121:0/3091510124 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e390b7700 1 --2- 172.21.15.121:0/3091510124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560783f6edd0 0x560784040e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e398b8700 1 --2- 172.21.15.121:0/3091510124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560784041830 0x560783fa2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e308b6700 1 -- 172.21.15.121:0/3091510124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560784041830 msgr2=0x560783fa2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e308b6700 1 --2- 172.21.15.121:0/3091510124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560784041830 0x560783fa2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e308b6700 1 -- 172.21.15.121:0/3091510124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560783f6edd0 msgr2=0x560784040e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e308b6700 1 --2- 172.21.15.121:0/3091510124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560783f6edd0 0x560784040e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e308b6700 1 -- 172.21.15.121:0/3091510124 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c005cf0 con 0x560784047670 2023-11-15T03:03:32.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e308b6700 1 --2- 172.21.15.121:0/3091510124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560784047670 0x560783fa1e70 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7f2e24001d30 tx=0x7f2e24001810).ready entity=mon.0 client_cookie=e0193c0f310ac157 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e33fff700 1 -- 172.21.15.121:0/3091510124 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2400bcc0 con 0x560784047670 2023-11-15T03:03:32.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e33fff700 1 -- 172.21.15.121:0/3091510124 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2400be20 con 0x560784047670 2023-11-15T03:03:32.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e33fff700 1 -- 172.21.15.121:0/3091510124 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2400e6a0 con 0x560784047670 2023-11-15T03:03:32.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e44b2fd00 1 -- 172.21.15.121:0/3091510124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560784047670 msgr2=0x560783fa1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e44b2fd00 1 --2- 172.21.15.121:0/3091510124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560784047670 0x560783fa1e70 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7f2e24001d30 tx=0x7f2e24001810).stop 2023-11-15T03:03:32.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e44b2fd00 1 -- 172.21.15.121:0/3091510124 shutdown_connections 2023-11-15T03:03:32.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e44b2fd00 1 --2- 172.21.15.121:0/3091510124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560784041830 0x560783fa2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e44b2fd00 1 --2- 172.21.15.121:0/3091510124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560784047670 0x560783fa1e70 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e44b2fd00 1 --2- 172.21.15.121:0/3091510124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560783f6edd0 0x560784040e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e44b2fd00 1 -- 172.21.15.121:0/3091510124 >> 172.21.15.121:0/3091510124 conn(0x560783fbe080 msgr2=0x560783fbe460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e44b2fd00 1 -- 172.21.15.121:0/3091510124 shutdown_connections 2023-11-15T03:03:32.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.377+0000 7f2e44b2fd00 1 -- 172.21.15.121:0/3091510124 wait complete. 2023-11-15T03:03:32.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.381+0000 7f2e44b2fd00 1 Processor -- start 2023-11-15T03:03:32.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.381+0000 7f2e44b2fd00 1 -- start start 2023-11-15T03:03:32.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.381+0000 7f2e44b2fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560783f6edd0 0x560784058400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.381+0000 7f2e44b2fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560784041830 0x56078405b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.381+0000 7f2e44b2fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560784047670 0x560783fa1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.381+0000 7f2e44b2fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560783fb31f0 con 0x560784047670 2023-11-15T03:03:32.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.381+0000 7f2e44b2fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560783fa0410 con 0x560784041830 2023-11-15T03:03:32.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.381+0000 7f2e44b2fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560783fa06b0 con 0x560783f6edd0 2023-11-15T03:03:32.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.381+0000 7f2e398b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560784047670 0x560783fa1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.381+0000 7f2e398b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560784047670 0x560783fa1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50056/0 (socket says 172.21.15.121:50056) 2023-11-15T03:03:32.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.381+0000 7f2e398b8700 1 -- 172.21.15.121:0/2152999641 learned_addr learned my addr 172.21.15.121:0/2152999641 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.381+0000 7f2e308b6700 1 --2- 172.21.15.121:0/2152999641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560784041830 0x56078405b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.381+0000 7f2e390b7700 1 --2- 172.21.15.121:0/2152999641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560783f6edd0 0x560784058400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.381+0000 7f2e398b8700 1 -- 172.21.15.121:0/2152999641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560783f6edd0 msgr2=0x560784058400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.381+0000 7f2e398b8700 1 --2- 172.21.15.121:0/2152999641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560783f6edd0 0x560784058400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.381+0000 7f2e398b8700 1 -- 172.21.15.121:0/2152999641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560784041830 msgr2=0x56078405b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.381+0000 7f2e398b8700 1 --2- 172.21.15.121:0/2152999641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560784041830 0x56078405b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.381+0000 7f2e398b8700 1 -- 172.21.15.121:0/2152999641 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e28001e10 con 0x560784047670 2023-11-15T03:03:32.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.385+0000 7f2e398b8700 1 --2- 172.21.15.121:0/2152999641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560784047670 0x560783fa1ca0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7f2e3400f900 tx=0x7f2e34007040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.385+0000 7f2e327fc700 1 -- 172.21.15.121:0/2152999641 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e340024c0 con 0x560784047670 2023-11-15T03:03:32.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.385+0000 7f2e327fc700 1 -- 172.21.15.121:0/2152999641 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e34011610 con 0x560784047670 2023-11-15T03:03:32.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.385+0000 7f2e327fc700 1 -- 172.21.15.121:0/2152999641 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34011990 con 0x560784047670 2023-11-15T03:03:32.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.385+0000 7f2e44b2fd00 1 -- 172.21.15.121:0/2152999641 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560783f10770 con 0x560784047670 2023-11-15T03:03:32.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.385+0000 7f2e44b2fd00 1 -- 172.21.15.121:0/2152999641 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560783edfa70 con 0x560784047670 2023-11-15T03:03:32.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.385+0000 7f2e327fc700 1 -- 172.21.15.121:0/2152999641 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2e34003040 con 0x560784047670 2023-11-15T03:03:32.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.389+0000 7f2e327fc700 1 --2- 172.21.15.121:0/2152999641 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2e1c059020 0x7f2e1c05b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.389+0000 7f2e327fc700 1 -- 172.21.15.121:0/2152999641 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f2e34073bb0 con 0x560784047670 2023-11-15T03:03:32.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.389+0000 7f2e390b7700 1 --2- 172.21.15.121:0/2152999641 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2e1c059020 0x7f2e1c05b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.389+0000 7f2e44b2fd00 1 --2- 172.21.15.121:0/2152999641 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x560783f9dc00 0x560783fb4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.389+0000 7f2e44b2fd00 1 -- 172.21.15.121:0/2152999641 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 13.26 13:654134d2:::mds0_openfiles.0:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x56078405b510 con 0x560783f9dc00 2023-11-15T03:03:32.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.389+0000 7f2e308b6700 1 --2- 172.21.15.121:0/2152999641 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x560783f9dc00 0x560783fb4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.389+0000 7f2e390b7700 1 --2- 172.21.15.121:0/2152999641 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2e1c059020 0x7f2e1c05b4e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x560783f5bd20 tx=0x7f2e2c015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.389+0000 7f2e308b6700 1 --2- 172.21.15.121:0/2152999641 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x560783f9dc00 0x560783fb4170 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.389+0000 7f2e308b6700 1 -- 172.21.15.121:0/2152999641 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-get-header out=42b] v0'0 uv2 ondisk = 0) v8 ==== 160+0+42 (crc 0 0 0) 0x7f2e2400e470 con 0x560783f9dc00 2023-11-15T03:03:32.405 INFO:teuthology.orchestra.run.smithi121.stdout:header (42 bytes) : 2023-11-15T03:03:32.405 INFO:teuthology.orchestra.run.smithi121.stdout:00000000 13 00 00 00 63 65 70 68 20 66 73 20 76 6f 6c 75 |....ceph fs volu| 2023-11-15T03:03:32.405 INFO:teuthology.orchestra.run.smithi121.stdout:00000010 6d 65 20 76 30 31 31 01 01 0d 00 00 00 02 00 00 |me v011.........| 2023-11-15T03:03:32.406 INFO:teuthology.orchestra.run.smithi121.stdout:00000020 00 00 00 00 00 01 00 00 00 00 |..........| 2023-11-15T03:03:32.406 INFO:teuthology.orchestra.run.smithi121.stdout:0000002a 2023-11-15T03:03:32.406 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:32.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.389+0000 7f2e44b2fd00 1 -- 172.21.15.121:0/2152999641 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x560783f9dc00 msgr2=0x560783fb4170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.393+0000 7f2e44b2fd00 1 --2- 172.21.15.121:0/2152999641 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x560783f9dc00 0x560783fb4170 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.393+0000 7f2e44b2fd00 1 -- 172.21.15.121:0/2152999641 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2e1c059020 msgr2=0x7f2e1c05b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.393+0000 7f2e44b2fd00 1 --2- 172.21.15.121:0/2152999641 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2e1c059020 0x7f2e1c05b4e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x560783f5bd20 tx=0x7f2e2c015000).stop 2023-11-15T03:03:32.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.393+0000 7f2e44b2fd00 1 -- 172.21.15.121:0/2152999641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560784047670 msgr2=0x560783fa1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.393+0000 7f2e44b2fd00 1 --2- 172.21.15.121:0/2152999641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560784047670 0x560783fa1ca0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7f2e3400f900 tx=0x7f2e34007040).stop 2023-11-15T03:03:32.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.393+0000 7f2e398b8700 1 -- 172.21.15.121:0/2152999641 reap_dead start 2023-11-15T03:03:32.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.393+0000 7f2e44b2fd00 1 -- 172.21.15.121:0/2152999641 shutdown_connections 2023-11-15T03:03:32.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.393+0000 7f2e44b2fd00 1 -- 172.21.15.121:0/2152999641 >> 172.21.15.121:0/2152999641 conn(0x560783fbe080 msgr2=0x560783fbe460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.393+0000 7f2e44b2fd00 1 -- 172.21.15.121:0/2152999641 shutdown_connections 2023-11-15T03:03:32.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.393+0000 7f2e44b2fd00 1 -- 172.21.15.121:0/2152999641 wait complete. 2023-11-15T03:03:32.412 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 500.00000000 2023-11-15T03:03:32.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.477+0000 7f16d3b3cd00 1 Processor -- start 2023-11-15T03:03:32.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.477+0000 7f16d3b3cd00 1 -- start start 2023-11-15T03:03:32.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.477+0000 7f16d3b3cd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ab3a14860 0x557ab3ae5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.477+0000 7f16d3b3cd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab3ae9ec0 0x557ab3a5c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.477+0000 7f16d3b3cd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab3aea2a0 0x557ab3a3f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.477+0000 7f16d3b3cd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab3ae5900 con 0x557ab3a14860 2023-11-15T03:03:32.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.477+0000 7f16d3b3cd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab3a5bf40 con 0x557ab3aea2a0 2023-11-15T03:03:32.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.477+0000 7f16d3b3cd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ab3a5c8c0 con 0x557ab3ae9ec0 2023-11-15T03:03:32.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.477+0000 7f16c8147700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ab3a14860 0x557ab3ae5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.477+0000 7f16c8147700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ab3a14860 0x557ab3ae5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50062/0 (socket says 172.21.15.121:50062) 2023-11-15T03:03:32.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.477+0000 7f16c8147700 1 -- 172.21.15.121:0/2230437729 learned_addr learned my addr 172.21.15.121:0/2230437729 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16bf946700 1 --2- 172.21.15.121:0/2230437729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab3ae9ec0 0x557ab3a5c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16c8948700 1 --2- 172.21.15.121:0/2230437729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab3aea2a0 0x557ab3a3f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16c8147700 1 -- 172.21.15.121:0/2230437729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab3ae9ec0 msgr2=0x557ab3a5c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16c8147700 1 --2- 172.21.15.121:0/2230437729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab3ae9ec0 0x557ab3a5c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16c8147700 1 -- 172.21.15.121:0/2230437729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab3aea2a0 msgr2=0x557ab3a3f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16c8147700 1 --2- 172.21.15.121:0/2230437729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab3aea2a0 0x557ab3a3f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16c8147700 1 -- 172.21.15.121:0/2230437729 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c000a180 con 0x557ab3a14860 2023-11-15T03:03:32.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16c8948700 1 --2- 172.21.15.121:0/2230437729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab3aea2a0 0x557ab3a3f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:32.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16c8147700 1 --2- 172.21.15.121:0/2230437729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ab3a14860 0x557ab3ae5250 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f16b8009800 tx=0x7f16b800c040).ready entity=mon.0 client_cookie=14997a887e1532e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16c7145700 1 -- 172.21.15.121:0/2230437729 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b800f040 con 0x557ab3a14860 2023-11-15T03:03:32.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16c7145700 1 -- 172.21.15.121:0/2230437729 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b8008860 con 0x557ab3a14860 2023-11-15T03:03:32.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16c7145700 1 -- 172.21.15.121:0/2230437729 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8008be0 con 0x557ab3a14860 2023-11-15T03:03:32.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16d3b3cd00 1 -- 172.21.15.121:0/2230437729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ab3a14860 msgr2=0x557ab3ae5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16d3b3cd00 1 --2- 172.21.15.121:0/2230437729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ab3a14860 0x557ab3ae5250 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f16b8009800 tx=0x7f16b800c040).stop 2023-11-15T03:03:32.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16d3b3cd00 1 -- 172.21.15.121:0/2230437729 shutdown_connections 2023-11-15T03:03:32.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16d3b3cd00 1 --2- 172.21.15.121:0/2230437729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab3aea2a0 0x557ab3a3f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16d3b3cd00 1 --2- 172.21.15.121:0/2230437729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab3ae9ec0 0x557ab3a5c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16d3b3cd00 1 --2- 172.21.15.121:0/2230437729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ab3a14860 0x557ab3ae5250 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16d3b3cd00 1 -- 172.21.15.121:0/2230437729 >> 172.21.15.121:0/2230437729 conn(0x557ab3a127d0 msgr2=0x557ab3aed370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16d3b3cd00 1 -- 172.21.15.121:0/2230437729 shutdown_connections 2023-11-15T03:03:32.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16d3b3cd00 1 -- 172.21.15.121:0/2230437729 wait complete. 2023-11-15T03:03:32.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.481+0000 7f16d3b3cd00 1 Processor -- start 2023-11-15T03:03:32.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16d3b3cd00 1 -- start start 2023-11-15T03:03:32.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16d3b3cd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ab3a14860 0x557ab3adcda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16d3b3cd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab3ae9ec0 0x557ab3addd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16d3b3cd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab3aea2a0 0x557ab3ad9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16d3b3cd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab3af3320 con 0x557ab3a14860 2023-11-15T03:03:32.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16d3b3cd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab3ae7190 con 0x557ab3ae9ec0 2023-11-15T03:03:32.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16d3b3cd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ab3aeb2e0 con 0x557ab3aea2a0 2023-11-15T03:03:32.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16c8147700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ab3a14860 0x557ab3adcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16c8147700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ab3a14860 0x557ab3adcda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50074/0 (socket says 172.21.15.121:50074) 2023-11-15T03:03:32.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16c8147700 1 -- 172.21.15.121:0/2890067521 learned_addr learned my addr 172.21.15.121:0/2890067521 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16bf946700 1 --2- 172.21.15.121:0/2890067521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab3ae9ec0 0x557ab3addd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16c8948700 1 --2- 172.21.15.121:0/2890067521 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab3aea2a0 0x557ab3ad9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16c8147700 1 -- 172.21.15.121:0/2890067521 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab3aea2a0 msgr2=0x557ab3ad9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16c8147700 1 --2- 172.21.15.121:0/2890067521 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab3aea2a0 0x557ab3ad9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16c8147700 1 -- 172.21.15.121:0/2890067521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab3ae9ec0 msgr2=0x557ab3addd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16c8147700 1 --2- 172.21.15.121:0/2890067521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab3ae9ec0 0x557ab3addd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16c8147700 1 -- 172.21.15.121:0/2890067521 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b8009430 con 0x557ab3a14860 2023-11-15T03:03:32.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16bf946700 1 --2- 172.21.15.121:0/2890067521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab3ae9ec0 0x557ab3addd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:32.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16c8948700 1 --2- 172.21.15.121:0/2890067521 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab3aea2a0 0x557ab3ad9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:32.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16c8147700 1 --2- 172.21.15.121:0/2890067521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ab3a14860 0x557ab3adcda0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x557ab3ae5a40 tx=0x7f16b8018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16c5942700 1 -- 172.21.15.121:0/2890067521 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b800f070 con 0x557ab3a14860 2023-11-15T03:03:32.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16c5942700 1 -- 172.21.15.121:0/2890067521 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b8008be0 con 0x557ab3a14860 2023-11-15T03:03:32.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16c5942700 1 -- 172.21.15.121:0/2890067521 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b801e6b0 con 0x557ab3a14860 2023-11-15T03:03:32.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16d3b3cd00 1 -- 172.21.15.121:0/2890067521 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557ab39b3a20 con 0x557ab3a14860 2023-11-15T03:03:32.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.485+0000 7f16d3b3cd00 1 -- 172.21.15.121:0/2890067521 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ab39b3770 con 0x557ab3a14860 2023-11-15T03:03:32.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.489+0000 7f16c5942700 1 -- 172.21.15.121:0/2890067521 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f16b80133e0 con 0x557ab3a14860 2023-11-15T03:03:32.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.489+0000 7f16c5942700 1 --2- 172.21.15.121:0/2890067521 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f16a8058ff0 0x7f16a805b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.489+0000 7f16c5942700 1 -- 172.21.15.121:0/2890067521 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f16b8074de0 con 0x557ab3a14860 2023-11-15T03:03:32.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.493+0000 7f16bf946700 1 --2- 172.21.15.121:0/2890067521 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f16a8058ff0 0x7f16a805b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.493+0000 7f16d3b3cd00 1 --2- 172.21.15.121:0/2890067521 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x557ab3ae3e80 0x557ab3ae4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.493+0000 7f16d3b3cd00 1 -- 172.21.15.121:0/2890067521 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x557ab3ae4750 con 0x557ab3ae3e80 2023-11-15T03:03:32.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.493+0000 7f16bf946700 1 --2- 172.21.15.121:0/2890067521 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f16a8058ff0 0x7f16a805b4b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x557ab3ae94c0 tx=0x7f16b000c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.493+0000 7f16c8948700 1 --2- 172.21.15.121:0/2890067521 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x557ab3ae3e80 0x557ab3ae4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.493+0000 7f16c8948700 1 --2- 172.21.15.121:0/2890067521 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x557ab3ae3e80 0x557ab3ae4260 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.493+0000 7f16c8948700 1 -- 172.21.15.121:0/2890067521 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(1 500.00000000 [omap-get-header] v0'0 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c00116e0 con 0x557ab3ae3e80 2023-11-15T03:03:32.502 INFO:teuthology.orchestra.run.smithi121.stdout:header (0 bytes) : 2023-11-15T03:03:32.503 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:32.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.493+0000 7f16d3b3cd00 1 -- 172.21.15.121:0/2890067521 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x557ab3ae3e80 msgr2=0x557ab3ae4260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.493+0000 7f16d3b3cd00 1 --2- 172.21.15.121:0/2890067521 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x557ab3ae3e80 0x557ab3ae4260 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.493+0000 7f16d3b3cd00 1 -- 172.21.15.121:0/2890067521 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f16a8058ff0 msgr2=0x7f16a805b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.493+0000 7f16d3b3cd00 1 --2- 172.21.15.121:0/2890067521 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f16a8058ff0 0x7f16a805b4b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x557ab3ae94c0 tx=0x7f16b000c040).stop 2023-11-15T03:03:32.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.493+0000 7f16d3b3cd00 1 -- 172.21.15.121:0/2890067521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ab3a14860 msgr2=0x557ab3adcda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.493+0000 7f16d3b3cd00 1 --2- 172.21.15.121:0/2890067521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ab3a14860 0x557ab3adcda0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x557ab3ae5a40 tx=0x7f16b8018000).stop 2023-11-15T03:03:32.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.493+0000 7f16c8948700 1 -- 172.21.15.121:0/2890067521 reap_dead start 2023-11-15T03:03:32.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.493+0000 7f16d3b3cd00 1 -- 172.21.15.121:0/2890067521 shutdown_connections 2023-11-15T03:03:32.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.493+0000 7f16d3b3cd00 1 -- 172.21.15.121:0/2890067521 >> 172.21.15.121:0/2890067521 conn(0x557ab3a127d0 msgr2=0x557ab3a41dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.493+0000 7f16d3b3cd00 1 -- 172.21.15.121:0/2890067521 shutdown_connections 2023-11-15T03:03:32.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.493+0000 7f16d3b3cd00 1 -- 172.21.15.121:0/2890067521 wait complete. 2023-11-15T03:03:32.507 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds_snaptable 2023-11-15T03:03:32.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.561+0000 7ff0abfdad00 1 Processor -- start 2023-11-15T03:03:32.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.561+0000 7ff0abfdad00 1 -- start start 2023-11-15T03:03:32.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0abfdad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f79b07dd0 0x560f79be0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0abfdad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f79b51b00 0x560f79bd78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0abfdad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f79be0670 0x560f79bed0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0abfdad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f79ab5cc0 con 0x560f79b51b00 2023-11-15T03:03:32.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0abfdad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f79a80220 con 0x560f79be0670 2023-11-15T03:03:32.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0abfdad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f79bdfa30 con 0x560f79b07dd0 2023-11-15T03:03:32.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0a140b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f79b51b00 0x560f79bd78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0a140b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f79b51b00 0x560f79bd78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50082/0 (socket says 172.21.15.121:50082) 2023-11-15T03:03:32.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0a140b700 1 -- 172.21.15.121:0/2762179197 learned_addr learned my addr 172.21.15.121:0/2762179197 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0a9c0c700 1 --2- 172.21.15.121:0/2762179197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f79b07dd0 0x560f79be0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0aa40d700 1 --2- 172.21.15.121:0/2762179197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f79be0670 0x560f79bed0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0a140b700 1 -- 172.21.15.121:0/2762179197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f79b07dd0 msgr2=0x560f79be0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0a140b700 1 --2- 172.21.15.121:0/2762179197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f79b07dd0 0x560f79be0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0a140b700 1 -- 172.21.15.121:0/2762179197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f79be0670 msgr2=0x560f79bed0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0a140b700 1 --2- 172.21.15.121:0/2762179197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f79be0670 0x560f79bed0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0a140b700 1 -- 172.21.15.121:0/2762179197 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c005cf0 con 0x560f79b51b00 2023-11-15T03:03:32.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0a9c0c700 1 --2- 172.21.15.121:0/2762179197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f79b07dd0 0x560f79be0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:32.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0aa40d700 1 --2- 172.21.15.121:0/2762179197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f79be0670 0x560f79bed0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:32.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0a140b700 1 --2- 172.21.15.121:0/2762179197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f79b51b00 0x560f79bd78a0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7ff0940014b0 tx=0x7ff094006b60).ready entity=mon.0 client_cookie=261e52c6091fe9a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0a8c0a700 1 -- 172.21.15.121:0/2762179197 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff094006110 con 0x560f79b51b00 2023-11-15T03:03:32.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0a8c0a700 1 -- 172.21.15.121:0/2762179197 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff094006270 con 0x560f79b51b00 2023-11-15T03:03:32.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0a8c0a700 1 -- 172.21.15.121:0/2762179197 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff094004690 con 0x560f79b51b00 2023-11-15T03:03:32.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0abfdad00 1 -- 172.21.15.121:0/2762179197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f79b51b00 msgr2=0x560f79bd78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0abfdad00 1 --2- 172.21.15.121:0/2762179197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f79b51b00 0x560f79bd78a0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7ff0940014b0 tx=0x7ff094006b60).stop 2023-11-15T03:03:32.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0abfdad00 1 -- 172.21.15.121:0/2762179197 shutdown_connections 2023-11-15T03:03:32.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0abfdad00 1 --2- 172.21.15.121:0/2762179197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f79be0670 0x560f79bed0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0abfdad00 1 --2- 172.21.15.121:0/2762179197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f79b51b00 0x560f79bd78a0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0abfdad00 1 --2- 172.21.15.121:0/2762179197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f79b07dd0 0x560f79be0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0abfdad00 1 -- 172.21.15.121:0/2762179197 >> 172.21.15.121:0/2762179197 conn(0x560f79b57080 msgr2=0x560f79b57460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0abfdad00 1 -- 172.21.15.121:0/2762179197 shutdown_connections 2023-11-15T03:03:32.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.565+0000 7ff0abfdad00 1 -- 172.21.15.121:0/2762179197 wait complete. 2023-11-15T03:03:32.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.569+0000 7ff0abfdad00 1 Processor -- start 2023-11-15T03:03:32.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.569+0000 7ff0abfdad00 1 -- start start 2023-11-15T03:03:32.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.569+0000 7ff0abfdad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f79b07dd0 0x560f79b4d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.569+0000 7ff0abfdad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f79b51b00 0x560f79be9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.569+0000 7ff0abfdad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f79be0670 0x560f79b3a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.569+0000 7ff0abfdad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f79bd8930 con 0x560f79b51b00 2023-11-15T03:03:32.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.569+0000 7ff0abfdad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f79bddbe0 con 0x560f79be0670 2023-11-15T03:03:32.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.569+0000 7ff0abfdad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f79bddec0 con 0x560f79b07dd0 2023-11-15T03:03:32.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.569+0000 7ff0a140b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f79b51b00 0x560f79be9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.569+0000 7ff0a140b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f79b51b00 0x560f79be9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50086/0 (socket says 172.21.15.121:50086) 2023-11-15T03:03:32.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.569+0000 7ff0a140b700 1 -- 172.21.15.121:0/1188739436 learned_addr learned my addr 172.21.15.121:0/1188739436 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.569+0000 7ff0aa40d700 1 --2- 172.21.15.121:0/1188739436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f79be0670 0x560f79b3a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.569+0000 7ff0a9c0c700 1 --2- 172.21.15.121:0/1188739436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f79b07dd0 0x560f79b4d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.569+0000 7ff0a140b700 1 -- 172.21.15.121:0/1188739436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f79b07dd0 msgr2=0x560f79b4d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.569+0000 7ff0a140b700 1 --2- 172.21.15.121:0/1188739436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f79b07dd0 0x560f79b4d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.573+0000 7ff0a140b700 1 -- 172.21.15.121:0/1188739436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f79be0670 msgr2=0x560f79b3a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.573+0000 7ff0a140b700 1 --2- 172.21.15.121:0/1188739436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f79be0670 0x560f79b3a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.573+0000 7ff0a140b700 1 -- 172.21.15.121:0/1188739436 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f79aa9a20 con 0x560f79b51b00 2023-11-15T03:03:32.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.573+0000 7ff0a140b700 1 --2- 172.21.15.121:0/1188739436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f79b51b00 0x560f79be9e70 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7ff094008000 tx=0x7ff094014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.573+0000 7ff0a2ffd700 1 -- 172.21.15.121:0/1188739436 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff094006070 con 0x560f79b51b00 2023-11-15T03:03:32.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.573+0000 7ff0a2ffd700 1 -- 172.21.15.121:0/1188739436 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff094013620 con 0x560f79b51b00 2023-11-15T03:03:32.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.573+0000 7ff0a2ffd700 1 -- 172.21.15.121:0/1188739436 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0940139e0 con 0x560f79b51b00 2023-11-15T03:03:32.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.573+0000 7ff0abfdad00 1 -- 172.21.15.121:0/1188739436 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560f79a78a70 con 0x560f79b51b00 2023-11-15T03:03:32.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.573+0000 7ff0abfdad00 1 -- 172.21.15.121:0/1188739436 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f79a84010 con 0x560f79b51b00 2023-11-15T03:03:32.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.577+0000 7ff0a2ffd700 1 -- 172.21.15.121:0/1188739436 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff094021470 con 0x560f79b51b00 2023-11-15T03:03:32.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.577+0000 7ff0a2ffd700 1 --2- 172.21.15.121:0/1188739436 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff08c058ff0 0x7ff08c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.577+0000 7ff0a2ffd700 1 -- 172.21.15.121:0/1188739436 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ff09401b040 con 0x560f79b51b00 2023-11-15T03:03:32.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.577+0000 7ff0a9c0c700 1 --2- 172.21.15.121:0/1188739436 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff08c058ff0 0x7ff08c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.577+0000 7ff0abfdad00 1 --2- 172.21.15.121:0/1188739436 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x560f79b4d200 0x560f79bd9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.577+0000 7ff0abfdad00 1 -- 172.21.15.121:0/1188739436 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:1 13.2d 13:b50e409b:::mds_snaptable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x560f79bd9d80 con 0x560f79b4d200 2023-11-15T03:03:32.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.577+0000 7ff0aa40d700 1 --2- 172.21.15.121:0/1188739436 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x560f79b4d200 0x560f79bd9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.577+0000 7ff0a9c0c700 1 --2- 172.21.15.121:0/1188739436 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff08c058ff0 0x7ff08c05b4b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7ff09c00c010 tx=0x7ff09c016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.577+0000 7ff0aa40d700 1 --2- 172.21.15.121:0/1188739436 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x560f79b4d200 0x560f79bd9890 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.581+0000 7ff0aa40d700 1 -- 172.21.15.121:0/1188739436 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(1 mds_snaptable [omap-get-header] v0'0 uv1 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff0a400e480 con 0x560f79b4d200 2023-11-15T03:03:32.596 INFO:teuthology.orchestra.run.smithi121.stdout:header (0 bytes) : 2023-11-15T03:03:32.597 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:32.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.581+0000 7ff0abfdad00 1 -- 172.21.15.121:0/1188739436 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x560f79b4d200 msgr2=0x560f79bd9890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.581+0000 7ff0abfdad00 1 --2- 172.21.15.121:0/1188739436 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x560f79b4d200 0x560f79bd9890 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.581+0000 7ff0abfdad00 1 -- 172.21.15.121:0/1188739436 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff08c058ff0 msgr2=0x7ff08c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.581+0000 7ff0abfdad00 1 --2- 172.21.15.121:0/1188739436 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff08c058ff0 0x7ff08c05b4b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7ff09c00c010 tx=0x7ff09c016000).stop 2023-11-15T03:03:32.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.581+0000 7ff0abfdad00 1 -- 172.21.15.121:0/1188739436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f79b51b00 msgr2=0x560f79be9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.581+0000 7ff0abfdad00 1 --2- 172.21.15.121:0/1188739436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f79b51b00 0x560f79be9e70 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7ff094008000 tx=0x7ff094014000).stop 2023-11-15T03:03:32.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.581+0000 7ff0aa40d700 1 -- 172.21.15.121:0/1188739436 reap_dead start 2023-11-15T03:03:32.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.581+0000 7ff0abfdad00 1 -- 172.21.15.121:0/1188739436 shutdown_connections 2023-11-15T03:03:32.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.581+0000 7ff0abfdad00 1 -- 172.21.15.121:0/1188739436 >> 172.21.15.121:0/1188739436 conn(0x560f79b57080 msgr2=0x560f79b57460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.581+0000 7ff0abfdad00 1 -- 172.21.15.121:0/1188739436 shutdown_connections 2023-11-15T03:03:32.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.581+0000 7ff0abfdad00 1 -- 172.21.15.121:0/1188739436 wait complete. 2023-11-15T03:03:32.601 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_inotable 2023-11-15T03:03:32.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.657+0000 7ff44d5dad00 1 Processor -- start 2023-11-15T03:03:32.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.657+0000 7ff44d5dad00 1 -- start start 2023-11-15T03:03:32.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.661+0000 7ff44d5dad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56084d94cdd0 0x56084d999190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.661+0000 7ff44d5dad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56084d996b00 0x56084da31aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.661+0000 7ff44d5dad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56084da25670 0x56084da31f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.661+0000 7ff44d5dad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56084d8c5220 con 0x56084da25670 2023-11-15T03:03:32.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.661+0000 7ff44d5dad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56084d97f170 con 0x56084d996b00 2023-11-15T03:03:32.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.661+0000 7ff44d5dad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56084d992850 con 0x56084d94cdd0 2023-11-15T03:03:32.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.661+0000 7ff44ba0b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56084da25670 0x56084da31f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.661+0000 7ff44ba0b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56084da25670 0x56084da31f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50098/0 (socket says 172.21.15.121:50098) 2023-11-15T03:03:32.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.661+0000 7ff44b20a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56084d94cdd0 0x56084d999190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.661+0000 7ff44ba0b700 1 -- 172.21.15.121:0/3540137149 learned_addr learned my addr 172.21.15.121:0/3540137149 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.661+0000 7ff442a09700 1 --2- 172.21.15.121:0/3540137149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56084d996b00 0x56084da31aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.661+0000 7ff44ba0b700 1 -- 172.21.15.121:0/3540137149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56084d94cdd0 msgr2=0x56084d999190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.661+0000 7ff44ba0b700 1 --2- 172.21.15.121:0/3540137149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56084d94cdd0 0x56084d999190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.661+0000 7ff44ba0b700 1 -- 172.21.15.121:0/3540137149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56084d996b00 msgr2=0x56084da31aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.661+0000 7ff44ba0b700 1 --2- 172.21.15.121:0/3540137149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56084d996b00 0x56084da31aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.661+0000 7ff44ba0b700 1 -- 172.21.15.121:0/3540137149 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44400a180 con 0x56084da25670 2023-11-15T03:03:32.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.661+0000 7ff442a09700 1 --2- 172.21.15.121:0/3540137149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56084d996b00 0x56084da31aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:32.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.661+0000 7ff44ba0b700 1 --2- 172.21.15.121:0/3540137149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56084da25670 0x56084da31f90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7ff44400acb0 tx=0x7ff444014000).ready entity=mon.0 client_cookie=52e44dbf49c8d37b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.661+0000 7ff44a208700 1 -- 172.21.15.121:0/3540137149 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44400c4a0 con 0x56084da25670 2023-11-15T03:03:32.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.665+0000 7ff44a208700 1 -- 172.21.15.121:0/3540137149 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44400ccf0 con 0x56084da25670 2023-11-15T03:03:32.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.665+0000 7ff44a208700 1 -- 172.21.15.121:0/3540137149 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44400ce60 con 0x56084da25670 2023-11-15T03:03:32.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.665+0000 7ff44d5dad00 1 -- 172.21.15.121:0/3540137149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56084da25670 msgr2=0x56084da31f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.665+0000 7ff44d5dad00 1 --2- 172.21.15.121:0/3540137149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56084da25670 0x56084da31f90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7ff44400acb0 tx=0x7ff444014000).stop 2023-11-15T03:03:32.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.665+0000 7ff44d5dad00 1 -- 172.21.15.121:0/3540137149 shutdown_connections 2023-11-15T03:03:32.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.665+0000 7ff44d5dad00 1 --2- 172.21.15.121:0/3540137149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56084da25670 0x56084da31f90 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.665+0000 7ff44d5dad00 1 --2- 172.21.15.121:0/3540137149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56084d996b00 0x56084da31aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.665+0000 7ff44d5dad00 1 --2- 172.21.15.121:0/3540137149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56084d94cdd0 0x56084d999190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.665+0000 7ff44d5dad00 1 -- 172.21.15.121:0/3540137149 >> 172.21.15.121:0/3540137149 conn(0x56084d99c080 msgr2=0x56084d981810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.665+0000 7ff44d5dad00 1 -- 172.21.15.121:0/3540137149 shutdown_connections 2023-11-15T03:03:32.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.665+0000 7ff44d5dad00 1 -- 172.21.15.121:0/3540137149 wait complete. 2023-11-15T03:03:32.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.665+0000 7ff44d5dad00 1 Processor -- start 2023-11-15T03:03:32.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.665+0000 7ff44d5dad00 1 -- start start 2023-11-15T03:03:32.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff44d5dad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56084d94cdd0 0x56084d981530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff44d5dad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56084d996b00 0x56084da0c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff44d5dad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56084da25670 0x56084da0cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff44d5dad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56084da21260 con 0x56084d996b00 2023-11-15T03:03:32.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff44d5dad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56084da0d010 con 0x56084da25670 2023-11-15T03:03:32.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff44d5dad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56084d97e030 con 0x56084d94cdd0 2023-11-15T03:03:32.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff442a09700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56084d996b00 0x56084da0c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff442a09700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56084d996b00 0x56084da0c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50114/0 (socket says 172.21.15.121:50114) 2023-11-15T03:03:32.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff442a09700 1 -- 172.21.15.121:0/1660204174 learned_addr learned my addr 172.21.15.121:0/1660204174 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff44b20a700 1 --2- 172.21.15.121:0/1660204174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56084d94cdd0 0x56084d981530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff44ba0b700 1 --2- 172.21.15.121:0/1660204174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56084da25670 0x56084da0cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff442a09700 1 -- 172.21.15.121:0/1660204174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56084d94cdd0 msgr2=0x56084d981530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff442a09700 1 --2- 172.21.15.121:0/1660204174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56084d94cdd0 0x56084d981530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff442a09700 1 -- 172.21.15.121:0/1660204174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56084da25670 msgr2=0x56084da0cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff442a09700 1 --2- 172.21.15.121:0/1660204174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56084da25670 0x56084da0cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff442a09700 1 -- 172.21.15.121:0/1660204174 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56084d8eea20 con 0x56084d996b00 2023-11-15T03:03:32.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff442a09700 1 --2- 172.21.15.121:0/1660204174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56084d996b00 0x56084da0c630 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7ff434001d40 tx=0x7ff434001930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff448a05700 1 -- 172.21.15.121:0/1660204174 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43400bbc0 con 0x56084d996b00 2023-11-15T03:03:32.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff448a05700 1 -- 172.21.15.121:0/1660204174 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff434008040 con 0x56084d996b00 2023-11-15T03:03:32.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff44d5dad00 1 -- 172.21.15.121:0/1660204174 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56084d8ee770 con 0x56084d996b00 2023-11-15T03:03:32.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff448a05700 1 -- 172.21.15.121:0/1660204174 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff434011480 con 0x56084d996b00 2023-11-15T03:03:32.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.669+0000 7ff44d5dad00 1 -- 172.21.15.121:0/1660204174 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56084d8bda70 con 0x56084d996b00 2023-11-15T03:03:32.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.673+0000 7ff448a05700 1 -- 172.21.15.121:0/1660204174 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff4340103e0 con 0x56084d996b00 2023-11-15T03:03:32.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.673+0000 7ff448a05700 1 --2- 172.21.15.121:0/1660204174 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff42c058ff0 0x7ff42c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.673+0000 7ff448a05700 1 -- 172.21.15.121:0/1660204174 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ff434072b70 con 0x56084d996b00 2023-11-15T03:03:32.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.673+0000 7ff44b20a700 1 --2- 172.21.15.121:0/1660204174 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff42c058ff0 0x7ff42c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.673+0000 7ff44d5dad00 1 --2- 172.21.15.121:0/1660204174 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56084d981a20 0x56084d97d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.673+0000 7ff44d5dad00 1 -- 172.21.15.121:0/1660204174 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x56084d994d60 con 0x56084d981a20 2023-11-15T03:03:32.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.673+0000 7ff44b20a700 1 --2- 172.21.15.121:0/1660204174 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff42c058ff0 0x7ff42c05b4b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7ff43c005fd0 tx=0x7ff43c00c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.673+0000 7ff44ba0b700 1 --2- 172.21.15.121:0/1660204174 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56084d981a20 0x56084d97d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.677+0000 7ff44ba0b700 1 --2- 172.21.15.121:0/1660204174 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56084d981a20 0x56084d97d5b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.677+0000 7ff44ba0b700 1 -- 172.21.15.121:0/1660204174 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 mds0_inotable [omap-get-header] v0'0 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff44400c770 con 0x56084d981a20 2023-11-15T03:03:32.688 INFO:teuthology.orchestra.run.smithi121.stdout:header (0 bytes) : 2023-11-15T03:03:32.688 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:32.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.677+0000 7ff44d5dad00 1 -- 172.21.15.121:0/1660204174 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56084d981a20 msgr2=0x56084d97d5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.677+0000 7ff44d5dad00 1 --2- 172.21.15.121:0/1660204174 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56084d981a20 0x56084d97d5b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.677+0000 7ff44d5dad00 1 -- 172.21.15.121:0/1660204174 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff42c058ff0 msgr2=0x7ff42c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.677+0000 7ff44d5dad00 1 --2- 172.21.15.121:0/1660204174 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff42c058ff0 0x7ff42c05b4b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7ff43c005fd0 tx=0x7ff43c00c040).stop 2023-11-15T03:03:32.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.677+0000 7ff44d5dad00 1 -- 172.21.15.121:0/1660204174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56084d996b00 msgr2=0x56084da0c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.677+0000 7ff44d5dad00 1 --2- 172.21.15.121:0/1660204174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56084d996b00 0x56084da0c630 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7ff434001d40 tx=0x7ff434001930).stop 2023-11-15T03:03:32.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.677+0000 7ff44ba0b700 1 -- 172.21.15.121:0/1660204174 reap_dead start 2023-11-15T03:03:32.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.677+0000 7ff44d5dad00 1 -- 172.21.15.121:0/1660204174 shutdown_connections 2023-11-15T03:03:32.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.677+0000 7ff44d5dad00 1 -- 172.21.15.121:0/1660204174 >> 172.21.15.121:0/1660204174 conn(0x56084d99c080 msgr2=0x56084d999190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.677+0000 7ff44d5dad00 1 -- 172.21.15.121:0/1660204174 shutdown_connections 2023-11-15T03:03:32.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.677+0000 7ff44d5dad00 1 -- 172.21.15.121:0/1660204174 wait complete. 2023-11-15T03:03:32.693 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 100.00000000 2023-11-15T03:03:32.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.745+0000 7f489173ed00 1 Processor -- start 2023-11-15T03:03:32.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.745+0000 7f489173ed00 1 -- start start 2023-11-15T03:03:32.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f489173ed00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c58930bc0 0x555c58a13da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f489173ed00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555c5895ee40 0x555c58979590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f489173ed00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c58a0c3b0 0x555c58960080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f489173ed00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c58a00430 con 0x555c58a0c3b0 2023-11-15T03:03:32.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f489173ed00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5897a910 con 0x555c58930bc0 2023-11-15T03:03:32.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f489173ed00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c5897abf0 con 0x555c5895ee40 2023-11-15T03:03:32.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f48865a2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c58a0c3b0 0x555c58960080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f48865a2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c58a0c3b0 0x555c58960080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50134/0 (socket says 172.21.15.121:50134) 2023-11-15T03:03:32.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f48865a2700 1 -- 172.21.15.121:0/3593287309 learned_addr learned my addr 172.21.15.121:0/3593287309 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f4885da1700 1 --2- 172.21.15.121:0/3593287309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c58930bc0 0x555c58a13da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f487d5a0700 1 --2- 172.21.15.121:0/3593287309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555c5895ee40 0x555c58979590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f48865a2700 1 -- 172.21.15.121:0/3593287309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555c5895ee40 msgr2=0x555c58979590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f48865a2700 1 --2- 172.21.15.121:0/3593287309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555c5895ee40 0x555c58979590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f48865a2700 1 -- 172.21.15.121:0/3593287309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c58930bc0 msgr2=0x555c58a13da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f48865a2700 1 --2- 172.21.15.121:0/3593287309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c58930bc0 0x555c58a13da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f48865a2700 1 -- 172.21.15.121:0/3593287309 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4870005db0 con 0x555c58a0c3b0 2023-11-15T03:03:32.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f48865a2700 1 --2- 172.21.15.121:0/3593287309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c58a0c3b0 0x555c58960080 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f48800098f0 tx=0x7f488000c040).ready entity=mon.0 client_cookie=80f12563abd90911 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f4884d9f700 1 -- 172.21.15.121:0/3593287309 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488000f040 con 0x555c58a0c3b0 2023-11-15T03:03:32.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f4884d9f700 1 -- 172.21.15.121:0/3593287309 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4880008800 con 0x555c58a0c3b0 2023-11-15T03:03:32.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f4884d9f700 1 -- 172.21.15.121:0/3593287309 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4880008ba0 con 0x555c58a0c3b0 2023-11-15T03:03:32.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f489173ed00 1 -- 172.21.15.121:0/3593287309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c58a0c3b0 msgr2=0x555c58960080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f489173ed00 1 --2- 172.21.15.121:0/3593287309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c58a0c3b0 0x555c58960080 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f48800098f0 tx=0x7f488000c040).stop 2023-11-15T03:03:32.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f489173ed00 1 -- 172.21.15.121:0/3593287309 shutdown_connections 2023-11-15T03:03:32.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f489173ed00 1 --2- 172.21.15.121:0/3593287309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c58a0c3b0 0x555c58960080 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f489173ed00 1 --2- 172.21.15.121:0/3593287309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555c5895ee40 0x555c58979590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f489173ed00 1 --2- 172.21.15.121:0/3593287309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c58930bc0 0x555c58a13da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f489173ed00 1 -- 172.21.15.121:0/3593287309 >> 172.21.15.121:0/3593287309 conn(0x555c58977f80 msgr2=0x555c5895f860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f489173ed00 1 -- 172.21.15.121:0/3593287309 shutdown_connections 2023-11-15T03:03:32.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.749+0000 7f489173ed00 1 -- 172.21.15.121:0/3593287309 wait complete. 2023-11-15T03:03:32.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.753+0000 7f489173ed00 1 Processor -- start 2023-11-15T03:03:32.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.753+0000 7f489173ed00 1 -- start start 2023-11-15T03:03:32.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.753+0000 7f489173ed00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c58930bc0 0x555c589fecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.753+0000 7f489173ed00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c5895ee40 0x555c58a13790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.753+0000 7f489173ed00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555c58a0c3b0 0x555c58a13c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.753+0000 7f489173ed00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c58a13300 con 0x555c58930bc0 2023-11-15T03:03:32.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.753+0000 7f489173ed00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c58a11800 con 0x555c5895ee40 2023-11-15T03:03:32.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.753+0000 7f489173ed00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c58a06b20 con 0x555c58a0c3b0 2023-11-15T03:03:32.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.753+0000 7f4885da1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c58930bc0 0x555c589fecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.753+0000 7f4885da1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c58930bc0 0x555c589fecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50148/0 (socket says 172.21.15.121:50148) 2023-11-15T03:03:32.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.753+0000 7f4885da1700 1 -- 172.21.15.121:0/1694375461 learned_addr learned my addr 172.21.15.121:0/1694375461 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.753+0000 7f48865a2700 1 --2- 172.21.15.121:0/1694375461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555c58a0c3b0 0x555c58a13c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.753+0000 7f487d5a0700 1 --2- 172.21.15.121:0/1694375461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c5895ee40 0x555c58a13790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.753+0000 7f4885da1700 1 -- 172.21.15.121:0/1694375461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555c58a0c3b0 msgr2=0x555c58a13c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.753+0000 7f4885da1700 1 --2- 172.21.15.121:0/1694375461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555c58a0c3b0 0x555c58a13c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.753+0000 7f4885da1700 1 -- 172.21.15.121:0/1694375461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c5895ee40 msgr2=0x555c58a13790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.753+0000 7f4885da1700 1 --2- 172.21.15.121:0/1694375461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c5895ee40 0x555c58a13790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.753+0000 7f4885da1700 1 -- 172.21.15.121:0/1694375461 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48800095a0 con 0x555c58930bc0 2023-11-15T03:03:32.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.753+0000 7f48865a2700 1 --2- 172.21.15.121:0/1694375461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555c58a0c3b0 0x555c58a13c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:32.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.757+0000 7f4885da1700 1 --2- 172.21.15.121:0/1694375461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c58930bc0 0x555c589fecf0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7f48780109c0 tx=0x7f4878005d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.757+0000 7f487effd700 1 -- 172.21.15.121:0/1694375461 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487800b440 con 0x555c58930bc0 2023-11-15T03:03:32.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.757+0000 7f487effd700 1 -- 172.21.15.121:0/1694375461 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487800e940 con 0x555c58930bc0 2023-11-15T03:03:32.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.757+0000 7f489173ed00 1 -- 172.21.15.121:0/1694375461 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555c588d1a20 con 0x555c58930bc0 2023-11-15T03:03:32.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.757+0000 7f487effd700 1 -- 172.21.15.121:0/1694375461 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487800ecc0 con 0x555c58930bc0 2023-11-15T03:03:32.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.757+0000 7f489173ed00 1 -- 172.21.15.121:0/1694375461 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c588d1770 con 0x555c58930bc0 2023-11-15T03:03:32.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.757+0000 7f487effd700 1 -- 172.21.15.121:0/1694375461 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4878013650 con 0x555c58930bc0 2023-11-15T03:03:32.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.761+0000 7f487effd700 1 --2- 172.21.15.121:0/1694375461 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4868058ff0 0x7f486805b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.761+0000 7f487effd700 1 -- 172.21.15.121:0/1694375461 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f4878073b20 con 0x555c58930bc0 2023-11-15T03:03:32.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.761+0000 7f487d5a0700 1 --2- 172.21.15.121:0/1694375461 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4868058ff0 0x7f486805b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.761+0000 7f489173ed00 1 --2- 172.21.15.121:0/1694375461 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x555c58960cf0 0x555c58a0ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.761+0000 7f489173ed00 1 -- 172.21.15.121:0/1694375461 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 13.33 13:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x555c58a15b40 con 0x555c58960cf0 2023-11-15T03:03:32.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.761+0000 7f48865a2700 1 --2- 172.21.15.121:0/1694375461 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x555c58960cf0 0x555c58a0ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.761+0000 7f487d5a0700 1 --2- 172.21.15.121:0/1694375461 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4868058ff0 0x7f486805b4b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x555c589353c0 tx=0x7f4870016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.761+0000 7f48865a2700 1 --2- 172.21.15.121:0/1694375461 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x555c58960cf0 0x555c58a0ff60 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.761+0000 7f48865a2700 1 -- 172.21.15.121:0/1694375461 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 100.00000000 [omap-get-header out=274b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7f4880014430 con 0x555c58960cf0 2023-11-15T03:03:32.780 INFO:teuthology.orchestra.run.smithi121.stdout:header (274 bytes) : 2023-11-15T03:03:32.780 INFO:teuthology.orchestra.run.smithi121.stdout:00000000 04 03 0c 01 00 00 01 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:32.780 INFO:teuthology.orchestra.run.smithi121.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2023-11-15T03:03:32.780 INFO:teuthology.orchestra.run.smithi121.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:32.781 INFO:teuthology.orchestra.run.smithi121.stdout:00000030 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:32.781 INFO:teuthology.orchestra.run.smithi121.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2023-11-15T03:03:32.781 INFO:teuthology.orchestra.run.smithi121.stdout:00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:32.781 INFO:teuthology.orchestra.run.smithi121.stdout:00000060 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:32.781 INFO:teuthology.orchestra.run.smithi121.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2023-11-15T03:03:32.782 INFO:teuthology.orchestra.run.smithi121.stdout:00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:32.783 INFO:teuthology.orchestra.run.smithi121.stdout:00000090 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:32.783 INFO:teuthology.orchestra.run.smithi121.stdout:000000a0 00 00 00 00 00 00 00 00 f9 34 54 65 aa d9 64 2f |.........4Te..d/| 2023-11-15T03:03:32.784 INFO:teuthology.orchestra.run.smithi121.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2023-11-15T03:03:32.784 INFO:teuthology.orchestra.run.smithi121.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 |................| 2023-11-15T03:03:32.784 INFO:teuthology.orchestra.run.smithi121.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:32.784 INFO:teuthology.orchestra.run.smithi121.stdout:000000e0 00 00 00 00 00 00 f9 34 54 65 aa d9 64 2f 00 00 |.......4Te..d/..| 2023-11-15T03:03:32.785 INFO:teuthology.orchestra.run.smithi121.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:32.785 INFO:teuthology.orchestra.run.smithi121.stdout:* 2023-11-15T03:03:32.785 INFO:teuthology.orchestra.run.smithi121.stdout:00000110 00 00 |..| 2023-11-15T03:03:32.785 INFO:teuthology.orchestra.run.smithi121.stdout:00000112 2023-11-15T03:03:32.785 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:32.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.765+0000 7f489173ed00 1 -- 172.21.15.121:0/1694375461 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x555c58960cf0 msgr2=0x555c58a0ff60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.765+0000 7f489173ed00 1 --2- 172.21.15.121:0/1694375461 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x555c58960cf0 0x555c58a0ff60 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.765+0000 7f489173ed00 1 -- 172.21.15.121:0/1694375461 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4868058ff0 msgr2=0x7f486805b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.765+0000 7f489173ed00 1 --2- 172.21.15.121:0/1694375461 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4868058ff0 0x7f486805b4b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x555c589353c0 tx=0x7f4870016000).stop 2023-11-15T03:03:32.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.765+0000 7f489173ed00 1 -- 172.21.15.121:0/1694375461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c58930bc0 msgr2=0x555c589fecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.765+0000 7f489173ed00 1 --2- 172.21.15.121:0/1694375461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c58930bc0 0x555c589fecf0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7f48780109c0 tx=0x7f4878005d40).stop 2023-11-15T03:03:32.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.765+0000 7f48865a2700 1 -- 172.21.15.121:0/1694375461 reap_dead start 2023-11-15T03:03:32.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.765+0000 7f489173ed00 1 -- 172.21.15.121:0/1694375461 shutdown_connections 2023-11-15T03:03:32.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.765+0000 7f489173ed00 1 -- 172.21.15.121:0/1694375461 >> 172.21.15.121:0/1694375461 conn(0x555c58977f80 msgr2=0x555c58979590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.765+0000 7f489173ed00 1 -- 172.21.15.121:0/1694375461 shutdown_connections 2023-11-15T03:03:32.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.765+0000 7f489173ed00 1 -- 172.21.15.121:0/1694375461 wait complete. 2023-11-15T03:03:32.790 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_sessionmap 2023-11-15T03:03:32.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f0016937d00 1 Processor -- start 2023-11-15T03:03:32.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f0016937d00 1 -- start start 2023-11-15T03:03:32.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f0016937d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55933e53b230 0x55933e603a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f0016937d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55933e569830 0x55933e567260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f0016937d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55933e60fa30 0x55933e567750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f0016937d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933e603f70 con 0x55933e60fa30 2023-11-15T03:03:32.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f0016937d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933e60c490 con 0x55933e569830 2023-11-15T03:03:32.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f0016937d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55933e566dd0 con 0x55933e53b230 2023-11-15T03:03:32.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f000b71b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55933e60fa30 0x55933e567750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f000b71b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55933e60fa30 0x55933e567750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50166/0 (socket says 172.21.15.121:50166) 2023-11-15T03:03:32.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f000b71b700 1 -- 172.21.15.121:0/3763666313 learned_addr learned my addr 172.21.15.121:0/3763666313 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f000af1a700 1 --2- 172.21.15.121:0/3763666313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55933e53b230 0x55933e603a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f0002719700 1 --2- 172.21.15.121:0/3763666313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55933e569830 0x55933e567260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f000b71b700 1 -- 172.21.15.121:0/3763666313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55933e53b230 msgr2=0x55933e603a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f000b71b700 1 --2- 172.21.15.121:0/3763666313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55933e53b230 0x55933e603a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f000b71b700 1 -- 172.21.15.121:0/3763666313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55933e569830 msgr2=0x55933e567260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f000b71b700 1 --2- 172.21.15.121:0/3763666313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55933e569830 0x55933e567260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f000b71b700 1 -- 172.21.15.121:0/3763666313 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000400a180 con 0x55933e60fa30 2023-11-15T03:03:32.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f0002719700 1 --2- 172.21.15.121:0/3763666313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55933e569830 0x55933e567260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:32.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f000b71b700 1 --2- 172.21.15.121:0/3763666313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55933e60fa30 0x55933e567750 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f000400ac00 tx=0x7f0004014000).ready entity=mon.0 client_cookie=4b20b05ff1b56f69 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f0009f18700 1 -- 172.21.15.121:0/3763666313 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00040116e0 con 0x55933e60fa30 2023-11-15T03:03:32.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f0009f18700 1 -- 172.21.15.121:0/3763666313 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0004011840 con 0x55933e60fa30 2023-11-15T03:03:32.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.849+0000 7f0009f18700 1 -- 172.21.15.121:0/3763666313 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0004011e70 con 0x55933e60fa30 2023-11-15T03:03:32.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.853+0000 7f0016937d00 1 -- 172.21.15.121:0/3763666313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55933e60fa30 msgr2=0x55933e567750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.853+0000 7f0016937d00 1 --2- 172.21.15.121:0/3763666313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55933e60fa30 0x55933e567750 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f000400ac00 tx=0x7f0004014000).stop 2023-11-15T03:03:32.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.853+0000 7f0016937d00 1 -- 172.21.15.121:0/3763666313 shutdown_connections 2023-11-15T03:03:32.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.853+0000 7f0016937d00 1 --2- 172.21.15.121:0/3763666313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55933e60fa30 0x55933e567750 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.853+0000 7f0016937d00 1 --2- 172.21.15.121:0/3763666313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55933e569830 0x55933e567260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.853+0000 7f0016937d00 1 --2- 172.21.15.121:0/3763666313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55933e53b230 0x55933e603a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.853+0000 7f0016937d00 1 -- 172.21.15.121:0/3763666313 >> 172.21.15.121:0/3763666313 conn(0x55933e5397d0 msgr2=0x55933e5386c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.853+0000 7f0016937d00 1 -- 172.21.15.121:0/3763666313 shutdown_connections 2023-11-15T03:03:32.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.853+0000 7f0016937d00 1 -- 172.21.15.121:0/3763666313 wait complete. 2023-11-15T03:03:32.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.853+0000 7f0016937d00 1 Processor -- start 2023-11-15T03:03:32.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.853+0000 7f0016937d00 1 -- start start 2023-11-15T03:03:32.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f0016937d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55933e53b230 0x55933e602d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f0016937d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55933e569830 0x55933e611230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f0016937d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55933e60fa30 0x55933e611720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f0016937d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933e5ff980 con 0x55933e53b230 2023-11-15T03:03:32.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f0016937d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933e565280 con 0x55933e60fa30 2023-11-15T03:03:32.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f0016937d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55933e601e20 con 0x55933e569830 2023-11-15T03:03:32.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f000af1a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55933e53b230 0x55933e602d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f000af1a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55933e53b230 0x55933e602d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50180/0 (socket says 172.21.15.121:50180) 2023-11-15T03:03:32.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f000af1a700 1 -- 172.21.15.121:0/3985980260 learned_addr learned my addr 172.21.15.121:0/3985980260 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f000b71b700 1 --2- 172.21.15.121:0/3985980260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55933e60fa30 0x55933e611720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f0002719700 1 --2- 172.21.15.121:0/3985980260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55933e569830 0x55933e611230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f000af1a700 1 -- 172.21.15.121:0/3985980260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55933e569830 msgr2=0x55933e611230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f000af1a700 1 --2- 172.21.15.121:0/3985980260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55933e569830 0x55933e611230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f000af1a700 1 -- 172.21.15.121:0/3985980260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55933e60fa30 msgr2=0x55933e611720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f000af1a700 1 --2- 172.21.15.121:0/3985980260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55933e60fa30 0x55933e611720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f000af1a700 1 -- 172.21.15.121:0/3985980260 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55933e4df7f0 con 0x55933e53b230 2023-11-15T03:03:32.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f000b71b700 1 --2- 172.21.15.121:0/3985980260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55933e60fa30 0x55933e611720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:32.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f000af1a700 1 --2- 172.21.15.121:0/3985980260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55933e53b230 0x55933e602d40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7efffc008b50 tx=0x7efffc00c040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f0003fff700 1 -- 172.21.15.121:0/3985980260 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc00f030 con 0x55933e53b230 2023-11-15T03:03:32.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f0003fff700 1 -- 172.21.15.121:0/3985980260 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efffc010040 con 0x55933e53b230 2023-11-15T03:03:32.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f0003fff700 1 -- 172.21.15.121:0/3985980260 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc006cb0 con 0x55933e53b230 2023-11-15T03:03:32.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f0016937d00 1 -- 172.21.15.121:0/3985980260 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55933e4daa20 con 0x55933e53b230 2023-11-15T03:03:32.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.857+0000 7f0016937d00 1 -- 172.21.15.121:0/3985980260 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55933e4da770 con 0x55933e53b230 2023-11-15T03:03:32.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.861+0000 7f0003fff700 1 -- 172.21.15.121:0/3985980260 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7efffc015670 con 0x55933e53b230 2023-11-15T03:03:32.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.861+0000 7f0003fff700 1 --2- 172.21.15.121:0/3985980260 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effec058ff0 0x7effec05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.861+0000 7f0003fff700 1 -- 172.21.15.121:0/3985980260 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7efffc0735b0 con 0x55933e53b230 2023-11-15T03:03:32.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.861+0000 7f0002719700 1 --2- 172.21.15.121:0/3985980260 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effec058ff0 0x7effec05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.861+0000 7f0016937d00 1 --2- 172.21.15.121:0/3985980260 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55933e62a3c0 0x55933e5ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.861+0000 7f0016937d00 1 -- 172.21.15.121:0/3985980260 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55933e622320 con 0x55933e62a3c0 2023-11-15T03:03:32.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.861+0000 7f0002719700 1 --2- 172.21.15.121:0/3985980260 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effec058ff0 0x7effec05b4b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x55933e615dd0 tx=0x7efff4005a50).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.861+0000 7f000b71b700 1 --2- 172.21.15.121:0/3985980260 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55933e62a3c0 0x55933e5ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.865+0000 7f000b71b700 1 --2- 172.21.15.121:0/3985980260 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55933e62a3c0 0x55933e5ffff0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.865+0000 7f000b71b700 1 -- 172.21.15.121:0/3985980260 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-header out=14b] v0'0 uv2 ondisk = 0) v8 ==== 159+0+14 (crc 0 0 0) 0x7f00040119b0 con 0x55933e62a3c0 2023-11-15T03:03:32.890 INFO:teuthology.orchestra.run.smithi121.stdout:header (14 bytes) : 2023-11-15T03:03:32.890 INFO:teuthology.orchestra.run.smithi121.stdout:00000000 01 01 08 00 00 00 04 00 00 00 00 00 00 00 |..............| 2023-11-15T03:03:32.890 INFO:teuthology.orchestra.run.smithi121.stdout:0000000e 2023-11-15T03:03:32.890 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:32.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.865+0000 7f0016937d00 1 -- 172.21.15.121:0/3985980260 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55933e62a3c0 msgr2=0x55933e5ffff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.865+0000 7f0016937d00 1 --2- 172.21.15.121:0/3985980260 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55933e62a3c0 0x55933e5ffff0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.865+0000 7f0016937d00 1 -- 172.21.15.121:0/3985980260 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effec058ff0 msgr2=0x7effec05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.865+0000 7f0016937d00 1 --2- 172.21.15.121:0/3985980260 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7effec058ff0 0x7effec05b4b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x55933e615dd0 tx=0x7efff4005a50).stop 2023-11-15T03:03:32.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.865+0000 7f0016937d00 1 -- 172.21.15.121:0/3985980260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55933e53b230 msgr2=0x55933e602d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.865+0000 7f0016937d00 1 --2- 172.21.15.121:0/3985980260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55933e53b230 0x55933e602d40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7efffc008b50 tx=0x7efffc00c040).stop 2023-11-15T03:03:32.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.865+0000 7f000b71b700 1 -- 172.21.15.121:0/3985980260 reap_dead start 2023-11-15T03:03:32.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.865+0000 7f0016937d00 1 -- 172.21.15.121:0/3985980260 shutdown_connections 2023-11-15T03:03:32.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.865+0000 7f0016937d00 1 -- 172.21.15.121:0/3985980260 >> 172.21.15.121:0/3985980260 conn(0x55933e5397d0 msgr2=0x55933e60e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.865+0000 7f0016937d00 1 -- 172.21.15.121:0/3985980260 shutdown_connections 2023-11-15T03:03:32.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.865+0000 7f0016937d00 1 -- 172.21.15.121:0/3985980260 wait complete. 2023-11-15T03:03:32.896 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 400.00000000 2023-11-15T03:03:32.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.953+0000 7f74cfadad00 1 Processor -- start 2023-11-15T03:03:32.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.953+0000 7f74cfadad00 1 -- start start 2023-11-15T03:03:32.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.953+0000 7f74cfadad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d19140bb0 0x556d19175140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.953+0000 7f74cfadad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d1918ab00 0x556d19173240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.953+0000 7f74cfadad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556d1921d650 0x556d19185ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.953+0000 7f74cfadad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d190b9220 con 0x556d19140bb0 2023-11-15T03:03:32.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cfadad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d1921bd00 con 0x556d1918ab00 2023-11-15T03:03:32.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cfadad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d1920aa00 con 0x556d1921d650 2023-11-15T03:03:32.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cd35e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d19140bb0 0x556d19175140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cd35e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d19140bb0 0x556d19175140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50192/0 (socket says 172.21.15.121:50192) 2023-11-15T03:03:32.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cd35e700 1 -- 172.21.15.121:0/1175700009 learned_addr learned my addr 172.21.15.121:0/1175700009 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cdb5f700 1 --2- 172.21.15.121:0/1175700009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556d1921d650 0x556d19185ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74c4b5d700 1 --2- 172.21.15.121:0/1175700009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d1918ab00 0x556d19173240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cd35e700 1 -- 172.21.15.121:0/1175700009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556d1921d650 msgr2=0x556d19185ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cd35e700 1 --2- 172.21.15.121:0/1175700009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556d1921d650 0x556d19185ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cd35e700 1 -- 172.21.15.121:0/1175700009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d1918ab00 msgr2=0x556d19173240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:32.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cd35e700 1 --2- 172.21.15.121:0/1175700009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d1918ab00 0x556d19173240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cd35e700 1 -- 172.21.15.121:0/1175700009 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c8005cf0 con 0x556d19140bb0 2023-11-15T03:03:32.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cd35e700 1 --2- 172.21.15.121:0/1175700009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d19140bb0 0x556d19175140 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f74c0006f30 tx=0x7f74c0006a10).ready entity=mon.0 client_cookie=7112971d32b5835 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74c7fff700 1 -- 172.21.15.121:0/1175700009 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0006110 con 0x556d19140bb0 2023-11-15T03:03:32.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74c7fff700 1 -- 172.21.15.121:0/1175700009 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c0006270 con 0x556d19140bb0 2023-11-15T03:03:32.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74c7fff700 1 -- 172.21.15.121:0/1175700009 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0004770 con 0x556d19140bb0 2023-11-15T03:03:32.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cfadad00 1 -- 172.21.15.121:0/1175700009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d19140bb0 msgr2=0x556d19175140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cfadad00 1 --2- 172.21.15.121:0/1175700009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d19140bb0 0x556d19175140 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f74c0006f30 tx=0x7f74c0006a10).stop 2023-11-15T03:03:32.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cfadad00 1 -- 172.21.15.121:0/1175700009 shutdown_connections 2023-11-15T03:03:32.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cfadad00 1 --2- 172.21.15.121:0/1175700009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556d1921d650 0x556d19185ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cfadad00 1 --2- 172.21.15.121:0/1175700009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d1918ab00 0x556d19173240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cfadad00 1 --2- 172.21.15.121:0/1175700009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d19140bb0 0x556d19175140 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cfadad00 1 -- 172.21.15.121:0/1175700009 >> 172.21.15.121:0/1175700009 conn(0x556d19190080 msgr2=0x556d19220db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:32.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cfadad00 1 -- 172.21.15.121:0/1175700009 shutdown_connections 2023-11-15T03:03:32.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.957+0000 7f74cfadad00 1 -- 172.21.15.121:0/1175700009 wait complete. 2023-11-15T03:03:32.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74cfadad00 1 Processor -- start 2023-11-15T03:03:32.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74cfadad00 1 -- start start 2023-11-15T03:03:32.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74cfadad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d19140bb0 0x556d19220e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74cfadad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556d1918ab00 0x556d192131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74cfadad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d1921d650 0x556d19185fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74cfadad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d19221350 con 0x556d19140bb0 2023-11-15T03:03:32.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74cfadad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d1918a080 con 0x556d1921d650 2023-11-15T03:03:32.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74cfadad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d1918a360 con 0x556d1918ab00 2023-11-15T03:03:32.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74cd35e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d19140bb0 0x556d19220e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74cd35e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d19140bb0 0x556d19220e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50200/0 (socket says 172.21.15.121:50200) 2023-11-15T03:03:32.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74cd35e700 1 -- 172.21.15.121:0/2753673780 learned_addr learned my addr 172.21.15.121:0/2753673780 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:32.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74cdb5f700 1 --2- 172.21.15.121:0/2753673780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d1921d650 0x556d19185fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74c4b5d700 1 --2- 172.21.15.121:0/2753673780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556d1918ab00 0x556d192131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74cd35e700 1 -- 172.21.15.121:0/2753673780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556d1918ab00 msgr2=0x556d192131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74cd35e700 1 --2- 172.21.15.121:0/2753673780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556d1918ab00 0x556d192131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74cd35e700 1 -- 172.21.15.121:0/2753673780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d1921d650 msgr2=0x556d19185fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74cd35e700 1 --2- 172.21.15.121:0/2753673780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d1921d650 0x556d19185fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74cd35e700 1 -- 172.21.15.121:0/2753673780 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc001e10 con 0x556d19140bb0 2023-11-15T03:03:32.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74cd35e700 1 --2- 172.21.15.121:0/2753673780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d19140bb0 0x556d19220e60 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x556d192214c0 tx=0x7f74c0012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74c67fc700 1 -- 172.21.15.121:0/2753673780 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0006070 con 0x556d19140bb0 2023-11-15T03:03:32.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74c67fc700 1 -- 172.21.15.121:0/2753673780 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c0013620 con 0x556d19140bb0 2023-11-15T03:03:32.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74c67fc700 1 -- 172.21.15.121:0/2753673780 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c00139e0 con 0x556d19140bb0 2023-11-15T03:03:32.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.961+0000 7f74cfadad00 1 -- 172.21.15.121:0/2753673780 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556d190e2770 con 0x556d19140bb0 2023-11-15T03:03:32.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.965+0000 7f74cfadad00 1 -- 172.21.15.121:0/2753673780 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d190b1a70 con 0x556d19140bb0 2023-11-15T03:03:32.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.965+0000 7f74c67fc700 1 -- 172.21.15.121:0/2753673780 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f74c00123c0 con 0x556d19140bb0 2023-11-15T03:03:32.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74c67fc700 1 --2- 172.21.15.121:0/2753673780 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f74b0058ff0 0x7f74b005b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74c67fc700 1 -- 172.21.15.121:0/2753673780 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f74c00737e0 con 0x556d19140bb0 2023-11-15T03:03:32.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74c4b5d700 1 --2- 172.21.15.121:0/2753673780 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f74b0058ff0 0x7f74b005b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74cfadad00 1 --2- 172.21.15.121:0/2753673780 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556d1921dbe0 0x556d1921f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:32.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74cfadad00 1 -- 172.21.15.121:0/2753673780 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x556d1921fd80 con 0x556d1921dbe0 2023-11-15T03:03:32.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74cdb5f700 1 --2- 172.21.15.121:0/2753673780 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556d1921dbe0 0x556d1921f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:32.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74c4b5d700 1 --2- 172.21.15.121:0/2753673780 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f74b0058ff0 0x7f74b005b4b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x556d1918bcc0 tx=0x7f74b800c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74cdb5f700 1 --2- 172.21.15.121:0/2753673780 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556d1921dbe0 0x556d1921f890 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:32.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74cdb5f700 1 -- 172.21.15.121:0/2753673780 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 400.00000000 [omap-get-header] v0'0 uv1 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f74c80078e0 con 0x556d1921dbe0 2023-11-15T03:03:32.995 INFO:teuthology.orchestra.run.smithi121.stdout:header (0 bytes) : 2023-11-15T03:03:32.995 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:32.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74cfadad00 1 -- 172.21.15.121:0/2753673780 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556d1921dbe0 msgr2=0x556d1921f890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74cfadad00 1 --2- 172.21.15.121:0/2753673780 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556d1921dbe0 0x556d1921f890 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:32.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74cfadad00 1 -- 172.21.15.121:0/2753673780 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f74b0058ff0 msgr2=0x7f74b005b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74cfadad00 1 --2- 172.21.15.121:0/2753673780 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f74b0058ff0 0x7f74b005b4b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x556d1918bcc0 tx=0x7f74b800c040).stop 2023-11-15T03:03:32.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74cfadad00 1 -- 172.21.15.121:0/2753673780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d19140bb0 msgr2=0x556d19220e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:32.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74cfadad00 1 --2- 172.21.15.121:0/2753673780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d19140bb0 0x556d19220e60 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x556d192214c0 tx=0x7f74c0012000).stop 2023-11-15T03:03:32.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74cdb5f700 1 -- 172.21.15.121:0/2753673780 reap_dead start 2023-11-15T03:03:32.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74cfadad00 1 -- 172.21.15.121:0/2753673780 shutdown_connections 2023-11-15T03:03:32.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74cfadad00 1 -- 172.21.15.121:0/2753673780 >> 172.21.15.121:0/2753673780 conn(0x556d19190080 msgr2=0x556d19175140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74cfadad00 1 -- 172.21.15.121:0/2753673780 shutdown_connections 2023-11-15T03:03:33.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:32.969+0000 7f74cfadad00 1 -- 172.21.15.121:0/2753673780 wait complete. 2023-11-15T03:03:33.000 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 100.00000000.inode 2023-11-15T03:03:33.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.053+0000 7f2e3493ad00 1 Processor -- start 2023-11-15T03:03:33.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.053+0000 7f2e3493ad00 1 -- start start 2023-11-15T03:03:33.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e3493ad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f56876bd0 0x560f5684cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e3493ad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f56921dc0 0x560f5690b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e3493ad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f56925c30 0x560f5692b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e3493ad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f567c4220 con 0x560f56921dc0 2023-11-15T03:03:33.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e3493ad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5690aec0 con 0x560f56876bd0 2023-11-15T03:03:33.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e3493ad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f56893910 con 0x560f56925c30 2023-11-15T03:03:33.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e20818700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f56921dc0 0x560f5690b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e20818700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f56921dc0 0x560f5690b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50216/0 (socket says 172.21.15.121:50216) 2023-11-15T03:03:33.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e20818700 1 -- 172.21.15.121:0/555503186 learned_addr learned my addr 172.21.15.121:0/555503186 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e29019700 1 --2- 172.21.15.121:0/555503186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f56876bd0 0x560f5684cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e2981a700 1 --2- 172.21.15.121:0/555503186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f56925c30 0x560f5692b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e20818700 1 -- 172.21.15.121:0/555503186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f56925c30 msgr2=0x560f5692b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e20818700 1 --2- 172.21.15.121:0/555503186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f56925c30 0x560f5692b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e20818700 1 -- 172.21.15.121:0/555503186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f56876bd0 msgr2=0x560f5684cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e20818700 1 --2- 172.21.15.121:0/555503186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f56876bd0 0x560f5684cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e20818700 1 -- 172.21.15.121:0/555503186 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c005cf0 con 0x560f56921dc0 2023-11-15T03:03:33.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e2981a700 1 --2- 172.21.15.121:0/555503186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f56925c30 0x560f5692b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:33.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e20818700 1 --2- 172.21.15.121:0/555503186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f56921dc0 0x560f5690b310 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f2e14001d70 tx=0x7f2e14001820).ready entity=mon.0 client_cookie=bf7fd057c292130f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e23fff700 1 -- 172.21.15.121:0/555503186 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1400bb90 con 0x560f56921dc0 2023-11-15T03:03:33.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e23fff700 1 -- 172.21.15.121:0/555503186 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1400bcf0 con 0x560f56921dc0 2023-11-15T03:03:33.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e23fff700 1 -- 172.21.15.121:0/555503186 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1400e680 con 0x560f56921dc0 2023-11-15T03:03:33.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e3493ad00 1 -- 172.21.15.121:0/555503186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f56921dc0 msgr2=0x560f5690b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e3493ad00 1 --2- 172.21.15.121:0/555503186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f56921dc0 0x560f5690b310 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f2e14001d70 tx=0x7f2e14001820).stop 2023-11-15T03:03:33.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e3493ad00 1 -- 172.21.15.121:0/555503186 shutdown_connections 2023-11-15T03:03:33.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e3493ad00 1 --2- 172.21.15.121:0/555503186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f56925c30 0x560f5692b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e3493ad00 1 --2- 172.21.15.121:0/555503186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f56921dc0 0x560f5690b310 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e3493ad00 1 --2- 172.21.15.121:0/555503186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f56876bd0 0x560f5684cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e3493ad00 1 -- 172.21.15.121:0/555503186 >> 172.21.15.121:0/555503186 conn(0x560f56893f80 msgr2=0x560f569221a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e3493ad00 1 -- 172.21.15.121:0/555503186 shutdown_connections 2023-11-15T03:03:33.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e3493ad00 1 -- 172.21.15.121:0/555503186 wait complete. 2023-11-15T03:03:33.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.057+0000 7f2e3493ad00 1 Processor -- start 2023-11-15T03:03:33.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e3493ad00 1 -- start start 2023-11-15T03:03:33.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e3493ad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f56876bd0 0x560f5691b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e3493ad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f56925c30 0x560f5691e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e3493ad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f5691d070 0x560f5691f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e3493ad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5691cc50 con 0x560f5691d070 2023-11-15T03:03:33.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e3493ad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5691c2b0 con 0x560f56925c30 2023-11-15T03:03:33.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e3493ad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f5691c590 con 0x560f56876bd0 2023-11-15T03:03:33.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e2981a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f5691d070 0x560f5691f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e2981a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f5691d070 0x560f5691f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50218/0 (socket says 172.21.15.121:50218) 2023-11-15T03:03:33.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e2981a700 1 -- 172.21.15.121:0/2179756644 learned_addr learned my addr 172.21.15.121:0/2179756644 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e20818700 1 --2- 172.21.15.121:0/2179756644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f56925c30 0x560f5691e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e29019700 1 --2- 172.21.15.121:0/2179756644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f56876bd0 0x560f5691b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e2981a700 1 -- 172.21.15.121:0/2179756644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f56876bd0 msgr2=0x560f5691b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e2981a700 1 --2- 172.21.15.121:0/2179756644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560f56876bd0 0x560f5691b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e2981a700 1 -- 172.21.15.121:0/2179756644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f56925c30 msgr2=0x560f5691e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e2981a700 1 --2- 172.21.15.121:0/2179756644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f56925c30 0x560f5691e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e2981a700 1 -- 172.21.15.121:0/2179756644 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e18001e10 con 0x560f5691d070 2023-11-15T03:03:33.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e2981a700 1 --2- 172.21.15.121:0/2179756644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f5691d070 0x560f5691f770 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7f2e2400f980 tx=0x7f2e24007040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e227fc700 1 -- 172.21.15.121:0/2179756644 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2400d3f0 con 0x560f5691d070 2023-11-15T03:03:33.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e227fc700 1 -- 172.21.15.121:0/2179756644 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e24011610 con 0x560f5691d070 2023-11-15T03:03:33.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e3493ad00 1 -- 172.21.15.121:0/2179756644 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560f567ed770 con 0x560f5691d070 2023-11-15T03:03:33.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e227fc700 1 -- 172.21.15.121:0/2179756644 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e240119b0 con 0x560f5691d070 2023-11-15T03:03:33.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.061+0000 7f2e3493ad00 1 -- 172.21.15.121:0/2179756644 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f567bca70 con 0x560f5691d070 2023-11-15T03:03:33.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.065+0000 7f2e227fc700 1 -- 172.21.15.121:0/2179756644 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2e24011d20 con 0x560f5691d070 2023-11-15T03:03:33.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.065+0000 7f2e227fc700 1 --2- 172.21.15.121:0/2179756644 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2e0c059020 0x7f2e0c05b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.065+0000 7f2e227fc700 1 -- 172.21.15.121:0/2179756644 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f2e24073b90 con 0x560f5691d070 2023-11-15T03:03:33.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.065+0000 7f2e29019700 1 --2- 172.21.15.121:0/2179756644 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2e0c059020 0x7f2e0c05b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.065+0000 7f2e3493ad00 1 --2- 172.21.15.121:0/2179756644 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x560f5691d4f0 0x560f569328e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.065+0000 7f2e3493ad00 1 -- 172.21.15.121:0/2179756644 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x560f56932dd0 con 0x560f5691d4f0 2023-11-15T03:03:33.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.069+0000 7f2e20818700 1 --2- 172.21.15.121:0/2179756644 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x560f5691d4f0 0x560f569328e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.069+0000 7f2e29019700 1 --2- 172.21.15.121:0/2179756644 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2e0c059020 0x7f2e0c05b4e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f2e1c00a520 tx=0x7f2e1c015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.069+0000 7f2e20818700 1 --2- 172.21.15.121:0/2179756644 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x560f5691d4f0 0x560f569328e0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.069+0000 7f2e20818700 1 -- 172.21.15.121:0/2179756644 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 100.00000000.inode [omap-get-header] v0'0 uv3 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2e14000bc0 con 0x560f5691d4f0 2023-11-15T03:03:33.085 INFO:teuthology.orchestra.run.smithi121.stdout:header (0 bytes) : 2023-11-15T03:03:33.085 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:33.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.069+0000 7f2e3493ad00 1 -- 172.21.15.121:0/2179756644 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x560f5691d4f0 msgr2=0x560f569328e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.069+0000 7f2e3493ad00 1 --2- 172.21.15.121:0/2179756644 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x560f5691d4f0 0x560f569328e0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.069+0000 7f2e3493ad00 1 -- 172.21.15.121:0/2179756644 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2e0c059020 msgr2=0x7f2e0c05b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.069+0000 7f2e3493ad00 1 --2- 172.21.15.121:0/2179756644 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2e0c059020 0x7f2e0c05b4e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f2e1c00a520 tx=0x7f2e1c015000).stop 2023-11-15T03:03:33.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.069+0000 7f2e3493ad00 1 -- 172.21.15.121:0/2179756644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f5691d070 msgr2=0x560f5691f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.069+0000 7f2e3493ad00 1 --2- 172.21.15.121:0/2179756644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560f5691d070 0x560f5691f770 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7f2e2400f980 tx=0x7f2e24007040).stop 2023-11-15T03:03:33.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.069+0000 7f2e2981a700 1 -- 172.21.15.121:0/2179756644 reap_dead start 2023-11-15T03:03:33.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.069+0000 7f2e3493ad00 1 -- 172.21.15.121:0/2179756644 shutdown_connections 2023-11-15T03:03:33.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.069+0000 7f2e3493ad00 1 -- 172.21.15.121:0/2179756644 >> 172.21.15.121:0/2179756644 conn(0x560f56893f80 msgr2=0x560f56923320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.069+0000 7f2e3493ad00 1 -- 172.21.15.121:0/2179756644 shutdown_connections 2023-11-15T03:03:33.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.069+0000 7f2e3493ad00 1 -- 172.21.15.121:0/2179756644 wait complete. 2023-11-15T03:03:33.089 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 1.00000000 2023-11-15T03:03:33.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c4813dd00 1 Processor -- start 2023-11-15T03:03:33.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c4813dd00 1 -- start start 2023-11-15T03:03:33.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c4813dd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556e5c468dd0 0x556e5c49b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c4813dd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e5c4b2b00 0x556e5c4ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c4813dd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556e5c541670 0x556e5c5431c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c4813dd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e5c3e1220 con 0x556e5c4b2b00 2023-11-15T03:03:33.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c4813dd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e5c4b6d60 con 0x556e5c541670 2023-11-15T03:03:33.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c4813dd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556e5c4b1a60 con 0x556e5c468dd0 2023-11-15T03:03:33.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c34018700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e5c4b2b00 0x556e5c4ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c34018700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e5c4b2b00 0x556e5c4ae1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50240/0 (socket says 172.21.15.121:50240) 2023-11-15T03:03:33.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c34018700 1 -- 172.21.15.121:0/1657527499 learned_addr learned my addr 172.21.15.121:0/1657527499 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c3d01a700 1 --2- 172.21.15.121:0/1657527499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556e5c541670 0x556e5c5431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c3c819700 1 --2- 172.21.15.121:0/1657527499 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556e5c468dd0 0x556e5c49b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c34018700 1 -- 172.21.15.121:0/1657527499 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556e5c468dd0 msgr2=0x556e5c49b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c34018700 1 --2- 172.21.15.121:0/1657527499 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556e5c468dd0 0x556e5c49b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c34018700 1 -- 172.21.15.121:0/1657527499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556e5c541670 msgr2=0x556e5c5431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c34018700 1 --2- 172.21.15.121:0/1657527499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556e5c541670 0x556e5c5431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c34018700 1 -- 172.21.15.121:0/1657527499 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c005cf0 con 0x556e5c4b2b00 2023-11-15T03:03:33.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c34018700 1 --2- 172.21.15.121:0/1657527499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e5c4b2b00 0x556e5c4ae1a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7f4c24001d60 tx=0x7f4c240017e0).ready entity=mon.0 client_cookie=d33d177bd2b0a703 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c377fe700 1 -- 172.21.15.121:0/1657527499 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400bc80 con 0x556e5c4b2b00 2023-11-15T03:03:33.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c377fe700 1 -- 172.21.15.121:0/1657527499 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2400bde0 con 0x556e5c4b2b00 2023-11-15T03:03:33.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.145+0000 7f4c377fe700 1 -- 172.21.15.121:0/1657527499 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400e680 con 0x556e5c4b2b00 2023-11-15T03:03:33.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.149+0000 7f4c4813dd00 1 -- 172.21.15.121:0/1657527499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e5c4b2b00 msgr2=0x556e5c4ae1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.149+0000 7f4c4813dd00 1 --2- 172.21.15.121:0/1657527499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e5c4b2b00 0x556e5c4ae1a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7f4c24001d60 tx=0x7f4c240017e0).stop 2023-11-15T03:03:33.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.149+0000 7f4c4813dd00 1 -- 172.21.15.121:0/1657527499 shutdown_connections 2023-11-15T03:03:33.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.149+0000 7f4c4813dd00 1 --2- 172.21.15.121:0/1657527499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556e5c541670 0x556e5c5431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.149+0000 7f4c4813dd00 1 --2- 172.21.15.121:0/1657527499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e5c4b2b00 0x556e5c4ae1a0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.149+0000 7f4c4813dd00 1 --2- 172.21.15.121:0/1657527499 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556e5c468dd0 0x556e5c49b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.149+0000 7f4c4813dd00 1 -- 172.21.15.121:0/1657527499 >> 172.21.15.121:0/1657527499 conn(0x556e5c4b8080 msgr2=0x556e5c4b8460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.149+0000 7f4c4813dd00 1 -- 172.21.15.121:0/1657527499 shutdown_connections 2023-11-15T03:03:33.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.149+0000 7f4c4813dd00 1 -- 172.21.15.121:0/1657527499 wait complete. 2023-11-15T03:03:33.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.149+0000 7f4c4813dd00 1 Processor -- start 2023-11-15T03:03:33.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.149+0000 7f4c4813dd00 1 -- start start 2023-11-15T03:03:33.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.149+0000 7f4c4813dd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556e5c468dd0 0x556e5c540a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.149+0000 7f4c4813dd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e5c4b2b00 0x556e5c499520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.149+0000 7f4c4813dd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556e5c541670 0x556e5c53a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.149+0000 7f4c4813dd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e5c499cd0 con 0x556e5c4b2b00 2023-11-15T03:03:33.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.149+0000 7f4c4813dd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e5c499f20 con 0x556e5c541670 2023-11-15T03:03:33.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.149+0000 7f4c4813dd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556e5c49a200 con 0x556e5c468dd0 2023-11-15T03:03:33.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.153+0000 7f4c34018700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e5c4b2b00 0x556e5c499520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.153+0000 7f4c34018700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e5c4b2b00 0x556e5c499520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50242/0 (socket says 172.21.15.121:50242) 2023-11-15T03:03:33.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.153+0000 7f4c34018700 1 -- 172.21.15.121:0/2948429331 learned_addr learned my addr 172.21.15.121:0/2948429331 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.153+0000 7f4c3c819700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556e5c468dd0 0x556e5c540a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.153+0000 7f4c3d01a700 1 --2- 172.21.15.121:0/2948429331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556e5c541670 0x556e5c53a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.153+0000 7f4c34018700 1 -- 172.21.15.121:0/2948429331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556e5c468dd0 msgr2=0x556e5c540a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.153+0000 7f4c34018700 1 --2- 172.21.15.121:0/2948429331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556e5c468dd0 0x556e5c540a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.153+0000 7f4c34018700 1 -- 172.21.15.121:0/2948429331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556e5c541670 msgr2=0x556e5c53a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.153+0000 7f4c34018700 1 --2- 172.21.15.121:0/2948429331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556e5c541670 0x556e5c53a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.153+0000 7f4c34018700 1 -- 172.21.15.121:0/2948429331 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e5c40f7f0 con 0x556e5c4b2b00 2023-11-15T03:03:33.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.153+0000 7f4c34018700 1 --2- 172.21.15.121:0/2948429331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e5c4b2b00 0x556e5c499520 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x556e5c46b1b0 tx=0x7f4c240017e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.153+0000 7f4c35ffb700 1 -- 172.21.15.121:0/2948429331 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400ee10 con 0x556e5c4b2b00 2023-11-15T03:03:33.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.153+0000 7f4c35ffb700 1 -- 172.21.15.121:0/2948429331 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2400d8c0 con 0x556e5c4b2b00 2023-11-15T03:03:33.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.153+0000 7f4c4813dd00 1 -- 172.21.15.121:0/2948429331 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556e5c40a770 con 0x556e5c4b2b00 2023-11-15T03:03:33.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.153+0000 7f4c35ffb700 1 -- 172.21.15.121:0/2948429331 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400dc80 con 0x556e5c4b2b00 2023-11-15T03:03:33.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.153+0000 7f4c4813dd00 1 -- 172.21.15.121:0/2948429331 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e5c3d9a70 con 0x556e5c4b2b00 2023-11-15T03:03:33.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.157+0000 7f4c35ffb700 1 -- 172.21.15.121:0/2948429331 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4c2401a640 con 0x556e5c4b2b00 2023-11-15T03:03:33.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.157+0000 7f4c35ffb700 1 --2- 172.21.15.121:0/2948429331 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4c1c058ff0 0x7f4c1c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.157+0000 7f4c35ffb700 1 -- 172.21.15.121:0/2948429331 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f4c24075f00 con 0x556e5c4b2b00 2023-11-15T03:03:33.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.157+0000 7f4c3c819700 1 --2- 172.21.15.121:0/2948429331 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4c1c058ff0 0x7f4c1c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.157+0000 7f4c4813dd00 1 --2- 172.21.15.121:0/2948429331 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556e5c542310 0x556e5c498010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.157+0000 7f4c4813dd00 1 -- 172.21.15.121:0/2948429331 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.3f 13:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x556e5c498500 con 0x556e5c542310 2023-11-15T03:03:33.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.157+0000 7f4c3c819700 1 --2- 172.21.15.121:0/2948429331 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4c1c058ff0 0x7f4c1c05b4b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f4c2c00afd0 tx=0x7f4c2c016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.157+0000 7f4c3d01a700 1 --2- 172.21.15.121:0/2948429331 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556e5c542310 0x556e5c498010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.157+0000 7f4c3d01a700 1 --2- 172.21.15.121:0/2948429331 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556e5c542310 0x556e5c498010 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.161+0000 7f4c3d01a700 1 -- 172.21.15.121:0/2948429331 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 1.00000000 [omap-get-header out=274b] v0'0 uv5 ondisk = 0) v8 ==== 154+0+274 (crc 0 0 0) 0x7f4c3800ca20 con 0x556e5c542310 2023-11-15T03:03:33.170 INFO:teuthology.orchestra.run.smithi121.stdout:header (274 bytes) : 2023-11-15T03:03:33.174 INFO:teuthology.orchestra.run.smithi121.stdout:00000000 04 03 0c 01 00 00 0d 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:33.175 INFO:teuthology.orchestra.run.smithi121.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2023-11-15T03:03:33.175 INFO:teuthology.orchestra.run.smithi121.stdout:00000020 00 00 00 00 01 35 54 65 e0 96 c4 10 00 00 00 00 |.....5Te........| 2023-11-15T03:03:33.175 INFO:teuthology.orchestra.run.smithi121.stdout:00000030 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 |................| 2023-11-15T03:03:33.175 INFO:teuthology.orchestra.run.smithi121.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2023-11-15T03:03:33.176 INFO:teuthology.orchestra.run.smithi121.stdout:00000050 00 00 01 35 54 65 e0 96 c4 10 00 00 00 00 00 00 |...5Te..........| 2023-11-15T03:03:33.176 INFO:teuthology.orchestra.run.smithi121.stdout:00000060 00 00 01 00 00 00 00 00 00 00 01 00 00 00 00 00 |................| 2023-11-15T03:03:33.176 INFO:teuthology.orchestra.run.smithi121.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2023-11-15T03:03:33.176 INFO:teuthology.orchestra.run.smithi121.stdout:00000080 00 00 60 00 00 00 00 00 01 00 00 00 00 00 00 00 |..`.............| 2023-11-15T03:03:33.177 INFO:teuthology.orchestra.run.smithi121.stdout:00000090 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:33.177 INFO:teuthology.orchestra.run.smithi121.stdout:000000a0 00 00 00 00 00 00 00 00 01 35 54 65 6d 91 ae 17 |.........5Tem...| 2023-11-15T03:03:33.177 INFO:teuthology.orchestra.run.smithi121.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2023-11-15T03:03:33.177 INFO:teuthology.orchestra.run.smithi121.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 |................| 2023-11-15T03:03:33.183 INFO:teuthology.orchestra.run.smithi121.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:33.184 INFO:teuthology.orchestra.run.smithi121.stdout:000000e0 00 00 00 00 00 00 01 35 54 65 e0 96 c4 10 00 00 |.......5Te......| 2023-11-15T03:03:33.184 INFO:teuthology.orchestra.run.smithi121.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-11-15T03:03:33.184 INFO:teuthology.orchestra.run.smithi121.stdout:* 2023-11-15T03:03:33.184 INFO:teuthology.orchestra.run.smithi121.stdout:00000110 00 00 |..| 2023-11-15T03:03:33.185 INFO:teuthology.orchestra.run.smithi121.stdout:00000112 2023-11-15T03:03:33.185 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:33.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.161+0000 7f4c4813dd00 1 -- 172.21.15.121:0/2948429331 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556e5c542310 msgr2=0x556e5c498010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.161+0000 7f4c4813dd00 1 --2- 172.21.15.121:0/2948429331 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556e5c542310 0x556e5c498010 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.161+0000 7f4c4813dd00 1 -- 172.21.15.121:0/2948429331 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4c1c058ff0 msgr2=0x7f4c1c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.161+0000 7f4c4813dd00 1 --2- 172.21.15.121:0/2948429331 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4c1c058ff0 0x7f4c1c05b4b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f4c2c00afd0 tx=0x7f4c2c016000).stop 2023-11-15T03:03:33.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.161+0000 7f4c4813dd00 1 -- 172.21.15.121:0/2948429331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e5c4b2b00 msgr2=0x556e5c499520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.161+0000 7f4c4813dd00 1 --2- 172.21.15.121:0/2948429331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e5c4b2b00 0x556e5c499520 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x556e5c46b1b0 tx=0x7f4c240017e0).stop 2023-11-15T03:03:33.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.161+0000 7f4c3d01a700 1 -- 172.21.15.121:0/2948429331 reap_dead start 2023-11-15T03:03:33.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.161+0000 7f4c4813dd00 1 -- 172.21.15.121:0/2948429331 shutdown_connections 2023-11-15T03:03:33.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.161+0000 7f4c4813dd00 1 -- 172.21.15.121:0/2948429331 >> 172.21.15.121:0/2948429331 conn(0x556e5c4b8080 msgr2=0x556e5c5427e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.161+0000 7f4c4813dd00 1 -- 172.21.15.121:0/2948429331 shutdown_connections 2023-11-15T03:03:33.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.161+0000 7f4c4813dd00 1 -- 172.21.15.121:0/2948429331 wait complete. 2023-11-15T03:03:33.190 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000 2023-11-15T03:03:33.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.249+0000 7f4a31aedd00 1 Processor -- start 2023-11-15T03:03:33.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.249+0000 7f4a31aedd00 1 -- start start 2023-11-15T03:03:33.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.249+0000 7f4a31aedd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a8cfe956c0 0x55a8cff67ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.249+0000 7f4a31aedd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a8cfedfb00 0x55a8cfec4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.249+0000 7f4a31aedd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8cff6e670 0x55a8cfec50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.249+0000 7f4a31aedd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8cfe0e220 con 0x55a8cfedfb00 2023-11-15T03:03:33.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.249+0000 7f4a31aedd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8cfec76d0 con 0x55a8cff6e670 2023-11-15T03:03:33.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.249+0000 7f4a31aedd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8cfee0200 con 0x55a8cfe956c0 2023-11-15T03:03:33.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.249+0000 7f4a26bcc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a8cfedfb00 0x55a8cfec4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.249+0000 7f4a26bcc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a8cfedfb00 0x55a8cfec4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50256/0 (socket says 172.21.15.121:50256) 2023-11-15T03:03:33.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.249+0000 7f4a26bcc700 1 -- 172.21.15.121:0/274389416 learned_addr learned my addr 172.21.15.121:0/274389416 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.249+0000 7f4a2fbce700 1 --2- 172.21.15.121:0/274389416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8cff6e670 0x55a8cfec50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.249+0000 7f4a2f3cd700 1 --2- 172.21.15.121:0/274389416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a8cfe956c0 0x55a8cff67ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.253+0000 7f4a26bcc700 1 -- 172.21.15.121:0/274389416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a8cfe956c0 msgr2=0x55a8cff67ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.253+0000 7f4a26bcc700 1 --2- 172.21.15.121:0/274389416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a8cfe956c0 0x55a8cff67ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.253+0000 7f4a26bcc700 1 -- 172.21.15.121:0/274389416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8cff6e670 msgr2=0x55a8cfec50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.253+0000 7f4a26bcc700 1 --2- 172.21.15.121:0/274389416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8cff6e670 0x55a8cfec50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.253+0000 7f4a26bcc700 1 -- 172.21.15.121:0/274389416 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a18005cf0 con 0x55a8cfedfb00 2023-11-15T03:03:33.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.253+0000 7f4a26bcc700 1 --2- 172.21.15.121:0/274389416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a8cfedfb00 0x55a8cfec4c00 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f4a1800acc0 tx=0x7f4a18016000).ready entity=mon.0 client_cookie=6541df0a7a2e88a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.253+0000 7f4a2e3cb700 1 -- 172.21.15.121:0/274389416 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a180086a0 con 0x55a8cfedfb00 2023-11-15T03:03:33.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.253+0000 7f4a2e3cb700 1 -- 172.21.15.121:0/274389416 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a18005350 con 0x55a8cfedfb00 2023-11-15T03:03:33.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.253+0000 7f4a2e3cb700 1 -- 172.21.15.121:0/274389416 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a18000bc0 con 0x55a8cfedfb00 2023-11-15T03:03:33.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.253+0000 7f4a31aedd00 1 -- 172.21.15.121:0/274389416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a8cfedfb00 msgr2=0x55a8cfec4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.253+0000 7f4a31aedd00 1 --2- 172.21.15.121:0/274389416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a8cfedfb00 0x55a8cfec4c00 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f4a1800acc0 tx=0x7f4a18016000).stop 2023-11-15T03:03:33.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.253+0000 7f4a31aedd00 1 -- 172.21.15.121:0/274389416 shutdown_connections 2023-11-15T03:03:33.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.253+0000 7f4a31aedd00 1 --2- 172.21.15.121:0/274389416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8cff6e670 0x55a8cfec50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.253+0000 7f4a31aedd00 1 --2- 172.21.15.121:0/274389416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a8cfedfb00 0x55a8cfec4c00 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.253+0000 7f4a31aedd00 1 --2- 172.21.15.121:0/274389416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a8cfe956c0 0x55a8cff67ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.253+0000 7f4a31aedd00 1 -- 172.21.15.121:0/274389416 >> 172.21.15.121:0/274389416 conn(0x55a8cfee5080 msgr2=0x55a8cff70980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.253+0000 7f4a31aedd00 1 -- 172.21.15.121:0/274389416 shutdown_connections 2023-11-15T03:03:33.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.253+0000 7f4a31aedd00 1 -- 172.21.15.121:0/274389416 wait complete. 2023-11-15T03:03:33.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.253+0000 7f4a31aedd00 1 Processor -- start 2023-11-15T03:03:33.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.257+0000 7f4a31aedd00 1 -- start start 2023-11-15T03:03:33.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.257+0000 7f4a31aedd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8cfe956c0 0x55a8cfed9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.257+0000 7f4a31aedd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a8cfedfb00 0x55a8cfedf190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.257+0000 7f4a31aedd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a8cff6e670 0x55a8cff66bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.257+0000 7f4a31aedd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8cfeded00 con 0x55a8cfedfb00 2023-11-15T03:03:33.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.257+0000 7f4a31aedd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8cfede3d0 con 0x55a8cfe956c0 2023-11-15T03:03:33.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.257+0000 7f4a31aedd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8cfede6b0 con 0x55a8cff6e670 2023-11-15T03:03:33.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.257+0000 7f4a26bcc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a8cfedfb00 0x55a8cfedf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.257+0000 7f4a26bcc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a8cfedfb00 0x55a8cfedf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50264/0 (socket says 172.21.15.121:50264) 2023-11-15T03:03:33.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.257+0000 7f4a26bcc700 1 -- 172.21.15.121:0/656185964 learned_addr learned my addr 172.21.15.121:0/656185964 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.257+0000 7f4a2f3cd700 1 --2- 172.21.15.121:0/656185964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8cfe956c0 0x55a8cfed9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.257+0000 7f4a2fbce700 1 --2- 172.21.15.121:0/656185964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a8cff6e670 0x55a8cff66bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.257+0000 7f4a26bcc700 1 -- 172.21.15.121:0/656185964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a8cff6e670 msgr2=0x55a8cff66bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.257+0000 7f4a26bcc700 1 --2- 172.21.15.121:0/656185964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a8cff6e670 0x55a8cff66bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.257+0000 7f4a26bcc700 1 -- 172.21.15.121:0/656185964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8cfe956c0 msgr2=0x55a8cfed9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.257+0000 7f4a26bcc700 1 --2- 172.21.15.121:0/656185964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8cfe956c0 0x55a8cfed9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.257+0000 7f4a26bcc700 1 -- 172.21.15.121:0/656185964 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8cfe37a20 con 0x55a8cfedfb00 2023-11-15T03:03:33.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.257+0000 7f4a26bcc700 1 --2- 172.21.15.121:0/656185964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a8cfedfb00 0x55a8cfedf190 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7f4a1801f040 tx=0x7f4a1801c000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.257+0000 7f4a2cbc8700 1 -- 172.21.15.121:0/656185964 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a180085c0 con 0x55a8cfedfb00 2023-11-15T03:03:33.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.261+0000 7f4a2cbc8700 1 -- 172.21.15.121:0/656185964 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1802a070 con 0x55a8cfedfb00 2023-11-15T03:03:33.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.261+0000 7f4a31aedd00 1 -- 172.21.15.121:0/656185964 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a8cfe37770 con 0x55a8cfedfb00 2023-11-15T03:03:33.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.261+0000 7f4a2cbc8700 1 -- 172.21.15.121:0/656185964 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a18013690 con 0x55a8cfedfb00 2023-11-15T03:03:33.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.261+0000 7f4a31aedd00 1 -- 172.21.15.121:0/656185964 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8cfe06a70 con 0x55a8cfedfb00 2023-11-15T03:03:33.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.261+0000 7f4a2cbc8700 1 -- 172.21.15.121:0/656185964 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4a1802c030 con 0x55a8cfedfb00 2023-11-15T03:03:33.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.265+0000 7f4a2cbc8700 1 --2- 172.21.15.121:0/656185964 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a10058ff0 0x7f4a1005b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.265+0000 7f4a2cbc8700 1 -- 172.21.15.121:0/656185964 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f4a180763a0 con 0x55a8cfedfb00 2023-11-15T03:03:33.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.265+0000 7f4a2f3cd700 1 --2- 172.21.15.121:0/656185964 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a10058ff0 0x7f4a1005b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.265+0000 7f4a31aedd00 1 --2- 172.21.15.121:0/656185964 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55a8cff701f0 0x55a8cff7ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.265+0000 7f4a31aedd00 1 -- 172.21.15.121:0/656185964 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:1 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55a8cff7ef10 con 0x55a8cff701f0 2023-11-15T03:03:33.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.265+0000 7f4a2f3cd700 1 --2- 172.21.15.121:0/656185964 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a10058ff0 0x7f4a1005b4b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f4a20005fd0 tx=0x7f4a2000c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.265+0000 7f4a2fbce700 1 --2- 172.21.15.121:0/656185964 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55a8cff701f0 0x55a8cff7ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.265+0000 7f4a2fbce700 1 --2- 172.21.15.121:0/656185964 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55a8cff701f0 0x55a8cff7ea20 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.265+0000 7f4a2fbce700 1 -- 172.21.15.121:0/656185964 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-keys out=21b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+21 (crc 0 0 0) 0x7f4a28004020 con 0x55a8cff701f0 2023-11-15T03:03:33.276 INFO:teuthology.orchestra.run.smithi121.stdout:sixmegs_head 2023-11-15T03:03:33.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.265+0000 7f4a31aedd00 1 -- 172.21.15.121:0/656185964 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55a8cff701f0 msgr2=0x55a8cff7ea20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.265+0000 7f4a31aedd00 1 --2- 172.21.15.121:0/656185964 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55a8cff701f0 0x55a8cff7ea20 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.265+0000 7f4a31aedd00 1 -- 172.21.15.121:0/656185964 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a10058ff0 msgr2=0x7f4a1005b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.265+0000 7f4a31aedd00 1 --2- 172.21.15.121:0/656185964 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a10058ff0 0x7f4a1005b4b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f4a20005fd0 tx=0x7f4a2000c040).stop 2023-11-15T03:03:33.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.265+0000 7f4a31aedd00 1 -- 172.21.15.121:0/656185964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a8cfedfb00 msgr2=0x55a8cfedf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.265+0000 7f4a31aedd00 1 --2- 172.21.15.121:0/656185964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a8cfedfb00 0x55a8cfedf190 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7f4a1801f040 tx=0x7f4a1801c000).stop 2023-11-15T03:03:33.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.265+0000 7f4a2fbce700 1 -- 172.21.15.121:0/656185964 reap_dead start 2023-11-15T03:03:33.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.269+0000 7f4a31aedd00 1 -- 172.21.15.121:0/656185964 shutdown_connections 2023-11-15T03:03:33.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.269+0000 7f4a31aedd00 1 -- 172.21.15.121:0/656185964 >> 172.21.15.121:0/656185964 conn(0x55a8cfee5080 msgr2=0x55a8cfec8110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.269+0000 7f4a31aedd00 1 -- 172.21.15.121:0/656185964 shutdown_connections 2023-11-15T03:03:33.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.269+0000 7f4a31aedd00 1 -- 172.21.15.121:0/656185964 wait complete. 2023-11-15T03:03:33.281 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 600.00000000 2023-11-15T03:03:33.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.333+0000 7f282af38d00 1 Processor -- start 2023-11-15T03:03:33.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.333+0000 7f282af38d00 1 -- start start 2023-11-15T03:03:33.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f282af38d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556541254dd0 0x5565412545f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f282af38d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55654129eb00 0x55654132ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f282af38d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55654132d670 0x55654129a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f282af38d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565411cd220 con 0x556541254dd0 2023-11-15T03:03:33.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f282af38d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556541330430 con 0x55654132d670 2023-11-15T03:03:33.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f282af38d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55654129dfa0 con 0x55654129eb00 2023-11-15T03:03:33.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f281f619700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556541254dd0 0x5565412545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f281f619700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556541254dd0 0x5565412545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50270/0 (socket says 172.21.15.121:50270) 2023-11-15T03:03:33.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f281f619700 1 -- 172.21.15.121:0/478180377 learned_addr learned my addr 172.21.15.121:0/478180377 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f281fe1a700 1 --2- 172.21.15.121:0/478180377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55654132d670 0x55654129a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f2817fff700 1 --2- 172.21.15.121:0/478180377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55654129eb00 0x55654132ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f281f619700 1 -- 172.21.15.121:0/478180377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55654129eb00 msgr2=0x55654132ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f281f619700 1 --2- 172.21.15.121:0/478180377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55654129eb00 0x55654132ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f281f619700 1 -- 172.21.15.121:0/478180377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55654132d670 msgr2=0x55654129a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f281f619700 1 --2- 172.21.15.121:0/478180377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55654132d670 0x55654129a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f281f619700 1 -- 172.21.15.121:0/478180377 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2810005cf0 con 0x556541254dd0 2023-11-15T03:03:33.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f2817fff700 1 --2- 172.21.15.121:0/478180377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55654129eb00 0x55654132ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:33.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f281f619700 1 --2- 172.21.15.121:0/478180377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556541254dd0 0x5565412545f0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7f281000acf0 tx=0x7f2810016000).ready entity=mon.0 client_cookie=de881dfd7ab4d887 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f281e617700 1 -- 172.21.15.121:0/478180377 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28100086a0 con 0x556541254dd0 2023-11-15T03:03:33.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f281e617700 1 -- 172.21.15.121:0/478180377 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2810005330 con 0x556541254dd0 2023-11-15T03:03:33.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f281e617700 1 -- 172.21.15.121:0/478180377 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2810000bc0 con 0x556541254dd0 2023-11-15T03:03:33.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f282af38d00 1 -- 172.21.15.121:0/478180377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556541254dd0 msgr2=0x5565412545f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f282af38d00 1 --2- 172.21.15.121:0/478180377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556541254dd0 0x5565412545f0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7f281000acf0 tx=0x7f2810016000).stop 2023-11-15T03:03:33.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f282af38d00 1 -- 172.21.15.121:0/478180377 shutdown_connections 2023-11-15T03:03:33.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f282af38d00 1 --2- 172.21.15.121:0/478180377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55654132d670 0x55654129a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f282af38d00 1 --2- 172.21.15.121:0/478180377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55654129eb00 0x55654132ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f282af38d00 1 --2- 172.21.15.121:0/478180377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556541254dd0 0x5565412545f0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f282af38d00 1 -- 172.21.15.121:0/478180377 >> 172.21.15.121:0/478180377 conn(0x5565412a4080 msgr2=0x556541286700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f282af38d00 1 -- 172.21.15.121:0/478180377 shutdown_connections 2023-11-15T03:03:33.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.337+0000 7f282af38d00 1 -- 172.21.15.121:0/478180377 wait complete. 2023-11-15T03:03:33.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f282af38d00 1 Processor -- start 2023-11-15T03:03:33.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f282af38d00 1 -- start start 2023-11-15T03:03:33.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f282af38d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556541254dd0 0x55654132c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f282af38d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55654129eb00 0x55654132cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f282af38d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55654132d670 0x55654129c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f282af38d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55654132cfc0 con 0x55654132d670 2023-11-15T03:03:33.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f282af38d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55654132a090 con 0x55654129eb00 2023-11-15T03:03:33.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f282af38d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55654132ac10 con 0x556541254dd0 2023-11-15T03:03:33.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f281fe1a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55654132d670 0x55654129c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f2817fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55654129eb00 0x55654132cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f281f619700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556541254dd0 0x55654132c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f2817fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55654129eb00 0x55654132cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:50886/0 (socket says 172.21.15.121:50886) 2023-11-15T03:03:33.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f2817fff700 1 -- 172.21.15.121:0/64380617 learned_addr learned my addr 172.21.15.121:0/64380617 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f281fe1a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55654132d670 0x55654129c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50276/0 (socket says 172.21.15.121:50276) 2023-11-15T03:03:33.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f2817fff700 1 -- 172.21.15.121:0/64380617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556541254dd0 msgr2=0x55654132c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f2817fff700 1 --2- 172.21.15.121:0/64380617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556541254dd0 0x55654132c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f2817fff700 1 -- 172.21.15.121:0/64380617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55654132d670 msgr2=0x55654129c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f2817fff700 1 --2- 172.21.15.121:0/64380617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55654132d670 0x55654129c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f2817fff700 1 -- 172.21.15.121:0/64380617 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565411f6a20 con 0x55654129eb00 2023-11-15T03:03:33.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.341+0000 7f281fe1a700 1 --2- 172.21.15.121:0/64380617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55654132d670 0x55654129c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:03:33.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.345+0000 7f2817fff700 1 --2- 172.21.15.121:0/64380617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55654129eb00 0x55654132cad0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f280800df10 tx=0x7f2808006a30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.345+0000 7f281ce14700 1 -- 172.21.15.121:0/64380617 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280800bda0 con 0x55654129eb00 2023-11-15T03:03:33.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.345+0000 7f281ce14700 1 -- 172.21.15.121:0/64380617 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280800f450 con 0x55654129eb00 2023-11-15T03:03:33.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.345+0000 7f281ce14700 1 -- 172.21.15.121:0/64380617 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280800f7f0 con 0x55654129eb00 2023-11-15T03:03:33.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.345+0000 7f282af38d00 1 -- 172.21.15.121:0/64380617 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5565411f6770 con 0x55654129eb00 2023-11-15T03:03:33.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.345+0000 7f282af38d00 1 -- 172.21.15.121:0/64380617 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565411c5a70 con 0x55654129eb00 2023-11-15T03:03:33.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.349+0000 7f281ce14700 1 -- 172.21.15.121:0/64380617 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f280800fc20 con 0x55654129eb00 2023-11-15T03:03:33.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.349+0000 7f281ce14700 1 --2- 172.21.15.121:0/64380617 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2800059040 0x7f280005b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.349+0000 7f281ce14700 1 -- 172.21.15.121:0/64380617 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f28080736e0 con 0x55654129eb00 2023-11-15T03:03:33.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.349+0000 7f282af38d00 1 --2- 172.21.15.121:0/64380617 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x556541323ed0 0x55654129cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.349+0000 7f281f619700 1 --2- 172.21.15.121:0/64380617 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2800059040 0x7f280005b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.349+0000 7f282af38d00 1 -- 172.21.15.121:0/64380617 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:1 13.30 13:0d82a743:::600.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55654129d5d0 con 0x556541323ed0 2023-11-15T03:03:33.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.349+0000 7f281fe1a700 1 --2- 172.21.15.121:0/64380617 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x556541323ed0 0x55654129cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.349+0000 7f281f619700 1 --2- 172.21.15.121:0/64380617 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2800059040 0x7f280005b500 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f281000acf0 tx=0x7f281001c000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.349+0000 7f281fe1a700 1 --2- 172.21.15.121:0/64380617 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x556541323ed0 0x55654129cdb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.353+0000 7f281fe1a700 1 -- 172.21.15.121:0/64380617 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(1 600.00000000 [omap-get-keys out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7f2818013540 con 0x556541323ed0 2023-11-15T03:03:33.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.353+0000 7f282af38d00 1 -- 172.21.15.121:0/64380617 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x556541323ed0 msgr2=0x55654129cdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.353+0000 7f282af38d00 1 --2- 172.21.15.121:0/64380617 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x556541323ed0 0x55654129cdb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.353+0000 7f282af38d00 1 -- 172.21.15.121:0/64380617 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2800059040 msgr2=0x7f280005b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.353+0000 7f282af38d00 1 --2- 172.21.15.121:0/64380617 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2800059040 0x7f280005b500 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f281000acf0 tx=0x7f281001c000).stop 2023-11-15T03:03:33.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.353+0000 7f282af38d00 1 -- 172.21.15.121:0/64380617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55654129eb00 msgr2=0x55654132cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.353+0000 7f282af38d00 1 --2- 172.21.15.121:0/64380617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55654129eb00 0x55654132cad0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f280800df10 tx=0x7f2808006a30).stop 2023-11-15T03:03:33.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.353+0000 7f281fe1a700 1 -- 172.21.15.121:0/64380617 reap_dead start 2023-11-15T03:03:33.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.353+0000 7f282af38d00 1 -- 172.21.15.121:0/64380617 shutdown_connections 2023-11-15T03:03:33.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.353+0000 7f282af38d00 1 -- 172.21.15.121:0/64380617 >> 172.21.15.121:0/64380617 conn(0x5565412a4080 msgr2=0x556541289900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.353+0000 7f282af38d00 1 -- 172.21.15.121:0/64380617 shutdown_connections 2023-11-15T03:03:33.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.353+0000 7f282af38d00 1 -- 172.21.15.121:0/64380617 wait complete. 2023-11-15T03:03:33.369 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000.inode 2023-11-15T03:03:33.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13c1da6d00 1 Processor -- start 2023-11-15T03:03:33.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13c1da6d00 1 -- start start 2023-11-15T03:03:33.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13c1da6d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558410cad6c0 0x558410ce0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13c1da6d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558410cf7b00 0x558410cf97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13c1da6d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558410d86670 0x558410cf9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13c1da6d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558410d89b90 con 0x558410d86670 2023-11-15T03:03:33.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13c1da6d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558410d7eef0 con 0x558410cad6c0 2023-11-15T03:03:33.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13c1da6d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558410d7f1d0 con 0x558410cf7b00 2023-11-15T03:03:33.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13bfeb8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558410d86670 0x558410cf9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13bfeb8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558410d86670 0x558410cf9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50294/0 (socket says 172.21.15.121:50294) 2023-11-15T03:03:33.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13bfeb8700 1 -- 172.21.15.121:0/2780661973 learned_addr learned my addr 172.21.15.121:0/2780661973 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13bf6b7700 1 --2- 172.21.15.121:0/2780661973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558410cad6c0 0x558410ce0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13b6eb6700 1 --2- 172.21.15.121:0/2780661973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558410cf7b00 0x558410cf97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13bfeb8700 1 -- 172.21.15.121:0/2780661973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558410cf7b00 msgr2=0x558410cf97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13bfeb8700 1 --2- 172.21.15.121:0/2780661973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558410cf7b00 0x558410cf97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13bfeb8700 1 -- 172.21.15.121:0/2780661973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558410cad6c0 msgr2=0x558410ce0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13bfeb8700 1 --2- 172.21.15.121:0/2780661973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558410cad6c0 0x558410ce0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13bfeb8700 1 -- 172.21.15.121:0/2780661973 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b800a180 con 0x558410d86670 2023-11-15T03:03:33.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13bfeb8700 1 --2- 172.21.15.121:0/2780661973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558410d86670 0x558410cf9cb0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x558410d80cd0 tx=0x7f13b8014000).ready entity=mon.0 client_cookie=10861671e43f258e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13be6b5700 1 -- 172.21.15.121:0/2780661973 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b8011410 con 0x558410d86670 2023-11-15T03:03:33.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13be6b5700 1 -- 172.21.15.121:0/2780661973 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b80056d0 con 0x558410d86670 2023-11-15T03:03:33.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.421+0000 7f13be6b5700 1 -- 172.21.15.121:0/2780661973 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b8011c60 con 0x558410d86670 2023-11-15T03:03:33.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13c1da6d00 1 -- 172.21.15.121:0/2780661973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558410d86670 msgr2=0x558410cf9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13c1da6d00 1 --2- 172.21.15.121:0/2780661973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558410d86670 0x558410cf9cb0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x558410d80cd0 tx=0x7f13b8014000).stop 2023-11-15T03:03:33.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13c1da6d00 1 -- 172.21.15.121:0/2780661973 shutdown_connections 2023-11-15T03:03:33.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13c1da6d00 1 --2- 172.21.15.121:0/2780661973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558410d86670 0x558410cf9cb0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13c1da6d00 1 --2- 172.21.15.121:0/2780661973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558410cf7b00 0x558410cf97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13c1da6d00 1 --2- 172.21.15.121:0/2780661973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558410cad6c0 0x558410ce0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13c1da6d00 1 -- 172.21.15.121:0/2780661973 >> 172.21.15.121:0/2780661973 conn(0x558410cfd080 msgr2=0x558410d88820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13c1da6d00 1 -- 172.21.15.121:0/2780661973 shutdown_connections 2023-11-15T03:03:33.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13c1da6d00 1 -- 172.21.15.121:0/2780661973 wait complete. 2023-11-15T03:03:33.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13c1da6d00 1 Processor -- start 2023-11-15T03:03:33.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13c1da6d00 1 -- start start 2023-11-15T03:03:33.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13c1da6d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558410cad6c0 0x558410cf5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13c1da6d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558410cf7b00 0x558410cf6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13c1da6d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558410d86670 0x558410d7ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13c1da6d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558410d7c470 con 0x558410cf7b00 2023-11-15T03:03:33.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13c1da6d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558410cddd10 con 0x558410cad6c0 2023-11-15T03:03:33.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13c1da6d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558410cddff0 con 0x558410d86670 2023-11-15T03:03:33.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13b6eb6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558410cf7b00 0x558410cf6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13bf6b7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558410cad6c0 0x558410cf5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13b6eb6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558410cf7b00 0x558410cf6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50298/0 (socket says 172.21.15.121:50298) 2023-11-15T03:03:33.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13b6eb6700 1 -- 172.21.15.121:0/1578195463 learned_addr learned my addr 172.21.15.121:0/1578195463 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.425+0000 7f13bfeb8700 1 --2- 172.21.15.121:0/1578195463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558410d86670 0x558410d7ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.429+0000 7f13b6eb6700 1 -- 172.21.15.121:0/1578195463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558410d86670 msgr2=0x558410d7ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.429+0000 7f13b6eb6700 1 --2- 172.21.15.121:0/1578195463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558410d86670 0x558410d7ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.429+0000 7f13b6eb6700 1 -- 172.21.15.121:0/1578195463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558410cad6c0 msgr2=0x558410cf5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.429+0000 7f13b6eb6700 1 --2- 172.21.15.121:0/1578195463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558410cad6c0 0x558410cf5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.429+0000 7f13b6eb6700 1 -- 172.21.15.121:0/1578195463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558410c547f0 con 0x558410cf7b00 2023-11-15T03:03:33.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.429+0000 7f13bf6b7700 1 --2- 172.21.15.121:0/1578195463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558410cad6c0 0x558410cf5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:03:33.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.429+0000 7f13b6eb6700 1 --2- 172.21.15.121:0/1578195463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558410cf7b00 0x558410cf6380 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7f13a8008af0 tx=0x7f13a800c040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.429+0000 7f13bceb2700 1 -- 172.21.15.121:0/1578195463 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a800f040 con 0x558410cf7b00 2023-11-15T03:03:33.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.429+0000 7f13bceb2700 1 -- 172.21.15.121:0/1578195463 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a80094e0 con 0x558410cf7b00 2023-11-15T03:03:33.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.429+0000 7f13c1da6d00 1 -- 172.21.15.121:0/1578195463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558410c4fa20 con 0x558410cf7b00 2023-11-15T03:03:33.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.429+0000 7f13bceb2700 1 -- 172.21.15.121:0/1578195463 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8010ac0 con 0x558410cf7b00 2023-11-15T03:03:33.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.429+0000 7f13c1da6d00 1 -- 172.21.15.121:0/1578195463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558410c4f770 con 0x558410cf7b00 2023-11-15T03:03:33.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.429+0000 7f13bceb2700 1 -- 172.21.15.121:0/1578195463 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f13a8015430 con 0x558410cf7b00 2023-11-15T03:03:33.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.433+0000 7f13bceb2700 1 --2- 172.21.15.121:0/1578195463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f13a0058ff0 0x7f13a005b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.433+0000 7f13bceb2700 1 -- 172.21.15.121:0/1578195463 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f13a8073e80 con 0x558410cf7b00 2023-11-15T03:03:33.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.433+0000 7f13bf6b7700 1 --2- 172.21.15.121:0/1578195463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f13a0058ff0 0x7f13a005b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.433+0000 7f13c1da6d00 1 --2- 172.21.15.121:0/1578195463 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x558410cdf800 0x558410da2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.433+0000 7f13c1da6d00 1 -- 172.21.15.121:0/1578195463 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x558410d7b180 con 0x558410cdf800 2023-11-15T03:03:33.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.433+0000 7f13bfeb8700 1 --2- 172.21.15.121:0/1578195463 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x558410cdf800 0x558410da2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.433+0000 7f13bf6b7700 1 --2- 172.21.15.121:0/1578195463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f13a0058ff0 0x7f13a005b4b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f13b0009e80 tx=0x7f13b000c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.433+0000 7f13bfeb8700 1 --2- 172.21.15.121:0/1578195463 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x558410cdf800 0x558410da2150 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.433+0000 7f13bfeb8700 1 -- 172.21.15.121:0/1578195463 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(1 1.00000000.inode [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) v8 ==== 160+0+5 (crc 0 0 0) 0x7f13b8011770 con 0x558410cdf800 2023-11-15T03:03:33.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.433+0000 7f13c1da6d00 1 -- 172.21.15.121:0/1578195463 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x558410cdf800 msgr2=0x558410da2150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.433+0000 7f13c1da6d00 1 --2- 172.21.15.121:0/1578195463 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x558410cdf800 0x558410da2150 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.433+0000 7f13c1da6d00 1 -- 172.21.15.121:0/1578195463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f13a0058ff0 msgr2=0x7f13a005b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.433+0000 7f13c1da6d00 1 --2- 172.21.15.121:0/1578195463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f13a0058ff0 0x7f13a005b4b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f13b0009e80 tx=0x7f13b000c040).stop 2023-11-15T03:03:33.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.433+0000 7f13c1da6d00 1 -- 172.21.15.121:0/1578195463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558410cf7b00 msgr2=0x558410cf6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.433+0000 7f13c1da6d00 1 --2- 172.21.15.121:0/1578195463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558410cf7b00 0x558410cf6380 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7f13a8008af0 tx=0x7f13a800c040).stop 2023-11-15T03:03:33.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.433+0000 7f13bfeb8700 1 -- 172.21.15.121:0/1578195463 reap_dead start 2023-11-15T03:03:33.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.433+0000 7f13c1da6d00 1 -- 172.21.15.121:0/1578195463 shutdown_connections 2023-11-15T03:03:33.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.433+0000 7f13c1da6d00 1 -- 172.21.15.121:0/1578195463 >> 172.21.15.121:0/1578195463 conn(0x558410cfd080 msgr2=0x558410d7fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.433+0000 7f13c1da6d00 1 -- 172.21.15.121:0/1578195463 shutdown_connections 2023-11-15T03:03:33.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.437+0000 7f13c1da6d00 1 -- 172.21.15.121:0/1578195463 wait complete. 2023-11-15T03:03:33.454 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 200.00000000 2023-11-15T03:03:33.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.513+0000 7f1266131d00 1 Processor -- start 2023-11-15T03:03:33.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.513+0000 7f1266131d00 1 -- start start 2023-11-15T03:03:33.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.513+0000 7f1266131d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0f90fcde0 0x55b0f91ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.513+0000 7f1266131d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0f912e420 0x55b0f912a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.513+0000 7f1266131d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b0f91d29f0 0x55b0f91cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.513+0000 7f1266131d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f9075220 con 0x55b0f912e420 2023-11-15T03:03:33.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.513+0000 7f1266131d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f91ce5f0 con 0x55b0f90fcde0 2023-11-15T03:03:33.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.513+0000 7f1266131d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0f91cef70 con 0x55b0f91d29f0 2023-11-15T03:03:33.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.513+0000 7f1252010700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0f912e420 0x55b0f912a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.513+0000 7f125b012700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b0f91d29f0 0x55b0f91cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.513+0000 7f125b012700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b0f91d29f0 0x55b0f91cf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:52930/0 (socket says 172.21.15.121:52930) 2023-11-15T03:03:33.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.513+0000 7f125b012700 1 -- 172.21.15.121:0/948953578 learned_addr learned my addr 172.21.15.121:0/948953578 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.513+0000 7f125a811700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0f90fcde0 0x55b0f91ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.513+0000 7f125b012700 1 -- 172.21.15.121:0/948953578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0f90fcde0 msgr2=0x55b0f91ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.513+0000 7f125b012700 1 --2- 172.21.15.121:0/948953578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0f90fcde0 0x55b0f91ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.513+0000 7f125b012700 1 -- 172.21.15.121:0/948953578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0f912e420 msgr2=0x55b0f912a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.513+0000 7f125b012700 1 --2- 172.21.15.121:0/948953578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0f912e420 0x55b0f912a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.513+0000 7f125b012700 1 -- 172.21.15.121:0/948953578 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c005cf0 con 0x55b0f91d29f0 2023-11-15T03:03:33.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f125b012700 1 --2- 172.21.15.121:0/948953578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b0f91d29f0 0x55b0f91cf130 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f125400b7b0 tx=0x7f1254005d40).ready entity=mon.2 client_cookie=78185cd92e8fe6ce server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f125980f700 1 -- 172.21.15.121:0/948953578 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125400b240 con 0x55b0f91d29f0 2023-11-15T03:03:33.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f125980f700 1 -- 172.21.15.121:0/948953578 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125400f590 con 0x55b0f91d29f0 2023-11-15T03:03:33.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f125980f700 1 -- 172.21.15.121:0/948953578 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125400f910 con 0x55b0f91d29f0 2023-11-15T03:03:33.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1266131d00 1 -- 172.21.15.121:0/948953578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b0f91d29f0 msgr2=0x55b0f91cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1266131d00 1 --2- 172.21.15.121:0/948953578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b0f91d29f0 0x55b0f91cf130 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f125400b7b0 tx=0x7f1254005d40).stop 2023-11-15T03:03:33.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1266131d00 1 -- 172.21.15.121:0/948953578 shutdown_connections 2023-11-15T03:03:33.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1266131d00 1 --2- 172.21.15.121:0/948953578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b0f91d29f0 0x55b0f91cf130 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1266131d00 1 --2- 172.21.15.121:0/948953578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0f912e420 0x55b0f912a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1266131d00 1 --2- 172.21.15.121:0/948953578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0f90fcde0 0x55b0f91ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1266131d00 1 -- 172.21.15.121:0/948953578 >> 172.21.15.121:0/948953578 conn(0x55b0f90fd7d0 msgr2=0x55b0f9148a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1266131d00 1 -- 172.21.15.121:0/948953578 shutdown_connections 2023-11-15T03:03:33.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1266131d00 1 -- 172.21.15.121:0/948953578 wait complete. 2023-11-15T03:03:33.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1266131d00 1 Processor -- start 2023-11-15T03:03:33.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1266131d00 1 -- start start 2023-11-15T03:03:33.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1266131d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b0f90fcde0 0x55b0f91cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1266131d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0f912e420 0x55b0f91cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1266131d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0f91d29f0 0x55b0f91cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1266131d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f912a5c0 con 0x55b0f912e420 2023-11-15T03:03:33.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1266131d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f91d5300 con 0x55b0f91d29f0 2023-11-15T03:03:33.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1266131d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0f91d7b80 con 0x55b0f90fcde0 2023-11-15T03:03:33.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1252010700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0f912e420 0x55b0f91cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1252010700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0f912e420 0x55b0f91cc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50318/0 (socket says 172.21.15.121:50318) 2023-11-15T03:03:33.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.517+0000 7f1252010700 1 -- 172.21.15.121:0/3787264241 learned_addr learned my addr 172.21.15.121:0/3787264241 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.521+0000 7f125b012700 1 --2- 172.21.15.121:0/3787264241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0f91d29f0 0x55b0f91cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.521+0000 7f125a811700 1 --2- 172.21.15.121:0/3787264241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b0f90fcde0 0x55b0f91cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.521+0000 7f1252010700 1 -- 172.21.15.121:0/3787264241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b0f90fcde0 msgr2=0x55b0f91cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.521+0000 7f1252010700 1 --2- 172.21.15.121:0/3787264241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b0f90fcde0 0x55b0f91cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.521+0000 7f1252010700 1 -- 172.21.15.121:0/3787264241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0f91d29f0 msgr2=0x55b0f91cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.521+0000 7f1252010700 1 --2- 172.21.15.121:0/3787264241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0f91d29f0 0x55b0f91cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.521+0000 7f1252010700 1 -- 172.21.15.121:0/3787264241 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125400b430 con 0x55b0f912e420 2023-11-15T03:03:33.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.521+0000 7f125a811700 1 --2- 172.21.15.121:0/3787264241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b0f90fcde0 0x55b0f91cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:33.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.521+0000 7f1252010700 1 --2- 172.21.15.121:0/3787264241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0f912e420 0x55b0f91cc3d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f124400c390 tx=0x7f1244001730).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.521+0000 7f1253fff700 1 -- 172.21.15.121:0/3787264241 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244001b00 con 0x55b0f912e420 2023-11-15T03:03:33.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.521+0000 7f1253fff700 1 -- 172.21.15.121:0/3787264241 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1244005070 con 0x55b0f912e420 2023-11-15T03:03:33.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.521+0000 7f1266131d00 1 -- 172.21.15.121:0/3787264241 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b0f909e770 con 0x55b0f912e420 2023-11-15T03:03:33.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.521+0000 7f1253fff700 1 -- 172.21.15.121:0/3787264241 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124400f450 con 0x55b0f912e420 2023-11-15T03:03:33.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.521+0000 7f1266131d00 1 -- 172.21.15.121:0/3787264241 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0f906da70 con 0x55b0f912e420 2023-11-15T03:03:33.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.525+0000 7f1253fff700 1 -- 172.21.15.121:0/3787264241 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f124400f800 con 0x55b0f912e420 2023-11-15T03:03:33.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.525+0000 7f1253fff700 1 --2- 172.21.15.121:0/3787264241 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f123c058ff0 0x7f123c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.525+0000 7f1253fff700 1 -- 172.21.15.121:0/3787264241 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f1244073830 con 0x55b0f912e420 2023-11-15T03:03:33.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.525+0000 7f125a811700 1 --2- 172.21.15.121:0/3787264241 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f123c058ff0 0x7f123c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.525+0000 7f1266131d00 1 --2- 172.21.15.121:0/3787264241 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55b0f9142d00 0x55b0f91430e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.525+0000 7f1266131d00 1 -- 172.21.15.121:0/3787264241 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55b0f91cc000 con 0x55b0f9142d00 2023-11-15T03:03:33.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.525+0000 7f125b012700 1 --2- 172.21.15.121:0/3787264241 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55b0f9142d00 0x55b0f91430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.525+0000 7f125a811700 1 --2- 172.21.15.121:0/3787264241 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f123c058ff0 0x7f123c05b4b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f124c005a90 tx=0x7f124c016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.525+0000 7f125b012700 1 --2- 172.21.15.121:0/3787264241 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55b0f9142d00 0x55b0f91430e0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.525+0000 7f125b012700 1 -- 172.21.15.121:0/3787264241 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(1 200.00000000 [omap-get-keys out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7f1254019040 con 0x55b0f9142d00 2023-11-15T03:03:33.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.525+0000 7f1266131d00 1 -- 172.21.15.121:0/3787264241 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55b0f9142d00 msgr2=0x55b0f91430e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.525+0000 7f1266131d00 1 --2- 172.21.15.121:0/3787264241 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55b0f9142d00 0x55b0f91430e0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.525+0000 7f1266131d00 1 -- 172.21.15.121:0/3787264241 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f123c058ff0 msgr2=0x7f123c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.525+0000 7f1266131d00 1 --2- 172.21.15.121:0/3787264241 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f123c058ff0 0x7f123c05b4b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f124c005a90 tx=0x7f124c016000).stop 2023-11-15T03:03:33.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.525+0000 7f1266131d00 1 -- 172.21.15.121:0/3787264241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0f912e420 msgr2=0x55b0f91cc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.525+0000 7f1266131d00 1 --2- 172.21.15.121:0/3787264241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0f912e420 0x55b0f91cc3d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f124400c390 tx=0x7f1244001730).stop 2023-11-15T03:03:33.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.525+0000 7f125b012700 1 -- 172.21.15.121:0/3787264241 reap_dead start 2023-11-15T03:03:33.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.525+0000 7f1266131d00 1 -- 172.21.15.121:0/3787264241 shutdown_connections 2023-11-15T03:03:33.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.525+0000 7f1266131d00 1 -- 172.21.15.121:0/3787264241 >> 172.21.15.121:0/3787264241 conn(0x55b0f90fd7d0 msgr2=0x55b0f912ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.529+0000 7f1266131d00 1 -- 172.21.15.121:0/3787264241 shutdown_connections 2023-11-15T03:03:33.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.529+0000 7f1266131d00 1 -- 172.21.15.121:0/3787264241 wait complete. 2023-11-15T03:03:33.554 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 200.00000001 2023-11-15T03:03:33.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fb6530d00 1 Processor -- start 2023-11-15T03:03:33.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fb6530d00 1 -- start start 2023-11-15T03:03:33.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fb6530d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5a9f0cbd0 0x55e5a9fb3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fb6530d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e5a9fb7dc0 0x55e5a9fb95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fb6530d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5a9fbbc30 0x55e5a9ee24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fb6530d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5a9e8fcc0 con 0x55e5a9fb7dc0 2023-11-15T03:03:33.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fb6530d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5a9e5a220 con 0x55e5a9fbbc30 2023-11-15T03:03:33.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fb6530d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5a9fb75e0 con 0x55e5a9f0cbd0 2023-11-15T03:03:33.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fa237f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e5a9fb7dc0 0x55e5a9fb95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fa237f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e5a9fb7dc0 0x55e5a9fb95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50324/0 (socket says 172.21.15.121:50324) 2023-11-15T03:03:33.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fa237f700 1 -- 172.21.15.121:0/2472600314 learned_addr learned my addr 172.21.15.121:0/2472600314 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fa237f700 1 -- 172.21.15.121:0/2472600314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5a9f0cbd0 msgr2=0x55e5a9fb3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fa237f700 1 --2- 172.21.15.121:0/2472600314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5a9f0cbd0 0x55e5a9fb3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:03:33.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fa237f700 1 -- 172.21.15.121:0/2472600314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5a9fbbc30 msgr2=0x55e5a9ee24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fa237f700 1 --2- 172.21.15.121:0/2472600314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5a9fbbc30 0x55e5a9ee24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:03:33.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fa237f700 1 -- 172.21.15.121:0/2472600314 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f94005cf0 con 0x55e5a9fb7dc0 2023-11-15T03:03:33.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fa237f700 1 --2- 172.21.15.121:0/2472600314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e5a9fb7dc0 0x55e5a9fb95a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f3f9400f000 tx=0x7f3f94015000).ready entity=mon.0 client_cookie=70242619e9f40cca server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fa9b7e700 1 -- 172.21.15.121:0/2472600314 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f94007e60 con 0x55e5a9fb7dc0 2023-11-15T03:03:33.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fa9b7e700 1 -- 172.21.15.121:0/2472600314 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f94000bc0 con 0x55e5a9fb7dc0 2023-11-15T03:03:33.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fa9b7e700 1 -- 172.21.15.121:0/2472600314 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f94007af0 con 0x55e5a9fb7dc0 2023-11-15T03:03:33.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fb6530d00 1 -- 172.21.15.121:0/2472600314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e5a9fb7dc0 msgr2=0x55e5a9fb95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fb6530d00 1 --2- 172.21.15.121:0/2472600314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e5a9fb7dc0 0x55e5a9fb95a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f3f9400f000 tx=0x7f3f94015000).stop 2023-11-15T03:03:33.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fb6530d00 1 -- 172.21.15.121:0/2472600314 shutdown_connections 2023-11-15T03:03:33.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fb6530d00 1 --2- 172.21.15.121:0/2472600314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5a9fbbc30 0x55e5a9ee24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:03:33.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fb6530d00 1 --2- 172.21.15.121:0/2472600314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e5a9fb7dc0 0x55e5a9fb95a0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.609+0000 7f3fb6530d00 1 --2- 172.21.15.121:0/2472600314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5a9f0cbd0 0x55e5a9fb3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:03:33.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.613+0000 7f3fb6530d00 1 -- 172.21.15.121:0/2472600314 >> 172.21.15.121:0/2472600314 conn(0x55e5a9f29f80 msgr2=0x55e5a9fbc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.613+0000 7f3fb6530d00 1 -- 172.21.15.121:0/2472600314 shutdown_connections 2023-11-15T03:03:33.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.613+0000 7f3fb6530d00 1 -- 172.21.15.121:0/2472600314 wait complete. 2023-11-15T03:03:33.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.613+0000 7f3fb6530d00 1 Processor -- start 2023-11-15T03:03:33.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.613+0000 7f3fb6530d00 1 -- start start 2023-11-15T03:03:33.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.613+0000 7f3fb6530d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e5a9f0cbd0 0x55e5a9f0ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.613+0000 7f3fb6530d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5a9fb7dc0 0x55e5a9faf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.613+0000 7f3fb6530d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5a9fbbc30 0x55e5a9fcb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.613+0000 7f3fb6530d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5a9fcb430 con 0x55e5a9f0cbd0 2023-11-15T03:03:33.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.613+0000 7f3fb6530d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5a9fcb100 con 0x55e5a9fb7dc0 2023-11-15T03:03:33.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.613+0000 7f3fb6530d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5a9fb8240 con 0x55e5a9fbbc30 2023-11-15T03:03:33.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.613+0000 7f3faab80700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e5a9f0cbd0 0x55e5a9f0ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.613+0000 7f3faab80700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e5a9f0cbd0 0x55e5a9f0ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50328/0 (socket says 172.21.15.121:50328) 2023-11-15T03:03:33.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.613+0000 7f3faab80700 1 -- 172.21.15.121:0/3500984772 learned_addr learned my addr 172.21.15.121:0/3500984772 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.617+0000 7f3fab381700 1 --2- 172.21.15.121:0/3500984772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5a9fbbc30 0x55e5a9fcb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.617+0000 7f3fa237f700 1 --2- 172.21.15.121:0/3500984772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5a9fb7dc0 0x55e5a9faf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.617+0000 7f3faab80700 1 -- 172.21.15.121:0/3500984772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5a9fbbc30 msgr2=0x55e5a9fcb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.617+0000 7f3faab80700 1 --2- 172.21.15.121:0/3500984772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5a9fbbc30 0x55e5a9fcb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.617+0000 7f3faab80700 1 -- 172.21.15.121:0/3500984772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5a9fb7dc0 msgr2=0x55e5a9faf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.617+0000 7f3faab80700 1 --2- 172.21.15.121:0/3500984772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5a9fb7dc0 0x55e5a9faf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.617+0000 7f3faab80700 1 -- 172.21.15.121:0/3500984772 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5a9e83770 con 0x55e5a9f0cbd0 2023-11-15T03:03:33.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.617+0000 7f3faab80700 1 --2- 172.21.15.121:0/3500984772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e5a9f0cbd0 0x55e5a9f0ef60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7f3f9c00b410 tx=0x7f3f9c0069d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.617+0000 7f3fa3fff700 1 -- 172.21.15.121:0/3500984772 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c00bea0 con 0x55e5a9f0cbd0 2023-11-15T03:03:33.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.617+0000 7f3fa3fff700 1 -- 172.21.15.121:0/3500984772 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9c006170 con 0x55e5a9f0cbd0 2023-11-15T03:03:33.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.617+0000 7f3fa3fff700 1 -- 172.21.15.121:0/3500984772 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c003660 con 0x55e5a9f0cbd0 2023-11-15T03:03:33.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.617+0000 7f3fb6530d00 1 -- 172.21.15.121:0/3500984772 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e5a9e52a70 con 0x55e5a9f0cbd0 2023-11-15T03:03:33.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.617+0000 7f3fb6530d00 1 -- 172.21.15.121:0/3500984772 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5a9e5e010 con 0x55e5a9f0cbd0 2023-11-15T03:03:33.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.621+0000 7f3fa3fff700 1 -- 172.21.15.121:0/3500984772 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3f9c003a50 con 0x55e5a9f0cbd0 2023-11-15T03:03:33.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.621+0000 7f3fa3fff700 1 --2- 172.21.15.121:0/3500984772 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3f8c058ff0 0x7f3f8c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.621+0000 7f3fa3fff700 1 -- 172.21.15.121:0/3500984772 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f3f9c072d50 con 0x55e5a9f0cbd0 2023-11-15T03:03:33.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.621+0000 7f3fa237f700 1 --2- 172.21.15.121:0/3500984772 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3f8c058ff0 0x7f3f8c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.621+0000 7f3fb6530d00 1 --2- 172.21.15.121:0/3500984772 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e5a9ee2420 0x55e5a9fcc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.621+0000 7f3fb6530d00 1 -- 172.21.15.121:0/3500984772 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.34 13:2e2fa760:::200.00000001:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55e5a9fcbdb0 con 0x55e5a9ee2420 2023-11-15T03:03:33.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.621+0000 7f3fab381700 1 --2- 172.21.15.121:0/3500984772 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e5a9ee2420 0x55e5a9fcc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.621+0000 7f3fa237f700 1 --2- 172.21.15.121:0/3500984772 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3f8c058ff0 0x7f3f8c05b4b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f3f9401f000 tx=0x7f3f9401a000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.621+0000 7f3fab381700 1 --2- 172.21.15.121:0/3500984772 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e5a9ee2420 0x55e5a9fcc1a0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.625+0000 7f3fab381700 1 -- 172.21.15.121:0/3500984772 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 200.00000001 [omap-get-keys out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7f3fa4002960 con 0x55e5a9ee2420 2023-11-15T03:03:33.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.625+0000 7f3fb6530d00 1 -- 172.21.15.121:0/3500984772 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e5a9ee2420 msgr2=0x55e5a9fcc1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.625+0000 7f3fb6530d00 1 --2- 172.21.15.121:0/3500984772 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e5a9ee2420 0x55e5a9fcc1a0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.625+0000 7f3fb6530d00 1 -- 172.21.15.121:0/3500984772 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3f8c058ff0 msgr2=0x7f3f8c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.625+0000 7f3fb6530d00 1 --2- 172.21.15.121:0/3500984772 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3f8c058ff0 0x7f3f8c05b4b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f3f9401f000 tx=0x7f3f9401a000).stop 2023-11-15T03:03:33.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.625+0000 7f3fb6530d00 1 -- 172.21.15.121:0/3500984772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e5a9f0cbd0 msgr2=0x55e5a9f0ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.625+0000 7f3fb6530d00 1 --2- 172.21.15.121:0/3500984772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e5a9f0cbd0 0x55e5a9f0ef60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7f3f9c00b410 tx=0x7f3f9c0069d0).stop 2023-11-15T03:03:33.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.625+0000 7f3fab381700 1 -- 172.21.15.121:0/3500984772 reap_dead start 2023-11-15T03:03:33.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.625+0000 7f3fb6530d00 1 -- 172.21.15.121:0/3500984772 shutdown_connections 2023-11-15T03:03:33.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.625+0000 7f3fb6530d00 1 -- 172.21.15.121:0/3500984772 >> 172.21.15.121:0/3500984772 conn(0x55e5a9f29f80 msgr2=0x55e5a9ee0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.625+0000 7f3fb6530d00 1 -- 172.21.15.121:0/3500984772 shutdown_connections 2023-11-15T03:03:33.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.625+0000 7f3fb6530d00 1 -- 172.21.15.121:0/3500984772 wait complete. 2023-11-15T03:03:33.655 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_openfiles.0 2023-11-15T03:03:33.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.709+0000 7f28b7945d00 1 Processor -- start 2023-11-15T03:03:33.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.709+0000 7f28b7945d00 1 -- start start 2023-11-15T03:03:33.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28b7945d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562e45d59bb0 0x562e45d595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28b7945d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562e45da3b00 0x562e45e36ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28b7945d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e45e36650 0x562e45d8c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28b7945d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e45cd2220 con 0x562e45d59bb0 2023-11-15T03:03:33.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28b7945d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e45d8d860 con 0x562e45e36650 2023-11-15T03:03:33.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28b7945d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e45d8ec40 con 0x562e45da3b00 2023-11-15T03:03:33.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28abfaa700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562e45d59bb0 0x562e45d595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28abfaa700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562e45d59bb0 0x562e45d595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50348/0 (socket says 172.21.15.121:50348) 2023-11-15T03:03:33.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28abfaa700 1 -- 172.21.15.121:0/1747694786 learned_addr learned my addr 172.21.15.121:0/1747694786 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28a37a9700 1 --2- 172.21.15.121:0/1747694786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562e45da3b00 0x562e45e36ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28ac7ab700 1 --2- 172.21.15.121:0/1747694786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e45e36650 0x562e45d8c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28abfaa700 1 -- 172.21.15.121:0/1747694786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562e45da3b00 msgr2=0x562e45e36ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28abfaa700 1 --2- 172.21.15.121:0/1747694786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562e45da3b00 0x562e45e36ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28abfaa700 1 -- 172.21.15.121:0/1747694786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e45e36650 msgr2=0x562e45d8c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28abfaa700 1 --2- 172.21.15.121:0/1747694786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e45e36650 0x562e45d8c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28abfaa700 1 -- 172.21.15.121:0/1747694786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a4005cf0 con 0x562e45d59bb0 2023-11-15T03:03:33.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28abfaa700 1 --2- 172.21.15.121:0/1747694786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562e45d59bb0 0x562e45d595f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7f289c001760 tx=0x7f289c0069b0).ready entity=mon.0 client_cookie=869ec83e2a5ff18d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28aafa8700 1 -- 172.21.15.121:0/1747694786 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c006320 con 0x562e45d59bb0 2023-11-15T03:03:33.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28aafa8700 1 -- 172.21.15.121:0/1747694786 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289c004540 con 0x562e45d59bb0 2023-11-15T03:03:33.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28aafa8700 1 -- 172.21.15.121:0/1747694786 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c0048e0 con 0x562e45d59bb0 2023-11-15T03:03:33.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28b7945d00 1 -- 172.21.15.121:0/1747694786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562e45d59bb0 msgr2=0x562e45d595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28b7945d00 1 --2- 172.21.15.121:0/1747694786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562e45d59bb0 0x562e45d595f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7f289c001760 tx=0x7f289c0069b0).stop 2023-11-15T03:03:33.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28b7945d00 1 -- 172.21.15.121:0/1747694786 shutdown_connections 2023-11-15T03:03:33.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28b7945d00 1 --2- 172.21.15.121:0/1747694786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e45e36650 0x562e45d8c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28b7945d00 1 --2- 172.21.15.121:0/1747694786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562e45da3b00 0x562e45e36ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28b7945d00 1 --2- 172.21.15.121:0/1747694786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562e45d59bb0 0x562e45d595f0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28b7945d00 1 -- 172.21.15.121:0/1747694786 >> 172.21.15.121:0/1747694786 conn(0x562e45da9080 msgr2=0x562e45d9dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28b7945d00 1 -- 172.21.15.121:0/1747694786 shutdown_connections 2023-11-15T03:03:33.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.713+0000 7f28b7945d00 1 -- 172.21.15.121:0/1747694786 wait complete. 2023-11-15T03:03:33.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.717+0000 7f28b7945d00 1 Processor -- start 2023-11-15T03:03:33.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.717+0000 7f28b7945d00 1 -- start start 2023-11-15T03:03:33.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.717+0000 7f28b7945d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562e45d59bb0 0x562e45e25900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.717+0000 7f28b7945d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562e45da3b00 0x562e45e24dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.717+0000 7f28b7945d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e45e36650 0x562e45da2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.717+0000 7f28b7945d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e45e26310 con 0x562e45d59bb0 2023-11-15T03:03:33.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.717+0000 7f28b7945d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e45d8ba10 con 0x562e45e36650 2023-11-15T03:03:33.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.717+0000 7f28b7945d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e45e276b0 con 0x562e45da3b00 2023-11-15T03:03:33.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.717+0000 7f28abfaa700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562e45d59bb0 0x562e45e25900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.717+0000 7f28abfaa700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562e45d59bb0 0x562e45e25900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50356/0 (socket says 172.21.15.121:50356) 2023-11-15T03:03:33.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.717+0000 7f28abfaa700 1 -- 172.21.15.121:0/3403487553 learned_addr learned my addr 172.21.15.121:0/3403487553 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.717+0000 7f28a37a9700 1 --2- 172.21.15.121:0/3403487553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562e45da3b00 0x562e45e24dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.717+0000 7f28ac7ab700 1 --2- 172.21.15.121:0/3403487553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e45e36650 0x562e45da2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.717+0000 7f28abfaa700 1 -- 172.21.15.121:0/3403487553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562e45da3b00 msgr2=0x562e45e24dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.717+0000 7f28abfaa700 1 --2- 172.21.15.121:0/3403487553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562e45da3b00 0x562e45e24dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.717+0000 7f28abfaa700 1 -- 172.21.15.121:0/3403487553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e45e36650 msgr2=0x562e45da2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.721+0000 7f28abfaa700 1 --2- 172.21.15.121:0/3403487553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e45e36650 0x562e45da2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.721+0000 7f28abfaa700 1 -- 172.21.15.121:0/3403487553 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289c006c90 con 0x562e45d59bb0 2023-11-15T03:03:33.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.721+0000 7f28ac7ab700 1 --2- 172.21.15.121:0/3403487553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e45e36650 0x562e45da2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:33.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.721+0000 7f28abfaa700 1 --2- 172.21.15.121:0/3403487553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562e45d59bb0 0x562e45e25900 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7f289c001eb0 tx=0x7f289c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.721+0000 7f28a97a5700 1 -- 172.21.15.121:0/3403487553 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c0136a0 con 0x562e45d59bb0 2023-11-15T03:03:33.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.721+0000 7f28a97a5700 1 -- 172.21.15.121:0/3403487553 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289c013800 con 0x562e45d59bb0 2023-11-15T03:03:33.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.721+0000 7f28b7945d00 1 -- 172.21.15.121:0/3403487553 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562e45cfb770 con 0x562e45d59bb0 2023-11-15T03:03:33.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.721+0000 7f28a97a5700 1 -- 172.21.15.121:0/3403487553 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c013bc0 con 0x562e45d59bb0 2023-11-15T03:03:33.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.721+0000 7f28b7945d00 1 -- 172.21.15.121:0/3403487553 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e45ccaa70 con 0x562e45d59bb0 2023-11-15T03:03:33.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.721+0000 7f28a97a5700 1 -- 172.21.15.121:0/3403487553 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f289c01e3f0 con 0x562e45d59bb0 2023-11-15T03:03:33.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.725+0000 7f28a97a5700 1 --2- 172.21.15.121:0/3403487553 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f288c058ff0 0x7f288c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.725+0000 7f28a97a5700 1 -- 172.21.15.121:0/3403487553 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f289c074740 con 0x562e45d59bb0 2023-11-15T03:03:33.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.725+0000 7f28a37a9700 1 --2- 172.21.15.121:0/3403487553 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f288c058ff0 0x7f288c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.725+0000 7f28b7945d00 1 --2- 172.21.15.121:0/3403487553 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562e45d8d110 0x562e45e46810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.725+0000 7f28b7945d00 1 -- 172.21.15.121:0/3403487553 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 13.26 13:654134d2:::mds0_openfiles.0:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x562e45e252b0 con 0x562e45d8d110 2023-11-15T03:03:33.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.725+0000 7f28ac7ab700 1 --2- 172.21.15.121:0/3403487553 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562e45d8d110 0x562e45e46810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.725+0000 7f28a37a9700 1 --2- 172.21.15.121:0/3403487553 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f288c058ff0 0x7f288c05b4b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x562e45da8910 tx=0x7f289400c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.725+0000 7f28ac7ab700 1 --2- 172.21.15.121:0/3403487553 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562e45d8d110 0x562e45e46810 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.725+0000 7f28ac7ab700 1 -- 172.21.15.121:0/3403487553 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-get-keys out=5b] v0'0 uv2 ondisk = 0) v8 ==== 160+0+5 (crc 0 0 0) 0x7f28a4008a00 con 0x562e45d8d110 2023-11-15T03:03:33.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.725+0000 7f28b7945d00 1 -- 172.21.15.121:0/3403487553 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562e45d8d110 msgr2=0x562e45e46810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.725+0000 7f28b7945d00 1 --2- 172.21.15.121:0/3403487553 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562e45d8d110 0x562e45e46810 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.725+0000 7f28b7945d00 1 -- 172.21.15.121:0/3403487553 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f288c058ff0 msgr2=0x7f288c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.725+0000 7f28b7945d00 1 --2- 172.21.15.121:0/3403487553 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f288c058ff0 0x7f288c05b4b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x562e45da8910 tx=0x7f289400c040).stop 2023-11-15T03:03:33.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.725+0000 7f28b7945d00 1 -- 172.21.15.121:0/3403487553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562e45d59bb0 msgr2=0x562e45e25900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.725+0000 7f28b7945d00 1 --2- 172.21.15.121:0/3403487553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562e45d59bb0 0x562e45e25900 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7f289c001eb0 tx=0x7f289c014000).stop 2023-11-15T03:03:33.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.725+0000 7f28ac7ab700 1 -- 172.21.15.121:0/3403487553 reap_dead start 2023-11-15T03:03:33.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.729+0000 7f28b7945d00 1 -- 172.21.15.121:0/3403487553 shutdown_connections 2023-11-15T03:03:33.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.729+0000 7f28b7945d00 1 -- 172.21.15.121:0/3403487553 >> 172.21.15.121:0/3403487553 conn(0x562e45da9080 msgr2=0x562e45d8aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.729+0000 7f28b7945d00 1 -- 172.21.15.121:0/3403487553 shutdown_connections 2023-11-15T03:03:33.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.729+0000 7f28b7945d00 1 -- 172.21.15.121:0/3403487553 wait complete. 2023-11-15T03:03:33.751 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 500.00000000 2023-11-15T03:03:33.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.805+0000 7f0a09ef2d00 1 Processor -- start 2023-11-15T03:03:33.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.809+0000 7f0a09ef2d00 1 -- start start 2023-11-15T03:03:33.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.809+0000 7f0a09ef2d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08be2d440 0x55c08beb86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.809+0000 7f0a09ef2d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c08be47c60 0x55c08beb4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.809+0000 7f0a09ef2d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c08beba2e0 0x55c08bed5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.809+0000 7f0a09ef2d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08bd74220 con 0x55c08beba2e0 2023-11-15T03:03:33.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.809+0000 7f0a09ef2d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08beb8bd0 con 0x55c08be2d440 2023-11-15T03:03:33.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.809+0000 7f0a09ef2d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c08bed10d0 con 0x55c08be47c60 2023-11-15T03:03:33.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.809+0000 7f0a08023700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c08beba2e0 0x55c08bed5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.809+0000 7f0a08023700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c08beba2e0 0x55c08bed5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50372/0 (socket says 172.21.15.121:50372) 2023-11-15T03:03:33.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.809+0000 7f0a08023700 1 -- 172.21.15.121:0/2455144085 learned_addr learned my addr 172.21.15.121:0/2455144085 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.809+0000 7f09ff021700 1 --2- 172.21.15.121:0/2455144085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c08be47c60 0x55c08beb4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.809+0000 7f0a07822700 1 --2- 172.21.15.121:0/2455144085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08be2d440 0x55c08beb86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.809+0000 7f0a08023700 1 -- 172.21.15.121:0/2455144085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c08be47c60 msgr2=0x55c08beb4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.809+0000 7f0a08023700 1 --2- 172.21.15.121:0/2455144085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c08be47c60 0x55c08beb4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.809+0000 7f0a08023700 1 -- 172.21.15.121:0/2455144085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08be2d440 msgr2=0x55c08beb86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.809+0000 7f0a08023700 1 --2- 172.21.15.121:0/2455144085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08be2d440 0x55c08beb86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.809+0000 7f0a08023700 1 -- 172.21.15.121:0/2455144085 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a00005cf0 con 0x55c08beba2e0 2023-11-15T03:03:33.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.809+0000 7f0a08023700 1 --2- 172.21.15.121:0/2455144085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c08beba2e0 0x55c08bed5a70 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7f0a0000acf0 tx=0x7f0a00016000).ready entity=mon.0 client_cookie=e8d6e10e434af904 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.809+0000 7f0a06820700 1 -- 172.21.15.121:0/2455144085 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a000086a0 con 0x55c08beba2e0 2023-11-15T03:03:33.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.813+0000 7f0a06820700 1 -- 172.21.15.121:0/2455144085 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a00008970 con 0x55c08beba2e0 2023-11-15T03:03:33.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.813+0000 7f0a06820700 1 -- 172.21.15.121:0/2455144085 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00008b20 con 0x55c08beba2e0 2023-11-15T03:03:33.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.813+0000 7f0a09ef2d00 1 -- 172.21.15.121:0/2455144085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c08beba2e0 msgr2=0x55c08bed5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.813+0000 7f0a09ef2d00 1 --2- 172.21.15.121:0/2455144085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c08beba2e0 0x55c08bed5a70 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7f0a0000acf0 tx=0x7f0a00016000).stop 2023-11-15T03:03:33.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.813+0000 7f0a09ef2d00 1 -- 172.21.15.121:0/2455144085 shutdown_connections 2023-11-15T03:03:33.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.813+0000 7f0a09ef2d00 1 --2- 172.21.15.121:0/2455144085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c08beba2e0 0x55c08bed5a70 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.813+0000 7f0a09ef2d00 1 --2- 172.21.15.121:0/2455144085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c08be47c60 0x55c08beb4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.813+0000 7f0a09ef2d00 1 --2- 172.21.15.121:0/2455144085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08be2d440 0x55c08beb86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.813+0000 7f0a09ef2d00 1 -- 172.21.15.121:0/2455144085 >> 172.21.15.121:0/2455144085 conn(0x55c08bdfc7d0 msgr2=0x55c08bed2700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.813+0000 7f0a09ef2d00 1 -- 172.21.15.121:0/2455144085 shutdown_connections 2023-11-15T03:03:33.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.813+0000 7f0a09ef2d00 1 -- 172.21.15.121:0/2455144085 wait complete. 2023-11-15T03:03:33.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.813+0000 7f0a09ef2d00 1 Processor -- start 2023-11-15T03:03:33.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.813+0000 7f0a09ef2d00 1 -- start start 2023-11-15T03:03:33.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.813+0000 7f0a09ef2d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c08be2d440 0x55c08beb4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a09ef2d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c08be47c60 0x55c08beb7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a09ef2d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08beba2e0 0x55c08beb7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a09ef2d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08beb57e0 con 0x55c08be2d440 2023-11-15T03:03:33.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a09ef2d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c08beb5ac0 con 0x55c08beba2e0 2023-11-15T03:03:33.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a09ef2d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c08beb5da0 con 0x55c08be47c60 2023-11-15T03:03:33.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a07822700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c08be2d440 0x55c08beb4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a07822700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c08be2d440 0x55c08beb4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50376/0 (socket says 172.21.15.121:50376) 2023-11-15T03:03:33.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a07822700 1 -- 172.21.15.121:0/198465703 learned_addr learned my addr 172.21.15.121:0/198465703 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f09ff021700 1 --2- 172.21.15.121:0/198465703 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c08be47c60 0x55c08beb7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a08023700 1 --2- 172.21.15.121:0/198465703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08beba2e0 0x55c08beb7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a07822700 1 -- 172.21.15.121:0/198465703 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c08be47c60 msgr2=0x55c08beb7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a07822700 1 --2- 172.21.15.121:0/198465703 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c08be47c60 0x55c08beb7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a07822700 1 -- 172.21.15.121:0/198465703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08beba2e0 msgr2=0x55c08beb7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a07822700 1 --2- 172.21.15.121:0/198465703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c08beba2e0 0x55c08beb7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a07822700 1 -- 172.21.15.121:0/198465703 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a00017040 con 0x55c08be2d440 2023-11-15T03:03:33.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a07822700 1 --2- 172.21.15.121:0/198465703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c08be2d440 0x55c08beb4db0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f09f80109c0 tx=0x7f09f8007040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a0501d700 1 -- 172.21.15.121:0/198465703 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f800a4d0 con 0x55c08be2d440 2023-11-15T03:03:33.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a0501d700 1 -- 172.21.15.121:0/198465703 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f8002980 con 0x55c08be2d440 2023-11-15T03:03:33.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a09ef2d00 1 -- 172.21.15.121:0/198465703 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c08bd9d770 con 0x55c08be2d440 2023-11-15T03:03:33.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a0501d700 1 -- 172.21.15.121:0/198465703 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8002d00 con 0x55c08be2d440 2023-11-15T03:03:33.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.817+0000 7f0a09ef2d00 1 -- 172.21.15.121:0/198465703 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c08bd6ca70 con 0x55c08be2d440 2023-11-15T03:03:33.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.821+0000 7f0a0501d700 1 -- 172.21.15.121:0/198465703 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f09f8003040 con 0x55c08be2d440 2023-11-15T03:03:33.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.821+0000 7f0a0501d700 1 --2- 172.21.15.121:0/198465703 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f09e8059020 0x7f09e805b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.821+0000 7f0a0501d700 1 -- 172.21.15.121:0/198465703 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f09f80741a0 con 0x55c08be2d440 2023-11-15T03:03:33.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.821+0000 7f09ff021700 1 --2- 172.21.15.121:0/198465703 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f09e8059020 0x7f09e805b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.821+0000 7f0a09ef2d00 1 --2- 172.21.15.121:0/198465703 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c08beb52a0 0x55c08bef0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.821+0000 7f0a09ef2d00 1 -- 172.21.15.121:0/198465703 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55c08beb2320 con 0x55c08beb52a0 2023-11-15T03:03:33.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.821+0000 7f09ff021700 1 --2- 172.21.15.121:0/198465703 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f09e8059020 0x7f09e805b4e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x55c08beb9d70 tx=0x7f09f00016f0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.821+0000 7f0a08023700 1 --2- 172.21.15.121:0/198465703 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c08beb52a0 0x55c08bef0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.821+0000 7f0a08023700 1 --2- 172.21.15.121:0/198465703 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c08beb52a0 0x55c08bef0600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.825+0000 7f0a08023700 1 -- 172.21.15.121:0/198465703 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(1 500.00000000 [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7f0a00024040 con 0x55c08beb52a0 2023-11-15T03:03:33.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.825+0000 7f0a09ef2d00 1 -- 172.21.15.121:0/198465703 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c08beb52a0 msgr2=0x55c08bef0600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.825+0000 7f0a09ef2d00 1 --2- 172.21.15.121:0/198465703 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c08beb52a0 0x55c08bef0600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.825+0000 7f0a09ef2d00 1 -- 172.21.15.121:0/198465703 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f09e8059020 msgr2=0x7f09e805b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.825+0000 7f0a09ef2d00 1 --2- 172.21.15.121:0/198465703 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f09e8059020 0x7f09e805b4e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x55c08beb9d70 tx=0x7f09f00016f0).stop 2023-11-15T03:03:33.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.825+0000 7f0a09ef2d00 1 -- 172.21.15.121:0/198465703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c08be2d440 msgr2=0x55c08beb4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.825+0000 7f0a09ef2d00 1 --2- 172.21.15.121:0/198465703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c08be2d440 0x55c08beb4db0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f09f80109c0 tx=0x7f09f8007040).stop 2023-11-15T03:03:33.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.825+0000 7f0a08023700 1 -- 172.21.15.121:0/198465703 reap_dead start 2023-11-15T03:03:33.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.825+0000 7f0a09ef2d00 1 -- 172.21.15.121:0/198465703 shutdown_connections 2023-11-15T03:03:33.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.825+0000 7f0a09ef2d00 1 -- 172.21.15.121:0/198465703 >> 172.21.15.121:0/198465703 conn(0x55c08bdfc7d0 msgr2=0x55c08bdfb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.825+0000 7f0a09ef2d00 1 -- 172.21.15.121:0/198465703 shutdown_connections 2023-11-15T03:03:33.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.825+0000 7f0a09ef2d00 1 -- 172.21.15.121:0/198465703 wait complete. 2023-11-15T03:03:33.844 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds_snaptable 2023-11-15T03:03:33.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.901+0000 7fe917b24d00 1 Processor -- start 2023-11-15T03:03:33.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.901+0000 7fe917b24d00 1 -- start start 2023-11-15T03:03:33.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.901+0000 7fe917b24d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564deb8ff440 0x564deb985250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.901+0000 7fe917b24d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564deb919c60 0x564deb918530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.901+0000 7fe917b24d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564deb98c2e0 0x564deb9a5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.901+0000 7fe917b24d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564deb846220 con 0x564deb8ff440 2023-11-15T03:03:33.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.901+0000 7fe917b24d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564deb984d50 con 0x564deb98c2e0 2023-11-15T03:03:33.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.901+0000 7fe917b24d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564deb9a4720 con 0x564deb919c60 2023-11-15T03:03:33.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.901+0000 7fe915445700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564deb8ff440 0x564deb985250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.901+0000 7fe915445700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564deb8ff440 0x564deb985250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50394/0 (socket says 172.21.15.121:50394) 2023-11-15T03:03:33.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.901+0000 7fe915445700 1 -- 172.21.15.121:0/3819680650 learned_addr learned my addr 172.21.15.121:0/3819680650 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.901+0000 7fe915c46700 1 --2- 172.21.15.121:0/3819680650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564deb98c2e0 0x564deb9a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.901+0000 7fe914c44700 1 --2- 172.21.15.121:0/3819680650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564deb919c60 0x564deb918530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.901+0000 7fe915445700 1 -- 172.21.15.121:0/3819680650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564deb919c60 msgr2=0x564deb918530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.901+0000 7fe915445700 1 --2- 172.21.15.121:0/3819680650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564deb919c60 0x564deb918530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.901+0000 7fe915445700 1 -- 172.21.15.121:0/3819680650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564deb98c2e0 msgr2=0x564deb9a5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:33.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.901+0000 7fe915445700 1 --2- 172.21.15.121:0/3819680650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564deb98c2e0 0x564deb9a5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.901+0000 7fe915445700 1 -- 172.21.15.121:0/3819680650 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90800c3d0 con 0x564deb8ff440 2023-11-15T03:03:33.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.901+0000 7fe915c46700 1 --2- 172.21.15.121:0/3819680650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564deb98c2e0 0x564deb9a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:33.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.905+0000 7fe915445700 1 --2- 172.21.15.121:0/3819680650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564deb8ff440 0x564deb985250 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x564deb9170e0 tx=0x564deb917110).ready entity=mon.0 client_cookie=ebe0693a04badcda server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.905+0000 7fe9077fe700 1 -- 172.21.15.121:0/3819680650 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90801f070 con 0x564deb8ff440 2023-11-15T03:03:33.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.905+0000 7fe9077fe700 1 -- 172.21.15.121:0/3819680650 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe908000f40 con 0x564deb8ff440 2023-11-15T03:03:33.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.905+0000 7fe9077fe700 1 -- 172.21.15.121:0/3819680650 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90800a770 con 0x564deb8ff440 2023-11-15T03:03:33.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.905+0000 7fe917b24d00 1 -- 172.21.15.121:0/3819680650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564deb8ff440 msgr2=0x564deb985250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.905+0000 7fe917b24d00 1 --2- 172.21.15.121:0/3819680650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564deb8ff440 0x564deb985250 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x564deb9170e0 tx=0x564deb917110).stop 2023-11-15T03:03:33.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.905+0000 7fe917b24d00 1 -- 172.21.15.121:0/3819680650 shutdown_connections 2023-11-15T03:03:33.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.905+0000 7fe917b24d00 1 --2- 172.21.15.121:0/3819680650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564deb98c2e0 0x564deb9a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.905+0000 7fe917b24d00 1 --2- 172.21.15.121:0/3819680650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564deb919c60 0x564deb918530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.905+0000 7fe917b24d00 1 --2- 172.21.15.121:0/3819680650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564deb8ff440 0x564deb985250 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.905+0000 7fe917b24d00 1 -- 172.21.15.121:0/3819680650 >> 172.21.15.121:0/3819680650 conn(0x564deb8ce7d0 msgr2=0x564deb9a5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.905+0000 7fe917b24d00 1 -- 172.21.15.121:0/3819680650 shutdown_connections 2023-11-15T03:03:33.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.905+0000 7fe917b24d00 1 -- 172.21.15.121:0/3819680650 wait complete. 2023-11-15T03:03:33.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.905+0000 7fe917b24d00 1 Processor -- start 2023-11-15T03:03:33.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.905+0000 7fe917b24d00 1 -- start start 2023-11-15T03:03:33.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe917b24d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564deb8ff440 0x564deb9a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe917b24d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564deb919c60 0x564deb983930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe917b24d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564deb98c2e0 0x564deb9a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe917b24d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564deb9898d0 con 0x564deb8ff440 2023-11-15T03:03:33.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe917b24d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564deb9a51f0 con 0x564deb98c2e0 2023-11-15T03:03:33.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe917b24d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564deb989030 con 0x564deb919c60 2023-11-15T03:03:33.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe915445700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564deb8ff440 0x564deb9a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe915445700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564deb8ff440 0x564deb9a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50408/0 (socket says 172.21.15.121:50408) 2023-11-15T03:03:33.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe915445700 1 -- 172.21.15.121:0/1914061013 learned_addr learned my addr 172.21.15.121:0/1914061013 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:33.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe914c44700 1 --2- 172.21.15.121:0/1914061013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564deb919c60 0x564deb983930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe915c46700 1 --2- 172.21.15.121:0/1914061013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564deb98c2e0 0x564deb9a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe915445700 1 -- 172.21.15.121:0/1914061013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564deb919c60 msgr2=0x564deb983930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe915445700 1 --2- 172.21.15.121:0/1914061013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564deb919c60 0x564deb983930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe915445700 1 -- 172.21.15.121:0/1914061013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564deb98c2e0 msgr2=0x564deb9a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe915445700 1 --2- 172.21.15.121:0/1914061013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564deb98c2e0 0x564deb9a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe915445700 1 -- 172.21.15.121:0/1914061013 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564deb86fa20 con 0x564deb8ff440 2023-11-15T03:03:33.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe915445700 1 --2- 172.21.15.121:0/1914061013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564deb8ff440 0x564deb9a5680 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7fe908005fa0 tx=0x7fe90801f000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe905ffb700 1 -- 172.21.15.121:0/1914061013 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90800fea0 con 0x564deb8ff440 2023-11-15T03:03:33.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe905ffb700 1 -- 172.21.15.121:0/1914061013 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe908018420 con 0x564deb8ff440 2023-11-15T03:03:33.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe905ffb700 1 -- 172.21.15.121:0/1914061013 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9080187c0 con 0x564deb8ff440 2023-11-15T03:03:33.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe917b24d00 1 -- 172.21.15.121:0/1914061013 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564deb86f770 con 0x564deb8ff440 2023-11-15T03:03:33.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.909+0000 7fe917b24d00 1 -- 172.21.15.121:0/1914061013 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564deb83ea70 con 0x564deb8ff440 2023-11-15T03:03:33.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.913+0000 7fe905ffb700 1 -- 172.21.15.121:0/1914061013 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe908003030 con 0x564deb8ff440 2023-11-15T03:03:33.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.913+0000 7fe905ffb700 1 --2- 172.21.15.121:0/1914061013 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe8f8059020 0x7fe8f805b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.913+0000 7fe905ffb700 1 -- 172.21.15.121:0/1914061013 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fe9080756d0 con 0x564deb8ff440 2023-11-15T03:03:33.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.913+0000 7fe914c44700 1 --2- 172.21.15.121:0/1914061013 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe8f8059020 0x7fe8f805b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.913+0000 7fe917b24d00 1 --2- 172.21.15.121:0/1914061013 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x564deb9b6e40 0x564deb9b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:33.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.913+0000 7fe917b24d00 1 -- 172.21.15.121:0/1914061013 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:1 13.2d 13:b50e409b:::mds_snaptable:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x564deb9b7740 con 0x564deb9b6e40 2023-11-15T03:03:33.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.913+0000 7fe915c46700 1 --2- 172.21.15.121:0/1914061013 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x564deb9b6e40 0x564deb9b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:33.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.913+0000 7fe914c44700 1 --2- 172.21.15.121:0/1914061013 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe8f8059020 0x7fe8f805b4e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7fe90c001ed0 tx=0x7fe90c0017c0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.913+0000 7fe915c46700 1 --2- 172.21.15.121:0/1914061013 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x564deb9b6e40 0x564deb9b7250 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:33.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.917+0000 7fe915c46700 1 -- 172.21.15.121:0/1914061013 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(1 mds_snaptable [omap-get-keys out=5b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+5 (crc 0 0 0) 0x7fe9100098f0 con 0x564deb9b6e40 2023-11-15T03:03:33.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.917+0000 7fe917b24d00 1 -- 172.21.15.121:0/1914061013 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x564deb9b6e40 msgr2=0x564deb9b7250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.917+0000 7fe917b24d00 1 --2- 172.21.15.121:0/1914061013 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x564deb9b6e40 0x564deb9b7250 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:33.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.917+0000 7fe917b24d00 1 -- 172.21.15.121:0/1914061013 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe8f8059020 msgr2=0x7fe8f805b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.917+0000 7fe917b24d00 1 --2- 172.21.15.121:0/1914061013 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe8f8059020 0x7fe8f805b4e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7fe90c001ed0 tx=0x7fe90c0017c0).stop 2023-11-15T03:03:33.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.917+0000 7fe917b24d00 1 -- 172.21.15.121:0/1914061013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564deb8ff440 msgr2=0x564deb9a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:33.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.917+0000 7fe917b24d00 1 --2- 172.21.15.121:0/1914061013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564deb8ff440 0x564deb9a5680 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7fe908005fa0 tx=0x7fe90801f000).stop 2023-11-15T03:03:33.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.917+0000 7fe915c46700 1 -- 172.21.15.121:0/1914061013 reap_dead start 2023-11-15T03:03:33.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.917+0000 7fe917b24d00 1 -- 172.21.15.121:0/1914061013 shutdown_connections 2023-11-15T03:03:33.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.917+0000 7fe917b24d00 1 -- 172.21.15.121:0/1914061013 >> 172.21.15.121:0/1914061013 conn(0x564deb8ce7d0 msgr2=0x564deb918530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:33.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.917+0000 7fe917b24d00 1 -- 172.21.15.121:0/1914061013 shutdown_connections 2023-11-15T03:03:33.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.917+0000 7fe917b24d00 1 -- 172.21.15.121:0/1914061013 wait complete. 2023-11-15T03:03:33.929 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_inotable 2023-11-15T03:03:33.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.993+0000 7f17a032ad00 1 Processor -- start 2023-11-15T03:03:33.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.993+0000 7f17a032ad00 1 -- start start 2023-11-15T03:03:34.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.993+0000 7f17a032ad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556d9ff5b920 0x556da00229a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.993+0000 7f17a032ad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d9ffa32f0 0x556d9ff88310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.993+0000 7f17a032ad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556da0031a20 0x556d9ff88800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.993+0000 7f17a032ad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9ff05cc0 con 0x556da0031a20 2023-11-15T03:03:34.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.993+0000 7f17a032ad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9fed0220 con 0x556d9ffa32f0 2023-11-15T03:03:34.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.993+0000 7f17a032ad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d9ff89d80 con 0x556d9ff5b920 2023-11-15T03:03:34.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f179e446700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556da0031a20 0x556d9ff88800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f179e446700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556da0031a20 0x556d9ff88800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50416/0 (socket says 172.21.15.121:50416) 2023-11-15T03:03:34.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f179e446700 1 -- 172.21.15.121:0/1537984002 learned_addr learned my addr 172.21.15.121:0/1537984002 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f179dc45700 1 --2- 172.21.15.121:0/1537984002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556d9ff5b920 0x556da00229a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f1795444700 1 --2- 172.21.15.121:0/1537984002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d9ffa32f0 0x556d9ff88310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f179e446700 1 -- 172.21.15.121:0/1537984002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556d9ff5b920 msgr2=0x556da00229a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f179e446700 1 --2- 172.21.15.121:0/1537984002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556d9ff5b920 0x556da00229a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f179e446700 1 -- 172.21.15.121:0/1537984002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d9ffa32f0 msgr2=0x556d9ff88310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f179e446700 1 --2- 172.21.15.121:0/1537984002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d9ffa32f0 0x556d9ff88310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f179e446700 1 -- 172.21.15.121:0/1537984002 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179000a0e0 con 0x556da0031a20 2023-11-15T03:03:34.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f179e446700 1 --2- 172.21.15.121:0/1537984002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556da0031a20 0x556d9ff88800 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f1798002df0 tx=0x7f17980028c0).ready entity=mon.0 client_cookie=a0f1cab1c0a01e16 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f179cc43700 1 -- 172.21.15.121:0/1537984002 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179800bb90 con 0x556da0031a20 2023-11-15T03:03:34.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f179cc43700 1 -- 172.21.15.121:0/1537984002 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1798008070 con 0x556da0031a20 2023-11-15T03:03:34.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f179cc43700 1 -- 172.21.15.121:0/1537984002 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179800bcf0 con 0x556da0031a20 2023-11-15T03:03:34.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f17a032ad00 1 -- 172.21.15.121:0/1537984002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556da0031a20 msgr2=0x556d9ff88800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f17a032ad00 1 --2- 172.21.15.121:0/1537984002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556da0031a20 0x556d9ff88800 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f1798002df0 tx=0x7f17980028c0).stop 2023-11-15T03:03:34.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f17a032ad00 1 -- 172.21.15.121:0/1537984002 shutdown_connections 2023-11-15T03:03:34.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f17a032ad00 1 --2- 172.21.15.121:0/1537984002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556da0031a20 0x556d9ff88800 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f17a032ad00 1 --2- 172.21.15.121:0/1537984002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d9ffa32f0 0x556d9ff88310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f17a032ad00 1 --2- 172.21.15.121:0/1537984002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556d9ff5b920 0x556da00229a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f17a032ad00 1 -- 172.21.15.121:0/1537984002 >> 172.21.15.121:0/1537984002 conn(0x556d9ffa18f0 msgr2=0x556d9ffa0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f17a032ad00 1 -- 172.21.15.121:0/1537984002 shutdown_connections 2023-11-15T03:03:34.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:33.997+0000 7f17a032ad00 1 -- 172.21.15.121:0/1537984002 wait complete. 2023-11-15T03:03:34.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.001+0000 7f17a032ad00 1 Processor -- start 2023-11-15T03:03:34.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.001+0000 7f17a032ad00 1 -- start start 2023-11-15T03:03:34.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.001+0000 7f17a032ad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556d9ff5b920 0x556d9ff9ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.001+0000 7f17a032ad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d9ffa32f0 0x556da002e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.001+0000 7f17a032ad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556da0031a20 0x556da0021aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.001+0000 7f17a032ad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da002e950 con 0x556d9ffa32f0 2023-11-15T03:03:34.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.001+0000 7f17a032ad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9ff88310 con 0x556da0031a20 2023-11-15T03:03:34.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.001+0000 7f17a032ad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d9ff888d0 con 0x556d9ff5b920 2023-11-15T03:03:34.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.001+0000 7f1795444700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d9ffa32f0 0x556da002e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.001+0000 7f1795444700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d9ffa32f0 0x556da002e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50420/0 (socket says 172.21.15.121:50420) 2023-11-15T03:03:34.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.001+0000 7f1795444700 1 -- 172.21.15.121:0/1254109019 learned_addr learned my addr 172.21.15.121:0/1254109019 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.001+0000 7f179dc45700 1 --2- 172.21.15.121:0/1254109019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556d9ff5b920 0x556d9ff9ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.001+0000 7f179e446700 1 --2- 172.21.15.121:0/1254109019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556da0031a20 0x556da0021aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.001+0000 7f1795444700 1 -- 172.21.15.121:0/1254109019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556d9ff5b920 msgr2=0x556d9ff9ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.001+0000 7f1795444700 1 --2- 172.21.15.121:0/1254109019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556d9ff5b920 0x556d9ff9ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.001+0000 7f1795444700 1 -- 172.21.15.121:0/1254109019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556da0031a20 msgr2=0x556da0021aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.001+0000 7f1795444700 1 --2- 172.21.15.121:0/1254109019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556da0031a20 0x556da0021aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.001+0000 7f1795444700 1 -- 172.21.15.121:0/1254109019 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179800a040 con 0x556d9ffa32f0 2023-11-15T03:03:34.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.001+0000 7f1795444700 1 --2- 172.21.15.121:0/1254109019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d9ffa32f0 0x556da002e460 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x556d9ff5d2a0 tx=0x7f17880016d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.005+0000 7f1796ffd700 1 -- 172.21.15.121:0/1254109019 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178800cec0 con 0x556d9ffa32f0 2023-11-15T03:03:34.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.005+0000 7f1796ffd700 1 -- 172.21.15.121:0/1254109019 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1788001a40 con 0x556d9ffa32f0 2023-11-15T03:03:34.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.005+0000 7f17a032ad00 1 -- 172.21.15.121:0/1254109019 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556d9fec8a70 con 0x556d9ffa32f0 2023-11-15T03:03:34.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.005+0000 7f1796ffd700 1 -- 172.21.15.121:0/1254109019 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178800f660 con 0x556d9ffa32f0 2023-11-15T03:03:34.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.005+0000 7f17a032ad00 1 -- 172.21.15.121:0/1254109019 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d9fed4010 con 0x556d9ffa32f0 2023-11-15T03:03:34.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.005+0000 7f1796ffd700 1 -- 172.21.15.121:0/1254109019 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f178800fa10 con 0x556d9ffa32f0 2023-11-15T03:03:34.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.009+0000 7f1796ffd700 1 --2- 172.21.15.121:0/1254109019 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1780059020 0x7f178005b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.009+0000 7f1796ffd700 1 -- 172.21.15.121:0/1254109019 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f1788074050 con 0x556d9ffa32f0 2023-11-15T03:03:34.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.009+0000 7f179dc45700 1 --2- 172.21.15.121:0/1254109019 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1780059020 0x7f178005b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.009+0000 7f17a032ad00 1 --2- 172.21.15.121:0/1254109019 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x556da0021f90 0x556da002cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.009+0000 7f17a032ad00 1 -- 172.21.15.121:0/1254109019 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x556da002d430 con 0x556da0021f90 2023-11-15T03:03:34.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.009+0000 7f179e446700 1 --2- 172.21.15.121:0/1254109019 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x556da0021f90 0x556da002cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.009+0000 7f179dc45700 1 --2- 172.21.15.121:0/1254109019 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1780059020 0x7f178005b4e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f179000a520 tx=0x7f1790015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.009+0000 7f179e446700 1 --2- 172.21.15.121:0/1254109019 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x556da0021f90 0x556da002cf40 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.009+0000 7f179e446700 1 -- 172.21.15.121:0/1254109019 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 mds0_inotable [omap-get-keys out=5b] v0'0 uv2 ondisk = 0) v8 ==== 157+0+5 (crc 0 0 0) 0x7f179800bb90 con 0x556da0021f90 2023-11-15T03:03:34.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.009+0000 7f17a032ad00 1 -- 172.21.15.121:0/1254109019 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x556da0021f90 msgr2=0x556da002cf40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.009+0000 7f17a032ad00 1 --2- 172.21.15.121:0/1254109019 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x556da0021f90 0x556da002cf40 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.013+0000 7f17a032ad00 1 -- 172.21.15.121:0/1254109019 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1780059020 msgr2=0x7f178005b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.013+0000 7f17a032ad00 1 --2- 172.21.15.121:0/1254109019 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1780059020 0x7f178005b4e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f179000a520 tx=0x7f1790015000).stop 2023-11-15T03:03:34.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.013+0000 7f17a032ad00 1 -- 172.21.15.121:0/1254109019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d9ffa32f0 msgr2=0x556da002e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.013+0000 7f17a032ad00 1 --2- 172.21.15.121:0/1254109019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d9ffa32f0 0x556da002e460 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x556d9ff5d2a0 tx=0x7f17880016d0).stop 2023-11-15T03:03:34.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.013+0000 7f179e446700 1 -- 172.21.15.121:0/1254109019 reap_dead start 2023-11-15T03:03:34.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.013+0000 7f17a032ad00 1 -- 172.21.15.121:0/1254109019 shutdown_connections 2023-11-15T03:03:34.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.013+0000 7f17a032ad00 1 -- 172.21.15.121:0/1254109019 >> 172.21.15.121:0/1254109019 conn(0x556d9ffa18f0 msgr2=0x556d9ff8a950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.013+0000 7f17a032ad00 1 -- 172.21.15.121:0/1254109019 shutdown_connections 2023-11-15T03:03:34.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.013+0000 7f17a032ad00 1 -- 172.21.15.121:0/1254109019 wait complete. 2023-11-15T03:03:34.029 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 100.00000000 2023-11-15T03:03:34.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.081+0000 7fca7dd3fd00 1 Processor -- start 2023-11-15T03:03:34.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.081+0000 7fca7dd3fd00 1 -- start start 2023-11-15T03:03:34.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.085+0000 7fca7dd3fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7dc2b0dd0 0x55a7dc2e0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.085+0000 7fca7dd3fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7dc2fab00 0x55a7dc2e5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.085+0000 7fca7dd3fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7dc38d650 0x55a7dc2e1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.085+0000 7fca7dd3fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dc25ecc0 con 0x55a7dc2fab00 2023-11-15T03:03:34.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.085+0000 7fca7dd3fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dc229220 con 0x55a7dc38d650 2023-11-15T03:03:34.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.085+0000 7fca7dd3fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7dc383b00 con 0x55a7dc2b0dd0 2023-11-15T03:03:34.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.085+0000 7fca72e44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7dc2fab00 0x55a7dc2e5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.085+0000 7fca72e44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7dc2fab00 0x55a7dc2e5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50436/0 (socket says 172.21.15.121:50436) 2023-11-15T03:03:34.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.085+0000 7fca72e44700 1 -- 172.21.15.121:0/206510432 learned_addr learned my addr 172.21.15.121:0/206510432 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.085+0000 7fca7be46700 1 --2- 172.21.15.121:0/206510432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7dc38d650 0x55a7dc2e1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.085+0000 7fca7b645700 1 --2- 172.21.15.121:0/206510432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7dc2b0dd0 0x55a7dc2e0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.085+0000 7fca72e44700 1 -- 172.21.15.121:0/206510432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7dc2b0dd0 msgr2=0x55a7dc2e0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.085+0000 7fca72e44700 1 --2- 172.21.15.121:0/206510432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7dc2b0dd0 0x55a7dc2e0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.085+0000 7fca72e44700 1 -- 172.21.15.121:0/206510432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7dc38d650 msgr2=0x55a7dc2e1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.085+0000 7fca72e44700 1 --2- 172.21.15.121:0/206510432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7dc38d650 0x55a7dc2e1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.085+0000 7fca72e44700 1 -- 172.21.15.121:0/206510432 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7400a0e0 con 0x55a7dc2fab00 2023-11-15T03:03:34.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.085+0000 7fca7b645700 1 --2- 172.21.15.121:0/206510432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7dc2b0dd0 0x55a7dc2e0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:34.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.085+0000 7fca72e44700 1 --2- 172.21.15.121:0/206510432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7dc2fab00 0x55a7dc2e5930 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7fca64001d40 tx=0x7fca64001810).ready entity=mon.0 client_cookie=49dc770f2aa869b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.085+0000 7fca7a643700 1 -- 172.21.15.121:0/206510432 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca64001ea0 con 0x55a7dc2fab00 2023-11-15T03:03:34.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.089+0000 7fca7a643700 1 -- 172.21.15.121:0/206510432 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca64008070 con 0x55a7dc2fab00 2023-11-15T03:03:34.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.089+0000 7fca7a643700 1 -- 172.21.15.121:0/206510432 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6400ba80 con 0x55a7dc2fab00 2023-11-15T03:03:34.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.089+0000 7fca7dd3fd00 1 -- 172.21.15.121:0/206510432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7dc2fab00 msgr2=0x55a7dc2e5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.089+0000 7fca7dd3fd00 1 --2- 172.21.15.121:0/206510432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7dc2fab00 0x55a7dc2e5930 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7fca64001d40 tx=0x7fca64001810).stop 2023-11-15T03:03:34.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.089+0000 7fca7dd3fd00 1 -- 172.21.15.121:0/206510432 shutdown_connections 2023-11-15T03:03:34.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.089+0000 7fca7dd3fd00 1 --2- 172.21.15.121:0/206510432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7dc38d650 0x55a7dc2e1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.089+0000 7fca7dd3fd00 1 --2- 172.21.15.121:0/206510432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7dc2fab00 0x55a7dc2e5930 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.089+0000 7fca7dd3fd00 1 --2- 172.21.15.121:0/206510432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7dc2b0dd0 0x55a7dc2e0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.089+0000 7fca7dd3fd00 1 -- 172.21.15.121:0/206510432 >> 172.21.15.121:0/206510432 conn(0x55a7dc300080 msgr2=0x55a7dc38bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.089+0000 7fca7dd3fd00 1 -- 172.21.15.121:0/206510432 shutdown_connections 2023-11-15T03:03:34.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.089+0000 7fca7dd3fd00 1 -- 172.21.15.121:0/206510432 wait complete. 2023-11-15T03:03:34.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.089+0000 7fca7dd3fd00 1 Processor -- start 2023-11-15T03:03:34.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.089+0000 7fca7dd3fd00 1 -- start start 2023-11-15T03:03:34.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca7dd3fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7dc2b0dd0 0x55a7dc382050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca7dd3fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7dc2fab00 0x55a7dc2f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca7dd3fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7dc38d650 0x55a7dc2f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca7dd3fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dc2f5ed0 con 0x55a7dc2fab00 2023-11-15T03:03:34.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca7dd3fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dc38a7b0 con 0x55a7dc38d650 2023-11-15T03:03:34.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca7dd3fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7dc38b330 con 0x55a7dc2b0dd0 2023-11-15T03:03:34.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca72e44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7dc2fab00 0x55a7dc2f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca7b645700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7dc2b0dd0 0x55a7dc382050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca72e44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7dc2fab00 0x55a7dc2f54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50452/0 (socket says 172.21.15.121:50452) 2023-11-15T03:03:34.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca72e44700 1 -- 172.21.15.121:0/1615745605 learned_addr learned my addr 172.21.15.121:0/1615745605 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca7be46700 1 --2- 172.21.15.121:0/1615745605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7dc38d650 0x55a7dc2f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca72e44700 1 -- 172.21.15.121:0/1615745605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7dc2b0dd0 msgr2=0x55a7dc382050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca72e44700 1 --2- 172.21.15.121:0/1615745605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7dc2b0dd0 0x55a7dc382050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca72e44700 1 -- 172.21.15.121:0/1615745605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7dc38d650 msgr2=0x55a7dc2f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca72e44700 1 --2- 172.21.15.121:0/1615745605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7dc38d650 0x55a7dc2f59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca72e44700 1 -- 172.21.15.121:0/1615745605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6400a040 con 0x55a7dc2fab00 2023-11-15T03:03:34.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca72e44700 1 --2- 172.21.15.121:0/1615745605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7dc2fab00 0x55a7dc2f54f0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x55a7dc38a420 tx=0x55a7dc2e3930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca78e40700 1 -- 172.21.15.121:0/1615745605 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca64001ea0 con 0x55a7dc2fab00 2023-11-15T03:03:34.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca78e40700 1 -- 172.21.15.121:0/1615745605 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6400ba80 con 0x55a7dc2fab00 2023-11-15T03:03:34.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca78e40700 1 -- 172.21.15.121:0/1615745605 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6401b610 con 0x55a7dc2fab00 2023-11-15T03:03:34.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca7dd3fd00 1 -- 172.21.15.121:0/1615745605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a7dc221a70 con 0x55a7dc2fab00 2023-11-15T03:03:34.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.093+0000 7fca7dd3fd00 1 -- 172.21.15.121:0/1615745605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7dc22d010 con 0x55a7dc2fab00 2023-11-15T03:03:34.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.097+0000 7fca78e40700 1 -- 172.21.15.121:0/1615745605 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fca64016070 con 0x55a7dc2fab00 2023-11-15T03:03:34.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.097+0000 7fca78e40700 1 --2- 172.21.15.121:0/1615745605 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fca5c058ff0 0x7fca5c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.101+0000 7fca78e40700 1 -- 172.21.15.121:0/1615745605 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fca64073f40 con 0x55a7dc2fab00 2023-11-15T03:03:34.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.101+0000 7fca7b645700 1 --2- 172.21.15.121:0/1615745605 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fca5c058ff0 0x7fca5c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.101+0000 7fca7dd3fd00 1 --2- 172.21.15.121:0/1615745605 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a7dc38bc90 0x55a7dc38fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.101+0000 7fca7dd3fd00 1 -- 172.21.15.121:0/1615745605 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 13.33 13:cd5a64a3:::100.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55a7dc38b7c0 con 0x55a7dc38bc90 2023-11-15T03:03:34.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.101+0000 7fca7b645700 1 --2- 172.21.15.121:0/1615745605 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fca5c058ff0 0x7fca5c05b4b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7fca6c000c00 tx=0x7fca6c001920).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.101+0000 7fca7be46700 1 --2- 172.21.15.121:0/1615745605 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a7dc38bc90 0x55a7dc38fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.101+0000 7fca7be46700 1 --2- 172.21.15.121:0/1615745605 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a7dc38bc90 0x55a7dc38fd30 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.101+0000 7fca7be46700 1 -- 172.21.15.121:0/1615745605 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 100.00000000 [omap-get-keys out=155b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+155 (crc 0 0 0) 0x7fca7400d7d0 con 0x55a7dc38bc90 2023-11-15T03:03:34.117 INFO:teuthology.orchestra.run.smithi121.stdout:stray0_head 2023-11-15T03:03:34.117 INFO:teuthology.orchestra.run.smithi121.stdout:stray1_head 2023-11-15T03:03:34.117 INFO:teuthology.orchestra.run.smithi121.stdout:stray2_head 2023-11-15T03:03:34.118 INFO:teuthology.orchestra.run.smithi121.stdout:stray3_head 2023-11-15T03:03:34.118 INFO:teuthology.orchestra.run.smithi121.stdout:stray4_head 2023-11-15T03:03:34.118 INFO:teuthology.orchestra.run.smithi121.stdout:stray5_head 2023-11-15T03:03:34.118 INFO:teuthology.orchestra.run.smithi121.stdout:stray6_head 2023-11-15T03:03:34.118 INFO:teuthology.orchestra.run.smithi121.stdout:stray7_head 2023-11-15T03:03:34.119 INFO:teuthology.orchestra.run.smithi121.stdout:stray8_head 2023-11-15T03:03:34.119 INFO:teuthology.orchestra.run.smithi121.stdout:stray9_head 2023-11-15T03:03:34.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.101+0000 7fca7dd3fd00 1 -- 172.21.15.121:0/1615745605 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a7dc38bc90 msgr2=0x55a7dc38fd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.101+0000 7fca7dd3fd00 1 --2- 172.21.15.121:0/1615745605 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a7dc38bc90 0x55a7dc38fd30 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.101+0000 7fca7dd3fd00 1 -- 172.21.15.121:0/1615745605 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fca5c058ff0 msgr2=0x7fca5c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.101+0000 7fca7dd3fd00 1 --2- 172.21.15.121:0/1615745605 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fca5c058ff0 0x7fca5c05b4b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7fca6c000c00 tx=0x7fca6c001920).stop 2023-11-15T03:03:34.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.101+0000 7fca7dd3fd00 1 -- 172.21.15.121:0/1615745605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7dc2fab00 msgr2=0x55a7dc2f54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.101+0000 7fca7dd3fd00 1 --2- 172.21.15.121:0/1615745605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7dc2fab00 0x55a7dc2f54f0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x55a7dc38a420 tx=0x55a7dc2e3930).stop 2023-11-15T03:03:34.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.101+0000 7fca7be46700 1 -- 172.21.15.121:0/1615745605 reap_dead start 2023-11-15T03:03:34.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.101+0000 7fca7dd3fd00 1 -- 172.21.15.121:0/1615745605 shutdown_connections 2023-11-15T03:03:34.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.101+0000 7fca7dd3fd00 1 -- 172.21.15.121:0/1615745605 >> 172.21.15.121:0/1615745605 conn(0x55a7dc300080 msgr2=0x55a7dc2dffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.101+0000 7fca7dd3fd00 1 -- 172.21.15.121:0/1615745605 shutdown_connections 2023-11-15T03:03:34.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.101+0000 7fca7dd3fd00 1 -- 172.21.15.121:0/1615745605 wait complete. 2023-11-15T03:03:34.123 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_sessionmap 2023-11-15T03:03:34.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.177+0000 7f17f42d8d00 1 Processor -- start 2023-11-15T03:03:34.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17f42d8d00 1 -- start start 2023-11-15T03:03:34.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17f42d8d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537c5add0 0x55a537d359a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17f42d8d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a537ca4b00 0x55a537d3fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17f42d8d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a537d37650 0x55a537d40350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17f42d8d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a537c08cc0 con 0x55a537ca4b00 2023-11-15T03:03:34.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17f42d8d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a537bd3220 con 0x55a537c5add0 2023-11-15T03:03:34.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17f42d8d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a537d2d460 con 0x55a537d37650 2023-11-15T03:03:34.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17ebfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a537ca4b00 0x55a537d3fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17ebfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a537ca4b00 0x55a537d3fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50454/0 (socket says 172.21.15.121:50454) 2023-11-15T03:03:34.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17ebfff700 1 -- 172.21.15.121:0/3484890637 learned_addr learned my addr 172.21.15.121:0/3484890637 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17f2311700 1 --2- 172.21.15.121:0/3484890637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a537d37650 0x55a537d40350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17f1b10700 1 --2- 172.21.15.121:0/3484890637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537c5add0 0x55a537d359a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17ebfff700 1 -- 172.21.15.121:0/3484890637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a537d37650 msgr2=0x55a537d40350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17ebfff700 1 --2- 172.21.15.121:0/3484890637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a537d37650 0x55a537d40350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17ebfff700 1 -- 172.21.15.121:0/3484890637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537c5add0 msgr2=0x55a537d359a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17ebfff700 1 --2- 172.21.15.121:0/3484890637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537c5add0 0x55a537d359a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17ebfff700 1 -- 172.21.15.121:0/3484890637 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ec00a0e0 con 0x55a537ca4b00 2023-11-15T03:03:34.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17ebfff700 1 --2- 172.21.15.121:0/3484890637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a537ca4b00 0x55a537d3fe60 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7f17dc002e10 tx=0x7f17dc0028b0).ready entity=mon.0 client_cookie=4e7e18c2371fc530 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17f0b0e700 1 -- 172.21.15.121:0/3484890637 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc00bbd0 con 0x55a537ca4b00 2023-11-15T03:03:34.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17f0b0e700 1 -- 172.21.15.121:0/3484890637 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc008070 con 0x55a537ca4b00 2023-11-15T03:03:34.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17f0b0e700 1 -- 172.21.15.121:0/3484890637 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc00bd50 con 0x55a537ca4b00 2023-11-15T03:03:34.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17f42d8d00 1 -- 172.21.15.121:0/3484890637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a537ca4b00 msgr2=0x55a537d3fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17f42d8d00 1 --2- 172.21.15.121:0/3484890637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a537ca4b00 0x55a537d3fe60 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7f17dc002e10 tx=0x7f17dc0028b0).stop 2023-11-15T03:03:34.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17f42d8d00 1 -- 172.21.15.121:0/3484890637 shutdown_connections 2023-11-15T03:03:34.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17f42d8d00 1 --2- 172.21.15.121:0/3484890637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a537d37650 0x55a537d40350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17f42d8d00 1 --2- 172.21.15.121:0/3484890637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a537ca4b00 0x55a537d3fe60 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17f42d8d00 1 --2- 172.21.15.121:0/3484890637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537c5add0 0x55a537d359a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.181+0000 7f17f42d8d00 1 -- 172.21.15.121:0/3484890637 >> 172.21.15.121:0/3484890637 conn(0x55a537caa080 msgr2=0x55a537ca6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.185+0000 7f17f42d8d00 1 -- 172.21.15.121:0/3484890637 shutdown_connections 2023-11-15T03:03:34.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.185+0000 7f17f42d8d00 1 -- 172.21.15.121:0/3484890637 wait complete. 2023-11-15T03:03:34.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.185+0000 7f17f42d8d00 1 Processor -- start 2023-11-15T03:03:34.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.185+0000 7f17f42d8d00 1 -- start start 2023-11-15T03:03:34.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.185+0000 7f17f42d8d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537c5add0 0x55a537d37ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.185+0000 7f17f42d8d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a537ca4b00 0x55a537d359a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.185+0000 7f17f42d8d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a537d37650 0x55a537d35e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.185+0000 7f17f42d8d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a537d2c640 con 0x55a537ca4b00 2023-11-15T03:03:34.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.185+0000 7f17f42d8d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a537d36380 con 0x55a537c5add0 2023-11-15T03:03:34.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.185+0000 7f17f42d8d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a537c8bd80 con 0x55a537d37650 2023-11-15T03:03:34.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.185+0000 7f17ebfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a537ca4b00 0x55a537d359a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.185+0000 7f17ebfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a537ca4b00 0x55a537d359a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50468/0 (socket says 172.21.15.121:50468) 2023-11-15T03:03:34.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.185+0000 7f17ebfff700 1 -- 172.21.15.121:0/2830434161 learned_addr learned my addr 172.21.15.121:0/2830434161 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.185+0000 7f17f1b10700 1 --2- 172.21.15.121:0/2830434161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537c5add0 0x55a537d37ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.185+0000 7f17f2311700 1 --2- 172.21.15.121:0/2830434161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a537d37650 0x55a537d35e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.189+0000 7f17ebfff700 1 -- 172.21.15.121:0/2830434161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a537d37650 msgr2=0x55a537d35e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.189+0000 7f17ebfff700 1 --2- 172.21.15.121:0/2830434161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a537d37650 0x55a537d35e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.189+0000 7f17ebfff700 1 -- 172.21.15.121:0/2830434161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537c5add0 msgr2=0x55a537d37ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.189+0000 7f17ebfff700 1 --2- 172.21.15.121:0/2830434161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537c5add0 0x55a537d37ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.189+0000 7f17ebfff700 1 -- 172.21.15.121:0/2830434161 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17dc00a040 con 0x55a537ca4b00 2023-11-15T03:03:34.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.189+0000 7f17ebfff700 1 --2- 172.21.15.121:0/2830434161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a537ca4b00 0x55a537d359a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x55a537ca1e90 tx=0x7f17dc0028b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.189+0000 7f17ea7fc700 1 -- 172.21.15.121:0/2830434161 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc011e20 con 0x55a537ca4b00 2023-11-15T03:03:34.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.189+0000 7f17ea7fc700 1 -- 172.21.15.121:0/2830434161 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc01b3f0 con 0x55a537ca4b00 2023-11-15T03:03:34.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.189+0000 7f17ea7fc700 1 -- 172.21.15.121:0/2830434161 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc01b770 con 0x55a537ca4b00 2023-11-15T03:03:34.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.189+0000 7f17f42d8d00 1 -- 172.21.15.121:0/2830434161 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a537bcba70 con 0x55a537ca4b00 2023-11-15T03:03:34.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.189+0000 7f17f42d8d00 1 -- 172.21.15.121:0/2830434161 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a537bd7010 con 0x55a537ca4b00 2023-11-15T03:03:34.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.193+0000 7f17ea7fc700 1 -- 172.21.15.121:0/2830434161 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f17dc017070 con 0x55a537ca4b00 2023-11-15T03:03:34.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.193+0000 7f17ea7fc700 1 --2- 172.21.15.121:0/2830434161 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f17d4058ff0 0x7f17d405b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.193+0000 7f17ea7fc700 1 -- 172.21.15.121:0/2830434161 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f17dc074c40 con 0x55a537ca4b00 2023-11-15T03:03:34.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.193+0000 7f17f42d8d00 1 --2- 172.21.15.121:0/2830434161 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55a537d2bef0 0x55a537d54180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.193+0000 7f17f1b10700 1 --2- 172.21.15.121:0/2830434161 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f17d4058ff0 0x7f17d405b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.193+0000 7f17f42d8d00 1 -- 172.21.15.121:0/2830434161 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55a537c8fdb0 con 0x55a537d2bef0 2023-11-15T03:03:34.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.193+0000 7f17f2311700 1 --2- 172.21.15.121:0/2830434161 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55a537d2bef0 0x55a537d54180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.193+0000 7f17f1b10700 1 --2- 172.21.15.121:0/2830434161 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f17d4058ff0 0x7f17d405b4b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f17e4000c00 tx=0x7f17e4001a80).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.193+0000 7f17f2311700 1 --2- 172.21.15.121:0/2830434161 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55a537d2bef0 0x55a537d54180 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.197+0000 7f17f2311700 1 -- 172.21.15.121:0/2830434161 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-keys out=5b] v0'0 uv2 ondisk = 0) v8 ==== 159+0+5 (crc 0 0 0) 0x7f17ec017040 con 0x55a537d2bef0 2023-11-15T03:03:34.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.197+0000 7f17f42d8d00 1 -- 172.21.15.121:0/2830434161 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55a537d2bef0 msgr2=0x55a537d54180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.197+0000 7f17f42d8d00 1 --2- 172.21.15.121:0/2830434161 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55a537d2bef0 0x55a537d54180 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.197+0000 7f17f42d8d00 1 -- 172.21.15.121:0/2830434161 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f17d4058ff0 msgr2=0x7f17d405b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.197+0000 7f17f42d8d00 1 --2- 172.21.15.121:0/2830434161 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f17d4058ff0 0x7f17d405b4b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f17e4000c00 tx=0x7f17e4001a80).stop 2023-11-15T03:03:34.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.197+0000 7f17f42d8d00 1 -- 172.21.15.121:0/2830434161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a537ca4b00 msgr2=0x55a537d359a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.197+0000 7f17f42d8d00 1 --2- 172.21.15.121:0/2830434161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a537ca4b00 0x55a537d359a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x55a537ca1e90 tx=0x7f17dc0028b0).stop 2023-11-15T03:03:34.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.197+0000 7f17f2311700 1 -- 172.21.15.121:0/2830434161 reap_dead start 2023-11-15T03:03:34.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.197+0000 7f17f42d8d00 1 -- 172.21.15.121:0/2830434161 shutdown_connections 2023-11-15T03:03:34.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.197+0000 7f17f42d8d00 1 -- 172.21.15.121:0/2830434161 >> 172.21.15.121:0/2830434161 conn(0x55a537caa080 msgr2=0x55a537ca2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.197+0000 7f17f42d8d00 1 -- 172.21.15.121:0/2830434161 shutdown_connections 2023-11-15T03:03:34.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.197+0000 7f17f42d8d00 1 -- 172.21.15.121:0/2830434161 wait complete. 2023-11-15T03:03:34.214 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 400.00000000 2023-11-15T03:03:34.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.269+0000 7fd13ab30d00 1 Processor -- start 2023-11-15T03:03:34.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.269+0000 7fd13ab30d00 1 -- start start 2023-11-15T03:03:34.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd13ab30d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557974996bd0 0x557974a3a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd13ab30d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55797499d060 0x55797496c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd13ab30d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557974a41dc0 0x55797496c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd13ab30d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579748e4220 con 0x557974a41dc0 2023-11-15T03:03:34.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd13ab30d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797496ce10 con 0x55797499d060 2023-11-15T03:03:34.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd13ab30d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557974a42ca0 con 0x557974996bd0 2023-11-15T03:03:34.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd12f916700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557974a41dc0 0x55797496c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd12f916700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557974a41dc0 0x55797496c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50474/0 (socket says 172.21.15.121:50474) 2023-11-15T03:03:34.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd12f916700 1 -- 172.21.15.121:0/2728437610 learned_addr learned my addr 172.21.15.121:0/2728437610 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd126914700 1 --2- 172.21.15.121:0/2728437610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55797499d060 0x55797496c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd12f115700 1 --2- 172.21.15.121:0/2728437610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557974996bd0 0x557974a3a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd12f916700 1 -- 172.21.15.121:0/2728437610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557974996bd0 msgr2=0x557974a3a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd12f916700 1 --2- 172.21.15.121:0/2728437610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557974996bd0 0x557974a3a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd12f916700 1 -- 172.21.15.121:0/2728437610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55797499d060 msgr2=0x55797496c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd12f916700 1 --2- 172.21.15.121:0/2728437610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55797499d060 0x55797496c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd12f916700 1 -- 172.21.15.121:0/2728437610 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd118005db0 con 0x557974a41dc0 2023-11-15T03:03:34.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd12f916700 1 --2- 172.21.15.121:0/2728437610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557974a41dc0 0x55797496c920 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7fd128006fd0 tx=0x7fd128006ac0).ready entity=mon.0 client_cookie=81c3dad43b9a0dee server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd12e113700 1 -- 172.21.15.121:0/2728437610 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1280030d0 con 0x557974a41dc0 2023-11-15T03:03:34.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd12e113700 1 -- 172.21.15.121:0/2728437610 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd128012400 con 0x557974a41dc0 2023-11-15T03:03:34.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd12e113700 1 -- 172.21.15.121:0/2728437610 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1280127c0 con 0x557974a41dc0 2023-11-15T03:03:34.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd13ab30d00 1 -- 172.21.15.121:0/2728437610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557974a41dc0 msgr2=0x55797496c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd13ab30d00 1 --2- 172.21.15.121:0/2728437610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557974a41dc0 0x55797496c920 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7fd128006fd0 tx=0x7fd128006ac0).stop 2023-11-15T03:03:34.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd13ab30d00 1 -- 172.21.15.121:0/2728437610 shutdown_connections 2023-11-15T03:03:34.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd13ab30d00 1 --2- 172.21.15.121:0/2728437610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557974a41dc0 0x55797496c920 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd13ab30d00 1 --2- 172.21.15.121:0/2728437610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55797499d060 0x55797496c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd13ab30d00 1 --2- 172.21.15.121:0/2728437610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557974996bd0 0x557974a3a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd13ab30d00 1 -- 172.21.15.121:0/2728437610 >> 172.21.15.121:0/2728437610 conn(0x5579749b3f80 msgr2=0x557974a41460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd13ab30d00 1 -- 172.21.15.121:0/2728437610 shutdown_connections 2023-11-15T03:03:34.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.273+0000 7fd13ab30d00 1 -- 172.21.15.121:0/2728437610 wait complete. 2023-11-15T03:03:34.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.277+0000 7fd13ab30d00 1 Processor -- start 2023-11-15T03:03:34.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.277+0000 7fd13ab30d00 1 -- start start 2023-11-15T03:03:34.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.277+0000 7fd13ab30d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557974996bd0 0x557974a336c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.277+0000 7fd13ab30d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55797499d060 0x557974a35b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.277+0000 7fd13ab30d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557974a41dc0 0x557974a36070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.277+0000 7fd13ab30d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557974a371a0 con 0x55797499d060 2023-11-15T03:03:34.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.277+0000 7fd13ab30d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557974a36560 con 0x557974a41dc0 2023-11-15T03:03:34.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.277+0000 7fd13ab30d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557974a39760 con 0x557974996bd0 2023-11-15T03:03:34.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.277+0000 7fd126914700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55797499d060 0x557974a35b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.277+0000 7fd126914700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55797499d060 0x557974a35b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50486/0 (socket says 172.21.15.121:50486) 2023-11-15T03:03:34.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.277+0000 7fd126914700 1 -- 172.21.15.121:0/1749387227 learned_addr learned my addr 172.21.15.121:0/1749387227 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.277+0000 7fd12f115700 1 --2- 172.21.15.121:0/1749387227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557974996bd0 0x557974a336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.277+0000 7fd12f916700 1 --2- 172.21.15.121:0/1749387227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557974a41dc0 0x557974a36070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.277+0000 7fd126914700 1 -- 172.21.15.121:0/1749387227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557974996bd0 msgr2=0x557974a336c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.277+0000 7fd126914700 1 --2- 172.21.15.121:0/1749387227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557974996bd0 0x557974a336c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.277+0000 7fd126914700 1 -- 172.21.15.121:0/1749387227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557974a41dc0 msgr2=0x557974a36070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.277+0000 7fd126914700 1 --2- 172.21.15.121:0/1749387227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557974a41dc0 0x557974a36070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.277+0000 7fd126914700 1 -- 172.21.15.121:0/1749387227 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579749127f0 con 0x55797499d060 2023-11-15T03:03:34.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.281+0000 7fd126914700 1 --2- 172.21.15.121:0/1749387227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55797499d060 0x557974a35b80 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7fd118010000 tx=0x7fd118016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.281+0000 7fd12c910700 1 -- 172.21.15.121:0/1749387227 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118013510 con 0x55797499d060 2023-11-15T03:03:34.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.281+0000 7fd12c910700 1 -- 172.21.15.121:0/1749387227 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd118013670 con 0x55797499d060 2023-11-15T03:03:34.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.281+0000 7fd13ab30d00 1 -- 172.21.15.121:0/1749387227 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55797490d770 con 0x55797499d060 2023-11-15T03:03:34.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.281+0000 7fd12c910700 1 -- 172.21.15.121:0/1749387227 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1180139f0 con 0x55797499d060 2023-11-15T03:03:34.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.281+0000 7fd13ab30d00 1 -- 172.21.15.121:0/1749387227 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579748dca70 con 0x55797499d060 2023-11-15T03:03:34.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.281+0000 7fd12c910700 1 -- 172.21.15.121:0/1749387227 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd118028460 con 0x55797499d060 2023-11-15T03:03:34.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.285+0000 7fd12c910700 1 --2- 172.21.15.121:0/1749387227 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd110059020 0x7fd11005b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.285+0000 7fd12c910700 1 -- 172.21.15.121:0/1749387227 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fd1180750d0 con 0x55797499d060 2023-11-15T03:03:34.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.285+0000 7fd13ab30d00 1 --2- 172.21.15.121:0/1749387227 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x557974a3f640 0x557974a38900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.285+0000 7fd12f115700 1 --2- 172.21.15.121:0/1749387227 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd110059020 0x7fd11005b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.285+0000 7fd13ab30d00 1 -- 172.21.15.121:0/1749387227 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x5579749b6500 con 0x557974a3f640 2023-11-15T03:03:34.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.285+0000 7fd12f916700 1 --2- 172.21.15.121:0/1749387227 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x557974a3f640 0x557974a38900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.285+0000 7fd12f115700 1 --2- 172.21.15.121:0/1749387227 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd110059020 0x7fd11005b4e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fd120009a40 tx=0x7fd12000c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.285+0000 7fd12f916700 1 --2- 172.21.15.121:0/1749387227 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x557974a3f640 0x557974a38900 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.285+0000 7fd12f916700 1 -- 172.21.15.121:0/1749387227 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 400.00000000 [omap-get-keys out=5b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fd128000f40 con 0x557974a3f640 2023-11-15T03:03:34.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.285+0000 7fd13ab30d00 1 -- 172.21.15.121:0/1749387227 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x557974a3f640 msgr2=0x557974a38900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.285+0000 7fd13ab30d00 1 --2- 172.21.15.121:0/1749387227 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x557974a3f640 0x557974a38900 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.289+0000 7fd13ab30d00 1 -- 172.21.15.121:0/1749387227 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd110059020 msgr2=0x7fd11005b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.289+0000 7fd13ab30d00 1 --2- 172.21.15.121:0/1749387227 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd110059020 0x7fd11005b4e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fd120009a40 tx=0x7fd12000c040).stop 2023-11-15T03:03:34.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.289+0000 7fd13ab30d00 1 -- 172.21.15.121:0/1749387227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55797499d060 msgr2=0x557974a35b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.289+0000 7fd13ab30d00 1 --2- 172.21.15.121:0/1749387227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55797499d060 0x557974a35b80 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7fd118010000 tx=0x7fd118016000).stop 2023-11-15T03:03:34.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.289+0000 7fd12f916700 1 -- 172.21.15.121:0/1749387227 reap_dead start 2023-11-15T03:03:34.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.289+0000 7fd13ab30d00 1 -- 172.21.15.121:0/1749387227 shutdown_connections 2023-11-15T03:03:34.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.289+0000 7fd13ab30d00 1 -- 172.21.15.121:0/1749387227 >> 172.21.15.121:0/1749387227 conn(0x5579749b3f80 msgr2=0x557974a41460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.289+0000 7fd13ab30d00 1 -- 172.21.15.121:0/1749387227 shutdown_connections 2023-11-15T03:03:34.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.289+0000 7fd13ab30d00 1 -- 172.21.15.121:0/1749387227 wait complete. 2023-11-15T03:03:34.309 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 100.00000000.inode 2023-11-15T03:03:34.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.361+0000 7fece06dad00 1 Processor -- start 2023-11-15T03:03:34.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.361+0000 7fece06dad00 1 -- start start 2023-11-15T03:03:34.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fece06dad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fafdf17a30 0x55fafdfee330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fece06dad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafdf62e70 0x55fafdff6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fece06dad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fafdff8000 0x55fafdf5bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fece06dad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fafde90220 con 0x55fafdf17a30 2023-11-15T03:03:34.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fece06dad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fafdfed8d0 con 0x55fafdf62e70 2023-11-15T03:03:34.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fece06dad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fafdf494e0 con 0x55fafdff8000 2023-11-15T03:03:34.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fecddf5e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fafdf17a30 0x55fafdfee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fecddf5e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fafdf17a30 0x55fafdfee330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50504/0 (socket says 172.21.15.121:50504) 2023-11-15T03:03:34.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fecddf5e700 1 -- 172.21.15.121:0/1258622509 learned_addr learned my addr 172.21.15.121:0/1258622509 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fecd575d700 1 --2- 172.21.15.121:0/1258622509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafdf62e70 0x55fafdff6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fecde75f700 1 --2- 172.21.15.121:0/1258622509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fafdff8000 0x55fafdf5bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fecddf5e700 1 -- 172.21.15.121:0/1258622509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fafdff8000 msgr2=0x55fafdf5bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fecddf5e700 1 --2- 172.21.15.121:0/1258622509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fafdff8000 0x55fafdf5bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fecddf5e700 1 -- 172.21.15.121:0/1258622509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafdf62e70 msgr2=0x55fafdff6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fecddf5e700 1 --2- 172.21.15.121:0/1258622509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafdf62e70 0x55fafdff6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fecddf5e700 1 -- 172.21.15.121:0/1258622509 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd0005cf0 con 0x55fafdf17a30 2023-11-15T03:03:34.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fecddf5e700 1 --2- 172.21.15.121:0/1258622509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fafdf17a30 0x55fafdfee330 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7fecd0009a70 tx=0x7fecd000c040).ready entity=mon.0 client_cookie=70d488ba5f68b164 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fecdcf5c700 1 -- 172.21.15.121:0/1258622509 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd000f040 con 0x55fafdf17a30 2023-11-15T03:03:34.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fecdcf5c700 1 -- 172.21.15.121:0/1258622509 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd0008b30 con 0x55fafdf17a30 2023-11-15T03:03:34.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fecdcf5c700 1 -- 172.21.15.121:0/1258622509 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd00143f0 con 0x55fafdf17a30 2023-11-15T03:03:34.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fece06dad00 1 -- 172.21.15.121:0/1258622509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fafdf17a30 msgr2=0x55fafdfee330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fece06dad00 1 --2- 172.21.15.121:0/1258622509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fafdf17a30 0x55fafdfee330 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7fecd0009a70 tx=0x7fecd000c040).stop 2023-11-15T03:03:34.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fece06dad00 1 -- 172.21.15.121:0/1258622509 shutdown_connections 2023-11-15T03:03:34.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fece06dad00 1 --2- 172.21.15.121:0/1258622509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fafdff8000 0x55fafdf5bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fece06dad00 1 --2- 172.21.15.121:0/1258622509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafdf62e70 0x55fafdff6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fece06dad00 1 --2- 172.21.15.121:0/1258622509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fafdf17a30 0x55fafdfee330 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fece06dad00 1 -- 172.21.15.121:0/1258622509 >> 172.21.15.121:0/1258622509 conn(0x55fafdf67080 msgr2=0x55fafdf16ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fece06dad00 1 -- 172.21.15.121:0/1258622509 shutdown_connections 2023-11-15T03:03:34.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.365+0000 7fece06dad00 1 -- 172.21.15.121:0/1258622509 wait complete. 2023-11-15T03:03:34.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fece06dad00 1 Processor -- start 2023-11-15T03:03:34.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fece06dad00 1 -- start start 2023-11-15T03:03:34.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fece06dad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fafdf17a30 0x55fafdfec6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fece06dad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fafdf62e70 0x55fafdff0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fece06dad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafdff8000 0x55fafdff1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fece06dad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fafdff1540 con 0x55fafdf17a30 2023-11-15T03:03:34.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fece06dad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fafdf5ee10 con 0x55fafdff8000 2023-11-15T03:03:34.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fece06dad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fafdfecba0 con 0x55fafdf62e70 2023-11-15T03:03:34.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fecddf5e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fafdf17a30 0x55fafdfec6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fecddf5e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fafdf17a30 0x55fafdfec6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50508/0 (socket says 172.21.15.121:50508) 2023-11-15T03:03:34.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fecddf5e700 1 -- 172.21.15.121:0/2460219238 learned_addr learned my addr 172.21.15.121:0/2460219238 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fecd575d700 1 --2- 172.21.15.121:0/2460219238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fafdf62e70 0x55fafdff0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fecde75f700 1 --2- 172.21.15.121:0/2460219238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafdff8000 0x55fafdff1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fecddf5e700 1 -- 172.21.15.121:0/2460219238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fafdf62e70 msgr2=0x55fafdff0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fecddf5e700 1 --2- 172.21.15.121:0/2460219238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fafdf62e70 0x55fafdff0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fecddf5e700 1 -- 172.21.15.121:0/2460219238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafdff8000 msgr2=0x55fafdff1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fecddf5e700 1 --2- 172.21.15.121:0/2460219238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fafdff8000 0x55fafdff1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fecddf5e700 1 -- 172.21.15.121:0/2460219238 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd0009670 con 0x55fafdf17a30 2023-11-15T03:03:34.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fecddf5e700 1 --2- 172.21.15.121:0/2460219238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fafdf17a30 0x55fafdfec6b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7fecd00086a0 tx=0x7fecd0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.369+0000 7fecd6ffd700 1 -- 172.21.15.121:0/2460219238 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd000f040 con 0x55fafdf17a30 2023-11-15T03:03:34.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.373+0000 7fecd6ffd700 1 -- 172.21.15.121:0/2460219238 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd0014ac0 con 0x55fafdf17a30 2023-11-15T03:03:34.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.373+0000 7fecd6ffd700 1 -- 172.21.15.121:0/2460219238 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd0014e60 con 0x55fafdf17a30 2023-11-15T03:03:34.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.373+0000 7fece06dad00 1 -- 172.21.15.121:0/2460219238 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd800a0e0 con 0x55fafdf17a30 2023-11-15T03:03:34.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.373+0000 7fece06dad00 1 -- 172.21.15.121:0/2460219238 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fafdeb9770 con 0x55fafdf17a30 2023-11-15T03:03:34.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.373+0000 7fecd6ffd700 1 -- 172.21.15.121:0/2460219238 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fecd00263b0 con 0x55fafdf17a30 2023-11-15T03:03:34.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.373+0000 7fecd6ffd700 1 --2- 172.21.15.121:0/2460219238 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fecc0058ff0 0x7fecc005b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.373+0000 7fecd6ffd700 1 -- 172.21.15.121:0/2460219238 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fecd0074830 con 0x55fafdf17a30 2023-11-15T03:03:34.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.377+0000 7fecd575d700 1 --2- 172.21.15.121:0/2460219238 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fecc0058ff0 0x7fecc005b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.377+0000 7fece06dad00 1 --2- 172.21.15.121:0/2460219238 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55fafdf5f2a0 0x55fafdf5f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.377+0000 7fece06dad00 1 -- 172.21.15.121:0/2460219238 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55fafdf5fb70 con 0x55fafdf5f2a0 2023-11-15T03:03:34.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.377+0000 7fecd575d700 1 --2- 172.21.15.121:0/2460219238 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fecc0058ff0 0x7fecc005b4b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x55fafdf64830 tx=0x7fecc8001870).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.377+0000 7fecde75f700 1 --2- 172.21.15.121:0/2460219238 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55fafdf5f2a0 0x55fafdf5f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.377+0000 7fecde75f700 1 --2- 172.21.15.121:0/2460219238 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55fafdf5f2a0 0x55fafdf5f680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.377+0000 7fecde75f700 1 -- 172.21.15.121:0/2460219238 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 100.00000000.inode [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+5 (crc 0 0 0) 0x7fecd8017040 con 0x55fafdf5f2a0 2023-11-15T03:03:34.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.377+0000 7fece06dad00 1 -- 172.21.15.121:0/2460219238 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55fafdf5f2a0 msgr2=0x55fafdf5f680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.377+0000 7fece06dad00 1 --2- 172.21.15.121:0/2460219238 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55fafdf5f2a0 0x55fafdf5f680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.377+0000 7fece06dad00 1 -- 172.21.15.121:0/2460219238 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fecc0058ff0 msgr2=0x7fecc005b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.377+0000 7fece06dad00 1 --2- 172.21.15.121:0/2460219238 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fecc0058ff0 0x7fecc005b4b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x55fafdf64830 tx=0x7fecc8001870).stop 2023-11-15T03:03:34.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.377+0000 7fece06dad00 1 -- 172.21.15.121:0/2460219238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fafdf17a30 msgr2=0x55fafdfec6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.377+0000 7fece06dad00 1 --2- 172.21.15.121:0/2460219238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fafdf17a30 0x55fafdfec6b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7fecd00086a0 tx=0x7fecd0017000).stop 2023-11-15T03:03:34.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.377+0000 7fecde75f700 1 -- 172.21.15.121:0/2460219238 reap_dead start 2023-11-15T03:03:34.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.377+0000 7fece06dad00 1 -- 172.21.15.121:0/2460219238 shutdown_connections 2023-11-15T03:03:34.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.377+0000 7fece06dad00 1 -- 172.21.15.121:0/2460219238 >> 172.21.15.121:0/2460219238 conn(0x55fafdf67080 msgr2=0x55fafdf16ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.377+0000 7fece06dad00 1 -- 172.21.15.121:0/2460219238 shutdown_connections 2023-11-15T03:03:34.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.377+0000 7fece06dad00 1 -- 172.21.15.121:0/2460219238 wait complete. 2023-11-15T03:03:34.394 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000 2023-11-15T03:03:34.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.449+0000 7fe9806ebd00 1 Processor -- start 2023-11-15T03:03:34.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.449+0000 7fe9806ebd00 1 -- start start 2023-11-15T03:03:34.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9806ebd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558de9c051e0 0x558de9bc06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9806ebd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558de9c08030 0x558de9bece40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9806ebd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9c98220 0x558de9bed330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9806ebd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558de9c97650 con 0x558de9c08030 2023-11-15T03:03:34.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9806ebd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558de9bed820 con 0x558de9c98220 2023-11-15T03:03:34.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9806ebd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558de9bbc3a0 con 0x558de9c051e0 2023-11-15T03:03:34.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9757cc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558de9c08030 0x558de9bece40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9757cc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558de9c08030 0x558de9bece40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50524/0 (socket says 172.21.15.121:50524) 2023-11-15T03:03:34.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9757cc700 1 -- 172.21.15.121:0/3183524921 learned_addr learned my addr 172.21.15.121:0/3183524921 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe97dfcd700 1 --2- 172.21.15.121:0/3183524921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558de9c051e0 0x558de9bc06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe97e7ce700 1 --2- 172.21.15.121:0/3183524921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9c98220 0x558de9bed330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9757cc700 1 -- 172.21.15.121:0/3183524921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558de9c051e0 msgr2=0x558de9bc06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9757cc700 1 --2- 172.21.15.121:0/3183524921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558de9c051e0 0x558de9bc06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9757cc700 1 -- 172.21.15.121:0/3183524921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9c98220 msgr2=0x558de9bed330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9757cc700 1 --2- 172.21.15.121:0/3183524921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9c98220 0x558de9bed330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9757cc700 1 -- 172.21.15.121:0/3183524921 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe970005cf0 con 0x558de9c08030 2023-11-15T03:03:34.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9757cc700 1 --2- 172.21.15.121:0/3183524921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558de9c08030 0x558de9bece40 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0x7fe9680097d0 tx=0x7fe96800c040).ready entity=mon.0 client_cookie=5bf8b9635cc4e633 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe97cfcb700 1 -- 172.21.15.121:0/3183524921 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96800f040 con 0x558de9c08030 2023-11-15T03:03:34.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe97cfcb700 1 -- 172.21.15.121:0/3183524921 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe968008860 con 0x558de9c08030 2023-11-15T03:03:34.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe97cfcb700 1 -- 172.21.15.121:0/3183524921 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe968008be0 con 0x558de9c08030 2023-11-15T03:03:34.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9806ebd00 1 -- 172.21.15.121:0/3183524921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558de9c08030 msgr2=0x558de9bece40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9806ebd00 1 --2- 172.21.15.121:0/3183524921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558de9c08030 0x558de9bece40 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0x7fe9680097d0 tx=0x7fe96800c040).stop 2023-11-15T03:03:34.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9806ebd00 1 -- 172.21.15.121:0/3183524921 shutdown_connections 2023-11-15T03:03:34.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9806ebd00 1 --2- 172.21.15.121:0/3183524921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9c98220 0x558de9bed330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9806ebd00 1 --2- 172.21.15.121:0/3183524921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558de9c08030 0x558de9bece40 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9806ebd00 1 --2- 172.21.15.121:0/3183524921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558de9c051e0 0x558de9bc06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.453+0000 7fe9806ebd00 1 -- 172.21.15.121:0/3183524921 >> 172.21.15.121:0/3183524921 conn(0x558de9c058f0 msgr2=0x558de9bbf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.457+0000 7fe9806ebd00 1 -- 172.21.15.121:0/3183524921 shutdown_connections 2023-11-15T03:03:34.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.457+0000 7fe9806ebd00 1 -- 172.21.15.121:0/3183524921 wait complete. 2023-11-15T03:03:34.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.457+0000 7fe9806ebd00 1 Processor -- start 2023-11-15T03:03:34.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.457+0000 7fe9806ebd00 1 -- start start 2023-11-15T03:03:34.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.457+0000 7fe9806ebd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9c051e0 0x558de9c9b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.457+0000 7fe9806ebd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558de9c08030 0x558de9c95d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.457+0000 7fe9806ebd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558de9c98220 0x558de9c96270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.457+0000 7fe9806ebd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558de9be8960 con 0x558de9c98220 2023-11-15T03:03:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.457+0000 7fe9806ebd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558de9c9aac0 con 0x558de9c051e0 2023-11-15T03:03:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.457+0000 7fe9806ebd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558de9c9ad60 con 0x558de9c08030 2023-11-15T03:03:34.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.461+0000 7fe97e7ce700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558de9c98220 0x558de9c96270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.461+0000 7fe97dfcd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9c051e0 0x558de9c9b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.461+0000 7fe9757cc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558de9c08030 0x558de9c95d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.461+0000 7fe97e7ce700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558de9c98220 0x558de9c96270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50540/0 (socket says 172.21.15.121:50540) 2023-11-15T03:03:34.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.461+0000 7fe97e7ce700 1 -- 172.21.15.121:0/2821885585 learned_addr learned my addr 172.21.15.121:0/2821885585 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.461+0000 7fe97e7ce700 1 -- 172.21.15.121:0/2821885585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558de9c08030 msgr2=0x558de9c95d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.461+0000 7fe97e7ce700 1 --2- 172.21.15.121:0/2821885585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558de9c08030 0x558de9c95d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.461+0000 7fe97e7ce700 1 -- 172.21.15.121:0/2821885585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9c051e0 msgr2=0x558de9c9b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.461+0000 7fe97e7ce700 1 --2- 172.21.15.121:0/2821885585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9c051e0 0x558de9c9b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.461+0000 7fe97e7ce700 1 -- 172.21.15.121:0/2821885585 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe968009430 con 0x558de9c98220 2023-11-15T03:03:34.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.461+0000 7fe97e7ce700 1 --2- 172.21.15.121:0/2821885585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558de9c98220 0x558de9c96270 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7fe97800e000 tx=0x7fe978014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.461+0000 7fe976ffd700 1 -- 172.21.15.121:0/2821885585 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978011410 con 0x558de9c98220 2023-11-15T03:03:34.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.461+0000 7fe976ffd700 1 -- 172.21.15.121:0/2821885585 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe978005a80 con 0x558de9c98220 2023-11-15T03:03:34.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.461+0000 7fe9806ebd00 1 -- 172.21.15.121:0/2821885585 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97800a180 con 0x558de9c98220 2023-11-15T03:03:34.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.461+0000 7fe976ffd700 1 -- 172.21.15.121:0/2821885585 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9780118f0 con 0x558de9c98220 2023-11-15T03:03:34.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.461+0000 7fe9806ebd00 1 -- 172.21.15.121:0/2821885585 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558de9b5da20 con 0x558de9c98220 2023-11-15T03:03:34.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.465+0000 7fe976ffd700 1 -- 172.21.15.121:0/2821885585 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe978003040 con 0x558de9c98220 2023-11-15T03:03:34.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.465+0000 7fe976ffd700 1 --2- 172.21.15.121:0/2821885585 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe960058ff0 0x7fe96005b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.465+0000 7fe976ffd700 1 -- 172.21.15.121:0/2821885585 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fe978074fc0 con 0x558de9c98220 2023-11-15T03:03:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.465+0000 7fe97dfcd700 1 --2- 172.21.15.121:0/2821885585 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe960058ff0 0x7fe96005b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.465+0000 7fe9806ebd00 1 --2- 172.21.15.121:0/2821885585 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x558de9be9e20 0x558de9c01430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.465+0000 7fe9806ebd00 1 -- 172.21.15.121:0/2821885585 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.3f 13:ff5b34d6:::1.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x558de9c01920 con 0x558de9be9e20 2023-11-15T03:03:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.465+0000 7fe9757cc700 1 --2- 172.21.15.121:0/2821885585 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x558de9be9e20 0x558de9c01430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.465+0000 7fe97dfcd700 1 --2- 172.21.15.121:0/2821885585 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe960058ff0 0x7fe96005b4b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fe970009d10 tx=0x7fe97000c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.465+0000 7fe9757cc700 1 --2- 172.21.15.121:0/2821885585 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x558de9be9e20 0x558de9c01430 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.469+0000 7fe9757cc700 1 -- 172.21.15.121:0/2821885585 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 1.00000000 [omap-get-keys out=20b] v0'0 uv5 ondisk = 0) v8 ==== 154+0+20 (crc 0 0 0) 0x7fe968014430 con 0x558de9be9e20 2023-11-15T03:03:34.477 INFO:teuthology.orchestra.run.smithi121.stdout:subdir_head 2023-11-15T03:03:34.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.469+0000 7fe9806ebd00 1 -- 172.21.15.121:0/2821885585 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x558de9be9e20 msgr2=0x558de9c01430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.469+0000 7fe9806ebd00 1 --2- 172.21.15.121:0/2821885585 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x558de9be9e20 0x558de9c01430 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.469+0000 7fe9806ebd00 1 -- 172.21.15.121:0/2821885585 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe960058ff0 msgr2=0x7fe96005b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.469+0000 7fe9806ebd00 1 --2- 172.21.15.121:0/2821885585 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe960058ff0 0x7fe96005b4b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fe970009d10 tx=0x7fe97000c040).stop 2023-11-15T03:03:34.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.469+0000 7fe9806ebd00 1 -- 172.21.15.121:0/2821885585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558de9c98220 msgr2=0x558de9c96270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.469+0000 7fe9806ebd00 1 --2- 172.21.15.121:0/2821885585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558de9c98220 0x558de9c96270 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7fe97800e000 tx=0x7fe978014000).stop 2023-11-15T03:03:34.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.469+0000 7fe97e7ce700 1 -- 172.21.15.121:0/2821885585 reap_dead start 2023-11-15T03:03:34.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.469+0000 7fe9806ebd00 1 -- 172.21.15.121:0/2821885585 shutdown_connections 2023-11-15T03:03:34.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.469+0000 7fe9806ebd00 1 -- 172.21.15.121:0/2821885585 >> 172.21.15.121:0/2821885585 conn(0x558de9c058f0 msgr2=0x558de9bbf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.469+0000 7fe9806ebd00 1 -- 172.21.15.121:0/2821885585 shutdown_connections 2023-11-15T03:03:34.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.469+0000 7fe9806ebd00 1 -- 172.21.15.121:0/2821885585 wait complete. 2023-11-15T03:03:34.481 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 10000000000.00000000 2023-11-15T03:03:34.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.529+0000 7f6b8412fd00 1 Processor -- start 2023-11-15T03:03:34.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.529+0000 7f6b8412fd00 1 -- start start 2023-11-15T03:03:34.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.529+0000 7f6b8412fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56268b2f8dd0 0x56268b33e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.529+0000 7f6b8412fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56268b342b00 0x56268b33cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.529+0000 7f6b8412fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56268b3d1670 0x56268b33d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.529+0000 7f6b8412fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56268b271220 con 0x56268b2f8dd0 2023-11-15T03:03:34.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.529+0000 7f6b8412fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56268b3d47e0 con 0x56268b342b00 2023-11-15T03:03:34.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.529+0000 7f6b8412fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56268b348460 con 0x56268b3d1670 2023-11-15T03:03:34.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.529+0000 7f6b786b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56268b2f8dd0 0x56268b33e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.529+0000 7f6b786b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56268b2f8dd0 0x56268b33e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50550/0 (socket says 172.21.15.121:50550) 2023-11-15T03:03:34.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.529+0000 7f6b786b7700 1 -- 172.21.15.121:0/134351991 learned_addr learned my addr 172.21.15.121:0/134351991 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.529+0000 7f6b78eb8700 1 --2- 172.21.15.121:0/134351991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56268b3d1670 0x56268b33d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.529+0000 7f6b6feb6700 1 --2- 172.21.15.121:0/134351991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56268b342b00 0x56268b33cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.529+0000 7f6b786b7700 1 -- 172.21.15.121:0/134351991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56268b3d1670 msgr2=0x56268b33d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.529+0000 7f6b786b7700 1 --2- 172.21.15.121:0/134351991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56268b3d1670 0x56268b33d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.529+0000 7f6b786b7700 1 -- 172.21.15.121:0/134351991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56268b342b00 msgr2=0x56268b33cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.529+0000 7f6b786b7700 1 --2- 172.21.15.121:0/134351991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56268b342b00 0x56268b33cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.529+0000 7f6b786b7700 1 -- 172.21.15.121:0/134351991 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b68005db0 con 0x56268b2f8dd0 2023-11-15T03:03:34.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.529+0000 7f6b78eb8700 1 --2- 172.21.15.121:0/134351991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56268b3d1670 0x56268b33d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:34.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.533+0000 7f6b786b7700 1 --2- 172.21.15.121:0/134351991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56268b2f8dd0 0x56268b33e3d0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f6b68002f90 tx=0x7f6b68016000).ready entity=mon.0 client_cookie=aa704bfb3b5a2392 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.533+0000 7f6b776b5700 1 -- 172.21.15.121:0/134351991 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b68005350 con 0x56268b2f8dd0 2023-11-15T03:03:34.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.533+0000 7f6b776b5700 1 -- 172.21.15.121:0/134351991 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b68000bc0 con 0x56268b2f8dd0 2023-11-15T03:03:34.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.533+0000 7f6b776b5700 1 -- 172.21.15.121:0/134351991 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b68008ab0 con 0x56268b2f8dd0 2023-11-15T03:03:34.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.533+0000 7f6b8412fd00 1 -- 172.21.15.121:0/134351991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56268b2f8dd0 msgr2=0x56268b33e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.533+0000 7f6b8412fd00 1 --2- 172.21.15.121:0/134351991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56268b2f8dd0 0x56268b33e3d0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f6b68002f90 tx=0x7f6b68016000).stop 2023-11-15T03:03:34.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.533+0000 7f6b8412fd00 1 -- 172.21.15.121:0/134351991 shutdown_connections 2023-11-15T03:03:34.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.533+0000 7f6b8412fd00 1 --2- 172.21.15.121:0/134351991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56268b3d1670 0x56268b33d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.533+0000 7f6b8412fd00 1 --2- 172.21.15.121:0/134351991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56268b342b00 0x56268b33cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.533+0000 7f6b8412fd00 1 --2- 172.21.15.121:0/134351991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56268b2f8dd0 0x56268b33e3d0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.533+0000 7f6b8412fd00 1 -- 172.21.15.121:0/134351991 >> 172.21.15.121:0/134351991 conn(0x56268b348080 msgr2=0x56268b3466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.533+0000 7f6b8412fd00 1 -- 172.21.15.121:0/134351991 shutdown_connections 2023-11-15T03:03:34.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.533+0000 7f6b8412fd00 1 -- 172.21.15.121:0/134351991 wait complete. 2023-11-15T03:03:34.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.533+0000 7f6b8412fd00 1 Processor -- start 2023-11-15T03:03:34.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.533+0000 7f6b8412fd00 1 -- start start 2023-11-15T03:03:34.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b8412fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56268b2f8dd0 0x56268b3c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b8412fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56268b342b00 0x56268b3d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b8412fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56268b3d1670 0x56268b3d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b8412fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56268b3d31c0 con 0x56268b3d1670 2023-11-15T03:03:34.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b8412fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56268b3d13f0 con 0x56268b342b00 2023-11-15T03:03:34.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b8412fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56268b3e2040 con 0x56268b2f8dd0 2023-11-15T03:03:34.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b78eb8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56268b3d1670 0x56268b3d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b786b7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56268b2f8dd0 0x56268b3c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b78eb8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56268b3d1670 0x56268b3d2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50560/0 (socket says 172.21.15.121:50560) 2023-11-15T03:03:34.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b78eb8700 1 -- 172.21.15.121:0/4060990339 learned_addr learned my addr 172.21.15.121:0/4060990339 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b6feb6700 1 --2- 172.21.15.121:0/4060990339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56268b342b00 0x56268b3d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b78eb8700 1 -- 172.21.15.121:0/4060990339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56268b2f8dd0 msgr2=0x56268b3c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b78eb8700 1 --2- 172.21.15.121:0/4060990339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56268b2f8dd0 0x56268b3c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b78eb8700 1 -- 172.21.15.121:0/4060990339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56268b342b00 msgr2=0x56268b3d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b78eb8700 1 --2- 172.21.15.121:0/4060990339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56268b342b00 0x56268b3d27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b78eb8700 1 -- 172.21.15.121:0/4060990339 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56268b29aa20 con 0x56268b3d1670 2023-11-15T03:03:34.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b78eb8700 1 --2- 172.21.15.121:0/4060990339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56268b3d1670 0x56268b3d2cd0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7f6b7000e560 tx=0x7f6b70007040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b75eb2700 1 -- 172.21.15.121:0/4060990339 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70012400 con 0x56268b3d1670 2023-11-15T03:03:34.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b75eb2700 1 -- 172.21.15.121:0/4060990339 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b70012560 con 0x56268b3d1670 2023-11-15T03:03:34.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b75eb2700 1 -- 172.21.15.121:0/4060990339 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70012900 con 0x56268b3d1670 2023-11-15T03:03:34.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b8412fd00 1 -- 172.21.15.121:0/4060990339 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56268b29a770 con 0x56268b3d1670 2023-11-15T03:03:34.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.537+0000 7f6b8412fd00 1 -- 172.21.15.121:0/4060990339 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56268b269a70 con 0x56268b3d1670 2023-11-15T03:03:34.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.541+0000 7f6b75eb2700 1 -- 172.21.15.121:0/4060990339 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6b70012cb0 con 0x56268b3d1670 2023-11-15T03:03:34.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.541+0000 7f6b75eb2700 1 --2- 172.21.15.121:0/4060990339 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6b58058ff0 0x7f6b5805b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.541+0000 7f6b75eb2700 1 -- 172.21.15.121:0/4060990339 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f6b700735b0 con 0x56268b3d1670 2023-11-15T03:03:34.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.541+0000 7f6b786b7700 1 --2- 172.21.15.121:0/4060990339 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6b58058ff0 0x7f6b5805b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.541+0000 7f6b8412fd00 1 --2- 172.21.15.121:0/4060990339 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56268b33dc70 0x56268b3295b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.541+0000 7f6b8412fd00 1 -- 172.21.15.121:0/4060990339 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:1 13.30 13:0cf6ad0f:::10000000000.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x56268b3d40c0 con 0x56268b33dc70 2023-11-15T03:03:34.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.541+0000 7f6b6feb6700 1 --2- 172.21.15.121:0/4060990339 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56268b33dc70 0x56268b3295b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.541+0000 7f6b786b7700 1 --2- 172.21.15.121:0/4060990339 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6b58058ff0 0x7f6b5805b4b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f6b68002f90 tx=0x7f6b6801c000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.545+0000 7f6b6feb6700 1 --2- 172.21.15.121:0/4060990339 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56268b33dc70 0x56268b3295b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.545+0000 7f6b6feb6700 1 -- 172.21.15.121:0/4060990339 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(1 10000000000.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x7f6b6000e720 con 0x56268b33dc70 2023-11-15T03:03:34.554 INFO:teuthology.orchestra.run.smithi121.stdout:cephfs_metadata/10000000000.00000000 mtime 2023-11-15T03:03:30.000000+0000, size 0 2023-11-15T03:03:34.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.545+0000 7f6b8412fd00 1 -- 172.21.15.121:0/4060990339 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56268b33dc70 msgr2=0x56268b3295b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.545+0000 7f6b8412fd00 1 --2- 172.21.15.121:0/4060990339 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56268b33dc70 0x56268b3295b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.545+0000 7f6b8412fd00 1 -- 172.21.15.121:0/4060990339 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6b58058ff0 msgr2=0x7f6b5805b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.545+0000 7f6b8412fd00 1 --2- 172.21.15.121:0/4060990339 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6b58058ff0 0x7f6b5805b4b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f6b68002f90 tx=0x7f6b6801c000).stop 2023-11-15T03:03:34.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.545+0000 7f6b8412fd00 1 -- 172.21.15.121:0/4060990339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56268b3d1670 msgr2=0x56268b3d2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.545+0000 7f6b8412fd00 1 --2- 172.21.15.121:0/4060990339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56268b3d1670 0x56268b3d2cd0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7f6b7000e560 tx=0x7f6b70007040).stop 2023-11-15T03:03:34.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.545+0000 7f6b78eb8700 1 -- 172.21.15.121:0/4060990339 reap_dead start 2023-11-15T03:03:34.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.545+0000 7f6b8412fd00 1 -- 172.21.15.121:0/4060990339 shutdown_connections 2023-11-15T03:03:34.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.545+0000 7f6b8412fd00 1 -- 172.21.15.121:0/4060990339 >> 172.21.15.121:0/4060990339 conn(0x56268b348080 msgr2=0x56268b3ca4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.545+0000 7f6b8412fd00 1 -- 172.21.15.121:0/4060990339 shutdown_connections 2023-11-15T03:03:34.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.545+0000 7f6b8412fd00 1 -- 172.21.15.121:0/4060990339 wait complete. 2023-11-15T03:03:34.558 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 600.00000000 2023-11-15T03:03:34.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.609+0000 7f2faa146d00 1 Processor -- start 2023-11-15T03:03:34.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.609+0000 7f2faa146d00 1 -- start start 2023-11-15T03:03:34.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2faa146d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x565415407450 0x5654154d5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2faa146d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565415435830 0x56541544db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2faa146d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654154dba30 0x5654154ee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2faa146d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541537d220 con 0x5654154dba30 2023-11-15T03:03:34.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2faa146d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654154daab0 con 0x565415435830 2023-11-15T03:03:34.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2faa146d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56541544eb40 con 0x565415407450 2023-11-15T03:03:34.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2f9efab700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654154dba30 0x5654154ee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2f9efab700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654154dba30 0x5654154ee0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50580/0 (socket says 172.21.15.121:50580) 2023-11-15T03:03:34.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2f9efab700 1 -- 172.21.15.121:0/204770856 learned_addr learned my addr 172.21.15.121:0/204770856 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2f9e7aa700 1 --2- 172.21.15.121:0/204770856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x565415407450 0x5654154d5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2f95fa9700 1 --2- 172.21.15.121:0/204770856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565415435830 0x56541544db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2f9efab700 1 -- 172.21.15.121:0/204770856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x565415407450 msgr2=0x5654154d5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2f9efab700 1 --2- 172.21.15.121:0/204770856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x565415407450 0x5654154d5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2f9efab700 1 -- 172.21.15.121:0/204770856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565415435830 msgr2=0x56541544db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2f9efab700 1 --2- 172.21.15.121:0/204770856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565415435830 0x56541544db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2f9efab700 1 -- 172.21.15.121:0/204770856 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9800a0e0 con 0x5654154dba30 2023-11-15T03:03:34.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2f9efab700 1 --2- 172.21.15.121:0/204770856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654154dba30 0x5654154ee0e0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7f2f98005d40 tx=0x7f2f98015000).ready entity=mon.0 client_cookie=c5e165f78a587b82 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2f9d7a8700 1 -- 172.21.15.121:0/204770856 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9800d470 con 0x5654154dba30 2023-11-15T03:03:34.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2f9d7a8700 1 -- 172.21.15.121:0/204770856 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9800dd00 con 0x5654154dba30 2023-11-15T03:03:34.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2f9d7a8700 1 -- 172.21.15.121:0/204770856 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9800d970 con 0x5654154dba30 2023-11-15T03:03:34.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2faa146d00 1 -- 172.21.15.121:0/204770856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654154dba30 msgr2=0x5654154ee0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2faa146d00 1 --2- 172.21.15.121:0/204770856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654154dba30 0x5654154ee0e0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7f2f98005d40 tx=0x7f2f98015000).stop 2023-11-15T03:03:34.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2faa146d00 1 -- 172.21.15.121:0/204770856 shutdown_connections 2023-11-15T03:03:34.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2faa146d00 1 --2- 172.21.15.121:0/204770856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654154dba30 0x5654154ee0e0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2faa146d00 1 --2- 172.21.15.121:0/204770856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565415435830 0x56541544db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2faa146d00 1 --2- 172.21.15.121:0/204770856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x565415407450 0x5654154d5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2faa146d00 1 -- 172.21.15.121:0/204770856 >> 172.21.15.121:0/204770856 conn(0x5654154057d0 msgr2=0x565415403ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2faa146d00 1 -- 172.21.15.121:0/204770856 shutdown_connections 2023-11-15T03:03:34.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.613+0000 7f2faa146d00 1 -- 172.21.15.121:0/204770856 wait complete. 2023-11-15T03:03:34.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2faa146d00 1 Processor -- start 2023-11-15T03:03:34.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2faa146d00 1 -- start start 2023-11-15T03:03:34.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2faa146d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565415407450 0x5654154d1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2faa146d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x565415435830 0x5654154d4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2faa146d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654154dba30 0x5654154d5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2faa146d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654154d8690 con 0x565415407450 2023-11-15T03:03:34.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2faa146d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654154d3470 con 0x5654154dba30 2023-11-15T03:03:34.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2faa146d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5654154d2b40 con 0x565415435830 2023-11-15T03:03:34.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2f9e7aa700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565415407450 0x5654154d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2f9e7aa700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565415407450 0x5654154d1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50584/0 (socket says 172.21.15.121:50584) 2023-11-15T03:03:34.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2f9e7aa700 1 -- 172.21.15.121:0/3964268014 learned_addr learned my addr 172.21.15.121:0/3964268014 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2f95fa9700 1 --2- 172.21.15.121:0/3964268014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x565415435830 0x5654154d4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2f9efab700 1 --2- 172.21.15.121:0/3964268014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654154dba30 0x5654154d5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2f9e7aa700 1 -- 172.21.15.121:0/3964268014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x565415435830 msgr2=0x5654154d4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2f9e7aa700 1 --2- 172.21.15.121:0/3964268014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x565415435830 0x5654154d4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2f9e7aa700 1 -- 172.21.15.121:0/3964268014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654154dba30 msgr2=0x5654154d5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2f9e7aa700 1 --2- 172.21.15.121:0/3964268014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654154dba30 0x5654154d5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2f9e7aa700 1 -- 172.21.15.121:0/3964268014 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654153a6a20 con 0x565415407450 2023-11-15T03:03:34.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2f9e7aa700 1 --2- 172.21.15.121:0/3964268014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565415407450 0x5654154d1510 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7f2f9000de90 tx=0x7f2f90000bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2f977fe700 1 -- 172.21.15.121:0/3964268014 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9000bd80 con 0x565415407450 2023-11-15T03:03:34.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2f977fe700 1 -- 172.21.15.121:0/3964268014 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9000f450 con 0x565415407450 2023-11-15T03:03:34.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2f977fe700 1 -- 172.21.15.121:0/3964268014 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9000f7d0 con 0x565415407450 2023-11-15T03:03:34.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.617+0000 7f2faa146d00 1 -- 172.21.15.121:0/3964268014 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5654153a6770 con 0x565415407450 2023-11-15T03:03:34.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.621+0000 7f2faa146d00 1 -- 172.21.15.121:0/3964268014 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565415375a70 con 0x565415407450 2023-11-15T03:03:34.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.621+0000 7f2f977fe700 1 -- 172.21.15.121:0/3964268014 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2f9000fb80 con 0x565415407450 2023-11-15T03:03:34.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.621+0000 7f2f977fe700 1 --2- 172.21.15.121:0/3964268014 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2f84058ff0 0x7f2f8405b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.621+0000 7f2f977fe700 1 -- 172.21.15.121:0/3964268014 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f2f90073850 con 0x565415407450 2023-11-15T03:03:34.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.621+0000 7f2f95fa9700 1 --2- 172.21.15.121:0/3964268014 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2f84058ff0 0x7f2f8405b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.621+0000 7f2faa146d00 1 --2- 172.21.15.121:0/3964268014 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5654154cfc40 0x565415431220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.621+0000 7f2faa146d00 1 -- 172.21.15.121:0/3964268014 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:1 13.30 13:0d82a743:::600.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x5654154e1e20 con 0x5654154cfc40 2023-11-15T03:03:34.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.625+0000 7f2f9efab700 1 --2- 172.21.15.121:0/3964268014 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5654154cfc40 0x565415431220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.625+0000 7f2f95fa9700 1 --2- 172.21.15.121:0/3964268014 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2f84058ff0 0x7f2f8405b4b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f2f88005fd0 tx=0x7f2f8800c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.625+0000 7f2f9efab700 1 --2- 172.21.15.121:0/3964268014 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5654154cfc40 0x565415431220 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.625+0000 7f2f9efab700 1 -- 172.21.15.121:0/3964268014 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(1 600.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x7f2f9800d330 con 0x5654154cfc40 2023-11-15T03:03:34.646 INFO:teuthology.orchestra.run.smithi121.stdout:cephfs_metadata/600.00000000 mtime 2023-11-15T03:03:21.000000+0000, size 0 2023-11-15T03:03:34.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.625+0000 7f2faa146d00 1 -- 172.21.15.121:0/3964268014 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5654154cfc40 msgr2=0x565415431220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.625+0000 7f2faa146d00 1 --2- 172.21.15.121:0/3964268014 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5654154cfc40 0x565415431220 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.625+0000 7f2faa146d00 1 -- 172.21.15.121:0/3964268014 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2f84058ff0 msgr2=0x7f2f8405b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.625+0000 7f2faa146d00 1 --2- 172.21.15.121:0/3964268014 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2f84058ff0 0x7f2f8405b4b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f2f88005fd0 tx=0x7f2f8800c040).stop 2023-11-15T03:03:34.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.625+0000 7f2faa146d00 1 -- 172.21.15.121:0/3964268014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565415407450 msgr2=0x5654154d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.625+0000 7f2faa146d00 1 --2- 172.21.15.121:0/3964268014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565415407450 0x5654154d1510 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7f2f9000de90 tx=0x7f2f90000bc0).stop 2023-11-15T03:03:34.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.625+0000 7f2f9efab700 1 -- 172.21.15.121:0/3964268014 reap_dead start 2023-11-15T03:03:34.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.625+0000 7f2faa146d00 1 -- 172.21.15.121:0/3964268014 shutdown_connections 2023-11-15T03:03:34.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.625+0000 7f2faa146d00 1 -- 172.21.15.121:0/3964268014 >> 172.21.15.121:0/3964268014 conn(0x5654154057d0 msgr2=0x565415403ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.625+0000 7f2faa146d00 1 -- 172.21.15.121:0/3964268014 shutdown_connections 2023-11-15T03:03:34.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.625+0000 7f2faa146d00 1 -- 172.21.15.121:0/3964268014 wait complete. 2023-11-15T03:03:34.650 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 1.00000000.inode 2023-11-15T03:03:34.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.701+0000 7fa13d0f2d00 1 Processor -- start 2023-11-15T03:03:34.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa13d0f2d00 1 -- start start 2023-11-15T03:03:34.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa13d0f2d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c835a0f920 0x55c835adb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa13d0f2d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c835a572f0 0x55c835ae49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa13d0f2d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c835ae5a20 0x55c835aeb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa13d0f2d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8359b9cc0 con 0x55c835a572f0 2023-11-15T03:03:34.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa13d0f2d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c835984220 con 0x55c835ae5a20 2023-11-15T03:03:34.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa13d0f2d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c835ae3f90 con 0x55c835a0f920 2023-11-15T03:03:34.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa132221700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c835a572f0 0x55c835ae49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa132221700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c835a572f0 0x55c835ae49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59288/0 (socket says 172.21.15.121:59288) 2023-11-15T03:03:34.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa132221700 1 -- 172.21.15.121:0/3219334912 learned_addr learned my addr 172.21.15.121:0/3219334912 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa13aa22700 1 --2- 172.21.15.121:0/3219334912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c835a0f920 0x55c835adb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa13b223700 1 --2- 172.21.15.121:0/3219334912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c835ae5a20 0x55c835aeb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa132221700 1 -- 172.21.15.121:0/3219334912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c835a0f920 msgr2=0x55c835adb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa132221700 1 --2- 172.21.15.121:0/3219334912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c835a0f920 0x55c835adb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa132221700 1 -- 172.21.15.121:0/3219334912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c835ae5a20 msgr2=0x55c835aeb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa132221700 1 --2- 172.21.15.121:0/3219334912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c835ae5a20 0x55c835aeb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa132221700 1 -- 172.21.15.121:0/3219334912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c005cf0 con 0x55c835a572f0 2023-11-15T03:03:34.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa13b223700 1 --2- 172.21.15.121:0/3219334912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c835ae5a20 0x55c835aeb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:34.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa132221700 1 --2- 172.21.15.121:0/3219334912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c835a572f0 0x55c835ae49e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7fa124001d60 tx=0x7fa124001830).ready entity=mon.0 client_cookie=812a5f7e851fa61a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa139a20700 1 -- 172.21.15.121:0/3219334912 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12400bae0 con 0x55c835a572f0 2023-11-15T03:03:34.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa139a20700 1 -- 172.21.15.121:0/3219334912 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa124008070 con 0x55c835a572f0 2023-11-15T03:03:34.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa139a20700 1 -- 172.21.15.121:0/3219334912 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12400bc40 con 0x55c835a572f0 2023-11-15T03:03:34.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa13d0f2d00 1 -- 172.21.15.121:0/3219334912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c835a572f0 msgr2=0x55c835ae49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.705+0000 7fa13d0f2d00 1 --2- 172.21.15.121:0/3219334912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c835a572f0 0x55c835ae49e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7fa124001d60 tx=0x7fa124001830).stop 2023-11-15T03:03:34.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa13d0f2d00 1 -- 172.21.15.121:0/3219334912 shutdown_connections 2023-11-15T03:03:34.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa13d0f2d00 1 --2- 172.21.15.121:0/3219334912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c835ae5a20 0x55c835aeb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa13d0f2d00 1 --2- 172.21.15.121:0/3219334912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c835a572f0 0x55c835ae49e0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa13d0f2d00 1 --2- 172.21.15.121:0/3219334912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c835a0f920 0x55c835adb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa13d0f2d00 1 -- 172.21.15.121:0/3219334912 >> 172.21.15.121:0/3219334912 conn(0x55c835a558f0 msgr2=0x55c835a54cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa13d0f2d00 1 -- 172.21.15.121:0/3219334912 shutdown_connections 2023-11-15T03:03:34.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa13d0f2d00 1 -- 172.21.15.121:0/3219334912 wait complete. 2023-11-15T03:03:34.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa13d0f2d00 1 Processor -- start 2023-11-15T03:03:34.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa13d0f2d00 1 -- start start 2023-11-15T03:03:34.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa13d0f2d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c835a0f920 0x55c835ae2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa13d0f2d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c835a572f0 0x55c835ad9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa13d0f2d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c835ae5a20 0x55c835a4fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa13d0f2d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c835ad9a90 con 0x55c835a572f0 2023-11-15T03:03:34.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa13d0f2d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c835ae2f50 con 0x55c835ae5a20 2023-11-15T03:03:34.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa13d0f2d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c835ade640 con 0x55c835a0f920 2023-11-15T03:03:34.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa132221700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c835a572f0 0x55c835ad9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa132221700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c835a572f0 0x55c835ad9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59296/0 (socket says 172.21.15.121:59296) 2023-11-15T03:03:34.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa132221700 1 -- 172.21.15.121:0/4036036006 learned_addr learned my addr 172.21.15.121:0/4036036006 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa13aa22700 1 --2- 172.21.15.121:0/4036036006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c835a0f920 0x55c835ae2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa13b223700 1 --2- 172.21.15.121:0/4036036006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c835ae5a20 0x55c835a4fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa132221700 1 -- 172.21.15.121:0/4036036006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c835a0f920 msgr2=0x55c835ae2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa132221700 1 --2- 172.21.15.121:0/4036036006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c835a0f920 0x55c835ae2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa132221700 1 -- 172.21.15.121:0/4036036006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c835ae5a20 msgr2=0x55c835a4fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa132221700 1 --2- 172.21.15.121:0/4036036006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c835ae5a20 0x55c835a4fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa132221700 1 -- 172.21.15.121:0/4036036006 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12400a040 con 0x55c835a572f0 2023-11-15T03:03:34.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.709+0000 7fa13aa22700 1 --2- 172.21.15.121:0/4036036006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c835a0f920 0x55c835ae2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:03:34.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.713+0000 7fa132221700 1 --2- 172.21.15.121:0/4036036006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c835a572f0 0x55c835ad9eb0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7fa12400f000 tx=0x55c835a51f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.713+0000 7fa133fff700 1 -- 172.21.15.121:0/4036036006 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12401d460 con 0x55c835a572f0 2023-11-15T03:03:34.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.713+0000 7fa133fff700 1 -- 172.21.15.121:0/4036036006 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12401d5c0 con 0x55c835a572f0 2023-11-15T03:03:34.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.713+0000 7fa13d0f2d00 1 -- 172.21.15.121:0/4036036006 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c83597ca70 con 0x55c835a572f0 2023-11-15T03:03:34.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.713+0000 7fa133fff700 1 -- 172.21.15.121:0/4036036006 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12401d940 con 0x55c835a572f0 2023-11-15T03:03:34.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.713+0000 7fa13d0f2d00 1 -- 172.21.15.121:0/4036036006 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c835988010 con 0x55c835a572f0 2023-11-15T03:03:34.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.713+0000 7fa133fff700 1 -- 172.21.15.121:0/4036036006 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa12401c3c0 con 0x55c835a572f0 2023-11-15T03:03:34.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa133fff700 1 --2- 172.21.15.121:0/4036036006 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa11c058ff0 0x7fa11c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa133fff700 1 -- 172.21.15.121:0/4036036006 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fa124074f50 con 0x55c835a572f0 2023-11-15T03:03:34.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa13aa22700 1 --2- 172.21.15.121:0/4036036006 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa11c058ff0 0x7fa11c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa13d0f2d00 1 --2- 172.21.15.121:0/4036036006 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c835a0bda0 0x55c835adcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa13d0f2d00 1 -- 172.21.15.121:0/4036036006 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55c835ade1d0 con 0x55c835a0bda0 2023-11-15T03:03:34.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa13b223700 1 --2- 172.21.15.121:0/4036036006 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c835a0bda0 0x55c835adcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa13aa22700 1 --2- 172.21.15.121:0/4036036006 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa11c058ff0 0x7fa11c05b4b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fa12c00a520 tx=0x7fa12c016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa13b223700 1 --2- 172.21.15.121:0/4036036006 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c835a0bda0 0x55c835adcfd0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa13b223700 1 -- 172.21.15.121:0/4036036006 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x7fa134001630 con 0x55c835a0bda0 2023-11-15T03:03:34.731 INFO:teuthology.orchestra.run.smithi121.stdout:cephfs_metadata/1.00000000.inode mtime 2023-11-15T03:03:30.000000+0000, size 554 2023-11-15T03:03:34.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa13d0f2d00 1 -- 172.21.15.121:0/4036036006 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c835a0bda0 msgr2=0x55c835adcfd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa13d0f2d00 1 --2- 172.21.15.121:0/4036036006 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c835a0bda0 0x55c835adcfd0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa13d0f2d00 1 -- 172.21.15.121:0/4036036006 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa11c058ff0 msgr2=0x7fa11c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa13d0f2d00 1 --2- 172.21.15.121:0/4036036006 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa11c058ff0 0x7fa11c05b4b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fa12c00a520 tx=0x7fa12c016000).stop 2023-11-15T03:03:34.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa13d0f2d00 1 -- 172.21.15.121:0/4036036006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c835a572f0 msgr2=0x55c835ad9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa13d0f2d00 1 --2- 172.21.15.121:0/4036036006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c835a572f0 0x55c835ad9eb0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7fa12400f000 tx=0x55c835a51f50).stop 2023-11-15T03:03:34.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa13b223700 1 -- 172.21.15.121:0/4036036006 reap_dead start 2023-11-15T03:03:34.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa13d0f2d00 1 -- 172.21.15.121:0/4036036006 shutdown_connections 2023-11-15T03:03:34.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa13d0f2d00 1 -- 172.21.15.121:0/4036036006 >> 172.21.15.121:0/4036036006 conn(0x55c835a558f0 msgr2=0x55c835ae3580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa13d0f2d00 1 -- 172.21.15.121:0/4036036006 shutdown_connections 2023-11-15T03:03:34.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.717+0000 7fa13d0f2d00 1 -- 172.21.15.121:0/4036036006 wait complete. 2023-11-15T03:03:34.745 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 200.00000000 2023-11-15T03:03:34.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f5101940d00 1 Processor -- start 2023-11-15T03:03:34.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f5101940d00 1 -- start start 2023-11-15T03:03:34.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f5101940d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cfe132dc0 0x559cfe1a6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f5101940d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cfe1a8650 0x559cfe1a7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f5101940d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cfe1a8a30 0x559cfe1cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f5101940d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cfe05f220 con 0x559cfe132dc0 2023-11-15T03:03:34.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f5101940d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cfe12e430 con 0x559cfe1a8650 2023-11-15T03:03:34.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f5101940d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cfe115ff0 con 0x559cfe1a8a30 2023-11-15T03:03:34.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f50f6045700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cfe132dc0 0x559cfe1a6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f50f6045700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cfe132dc0 0x559cfe1a6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59312/0 (socket says 172.21.15.121:59312) 2023-11-15T03:03:34.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f50f6045700 1 -- 172.21.15.121:0/3998348347 learned_addr learned my addr 172.21.15.121:0/3998348347 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f50f6846700 1 --2- 172.21.15.121:0/3998348347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cfe1a8a30 0x559cfe1cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f50ed844700 1 --2- 172.21.15.121:0/3998348347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cfe1a8650 0x559cfe1a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f50f6045700 1 -- 172.21.15.121:0/3998348347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cfe1a8a30 msgr2=0x559cfe1cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f50f6045700 1 --2- 172.21.15.121:0/3998348347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cfe1a8a30 0x559cfe1cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f50f6045700 1 -- 172.21.15.121:0/3998348347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cfe1a8650 msgr2=0x559cfe1a7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f50f6045700 1 --2- 172.21.15.121:0/3998348347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cfe1a8650 0x559cfe1a7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f50f6045700 1 -- 172.21.15.121:0/3998348347 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f000a0e0 con 0x559cfe132dc0 2023-11-15T03:03:34.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f50f6846700 1 --2- 172.21.15.121:0/3998348347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cfe1a8a30 0x559cfe1cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:34.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f50ed844700 1 --2- 172.21.15.121:0/3998348347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cfe1a8650 0x559cfe1a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:34.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f50f6045700 1 --2- 172.21.15.121:0/3998348347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cfe132dc0 0x559cfe1a6c70 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f50e8001c90 tx=0x7f50e8001770).ready entity=mon.0 client_cookie=8f72273ce115ed51 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f50f5043700 1 -- 172.21.15.121:0/3998348347 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e800bbf0 con 0x559cfe132dc0 2023-11-15T03:03:34.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f50f5043700 1 -- 172.21.15.121:0/3998348347 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e800bd50 con 0x559cfe132dc0 2023-11-15T03:03:34.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.805+0000 7f50f5043700 1 -- 172.21.15.121:0/3998348347 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e800e680 con 0x559cfe132dc0 2023-11-15T03:03:34.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f5101940d00 1 -- 172.21.15.121:0/3998348347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cfe132dc0 msgr2=0x559cfe1a6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f5101940d00 1 --2- 172.21.15.121:0/3998348347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cfe132dc0 0x559cfe1a6c70 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f50e8001c90 tx=0x7f50e8001770).stop 2023-11-15T03:03:34.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f5101940d00 1 -- 172.21.15.121:0/3998348347 shutdown_connections 2023-11-15T03:03:34.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f5101940d00 1 --2- 172.21.15.121:0/3998348347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cfe1a8a30 0x559cfe1cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f5101940d00 1 --2- 172.21.15.121:0/3998348347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cfe1a8650 0x559cfe1a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f5101940d00 1 --2- 172.21.15.121:0/3998348347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cfe132dc0 0x559cfe1a6c70 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f5101940d00 1 -- 172.21.15.121:0/3998348347 >> 172.21.15.121:0/3998348347 conn(0x559cfe1308f0 msgr2=0x559cfe0e6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f5101940d00 1 -- 172.21.15.121:0/3998348347 shutdown_connections 2023-11-15T03:03:34.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f5101940d00 1 -- 172.21.15.121:0/3998348347 wait complete. 2023-11-15T03:03:34.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f5101940d00 1 Processor -- start 2023-11-15T03:03:34.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f5101940d00 1 -- start start 2023-11-15T03:03:34.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f5101940d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cfe132dc0 0x559cfe1be840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f5101940d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cfe1a8650 0x559cfe1a3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f5101940d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cfe1a8a30 0x559cfe0e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f5101940d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cfe1cfb70 con 0x559cfe1a8650 2023-11-15T03:03:34.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f5101940d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cfe0ea340 con 0x559cfe1a8a30 2023-11-15T03:03:34.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f5101940d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cfe129b70 con 0x559cfe132dc0 2023-11-15T03:03:34.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f50ed844700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cfe1a8650 0x559cfe1a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f50ed844700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cfe1a8650 0x559cfe1a3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59326/0 (socket says 172.21.15.121:59326) 2023-11-15T03:03:34.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.809+0000 7f50ed844700 1 -- 172.21.15.121:0/1441509076 learned_addr learned my addr 172.21.15.121:0/1441509076 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.813+0000 7f50f6045700 1 --2- 172.21.15.121:0/1441509076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cfe132dc0 0x559cfe1be840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.813+0000 7f50f6846700 1 --2- 172.21.15.121:0/1441509076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cfe1a8a30 0x559cfe0e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.813+0000 7f50ed844700 1 -- 172.21.15.121:0/1441509076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cfe132dc0 msgr2=0x559cfe1be840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.813+0000 7f50ed844700 1 --2- 172.21.15.121:0/1441509076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cfe132dc0 0x559cfe1be840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.813+0000 7f50ed844700 1 -- 172.21.15.121:0/1441509076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cfe1a8a30 msgr2=0x559cfe0e6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.813+0000 7f50ed844700 1 --2- 172.21.15.121:0/1441509076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cfe1a8a30 0x559cfe0e6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.813+0000 7f50ed844700 1 -- 172.21.15.121:0/1441509076 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e4001e10 con 0x559cfe1a8650 2023-11-15T03:03:34.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.813+0000 7f50f6045700 1 --2- 172.21.15.121:0/1441509076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cfe132dc0 0x559cfe1be840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:34.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.813+0000 7f50f6846700 1 --2- 172.21.15.121:0/1441509076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cfe1a8a30 0x559cfe0e6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:34.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.813+0000 7f50ed844700 1 --2- 172.21.15.121:0/1441509076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cfe1a8650 0x559cfe1a3560 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f50e0008b50 tx=0x7f50e000c040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.813+0000 7f50ef7fe700 1 -- 172.21.15.121:0/1441509076 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e000f030 con 0x559cfe1a8650 2023-11-15T03:03:34.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.813+0000 7f50ef7fe700 1 -- 172.21.15.121:0/1441509076 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e0010040 con 0x559cfe1a8650 2023-11-15T03:03:34.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.813+0000 7f5101940d00 1 -- 172.21.15.121:0/1441509076 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559cfe088770 con 0x559cfe1a8650 2023-11-15T03:03:34.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.813+0000 7f50ef7fe700 1 -- 172.21.15.121:0/1441509076 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0006c50 con 0x559cfe1a8650 2023-11-15T03:03:34.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.813+0000 7f5101940d00 1 -- 172.21.15.121:0/1441509076 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559cfe057a70 con 0x559cfe1a8650 2023-11-15T03:03:34.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.813+0000 7f50ef7fe700 1 -- 172.21.15.121:0/1441509076 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f50e00143c0 con 0x559cfe1a8650 2023-11-15T03:03:34.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f50ef7fe700 1 --2- 172.21.15.121:0/1441509076 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50d8059020 0x7f50d805b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f50ef7fe700 1 -- 172.21.15.121:0/1441509076 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f50e00736c0 con 0x559cfe1a8650 2023-11-15T03:03:34.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f5101940d00 1 --2- 172.21.15.121:0/1441509076 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x559cfe1a3a50 0x559cfe1a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f50f6045700 1 --2- 172.21.15.121:0/1441509076 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50d8059020 0x7f50d805b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f5101940d00 1 -- 172.21.15.121:0/1441509076 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x559cfe1a5b90 con 0x559cfe1a3a50 2023-11-15T03:03:34.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f50f6846700 1 --2- 172.21.15.121:0/1441509076 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x559cfe1a3a50 0x559cfe1a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f50f6045700 1 --2- 172.21.15.121:0/1441509076 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50d8059020 0x7f50d805b4e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f50e80017d0 tx=0x7f50e8014000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f50f6846700 1 --2- 172.21.15.121:0/1441509076 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x559cfe1a3a50 0x559cfe1a56a0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f50f6846700 1 -- 172.21.15.121:0/1441509076 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(1 200.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x7f50f000dd00 con 0x559cfe1a3a50 2023-11-15T03:03:34.845 INFO:teuthology.orchestra.run.smithi121.stdout:cephfs_metadata/200.00000000 mtime 2023-11-15T03:03:30.000000+0000, size 90 2023-11-15T03:03:34.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f5101940d00 1 -- 172.21.15.121:0/1441509076 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x559cfe1a3a50 msgr2=0x559cfe1a56a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f5101940d00 1 --2- 172.21.15.121:0/1441509076 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x559cfe1a3a50 0x559cfe1a56a0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f5101940d00 1 -- 172.21.15.121:0/1441509076 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50d8059020 msgr2=0x7f50d805b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f5101940d00 1 --2- 172.21.15.121:0/1441509076 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50d8059020 0x7f50d805b4e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f50e80017d0 tx=0x7f50e8014000).stop 2023-11-15T03:03:34.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f5101940d00 1 -- 172.21.15.121:0/1441509076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cfe1a8650 msgr2=0x559cfe1a3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f5101940d00 1 --2- 172.21.15.121:0/1441509076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cfe1a8650 0x559cfe1a3560 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f50e0008b50 tx=0x7f50e000c040).stop 2023-11-15T03:03:34.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f50f6846700 1 -- 172.21.15.121:0/1441509076 reap_dead start 2023-11-15T03:03:34.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f5101940d00 1 -- 172.21.15.121:0/1441509076 shutdown_connections 2023-11-15T03:03:34.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f5101940d00 1 -- 172.21.15.121:0/1441509076 >> 172.21.15.121:0/1441509076 conn(0x559cfe1308f0 msgr2=0x559cfe1196d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f5101940d00 1 -- 172.21.15.121:0/1441509076 shutdown_connections 2023-11-15T03:03:34.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.817+0000 7f5101940d00 1 -- 172.21.15.121:0/1441509076 wait complete. 2023-11-15T03:03:34.854 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 200.00000001 2023-11-15T03:03:34.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.913+0000 7fe74c93dd00 1 Processor -- start 2023-11-15T03:03:34.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.913+0000 7fe74c93dd00 1 -- start start 2023-11-15T03:03:34.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.913+0000 7fe74c93dd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564579453bb0 0x564579525e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.913+0000 7fe74c93dd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56457949db00 0x564579526770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.913+0000 7fe74c93dd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564579530650 0x56457952e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.913+0000 7fe74c93dd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645793cc220 con 0x564579530650 2023-11-15T03:03:34.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.913+0000 7fe74c93dd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564579486690 con 0x564579453bb0 2023-11-15T03:03:34.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.913+0000 7fe74c93dd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645794882e0 con 0x56457949db00 2023-11-15T03:03:34.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.913+0000 7fe7417ce700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564579530650 0x56457952e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.913+0000 7fe7417ce700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564579530650 0x56457952e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59334/0 (socket says 172.21.15.121:59334) 2023-11-15T03:03:34.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.913+0000 7fe7417ce700 1 -- 172.21.15.121:0/2178973453 learned_addr learned my addr 172.21.15.121:0/2178973453 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.913+0000 7fe740fcd700 1 --2- 172.21.15.121:0/2178973453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564579453bb0 0x564579525e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe7387cc700 1 --2- 172.21.15.121:0/2178973453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56457949db00 0x564579526770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe7417ce700 1 -- 172.21.15.121:0/2178973453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56457949db00 msgr2=0x564579526770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe7417ce700 1 --2- 172.21.15.121:0/2178973453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56457949db00 0x564579526770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe7417ce700 1 -- 172.21.15.121:0/2178973453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564579453bb0 msgr2=0x564579525e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:34.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe7417ce700 1 --2- 172.21.15.121:0/2178973453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564579453bb0 0x564579525e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe7417ce700 1 -- 172.21.15.121:0/2178973453 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe728005cf0 con 0x564579530650 2023-11-15T03:03:34.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe7417ce700 1 --2- 172.21.15.121:0/2178973453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564579530650 0x56457952e1b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7fe73c010000 tx=0x7fe73c016000).ready entity=mon.0 client_cookie=57e7fde61ffd7740 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe73b7fe700 1 -- 172.21.15.121:0/2178973453 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c008aa0 con 0x564579530650 2023-11-15T03:03:34.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe73b7fe700 1 -- 172.21.15.121:0/2178973453 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73c005350 con 0x564579530650 2023-11-15T03:03:34.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe73b7fe700 1 -- 172.21.15.121:0/2178973453 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c0054e0 con 0x564579530650 2023-11-15T03:03:34.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe74c93dd00 1 -- 172.21.15.121:0/2178973453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564579530650 msgr2=0x56457952e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe74c93dd00 1 --2- 172.21.15.121:0/2178973453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564579530650 0x56457952e1b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7fe73c010000 tx=0x7fe73c016000).stop 2023-11-15T03:03:34.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe74c93dd00 1 -- 172.21.15.121:0/2178973453 shutdown_connections 2023-11-15T03:03:34.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe74c93dd00 1 --2- 172.21.15.121:0/2178973453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564579530650 0x56457952e1b0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe74c93dd00 1 --2- 172.21.15.121:0/2178973453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56457949db00 0x564579526770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe74c93dd00 1 --2- 172.21.15.121:0/2178973453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564579453bb0 0x564579525e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe74c93dd00 1 -- 172.21.15.121:0/2178973453 >> 172.21.15.121:0/2178973453 conn(0x5645794a3080 msgr2=0x564579488760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe74c93dd00 1 -- 172.21.15.121:0/2178973453 shutdown_connections 2023-11-15T03:03:34.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe74c93dd00 1 -- 172.21.15.121:0/2178973453 wait complete. 2023-11-15T03:03:34.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe74c93dd00 1 Processor -- start 2023-11-15T03:03:34.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.917+0000 7fe74c93dd00 1 -- start start 2023-11-15T03:03:34.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe74c93dd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564579453bb0 0x5645794a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe74c93dd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56457949db00 0x564579524c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe74c93dd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564579530650 0x564579488500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe74c93dd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457953cb50 con 0x56457949db00 2023-11-15T03:03:34.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe74c93dd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457953ce30 con 0x564579530650 2023-11-15T03:03:34.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe74c93dd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56457953d110 con 0x564579453bb0 2023-11-15T03:03:34.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe7387cc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56457949db00 0x564579524c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe7387cc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56457949db00 0x564579524c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59350/0 (socket says 172.21.15.121:59350) 2023-11-15T03:03:34.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe7387cc700 1 -- 172.21.15.121:0/1897797189 learned_addr learned my addr 172.21.15.121:0/1897797189 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:34.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe740fcd700 1 --2- 172.21.15.121:0/1897797189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564579453bb0 0x5645794a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe7417ce700 1 --2- 172.21.15.121:0/1897797189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564579530650 0x564579488500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe7387cc700 1 -- 172.21.15.121:0/1897797189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564579453bb0 msgr2=0x5645794a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe7387cc700 1 --2- 172.21.15.121:0/1897797189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564579453bb0 0x5645794a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe7387cc700 1 -- 172.21.15.121:0/1897797189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564579530650 msgr2=0x564579488500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe7387cc700 1 --2- 172.21.15.121:0/1897797189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564579530650 0x564579488500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe7387cc700 1 -- 172.21.15.121:0/1897797189 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645793f5a20 con 0x56457949db00 2023-11-15T03:03:34.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe7387cc700 1 --2- 172.21.15.121:0/1897797189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56457949db00 0x564579524c70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7fe728008b20 tx=0x7fe72800c040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe739ffb700 1 -- 172.21.15.121:0/1897797189 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72800f030 con 0x56457949db00 2023-11-15T03:03:34.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe739ffb700 1 -- 172.21.15.121:0/1897797189 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe728010040 con 0x56457949db00 2023-11-15T03:03:34.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe74c93dd00 1 -- 172.21.15.121:0/1897797189 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe73c005db0 con 0x56457949db00 2023-11-15T03:03:34.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe739ffb700 1 -- 172.21.15.121:0/1897797189 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe728006cb0 con 0x56457949db00 2023-11-15T03:03:34.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.921+0000 7fe74c93dd00 1 -- 172.21.15.121:0/1897797189 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645793f5770 con 0x56457949db00 2023-11-15T03:03:34.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.925+0000 7fe739ffb700 1 -- 172.21.15.121:0/1897797189 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe728015670 con 0x56457949db00 2023-11-15T03:03:34.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.925+0000 7fe739ffb700 1 --2- 172.21.15.121:0/1897797189 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe720059020 0x7fe72005b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.925+0000 7fe739ffb700 1 -- 172.21.15.121:0/1897797189 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fe728073630 con 0x56457949db00 2023-11-15T03:03:34.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.925+0000 7fe740fcd700 1 --2- 172.21.15.121:0/1897797189 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe720059020 0x7fe72005b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.925+0000 7fe74c93dd00 1 --2- 172.21.15.121:0/1897797189 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56457949a480 0x56457949c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.925+0000 7fe74c93dd00 1 -- 172.21.15.121:0/1897797189 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.34 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x564579525160 con 0x56457949a480 2023-11-15T03:03:34.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.925+0000 7fe740fcd700 1 --2- 172.21.15.121:0/1897797189 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe720059020 0x7fe72005b4e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7fe730006fd0 tx=0x7fe730008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.929+0000 7fe7417ce700 1 --2- 172.21.15.121:0/1897797189 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56457949a480 0x56457949c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:34.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.929+0000 7fe7417ce700 1 --2- 172.21.15.121:0/1897797189 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56457949a480 0x56457949c130 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:34.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.929+0000 7fe7417ce700 1 -- 172.21.15.121:0/1897797189 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 200.00000001 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x7fe73c022020 con 0x56457949a480 2023-11-15T03:03:34.937 INFO:teuthology.orchestra.run.smithi121.stdout:cephfs_metadata/200.00000001 mtime 2023-11-15T03:03:30.000000+0000, size 16484 2023-11-15T03:03:34.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.929+0000 7fe74c93dd00 1 -- 172.21.15.121:0/1897797189 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56457949a480 msgr2=0x56457949c130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.929+0000 7fe74c93dd00 1 --2- 172.21.15.121:0/1897797189 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56457949a480 0x56457949c130 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:34.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.929+0000 7fe74c93dd00 1 -- 172.21.15.121:0/1897797189 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe720059020 msgr2=0x7fe72005b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.929+0000 7fe74c93dd00 1 --2- 172.21.15.121:0/1897797189 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe720059020 0x7fe72005b4e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7fe730006fd0 tx=0x7fe730008040).stop 2023-11-15T03:03:34.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.929+0000 7fe74c93dd00 1 -- 172.21.15.121:0/1897797189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56457949db00 msgr2=0x564579524c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:34.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.929+0000 7fe74c93dd00 1 --2- 172.21.15.121:0/1897797189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56457949db00 0x564579524c70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7fe728008b20 tx=0x7fe72800c040).stop 2023-11-15T03:03:34.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.929+0000 7fe7417ce700 1 -- 172.21.15.121:0/1897797189 reap_dead start 2023-11-15T03:03:34.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.929+0000 7fe74c93dd00 1 -- 172.21.15.121:0/1897797189 shutdown_connections 2023-11-15T03:03:34.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.929+0000 7fe74c93dd00 1 -- 172.21.15.121:0/1897797189 >> 172.21.15.121:0/1897797189 conn(0x5645794a3080 msgr2=0x564579525bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:34.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.929+0000 7fe74c93dd00 1 -- 172.21.15.121:0/1897797189 shutdown_connections 2023-11-15T03:03:34.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.929+0000 7fe74c93dd00 1 -- 172.21.15.121:0/1897797189 wait complete. 2023-11-15T03:03:34.941 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_openfiles.0 2023-11-15T03:03:34.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.993+0000 7fb68195bd00 1 Processor -- start 2023-11-15T03:03:34.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.993+0000 7fb68195bd00 1 -- start start 2023-11-15T03:03:34.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.993+0000 7fb68195bd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fe8fe508d0 0x55fe8fe081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.993+0000 7fb68195bd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe8fec6650 0x55fe8fedce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:34.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.993+0000 7fb68195bd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe8fec6a30 0x55fe8fe4fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.993+0000 7fb68195bd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8fd7d220 con 0x55fe8fec6a30 2023-11-15T03:03:35.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.993+0000 7fb68195bd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8fe50d60 con 0x55fe8fec6650 2023-11-15T03:03:35.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.993+0000 7fb68195bd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe8fe04d30 con 0x55fe8fe508d0 2023-11-15T03:03:35.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.993+0000 7fb67fa46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe8fec6a30 0x55fe8fe4fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.993+0000 7fb67fa46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe8fec6a30 0x55fe8fe4fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59370/0 (socket says 172.21.15.121:59370) 2023-11-15T03:03:35.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.993+0000 7fb67fa46700 1 -- 172.21.15.121:0/1440746114 learned_addr learned my addr 172.21.15.121:0/1440746114 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:35.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.993+0000 7fb67f245700 1 --2- 172.21.15.121:0/1440746114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fe8fe508d0 0x55fe8fe081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.993+0000 7fb676a44700 1 --2- 172.21.15.121:0/1440746114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe8fec6650 0x55fe8fedce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb67fa46700 1 -- 172.21.15.121:0/1440746114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fe8fe508d0 msgr2=0x55fe8fe081b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:35.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb67fa46700 1 --2- 172.21.15.121:0/1440746114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fe8fe508d0 0x55fe8fe081b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb67fa46700 1 -- 172.21.15.121:0/1440746114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe8fec6650 msgr2=0x55fe8fedce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:35.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb67fa46700 1 --2- 172.21.15.121:0/1440746114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe8fec6650 0x55fe8fedce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb67fa46700 1 -- 172.21.15.121:0/1440746114 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67000a180 con 0x55fe8fec6a30 2023-11-15T03:03:35.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb67f245700 1 --2- 172.21.15.121:0/1440746114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fe8fe508d0 0x55fe8fe081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:35.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb67fa46700 1 --2- 172.21.15.121:0/1440746114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe8fec6a30 0x55fe8fe4fe20 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7fb678001cb0 tx=0x7fb678001780).ready entity=mon.0 client_cookie=858955ad608c9f86 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb67e243700 1 -- 172.21.15.121:0/1440746114 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67800bb70 con 0x55fe8fec6a30 2023-11-15T03:03:35.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb67e243700 1 -- 172.21.15.121:0/1440746114 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb678008070 con 0x55fe8fec6a30 2023-11-15T03:03:35.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb67e243700 1 -- 172.21.15.121:0/1440746114 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67800bcd0 con 0x55fe8fec6a30 2023-11-15T03:03:35.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb68195bd00 1 -- 172.21.15.121:0/1440746114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe8fec6a30 msgr2=0x55fe8fe4fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb68195bd00 1 --2- 172.21.15.121:0/1440746114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe8fec6a30 0x55fe8fe4fe20 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7fb678001cb0 tx=0x7fb678001780).stop 2023-11-15T03:03:35.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb68195bd00 1 -- 172.21.15.121:0/1440746114 shutdown_connections 2023-11-15T03:03:35.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb68195bd00 1 --2- 172.21.15.121:0/1440746114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe8fec6a30 0x55fe8fe4fe20 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb68195bd00 1 --2- 172.21.15.121:0/1440746114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe8fec6650 0x55fe8fedce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb68195bd00 1 --2- 172.21.15.121:0/1440746114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fe8fe508d0 0x55fe8fe081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb68195bd00 1 -- 172.21.15.121:0/1440746114 >> 172.21.15.121:0/1440746114 conn(0x55fe8fe4e8f0 msgr2=0x55fe8fe09820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:35.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb68195bd00 1 -- 172.21.15.121:0/1440746114 shutdown_connections 2023-11-15T03:03:35.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb68195bd00 1 -- 172.21.15.121:0/1440746114 wait complete. 2023-11-15T03:03:35.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:34.997+0000 7fb68195bd00 1 Processor -- start 2023-11-15T03:03:35.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.001+0000 7fb68195bd00 1 -- start start 2023-11-15T03:03:35.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.001+0000 7fb68195bd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fe8fe508d0 0x55fe8febd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.001+0000 7fb68195bd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe8fec6650 0x55fe8fe36c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.001+0000 7fb68195bd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe8fec6a30 0x55fe8fef1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.001+0000 7fb68195bd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8fe4a790 con 0x55fe8fec6650 2023-11-15T03:03:35.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.001+0000 7fb68195bd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8fe49ec0 con 0x55fe8fec6a30 2023-11-15T03:03:35.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.001+0000 7fb68195bd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe8fe37140 con 0x55fe8fe508d0 2023-11-15T03:03:35.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.001+0000 7fb676a44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe8fec6650 0x55fe8fe36c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.001+0000 7fb676a44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe8fec6650 0x55fe8fe36c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59378/0 (socket says 172.21.15.121:59378) 2023-11-15T03:03:35.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.001+0000 7fb676a44700 1 -- 172.21.15.121:0/341452967 learned_addr learned my addr 172.21.15.121:0/341452967 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:35.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.001+0000 7fb67fa46700 1 --2- 172.21.15.121:0/341452967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe8fec6a30 0x55fe8fef1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.001+0000 7fb67f245700 1 --2- 172.21.15.121:0/341452967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fe8fe508d0 0x55fe8febd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.001+0000 7fb676a44700 1 -- 172.21.15.121:0/341452967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fe8fe508d0 msgr2=0x55fe8febd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.001+0000 7fb676a44700 1 --2- 172.21.15.121:0/341452967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fe8fe508d0 0x55fe8febd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.001+0000 7fb676a44700 1 -- 172.21.15.121:0/341452967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe8fec6a30 msgr2=0x55fe8fef1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.001+0000 7fb676a44700 1 --2- 172.21.15.121:0/341452967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe8fec6a30 0x55fe8fef1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.001+0000 7fb676a44700 1 -- 172.21.15.121:0/341452967 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67800a040 con 0x55fe8fec6650 2023-11-15T03:03:35.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.001+0000 7fb676a44700 1 --2- 172.21.15.121:0/341452967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe8fec6650 0x55fe8fe36c50 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7fb668008af0 tx=0x7fb66800c040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.005+0000 7fb67ca40700 1 -- 172.21.15.121:0/341452967 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66800f040 con 0x55fe8fec6650 2023-11-15T03:03:35.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.005+0000 7fb67ca40700 1 -- 172.21.15.121:0/341452967 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6680094e0 con 0x55fe8fec6650 2023-11-15T03:03:35.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.005+0000 7fb68195bd00 1 -- 172.21.15.121:0/341452967 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fe8fda6770 con 0x55fe8fec6650 2023-11-15T03:03:35.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.005+0000 7fb67ca40700 1 -- 172.21.15.121:0/341452967 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668010ac0 con 0x55fe8fec6650 2023-11-15T03:03:35.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.005+0000 7fb68195bd00 1 -- 172.21.15.121:0/341452967 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe8fd75a70 con 0x55fe8fec6650 2023-11-15T03:03:35.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.005+0000 7fb67ca40700 1 -- 172.21.15.121:0/341452967 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb668015430 con 0x55fe8fec6650 2023-11-15T03:03:35.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.005+0000 7fb67ca40700 1 --2- 172.21.15.121:0/341452967 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb660059040 0x7fb66005b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.009+0000 7fb67ca40700 1 -- 172.21.15.121:0/341452967 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fb668073db0 con 0x55fe8fec6650 2023-11-15T03:03:35.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.009+0000 7fb67f245700 1 --2- 172.21.15.121:0/341452967 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb660059040 0x7fb66005b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.009+0000 7fb68195bd00 1 --2- 172.21.15.121:0/341452967 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55fe8fef5310 0x55fe8fef5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.009+0000 7fb68195bd00 1 -- 172.21.15.121:0/341452967 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 13.26 13:654134d2:::mds0_openfiles.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55fe8fef1a40 con 0x55fe8fef5310 2023-11-15T03:03:35.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.009+0000 7fb67fa46700 1 --2- 172.21.15.121:0/341452967 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55fe8fef5310 0x55fe8fef5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.009+0000 7fb67f245700 1 --2- 172.21.15.121:0/341452967 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb660059040 0x7fb66005b500 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7fb670006fd0 tx=0x7fb670014000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.009+0000 7fb67fa46700 1 --2- 172.21.15.121:0/341452967 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55fe8fef5310 0x55fe8fef5720 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.009+0000 7fb67fa46700 1 -- 172.21.15.121:0/341452967 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 mds0_openfiles.0 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x7fb67800f610 con 0x55fe8fef5310 2023-11-15T03:03:35.022 INFO:teuthology.orchestra.run.smithi121.stdout:cephfs_metadata/mds0_openfiles.0 mtime 2023-11-15T03:03:30.000000+0000, size 0 2023-11-15T03:03:35.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.009+0000 7fb68195bd00 1 -- 172.21.15.121:0/341452967 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55fe8fef5310 msgr2=0x55fe8fef5720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.009+0000 7fb68195bd00 1 --2- 172.21.15.121:0/341452967 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55fe8fef5310 0x55fe8fef5720 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.009+0000 7fb68195bd00 1 -- 172.21.15.121:0/341452967 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb660059040 msgr2=0x7fb66005b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.009+0000 7fb68195bd00 1 --2- 172.21.15.121:0/341452967 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb660059040 0x7fb66005b500 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7fb670006fd0 tx=0x7fb670014000).stop 2023-11-15T03:03:35.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.009+0000 7fb68195bd00 1 -- 172.21.15.121:0/341452967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe8fec6650 msgr2=0x55fe8fe36c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.009+0000 7fb68195bd00 1 --2- 172.21.15.121:0/341452967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe8fec6650 0x55fe8fe36c50 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7fb668008af0 tx=0x7fb66800c040).stop 2023-11-15T03:03:35.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.009+0000 7fb67fa46700 1 -- 172.21.15.121:0/341452967 reap_dead start 2023-11-15T03:03:35.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.009+0000 7fb68195bd00 1 -- 172.21.15.121:0/341452967 shutdown_connections 2023-11-15T03:03:35.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.009+0000 7fb68195bd00 1 -- 172.21.15.121:0/341452967 >> 172.21.15.121:0/341452967 conn(0x55fe8fe4e8f0 msgr2=0x55fe8fe09820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:35.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.009+0000 7fb68195bd00 1 -- 172.21.15.121:0/341452967 shutdown_connections 2023-11-15T03:03:35.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.009+0000 7fb68195bd00 1 -- 172.21.15.121:0/341452967 wait complete. 2023-11-15T03:03:35.027 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 500.00000000 2023-11-15T03:03:35.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.081+0000 7f0eecb3fd00 1 Processor -- start 2023-11-15T03:03:35.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.081+0000 7f0eecb3fd00 1 -- start start 2023-11-15T03:03:35.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eecb3fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56236550c920 0x56236553b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eecb3fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5623655542f0 0x56236554cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eecb3fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623655e2a20 0x56236554d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eecb3fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562365481220 con 0x56236550c920 2023-11-15T03:03:35.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eecb3fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623655d8d50 con 0x5623655e2a20 2023-11-15T03:03:35.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eecb3fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56236553b250 con 0x5623655542f0 2023-11-15T03:03:35.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eea445700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56236550c920 0x56236553b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eea445700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56236550c920 0x56236553b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59382/0 (socket says 172.21.15.121:59382) 2023-11-15T03:03:35.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eea445700 1 -- 172.21.15.121:0/3768771333 learned_addr learned my addr 172.21.15.121:0/3768771333 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:35.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eeac46700 1 --2- 172.21.15.121:0/3768771333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623655e2a20 0x56236554d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0ee1c44700 1 --2- 172.21.15.121:0/3768771333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5623655542f0 0x56236554cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eea445700 1 -- 172.21.15.121:0/3768771333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5623655542f0 msgr2=0x56236554cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:35.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eea445700 1 --2- 172.21.15.121:0/3768771333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5623655542f0 0x56236554cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eea445700 1 -- 172.21.15.121:0/3768771333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623655e2a20 msgr2=0x56236554d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:35.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eea445700 1 --2- 172.21.15.121:0/3768771333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623655e2a20 0x56236554d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eea445700 1 -- 172.21.15.121:0/3768771333 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee400a0e0 con 0x56236550c920 2023-11-15T03:03:35.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eeac46700 1 --2- 172.21.15.121:0/3768771333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623655e2a20 0x56236554d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:03:35.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eea445700 1 --2- 172.21.15.121:0/3768771333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56236550c920 0x56236553b6e0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f0edc00ac40 tx=0x7f0edc015000).ready entity=mon.0 client_cookie=b0a1f7fe70bbad70 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0ee9443700 1 -- 172.21.15.121:0/3768771333 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc007610 con 0x56236550c920 2023-11-15T03:03:35.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0ee9443700 1 -- 172.21.15.121:0/3768771333 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0edc005330 con 0x56236550c920 2023-11-15T03:03:35.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0ee9443700 1 -- 172.21.15.121:0/3768771333 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc0078e0 con 0x56236550c920 2023-11-15T03:03:35.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eecb3fd00 1 -- 172.21.15.121:0/3768771333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56236550c920 msgr2=0x56236553b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eecb3fd00 1 --2- 172.21.15.121:0/3768771333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56236550c920 0x56236553b6e0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f0edc00ac40 tx=0x7f0edc015000).stop 2023-11-15T03:03:35.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eecb3fd00 1 -- 172.21.15.121:0/3768771333 shutdown_connections 2023-11-15T03:03:35.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eecb3fd00 1 --2- 172.21.15.121:0/3768771333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623655e2a20 0x56236554d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eecb3fd00 1 --2- 172.21.15.121:0/3768771333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5623655542f0 0x56236554cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eecb3fd00 1 --2- 172.21.15.121:0/3768771333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56236550c920 0x56236553b6e0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eecb3fd00 1 -- 172.21.15.121:0/3768771333 >> 172.21.15.121:0/3768771333 conn(0x5623655528f0 msgr2=0x5623655e0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:35.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eecb3fd00 1 -- 172.21.15.121:0/3768771333 shutdown_connections 2023-11-15T03:03:35.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.085+0000 7f0eecb3fd00 1 -- 172.21.15.121:0/3768771333 wait complete. 2023-11-15T03:03:35.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.089+0000 7f0eecb3fd00 1 Processor -- start 2023-11-15T03:03:35.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.089+0000 7f0eecb3fd00 1 -- start start 2023-11-15T03:03:35.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.089+0000 7f0eecb3fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56236550c920 0x56236554fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.089+0000 7f0eecb3fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623655542f0 0x5623655d80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.089+0000 7f0eecb3fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623655e2a20 0x56236550af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.089+0000 7f0eecb3fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236550aac0 con 0x5623655542f0 2023-11-15T03:03:35.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.089+0000 7f0eecb3fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236554def0 con 0x5623655e2a20 2023-11-15T03:03:35.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.089+0000 7f0eecb3fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56236554e1d0 con 0x56236550c920 2023-11-15T03:03:35.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.089+0000 7f0ee1c44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623655542f0 0x5623655d80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.089+0000 7f0ee1c44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623655542f0 0x5623655d80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59392/0 (socket says 172.21.15.121:59392) 2023-11-15T03:03:35.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.089+0000 7f0ee1c44700 1 -- 172.21.15.121:0/548624206 learned_addr learned my addr 172.21.15.121:0/548624206 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:35.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.089+0000 7f0eeac46700 1 --2- 172.21.15.121:0/548624206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623655e2a20 0x56236550af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.089+0000 7f0eea445700 1 --2- 172.21.15.121:0/548624206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56236550c920 0x56236554fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.089+0000 7f0ee1c44700 1 -- 172.21.15.121:0/548624206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56236550c920 msgr2=0x56236554fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.089+0000 7f0ee1c44700 1 --2- 172.21.15.121:0/548624206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56236550c920 0x56236554fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.089+0000 7f0ee1c44700 1 -- 172.21.15.121:0/548624206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623655e2a20 msgr2=0x56236550af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.089+0000 7f0ee1c44700 1 --2- 172.21.15.121:0/548624206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623655e2a20 0x56236550af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.089+0000 7f0ee1c44700 1 -- 172.21.15.121:0/548624206 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee400a520 con 0x5623655542f0 2023-11-15T03:03:35.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.093+0000 7f0ee1c44700 1 --2- 172.21.15.121:0/548624206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623655542f0 0x5623655d80b0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f0ed400de30 tx=0x7f0ed4001840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.093+0000 7f0ee37fe700 1 -- 172.21.15.121:0/548624206 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed400bc50 con 0x5623655542f0 2023-11-15T03:03:35.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.093+0000 7f0ee37fe700 1 -- 172.21.15.121:0/548624206 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed400bdb0 con 0x5623655542f0 2023-11-15T03:03:35.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.093+0000 7f0ee37fe700 1 -- 172.21.15.121:0/548624206 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed400f660 con 0x5623655542f0 2023-11-15T03:03:35.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.093+0000 7f0eecb3fd00 1 -- 172.21.15.121:0/548624206 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5623654aaa20 con 0x5623655542f0 2023-11-15T03:03:35.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.093+0000 7f0eecb3fd00 1 -- 172.21.15.121:0/548624206 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0edc005cf0 con 0x5623655542f0 2023-11-15T03:03:35.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.093+0000 7f0ee37fe700 1 -- 172.21.15.121:0/548624206 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0ed400f9d0 con 0x5623655542f0 2023-11-15T03:03:35.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.097+0000 7f0ee37fe700 1 --2- 172.21.15.121:0/548624206 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0ecc058ff0 0x7f0ecc05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.097+0000 7f0ee37fe700 1 -- 172.21.15.121:0/548624206 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f0ed4073b80 con 0x5623655542f0 2023-11-15T03:03:35.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.097+0000 7f0eecb3fd00 1 --2- 172.21.15.121:0/548624206 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56236554d9f0 0x5623655e8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.097+0000 7f0eecb3fd00 1 -- 172.21.15.121:0/548624206 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x5623655e0e40 con 0x56236554d9f0 2023-11-15T03:03:35.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.097+0000 7f0eea445700 1 --2- 172.21.15.121:0/548624206 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0ecc058ff0 0x7f0ecc05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.097+0000 7f0eeac46700 1 --2- 172.21.15.121:0/548624206 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56236554d9f0 0x5623655e8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.097+0000 7f0eea445700 1 --2- 172.21.15.121:0/548624206 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0ecc058ff0 0x7f0ecc05b4b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f0edc00ac40 tx=0x7f0edc01a000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.097+0000 7f0eeac46700 1 --2- 172.21.15.121:0/548624206 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56236554d9f0 0x5623655e8820 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.097+0000 7f0eeac46700 1 -- 172.21.15.121:0/548624206 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(1 500.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x7f0ee4014420 con 0x56236554d9f0 2023-11-15T03:03:35.113 INFO:teuthology.orchestra.run.smithi121.stdout:cephfs_metadata/500.00000000 mtime 2023-11-15T03:03:21.000000+0000, size 90 2023-11-15T03:03:35.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.097+0000 7f0eecb3fd00 1 -- 172.21.15.121:0/548624206 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56236554d9f0 msgr2=0x5623655e8820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.097+0000 7f0eecb3fd00 1 --2- 172.21.15.121:0/548624206 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56236554d9f0 0x5623655e8820 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.101+0000 7f0eecb3fd00 1 -- 172.21.15.121:0/548624206 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0ecc058ff0 msgr2=0x7f0ecc05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.101+0000 7f0eecb3fd00 1 --2- 172.21.15.121:0/548624206 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0ecc058ff0 0x7f0ecc05b4b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f0edc00ac40 tx=0x7f0edc01a000).stop 2023-11-15T03:03:35.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.101+0000 7f0eecb3fd00 1 -- 172.21.15.121:0/548624206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623655542f0 msgr2=0x5623655d80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.101+0000 7f0eecb3fd00 1 --2- 172.21.15.121:0/548624206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623655542f0 0x5623655d80b0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f0ed400de30 tx=0x7f0ed4001840).stop 2023-11-15T03:03:35.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.101+0000 7f0eeac46700 1 -- 172.21.15.121:0/548624206 reap_dead start 2023-11-15T03:03:35.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.101+0000 7f0eecb3fd00 1 -- 172.21.15.121:0/548624206 shutdown_connections 2023-11-15T03:03:35.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.101+0000 7f0eecb3fd00 1 -- 172.21.15.121:0/548624206 >> 172.21.15.121:0/548624206 conn(0x5623655528f0 msgr2=0x5623655e0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:35.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.101+0000 7f0eecb3fd00 1 -- 172.21.15.121:0/548624206 shutdown_connections 2023-11-15T03:03:35.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.101+0000 7f0eecb3fd00 1 -- 172.21.15.121:0/548624206 wait complete. 2023-11-15T03:03:35.124 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds_snaptable 2023-11-15T03:03:35.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.181+0000 7fdc4c4d8d00 1 Processor -- start 2023-11-15T03:03:35.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.181+0000 7fdc4c4d8d00 1 -- start start 2023-11-15T03:03:35.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.181+0000 7fdc4c4d8d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a71e7cc6c0 0x55a71e89ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.181+0000 7fdc4c4d8d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a71e816b00 0x55a71e811ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.181+0000 7fdc4c4d8d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a71e8a5670 0x55a71e8124e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.181+0000 7fdc4c4d8d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71e745220 con 0x55a71e8a5670 2023-11-15T03:03:35.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.181+0000 7fdc4c4d8d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71e7ff700 con 0x55a71e816b00 2023-11-15T03:03:35.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.181+0000 7fdc4c4d8d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a71e8129d0 con 0x55a71e7cc6c0 2023-11-15T03:03:35.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.181+0000 7fdc4a5bb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a71e8a5670 0x55a71e8124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.181+0000 7fdc4a5bb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a71e8a5670 0x55a71e8124e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59402/0 (socket says 172.21.15.121:59402) 2023-11-15T03:03:35.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.181+0000 7fdc4a5bb700 1 -- 172.21.15.121:0/1125473452 learned_addr learned my addr 172.21.15.121:0/1125473452 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:35.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.181+0000 7fdc415b9700 1 --2- 172.21.15.121:0/1125473452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a71e816b00 0x55a71e811ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.181+0000 7fdc49dba700 1 --2- 172.21.15.121:0/1125473452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a71e7cc6c0 0x55a71e89ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.181+0000 7fdc4a5bb700 1 -- 172.21.15.121:0/1125473452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a71e7cc6c0 msgr2=0x55a71e89ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:35.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.181+0000 7fdc4a5bb700 1 --2- 172.21.15.121:0/1125473452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a71e7cc6c0 0x55a71e89ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.181+0000 7fdc4a5bb700 1 -- 172.21.15.121:0/1125473452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a71e816b00 msgr2=0x55a71e811ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:35.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.181+0000 7fdc4a5bb700 1 --2- 172.21.15.121:0/1125473452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a71e816b00 0x55a71e811ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.181+0000 7fdc4a5bb700 1 -- 172.21.15.121:0/1125473452 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c005cf0 con 0x55a71e8a5670 2023-11-15T03:03:35.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.185+0000 7fdc4a5bb700 1 --2- 172.21.15.121:0/1125473452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a71e8a5670 0x55a71e8124e0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7fdc44001420 tx=0x7fdc44006b20).ready entity=mon.0 client_cookie=5dd0c61073b51919 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.185+0000 7fdc48db8700 1 -- 172.21.15.121:0/1125473452 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc440030d0 con 0x55a71e8a5670 2023-11-15T03:03:35.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.185+0000 7fdc48db8700 1 -- 172.21.15.121:0/1125473452 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc44012400 con 0x55a71e8a5670 2023-11-15T03:03:35.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.185+0000 7fdc48db8700 1 -- 172.21.15.121:0/1125473452 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc44012780 con 0x55a71e8a5670 2023-11-15T03:03:35.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.185+0000 7fdc4c4d8d00 1 -- 172.21.15.121:0/1125473452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a71e8a5670 msgr2=0x55a71e8124e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.185+0000 7fdc4c4d8d00 1 --2- 172.21.15.121:0/1125473452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a71e8a5670 0x55a71e8124e0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7fdc44001420 tx=0x7fdc44006b20).stop 2023-11-15T03:03:35.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.185+0000 7fdc4c4d8d00 1 -- 172.21.15.121:0/1125473452 shutdown_connections 2023-11-15T03:03:35.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.185+0000 7fdc4c4d8d00 1 --2- 172.21.15.121:0/1125473452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a71e8a5670 0x55a71e8124e0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.185+0000 7fdc4c4d8d00 1 --2- 172.21.15.121:0/1125473452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a71e816b00 0x55a71e811ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.185+0000 7fdc4c4d8d00 1 --2- 172.21.15.121:0/1125473452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a71e7cc6c0 0x55a71e89ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.185+0000 7fdc4c4d8d00 1 -- 172.21.15.121:0/1125473452 >> 172.21.15.121:0/1125473452 conn(0x55a71e81c080 msgr2=0x55a71e8a4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:35.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.185+0000 7fdc4c4d8d00 1 -- 172.21.15.121:0/1125473452 shutdown_connections 2023-11-15T03:03:35.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.185+0000 7fdc4c4d8d00 1 -- 172.21.15.121:0/1125473452 wait complete. 2023-11-15T03:03:35.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.185+0000 7fdc4c4d8d00 1 Processor -- start 2023-11-15T03:03:35.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.185+0000 7fdc4c4d8d00 1 -- start start 2023-11-15T03:03:35.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc4c4d8d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a71e7cc6c0 0x55a71e89b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc4c4d8d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a71e816b00 0x55a71e89bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc4c4d8d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a71e8a5670 0x55a71e8b9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc4c4d8d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71e89c1d0 con 0x55a71e816b00 2023-11-15T03:03:35.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc4c4d8d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71e8ba770 con 0x55a71e8a5670 2023-11-15T03:03:35.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc4c4d8d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a71e89dba0 con 0x55a71e7cc6c0 2023-11-15T03:03:35.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc415b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a71e816b00 0x55a71e89bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc415b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a71e816b00 0x55a71e89bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59404/0 (socket says 172.21.15.121:59404) 2023-11-15T03:03:35.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc415b9700 1 -- 172.21.15.121:0/16906072 learned_addr learned my addr 172.21.15.121:0/16906072 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:35.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc4a5bb700 1 --2- 172.21.15.121:0/16906072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a71e8a5670 0x55a71e8b9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc49dba700 1 --2- 172.21.15.121:0/16906072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a71e7cc6c0 0x55a71e89b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc415b9700 1 -- 172.21.15.121:0/16906072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a71e7cc6c0 msgr2=0x55a71e89b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc415b9700 1 --2- 172.21.15.121:0/16906072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a71e7cc6c0 0x55a71e89b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc415b9700 1 -- 172.21.15.121:0/16906072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a71e8a5670 msgr2=0x55a71e8b9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc415b9700 1 --2- 172.21.15.121:0/16906072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a71e8a5670 0x55a71e8b9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc415b9700 1 -- 172.21.15.121:0/16906072 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a71e7737f0 con 0x55a71e816b00 2023-11-15T03:03:35.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc4a5bb700 1 --2- 172.21.15.121:0/16906072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a71e8a5670 0x55a71e8b9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:35.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc415b9700 1 --2- 172.21.15.121:0/16906072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a71e816b00 0x55a71e89bce0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7fdc3400c010 tx=0x7fdc34016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc42ffd700 1 -- 172.21.15.121:0/16906072 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34022070 con 0x55a71e816b00 2023-11-15T03:03:35.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc42ffd700 1 -- 172.21.15.121:0/16906072 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc34013510 con 0x55a71e816b00 2023-11-15T03:03:35.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc4c4d8d00 1 -- 172.21.15.121:0/16906072 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc34005db0 con 0x55a71e816b00 2023-11-15T03:03:35.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc42ffd700 1 -- 172.21.15.121:0/16906072 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc340138d0 con 0x55a71e816b00 2023-11-15T03:03:35.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.189+0000 7fdc4c4d8d00 1 -- 172.21.15.121:0/16906072 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a71e76e770 con 0x55a71e816b00 2023-11-15T03:03:35.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.193+0000 7fdc42ffd700 1 -- 172.21.15.121:0/16906072 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fdc34013c80 con 0x55a71e816b00 2023-11-15T03:03:35.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.193+0000 7fdc42ffd700 1 --2- 172.21.15.121:0/16906072 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdc2c059020 0x7fdc2c05b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.193+0000 7fdc42ffd700 1 -- 172.21.15.121:0/16906072 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fdc34074ca0 con 0x55a71e816b00 2023-11-15T03:03:35.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.193+0000 7fdc49dba700 1 --2- 172.21.15.121:0/16906072 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdc2c059020 0x7fdc2c05b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.193+0000 7fdc4c4d8d00 1 --2- 172.21.15.121:0/16906072 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a71e801990 0x55a71e89c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.193+0000 7fdc4c4d8d00 1 -- 172.21.15.121:0/16906072 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:1 13.2d 13:b50e409b:::mds_snaptable:head [stat] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55a71e8a5060 con 0x55a71e801990 2023-11-15T03:03:35.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.193+0000 7fdc4a5bb700 1 --2- 172.21.15.121:0/16906072 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a71e801990 0x55a71e89c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.197+0000 7fdc49dba700 1 --2- 172.21.15.121:0/16906072 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdc2c059020 0x7fdc2c05b4e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fdc3c005cc0 tx=0x7fdc3c00c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.197+0000 7fdc4a5bb700 1 --2- 172.21.15.121:0/16906072 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a71e801990 0x55a71e89c9b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.197+0000 7fdc4a5bb700 1 -- 172.21.15.121:0/16906072 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(1 mds_snaptable [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+16 (crc 0 0 0) 0x7fdc44012660 con 0x55a71e801990 2023-11-15T03:03:35.212 INFO:teuthology.orchestra.run.smithi121.stdout:cephfs_metadata/mds_snaptable mtime 2023-11-15T03:03:21.000000+0000, size 70 2023-11-15T03:03:35.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.197+0000 7fdc4c4d8d00 1 -- 172.21.15.121:0/16906072 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a71e801990 msgr2=0x55a71e89c9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.197+0000 7fdc4c4d8d00 1 --2- 172.21.15.121:0/16906072 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a71e801990 0x55a71e89c9b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.197+0000 7fdc4c4d8d00 1 -- 172.21.15.121:0/16906072 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdc2c059020 msgr2=0x7fdc2c05b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.197+0000 7fdc4c4d8d00 1 --2- 172.21.15.121:0/16906072 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdc2c059020 0x7fdc2c05b4e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fdc3c005cc0 tx=0x7fdc3c00c040).stop 2023-11-15T03:03:35.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.197+0000 7fdc4c4d8d00 1 -- 172.21.15.121:0/16906072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a71e816b00 msgr2=0x55a71e89bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.197+0000 7fdc4c4d8d00 1 --2- 172.21.15.121:0/16906072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a71e816b00 0x55a71e89bce0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7fdc3400c010 tx=0x7fdc34016000).stop 2023-11-15T03:03:35.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.197+0000 7fdc4a5bb700 1 -- 172.21.15.121:0/16906072 reap_dead start 2023-11-15T03:03:35.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.197+0000 7fdc4c4d8d00 1 -- 172.21.15.121:0/16906072 shutdown_connections 2023-11-15T03:03:35.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.197+0000 7fdc4c4d8d00 1 -- 172.21.15.121:0/16906072 >> 172.21.15.121:0/16906072 conn(0x55a71e81c080 msgr2=0x55a71e7fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:35.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.197+0000 7fdc4c4d8d00 1 -- 172.21.15.121:0/16906072 shutdown_connections 2023-11-15T03:03:35.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.197+0000 7fdc4c4d8d00 1 -- 172.21.15.121:0/16906072 wait complete. 2023-11-15T03:03:35.219 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_inotable 2023-11-15T03:03:35.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.273+0000 7fd4b94dad00 1 Processor -- start 2023-11-15T03:03:35.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.273+0000 7fd4b94dad00 1 -- start start 2023-11-15T03:03:35.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.273+0000 7fd4b94dad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f606b356c0 0x55f606c0ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.273+0000 7fd4b94dad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f606b7fb00 0x55f606c10e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.273+0000 7fd4b94dad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f606c0e670 0x55f606c07f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.273+0000 7fd4b94dad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f606aae220 con 0x55f606b356c0 2023-11-15T03:03:35.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.273+0000 7fd4b94dad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f606c11380 con 0x55f606b7fb00 2023-11-15T03:03:35.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.273+0000 7fd4b94dad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f606c10a00 con 0x55f606c0e670 2023-11-15T03:03:35.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.273+0000 7fd4b6d31700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f606b356c0 0x55f606c0ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.273+0000 7fd4b6d31700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f606b356c0 0x55f606c0ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59412/0 (socket says 172.21.15.121:59412) 2023-11-15T03:03:35.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.273+0000 7fd4b6d31700 1 -- 172.21.15.121:0/531411 learned_addr learned my addr 172.21.15.121:0/531411 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:35.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.273+0000 7fd4b7532700 1 --2- 172.21.15.121:0/531411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f606c0e670 0x55f606c07f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.273+0000 7fd4ae530700 1 --2- 172.21.15.121:0/531411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f606b7fb00 0x55f606c10e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b6d31700 1 -- 172.21.15.121:0/531411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f606c0e670 msgr2=0x55f606c07f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:35.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b6d31700 1 --2- 172.21.15.121:0/531411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f606c0e670 0x55f606c07f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b6d31700 1 -- 172.21.15.121:0/531411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f606b7fb00 msgr2=0x55f606c10e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:35.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b6d31700 1 --2- 172.21.15.121:0/531411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f606b7fb00 0x55f606c10e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b6d31700 1 -- 172.21.15.121:0/531411 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a0005cf0 con 0x55f606b356c0 2023-11-15T03:03:35.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b6d31700 1 --2- 172.21.15.121:0/531411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f606b356c0 0x55f606c0ead0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7fd4a800d4f0 tx=0x7fd4a8015000).ready entity=mon.0 client_cookie=b4cecabc158794ff server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b5d2f700 1 -- 172.21.15.121:0/531411 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a800ac70 con 0x55f606b356c0 2023-11-15T03:03:35.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b5d2f700 1 -- 172.21.15.121:0/531411 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a8000f40 con 0x55f606b356c0 2023-11-15T03:03:35.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b5d2f700 1 -- 172.21.15.121:0/531411 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a800d740 con 0x55f606b356c0 2023-11-15T03:03:35.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b94dad00 1 -- 172.21.15.121:0/531411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f606b356c0 msgr2=0x55f606c0ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b94dad00 1 --2- 172.21.15.121:0/531411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f606b356c0 0x55f606c0ead0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7fd4a800d4f0 tx=0x7fd4a8015000).stop 2023-11-15T03:03:35.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b94dad00 1 -- 172.21.15.121:0/531411 shutdown_connections 2023-11-15T03:03:35.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b94dad00 1 --2- 172.21.15.121:0/531411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f606c0e670 0x55f606c07f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b94dad00 1 --2- 172.21.15.121:0/531411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f606b7fb00 0x55f606c10e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b94dad00 1 --2- 172.21.15.121:0/531411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f606b356c0 0x55f606c0ead0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b94dad00 1 -- 172.21.15.121:0/531411 >> 172.21.15.121:0/531411 conn(0x55f606b85080 msgr2=0x55f606b79fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:35.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b94dad00 1 -- 172.21.15.121:0/531411 shutdown_connections 2023-11-15T03:03:35.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b94dad00 1 -- 172.21.15.121:0/531411 wait complete. 2023-11-15T03:03:35.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b94dad00 1 Processor -- start 2023-11-15T03:03:35.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.277+0000 7fd4b94dad00 1 -- start start 2023-11-15T03:03:35.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4b94dad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f606b356c0 0x55f606bf5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4b94dad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f606b7fb00 0x55f606b649e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4b94dad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f606c0e670 0x55f606b64ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4b94dad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f606b653c0 con 0x55f606b356c0 2023-11-15T03:03:35.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4b94dad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f606c054a0 con 0x55f606b7fb00 2023-11-15T03:03:35.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4b94dad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f606c056f0 con 0x55f606c0e670 2023-11-15T03:03:35.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4b6d31700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f606b356c0 0x55f606bf5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4b6d31700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f606b356c0 0x55f606bf5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59416/0 (socket says 172.21.15.121:59416) 2023-11-15T03:03:35.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4b6d31700 1 -- 172.21.15.121:0/910146019 learned_addr learned my addr 172.21.15.121:0/910146019 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:35.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4ae530700 1 --2- 172.21.15.121:0/910146019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f606b7fb00 0x55f606b649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4b7532700 1 --2- 172.21.15.121:0/910146019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f606c0e670 0x55f606b64ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4b6d31700 1 -- 172.21.15.121:0/910146019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f606c0e670 msgr2=0x55f606b64ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4b6d31700 1 --2- 172.21.15.121:0/910146019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f606c0e670 0x55f606b64ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4b6d31700 1 -- 172.21.15.121:0/910146019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f606b7fb00 msgr2=0x55f606b649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4b6d31700 1 --2- 172.21.15.121:0/910146019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f606b7fb00 0x55f606b649e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4b6d31700 1 -- 172.21.15.121:0/910146019 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f606ad7a20 con 0x55f606b356c0 2023-11-15T03:03:35.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4b6d31700 1 --2- 172.21.15.121:0/910146019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f606b356c0 0x55f606bf5a70 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7fd4a8005ba0 tx=0x7fd4a801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4affff700 1 -- 172.21.15.121:0/910146019 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a800d930 con 0x55f606b356c0 2023-11-15T03:03:35.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4affff700 1 -- 172.21.15.121:0/910146019 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a8005730 con 0x55f606b356c0 2023-11-15T03:03:35.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4affff700 1 -- 172.21.15.121:0/910146019 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8010650 con 0x55f606b356c0 2023-11-15T03:03:35.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4b94dad00 1 -- 172.21.15.121:0/910146019 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a800a0e0 con 0x55f606b356c0 2023-11-15T03:03:35.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.281+0000 7fd4b94dad00 1 -- 172.21.15.121:0/910146019 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f606ad7770 con 0x55f606b356c0 2023-11-15T03:03:35.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.285+0000 7fd4affff700 1 -- 172.21.15.121:0/910146019 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd4a8017070 con 0x55f606b356c0 2023-11-15T03:03:35.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.285+0000 7fd4affff700 1 --2- 172.21.15.121:0/910146019 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd498058ff0 0x7fd49805b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.285+0000 7fd4affff700 1 -- 172.21.15.121:0/910146019 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fd4a8075cf0 con 0x55f606b356c0 2023-11-15T03:03:35.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.285+0000 7fd4ae530700 1 --2- 172.21.15.121:0/910146019 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd498058ff0 0x7fd49805b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.285+0000 7fd4b94dad00 1 --2- 172.21.15.121:0/910146019 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55f606b7cb80 0x55f606c06eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.285+0000 7fd4b94dad00 1 -- 172.21.15.121:0/910146019 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [stat] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55f606c0f020 con 0x55f606b7cb80 2023-11-15T03:03:35.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.285+0000 7fd4b7532700 1 --2- 172.21.15.121:0/910146019 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55f606b7cb80 0x55f606c06eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.285+0000 7fd4ae530700 1 --2- 172.21.15.121:0/910146019 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd498058ff0 0x7fd49805b4b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x55f606b658a0 tx=0x7fd4a000c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.289+0000 7fd4b7532700 1 --2- 172.21.15.121:0/910146019 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55f606b7cb80 0x55f606c06eb0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.289+0000 7fd4b7532700 1 -- 172.21.15.121:0/910146019 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 mds0_inotable [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 157+0+16 (crc 0 0 0) 0x7fd4b000f860 con 0x55f606b7cb80 2023-11-15T03:03:35.299 INFO:teuthology.orchestra.run.smithi121.stdout:cephfs_metadata/mds0_inotable mtime 2023-11-15T03:03:30.000000+0000, size 34 2023-11-15T03:03:35.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.289+0000 7fd4b94dad00 1 -- 172.21.15.121:0/910146019 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55f606b7cb80 msgr2=0x55f606c06eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.289+0000 7fd4b94dad00 1 --2- 172.21.15.121:0/910146019 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55f606b7cb80 0x55f606c06eb0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.289+0000 7fd4b94dad00 1 -- 172.21.15.121:0/910146019 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd498058ff0 msgr2=0x7fd49805b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.289+0000 7fd4b94dad00 1 --2- 172.21.15.121:0/910146019 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd498058ff0 0x7fd49805b4b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x55f606b658a0 tx=0x7fd4a000c040).stop 2023-11-15T03:03:35.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.289+0000 7fd4b94dad00 1 -- 172.21.15.121:0/910146019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f606b356c0 msgr2=0x55f606bf5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.289+0000 7fd4b94dad00 1 --2- 172.21.15.121:0/910146019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f606b356c0 0x55f606bf5a70 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7fd4a8005ba0 tx=0x7fd4a801a000).stop 2023-11-15T03:03:35.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.289+0000 7fd4b7532700 1 -- 172.21.15.121:0/910146019 reap_dead start 2023-11-15T03:03:35.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.289+0000 7fd4b94dad00 1 -- 172.21.15.121:0/910146019 shutdown_connections 2023-11-15T03:03:35.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.289+0000 7fd4b94dad00 1 -- 172.21.15.121:0/910146019 >> 172.21.15.121:0/910146019 conn(0x55f606b85080 msgr2=0x55f606b676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:35.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.289+0000 7fd4b94dad00 1 -- 172.21.15.121:0/910146019 shutdown_connections 2023-11-15T03:03:35.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.289+0000 7fd4b94dad00 1 -- 172.21.15.121:0/910146019 wait complete. 2023-11-15T03:03:35.303 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 100.00000000 2023-11-15T03:03:35.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71f54eed00 1 Processor -- start 2023-11-15T03:03:35.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71f54eed00 1 -- start start 2023-11-15T03:03:35.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71f54eed00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e463987bb0 0x55e4639ba4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71f54eed00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4639d1b00 0x55e463a6cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71f54eed00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e463a64650 0x55e463a6d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71f54eed00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e463935cc0 con 0x55e4639d1b00 2023-11-15T03:03:35.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71f54eed00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e463900220 con 0x55e463a64650 2023-11-15T03:03:35.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71f54eed00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e463a5a7b0 con 0x55e463987bb0 2023-11-15T03:03:35.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71ea621700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4639d1b00 0x55e463a6cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71ea621700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4639d1b00 0x55e463a6cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59428/0 (socket says 172.21.15.121:59428) 2023-11-15T03:03:35.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71ea621700 1 -- 172.21.15.121:0/3955902268 learned_addr learned my addr 172.21.15.121:0/3955902268 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:35.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71f3623700 1 --2- 172.21.15.121:0/3955902268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e463a64650 0x55e463a6d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71f2e22700 1 --2- 172.21.15.121:0/3955902268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e463987bb0 0x55e4639ba4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71ea621700 1 -- 172.21.15.121:0/3955902268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e463987bb0 msgr2=0x55e4639ba4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:35.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71ea621700 1 --2- 172.21.15.121:0/3955902268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e463987bb0 0x55e4639ba4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71ea621700 1 -- 172.21.15.121:0/3955902268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e463a64650 msgr2=0x55e463a6d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:35.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71ea621700 1 --2- 172.21.15.121:0/3955902268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e463a64650 0x55e463a6d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71ea621700 1 -- 172.21.15.121:0/3955902268 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71dc005cf0 con 0x55e4639d1b00 2023-11-15T03:03:35.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71f2e22700 1 --2- 172.21.15.121:0/3955902268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e463987bb0 0x55e4639ba4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:35.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71f3623700 1 --2- 172.21.15.121:0/3955902268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e463a64650 0x55e463a6d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:35.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71ea621700 1 --2- 172.21.15.121:0/3955902268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4639d1b00 0x55e463a6cec0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x55e4639d6e50 tx=0x7f71dc014000).ready entity=mon.0 client_cookie=4100ce4d33ce5288 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.361+0000 7f71f1e20700 1 -- 172.21.15.121:0/3955902268 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71dc0075b0 con 0x55e4639d1b00 2023-11-15T03:03:35.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f1e20700 1 -- 172.21.15.121:0/3955902268 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71dc007710 con 0x55e4639d1b00 2023-11-15T03:03:35.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f1e20700 1 -- 172.21.15.121:0/3955902268 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71dc007b30 con 0x55e4639d1b00 2023-11-15T03:03:35.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f54eed00 1 -- 172.21.15.121:0/3955902268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4639d1b00 msgr2=0x55e463a6cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f54eed00 1 --2- 172.21.15.121:0/3955902268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4639d1b00 0x55e463a6cec0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x55e4639d6e50 tx=0x7f71dc014000).stop 2023-11-15T03:03:35.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f54eed00 1 -- 172.21.15.121:0/3955902268 shutdown_connections 2023-11-15T03:03:35.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f54eed00 1 --2- 172.21.15.121:0/3955902268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e463a64650 0x55e463a6d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f54eed00 1 --2- 172.21.15.121:0/3955902268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4639d1b00 0x55e463a6cec0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f54eed00 1 --2- 172.21.15.121:0/3955902268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e463987bb0 0x55e4639ba4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f54eed00 1 -- 172.21.15.121:0/3955902268 >> 172.21.15.121:0/3955902268 conn(0x55e4639d7080 msgr2=0x55e4639cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:35.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f54eed00 1 -- 172.21.15.121:0/3955902268 shutdown_connections 2023-11-15T03:03:35.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f54eed00 1 -- 172.21.15.121:0/3955902268 wait complete. 2023-11-15T03:03:35.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f54eed00 1 Processor -- start 2023-11-15T03:03:35.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f54eed00 1 -- start start 2023-11-15T03:03:35.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f54eed00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e463987bb0 0x55e4639cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f54eed00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e4639d1b00 0x55e463a70390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f54eed00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e463a64650 0x55e4639cf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f54eed00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4639b86e0 con 0x55e463a64650 2023-11-15T03:03:35.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f54eed00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4639b7ad0 con 0x55e463987bb0 2023-11-15T03:03:35.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f54eed00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4639b7db0 con 0x55e4639d1b00 2023-11-15T03:03:35.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f3623700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e463a64650 0x55e4639cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f3623700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e463a64650 0x55e4639cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59440/0 (socket says 172.21.15.121:59440) 2023-11-15T03:03:35.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f3623700 1 -- 172.21.15.121:0/3783396746 learned_addr learned my addr 172.21.15.121:0/3783396746 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:35.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71f2e22700 1 --2- 172.21.15.121:0/3783396746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e463987bb0 0x55e4639cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.365+0000 7f71ea621700 1 --2- 172.21.15.121:0/3783396746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e4639d1b00 0x55e463a70390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.369+0000 7f71f3623700 1 -- 172.21.15.121:0/3783396746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e4639d1b00 msgr2=0x55e463a70390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.369+0000 7f71f3623700 1 --2- 172.21.15.121:0/3783396746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e4639d1b00 0x55e463a70390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.369+0000 7f71f3623700 1 -- 172.21.15.121:0/3783396746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e463987bb0 msgr2=0x55e4639cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.369+0000 7f71f3623700 1 --2- 172.21.15.121:0/3783396746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e463987bb0 0x55e4639cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.369+0000 7f71f3623700 1 -- 172.21.15.121:0/3783396746 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e463929770 con 0x55e463a64650 2023-11-15T03:03:35.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.369+0000 7f71f3623700 1 --2- 172.21.15.121:0/3783396746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e463a64650 0x55e4639cf980 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0x7f71ec00dfd0 tx=0x7f71ec0018b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.369+0000 7f71ebfff700 1 -- 172.21.15.121:0/3783396746 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec001d40 con 0x55e463a64650 2023-11-15T03:03:35.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.369+0000 7f71ebfff700 1 -- 172.21.15.121:0/3783396746 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ec008040 con 0x55e463a64650 2023-11-15T03:03:35.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.369+0000 7f71ebfff700 1 -- 172.21.15.121:0/3783396746 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec00bdf0 con 0x55e463a64650 2023-11-15T03:03:35.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.369+0000 7f71f54eed00 1 -- 172.21.15.121:0/3783396746 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e4638f8a70 con 0x55e463a64650 2023-11-15T03:03:35.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.369+0000 7f71f54eed00 1 -- 172.21.15.121:0/3783396746 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e463904010 con 0x55e463a64650 2023-11-15T03:03:35.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.373+0000 7f71ebfff700 1 -- 172.21.15.121:0/3783396746 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f71ec0103e0 con 0x55e463a64650 2023-11-15T03:03:35.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.373+0000 7f71ebfff700 1 --2- 172.21.15.121:0/3783396746 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71d4058ff0 0x7f71d405b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.373+0000 7f71ebfff700 1 -- 172.21.15.121:0/3783396746 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f71ec072be0 con 0x55e463a64650 2023-11-15T03:03:35.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.373+0000 7f71f2e22700 1 --2- 172.21.15.121:0/3783396746 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71d4058ff0 0x7f71d405b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.373+0000 7f71f54eed00 1 --2- 172.21.15.121:0/3783396746 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e4639b75f0 0x55e463a62c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.373+0000 7f71f54eed00 1 -- 172.21.15.121:0/3783396746 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 13.33 13:cd5a64a3:::100.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55e463a63180 con 0x55e4639b75f0 2023-11-15T03:03:35.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.373+0000 7f71f2e22700 1 --2- 172.21.15.121:0/3783396746 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71d4058ff0 0x7f71d405b4b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f71e4002250 tx=0x7f71e4006040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.373+0000 7f71ea621700 1 --2- 172.21.15.121:0/3783396746 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e4639b75f0 0x55e463a62c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.377+0000 7f71ea621700 1 --2- 172.21.15.121:0/3783396746 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e4639b75f0 0x55e463a62c90 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.377+0000 7f71ea621700 1 -- 172.21.15.121:0/3783396746 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 100.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x7f71dc027440 con 0x55e4639b75f0 2023-11-15T03:03:35.411 INFO:teuthology.orchestra.run.smithi121.stdout:cephfs_metadata/100.00000000 mtime 2023-11-15T03:03:21.000000+0000, size 0 2023-11-15T03:03:35.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.377+0000 7f71f54eed00 1 -- 172.21.15.121:0/3783396746 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e4639b75f0 msgr2=0x55e463a62c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.377+0000 7f71f54eed00 1 --2- 172.21.15.121:0/3783396746 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e4639b75f0 0x55e463a62c90 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.377+0000 7f71f54eed00 1 -- 172.21.15.121:0/3783396746 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71d4058ff0 msgr2=0x7f71d405b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.377+0000 7f71f54eed00 1 --2- 172.21.15.121:0/3783396746 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71d4058ff0 0x7f71d405b4b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f71e4002250 tx=0x7f71e4006040).stop 2023-11-15T03:03:35.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.377+0000 7f71f54eed00 1 -- 172.21.15.121:0/3783396746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e463a64650 msgr2=0x55e4639cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.377+0000 7f71f54eed00 1 --2- 172.21.15.121:0/3783396746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e463a64650 0x55e4639cf980 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0x7f71ec00dfd0 tx=0x7f71ec0018b0).stop 2023-11-15T03:03:35.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.377+0000 7f71f3623700 1 -- 172.21.15.121:0/3783396746 reap_dead start 2023-11-15T03:03:35.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.377+0000 7f71f54eed00 1 -- 172.21.15.121:0/3783396746 shutdown_connections 2023-11-15T03:03:35.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.377+0000 7f71f54eed00 1 -- 172.21.15.121:0/3783396746 >> 172.21.15.121:0/3783396746 conn(0x55e4639d7080 msgr2=0x55e4639ba4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:35.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.377+0000 7f71f54eed00 1 -- 172.21.15.121:0/3783396746 shutdown_connections 2023-11-15T03:03:35.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.377+0000 7f71f54eed00 1 -- 172.21.15.121:0/3783396746 wait complete. 2023-11-15T03:03:35.416 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_sessionmap 2023-11-15T03:03:35.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.469+0000 7f12ef538d00 1 Processor -- start 2023-11-15T03:03:35.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.469+0000 7f12ef538d00 1 -- start start 2023-11-15T03:03:35.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.469+0000 7f12ef538d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5608f6405de0 0x5608f64d8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.469+0000 7f12ef538d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5608f6437420 0x5608f64dcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.469+0000 7f12ef538d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f64db9f0 0x5608f6407ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.469+0000 7f12ef538d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608f637e220 con 0x5608f6437420 2023-11-15T03:03:35.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.469+0000 7f12ef538d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608f64d7610 con 0x5608f64db9f0 2023-11-15T03:03:35.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.469+0000 7f12ef538d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608f64dc8b0 con 0x5608f6405de0 2023-11-15T03:03:35.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.469+0000 7f12db418700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5608f6437420 0x5608f64dcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.469+0000 7f12db418700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5608f6437420 0x5608f64dcd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59456/0 (socket says 172.21.15.121:59456) 2023-11-15T03:03:35.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.469+0000 7f12db418700 1 -- 172.21.15.121:0/386744424 learned_addr learned my addr 172.21.15.121:0/386744424 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:35.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.469+0000 7f12e3c19700 1 --2- 172.21.15.121:0/386744424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5608f6405de0 0x5608f64d8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.469+0000 7f12e441a700 1 --2- 172.21.15.121:0/386744424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f64db9f0 0x5608f6407ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.469+0000 7f12db418700 1 -- 172.21.15.121:0/386744424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5608f6405de0 msgr2=0x5608f64d8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:35.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.469+0000 7f12db418700 1 --2- 172.21.15.121:0/386744424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5608f6405de0 0x5608f64d8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.469+0000 7f12db418700 1 -- 172.21.15.121:0/386744424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f64db9f0 msgr2=0x5608f6407ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:35.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.473+0000 7f12db418700 1 --2- 172.21.15.121:0/386744424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f64db9f0 0x5608f6407ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.473+0000 7f12db418700 1 -- 172.21.15.121:0/386744424 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d4005db0 con 0x5608f6437420 2023-11-15T03:03:35.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.473+0000 7f12db418700 1 --2- 172.21.15.121:0/386744424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5608f6437420 0x5608f64dcd40 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7f12cc009920 tx=0x7f12cc00c040).ready entity=mon.0 client_cookie=950cd065a45a9245 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.473+0000 7f12e2c17700 1 -- 172.21.15.121:0/386744424 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc00f040 con 0x5608f6437420 2023-11-15T03:03:35.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.473+0000 7f12e2c17700 1 -- 172.21.15.121:0/386744424 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12cc008ac0 con 0x5608f6437420 2023-11-15T03:03:35.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.473+0000 7f12e2c17700 1 -- 172.21.15.121:0/386744424 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc008e40 con 0x5608f6437420 2023-11-15T03:03:35.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.473+0000 7f12ef538d00 1 -- 172.21.15.121:0/386744424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5608f6437420 msgr2=0x5608f64dcd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.473+0000 7f12ef538d00 1 --2- 172.21.15.121:0/386744424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5608f6437420 0x5608f64dcd40 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7f12cc009920 tx=0x7f12cc00c040).stop 2023-11-15T03:03:35.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.473+0000 7f12ef538d00 1 -- 172.21.15.121:0/386744424 shutdown_connections 2023-11-15T03:03:35.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.473+0000 7f12ef538d00 1 --2- 172.21.15.121:0/386744424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f64db9f0 0x5608f6407ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.473+0000 7f12ef538d00 1 --2- 172.21.15.121:0/386744424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5608f6437420 0x5608f64dcd40 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.473+0000 7f12ef538d00 1 --2- 172.21.15.121:0/386744424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5608f6405de0 0x5608f64d8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.473+0000 7f12ef538d00 1 -- 172.21.15.121:0/386744424 >> 172.21.15.121:0/386744424 conn(0x5608f64067d0 msgr2=0x5608f6404ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:35.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.473+0000 7f12ef538d00 1 -- 172.21.15.121:0/386744424 shutdown_connections 2023-11-15T03:03:35.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.473+0000 7f12ef538d00 1 -- 172.21.15.121:0/386744424 wait complete. 2023-11-15T03:03:35.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.473+0000 7f12ef538d00 1 Processor -- start 2023-11-15T03:03:35.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.473+0000 7f12ef538d00 1 -- start start 2023-11-15T03:03:35.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12ef538d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f6405de0 0x5608f6433eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12ef538d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5608f6437420 0x5608f6407c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12ef538d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5608f64db9f0 0x5608f64d9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12ef538d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608f6434790 con 0x5608f6437420 2023-11-15T03:03:35.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12ef538d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608f64e7c10 con 0x5608f6405de0 2023-11-15T03:03:35.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12ef538d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608f64e7ef0 con 0x5608f64db9f0 2023-11-15T03:03:35.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12db418700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5608f6437420 0x5608f6407c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12e3c19700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f6405de0 0x5608f6433eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12db418700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5608f6437420 0x5608f6407c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59472/0 (socket says 172.21.15.121:59472) 2023-11-15T03:03:35.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12e441a700 1 --2- 172.21.15.121:0/2750461315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5608f64db9f0 0x5608f64d9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12db418700 1 -- 172.21.15.121:0/2750461315 learned_addr learned my addr 172.21.15.121:0/2750461315 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:35.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12db418700 1 -- 172.21.15.121:0/2750461315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5608f64db9f0 msgr2=0x5608f64d9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12db418700 1 --2- 172.21.15.121:0/2750461315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5608f64db9f0 0x5608f64d9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12db418700 1 -- 172.21.15.121:0/2750461315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f6405de0 msgr2=0x5608f6433eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12db418700 1 --2- 172.21.15.121:0/2750461315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f6405de0 0x5608f6433eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12db418700 1 -- 172.21.15.121:0/2750461315 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc009580 con 0x5608f6437420 2023-11-15T03:03:35.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12db418700 1 --2- 172.21.15.121:0/2750461315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5608f6437420 0x5608f6407c10 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7f12cc005370 tx=0x7f12cc018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12e1414700 1 -- 172.21.15.121:0/2750461315 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc00f040 con 0x5608f6437420 2023-11-15T03:03:35.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12e1414700 1 -- 172.21.15.121:0/2750461315 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12cc014df0 con 0x5608f6437420 2023-11-15T03:03:35.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12e1414700 1 -- 172.21.15.121:0/2750461315 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc01e690 con 0x5608f6437420 2023-11-15T03:03:35.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12ef538d00 1 -- 172.21.15.121:0/2750461315 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5608f63a7770 con 0x5608f6437420 2023-11-15T03:03:35.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.477+0000 7f12ef538d00 1 -- 172.21.15.121:0/2750461315 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608f6376a70 con 0x5608f6437420 2023-11-15T03:03:35.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.481+0000 7f12e1414700 1 -- 172.21.15.121:0/2750461315 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f12cc01eac0 con 0x5608f6437420 2023-11-15T03:03:35.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.481+0000 7f12e1414700 1 --2- 172.21.15.121:0/2750461315 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f12c4058ff0 0x7f12c405b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.481+0000 7f12e1414700 1 -- 172.21.15.121:0/2750461315 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f12cc074f20 con 0x5608f6437420 2023-11-15T03:03:35.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.481+0000 7f12e3c19700 1 --2- 172.21.15.121:0/2750461315 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f12c4058ff0 0x7f12c405b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.481+0000 7f12ef538d00 1 --2- 172.21.15.121:0/2750461315 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5608f64dd290 0x5608f64f2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.481+0000 7f12ef538d00 1 -- 172.21.15.121:0/2750461315 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [stat] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x5608f6408100 con 0x5608f64dd290 2023-11-15T03:03:35.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.485+0000 7f12e441a700 1 --2- 172.21.15.121:0/2750461315 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5608f64dd290 0x5608f64f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.485+0000 7f12e3c19700 1 --2- 172.21.15.121:0/2750461315 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f12c4058ff0 0x7f12c405b4b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f12d400c010 tx=0x7f12d4016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.485+0000 7f12e441a700 1 --2- 172.21.15.121:0/2750461315 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5608f64dd290 0x5608f64f2540 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.485+0000 7f12e441a700 1 -- 172.21.15.121:0/2750461315 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(1 mds0_sessionmap [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 159+0+16 (crc 0 0 0) 0x7f12dc00e700 con 0x5608f64dd290 2023-11-15T03:03:35.494 INFO:teuthology.orchestra.run.smithi121.stdout:cephfs_metadata/mds0_sessionmap mtime 2023-11-15T03:03:30.000000+0000, size 0 2023-11-15T03:03:35.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.485+0000 7f12ef538d00 1 -- 172.21.15.121:0/2750461315 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5608f64dd290 msgr2=0x5608f64f2540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.485+0000 7f12ef538d00 1 --2- 172.21.15.121:0/2750461315 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5608f64dd290 0x5608f64f2540 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.485+0000 7f12ef538d00 1 -- 172.21.15.121:0/2750461315 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f12c4058ff0 msgr2=0x7f12c405b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.485+0000 7f12ef538d00 1 --2- 172.21.15.121:0/2750461315 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f12c4058ff0 0x7f12c405b4b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f12d400c010 tx=0x7f12d4016000).stop 2023-11-15T03:03:35.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.485+0000 7f12ef538d00 1 -- 172.21.15.121:0/2750461315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5608f6437420 msgr2=0x5608f6407c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.485+0000 7f12ef538d00 1 --2- 172.21.15.121:0/2750461315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5608f6437420 0x5608f6407c10 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7f12cc005370 tx=0x7f12cc018000).stop 2023-11-15T03:03:35.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.485+0000 7f12e441a700 1 -- 172.21.15.121:0/2750461315 reap_dead start 2023-11-15T03:03:35.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.485+0000 7f12ef538d00 1 -- 172.21.15.121:0/2750461315 shutdown_connections 2023-11-15T03:03:35.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.485+0000 7f12ef538d00 1 -- 172.21.15.121:0/2750461315 >> 172.21.15.121:0/2750461315 conn(0x5608f64067d0 msgr2=0x5608f64da8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:35.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.485+0000 7f12ef538d00 1 -- 172.21.15.121:0/2750461315 shutdown_connections 2023-11-15T03:03:35.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.485+0000 7f12ef538d00 1 -- 172.21.15.121:0/2750461315 wait complete. 2023-11-15T03:03:35.498 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 400.00000000 2023-11-15T03:03:35.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.553+0000 7f3e22adad00 1 Processor -- start 2023-11-15T03:03:35.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.553+0000 7f3e22adad00 1 -- start start 2023-11-15T03:03:35.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.553+0000 7f3e22adad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55870afffa30 0x55870b032220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.553+0000 7f3e22adad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55870b04eb20 0x55870b0d5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.553+0000 7f3e22adad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55870b034bd0 0x55870b0d5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.553+0000 7f3e22adad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870af78220 con 0x55870b034bd0 2023-11-15T03:03:35.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.553+0000 7f3e22adad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870b04b740 con 0x55870b04eb20 2023-11-15T03:03:35.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.553+0000 7f3e22adad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55870b0d5ef0 con 0x55870afffa30 2023-11-15T03:03:35.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.553+0000 7f3e20bbd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55870b034bd0 0x55870b0d5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.553+0000 7f3e20bbd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55870b034bd0 0x55870b0d5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59494/0 (socket says 172.21.15.121:59494) 2023-11-15T03:03:35.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.553+0000 7f3e20bbd700 1 -- 172.21.15.121:0/4053278221 learned_addr learned my addr 172.21.15.121:0/4053278221 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:35.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.553+0000 7f3e17bbb700 1 --2- 172.21.15.121:0/4053278221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55870b04eb20 0x55870b0d5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.553+0000 7f3e203bc700 1 --2- 172.21.15.121:0/4053278221 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55870afffa30 0x55870b032220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e20bbd700 1 -- 172.21.15.121:0/4053278221 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55870afffa30 msgr2=0x55870b032220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:35.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e20bbd700 1 --2- 172.21.15.121:0/4053278221 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55870afffa30 0x55870b032220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e20bbd700 1 -- 172.21.15.121:0/4053278221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55870b04eb20 msgr2=0x55870b0d5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:35.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e20bbd700 1 --2- 172.21.15.121:0/4053278221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55870b04eb20 0x55870b0d5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e20bbd700 1 -- 172.21.15.121:0/4053278221 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e18005cf0 con 0x55870b034bd0 2023-11-15T03:03:35.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e20bbd700 1 --2- 172.21.15.121:0/4053278221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55870b034bd0 0x55870b0d5a00 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7f3e18009920 tx=0x7f3e1800c040).ready entity=mon.0 client_cookie=7012ec6814734944 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e1f3ba700 1 -- 172.21.15.121:0/4053278221 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1800f040 con 0x55870b034bd0 2023-11-15T03:03:35.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e1f3ba700 1 -- 172.21.15.121:0/4053278221 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e18008800 con 0x55870b034bd0 2023-11-15T03:03:35.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e1f3ba700 1 -- 172.21.15.121:0/4053278221 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e18008ba0 con 0x55870b034bd0 2023-11-15T03:03:35.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e22adad00 1 -- 172.21.15.121:0/4053278221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55870b034bd0 msgr2=0x55870b0d5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e22adad00 1 --2- 172.21.15.121:0/4053278221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55870b034bd0 0x55870b0d5a00 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7f3e18009920 tx=0x7f3e1800c040).stop 2023-11-15T03:03:35.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e22adad00 1 -- 172.21.15.121:0/4053278221 shutdown_connections 2023-11-15T03:03:35.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e22adad00 1 --2- 172.21.15.121:0/4053278221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55870b034bd0 0x55870b0d5a00 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e22adad00 1 --2- 172.21.15.121:0/4053278221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55870b04eb20 0x55870b0d5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e22adad00 1 --2- 172.21.15.121:0/4053278221 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55870afffa30 0x55870b032220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e22adad00 1 -- 172.21.15.121:0/4053278221 >> 172.21.15.121:0/4053278221 conn(0x55870b04f080 msgr2=0x55870b044800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:35.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e22adad00 1 -- 172.21.15.121:0/4053278221 shutdown_connections 2023-11-15T03:03:35.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e22adad00 1 -- 172.21.15.121:0/4053278221 wait complete. 2023-11-15T03:03:35.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e22adad00 1 Processor -- start 2023-11-15T03:03:35.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.557+0000 7f3e22adad00 1 -- start start 2023-11-15T03:03:35.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e22adad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55870afffa30 0x55870b0c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e22adad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55870b034bd0 0x55870b044fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e22adad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55870b04eb20 0x55870b0454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e22adad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870b0d5bb0 con 0x55870b034bd0 2023-11-15T03:03:35.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e22adad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870b0d2a40 con 0x55870b04eb20 2023-11-15T03:03:35.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e22adad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55870b0d37a0 con 0x55870afffa30 2023-11-15T03:03:35.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e17bbb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55870b034bd0 0x55870b044fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e17bbb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55870b034bd0 0x55870b044fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59496/0 (socket says 172.21.15.121:59496) 2023-11-15T03:03:35.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e17bbb700 1 -- 172.21.15.121:0/3941875542 learned_addr learned my addr 172.21.15.121:0/3941875542 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:35.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e20bbd700 1 --2- 172.21.15.121:0/3941875542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55870b04eb20 0x55870b0454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e203bc700 1 --2- 172.21.15.121:0/3941875542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55870afffa30 0x55870b0c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e17bbb700 1 -- 172.21.15.121:0/3941875542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55870afffa30 msgr2=0x55870b0c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e17bbb700 1 --2- 172.21.15.121:0/3941875542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55870afffa30 0x55870b0c84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e17bbb700 1 -- 172.21.15.121:0/3941875542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55870b04eb20 msgr2=0x55870b0454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e17bbb700 1 --2- 172.21.15.121:0/3941875542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55870b04eb20 0x55870b0454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e17bbb700 1 -- 172.21.15.121:0/3941875542 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e18009570 con 0x55870b034bd0 2023-11-15T03:03:35.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e203bc700 1 --2- 172.21.15.121:0/3941875542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55870afffa30 0x55870b0c84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:35.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e17bbb700 1 --2- 172.21.15.121:0/3941875542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55870b034bd0 0x55870b044fb0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x55870afecd50 tx=0x7f3e08015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e1dbb7700 1 -- 172.21.15.121:0/3941875542 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e08005870 con 0x55870b034bd0 2023-11-15T03:03:35.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e1dbb7700 1 -- 172.21.15.121:0/3941875542 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0800dd00 con 0x55870b034bd0 2023-11-15T03:03:35.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e22adad00 1 -- 172.21.15.121:0/3941875542 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0800a0e0 con 0x55870b034bd0 2023-11-15T03:03:35.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e1dbb7700 1 -- 172.21.15.121:0/3941875542 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0800de70 con 0x55870b034bd0 2023-11-15T03:03:35.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.561+0000 7f3e22adad00 1 -- 172.21.15.121:0/3941875542 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55870afa1770 con 0x55870b034bd0 2023-11-15T03:03:35.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.565+0000 7f3e1dbb7700 1 -- 172.21.15.121:0/3941875542 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3e08021020 con 0x55870b034bd0 2023-11-15T03:03:35.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.565+0000 7f3e1dbb7700 1 --2- 172.21.15.121:0/3941875542 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e00058ff0 0x7f3e0005b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.565+0000 7f3e1dbb7700 1 -- 172.21.15.121:0/3941875542 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f3e08074f30 con 0x55870b034bd0 2023-11-15T03:03:35.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.565+0000 7f3e203bc700 1 --2- 172.21.15.121:0/3941875542 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e00058ff0 0x7f3e0005b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.565+0000 7f3e22adad00 1 --2- 172.21.15.121:0/3941875542 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55870b0d9390 0x55870b0d9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.565+0000 7f3e22adad00 1 -- 172.21.15.121:0/3941875542 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55870b0d3c30 con 0x55870b0d9390 2023-11-15T03:03:35.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.565+0000 7f3e203bc700 1 --2- 172.21.15.121:0/3941875542 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e00058ff0 0x7f3e0005b4b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f3e10004170 tx=0x7f3e10000bc0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.565+0000 7f3e20bbd700 1 --2- 172.21.15.121:0/3941875542 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55870b0d9390 0x55870b0d9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.569+0000 7f3e20bbd700 1 --2- 172.21.15.121:0/3941875542 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55870b0d9390 0x55870b0d9770 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.569+0000 7f3e20bbd700 1 -- 172.21.15.121:0/3941875542 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 400.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x7f3e18014430 con 0x55870b0d9390 2023-11-15T03:03:35.579 INFO:teuthology.orchestra.run.smithi121.stdout:cephfs_metadata/400.00000000 mtime 2023-11-15T03:03:21.000000+0000, size 22 2023-11-15T03:03:35.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.569+0000 7f3e22adad00 1 -- 172.21.15.121:0/3941875542 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55870b0d9390 msgr2=0x55870b0d9770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.569+0000 7f3e22adad00 1 --2- 172.21.15.121:0/3941875542 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55870b0d9390 0x55870b0d9770 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.569+0000 7f3e22adad00 1 -- 172.21.15.121:0/3941875542 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e00058ff0 msgr2=0x7f3e0005b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.569+0000 7f3e22adad00 1 --2- 172.21.15.121:0/3941875542 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e00058ff0 0x7f3e0005b4b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f3e10004170 tx=0x7f3e10000bc0).stop 2023-11-15T03:03:35.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.569+0000 7f3e22adad00 1 -- 172.21.15.121:0/3941875542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55870b034bd0 msgr2=0x55870b044fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.569+0000 7f3e22adad00 1 --2- 172.21.15.121:0/3941875542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55870b034bd0 0x55870b044fb0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x55870afecd50 tx=0x7f3e08015000).stop 2023-11-15T03:03:35.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.569+0000 7f3e20bbd700 1 -- 172.21.15.121:0/3941875542 reap_dead start 2023-11-15T03:03:35.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.569+0000 7f3e22adad00 1 -- 172.21.15.121:0/3941875542 shutdown_connections 2023-11-15T03:03:35.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.569+0000 7f3e22adad00 1 -- 172.21.15.121:0/3941875542 >> 172.21.15.121:0/3941875542 conn(0x55870b04f080 msgr2=0x55870b04a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:35.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.569+0000 7f3e22adad00 1 -- 172.21.15.121:0/3941875542 shutdown_connections 2023-11-15T03:03:35.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.569+0000 7f3e22adad00 1 -- 172.21.15.121:0/3941875542 wait complete. 2023-11-15T03:03:35.586 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 100.00000000.inode 2023-11-15T03:03:35.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.645+0000 7fe0224ddd00 1 Processor -- start 2023-11-15T03:03:35.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.645+0000 7fe0224ddd00 1 -- start start 2023-11-15T03:03:35.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.645+0000 7fe0224ddd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0a058cbb0 0x55c0a05bf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.645+0000 7fe0224ddd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c0a05d6b00 0x55c0a065a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.645+0000 7fe0224ddd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0a0669650 0x55c0a065adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.645+0000 7fe0224ddd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a053acc0 con 0x55c0a058cbb0 2023-11-15T03:03:35.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.645+0000 7fe0224ddd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a0505220 con 0x55c0a0669650 2023-11-15T03:03:35.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.645+0000 7fe0224ddd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c0a065ef90 con 0x55c0a05d6b00 2023-11-15T03:03:35.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.645+0000 7fe01fe37700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0a058cbb0 0x55c0a05bf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.645+0000 7fe01fe37700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0a058cbb0 0x55c0a05bf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59518/0 (socket says 172.21.15.121:59518) 2023-11-15T03:03:35.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.645+0000 7fe01fe37700 1 -- 172.21.15.121:0/2961182053 learned_addr learned my addr 172.21.15.121:0/2961182053 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:35.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.645+0000 7fe020638700 1 --2- 172.21.15.121:0/2961182053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0a0669650 0x55c0a065adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.645+0000 7fe017636700 1 --2- 172.21.15.121:0/2961182053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c0a05d6b00 0x55c0a065a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.645+0000 7fe01fe37700 1 -- 172.21.15.121:0/2961182053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c0a05d6b00 msgr2=0x55c0a065a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:35.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.645+0000 7fe01fe37700 1 --2- 172.21.15.121:0/2961182053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c0a05d6b00 0x55c0a065a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.645+0000 7fe01fe37700 1 -- 172.21.15.121:0/2961182053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0a0669650 msgr2=0x55c0a065adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:35.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.645+0000 7fe01fe37700 1 --2- 172.21.15.121:0/2961182053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0a0669650 0x55c0a065adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.645+0000 7fe01fe37700 1 -- 172.21.15.121:0/2961182053 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008005cf0 con 0x55c0a058cbb0 2023-11-15T03:03:35.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.645+0000 7fe020638700 1 --2- 172.21.15.121:0/2961182053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0a0669650 0x55c0a065adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:35.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.649+0000 7fe01fe37700 1 --2- 172.21.15.121:0/2961182053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0a058cbb0 0x55c0a05bf390 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0x7fe010001560 tx=0x7fe010006b60).ready entity=mon.0 client_cookie=4a0e6d8aca3adc18 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.649+0000 7fe01ee35700 1 -- 172.21.15.121:0/2961182053 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010003090 con 0x55c0a058cbb0 2023-11-15T03:03:35.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.649+0000 7fe01ee35700 1 -- 172.21.15.121:0/2961182053 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe010012450 con 0x55c0a058cbb0 2023-11-15T03:03:35.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.649+0000 7fe01ee35700 1 -- 172.21.15.121:0/2961182053 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0100127f0 con 0x55c0a058cbb0 2023-11-15T03:03:35.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.649+0000 7fe0224ddd00 1 -- 172.21.15.121:0/2961182053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0a058cbb0 msgr2=0x55c0a05bf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.649+0000 7fe0224ddd00 1 --2- 172.21.15.121:0/2961182053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0a058cbb0 0x55c0a05bf390 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0x7fe010001560 tx=0x7fe010006b60).stop 2023-11-15T03:03:35.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.649+0000 7fe0224ddd00 1 -- 172.21.15.121:0/2961182053 shutdown_connections 2023-11-15T03:03:35.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.649+0000 7fe0224ddd00 1 --2- 172.21.15.121:0/2961182053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0a0669650 0x55c0a065adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.649+0000 7fe0224ddd00 1 --2- 172.21.15.121:0/2961182053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c0a05d6b00 0x55c0a065a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.649+0000 7fe0224ddd00 1 --2- 172.21.15.121:0/2961182053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0a058cbb0 0x55c0a05bf390 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.649+0000 7fe0224ddd00 1 -- 172.21.15.121:0/2961182053 >> 172.21.15.121:0/2961182053 conn(0x55c0a05dc080 msgr2=0x55c0a05d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:35.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.649+0000 7fe0224ddd00 1 -- 172.21.15.121:0/2961182053 shutdown_connections 2023-11-15T03:03:35.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.649+0000 7fe0224ddd00 1 -- 172.21.15.121:0/2961182053 wait complete. 2023-11-15T03:03:35.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.649+0000 7fe0224ddd00 1 Processor -- start 2023-11-15T03:03:35.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.649+0000 7fe0224ddd00 1 -- start start 2023-11-15T03:03:35.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe0224ddd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0a05d6b00 0x55c0a05be3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe0224ddd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c0a0669650 0x55c0a05d4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe0224ddd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0a0658c00 0x55c0a05d4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe0224ddd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a05d4df0 con 0x55c0a05d6b00 2023-11-15T03:03:35.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe0224ddd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a0659ca0 con 0x55c0a0658c00 2023-11-15T03:03:35.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe0224ddd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c0a05be890 con 0x55c0a0669650 2023-11-15T03:03:35.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe01fe37700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0a05d6b00 0x55c0a05be3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe01fe37700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0a05d6b00 0x55c0a05be3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59522/0 (socket says 172.21.15.121:59522) 2023-11-15T03:03:35.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe01fe37700 1 -- 172.21.15.121:0/360672237 learned_addr learned my addr 172.21.15.121:0/360672237 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:35.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe017636700 1 --2- 172.21.15.121:0/360672237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c0a0669650 0x55c0a05d4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe020638700 1 --2- 172.21.15.121:0/360672237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0a0658c00 0x55c0a05d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe01fe37700 1 -- 172.21.15.121:0/360672237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c0a0669650 msgr2=0x55c0a05d4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe01fe37700 1 --2- 172.21.15.121:0/360672237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c0a0669650 0x55c0a05d4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe01fe37700 1 -- 172.21.15.121:0/360672237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0a0658c00 msgr2=0x55c0a05d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe01fe37700 1 --2- 172.21.15.121:0/360672237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0a0658c00 0x55c0a05d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe01fe37700 1 -- 172.21.15.121:0/360672237 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0a052ea20 con 0x55c0a05d6b00 2023-11-15T03:03:35.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe01fe37700 1 --2- 172.21.15.121:0/360672237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0a05d6b00 0x55c0a05be3a0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7fe010009000 tx=0x7fe010014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe01d632700 1 -- 172.21.15.121:0/360672237 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01000d5e0 con 0x55c0a05d6b00 2023-11-15T03:03:35.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe01d632700 1 -- 172.21.15.121:0/360672237 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01000d780 con 0x55c0a05d6b00 2023-11-15T03:03:35.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe01d632700 1 -- 172.21.15.121:0/360672237 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01000db40 con 0x55c0a05d6b00 2023-11-15T03:03:35.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe0224ddd00 1 -- 172.21.15.121:0/360672237 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c0a04fda70 con 0x55c0a05d6b00 2023-11-15T03:03:35.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.653+0000 7fe0224ddd00 1 -- 172.21.15.121:0/360672237 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0a0509010 con 0x55c0a05d6b00 2023-11-15T03:03:35.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.657+0000 7fe01d632700 1 -- 172.21.15.121:0/360672237 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe010028440 con 0x55c0a05d6b00 2023-11-15T03:03:35.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.657+0000 7fe01d632700 1 --2- 172.21.15.121:0/360672237 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe000058ff0 0x7fe00005b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.657+0000 7fe01d632700 1 -- 172.21.15.121:0/360672237 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fe010074690 con 0x55c0a05d6b00 2023-11-15T03:03:35.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.657+0000 7fe017636700 1 --2- 172.21.15.121:0/360672237 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe000058ff0 0x7fe00005b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.657+0000 7fe0224ddd00 1 --2- 172.21.15.121:0/360672237 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c0a05bf390 0x55c0a0679060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.657+0000 7fe0224ddd00 1 -- 172.21.15.121:0/360672237 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.3f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55c0a05bbbe0 con 0x55c0a05bf390 2023-11-15T03:03:35.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.657+0000 7fe017636700 1 --2- 172.21.15.121:0/360672237 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe000058ff0 0x7fe00005b4b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7fe00800afd0 tx=0x7fe008016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.661+0000 7fe020638700 1 --2- 172.21.15.121:0/360672237 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c0a05bf390 0x55c0a0679060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.661+0000 7fe020638700 1 --2- 172.21.15.121:0/360672237 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c0a05bf390 0x55c0a0679060 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.661+0000 7fe020638700 1 -- 172.21.15.121:0/360672237 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x7fe01800f600 con 0x55c0a05bf390 2023-11-15T03:03:35.674 INFO:teuthology.orchestra.run.smithi121.stdout:cephfs_metadata/100.00000000.inode mtime 2023-11-15T03:03:21.000000+0000, size 554 2023-11-15T03:03:35.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.661+0000 7fe0224ddd00 1 -- 172.21.15.121:0/360672237 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c0a05bf390 msgr2=0x55c0a0679060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.661+0000 7fe0224ddd00 1 --2- 172.21.15.121:0/360672237 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c0a05bf390 0x55c0a0679060 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.661+0000 7fe0224ddd00 1 -- 172.21.15.121:0/360672237 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe000058ff0 msgr2=0x7fe00005b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.661+0000 7fe0224ddd00 1 --2- 172.21.15.121:0/360672237 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe000058ff0 0x7fe00005b4b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7fe00800afd0 tx=0x7fe008016000).stop 2023-11-15T03:03:35.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.661+0000 7fe0224ddd00 1 -- 172.21.15.121:0/360672237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0a05d6b00 msgr2=0x55c0a05be3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.661+0000 7fe0224ddd00 1 --2- 172.21.15.121:0/360672237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0a05d6b00 0x55c0a05be3a0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7fe010009000 tx=0x7fe010014000).stop 2023-11-15T03:03:35.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.661+0000 7fe020638700 1 -- 172.21.15.121:0/360672237 reap_dead start 2023-11-15T03:03:35.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.661+0000 7fe0224ddd00 1 -- 172.21.15.121:0/360672237 shutdown_connections 2023-11-15T03:03:35.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.661+0000 7fe0224ddd00 1 -- 172.21.15.121:0/360672237 >> 172.21.15.121:0/360672237 conn(0x55c0a05dc080 msgr2=0x55c0a05d9270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:35.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.661+0000 7fe0224ddd00 1 -- 172.21.15.121:0/360672237 shutdown_connections 2023-11-15T03:03:35.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.661+0000 7fe0224ddd00 1 -- 172.21.15.121:0/360672237 wait complete. 2023-11-15T03:03:35.678 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 1.00000000 2023-11-15T03:03:35.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.733+0000 7efdb94dad00 1 Processor -- start 2023-11-15T03:03:35.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.733+0000 7efdb94dad00 1 -- start start 2023-11-15T03:03:35.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdb94dad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f633dbea30 0x55f633e0b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdb94dad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f633e09e70 0x55f633dbe050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdb94dad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f633e9f000 0x55f633df1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdb94dad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f633d37220 con 0x55f633e09e70 2023-11-15T03:03:35.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdb94dad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f633e0b9f0 con 0x55f633dbea30 2023-11-15T03:03:35.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdb94dad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f633e94d90 con 0x55f633e9f000 2023-11-15T03:03:35.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdaffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f633e09e70 0x55f633dbe050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdaffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f633e09e70 0x55f633dbe050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59542/0 (socket says 172.21.15.121:59542) 2023-11-15T03:03:35.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdaffff700 1 -- 172.21.15.121:0/4093148056 learned_addr learned my addr 172.21.15.121:0/4093148056 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:35.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdb755f700 1 --2- 172.21.15.121:0/4093148056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f633e9f000 0x55f633df1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdb6d5e700 1 --2- 172.21.15.121:0/4093148056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f633dbea30 0x55f633e0b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdaffff700 1 -- 172.21.15.121:0/4093148056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f633e9f000 msgr2=0x55f633df1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:35.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdaffff700 1 --2- 172.21.15.121:0/4093148056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f633e9f000 0x55f633df1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdaffff700 1 -- 172.21.15.121:0/4093148056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f633dbea30 msgr2=0x55f633e0b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:35.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdaffff700 1 --2- 172.21.15.121:0/4093148056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f633dbea30 0x55f633e0b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdaffff700 1 -- 172.21.15.121:0/4093148056 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda8005db0 con 0x55f633e09e70 2023-11-15T03:03:35.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdaffff700 1 --2- 172.21.15.121:0/4093148056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f633e09e70 0x55f633dbe050 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0x7efda000b230 tx=0x7efda00017b0).ready entity=mon.0 client_cookie=a50b28866af8ff37 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdb5d5c700 1 -- 172.21.15.121:0/4093148056 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda000f450 con 0x55f633e09e70 2023-11-15T03:03:35.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdb5d5c700 1 -- 172.21.15.121:0/4093148056 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda000f5b0 con 0x55f633e09e70 2023-11-15T03:03:35.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdb5d5c700 1 -- 172.21.15.121:0/4093148056 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda000f950 con 0x55f633e09e70 2023-11-15T03:03:35.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdb94dad00 1 -- 172.21.15.121:0/4093148056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f633e09e70 msgr2=0x55f633dbe050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdb94dad00 1 --2- 172.21.15.121:0/4093148056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f633e09e70 0x55f633dbe050 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0x7efda000b230 tx=0x7efda00017b0).stop 2023-11-15T03:03:35.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdb94dad00 1 -- 172.21.15.121:0/4093148056 shutdown_connections 2023-11-15T03:03:35.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdb94dad00 1 --2- 172.21.15.121:0/4093148056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f633e9f000 0x55f633df1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdb94dad00 1 --2- 172.21.15.121:0/4093148056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f633e09e70 0x55f633dbe050 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdb94dad00 1 --2- 172.21.15.121:0/4093148056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f633dbea30 0x55f633e0b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.737+0000 7efdb94dad00 1 -- 172.21.15.121:0/4093148056 >> 172.21.15.121:0/4093148056 conn(0x55f633e0e080 msgr2=0x55f633e0e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:35.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.741+0000 7efdb94dad00 1 -- 172.21.15.121:0/4093148056 shutdown_connections 2023-11-15T03:03:35.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.741+0000 7efdb94dad00 1 -- 172.21.15.121:0/4093148056 wait complete. 2023-11-15T03:03:35.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.741+0000 7efdb94dad00 1 Processor -- start 2023-11-15T03:03:35.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.741+0000 7efdb94dad00 1 -- start start 2023-11-15T03:03:35.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.741+0000 7efdb94dad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f633dbea30 0x55f633e98e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.741+0000 7efdb94dad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f633e09e70 0x55f633e904b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.741+0000 7efdb94dad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f633e9f000 0x55f633df1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.741+0000 7efdb94dad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f633e94b90 con 0x55f633dbea30 2023-11-15T03:03:35.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.741+0000 7efdb94dad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f633e909a0 con 0x55f633e09e70 2023-11-15T03:03:35.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.741+0000 7efdb94dad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f633e91530 con 0x55f633e9f000 2023-11-15T03:03:35.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.741+0000 7efdb6d5e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f633dbea30 0x55f633e98e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.745+0000 7efdb6d5e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f633dbea30 0x55f633e98e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59546/0 (socket says 172.21.15.121:59546) 2023-11-15T03:03:35.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.745+0000 7efdb6d5e700 1 -- 172.21.15.121:0/1290662006 learned_addr learned my addr 172.21.15.121:0/1290662006 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:35.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.745+0000 7efdb755f700 1 --2- 172.21.15.121:0/1290662006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f633e9f000 0x55f633df1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.745+0000 7efdaffff700 1 --2- 172.21.15.121:0/1290662006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f633e09e70 0x55f633e904b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.745+0000 7efdb6d5e700 1 -- 172.21.15.121:0/1290662006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f633e9f000 msgr2=0x55f633df1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.745+0000 7efdb6d5e700 1 --2- 172.21.15.121:0/1290662006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f633e9f000 0x55f633df1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.745+0000 7efdb6d5e700 1 -- 172.21.15.121:0/1290662006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f633e09e70 msgr2=0x55f633e904b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.745+0000 7efdb6d5e700 1 --2- 172.21.15.121:0/1290662006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f633e09e70 0x55f633e904b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.745+0000 7efdb6d5e700 1 -- 172.21.15.121:0/1290662006 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda0001a90 con 0x55f633dbea30 2023-11-15T03:03:35.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.745+0000 7efdb6d5e700 1 --2- 172.21.15.121:0/1290662006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f633dbea30 0x55f633e98e90 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7efda8008bf0 tx=0x7efda800c040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.745+0000 7efdaf7fe700 1 -- 172.21.15.121:0/1290662006 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda800f030 con 0x55f633dbea30 2023-11-15T03:03:35.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.745+0000 7efdaf7fe700 1 -- 172.21.15.121:0/1290662006 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda8010040 con 0x55f633dbea30 2023-11-15T03:03:35.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.745+0000 7efdb94dad00 1 -- 172.21.15.121:0/1290662006 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb000a0e0 con 0x55f633dbea30 2023-11-15T03:03:35.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.745+0000 7efdaf7fe700 1 -- 172.21.15.121:0/1290662006 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda8006d80 con 0x55f633dbea30 2023-11-15T03:03:35.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.745+0000 7efdb94dad00 1 -- 172.21.15.121:0/1290662006 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f633d60770 con 0x55f633dbea30 2023-11-15T03:03:35.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.749+0000 7efdaf7fe700 1 -- 172.21.15.121:0/1290662006 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7efda80143c0 con 0x55f633dbea30 2023-11-15T03:03:35.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.749+0000 7efdaf7fe700 1 --2- 172.21.15.121:0/1290662006 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efd98059020 0x7efd9805b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.749+0000 7efdaf7fe700 1 -- 172.21.15.121:0/1290662006 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7efda8073630 con 0x55f633dbea30 2023-11-15T03:03:35.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.749+0000 7efdaffff700 1 --2- 172.21.15.121:0/1290662006 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efd98059020 0x7efd9805b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.749+0000 7efdb94dad00 1 --2- 172.21.15.121:0/1290662006 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f633e0b500 0x55f633e936f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:35.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.749+0000 7efdb94dad00 1 -- 172.21.15.121:0/1290662006 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.3f 13:ff5b34d6:::1.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected e93) v8 -- 0x55f633df19f0 con 0x55f633e0b500 2023-11-15T03:03:35.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.749+0000 7efdaffff700 1 --2- 172.21.15.121:0/1290662006 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efd98059020 0x7efd9805b4e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7efda000b230 tx=0x7efda00017b0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.749+0000 7efdb755f700 1 --2- 172.21.15.121:0/1290662006 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f633e0b500 0x55f633e936f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:35.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.749+0000 7efdb755f700 1 --2- 172.21.15.121:0/1290662006 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f633e0b500 0x55f633e936f0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:35.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.753+0000 7efdb755f700 1 -- 172.21.15.121:0/1290662006 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 1.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 154+0+16 (crc 0 0 0) 0x7efdb000d740 con 0x55f633e0b500 2023-11-15T03:03:35.763 INFO:teuthology.orchestra.run.smithi121.stdout:cephfs_metadata/1.00000000 mtime 2023-11-15T03:03:30.000000+0000, size 0 2023-11-15T03:03:35.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.753+0000 7efdb94dad00 1 -- 172.21.15.121:0/1290662006 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f633e0b500 msgr2=0x55f633e936f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.753+0000 7efdb94dad00 1 --2- 172.21.15.121:0/1290662006 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f633e0b500 0x55f633e936f0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:35.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.753+0000 7efdb94dad00 1 -- 172.21.15.121:0/1290662006 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efd98059020 msgr2=0x7efd9805b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.753+0000 7efdb94dad00 1 --2- 172.21.15.121:0/1290662006 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efd98059020 0x7efd9805b4e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7efda000b230 tx=0x7efda00017b0).stop 2023-11-15T03:03:35.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.753+0000 7efdb94dad00 1 -- 172.21.15.121:0/1290662006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f633dbea30 msgr2=0x55f633e98e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:35.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.753+0000 7efdb94dad00 1 --2- 172.21.15.121:0/1290662006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f633dbea30 0x55f633e98e90 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7efda8008bf0 tx=0x7efda800c040).stop 2023-11-15T03:03:35.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.753+0000 7efdb755f700 1 -- 172.21.15.121:0/1290662006 reap_dead start 2023-11-15T03:03:35.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.753+0000 7efdb94dad00 1 -- 172.21.15.121:0/1290662006 shutdown_connections 2023-11-15T03:03:35.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.753+0000 7efdb94dad00 1 -- 172.21.15.121:0/1290662006 >> 172.21.15.121:0/1290662006 conn(0x55f633e0e080 msgr2=0x55f633e0e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:35.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.753+0000 7efdb94dad00 1 -- 172.21.15.121:0/1290662006 shutdown_connections 2023-11-15T03:03:35.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:35.753+0000 7efdb94dad00 1 -- 172.21.15.121:0/1290662006 wait complete. 2023-11-15T03:03:35.769 INFO:tasks.cephfs.test_damage:Expectation on rm '10000000000.00000000' will be 'no damage' 2023-11-15T03:03:35.769 INFO:tasks.cephfs.test_damage:Expectation on rm '600.00000000' will be 'damaged_on_start' 2023-11-15T03:03:35.769 INFO:tasks.cephfs.test_damage:Expectation on rm '1.00000000.inode' will be 'damaged_on_start' 2023-11-15T03:03:35.770 INFO:tasks.cephfs.test_damage:Expectation on rm '200.00000000' will be 'damaged_on_start' 2023-11-15T03:03:35.770 INFO:tasks.cephfs.test_damage:Expectation on rm '200.00000001' will be 'damaged_on_start' 2023-11-15T03:03:35.771 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_openfiles.0' will be 'no damage' 2023-11-15T03:03:35.771 INFO:tasks.cephfs.test_damage:Expectation on rm '500.00000000' will be 'no damage' 2023-11-15T03:03:35.771 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds_snaptable' will be 'damaged_on_start' 2023-11-15T03:03:35.772 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_inotable' will be 'damaged_on_start' 2023-11-15T03:03:35.772 INFO:tasks.cephfs.test_damage:Expectation on rm '100.00000000' will be 'damaged_on_start' 2023-11-15T03:03:35.772 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_sessionmap' will be 'damaged_on_start' 2023-11-15T03:03:35.773 INFO:tasks.cephfs.test_damage:Expectation on rm '400.00000000' will be 'no damage' 2023-11-15T03:03:35.773 INFO:tasks.cephfs.test_damage:Expectation on rm '100.00000000.inode' will be 'damaged_on_start' 2023-11-15T03:03:35.773 INFO:tasks.cephfs.test_damage:Expectation on rm '1.00000000' will be 'damaged_on_start' 2023-11-15T03:03:35.774 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '10000000000.00000000' will be 'no damage' 2023-11-15T03:03:35.774 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '600.00000000' will be 'damaged_on_start' 2023-11-15T03:03:35.775 INFO:tasks.cephfs.test_damage:Expectation on corrupt header 'mds0_openfiles.0' will be 'no damage' 2023-11-15T03:03:35.775 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '100.00000000' will be 'damaged_on_start' 2023-11-15T03:03:35.775 INFO:tasks.cephfs.test_damage:Expectation on corrupt header 'mds0_sessionmap' will be 'damaged_on_start' 2023-11-15T03:03:35.776 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '1.00000000' will be 'damaged_on_start' 2023-11-15T03:03:35.776 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 10000000000.00000000' 2023-11-15T03:03:35.776 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:03:35.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:35.777 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:03:35.800 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:03:35.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:03:35.801 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:03:36.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.001+0000 7f4059e8f700 1 Processor -- start 2023-11-15T03:03:36.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.001+0000 7f4059e8f700 1 -- start start 2023-11-15T03:03:36.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.005+0000 7f4059e8f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40540c8e70 0x7f40540c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:36.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.005+0000 7f4059e8f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f40540c43b0 0x7f40540c4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:36.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.005+0000 7f4059e8f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40540a9b40 con 0x7f4054156f10 2023-11-15T03:03:36.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.005+0000 7f4059e8f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40540c3730 con 0x7f40540c43b0 2023-11-15T03:03:36.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.005+0000 7f4059e8f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40540bddd0 con 0x7f40540c8e70 2023-11-15T03:03:36.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.005+0000 7f4058e8d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4054156f10 0x7f40540bd6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:57214/0 (socket says 172.21.15.121:57214) 2023-11-15T03:03:36.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.005+0000 7f4058e8d700 1 -- 172.21.15.121:0/1241000797 learned_addr learned my addr 172.21.15.121:0/1241000797 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:36.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.005+0000 7f405968e700 1 --2- 172.21.15.121:0/1241000797 >> v2:172.21.15.190:3300/0 conn(0x7f40540c43b0 0x7f40540c4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:36.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.005+0000 7f4053fff700 1 --2- 172.21.15.121:0/1241000797 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40540c8e70 0x7f40540c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:36.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.005+0000 7f40537fe700 1 -- 172.21.15.121:0/1241000797 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593077043 0 0) 0x7f40540a9b40 con 0x7f4054156f10 2023-11-15T03:03:36.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.005+0000 7f40537fe700 1 -- 172.21.15.121:0/1241000797 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40440033f0 con 0x7f4054156f10 2023-11-15T03:03:36.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.005+0000 7f405968e700 1 -- 172.21.15.121:0/1241000797 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40540c8e70 msgr2=0x7f40540c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:36.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.005+0000 7f405968e700 1 --2- 172.21.15.121:0/1241000797 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40540c8e70 0x7f40540c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:36.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.005+0000 7f405968e700 1 -- 172.21.15.121:0/1241000797 >> v1:172.21.15.121:6789/0 conn(0x7f4054156f10 legacy=0x7f40540bd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:36.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.005+0000 7f405968e700 1 -- 172.21.15.121:0/1241000797 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40540c4d30 con 0x7f40540c43b0 2023-11-15T03:03:36.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.005+0000 7f405968e700 1 --2- 172.21.15.121:0/1241000797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40540c43b0 0x7f40540c4840 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f404c009e60 tx=0x7f404c009780).ready entity=mon.1 client_cookie=f0f1be605a5032ae server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:36.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.005+0000 7f40537fe700 1 -- 172.21.15.121:0/1241000797 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c00e070 con 0x7f40540c43b0 2023-11-15T03:03:36.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.009+0000 7f40537fe700 1 -- 172.21.15.121:0/1241000797 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404c00bab0 con 0x7f40540c43b0 2023-11-15T03:03:36.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.009+0000 7f40537fe700 1 -- 172.21.15.121:0/1241000797 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c00be30 con 0x7f40540c43b0 2023-11-15T03:03:36.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.009+0000 7f4059e8f700 1 -- 172.21.15.121:0/1241000797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40540c43b0 msgr2=0x7f40540c4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:36.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.009+0000 7f4059e8f700 1 --2- 172.21.15.121:0/1241000797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40540c43b0 0x7f40540c4840 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f404c009e60 tx=0x7f404c009780).stop 2023-11-15T03:03:36.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.009+0000 7f4059e8f700 1 -- 172.21.15.121:0/1241000797 shutdown_connections 2023-11-15T03:03:36.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.009+0000 7f4059e8f700 1 --2- 172.21.15.121:0/1241000797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40540c43b0 0x7f40540c4840 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:36.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.009+0000 7f4059e8f700 1 --2- 172.21.15.121:0/1241000797 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40540c8e70 0x7f40540c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:36.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.009+0000 7f4059e8f700 1 -- 172.21.15.121:0/1241000797 >> 172.21.15.121:0/1241000797 conn(0x7f40540684f0 msgr2=0x7f40540c8760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:36.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.009+0000 7f4059e8f700 1 -- 172.21.15.121:0/1241000797 shutdown_connections 2023-11-15T03:03:36.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.009+0000 7f4059e8f700 1 -- 172.21.15.121:0/1241000797 wait complete. 2023-11-15T03:03:36.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.013+0000 7f4059e8f700 1 Processor -- start 2023-11-15T03:03:36.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.013+0000 7f4059e8f700 1 -- start start 2023-11-15T03:03:36.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.013+0000 7f4059e8f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40540c43b0 0x7f4054151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:36.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.013+0000 7f4059e8f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40540c8e70 0x7f4054164980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:36.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.013+0000 7f4059e8f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4054156f10 0x7f4054159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:36.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.013+0000 7f4059e8f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4054151fc0 con 0x7f40540c43b0 2023-11-15T03:03:36.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.013+0000 7f4059e8f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4054164e70 con 0x7f40540c8e70 2023-11-15T03:03:36.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.013+0000 7f4059e8f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40541540f0 con 0x7f4054156f10 2023-11-15T03:03:36.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.013+0000 7f4058e8d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40540c43b0 0x7f4054151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:36.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.013+0000 7f4058e8d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40540c43b0 0x7f4054151ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59558/0 (socket says 172.21.15.121:59558) 2023-11-15T03:03:36.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.013+0000 7f4058e8d700 1 -- 172.21.15.121:0/3281024511 learned_addr learned my addr 172.21.15.121:0/3281024511 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:36.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.013+0000 7f405968e700 1 --2- 172.21.15.121:0/3281024511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4054156f10 0x7f4054159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:36.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.013+0000 7f4053fff700 1 --2- 172.21.15.121:0/3281024511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40540c8e70 0x7f4054164980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:36.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.017+0000 7f4058e8d700 1 -- 172.21.15.121:0/3281024511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4054156f10 msgr2=0x7f4054159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:36.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.017+0000 7f4058e8d700 1 --2- 172.21.15.121:0/3281024511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4054156f10 0x7f4054159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:36.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.017+0000 7f4058e8d700 1 -- 172.21.15.121:0/3281024511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40540c8e70 msgr2=0x7f4054164980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:36.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.017+0000 7f4058e8d700 1 --2- 172.21.15.121:0/3281024511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40540c8e70 0x7f4054164980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:36.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.017+0000 7f4058e8d700 1 -- 172.21.15.121:0/3281024511 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404c009a20 con 0x7f40540c43b0 2023-11-15T03:03:36.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.017+0000 7f4058e8d700 1 --2- 172.21.15.121:0/3281024511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40540c43b0 0x7f4054151ad0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7f404800c420 tx=0x7f4048007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:36.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.017+0000 7f4050ff9700 1 -- 172.21.15.121:0/3281024511 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40480070d0 con 0x7f40540c43b0 2023-11-15T03:03:36.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.017+0000 7f4050ff9700 1 -- 172.21.15.121:0/3281024511 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4048007230 con 0x7f40540c43b0 2023-11-15T03:03:36.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.017+0000 7f4050ff9700 1 -- 172.21.15.121:0/3281024511 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40480075b0 con 0x7f40540c43b0 2023-11-15T03:03:36.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.017+0000 7f4059e8f700 1 -- 172.21.15.121:0/3281024511 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405415a210 con 0x7f40540c43b0 2023-11-15T03:03:36.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.017+0000 7f4059e8f700 1 -- 172.21.15.121:0/3281024511 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40541537c0 con 0x7f40540c43b0 2023-11-15T03:03:36.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.017+0000 7f4059e8f700 1 -- 172.21.15.121:0/3281024511 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405406bcf0 con 0x7f40540c43b0 2023-11-15T03:03:36.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.021+0000 7f4050ff9700 1 -- 172.21.15.121:0/3281024511 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f40480059b0 con 0x7f40540c43b0 2023-11-15T03:03:36.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.021+0000 7f4050ff9700 1 --2- 172.21.15.121:0/3281024511 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f403805a510 0x7f403805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:36.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.021+0000 7f4050ff9700 1 -- 172.21.15.121:0/3281024511 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f4048073f30 con 0x7f40540c43b0 2023-11-15T03:03:36.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.021+0000 7f4053fff700 1 --2- 172.21.15.121:0/3281024511 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f403805a510 0x7f403805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:36.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.021+0000 7f4053fff700 1 --2- 172.21.15.121:0/3281024511 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f403805a510 0x7f403805c9d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f4054156980 tx=0x7f4040009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:36.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.025+0000 7f4050ff9700 1 -- 172.21.15.121:0/3281024511 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4048047600 con 0x7f40540c43b0 2023-11-15T03:03:36.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:36.217+0000 7f4059e8f700 1 -- 172.21.15.121:0/3281024511 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4054068a70 con 0x7f40540c43b0 2023-11-15T03:03:37.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.169+0000 7f4050ff9700 1 -- 172.21.15.121:0/3281024511 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7f404804ac20 con 0x7f40540c43b0 2023-11-15T03:03:37.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.173+0000 7f4059e8f700 1 -- 172.21.15.121:0/3281024511 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f403805a510 msgr2=0x7f403805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:37.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.173+0000 7f4059e8f700 1 --2- 172.21.15.121:0/3281024511 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f403805a510 0x7f403805c9d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f4054156980 tx=0x7f4040009060).stop 2023-11-15T03:03:37.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.173+0000 7f4059e8f700 1 -- 172.21.15.121:0/3281024511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40540c43b0 msgr2=0x7f4054151ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:37.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.173+0000 7f4059e8f700 1 --2- 172.21.15.121:0/3281024511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40540c43b0 0x7f4054151ad0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7f404800c420 tx=0x7f4048007c80).stop 2023-11-15T03:03:37.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.173+0000 7f4059e8f700 1 -- 172.21.15.121:0/3281024511 shutdown_connections 2023-11-15T03:03:37.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.173+0000 7f4059e8f700 1 --2- 172.21.15.121:0/3281024511 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f403805a510 0x7f403805c9d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:37.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.173+0000 7f4059e8f700 1 --2- 172.21.15.121:0/3281024511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4054156f10 0x7f4054159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:37.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.173+0000 7f4059e8f700 1 --2- 172.21.15.121:0/3281024511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40540c8e70 0x7f4054164980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:37.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.173+0000 7f4059e8f700 1 --2- 172.21.15.121:0/3281024511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40540c43b0 0x7f4054151ad0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:37.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.173+0000 7f4059e8f700 1 -- 172.21.15.121:0/3281024511 >> 172.21.15.121:0/3281024511 conn(0x7f40540684f0 msgr2=0x7f40540bd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:37.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.173+0000 7f4059e8f700 1 -- 172.21.15.121:0/3281024511 shutdown_connections 2023-11-15T03:03:37.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.173+0000 7f4059e8f700 1 -- 172.21.15.121:0/3281024511 wait complete. 2023-11-15T03:03:37.181 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:03:37.201 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:03:37.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.357+0000 7fac659b9700 1 Processor -- start 2023-11-15T03:03:37.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac659b9700 1 -- start start 2023-11-15T03:03:37.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac659b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac60062710 0x7fac6014fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:37.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac659b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fac60150110 0x7fac60065680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:37.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac659b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac600bd3f0 con 0x7fac60062330 2023-11-15T03:03:37.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac659b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac60062b70 con 0x7fac60150110 2023-11-15T03:03:37.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac659b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac600c14d0 con 0x7fac60062710 2023-11-15T03:03:37.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac649b7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fac60062330 0x7fac600bcce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:57218/0 (socket says 172.21.15.121:57218) 2023-11-15T03:03:37.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac649b7700 1 -- 172.21.15.121:0/2779691108 learned_addr learned my addr 172.21.15.121:0/2779691108 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:37.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac651b8700 1 --2- 172.21.15.121:0/2779691108 >> v2:172.21.15.121:3300/0 conn(0x7fac60150110 0x7fac60065680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:37.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac5ffff700 1 --2- 172.21.15.121:0/2779691108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac60062710 0x7fac6014fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:37.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac5f7fe700 1 -- 172.21.15.121:0/2779691108 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4014531873 0 0) 0x7fac600bd3f0 con 0x7fac60062330 2023-11-15T03:03:37.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac5f7fe700 1 -- 172.21.15.121:0/2779691108 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac500033f0 con 0x7fac60062330 2023-11-15T03:03:37.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac651b8700 1 -- 172.21.15.121:0/2779691108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac60062710 msgr2=0x7fac6014fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:37.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac651b8700 1 --2- 172.21.15.121:0/2779691108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac60062710 0x7fac6014fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:37.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac651b8700 1 -- 172.21.15.121:0/2779691108 >> v1:172.21.15.121:6789/0 conn(0x7fac60062330 legacy=0x7fac600bcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:37.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac651b8700 1 -- 172.21.15.121:0/2779691108 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac600c2330 con 0x7fac60150110 2023-11-15T03:03:37.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac651b8700 1 --2- 172.21.15.121:0/2779691108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac60150110 0x7fac60065680 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7fac58009e60 tx=0x7fac58009780).ready entity=mon.0 client_cookie=5c7cab4987b18536 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:37.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac5f7fe700 1 -- 172.21.15.121:0/2779691108 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5800e070 con 0x7fac60150110 2023-11-15T03:03:37.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac5f7fe700 1 -- 172.21.15.121:0/2779691108 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5800bab0 con 0x7fac60150110 2023-11-15T03:03:37.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.361+0000 7fac5f7fe700 1 -- 172.21.15.121:0/2779691108 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5800be30 con 0x7fac60150110 2023-11-15T03:03:37.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.365+0000 7fac659b9700 1 -- 172.21.15.121:0/2779691108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac60150110 msgr2=0x7fac60065680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:37.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.365+0000 7fac659b9700 1 --2- 172.21.15.121:0/2779691108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac60150110 0x7fac60065680 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7fac58009e60 tx=0x7fac58009780).stop 2023-11-15T03:03:37.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.365+0000 7fac659b9700 1 -- 172.21.15.121:0/2779691108 shutdown_connections 2023-11-15T03:03:37.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.365+0000 7fac659b9700 1 --2- 172.21.15.121:0/2779691108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac60150110 0x7fac60065680 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:37.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.365+0000 7fac659b9700 1 --2- 172.21.15.121:0/2779691108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac60062710 0x7fac6014fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:37.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.365+0000 7fac659b9700 1 -- 172.21.15.121:0/2779691108 >> 172.21.15.121:0/2779691108 conn(0x7fac60067a60 msgr2=0x7fac60067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:37.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.365+0000 7fac659b9700 1 -- 172.21.15.121:0/2779691108 shutdown_connections 2023-11-15T03:03:37.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.365+0000 7fac659b9700 1 -- 172.21.15.121:0/2779691108 wait complete. 2023-11-15T03:03:37.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.365+0000 7fac659b9700 1 Processor -- start 2023-11-15T03:03:37.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.365+0000 7fac659b9700 1 -- start start 2023-11-15T03:03:37.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac659b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac60062330 0x7fac60147140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:37.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac659b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac60062710 0x7fac601498e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:37.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac659b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac60150110 0x7fac60149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:37.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac659b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6014a2c0 con 0x7fac60062710 2023-11-15T03:03:37.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac659b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac60147630 con 0x7fac60062330 2023-11-15T03:03:37.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac659b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac600aa2d0 con 0x7fac60150110 2023-11-15T03:03:37.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac5ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac60062710 0x7fac601498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:37.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac5ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac60062710 0x7fac601498e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59590/0 (socket says 172.21.15.121:59590) 2023-11-15T03:03:37.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac5ffff700 1 -- 172.21.15.121:0/4238642345 learned_addr learned my addr 172.21.15.121:0/4238642345 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:37.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac651b8700 1 --2- 172.21.15.121:0/4238642345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac60150110 0x7fac60149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:37.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac649b7700 1 --2- 172.21.15.121:0/4238642345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac60062330 0x7fac60147140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:37.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac5ffff700 1 -- 172.21.15.121:0/4238642345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac60150110 msgr2=0x7fac60149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:37.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac5ffff700 1 --2- 172.21.15.121:0/4238642345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac60150110 0x7fac60149dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:37.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac5ffff700 1 -- 172.21.15.121:0/4238642345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac60062330 msgr2=0x7fac60147140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:37.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac5ffff700 1 --2- 172.21.15.121:0/4238642345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac60062330 0x7fac60147140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:37.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac5ffff700 1 -- 172.21.15.121:0/4238642345 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac58009a20 con 0x7fac60062710 2023-11-15T03:03:37.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac649b7700 1 --2- 172.21.15.121:0/4238642345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac60062330 0x7fac60147140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:37.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac5ffff700 1 --2- 172.21.15.121:0/4238642345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac60062710 0x7fac601498e0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7fac4c00e8d0 tx=0x7fac4c006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:37.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac5cff9700 1 -- 172.21.15.121:0/4238642345 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c012600 con 0x7fac60062710 2023-11-15T03:03:37.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac5cff9700 1 -- 172.21.15.121:0/4238642345 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac4c012760 con 0x7fac60062710 2023-11-15T03:03:37.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac659b9700 1 -- 172.21.15.121:0/4238642345 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac600aa5b0 con 0x7fac60062710 2023-11-15T03:03:37.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac5cff9700 1 -- 172.21.15.121:0/4238642345 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c012ae0 con 0x7fac60062710 2023-11-15T03:03:37.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.369+0000 7fac659b9700 1 -- 172.21.15.121:0/4238642345 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac600aab70 con 0x7fac60062710 2023-11-15T03:03:37.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.373+0000 7fac659b9700 1 -- 172.21.15.121:0/4238642345 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6006bcf0 con 0x7fac60062710 2023-11-15T03:03:37.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.373+0000 7fac5cff9700 1 -- 172.21.15.121:0/4238642345 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fac4c01b400 con 0x7fac60062710 2023-11-15T03:03:37.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.373+0000 7fac5cff9700 1 --2- 172.21.15.121:0/4238642345 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fac4405a510 0x7fac4405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:37.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.373+0000 7fac5cff9700 1 -- 172.21.15.121:0/4238642345 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fac4c074c90 con 0x7fac60062710 2023-11-15T03:03:37.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.373+0000 7fac649b7700 1 --2- 172.21.15.121:0/4238642345 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fac4405a510 0x7fac4405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:37.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.377+0000 7fac649b7700 1 --2- 172.21.15.121:0/4238642345 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fac4405a510 0x7fac4405c9d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fac54007fd0 tx=0x7fac54009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:37.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.381+0000 7fac5cff9700 1 -- 172.21.15.121:0/4238642345 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fac4c0483c0 con 0x7fac60062710 2023-11-15T03:03:37.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:37.601+0000 7fac659b9700 1 -- 172.21.15.121:0/4238642345 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fac601496c0 con 0x7fac60062710 2023-11-15T03:03:38.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.177+0000 7fac5cff9700 1 -- 172.21.15.121:0/4238642345 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v85) v1 ==== 107+0+0 (secure 0 0 0) 0x7fac4c016030 con 0x7fac60062710 2023-11-15T03:03:38.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.181+0000 7fac659b9700 1 -- 172.21.15.121:0/4238642345 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fac4405a510 msgr2=0x7fac4405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.181+0000 7fac659b9700 1 --2- 172.21.15.121:0/4238642345 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fac4405a510 0x7fac4405c9d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fac54007fd0 tx=0x7fac54009040).stop 2023-11-15T03:03:38.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.181+0000 7fac659b9700 1 -- 172.21.15.121:0/4238642345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac60062710 msgr2=0x7fac601498e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.181+0000 7fac659b9700 1 --2- 172.21.15.121:0/4238642345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac60062710 0x7fac601498e0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7fac4c00e8d0 tx=0x7fac4c006f40).stop 2023-11-15T03:03:38.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.181+0000 7fac659b9700 1 -- 172.21.15.121:0/4238642345 shutdown_connections 2023-11-15T03:03:38.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.181+0000 7fac659b9700 1 --2- 172.21.15.121:0/4238642345 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fac4405a510 0x7fac4405c9d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.181+0000 7fac659b9700 1 --2- 172.21.15.121:0/4238642345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac60150110 0x7fac60149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.181+0000 7fac659b9700 1 --2- 172.21.15.121:0/4238642345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac60062710 0x7fac601498e0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.181+0000 7fac659b9700 1 --2- 172.21.15.121:0/4238642345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac60062330 0x7fac60147140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.181+0000 7fac659b9700 1 -- 172.21.15.121:0/4238642345 >> 172.21.15.121:0/4238642345 conn(0x7fac60067a60 msgr2=0x7fac600bce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:38.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.185+0000 7fac659b9700 1 -- 172.21.15.121:0/4238642345 shutdown_connections 2023-11-15T03:03:38.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.185+0000 7fac659b9700 1 -- 172.21.15.121:0/4238642345 wait complete. 2023-11-15T03:03:38.191 INFO:teuthology.orchestra.run.smithi121.stderr:nothing to do: rank is not damaged 2023-11-15T03:03:38.210 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:03:38.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.265+0000 7f28710d9d00 1 Processor -- start 2023-11-15T03:03:38.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.265+0000 7f28710d9d00 1 -- start start 2023-11-15T03:03:38.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.265+0000 7f28710d9d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c83cb8d920 0x55c83cc62b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.265+0000 7f28710d9d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c83cbd52f0 0x55c83cbba250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.265+0000 7f28710d9d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c83cc63a20 0x55c83cc58de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.265+0000 7f28710d9d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c83cb37cc0 con 0x55c83cb8d920 2023-11-15T03:03:38.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.265+0000 7f28710d9d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c83cb02220 con 0x55c83cbd52f0 2023-11-15T03:03:38.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.265+0000 7f28710d9d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c83cb8c220 con 0x55c83cc63a20 2023-11-15T03:03:38.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.265+0000 7f286ea32700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c83cb8d920 0x55c83cc62b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.265+0000 7f2866231700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c83cbd52f0 0x55c83cbba250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.265+0000 7f286ea32700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c83cb8d920 0x55c83cc62b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59600/0 (socket says 172.21.15.121:59600) 2023-11-15T03:03:38.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.265+0000 7f2866231700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c83cbd52f0 0x55c83cbba250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:55236/0 (socket says 172.21.15.121:55236) 2023-11-15T03:03:38.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.265+0000 7f286ea32700 1 -- 172.21.15.121:0/736932057 learned_addr learned my addr 172.21.15.121:0/736932057 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:38.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.265+0000 7f286f233700 1 --2- 172.21.15.121:0/736932057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c83cc63a20 0x55c83cc58de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.265+0000 7f286ea32700 1 -- 172.21.15.121:0/736932057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c83cc63a20 msgr2=0x55c83cc58de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:38.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.265+0000 7f286ea32700 1 --2- 172.21.15.121:0/736932057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c83cc63a20 0x55c83cc58de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.265+0000 7f286ea32700 1 -- 172.21.15.121:0/736932057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c83cbd52f0 msgr2=0x55c83cbba250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:38.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.265+0000 7f286ea32700 1 --2- 172.21.15.121:0/736932057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c83cbd52f0 0x55c83cbba250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.265+0000 7f286ea32700 1 -- 172.21.15.121:0/736932057 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2860005cf0 con 0x55c83cb8d920 2023-11-15T03:03:38.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.269+0000 7f286ea32700 1 --2- 172.21.15.121:0/736932057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c83cb8d920 0x55c83cc62b50 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x55c83cb78c20 tx=0x7f2860016000).ready entity=mon.0 client_cookie=4b0a97209c66a477 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:38.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.269+0000 7f286da30700 1 -- 172.21.15.121:0/736932057 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28600086a0 con 0x55c83cb8d920 2023-11-15T03:03:38.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.269+0000 7f286da30700 1 -- 172.21.15.121:0/736932057 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2860000bc0 con 0x55c83cb8d920 2023-11-15T03:03:38.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.269+0000 7f286da30700 1 -- 172.21.15.121:0/736932057 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2860008970 con 0x55c83cb8d920 2023-11-15T03:03:38.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.269+0000 7f28710d9d00 1 -- 172.21.15.121:0/736932057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c83cb8d920 msgr2=0x55c83cc62b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.269+0000 7f28710d9d00 1 --2- 172.21.15.121:0/736932057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c83cb8d920 0x55c83cc62b50 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x55c83cb78c20 tx=0x7f2860016000).stop 2023-11-15T03:03:38.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.269+0000 7f28710d9d00 1 -- 172.21.15.121:0/736932057 shutdown_connections 2023-11-15T03:03:38.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.269+0000 7f28710d9d00 1 --2- 172.21.15.121:0/736932057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c83cc63a20 0x55c83cc58de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.269+0000 7f28710d9d00 1 --2- 172.21.15.121:0/736932057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c83cbd52f0 0x55c83cbba250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.269+0000 7f28710d9d00 1 --2- 172.21.15.121:0/736932057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c83cb8d920 0x55c83cc62b50 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.269+0000 7f28710d9d00 1 -- 172.21.15.121:0/736932057 >> 172.21.15.121:0/736932057 conn(0x55c83cbd38f0 msgr2=0x55c83cbbc820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:38.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.269+0000 7f28710d9d00 1 -- 172.21.15.121:0/736932057 shutdown_connections 2023-11-15T03:03:38.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.269+0000 7f28710d9d00 1 -- 172.21.15.121:0/736932057 wait complete. 2023-11-15T03:03:38.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.269+0000 7f28710d9d00 1 Processor -- start 2023-11-15T03:03:38.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.269+0000 7f28710d9d00 1 -- start start 2023-11-15T03:03:38.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f28710d9d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c83cb8d920 0x55c83cc63080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f28710d9d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c83cbd52f0 0x55c83cc5f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f28710d9d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c83cc63a20 0x55c83cc55830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f28710d9d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c83cc63570 con 0x55c83cb8d920 2023-11-15T03:03:38.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f28710d9d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c83cc5fdd0 con 0x55c83cbd52f0 2023-11-15T03:03:38.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f28710d9d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c83cbce970 con 0x55c83cc63a20 2023-11-15T03:03:38.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f286ea32700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c83cb8d920 0x55c83cc63080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f286ea32700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c83cb8d920 0x55c83cc63080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59612/0 (socket says 172.21.15.121:59612) 2023-11-15T03:03:38.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 learned_addr learned my addr 172.21.15.121:0/2359048650 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:38.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f2866231700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c83cbd52f0 0x55c83cc5f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f286f233700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c83cc63a20 0x55c83cc55830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c83cc63a20 msgr2=0x55c83cc55830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f286ea32700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c83cc63a20 0x55c83cc55830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c83cbd52f0 msgr2=0x55c83cc5f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f286ea32700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c83cbd52f0 0x55c83cc5f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c83cb2b770 con 0x55c83cb8d920 2023-11-15T03:03:38.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f286ea32700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c83cb8d920 0x55c83cc63080 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7f2860021040 tx=0x7f286001c000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:38.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f2867fff700 1 -- 172.21.15.121:0/2359048650 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286001e680 con 0x55c83cb8d920 2023-11-15T03:03:38.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f2867fff700 1 -- 172.21.15.121:0/2359048650 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286001e7e0 con 0x55c83cb8d920 2023-11-15T03:03:38.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c83cafaa70 con 0x55c83cb8d920 2023-11-15T03:03:38.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f2867fff700 1 -- 172.21.15.121:0/2359048650 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286001eba0 con 0x55c83cb8d920 2023-11-15T03:03:38.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.273+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c83cb06010 con 0x55c83cb8d920 2023-11-15T03:03:38.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.277+0000 7f2867fff700 1 -- 172.21.15.121:0/2359048650 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2860025040 con 0x55c83cb8d920 2023-11-15T03:03:38.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.277+0000 7f2867fff700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2850058ff0 0x7f285005b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.277+0000 7f2867fff700 1 -- 172.21.15.121:0/2359048650 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f28600756c0 con 0x55c83cb8d920 2023-11-15T03:03:38.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.277+0000 7f2866231700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2850058ff0 0x7f285005b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.292 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:03:38.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.277+0000 7f28710d9d00 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c83cc55d20 0x55c83cbcf130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.277+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cbcf620 con 0x55c83cc55d20 2023-11-15T03:03:38.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.281+0000 7f286f233700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c83cc55d20 0x55c83cbcf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.281+0000 7f2866231700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2850058ff0 0x7f285005b4b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x55c83cb8f600 tx=0x7f2858000bc0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:38.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.281+0000 7f286f233700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c83cc55d20 0x55c83cbcf130 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:38.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.281+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v93'5 uv2 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc55d20 2023-11-15T03:03:38.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.281+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cbcf9d0 con 0x55c83cc55d20 2023-11-15T03:03:38.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.285+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v93'6 uv6 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc55d20 2023-11-15T03:03:38.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.285+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cbcfd80 con 0x55c83cc55d20 2023-11-15T03:03:38.296 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:03:38.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.285+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v93'7 uv7 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc55d20 2023-11-15T03:03:38.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.285+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cbd0130 con 0x55c83cc55d20 2023-11-15T03:03:38.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.289+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v93'8 uv8 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc55d20 2023-11-15T03:03:38.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.289+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc7fd40 con 0x55c83cc55d20 2023-11-15T03:03:38.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.289+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v93'9 uv9 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc55d20 2023-11-15T03:03:38.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.289+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc7f220 con 0x55c83cc55d20 2023-11-15T03:03:38.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.293+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v93'10 uv10 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc55d20 2023-11-15T03:03:38.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.293+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc7f650 con 0x55c83cc55d20 2023-11-15T03:03:38.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.293+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v93'11 uv4 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc55d20 2023-11-15T03:03:38.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.293+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc800f0 con 0x55c83cc55d20 2023-11-15T03:03:38.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.297+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v93'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc55d20 2023-11-15T03:03:38.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.297+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc800f0 con 0x55c83cc55d20 2023-11-15T03:03:38.303 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:03:38.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.297+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v93'13 uv13 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc55d20 2023-11-15T03:03:38.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.297+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc81100 con 0x55c83cc55d20 2023-11-15T03:03:38.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.301+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v93'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc55d20 2023-11-15T03:03:38.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.301+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc81100 con 0x55c83cc55d20 2023-11-15T03:03:38.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.301+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v93'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc55d20 2023-11-15T03:03:38.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.301+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc800f0 con 0x55c83cc55d20 2023-11-15T03:03:38.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.305+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v93'16 uv16 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc55d20 2023-11-15T03:03:38.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.305+0000 7f28710d9d00 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c83cc80600 0x55c83cc829f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.305+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc82ee0 con 0x55c83cc80600 2023-11-15T03:03:38.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.305+0000 7f286ea32700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c83cc80600 0x55c83cc829f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.305+0000 7f286ea32700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c83cc80600 0x55c83cc829f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:38.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.309+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v93'7 uv6 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.309+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc83290 con 0x55c83cc80600 2023-11-15T03:03:38.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.313+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v93'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.313+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc83640 con 0x55c83cc80600 2023-11-15T03:03:38.320 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:03:38.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.313+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v93'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.317+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc84a00 con 0x55c83cc80600 2023-11-15T03:03:38.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.317+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v93'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.317+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc84db0 con 0x55c83cc80600 2023-11-15T03:03:38.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.321+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v93'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.321+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc839f0 con 0x55c83cc80600 2023-11-15T03:03:38.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.321+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v93'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.321+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc842e0 con 0x55c83cc80600 2023-11-15T03:03:38.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.325+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v93'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.325+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc85160 con 0x55c83cc80600 2023-11-15T03:03:38.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.329+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v93'14 uv2 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.329+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc85160 con 0x55c83cc80600 2023-11-15T03:03:38.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.329+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v93'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.329+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc85160 con 0x55c83cc80600 2023-11-15T03:03:38.339 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:03:38.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.333+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v93'16 uv16 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.333+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc86170 con 0x55c83cc80600 2023-11-15T03:03:38.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.337+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v93'17 uv17 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.337+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc86170 con 0x55c83cc80600 2023-11-15T03:03:38.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.341+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v93'18 uv18 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.341+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc85160 con 0x55c83cc80600 2023-11-15T03:03:38.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.341+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v93'19 uv19 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.341+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc85160 con 0x55c83cc80600 2023-11-15T03:03:38.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.345+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v93'20 uv4 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.345+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc85160 con 0x55c83cc80600 2023-11-15T03:03:38.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.349+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v93'21 uv21 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.349+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc85160 con 0x55c83cc80600 2023-11-15T03:03:38.358 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:03:38.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.353+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v93'22 uv22 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.353+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc86170 con 0x55c83cc80600 2023-11-15T03:03:38.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.357+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v93'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.357+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc86170 con 0x55c83cc80600 2023-11-15T03:03:38.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.357+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v93'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.357+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc85160 con 0x55c83cc80600 2023-11-15T03:03:38.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.361+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v93'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.361+0000 7f28710d9d00 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c83cc85670 0x55c83cc87a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.361+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc87f50 con 0x55c83cc85670 2023-11-15T03:03:38.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.361+0000 7f2866231700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c83cc85670 0x55c83cc87a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.361+0000 7f2866231700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c83cc85670 0x55c83cc87a60 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:38.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.365+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v93'6 uv3 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc85670 2023-11-15T03:03:38.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.365+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc88300 con 0x55c83cc85670 2023-11-15T03:03:38.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.369+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v93'7 uv7 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc85670 2023-11-15T03:03:38.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.369+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc886b0 con 0x55c83cc85670 2023-11-15T03:03:38.375 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:03:38.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.369+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v93'8 uv8 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc85670 2023-11-15T03:03:38.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.369+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc88a60 con 0x55c83cc85670 2023-11-15T03:03:38.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.373+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v93'9 uv9 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc85670 2023-11-15T03:03:38.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.373+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc83ef0 con 0x55c83cc85670 2023-11-15T03:03:38.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.377+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v93'10 uv10 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc85670 2023-11-15T03:03:38.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.377+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc893a0 con 0x55c83cc85670 2023-11-15T03:03:38.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.377+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v93'11 uv5 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc85670 2023-11-15T03:03:38.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.377+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc89750 con 0x55c83cc85670 2023-11-15T03:03:38.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.381+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v93'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc85670 2023-11-15T03:03:38.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.381+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc89b00 con 0x55c83cc85670 2023-11-15T03:03:38.395 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:03:38.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.385+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v93'13 uv13 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc85670 2023-11-15T03:03:38.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.385+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc89b00 con 0x55c83cc85670 2023-11-15T03:03:38.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.389+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v93'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc85670 2023-11-15T03:03:38.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.389+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc88ee0 con 0x55c83cc85670 2023-11-15T03:03:38.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.389+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v93'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc85670 2023-11-15T03:03:38.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.389+0000 7f28710d9d00 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c83cc8a500 0x55c83cc8c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.389+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc8cde0 con 0x55c83cc8a500 2023-11-15T03:03:38.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.389+0000 7f286f233700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c83cc8a500 0x55c83cc8c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.393+0000 7f286f233700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c83cc8a500 0x55c83cc8c8f0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:38.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.393+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v93'14 uv13 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.393+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc8cde0 con 0x55c83cc8a500 2023-11-15T03:03:38.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.397+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v93'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.397+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc8cde0 con 0x55c83cc8a500 2023-11-15T03:03:38.407 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:03:38.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.401+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v93'16 uv16 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.401+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc90e70 con 0x55c83cc8a500 2023-11-15T03:03:38.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.405+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v93'17 uv17 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.405+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc90e70 con 0x55c83cc8a500 2023-11-15T03:03:38.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.409+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v93'18 uv18 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.409+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc8cde0 con 0x55c83cc8a500 2023-11-15T03:03:38.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.409+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v93'19 uv2 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.409+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc8cde0 con 0x55c83cc8a500 2023-11-15T03:03:38.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.413+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v93'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.413+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc8cde0 con 0x55c83cc8a500 2023-11-15T03:03:38.422 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:03:38.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.417+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v93'21 uv21 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.417+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc8ddf0 con 0x55c83cc8a500 2023-11-15T03:03:38.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.421+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v93'22 uv22 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.421+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc8ddf0 con 0x55c83cc8a500 2023-11-15T03:03:38.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.421+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v93'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.421+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc8cde0 con 0x55c83cc8a500 2023-11-15T03:03:38.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.425+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v93'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.425+0000 7f28710d9d00 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c83cc8d460 0x55c83cc8f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.425+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc8fe70 con 0x55c83cc8d460 2023-11-15T03:03:38.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.425+0000 7f286ea32700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c83cc8d460 0x55c83cc8f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.429+0000 7f286ea32700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c83cc8d460 0x55c83cc8f850 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:38.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.429+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v93'3 uv2 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc8d460 2023-11-15T03:03:38.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.429+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc8ff50 con 0x55c83cc8d460 2023-11-15T03:03:38.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.433+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v93'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc8d460 2023-11-15T03:03:38.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.433+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc90380 con 0x55c83cc8d460 2023-11-15T03:03:38.453 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:03:38.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.437+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v93'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc8d460 2023-11-15T03:03:38.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.437+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc91390 con 0x55c83cc8d460 2023-11-15T03:03:38.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.441+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v93'6 uv6 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc8d460 2023-11-15T03:03:38.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.441+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc91390 con 0x55c83cc8d460 2023-11-15T03:03:38.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.441+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v93'7 uv7 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc8d460 2023-11-15T03:03:38.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.441+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc90380 con 0x55c83cc8d460 2023-11-15T03:03:38.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.445+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v93'8 uv8 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc8d460 2023-11-15T03:03:38.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.445+0000 7f28710d9d00 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c83cc90890 0x55c83cc92c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.445+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc93330 con 0x55c83cc90890 2023-11-15T03:03:38.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.445+0000 7f2866231700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c83cc90890 0x55c83cc92c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.445+0000 7f2866231700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c83cc90890 0x55c83cc92c80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:38.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.449+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v93'3 uv2 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.449+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc93710 con 0x55c83cc90890 2023-11-15T03:03:38.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.453+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v93'4 uv4 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.453+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc93710 con 0x55c83cc90890 2023-11-15T03:03:38.462 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:03:38.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.457+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v93'5 uv5 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.457+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc93710 con 0x55c83cc90890 2023-11-15T03:03:38.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.461+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v93'6 uv6 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.461+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc93710 con 0x55c83cc90890 2023-11-15T03:03:38.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.461+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v93'3 uv2 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.461+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc93710 con 0x55c83cc90890 2023-11-15T03:03:38.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.465+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v93'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.465+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc93710 con 0x55c83cc90890 2023-11-15T03:03:38.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.465+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v93'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.479 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:03:38.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.469+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc94720 con 0x55c83cc90890 2023-11-15T03:03:38.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.469+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v93'6 uv6 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.469+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc94720 con 0x55c83cc90890 2023-11-15T03:03:38.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.473+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v93'7 uv7 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.473+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc93710 con 0x55c83cc90890 2023-11-15T03:03:38.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.477+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v93'8 uv8 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.477+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc93710 con 0x55c83cc80600 2023-11-15T03:03:38.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.477+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v93'4 uv2 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.477+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc93710 con 0x55c83cc80600 2023-11-15T03:03:38.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.481+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v93'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.481+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc93710 con 0x55c83cc80600 2023-11-15T03:03:38.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.485+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v93'6 uv6 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.492 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:03:38.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.485+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc94720 con 0x55c83cc80600 2023-11-15T03:03:38.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.489+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v93'7 uv7 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.489+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc94720 con 0x55c83cc80600 2023-11-15T03:03:38.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.493+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v93'8 uv8 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.493+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc93710 con 0x55c83cc80600 2023-11-15T03:03:38.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.497+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v93'9 uv9 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.497+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc93710 con 0x55c83cc80600 2023-11-15T03:03:38.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.497+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v93'10 uv3 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.497+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc93710 con 0x55c83cc80600 2023-11-15T03:03:38.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.501+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v93'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.501+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc93710 con 0x55c83cc80600 2023-11-15T03:03:38.511 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:03:38.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.505+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v93'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.505+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc93710 con 0x55c83cc80600 2023-11-15T03:03:38.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.509+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v93'13 uv13 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.509+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc80600 2023-11-15T03:03:38.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.513+0000 7f286ea32700 1 -- 172.21.15.121:0/2359048650 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v93'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f28600064b0 con 0x55c83cc80600 2023-11-15T03:03:38.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.513+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc85670 2023-11-15T03:03:38.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.513+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v93'2 uv1 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc85670 2023-11-15T03:03:38.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.513+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc85670 2023-11-15T03:03:38.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.517+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v93'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc85670 2023-11-15T03:03:38.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.517+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc85670 2023-11-15T03:03:38.529 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:03:38.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.517+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v93'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc85670 2023-11-15T03:03:38.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.521+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc85670 2023-11-15T03:03:38.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.525+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v93'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc85670 2023-11-15T03:03:38.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.525+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc85670 2023-11-15T03:03:38.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.525+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v93'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc85670 2023-11-15T03:03:38.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.525+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc90890 2023-11-15T03:03:38.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.529+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v93'3 uv2 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.529+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc90890 2023-11-15T03:03:38.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.533+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v93'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.533+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc90890 2023-11-15T03:03:38.543 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:03:38.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.537+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v93'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.537+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc90890 2023-11-15T03:03:38.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.541+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v93'6 uv6 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.541+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc90890 2023-11-15T03:03:38.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.541+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v93'7 uv7 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.541+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc90890 2023-11-15T03:03:38.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.545+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v93'8 uv8 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.545+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc90890 2023-11-15T03:03:38.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.549+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v93'3 uv2 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.549+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc90890 2023-11-15T03:03:38.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.553+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v93'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.553+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc90890 2023-11-15T03:03:38.562 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:03:38.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.557+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v93'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.557+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc90890 2023-11-15T03:03:38.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.557+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v93'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.557+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc90890 2023-11-15T03:03:38.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.561+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v93'7 uv7 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.561+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc90890 2023-11-15T03:03:38.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.565+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v93'2 uv1 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.565+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc90890 2023-11-15T03:03:38.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.569+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v93'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.569+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc90890 2023-11-15T03:03:38.576 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:03:38.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.569+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v93'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.569+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc89b80 con 0x55c83cc90890 2023-11-15T03:03:38.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.573+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v93'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.573+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc96640 con 0x55c83cc90890 2023-11-15T03:03:38.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.577+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v93'6 uv6 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.577+0000 7f28710d9d00 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c83cc96820 0x55c83cc96c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.577+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc96820 2023-11-15T03:03:38.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.577+0000 7f286f233700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c83cc96820 0x55c83cc96c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.577+0000 7f286f233700 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c83cc96820 0x55c83cc96c60 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:38.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.581+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v93'3 uv2 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc96820 2023-11-15T03:03:38.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.581+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc96820 2023-11-15T03:03:38.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.585+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v93'4 uv4 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc96820 2023-11-15T03:03:38.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.585+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc96820 2023-11-15T03:03:38.593 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:03:38.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.585+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v93'5 uv5 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc96820 2023-11-15T03:03:38.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.585+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc96820 2023-11-15T03:03:38.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.589+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v93'6 uv6 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc96820 2023-11-15T03:03:38.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.589+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc90890 2023-11-15T03:03:38.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.593+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v93'3 uv2 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.593+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc90890 2023-11-15T03:03:38.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.597+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v93'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.597+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc90890 2023-11-15T03:03:38.608 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:03:38.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.597+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v93'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.597+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc98160 con 0x55c83cc90890 2023-11-15T03:03:38.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.601+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v93'6 uv6 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.601+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc98160 con 0x55c83cc90890 2023-11-15T03:03:38.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.601+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v93'7 uv7 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.605+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc90890 2023-11-15T03:03:38.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.605+0000 7f2866231700 1 -- 172.21.15.121:0/2359048650 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v93'8 uv8 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f285800e900 con 0x55c83cc90890 2023-11-15T03:03:38.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.605+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc8a500 2023-11-15T03:03:38.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.609+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v93'2 uv1 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.609+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc8a500 2023-11-15T03:03:38.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.609+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v93'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.609+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc8a500 2023-11-15T03:03:38.625 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:03:38.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.617+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v93'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.617+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc8a500 2023-11-15T03:03:38.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.617+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v93'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.617+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc8a500 2023-11-15T03:03:38.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.625+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v93'6 uv6 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.625+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc8a500 2023-11-15T03:03:38.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.625+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v93'6 uv3 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.625+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc8a500 2023-11-15T03:03:38.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.629+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v93'7 uv7 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.629+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc8a500 2023-11-15T03:03:38.637 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:03:38.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.633+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v93'8 uv8 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.633+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc8a500 2023-11-15T03:03:38.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.633+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v93'9 uv9 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.633+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc8a500 2023-11-15T03:03:38.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.637+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v93'10 uv10 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.637+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc8a500 2023-11-15T03:03:38.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.641+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v93'11 uv5 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.641+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc8a500 2023-11-15T03:03:38.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.645+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v93'12 uv12 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.645+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc8a500 2023-11-15T03:03:38.651 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:03:38.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.645+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v93'13 uv13 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.645+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc98160 con 0x55c83cc8a500 2023-11-15T03:03:38.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.649+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v93'14 uv14 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.649+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc98160 con 0x55c83cc8a500 2023-11-15T03:03:38.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.653+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v93'15 uv15 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.653+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc97150 con 0x55c83cc8a500 2023-11-15T03:03:38.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.653+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v93'16 uv16 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.653+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e93) v8 -- 0x55c83cc97640 con 0x55c83cc8a500 2023-11-15T03:03:38.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v93'17 uv17 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2868002980 con 0x55c83cc8a500 2023-11-15T03:03:38.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c83cc85670 msgr2=0x55c83cc87a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c83cc85670 0x55c83cc87a60 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c83cc55d20 msgr2=0x55c83cbcf130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c83cc55d20 0x55c83cbcf130 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c83cc8d460 msgr2=0x55c83cc8f850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c83cc8d460 0x55c83cc8f850 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c83cc80600 msgr2=0x55c83cc829f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c83cc80600 0x55c83cc829f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c83cc96820 msgr2=0x55c83cc96c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c83cc96820 0x55c83cc96c60 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c83cc8a500 msgr2=0x55c83cc8c8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c83cc8a500 0x55c83cc8c8f0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c83cc90890 msgr2=0x55c83cc92c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c83cc90890 0x55c83cc92c80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 reap_dead start 2023-11-15T03:03:38.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 reap_dead start 2023-11-15T03:03:38.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 reap_dead start 2023-11-15T03:03:38.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f286f233700 1 -- 172.21.15.121:0/2359048650 reap_dead start 2023-11-15T03:03:38.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2850058ff0 msgr2=0x7f285005b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2850058ff0 0x7f285005b4b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x55c83cb8f600 tx=0x7f2858000bc0).stop 2023-11-15T03:03:38.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c83cb8d920 msgr2=0x55c83cc63080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c83cb8d920 0x55c83cc63080 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7f2860021040 tx=0x7f286001c000).stop 2023-11-15T03:03:38.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 shutdown_connections 2023-11-15T03:03:38.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2850058ff0 0x7f285005b4b0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 --2- 172.21.15.121:0/2359048650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c83cb8d920 0x55c83cc63080 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 >> 172.21.15.121:0/2359048650 conn(0x55c83cbd38f0 msgr2=0x55c83cb89d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:38.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.657+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 shutdown_connections 2023-11-15T03:03:38.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.661+0000 7f28710d9d00 1 -- 172.21.15.121:0/2359048650 wait complete. 2023-11-15T03:03:38.674 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2023-11-15T03:03:38.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.729+0000 7f9e1072fd00 1 Processor -- start 2023-11-15T03:03:38.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.729+0000 7f9e1072fd00 1 -- start start 2023-11-15T03:03:38.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9e1072fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558484156920 0x55848422f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9e1072fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55848419bf50 0x55848422f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9e1072fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55848422d130 0x5584841526c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9e1072fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558484100cc0 con 0x55848419bf50 2023-11-15T03:03:38.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9e1072fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584840cb220 con 0x55848422d130 2023-11-15T03:03:38.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9e1072fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584841551c0 con 0x558484156920 2023-11-15T03:03:38.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9dfffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55848419bf50 0x55848422f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9e05623700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55848422d130 0x5584841526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9dfffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55848419bf50 0x55848422f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59624/0 (socket says 172.21.15.121:59624) 2023-11-15T03:03:38.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9dfffff700 1 -- 172.21.15.121:0/1632198729 learned_addr learned my addr 172.21.15.121:0/1632198729 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:38.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9e04e22700 1 --2- 172.21.15.121:0/1632198729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558484156920 0x55848422f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9dfffff700 1 -- 172.21.15.121:0/1632198729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558484156920 msgr2=0x55848422f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:38.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9dfffff700 1 --2- 172.21.15.121:0/1632198729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558484156920 0x55848422f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9dfffff700 1 -- 172.21.15.121:0/1632198729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55848422d130 msgr2=0x5584841526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:38.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9dfffff700 1 --2- 172.21.15.121:0/1632198729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55848422d130 0x5584841526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9dfffff700 1 -- 172.21.15.121:0/1632198729 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0000a0e0 con 0x55848419bf50 2023-11-15T03:03:38.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9dfffff700 1 --2- 172.21.15.121:0/1632198729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55848419bf50 0x55848422f7f0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7f9df4002d60 tx=0x7f9df4002830).ready entity=mon.0 client_cookie=e98a3e12bfbd2a89 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:38.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9dfeffd700 1 -- 172.21.15.121:0/1632198729 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df400ba30 con 0x55848419bf50 2023-11-15T03:03:38.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9dfeffd700 1 -- 172.21.15.121:0/1632198729 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df4008070 con 0x55848419bf50 2023-11-15T03:03:38.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9dfeffd700 1 -- 172.21.15.121:0/1632198729 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df400bbb0 con 0x55848419bf50 2023-11-15T03:03:38.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9e1072fd00 1 -- 172.21.15.121:0/1632198729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55848419bf50 msgr2=0x55848422f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9e1072fd00 1 --2- 172.21.15.121:0/1632198729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55848419bf50 0x55848422f7f0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7f9df4002d60 tx=0x7f9df4002830).stop 2023-11-15T03:03:38.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9e1072fd00 1 -- 172.21.15.121:0/1632198729 shutdown_connections 2023-11-15T03:03:38.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9e1072fd00 1 --2- 172.21.15.121:0/1632198729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55848422d130 0x5584841526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9e1072fd00 1 --2- 172.21.15.121:0/1632198729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55848419bf50 0x55848422f7f0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9e1072fd00 1 --2- 172.21.15.121:0/1632198729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558484156920 0x55848422f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9e1072fd00 1 -- 172.21.15.121:0/1632198729 >> 172.21.15.121:0/1632198729 conn(0x55848419c8f0 msgr2=0x55848422af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:38.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9e1072fd00 1 -- 172.21.15.121:0/1632198729 shutdown_connections 2023-11-15T03:03:38.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.733+0000 7f9e1072fd00 1 -- 172.21.15.121:0/1632198729 wait complete. 2023-11-15T03:03:38.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9e1072fd00 1 Processor -- start 2023-11-15T03:03:38.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9e1072fd00 1 -- start start 2023-11-15T03:03:38.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9e1072fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558484156920 0x558484152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9e1072fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55848419bf50 0x55848423a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9e1072fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55848422d130 0x55848423ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9e1072fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558484155810 con 0x55848422d130 2023-11-15T03:03:38.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9e1072fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558484196f80 con 0x558484156920 2023-11-15T03:03:38.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9e1072fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558484229570 con 0x55848419bf50 2023-11-15T03:03:38.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9e05623700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55848422d130 0x55848423ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9e05623700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55848422d130 0x55848423ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59630/0 (socket says 172.21.15.121:59630) 2023-11-15T03:03:38.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9e05623700 1 -- 172.21.15.121:0/200033094 learned_addr learned my addr 172.21.15.121:0/200033094 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:38.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9dfffff700 1 --2- 172.21.15.121:0/200033094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55848419bf50 0x55848423a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9e04e22700 1 --2- 172.21.15.121:0/200033094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558484156920 0x558484152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9e05623700 1 -- 172.21.15.121:0/200033094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55848419bf50 msgr2=0x55848423a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9e05623700 1 --2- 172.21.15.121:0/200033094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55848419bf50 0x55848423a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9e05623700 1 -- 172.21.15.121:0/200033094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558484156920 msgr2=0x558484152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9e05623700 1 --2- 172.21.15.121:0/200033094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558484156920 0x558484152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9e05623700 1 -- 172.21.15.121:0/200033094 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df400a040 con 0x55848422d130 2023-11-15T03:03:38.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9e04e22700 1 --2- 172.21.15.121:0/200033094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558484156920 0x558484152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:38.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.737+0000 7f9e05623700 1 --2- 172.21.15.121:0/200033094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55848422d130 0x55848423ae30 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x558484141c20 tx=0x7f9e00015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:38.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.741+0000 7f9dfd7fa700 1 -- 172.21.15.121:0/200033094 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0000d740 con 0x55848422d130 2023-11-15T03:03:38.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.741+0000 7f9dfd7fa700 1 -- 172.21.15.121:0/200033094 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0000d8a0 con 0x55848422d130 2023-11-15T03:03:38.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.741+0000 7f9e1072fd00 1 -- 172.21.15.121:0/200033094 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5584840c3a70 con 0x55848422d130 2023-11-15T03:03:38.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.741+0000 7f9dfd7fa700 1 -- 172.21.15.121:0/200033094 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00024420 con 0x55848422d130 2023-11-15T03:03:38.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.741+0000 7f9e1072fd00 1 -- 172.21.15.121:0/200033094 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584840cf010 con 0x55848422d130 2023-11-15T03:03:38.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.741+0000 7f9dfd7fa700 1 -- 172.21.15.121:0/200033094 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9e00021020 con 0x55848422d130 2023-11-15T03:03:38.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.745+0000 7f9dfd7fa700 1 --2- 172.21.15.121:0/200033094 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9df0059020 0x7f9df005b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.745+0000 7f9dfd7fa700 1 -- 172.21.15.121:0/200033094 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f9e00074f90 con 0x55848422d130 2023-11-15T03:03:38.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.745+0000 7f9e04e22700 1 --2- 172.21.15.121:0/200033094 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9df0059020 0x7f9df005b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.745+0000 7f9e1072fd00 1 --2- 172.21.15.121:0/200033094 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5584841572c0 0x558484227f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.745+0000 7f9e1072fd00 1 -- 172.21.15.121:0/200033094 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:1 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e93) v8 -- 0x558484152db0 con 0x5584841572c0 2023-11-15T03:03:38.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.745+0000 7f9e04e22700 1 --2- 172.21.15.121:0/200033094 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9df0059020 0x7f9df005b4e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f9df80011e0 tx=0x7f9df8001710).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:38.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.745+0000 7f9dfffff700 1 --2- 172.21.15.121:0/200033094 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5584841572c0 0x558484227f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.745+0000 7f9dfffff700 1 --2- 172.21.15.121:0/200033094 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5584841572c0 0x558484227f90 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:38.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.749+0000 7f9dfffff700 1 -- 172.21.15.121:0/200033094 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v93'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9df4016040 con 0x5584841572c0 2023-11-15T03:03:38.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.749+0000 7f9e1072fd00 1 -- 172.21.15.121:0/200033094 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5584841572c0 msgr2=0x558484227f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.749+0000 7f9e1072fd00 1 --2- 172.21.15.121:0/200033094 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5584841572c0 0x558484227f90 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.753+0000 7f9e1072fd00 1 -- 172.21.15.121:0/200033094 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9df0059020 msgr2=0x7f9df005b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.753+0000 7f9e1072fd00 1 --2- 172.21.15.121:0/200033094 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9df0059020 0x7f9df005b4e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f9df80011e0 tx=0x7f9df8001710).stop 2023-11-15T03:03:38.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.753+0000 7f9e1072fd00 1 -- 172.21.15.121:0/200033094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55848422d130 msgr2=0x55848423ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.753+0000 7f9e1072fd00 1 --2- 172.21.15.121:0/200033094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55848422d130 0x55848423ae30 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x558484141c20 tx=0x7f9e00015000).stop 2023-11-15T03:03:38.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.753+0000 7f9e05623700 1 -- 172.21.15.121:0/200033094 reap_dead start 2023-11-15T03:03:38.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.753+0000 7f9e1072fd00 1 -- 172.21.15.121:0/200033094 shutdown_connections 2023-11-15T03:03:38.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.753+0000 7f9e1072fd00 1 -- 172.21.15.121:0/200033094 >> 172.21.15.121:0/200033094 conn(0x55848419c8f0 msgr2=0x5584842223c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:38.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.753+0000 7f9e1072fd00 1 -- 172.21.15.121:0/200033094 shutdown_connections 2023-11-15T03:03:38.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.753+0000 7f9e1072fd00 1 -- 172.21.15.121:0/200033094 wait complete. 2023-11-15T03:03:38.763 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:03:38.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.937+0000 7ffa128e4700 1 Processor -- start 2023-11-15T03:03:38.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.937+0000 7ffa128e4700 1 -- start start 2023-11-15T03:03:38.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.941+0000 7ffa128e4700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa0c151af0 0x7ffa0c0a7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.941+0000 7ffa128e4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffa0c0c3650 0x7ffa0c065d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.941+0000 7ffa128e4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa0c1528b0 con 0x7ffa0c0a5720 2023-11-15T03:03:38.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.941+0000 7ffa128e4700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0c151ff0 con 0x7ffa0c151af0 2023-11-15T03:03:38.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.941+0000 7ffa128e4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0c152260 con 0x7ffa0c0c3650 2023-11-15T03:03:38.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.941+0000 7ffa120e3700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffa0c0c3650 0x7ffa0c065d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.941+0000 7ffa118e2700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa0c151af0 0x7ffa0c0a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.941+0000 7ffa110e1700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ffa0c0a5720 0x7ffa0c0c2f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:53864/0 (socket says 172.21.15.121:53864) 2023-11-15T03:03:38.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.941+0000 7ffa110e1700 1 -- 172.21.15.121:0/2804188138 learned_addr learned my addr 172.21.15.121:0/2804188138 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:38.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.941+0000 7ffa120e3700 1 -- 172.21.15.121:0/2804188138 >> v2:172.21.15.121:3300/0 conn(0x7ffa0c151af0 msgr2=0x7ffa0c0a7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:38.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.941+0000 7ffa120e3700 1 --2- 172.21.15.121:0/2804188138 >> v2:172.21.15.121:3300/0 conn(0x7ffa0c151af0 0x7ffa0c0a7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.941+0000 7ffa120e3700 1 -- 172.21.15.121:0/2804188138 >> v1:172.21.15.190:6789/0 conn(0x7ffa0c0a5720 legacy=0x7ffa0c0c2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.941+0000 7ffa120e3700 1 -- 172.21.15.121:0/2804188138 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0c152540 con 0x7ffa0c0c3650 2023-11-15T03:03:38.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.941+0000 7ffa120e3700 1 --2- 172.21.15.121:0/2804188138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0c0c3650 0x7ffa0c065d70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7ffa04009e60 tx=0x7ffa04009780).ready entity=mon.1 client_cookie=36eacdd507343a33 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:38.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.941+0000 7ffa108e0700 1 -- 172.21.15.121:0/2804188138 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400e070 con 0x7ffa0c0c3650 2023-11-15T03:03:38.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.941+0000 7ffa108e0700 1 -- 172.21.15.121:0/2804188138 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0400bb20 con 0x7ffa0c0c3650 2023-11-15T03:03:38.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.941+0000 7ffa108e0700 1 -- 172.21.15.121:0/2804188138 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400bea0 con 0x7ffa0c0c3650 2023-11-15T03:03:38.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.945+0000 7ffa128e4700 1 -- 172.21.15.121:0/2804188138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0c0c3650 msgr2=0x7ffa0c065d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.945+0000 7ffa128e4700 1 --2- 172.21.15.121:0/2804188138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0c0c3650 0x7ffa0c065d70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7ffa04009e60 tx=0x7ffa04009780).stop 2023-11-15T03:03:38.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.945+0000 7ffa128e4700 1 -- 172.21.15.121:0/2804188138 shutdown_connections 2023-11-15T03:03:38.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.945+0000 7ffa128e4700 1 --2- 172.21.15.121:0/2804188138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0c0c3650 0x7ffa0c065d70 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.945+0000 7ffa128e4700 1 --2- 172.21.15.121:0/2804188138 >> v2:172.21.15.121:3300/0 conn(0x7ffa0c151af0 0x7ffa0c0a7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.945+0000 7ffa128e4700 1 -- 172.21.15.121:0/2804188138 >> 172.21.15.121:0/2804188138 conn(0x7ffa0c067a60 msgr2=0x7ffa0c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:38.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.945+0000 7ffa128e4700 1 -- 172.21.15.121:0/2804188138 shutdown_connections 2023-11-15T03:03:38.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.945+0000 7ffa128e4700 1 -- 172.21.15.121:0/2804188138 wait complete. 2023-11-15T03:03:38.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.945+0000 7ffa128e4700 1 Processor -- start 2023-11-15T03:03:38.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.945+0000 7ffa128e4700 1 -- start start 2023-11-15T03:03:38.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa128e4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0c0a5720 0x7ffa0c0aa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa128e4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa0c0c3650 0x7ffa0c0aa590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa128e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0c163a40 0x7ffa0c163ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa128e4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c0ab0f0 con 0x7ffa0c0c3650 2023-11-15T03:03:38.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa128e4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c0aae10 con 0x7ffa0c0a5720 2023-11-15T03:03:38.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa128e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c164f30 con 0x7ffa0c163a40 2023-11-15T03:03:38.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa110e1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa0c0c3650 0x7ffa0c0aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa110e1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa0c0c3650 0x7ffa0c0aa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59662/0 (socket says 172.21.15.121:59662) 2023-11-15T03:03:38.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa110e1700 1 -- 172.21.15.121:0/471894605 learned_addr learned my addr 172.21.15.121:0/471894605 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:38.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa118e2700 1 --2- 172.21.15.121:0/471894605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0c0a5720 0x7ffa0c0aa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa120e3700 1 --2- 172.21.15.121:0/471894605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0c163a40 0x7ffa0c163ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa110e1700 1 -- 172.21.15.121:0/471894605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0c163a40 msgr2=0x7ffa0c163ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa110e1700 1 --2- 172.21.15.121:0/471894605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0c163a40 0x7ffa0c163ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa110e1700 1 -- 172.21.15.121:0/471894605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0c0a5720 msgr2=0x7ffa0c0aa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:38.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa110e1700 1 --2- 172.21.15.121:0/471894605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0c0a5720 0x7ffa0c0aa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:38.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa110e1700 1 -- 172.21.15.121:0/471894605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa04009a20 con 0x7ffa0c0c3650 2023-11-15T03:03:38.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa110e1700 1 --2- 172.21.15.121:0/471894605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa0c0c3650 0x7ffa0c0aa590 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7ff9fc001eb0 tx=0x7ff9fc007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:38.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa01ffb700 1 -- 172.21.15.121:0/471894605 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0070d0 con 0x7ffa0c0c3650 2023-11-15T03:03:38.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa01ffb700 1 -- 172.21.15.121:0/471894605 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9fc007230 con 0x7ffa0c0c3650 2023-11-15T03:03:38.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa01ffb700 1 -- 172.21.15.121:0/471894605 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0075d0 con 0x7ffa0c0c3650 2023-11-15T03:03:38.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa128e4700 1 -- 172.21.15.121:0/471894605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0c165210 con 0x7ffa0c0c3650 2023-11-15T03:03:38.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.949+0000 7ffa128e4700 1 -- 172.21.15.121:0/471894605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0c0aaa80 con 0x7ffa0c0c3650 2023-11-15T03:03:38.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.953+0000 7ffa128e4700 1 -- 172.21.15.121:0/471894605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0c06bcf0 con 0x7ffa0c0c3650 2023-11-15T03:03:38.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.953+0000 7ffa01ffb700 1 -- 172.21.15.121:0/471894605 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff9fc0059b0 con 0x7ffa0c0c3650 2023-11-15T03:03:38.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.953+0000 7ffa01ffb700 1 --2- 172.21.15.121:0/471894605 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff9f005a510 0x7ff9f005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:38.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.953+0000 7ffa01ffb700 1 -- 172.21.15.121:0/471894605 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ff9fc073e30 con 0x7ffa0c0c3650 2023-11-15T03:03:38.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.957+0000 7ffa118e2700 1 --2- 172.21.15.121:0/471894605 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff9f005a510 0x7ff9f005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:38.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.957+0000 7ffa118e2700 1 --2- 172.21.15.121:0/471894605 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff9f005a510 0x7ff9f005c9d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7ffa0c0a56f0 tx=0x7ffa08009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:38.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:38.961+0000 7ffa01ffb700 1 -- 172.21.15.121:0/471894605 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff9fc047500 con 0x7ffa0c0c3650 2023-11-15T03:03:39.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:39.165+0000 7ffa128e4700 1 -- 172.21.15.121:0/471894605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7ffa0c068a70 con 0x7ffa0c0c3650 2023-11-15T03:03:40.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.013+0000 7ffa01ffb700 1 -- 172.21.15.121:0/471894605 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v86) v1 ==== 162+0+0 (secure 0 0 0) 0x7ff9fc04ab20 con 0x7ffa0c0c3650 2023-11-15T03:03:40.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.013+0000 7ffa128e4700 1 -- 172.21.15.121:0/471894605 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff9f005a510 msgr2=0x7ff9f005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:40.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.013+0000 7ffa128e4700 1 --2- 172.21.15.121:0/471894605 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff9f005a510 0x7ff9f005c9d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7ffa0c0a56f0 tx=0x7ffa08009060).stop 2023-11-15T03:03:40.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.017+0000 7ffa128e4700 1 -- 172.21.15.121:0/471894605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa0c0c3650 msgr2=0x7ffa0c0aa590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:40.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.017+0000 7ffa128e4700 1 --2- 172.21.15.121:0/471894605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa0c0c3650 0x7ffa0c0aa590 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7ff9fc001eb0 tx=0x7ff9fc007e50).stop 2023-11-15T03:03:40.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.017+0000 7ffa128e4700 1 -- 172.21.15.121:0/471894605 shutdown_connections 2023-11-15T03:03:40.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.017+0000 7ffa128e4700 1 --2- 172.21.15.121:0/471894605 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff9f005a510 0x7ff9f005c9d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:40.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.017+0000 7ffa128e4700 1 --2- 172.21.15.121:0/471894605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0c163a40 0x7ffa0c163ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:40.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.017+0000 7ffa128e4700 1 --2- 172.21.15.121:0/471894605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa0c0c3650 0x7ffa0c0aa590 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:40.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.017+0000 7ffa128e4700 1 --2- 172.21.15.121:0/471894605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0c0a5720 0x7ffa0c0aa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:40.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.017+0000 7ffa128e4700 1 -- 172.21.15.121:0/471894605 >> 172.21.15.121:0/471894605 conn(0x7ffa0c067a60 msgr2=0x7ffa0c0bf980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:40.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.017+0000 7ffa128e4700 1 -- 172.21.15.121:0/471894605 shutdown_connections 2023-11-15T03:03:40.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.017+0000 7ffa128e4700 1 -- 172.21.15.121:0/471894605 wait complete. 2023-11-15T03:03:40.024 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:03:40.446 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:03:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.605+0000 7f63519a9700 1 Processor -- start 2023-11-15T03:03:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.605+0000 7f63519a9700 1 -- start start 2023-11-15T03:03:40.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.609+0000 7f63519a9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f634c153a30 0x7f634c153e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:40.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.609+0000 7f63519a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c0cb9d0 0x7f634c0c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:40.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.609+0000 7f63519a9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f634c0c2920 con 0x7f634c0ab620 2023-11-15T03:03:40.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.609+0000 7f63519a9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f634c067a60 con 0x7f634c153a30 2023-11-15T03:03:40.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.609+0000 7f63519a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f634c0c2c00 con 0x7f634c0cb9d0 2023-11-15T03:03:40.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.609+0000 7f634affd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f634c153a30 0x7f634c153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:40.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.613+0000 7f634affd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f634c153a30 0x7f634c153e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59678/0 (socket says 172.21.15.121:59678) 2023-11-15T03:03:40.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.613+0000 7f634affd700 1 -- 172.21.15.121:0/3408045382 learned_addr learned my addr 172.21.15.121:0/3408045382 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:40.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.613+0000 7f634bfff700 1 --2- 172.21.15.121:0/3408045382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c0cb9d0 0x7f634c0c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:40.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.613+0000 7f634affd700 1 -- 172.21.15.121:0/3408045382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c0cb9d0 msgr2=0x7f634c0c22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:40.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.613+0000 7f634affd700 1 --2- 172.21.15.121:0/3408045382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c0cb9d0 0x7f634c0c22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:40.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.613+0000 7f634affd700 1 -- 172.21.15.121:0/3408045382 >> v1:172.21.15.190:6789/0 conn(0x7f634c0ab620 legacy=0x7f634c0cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:40.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.613+0000 7f634affd700 1 -- 172.21.15.121:0/3408045382 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634c0c2ee0 con 0x7f634c153a30 2023-11-15T03:03:40.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.613+0000 7f634affd700 1 --2- 172.21.15.121:0/3408045382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f634c153a30 0x7f634c153e10 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7f633c006fd0 tx=0x7f633c0090f0).ready entity=mon.0 client_cookie=5f5c62c7d7424e3a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:40.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.613+0000 7f6349ffb700 1 -- 172.21.15.121:0/3408045382 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c00b580 con 0x7f634c153a30 2023-11-15T03:03:40.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.613+0000 7f6349ffb700 1 -- 172.21.15.121:0/3408045382 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633c00b6e0 con 0x7f634c153a30 2023-11-15T03:03:40.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.613+0000 7f6349ffb700 1 -- 172.21.15.121:0/3408045382 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c00ba60 con 0x7f634c153a30 2023-11-15T03:03:40.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.613+0000 7f63519a9700 1 -- 172.21.15.121:0/3408045382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f634c153a30 msgr2=0x7f634c153e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:40.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.613+0000 7f63519a9700 1 --2- 172.21.15.121:0/3408045382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f634c153a30 0x7f634c153e10 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7f633c006fd0 tx=0x7f633c0090f0).stop 2023-11-15T03:03:40.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.613+0000 7f63519a9700 1 -- 172.21.15.121:0/3408045382 shutdown_connections 2023-11-15T03:03:40.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.613+0000 7f63519a9700 1 --2- 172.21.15.121:0/3408045382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c0cb9d0 0x7f634c0c22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:40.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.613+0000 7f63519a9700 1 --2- 172.21.15.121:0/3408045382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f634c153a30 0x7f634c153e10 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:40.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.613+0000 7f63519a9700 1 -- 172.21.15.121:0/3408045382 >> 172.21.15.121:0/3408045382 conn(0x7f634c095820 msgr2=0x7f634c069820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:40.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.613+0000 7f63519a9700 1 -- 172.21.15.121:0/3408045382 shutdown_connections 2023-11-15T03:03:40.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.613+0000 7f63519a9700 1 -- 172.21.15.121:0/3408045382 wait complete. 2023-11-15T03:03:40.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.617+0000 7f63519a9700 1 Processor -- start 2023-11-15T03:03:40.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.617+0000 7f63519a9700 1 -- start start 2023-11-15T03:03:40.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f63519a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f634c0ab620 0x7f634c151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:40.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f63519a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f634c0cb9d0 0x7f634c0b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:40.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f63519a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c153a30 0x7f634c0b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:40.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f63519a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c085420 con 0x7f634c0cb9d0 2023-11-15T03:03:40.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f63519a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c082df0 con 0x7f634c0ab620 2023-11-15T03:03:40.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f63519a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f634c0830d0 con 0x7f634c153a30 2023-11-15T03:03:40.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f634a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f634c0cb9d0 0x7f634c0b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:40.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f634a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f634c0cb9d0 0x7f634c0b02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59684/0 (socket says 172.21.15.121:59684) 2023-11-15T03:03:40.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f634a7fc700 1 -- 172.21.15.121:0/1834254323 learned_addr learned my addr 172.21.15.121:0/1834254323 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:40.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f634affd700 1 --2- 172.21.15.121:0/1834254323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f634c0ab620 0x7f634c151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:40.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f634bfff700 1 --2- 172.21.15.121:0/1834254323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c153a30 0x7f634c0b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:40.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f634a7fc700 1 -- 172.21.15.121:0/1834254323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c153a30 msgr2=0x7f634c0b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:40.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f634a7fc700 1 --2- 172.21.15.121:0/1834254323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c153a30 0x7f634c0b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:40.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f634a7fc700 1 -- 172.21.15.121:0/1834254323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f634c0ab620 msgr2=0x7f634c151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:40.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f634a7fc700 1 --2- 172.21.15.121:0/1834254323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f634c0ab620 0x7f634c151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:40.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f634a7fc700 1 -- 172.21.15.121:0/1834254323 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c009370 con 0x7f634c0cb9d0 2023-11-15T03:03:40.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f634a7fc700 1 --2- 172.21.15.121:0/1834254323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f634c0cb9d0 0x7f634c0b02d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 rx=0x7f634000c4c0 tx=0x7f6340007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:40.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f6332ffd700 1 -- 172.21.15.121:0/1834254323 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634000ce70 con 0x7f634c0cb9d0 2023-11-15T03:03:40.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f6332ffd700 1 -- 172.21.15.121:0/1834254323 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6340001e90 con 0x7f634c0cb9d0 2023-11-15T03:03:40.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f63519a9700 1 -- 172.21.15.121:0/1834254323 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f634c0833b0 con 0x7f634c0cb9d0 2023-11-15T03:03:40.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f6332ffd700 1 -- 172.21.15.121:0/1834254323 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6340007250 con 0x7f634c0cb9d0 2023-11-15T03:03:40.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.621+0000 7f63519a9700 1 -- 172.21.15.121:0/1834254323 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f634c169970 con 0x7f634c0cb9d0 2023-11-15T03:03:40.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.625+0000 7f63519a9700 1 -- 172.21.15.121:0/1834254323 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f634c0838b0 con 0x7f634c0cb9d0 2023-11-15T03:03:40.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.625+0000 7f6332ffd700 1 -- 172.21.15.121:0/1834254323 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f63400066a0 con 0x7f634c0cb9d0 2023-11-15T03:03:40.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.629+0000 7f6332ffd700 1 --2- 172.21.15.121:0/1834254323 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f633805a510 0x7f633805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:40.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.629+0000 7f6332ffd700 1 -- 172.21.15.121:0/1834254323 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6352+0+0 (secure 0 0 0) 0x7f63400740e0 con 0x7f634c0cb9d0 2023-11-15T03:03:40.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.629+0000 7f634affd700 1 --2- 172.21.15.121:0/1834254323 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f633805a510 0x7f633805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:40.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.629+0000 7f634affd700 1 --2- 172.21.15.121:0/1834254323 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f633805a510 0x7f633805c9d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f633c009fd0 tx=0x7f633c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:40.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.633+0000 7f6332ffd700 1 -- 172.21.15.121:0/1834254323 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f63400477e0 con 0x7f634c0cb9d0 2023-11-15T03:03:40.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.829+0000 7f63519a9700 1 -- 172.21.15.121:0/1834254323 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f634c047060 con 0x7f634c0cb9d0 2023-11-15T03:03:40.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.829+0000 7f6332ffd700 1 -- 172.21.15.121:0/1834254323 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4498 (secure 0 0 0) 0x7f634004ae00 con 0x7f634c0cb9d0 2023-11-15T03:03:40.835 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:40.836 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":87,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:03:40.014589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7140},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7140":{"gid":7140,"name":"a","rank":0,"incarnation":87,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/763933658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":763933658},{"type":"v1","addr":"172.21.15.121:6836","nonce":763933658}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:03:40.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.833+0000 7f63519a9700 1 -- 172.21.15.121:0/1834254323 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f633805a510 msgr2=0x7f633805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:40.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.833+0000 7f63519a9700 1 --2- 172.21.15.121:0/1834254323 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f633805a510 0x7f633805c9d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f633c009fd0 tx=0x7f633c017000).stop 2023-11-15T03:03:40.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.833+0000 7f63519a9700 1 -- 172.21.15.121:0/1834254323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f634c0cb9d0 msgr2=0x7f634c0b02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:40.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.833+0000 7f63519a9700 1 --2- 172.21.15.121:0/1834254323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f634c0cb9d0 0x7f634c0b02d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 rx=0x7f634000c4c0 tx=0x7f6340007e00).stop 2023-11-15T03:03:40.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.833+0000 7f63519a9700 1 -- 172.21.15.121:0/1834254323 shutdown_connections 2023-11-15T03:03:40.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.833+0000 7f63519a9700 1 --2- 172.21.15.121:0/1834254323 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f633805a510 0x7f633805c9d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:40.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.833+0000 7f63519a9700 1 --2- 172.21.15.121:0/1834254323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c153a30 0x7f634c0b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:40.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.833+0000 7f63519a9700 1 --2- 172.21.15.121:0/1834254323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f634c0cb9d0 0x7f634c0b02d0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:40.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.833+0000 7f63519a9700 1 --2- 172.21.15.121:0/1834254323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f634c0ab620 0x7f634c151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:40.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.833+0000 7f63519a9700 1 -- 172.21.15.121:0/1834254323 >> 172.21.15.121:0/1834254323 conn(0x7f634c095820 msgr2=0x7f634c0aa550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:40.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.833+0000 7f63519a9700 1 -- 172.21.15.121:0/1834254323 shutdown_connections 2023-11-15T03:03:40.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:40.833+0000 7f63519a9700 1 -- 172.21.15.121:0/1834254323 wait complete. 2023-11-15T03:03:40.840 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 87 2023-11-15T03:03:40.855 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:03:40.014589+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7140}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7140': {'gid': 7140, 'name': 'a', 'rank': 0, 'incarnation': 87, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.121:6836/763933658', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 763933658}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 763933658}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:03:40.856 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7140:up:replay 2023-11-15T03:03:40.856 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:03:41.230 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:03:41.239 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:03:46.260 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:03:46.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.420+0000 7f9d0d2db700 1 Processor -- start 2023-11-15T03:03:46.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.420+0000 7f9d0d2db700 1 -- start start 2023-11-15T03:03:46.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d0d2db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9d08151920 0x7f9d08151d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d0d2db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d08063fe0 0x7f9d08064440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d0d2db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9d080c4720 0x7f9d080c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d0d2db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d08064930 con 0x7f9d080c4720 2023-11-15T03:03:46.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d0d2db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d08068430 con 0x7f9d08151920 2023-11-15T03:03:46.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d0d2db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d080c50a0 con 0x7f9d08063fe0 2023-11-15T03:03:46.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d07fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9d080c4720 0x7f9d080c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d07fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9d080c4720 0x7f9d080c4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44992/0 (socket says 172.21.15.121:44992) 2023-11-15T03:03:46.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d07fff700 1 -- 172.21.15.121:0/2121509602 learned_addr learned my addr 172.21.15.121:0/2121509602 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:46.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d067fc700 1 --2- 172.21.15.121:0/2121509602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d08063fe0 0x7f9d08064440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d06ffd700 1 --2- 172.21.15.121:0/2121509602 >> v2:172.21.15.190:3300/0 conn(0x7f9d08151920 0x7f9d08151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d07fff700 1 -- 172.21.15.121:0/2121509602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d08063fe0 msgr2=0x7f9d08064440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:46.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d07fff700 1 --2- 172.21.15.121:0/2121509602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d08063fe0 0x7f9d08064440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d07fff700 1 -- 172.21.15.121:0/2121509602 >> v2:172.21.15.190:3300/0 conn(0x7f9d08151920 msgr2=0x7f9d08151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:46.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d07fff700 1 --2- 172.21.15.121:0/2121509602 >> v2:172.21.15.190:3300/0 conn(0x7f9d08151920 0x7f9d08151d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d07fff700 1 -- 172.21.15.121:0/2121509602 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d08064b50 con 0x7f9d080c4720 2023-11-15T03:03:46.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d07fff700 1 --2- 172.21.15.121:0/2121509602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d080c4720 0x7f9d080c4bb0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f9d080626c0 tx=0x7f9d00009840).ready entity=mon.0 client_cookie=c2893425f15b9b5f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:46.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d05ffb700 1 -- 172.21.15.121:0/2121509602 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0000e070 con 0x7f9d080c4720 2023-11-15T03:03:46.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d05ffb700 1 -- 172.21.15.121:0/2121509602 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0000ba50 con 0x7f9d080c4720 2023-11-15T03:03:46.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d05ffb700 1 -- 172.21.15.121:0/2121509602 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0000bdd0 con 0x7f9d080c4720 2023-11-15T03:03:46.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d0d2db700 1 -- 172.21.15.121:0/2121509602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d080c4720 msgr2=0x7f9d080c4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:46.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d0d2db700 1 --2- 172.21.15.121:0/2121509602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d080c4720 0x7f9d080c4bb0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f9d080626c0 tx=0x7f9d00009840).stop 2023-11-15T03:03:46.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d0d2db700 1 -- 172.21.15.121:0/2121509602 shutdown_connections 2023-11-15T03:03:46.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d0d2db700 1 --2- 172.21.15.121:0/2121509602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d080c4720 0x7f9d080c4bb0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d0d2db700 1 --2- 172.21.15.121:0/2121509602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d08063fe0 0x7f9d08064440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.424+0000 7f9d0d2db700 1 --2- 172.21.15.121:0/2121509602 >> v2:172.21.15.190:3300/0 conn(0x7f9d08151920 0x7f9d08151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.428+0000 7f9d0d2db700 1 -- 172.21.15.121:0/2121509602 >> 172.21.15.121:0/2121509602 conn(0x7f9d08069ac0 msgr2=0x7f9d08090a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:46.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.428+0000 7f9d0d2db700 1 -- 172.21.15.121:0/2121509602 shutdown_connections 2023-11-15T03:03:46.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.428+0000 7f9d0d2db700 1 -- 172.21.15.121:0/2121509602 wait complete. 2023-11-15T03:03:46.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.428+0000 7f9d0d2db700 1 Processor -- start 2023-11-15T03:03:46.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9d0d2db700 1 -- start start 2023-11-15T03:03:46.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9d0d2db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d08063fe0 0x7f9d081566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9d0d2db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d080c4720 0x7f9d080c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9d0d2db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d08151920 0x7f9d080c0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9d0d2db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d08154d60 con 0x7f9d080c4720 2023-11-15T03:03:46.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9d0d2db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d08154a80 con 0x7f9d08063fe0 2023-11-15T03:03:46.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9d0d2db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d08155c10 con 0x7f9d08151920 2023-11-15T03:03:46.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9d067fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d080c4720 0x7f9d080c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9d067fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d080c4720 0x7f9d080c70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45004/0 (socket says 172.21.15.121:45004) 2023-11-15T03:03:46.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9d067fc700 1 -- 172.21.15.121:0/1724458344 learned_addr learned my addr 172.21.15.121:0/1724458344 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:46.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9d06ffd700 1 --2- 172.21.15.121:0/1724458344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d08063fe0 0x7f9d081566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9d07fff700 1 --2- 172.21.15.121:0/1724458344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d08151920 0x7f9d080c0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9d067fc700 1 -- 172.21.15.121:0/1724458344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d08151920 msgr2=0x7f9d080c0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:46.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9d067fc700 1 --2- 172.21.15.121:0/1724458344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d08151920 0x7f9d080c0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9d067fc700 1 -- 172.21.15.121:0/1724458344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d08063fe0 msgr2=0x7f9d081566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:46.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9d067fc700 1 --2- 172.21.15.121:0/1724458344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d08063fe0 0x7f9d081566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9d067fc700 1 -- 172.21.15.121:0/1724458344 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d00009ae0 con 0x7f9d080c4720 2023-11-15T03:03:46.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9d067fc700 1 --2- 172.21.15.121:0/1724458344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d080c4720 0x7f9d080c70f0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7f9cfc00d890 tx=0x7f9cfc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:46.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9ceeffd700 1 -- 172.21.15.121:0/1724458344 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc011590 con 0x7f9d080c4720 2023-11-15T03:03:46.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9ceeffd700 1 -- 172.21.15.121:0/1724458344 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc0116f0 con 0x7f9d080c4720 2023-11-15T03:03:46.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9ceeffd700 1 -- 172.21.15.121:0/1724458344 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc011a70 con 0x7f9d080c4720 2023-11-15T03:03:46.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.432+0000 7f9d0d2db700 1 -- 172.21.15.121:0/1724458344 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d08155ef0 con 0x7f9d080c4720 2023-11-15T03:03:46.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.436+0000 7f9d0d2db700 1 -- 172.21.15.121:0/1724458344 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d080c75e0 con 0x7f9d080c4720 2023-11-15T03:03:46.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.436+0000 7f9d0d2db700 1 -- 172.21.15.121:0/1724458344 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d081563f0 con 0x7f9d080c4720 2023-11-15T03:03:46.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.436+0000 7f9ceeffd700 1 -- 172.21.15.121:0/1724458344 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9cfc01c460 con 0x7f9d080c4720 2023-11-15T03:03:46.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.436+0000 7f9ceeffd700 1 --2- 172.21.15.121:0/1724458344 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9cf405a560 0x7f9cf405ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.436+0000 7f9ceeffd700 1 -- 172.21.15.121:0/1724458344 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6352+0+0 (secure 0 0 0) 0x7f9cfc0746c0 con 0x7f9d080c4720 2023-11-15T03:03:46.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.436+0000 7f9d06ffd700 1 --2- 172.21.15.121:0/1724458344 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9cf405a560 0x7f9cf405ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.440+0000 7f9d06ffd700 1 --2- 172.21.15.121:0/1724458344 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9cf405a560 0x7f9cf405ca20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f9cf800bb10 tx=0x7f9cf8008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:46.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.444+0000 7f9ceeffd700 1 -- 172.21.15.121:0/1724458344 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9cfc047d40 con 0x7f9d080c4720 2023-11-15T03:03:46.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.644+0000 7f9d0d2db700 1 -- 172.21.15.121:0/1724458344 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d08047060 con 0x7f9d080c4720 2023-11-15T03:03:46.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.644+0000 7f9ceeffd700 1 -- 172.21.15.121:0/1724458344 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4498 (secure 0 0 0) 0x7f9cfc04b360 con 0x7f9d080c4720 2023-11-15T03:03:46.654 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:46.654 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":90,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:03:43.035794+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7140},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7140":{"gid":7140,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6836/763933658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":763933658},{"type":"v1","addr":"172.21.15.121:6836","nonce":763933658}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:03:46.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.648+0000 7f9d0d2db700 1 -- 172.21.15.121:0/1724458344 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9cf405a560 msgr2=0x7f9cf405ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:46.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.648+0000 7f9d0d2db700 1 --2- 172.21.15.121:0/1724458344 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9cf405a560 0x7f9cf405ca20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f9cf800bb10 tx=0x7f9cf8008040).stop 2023-11-15T03:03:46.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.648+0000 7f9d0d2db700 1 -- 172.21.15.121:0/1724458344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d080c4720 msgr2=0x7f9d080c70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:46.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.648+0000 7f9d0d2db700 1 --2- 172.21.15.121:0/1724458344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d080c4720 0x7f9d080c70f0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7f9cfc00d890 tx=0x7f9cfc008040).stop 2023-11-15T03:03:46.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.648+0000 7f9d0d2db700 1 -- 172.21.15.121:0/1724458344 shutdown_connections 2023-11-15T03:03:46.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.648+0000 7f9d0d2db700 1 --2- 172.21.15.121:0/1724458344 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9cf405a560 0x7f9cf405ca20 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.648+0000 7f9d0d2db700 1 --2- 172.21.15.121:0/1724458344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d08151920 0x7f9d080c0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.648+0000 7f9d0d2db700 1 --2- 172.21.15.121:0/1724458344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d080c4720 0x7f9d080c70f0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.648+0000 7f9d0d2db700 1 --2- 172.21.15.121:0/1724458344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d08063fe0 0x7f9d081566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.648+0000 7f9d0d2db700 1 -- 172.21.15.121:0/1724458344 >> 172.21.15.121:0/1724458344 conn(0x7f9d08069ac0 msgr2=0x7f9d080ad7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:46.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.648+0000 7f9d0d2db700 1 -- 172.21.15.121:0/1724458344 shutdown_connections 2023-11-15T03:03:46.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.648+0000 7f9d0d2db700 1 -- 172.21.15.121:0/1724458344 wait complete. 2023-11-15T03:03:46.661 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 90 2023-11-15T03:03:46.676 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:03:43.035794+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7140}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7140': {'gid': 7140, 'name': 'a', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.121:6836/763933658', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 763933658}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 763933658}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:03:46.677 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T03:03:46.677 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-15T03:03:46.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.836+0000 7fd7fbbdb700 1 Processor -- start 2023-11-15T03:03:46.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.836+0000 7fd7fbbdb700 1 -- start start 2023-11-15T03:03:46.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.836+0000 7fd7fbbdb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f4157e30 0x7fd7f4158210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.836+0000 7fd7fbbdb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f40aaf80 con 0x7fd7f4062190 2023-11-15T03:03:46.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.836+0000 7fd7fbbdb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f40aa970 con 0x7fd7f40ac420 2023-11-15T03:03:46.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.836+0000 7fd7fbbdb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f4066e10 con 0x7fd7f4157e30 2023-11-15T03:03:46.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.840+0000 7fd7f9977700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f4157e30 0x7fd7f4158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.840+0000 7fd7f9176700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fd7f4062190 0x7fd7f40660e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58180/0 (socket says 172.21.15.121:58180) 2023-11-15T03:03:46.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.840+0000 7fd7f9176700 1 -- 172.21.15.121:0/1257976767 learned_addr learned my addr 172.21.15.121:0/1257976767 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:46.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.840+0000 7fd7f9977700 1 -- 172.21.15.121:0/1257976767 >> v1:172.21.15.190:6789/0 conn(0x7fd7f40ac420 legacy=0x7fd7f4161c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:46.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.840+0000 7fd7f9977700 1 -- 172.21.15.121:0/1257976767 >> v1:172.21.15.121:6789/0 conn(0x7fd7f4062190 legacy=0x7fd7f40660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:46.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.840+0000 7fd7f9977700 1 -- 172.21.15.121:0/1257976767 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f40670f0 con 0x7fd7f4157e30 2023-11-15T03:03:46.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.840+0000 7fd7f9977700 1 --2- 172.21.15.121:0/1257976767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f4157e30 0x7fd7f4158210 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fd7f0009790 tx=0x7fd7f0009060).ready entity=mon.2 client_cookie=20de593147ea802d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:46.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.840+0000 7fd7f8975700 1 -- 172.21.15.121:0/1257976767 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f000b5a0 con 0x7fd7f4157e30 2023-11-15T03:03:46.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.840+0000 7fd7f8975700 1 -- 172.21.15.121:0/1257976767 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f000b700 con 0x7fd7f4157e30 2023-11-15T03:03:46.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.840+0000 7fd7f8975700 1 -- 172.21.15.121:0/1257976767 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f000ba80 con 0x7fd7f4157e30 2023-11-15T03:03:46.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.840+0000 7fd7fbbdb700 1 -- 172.21.15.121:0/1257976767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f4157e30 msgr2=0x7fd7f4158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:46.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.840+0000 7fd7fbbdb700 1 --2- 172.21.15.121:0/1257976767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f4157e30 0x7fd7f4158210 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fd7f0009790 tx=0x7fd7f0009060).stop 2023-11-15T03:03:46.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.844+0000 7fd7fbbdb700 1 -- 172.21.15.121:0/1257976767 shutdown_connections 2023-11-15T03:03:46.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.844+0000 7fd7fbbdb700 1 --2- 172.21.15.121:0/1257976767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f4157e30 0x7fd7f4158210 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.844+0000 7fd7fbbdb700 1 -- 172.21.15.121:0/1257976767 >> 172.21.15.121:0/1257976767 conn(0x7fd7f4067a60 msgr2=0x7fd7f4067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:46.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.844+0000 7fd7fbbdb700 1 -- 172.21.15.121:0/1257976767 shutdown_connections 2023-11-15T03:03:46.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.844+0000 7fd7fbbdb700 1 -- 172.21.15.121:0/1257976767 wait complete. 2023-11-15T03:03:46.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.844+0000 7fd7fbbdb700 1 Processor -- start 2023-11-15T03:03:46.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.844+0000 7fd7fbbdb700 1 -- start start 2023-11-15T03:03:46.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7fbbdb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f4062190 0x7fd7f40bd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7fbbdb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7f40ac420 0x7fd7f40bda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7fbbdb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7f4157e30 0x7fd7f4168fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7fbbdb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f40bf080 con 0x7fd7f40ac420 2023-11-15T03:03:46.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7fbbdb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f40bdf00 con 0x7fd7f4157e30 2023-11-15T03:03:46.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7fbbdb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f416c460 con 0x7fd7f4062190 2023-11-15T03:03:46.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7f9176700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7f40ac420 0x7fd7f40bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7f9176700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7f40ac420 0x7fd7f40bda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45010/0 (socket says 172.21.15.121:45010) 2023-11-15T03:03:46.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7f9176700 1 -- 172.21.15.121:0/3885376271 learned_addr learned my addr 172.21.15.121:0/3885376271 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:46.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7fa178700 1 --2- 172.21.15.121:0/3885376271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7f4157e30 0x7fd7f4168fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7f9977700 1 --2- 172.21.15.121:0/3885376271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f4062190 0x7fd7f40bd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7f9176700 1 -- 172.21.15.121:0/3885376271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f4062190 msgr2=0x7fd7f40bd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:46.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7f9176700 1 --2- 172.21.15.121:0/3885376271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f4062190 0x7fd7f40bd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7f9176700 1 -- 172.21.15.121:0/3885376271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7f4157e30 msgr2=0x7fd7f4168fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:46.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7f9176700 1 --2- 172.21.15.121:0/3885376271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7f4157e30 0x7fd7f4168fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7f9176700 1 -- 172.21.15.121:0/3885376271 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f00092a0 con 0x7fd7f40ac420 2023-11-15T03:03:46.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7fa178700 1 --2- 172.21.15.121:0/3885376271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7f4157e30 0x7fd7f4168fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:46.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7f9176700 1 --2- 172.21.15.121:0/3885376271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7f40ac420 0x7fd7f40bda10 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7fd7e400c4c0 tx=0x7fd7e4007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:46.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7e9ffb700 1 -- 172.21.15.121:0/3885376271 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e40072e0 con 0x7fd7f40ac420 2023-11-15T03:03:46.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7e9ffb700 1 -- 172.21.15.121:0/3885376271 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e4007440 con 0x7fd7f40ac420 2023-11-15T03:03:46.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7fbbdb700 1 -- 172.21.15.121:0/3885376271 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f416c740 con 0x7fd7f40ac420 2023-11-15T03:03:46.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7e9ffb700 1 -- 172.21.15.121:0/3885376271 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e40057a0 con 0x7fd7f40ac420 2023-11-15T03:03:46.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.848+0000 7fd7fbbdb700 1 -- 172.21.15.121:0/3885376271 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f416cd00 con 0x7fd7f40ac420 2023-11-15T03:03:46.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.852+0000 7fd7e9ffb700 1 -- 172.21.15.121:0/3885376271 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd7e4005bd0 con 0x7fd7f40ac420 2023-11-15T03:03:46.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.852+0000 7fd7e9ffb700 1 --2- 172.21.15.121:0/3885376271 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd7d805a510 0x7fd7d805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.852+0000 7fd7e9ffb700 1 -- 172.21.15.121:0/3885376271 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6352+0+0 (secure 0 0 0) 0x7fd7e4073f90 con 0x7fd7f40ac420 2023-11-15T03:03:46.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd7f9977700 1 --2- 172.21.15.121:0/3885376271 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd7d805a510 0x7fd7d805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd805f73740 1 Processor -- start 2023-11-15T03:03:46.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd805f73740 1 -- start start 2023-11-15T03:03:46.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd7f9977700 1 --2- 172.21.15.121:0/3885376271 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd7d805a510 0x7fd7d805c9d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7fd7f40c4d10 tx=0x7fd7f0015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:46.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd805f73740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 0x14d45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd805f73740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148c510 0x148d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd805f73740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 0x14d2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd805f73740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x140d410 con 0x148d300 2023-11-15T03:03:46.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd805f73740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x140d550 con 0x14d2ab0 2023-11-15T03:03:46.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd805f73740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x140da00 con 0x148c510 2023-11-15T03:03:46.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd7f9977700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 0x14d45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd7f9977700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 0x14d45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45024/0 (socket says 172.21.15.121:45024) 2023-11-15T03:03:46.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd7f9977700 1 -- 172.21.15.121:0/133825041 learned_addr learned my addr 172.21.15.121:0/133825041 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:46.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd7f9176700 1 --2- 172.21.15.121:0/133825041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148c510 0x148d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd7f9977700 1 -- 172.21.15.121:0/133825041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148c510 msgr2=0x148d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:46.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd7f9977700 1 --2- 172.21.15.121:0/133825041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148c510 0x148d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd7fa178700 1 --2- 172.21.15.121:0/133825041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 0x14d2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd7f9977700 1 -- 172.21.15.121:0/133825041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 msgr2=0x14d2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:46.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd7f9977700 1 --2- 172.21.15.121:0/133825041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 0x14d2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd7f9977700 1 -- 172.21.15.121:0/133825041 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1406730 con 0x148d300 2023-11-15T03:03:46.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd7fa178700 1 --2- 172.21.15.121:0/133825041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 0x14d2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:46.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd7f9977700 1 --2- 172.21.15.121:0/133825041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 0x14d45b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7fd7f00096e0 tx=0x7fd7f001e070).ready entity=mon.0 client_cookie=5c08a92a89130ac9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:46.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd7deffd700 1 -- 172.21.15.121:0/133825041 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f001c8d0 con 0x148d300 2023-11-15T03:03:46.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd7deffd700 1 -- 172.21.15.121:0/133825041 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f001ca30 con 0x148d300 2023-11-15T03:03:46.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd7deffd700 1 -- 172.21.15.121:0/133825041 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f001cdb0 con 0x148d300 2023-11-15T03:03:46.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd805f73740 1 -- 172.21.15.121:0/133825041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 msgr2=0x14d45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:46.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.856+0000 7fd805f73740 1 --2- 172.21.15.121:0/133825041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 0x14d45b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7fd7f00096e0 tx=0x7fd7f001e070).stop 2023-11-15T03:03:46.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.860+0000 7fd805f73740 1 -- 172.21.15.121:0/133825041 shutdown_connections 2023-11-15T03:03:46.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.860+0000 7fd805f73740 1 --2- 172.21.15.121:0/133825041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 0x14d2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.860+0000 7fd805f73740 1 --2- 172.21.15.121:0/133825041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148c510 0x148d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.860+0000 7fd805f73740 1 --2- 172.21.15.121:0/133825041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 0x14d45b0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.860+0000 7fd805f73740 1 -- 172.21.15.121:0/133825041 >> 172.21.15.121:0/133825041 conn(0x126b550 msgr2=0x14d3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:46.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.860+0000 7fd805f73740 1 -- 172.21.15.121:0/133825041 shutdown_connections 2023-11-15T03:03:46.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.860+0000 7fd805f73740 1 -- 172.21.15.121:0/133825041 wait complete. 2023-11-15T03:03:46.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.860+0000 7fd805f73740 1 -- start start 2023-11-15T03:03:46.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.860+0000 7fd805f73740 1 Processor -- start 2023-11-15T03:03:46.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd805f73740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148c510 0x14cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd805f73740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 0x14b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd805f73740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 0x14b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd805f73740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x13d8650 con 0x148d300 2023-11-15T03:03:46.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd805f73740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x14cc6c0 con 0x14d2ab0 2023-11-15T03:03:46.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd805f73740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x14cc9a0 con 0x148c510 2023-11-15T03:03:46.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd805f73740 10 client.0 ll_register_callbacks cb 0x1445020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:03:46.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd7f9176700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 0x14b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd7f9176700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 0x14b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45026/0 (socket says 172.21.15.121:45026) 2023-11-15T03:03:46.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd7f9176700 1 -- 172.21.15.121:0/3707209817 learned_addr learned my addr 172.21.15.121:0/3707209817 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:46.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd7f9977700 1 --2- 172.21.15.121:0/3707209817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148c510 0x14cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd7fa178700 1 --2- 172.21.15.121:0/3707209817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 0x14b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd7f9176700 1 -- 172.21.15.121:0/3707209817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148c510 msgr2=0x14cb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:46.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd7f9176700 1 --2- 172.21.15.121:0/3707209817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148c510 0x14cb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd7f9176700 1 -- 172.21.15.121:0/3707209817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 msgr2=0x14b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:46.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd7f9176700 1 --2- 172.21.15.121:0/3707209817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 0x14b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd7f9176700 1 -- 172.21.15.121:0/3707209817 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f40670f0 con 0x148d300 2023-11-15T03:03:46.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd7fa178700 1 --2- 172.21.15.121:0/3707209817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 0x14b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:46.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd7f9176700 1 --2- 172.21.15.121:0/3707209817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 0x14b3090 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7fd7e4072420 tx=0x7fd7e404d040).ready entity=mon.0 client_cookie=857d35dca8c3bbb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:46.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd7d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:03:46.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd7d3fff700 1 -- 172.21.15.121:0/3707209817 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e401ab70 con 0x148d300 2023-11-15T03:03:46.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd7d3fff700 1 -- 172.21.15.121:0/3707209817 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6352+0+0 (secure 0 0 0) 0x7fd7e4050070 con 0x148d300 2023-11-15T03:03:46.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd805f73740 1 -- 172.21.15.121:0/3707209817 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x14031a0 con 0x148d300 2023-11-15T03:03:46.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd7d3fff700 1 -- 172.21.15.121:0/3707209817 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7e4047140 con 0x148d300 2023-11-15T03:03:46.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd805f73740 10 client.7632 fetch_fsmap learned FSMap version 90 2023-11-15T03:03:46.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd805f73740 1 -- 172.21.15.121:0/3707209817 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x1401960 con 0x148d300 2023-11-15T03:03:46.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd7d3fff700 1 -- 172.21.15.121:0/3707209817 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x14031a0 con 0x148d300 2023-11-15T03:03:46.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd7d3fff700 1 -- 172.21.15.121:0/3707209817 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 90) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd7e40472a0 con 0x148d300 2023-11-15T03:03:46.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd805f73740 10 client.7632 fetch_fsmap finished waiting for FSMap version 90 2023-11-15T03:03:46.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd805f73740 10 client.7632 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:03:46.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd805f73740 20 client.7632 populate_metadata read hostname 'smithi121' 2023-11-15T03:03:46.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd805f73740 1 --2- 172.21.15.121:0/3707209817 >> [v2:172.21.15.121:6834/763933658,v1:172.21.15.121:6836/763933658] conn(0x14d2f10 0x14cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd805f73740 4 client.7632 mds_command: new command op to 7140 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:03:46.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd805f73740 1 -- 172.21.15.121:0/3707209817 --> [v2:172.21.15.121:6834/763933658,v1:172.21.15.121:6836/763933658] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x13f96f0 con 0x14d2f10 2023-11-15T03:03:46.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.864+0000 7fd7f9977700 1 --2- 172.21.15.121:0/3707209817 >> [v2:172.21.15.121:6834/763933658,v1:172.21.15.121:6836/763933658] conn(0x14d2f10 0x14cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.868+0000 7fd7d3fff700 10 client.7632 ms_handle_connect on v2:172.21.15.121:6834/763933658 2023-11-15T03:03:46.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.868+0000 7fd7f9977700 1 --2- 172.21.15.121:0/3707209817 >> [v2:172.21.15.121:6834/763933658,v1:172.21.15.121:6836/763933658] conn(0x14d2f10 0x14cd5d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=b95b5c3e549f70e3 server_cookie=989fa302ce920ff7 in_seq=0 out_seq=0 2023-11-15T03:03:46.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.868+0000 7fd7d3fff700 1 -- 172.21.15.121:0/3707209817 <== mds.0 v2:172.21.15.121:6834/763933658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x13f96f0 con 0x14d2f10 2023-11-15T03:03:46.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.868+0000 7fd7d3fff700 10 client.7632 handle_command_reply: tid=0 2023-11-15T03:03:46.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.868+0000 7fd805f73740 1 client.7632 shutdown 2023-11-15T03:03:46.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.868+0000 7fd805f73740 1 -- 172.21.15.121:0/3707209817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 msgr2=0x14b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:46.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.868+0000 7fd805f73740 1 --2- 172.21.15.121:0/3707209817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 0x14b3090 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7fd7e4072420 tx=0x7fd7e404d040).stop 2023-11-15T03:03:46.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.868+0000 7fd805f73740 1 -- 172.21.15.121:0/3707209817 shutdown_connections 2023-11-15T03:03:46.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.868+0000 7fd805f73740 1 --2- 172.21.15.121:0/3707209817 >> [v2:172.21.15.121:6834/763933658,v1:172.21.15.121:6836/763933658] conn(0x14d2f10 0x14cd5d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.868+0000 7fd805f73740 1 --2- 172.21.15.121:0/3707209817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 0x14b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.868+0000 7fd805f73740 1 --2- 172.21.15.121:0/3707209817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 0x14b3090 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.868+0000 7fd805f73740 1 --2- 172.21.15.121:0/3707209817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148c510 0x14cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.868+0000 7fd7d3fff700 0 client.7632 ms_handle_reset on v2:172.21.15.121:6834/763933658 2023-11-15T03:03:46.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.868+0000 7fd805f73740 1 -- 172.21.15.121:0/3707209817 >> 172.21.15.121:0/3707209817 conn(0x126b550 msgr2=0x148c990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:46.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.868+0000 7fd805f73740 1 -- 172.21.15.121:0/3707209817 shutdown_connections 2023-11-15T03:03:46.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.868+0000 7fd805f73740 1 -- 172.21.15.121:0/3707209817 wait complete. 2023-11-15T03:03:46.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.868+0000 7fd805f73740 20 client.7632 trim_cache size 0 max 16384 2023-11-15T03:03:46.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.888+0000 7fd805f73740 1 Processor -- start 2023-11-15T03:03:46.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.888+0000 7fd805f73740 1 -- start start 2023-11-15T03:03:46.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.888+0000 7fd805f73740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148c510 0x14e2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.888+0000 7fd805f73740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 0x14b6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.888+0000 7fd805f73740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 0x14d3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.888+0000 7fd805f73740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x14cfdd0 con 0x148d300 2023-11-15T03:03:46.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.888+0000 7fd805f73740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1450e60 con 0x14d2ab0 2023-11-15T03:03:46.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.888+0000 7fd805f73740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x14031a0 con 0x148c510 2023-11-15T03:03:46.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.888+0000 7fd7f9176700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 0x14b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.888+0000 7fd7f9176700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 0x14b6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45034/0 (socket says 172.21.15.121:45034) 2023-11-15T03:03:46.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.888+0000 7fd7f9176700 1 -- 172.21.15.121:0/962556957 learned_addr learned my addr 172.21.15.121:0/962556957 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:46.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.888+0000 7fd7f9977700 1 --2- 172.21.15.121:0/962556957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148c510 0x14e2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.888+0000 7fd7fa178700 1 --2- 172.21.15.121:0/962556957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 0x14d3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd7f9977700 1 -- 172.21.15.121:0/962556957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 msgr2=0x14d3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:46.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd7f9977700 1 --2- 172.21.15.121:0/962556957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 0x14d3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd7f9977700 1 -- 172.21.15.121:0/962556957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 msgr2=0x14b6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:46.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd7f9977700 1 --2- 172.21.15.121:0/962556957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 0x14b6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd7f9977700 1 -- 172.21.15.121:0/962556957 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f4169800 con 0x148c510 2023-11-15T03:03:46.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd7f9176700 1 --2- 172.21.15.121:0/962556957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 0x14b6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:03:46.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd7f9977700 1 --2- 172.21.15.121:0/962556957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148c510 0x14e2180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fd7f0029000 tx=0x7fd7f0012000).ready entity=mon.2 client_cookie=b4159e371e38a6cf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:46.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd7df7fe700 1 -- 172.21.15.121:0/962556957 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0035040 con 0x148c510 2023-11-15T03:03:46.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd7df7fe700 1 -- 172.21.15.121:0/962556957 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f000bcf0 con 0x148c510 2023-11-15T03:03:46.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd7df7fe700 1 -- 172.21.15.121:0/962556957 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f001c700 con 0x148c510 2023-11-15T03:03:46.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd805f73740 1 -- 172.21.15.121:0/962556957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148c510 msgr2=0x14e2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:46.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd805f73740 1 --2- 172.21.15.121:0/962556957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148c510 0x14e2180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fd7f0029000 tx=0x7fd7f0012000).stop 2023-11-15T03:03:46.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd805f73740 1 -- 172.21.15.121:0/962556957 shutdown_connections 2023-11-15T03:03:46.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd805f73740 1 --2- 172.21.15.121:0/962556957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 0x14d3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd805f73740 1 --2- 172.21.15.121:0/962556957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148d300 0x14b6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd805f73740 1 --2- 172.21.15.121:0/962556957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148c510 0x14e2180 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd805f73740 1 -- 172.21.15.121:0/962556957 >> 172.21.15.121:0/962556957 conn(0x126b550 msgr2=0x14ca090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:46.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd805f73740 1 -- 172.21.15.121:0/962556957 shutdown_connections 2023-11-15T03:03:46.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd805f73740 1 -- 172.21.15.121:0/962556957 wait complete. 2023-11-15T03:03:46.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd805f73740 1 -- start start 2023-11-15T03:03:46.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.892+0000 7fd805f73740 1 Processor -- start 2023-11-15T03:03:46.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd805f73740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148c510 0x15383d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd805f73740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148d300 0x1487de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd805f73740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 0x14882d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd805f73740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x14cd220 con 0x148c510 2023-11-15T03:03:46.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd805f73740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x14887c0 con 0x14d2ab0 2023-11-15T03:03:46.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd805f73740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x14e8570 con 0x148d300 2023-11-15T03:03:46.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd805f73740 10 client.0 ll_register_callbacks cb 0x7fd7e404b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:03:46.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7f9977700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148c510 0x15383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7f9977700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148c510 0x15383d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45048/0 (socket says 172.21.15.121:45048) 2023-11-15T03:03:46.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7f9977700 1 -- 172.21.15.121:0/769314068 learned_addr learned my addr 172.21.15.121:0/769314068 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:46.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7f9176700 1 --2- 172.21.15.121:0/769314068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148d300 0x1487de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7fa178700 1 --2- 172.21.15.121:0/769314068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 0x14882d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7f9977700 1 -- 172.21.15.121:0/769314068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148d300 msgr2=0x1487de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:46.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7f9977700 1 --2- 172.21.15.121:0/769314068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148d300 0x1487de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7f9977700 1 -- 172.21.15.121:0/769314068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 msgr2=0x14882d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:46.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7f9977700 1 --2- 172.21.15.121:0/769314068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 0x14882d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7f9977700 1 -- 172.21.15.121:0/769314068 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1401960 con 0x148c510 2023-11-15T03:03:46.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7fa178700 1 --2- 172.21.15.121:0/769314068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 0x14882d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:46.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7f9176700 1 --2- 172.21.15.121:0/769314068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148d300 0x1487de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:46.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7f9977700 1 --2- 172.21.15.121:0/769314068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148c510 0x15383d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7fd7f000b990 tx=0x7fd7f0027000).ready entity=mon.0 client_cookie=c70ad49013ea7e48 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:46.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:03:46.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7d3fff700 1 -- 172.21.15.121:0/769314068 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0035040 con 0x148c510 2023-11-15T03:03:46.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7d3fff700 1 -- 172.21.15.121:0/769314068 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6352+0+0 (secure 0 0 0) 0x7fd7f003ce00 con 0x148c510 2023-11-15T03:03:46.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd805f73740 1 -- 172.21.15.121:0/769314068 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7f001ca30 con 0x148c510 2023-11-15T03:03:46.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7d3fff700 1 -- 172.21.15.121:0/769314068 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7f0026e60 con 0x148c510 2023-11-15T03:03:46.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd805f73740 10 client.7638 fetch_fsmap learned FSMap version 90 2023-11-15T03:03:46.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd805f73740 1 -- 172.21.15.121:0/769314068 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x1406730 con 0x148c510 2023-11-15T03:03:46.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7d3fff700 1 -- 172.21.15.121:0/769314068 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f001ca30 con 0x148c510 2023-11-15T03:03:46.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7d3fff700 1 -- 172.21.15.121:0/769314068 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 90) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd7f003b4c0 con 0x148c510 2023-11-15T03:03:46.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd805f73740 10 client.7638 fetch_fsmap finished waiting for FSMap version 90 2023-11-15T03:03:46.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd805f73740 10 client.7638 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:03:46.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd805f73740 20 client.7638 populate_metadata read hostname 'smithi121' 2023-11-15T03:03:46.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd805f73740 1 --2- 172.21.15.121:0/769314068 >> [v2:172.21.15.121:6834/763933658,v1:172.21.15.121:6836/763933658] conn(0x14ce1f0 0x14863c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:46.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd805f73740 4 client.7638 mds_command: new command op to 7140 tid=0[{"prefix": "status"}] 2023-11-15T03:03:46.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd805f73740 1 -- 172.21.15.121:0/769314068 --> [v2:172.21.15.121:6834/763933658,v1:172.21.15.121:6836/763933658] -- command(tid 0: {"prefix": "status"}) v1 -- 0x12c24e0 con 0x14ce1f0 2023-11-15T03:03:46.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.896+0000 7fd7f9176700 1 --2- 172.21.15.121:0/769314068 >> [v2:172.21.15.121:6834/763933658,v1:172.21.15.121:6836/763933658] conn(0x14ce1f0 0x14863c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:46.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd7f9176700 1 --2- 172.21.15.121:0/769314068 >> [v2:172.21.15.121:6834/763933658,v1:172.21.15.121:6836/763933658] conn(0x14ce1f0 0x14863c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=8a2d60cd29a93b89 server_cookie=8dd1cb17d99459f4 in_seq=0 out_seq=0 2023-11-15T03:03:46.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd7d3fff700 10 client.7638 ms_handle_connect on v2:172.21.15.121:6834/763933658 2023-11-15T03:03:46.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd7d3fff700 1 -- 172.21.15.121:0/769314068 <== mds.0 v2:172.21.15.121:6834/763933658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x12c24e0 con 0x14ce1f0 2023-11-15T03:03:46.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd7d3fff700 10 client.7638 handle_command_reply: tid=0 2023-11-15T03:03:46.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd805f73740 1 client.7638 shutdown 2023-11-15T03:03:46.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd805f73740 1 -- 172.21.15.121:0/769314068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148c510 msgr2=0x15383d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:46.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd805f73740 1 --2- 172.21.15.121:0/769314068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148c510 0x15383d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7fd7f000b990 tx=0x7fd7f0027000).stop 2023-11-15T03:03:46.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd805f73740 1 -- 172.21.15.121:0/769314068 shutdown_connections 2023-11-15T03:03:46.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd805f73740 1 --2- 172.21.15.121:0/769314068 >> [v2:172.21.15.121:6834/763933658,v1:172.21.15.121:6836/763933658] conn(0x14ce1f0 0x14863c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd805f73740 1 --2- 172.21.15.121:0/769314068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x14d2ab0 0x14882d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd805f73740 1 --2- 172.21.15.121:0/769314068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x148d300 0x1487de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd7d3fff700 0 client.7638 ms_handle_reset on v2:172.21.15.121:6834/763933658 2023-11-15T03:03:46.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd805f73740 1 --2- 172.21.15.121:0/769314068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x148c510 0x15383d0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd805f73740 1 -- 172.21.15.121:0/769314068 >> 172.21.15.121:0/769314068 conn(0x126b550 msgr2=0x14d3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:46.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd805f73740 1 -- 172.21.15.121:0/769314068 shutdown_connections 2023-11-15T03:03:46.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd805f73740 1 -- 172.21.15.121:0/769314068 wait complete. 2023-11-15T03:03:46.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd805f73740 20 client.7638 trim_cache size 0 max 16384 2023-11-15T03:03:46.923 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:03:46.923 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T03:03:46.923 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T03:03:46.923 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 7140, 2023-11-15T03:03:46.924 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T03:03:46.924 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T03:03:46.924 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T03:03:46.924 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 6.8834760419999999, 2023-11-15T03:03:46.925 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 90, 2023-11-15T03:03:46.925 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 95, 2023-11-15T03:03:46.925 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 95, 2023-11-15T03:03:46.925 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 15.700906196 2023-11-15T03:03:46.925 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:03:46.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd7fbbdb700 1 -- 172.21.15.121:0/3885376271 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd7d805a510 msgr2=0x7fd7d805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:46.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd7fbbdb700 1 --2- 172.21.15.121:0/3885376271 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd7d805a510 0x7fd7d805c9d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7fd7f40c4d10 tx=0x7fd7f0015000).stop 2023-11-15T03:03:46.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd7fbbdb700 1 -- 172.21.15.121:0/3885376271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7f40ac420 msgr2=0x7fd7f40bda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:46.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd7fbbdb700 1 --2- 172.21.15.121:0/3885376271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7f40ac420 0x7fd7f40bda10 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7fd7e400c4c0 tx=0x7fd7e4007dc0).stop 2023-11-15T03:03:46.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd7fbbdb700 1 -- 172.21.15.121:0/3885376271 shutdown_connections 2023-11-15T03:03:46.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd7fbbdb700 1 --2- 172.21.15.121:0/3885376271 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd7d805a510 0x7fd7d805c9d0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd7fbbdb700 1 --2- 172.21.15.121:0/3885376271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7f4157e30 0x7fd7f4168fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd7fbbdb700 1 --2- 172.21.15.121:0/3885376271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7f40ac420 0x7fd7f40bda10 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd7fbbdb700 1 --2- 172.21.15.121:0/3885376271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f4062190 0x7fd7f40bd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:46.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd7fbbdb700 1 -- 172.21.15.121:0/3885376271 >> 172.21.15.121:0/3885376271 conn(0x7fd7f4067a60 msgr2=0x7fd7f4158950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:46.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd7fbbdb700 1 -- 172.21.15.121:0/3885376271 shutdown_connections 2023-11-15T03:03:46.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:46.900+0000 7fd7fbbdb700 1 -- 172.21.15.121:0/3885376271 wait complete. 2023-11-15T03:03:46.929 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:03:46.930 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 10000000000.00000000', proceeding to ls 2023-11-15T03:03:46.930 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T03:03:47.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.064+0000 7fe0dee9b700 1 Processor -- start 2023-11-15T03:03:47.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.064+0000 7fe0dee9b700 1 -- start start 2023-11-15T03:03:47.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.064+0000 7fe0dee9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0d8151c70 0x7fe0d80c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:47.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.064+0000 7fe0dee9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe0d80c25a0 0x7fe0d80bb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:47.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.064+0000 7fe0dee9b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d8156500 con 0x7fe0d8151890 2023-11-15T03:03:47.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.064+0000 7fe0dee9b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d80c2a30 con 0x7fe0d80c25a0 2023-11-15T03:03:47.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.064+0000 7fe0dee9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d81558f0 con 0x7fe0d8151c70 2023-11-15T03:03:47.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.064+0000 7fe0dd438700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe0d80c25a0 0x7fe0d80bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:47.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.064+0000 7fe0cffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0d8151c70 0x7fe0d80c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:47.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.064+0000 7fe0dcc37700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe0d8151890 0x7fe0d8063ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:36922/0 (socket says 172.21.15.121:36922) 2023-11-15T03:03:47.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.064+0000 7fe0dcc37700 1 -- 172.21.15.121:0/685681326 learned_addr learned my addr 172.21.15.121:0/685681326 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:47.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.068+0000 7fe0dd438700 1 -- 172.21.15.121:0/685681326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0d8151c70 msgr2=0x7fe0d80c20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:47.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.068+0000 7fe0dd438700 1 --2- 172.21.15.121:0/685681326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0d8151c70 0x7fe0d80c20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:47.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.068+0000 7fe0dd438700 1 -- 172.21.15.121:0/685681326 >> v1:172.21.15.190:6789/0 conn(0x7fe0d8151890 legacy=0x7fe0d8063ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:47.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.068+0000 7fe0dd438700 1 -- 172.21.15.121:0/685681326 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d8155bd0 con 0x7fe0d80c25a0 2023-11-15T03:03:47.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.068+0000 7fe0dd438700 1 --2- 172.21.15.121:0/685681326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0d80c25a0 0x7fe0d80bb910 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7fe0d0009760 tx=0x7fe0d0009060).ready entity=mon.0 client_cookie=55e1da7266131b4a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:47.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.068+0000 7fe0cf7fe700 1 -- 172.21.15.121:0/685681326 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d000f510 con 0x7fe0d80c25a0 2023-11-15T03:03:47.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.068+0000 7fe0cf7fe700 1 -- 172.21.15.121:0/685681326 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d000f670 con 0x7fe0d80c25a0 2023-11-15T03:03:47.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.068+0000 7fe0cf7fe700 1 -- 172.21.15.121:0/685681326 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d000f9f0 con 0x7fe0d80c25a0 2023-11-15T03:03:47.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.068+0000 7fe0dee9b700 1 -- 172.21.15.121:0/685681326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0d80c25a0 msgr2=0x7fe0d80bb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:47.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.068+0000 7fe0dee9b700 1 --2- 172.21.15.121:0/685681326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0d80c25a0 0x7fe0d80bb910 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7fe0d0009760 tx=0x7fe0d0009060).stop 2023-11-15T03:03:47.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.068+0000 7fe0dee9b700 1 -- 172.21.15.121:0/685681326 shutdown_connections 2023-11-15T03:03:47.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.068+0000 7fe0dee9b700 1 --2- 172.21.15.121:0/685681326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0d80c25a0 0x7fe0d80bb910 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:47.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.068+0000 7fe0dee9b700 1 --2- 172.21.15.121:0/685681326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0d8151c70 0x7fe0d80c20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:47.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.068+0000 7fe0dee9b700 1 -- 172.21.15.121:0/685681326 >> 172.21.15.121:0/685681326 conn(0x7fe0d80c66d0 msgr2=0x7fe0d80c6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:47.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.068+0000 7fe0dee9b700 1 -- 172.21.15.121:0/685681326 shutdown_connections 2023-11-15T03:03:47.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.068+0000 7fe0dee9b700 1 -- 172.21.15.121:0/685681326 wait complete. 2023-11-15T03:03:47.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.068+0000 7fe0dee9b700 1 Processor -- start 2023-11-15T03:03:47.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0dee9b700 1 -- start start 2023-11-15T03:03:47.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0dee9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0d80c25a0 0x7fe0d80a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:47.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0dee9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0d8151890 0x7fe0d8162160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:47.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0dee9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0d80a7e90 0x7fe0d8149af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:47.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0dee9b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d8163020 con 0x7fe0d80a7e90 2023-11-15T03:03:47.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0dee9b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d8162a60 con 0x7fe0d8151890 2023-11-15T03:03:47.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0dee9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d8162d40 con 0x7fe0d80c25a0 2023-11-15T03:03:47.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0dd438700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0d80a7e90 0x7fe0d8149af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:47.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0dd438700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0d80a7e90 0x7fe0d8149af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45066/0 (socket says 172.21.15.121:45066) 2023-11-15T03:03:47.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0dd438700 1 -- 172.21.15.121:0/928740348 learned_addr learned my addr 172.21.15.121:0/928740348 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:47.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0dcc37700 1 --2- 172.21.15.121:0/928740348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0d80c25a0 0x7fe0d80a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:47.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0cffff700 1 --2- 172.21.15.121:0/928740348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0d8151890 0x7fe0d8162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:47.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0dd438700 1 -- 172.21.15.121:0/928740348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0d80c25a0 msgr2=0x7fe0d80a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:47.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0dd438700 1 --2- 172.21.15.121:0/928740348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0d80c25a0 0x7fe0d80a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:47.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0dd438700 1 -- 172.21.15.121:0/928740348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0d8151890 msgr2=0x7fe0d8162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:47.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0dd438700 1 --2- 172.21.15.121:0/928740348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0d8151890 0x7fe0d8162160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:47.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0dd438700 1 -- 172.21.15.121:0/928740348 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d00092a0 con 0x7fe0d80a7e90 2023-11-15T03:03:47.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0dcc37700 1 --2- 172.21.15.121:0/928740348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0d80c25a0 0x7fe0d80a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:47.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0dd438700 1 --2- 172.21.15.121:0/928740348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0d80a7e90 0x7fe0d8149af0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7fe0d8046990 tx=0x7fe0d0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:47.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0ccff9700 1 -- 172.21.15.121:0/928740348 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d000f510 con 0x7fe0d80a7e90 2023-11-15T03:03:47.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0ccff9700 1 -- 172.21.15.121:0/928740348 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d000f9f0 con 0x7fe0d80a7e90 2023-11-15T03:03:47.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.072+0000 7fe0dee9b700 1 -- 172.21.15.121:0/928740348 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d80a6100 con 0x7fe0d80a7e90 2023-11-15T03:03:47.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.076+0000 7fe0ccff9700 1 -- 172.21.15.121:0/928740348 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d001b640 con 0x7fe0d80a7e90 2023-11-15T03:03:47.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.076+0000 7fe0dee9b700 1 -- 172.21.15.121:0/928740348 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d80a6df0 con 0x7fe0d80a7e90 2023-11-15T03:03:47.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.076+0000 7fe0dee9b700 1 -- 172.21.15.121:0/928740348 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d80a6580 con 0x7fe0d80a7e90 2023-11-15T03:03:47.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.076+0000 7fe0ccff9700 1 -- 172.21.15.121:0/928740348 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe0d001ba70 con 0x7fe0d80a7e90 2023-11-15T03:03:47.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.080+0000 7fe0ccff9700 1 --2- 172.21.15.121:0/928740348 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe0c405a510 0x7fe0c405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:47.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.080+0000 7fe0ccff9700 1 -- 172.21.15.121:0/928740348 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6352+0+0 (secure 0 0 0) 0x7fe0d00765d0 con 0x7fe0d80a7e90 2023-11-15T03:03:47.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.080+0000 7fe0dcc37700 1 --2- 172.21.15.121:0/928740348 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe0c405a510 0x7fe0c405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:47.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.080+0000 7fe0dcc37700 1 --2- 172.21.15.121:0/928740348 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe0c405a510 0x7fe0c405c9d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fe0d4007fd0 tx=0x7fe0d4009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:47.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.084+0000 7fe0ccff9700 1 -- 172.21.15.121:0/928740348 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe0d0049c30 con 0x7fe0d80a7e90 2023-11-15T03:03:47.157 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:03:47.152+0000 7f792edf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:03:47.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.280+0000 7fe0dee9b700 1 -- 172.21.15.121:0/928740348 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe0d805eeb0 con 0x7fe0d80a7e90 2023-11-15T03:03:47.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.284+0000 7fe0ccff9700 1 -- 172.21.15.121:0/928740348 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe0d004d250 con 0x7fe0d80a7e90 2023-11-15T03:03:47.289 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T03:03:47.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.284+0000 7fe0dee9b700 1 -- 172.21.15.121:0/928740348 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe0c405a510 msgr2=0x7fe0c405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:47.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.284+0000 7fe0dee9b700 1 --2- 172.21.15.121:0/928740348 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe0c405a510 0x7fe0c405c9d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fe0d4007fd0 tx=0x7fe0d4009040).stop 2023-11-15T03:03:47.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.284+0000 7fe0dee9b700 1 -- 172.21.15.121:0/928740348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0d80a7e90 msgr2=0x7fe0d8149af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:47.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.284+0000 7fe0dee9b700 1 --2- 172.21.15.121:0/928740348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0d80a7e90 0x7fe0d8149af0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7fe0d8046990 tx=0x7fe0d0017000).stop 2023-11-15T03:03:47.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.284+0000 7fe0dee9b700 1 -- 172.21.15.121:0/928740348 shutdown_connections 2023-11-15T03:03:47.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.284+0000 7fe0dee9b700 1 --2- 172.21.15.121:0/928740348 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe0c405a510 0x7fe0c405c9d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:47.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.284+0000 7fe0dee9b700 1 --2- 172.21.15.121:0/928740348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0d80a7e90 0x7fe0d8149af0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:47.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.284+0000 7fe0dee9b700 1 --2- 172.21.15.121:0/928740348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0d8151890 0x7fe0d8162160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:47.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.284+0000 7fe0dee9b700 1 --2- 172.21.15.121:0/928740348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0d80c25a0 0x7fe0d80a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:47.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.284+0000 7fe0dee9b700 1 -- 172.21.15.121:0/928740348 >> 172.21.15.121:0/928740348 conn(0x7fe0d80c66d0 msgr2=0x7fe0d814aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:47.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.288+0000 7fe0dee9b700 1 -- 172.21.15.121:0/928740348 shutdown_connections 2023-11-15T03:03:47.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:47.288+0000 7fe0dee9b700 1 -- 172.21.15.121:0/928740348 wait complete. 2023-11-15T03:03:47.308 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T03:03:47.308 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T03:03:47.308 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T03:03:47.309 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T03:03:47.309 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T03:03:47.309 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T03:03:47.310 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T03:03:47.310 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T03:03:47.310 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:03:47.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:47.311 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T03:03:47.334 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T03:03:47.334 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T03:03:47.334 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T03:03:47.335 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:03:47.335 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T03:03:47.335 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:03:47.335 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:03:47.335 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:03:47.336 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T03:03:47.336 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:03:47.336 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:03:47.336 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T03:03:47.336 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:03:47.337 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:03:47.337 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T03:03:47.337 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:03:47.337 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T03:03:47.337 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42012sec preferred_lft 42012sec 2023-11-15T03:03:47.338 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T03:03:47.338 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:03:47.338 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T03:03:47.338 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:03:47.338 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T03:03:47.339 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:03:47.339 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T03:03:47.339 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T03:03:47.340 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T03:03:47.340 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:03:47.340 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T03:03:47.341 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:03:47.341 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:03:47.341 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T03:03:47.342 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T03:03:47.342 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:03:47.342 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:03:47.342 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T03:03:47.342 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T03:03:47.343 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:03:47.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:47.344 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T03:03:47.427 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T03:03:47.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:47.431 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T03:03:47.500 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T03:03:47.500 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T03:03:47.501 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T03:03:47.501 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T03:03:47.501 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T03:03:47.501 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T03:03:47.502 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T03:03:47.502 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T03:03:47.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:47.503 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T03:03:47.504 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T03:03:47.504 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T03:03:47.505 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T03:03:47.505 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T03:03:47.505 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T03:03:47.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:47.642 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T03:03:47.659 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T03:03:47.660 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T03:03:47.660 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:03:47.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:47.661 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T03:03:47.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:47.739 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T03:03:47.799 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T03:03:47.799 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:03:47.914 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:03:47.914 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:03:47.935 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:03:47.935 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:03:47.941 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T03:03:47.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:47.942 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T03:03:47.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:03:47.970 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2023-11-15T03:03:48.024 INFO:teuthology.orchestra.run:waiting for 20 2023-11-15T03:03:48.039 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:03:48.032+0000 7f792b5ea700 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/subdir) 2023-11-15T03:03:48.039 INFO:teuthology.orchestra.run.smithi121.stdout:.: 2023-11-15T03:03:48.040 INFO:teuthology.orchestra.run.smithi121.stdout:subdir 2023-11-15T03:03:48.040 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:48.040 INFO:teuthology.orchestra.run.smithi121.stdout:./subdir: 2023-11-15T03:03:48.041 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 10000000000.00000000' did not caused DAMAGED state 2023-11-15T03:03:48.042 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 600.00000000' 2023-11-15T03:03:48.042 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T03:03:48.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:03:48.043 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-15T03:03:48.133 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:03:48.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:48.134 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:03:48.202 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:03:48.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.408+0000 7f36716db700 1 Processor -- start 2023-11-15T03:03:48.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.408+0000 7f36716db700 1 -- start start 2023-11-15T03:03:48.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.408+0000 7f36716db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f366c154e10 0x7f366c0a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:48.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.408+0000 7f36716db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f366c0a89e0 0x7f366c062190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:48.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.408+0000 7f36716db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f366c0a9250 con 0x7f366c1551f0 2023-11-15T03:03:48.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.408+0000 7f36716db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f366c0663a0 con 0x7f366c0a89e0 2023-11-15T03:03:48.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.408+0000 7f36716db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f366c0a6d80 con 0x7f366c154e10 2023-11-15T03:03:48.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.408+0000 7f36627fc700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f366c1551f0 0x7f366c0a9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58196/0 (socket says 172.21.15.121:58196) 2023-11-15T03:03:48.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.408+0000 7f366b7fe700 1 --2- 172.21.15.121:0/2339725630 >> v2:172.21.15.121:3300/0 conn(0x7f366c0a89e0 0x7f366c062190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:48.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.412+0000 7f36627fc700 1 -- 172.21.15.121:0/2339725630 learned_addr learned my addr 172.21.15.121:0/2339725630 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:48.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.412+0000 7f366affd700 1 --2- 172.21.15.121:0/2339725630 >> v2:172.21.15.190:3300/0 conn(0x7f366c154e10 0x7f366c0a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:48.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.412+0000 7f366b7fe700 1 -- 172.21.15.121:0/2339725630 >> v2:172.21.15.190:3300/0 conn(0x7f366c154e10 msgr2=0x7f366c0a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:48.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.412+0000 7f366b7fe700 1 --2- 172.21.15.121:0/2339725630 >> v2:172.21.15.190:3300/0 conn(0x7f366c154e10 0x7f366c0a5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:48.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.412+0000 7f366b7fe700 1 -- 172.21.15.121:0/2339725630 >> v1:172.21.15.121:6789/0 conn(0x7f366c1551f0 legacy=0x7f366c0a9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:48.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.412+0000 7f366b7fe700 1 -- 172.21.15.121:0/2339725630 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366c0a7060 con 0x7f366c0a89e0 2023-11-15T03:03:48.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.412+0000 7f366b7fe700 1 --2- 172.21.15.121:0/2339725630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f366c0a89e0 0x7f366c062190 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f366c0c6470 tx=0x7f3664009780).ready entity=mon.0 client_cookie=afe4ded176305eac server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:48.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.412+0000 7f366a7fc700 1 -- 172.21.15.121:0/2339725630 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366400e070 con 0x7f366c0a89e0 2023-11-15T03:03:48.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.412+0000 7f366a7fc700 1 -- 172.21.15.121:0/2339725630 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366400bb60 con 0x7f366c0a89e0 2023-11-15T03:03:48.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.412+0000 7f366a7fc700 1 -- 172.21.15.121:0/2339725630 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3664012410 con 0x7f366c0a89e0 2023-11-15T03:03:48.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.412+0000 7f36716db700 1 -- 172.21.15.121:0/2339725630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f366c0a89e0 msgr2=0x7f366c062190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:48.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.412+0000 7f36716db700 1 --2- 172.21.15.121:0/2339725630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f366c0a89e0 0x7f366c062190 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f366c0c6470 tx=0x7f3664009780).stop 2023-11-15T03:03:48.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.412+0000 7f36716db700 1 -- 172.21.15.121:0/2339725630 shutdown_connections 2023-11-15T03:03:48.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.412+0000 7f36716db700 1 --2- 172.21.15.121:0/2339725630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f366c0a89e0 0x7f366c062190 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:48.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.412+0000 7f36716db700 1 --2- 172.21.15.121:0/2339725630 >> v2:172.21.15.190:3300/0 conn(0x7f366c154e10 0x7f366c0a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:48.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.412+0000 7f36716db700 1 -- 172.21.15.121:0/2339725630 >> 172.21.15.121:0/2339725630 conn(0x7f366c067a60 msgr2=0x7f366c067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:48.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.412+0000 7f36716db700 1 -- 172.21.15.121:0/2339725630 shutdown_connections 2023-11-15T03:03:48.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.412+0000 7f36716db700 1 -- 172.21.15.121:0/2339725630 wait complete. 2023-11-15T03:03:48.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.416+0000 7f36716db700 1 Processor -- start 2023-11-15T03:03:48.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.416+0000 7f36716db700 1 -- start start 2023-11-15T03:03:48.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.416+0000 7f36716db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f366c0a89e0 0x7f366c155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:48.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.416+0000 7f36716db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f366c154e10 0x7f366c158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:48.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.416+0000 7f36716db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f366c1551f0 0x7f366c143de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:48.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.416+0000 7f36716db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c156800 con 0x7f366c1551f0 2023-11-15T03:03:48.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.416+0000 7f36716db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c155bf0 con 0x7f366c0a89e0 2023-11-15T03:03:48.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.416+0000 7f36716db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366c155ed0 con 0x7f366c154e10 2023-11-15T03:03:48.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.416+0000 7f366b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f366c1551f0 0x7f366c143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:48.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.416+0000 7f366b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f366c1551f0 0x7f366c143de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45084/0 (socket says 172.21.15.121:45084) 2023-11-15T03:03:48.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.416+0000 7f366b7fe700 1 -- 172.21.15.121:0/4279755507 learned_addr learned my addr 172.21.15.121:0/4279755507 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:48.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.416+0000 7f366affd700 1 --2- 172.21.15.121:0/4279755507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f366c0a89e0 0x7f366c155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:48.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.416+0000 7f36627fc700 1 --2- 172.21.15.121:0/4279755507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f366c154e10 0x7f366c158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:48.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.416+0000 7f366b7fe700 1 -- 172.21.15.121:0/4279755507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f366c154e10 msgr2=0x7f366c158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:48.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.416+0000 7f366b7fe700 1 --2- 172.21.15.121:0/4279755507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f366c154e10 0x7f366c158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:48.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.416+0000 7f366b7fe700 1 -- 172.21.15.121:0/4279755507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f366c0a89e0 msgr2=0x7f366c155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:48.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.416+0000 7f366b7fe700 1 --2- 172.21.15.121:0/4279755507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f366c0a89e0 0x7f366c155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:48.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.416+0000 7f366b7fe700 1 -- 172.21.15.121:0/4279755507 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3664009a20 con 0x7f366c1551f0 2023-11-15T03:03:48.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.416+0000 7f366b7fe700 1 --2- 172.21.15.121:0/4279755507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f366c1551f0 0x7f366c143de0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7f366400b6c0 tx=0x7f3664009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:48.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.420+0000 7f36637fe700 1 -- 172.21.15.121:0/4279755507 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366401d630 con 0x7f366c1551f0 2023-11-15T03:03:48.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.420+0000 7f36637fe700 1 -- 172.21.15.121:0/4279755507 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366401d790 con 0x7f366c1551f0 2023-11-15T03:03:48.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.420+0000 7f36716db700 1 -- 172.21.15.121:0/4279755507 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366c1561b0 con 0x7f366c1551f0 2023-11-15T03:03:48.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.420+0000 7f36637fe700 1 -- 172.21.15.121:0/4279755507 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366401db10 con 0x7f366c1551f0 2023-11-15T03:03:48.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.420+0000 7f36716db700 1 -- 172.21.15.121:0/4279755507 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366c1593f0 con 0x7f366c1551f0 2023-11-15T03:03:48.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.420+0000 7f36716db700 1 -- 172.21.15.121:0/4279755507 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366c156970 con 0x7f366c1551f0 2023-11-15T03:03:48.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.420+0000 7f36637fe700 1 -- 172.21.15.121:0/4279755507 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3664003040 con 0x7f366c1551f0 2023-11-15T03:03:48.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.424+0000 7f36637fe700 1 --2- 172.21.15.121:0/4279755507 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f364c05a540 0x7f364c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:48.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.424+0000 7f36637fe700 1 -- 172.21.15.121:0/4279755507 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6352+0+0 (secure 0 0 0) 0x7f3664076680 con 0x7f366c1551f0 2023-11-15T03:03:48.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.424+0000 7f366affd700 1 --2- 172.21.15.121:0/4279755507 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f364c05a540 0x7f364c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:48.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.424+0000 7f366affd700 1 --2- 172.21.15.121:0/4279755507 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f364c05a540 0x7f364c05ca00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f365c009c30 tx=0x7f365c009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:48.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.428+0000 7f36637fe700 1 -- 172.21.15.121:0/4279755507 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3664049d80 con 0x7f366c1551f0 2023-11-15T03:03:48.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:48.624+0000 7f36716db700 1 -- 172.21.15.121:0/4279755507 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f366c047060 con 0x7f366c1551f0 2023-11-15T03:03:49.138 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:03:49.132+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:03:49.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.140+0000 7f36637fe700 1 -- 172.21.15.121:0/4279755507 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) v1 ==== 161+0+0 (secure 0 0 0) 0x7f366404d3a0 con 0x7f366c1551f0 2023-11-15T03:03:49.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.144+0000 7f36716db700 1 -- 172.21.15.121:0/4279755507 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f364c05a540 msgr2=0x7f364c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:49.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.144+0000 7f36716db700 1 --2- 172.21.15.121:0/4279755507 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f364c05a540 0x7f364c05ca00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f365c009c30 tx=0x7f365c009060).stop 2023-11-15T03:03:49.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.144+0000 7f36716db700 1 -- 172.21.15.121:0/4279755507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f366c1551f0 msgr2=0x7f366c143de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:49.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.144+0000 7f36716db700 1 --2- 172.21.15.121:0/4279755507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f366c1551f0 0x7f366c143de0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7f366400b6c0 tx=0x7f3664009780).stop 2023-11-15T03:03:49.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.144+0000 7f36716db700 1 -- 172.21.15.121:0/4279755507 shutdown_connections 2023-11-15T03:03:49.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.144+0000 7f36716db700 1 --2- 172.21.15.121:0/4279755507 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f364c05a540 0x7f364c05ca00 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:49.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.144+0000 7f36716db700 1 --2- 172.21.15.121:0/4279755507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f366c1551f0 0x7f366c143de0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:49.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.144+0000 7f36716db700 1 --2- 172.21.15.121:0/4279755507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f366c154e10 0x7f366c158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:49.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.144+0000 7f36716db700 1 --2- 172.21.15.121:0/4279755507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f366c0a89e0 0x7f366c155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:49.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.144+0000 7f36716db700 1 -- 172.21.15.121:0/4279755507 >> 172.21.15.121:0/4279755507 conn(0x7f366c067a60 msgr2=0x7f366c0c4520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:49.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.144+0000 7f36716db700 1 -- 172.21.15.121:0/4279755507 shutdown_connections 2023-11-15T03:03:49.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.144+0000 7f36716db700 1 -- 172.21.15.121:0/4279755507 wait complete. 2023-11-15T03:03:49.155 INFO:tasks.ceph.mds.a.smithi121.stderr: -339> 2023-11-15T03:03:47.152+0000 7f792edf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:03:49.156 INFO:tasks.ceph.mds.a.smithi121.stderr: -122> 2023-11-15T03:03:48.032+0000 7f792b5ea700 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/subdir) 2023-11-15T03:03:49.156 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:03:49.174 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:03:49.194 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:03:49.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.332+0000 7f816da5b700 1 Processor -- start 2023-11-15T03:03:49.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.332+0000 7f816da5b700 1 -- start start 2023-11-15T03:03:49.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.332+0000 7f816da5b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f81680aafe0 0x7f81680be8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:49.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.332+0000 7f816da5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81680bedd0 0x7f81680c5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:49.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.332+0000 7f816da5b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81680a8ee0 con 0x7f81680c54f0 2023-11-15T03:03:49.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.332+0000 7f816da5b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8168152010 con 0x7f81680aafe0 2023-11-15T03:03:49.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.332+0000 7f816da5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81680bf230 con 0x7f81680bedd0 2023-11-15T03:03:49.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.332+0000 7f81677fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f81680c54f0 0x7f81680643a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58220/0 (socket says 172.21.15.121:58220) 2023-11-15T03:03:49.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.332+0000 7f81677fe700 1 -- 172.21.15.121:0/2423630943 learned_addr learned my addr 172.21.15.121:0/2423630943 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:49.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.332+0000 7f81667fc700 1 --2- 172.21.15.121:0/2423630943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81680bedd0 0x7f81680c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:49.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.332+0000 7f8166ffd700 1 --2- 172.21.15.121:0/2423630943 >> v2:172.21.15.190:3300/0 conn(0x7f81680aafe0 0x7f81680be8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:49.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8165ffb700 1 -- 172.21.15.121:0/2423630943 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009923752 0 0) 0x7f81680a8ee0 con 0x7f81680c54f0 2023-11-15T03:03:49.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8165ffb700 1 -- 172.21.15.121:0/2423630943 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81500033f0 con 0x7f81680c54f0 2023-11-15T03:03:49.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8165ffb700 1 -- 172.21.15.121:0/2423630943 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1844909065 0 0) 0x7f81500033f0 con 0x7f81680c54f0 2023-11-15T03:03:49.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8165ffb700 1 -- 172.21.15.121:0/2423630943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81680bedd0 msgr2=0x7f81680c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:49.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8165ffb700 1 --2- 172.21.15.121:0/2423630943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81680bedd0 0x7f81680c5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:49.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8165ffb700 1 -- 172.21.15.121:0/2423630943 >> v2:172.21.15.190:3300/0 conn(0x7f81680aafe0 msgr2=0x7f81680be8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:49.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8165ffb700 1 --2- 172.21.15.121:0/2423630943 >> v2:172.21.15.190:3300/0 conn(0x7f81680aafe0 0x7f81680be8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:49.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8165ffb700 1 -- 172.21.15.121:0/2423630943 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81680bf730 con 0x7f81680c54f0 2023-11-15T03:03:49.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8165ffb700 1 -- 172.21.15.121:0/2423630943 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f81600017f0 con 0x7f81680c54f0 2023-11-15T03:03:49.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8165ffb700 1 -- 172.21.15.121:0/2423630943 >> v1:172.21.15.121:6789/0 conn(0x7f81680c54f0 legacy=0x7f81680643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:49.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8165ffb700 1 --2- 172.21.15.121:0/2423630943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8150003910 0x7f8150003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:49.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8165ffb700 1 --2- 172.21.15.121:0/2423630943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8150004690 0x7f8150004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:49.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8165ffb700 1 --2- 172.21.15.121:0/2423630943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81680bedd0 0x7f8150005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:49.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8165ffb700 1 -- 172.21.15.121:0/2423630943 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8168152010 con 0x7f8150003910 2023-11-15T03:03:49.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8165ffb700 1 -- 172.21.15.121:0/2423630943 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81680bf230 con 0x7f8150004690 2023-11-15T03:03:49.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8165ffb700 1 -- 172.21.15.121:0/2423630943 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815000b8e0 con 0x7f81680bedd0 2023-11-15T03:03:49.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8166ffd700 1 --2- 172.21.15.121:0/2423630943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8150003910 0x7f8150003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:49.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f81677fe700 1 --2- 172.21.15.121:0/2423630943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81680bedd0 0x7f8150005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:49.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f81667fc700 1 --2- 172.21.15.121:0/2423630943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8150004690 0x7f8150004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:49.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8166ffd700 1 -- 172.21.15.121:0/2423630943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81680bedd0 msgr2=0x7f8150005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:49.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8166ffd700 1 --2- 172.21.15.121:0/2423630943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81680bedd0 0x7f8150005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:49.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8166ffd700 1 -- 172.21.15.121:0/2423630943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8150004690 msgr2=0x7f8150004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:49.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8166ffd700 1 --2- 172.21.15.121:0/2423630943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8150004690 0x7f8150004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:49.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8166ffd700 1 -- 172.21.15.121:0/2423630943 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81500033f0 con 0x7f8150003910 2023-11-15T03:03:49.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8166ffd700 1 --2- 172.21.15.121:0/2423630943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8150003910 0x7f8150003d30 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7f8158009a80 tx=0x7f8158009060).ready entity=mon.0 client_cookie=f3075f5c05f17004 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:49.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8165ffb700 1 -- 172.21.15.121:0/2423630943 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8158009430 con 0x7f8150003910 2023-11-15T03:03:49.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8165ffb700 1 -- 172.21.15.121:0/2423630943 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815800e640 con 0x7f8150003910 2023-11-15T03:03:49.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.336+0000 7f8165ffb700 1 -- 172.21.15.121:0/2423630943 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815800e9c0 con 0x7f8150003910 2023-11-15T03:03:49.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.340+0000 7f816da5b700 1 -- 172.21.15.121:0/2423630943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8150003910 msgr2=0x7f8150003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:49.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.340+0000 7f816da5b700 1 --2- 172.21.15.121:0/2423630943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8150003910 0x7f8150003d30 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7f8158009a80 tx=0x7f8158009060).stop 2023-11-15T03:03:49.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.340+0000 7f81677fe700 1 -- 172.21.15.121:0/2423630943 reap_dead start 2023-11-15T03:03:49.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.340+0000 7f816da5b700 1 -- 172.21.15.121:0/2423630943 shutdown_connections 2023-11-15T03:03:49.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.340+0000 7f816da5b700 1 -- 172.21.15.121:0/2423630943 >> 172.21.15.121:0/2423630943 conn(0x7f8168067a60 msgr2=0x7f8168067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:49.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.340+0000 7f816da5b700 1 -- 172.21.15.121:0/2423630943 shutdown_connections 2023-11-15T03:03:49.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.340+0000 7f816da5b700 1 -- 172.21.15.121:0/2423630943 wait complete. 2023-11-15T03:03:49.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f816da5b700 1 Processor -- start 2023-11-15T03:03:49.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f816da5b700 1 -- start start 2023-11-15T03:03:49.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f816da5b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8150003910 0x7f81680a7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:49.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f816da5b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81680c04b0 0x7f81680ac270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:49.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f816da5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81680adcf0 0x7f81680ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:49.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f816da5b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81680c1550 con 0x7f81680c04b0 2023-11-15T03:03:49.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f816da5b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81680c2570 con 0x7f8150003910 2023-11-15T03:03:49.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f816da5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81680a8150 con 0x7f81680adcf0 2023-11-15T03:03:49.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f81667fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81680c04b0 0x7f81680ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:49.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f81667fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81680c04b0 0x7f81680ac270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45116/0 (socket says 172.21.15.121:45116) 2023-11-15T03:03:49.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f81667fc700 1 -- 172.21.15.121:0/2792912538 learned_addr learned my addr 172.21.15.121:0/2792912538 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:49.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f8166ffd700 1 --2- 172.21.15.121:0/2792912538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8150003910 0x7f81680a7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:49.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f81677fe700 1 --2- 172.21.15.121:0/2792912538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81680adcf0 0x7f81680ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:49.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f81667fc700 1 -- 172.21.15.121:0/2792912538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81680adcf0 msgr2=0x7f81680ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:49.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f81667fc700 1 --2- 172.21.15.121:0/2792912538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81680adcf0 0x7f81680ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:49.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f81667fc700 1 -- 172.21.15.121:0/2792912538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8150003910 msgr2=0x7f81680a7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:49.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f81667fc700 1 --2- 172.21.15.121:0/2792912538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8150003910 0x7f81680a7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:49.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f81667fc700 1 -- 172.21.15.121:0/2792912538 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81680ac760 con 0x7f81680c04b0 2023-11-15T03:03:49.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f81677fe700 1 --2- 172.21.15.121:0/2792912538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81680adcf0 0x7f81680ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:49.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f8166ffd700 1 --2- 172.21.15.121:0/2792912538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8150003910 0x7f81680a7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:49.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f81667fc700 1 --2- 172.21.15.121:0/2792912538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81680c04b0 0x7f81680ac270 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7f815c0119f0 tx=0x7f815c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:49.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.344+0000 7f814effd700 1 -- 172.21.15.121:0/2792912538 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c00f940 con 0x7f81680c04b0 2023-11-15T03:03:49.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.348+0000 7f814effd700 1 -- 172.21.15.121:0/2792912538 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815c00faa0 con 0x7f81680c04b0 2023-11-15T03:03:49.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.348+0000 7f816da5b700 1 -- 172.21.15.121:0/2792912538 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81680c0940 con 0x7f81680c04b0 2023-11-15T03:03:49.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.348+0000 7f814effd700 1 -- 172.21.15.121:0/2792912538 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c00fe20 con 0x7f81680c04b0 2023-11-15T03:03:49.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.348+0000 7f816da5b700 1 -- 172.21.15.121:0/2792912538 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81680c0f00 con 0x7f81680c04b0 2023-11-15T03:03:49.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.348+0000 7f816da5b700 1 -- 172.21.15.121:0/2792912538 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81680c1700 con 0x7f81680c04b0 2023-11-15T03:03:49.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.348+0000 7f814effd700 1 -- 172.21.15.121:0/2792912538 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f815c014650 con 0x7f81680c04b0 2023-11-15T03:03:49.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.352+0000 7f814effd700 1 --2- 172.21.15.121:0/2792912538 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f815405a510 0x7f815405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:49.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.352+0000 7f814effd700 1 -- 172.21.15.121:0/2792912538 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f815c075610 con 0x7f81680c04b0 2023-11-15T03:03:49.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.352+0000 7f8166ffd700 1 --2- 172.21.15.121:0/2792912538 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f815405a510 0x7f815405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:49.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.352+0000 7f8166ffd700 1 --2- 172.21.15.121:0/2792912538 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f815405a510 0x7f815405c9d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f81680bfa90 tx=0x7f815801a000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:49.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.360+0000 7f814effd700 1 -- 172.21.15.121:0/2792912538 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f81680c1700 con 0x7f81680c04b0 2023-11-15T03:03:49.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:49.584+0000 7f816da5b700 1 -- 172.21.15.121:0/2792912538 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f81680c1690 con 0x7f81680c04b0 2023-11-15T03:03:50.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.152+0000 7f814effd700 1 -- 172.21.15.121:0/2792912538 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v92) v1 ==== 107+0+0 (secure 0 0 0) 0x7f815c048c30 con 0x7f81680c04b0 2023-11-15T03:03:50.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.156+0000 7f816da5b700 1 -- 172.21.15.121:0/2792912538 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f815405a510 msgr2=0x7f815405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.156+0000 7f816da5b700 1 --2- 172.21.15.121:0/2792912538 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f815405a510 0x7f815405c9d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f81680bfa90 tx=0x7f815801a000).stop 2023-11-15T03:03:50.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.156+0000 7f816da5b700 1 -- 172.21.15.121:0/2792912538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81680c04b0 msgr2=0x7f81680ac270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.156+0000 7f816da5b700 1 --2- 172.21.15.121:0/2792912538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81680c04b0 0x7f81680ac270 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7f815c0119f0 tx=0x7f815c009060).stop 2023-11-15T03:03:50.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.156+0000 7f816da5b700 1 -- 172.21.15.121:0/2792912538 shutdown_connections 2023-11-15T03:03:50.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.156+0000 7f816da5b700 1 --2- 172.21.15.121:0/2792912538 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f815405a510 0x7f815405c9d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.156+0000 7f816da5b700 1 --2- 172.21.15.121:0/2792912538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81680adcf0 0x7f81680ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.156+0000 7f816da5b700 1 --2- 172.21.15.121:0/2792912538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81680c04b0 0x7f81680ac270 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.156+0000 7f816da5b700 1 --2- 172.21.15.121:0/2792912538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8150003910 0x7f81680a7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.156+0000 7f816da5b700 1 -- 172.21.15.121:0/2792912538 >> 172.21.15.121:0/2792912538 conn(0x7f8168067a60 msgr2=0x7f81681514a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:50.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.156+0000 7f816da5b700 1 -- 172.21.15.121:0/2792912538 shutdown_connections 2023-11-15T03:03:50.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.156+0000 7f816da5b700 1 -- 172.21.15.121:0/2792912538 wait complete. 2023-11-15T03:03:50.167 INFO:teuthology.orchestra.run.smithi121.stderr:nothing to do: rank is not damaged 2023-11-15T03:03:50.186 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:03:50.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.280+0000 7f7c85d3ad00 1 Processor -- start 2023-11-15T03:03:50.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.280+0000 7f7c85d3ad00 1 -- start start 2023-11-15T03:03:50.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c85d3ad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56262ce81dd0 0x56262cf54430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c85d3ad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56262cf5a670 0x56262ceca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c85d3ad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56262cec6ff0 0x56262cec7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c85d3ad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56262cdfa220 con 0x56262cec6ff0 2023-11-15T03:03:50.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c85d3ad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56262ceca550 con 0x56262ce81dd0 2023-11-15T03:03:50.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c85d3ad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56262cec7960 con 0x56262cf5a670 2023-11-15T03:03:50.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c7aa46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56262cec6ff0 0x56262cec7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c7aa46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56262cec6ff0 0x56262cec7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45130/0 (socket says 172.21.15.121:45130) 2023-11-15T03:03:50.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c7aa46700 1 -- 172.21.15.121:0/1819112732 learned_addr learned my addr 172.21.15.121:0/1819112732 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:50.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c79a44700 1 --2- 172.21.15.121:0/1819112732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56262cf5a670 0x56262ceca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c7a245700 1 --2- 172.21.15.121:0/1819112732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56262ce81dd0 0x56262cf54430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c7aa46700 1 -- 172.21.15.121:0/1819112732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56262cf5a670 msgr2=0x56262ceca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:50.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c7aa46700 1 --2- 172.21.15.121:0/1819112732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56262cf5a670 0x56262ceca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c7aa46700 1 -- 172.21.15.121:0/1819112732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56262ce81dd0 msgr2=0x56262cf54430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:50.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c7aa46700 1 --2- 172.21.15.121:0/1819112732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56262ce81dd0 0x56262cf54430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c7aa46700 1 -- 172.21.15.121:0/1819112732 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c005db0 con 0x56262cec6ff0 2023-11-15T03:03:50.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c7aa46700 1 --2- 172.21.15.121:0/1819112732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56262cec6ff0 0x56262cec7470 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7f7c74002920 tx=0x7f7c74005d40).ready entity=mon.0 client_cookie=8ae6f12a542c8e29 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:50.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c78a42700 1 -- 172.21.15.121:0/1819112732 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7400d8d0 con 0x56262cec6ff0 2023-11-15T03:03:50.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c78a42700 1 -- 172.21.15.121:0/1819112732 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7400da30 con 0x56262cec6ff0 2023-11-15T03:03:50.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c78a42700 1 -- 172.21.15.121:0/1819112732 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7400ddd0 con 0x56262cec6ff0 2023-11-15T03:03:50.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/1819112732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56262cec6ff0 msgr2=0x56262cec7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/1819112732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56262cec6ff0 0x56262cec7470 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7f7c74002920 tx=0x7f7c74005d40).stop 2023-11-15T03:03:50.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/1819112732 shutdown_connections 2023-11-15T03:03:50.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/1819112732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56262cec6ff0 0x56262cec7470 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/1819112732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56262cf5a670 0x56262ceca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/1819112732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56262ce81dd0 0x56262cf54430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/1819112732 >> 172.21.15.121:0/1819112732 conn(0x56262ced1080 msgr2=0x56262ced1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:50.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/1819112732 shutdown_connections 2023-11-15T03:03:50.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.284+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/1819112732 wait complete. 2023-11-15T03:03:50.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c85d3ad00 1 Processor -- start 2023-11-15T03:03:50.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c85d3ad00 1 -- start start 2023-11-15T03:03:50.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c85d3ad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56262ce81dd0 0x56262cec73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c85d3ad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56262cec6ff0 0x56262cecbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c85d3ad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56262cf5a670 0x56262cecf050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c85d3ad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56262ceb3570 con 0x56262cec6ff0 2023-11-15T03:03:50.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c85d3ad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56262cf63ec0 con 0x56262ce81dd0 2023-11-15T03:03:50.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c85d3ad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56262cf63940 con 0x56262cf5a670 2023-11-15T03:03:50.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c79a44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56262cec6ff0 0x56262cecbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c79a44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56262cec6ff0 0x56262cecbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45140/0 (socket says 172.21.15.121:45140) 2023-11-15T03:03:50.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 learned_addr learned my addr 172.21.15.121:0/2992459667 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:50.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c7a245700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56262ce81dd0 0x56262cec73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c7aa46700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56262cf5a670 0x56262cecf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56262cf5a670 msgr2=0x56262cecf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c79a44700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56262cf5a670 0x56262cecf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56262ce81dd0 msgr2=0x56262cec73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c79a44700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56262ce81dd0 0x56262cec73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c74002600 con 0x56262cec6ff0 2023-11-15T03:03:50.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c79a44700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56262cec6ff0 0x56262cecbf50 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f7c7000b330 tx=0x7f7c70001980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:50.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c6affd700 1 -- 172.21.15.121:0/2992459667 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7000bb30 con 0x56262cec6ff0 2023-11-15T03:03:50.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c6affd700 1 -- 172.21.15.121:0/2992459667 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7000bc90 con 0x56262cec6ff0 2023-11-15T03:03:50.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.288+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56262ce23770 con 0x56262cec6ff0 2023-11-15T03:03:50.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.292+0000 7f7c6affd700 1 -- 172.21.15.121:0/2992459667 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70010660 con 0x56262cec6ff0 2023-11-15T03:03:50.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.292+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56262cdf2a70 con 0x56262cec6ff0 2023-11-15T03:03:50.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.292+0000 7f7c6affd700 1 -- 172.21.15.121:0/2992459667 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7c700109d0 con 0x56262cec6ff0 2023-11-15T03:03:50.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.292+0000 7f7c6affd700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c5c058ff0 0x7f7c5c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.292+0000 7f7c6affd700 1 -- 172.21.15.121:0/2992459667 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f7c70008070 con 0x56262cec6ff0 2023-11-15T03:03:50.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.296+0000 7f7c7a245700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c5c058ff0 0x7f7c5c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.314 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:03:50.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.296+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56262ceb3a00 0x56262cf6f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.296+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cecf540 con 0x56262ceb3a00 2023-11-15T03:03:50.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.296+0000 7f7c7aa46700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56262ceb3a00 0x56262cf6f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.296+0000 7f7c7a245700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c5c058ff0 0x7f7c5c05b4b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x56262ce6ed20 tx=0x7f7c6c016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:50.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.296+0000 7f7c7aa46700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56262ceb3a00 0x56262cf6f6f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:50.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.296+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v96'17 uv10 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262ceb3a00 2023-11-15T03:03:50.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.296+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf65300 con 0x56262ceb3a00 2023-11-15T03:03:50.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.300+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v96'18 uv18 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262ceb3a00 2023-11-15T03:03:50.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.300+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf656e0 con 0x56262ceb3a00 2023-11-15T03:03:50.319 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:03:50.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.304+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v96'19 uv19 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262ceb3a00 2023-11-15T03:03:50.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.304+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf65ae0 con 0x56262ceb3a00 2023-11-15T03:03:50.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.304+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v96'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262ceb3a00 2023-11-15T03:03:50.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.304+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf65ee0 con 0x56262ceb3a00 2023-11-15T03:03:50.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.308+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v96'21 uv21 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262ceb3a00 2023-11-15T03:03:50.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.308+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf784c0 con 0x56262ceb3a00 2023-11-15T03:03:50.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.312+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v96'22 uv22 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262ceb3a00 2023-11-15T03:03:50.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.312+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf78870 con 0x56262ceb3a00 2023-11-15T03:03:50.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.316+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v96'23 uv16 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262ceb3a00 2023-11-15T03:03:50.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.316+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf78870 con 0x56262ceb3a00 2023-11-15T03:03:50.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.316+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v96'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262ceb3a00 2023-11-15T03:03:50.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.316+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf78870 con 0x56262ceb3a00 2023-11-15T03:03:50.329 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:03:50.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.320+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v96'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262ceb3a00 2023-11-15T03:03:50.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.320+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf79880 con 0x56262ceb3a00 2023-11-15T03:03:50.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.324+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v96'26 uv26 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262ceb3a00 2023-11-15T03:03:50.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.324+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf79880 con 0x56262ceb3a00 2023-11-15T03:03:50.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.328+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v96'27 uv27 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262ceb3a00 2023-11-15T03:03:50.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.328+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf78870 con 0x56262ceb3a00 2023-11-15T03:03:50.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.332+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v96'28 uv28 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262ceb3a00 2023-11-15T03:03:50.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.332+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56262cf78d80 0x56262cf7b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.332+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf7b660 con 0x56262cf78d80 2023-11-15T03:03:50.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.332+0000 7f7c7a245700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56262cf78d80 0x56262cf7b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.332+0000 7f7c7a245700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56262cf78d80 0x56262cf7b170 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:50.349 INFO:teuthology.orchestra.run.smithi121.stdout:Write #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:03:50.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.336+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v96'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.336+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf7ca20 con 0x56262cf78d80 2023-11-15T03:03:50.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.340+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [setxattr (30)] v96'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.340+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf7cdd0 con 0x56262cf78d80 2023-11-15T03:03:50.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.344+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [setxattr (62)] v96'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.344+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf7ba10 con 0x56262cf78d80 2023-11-15T03:03:50.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.344+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [omap-set-header] v96'30 uv30 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.344+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf7c300 con 0x56262cf78d80 2023-11-15T03:03:50.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.348+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [omap-set-vals] v96'31 uv31 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.348+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf7d180 con 0x56262cf78d80 2023-11-15T03:03:50.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.352+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 600.00000000 [create] v96'32 uv19 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.352+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf7d530 con 0x56262cf78d80 2023-11-15T03:03:50.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.356+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 600.00000000 [delete] v96'33 uv33 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.356+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf7d8e0 con 0x56262cf78d80 2023-11-15T03:03:50.364 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:03:50.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.360+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v96'34 uv34 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.360+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf7e8f0 con 0x56262cf78d80 2023-11-15T03:03:50.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.360+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [setxattr (30)] v96'35 uv35 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.360+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf7e8f0 con 0x56262cf78d80 2023-11-15T03:03:50.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.368+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [setxattr (62)] v96'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.368+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf7d8e0 con 0x56262cf78d80 2023-11-15T03:03:50.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.368+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [omap-set-header] v96'37 uv37 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.368+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf7d8e0 con 0x56262cf78d80 2023-11-15T03:03:50.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.372+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 603.00000000 [create] v96'38 uv25 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.372+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf7d8e0 con 0x56262cf78d80 2023-11-15T03:03:50.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.376+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 603.00000000 [delete] v96'39 uv39 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.376+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf7d8e0 con 0x56262cf78d80 2023-11-15T03:03:50.388 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:03:50.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.380+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v96'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.380+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf7e8f0 con 0x56262cf78d80 2023-11-15T03:03:50.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.384+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [setxattr (30)] v96'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.384+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf7e8f0 con 0x56262cf78d80 2023-11-15T03:03:50.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.388+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [setxattr (62)] v96'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.388+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf7d8e0 con 0x56262cf78d80 2023-11-15T03:03:50.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.392+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [omap-set-header] v96'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.392+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x56262cf7ddf0 0x56262cf801e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.392+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:30 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf806d0 con 0x56262cf7ddf0 2023-11-15T03:03:50.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.392+0000 7f7c79a44700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x56262cf7ddf0 0x56262cf801e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.392+0000 7f7c79a44700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x56262cf7ddf0 0x56262cf801e0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:50.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.396+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(30 1.00000000.inode [create] v96'17 uv10 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf7ddf0 2023-11-15T03:03:50.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.396+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:31 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf80a80 con 0x56262cf7ddf0 2023-11-15T03:03:50.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.400+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(31 1.00000000.inode [delete] v96'18 uv18 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf7ddf0 2023-11-15T03:03:50.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.400+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf80e30 con 0x56262cf7ddf0 2023-11-15T03:03:50.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.404+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(32 1.00000000.inode [create] v96'19 uv19 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf7ddf0 2023-11-15T03:03:50.409 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:03:50.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.404+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf816d0 con 0x56262cf7ddf0 2023-11-15T03:03:50.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.408+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(33 1.00000000.inode [write 0~554] v96'20 uv20 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf7ddf0 2023-11-15T03:03:50.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.408+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf81a80 con 0x56262cf7ddf0 2023-11-15T03:03:50.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.412+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v96'21 uv21 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf7ddf0 2023-11-15T03:03:50.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.412+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf81e30 con 0x56262cf7ddf0 2023-11-15T03:03:50.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.416+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(35 200.00000000 [create] v96'22 uv16 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf7ddf0 2023-11-15T03:03:50.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.416+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf821e0 con 0x56262cf7ddf0 2023-11-15T03:03:50.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.416+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(36 200.00000000 [delete] v96'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf7ddf0 2023-11-15T03:03:50.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.416+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf82590 con 0x56262cf7ddf0 2023-11-15T03:03:50.426 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:03:50.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.420+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(37 200.00000000 [create] v96'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf7ddf0 2023-11-15T03:03:50.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.420+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf82590 con 0x56262cf7ddf0 2023-11-15T03:03:50.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.424+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(38 200.00000000 [write 0~90] v96'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf7ddf0 2023-11-15T03:03:50.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.424+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf82590 con 0x56262cf7ddf0 2023-11-15T03:03:50.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.428+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(39 200.00000000 [omap-set-header] v96'26 uv26 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf7ddf0 2023-11-15T03:03:50.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.428+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56262cf82ea0 0x56262cf85290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.428+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:40 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf85780 con 0x56262cf82ea0 2023-11-15T03:03:50.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.428+0000 7f7c7aa46700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56262cf82ea0 0x56262cf85290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.428+0000 7f7c7aa46700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56262cf82ea0 0x56262cf85290 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:50.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.432+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(40 200.00000001 [create] v96'28 uv27 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.432+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:41 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf85780 con 0x56262cf82ea0 2023-11-15T03:03:50.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.432+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(41 200.00000001 [delete] v96'29 uv29 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.432+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf85780 con 0x56262cf82ea0 2023-11-15T03:03:50.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.436+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(42 200.00000001 [create] v96'30 uv30 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.443 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:03:50.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.436+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf89810 con 0x56262cf82ea0 2023-11-15T03:03:50.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.440+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(43 200.00000001 [write 0~16484] v96'31 uv31 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.440+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf81260 con 0x56262cf82ea0 2023-11-15T03:03:50.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.444+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(44 200.00000001 [omap-set-header] v96'32 uv32 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.444+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf81260 con 0x56262cf82ea0 2023-11-15T03:03:50.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.448+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(45 606.00000000 [create] v96'33 uv24 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.448+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf811e0 con 0x56262cf82ea0 2023-11-15T03:03:50.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.448+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(46 606.00000000 [delete] v96'34 uv34 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.448+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf811e0 con 0x56262cf82ea0 2023-11-15T03:03:50.458 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:03:50.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.452+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(47 606.00000000 [create] v96'35 uv35 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.452+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf811e0 con 0x56262cf82ea0 2023-11-15T03:03:50.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.456+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v96'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.456+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf811e0 con 0x56262cf82ea0 2023-11-15T03:03:50.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.456+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v96'37 uv37 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.456+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf811e0 con 0x56262cf82ea0 2023-11-15T03:03:50.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.460+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(50 606.00000000 [omap-set-header] v96'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.460+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x56262cf86180 0x56262cf88570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.460+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:51 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf88a60 con 0x56262cf86180 2023-11-15T03:03:50.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.460+0000 7f7c7a245700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x56262cf86180 0x56262cf88570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.460+0000 7f7c7a245700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x56262cf86180 0x56262cf88570 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:50.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.464+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(51 607.00000000 [create] v96'9 uv8 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf86180 2023-11-15T03:03:50.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.464+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:52 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf88aa0 con 0x56262cf86180 2023-11-15T03:03:50.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.468+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(52 607.00000000 [delete] v96'10 uv10 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf86180 2023-11-15T03:03:50.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.468+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf88aa0 con 0x56262cf86180 2023-11-15T03:03:50.475 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:03:50.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.468+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(53 607.00000000 [create] v96'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf86180 2023-11-15T03:03:50.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.468+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf89f00 con 0x56262cf86180 2023-11-15T03:03:50.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.472+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v96'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf86180 2023-11-15T03:03:50.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.472+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf89f00 con 0x56262cf86180 2023-11-15T03:03:50.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.476+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v96'13 uv13 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf86180 2023-11-15T03:03:50.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.476+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf88ef0 con 0x56262cf86180 2023-11-15T03:03:50.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.476+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v96'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf86180 2023-11-15T03:03:50.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.476+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56262cf89400 0x56262cf8b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.476+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:57 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8be70 con 0x56262cf89400 2023-11-15T03:03:50.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.476+0000 7f7c79a44700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56262cf89400 0x56262cf8b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.476+0000 7f7c79a44700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56262cf89400 0x56262cf8b7f0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:50.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.480+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v96'8 uv7 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.480+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:58 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf8c250 con 0x56262cf89400 2023-11-15T03:03:50.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.484+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v96'9 uv9 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.484+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8c250 con 0x56262cf89400 2023-11-15T03:03:50.492 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:03:50.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.484+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v96'10 uv10 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.484+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8c250 con 0x56262cf89400 2023-11-15T03:03:50.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.488+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v96'11 uv11 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.488+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8c250 con 0x56262cf89400 2023-11-15T03:03:50.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.492+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(61 608.00000000 [create] v96'9 uv8 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.492+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf8c250 con 0x56262cf89400 2023-11-15T03:03:50.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.492+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(62 608.00000000 [delete] v96'10 uv10 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.492+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8c250 con 0x56262cf89400 2023-11-15T03:03:50.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.496+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(63 608.00000000 [create] v96'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.503 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:03:50.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.496+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8d260 con 0x56262cf89400 2023-11-15T03:03:50.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.500+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v96'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.500+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8d260 con 0x56262cf89400 2023-11-15T03:03:50.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.500+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v96'13 uv13 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.500+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8c250 con 0x56262cf89400 2023-11-15T03:03:50.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.504+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(66 608.00000000 [omap-set-header] v96'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.504+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8c250 con 0x56262cf78d80 2023-11-15T03:03:50.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.508+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(67 604.00000000 [create] v96'15 uv9 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.508+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf8c250 con 0x56262cf78d80 2023-11-15T03:03:50.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.512+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(68 604.00000000 [delete] v96'16 uv16 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.512+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8c250 con 0x56262cf78d80 2023-11-15T03:03:50.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.516+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v96'17 uv17 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.521 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:03:50.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.516+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8d260 con 0x56262cf78d80 2023-11-15T03:03:50.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.516+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v96'18 uv18 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.516+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8d260 con 0x56262cf78d80 2023-11-15T03:03:50.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.520+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v96'19 uv19 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.520+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8c250 con 0x56262cf78d80 2023-11-15T03:03:50.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.524+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [omap-set-header] v96'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.524+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8c250 con 0x56262cf78d80 2023-11-15T03:03:50.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.528+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 500.00000000 [create] v96'21 uv14 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.528+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf8c250 con 0x56262cf78d80 2023-11-15T03:03:50.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.532+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 500.00000000 [delete] v96'22 uv22 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.532+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8c250 con 0x56262cf78d80 2023-11-15T03:03:50.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.532+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v96'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.540 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:03:50.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.532+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8c250 con 0x56262cf78d80 2023-11-15T03:03:50.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.536+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [write 0~90] v96'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.536+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf85800 con 0x56262cf78d80 2023-11-15T03:03:50.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.540+0000 7f7c7a245700 1 -- 172.21.15.121:0/2992459667 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [omap-set-header] v96'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c6c008a60 con 0x56262cf78d80 2023-11-15T03:03:50.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.540+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:78 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf85800 con 0x56262cf7ddf0 2023-11-15T03:03:50.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.540+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(78 mds_snaptable [create] v96'7 uv6 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf7ddf0 2023-11-15T03:03:50.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.540+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:79 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf85800 con 0x56262cf7ddf0 2023-11-15T03:03:50.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.544+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(79 mds_snaptable [delete] v96'8 uv8 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf7ddf0 2023-11-15T03:03:50.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.544+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf85800 con 0x56262cf7ddf0 2023-11-15T03:03:50.555 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:03:50.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.548+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(80 mds_snaptable [create] v96'9 uv9 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf7ddf0 2023-11-15T03:03:50.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.548+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf85800 con 0x56262cf7ddf0 2023-11-15T03:03:50.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.548+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(81 mds_snaptable [write 0~70] v96'10 uv10 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf7ddf0 2023-11-15T03:03:50.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.548+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf85800 con 0x56262cf7ddf0 2023-11-15T03:03:50.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.552+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v96'11 uv11 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf7ddf0 2023-11-15T03:03:50.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.552+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf85800 con 0x56262cf89400 2023-11-15T03:03:50.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.556+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(83 605.00000000 [create] v96'9 uv8 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.556+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf85800 con 0x56262cf89400 2023-11-15T03:03:50.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.556+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(84 605.00000000 [delete] v96'10 uv10 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.556+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf85800 con 0x56262cf89400 2023-11-15T03:03:50.567 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:03:50.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.560+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(85 605.00000000 [create] v96'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.560+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf85800 con 0x56262cf89400 2023-11-15T03:03:50.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.564+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v96'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.564+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf85800 con 0x56262cf89400 2023-11-15T03:03:50.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.568+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v96'13 uv13 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.568+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf85800 con 0x56262cf89400 2023-11-15T03:03:50.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.568+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(88 605.00000000 [omap-set-header] v96'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.568+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf85800 con 0x56262cf89400 2023-11-15T03:03:50.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.572+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(89 mds0_inotable [create] v96'8 uv7 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.572+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf85800 con 0x56262cf89400 2023-11-15T03:03:50.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.576+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(90 mds0_inotable [delete] v96'9 uv9 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.576+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf85800 con 0x56262cf89400 2023-11-15T03:03:50.585 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:03:50.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.580+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(91 mds0_inotable [create] v96'10 uv10 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.580+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf85800 con 0x56262cf89400 2023-11-15T03:03:50.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.584+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(92 mds0_inotable [write 0~34] v96'11 uv11 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.584+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8c740 con 0x56262cf89400 2023-11-15T03:03:50.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.584+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v96'12 uv12 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.584+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8c740 con 0x56262cf89400 2023-11-15T03:03:50.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.588+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(94 100.00000000 [create] v96'7 uv6 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.588+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf85800 con 0x56262cf89400 2023-11-15T03:03:50.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.592+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(95 100.00000000 [delete] v96'8 uv8 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.592+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf85800 con 0x56262cf89400 2023-11-15T03:03:50.607 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:03:50.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.596+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(96 100.00000000 [create] v96'9 uv9 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.596+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8c740 con 0x56262cf89400 2023-11-15T03:03:50.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.600+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(97 100.00000000 [omap-set-header] v96'10 uv10 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.600+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8f120 con 0x56262cf89400 2023-11-15T03:03:50.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.600+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v96'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.604+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56262cf8f300 0x56262cf8f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.604+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf8f300 2023-11-15T03:03:50.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.604+0000 7f7c7aa46700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56262cf8f300 0x56262cf8f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.604+0000 7f7c7aa46700 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56262cf8f300 0x56262cf8f770 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:50.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.608+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(99 mds0_sessionmap [create] v96'7 uv6 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf8f300 2023-11-15T03:03:50.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.608+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf8f300 2023-11-15T03:03:50.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.612+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(100 mds0_sessionmap [delete] v96'8 uv8 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf8f300 2023-11-15T03:03:50.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.612+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf8f300 2023-11-15T03:03:50.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.612+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(101 mds0_sessionmap [create] v96'9 uv9 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf8f300 2023-11-15T03:03:50.619 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:03:50.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.612+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf8f300 2023-11-15T03:03:50.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.616+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v96'10 uv10 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf8f300 2023-11-15T03:03:50.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.616+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf89400 2023-11-15T03:03:50.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.620+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(103 609.00000000 [create] v96'9 uv8 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.620+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf89400 2023-11-15T03:03:50.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.620+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(104 609.00000000 [delete] v96'10 uv10 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.620+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf89400 2023-11-15T03:03:50.632 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:03:50.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.624+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(105 609.00000000 [create] v96'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.624+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf90c70 con 0x56262cf89400 2023-11-15T03:03:50.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.628+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v96'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.628+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf90c70 con 0x56262cf89400 2023-11-15T03:03:50.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.632+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v96'13 uv13 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.632+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf89400 2023-11-15T03:03:50.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.632+0000 7f7c79a44700 1 -- 172.21.15.121:0/2992459667 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(108 609.00000000 [omap-set-header] v96'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c70048a50 con 0x56262cf89400 2023-11-15T03:03:50.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.632+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf82ea0 2023-11-15T03:03:50.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.636+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(109 400.00000000 [create] v96'7 uv6 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.636+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf82ea0 2023-11-15T03:03:50.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.640+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(110 400.00000000 [delete] v96'8 uv8 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.640+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf82ea0 2023-11-15T03:03:50.647 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:03:50.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.640+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(111 400.00000000 [create] v96'9 uv9 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.640+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf82ea0 2023-11-15T03:03:50.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.644+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(112 400.00000000 [write 0~22] v96'10 uv10 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.644+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf82ea0 2023-11-15T03:03:50.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.648+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(113 400.00000000 [omap-set-header] v96'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.648+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf82ea0 2023-11-15T03:03:50.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.648+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(114 100.00000000.inode [create] v96'18 uv10 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.648+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf82ea0 2023-11-15T03:03:50.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.652+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(115 100.00000000.inode [delete] v96'19 uv19 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.652+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf82ea0 2023-11-15T03:03:50.660 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:03:50.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.656+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(116 100.00000000.inode [create] v96'20 uv20 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.656+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf82ea0 2023-11-15T03:03:50.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.656+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v96'21 uv21 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.656+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf82ea0 2023-11-15T03:03:50.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.660+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v96'22 uv22 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.660+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf82ea0 2023-11-15T03:03:50.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.664+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(119 1.00000000 [create] v96'23 uv17 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.664+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf82ea0 2023-11-15T03:03:50.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.664+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(120 1.00000000 [delete] v96'24 uv24 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.664+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf82ea0 2023-11-15T03:03:50.674 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:03:50.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.668+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(121 1.00000000 [create] v96'25 uv25 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.668+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf90c70 con 0x56262cf82ea0 2023-11-15T03:03:50.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.672+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v96'26 uv26 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.672+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf90c70 con 0x56262cf82ea0 2023-11-15T03:03:50.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.672+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v96'27 uv27 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.672+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf8fc60 con 0x56262cf82ea0 2023-11-15T03:03:50.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.676+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v96'28 uv28 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.676+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e96) v8 -- 0x56262cf90150 con 0x56262cf82ea0 2023-11-15T03:03:50.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v96'29 uv29 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7c74017020 con 0x56262cf82ea0 2023-11-15T03:03:50.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x56262cf7ddf0 msgr2=0x56262cf801e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x56262cf7ddf0 0x56262cf801e0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56262ceb3a00 msgr2=0x56262cf6f6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56262ceb3a00 0x56262cf6f6f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x56262cf86180 msgr2=0x56262cf88570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x56262cf86180 0x56262cf88570 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56262cf78d80 msgr2=0x56262cf7b170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56262cf78d80 0x56262cf7b170 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56262cf8f300 msgr2=0x56262cf8f770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56262cf8f300 0x56262cf8f770 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56262cf82ea0 msgr2=0x56262cf85290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56262cf82ea0 0x56262cf85290 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56262cf89400 msgr2=0x56262cf8b7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56262cf89400 0x56262cf8b7f0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 reap_dead start 2023-11-15T03:03:50.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 reap_dead start 2023-11-15T03:03:50.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 reap_dead start 2023-11-15T03:03:50.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c7aa46700 1 -- 172.21.15.121:0/2992459667 reap_dead start 2023-11-15T03:03:50.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c5c058ff0 msgr2=0x7f7c5c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c5c058ff0 0x7f7c5c05b4b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x56262ce6ed20 tx=0x7f7c6c016000).stop 2023-11-15T03:03:50.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56262cec6ff0 msgr2=0x56262cecbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56262cec6ff0 0x56262cecbf50 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f7c7000b330 tx=0x7f7c70001980).stop 2023-11-15T03:03:50.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 shutdown_connections 2023-11-15T03:03:50.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c5c058ff0 0x7f7c5c05b4b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 --2- 172.21.15.121:0/2992459667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56262cec6ff0 0x56262cecbf50 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 >> 172.21.15.121:0/2992459667 conn(0x56262ced1080 msgr2=0x56262ced1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:50.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 shutdown_connections 2023-11-15T03:03:50.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.680+0000 7f7c85d3ad00 1 -- 172.21.15.121:0/2992459667 wait complete. 2023-11-15T03:03:50.697 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 600.00000000 2023-11-15T03:03:50.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7b291ed00 1 Processor -- start 2023-11-15T03:03:50.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7b291ed00 1 -- start start 2023-11-15T03:03:50.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7b291ed00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559350c7cbb0 0x559350cb1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7b291ed00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559350cc6b00 0x559350cb1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7b291ed00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559350d59650 0x559350d4e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7b291ed00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559350bf5220 con 0x559350c7cbb0 2023-11-15T03:03:50.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7b291ed00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559350cb1b40 con 0x559350d59650 2023-11-15T03:03:50.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7b291ed00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x559350caf420 con 0x559350cc6b00 2023-11-15T03:03:50.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7b0245700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559350c7cbb0 0x559350cb1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7b0245700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559350c7cbb0 0x559350cb1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45152/0 (socket says 172.21.15.121:45152) 2023-11-15T03:03:50.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7b0245700 1 -- 172.21.15.121:0/4191224543 learned_addr learned my addr 172.21.15.121:0/4191224543 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:50.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7b0a46700 1 --2- 172.21.15.121:0/4191224543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559350d59650 0x559350d4e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7a7a44700 1 --2- 172.21.15.121:0/4191224543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559350cc6b00 0x559350cb1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7b0245700 1 -- 172.21.15.121:0/4191224543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559350cc6b00 msgr2=0x559350cb1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:50.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7b0245700 1 --2- 172.21.15.121:0/4191224543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559350cc6b00 0x559350cb1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7b0245700 1 -- 172.21.15.121:0/4191224543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559350d59650 msgr2=0x559350d4e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:50.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7b0245700 1 --2- 172.21.15.121:0/4191224543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559350d59650 0x559350d4e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7b0245700 1 -- 172.21.15.121:0/4191224543 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a8005db0 con 0x559350c7cbb0 2023-11-15T03:03:50.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7b0a46700 1 --2- 172.21.15.121:0/4191224543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559350d59650 0x559350d4e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:50.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.756+0000 7ff7b0245700 1 --2- 172.21.15.121:0/4191224543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559350c7cbb0 0x559350cb1160 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7ff7a0001550 tx=0x7ff7a0006b60).ready entity=mon.0 client_cookie=88333f491834fd5a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:50.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.760+0000 7ff7af243700 1 -- 172.21.15.121:0/4191224543 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00030d0 con 0x559350c7cbb0 2023-11-15T03:03:50.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.760+0000 7ff7af243700 1 -- 172.21.15.121:0/4191224543 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a0012400 con 0x559350c7cbb0 2023-11-15T03:03:50.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.760+0000 7ff7af243700 1 -- 172.21.15.121:0/4191224543 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00127a0 con 0x559350c7cbb0 2023-11-15T03:03:50.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.760+0000 7ff7b291ed00 1 -- 172.21.15.121:0/4191224543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559350c7cbb0 msgr2=0x559350cb1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.760+0000 7ff7b291ed00 1 --2- 172.21.15.121:0/4191224543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559350c7cbb0 0x559350cb1160 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7ff7a0001550 tx=0x7ff7a0006b60).stop 2023-11-15T03:03:50.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.760+0000 7ff7b291ed00 1 -- 172.21.15.121:0/4191224543 shutdown_connections 2023-11-15T03:03:50.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.760+0000 7ff7b291ed00 1 --2- 172.21.15.121:0/4191224543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559350d59650 0x559350d4e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.760+0000 7ff7b291ed00 1 --2- 172.21.15.121:0/4191224543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559350cc6b00 0x559350cb1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.760+0000 7ff7b291ed00 1 --2- 172.21.15.121:0/4191224543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559350c7cbb0 0x559350cb1160 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.760+0000 7ff7b291ed00 1 -- 172.21.15.121:0/4191224543 >> 172.21.15.121:0/4191224543 conn(0x559350ccc080 msgr2=0x559350d4f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:50.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.760+0000 7ff7b291ed00 1 -- 172.21.15.121:0/4191224543 shutdown_connections 2023-11-15T03:03:50.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.760+0000 7ff7b291ed00 1 -- 172.21.15.121:0/4191224543 wait complete. 2023-11-15T03:03:50.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.760+0000 7ff7b291ed00 1 Processor -- start 2023-11-15T03:03:50.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.760+0000 7ff7b291ed00 1 -- start start 2023-11-15T03:03:50.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7b291ed00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559350c7cbb0 0x559350cabbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7b291ed00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559350cc6b00 0x559350d3d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7b291ed00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559350d59650 0x559350d567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7b291ed00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559350cae830 con 0x559350d59650 2023-11-15T03:03:50.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7b291ed00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559350d4d840 con 0x559350cc6b00 2023-11-15T03:03:50.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7b291ed00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x559350cae1e0 con 0x559350c7cbb0 2023-11-15T03:03:50.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7b0a46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559350d59650 0x559350d567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7b0a46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559350d59650 0x559350d567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45154/0 (socket says 172.21.15.121:45154) 2023-11-15T03:03:50.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7b0a46700 1 -- 172.21.15.121:0/4018298529 learned_addr learned my addr 172.21.15.121:0/4018298529 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:50.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7b0245700 1 --2- 172.21.15.121:0/4018298529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559350c7cbb0 0x559350cabbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7a7a44700 1 --2- 172.21.15.121:0/4018298529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559350cc6b00 0x559350d3d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7b0a46700 1 -- 172.21.15.121:0/4018298529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559350c7cbb0 msgr2=0x559350cabbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7b0a46700 1 --2- 172.21.15.121:0/4018298529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559350c7cbb0 0x559350cabbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7b0a46700 1 -- 172.21.15.121:0/4018298529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559350cc6b00 msgr2=0x559350d3d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7b0a46700 1 --2- 172.21.15.121:0/4018298529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559350cc6b00 0x559350d3d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7b0a46700 1 -- 172.21.15.121:0/4018298529 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559350c237f0 con 0x559350d59650 2023-11-15T03:03:50.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7b0a46700 1 --2- 172.21.15.121:0/4018298529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559350d59650 0x559350d567e0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 rx=0x7ff7a8008fd0 tx=0x7ff7a8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:50.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7ada40700 1 -- 172.21.15.121:0/4018298529 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8013650 con 0x559350d59650 2023-11-15T03:03:50.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7ada40700 1 -- 172.21.15.121:0/4018298529 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a80137f0 con 0x559350d59650 2023-11-15T03:03:50.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7ada40700 1 -- 172.21.15.121:0/4018298529 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8013bb0 con 0x559350d59650 2023-11-15T03:03:50.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7b291ed00 1 -- 172.21.15.121:0/4018298529 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559350c1e770 con 0x559350d59650 2023-11-15T03:03:50.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.764+0000 7ff7b291ed00 1 -- 172.21.15.121:0/4018298529 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559350beda70 con 0x559350d59650 2023-11-15T03:03:50.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.768+0000 7ff7ada40700 1 -- 172.21.15.121:0/4018298529 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff7a801d3e0 con 0x559350d59650 2023-11-15T03:03:50.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.768+0000 7ff7ada40700 1 --2- 172.21.15.121:0/4018298529 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff790058ff0 0x7ff79005b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.772+0000 7ff7ada40700 1 -- 172.21.15.121:0/4018298529 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6438+0+0 (secure 0 0 0) 0x7ff7a80745e0 con 0x559350d59650 2023-11-15T03:03:50.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.772+0000 7ff7b0245700 1 --2- 172.21.15.121:0/4018298529 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff790058ff0 0x7ff79005b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.772+0000 7ff7b291ed00 1 --2- 172.21.15.121:0/4018298529 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x559350d3ddb0 0x559350cc14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.772+0000 7ff7b291ed00 1 -- 172.21.15.121:0/4018298529 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:1 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e96) v8 -- 0x559350cc5ed0 con 0x559350d3ddb0 2023-11-15T03:03:50.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.772+0000 7ff7b0245700 1 --2- 172.21.15.121:0/4018298529 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff790058ff0 0x7ff79005b4b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7ff7a0006bc0 tx=0x7ff7a0011000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:50.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.772+0000 7ff7a7a44700 1 --2- 172.21.15.121:0/4018298529 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x559350d3ddb0 0x559350cc14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.772+0000 7ff7a7a44700 1 --2- 172.21.15.121:0/4018298529 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x559350d3ddb0 0x559350cc14f0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:50.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.776+0000 7ff7a7a44700 1 -- 172.21.15.121:0/4018298529 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(1 600.00000000 [delete] v96'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff798009bb0 con 0x559350d3ddb0 2023-11-15T03:03:50.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.776+0000 7ff7b291ed00 1 -- 172.21.15.121:0/4018298529 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x559350d3ddb0 msgr2=0x559350cc14f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.776+0000 7ff7b291ed00 1 --2- 172.21.15.121:0/4018298529 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x559350d3ddb0 0x559350cc14f0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.776+0000 7ff7b291ed00 1 -- 172.21.15.121:0/4018298529 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff790058ff0 msgr2=0x7ff79005b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.776+0000 7ff7b291ed00 1 --2- 172.21.15.121:0/4018298529 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff790058ff0 0x7ff79005b4b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7ff7a0006bc0 tx=0x7ff7a0011000).stop 2023-11-15T03:03:50.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.776+0000 7ff7b291ed00 1 -- 172.21.15.121:0/4018298529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559350d59650 msgr2=0x559350d567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.776+0000 7ff7b291ed00 1 --2- 172.21.15.121:0/4018298529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559350d59650 0x559350d567e0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 rx=0x7ff7a8008fd0 tx=0x7ff7a8016000).stop 2023-11-15T03:03:50.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.776+0000 7ff7b0a46700 1 -- 172.21.15.121:0/4018298529 reap_dead start 2023-11-15T03:03:50.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.776+0000 7ff7b291ed00 1 -- 172.21.15.121:0/4018298529 shutdown_connections 2023-11-15T03:03:50.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.776+0000 7ff7b291ed00 1 -- 172.21.15.121:0/4018298529 >> 172.21.15.121:0/4018298529 conn(0x559350ccc080 msgr2=0x559350d4f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:50.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.776+0000 7ff7b291ed00 1 -- 172.21.15.121:0/4018298529 shutdown_connections 2023-11-15T03:03:50.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.776+0000 7ff7b291ed00 1 -- 172.21.15.121:0/4018298529 wait complete. 2023-11-15T03:03:50.793 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:03:50.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.960+0000 7f31b488f700 1 Processor -- start 2023-11-15T03:03:50.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.960+0000 7f31b488f700 1 -- start start 2023-11-15T03:03:50.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.960+0000 7f31b488f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b0151af0 0x7f31b00a7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.960+0000 7f31b488f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b01528b0 con 0x7f31b00c3650 2023-11-15T03:03:50.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.960+0000 7f31b488f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b0151ff0 con 0x7f31b00a5720 2023-11-15T03:03:50.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.960+0000 7f31b488f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b0152260 con 0x7f31b0151af0 2023-11-15T03:03:50.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.960+0000 7f31affff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f31b00c3650 0x7f31b0065d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58236/0 (socket says 172.21.15.121:58236) 2023-11-15T03:03:50.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.960+0000 7f31a6ffd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f31b00a5720 0x7f31b00c2f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:36940/0 (socket says 172.21.15.121:36940) 2023-11-15T03:03:50.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.960+0000 7f31affff700 1 -- 172.21.15.121:0/3338599385 learned_addr learned my addr 172.21.15.121:0/3338599385 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:50.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.960+0000 7f31af7fe700 1 --2- 172.21.15.121:0/3338599385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b0151af0 0x7f31b00a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.960+0000 7f31af7fe700 1 -- 172.21.15.121:0/3338599385 >> v1:172.21.15.190:6789/0 conn(0x7f31b00a5720 legacy=0x7f31b00c2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.960+0000 7f31af7fe700 1 -- 172.21.15.121:0/3338599385 >> v1:172.21.15.121:6789/0 conn(0x7f31b00c3650 legacy=0x7f31b0065d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.960+0000 7f31af7fe700 1 -- 172.21.15.121:0/3338599385 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b0152540 con 0x7f31b0151af0 2023-11-15T03:03:50.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.964+0000 7f31af7fe700 1 --2- 172.21.15.121:0/3338599385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b0151af0 0x7f31b00a7070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f31a0009790 tx=0x7f31a0009060).ready entity=mon.2 client_cookie=3359826ba59d6d8b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:50.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.964+0000 7f31aeffd700 1 -- 172.21.15.121:0/3338599385 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a000b5a0 con 0x7f31b0151af0 2023-11-15T03:03:50.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.964+0000 7f31aeffd700 1 -- 172.21.15.121:0/3338599385 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a000b700 con 0x7f31b0151af0 2023-11-15T03:03:50.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.964+0000 7f31aeffd700 1 -- 172.21.15.121:0/3338599385 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a000ba80 con 0x7f31b0151af0 2023-11-15T03:03:50.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.964+0000 7f31b488f700 1 -- 172.21.15.121:0/3338599385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b0151af0 msgr2=0x7f31b00a7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.964+0000 7f31b488f700 1 --2- 172.21.15.121:0/3338599385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b0151af0 0x7f31b00a7070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f31a0009790 tx=0x7f31a0009060).stop 2023-11-15T03:03:50.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.964+0000 7f31b488f700 1 -- 172.21.15.121:0/3338599385 shutdown_connections 2023-11-15T03:03:50.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.964+0000 7f31b488f700 1 --2- 172.21.15.121:0/3338599385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b0151af0 0x7f31b00a7070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.964+0000 7f31b488f700 1 -- 172.21.15.121:0/3338599385 >> 172.21.15.121:0/3338599385 conn(0x7f31b0067a60 msgr2=0x7f31b0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:50.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.964+0000 7f31b488f700 1 -- 172.21.15.121:0/3338599385 shutdown_connections 2023-11-15T03:03:50.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.964+0000 7f31b488f700 1 -- 172.21.15.121:0/3338599385 wait complete. 2023-11-15T03:03:50.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.964+0000 7f31b488f700 1 Processor -- start 2023-11-15T03:03:50.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31b488f700 1 -- start start 2023-11-15T03:03:50.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31b488f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31b00a5720 0x7f31b00bf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31b488f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b00c3650 0x7f31b0168450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31b488f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b01446f0 0x7f31b00c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31b488f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b0169460 con 0x7f31b00a5720 2023-11-15T03:03:50.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31b488f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b00bb7a0 con 0x7f31b00c3650 2023-11-15T03:03:50.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31b488f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b0168940 con 0x7f31b01446f0 2023-11-15T03:03:50.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31af7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31b00a5720 0x7f31b00bf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31af7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31b00a5720 0x7f31b00bf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45156/0 (socket says 172.21.15.121:45156) 2023-11-15T03:03:50.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31af7fe700 1 -- 172.21.15.121:0/965149705 learned_addr learned my addr 172.21.15.121:0/965149705 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:50.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31af7fe700 1 -- 172.21.15.121:0/965149705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b01446f0 msgr2=0x7f31b00c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31af7fe700 1 --2- 172.21.15.121:0/965149705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b01446f0 0x7f31b00c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:03:50.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31af7fe700 1 -- 172.21.15.121:0/965149705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b00c3650 msgr2=0x7f31b0168450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:50.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31a6ffd700 1 --2- 172.21.15.121:0/965149705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b00c3650 0x7f31b0168450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31af7fe700 1 --2- 172.21.15.121:0/965149705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b00c3650 0x7f31b0168450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:50.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31af7fe700 1 -- 172.21.15.121:0/965149705 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a00092a0 con 0x7f31b00a5720 2023-11-15T03:03:50.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31a6ffd700 1 --2- 172.21.15.121:0/965149705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b00c3650 0x7f31b0168450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:50.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31af7fe700 1 --2- 172.21.15.121:0/965149705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31b00a5720 0x7f31b00bf0c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7f31b00c75a0 tx=0x7f31a0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:50.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31a7fff700 1 -- 172.21.15.121:0/965149705 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a000ba80 con 0x7f31b00a5720 2023-11-15T03:03:50.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31a7fff700 1 -- 172.21.15.121:0/965149705 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a0000c90 con 0x7f31b00a5720 2023-11-15T03:03:50.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31a7fff700 1 -- 172.21.15.121:0/965149705 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a001b660 con 0x7f31b00a5720 2023-11-15T03:03:50.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31b488f700 1 -- 172.21.15.121:0/965149705 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b0168bc0 con 0x7f31b00a5720 2023-11-15T03:03:50.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.968+0000 7f31b488f700 1 -- 172.21.15.121:0/965149705 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b0169180 con 0x7f31b00a5720 2023-11-15T03:03:50.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.972+0000 7f31b488f700 1 -- 172.21.15.121:0/965149705 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b006bcf0 con 0x7f31b00a5720 2023-11-15T03:03:50.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.972+0000 7f31a7fff700 1 -- 172.21.15.121:0/965149705 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f31a001b9d0 con 0x7f31b00a5720 2023-11-15T03:03:50.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.972+0000 7f31a7fff700 1 --2- 172.21.15.121:0/965149705 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f319005a4e0 0x7f319005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:50.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.972+0000 7f31a7fff700 1 -- 172.21.15.121:0/965149705 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f31a0076320 con 0x7f31b00a5720 2023-11-15T03:03:50.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.972+0000 7f31a6ffd700 1 --2- 172.21.15.121:0/965149705 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f319005a4e0 0x7f319005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:50.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.976+0000 7f31a6ffd700 1 --2- 172.21.15.121:0/965149705 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f319005a4e0 0x7f319005c9a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f31b00a5650 tx=0x7f3198009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:50.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:50.980+0000 7f31a7fff700 1 -- 172.21.15.121:0/965149705 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f31a0049900 con 0x7f31b00a5720 2023-11-15T03:03:51.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:51.180+0000 7f31b488f700 1 -- 172.21.15.121:0/965149705 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f31b0068a70 con 0x7f31b00a5720 2023-11-15T03:03:52.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.176+0000 7f31a7fff700 1 -- 172.21.15.121:0/965149705 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v93) v1 ==== 162+0+0 (secure 0 0 0) 0x7f31a004cf20 con 0x7f31b00a5720 2023-11-15T03:03:52.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.180+0000 7f31b488f700 1 -- 172.21.15.121:0/965149705 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f319005a4e0 msgr2=0x7f319005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:52.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.180+0000 7f31b488f700 1 --2- 172.21.15.121:0/965149705 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f319005a4e0 0x7f319005c9a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f31b00a5650 tx=0x7f3198009040).stop 2023-11-15T03:03:52.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.180+0000 7f31b488f700 1 -- 172.21.15.121:0/965149705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31b00a5720 msgr2=0x7f31b00bf0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:52.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.180+0000 7f31b488f700 1 --2- 172.21.15.121:0/965149705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31b00a5720 0x7f31b00bf0c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7f31b00c75a0 tx=0x7f31a0015000).stop 2023-11-15T03:03:52.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.180+0000 7f31b488f700 1 -- 172.21.15.121:0/965149705 shutdown_connections 2023-11-15T03:03:52.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.180+0000 7f31b488f700 1 --2- 172.21.15.121:0/965149705 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f319005a4e0 0x7f319005c9a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:52.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.180+0000 7f31b488f700 1 --2- 172.21.15.121:0/965149705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b01446f0 0x7f31b00c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:03:52.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.180+0000 7f31b488f700 1 --2- 172.21.15.121:0/965149705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b00c3650 0x7f31b0168450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:52.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.180+0000 7f31b488f700 1 --2- 172.21.15.121:0/965149705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31b00a5720 0x7f31b00bf0c0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:52.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.180+0000 7f31b488f700 1 -- 172.21.15.121:0/965149705 >> 172.21.15.121:0/965149705 conn(0x7f31b0067a60 msgr2=0x7f31b0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:52.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.180+0000 7f31b488f700 1 -- 172.21.15.121:0/965149705 shutdown_connections 2023-11-15T03:03:52.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.180+0000 7f31b488f700 1 -- 172.21.15.121:0/965149705 wait complete. 2023-11-15T03:03:52.190 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:03:52.210 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:03:52.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.368+0000 7f6f4b4f1700 1 Processor -- start 2023-11-15T03:03:52.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.372+0000 7f6f4b4f1700 1 -- start start 2023-11-15T03:03:52.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.372+0000 7f6f4b4f1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6f44153da0 0x7f6f440c1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:52.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.372+0000 7f6f4b4f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f44157f70 0x7f6f440c2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:52.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.372+0000 7f6f4b4f1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f440c3320 con 0x7f6f440c5e10 2023-11-15T03:03:52.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.372+0000 7f6f4b4f1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f44155190 con 0x7f6f44153da0 2023-11-15T03:03:52.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.372+0000 7f6f4b4f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f440c2710 con 0x7f6f44157f70 2023-11-15T03:03:52.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.372+0000 7f6f48a8c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6f440c5e10 0x7f6f44157860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58252/0 (socket says 172.21.15.121:58252) 2023-11-15T03:03:52.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.372+0000 7f6f48a8c700 1 -- 172.21.15.121:0/2990550965 learned_addr learned my addr 172.21.15.121:0/2990550965 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:52.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.372+0000 7f6f49a8e700 1 --2- 172.21.15.121:0/2990550965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f44157f70 0x7f6f440c2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:52.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.372+0000 7f6f4928d700 1 --2- 172.21.15.121:0/2990550965 >> v2:172.21.15.121:3300/0 conn(0x7f6f44153da0 0x7f6f440c1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:52.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.372+0000 7f6f3bfff700 1 -- 172.21.15.121:0/2990550965 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1867625874 0 0) 0x7f6f440c3320 con 0x7f6f440c5e10 2023-11-15T03:03:52.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.372+0000 7f6f3bfff700 1 -- 172.21.15.121:0/2990550965 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f300033f0 con 0x7f6f440c5e10 2023-11-15T03:03:52.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.376+0000 7f6f49a8e700 1 -- 172.21.15.121:0/2990550965 >> v2:172.21.15.121:3300/0 conn(0x7f6f44153da0 msgr2=0x7f6f440c1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:52.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.376+0000 7f6f49a8e700 1 --2- 172.21.15.121:0/2990550965 >> v2:172.21.15.121:3300/0 conn(0x7f6f44153da0 0x7f6f440c1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:52.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.376+0000 7f6f49a8e700 1 -- 172.21.15.121:0/2990550965 >> v1:172.21.15.121:6789/0 conn(0x7f6f440c5e10 legacy=0x7f6f44157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:52.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.376+0000 7f6f49a8e700 1 -- 172.21.15.121:0/2990550965 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f440c29f0 con 0x7f6f44157f70 2023-11-15T03:03:52.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.376+0000 7f6f49a8e700 1 --2- 172.21.15.121:0/2990550965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f44157f70 0x7f6f440c2140 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f6f3c009760 tx=0x7f6f3c009060).ready entity=mon.2 client_cookie=c9e804f6979ef259 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:52.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.376+0000 7f6f3bfff700 1 -- 172.21.15.121:0/2990550965 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c00d4a0 con 0x7f6f44157f70 2023-11-15T03:03:52.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.376+0000 7f6f3bfff700 1 -- 172.21.15.121:0/2990550965 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3c00d600 con 0x7f6f44157f70 2023-11-15T03:03:52.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.376+0000 7f6f3bfff700 1 -- 172.21.15.121:0/2990550965 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c00d980 con 0x7f6f44157f70 2023-11-15T03:03:52.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.376+0000 7f6f4b4f1700 1 -- 172.21.15.121:0/2990550965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f44157f70 msgr2=0x7f6f440c2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:52.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.376+0000 7f6f4b4f1700 1 --2- 172.21.15.121:0/2990550965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f44157f70 0x7f6f440c2140 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f6f3c009760 tx=0x7f6f3c009060).stop 2023-11-15T03:03:52.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.376+0000 7f6f4b4f1700 1 -- 172.21.15.121:0/2990550965 shutdown_connections 2023-11-15T03:03:52.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.376+0000 7f6f4b4f1700 1 --2- 172.21.15.121:0/2990550965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f44157f70 0x7f6f440c2140 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:52.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.376+0000 7f6f4b4f1700 1 --2- 172.21.15.121:0/2990550965 >> v2:172.21.15.121:3300/0 conn(0x7f6f44153da0 0x7f6f440c1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:52.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.376+0000 7f6f4b4f1700 1 -- 172.21.15.121:0/2990550965 >> 172.21.15.121:0/2990550965 conn(0x7f6f440c7ac0 msgr2=0x7f6f440c3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:52.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.376+0000 7f6f4b4f1700 1 -- 172.21.15.121:0/2990550965 shutdown_connections 2023-11-15T03:03:52.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.376+0000 7f6f4b4f1700 1 -- 172.21.15.121:0/2990550965 wait complete. 2023-11-15T03:03:52.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.380+0000 7f6f4b4f1700 1 Processor -- start 2023-11-15T03:03:52.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.380+0000 7f6f4b4f1700 1 -- start start 2023-11-15T03:03:52.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.380+0000 7f6f4b4f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f440c5e10 0x7f6f440ab850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:52.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.380+0000 7f6f4b4f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f44153da0 0x7f6f440a8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:52.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.380+0000 7f6f4b4f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f44157f70 0x7f6f4414da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:52.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.380+0000 7f6f4b4f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f440a9330 con 0x7f6f44157f70 2023-11-15T03:03:52.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.380+0000 7f6f4b4f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f440a9610 con 0x7f6f44153da0 2023-11-15T03:03:52.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.380+0000 7f6f4b4f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f440a98f0 con 0x7f6f440c5e10 2023-11-15T03:03:52.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.380+0000 7f6f49a8e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f44157f70 0x7f6f4414da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:52.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.380+0000 7f6f49a8e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f44157f70 0x7f6f4414da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45178/0 (socket says 172.21.15.121:45178) 2023-11-15T03:03:52.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.380+0000 7f6f49a8e700 1 -- 172.21.15.121:0/2395330542 learned_addr learned my addr 172.21.15.121:0/2395330542 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:52.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.384+0000 7f6f48a8c700 1 --2- 172.21.15.121:0/2395330542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f44153da0 0x7f6f440a8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:52.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.384+0000 7f6f4928d700 1 --2- 172.21.15.121:0/2395330542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f440c5e10 0x7f6f440ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:52.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.384+0000 7f6f49a8e700 1 -- 172.21.15.121:0/2395330542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f440c5e10 msgr2=0x7f6f440ab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:52.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.384+0000 7f6f49a8e700 1 --2- 172.21.15.121:0/2395330542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f440c5e10 0x7f6f440ab850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:52.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.384+0000 7f6f49a8e700 1 -- 172.21.15.121:0/2395330542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f44153da0 msgr2=0x7f6f440a8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:52.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.384+0000 7f6f49a8e700 1 --2- 172.21.15.121:0/2395330542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f44153da0 0x7f6f440a8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:52.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.384+0000 7f6f49a8e700 1 -- 172.21.15.121:0/2395330542 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c0092a0 con 0x7f6f44157f70 2023-11-15T03:03:52.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.384+0000 7f6f49a8e700 1 --2- 172.21.15.121:0/2395330542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f44157f70 0x7f6f4414da80 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7f6f3c010000 tx=0x7f6f3c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:52.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.384+0000 7f6f397fa700 1 -- 172.21.15.121:0/2395330542 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c00d770 con 0x7f6f44157f70 2023-11-15T03:03:52.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.384+0000 7f6f397fa700 1 -- 172.21.15.121:0/2395330542 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3c00d8d0 con 0x7f6f44157f70 2023-11-15T03:03:52.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.384+0000 7f6f397fa700 1 -- 172.21.15.121:0/2395330542 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c01d640 con 0x7f6f44157f70 2023-11-15T03:03:52.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.384+0000 7f6f4b4f1700 1 -- 172.21.15.121:0/2395330542 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f440a71d0 con 0x7f6f44157f70 2023-11-15T03:03:52.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.384+0000 7f6f4b4f1700 1 -- 172.21.15.121:0/2395330542 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f440a90d0 con 0x7f6f44157f70 2023-11-15T03:03:52.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.384+0000 7f6f4b4f1700 1 -- 172.21.15.121:0/2395330542 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f440a7010 con 0x7f6f44157f70 2023-11-15T03:03:52.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.388+0000 7f6f397fa700 1 -- 172.21.15.121:0/2395330542 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6f3c01d9b0 con 0x7f6f44157f70 2023-11-15T03:03:52.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.388+0000 7f6f397fa700 1 --2- 172.21.15.121:0/2395330542 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6f2805a510 0x7f6f2805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:52.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.388+0000 7f6f397fa700 1 -- 172.21.15.121:0/2395330542 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f6f3c076320 con 0x7f6f44157f70 2023-11-15T03:03:52.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.388+0000 7f6f4928d700 1 --2- 172.21.15.121:0/2395330542 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6f2805a510 0x7f6f2805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:52.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.388+0000 7f6f4928d700 1 --2- 172.21.15.121:0/2395330542 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6f2805a510 0x7f6f2805c9d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f6f400076a0 tx=0x7f6f4000b040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:52.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.396+0000 7f6f397fa700 1 -- 172.21.15.121:0/2395330542 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6f3c0499c0 con 0x7f6f44157f70 2023-11-15T03:03:52.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.596+0000 7f6f4b4f1700 1 -- 172.21.15.121:0/2395330542 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f440a7410 con 0x7f6f44157f70 2023-11-15T03:03:52.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.600+0000 7f6f397fa700 1 -- 172.21.15.121:0/2395330542 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4501 (secure 0 0 0) 0x7f6f3c04cfe0 con 0x7f6f44157f70 2023-11-15T03:03:52.607 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:52.607 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:03:52.178096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7659},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7659":{"gid":7659,"name":"a","rank":0,"incarnation":94,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/1378715461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1378715461},{"type":"v1","addr":"172.21.15.121:6836","nonce":1378715461}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:03:52.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.600+0000 7f6f4b4f1700 1 -- 172.21.15.121:0/2395330542 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6f2805a510 msgr2=0x7f6f2805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:52.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.600+0000 7f6f4b4f1700 1 --2- 172.21.15.121:0/2395330542 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6f2805a510 0x7f6f2805c9d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f6f400076a0 tx=0x7f6f4000b040).stop 2023-11-15T03:03:52.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.600+0000 7f6f4b4f1700 1 -- 172.21.15.121:0/2395330542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f44157f70 msgr2=0x7f6f4414da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:52.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.600+0000 7f6f4b4f1700 1 --2- 172.21.15.121:0/2395330542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f44157f70 0x7f6f4414da80 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7f6f3c010000 tx=0x7f6f3c016000).stop 2023-11-15T03:03:52.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.604+0000 7f6f4b4f1700 1 -- 172.21.15.121:0/2395330542 shutdown_connections 2023-11-15T03:03:52.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.604+0000 7f6f4b4f1700 1 --2- 172.21.15.121:0/2395330542 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6f2805a510 0x7f6f2805c9d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:52.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.604+0000 7f6f4b4f1700 1 --2- 172.21.15.121:0/2395330542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f44157f70 0x7f6f4414da80 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:52.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.604+0000 7f6f4b4f1700 1 --2- 172.21.15.121:0/2395330542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f44153da0 0x7f6f440a8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:52.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.604+0000 7f6f4b4f1700 1 --2- 172.21.15.121:0/2395330542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f440c5e10 0x7f6f440ab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:52.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.604+0000 7f6f4b4f1700 1 -- 172.21.15.121:0/2395330542 >> 172.21.15.121:0/2395330542 conn(0x7f6f440c7ac0 msgr2=0x7f6f44068970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:52.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.604+0000 7f6f4b4f1700 1 -- 172.21.15.121:0/2395330542 shutdown_connections 2023-11-15T03:03:52.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:52.604+0000 7f6f4b4f1700 1 -- 172.21.15.121:0/2395330542 wait complete. 2023-11-15T03:03:52.613 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 94 2023-11-15T03:03:52.630 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:03:55.223 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:03:55.216+0000 7fa046746700 -1 log_channel(cluster) log [ERR] : dir 0x600 object missing on disk; some files may be lost (~mds0/stray0) 2023-11-15T03:03:55.223 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:03:55.216+0000 7fa046746700 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2023-11-15T03:03:56.218 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:03:56.212+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:03:56.219 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:03:56.212+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:03:56.234 INFO:tasks.ceph.mds.a.smithi121.stderr: -17> 2023-11-15T03:03:55.216+0000 7fa046746700 -1 log_channel(cluster) log [ERR] : dir 0x600 object missing on disk; some files may be lost (~mds0/stray0) 2023-11-15T03:03:56.234 INFO:tasks.ceph.mds.a.smithi121.stderr: -15> 2023-11-15T03:03:55.216+0000 7fa046746700 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2023-11-15T03:03:56.271 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:03:57.631 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:03:57.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.788+0000 7fe951c1b700 1 Processor -- start 2023-11-15T03:03:57.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.788+0000 7fe951c1b700 1 -- start start 2023-11-15T03:03:57.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.792+0000 7fe951c1b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe94c0a5ed0 0x7fe94c155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:57.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.792+0000 7fe951c1b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe94c0c4c30 con 0x7fe94c154ca0 2023-11-15T03:03:57.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.792+0000 7fe951c1b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe94c0a6f50 con 0x7fe94c1562f0 2023-11-15T03:03:57.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.792+0000 7fe951c1b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe94c0a8280 con 0x7fe94c0a5ed0 2023-11-15T03:03:57.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.792+0000 7fe94b7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe94c154ca0 0x7fe94c0a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:38208/0 (socket says 172.21.15.121:38208) 2023-11-15T03:03:57.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.792+0000 7fe94b7fe700 1 -- 172.21.15.121:0/4031791577 learned_addr learned my addr 172.21.15.121:0/4031791577 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:57.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.792+0000 7fe94bfff700 1 --2- 172.21.15.121:0/4031791577 >> v2:172.21.15.190:3300/0 conn(0x7fe94c0a5ed0 0x7fe94c155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:57.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.792+0000 7fe94a7fc700 1 -- 172.21.15.121:0/4031791577 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203029880 0 0) 0x7fe94c0c4c30 con 0x7fe94c154ca0 2023-11-15T03:03:57.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.792+0000 7fe94a7fc700 1 -- 172.21.15.121:0/4031791577 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9340033f0 con 0x7fe94c154ca0 2023-11-15T03:03:57.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.792+0000 7fe94bfff700 1 -- 172.21.15.121:0/4031791577 >> v1:172.21.15.190:6789/0 conn(0x7fe94c1562f0 legacy=0x7fe94c0a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:57.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.792+0000 7fe94a7fc700 1 -- 172.21.15.121:0/4031791577 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1362893329 0 0) 0x7fe9340033f0 con 0x7fe94c154ca0 2023-11-15T03:03:57.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.792+0000 7fe94bfff700 1 -- 172.21.15.121:0/4031791577 >> v1:172.21.15.121:6789/0 conn(0x7fe94c154ca0 legacy=0x7fe94c0a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:57.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.792+0000 7fe94bfff700 1 -- 172.21.15.121:0/4031791577 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94c0a7230 con 0x7fe94c0a5ed0 2023-11-15T03:03:57.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.792+0000 7fe94bfff700 1 --2- 172.21.15.121:0/4031791577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94c0a5ed0 0x7fe94c155080 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fe944009e60 tx=0x7fe944009780).ready entity=mon.1 client_cookie=93a1ed18092433ea server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:57.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.796+0000 7fe94a7fc700 1 -- 172.21.15.121:0/4031791577 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94400e070 con 0x7fe94c0a5ed0 2023-11-15T03:03:57.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.796+0000 7fe94a7fc700 1 -- 172.21.15.121:0/4031791577 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94400b850 con 0x7fe94c0a5ed0 2023-11-15T03:03:57.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.796+0000 7fe94a7fc700 1 -- 172.21.15.121:0/4031791577 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c001470 con 0x7fe94c0a5ed0 2023-11-15T03:03:57.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.796+0000 7fe951c1b700 1 -- 172.21.15.121:0/4031791577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94c0a5ed0 msgr2=0x7fe94c155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:57.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.796+0000 7fe951c1b700 1 --2- 172.21.15.121:0/4031791577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94c0a5ed0 0x7fe94c155080 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fe944009e60 tx=0x7fe944009780).stop 2023-11-15T03:03:57.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.796+0000 7fe951c1b700 1 -- 172.21.15.121:0/4031791577 shutdown_connections 2023-11-15T03:03:57.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.796+0000 7fe951c1b700 1 --2- 172.21.15.121:0/4031791577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94c0a5ed0 0x7fe94c155080 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:57.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.796+0000 7fe951c1b700 1 -- 172.21.15.121:0/4031791577 >> 172.21.15.121:0/4031791577 conn(0x7fe94c062190 msgr2=0x7fe94c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:57.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.796+0000 7fe951c1b700 1 -- 172.21.15.121:0/4031791577 shutdown_connections 2023-11-15T03:03:57.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.796+0000 7fe951c1b700 1 -- 172.21.15.121:0/4031791577 wait complete. 2023-11-15T03:03:57.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.800+0000 7fe951c1b700 1 Processor -- start 2023-11-15T03:03:57.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.800+0000 7fe951c1b700 1 -- start start 2023-11-15T03:03:57.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.800+0000 7fe951c1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe94c0a5ed0 0x7fe94c0be340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:57.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.800+0000 7fe951c1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe94c154ca0 0x7fe94c0bb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:57.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.800+0000 7fe951c1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94c1562f0 0x7fe94c14d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:57.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.800+0000 7fe951c1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c0be830 con 0x7fe94c154ca0 2023-11-15T03:03:57.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.800+0000 7fe951c1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c0bea20 con 0x7fe94c1562f0 2023-11-15T03:03:57.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.800+0000 7fe951c1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c0bed00 con 0x7fe94c0a5ed0 2023-11-15T03:03:57.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.800+0000 7fe94affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe94c154ca0 0x7fe94c0bb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:57.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.800+0000 7fe94affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe94c154ca0 0x7fe94c0bb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38442/0 (socket says 172.21.15.121:38442) 2023-11-15T03:03:57.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.800+0000 7fe94affd700 1 -- 172.21.15.121:0/4007561280 learned_addr learned my addr 172.21.15.121:0/4007561280 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:57.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.800+0000 7fe94b7fe700 1 --2- 172.21.15.121:0/4007561280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe94c0a5ed0 0x7fe94c0be340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:57.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.800+0000 7fe94bfff700 1 --2- 172.21.15.121:0/4007561280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94c1562f0 0x7fe94c14d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:57.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.800+0000 7fe94affd700 1 -- 172.21.15.121:0/4007561280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe94c0a5ed0 msgr2=0x7fe94c0be340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:57.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.800+0000 7fe94affd700 1 --2- 172.21.15.121:0/4007561280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe94c0a5ed0 0x7fe94c0be340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:57.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.800+0000 7fe94affd700 1 -- 172.21.15.121:0/4007561280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94c1562f0 msgr2=0x7fe94c14d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:57.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.800+0000 7fe94affd700 1 --2- 172.21.15.121:0/4007561280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94c1562f0 0x7fe94c14d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:57.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.800+0000 7fe94affd700 1 -- 172.21.15.121:0/4007561280 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe944009a20 con 0x7fe94c154ca0 2023-11-15T03:03:57.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.804+0000 7fe94affd700 1 --2- 172.21.15.121:0/4007561280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe94c154ca0 0x7fe94c0bb980 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7fe94000c440 tx=0x7fe940007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:57.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.804+0000 7fe9337fe700 1 -- 172.21.15.121:0/4007561280 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940007020 con 0x7fe94c154ca0 2023-11-15T03:03:57.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.804+0000 7fe9337fe700 1 -- 172.21.15.121:0/4007561280 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe940007180 con 0x7fe94c154ca0 2023-11-15T03:03:57.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.804+0000 7fe9337fe700 1 -- 172.21.15.121:0/4007561280 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940007520 con 0x7fe94c154ca0 2023-11-15T03:03:57.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.804+0000 7fe951c1b700 1 -- 172.21.15.121:0/4007561280 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94c0befe0 con 0x7fe94c154ca0 2023-11-15T03:03:57.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.804+0000 7fe951c1b700 1 -- 172.21.15.121:0/4007561280 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94c0bf5a0 con 0x7fe94c154ca0 2023-11-15T03:03:57.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.804+0000 7fe951c1b700 1 -- 172.21.15.121:0/4007561280 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe94c0bbed0 con 0x7fe94c154ca0 2023-11-15T03:03:57.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.804+0000 7fe9337fe700 1 -- 172.21.15.121:0/4007561280 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe9400068b0 con 0x7fe94c154ca0 2023-11-15T03:03:57.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.808+0000 7fe9337fe700 1 --2- 172.21.15.121:0/4007561280 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe93805a510 0x7fe93805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:57.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.808+0000 7fe9337fe700 1 -- 172.21.15.121:0/4007561280 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6524+0+0 (secure 0 0 0) 0x7fe9400742b0 con 0x7fe94c154ca0 2023-11-15T03:03:57.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.808+0000 7fe94b7fe700 1 --2- 172.21.15.121:0/4007561280 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe93805a510 0x7fe93805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:57.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.808+0000 7fe94b7fe700 1 --2- 172.21.15.121:0/4007561280 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe93805a510 0x7fe93805c9d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7fe93c001ed0 tx=0x7fe93c000bc0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:57.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:57.812+0000 7fe9337fe700 1 -- 172.21.15.121:0/4007561280 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe940047840 con 0x7fe94c154ca0 2023-11-15T03:03:58.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.004+0000 7fe951c1b700 1 -- 172.21.15.121:0/4007561280 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe94c06c150 con 0x7fe94c154ca0 2023-11-15T03:03:58.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.008+0000 7fe9337fe700 1 -- 172.21.15.121:0/4007561280 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4492 (secure 0 0 0) 0x7fe94004ae60 con 0x7fe94c154ca0 2023-11-15T03:03:58.014 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:58.014 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":99,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7704,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3764682615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3764682615},{"type":"v1","addr":"172.21.15.121:6836","nonce":3764682615}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:03:56.215692+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:03:58.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.008+0000 7fe951c1b700 1 -- 172.21.15.121:0/4007561280 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe93805a510 msgr2=0x7fe93805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:58.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.008+0000 7fe951c1b700 1 --2- 172.21.15.121:0/4007561280 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe93805a510 0x7fe93805c9d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7fe93c001ed0 tx=0x7fe93c000bc0).stop 2023-11-15T03:03:58.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.008+0000 7fe951c1b700 1 -- 172.21.15.121:0/4007561280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe94c154ca0 msgr2=0x7fe94c0bb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:58.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.008+0000 7fe951c1b700 1 --2- 172.21.15.121:0/4007561280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe94c154ca0 0x7fe94c0bb980 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7fe94000c440 tx=0x7fe940007dd0).stop 2023-11-15T03:03:58.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.008+0000 7fe951c1b700 1 -- 172.21.15.121:0/4007561280 shutdown_connections 2023-11-15T03:03:58.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.008+0000 7fe951c1b700 1 --2- 172.21.15.121:0/4007561280 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe93805a510 0x7fe93805c9d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.008+0000 7fe951c1b700 1 --2- 172.21.15.121:0/4007561280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94c1562f0 0x7fe94c14d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.008+0000 7fe951c1b700 1 --2- 172.21.15.121:0/4007561280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe94c154ca0 0x7fe94c0bb980 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.008+0000 7fe951c1b700 1 --2- 172.21.15.121:0/4007561280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe94c0a5ed0 0x7fe94c0be340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.008+0000 7fe951c1b700 1 -- 172.21.15.121:0/4007561280 >> 172.21.15.121:0/4007561280 conn(0x7fe94c062190 msgr2=0x7fe94c0a8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:58.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.008+0000 7fe951c1b700 1 -- 172.21.15.121:0/4007561280 shutdown_connections 2023-11-15T03:03:58.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.008+0000 7fe951c1b700 1 -- 172.21.15.121:0/4007561280 wait complete. 2023-11-15T03:03:58.023 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 99 2023-11-15T03:03:58.036 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:03:58.037 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:03:58.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.192+0000 7fc994a79700 1 Processor -- start 2023-11-15T03:03:58.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.192+0000 7fc994a79700 1 -- start start 2023-11-15T03:03:58.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc994a79700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc990066320 0x7fc990154110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:58.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc994a79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9900acf10 0x7fc990062190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:58.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc994a79700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9900c73e0 con 0x7fc990066700 2023-11-15T03:03:58.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc994a79700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9901546c0 con 0x7fc990066320 2023-11-15T03:03:58.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc994a79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9901549a0 con 0x7fc9900acf10 2023-11-15T03:03:58.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc98effd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc990066700 0x7fc9900656a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:38222/0 (socket says 172.21.15.121:38222) 2023-11-15T03:03:58.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc98effd700 1 -- 172.21.15.121:0/1941158687 learned_addr learned my addr 172.21.15.121:0/1941158687 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:03:58.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc98f7fe700 1 --2- 172.21.15.121:0/1941158687 >> v2:172.21.15.190:3300/0 conn(0x7fc990066320 0x7fc990154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:58.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc98ffff700 1 --2- 172.21.15.121:0/1941158687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9900acf10 0x7fc990062190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:58.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc98e7fc700 1 -- 172.21.15.121:0/1941158687 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 394835039 0 0) 0x7fc9900c73e0 con 0x7fc990066700 2023-11-15T03:03:58.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc98e7fc700 1 -- 172.21.15.121:0/1941158687 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9780033f0 con 0x7fc990066700 2023-11-15T03:03:58.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc98ffff700 1 -- 172.21.15.121:0/1941158687 >> v2:172.21.15.190:3300/0 conn(0x7fc990066320 msgr2=0x7fc990154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:58.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc98ffff700 1 --2- 172.21.15.121:0/1941158687 >> v2:172.21.15.190:3300/0 conn(0x7fc990066320 0x7fc990154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc98ffff700 1 -- 172.21.15.121:0/1941158687 >> v1:172.21.15.121:6789/0 conn(0x7fc990066700 legacy=0x7fc9900656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:58.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc98ffff700 1 -- 172.21.15.121:0/1941158687 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc990154c80 con 0x7fc9900acf10 2023-11-15T03:03:58.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc98f7fe700 1 --2- 172.21.15.121:0/1941158687 >> v2:172.21.15.190:3300/0 conn(0x7fc990066320 0x7fc990154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:03:58.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc98ffff700 1 --2- 172.21.15.121:0/1941158687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9900acf10 0x7fc990062190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fc988009e60 tx=0x7fc988009780).ready entity=mon.2 client_cookie=4eb7a9b83cf18e87 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:58.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc98e7fc700 1 -- 172.21.15.121:0/1941158687 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98800e070 con 0x7fc9900acf10 2023-11-15T03:03:58.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc98e7fc700 1 -- 172.21.15.121:0/1941158687 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98800bab0 con 0x7fc9900acf10 2023-11-15T03:03:58.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.196+0000 7fc98e7fc700 1 -- 172.21.15.121:0/1941158687 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98800be30 con 0x7fc9900acf10 2023-11-15T03:03:58.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.200+0000 7fc994a79700 1 -- 172.21.15.121:0/1941158687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9900acf10 msgr2=0x7fc990062190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:58.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.200+0000 7fc994a79700 1 --2- 172.21.15.121:0/1941158687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9900acf10 0x7fc990062190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fc988009e60 tx=0x7fc988009780).stop 2023-11-15T03:03:58.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.200+0000 7fc994a79700 1 -- 172.21.15.121:0/1941158687 shutdown_connections 2023-11-15T03:03:58.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.200+0000 7fc994a79700 1 --2- 172.21.15.121:0/1941158687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9900acf10 0x7fc990062190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.200+0000 7fc994a79700 1 --2- 172.21.15.121:0/1941158687 >> v2:172.21.15.190:3300/0 conn(0x7fc990066320 0x7fc990154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.200+0000 7fc994a79700 1 -- 172.21.15.121:0/1941158687 >> 172.21.15.121:0/1941158687 conn(0x7fc990067a60 msgr2=0x7fc990067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:58.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.200+0000 7fc994a79700 1 -- 172.21.15.121:0/1941158687 shutdown_connections 2023-11-15T03:03:58.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.200+0000 7fc994a79700 1 -- 172.21.15.121:0/1941158687 wait complete. 2023-11-15T03:03:58.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.200+0000 7fc994a79700 1 Processor -- start 2023-11-15T03:03:58.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.200+0000 7fc994a79700 1 -- start start 2023-11-15T03:03:58.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc994a79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc990066320 0x7fc99014b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:58.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc994a79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc990066700 0x7fc99014a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:58.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc994a79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9900acf10 0x7fc99014dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:58.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc994a79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc990167f00 con 0x7fc9900acf10 2023-11-15T03:03:58.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc994a79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9901672f0 con 0x7fc990066320 2023-11-15T03:03:58.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc994a79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9901675d0 con 0x7fc990066700 2023-11-15T03:03:58.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc98ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9900acf10 0x7fc99014dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:58.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc98ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9900acf10 0x7fc99014dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38456/0 (socket says 172.21.15.121:38456) 2023-11-15T03:03:58.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc98ffff700 1 -- 172.21.15.121:0/973501532 learned_addr learned my addr 172.21.15.121:0/973501532 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:58.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc98effd700 1 --2- 172.21.15.121:0/973501532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc990066700 0x7fc99014a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:58.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc98f7fe700 1 --2- 172.21.15.121:0/973501532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc990066320 0x7fc99014b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:58.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc98ffff700 1 -- 172.21.15.121:0/973501532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc990066700 msgr2=0x7fc99014a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:58.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc98ffff700 1 --2- 172.21.15.121:0/973501532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc990066700 0x7fc99014a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc98ffff700 1 -- 172.21.15.121:0/973501532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc990066320 msgr2=0x7fc99014b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:58.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc98ffff700 1 --2- 172.21.15.121:0/973501532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc990066320 0x7fc99014b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc98ffff700 1 -- 172.21.15.121:0/973501532 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9800092a0 con 0x7fc9900acf10 2023-11-15T03:03:58.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc98f7fe700 1 --2- 172.21.15.121:0/973501532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc990066320 0x7fc99014b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:58.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc98ffff700 1 --2- 172.21.15.121:0/973501532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9900acf10 0x7fc99014dd90 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7fc98800b610 tx=0x7fc988009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:58.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc9777fe700 1 -- 172.21.15.121:0/973501532 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98801d630 con 0x7fc9900acf10 2023-11-15T03:03:58.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc9777fe700 1 -- 172.21.15.121:0/973501532 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98801d790 con 0x7fc9900acf10 2023-11-15T03:03:58.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc9777fe700 1 -- 172.21.15.121:0/973501532 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98801db50 con 0x7fc9900acf10 2023-11-15T03:03:58.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc994a79700 1 -- 172.21.15.121:0/973501532 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc988009a20 con 0x7fc9900acf10 2023-11-15T03:03:58.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc994a79700 1 -- 172.21.15.121:0/973501532 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc990167b90 con 0x7fc9900acf10 2023-11-15T03:03:58.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.204+0000 7fc994a79700 1 -- 172.21.15.121:0/973501532 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9901680b0 con 0x7fc9900acf10 2023-11-15T03:03:58.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.208+0000 7fc9777fe700 1 -- 172.21.15.121:0/973501532 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc988003040 con 0x7fc9900acf10 2023-11-15T03:03:58.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.208+0000 7fc9777fe700 1 --2- 172.21.15.121:0/973501532 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc97c05a510 0x7fc97c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:58.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.208+0000 7fc9777fe700 1 -- 172.21.15.121:0/973501532 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6524+0+0 (secure 0 0 0) 0x7fc988076860 con 0x7fc9900acf10 2023-11-15T03:03:58.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.208+0000 7fc98f7fe700 1 --2- 172.21.15.121:0/973501532 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc97c05a510 0x7fc97c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:58.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.212+0000 7fc98f7fe700 1 --2- 172.21.15.121:0/973501532 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc97c05a510 0x7fc97c05c9d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7fc980009270 tx=0x7fc980008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:58.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.212+0000 7fc9777fe700 1 -- 172.21.15.121:0/973501532 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc9901680b0 con 0x7fc9900acf10 2023-11-15T03:03:58.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.408+0000 7fc994a79700 1 -- 172.21.15.121:0/973501532 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9901680b0 con 0x7fc9900acf10 2023-11-15T03:03:58.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.408+0000 7fc9777fe700 1 -- 172.21.15.121:0/973501532 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4492 (secure 0 0 0) 0x7fc9901680b0 con 0x7fc9900acf10 2023-11-15T03:03:58.417 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:58.417 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":99,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7704,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3764682615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3764682615},{"type":"v1","addr":"172.21.15.121:6836","nonce":3764682615}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:03:56.215692+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:03:58.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.412+0000 7fc994a79700 1 -- 172.21.15.121:0/973501532 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc97c05a510 msgr2=0x7fc97c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:58.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.412+0000 7fc994a79700 1 --2- 172.21.15.121:0/973501532 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc97c05a510 0x7fc97c05c9d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7fc980009270 tx=0x7fc980008040).stop 2023-11-15T03:03:58.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.412+0000 7fc994a79700 1 -- 172.21.15.121:0/973501532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9900acf10 msgr2=0x7fc99014dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:58.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.412+0000 7fc994a79700 1 --2- 172.21.15.121:0/973501532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9900acf10 0x7fc99014dd90 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7fc98800b610 tx=0x7fc988009780).stop 2023-11-15T03:03:58.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.416+0000 7fc994a79700 1 -- 172.21.15.121:0/973501532 shutdown_connections 2023-11-15T03:03:58.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.416+0000 7fc994a79700 1 --2- 172.21.15.121:0/973501532 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc97c05a510 0x7fc97c05c9d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.416+0000 7fc994a79700 1 --2- 172.21.15.121:0/973501532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9900acf10 0x7fc99014dd90 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.416+0000 7fc994a79700 1 --2- 172.21.15.121:0/973501532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc990066700 0x7fc99014a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.416+0000 7fc994a79700 1 --2- 172.21.15.121:0/973501532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc990066320 0x7fc99014b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.416+0000 7fc994a79700 1 -- 172.21.15.121:0/973501532 >> 172.21.15.121:0/973501532 conn(0x7fc990067a60 msgr2=0x7fc990067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:58.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.416+0000 7fc994a79700 1 -- 172.21.15.121:0/973501532 shutdown_connections 2023-11-15T03:03:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.416+0000 7fc994a79700 1 -- 172.21.15.121:0/973501532 wait complete. 2023-11-15T03:03:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 99 2023-11-15T03:03:58.443 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:03:58.444 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:03:58.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.600+0000 7fc398df9700 1 Processor -- start 2023-11-15T03:03:58.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.600+0000 7fc398df9700 1 -- start start 2023-11-15T03:03:58.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.604+0000 7fc398df9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc394147910 0x7fc3940a8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:58.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.604+0000 7fc398df9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc3940c2ce0 0x7fc3940657f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:58.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.604+0000 7fc398df9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc394151810 con 0x7fc3940a8c40 2023-11-15T03:03:58.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.604+0000 7fc398df9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3940a9aa0 con 0x7fc3940c2ce0 2023-11-15T03:03:58.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.604+0000 7fc398df9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3940670b0 con 0x7fc394147910 2023-11-15T03:03:58.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.604+0000 7fc393fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc3940c2ce0 0x7fc3940657f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:58.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.604+0000 7fc393fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc3940c2ce0 0x7fc3940657f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38458/0 (socket says 172.21.15.121:38458) 2023-11-15T03:03:58.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.604+0000 7fc393fff700 1 -- 172.21.15.121:0/168498162 learned_addr learned my addr 172.21.15.121:0/168498162 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:58.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.604+0000 7fc3937fe700 1 --2- 172.21.15.121:0/168498162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc394147910 0x7fc3940a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:58.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.604+0000 7fc393fff700 1 -- 172.21.15.121:0/168498162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc394147910 msgr2=0x7fc3940a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:58.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.604+0000 7fc393fff700 1 --2- 172.21.15.121:0/168498162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc394147910 0x7fc3940a8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.604+0000 7fc393fff700 1 -- 172.21.15.121:0/168498162 >> v1:172.21.15.190:6789/0 conn(0x7fc3940a8c40 legacy=0x7fc3940c25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:58.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.608+0000 7fc393fff700 1 -- 172.21.15.121:0/168498162 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc394150f00 con 0x7fc3940c2ce0 2023-11-15T03:03:58.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.608+0000 7fc3937fe700 1 --2- 172.21.15.121:0/168498162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc394147910 0x7fc3940a8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:58.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.608+0000 7fc393fff700 1 --2- 172.21.15.121:0/168498162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3940c2ce0 0x7fc3940657f0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0x7fc38c009f20 tx=0x7fc38c009840).ready entity=mon.0 client_cookie=b9555bfbc0c15624 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:58.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.608+0000 7fc3927fc700 1 -- 172.21.15.121:0/168498162 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c00e070 con 0x7fc3940c2ce0 2023-11-15T03:03:58.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.608+0000 7fc3927fc700 1 -- 172.21.15.121:0/168498162 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c00ba80 con 0x7fc3940c2ce0 2023-11-15T03:03:58.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.608+0000 7fc3927fc700 1 -- 172.21.15.121:0/168498162 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c00be00 con 0x7fc3940c2ce0 2023-11-15T03:03:58.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.608+0000 7fc398df9700 1 -- 172.21.15.121:0/168498162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3940c2ce0 msgr2=0x7fc3940657f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:58.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.608+0000 7fc398df9700 1 --2- 172.21.15.121:0/168498162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3940c2ce0 0x7fc3940657f0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0x7fc38c009f20 tx=0x7fc38c009840).stop 2023-11-15T03:03:58.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.608+0000 7fc398df9700 1 -- 172.21.15.121:0/168498162 shutdown_connections 2023-11-15T03:03:58.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.608+0000 7fc398df9700 1 --2- 172.21.15.121:0/168498162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3940c2ce0 0x7fc3940657f0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.608+0000 7fc398df9700 1 --2- 172.21.15.121:0/168498162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc394147910 0x7fc3940a8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.608+0000 7fc398df9700 1 -- 172.21.15.121:0/168498162 >> 172.21.15.121:0/168498162 conn(0x7fc394062190 msgr2=0x7fc394062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:58.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.608+0000 7fc398df9700 1 -- 172.21.15.121:0/168498162 shutdown_connections 2023-11-15T03:03:58.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.608+0000 7fc398df9700 1 -- 172.21.15.121:0/168498162 wait complete. 2023-11-15T03:03:58.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.612+0000 7fc398df9700 1 Processor -- start 2023-11-15T03:03:58.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.612+0000 7fc398df9700 1 -- start start 2023-11-15T03:03:58.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc398df9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3940a8c40 0x7fc3941419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:58.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc398df9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3940c2ce0 0x7fc394159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:58.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc398df9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc394147910 0x7fc39415a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:58.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc398df9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39415a6a0 con 0x7fc3940c2ce0 2023-11-15T03:03:58.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc398df9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc394143570 con 0x7fc394147910 2023-11-15T03:03:58.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc398df9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3940c5670 con 0x7fc3940a8c40 2023-11-15T03:03:58.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc392ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3940c2ce0 0x7fc394159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:58.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc392ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3940c2ce0 0x7fc394159cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38470/0 (socket says 172.21.15.121:38470) 2023-11-15T03:03:58.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc392ffd700 1 -- 172.21.15.121:0/2500555132 learned_addr learned my addr 172.21.15.121:0/2500555132 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:58.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc393fff700 1 --2- 172.21.15.121:0/2500555132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc394147910 0x7fc39415a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:58.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc3937fe700 1 --2- 172.21.15.121:0/2500555132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3940a8c40 0x7fc3941419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:58.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc392ffd700 1 -- 172.21.15.121:0/2500555132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3940a8c40 msgr2=0x7fc3941419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:58.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc392ffd700 1 --2- 172.21.15.121:0/2500555132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3940a8c40 0x7fc3941419d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc392ffd700 1 -- 172.21.15.121:0/2500555132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc394147910 msgr2=0x7fc39415a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:58.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc392ffd700 1 --2- 172.21.15.121:0/2500555132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc394147910 0x7fc39415a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc392ffd700 1 -- 172.21.15.121:0/2500555132 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c009ae0 con 0x7fc3940c2ce0 2023-11-15T03:03:58.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc3937fe700 1 --2- 172.21.15.121:0/2500555132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3940a8c40 0x7fc3941419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:03:58.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc392ffd700 1 --2- 172.21.15.121:0/2500555132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3940c2ce0 0x7fc394159cc0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7fc38800c4c0 tx=0x7fc388007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:58.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc37b7fe700 1 -- 172.21.15.121:0/2500555132 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388007020 con 0x7fc3940c2ce0 2023-11-15T03:03:58.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc37b7fe700 1 -- 172.21.15.121:0/2500555132 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc388007180 con 0x7fc3940c2ce0 2023-11-15T03:03:58.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc37b7fe700 1 -- 172.21.15.121:0/2500555132 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388007520 con 0x7fc3940c2ce0 2023-11-15T03:03:58.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc398df9700 1 -- 172.21.15.121:0/2500555132 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc394142c40 con 0x7fc3940c2ce0 2023-11-15T03:03:58.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.616+0000 7fc398df9700 1 -- 172.21.15.121:0/2500555132 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc394143200 con 0x7fc3940c2ce0 2023-11-15T03:03:58.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.620+0000 7fc398df9700 1 -- 172.21.15.121:0/2500555132 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc394143700 con 0x7fc3940c2ce0 2023-11-15T03:03:58.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.620+0000 7fc37b7fe700 1 -- 172.21.15.121:0/2500555132 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc3880068b0 con 0x7fc3940c2ce0 2023-11-15T03:03:58.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.620+0000 7fc37b7fe700 1 --2- 172.21.15.121:0/2500555132 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc38005a510 0x7fc38005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:58.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.620+0000 7fc37b7fe700 1 -- 172.21.15.121:0/2500555132 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6524+0+0 (secure 0 0 0) 0x7fc388073dc0 con 0x7fc3940c2ce0 2023-11-15T03:03:58.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.620+0000 7fc3937fe700 1 --2- 172.21.15.121:0/2500555132 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc38005a510 0x7fc38005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:58.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.620+0000 7fc3937fe700 1 --2- 172.21.15.121:0/2500555132 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc38005a510 0x7fc38005c9d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fc384006d80 tx=0x7fc384008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:58.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.628+0000 7fc37b7fe700 1 -- 172.21.15.121:0/2500555132 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc388047410 con 0x7fc3940c2ce0 2023-11-15T03:03:58.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.820+0000 7fc398df9700 1 -- 172.21.15.121:0/2500555132 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc39406bcf0 con 0x7fc3940c2ce0 2023-11-15T03:03:58.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.820+0000 7fc37b7fe700 1 -- 172.21.15.121:0/2500555132 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4492 (secure 0 0 0) 0x7fc38804aa30 con 0x7fc3940c2ce0 2023-11-15T03:03:58.829 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:03:58.829 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":99,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7704,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3764682615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3764682615},{"type":"v1","addr":"172.21.15.121:6836","nonce":3764682615}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:03:56.215692+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:03:58.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.824+0000 7fc398df9700 1 -- 172.21.15.121:0/2500555132 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc38005a510 msgr2=0x7fc38005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:58.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.824+0000 7fc398df9700 1 --2- 172.21.15.121:0/2500555132 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc38005a510 0x7fc38005c9d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fc384006d80 tx=0x7fc384008040).stop 2023-11-15T03:03:58.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.824+0000 7fc398df9700 1 -- 172.21.15.121:0/2500555132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3940c2ce0 msgr2=0x7fc394159cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:58.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.824+0000 7fc398df9700 1 --2- 172.21.15.121:0/2500555132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3940c2ce0 0x7fc394159cc0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7fc38800c4c0 tx=0x7fc388007dd0).stop 2023-11-15T03:03:58.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.824+0000 7fc398df9700 1 -- 172.21.15.121:0/2500555132 shutdown_connections 2023-11-15T03:03:58.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.824+0000 7fc398df9700 1 --2- 172.21.15.121:0/2500555132 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc38005a510 0x7fc38005c9d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.824+0000 7fc398df9700 1 --2- 172.21.15.121:0/2500555132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc394147910 0x7fc39415a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.824+0000 7fc398df9700 1 --2- 172.21.15.121:0/2500555132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3940c2ce0 0x7fc394159cc0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.824+0000 7fc398df9700 1 --2- 172.21.15.121:0/2500555132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3940a8c40 0x7fc3941419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:58.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.824+0000 7fc398df9700 1 -- 172.21.15.121:0/2500555132 >> 172.21.15.121:0/2500555132 conn(0x7fc394062190 msgr2=0x7fc3940c5990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:58.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.824+0000 7fc398df9700 1 -- 172.21.15.121:0/2500555132 shutdown_connections 2023-11-15T03:03:58.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:58.824+0000 7fc398df9700 1 -- 172.21.15.121:0/2500555132 wait complete. 2023-11-15T03:03:58.834 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 99 2023-11-15T03:03:58.851 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 600.00000000' led to DAMAGED state 2023-11-15T03:03:58.852 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 1.00000000.inode' 2023-11-15T03:03:58.852 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:03:58.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:03:58.853 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:03:58.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:03:58.875 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:03:58.876 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:03:59.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.072+0000 7f7af13b9700 1 Processor -- start 2023-11-15T03:03:59.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.072+0000 7f7af13b9700 1 -- start start 2023-11-15T03:03:59.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.072+0000 7f7af13b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aec0abaa0 0x7f7aec0abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:59.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.072+0000 7f7af13b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7aec0bb510 0x7f7aec063570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:59.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.072+0000 7f7af13b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aec0c2c00 con 0x7f7aec0a8880 2023-11-15T03:03:59.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.072+0000 7f7af13b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aec063a60 con 0x7f7aec0bb510 2023-11-15T03:03:59.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.072+0000 7f7af13b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0657b0 con 0x7f7aec0abaa0 2023-11-15T03:03:59.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.072+0000 7f7ae37fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7aec0bb510 0x7f7aec063570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:59.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.072+0000 7f7ae37fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7aec0bb510 0x7f7aec063570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38478/0 (socket says 172.21.15.121:38478) 2023-11-15T03:03:59.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.072+0000 7f7ae37fe700 1 -- 172.21.15.121:0/1603252380 learned_addr learned my addr 172.21.15.121:0/1603252380 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:59.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.072+0000 7f7aebfff700 1 --2- 172.21.15.121:0/1603252380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aec0abaa0 0x7f7aec0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:59.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.076+0000 7f7ae37fe700 1 -- 172.21.15.121:0/1603252380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aec0abaa0 msgr2=0x7f7aec0abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:03:59.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.076+0000 7f7ae37fe700 1 --2- 172.21.15.121:0/1603252380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aec0abaa0 0x7f7aec0abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:59.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.076+0000 7f7ae37fe700 1 -- 172.21.15.121:0/1603252380 >> v1:172.21.15.190:6789/0 conn(0x7f7aec0a8880 legacy=0x7f7aec151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:59.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.076+0000 7f7ae37fe700 1 -- 172.21.15.121:0/1603252380 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec065a90 con 0x7f7aec0bb510 2023-11-15T03:03:59.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.076+0000 7f7ae37fe700 1 --2- 172.21.15.121:0/1603252380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aec0bb510 0x7f7aec063570 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7f7ad4009810 tx=0x7f7ad4009060).ready entity=mon.0 client_cookie=b609d8bd28c46732 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:59.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.076+0000 7f7aeb7fe700 1 -- 172.21.15.121:0/1603252380 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4005880 con 0x7f7aec0bb510 2023-11-15T03:03:59.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.076+0000 7f7aeb7fe700 1 -- 172.21.15.121:0/1603252380 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad40059e0 con 0x7f7aec0bb510 2023-11-15T03:03:59.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.076+0000 7f7aeb7fe700 1 -- 172.21.15.121:0/1603252380 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4005d60 con 0x7f7aec0bb510 2023-11-15T03:03:59.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.076+0000 7f7af13b9700 1 -- 172.21.15.121:0/1603252380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aec0bb510 msgr2=0x7f7aec063570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:59.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.076+0000 7f7af13b9700 1 --2- 172.21.15.121:0/1603252380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aec0bb510 0x7f7aec063570 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7f7ad4009810 tx=0x7f7ad4009060).stop 2023-11-15T03:03:59.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.076+0000 7f7af13b9700 1 -- 172.21.15.121:0/1603252380 shutdown_connections 2023-11-15T03:03:59.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.076+0000 7f7af13b9700 1 --2- 172.21.15.121:0/1603252380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aec0bb510 0x7f7aec063570 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:59.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.076+0000 7f7af13b9700 1 --2- 172.21.15.121:0/1603252380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aec0abaa0 0x7f7aec0abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:59.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.076+0000 7f7af13b9700 1 -- 172.21.15.121:0/1603252380 >> 172.21.15.121:0/1603252380 conn(0x7f7aec067a60 msgr2=0x7f7aec067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:03:59.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.076+0000 7f7af13b9700 1 -- 172.21.15.121:0/1603252380 shutdown_connections 2023-11-15T03:03:59.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.076+0000 7f7af13b9700 1 -- 172.21.15.121:0/1603252380 wait complete. 2023-11-15T03:03:59.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7af13b9700 1 Processor -- start 2023-11-15T03:03:59.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7af13b9700 1 -- start start 2023-11-15T03:03:59.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7af13b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aec0a8880 0x7f7aec0c4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:59.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7af13b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aec0abaa0 0x7f7aec0a9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:59.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7af13b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aec0bb510 0x7f7aec167a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:59.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7af13b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0bd780 con 0x7f7aec0a8880 2023-11-15T03:03:59.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7af13b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0aa7b0 con 0x7f7aec0bb510 2023-11-15T03:03:59.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7af13b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0aaa90 con 0x7f7aec0abaa0 2023-11-15T03:03:59.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7aebfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aec0a8880 0x7f7aec0c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:59.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7aebfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aec0a8880 0x7f7aec0c4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38490/0 (socket says 172.21.15.121:38490) 2023-11-15T03:03:59.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7aebfff700 1 -- 172.21.15.121:0/3374511437 learned_addr learned my addr 172.21.15.121:0/3374511437 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:03:59.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7ae37fe700 1 --2- 172.21.15.121:0/3374511437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aec0abaa0 0x7f7aec0a9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:59.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7af0bb8700 1 --2- 172.21.15.121:0/3374511437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aec0bb510 0x7f7aec167a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:59.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7aebfff700 1 -- 172.21.15.121:0/3374511437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aec0abaa0 msgr2=0x7f7aec0a9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:59.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7aebfff700 1 --2- 172.21.15.121:0/3374511437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aec0abaa0 0x7f7aec0a9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:59.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7aebfff700 1 -- 172.21.15.121:0/3374511437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aec0bb510 msgr2=0x7f7aec167a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:03:59.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7aebfff700 1 --2- 172.21.15.121:0/3374511437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aec0bb510 0x7f7aec167a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:03:59.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7aebfff700 1 -- 172.21.15.121:0/3374511437 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad4009300 con 0x7f7aec0a8880 2023-11-15T03:03:59.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7ae37fe700 1 --2- 172.21.15.121:0/3374511437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aec0abaa0 0x7f7aec0a9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:59.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7af0bb8700 1 --2- 172.21.15.121:0/3374511437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aec0bb510 0x7f7aec167a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:03:59.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.080+0000 7f7aebfff700 1 --2- 172.21.15.121:0/3374511437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aec0a8880 0x7f7aec0c4260 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7f7adc00d890 tx=0x7f7adc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:59.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.084+0000 7f7ae8ff9700 1 -- 172.21.15.121:0/3374511437 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc0125f0 con 0x7f7aec0a8880 2023-11-15T03:03:59.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.084+0000 7f7ae8ff9700 1 -- 172.21.15.121:0/3374511437 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7adc012750 con 0x7f7aec0a8880 2023-11-15T03:03:59.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.084+0000 7f7af13b9700 1 -- 172.21.15.121:0/3374511437 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aec0aad70 con 0x7f7aec0a8880 2023-11-15T03:03:59.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.084+0000 7f7ae8ff9700 1 -- 172.21.15.121:0/3374511437 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc012ad0 con 0x7f7aec0a8880 2023-11-15T03:03:59.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.084+0000 7f7af13b9700 1 -- 172.21.15.121:0/3374511437 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aec0aa480 con 0x7f7aec0a8880 2023-11-15T03:03:59.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.084+0000 7f7af13b9700 1 -- 172.21.15.121:0/3374511437 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aec0bdb70 con 0x7f7aec0a8880 2023-11-15T03:03:59.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.084+0000 7f7ae8ff9700 1 -- 172.21.15.121:0/3374511437 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7adc013480 con 0x7f7aec0a8880 2023-11-15T03:03:59.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.088+0000 7f7ae8ff9700 1 --2- 172.21.15.121:0/3374511437 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7acc05a510 0x7f7acc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:03:59.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.088+0000 7f7ae8ff9700 1 -- 172.21.15.121:0/3374511437 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6524+0+0 (secure 0 0 0) 0x7f7adc0746b0 con 0x7f7aec0a8880 2023-11-15T03:03:59.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.088+0000 7f7ae37fe700 1 --2- 172.21.15.121:0/3374511437 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7acc05a510 0x7f7acc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:03:59.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.088+0000 7f7ae37fe700 1 --2- 172.21.15.121:0/3374511437 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7acc05a510 0x7f7acc05c9d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f7ad4009810 tx=0x7f7ad4016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:03:59.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.092+0000 7f7ae8ff9700 1 -- 172.21.15.121:0/3374511437 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7adc047ca0 con 0x7f7aec0a8880 2023-11-15T03:03:59.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:03:59.292+0000 7f7af13b9700 1 -- 172.21.15.121:0/3374511437 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7aec06bcf0 con 0x7f7aec0a8880 2023-11-15T03:04:00.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.248+0000 7f7ae8ff9700 1 -- 172.21.15.121:0/3374511437 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v100) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7adc04b2c0 con 0x7f7aec0a8880 2023-11-15T03:04:00.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.252+0000 7f7af13b9700 1 -- 172.21.15.121:0/3374511437 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7acc05a510 msgr2=0x7f7acc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:00.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.252+0000 7f7af13b9700 1 --2- 172.21.15.121:0/3374511437 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7acc05a510 0x7f7acc05c9d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f7ad4009810 tx=0x7f7ad4016000).stop 2023-11-15T03:04:00.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.252+0000 7f7af13b9700 1 -- 172.21.15.121:0/3374511437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aec0a8880 msgr2=0x7f7aec0c4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:00.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.252+0000 7f7af13b9700 1 --2- 172.21.15.121:0/3374511437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aec0a8880 0x7f7aec0c4260 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7f7adc00d890 tx=0x7f7adc008040).stop 2023-11-15T03:04:00.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.252+0000 7f7af13b9700 1 -- 172.21.15.121:0/3374511437 shutdown_connections 2023-11-15T03:04:00.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.252+0000 7f7af13b9700 1 --2- 172.21.15.121:0/3374511437 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7acc05a510 0x7f7acc05c9d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:00.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.252+0000 7f7af13b9700 1 --2- 172.21.15.121:0/3374511437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aec0bb510 0x7f7aec167a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:00.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.252+0000 7f7af13b9700 1 --2- 172.21.15.121:0/3374511437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aec0abaa0 0x7f7aec0a9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:00.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.252+0000 7f7af13b9700 1 --2- 172.21.15.121:0/3374511437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aec0a8880 0x7f7aec0c4260 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:00.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.252+0000 7f7af13b9700 1 -- 172.21.15.121:0/3374511437 >> 172.21.15.121:0/3374511437 conn(0x7f7aec067a60 msgr2=0x7f7aec150070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:00.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.252+0000 7f7af13b9700 1 -- 172.21.15.121:0/3374511437 shutdown_connections 2023-11-15T03:04:00.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.252+0000 7f7af13b9700 1 -- 172.21.15.121:0/3374511437 wait complete. 2023-11-15T03:04:00.263 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:04:00.281 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:04:00.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.440+0000 7f970db1b700 1 Processor -- start 2023-11-15T03:04:00.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.440+0000 7f970db1b700 1 -- start start 2023-11-15T03:04:00.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.444+0000 7f970db1b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f97080a8de0 0x7f9708066420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:00.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.444+0000 7f970db1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97080628a0 0x7f9708151410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:00.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.444+0000 7f970db1b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9708065f90 con 0x7f9708150f90 2023-11-15T03:04:00.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.444+0000 7f970db1b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9708151900 con 0x7f97080a8de0 2023-11-15T03:04:00.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.444+0000 7f970db1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9708147ea0 con 0x7f97080628a0 2023-11-15T03:04:00.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.444+0000 7f9706ffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9708150f90 0x7f9708062190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:38236/0 (socket says 172.21.15.121:38236) 2023-11-15T03:04:00.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.444+0000 7f9706ffd700 1 -- 172.21.15.121:0/678051673 learned_addr learned my addr 172.21.15.121:0/678051673 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:00.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.444+0000 7f97077fe700 1 --2- 172.21.15.121:0/678051673 >> v2:172.21.15.190:3300/0 conn(0x7f97080a8de0 0x7f9708066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:00.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.444+0000 7f9707fff700 1 --2- 172.21.15.121:0/678051673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97080628a0 0x7f9708151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:00.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.444+0000 7f97067fc700 1 -- 172.21.15.121:0/678051673 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4044115872 0 0) 0x7f9708065f90 con 0x7f9708150f90 2023-11-15T03:04:00.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.444+0000 7f97067fc700 1 -- 172.21.15.121:0/678051673 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96f00033f0 con 0x7f9708150f90 2023-11-15T03:04:00.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.444+0000 7f97077fe700 1 -- 172.21.15.121:0/678051673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97080628a0 msgr2=0x7f9708151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:00.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.444+0000 7f97077fe700 1 --2- 172.21.15.121:0/678051673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97080628a0 0x7f9708151410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:00.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.444+0000 7f97077fe700 1 -- 172.21.15.121:0/678051673 >> v1:172.21.15.121:6789/0 conn(0x7f9708150f90 legacy=0x7f9708062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:00.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.444+0000 7f97077fe700 1 -- 172.21.15.121:0/678051673 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9708151e30 con 0x7f97080a8de0 2023-11-15T03:04:00.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.444+0000 7f97077fe700 1 --2- 172.21.15.121:0/678051673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97080a8de0 0x7f9708066420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f96f8009760 tx=0x7f96f8009060).ready entity=mon.1 client_cookie=6e00aa56a35d3995 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:00.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.444+0000 7f97067fc700 1 -- 172.21.15.121:0/678051673 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f800d4a0 con 0x7f97080a8de0 2023-11-15T03:04:00.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.444+0000 7f97067fc700 1 -- 172.21.15.121:0/678051673 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f800d600 con 0x7f97080a8de0 2023-11-15T03:04:00.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.448+0000 7f97067fc700 1 -- 172.21.15.121:0/678051673 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f800d980 con 0x7f97080a8de0 2023-11-15T03:04:00.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.448+0000 7f970db1b700 1 -- 172.21.15.121:0/678051673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97080a8de0 msgr2=0x7f9708066420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:00.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.448+0000 7f970db1b700 1 --2- 172.21.15.121:0/678051673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97080a8de0 0x7f9708066420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f96f8009760 tx=0x7f96f8009060).stop 2023-11-15T03:04:00.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.448+0000 7f970db1b700 1 -- 172.21.15.121:0/678051673 shutdown_connections 2023-11-15T03:04:00.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.448+0000 7f970db1b700 1 --2- 172.21.15.121:0/678051673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97080628a0 0x7f9708151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:00.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.448+0000 7f970db1b700 1 --2- 172.21.15.121:0/678051673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97080a8de0 0x7f9708066420 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:00.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.448+0000 7f970db1b700 1 -- 172.21.15.121:0/678051673 >> 172.21.15.121:0/678051673 conn(0x7f9708067a60 msgr2=0x7f9708067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:00.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.448+0000 7f970db1b700 1 -- 172.21.15.121:0/678051673 shutdown_connections 2023-11-15T03:04:00.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.448+0000 7f970db1b700 1 -- 172.21.15.121:0/678051673 wait complete. 2023-11-15T03:04:00.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.448+0000 7f970db1b700 1 Processor -- start 2023-11-15T03:04:00.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.448+0000 7f970db1b700 1 -- start start 2023-11-15T03:04:00.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f970db1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97080628a0 0x7f970814de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:00.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f970db1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97080a8de0 0x7f97080aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:00.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f970db1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9708150f90 0x7f970814c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:00.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f970db1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9708149130 con 0x7f97080628a0 2023-11-15T03:04:00.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f970db1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97080aa650 con 0x7f97080a8de0 2023-11-15T03:04:00.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f970db1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97080a9760 con 0x7f9708150f90 2023-11-15T03:04:00.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f97077fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97080628a0 0x7f970814de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:00.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f97077fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97080628a0 0x7f970814de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38500/0 (socket says 172.21.15.121:38500) 2023-11-15T03:04:00.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f97077fe700 1 -- 172.21.15.121:0/2333114273 learned_addr learned my addr 172.21.15.121:0/2333114273 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:00.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f9706ffd700 1 --2- 172.21.15.121:0/2333114273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97080a8de0 0x7f97080aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:00.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f9707fff700 1 --2- 172.21.15.121:0/2333114273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9708150f90 0x7f970814c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:00.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f97077fe700 1 -- 172.21.15.121:0/2333114273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9708150f90 msgr2=0x7f970814c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:00.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f97077fe700 1 --2- 172.21.15.121:0/2333114273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9708150f90 0x7f970814c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:00.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f97077fe700 1 -- 172.21.15.121:0/2333114273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97080a8de0 msgr2=0x7f97080aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:00.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f97077fe700 1 --2- 172.21.15.121:0/2333114273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97080a8de0 0x7f97080aaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:00.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f97077fe700 1 -- 172.21.15.121:0/2333114273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f80092a0 con 0x7f97080628a0 2023-11-15T03:04:00.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f9707fff700 1 --2- 172.21.15.121:0/2333114273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9708150f90 0x7f970814c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:00.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f9706ffd700 1 --2- 172.21.15.121:0/2333114273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97080a8de0 0x7f97080aaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:00.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f97077fe700 1 --2- 172.21.15.121:0/2333114273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97080628a0 0x7f970814de90 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f96f8009d50 tx=0x7f96f8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:00.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f96ef7fe700 1 -- 172.21.15.121:0/2333114273 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8000d10 con 0x7f97080628a0 2023-11-15T03:04:00.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f96ef7fe700 1 -- 172.21.15.121:0/2333114273 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f800d770 con 0x7f97080628a0 2023-11-15T03:04:00.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f96ef7fe700 1 -- 172.21.15.121:0/2333114273 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f801d640 con 0x7f97080628a0 2023-11-15T03:04:00.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f970db1b700 1 -- 172.21.15.121:0/2333114273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97080a9a40 con 0x7f97080628a0 2023-11-15T03:04:00.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.452+0000 7f970db1b700 1 -- 172.21.15.121:0/2333114273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97080aa000 con 0x7f97080628a0 2023-11-15T03:04:00.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.456+0000 7f970db1b700 1 -- 172.21.15.121:0/2333114273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9708047060 con 0x7f97080628a0 2023-11-15T03:04:00.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.456+0000 7f96ef7fe700 1 -- 172.21.15.121:0/2333114273 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f96f801da70 con 0x7f97080628a0 2023-11-15T03:04:00.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.456+0000 7f96ef7fe700 1 --2- 172.21.15.121:0/2333114273 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f96f405a510 0x7f96f405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:00.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.456+0000 7f96ef7fe700 1 -- 172.21.15.121:0/2333114273 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6524+0+0 (secure 0 0 0) 0x7f96f8076240 con 0x7f97080628a0 2023-11-15T03:04:00.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.460+0000 7f9706ffd700 1 --2- 172.21.15.121:0/2333114273 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f96f405a510 0x7f96f405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:00.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.460+0000 7f9706ffd700 1 --2- 172.21.15.121:0/2333114273 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f96f405a510 0x7f96f405c9d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f96fc007fd0 tx=0x7f96fc009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:00.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.464+0000 7f96ef7fe700 1 -- 172.21.15.121:0/2333114273 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f96f80497d0 con 0x7f97080628a0 2023-11-15T03:04:00.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:00.684+0000 7f970db1b700 1 -- 172.21.15.121:0/2333114273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f97080c2740 con 0x7f97080628a0 2023-11-15T03:04:01.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.256+0000 7f96ef7fe700 1 -- 172.21.15.121:0/2333114273 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v101) v1 ==== 101+0+0 (secure 0 0 0) 0x7f96f804cdf0 con 0x7f97080628a0 2023-11-15T03:04:01.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.260+0000 7f970db1b700 1 -- 172.21.15.121:0/2333114273 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f96f405a510 msgr2=0x7f96f405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.260+0000 7f970db1b700 1 --2- 172.21.15.121:0/2333114273 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f96f405a510 0x7f96f405c9d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f96fc007fd0 tx=0x7f96fc009040).stop 2023-11-15T03:04:01.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.260+0000 7f970db1b700 1 -- 172.21.15.121:0/2333114273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97080628a0 msgr2=0x7f970814de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.260+0000 7f970db1b700 1 --2- 172.21.15.121:0/2333114273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97080628a0 0x7f970814de90 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f96f8009d50 tx=0x7f96f8016000).stop 2023-11-15T03:04:01.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.260+0000 7f970db1b700 1 -- 172.21.15.121:0/2333114273 shutdown_connections 2023-11-15T03:04:01.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.260+0000 7f970db1b700 1 --2- 172.21.15.121:0/2333114273 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f96f405a510 0x7f96f405c9d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.260+0000 7f970db1b700 1 --2- 172.21.15.121:0/2333114273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9708150f90 0x7f970814c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.260+0000 7f970db1b700 1 --2- 172.21.15.121:0/2333114273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97080a8de0 0x7f97080aaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.260+0000 7f970db1b700 1 --2- 172.21.15.121:0/2333114273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97080628a0 0x7f970814de90 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.260+0000 7f970db1b700 1 -- 172.21.15.121:0/2333114273 >> 172.21.15.121:0/2333114273 conn(0x7f9708067a60 msgr2=0x7f97080c4500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:01.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.260+0000 7f970db1b700 1 -- 172.21.15.121:0/2333114273 shutdown_connections 2023-11-15T03:04:01.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.260+0000 7f970db1b700 1 -- 172.21.15.121:0/2333114273 wait complete. 2023-11-15T03:04:01.268 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:04:01.285 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:04:01.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.332+0000 7eff9d930d00 1 Processor -- start 2023-11-15T03:04:01.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.332+0000 7eff9d930d00 1 -- start start 2023-11-15T03:04:01.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff9d930d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56508cee16c0 0x56508cfc6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff9d930d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56508cf2bb00 0x56508cfc3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff9d930d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56508cfc4160 0x56508cf12a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff9d930d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56508ce5a220 con 0x56508cee16c0 2023-11-15T03:04:01.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff9d930d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56508cf13d10 con 0x56508cf2bb00 2023-11-15T03:04:01.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff9d930d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56508cfbdb70 con 0x56508cfc4160 2023-11-15T03:04:01.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff91e45700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56508cee16c0 0x56508cfc6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff91e45700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56508cee16c0 0x56508cfc6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38518/0 (socket says 172.21.15.121:38518) 2023-11-15T03:04:01.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff91e45700 1 -- 172.21.15.121:0/546841446 learned_addr learned my addr 172.21.15.121:0/546841446 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:01.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff89644700 1 --2- 172.21.15.121:0/546841446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56508cf2bb00 0x56508cfc3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff92646700 1 --2- 172.21.15.121:0/546841446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56508cfc4160 0x56508cf12a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff91e45700 1 -- 172.21.15.121:0/546841446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56508cfc4160 msgr2=0x56508cf12a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:01.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff91e45700 1 --2- 172.21.15.121:0/546841446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56508cfc4160 0x56508cf12a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff91e45700 1 -- 172.21.15.121:0/546841446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56508cf2bb00 msgr2=0x56508cfc3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:01.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff91e45700 1 --2- 172.21.15.121:0/546841446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56508cf2bb00 0x56508cfc3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff91e45700 1 -- 172.21.15.121:0/546841446 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7c005cf0 con 0x56508cee16c0 2023-11-15T03:04:01.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff89644700 1 --2- 172.21.15.121:0/546841446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56508cf2bb00 0x56508cfc3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:01.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff91e45700 1 --2- 172.21.15.121:0/546841446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56508cee16c0 0x56508cfc6b40 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7eff84009950 tx=0x7eff8400c040).ready entity=mon.0 client_cookie=11f5e6c2862a8fcc server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:01.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff90e43700 1 -- 172.21.15.121:0/546841446 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8400f040 con 0x56508cee16c0 2023-11-15T03:04:01.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff90e43700 1 -- 172.21.15.121:0/546841446 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff84008a60 con 0x56508cee16c0 2023-11-15T03:04:01.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff90e43700 1 -- 172.21.15.121:0/546841446 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff84008de0 con 0x56508cee16c0 2023-11-15T03:04:01.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff9d930d00 1 -- 172.21.15.121:0/546841446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56508cee16c0 msgr2=0x56508cfc6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff9d930d00 1 --2- 172.21.15.121:0/546841446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56508cee16c0 0x56508cfc6b40 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7eff84009950 tx=0x7eff8400c040).stop 2023-11-15T03:04:01.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff9d930d00 1 -- 172.21.15.121:0/546841446 shutdown_connections 2023-11-15T03:04:01.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff9d930d00 1 --2- 172.21.15.121:0/546841446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56508cfc4160 0x56508cf12a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff9d930d00 1 --2- 172.21.15.121:0/546841446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56508cf2bb00 0x56508cfc3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff9d930d00 1 --2- 172.21.15.121:0/546841446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56508cee16c0 0x56508cfc6b40 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.336+0000 7eff9d930d00 1 -- 172.21.15.121:0/546841446 >> 172.21.15.121:0/546841446 conn(0x56508cf31080 msgr2=0x56508cf25fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:01.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff9d930d00 1 -- 172.21.15.121:0/546841446 shutdown_connections 2023-11-15T03:04:01.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff9d930d00 1 -- 172.21.15.121:0/546841446 wait complete. 2023-11-15T03:04:01.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff9d930d00 1 Processor -- start 2023-11-15T03:04:01.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff9d930d00 1 -- start start 2023-11-15T03:04:01.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff9d930d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56508cee16c0 0x56508cf12be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff9d930d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56508cf2bb00 0x56508cfc5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff9d930d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56508cf16c50 0x56508cfc5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff9d930d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56508cfbcd20 con 0x56508cf2bb00 2023-11-15T03:04:01.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff9d930d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56508cfbe630 con 0x56508cf16c50 2023-11-15T03:04:01.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff9d930d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56508cf281a0 con 0x56508cee16c0 2023-11-15T03:04:01.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff89644700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56508cf2bb00 0x56508cfc5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff89644700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56508cf2bb00 0x56508cfc5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38524/0 (socket says 172.21.15.121:38524) 2023-11-15T03:04:01.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 learned_addr learned my addr 172.21.15.121:0/2564052450 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:01.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff91e45700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56508cee16c0 0x56508cf12be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff92646700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56508cf16c50 0x56508cfc5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56508cee16c0 msgr2=0x56508cf12be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff89644700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56508cee16c0 0x56508cf12be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56508cf16c50 msgr2=0x56508cfc5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff89644700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56508cf16c50 0x56508cfc5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff84009580 con 0x56508cf2bb00 2023-11-15T03:04:01.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.340+0000 7eff92646700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56508cf16c50 0x56508cfc5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:01.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.344+0000 7eff89644700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56508cf2bb00 0x56508cfc5520 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7eff7c010000 tx=0x7eff7c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:01.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.344+0000 7eff8affd700 1 -- 172.21.15.121:0/2564052450 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c01e440 con 0x56508cf2bb00 2023-11-15T03:04:01.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.344+0000 7eff8affd700 1 -- 172.21.15.121:0/2564052450 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7c01e5a0 con 0x56508cf2bb00 2023-11-15T03:04:01.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.344+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56508ce83770 con 0x56508cf2bb00 2023-11-15T03:04:01.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.344+0000 7eff8affd700 1 -- 172.21.15.121:0/2564052450 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c01e940 con 0x56508cf2bb00 2023-11-15T03:04:01.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.344+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56508ce52a70 con 0x56508cf2bb00 2023-11-15T03:04:01.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.348+0000 7eff8affd700 1 -- 172.21.15.121:0/2564052450 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7eff7c013440 con 0x56508cf2bb00 2023-11-15T03:04:01.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.348+0000 7eff8affd700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7eff74058ff0 0x7eff7405b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.348+0000 7eff8affd700 1 -- 172.21.15.121:0/2564052450 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6524+0+0 (secure 0 0 0) 0x7eff7c074970 con 0x56508cf2bb00 2023-11-15T03:04:01.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.348+0000 7eff91e45700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7eff74058ff0 0x7eff7405b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.360 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:04:01.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.348+0000 7eff9d930d00 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56508cfb3e50 0x56508cf28910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.348+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cf28e00 con 0x56508cfb3e50 2023-11-15T03:04:01.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.348+0000 7eff92646700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56508cfb3e50 0x56508cf28910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.348+0000 7eff91e45700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7eff74058ff0 0x7eff7405b4b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7eff840097f0 tx=0x7eff84018000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:01.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.352+0000 7eff92646700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56508cfb3e50 0x56508cf28910 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:01.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.352+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v97'29 uv22 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfb3e50 2023-11-15T03:04:01.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.352+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cf291b0 con 0x56508cfb3e50 2023-11-15T03:04:01.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.356+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v97'30 uv30 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfb3e50 2023-11-15T03:04:01.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.356+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cf29590 con 0x56508cfb3e50 2023-11-15T03:04:01.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.356+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v97'31 uv31 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfb3e50 2023-11-15T03:04:01.364 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:04:01.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.356+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cf29940 con 0x56508cfb3e50 2023-11-15T03:04:01.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.360+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v97'32 uv32 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfb3e50 2023-11-15T03:04:01.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.360+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfd9710 con 0x56508cfb3e50 2023-11-15T03:04:01.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.360+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v97'33 uv33 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfb3e50 2023-11-15T03:04:01.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.360+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfd8700 con 0x56508cfb3e50 2023-11-15T03:04:01.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.368+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v97'34 uv34 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfb3e50 2023-11-15T03:04:01.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.368+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfd8b30 con 0x56508cfb3e50 2023-11-15T03:04:01.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.368+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v97'35 uv28 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfb3e50 2023-11-15T03:04:01.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.368+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cfd8f30 con 0x56508cfb3e50 2023-11-15T03:04:01.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.372+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v97'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfb3e50 2023-11-15T03:04:01.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.372+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfd8f30 con 0x56508cfb3e50 2023-11-15T03:04:01.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.376+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v97'37 uv37 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfb3e50 2023-11-15T03:04:01.383 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:04:01.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.376+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfd8f30 con 0x56508cfb3e50 2023-11-15T03:04:01.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.380+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v97'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfb3e50 2023-11-15T03:04:01.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.380+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfd8f30 con 0x56508cfb3e50 2023-11-15T03:04:01.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.380+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v97'39 uv39 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfb3e50 2023-11-15T03:04:01.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.380+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfd8f30 con 0x56508cfb3e50 2023-11-15T03:04:01.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.384+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v97'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfb3e50 2023-11-15T03:04:01.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.384+0000 7eff9d930d00 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56508cfd9330 0x56508cfdbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.384+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfd8f30 con 0x56508cfd9330 2023-11-15T03:04:01.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.384+0000 7eff91e45700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56508cfd9330 0x56508cfdbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.384+0000 7eff91e45700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56508cfd9330 0x56508cfdbfe0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:01.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.388+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v97'45 uv31 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.388+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cfdc4d0 con 0x56508cfd9330 2023-11-15T03:04:01.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.392+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v97'46 uv46 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.392+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfdc880 con 0x56508cfd9330 2023-11-15T03:04:01.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.392+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v97'47 uv47 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.402 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:04:01.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.392+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfddcc0 con 0x56508cfd9330 2023-11-15T03:04:01.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.396+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v97'48 uv48 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.396+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfde140 con 0x56508cfd9330 2023-11-15T03:04:01.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.400+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v97'49 uv49 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.400+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfdcc30 con 0x56508cfd9330 2023-11-15T03:04:01.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.404+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v97'50 uv50 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.404+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfdd560 con 0x56508cfd9330 2023-11-15T03:04:01.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.408+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v97'51 uv51 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.408+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfde540 con 0x56508cfd9330 2023-11-15T03:04:01.415 INFO:teuthology.orchestra.run.smithi121.stdout:Write #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:04:01.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.408+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v97'52 uv52 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.408+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfdf550 con 0x56508cfd9330 2023-11-15T03:04:01.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.412+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [setxattr (30)] v97'53 uv53 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.412+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfdf550 con 0x56508cfd9330 2023-11-15T03:04:01.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.412+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [setxattr (62)] v97'54 uv54 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.416+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfdd910 con 0x56508cfd9330 2023-11-15T03:04:01.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.416+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [omap-set-header] v97'55 uv55 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.416+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfdd910 con 0x56508cfd9330 2023-11-15T03:04:01.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.420+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 603.00000000 [create] v97'56 uv43 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.420+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cfdd910 con 0x56508cfd9330 2023-11-15T03:04:01.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.420+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 603.00000000 [delete] v97'57 uv57 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.424+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfdd910 con 0x56508cfd9330 2023-11-15T03:04:01.431 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:04:01.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.424+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v97'58 uv58 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.424+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfdd910 con 0x56508cfd9330 2023-11-15T03:04:01.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.428+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [setxattr (30)] v97'59 uv59 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.428+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfdd910 con 0x56508cfd9330 2023-11-15T03:04:01.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.432+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [setxattr (62)] v97'60 uv60 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.432+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfdd910 con 0x56508cfd9330 2023-11-15T03:04:01.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.432+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [omap-set-header] v97'61 uv61 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.432+0000 7eff9d930d00 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x56508cfdea50 0x56508cfe0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.432+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:30 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfe1330 con 0x56508cfdea50 2023-11-15T03:04:01.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.432+0000 7eff89644700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x56508cfdea50 0x56508cfe0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.436+0000 7eff89644700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x56508cfdea50 0x56508cfe0e40 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:01.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.436+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(30 1.00000000.inode [create] v97'27 uv21 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfdea50 2023-11-15T03:04:01.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.436+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:31 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cfe1720 con 0x56508cfdea50 2023-11-15T03:04:01.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.440+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(31 1.00000000.inode [delete] v97'28 uv28 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfdea50 2023-11-15T03:04:01.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.440+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfe1bd0 con 0x56508cfdea50 2023-11-15T03:04:01.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.444+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(32 1.00000000.inode [create] v97'29 uv29 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfdea50 2023-11-15T03:04:01.450 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:04:01.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.444+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfe24c0 con 0x56508cfdea50 2023-11-15T03:04:01.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.448+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(33 1.00000000.inode [write 0~554] v97'30 uv30 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfdea50 2023-11-15T03:04:01.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.448+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfe2870 con 0x56508cfdea50 2023-11-15T03:04:01.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.448+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v97'31 uv31 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfdea50 2023-11-15T03:04:01.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.448+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfe2c70 con 0x56508cfdea50 2023-11-15T03:04:01.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.452+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(35 200.00000000 [create] v97'32 uv26 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfdea50 2023-11-15T03:04:01.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.452+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cfe3020 con 0x56508cfdea50 2023-11-15T03:04:01.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.456+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(36 200.00000000 [delete] v97'33 uv33 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfdea50 2023-11-15T03:04:01.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.456+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfe33d0 con 0x56508cfdea50 2023-11-15T03:04:01.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.456+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(37 200.00000000 [create] v97'34 uv34 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfdea50 2023-11-15T03:04:01.465 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:04:01.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.456+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfe33d0 con 0x56508cfdea50 2023-11-15T03:04:01.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.460+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(38 200.00000000 [write 0~90] v97'35 uv35 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfdea50 2023-11-15T03:04:01.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.460+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfe37d0 con 0x56508cfdea50 2023-11-15T03:04:01.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.464+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(39 200.00000000 [omap-set-header] v97'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfdea50 2023-11-15T03:04:01.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.464+0000 7eff9d930d00 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56508cfe3ce0 0x56508cfe60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.464+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:40 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfe65c0 con 0x56508cfe3ce0 2023-11-15T03:04:01.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.464+0000 7eff92646700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56508cfe3ce0 0x56508cfe60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.464+0000 7eff92646700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56508cfe3ce0 0x56508cfe60d0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:01.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.468+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(40 200.00000001 [create] v97'40 uv32 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.468+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:41 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cfe65c0 con 0x56508cfe3ce0 2023-11-15T03:04:01.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.472+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(41 200.00000001 [delete] v97'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.472+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfe65c0 con 0x56508cfe3ce0 2023-11-15T03:04:01.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.476+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(42 200.00000001 [create] v97'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.481 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:04:01.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.476+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfea650 con 0x56508cfe3ce0 2023-11-15T03:04:01.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.480+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(43 200.00000001 [write 0~16484] v97'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.480+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfeaa80 con 0x56508cfe3ce0 2023-11-15T03:04:01.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.480+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(44 200.00000001 [omap-set-header] v97'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.480+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfe65c0 con 0x56508cfe3ce0 2023-11-15T03:04:01.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.484+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(45 606.00000000 [create] v97'45 uv38 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.484+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cfe65c0 con 0x56508cfe3ce0 2023-11-15T03:04:01.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.488+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(46 606.00000000 [delete] v97'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.488+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfe65c0 con 0x56508cfe3ce0 2023-11-15T03:04:01.498 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:04:01.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.492+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(47 606.00000000 [create] v97'47 uv47 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.492+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfe65c0 con 0x56508cfe3ce0 2023-11-15T03:04:01.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.492+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v97'48 uv48 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.496+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfe65c0 con 0x56508cfe3ce0 2023-11-15T03:04:01.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.496+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v97'49 uv49 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.496+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfe6600 con 0x56508cfe3ce0 2023-11-15T03:04:01.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.500+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(50 606.00000000 [omap-set-header] v97'50 uv50 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.500+0000 7eff9d930d00 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x56508cfe6f10 0x56508cfe9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.500+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:51 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfe97f0 con 0x56508cfe6f10 2023-11-15T03:04:01.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.500+0000 7eff91e45700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x56508cfe6f10 0x56508cfe9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.504+0000 7eff91e45700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x56508cfe6f10 0x56508cfe9300 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:01.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.504+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(51 607.00000000 [create] v97'15 uv14 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfe6f10 2023-11-15T03:04:01.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.504+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:52 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cfe9890 con 0x56508cfe6f10 2023-11-15T03:04:01.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.508+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(52 607.00000000 [delete] v97'16 uv16 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfe6f10 2023-11-15T03:04:01.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.508+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfe9cc0 con 0x56508cfe6f10 2023-11-15T03:04:01.519 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:04:01.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.508+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(53 607.00000000 [create] v97'17 uv17 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfe6f10 2023-11-15T03:04:01.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.508+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfeacd0 con 0x56508cfe6f10 2023-11-15T03:04:01.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.512+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v97'18 uv18 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfe6f10 2023-11-15T03:04:01.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.512+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfeacd0 con 0x56508cfe6f10 2023-11-15T03:04:01.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.516+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v97'19 uv19 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfe6f10 2023-11-15T03:04:01.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.516+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfe9cc0 con 0x56508cfe6f10 2023-11-15T03:04:01.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.516+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v97'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfe6f10 2023-11-15T03:04:01.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.516+0000 7eff9d930d00 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56508cfea1d0 0x56508cfec5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.516+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:57 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfeccb0 con 0x56508cfea1d0 2023-11-15T03:04:01.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.516+0000 7eff89644700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56508cfea1d0 0x56508cfec5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.516+0000 7eff89644700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56508cfea1d0 0x56508cfec5c0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:01.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.520+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v97'12 uv11 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.520+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:58 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.524+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v97'13 uv13 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.524+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.534 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:04:01.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.528+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v97'14 uv14 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.528+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.528+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v97'15 uv15 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.528+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.532+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(61 608.00000000 [create] v97'15 uv14 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.532+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.536+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(62 608.00000000 [delete] v97'16 uv16 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.536+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.544 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:04:01.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.536+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(63 608.00000000 [create] v97'17 uv17 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.536+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfee0f0 con 0x56508cfea1d0 2023-11-15T03:04:01.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.540+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v97'18 uv18 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.540+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfee0f0 con 0x56508cfea1d0 2023-11-15T03:04:01.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.540+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v97'19 uv19 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.540+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.544+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(66 608.00000000 [omap-set-header] v97'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.544+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfd9330 2023-11-15T03:04:01.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.548+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(67 604.00000000 [create] v97'26 uv20 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.548+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfd9330 2023-11-15T03:04:01.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.548+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(68 604.00000000 [delete] v97'27 uv27 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.548+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfd9330 2023-11-15T03:04:01.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.552+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v97'28 uv28 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.560 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:04:01.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.552+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfee0f0 con 0x56508cfd9330 2023-11-15T03:04:01.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.556+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v97'29 uv29 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.556+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfee0f0 con 0x56508cfd9330 2023-11-15T03:04:01.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.560+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v97'30 uv30 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.560+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfd9330 2023-11-15T03:04:01.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.560+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [omap-set-header] v97'31 uv31 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.560+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfd9330 2023-11-15T03:04:01.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.568+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 500.00000000 [create] v97'32 uv25 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.568+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfd9330 2023-11-15T03:04:01.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.572+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 500.00000000 [delete] v97'33 uv33 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.572+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfd9330 2023-11-15T03:04:01.581 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:04:01.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.572+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v97'34 uv34 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.572+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfd9330 2023-11-15T03:04:01.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.576+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [write 0~90] v97'35 uv35 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.576+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfd9330 2023-11-15T03:04:01.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.580+0000 7eff91e45700 1 -- 172.21.15.121:0/2564052450 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [omap-set-header] v97'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff84014640 con 0x56508cfd9330 2023-11-15T03:04:01.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.580+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:78 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfdea50 2023-11-15T03:04:01.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.584+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(78 mds_snaptable [create] v97'12 uv11 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfdea50 2023-11-15T03:04:01.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.584+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:79 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfdea50 2023-11-15T03:04:01.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.588+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(79 mds_snaptable [delete] v97'13 uv13 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfdea50 2023-11-15T03:04:01.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.588+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfdea50 2023-11-15T03:04:01.597 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:04:01.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.588+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(80 mds_snaptable [create] v97'14 uv14 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfdea50 2023-11-15T03:04:01.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.588+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfdea50 2023-11-15T03:04:01.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.592+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(81 mds_snaptable [write 0~70] v97'15 uv15 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfdea50 2023-11-15T03:04:01.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.592+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfdea50 2023-11-15T03:04:01.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.592+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v97'16 uv16 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfdea50 2023-11-15T03:04:01.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.592+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.596+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(83 605.00000000 [create] v97'15 uv14 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.596+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.600+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(84 605.00000000 [delete] v97'16 uv16 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.600+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.604+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(85 605.00000000 [create] v97'17 uv17 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.609 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:04:01.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.604+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfee0f0 con 0x56508cfea1d0 2023-11-15T03:04:01.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.604+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v97'18 uv18 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.604+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfee0f0 con 0x56508cfea1d0 2023-11-15T03:04:01.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.608+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v97'19 uv19 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.608+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.612+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(88 605.00000000 [omap-set-header] v97'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.612+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.612+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(89 mds0_inotable [create] v97'13 uv12 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.612+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.616+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(90 mds0_inotable [delete] v97'14 uv14 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.616+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.620+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(91 mds0_inotable [create] v97'15 uv15 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.627 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:04:01.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.620+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.624+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(92 mds0_inotable [write 0~34] v97'16 uv16 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.624+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.628+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v97'17 uv17 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.628+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.632+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(94 100.00000000 [create] v97'12 uv11 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.632+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.636+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(95 100.00000000 [delete] v97'13 uv13 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.636+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.636+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(96 100.00000000 [create] v97'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.645 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:04:01.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.640+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfed0e0 con 0x56508cfea1d0 2023-11-15T03:04:01.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.644+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(97 100.00000000 [omap-set-header] v97'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.644+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfefda0 con 0x56508cfea1d0 2023-11-15T03:04:01.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.648+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v97'16 uv16 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.648+0000 7eff9d930d00 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56508cfed4e0 0x56508cff03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.648+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfef8d0 con 0x56508cfed4e0 2023-11-15T03:04:01.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.648+0000 7eff92646700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56508cfed4e0 0x56508cff03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.648+0000 7eff92646700 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56508cfed4e0 0x56508cff03f0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:01.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.652+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(99 mds0_sessionmap [create] v97'11 uv10 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfed4e0 2023-11-15T03:04:01.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.652+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cfef8d0 con 0x56508cfed4e0 2023-11-15T03:04:01.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.656+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(100 mds0_sessionmap [delete] v97'12 uv12 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfed4e0 2023-11-15T03:04:01.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.656+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfef8d0 con 0x56508cfed4e0 2023-11-15T03:04:01.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.656+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(101 mds0_sessionmap [create] v97'13 uv13 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfed4e0 2023-11-15T03:04:01.664 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:04:01.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.656+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfef8d0 con 0x56508cfed4e0 2023-11-15T03:04:01.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.660+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v97'14 uv14 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfed4e0 2023-11-15T03:04:01.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.660+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfef8d0 con 0x56508cfea1d0 2023-11-15T03:04:01.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.664+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(103 609.00000000 [create] v97'15 uv14 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.664+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cfef8d0 con 0x56508cfea1d0 2023-11-15T03:04:01.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.668+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(104 609.00000000 [delete] v97'16 uv16 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.668+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfef8d0 con 0x56508cfea1d0 2023-11-15T03:04:01.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.668+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(105 609.00000000 [create] v97'17 uv17 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.680 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:04:01.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.668+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfef8d0 con 0x56508cfea1d0 2023-11-15T03:04:01.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.672+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v97'18 uv18 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.672+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfef8d0 con 0x56508cfea1d0 2023-11-15T03:04:01.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.676+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v97'19 uv19 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.676+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfef8d0 con 0x56508cfea1d0 2023-11-15T03:04:01.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.680+0000 7eff89644700 1 -- 172.21.15.121:0/2564052450 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(108 609.00000000 [omap-set-header] v97'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff7c049fb0 con 0x56508cfea1d0 2023-11-15T03:04:01.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.680+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfef8d0 con 0x56508cfe3ce0 2023-11-15T03:04:01.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.684+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(109 400.00000000 [create] v97'12 uv11 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.684+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cfef8d0 con 0x56508cfe3ce0 2023-11-15T03:04:01.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.684+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(110 400.00000000 [delete] v97'13 uv13 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.684+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cfef8d0 con 0x56508cfe3ce0 2023-11-15T03:04:01.695 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:04:01.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.688+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(111 400.00000000 [create] v97'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.688+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cff08e0 con 0x56508cfe3ce0 2023-11-15T03:04:01.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.692+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(112 400.00000000 [write 0~22] v97'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.692+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cff08e0 con 0x56508cfe3ce0 2023-11-15T03:04:01.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.692+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(113 400.00000000 [omap-set-header] v97'16 uv16 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.696+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cff08e0 con 0x56508cfe3ce0 2023-11-15T03:04:01.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.696+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(114 100.00000000.inode [create] v97'30 uv22 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.696+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cff08e0 con 0x56508cfe3ce0 2023-11-15T03:04:01.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.700+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(115 100.00000000.inode [delete] v97'31 uv31 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.700+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cff08e0 con 0x56508cfe3ce0 2023-11-15T03:04:01.710 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:04:01.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.704+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(116 100.00000000.inode [create] v97'32 uv32 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.704+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cff08e0 con 0x56508cfe3ce0 2023-11-15T03:04:01.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.704+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v97'33 uv33 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.704+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cff08e0 con 0x56508cfe3ce0 2023-11-15T03:04:01.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.708+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v97'34 uv34 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.708+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cff08e0 con 0x56508cfe3ce0 2023-11-15T03:04:01.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.712+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(119 1.00000000 [create] v97'35 uv29 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.712+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x56508cff08e0 con 0x56508cfe3ce0 2023-11-15T03:04:01.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.712+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(120 1.00000000 [delete] v97'36 uv36 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.712+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cff08e0 con 0x56508cfe3ce0 2023-11-15T03:04:01.722 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:04:01.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.716+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(121 1.00000000 [create] v97'37 uv37 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.716+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cff18f0 con 0x56508cfe3ce0 2023-11-15T03:04:01.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.720+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v97'38 uv38 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.720+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cff18f0 con 0x56508cfe3ce0 2023-11-15T03:04:01.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.720+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v97'39 uv39 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.720+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cff08e0 con 0x56508cfe3ce0 2023-11-15T03:04:01.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.724+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v97'40 uv40 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.724+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e97) v8 -- 0x56508cff0dd0 con 0x56508cfe3ce0 2023-11-15T03:04:01.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v97'41 uv41 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7eff8c004520 con 0x56508cfe3ce0 2023-11-15T03:04:01.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x56508cfdea50 msgr2=0x56508cfe0e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x56508cfdea50 0x56508cfe0e40 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56508cfb3e50 msgr2=0x56508cf28910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56508cfb3e50 0x56508cf28910 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x56508cfe6f10 msgr2=0x56508cfe9300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x56508cfe6f10 0x56508cfe9300 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56508cfd9330 msgr2=0x56508cfdbfe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56508cfd9330 0x56508cfdbfe0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56508cfed4e0 msgr2=0x56508cff03f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56508cfed4e0 0x56508cff03f0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56508cfe3ce0 msgr2=0x56508cfe60d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56508cfe3ce0 0x56508cfe60d0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56508cfea1d0 msgr2=0x56508cfec5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56508cfea1d0 0x56508cfec5c0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 reap_dead start 2023-11-15T03:04:01.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 reap_dead start 2023-11-15T03:04:01.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 reap_dead start 2023-11-15T03:04:01.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff92646700 1 -- 172.21.15.121:0/2564052450 reap_dead start 2023-11-15T03:04:01.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7eff74058ff0 msgr2=0x7eff7405b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7eff74058ff0 0x7eff7405b4b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7eff840097f0 tx=0x7eff84018000).stop 2023-11-15T03:04:01.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56508cf2bb00 msgr2=0x56508cfc5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56508cf2bb00 0x56508cfc5520 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7eff7c010000 tx=0x7eff7c016000).stop 2023-11-15T03:04:01.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 shutdown_connections 2023-11-15T03:04:01.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7eff74058ff0 0x7eff7405b4b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 --2- 172.21.15.121:0/2564052450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56508cf2bb00 0x56508cfc5520 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 >> 172.21.15.121:0/2564052450 conn(0x56508cf31080 msgr2=0x56508cf10c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:01.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 shutdown_connections 2023-11-15T03:04:01.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.728+0000 7eff9d930d00 1 -- 172.21.15.121:0/2564052450 wait complete. 2023-11-15T03:04:01.743 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 1.00000000.inode 2023-11-15T03:04:01.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.836+0000 7f26d2337d00 1 Processor -- start 2023-11-15T03:04:01.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.836+0000 7f26d2337d00 1 -- start start 2023-11-15T03:04:01.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26d2337d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fded982920 0x55fded9b1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26d2337d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fded9c9be0 0x55fdeda5b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26d2337d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fdeda59130 0x55fdeda5b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26d2337d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fded8f7220 con 0x55fded9c9be0 2023-11-15T03:04:01.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26d2337d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fded982ee0 con 0x55fded982920 2023-11-15T03:04:01.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26d2337d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fdeda4eb70 con 0x55fdeda59130 2023-11-15T03:04:01.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26c61cc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fded9c9be0 0x55fdeda5b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26c61cc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fded9c9be0 0x55fdeda5b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38538/0 (socket says 172.21.15.121:38538) 2023-11-15T03:04:01.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26c61cc700 1 -- 172.21.15.121:0/1448664117 learned_addr learned my addr 172.21.15.121:0/1448664117 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:01.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26c69cd700 1 --2- 172.21.15.121:0/1448664117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fded982920 0x55fded9b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26c71ce700 1 --2- 172.21.15.121:0/1448664117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fdeda59130 0x55fdeda5b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26c61cc700 1 -- 172.21.15.121:0/1448664117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fdeda59130 msgr2=0x55fdeda5b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:01.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26c61cc700 1 --2- 172.21.15.121:0/1448664117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fdeda59130 0x55fdeda5b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26c61cc700 1 -- 172.21.15.121:0/1448664117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fded982920 msgr2=0x55fded9b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:01.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26c61cc700 1 --2- 172.21.15.121:0/1448664117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fded982920 0x55fded9b1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26c61cc700 1 -- 172.21.15.121:0/1448664117 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc005db0 con 0x55fded9c9be0 2023-11-15T03:04:01.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26c69cd700 1 --2- 172.21.15.121:0/1448664117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fded982920 0x55fded9b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:01.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26c61cc700 1 --2- 172.21.15.121:0/1448664117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fded9c9be0 0x55fdeda5b330 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7f26bc005720 tx=0x7f26bc00c040).ready entity=mon.0 client_cookie=abb172ce6f0d987f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:01.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26c51ca700 1 -- 172.21.15.121:0/1448664117 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc00f040 con 0x55fded9c9be0 2023-11-15T03:04:01.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26c51ca700 1 -- 172.21.15.121:0/1448664117 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26bc009430 con 0x55fded9c9be0 2023-11-15T03:04:01.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26c51ca700 1 -- 172.21.15.121:0/1448664117 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc008cd0 con 0x55fded9c9be0 2023-11-15T03:04:01.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26d2337d00 1 -- 172.21.15.121:0/1448664117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fded9c9be0 msgr2=0x55fdeda5b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26d2337d00 1 --2- 172.21.15.121:0/1448664117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fded9c9be0 0x55fdeda5b330 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7f26bc005720 tx=0x7f26bc00c040).stop 2023-11-15T03:04:01.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26d2337d00 1 -- 172.21.15.121:0/1448664117 shutdown_connections 2023-11-15T03:04:01.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26d2337d00 1 --2- 172.21.15.121:0/1448664117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fdeda59130 0x55fdeda5b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26d2337d00 1 --2- 172.21.15.121:0/1448664117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fded9c9be0 0x55fdeda5b330 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26d2337d00 1 --2- 172.21.15.121:0/1448664117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fded982920 0x55fded9b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.840+0000 7f26d2337d00 1 -- 172.21.15.121:0/1448664117 >> 172.21.15.121:0/1448664117 conn(0x55fded9c88f0 msgr2=0x55fded9b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:01.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.844+0000 7f26d2337d00 1 -- 172.21.15.121:0/1448664117 shutdown_connections 2023-11-15T03:04:01.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.844+0000 7f26d2337d00 1 -- 172.21.15.121:0/1448664117 wait complete. 2023-11-15T03:04:01.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.844+0000 7f26d2337d00 1 Processor -- start 2023-11-15T03:04:01.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.844+0000 7f26d2337d00 1 -- start start 2023-11-15T03:04:01.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.844+0000 7f26d2337d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fded982920 0x55fdeda67150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.844+0000 7f26d2337d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fded9c9be0 0x55fdeda54110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.844+0000 7f26d2337d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fdeda59130 0x55fdeda54600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.844+0000 7f26d2337d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fded97f3d0 con 0x55fded9c9be0 2023-11-15T03:04:01.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.844+0000 7f26d2337d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdeda54af0 con 0x55fded982920 2023-11-15T03:04:01.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.844+0000 7f26d2337d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fded9805a0 con 0x55fdeda59130 2023-11-15T03:04:01.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.844+0000 7f26c61cc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fded9c9be0 0x55fdeda54110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.844+0000 7f26c61cc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fded9c9be0 0x55fdeda54110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38550/0 (socket says 172.21.15.121:38550) 2023-11-15T03:04:01.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.844+0000 7f26c61cc700 1 -- 172.21.15.121:0/1216570471 learned_addr learned my addr 172.21.15.121:0/1216570471 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:01.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.848+0000 7f26c69cd700 1 --2- 172.21.15.121:0/1216570471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fded982920 0x55fdeda67150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.848+0000 7f26c71ce700 1 --2- 172.21.15.121:0/1216570471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fdeda59130 0x55fdeda54600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.848+0000 7f26c61cc700 1 -- 172.21.15.121:0/1216570471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fdeda59130 msgr2=0x55fdeda54600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.848+0000 7f26c61cc700 1 --2- 172.21.15.121:0/1216570471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fdeda59130 0x55fdeda54600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.848+0000 7f26c61cc700 1 -- 172.21.15.121:0/1216570471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fded982920 msgr2=0x55fdeda67150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.848+0000 7f26c61cc700 1 --2- 172.21.15.121:0/1216570471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fded982920 0x55fdeda67150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.848+0000 7f26c61cc700 1 -- 172.21.15.121:0/1216570471 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc0095a0 con 0x55fded9c9be0 2023-11-15T03:04:01.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.848+0000 7f26c61cc700 1 --2- 172.21.15.121:0/1216570471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fded9c9be0 0x55fdeda54110 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7f26bc012000 tx=0x7f26bc018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:01.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.848+0000 7f26b77fe700 1 -- 172.21.15.121:0/1216570471 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc00f040 con 0x55fded9c9be0 2023-11-15T03:04:01.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.848+0000 7f26b77fe700 1 -- 172.21.15.121:0/1216570471 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26bc014d50 con 0x55fded9c9be0 2023-11-15T03:04:01.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.848+0000 7f26b77fe700 1 -- 172.21.15.121:0/1216570471 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc01e620 con 0x55fded9c9be0 2023-11-15T03:04:01.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.848+0000 7f26d2337d00 1 -- 172.21.15.121:0/1216570471 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c000a0e0 con 0x55fded9c9be0 2023-11-15T03:04:01.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.848+0000 7f26d2337d00 1 -- 172.21.15.121:0/1216570471 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fded920770 con 0x55fded9c9be0 2023-11-15T03:04:01.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.852+0000 7f26b77fe700 1 -- 172.21.15.121:0/1216570471 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f26bc01ea50 con 0x55fded9c9be0 2023-11-15T03:04:01.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.852+0000 7f26b77fe700 1 --2- 172.21.15.121:0/1216570471 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f26ac058ff0 0x7f26ac05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.852+0000 7f26b77fe700 1 -- 172.21.15.121:0/1216570471 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6524+0+0 (secure 0 0 0) 0x7f26bc074e70 con 0x55fded9c9be0 2023-11-15T03:04:01.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.852+0000 7f26c69cd700 1 --2- 172.21.15.121:0/1216570471 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f26ac058ff0 0x7f26ac05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.852+0000 7f26d2337d00 1 --2- 172.21.15.121:0/1216570471 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55fded9affd0 0x55fdeda539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:01.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.852+0000 7f26d2337d00 1 -- 172.21.15.121:0/1216570471 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e97) v8 -- 0x55fdeda4fec0 con 0x55fded9affd0 2023-11-15T03:04:01.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.852+0000 7f26c71ce700 1 --2- 172.21.15.121:0/1216570471 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55fded9affd0 0x55fdeda539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:01.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.852+0000 7f26c69cd700 1 --2- 172.21.15.121:0/1216570471 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f26ac058ff0 0x7f26ac05b4b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f26b8000c00 tx=0x7f26b80019c0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:01.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.852+0000 7f26c71ce700 1 --2- 172.21.15.121:0/1216570471 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55fded9affd0 0x55fdeda539e0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:01.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.856+0000 7f26c71ce700 1 -- 172.21.15.121:0/1216570471 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(1 1.00000000.inode [delete] v97'37 uv37 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f26c000d740 con 0x55fded9affd0 2023-11-15T03:04:01.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.856+0000 7f26d2337d00 1 -- 172.21.15.121:0/1216570471 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55fded9affd0 msgr2=0x55fdeda539e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.856+0000 7f26d2337d00 1 --2- 172.21.15.121:0/1216570471 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55fded9affd0 0x55fdeda539e0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:01.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.856+0000 7f26d2337d00 1 -- 172.21.15.121:0/1216570471 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f26ac058ff0 msgr2=0x7f26ac05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.856+0000 7f26d2337d00 1 --2- 172.21.15.121:0/1216570471 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f26ac058ff0 0x7f26ac05b4b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f26b8000c00 tx=0x7f26b80019c0).stop 2023-11-15T03:04:01.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.856+0000 7f26d2337d00 1 -- 172.21.15.121:0/1216570471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fded9c9be0 msgr2=0x55fdeda54110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:01.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.856+0000 7f26d2337d00 1 --2- 172.21.15.121:0/1216570471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fded9c9be0 0x55fdeda54110 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7f26bc012000 tx=0x7f26bc018000).stop 2023-11-15T03:04:01.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.856+0000 7f26c71ce700 1 -- 172.21.15.121:0/1216570471 reap_dead start 2023-11-15T03:04:01.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.856+0000 7f26d2337d00 1 -- 172.21.15.121:0/1216570471 shutdown_connections 2023-11-15T03:04:01.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.856+0000 7f26d2337d00 1 -- 172.21.15.121:0/1216570471 >> 172.21.15.121:0/1216570471 conn(0x55fded9c88f0 msgr2=0x55fded9b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:01.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.856+0000 7f26d2337d00 1 -- 172.21.15.121:0/1216570471 shutdown_connections 2023-11-15T03:04:01.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:01.856+0000 7f26d2337d00 1 -- 172.21.15.121:0/1216570471 wait complete. 2023-11-15T03:04:01.881 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:04:02.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.048+0000 7ffa8f369700 1 Processor -- start 2023-11-15T03:04:02.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.048+0000 7ffa8f369700 1 -- start start 2023-11-15T03:04:02.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.048+0000 7ffa8f369700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa880682a0 0x7ffa880c10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:02.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.048+0000 7ffa8f369700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa88153700 0x7ffa88153b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:02.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.048+0000 7ffa8f369700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa880c15a0 con 0x7ffa88067ec0 2023-11-15T03:04:02.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.048+0000 7ffa8f369700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa880686e0 con 0x7ffa880682a0 2023-11-15T03:04:02.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.048+0000 7ffa8f369700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa880c0c20 con 0x7ffa88153700 2023-11-15T03:04:02.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.048+0000 7ffa8c904700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa880682a0 0x7ffa880c10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:02.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.048+0000 7ffa8d105700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ffa88067ec0 0x7ffa88062c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:39296/0 (socket says 172.21.15.121:39296) 2023-11-15T03:04:02.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.048+0000 7ffa8d105700 1 -- 172.21.15.121:0/2035871992 learned_addr learned my addr 172.21.15.121:0/2035871992 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:02.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.048+0000 7ffa8d906700 1 --2- 172.21.15.121:0/2035871992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa88153700 0x7ffa88153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:02.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.048+0000 7ffa8c904700 1 -- 172.21.15.121:0/2035871992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa88153700 msgr2=0x7ffa88153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:02.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.052+0000 7ffa8c904700 1 --2- 172.21.15.121:0/2035871992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa88153700 0x7ffa88153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:02.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.052+0000 7ffa8c904700 1 -- 172.21.15.121:0/2035871992 >> v1:172.21.15.190:6789/0 conn(0x7ffa88067ec0 legacy=0x7ffa88062c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:02.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.052+0000 7ffa7ffff700 1 -- 172.21.15.121:0/2035871992 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217459110 0 0) 0x7ffa84001110 con 0x7ffa88067ec0 2023-11-15T03:04:02.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.052+0000 7ffa8c904700 1 -- 172.21.15.121:0/2035871992 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa88158730 con 0x7ffa880682a0 2023-11-15T03:04:02.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.052+0000 7ffa8c904700 1 --2- 172.21.15.121:0/2035871992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa880682a0 0x7ffa880c10b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7ffa78009760 tx=0x7ffa78009060).ready entity=mon.0 client_cookie=c3e4c97eb341537e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:02.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.052+0000 7ffa7ffff700 1 -- 172.21.15.121:0/2035871992 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7800b580 con 0x7ffa880682a0 2023-11-15T03:04:02.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.052+0000 7ffa7ffff700 1 -- 172.21.15.121:0/2035871992 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7800b6e0 con 0x7ffa880682a0 2023-11-15T03:04:02.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.052+0000 7ffa7ffff700 1 -- 172.21.15.121:0/2035871992 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7800ba60 con 0x7ffa880682a0 2023-11-15T03:04:02.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.052+0000 7ffa8f369700 1 -- 172.21.15.121:0/2035871992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa880682a0 msgr2=0x7ffa880c10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:02.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.052+0000 7ffa8f369700 1 --2- 172.21.15.121:0/2035871992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa880682a0 0x7ffa880c10b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7ffa78009760 tx=0x7ffa78009060).stop 2023-11-15T03:04:02.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.052+0000 7ffa8f369700 1 -- 172.21.15.121:0/2035871992 shutdown_connections 2023-11-15T03:04:02.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.052+0000 7ffa8f369700 1 --2- 172.21.15.121:0/2035871992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa88153700 0x7ffa88153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:02.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.052+0000 7ffa8f369700 1 --2- 172.21.15.121:0/2035871992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa880682a0 0x7ffa880c10b0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:02.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.052+0000 7ffa8f369700 1 -- 172.21.15.121:0/2035871992 >> 172.21.15.121:0/2035871992 conn(0x7ffa880cad40 msgr2=0x7ffa880cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:02.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.052+0000 7ffa8f369700 1 -- 172.21.15.121:0/2035871992 shutdown_connections 2023-11-15T03:04:02.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.052+0000 7ffa8f369700 1 -- 172.21.15.121:0/2035871992 wait complete. 2023-11-15T03:04:02.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.056+0000 7ffa8f369700 1 Processor -- start 2023-11-15T03:04:02.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.056+0000 7ffa8f369700 1 -- start start 2023-11-15T03:04:02.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.056+0000 7ffa8f369700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa88067ec0 0x7ffa880af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:02.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.056+0000 7ffa8f369700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa880682a0 0x7ffa88164cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:02.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.056+0000 7ffa8f369700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa88153700 0x7ffa88166ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:02.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.056+0000 7ffa8f369700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa880af960 con 0x7ffa88067ec0 2023-11-15T03:04:02.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.056+0000 7ffa8f369700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa880afc40 con 0x7ffa880682a0 2023-11-15T03:04:02.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.056+0000 7ffa8f369700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa880aff20 con 0x7ffa88153700 2023-11-15T03:04:02.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.060+0000 7ffa8d105700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa88067ec0 0x7ffa880af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:02.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.060+0000 7ffa8d105700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa88067ec0 0x7ffa880af380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38558/0 (socket says 172.21.15.121:38558) 2023-11-15T03:04:02.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.060+0000 7ffa8d105700 1 -- 172.21.15.121:0/2520645134 learned_addr learned my addr 172.21.15.121:0/2520645134 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:02.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.060+0000 7ffa8d906700 1 --2- 172.21.15.121:0/2520645134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa88153700 0x7ffa88166ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:02.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.060+0000 7ffa8c904700 1 --2- 172.21.15.121:0/2520645134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa880682a0 0x7ffa88164cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:02.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.060+0000 7ffa8d105700 1 -- 172.21.15.121:0/2520645134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa88153700 msgr2=0x7ffa88166ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:02.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.060+0000 7ffa8d105700 1 --2- 172.21.15.121:0/2520645134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa88153700 0x7ffa88166ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:02.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.060+0000 7ffa8d105700 1 -- 172.21.15.121:0/2520645134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa880682a0 msgr2=0x7ffa88164cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:02.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.060+0000 7ffa8d105700 1 --2- 172.21.15.121:0/2520645134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa880682a0 0x7ffa88164cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:02.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.060+0000 7ffa8d105700 1 -- 172.21.15.121:0/2520645134 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa780092a0 con 0x7ffa88067ec0 2023-11-15T03:04:02.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.060+0000 7ffa8d105700 1 --2- 172.21.15.121:0/2520645134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa88067ec0 0x7ffa880af380 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7ffa8400c510 tx=0x7ffa84007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:02.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.060+0000 7ffa7d7fa700 1 -- 172.21.15.121:0/2520645134 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa840072e0 con 0x7ffa88067ec0 2023-11-15T03:04:02.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.060+0000 7ffa7d7fa700 1 -- 172.21.15.121:0/2520645134 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa84007440 con 0x7ffa88067ec0 2023-11-15T03:04:02.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.060+0000 7ffa8f369700 1 -- 172.21.15.121:0/2520645134 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa880b0200 con 0x7ffa88067ec0 2023-11-15T03:04:02.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.060+0000 7ffa7d7fa700 1 -- 172.21.15.121:0/2520645134 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa840057a0 con 0x7ffa88067ec0 2023-11-15T03:04:02.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.060+0000 7ffa8f369700 1 -- 172.21.15.121:0/2520645134 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa881673c0 con 0x7ffa88067ec0 2023-11-15T03:04:02.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.060+0000 7ffa8f369700 1 -- 172.21.15.121:0/2520645134 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa88167640 con 0x7ffa88067ec0 2023-11-15T03:04:02.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.064+0000 7ffa7d7fa700 1 -- 172.21.15.121:0/2520645134 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ffa84005bd0 con 0x7ffa88067ec0 2023-11-15T03:04:02.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.064+0000 7ffa7d7fa700 1 --2- 172.21.15.121:0/2520645134 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa6c05a510 0x7ffa6c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:02.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.064+0000 7ffa7d7fa700 1 -- 172.21.15.121:0/2520645134 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6524+0+0 (secure 0 0 0) 0x7ffa840740f0 con 0x7ffa88067ec0 2023-11-15T03:04:02.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.064+0000 7ffa8c904700 1 --2- 172.21.15.121:0/2520645134 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa6c05a510 0x7ffa6c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:02.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.068+0000 7ffa8c904700 1 --2- 172.21.15.121:0/2520645134 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa6c05a510 0x7ffa6c05c9d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7ffa88165430 tx=0x7ffa78017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:02.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.072+0000 7ffa7d7fa700 1 -- 172.21.15.121:0/2520645134 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ffa84047740 con 0x7ffa88067ec0 2023-11-15T03:04:02.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:02.276+0000 7ffa8f369700 1 -- 172.21.15.121:0/2520645134 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7ffa880c15a0 con 0x7ffa88067ec0 2023-11-15T03:04:03.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.012+0000 7ffa7d7fa700 1 -- 172.21.15.121:0/2520645134 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v102) v1 ==== 162+0+0 (secure 0 0 0) 0x7ffa880c15a0 con 0x7ffa88067ec0 2023-11-15T03:04:03.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.020+0000 7ffa8f369700 1 -- 172.21.15.121:0/2520645134 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa6c05a510 msgr2=0x7ffa6c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:03.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.020+0000 7ffa8f369700 1 --2- 172.21.15.121:0/2520645134 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa6c05a510 0x7ffa6c05c9d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7ffa88165430 tx=0x7ffa78017000).stop 2023-11-15T03:04:03.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.020+0000 7ffa8f369700 1 -- 172.21.15.121:0/2520645134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa88067ec0 msgr2=0x7ffa880af380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:03.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.020+0000 7ffa8f369700 1 --2- 172.21.15.121:0/2520645134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa88067ec0 0x7ffa880af380 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7ffa8400c510 tx=0x7ffa84007ad0).stop 2023-11-15T03:04:03.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.020+0000 7ffa8f369700 1 -- 172.21.15.121:0/2520645134 shutdown_connections 2023-11-15T03:04:03.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.020+0000 7ffa8f369700 1 --2- 172.21.15.121:0/2520645134 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa6c05a510 0x7ffa6c05c9d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:03.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.020+0000 7ffa8f369700 1 --2- 172.21.15.121:0/2520645134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa88153700 0x7ffa88166ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:03.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.020+0000 7ffa8f369700 1 --2- 172.21.15.121:0/2520645134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa880682a0 0x7ffa88164cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:03.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.020+0000 7ffa8f369700 1 --2- 172.21.15.121:0/2520645134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa88067ec0 0x7ffa880af380 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:03.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.020+0000 7ffa8f369700 1 -- 172.21.15.121:0/2520645134 >> 172.21.15.121:0/2520645134 conn(0x7ffa880cad40 msgr2=0x7ffa88150940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:03.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.020+0000 7ffa8f369700 1 -- 172.21.15.121:0/2520645134 shutdown_connections 2023-11-15T03:04:03.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.020+0000 7ffa8f369700 1 -- 172.21.15.121:0/2520645134 wait complete. 2023-11-15T03:04:03.029 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:04:03.029 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:04:03.024+0000 7f989baae700 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2023-11-15T03:04:03.029 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:04:03.024+0000 7f989baae700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2023-11-15T03:04:03.050 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:04:03.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.212+0000 7f32d0e79700 1 Processor -- start 2023-11-15T03:04:03.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.212+0000 7f32d0e79700 1 -- start start 2023-11-15T03:04:03.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.216+0000 7f32d0e79700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f32cc066700 0x7f32cc154110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:03.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.216+0000 7f32d0e79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32cc0acf10 0x7f32cc062190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:03.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.216+0000 7f32d0e79700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32cc0c73e0 con 0x7f32cc066320 2023-11-15T03:04:03.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.216+0000 7f32d0e79700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32cc1546c0 con 0x7f32cc066700 2023-11-15T03:04:03.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.216+0000 7f32d0e79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc1549a0 con 0x7f32cc0acf10 2023-11-15T03:04:03.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.216+0000 7f32cb7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f32cc066320 0x7f32cc0656a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:38246/0 (socket says 172.21.15.121:38246) 2023-11-15T03:04:03.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.216+0000 7f32cb7fe700 1 -- 172.21.15.121:0/309354320 learned_addr learned my addr 172.21.15.121:0/309354320 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:03.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.216+0000 7f32caffd700 1 --2- 172.21.15.121:0/309354320 >> v2:172.21.15.190:3300/0 conn(0x7f32cc066700 0x7f32cc154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:03.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.216+0000 7f32cbfff700 1 --2- 172.21.15.121:0/309354320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32cc0acf10 0x7f32cc062190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:03.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.216+0000 7f32ca7fc700 1 -- 172.21.15.121:0/309354320 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413996304 0 0) 0x7f32cc0c73e0 con 0x7f32cc066320 2023-11-15T03:04:03.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.216+0000 7f32ca7fc700 1 -- 172.21.15.121:0/309354320 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32b40033f0 con 0x7f32cc066320 2023-11-15T03:04:03.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.216+0000 7f32caffd700 1 -- 172.21.15.121:0/309354320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32cc0acf10 msgr2=0x7f32cc062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:03.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.216+0000 7f32caffd700 1 --2- 172.21.15.121:0/309354320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32cc0acf10 0x7f32cc062190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:03.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.216+0000 7f32caffd700 1 -- 172.21.15.121:0/309354320 >> v1:172.21.15.121:6789/0 conn(0x7f32cc066320 legacy=0x7f32cc0656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:03.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.216+0000 7f32caffd700 1 -- 172.21.15.121:0/309354320 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32cc154c80 con 0x7f32cc066700 2023-11-15T03:04:03.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.216+0000 7f32caffd700 1 --2- 172.21.15.121:0/309354320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32cc066700 0x7f32cc154110 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f32c0009760 tx=0x7f32c0009060).ready entity=mon.1 client_cookie=91c5a5ae46ec9103 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:03.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.216+0000 7f32ca7fc700 1 -- 172.21.15.121:0/309354320 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c000d4a0 con 0x7f32cc066700 2023-11-15T03:04:03.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.216+0000 7f32ca7fc700 1 -- 172.21.15.121:0/309354320 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c000d600 con 0x7f32cc066700 2023-11-15T03:04:03.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.216+0000 7f32ca7fc700 1 -- 172.21.15.121:0/309354320 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c000d980 con 0x7f32cc066700 2023-11-15T03:04:03.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.220+0000 7f32d0e79700 1 -- 172.21.15.121:0/309354320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32cc066700 msgr2=0x7f32cc154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:03.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.220+0000 7f32d0e79700 1 --2- 172.21.15.121:0/309354320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32cc066700 0x7f32cc154110 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f32c0009760 tx=0x7f32c0009060).stop 2023-11-15T03:04:03.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.220+0000 7f32d0e79700 1 -- 172.21.15.121:0/309354320 shutdown_connections 2023-11-15T03:04:03.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.220+0000 7f32d0e79700 1 --2- 172.21.15.121:0/309354320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32cc0acf10 0x7f32cc062190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:03.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.220+0000 7f32d0e79700 1 --2- 172.21.15.121:0/309354320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32cc066700 0x7f32cc154110 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:03.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.220+0000 7f32d0e79700 1 -- 172.21.15.121:0/309354320 >> 172.21.15.121:0/309354320 conn(0x7f32cc067a60 msgr2=0x7f32cc067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:03.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.220+0000 7f32d0e79700 1 -- 172.21.15.121:0/309354320 shutdown_connections 2023-11-15T03:04:03.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.220+0000 7f32d0e79700 1 -- 172.21.15.121:0/309354320 wait complete. 2023-11-15T03:04:03.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.220+0000 7f32d0e79700 1 Processor -- start 2023-11-15T03:04:03.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.220+0000 7f32d0e79700 1 -- start start 2023-11-15T03:04:03.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32d0e79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32cc066320 0x7f32cc0c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:03.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32d0e79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32cc066700 0x7f32cc0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:03.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32d0e79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32cc0acf10 0x7f32cc0a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:03.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32d0e79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc0a7110 con 0x7f32cc066700 2023-11-15T03:04:03.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32d0e79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc0a6500 con 0x7f32cc0acf10 2023-11-15T03:04:03.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32d0e79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc0a67e0 con 0x7f32cc066320 2023-11-15T03:04:03.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32caffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32cc066700 0x7f32cc0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:03.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32caffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32cc066700 0x7f32cc0a5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38562/0 (socket says 172.21.15.121:38562) 2023-11-15T03:04:03.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32caffd700 1 -- 172.21.15.121:0/2271073275 learned_addr learned my addr 172.21.15.121:0/2271073275 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:03.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32cbfff700 1 --2- 172.21.15.121:0/2271073275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32cc0acf10 0x7f32cc0a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:03.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32cb7fe700 1 --2- 172.21.15.121:0/2271073275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32cc066320 0x7f32cc0c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:03.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32caffd700 1 -- 172.21.15.121:0/2271073275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32cc066320 msgr2=0x7f32cc0c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:03.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32caffd700 1 --2- 172.21.15.121:0/2271073275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32cc066320 0x7f32cc0c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:03.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32caffd700 1 -- 172.21.15.121:0/2271073275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32cc0acf10 msgr2=0x7f32cc0a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:03.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32caffd700 1 --2- 172.21.15.121:0/2271073275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32cc0acf10 0x7f32cc0a5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:03.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32caffd700 1 -- 172.21.15.121:0/2271073275 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c00092a0 con 0x7f32cc066700 2023-11-15T03:04:03.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32caffd700 1 --2- 172.21.15.121:0/2271073275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32cc066700 0x7f32cc0a5a10 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7f32cc064af0 tx=0x7f32c0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:03.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32b37fe700 1 -- 172.21.15.121:0/2271073275 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0000c90 con 0x7f32cc066700 2023-11-15T03:04:03.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32b37fe700 1 -- 172.21.15.121:0/2271073275 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c000d770 con 0x7f32cc066700 2023-11-15T03:04:03.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32d0e79700 1 -- 172.21.15.121:0/2271073275 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32cc0a6ac0 con 0x7f32cc066700 2023-11-15T03:04:03.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32b37fe700 1 -- 172.21.15.121:0/2271073275 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c001d640 con 0x7f32cc066700 2023-11-15T03:04:03.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.224+0000 7f32d0e79700 1 -- 172.21.15.121:0/2271073275 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32cc0a75a0 con 0x7f32cc066700 2023-11-15T03:04:03.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.228+0000 7f32d0e79700 1 -- 172.21.15.121:0/2271073275 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32cc0a7280 con 0x7f32cc066700 2023-11-15T03:04:03.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.228+0000 7f32b37fe700 1 -- 172.21.15.121:0/2271073275 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f32c001da70 con 0x7f32cc066700 2023-11-15T03:04:03.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.228+0000 7f32b37fe700 1 --2- 172.21.15.121:0/2271073275 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f32b805a510 0x7f32b805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:03.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.228+0000 7f32b37fe700 1 -- 172.21.15.121:0/2271073275 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6610+0+0 (secure 0 0 0) 0x7f32c00761d0 con 0x7f32cc066700 2023-11-15T03:04:03.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.228+0000 7f32cb7fe700 1 --2- 172.21.15.121:0/2271073275 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f32b805a510 0x7f32b805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:03.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.228+0000 7f32cb7fe700 1 --2- 172.21.15.121:0/2271073275 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f32b805a510 0x7f32b805c9d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f32bc005750 tx=0x7f32bc009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:03.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.236+0000 7f32b37fe700 1 -- 172.21.15.121:0/2271073275 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f32c00497c0 con 0x7f32cc066700 2023-11-15T03:04:03.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.440+0000 7f32d0e79700 1 -- 172.21.15.121:0/2271073275 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32cc154f60 con 0x7f32cc066700 2023-11-15T03:04:03.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.444+0000 7f32b37fe700 1 -- 172.21.15.121:0/2271073275 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4504 (secure 0 0 0) 0x7f32c004cde0 con 0x7f32cc066700 2023-11-15T03:04:03.450 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:04:03.450 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":102,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:04:03.009087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7704":{"gid":7704,"name":"a","rank":0,"incarnation":102,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/3764682615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3764682615},{"type":"v1","addr":"172.21.15.121:6836","nonce":3764682615}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:04:03.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.448+0000 7f32d0e79700 1 -- 172.21.15.121:0/2271073275 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f32b805a510 msgr2=0x7f32b805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:03.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.448+0000 7f32d0e79700 1 --2- 172.21.15.121:0/2271073275 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f32b805a510 0x7f32b805c9d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f32bc005750 tx=0x7f32bc009040).stop 2023-11-15T03:04:03.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.448+0000 7f32d0e79700 1 -- 172.21.15.121:0/2271073275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32cc066700 msgr2=0x7f32cc0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:03.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.448+0000 7f32d0e79700 1 --2- 172.21.15.121:0/2271073275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32cc066700 0x7f32cc0a5a10 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7f32cc064af0 tx=0x7f32c0016000).stop 2023-11-15T03:04:03.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.448+0000 7f32d0e79700 1 -- 172.21.15.121:0/2271073275 shutdown_connections 2023-11-15T03:04:03.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.448+0000 7f32d0e79700 1 --2- 172.21.15.121:0/2271073275 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f32b805a510 0x7f32b805c9d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:03.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.448+0000 7f32d0e79700 1 --2- 172.21.15.121:0/2271073275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32cc0acf10 0x7f32cc0a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:03.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.448+0000 7f32d0e79700 1 --2- 172.21.15.121:0/2271073275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32cc066700 0x7f32cc0a5a10 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:03.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.448+0000 7f32d0e79700 1 --2- 172.21.15.121:0/2271073275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32cc066320 0x7f32cc0c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:03.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.448+0000 7f32d0e79700 1 -- 172.21.15.121:0/2271073275 >> 172.21.15.121:0/2271073275 conn(0x7f32cc067a60 msgr2=0x7f32cc0aaba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:03.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.448+0000 7f32d0e79700 1 -- 172.21.15.121:0/2271073275 shutdown_connections 2023-11-15T03:04:03.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:03.448+0000 7f32d0e79700 1 -- 172.21.15.121:0/2271073275 wait complete. 2023-11-15T03:04:03.457 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 102 2023-11-15T03:04:03.476 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:04:04.024 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:04:04.016+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:04:04.024 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:04:04.016+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:04:04.037 INFO:tasks.ceph.mds.a.smithi121.stderr: -17> 2023-11-15T03:04:03.024+0000 7f989baae700 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2023-11-15T03:04:04.037 INFO:tasks.ceph.mds.a.smithi121.stderr: -15> 2023-11-15T03:04:03.024+0000 7f989baae700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2023-11-15T03:04:04.081 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:04:08.477 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:04:08.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.636+0000 7fdb8b59e700 1 Processor -- start 2023-11-15T03:04:08.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.636+0000 7fdb8b59e700 1 -- start start 2023-11-15T03:04:08.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.636+0000 7fdb8b59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdb8c0cafb0 0x7fdb8c066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:08.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.636+0000 7fdb8b59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdb8c067270 0x7fdb8c0ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:08.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.636+0000 7fdb8b59e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8c0ad3a0 con 0x7fdb8c0cabd0 2023-11-15T03:04:08.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.636+0000 7fdb8b59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb8c0cb410 con 0x7fdb8c067270 2023-11-15T03:04:08.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.636+0000 7fdb8b59e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb8c0ad650 con 0x7fdb8c0cafb0 2023-11-15T03:04:08.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.636+0000 7fdb8ad9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdb8c067270 0x7fdb8c0ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:08.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.636+0000 7fdb89d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdb8c0cafb0 0x7fdb8c066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:08.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.636+0000 7fdb8ad9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdb8c067270 0x7fdb8c0ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:52836/0 (socket says 172.21.15.121:52836) 2023-11-15T03:04:08.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.636+0000 7fdb8ad9d700 1 -- 172.21.15.121:0/1575289162 learned_addr learned my addr 172.21.15.121:0/1575289162 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:08.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.640+0000 7fdb8959a700 1 -- 172.21.15.121:0/1575289162 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296991504 0 0) 0x7fdb8c0ad3a0 con 0x7fdb8c0cabd0 2023-11-15T03:04:08.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.640+0000 7fdb8959a700 1 -- 172.21.15.121:0/1575289162 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb740033f0 con 0x7fdb8c0cabd0 2023-11-15T03:04:08.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.640+0000 7fdb8ad9d700 1 -- 172.21.15.121:0/1575289162 >> v2:172.21.15.190:3300/0 conn(0x7fdb8c0cafb0 msgr2=0x7fdb8c066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:08.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.640+0000 7fdb8ad9d700 1 --2- 172.21.15.121:0/1575289162 >> v2:172.21.15.190:3300/0 conn(0x7fdb8c0cafb0 0x7fdb8c066d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:08.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.640+0000 7fdb8ad9d700 1 -- 172.21.15.121:0/1575289162 >> v1:172.21.15.190:6789/0 conn(0x7fdb8c0cabd0 legacy=0x7fdb8c0acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:08.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.640+0000 7fdb8ad9d700 1 -- 172.21.15.121:0/1575289162 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c065fc0 con 0x7fdb8c067270 2023-11-15T03:04:08.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.640+0000 7fdb8ad9d700 1 --2- 172.21.15.121:0/1575289162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c067270 0x7fdb8c0ae190 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7fdb8c0c8860 tx=0x7fdb84009840).ready entity=mon.0 client_cookie=db579c6d3c6583d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:08.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.640+0000 7fdb8959a700 1 -- 172.21.15.121:0/1575289162 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8400e070 con 0x7fdb8c067270 2023-11-15T03:04:08.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.640+0000 7fdb8959a700 1 -- 172.21.15.121:0/1575289162 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8400b970 con 0x7fdb8c067270 2023-11-15T03:04:08.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.640+0000 7fdb8959a700 1 -- 172.21.15.121:0/1575289162 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8400bcf0 con 0x7fdb8c067270 2023-11-15T03:04:08.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.640+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1575289162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c067270 msgr2=0x7fdb8c0ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:08.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.640+0000 7fdb8b59e700 1 --2- 172.21.15.121:0/1575289162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c067270 0x7fdb8c0ae190 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7fdb8c0c8860 tx=0x7fdb84009840).stop 2023-11-15T03:04:08.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.640+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1575289162 shutdown_connections 2023-11-15T03:04:08.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.640+0000 7fdb8b59e700 1 --2- 172.21.15.121:0/1575289162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c067270 0x7fdb8c0ae190 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:08.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.640+0000 7fdb8b59e700 1 --2- 172.21.15.121:0/1575289162 >> v2:172.21.15.190:3300/0 conn(0x7fdb8c0cafb0 0x7fdb8c066d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:08.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.640+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1575289162 >> 172.21.15.121:0/1575289162 conn(0x7fdb8c062190 msgr2=0x7fdb8c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:08.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.640+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1575289162 shutdown_connections 2023-11-15T03:04:08.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.640+0000 7fdb8b59e700 1 -- 172.21.15.121:0/1575289162 wait complete. 2023-11-15T03:04:08.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb8b59e700 1 Processor -- start 2023-11-15T03:04:08.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb8b59e700 1 -- start start 2023-11-15T03:04:08.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb8b59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb8c067270 0x7fdb8c1498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:08.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb8b59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c0cabd0 0x7fdb8c14a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:08.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb8b59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb8c0cafb0 0x7fdb8c14e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:08.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb8b59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c151a20 con 0x7fdb8c0cabd0 2023-11-15T03:04:08.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb8b59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c149de0 con 0x7fdb8c067270 2023-11-15T03:04:08.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb8b59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c14a600 con 0x7fdb8c0cafb0 2023-11-15T03:04:08.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb89d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c0cabd0 0x7fdb8c14a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:08.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb89d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c0cabd0 0x7fdb8c14a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:52844/0 (socket says 172.21.15.121:52844) 2023-11-15T03:04:08.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb89d9b700 1 -- 172.21.15.121:0/2932967437 learned_addr learned my addr 172.21.15.121:0/2932967437 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:08.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb8a59c700 1 --2- 172.21.15.121:0/2932967437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb8c067270 0x7fdb8c1498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:08.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb8ad9d700 1 --2- 172.21.15.121:0/2932967437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb8c0cafb0 0x7fdb8c14e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:08.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb89d9b700 1 -- 172.21.15.121:0/2932967437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb8c0cafb0 msgr2=0x7fdb8c14e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:08.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb89d9b700 1 --2- 172.21.15.121:0/2932967437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb8c0cafb0 0x7fdb8c14e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:08.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb89d9b700 1 -- 172.21.15.121:0/2932967437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb8c067270 msgr2=0x7fdb8c1498f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:08.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb89d9b700 1 --2- 172.21.15.121:0/2932967437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb8c067270 0x7fdb8c1498f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:08.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb89d9b700 1 -- 172.21.15.121:0/2932967437 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb84009ae0 con 0x7fdb8c0cabd0 2023-11-15T03:04:08.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb89d9b700 1 --2- 172.21.15.121:0/2932967437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c0cabd0 0x7fdb8c14a110 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7fdb8000e900 tx=0x7fdb80009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:08.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb7a7fc700 1 -- 172.21.15.121:0/2932967437 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8000c640 con 0x7fdb8c0cabd0 2023-11-15T03:04:08.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb7a7fc700 1 -- 172.21.15.121:0/2932967437 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8000c7a0 con 0x7fdb8c0cabd0 2023-11-15T03:04:08.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb8b59e700 1 -- 172.21.15.121:0/2932967437 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8c153680 con 0x7fdb8c0cabd0 2023-11-15T03:04:08.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb7a7fc700 1 -- 172.21.15.121:0/2932967437 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8000cb20 con 0x7fdb8c0cabd0 2023-11-15T03:04:08.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.644+0000 7fdb8b59e700 1 -- 172.21.15.121:0/2932967437 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8c153c40 con 0x7fdb8c0cabd0 2023-11-15T03:04:08.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.648+0000 7fdb8b59e700 1 -- 172.21.15.121:0/2932967437 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb8c06bcf0 con 0x7fdb8c0cabd0 2023-11-15T03:04:08.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.648+0000 7fdb7a7fc700 1 -- 172.21.15.121:0/2932967437 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fdb8001b400 con 0x7fdb8c0cabd0 2023-11-15T03:04:08.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.652+0000 7fdb7a7fc700 1 --2- 172.21.15.121:0/2932967437 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdb6c05a4e0 0x7fdb6c05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:08.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.652+0000 7fdb7a7fc700 1 -- 172.21.15.121:0/2932967437 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6610+0+0 (secure 0 0 0) 0x7fdb80074e50 con 0x7fdb8c0cabd0 2023-11-15T03:04:08.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.652+0000 7fdb8a59c700 1 --2- 172.21.15.121:0/2932967437 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdb6c05a4e0 0x7fdb6c05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:08.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.652+0000 7fdb8a59c700 1 --2- 172.21.15.121:0/2932967437 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdb6c05a4e0 0x7fdb6c05c9a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fdb7c003700 tx=0x7fdb7c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:08.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.656+0000 7fdb7a7fc700 1 -- 172.21.15.121:0/2932967437 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdb80048320 con 0x7fdb8c0cabd0 2023-11-15T03:04:08.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.848+0000 7fdb8b59e700 1 -- 172.21.15.121:0/2932967437 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb8c0662a0 con 0x7fdb8c0cabd0 2023-11-15T03:04:08.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.852+0000 7fdb7a7fc700 1 -- 172.21.15.121:0/2932967437 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4489 (secure 0 0 0) 0x7fdb80016030 con 0x7fdb8c0cabd0 2023-11-15T03:04:08.859 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:04:08.859 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":104,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7770,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/57668150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":57668150},{"type":"v1","addr":"172.21.15.121:6836","nonce":57668150}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:04:04.020764+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:04:08.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.856+0000 7fdb8b59e700 1 -- 172.21.15.121:0/2932967437 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdb6c05a4e0 msgr2=0x7fdb6c05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:08.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.856+0000 7fdb8b59e700 1 --2- 172.21.15.121:0/2932967437 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdb6c05a4e0 0x7fdb6c05c9a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fdb7c003700 tx=0x7fdb7c009040).stop 2023-11-15T03:04:08.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.856+0000 7fdb8b59e700 1 -- 172.21.15.121:0/2932967437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c0cabd0 msgr2=0x7fdb8c14a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:08.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.856+0000 7fdb8b59e700 1 --2- 172.21.15.121:0/2932967437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c0cabd0 0x7fdb8c14a110 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7fdb8000e900 tx=0x7fdb80009060).stop 2023-11-15T03:04:08.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.856+0000 7fdb8b59e700 1 -- 172.21.15.121:0/2932967437 shutdown_connections 2023-11-15T03:04:08.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.856+0000 7fdb8b59e700 1 --2- 172.21.15.121:0/2932967437 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdb6c05a4e0 0x7fdb6c05c9a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:08.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.856+0000 7fdb8b59e700 1 --2- 172.21.15.121:0/2932967437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb8c0cafb0 0x7fdb8c14e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:08.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.856+0000 7fdb8b59e700 1 --2- 172.21.15.121:0/2932967437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c0cabd0 0x7fdb8c14a110 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:08.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.856+0000 7fdb8b59e700 1 --2- 172.21.15.121:0/2932967437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb8c067270 0x7fdb8c1498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:08.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.856+0000 7fdb8b59e700 1 -- 172.21.15.121:0/2932967437 >> 172.21.15.121:0/2932967437 conn(0x7fdb8c062190 msgr2=0x7fdb8c0c70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:08.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.856+0000 7fdb8b59e700 1 -- 172.21.15.121:0/2932967437 shutdown_connections 2023-11-15T03:04:08.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:08.856+0000 7fdb8b59e700 1 -- 172.21.15.121:0/2932967437 wait complete. 2023-11-15T03:04:08.868 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 104 2023-11-15T03:04:08.882 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:04:08.883 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:04:09.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.036+0000 7f6d917db700 1 Processor -- start 2023-11-15T03:04:09.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.036+0000 7f6d917db700 1 -- start start 2023-11-15T03:04:09.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.040+0000 7f6d917db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6d8c0aa240 0x7f6d8c067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.040+0000 7f6d917db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8c0cb5c0 con 0x7f6d8c067f50 2023-11-15T03:04:09.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.040+0000 7f6d917db700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8c0c9ca0 con 0x7f6d8c153bb0 2023-11-15T03:04:09.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.040+0000 7f6d917db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d8c0aa6f0 con 0x7f6d8c0aa240 2023-11-15T03:04:09.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.040+0000 7f6d8b7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6d8c067f50 0x7f6d8c065d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53422/0 (socket says 172.21.15.121:53422) 2023-11-15T03:04:09.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.040+0000 7f6d8b7fe700 1 -- 172.21.15.121:0/2922171628 learned_addr learned my addr 172.21.15.121:0/2922171628 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:09.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.040+0000 7f6d83fff700 1 --2- 172.21.15.121:0/2922171628 >> v2:172.21.15.121:3300/0 conn(0x7f6d8c0aa240 0x7f6d8c067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.040+0000 7f6d8a7fc700 1 -- 172.21.15.121:0/2922171628 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1126017960 0 0) 0x7f6d8c0cb5c0 con 0x7f6d8c067f50 2023-11-15T03:04:09.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.040+0000 7f6d8a7fc700 1 -- 172.21.15.121:0/2922171628 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d780033f0 con 0x7f6d8c067f50 2023-11-15T03:04:09.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.040+0000 7f6d8a7fc700 1 -- 172.21.15.121:0/2922171628 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024110924 0 0) 0x7f6d8c0c9ca0 con 0x7f6d8c153bb0 2023-11-15T03:04:09.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.040+0000 7f6d8a7fc700 1 -- 172.21.15.121:0/2922171628 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d8c0cb5c0 con 0x7f6d8c153bb0 2023-11-15T03:04:09.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.040+0000 7f6d8a7fc700 1 -- 172.21.15.121:0/2922171628 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1991315562 0 0) 0x7f6d780033f0 con 0x7f6d8c067f50 2023-11-15T03:04:09.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.040+0000 7f6d83fff700 1 -- 172.21.15.121:0/2922171628 >> v1:172.21.15.190:6789/0 conn(0x7f6d8c153bb0 legacy=0x7f6d8c0a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:09.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.040+0000 7f6d83fff700 1 -- 172.21.15.121:0/2922171628 >> v1:172.21.15.121:6789/0 conn(0x7f6d8c067f50 legacy=0x7f6d8c065d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:09.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.040+0000 7f6d83fff700 1 -- 172.21.15.121:0/2922171628 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c0caa20 con 0x7f6d8c0aa240 2023-11-15T03:04:09.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.044+0000 7f6d83fff700 1 --2- 172.21.15.121:0/2922171628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d8c0aa240 0x7f6d8c067a60 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7f6d7c009760 tx=0x7f6d7c009060).ready entity=mon.0 client_cookie=df7e3814b1326acf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:09.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.044+0000 7f6d8a7fc700 1 -- 172.21.15.121:0/2922171628 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d840017f0 con 0x7f6d8c0aa240 2023-11-15T03:04:09.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.044+0000 7f6d8a7fc700 1 -- 172.21.15.121:0/2922171628 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7c011400 con 0x7f6d8c0aa240 2023-11-15T03:04:09.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.044+0000 7f6d8a7fc700 1 -- 172.21.15.121:0/2922171628 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c011780 con 0x7f6d8c0aa240 2023-11-15T03:04:09.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.044+0000 7f6d917db700 1 -- 172.21.15.121:0/2922171628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d8c0aa240 msgr2=0x7f6d8c067a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:09.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.044+0000 7f6d917db700 1 --2- 172.21.15.121:0/2922171628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d8c0aa240 0x7f6d8c067a60 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7f6d7c009760 tx=0x7f6d7c009060).stop 2023-11-15T03:04:09.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.044+0000 7f6d917db700 1 -- 172.21.15.121:0/2922171628 shutdown_connections 2023-11-15T03:04:09.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.044+0000 7f6d917db700 1 --2- 172.21.15.121:0/2922171628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d8c0aa240 0x7f6d8c067a60 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.044+0000 7f6d917db700 1 -- 172.21.15.121:0/2922171628 >> 172.21.15.121:0/2922171628 conn(0x7f6d8c058c50 msgr2=0x7f6d8c059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:09.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.044+0000 7f6d917db700 1 -- 172.21.15.121:0/2922171628 shutdown_connections 2023-11-15T03:04:09.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.044+0000 7f6d917db700 1 -- 172.21.15.121:0/2922171628 wait complete. 2023-11-15T03:04:09.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d917db700 1 Processor -- start 2023-11-15T03:04:09.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d917db700 1 -- start start 2023-11-15T03:04:09.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d917db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d8c067f50 0x7f6d8c14e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d917db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d8c0aa240 0x7f6d8c155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d917db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d8c153bb0 0x7f6d8c155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d917db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c155e10 con 0x7f6d8c0aa240 2023-11-15T03:04:09.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d917db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c164840 con 0x7f6d8c067f50 2023-11-15T03:04:09.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d917db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c0c4190 con 0x7f6d8c153bb0 2023-11-15T03:04:09.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d83fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d8c0aa240 0x7f6d8c155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d83fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d8c0aa240 0x7f6d8c155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:52870/0 (socket says 172.21.15.121:52870) 2023-11-15T03:04:09.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d83fff700 1 -- 172.21.15.121:0/1009000405 learned_addr learned my addr 172.21.15.121:0/1009000405 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:09.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d8b7fe700 1 --2- 172.21.15.121:0/1009000405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d8c153bb0 0x7f6d8c155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d8affd700 1 --2- 172.21.15.121:0/1009000405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d8c067f50 0x7f6d8c14e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d83fff700 1 -- 172.21.15.121:0/1009000405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d8c153bb0 msgr2=0x7f6d8c155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:09.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d83fff700 1 --2- 172.21.15.121:0/1009000405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d8c153bb0 0x7f6d8c155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d83fff700 1 -- 172.21.15.121:0/1009000405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d8c067f50 msgr2=0x7f6d8c14e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:09.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d83fff700 1 --2- 172.21.15.121:0/1009000405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d8c067f50 0x7f6d8c14e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d83fff700 1 -- 172.21.15.121:0/1009000405 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c0092a0 con 0x7f6d8c0aa240 2023-11-15T03:04:09.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d83fff700 1 --2- 172.21.15.121:0/1009000405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d8c0aa240 0x7f6d8c155430 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7f6d8c065560 tx=0x7f6d7c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:09.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d82ffd700 1 -- 172.21.15.121:0/1009000405 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c0108b0 con 0x7f6d8c0aa240 2023-11-15T03:04:09.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d82ffd700 1 -- 172.21.15.121:0/1009000405 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7c010a10 con 0x7f6d8c0aa240 2023-11-15T03:04:09.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d82ffd700 1 -- 172.21.15.121:0/1009000405 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c010d90 con 0x7f6d8c0aa240 2023-11-15T03:04:09.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d917db700 1 -- 172.21.15.121:0/1009000405 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8c0c4470 con 0x7f6d8c0aa240 2023-11-15T03:04:09.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.048+0000 7f6d917db700 1 -- 172.21.15.121:0/1009000405 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8c0c4a30 con 0x7f6d8c0aa240 2023-11-15T03:04:09.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.052+0000 7f6d917db700 1 -- 172.21.15.121:0/1009000405 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8c06c150 con 0x7f6d8c0aa240 2023-11-15T03:04:09.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.052+0000 7f6d82ffd700 1 -- 172.21.15.121:0/1009000405 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6d7c01d640 con 0x7f6d8c0aa240 2023-11-15T03:04:09.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.052+0000 7f6d82ffd700 1 --2- 172.21.15.121:0/1009000405 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6d6c05a510 0x7f6d6c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.056+0000 7f6d82ffd700 1 -- 172.21.15.121:0/1009000405 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6610+0+0 (secure 0 0 0) 0x7f6d7c075f80 con 0x7f6d8c0aa240 2023-11-15T03:04:09.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.056+0000 7f6d8affd700 1 --2- 172.21.15.121:0/1009000405 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6d6c05a510 0x7f6d6c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.056+0000 7f6d8affd700 1 --2- 172.21.15.121:0/1009000405 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6d6c05a510 0x7f6d6c05c9d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f6d74003660 tx=0x7f6d74009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:09.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.060+0000 7f6d82ffd700 1 -- 172.21.15.121:0/1009000405 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6d7c0a4af0 con 0x7f6d8c0aa240 2023-11-15T03:04:09.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.256+0000 7f6d917db700 1 -- 172.21.15.121:0/1009000405 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d8c0c5d20 con 0x7f6d8c0aa240 2023-11-15T03:04:09.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.260+0000 7f6d82ffd700 1 -- 172.21.15.121:0/1009000405 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4489 (secure 0 0 0) 0x7f6d7c049570 con 0x7f6d8c0aa240 2023-11-15T03:04:09.265 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:04:09.265 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":104,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7770,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/57668150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":57668150},{"type":"v1","addr":"172.21.15.121:6836","nonce":57668150}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:04:04.020764+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:04:09.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.260+0000 7f6d917db700 1 -- 172.21.15.121:0/1009000405 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6d6c05a510 msgr2=0x7f6d6c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:09.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.260+0000 7f6d917db700 1 --2- 172.21.15.121:0/1009000405 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6d6c05a510 0x7f6d6c05c9d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f6d74003660 tx=0x7f6d74009040).stop 2023-11-15T03:04:09.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.260+0000 7f6d917db700 1 -- 172.21.15.121:0/1009000405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d8c0aa240 msgr2=0x7f6d8c155430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:09.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.260+0000 7f6d917db700 1 --2- 172.21.15.121:0/1009000405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d8c0aa240 0x7f6d8c155430 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7f6d8c065560 tx=0x7f6d7c016000).stop 2023-11-15T03:04:09.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.260+0000 7f6d917db700 1 -- 172.21.15.121:0/1009000405 shutdown_connections 2023-11-15T03:04:09.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.260+0000 7f6d917db700 1 --2- 172.21.15.121:0/1009000405 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6d6c05a510 0x7f6d6c05c9d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.260+0000 7f6d917db700 1 --2- 172.21.15.121:0/1009000405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d8c153bb0 0x7f6d8c155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.260+0000 7f6d917db700 1 --2- 172.21.15.121:0/1009000405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d8c0aa240 0x7f6d8c155430 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.260+0000 7f6d917db700 1 --2- 172.21.15.121:0/1009000405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d8c067f50 0x7f6d8c14e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.260+0000 7f6d917db700 1 -- 172.21.15.121:0/1009000405 >> 172.21.15.121:0/1009000405 conn(0x7f6d8c058c50 msgr2=0x7f6d8c0b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:09.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.260+0000 7f6d917db700 1 -- 172.21.15.121:0/1009000405 shutdown_connections 2023-11-15T03:04:09.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.260+0000 7f6d917db700 1 -- 172.21.15.121:0/1009000405 wait complete. 2023-11-15T03:04:09.273 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 104 2023-11-15T03:04:09.288 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:04:09.288 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:04:09.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.444+0000 7f1d8f0db700 1 Processor -- start 2023-11-15T03:04:09.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.444+0000 7f1d8f0db700 1 -- start start 2023-11-15T03:04:09.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.448+0000 7f1d8f0db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1d88153a10 0x7f1d880a8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.448+0000 7f1d8f0db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d880a93d0 0x7f1d88157830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.448+0000 7f1d8f0db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1d88153df0 0x7f1d88157d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.448+0000 7f1d8f0db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d88158210 con 0x7f1d88153a10 2023-11-15T03:04:09.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.448+0000 7f1d8f0db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d880c3a30 con 0x7f1d88153df0 2023-11-15T03:04:09.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.448+0000 7f1d8f0db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880c7a20 con 0x7f1d880a93d0 2023-11-15T03:04:09.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.448+0000 7f1d8ce77700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1d88153a10 0x7f1d880a8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.448+0000 7f1d8ce77700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1d88153a10 0x7f1d880a8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:52878/0 (socket says 172.21.15.121:52878) 2023-11-15T03:04:09.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.448+0000 7f1d8ce77700 1 -- 172.21.15.121:0/2413682805 learned_addr learned my addr 172.21.15.121:0/2413682805 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:09.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.448+0000 7f1d8d678700 1 --2- 172.21.15.121:0/2413682805 >> v2:172.21.15.190:3300/0 conn(0x7f1d88153df0 0x7f1d88157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.448+0000 7f1d87fff700 1 --2- 172.21.15.121:0/2413682805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d880a93d0 0x7f1d88157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.448+0000 7f1d8ce77700 1 -- 172.21.15.121:0/2413682805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d880a93d0 msgr2=0x7f1d88157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:09.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.448+0000 7f1d8ce77700 1 --2- 172.21.15.121:0/2413682805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d880a93d0 0x7f1d88157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.448+0000 7f1d8ce77700 1 -- 172.21.15.121:0/2413682805 >> v2:172.21.15.190:3300/0 conn(0x7f1d88153df0 msgr2=0x7f1d88157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:09.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.448+0000 7f1d8ce77700 1 --2- 172.21.15.121:0/2413682805 >> v2:172.21.15.190:3300/0 conn(0x7f1d88153df0 0x7f1d88157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.448+0000 7f1d8ce77700 1 -- 172.21.15.121:0/2413682805 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d880c7d00 con 0x7f1d88153a10 2023-11-15T03:04:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.448+0000 7f1d8d678700 1 --2- 172.21.15.121:0/2413682805 >> v2:172.21.15.190:3300/0 conn(0x7f1d88153df0 0x7f1d88157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.452+0000 7f1d8ce77700 1 --2- 172.21.15.121:0/2413682805 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d88153a10 0x7f1d880a8ee0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7f1d88069ba0 tx=0x7f1d7c0090f0).ready entity=mon.0 client_cookie=448ef71a79c74a15 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.452+0000 7f1d877fe700 1 -- 172.21.15.121:0/2413682805 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c00b600 con 0x7f1d88153a10 2023-11-15T03:04:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.452+0000 7f1d877fe700 1 -- 172.21.15.121:0/2413682805 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7c00b760 con 0x7f1d88153a10 2023-11-15T03:04:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.452+0000 7f1d877fe700 1 -- 172.21.15.121:0/2413682805 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c00bae0 con 0x7f1d88153a10 2023-11-15T03:04:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.452+0000 7f1d8f0db700 1 -- 172.21.15.121:0/2413682805 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d88153a10 msgr2=0x7f1d880a8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.452+0000 7f1d8f0db700 1 --2- 172.21.15.121:0/2413682805 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d88153a10 0x7f1d880a8ee0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7f1d88069ba0 tx=0x7f1d7c0090f0).stop 2023-11-15T03:04:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.452+0000 7f1d8f0db700 1 -- 172.21.15.121:0/2413682805 shutdown_connections 2023-11-15T03:04:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.452+0000 7f1d8f0db700 1 --2- 172.21.15.121:0/2413682805 >> v2:172.21.15.190:3300/0 conn(0x7f1d88153df0 0x7f1d88157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.452+0000 7f1d8f0db700 1 --2- 172.21.15.121:0/2413682805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d880a93d0 0x7f1d88157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.452+0000 7f1d8f0db700 1 --2- 172.21.15.121:0/2413682805 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d88153a10 0x7f1d880a8ee0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.452+0000 7f1d8f0db700 1 -- 172.21.15.121:0/2413682805 >> 172.21.15.121:0/2413682805 conn(0x7f1d880c8960 msgr2=0x7f1d880c8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:09.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.452+0000 7f1d8f0db700 1 -- 172.21.15.121:0/2413682805 shutdown_connections 2023-11-15T03:04:09.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.452+0000 7f1d8f0db700 1 -- 172.21.15.121:0/2413682805 wait complete. 2023-11-15T03:04:09.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.452+0000 7f1d8f0db700 1 Processor -- start 2023-11-15T03:04:09.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.452+0000 7f1d8f0db700 1 -- start start 2023-11-15T03:04:09.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d8f0db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d880a93d0 0x7f1d880c2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d8f0db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d88153a10 0x7f1d880bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d8f0db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d88153df0 0x7f1d8814d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d8f0db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880c0080 con 0x7f1d880a93d0 2023-11-15T03:04:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d8f0db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d88168780 con 0x7f1d88153df0 2023-11-15T03:04:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d8f0db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d88155230 con 0x7f1d88153a10 2023-11-15T03:04:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d8ce77700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d880a93d0 0x7f1d880c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d8ce77700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d880a93d0 0x7f1d880c2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:52886/0 (socket says 172.21.15.121:52886) 2023-11-15T03:04:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d8ce77700 1 -- 172.21.15.121:0/460757066 learned_addr learned my addr 172.21.15.121:0/460757066 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d87fff700 1 --2- 172.21.15.121:0/460757066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d88153a10 0x7f1d880bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d8d678700 1 --2- 172.21.15.121:0/460757066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d88153df0 0x7f1d8814d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d8ce77700 1 -- 172.21.15.121:0/460757066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d88153a10 msgr2=0x7f1d880bfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:09.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d8ce77700 1 --2- 172.21.15.121:0/460757066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d88153a10 0x7f1d880bfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d8ce77700 1 -- 172.21.15.121:0/460757066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d88153df0 msgr2=0x7f1d8814d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:09.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d8ce77700 1 --2- 172.21.15.121:0/460757066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d88153df0 0x7f1d8814d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d8ce77700 1 -- 172.21.15.121:0/460757066 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c009370 con 0x7f1d880a93d0 2023-11-15T03:04:09.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d8ce77700 1 --2- 172.21.15.121:0/460757066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d880a93d0 0x7f1d880c2580 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7f1d7c006fd0 tx=0x7f1d7c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:09.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d84ff9700 1 -- 172.21.15.121:0/460757066 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c00b8d0 con 0x7f1d880a93d0 2023-11-15T03:04:09.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d84ff9700 1 -- 172.21.15.121:0/460757066 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7c00ba30 con 0x7f1d880a93d0 2023-11-15T03:04:09.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d8f0db700 1 -- 172.21.15.121:0/460757066 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d88155ad0 con 0x7f1d880a93d0 2023-11-15T03:04:09.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d84ff9700 1 -- 172.21.15.121:0/460757066 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c01b660 con 0x7f1d880a93d0 2023-11-15T03:04:09.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.456+0000 7f1d8f0db700 1 -- 172.21.15.121:0/460757066 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d88155480 con 0x7f1d880a93d0 2023-11-15T03:04:09.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.460+0000 7f1d8f0db700 1 -- 172.21.15.121:0/460757066 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d8806c150 con 0x7f1d880a93d0 2023-11-15T03:04:09.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.460+0000 7f1d84ff9700 1 -- 172.21.15.121:0/460757066 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1d7c01ba10 con 0x7f1d880a93d0 2023-11-15T03:04:09.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.460+0000 7f1d84ff9700 1 --2- 172.21.15.121:0/460757066 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d6c05a510 0x7f1d6c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.460+0000 7f1d84ff9700 1 -- 172.21.15.121:0/460757066 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6610+0+0 (secure 0 0 0) 0x7f1d7c076270 con 0x7f1d880a93d0 2023-11-15T03:04:09.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.460+0000 7f1d87fff700 1 --2- 172.21.15.121:0/460757066 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d6c05a510 0x7f1d6c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.460+0000 7f1d87fff700 1 --2- 172.21.15.121:0/460757066 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d6c05a510 0x7f1d6c05c9d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f1d881574a0 tx=0x7f1d74008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:09.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.468+0000 7f1d84ff9700 1 -- 172.21.15.121:0/460757066 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1d7c0497a0 con 0x7f1d880a93d0 2023-11-15T03:04:09.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.664+0000 7f1d8f0db700 1 -- 172.21.15.121:0/460757066 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d880ac040 con 0x7f1d880a93d0 2023-11-15T03:04:09.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.668+0000 7f1d84ff9700 1 -- 172.21.15.121:0/460757066 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4489 (secure 0 0 0) 0x7f1d7c029020 con 0x7f1d880a93d0 2023-11-15T03:04:09.674 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:04:09.674 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":104,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7770,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/57668150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":57668150},{"type":"v1","addr":"172.21.15.121:6836","nonce":57668150}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:04:04.020764+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:04:09.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.668+0000 7f1d8f0db700 1 -- 172.21.15.121:0/460757066 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d6c05a510 msgr2=0x7f1d6c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:09.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.668+0000 7f1d8f0db700 1 --2- 172.21.15.121:0/460757066 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d6c05a510 0x7f1d6c05c9d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f1d881574a0 tx=0x7f1d74008040).stop 2023-11-15T03:04:09.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.668+0000 7f1d8f0db700 1 -- 172.21.15.121:0/460757066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d880a93d0 msgr2=0x7f1d880c2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:09.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.668+0000 7f1d8f0db700 1 --2- 172.21.15.121:0/460757066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d880a93d0 0x7f1d880c2580 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7f1d7c006fd0 tx=0x7f1d7c017000).stop 2023-11-15T03:04:09.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.668+0000 7f1d8f0db700 1 -- 172.21.15.121:0/460757066 shutdown_connections 2023-11-15T03:04:09.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.668+0000 7f1d8f0db700 1 --2- 172.21.15.121:0/460757066 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d6c05a510 0x7f1d6c05c9d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.668+0000 7f1d8f0db700 1 --2- 172.21.15.121:0/460757066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d88153df0 0x7f1d8814d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.668+0000 7f1d8f0db700 1 --2- 172.21.15.121:0/460757066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d88153a10 0x7f1d880bfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.668+0000 7f1d8f0db700 1 --2- 172.21.15.121:0/460757066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d880a93d0 0x7f1d880c2580 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.668+0000 7f1d8f0db700 1 -- 172.21.15.121:0/460757066 >> 172.21.15.121:0/460757066 conn(0x7f1d880c8960 msgr2=0x7f1d88152640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:09.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.668+0000 7f1d8f0db700 1 -- 172.21.15.121:0/460757066 shutdown_connections 2023-11-15T03:04:09.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.668+0000 7f1d8f0db700 1 -- 172.21.15.121:0/460757066 wait complete. 2023-11-15T03:04:09.679 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 104 2023-11-15T03:04:09.697 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 1.00000000.inode' led to DAMAGED state 2023-11-15T03:04:09.697 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 200.00000000' 2023-11-15T03:04:09.697 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:04:09.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:09.698 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:04:09.716 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:04:09.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:04:09.719 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:04:09.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.920+0000 7fb4172db700 1 Processor -- start 2023-11-15T03:04:09.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.920+0000 7fb4172db700 1 -- start start 2023-11-15T03:04:09.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb4172db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb410153aa0 0x7fb410066680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb4172db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4100c5c50 con 0x7fb4100c0490 2023-11-15T03:04:09.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb4172db700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4100c5920 con 0x7fb410153e80 2023-11-15T03:04:09.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb4172db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb410066b70 con 0x7fb410153aa0 2023-11-15T03:04:09.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb415878700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb4100c0490 0x7fb4100c60e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53436/0 (socket says 172.21.15.121:53436) 2023-11-15T03:04:09.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb415878700 1 -- 172.21.15.121:0/2713266781 learned_addr learned my addr 172.21.15.121:0/2713266781 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:09.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb415077700 1 --2- 172.21.15.121:0/2713266781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb410153aa0 0x7fb410066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb407fff700 1 -- 172.21.15.121:0/2713266781 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3917880933 0 0) 0x7fb4100c5c50 con 0x7fb4100c0490 2023-11-15T03:04:09.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb407fff700 1 -- 172.21.15.121:0/2713266781 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3fc0033f0 con 0x7fb4100c0490 2023-11-15T03:04:09.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb407fff700 1 -- 172.21.15.121:0/2713266781 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1292996180 0 0) 0x7fb3fc0033f0 con 0x7fb4100c0490 2023-11-15T03:04:09.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb407fff700 1 -- 172.21.15.121:0/2713266781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb410153aa0 msgr2=0x7fb410066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:09.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb407fff700 1 --2- 172.21.15.121:0/2713266781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb410153aa0 0x7fb410066680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb407fff700 1 -- 172.21.15.121:0/2713266781 >> v1:172.21.15.190:6789/0 conn(0x7fb410153e80 legacy=0x7fb4100ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:09.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb407fff700 1 -- 172.21.15.121:0/2713266781 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb410066d50 con 0x7fb4100c0490 2023-11-15T03:04:09.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb407fff700 1 -- 172.21.15.121:0/2713266781 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fb4080017f0 con 0x7fb4100c0490 2023-11-15T03:04:09.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb407fff700 1 -- 172.21.15.121:0/2713266781 >> v1:172.21.15.121:6789/0 conn(0x7fb4100c0490 legacy=0x7fb4100c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:09.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb407fff700 1 --2- 172.21.15.121:0/2713266781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3fc003910 0x7fb3fc003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb407fff700 1 --2- 172.21.15.121:0/2713266781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3fc004220 0x7fb3fc0046f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb407fff700 1 --2- 172.21.15.121:0/2713266781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3fc004ea0 0x7fb3fc00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb407fff700 1 -- 172.21.15.121:0/2713266781 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb410066b70 con 0x7fb3fc004ea0 2023-11-15T03:04:09.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb407fff700 1 -- 172.21.15.121:0/2713266781 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc00be00 con 0x7fb3fc004220 2023-11-15T03:04:09.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb407fff700 1 -- 172.21.15.121:0/2713266781 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc00bfc0 con 0x7fb3fc003910 2023-11-15T03:04:09.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb415878700 1 --2- 172.21.15.121:0/2713266781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3fc004ea0 0x7fb3fc00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb415077700 1 --2- 172.21.15.121:0/2713266781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3fc003910 0x7fb3fc003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb414876700 1 --2- 172.21.15.121:0/2713266781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3fc004220 0x7fb3fc0046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb415878700 1 -- 172.21.15.121:0/2713266781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3fc003910 msgr2=0x7fb3fc003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:09.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb415878700 1 --2- 172.21.15.121:0/2713266781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3fc003910 0x7fb3fc003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb415878700 1 -- 172.21.15.121:0/2713266781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3fc004220 msgr2=0x7fb3fc0046f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:09.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb415878700 1 --2- 172.21.15.121:0/2713266781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3fc004220 0x7fb3fc0046f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb415878700 1 -- 172.21.15.121:0/2713266781 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3fc0033f0 con 0x7fb3fc004ea0 2023-11-15T03:04:09.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb414876700 1 --2- 172.21.15.121:0/2713266781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3fc004220 0x7fb3fc0046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:09.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.924+0000 7fb415878700 1 --2- 172.21.15.121:0/2713266781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3fc004ea0 0x7fb3fc00b8e0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7fb40800c5f0 tx=0x7fb4100c5bc0).ready entity=mon.0 client_cookie=2196ce16e10dbbc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:09.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.928+0000 7fb407fff700 1 -- 172.21.15.121:0/2713266781 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40800cdf0 con 0x7fb3fc004ea0 2023-11-15T03:04:09.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.928+0000 7fb407fff700 1 -- 172.21.15.121:0/2713266781 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb408005b20 con 0x7fb3fc004ea0 2023-11-15T03:04:09.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.928+0000 7fb407fff700 1 -- 172.21.15.121:0/2713266781 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080070d0 con 0x7fb3fc004ea0 2023-11-15T03:04:09.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.928+0000 7fb4172db700 1 -- 172.21.15.121:0/2713266781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3fc004ea0 msgr2=0x7fb3fc00b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:09.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.928+0000 7fb4172db700 1 --2- 172.21.15.121:0/2713266781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3fc004ea0 0x7fb3fc00b8e0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7fb40800c5f0 tx=0x7fb4100c5bc0).stop 2023-11-15T03:04:09.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.928+0000 7fb415878700 1 -- 172.21.15.121:0/2713266781 reap_dead start 2023-11-15T03:04:09.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.928+0000 7fb4172db700 1 -- 172.21.15.121:0/2713266781 shutdown_connections 2023-11-15T03:04:09.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.928+0000 7fb4172db700 1 -- 172.21.15.121:0/2713266781 >> 172.21.15.121:0/2713266781 conn(0x7fb410067a60 msgr2=0x7fb410067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:09.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.928+0000 7fb4172db700 1 -- 172.21.15.121:0/2713266781 shutdown_connections 2023-11-15T03:04:09.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.928+0000 7fb4172db700 1 -- 172.21.15.121:0/2713266781 wait complete. 2023-11-15T03:04:09.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.932+0000 7fb4172db700 1 Processor -- start 2023-11-15T03:04:09.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.932+0000 7fb4172db700 1 -- start start 2023-11-15T03:04:09.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.932+0000 7fb4172db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3fc004ea0 0x7fb41014dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.932+0000 7fb4172db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4101562e0 0x7fb410154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.932+0000 7fb4172db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4100b0100 0x7fb4100b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.932+0000 7fb4172db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41014e270 con 0x7fb4100b0100 2023-11-15T03:04:09.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.932+0000 7fb4172db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4100b0a50 con 0x7fb4101562e0 2023-11-15T03:04:09.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.932+0000 7fb4172db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb410156ae0 con 0x7fb3fc004ea0 2023-11-15T03:04:09.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.932+0000 7fb415878700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4100b0100 0x7fb4100b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.932+0000 7fb415878700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4100b0100 0x7fb4100b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:52906/0 (socket says 172.21.15.121:52906) 2023-11-15T03:04:09.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.932+0000 7fb415878700 1 -- 172.21.15.121:0/1604975627 learned_addr learned my addr 172.21.15.121:0/1604975627 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:09.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.932+0000 7fb415077700 1 --2- 172.21.15.121:0/1604975627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3fc004ea0 0x7fb41014dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.932+0000 7fb414876700 1 --2- 172.21.15.121:0/1604975627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4101562e0 0x7fb410154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.936+0000 7fb415878700 1 -- 172.21.15.121:0/1604975627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3fc004ea0 msgr2=0x7fb41014dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:09.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.936+0000 7fb415878700 1 --2- 172.21.15.121:0/1604975627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3fc004ea0 0x7fb41014dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.936+0000 7fb415878700 1 -- 172.21.15.121:0/1604975627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4101562e0 msgr2=0x7fb410154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:09.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.936+0000 7fb415878700 1 --2- 172.21.15.121:0/1604975627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4101562e0 0x7fb410154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:09.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.936+0000 7fb415878700 1 -- 172.21.15.121:0/1604975627 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb410156dc0 con 0x7fb4100b0100 2023-11-15T03:04:09.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.936+0000 7fb415077700 1 --2- 172.21.15.121:0/1604975627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3fc004ea0 0x7fb41014dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:09.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.936+0000 7fb414876700 1 --2- 172.21.15.121:0/1604975627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4101562e0 0x7fb410154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:09.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.936+0000 7fb415878700 1 --2- 172.21.15.121:0/1604975627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4100b0100 0x7fb4100b0560 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7fb408000f80 tx=0x7fb408015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:09.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.936+0000 7fb4057fa700 1 -- 172.21.15.121:0/1604975627 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080017f0 con 0x7fb4100b0100 2023-11-15T03:04:09.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.936+0000 7fb4057fa700 1 -- 172.21.15.121:0/1604975627 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb408001a10 con 0x7fb4100b0100 2023-11-15T03:04:09.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.936+0000 7fb4057fa700 1 -- 172.21.15.121:0/1604975627 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408013450 con 0x7fb4100b0100 2023-11-15T03:04:09.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.936+0000 7fb4172db700 1 -- 172.21.15.121:0/1604975627 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4101570a0 con 0x7fb4100b0100 2023-11-15T03:04:09.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.936+0000 7fb4172db700 1 -- 172.21.15.121:0/1604975627 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4100ade50 con 0x7fb4100b0100 2023-11-15T03:04:09.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.936+0000 7fb4172db700 1 -- 172.21.15.121:0/1604975627 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4100ae040 con 0x7fb4100b0100 2023-11-15T03:04:09.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.940+0000 7fb4057fa700 1 -- 172.21.15.121:0/1604975627 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb408013a10 con 0x7fb4100b0100 2023-11-15T03:04:09.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.940+0000 7fb4057fa700 1 --2- 172.21.15.121:0/1604975627 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3f405a510 0x7fb3f405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:09.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.940+0000 7fb4057fa700 1 -- 172.21.15.121:0/1604975627 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6610+0+0 (secure 0 0 0) 0x7fb408075b60 con 0x7fb4100b0100 2023-11-15T03:04:09.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.940+0000 7fb415077700 1 --2- 172.21.15.121:0/1604975627 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3f405a510 0x7fb3f405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:09.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.940+0000 7fb415077700 1 --2- 172.21.15.121:0/1604975627 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3f405a510 0x7fb3f405c9d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7fb40c006a10 tx=0x7fb40c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:09.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:09.948+0000 7fb4057fa700 1 -- 172.21.15.121:0/1604975627 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb408049150 con 0x7fb4100b0100 2023-11-15T03:04:10.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:10.152+0000 7fb4172db700 1 -- 172.21.15.121:0/1604975627 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb410047060 con 0x7fb4100b0100 2023-11-15T03:04:11.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.040+0000 7fb4057fa700 1 -- 172.21.15.121:0/1604975627 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v105) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb40801a030 con 0x7fb4100b0100 2023-11-15T03:04:11.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.044+0000 7fb4172db700 1 -- 172.21.15.121:0/1604975627 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3f405a510 msgr2=0x7fb3f405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:11.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.044+0000 7fb4172db700 1 --2- 172.21.15.121:0/1604975627 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3f405a510 0x7fb3f405c9d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7fb40c006a10 tx=0x7fb40c008040).stop 2023-11-15T03:04:11.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.044+0000 7fb4172db700 1 -- 172.21.15.121:0/1604975627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4100b0100 msgr2=0x7fb4100b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:11.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.044+0000 7fb4172db700 1 --2- 172.21.15.121:0/1604975627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4100b0100 0x7fb4100b0560 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7fb408000f80 tx=0x7fb408015000).stop 2023-11-15T03:04:11.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.044+0000 7fb4172db700 1 -- 172.21.15.121:0/1604975627 shutdown_connections 2023-11-15T03:04:11.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.044+0000 7fb4172db700 1 --2- 172.21.15.121:0/1604975627 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3f405a510 0x7fb3f405c9d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:11.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.044+0000 7fb4172db700 1 --2- 172.21.15.121:0/1604975627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4100b0100 0x7fb4100b0560 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:11.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.044+0000 7fb4172db700 1 --2- 172.21.15.121:0/1604975627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4101562e0 0x7fb410154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:11.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.044+0000 7fb4172db700 1 --2- 172.21.15.121:0/1604975627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3fc004ea0 0x7fb41014dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:11.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.044+0000 7fb4172db700 1 -- 172.21.15.121:0/1604975627 >> 172.21.15.121:0/1604975627 conn(0x7fb410067a60 msgr2=0x7fb4100c7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:11.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.044+0000 7fb4172db700 1 -- 172.21.15.121:0/1604975627 shutdown_connections 2023-11-15T03:04:11.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.044+0000 7fb4172db700 1 -- 172.21.15.121:0/1604975627 wait complete. 2023-11-15T03:04:11.054 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:04:11.075 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:04:11.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.228+0000 7fd3ee69b700 1 Processor -- start 2023-11-15T03:04:11.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.228+0000 7fd3ee69b700 1 -- start start 2023-11-15T03:04:11.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.228+0000 7fd3ee69b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd3e8151920 0x7fd3e80bb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:11.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.228+0000 7fd3ee69b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd3e8067150 0x7fd3e80aba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:11.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3ee69b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3e8156270 0x7fd3e80abf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:11.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3ee69b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3e80c4060 con 0x7fd3e8151920 2023-11-15T03:04:11.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3ee69b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3e80ac420 con 0x7fd3e8067150 2023-11-15T03:04:11.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3ee69b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e8155de0 con 0x7fd3e8156270 2023-11-15T03:04:11.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3e7fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd3e8151920 0x7fd3e80bb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:11.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3e7fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd3e8151920 0x7fd3e80bb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:52912/0 (socket says 172.21.15.121:52912) 2023-11-15T03:04:11.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3e7fff700 1 -- 172.21.15.121:0/1495194708 learned_addr learned my addr 172.21.15.121:0/1495194708 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:11.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3dffff700 1 --2- 172.21.15.121:0/1495194708 >> v2:172.21.15.190:3300/0 conn(0x7fd3e8067150 0x7fd3e80aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:11.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3ecc38700 1 --2- 172.21.15.121:0/1495194708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3e8156270 0x7fd3e80abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:11.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3e7fff700 1 -- 172.21.15.121:0/1495194708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3e8156270 msgr2=0x7fd3e80abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:11.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3e7fff700 1 --2- 172.21.15.121:0/1495194708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3e8156270 0x7fd3e80abf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:11.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3e7fff700 1 -- 172.21.15.121:0/1495194708 >> v2:172.21.15.190:3300/0 conn(0x7fd3e8067150 msgr2=0x7fd3e80aba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:11.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3e7fff700 1 --2- 172.21.15.121:0/1495194708 >> v2:172.21.15.190:3300/0 conn(0x7fd3e8067150 0x7fd3e80aba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:11.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3e7fff700 1 -- 172.21.15.121:0/1495194708 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e80c3aa0 con 0x7fd3e8151920 2023-11-15T03:04:11.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3e7fff700 1 --2- 172.21.15.121:0/1495194708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3e8151920 0x7fd3e80bb6b0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7fd3e80c7730 tx=0x7fd3d80090f0).ready entity=mon.0 client_cookie=42ff44aefdf0b2e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:11.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3e77fe700 1 -- 172.21.15.121:0/1495194708 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d800f510 con 0x7fd3e8151920 2023-11-15T03:04:11.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3e77fe700 1 -- 172.21.15.121:0/1495194708 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d800f670 con 0x7fd3e8151920 2023-11-15T03:04:11.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3e77fe700 1 -- 172.21.15.121:0/1495194708 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d800f9f0 con 0x7fd3e8151920 2023-11-15T03:04:11.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3ee69b700 1 -- 172.21.15.121:0/1495194708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3e8151920 msgr2=0x7fd3e80bb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:11.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3ee69b700 1 --2- 172.21.15.121:0/1495194708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3e8151920 0x7fd3e80bb6b0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7fd3e80c7730 tx=0x7fd3d80090f0).stop 2023-11-15T03:04:11.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3ee69b700 1 -- 172.21.15.121:0/1495194708 shutdown_connections 2023-11-15T03:04:11.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3ee69b700 1 --2- 172.21.15.121:0/1495194708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3e8156270 0x7fd3e80abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:11.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3ee69b700 1 --2- 172.21.15.121:0/1495194708 >> v2:172.21.15.190:3300/0 conn(0x7fd3e8067150 0x7fd3e80aba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:11.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3ee69b700 1 --2- 172.21.15.121:0/1495194708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3e8151920 0x7fd3e80bb6b0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:11.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3ee69b700 1 -- 172.21.15.121:0/1495194708 >> 172.21.15.121:0/1495194708 conn(0x7fd3e8067a60 msgr2=0x7fd3e8067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:11.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3ee69b700 1 -- 172.21.15.121:0/1495194708 shutdown_connections 2023-11-15T03:04:11.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.232+0000 7fd3ee69b700 1 -- 172.21.15.121:0/1495194708 wait complete. 2023-11-15T03:04:11.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.236+0000 7fd3ee69b700 1 Processor -- start 2023-11-15T03:04:11.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.236+0000 7fd3ee69b700 1 -- start start 2023-11-15T03:04:11.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.236+0000 7fd3ee69b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3e8067150 0x7fd3e8150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:11.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.236+0000 7fd3ee69b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3e8151920 0x7fd3e814a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:11.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.236+0000 7fd3ee69b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3e8156270 0x7fd3e813d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:11.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.236+0000 7fd3ee69b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e8150a20 con 0x7fd3e8156270 2023-11-15T03:04:11.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.236+0000 7fd3ee69b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e8157de0 con 0x7fd3e8151920 2023-11-15T03:04:11.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.236+0000 7fd3ee69b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e814ab30 con 0x7fd3e8067150 2023-11-15T03:04:11.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.236+0000 7fd3ecc38700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3e8156270 0x7fd3e813d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:11.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.236+0000 7fd3ecc38700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3e8156270 0x7fd3e813d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:52928/0 (socket says 172.21.15.121:52928) 2023-11-15T03:04:11.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.236+0000 7fd3ecc38700 1 -- 172.21.15.121:0/2705355754 learned_addr learned my addr 172.21.15.121:0/2705355754 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:11.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.236+0000 7fd3ecc38700 1 -- 172.21.15.121:0/2705355754 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3e8067150 msgr2=0x7fd3e8150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:11.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.236+0000 7fd3ecc38700 1 --2- 172.21.15.121:0/2705355754 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3e8067150 0x7fd3e8150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:04:11.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.236+0000 7fd3ecc38700 1 -- 172.21.15.121:0/2705355754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3e8151920 msgr2=0x7fd3e814a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:11.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.236+0000 7fd3ecc38700 1 --2- 172.21.15.121:0/2705355754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3e8151920 0x7fd3e814a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:04:11.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.236+0000 7fd3ecc38700 1 -- 172.21.15.121:0/2705355754 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d8009370 con 0x7fd3e8156270 2023-11-15T03:04:11.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.236+0000 7fd3ecc38700 1 --2- 172.21.15.121:0/2705355754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3e8156270 0x7fd3e813d180 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 rx=0x7fd3e0009df0 tx=0x7fd3e000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:11.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.240+0000 7fd3e4ff9700 1 -- 172.21.15.121:0/2705355754 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e000caf0 con 0x7fd3e8156270 2023-11-15T03:04:11.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.240+0000 7fd3e4ff9700 1 -- 172.21.15.121:0/2705355754 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e000f040 con 0x7fd3e8156270 2023-11-15T03:04:11.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.240+0000 7fd3ee69b700 1 -- 172.21.15.121:0/2705355754 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e813d670 con 0x7fd3e8156270 2023-11-15T03:04:11.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.240+0000 7fd3e4ff9700 1 -- 172.21.15.121:0/2705355754 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e000ce60 con 0x7fd3e8156270 2023-11-15T03:04:11.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.240+0000 7fd3ee69b700 1 -- 172.21.15.121:0/2705355754 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e8150ef0 con 0x7fd3e8156270 2023-11-15T03:04:11.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.240+0000 7fd3ee69b700 1 -- 172.21.15.121:0/2705355754 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e8047060 con 0x7fd3e8156270 2023-11-15T03:04:11.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.240+0000 7fd3e4ff9700 1 -- 172.21.15.121:0/2705355754 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd3e0013610 con 0x7fd3e8156270 2023-11-15T03:04:11.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.244+0000 7fd3e4ff9700 1 --2- 172.21.15.121:0/2705355754 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd3c805a540 0x7fd3c805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:11.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.244+0000 7fd3e4ff9700 1 -- 172.21.15.121:0/2705355754 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6610+0+0 (secure 0 0 0) 0x7fd3e0075230 con 0x7fd3e8156270 2023-11-15T03:04:11.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.244+0000 7fd3e7fff700 1 --2- 172.21.15.121:0/2705355754 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd3c805a540 0x7fd3c805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:11.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.244+0000 7fd3e7fff700 1 --2- 172.21.15.121:0/2705355754 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd3c805a540 0x7fd3c805ca00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fd3e80c7730 tx=0x7fd3d8017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:11.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.248+0000 7fd3e4ff9700 1 -- 172.21.15.121:0/2705355754 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd3e0048760 con 0x7fd3e8156270 2023-11-15T03:04:11.309 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:04:11.317 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:04:11.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:11.468+0000 7fd3ee69b700 1 -- 172.21.15.121:0/2705355754 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fd3e8157f70 con 0x7fd3e8156270 2023-11-15T03:04:12.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.048+0000 7fd3e4ff9700 1 -- 172.21.15.121:0/2705355754 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v106) v1 ==== 101+0+0 (secure 0 0 0) 0x7fd3e004bd80 con 0x7fd3e8156270 2023-11-15T03:04:12.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.052+0000 7fd3ee69b700 1 -- 172.21.15.121:0/2705355754 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd3c805a540 msgr2=0x7fd3c805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.052+0000 7fd3ee69b700 1 --2- 172.21.15.121:0/2705355754 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd3c805a540 0x7fd3c805ca00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fd3e80c7730 tx=0x7fd3d8017000).stop 2023-11-15T03:04:12.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.052+0000 7fd3ee69b700 1 -- 172.21.15.121:0/2705355754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3e8156270 msgr2=0x7fd3e813d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.052+0000 7fd3ee69b700 1 --2- 172.21.15.121:0/2705355754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3e8156270 0x7fd3e813d180 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 rx=0x7fd3e0009df0 tx=0x7fd3e000b040).stop 2023-11-15T03:04:12.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.052+0000 7fd3ee69b700 1 -- 172.21.15.121:0/2705355754 shutdown_connections 2023-11-15T03:04:12.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.052+0000 7fd3ee69b700 1 --2- 172.21.15.121:0/2705355754 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd3c805a540 0x7fd3c805ca00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.052+0000 7fd3ee69b700 1 --2- 172.21.15.121:0/2705355754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3e8156270 0x7fd3e813d180 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.052+0000 7fd3ee69b700 1 --2- 172.21.15.121:0/2705355754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3e8151920 0x7fd3e814a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:04:12.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.052+0000 7fd3ee69b700 1 --2- 172.21.15.121:0/2705355754 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3e8067150 0x7fd3e8150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:04:12.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.052+0000 7fd3ee69b700 1 -- 172.21.15.121:0/2705355754 >> 172.21.15.121:0/2705355754 conn(0x7fd3e8067a60 msgr2=0x7fd3e80c5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:12.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.052+0000 7fd3ee69b700 1 -- 172.21.15.121:0/2705355754 shutdown_connections 2023-11-15T03:04:12.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.052+0000 7fd3ee69b700 1 -- 172.21.15.121:0/2705355754 wait complete. 2023-11-15T03:04:12.062 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:04:12.086 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:04:12.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.140+0000 7f71cbf39d00 1 Processor -- start 2023-11-15T03:04:12.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.140+0000 7f71cbf39d00 1 -- start start 2023-11-15T03:04:12.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71cbf39d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5590173d0a30 0x5590174b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71cbf39d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55901741be70 0x5590174aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71cbf39d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590174b1000 0x559017414fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71cbf39d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559017349220 con 0x55901741be70 2023-11-15T03:04:12.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71cbf39d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559017403bb0 con 0x5590174b1000 2023-11-15T03:04:12.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71cbf39d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590173ffbc0 con 0x5590173d0a30 2023-11-15T03:04:12.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71b7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55901741be70 0x5590174aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71c05a1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5590173d0a30 0x5590174b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71c05a1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5590173d0a30 0x5590174b05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:34372/0 (socket says 172.21.15.121:34372) 2023-11-15T03:04:12.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71c05a1700 1 -- 172.21.15.121:0/4143871030 learned_addr learned my addr 172.21.15.121:0/4143871030 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:12.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71c0da2700 1 --2- 172.21.15.121:0/4143871030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590174b1000 0x559017414fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71c05a1700 1 -- 172.21.15.121:0/4143871030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590174b1000 msgr2=0x559017414fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:12.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71c05a1700 1 --2- 172.21.15.121:0/4143871030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590174b1000 0x559017414fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71c05a1700 1 -- 172.21.15.121:0/4143871030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55901741be70 msgr2=0x5590174aefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:12.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71c05a1700 1 --2- 172.21.15.121:0/4143871030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55901741be70 0x5590174aefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71c05a1700 1 -- 172.21.15.121:0/4143871030 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b800a180 con 0x5590173d0a30 2023-11-15T03:04:12.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71b7fff700 1 --2- 172.21.15.121:0/4143871030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55901741be70 0x5590174aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:04:12.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71c05a1700 1 --2- 172.21.15.121:0/4143871030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5590173d0a30 0x5590174b05e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f71b0009800 tx=0x7f71b000c040).ready entity=mon.2 client_cookie=317216bcac9dbe01 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:12.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71bf59f700 1 -- 172.21.15.121:0/4143871030 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b000f040 con 0x5590173d0a30 2023-11-15T03:04:12.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71bf59f700 1 -- 172.21.15.121:0/4143871030 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b0008700 con 0x5590173d0a30 2023-11-15T03:04:12.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71bf59f700 1 -- 172.21.15.121:0/4143871030 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b0008a80 con 0x5590173d0a30 2023-11-15T03:04:12.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71cbf39d00 1 -- 172.21.15.121:0/4143871030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5590173d0a30 msgr2=0x5590174b05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/4143871030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5590173d0a30 0x5590174b05e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f71b0009800 tx=0x7f71b000c040).stop 2023-11-15T03:04:12.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71cbf39d00 1 -- 172.21.15.121:0/4143871030 shutdown_connections 2023-11-15T03:04:12.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/4143871030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590174b1000 0x559017414fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/4143871030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55901741be70 0x5590174aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/4143871030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5590173d0a30 0x5590174b05e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71cbf39d00 1 -- 172.21.15.121:0/4143871030 >> 172.21.15.121:0/4143871030 conn(0x559017420080 msgr2=0x5590174afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:12.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71cbf39d00 1 -- 172.21.15.121:0/4143871030 shutdown_connections 2023-11-15T03:04:12.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.144+0000 7f71cbf39d00 1 -- 172.21.15.121:0/4143871030 wait complete. 2023-11-15T03:04:12.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.148+0000 7f71cbf39d00 1 Processor -- start 2023-11-15T03:04:12.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.148+0000 7f71cbf39d00 1 -- start start 2023-11-15T03:04:12.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.148+0000 7f71cbf39d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5590173d0a30 0x5590174056a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.148+0000 7f71cbf39d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901741be70 0x559017405ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.148+0000 7f71cbf39d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590174b1000 0x5590174a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.148+0000 7f71cbf39d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590174a82f0 con 0x5590174b1000 2023-11-15T03:04:12.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.148+0000 7f71cbf39d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590174a85d0 con 0x55901741be70 2023-11-15T03:04:12.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.148+0000 7f71cbf39d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590174a88b0 con 0x5590173d0a30 2023-11-15T03:04:12.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.148+0000 7f71c0da2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590174b1000 0x5590174a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.148+0000 7f71c0da2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590174b1000 0x5590174a63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:52944/0 (socket says 172.21.15.121:52944) 2023-11-15T03:04:12.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.148+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 learned_addr learned my addr 172.21.15.121:0/3582279857 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:12.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.148+0000 7f71c05a1700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5590173d0a30 0x5590174056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.148+0000 7f71b7fff700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901741be70 0x559017405ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.152+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5590173d0a30 msgr2=0x5590174056a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.152+0000 7f71c0da2700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5590173d0a30 0x5590174056a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.152+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901741be70 msgr2=0x559017405ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.152+0000 7f71c0da2700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901741be70 0x559017405ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.152+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a800a040 con 0x5590174b1000 2023-11-15T03:04:12.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.152+0000 7f71b7fff700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55901741be70 0x559017405ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:04:12.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.152+0000 7f71c0da2700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590174b1000 0x5590174a63b0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f71b800b010 tx=0x7f71b8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:12.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.152+0000 7f71bdd9c700 1 -- 172.21.15.121:0/3582279857 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b800cd30 con 0x5590174b1000 2023-11-15T03:04:12.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.152+0000 7f71bdd9c700 1 -- 172.21.15.121:0/3582279857 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b800ce90 con 0x5590174b1000 2023-11-15T03:04:12.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.152+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b0009430 con 0x5590174b1000 2023-11-15T03:04:12.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.152+0000 7f71bdd9c700 1 -- 172.21.15.121:0/3582279857 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b800c9a0 con 0x5590174b1000 2023-11-15T03:04:12.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.152+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559017372770 con 0x5590174b1000 2023-11-15T03:04:12.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.152+0000 7f71bdd9c700 1 -- 172.21.15.121:0/3582279857 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f71b8003040 con 0x5590174b1000 2023-11-15T03:04:12.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.156+0000 7f71bdd9c700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71a0058ff0 0x7f71a005b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.156+0000 7f71bdd9c700 1 -- 172.21.15.121:0/3582279857 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6610+0+0 (secure 0 0 0) 0x7f71b8075bd0 con 0x5590174b1000 2023-11-15T03:04:12.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.156+0000 7f71c05a1700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71a0058ff0 0x7f71a005b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.186 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:04:12.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.156+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x559017418c30 0x55901741c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.156+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174a9390 con 0x559017418c30 2023-11-15T03:04:12.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.156+0000 7f71b7fff700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x559017418c30 0x55901741c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.156+0000 7f71c05a1700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71a0058ff0 0x7f71a005b4b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f71b0006010 tx=0x7f71b0018000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:12.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.156+0000 7f71b7fff700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x559017418c30 0x55901741c790 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:12.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.160+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v98'41 uv34 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x559017418c30 2023-11-15T03:04:12.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.160+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174a9740 con 0x559017418c30 2023-11-15T03:04:12.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.160+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v98'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x559017418c30 2023-11-15T03:04:12.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.160+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174a9af0 con 0x559017418c30 2023-11-15T03:04:12.190 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:04:12.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.164+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v98'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x559017418c30 2023-11-15T03:04:12.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.164+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174cb750 con 0x559017418c30 2023-11-15T03:04:12.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.168+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v98'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x559017418c30 2023-11-15T03:04:12.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.168+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174cbb00 con 0x559017418c30 2023-11-15T03:04:12.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.172+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v98'45 uv45 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x559017418c30 2023-11-15T03:04:12.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.172+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174cac30 con 0x559017418c30 2023-11-15T03:04:12.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.172+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v98'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x559017418c30 2023-11-15T03:04:12.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.172+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174cb030 con 0x559017418c30 2023-11-15T03:04:12.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.180+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v98'47 uv40 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x559017418c30 2023-11-15T03:04:12.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.180+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174cbeb0 con 0x559017418c30 2023-11-15T03:04:12.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.180+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v98'48 uv48 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x559017418c30 2023-11-15T03:04:12.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.180+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174cbeb0 con 0x559017418c30 2023-11-15T03:04:12.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.184+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v98'49 uv49 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x559017418c30 2023-11-15T03:04:12.195 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:04:12.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.184+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174ccec0 con 0x559017418c30 2023-11-15T03:04:12.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.188+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v98'50 uv50 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x559017418c30 2023-11-15T03:04:12.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.188+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174ccec0 con 0x559017418c30 2023-11-15T03:04:12.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.192+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v98'51 uv51 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x559017418c30 2023-11-15T03:04:12.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.192+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174cbeb0 con 0x559017418c30 2023-11-15T03:04:12.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.196+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v98'52 uv52 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x559017418c30 2023-11-15T03:04:12.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.196+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5590174cc3c0 0x5590174ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.196+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174ceca0 con 0x5590174cc3c0 2023-11-15T03:04:12.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.196+0000 7f71c05a1700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5590174cc3c0 0x5590174ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.196+0000 7f71c05a1700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5590174cc3c0 0x5590174ce7b0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:12.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.200+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v98'62 uv51 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.200+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174cf050 con 0x5590174cc3c0 2023-11-15T03:04:12.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.204+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v98'63 uv63 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.204+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174cf400 con 0x5590174cc3c0 2023-11-15T03:04:12.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.204+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v98'64 uv64 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.213 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:04:12.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.204+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d07c0 con 0x5590174cc3c0 2023-11-15T03:04:12.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.208+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v98'65 uv65 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.208+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d0b70 con 0x5590174cc3c0 2023-11-15T03:04:12.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.212+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v98'66 uv66 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.212+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174cf7b0 con 0x5590174cc3c0 2023-11-15T03:04:12.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.216+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v98'67 uv67 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.216+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d00a0 con 0x5590174cc3c0 2023-11-15T03:04:12.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.216+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v98'68 uv68 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.216+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d00a0 con 0x5590174cc3c0 2023-11-15T03:04:12.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.220+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v98'69 uv55 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.220+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174d00a0 con 0x5590174cc3c0 2023-11-15T03:04:12.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.220+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v98'70 uv70 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.220+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d00a0 con 0x5590174cc3c0 2023-11-15T03:04:12.231 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:04:12.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.224+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v98'71 uv71 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.224+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d00a0 con 0x5590174cc3c0 2023-11-15T03:04:12.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.228+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v98'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.228+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d00a0 con 0x5590174cc3c0 2023-11-15T03:04:12.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.228+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v98'73 uv73 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.228+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d00a0 con 0x5590174cc3c0 2023-11-15T03:04:12.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.232+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v98'74 uv74 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.232+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d00a0 con 0x5590174cc3c0 2023-11-15T03:04:12.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.236+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v98'75 uv61 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.236+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174d00a0 con 0x5590174cc3c0 2023-11-15T03:04:12.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.236+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v98'76 uv76 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.236+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d00a0 con 0x5590174cc3c0 2023-11-15T03:04:12.246 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:04:12.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.240+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v98'77 uv77 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.240+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d00a0 con 0x5590174cc3c0 2023-11-15T03:04:12.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.244+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v98'78 uv78 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.244+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d00a0 con 0x5590174cc3c0 2023-11-15T03:04:12.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.244+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v98'79 uv79 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.244+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d00a0 con 0x5590174cc3c0 2023-11-15T03:04:12.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.248+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v98'80 uv80 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.248+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5590174d0f20 0x5590174d3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.248+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d3800 con 0x5590174d0f20 2023-11-15T03:04:12.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.248+0000 7f71b7fff700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5590174d0f20 0x5590174d3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.248+0000 7f71b7fff700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5590174d0f20 0x5590174d3310 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:12.259 INFO:teuthology.orchestra.run.smithi121.stdout:Write #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:04:12.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.252+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v98'38 uv38 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174d0f20 2023-11-15T03:04:12.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.252+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d3cf0 con 0x5590174d0f20 2023-11-15T03:04:12.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.256+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [write 0~554] v98'39 uv39 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174d0f20 2023-11-15T03:04:12.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.256+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d3cf0 con 0x5590174d0f20 2023-11-15T03:04:12.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.260+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v98'40 uv40 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174d0f20 2023-11-15T03:04:12.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.260+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d3cf0 con 0x5590174d0f20 2023-11-15T03:04:12.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.264+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 200.00000000 [create] v98'41 uv36 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174d0f20 2023-11-15T03:04:12.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.264+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174d3cf0 con 0x5590174d0f20 2023-11-15T03:04:12.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.264+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 200.00000000 [delete] v98'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174d0f20 2023-11-15T03:04:12.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.264+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d3cf0 con 0x5590174d0f20 2023-11-15T03:04:12.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.268+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v98'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174d0f20 2023-11-15T03:04:12.276 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:04:12.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.268+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d3cf0 con 0x5590174d0f20 2023-11-15T03:04:12.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.272+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [write 0~90] v98'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174d0f20 2023-11-15T03:04:12.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.272+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d3cf0 con 0x5590174d0f20 2023-11-15T03:04:12.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.276+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [omap-set-header] v98'45 uv45 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174d0f20 2023-11-15T03:04:12.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.276+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5590174d4200 0x5590174d65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.276+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:40 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d6ae0 con 0x5590174d4200 2023-11-15T03:04:12.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.276+0000 7f71c0da2700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5590174d4200 0x5590174d65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.276+0000 7f71c0da2700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5590174d4200 0x5590174d65f0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:12.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.280+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(40 200.00000001 [create] v98'51 uv44 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.280+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:41 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174d6e90 con 0x5590174d4200 2023-11-15T03:04:12.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.284+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(41 200.00000001 [delete] v98'52 uv52 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.284+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d7240 con 0x5590174d4200 2023-11-15T03:04:12.294 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:04:12.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.288+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(42 200.00000001 [create] v98'53 uv53 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.288+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174db680 con 0x5590174d4200 2023-11-15T03:04:12.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.292+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(43 200.00000001 [write 0~16484] v98'54 uv54 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.292+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d3900 con 0x5590174d4200 2023-11-15T03:04:12.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.292+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(44 200.00000001 [omap-set-header] v98'55 uv55 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.296+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d75f0 con 0x5590174d4200 2023-11-15T03:04:12.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.296+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(45 606.00000000 [create] v98'56 uv50 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.296+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174d79f0 con 0x5590174d4200 2023-11-15T03:04:12.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.300+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(46 606.00000000 [delete] v98'57 uv57 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.300+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d7da0 con 0x5590174d4200 2023-11-15T03:04:12.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.304+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(47 606.00000000 [create] v98'58 uv58 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.313 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:04:12.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.304+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d8db0 con 0x5590174d4200 2023-11-15T03:04:12.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.304+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v98'59 uv59 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.304+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d8db0 con 0x5590174d4200 2023-11-15T03:04:12.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.308+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v98'60 uv60 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.308+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174d7da0 con 0x5590174d4200 2023-11-15T03:04:12.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.312+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(50 606.00000000 [omap-set-header] v98'61 uv61 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.312+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5590174d86b0 0x5590174daaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.312+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:51 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174daf90 con 0x5590174d86b0 2023-11-15T03:04:12.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.312+0000 7f71c05a1700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5590174d86b0 0x5590174daaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.316+0000 7f71c05a1700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5590174d86b0 0x5590174daaa0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:12.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.316+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(51 607.00000000 [create] v98'21 uv20 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174d86b0 2023-11-15T03:04:12.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.316+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:52 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174daf90 con 0x5590174d86b0 2023-11-15T03:04:12.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.320+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(52 607.00000000 [delete] v98'22 uv22 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174d86b0 2023-11-15T03:04:12.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.320+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174daf90 con 0x5590174d86b0 2023-11-15T03:04:12.330 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:04:12.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.324+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(53 607.00000000 [create] v98'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174d86b0 2023-11-15T03:04:12.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.324+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174daf90 con 0x5590174d86b0 2023-11-15T03:04:12.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.324+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v98'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174d86b0 2023-11-15T03:04:12.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.324+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174daf90 con 0x5590174d86b0 2023-11-15T03:04:12.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.328+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v98'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174d86b0 2023-11-15T03:04:12.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.328+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174daf90 con 0x5590174d86b0 2023-11-15T03:04:12.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.332+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v98'26 uv26 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174d86b0 2023-11-15T03:04:12.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.332+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5590174dc4d0 0x5590174de8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.332+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:57 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174dedb0 con 0x5590174dc4d0 2023-11-15T03:04:12.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.332+0000 7f71b7fff700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5590174dc4d0 0x5590174de8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.332+0000 7f71b7fff700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5590174dc4d0 0x5590174de8c0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:12.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.336+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v98'16 uv15 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.336+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:58 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174df1b0 con 0x5590174dc4d0 2023-11-15T03:04:12.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.336+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v98'17 uv17 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.336+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174df1b0 con 0x5590174dc4d0 2023-11-15T03:04:12.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.340+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v98'18 uv18 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.347 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:04:12.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.340+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174df1b0 con 0x5590174dc4d0 2023-11-15T03:04:12.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.344+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v98'19 uv19 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.344+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174df1b0 con 0x5590174dc4d0 2023-11-15T03:04:12.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.348+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(61 608.00000000 [create] v98'21 uv20 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.348+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174df1b0 con 0x5590174dc4d0 2023-11-15T03:04:12.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.348+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(62 608.00000000 [delete] v98'22 uv22 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.348+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174df1b0 con 0x5590174dc4d0 2023-11-15T03:04:12.358 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:04:12.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.352+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(63 608.00000000 [create] v98'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.352+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e01c0 con 0x5590174dc4d0 2023-11-15T03:04:12.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.356+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v98'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.356+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e01c0 con 0x5590174dc4d0 2023-11-15T03:04:12.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.356+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v98'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.356+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174df1b0 con 0x5590174dc4d0 2023-11-15T03:04:12.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.360+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(66 608.00000000 [omap-set-header] v98'26 uv26 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.360+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174df1b0 con 0x5590174cc3c0 2023-11-15T03:04:12.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.364+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(67 604.00000000 [create] v98'37 uv31 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.364+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174df1b0 con 0x5590174cc3c0 2023-11-15T03:04:12.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.368+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(68 604.00000000 [delete] v98'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.368+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174df1b0 con 0x5590174cc3c0 2023-11-15T03:04:12.378 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:04:12.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.372+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(69 604.00000000 [create] v98'39 uv39 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.372+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e01c0 con 0x5590174cc3c0 2023-11-15T03:04:12.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.376+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v98'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.376+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e01c0 con 0x5590174cc3c0 2023-11-15T03:04:12.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.380+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v98'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.380+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174df1b0 con 0x5590174cc3c0 2023-11-15T03:04:12.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.380+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(72 604.00000000 [omap-set-header] v98'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.380+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174df1b0 con 0x5590174cc3c0 2023-11-15T03:04:12.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.384+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(73 500.00000000 [create] v98'43 uv36 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.384+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174df1b0 con 0x5590174cc3c0 2023-11-15T03:04:12.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.388+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(74 500.00000000 [delete] v98'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.388+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174df1b0 con 0x5590174cc3c0 2023-11-15T03:04:12.397 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:04:12.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.392+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(75 500.00000000 [create] v98'45 uv45 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.392+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174df1b0 con 0x5590174cc3c0 2023-11-15T03:04:12.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.396+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(76 500.00000000 [write 0~90] v98'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.396+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174cc3c0 2023-11-15T03:04:12.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.396+0000 7f71c05a1700 1 -- 172.21.15.121:0/3582279857 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(77 500.00000000 [omap-set-header] v98'47 uv47 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b0014430 con 0x5590174cc3c0 2023-11-15T03:04:12.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.396+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:78 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174d0f20 2023-11-15T03:04:12.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.400+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(78 mds_snaptable [create] v98'17 uv16 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174d0f20 2023-11-15T03:04:12.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.400+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:79 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174d0f20 2023-11-15T03:04:12.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.404+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(79 mds_snaptable [delete] v98'18 uv18 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174d0f20 2023-11-15T03:04:12.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.404+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174d0f20 2023-11-15T03:04:12.412 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:04:12.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.404+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v98'19 uv19 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174d0f20 2023-11-15T03:04:12.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.404+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174d0f20 2023-11-15T03:04:12.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.408+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [write 0~70] v98'20 uv20 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174d0f20 2023-11-15T03:04:12.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.408+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174d0f20 2023-11-15T03:04:12.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.412+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v98'21 uv21 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174d0f20 2023-11-15T03:04:12.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.412+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174dc4d0 2023-11-15T03:04:12.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.412+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(83 605.00000000 [create] v98'21 uv20 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.412+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174dc4d0 2023-11-15T03:04:12.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.416+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(84 605.00000000 [delete] v98'22 uv22 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.416+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174dc4d0 2023-11-15T03:04:12.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.420+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(85 605.00000000 [create] v98'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.427 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:04:12.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.420+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174dc4d0 2023-11-15T03:04:12.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.424+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v98'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.424+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174dc4d0 2023-11-15T03:04:12.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.424+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v98'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.424+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174dc4d0 2023-11-15T03:04:12.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.428+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(88 605.00000000 [omap-set-header] v98'26 uv26 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.428+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174dc4d0 2023-11-15T03:04:12.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.432+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(89 mds0_inotable [create] v98'18 uv17 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.432+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174dc4d0 2023-11-15T03:04:12.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.436+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(90 mds0_inotable [delete] v98'19 uv19 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.436+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174dc4d0 2023-11-15T03:04:12.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.436+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(91 mds0_inotable [create] v98'20 uv20 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.447 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:04:12.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.436+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174dc4d0 2023-11-15T03:04:12.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.440+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(92 mds0_inotable [write 0~34] v98'21 uv21 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.440+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174dc4d0 2023-11-15T03:04:12.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.444+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v98'22 uv22 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.444+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174dc4d0 2023-11-15T03:04:12.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.448+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(94 100.00000000 [create] v98'17 uv16 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.448+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174dc4d0 2023-11-15T03:04:12.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.448+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(95 100.00000000 [delete] v98'18 uv18 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.448+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174dc4d0 2023-11-15T03:04:12.461 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:04:12.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.452+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(96 100.00000000 [create] v98'19 uv19 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.452+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174dbb00 con 0x5590174dc4d0 2023-11-15T03:04:12.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.456+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(97 100.00000000 [omap-set-header] v98'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.456+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e2020 con 0x5590174dc4d0 2023-11-15T03:04:12.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.460+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v98'21 uv21 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.460+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5590174df830 0x5590174e2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.460+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174df830 2023-11-15T03:04:12.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.460+0000 7f71c0da2700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5590174df830 0x5590174e2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.460+0000 7f71c0da2700 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5590174df830 0x5590174e2640 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:12.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.464+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(99 mds0_sessionmap [create] v98'15 uv14 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174df830 2023-11-15T03:04:12.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.464+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174df830 2023-11-15T03:04:12.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.464+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(100 mds0_sessionmap [delete] v98'16 uv16 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174df830 2023-11-15T03:04:12.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.464+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174df830 2023-11-15T03:04:12.476 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:04:12.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.468+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(101 mds0_sessionmap [create] v98'17 uv17 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174df830 2023-11-15T03:04:12.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.468+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174df830 2023-11-15T03:04:12.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.472+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v98'18 uv18 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174df830 2023-11-15T03:04:12.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.472+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174dc4d0 2023-11-15T03:04:12.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.472+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(103 609.00000000 [create] v98'21 uv20 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.472+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174dc4d0 2023-11-15T03:04:12.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.476+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(104 609.00000000 [delete] v98'22 uv22 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.476+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174dc4d0 2023-11-15T03:04:12.486 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:04:12.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.480+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(105 609.00000000 [create] v98'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.480+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e3b40 con 0x5590174dc4d0 2023-11-15T03:04:12.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.484+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v98'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.484+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e3b40 con 0x5590174dc4d0 2023-11-15T03:04:12.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.484+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v98'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.484+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174dc4d0 2023-11-15T03:04:12.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.488+0000 7f71b7fff700 1 -- 172.21.15.121:0/3582279857 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(108 609.00000000 [omap-set-header] v98'26 uv26 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71a800ec30 con 0x5590174dc4d0 2023-11-15T03:04:12.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.488+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174d4200 2023-11-15T03:04:12.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.492+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(109 400.00000000 [create] v98'17 uv16 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.492+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174d4200 2023-11-15T03:04:12.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.492+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(110 400.00000000 [delete] v98'18 uv18 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.492+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174d4200 2023-11-15T03:04:12.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.496+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(111 400.00000000 [create] v98'19 uv19 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.502 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:04:12.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.496+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174d4200 2023-11-15T03:04:12.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.500+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(112 400.00000000 [write 0~22] v98'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.500+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174d4200 2023-11-15T03:04:12.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.500+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(113 400.00000000 [omap-set-header] v98'21 uv21 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.500+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174d4200 2023-11-15T03:04:12.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.504+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(114 100.00000000.inode [create] v98'42 uv34 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.504+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174d4200 2023-11-15T03:04:12.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.508+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(115 100.00000000.inode [delete] v98'43 uv43 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.508+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174d4200 2023-11-15T03:04:12.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.508+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(116 100.00000000.inode [create] v98'44 uv44 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.516 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:04:12.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.508+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174d4200 2023-11-15T03:04:12.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.512+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v98'45 uv45 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.512+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174d4200 2023-11-15T03:04:12.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.516+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v98'46 uv46 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.516+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174d4200 2023-11-15T03:04:12.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.516+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(119 1.00000000 [create] v98'47 uv41 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.516+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174d4200 2023-11-15T03:04:12.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.520+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(120 1.00000000 [delete] v98'48 uv48 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.520+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174d4200 2023-11-15T03:04:12.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.524+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(121 1.00000000 [create] v98'49 uv49 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.530 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:04:12.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.524+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e3b40 con 0x5590174d4200 2023-11-15T03:04:12.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.524+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v98'50 uv50 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.524+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e3b40 con 0x5590174d4200 2023-11-15T03:04:12.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.528+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v98'51 uv51 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.528+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174d4200 2023-11-15T03:04:12.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.532+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v98'52 uv52 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.532+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e98) v8 -- 0x5590174e2b30 con 0x5590174d4200 2023-11-15T03:04:12.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.532+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v98'53 uv53 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f71b8024020 con 0x5590174d4200 2023-11-15T03:04:12.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.532+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5590174d0f20 msgr2=0x5590174d3310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.532+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5590174d0f20 0x5590174d3310 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.532+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x559017418c30 msgr2=0x55901741c790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.532+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x559017418c30 0x55901741c790 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.532+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5590174d86b0 msgr2=0x5590174daaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.532+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5590174d86b0 0x5590174daaa0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.532+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5590174cc3c0 msgr2=0x5590174ce7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.532+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5590174cc3c0 0x5590174ce7b0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.532+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5590174df830 msgr2=0x5590174e2640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.532+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5590174df830 0x5590174e2640 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.532+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5590174d4200 msgr2=0x5590174d65f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.532+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5590174d4200 0x5590174d65f0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.532+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 reap_dead start 2023-11-15T03:04:12.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.532+0000 7f71c0da2700 1 -- 172.21.15.121:0/3582279857 reap_dead start 2023-11-15T03:04:12.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.532+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5590174dc4d0 msgr2=0x5590174de8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.532+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5590174dc4d0 0x5590174de8c0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.536+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71a0058ff0 msgr2=0x7f71a005b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.536+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71a0058ff0 0x7f71a005b4b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f71b0006010 tx=0x7f71b0018000).stop 2023-11-15T03:04:12.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.536+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590174b1000 msgr2=0x5590174a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.536+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590174b1000 0x5590174a63b0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f71b800b010 tx=0x7f71b8014000).stop 2023-11-15T03:04:12.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.536+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 shutdown_connections 2023-11-15T03:04:12.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.536+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5590174dc4d0 0x5590174de8c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.536+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5590174d4200 0x5590174d65f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.536+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f71a0058ff0 0x7f71a005b4b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.536+0000 7f71cbf39d00 1 --2- 172.21.15.121:0/3582279857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590174b1000 0x5590174a63b0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.536+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 >> 172.21.15.121:0/3582279857 conn(0x559017420080 msgr2=0x5590174aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:12.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.536+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 shutdown_connections 2023-11-15T03:04:12.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.536+0000 7f71cbf39d00 1 -- 172.21.15.121:0/3582279857 wait complete. 2023-11-15T03:04:12.552 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 200.00000000 2023-11-15T03:04:12.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.652+0000 7f38bcd3ad00 1 Processor -- start 2023-11-15T03:04:12.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.652+0000 7f38bcd3ad00 1 -- start start 2023-11-15T03:04:12.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.652+0000 7f38bcd3ad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598a7635dd0 0x5598a77118d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.652+0000 7f38bcd3ad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598a767fb00 0x5598a767b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.652+0000 7f38bcd3ad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598a770e670 0x5598a771afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.652+0000 7f38bcd3ad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598a75e3cc0 con 0x5598a7635dd0 2023-11-15T03:04:12.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.652+0000 7f38bcd3ad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598a75ae220 con 0x5598a767fb00 2023-11-15T03:04:12.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.652+0000 7f38bcd3ad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598a770f7b0 con 0x5598a770e670 2023-11-15T03:04:12.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.652+0000 7f38b1245700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598a7635dd0 0x5598a77118d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.652+0000 7f38b1a46700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598a770e670 0x5598a771afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.652+0000 7f38a8a44700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598a767fb00 0x5598a767b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.652+0000 7f38b1245700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598a7635dd0 0x5598a77118d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:52960/0 (socket says 172.21.15.121:52960) 2023-11-15T03:04:12.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.652+0000 7f38b1245700 1 -- 172.21.15.121:0/3899446596 learned_addr learned my addr 172.21.15.121:0/3899446596 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:12.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.652+0000 7f38b1245700 1 -- 172.21.15.121:0/3899446596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598a770e670 msgr2=0x5598a771afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:12.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.652+0000 7f38b1245700 1 --2- 172.21.15.121:0/3899446596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598a770e670 0x5598a771afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.652+0000 7f38b1245700 1 -- 172.21.15.121:0/3899446596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598a767fb00 msgr2=0x5598a767b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:12.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.652+0000 7f38b1245700 1 --2- 172.21.15.121:0/3899446596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598a767fb00 0x5598a767b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.652+0000 7f38b1245700 1 -- 172.21.15.121:0/3899446596 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a4005cf0 con 0x5598a7635dd0 2023-11-15T03:04:12.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.652+0000 7f38b1245700 1 --2- 172.21.15.121:0/3899446596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598a7635dd0 0x5598a77118d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f38a4010000 tx=0x7f38a4016000).ready entity=mon.0 client_cookie=bd1e2ee81cbd57e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:12.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.656+0000 7f38abfff700 1 -- 172.21.15.121:0/3899446596 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a40053b0 con 0x5598a7635dd0 2023-11-15T03:04:12.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.656+0000 7f38abfff700 1 -- 172.21.15.121:0/3899446596 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a4000bc0 con 0x5598a7635dd0 2023-11-15T03:04:12.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.656+0000 7f38abfff700 1 -- 172.21.15.121:0/3899446596 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a40089d0 con 0x5598a7635dd0 2023-11-15T03:04:12.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.656+0000 7f38bcd3ad00 1 -- 172.21.15.121:0/3899446596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598a7635dd0 msgr2=0x5598a77118d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.656+0000 7f38bcd3ad00 1 --2- 172.21.15.121:0/3899446596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598a7635dd0 0x5598a77118d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f38a4010000 tx=0x7f38a4016000).stop 2023-11-15T03:04:12.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.656+0000 7f38bcd3ad00 1 -- 172.21.15.121:0/3899446596 shutdown_connections 2023-11-15T03:04:12.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.656+0000 7f38bcd3ad00 1 --2- 172.21.15.121:0/3899446596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598a770e670 0x5598a771afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.656+0000 7f38bcd3ad00 1 --2- 172.21.15.121:0/3899446596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598a767fb00 0x5598a767b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.656+0000 7f38bcd3ad00 1 --2- 172.21.15.121:0/3899446596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598a7635dd0 0x5598a77118d0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.656+0000 7f38bcd3ad00 1 -- 172.21.15.121:0/3899446596 >> 172.21.15.121:0/3899446596 conn(0x5598a7685080 msgr2=0x5598a7685460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:12.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.656+0000 7f38bcd3ad00 1 -- 172.21.15.121:0/3899446596 shutdown_connections 2023-11-15T03:04:12.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.656+0000 7f38bcd3ad00 1 -- 172.21.15.121:0/3899446596 wait complete. 2023-11-15T03:04:12.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.656+0000 7f38bcd3ad00 1 Processor -- start 2023-11-15T03:04:12.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.656+0000 7f38bcd3ad00 1 -- start start 2023-11-15T03:04:12.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38bcd3ad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598a767fb00 0x5598a767e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38bcd3ad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598a770e670 0x5598a7706510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38bcd3ad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598a7708050 0x5598a7706a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38bcd3ad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598a7706ef0 con 0x5598a770e670 2023-11-15T03:04:12.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38bcd3ad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598a7707290 con 0x5598a767fb00 2023-11-15T03:04:12.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38bcd3ad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598a7707570 con 0x5598a7708050 2023-11-15T03:04:12.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38a8a44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598a770e670 0x5598a7706510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38a8a44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598a770e670 0x5598a7706510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:52966/0 (socket says 172.21.15.121:52966) 2023-11-15T03:04:12.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38a8a44700 1 -- 172.21.15.121:0/3190391269 learned_addr learned my addr 172.21.15.121:0/3190391269 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:12.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38b1245700 1 --2- 172.21.15.121:0/3190391269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598a767fb00 0x5598a767e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38b1a46700 1 --2- 172.21.15.121:0/3190391269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598a7708050 0x5598a7706a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38a8a44700 1 -- 172.21.15.121:0/3190391269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598a7708050 msgr2=0x5598a7706a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38a8a44700 1 --2- 172.21.15.121:0/3190391269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598a7708050 0x5598a7706a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38a8a44700 1 -- 172.21.15.121:0/3190391269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598a767fb00 msgr2=0x5598a767e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38a8a44700 1 --2- 172.21.15.121:0/3190391269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598a767fb00 0x5598a767e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38a8a44700 1 -- 172.21.15.121:0/3190391269 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598a75d7770 con 0x5598a770e670 2023-11-15T03:04:12.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38b1245700 1 --2- 172.21.15.121:0/3190391269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598a767fb00 0x5598a767e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:12.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38a8a44700 1 --2- 172.21.15.121:0/3190391269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598a770e670 0x5598a7706510 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f389c002e80 tx=0x7f389c0017a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:12.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38aa7fc700 1 -- 172.21.15.121:0/3190391269 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c002b60 con 0x5598a770e670 2023-11-15T03:04:12.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38aa7fc700 1 -- 172.21.15.121:0/3190391269 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c008070 con 0x5598a770e670 2023-11-15T03:04:12.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38bcd3ad00 1 -- 172.21.15.121:0/3190391269 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5598a75a6a70 con 0x5598a770e670 2023-11-15T03:04:12.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38aa7fc700 1 -- 172.21.15.121:0/3190391269 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c00cd50 con 0x5598a770e670 2023-11-15T03:04:12.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.660+0000 7f38bcd3ad00 1 -- 172.21.15.121:0/3190391269 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598a75b2010 con 0x5598a770e670 2023-11-15T03:04:12.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.664+0000 7f38aa7fc700 1 -- 172.21.15.121:0/3190391269 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f389c011660 con 0x5598a770e670 2023-11-15T03:04:12.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.664+0000 7f38aa7fc700 1 --2- 172.21.15.121:0/3190391269 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3894059020 0x7f389405b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.664+0000 7f38aa7fc700 1 -- 172.21.15.121:0/3190391269 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6610+0+0 (secure 0 0 0) 0x7f389c073430 con 0x5598a770e670 2023-11-15T03:04:12.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.664+0000 7f38b1245700 1 --2- 172.21.15.121:0/3190391269 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3894059020 0x7f389405b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.664+0000 7f38bcd3ad00 1 --2- 172.21.15.121:0/3190391269 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5598a767b600 0x5598a77179d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.664+0000 7f38bcd3ad00 1 -- 172.21.15.121:0/3190391269 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e98) v8 -- 0x5598a7717ec0 con 0x5598a767b600 2023-11-15T03:04:12.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.664+0000 7f38b1a46700 1 --2- 172.21.15.121:0/3190391269 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5598a767b600 0x5598a77179d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.668+0000 7f38b1245700 1 --2- 172.21.15.121:0/3190391269 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3894059020 0x7f389405b4e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f38a4021000 tx=0x7f38a401c000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:12.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.668+0000 7f38b1a46700 1 --2- 172.21.15.121:0/3190391269 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5598a767b600 0x5598a77179d0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:12.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.668+0000 7f38b1a46700 1 -- 172.21.15.121:0/3190391269 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(1 200.00000000 [delete] v98'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38ac003450 con 0x5598a767b600 2023-11-15T03:04:12.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.668+0000 7f38bcd3ad00 1 -- 172.21.15.121:0/3190391269 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5598a767b600 msgr2=0x5598a77179d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.672+0000 7f38bcd3ad00 1 --2- 172.21.15.121:0/3190391269 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5598a767b600 0x5598a77179d0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.672+0000 7f38bcd3ad00 1 -- 172.21.15.121:0/3190391269 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3894059020 msgr2=0x7f389405b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.672+0000 7f38bcd3ad00 1 --2- 172.21.15.121:0/3190391269 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3894059020 0x7f389405b4e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f38a4021000 tx=0x7f38a401c000).stop 2023-11-15T03:04:12.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.672+0000 7f38bcd3ad00 1 -- 172.21.15.121:0/3190391269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598a770e670 msgr2=0x5598a7706510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.672+0000 7f38bcd3ad00 1 --2- 172.21.15.121:0/3190391269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598a770e670 0x5598a7706510 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f389c002e80 tx=0x7f389c0017a0).stop 2023-11-15T03:04:12.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.672+0000 7f38b1a46700 1 -- 172.21.15.121:0/3190391269 reap_dead start 2023-11-15T03:04:12.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.672+0000 7f38bcd3ad00 1 -- 172.21.15.121:0/3190391269 shutdown_connections 2023-11-15T03:04:12.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.672+0000 7f38bcd3ad00 1 -- 172.21.15.121:0/3190391269 >> 172.21.15.121:0/3190391269 conn(0x5598a7685080 msgr2=0x5598a7685460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:12.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.672+0000 7f38bcd3ad00 1 -- 172.21.15.121:0/3190391269 shutdown_connections 2023-11-15T03:04:12.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.672+0000 7f38bcd3ad00 1 -- 172.21.15.121:0/3190391269 wait complete. 2023-11-15T03:04:12.685 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:04:12.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.848+0000 7f535e507700 1 Processor -- start 2023-11-15T03:04:12.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.848+0000 7f535e507700 1 -- start start 2023-11-15T03:04:12.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.852+0000 7f535e507700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53580a8d40 0x7f53580c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.852+0000 7f535e507700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53580657f0 con 0x7f53580a8960 2023-11-15T03:04:12.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.852+0000 7f535e507700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53580c1b60 con 0x7f5358066520 2023-11-15T03:04:12.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.852+0000 7f535e507700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53581583d0 con 0x7f53580a8d40 2023-11-15T03:04:12.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.852+0000 7f535d505700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f53580a8960 0x7f535815c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53456/0 (socket says 172.21.15.121:53456) 2023-11-15T03:04:12.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.852+0000 7f535d505700 1 -- 172.21.15.121:0/3471349865 learned_addr learned my addr 172.21.15.121:0/3471349865 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:12.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.852+0000 7f535cd04700 1 --2- 172.21.15.121:0/3471349865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53580a8d40 0x7f53580c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.852+0000 7f534ffff700 1 -- 172.21.15.121:0/3471349865 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887275811 0 0) 0x7f53580657f0 con 0x7f53580a8960 2023-11-15T03:04:12.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.852+0000 7f534ffff700 1 -- 172.21.15.121:0/3471349865 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53440033f0 con 0x7f53580a8960 2023-11-15T03:04:12.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f534ffff700 1 -- 172.21.15.121:0/3471349865 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2519463421 0 0) 0x7f53440033f0 con 0x7f53580a8960 2023-11-15T03:04:12.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f534ffff700 1 -- 172.21.15.121:0/3471349865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53580a8d40 msgr2=0x7f53580c15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:12.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f534ffff700 1 --2- 172.21.15.121:0/3471349865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53580a8d40 0x7f53580c15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f534ffff700 1 -- 172.21.15.121:0/3471349865 >> v1:172.21.15.190:6789/0 conn(0x7f5358066520 legacy=0x7f53580c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f534ffff700 1 -- 172.21.15.121:0/3471349865 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5358065a40 con 0x7f53580a8960 2023-11-15T03:04:12.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f534ffff700 1 -- 172.21.15.121:0/3471349865 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f5354001470 con 0x7f53580a8960 2023-11-15T03:04:12.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f534ffff700 1 -- 172.21.15.121:0/3471349865 >> v1:172.21.15.121:6789/0 conn(0x7f53580a8960 legacy=0x7f535815c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f534ffff700 1 --2- 172.21.15.121:0/3471349865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5344003910 0x7f5344003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f534ffff700 1 --2- 172.21.15.121:0/3471349865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53580a8d40 0x7f5344004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f534ffff700 1 --2- 172.21.15.121:0/3471349865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5344004b80 0x7f5344005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f534ffff700 1 -- 172.21.15.121:0/3471349865 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53581583d0 con 0x7f5344004b80 2023-11-15T03:04:12.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f534ffff700 1 -- 172.21.15.121:0/3471349865 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534400b910 con 0x7f5344003910 2023-11-15T03:04:12.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f534ffff700 1 -- 172.21.15.121:0/3471349865 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534400bad0 con 0x7f53580a8d40 2023-11-15T03:04:12.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f535dd06700 1 --2- 172.21.15.121:0/3471349865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5344004b80 0x7f5344005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f535cd04700 1 --2- 172.21.15.121:0/3471349865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53580a8d40 0x7f5344004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f535d505700 1 --2- 172.21.15.121:0/3471349865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5344003910 0x7f5344003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f535dd06700 1 -- 172.21.15.121:0/3471349865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53580a8d40 msgr2=0x7f5344004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:12.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f535dd06700 1 --2- 172.21.15.121:0/3471349865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53580a8d40 0x7f5344004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f535dd06700 1 -- 172.21.15.121:0/3471349865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5344003910 msgr2=0x7f5344003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:12.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f535dd06700 1 --2- 172.21.15.121:0/3471349865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5344003910 0x7f5344003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f535dd06700 1 -- 172.21.15.121:0/3471349865 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53440033f0 con 0x7f5344004b80 2023-11-15T03:04:12.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f535d505700 1 --2- 172.21.15.121:0/3471349865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5344003910 0x7f5344003d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:12.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f535cd04700 1 --2- 172.21.15.121:0/3471349865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53580a8d40 0x7f5344004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:12.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f535dd06700 1 --2- 172.21.15.121:0/3471349865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5344004b80 0x7f5344005000 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7f5350001e30 tx=0x7f5350007f30).ready entity=mon.0 client_cookie=444e0ee0302e3969 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:12.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f534ffff700 1 -- 172.21.15.121:0/3471349865 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350006a20 con 0x7f5344004b80 2023-11-15T03:04:12.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f534ffff700 1 -- 172.21.15.121:0/3471349865 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5350006b80 con 0x7f5344004b80 2023-11-15T03:04:12.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f534ffff700 1 -- 172.21.15.121:0/3471349865 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535000c5d0 con 0x7f5344004b80 2023-11-15T03:04:12.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f535e507700 1 -- 172.21.15.121:0/3471349865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5344004b80 msgr2=0x7f5344005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f535e507700 1 --2- 172.21.15.121:0/3471349865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5344004b80 0x7f5344005000 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7f5350001e30 tx=0x7f5350007f30).stop 2023-11-15T03:04:12.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f535dd06700 1 -- 172.21.15.121:0/3471349865 reap_dead start 2023-11-15T03:04:12.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f535e507700 1 -- 172.21.15.121:0/3471349865 shutdown_connections 2023-11-15T03:04:12.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f535e507700 1 -- 172.21.15.121:0/3471349865 >> 172.21.15.121:0/3471349865 conn(0x7f5358067a60 msgr2=0x7f5358067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:12.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f535e507700 1 -- 172.21.15.121:0/3471349865 shutdown_connections 2023-11-15T03:04:12.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.856+0000 7f535e507700 1 -- 172.21.15.121:0/3471349865 wait complete. 2023-11-15T03:04:12.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.860+0000 7f535e507700 1 Processor -- start 2023-11-15T03:04:12.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.860+0000 7f535e507700 1 -- start start 2023-11-15T03:04:12.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.860+0000 7f535e507700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5344004b80 0x7f535815b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.860+0000 7f535e507700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5358143af0 0x7f53580a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.860+0000 7f535e507700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53580aa3d0 0x7f53581595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.860+0000 7f535e507700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535815af80 con 0x7f5344004b80 2023-11-15T03:04:12.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.860+0000 7f535e507700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53580aa850 con 0x7f5358143af0 2023-11-15T03:04:12.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.860+0000 7f535e507700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535815a370 con 0x7f53580aa3d0 2023-11-15T03:04:12.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.864+0000 7f535d505700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5344004b80 0x7f535815b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.864+0000 7f535d505700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5344004b80 0x7f535815b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:52994/0 (socket says 172.21.15.121:52994) 2023-11-15T03:04:12.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.864+0000 7f535d505700 1 -- 172.21.15.121:0/4044466676 learned_addr learned my addr 172.21.15.121:0/4044466676 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:12.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.864+0000 7f535cd04700 1 --2- 172.21.15.121:0/4044466676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5358143af0 0x7f53580a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.864+0000 7f535dd06700 1 --2- 172.21.15.121:0/4044466676 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53580aa3d0 0x7f53581595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.864+0000 7f535d505700 1 -- 172.21.15.121:0/4044466676 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53580aa3d0 msgr2=0x7f53581595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.864+0000 7f535d505700 1 --2- 172.21.15.121:0/4044466676 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53580aa3d0 0x7f53581595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.864+0000 7f535d505700 1 -- 172.21.15.121:0/4044466676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5358143af0 msgr2=0x7f53580a9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:12.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.864+0000 7f535d505700 1 --2- 172.21.15.121:0/4044466676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5358143af0 0x7f53580a9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:12.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.864+0000 7f535d505700 1 -- 172.21.15.121:0/4044466676 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535815a650 con 0x7f5344004b80 2023-11-15T03:04:12.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.864+0000 7f535d505700 1 --2- 172.21.15.121:0/4044466676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5344004b80 0x7f535815b410 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7f535400e790 tx=0x7f53540014c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:12.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.864+0000 7f534d7fa700 1 -- 172.21.15.121:0/4044466676 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354010490 con 0x7f5344004b80 2023-11-15T03:04:12.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.864+0000 7f534d7fa700 1 -- 172.21.15.121:0/4044466676 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5354010040 con 0x7f5344004b80 2023-11-15T03:04:12.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.864+0000 7f535e507700 1 -- 172.21.15.121:0/4044466676 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535815a930 con 0x7f5344004b80 2023-11-15T03:04:12.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.864+0000 7f534d7fa700 1 -- 172.21.15.121:0/4044466676 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354010800 con 0x7f5344004b80 2023-11-15T03:04:12.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.864+0000 7f535e507700 1 -- 172.21.15.121:0/4044466676 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535815b900 con 0x7f5344004b80 2023-11-15T03:04:12.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.864+0000 7f535e507700 1 -- 172.21.15.121:0/4044466676 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535815adb0 con 0x7f5344004b80 2023-11-15T03:04:12.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.868+0000 7f534d7fa700 1 -- 172.21.15.121:0/4044466676 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5354010bf0 con 0x7f5344004b80 2023-11-15T03:04:12.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.868+0000 7f534d7fa700 1 --2- 172.21.15.121:0/4044466676 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f533c05a540 0x7f533c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:12.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.868+0000 7f534d7fa700 1 -- 172.21.15.121:0/4044466676 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6610+0+0 (secure 0 0 0) 0x7f53540741d0 con 0x7f5344004b80 2023-11-15T03:04:12.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.868+0000 7f535cd04700 1 --2- 172.21.15.121:0/4044466676 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f533c05a540 0x7f533c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:12.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.868+0000 7f535cd04700 1 --2- 172.21.15.121:0/4044466676 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f533c05a540 0x7f533c05ca00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f53580aadb0 tx=0x7f5348008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:12.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:12.872+0000 7f534d7fa700 1 -- 172.21.15.121:0/4044466676 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5354047700 con 0x7f5344004b80 2023-11-15T03:04:13.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:13.072+0000 7f535e507700 1 -- 172.21.15.121:0/4044466676 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f535806bcf0 con 0x7f5344004b80 2023-11-15T03:04:14.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.076+0000 7f534d7fa700 1 -- 172.21.15.121:0/4044466676 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v107) v1 ==== 162+0+0 (secure 0 0 0) 0x7f535404ad20 con 0x7f5344004b80 2023-11-15T03:04:14.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.080+0000 7f535e507700 1 -- 172.21.15.121:0/4044466676 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f533c05a540 msgr2=0x7f533c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:14.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.080+0000 7f535e507700 1 --2- 172.21.15.121:0/4044466676 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f533c05a540 0x7f533c05ca00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f53580aadb0 tx=0x7f5348008040).stop 2023-11-15T03:04:14.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.080+0000 7f535e507700 1 -- 172.21.15.121:0/4044466676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5344004b80 msgr2=0x7f535815b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:14.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.080+0000 7f535e507700 1 --2- 172.21.15.121:0/4044466676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5344004b80 0x7f535815b410 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7f535400e790 tx=0x7f53540014c0).stop 2023-11-15T03:04:14.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.080+0000 7f535e507700 1 -- 172.21.15.121:0/4044466676 shutdown_connections 2023-11-15T03:04:14.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.080+0000 7f535e507700 1 --2- 172.21.15.121:0/4044466676 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f533c05a540 0x7f533c05ca00 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:14.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.080+0000 7f535e507700 1 --2- 172.21.15.121:0/4044466676 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53580aa3d0 0x7f53581595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:14.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.080+0000 7f535e507700 1 --2- 172.21.15.121:0/4044466676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5358143af0 0x7f53580a9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:14.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.080+0000 7f535e507700 1 --2- 172.21.15.121:0/4044466676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5344004b80 0x7f535815b410 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:14.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.080+0000 7f535e507700 1 -- 172.21.15.121:0/4044466676 >> 172.21.15.121:0/4044466676 conn(0x7f5358067a60 msgr2=0x7f53580aae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:14.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.080+0000 7f535e507700 1 -- 172.21.15.121:0/4044466676 shutdown_connections 2023-11-15T03:04:14.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.080+0000 7f535e507700 1 -- 172.21.15.121:0/4044466676 wait complete. 2023-11-15T03:04:14.089 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:04:14.093 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:04:14.088+0000 7fc31e096700 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (2) No such file or directory 2023-11-15T03:04:14.109 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:04:14.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.260+0000 7fd059c39700 1 Processor -- start 2023-11-15T03:04:14.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.260+0000 7fd059c39700 1 -- start start 2023-11-15T03:04:14.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.260+0000 7fd059c39700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd054151870 0x7fd054151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:14.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.260+0000 7fd059c39700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd054152140 0x7fd0540a9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:14.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.260+0000 7fd059c39700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd0540a9d20 0x7fd0540c30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:14.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.260+0000 7fd059c39700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0540657f0 con 0x7fd0540a9d20 2023-11-15T03:04:14.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.260+0000 7fd059c39700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd054065ad0 con 0x7fd054151870 2023-11-15T03:04:14.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.260+0000 7fd059c39700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540aa260 con 0x7fd054152140 2023-11-15T03:04:14.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.260+0000 7fd059438700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd0540a9d20 0x7fd0540c30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:14.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.260+0000 7fd04bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd054152140 0x7fd0540a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:14.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd04bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd054152140 0x7fd0540a9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:34446/0 (socket says 172.21.15.121:34446) 2023-11-15T03:04:14.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.260+0000 7fd058c37700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd054151870 0x7fd054151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:14.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd04bfff700 1 -- 172.21.15.121:0/851841118 learned_addr learned my addr 172.21.15.121:0/851841118 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:14.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd059438700 1 -- 172.21.15.121:0/851841118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd054152140 msgr2=0x7fd0540a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:14.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd059438700 1 --2- 172.21.15.121:0/851841118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd054152140 0x7fd0540a9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:14.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd059438700 1 -- 172.21.15.121:0/851841118 >> v2:172.21.15.190:3300/0 conn(0x7fd054151870 msgr2=0x7fd054151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:14.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd059438700 1 --2- 172.21.15.121:0/851841118 >> v2:172.21.15.190:3300/0 conn(0x7fd054151870 0x7fd054151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:14.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd059438700 1 -- 172.21.15.121:0/851841118 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0540aa540 con 0x7fd0540a9d20 2023-11-15T03:04:14.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd04bfff700 1 --2- 172.21.15.121:0/851841118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd054152140 0x7fd0540a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:04:14.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd059438700 1 --2- 172.21.15.121:0/851841118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0540a9d20 0x7fd0540c30e0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7fd054067720 tx=0x7fd04c009780).ready entity=mon.0 client_cookie=219871783821da72 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:14.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd04b7fe700 1 -- 172.21.15.121:0/851841118 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c00e070 con 0x7fd0540a9d20 2023-11-15T03:04:14.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd04b7fe700 1 -- 172.21.15.121:0/851841118 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04c00bab0 con 0x7fd0540a9d20 2023-11-15T03:04:14.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd04b7fe700 1 -- 172.21.15.121:0/851841118 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c00be30 con 0x7fd0540a9d20 2023-11-15T03:04:14.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd059c39700 1 -- 172.21.15.121:0/851841118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0540a9d20 msgr2=0x7fd0540c30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:14.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd059c39700 1 --2- 172.21.15.121:0/851841118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0540a9d20 0x7fd0540c30e0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7fd054067720 tx=0x7fd04c009780).stop 2023-11-15T03:04:14.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd059c39700 1 -- 172.21.15.121:0/851841118 shutdown_connections 2023-11-15T03:04:14.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd059c39700 1 --2- 172.21.15.121:0/851841118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0540a9d20 0x7fd0540c30e0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:14.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd059c39700 1 --2- 172.21.15.121:0/851841118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd054152140 0x7fd0540a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:14.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd059c39700 1 --2- 172.21.15.121:0/851841118 >> v2:172.21.15.190:3300/0 conn(0x7fd054151870 0x7fd054151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:14.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd059c39700 1 -- 172.21.15.121:0/851841118 >> 172.21.15.121:0/851841118 conn(0x7fd054062190 msgr2=0x7fd054062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:14.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd059c39700 1 -- 172.21.15.121:0/851841118 shutdown_connections 2023-11-15T03:04:14.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.264+0000 7fd059c39700 1 -- 172.21.15.121:0/851841118 wait complete. 2023-11-15T03:04:14.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.268+0000 7fd059c39700 1 Processor -- start 2023-11-15T03:04:14.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.268+0000 7fd059c39700 1 -- start start 2023-11-15T03:04:14.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.268+0000 7fd059c39700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0540a9d20 0x7fd054139a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:14.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.268+0000 7fd059c39700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd054151870 0x7fd0540c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:14.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.268+0000 7fd059c39700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd054152140 0x7fd0540c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:14.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.268+0000 7fd059c39700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540c6910 con 0x7fd0540a9d20 2023-11-15T03:04:14.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.268+0000 7fd059c39700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540c6bf0 con 0x7fd054152140 2023-11-15T03:04:14.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.268+0000 7fd059c39700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540c6ed0 con 0x7fd054151870 2023-11-15T03:04:14.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.272+0000 7fd058c37700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0540a9d20 0x7fd054139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:14.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.272+0000 7fd058c37700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0540a9d20 0x7fd054139a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53024/0 (socket says 172.21.15.121:53024) 2023-11-15T03:04:14.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.272+0000 7fd058c37700 1 -- 172.21.15.121:0/1963136523 learned_addr learned my addr 172.21.15.121:0/1963136523 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:14.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.272+0000 7fd04bfff700 1 --2- 172.21.15.121:0/1963136523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd054151870 0x7fd0540c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:14.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.272+0000 7fd059438700 1 --2- 172.21.15.121:0/1963136523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd054152140 0x7fd0540c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:14.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.272+0000 7fd058c37700 1 -- 172.21.15.121:0/1963136523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd054151870 msgr2=0x7fd0540c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:14.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.272+0000 7fd058c37700 1 --2- 172.21.15.121:0/1963136523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd054151870 0x7fd0540c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:14.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.272+0000 7fd058c37700 1 -- 172.21.15.121:0/1963136523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd054152140 msgr2=0x7fd0540c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:14.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.272+0000 7fd058c37700 1 --2- 172.21.15.121:0/1963136523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd054152140 0x7fd0540c6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:14.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.272+0000 7fd058c37700 1 -- 172.21.15.121:0/1963136523 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd044009370 con 0x7fd0540a9d20 2023-11-15T03:04:14.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.272+0000 7fd058c37700 1 --2- 172.21.15.121:0/1963136523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0540a9d20 0x7fd054139a60 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7fd05000e930 tx=0x7fd050009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:14.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.272+0000 7fd048ff9700 1 -- 172.21.15.121:0/1963136523 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05000eed0 con 0x7fd0540a9d20 2023-11-15T03:04:14.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.272+0000 7fd048ff9700 1 -- 172.21.15.121:0/1963136523 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05000c850 con 0x7fd0540a9d20 2023-11-15T03:04:14.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.272+0000 7fd059c39700 1 -- 172.21.15.121:0/1963136523 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04c009a20 con 0x7fd0540a9d20 2023-11-15T03:04:14.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.272+0000 7fd048ff9700 1 -- 172.21.15.121:0/1963136523 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05000cbd0 con 0x7fd0540a9d20 2023-11-15T03:04:14.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.272+0000 7fd059c39700 1 -- 172.21.15.121:0/1963136523 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0541627b0 con 0x7fd0540a9d20 2023-11-15T03:04:14.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.272+0000 7fd059c39700 1 -- 172.21.15.121:0/1963136523 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05406bcf0 con 0x7fd0540a9d20 2023-11-15T03:04:14.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.276+0000 7fd048ff9700 1 -- 172.21.15.121:0/1963136523 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd05001b400 con 0x7fd0540a9d20 2023-11-15T03:04:14.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.276+0000 7fd048ff9700 1 --2- 172.21.15.121:0/1963136523 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd04005a510 0x7fd04005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:14.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.276+0000 7fd048ff9700 1 -- 172.21.15.121:0/1963136523 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6696+0+0 (secure 0 0 0) 0x7fd050075110 con 0x7fd0540a9d20 2023-11-15T03:04:14.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.276+0000 7fd04bfff700 1 --2- 172.21.15.121:0/1963136523 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd04005a510 0x7fd04005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:14.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.276+0000 7fd04bfff700 1 --2- 172.21.15.121:0/1963136523 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd04005a510 0x7fd04005c9d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fd044009340 tx=0x7fd044008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:14.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.280+0000 7fd048ff9700 1 -- 172.21.15.121:0/1963136523 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd050048590 con 0x7fd0540a9d20 2023-11-15T03:04:14.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.480+0000 7fd059c39700 1 -- 172.21.15.121:0/1963136523 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0540aa820 con 0x7fd0540a9d20 2023-11-15T03:04:14.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.480+0000 7fd048ff9700 1 -- 172.21.15.121:0/1963136523 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4498 (secure 0 0 0) 0x7fd05004bbb0 con 0x7fd0540a9d20 2023-11-15T03:04:14.487 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:04:14.488 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":108,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:04:14.078039+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7770},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7770":{"gid":7770,"name":"a","rank":0,"incarnation":108,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/57668150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":57668150},{"type":"v1","addr":"172.21.15.121:6836","nonce":57668150}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:04:14.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.484+0000 7fd059c39700 1 -- 172.21.15.121:0/1963136523 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd04005a510 msgr2=0x7fd04005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:14.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.484+0000 7fd059c39700 1 --2- 172.21.15.121:0/1963136523 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd04005a510 0x7fd04005c9d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fd044009340 tx=0x7fd044008040).stop 2023-11-15T03:04:14.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.484+0000 7fd059c39700 1 -- 172.21.15.121:0/1963136523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0540a9d20 msgr2=0x7fd054139a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:14.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.484+0000 7fd059c39700 1 --2- 172.21.15.121:0/1963136523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0540a9d20 0x7fd054139a60 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7fd05000e930 tx=0x7fd050009060).stop 2023-11-15T03:04:14.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.484+0000 7fd059c39700 1 -- 172.21.15.121:0/1963136523 shutdown_connections 2023-11-15T03:04:14.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.484+0000 7fd059c39700 1 --2- 172.21.15.121:0/1963136523 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd04005a510 0x7fd04005c9d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:14.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.484+0000 7fd059c39700 1 --2- 172.21.15.121:0/1963136523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd054152140 0x7fd0540c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:14.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.484+0000 7fd059c39700 1 --2- 172.21.15.121:0/1963136523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd054151870 0x7fd0540c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:14.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.484+0000 7fd059c39700 1 --2- 172.21.15.121:0/1963136523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0540a9d20 0x7fd054139a60 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:14.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.484+0000 7fd059c39700 1 -- 172.21.15.121:0/1963136523 >> 172.21.15.121:0/1963136523 conn(0x7fd054062190 msgr2=0x7fd05414f030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:14.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.484+0000 7fd059c39700 1 -- 172.21.15.121:0/1963136523 shutdown_connections 2023-11-15T03:04:14.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:14.484+0000 7fd059c39700 1 -- 172.21.15.121:0/1963136523 wait complete. 2023-11-15T03:04:14.495 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 108 2023-11-15T03:04:14.514 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:04:15.091 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:04:15.080+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:04:15.092 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:04:15.080+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:04:15.106 INFO:tasks.ceph.mds.a.smithi121.stderr: -16> 2023-11-15T03:04:14.088+0000 7fc31e096700 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (2) No such file or directory 2023-11-15T03:04:15.145 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:04:19.515 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:04:19.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.672+0000 7f516adb9700 1 Processor -- start 2023-11-15T03:04:19.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.672+0000 7f516adb9700 1 -- start start 2023-11-15T03:04:19.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.676+0000 7f516adb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5164152850 0x7f51640c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:19.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.676+0000 7f516adb9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51640c1d50 con 0x7f516415be30 2023-11-15T03:04:19.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.676+0000 7f516adb9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516415cb80 con 0x7f51640c3280 2023-11-15T03:04:19.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.676+0000 7f516adb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51640c1860 con 0x7f5164152850 2023-11-15T03:04:19.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.676+0000 7f516a5b8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f516415be30 0x7f516415c280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44180/0 (socket says 172.21.15.121:44180) 2023-11-15T03:04:19.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.676+0000 7f516a5b8700 1 -- 172.21.15.121:0/3789126927 learned_addr learned my addr 172.21.15.121:0/3789126927 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:19.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.676+0000 7f51695b6700 1 --2- 172.21.15.121:0/3789126927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5164152850 0x7f51640c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:19.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.676+0000 7f5168db5700 1 -- 172.21.15.121:0/3789126927 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 485806921 0 0) 0x7f51640c1d50 con 0x7f516415be30 2023-11-15T03:04:19.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.676+0000 7f5168db5700 1 -- 172.21.15.121:0/3789126927 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51500033f0 con 0x7f516415be30 2023-11-15T03:04:19.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.676+0000 7f51695b6700 1 -- 172.21.15.121:0/3789126927 >> v1:172.21.15.190:6789/0 conn(0x7f51640c3280 legacy=0x7f5164152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:19.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.676+0000 7f51695b6700 1 -- 172.21.15.121:0/3789126927 >> v1:172.21.15.121:6789/0 conn(0x7f516415be30 legacy=0x7f516415c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:19.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.676+0000 7f51695b6700 1 -- 172.21.15.121:0/3789126927 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51640a9240 con 0x7f5164152850 2023-11-15T03:04:19.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.676+0000 7f51695b6700 1 --2- 172.21.15.121:0/3789126927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5164152850 0x7f51640c3660 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f5154009790 tx=0x7f5154009060).ready entity=mon.2 client_cookie=65ad138475062257 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:19.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.680+0000 7f5168db5700 1 -- 172.21.15.121:0/3789126927 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515400f510 con 0x7f5164152850 2023-11-15T03:04:19.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.680+0000 7f5168db5700 1 -- 172.21.15.121:0/3789126927 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515400f670 con 0x7f5164152850 2023-11-15T03:04:19.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.680+0000 7f5168db5700 1 -- 172.21.15.121:0/3789126927 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515400f9f0 con 0x7f5164152850 2023-11-15T03:04:19.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.680+0000 7f516adb9700 1 -- 172.21.15.121:0/3789126927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5164152850 msgr2=0x7f51640c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:19.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.680+0000 7f516adb9700 1 --2- 172.21.15.121:0/3789126927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5164152850 0x7f51640c3660 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f5154009790 tx=0x7f5154009060).stop 2023-11-15T03:04:19.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.680+0000 7f516adb9700 1 -- 172.21.15.121:0/3789126927 shutdown_connections 2023-11-15T03:04:19.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.680+0000 7f516adb9700 1 --2- 172.21.15.121:0/3789126927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5164152850 0x7f51640c3660 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:19.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.680+0000 7f516adb9700 1 -- 172.21.15.121:0/3789126927 >> 172.21.15.121:0/3789126927 conn(0x7f5164062190 msgr2=0x7f5164062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:19.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.680+0000 7f516adb9700 1 -- 172.21.15.121:0/3789126927 shutdown_connections 2023-11-15T03:04:19.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.680+0000 7f516adb9700 1 -- 172.21.15.121:0/3789126927 wait complete. 2023-11-15T03:04:19.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.680+0000 7f516adb9700 1 Processor -- start 2023-11-15T03:04:19.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.684+0000 7f516adb9700 1 -- start start 2023-11-15T03:04:19.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.684+0000 7f516adb9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51640c3280 0x7f516414d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:19.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.684+0000 7f516adb9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5164152850 0x7f5164066840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:19.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.684+0000 7f516adb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f516415be30 0x7f5164066d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:19.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.684+0000 7f516adb9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5164067220 con 0x7f5164152850 2023-11-15T03:04:19.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.684+0000 7f516adb9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516414da50 con 0x7f51640c3280 2023-11-15T03:04:19.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.684+0000 7f516adb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51640c65a0 con 0x7f516415be30 2023-11-15T03:04:19.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.684+0000 7f51695b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5164152850 0x7f5164066840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:19.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.684+0000 7f51695b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5164152850 0x7f5164066840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56384/0 (socket says 172.21.15.121:56384) 2023-11-15T03:04:19.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.684+0000 7f51695b6700 1 -- 172.21.15.121:0/1055036946 learned_addr learned my addr 172.21.15.121:0/1055036946 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:19.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.684+0000 7f5169db7700 1 --2- 172.21.15.121:0/1055036946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51640c3280 0x7f516414d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:19.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.684+0000 7f516a5b8700 1 --2- 172.21.15.121:0/1055036946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f516415be30 0x7f5164066d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:19.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.684+0000 7f51695b6700 1 -- 172.21.15.121:0/1055036946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f516415be30 msgr2=0x7f5164066d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:19.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.684+0000 7f51695b6700 1 --2- 172.21.15.121:0/1055036946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f516415be30 0x7f5164066d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:19.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.684+0000 7f51695b6700 1 -- 172.21.15.121:0/1055036946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51640c3280 msgr2=0x7f516414d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:19.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.684+0000 7f51695b6700 1 --2- 172.21.15.121:0/1055036946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51640c3280 0x7f516414d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:19.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.684+0000 7f51695b6700 1 -- 172.21.15.121:0/1055036946 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51540092a0 con 0x7f5164152850 2023-11-15T03:04:19.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.684+0000 7f51695b6700 1 --2- 172.21.15.121:0/1055036946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5164152850 0x7f5164066840 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7f515400f860 tx=0x7f5154017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:19.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.688+0000 7f5159ffb700 1 -- 172.21.15.121:0/1055036946 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5154023430 con 0x7f5164152850 2023-11-15T03:04:19.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.688+0000 7f5159ffb700 1 -- 172.21.15.121:0/1055036946 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515401e070 con 0x7f5164152850 2023-11-15T03:04:19.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.688+0000 7f516adb9700 1 -- 172.21.15.121:0/1055036946 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51640c51d0 con 0x7f5164152850 2023-11-15T03:04:19.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.688+0000 7f5159ffb700 1 -- 172.21.15.121:0/1055036946 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5154023800 con 0x7f5164152850 2023-11-15T03:04:19.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.688+0000 7f516adb9700 1 -- 172.21.15.121:0/1055036946 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51640c6b20 con 0x7f5164152850 2023-11-15T03:04:19.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.688+0000 7f516adb9700 1 -- 172.21.15.121:0/1055036946 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516406bcf0 con 0x7f5164152850 2023-11-15T03:04:19.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.688+0000 7f5159ffb700 1 -- 172.21.15.121:0/1055036946 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5154020030 con 0x7f5164152850 2023-11-15T03:04:19.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.692+0000 7f5159ffb700 1 --2- 172.21.15.121:0/1055036946 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f514805a510 0x7f514805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:19.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.692+0000 7f5159ffb700 1 -- 172.21.15.121:0/1055036946 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6696+0+0 (secure 0 0 0) 0x7f5154075a00 con 0x7f5164152850 2023-11-15T03:04:19.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.692+0000 7f5169db7700 1 --2- 172.21.15.121:0/1055036946 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f514805a510 0x7f514805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:19.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.692+0000 7f5169db7700 1 --2- 172.21.15.121:0/1055036946 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f514805a510 0x7f514805c9d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f51641531a0 tx=0x7f5160009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:19.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.696+0000 7f5159ffb700 1 -- 172.21.15.121:0/1055036946 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5154048f40 con 0x7f5164152850 2023-11-15T03:04:19.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.892+0000 7f516adb9700 1 -- 172.21.15.121:0/1055036946 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51640a7cc0 con 0x7f5164152850 2023-11-15T03:04:19.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.892+0000 7f5159ffb700 1 -- 172.21.15.121:0/1055036946 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4495 (secure 0 0 0) 0x7f515404c560 con 0x7f5164152850 2023-11-15T03:04:19.900 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:04:19.901 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":110,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7836,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3956345798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3956345798},{"type":"v1","addr":"172.21.15.121:6836","nonce":3956345798}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:04:15.085570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:04:19.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.896+0000 7f516adb9700 1 -- 172.21.15.121:0/1055036946 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f514805a510 msgr2=0x7f514805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:19.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.896+0000 7f516adb9700 1 --2- 172.21.15.121:0/1055036946 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f514805a510 0x7f514805c9d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f51641531a0 tx=0x7f5160009040).stop 2023-11-15T03:04:19.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.896+0000 7f516adb9700 1 -- 172.21.15.121:0/1055036946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5164152850 msgr2=0x7f5164066840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:19.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.896+0000 7f516adb9700 1 --2- 172.21.15.121:0/1055036946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5164152850 0x7f5164066840 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7f515400f860 tx=0x7f5154017000).stop 2023-11-15T03:04:19.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.896+0000 7f516adb9700 1 -- 172.21.15.121:0/1055036946 shutdown_connections 2023-11-15T03:04:19.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.896+0000 7f516adb9700 1 --2- 172.21.15.121:0/1055036946 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f514805a510 0x7f514805c9d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:19.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.896+0000 7f516adb9700 1 --2- 172.21.15.121:0/1055036946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f516415be30 0x7f5164066d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:19.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.896+0000 7f516adb9700 1 --2- 172.21.15.121:0/1055036946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5164152850 0x7f5164066840 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:19.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.896+0000 7f516adb9700 1 --2- 172.21.15.121:0/1055036946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51640c3280 0x7f516414d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:19.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.896+0000 7f516adb9700 1 -- 172.21.15.121:0/1055036946 >> 172.21.15.121:0/1055036946 conn(0x7f5164062190 msgr2=0x7f516415a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:19.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.896+0000 7f516adb9700 1 -- 172.21.15.121:0/1055036946 shutdown_connections 2023-11-15T03:04:19.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:19.896+0000 7f516adb9700 1 -- 172.21.15.121:0/1055036946 wait complete. 2023-11-15T03:04:19.907 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 110 2023-11-15T03:04:19.923 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:04:19.924 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:04:20.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.080+0000 7fa30f479700 1 Processor -- start 2023-11-15T03:04:20.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.080+0000 7fa30f479700 1 -- start start 2023-11-15T03:04:20.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.080+0000 7fa30f479700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa308151870 0x7fa308151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:20.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.080+0000 7fa30f479700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3080c30e0 0x7fa3080c3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:20.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.080+0000 7fa30f479700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3080c3a80 con 0x7fa308152140 2023-11-15T03:04:20.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.080+0000 7fa30f479700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3080657f0 con 0x7fa308151870 2023-11-15T03:04:20.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.080+0000 7fa30f479700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa308065ad0 con 0x7fa3080c30e0 2023-11-15T03:04:20.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30e477700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa308151870 0x7fa308151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:20.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30e477700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa308151870 0x7fa308151c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56396/0 (socket says 172.21.15.121:56396) 2023-11-15T03:04:20.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30e477700 1 -- 172.21.15.121:0/1002146454 learned_addr learned my addr 172.21.15.121:0/1002146454 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:20.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30ec78700 1 --2- 172.21.15.121:0/1002146454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3080c30e0 0x7fa3080c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:20.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30e477700 1 -- 172.21.15.121:0/1002146454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3080c30e0 msgr2=0x7fa3080c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:20.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30e477700 1 --2- 172.21.15.121:0/1002146454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3080c30e0 0x7fa3080c3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:20.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30e477700 1 -- 172.21.15.121:0/1002146454 >> v1:172.21.15.190:6789/0 conn(0x7fa308152140 legacy=0x7fa3080a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:20.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30e477700 1 -- 172.21.15.121:0/1002146454 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3080aa260 con 0x7fa308151870 2023-11-15T03:04:20.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30e477700 1 --2- 172.21.15.121:0/1002146454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa308151870 0x7fa308151c50 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7fa304006fd0 tx=0x7fa3040090f0).ready entity=mon.0 client_cookie=aea738670e35c31d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:20.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30d475700 1 -- 172.21.15.121:0/1002146454 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30400b580 con 0x7fa308151870 2023-11-15T03:04:20.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30d475700 1 -- 172.21.15.121:0/1002146454 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30400b6e0 con 0x7fa308151870 2023-11-15T03:04:20.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30d475700 1 -- 172.21.15.121:0/1002146454 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30400ba60 con 0x7fa308151870 2023-11-15T03:04:20.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30f479700 1 -- 172.21.15.121:0/1002146454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa308151870 msgr2=0x7fa308151c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:20.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30f479700 1 --2- 172.21.15.121:0/1002146454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa308151870 0x7fa308151c50 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7fa304006fd0 tx=0x7fa3040090f0).stop 2023-11-15T03:04:20.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30f479700 1 -- 172.21.15.121:0/1002146454 shutdown_connections 2023-11-15T03:04:20.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30f479700 1 --2- 172.21.15.121:0/1002146454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3080c30e0 0x7fa3080c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:20.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30f479700 1 --2- 172.21.15.121:0/1002146454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa308151870 0x7fa308151c50 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:20.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30f479700 1 -- 172.21.15.121:0/1002146454 >> 172.21.15.121:0/1002146454 conn(0x7fa308062190 msgr2=0x7fa308062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:20.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30f479700 1 -- 172.21.15.121:0/1002146454 shutdown_connections 2023-11-15T03:04:20.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.084+0000 7fa30f479700 1 -- 172.21.15.121:0/1002146454 wait complete. 2023-11-15T03:04:20.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.088+0000 7fa30f479700 1 Processor -- start 2023-11-15T03:04:20.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.088+0000 7fa30f479700 1 -- start start 2023-11-15T03:04:20.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa30f479700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3080c30e0 0x7fa3081683c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:20.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa30f479700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa308151870 0x7fa3080a8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:20.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa30f479700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa308152140 0x7fa30814c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:20.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa30f479700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3081688b0 con 0x7fa3080c30e0 2023-11-15T03:04:20.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa30f479700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3080a90f0 con 0x7fa308152140 2023-11-15T03:04:20.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa30f479700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3080c2760 con 0x7fa308151870 2023-11-15T03:04:20.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa30e477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3080c30e0 0x7fa3081683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:20.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa30e477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3080c30e0 0x7fa3081683c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56406/0 (socket says 172.21.15.121:56406) 2023-11-15T03:04:20.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa30e477700 1 -- 172.21.15.121:0/3613797381 learned_addr learned my addr 172.21.15.121:0/3613797381 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:20.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa30dc76700 1 --2- 172.21.15.121:0/3613797381 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa308151870 0x7fa3080a8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:20.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa30ec78700 1 --2- 172.21.15.121:0/3613797381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa308152140 0x7fa30814c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:20.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa30e477700 1 -- 172.21.15.121:0/3613797381 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa308151870 msgr2=0x7fa3080a8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:20.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa30e477700 1 --2- 172.21.15.121:0/3613797381 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa308151870 0x7fa3080a8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:20.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa30e477700 1 -- 172.21.15.121:0/3613797381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa308152140 msgr2=0x7fa30814c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:20.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa30e477700 1 --2- 172.21.15.121:0/3613797381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa308152140 0x7fa30814c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:20.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa30e477700 1 -- 172.21.15.121:0/3613797381 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa304009370 con 0x7fa3080c30e0 2023-11-15T03:04:20.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa30e477700 1 --2- 172.21.15.121:0/3613797381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3080c30e0 0x7fa3081683c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7fa304009130 tx=0x7fa304017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:20.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa2fe7fc700 1 -- 172.21.15.121:0/3613797381 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa304000bc0 con 0x7fa3080c30e0 2023-11-15T03:04:20.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa2fe7fc700 1 -- 172.21.15.121:0/3613797381 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30400b850 con 0x7fa3080c30e0 2023-11-15T03:04:20.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa2fe7fc700 1 -- 172.21.15.121:0/3613797381 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa304000d20 con 0x7fa3080c30e0 2023-11-15T03:04:20.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa30f479700 1 -- 172.21.15.121:0/3613797381 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3080c2a40 con 0x7fa3080c30e0 2023-11-15T03:04:20.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.092+0000 7fa30f479700 1 -- 172.21.15.121:0/3613797381 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3080bf930 con 0x7fa3080c30e0 2023-11-15T03:04:20.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.096+0000 7fa30f479700 1 -- 172.21.15.121:0/3613797381 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa30806bcf0 con 0x7fa3080c30e0 2023-11-15T03:04:20.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.096+0000 7fa2fe7fc700 1 -- 172.21.15.121:0/3613797381 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa30401cb10 con 0x7fa3080c30e0 2023-11-15T03:04:20.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.100+0000 7fa2fe7fc700 1 --2- 172.21.15.121:0/3613797381 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa2ec05a510 0x7fa2ec05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:20.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.100+0000 7fa2fe7fc700 1 -- 172.21.15.121:0/3613797381 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6696+0+0 (secure 0 0 0) 0x7fa304076d50 con 0x7fa3080c30e0 2023-11-15T03:04:20.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.100+0000 7fa30dc76700 1 --2- 172.21.15.121:0/3613797381 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa2ec05a510 0x7fa2ec05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:20.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.100+0000 7fa30dc76700 1 --2- 172.21.15.121:0/3613797381 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa2ec05a510 0x7fa2ec05c9d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fa3080c6530 tx=0x7fa2f8007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:20.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.104+0000 7fa2fe7fc700 1 -- 172.21.15.121:0/3613797381 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa30404a2b0 con 0x7fa3080c30e0 2023-11-15T03:04:20.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.308+0000 7fa30f479700 1 -- 172.21.15.121:0/3613797381 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3080aa540 con 0x7fa3080c30e0 2023-11-15T03:04:20.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.308+0000 7fa2fe7fc700 1 -- 172.21.15.121:0/3613797381 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4495 (secure 0 0 0) 0x7fa30402a020 con 0x7fa3080c30e0 2023-11-15T03:04:20.315 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:04:20.316 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":110,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7836,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3956345798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3956345798},{"type":"v1","addr":"172.21.15.121:6836","nonce":3956345798}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:04:15.085570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:04:20.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.312+0000 7fa30f479700 1 -- 172.21.15.121:0/3613797381 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa2ec05a510 msgr2=0x7fa2ec05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:20.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.312+0000 7fa30f479700 1 --2- 172.21.15.121:0/3613797381 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa2ec05a510 0x7fa2ec05c9d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fa3080c6530 tx=0x7fa2f8007d40).stop 2023-11-15T03:04:20.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.312+0000 7fa30f479700 1 -- 172.21.15.121:0/3613797381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3080c30e0 msgr2=0x7fa3081683c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:20.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.312+0000 7fa30f479700 1 --2- 172.21.15.121:0/3613797381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3080c30e0 0x7fa3081683c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7fa304009130 tx=0x7fa304017000).stop 2023-11-15T03:04:20.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.312+0000 7fa30f479700 1 -- 172.21.15.121:0/3613797381 shutdown_connections 2023-11-15T03:04:20.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.312+0000 7fa30f479700 1 --2- 172.21.15.121:0/3613797381 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa2ec05a510 0x7fa2ec05c9d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:20.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.312+0000 7fa30f479700 1 --2- 172.21.15.121:0/3613797381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa308152140 0x7fa30814c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:20.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.312+0000 7fa30f479700 1 --2- 172.21.15.121:0/3613797381 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa308151870 0x7fa3080a8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:20.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.312+0000 7fa30f479700 1 --2- 172.21.15.121:0/3613797381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3080c30e0 0x7fa3081683c0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:20.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.312+0000 7fa30f479700 1 -- 172.21.15.121:0/3613797381 >> 172.21.15.121:0/3613797381 conn(0x7fa308062190 msgr2=0x7fa30814e110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:20.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.312+0000 7fa30f479700 1 -- 172.21.15.121:0/3613797381 shutdown_connections 2023-11-15T03:04:20.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.312+0000 7fa30f479700 1 -- 172.21.15.121:0/3613797381 wait complete. 2023-11-15T03:04:20.327 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 110 2023-11-15T03:04:20.339 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:04:20.340 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:04:20.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.492+0000 7f91432db700 1 Processor -- start 2023-11-15T03:04:20.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.492+0000 7f91432db700 1 -- start start 2023-11-15T03:04:20.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.496+0000 7f91432db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f913c1562f0 0x7f913c155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:20.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.496+0000 7f91432db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f913c0a57c0 0x7f913c0a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:20.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.496+0000 7f91432db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913c0c4c30 con 0x7f913c154ca0 2023-11-15T03:04:20.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.496+0000 7f91432db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f913c0a8280 con 0x7f913c1562f0 2023-11-15T03:04:20.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.496+0000 7f91432db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0a6d40 con 0x7f913c0a57c0 2023-11-15T03:04:20.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.496+0000 7f9141077700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f913c154ca0 0x7f913c0a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44194/0 (socket says 172.21.15.121:44194) 2023-11-15T03:04:20.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.496+0000 7f9141077700 1 -- 172.21.15.121:0/292040573 learned_addr learned my addr 172.21.15.121:0/292040573 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:20.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.496+0000 7f9140876700 1 --2- 172.21.15.121:0/292040573 >> v2:172.21.15.190:3300/0 conn(0x7f913c1562f0 0x7f913c155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:20.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.496+0000 7f9141878700 1 --2- 172.21.15.121:0/292040573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f913c0a57c0 0x7f913c0a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:20.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.496+0000 7f9133fff700 1 -- 172.21.15.121:0/292040573 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3501735186 0 0) 0x7f913c0c4c30 con 0x7f913c154ca0 2023-11-15T03:04:20.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.496+0000 7f9133fff700 1 -- 172.21.15.121:0/292040573 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91280033f0 con 0x7f913c154ca0 2023-11-15T03:04:20.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.500+0000 7f9141878700 1 -- 172.21.15.121:0/292040573 >> v2:172.21.15.190:3300/0 conn(0x7f913c1562f0 msgr2=0x7f913c155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:20.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.500+0000 7f9141878700 1 --2- 172.21.15.121:0/292040573 >> v2:172.21.15.190:3300/0 conn(0x7f913c1562f0 0x7f913c155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:20.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.500+0000 7f9141878700 1 -- 172.21.15.121:0/292040573 >> v1:172.21.15.121:6789/0 conn(0x7f913c154ca0 legacy=0x7f913c0a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:20.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.500+0000 7f9141878700 1 -- 172.21.15.121:0/292040573 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c0a6140 con 0x7f913c0a57c0 2023-11-15T03:04:20.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.500+0000 7f9141878700 1 --2- 172.21.15.121:0/292040573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f913c0a57c0 0x7f913c0a5c50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f9134009e60 tx=0x7f9134009780).ready entity=mon.2 client_cookie=cc778f4bc8e6708b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:20.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.500+0000 7f9133fff700 1 -- 172.21.15.121:0/292040573 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913400e070 con 0x7f913c0a57c0 2023-11-15T03:04:20.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.500+0000 7f9133fff700 1 -- 172.21.15.121:0/292040573 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913400bab0 con 0x7f913c0a57c0 2023-11-15T03:04:20.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.500+0000 7f9133fff700 1 -- 172.21.15.121:0/292040573 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913400be30 con 0x7f913c0a57c0 2023-11-15T03:04:20.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.500+0000 7f91432db700 1 -- 172.21.15.121:0/292040573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f913c0a57c0 msgr2=0x7f913c0a5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:20.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.500+0000 7f91432db700 1 --2- 172.21.15.121:0/292040573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f913c0a57c0 0x7f913c0a5c50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f9134009e60 tx=0x7f9134009780).stop 2023-11-15T03:04:20.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.500+0000 7f91432db700 1 -- 172.21.15.121:0/292040573 shutdown_connections 2023-11-15T03:04:20.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.500+0000 7f91432db700 1 --2- 172.21.15.121:0/292040573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f913c0a57c0 0x7f913c0a5c50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:20.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.500+0000 7f91432db700 1 --2- 172.21.15.121:0/292040573 >> v2:172.21.15.190:3300/0 conn(0x7f913c1562f0 0x7f913c155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:20.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.500+0000 7f91432db700 1 -- 172.21.15.121:0/292040573 >> 172.21.15.121:0/292040573 conn(0x7f913c062190 msgr2=0x7f913c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:20.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.500+0000 7f91432db700 1 -- 172.21.15.121:0/292040573 shutdown_connections 2023-11-15T03:04:20.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.500+0000 7f91432db700 1 -- 172.21.15.121:0/292040573 wait complete. 2023-11-15T03:04:20.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.504+0000 7f91432db700 1 Processor -- start 2023-11-15T03:04:20.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.504+0000 7f91432db700 1 -- start start 2023-11-15T03:04:20.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.504+0000 7f91432db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f913c0a57c0 0x7f913c13bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:20.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.504+0000 7f91432db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f913c154ca0 0x7f913c0bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:20.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.504+0000 7f91432db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f913c1562f0 0x7f913c0c0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:20.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.504+0000 7f91432db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0c0570 con 0x7f913c154ca0 2023-11-15T03:04:20.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.504+0000 7f91432db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c13c9e0 con 0x7f913c0a57c0 2023-11-15T03:04:20.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.504+0000 7f91432db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913c13ccc0 con 0x7f913c1562f0 2023-11-15T03:04:20.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.504+0000 7f9140876700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f913c154ca0 0x7f913c0bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:20.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.504+0000 7f9140876700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f913c154ca0 0x7f913c0bfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56416/0 (socket says 172.21.15.121:56416) 2023-11-15T03:04:20.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.504+0000 7f9140876700 1 -- 172.21.15.121:0/3221401140 learned_addr learned my addr 172.21.15.121:0/3221401140 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:20.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.504+0000 7f9141077700 1 --2- 172.21.15.121:0/3221401140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f913c0a57c0 0x7f913c13bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:20.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.504+0000 7f9141878700 1 --2- 172.21.15.121:0/3221401140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f913c1562f0 0x7f913c0c0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:20.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.504+0000 7f9140876700 1 -- 172.21.15.121:0/3221401140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f913c1562f0 msgr2=0x7f913c0c0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:20.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.504+0000 7f9140876700 1 --2- 172.21.15.121:0/3221401140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f913c1562f0 0x7f913c0c0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:20.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.504+0000 7f9140876700 1 -- 172.21.15.121:0/3221401140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f913c0a57c0 msgr2=0x7f913c13bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:20.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.504+0000 7f9140876700 1 --2- 172.21.15.121:0/3221401140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f913c0a57c0 0x7f913c13bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:20.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.504+0000 7f9140876700 1 -- 172.21.15.121:0/3221401140 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9134009a20 con 0x7f913c154ca0 2023-11-15T03:04:20.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.508+0000 7f9140876700 1 --2- 172.21.15.121:0/3221401140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f913c154ca0 0x7f913c0bfb90 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7f912c00e900 tx=0x7f912c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:20.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.508+0000 7f91317fa700 1 -- 172.21.15.121:0/3221401140 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c00c640 con 0x7f913c154ca0 2023-11-15T03:04:20.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.508+0000 7f91317fa700 1 -- 172.21.15.121:0/3221401140 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c00c7a0 con 0x7f913c154ca0 2023-11-15T03:04:20.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.508+0000 7f91432db700 1 -- 172.21.15.121:0/3221401140 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913c153de0 con 0x7f913c154ca0 2023-11-15T03:04:20.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.508+0000 7f91317fa700 1 -- 172.21.15.121:0/3221401140 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c00cb20 con 0x7f913c154ca0 2023-11-15T03:04:20.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.508+0000 7f91432db700 1 -- 172.21.15.121:0/3221401140 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913c1543a0 con 0x7f913c154ca0 2023-11-15T03:04:20.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.508+0000 7f91432db700 1 -- 172.21.15.121:0/3221401140 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913c06c150 con 0x7f913c154ca0 2023-11-15T03:04:20.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.508+0000 7f91317fa700 1 -- 172.21.15.121:0/3221401140 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f912c01b400 con 0x7f913c154ca0 2023-11-15T03:04:20.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.512+0000 7f91317fa700 1 --2- 172.21.15.121:0/3221401140 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f912005a510 0x7f912005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:20.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.512+0000 7f91317fa700 1 -- 172.21.15.121:0/3221401140 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6696+0+0 (secure 0 0 0) 0x7f912c074ed0 con 0x7f913c154ca0 2023-11-15T03:04:20.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.512+0000 7f9141077700 1 --2- 172.21.15.121:0/3221401140 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f912005a510 0x7f912005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:20.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.512+0000 7f9141077700 1 --2- 172.21.15.121:0/3221401140 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f912005a510 0x7f912005c9d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f9138005750 tx=0x7f9138009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:20.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.520+0000 7f91317fa700 1 -- 172.21.15.121:0/3221401140 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f912c048400 con 0x7f913c154ca0 2023-11-15T03:04:20.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.720+0000 7f91432db700 1 -- 172.21.15.121:0/3221401140 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f913c0c7550 con 0x7f913c154ca0 2023-11-15T03:04:20.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.720+0000 7f91317fa700 1 -- 172.21.15.121:0/3221401140 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4495 (secure 0 0 0) 0x7f912c04ba20 con 0x7f913c154ca0 2023-11-15T03:04:20.729 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:04:20.729 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":110,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7836,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3956345798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3956345798},{"type":"v1","addr":"172.21.15.121:6836","nonce":3956345798}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:04:15.085570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:04:20.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.728+0000 7f91432db700 1 -- 172.21.15.121:0/3221401140 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f912005a510 msgr2=0x7f912005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:20.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.728+0000 7f91432db700 1 --2- 172.21.15.121:0/3221401140 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f912005a510 0x7f912005c9d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f9138005750 tx=0x7f9138009040).stop 2023-11-15T03:04:20.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.728+0000 7f91432db700 1 -- 172.21.15.121:0/3221401140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f913c154ca0 msgr2=0x7f913c0bfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:20.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.728+0000 7f91432db700 1 --2- 172.21.15.121:0/3221401140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f913c154ca0 0x7f913c0bfb90 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7f912c00e900 tx=0x7f912c009060).stop 2023-11-15T03:04:20.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.728+0000 7f91432db700 1 -- 172.21.15.121:0/3221401140 shutdown_connections 2023-11-15T03:04:20.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.728+0000 7f91432db700 1 --2- 172.21.15.121:0/3221401140 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f912005a510 0x7f912005c9d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:20.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.728+0000 7f91432db700 1 --2- 172.21.15.121:0/3221401140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f913c1562f0 0x7f913c0c0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:20.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.728+0000 7f91432db700 1 --2- 172.21.15.121:0/3221401140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f913c154ca0 0x7f913c0bfb90 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:20.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.728+0000 7f91432db700 1 --2- 172.21.15.121:0/3221401140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f913c0a57c0 0x7f913c13bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:20.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.728+0000 7f91432db700 1 -- 172.21.15.121:0/3221401140 >> 172.21.15.121:0/3221401140 conn(0x7f913c062190 msgr2=0x7f913c0a8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:20.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.728+0000 7f91432db700 1 -- 172.21.15.121:0/3221401140 shutdown_connections 2023-11-15T03:04:20.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.728+0000 7f91432db700 1 -- 172.21.15.121:0/3221401140 wait complete. 2023-11-15T03:04:20.739 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 110 2023-11-15T03:04:20.755 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 200.00000000' led to DAMAGED state 2023-11-15T03:04:20.755 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 200.00000001' 2023-11-15T03:04:20.756 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:04:20.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:20.757 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:04:20.780 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:04:20.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:04:20.781 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:04:20.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.984+0000 7fd534945700 1 Processor -- start 2023-11-15T03:04:20.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.984+0000 7fd534945700 1 -- start start 2023-11-15T03:04:20.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.984+0000 7fd534945700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd5300a79d0 0x7fd5300643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:20.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.984+0000 7fd534945700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd530152040 con 0x7fd5300a7db0 2023-11-15T03:04:20.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.984+0000 7fd534945700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5300a8d80 con 0x7fd5300c3bb0 2023-11-15T03:04:20.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.984+0000 7fd534945700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd530064e60 con 0x7fd5300a79d0 2023-11-15T03:04:20.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.984+0000 7fd52e59c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fd5300a7db0 0x7fd5300c34a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44204/0 (socket says 172.21.15.121:44204) 2023-11-15T03:04:20.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.984+0000 7fd52e59c700 1 -- 172.21.15.121:0/3408893494 learned_addr learned my addr 172.21.15.121:0/3408893494 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:20.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.984+0000 7fd52ed9d700 1 --2- 172.21.15.121:0/3408893494 >> v2:172.21.15.190:3300/0 conn(0x7fd5300a79d0 0x7fd5300643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:20.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.988+0000 7fd52dd9b700 1 -- 172.21.15.121:0/3408893494 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1504753383 0 0) 0x7fd5300a8d80 con 0x7fd5300c3bb0 2023-11-15T03:04:20.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.988+0000 7fd52dd9b700 1 -- 172.21.15.121:0/3408893494 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5180033f0 con 0x7fd5300c3bb0 2023-11-15T03:04:20.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.988+0000 7fd52dd9b700 1 -- 172.21.15.121:0/3408893494 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274903941 0 0) 0x7fd524001040 con 0x7fd5300a7db0 2023-11-15T03:04:20.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.988+0000 7fd52dd9b700 1 -- 172.21.15.121:0/3408893494 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5300a8d80 con 0x7fd5300a7db0 2023-11-15T03:04:20.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.988+0000 7fd52ed9d700 1 -- 172.21.15.121:0/3408893494 >> v1:172.21.15.190:6789/0 conn(0x7fd5300c3bb0 legacy=0x7fd5300a9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:20.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.988+0000 7fd52ed9d700 1 -- 172.21.15.121:0/3408893494 >> v1:172.21.15.121:6789/0 conn(0x7fd5300a7db0 legacy=0x7fd5300c34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:20.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.988+0000 7fd52dd9b700 1 -- 172.21.15.121:0/3408893494 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1711794975 0 0) 0x7fd5300a8d80 con 0x7fd5300a7db0 2023-11-15T03:04:20.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.988+0000 7fd52ed9d700 1 -- 172.21.15.121:0/3408893494 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5300a8770 con 0x7fd5300a79d0 2023-11-15T03:04:20.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.988+0000 7fd52ed9d700 1 --2- 172.21.15.121:0/3408893494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5300a79d0 0x7fd5300643a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7fd520009790 tx=0x7fd520009060).ready entity=mon.1 client_cookie=28bbce24e0c732c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:20.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.988+0000 7fd52dd9b700 1 -- 172.21.15.121:0/3408893494 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd524001530 con 0x7fd5300a79d0 2023-11-15T03:04:20.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.988+0000 7fd52dd9b700 1 -- 172.21.15.121:0/3408893494 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd520011400 con 0x7fd5300a79d0 2023-11-15T03:04:20.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.988+0000 7fd52dd9b700 1 -- 172.21.15.121:0/3408893494 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd520011780 con 0x7fd5300a79d0 2023-11-15T03:04:20.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.988+0000 7fd534945700 1 -- 172.21.15.121:0/3408893494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5300a79d0 msgr2=0x7fd5300643a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:20.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.988+0000 7fd534945700 1 --2- 172.21.15.121:0/3408893494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5300a79d0 0x7fd5300643a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7fd520009790 tx=0x7fd520009060).stop 2023-11-15T03:04:21.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.988+0000 7fd534945700 1 -- 172.21.15.121:0/3408893494 shutdown_connections 2023-11-15T03:04:21.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.988+0000 7fd534945700 1 --2- 172.21.15.121:0/3408893494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5300a79d0 0x7fd5300643a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:21.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.988+0000 7fd534945700 1 -- 172.21.15.121:0/3408893494 >> 172.21.15.121:0/3408893494 conn(0x7fd5300684f0 msgr2=0x7fd5300c8760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:21.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.988+0000 7fd534945700 1 -- 172.21.15.121:0/3408893494 shutdown_connections 2023-11-15T03:04:21.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.988+0000 7fd534945700 1 -- 172.21.15.121:0/3408893494 wait complete. 2023-11-15T03:04:21.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.992+0000 7fd534945700 1 Processor -- start 2023-11-15T03:04:21.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.992+0000 7fd534945700 1 -- start start 2023-11-15T03:04:21.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd534945700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5300a79d0 0x7fd5300ac600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:21.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd534945700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5300a7db0 0x7fd5300ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:21.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd534945700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5300c3bb0 0x7fd530150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:21.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd534945700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300acaf0 con 0x7fd5300c3bb0 2023-11-15T03:04:21.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd534945700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300ad5c0 con 0x7fd5300a79d0 2023-11-15T03:04:21.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd534945700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300ae410 con 0x7fd5300a7db0 2023-11-15T03:04:21.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd52f59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5300c3bb0 0x7fd530150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:21.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd52f59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5300c3bb0 0x7fd530150090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56424/0 (socket says 172.21.15.121:56424) 2023-11-15T03:04:21.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd52f59e700 1 -- 172.21.15.121:0/3282091039 learned_addr learned my addr 172.21.15.121:0/3282091039 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:21.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd52ed9d700 1 --2- 172.21.15.121:0/3282091039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5300a79d0 0x7fd5300ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:21.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd52e59c700 1 --2- 172.21.15.121:0/3282091039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5300a7db0 0x7fd5300ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:21.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd52f59e700 1 -- 172.21.15.121:0/3282091039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5300a7db0 msgr2=0x7fd5300ad0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:21.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd52f59e700 1 --2- 172.21.15.121:0/3282091039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5300a7db0 0x7fd5300ad0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:21.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd52f59e700 1 -- 172.21.15.121:0/3282091039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5300a79d0 msgr2=0x7fd5300ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:21.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd52f59e700 1 --2- 172.21.15.121:0/3282091039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5300a79d0 0x7fd5300ac600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:21.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd52f59e700 1 -- 172.21.15.121:0/3282091039 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5200092a0 con 0x7fd5300c3bb0 2023-11-15T03:04:21.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd52f59e700 1 --2- 172.21.15.121:0/3282091039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5300c3bb0 0x7fd530150090 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7fd52800c7f0 tx=0x7fd52800cbb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:21.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd516ffd700 1 -- 172.21.15.121:0/3282091039 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528002020 con 0x7fd5300c3bb0 2023-11-15T03:04:21.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd516ffd700 1 -- 172.21.15.121:0/3282091039 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd528002180 con 0x7fd5300c3bb0 2023-11-15T03:04:21.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd516ffd700 1 -- 172.21.15.121:0/3282091039 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528001590 con 0x7fd5300c3bb0 2023-11-15T03:04:21.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd534945700 1 -- 172.21.15.121:0/3282091039 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5300ad800 con 0x7fd5300c3bb0 2023-11-15T03:04:21.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:20.996+0000 7fd534945700 1 -- 172.21.15.121:0/3282091039 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5300addc0 con 0x7fd5300c3bb0 2023-11-15T03:04:21.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:21.000+0000 7fd534945700 1 -- 172.21.15.121:0/3282091039 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd53006bcf0 con 0x7fd5300c3bb0 2023-11-15T03:04:21.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:21.000+0000 7fd516ffd700 1 -- 172.21.15.121:0/3282091039 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd528001980 con 0x7fd5300c3bb0 2023-11-15T03:04:21.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:21.000+0000 7fd516ffd700 1 --2- 172.21.15.121:0/3282091039 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd51c05a4e0 0x7fd51c05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:21.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:21.000+0000 7fd516ffd700 1 -- 172.21.15.121:0/3282091039 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6696+0+0 (secure 0 0 0) 0x7fd5280740b0 con 0x7fd5300c3bb0 2023-11-15T03:04:21.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:21.000+0000 7fd52ed9d700 1 --2- 172.21.15.121:0/3282091039 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd51c05a4e0 0x7fd51c05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:21.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:21.004+0000 7fd52ed9d700 1 --2- 172.21.15.121:0/3282091039 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd51c05a4e0 0x7fd51c05c9a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7fd520009760 tx=0x7fd520016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:21.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:21.008+0000 7fd516ffd700 1 -- 172.21.15.121:0/3282091039 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd528047650 con 0x7fd5300c3bb0 2023-11-15T03:04:21.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:21.208+0000 7fd534945700 1 -- 172.21.15.121:0/3282091039 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd5300a84b0 con 0x7fd5300c3bb0 2023-11-15T03:04:22.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.056+0000 7fd516ffd700 1 -- 172.21.15.121:0/3282091039 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v111) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd528001cb0 con 0x7fd5300c3bb0 2023-11-15T03:04:22.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.060+0000 7fd534945700 1 -- 172.21.15.121:0/3282091039 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd51c05a4e0 msgr2=0x7fd51c05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:22.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.060+0000 7fd534945700 1 --2- 172.21.15.121:0/3282091039 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd51c05a4e0 0x7fd51c05c9a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7fd520009760 tx=0x7fd520016000).stop 2023-11-15T03:04:22.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.060+0000 7fd534945700 1 -- 172.21.15.121:0/3282091039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5300c3bb0 msgr2=0x7fd530150090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:22.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.060+0000 7fd534945700 1 --2- 172.21.15.121:0/3282091039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5300c3bb0 0x7fd530150090 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7fd52800c7f0 tx=0x7fd52800cbb0).stop 2023-11-15T03:04:22.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.060+0000 7fd534945700 1 -- 172.21.15.121:0/3282091039 shutdown_connections 2023-11-15T03:04:22.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.060+0000 7fd534945700 1 --2- 172.21.15.121:0/3282091039 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd51c05a4e0 0x7fd51c05c9a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:22.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.060+0000 7fd534945700 1 --2- 172.21.15.121:0/3282091039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5300c3bb0 0x7fd530150090 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:22.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.060+0000 7fd534945700 1 --2- 172.21.15.121:0/3282091039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5300a7db0 0x7fd5300ad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:22.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.060+0000 7fd534945700 1 --2- 172.21.15.121:0/3282091039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5300a79d0 0x7fd5300ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:22.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.060+0000 7fd534945700 1 -- 172.21.15.121:0/3282091039 >> 172.21.15.121:0/3282091039 conn(0x7fd5300684f0 msgr2=0x7fd5300a9870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:22.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.060+0000 7fd534945700 1 -- 172.21.15.121:0/3282091039 shutdown_connections 2023-11-15T03:04:22.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.060+0000 7fd534945700 1 -- 172.21.15.121:0/3282091039 wait complete. 2023-11-15T03:04:22.070 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:04:22.089 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:04:22.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.252+0000 7fe03af46700 1 Processor -- start 2023-11-15T03:04:22.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.252+0000 7fe03af46700 1 -- start start 2023-11-15T03:04:22.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.256+0000 7fe03af46700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe034157e30 0x7fe034158210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:22.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.256+0000 7fe03af46700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe034062190 0x7fe0340625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:22.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.256+0000 7fe03af46700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0340aaf80 con 0x7fe0340660e0 2023-11-15T03:04:22.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.256+0000 7fe03af46700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe034062ae0 con 0x7fe034157e30 2023-11-15T03:04:22.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.256+0000 7fe03af46700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0340aa970 con 0x7fe034062190 2023-11-15T03:04:22.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.256+0000 7fe038ce2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe034157e30 0x7fe034158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:22.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.256+0000 7fe0394e3700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe0340660e0 0x7fe034066550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:58714/0 (socket says 172.21.15.121:58714) 2023-11-15T03:04:22.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.256+0000 7fe0394e3700 1 -- 172.21.15.121:0/2630807111 learned_addr learned my addr 172.21.15.121:0/2630807111 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:22.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.256+0000 7fe033fff700 1 --2- 172.21.15.121:0/2630807111 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe034062190 0x7fe0340625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:22.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.256+0000 7fe0337fe700 1 -- 172.21.15.121:0/2630807111 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078551858 0 0) 0x7fe0340aaf80 con 0x7fe0340660e0 2023-11-15T03:04:22.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.256+0000 7fe0337fe700 1 -- 172.21.15.121:0/2630807111 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0240033f0 con 0x7fe0340660e0 2023-11-15T03:04:22.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.256+0000 7fe033fff700 1 -- 172.21.15.121:0/2630807111 >> v2:172.21.15.190:3300/0 conn(0x7fe034157e30 msgr2=0x7fe034158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:22.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.256+0000 7fe033fff700 1 --2- 172.21.15.121:0/2630807111 >> v2:172.21.15.190:3300/0 conn(0x7fe034157e30 0x7fe034158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:22.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.256+0000 7fe033fff700 1 -- 172.21.15.121:0/2630807111 >> v1:172.21.15.190:6789/0 conn(0x7fe0340660e0 legacy=0x7fe034066550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:22.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.256+0000 7fe033fff700 1 -- 172.21.15.121:0/2630807111 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe034066e10 con 0x7fe034062190 2023-11-15T03:04:22.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.256+0000 7fe033fff700 1 --2- 172.21.15.121:0/2630807111 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe034062190 0x7fe0340625f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fe020009760 tx=0x7fe020009060).ready entity=mon.2 client_cookie=2127961fd74428e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:22.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.256+0000 7fe0337fe700 1 -- 172.21.15.121:0/2630807111 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02000d4a0 con 0x7fe034062190 2023-11-15T03:04:22.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.256+0000 7fe0337fe700 1 -- 172.21.15.121:0/2630807111 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02000d600 con 0x7fe034062190 2023-11-15T03:04:22.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.256+0000 7fe0337fe700 1 -- 172.21.15.121:0/2630807111 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02000d980 con 0x7fe034062190 2023-11-15T03:04:22.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.260+0000 7fe03af46700 1 -- 172.21.15.121:0/2630807111 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe034062190 msgr2=0x7fe0340625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:22.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.260+0000 7fe03af46700 1 --2- 172.21.15.121:0/2630807111 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe034062190 0x7fe0340625f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fe020009760 tx=0x7fe020009060).stop 2023-11-15T03:04:22.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.260+0000 7fe03af46700 1 -- 172.21.15.121:0/2630807111 shutdown_connections 2023-11-15T03:04:22.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.260+0000 7fe03af46700 1 --2- 172.21.15.121:0/2630807111 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe034062190 0x7fe0340625f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:22.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.260+0000 7fe03af46700 1 --2- 172.21.15.121:0/2630807111 >> v2:172.21.15.190:3300/0 conn(0x7fe034157e30 0x7fe034158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:22.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.260+0000 7fe03af46700 1 -- 172.21.15.121:0/2630807111 >> 172.21.15.121:0/2630807111 conn(0x7fe034067a60 msgr2=0x7fe034067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:22.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.260+0000 7fe03af46700 1 -- 172.21.15.121:0/2630807111 shutdown_connections 2023-11-15T03:04:22.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.260+0000 7fe03af46700 1 -- 172.21.15.121:0/2630807111 wait complete. 2023-11-15T03:04:22.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.264+0000 7fe03af46700 1 Processor -- start 2023-11-15T03:04:22.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.264+0000 7fe03af46700 1 -- start start 2023-11-15T03:04:22.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.264+0000 7fe03af46700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe034062190 0x7fe034153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:22.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.264+0000 7fe03af46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0340660e0 0x7fe034153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:22.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.264+0000 7fe03af46700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe034157e30 0x7fe034168940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:22.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.264+0000 7fe03af46700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe034150eb0 con 0x7fe0340660e0 2023-11-15T03:04:22.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.264+0000 7fe03af46700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe034154310 con 0x7fe034062190 2023-11-15T03:04:22.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.264+0000 7fe03af46700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0341545f0 con 0x7fe034157e30 2023-11-15T03:04:22.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.264+0000 7fe033fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0340660e0 0x7fe034153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:22.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.264+0000 7fe033fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0340660e0 0x7fe034153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56426/0 (socket says 172.21.15.121:56426) 2023-11-15T03:04:22.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.264+0000 7fe033fff700 1 -- 172.21.15.121:0/3163938302 learned_addr learned my addr 172.21.15.121:0/3163938302 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:22.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.264+0000 7fe038ce2700 1 --2- 172.21.15.121:0/3163938302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe034062190 0x7fe034153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:22.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.264+0000 7fe0394e3700 1 --2- 172.21.15.121:0/3163938302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe034157e30 0x7fe034168940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:22.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.268+0000 7fe033fff700 1 -- 172.21.15.121:0/3163938302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe034157e30 msgr2=0x7fe034168940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:22.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.268+0000 7fe033fff700 1 --2- 172.21.15.121:0/3163938302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe034157e30 0x7fe034168940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:22.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.268+0000 7fe033fff700 1 -- 172.21.15.121:0/3163938302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe034062190 msgr2=0x7fe034153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:22.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.268+0000 7fe033fff700 1 --2- 172.21.15.121:0/3163938302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe034062190 0x7fe034153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:22.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.268+0000 7fe033fff700 1 -- 172.21.15.121:0/3163938302 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0200092a0 con 0x7fe0340660e0 2023-11-15T03:04:22.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.268+0000 7fe0394e3700 1 --2- 172.21.15.121:0/3163938302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe034157e30 0x7fe034168940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:22.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.268+0000 7fe033fff700 1 --2- 172.21.15.121:0/3163938302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0340660e0 0x7fe034153a80 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7fe020009d50 tx=0x7fe020016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:22.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.268+0000 7fe030ff9700 1 -- 172.21.15.121:0/3163938302 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe020000c90 con 0x7fe0340660e0 2023-11-15T03:04:22.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.268+0000 7fe030ff9700 1 -- 172.21.15.121:0/3163938302 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02000d770 con 0x7fe0340660e0 2023-11-15T03:04:22.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.268+0000 7fe03af46700 1 -- 172.21.15.121:0/3163938302 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0341548d0 con 0x7fe0340660e0 2023-11-15T03:04:22.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.268+0000 7fe030ff9700 1 -- 172.21.15.121:0/3163938302 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02001d430 con 0x7fe0340660e0 2023-11-15T03:04:22.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.268+0000 7fe03af46700 1 -- 172.21.15.121:0/3163938302 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03414ea50 con 0x7fe0340660e0 2023-11-15T03:04:22.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.268+0000 7fe03af46700 1 -- 172.21.15.121:0/3163938302 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe034151020 con 0x7fe0340660e0 2023-11-15T03:04:22.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.272+0000 7fe030ff9700 1 -- 172.21.15.121:0/3163938302 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe02001d860 con 0x7fe0340660e0 2023-11-15T03:04:22.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.272+0000 7fe030ff9700 1 --2- 172.21.15.121:0/3163938302 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe01805a510 0x7fe01805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:22.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.272+0000 7fe030ff9700 1 -- 172.21.15.121:0/3163938302 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6696+0+0 (secure 0 0 0) 0x7fe020076330 con 0x7fe0340660e0 2023-11-15T03:04:22.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.272+0000 7fe038ce2700 1 --2- 172.21.15.121:0/3163938302 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe01805a510 0x7fe01805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:22.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.272+0000 7fe038ce2700 1 --2- 172.21.15.121:0/3163938302 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe01805a510 0x7fe01805c9d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fe028009c30 tx=0x7fe028009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:22.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.276+0000 7fe030ff9700 1 -- 172.21.15.121:0/3163938302 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe0200498d0 con 0x7fe0340660e0 2023-11-15T03:04:22.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:22.500+0000 7fe03af46700 1 -- 172.21.15.121:0/3163938302 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fe03415b990 con 0x7fe0340660e0 2023-11-15T03:04:23.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.068+0000 7fe030ff9700 1 -- 172.21.15.121:0/3163938302 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v112) v1 ==== 101+0+0 (secure 0 0 0) 0x7fe02004cef0 con 0x7fe0340660e0 2023-11-15T03:04:23.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.076+0000 7fe03af46700 1 -- 172.21.15.121:0/3163938302 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe01805a510 msgr2=0x7fe01805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.076+0000 7fe03af46700 1 --2- 172.21.15.121:0/3163938302 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe01805a510 0x7fe01805c9d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fe028009c30 tx=0x7fe028009060).stop 2023-11-15T03:04:23.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.076+0000 7fe03af46700 1 -- 172.21.15.121:0/3163938302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0340660e0 msgr2=0x7fe034153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.076+0000 7fe03af46700 1 --2- 172.21.15.121:0/3163938302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0340660e0 0x7fe034153a80 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7fe020009d50 tx=0x7fe020016000).stop 2023-11-15T03:04:23.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.076+0000 7fe03af46700 1 -- 172.21.15.121:0/3163938302 shutdown_connections 2023-11-15T03:04:23.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.076+0000 7fe03af46700 1 --2- 172.21.15.121:0/3163938302 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe01805a510 0x7fe01805c9d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.076+0000 7fe03af46700 1 --2- 172.21.15.121:0/3163938302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe034157e30 0x7fe034168940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.076+0000 7fe03af46700 1 --2- 172.21.15.121:0/3163938302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0340660e0 0x7fe034153a80 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.076+0000 7fe03af46700 1 --2- 172.21.15.121:0/3163938302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe034062190 0x7fe034153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.076+0000 7fe03af46700 1 -- 172.21.15.121:0/3163938302 >> 172.21.15.121:0/3163938302 conn(0x7fe034067a60 msgr2=0x7fe0340c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:23.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.076+0000 7fe03af46700 1 -- 172.21.15.121:0/3163938302 shutdown_connections 2023-11-15T03:04:23.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.076+0000 7fe03af46700 1 -- 172.21.15.121:0/3163938302 wait complete. 2023-11-15T03:04:23.087 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:04:23.104 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:04:23.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f9279b3ad00 1 Processor -- start 2023-11-15T03:04:23.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f9279b3ad00 1 -- start start 2023-11-15T03:04:23.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f9279b3ad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598beced0d0 0x5598bec4d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f9279b3ad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598bec4dab0 0x5598bec611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f9279b3ad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598bec61690 0x5598bec61b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f9279b3ad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598bebc8cc0 con 0x5598bec4dab0 2023-11-15T03:04:23.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f9279b3ad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598beb93220 con 0x5598beced0d0 2023-11-15T03:04:23.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f9279b3ad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598bec6a6a0 con 0x5598bec61690 2023-11-15T03:04:23.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f926d844700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598bec4dab0 0x5598bec611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f926d844700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598bec4dab0 0x5598bec611a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56446/0 (socket says 172.21.15.121:56446) 2023-11-15T03:04:23.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f926d844700 1 -- 172.21.15.121:0/741797849 learned_addr learned my addr 172.21.15.121:0/741797849 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:23.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f926e045700 1 --2- 172.21.15.121:0/741797849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598beced0d0 0x5598bec4d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f926e846700 1 --2- 172.21.15.121:0/741797849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598bec61690 0x5598bec61b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f926d844700 1 -- 172.21.15.121:0/741797849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598bec61690 msgr2=0x5598bec61b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:23.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f926d844700 1 --2- 172.21.15.121:0/741797849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598bec61690 0x5598bec61b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f926d844700 1 -- 172.21.15.121:0/741797849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598beced0d0 msgr2=0x5598bec4d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:23.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f926d844700 1 --2- 172.21.15.121:0/741797849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598beced0d0 0x5598bec4d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f926d844700 1 -- 172.21.15.121:0/741797849 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598beb8ba70 con 0x5598bec4dab0 2023-11-15T03:04:23.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f926e846700 1 --2- 172.21.15.121:0/741797849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598bec61690 0x5598bec61b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:04:23.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f926d844700 1 --2- 172.21.15.121:0/741797849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598bec4dab0 0x5598bec611a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7f925800a3b0 tx=0x7f9264016000).ready entity=mon.0 client_cookie=ca83b3aa6f058f01 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:23.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f926c842700 1 -- 172.21.15.121:0/741797849 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926400f040 con 0x5598bec4dab0 2023-11-15T03:04:23.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f926c842700 1 -- 172.21.15.121:0/741797849 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9264008490 con 0x5598bec4dab0 2023-11-15T03:04:23.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.156+0000 7f926c842700 1 -- 172.21.15.121:0/741797849 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264008c40 con 0x5598bec4dab0 2023-11-15T03:04:23.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f9279b3ad00 1 -- 172.21.15.121:0/741797849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598bec4dab0 msgr2=0x5598bec611a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/741797849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598bec4dab0 0x5598bec611a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7f925800a3b0 tx=0x7f9264016000).stop 2023-11-15T03:04:23.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f9279b3ad00 1 -- 172.21.15.121:0/741797849 shutdown_connections 2023-11-15T03:04:23.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/741797849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598bec61690 0x5598bec61b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/741797849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598bec4dab0 0x5598bec611a0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/741797849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598beced0d0 0x5598bec4d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f9279b3ad00 1 -- 172.21.15.121:0/741797849 >> 172.21.15.121:0/741797849 conn(0x5598bec6a080 msgr2=0x5598bec4e660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:23.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f9279b3ad00 1 -- 172.21.15.121:0/741797849 shutdown_connections 2023-11-15T03:04:23.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f9279b3ad00 1 -- 172.21.15.121:0/741797849 wait complete. 2023-11-15T03:04:23.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f9279b3ad00 1 Processor -- start 2023-11-15T03:04:23.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f9279b3ad00 1 -- start start 2023-11-15T03:04:23.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f9279b3ad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598bec4dab0 0x5598becebe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f9279b3ad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598bec61690 0x5598bec5f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f9279b3ad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598beced0d0 0x5598bec5f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f9279b3ad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598bec4f940 con 0x5598beced0d0 2023-11-15T03:04:23.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f9279b3ad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598bec5fed0 con 0x5598bec61690 2023-11-15T03:04:23.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f9279b3ad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598beceeff0 con 0x5598bec4dab0 2023-11-15T03:04:23.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f926e846700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598beced0d0 0x5598bec5f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f926e846700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598beced0d0 0x5598bec5f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56462/0 (socket says 172.21.15.121:56462) 2023-11-15T03:04:23.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 learned_addr learned my addr 172.21.15.121:0/2022118472 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:23.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f926e045700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598bec4dab0 0x5598becebe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f926d844700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598bec61690 0x5598bec5f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598bec4dab0 msgr2=0x5598becebe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f926e846700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598bec4dab0 0x5598becebe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598bec61690 msgr2=0x5598bec5f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f926e846700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598bec61690 0x5598bec5f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9258001e10 con 0x5598beced0d0 2023-11-15T03:04:23.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f926d844700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598bec61690 0x5598bec5f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:23.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.160+0000 7f926e846700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598beced0d0 0x5598bec5f9e0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x5598bec6ac90 tx=0x7f9268006910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:23.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.164+0000 7f925effd700 1 -- 172.21.15.121:0/2022118472 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926800ec80 con 0x5598beced0d0 2023-11-15T03:04:23.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.164+0000 7f925effd700 1 -- 172.21.15.121:0/2022118472 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926800ee00 con 0x5598beced0d0 2023-11-15T03:04:23.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.164+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5598beb97010 con 0x5598beced0d0 2023-11-15T03:04:23.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.164+0000 7f925effd700 1 -- 172.21.15.121:0/2022118472 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9268006280 con 0x5598beced0d0 2023-11-15T03:04:23.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.164+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598bec64070 con 0x5598beced0d0 2023-11-15T03:04:23.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.164+0000 7f925effd700 1 -- 172.21.15.121:0/2022118472 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9268005650 con 0x5598beced0d0 2023-11-15T03:04:23.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.168+0000 7f925effd700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9250059020 0x7f925005b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.168+0000 7f925effd700 1 -- 172.21.15.121:0/2022118472 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6696+0+0 (secure 0 0 0) 0x7f9268072960 con 0x5598beced0d0 2023-11-15T03:04:23.189 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:04:23.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.168+0000 7f926e045700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9250059020 0x7f925005b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.168+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5598bec611a0 0x5598bed05440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.168+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598becedd10 con 0x5598bec611a0 2023-11-15T03:04:23.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.168+0000 7f926d844700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5598bec611a0 0x5598bed05440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.168+0000 7f926e045700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9250059020 0x7f925005b4e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f92600015a0 tx=0x5598becf7130).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:23.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.168+0000 7f926d844700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5598bec611a0 0x5598bed05440 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:23.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.172+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v99'53 uv46 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bec611a0 2023-11-15T03:04:23.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.172+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed05930 con 0x5598bec611a0 2023-11-15T03:04:23.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.172+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v99'54 uv54 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bec611a0 2023-11-15T03:04:23.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.172+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed10ff0 con 0x5598bec611a0 2023-11-15T03:04:23.193 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:04:23.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.176+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v99'55 uv55 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bec611a0 2023-11-15T03:04:23.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.176+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed123b0 con 0x5598bec611a0 2023-11-15T03:04:23.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.176+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v99'56 uv56 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bec611a0 2023-11-15T03:04:23.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.180+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed12760 con 0x5598bec611a0 2023-11-15T03:04:23.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.180+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v99'57 uv57 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bec611a0 2023-11-15T03:04:23.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.180+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed113a0 con 0x5598bec611a0 2023-11-15T03:04:23.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.184+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v99'58 uv58 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bec611a0 2023-11-15T03:04:23.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.184+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed11750 con 0x5598bec611a0 2023-11-15T03:04:23.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.188+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v99'59 uv52 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bec611a0 2023-11-15T03:04:23.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.188+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed11b00 con 0x5598bec611a0 2023-11-15T03:04:23.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.192+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v99'60 uv60 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bec611a0 2023-11-15T03:04:23.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.192+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed11b00 con 0x5598bec611a0 2023-11-15T03:04:23.200 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:04:23.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.192+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v99'61 uv61 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bec611a0 2023-11-15T03:04:23.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.196+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed11b00 con 0x5598bec611a0 2023-11-15T03:04:23.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.196+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v99'62 uv62 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bec611a0 2023-11-15T03:04:23.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.196+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed11b00 con 0x5598bec611a0 2023-11-15T03:04:23.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.200+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v99'63 uv63 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bec611a0 2023-11-15T03:04:23.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.200+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed11b00 con 0x5598bec611a0 2023-11-15T03:04:23.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.200+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v99'64 uv64 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bec611a0 2023-11-15T03:04:23.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.200+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5598bed12b10 0x5598bed14f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.200+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed153f0 con 0x5598bed12b10 2023-11-15T03:04:23.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.204+0000 7f926e045700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5598bed12b10 0x5598bed14f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.204+0000 7f926e045700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5598bed12b10 0x5598bed14f00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:23.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.208+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v99'81 uv68 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.208+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed157a0 con 0x5598bed12b10 2023-11-15T03:04:23.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.212+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v99'82 uv82 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.212+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed15b50 con 0x5598bed12b10 2023-11-15T03:04:23.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.212+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v99'83 uv83 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.221 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:04:23.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.212+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed16f10 con 0x5598bed12b10 2023-11-15T03:04:23.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.220+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v99'84 uv84 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.220+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed172c0 con 0x5598bed12b10 2023-11-15T03:04:23.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.220+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v99'85 uv85 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.220+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed15f00 con 0x5598bed12b10 2023-11-15T03:04:23.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.224+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v99'86 uv86 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.224+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed16820 con 0x5598bed12b10 2023-11-15T03:04:23.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.228+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v99'87 uv87 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.228+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed17670 con 0x5598bed12b10 2023-11-15T03:04:23.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.228+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v99'88 uv74 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.228+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed17670 con 0x5598bed12b10 2023-11-15T03:04:23.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.232+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v99'89 uv89 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.232+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed17670 con 0x5598bed12b10 2023-11-15T03:04:23.243 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:04:23.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.236+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v99'90 uv90 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.236+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed18680 con 0x5598bed12b10 2023-11-15T03:04:23.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.236+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v99'91 uv91 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.236+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed18680 con 0x5598bed12b10 2023-11-15T03:04:23.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.240+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v99'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.240+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed17670 con 0x5598bed12b10 2023-11-15T03:04:23.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.244+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v99'93 uv93 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.244+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed17670 con 0x5598bed12b10 2023-11-15T03:04:23.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.244+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v99'94 uv80 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.244+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed17670 con 0x5598bed12b10 2023-11-15T03:04:23.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.248+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v99'95 uv95 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.248+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed17670 con 0x5598bed12b10 2023-11-15T03:04:23.258 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:04:23.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.252+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v99'96 uv96 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.252+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed18680 con 0x5598bed12b10 2023-11-15T03:04:23.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.252+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v99'97 uv97 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.252+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed18680 con 0x5598bed12b10 2023-11-15T03:04:23.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.256+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v99'98 uv98 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.256+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed17670 con 0x5598bed12b10 2023-11-15T03:04:23.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.260+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v99'99 uv99 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.260+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5598bed17b80 0x5598bed19f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.260+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1a460 con 0x5598bed17b80 2023-11-15T03:04:23.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.260+0000 7f926d844700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5598bed17b80 0x5598bed19f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.260+0000 7f926d844700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5598bed17b80 0x5598bed19f70 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:23.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.264+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v99'47 uv40 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed17b80 2023-11-15T03:04:23.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.264+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed1a460 con 0x5598bed17b80 2023-11-15T03:04:23.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.268+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v99'48 uv48 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed17b80 2023-11-15T03:04:23.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.268+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1a460 con 0x5598bed17b80 2023-11-15T03:04:23.276 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:04:23.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.268+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v99'49 uv49 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed17b80 2023-11-15T03:04:23.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.268+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1a950 con 0x5598bed17b80 2023-11-15T03:04:23.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.272+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v99'50 uv50 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed17b80 2023-11-15T03:04:23.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.276+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1a950 con 0x5598bed17b80 2023-11-15T03:04:23.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.276+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v99'51 uv51 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed17b80 2023-11-15T03:04:23.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.276+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1a950 con 0x5598bed17b80 2023-11-15T03:04:23.286 INFO:teuthology.orchestra.run.smithi121.stdout:Write #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:04:23.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.280+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v99'52 uv52 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed17b80 2023-11-15T03:04:23.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.280+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1a950 con 0x5598bed17b80 2023-11-15T03:04:23.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.284+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [write 0~90] v99'53 uv53 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed17b80 2023-11-15T03:04:23.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.284+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1a950 con 0x5598bed17b80 2023-11-15T03:04:23.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.288+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [omap-set-header] v99'54 uv54 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed17b80 2023-11-15T03:04:23.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.288+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5598bed1ae60 0x5598bed1d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.288+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:40 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1d810 con 0x5598bed1ae60 2023-11-15T03:04:23.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.288+0000 7f926e846700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5598bed1ae60 0x5598bed1d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.288+0000 7f926e846700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5598bed1ae60 0x5598bed1d250 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:23.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.292+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(40 200.00000001 [create] v99'62 uv55 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.292+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:41 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed1dd10 con 0x5598bed1ae60 2023-11-15T03:04:23.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.296+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(41 200.00000001 [delete] v99'63 uv63 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.296+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1e200 con 0x5598bed1ae60 2023-11-15T03:04:23.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.296+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(42 200.00000001 [create] v99'64 uv64 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.304 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:04:23.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.296+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1a540 con 0x5598bed1ae60 2023-11-15T03:04:23.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.300+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(43 200.00000001 [write 0~16484] v99'65 uv65 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.304+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed22c80 con 0x5598bed1ae60 2023-11-15T03:04:23.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.304+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(44 200.00000001 [omap-set-header] v99'66 uv66 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.304+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1e600 con 0x5598bed1ae60 2023-11-15T03:04:23.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.308+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(45 606.00000000 [create] v99'67 uv61 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.308+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed1ea00 con 0x5598bed1ae60 2023-11-15T03:04:23.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.312+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(46 606.00000000 [delete] v99'68 uv68 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.312+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1f370 con 0x5598bed1ae60 2023-11-15T03:04:23.322 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:04:23.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.316+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(47 606.00000000 [create] v99'69 uv69 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.316+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1f370 con 0x5598bed1ae60 2023-11-15T03:04:23.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.316+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v99'70 uv70 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.316+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed207d0 con 0x5598bed1ae60 2023-11-15T03:04:23.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.320+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v99'71 uv71 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.320+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1f3d0 con 0x5598bed1ae60 2023-11-15T03:04:23.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.324+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(50 606.00000000 [omap-set-header] v99'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.324+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5598bed1f390 0x5598bed21e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.324+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:51 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed22580 con 0x5598bed1f390 2023-11-15T03:04:23.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.324+0000 7f926e045700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5598bed1f390 0x5598bed21e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.324+0000 7f926e045700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5598bed1f390 0x5598bed21e80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:23.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.328+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(51 607.00000000 [create] v99'27 uv26 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed1f390 2023-11-15T03:04:23.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.328+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:52 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed22580 con 0x5598bed1f390 2023-11-15T03:04:23.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.332+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(52 607.00000000 [delete] v99'28 uv28 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed1f390 2023-11-15T03:04:23.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.332+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed22580 con 0x5598bed1f390 2023-11-15T03:04:23.339 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:04:23.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.332+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(53 607.00000000 [create] v99'29 uv29 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed1f390 2023-11-15T03:04:23.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.332+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed22580 con 0x5598bed1f390 2023-11-15T03:04:23.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.336+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v99'30 uv30 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed1f390 2023-11-15T03:04:23.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.336+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed22580 con 0x5598bed1f390 2023-11-15T03:04:23.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.336+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v99'31 uv31 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed1f390 2023-11-15T03:04:23.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.336+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed22580 con 0x5598bed1f390 2023-11-15T03:04:23.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.340+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v99'32 uv32 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed1f390 2023-11-15T03:04:23.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.340+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5598bed23590 0x5598bed23990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.340+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:57 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed260f0 con 0x5598bed23590 2023-11-15T03:04:23.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.340+0000 7f926d844700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5598bed23590 0x5598bed23990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.340+0000 7f926d844700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5598bed23590 0x5598bed23990 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:23.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.344+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v99'20 uv19 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.344+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:58 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed260f0 con 0x5598bed23590 2023-11-15T03:04:23.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.344+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v99'21 uv21 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.344+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed260f0 con 0x5598bed23590 2023-11-15T03:04:23.353 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:04:23.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.348+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v99'22 uv22 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.348+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1eee0 con 0x5598bed23590 2023-11-15T03:04:23.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.348+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v99'23 uv23 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.348+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1eea0 con 0x5598bed23590 2023-11-15T03:04:23.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.352+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(61 608.00000000 [create] v99'27 uv26 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.352+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed1eea0 con 0x5598bed23590 2023-11-15T03:04:23.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.356+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(62 608.00000000 [delete] v99'28 uv28 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.356+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1eea0 con 0x5598bed23590 2023-11-15T03:04:23.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.356+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(63 608.00000000 [create] v99'29 uv29 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.369 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:04:23.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.356+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1eea0 con 0x5598bed23590 2023-11-15T03:04:23.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.360+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v99'30 uv30 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.360+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1eea0 con 0x5598bed23590 2023-11-15T03:04:23.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.364+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v99'31 uv31 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.364+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1eea0 con 0x5598bed23590 2023-11-15T03:04:23.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.364+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(66 608.00000000 [omap-set-header] v99'32 uv32 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.364+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1eea0 con 0x5598bed12b10 2023-11-15T03:04:23.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.368+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(67 604.00000000 [create] v99'48 uv42 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.368+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed1eea0 con 0x5598bed12b10 2023-11-15T03:04:23.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.372+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(68 604.00000000 [delete] v99'49 uv49 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.372+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1eea0 con 0x5598bed12b10 2023-11-15T03:04:23.383 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:04:23.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.376+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(69 604.00000000 [create] v99'50 uv50 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.376+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1eea0 con 0x5598bed12b10 2023-11-15T03:04:23.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.380+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v99'51 uv51 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.380+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1eea0 con 0x5598bed12b10 2023-11-15T03:04:23.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.380+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v99'52 uv52 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.384+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1eea0 con 0x5598bed12b10 2023-11-15T03:04:23.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.384+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(72 604.00000000 [omap-set-header] v99'53 uv53 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.384+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1eea0 con 0x5598bed12b10 2023-11-15T03:04:23.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.388+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(73 500.00000000 [create] v99'54 uv47 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.388+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed1eea0 con 0x5598bed12b10 2023-11-15T03:04:23.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.388+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(74 500.00000000 [delete] v99'55 uv55 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.388+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1eea0 con 0x5598bed12b10 2023-11-15T03:04:23.399 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:04:23.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.392+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(75 500.00000000 [create] v99'56 uv56 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.392+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed1eea0 con 0x5598bed12b10 2023-11-15T03:04:23.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.396+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(76 500.00000000 [write 0~90] v99'57 uv57 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.396+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed265e0 con 0x5598bed12b10 2023-11-15T03:04:23.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.400+0000 7f926e045700 1 -- 172.21.15.121:0/2022118472 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(77 500.00000000 [omap-set-header] v99'58 uv58 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9260011d50 con 0x5598bed12b10 2023-11-15T03:04:23.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.400+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:78 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed265e0 con 0x5598bed17b80 2023-11-15T03:04:23.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.400+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(78 mds_snaptable [create] v99'22 uv21 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed17b80 2023-11-15T03:04:23.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.400+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:79 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed265e0 con 0x5598bed17b80 2023-11-15T03:04:23.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.404+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(79 mds_snaptable [delete] v99'23 uv23 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed17b80 2023-11-15T03:04:23.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.404+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed265e0 con 0x5598bed17b80 2023-11-15T03:04:23.412 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:04:23.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.404+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v99'24 uv24 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed17b80 2023-11-15T03:04:23.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.404+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed265e0 con 0x5598bed17b80 2023-11-15T03:04:23.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.408+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [write 0~70] v99'25 uv25 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed17b80 2023-11-15T03:04:23.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.408+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed265e0 con 0x5598bed17b80 2023-11-15T03:04:23.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.412+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v99'26 uv26 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed17b80 2023-11-15T03:04:23.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.412+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed265e0 con 0x5598bed23590 2023-11-15T03:04:23.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.412+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(83 605.00000000 [create] v99'27 uv26 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.412+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed265e0 con 0x5598bed23590 2023-11-15T03:04:23.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.416+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(84 605.00000000 [delete] v99'28 uv28 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.416+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed265e0 con 0x5598bed23590 2023-11-15T03:04:23.428 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:04:23.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.420+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(85 605.00000000 [create] v99'29 uv29 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.420+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed275f0 con 0x5598bed23590 2023-11-15T03:04:23.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.424+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v99'30 uv30 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.424+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed275f0 con 0x5598bed23590 2023-11-15T03:04:23.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.428+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v99'31 uv31 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.428+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed265e0 con 0x5598bed23590 2023-11-15T03:04:23.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.428+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(88 605.00000000 [omap-set-header] v99'32 uv32 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.428+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed265e0 con 0x5598bed23590 2023-11-15T03:04:23.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.432+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(89 mds0_inotable [create] v99'23 uv22 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.432+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed265e0 con 0x5598bed23590 2023-11-15T03:04:23.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.436+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(90 mds0_inotable [delete] v99'24 uv24 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.436+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed265e0 con 0x5598bed23590 2023-11-15T03:04:23.446 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:04:23.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.440+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(91 mds0_inotable [create] v99'25 uv25 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.440+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed265e0 con 0x5598bed23590 2023-11-15T03:04:23.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.444+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(92 mds0_inotable [write 0~34] v99'26 uv26 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.444+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed265e0 con 0x5598bed23590 2023-11-15T03:04:23.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.444+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v99'27 uv27 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.444+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed265e0 con 0x5598bed23590 2023-11-15T03:04:23.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.448+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(94 100.00000000 [create] v99'22 uv21 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.448+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed265e0 con 0x5598bed23590 2023-11-15T03:04:23.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.452+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(95 100.00000000 [delete] v99'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.452+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed265e0 con 0x5598bed23590 2023-11-15T03:04:23.464 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:04:23.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.456+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(96 100.00000000 [create] v99'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.456+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed265e0 con 0x5598bed23590 2023-11-15T03:04:23.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.456+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(97 100.00000000 [omap-set-header] v99'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.456+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed29060 con 0x5598bed23590 2023-11-15T03:04:23.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.460+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v99'26 uv26 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.460+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5598bed29240 0x5598bed296b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.460+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed29ba0 con 0x5598bed29240 2023-11-15T03:04:23.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.460+0000 7f926e846700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5598bed29240 0x5598bed296b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.460+0000 7f926e846700 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5598bed29240 0x5598bed296b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:23.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.464+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(99 mds0_sessionmap [create] v99'19 uv18 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed29240 2023-11-15T03:04:23.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.464+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed29ba0 con 0x5598bed29240 2023-11-15T03:04:23.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.468+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(100 mds0_sessionmap [delete] v99'20 uv20 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed29240 2023-11-15T03:04:23.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.468+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed29ba0 con 0x5598bed29240 2023-11-15T03:04:23.477 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:04:23.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.472+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(101 mds0_sessionmap [create] v99'21 uv21 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed29240 2023-11-15T03:04:23.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.472+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed29ba0 con 0x5598bed29240 2023-11-15T03:04:23.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.472+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v99'22 uv22 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed29240 2023-11-15T03:04:23.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.472+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed29ba0 con 0x5598bed23590 2023-11-15T03:04:23.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.476+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(103 609.00000000 [create] v99'27 uv26 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.476+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed29ba0 con 0x5598bed23590 2023-11-15T03:04:23.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.480+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(104 609.00000000 [delete] v99'28 uv28 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.480+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed29ba0 con 0x5598bed23590 2023-11-15T03:04:23.489 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:04:23.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.484+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(105 609.00000000 [create] v99'29 uv29 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.484+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed2abb0 con 0x5598bed23590 2023-11-15T03:04:23.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.484+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v99'30 uv30 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.484+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed2abb0 con 0x5598bed23590 2023-11-15T03:04:23.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.488+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v99'31 uv31 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.488+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed29ba0 con 0x5598bed23590 2023-11-15T03:04:23.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.492+0000 7f926d844700 1 -- 172.21.15.121:0/2022118472 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(108 609.00000000 [omap-set-header] v99'32 uv32 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9264008b20 con 0x5598bed23590 2023-11-15T03:04:23.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.492+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed28610 con 0x5598bed1ae60 2023-11-15T03:04:23.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.492+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(109 400.00000000 [create] v99'22 uv21 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.492+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed29ba0 con 0x5598bed1ae60 2023-11-15T03:04:23.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.496+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(110 400.00000000 [delete] v99'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.496+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed29ba0 con 0x5598bed1ae60 2023-11-15T03:04:23.505 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:04:23.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.500+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(111 400.00000000 [create] v99'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.500+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed28ad0 con 0x5598bed1ae60 2023-11-15T03:04:23.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.500+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(112 400.00000000 [write 0~22] v99'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.500+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed2a090 con 0x5598bed1ae60 2023-11-15T03:04:23.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.504+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(113 400.00000000 [omap-set-header] v99'26 uv26 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.504+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed2a090 con 0x5598bed1ae60 2023-11-15T03:04:23.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.508+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(114 100.00000000.inode [create] v99'54 uv46 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.508+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed28ad0 con 0x5598bed1ae60 2023-11-15T03:04:23.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.508+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(115 100.00000000.inode [delete] v99'55 uv55 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.508+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed28ad0 con 0x5598bed1ae60 2023-11-15T03:04:23.519 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:04:23.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.512+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(116 100.00000000.inode [create] v99'56 uv56 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.512+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed28b10 con 0x5598bed1ae60 2023-11-15T03:04:23.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.516+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v99'57 uv57 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.516+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed29c20 con 0x5598bed1ae60 2023-11-15T03:04:23.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.516+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v99'58 uv58 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.516+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed29c20 con 0x5598bed1ae60 2023-11-15T03:04:23.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.520+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(119 1.00000000 [create] v99'59 uv53 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.520+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x5598bed2a090 con 0x5598bed1ae60 2023-11-15T03:04:23.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.524+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(120 1.00000000 [delete] v99'60 uv60 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.524+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed2a090 con 0x5598bed1ae60 2023-11-15T03:04:23.536 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:04:23.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.524+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(121 1.00000000 [create] v99'61 uv61 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.524+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed2b0a0 con 0x5598bed1ae60 2023-11-15T03:04:23.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.528+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v99'62 uv62 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.528+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed2b0a0 con 0x5598bed1ae60 2023-11-15T03:04:23.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.532+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v99'63 uv63 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.532+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed2a220 con 0x5598bed1ae60 2023-11-15T03:04:23.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v99'64 uv64 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e99) v8 -- 0x5598bed2a490 con 0x5598bed1ae60 2023-11-15T03:04:23.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v99'65 uv65 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9268048010 con 0x5598bed1ae60 2023-11-15T03:04:23.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5598bed17b80 msgr2=0x5598bed19f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5598bed17b80 0x5598bed19f70 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5598bec611a0 msgr2=0x5598bed05440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5598bec611a0 0x5598bed05440 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5598bed1f390 msgr2=0x5598bed21e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5598bed1f390 0x5598bed21e80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5598bed12b10 msgr2=0x5598bed14f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5598bed12b10 0x5598bed14f00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 reap_dead start 2023-11-15T03:04:23.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5598bed29240 msgr2=0x5598bed296b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5598bed29240 0x5598bed296b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5598bed1ae60 msgr2=0x5598bed1d250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5598bed1ae60 0x5598bed1d250 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5598bed23590 msgr2=0x5598bed23990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5598bed23590 0x5598bed23990 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9250059020 msgr2=0x7f925005b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9250059020 0x7f925005b4e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f92600015a0 tx=0x5598becf7130).stop 2023-11-15T03:04:23.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.536+0000 7f926e846700 1 -- 172.21.15.121:0/2022118472 reap_dead start 2023-11-15T03:04:23.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.540+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598beced0d0 msgr2=0x5598bec5f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.540+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598beced0d0 0x5598bec5f9e0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x5598bec6ac90 tx=0x7f9268006910).stop 2023-11-15T03:04:23.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.540+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 shutdown_connections 2023-11-15T03:04:23.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.540+0000 7f9279b3ad00 1 --2- 172.21.15.121:0/2022118472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5598beced0d0 0x5598bec5f9e0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.540+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 >> 172.21.15.121:0/2022118472 conn(0x5598bec6a080 msgr2=0x5598bec1c410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:23.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.540+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 shutdown_connections 2023-11-15T03:04:23.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.540+0000 7f9279b3ad00 1 -- 172.21.15.121:0/2022118472 wait complete. 2023-11-15T03:04:23.558 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 200.00000001 2023-11-15T03:04:23.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.656+0000 7fa54592fd00 1 Processor -- start 2023-11-15T03:04:23.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.656+0000 7fa54592fd00 1 -- start start 2023-11-15T03:04:23.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.656+0000 7fa54592fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c671cbfa30 0x55c671d95d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.656+0000 7fa54592fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c671d0ae70 0x55c671da13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.656+0000 7fa54592fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c671da0000 0x55c671cbeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.656+0000 7fa54592fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c671c38220 con 0x55c671da0000 2023-11-15T03:04:23.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.656+0000 7fa54592fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c671d0c430 con 0x55c671cbfa30 2023-11-15T03:04:23.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.656+0000 7fa54592fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c671da18c0 con 0x55c671d0ae70 2023-11-15T03:04:23.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.656+0000 7fa53a781700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c671da0000 0x55c671cbeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.656+0000 7fa53a781700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c671da0000 0x55c671cbeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56482/0 (socket says 172.21.15.121:56482) 2023-11-15T03:04:23.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.656+0000 7fa53a781700 1 -- 172.21.15.121:0/2299019996 learned_addr learned my addr 172.21.15.121:0/2299019996 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:23.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.656+0000 7fa539f80700 1 --2- 172.21.15.121:0/2299019996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c671cbfa30 0x55c671d95d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.656+0000 7fa53177f700 1 --2- 172.21.15.121:0/2299019996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c671d0ae70 0x55c671da13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa53a781700 1 -- 172.21.15.121:0/2299019996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c671d0ae70 msgr2=0x55c671da13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:23.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa53a781700 1 --2- 172.21.15.121:0/2299019996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c671d0ae70 0x55c671da13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa53a781700 1 -- 172.21.15.121:0/2299019996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c671cbfa30 msgr2=0x55c671d95d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:23.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa53a781700 1 --2- 172.21.15.121:0/2299019996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c671cbfa30 0x55c671d95d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa53a781700 1 -- 172.21.15.121:0/2299019996 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c005db0 con 0x55c671da0000 2023-11-15T03:04:23.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa53a781700 1 --2- 172.21.15.121:0/2299019996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c671da0000 0x55c671cbeea0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7fa534002920 tx=0x7fa534005d40).ready entity=mon.0 client_cookie=94808da0b5a6830 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:23.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa538f7e700 1 -- 172.21.15.121:0/2299019996 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53400d720 con 0x55c671da0000 2023-11-15T03:04:23.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa538f7e700 1 -- 172.21.15.121:0/2299019996 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53400d880 con 0x55c671da0000 2023-11-15T03:04:23.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa538f7e700 1 -- 172.21.15.121:0/2299019996 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53400dc20 con 0x55c671da0000 2023-11-15T03:04:23.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa54592fd00 1 -- 172.21.15.121:0/2299019996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c671da0000 msgr2=0x55c671cbeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa54592fd00 1 --2- 172.21.15.121:0/2299019996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c671da0000 0x55c671cbeea0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7fa534002920 tx=0x7fa534005d40).stop 2023-11-15T03:04:23.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa54592fd00 1 -- 172.21.15.121:0/2299019996 shutdown_connections 2023-11-15T03:04:23.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa54592fd00 1 --2- 172.21.15.121:0/2299019996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c671da0000 0x55c671cbeea0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa54592fd00 1 --2- 172.21.15.121:0/2299019996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c671d0ae70 0x55c671da13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa54592fd00 1 --2- 172.21.15.121:0/2299019996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c671cbfa30 0x55c671d95d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa54592fd00 1 -- 172.21.15.121:0/2299019996 >> 172.21.15.121:0/2299019996 conn(0x55c671d0f080 msgr2=0x55c671d9c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:23.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa54592fd00 1 -- 172.21.15.121:0/2299019996 shutdown_connections 2023-11-15T03:04:23.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa54592fd00 1 -- 172.21.15.121:0/2299019996 wait complete. 2023-11-15T03:04:23.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa54592fd00 1 Processor -- start 2023-11-15T03:04:23.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.660+0000 7fa54592fd00 1 -- start start 2023-11-15T03:04:23.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa54592fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c671cbfa30 0x55c671d99090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa54592fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c671d0ae70 0x55c671dacce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa54592fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c671da0000 0x55c671cbeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa54592fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c671d99580 con 0x55c671cbfa30 2023-11-15T03:04:23.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa54592fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c671dad1d0 con 0x55c671da0000 2023-11-15T03:04:23.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa54592fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c671d99820 con 0x55c671d0ae70 2023-11-15T03:04:23.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa539f80700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c671cbfa30 0x55c671d99090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa539f80700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c671cbfa30 0x55c671d99090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56490/0 (socket says 172.21.15.121:56490) 2023-11-15T03:04:23.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa539f80700 1 -- 172.21.15.121:0/3170382024 learned_addr learned my addr 172.21.15.121:0/3170382024 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:23.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa53177f700 1 --2- 172.21.15.121:0/3170382024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c671d0ae70 0x55c671dacce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa53a781700 1 --2- 172.21.15.121:0/3170382024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c671da0000 0x55c671cbeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa539f80700 1 -- 172.21.15.121:0/3170382024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c671d0ae70 msgr2=0x55c671dacce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa539f80700 1 --2- 172.21.15.121:0/3170382024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c671d0ae70 0x55c671dacce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa539f80700 1 -- 172.21.15.121:0/3170382024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c671da0000 msgr2=0x55c671cbeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa539f80700 1 --2- 172.21.15.121:0/3170382024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c671da0000 0x55c671cbeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa539f80700 1 -- 172.21.15.121:0/3170382024 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa534002600 con 0x55c671cbfa30 2023-11-15T03:04:23.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa539f80700 1 --2- 172.21.15.121:0/3170382024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c671cbfa30 0x55c671d99090 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7fa52c00a5a0 tx=0x7fa52c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:23.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa532ffd700 1 -- 172.21.15.121:0/3170382024 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c013510 con 0x55c671cbfa30 2023-11-15T03:04:23.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa532ffd700 1 -- 172.21.15.121:0/3170382024 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c013670 con 0x55c671cbfa30 2023-11-15T03:04:23.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa532ffd700 1 -- 172.21.15.121:0/3170382024 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0139f0 con 0x55c671cbfa30 2023-11-15T03:04:23.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa54592fd00 1 -- 172.21.15.121:0/3170382024 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c671c61770 con 0x55c671cbfa30 2023-11-15T03:04:23.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.664+0000 7fa54592fd00 1 -- 172.21.15.121:0/3170382024 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c671c30a70 con 0x55c671cbfa30 2023-11-15T03:04:23.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.668+0000 7fa532ffd700 1 -- 172.21.15.121:0/3170382024 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa52c028460 con 0x55c671cbfa30 2023-11-15T03:04:23.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.668+0000 7fa532ffd700 1 --2- 172.21.15.121:0/3170382024 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa51c058ff0 0x7fa51c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.668+0000 7fa532ffd700 1 -- 172.21.15.121:0/3170382024 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6696+0+0 (secure 0 0 0) 0x7fa52c0753c0 con 0x55c671cbfa30 2023-11-15T03:04:23.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.668+0000 7fa53177f700 1 --2- 172.21.15.121:0/3170382024 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa51c058ff0 0x7fa51c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.668+0000 7fa54592fd00 1 --2- 172.21.15.121:0/3170382024 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c671d0e990 0x55c671cf3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.668+0000 7fa54592fd00 1 -- 172.21.15.121:0/3170382024 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e99) v8 -- 0x55c671cf3c80 con 0x55c671d0e990 2023-11-15T03:04:23.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.668+0000 7fa53177f700 1 --2- 172.21.15.121:0/3170382024 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa51c058ff0 0x7fa51c05b4b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x55c671d0e740 tx=0x7fa524003050).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:23.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.668+0000 7fa53a781700 1 --2- 172.21.15.121:0/3170382024 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c671d0e990 0x55c671cf3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.672+0000 7fa53a781700 1 --2- 172.21.15.121:0/3170382024 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c671d0e990 0x55c671cf3790 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:23.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.676+0000 7fa53a781700 1 -- 172.21.15.121:0/3170382024 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 200.00000001 [delete] v99'73 uv73 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa534017020 con 0x55c671d0e990 2023-11-15T03:04:23.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.676+0000 7fa54592fd00 1 -- 172.21.15.121:0/3170382024 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c671d0e990 msgr2=0x55c671cf3790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.676+0000 7fa54592fd00 1 --2- 172.21.15.121:0/3170382024 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c671d0e990 0x55c671cf3790 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.676+0000 7fa54592fd00 1 -- 172.21.15.121:0/3170382024 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa51c058ff0 msgr2=0x7fa51c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.676+0000 7fa54592fd00 1 --2- 172.21.15.121:0/3170382024 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa51c058ff0 0x7fa51c05b4b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x55c671d0e740 tx=0x7fa524003050).stop 2023-11-15T03:04:23.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.676+0000 7fa54592fd00 1 -- 172.21.15.121:0/3170382024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c671cbfa30 msgr2=0x55c671d99090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.676+0000 7fa54592fd00 1 --2- 172.21.15.121:0/3170382024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c671cbfa30 0x55c671d99090 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7fa52c00a5a0 tx=0x7fa52c016000).stop 2023-11-15T03:04:23.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.676+0000 7fa53a781700 1 -- 172.21.15.121:0/3170382024 reap_dead start 2023-11-15T03:04:23.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.676+0000 7fa54592fd00 1 -- 172.21.15.121:0/3170382024 shutdown_connections 2023-11-15T03:04:23.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.676+0000 7fa54592fd00 1 -- 172.21.15.121:0/3170382024 >> 172.21.15.121:0/3170382024 conn(0x55c671d0f080 msgr2=0x55c671d9c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:23.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.676+0000 7fa54592fd00 1 -- 172.21.15.121:0/3170382024 shutdown_connections 2023-11-15T03:04:23.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.676+0000 7fa54592fd00 1 -- 172.21.15.121:0/3170382024 wait complete. 2023-11-15T03:04:23.687 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:04:23.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.848+0000 7fa19ef07700 1 Processor -- start 2023-11-15T03:04:23.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.848+0000 7fa19ef07700 1 -- start start 2023-11-15T03:04:23.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.852+0000 7fa19ef07700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa1980c5c70 0x7fa198153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.852+0000 7fa19ef07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa198066330 0x7fa1980ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.852+0000 7fa19ef07700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa198065ea0 con 0x7fa1980acd00 2023-11-15T03:04:23.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.852+0000 7fa19ef07700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa198066bb0 con 0x7fa1980c5c70 2023-11-15T03:04:23.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.852+0000 7fa19ef07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa198065890 con 0x7fa198066330 2023-11-15T03:04:23.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.852+0000 7fa19d704700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa1980c5c70 0x7fa198153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.852+0000 7fa19e706700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa198066330 0x7fa1980ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.852+0000 7fa19d704700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa1980c5c70 0x7fa198153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56492/0 (socket says 172.21.15.121:56492) 2023-11-15T03:04:23.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.852+0000 7fa19d704700 1 -- 172.21.15.121:0/1376106698 learned_addr learned my addr 172.21.15.121:0/1376106698 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:23.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.852+0000 7fa19d704700 1 -- 172.21.15.121:0/1376106698 >> v2:172.21.15.190:3300/0 conn(0x7fa198066330 msgr2=0x7fa1980ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:23.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.852+0000 7fa19d704700 1 --2- 172.21.15.121:0/1376106698 >> v2:172.21.15.190:3300/0 conn(0x7fa198066330 0x7fa1980ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.852+0000 7fa19d704700 1 -- 172.21.15.121:0/1376106698 >> v1:172.21.15.190:6789/0 conn(0x7fa1980acd00 legacy=0x7fa1981536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.852+0000 7fa19d704700 1 -- 172.21.15.121:0/1376106698 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1980c64a0 con 0x7fa1980c5c70 2023-11-15T03:04:23.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.852+0000 7fa19d704700 1 --2- 172.21.15.121:0/1376106698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1980c5c70 0x7fa198153dd0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7fa1980c4f20 tx=0x7fa1880090f0).ready entity=mon.0 client_cookie=2b4c79a1a6597ce5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:23.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.856+0000 7fa19cf03700 1 -- 172.21.15.121:0/1376106698 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18800b520 con 0x7fa1980c5c70 2023-11-15T03:04:23.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.856+0000 7fa19cf03700 1 -- 172.21.15.121:0/1376106698 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18800b680 con 0x7fa1980c5c70 2023-11-15T03:04:23.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.856+0000 7fa19cf03700 1 -- 172.21.15.121:0/1376106698 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18800ba00 con 0x7fa1980c5c70 2023-11-15T03:04:23.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.856+0000 7fa19ef07700 1 -- 172.21.15.121:0/1376106698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1980c5c70 msgr2=0x7fa198153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.856+0000 7fa19ef07700 1 --2- 172.21.15.121:0/1376106698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1980c5c70 0x7fa198153dd0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7fa1980c4f20 tx=0x7fa1880090f0).stop 2023-11-15T03:04:23.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.856+0000 7fa19ef07700 1 -- 172.21.15.121:0/1376106698 shutdown_connections 2023-11-15T03:04:23.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.856+0000 7fa19ef07700 1 --2- 172.21.15.121:0/1376106698 >> v2:172.21.15.190:3300/0 conn(0x7fa198066330 0x7fa1980ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.856+0000 7fa19ef07700 1 --2- 172.21.15.121:0/1376106698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1980c5c70 0x7fa198153dd0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.856+0000 7fa19ef07700 1 -- 172.21.15.121:0/1376106698 >> 172.21.15.121:0/1376106698 conn(0x7fa198063fc0 msgr2=0x7fa198067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:23.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.856+0000 7fa19ef07700 1 -- 172.21.15.121:0/1376106698 shutdown_connections 2023-11-15T03:04:23.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.856+0000 7fa19ef07700 1 -- 172.21.15.121:0/1376106698 wait complete. 2023-11-15T03:04:23.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.856+0000 7fa19ef07700 1 Processor -- start 2023-11-15T03:04:23.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19ef07700 1 -- start start 2023-11-15T03:04:23.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19ef07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa198066330 0x7fa1980c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19ef07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1980acd00 0x7fa1980ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19ef07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1980c5c70 0x7fa198158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19ef07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980af410 con 0x7fa1980acd00 2023-11-15T03:04:23.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19ef07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980af130 con 0x7fa198066330 2023-11-15T03:04:23.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19ef07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980c37e0 con 0x7fa1980c5c70 2023-11-15T03:04:23.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19d704700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1980acd00 0x7fa1980ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19d704700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1980acd00 0x7fa1980ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56508/0 (socket says 172.21.15.121:56508) 2023-11-15T03:04:23.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19d704700 1 -- 172.21.15.121:0/2397848869 learned_addr learned my addr 172.21.15.121:0/2397848869 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:23.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19df05700 1 --2- 172.21.15.121:0/2397848869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa198066330 0x7fa1980c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19e706700 1 --2- 172.21.15.121:0/2397848869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1980c5c70 0x7fa198158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19d704700 1 -- 172.21.15.121:0/2397848869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1980c5c70 msgr2=0x7fa198158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19d704700 1 --2- 172.21.15.121:0/2397848869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1980c5c70 0x7fa198158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19d704700 1 -- 172.21.15.121:0/2397848869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa198066330 msgr2=0x7fa1980c42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:23.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19d704700 1 --2- 172.21.15.121:0/2397848869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa198066330 0x7fa1980c42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:23.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19d704700 1 -- 172.21.15.121:0/2397848869 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa188009370 con 0x7fa1980acd00 2023-11-15T03:04:23.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19e706700 1 --2- 172.21.15.121:0/2397848869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1980c5c70 0x7fa198158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:04:23.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19d704700 1 --2- 172.21.15.121:0/2397848869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1980acd00 0x7fa1980ae830 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7fa198064f20 tx=0x7fa188017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:23.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa18dffb700 1 -- 172.21.15.121:0/2397848869 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18800b7f0 con 0x7fa1980acd00 2023-11-15T03:04:23.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa18dffb700 1 -- 172.21.15.121:0/2397848869 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18800b950 con 0x7fa1980acd00 2023-11-15T03:04:23.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19ef07700 1 -- 172.21.15.121:0/2397848869 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1980c3ac0 con 0x7fa1980acd00 2023-11-15T03:04:23.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa18dffb700 1 -- 172.21.15.121:0/2397848869 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18801c670 con 0x7fa1980acd00 2023-11-15T03:04:23.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.860+0000 7fa19ef07700 1 -- 172.21.15.121:0/2397848869 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1980c47b0 con 0x7fa1980acd00 2023-11-15T03:04:23.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.864+0000 7fa19ef07700 1 -- 172.21.15.121:0/2397848869 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1980c3640 con 0x7fa1980acd00 2023-11-15T03:04:23.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.864+0000 7fa18dffb700 1 -- 172.21.15.121:0/2397848869 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa18801c9e0 con 0x7fa1980acd00 2023-11-15T03:04:23.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.868+0000 7fa18dffb700 1 --2- 172.21.15.121:0/2397848869 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa17c05a510 0x7fa17c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:23.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.868+0000 7fa18dffb700 1 -- 172.21.15.121:0/2397848869 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6696+0+0 (secure 0 0 0) 0x7fa188076610 con 0x7fa1980acd00 2023-11-15T03:04:23.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.868+0000 7fa19df05700 1 --2- 172.21.15.121:0/2397848869 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa17c05a510 0x7fa17c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:23.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.868+0000 7fa19df05700 1 --2- 172.21.15.121:0/2397848869 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa17c05a510 0x7fa17c05c9d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fa194003710 tx=0x7fa194009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:23.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:23.872+0000 7fa18dffb700 1 -- 172.21.15.121:0/2397848869 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa1980c3640 con 0x7fa1980acd00 2023-11-15T03:04:24.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:24.072+0000 7fa19ef07700 1 -- 172.21.15.121:0/2397848869 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fa1980c3640 con 0x7fa1980acd00 2023-11-15T03:04:25.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.088+0000 7fa18dffb700 1 -- 172.21.15.121:0/2397848869 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v113) v1 ==== 162+0+0 (secure 0 0 0) 0x7fa1980c3640 con 0x7fa1980acd00 2023-11-15T03:04:25.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.092+0000 7fa19ef07700 1 -- 172.21.15.121:0/2397848869 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa17c05a510 msgr2=0x7fa17c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:25.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.092+0000 7fa19ef07700 1 --2- 172.21.15.121:0/2397848869 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa17c05a510 0x7fa17c05c9d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fa194003710 tx=0x7fa194009040).stop 2023-11-15T03:04:25.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.092+0000 7fa19ef07700 1 -- 172.21.15.121:0/2397848869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1980acd00 msgr2=0x7fa1980ae830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:25.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.092+0000 7fa19ef07700 1 --2- 172.21.15.121:0/2397848869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1980acd00 0x7fa1980ae830 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7fa198064f20 tx=0x7fa188017000).stop 2023-11-15T03:04:25.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.092+0000 7fa19ef07700 1 -- 172.21.15.121:0/2397848869 shutdown_connections 2023-11-15T03:04:25.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.092+0000 7fa19ef07700 1 --2- 172.21.15.121:0/2397848869 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa17c05a510 0x7fa17c05c9d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:25.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.092+0000 7fa19ef07700 1 --2- 172.21.15.121:0/2397848869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1980c5c70 0x7fa198158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:25.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.092+0000 7fa19ef07700 1 --2- 172.21.15.121:0/2397848869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1980acd00 0x7fa1980ae830 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:25.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.092+0000 7fa19ef07700 1 --2- 172.21.15.121:0/2397848869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa198066330 0x7fa1980c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:25.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.092+0000 7fa19ef07700 1 -- 172.21.15.121:0/2397848869 >> 172.21.15.121:0/2397848869 conn(0x7fa198063fc0 msgr2=0x7fa19814f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:25.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.092+0000 7fa19ef07700 1 -- 172.21.15.121:0/2397848869 shutdown_connections 2023-11-15T03:04:25.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.092+0000 7fa19ef07700 1 -- 172.21.15.121:0/2397848869 wait complete. 2023-11-15T03:04:25.103 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:04:25.111 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:04:25.104+0000 7fb3883e7700 -1 log_channel(cluster) log [ERR] : missing journal object 2023-11-15T03:04:25.128 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:04:25.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.280+0000 7fec960db700 1 Processor -- start 2023-11-15T03:04:25.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.280+0000 7fec960db700 1 -- start start 2023-11-15T03:04:25.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.280+0000 7fec960db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fec90152780 0x7fec90159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:25.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.280+0000 7fec960db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec900c65d0 0x7fec900c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:25.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.280+0000 7fec960db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fec900a5510 0x7fec900a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:25.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.280+0000 7fec960db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec900c3fa0 con 0x7fec90152780 2023-11-15T03:04:25.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.280+0000 7fec960db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec90159630 con 0x7fec900a5510 2023-11-15T03:04:25.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.280+0000 7fec960db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec900c30b0 con 0x7fec900c65d0 2023-11-15T03:04:25.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.280+0000 7fec8f7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fec90152780 0x7fec90159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:25.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.280+0000 7fec8f7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fec90152780 0x7fec90159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:47898/0 (socket says 172.21.15.121:47898) 2023-11-15T03:04:25.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.280+0000 7fec8f7fe700 1 -- 172.21.15.121:0/1549736307 learned_addr learned my addr 172.21.15.121:0/1549736307 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:25.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.280+0000 7fec8effd700 1 --2- 172.21.15.121:0/1549736307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec900c65d0 0x7fec900c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:25.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.280+0000 7fec8ffff700 1 --2- 172.21.15.121:0/1549736307 >> v2:172.21.15.190:3300/0 conn(0x7fec900a5510 0x7fec900a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:25.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.280+0000 7fec8effd700 1 -- 172.21.15.121:0/1549736307 >> v2:172.21.15.190:3300/0 conn(0x7fec900a5510 msgr2=0x7fec900a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:25.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.280+0000 7fec8effd700 1 --2- 172.21.15.121:0/1549736307 >> v2:172.21.15.190:3300/0 conn(0x7fec900a5510 0x7fec900a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:25.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.280+0000 7fec8effd700 1 -- 172.21.15.121:0/1549736307 >> v2:172.21.15.121:3300/0 conn(0x7fec90152780 msgr2=0x7fec90159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:25.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.280+0000 7fec8effd700 1 --2- 172.21.15.121:0/1549736307 >> v2:172.21.15.121:3300/0 conn(0x7fec90152780 0x7fec90159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:25.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.280+0000 7fec8effd700 1 -- 172.21.15.121:0/1549736307 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec900c3390 con 0x7fec900c65d0 2023-11-15T03:04:25.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.280+0000 7fec8f7fe700 1 --2- 172.21.15.121:0/1549736307 >> v2:172.21.15.121:3300/0 conn(0x7fec90152780 0x7fec90159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:04:25.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.284+0000 7fec8effd700 1 --2- 172.21.15.121:0/1549736307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec900c65d0 0x7fec900c6a50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fec90065600 tx=0x7fec84009060).ready entity=mon.2 client_cookie=8f077dc12d867b0b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:25.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.284+0000 7fec8e7fc700 1 -- 172.21.15.121:0/1549736307 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8400f510 con 0x7fec900c65d0 2023-11-15T03:04:25.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.284+0000 7fec8e7fc700 1 -- 172.21.15.121:0/1549736307 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8400f670 con 0x7fec900c65d0 2023-11-15T03:04:25.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.284+0000 7fec8e7fc700 1 -- 172.21.15.121:0/1549736307 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8400f9f0 con 0x7fec900c65d0 2023-11-15T03:04:25.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.284+0000 7fec960db700 1 -- 172.21.15.121:0/1549736307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec900c65d0 msgr2=0x7fec900c6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:25.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.284+0000 7fec960db700 1 --2- 172.21.15.121:0/1549736307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec900c65d0 0x7fec900c6a50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fec90065600 tx=0x7fec84009060).stop 2023-11-15T03:04:25.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.284+0000 7fec960db700 1 -- 172.21.15.121:0/1549736307 shutdown_connections 2023-11-15T03:04:25.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.284+0000 7fec960db700 1 --2- 172.21.15.121:0/1549736307 >> v2:172.21.15.190:3300/0 conn(0x7fec900a5510 0x7fec900a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:25.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.284+0000 7fec960db700 1 --2- 172.21.15.121:0/1549736307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec900c65d0 0x7fec900c6a50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:25.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.284+0000 7fec960db700 1 --2- 172.21.15.121:0/1549736307 >> v2:172.21.15.121:3300/0 conn(0x7fec90152780 0x7fec90159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:25.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.284+0000 7fec960db700 1 -- 172.21.15.121:0/1549736307 >> 172.21.15.121:0/1549736307 conn(0x7fec90062190 msgr2=0x7fec900625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:25.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.284+0000 7fec960db700 1 -- 172.21.15.121:0/1549736307 shutdown_connections 2023-11-15T03:04:25.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.284+0000 7fec960db700 1 -- 172.21.15.121:0/1549736307 wait complete. 2023-11-15T03:04:25.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec960db700 1 Processor -- start 2023-11-15T03:04:25.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec960db700 1 -- start start 2023-11-15T03:04:25.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec960db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec900a5510 0x7fec90147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:25.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec960db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec900c65d0 0x7fec90168c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:25.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec960db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec90152780 0x7fec9015bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:25.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec960db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9015dc10 con 0x7fec90152780 2023-11-15T03:04:25.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec960db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec90169ca0 con 0x7fec900a5510 2023-11-15T03:04:25.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec960db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec90147b30 con 0x7fec900c65d0 2023-11-15T03:04:25.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec8ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec90152780 0x7fec9015bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:25.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec8ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec90152780 0x7fec9015bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:47910/0 (socket says 172.21.15.121:47910) 2023-11-15T03:04:25.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec8ffff700 1 -- 172.21.15.121:0/2173302216 learned_addr learned my addr 172.21.15.121:0/2173302216 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:25.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec8effd700 1 --2- 172.21.15.121:0/2173302216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec900c65d0 0x7fec90168c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:25.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec8f7fe700 1 --2- 172.21.15.121:0/2173302216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec900a5510 0x7fec90147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:25.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec8ffff700 1 -- 172.21.15.121:0/2173302216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec900c65d0 msgr2=0x7fec90168c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:25.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec8ffff700 1 --2- 172.21.15.121:0/2173302216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec900c65d0 0x7fec90168c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:25.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec8ffff700 1 -- 172.21.15.121:0/2173302216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec900a5510 msgr2=0x7fec90147640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:25.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec8ffff700 1 --2- 172.21.15.121:0/2173302216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec900a5510 0x7fec90147640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:25.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec8ffff700 1 -- 172.21.15.121:0/2173302216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec80009370 con 0x7fec90152780 2023-11-15T03:04:25.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec8f7fe700 1 --2- 172.21.15.121:0/2173302216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec900a5510 0x7fec90147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:25.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec8ffff700 1 --2- 172.21.15.121:0/2173302216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec90152780 0x7fec9015bc20 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7fec88009fa0 tx=0x7fec88009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:25.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec777fe700 1 -- 172.21.15.121:0/2173302216 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8800ec10 con 0x7fec90152780 2023-11-15T03:04:25.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec777fe700 1 -- 172.21.15.121:0/2173302216 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8800ed70 con 0x7fec90152780 2023-11-15T03:04:25.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.288+0000 7fec960db700 1 -- 172.21.15.121:0/2173302216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec840092a0 con 0x7fec90152780 2023-11-15T03:04:25.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.292+0000 7fec777fe700 1 -- 172.21.15.121:0/2173302216 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec88014610 con 0x7fec90152780 2023-11-15T03:04:25.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.292+0000 7fec960db700 1 -- 172.21.15.121:0/2173302216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec901696e0 con 0x7fec90152780 2023-11-15T03:04:25.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.292+0000 7fec960db700 1 -- 172.21.15.121:0/2173302216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9006c150 con 0x7fec90152780 2023-11-15T03:04:25.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.292+0000 7fec777fe700 1 -- 172.21.15.121:0/2173302216 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fec8801d3b0 con 0x7fec90152780 2023-11-15T03:04:25.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.296+0000 7fec777fe700 1 --2- 172.21.15.121:0/2173302216 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fec7c05a4e0 0x7fec7c05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:25.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.296+0000 7fec777fe700 1 -- 172.21.15.121:0/2173302216 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6782+0+0 (secure 0 0 0) 0x7fec88074f90 con 0x7fec90152780 2023-11-15T03:04:25.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.296+0000 7fec8f7fe700 1 --2- 172.21.15.121:0/2173302216 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fec7c05a4e0 0x7fec7c05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:25.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.296+0000 7fec8f7fe700 1 --2- 172.21.15.121:0/2173302216 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fec7c05a4e0 0x7fec7c05c9a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fec80006fd0 tx=0x7fec80008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:25.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.300+0000 7fec777fe700 1 -- 172.21.15.121:0/2173302216 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fec880483c0 con 0x7fec90152780 2023-11-15T03:04:25.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.504+0000 7fec960db700 1 -- 172.21.15.121:0/2173302216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec9015ddf0 con 0x7fec90152780 2023-11-15T03:04:25.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.508+0000 7fec777fe700 1 -- 172.21.15.121:0/2173302216 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4504 (secure 0 0 0) 0x7fec8804b9e0 con 0x7fec90152780 2023-11-15T03:04:25.515 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:04:25.517 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":114,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:04:25.091263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7836},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7836":{"gid":7836,"name":"a","rank":0,"incarnation":114,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/3956345798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3956345798},{"type":"v1","addr":"172.21.15.121:6836","nonce":3956345798}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:04:25.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.508+0000 7fec960db700 1 -- 172.21.15.121:0/2173302216 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fec7c05a4e0 msgr2=0x7fec7c05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:25.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.508+0000 7fec960db700 1 --2- 172.21.15.121:0/2173302216 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fec7c05a4e0 0x7fec7c05c9a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fec80006fd0 tx=0x7fec80008040).stop 2023-11-15T03:04:25.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.512+0000 7fec960db700 1 -- 172.21.15.121:0/2173302216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec90152780 msgr2=0x7fec9015bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:25.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.512+0000 7fec960db700 1 --2- 172.21.15.121:0/2173302216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec90152780 0x7fec9015bc20 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7fec88009fa0 tx=0x7fec88009900).stop 2023-11-15T03:04:25.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.512+0000 7fec960db700 1 -- 172.21.15.121:0/2173302216 shutdown_connections 2023-11-15T03:04:25.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.512+0000 7fec960db700 1 --2- 172.21.15.121:0/2173302216 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fec7c05a4e0 0x7fec7c05c9a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:25.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.512+0000 7fec960db700 1 --2- 172.21.15.121:0/2173302216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec90152780 0x7fec9015bc20 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:25.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.512+0000 7fec960db700 1 --2- 172.21.15.121:0/2173302216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec900c65d0 0x7fec90168c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:25.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.512+0000 7fec960db700 1 --2- 172.21.15.121:0/2173302216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec900a5510 0x7fec90147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:25.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.512+0000 7fec960db700 1 -- 172.21.15.121:0/2173302216 >> 172.21.15.121:0/2173302216 conn(0x7fec90062190 msgr2=0x7fec9015e080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:25.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.512+0000 7fec960db700 1 -- 172.21.15.121:0/2173302216 shutdown_connections 2023-11-15T03:04:25.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:25.512+0000 7fec960db700 1 -- 172.21.15.121:0/2173302216 wait complete. 2023-11-15T03:04:25.522 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 114 2023-11-15T03:04:25.537 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:04:26.101 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:04:26.092+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:04:26.101 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:04:26.096+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:04:26.114 INFO:tasks.ceph.mds.a.smithi121.stderr: -15> 2023-11-15T03:04:25.104+0000 7fb3883e7700 -1 log_channel(cluster) log [ERR] : missing journal object 2023-11-15T03:04:26.151 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:04:30.538 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:04:30.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.695+0000 7f81a97b9700 1 Processor -- start 2023-11-15T03:04:30.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.695+0000 7f81a97b9700 1 -- start start 2023-11-15T03:04:30.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.695+0000 7f81a97b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f81a4151870 0x7f81a4151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:30.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.695+0000 7f81a97b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81a4152140 0x7f81a40a9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:30.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.695+0000 7f81a97b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a40657f0 con 0x7f81a40a9d20 2023-11-15T03:04:30.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.695+0000 7f81a97b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a4065ad0 con 0x7f81a4151870 2023-11-15T03:04:30.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.695+0000 7f81a97b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a40aa260 con 0x7f81a4152140 2023-11-15T03:04:30.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.695+0000 7f81a3fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f81a4151870 0x7f81a4151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:30.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.695+0000 7f81a3fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f81a4151870 0x7f81a4151c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:47914/0 (socket says 172.21.15.121:47914) 2023-11-15T03:04:30.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.695+0000 7f81a3fff700 1 -- 172.21.15.121:0/1009537109 learned_addr learned my addr 172.21.15.121:0/1009537109 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:30.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.695+0000 7f81a37fe700 1 --2- 172.21.15.121:0/1009537109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81a4152140 0x7f81a40a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:30.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.699+0000 7f81a3fff700 1 -- 172.21.15.121:0/1009537109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81a4152140 msgr2=0x7f81a40a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:30.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.699+0000 7f81a3fff700 1 --2- 172.21.15.121:0/1009537109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81a4152140 0x7f81a40a9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:30.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.699+0000 7f81a3fff700 1 -- 172.21.15.121:0/1009537109 >> v1:172.21.15.190:6789/0 conn(0x7f81a40a9d20 legacy=0x7f81a40c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:30.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.699+0000 7f81a3fff700 1 -- 172.21.15.121:0/1009537109 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a40aa540 con 0x7f81a4151870 2023-11-15T03:04:30.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.699+0000 7f81a3fff700 1 --2- 172.21.15.121:0/1009537109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81a4151870 0x7f81a4151c50 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7f8194006fd0 tx=0x7f81940090f0).ready entity=mon.0 client_cookie=271a2870b041ed6d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:30.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.699+0000 7f81a2ffd700 1 -- 172.21.15.121:0/1009537109 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819400b580 con 0x7f81a4151870 2023-11-15T03:04:30.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.699+0000 7f81a2ffd700 1 -- 172.21.15.121:0/1009537109 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819400b6e0 con 0x7f81a4151870 2023-11-15T03:04:30.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.699+0000 7f81a2ffd700 1 -- 172.21.15.121:0/1009537109 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819400ba60 con 0x7f81a4151870 2023-11-15T03:04:30.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.699+0000 7f81a97b9700 1 -- 172.21.15.121:0/1009537109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81a4151870 msgr2=0x7f81a4151c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:30.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.699+0000 7f81a97b9700 1 --2- 172.21.15.121:0/1009537109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81a4151870 0x7f81a4151c50 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7f8194006fd0 tx=0x7f81940090f0).stop 2023-11-15T03:04:30.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.699+0000 7f81a97b9700 1 -- 172.21.15.121:0/1009537109 shutdown_connections 2023-11-15T03:04:30.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.699+0000 7f81a97b9700 1 --2- 172.21.15.121:0/1009537109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81a4152140 0x7f81a40a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:30.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.699+0000 7f81a97b9700 1 --2- 172.21.15.121:0/1009537109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81a4151870 0x7f81a4151c50 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:30.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.699+0000 7f81a97b9700 1 -- 172.21.15.121:0/1009537109 >> 172.21.15.121:0/1009537109 conn(0x7f81a4062190 msgr2=0x7f81a4062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:30.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.699+0000 7f81a97b9700 1 -- 172.21.15.121:0/1009537109 shutdown_connections 2023-11-15T03:04:30.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.699+0000 7f81a97b9700 1 -- 172.21.15.121:0/1009537109 wait complete. 2023-11-15T03:04:30.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.703+0000 7f81a97b9700 1 Processor -- start 2023-11-15T03:04:30.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.703+0000 7f81a97b9700 1 -- start start 2023-11-15T03:04:30.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a97b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81a40a9d20 0x7f81a4153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:30.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a97b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81a4151870 0x7f81a4165630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:30.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a97b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81a4152140 0x7f81a4165b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:30.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a97b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a414fb60 con 0x7f81a4151870 2023-11-15T03:04:30.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a97b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a40c4b30 con 0x7f81a4152140 2023-11-15T03:04:30.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a97b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a41661a0 con 0x7f81a40a9d20 2023-11-15T03:04:30.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81a4151870 0x7f81a4165630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:30.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81a4151870 0x7f81a4165630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:47918/0 (socket says 172.21.15.121:47918) 2023-11-15T03:04:30.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a37fe700 1 -- 172.21.15.121:0/3764748472 learned_addr learned my addr 172.21.15.121:0/3764748472 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:30.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a3fff700 1 --2- 172.21.15.121:0/3764748472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81a40a9d20 0x7f81a4153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:30.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a8fb8700 1 --2- 172.21.15.121:0/3764748472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81a4152140 0x7f81a4165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:30.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a37fe700 1 -- 172.21.15.121:0/3764748472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81a40a9d20 msgr2=0x7f81a4153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:30.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a37fe700 1 --2- 172.21.15.121:0/3764748472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81a40a9d20 0x7f81a4153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:30.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a37fe700 1 -- 172.21.15.121:0/3764748472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81a4152140 msgr2=0x7f81a4165b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:30.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a37fe700 1 --2- 172.21.15.121:0/3764748472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81a4152140 0x7f81a4165b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:30.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a37fe700 1 -- 172.21.15.121:0/3764748472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8194009370 con 0x7f81a4151870 2023-11-15T03:04:30.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a3fff700 1 --2- 172.21.15.121:0/3764748472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81a40a9d20 0x7f81a4153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:30.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a8fb8700 1 --2- 172.21.15.121:0/3764748472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81a4152140 0x7f81a4165b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:30.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a37fe700 1 --2- 172.21.15.121:0/3764748472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81a4151870 0x7f81a4165630 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7f819800e8d0 tx=0x7f8198009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:30.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f818bfff700 1 -- 172.21.15.121:0/3764748472 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819800c640 con 0x7f81a4151870 2023-11-15T03:04:30.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f818bfff700 1 -- 172.21.15.121:0/3764748472 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819800c7a0 con 0x7f81a4151870 2023-11-15T03:04:30.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a97b9700 1 -- 172.21.15.121:0/3764748472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a4166480 con 0x7f81a4151870 2023-11-15T03:04:30.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f818bfff700 1 -- 172.21.15.121:0/3764748472 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819800cb20 con 0x7f81a4151870 2023-11-15T03:04:30.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.707+0000 7f81a97b9700 1 -- 172.21.15.121:0/3764748472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a40c47c0 con 0x7f81a4151870 2023-11-15T03:04:30.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.711+0000 7f81a97b9700 1 -- 172.21.15.121:0/3764748472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a40c4ce0 con 0x7f81a4151870 2023-11-15T03:04:30.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.711+0000 7f818bfff700 1 -- 172.21.15.121:0/3764748472 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f819801b400 con 0x7f81a4151870 2023-11-15T03:04:30.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.715+0000 7f818bfff700 1 --2- 172.21.15.121:0/3764748472 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f819005a4e0 0x7f819005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:30.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.715+0000 7f818bfff700 1 -- 172.21.15.121:0/3764748472 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6782+0+0 (secure 0 0 0) 0x7f8198074ea0 con 0x7f81a4151870 2023-11-15T03:04:30.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.715+0000 7f81a3fff700 1 --2- 172.21.15.121:0/3764748472 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f819005a4e0 0x7f819005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:30.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.715+0000 7f81a3fff700 1 --2- 172.21.15.121:0/3764748472 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f819005a4e0 0x7f819005c9a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f8194009fd0 tx=0x7f8194017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:30.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.719+0000 7f818bfff700 1 -- 172.21.15.121:0/3764748472 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f81a40c4ce0 con 0x7f81a4151870 2023-11-15T03:04:30.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.915+0000 7f81a97b9700 1 -- 172.21.15.121:0/3764748472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81a40c4ca0 con 0x7f81a4151870 2023-11-15T03:04:30.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.915+0000 7f818bfff700 1 -- 172.21.15.121:0/3764748472 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f81980482d0 con 0x7f81a4151870 2023-11-15T03:04:30.923 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:04:30.923 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":116,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2831707405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2831707405},{"type":"v1","addr":"172.21.15.121:6836","nonce":2831707405}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:04:26.097593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:04:30.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.919+0000 7f81a97b9700 1 -- 172.21.15.121:0/3764748472 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f819005a4e0 msgr2=0x7f819005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:30.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.919+0000 7f81a97b9700 1 --2- 172.21.15.121:0/3764748472 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f819005a4e0 0x7f819005c9a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f8194009fd0 tx=0x7f8194017000).stop 2023-11-15T03:04:30.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.919+0000 7f81a97b9700 1 -- 172.21.15.121:0/3764748472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81a4151870 msgr2=0x7f81a4165630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:30.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.919+0000 7f81a97b9700 1 --2- 172.21.15.121:0/3764748472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81a4151870 0x7f81a4165630 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7f819800e8d0 tx=0x7f8198009060).stop 2023-11-15T03:04:30.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.919+0000 7f81a97b9700 1 -- 172.21.15.121:0/3764748472 shutdown_connections 2023-11-15T03:04:30.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.919+0000 7f81a97b9700 1 --2- 172.21.15.121:0/3764748472 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f819005a4e0 0x7f819005c9a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:30.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.919+0000 7f81a97b9700 1 --2- 172.21.15.121:0/3764748472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81a4152140 0x7f81a4165b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:30.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.919+0000 7f81a97b9700 1 --2- 172.21.15.121:0/3764748472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81a4151870 0x7f81a4165630 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:30.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.919+0000 7f81a97b9700 1 --2- 172.21.15.121:0/3764748472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81a40a9d20 0x7f81a4153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:30.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.919+0000 7f81a97b9700 1 -- 172.21.15.121:0/3764748472 >> 172.21.15.121:0/3764748472 conn(0x7f81a4062190 msgr2=0x7f81a40a8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:30.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.919+0000 7f81a97b9700 1 -- 172.21.15.121:0/3764748472 shutdown_connections 2023-11-15T03:04:30.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:30.919+0000 7f81a97b9700 1 -- 172.21.15.121:0/3764748472 wait complete. 2023-11-15T03:04:30.928 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 116 2023-11-15T03:04:30.945 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:04:30.946 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:04:31.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.111+0000 7f02a1df9700 1 Processor -- start 2023-11-15T03:04:31.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.111+0000 7f02a1df9700 1 -- start start 2023-11-15T03:04:31.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.115+0000 7f02a1df9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f029c0a87f0 0x7f029c067150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:31.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.115+0000 7f02a1df9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f029c0c29a0 0x7f029c14e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:31.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.115+0000 7f02a1df9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f029c0a9350 con 0x7f029c066d70 2023-11-15T03:04:31.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.115+0000 7f02a1df9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f029c150ed0 con 0x7f029c0c29a0 2023-11-15T03:04:31.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.115+0000 7f02a1df9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f029c151420 con 0x7f029c0a87f0 2023-11-15T03:04:31.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.115+0000 7f02a15f8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f029c0c29a0 0x7f029c14e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:31.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.115+0000 7f02a0df7700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f029c066d70 0x7f029c1518b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:41252/0 (socket says 172.21.15.121:41252) 2023-11-15T03:04:31.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.115+0000 7f0293fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f029c0a87f0 0x7f029c067150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:31.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.115+0000 7f02a0df7700 1 -- 172.21.15.121:0/892634955 learned_addr learned my addr 172.21.15.121:0/892634955 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:31.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.115+0000 7f02937fe700 1 -- 172.21.15.121:0/892634955 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332046777 0 0) 0x7f029c0a9350 con 0x7f029c066d70 2023-11-15T03:04:31.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.115+0000 7f02a15f8700 1 -- 172.21.15.121:0/892634955 >> v2:172.21.15.190:3300/0 conn(0x7f029c0a87f0 msgr2=0x7f029c067150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:31.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.115+0000 7f02a15f8700 1 --2- 172.21.15.121:0/892634955 >> v2:172.21.15.190:3300/0 conn(0x7f029c0a87f0 0x7f029c067150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:31.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.115+0000 7f02a15f8700 1 -- 172.21.15.121:0/892634955 >> v1:172.21.15.190:6789/0 conn(0x7f029c066d70 legacy=0x7f029c1518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:31.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.115+0000 7f02a15f8700 1 -- 172.21.15.121:0/892634955 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029c0a8d90 con 0x7f029c0c29a0 2023-11-15T03:04:31.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.119+0000 7f02a15f8700 1 --2- 172.21.15.121:0/892634955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f029c0c29a0 0x7f029c14e000 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f0294009e60 tx=0x7f0294009780).ready entity=mon.0 client_cookie=b35eb8ccd1f51770 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:31.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.119+0000 7f02937fe700 1 -- 172.21.15.121:0/892634955 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029400e070 con 0x7f029c0c29a0 2023-11-15T03:04:31.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.119+0000 7f02937fe700 1 -- 172.21.15.121:0/892634955 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029400bb20 con 0x7f029c0c29a0 2023-11-15T03:04:31.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.119+0000 7f02937fe700 1 -- 172.21.15.121:0/892634955 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029400bea0 con 0x7f029c0c29a0 2023-11-15T03:04:31.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.119+0000 7f02a1df9700 1 -- 172.21.15.121:0/892634955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f029c0c29a0 msgr2=0x7f029c14e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:31.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.119+0000 7f02a1df9700 1 --2- 172.21.15.121:0/892634955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f029c0c29a0 0x7f029c14e000 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f0294009e60 tx=0x7f0294009780).stop 2023-11-15T03:04:31.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.119+0000 7f02a1df9700 1 -- 172.21.15.121:0/892634955 shutdown_connections 2023-11-15T03:04:31.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.119+0000 7f02a1df9700 1 --2- 172.21.15.121:0/892634955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f029c0c29a0 0x7f029c14e000 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:31.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.119+0000 7f02a1df9700 1 --2- 172.21.15.121:0/892634955 >> v2:172.21.15.190:3300/0 conn(0x7f029c0a87f0 0x7f029c067150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:31.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.119+0000 7f02a1df9700 1 -- 172.21.15.121:0/892634955 >> 172.21.15.121:0/892634955 conn(0x7f029c062190 msgr2=0x7f029c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:31.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.119+0000 7f02a1df9700 1 -- 172.21.15.121:0/892634955 shutdown_connections 2023-11-15T03:04:31.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.119+0000 7f02a1df9700 1 -- 172.21.15.121:0/892634955 wait complete. 2023-11-15T03:04:31.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.123+0000 7f02a1df9700 1 Processor -- start 2023-11-15T03:04:31.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.123+0000 7f02a1df9700 1 -- start start 2023-11-15T03:04:31.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.123+0000 7f02a1df9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f029c066d70 0x7f029c1410d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:31.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.123+0000 7f02a1df9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f029c0a87f0 0x7f029c161be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:31.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f02a1df9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f029c0c29a0 0x7f029c0bfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:31.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f02a1df9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c14c8b0 con 0x7f029c066d70 2023-11-15T03:04:31.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f02a1df9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c0bc5f0 con 0x7f029c0c29a0 2023-11-15T03:04:31.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f02a1df9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f029c0bb9e0 con 0x7f029c0a87f0 2023-11-15T03:04:31.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f02a0df7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f029c066d70 0x7f029c1410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:31.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f02a0df7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f029c066d70 0x7f029c1410d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:47938/0 (socket says 172.21.15.121:47938) 2023-11-15T03:04:31.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f02a0df7700 1 -- 172.21.15.121:0/1043863316 learned_addr learned my addr 172.21.15.121:0/1043863316 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:31.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f02a15f8700 1 --2- 172.21.15.121:0/1043863316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f029c0c29a0 0x7f029c0bfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:31.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f02a0df7700 1 -- 172.21.15.121:0/1043863316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f029c0a87f0 msgr2=0x7f029c161be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:31.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f02a0df7700 1 --2- 172.21.15.121:0/1043863316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f029c0a87f0 0x7f029c161be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:04:31.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f02a0df7700 1 -- 172.21.15.121:0/1043863316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f029c0c29a0 msgr2=0x7f029c0bfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:31.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f02a0df7700 1 --2- 172.21.15.121:0/1043863316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f029c0c29a0 0x7f029c0bfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:31.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f02a0df7700 1 -- 172.21.15.121:0/1043863316 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0294009a20 con 0x7f029c066d70 2023-11-15T03:04:31.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f02a0df7700 1 --2- 172.21.15.121:0/1043863316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f029c066d70 0x7f029c1410d0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7f029800c460 tx=0x7f0298007e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:31.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f0290ff9700 1 -- 172.21.15.121:0/1043863316 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02980072e0 con 0x7f029c066d70 2023-11-15T03:04:31.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f0290ff9700 1 -- 172.21.15.121:0/1043863316 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0298007440 con 0x7f029c066d70 2023-11-15T03:04:31.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f0290ff9700 1 -- 172.21.15.121:0/1043863316 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02980057a0 con 0x7f029c066d70 2023-11-15T03:04:31.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f02a1df9700 1 -- 172.21.15.121:0/1043863316 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029c0bbcc0 con 0x7f029c066d70 2023-11-15T03:04:31.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f02a1df9700 1 -- 172.21.15.121:0/1043863316 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029c0bc280 con 0x7f029c066d70 2023-11-15T03:04:31.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.127+0000 7f02a1df9700 1 -- 172.21.15.121:0/1043863316 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029c06bcf0 con 0x7f029c066d70 2023-11-15T03:04:31.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.131+0000 7f0290ff9700 1 -- 172.21.15.121:0/1043863316 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0298005b90 con 0x7f029c066d70 2023-11-15T03:04:31.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.131+0000 7f0290ff9700 1 --2- 172.21.15.121:0/1043863316 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f028805a510 0x7f028805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:31.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.131+0000 7f0290ff9700 1 -- 172.21.15.121:0/1043863316 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6782+0+0 (secure 0 0 0) 0x7f0298074940 con 0x7f029c066d70 2023-11-15T03:04:31.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.131+0000 7f0293fff700 1 --2- 172.21.15.121:0/1043863316 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f028805a510 0x7f028805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:31.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.131+0000 7f0293fff700 1 --2- 172.21.15.121:0/1043863316 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f028805a510 0x7f028805c9d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f029c1540f0 tx=0x7f028c009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:31.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.135+0000 7f0290ff9700 1 -- 172.21.15.121:0/1043863316 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0298047e50 con 0x7f029c066d70 2023-11-15T03:04:31.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.339+0000 7f02a1df9700 1 -- 172.21.15.121:0/1043863316 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f029c0669b0 con 0x7f029c066d70 2023-11-15T03:04:31.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.339+0000 7f0290ff9700 1 -- 172.21.15.121:0/1043863316 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f029804b470 con 0x7f029c066d70 2023-11-15T03:04:31.349 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:04:31.349 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":116,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2831707405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2831707405},{"type":"v1","addr":"172.21.15.121:6836","nonce":2831707405}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:04:26.097593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:04:31.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.343+0000 7f02a1df9700 1 -- 172.21.15.121:0/1043863316 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f028805a510 msgr2=0x7f028805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:31.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.343+0000 7f02a1df9700 1 --2- 172.21.15.121:0/1043863316 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f028805a510 0x7f028805c9d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f029c1540f0 tx=0x7f028c009060).stop 2023-11-15T03:04:31.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.343+0000 7f02a1df9700 1 -- 172.21.15.121:0/1043863316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f029c066d70 msgr2=0x7f029c1410d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:31.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.343+0000 7f02a1df9700 1 --2- 172.21.15.121:0/1043863316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f029c066d70 0x7f029c1410d0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7f029800c460 tx=0x7f0298007e90).stop 2023-11-15T03:04:31.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.343+0000 7f02a1df9700 1 -- 172.21.15.121:0/1043863316 shutdown_connections 2023-11-15T03:04:31.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.343+0000 7f02a1df9700 1 --2- 172.21.15.121:0/1043863316 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f028805a510 0x7f028805c9d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:31.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.343+0000 7f02a1df9700 1 --2- 172.21.15.121:0/1043863316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f029c0c29a0 0x7f029c0bfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:31.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.343+0000 7f02a1df9700 1 --2- 172.21.15.121:0/1043863316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f029c0a87f0 0x7f029c161be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:04:31.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.343+0000 7f02a1df9700 1 --2- 172.21.15.121:0/1043863316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f029c066d70 0x7f029c1410d0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:31.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.343+0000 7f02a1df9700 1 -- 172.21.15.121:0/1043863316 >> 172.21.15.121:0/1043863316 conn(0x7f029c062190 msgr2=0x7f029c1518b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:31.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.343+0000 7f02a1df9700 1 -- 172.21.15.121:0/1043863316 shutdown_connections 2023-11-15T03:04:31.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.343+0000 7f02a1df9700 1 -- 172.21.15.121:0/1043863316 wait complete. 2023-11-15T03:04:31.356 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 116 2023-11-15T03:04:31.372 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:04:31.376 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:04:31.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.531+0000 7fafd82db700 1 Processor -- start 2023-11-15T03:04:31.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.531+0000 7fafd82db700 1 -- start start 2023-11-15T03:04:31.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.531+0000 7fafd82db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fafd00c6d50 0x7fafd0151470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:31.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.531+0000 7fafd82db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd0152000 con 0x7fafd014af80 2023-11-15T03:04:31.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.531+0000 7fafd82db700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd00c1270 con 0x7fafd0150ff0 2023-11-15T03:04:31.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.531+0000 7fafd82db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafd00c1e10 con 0x7fafd00c6d50 2023-11-15T03:04:31.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.531+0000 7fafd6878700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fafd00c6d50 0x7fafd0151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:31.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.531+0000 7fafd6077700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fafd014af80 0x7fafd00c1700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:41100/0 (socket says 172.21.15.121:41100) 2023-11-15T03:04:31.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.531+0000 7fafd6077700 1 -- 172.21.15.121:0/3781307948 learned_addr learned my addr 172.21.15.121:0/3781307948 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:31.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.535+0000 7fafd6878700 1 -- 172.21.15.121:0/3781307948 >> v1:172.21.15.190:6789/0 conn(0x7fafd0150ff0 legacy=0x7fafd00c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:31.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.535+0000 7fafd6878700 1 -- 172.21.15.121:0/3781307948 >> v1:172.21.15.121:6789/0 conn(0x7fafd014af80 legacy=0x7fafd00c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:31.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.535+0000 7fafd6878700 1 -- 172.21.15.121:0/3781307948 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd0151a40 con 0x7fafd00c6d50 2023-11-15T03:04:31.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.535+0000 7fafd6878700 1 --2- 172.21.15.121:0/3781307948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafd00c6d50 0x7fafd0151470 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7fafc8009e60 tx=0x7fafc8009780).ready entity=mon.0 client_cookie=275e853c3f80e306 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:31.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.535+0000 7fafd5075700 1 -- 172.21.15.121:0/3781307948 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc800e070 con 0x7fafd00c6d50 2023-11-15T03:04:31.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.535+0000 7fafd5075700 1 -- 172.21.15.121:0/3781307948 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc800bc30 con 0x7fafd00c6d50 2023-11-15T03:04:31.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.535+0000 7fafd5075700 1 -- 172.21.15.121:0/3781307948 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc8012410 con 0x7fafd00c6d50 2023-11-15T03:04:31.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.535+0000 7fafd82db700 1 -- 172.21.15.121:0/3781307948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafd00c6d50 msgr2=0x7fafd0151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:31.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.535+0000 7fafd82db700 1 --2- 172.21.15.121:0/3781307948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafd00c6d50 0x7fafd0151470 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7fafc8009e60 tx=0x7fafc8009780).stop 2023-11-15T03:04:31.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.535+0000 7fafd82db700 1 -- 172.21.15.121:0/3781307948 shutdown_connections 2023-11-15T03:04:31.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.535+0000 7fafd82db700 1 --2- 172.21.15.121:0/3781307948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafd00c6d50 0x7fafd0151470 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:31.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.535+0000 7fafd82db700 1 -- 172.21.15.121:0/3781307948 >> 172.21.15.121:0/3781307948 conn(0x7fafd0062190 msgr2=0x7fafd00625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:31.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.535+0000 7fafd82db700 1 -- 172.21.15.121:0/3781307948 shutdown_connections 2023-11-15T03:04:31.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.535+0000 7fafd82db700 1 -- 172.21.15.121:0/3781307948 wait complete. 2023-11-15T03:04:31.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.539+0000 7fafd82db700 1 Processor -- start 2023-11-15T03:04:31.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.539+0000 7fafd82db700 1 -- start start 2023-11-15T03:04:31.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.539+0000 7fafd82db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafd00c6d50 0x7fafd0067220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:31.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.539+0000 7fafd82db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafd014af80 0x7fafd0164340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:31.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.539+0000 7fafd82db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafd0150ff0 0x7fafd0164830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:31.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.539+0000 7fafd82db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd014ec40 con 0x7fafd0150ff0 2023-11-15T03:04:31.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.539+0000 7fafd82db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd014f200 con 0x7fafd014af80 2023-11-15T03:04:31.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.539+0000 7fafd82db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd0164e70 con 0x7fafd00c6d50 2023-11-15T03:04:31.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.539+0000 7fafd6878700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafd0150ff0 0x7fafd0164830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:31.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.539+0000 7fafd6878700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafd0150ff0 0x7fafd0164830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:47954/0 (socket says 172.21.15.121:47954) 2023-11-15T03:04:31.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.539+0000 7fafd6878700 1 -- 172.21.15.121:0/2839697890 learned_addr learned my addr 172.21.15.121:0/2839697890 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:31.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.539+0000 7fafd5876700 1 --2- 172.21.15.121:0/2839697890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafd014af80 0x7fafd0164340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:31.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.539+0000 7fafd6077700 1 --2- 172.21.15.121:0/2839697890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafd00c6d50 0x7fafd0067220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:31.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.539+0000 7fafd6878700 1 -- 172.21.15.121:0/2839697890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafd00c6d50 msgr2=0x7fafd0067220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:31.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.539+0000 7fafd6878700 1 --2- 172.21.15.121:0/2839697890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafd00c6d50 0x7fafd0067220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:31.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.539+0000 7fafd6878700 1 -- 172.21.15.121:0/2839697890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafd014af80 msgr2=0x7fafd0164340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:31.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.539+0000 7fafd6878700 1 --2- 172.21.15.121:0/2839697890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafd014af80 0x7fafd0164340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:31.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.539+0000 7fafd6878700 1 -- 172.21.15.121:0/2839697890 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc8009a20 con 0x7fafd0150ff0 2023-11-15T03:04:31.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.539+0000 7fafd5876700 1 --2- 172.21.15.121:0/2839697890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafd014af80 0x7fafd0164340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:31.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.543+0000 7fafd6878700 1 --2- 172.21.15.121:0/2839697890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafd0150ff0 0x7fafd0164830 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7fafd0064f10 tx=0x7fafc8009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:31.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.543+0000 7fafc67fc700 1 -- 172.21.15.121:0/2839697890 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc80064a0 con 0x7fafd0150ff0 2023-11-15T03:04:31.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.543+0000 7fafc67fc700 1 -- 172.21.15.121:0/2839697890 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc8006600 con 0x7fafd0150ff0 2023-11-15T03:04:31.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.543+0000 7fafc67fc700 1 -- 172.21.15.121:0/2839697890 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc80069e0 con 0x7fafd0150ff0 2023-11-15T03:04:31.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.543+0000 7fafd82db700 1 -- 172.21.15.121:0/2839697890 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd0165150 con 0x7fafd0150ff0 2023-11-15T03:04:31.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.543+0000 7fafd82db700 1 -- 172.21.15.121:0/2839697890 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd01523a0 con 0x7fafd0150ff0 2023-11-15T03:04:31.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.543+0000 7fafd82db700 1 -- 172.21.15.121:0/2839697890 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd014f370 con 0x7fafd0150ff0 2023-11-15T03:04:31.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.543+0000 7fafc67fc700 1 -- 172.21.15.121:0/2839697890 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fafc801d630 con 0x7fafd0150ff0 2023-11-15T03:04:31.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.547+0000 7fafc67fc700 1 --2- 172.21.15.121:0/2839697890 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fafb405a590 0x7fafb405ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:31.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.547+0000 7fafc67fc700 1 -- 172.21.15.121:0/2839697890 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6782+0+0 (secure 0 0 0) 0x7fafc8076fd0 con 0x7fafd0150ff0 2023-11-15T03:04:31.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.547+0000 7fafd6077700 1 --2- 172.21.15.121:0/2839697890 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fafb405a590 0x7fafb405ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:31.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.547+0000 7fafd6077700 1 --2- 172.21.15.121:0/2839697890 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fafb405a590 0x7fafb405ca50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7fafd00c0aa0 tx=0x7fafcc007d30).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:31.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.551+0000 7fafc67fc700 1 -- 172.21.15.121:0/2839697890 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fafc804a480 con 0x7fafd0150ff0 2023-11-15T03:04:31.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.747+0000 7fafd82db700 1 -- 172.21.15.121:0/2839697890 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafd006c150 con 0x7fafd0150ff0 2023-11-15T03:04:31.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.747+0000 7fafc67fc700 1 -- 172.21.15.121:0/2839697890 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4496 (secure 0 0 0) 0x7fafc804daa0 con 0x7fafd0150ff0 2023-11-15T03:04:31.757 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:04:31.757 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":116,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2831707405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2831707405},{"type":"v1","addr":"172.21.15.121:6836","nonce":2831707405}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:04:26.097593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:04:31.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.755+0000 7fafd82db700 1 -- 172.21.15.121:0/2839697890 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fafb405a590 msgr2=0x7fafb405ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:31.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.755+0000 7fafd82db700 1 --2- 172.21.15.121:0/2839697890 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fafb405a590 0x7fafb405ca50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7fafd00c0aa0 tx=0x7fafcc007d30).stop 2023-11-15T03:04:31.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.755+0000 7fafd82db700 1 -- 172.21.15.121:0/2839697890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafd0150ff0 msgr2=0x7fafd0164830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:31.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.755+0000 7fafd82db700 1 --2- 172.21.15.121:0/2839697890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafd0150ff0 0x7fafd0164830 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7fafd0064f10 tx=0x7fafc8009780).stop 2023-11-15T03:04:31.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.755+0000 7fafd82db700 1 -- 172.21.15.121:0/2839697890 shutdown_connections 2023-11-15T03:04:31.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.755+0000 7fafd82db700 1 --2- 172.21.15.121:0/2839697890 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fafb405a590 0x7fafb405ca50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:31.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.755+0000 7fafd82db700 1 --2- 172.21.15.121:0/2839697890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafd0150ff0 0x7fafd0164830 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:31.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.755+0000 7fafd82db700 1 --2- 172.21.15.121:0/2839697890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafd014af80 0x7fafd0164340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:31.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.755+0000 7fafd82db700 1 --2- 172.21.15.121:0/2839697890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafd00c6d50 0x7fafd0067220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:31.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.755+0000 7fafd82db700 1 -- 172.21.15.121:0/2839697890 >> 172.21.15.121:0/2839697890 conn(0x7fafd0062190 msgr2=0x7fafd014a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:31.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.755+0000 7fafd82db700 1 -- 172.21.15.121:0/2839697890 shutdown_connections 2023-11-15T03:04:31.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:31.755+0000 7fafd82db700 1 -- 172.21.15.121:0/2839697890 wait complete. 2023-11-15T03:04:31.765 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 116 2023-11-15T03:04:31.783 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 200.00000001' led to DAMAGED state 2023-11-15T03:04:31.784 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_openfiles.0' 2023-11-15T03:04:31.784 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:04:31.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:31.786 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:04:31.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:04:31.809 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:04:31.809 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:04:32.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.011+0000 7f9627307700 1 Processor -- start 2023-11-15T03:04:32.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.011+0000 7f9627307700 1 -- start start 2023-11-15T03:04:32.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9627307700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96200a8880 0x7f9620151b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:32.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9627307700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96200c2c00 con 0x7f96200bb510 2023-11-15T03:04:32.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9627307700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96200657b0 con 0x7f96200abaa0 2023-11-15T03:04:32.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9627307700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9620065a90 con 0x7f96200a8880 2023-11-15T03:04:32.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625b04700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f96200bb510 0x7f9620151400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:41102/0 (socket says 172.21.15.121:41102) 2023-11-15T03:04:32.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625b04700 1 -- 172.21.15.121:0/313677826 learned_addr learned my addr 172.21.15.121:0/313677826 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:32.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9626b06700 1 --2- 172.21.15.121:0/313677826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96200a8880 0x7f9620151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:32.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625303700 1 -- 172.21.15.121:0/313677826 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1796051472 0 0) 0x7f96200c2c00 con 0x7f96200bb510 2023-11-15T03:04:32.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625303700 1 -- 172.21.15.121:0/313677826 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f960c0033f0 con 0x7f96200bb510 2023-11-15T03:04:32.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625303700 1 -- 172.21.15.121:0/313677826 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1896643125 0 0) 0x7f96200657b0 con 0x7f96200abaa0 2023-11-15T03:04:32.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625303700 1 -- 172.21.15.121:0/313677826 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96200c2c00 con 0x7f96200abaa0 2023-11-15T03:04:32.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625303700 1 -- 172.21.15.121:0/313677826 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 521156104 0 0) 0x7f960c0033f0 con 0x7f96200bb510 2023-11-15T03:04:32.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625303700 1 -- 172.21.15.121:0/313677826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96200a8880 msgr2=0x7f9620151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:32.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625303700 1 --2- 172.21.15.121:0/313677826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96200a8880 0x7f9620151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:32.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625303700 1 -- 172.21.15.121:0/313677826 >> v1:172.21.15.190:6789/0 conn(0x7f96200abaa0 legacy=0x7f9620063570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:32.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625303700 1 -- 172.21.15.121:0/313677826 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96200c2890 con 0x7f96200bb510 2023-11-15T03:04:32.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625303700 1 -- 172.21.15.121:0/313677826 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f96100015f0 con 0x7f96200bb510 2023-11-15T03:04:32.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9626b06700 1 --2- 172.21.15.121:0/313677826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96200a8880 0x7f9620151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:04:32.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625303700 1 -- 172.21.15.121:0/313677826 >> v1:172.21.15.121:6789/0 conn(0x7f96200bb510 legacy=0x7f9620151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:32.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625303700 1 --2- 172.21.15.121:0/313677826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f960c003aa0 0x7f960c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:32.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625303700 1 --2- 172.21.15.121:0/313677826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96200a8880 0x7f960c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:32.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625303700 1 --2- 172.21.15.121:0/313677826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f960c004b80 0x7f960c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:32.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625303700 1 -- 172.21.15.121:0/313677826 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9620065a90 con 0x7f960c004b80 2023-11-15T03:04:32.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625303700 1 -- 172.21.15.121:0/313677826 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c00b990 con 0x7f960c003aa0 2023-11-15T03:04:32.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625303700 1 -- 172.21.15.121:0/313677826 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960c00bb20 con 0x7f96200a8880 2023-11-15T03:04:32.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9626b06700 1 --2- 172.21.15.121:0/313677826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f960c004b80 0x7f960c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:32.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.015+0000 7f9625b04700 1 --2- 172.21.15.121:0/313677826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96200a8880 0x7f960c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:32.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.019+0000 7f9626305700 1 --2- 172.21.15.121:0/313677826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f960c003aa0 0x7f960c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:32.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.019+0000 7f9626b06700 1 -- 172.21.15.121:0/313677826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96200a8880 msgr2=0x7f960c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:32.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.019+0000 7f9626b06700 1 --2- 172.21.15.121:0/313677826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96200a8880 0x7f960c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:32.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.019+0000 7f9626b06700 1 -- 172.21.15.121:0/313677826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f960c003aa0 msgr2=0x7f960c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:32.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.019+0000 7f9626b06700 1 --2- 172.21.15.121:0/313677826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f960c003aa0 0x7f960c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:32.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.019+0000 7f9626b06700 1 -- 172.21.15.121:0/313677826 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9618009a20 con 0x7f960c004b80 2023-11-15T03:04:32.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.019+0000 7f9626305700 1 --2- 172.21.15.121:0/313677826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f960c003aa0 0x7f960c003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:32.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.019+0000 7f9626b06700 1 --2- 172.21.15.121:0/313677826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f960c004b80 0x7f960c005000 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7f9620152000 tx=0x7f961800b040).ready entity=mon.0 client_cookie=cf1b70a6e899777 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:32.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.019+0000 7f9625303700 1 -- 172.21.15.121:0/313677826 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9618017070 con 0x7f960c004b80 2023-11-15T03:04:32.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.019+0000 7f9625303700 1 -- 172.21.15.121:0/313677826 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961800f040 con 0x7f960c004b80 2023-11-15T03:04:32.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.019+0000 7f9625303700 1 -- 172.21.15.121:0/313677826 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961800edf0 con 0x7f960c004b80 2023-11-15T03:04:32.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.019+0000 7f9627307700 1 -- 172.21.15.121:0/313677826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f960c004b80 msgr2=0x7f960c005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:32.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.019+0000 7f9627307700 1 --2- 172.21.15.121:0/313677826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f960c004b80 0x7f960c005000 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7f9620152000 tx=0x7f961800b040).stop 2023-11-15T03:04:32.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.019+0000 7f9626b06700 1 -- 172.21.15.121:0/313677826 reap_dead start 2023-11-15T03:04:32.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.019+0000 7f9627307700 1 -- 172.21.15.121:0/313677826 shutdown_connections 2023-11-15T03:04:32.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.019+0000 7f9627307700 1 -- 172.21.15.121:0/313677826 >> 172.21.15.121:0/313677826 conn(0x7f9620067a60 msgr2=0x7f9620067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:32.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.019+0000 7f9627307700 1 -- 172.21.15.121:0/313677826 shutdown_connections 2023-11-15T03:04:32.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.019+0000 7f9627307700 1 -- 172.21.15.121:0/313677826 wait complete. 2023-11-15T03:04:32.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.023+0000 7f9627307700 1 Processor -- start 2023-11-15T03:04:32.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.023+0000 7f9627307700 1 -- start start 2023-11-15T03:04:32.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.023+0000 7f9627307700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f960c004b80 0x7f9620140300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:32.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.023+0000 7f9627307700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9620152a00 0x7f9620152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:32.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.023+0000 7f9627307700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9620153e40 0x7f96201542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:32.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.023+0000 7f9627307700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9620153a50 con 0x7f960c004b80 2023-11-15T03:04:32.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.023+0000 7f9627307700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96200bdac0 con 0x7f9620153e40 2023-11-15T03:04:32.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.023+0000 7f9627307700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96201547b0 con 0x7f9620152a00 2023-11-15T03:04:32.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.023+0000 7f9626305700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f960c004b80 0x7f9620140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:32.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.023+0000 7f9626305700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f960c004b80 0x7f9620140300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:47978/0 (socket says 172.21.15.121:47978) 2023-11-15T03:04:32.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.027+0000 7f9626305700 1 -- 172.21.15.121:0/2725064255 learned_addr learned my addr 172.21.15.121:0/2725064255 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:32.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.027+0000 7f9626305700 1 -- 172.21.15.121:0/2725064255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9620152a00 msgr2=0x7f9620152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:32.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.027+0000 7f9626b06700 1 --2- 172.21.15.121:0/2725064255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9620153e40 0x7f96201542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:32.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.027+0000 7f9626305700 1 --2- 172.21.15.121:0/2725064255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9620152a00 0x7f9620152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:04:32.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.027+0000 7f9626305700 1 -- 172.21.15.121:0/2725064255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9620153e40 msgr2=0x7f96201542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:32.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.027+0000 7f9626305700 1 --2- 172.21.15.121:0/2725064255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9620153e40 0x7f96201542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:32.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.027+0000 7f9626305700 1 -- 172.21.15.121:0/2725064255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962013da10 con 0x7f960c004b80 2023-11-15T03:04:32.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.027+0000 7f9626305700 1 --2- 172.21.15.121:0/2725064255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f960c004b80 0x7f9620140300 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f961c00e4e0 tx=0x7f961c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:32.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.027+0000 7f96167fc700 1 -- 172.21.15.121:0/2725064255 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c001570 con 0x7f960c004b80 2023-11-15T03:04:32.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.027+0000 7f96167fc700 1 -- 172.21.15.121:0/2725064255 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961c010040 con 0x7f960c004b80 2023-11-15T03:04:32.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.027+0000 7f9627307700 1 -- 172.21.15.121:0/2725064255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96200bd790 con 0x7f960c004b80 2023-11-15T03:04:32.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.027+0000 7f96167fc700 1 -- 172.21.15.121:0/2725064255 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c0018e0 con 0x7f960c004b80 2023-11-15T03:04:32.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.027+0000 7f9627307700 1 -- 172.21.15.121:0/2725064255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9620153770 con 0x7f960c004b80 2023-11-15T03:04:32.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.027+0000 7f9627307700 1 -- 172.21.15.121:0/2725064255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96200bd5d0 con 0x7f960c004b80 2023-11-15T03:04:32.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.027+0000 7f96167fc700 1 -- 172.21.15.121:0/2725064255 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f961c001c50 con 0x7f960c004b80 2023-11-15T03:04:32.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.031+0000 7f96167fc700 1 --2- 172.21.15.121:0/2725064255 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f960405a540 0x7f960405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:32.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.031+0000 7f96167fc700 1 -- 172.21.15.121:0/2725064255 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6782+0+0 (secure 0 0 0) 0x7f961c0745f0 con 0x7f960c004b80 2023-11-15T03:04:32.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.031+0000 7f9625b04700 1 --2- 172.21.15.121:0/2725064255 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f960405a540 0x7f960405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:32.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.031+0000 7f9625b04700 1 --2- 172.21.15.121:0/2725064255 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f960405a540 0x7f960405ca00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f9610002e30 tx=0x7f9610009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:32.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.035+0000 7f96167fc700 1 -- 172.21.15.121:0/2725064255 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f961c047b40 con 0x7f960c004b80 2023-11-15T03:04:32.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:32.235+0000 7f9627307700 1 -- 172.21.15.121:0/2725064255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9620068a70 con 0x7f960c004b80 2023-11-15T03:04:33.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.155+0000 7f96167fc700 1 -- 172.21.15.121:0/2725064255 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v117) v1 ==== 161+0+0 (secure 0 0 0) 0x7f961c04b160 con 0x7f960c004b80 2023-11-15T03:04:33.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.159+0000 7f9627307700 1 -- 172.21.15.121:0/2725064255 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f960405a540 msgr2=0x7f960405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:33.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.159+0000 7f9627307700 1 --2- 172.21.15.121:0/2725064255 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f960405a540 0x7f960405ca00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f9610002e30 tx=0x7f9610009040).stop 2023-11-15T03:04:33.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.159+0000 7f9627307700 1 -- 172.21.15.121:0/2725064255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f960c004b80 msgr2=0x7f9620140300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:33.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.159+0000 7f9627307700 1 --2- 172.21.15.121:0/2725064255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f960c004b80 0x7f9620140300 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f961c00e4e0 tx=0x7f961c009040).stop 2023-11-15T03:04:33.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.159+0000 7f9627307700 1 -- 172.21.15.121:0/2725064255 shutdown_connections 2023-11-15T03:04:33.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.159+0000 7f9627307700 1 --2- 172.21.15.121:0/2725064255 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f960405a540 0x7f960405ca00 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:33.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.159+0000 7f9627307700 1 --2- 172.21.15.121:0/2725064255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9620153e40 0x7f96201542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:33.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.159+0000 7f9627307700 1 --2- 172.21.15.121:0/2725064255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9620152a00 0x7f9620152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:04:33.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.159+0000 7f9627307700 1 --2- 172.21.15.121:0/2725064255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f960c004b80 0x7f9620140300 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:33.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.159+0000 7f9627307700 1 -- 172.21.15.121:0/2725064255 >> 172.21.15.121:0/2725064255 conn(0x7f9620067a60 msgr2=0x7f9620066370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:33.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.163+0000 7f9627307700 1 -- 172.21.15.121:0/2725064255 shutdown_connections 2023-11-15T03:04:33.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.163+0000 7f9627307700 1 -- 172.21.15.121:0/2725064255 wait complete. 2023-11-15T03:04:33.171 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:04:33.191 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:04:33.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.359+0000 7f1bbd3f9700 1 Processor -- start 2023-11-15T03:04:33.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.359+0000 7f1bbd3f9700 1 -- start start 2023-11-15T03:04:33.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.359+0000 7f1bbd3f9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1bb80ab0d0 0x7f1bb80ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:33.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.359+0000 7f1bbd3f9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb8066c90 con 0x7f1bb80add40 2023-11-15T03:04:33.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.359+0000 7f1bbd3f9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb8065e20 con 0x7f1bb80ad1b0 2023-11-15T03:04:33.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.359+0000 7f1bbd3f9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb8066080 con 0x7f1bb80ab0d0 2023-11-15T03:04:33.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.359+0000 7f1bbcbf8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f1bb80add40 0x7f1bb80ae1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:41104/0 (socket says 172.21.15.121:41104) 2023-11-15T03:04:33.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.359+0000 7f1bbcbf8700 1 -- 172.21.15.121:0/55498328 learned_addr learned my addr 172.21.15.121:0/55498328 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:33.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.359+0000 7f1bb7fff700 1 --2- 172.21.15.121:0/55498328 >> v2:172.21.15.190:3300/0 conn(0x7f1bb80ab0d0 0x7f1bb80ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:33.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.359+0000 7f1bb6ffd700 1 -- 172.21.15.121:0/55498328 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054293079 0 0) 0x7f1bb8066c90 con 0x7f1bb80add40 2023-11-15T03:04:33.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.359+0000 7f1bb6ffd700 1 -- 172.21.15.121:0/55498328 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ba00033f0 con 0x7f1bb80add40 2023-11-15T03:04:33.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.359+0000 7f1bb6ffd700 1 -- 172.21.15.121:0/55498328 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437888407 0 0) 0x7f1bb8065e20 con 0x7f1bb80ad1b0 2023-11-15T03:04:33.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.359+0000 7f1bb6ffd700 1 -- 172.21.15.121:0/55498328 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bb8066c90 con 0x7f1bb80ad1b0 2023-11-15T03:04:33.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.359+0000 7f1bb7fff700 1 -- 172.21.15.121:0/55498328 >> v1:172.21.15.190:6789/0 conn(0x7f1bb80ad1b0 legacy=0x7f1bb8065710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:33.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.359+0000 7f1bb7fff700 1 -- 172.21.15.121:0/55498328 >> v1:172.21.15.121:6789/0 conn(0x7f1bb80add40 legacy=0x7f1bb80ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:33.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.359+0000 7f1bb7fff700 1 -- 172.21.15.121:0/55498328 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb8066360 con 0x7f1bb80ab0d0 2023-11-15T03:04:33.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.363+0000 7f1bb7fff700 1 --2- 172.21.15.121:0/55498328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bb80ab0d0 0x7f1bb80ab4b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f1ba8006fd0 tx=0x7f1ba8006f40).ready entity=mon.1 client_cookie=2687cabfa00144ad server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:33.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.363+0000 7f1bb6ffd700 1 -- 172.21.15.121:0/55498328 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8005880 con 0x7f1bb80ab0d0 2023-11-15T03:04:33.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.363+0000 7f1bb6ffd700 1 -- 172.21.15.121:0/55498328 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba80059e0 con 0x7f1bb80ab0d0 2023-11-15T03:04:33.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.363+0000 7f1bb6ffd700 1 -- 172.21.15.121:0/55498328 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8005d60 con 0x7f1bb80ab0d0 2023-11-15T03:04:33.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.363+0000 7f1bbd3f9700 1 -- 172.21.15.121:0/55498328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bb80ab0d0 msgr2=0x7f1bb80ab4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:33.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.363+0000 7f1bbd3f9700 1 --2- 172.21.15.121:0/55498328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bb80ab0d0 0x7f1bb80ab4b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f1ba8006fd0 tx=0x7f1ba8006f40).stop 2023-11-15T03:04:33.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.363+0000 7f1bbd3f9700 1 -- 172.21.15.121:0/55498328 shutdown_connections 2023-11-15T03:04:33.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.363+0000 7f1bbd3f9700 1 --2- 172.21.15.121:0/55498328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bb80ab0d0 0x7f1bb80ab4b0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:33.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.363+0000 7f1bbd3f9700 1 -- 172.21.15.121:0/55498328 >> 172.21.15.121:0/55498328 conn(0x7f1bb8067a60 msgr2=0x7f1bb8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:33.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.363+0000 7f1bbd3f9700 1 -- 172.21.15.121:0/55498328 shutdown_connections 2023-11-15T03:04:33.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.363+0000 7f1bbd3f9700 1 -- 172.21.15.121:0/55498328 wait complete. 2023-11-15T03:04:33.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.363+0000 7f1bbd3f9700 1 Processor -- start 2023-11-15T03:04:33.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.363+0000 7f1bbd3f9700 1 -- start start 2023-11-15T03:04:33.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1bbd3f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bb80ab0d0 0x7f1bb80c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:33.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1bbd3f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bb80ad1b0 0x7f1bb80a8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:33.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1bbd3f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bb80add40 0x7f1bb80a8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:33.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1bbd3f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb80a9320 con 0x7f1bb80ad1b0 2023-11-15T03:04:33.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1bbd3f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb80a51c0 con 0x7f1bb80ab0d0 2023-11-15T03:04:33.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1bbd3f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb80a6a20 con 0x7f1bb80add40 2023-11-15T03:04:33.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1bb77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bb80ad1b0 0x7f1bb80a8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:33.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1bb7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bb80ab0d0 0x7f1bb80c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:33.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1bb77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bb80ad1b0 0x7f1bb80a8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:47982/0 (socket says 172.21.15.121:47982) 2023-11-15T03:04:33.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1bbcbf8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bb80add40 0x7f1bb80a8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:33.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1bb77fe700 1 -- 172.21.15.121:0/2645053512 learned_addr learned my addr 172.21.15.121:0/2645053512 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:33.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1bb77fe700 1 -- 172.21.15.121:0/2645053512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bb80add40 msgr2=0x7f1bb80a8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:33.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1bb77fe700 1 --2- 172.21.15.121:0/2645053512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bb80add40 0x7f1bb80a8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:33.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1bb77fe700 1 -- 172.21.15.121:0/2645053512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bb80ab0d0 msgr2=0x7f1bb80c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:33.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1bb77fe700 1 --2- 172.21.15.121:0/2645053512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bb80ab0d0 0x7f1bb80c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:33.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1bb77fe700 1 -- 172.21.15.121:0/2645053512 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba8009250 con 0x7f1bb80ad1b0 2023-11-15T03:04:33.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1bb77fe700 1 --2- 172.21.15.121:0/2645053512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bb80ad1b0 0x7f1bb80a8940 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7f1bac00c490 tx=0x7f1bac007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:33.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1b9ffff700 1 -- 172.21.15.121:0/2645053512 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac00ce40 con 0x7f1bb80ad1b0 2023-11-15T03:04:33.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1b9ffff700 1 -- 172.21.15.121:0/2645053512 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bac00f470 con 0x7f1bb80ad1b0 2023-11-15T03:04:33.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1b9ffff700 1 -- 172.21.15.121:0/2645053512 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac00f830 con 0x7f1bb80ad1b0 2023-11-15T03:04:33.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1bbd3f9700 1 -- 172.21.15.121:0/2645053512 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb80a6d00 con 0x7f1bb80ad1b0 2023-11-15T03:04:33.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.367+0000 7f1bbd3f9700 1 -- 172.21.15.121:0/2645053512 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb80c6780 con 0x7f1bb80ad1b0 2023-11-15T03:04:33.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.371+0000 7f1bbd3f9700 1 -- 172.21.15.121:0/2645053512 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb806bcf0 con 0x7f1bb80ad1b0 2023-11-15T03:04:33.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.371+0000 7f1b9ffff700 1 -- 172.21.15.121:0/2645053512 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1bac00fbe0 con 0x7f1bb80ad1b0 2023-11-15T03:04:33.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.371+0000 7f1b9ffff700 1 --2- 172.21.15.121:0/2645053512 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1ba405a510 0x7f1ba405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:33.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.371+0000 7f1b9ffff700 1 -- 172.21.15.121:0/2645053512 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6782+0+0 (secure 0 0 0) 0x7f1bac073ff0 con 0x7f1bb80ad1b0 2023-11-15T03:04:33.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.371+0000 7f1bb7fff700 1 --2- 172.21.15.121:0/2645053512 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1ba405a510 0x7f1ba405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:33.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.375+0000 7f1bb7fff700 1 --2- 172.21.15.121:0/2645053512 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1ba405a510 0x7f1ba405c9d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f1ba8006fa0 tx=0x7f1ba8015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:33.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.379+0000 7f1b9ffff700 1 -- 172.21.15.121:0/2645053512 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1bac0a2af0 con 0x7f1bb80ad1b0 2023-11-15T03:04:33.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:33.599+0000 7f1bbd3f9700 1 -- 172.21.15.121:0/2645053512 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f1bb80c7160 con 0x7f1bb80ad1b0 2023-11-15T03:04:34.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.175+0000 7f1b9ffff700 1 -- 172.21.15.121:0/2645053512 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v118) v1 ==== 101+0+0 (secure 0 0 0) 0x7f1bac0474a0 con 0x7f1bb80ad1b0 2023-11-15T03:04:34.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.179+0000 7f1bbd3f9700 1 -- 172.21.15.121:0/2645053512 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1ba405a510 msgr2=0x7f1ba405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.179+0000 7f1bbd3f9700 1 --2- 172.21.15.121:0/2645053512 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1ba405a510 0x7f1ba405c9d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f1ba8006fa0 tx=0x7f1ba8015000).stop 2023-11-15T03:04:34.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.179+0000 7f1bbd3f9700 1 -- 172.21.15.121:0/2645053512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bb80ad1b0 msgr2=0x7f1bb80a8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.179+0000 7f1bbd3f9700 1 --2- 172.21.15.121:0/2645053512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bb80ad1b0 0x7f1bb80a8940 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7f1bac00c490 tx=0x7f1bac007c60).stop 2023-11-15T03:04:34.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.179+0000 7f1bbd3f9700 1 -- 172.21.15.121:0/2645053512 shutdown_connections 2023-11-15T03:04:34.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.179+0000 7f1bbd3f9700 1 --2- 172.21.15.121:0/2645053512 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1ba405a510 0x7f1ba405c9d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.179+0000 7f1bbd3f9700 1 --2- 172.21.15.121:0/2645053512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bb80add40 0x7f1bb80a8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.179+0000 7f1bbd3f9700 1 --2- 172.21.15.121:0/2645053512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bb80ad1b0 0x7f1bb80a8940 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.179+0000 7f1bbd3f9700 1 --2- 172.21.15.121:0/2645053512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bb80ab0d0 0x7f1bb80c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.179+0000 7f1bbd3f9700 1 -- 172.21.15.121:0/2645053512 >> 172.21.15.121:0/2645053512 conn(0x7f1bb8067a60 msgr2=0x7f1bb81541a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:34.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.179+0000 7f1bbd3f9700 1 -- 172.21.15.121:0/2645053512 shutdown_connections 2023-11-15T03:04:34.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.179+0000 7f1bbd3f9700 1 -- 172.21.15.121:0/2645053512 wait complete. 2023-11-15T03:04:34.190 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:04:34.209 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:04:34.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.263+0000 7f2a8ced8d00 1 Processor -- start 2023-11-15T03:04:34.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a8ced8d00 1 -- start start 2023-11-15T03:04:34.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a8ced8d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c67d537860 0x55c67d607de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a8ced8d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c67d60d2a0 0x55c67d564dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a8ced8d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67d563470 0x55c67d5652c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a8ced8d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c67d5657b0 con 0x55c67d537860 2023-11-15T03:04:34.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a8ced8d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c67d57f440 con 0x55c67d563470 2023-11-15T03:04:34.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a8ced8d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c67d57f720 con 0x55c67d60d2a0 2023-11-15T03:04:34.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a8a778700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c67d537860 0x55c67d607de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a8a778700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c67d537860 0x55c67d607de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48012/0 (socket says 172.21.15.121:48012) 2023-11-15T03:04:34.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a8a778700 1 -- 172.21.15.121:0/3360934295 learned_addr learned my addr 172.21.15.121:0/3360934295 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:34.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a8af79700 1 --2- 172.21.15.121:0/3360934295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67d563470 0x55c67d5652c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a81f77700 1 --2- 172.21.15.121:0/3360934295 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c67d60d2a0 0x55c67d564dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a8a778700 1 -- 172.21.15.121:0/3360934295 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c67d60d2a0 msgr2=0x55c67d564dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:34.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a8a778700 1 --2- 172.21.15.121:0/3360934295 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c67d60d2a0 0x55c67d564dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a8a778700 1 -- 172.21.15.121:0/3360934295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67d563470 msgr2=0x55c67d5652c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:34.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a8a778700 1 --2- 172.21.15.121:0/3360934295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67d563470 0x55c67d5652c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a8a778700 1 -- 172.21.15.121:0/3360934295 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c005db0 con 0x55c67d537860 2023-11-15T03:04:34.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a8a778700 1 --2- 172.21.15.121:0/3360934295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c67d537860 0x55c67d607de0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7f2a7c010000 tx=0x7f2a7c016000).ready entity=mon.0 client_cookie=8d69dd710ced3243 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:34.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a89776700 1 -- 172.21.15.121:0/3360934295 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c0054f0 con 0x55c67d537860 2023-11-15T03:04:34.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a89776700 1 -- 172.21.15.121:0/3360934295 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c000bc0 con 0x55c67d537860 2023-11-15T03:04:34.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.267+0000 7f2a89776700 1 -- 172.21.15.121:0/3360934295 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c008970 con 0x55c67d537860 2023-11-15T03:04:34.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.271+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3360934295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c67d537860 msgr2=0x55c67d607de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.271+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3360934295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c67d537860 0x55c67d607de0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7f2a7c010000 tx=0x7f2a7c016000).stop 2023-11-15T03:04:34.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.271+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3360934295 shutdown_connections 2023-11-15T03:04:34.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.271+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3360934295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67d563470 0x55c67d5652c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.271+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3360934295 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c67d60d2a0 0x55c67d564dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.271+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3360934295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c67d537860 0x55c67d607de0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.271+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3360934295 >> 172.21.15.121:0/3360934295 conn(0x55c67d5357d0 msgr2=0x55c67d608b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:34.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.271+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3360934295 shutdown_connections 2023-11-15T03:04:34.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.271+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3360934295 wait complete. 2023-11-15T03:04:34.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.271+0000 7f2a8ced8d00 1 Processor -- start 2023-11-15T03:04:34.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.271+0000 7f2a8ced8d00 1 -- start start 2023-11-15T03:04:34.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.271+0000 7f2a8ced8d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c67d537860 0x55c67d608b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.271+0000 7f2a8ced8d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67d563470 0x55c67d6047c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.271+0000 7f2a8ced8d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c67d60d2a0 0x55c67d5604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.271+0000 7f2a8ced8d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c67d609110 con 0x55c67d537860 2023-11-15T03:04:34.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.271+0000 7f2a8ced8d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c67d57c250 con 0x55c67d563470 2023-11-15T03:04:34.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.271+0000 7f2a8ced8d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c67d57c530 con 0x55c67d60d2a0 2023-11-15T03:04:34.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.275+0000 7f2a8a778700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c67d537860 0x55c67d608b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.275+0000 7f2a8a778700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c67d537860 0x55c67d608b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48028/0 (socket says 172.21.15.121:48028) 2023-11-15T03:04:34.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.275+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 learned_addr learned my addr 172.21.15.121:0/3458580343 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:34.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.275+0000 7f2a81f77700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67d563470 0x55c67d6047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.275+0000 7f2a8af79700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c67d60d2a0 0x55c67d5604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.275+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c67d60d2a0 msgr2=0x55c67d5604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.275+0000 7f2a8a778700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c67d60d2a0 0x55c67d5604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.275+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67d563470 msgr2=0x55c67d6047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.275+0000 7f2a8a778700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67d563470 0x55c67d6047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.275+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c67d4db7f0 con 0x55c67d537860 2023-11-15T03:04:34.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.275+0000 7f2a8a778700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c67d537860 0x55c67d608b50 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 rx=0x7f2a7c005c00 tx=0x7f2a7c01c000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:34.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.275+0000 7f2a837fe700 1 -- 172.21.15.121:0/3458580343 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c008bc0 con 0x55c67d537860 2023-11-15T03:04:34.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.275+0000 7f2a837fe700 1 -- 172.21.15.121:0/3458580343 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c013860 con 0x55c67d537860 2023-11-15T03:04:34.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.275+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c67d4d6a20 con 0x55c67d537860 2023-11-15T03:04:34.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.275+0000 7f2a837fe700 1 -- 172.21.15.121:0/3458580343 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c013c20 con 0x55c67d537860 2023-11-15T03:04:34.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.275+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c67d4d6770 con 0x55c67d537860 2023-11-15T03:04:34.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.279+0000 7f2a837fe700 1 -- 172.21.15.121:0/3458580343 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2a7c01d620 con 0x55c67d537860 2023-11-15T03:04:34.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.279+0000 7f2a837fe700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2a6c058ff0 0x7f2a6c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.279+0000 7f2a837fe700 1 -- 172.21.15.121:0/3458580343 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6782+0+0 (secure 0 0 0) 0x7f2a7c0766d0 con 0x55c67d537860 2023-11-15T03:04:34.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.279+0000 7f2a81f77700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2a6c058ff0 0x7f2a6c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.297 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:04:34.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.279+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c67d604cb0 0x55c67d609f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.279+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d561530 con 0x55c67d604cb0 2023-11-15T03:04:34.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.279+0000 7f2a8af79700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c67d604cb0 0x55c67d609f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.279+0000 7f2a81f77700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2a6c058ff0 0x7f2a6c05b4b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x55c67d609040 tx=0x7f2a7400c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:34.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.283+0000 7f2a8af79700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c67d604cb0 0x55c67d609f40 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:34.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.283+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v100'65 uv58 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d604cb0 2023-11-15T03:04:34.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.283+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d60a430 con 0x55c67d604cb0 2023-11-15T03:04:34.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.287+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v100'66 uv66 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d604cb0 2023-11-15T03:04:34.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.287+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d60a810 con 0x55c67d604cb0 2023-11-15T03:04:34.301 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:04:34.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.287+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v100'67 uv67 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d604cb0 2023-11-15T03:04:34.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.287+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62b8f0 con 0x55c67d604cb0 2023-11-15T03:04:34.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.291+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v100'68 uv68 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d604cb0 2023-11-15T03:04:34.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.291+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62bca0 con 0x55c67d604cb0 2023-11-15T03:04:34.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.295+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v100'69 uv69 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d604cb0 2023-11-15T03:04:34.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.295+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62a8e0 con 0x55c67d604cb0 2023-11-15T03:04:34.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.295+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v100'70 uv70 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d604cb0 2023-11-15T03:04:34.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.295+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62ad10 con 0x55c67d604cb0 2023-11-15T03:04:34.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.299+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v100'71 uv64 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d604cb0 2023-11-15T03:04:34.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.299+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d62b110 con 0x55c67d604cb0 2023-11-15T03:04:34.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.299+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v100'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d604cb0 2023-11-15T03:04:34.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.299+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62b110 con 0x55c67d604cb0 2023-11-15T03:04:34.310 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:04:34.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.303+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v100'73 uv73 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d604cb0 2023-11-15T03:04:34.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.303+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62b510 con 0x55c67d604cb0 2023-11-15T03:04:34.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.303+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v100'74 uv74 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d604cb0 2023-11-15T03:04:34.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.303+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62b0c0 con 0x55c67d604cb0 2023-11-15T03:04:34.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.307+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v100'75 uv75 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d604cb0 2023-11-15T03:04:34.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.307+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62b0c0 con 0x55c67d604cb0 2023-11-15T03:04:34.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.311+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v100'76 uv76 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d604cb0 2023-11-15T03:04:34.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.311+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c67d62b4c0 0x55c67d62e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.311+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62b0c0 con 0x55c67d62b4c0 2023-11-15T03:04:34.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.311+0000 7f2a8a778700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c67d62b4c0 0x55c67d62e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.311+0000 7f2a8a778700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c67d62b4c0 0x55c67d62e570 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:34.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.315+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v100'100 uv87 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.315+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d62ea60 con 0x55c67d62b4c0 2023-11-15T03:04:34.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.319+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v100'101 uv101 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.319+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62ee10 con 0x55c67d62b4c0 2023-11-15T03:04:34.330 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:04:34.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.323+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v100'102 uv102 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.323+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6301d0 con 0x55c67d62b4c0 2023-11-15T03:04:34.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.323+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v100'103 uv103 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.323+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d630580 con 0x55c67d62b4c0 2023-11-15T03:04:34.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.327+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v100'104 uv104 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.327+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62f1c0 con 0x55c67d62b4c0 2023-11-15T03:04:34.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.331+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v100'105 uv105 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.331+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62fab0 con 0x55c67d62b4c0 2023-11-15T03:04:34.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.331+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v100'106 uv106 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.331+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62fab0 con 0x55c67d62b4c0 2023-11-15T03:04:34.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.335+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v100'107 uv93 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.335+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d62fab0 con 0x55c67d62b4c0 2023-11-15T03:04:34.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.339+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v100'108 uv108 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.339+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62fab0 con 0x55c67d62b4c0 2023-11-15T03:04:34.348 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:04:34.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.339+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v100'109 uv109 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.339+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62fab0 con 0x55c67d62b4c0 2023-11-15T03:04:34.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.343+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v100'110 uv110 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.343+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62fab0 con 0x55c67d62b4c0 2023-11-15T03:04:34.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.347+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v100'111 uv111 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.347+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62fab0 con 0x55c67d62b4c0 2023-11-15T03:04:34.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.347+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v100'112 uv112 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.351+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62fab0 con 0x55c67d62b4c0 2023-11-15T03:04:34.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.351+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v100'113 uv99 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.351+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d62fab0 con 0x55c67d62b4c0 2023-11-15T03:04:34.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.355+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v100'114 uv114 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.355+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62fab0 con 0x55c67d62b4c0 2023-11-15T03:04:34.367 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:04:34.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.355+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v100'115 uv115 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.355+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62fab0 con 0x55c67d62b4c0 2023-11-15T03:04:34.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.359+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v100'116 uv116 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.359+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62fab0 con 0x55c67d62b4c0 2023-11-15T03:04:34.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.363+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v100'117 uv117 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.363+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62fab0 con 0x55c67d62b4c0 2023-11-15T03:04:34.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.363+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v100'118 uv118 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.367+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c67d630e40 0x55c67d633230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.367+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d62fab0 con 0x55c67d630e40 2023-11-15T03:04:34.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.367+0000 7f2a81f77700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c67d630e40 0x55c67d633230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.367+0000 7f2a81f77700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c67d630e40 0x55c67d633230 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:34.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.371+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v100'55 uv51 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d630e40 2023-11-15T03:04:34.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.371+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d633720 con 0x55c67d630e40 2023-11-15T03:04:34.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.375+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v100'56 uv56 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d630e40 2023-11-15T03:04:34.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.375+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d633ad0 con 0x55c67d630e40 2023-11-15T03:04:34.385 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:04:34.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.379+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v100'57 uv57 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d630e40 2023-11-15T03:04:34.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.379+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d634370 con 0x55c67d630e40 2023-11-15T03:04:34.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.379+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v100'58 uv58 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d630e40 2023-11-15T03:04:34.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.379+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d633f00 con 0x55c67d630e40 2023-11-15T03:04:34.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.383+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v100'59 uv59 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d630e40 2023-11-15T03:04:34.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.383+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d634c90 con 0x55c67d630e40 2023-11-15T03:04:34.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.387+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v100'60 uv54 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d630e40 2023-11-15T03:04:34.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.387+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d635040 con 0x55c67d630e40 2023-11-15T03:04:34.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.391+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v100'61 uv61 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d630e40 2023-11-15T03:04:34.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.391+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6353f0 con 0x55c67d630e40 2023-11-15T03:04:34.402 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:04:34.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.395+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v100'62 uv62 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d630e40 2023-11-15T03:04:34.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.395+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6353f0 con 0x55c67d630e40 2023-11-15T03:04:34.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.395+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v100'63 uv63 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d630e40 2023-11-15T03:04:34.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.395+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d634820 con 0x55c67d630e40 2023-11-15T03:04:34.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.399+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v100'64 uv64 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d630e40 2023-11-15T03:04:34.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.399+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c67d634820 0x55c67d637e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.399+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6383c0 con 0x55c67d634820 2023-11-15T03:04:34.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.399+0000 7f2a8af79700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c67d634820 0x55c67d637e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.399+0000 7f2a8af79700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c67d634820 0x55c67d637e00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:34.411 INFO:teuthology.orchestra.run.smithi121.stdout:Write #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:04:34.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.403+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v100'74 uv74 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.403+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63c570 con 0x55c67d634820 2023-11-15T03:04:34.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.407+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [write 0~16484] v100'75 uv75 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.407+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63c9a0 con 0x55c67d634820 2023-11-15T03:04:34.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.411+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [omap-set-header] v100'76 uv76 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.411+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6383c0 con 0x55c67d634820 2023-11-15T03:04:34.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.411+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 606.00000000 [create] v100'77 uv72 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.411+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d6383c0 con 0x55c67d634820 2023-11-15T03:04:34.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.415+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 606.00000000 [delete] v100'78 uv78 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.415+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6383c0 con 0x55c67d634820 2023-11-15T03:04:34.427 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:04:34.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.419+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v100'79 uv79 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.419+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6393d0 con 0x55c67d634820 2023-11-15T03:04:34.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.423+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v100'80 uv80 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.423+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6393d0 con 0x55c67d634820 2023-11-15T03:04:34.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.423+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v100'81 uv81 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.423+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6383c0 con 0x55c67d634820 2023-11-15T03:04:34.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.427+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [omap-set-header] v100'82 uv82 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.427+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c67d6388d0 0x55c67d63ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.427+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:51 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63b2d0 con 0x55c67d6388d0 2023-11-15T03:04:34.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.427+0000 7f2a8a778700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c67d6388d0 0x55c67d63ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.427+0000 7f2a8a778700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c67d6388d0 0x55c67d63ad00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:34.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.431+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(51 607.00000000 [create] v100'33 uv32 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d6388d0 2023-11-15T03:04:34.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.431+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:52 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d63b6b0 con 0x55c67d6388d0 2023-11-15T03:04:34.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.435+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(52 607.00000000 [delete] v100'34 uv34 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d6388d0 2023-11-15T03:04:34.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.435+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63b6b0 con 0x55c67d6388d0 2023-11-15T03:04:34.446 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:04:34.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.439+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(53 607.00000000 [create] v100'35 uv35 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d6388d0 2023-11-15T03:04:34.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.439+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63b6b0 con 0x55c67d6388d0 2023-11-15T03:04:34.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.439+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v100'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d6388d0 2023-11-15T03:04:34.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.439+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63b6b0 con 0x55c67d6388d0 2023-11-15T03:04:34.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.443+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v100'37 uv37 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d6388d0 2023-11-15T03:04:34.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.443+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63b6b0 con 0x55c67d6388d0 2023-11-15T03:04:34.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.447+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v100'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d6388d0 2023-11-15T03:04:34.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.447+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c67d63bff0 0x55c67d63e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.447+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:57 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63eea0 con 0x55c67d63bff0 2023-11-15T03:04:34.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.451+0000 7f2a81f77700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c67d63bff0 0x55c67d63e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.451+0000 7f2a81f77700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c67d63bff0 0x55c67d63e9b0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:34.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.455+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v100'24 uv23 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.455+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:58 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d63eea0 con 0x55c67d63bff0 2023-11-15T03:04:34.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.455+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v100'25 uv25 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.455+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63eea0 con 0x55c67d63bff0 2023-11-15T03:04:34.466 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:04:34.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.459+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v100'26 uv26 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.459+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d63bff0 2023-11-15T03:04:34.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.463+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v100'27 uv27 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.463+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d63bff0 2023-11-15T03:04:34.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.467+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(61 608.00000000 [create] v100'33 uv32 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.467+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d63bff0 2023-11-15T03:04:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.467+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(62 608.00000000 [delete] v100'34 uv34 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.467+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d63bff0 2023-11-15T03:04:34.479 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:04:34.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.471+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(63 608.00000000 [create] v100'35 uv35 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.471+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d640300 con 0x55c67d63bff0 2023-11-15T03:04:34.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.475+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v100'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.475+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d640300 con 0x55c67d63bff0 2023-11-15T03:04:34.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.475+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v100'37 uv37 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.475+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d63bff0 2023-11-15T03:04:34.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.479+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(66 608.00000000 [omap-set-header] v100'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.479+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d62b4c0 2023-11-15T03:04:34.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.483+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(67 604.00000000 [create] v100'59 uv53 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.483+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d62b4c0 2023-11-15T03:04:34.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.487+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(68 604.00000000 [delete] v100'60 uv60 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.487+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d62b4c0 2023-11-15T03:04:34.498 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:04:34.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.491+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(69 604.00000000 [create] v100'61 uv61 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.491+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d640300 con 0x55c67d62b4c0 2023-11-15T03:04:34.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.495+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v100'62 uv62 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.495+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d640300 con 0x55c67d62b4c0 2023-11-15T03:04:34.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.495+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v100'63 uv63 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.495+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d62b4c0 2023-11-15T03:04:34.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.499+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(72 604.00000000 [omap-set-header] v100'64 uv64 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.499+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d62b4c0 2023-11-15T03:04:34.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.503+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(73 500.00000000 [create] v100'65 uv58 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.503+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d62b4c0 2023-11-15T03:04:34.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.503+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(74 500.00000000 [delete] v100'66 uv66 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.503+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d62b4c0 2023-11-15T03:04:34.515 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:04:34.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.507+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(75 500.00000000 [create] v100'67 uv67 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.507+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d62b4c0 2023-11-15T03:04:34.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.511+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(76 500.00000000 [write 0~90] v100'68 uv68 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.511+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d62b4c0 2023-11-15T03:04:34.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.515+0000 7f2a8a778700 1 -- 172.21.15.121:0/3458580343 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(77 500.00000000 [omap-set-header] v100'69 uv69 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a7c023020 con 0x55c67d62b4c0 2023-11-15T03:04:34.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.515+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:78 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d630e40 2023-11-15T03:04:34.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.519+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(78 mds_snaptable [create] v100'27 uv26 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d630e40 2023-11-15T03:04:34.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.519+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:79 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d630e40 2023-11-15T03:04:34.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.523+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(79 mds_snaptable [delete] v100'28 uv28 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d630e40 2023-11-15T03:04:34.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.523+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d630e40 2023-11-15T03:04:34.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.523+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(80 mds_snaptable [create] v100'29 uv29 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d630e40 2023-11-15T03:04:34.534 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:04:34.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.523+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d630e40 2023-11-15T03:04:34.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.527+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(81 mds_snaptable [write 0~70] v100'30 uv30 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d630e40 2023-11-15T03:04:34.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.527+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d630e40 2023-11-15T03:04:34.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.531+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v100'31 uv31 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d630e40 2023-11-15T03:04:34.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.531+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d63bff0 2023-11-15T03:04:34.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.531+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(83 605.00000000 [create] v100'33 uv32 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.531+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d63bff0 2023-11-15T03:04:34.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.535+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(84 605.00000000 [delete] v100'34 uv34 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.535+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d63bff0 2023-11-15T03:04:34.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.535+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(85 605.00000000 [create] v100'35 uv35 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.550 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:04:34.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.539+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d640300 con 0x55c67d63bff0 2023-11-15T03:04:34.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.539+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v100'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.539+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d640300 con 0x55c67d63bff0 2023-11-15T03:04:34.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.543+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v100'37 uv37 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.543+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d63bff0 2023-11-15T03:04:34.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.547+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(88 605.00000000 [omap-set-header] v100'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.547+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d63bff0 2023-11-15T03:04:34.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.551+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(89 mds0_inotable [create] v100'28 uv27 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.551+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d63bff0 2023-11-15T03:04:34.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.551+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(90 mds0_inotable [delete] v100'29 uv29 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.551+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d63bff0 2023-11-15T03:04:34.564 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:04:34.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.555+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(91 mds0_inotable [create] v100'30 uv30 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.555+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d63bff0 2023-11-15T03:04:34.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.559+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(92 mds0_inotable [write 0~34] v100'31 uv31 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.559+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d63bff0 2023-11-15T03:04:34.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.563+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v100'32 uv32 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.563+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d63bff0 2023-11-15T03:04:34.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.563+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(94 100.00000000 [create] v100'27 uv26 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.563+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d63bff0 2023-11-15T03:04:34.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.567+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(95 100.00000000 [delete] v100'28 uv28 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.567+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d63bff0 2023-11-15T03:04:34.578 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:04:34.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.571+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(96 100.00000000 [create] v100'29 uv29 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.571+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d63f2f0 con 0x55c67d63bff0 2023-11-15T03:04:34.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.575+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(97 100.00000000 [omap-set-header] v100'30 uv30 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.575+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d642060 con 0x55c67d63bff0 2023-11-15T03:04:34.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.575+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v100'31 uv31 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.575+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c67d63fe60 0x55c67d640240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.575+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d63fe60 2023-11-15T03:04:34.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.579+0000 7f2a8af79700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c67d63fe60 0x55c67d640240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.579+0000 7f2a8af79700 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c67d63fe60 0x55c67d640240 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:34.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.579+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(99 mds0_sessionmap [create] v100'23 uv22 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d63fe60 2023-11-15T03:04:34.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.579+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d63fe60 2023-11-15T03:04:34.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.583+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(100 mds0_sessionmap [delete] v100'24 uv24 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d63fe60 2023-11-15T03:04:34.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.583+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d63fe60 2023-11-15T03:04:34.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.587+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(101 mds0_sessionmap [create] v100'25 uv25 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d63fe60 2023-11-15T03:04:34.595 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:04:34.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.587+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d63fe60 2023-11-15T03:04:34.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.591+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v100'26 uv26 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d63fe60 2023-11-15T03:04:34.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.591+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d63bff0 2023-11-15T03:04:34.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.591+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(103 609.00000000 [create] v100'33 uv32 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.595+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d63bff0 2023-11-15T03:04:34.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.595+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(104 609.00000000 [delete] v100'34 uv34 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.595+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d63bff0 2023-11-15T03:04:34.606 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:04:34.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.599+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(105 609.00000000 [create] v100'35 uv35 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.599+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d63bff0 2023-11-15T03:04:34.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.603+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v100'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.603+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d63bff0 2023-11-15T03:04:34.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.603+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v100'37 uv37 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.603+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d63bff0 2023-11-15T03:04:34.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.607+0000 7f2a81f77700 1 -- 172.21.15.121:0/3458580343 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(108 609.00000000 [omap-set-header] v100'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a74007cb0 con 0x55c67d63bff0 2023-11-15T03:04:34.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.607+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d634820 2023-11-15T03:04:34.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.611+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(109 400.00000000 [create] v100'27 uv26 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.611+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d634820 2023-11-15T03:04:34.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.611+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(110 400.00000000 [delete] v100'28 uv28 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.611+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d634820 2023-11-15T03:04:34.622 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:04:34.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.615+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v100'29 uv29 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.615+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d634820 2023-11-15T03:04:34.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.619+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [write 0~22] v100'30 uv30 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.619+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d634820 2023-11-15T03:04:34.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.619+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [omap-set-header] v100'31 uv31 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.619+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d634820 2023-11-15T03:04:34.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.623+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 100.00000000.inode [create] v100'66 uv58 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.623+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d634820 2023-11-15T03:04:34.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.623+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 100.00000000.inode [delete] v100'67 uv67 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.627+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d634820 2023-11-15T03:04:34.635 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:04:34.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.627+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v100'68 uv68 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.627+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d634820 2023-11-15T03:04:34.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.631+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v100'69 uv69 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.631+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d640bf0 con 0x55c67d634820 2023-11-15T03:04:34.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.635+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v100'70 uv70 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.635+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d640bf0 con 0x55c67d634820 2023-11-15T03:04:34.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.635+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 1.00000000 [create] v100'71 uv65 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.635+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d634820 2023-11-15T03:04:34.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.639+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 1.00000000 [delete] v100'72 uv72 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.639+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d634820 2023-11-15T03:04:34.649 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:04:34.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.639+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v100'73 uv73 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.643+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d634820 2023-11-15T03:04:34.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.643+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v100'74 uv74 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.643+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d634820 2023-11-15T03:04:34.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.647+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v100'75 uv75 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.647+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d6407a0 con 0x55c67d634820 2023-11-15T03:04:34.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.647+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [omap-set-header] v100'76 uv76 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.647+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e100) v8 -- 0x55c67d640ba0 con 0x55c67d634820 2023-11-15T03:04:34.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v100'77 uv77 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2a8400dc00 con 0x55c67d634820 2023-11-15T03:04:34.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c67d630e40 msgr2=0x55c67d633230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c67d630e40 0x55c67d633230 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c67d604cb0 msgr2=0x55c67d609f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c67d604cb0 0x55c67d609f40 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c67d6388d0 msgr2=0x55c67d63ad00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c67d6388d0 0x55c67d63ad00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c67d62b4c0 msgr2=0x55c67d62e570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c67d62b4c0 0x55c67d62e570 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c67d63fe60 msgr2=0x55c67d640240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c67d63fe60 0x55c67d640240 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c67d634820 msgr2=0x55c67d637e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c67d634820 0x55c67d637e00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c67d63bff0 msgr2=0x55c67d63e9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c67d63bff0 0x55c67d63e9b0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 reap_dead start 2023-11-15T03:04:34.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 reap_dead start 2023-11-15T03:04:34.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 reap_dead start 2023-11-15T03:04:34.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8af79700 1 -- 172.21.15.121:0/3458580343 reap_dead start 2023-11-15T03:04:34.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2a6c058ff0 msgr2=0x7f2a6c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2a6c058ff0 0x7f2a6c05b4b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x55c67d609040 tx=0x7f2a7400c040).stop 2023-11-15T03:04:34.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c67d537860 msgr2=0x55c67d608b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c67d537860 0x55c67d608b50 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 rx=0x7f2a7c005c00 tx=0x7f2a7c01c000).stop 2023-11-15T03:04:34.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 shutdown_connections 2023-11-15T03:04:34.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2a6c058ff0 0x7f2a6c05b4b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 --2- 172.21.15.121:0/3458580343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c67d537860 0x55c67d608b50 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 >> 172.21.15.121:0/3458580343 conn(0x55c67d5357d0 msgr2=0x55c67d561a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:34.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 shutdown_connections 2023-11-15T03:04:34.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.651+0000 7f2a8ced8d00 1 -- 172.21.15.121:0/3458580343 wait complete. 2023-11-15T03:04:34.667 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_openfiles.0 2023-11-15T03:04:34.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.759+0000 7f6b4d932d00 1 Processor -- start 2023-11-15T03:04:34.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.759+0000 7f6b4d932d00 1 -- start start 2023-11-15T03:04:34.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b4d932d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dd1f7ff7e0 0x55dd1f844d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b4d932d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd1f8bd650 0x55dd1f845210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b4d932d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55dd1f8bda30 0x55dd1f8bb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b4d932d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1f774220 con 0x55dd1f7ff7e0 2023-11-15T03:04:34.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b4d932d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1f845700 con 0x55dd1f8bd650 2023-11-15T03:04:34.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b4d932d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd1f7fbd90 con 0x55dd1f8bda30 2023-11-15T03:04:34.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b41eb7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dd1f7ff7e0 0x55dd1f844d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b41eb7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dd1f7ff7e0 0x55dd1f844d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42440/0 (socket says 172.21.15.121:42440) 2023-11-15T03:04:34.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b41eb7700 1 -- 172.21.15.121:0/2235495069 learned_addr learned my addr 172.21.15.121:0/2235495069 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:34.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b396b6700 1 --2- 172.21.15.121:0/2235495069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd1f8bd650 0x55dd1f845210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b426b8700 1 --2- 172.21.15.121:0/2235495069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55dd1f8bda30 0x55dd1f8bb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b41eb7700 1 -- 172.21.15.121:0/2235495069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55dd1f8bda30 msgr2=0x55dd1f8bb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:34.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b41eb7700 1 --2- 172.21.15.121:0/2235495069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55dd1f8bda30 0x55dd1f8bb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b41eb7700 1 -- 172.21.15.121:0/2235495069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd1f8bd650 msgr2=0x55dd1f845210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:34.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b41eb7700 1 --2- 172.21.15.121:0/2235495069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd1f8bd650 0x55dd1f845210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b41eb7700 1 -- 172.21.15.121:0/2235495069 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b34005db0 con 0x55dd1f7ff7e0 2023-11-15T03:04:34.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b41eb7700 1 --2- 172.21.15.121:0/2235495069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dd1f7ff7e0 0x55dd1f844d20 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x55dd1f7febf0 tx=0x7f6b34016000).ready entity=mon.0 client_cookie=2f0632264374a484 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:34.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b40eb5700 1 -- 172.21.15.121:0/2235495069 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b34005330 con 0x55dd1f7ff7e0 2023-11-15T03:04:34.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b40eb5700 1 -- 172.21.15.121:0/2235495069 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b34000bc0 con 0x55dd1f7ff7e0 2023-11-15T03:04:34.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b40eb5700 1 -- 172.21.15.121:0/2235495069 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b34008a50 con 0x55dd1f7ff7e0 2023-11-15T03:04:34.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b4d932d00 1 -- 172.21.15.121:0/2235495069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dd1f7ff7e0 msgr2=0x55dd1f844d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b4d932d00 1 --2- 172.21.15.121:0/2235495069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dd1f7ff7e0 0x55dd1f844d20 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x55dd1f7febf0 tx=0x7f6b34016000).stop 2023-11-15T03:04:34.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b4d932d00 1 -- 172.21.15.121:0/2235495069 shutdown_connections 2023-11-15T03:04:34.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b4d932d00 1 --2- 172.21.15.121:0/2235495069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55dd1f8bda30 0x55dd1f8bb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b4d932d00 1 --2- 172.21.15.121:0/2235495069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd1f8bd650 0x55dd1f845210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b4d932d00 1 --2- 172.21.15.121:0/2235495069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dd1f7ff7e0 0x55dd1f844d20 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b4d932d00 1 -- 172.21.15.121:0/2235495069 >> 172.21.15.121:0/2235495069 conn(0x55dd1f8458f0 msgr2=0x55dd1f800af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:34.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b4d932d00 1 -- 172.21.15.121:0/2235495069 shutdown_connections 2023-11-15T03:04:34.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.763+0000 7f6b4d932d00 1 -- 172.21.15.121:0/2235495069 wait complete. 2023-11-15T03:04:34.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b4d932d00 1 Processor -- start 2023-11-15T03:04:34.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b4d932d00 1 -- start start 2023-11-15T03:04:34.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b4d932d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd1f7ff7e0 0x55dd1f83e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b4d932d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55dd1f8bd650 0x55dd1f83ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b4d932d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dd1f8bda30 0x55dd1f82cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b4d932d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1f83ef20 con 0x55dd1f8bda30 2023-11-15T03:04:34.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b4d932d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1f8d6c10 con 0x55dd1f7ff7e0 2023-11-15T03:04:34.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b4d932d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd1f8d6ef0 con 0x55dd1f8bd650 2023-11-15T03:04:34.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b426b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dd1f8bda30 0x55dd1f82cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b426b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dd1f8bda30 0x55dd1f82cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42446/0 (socket says 172.21.15.121:42446) 2023-11-15T03:04:34.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b426b8700 1 -- 172.21.15.121:0/2933338954 learned_addr learned my addr 172.21.15.121:0/2933338954 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:34.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b41eb7700 1 --2- 172.21.15.121:0/2933338954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd1f7ff7e0 0x55dd1f83e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b396b6700 1 --2- 172.21.15.121:0/2933338954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55dd1f8bd650 0x55dd1f83ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b426b8700 1 -- 172.21.15.121:0/2933338954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55dd1f8bd650 msgr2=0x55dd1f83ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b426b8700 1 --2- 172.21.15.121:0/2933338954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55dd1f8bd650 0x55dd1f83ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b426b8700 1 -- 172.21.15.121:0/2933338954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd1f7ff7e0 msgr2=0x55dd1f83e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b426b8700 1 --2- 172.21.15.121:0/2933338954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd1f7ff7e0 0x55dd1f83e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b426b8700 1 -- 172.21.15.121:0/2933338954 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b34017040 con 0x55dd1f8bda30 2023-11-15T03:04:34.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b41eb7700 1 --2- 172.21.15.121:0/2933338954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd1f7ff7e0 0x55dd1f83e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:04:34.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b426b8700 1 --2- 172.21.15.121:0/2933338954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dd1f8bda30 0x55dd1f82cd90 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7f6b3c00e380 tx=0x7f6b3c007040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:34.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.767+0000 7f6b3affd700 1 -- 172.21.15.121:0/2933338954 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c00eec0 con 0x55dd1f8bda30 2023-11-15T03:04:34.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.771+0000 7f6b3affd700 1 -- 172.21.15.121:0/2933338954 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3c012400 con 0x55dd1f8bda30 2023-11-15T03:04:34.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.771+0000 7f6b4d932d00 1 -- 172.21.15.121:0/2933338954 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dd1f79d770 con 0x55dd1f8bda30 2023-11-15T03:04:34.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.771+0000 7f6b3affd700 1 -- 172.21.15.121:0/2933338954 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c012780 con 0x55dd1f8bda30 2023-11-15T03:04:34.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.771+0000 7f6b4d932d00 1 -- 172.21.15.121:0/2933338954 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd1f76ca70 con 0x55dd1f8bda30 2023-11-15T03:04:34.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.771+0000 7f6b3affd700 1 -- 172.21.15.121:0/2933338954 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6b3c012af0 con 0x55dd1f8bda30 2023-11-15T03:04:34.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.775+0000 7f6b3affd700 1 --2- 172.21.15.121:0/2933338954 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6b24058ff0 0x7f6b2405b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.775+0000 7f6b3affd700 1 -- 172.21.15.121:0/2933338954 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6782+0+0 (secure 0 0 0) 0x7f6b3c0739d0 con 0x55dd1f8bda30 2023-11-15T03:04:34.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.775+0000 7f6b41eb7700 1 --2- 172.21.15.121:0/2933338954 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6b24058ff0 0x7f6b2405b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.775+0000 7f6b4d932d00 1 --2- 172.21.15.121:0/2933338954 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55dd1f82c690 0x55dd1f82dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.775+0000 7f6b4d932d00 1 -- 172.21.15.121:0/2933338954 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e100) v8 -- 0x55dd1f8d5100 con 0x55dd1f82c690 2023-11-15T03:04:34.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.775+0000 7f6b41eb7700 1 --2- 172.21.15.121:0/2933338954 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6b24058ff0 0x7f6b2405b4b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x55dd1f7febf0 tx=0x7f6b3401b000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:34.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.775+0000 7f6b396b6700 1 --2- 172.21.15.121:0/2933338954 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55dd1f82c690 0x55dd1f82dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.775+0000 7f6b396b6700 1 --2- 172.21.15.121:0/2933338954 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55dd1f82c690 0x55dd1f82dc80 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:34.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.779+0000 7f6b396b6700 1 -- 172.21.15.121:0/2933338954 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 mds0_openfiles.0 [delete] v100'28 uv28 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6b2c0106e0 con 0x55dd1f82c690 2023-11-15T03:04:34.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.779+0000 7f6b4d932d00 1 -- 172.21.15.121:0/2933338954 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55dd1f82c690 msgr2=0x55dd1f82dc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.779+0000 7f6b4d932d00 1 --2- 172.21.15.121:0/2933338954 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55dd1f82c690 0x55dd1f82dc80 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.779+0000 7f6b4d932d00 1 -- 172.21.15.121:0/2933338954 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6b24058ff0 msgr2=0x7f6b2405b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.779+0000 7f6b4d932d00 1 --2- 172.21.15.121:0/2933338954 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6b24058ff0 0x7f6b2405b4b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x55dd1f7febf0 tx=0x7f6b3401b000).stop 2023-11-15T03:04:34.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.779+0000 7f6b4d932d00 1 -- 172.21.15.121:0/2933338954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dd1f8bda30 msgr2=0x55dd1f82cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.779+0000 7f6b4d932d00 1 --2- 172.21.15.121:0/2933338954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dd1f8bda30 0x55dd1f82cd90 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7f6b3c00e380 tx=0x7f6b3c007040).stop 2023-11-15T03:04:34.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.779+0000 7f6b426b8700 1 -- 172.21.15.121:0/2933338954 reap_dead start 2023-11-15T03:04:34.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.779+0000 7f6b4d932d00 1 -- 172.21.15.121:0/2933338954 shutdown_connections 2023-11-15T03:04:34.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.779+0000 7f6b4d932d00 1 -- 172.21.15.121:0/2933338954 >> 172.21.15.121:0/2933338954 conn(0x55dd1f8458f0 msgr2=0x55dd1f847b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:34.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.779+0000 7f6b4d932d00 1 -- 172.21.15.121:0/2933338954 shutdown_connections 2023-11-15T03:04:34.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.779+0000 7f6b4d932d00 1 -- 172.21.15.121:0/2933338954 wait complete. 2023-11-15T03:04:34.810 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:04:34.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.979+0000 7fa1aa09b700 1 Processor -- start 2023-11-15T03:04:34.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.979+0000 7fa1aa09b700 1 -- start start 2023-11-15T03:04:34.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.983+0000 7fa1aa09b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa1a4067ec0 0x7fa1a40682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.983+0000 7fa1aa09b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa1a4153700 0x7fa1a4153b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:34.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.983+0000 7fa1aa09b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a40c0c20 con 0x7fa1a40c10b0 2023-11-15T03:04:34.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.983+0000 7fa1aa09b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a4158730 con 0x7fa1a4067ec0 2023-11-15T03:04:34.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.983+0000 7fa1aa09b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a40c02f0 con 0x7fa1a4153700 2023-11-15T03:04:34.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.983+0000 7fa1a37fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa1a4067ec0 0x7fa1a40682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.983+0000 7fa1a37fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa1a4067ec0 0x7fa1a40682a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42462/0 (socket says 172.21.15.121:42462) 2023-11-15T03:04:34.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.983+0000 7fa1a37fe700 1 -- 172.21.15.121:0/1477728730 learned_addr learned my addr 172.21.15.121:0/1477728730 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:34.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.983+0000 7fa1a3fff700 1 --2- 172.21.15.121:0/1477728730 >> v2:172.21.15.190:3300/0 conn(0x7fa1a4153700 0x7fa1a4153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:34.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.983+0000 7fa1a37fe700 1 -- 172.21.15.121:0/1477728730 >> v2:172.21.15.190:3300/0 conn(0x7fa1a4153700 msgr2=0x7fa1a4153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:34.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.983+0000 7fa1a37fe700 1 --2- 172.21.15.121:0/1477728730 >> v2:172.21.15.190:3300/0 conn(0x7fa1a4153700 0x7fa1a4153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.983+0000 7fa1a37fe700 1 -- 172.21.15.121:0/1477728730 >> v1:172.21.15.190:6789/0 conn(0x7fa1a40c10b0 legacy=0x7fa1a4062c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.983+0000 7fa1a37fe700 1 -- 172.21.15.121:0/1477728730 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a40c05d0 con 0x7fa1a4067ec0 2023-11-15T03:04:34.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.983+0000 7fa1a37fe700 1 --2- 172.21.15.121:0/1477728730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a4067ec0 0x7fa1a40682a0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7fa1a405fa90 tx=0x7fa1940090f0).ready entity=mon.0 client_cookie=99e1b164f9065f95 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:34.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.983+0000 7fa1a27fc700 1 -- 172.21.15.121:0/1477728730 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19400b600 con 0x7fa1a4067ec0 2023-11-15T03:04:34.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.983+0000 7fa1a27fc700 1 -- 172.21.15.121:0/1477728730 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19400b760 con 0x7fa1a4067ec0 2023-11-15T03:04:34.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.983+0000 7fa1a27fc700 1 -- 172.21.15.121:0/1477728730 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19400bae0 con 0x7fa1a4067ec0 2023-11-15T03:04:34.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.987+0000 7fa1aa09b700 1 -- 172.21.15.121:0/1477728730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a4067ec0 msgr2=0x7fa1a40682a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:34.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.987+0000 7fa1aa09b700 1 --2- 172.21.15.121:0/1477728730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a4067ec0 0x7fa1a40682a0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7fa1a405fa90 tx=0x7fa1940090f0).stop 2023-11-15T03:04:34.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.987+0000 7fa1aa09b700 1 -- 172.21.15.121:0/1477728730 shutdown_connections 2023-11-15T03:04:34.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.987+0000 7fa1aa09b700 1 --2- 172.21.15.121:0/1477728730 >> v2:172.21.15.190:3300/0 conn(0x7fa1a4153700 0x7fa1a4153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.987+0000 7fa1aa09b700 1 --2- 172.21.15.121:0/1477728730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a4067ec0 0x7fa1a40682a0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:34.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.987+0000 7fa1aa09b700 1 -- 172.21.15.121:0/1477728730 >> 172.21.15.121:0/1477728730 conn(0x7fa1a40cad40 msgr2=0x7fa1a40cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:34.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.987+0000 7fa1aa09b700 1 -- 172.21.15.121:0/1477728730 shutdown_connections 2023-11-15T03:04:34.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.987+0000 7fa1aa09b700 1 -- 172.21.15.121:0/1477728730 wait complete. 2023-11-15T03:04:34.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.987+0000 7fa1aa09b700 1 Processor -- start 2023-11-15T03:04:34.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.987+0000 7fa1aa09b700 1 -- start start 2023-11-15T03:04:35.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa1aa09b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1a4067ec0 0x7fa1a414ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:35.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa1aa09b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1a40c10b0 0x7fa1a4153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:35.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa1aa09b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a4153700 0x7fa1a41652f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:35.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa1aa09b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40ad5e0 con 0x7fa1a4153700 2023-11-15T03:04:35.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa1aa09b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a4164250 con 0x7fa1a40c10b0 2023-11-15T03:04:35.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa1aa09b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a4154870 con 0x7fa1a4067ec0 2023-11-15T03:04:35.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa1a3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a4153700 0x7fa1a41652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:35.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa1a3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a4153700 0x7fa1a41652f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42476/0 (socket says 172.21.15.121:42476) 2023-11-15T03:04:35.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa1a3fff700 1 -- 172.21.15.121:0/4040901291 learned_addr learned my addr 172.21.15.121:0/4040901291 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:35.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa1a37fe700 1 --2- 172.21.15.121:0/4040901291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1a4067ec0 0x7fa1a414ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:35.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa1a2ffd700 1 --2- 172.21.15.121:0/4040901291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1a40c10b0 0x7fa1a4153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:35.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa1a3fff700 1 -- 172.21.15.121:0/4040901291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1a4067ec0 msgr2=0x7fa1a414ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:35.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa1a3fff700 1 --2- 172.21.15.121:0/4040901291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1a4067ec0 0x7fa1a414ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:35.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa1a3fff700 1 -- 172.21.15.121:0/4040901291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1a40c10b0 msgr2=0x7fa1a4153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:35.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa1a3fff700 1 --2- 172.21.15.121:0/4040901291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1a40c10b0 0x7fa1a4153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:35.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa1a3fff700 1 -- 172.21.15.121:0/4040901291 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa194009370 con 0x7fa1a4153700 2023-11-15T03:04:35.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa1a3fff700 1 --2- 172.21.15.121:0/4040901291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a4153700 0x7fa1a41652f0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7fa19c007f20 tx=0x7fa19c009860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:35.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa18b7fe700 1 -- 172.21.15.121:0/4040901291 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c00eb90 con 0x7fa1a4153700 2023-11-15T03:04:35.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa18b7fe700 1 -- 172.21.15.121:0/4040901291 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19c00ecf0 con 0x7fa1a4153700 2023-11-15T03:04:35.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa1aa09b700 1 -- 172.21.15.121:0/4040901291 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a4154b50 con 0x7fa1a4153700 2023-11-15T03:04:35.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa18b7fe700 1 -- 172.21.15.121:0/4040901291 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c014610 con 0x7fa1a4153700 2023-11-15T03:04:35.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.991+0000 7fa1aa09b700 1 -- 172.21.15.121:0/4040901291 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a4154060 con 0x7fa1a4153700 2023-11-15T03:04:35.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.995+0000 7fa1aa09b700 1 -- 172.21.15.121:0/4040901291 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a4047060 con 0x7fa1a4153700 2023-11-15T03:04:35.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.995+0000 7fa18b7fe700 1 -- 172.21.15.121:0/4040901291 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa19c014980 con 0x7fa1a4153700 2023-11-15T03:04:35.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.995+0000 7fa18b7fe700 1 --2- 172.21.15.121:0/4040901291 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa19005a510 0x7fa19005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:35.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.995+0000 7fa18b7fe700 1 -- 172.21.15.121:0/4040901291 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6782+0+0 (secure 0 0 0) 0x7fa19c074e00 con 0x7fa1a4153700 2023-11-15T03:04:35.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.995+0000 7fa1a37fe700 1 --2- 172.21.15.121:0/4040901291 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa19005a510 0x7fa19005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:35.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:34.999+0000 7fa1a37fe700 1 --2- 172.21.15.121:0/4040901291 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa19005a510 0x7fa19005c9d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fa194009fd0 tx=0x7fa194017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:35.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:35.003+0000 7fa18b7fe700 1 -- 172.21.15.121:0/4040901291 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa19c048350 con 0x7fa1a4153700 2023-11-15T03:04:35.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:35.195+0000 7fa1aa09b700 1 -- 172.21.15.121:0/4040901291 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fa1a40adc40 con 0x7fa1a4153700 2023-11-15T03:04:36.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.183+0000 7fa18b7fe700 1 -- 172.21.15.121:0/4040901291 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v119) v1 ==== 162+0+0 (secure 0 0 0) 0x7fa19c00ee60 con 0x7fa1a4153700 2023-11-15T03:04:36.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.187+0000 7fa1aa09b700 1 -- 172.21.15.121:0/4040901291 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa19005a510 msgr2=0x7fa19005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:36.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.187+0000 7fa1aa09b700 1 --2- 172.21.15.121:0/4040901291 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa19005a510 0x7fa19005c9d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fa194009fd0 tx=0x7fa194017000).stop 2023-11-15T03:04:36.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.187+0000 7fa1aa09b700 1 -- 172.21.15.121:0/4040901291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a4153700 msgr2=0x7fa1a41652f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:36.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.187+0000 7fa1aa09b700 1 --2- 172.21.15.121:0/4040901291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a4153700 0x7fa1a41652f0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7fa19c007f20 tx=0x7fa19c009860).stop 2023-11-15T03:04:36.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.187+0000 7fa1aa09b700 1 -- 172.21.15.121:0/4040901291 shutdown_connections 2023-11-15T03:04:36.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.187+0000 7fa1aa09b700 1 --2- 172.21.15.121:0/4040901291 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa19005a510 0x7fa19005c9d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:36.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.187+0000 7fa1aa09b700 1 --2- 172.21.15.121:0/4040901291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a4153700 0x7fa1a41652f0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:36.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.187+0000 7fa1aa09b700 1 --2- 172.21.15.121:0/4040901291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1a40c10b0 0x7fa1a4153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:36.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.187+0000 7fa1aa09b700 1 --2- 172.21.15.121:0/4040901291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1a4067ec0 0x7fa1a414ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:36.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.187+0000 7fa1aa09b700 1 -- 172.21.15.121:0/4040901291 >> 172.21.15.121:0/4040901291 conn(0x7fa1a40cad40 msgr2=0x7fa1a40b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:36.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.187+0000 7fa1aa09b700 1 -- 172.21.15.121:0/4040901291 shutdown_connections 2023-11-15T03:04:36.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.187+0000 7fa1aa09b700 1 -- 172.21.15.121:0/4040901291 wait complete. 2023-11-15T03:04:36.210 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:04:36.218 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:04:36.195+0000 7febfb572700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-11-15T03:04:36.626 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:04:36.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.791+0000 7f4ae4df9700 1 Processor -- start 2023-11-15T03:04:36.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.791+0000 7f4ae4df9700 1 -- start start 2023-11-15T03:04:36.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ae4df9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4ae0066d70 0x7f4ae00a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:36.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ae4df9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ae00a9350 con 0x7f4ae00c29a0 2023-11-15T03:04:36.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ae4df9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ae0151420 con 0x7f4ae0067150 2023-11-15T03:04:36.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ae4df9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae0150ed0 con 0x7f4ae0066d70 2023-11-15T03:04:36.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4adffff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4ae00c29a0 0x7f4ae014e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:55606/0 (socket says 172.21.15.121:55606) 2023-11-15T03:04:36.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4adffff700 1 -- 172.21.15.121:0/3503829087 learned_addr learned my addr 172.21.15.121:0/3503829087 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:36.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4adf7fe700 1 --2- 172.21.15.121:0/3503829087 >> v2:172.21.15.190:3300/0 conn(0x7f4ae0066d70 0x7f4ae00a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:36.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ade7fc700 1 -- 172.21.15.121:0/3503829087 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615662556 0 0) 0x7f4ae00a9350 con 0x7f4ae00c29a0 2023-11-15T03:04:36.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ade7fc700 1 -- 172.21.15.121:0/3503829087 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ac80033f0 con 0x7f4ae00c29a0 2023-11-15T03:04:36.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ade7fc700 1 -- 172.21.15.121:0/3503829087 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 590294524 0 0) 0x7f4ae0151420 con 0x7f4ae0067150 2023-11-15T03:04:36.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ade7fc700 1 -- 172.21.15.121:0/3503829087 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ae00a9350 con 0x7f4ae0067150 2023-11-15T03:04:36.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ade7fc700 1 -- 172.21.15.121:0/3503829087 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3354177682 0 0) 0x7f4ac80033f0 con 0x7f4ae00c29a0 2023-11-15T03:04:36.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ade7fc700 1 -- 172.21.15.121:0/3503829087 >> v2:172.21.15.190:3300/0 conn(0x7f4ae0066d70 msgr2=0x7f4ae00a87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:36.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ade7fc700 1 --2- 172.21.15.121:0/3503829087 >> v2:172.21.15.190:3300/0 conn(0x7f4ae0066d70 0x7f4ae00a87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:36.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ade7fc700 1 -- 172.21.15.121:0/3503829087 >> v1:172.21.15.190:6789/0 conn(0x7f4ae0067150 legacy=0x7f4ae01518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:36.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ade7fc700 1 -- 172.21.15.121:0/3503829087 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae00a8d90 con 0x7f4ae00c29a0 2023-11-15T03:04:36.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ade7fc700 1 -- 172.21.15.121:0/3503829087 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f4ad80017f0 con 0x7f4ae00c29a0 2023-11-15T03:04:36.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4adf7fe700 1 --2- 172.21.15.121:0/3503829087 >> v2:172.21.15.190:3300/0 conn(0x7f4ae0066d70 0x7f4ae00a87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:04:36.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ade7fc700 1 -- 172.21.15.121:0/3503829087 >> v1:172.21.15.121:6789/0 conn(0x7f4ae00c29a0 legacy=0x7f4ae014e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:36.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ade7fc700 1 --2- 172.21.15.121:0/3503829087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac8003a70 0x7f4ac8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:36.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ade7fc700 1 --2- 172.21.15.121:0/3503829087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac8004690 0x7f4ac8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:36.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ade7fc700 1 --2- 172.21.15.121:0/3503829087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac8005000 0x7f4ac800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:36.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ade7fc700 1 -- 172.21.15.121:0/3503829087 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae0150ed0 con 0x7f4ac8004690 2023-11-15T03:04:36.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ade7fc700 1 -- 172.21.15.121:0/3503829087 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac800be80 con 0x7f4ac8003a70 2023-11-15T03:04:36.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4ade7fc700 1 -- 172.21.15.121:0/3503829087 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac800c040 con 0x7f4ac8005000 2023-11-15T03:04:36.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4adeffd700 1 --2- 172.21.15.121:0/3503829087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac8004690 0x7f4ac8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:36.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.795+0000 7f4adf7fe700 1 --2- 172.21.15.121:0/3503829087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac8003a70 0x7f4ac8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:36.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.799+0000 7f4adffff700 1 --2- 172.21.15.121:0/3503829087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac8005000 0x7f4ac800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:36.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.799+0000 7f4adeffd700 1 -- 172.21.15.121:0/3503829087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac8005000 msgr2=0x7f4ac800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:36.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.799+0000 7f4adeffd700 1 --2- 172.21.15.121:0/3503829087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac8005000 0x7f4ac800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:36.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.799+0000 7f4adeffd700 1 -- 172.21.15.121:0/3503829087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac8003a70 msgr2=0x7f4ac8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:36.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.799+0000 7f4adeffd700 1 --2- 172.21.15.121:0/3503829087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac8003a70 0x7f4ac8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:36.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.799+0000 7f4adeffd700 1 -- 172.21.15.121:0/3503829087 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac80033f0 con 0x7f4ac8004690 2023-11-15T03:04:36.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.799+0000 7f4adffff700 1 -- 172.21.15.121:0/3503829087 reap_dead start 2023-11-15T03:04:36.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.799+0000 7f4adeffd700 1 --2- 172.21.15.121:0/3503829087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac8004690 0x7f4ac8004b10 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f4ad4001860 tx=0x7f4ad4007bb0).ready entity=mon.0 client_cookie=d020aeffe302fbfa server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:36.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.799+0000 7f4ade7fc700 1 -- 172.21.15.121:0/3503829087 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4007020 con 0x7f4ac8004690 2023-11-15T03:04:36.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.799+0000 7f4ade7fc700 1 -- 172.21.15.121:0/3503829087 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad4007180 con 0x7f4ac8004690 2023-11-15T03:04:36.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.799+0000 7f4ade7fc700 1 -- 172.21.15.121:0/3503829087 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4007500 con 0x7f4ac8004690 2023-11-15T03:04:36.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.799+0000 7f4ae4df9700 1 -- 172.21.15.121:0/3503829087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac8004690 msgr2=0x7f4ac8004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:36.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.799+0000 7f4ae4df9700 1 --2- 172.21.15.121:0/3503829087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac8004690 0x7f4ac8004b10 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f4ad4001860 tx=0x7f4ad4007bb0).stop 2023-11-15T03:04:36.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.799+0000 7f4ae4df9700 1 -- 172.21.15.121:0/3503829087 shutdown_connections 2023-11-15T03:04:36.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.799+0000 7f4ae4df9700 1 --2- 172.21.15.121:0/3503829087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac8004690 0x7f4ac8004b10 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:36.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.799+0000 7f4ae4df9700 1 -- 172.21.15.121:0/3503829087 >> 172.21.15.121:0/3503829087 conn(0x7f4ae0062190 msgr2=0x7f4ae0062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:36.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.799+0000 7f4ae4df9700 1 -- 172.21.15.121:0/3503829087 shutdown_connections 2023-11-15T03:04:36.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.799+0000 7f4ae4df9700 1 -- 172.21.15.121:0/3503829087 wait complete. 2023-11-15T03:04:36.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.803+0000 7f4ae4df9700 1 Processor -- start 2023-11-15T03:04:36.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.803+0000 7f4ae4df9700 1 -- start start 2023-11-15T03:04:36.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.803+0000 7f4ae4df9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac8004690 0x7f4ae00bfc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:36.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.803+0000 7f4ae4df9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ae00c0150 0x7f4ae00a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:36.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.803+0000 7f4ae4df9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ae00a1ea0 0x7f4ae01679b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:36.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.803+0000 7f4ae4df9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae00675d0 con 0x7f4ae00c0150 2023-11-15T03:04:36.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.803+0000 7f4ae4df9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae00a8680 con 0x7f4ac8004690 2023-11-15T03:04:36.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.803+0000 7f4ae4df9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae0168720 con 0x7f4ae00a1ea0 2023-11-15T03:04:36.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.807+0000 7f4adeffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ae00c0150 0x7f4ae00a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:36.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.807+0000 7f4adeffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ae00c0150 0x7f4ae00a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42496/0 (socket says 172.21.15.121:42496) 2023-11-15T03:04:36.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.807+0000 7f4adeffd700 1 -- 172.21.15.121:0/4052957840 learned_addr learned my addr 172.21.15.121:0/4052957840 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:36.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.807+0000 7f4adf7fe700 1 --2- 172.21.15.121:0/4052957840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac8004690 0x7f4ae00bfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:36.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.807+0000 7f4adffff700 1 --2- 172.21.15.121:0/4052957840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ae00a1ea0 0x7f4ae01679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:36.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.807+0000 7f4adeffd700 1 -- 172.21.15.121:0/4052957840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ae00a1ea0 msgr2=0x7f4ae01679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:36.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.807+0000 7f4adeffd700 1 --2- 172.21.15.121:0/4052957840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ae00a1ea0 0x7f4ae01679b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:36.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.807+0000 7f4adeffd700 1 -- 172.21.15.121:0/4052957840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac8004690 msgr2=0x7f4ae00bfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:36.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.807+0000 7f4adeffd700 1 --2- 172.21.15.121:0/4052957840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac8004690 0x7f4ae00bfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:36.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.807+0000 7f4adeffd700 1 -- 172.21.15.121:0/4052957840 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae0164650 con 0x7f4ae00c0150 2023-11-15T03:04:36.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.807+0000 7f4adf7fe700 1 --2- 172.21.15.121:0/4052957840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac8004690 0x7f4ae00bfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:04:36.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.807+0000 7f4adeffd700 1 --2- 172.21.15.121:0/4052957840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ae00c0150 0x7f4ae00a19b0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f4ad4006be0 tx=0x7f4ad4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:36.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.807+0000 7f4ac77fe700 1 -- 172.21.15.121:0/4052957840 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad401d400 con 0x7f4ae00c0150 2023-11-15T03:04:36.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.807+0000 7f4ac77fe700 1 -- 172.21.15.121:0/4052957840 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad401d560 con 0x7f4ae00c0150 2023-11-15T03:04:36.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.807+0000 7f4ae4df9700 1 -- 172.21.15.121:0/4052957840 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae0164930 con 0x7f4ae00c0150 2023-11-15T03:04:36.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.807+0000 7f4ac77fe700 1 -- 172.21.15.121:0/4052957840 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad401d6d0 con 0x7f4ae00c0150 2023-11-15T03:04:36.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.807+0000 7f4ae4df9700 1 -- 172.21.15.121:0/4052957840 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae01642c0 con 0x7f4ae00c0150 2023-11-15T03:04:36.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.807+0000 7f4ae4df9700 1 -- 172.21.15.121:0/4052957840 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae00ac810 con 0x7f4ae00c0150 2023-11-15T03:04:36.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.811+0000 7f4ac77fe700 1 -- 172.21.15.121:0/4052957840 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4ad401b040 con 0x7f4ae00c0150 2023-11-15T03:04:36.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.811+0000 7f4ac77fe700 1 --2- 172.21.15.121:0/4052957840 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4acc05a540 0x7f4acc05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:36.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.811+0000 7f4ac77fe700 1 -- 172.21.15.121:0/4052957840 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6782+0+0 (secure 0 0 0) 0x7f4ad4074fb0 con 0x7f4ae00c0150 2023-11-15T03:04:36.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.811+0000 7f4adf7fe700 1 --2- 172.21.15.121:0/4052957840 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4acc05a540 0x7f4acc05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:36.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.815+0000 7f4adf7fe700 1 --2- 172.21.15.121:0/4052957840 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4acc05a540 0x7f4acc05ca00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f4ae00c0c90 tx=0x7f4ad0009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:36.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:36.819+0000 7f4ac77fe700 1 -- 172.21.15.121:0/4052957840 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4ad40a3af0 con 0x7f4ae00c0150 2023-11-15T03:04:37.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:37.015+0000 7f4ae4df9700 1 -- 172.21.15.121:0/4052957840 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ae006bcf0 con 0x7f4ae00c0150 2023-11-15T03:04:37.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:37.019+0000 7f4ac77fe700 1 -- 172.21.15.121:0/4052957840 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f4ae00ac810 con 0x7f4ae00c0150 2023-11-15T03:04:37.026 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:04:37.026 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":120,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:04:36.187625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7896},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7896":{"gid":7896,"name":"a","rank":0,"incarnation":120,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/2831707405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2831707405},{"type":"v1","addr":"172.21.15.121:6836","nonce":2831707405}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:04:37.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:37.019+0000 7f4ae4df9700 1 -- 172.21.15.121:0/4052957840 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4acc05a540 msgr2=0x7f4acc05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:37.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:37.019+0000 7f4ae4df9700 1 --2- 172.21.15.121:0/4052957840 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4acc05a540 0x7f4acc05ca00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f4ae00c0c90 tx=0x7f4ad0009060).stop 2023-11-15T03:04:37.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:37.019+0000 7f4ae4df9700 1 -- 172.21.15.121:0/4052957840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ae00c0150 msgr2=0x7f4ae00a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:37.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:37.019+0000 7f4ae4df9700 1 --2- 172.21.15.121:0/4052957840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ae00c0150 0x7f4ae00a19b0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f4ad4006be0 tx=0x7f4ad4015000).stop 2023-11-15T03:04:37.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:37.019+0000 7f4ae4df9700 1 -- 172.21.15.121:0/4052957840 shutdown_connections 2023-11-15T03:04:37.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:37.019+0000 7f4ae4df9700 1 --2- 172.21.15.121:0/4052957840 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4acc05a540 0x7f4acc05ca00 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:37.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:37.019+0000 7f4ae4df9700 1 --2- 172.21.15.121:0/4052957840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ae00a1ea0 0x7f4ae01679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:37.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:37.019+0000 7f4ae4df9700 1 --2- 172.21.15.121:0/4052957840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ae00c0150 0x7f4ae00a19b0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:37.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:37.019+0000 7f4ae4df9700 1 --2- 172.21.15.121:0/4052957840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac8004690 0x7f4ae00bfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:37.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:37.019+0000 7f4ae4df9700 1 -- 172.21.15.121:0/4052957840 >> 172.21.15.121:0/4052957840 conn(0x7f4ae0062190 msgr2=0x7f4ae014d460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:37.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:37.019+0000 7f4ae4df9700 1 -- 172.21.15.121:0/4052957840 shutdown_connections 2023-11-15T03:04:37.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:37.019+0000 7f4ae4df9700 1 -- 172.21.15.121:0/4052957840 wait complete. 2023-11-15T03:04:37.033 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 120 2023-11-15T03:04:37.047 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 120, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:04:36.187625+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7896}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7896': {'gid': 7896, 'name': 'a', 'rank': 0, 'incarnation': 120, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.121:6836/2831707405', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 2831707405}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 2831707405}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:04:37.048 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7896:up:replay 2023-11-15T03:04:37.048 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:04:41.388 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:04:41.397 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:04:42.137 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:04:42.127+0000 7febfed79700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:04:42.453 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:04:42.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.607+0000 7f50f365b700 1 Processor -- start 2023-11-15T03:04:42.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.607+0000 7f50f365b700 1 -- start start 2023-11-15T03:04:42.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.607+0000 7f50f365b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f50ec1562f0 0x7f50ec155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:42.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.607+0000 7f50f365b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ec0c4c30 con 0x7f50ec154ca0 2023-11-15T03:04:42.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.607+0000 7f50f365b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ec0a6f50 con 0x7f50ec0a57c0 2023-11-15T03:04:42.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.607+0000 7f50f365b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50ec0a8280 con 0x7f50ec1562f0 2023-11-15T03:04:42.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.611+0000 7f50f13f7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f50ec154ca0 0x7f50ec0a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:55618/0 (socket says 172.21.15.121:55618) 2023-11-15T03:04:42.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.611+0000 7f50f13f7700 1 -- 172.21.15.121:0/127123178 learned_addr learned my addr 172.21.15.121:0/127123178 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:42.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.611+0000 7f50f0bf6700 1 --2- 172.21.15.121:0/127123178 >> v2:172.21.15.190:3300/0 conn(0x7f50ec1562f0 0x7f50ec155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:42.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.611+0000 7f50e3fff700 1 -- 172.21.15.121:0/127123178 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238019940 0 0) 0x7f50ec0c4c30 con 0x7f50ec154ca0 2023-11-15T03:04:42.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.611+0000 7f50e3fff700 1 -- 172.21.15.121:0/127123178 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50d80033f0 con 0x7f50ec154ca0 2023-11-15T03:04:42.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.611+0000 7f50f0bf6700 1 -- 172.21.15.121:0/127123178 >> v1:172.21.15.190:6789/0 conn(0x7f50ec0a57c0 legacy=0x7f50ec0c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:42.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.611+0000 7f50f0bf6700 1 -- 172.21.15.121:0/127123178 >> v1:172.21.15.121:6789/0 conn(0x7f50ec154ca0 legacy=0x7f50ec0a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:42.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.611+0000 7f50f0bf6700 1 -- 172.21.15.121:0/127123178 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec0a7230 con 0x7f50ec1562f0 2023-11-15T03:04:42.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.611+0000 7f50f0bf6700 1 --2- 172.21.15.121:0/127123178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec1562f0 0x7f50ec155080 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f50dc009760 tx=0x7f50dc009060).ready entity=mon.1 client_cookie=d063f526e35b3885 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:42.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.611+0000 7f50e3fff700 1 -- 172.21.15.121:0/127123178 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc00f510 con 0x7f50ec1562f0 2023-11-15T03:04:42.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.611+0000 7f50e3fff700 1 -- 172.21.15.121:0/127123178 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc00f670 con 0x7f50ec1562f0 2023-11-15T03:04:42.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.611+0000 7f50e3fff700 1 -- 172.21.15.121:0/127123178 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc00f9f0 con 0x7f50ec1562f0 2023-11-15T03:04:42.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.611+0000 7f50f365b700 1 -- 172.21.15.121:0/127123178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec1562f0 msgr2=0x7f50ec155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:42.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.611+0000 7f50f365b700 1 --2- 172.21.15.121:0/127123178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec1562f0 0x7f50ec155080 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f50dc009760 tx=0x7f50dc009060).stop 2023-11-15T03:04:42.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.611+0000 7f50f365b700 1 -- 172.21.15.121:0/127123178 shutdown_connections 2023-11-15T03:04:42.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.611+0000 7f50f365b700 1 --2- 172.21.15.121:0/127123178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec1562f0 0x7f50ec155080 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:42.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.611+0000 7f50f365b700 1 -- 172.21.15.121:0/127123178 >> 172.21.15.121:0/127123178 conn(0x7f50ec062190 msgr2=0x7f50ec0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:42.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.615+0000 7f50f365b700 1 -- 172.21.15.121:0/127123178 shutdown_connections 2023-11-15T03:04:42.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.615+0000 7f50f365b700 1 -- 172.21.15.121:0/127123178 wait complete. 2023-11-15T03:04:42.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.615+0000 7f50f365b700 1 Processor -- start 2023-11-15T03:04:42.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.615+0000 7f50f365b700 1 -- start start 2023-11-15T03:04:42.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.615+0000 7f50f365b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0a57c0 0x7f50ec13a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:42.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.615+0000 7f50f365b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50ec154ca0 0x7f50ec0c2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:42.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.615+0000 7f50f365b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec1562f0 0x7f50ec0c2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:42.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.615+0000 7f50f365b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec0c2c30 con 0x7f50ec154ca0 2023-11-15T03:04:42.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.615+0000 7f50f365b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec164d50 con 0x7f50ec1562f0 2023-11-15T03:04:42.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.615+0000 7f50f365b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec1594b0 con 0x7f50ec0a57c0 2023-11-15T03:04:42.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.619+0000 7f50f0bf6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50ec154ca0 0x7f50ec0c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:42.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.619+0000 7f50f0bf6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50ec154ca0 0x7f50ec0c2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42500/0 (socket says 172.21.15.121:42500) 2023-11-15T03:04:42.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.619+0000 7f50f0bf6700 1 -- 172.21.15.121:0/3629847085 learned_addr learned my addr 172.21.15.121:0/3629847085 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:42.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.619+0000 7f50f13f7700 1 --2- 172.21.15.121:0/3629847085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0a57c0 0x7f50ec13a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:42.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.619+0000 7f50f1bf8700 1 --2- 172.21.15.121:0/3629847085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec1562f0 0x7f50ec0c2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:42.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.619+0000 7f50f0bf6700 1 -- 172.21.15.121:0/3629847085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0a57c0 msgr2=0x7f50ec13a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:42.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.619+0000 7f50f0bf6700 1 --2- 172.21.15.121:0/3629847085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0a57c0 0x7f50ec13a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:42.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.619+0000 7f50f0bf6700 1 -- 172.21.15.121:0/3629847085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec1562f0 msgr2=0x7f50ec0c2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:42.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.619+0000 7f50f0bf6700 1 --2- 172.21.15.121:0/3629847085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec1562f0 0x7f50ec0c2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:42.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.619+0000 7f50f0bf6700 1 -- 172.21.15.121:0/3629847085 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc0092a0 con 0x7f50ec154ca0 2023-11-15T03:04:42.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.619+0000 7f50f13f7700 1 --2- 172.21.15.121:0/3629847085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0a57c0 0x7f50ec13a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:04:42.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.619+0000 7f50f1bf8700 1 --2- 172.21.15.121:0/3629847085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec1562f0 0x7f50ec0c2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:04:42.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.619+0000 7f50f0bf6700 1 --2- 172.21.15.121:0/3629847085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50ec154ca0 0x7f50ec0c2250 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7f50ec063d40 tx=0x7f50dc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:42.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.619+0000 7f50e17fa700 1 -- 172.21.15.121:0/3629847085 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc006cd0 con 0x7f50ec154ca0 2023-11-15T03:04:42.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.619+0000 7f50e17fa700 1 -- 172.21.15.121:0/3629847085 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc000c90 con 0x7f50ec154ca0 2023-11-15T03:04:42.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.619+0000 7f50f365b700 1 -- 172.21.15.121:0/3629847085 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ec164700 con 0x7f50ec154ca0 2023-11-15T03:04:42.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.619+0000 7f50e17fa700 1 -- 172.21.15.121:0/3629847085 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc00f9f0 con 0x7f50ec154ca0 2023-11-15T03:04:42.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.619+0000 7f50f365b700 1 -- 172.21.15.121:0/3629847085 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ec164400 con 0x7f50ec154ca0 2023-11-15T03:04:42.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.619+0000 7f50f365b700 1 -- 172.21.15.121:0/3629847085 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ec164ee0 con 0x7f50ec154ca0 2023-11-15T03:04:42.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.623+0000 7f50e17fa700 1 -- 172.21.15.121:0/3629847085 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f50dc01b8b0 con 0x7f50ec154ca0 2023-11-15T03:04:42.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.623+0000 7f50e17fa700 1 --2- 172.21.15.121:0/3629847085 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50d005a510 0x7f50d005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:42.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.623+0000 7f50e17fa700 1 -- 172.21.15.121:0/3629847085 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6782+0+0 (secure 0 0 0) 0x7f50dc076420 con 0x7f50ec154ca0 2023-11-15T03:04:42.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.623+0000 7f50f13f7700 1 --2- 172.21.15.121:0/3629847085 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50d005a510 0x7f50d005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:42.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.623+0000 7f50f13f7700 1 --2- 172.21.15.121:0/3629847085 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50d005a510 0x7f50d005c9d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f50e8005750 tx=0x7f50e8009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:42.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.631+0000 7f50e17fa700 1 -- 172.21.15.121:0/3629847085 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f50dc049a10 con 0x7f50ec154ca0 2023-11-15T03:04:42.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.827+0000 7f50f365b700 1 -- 172.21.15.121:0/3629847085 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50ec06c150 con 0x7f50ec154ca0 2023-11-15T03:04:42.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.827+0000 7f50e17fa700 1 -- 172.21.15.121:0/3629847085 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f50dc029090 con 0x7f50ec154ca0 2023-11-15T03:04:42.835 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:04:42.835 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":123,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:04:39.211016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7896},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7896":{"gid":7896,"name":"a","rank":0,"incarnation":120,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6836/2831707405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2831707405},{"type":"v1","addr":"172.21.15.121:6836","nonce":2831707405}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:04:42.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.831+0000 7f50f365b700 1 -- 172.21.15.121:0/3629847085 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50d005a510 msgr2=0x7f50d005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:42.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.831+0000 7f50f365b700 1 --2- 172.21.15.121:0/3629847085 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50d005a510 0x7f50d005c9d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f50e8005750 tx=0x7f50e8009040).stop 2023-11-15T03:04:42.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.831+0000 7f50f365b700 1 -- 172.21.15.121:0/3629847085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50ec154ca0 msgr2=0x7f50ec0c2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:42.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.831+0000 7f50f365b700 1 --2- 172.21.15.121:0/3629847085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50ec154ca0 0x7f50ec0c2250 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7f50ec063d40 tx=0x7f50dc017000).stop 2023-11-15T03:04:42.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.831+0000 7f50f365b700 1 -- 172.21.15.121:0/3629847085 shutdown_connections 2023-11-15T03:04:42.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.831+0000 7f50f365b700 1 --2- 172.21.15.121:0/3629847085 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f50d005a510 0x7f50d005c9d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:42.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.831+0000 7f50f365b700 1 --2- 172.21.15.121:0/3629847085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec1562f0 0x7f50ec0c2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:42.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.831+0000 7f50f365b700 1 --2- 172.21.15.121:0/3629847085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50ec154ca0 0x7f50ec0c2250 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:42.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.831+0000 7f50f365b700 1 --2- 172.21.15.121:0/3629847085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0a57c0 0x7f50ec13a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:42.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.831+0000 7f50f365b700 1 -- 172.21.15.121:0/3629847085 >> 172.21.15.121:0/3629847085 conn(0x7f50ec062190 msgr2=0x7f50ec0c0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:42.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.831+0000 7f50f365b700 1 -- 172.21.15.121:0/3629847085 shutdown_connections 2023-11-15T03:04:42.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:42.831+0000 7f50f365b700 1 -- 172.21.15.121:0/3629847085 wait complete. 2023-11-15T03:04:42.843 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 123 2023-11-15T03:04:42.861 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 123, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:04:39.211016+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7896}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7896': {'gid': 7896, 'name': 'a', 'rank': 0, 'incarnation': 120, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.121:6836/2831707405', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 2831707405}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 2831707405}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:04:42.861 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T03:04:42.862 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-15T03:04:43.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.019+0000 7ff4c95b9700 1 Processor -- start 2023-11-15T03:04:43.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.019+0000 7ff4c95b9700 1 -- start start 2023-11-15T03:04:43.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.023+0000 7ff4c95b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff4c40c7a90 0x7ff4c4158260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.023+0000 7ff4c95b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c40c6170 con 0x7ff4c40a9f80 2023-11-15T03:04:43.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.027+0000 7ff4c95b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c4157dd0 con 0x7ff4c40c5a00 2023-11-15T03:04:43.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.027+0000 7ff4c95b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c4154000 con 0x7ff4c40c7a90 2023-11-15T03:04:43.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.027+0000 7ff4c3fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff4c40a9f80 0x7ff4c40c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:55634/0 (socket says 172.21.15.121:55634) 2023-11-15T03:04:43.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.027+0000 7ff4c3fff700 1 -- 172.21.15.121:0/3098852478 learned_addr learned my addr 172.21.15.121:0/3098852478 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:43.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.027+0000 7ff4c8db8700 1 --2- 172.21.15.121:0/3098852478 >> v2:172.21.15.190:3300/0 conn(0x7ff4c40c7a90 0x7ff4c4158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.027+0000 7ff4c8db8700 1 -- 172.21.15.121:0/3098852478 >> v1:172.21.15.190:6789/0 conn(0x7ff4c40c5a00 legacy=0x7ff4c4065710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:43.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.027+0000 7ff4c8db8700 1 -- 172.21.15.121:0/3098852478 >> v1:172.21.15.121:6789/0 conn(0x7ff4c40a9f80 legacy=0x7ff4c40c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:43.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.027+0000 7ff4c8db8700 1 -- 172.21.15.121:0/3098852478 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c40c5e90 con 0x7ff4c40c7a90 2023-11-15T03:04:43.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.027+0000 7ff4c8db8700 1 --2- 172.21.15.121:0/3098852478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4c40c7a90 0x7ff4c4158260 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7ff4b4009760 tx=0x7ff4b4009060).ready entity=mon.1 client_cookie=52a8b76cbaa327e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:43.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.031+0000 7ff4c2ffd700 1 -- 172.21.15.121:0/3098852478 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b400b580 con 0x7ff4c40c7a90 2023-11-15T03:04:43.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.031+0000 7ff4c2ffd700 1 -- 172.21.15.121:0/3098852478 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b400b6e0 con 0x7ff4c40c7a90 2023-11-15T03:04:43.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.031+0000 7ff4c2ffd700 1 -- 172.21.15.121:0/3098852478 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b400ba60 con 0x7ff4c40c7a90 2023-11-15T03:04:43.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.031+0000 7ff4c95b9700 1 -- 172.21.15.121:0/3098852478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4c40c7a90 msgr2=0x7ff4c4158260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:43.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.031+0000 7ff4c95b9700 1 --2- 172.21.15.121:0/3098852478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4c40c7a90 0x7ff4c4158260 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7ff4b4009760 tx=0x7ff4b4009060).stop 2023-11-15T03:04:43.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.031+0000 7ff4c95b9700 1 -- 172.21.15.121:0/3098852478 shutdown_connections 2023-11-15T03:04:43.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.031+0000 7ff4c95b9700 1 --2- 172.21.15.121:0/3098852478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4c40c7a90 0x7ff4c4158260 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.031+0000 7ff4c95b9700 1 -- 172.21.15.121:0/3098852478 >> 172.21.15.121:0/3098852478 conn(0x7ff4c4067a60 msgr2=0x7ff4c4067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:43.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.031+0000 7ff4c95b9700 1 -- 172.21.15.121:0/3098852478 shutdown_connections 2023-11-15T03:04:43.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.031+0000 7ff4c95b9700 1 -- 172.21.15.121:0/3098852478 wait complete. 2023-11-15T03:04:43.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.035+0000 7ff4c95b9700 1 Processor -- start 2023-11-15T03:04:43.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.035+0000 7ff4c95b9700 1 -- start start 2023-11-15T03:04:43.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.035+0000 7ff4c95b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4c40a9f80 0x7ff4c40ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.035+0000 7ff4c95b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4c40c7a90 0x7ff4c40ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.035+0000 7ff4c95b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4c4164320 0x7ff4c4164780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.035+0000 7ff4c95b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c414e6f0 con 0x7ff4c4164320 2023-11-15T03:04:43.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.035+0000 7ff4c95b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c40cb0e0 con 0x7ff4c40a9f80 2023-11-15T03:04:43.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.035+0000 7ff4c95b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c41548f0 con 0x7ff4c40c7a90 2023-11-15T03:04:43.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.035+0000 7ff4c8db8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4c4164320 0x7ff4c4164780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.035+0000 7ff4c8db8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4c4164320 0x7ff4c4164780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42504/0 (socket says 172.21.15.121:42504) 2023-11-15T03:04:43.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.035+0000 7ff4c8db8700 1 -- 172.21.15.121:0/3259910018 learned_addr learned my addr 172.21.15.121:0/3259910018 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:43.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.035+0000 7ff4c3fff700 1 --2- 172.21.15.121:0/3259910018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4c40a9f80 0x7ff4c40ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.035+0000 7ff4c37fe700 1 --2- 172.21.15.121:0/3259910018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4c40c7a90 0x7ff4c40ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.035+0000 7ff4c8db8700 1 -- 172.21.15.121:0/3259910018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4c40c7a90 msgr2=0x7ff4c40ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:43.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.035+0000 7ff4c8db8700 1 --2- 172.21.15.121:0/3259910018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4c40c7a90 0x7ff4c40ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.035+0000 7ff4c8db8700 1 -- 172.21.15.121:0/3259910018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4c40a9f80 msgr2=0x7ff4c40ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:43.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.035+0000 7ff4c8db8700 1 --2- 172.21.15.121:0/3259910018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4c40a9f80 0x7ff4c40ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.035+0000 7ff4c8db8700 1 -- 172.21.15.121:0/3259910018 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b40092a0 con 0x7ff4c4164320 2023-11-15T03:04:43.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.039+0000 7ff4c8db8700 1 --2- 172.21.15.121:0/3259910018 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4c4164320 0x7ff4c4164780 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7ff4b4011f30 tx=0x7ff4b4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:43.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.039+0000 7ff4abfff700 1 -- 172.21.15.121:0/3259910018 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b400ba60 con 0x7ff4c4164320 2023-11-15T03:04:43.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.039+0000 7ff4abfff700 1 -- 172.21.15.121:0/3259910018 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b401b440 con 0x7ff4c4164320 2023-11-15T03:04:43.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.039+0000 7ff4c95b9700 1 -- 172.21.15.121:0/3259910018 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c414c370 con 0x7ff4c4164320 2023-11-15T03:04:43.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.039+0000 7ff4abfff700 1 -- 172.21.15.121:0/3259910018 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b401b7c0 con 0x7ff4c4164320 2023-11-15T03:04:43.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.039+0000 7ff4c95b9700 1 -- 172.21.15.121:0/3259910018 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c40cae00 con 0x7ff4c4164320 2023-11-15T03:04:43.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.043+0000 7ff4abfff700 1 -- 172.21.15.121:0/3259910018 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff4b401bb70 con 0x7ff4c4164320 2023-11-15T03:04:43.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.043+0000 7ff4abfff700 1 --2- 172.21.15.121:0/3259910018 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff4b005a510 0x7ff4b005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.043+0000 7ff4abfff700 1 -- 172.21.15.121:0/3259910018 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6782+0+0 (secure 0 0 0) 0x7ff4b4076420 con 0x7ff4c4164320 2023-11-15T03:04:43.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.043+0000 7ff4c3fff700 1 --2- 172.21.15.121:0/3259910018 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff4b005a510 0x7ff4b005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.043+0000 7ff4c3fff700 1 --2- 172.21.15.121:0/3259910018 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff4b005a510 0x7ff4b005c9d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7ff4bc001a90 tx=0x7ff4bc001560).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:43.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4d4b90740 1 Processor -- start 2023-11-15T03:04:43.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4d4b90740 1 -- start start 2023-11-15T03:04:43.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4d4b90740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2627ad0 0x264e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4d4b90740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2627eb0 0x2674140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4d4b90740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2674630 0x2674ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4d4b90740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x25a8410 con 0x2627eb0 2023-11-15T03:04:43.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4d4b90740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x25a8550 con 0x2674630 2023-11-15T03:04:43.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4d4b90740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x25a8a00 con 0x2627ad0 2023-11-15T03:04:43.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4c37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2627eb0 0x2674140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4c37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2627eb0 0x2674140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42508/0 (socket says 172.21.15.121:42508) 2023-11-15T03:04:43.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4c37fe700 1 -- 172.21.15.121:0/454525371 learned_addr learned my addr 172.21.15.121:0/454525371 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:43.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4c3fff700 1 --2- 172.21.15.121:0/454525371 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2627ad0 0x264e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4c8db8700 1 --2- 172.21.15.121:0/454525371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2674630 0x2674ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4c37fe700 1 -- 172.21.15.121:0/454525371 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2627ad0 msgr2=0x264e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:43.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4c37fe700 1 --2- 172.21.15.121:0/454525371 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2627ad0 0x264e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4c37fe700 1 -- 172.21.15.121:0/454525371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2674630 msgr2=0x2674ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:43.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4c37fe700 1 --2- 172.21.15.121:0/454525371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2674630 0x2674ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4c37fe700 1 -- 172.21.15.121:0/454525371 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x25a1730 con 0x2627eb0 2023-11-15T03:04:43.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4c37fe700 1 --2- 172.21.15.121:0/454525371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2627eb0 0x2674140 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x7ff4b8003650 tx=0x7ff4b8007d40).ready entity=mon.0 client_cookie=582e034908fcaf26 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:43.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4a97fa700 1 -- 172.21.15.121:0/454525371 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8003040 con 0x2627eb0 2023-11-15T03:04:43.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4a97fa700 1 -- 172.21.15.121:0/454525371 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b8003270 con 0x2627eb0 2023-11-15T03:04:43.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.047+0000 7ff4a97fa700 1 -- 172.21.15.121:0/454525371 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8007450 con 0x2627eb0 2023-11-15T03:04:43.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4d4b90740 1 -- 172.21.15.121:0/454525371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2627eb0 msgr2=0x2674140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:43.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/454525371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2627eb0 0x2674140 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x7ff4b8003650 tx=0x7ff4b8007d40).stop 2023-11-15T03:04:43.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4d4b90740 1 -- 172.21.15.121:0/454525371 shutdown_connections 2023-11-15T03:04:43.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/454525371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2674630 0x2674ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/454525371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2627eb0 0x2674140 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/454525371 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2627ad0 0x264e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4d4b90740 1 -- 172.21.15.121:0/454525371 >> 172.21.15.121:0/454525371 conn(0x2406550 msgr2=0x26267e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:43.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4d4b90740 1 -- 172.21.15.121:0/454525371 shutdown_connections 2023-11-15T03:04:43.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4d4b90740 1 -- 172.21.15.121:0/454525371 wait complete. 2023-11-15T03:04:43.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4d4b90740 1 -- start start 2023-11-15T03:04:43.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4d4b90740 1 Processor -- start 2023-11-15T03:04:43.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4d4b90740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2627ad0 0x2667310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4d4b90740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2627eb0 0x264a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4d4b90740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2674630 0x266ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4d4b90740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2573650 con 0x2674630 2023-11-15T03:04:43.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4d4b90740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x26709d0 con 0x2627eb0 2023-11-15T03:04:43.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4d4b90740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2667a10 con 0x2627ad0 2023-11-15T03:04:43.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4d4b90740 10 client.0 ll_register_callbacks cb 0x25e0020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:04:43.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4c8db8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2674630 0x266ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4c8db8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2674630 0x266ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42514/0 (socket says 172.21.15.121:42514) 2023-11-15T03:04:43.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4c8db8700 1 -- 172.21.15.121:0/3531894823 learned_addr learned my addr 172.21.15.121:0/3531894823 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:43.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4c3fff700 1 --2- 172.21.15.121:0/3531894823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2627ad0 0x2667310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.051+0000 7ff4c37fe700 1 --2- 172.21.15.121:0/3531894823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2627eb0 0x264a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4c8db8700 1 -- 172.21.15.121:0/3531894823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2627ad0 msgr2=0x2667310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:43.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4c8db8700 1 --2- 172.21.15.121:0/3531894823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2627ad0 0x2667310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4c8db8700 1 -- 172.21.15.121:0/3531894823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2627eb0 msgr2=0x264a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:43.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4c8db8700 1 --2- 172.21.15.121:0/3531894823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2627eb0 0x264a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4c8db8700 1 -- 172.21.15.121:0/3531894823 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c4157dd0 con 0x2674630 2023-11-15T03:04:43.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4c37fe700 1 --2- 172.21.15.121:0/3531894823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2627eb0 0x264a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:43.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4c8db8700 1 --2- 172.21.15.121:0/3531894823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2674630 0x266ff50 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7ff4b4009210 tx=0x7ff4b4076040).ready entity=mon.0 client_cookie=3c58c91be094374b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:43.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4a27fc700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:04:43.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4a27fc700 1 -- 172.21.15.121:0/3531894823 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b40760b0 con 0x2674630 2023-11-15T03:04:43.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4a27fc700 1 -- 172.21.15.121:0/3531894823 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6782+0+0 (secure 0 0 0) 0x7ff4b4076210 con 0x2674630 2023-11-15T03:04:43.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4d4b90740 1 -- 172.21.15.121:0/3531894823 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x259e1a0 con 0x2674630 2023-11-15T03:04:43.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4a27fc700 1 -- 172.21.15.121:0/3531894823 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4b404cd30 con 0x2674630 2023-11-15T03:04:43.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4d4b90740 10 client.7974 fetch_fsmap learned FSMap version 123 2023-11-15T03:04:43.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4d4b90740 1 -- 172.21.15.121:0/3531894823 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=2+}) v3 -- 0x259c960 con 0x2674630 2023-11-15T03:04:43.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4a27fc700 1 -- 172.21.15.121:0/3531894823 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x259e1a0 con 0x2674630 2023-11-15T03:04:43.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4a27fc700 1 -- 172.21.15.121:0/3531894823 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 123) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ff4b4049420 con 0x2674630 2023-11-15T03:04:43.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4d4b90740 10 client.7974 fetch_fsmap finished waiting for FSMap version 123 2023-11-15T03:04:43.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4d4b90740 10 client.7974 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:04:43.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4d4b90740 20 client.7974 populate_metadata read hostname 'smithi121' 2023-11-15T03:04:43.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/3531894823 >> [v2:172.21.15.121:6834/2831707405,v1:172.21.15.121:6836/2831707405] conn(0x2670440 0x2644d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4d4b90740 4 client.7974 mds_command: new command op to 7896 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:04:43.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4d4b90740 1 -- 172.21.15.121:0/3531894823 --> [v2:172.21.15.121:6834/2831707405,v1:172.21.15.121:6836/2831707405] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x25946f0 con 0x2670440 2023-11-15T03:04:43.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4c3fff700 1 --2- 172.21.15.121:0/3531894823 >> [v2:172.21.15.121:6834/2831707405,v1:172.21.15.121:6836/2831707405] conn(0x2670440 0x2644d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4c3fff700 1 --2- 172.21.15.121:0/3531894823 >> [v2:172.21.15.121:6834/2831707405,v1:172.21.15.121:6836/2831707405] conn(0x2670440 0x2644d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=56998b041a869a95 server_cookie=d7100f2de26c5a63 in_seq=0 out_seq=0 2023-11-15T03:04:43.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4a27fc700 10 client.7974 ms_handle_connect on v2:172.21.15.121:6834/2831707405 2023-11-15T03:04:43.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4a27fc700 1 -- 172.21.15.121:0/3531894823 <== mds.0 v2:172.21.15.121:6834/2831707405 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x25946f0 con 0x2670440 2023-11-15T03:04:43.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4a27fc700 10 client.7974 handle_command_reply: tid=0 2023-11-15T03:04:43.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.055+0000 7ff4d4b90740 1 client.7974 shutdown 2023-11-15T03:04:43.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.059+0000 7ff4d4b90740 1 -- 172.21.15.121:0/3531894823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2674630 msgr2=0x266ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:43.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.059+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/3531894823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2674630 0x266ff50 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7ff4b4009210 tx=0x7ff4b4076040).stop 2023-11-15T03:04:43.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.059+0000 7ff4d4b90740 1 -- 172.21.15.121:0/3531894823 shutdown_connections 2023-11-15T03:04:43.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.059+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/3531894823 >> [v2:172.21.15.121:6834/2831707405,v1:172.21.15.121:6836/2831707405] conn(0x2670440 0x2644d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.059+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/3531894823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2674630 0x266ff50 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.059+0000 7ff4a27fc700 0 client.7974 ms_handle_reset on v2:172.21.15.121:6834/2831707405 2023-11-15T03:04:43.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.059+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/3531894823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2627eb0 0x264a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.059+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/3531894823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2627ad0 0x2667310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.059+0000 7ff4d4b90740 1 -- 172.21.15.121:0/3531894823 >> 172.21.15.121:0/3531894823 conn(0x2406550 msgr2=0x266f580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:43.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.059+0000 7ff4d4b90740 1 -- 172.21.15.121:0/3531894823 shutdown_connections 2023-11-15T03:04:43.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.059+0000 7ff4d4b90740 1 -- 172.21.15.121:0/3531894823 wait complete. 2023-11-15T03:04:43.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.059+0000 7ff4d4b90740 20 client.7974 trim_cache size 0 max 16384 2023-11-15T03:04:43.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.075+0000 7ff4d4b90740 1 Processor -- start 2023-11-15T03:04:43.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.075+0000 7ff4d4b90740 1 -- start start 2023-11-15T03:04:43.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4d4b90740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2627ad0 0x26430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4d4b90740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2627eb0 0x2668270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4d4b90740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2674630 0x264da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4d4b90740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2583a00 con 0x2674630 2023-11-15T03:04:43.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4d4b90740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x251eeb0 con 0x2627eb0 2023-11-15T03:04:43.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4d4b90740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2622bb0 con 0x2627ad0 2023-11-15T03:04:43.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4c8db8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2674630 0x264da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4c3fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2627ad0 0x26430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4c8db8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2674630 0x264da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42516/0 (socket says 172.21.15.121:42516) 2023-11-15T03:04:43.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4c8db8700 1 -- 172.21.15.121:0/3467150818 learned_addr learned my addr 172.21.15.121:0/3467150818 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:43.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4c37fe700 1 --2- 172.21.15.121:0/3467150818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2627eb0 0x2668270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4c8db8700 1 -- 172.21.15.121:0/3467150818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2627ad0 msgr2=0x26430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:43.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4c8db8700 1 --2- 172.21.15.121:0/3467150818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2627ad0 0x26430f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4c8db8700 1 -- 172.21.15.121:0/3467150818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2627eb0 msgr2=0x2668270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:43.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4c8db8700 1 --2- 172.21.15.121:0/3467150818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2627eb0 0x2668270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4c8db8700 1 -- 172.21.15.121:0/3467150818 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c4164fd0 con 0x2674630 2023-11-15T03:04:43.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4c8db8700 1 --2- 172.21.15.121:0/3467150818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2674630 0x264da40 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7ff4b4006d10 tx=0x7ff4b4030000).ready entity=mon.0 client_cookie=f08477edcfad3da5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:43.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4a9ffb700 1 -- 172.21.15.121:0/3467150818 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b401b960 con 0x2674630 2023-11-15T03:04:43.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4a9ffb700 1 -- 172.21.15.121:0/3467150818 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b404cd30 con 0x2674630 2023-11-15T03:04:43.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4a9ffb700 1 -- 172.21.15.121:0/3467150818 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4076030 con 0x2674630 2023-11-15T03:04:43.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4d4b90740 1 -- 172.21.15.121:0/3467150818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2674630 msgr2=0x264da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:43.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/3467150818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2674630 0x264da40 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7ff4b4006d10 tx=0x7ff4b4030000).stop 2023-11-15T03:04:43.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4d4b90740 1 -- 172.21.15.121:0/3467150818 shutdown_connections 2023-11-15T03:04:43.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/3467150818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2674630 0x264da40 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/3467150818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2627eb0 0x2668270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/3467150818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2627ad0 0x26430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4d4b90740 1 -- 172.21.15.121:0/3467150818 >> 172.21.15.121:0/3467150818 conn(0x2406550 msgr2=0x2644d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:43.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4d4b90740 1 -- 172.21.15.121:0/3467150818 shutdown_connections 2023-11-15T03:04:43.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.079+0000 7ff4d4b90740 1 -- 172.21.15.121:0/3467150818 wait complete. 2023-11-15T03:04:43.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4d4b90740 1 -- start start 2023-11-15T03:04:43.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4d4b90740 1 Processor -- start 2023-11-15T03:04:43.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4d4b90740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2627ad0 0x2647540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4d4b90740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2627eb0 0x26d8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4d4b90740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2674630 0x2665150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4d4b90740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x26662b0 con 0x2627ad0 2023-11-15T03:04:43.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4d4b90740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x26450a0 con 0x2627eb0 2023-11-15T03:04:43.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4d4b90740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x267ccc0 con 0x2674630 2023-11-15T03:04:43.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4d4b90740 10 client.0 ll_register_callbacks cb 0x7ff4b400b2a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:04:43.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4c3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2627ad0 0x2647540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4c3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2627ad0 0x2647540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42522/0 (socket says 172.21.15.121:42522) 2023-11-15T03:04:43.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4c3fff700 1 -- 172.21.15.121:0/1432043121 learned_addr learned my addr 172.21.15.121:0/1432043121 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:43.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4c8db8700 1 --2- 172.21.15.121:0/1432043121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2674630 0x2665150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4c37fe700 1 --2- 172.21.15.121:0/1432043121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2627eb0 0x26d8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4c3fff700 1 -- 172.21.15.121:0/1432043121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2674630 msgr2=0x2665150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:43.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4c3fff700 1 --2- 172.21.15.121:0/1432043121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2674630 0x2665150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4c3fff700 1 -- 172.21.15.121:0/1432043121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2627eb0 msgr2=0x26d8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:43.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4c3fff700 1 --2- 172.21.15.121:0/1432043121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2627eb0 0x26d8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4c3fff700 1 -- 172.21.15.121:0/1432043121 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x259c960 con 0x2627ad0 2023-11-15T03:04:43.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4c37fe700 1 --2- 172.21.15.121:0/1432043121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2627eb0 0x26d8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:43.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4c3fff700 1 --2- 172.21.15.121:0/1432043121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2627ad0 0x2647540 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x26490b0 tx=0x7ff4bc019040).ready entity=mon.0 client_cookie=d03b2d6b795c89a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:43.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4a27fc700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:04:43.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4a27fc700 1 -- 172.21.15.121:0/1432043121 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc016ac0 con 0x2627ad0 2023-11-15T03:04:43.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4a27fc700 1 -- 172.21.15.121:0/1432043121 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6782+0+0 (secure 0 0 0) 0x7ff4bc016c20 con 0x2627ad0 2023-11-15T03:04:43.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.083+0000 7ff4d4b90740 1 -- 172.21.15.121:0/1432043121 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4b8003270 con 0x2627ad0 2023-11-15T03:04:43.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4a27fc700 1 -- 172.21.15.121:0/1432043121 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4bc014430 con 0x2627ad0 2023-11-15T03:04:43.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4d4b90740 10 client.7980 fetch_fsmap learned FSMap version 123 2023-11-15T03:04:43.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4d4b90740 1 -- 172.21.15.121:0/1432043121 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=2+}) v3 -- 0x259c6b0 con 0x2627ad0 2023-11-15T03:04:43.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4a27fc700 1 -- 172.21.15.121:0/1432043121 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b8003270 con 0x2627ad0 2023-11-15T03:04:43.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4a27fc700 1 -- 172.21.15.121:0/1432043121 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 123) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ff4bc02e380 con 0x2627ad0 2023-11-15T03:04:43.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4d4b90740 10 client.7980 fetch_fsmap finished waiting for FSMap version 123 2023-11-15T03:04:43.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4d4b90740 10 client.7980 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:04:43.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4d4b90740 20 client.7980 populate_metadata read hostname 'smithi121' 2023-11-15T03:04:43.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/1432043121 >> [v2:172.21.15.121:6834/2831707405,v1:172.21.15.121:6836/2831707405] conn(0x26843c0 0x26847a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4d4b90740 4 client.7980 mds_command: new command op to 7896 tid=0[{"prefix": "status"}] 2023-11-15T03:04:43.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4d4b90740 1 -- 172.21.15.121:0/1432043121 --> [v2:172.21.15.121:6834/2831707405,v1:172.21.15.121:6836/2831707405] -- command(tid 0: {"prefix": "status"}) v1 -- 0x245d4e0 con 0x26843c0 2023-11-15T03:04:43.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4c37fe700 1 --2- 172.21.15.121:0/1432043121 >> [v2:172.21.15.121:6834/2831707405,v1:172.21.15.121:6836/2831707405] conn(0x26843c0 0x26847a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4a27fc700 10 client.7980 ms_handle_connect on v2:172.21.15.121:6834/2831707405 2023-11-15T03:04:43.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4c37fe700 1 --2- 172.21.15.121:0/1432043121 >> [v2:172.21.15.121:6834/2831707405,v1:172.21.15.121:6836/2831707405] conn(0x26843c0 0x26847a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=edd13a294f41d293 server_cookie=b6c8ccfd16f380d0 in_seq=0 out_seq=0 2023-11-15T03:04:43.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4a27fc700 1 -- 172.21.15.121:0/1432043121 <== mds.0 v2:172.21.15.121:6834/2831707405 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x245d4e0 con 0x26843c0 2023-11-15T03:04:43.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4a27fc700 10 client.7980 handle_command_reply: tid=0 2023-11-15T03:04:43.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4d4b90740 1 client.7980 shutdown 2023-11-15T03:04:43.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4d4b90740 1 -- 172.21.15.121:0/1432043121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2627ad0 msgr2=0x2647540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:43.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/1432043121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2627ad0 0x2647540 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x26490b0 tx=0x7ff4bc019040).stop 2023-11-15T03:04:43.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4d4b90740 1 -- 172.21.15.121:0/1432043121 shutdown_connections 2023-11-15T03:04:43.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/1432043121 >> [v2:172.21.15.121:6834/2831707405,v1:172.21.15.121:6836/2831707405] conn(0x26843c0 0x26847a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/1432043121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2674630 0x2665150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/1432043121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2627eb0 0x26d8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4d4b90740 1 --2- 172.21.15.121:0/1432043121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2627ad0 0x2647540 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4a27fc700 0 client.7980 ms_handle_reset on v2:172.21.15.121:6834/2831707405 2023-11-15T03:04:43.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4d4b90740 1 -- 172.21.15.121:0/1432043121 >> 172.21.15.121:0/1432043121 conn(0x2406550 msgr2=0x26284e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:43.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4d4b90740 1 -- 172.21.15.121:0/1432043121 shutdown_connections 2023-11-15T03:04:43.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.087+0000 7ff4d4b90740 1 -- 172.21.15.121:0/1432043121 wait complete. 2023-11-15T03:04:43.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.091+0000 7ff4d4b90740 20 client.7980 trim_cache size 0 max 16384 2023-11-15T03:04:43.112 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:04:43.113 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T03:04:43.113 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T03:04:43.113 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 7896, 2023-11-15T03:04:43.113 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T03:04:43.113 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T03:04:43.114 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T03:04:43.114 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 6.9011091540000002, 2023-11-15T03:04:43.114 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 123, 2023-11-15T03:04:43.114 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 100, 2023-11-15T03:04:43.114 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 100, 2023-11-15T03:04:43.115 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 16.939151441 2023-11-15T03:04:43.115 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:04:43.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.091+0000 7ff4c95b9700 1 -- 172.21.15.121:0/3259910018 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff4b005a510 msgr2=0x7ff4b005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:43.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.091+0000 7ff4c95b9700 1 --2- 172.21.15.121:0/3259910018 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff4b005a510 0x7ff4b005c9d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7ff4bc001a90 tx=0x7ff4bc001560).stop 2023-11-15T03:04:43.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.091+0000 7ff4c95b9700 1 -- 172.21.15.121:0/3259910018 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4c4164320 msgr2=0x7ff4c4164780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:43.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.091+0000 7ff4c95b9700 1 --2- 172.21.15.121:0/3259910018 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4c4164320 0x7ff4c4164780 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7ff4b4011f30 tx=0x7ff4b4017000).stop 2023-11-15T03:04:43.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.091+0000 7ff4c95b9700 1 -- 172.21.15.121:0/3259910018 shutdown_connections 2023-11-15T03:04:43.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.091+0000 7ff4c95b9700 1 --2- 172.21.15.121:0/3259910018 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff4b005a510 0x7ff4b005c9d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.091+0000 7ff4c95b9700 1 --2- 172.21.15.121:0/3259910018 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4c4164320 0x7ff4c4164780 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.091+0000 7ff4c95b9700 1 --2- 172.21.15.121:0/3259910018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4c40c7a90 0x7ff4c40ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.091+0000 7ff4c95b9700 1 --2- 172.21.15.121:0/3259910018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4c40a9f80 0x7ff4c40ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.091+0000 7ff4c95b9700 1 -- 172.21.15.121:0/3259910018 >> 172.21.15.121:0/3259910018 conn(0x7ff4c4067a60 msgr2=0x7ff4c4065c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:43.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.091+0000 7ff4c95b9700 1 -- 172.21.15.121:0/3259910018 shutdown_connections 2023-11-15T03:04:43.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.091+0000 7ff4c95b9700 1 -- 172.21.15.121:0/3259910018 wait complete. 2023-11-15T03:04:43.121 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:04:43.121 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete mds0_openfiles.0', proceeding to ls 2023-11-15T03:04:43.121 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T03:04:43.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.247+0000 7f6398879700 1 Processor -- start 2023-11-15T03:04:43.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.247+0000 7f6398879700 1 -- start start 2023-11-15T03:04:43.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.247+0000 7f6398879700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f639415b450 0x7f63940657a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.247+0000 7f6398879700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f63940c2380 0x7f63940c2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.247+0000 7f6398879700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63940c2d20 con 0x7f639415b830 2023-11-15T03:04:43.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.247+0000 7f6398879700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6394064a10 con 0x7f63940c2380 2023-11-15T03:04:43.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.247+0000 7f6398879700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6394064280 con 0x7f639415b450 2023-11-15T03:04:43.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.247+0000 7f6393fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f63940c2380 0x7f63940c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.247+0000 7f6393fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f63940c2380 0x7f63940c2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42536/0 (socket says 172.21.15.121:42536) 2023-11-15T03:04:43.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.247+0000 7f6393fff700 1 -- 172.21.15.121:0/3368982874 learned_addr learned my addr 172.21.15.121:0/3368982874 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:43.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.247+0000 7f63937fe700 1 --2- 172.21.15.121:0/3368982874 >> v2:172.21.15.190:3300/0 conn(0x7f639415b450 0x7f63940657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.251+0000 7f6393fff700 1 -- 172.21.15.121:0/3368982874 >> v2:172.21.15.190:3300/0 conn(0x7f639415b450 msgr2=0x7f63940657a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:43.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.251+0000 7f6393fff700 1 --2- 172.21.15.121:0/3368982874 >> v2:172.21.15.190:3300/0 conn(0x7f639415b450 0x7f63940657a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.251+0000 7f6393fff700 1 -- 172.21.15.121:0/3368982874 >> v1:172.21.15.190:6789/0 conn(0x7f639415b830 legacy=0x7f6394063aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:43.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.251+0000 7f6393fff700 1 -- 172.21.15.121:0/3368982874 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63940c1790 con 0x7f63940c2380 2023-11-15T03:04:43.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.251+0000 7f6393fff700 1 --2- 172.21.15.121:0/3368982874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63940c2380 0x7f63940c2830 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7f6394063790 tx=0x7f638c009840).ready entity=mon.0 client_cookie=8c2a3aff2067e607 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:43.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.251+0000 7f63927fc700 1 -- 172.21.15.121:0/3368982874 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c00e070 con 0x7f63940c2380 2023-11-15T03:04:43.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.251+0000 7f63927fc700 1 -- 172.21.15.121:0/3368982874 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638c00ba50 con 0x7f63940c2380 2023-11-15T03:04:43.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.251+0000 7f63927fc700 1 -- 172.21.15.121:0/3368982874 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c00bdd0 con 0x7f63940c2380 2023-11-15T03:04:43.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.251+0000 7f6398879700 1 -- 172.21.15.121:0/3368982874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63940c2380 msgr2=0x7f63940c2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:43.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.251+0000 7f6398879700 1 --2- 172.21.15.121:0/3368982874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63940c2380 0x7f63940c2830 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7f6394063790 tx=0x7f638c009840).stop 2023-11-15T03:04:43.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.251+0000 7f6398879700 1 -- 172.21.15.121:0/3368982874 shutdown_connections 2023-11-15T03:04:43.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.251+0000 7f6398879700 1 --2- 172.21.15.121:0/3368982874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63940c2380 0x7f63940c2830 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.251+0000 7f6398879700 1 --2- 172.21.15.121:0/3368982874 >> v2:172.21.15.190:3300/0 conn(0x7f639415b450 0x7f63940657a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.251+0000 7f6398879700 1 -- 172.21.15.121:0/3368982874 >> 172.21.15.121:0/3368982874 conn(0x7f63940677c0 msgr2=0x7f63940c65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:43.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.251+0000 7f6398879700 1 -- 172.21.15.121:0/3368982874 shutdown_connections 2023-11-15T03:04:43.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.251+0000 7f6398879700 1 -- 172.21.15.121:0/3368982874 wait complete. 2023-11-15T03:04:43.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f6398879700 1 Processor -- start 2023-11-15T03:04:43.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f6398879700 1 -- start start 2023-11-15T03:04:43.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f6398879700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63940c2380 0x7f63940aba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f6398879700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f639415b450 0x7f63940a8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f6398879700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f639415b830 0x7f639415a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f6398879700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63940ac400 con 0x7f639415b830 2023-11-15T03:04:43.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f6398879700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63940bd870 con 0x7f63940c2380 2023-11-15T03:04:43.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f6398879700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63940bdb50 con 0x7f639415b450 2023-11-15T03:04:43.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f6393fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f639415b830 0x7f639415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f6393fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f639415b830 0x7f639415a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42548/0 (socket says 172.21.15.121:42548) 2023-11-15T03:04:43.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f6393fff700 1 -- 172.21.15.121:0/428149451 learned_addr learned my addr 172.21.15.121:0/428149451 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:43.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f63937fe700 1 --2- 172.21.15.121:0/428149451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63940c2380 0x7f63940aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f6393fff700 1 -- 172.21.15.121:0/428149451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f639415b450 msgr2=0x7f63940a8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:43.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f6393fff700 1 --2- 172.21.15.121:0/428149451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f639415b450 0x7f63940a8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:04:43.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f6393fff700 1 -- 172.21.15.121:0/428149451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63940c2380 msgr2=0x7f63940aba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:43.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f6393fff700 1 --2- 172.21.15.121:0/428149451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63940c2380 0x7f63940aba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f6393fff700 1 -- 172.21.15.121:0/428149451 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c009ae0 con 0x7f639415b830 2023-11-15T03:04:43.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f6393fff700 1 --2- 172.21.15.121:0/428149451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f639415b830 0x7f639415a030 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7f638c009fd0 tx=0x7f638c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:43.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f637b7fe700 1 -- 172.21.15.121:0/428149451 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c009500 con 0x7f639415b830 2023-11-15T03:04:43.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f637b7fe700 1 -- 172.21.15.121:0/428149451 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638c0092e0 con 0x7f639415b830 2023-11-15T03:04:43.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f6398879700 1 -- 172.21.15.121:0/428149451 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63940bde30 con 0x7f639415b830 2023-11-15T03:04:43.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f637b7fe700 1 -- 172.21.15.121:0/428149451 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c01d620 con 0x7f639415b830 2023-11-15T03:04:43.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.255+0000 7f6398879700 1 -- 172.21.15.121:0/428149451 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6394157bc0 con 0x7f639415b830 2023-11-15T03:04:43.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.259+0000 7f6398879700 1 -- 172.21.15.121:0/428149451 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63940bd6d0 con 0x7f639415b830 2023-11-15T03:04:43.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.259+0000 7f637b7fe700 1 -- 172.21.15.121:0/428149451 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f638c01da50 con 0x7f639415b830 2023-11-15T03:04:43.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.259+0000 7f637b7fe700 1 --2- 172.21.15.121:0/428149451 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f638005a4e0 0x7f638005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:43.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.259+0000 7f637b7fe700 1 -- 172.21.15.121:0/428149451 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6782+0+0 (secure 0 0 0) 0x7f638c0765f0 con 0x7f639415b830 2023-11-15T03:04:43.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.259+0000 7f63937fe700 1 --2- 172.21.15.121:0/428149451 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f638005a4e0 0x7f638005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:43.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.259+0000 7f63937fe700 1 --2- 172.21.15.121:0/428149451 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f638005a4e0 0x7f638005c9a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f638400bb20 tx=0x7f6384008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:43.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.267+0000 7f637b7fe700 1 -- 172.21.15.121:0/428149451 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f63940bd6d0 con 0x7f639415b830 2023-11-15T03:04:43.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.459+0000 7f6398879700 1 -- 172.21.15.121:0/428149451 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f63940bd6d0 con 0x7f639415b830 2023-11-15T03:04:43.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.463+0000 7f637b7fe700 1 -- 172.21.15.121:0/428149451 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v123) v1 ==== 53+0+73 (secure 0 0 0) 0x7f63940bd6d0 con 0x7f639415b830 2023-11-15T03:04:43.470 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T03:04:43.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.463+0000 7f6398879700 1 -- 172.21.15.121:0/428149451 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f638005a4e0 msgr2=0x7f638005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:43.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.463+0000 7f6398879700 1 --2- 172.21.15.121:0/428149451 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f638005a4e0 0x7f638005c9a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f638400bb20 tx=0x7f6384008040).stop 2023-11-15T03:04:43.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.463+0000 7f6398879700 1 -- 172.21.15.121:0/428149451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f639415b830 msgr2=0x7f639415a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:43.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.463+0000 7f6398879700 1 --2- 172.21.15.121:0/428149451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f639415b830 0x7f639415a030 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7f638c009fd0 tx=0x7f638c009840).stop 2023-11-15T03:04:43.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.463+0000 7f6398879700 1 -- 172.21.15.121:0/428149451 shutdown_connections 2023-11-15T03:04:43.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.463+0000 7f6398879700 1 --2- 172.21.15.121:0/428149451 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f638005a4e0 0x7f638005c9a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.463+0000 7f6398879700 1 --2- 172.21.15.121:0/428149451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f639415b830 0x7f639415a030 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.463+0000 7f6398879700 1 --2- 172.21.15.121:0/428149451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f639415b450 0x7f63940a8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:04:43.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.463+0000 7f6398879700 1 --2- 172.21.15.121:0/428149451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63940c2380 0x7f63940aba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:43.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.463+0000 7f6398879700 1 -- 172.21.15.121:0/428149451 >> 172.21.15.121:0/428149451 conn(0x7f63940677c0 msgr2=0x7f63940c54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:43.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.463+0000 7f6398879700 1 -- 172.21.15.121:0/428149451 shutdown_connections 2023-11-15T03:04:43.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:43.463+0000 7f6398879700 1 -- 172.21.15.121:0/428149451 wait complete. 2023-11-15T03:04:43.490 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T03:04:43.491 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T03:04:43.491 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T03:04:43.491 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T03:04:43.492 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T03:04:43.492 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T03:04:43.493 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T03:04:43.493 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T03:04:43.494 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:04:43.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:43.494 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T03:04:43.515 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T03:04:43.516 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T03:04:43.516 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T03:04:43.517 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:04:43.517 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T03:04:43.518 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:04:43.518 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:04:43.518 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:04:43.518 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T03:04:43.519 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:04:43.519 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:04:43.519 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T03:04:43.519 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:04:43.520 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:04:43.520 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T03:04:43.520 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:04:43.520 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T03:04:43.520 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 41956sec preferred_lft 41956sec 2023-11-15T03:04:43.521 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T03:04:43.521 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:04:43.521 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T03:04:43.521 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:04:43.522 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T03:04:43.522 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:04:43.522 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T03:04:43.522 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T03:04:43.523 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T03:04:43.523 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:04:43.523 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T03:04:43.523 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:04:43.524 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:04:43.524 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T03:04:43.524 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T03:04:43.524 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:04:43.524 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:04:43.525 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T03:04:43.525 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T03:04:43.525 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:04:43.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:43.527 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T03:04:43.606 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T03:04:43.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:43.611 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T03:04:43.675 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T03:04:43.675 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T03:04:43.675 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T03:04:43.675 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T03:04:43.676 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T03:04:43.676 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T03:04:43.676 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T03:04:43.676 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T03:04:43.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:43.677 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T03:04:43.678 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T03:04:43.678 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T03:04:43.678 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T03:04:43.679 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T03:04:43.679 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T03:04:43.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:43.821 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T03:04:43.842 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T03:04:43.842 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T03:04:43.843 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:04:43.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:43.844 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T03:04:43.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:43.927 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T03:04:43.991 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T03:04:43.992 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:04:44.115 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:04:44.116 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:04:44.140 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:04:44.140 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:04:44.145 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T03:04:44.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:44.146 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T03:04:44.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:04:44.180 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2023-11-15T03:04:44.231 INFO:teuthology.orchestra.run:waiting for 20 2023-11-15T03:04:44.247 INFO:teuthology.orchestra.run.smithi121.stdout:.: 2023-11-15T03:04:44.248 INFO:teuthology.orchestra.run.smithi121.stdout:subdir 2023-11-15T03:04:44.248 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:04:44.248 INFO:teuthology.orchestra.run.smithi121.stdout:./subdir: 2023-11-15T03:04:44.248 INFO:teuthology.orchestra.run.smithi121.stdout:sixmegs 2023-11-15T03:04:44.249 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_openfiles.0' did not caused DAMAGED state 2023-11-15T03:04:44.250 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 500.00000000' 2023-11-15T03:04:44.251 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T03:04:44.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:04:44.252 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-15T03:04:44.333 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:04:44.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:44.334 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:04:44.395 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:04:44.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.595+0000 7f0e574e4700 1 Processor -- start 2023-11-15T03:04:44.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.595+0000 7f0e574e4700 1 -- start start 2023-11-15T03:04:44.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.599+0000 7f0e574e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e500bf210 0x7f0e500643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:44.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.599+0000 7f0e574e4700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e500bed80 con 0x7f0e50064890 2023-11-15T03:04:44.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.599+0000 7f0e574e4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e50064d20 con 0x7f0e50154e40 2023-11-15T03:04:44.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.599+0000 7f0e574e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e500be170 con 0x7f0e500bf210 2023-11-15T03:04:44.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.599+0000 7f0e56ce3700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f0e50064890 0x7f0e5015b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:55636/0 (socket says 172.21.15.121:55636) 2023-11-15T03:04:44.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.599+0000 7f0e56ce3700 1 -- 172.21.15.121:0/2020066953 learned_addr learned my addr 172.21.15.121:0/2020066953 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:44.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.599+0000 7f0e564e2700 1 --1- 172.21.15.121:0/2020066953 >> v1:172.21.15.190:6789/0 conn(0x7f0e50154e40 0x7f0e500c3770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:50078/0 (socket says 172.21.15.121:50078) 2023-11-15T03:04:44.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.599+0000 7f0e55ce1700 1 --2- 172.21.15.121:0/2020066953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e500bf210 0x7f0e500643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:44.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.599+0000 7f0e554e0700 1 -- 172.21.15.121:0/2020066953 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1010382002 0 0) 0x7f0e50064d20 con 0x7f0e50154e40 2023-11-15T03:04:44.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.599+0000 7f0e554e0700 1 -- 172.21.15.121:0/2020066953 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e3c0033f0 con 0x7f0e50154e40 2023-11-15T03:04:44.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.599+0000 7f0e554e0700 1 -- 172.21.15.121:0/2020066953 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2994599803 0 0) 0x7f0e500bed80 con 0x7f0e50064890 2023-11-15T03:04:44.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.599+0000 7f0e554e0700 1 -- 172.21.15.121:0/2020066953 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e50064d20 con 0x7f0e50064890 2023-11-15T03:04:44.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.599+0000 7f0e55ce1700 1 -- 172.21.15.121:0/2020066953 >> v1:172.21.15.190:6789/0 conn(0x7f0e50154e40 legacy=0x7f0e500c3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:44.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.599+0000 7f0e55ce1700 1 -- 172.21.15.121:0/2020066953 >> v1:172.21.15.121:6789/0 conn(0x7f0e50064890 legacy=0x7f0e5015b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:44.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.599+0000 7f0e55ce1700 1 -- 172.21.15.121:0/2020066953 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e500be450 con 0x7f0e500bf210 2023-11-15T03:04:44.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.599+0000 7f0e55ce1700 1 --2- 172.21.15.121:0/2020066953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e500bf210 0x7f0e500643a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f0e40009790 tx=0x7f0e40009060).ready entity=mon.2 client_cookie=d2b8c8d278312cfc server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:44.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.603+0000 7f0e554e0700 1 -- 172.21.15.121:0/2020066953 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4000f510 con 0x7f0e500bf210 2023-11-15T03:04:44.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.603+0000 7f0e554e0700 1 -- 172.21.15.121:0/2020066953 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e4000f670 con 0x7f0e500bf210 2023-11-15T03:04:44.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.603+0000 7f0e554e0700 1 -- 172.21.15.121:0/2020066953 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4000f9f0 con 0x7f0e500bf210 2023-11-15T03:04:44.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.603+0000 7f0e574e4700 1 -- 172.21.15.121:0/2020066953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e500bf210 msgr2=0x7f0e500643a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:44.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.603+0000 7f0e574e4700 1 --2- 172.21.15.121:0/2020066953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e500bf210 0x7f0e500643a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f0e40009790 tx=0x7f0e40009060).stop 2023-11-15T03:04:44.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.603+0000 7f0e574e4700 1 -- 172.21.15.121:0/2020066953 shutdown_connections 2023-11-15T03:04:44.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.603+0000 7f0e574e4700 1 --2- 172.21.15.121:0/2020066953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e500bf210 0x7f0e500643a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:44.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.603+0000 7f0e574e4700 1 -- 172.21.15.121:0/2020066953 >> 172.21.15.121:0/2020066953 conn(0x7f0e500684e0 msgr2=0x7f0e500c8760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:44.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.603+0000 7f0e574e4700 1 -- 172.21.15.121:0/2020066953 shutdown_connections 2023-11-15T03:04:44.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.603+0000 7f0e574e4700 1 -- 172.21.15.121:0/2020066953 wait complete. 2023-11-15T03:04:44.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.607+0000 7f0e574e4700 1 Processor -- start 2023-11-15T03:04:44.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.607+0000 7f0e574e4700 1 -- start start 2023-11-15T03:04:44.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.607+0000 7f0e574e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e50064890 0x7f0e500c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:44.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.607+0000 7f0e574e4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e500bf210 0x7f0e500c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:44.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.607+0000 7f0e574e4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e50154e40 0x7f0e500c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:44.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.607+0000 7f0e574e4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e500c11f0 con 0x7f0e50154e40 2023-11-15T03:04:44.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.607+0000 7f0e574e4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e500c1460 con 0x7f0e500bf210 2023-11-15T03:04:44.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.607+0000 7f0e574e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e500c1740 con 0x7f0e50064890 2023-11-15T03:04:44.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.607+0000 7f0e56ce3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e50154e40 0x7f0e500c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:44.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.607+0000 7f0e564e2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e50064890 0x7f0e500c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:44.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.607+0000 7f0e55ce1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e500bf210 0x7f0e500c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:44.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.607+0000 7f0e56ce3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e50154e40 0x7f0e500c8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42564/0 (socket says 172.21.15.121:42564) 2023-11-15T03:04:44.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.607+0000 7f0e56ce3700 1 -- 172.21.15.121:0/1262434319 learned_addr learned my addr 172.21.15.121:0/1262434319 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:44.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.611+0000 7f0e56ce3700 1 -- 172.21.15.121:0/1262434319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e50064890 msgr2=0x7f0e500c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:44.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.611+0000 7f0e56ce3700 1 --2- 172.21.15.121:0/1262434319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e50064890 0x7f0e500c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:44.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.611+0000 7f0e56ce3700 1 -- 172.21.15.121:0/1262434319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e500bf210 msgr2=0x7f0e500c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:44.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.611+0000 7f0e56ce3700 1 --2- 172.21.15.121:0/1262434319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e500bf210 0x7f0e500c7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:44.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.611+0000 7f0e56ce3700 1 -- 172.21.15.121:0/1262434319 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e400092a0 con 0x7f0e50154e40 2023-11-15T03:04:44.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.611+0000 7f0e56ce3700 1 --2- 172.21.15.121:0/1262434319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e50154e40 0x7f0e500c8850 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7f0e48005ad0 tx=0x7f0e48001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:44.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.611+0000 7f0e467fc700 1 -- 172.21.15.121:0/1262434319 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48005d30 con 0x7f0e50154e40 2023-11-15T03:04:44.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.611+0000 7f0e467fc700 1 -- 172.21.15.121:0/1262434319 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e48005e90 con 0x7f0e50154e40 2023-11-15T03:04:44.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.611+0000 7f0e574e4700 1 -- 172.21.15.121:0/1262434319 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e500c1a20 con 0x7f0e50154e40 2023-11-15T03:04:44.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.611+0000 7f0e467fc700 1 -- 172.21.15.121:0/1262434319 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48007320 con 0x7f0e50154e40 2023-11-15T03:04:44.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.611+0000 7f0e574e4700 1 -- 172.21.15.121:0/1262434319 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e500c5710 con 0x7f0e50154e40 2023-11-15T03:04:44.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.611+0000 7f0e574e4700 1 -- 172.21.15.121:0/1262434319 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e5006bcf0 con 0x7f0e50154e40 2023-11-15T03:04:44.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.615+0000 7f0e467fc700 1 -- 172.21.15.121:0/1262434319 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0e48007710 con 0x7f0e50154e40 2023-11-15T03:04:44.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.615+0000 7f0e467fc700 1 --2- 172.21.15.121:0/1262434319 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e3405a510 0x7f0e3405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:44.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.615+0000 7f0e467fc700 1 -- 172.21.15.121:0/1262434319 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6782+0+0 (secure 0 0 0) 0x7f0e48074a60 con 0x7f0e50154e40 2023-11-15T03:04:44.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.615+0000 7f0e564e2700 1 --2- 172.21.15.121:0/1262434319 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e3405a510 0x7f0e3405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:44.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.619+0000 7f0e564e2700 1 --2- 172.21.15.121:0/1262434319 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e3405a510 0x7f0e3405c9d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f0e4c0076c0 tx=0x7f0e4c007be0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:44.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.623+0000 7f0e467fc700 1 -- 172.21.15.121:0/1262434319 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0e48047ef0 con 0x7f0e50154e40 2023-11-15T03:04:44.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:44.815+0000 7f0e574e4700 1 -- 172.21.15.121:0/1262434319 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0e50068a70 con 0x7f0e50154e40 2023-11-15T03:04:45.169 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:04:45.159+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:04:45.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.171+0000 7f0e467fc700 1 -- 172.21.15.121:0/1262434319 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v124) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0e4804b510 con 0x7f0e50154e40 2023-11-15T03:04:45.182 INFO:tasks.ceph.mds.a.smithi121.stderr: -1425> 2023-11-15T03:04:36.195+0000 7febfb572700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-11-15T03:04:45.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.175+0000 7f0e574e4700 1 -- 172.21.15.121:0/1262434319 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e3405a510 msgr2=0x7f0e3405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:45.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.175+0000 7f0e574e4700 1 --2- 172.21.15.121:0/1262434319 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e3405a510 0x7f0e3405c9d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f0e4c0076c0 tx=0x7f0e4c007be0).stop 2023-11-15T03:04:45.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.175+0000 7f0e574e4700 1 -- 172.21.15.121:0/1262434319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e50154e40 msgr2=0x7f0e500c8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:45.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.175+0000 7f0e574e4700 1 --2- 172.21.15.121:0/1262434319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e50154e40 0x7f0e500c8850 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7f0e48005ad0 tx=0x7f0e48001650).stop 2023-11-15T03:04:45.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.175+0000 7f0e574e4700 1 -- 172.21.15.121:0/1262434319 shutdown_connections 2023-11-15T03:04:45.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.175+0000 7f0e574e4700 1 --2- 172.21.15.121:0/1262434319 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e3405a510 0x7f0e3405c9d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:45.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.175+0000 7f0e574e4700 1 --2- 172.21.15.121:0/1262434319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e50154e40 0x7f0e500c8850 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:45.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.175+0000 7f0e574e4700 1 --2- 172.21.15.121:0/1262434319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e500bf210 0x7f0e500c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:45.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.175+0000 7f0e574e4700 1 --2- 172.21.15.121:0/1262434319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e50064890 0x7f0e500c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:45.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.175+0000 7f0e574e4700 1 -- 172.21.15.121:0/1262434319 >> 172.21.15.121:0/1262434319 conn(0x7f0e500684e0 msgr2=0x7f0e500a9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:45.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.175+0000 7f0e574e4700 1 -- 172.21.15.121:0/1262434319 shutdown_connections 2023-11-15T03:04:45.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.175+0000 7f0e574e4700 1 -- 172.21.15.121:0/1262434319 wait complete. 2023-11-15T03:04:45.186 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:04:45.188 INFO:tasks.ceph.mds.a.smithi121.stderr: -417> 2023-11-15T03:04:42.127+0000 7febfed79700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:04:45.204 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:04:45.230 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:04:45.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.367+0000 7f3d2359e700 1 Processor -- start 2023-11-15T03:04:45.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.367+0000 7f3d2359e700 1 -- start start 2023-11-15T03:04:45.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.371+0000 7f3d2359e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3d240c45d0 0x7f3d240c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:45.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.371+0000 7f3d2359e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3d240c8e70 0x7f3d240c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:45.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.371+0000 7f3d2359e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d240aaa50 con 0x7f3d240c3250 2023-11-15T03:04:45.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.371+0000 7f3d2359e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d240bdb40 con 0x7f3d240c8e70 2023-11-15T03:04:45.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.371+0000 7f3d2359e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d240aa490 con 0x7f3d240c45d0 2023-11-15T03:04:45.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.371+0000 7f3d21d9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3d240c8e70 0x7f3d240c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:45.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.371+0000 7f3d21d9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3d240c8e70 0x7f3d240c2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33780/0 (socket says 172.21.15.121:33780) 2023-11-15T03:04:45.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.371+0000 7f3d21d9b700 1 -- 172.21.15.121:0/3939987173 learned_addr learned my addr 172.21.15.121:0/3939987173 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:45.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.371+0000 7f3d2259c700 1 --2- 172.21.15.121:0/3939987173 >> v2:172.21.15.190:3300/0 conn(0x7f3d240c45d0 0x7f3d240c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:45.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.371+0000 7f3d21d9b700 1 -- 172.21.15.121:0/3939987173 >> v2:172.21.15.190:3300/0 conn(0x7f3d240c45d0 msgr2=0x7f3d240c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:45.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.371+0000 7f3d21d9b700 1 --2- 172.21.15.121:0/3939987173 >> v2:172.21.15.190:3300/0 conn(0x7f3d240c45d0 0x7f3d240c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:45.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.371+0000 7f3d21d9b700 1 -- 172.21.15.121:0/3939987173 >> v1:172.21.15.190:6789/0 conn(0x7f3d240c3250 legacy=0x7f3d240c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:45.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.371+0000 7f3d21d9b700 1 -- 172.21.15.121:0/3939987173 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d240aa770 con 0x7f3d240c8e70 2023-11-15T03:04:45.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.371+0000 7f3d21d9b700 1 --2- 172.21.15.121:0/3939987173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d240c8e70 0x7f3d240c2d60 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f3d24068a40 tx=0x7f3d180090f0).ready entity=mon.0 client_cookie=afb645f690801e2e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:45.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.371+0000 7f3d2159a700 1 -- 172.21.15.121:0/3939987173 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1800b600 con 0x7f3d240c8e70 2023-11-15T03:04:45.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.371+0000 7f3d2159a700 1 -- 172.21.15.121:0/3939987173 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1800b760 con 0x7f3d240c8e70 2023-11-15T03:04:45.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.371+0000 7f3d2159a700 1 -- 172.21.15.121:0/3939987173 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1800bb00 con 0x7f3d240c8e70 2023-11-15T03:04:45.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.375+0000 7f3d2359e700 1 -- 172.21.15.121:0/3939987173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d240c8e70 msgr2=0x7f3d240c2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:45.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.375+0000 7f3d2359e700 1 --2- 172.21.15.121:0/3939987173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d240c8e70 0x7f3d240c2d60 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f3d24068a40 tx=0x7f3d180090f0).stop 2023-11-15T03:04:45.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.375+0000 7f3d2359e700 1 -- 172.21.15.121:0/3939987173 shutdown_connections 2023-11-15T03:04:45.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.375+0000 7f3d2359e700 1 --2- 172.21.15.121:0/3939987173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d240c8e70 0x7f3d240c2d60 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:45.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.375+0000 7f3d2359e700 1 --2- 172.21.15.121:0/3939987173 >> v2:172.21.15.190:3300/0 conn(0x7f3d240c45d0 0x7f3d240c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:45.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.375+0000 7f3d2359e700 1 -- 172.21.15.121:0/3939987173 >> 172.21.15.121:0/3939987173 conn(0x7f3d240684e0 msgr2=0x7f3d240c8760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:45.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.375+0000 7f3d2359e700 1 -- 172.21.15.121:0/3939987173 shutdown_connections 2023-11-15T03:04:45.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.375+0000 7f3d2359e700 1 -- 172.21.15.121:0/3939987173 wait complete. 2023-11-15T03:04:45.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.375+0000 7f3d2359e700 1 Processor -- start 2023-11-15T03:04:45.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.375+0000 7f3d2359e700 1 -- start start 2023-11-15T03:04:45.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d2359e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d240c3250 0x7f3d2414b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:45.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d2359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d240c45d0 0x7f3d240a6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:45.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d2359e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d240c8e70 0x7f3d240a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:45.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d2359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2414b860 con 0x7f3d240c45d0 2023-11-15T03:04:45.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d2359e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d240a7ea0 con 0x7f3d240c3250 2023-11-15T03:04:45.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d2359e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d240a7290 con 0x7f3d240c8e70 2023-11-15T03:04:45.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d21d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d240c45d0 0x7f3d240a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:45.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d2259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d240c3250 0x7f3d2414b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:45.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d21d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d240c45d0 0x7f3d240a6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33786/0 (socket says 172.21.15.121:33786) 2023-11-15T03:04:45.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d21d9b700 1 -- 172.21.15.121:0/2072071081 learned_addr learned my addr 172.21.15.121:0/2072071081 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:45.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d22d9d700 1 --2- 172.21.15.121:0/2072071081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d240c8e70 0x7f3d240a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:45.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d21d9b700 1 -- 172.21.15.121:0/2072071081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d240c8e70 msgr2=0x7f3d240a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:45.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d21d9b700 1 --2- 172.21.15.121:0/2072071081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d240c8e70 0x7f3d240a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:45.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d21d9b700 1 -- 172.21.15.121:0/2072071081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d240c3250 msgr2=0x7f3d2414b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:45.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d21d9b700 1 --2- 172.21.15.121:0/2072071081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d240c3250 0x7f3d2414b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:45.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d21d9b700 1 -- 172.21.15.121:0/2072071081 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d18009370 con 0x7f3d240c45d0 2023-11-15T03:04:45.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d21d9b700 1 --2- 172.21.15.121:0/2072071081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d240c45d0 0x7f3d240a6840 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0x7f3d24066d70 tx=0x7f3d18017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:45.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d127fc700 1 -- 172.21.15.121:0/2072071081 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18006d90 con 0x7f3d240c45d0 2023-11-15T03:04:45.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d127fc700 1 -- 172.21.15.121:0/2072071081 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1800bec0 con 0x7f3d240c45d0 2023-11-15T03:04:45.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d2359e700 1 -- 172.21.15.121:0/2072071081 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d240a7570 con 0x7f3d240c45d0 2023-11-15T03:04:45.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d127fc700 1 -- 172.21.15.121:0/2072071081 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18024470 con 0x7f3d240c45d0 2023-11-15T03:04:45.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.379+0000 7f3d2359e700 1 -- 172.21.15.121:0/2072071081 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d240a7b30 con 0x7f3d240c45d0 2023-11-15T03:04:45.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.383+0000 7f3d2359e700 1 -- 172.21.15.121:0/2072071081 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d240a8050 con 0x7f3d240c45d0 2023-11-15T03:04:45.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.383+0000 7f3d127fc700 1 -- 172.21.15.121:0/2072071081 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3d18021070 con 0x7f3d240c45d0 2023-11-15T03:04:45.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.383+0000 7f3d127fc700 1 --2- 172.21.15.121:0/2072071081 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3d0405a540 0x7f3d0405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:45.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.387+0000 7f3d127fc700 1 -- 172.21.15.121:0/2072071081 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6868+0+0 (secure 0 0 0) 0x7f3d180765c0 con 0x7f3d240c45d0 2023-11-15T03:04:45.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.387+0000 7f3d2259c700 1 --2- 172.21.15.121:0/2072071081 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3d0405a540 0x7f3d0405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:45.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.387+0000 7f3d2259c700 1 --2- 172.21.15.121:0/2072071081 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3d0405a540 0x7f3d0405ca00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f3d1400bb10 tx=0x7f3d14008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:45.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.391+0000 7f3d127fc700 1 -- 172.21.15.121:0/2072071081 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3d240a8050 con 0x7f3d240c45d0 2023-11-15T03:04:45.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:45.607+0000 7f3d2359e700 1 -- 172.21.15.121:0/2072071081 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f3d240a8010 con 0x7f3d240c45d0 2023-11-15T03:04:46.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.191+0000 7f3d127fc700 1 -- 172.21.15.121:0/2072071081 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v125) v1 ==== 107+0+0 (secure 0 0 0) 0x7f3d18049ab0 con 0x7f3d240c45d0 2023-11-15T03:04:46.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.195+0000 7f3d2359e700 1 -- 172.21.15.121:0/2072071081 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3d0405a540 msgr2=0x7f3d0405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.195+0000 7f3d2359e700 1 --2- 172.21.15.121:0/2072071081 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3d0405a540 0x7f3d0405ca00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f3d1400bb10 tx=0x7f3d14008040).stop 2023-11-15T03:04:46.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.195+0000 7f3d2359e700 1 -- 172.21.15.121:0/2072071081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d240c45d0 msgr2=0x7f3d240a6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.195+0000 7f3d2359e700 1 --2- 172.21.15.121:0/2072071081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d240c45d0 0x7f3d240a6840 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0x7f3d24066d70 tx=0x7f3d18017000).stop 2023-11-15T03:04:46.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.195+0000 7f3d2359e700 1 -- 172.21.15.121:0/2072071081 shutdown_connections 2023-11-15T03:04:46.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.195+0000 7f3d2359e700 1 --2- 172.21.15.121:0/2072071081 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3d0405a540 0x7f3d0405ca00 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.195+0000 7f3d2359e700 1 --2- 172.21.15.121:0/2072071081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d240c8e70 0x7f3d240a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.195+0000 7f3d2359e700 1 --2- 172.21.15.121:0/2072071081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d240c45d0 0x7f3d240a6840 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.195+0000 7f3d2359e700 1 --2- 172.21.15.121:0/2072071081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d240c3250 0x7f3d2414b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.195+0000 7f3d2359e700 1 -- 172.21.15.121:0/2072071081 >> 172.21.15.121:0/2072071081 conn(0x7f3d240684e0 msgr2=0x7f3d240c7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:46.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.195+0000 7f3d2359e700 1 -- 172.21.15.121:0/2072071081 shutdown_connections 2023-11-15T03:04:46.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.195+0000 7f3d2359e700 1 -- 172.21.15.121:0/2072071081 wait complete. 2023-11-15T03:04:46.206 INFO:teuthology.orchestra.run.smithi121.stderr:nothing to do: rank is not damaged 2023-11-15T03:04:46.228 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:04:46.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.327+0000 7fadb897bd00 1 Processor -- start 2023-11-15T03:04:46.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.327+0000 7fadb897bd00 1 -- start start 2023-11-15T03:04:46.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.327+0000 7fadb897bd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af65a3c90 0x562af6585500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.327+0000 7fadb897bd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562af662fec0 0x562af65a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.327+0000 7fadb897bd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562af66302a0 0x562af65a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.327+0000 7fadb897bd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af64d0220 con 0x562af65a3c90 2023-11-15T03:04:46.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.327+0000 7fadb897bd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af6586ab0 con 0x562af66302a0 2023-11-15T03:04:46.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.327+0000 7fadb897bd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x562af6589ff0 con 0x562af662fec0 2023-11-15T03:04:46.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.327+0000 7fadb62b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af65a3c90 0x562af6585500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.327+0000 7fadb62b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af65a3c90 0x562af6585500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33808/0 (socket says 172.21.15.121:33808) 2023-11-15T03:04:46.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.327+0000 7fadb62b7700 1 -- 172.21.15.121:0/2294222536 learned_addr learned my addr 172.21.15.121:0/2294222536 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:46.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.327+0000 7fadadab6700 1 --2- 172.21.15.121:0/2294222536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562af662fec0 0x562af65a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.327+0000 7fadb6ab8700 1 --2- 172.21.15.121:0/2294222536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562af66302a0 0x562af65a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb62b7700 1 -- 172.21.15.121:0/2294222536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562af662fec0 msgr2=0x562af65a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:46.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb62b7700 1 --2- 172.21.15.121:0/2294222536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562af662fec0 0x562af65a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb62b7700 1 -- 172.21.15.121:0/2294222536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562af66302a0 msgr2=0x562af65a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:46.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb62b7700 1 --2- 172.21.15.121:0/2294222536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562af66302a0 0x562af65a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb62b7700 1 -- 172.21.15.121:0/2294222536 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada0005cf0 con 0x562af65a3c90 2023-11-15T03:04:46.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadadab6700 1 --2- 172.21.15.121:0/2294222536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562af662fec0 0x562af65a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:46.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb62b7700 1 --2- 172.21.15.121:0/2294222536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af65a3c90 0x562af6585500 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7fada8001d30 tx=0x7fada8001810).ready entity=mon.0 client_cookie=8506e6097bb7af3e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:46.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb52b5700 1 -- 172.21.15.121:0/2294222536 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada800bcc0 con 0x562af65a3c90 2023-11-15T03:04:46.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb52b5700 1 -- 172.21.15.121:0/2294222536 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada800be20 con 0x562af65a3c90 2023-11-15T03:04:46.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb52b5700 1 -- 172.21.15.121:0/2294222536 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada800e6a0 con 0x562af65a3c90 2023-11-15T03:04:46.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb897bd00 1 -- 172.21.15.121:0/2294222536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af65a3c90 msgr2=0x562af6585500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2294222536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af65a3c90 0x562af6585500 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7fada8001d30 tx=0x7fada8001810).stop 2023-11-15T03:04:46.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb897bd00 1 -- 172.21.15.121:0/2294222536 shutdown_connections 2023-11-15T03:04:46.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2294222536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562af66302a0 0x562af65a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2294222536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562af662fec0 0x562af65a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2294222536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af65a3c90 0x562af6585500 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb897bd00 1 -- 172.21.15.121:0/2294222536 >> 172.21.15.121:0/2294222536 conn(0x562af65587d0 msgr2=0x562af65a2010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:46.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb897bd00 1 -- 172.21.15.121:0/2294222536 shutdown_connections 2023-11-15T03:04:46.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb897bd00 1 -- 172.21.15.121:0/2294222536 wait complete. 2023-11-15T03:04:46.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb897bd00 1 Processor -- start 2023-11-15T03:04:46.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.331+0000 7fadb897bd00 1 -- start start 2023-11-15T03:04:46.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadb897bd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562af65a3c90 0x562af6634050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadb897bd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af662fec0 0x562af6625970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadb897bd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562af66302a0 0x562af6625e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadb897bd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af65a2da0 con 0x562af662fec0 2023-11-15T03:04:46.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadb897bd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af659e5a0 con 0x562af66302a0 2023-11-15T03:04:46.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadb897bd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x562af659f2c0 con 0x562af65a3c90 2023-11-15T03:04:46.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadadab6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af662fec0 0x562af6625970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadadab6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af662fec0 0x562af6625970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33818/0 (socket says 172.21.15.121:33818) 2023-11-15T03:04:46.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 learned_addr learned my addr 172.21.15.121:0/2506771385 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:46.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadb62b7700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562af65a3c90 0x562af6634050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadb6ab8700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562af66302a0 0x562af6625e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562af65a3c90 msgr2=0x562af6634050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadadab6700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562af65a3c90 0x562af6634050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562af66302a0 msgr2=0x562af6625e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadadab6700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562af66302a0 0x562af6625e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada4001e10 con 0x562af662fec0 2023-11-15T03:04:46.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadadab6700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af662fec0 0x562af6625970 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x562af6559790 tx=0x7fada0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:46.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadaf7fe700 1 -- 172.21.15.121:0/2506771385 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0007610 con 0x562af662fec0 2023-11-15T03:04:46.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadaf7fe700 1 -- 172.21.15.121:0/2506771385 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada001b040 con 0x562af662fec0 2023-11-15T03:04:46.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562af64f9770 con 0x562af662fec0 2023-11-15T03:04:46.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadaf7fe700 1 -- 172.21.15.121:0/2506771385 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada000aea0 con 0x562af662fec0 2023-11-15T03:04:46.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.335+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562af64c8a70 con 0x562af662fec0 2023-11-15T03:04:46.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.339+0000 7fadaf7fe700 1 -- 172.21.15.121:0/2506771385 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fada0029440 con 0x562af662fec0 2023-11-15T03:04:46.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.339+0000 7fadaf7fe700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fad98059020 0x7fad9805b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.339+0000 7fadaf7fe700 1 -- 172.21.15.121:0/2506771385 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6868+0+0 (secure 0 0 0) 0x7fada0074980 con 0x562af662fec0 2023-11-15T03:04:46.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.339+0000 7fadb62b7700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fad98059020 0x7fad9805b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.373 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:04:46.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.339+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x562af65a0080 0x562af659dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.339+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6634540 con 0x562af65a0080 2023-11-15T03:04:46.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.343+0000 7fadb6ab8700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x562af65a0080 0x562af659dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.343+0000 7fadb62b7700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fad98059020 0x7fad9805b4e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7fada800c010 tx=0x7fada8014000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:46.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.343+0000 7fadb6ab8700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x562af65a0080 0x562af659dcf0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:46.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.343+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v101'77 uv70 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af65a0080 2023-11-15T03:04:46.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.343+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af664d700 con 0x562af65a0080 2023-11-15T03:04:46.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.347+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v101'78 uv78 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af65a0080 2023-11-15T03:04:46.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.347+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af664dab0 con 0x562af65a0080 2023-11-15T03:04:46.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.347+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v101'79 uv79 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af65a0080 2023-11-15T03:04:46.377 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:04:46.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.347+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af664ee70 con 0x562af65a0080 2023-11-15T03:04:46.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.351+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v101'80 uv80 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af65a0080 2023-11-15T03:04:46.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.351+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af664f220 con 0x562af65a0080 2023-11-15T03:04:46.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.355+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v101'81 uv81 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af65a0080 2023-11-15T03:04:46.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.355+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af664de60 con 0x562af65a0080 2023-11-15T03:04:46.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.355+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v101'82 uv82 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af65a0080 2023-11-15T03:04:46.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.355+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af664e260 con 0x562af65a0080 2023-11-15T03:04:46.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.359+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v101'83 uv76 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af65a0080 2023-11-15T03:04:46.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.359+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af664e610 con 0x562af65a0080 2023-11-15T03:04:46.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.363+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v101'84 uv84 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af65a0080 2023-11-15T03:04:46.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.363+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af664e610 con 0x562af65a0080 2023-11-15T03:04:46.381 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:04:46.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.363+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v101'85 uv85 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af65a0080 2023-11-15T03:04:46.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.363+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af664e610 con 0x562af65a0080 2023-11-15T03:04:46.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.367+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v101'86 uv86 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af65a0080 2023-11-15T03:04:46.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.367+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af664e610 con 0x562af65a0080 2023-11-15T03:04:46.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.371+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v101'87 uv87 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af65a0080 2023-11-15T03:04:46.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.371+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af664e610 con 0x562af65a0080 2023-11-15T03:04:46.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.371+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v101'88 uv88 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af65a0080 2023-11-15T03:04:46.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.371+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x562af664f5d0 0x562af66519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.371+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6651eb0 con 0x562af664f5d0 2023-11-15T03:04:46.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.375+0000 7fadb62b7700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x562af664f5d0 0x562af66519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.375+0000 7fadb62b7700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x562af664f5d0 0x562af66519c0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:46.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.379+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v101'119 uv106 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.379+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af6652260 con 0x562af664f5d0 2023-11-15T03:04:46.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.383+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v101'120 uv120 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.383+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6652610 con 0x562af664f5d0 2023-11-15T03:04:46.399 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:04:46.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.383+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v101'121 uv121 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.383+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66539d0 con 0x562af664f5d0 2023-11-15T03:04:46.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.387+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v101'122 uv122 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.387+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6653d80 con 0x562af664f5d0 2023-11-15T03:04:46.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.391+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v101'123 uv123 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.391+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66529c0 con 0x562af664f5d0 2023-11-15T03:04:46.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.391+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v101'124 uv124 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.395+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6652dc0 con 0x562af664f5d0 2023-11-15T03:04:46.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.395+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v101'125 uv125 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.395+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66531c0 con 0x562af664f5d0 2023-11-15T03:04:46.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.399+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v101'126 uv112 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.399+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af66531c0 con 0x562af664f5d0 2023-11-15T03:04:46.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.403+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v101'127 uv127 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.403+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6653170 con 0x562af664f5d0 2023-11-15T03:04:46.414 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:04:46.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.403+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v101'128 uv128 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.403+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6653170 con 0x562af664f5d0 2023-11-15T03:04:46.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.407+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v101'129 uv129 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.407+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6653170 con 0x562af664f5d0 2023-11-15T03:04:46.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.411+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v101'130 uv130 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.411+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6653170 con 0x562af664f5d0 2023-11-15T03:04:46.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.411+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v101'131 uv131 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.411+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6653170 con 0x562af664f5d0 2023-11-15T03:04:46.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.415+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v101'132 uv118 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.415+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af6653170 con 0x562af664f5d0 2023-11-15T03:04:46.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.419+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v101'133 uv133 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.419+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6653170 con 0x562af664f5d0 2023-11-15T03:04:46.433 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:04:46.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.419+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v101'134 uv134 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.419+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6653170 con 0x562af664f5d0 2023-11-15T03:04:46.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.423+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v101'135 uv135 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.423+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6653170 con 0x562af664f5d0 2023-11-15T03:04:46.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.427+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v101'136 uv136 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.427+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6653170 con 0x562af664f5d0 2023-11-15T03:04:46.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.427+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v101'137 uv137 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.427+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x562af6653570 0x562af6656650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.427+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6653170 con 0x562af6653570 2023-11-15T03:04:46.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.427+0000 7fadadab6700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x562af6653570 0x562af6656650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.431+0000 7fadadab6700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x562af6653570 0x562af6656650 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:46.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.431+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v101'66 uv59 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af6653570 2023-11-15T03:04:46.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.431+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af6656b40 con 0x562af6653570 2023-11-15T03:04:46.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.435+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v101'67 uv67 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af6653570 2023-11-15T03:04:46.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.435+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6656ef0 con 0x562af6653570 2023-11-15T03:04:46.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.439+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v101'68 uv68 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af6653570 2023-11-15T03:04:46.448 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:04:46.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.439+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66572a0 con 0x562af6653570 2023-11-15T03:04:46.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.443+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v101'69 uv69 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af6653570 2023-11-15T03:04:46.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.443+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af659d3f0 con 0x562af6653570 2023-11-15T03:04:46.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.443+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v101'70 uv70 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af6653570 2023-11-15T03:04:46.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.443+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6657bc0 con 0x562af6653570 2023-11-15T03:04:46.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.447+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v101'71 uv65 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af6653570 2023-11-15T03:04:46.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.447+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af6657f70 con 0x562af6653570 2023-11-15T03:04:46.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.451+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v101'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af6653570 2023-11-15T03:04:46.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.451+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6657f70 con 0x562af6653570 2023-11-15T03:04:46.461 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:04:46.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.451+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v101'73 uv73 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af6653570 2023-11-15T03:04:46.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.451+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6657f70 con 0x562af6653570 2023-11-15T03:04:46.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.455+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v101'74 uv74 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af6653570 2023-11-15T03:04:46.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.455+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6657f70 con 0x562af6653570 2023-11-15T03:04:46.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.459+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v101'75 uv75 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af6653570 2023-11-15T03:04:46.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.459+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x562af6658480 0x562af665a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.459+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af665ae40 con 0x562af6658480 2023-11-15T03:04:46.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.459+0000 7fadb6ab8700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x562af6658480 0x562af665a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.459+0000 7fadb6ab8700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x562af6658480 0x562af665a870 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:46.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.463+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v101'86 uv85 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.463+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af665b220 con 0x562af6658480 2023-11-15T03:04:46.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.467+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v101'87 uv87 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.467+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af665b220 con 0x562af6658480 2023-11-15T03:04:46.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.467+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v101'88 uv88 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.477 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:04:46.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.467+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af665f2b0 con 0x562af6658480 2023-11-15T03:04:46.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.471+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v101'89 uv89 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.471+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af665f2b0 con 0x562af6658480 2023-11-15T03:04:46.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.475+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v101'90 uv90 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.475+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af665b220 con 0x562af6658480 2023-11-15T03:04:46.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.479+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v101'91 uv82 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.479+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af665b220 con 0x562af6658480 2023-11-15T03:04:46.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.483+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v101'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.483+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af665b220 con 0x562af6658480 2023-11-15T03:04:46.496 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:04:46.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.487+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v101'93 uv93 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.487+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af665c230 con 0x562af6658480 2023-11-15T03:04:46.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.491+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v101'94 uv94 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.491+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af665c230 con 0x562af6658480 2023-11-15T03:04:46.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.491+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v101'95 uv95 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.491+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af665b220 con 0x562af6658480 2023-11-15T03:04:46.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.495+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v101'96 uv96 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.495+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x562af665b8a0 0x562af665dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.495+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af665e260 con 0x562af665b8a0 2023-11-15T03:04:46.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.495+0000 7fadb62b7700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x562af665b8a0 0x562af665dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.495+0000 7fadb62b7700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x562af665b8a0 0x562af665dc90 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:46.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.499+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v101'39 uv38 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af665b8a0 2023-11-15T03:04:46.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.499+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af665e340 con 0x562af665b8a0 2023-11-15T03:04:46.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.503+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v101'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af665b8a0 2023-11-15T03:04:46.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.503+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af665e770 con 0x562af665b8a0 2023-11-15T03:04:46.515 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:04:46.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.503+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v101'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af665b8a0 2023-11-15T03:04:46.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.503+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af665f780 con 0x562af665b8a0 2023-11-15T03:04:46.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.507+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v101'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af665b8a0 2023-11-15T03:04:46.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.507+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af665f780 con 0x562af665b8a0 2023-11-15T03:04:46.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.511+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v101'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af665b8a0 2023-11-15T03:04:46.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.511+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af665e770 con 0x562af665b8a0 2023-11-15T03:04:46.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.511+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v101'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af665b8a0 2023-11-15T03:04:46.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.511+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562af665ec80 0x562af6661070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.511+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661720 con 0x562af665ec80 2023-11-15T03:04:46.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.511+0000 7fadadab6700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562af665ec80 0x562af6661070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.515+0000 7fadadab6700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562af665ec80 0x562af6661070 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:46.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.515+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v101'30 uv29 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.515+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af6661b00 con 0x562af665ec80 2023-11-15T03:04:46.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.519+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v101'31 uv31 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.519+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661b00 con 0x562af665ec80 2023-11-15T03:04:46.529 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:04:46.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.523+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v101'32 uv32 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.523+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661b00 con 0x562af665ec80 2023-11-15T03:04:46.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.527+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v101'33 uv33 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.527+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661b00 con 0x562af665ec80 2023-11-15T03:04:46.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.531+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v101'39 uv38 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.531+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af6661b00 con 0x562af665ec80 2023-11-15T03:04:46.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.531+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v101'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.531+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661b00 con 0x562af665ec80 2023-11-15T03:04:46.542 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:04:46.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.535+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v101'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.535+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6662b10 con 0x562af665ec80 2023-11-15T03:04:46.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.535+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v101'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.539+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6662b10 con 0x562af665ec80 2023-11-15T03:04:46.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.543+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v101'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.543+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661b00 con 0x562af665ec80 2023-11-15T03:04:46.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.543+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v101'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.543+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661b00 con 0x562af664f5d0 2023-11-15T03:04:46.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.547+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v101'70 uv64 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.547+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af6661b00 con 0x562af664f5d0 2023-11-15T03:04:46.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.551+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v101'71 uv71 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.551+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661b00 con 0x562af664f5d0 2023-11-15T03:04:46.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.555+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v101'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.563 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:04:46.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.555+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661b00 con 0x562af664f5d0 2023-11-15T03:04:46.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.559+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v101'73 uv73 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.559+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661b00 con 0x562af664f5d0 2023-11-15T03:04:46.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.559+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v101'74 uv74 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.559+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661b00 con 0x562af664f5d0 2023-11-15T03:04:46.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.563+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v101'75 uv75 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.563+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661b00 con 0x562af664f5d0 2023-11-15T03:04:46.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.567+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v101'76 uv69 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.567+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af6661b00 con 0x562af664f5d0 2023-11-15T03:04:46.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.567+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v101'77 uv77 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.567+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661b00 con 0x562af664f5d0 2023-11-15T03:04:46.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.571+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v101'78 uv78 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.579 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:04:46.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.571+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661b00 con 0x562af664f5d0 2023-11-15T03:04:46.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.575+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v101'79 uv79 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.575+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af664f5d0 2023-11-15T03:04:46.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.579+0000 7fadb62b7700 1 -- 172.21.15.121:0/2506771385 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v101'80 uv80 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada800ea50 con 0x562af664f5d0 2023-11-15T03:04:46.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.579+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af6653570 2023-11-15T03:04:46.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.579+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v101'32 uv31 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af6653570 2023-11-15T03:04:46.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.579+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af6653570 2023-11-15T03:04:46.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.583+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v101'33 uv33 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af6653570 2023-11-15T03:04:46.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.583+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af6653570 2023-11-15T03:04:46.593 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:04:46.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.583+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v101'34 uv34 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af6653570 2023-11-15T03:04:46.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.583+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af6653570 2023-11-15T03:04:46.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.587+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v101'35 uv35 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af6653570 2023-11-15T03:04:46.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.587+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af6653570 2023-11-15T03:04:46.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.591+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v101'36 uv36 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af6653570 2023-11-15T03:04:46.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.591+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af665ec80 2023-11-15T03:04:46.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.591+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v101'39 uv38 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.591+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af665ec80 2023-11-15T03:04:46.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.595+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v101'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.595+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af665ec80 2023-11-15T03:04:46.607 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:04:46.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.599+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v101'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.599+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af665ec80 2023-11-15T03:04:46.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.603+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v101'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.603+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af665ec80 2023-11-15T03:04:46.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.603+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v101'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.603+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af665ec80 2023-11-15T03:04:46.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.607+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v101'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.607+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af665ec80 2023-11-15T03:04:46.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.611+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v101'33 uv32 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.611+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af665ec80 2023-11-15T03:04:46.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.615+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v101'34 uv34 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.615+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af665ec80 2023-11-15T03:04:46.624 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:04:46.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.615+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v101'35 uv35 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.615+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af665ec80 2023-11-15T03:04:46.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.619+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v101'36 uv36 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.619+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af665ec80 2023-11-15T03:04:46.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.623+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v101'37 uv37 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.623+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af665ec80 2023-11-15T03:04:46.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.623+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v101'32 uv31 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.627+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af665ec80 2023-11-15T03:04:46.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.627+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v101'33 uv33 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.627+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af665ec80 2023-11-15T03:04:46.639 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:04:46.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.631+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v101'34 uv34 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.631+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6661f20 con 0x562af665ec80 2023-11-15T03:04:46.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.635+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v101'35 uv35 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.635+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6664ba0 con 0x562af665ec80 2023-11-15T03:04:46.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.635+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v101'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.635+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x562af6657740 0x562af66650f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.635+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66655e0 con 0x562af6657740 2023-11-15T03:04:46.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.639+0000 7fadb6ab8700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x562af6657740 0x562af66650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.639+0000 7fadb6ab8700 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x562af6657740 0x562af66650f0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:46.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.643+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v101'27 uv26 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6657740 2023-11-15T03:04:46.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.643+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af66655e0 con 0x562af6657740 2023-11-15T03:04:46.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.647+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v101'28 uv28 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6657740 2023-11-15T03:04:46.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.647+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66655e0 con 0x562af6657740 2023-11-15T03:04:46.656 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:04:46.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.647+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v101'29 uv29 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6657740 2023-11-15T03:04:46.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.647+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66655e0 con 0x562af6657740 2023-11-15T03:04:46.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.651+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v101'30 uv30 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6657740 2023-11-15T03:04:46.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.651+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66655e0 con 0x562af665ec80 2023-11-15T03:04:46.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.655+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v101'39 uv38 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.655+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af66655e0 con 0x562af665ec80 2023-11-15T03:04:46.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.659+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v101'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.659+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6664d30 con 0x562af665ec80 2023-11-15T03:04:46.674 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:04:46.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.663+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v101'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.663+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66665f0 con 0x562af665ec80 2023-11-15T03:04:46.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.663+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v101'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.663+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66665f0 con 0x562af665ec80 2023-11-15T03:04:46.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.667+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v101'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.667+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66655e0 con 0x562af665ec80 2023-11-15T03:04:46.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.671+0000 7fadadab6700 1 -- 172.21.15.121:0/2506771385 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v101'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fada0072db0 con 0x562af665ec80 2023-11-15T03:04:46.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.671+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66655e0 con 0x562af6658480 2023-11-15T03:04:46.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.671+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v101'32 uv31 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.675+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af66655e0 con 0x562af6658480 2023-11-15T03:04:46.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.675+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v101'33 uv33 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.675+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66655e0 con 0x562af6658480 2023-11-15T03:04:46.686 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:04:46.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.679+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v101'34 uv34 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.679+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66655e0 con 0x562af6658480 2023-11-15T03:04:46.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.679+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v101'35 uv35 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.679+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66623a0 con 0x562af6658480 2023-11-15T03:04:46.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.683+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v101'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.683+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66623a0 con 0x562af6658480 2023-11-15T03:04:46.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.687+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v101'78 uv70 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.687+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af66655e0 con 0x562af6658480 2023-11-15T03:04:46.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.687+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v101'79 uv79 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.687+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66655e0 con 0x562af6658480 2023-11-15T03:04:46.699 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:04:46.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.691+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v101'80 uv80 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.691+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66655e0 con 0x562af6658480 2023-11-15T03:04:46.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.695+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v101'81 uv81 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.695+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6665a30 con 0x562af6658480 2023-11-15T03:04:46.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.699+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v101'82 uv82 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.699+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6665a30 con 0x562af6658480 2023-11-15T03:04:46.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.699+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v101'83 uv77 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.699+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x562af66656b0 con 0x562af6658480 2023-11-15T03:04:46.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.703+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v101'84 uv84 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.703+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66655e0 con 0x562af6658480 2023-11-15T03:04:46.712 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:04:46.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.703+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v101'85 uv85 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.703+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66655e0 con 0x562af6658480 2023-11-15T03:04:46.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.707+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v101'86 uv86 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.707+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af66655e0 con 0x562af6658480 2023-11-15T03:04:46.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.711+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v101'87 uv87 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.711+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6665770 con 0x562af6658480 2023-11-15T03:04:46.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.711+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v101'88 uv88 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.711+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e101) v8 -- 0x562af6665c60 con 0x562af6658480 2023-11-15T03:04:46.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v101'89 uv89 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fadb000bc10 con 0x562af6658480 2023-11-15T03:04:46.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x562af6653570 msgr2=0x562af6656650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x562af6653570 0x562af6656650 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x562af65a0080 msgr2=0x562af659dcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x562af65a0080 0x562af659dcf0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x562af665b8a0 msgr2=0x562af665dc90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x562af665b8a0 0x562af665dc90 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x562af664f5d0 msgr2=0x562af66519c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x562af664f5d0 0x562af66519c0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x562af6657740 msgr2=0x562af66650f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 reap_dead start 2023-11-15T03:04:46.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x562af6657740 0x562af66650f0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x562af6658480 msgr2=0x562af665a870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x562af6658480 0x562af665a870 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562af665ec80 msgr2=0x562af6661070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562af665ec80 0x562af6661070 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fad98059020 msgr2=0x7fad9805b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fad98059020 0x7fad9805b4e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7fada800c010 tx=0x7fada8014000).stop 2023-11-15T03:04:46.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af662fec0 msgr2=0x562af6625970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 --2- 172.21.15.121:0/2506771385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af662fec0 0x562af6625970 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x562af6559790 tx=0x7fada0015000).stop 2023-11-15T03:04:46.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb6ab8700 1 -- 172.21.15.121:0/2506771385 reap_dead start 2023-11-15T03:04:46.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 shutdown_connections 2023-11-15T03:04:46.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 >> 172.21.15.121:0/2506771385 conn(0x562af65587d0 msgr2=0x562af65a1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:46.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 shutdown_connections 2023-11-15T03:04:46.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.715+0000 7fadb897bd00 1 -- 172.21.15.121:0/2506771385 wait complete. 2023-11-15T03:04:46.731 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 500.00000000 2023-11-15T03:04:46.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.779+0000 7fd594d3cd00 1 Processor -- start 2023-11-15T03:04:46.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.779+0000 7fd594d3cd00 1 -- start start 2023-11-15T03:04:46.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd594d3cd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4fbafe6c0 0x55d4fbbd0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd594d3cd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4fbb48b00 0x55d4fbbd1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd594d3cd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d4fbbd7670 0x55d4fbb4a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd594d3cd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fbbd9f10 con 0x55d4fbb48b00 2023-11-15T03:04:46.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd594d3cd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fbbd9900 con 0x55d4fbafe6c0 2023-11-15T03:04:46.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd594d3cd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4fbbdac30 con 0x55d4fbbd7670 2023-11-15T03:04:46.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd580ab6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4fbb48b00 0x55d4fbbd1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd580ab6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4fbb48b00 0x55d4fbbd1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33836/0 (socket says 172.21.15.121:33836) 2023-11-15T03:04:46.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd580ab6700 1 -- 172.21.15.121:0/733025447 learned_addr learned my addr 172.21.15.121:0/733025447 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:46.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd5892b7700 1 --2- 172.21.15.121:0/733025447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4fbafe6c0 0x55d4fbbd0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd589ab8700 1 --2- 172.21.15.121:0/733025447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d4fbbd7670 0x55d4fbb4a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd580ab6700 1 -- 172.21.15.121:0/733025447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d4fbbd7670 msgr2=0x55d4fbb4a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:46.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd580ab6700 1 --2- 172.21.15.121:0/733025447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d4fbbd7670 0x55d4fbb4a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd580ab6700 1 -- 172.21.15.121:0/733025447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4fbafe6c0 msgr2=0x55d4fbbd0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:46.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd580ab6700 1 --2- 172.21.15.121:0/733025447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4fbafe6c0 0x55d4fbbd0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd580ab6700 1 -- 172.21.15.121:0/733025447 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd584005db0 con 0x55d4fbb48b00 2023-11-15T03:04:46.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd580ab6700 1 --2- 172.21.15.121:0/733025447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4fbb48b00 0x55d4fbbd1220 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7fd57400a7c0 tx=0x7fd574005cd0).ready entity=mon.0 client_cookie=43dd8bf07ca7d13b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:46.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd583fff700 1 -- 172.21.15.121:0/733025447 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57400f070 con 0x55d4fbb48b00 2023-11-15T03:04:46.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd583fff700 1 -- 172.21.15.121:0/733025447 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5740087e0 con 0x55d4fbb48b00 2023-11-15T03:04:46.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.783+0000 7fd583fff700 1 -- 172.21.15.121:0/733025447 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574008b60 con 0x55d4fbb48b00 2023-11-15T03:04:46.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.787+0000 7fd594d3cd00 1 -- 172.21.15.121:0/733025447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4fbb48b00 msgr2=0x55d4fbbd1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.787+0000 7fd594d3cd00 1 --2- 172.21.15.121:0/733025447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4fbb48b00 0x55d4fbbd1220 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7fd57400a7c0 tx=0x7fd574005cd0).stop 2023-11-15T03:04:46.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.787+0000 7fd594d3cd00 1 -- 172.21.15.121:0/733025447 shutdown_connections 2023-11-15T03:04:46.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.787+0000 7fd594d3cd00 1 --2- 172.21.15.121:0/733025447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d4fbbd7670 0x55d4fbb4a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.787+0000 7fd594d3cd00 1 --2- 172.21.15.121:0/733025447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4fbb48b00 0x55d4fbbd1220 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.787+0000 7fd594d3cd00 1 --2- 172.21.15.121:0/733025447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4fbafe6c0 0x55d4fbbd0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.787+0000 7fd594d3cd00 1 -- 172.21.15.121:0/733025447 >> 172.21.15.121:0/733025447 conn(0x55d4fbb4e080 msgr2=0x55d4fbb33a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:46.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.787+0000 7fd594d3cd00 1 -- 172.21.15.121:0/733025447 shutdown_connections 2023-11-15T03:04:46.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.787+0000 7fd594d3cd00 1 -- 172.21.15.121:0/733025447 wait complete. 2023-11-15T03:04:46.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.787+0000 7fd594d3cd00 1 Processor -- start 2023-11-15T03:04:46.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.787+0000 7fd594d3cd00 1 -- start start 2023-11-15T03:04:46.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd594d3cd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4fbafe6c0 0x55d4fbbcfda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd594d3cd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d4fbb48b00 0x55d4fbbd3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd594d3cd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4fbbd7670 0x55d4fbbd4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd594d3cd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fbbd4970 con 0x55d4fbafe6c0 2023-11-15T03:04:46.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd594d3cd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fbbd3d10 con 0x55d4fbbd7670 2023-11-15T03:04:46.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd594d3cd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4fbb33400 con 0x55d4fbb48b00 2023-11-15T03:04:46.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd5892b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4fbafe6c0 0x55d4fbbcfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd5892b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4fbafe6c0 0x55d4fbbcfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33838/0 (socket says 172.21.15.121:33838) 2023-11-15T03:04:46.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd5892b7700 1 -- 172.21.15.121:0/1125198367 learned_addr learned my addr 172.21.15.121:0/1125198367 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:46.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd589ab8700 1 --2- 172.21.15.121:0/1125198367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4fbbd7670 0x55d4fbbd4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd580ab6700 1 --2- 172.21.15.121:0/1125198367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d4fbb48b00 0x55d4fbbd3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd5892b7700 1 -- 172.21.15.121:0/1125198367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d4fbb48b00 msgr2=0x55d4fbbd3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd5892b7700 1 --2- 172.21.15.121:0/1125198367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d4fbb48b00 0x55d4fbbd3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd5892b7700 1 -- 172.21.15.121:0/1125198367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4fbbd7670 msgr2=0x55d4fbbd4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd5892b7700 1 --2- 172.21.15.121:0/1125198367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4fbbd7670 0x55d4fbbd4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd5892b7700 1 -- 172.21.15.121:0/1125198367 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57400a440 con 0x55d4fbafe6c0 2023-11-15T03:04:46.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd5892b7700 1 --2- 172.21.15.121:0/1125198367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4fbafe6c0 0x55d4fbbcfda0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7fd57c0109c0 tx=0x7fd57c005d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:46.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd5827fc700 1 -- 172.21.15.121:0/1125198367 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c00b470 con 0x55d4fbafe6c0 2023-11-15T03:04:46.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd5827fc700 1 -- 172.21.15.121:0/1125198367 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57c010e50 con 0x55d4fbafe6c0 2023-11-15T03:04:46.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd594d3cd00 1 -- 172.21.15.121:0/1125198367 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d4fbaa0a20 con 0x55d4fbafe6c0 2023-11-15T03:04:46.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd5827fc700 1 -- 172.21.15.121:0/1125198367 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c00eb50 con 0x55d4fbafe6c0 2023-11-15T03:04:46.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.791+0000 7fd594d3cd00 1 -- 172.21.15.121:0/1125198367 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4fbaa0770 con 0x55d4fbafe6c0 2023-11-15T03:04:46.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.795+0000 7fd5827fc700 1 -- 172.21.15.121:0/1125198367 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd57c013440 con 0x55d4fbafe6c0 2023-11-15T03:04:46.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.795+0000 7fd5827fc700 1 --2- 172.21.15.121:0/1125198367 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd56c059020 0x7fd56c05b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.795+0000 7fd5827fc700 1 -- 172.21.15.121:0/1125198367 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6868+0+0 (secure 0 0 0) 0x7fd57c074370 con 0x55d4fbafe6c0 2023-11-15T03:04:46.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.795+0000 7fd580ab6700 1 --2- 172.21.15.121:0/1125198367 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd56c059020 0x7fd56c05b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.795+0000 7fd594d3cd00 1 --2- 172.21.15.121:0/1125198367 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55d4fbbd0290 0x55d4fbbf2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.795+0000 7fd594d3cd00 1 -- 172.21.15.121:0/1125198367 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e101) v8 -- 0x55d4fbb4aed0 con 0x55d4fbbd0290 2023-11-15T03:04:46.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.795+0000 7fd589ab8700 1 --2- 172.21.15.121:0/1125198367 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55d4fbbd0290 0x55d4fbbf2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.795+0000 7fd580ab6700 1 --2- 172.21.15.121:0/1125198367 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd56c059020 0x7fd56c05b4e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x55d4fbbd7e20 tx=0x7fd574016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:46.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.795+0000 7fd589ab8700 1 --2- 172.21.15.121:0/1125198367 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55d4fbbd0290 0x55d4fbbf2c50 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:46.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.799+0000 7fd589ab8700 1 -- 172.21.15.121:0/1125198367 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(1 500.00000000 [delete] v101'81 uv81 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd5840054f0 con 0x55d4fbbd0290 2023-11-15T03:04:46.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.799+0000 7fd594d3cd00 1 -- 172.21.15.121:0/1125198367 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55d4fbbd0290 msgr2=0x55d4fbbf2c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.799+0000 7fd594d3cd00 1 --2- 172.21.15.121:0/1125198367 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55d4fbbd0290 0x55d4fbbf2c50 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.799+0000 7fd594d3cd00 1 -- 172.21.15.121:0/1125198367 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd56c059020 msgr2=0x7fd56c05b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.799+0000 7fd594d3cd00 1 --2- 172.21.15.121:0/1125198367 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd56c059020 0x7fd56c05b4e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x55d4fbbd7e20 tx=0x7fd574016000).stop 2023-11-15T03:04:46.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.799+0000 7fd594d3cd00 1 -- 172.21.15.121:0/1125198367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4fbafe6c0 msgr2=0x55d4fbbcfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.799+0000 7fd594d3cd00 1 --2- 172.21.15.121:0/1125198367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4fbafe6c0 0x55d4fbbcfda0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7fd57c0109c0 tx=0x7fd57c005d40).stop 2023-11-15T03:04:46.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.799+0000 7fd589ab8700 1 -- 172.21.15.121:0/1125198367 reap_dead start 2023-11-15T03:04:46.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.799+0000 7fd594d3cd00 1 -- 172.21.15.121:0/1125198367 shutdown_connections 2023-11-15T03:04:46.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.799+0000 7fd594d3cd00 1 -- 172.21.15.121:0/1125198367 >> 172.21.15.121:0/1125198367 conn(0x55d4fbb4e080 msgr2=0x55d4fbb4a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:46.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.799+0000 7fd594d3cd00 1 -- 172.21.15.121:0/1125198367 shutdown_connections 2023-11-15T03:04:46.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.799+0000 7fd594d3cd00 1 -- 172.21.15.121:0/1125198367 wait complete. 2023-11-15T03:04:46.814 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:04:46.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.971+0000 7fa4f9bb9700 1 Processor -- start 2023-11-15T03:04:46.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.971+0000 7fa4f9bb9700 1 -- start start 2023-11-15T03:04:46.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.971+0000 7fa4f9bb9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa4f40a8960 0x7fa4f40c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.971+0000 7fa4f9bb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f40a8d40 0x7fa4f415c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.971+0000 7fa4f9bb9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa4f4066520 0x7fa4f4066990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.971+0000 7fa4f9bb9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f41583d0 con 0x7fa4f4066520 2023-11-15T03:04:46.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.971+0000 7fa4f9bb9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f40657f0 con 0x7fa4f40a8960 2023-11-15T03:04:46.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.971+0000 7fa4f9bb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f40c1b60 con 0x7fa4f40a8d40 2023-11-15T03:04:46.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4f93b8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa4f4066520 0x7fa4f4066990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4ebfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f40a8d40 0x7fa4f415c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4ebfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f40a8d40 0x7fa4f415c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:53040/0 (socket says 172.21.15.121:53040) 2023-11-15T03:04:46.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4ebfff700 1 -- 172.21.15.121:0/745752927 learned_addr learned my addr 172.21.15.121:0/745752927 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:46.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4f93b8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa4f4066520 0x7fa4f4066990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33844/0 (socket says 172.21.15.121:33844) 2023-11-15T03:04:46.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4f8bb7700 1 --2- 172.21.15.121:0/745752927 >> v2:172.21.15.190:3300/0 conn(0x7fa4f40a8960 0x7fa4f40c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4ebfff700 1 -- 172.21.15.121:0/745752927 >> v2:172.21.15.190:3300/0 conn(0x7fa4f40a8960 msgr2=0x7fa4f40c15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:46.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4ebfff700 1 --2- 172.21.15.121:0/745752927 >> v2:172.21.15.190:3300/0 conn(0x7fa4f40a8960 0x7fa4f40c15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4ebfff700 1 -- 172.21.15.121:0/745752927 >> v2:172.21.15.121:3300/0 conn(0x7fa4f4066520 msgr2=0x7fa4f4066990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:46.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4ebfff700 1 --2- 172.21.15.121:0/745752927 >> v2:172.21.15.121:3300/0 conn(0x7fa4f4066520 0x7fa4f4066990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4ebfff700 1 -- 172.21.15.121:0/745752927 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f4065a40 con 0x7fa4f40a8d40 2023-11-15T03:04:46.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4f93b8700 1 --2- 172.21.15.121:0/745752927 >> v2:172.21.15.121:3300/0 conn(0x7fa4f4066520 0x7fa4f4066990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:04:46.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4ebfff700 1 --2- 172.21.15.121:0/745752927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f40a8d40 0x7fa4f415c4c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fa4f4063c80 tx=0x7fa4e40090f0).ready entity=mon.2 client_cookie=572288d05bff9dbf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:46.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4eb7fe700 1 -- 172.21.15.121:0/745752927 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e400f510 con 0x7fa4f40a8d40 2023-11-15T03:04:46.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4eb7fe700 1 -- 172.21.15.121:0/745752927 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e400f670 con 0x7fa4f40a8d40 2023-11-15T03:04:46.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4eb7fe700 1 -- 172.21.15.121:0/745752927 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e400fa10 con 0x7fa4f40a8d40 2023-11-15T03:04:46.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4f9bb9700 1 -- 172.21.15.121:0/745752927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f40a8d40 msgr2=0x7fa4f415c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:46.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4f9bb9700 1 --2- 172.21.15.121:0/745752927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f40a8d40 0x7fa4f415c4c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fa4f4063c80 tx=0x7fa4e40090f0).stop 2023-11-15T03:04:46.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4f9bb9700 1 -- 172.21.15.121:0/745752927 shutdown_connections 2023-11-15T03:04:46.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4f9bb9700 1 --2- 172.21.15.121:0/745752927 >> v2:172.21.15.121:3300/0 conn(0x7fa4f4066520 0x7fa4f4066990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4f9bb9700 1 --2- 172.21.15.121:0/745752927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f40a8d40 0x7fa4f415c4c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4f9bb9700 1 --2- 172.21.15.121:0/745752927 >> v2:172.21.15.190:3300/0 conn(0x7fa4f40a8960 0x7fa4f40c15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:46.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4f9bb9700 1 -- 172.21.15.121:0/745752927 >> 172.21.15.121:0/745752927 conn(0x7fa4f4067a60 msgr2=0x7fa4f4067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:46.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4f9bb9700 1 -- 172.21.15.121:0/745752927 shutdown_connections 2023-11-15T03:04:46.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.975+0000 7fa4f9bb9700 1 -- 172.21.15.121:0/745752927 wait complete. 2023-11-15T03:04:46.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.979+0000 7fa4f9bb9700 1 Processor -- start 2023-11-15T03:04:46.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f9bb9700 1 -- start start 2023-11-15T03:04:46.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f9bb9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f4066520 0x7fa4f415d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f9bb9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4f40a8960 0x7fa4f415e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f9bb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f40a8d40 0x7fa4f40aa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:46.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f9bb9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f415a480 con 0x7fa4f4066520 2023-11-15T03:04:46.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f9bb9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f415c510 con 0x7fa4f40a8960 2023-11-15T03:04:46.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f9bb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f415c7f0 con 0x7fa4f40a8d40 2023-11-15T03:04:46.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f8bb7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f4066520 0x7fa4f415d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:46.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f8bb7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f4066520 0x7fa4f415d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33856/0 (socket says 172.21.15.121:33856) 2023-11-15T03:04:46.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f8bb7700 1 -- 172.21.15.121:0/1102143759 learned_addr learned my addr 172.21.15.121:0/1102143759 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:46.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4ebfff700 1 --2- 172.21.15.121:0/1102143759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4f40a8960 0x7fa4f415e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:47.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f93b8700 1 --2- 172.21.15.121:0/1102143759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f40a8d40 0x7fa4f40aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:47.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f8bb7700 1 -- 172.21.15.121:0/1102143759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f40a8d40 msgr2=0x7fa4f40aa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:47.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f8bb7700 1 --2- 172.21.15.121:0/1102143759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f40a8d40 0x7fa4f40aa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:47.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f8bb7700 1 -- 172.21.15.121:0/1102143759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4f40a8960 msgr2=0x7fa4f415e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:47.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f8bb7700 1 --2- 172.21.15.121:0/1102143759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4f40a8960 0x7fa4f415e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:47.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f8bb7700 1 -- 172.21.15.121:0/1102143759 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ec009a20 con 0x7fa4f4066520 2023-11-15T03:04:47.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f93b8700 1 --2- 172.21.15.121:0/1102143759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f40a8d40 0x7fa4f40aa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:47.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f8bb7700 1 --2- 172.21.15.121:0/1102143759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f4066520 0x7fa4f415d2d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7fa4f000e930 tx=0x7fa4f0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:47.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4e8ff9700 1 -- 172.21.15.121:0/1102143759 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f000c670 con 0x7fa4f4066520 2023-11-15T03:04:47.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4e8ff9700 1 -- 172.21.15.121:0/1102143759 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f000c7d0 con 0x7fa4f4066520 2023-11-15T03:04:47.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f9bb9700 1 -- 172.21.15.121:0/1102143759 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e4009370 con 0x7fa4f4066520 2023-11-15T03:04:47.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4e8ff9700 1 -- 172.21.15.121:0/1102143759 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f000cb50 con 0x7fa4f4066520 2023-11-15T03:04:47.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.983+0000 7fa4f9bb9700 1 -- 172.21.15.121:0/1102143759 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f415cdb0 con 0x7fa4f4066520 2023-11-15T03:04:47.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.987+0000 7fa4f9bb9700 1 -- 172.21.15.121:0/1102143759 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f406bcf0 con 0x7fa4f4066520 2023-11-15T03:04:47.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.987+0000 7fa4e8ff9700 1 -- 172.21.15.121:0/1102143759 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa4f001b400 con 0x7fa4f4066520 2023-11-15T03:04:47.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.991+0000 7fa4e8ff9700 1 --2- 172.21.15.121:0/1102143759 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa4e005a540 0x7fa4e005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:47.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.991+0000 7fa4e8ff9700 1 -- 172.21.15.121:0/1102143759 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6868+0+0 (secure 0 0 0) 0x7fa4f0074f70 con 0x7fa4f4066520 2023-11-15T03:04:47.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.991+0000 7fa4ebfff700 1 --2- 172.21.15.121:0/1102143759 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa4e005a540 0x7fa4e005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:47.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.991+0000 7fa4ebfff700 1 --2- 172.21.15.121:0/1102143759 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa4e005a540 0x7fa4e005ca00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7fa4f40a9160 tx=0x7fa4e4017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:47.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:46.995+0000 7fa4e8ff9700 1 -- 172.21.15.121:0/1102143759 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa4f00a3af0 con 0x7fa4f4066520 2023-11-15T03:04:47.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:47.191+0000 7fa4f9bb9700 1 -- 172.21.15.121:0/1102143759 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fa4f40a8050 con 0x7fa4f4066520 2023-11-15T03:04:48.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.031+0000 7fa4e8ff9700 1 -- 172.21.15.121:0/1102143759 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v126) v1 ==== 162+0+0 (secure 0 0 0) 0x7fa4f0048400 con 0x7fa4f4066520 2023-11-15T03:04:48.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.039+0000 7fa4f9bb9700 1 -- 172.21.15.121:0/1102143759 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa4e005a540 msgr2=0x7fa4e005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:48.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.039+0000 7fa4f9bb9700 1 --2- 172.21.15.121:0/1102143759 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa4e005a540 0x7fa4e005ca00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7fa4f40a9160 tx=0x7fa4e4017000).stop 2023-11-15T03:04:48.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.039+0000 7fa4f9bb9700 1 -- 172.21.15.121:0/1102143759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f4066520 msgr2=0x7fa4f415d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:48.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.039+0000 7fa4f9bb9700 1 --2- 172.21.15.121:0/1102143759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f4066520 0x7fa4f415d2d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7fa4f000e930 tx=0x7fa4f0009060).stop 2023-11-15T03:04:48.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.039+0000 7fa4f9bb9700 1 -- 172.21.15.121:0/1102143759 shutdown_connections 2023-11-15T03:04:48.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.039+0000 7fa4f9bb9700 1 --2- 172.21.15.121:0/1102143759 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa4e005a540 0x7fa4e005ca00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:48.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.039+0000 7fa4f9bb9700 1 --2- 172.21.15.121:0/1102143759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4f40a8d40 0x7fa4f40aa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:48.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.039+0000 7fa4f9bb9700 1 --2- 172.21.15.121:0/1102143759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4f40a8960 0x7fa4f415e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:48.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.039+0000 7fa4f9bb9700 1 --2- 172.21.15.121:0/1102143759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f4066520 0x7fa4f415d2d0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:48.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.039+0000 7fa4f9bb9700 1 -- 172.21.15.121:0/1102143759 >> 172.21.15.121:0/1102143759 conn(0x7fa4f4067a60 msgr2=0x7fa4f40ac180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:48.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.039+0000 7fa4f9bb9700 1 -- 172.21.15.121:0/1102143759 shutdown_connections 2023-11-15T03:04:48.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.039+0000 7fa4f9bb9700 1 -- 172.21.15.121:0/1102143759 wait complete. 2023-11-15T03:04:48.049 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:04:48.470 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:04:48.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.643+0000 7f6af4969700 1 Processor -- start 2023-11-15T03:04:48.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.643+0000 7f6af4969700 1 -- start start 2023-11-15T03:04:48.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.647+0000 7f6af4969700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6af0150870 0x7f6af00c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:48.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.647+0000 7f6af4969700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af0153d20 0x7f6af00c3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:48.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.647+0000 7f6af4969700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af00c6350 con 0x7f6af0150c50 2023-11-15T03:04:48.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.647+0000 7f6af4969700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af00c3b70 con 0x7f6af0150870 2023-11-15T03:04:48.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.647+0000 7f6af4969700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af01541b0 con 0x7f6af0153d20 2023-11-15T03:04:48.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.647+0000 7f6aee59c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6af0150870 0x7f6af00c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:48.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.647+0000 7f6aedd9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af0153d20 0x7f6af00c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:48.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.647+0000 7f6aeed9d700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f6af0150c50 0x7f6af01510c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:44434/0 (socket says 172.21.15.121:44434) 2023-11-15T03:04:48.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.647+0000 7f6aee59c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6af0150870 0x7f6af00c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:46222/0 (socket says 172.21.15.121:46222) 2023-11-15T03:04:48.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.647+0000 7f6aeed9d700 1 -- 172.21.15.121:0/3036597770 learned_addr learned my addr 172.21.15.121:0/3036597770 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:48.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.647+0000 7f6aedd9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af0153d20 0x7f6af00c3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:53054/0 (socket says 172.21.15.121:53054) 2023-11-15T03:04:48.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.651+0000 7f6aed59a700 1 -- 172.21.15.121:0/3036597770 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1463204419 0 0) 0x7f6af00c6350 con 0x7f6af0150c50 2023-11-15T03:04:48.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.651+0000 7f6aee59c700 1 -- 172.21.15.121:0/3036597770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af0153d20 msgr2=0x7f6af00c3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:48.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.651+0000 7f6aee59c700 1 --2- 172.21.15.121:0/3036597770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af0153d20 0x7f6af00c3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:48.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.651+0000 7f6aee59c700 1 -- 172.21.15.121:0/3036597770 >> v1:172.21.15.190:6789/0 conn(0x7f6af0150c50 legacy=0x7f6af01510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:48.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.651+0000 7f6aee59c700 1 -- 172.21.15.121:0/3036597770 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af0154490 con 0x7f6af0150870 2023-11-15T03:04:48.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.651+0000 7f6aedd9b700 1 --2- 172.21.15.121:0/3036597770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af0153d20 0x7f6af00c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:04:48.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.651+0000 7f6aee59c700 1 --2- 172.21.15.121:0/3036597770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af0150870 0x7f6af00c3190 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f6ae0009760 tx=0x7f6ae0009060).ready entity=mon.1 client_cookie=ad3808c73eb31d64 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:48.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.651+0000 7f6aed59a700 1 -- 172.21.15.121:0/3036597770 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000f510 con 0x7f6af0150870 2023-11-15T03:04:48.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.651+0000 7f6aed59a700 1 -- 172.21.15.121:0/3036597770 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae000f670 con 0x7f6af0150870 2023-11-15T03:04:48.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.651+0000 7f6aed59a700 1 -- 172.21.15.121:0/3036597770 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000f9f0 con 0x7f6af0150870 2023-11-15T03:04:48.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.651+0000 7f6af4969700 1 -- 172.21.15.121:0/3036597770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af0150870 msgr2=0x7f6af00c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:48.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.651+0000 7f6af4969700 1 --2- 172.21.15.121:0/3036597770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af0150870 0x7f6af00c3190 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f6ae0009760 tx=0x7f6ae0009060).stop 2023-11-15T03:04:48.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.651+0000 7f6af4969700 1 -- 172.21.15.121:0/3036597770 shutdown_connections 2023-11-15T03:04:48.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.651+0000 7f6af4969700 1 --2- 172.21.15.121:0/3036597770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af0153d20 0x7f6af00c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:48.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.651+0000 7f6af4969700 1 --2- 172.21.15.121:0/3036597770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af0150870 0x7f6af00c3190 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:48.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.651+0000 7f6af4969700 1 -- 172.21.15.121:0/3036597770 >> 172.21.15.121:0/3036597770 conn(0x7f6af0062190 msgr2=0x7f6af00625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:48.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.651+0000 7f6af4969700 1 -- 172.21.15.121:0/3036597770 shutdown_connections 2023-11-15T03:04:48.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.651+0000 7f6af4969700 1 -- 172.21.15.121:0/3036597770 wait complete. 2023-11-15T03:04:48.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.655+0000 7f6af4969700 1 Processor -- start 2023-11-15T03:04:48.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.655+0000 7f6af4969700 1 -- start start 2023-11-15T03:04:48.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.655+0000 7f6af4969700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af0150870 0x7f6af00bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:48.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.655+0000 7f6af4969700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af0150c50 0x7f6af00c0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:48.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.655+0000 7f6af4969700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6af00ab940 0x7f6af00abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:48.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.655+0000 7f6af4969700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af0167800 con 0x7f6af00ab940 2023-11-15T03:04:48.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.655+0000 7f6af4969700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af00c0570 con 0x7f6af0150c50 2023-11-15T03:04:48.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.655+0000 7f6af4969700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af014c360 con 0x7f6af0150870 2023-11-15T03:04:48.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.655+0000 7f6aeed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6af00ab940 0x7f6af00abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:48.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.655+0000 7f6aeed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6af00ab940 0x7f6af00abdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33866/0 (socket says 172.21.15.121:33866) 2023-11-15T03:04:48.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.655+0000 7f6aeed9d700 1 -- 172.21.15.121:0/3286890870 learned_addr learned my addr 172.21.15.121:0/3286890870 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:48.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.655+0000 7f6aedd9b700 1 --2- 172.21.15.121:0/3286890870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af0150c50 0x7f6af00c0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:48.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.655+0000 7f6aee59c700 1 --2- 172.21.15.121:0/3286890870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af0150870 0x7f6af00bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:48.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.655+0000 7f6aeed9d700 1 -- 172.21.15.121:0/3286890870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af0150870 msgr2=0x7f6af00bfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:48.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.655+0000 7f6aeed9d700 1 --2- 172.21.15.121:0/3286890870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af0150870 0x7f6af00bfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:48.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.655+0000 7f6aeed9d700 1 -- 172.21.15.121:0/3286890870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af0150c50 msgr2=0x7f6af00c0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:48.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.655+0000 7f6aeed9d700 1 --2- 172.21.15.121:0/3286890870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af0150c50 0x7f6af00c0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:48.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.655+0000 7f6aeed9d700 1 -- 172.21.15.121:0/3286890870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae40092a0 con 0x7f6af00ab940 2023-11-15T03:04:48.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.655+0000 7f6aeed9d700 1 --2- 172.21.15.121:0/3286890870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6af00ab940 0x7f6af00abdc0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7f6ae800c7c0 tx=0x7f6ae8007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:48.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.659+0000 7f6ade7fc700 1 -- 172.21.15.121:0/3286890870 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8005d30 con 0x7f6af00ab940 2023-11-15T03:04:48.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.659+0000 7f6ade7fc700 1 -- 172.21.15.121:0/3286890870 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae8007110 con 0x7f6af00ab940 2023-11-15T03:04:48.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.659+0000 7f6af4969700 1 -- 172.21.15.121:0/3286890870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae00092a0 con 0x7f6af00ab940 2023-11-15T03:04:48.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.659+0000 7f6ade7fc700 1 -- 172.21.15.121:0/3286890870 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae80074d0 con 0x7f6af00ab940 2023-11-15T03:04:48.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.659+0000 7f6af4969700 1 -- 172.21.15.121:0/3286890870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af0164a50 con 0x7f6af00ab940 2023-11-15T03:04:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.659+0000 7f6af4969700 1 -- 172.21.15.121:0/3286890870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af006c150 con 0x7f6af00ab940 2023-11-15T03:04:48.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.659+0000 7f6ade7fc700 1 -- 172.21.15.121:0/3286890870 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6ae8007840 con 0x7f6af00ab940 2023-11-15T03:04:48.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.663+0000 7f6ade7fc700 1 --2- 172.21.15.121:0/3286890870 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6ad005a510 0x7f6ad005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:48.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.663+0000 7f6ade7fc700 1 -- 172.21.15.121:0/3286890870 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6868+0+0 (secure 0 0 0) 0x7f6ae8074940 con 0x7f6af00ab940 2023-11-15T03:04:48.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.663+0000 7f6aee59c700 1 --2- 172.21.15.121:0/3286890870 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6ad005a510 0x7f6ad005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:48.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.663+0000 7f6aee59c700 1 --2- 172.21.15.121:0/3286890870 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6ad005a510 0x7f6ad005c9d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f6af014b050 tx=0x7f6ae0017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:48.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.667+0000 7f6ade7fc700 1 -- 172.21.15.121:0/3286890870 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6ae8047d70 con 0x7f6af00ab940 2023-11-15T03:04:48.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.871+0000 7f6af4969700 1 -- 172.21.15.121:0/3286890870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6af0068ae0 con 0x7f6af00ab940 2023-11-15T03:04:48.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.871+0000 7f6ade7fc700 1 -- 172.21.15.121:0/3286890870 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f6ae804b390 con 0x7f6af00ab940 2023-11-15T03:04:48.879 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:04:48.879 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":126,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:04:48.028397+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8001},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8001":{"gid":8001,"name":"a","rank":0,"incarnation":126,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/3831023402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3831023402},{"type":"v1","addr":"172.21.15.121:6836","nonce":3831023402}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:04:48.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.875+0000 7f6af4969700 1 -- 172.21.15.121:0/3286890870 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6ad005a510 msgr2=0x7f6ad005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:48.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.875+0000 7f6af4969700 1 --2- 172.21.15.121:0/3286890870 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6ad005a510 0x7f6ad005c9d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f6af014b050 tx=0x7f6ae0017000).stop 2023-11-15T03:04:48.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.875+0000 7f6af4969700 1 -- 172.21.15.121:0/3286890870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6af00ab940 msgr2=0x7f6af00abdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:48.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.875+0000 7f6af4969700 1 --2- 172.21.15.121:0/3286890870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6af00ab940 0x7f6af00abdc0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7f6ae800c7c0 tx=0x7f6ae8007f30).stop 2023-11-15T03:04:48.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.875+0000 7f6af4969700 1 -- 172.21.15.121:0/3286890870 shutdown_connections 2023-11-15T03:04:48.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.875+0000 7f6af4969700 1 --2- 172.21.15.121:0/3286890870 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6ad005a510 0x7f6ad005c9d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:48.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.875+0000 7f6af4969700 1 --2- 172.21.15.121:0/3286890870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6af00ab940 0x7f6af00abdc0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:48.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.875+0000 7f6af4969700 1 --2- 172.21.15.121:0/3286890870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af0150c50 0x7f6af00c0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:48.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.875+0000 7f6af4969700 1 --2- 172.21.15.121:0/3286890870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af0150870 0x7f6af00bfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:48.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.875+0000 7f6af4969700 1 -- 172.21.15.121:0/3286890870 >> 172.21.15.121:0/3286890870 conn(0x7f6af0062190 msgr2=0x7f6af0066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:48.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.875+0000 7f6af4969700 1 -- 172.21.15.121:0/3286890870 shutdown_connections 2023-11-15T03:04:48.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:48.875+0000 7f6af4969700 1 -- 172.21.15.121:0/3286890870 wait complete. 2023-11-15T03:04:48.884 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 126 2023-11-15T03:04:48.904 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 126, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:04:48.028397+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 101, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8001}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8001': {'gid': 8001, 'name': 'a', 'rank': 0, 'incarnation': 126, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.121:6836/3831023402', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 3831023402}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 3831023402}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:04:48.904 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8001:up:replay 2023-11-15T03:04:48.905 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:04:51.206 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:04:51.199+0000 7f1f278dc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:04:54.311 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:04:54.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.463+0000 7fcf1968f700 1 Processor -- start 2023-11-15T03:04:54.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.463+0000 7fcf1968f700 1 -- start start 2023-11-15T03:04:54.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.467+0000 7fcf1968f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcf14151870 0x7fcf14151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.467+0000 7fcf1968f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcf140c30e0 0x7fcf140c3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.467+0000 7fcf1968f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf140657f0 con 0x7fcf14152140 2023-11-15T03:04:54.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.467+0000 7fcf1968f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf14065ad0 con 0x7fcf140c30e0 2023-11-15T03:04:54.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.467+0000 7fcf1968f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf140c3a50 con 0x7fcf14151870 2023-11-15T03:04:54.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.467+0000 7fcf13fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcf14151870 0x7fcf14151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.467+0000 7fcf13fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcf14151870 0x7fcf14151c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:46254/0 (socket says 172.21.15.121:46254) 2023-11-15T03:04:54.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.467+0000 7fcf137fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fcf14152140 0x7fcf140a9830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:36032/0 (socket says 172.21.15.121:36032) 2023-11-15T03:04:54.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.467+0000 7fcf13fff700 1 -- 172.21.15.121:0/243293742 learned_addr learned my addr 172.21.15.121:0/243293742 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:54.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.467+0000 7fcf18e8e700 1 --2- 172.21.15.121:0/243293742 >> v2:172.21.15.121:3300/0 conn(0x7fcf140c30e0 0x7fcf140c3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.467+0000 7fcf13fff700 1 -- 172.21.15.121:0/243293742 >> v2:172.21.15.121:3300/0 conn(0x7fcf140c30e0 msgr2=0x7fcf140c3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:54.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.467+0000 7fcf13fff700 1 --2- 172.21.15.121:0/243293742 >> v2:172.21.15.121:3300/0 conn(0x7fcf140c30e0 0x7fcf140c3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.467+0000 7fcf12ffd700 1 -- 172.21.15.121:0/243293742 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525784068 0 0) 0x7fcf140657f0 con 0x7fcf14152140 2023-11-15T03:04:54.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.467+0000 7fcf13fff700 1 -- 172.21.15.121:0/243293742 >> v1:172.21.15.121:6789/0 conn(0x7fcf14152140 legacy=0x7fcf140a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:54.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.467+0000 7fcf13fff700 1 -- 172.21.15.121:0/243293742 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf140aa260 con 0x7fcf14151870 2023-11-15T03:04:54.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.467+0000 7fcf13fff700 1 --2- 172.21.15.121:0/243293742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf14151870 0x7fcf14151c50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7fcf14067720 tx=0x7fcf04009840).ready entity=mon.1 client_cookie=fac2566aee698f48 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:54.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.471+0000 7fcf12ffd700 1 -- 172.21.15.121:0/243293742 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0400e070 con 0x7fcf14151870 2023-11-15T03:04:54.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.471+0000 7fcf12ffd700 1 -- 172.21.15.121:0/243293742 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0400ba20 con 0x7fcf14151870 2023-11-15T03:04:54.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.471+0000 7fcf12ffd700 1 -- 172.21.15.121:0/243293742 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0400bda0 con 0x7fcf14151870 2023-11-15T03:04:54.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.471+0000 7fcf1968f700 1 -- 172.21.15.121:0/243293742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf14151870 msgr2=0x7fcf14151c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:54.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.471+0000 7fcf1968f700 1 --2- 172.21.15.121:0/243293742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf14151870 0x7fcf14151c50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7fcf14067720 tx=0x7fcf04009840).stop 2023-11-15T03:04:54.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.471+0000 7fcf1968f700 1 -- 172.21.15.121:0/243293742 shutdown_connections 2023-11-15T03:04:54.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.471+0000 7fcf1968f700 1 --2- 172.21.15.121:0/243293742 >> v2:172.21.15.121:3300/0 conn(0x7fcf140c30e0 0x7fcf140c3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.471+0000 7fcf1968f700 1 --2- 172.21.15.121:0/243293742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf14151870 0x7fcf14151c50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.471+0000 7fcf1968f700 1 -- 172.21.15.121:0/243293742 >> 172.21.15.121:0/243293742 conn(0x7fcf14062190 msgr2=0x7fcf14062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:54.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.471+0000 7fcf1968f700 1 -- 172.21.15.121:0/243293742 shutdown_connections 2023-11-15T03:04:54.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.471+0000 7fcf1968f700 1 -- 172.21.15.121:0/243293742 wait complete. 2023-11-15T03:04:54.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.471+0000 7fcf1968f700 1 Processor -- start 2023-11-15T03:04:54.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf1968f700 1 -- start start 2023-11-15T03:04:54.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf1968f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf140c30e0 0x7fcf1414a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf1968f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf14151870 0x7fcf141483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf1968f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf14152140 0x7fcf141431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf1968f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1414a8d0 con 0x7fcf14152140 2023-11-15T03:04:54.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf1968f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf14149740 con 0x7fcf140c30e0 2023-11-15T03:04:54.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf1968f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf141488e0 con 0x7fcf14151870 2023-11-15T03:04:54.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf18e8e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf14152140 0x7fcf141431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf18e8e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf14152140 0x7fcf141431c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33898/0 (socket says 172.21.15.121:33898) 2023-11-15T03:04:54.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf18e8e700 1 -- 172.21.15.121:0/2099950079 learned_addr learned my addr 172.21.15.121:0/2099950079 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:54.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf13fff700 1 --2- 172.21.15.121:0/2099950079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf140c30e0 0x7fcf1414a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf137fe700 1 --2- 172.21.15.121:0/2099950079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf14151870 0x7fcf141483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf18e8e700 1 -- 172.21.15.121:0/2099950079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf14151870 msgr2=0x7fcf141483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:54.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf18e8e700 1 --2- 172.21.15.121:0/2099950079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf14151870 0x7fcf141483f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf18e8e700 1 -- 172.21.15.121:0/2099950079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf140c30e0 msgr2=0x7fcf1414a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:54.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf18e8e700 1 --2- 172.21.15.121:0/2099950079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf140c30e0 0x7fcf1414a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf18e8e700 1 -- 172.21.15.121:0/2099950079 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf04009ae0 con 0x7fcf14152140 2023-11-15T03:04:54.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf137fe700 1 --2- 172.21.15.121:0/2099950079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf14151870 0x7fcf141483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:04:54.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf13fff700 1 --2- 172.21.15.121:0/2099950079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf140c30e0 0x7fcf1414a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:54.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf18e8e700 1 --2- 172.21.15.121:0/2099950079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf14152140 0x7fcf141431c0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7fcf0c00e8d0 tx=0x7fcf0c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:54.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcefbfff700 1 -- 172.21.15.121:0/2099950079 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c00ee70 con 0x7fcf14152140 2023-11-15T03:04:54.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf1968f700 1 -- 172.21.15.121:0/2099950079 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf14148b30 con 0x7fcf14152140 2023-11-15T03:04:54.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcefbfff700 1 -- 172.21.15.121:0/2099950079 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c00c850 con 0x7fcf14152140 2023-11-15T03:04:54.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcefbfff700 1 -- 172.21.15.121:0/2099950079 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c00cbd0 con 0x7fcf14152140 2023-11-15T03:04:54.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.475+0000 7fcf1968f700 1 -- 172.21.15.121:0/2099950079 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf141490f0 con 0x7fcf14152140 2023-11-15T03:04:54.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.479+0000 7fcf1968f700 1 -- 172.21.15.121:0/2099950079 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1406bcf0 con 0x7fcf14152140 2023-11-15T03:04:54.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.479+0000 7fcefbfff700 1 -- 172.21.15.121:0/2099950079 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fcf0c01b400 con 0x7fcf14152140 2023-11-15T03:04:54.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.479+0000 7fcefbfff700 1 --2- 172.21.15.121:0/2099950079 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcf0005a510 0x7fcf0005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.479+0000 7fcefbfff700 1 -- 172.21.15.121:0/2099950079 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6868+0+0 (secure 0 0 0) 0x7fcf0c0751c0 con 0x7fcf14152140 2023-11-15T03:04:54.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.483+0000 7fcf13fff700 1 --2- 172.21.15.121:0/2099950079 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcf0005a510 0x7fcf0005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.483+0000 7fcf13fff700 1 --2- 172.21.15.121:0/2099950079 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcf0005a510 0x7fcf0005c9d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7fcf140c35f0 tx=0x7fcf14065a40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:54.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.487+0000 7fcefbfff700 1 -- 172.21.15.121:0/2099950079 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcf0c048640 con 0x7fcf14152140 2023-11-15T03:04:54.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.683+0000 7fcf1968f700 1 -- 172.21.15.121:0/2099950079 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf14068a70 con 0x7fcf14152140 2023-11-15T03:04:54.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.683+0000 7fcefbfff700 1 -- 172.21.15.121:0/2099950079 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 129 v129) v1 ==== 95+0+4505 (secure 0 0 0) 0x7fcf0c016020 con 0x7fcf14152140 2023-11-15T03:04:54.692 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:04:54.692 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":129,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:04:51.060624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8001},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8001":{"gid":8001,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":4,"addr":"172.21.15.121:6836/3831023402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3831023402},{"type":"v1","addr":"172.21.15.121:6836","nonce":3831023402}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:04:54.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.687+0000 7fcf1968f700 1 -- 172.21.15.121:0/2099950079 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcf0005a510 msgr2=0x7fcf0005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:54.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.687+0000 7fcf1968f700 1 --2- 172.21.15.121:0/2099950079 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcf0005a510 0x7fcf0005c9d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7fcf140c35f0 tx=0x7fcf14065a40).stop 2023-11-15T03:04:54.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.687+0000 7fcf1968f700 1 -- 172.21.15.121:0/2099950079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf14152140 msgr2=0x7fcf141431c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:54.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.687+0000 7fcf1968f700 1 --2- 172.21.15.121:0/2099950079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf14152140 0x7fcf141431c0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7fcf0c00e8d0 tx=0x7fcf0c009060).stop 2023-11-15T03:04:54.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.687+0000 7fcf1968f700 1 -- 172.21.15.121:0/2099950079 shutdown_connections 2023-11-15T03:04:54.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.687+0000 7fcf1968f700 1 --2- 172.21.15.121:0/2099950079 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcf0005a510 0x7fcf0005c9d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.687+0000 7fcf1968f700 1 --2- 172.21.15.121:0/2099950079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf14152140 0x7fcf141431c0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.687+0000 7fcf1968f700 1 --2- 172.21.15.121:0/2099950079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf14151870 0x7fcf141483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.687+0000 7fcf1968f700 1 --2- 172.21.15.121:0/2099950079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf140c30e0 0x7fcf1414a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.687+0000 7fcf1968f700 1 -- 172.21.15.121:0/2099950079 >> 172.21.15.121:0/2099950079 conn(0x7fcf14062190 msgr2=0x7fcf14150b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:54.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.687+0000 7fcf1968f700 1 -- 172.21.15.121:0/2099950079 shutdown_connections 2023-11-15T03:04:54.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.691+0000 7fcf1968f700 1 -- 172.21.15.121:0/2099950079 wait complete. 2023-11-15T03:04:54.699 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 129 2023-11-15T03:04:54.717 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 129, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:04:51.060624+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 101, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8001}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8001': {'gid': 8001, 'name': 'a', 'rank': 0, 'incarnation': 126, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.121:6836/3831023402', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 3831023402}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 3831023402}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:04:54.717 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T03:04:54.718 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-15T03:04:54.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.875+0000 7f256adf9700 1 Processor -- start 2023-11-15T03:04:54.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.879+0000 7f256adf9700 1 -- start start 2023-11-15T03:04:54.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.879+0000 7f256adf9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f25640ab0d0 0x7f25640ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.879+0000 7f256adf9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25640ad1b0 0x7f2564065710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.879+0000 7f256adf9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2564066c90 con 0x7f2564065c00 2023-11-15T03:04:54.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.879+0000 7f256adf9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2564066080 con 0x7f25640ab0d0 2023-11-15T03:04:54.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.879+0000 7f256adf9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2564066360 con 0x7f25640ad1b0 2023-11-15T03:04:54.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.879+0000 7f256a5f8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f2564065c00 0x7f25640c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53974/0 (socket says 172.21.15.121:53974) 2023-11-15T03:04:54.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.879+0000 7f256a5f8700 1 -- 172.21.15.121:0/4154101302 learned_addr learned my addr 172.21.15.121:0/4154101302 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:54.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.879+0000 7f25695f6700 1 --2- 172.21.15.121:0/4154101302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25640ad1b0 0x7f2564065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.879+0000 7f2569df7700 1 --2- 172.21.15.121:0/4154101302 >> v2:172.21.15.190:3300/0 conn(0x7f25640ab0d0 0x7f25640ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.879+0000 7f2568df5700 1 -- 172.21.15.121:0/4154101302 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994764619 0 0) 0x7f2564066c90 con 0x7f2564065c00 2023-11-15T03:04:54.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.879+0000 7f2568df5700 1 -- 172.21.15.121:0/4154101302 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25500033f0 con 0x7f2564065c00 2023-11-15T03:04:54.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.879+0000 7f2569df7700 1 -- 172.21.15.121:0/4154101302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25640ad1b0 msgr2=0x7f2564065710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:54.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.879+0000 7f2569df7700 1 --2- 172.21.15.121:0/4154101302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25640ad1b0 0x7f2564065710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.879+0000 7f2569df7700 1 -- 172.21.15.121:0/4154101302 >> v1:172.21.15.121:6789/0 conn(0x7f2564065c00 legacy=0x7f25640c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:54.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.879+0000 7f2569df7700 1 -- 172.21.15.121:0/4154101302 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2564066640 con 0x7f25640ab0d0 2023-11-15T03:04:54.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.883+0000 7f2569df7700 1 --2- 172.21.15.121:0/4154101302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25640ab0d0 0x7f25640ab4b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f2560006fd0 tx=0x7f2560006f40).ready entity=mon.1 client_cookie=c9a46e7c6e1504d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:54.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.883+0000 7f2568df5700 1 -- 172.21.15.121:0/4154101302 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2560005880 con 0x7f25640ab0d0 2023-11-15T03:04:54.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.883+0000 7f2568df5700 1 -- 172.21.15.121:0/4154101302 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25600059e0 con 0x7f25640ab0d0 2023-11-15T03:04:54.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.883+0000 7f2568df5700 1 -- 172.21.15.121:0/4154101302 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2560005d60 con 0x7f25640ab0d0 2023-11-15T03:04:54.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.883+0000 7f256adf9700 1 -- 172.21.15.121:0/4154101302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25640ab0d0 msgr2=0x7f25640ab4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:54.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.883+0000 7f256adf9700 1 --2- 172.21.15.121:0/4154101302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25640ab0d0 0x7f25640ab4b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f2560006fd0 tx=0x7f2560006f40).stop 2023-11-15T03:04:54.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.883+0000 7f256adf9700 1 -- 172.21.15.121:0/4154101302 shutdown_connections 2023-11-15T03:04:54.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.883+0000 7f256adf9700 1 --2- 172.21.15.121:0/4154101302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25640ad1b0 0x7f2564065710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.883+0000 7f256adf9700 1 --2- 172.21.15.121:0/4154101302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25640ab0d0 0x7f25640ab4b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.883+0000 7f256adf9700 1 -- 172.21.15.121:0/4154101302 >> 172.21.15.121:0/4154101302 conn(0x7f2564067a60 msgr2=0x7f2564067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:54.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.883+0000 7f256adf9700 1 -- 172.21.15.121:0/4154101302 shutdown_connections 2023-11-15T03:04:54.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.883+0000 7f256adf9700 1 -- 172.21.15.121:0/4154101302 wait complete. 2023-11-15T03:04:54.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.887+0000 7f256adf9700 1 Processor -- start 2023-11-15T03:04:54.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.887+0000 7f256adf9700 1 -- start start 2023-11-15T03:04:54.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.887+0000 7f256adf9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2564065c00 0x7f25640c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.887+0000 7f256adf9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25640ab0d0 0x7f25640a8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.887+0000 7f256adf9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25640ad1b0 0x7f25640a9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.887+0000 7f256adf9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25640a9640 con 0x7f2564065c00 2023-11-15T03:04:54.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.887+0000 7f256adf9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25640a5150 con 0x7f25640ad1b0 2023-11-15T03:04:54.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.887+0000 7f256adf9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25640a4540 con 0x7f25640ab0d0 2023-11-15T03:04:54.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.887+0000 7f2569df7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2564065c00 0x7f25640c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.887+0000 7f2569df7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2564065c00 0x7f25640c63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34046/0 (socket says 172.21.15.121:34046) 2023-11-15T03:04:54.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.887+0000 7f2569df7700 1 -- 172.21.15.121:0/2303303939 learned_addr learned my addr 172.21.15.121:0/2303303939 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:54.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.887+0000 7f256a5f8700 1 --2- 172.21.15.121:0/2303303939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25640ad1b0 0x7f25640a9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.887+0000 7f2569df7700 1 -- 172.21.15.121:0/2303303939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25640ab0d0 msgr2=0x7f25640a8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:54.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.887+0000 7f25695f6700 1 --2- 172.21.15.121:0/2303303939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25640ab0d0 0x7f25640a8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.891+0000 7f2569df7700 1 --2- 172.21.15.121:0/2303303939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25640ab0d0 0x7f25640a8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.891+0000 7f2569df7700 1 -- 172.21.15.121:0/2303303939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25640ad1b0 msgr2=0x7f25640a9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:54.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.891+0000 7f2569df7700 1 --2- 172.21.15.121:0/2303303939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25640ad1b0 0x7f25640a9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.891+0000 7f2569df7700 1 -- 172.21.15.121:0/2303303939 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2560009250 con 0x7f2564065c00 2023-11-15T03:04:54.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.891+0000 7f256a5f8700 1 --2- 172.21.15.121:0/2303303939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25640ad1b0 0x7f25640a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:04:54.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.891+0000 7f25695f6700 1 --2- 172.21.15.121:0/2303303939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25640ab0d0 0x7f25640a8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:04:54.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.891+0000 7f2569df7700 1 --2- 172.21.15.121:0/2303303939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2564065c00 0x7f25640c63b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7f2560006f80 tx=0x7f2560015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:54.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.891+0000 7f2559ffb700 1 -- 172.21.15.121:0/2303303939 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256000d6b0 con 0x7f2564065c00 2023-11-15T03:04:54.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.891+0000 7f2559ffb700 1 -- 172.21.15.121:0/2303303939 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256000d810 con 0x7f2564065c00 2023-11-15T03:04:54.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.891+0000 7f256adf9700 1 -- 172.21.15.121:0/2303303939 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25640a4820 con 0x7f2564065c00 2023-11-15T03:04:54.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.891+0000 7f2559ffb700 1 -- 172.21.15.121:0/2303303939 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256000dbd0 con 0x7f2564065c00 2023-11-15T03:04:54.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.891+0000 7f256adf9700 1 -- 172.21.15.121:0/2303303939 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25640a4de0 con 0x7f2564065c00 2023-11-15T03:04:54.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.895+0000 7f2559ffb700 1 -- 172.21.15.121:0/2303303939 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f25600113f0 con 0x7f2564065c00 2023-11-15T03:04:54.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.895+0000 7f2559ffb700 1 --2- 172.21.15.121:0/2303303939 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f254805a510 0x7f254805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.895+0000 7f2559ffb700 1 -- 172.21.15.121:0/2303303939 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6868+0+0 (secure 0 0 0) 0x7f2560076140 con 0x7f2564065c00 2023-11-15T03:04:54.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f25695f6700 1 --2- 172.21.15.121:0/2303303939 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f254805a510 0x7f254805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f25763e0740 1 Processor -- start 2023-11-15T03:04:54.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f25763e0740 1 -- start start 2023-11-15T03:04:54.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f25695f6700 1 --2- 172.21.15.121:0/2303303939 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f254805a510 0x7f254805c9d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f2554006d80 tx=0x7f2554008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:54.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f25763e0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4ad0 0x16fb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f25763e0740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x16d4eb0 0x1721140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f25763e0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1721630 0x1721ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f25763e0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1655410 con 0x16d4ad0 2023-11-15T03:04:54.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f25763e0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1655550 con 0x1721630 2023-11-15T03:04:54.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f25763e0740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1655a00 con 0x16d4eb0 2023-11-15T03:04:54.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f2569df7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4ad0 0x16fb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f2569df7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4ad0 0x16fb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34052/0 (socket says 172.21.15.121:34052) 2023-11-15T03:04:54.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f2569df7700 1 -- 172.21.15.121:0/2457020138 learned_addr learned my addr 172.21.15.121:0/2457020138 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:54.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f2569df7700 1 -- 172.21.15.121:0/2457020138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x16d4eb0 msgr2=0x1721140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:54.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f2569df7700 1 --2- 172.21.15.121:0/2457020138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x16d4eb0 0x1721140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:04:54.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f2569df7700 1 -- 172.21.15.121:0/2457020138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1721630 msgr2=0x1721ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:54.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f256a5f8700 1 --2- 172.21.15.121:0/2457020138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1721630 0x1721ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f2569df7700 1 --2- 172.21.15.121:0/2457020138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1721630 0x1721ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f2569df7700 1 -- 172.21.15.121:0/2457020138 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x164e730 con 0x16d4ad0 2023-11-15T03:04:54.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f256a5f8700 1 --2- 172.21.15.121:0/2457020138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1721630 0x1721ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:04:54.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f2569df7700 1 --2- 172.21.15.121:0/2457020138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4ad0 0x16fb090 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7f256004d070 tx=0x7f256000e000).ready entity=mon.0 client_cookie=f1340bd05b1ae552 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:54.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f254effd700 1 -- 172.21.15.121:0/2457020138 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256004d170 con 0x16d4ad0 2023-11-15T03:04:54.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f254effd700 1 -- 172.21.15.121:0/2457020138 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2560052910 con 0x16d4ad0 2023-11-15T03:04:54.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.899+0000 7f254effd700 1 -- 172.21.15.121:0/2457020138 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25600498e0 con 0x16d4ad0 2023-11-15T03:04:54.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.903+0000 7f25763e0740 1 -- 172.21.15.121:0/2457020138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4ad0 msgr2=0x16fb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:54.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.903+0000 7f25763e0740 1 --2- 172.21.15.121:0/2457020138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4ad0 0x16fb090 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7f256004d070 tx=0x7f256000e000).stop 2023-11-15T03:04:54.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.903+0000 7f25763e0740 1 -- 172.21.15.121:0/2457020138 shutdown_connections 2023-11-15T03:04:54.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.903+0000 7f25763e0740 1 --2- 172.21.15.121:0/2457020138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1721630 0x1721ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.903+0000 7f25763e0740 1 --2- 172.21.15.121:0/2457020138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x16d4eb0 0x1721140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:04:54.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.903+0000 7f25763e0740 1 --2- 172.21.15.121:0/2457020138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4ad0 0x16fb090 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.903+0000 7f25763e0740 1 -- 172.21.15.121:0/2457020138 >> 172.21.15.121:0/2457020138 conn(0x14b3550 msgr2=0x16d37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:54.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.903+0000 7f25763e0740 1 -- 172.21.15.121:0/2457020138 shutdown_connections 2023-11-15T03:04:54.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.903+0000 7f25763e0740 1 -- 172.21.15.121:0/2457020138 wait complete. 2023-11-15T03:04:54.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.903+0000 7f25763e0740 1 -- start start 2023-11-15T03:04:54.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.903+0000 7f25763e0740 1 Processor -- start 2023-11-15T03:04:54.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f25763e0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x16d4ad0 0x1715790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f25763e0740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x16d4eb0 0x16fefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f25763e0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1721630 0x16fbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f25763e0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1620650 con 0x1721630 2023-11-15T03:04:54.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f25763e0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x16fc7e0 con 0x16d4ad0 2023-11-15T03:04:54.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f25763e0740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x17144a0 con 0x16d4eb0 2023-11-15T03:04:54.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f25763e0740 10 client.0 ll_register_callbacks cb 0x168d020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:04:54.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f256a5f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1721630 0x16fbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f256a5f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1721630 0x16fbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34068/0 (socket says 172.21.15.121:34068) 2023-11-15T03:04:54.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f256a5f8700 1 -- 172.21.15.121:0/1506495129 learned_addr learned my addr 172.21.15.121:0/1506495129 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:54.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f2569df7700 1 --2- 172.21.15.121:0/1506495129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x16d4ad0 0x1715790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f25695f6700 1 --2- 172.21.15.121:0/1506495129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x16d4eb0 0x16fefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f256a5f8700 1 -- 172.21.15.121:0/1506495129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x16d4eb0 msgr2=0x16fefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:54.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f256a5f8700 1 --2- 172.21.15.121:0/1506495129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x16d4eb0 0x16fefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f256a5f8700 1 -- 172.21.15.121:0/1506495129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x16d4ad0 msgr2=0x1715790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:54.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f256a5f8700 1 --2- 172.21.15.121:0/1506495129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x16d4ad0 0x1715790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f256a5f8700 1 -- 172.21.15.121:0/1506495129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25640a4de0 con 0x1721630 2023-11-15T03:04:54.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f256a5f8700 1 --2- 172.21.15.121:0/1506495129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1721630 0x16fbb60 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7f255c00e4f0 tx=0x7f255c009040).ready entity=mon.0 client_cookie=f8bc069a1dbec970 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:54.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f2543fff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:04:54.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f2543fff700 1 -- 172.21.15.121:0/1506495129 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c005b00 con 0x1721630 2023-11-15T03:04:54.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f2543fff700 1 -- 172.21.15.121:0/1506495129 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 6868+0+0 (secure 0 0 0) 0x7f255c005c60 con 0x1721630 2023-11-15T03:04:54.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f25763e0740 1 -- 172.21.15.121:0/1506495129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x164b1a0 con 0x1721630 2023-11-15T03:04:54.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f2543fff700 1 -- 172.21.15.121:0/1506495129 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7f255c005e20 con 0x1721630 2023-11-15T03:04:54.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f25763e0740 10 client.8058 fetch_fsmap learned FSMap version 129 2023-11-15T03:04:54.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f25763e0740 1 -- 172.21.15.121:0/1506495129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x1649960 con 0x1721630 2023-11-15T03:04:54.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f2543fff700 1 -- 172.21.15.121:0/1506495129 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x164b1a0 con 0x1721630 2023-11-15T03:04:54.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f2543fff700 1 -- 172.21.15.121:0/1506495129 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 129) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f255c0072e0 con 0x1721630 2023-11-15T03:04:54.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f25763e0740 10 client.8058 fetch_fsmap finished waiting for FSMap version 129 2023-11-15T03:04:54.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f25763e0740 10 client.8058 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:04:54.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f25763e0740 20 client.8058 populate_metadata read hostname 'smithi121' 2023-11-15T03:04:54.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f25763e0740 1 --2- 172.21.15.121:0/1506495129 >> [v2:172.21.15.121:6834/3831023402,v1:172.21.15.121:6836/3831023402] conn(0x16fc050 0x1701030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f25763e0740 4 client.8058 mds_command: new command op to 8001 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:04:54.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.907+0000 7f25763e0740 1 -- 172.21.15.121:0/1506495129 --> [v2:172.21.15.121:6834/3831023402,v1:172.21.15.121:6836/3831023402] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x16416f0 con 0x16fc050 2023-11-15T03:04:54.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.911+0000 7f2569df7700 1 --2- 172.21.15.121:0/1506495129 >> [v2:172.21.15.121:6834/3831023402,v1:172.21.15.121:6836/3831023402] conn(0x16fc050 0x1701030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.911+0000 7f2569df7700 1 --2- 172.21.15.121:0/1506495129 >> [v2:172.21.15.121:6834/3831023402,v1:172.21.15.121:6836/3831023402] conn(0x16fc050 0x1701030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=6691c87a3fbc12b8 server_cookie=de23fba6a030a7a0 in_seq=0 out_seq=0 2023-11-15T03:04:54.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.911+0000 7f2543fff700 10 client.8058 ms_handle_connect on v2:172.21.15.121:6834/3831023402 2023-11-15T03:04:54.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.911+0000 7f2543fff700 1 -- 172.21.15.121:0/1506495129 <== mds.0 v2:172.21.15.121:6834/3831023402 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x16416f0 con 0x16fc050 2023-11-15T03:04:54.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.911+0000 7f2543fff700 10 client.8058 handle_command_reply: tid=0 2023-11-15T03:04:54.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.911+0000 7f25763e0740 1 client.8058 shutdown 2023-11-15T03:04:54.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.911+0000 7f25763e0740 1 -- 172.21.15.121:0/1506495129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1721630 msgr2=0x16fbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:54.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.911+0000 7f25763e0740 1 --2- 172.21.15.121:0/1506495129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1721630 0x16fbb60 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7f255c00e4f0 tx=0x7f255c009040).stop 2023-11-15T03:04:54.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.911+0000 7f25763e0740 1 -- 172.21.15.121:0/1506495129 shutdown_connections 2023-11-15T03:04:54.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.911+0000 7f25763e0740 1 --2- 172.21.15.121:0/1506495129 >> [v2:172.21.15.121:6834/3831023402,v1:172.21.15.121:6836/3831023402] conn(0x16fc050 0x1701030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.911+0000 7f25763e0740 1 --2- 172.21.15.121:0/1506495129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1721630 0x16fbb60 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.911+0000 7f25763e0740 1 --2- 172.21.15.121:0/1506495129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x16d4eb0 0x16fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.911+0000 7f25763e0740 1 --2- 172.21.15.121:0/1506495129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x16d4ad0 0x1715790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.911+0000 7f25763e0740 1 -- 172.21.15.121:0/1506495129 >> 172.21.15.121:0/1506495129 conn(0x14b3550 msgr2=0x171c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:54.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.911+0000 7f2543fff700 0 client.8058 ms_handle_reset on v2:172.21.15.121:6834/3831023402 2023-11-15T03:04:54.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.911+0000 7f25763e0740 1 -- 172.21.15.121:0/1506495129 shutdown_connections 2023-11-15T03:04:54.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.911+0000 7f25763e0740 1 -- 172.21.15.121:0/1506495129 wait complete. 2023-11-15T03:04:54.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.911+0000 7f25763e0740 20 client.8058 trim_cache size 0 max 16384 2023-11-15T03:04:54.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f25763e0740 1 Processor -- start 2023-11-15T03:04:54.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f25763e0740 1 -- start start 2023-11-15T03:04:54.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f25763e0740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x16d4ad0 0x16ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f25763e0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4eb0 0x1701000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f25763e0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1721630 0x17014f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f25763e0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x16feda0 con 0x16d4eb0 2023-11-15T03:04:54.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f25763e0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1624c50 con 0x1721630 2023-11-15T03:04:54.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f25763e0740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x17019e0 con 0x16d4ad0 2023-11-15T03:04:54.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f25695f6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4eb0 0x1701000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f25695f6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4eb0 0x1701000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34080/0 (socket says 172.21.15.121:34080) 2023-11-15T03:04:54.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f25695f6700 1 -- 172.21.15.121:0/1717259382 learned_addr learned my addr 172.21.15.121:0/1717259382 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:54.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f2569df7700 1 --2- 172.21.15.121:0/1717259382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x16d4ad0 0x16ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f256a5f8700 1 --2- 172.21.15.121:0/1717259382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1721630 0x17014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f25695f6700 1 -- 172.21.15.121:0/1717259382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x16d4ad0 msgr2=0x16ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:54.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f25695f6700 1 --2- 172.21.15.121:0/1717259382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x16d4ad0 0x16ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f25695f6700 1 -- 172.21.15.121:0/1717259382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1721630 msgr2=0x17014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:54.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f25695f6700 1 --2- 172.21.15.121:0/1717259382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1721630 0x17014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f25695f6700 1 -- 172.21.15.121:0/1717259382 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2564154c10 con 0x16d4eb0 2023-11-15T03:04:54.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f256a5f8700 1 --2- 172.21.15.121:0/1717259382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1721630 0x17014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:54.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.931+0000 7f25695f6700 1 --2- 172.21.15.121:0/1717259382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4eb0 0x1701000 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7f2554010850 tx=0x7f2554017040).ready entity=mon.0 client_cookie=c1119e1452ff444f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:54.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.935+0000 7f254f7fe700 1 -- 172.21.15.121:0/1717259382 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554014870 con 0x16d4eb0 2023-11-15T03:04:54.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.935+0000 7f254f7fe700 1 -- 172.21.15.121:0/1717259382 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25540149d0 con 0x16d4eb0 2023-11-15T03:04:54.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.935+0000 7f254f7fe700 1 -- 172.21.15.121:0/1717259382 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554014d50 con 0x16d4eb0 2023-11-15T03:04:54.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.935+0000 7f25763e0740 1 -- 172.21.15.121:0/1717259382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4eb0 msgr2=0x1701000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:54.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.935+0000 7f25763e0740 1 --2- 172.21.15.121:0/1717259382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4eb0 0x1701000 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7f2554010850 tx=0x7f2554017040).stop 2023-11-15T03:04:54.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.935+0000 7f25763e0740 1 -- 172.21.15.121:0/1717259382 shutdown_connections 2023-11-15T03:04:54.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.935+0000 7f25763e0740 1 --2- 172.21.15.121:0/1717259382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1721630 0x17014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.935+0000 7f25763e0740 1 --2- 172.21.15.121:0/1717259382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4eb0 0x1701000 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.935+0000 7f25763e0740 1 --2- 172.21.15.121:0/1717259382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x16d4ad0 0x16ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.935+0000 7f25763e0740 1 -- 172.21.15.121:0/1717259382 >> 172.21.15.121:0/1717259382 conn(0x14b3550 msgr2=0x1712150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:54.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.935+0000 7f25763e0740 1 -- 172.21.15.121:0/1717259382 shutdown_connections 2023-11-15T03:04:54.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.935+0000 7f25763e0740 1 -- 172.21.15.121:0/1717259382 wait complete. 2023-11-15T03:04:54.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f25763e0740 1 -- start start 2023-11-15T03:04:54.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f25763e0740 1 Processor -- start 2023-11-15T03:04:54.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f25763e0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x16d4ad0 0x177d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f25763e0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4eb0 0x177d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f25763e0740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1721630 0x171f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f25763e0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x17204b0 con 0x16d4eb0 2023-11-15T03:04:54.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f25763e0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x16ee660 con 0x16d4ad0 2023-11-15T03:04:54.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f25763e0740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x16eeaf0 con 0x1721630 2023-11-15T03:04:54.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f25763e0740 10 client.0 ll_register_callbacks cb 0x7f255c001ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:04:54.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f25695f6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4eb0 0x177d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f25695f6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4eb0 0x177d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34092/0 (socket says 172.21.15.121:34092) 2023-11-15T03:04:54.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f25695f6700 1 -- 172.21.15.121:0/2876160052 learned_addr learned my addr 172.21.15.121:0/2876160052 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:54.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f2569df7700 1 --2- 172.21.15.121:0/2876160052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x16d4ad0 0x177d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f256a5f8700 1 --2- 172.21.15.121:0/2876160052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1721630 0x171f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f25695f6700 1 -- 172.21.15.121:0/2876160052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1721630 msgr2=0x171f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:54.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f25695f6700 1 --2- 172.21.15.121:0/2876160052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1721630 0x171f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f25695f6700 1 -- 172.21.15.121:0/2876160052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x16d4ad0 msgr2=0x177d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:54.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f25695f6700 1 --2- 172.21.15.121:0/2876160052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x16d4ad0 0x177d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f25695f6700 1 -- 172.21.15.121:0/2876160052 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255400e030 con 0x16d4eb0 2023-11-15T03:04:54.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f25695f6700 1 --2- 172.21.15.121:0/2876160052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4eb0 0x177d9c0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 rx=0x7f25540099e0 tx=0x7f2554018000).ready entity=mon.0 client_cookie=f140fee97f3ef599 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:54.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.939+0000 7f2543fff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:04:54.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f2543fff700 1 -- 172.21.15.121:0/2876160052 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554014870 con 0x16d4eb0 2023-11-15T03:04:54.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f2543fff700 1 -- 172.21.15.121:0/2876160052 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 6868+0+0 (secure 0 0 0) 0x7f25540218a0 con 0x16d4eb0 2023-11-15T03:04:54.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 1 -- 172.21.15.121:0/2876160052 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2560052910 con 0x16d4eb0 2023-11-15T03:04:54.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f2543fff700 1 -- 172.21.15.121:0/2876160052 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2554021c70 con 0x16d4eb0 2023-11-15T03:04:54.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 10 client.8064 fetch_fsmap learned FSMap version 129 2023-11-15T03:04:54.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 1 -- 172.21.15.121:0/2876160052 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x16496b0 con 0x16d4eb0 2023-11-15T03:04:54.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f2543fff700 1 -- 172.21.15.121:0/2876160052 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2560052910 con 0x16d4eb0 2023-11-15T03:04:54.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f2543fff700 1 -- 172.21.15.121:0/2876160052 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 129) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f2554031260 con 0x16d4eb0 2023-11-15T03:04:54.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 10 client.8064 fetch_fsmap finished waiting for FSMap version 129 2023-11-15T03:04:54.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 10 client.8064 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:04:54.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 20 client.8064 populate_metadata read hostname 'smithi121' 2023-11-15T03:04:54.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 1 --2- 172.21.15.121:0/2876160052 >> [v2:172.21.15.121:6834/3831023402,v1:172.21.15.121:6836/3831023402] conn(0x177deb0 0x171f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:54.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 4 client.8064 mds_command: new command op to 8001 tid=0[{"prefix": "status"}] 2023-11-15T03:04:54.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 1 -- 172.21.15.121:0/2876160052 --> [v2:172.21.15.121:6834/3831023402,v1:172.21.15.121:6836/3831023402] -- command(tid 0: {"prefix": "status"}) v1 -- 0x150a4e0 con 0x177deb0 2023-11-15T03:04:54.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f2569df7700 1 --2- 172.21.15.121:0/2876160052 >> [v2:172.21.15.121:6834/3831023402,v1:172.21.15.121:6836/3831023402] conn(0x177deb0 0x171f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:54.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f2569df7700 1 --2- 172.21.15.121:0/2876160052 >> [v2:172.21.15.121:6834/3831023402,v1:172.21.15.121:6836/3831023402] conn(0x177deb0 0x171f850 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=14b1478cb2dc2ca3 server_cookie=965a4f8bacd249f in_seq=0 out_seq=0 2023-11-15T03:04:54.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f2543fff700 10 client.8064 ms_handle_connect on v2:172.21.15.121:6834/3831023402 2023-11-15T03:04:54.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f2543fff700 1 -- 172.21.15.121:0/2876160052 <== mds.0 v2:172.21.15.121:6834/3831023402 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x150a4e0 con 0x177deb0 2023-11-15T03:04:54.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f2543fff700 10 client.8064 handle_command_reply: tid=0 2023-11-15T03:04:54.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 1 client.8064 shutdown 2023-11-15T03:04:54.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 1 -- 172.21.15.121:0/2876160052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4eb0 msgr2=0x177d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:54.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 1 --2- 172.21.15.121:0/2876160052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4eb0 0x177d9c0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 rx=0x7f25540099e0 tx=0x7f2554018000).stop 2023-11-15T03:04:54.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 1 -- 172.21.15.121:0/2876160052 shutdown_connections 2023-11-15T03:04:54.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 1 --2- 172.21.15.121:0/2876160052 >> [v2:172.21.15.121:6834/3831023402,v1:172.21.15.121:6836/3831023402] conn(0x177deb0 0x171f850 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 1 --2- 172.21.15.121:0/2876160052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1721630 0x171f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 1 --2- 172.21.15.121:0/2876160052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x16d4eb0 0x177d9c0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 1 --2- 172.21.15.121:0/2876160052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x16d4ad0 0x177d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 1 -- 172.21.15.121:0/2876160052 >> 172.21.15.121:0/2876160052 conn(0x14b3550 msgr2=0x16ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:54.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f2543fff700 0 client.8064 ms_handle_reset on v2:172.21.15.121:6834/3831023402 2023-11-15T03:04:54.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 1 -- 172.21.15.121:0/2876160052 shutdown_connections 2023-11-15T03:04:54.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 1 -- 172.21.15.121:0/2876160052 wait complete. 2023-11-15T03:04:54.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.943+0000 7f25763e0740 20 client.8064 trim_cache size 0 max 16384 2023-11-15T03:04:54.966 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:04:54.966 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T03:04:54.966 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T03:04:54.967 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 8001, 2023-11-15T03:04:54.967 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T03:04:54.967 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T03:04:54.967 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T03:04:54.967 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 6.9141538679999996, 2023-11-15T03:04:54.968 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 129, 2023-11-15T03:04:54.968 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 101, 2023-11-15T03:04:54.968 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 101, 2023-11-15T03:04:54.968 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 9.7171583520000002 2023-11-15T03:04:54.969 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:04:54.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.947+0000 7f256adf9700 1 -- 172.21.15.121:0/2303303939 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f254805a510 msgr2=0x7f254805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:54.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.947+0000 7f256adf9700 1 --2- 172.21.15.121:0/2303303939 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f254805a510 0x7f254805c9d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f2554006d80 tx=0x7f2554008040).stop 2023-11-15T03:04:54.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.947+0000 7f256adf9700 1 -- 172.21.15.121:0/2303303939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2564065c00 msgr2=0x7f25640c63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:54.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.947+0000 7f256adf9700 1 --2- 172.21.15.121:0/2303303939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2564065c00 0x7f25640c63b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7f2560006f80 tx=0x7f2560015000).stop 2023-11-15T03:04:54.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.947+0000 7f256adf9700 1 -- 172.21.15.121:0/2303303939 shutdown_connections 2023-11-15T03:04:54.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.947+0000 7f256adf9700 1 --2- 172.21.15.121:0/2303303939 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f254805a510 0x7f254805c9d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.947+0000 7f256adf9700 1 --2- 172.21.15.121:0/2303303939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25640ad1b0 0x7f25640a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.947+0000 7f256adf9700 1 --2- 172.21.15.121:0/2303303939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25640ab0d0 0x7f25640a8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.947+0000 7f256adf9700 1 --2- 172.21.15.121:0/2303303939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2564065c00 0x7f25640c63b0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:54.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.947+0000 7f256adf9700 1 -- 172.21.15.121:0/2303303939 >> 172.21.15.121:0/2303303939 conn(0x7f2564067a60 msgr2=0x7f25641541a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:54.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.947+0000 7f256adf9700 1 -- 172.21.15.121:0/2303303939 shutdown_connections 2023-11-15T03:04:54.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:54.947+0000 7f256adf9700 1 -- 172.21.15.121:0/2303303939 wait complete. 2023-11-15T03:04:54.976 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:04:54.977 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 500.00000000', proceeding to ls 2023-11-15T03:04:54.977 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T03:04:55.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.115+0000 7f922f49b700 1 Processor -- start 2023-11-15T03:04:55.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.115+0000 7f922f49b700 1 -- start start 2023-11-15T03:04:55.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.119+0000 7f922f49b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92280c7520 0x7f92280c1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:55.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.119+0000 7f922f49b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f92280c7900 0x7f92280c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:55.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.119+0000 7f922f49b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f92280aa980 0x7f92280aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:55.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.119+0000 7f922f49b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92280c0f70 con 0x7f92280c7900 2023-11-15T03:04:55.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.119+0000 7f922f49b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92280c5db0 con 0x7f92280aa980 2023-11-15T03:04:55.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.119+0000 7f922f49b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92280c6090 con 0x7f92280c7520 2023-11-15T03:04:55.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.119+0000 7f922ca36700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f92280c7900 0x7f92280c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:55.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.119+0000 7f922ca36700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f92280c7900 0x7f92280c7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34106/0 (socket says 172.21.15.121:34106) 2023-11-15T03:04:55.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.119+0000 7f922ca36700 1 -- 172.21.15.121:0/3688132650 learned_addr learned my addr 172.21.15.121:0/3688132650 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:55.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.119+0000 7f922d237700 1 --2- 172.21.15.121:0/3688132650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92280c7520 0x7f92280c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:55.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.119+0000 7f922da38700 1 --2- 172.21.15.121:0/3688132650 >> v2:172.21.15.190:3300/0 conn(0x7f92280aa980 0x7f92280aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:55.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.119+0000 7f922ca36700 1 -- 172.21.15.121:0/3688132650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92280c7520 msgr2=0x7f92280c1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:55.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.119+0000 7f922ca36700 1 --2- 172.21.15.121:0/3688132650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92280c7520 0x7f92280c1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:55.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.119+0000 7f922ca36700 1 -- 172.21.15.121:0/3688132650 >> v2:172.21.15.190:3300/0 conn(0x7f92280aa980 msgr2=0x7f92280aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:55.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.119+0000 7f922ca36700 1 --2- 172.21.15.121:0/3688132650 >> v2:172.21.15.190:3300/0 conn(0x7f92280aa980 0x7f92280aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:55.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.119+0000 7f922ca36700 1 -- 172.21.15.121:0/3688132650 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92280c0c00 con 0x7f92280c7900 2023-11-15T03:04:55.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.119+0000 7f922ca36700 1 --2- 172.21.15.121:0/3688132650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92280c7900 0x7f92280c7d60 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f92280caaa0 tx=0x7f92180090f0).ready entity=mon.0 client_cookie=73f0732e8e73d61c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:55.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.123+0000 7f921ffff700 1 -- 172.21.15.121:0/3688132650 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921800b600 con 0x7f92280c7900 2023-11-15T03:04:55.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.123+0000 7f921ffff700 1 -- 172.21.15.121:0/3688132650 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921800b760 con 0x7f92280c7900 2023-11-15T03:04:55.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.123+0000 7f921ffff700 1 -- 172.21.15.121:0/3688132650 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921800bae0 con 0x7f92280c7900 2023-11-15T03:04:55.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.123+0000 7f922f49b700 1 -- 172.21.15.121:0/3688132650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92280c7900 msgr2=0x7f92280c7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:55.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.123+0000 7f922f49b700 1 --2- 172.21.15.121:0/3688132650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92280c7900 0x7f92280c7d60 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f92280caaa0 tx=0x7f92180090f0).stop 2023-11-15T03:04:55.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.123+0000 7f922f49b700 1 -- 172.21.15.121:0/3688132650 shutdown_connections 2023-11-15T03:04:55.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.123+0000 7f922f49b700 1 --2- 172.21.15.121:0/3688132650 >> v2:172.21.15.190:3300/0 conn(0x7f92280aa980 0x7f92280aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:55.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.123+0000 7f922f49b700 1 --2- 172.21.15.121:0/3688132650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92280c7900 0x7f92280c7d60 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:55.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.123+0000 7f922f49b700 1 --2- 172.21.15.121:0/3688132650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92280c7520 0x7f92280c1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:55.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.123+0000 7f922f49b700 1 -- 172.21.15.121:0/3688132650 >> 172.21.15.121:0/3688132650 conn(0x7f9228067780 msgr2=0x7f92280681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:55.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.123+0000 7f922f49b700 1 -- 172.21.15.121:0/3688132650 shutdown_connections 2023-11-15T03:04:55.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.123+0000 7f922f49b700 1 -- 172.21.15.121:0/3688132650 wait complete. 2023-11-15T03:04:55.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.127+0000 7f922f49b700 1 Processor -- start 2023-11-15T03:04:55.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.127+0000 7f922f49b700 1 -- start start 2023-11-15T03:04:55.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.127+0000 7f922f49b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92280aa980 0x7f922814fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:55.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.127+0000 7f922f49b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92280c7520 0x7f922814d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:55.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.127+0000 7f922f49b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92280c7900 0x7f92280c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:55.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.127+0000 7f922f49b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9228164080 con 0x7f92280aa980 2023-11-15T03:04:55.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.127+0000 7f922f49b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9228150380 con 0x7f92280c7520 2023-11-15T03:04:55.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.127+0000 7f922f49b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92280c2f00 con 0x7f92280c7900 2023-11-15T03:04:55.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.127+0000 7f922d237700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92280aa980 0x7f922814fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:55.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.127+0000 7f922d237700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92280aa980 0x7f922814fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34116/0 (socket says 172.21.15.121:34116) 2023-11-15T03:04:55.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.127+0000 7f922d237700 1 -- 172.21.15.121:0/4272661751 learned_addr learned my addr 172.21.15.121:0/4272661751 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:55.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.131+0000 7f922da38700 1 --2- 172.21.15.121:0/4272661751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92280c7900 0x7f92280c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:55.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.131+0000 7f922ca36700 1 --2- 172.21.15.121:0/4272661751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92280c7520 0x7f922814d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:55.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.131+0000 7f922d237700 1 -- 172.21.15.121:0/4272661751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92280c7900 msgr2=0x7f92280c46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:55.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.131+0000 7f922d237700 1 --2- 172.21.15.121:0/4272661751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92280c7900 0x7f92280c46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:55.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.131+0000 7f922d237700 1 -- 172.21.15.121:0/4272661751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92280c7520 msgr2=0x7f922814d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:55.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.131+0000 7f922d237700 1 --2- 172.21.15.121:0/4272661751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92280c7520 0x7f922814d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:55.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.131+0000 7f922d237700 1 -- 172.21.15.121:0/4272661751 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9218009370 con 0x7f92280aa980 2023-11-15T03:04:55.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.131+0000 7f922d237700 1 --2- 172.21.15.121:0/4272661751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92280aa980 0x7f922814fe90 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7f922400d8c0 tx=0x7f9224008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:55.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.131+0000 7f921d7fa700 1 -- 172.21.15.121:0/4272661751 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92240095b0 con 0x7f92280aa980 2023-11-15T03:04:55.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.131+0000 7f921d7fa700 1 -- 172.21.15.121:0/4272661751 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9224009710 con 0x7f92280aa980 2023-11-15T03:04:55.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.131+0000 7f922f49b700 1 -- 172.21.15.121:0/4272661751 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922814d500 con 0x7f92280aa980 2023-11-15T03:04:55.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.131+0000 7f921d7fa700 1 -- 172.21.15.121:0/4272661751 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9224009a90 con 0x7f92280aa980 2023-11-15T03:04:55.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.131+0000 7f922f49b700 1 -- 172.21.15.121:0/4272661751 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9228164270 con 0x7f92280aa980 2023-11-15T03:04:55.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.131+0000 7f922f49b700 1 -- 172.21.15.121:0/4272661751 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922814d760 con 0x7f92280aa980 2023-11-15T03:04:55.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.135+0000 7f921d7fa700 1 -- 172.21.15.121:0/4272661751 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9224013480 con 0x7f92280aa980 2023-11-15T03:04:55.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.135+0000 7f921d7fa700 1 --2- 172.21.15.121:0/4272661751 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f920c05a510 0x7f920c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:55.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.135+0000 7f921d7fa700 1 -- 172.21.15.121:0/4272661751 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6868+0+0 (secure 0 0 0) 0x7f9224074980 con 0x7f92280aa980 2023-11-15T03:04:55.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.135+0000 7f922ca36700 1 --2- 172.21.15.121:0/4272661751 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f920c05a510 0x7f920c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:55.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.139+0000 7f922ca36700 1 --2- 172.21.15.121:0/4272661751 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f920c05a510 0x7f920c05c9d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f92280caaa0 tx=0x7f9218017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:55.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.143+0000 7f921d7fa700 1 -- 172.21.15.121:0/4272661751 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9224047d50 con 0x7f92280aa980 2023-11-15T03:04:55.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.335+0000 7f922f49b700 1 -- 172.21.15.121:0/4272661751 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f92280c18f0 con 0x7f92280aa980 2023-11-15T03:04:55.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.335+0000 7f921d7fa700 1 -- 172.21.15.121:0/4272661751 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v129) v1 ==== 53+0+73 (secure 0 0 0) 0x7f922404b370 con 0x7f92280aa980 2023-11-15T03:04:55.344 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T03:04:55.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.339+0000 7f922f49b700 1 -- 172.21.15.121:0/4272661751 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f920c05a510 msgr2=0x7f920c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:55.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.339+0000 7f922f49b700 1 --2- 172.21.15.121:0/4272661751 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f920c05a510 0x7f920c05c9d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f92280caaa0 tx=0x7f9218017000).stop 2023-11-15T03:04:55.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.339+0000 7f922f49b700 1 -- 172.21.15.121:0/4272661751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92280aa980 msgr2=0x7f922814fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:55.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.339+0000 7f922f49b700 1 --2- 172.21.15.121:0/4272661751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92280aa980 0x7f922814fe90 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7f922400d8c0 tx=0x7f9224008040).stop 2023-11-15T03:04:55.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.339+0000 7f922f49b700 1 -- 172.21.15.121:0/4272661751 shutdown_connections 2023-11-15T03:04:55.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.339+0000 7f922f49b700 1 --2- 172.21.15.121:0/4272661751 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f920c05a510 0x7f920c05c9d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:55.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.339+0000 7f922f49b700 1 --2- 172.21.15.121:0/4272661751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92280c7900 0x7f92280c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:55.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.339+0000 7f922f49b700 1 --2- 172.21.15.121:0/4272661751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92280c7520 0x7f922814d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:55.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.339+0000 7f922f49b700 1 --2- 172.21.15.121:0/4272661751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92280aa980 0x7f922814fe90 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:55.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.339+0000 7f922f49b700 1 -- 172.21.15.121:0/4272661751 >> 172.21.15.121:0/4272661751 conn(0x7f9228067780 msgr2=0x7f9228151fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:55.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.339+0000 7f922f49b700 1 -- 172.21.15.121:0/4272661751 shutdown_connections 2023-11-15T03:04:55.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:55.339+0000 7f922f49b700 1 -- 172.21.15.121:0/4272661751 wait complete. 2023-11-15T03:04:55.361 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T03:04:55.362 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T03:04:55.362 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T03:04:55.363 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T03:04:55.363 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T03:04:55.363 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T03:04:55.364 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T03:04:55.364 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T03:04:55.364 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:04:55.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:55.365 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T03:04:55.388 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T03:04:55.388 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T03:04:55.388 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T03:04:55.388 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:04:55.389 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T03:04:55.389 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:04:55.389 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:04:55.389 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:04:55.389 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T03:04:55.390 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:04:55.390 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:04:55.390 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T03:04:55.390 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:04:55.390 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:04:55.391 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T03:04:55.391 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:04:55.391 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T03:04:55.391 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 41944sec preferred_lft 41944sec 2023-11-15T03:04:55.391 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T03:04:55.392 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:04:55.392 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T03:04:55.392 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:04:55.392 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T03:04:55.392 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:04:55.392 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T03:04:55.393 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T03:04:55.393 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T03:04:55.393 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:04:55.393 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T03:04:55.393 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:04:55.394 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:04:55.394 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T03:04:55.394 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T03:04:55.394 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:04:55.394 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:04:55.395 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T03:04:55.395 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T03:04:55.395 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:04:55.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:55.397 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T03:04:55.477 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T03:04:55.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:55.481 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T03:04:55.548 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T03:04:55.549 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T03:04:55.549 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T03:04:55.549 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T03:04:55.549 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T03:04:55.550 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T03:04:55.550 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T03:04:55.550 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T03:04:55.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:55.551 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T03:04:55.551 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T03:04:55.552 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T03:04:55.552 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T03:04:55.552 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T03:04:55.553 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T03:04:55.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:55.688 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T03:04:55.709 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T03:04:55.710 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T03:04:55.711 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:04:55.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:55.712 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T03:04:55.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:55.794 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T03:04:55.855 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T03:04:55.856 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:04:55.974 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:04:55.975 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:04:55.996 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:04:55.996 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:04:56.003 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T03:04:56.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:56.004 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T03:04:56.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:04:56.039 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2023-11-15T03:04:56.091 INFO:teuthology.orchestra.run:waiting for 20 2023-11-15T03:04:56.107 INFO:teuthology.orchestra.run.smithi121.stdout:.: 2023-11-15T03:04:56.107 INFO:teuthology.orchestra.run.smithi121.stdout:subdir 2023-11-15T03:04:56.107 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:04:56.108 INFO:teuthology.orchestra.run.smithi121.stdout:./subdir: 2023-11-15T03:04:56.108 INFO:teuthology.orchestra.run.smithi121.stdout:sixmegs 2023-11-15T03:04:56.109 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 500.00000000' did not caused DAMAGED state 2023-11-15T03:04:56.109 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds_snaptable' 2023-11-15T03:04:56.109 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T03:04:56.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:04:56.110 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-15T03:04:56.195 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:04:56.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:04:56.196 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:04:56.271 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:04:56.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.507+0000 7f31523a9700 1 Processor -- start 2023-11-15T03:04:56.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.507+0000 7f31523a9700 1 -- start start 2023-11-15T03:04:56.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.507+0000 7f31523a9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f314c062330 0x7f314c062710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:56.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.507+0000 7f31523a9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f314c0aa130 0x7f314c154c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:56.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.507+0000 7f31523a9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314c0aac50 con 0x7f314c0ac150 2023-11-15T03:04:56.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.507+0000 7f31523a9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314c0aa6b0 con 0x7f314c062330 2023-11-15T03:04:56.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.507+0000 7f31523a9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314c0aa970 con 0x7f314c0aa130 2023-11-15T03:04:56.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.511+0000 7f3150946700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f314c0ac150 0x7f314c14ddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53980/0 (socket says 172.21.15.121:53980) 2023-11-15T03:04:56.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.511+0000 7f3150946700 1 -- 172.21.15.121:0/140916089 learned_addr learned my addr 172.21.15.121:0/140916089 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:56.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.511+0000 7f314bfff700 1 --2- 172.21.15.121:0/140916089 >> v2:172.21.15.121:3300/0 conn(0x7f314c062330 0x7f314c062710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:56.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.511+0000 7f31437fe700 1 --2- 172.21.15.121:0/140916089 >> v2:172.21.15.190:3300/0 conn(0x7f314c0aa130 0x7f314c154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:56.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.511+0000 7f314b7fe700 1 -- 172.21.15.121:0/140916089 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060999503 0 0) 0x7f314c0aac50 con 0x7f314c0ac150 2023-11-15T03:04:56.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.511+0000 7f314b7fe700 1 -- 172.21.15.121:0/140916089 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31380033f0 con 0x7f314c0ac150 2023-11-15T03:04:56.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.511+0000 7f314bfff700 1 -- 172.21.15.121:0/140916089 >> v2:172.21.15.190:3300/0 conn(0x7f314c0aa130 msgr2=0x7f314c154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:56.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.511+0000 7f314bfff700 1 --2- 172.21.15.121:0/140916089 >> v2:172.21.15.190:3300/0 conn(0x7f314c0aa130 0x7f314c154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:56.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.511+0000 7f314bfff700 1 -- 172.21.15.121:0/140916089 >> v1:172.21.15.121:6789/0 conn(0x7f314c0ac150 legacy=0x7f314c14ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:56.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.511+0000 7f314bfff700 1 -- 172.21.15.121:0/140916089 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314c154410 con 0x7f314c062330 2023-11-15T03:04:56.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.511+0000 7f314bfff700 1 --2- 172.21.15.121:0/140916089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f314c062330 0x7f314c062710 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f314c059c40 tx=0x7f313c009060).ready entity=mon.0 client_cookie=724c1aa87ba5abb1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:56.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.511+0000 7f314b7fe700 1 -- 172.21.15.121:0/140916089 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c00d4a0 con 0x7f314c062330 2023-11-15T03:04:56.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.511+0000 7f314b7fe700 1 -- 172.21.15.121:0/140916089 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c00d600 con 0x7f314c062330 2023-11-15T03:04:56.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.511+0000 7f314b7fe700 1 -- 172.21.15.121:0/140916089 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c00d980 con 0x7f314c062330 2023-11-15T03:04:56.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.511+0000 7f31523a9700 1 -- 172.21.15.121:0/140916089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f314c062330 msgr2=0x7f314c062710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:56.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.511+0000 7f31523a9700 1 --2- 172.21.15.121:0/140916089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f314c062330 0x7f314c062710 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f314c059c40 tx=0x7f313c009060).stop 2023-11-15T03:04:56.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.511+0000 7f31523a9700 1 -- 172.21.15.121:0/140916089 shutdown_connections 2023-11-15T03:04:56.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.511+0000 7f31523a9700 1 --2- 172.21.15.121:0/140916089 >> v2:172.21.15.190:3300/0 conn(0x7f314c0aa130 0x7f314c154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:56.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.511+0000 7f31523a9700 1 --2- 172.21.15.121:0/140916089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f314c062330 0x7f314c062710 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:56.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.515+0000 7f31523a9700 1 -- 172.21.15.121:0/140916089 >> 172.21.15.121:0/140916089 conn(0x7f314c067a60 msgr2=0x7f314c067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:56.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.515+0000 7f31523a9700 1 -- 172.21.15.121:0/140916089 shutdown_connections 2023-11-15T03:04:56.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.515+0000 7f31523a9700 1 -- 172.21.15.121:0/140916089 wait complete. 2023-11-15T03:04:56.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.515+0000 7f31523a9700 1 Processor -- start 2023-11-15T03:04:56.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f31523a9700 1 -- start start 2023-11-15T03:04:56.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f31523a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f314c062330 0x7f314c136bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:56.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f31523a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f314c0aa130 0x7f314c0c3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:56.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f31523a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f314c0ac150 0x7f314c156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:56.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f31523a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c14df00 con 0x7f314c062330 2023-11-15T03:04:56.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f31523a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0c2f00 con 0x7f314c0ac150 2023-11-15T03:04:56.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f31523a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0c3880 con 0x7f314c0aa130 2023-11-15T03:04:56.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f314bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f314c062330 0x7f314c136bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:56.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f314bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f314c062330 0x7f314c136bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34136/0 (socket says 172.21.15.121:34136) 2023-11-15T03:04:56.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f314bfff700 1 -- 172.21.15.121:0/542957896 learned_addr learned my addr 172.21.15.121:0/542957896 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:56.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f31437fe700 1 --2- 172.21.15.121:0/542957896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f314c0aa130 0x7f314c0c3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:56.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f3150946700 1 --2- 172.21.15.121:0/542957896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f314c0ac150 0x7f314c156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:56.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f314bfff700 1 -- 172.21.15.121:0/542957896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f314c0aa130 msgr2=0x7f314c0c3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:56.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f314bfff700 1 --2- 172.21.15.121:0/542957896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f314c0aa130 0x7f314c0c3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:56.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f314bfff700 1 -- 172.21.15.121:0/542957896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f314c0ac150 msgr2=0x7f314c156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:56.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f314bfff700 1 --2- 172.21.15.121:0/542957896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f314c0ac150 0x7f314c156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:56.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f314bfff700 1 -- 172.21.15.121:0/542957896 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313c0092a0 con 0x7f314c062330 2023-11-15T03:04:56.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f314bfff700 1 --2- 172.21.15.121:0/542957896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f314c062330 0x7f314c136bf0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7f313c011fb0 tx=0x7f313c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:56.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f3148ff9700 1 -- 172.21.15.121:0/542957896 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c00dd30 con 0x7f314c062330 2023-11-15T03:04:56.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f3148ff9700 1 -- 172.21.15.121:0/542957896 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c00de90 con 0x7f314c062330 2023-11-15T03:04:56.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f31523a9700 1 -- 172.21.15.121:0/542957896 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314c0c22f0 con 0x7f314c062330 2023-11-15T03:04:56.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.519+0000 7f3148ff9700 1 -- 172.21.15.121:0/542957896 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c00d9a0 con 0x7f314c062330 2023-11-15T03:04:56.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.523+0000 7f31523a9700 1 -- 172.21.15.121:0/542957896 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314c0c28b0 con 0x7f314c062330 2023-11-15T03:04:56.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.523+0000 7f31523a9700 1 -- 172.21.15.121:0/542957896 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f314c047060 con 0x7f314c062330 2023-11-15T03:04:56.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.523+0000 7f3148ff9700 1 -- 172.21.15.121:0/542957896 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f313c0263b0 con 0x7f314c062330 2023-11-15T03:04:56.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.527+0000 7f3148ff9700 1 --2- 172.21.15.121:0/542957896 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f312c05a510 0x7f312c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:56.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.527+0000 7f3148ff9700 1 -- 172.21.15.121:0/542957896 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6868+0+0 (secure 0 0 0) 0x7f313c0758e0 con 0x7f314c062330 2023-11-15T03:04:56.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.527+0000 7f31437fe700 1 --2- 172.21.15.121:0/542957896 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f312c05a510 0x7f312c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:56.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.527+0000 7f31437fe700 1 --2- 172.21.15.121:0/542957896 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f312c05a510 0x7f312c05c9d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f314c1672a0 tx=0x7f3134009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:56.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.531+0000 7f3148ff9700 1 -- 172.21.15.121:0/542957896 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f313c048cb0 con 0x7f314c062330 2023-11-15T03:04:56.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:56.735+0000 7f31523a9700 1 -- 172.21.15.121:0/542957896 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f314c1546f0 con 0x7f314c062330 2023-11-15T03:04:57.177 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:04:57.167+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:04:57.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.175+0000 7f3148ff9700 1 -- 172.21.15.121:0/542957896 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v130) v1 ==== 161+0+0 (secure 0 0 0) 0x7f313c022020 con 0x7f314c062330 2023-11-15T03:04:57.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.183+0000 7f31523a9700 1 -- 172.21.15.121:0/542957896 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f312c05a510 msgr2=0x7f312c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:57.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.183+0000 7f31523a9700 1 --2- 172.21.15.121:0/542957896 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f312c05a510 0x7f312c05c9d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f314c1672a0 tx=0x7f3134009060).stop 2023-11-15T03:04:57.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.183+0000 7f31523a9700 1 -- 172.21.15.121:0/542957896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f314c062330 msgr2=0x7f314c136bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:57.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.183+0000 7f31523a9700 1 --2- 172.21.15.121:0/542957896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f314c062330 0x7f314c136bf0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7f313c011fb0 tx=0x7f313c016000).stop 2023-11-15T03:04:57.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.183+0000 7f31523a9700 1 -- 172.21.15.121:0/542957896 shutdown_connections 2023-11-15T03:04:57.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.183+0000 7f31523a9700 1 --2- 172.21.15.121:0/542957896 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f312c05a510 0x7f312c05c9d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:57.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.183+0000 7f31523a9700 1 --2- 172.21.15.121:0/542957896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f314c0ac150 0x7f314c156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:57.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.183+0000 7f31523a9700 1 --2- 172.21.15.121:0/542957896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f314c0aa130 0x7f314c0c3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:57.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.183+0000 7f31523a9700 1 --2- 172.21.15.121:0/542957896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f314c062330 0x7f314c136bf0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:57.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.183+0000 7f31523a9700 1 -- 172.21.15.121:0/542957896 >> 172.21.15.121:0/542957896 conn(0x7f314c067a60 msgr2=0x7f314c065ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:57.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.183+0000 7f31523a9700 1 -- 172.21.15.121:0/542957896 shutdown_connections 2023-11-15T03:04:57.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.183+0000 7f31523a9700 1 -- 172.21.15.121:0/542957896 wait complete. 2023-11-15T03:04:57.192 INFO:tasks.ceph.mds.a.smithi121.stderr: -492> 2023-11-15T03:04:51.199+0000 7f1f278dc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:04:57.193 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:04:57.211 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:04:57.232 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:04:57.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.355+0000 7f5854db9700 1 Processor -- start 2023-11-15T03:04:57.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.355+0000 7f5854db9700 1 -- start start 2023-11-15T03:04:57.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f5854db9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f58500ab0d0 0x7f58500ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:57.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f5854db9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58500ad1b0 0x7f5850065710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:57.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f5854db9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5850066c90 con 0x7f5850065c00 2023-11-15T03:04:57.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f5854db9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5850066080 con 0x7f58500ab0d0 2023-11-15T03:04:57.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f5854db9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5850066360 con 0x7f58500ad1b0 2023-11-15T03:04:57.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f584f7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f58500ab0d0 0x7f58500ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:57.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f584effd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58500ad1b0 0x7f5850065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:57.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f584f7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f58500ab0d0 0x7f58500ab4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:41512/0 (socket says 172.21.15.121:41512) 2023-11-15T03:04:57.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f584f7fe700 1 -- 172.21.15.121:0/3576946627 learned_addr learned my addr 172.21.15.121:0/3576946627 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:57.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f584ffff700 1 --1- 172.21.15.121:0/3576946627 >> v1:172.21.15.190:6789/0 conn(0x7f5850065c00 0x7f58500c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:60408/0 (socket says 172.21.15.121:60408) 2023-11-15T03:04:57.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f584f7fe700 1 -- 172.21.15.121:0/3576946627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58500ad1b0 msgr2=0x7f5850065710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:57.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f584f7fe700 1 --2- 172.21.15.121:0/3576946627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58500ad1b0 0x7f5850065710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:57.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f584f7fe700 1 -- 172.21.15.121:0/3576946627 >> v1:172.21.15.190:6789/0 conn(0x7f5850065c00 legacy=0x7f58500c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:57.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f584f7fe700 1 -- 172.21.15.121:0/3576946627 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5850066640 con 0x7f58500ab0d0 2023-11-15T03:04:57.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f584f7fe700 1 --2- 172.21.15.121:0/3576946627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58500ab0d0 0x7f58500ab4b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f5840009810 tx=0x7f5840009060).ready entity=mon.1 client_cookie=e2f066f79e3224f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:57.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f584e7fc700 1 -- 172.21.15.121:0/3576946627 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840005880 con 0x7f58500ab0d0 2023-11-15T03:04:57.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f584e7fc700 1 -- 172.21.15.121:0/3576946627 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58400059e0 con 0x7f58500ab0d0 2023-11-15T03:04:57.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f584e7fc700 1 -- 172.21.15.121:0/3576946627 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840005d60 con 0x7f58500ab0d0 2023-11-15T03:04:57.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f5854db9700 1 -- 172.21.15.121:0/3576946627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58500ab0d0 msgr2=0x7f58500ab4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:57.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f5854db9700 1 --2- 172.21.15.121:0/3576946627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58500ab0d0 0x7f58500ab4b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f5840009810 tx=0x7f5840009060).stop 2023-11-15T03:04:57.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f5854db9700 1 -- 172.21.15.121:0/3576946627 shutdown_connections 2023-11-15T03:04:57.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f5854db9700 1 --2- 172.21.15.121:0/3576946627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58500ad1b0 0x7f5850065710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:57.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f5854db9700 1 --2- 172.21.15.121:0/3576946627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58500ab0d0 0x7f58500ab4b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:57.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f5854db9700 1 -- 172.21.15.121:0/3576946627 >> 172.21.15.121:0/3576946627 conn(0x7f5850067a60 msgr2=0x7f5850067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:57.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.359+0000 7f5854db9700 1 -- 172.21.15.121:0/3576946627 shutdown_connections 2023-11-15T03:04:57.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.363+0000 7f5854db9700 1 -- 172.21.15.121:0/3576946627 wait complete. 2023-11-15T03:04:57.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.363+0000 7f5854db9700 1 Processor -- start 2023-11-15T03:04:57.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.363+0000 7f5854db9700 1 -- start start 2023-11-15T03:04:57.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.363+0000 7f5854db9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5850065c00 0x7f5850148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:57.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.363+0000 7f5854db9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58500ab0d0 0x7f58500a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:57.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.363+0000 7f5854db9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58500ad1b0 0x7f58500a9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:57.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.363+0000 7f5854db9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58500a9850 con 0x7f58500ab0d0 2023-11-15T03:04:57.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.363+0000 7f5854db9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58500a5290 con 0x7f58500ad1b0 2023-11-15T03:04:57.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.363+0000 7f5854db9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58500a4710 con 0x7f5850065c00 2023-11-15T03:04:57.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.367+0000 7f584effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58500ab0d0 0x7f58500a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:57.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.367+0000 7f584effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58500ab0d0 0x7f58500a8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34160/0 (socket says 172.21.15.121:34160) 2023-11-15T03:04:57.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.367+0000 7f584effd700 1 -- 172.21.15.121:0/2747780680 learned_addr learned my addr 172.21.15.121:0/2747780680 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:57.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.367+0000 7f584ffff700 1 --2- 172.21.15.121:0/2747780680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58500ad1b0 0x7f58500a9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:57.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.367+0000 7f584f7fe700 1 --2- 172.21.15.121:0/2747780680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5850065c00 0x7f5850148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:57.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.367+0000 7f584effd700 1 -- 172.21.15.121:0/2747780680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5850065c00 msgr2=0x7f5850148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:57.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.367+0000 7f584effd700 1 --2- 172.21.15.121:0/2747780680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5850065c00 0x7f5850148240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:57.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.367+0000 7f584effd700 1 -- 172.21.15.121:0/2747780680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58500ad1b0 msgr2=0x7f58500a9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:57.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.367+0000 7f584effd700 1 --2- 172.21.15.121:0/2747780680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58500ad1b0 0x7f58500a9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:57.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.367+0000 7f584effd700 1 -- 172.21.15.121:0/2747780680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5840009300 con 0x7f58500ab0d0 2023-11-15T03:04:57.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.367+0000 7f584effd700 1 --2- 172.21.15.121:0/2747780680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58500ab0d0 0x7f58500a8e70 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7f584400e930 tx=0x7f5844009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:57.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.367+0000 7f58377fe700 1 -- 172.21.15.121:0/2747780680 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844012600 con 0x7f58500ab0d0 2023-11-15T03:04:57.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.367+0000 7f58377fe700 1 -- 172.21.15.121:0/2747780680 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5844012760 con 0x7f58500ab0d0 2023-11-15T03:04:57.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.367+0000 7f58377fe700 1 -- 172.21.15.121:0/2747780680 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844012ae0 con 0x7f58500ab0d0 2023-11-15T03:04:57.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.367+0000 7f5854db9700 1 -- 172.21.15.121:0/2747780680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58500a49f0 con 0x7f58500ab0d0 2023-11-15T03:04:57.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.367+0000 7f5854db9700 1 -- 172.21.15.121:0/2747780680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58500a4fb0 con 0x7f58500ab0d0 2023-11-15T03:04:57.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.367+0000 7f5854db9700 1 -- 172.21.15.121:0/2747780680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f585006bcf0 con 0x7f58500ab0d0 2023-11-15T03:04:57.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.367+0000 7f58377fe700 1 -- 172.21.15.121:0/2747780680 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f584401b400 con 0x7f58500ab0d0 2023-11-15T03:04:57.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.371+0000 7f58377fe700 1 --2- 172.21.15.121:0/2747780680 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f583c05a510 0x7f583c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:57.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.371+0000 7f58377fe700 1 -- 172.21.15.121:0/2747780680 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6954+0+0 (secure 0 0 0) 0x7f5844074f10 con 0x7f58500ab0d0 2023-11-15T03:04:57.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.371+0000 7f584f7fe700 1 --2- 172.21.15.121:0/2747780680 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f583c05a510 0x7f583c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:57.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.371+0000 7f584f7fe700 1 --2- 172.21.15.121:0/2747780680 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f583c05a510 0x7f583c05c9d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f58500cae40 tx=0x7f5840016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:57.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.379+0000 7f58377fe700 1 -- 172.21.15.121:0/2747780680 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5844048350 con 0x7f58500ab0d0 2023-11-15T03:04:57.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:57.599+0000 7f5854db9700 1 -- 172.21.15.121:0/2747780680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f58500a6720 con 0x7f58500ab0d0 2023-11-15T03:04:58.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.187+0000 7f58377fe700 1 -- 172.21.15.121:0/2747780680 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v131) v1 ==== 107+0+0 (secure 0 0 0) 0x7f5844016030 con 0x7f58500ab0d0 2023-11-15T03:04:58.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.195+0000 7f5854db9700 1 -- 172.21.15.121:0/2747780680 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f583c05a510 msgr2=0x7f583c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.195+0000 7f5854db9700 1 --2- 172.21.15.121:0/2747780680 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f583c05a510 0x7f583c05c9d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f58500cae40 tx=0x7f5840016000).stop 2023-11-15T03:04:58.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.195+0000 7f5854db9700 1 -- 172.21.15.121:0/2747780680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58500ab0d0 msgr2=0x7f58500a8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.195+0000 7f5854db9700 1 --2- 172.21.15.121:0/2747780680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58500ab0d0 0x7f58500a8e70 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7f584400e930 tx=0x7f5844009060).stop 2023-11-15T03:04:58.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.195+0000 7f5854db9700 1 -- 172.21.15.121:0/2747780680 shutdown_connections 2023-11-15T03:04:58.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.195+0000 7f5854db9700 1 --2- 172.21.15.121:0/2747780680 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f583c05a510 0x7f583c05c9d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.195+0000 7f5854db9700 1 --2- 172.21.15.121:0/2747780680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58500ad1b0 0x7f58500a9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.195+0000 7f5854db9700 1 --2- 172.21.15.121:0/2747780680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58500ab0d0 0x7f58500a8e70 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.195+0000 7f5854db9700 1 --2- 172.21.15.121:0/2747780680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5850065c00 0x7f5850148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.195+0000 7f5854db9700 1 -- 172.21.15.121:0/2747780680 >> 172.21.15.121:0/2747780680 conn(0x7f5850067a60 msgr2=0x7f58500afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:58.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.195+0000 7f5854db9700 1 -- 172.21.15.121:0/2747780680 shutdown_connections 2023-11-15T03:04:58.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.195+0000 7f5854db9700 1 -- 172.21.15.121:0/2747780680 wait complete. 2023-11-15T03:04:58.207 INFO:teuthology.orchestra.run.smithi121.stderr:nothing to do: rank is not damaged 2023-11-15T03:04:58.224 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:04:58.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.319+0000 7f1ca6cfad00 1 Processor -- start 2023-11-15T03:04:58.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.319+0000 7f1ca6cfad00 1 -- start start 2023-11-15T03:04:58.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.319+0000 7f1ca6cfad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cf1fb5a30 0x563cf2096080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.319+0000 7f1ca6cfad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563cf1fb5e10 0x563cf2093be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.319+0000 7f1ca6cfad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cf2004b20 0x563cf20940d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.319+0000 7f1ca6cfad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cf1f2e220 con 0x563cf1fb5a30 2023-11-15T03:04:58.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.319+0000 7f1ca6cfad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cf2001200 con 0x563cf2004b20 2023-11-15T03:04:58.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.319+0000 7f1ca6cfad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x563cf208c770 con 0x563cf1fb5e10 2023-11-15T03:04:58.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.319+0000 7f1ca492c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cf1fb5a30 0x563cf2096080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.319+0000 7f1ca492c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cf1fb5a30 0x563cf2096080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34186/0 (socket says 172.21.15.121:34186) 2023-11-15T03:04:58.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.319+0000 7f1ca492c700 1 -- 172.21.15.121:0/563541059 learned_addr learned my addr 172.21.15.121:0/563541059 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:58.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.319+0000 7f1ca512d700 1 --2- 172.21.15.121:0/563541059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cf2004b20 0x563cf20940d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.319+0000 7f1c9c12b700 1 --2- 172.21.15.121:0/563541059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563cf1fb5e10 0x563cf2093be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1ca492c700 1 -- 172.21.15.121:0/563541059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563cf1fb5e10 msgr2=0x563cf2093be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:58.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1ca492c700 1 --2- 172.21.15.121:0/563541059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563cf1fb5e10 0x563cf2093be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1ca492c700 1 -- 172.21.15.121:0/563541059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cf2004b20 msgr2=0x563cf20940d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:58.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1ca492c700 1 --2- 172.21.15.121:0/563541059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cf2004b20 0x563cf20940d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1ca492c700 1 -- 172.21.15.121:0/563541059 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8c005db0 con 0x563cf1fb5a30 2023-11-15T03:04:58.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1ca512d700 1 --2- 172.21.15.121:0/563541059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cf2004b20 0x563cf20940d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:58.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1ca492c700 1 --2- 172.21.15.121:0/563541059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cf1fb5a30 0x563cf2096080 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7f1c94001580 tx=0x7f1c94006b60).ready entity=mon.0 client_cookie=ceb99f333e228247 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:58.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1c9f7fe700 1 -- 172.21.15.121:0/563541059 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c940030b0 con 0x563cf1fb5a30 2023-11-15T03:04:58.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1c9f7fe700 1 -- 172.21.15.121:0/563541059 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c94012450 con 0x563cf1fb5a30 2023-11-15T03:04:58.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1c9f7fe700 1 -- 172.21.15.121:0/563541059 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c940127d0 con 0x563cf1fb5a30 2023-11-15T03:04:58.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/563541059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cf1fb5a30 msgr2=0x563cf2096080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/563541059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cf1fb5a30 0x563cf2096080 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7f1c94001580 tx=0x7f1c94006b60).stop 2023-11-15T03:04:58.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/563541059 shutdown_connections 2023-11-15T03:04:58.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/563541059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cf2004b20 0x563cf20940d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/563541059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563cf1fb5e10 0x563cf2093be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/563541059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cf1fb5a30 0x563cf2096080 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/563541059 >> 172.21.15.121:0/563541059 conn(0x563cf2005080 msgr2=0x563cf2095130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:58.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/563541059 shutdown_connections 2023-11-15T03:04:58.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/563541059 wait complete. 2023-11-15T03:04:58.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1ca6cfad00 1 Processor -- start 2023-11-15T03:04:58.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.323+0000 7f1ca6cfad00 1 -- start start 2023-11-15T03:04:58.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1ca6cfad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563cf1fb5a30 0x563cf2089ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1ca6cfad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cf1fb5e10 0x563cf2089fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1ca6cfad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cf2004b20 0x563cf1ffb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1ca6cfad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cf1fea1a0 con 0x563cf1fb5e10 2023-11-15T03:04:58.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1ca6cfad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cf1ffe080 con 0x563cf2004b20 2023-11-15T03:04:58.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1ca6cfad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x563cf1fea390 con 0x563cf1fb5a30 2023-11-15T03:04:58.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1c9c12b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cf1fb5e10 0x563cf2089fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1c9c12b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cf1fb5e10 0x563cf2089fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34194/0 (socket says 172.21.15.121:34194) 2023-11-15T03:04:58.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 learned_addr learned my addr 172.21.15.121:0/3603045108 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:58.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1ca512d700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cf2004b20 0x563cf1ffb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1ca492c700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563cf1fb5a30 0x563cf2089ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563cf1fb5a30 msgr2=0x563cf2089ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1c9c12b700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563cf1fb5a30 0x563cf2089ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cf2004b20 msgr2=0x563cf1ffb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1c9c12b700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cf2004b20 0x563cf1ffb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cf1f5c7f0 con 0x563cf1fb5e10 2023-11-15T03:04:58.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1c9c12b700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cf1fb5e10 0x563cf2089fc0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7f1c8c00a5a0 tx=0x7f1c8c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:58.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1c9dffb700 1 -- 172.21.15.121:0/3603045108 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c008ba0 con 0x563cf1fb5e10 2023-11-15T03:04:58.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1c9dffb700 1 -- 172.21.15.121:0/3603045108 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8c013440 con 0x563cf1fb5e10 2023-11-15T03:04:58.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563cf1f57770 con 0x563cf1fb5e10 2023-11-15T03:04:58.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1c9dffb700 1 -- 172.21.15.121:0/3603045108 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c0137c0 con 0x563cf1fb5e10 2023-11-15T03:04:58.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.327+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563cf1f26a70 con 0x563cf1fb5e10 2023-11-15T03:04:58.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.331+0000 7f1c9dffb700 1 -- 172.21.15.121:0/3603045108 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1c8c013b30 con 0x563cf1fb5e10 2023-11-15T03:04:58.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.331+0000 7f1c9dffb700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1c84059020 0x7f1c8405b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.331+0000 7f1c9dffb700 1 -- 172.21.15.121:0/3603045108 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6954+0+0 (secure 0 0 0) 0x7f1c8c075200 con 0x563cf1fb5e10 2023-11-15T03:04:58.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.331+0000 7f1ca492c700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1c84059020 0x7f1c8405b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.356 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:04:58.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.331+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x563cf208a5e0 0x563cf1ffea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.331+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf1ffef10 con 0x563cf208a5e0 2023-11-15T03:04:58.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.331+0000 7f1ca512d700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x563cf208a5e0 0x563cf1ffea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.331+0000 7f1ca492c700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1c84059020 0x7f1c8405b4e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x563cf2003160 tx=0x7f1c94012000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:58.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.335+0000 7f1ca512d700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x563cf208a5e0 0x563cf1ffea20 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:58.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.335+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v102'89 uv82 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf208a5e0 2023-11-15T03:04:58.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.335+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf1fff2f0 con 0x563cf208a5e0 2023-11-15T03:04:58.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.339+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v102'90 uv90 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf208a5e0 2023-11-15T03:04:58.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.339+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20af7a0 con 0x563cf208a5e0 2023-11-15T03:04:58.359 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:04:58.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.343+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v102'91 uv91 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf208a5e0 2023-11-15T03:04:58.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.343+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b0b60 con 0x563cf208a5e0 2023-11-15T03:04:58.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.343+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v102'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf208a5e0 2023-11-15T03:04:58.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.343+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b0f10 con 0x563cf208a5e0 2023-11-15T03:04:58.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.347+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v102'93 uv93 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf208a5e0 2023-11-15T03:04:58.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.347+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20afb50 con 0x563cf208a5e0 2023-11-15T03:04:58.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.351+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v102'94 uv94 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf208a5e0 2023-11-15T03:04:58.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.351+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20aff30 con 0x563cf208a5e0 2023-11-15T03:04:58.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.351+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v102'95 uv88 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf208a5e0 2023-11-15T03:04:58.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.351+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20aff30 con 0x563cf208a5e0 2023-11-15T03:04:58.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.355+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v102'96 uv96 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf208a5e0 2023-11-15T03:04:58.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.355+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20aff30 con 0x563cf208a5e0 2023-11-15T03:04:58.365 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:04:58.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.359+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v102'97 uv97 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf208a5e0 2023-11-15T03:04:58.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.359+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20aff00 con 0x563cf208a5e0 2023-11-15T03:04:58.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.359+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v102'98 uv98 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf208a5e0 2023-11-15T03:04:58.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.359+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20aff00 con 0x563cf208a5e0 2023-11-15T03:04:58.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.363+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v102'99 uv99 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf208a5e0 2023-11-15T03:04:58.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.363+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20aff00 con 0x563cf208a5e0 2023-11-15T03:04:58.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.367+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v102'100 uv100 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf208a5e0 2023-11-15T03:04:58.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.367+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x563cf20b12c0 0x563cf20b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.367+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20aff00 con 0x563cf20b12c0 2023-11-15T03:04:58.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.367+0000 7f1ca492c700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x563cf20b12c0 0x563cf20b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.367+0000 7f1ca492c700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x563cf20b12c0 0x563cf20b36b0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:58.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.371+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v102'138 uv125 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.371+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20b3ba0 con 0x563cf20b12c0 2023-11-15T03:04:58.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.375+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v102'139 uv139 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.375+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b3f50 con 0x563cf20b12c0 2023-11-15T03:04:58.386 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:04:58.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.379+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v102'140 uv140 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.379+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b5310 con 0x563cf20b12c0 2023-11-15T03:04:58.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.379+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v102'141 uv141 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.379+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b56c0 con 0x563cf20b12c0 2023-11-15T03:04:58.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.383+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v102'142 uv142 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.383+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b4300 con 0x563cf20b12c0 2023-11-15T03:04:58.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.387+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v102'143 uv143 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.387+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b4bf0 con 0x563cf20b12c0 2023-11-15T03:04:58.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.391+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v102'144 uv144 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.391+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b5a70 con 0x563cf20b12c0 2023-11-15T03:04:58.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.391+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v102'145 uv131 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.391+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20b5a70 con 0x563cf20b12c0 2023-11-15T03:04:58.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.395+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v102'146 uv146 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.395+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b5a70 con 0x563cf20b12c0 2023-11-15T03:04:58.407 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:04:58.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.399+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v102'147 uv147 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.399+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b6a80 con 0x563cf20b12c0 2023-11-15T03:04:58.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.403+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v102'148 uv148 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.403+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b6a80 con 0x563cf20b12c0 2023-11-15T03:04:58.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.403+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v102'149 uv149 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.403+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b5a70 con 0x563cf20b12c0 2023-11-15T03:04:58.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.407+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v102'150 uv150 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.407+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b5a70 con 0x563cf20b12c0 2023-11-15T03:04:58.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.411+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v102'151 uv137 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.411+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20b5a70 con 0x563cf20b12c0 2023-11-15T03:04:58.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.411+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v102'152 uv152 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.411+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b5a70 con 0x563cf20b12c0 2023-11-15T03:04:58.424 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:04:58.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.415+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v102'153 uv153 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.415+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b6a80 con 0x563cf20b12c0 2023-11-15T03:04:58.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.419+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v102'154 uv154 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.419+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b6a80 con 0x563cf20b12c0 2023-11-15T03:04:58.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.419+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v102'155 uv155 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.419+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b5a70 con 0x563cf20b12c0 2023-11-15T03:04:58.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.423+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v102'156 uv156 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.423+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x563cf20b5f80 0x563cf20b8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.423+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b8860 con 0x563cf20b5f80 2023-11-15T03:04:58.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.423+0000 7f1c9c12b700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x563cf20b5f80 0x563cf20b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.423+0000 7f1c9c12b700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x563cf20b5f80 0x563cf20b8370 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:58.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.427+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v102'77 uv70 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20b5f80 2023-11-15T03:04:58.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.427+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20b8c10 con 0x563cf20b5f80 2023-11-15T03:04:58.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.431+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v102'78 uv78 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20b5f80 2023-11-15T03:04:58.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.431+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b8fc0 con 0x563cf20b5f80 2023-11-15T03:04:58.440 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:04:58.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.431+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v102'79 uv79 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20b5f80 2023-11-15T03:04:58.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.431+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9860 con 0x563cf20b5f80 2023-11-15T03:04:58.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.435+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v102'80 uv80 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20b5f80 2023-11-15T03:04:58.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.435+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9c10 con 0x563cf20b5f80 2023-11-15T03:04:58.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.439+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v102'81 uv81 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20b5f80 2023-11-15T03:04:58.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.439+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9fc0 con 0x563cf20b5f80 2023-11-15T03:04:58.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.439+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v102'82 uv76 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20b5f80 2023-11-15T03:04:58.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.439+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20ba370 con 0x563cf20b5f80 2023-11-15T03:04:58.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.443+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v102'83 uv83 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20b5f80 2023-11-15T03:04:58.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.443+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20ba720 con 0x563cf20b5f80 2023-11-15T03:04:58.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.447+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v102'84 uv84 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20b5f80 2023-11-15T03:04:58.454 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:04:58.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.447+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20ba720 con 0x563cf20b5f80 2023-11-15T03:04:58.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.447+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v102'85 uv85 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20b5f80 2023-11-15T03:04:58.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.447+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20ba720 con 0x563cf20b5f80 2023-11-15T03:04:58.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.451+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v102'86 uv86 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20b5f80 2023-11-15T03:04:58.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.451+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x563cf20bac30 0x563cf20bd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.451+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20bd510 con 0x563cf20bac30 2023-11-15T03:04:58.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.451+0000 7f1ca512d700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x563cf20bac30 0x563cf20bd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.451+0000 7f1ca512d700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x563cf20bac30 0x563cf20bd020 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:58.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.455+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v102'100 uv99 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.455+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20bd5b0 con 0x563cf20bac30 2023-11-15T03:04:58.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.459+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v102'101 uv101 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.459+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20bd9b0 con 0x563cf20bac30 2023-11-15T03:04:58.471 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:04:58.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.463+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v102'102 uv102 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.463+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c1a40 con 0x563cf20bac30 2023-11-15T03:04:58.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.467+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v102'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.467+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c1a40 con 0x563cf20bac30 2023-11-15T03:04:58.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.471+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v102'104 uv104 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.471+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20bd9b0 con 0x563cf20bac30 2023-11-15T03:04:58.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.471+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v102'105 uv96 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.471+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20bd9b0 con 0x563cf20bac30 2023-11-15T03:04:58.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.475+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v102'106 uv106 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.475+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20bd9b0 con 0x563cf20bac30 2023-11-15T03:04:58.485 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:04:58.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.479+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v102'107 uv107 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.479+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20be9c0 con 0x563cf20bac30 2023-11-15T03:04:58.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.479+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v102'108 uv108 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.479+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20be9c0 con 0x563cf20bac30 2023-11-15T03:04:58.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.483+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v102'109 uv109 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.483+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20bd9b0 con 0x563cf20bac30 2023-11-15T03:04:58.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.487+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v102'110 uv110 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.487+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x563cf20bdec0 0x563cf20c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.487+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c08c0 con 0x563cf20bdec0 2023-11-15T03:04:58.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.487+0000 7f1ca492c700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x563cf20bdec0 0x563cf20c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.487+0000 7f1ca492c700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x563cf20bdec0 0x563cf20c02b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:58.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.491+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v102'45 uv44 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20bdec0 2023-11-15T03:04:58.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.491+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20c0ca0 con 0x563cf20bdec0 2023-11-15T03:04:58.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.491+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v102'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20bdec0 2023-11-15T03:04:58.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.491+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c0ca0 con 0x563cf20bdec0 2023-11-15T03:04:58.503 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:04:58.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.495+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v102'47 uv47 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20bdec0 2023-11-15T03:04:58.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.495+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c1cb0 con 0x563cf20bdec0 2023-11-15T03:04:58.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.499+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v102'48 uv48 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20bdec0 2023-11-15T03:04:58.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.499+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c1cb0 con 0x563cf20bdec0 2023-11-15T03:04:58.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.499+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v102'49 uv49 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20bdec0 2023-11-15T03:04:58.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.499+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c0ca0 con 0x563cf20bdec0 2023-11-15T03:04:58.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.503+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v102'50 uv50 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20bdec0 2023-11-15T03:04:58.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.503+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x563cf20c15b0 0x563cf20c1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.503+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c40c0 con 0x563cf20c15b0 2023-11-15T03:04:58.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.503+0000 7f1c9c12b700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x563cf20c15b0 0x563cf20c1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.503+0000 7f1c9c12b700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x563cf20c15b0 0x563cf20c1a00 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:58.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.507+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v102'35 uv34 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.507+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20c40c0 con 0x563cf20c15b0 2023-11-15T03:04:58.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.511+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v102'36 uv36 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.511+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c40c0 con 0x563cf20c15b0 2023-11-15T03:04:58.523 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:04:58.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.515+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v102'37 uv37 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.515+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20c15b0 2023-11-15T03:04:58.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.519+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v102'38 uv38 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.519+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20c15b0 2023-11-15T03:04:58.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.523+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v102'45 uv44 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.523+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20c15b0 2023-11-15T03:04:58.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.523+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v102'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.523+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20c15b0 2023-11-15T03:04:58.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.527+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v102'47 uv47 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.536 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:04:58.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.527+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20c15b0 2023-11-15T03:04:58.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.531+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v102'48 uv48 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.531+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20c15b0 2023-11-15T03:04:58.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.535+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v102'49 uv49 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.535+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20c15b0 2023-11-15T03:04:58.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.535+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v102'50 uv50 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.535+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20b12c0 2023-11-15T03:04:58.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.539+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v102'83 uv75 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.539+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20b12c0 2023-11-15T03:04:58.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.543+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v102'84 uv84 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.543+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20b12c0 2023-11-15T03:04:58.555 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:04:58.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.547+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v102'85 uv85 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.547+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20b12c0 2023-11-15T03:04:58.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.551+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v102'86 uv86 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.551+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20b12c0 2023-11-15T03:04:58.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.555+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v102'87 uv87 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.555+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20b12c0 2023-11-15T03:04:58.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.555+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v102'88 uv88 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.555+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20b12c0 2023-11-15T03:04:58.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.559+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v102'89 uv82 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.559+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20b12c0 2023-11-15T03:04:58.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.563+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v102'90 uv90 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.563+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20b12c0 2023-11-15T03:04:58.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.567+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v102'91 uv91 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.573 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:04:58.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.567+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20b12c0 2023-11-15T03:04:58.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.571+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v102'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.571+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20b12c0 2023-11-15T03:04:58.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.571+0000 7f1ca492c700 1 -- 172.21.15.121:0/3603045108 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v102'93 uv93 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c94012d70 con 0x563cf20b12c0 2023-11-15T03:04:58.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.571+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20b5f80 2023-11-15T03:04:58.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.575+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v102'37 uv36 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20b5f80 2023-11-15T03:04:58.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.575+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20b5f80 2023-11-15T03:04:58.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.579+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v102'38 uv38 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20b5f80 2023-11-15T03:04:58.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.579+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20b5f80 2023-11-15T03:04:58.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.579+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v102'39 uv39 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20b5f80 2023-11-15T03:04:58.588 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:04:58.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.579+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20b9490 con 0x563cf20b5f80 2023-11-15T03:04:58.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.583+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v102'40 uv40 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20b5f80 2023-11-15T03:04:58.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.583+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c45b0 con 0x563cf20b5f80 2023-11-15T03:04:58.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.587+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v102'41 uv41 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20b5f80 2023-11-15T03:04:58.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.587+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c45b0 con 0x563cf20c15b0 2023-11-15T03:04:58.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.587+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v102'45 uv44 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.587+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20c45b0 con 0x563cf20c15b0 2023-11-15T03:04:58.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.591+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v102'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.591+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c45b0 con 0x563cf20c15b0 2023-11-15T03:04:58.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.595+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v102'47 uv47 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.602 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:04:58.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.595+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c55c0 con 0x563cf20c15b0 2023-11-15T03:04:58.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.595+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v102'48 uv48 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.595+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c55c0 con 0x563cf20c15b0 2023-11-15T03:04:58.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.599+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v102'49 uv49 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.599+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c45b0 con 0x563cf20c15b0 2023-11-15T03:04:58.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.603+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v102'50 uv50 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.603+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c45b0 con 0x563cf20c15b0 2023-11-15T03:04:58.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.603+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v102'38 uv37 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.603+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20c45b0 con 0x563cf20c15b0 2023-11-15T03:04:58.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.607+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v102'39 uv39 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.607+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c45b0 con 0x563cf20c15b0 2023-11-15T03:04:58.619 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:04:58.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.611+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v102'40 uv40 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.611+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c45b0 con 0x563cf20c15b0 2023-11-15T03:04:58.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.615+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v102'41 uv41 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.615+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c45b0 con 0x563cf20c15b0 2023-11-15T03:04:58.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.615+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v102'42 uv42 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.615+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c45b0 con 0x563cf20c15b0 2023-11-15T03:04:58.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.619+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v102'37 uv36 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.619+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20c45b0 con 0x563cf20c15b0 2023-11-15T03:04:58.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.623+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v102'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.623+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c45b0 con 0x563cf20c15b0 2023-11-15T03:04:58.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.627+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v102'39 uv39 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.633 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:04:58.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.627+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c45b0 con 0x563cf20c15b0 2023-11-15T03:04:58.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.627+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v102'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.627+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c6fe0 con 0x563cf20c15b0 2023-11-15T03:04:58.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.631+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v102'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.631+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x563cf20c40c0 0x563cf20c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.631+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c64b0 con 0x563cf20c40c0 2023-11-15T03:04:58.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.631+0000 7f1ca512d700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x563cf20c40c0 0x563cf20c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.631+0000 7f1ca512d700 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x563cf20c40c0 0x563cf20c7630 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:58.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.635+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v102'31 uv30 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20c40c0 2023-11-15T03:04:58.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.635+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20c64b0 con 0x563cf20c40c0 2023-11-15T03:04:58.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.639+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v102'32 uv32 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20c40c0 2023-11-15T03:04:58.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.639+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c64b0 con 0x563cf20c40c0 2023-11-15T03:04:58.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.643+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v102'33 uv33 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20c40c0 2023-11-15T03:04:58.650 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:04:58.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.643+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c64b0 con 0x563cf20c40c0 2023-11-15T03:04:58.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.643+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v102'34 uv34 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20c40c0 2023-11-15T03:04:58.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.643+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c64b0 con 0x563cf20c15b0 2023-11-15T03:04:58.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.647+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v102'45 uv44 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.647+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20c64b0 con 0x563cf20c15b0 2023-11-15T03:04:58.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.651+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v102'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.651+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c64b0 con 0x563cf20c15b0 2023-11-15T03:04:58.661 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:04:58.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.655+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v102'47 uv47 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.655+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c64b0 con 0x563cf20c15b0 2023-11-15T03:04:58.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.655+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v102'48 uv48 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.655+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c64b0 con 0x563cf20c15b0 2023-11-15T03:04:58.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.659+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v102'49 uv49 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.659+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c64b0 con 0x563cf20c15b0 2023-11-15T03:04:58.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.663+0000 7f1c9c12b700 1 -- 172.21.15.121:0/3603045108 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v102'50 uv50 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c8c024020 con 0x563cf20c15b0 2023-11-15T03:04:58.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.663+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c64b0 con 0x563cf20bac30 2023-11-15T03:04:58.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.663+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v102'37 uv36 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.663+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20c64b0 con 0x563cf20bac30 2023-11-15T03:04:58.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.667+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v102'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.667+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c64b0 con 0x563cf20bac30 2023-11-15T03:04:58.677 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:04:58.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.671+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v102'39 uv39 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.671+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c7b20 con 0x563cf20bac30 2023-11-15T03:04:58.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.671+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v102'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.671+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c7b20 con 0x563cf20bac30 2023-11-15T03:04:58.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.675+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v102'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.675+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c7b20 con 0x563cf20bac30 2023-11-15T03:04:58.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.679+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v102'90 uv82 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.679+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20c7b20 con 0x563cf20bac30 2023-11-15T03:04:58.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.679+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v102'91 uv91 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.679+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c7b20 con 0x563cf20bac30 2023-11-15T03:04:58.691 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:04:58.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.683+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v102'92 uv92 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.683+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c7b20 con 0x563cf20bac30 2023-11-15T03:04:58.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.687+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v102'93 uv93 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.687+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c7b20 con 0x563cf20bac30 2023-11-15T03:04:58.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.691+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v102'94 uv94 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.691+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c7b20 con 0x563cf20bac30 2023-11-15T03:04:58.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.691+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v102'95 uv89 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.691+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x563cf20c7b20 con 0x563cf20bac30 2023-11-15T03:04:58.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.695+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v102'96 uv96 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.695+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c7b20 con 0x563cf20bac30 2023-11-15T03:04:58.705 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:04:58.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.699+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v102'97 uv97 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.699+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c8b30 con 0x563cf20bac30 2023-11-15T03:04:58.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.699+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v102'98 uv98 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.699+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c8b30 con 0x563cf20bac30 2023-11-15T03:04:58.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.703+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v102'99 uv99 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.703+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c7b20 con 0x563cf20bac30 2023-11-15T03:04:58.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v102'100 uv100 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e102) v8 -- 0x563cf20c8010 con 0x563cf20bac30 2023-11-15T03:04:58.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v102'101 uv101 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1ca0009b10 con 0x563cf20bac30 2023-11-15T03:04:58.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x563cf20b5f80 msgr2=0x563cf20b8370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x563cf20b5f80 0x563cf20b8370 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x563cf208a5e0 msgr2=0x563cf1ffea20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x563cf208a5e0 0x563cf1ffea20 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x563cf20bdec0 msgr2=0x563cf20c02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x563cf20bdec0 0x563cf20c02b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x563cf20b12c0 msgr2=0x563cf20b36b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x563cf20b12c0 0x563cf20b36b0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x563cf20c40c0 msgr2=0x563cf20c7630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x563cf20c40c0 0x563cf20c7630 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x563cf20bac30 msgr2=0x563cf20bd020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x563cf20bac30 0x563cf20bd020 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x563cf20c15b0 msgr2=0x563cf20c1a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x563cf20c15b0 0x563cf20c1a00 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 reap_dead start 2023-11-15T03:04:58.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 reap_dead start 2023-11-15T03:04:58.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 reap_dead start 2023-11-15T03:04:58.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca512d700 1 -- 172.21.15.121:0/3603045108 reap_dead start 2023-11-15T03:04:58.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1c84059020 msgr2=0x7f1c8405b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1c84059020 0x7f1c8405b4e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x563cf2003160 tx=0x7f1c94012000).stop 2023-11-15T03:04:58.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cf1fb5e10 msgr2=0x563cf2089fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.707+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cf1fb5e10 0x563cf2089fc0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7f1c8c00a5a0 tx=0x7f1c8c016000).stop 2023-11-15T03:04:58.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.711+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 shutdown_connections 2023-11-15T03:04:58.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.711+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1c84059020 0x7f1c8405b4e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.711+0000 7f1ca6cfad00 1 --2- 172.21.15.121:0/3603045108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cf1fb5e10 0x563cf2089fc0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.711+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 >> 172.21.15.121:0/3603045108 conn(0x563cf2005080 msgr2=0x563cf1fb4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:58.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.711+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 shutdown_connections 2023-11-15T03:04:58.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.711+0000 7f1ca6cfad00 1 -- 172.21.15.121:0/3603045108 wait complete. 2023-11-15T03:04:58.724 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds_snaptable 2023-11-15T03:04:58.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.819+0000 7fb0eb33dd00 1 Processor -- start 2023-11-15T03:04:58.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.819+0000 7fb0eb33dd00 1 -- start start 2023-11-15T03:04:58.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.819+0000 7fb0eb33dd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfdf24ddd0 0x55cfdf24d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.819+0000 7fb0eb33dd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cfdf297b00 0x55cfdf320430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.819+0000 7fb0eb33dd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfdf326670 0x55cfdf292ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.819+0000 7fb0eb33dd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfdf1c6220 con 0x55cfdf326670 2023-11-15T03:04:58.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.819+0000 7fb0eb33dd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfdf297000 con 0x55cfdf24ddd0 2023-11-15T03:04:58.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.819+0000 7fb0eb33dd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfdf296a40 con 0x55cfdf297b00 2023-11-15T03:04:58.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.819+0000 7fb0e0046700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfdf326670 0x55cfdf292ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.819+0000 7fb0e0046700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfdf326670 0x55cfdf292ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34214/0 (socket says 172.21.15.121:34214) 2023-11-15T03:04:58.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.819+0000 7fb0e0046700 1 -- 172.21.15.121:0/3241987010 learned_addr learned my addr 172.21.15.121:0/3241987010 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:58.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.819+0000 7fb0df845700 1 --2- 172.21.15.121:0/3241987010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfdf24ddd0 0x55cfdf24d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.819+0000 7fb0d7044700 1 --2- 172.21.15.121:0/3241987010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cfdf297b00 0x55cfdf320430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.819+0000 7fb0e0046700 1 -- 172.21.15.121:0/3241987010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cfdf297b00 msgr2=0x55cfdf320430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:58.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.819+0000 7fb0e0046700 1 --2- 172.21.15.121:0/3241987010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cfdf297b00 0x55cfdf320430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.823+0000 7fb0e0046700 1 -- 172.21.15.121:0/3241987010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfdf24ddd0 msgr2=0x55cfdf24d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:58.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.823+0000 7fb0e0046700 1 --2- 172.21.15.121:0/3241987010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfdf24ddd0 0x55cfdf24d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.823+0000 7fb0e0046700 1 -- 172.21.15.121:0/3241987010 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d000a180 con 0x55cfdf326670 2023-11-15T03:04:58.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.823+0000 7fb0e0046700 1 --2- 172.21.15.121:0/3241987010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfdf326670 0x55cfdf292ff0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7fb0d800a460 tx=0x7fb0d8005a00).ready entity=mon.0 client_cookie=49b36bc69e050d13 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:58.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.823+0000 7fb0de843700 1 -- 172.21.15.121:0/3241987010 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d800adb0 con 0x55cfdf326670 2023-11-15T03:04:58.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.823+0000 7fb0de843700 1 -- 172.21.15.121:0/3241987010 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d800c410 con 0x55cfdf326670 2023-11-15T03:04:58.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.823+0000 7fb0de843700 1 -- 172.21.15.121:0/3241987010 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d800c7d0 con 0x55cfdf326670 2023-11-15T03:04:58.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.823+0000 7fb0eb33dd00 1 -- 172.21.15.121:0/3241987010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfdf326670 msgr2=0x55cfdf292ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.823+0000 7fb0eb33dd00 1 --2- 172.21.15.121:0/3241987010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfdf326670 0x55cfdf292ff0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7fb0d800a460 tx=0x7fb0d8005a00).stop 2023-11-15T03:04:58.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.823+0000 7fb0eb33dd00 1 -- 172.21.15.121:0/3241987010 shutdown_connections 2023-11-15T03:04:58.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.823+0000 7fb0eb33dd00 1 --2- 172.21.15.121:0/3241987010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfdf326670 0x55cfdf292ff0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.823+0000 7fb0eb33dd00 1 --2- 172.21.15.121:0/3241987010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cfdf297b00 0x55cfdf320430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.823+0000 7fb0eb33dd00 1 --2- 172.21.15.121:0/3241987010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfdf24ddd0 0x55cfdf24d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.823+0000 7fb0eb33dd00 1 -- 172.21.15.121:0/3241987010 >> 172.21.15.121:0/3241987010 conn(0x55cfdf29d080 msgr2=0x55cfdf29b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:58.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.823+0000 7fb0eb33dd00 1 -- 172.21.15.121:0/3241987010 shutdown_connections 2023-11-15T03:04:58.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.823+0000 7fb0eb33dd00 1 -- 172.21.15.121:0/3241987010 wait complete. 2023-11-15T03:04:58.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.823+0000 7fb0eb33dd00 1 Processor -- start 2023-11-15T03:04:58.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.823+0000 7fb0eb33dd00 1 -- start start 2023-11-15T03:04:58.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0eb33dd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfdf24ddd0 0x55cfdf27f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0eb33dd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cfdf297b00 0x55cfdf31d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0eb33dd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfdf326670 0x55cfdf29aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0eb33dd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfdf31fa00 con 0x55cfdf326670 2023-11-15T03:04:58.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0eb33dd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfdf293890 con 0x55cfdf24ddd0 2023-11-15T03:04:58.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0eb33dd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfdf31f690 con 0x55cfdf297b00 2023-11-15T03:04:58.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0e0046700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfdf326670 0x55cfdf29aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0e0046700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfdf326670 0x55cfdf29aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34230/0 (socket says 172.21.15.121:34230) 2023-11-15T03:04:58.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0e0046700 1 -- 172.21.15.121:0/1571854450 learned_addr learned my addr 172.21.15.121:0/1571854450 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:58.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0df845700 1 --2- 172.21.15.121:0/1571854450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfdf24ddd0 0x55cfdf27f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0d7044700 1 --2- 172.21.15.121:0/1571854450 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cfdf297b00 0x55cfdf31d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0e0046700 1 -- 172.21.15.121:0/1571854450 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cfdf297b00 msgr2=0x55cfdf31d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0e0046700 1 --2- 172.21.15.121:0/1571854450 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cfdf297b00 0x55cfdf31d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0e0046700 1 -- 172.21.15.121:0/1571854450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfdf24ddd0 msgr2=0x55cfdf27f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0e0046700 1 --2- 172.21.15.121:0/1571854450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfdf24ddd0 0x55cfdf27f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0e0046700 1 -- 172.21.15.121:0/1571854450 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d8005ca0 con 0x55cfdf326670 2023-11-15T03:04:58.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0df845700 1 --2- 172.21.15.121:0/1571854450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfdf24ddd0 0x55cfdf27f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:04:58.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0e0046700 1 --2- 172.21.15.121:0/1571854450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfdf326670 0x55cfdf29aea0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7fb0d8008000 tx=0x7fb0d8005a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:58.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0dd040700 1 -- 172.21.15.121:0/1571854450 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d801c070 con 0x55cfdf326670 2023-11-15T03:04:58.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0dd040700 1 -- 172.21.15.121:0/1571854450 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d8005420 con 0x55cfdf326670 2023-11-15T03:04:58.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0dd040700 1 -- 172.21.15.121:0/1571854450 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80157f0 con 0x55cfdf326670 2023-11-15T03:04:58.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0eb33dd00 1 -- 172.21.15.121:0/1571854450 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cfdf1ef770 con 0x55cfdf326670 2023-11-15T03:04:58.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.827+0000 7fb0eb33dd00 1 -- 172.21.15.121:0/1571854450 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfdf1bea70 con 0x55cfdf326670 2023-11-15T03:04:58.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.831+0000 7fb0dd040700 1 -- 172.21.15.121:0/1571854450 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb0d8015c20 con 0x55cfdf326670 2023-11-15T03:04:58.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.831+0000 7fb0dd040700 1 --2- 172.21.15.121:0/1571854450 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb0c0058ff0 0x7fb0c005b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.831+0000 7fb0dd040700 1 -- 172.21.15.121:0/1571854450 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6954+0+0 (secure 0 0 0) 0x7fb0d8074c40 con 0x55cfdf326670 2023-11-15T03:04:58.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.835+0000 7fb0df845700 1 --2- 172.21.15.121:0/1571854450 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb0c0058ff0 0x7fb0c005b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.835+0000 7fb0eb33dd00 1 --2- 172.21.15.121:0/1571854450 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55cfdf326c50 0x55cfdf27d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:58.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.835+0000 7fb0eb33dd00 1 -- 172.21.15.121:0/1571854450 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:1 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e102) v8 -- 0x55cfdf293160 con 0x55cfdf326c50 2023-11-15T03:04:58.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.835+0000 7fb0d7044700 1 --2- 172.21.15.121:0/1571854450 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55cfdf326c50 0x55cfdf27d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:58.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.835+0000 7fb0df845700 1 --2- 172.21.15.121:0/1571854450 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb0c0058ff0 0x7fb0c005b4b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fb0d000a510 tx=0x7fb0d0014000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:58.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.835+0000 7fb0d7044700 1 --2- 172.21.15.121:0/1571854450 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55cfdf326c50 0x55cfdf27d110 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:58.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.839+0000 7fb0d7044700 1 -- 172.21.15.121:0/1571854450 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(1 mds_snaptable [delete] v102'42 uv42 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb0c800c900 con 0x55cfdf326c50 2023-11-15T03:04:58.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.839+0000 7fb0eb33dd00 1 -- 172.21.15.121:0/1571854450 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55cfdf326c50 msgr2=0x55cfdf27d110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.839+0000 7fb0eb33dd00 1 --2- 172.21.15.121:0/1571854450 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55cfdf326c50 0x55cfdf27d110 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:58.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.839+0000 7fb0eb33dd00 1 -- 172.21.15.121:0/1571854450 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb0c0058ff0 msgr2=0x7fb0c005b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.839+0000 7fb0eb33dd00 1 --2- 172.21.15.121:0/1571854450 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb0c0058ff0 0x7fb0c005b4b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fb0d000a510 tx=0x7fb0d0014000).stop 2023-11-15T03:04:58.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.839+0000 7fb0eb33dd00 1 -- 172.21.15.121:0/1571854450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfdf326670 msgr2=0x55cfdf29aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:58.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.839+0000 7fb0eb33dd00 1 --2- 172.21.15.121:0/1571854450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfdf326670 0x55cfdf29aea0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7fb0d8008000 tx=0x7fb0d8005a00).stop 2023-11-15T03:04:58.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.839+0000 7fb0e0046700 1 -- 172.21.15.121:0/1571854450 reap_dead start 2023-11-15T03:04:58.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.839+0000 7fb0eb33dd00 1 -- 172.21.15.121:0/1571854450 shutdown_connections 2023-11-15T03:04:58.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.839+0000 7fb0eb33dd00 1 -- 172.21.15.121:0/1571854450 >> 172.21.15.121:0/1571854450 conn(0x55cfdf29d080 msgr2=0x55cfdf2805a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:58.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.839+0000 7fb0eb33dd00 1 -- 172.21.15.121:0/1571854450 shutdown_connections 2023-11-15T03:04:58.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:58.839+0000 7fb0eb33dd00 1 -- 172.21.15.121:0/1571854450 wait complete. 2023-11-15T03:04:58.852 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:04:59.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.019+0000 7f571345b700 1 Processor -- start 2023-11-15T03:04:59.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.019+0000 7f571345b700 1 -- start start 2023-11-15T03:04:59.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.023+0000 7f571345b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f570c067020 0x7f570c1582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:59.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.023+0000 7f571345b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f570c068170 0x7f570c0bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:59.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.023+0000 7f571345b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f570c062da0 con 0x7f570c157ed0 2023-11-15T03:04:59.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.023+0000 7f571345b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f570c0a93a0 con 0x7f570c068170 2023-11-15T03:04:59.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.023+0000 7f571345b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570c062190 con 0x7f570c067020 2023-11-15T03:04:59.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.023+0000 7f57111f7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f570c157ed0 0x7f570c067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53992/0 (socket says 172.21.15.121:53992) 2023-11-15T03:04:59.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.023+0000 7f57111f7700 1 -- 172.21.15.121:0/1954915591 learned_addr learned my addr 172.21.15.121:0/1954915591 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:04:59.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.023+0000 7f57109f6700 1 --2- 172.21.15.121:0/1954915591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f570c067020 0x7f570c1582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:59.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.023+0000 7f57119f8700 1 --2- 172.21.15.121:0/1954915591 >> v2:172.21.15.190:3300/0 conn(0x7f570c068170 0x7f570c0bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:59.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.023+0000 7f5703fff700 1 -- 172.21.15.121:0/1954915591 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668040179 0 0) 0x7f570c062da0 con 0x7f570c157ed0 2023-11-15T03:04:59.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.023+0000 7f5703fff700 1 -- 172.21.15.121:0/1954915591 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56f80033f0 con 0x7f570c157ed0 2023-11-15T03:04:59.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.023+0000 7f57109f6700 1 -- 172.21.15.121:0/1954915591 >> v2:172.21.15.190:3300/0 conn(0x7f570c068170 msgr2=0x7f570c0bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:04:59.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.023+0000 7f57109f6700 1 --2- 172.21.15.121:0/1954915591 >> v2:172.21.15.190:3300/0 conn(0x7f570c068170 0x7f570c0bc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:59.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.023+0000 7f57109f6700 1 -- 172.21.15.121:0/1954915591 >> v1:172.21.15.121:6789/0 conn(0x7f570c157ed0 legacy=0x7f570c067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:59.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.023+0000 7f57109f6700 1 -- 172.21.15.121:0/1954915591 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570c062470 con 0x7f570c067020 2023-11-15T03:04:59.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.023+0000 7f57109f6700 1 --2- 172.21.15.121:0/1954915591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f570c067020 0x7f570c1582b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f56fc009760 tx=0x7f56fc009060).ready entity=mon.2 client_cookie=56af16e0a7531f8e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:59.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.027+0000 7f5703fff700 1 -- 172.21.15.121:0/1954915591 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc00d4a0 con 0x7f570c067020 2023-11-15T03:04:59.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.027+0000 7f5703fff700 1 -- 172.21.15.121:0/1954915591 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56fc00d600 con 0x7f570c067020 2023-11-15T03:04:59.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.027+0000 7f5703fff700 1 -- 172.21.15.121:0/1954915591 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc00d980 con 0x7f570c067020 2023-11-15T03:04:59.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.027+0000 7f571345b700 1 -- 172.21.15.121:0/1954915591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f570c067020 msgr2=0x7f570c1582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:59.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.027+0000 7f571345b700 1 --2- 172.21.15.121:0/1954915591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f570c067020 0x7f570c1582b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f56fc009760 tx=0x7f56fc009060).stop 2023-11-15T03:04:59.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.027+0000 7f571345b700 1 -- 172.21.15.121:0/1954915591 shutdown_connections 2023-11-15T03:04:59.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.027+0000 7f571345b700 1 --2- 172.21.15.121:0/1954915591 >> v2:172.21.15.190:3300/0 conn(0x7f570c068170 0x7f570c0bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:59.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.027+0000 7f571345b700 1 --2- 172.21.15.121:0/1954915591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f570c067020 0x7f570c1582b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:59.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.027+0000 7f571345b700 1 -- 172.21.15.121:0/1954915591 >> 172.21.15.121:0/1954915591 conn(0x7f570c0591c0 msgr2=0x7f570c0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:04:59.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.027+0000 7f571345b700 1 -- 172.21.15.121:0/1954915591 shutdown_connections 2023-11-15T03:04:59.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.027+0000 7f571345b700 1 -- 172.21.15.121:0/1954915591 wait complete. 2023-11-15T03:04:59.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.031+0000 7f571345b700 1 Processor -- start 2023-11-15T03:04:59.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.031+0000 7f571345b700 1 -- start start 2023-11-15T03:04:59.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.031+0000 7f571345b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f570c067020 0x7f570c146300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:59.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.031+0000 7f571345b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f570c068170 0x7f570c0be1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:59.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.031+0000 7f571345b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f570c157ed0 0x7f570c0c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:59.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.031+0000 7f571345b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c0a0410 con 0x7f570c157ed0 2023-11-15T03:04:59.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.031+0000 7f571345b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c0a0a50 con 0x7f570c068170 2023-11-15T03:04:59.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.031+0000 7f571345b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570c09f4b0 con 0x7f570c067020 2023-11-15T03:04:59.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.031+0000 7f57119f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f570c157ed0 0x7f570c0c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:59.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.035+0000 7f57119f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f570c157ed0 0x7f570c0c25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34234/0 (socket says 172.21.15.121:34234) 2023-11-15T03:04:59.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.035+0000 7f57119f8700 1 -- 172.21.15.121:0/1578041492 learned_addr learned my addr 172.21.15.121:0/1578041492 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:04:59.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.035+0000 7f57111f7700 1 --2- 172.21.15.121:0/1578041492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f570c067020 0x7f570c146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:59.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.035+0000 7f57109f6700 1 --2- 172.21.15.121:0/1578041492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f570c068170 0x7f570c0be1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:59.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.035+0000 7f57119f8700 1 -- 172.21.15.121:0/1578041492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f570c067020 msgr2=0x7f570c146300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:59.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.035+0000 7f57119f8700 1 --2- 172.21.15.121:0/1578041492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f570c067020 0x7f570c146300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:59.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.035+0000 7f57119f8700 1 -- 172.21.15.121:0/1578041492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f570c068170 msgr2=0x7f570c0be1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:04:59.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.035+0000 7f57119f8700 1 --2- 172.21.15.121:0/1578041492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f570c068170 0x7f570c0be1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:04:59.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.035+0000 7f57119f8700 1 -- 172.21.15.121:0/1578041492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56fc0092a0 con 0x7f570c157ed0 2023-11-15T03:04:59.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.035+0000 7f57109f6700 1 --2- 172.21.15.121:0/1578041492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f570c068170 0x7f570c0be1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:59.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.035+0000 7f57111f7700 1 --2- 172.21.15.121:0/1578041492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f570c067020 0x7f570c146300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:04:59.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.035+0000 7f57119f8700 1 --2- 172.21.15.121:0/1578041492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f570c157ed0 0x7f570c0c25a0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f5704009ec0 tx=0x7f570400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:59.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.035+0000 7f57017fa700 1 -- 172.21.15.121:0/1578041492 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570400cc20 con 0x7f570c157ed0 2023-11-15T03:04:59.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.035+0000 7f57017fa700 1 -- 172.21.15.121:0/1578041492 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570400f040 con 0x7f570c157ed0 2023-11-15T03:04:59.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.035+0000 7f57017fa700 1 -- 172.21.15.121:0/1578041492 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704013400 con 0x7f570c157ed0 2023-11-15T03:04:59.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.035+0000 7f571345b700 1 -- 172.21.15.121:0/1578041492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f570c09f790 con 0x7f570c157ed0 2023-11-15T03:04:59.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.035+0000 7f571345b700 1 -- 172.21.15.121:0/1578041492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f570c09fd50 con 0x7f570c157ed0 2023-11-15T03:04:59.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.035+0000 7f571345b700 1 -- 172.21.15.121:0/1578041492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f570c0a0550 con 0x7f570c157ed0 2023-11-15T03:04:59.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.039+0000 7f57017fa700 1 -- 172.21.15.121:0/1578041492 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5704013770 con 0x7f570c157ed0 2023-11-15T03:04:59.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.039+0000 7f57017fa700 1 --2- 172.21.15.121:0/1578041492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f56f005a510 0x7f56f005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:04:59.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.039+0000 7f57017fa700 1 -- 172.21.15.121:0/1578041492 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6954+0+0 (secure 0 0 0) 0x7f5704075540 con 0x7f570c157ed0 2023-11-15T03:04:59.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.039+0000 7f57111f7700 1 --2- 172.21.15.121:0/1578041492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f56f005a510 0x7f56f005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:04:59.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.039+0000 7f57111f7700 1 --2- 172.21.15.121:0/1578041492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f56f005a510 0x7f56f005c9d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f5708005750 tx=0x7f5708009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:04:59.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.047+0000 7f57017fa700 1 -- 172.21.15.121:0/1578041492 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f570c0a0550 con 0x7f570c157ed0 2023-11-15T03:04:59.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:04:59.247+0000 7f571345b700 1 -- 172.21.15.121:0/1578041492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f570c0a0550 con 0x7f570c157ed0 2023-11-15T03:05:00.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.211+0000 7f57017fa700 1 -- 172.21.15.121:0/1578041492 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v132) v1 ==== 162+0+0 (secure 0 0 0) 0x7f570c0a0550 con 0x7f570c157ed0 2023-11-15T03:05:00.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.215+0000 7f571345b700 1 -- 172.21.15.121:0/1578041492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f56f005a510 msgr2=0x7f56f005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:00.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.215+0000 7f571345b700 1 --2- 172.21.15.121:0/1578041492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f56f005a510 0x7f56f005c9d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f5708005750 tx=0x7f5708009040).stop 2023-11-15T03:05:00.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.215+0000 7f571345b700 1 -- 172.21.15.121:0/1578041492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f570c157ed0 msgr2=0x7f570c0c25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:00.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.215+0000 7f571345b700 1 --2- 172.21.15.121:0/1578041492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f570c157ed0 0x7f570c0c25a0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f5704009ec0 tx=0x7f570400b040).stop 2023-11-15T03:05:00.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.215+0000 7f571345b700 1 -- 172.21.15.121:0/1578041492 shutdown_connections 2023-11-15T03:05:00.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.215+0000 7f571345b700 1 --2- 172.21.15.121:0/1578041492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f56f005a510 0x7f56f005c9d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:00.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.215+0000 7f571345b700 1 --2- 172.21.15.121:0/1578041492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f570c157ed0 0x7f570c0c25a0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:00.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.215+0000 7f571345b700 1 --2- 172.21.15.121:0/1578041492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f570c068170 0x7f570c0be1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:00.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.215+0000 7f571345b700 1 --2- 172.21.15.121:0/1578041492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f570c067020 0x7f570c146300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:00.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.215+0000 7f571345b700 1 -- 172.21.15.121:0/1578041492 >> 172.21.15.121:0/1578041492 conn(0x7f570c0591c0 msgr2=0x7f570c0c5330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:00.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.215+0000 7f571345b700 1 -- 172.21.15.121:0/1578041492 shutdown_connections 2023-11-15T03:05:00.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.215+0000 7f571345b700 1 -- 172.21.15.121:0/1578041492 wait complete. 2023-11-15T03:05:00.226 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:05:00.231 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:05:00.223+0000 7fd1b05ea700 -1 mds.0.snaptable: load_2 could not read table: -2 2023-11-15T03:05:00.232 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:05:00.223+0000 7fd1b05ea700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2023-11-15T03:05:00.245 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:05:00.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.399+0000 7f3710ff9700 1 Processor -- start 2023-11-15T03:05:00.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.399+0000 7f3710ff9700 1 -- start start 2023-11-15T03:05:00.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.403+0000 7f3710ff9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f370c0cabd0 0x7f370c0cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:00.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.403+0000 7f3710ff9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f370c0ad0f0 0x7f370c0ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:00.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.403+0000 7f3710ff9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370c0ad650 con 0x7f370c0acc90 2023-11-15T03:05:00.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.403+0000 7f3710ff9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370c065fc0 con 0x7f370c0ad0f0 2023-11-15T03:05:00.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.403+0000 7f3710ff9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370c0662a0 con 0x7f370c0cabd0 2023-11-15T03:05:00.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.403+0000 7f370bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f370c0ad0f0 0x7f370c0ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:00.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.403+0000 7f370bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f370c0ad0f0 0x7f370c0ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34236/0 (socket says 172.21.15.121:34236) 2023-11-15T03:05:00.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.403+0000 7f370bfff700 1 -- 172.21.15.121:0/2222836914 learned_addr learned my addr 172.21.15.121:0/2222836914 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:00.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.403+0000 7f370b7fe700 1 --2- 172.21.15.121:0/2222836914 >> v2:172.21.15.190:3300/0 conn(0x7f370c0cabd0 0x7f370c0cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:00.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.403+0000 7f370bfff700 1 -- 172.21.15.121:0/2222836914 >> v2:172.21.15.190:3300/0 conn(0x7f370c0cabd0 msgr2=0x7f370c0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:00.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.403+0000 7f370bfff700 1 --2- 172.21.15.121:0/2222836914 >> v2:172.21.15.190:3300/0 conn(0x7f370c0cabd0 0x7f370c0cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:00.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.403+0000 7f370bfff700 1 -- 172.21.15.121:0/2222836914 >> v1:172.21.15.190:6789/0 conn(0x7f370c0acc90 legacy=0x7f370c066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:00.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.403+0000 7f370bfff700 1 -- 172.21.15.121:0/2222836914 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c066580 con 0x7f370c0ad0f0 2023-11-15T03:05:00.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.407+0000 7f370bfff700 1 --2- 172.21.15.121:0/2222836914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c0ad0f0 0x7f370c0ae190 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f370c064750 tx=0x7f3704009840).ready entity=mon.0 client_cookie=83c2e9dcd405c1ef server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:00.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.407+0000 7f370a7fc700 1 -- 172.21.15.121:0/2222836914 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370400e070 con 0x7f370c0ad0f0 2023-11-15T03:05:00.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.407+0000 7f370a7fc700 1 -- 172.21.15.121:0/2222836914 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370400ba50 con 0x7f370c0ad0f0 2023-11-15T03:05:00.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.407+0000 7f370a7fc700 1 -- 172.21.15.121:0/2222836914 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370400bdd0 con 0x7f370c0ad0f0 2023-11-15T03:05:00.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.407+0000 7f3710ff9700 1 -- 172.21.15.121:0/2222836914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c0ad0f0 msgr2=0x7f370c0ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:00.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.407+0000 7f3710ff9700 1 --2- 172.21.15.121:0/2222836914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c0ad0f0 0x7f370c0ae190 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f370c064750 tx=0x7f3704009840).stop 2023-11-15T03:05:00.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.407+0000 7f3710ff9700 1 -- 172.21.15.121:0/2222836914 shutdown_connections 2023-11-15T03:05:00.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.407+0000 7f3710ff9700 1 --2- 172.21.15.121:0/2222836914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c0ad0f0 0x7f370c0ae190 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:00.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.407+0000 7f3710ff9700 1 --2- 172.21.15.121:0/2222836914 >> v2:172.21.15.190:3300/0 conn(0x7f370c0cabd0 0x7f370c0cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:00.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.407+0000 7f3710ff9700 1 -- 172.21.15.121:0/2222836914 >> 172.21.15.121:0/2222836914 conn(0x7f370c062190 msgr2=0x7f370c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:00.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.407+0000 7f3710ff9700 1 -- 172.21.15.121:0/2222836914 shutdown_connections 2023-11-15T03:05:00.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.407+0000 7f3710ff9700 1 -- 172.21.15.121:0/2222836914 wait complete. 2023-11-15T03:05:00.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.407+0000 7f3710ff9700 1 Processor -- start 2023-11-15T03:05:00.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f3710ff9700 1 -- start start 2023-11-15T03:05:00.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f3710ff9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c0acc90 0x7f370c151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:00.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f3710ff9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f370c0ad0f0 0x7f370c140330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:00.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f3710ff9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f370c0cabd0 0x7f370c159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:00.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f3710ff9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c151750 con 0x7f370c0acc90 2023-11-15T03:05:00.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f3710ff9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c167930 con 0x7f370c0ad0f0 2023-11-15T03:05:00.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f3710ff9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c156150 con 0x7f370c0cabd0 2023-11-15T03:05:00.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f370b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c0acc90 0x7f370c151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:00.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f370b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c0acc90 0x7f370c151260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34246/0 (socket says 172.21.15.121:34246) 2023-11-15T03:05:00.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f370b7fe700 1 -- 172.21.15.121:0/597698003 learned_addr learned my addr 172.21.15.121:0/597698003 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:00.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f370bfff700 1 --2- 172.21.15.121:0/597698003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f370c0cabd0 0x7f370c159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:00.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f370affd700 1 --2- 172.21.15.121:0/597698003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f370c0ad0f0 0x7f370c140330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:00.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f370b7fe700 1 -- 172.21.15.121:0/597698003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f370c0cabd0 msgr2=0x7f370c159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:00.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f370b7fe700 1 --2- 172.21.15.121:0/597698003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f370c0cabd0 0x7f370c159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:00.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f370b7fe700 1 -- 172.21.15.121:0/597698003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f370c0ad0f0 msgr2=0x7f370c140330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:00.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f370b7fe700 1 --2- 172.21.15.121:0/597698003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f370c0ad0f0 0x7f370c140330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:00.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f370b7fe700 1 -- 172.21.15.121:0/597698003 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3704009ae0 con 0x7f370c0acc90 2023-11-15T03:05:00.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f370affd700 1 --2- 172.21.15.121:0/597698003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f370c0ad0f0 0x7f370c140330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:00.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f370b7fe700 1 --2- 172.21.15.121:0/597698003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c0acc90 0x7f370c151260 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0x7f36fc00d860 tx=0x7f36fc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:00.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f36f37fe700 1 -- 172.21.15.121:0/597698003 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc011590 con 0x7f370c0acc90 2023-11-15T03:05:00.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f36f37fe700 1 -- 172.21.15.121:0/597698003 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc0116f0 con 0x7f370c0acc90 2023-11-15T03:05:00.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f3710ff9700 1 -- 172.21.15.121:0/597698003 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370c156430 con 0x7f370c0acc90 2023-11-15T03:05:00.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f36f37fe700 1 -- 172.21.15.121:0/597698003 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc011a70 con 0x7f370c0acc90 2023-11-15T03:05:00.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.411+0000 7f3710ff9700 1 -- 172.21.15.121:0/597698003 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370c14daa0 con 0x7f370c0acc90 2023-11-15T03:05:00.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.415+0000 7f3710ff9700 1 -- 172.21.15.121:0/597698003 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370c06bcf0 con 0x7f370c0acc90 2023-11-15T03:05:00.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.415+0000 7f36f37fe700 1 -- 172.21.15.121:0/597698003 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f36fc012420 con 0x7f370c0acc90 2023-11-15T03:05:00.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.415+0000 7f36f37fe700 1 --2- 172.21.15.121:0/597698003 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f36f805a4e0 0x7f36f805c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:00.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.415+0000 7f36f37fe700 1 -- 172.21.15.121:0/597698003 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7040+0+0 (secure 0 0 0) 0x7f36fc0744f0 con 0x7f370c0acc90 2023-11-15T03:05:00.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.415+0000 7f370affd700 1 --2- 172.21.15.121:0/597698003 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f36f805a4e0 0x7f36f805c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:00.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.419+0000 7f370affd700 1 --2- 172.21.15.121:0/597698003 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f36f805a4e0 0x7f36f805c9a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f370c14ff60 tx=0x7f3700009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:00.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.419+0000 7f36f37fe700 1 -- 172.21.15.121:0/597698003 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f36fc0478e0 con 0x7f370c0acc90 2023-11-15T03:05:00.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.611+0000 7f3710ff9700 1 -- 172.21.15.121:0/597698003 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f370c066860 con 0x7f370c0acc90 2023-11-15T03:05:00.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.615+0000 7f36f37fe700 1 -- 172.21.15.121:0/597698003 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+4502 (secure 0 0 0) 0x7f36fc011e60 con 0x7f370c0acc90 2023-11-15T03:05:00.622 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:00.622 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":133,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":133,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:00.215408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8088},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8088":{"gid":8088,"name":"a","rank":0,"incarnation":133,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/682793496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":682793496},{"type":"v1","addr":"172.21.15.121:6836","nonce":682793496}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:05:00.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.619+0000 7f3710ff9700 1 -- 172.21.15.121:0/597698003 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f36f805a4e0 msgr2=0x7f36f805c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:00.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.619+0000 7f3710ff9700 1 --2- 172.21.15.121:0/597698003 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f36f805a4e0 0x7f36f805c9a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f370c14ff60 tx=0x7f3700009040).stop 2023-11-15T03:05:00.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.619+0000 7f3710ff9700 1 -- 172.21.15.121:0/597698003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c0acc90 msgr2=0x7f370c151260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:00.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.619+0000 7f3710ff9700 1 --2- 172.21.15.121:0/597698003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c0acc90 0x7f370c151260 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0x7f36fc00d860 tx=0x7f36fc008040).stop 2023-11-15T03:05:00.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.619+0000 7f3710ff9700 1 -- 172.21.15.121:0/597698003 shutdown_connections 2023-11-15T03:05:00.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.619+0000 7f3710ff9700 1 --2- 172.21.15.121:0/597698003 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f36f805a4e0 0x7f36f805c9a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:00.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.619+0000 7f3710ff9700 1 --2- 172.21.15.121:0/597698003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f370c0cabd0 0x7f370c159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:00.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.619+0000 7f3710ff9700 1 --2- 172.21.15.121:0/597698003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f370c0ad0f0 0x7f370c140330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:00.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.619+0000 7f3710ff9700 1 --2- 172.21.15.121:0/597698003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c0acc90 0x7f370c151260 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:00.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.619+0000 7f3710ff9700 1 -- 172.21.15.121:0/597698003 >> 172.21.15.121:0/597698003 conn(0x7f370c062190 msgr2=0x7f370c150590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:00.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.619+0000 7f3710ff9700 1 -- 172.21.15.121:0/597698003 shutdown_connections 2023-11-15T03:05:00.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:00.619+0000 7f3710ff9700 1 -- 172.21.15.121:0/597698003 wait complete. 2023-11-15T03:05:00.630 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 133 2023-11-15T03:05:00.649 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:05:01.226 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:05:01.219+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:05:01.226 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:05:01.219+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:05:01.234 INFO:tasks.ceph.mds.a.smithi121.stderr: -28> 2023-11-15T03:05:00.223+0000 7fd1b05ea700 -1 mds.0.snaptable: load_2 could not read table: -2 2023-11-15T03:05:01.235 INFO:tasks.ceph.mds.a.smithi121.stderr: -27> 2023-11-15T03:05:00.223+0000 7fd1b05ea700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2023-11-15T03:05:01.274 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:05:05.650 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:05:05.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.807+0000 7f4ee98db700 1 Processor -- start 2023-11-15T03:05:05.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.807+0000 7f4ee98db700 1 -- start start 2023-11-15T03:05:05.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.811+0000 7f4ee98db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4ee4150ff0 0x7f4ee4151470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:05.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.811+0000 7f4ee98db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee4152000 con 0x7f4ee414af80 2023-11-15T03:05:05.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.811+0000 7f4ee98db700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee40c1270 con 0x7f4ee40c6640 2023-11-15T03:05:05.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.811+0000 7f4ee98db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee40c1e10 con 0x7f4ee4150ff0 2023-11-15T03:05:05.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.811+0000 7f4ee2ffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4ee414af80 0x7f4ee40c1700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49614/0 (socket says 172.21.15.121:49614) 2023-11-15T03:05:05.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.811+0000 7f4ee2ffd700 1 -- 172.21.15.121:0/195462893 learned_addr learned my addr 172.21.15.121:0/195462893 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:05.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.811+0000 7f4edbfff700 1 --2- 172.21.15.121:0/195462893 >> v2:172.21.15.190:3300/0 conn(0x7f4ee4150ff0 0x7f4ee4151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:05.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.811+0000 7f4ee27fc700 1 -- 172.21.15.121:0/195462893 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987503177 0 0) 0x7f4ed4001110 con 0x7f4ee414af80 2023-11-15T03:05:05.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.811+0000 7f4ee27fc700 1 -- 172.21.15.121:0/195462893 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ed00033f0 con 0x7f4ee414af80 2023-11-15T03:05:05.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.811+0000 7f4edbfff700 1 -- 172.21.15.121:0/195462893 >> v1:172.21.15.190:6789/0 conn(0x7f4ee40c6640 legacy=0x7f4ee40a79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:05.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.811+0000 7f4edbfff700 1 -- 172.21.15.121:0/195462893 >> v1:172.21.15.121:6789/0 conn(0x7f4ee414af80 legacy=0x7f4ee40c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:05.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.815+0000 7f4edbfff700 1 -- 172.21.15.121:0/195462893 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee4151a40 con 0x7f4ee4150ff0 2023-11-15T03:05:05.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.815+0000 7f4edbfff700 1 --2- 172.21.15.121:0/195462893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee4150ff0 0x7f4ee4151470 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f4ecc009760 tx=0x7f4ecc009060).ready entity=mon.1 client_cookie=74450c0ef122e8d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:05.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.815+0000 7f4ee27fc700 1 -- 172.21.15.121:0/195462893 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc00f510 con 0x7f4ee4150ff0 2023-11-15T03:05:05.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.815+0000 7f4ee27fc700 1 -- 172.21.15.121:0/195462893 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ecc00f670 con 0x7f4ee4150ff0 2023-11-15T03:05:05.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.815+0000 7f4ee27fc700 1 -- 172.21.15.121:0/195462893 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc00f9f0 con 0x7f4ee4150ff0 2023-11-15T03:05:05.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.815+0000 7f4ee98db700 1 -- 172.21.15.121:0/195462893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee4150ff0 msgr2=0x7f4ee4151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:05.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.815+0000 7f4ee98db700 1 --2- 172.21.15.121:0/195462893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee4150ff0 0x7f4ee4151470 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f4ecc009760 tx=0x7f4ecc009060).stop 2023-11-15T03:05:05.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.815+0000 7f4ee98db700 1 -- 172.21.15.121:0/195462893 shutdown_connections 2023-11-15T03:05:05.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.815+0000 7f4ee98db700 1 --2- 172.21.15.121:0/195462893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee4150ff0 0x7f4ee4151470 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:05.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.815+0000 7f4ee98db700 1 -- 172.21.15.121:0/195462893 >> 172.21.15.121:0/195462893 conn(0x7f4ee4062190 msgr2=0x7f4ee40625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:05.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.815+0000 7f4ee98db700 1 -- 172.21.15.121:0/195462893 shutdown_connections 2023-11-15T03:05:05.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.815+0000 7f4ee98db700 1 -- 172.21.15.121:0/195462893 wait complete. 2023-11-15T03:05:05.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.819+0000 7f4ee98db700 1 Processor -- start 2023-11-15T03:05:05.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.819+0000 7f4ee98db700 1 -- start start 2023-11-15T03:05:05.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.819+0000 7f4ee98db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee40c6640 0x7f4ee414b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:05.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.819+0000 7f4ee98db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee414af80 0x7f4ee4160d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:05.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.819+0000 7f4ee98db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee4150ff0 0x7f4ee40a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:05.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.819+0000 7f4ee98db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee414bea0 con 0x7f4ee40c6640 2023-11-15T03:05:05.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.819+0000 7f4ee98db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee40ac7a0 con 0x7f4ee4150ff0 2023-11-15T03:05:05.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.819+0000 7f4ee98db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee40ab740 con 0x7f4ee414af80 2023-11-15T03:05:05.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.819+0000 7f4ee2ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee40c6640 0x7f4ee414b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:05.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.823+0000 7f4edbfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee414af80 0x7f4ee4160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:05.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.823+0000 7f4ee2ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee40c6640 0x7f4ee414b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33942/0 (socket says 172.21.15.121:33942) 2023-11-15T03:05:05.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.823+0000 7f4ee2ffd700 1 -- 172.21.15.121:0/3144800167 learned_addr learned my addr 172.21.15.121:0/3144800167 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:05.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.823+0000 7f4ee37fe700 1 --2- 172.21.15.121:0/3144800167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee4150ff0 0x7f4ee40a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:05.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.823+0000 7f4ee2ffd700 1 -- 172.21.15.121:0/3144800167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee414af80 msgr2=0x7f4ee4160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:05.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.823+0000 7f4ee2ffd700 1 --2- 172.21.15.121:0/3144800167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee414af80 0x7f4ee4160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:05.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.823+0000 7f4ee2ffd700 1 -- 172.21.15.121:0/3144800167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee4150ff0 msgr2=0x7f4ee40a99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:05.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.823+0000 7f4ee2ffd700 1 --2- 172.21.15.121:0/3144800167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee4150ff0 0x7f4ee40a99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:05.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.823+0000 7f4ee2ffd700 1 -- 172.21.15.121:0/3144800167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc0092a0 con 0x7f4ee40c6640 2023-11-15T03:05:05.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.823+0000 7f4edbfff700 1 --2- 172.21.15.121:0/3144800167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee414af80 0x7f4ee4160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:05:05.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.823+0000 7f4ee2ffd700 1 --2- 172.21.15.121:0/3144800167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee40c6640 0x7f4ee414b830 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7f4ed400c4a0 tx=0x7f4ed4007af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:05.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.823+0000 7f4edaffd700 1 -- 172.21.15.121:0/3144800167 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed400ce50 con 0x7f4ee40c6640 2023-11-15T03:05:05.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.823+0000 7f4edaffd700 1 -- 172.21.15.121:0/3144800167 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed40072e0 con 0x7f4ee40c6640 2023-11-15T03:05:05.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.823+0000 7f4ee98db700 1 -- 172.21.15.121:0/3144800167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee41611f0 con 0x7f4ee40c6640 2023-11-15T03:05:05.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.823+0000 7f4edaffd700 1 -- 172.21.15.121:0/3144800167 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed40057a0 con 0x7f4ee40c6640 2023-11-15T03:05:05.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.823+0000 7f4ee98db700 1 -- 172.21.15.121:0/3144800167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee40abcc0 con 0x7f4ee40c6640 2023-11-15T03:05:05.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.823+0000 7f4ee98db700 1 -- 172.21.15.121:0/3144800167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee406c150 con 0x7f4ee40c6640 2023-11-15T03:05:05.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.827+0000 7f4edaffd700 1 -- 172.21.15.121:0/3144800167 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4ed4005bd0 con 0x7f4ee40c6640 2023-11-15T03:05:05.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.827+0000 7f4edaffd700 1 --2- 172.21.15.121:0/3144800167 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4ec405a510 0x7f4ec405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:05.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.827+0000 7f4edaffd700 1 -- 172.21.15.121:0/3144800167 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7040+0+0 (secure 0 0 0) 0x7f4ed4074140 con 0x7f4ee40c6640 2023-11-15T03:05:05.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.827+0000 7f4edbfff700 1 --2- 172.21.15.121:0/3144800167 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4ec405a510 0x7f4ec405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:05.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.827+0000 7f4edbfff700 1 --2- 172.21.15.121:0/3144800167 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4ec405a510 0x7f4ec405c9d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f4ee4150c50 tx=0x7f4ecc017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:05.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:05.831+0000 7f4edaffd700 1 -- 172.21.15.121:0/3144800167 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4ed4047530 con 0x7f4ee40c6640 2023-11-15T03:05:06.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.027+0000 7f4ee98db700 1 -- 172.21.15.121:0/3144800167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ee4152000 con 0x7f4ee40c6640 2023-11-15T03:05:06.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.027+0000 7f4edaffd700 1 -- 172.21.15.121:0/3144800167 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f4ee4152000 con 0x7f4ee40c6640 2023-11-15T03:05:06.036 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:06.036 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":135,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8130,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2402059232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2402059232},{"type":"v1","addr":"172.21.15.121:6836","nonce":2402059232}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":135}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:01.222317+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:05:06.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.031+0000 7f4ee98db700 1 -- 172.21.15.121:0/3144800167 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4ec405a510 msgr2=0x7f4ec405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:06.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.031+0000 7f4ee98db700 1 --2- 172.21.15.121:0/3144800167 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4ec405a510 0x7f4ec405c9d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f4ee4150c50 tx=0x7f4ecc017000).stop 2023-11-15T03:05:06.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.031+0000 7f4ee98db700 1 -- 172.21.15.121:0/3144800167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee40c6640 msgr2=0x7f4ee414b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:06.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.031+0000 7f4ee98db700 1 --2- 172.21.15.121:0/3144800167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee40c6640 0x7f4ee414b830 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7f4ed400c4a0 tx=0x7f4ed4007af0).stop 2023-11-15T03:05:06.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.031+0000 7f4ee98db700 1 -- 172.21.15.121:0/3144800167 shutdown_connections 2023-11-15T03:05:06.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.031+0000 7f4ee98db700 1 --2- 172.21.15.121:0/3144800167 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4ec405a510 0x7f4ec405c9d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.031+0000 7f4ee98db700 1 --2- 172.21.15.121:0/3144800167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee4150ff0 0x7f4ee40a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.031+0000 7f4ee98db700 1 --2- 172.21.15.121:0/3144800167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee414af80 0x7f4ee4160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.031+0000 7f4ee98db700 1 --2- 172.21.15.121:0/3144800167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee40c6640 0x7f4ee414b830 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.031+0000 7f4ee98db700 1 -- 172.21.15.121:0/3144800167 >> 172.21.15.121:0/3144800167 conn(0x7f4ee4062190 msgr2=0x7f4ee4062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:06.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.031+0000 7f4ee98db700 1 -- 172.21.15.121:0/3144800167 shutdown_connections 2023-11-15T03:05:06.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.031+0000 7f4ee98db700 1 -- 172.21.15.121:0/3144800167 wait complete. 2023-11-15T03:05:06.042 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 135 2023-11-15T03:05:06.060 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:05:06.060 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:05:06.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.211+0000 7f02ddc79700 1 Processor -- start 2023-11-15T03:05:06.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.211+0000 7f02ddc79700 1 -- start start 2023-11-15T03:05:06.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.211+0000 7f02ddc79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02d80a9830 0x7f02d80a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:06.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.211+0000 7f02ddc79700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f02d80c30e0 0x7f02d80c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:06.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.211+0000 7f02ddc79700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d80657f0 con 0x7f02d8151870 2023-11-15T03:05:06.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.211+0000 7f02ddc79700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d8065ad0 con 0x7f02d80c30e0 2023-11-15T03:05:06.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.211+0000 7f02ddc79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d80c3a60 con 0x7f02d80a9830 2023-11-15T03:05:06.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02dd478700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f02d80c30e0 0x7f02d80c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:06.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02dd478700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f02d80c30e0 0x7f02d80c3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33958/0 (socket says 172.21.15.121:33958) 2023-11-15T03:05:06.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02dd478700 1 -- 172.21.15.121:0/3570311974 learned_addr learned my addr 172.21.15.121:0/3570311974 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:06.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02d7fff700 1 --2- 172.21.15.121:0/3570311974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02d80a9830 0x7f02d80a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:06.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02dd478700 1 -- 172.21.15.121:0/3570311974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02d80a9830 msgr2=0x7f02d80a9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:06.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02dd478700 1 --2- 172.21.15.121:0/3570311974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02d80a9830 0x7f02d80a9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02dd478700 1 -- 172.21.15.121:0/3570311974 >> v1:172.21.15.190:6789/0 conn(0x7f02d8151870 legacy=0x7f02d8151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:06.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02dd478700 1 -- 172.21.15.121:0/3570311974 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d80aa260 con 0x7f02d80c30e0 2023-11-15T03:05:06.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02d7fff700 1 --2- 172.21.15.121:0/3570311974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02d80a9830 0x7f02d80a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:05:06.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02dd478700 1 --2- 172.21.15.121:0/3570311974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02d80c30e0 0x7f02d80c3570 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7f02d0009f20 tx=0x7f02d0009840).ready entity=mon.0 client_cookie=22d2ce25f06d6e5b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:06.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02d77fe700 1 -- 172.21.15.121:0/3570311974 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d000e070 con 0x7f02d80c30e0 2023-11-15T03:05:06.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02d77fe700 1 -- 172.21.15.121:0/3570311974 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d000ba40 con 0x7f02d80c30e0 2023-11-15T03:05:06.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02d77fe700 1 -- 172.21.15.121:0/3570311974 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d000bdc0 con 0x7f02d80c30e0 2023-11-15T03:05:06.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02ddc79700 1 -- 172.21.15.121:0/3570311974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02d80c30e0 msgr2=0x7f02d80c3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:06.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02ddc79700 1 --2- 172.21.15.121:0/3570311974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02d80c30e0 0x7f02d80c3570 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7f02d0009f20 tx=0x7f02d0009840).stop 2023-11-15T03:05:06.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02ddc79700 1 -- 172.21.15.121:0/3570311974 shutdown_connections 2023-11-15T03:05:06.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02ddc79700 1 --2- 172.21.15.121:0/3570311974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02d80c30e0 0x7f02d80c3570 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02ddc79700 1 --2- 172.21.15.121:0/3570311974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02d80a9830 0x7f02d80a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02ddc79700 1 -- 172.21.15.121:0/3570311974 >> 172.21.15.121:0/3570311974 conn(0x7f02d8062190 msgr2=0x7f02d8062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:06.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02ddc79700 1 -- 172.21.15.121:0/3570311974 shutdown_connections 2023-11-15T03:05:06.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.215+0000 7f02ddc79700 1 -- 172.21.15.121:0/3570311974 wait complete. 2023-11-15T03:05:06.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.219+0000 7f02ddc79700 1 Processor -- start 2023-11-15T03:05:06.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.219+0000 7f02ddc79700 1 -- start start 2023-11-15T03:05:06.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02ddc79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02d80a9830 0x7f02d80c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:06.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02ddc79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02d80c30e0 0x7f02d80c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:06.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02ddc79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02d8151870 0x7f02d814a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:06.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02ddc79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d814bef0 con 0x7f02d80a9830 2023-11-15T03:05:06.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02ddc79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d80c69d0 con 0x7f02d80c30e0 2023-11-15T03:05:06.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02ddc79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d80c6cb0 con 0x7f02d8151870 2023-11-15T03:05:06.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02dcc77700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02d80a9830 0x7f02d80c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:06.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02dcc77700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02d80a9830 0x7f02d80c6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33970/0 (socket says 172.21.15.121:33970) 2023-11-15T03:05:06.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02dcc77700 1 -- 172.21.15.121:0/2923519697 learned_addr learned my addr 172.21.15.121:0/2923519697 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:06.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02dd478700 1 --2- 172.21.15.121:0/2923519697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02d8151870 0x7f02d814a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:06.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02d7fff700 1 --2- 172.21.15.121:0/2923519697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02d80c30e0 0x7f02d80c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:06.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02dcc77700 1 -- 172.21.15.121:0/2923519697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02d8151870 msgr2=0x7f02d814a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:06.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02dcc77700 1 --2- 172.21.15.121:0/2923519697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02d8151870 0x7f02d814a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02dcc77700 1 -- 172.21.15.121:0/2923519697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02d80c30e0 msgr2=0x7f02d80c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:06.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02dcc77700 1 --2- 172.21.15.121:0/2923519697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02d80c30e0 0x7f02d80c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02dcc77700 1 -- 172.21.15.121:0/2923519697 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d0009ae0 con 0x7f02d80a9830 2023-11-15T03:05:06.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02dcc77700 1 --2- 172.21.15.121:0/2923519697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02d80a9830 0x7f02d80c6380 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7f02cc00b460 tx=0x7f02cc006f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:06.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02d4ff9700 1 -- 172.21.15.121:0/2923519697 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc012410 con 0x7f02d80a9830 2023-11-15T03:05:06.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02d4ff9700 1 -- 172.21.15.121:0/2923519697 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02cc0125b0 con 0x7f02d80a9830 2023-11-15T03:05:06.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02d4ff9700 1 -- 172.21.15.121:0/2923519697 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc012970 con 0x7f02d80a9830 2023-11-15T03:05:06.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02ddc79700 1 -- 172.21.15.121:0/2923519697 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d80c6f90 con 0x7f02d80a9830 2023-11-15T03:05:06.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.223+0000 7f02ddc79700 1 -- 172.21.15.121:0/2923519697 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d80c6030 con 0x7f02d80a9830 2023-11-15T03:05:06.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.227+0000 7f02ddc79700 1 -- 172.21.15.121:0/2923519697 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d814c030 con 0x7f02d80a9830 2023-11-15T03:05:06.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.227+0000 7f02d4ff9700 1 -- 172.21.15.121:0/2923519697 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f02cc012d20 con 0x7f02d80a9830 2023-11-15T03:05:06.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.227+0000 7f02d4ff9700 1 --2- 172.21.15.121:0/2923519697 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f02bc05a4e0 0x7f02bc05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:06.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.227+0000 7f02d4ff9700 1 -- 172.21.15.121:0/2923519697 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7040+0+0 (secure 0 0 0) 0x7f02cc074870 con 0x7f02d80a9830 2023-11-15T03:05:06.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.231+0000 7f02d7fff700 1 --2- 172.21.15.121:0/2923519697 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f02bc05a4e0 0x7f02bc05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:06.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.231+0000 7f02d7fff700 1 --2- 172.21.15.121:0/2923519697 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f02bc05a4e0 0x7f02bc05c9a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f02d80a8de0 tx=0x7f02c4008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:06.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.235+0000 7f02d4ff9700 1 -- 172.21.15.121:0/2923519697 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f02cc047cc0 con 0x7f02d80a9830 2023-11-15T03:05:06.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.427+0000 7f02ddc79700 1 -- 172.21.15.121:0/2923519697 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02d806bcf0 con 0x7f02d80a9830 2023-11-15T03:05:06.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.427+0000 7f02d4ff9700 1 -- 172.21.15.121:0/2923519697 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f02cc04b2e0 con 0x7f02d80a9830 2023-11-15T03:05:06.436 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:06.436 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":135,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8130,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2402059232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2402059232},{"type":"v1","addr":"172.21.15.121:6836","nonce":2402059232}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":135}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:01.222317+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:05:06.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.431+0000 7f02ddc79700 1 -- 172.21.15.121:0/2923519697 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f02bc05a4e0 msgr2=0x7f02bc05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:06.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.431+0000 7f02ddc79700 1 --2- 172.21.15.121:0/2923519697 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f02bc05a4e0 0x7f02bc05c9a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f02d80a8de0 tx=0x7f02c4008040).stop 2023-11-15T03:05:06.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.431+0000 7f02ddc79700 1 -- 172.21.15.121:0/2923519697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02d80a9830 msgr2=0x7f02d80c6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:06.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.431+0000 7f02ddc79700 1 --2- 172.21.15.121:0/2923519697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02d80a9830 0x7f02d80c6380 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7f02cc00b460 tx=0x7f02cc006f20).stop 2023-11-15T03:05:06.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.431+0000 7f02ddc79700 1 -- 172.21.15.121:0/2923519697 shutdown_connections 2023-11-15T03:05:06.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.431+0000 7f02ddc79700 1 --2- 172.21.15.121:0/2923519697 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f02bc05a4e0 0x7f02bc05c9a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.431+0000 7f02ddc79700 1 --2- 172.21.15.121:0/2923519697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02d8151870 0x7f02d814a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.431+0000 7f02ddc79700 1 --2- 172.21.15.121:0/2923519697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02d80c30e0 0x7f02d80c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.431+0000 7f02ddc79700 1 --2- 172.21.15.121:0/2923519697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02d80a9830 0x7f02d80c6380 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.431+0000 7f02ddc79700 1 -- 172.21.15.121:0/2923519697 >> 172.21.15.121:0/2923519697 conn(0x7f02d8062190 msgr2=0x7f02d80c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:06.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.431+0000 7f02ddc79700 1 -- 172.21.15.121:0/2923519697 shutdown_connections 2023-11-15T03:05:06.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.431+0000 7f02ddc79700 1 -- 172.21.15.121:0/2923519697 wait complete. 2023-11-15T03:05:06.446 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 135 2023-11-15T03:05:06.457 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:05:06.458 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:05:06.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.603+0000 7f2b016b9700 1 Processor -- start 2023-11-15T03:05:06.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.607+0000 7f2b016b9700 1 -- start start 2023-11-15T03:05:06.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.607+0000 7f2b016b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2afc151870 0x7f2afc151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:06.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.607+0000 7f2b016b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2afc0657f0 con 0x7f2afc0c30e0 2023-11-15T03:05:06.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.607+0000 7f2b016b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2afc065ad0 con 0x7f2afc152140 2023-11-15T03:05:06.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.607+0000 7f2b016b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc0aa260 con 0x7f2afc151870 2023-11-15T03:05:06.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.607+0000 7f2b00eb8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f2afc0c30e0 0x7f2afc065f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49620/0 (socket says 172.21.15.121:49620) 2023-11-15T03:05:06.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.607+0000 7f2b00eb8700 1 -- 172.21.15.121:0/3011000341 learned_addr learned my addr 172.21.15.121:0/3011000341 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:06.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.607+0000 7f2afbfff700 1 --2- 172.21.15.121:0/3011000341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2afc151870 0x7f2afc151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:06.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.607+0000 7f2afb7fe700 1 -- 172.21.15.121:0/3011000341 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2805379600 0 0) 0x7f2afc0657f0 con 0x7f2afc0c30e0 2023-11-15T03:05:06.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.611+0000 7f2afb7fe700 1 -- 172.21.15.121:0/3011000341 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ae80033f0 con 0x7f2afc0c30e0 2023-11-15T03:05:06.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.611+0000 7f2afb7fe700 1 -- 172.21.15.121:0/3011000341 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2433707525 0 0) 0x7f2afc065ad0 con 0x7f2afc152140 2023-11-15T03:05:06.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.611+0000 7f2afbfff700 1 -- 172.21.15.121:0/3011000341 >> v1:172.21.15.190:6789/0 conn(0x7f2afc152140 legacy=0x7f2afc0a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:06.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.611+0000 7f2afbfff700 1 -- 172.21.15.121:0/3011000341 >> v1:172.21.15.121:6789/0 conn(0x7f2afc0c30e0 legacy=0x7f2afc065f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:06.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.611+0000 7f2afbfff700 1 -- 172.21.15.121:0/3011000341 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc0aa540 con 0x7f2afc151870 2023-11-15T03:05:06.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.611+0000 7f2afbfff700 1 --2- 172.21.15.121:0/3011000341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2afc151870 0x7f2afc151c50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f2aec009790 tx=0x7f2aec009060).ready entity=mon.2 client_cookie=853a74bb6ca404e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:06.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.611+0000 7f2afb7fe700 1 -- 172.21.15.121:0/3011000341 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec00f510 con 0x7f2afc151870 2023-11-15T03:05:06.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.611+0000 7f2afb7fe700 1 -- 172.21.15.121:0/3011000341 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec00f670 con 0x7f2afc151870 2023-11-15T03:05:06.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.611+0000 7f2afb7fe700 1 -- 172.21.15.121:0/3011000341 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec00f9f0 con 0x7f2afc151870 2023-11-15T03:05:06.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.611+0000 7f2b016b9700 1 -- 172.21.15.121:0/3011000341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2afc151870 msgr2=0x7f2afc151c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:06.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.611+0000 7f2b016b9700 1 --2- 172.21.15.121:0/3011000341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2afc151870 0x7f2afc151c50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f2aec009790 tx=0x7f2aec009060).stop 2023-11-15T03:05:06.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.611+0000 7f2b016b9700 1 -- 172.21.15.121:0/3011000341 shutdown_connections 2023-11-15T03:05:06.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.611+0000 7f2b016b9700 1 --2- 172.21.15.121:0/3011000341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2afc151870 0x7f2afc151c50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.611+0000 7f2b016b9700 1 -- 172.21.15.121:0/3011000341 >> 172.21.15.121:0/3011000341 conn(0x7f2afc062190 msgr2=0x7f2afc062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:06.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.611+0000 7f2b016b9700 1 -- 172.21.15.121:0/3011000341 shutdown_connections 2023-11-15T03:05:06.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.611+0000 7f2b016b9700 1 -- 172.21.15.121:0/3011000341 wait complete. 2023-11-15T03:05:06.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2b016b9700 1 Processor -- start 2023-11-15T03:05:06.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2b016b9700 1 -- start start 2023-11-15T03:05:06.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2b016b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2afc0c30e0 0x7f2afc147190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:06.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2b016b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2afc151870 0x7f2afc162a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:06.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2b016b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2afc152140 0x7f2afc148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:06.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2b016b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc0c44e0 con 0x7f2afc151870 2023-11-15T03:05:06.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2b016b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc0c5060 con 0x7f2afc152140 2023-11-15T03:05:06.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2b016b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc147680 con 0x7f2afc0c30e0 2023-11-15T03:05:06.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2af37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2afc151870 0x7f2afc162a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:06.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2afbfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2afc0c30e0 0x7f2afc147190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:06.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2af37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2afc151870 0x7f2afc162a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33984/0 (socket says 172.21.15.121:33984) 2023-11-15T03:05:06.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2af37fe700 1 -- 172.21.15.121:0/1076796965 learned_addr learned my addr 172.21.15.121:0/1076796965 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:06.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2b00eb8700 1 --2- 172.21.15.121:0/1076796965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2afc152140 0x7f2afc148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:06.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2afbfff700 1 -- 172.21.15.121:0/1076796965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2afc152140 msgr2=0x7f2afc148730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:06.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2afbfff700 1 --2- 172.21.15.121:0/1076796965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2afc152140 0x7f2afc148730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2afbfff700 1 -- 172.21.15.121:0/1076796965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2afc151870 msgr2=0x7f2afc162a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:06.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2afbfff700 1 --2- 172.21.15.121:0/1076796965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2afc151870 0x7f2afc162a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2afbfff700 1 -- 172.21.15.121:0/1076796965 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec0092a0 con 0x7f2afc0c30e0 2023-11-15T03:05:06.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2af37fe700 1 --2- 172.21.15.121:0/1076796965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2afc151870 0x7f2afc162a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:05:06.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.615+0000 7f2afbfff700 1 --2- 172.21.15.121:0/1076796965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2afc0c30e0 0x7f2afc147190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f2aec014000 tx=0x7f2aec017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:06.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.619+0000 7f2af8ff9700 1 -- 172.21.15.121:0/1076796965 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec01a610 con 0x7f2afc0c30e0 2023-11-15T03:05:06.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.619+0000 7f2af8ff9700 1 -- 172.21.15.121:0/1076796965 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec01a790 con 0x7f2afc0c30e0 2023-11-15T03:05:06.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.619+0000 7f2af8ff9700 1 -- 172.21.15.121:0/1076796965 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec01ab50 con 0x7f2afc0c30e0 2023-11-15T03:05:06.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.619+0000 7f2b016b9700 1 -- 172.21.15.121:0/1076796965 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2afc0c4730 con 0x7f2afc0c30e0 2023-11-15T03:05:06.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.619+0000 7f2b016b9700 1 -- 172.21.15.121:0/1076796965 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2afc0c4cf0 con 0x7f2afc0c30e0 2023-11-15T03:05:06.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.619+0000 7f2b016b9700 1 -- 172.21.15.121:0/1076796965 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2afc06bcf0 con 0x7f2afc0c30e0 2023-11-15T03:05:06.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.619+0000 7f2af8ff9700 1 -- 172.21.15.121:0/1076796965 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2aec020070 con 0x7f2afc0c30e0 2023-11-15T03:05:06.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.623+0000 7f2af8ff9700 1 --2- 172.21.15.121:0/1076796965 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2adc05a530 0x7f2adc05c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:06.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.623+0000 7f2af8ff9700 1 -- 172.21.15.121:0/1076796965 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7040+0+0 (secure 0 0 0) 0x7f2aec075cc0 con 0x7f2afc0c30e0 2023-11-15T03:05:06.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.623+0000 7f2af37fe700 1 --2- 172.21.15.121:0/1076796965 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2adc05a530 0x7f2adc05c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:06.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.623+0000 7f2af37fe700 1 --2- 172.21.15.121:0/1076796965 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2adc05a530 0x7f2adc05c9f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f2afc152110 tx=0x7f2ae4009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:06.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.627+0000 7f2af8ff9700 1 -- 172.21.15.121:0/1076796965 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2aec07c080 con 0x7f2afc0c30e0 2023-11-15T03:05:06.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.819+0000 7f2b016b9700 1 -- 172.21.15.121:0/1076796965 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2afc068a70 con 0x7f2afc0c30e0 2023-11-15T03:05:06.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.819+0000 7f2af8ff9700 1 -- 172.21.15.121:0/1076796965 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f2aec0490f0 con 0x7f2afc0c30e0 2023-11-15T03:05:06.828 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:06.828 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":135,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8130,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2402059232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2402059232},{"type":"v1","addr":"172.21.15.121:6836","nonce":2402059232}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":135}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:01.222317+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:05:06.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.823+0000 7f2b016b9700 1 -- 172.21.15.121:0/1076796965 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2adc05a530 msgr2=0x7f2adc05c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:06.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.823+0000 7f2b016b9700 1 --2- 172.21.15.121:0/1076796965 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2adc05a530 0x7f2adc05c9f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f2afc152110 tx=0x7f2ae4009040).stop 2023-11-15T03:05:06.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.823+0000 7f2b016b9700 1 -- 172.21.15.121:0/1076796965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2afc0c30e0 msgr2=0x7f2afc147190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:06.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.823+0000 7f2b016b9700 1 --2- 172.21.15.121:0/1076796965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2afc0c30e0 0x7f2afc147190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f2aec014000 tx=0x7f2aec017000).stop 2023-11-15T03:05:06.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.823+0000 7f2b016b9700 1 -- 172.21.15.121:0/1076796965 shutdown_connections 2023-11-15T03:05:06.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.823+0000 7f2b016b9700 1 --2- 172.21.15.121:0/1076796965 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2adc05a530 0x7f2adc05c9f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.823+0000 7f2b016b9700 1 --2- 172.21.15.121:0/1076796965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2afc152140 0x7f2afc148730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.823+0000 7f2b016b9700 1 --2- 172.21.15.121:0/1076796965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2afc151870 0x7f2afc162a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.823+0000 7f2b016b9700 1 --2- 172.21.15.121:0/1076796965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2afc0c30e0 0x7f2afc147190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:06.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.823+0000 7f2b016b9700 1 -- 172.21.15.121:0/1076796965 >> 172.21.15.121:0/1076796965 conn(0x7f2afc062190 msgr2=0x7f2afc154b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:06.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.823+0000 7f2b016b9700 1 -- 172.21.15.121:0/1076796965 shutdown_connections 2023-11-15T03:05:06.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:06.823+0000 7f2b016b9700 1 -- 172.21.15.121:0/1076796965 wait complete. 2023-11-15T03:05:06.834 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 135 2023-11-15T03:05:06.852 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds_snaptable' led to DAMAGED state 2023-11-15T03:05:06.852 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_inotable' 2023-11-15T03:05:06.853 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:05:06.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:05:06.853 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:05:06.871 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:05:06.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:05:06.873 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:05:07.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.071+0000 7fd15968f700 1 Processor -- start 2023-11-15T03:05:07.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.075+0000 7fd15968f700 1 -- start start 2023-11-15T03:05:07.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.075+0000 7fd15968f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd1540ab0d0 0x7fd1540ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:07.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.075+0000 7fd15968f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd1540add40 0x7fd1540ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:07.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.075+0000 7fd15968f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd154066c90 con 0x7fd1540ad1b0 2023-11-15T03:05:07.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.075+0000 7fd15968f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd154065e20 con 0x7fd1540ab0d0 2023-11-15T03:05:07.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.075+0000 7fd15968f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd154066080 con 0x7fd1540add40 2023-11-15T03:05:07.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.075+0000 7fd14b7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fd1540ad1b0 0x7fd154065710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49622/0 (socket says 172.21.15.121:49622) 2023-11-15T03:05:07.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.075+0000 7fd14b7fe700 1 -- 172.21.15.121:0/3687647293 learned_addr learned my addr 172.21.15.121:0/3687647293 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:07.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.075+0000 7fd153fff700 1 --2- 172.21.15.121:0/3687647293 >> v2:172.21.15.121:3300/0 conn(0x7fd1540ab0d0 0x7fd1540ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:07.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.075+0000 7fd158e8e700 1 --2- 172.21.15.121:0/3687647293 >> v2:172.21.15.190:3300/0 conn(0x7fd1540add40 0x7fd1540ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:07.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.075+0000 7fd1537fe700 1 -- 172.21.15.121:0/3687647293 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3540093922 0 0) 0x7fd154066c90 con 0x7fd1540ad1b0 2023-11-15T03:05:07.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.075+0000 7fd1537fe700 1 -- 172.21.15.121:0/3687647293 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1400033f0 con 0x7fd1540ad1b0 2023-11-15T03:05:07.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.075+0000 7fd153fff700 1 -- 172.21.15.121:0/3687647293 >> v2:172.21.15.190:3300/0 conn(0x7fd1540add40 msgr2=0x7fd1540ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:07.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.075+0000 7fd153fff700 1 --2- 172.21.15.121:0/3687647293 >> v2:172.21.15.190:3300/0 conn(0x7fd1540add40 0x7fd1540ae1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:07.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.075+0000 7fd153fff700 1 -- 172.21.15.121:0/3687647293 >> v1:172.21.15.121:6789/0 conn(0x7fd1540ad1b0 legacy=0x7fd154065710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:07.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.075+0000 7fd153fff700 1 -- 172.21.15.121:0/3687647293 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd154066360 con 0x7fd1540ab0d0 2023-11-15T03:05:07.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.079+0000 7fd153fff700 1 --2- 172.21.15.121:0/3687647293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1540ab0d0 0x7fd1540ab4b0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7fd154068ca0 tx=0x7fd144006f40).ready entity=mon.0 client_cookie=68386028f9827a00 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:07.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.079+0000 7fd1537fe700 1 -- 172.21.15.121:0/3687647293 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd144005880 con 0x7fd1540ab0d0 2023-11-15T03:05:07.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.079+0000 7fd1537fe700 1 -- 172.21.15.121:0/3687647293 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1440059e0 con 0x7fd1540ab0d0 2023-11-15T03:05:07.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.079+0000 7fd1537fe700 1 -- 172.21.15.121:0/3687647293 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd144005d60 con 0x7fd1540ab0d0 2023-11-15T03:05:07.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.079+0000 7fd15968f700 1 -- 172.21.15.121:0/3687647293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1540ab0d0 msgr2=0x7fd1540ab4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:07.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.079+0000 7fd15968f700 1 --2- 172.21.15.121:0/3687647293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1540ab0d0 0x7fd1540ab4b0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7fd154068ca0 tx=0x7fd144006f40).stop 2023-11-15T03:05:07.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.079+0000 7fd15968f700 1 -- 172.21.15.121:0/3687647293 shutdown_connections 2023-11-15T03:05:07.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.079+0000 7fd15968f700 1 --2- 172.21.15.121:0/3687647293 >> v2:172.21.15.190:3300/0 conn(0x7fd1540add40 0x7fd1540ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:07.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.079+0000 7fd15968f700 1 --2- 172.21.15.121:0/3687647293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1540ab0d0 0x7fd1540ab4b0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:07.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.079+0000 7fd15968f700 1 -- 172.21.15.121:0/3687647293 >> 172.21.15.121:0/3687647293 conn(0x7fd154067a60 msgr2=0x7fd154067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:07.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.079+0000 7fd15968f700 1 -- 172.21.15.121:0/3687647293 shutdown_connections 2023-11-15T03:05:07.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.079+0000 7fd15968f700 1 -- 172.21.15.121:0/3687647293 wait complete. 2023-11-15T03:05:07.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.079+0000 7fd15968f700 1 Processor -- start 2023-11-15T03:05:07.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.083+0000 7fd15968f700 1 -- start start 2023-11-15T03:05:07.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.083+0000 7fd15968f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1540ab0d0 0x7fd154163e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:07.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.083+0000 7fd15968f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1540ad1b0 0x7fd1540a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:07.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.083+0000 7fd15968f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1540add40 0x7fd1540c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:07.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.083+0000 7fd15968f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1540a3db0 con 0x7fd1540ad1b0 2023-11-15T03:05:07.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.083+0000 7fd15968f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1540a4bd0 con 0x7fd1540ab0d0 2023-11-15T03:05:07.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.083+0000 7fd15968f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1540a9630 con 0x7fd1540add40 2023-11-15T03:05:07.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.083+0000 7fd14b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1540ad1b0 0x7fd1540a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:07.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.083+0000 7fd14b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1540ad1b0 0x7fd1540a38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33996/0 (socket says 172.21.15.121:33996) 2023-11-15T03:05:07.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.083+0000 7fd14b7fe700 1 -- 172.21.15.121:0/4083747745 learned_addr learned my addr 172.21.15.121:0/4083747745 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.083+0000 7fd153fff700 1 --2- 172.21.15.121:0/4083747745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1540ab0d0 0x7fd154163e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.083+0000 7fd158e8e700 1 --2- 172.21.15.121:0/4083747745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1540add40 0x7fd1540c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.083+0000 7fd14b7fe700 1 -- 172.21.15.121:0/4083747745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1540add40 msgr2=0x7fd1540c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.083+0000 7fd14b7fe700 1 --2- 172.21.15.121:0/4083747745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1540add40 0x7fd1540c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.083+0000 7fd14b7fe700 1 -- 172.21.15.121:0/4083747745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1540ab0d0 msgr2=0x7fd154163e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:07.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.083+0000 7fd14b7fe700 1 --2- 172.21.15.121:0/4083747745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1540ab0d0 0x7fd154163e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:07.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.083+0000 7fd14b7fe700 1 -- 172.21.15.121:0/4083747745 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd144009250 con 0x7fd1540ad1b0 2023-11-15T03:05:07.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.083+0000 7fd153fff700 1 --2- 172.21.15.121:0/4083747745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1540ab0d0 0x7fd154163e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:07.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.083+0000 7fd14b7fe700 1 --2- 172.21.15.121:0/4083747745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1540ad1b0 0x7fd1540a38c0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7fd13c00c7f0 tx=0x7fd13c001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:07.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.087+0000 7fd150ff9700 1 -- 172.21.15.121:0/4083747745 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c005d30 con 0x7fd1540ad1b0 2023-11-15T03:05:07.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.087+0000 7fd150ff9700 1 -- 172.21.15.121:0/4083747745 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13c005e90 con 0x7fd1540ad1b0 2023-11-15T03:05:07.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.087+0000 7fd15968f700 1 -- 172.21.15.121:0/4083747745 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1540a3fc0 con 0x7fd1540ad1b0 2023-11-15T03:05:07.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.087+0000 7fd150ff9700 1 -- 172.21.15.121:0/4083747745 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c007360 con 0x7fd1540ad1b0 2023-11-15T03:05:07.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.087+0000 7fd15968f700 1 -- 172.21.15.121:0/4083747745 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1540a4580 con 0x7fd1540ad1b0 2023-11-15T03:05:07.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.091+0000 7fd15968f700 1 -- 172.21.15.121:0/4083747745 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1540a4d60 con 0x7fd1540ad1b0 2023-11-15T03:05:07.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.091+0000 7fd150ff9700 1 -- 172.21.15.121:0/4083747745 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd13c007790 con 0x7fd1540ad1b0 2023-11-15T03:05:07.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.095+0000 7fd150ff9700 1 --2- 172.21.15.121:0/4083747745 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd13405a510 0x7fd13405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:07.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.095+0000 7fd150ff9700 1 -- 172.21.15.121:0/4083747745 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7040+0+0 (secure 0 0 0) 0x7fd13c074d40 con 0x7fd1540ad1b0 2023-11-15T03:05:07.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.095+0000 7fd153fff700 1 --2- 172.21.15.121:0/4083747745 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd13405a510 0x7fd13405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:07.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.095+0000 7fd153fff700 1 --2- 172.21.15.121:0/4083747745 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd13405a510 0x7fd13405c9d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7fd144006fa0 tx=0x7fd144015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:07.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.103+0000 7fd150ff9700 1 -- 172.21.15.121:0/4083747745 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd13c0480f0 con 0x7fd1540ad1b0 2023-11-15T03:05:07.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:07.291+0000 7fd15968f700 1 -- 172.21.15.121:0/4083747745 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd15406bcf0 con 0x7fd1540ad1b0 2023-11-15T03:05:08.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.107+0000 7fd150ff9700 1 -- 172.21.15.121:0/4083747745 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v136) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd13c04b710 con 0x7fd1540ad1b0 2023-11-15T03:05:08.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.111+0000 7fd15968f700 1 -- 172.21.15.121:0/4083747745 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd13405a510 msgr2=0x7fd13405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:08.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.111+0000 7fd15968f700 1 --2- 172.21.15.121:0/4083747745 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd13405a510 0x7fd13405c9d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7fd144006fa0 tx=0x7fd144015000).stop 2023-11-15T03:05:08.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.111+0000 7fd15968f700 1 -- 172.21.15.121:0/4083747745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1540ad1b0 msgr2=0x7fd1540a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:08.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.111+0000 7fd15968f700 1 --2- 172.21.15.121:0/4083747745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1540ad1b0 0x7fd1540a38c0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7fd13c00c7f0 tx=0x7fd13c001650).stop 2023-11-15T03:05:08.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.111+0000 7fd15968f700 1 -- 172.21.15.121:0/4083747745 shutdown_connections 2023-11-15T03:05:08.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.111+0000 7fd15968f700 1 --2- 172.21.15.121:0/4083747745 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd13405a510 0x7fd13405c9d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:08.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.111+0000 7fd15968f700 1 --2- 172.21.15.121:0/4083747745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1540add40 0x7fd1540c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:08.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.111+0000 7fd15968f700 1 --2- 172.21.15.121:0/4083747745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1540ad1b0 0x7fd1540a38c0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:08.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.111+0000 7fd15968f700 1 --2- 172.21.15.121:0/4083747745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1540ab0d0 0x7fd154163e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:08.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.111+0000 7fd15968f700 1 -- 172.21.15.121:0/4083747745 >> 172.21.15.121:0/4083747745 conn(0x7fd154067a60 msgr2=0x7fd154154950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:08.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.111+0000 7fd15968f700 1 -- 172.21.15.121:0/4083747745 shutdown_connections 2023-11-15T03:05:08.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.111+0000 7fd15968f700 1 -- 172.21.15.121:0/4083747745 wait complete. 2023-11-15T03:05:08.123 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:05:08.142 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:05:08.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.299+0000 7f0d5d879700 1 Processor -- start 2023-11-15T03:05:08.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.299+0000 7f0d5d879700 1 -- start start 2023-11-15T03:05:08.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d5d879700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0d580c5a00 0x7f0d58158260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:08.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d5d879700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0d580c7a90 0x7f0d58154b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:08.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d5d879700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d580c6170 con 0x7f0d580a9f80 2023-11-15T03:05:08.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d5d879700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d58154000 con 0x7f0d580c7a90 2023-11-15T03:05:08.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d5d879700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d58157dd0 con 0x7f0d580c5a00 2023-11-15T03:05:08.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d5c877700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f0d580a9f80 0x7f0d580c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49628/0 (socket says 172.21.15.121:49628) 2023-11-15T03:05:08.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d5c877700 1 -- 172.21.15.121:0/1850180692 learned_addr learned my addr 172.21.15.121:0/1850180692 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:08.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d57fff700 1 --2- 172.21.15.121:0/1850180692 >> v2:172.21.15.190:3300/0 conn(0x7f0d580c5a00 0x7f0d58158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:08.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d5d078700 1 --2- 172.21.15.121:0/1850180692 >> v2:172.21.15.121:3300/0 conn(0x7f0d580c7a90 0x7f0d58154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:08.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d577fe700 1 -- 172.21.15.121:0/1850180692 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1721518357 0 0) 0x7f0d580c6170 con 0x7f0d580a9f80 2023-11-15T03:05:08.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d577fe700 1 -- 172.21.15.121:0/1850180692 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d480033f0 con 0x7f0d580a9f80 2023-11-15T03:05:08.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d5d078700 1 -- 172.21.15.121:0/1850180692 >> v2:172.21.15.190:3300/0 conn(0x7f0d580c5a00 msgr2=0x7f0d58158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:08.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d5d078700 1 --2- 172.21.15.121:0/1850180692 >> v2:172.21.15.190:3300/0 conn(0x7f0d580c5a00 0x7f0d58158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:08.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d577fe700 1 -- 172.21.15.121:0/1850180692 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 243233073 0 0) 0x7f0d480033f0 con 0x7f0d580a9f80 2023-11-15T03:05:08.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d5d078700 1 -- 172.21.15.121:0/1850180692 >> v1:172.21.15.121:6789/0 conn(0x7f0d580a9f80 legacy=0x7f0d580c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:08.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d5d078700 1 -- 172.21.15.121:0/1850180692 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d58155080 con 0x7f0d580c7a90 2023-11-15T03:05:08.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d57fff700 1 --2- 172.21.15.121:0/1850180692 >> v2:172.21.15.190:3300/0 conn(0x7f0d580c5a00 0x7f0d58158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:05:08.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d5d078700 1 --2- 172.21.15.121:0/1850180692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d580c7a90 0x7f0d58154b90 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f0d580aa860 tx=0x7f0d50009780).ready entity=mon.0 client_cookie=f769ed40390a6611 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:08.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d577fe700 1 -- 172.21.15.121:0/1850180692 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5000e070 con 0x7f0d580c7a90 2023-11-15T03:05:08.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d577fe700 1 -- 172.21.15.121:0/1850180692 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5000bab0 con 0x7f0d580c7a90 2023-11-15T03:05:08.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.303+0000 7f0d577fe700 1 -- 172.21.15.121:0/1850180692 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5000be30 con 0x7f0d580c7a90 2023-11-15T03:05:08.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.307+0000 7f0d5d879700 1 -- 172.21.15.121:0/1850180692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d580c7a90 msgr2=0x7f0d58154b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:08.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.307+0000 7f0d5d879700 1 --2- 172.21.15.121:0/1850180692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d580c7a90 0x7f0d58154b90 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f0d580aa860 tx=0x7f0d50009780).stop 2023-11-15T03:05:08.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.307+0000 7f0d5d879700 1 -- 172.21.15.121:0/1850180692 shutdown_connections 2023-11-15T03:05:08.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.307+0000 7f0d5d879700 1 --2- 172.21.15.121:0/1850180692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d580c7a90 0x7f0d58154b90 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:08.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.307+0000 7f0d5d879700 1 --2- 172.21.15.121:0/1850180692 >> v2:172.21.15.190:3300/0 conn(0x7f0d580c5a00 0x7f0d58158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:08.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.307+0000 7f0d5d879700 1 -- 172.21.15.121:0/1850180692 >> 172.21.15.121:0/1850180692 conn(0x7f0d58067a60 msgr2=0x7f0d58067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:08.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.307+0000 7f0d5d879700 1 -- 172.21.15.121:0/1850180692 shutdown_connections 2023-11-15T03:05:08.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.307+0000 7f0d5d879700 1 -- 172.21.15.121:0/1850180692 wait complete. 2023-11-15T03:05:08.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.307+0000 7f0d5d879700 1 Processor -- start 2023-11-15T03:05:08.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.307+0000 7f0d5d879700 1 -- start start 2023-11-15T03:05:08.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d5d879700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d580a9f80 0x7f0d58168320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:08.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d5d879700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d580c7a90 0x7f0d5814ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:08.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d5d879700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d58164280 0x7f0d58164700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:08.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d5d879700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d58164bf0 con 0x7f0d580c7a90 2023-11-15T03:05:08.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d5d879700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d580b0880 con 0x7f0d580a9f80 2023-11-15T03:05:08.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d5d879700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d58164e60 con 0x7f0d58164280 2023-11-15T03:05:08.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d57fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d580c7a90 0x7f0d5814ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:08.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d57fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d580c7a90 0x7f0d5814ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34016/0 (socket says 172.21.15.121:34016) 2023-11-15T03:05:08.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d57fff700 1 -- 172.21.15.121:0/2929291404 learned_addr learned my addr 172.21.15.121:0/2929291404 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:08.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d5d078700 1 --2- 172.21.15.121:0/2929291404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d58164280 0x7f0d58164700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:08.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d5c877700 1 --2- 172.21.15.121:0/2929291404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d580a9f80 0x7f0d58168320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:08.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d57fff700 1 -- 172.21.15.121:0/2929291404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d58164280 msgr2=0x7f0d58164700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:08.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d57fff700 1 --2- 172.21.15.121:0/2929291404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d58164280 0x7f0d58164700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:08.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d57fff700 1 -- 172.21.15.121:0/2929291404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d580a9f80 msgr2=0x7f0d58168320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:08.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d57fff700 1 --2- 172.21.15.121:0/2929291404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d580a9f80 0x7f0d58168320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:08.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d57fff700 1 -- 172.21.15.121:0/2929291404 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d440092a0 con 0x7f0d580c7a90 2023-11-15T03:05:08.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d5c877700 1 --2- 172.21.15.121:0/2929291404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d580a9f80 0x7f0d58168320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:08.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d57fff700 1 --2- 172.21.15.121:0/2929291404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d580c7a90 0x7f0d5814ed80 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f0d4400e9d0 tx=0x7f0d44009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:08.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d54ff9700 1 -- 172.21.15.121:0/2929291404 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4400eec0 con 0x7f0d580c7a90 2023-11-15T03:05:08.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d54ff9700 1 -- 172.21.15.121:0/2929291404 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4400c850 con 0x7f0d580c7a90 2023-11-15T03:05:08.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d5d879700 1 -- 172.21.15.121:0/2929291404 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d50009a20 con 0x7f0d580c7a90 2023-11-15T03:05:08.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.311+0000 7f0d54ff9700 1 -- 172.21.15.121:0/2929291404 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4400cbd0 con 0x7f0d580c7a90 2023-11-15T03:05:08.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.315+0000 7f0d5d879700 1 -- 172.21.15.121:0/2929291404 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d58167840 con 0x7f0d580c7a90 2023-11-15T03:05:08.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.315+0000 7f0d5d879700 1 -- 172.21.15.121:0/2929291404 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d5806bcf0 con 0x7f0d580c7a90 2023-11-15T03:05:08.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.315+0000 7f0d54ff9700 1 -- 172.21.15.121:0/2929291404 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0d4401b400 con 0x7f0d580c7a90 2023-11-15T03:05:08.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.319+0000 7f0d54ff9700 1 --2- 172.21.15.121:0/2929291404 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0d3c05a510 0x7f0d3c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:08.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.319+0000 7f0d54ff9700 1 -- 172.21.15.121:0/2929291404 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7040+0+0 (secure 0 0 0) 0x7f0d440751b0 con 0x7f0d580c7a90 2023-11-15T03:05:08.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.319+0000 7f0d5c877700 1 --2- 172.21.15.121:0/2929291404 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0d3c05a510 0x7f0d3c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:08.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.319+0000 7f0d5c877700 1 --2- 172.21.15.121:0/2929291404 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0d3c05a510 0x7f0d3c05c9d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f0d58167700 tx=0x7f0d4c007180).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:08.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.323+0000 7f0d54ff9700 1 -- 172.21.15.121:0/2929291404 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0d44048500 con 0x7f0d580c7a90 2023-11-15T03:05:08.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:08.547+0000 7f0d5d879700 1 -- 172.21.15.121:0/2929291404 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f0d580687c0 con 0x7f0d580c7a90 2023-11-15T03:05:09.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.115+0000 7f0d54ff9700 1 -- 172.21.15.121:0/2929291404 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v137) v1 ==== 101+0+0 (secure 0 0 0) 0x7f0d44016020 con 0x7f0d580c7a90 2023-11-15T03:05:09.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.119+0000 7f0d5d879700 1 -- 172.21.15.121:0/2929291404 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0d3c05a510 msgr2=0x7f0d3c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.119+0000 7f0d5d879700 1 --2- 172.21.15.121:0/2929291404 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0d3c05a510 0x7f0d3c05c9d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f0d58167700 tx=0x7f0d4c007180).stop 2023-11-15T03:05:09.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.119+0000 7f0d5d879700 1 -- 172.21.15.121:0/2929291404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d580c7a90 msgr2=0x7f0d5814ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.119+0000 7f0d5d879700 1 --2- 172.21.15.121:0/2929291404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d580c7a90 0x7f0d5814ed80 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f0d4400e9d0 tx=0x7f0d44009060).stop 2023-11-15T03:05:09.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.119+0000 7f0d5d879700 1 -- 172.21.15.121:0/2929291404 shutdown_connections 2023-11-15T03:05:09.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.119+0000 7f0d5d879700 1 --2- 172.21.15.121:0/2929291404 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0d3c05a510 0x7f0d3c05c9d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.119+0000 7f0d5d879700 1 --2- 172.21.15.121:0/2929291404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d58164280 0x7f0d58164700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.119+0000 7f0d5d879700 1 --2- 172.21.15.121:0/2929291404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d580c7a90 0x7f0d5814ed80 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.119+0000 7f0d5d879700 1 --2- 172.21.15.121:0/2929291404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d580a9f80 0x7f0d58168320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.119+0000 7f0d5d879700 1 -- 172.21.15.121:0/2929291404 >> 172.21.15.121:0/2929291404 conn(0x7f0d58067a60 msgr2=0x7f0d58067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:09.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.119+0000 7f0d5d879700 1 -- 172.21.15.121:0/2929291404 shutdown_connections 2023-11-15T03:05:09.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.119+0000 7f0d5d879700 1 -- 172.21.15.121:0/2929291404 wait complete. 2023-11-15T03:05:09.129 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:05:09.148 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:05:09.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.195+0000 7fa96bb37d00 1 Processor -- start 2023-11-15T03:05:09.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.195+0000 7fa96bb37d00 1 -- start start 2023-11-15T03:05:09.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.195+0000 7fa96bb37d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e967149a30 0x55e967220280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa96bb37d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e967149e10 0x55e967194d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa96bb37d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e967198b20 0x55e9672331e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa96bb37d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9670f7cc0 con 0x55e967149e10 2023-11-15T03:05:09.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa96bb37d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9670c2220 con 0x55e967198b20 2023-11-15T03:05:09.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa96bb37d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e967178940 con 0x55e967149a30 2023-11-15T03:05:09.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa957fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e967149e10 0x55e967194d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa957fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e967149e10 0x55e967194d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34030/0 (socket says 172.21.15.121:34030) 2023-11-15T03:05:09.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa957fff700 1 -- 172.21.15.121:0/650334965 learned_addr learned my addr 172.21.15.121:0/650334965 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:09.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa9601c4700 1 --2- 172.21.15.121:0/650334965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e967149a30 0x55e967220280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa9609c5700 1 --2- 172.21.15.121:0/650334965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e967198b20 0x55e9672331e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa957fff700 1 -- 172.21.15.121:0/650334965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e967149a30 msgr2=0x55e967220280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:09.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa957fff700 1 --2- 172.21.15.121:0/650334965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e967149a30 0x55e967220280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa957fff700 1 -- 172.21.15.121:0/650334965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e967198b20 msgr2=0x55e9672331e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:09.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa957fff700 1 --2- 172.21.15.121:0/650334965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e967198b20 0x55e9672331e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa957fff700 1 -- 172.21.15.121:0/650334965 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa948005cf0 con 0x55e967149e10 2023-11-15T03:05:09.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa957fff700 1 --2- 172.21.15.121:0/650334965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e967149e10 0x55e967194d10 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7fa948010000 tx=0x7fa948016000).ready entity=mon.0 client_cookie=2252c4d2c6a542da server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:09.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa95f1c2700 1 -- 172.21.15.121:0/650334965 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9480086a0 con 0x55e967149e10 2023-11-15T03:05:09.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa95f1c2700 1 -- 172.21.15.121:0/650334965 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa948000bc0 con 0x55e967149e10 2023-11-15T03:05:09.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa95f1c2700 1 -- 172.21.15.121:0/650334965 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948008970 con 0x55e967149e10 2023-11-15T03:05:09.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa96bb37d00 1 -- 172.21.15.121:0/650334965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e967149e10 msgr2=0x55e967194d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/650334965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e967149e10 0x55e967194d10 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7fa948010000 tx=0x7fa948016000).stop 2023-11-15T03:05:09.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa96bb37d00 1 -- 172.21.15.121:0/650334965 shutdown_connections 2023-11-15T03:05:09.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/650334965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e967198b20 0x55e9672331e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/650334965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e967149e10 0x55e967194d10 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/650334965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e967149a30 0x55e967220280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa96bb37d00 1 -- 172.21.15.121:0/650334965 >> 172.21.15.121:0/650334965 conn(0x55e967199080 msgr2=0x55e967199460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:09.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa96bb37d00 1 -- 172.21.15.121:0/650334965 shutdown_connections 2023-11-15T03:05:09.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.199+0000 7fa96bb37d00 1 -- 172.21.15.121:0/650334965 wait complete. 2023-11-15T03:05:09.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa96bb37d00 1 Processor -- start 2023-11-15T03:05:09.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa96bb37d00 1 -- start start 2023-11-15T03:05:09.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa96bb37d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e967149a30 0x55e96721f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa96bb37d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e967149e10 0x55e967228f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa96bb37d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e967198b20 0x55e9672248f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa96bb37d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e967223180 con 0x55e967198b20 2023-11-15T03:05:09.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa96bb37d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96718f2b0 con 0x55e967149e10 2023-11-15T03:05:09.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa96bb37d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e967191490 con 0x55e967149a30 2023-11-15T03:05:09.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa9609c5700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e967198b20 0x55e9672248f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa9609c5700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e967198b20 0x55e9672248f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34048/0 (socket says 172.21.15.121:34048) 2023-11-15T03:05:09.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 learned_addr learned my addr 172.21.15.121:0/2544867222 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:09.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa957fff700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e967149e10 0x55e967228f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa9601c4700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e967149a30 0x55e96721f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e967149a30 msgr2=0x55e96721f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa9609c5700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e967149a30 0x55e96721f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e967149e10 msgr2=0x55e967228f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa9609c5700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e967149e10 0x55e967228f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9670eb770 con 0x55e967198b20 2023-11-15T03:05:09.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa9609c5700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e967198b20 0x55e9672248f0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7fa95800d490 tx=0x7fa958006910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:09.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.203+0000 7fa95d9bf700 1 -- 172.21.15.121:0/2544867222 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95800dea0 con 0x55e967198b20 2023-11-15T03:05:09.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.207+0000 7fa95d9bf700 1 -- 172.21.15.121:0/2544867222 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa958013480 con 0x55e967198b20 2023-11-15T03:05:09.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.207+0000 7fa95d9bf700 1 -- 172.21.15.121:0/2544867222 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958013840 con 0x55e967198b20 2023-11-15T03:05:09.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.207+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e9670baa70 con 0x55e967198b20 2023-11-15T03:05:09.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.207+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9670c6010 con 0x55e967198b20 2023-11-15T03:05:09.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.207+0000 7fa95d9bf700 1 -- 172.21.15.121:0/2544867222 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa9580123e0 con 0x55e967198b20 2023-11-15T03:05:09.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.211+0000 7fa95d9bf700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa940058ff0 0x7fa94005b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.211+0000 7fa95d9bf700 1 -- 172.21.15.121:0/2544867222 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7040+0+0 (secure 0 0 0) 0x7fa958073640 con 0x55e967198b20 2023-11-15T03:05:09.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.211+0000 7fa9601c4700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa940058ff0 0x7fa94005b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.221 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:05:09.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.211+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55e967229400 0x55e967191e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.211+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967192320 con 0x55e967229400 2023-11-15T03:05:09.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.211+0000 7fa9601c4700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa940058ff0 0x7fa94005b4b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fa9500011e0 tx=0x7fa950001710).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:09.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.211+0000 7fa957fff700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55e967229400 0x55e967191e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.211+0000 7fa957fff700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55e967229400 0x55e967191e30 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:09.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.215+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v103'101 uv94 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967229400 2023-11-15T03:05:09.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.215+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e9671926d0 con 0x55e967229400 2023-11-15T03:05:09.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.215+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v103'102 uv102 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967229400 2023-11-15T03:05:09.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.215+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e9672432a0 con 0x55e967229400 2023-11-15T03:05:09.226 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:05:09.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.219+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v103'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967229400 2023-11-15T03:05:09.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.219+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967244660 con 0x55e967229400 2023-11-15T03:05:09.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.219+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v103'104 uv104 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967229400 2023-11-15T03:05:09.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.219+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967244a10 con 0x55e967229400 2023-11-15T03:05:09.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.223+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v103'105 uv105 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967229400 2023-11-15T03:05:09.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.223+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967243650 con 0x55e967229400 2023-11-15T03:05:09.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.223+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v103'106 uv106 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967229400 2023-11-15T03:05:09.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.223+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967243a50 con 0x55e967229400 2023-11-15T03:05:09.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.227+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v103'107 uv100 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967229400 2023-11-15T03:05:09.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.227+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e967243e50 con 0x55e967229400 2023-11-15T03:05:09.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.227+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v103'108 uv108 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967229400 2023-11-15T03:05:09.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.227+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967243e50 con 0x55e967229400 2023-11-15T03:05:09.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.231+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v103'109 uv109 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967229400 2023-11-15T03:05:09.239 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:05:09.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.231+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967243e50 con 0x55e967229400 2023-11-15T03:05:09.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.235+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v103'110 uv110 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967229400 2023-11-15T03:05:09.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.235+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967243e50 con 0x55e967229400 2023-11-15T03:05:09.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.235+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v103'111 uv111 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967229400 2023-11-15T03:05:09.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.235+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967243e50 con 0x55e967229400 2023-11-15T03:05:09.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.239+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v103'112 uv112 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967229400 2023-11-15T03:05:09.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.239+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e967244250 0x55e9672472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.239+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967243e50 con 0x55e967244250 2023-11-15T03:05:09.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.239+0000 7fa9601c4700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e967244250 0x55e9672472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.239+0000 7fa9601c4700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e967244250 0x55e9672472e0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:09.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.243+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v103'157 uv144 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.243+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e9672477d0 con 0x55e967244250 2023-11-15T03:05:09.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.247+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v103'158 uv158 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.247+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967247b80 con 0x55e967244250 2023-11-15T03:05:09.260 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:05:09.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.251+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v103'159 uv159 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.251+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967248f40 con 0x55e967244250 2023-11-15T03:05:09.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.255+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v103'160 uv160 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.255+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e9672492f0 con 0x55e967244250 2023-11-15T03:05:09.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.255+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v103'161 uv161 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.255+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967247f30 con 0x55e967244250 2023-11-15T03:05:09.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.259+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v103'162 uv162 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.259+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967248820 con 0x55e967244250 2023-11-15T03:05:09.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.263+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v103'163 uv163 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.263+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e9672496a0 con 0x55e967244250 2023-11-15T03:05:09.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.267+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v103'164 uv150 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.267+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e9672496a0 con 0x55e967244250 2023-11-15T03:05:09.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.267+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v103'165 uv165 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.267+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e9672496a0 con 0x55e967244250 2023-11-15T03:05:09.279 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:05:09.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.271+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v103'166 uv166 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.271+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96724a6b0 con 0x55e967244250 2023-11-15T03:05:09.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.275+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v103'167 uv167 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.275+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96724a6b0 con 0x55e967244250 2023-11-15T03:05:09.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.279+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v103'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.279+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e9672496a0 con 0x55e967244250 2023-11-15T03:05:09.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.279+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v103'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.279+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e9672496a0 con 0x55e967244250 2023-11-15T03:05:09.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.283+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v103'170 uv156 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.283+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e9672496a0 con 0x55e967244250 2023-11-15T03:05:09.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.287+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v103'171 uv171 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.287+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e9672496a0 con 0x55e967244250 2023-11-15T03:05:09.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.287+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v103'172 uv172 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.297 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:05:09.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.287+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96724a6b0 con 0x55e967244250 2023-11-15T03:05:09.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.291+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v103'173 uv173 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.291+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96724a6b0 con 0x55e967244250 2023-11-15T03:05:09.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.295+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v103'174 uv174 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.295+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e9672496a0 con 0x55e967244250 2023-11-15T03:05:09.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.295+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v103'175 uv175 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.295+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55e967249bb0 0x55e96724bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.299+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96724c490 con 0x55e967249bb0 2023-11-15T03:05:09.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.299+0000 7fa957fff700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55e967249bb0 0x55e96724bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.299+0000 7fa957fff700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55e967249bb0 0x55e96724bfa0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:09.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.303+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v103'87 uv81 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967249bb0 2023-11-15T03:05:09.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.303+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e96724c490 con 0x55e967249bb0 2023-11-15T03:05:09.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.303+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v103'88 uv88 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967249bb0 2023-11-15T03:05:09.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.303+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96724c490 con 0x55e967249bb0 2023-11-15T03:05:09.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.307+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v103'89 uv89 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967249bb0 2023-11-15T03:05:09.315 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:05:09.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.307+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96724c890 con 0x55e967249bb0 2023-11-15T03:05:09.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.311+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v103'90 uv90 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967249bb0 2023-11-15T03:05:09.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.311+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e9672483b0 con 0x55e967249bb0 2023-11-15T03:05:09.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.315+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v103'91 uv91 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967249bb0 2023-11-15T03:05:09.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.315+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e9672483b0 con 0x55e967249bb0 2023-11-15T03:05:09.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.319+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v103'92 uv86 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967249bb0 2023-11-15T03:05:09.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.319+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e967248330 con 0x55e967249bb0 2023-11-15T03:05:09.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.319+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v103'93 uv93 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967249bb0 2023-11-15T03:05:09.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.319+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96724c890 con 0x55e967249bb0 2023-11-15T03:05:09.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.327+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v103'94 uv94 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967249bb0 2023-11-15T03:05:09.334 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:05:09.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.327+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96724c890 con 0x55e967249bb0 2023-11-15T03:05:09.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.327+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v103'95 uv95 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967249bb0 2023-11-15T03:05:09.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.331+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96724c890 con 0x55e967249bb0 2023-11-15T03:05:09.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.331+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v103'96 uv96 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967249bb0 2023-11-15T03:05:09.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.331+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e96724cda0 0x55e96724f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.331+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96724f780 con 0x55e96724cda0 2023-11-15T03:05:09.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.335+0000 7fa9609c5700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e96724cda0 0x55e96724f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.335+0000 7fa9609c5700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e96724cda0 0x55e96724f190 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:09.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.335+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v103'111 uv104 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.339+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e96724fc80 con 0x55e96724cda0 2023-11-15T03:05:09.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.339+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v103'112 uv112 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.339+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967250130 con 0x55e96724cda0 2023-11-15T03:05:09.351 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:05:09.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.343+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v103'113 uv113 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.343+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967248430 con 0x55e96724cda0 2023-11-15T03:05:09.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.347+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v103'114 uv114 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.347+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e9672546c0 con 0x55e96724cda0 2023-11-15T03:05:09.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.351+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v103'115 uv115 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.351+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967250530 con 0x55e96724cda0 2023-11-15T03:05:09.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.355+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v103'116 uv110 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.355+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e967250930 con 0x55e96724cda0 2023-11-15T03:05:09.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.359+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v103'117 uv117 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.359+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967250930 con 0x55e96724cda0 2023-11-15T03:05:09.369 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:05:09.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.359+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v103'118 uv118 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.359+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967250930 con 0x55e96724cda0 2023-11-15T03:05:09.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.363+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v103'119 uv119 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.363+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967250930 con 0x55e96724cda0 2023-11-15T03:05:09.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.367+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v103'120 uv120 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.367+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967250930 con 0x55e96724cda0 2023-11-15T03:05:09.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.371+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v103'121 uv121 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.371+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55e9672514b0 0x55e9672538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.371+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967253d90 con 0x55e9672514b0 2023-11-15T03:05:09.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.371+0000 7fa9601c4700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55e9672514b0 0x55e9672538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.371+0000 7fa9601c4700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55e9672514b0 0x55e9672538a0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:09.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.375+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v103'51 uv50 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e9672514b0 2023-11-15T03:05:09.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.375+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e967253d90 con 0x55e9672514b0 2023-11-15T03:05:09.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.375+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v103'52 uv52 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e9672514b0 2023-11-15T03:05:09.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.375+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967253d90 con 0x55e9672514b0 2023-11-15T03:05:09.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.379+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v103'53 uv53 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e9672514b0 2023-11-15T03:05:09.387 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:05:09.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.379+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e9672541c0 con 0x55e9672514b0 2023-11-15T03:05:09.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.383+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v103'54 uv54 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e9672514b0 2023-11-15T03:05:09.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.383+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e9672541c0 con 0x55e9672514b0 2023-11-15T03:05:09.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.383+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v103'55 uv55 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e9672514b0 2023-11-15T03:05:09.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.383+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967253d90 con 0x55e9672514b0 2023-11-15T03:05:09.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.387+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v103'56 uv56 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e9672514b0 2023-11-15T03:05:09.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.387+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e9672541c0 0x55e967257560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.387+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967257a70 con 0x55e9672541c0 2023-11-15T03:05:09.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.387+0000 7fa957fff700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e9672541c0 0x55e967257560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.387+0000 7fa957fff700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e9672541c0 0x55e967257560 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:09.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.391+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v103'39 uv38 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.391+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e967257e50 con 0x55e9672541c0 2023-11-15T03:05:09.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.395+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v103'40 uv40 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.395+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967257e50 con 0x55e9672541c0 2023-11-15T03:05:09.404 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:05:09.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.395+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v103'41 uv41 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.395+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967257e50 con 0x55e9672541c0 2023-11-15T03:05:09.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.399+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v103'42 uv42 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.399+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967257e20 con 0x55e9672541c0 2023-11-15T03:05:09.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.403+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v103'51 uv50 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.403+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e967257de0 con 0x55e9672541c0 2023-11-15T03:05:09.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.407+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v103'52 uv52 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.407+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967257de0 con 0x55e9672541c0 2023-11-15T03:05:09.416 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:05:09.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.407+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v103'53 uv53 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.407+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967258df0 con 0x55e9672541c0 2023-11-15T03:05:09.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.411+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v103'54 uv54 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.411+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967258e30 con 0x55e9672541c0 2023-11-15T03:05:09.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.415+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v103'55 uv55 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.415+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967257dc0 con 0x55e9672541c0 2023-11-15T03:05:09.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.415+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v103'56 uv56 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.415+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967257dc0 con 0x55e967244250 2023-11-15T03:05:09.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.419+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v103'94 uv88 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.419+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e967257dc0 con 0x55e967244250 2023-11-15T03:05:09.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.423+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v103'95 uv95 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.423+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967257dc0 con 0x55e967244250 2023-11-15T03:05:09.434 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:05:09.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.427+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v103'96 uv96 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.427+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967257dc0 con 0x55e967244250 2023-11-15T03:05:09.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.431+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v103'97 uv97 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.431+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967257dc0 con 0x55e967244250 2023-11-15T03:05:09.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.431+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v103'98 uv98 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.431+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e9672581c0 con 0x55e967244250 2023-11-15T03:05:09.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.435+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v103'99 uv99 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.435+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e9672581c0 con 0x55e967244250 2023-11-15T03:05:09.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.439+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v103'100 uv93 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.439+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e967257dc0 con 0x55e967244250 2023-11-15T03:05:09.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.439+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v103'101 uv101 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.439+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967257dc0 con 0x55e967244250 2023-11-15T03:05:09.451 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:05:09.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.443+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v103'102 uv102 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.443+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967257dc0 con 0x55e967244250 2023-11-15T03:05:09.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.447+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v103'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.447+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967258240 con 0x55e967244250 2023-11-15T03:05:09.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.451+0000 7fa9601c4700 1 -- 172.21.15.121:0/2544867222 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v103'104 uv104 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa95000ea40 con 0x55e967244250 2023-11-15T03:05:09.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.451+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967258240 con 0x55e967249bb0 2023-11-15T03:05:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.455+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v103'43 uv43 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967249bb0 2023-11-15T03:05:09.462 INFO:teuthology.orchestra.run.smithi121.stdout:Write #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:05:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.455+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967258240 con 0x55e967249bb0 2023-11-15T03:05:09.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.455+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [write 0~70] v103'44 uv44 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967249bb0 2023-11-15T03:05:09.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.459+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967258240 con 0x55e967249bb0 2023-11-15T03:05:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.459+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v103'45 uv45 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e967249bb0 2023-11-15T03:05:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.459+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967258240 con 0x55e9672541c0 2023-11-15T03:05:09.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.463+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(83 605.00000000 [create] v103'51 uv50 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.463+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e967258240 con 0x55e9672541c0 2023-11-15T03:05:09.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.467+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(84 605.00000000 [delete] v103'52 uv52 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.467+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967258240 con 0x55e9672541c0 2023-11-15T03:05:09.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.467+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(85 605.00000000 [create] v103'53 uv53 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.476 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:05:09.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.467+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967258240 con 0x55e9672541c0 2023-11-15T03:05:09.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.471+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v103'54 uv54 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.471+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967258240 con 0x55e9672541c0 2023-11-15T03:05:09.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.475+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v103'55 uv55 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.475+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967258240 con 0x55e9672541c0 2023-11-15T03:05:09.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.475+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(88 605.00000000 [omap-set-header] v103'56 uv56 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.475+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967258240 con 0x55e9672541c0 2023-11-15T03:05:09.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.479+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(89 mds0_inotable [create] v103'43 uv42 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.479+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e967257fa0 con 0x55e9672541c0 2023-11-15T03:05:09.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.483+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(90 mds0_inotable [delete] v103'44 uv44 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.483+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967257eb0 con 0x55e9672541c0 2023-11-15T03:05:09.495 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:05:09.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.487+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(91 mds0_inotable [create] v103'45 uv45 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.487+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967257e60 con 0x55e9672541c0 2023-11-15T03:05:09.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.491+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(92 mds0_inotable [write 0~34] v103'46 uv46 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.491+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967257e60 con 0x55e9672541c0 2023-11-15T03:05:09.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.499+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v103'47 uv47 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.499+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967257e60 con 0x55e9672541c0 2023-11-15T03:05:09.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.503+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(94 100.00000000 [create] v103'42 uv41 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.503+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e967257e60 con 0x55e9672541c0 2023-11-15T03:05:09.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.507+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(95 100.00000000 [delete] v103'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.507+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967257e40 con 0x55e9672541c0 2023-11-15T03:05:09.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.511+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(96 100.00000000 [create] v103'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.521 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:05:09.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.511+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e967257e40 con 0x55e9672541c0 2023-11-15T03:05:09.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.515+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(97 100.00000000 [omap-set-header] v103'45 uv45 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.515+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725aec0 con 0x55e9672541c0 2023-11-15T03:05:09.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.519+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v103'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.519+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55e967254b70 0x55e96725b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.519+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e967254b70 2023-11-15T03:05:09.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.519+0000 7fa9609c5700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55e967254b70 0x55e96725b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.523+0000 7fa9609c5700 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55e967254b70 0x55e96725b3d0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:09.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.523+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(99 mds0_sessionmap [create] v103'35 uv34 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e967254b70 2023-11-15T03:05:09.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.523+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e967254b70 2023-11-15T03:05:09.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.527+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(100 mds0_sessionmap [delete] v103'36 uv36 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e967254b70 2023-11-15T03:05:09.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.527+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e967254b70 2023-11-15T03:05:09.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.531+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(101 mds0_sessionmap [create] v103'37 uv37 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e967254b70 2023-11-15T03:05:09.539 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:05:09.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.531+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e967254b70 2023-11-15T03:05:09.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.535+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v103'38 uv38 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e967254b70 2023-11-15T03:05:09.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.535+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e9672541c0 2023-11-15T03:05:09.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.539+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(103 609.00000000 [create] v103'51 uv50 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.539+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e9672541c0 2023-11-15T03:05:09.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.539+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(104 609.00000000 [delete] v103'52 uv52 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.539+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e9672541c0 2023-11-15T03:05:09.551 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:05:09.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.543+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(105 609.00000000 [create] v103'53 uv53 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.543+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725c8d0 con 0x55e9672541c0 2023-11-15T03:05:09.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.547+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v103'54 uv54 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.547+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725c8d0 con 0x55e9672541c0 2023-11-15T03:05:09.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.547+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v103'55 uv55 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.547+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e9672541c0 2023-11-15T03:05:09.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.551+0000 7fa957fff700 1 -- 172.21.15.121:0/2544867222 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(108 609.00000000 [omap-set-header] v103'56 uv56 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa948023020 con 0x55e9672541c0 2023-11-15T03:05:09.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.551+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e96724cda0 2023-11-15T03:05:09.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.555+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(109 400.00000000 [create] v103'42 uv41 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.555+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e96724cda0 2023-11-15T03:05:09.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.559+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(110 400.00000000 [delete] v103'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.559+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e96724cda0 2023-11-15T03:05:09.570 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:05:09.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.563+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(111 400.00000000 [create] v103'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.563+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e96724cda0 2023-11-15T03:05:09.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.567+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(112 400.00000000 [write 0~22] v103'45 uv45 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.567+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e96724cda0 2023-11-15T03:05:09.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.567+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(113 400.00000000 [omap-set-header] v103'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.567+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e96724cda0 2023-11-15T03:05:09.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.571+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(114 100.00000000.inode [create] v103'102 uv94 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.571+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e96724cda0 2023-11-15T03:05:09.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.575+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(115 100.00000000.inode [delete] v103'103 uv103 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.575+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e96724cda0 2023-11-15T03:05:09.584 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:05:09.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.575+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(116 100.00000000.inode [create] v103'104 uv104 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.575+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e96724cda0 2023-11-15T03:05:09.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.579+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v103'105 uv105 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.579+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e96724cda0 2023-11-15T03:05:09.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.583+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v103'106 uv106 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.583+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e96724cda0 2023-11-15T03:05:09.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.587+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(119 1.00000000 [create] v103'107 uv101 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.587+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e96724cda0 2023-11-15T03:05:09.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.587+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(120 1.00000000 [delete] v103'108 uv108 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.587+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725b8c0 con 0x55e96724cda0 2023-11-15T03:05:09.598 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:05:09.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.591+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(121 1.00000000 [create] v103'109 uv109 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.591+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725c8d0 con 0x55e96724cda0 2023-11-15T03:05:09.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.591+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v103'110 uv110 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.595+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725c8d0 con 0x55e96724cda0 2023-11-15T03:05:09.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.595+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v103'111 uv111 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.595+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725bdb0 con 0x55e96724cda0 2023-11-15T03:05:09.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.599+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v103'112 uv112 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.599+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e103) v8 -- 0x55e96725bdb0 con 0x55e96724cda0 2023-11-15T03:05:09.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v103'113 uv113 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa9580489f0 con 0x55e96724cda0 2023-11-15T03:05:09.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55e967249bb0 msgr2=0x55e96724bfa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55e967249bb0 0x55e96724bfa0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55e967229400 msgr2=0x55e967191e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55e967229400 0x55e967191e30 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55e9672514b0 msgr2=0x55e9672538a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55e9672514b0 0x55e9672538a0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e967244250 msgr2=0x55e9672472e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e967244250 0x55e9672472e0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55e967254b70 msgr2=0x55e96725b3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55e967254b70 0x55e96725b3d0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e96724cda0 msgr2=0x55e96724f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e96724cda0 0x55e96724f190 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e9672541c0 msgr2=0x55e967257560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 reap_dead start 2023-11-15T03:05:09.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e9672541c0 0x55e967257560 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 reap_dead start 2023-11-15T03:05:09.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa9609c5700 1 -- 172.21.15.121:0/2544867222 reap_dead start 2023-11-15T03:05:09.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa940058ff0 msgr2=0x7fa94005b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa940058ff0 0x7fa94005b4b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fa9500011e0 tx=0x7fa950001710).stop 2023-11-15T03:05:09.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e967198b20 msgr2=0x55e9672248f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.603+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e967198b20 0x55e9672248f0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7fa95800d490 tx=0x7fa958006910).stop 2023-11-15T03:05:09.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.607+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 shutdown_connections 2023-11-15T03:05:09.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.607+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa940058ff0 0x7fa94005b4b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.607+0000 7fa96bb37d00 1 --2- 172.21.15.121:0/2544867222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e967198b20 0x55e9672248f0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.607+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 >> 172.21.15.121:0/2544867222 conn(0x55e967199080 msgr2=0x55e967195d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:09.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.607+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 shutdown_connections 2023-11-15T03:05:09.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.607+0000 7fa96bb37d00 1 -- 172.21.15.121:0/2544867222 wait complete. 2023-11-15T03:05:09.621 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_inotable 2023-11-15T03:05:09.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.715+0000 7f74876f2d00 1 Processor -- start 2023-11-15T03:05:09.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.715+0000 7f74876f2d00 1 -- start start 2023-11-15T03:05:09.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.715+0000 7f74876f2d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560849d23c90 0x560849dab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.715+0000 7f74876f2d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560849dafec0 0x560849d22120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.715+0000 7f74876f2d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560849db02a0 0x560849d22610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.715+0000 7f74876f2d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560849c50220 con 0x560849db02a0 2023-11-15T03:05:09.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.715+0000 7f74876f2d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560849db26d0 con 0x560849dafec0 2023-11-15T03:05:09.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.715+0000 7f74876f2d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560849d06160 con 0x560849d23c90 2023-11-15T03:05:09.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.715+0000 7f7485823700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560849db02a0 0x560849d22610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.715+0000 7f7485823700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560849db02a0 0x560849d22610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34050/0 (socket says 172.21.15.121:34050) 2023-11-15T03:05:09.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f7485823700 1 -- 172.21.15.121:0/3030703273 learned_addr learned my addr 172.21.15.121:0/3030703273 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:09.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f7485022700 1 --2- 172.21.15.121:0/3030703273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560849d23c90 0x560849dab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f747c821700 1 --2- 172.21.15.121:0/3030703273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560849dafec0 0x560849d22120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f7485823700 1 -- 172.21.15.121:0/3030703273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560849d23c90 msgr2=0x560849dab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:09.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f7485823700 1 --2- 172.21.15.121:0/3030703273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560849d23c90 0x560849dab1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f7485823700 1 -- 172.21.15.121:0/3030703273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560849dafec0 msgr2=0x560849d22120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:09.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f7485823700 1 --2- 172.21.15.121:0/3030703273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560849dafec0 0x560849d22120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f7485823700 1 -- 172.21.15.121:0/3030703273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7480005cf0 con 0x560849db02a0 2023-11-15T03:05:09.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f7485022700 1 --2- 172.21.15.121:0/3030703273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560849d23c90 0x560849dab1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:09.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f747c821700 1 --2- 172.21.15.121:0/3030703273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560849dafec0 0x560849d22120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:09.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f7485823700 1 --2- 172.21.15.121:0/3030703273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560849db02a0 0x560849d22610 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7f748000ac40 tx=0x7f7480015000).ready entity=mon.0 client_cookie=40bcf0cae5d68309 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:09.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f747ffff700 1 -- 172.21.15.121:0/3030703273 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7480007ea0 con 0x560849db02a0 2023-11-15T03:05:09.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f747ffff700 1 -- 172.21.15.121:0/3030703273 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7480005330 con 0x560849db02a0 2023-11-15T03:05:09.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f747ffff700 1 -- 172.21.15.121:0/3030703273 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7480007610 con 0x560849db02a0 2023-11-15T03:05:09.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f74876f2d00 1 -- 172.21.15.121:0/3030703273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560849db02a0 msgr2=0x560849d22610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f74876f2d00 1 --2- 172.21.15.121:0/3030703273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560849db02a0 0x560849d22610 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7f748000ac40 tx=0x7f7480015000).stop 2023-11-15T03:05:09.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f74876f2d00 1 -- 172.21.15.121:0/3030703273 shutdown_connections 2023-11-15T03:05:09.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f74876f2d00 1 --2- 172.21.15.121:0/3030703273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560849db02a0 0x560849d22610 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f74876f2d00 1 --2- 172.21.15.121:0/3030703273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560849dafec0 0x560849d22120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f74876f2d00 1 --2- 172.21.15.121:0/3030703273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560849d23c90 0x560849dab1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f74876f2d00 1 -- 172.21.15.121:0/3030703273 >> 172.21.15.121:0/3030703273 conn(0x560849cd87d0 msgr2=0x560849cda0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:09.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f74876f2d00 1 -- 172.21.15.121:0/3030703273 shutdown_connections 2023-11-15T03:05:09.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.719+0000 7f74876f2d00 1 -- 172.21.15.121:0/3030703273 wait complete. 2023-11-15T03:05:09.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f74876f2d00 1 Processor -- start 2023-11-15T03:05:09.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f74876f2d00 1 -- start start 2023-11-15T03:05:09.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f74876f2d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560849d23c90 0x560849d244b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f74876f2d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560849dafec0 0x560849d22330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f74876f2d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560849db02a0 0x560849daae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f74876f2d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560849dbb3a0 con 0x560849d23c90 2023-11-15T03:05:09.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f74876f2d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560849dbb680 con 0x560849db02a0 2023-11-15T03:05:09.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f74876f2d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560849dbb960 con 0x560849dafec0 2023-11-15T03:05:09.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f7485022700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560849d23c90 0x560849d244b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f7485022700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560849d23c90 0x560849d244b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34066/0 (socket says 172.21.15.121:34066) 2023-11-15T03:05:09.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f7485022700 1 -- 172.21.15.121:0/3878120928 learned_addr learned my addr 172.21.15.121:0/3878120928 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:09.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f747c821700 1 --2- 172.21.15.121:0/3878120928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560849dafec0 0x560849d22330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f7485823700 1 --2- 172.21.15.121:0/3878120928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560849db02a0 0x560849daae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f7485022700 1 -- 172.21.15.121:0/3878120928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560849dafec0 msgr2=0x560849d22330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f7485022700 1 --2- 172.21.15.121:0/3878120928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560849dafec0 0x560849d22330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f7485022700 1 -- 172.21.15.121:0/3878120928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560849db02a0 msgr2=0x560849daae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f7485022700 1 --2- 172.21.15.121:0/3878120928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560849db02a0 0x560849daae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f7485022700 1 -- 172.21.15.121:0/3878120928 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560849c79a20 con 0x560849d23c90 2023-11-15T03:05:09.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f7485823700 1 --2- 172.21.15.121:0/3878120928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560849db02a0 0x560849daae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:09.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f7485022700 1 --2- 172.21.15.121:0/3878120928 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560849d23c90 0x560849d244b0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7f747800deb0 tx=0x7f7478000bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:09.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.723+0000 7f747e7fc700 1 -- 172.21.15.121:0/3878120928 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747800be50 con 0x560849d23c90 2023-11-15T03:05:09.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.727+0000 7f747e7fc700 1 -- 172.21.15.121:0/3878120928 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747800f450 con 0x560849d23c90 2023-11-15T03:05:09.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.727+0000 7f747e7fc700 1 -- 172.21.15.121:0/3878120928 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747800f810 con 0x560849d23c90 2023-11-15T03:05:09.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.727+0000 7f74876f2d00 1 -- 172.21.15.121:0/3878120928 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560849c79770 con 0x560849d23c90 2023-11-15T03:05:09.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.727+0000 7f74876f2d00 1 -- 172.21.15.121:0/3878120928 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560849c48a70 con 0x560849d23c90 2023-11-15T03:05:09.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.727+0000 7f747e7fc700 1 -- 172.21.15.121:0/3878120928 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f747800fb80 con 0x560849d23c90 2023-11-15T03:05:09.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.731+0000 7f747e7fc700 1 --2- 172.21.15.121:0/3878120928 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7468058ff0 0x7f746805b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.731+0000 7f747e7fc700 1 -- 172.21.15.121:0/3878120928 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7040+0+0 (secure 0 0 0) 0x7f7478074150 con 0x560849d23c90 2023-11-15T03:05:09.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.731+0000 7f747c821700 1 --2- 172.21.15.121:0/3878120928 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7468058ff0 0x7f746805b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.731+0000 7f74876f2d00 1 --2- 172.21.15.121:0/3878120928 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x560849d1d660 0x560849d1da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.731+0000 7f74876f2d00 1 -- 172.21.15.121:0/3878120928 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e103) v8 -- 0x560849d1df30 con 0x560849d1d660 2023-11-15T03:05:09.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.731+0000 7f747c821700 1 --2- 172.21.15.121:0/3878120928 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7468058ff0 0x7f746805b4b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f7470003440 tx=0x7f7470007040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:09.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.731+0000 7f7485823700 1 --2- 172.21.15.121:0/3878120928 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x560849d1d660 0x560849d1da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.731+0000 7f7485823700 1 --2- 172.21.15.121:0/3878120928 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x560849d1d660 0x560849d1da40 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:09.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.735+0000 7f7485823700 1 -- 172.21.15.121:0/3878120928 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 mds0_inotable [delete] v103'48 uv48 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7480007400 con 0x560849d1d660 2023-11-15T03:05:09.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.735+0000 7f74876f2d00 1 -- 172.21.15.121:0/3878120928 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x560849d1d660 msgr2=0x560849d1da40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.735+0000 7f74876f2d00 1 --2- 172.21.15.121:0/3878120928 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x560849d1d660 0x560849d1da40 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.735+0000 7f74876f2d00 1 -- 172.21.15.121:0/3878120928 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7468058ff0 msgr2=0x7f746805b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.735+0000 7f74876f2d00 1 --2- 172.21.15.121:0/3878120928 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7468058ff0 0x7f746805b4b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f7470003440 tx=0x7f7470007040).stop 2023-11-15T03:05:09.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.735+0000 7f74876f2d00 1 -- 172.21.15.121:0/3878120928 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560849d23c90 msgr2=0x560849d244b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.735+0000 7f74876f2d00 1 --2- 172.21.15.121:0/3878120928 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560849d23c90 0x560849d244b0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7f747800deb0 tx=0x7f7478000bc0).stop 2023-11-15T03:05:09.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.735+0000 7f7485823700 1 -- 172.21.15.121:0/3878120928 reap_dead start 2023-11-15T03:05:09.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.735+0000 7f74876f2d00 1 -- 172.21.15.121:0/3878120928 shutdown_connections 2023-11-15T03:05:09.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.735+0000 7f74876f2d00 1 -- 172.21.15.121:0/3878120928 >> 172.21.15.121:0/3878120928 conn(0x560849cd87d0 msgr2=0x560849db10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:09.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.735+0000 7f74876f2d00 1 -- 172.21.15.121:0/3878120928 shutdown_connections 2023-11-15T03:05:09.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.735+0000 7f74876f2d00 1 -- 172.21.15.121:0/3878120928 wait complete. 2023-11-15T03:05:09.750 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:05:09.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.915+0000 7f3cfe2e4700 1 Processor -- start 2023-11-15T03:05:09.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.915+0000 7f3cfe2e4700 1 -- start start 2023-11-15T03:05:09.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.919+0000 7f3cfe2e4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3cf80acd00 0x7f3cf80ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.919+0000 7f3cfe2e4700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3cf8066330 0x7f3cf8153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.919+0000 7f3cfe2e4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf8065ea0 con 0x7f3cf80c5c70 2023-11-15T03:05:09.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.919+0000 7f3cfe2e4700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cf8066bb0 con 0x7f3cf8066330 2023-11-15T03:05:09.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.919+0000 7f3cfe2e4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cf8065890 con 0x7f3cf80acd00 2023-11-15T03:05:09.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.919+0000 7f3cfdae3700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3cf8066330 0x7f3cf8153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.919+0000 7f3cfdae3700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3cf8066330 0x7f3cf8153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34076/0 (socket says 172.21.15.121:34076) 2023-11-15T03:05:09.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.919+0000 7f3cfdae3700 1 -- 172.21.15.121:0/305162190 learned_addr learned my addr 172.21.15.121:0/305162190 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:09.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.919+0000 7f3cfd2e2700 1 --2- 172.21.15.121:0/305162190 >> v2:172.21.15.190:3300/0 conn(0x7f3cf80acd00 0x7f3cf80ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.919+0000 7f3cfdae3700 1 -- 172.21.15.121:0/305162190 >> v2:172.21.15.190:3300/0 conn(0x7f3cf80acd00 msgr2=0x7f3cf80ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:09.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.919+0000 7f3cfdae3700 1 --2- 172.21.15.121:0/305162190 >> v2:172.21.15.190:3300/0 conn(0x7f3cf80acd00 0x7f3cf80ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.919+0000 7f3cfdae3700 1 -- 172.21.15.121:0/305162190 >> v1:172.21.15.190:6789/0 conn(0x7f3cf80c5c70 legacy=0x7f3cf81536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.919+0000 7f3cfdae3700 1 -- 172.21.15.121:0/305162190 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf80c64a0 con 0x7f3cf8066330 2023-11-15T03:05:09.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.919+0000 7f3cfdae3700 1 --2- 172.21.15.121:0/305162190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf8066330 0x7f3cf8153dd0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7f3cf806c280 tx=0x7f3cf0009840).ready entity=mon.0 client_cookie=c04cab6151df82c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:09.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.919+0000 7f3ceffff700 1 -- 172.21.15.121:0/305162190 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf000e070 con 0x7f3cf8066330 2023-11-15T03:05:09.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.919+0000 7f3ceffff700 1 -- 172.21.15.121:0/305162190 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf000ba50 con 0x7f3cf8066330 2023-11-15T03:05:09.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.919+0000 7f3ceffff700 1 -- 172.21.15.121:0/305162190 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf000bdd0 con 0x7f3cf8066330 2023-11-15T03:05:09.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.923+0000 7f3cfe2e4700 1 -- 172.21.15.121:0/305162190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf8066330 msgr2=0x7f3cf8153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.923+0000 7f3cfe2e4700 1 --2- 172.21.15.121:0/305162190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf8066330 0x7f3cf8153dd0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7f3cf806c280 tx=0x7f3cf0009840).stop 2023-11-15T03:05:09.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.923+0000 7f3cfe2e4700 1 -- 172.21.15.121:0/305162190 shutdown_connections 2023-11-15T03:05:09.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.923+0000 7f3cfe2e4700 1 --2- 172.21.15.121:0/305162190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf8066330 0x7f3cf8153dd0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.923+0000 7f3cfe2e4700 1 --2- 172.21.15.121:0/305162190 >> v2:172.21.15.190:3300/0 conn(0x7f3cf80acd00 0x7f3cf80ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.923+0000 7f3cfe2e4700 1 -- 172.21.15.121:0/305162190 >> 172.21.15.121:0/305162190 conn(0x7f3cf8063fc0 msgr2=0x7f3cf8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:09.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.923+0000 7f3cfe2e4700 1 -- 172.21.15.121:0/305162190 shutdown_connections 2023-11-15T03:05:09.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.923+0000 7f3cfe2e4700 1 -- 172.21.15.121:0/305162190 wait complete. 2023-11-15T03:05:09.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.923+0000 7f3cfe2e4700 1 Processor -- start 2023-11-15T03:05:09.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.923+0000 7f3cfe2e4700 1 -- start start 2023-11-15T03:05:09.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfe2e4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cf8066330 0x7f3cf80c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfe2e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cf80acd00 0x7f3cf80c3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfe2e4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf80c5c70 0x7f3cf8150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfe2e4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf80ca2c0 con 0x7f3cf80c5c70 2023-11-15T03:05:09.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfe2e4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf8154ca0 con 0x7f3cf8066330 2023-11-15T03:05:09.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfe2e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf8154370 con 0x7f3cf80acd00 2023-11-15T03:05:09.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfdae3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf80c5c70 0x7f3cf8150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfdae3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf80c5c70 0x7f3cf8150300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34084/0 (socket says 172.21.15.121:34084) 2023-11-15T03:05:09.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfdae3700 1 -- 172.21.15.121:0/2153868998 learned_addr learned my addr 172.21.15.121:0/2153868998 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:09.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfd2e2700 1 --2- 172.21.15.121:0/2153868998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cf8066330 0x7f3cf80c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfcae1700 1 --2- 172.21.15.121:0/2153868998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cf80acd00 0x7f3cf80c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfdae3700 1 -- 172.21.15.121:0/2153868998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cf80acd00 msgr2=0x7f3cf80c3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfdae3700 1 --2- 172.21.15.121:0/2153868998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cf80acd00 0x7f3cf80c3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfdae3700 1 -- 172.21.15.121:0/2153868998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cf8066330 msgr2=0x7f3cf80c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:09.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfdae3700 1 --2- 172.21.15.121:0/2153868998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cf8066330 0x7f3cf80c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:09.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfdae3700 1 -- 172.21.15.121:0/2153868998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf0009ae0 con 0x7f3cf80c5c70 2023-11-15T03:05:09.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfcae1700 1 --2- 172.21.15.121:0/2153868998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cf80acd00 0x7f3cf80c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:05:09.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfd2e2700 1 --2- 172.21.15.121:0/2153868998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cf8066330 0x7f3cf80c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:09.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfdae3700 1 --2- 172.21.15.121:0/2153868998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf80c5c70 0x7f3cf8150300 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7f3cf000bef0 tx=0x7f3cf0009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:09.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3ced7fa700 1 -- 172.21.15.121:0/2153868998 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf00092e0 con 0x7f3cf80c5c70 2023-11-15T03:05:09.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3ced7fa700 1 -- 172.21.15.121:0/2153868998 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf001d410 con 0x7f3cf80c5c70 2023-11-15T03:05:09.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3ced7fa700 1 -- 172.21.15.121:0/2153868998 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf001d790 con 0x7f3cf80c5c70 2023-11-15T03:05:09.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfe2e4700 1 -- 172.21.15.121:0/2153868998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf8154650 con 0x7f3cf80c5c70 2023-11-15T03:05:09.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.927+0000 7f3cfe2e4700 1 -- 172.21.15.121:0/2153868998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf80c9fc0 con 0x7f3cf80c5c70 2023-11-15T03:05:09.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.931+0000 7f3cfe2e4700 1 -- 172.21.15.121:0/2153868998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf8154e30 con 0x7f3cf80c5c70 2023-11-15T03:05:09.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.931+0000 7f3ced7fa700 1 -- 172.21.15.121:0/2153868998 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3cf001dbc0 con 0x7f3cf80c5c70 2023-11-15T03:05:09.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.931+0000 7f3ced7fa700 1 --2- 172.21.15.121:0/2153868998 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3cdc05a510 0x7f3cdc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:09.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.935+0000 7f3ced7fa700 1 -- 172.21.15.121:0/2153868998 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7040+0+0 (secure 0 0 0) 0x7f3cf00767f0 con 0x7f3cf80c5c70 2023-11-15T03:05:09.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.935+0000 7f3cfd2e2700 1 --2- 172.21.15.121:0/2153868998 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3cdc05a510 0x7f3cdc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:09.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.935+0000 7f3cfd2e2700 1 --2- 172.21.15.121:0/2153868998 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3cdc05a510 0x7f3cdc05c9d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f3cf4006fd0 tx=0x7f3cf4008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:09.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:09.939+0000 7f3ced7fa700 1 -- 172.21.15.121:0/2153868998 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3cf0049c40 con 0x7f3cf80c5c70 2023-11-15T03:05:10.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:10.143+0000 7f3cfe2e4700 1 -- 172.21.15.121:0/2153868998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f3cf806bcf0 con 0x7f3cf80c5c70 2023-11-15T03:05:11.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.139+0000 7f3ced7fa700 1 -- 172.21.15.121:0/2153868998 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v138) v1 ==== 162+0+0 (secure 0 0 0) 0x7f3cf004d260 con 0x7f3cf80c5c70 2023-11-15T03:05:11.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.143+0000 7f3cfe2e4700 1 -- 172.21.15.121:0/2153868998 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3cdc05a510 msgr2=0x7f3cdc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:11.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.143+0000 7f3cfe2e4700 1 --2- 172.21.15.121:0/2153868998 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3cdc05a510 0x7f3cdc05c9d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f3cf4006fd0 tx=0x7f3cf4008040).stop 2023-11-15T03:05:11.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.143+0000 7f3cfe2e4700 1 -- 172.21.15.121:0/2153868998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf80c5c70 msgr2=0x7f3cf8150300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:11.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.143+0000 7f3cfe2e4700 1 --2- 172.21.15.121:0/2153868998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf80c5c70 0x7f3cf8150300 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7f3cf000bef0 tx=0x7f3cf0009840).stop 2023-11-15T03:05:11.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.143+0000 7f3cfe2e4700 1 -- 172.21.15.121:0/2153868998 shutdown_connections 2023-11-15T03:05:11.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.143+0000 7f3cfe2e4700 1 --2- 172.21.15.121:0/2153868998 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3cdc05a510 0x7f3cdc05c9d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:11.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.143+0000 7f3cfe2e4700 1 --2- 172.21.15.121:0/2153868998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf80c5c70 0x7f3cf8150300 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:11.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.143+0000 7f3cfe2e4700 1 --2- 172.21.15.121:0/2153868998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cf80acd00 0x7f3cf80c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:11.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.143+0000 7f3cfe2e4700 1 --2- 172.21.15.121:0/2153868998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cf8066330 0x7f3cf80c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:11.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.143+0000 7f3cfe2e4700 1 -- 172.21.15.121:0/2153868998 >> 172.21.15.121:0/2153868998 conn(0x7f3cf8063fc0 msgr2=0x7f3cf81528e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:11.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.143+0000 7f3cfe2e4700 1 -- 172.21.15.121:0/2153868998 shutdown_connections 2023-11-15T03:05:11.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.143+0000 7f3cfe2e4700 1 -- 172.21.15.121:0/2153868998 wait complete. 2023-11-15T03:05:11.154 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:05:11.156 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:05:11.147+0000 7f694b5ea700 -1 mds.0.inotable: load_2 could not read table: -2 2023-11-15T03:05:11.156 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:05:11.147+0000 7f694b5ea700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-11-15T03:05:11.176 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:05:11.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.323+0000 7f6534ff9700 1 Processor -- start 2023-11-15T03:05:11.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.323+0000 7f6534ff9700 1 -- start start 2023-11-15T03:05:11.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f6534ff9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6530066320 0x7f6530154110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:11.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f6534ff9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65300c73e0 con 0x7f6530066700 2023-11-15T03:05:11.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f6534ff9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65301546c0 con 0x7f65300acf10 2023-11-15T03:05:11.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f6534ff9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65301549a0 con 0x7f6530066320 2023-11-15T03:05:11.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652effd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6530066700 0x7f65300656a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49650/0 (socket says 172.21.15.121:49650) 2023-11-15T03:05:11.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652effd700 1 -- 172.21.15.121:0/1870815643 learned_addr learned my addr 172.21.15.121:0/1870815643 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:11.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652f7fe700 1 --2- 172.21.15.121:0/1870815643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6530066320 0x7f6530154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:11.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652e7fc700 1 -- 172.21.15.121:0/1870815643 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437290283 0 0) 0x7f65300c73e0 con 0x7f6530066700 2023-11-15T03:05:11.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652e7fc700 1 -- 172.21.15.121:0/1870815643 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65180033f0 con 0x7f6530066700 2023-11-15T03:05:11.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652e7fc700 1 -- 172.21.15.121:0/1870815643 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1811260724 0 0) 0x7f65180033f0 con 0x7f6530066700 2023-11-15T03:05:11.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652e7fc700 1 -- 172.21.15.121:0/1870815643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6530066320 msgr2=0x7f6530154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:11.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652e7fc700 1 --2- 172.21.15.121:0/1870815643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6530066320 0x7f6530154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:11.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652e7fc700 1 -- 172.21.15.121:0/1870815643 >> v1:172.21.15.190:6789/0 conn(0x7f65300acf10 legacy=0x7f6530062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:11.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652e7fc700 1 -- 172.21.15.121:0/1870815643 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6530154c80 con 0x7f6530066700 2023-11-15T03:05:11.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652e7fc700 1 -- 172.21.15.121:0/1870815643 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f6524001470 con 0x7f6530066700 2023-11-15T03:05:11.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652e7fc700 1 -- 172.21.15.121:0/1870815643 >> v1:172.21.15.121:6789/0 conn(0x7f6530066700 legacy=0x7f65300656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:11.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652e7fc700 1 --2- 172.21.15.121:0/1870815643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6530066320 0x7f6518003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:11.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652e7fc700 1 --2- 172.21.15.121:0/1870815643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6518003e40 0x7f6518004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:11.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652e7fc700 1 --2- 172.21.15.121:0/1870815643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6518004b80 0x7f6518005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:11.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652e7fc700 1 -- 172.21.15.121:0/1870815643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65301549a0 con 0x7f6518004b80 2023-11-15T03:05:11.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652e7fc700 1 -- 172.21.15.121:0/1870815643 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651800b990 con 0x7f6518003e40 2023-11-15T03:05:11.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652e7fc700 1 -- 172.21.15.121:0/1870815643 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651800bb50 con 0x7f6530066320 2023-11-15T03:05:11.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652ffff700 1 --2- 172.21.15.121:0/1870815643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6518004b80 0x7f6518005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:11.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652effd700 1 --2- 172.21.15.121:0/1870815643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6518003e40 0x7f6518004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:11.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652f7fe700 1 --2- 172.21.15.121:0/1870815643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6530066320 0x7f6518003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:11.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652ffff700 1 -- 172.21.15.121:0/1870815643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6530066320 msgr2=0x7f6518003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:11.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652ffff700 1 --2- 172.21.15.121:0/1870815643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6530066320 0x7f6518003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:11.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652ffff700 1 -- 172.21.15.121:0/1870815643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6518003e40 msgr2=0x7f6518004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:11.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652ffff700 1 --2- 172.21.15.121:0/1870815643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6518003e40 0x7f6518004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:11.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652ffff700 1 -- 172.21.15.121:0/1870815643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65180033f0 con 0x7f6518004b80 2023-11-15T03:05:11.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652ffff700 1 --2- 172.21.15.121:0/1870815643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6518004b80 0x7f6518005000 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f6528001e30 tx=0x7f6528007f30).ready entity=mon.0 client_cookie=d7ee56b26145d514 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:11.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652e7fc700 1 -- 172.21.15.121:0/1870815643 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6528006a20 con 0x7f6518004b80 2023-11-15T03:05:11.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652e7fc700 1 -- 172.21.15.121:0/1870815643 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6528006b80 con 0x7f6518004b80 2023-11-15T03:05:11.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.327+0000 7f652e7fc700 1 -- 172.21.15.121:0/1870815643 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652800c5d0 con 0x7f6518004b80 2023-11-15T03:05:11.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.331+0000 7f6534ff9700 1 -- 172.21.15.121:0/1870815643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6518004b80 msgr2=0x7f6518005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:11.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.331+0000 7f6534ff9700 1 --2- 172.21.15.121:0/1870815643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6518004b80 0x7f6518005000 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f6528001e30 tx=0x7f6528007f30).stop 2023-11-15T03:05:11.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.331+0000 7f652ffff700 1 -- 172.21.15.121:0/1870815643 reap_dead start 2023-11-15T03:05:11.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.331+0000 7f6534ff9700 1 -- 172.21.15.121:0/1870815643 shutdown_connections 2023-11-15T03:05:11.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.331+0000 7f6534ff9700 1 -- 172.21.15.121:0/1870815643 >> 172.21.15.121:0/1870815643 conn(0x7f6530067a60 msgr2=0x7f6530067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:11.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.331+0000 7f6534ff9700 1 -- 172.21.15.121:0/1870815643 shutdown_connections 2023-11-15T03:05:11.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.331+0000 7f6534ff9700 1 -- 172.21.15.121:0/1870815643 wait complete. 2023-11-15T03:05:11.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.331+0000 7f6534ff9700 1 Processor -- start 2023-11-15T03:05:11.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.331+0000 7f6534ff9700 1 -- start start 2023-11-15T03:05:11.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f6534ff9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6518004b80 0x7f6530147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:11.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f6534ff9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65300af550 0x7f65300af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:11.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f6534ff9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6530164620 0x7f6530164aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:11.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f6534ff9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65300a4fe0 con 0x7f6530164620 2023-11-15T03:05:11.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f6534ff9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65300a5b60 con 0x7f6518004b80 2023-11-15T03:05:11.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f6534ff9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65300afea0 con 0x7f65300af550 2023-11-15T03:05:11.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f652ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6530164620 0x7f6530164aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:11.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f652ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6530164620 0x7f6530164aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34102/0 (socket says 172.21.15.121:34102) 2023-11-15T03:05:11.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f652ffff700 1 -- 172.21.15.121:0/301657116 learned_addr learned my addr 172.21.15.121:0/301657116 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:11.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f652effd700 1 --2- 172.21.15.121:0/301657116 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65300af550 0x7f65300af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:11.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f652f7fe700 1 --2- 172.21.15.121:0/301657116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6518004b80 0x7f6530147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:11.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f652ffff700 1 -- 172.21.15.121:0/301657116 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65300af550 msgr2=0x7f65300af9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:11.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f652ffff700 1 --2- 172.21.15.121:0/301657116 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65300af550 0x7f65300af9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:11.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f652ffff700 1 -- 172.21.15.121:0/301657116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6518004b80 msgr2=0x7f6530147500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:11.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f652ffff700 1 --2- 172.21.15.121:0/301657116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6518004b80 0x7f6530147500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:11.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f652ffff700 1 -- 172.21.15.121:0/301657116 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6530165470 con 0x7f6530164620 2023-11-15T03:05:11.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f652ffff700 1 --2- 172.21.15.121:0/301657116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6530164620 0x7f6530164aa0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0x7f6518005650 tx=0x7f6528016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:11.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f65177fe700 1 -- 172.21.15.121:0/301657116 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6528006a20 con 0x7f6530164620 2023-11-15T03:05:11.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f65177fe700 1 -- 172.21.15.121:0/301657116 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652800c5d0 con 0x7f6530164620 2023-11-15T03:05:11.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f6534ff9700 1 -- 172.21.15.121:0/301657116 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65300a5230 con 0x7f6530164620 2023-11-15T03:05:11.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f65177fe700 1 -- 172.21.15.121:0/301657116 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6528012840 con 0x7f6530164620 2023-11-15T03:05:11.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.335+0000 7f6534ff9700 1 -- 172.21.15.121:0/301657116 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65300a57f0 con 0x7f6530164620 2023-11-15T03:05:11.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.339+0000 7f6534ff9700 1 -- 172.21.15.121:0/301657116 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f653006bcf0 con 0x7f6530164620 2023-11-15T03:05:11.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.339+0000 7f65177fe700 1 -- 172.21.15.121:0/301657116 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6528012c70 con 0x7f6530164620 2023-11-15T03:05:11.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.339+0000 7f65177fe700 1 --2- 172.21.15.121:0/301657116 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f651c05a540 0x7f651c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:11.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.339+0000 7f65177fe700 1 -- 172.21.15.121:0/301657116 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7126+0+0 (secure 0 0 0) 0x7f65280756f0 con 0x7f6530164620 2023-11-15T03:05:11.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.339+0000 7f652f7fe700 1 --2- 172.21.15.121:0/301657116 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f651c05a540 0x7f651c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:11.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.343+0000 7f652f7fe700 1 --2- 172.21.15.121:0/301657116 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f651c05a540 0x7f651c05ca00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f6520009c30 tx=0x7f6520008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:11.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.347+0000 7f65177fe700 1 -- 172.21.15.121:0/301657116 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6528048b30 con 0x7f6530164620 2023-11-15T03:05:11.505 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:05:11.513 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:05:11.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.543+0000 7f6534ff9700 1 -- 172.21.15.121:0/301657116 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6530154f60 con 0x7f6530164620 2023-11-15T03:05:11.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.547+0000 7f65177fe700 1 -- 172.21.15.121:0/301657116 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f6528023070 con 0x7f6530164620 2023-11-15T03:05:11.554 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:11.555 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":139,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:11.142421+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8130},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8130":{"gid":8130,"name":"a","rank":0,"incarnation":139,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/2402059232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2402059232},{"type":"v1","addr":"172.21.15.121:6836","nonce":2402059232}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:05:11.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.547+0000 7f6534ff9700 1 -- 172.21.15.121:0/301657116 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f651c05a540 msgr2=0x7f651c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:11.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.547+0000 7f6534ff9700 1 --2- 172.21.15.121:0/301657116 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f651c05a540 0x7f651c05ca00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f6520009c30 tx=0x7f6520008040).stop 2023-11-15T03:05:11.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.547+0000 7f6534ff9700 1 -- 172.21.15.121:0/301657116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6530164620 msgr2=0x7f6530164aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:11.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.547+0000 7f6534ff9700 1 --2- 172.21.15.121:0/301657116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6530164620 0x7f6530164aa0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0x7f6518005650 tx=0x7f6528016000).stop 2023-11-15T03:05:11.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.547+0000 7f6534ff9700 1 -- 172.21.15.121:0/301657116 shutdown_connections 2023-11-15T03:05:11.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.547+0000 7f6534ff9700 1 --2- 172.21.15.121:0/301657116 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f651c05a540 0x7f651c05ca00 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:11.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.547+0000 7f6534ff9700 1 --2- 172.21.15.121:0/301657116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6530164620 0x7f6530164aa0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:11.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.547+0000 7f6534ff9700 1 --2- 172.21.15.121:0/301657116 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65300af550 0x7f65300af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:11.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.547+0000 7f6534ff9700 1 --2- 172.21.15.121:0/301657116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6518004b80 0x7f6530147500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:11.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.547+0000 7f6534ff9700 1 -- 172.21.15.121:0/301657116 >> 172.21.15.121:0/301657116 conn(0x7f6530067a60 msgr2=0x7f65300c9270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:11.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.547+0000 7f6534ff9700 1 -- 172.21.15.121:0/301657116 shutdown_connections 2023-11-15T03:05:11.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:11.547+0000 7f6534ff9700 1 -- 172.21.15.121:0/301657116 wait complete. 2023-11-15T03:05:11.559 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 139 2023-11-15T03:05:11.574 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:05:12.153 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:05:12.143+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:05:12.153 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:05:12.143+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:05:12.164 INFO:tasks.ceph.mds.a.smithi121.stderr: -28> 2023-11-15T03:05:11.147+0000 7f694b5ea700 -1 mds.0.inotable: load_2 could not read table: -2 2023-11-15T03:05:12.164 INFO:tasks.ceph.mds.a.smithi121.stderr: -27> 2023-11-15T03:05:11.147+0000 7f694b5ea700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-11-15T03:05:12.205 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:05:16.575 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:05:16.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.726+0000 7f8cfaee4700 1 Processor -- start 2023-11-15T03:05:16.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.726+0000 7f8cfaee4700 1 -- start start 2023-11-15T03:05:16.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cfaee4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf4157ef0 0x7f8cf4067140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:16.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cfaee4700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cf4066800 con 0x7f8cf4066370 2023-11-15T03:05:16.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cfaee4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cf41583f0 con 0x7f8cf4066d60 2023-11-15T03:05:16.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cfaee4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf40c00f0 con 0x7f8cf4157ef0 2023-11-15T03:05:16.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cfa6e3700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f8cf4066370 0x7f8cf41535c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:42960/0 (socket says 172.21.15.121:42960) 2023-11-15T03:05:16.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cfa6e3700 1 -- 172.21.15.121:0/856282737 learned_addr learned my addr 172.21.15.121:0/856282737 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:16.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf96e1700 1 --2- 172.21.15.121:0/856282737 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf4157ef0 0x7f8cf4067140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:16.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf8ee0700 1 -- 172.21.15.121:0/856282737 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2424145364 0 0) 0x7f8cf4066800 con 0x7f8cf4066370 2023-11-15T03:05:16.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf8ee0700 1 -- 172.21.15.121:0/856282737 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ce00033f0 con 0x7f8cf4066370 2023-11-15T03:05:16.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf8ee0700 1 -- 172.21.15.121:0/856282737 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 796825735 0 0) 0x7f8cf41583f0 con 0x7f8cf4066d60 2023-11-15T03:05:16.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf8ee0700 1 -- 172.21.15.121:0/856282737 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cf4066800 con 0x7f8cf4066d60 2023-11-15T03:05:16.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf8ee0700 1 -- 172.21.15.121:0/856282737 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2661655061 0 0) 0x7f8ce00033f0 con 0x7f8cf4066370 2023-11-15T03:05:16.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf8ee0700 1 -- 172.21.15.121:0/856282737 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf4157ef0 msgr2=0x7f8cf4067140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:16.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf8ee0700 1 --2- 172.21.15.121:0/856282737 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf4157ef0 0x7f8cf4067140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:16.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf8ee0700 1 -- 172.21.15.121:0/856282737 >> v1:172.21.15.190:6789/0 conn(0x7f8cf4066d60 legacy=0x7f8cf41577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:16.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf8ee0700 1 -- 172.21.15.121:0/856282737 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf41586d0 con 0x7f8cf4066370 2023-11-15T03:05:16.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf8ee0700 1 -- 172.21.15.121:0/856282737 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f8cec0017f0 con 0x7f8cf4066370 2023-11-15T03:05:16.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf8ee0700 1 -- 172.21.15.121:0/856282737 >> v1:172.21.15.121:6789/0 conn(0x7f8cf4066370 legacy=0x7f8cf41535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:16.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf8ee0700 1 --2- 172.21.15.121:0/856282737 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf4157ef0 0x7f8ce0003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:16.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf8ee0700 1 --2- 172.21.15.121:0/856282737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce0003fb0 0x7f8ce0004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:16.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf8ee0700 1 --2- 172.21.15.121:0/856282737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce0004b80 0x7f8ce0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:16.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf8ee0700 1 -- 172.21.15.121:0/856282737 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf40c00f0 con 0x7f8ce0003fb0 2023-11-15T03:05:16.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf8ee0700 1 -- 172.21.15.121:0/856282737 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce000b990 con 0x7f8ce0004b80 2023-11-15T03:05:16.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf8ee0700 1 -- 172.21.15.121:0/856282737 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce000bb50 con 0x7f8cf4157ef0 2023-11-15T03:05:16.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf96e1700 1 --2- 172.21.15.121:0/856282737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce0003fb0 0x7f8ce0004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:16.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cf9ee2700 1 --2- 172.21.15.121:0/856282737 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf4157ef0 0x7f8ce0003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:16.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.730+0000 7f8cfa6e3700 1 --2- 172.21.15.121:0/856282737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce0004b80 0x7f8ce0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:16.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.734+0000 7f8cf96e1700 1 -- 172.21.15.121:0/856282737 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf4157ef0 msgr2=0x7f8ce0003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:16.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.734+0000 7f8cf96e1700 1 --2- 172.21.15.121:0/856282737 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf4157ef0 0x7f8ce0003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:16.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.734+0000 7f8cf96e1700 1 -- 172.21.15.121:0/856282737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce0004b80 msgr2=0x7f8ce0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:16.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.734+0000 7f8cf96e1700 1 --2- 172.21.15.121:0/856282737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce0004b80 0x7f8ce0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:16.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.734+0000 7f8cf96e1700 1 -- 172.21.15.121:0/856282737 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce00033f0 con 0x7f8ce0003fb0 2023-11-15T03:05:16.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.734+0000 7f8cfa6e3700 1 --2- 172.21.15.121:0/856282737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce0004b80 0x7f8ce0005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:16.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.734+0000 7f8cf96e1700 1 --2- 172.21.15.121:0/856282737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce0003fb0 0x7f8ce0004690 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7f8ce4009a80 tx=0x7f8ce4009060).ready entity=mon.0 client_cookie=9184ab90f4272bb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:16.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.734+0000 7f8cf8ee0700 1 -- 172.21.15.121:0/856282737 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4009430 con 0x7f8ce0003fb0 2023-11-15T03:05:16.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.734+0000 7f8cf8ee0700 1 -- 172.21.15.121:0/856282737 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce400e640 con 0x7f8ce0003fb0 2023-11-15T03:05:16.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.734+0000 7f8cf8ee0700 1 -- 172.21.15.121:0/856282737 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400e9c0 con 0x7f8ce0003fb0 2023-11-15T03:05:16.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.734+0000 7f8cfaee4700 1 -- 172.21.15.121:0/856282737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce0003fb0 msgr2=0x7f8ce0004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:16.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.734+0000 7f8cfaee4700 1 --2- 172.21.15.121:0/856282737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce0003fb0 0x7f8ce0004690 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7f8ce4009a80 tx=0x7f8ce4009060).stop 2023-11-15T03:05:16.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.734+0000 7f8cfa6e3700 1 -- 172.21.15.121:0/856282737 reap_dead start 2023-11-15T03:05:16.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.734+0000 7f8cfaee4700 1 -- 172.21.15.121:0/856282737 shutdown_connections 2023-11-15T03:05:16.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.734+0000 7f8cfaee4700 1 -- 172.21.15.121:0/856282737 >> 172.21.15.121:0/856282737 conn(0x7f8cf4062190 msgr2=0x7f8cf4062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:16.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.734+0000 7f8cfaee4700 1 -- 172.21.15.121:0/856282737 shutdown_connections 2023-11-15T03:05:16.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.734+0000 7f8cfaee4700 1 -- 172.21.15.121:0/856282737 wait complete. 2023-11-15T03:05:16.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.738+0000 7f8cfaee4700 1 Processor -- start 2023-11-15T03:05:16.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.738+0000 7f8cfaee4700 1 -- start start 2023-11-15T03:05:16.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.738+0000 7f8cfaee4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce0003fb0 0x7f8cf4146290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:16.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.738+0000 7f8cfaee4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cf414ef60 0x7f8cf414f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:16.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.738+0000 7f8cfaee4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf40a7cc0 0x7f8cf40a8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:16.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.738+0000 7f8cfaee4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf4152eb0 con 0x7f8ce0003fb0 2023-11-15T03:05:16.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.738+0000 7f8cfaee4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf40aa440 con 0x7f8cf414ef60 2023-11-15T03:05:16.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.738+0000 7f8cfaee4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf414f8d0 con 0x7f8cf40a7cc0 2023-11-15T03:05:16.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.738+0000 7f8cf9ee2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce0003fb0 0x7f8cf4146290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:16.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.738+0000 7f8cf9ee2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce0003fb0 0x7f8cf4146290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39808/0 (socket says 172.21.15.121:39808) 2023-11-15T03:05:16.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.738+0000 7f8cf9ee2700 1 -- 172.21.15.121:0/4096429619 learned_addr learned my addr 172.21.15.121:0/4096429619 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:16.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.738+0000 7f8cf96e1700 1 --2- 172.21.15.121:0/4096429619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cf414ef60 0x7f8cf414f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:16.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.738+0000 7f8cfa6e3700 1 --2- 172.21.15.121:0/4096429619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf40a7cc0 0x7f8cf40a8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:16.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.738+0000 7f8cf9ee2700 1 -- 172.21.15.121:0/4096429619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf40a7cc0 msgr2=0x7f8cf40a8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:16.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.738+0000 7f8cf9ee2700 1 --2- 172.21.15.121:0/4096429619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf40a7cc0 0x7f8cf40a8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:16.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.738+0000 7f8cf9ee2700 1 -- 172.21.15.121:0/4096429619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cf414ef60 msgr2=0x7f8cf414f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:16.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.738+0000 7f8cf9ee2700 1 --2- 172.21.15.121:0/4096429619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cf414ef60 0x7f8cf414f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:16.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.738+0000 7f8cf9ee2700 1 -- 172.21.15.121:0/4096429619 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf40a8610 con 0x7f8ce0003fb0 2023-11-15T03:05:16.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.738+0000 7f8cf9ee2700 1 --2- 172.21.15.121:0/4096429619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce0003fb0 0x7f8cf4146290 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f8cf0002a50 tx=0x7f8cf0007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:16.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.742+0000 7f8ce9ffb700 1 -- 172.21.15.121:0/4096429619 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0002e30 con 0x7f8ce0003fb0 2023-11-15T03:05:16.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.742+0000 7f8ce9ffb700 1 -- 172.21.15.121:0/4096429619 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf000f8d0 con 0x7f8ce0003fb0 2023-11-15T03:05:16.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.742+0000 7f8cfaee4700 1 -- 172.21.15.121:0/4096429619 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf414fb40 con 0x7f8ce0003fb0 2023-11-15T03:05:16.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.742+0000 7f8ce9ffb700 1 -- 172.21.15.121:0/4096429619 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf000fc70 con 0x7f8ce0003fb0 2023-11-15T03:05:16.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.742+0000 7f8cfaee4700 1 -- 172.21.15.121:0/4096429619 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf4150100 con 0x7f8ce0003fb0 2023-11-15T03:05:16.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.742+0000 7f8cfaee4700 1 -- 172.21.15.121:0/4096429619 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf406bcf0 con 0x7f8ce0003fb0 2023-11-15T03:05:16.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.742+0000 7f8ce9ffb700 1 -- 172.21.15.121:0/4096429619 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8cf001d3b0 con 0x7f8ce0003fb0 2023-11-15T03:05:16.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.746+0000 7f8ce9ffb700 1 --2- 172.21.15.121:0/4096429619 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8cd805a540 0x7f8cd805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:16.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.746+0000 7f8ce9ffb700 1 -- 172.21.15.121:0/4096429619 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7126+0+0 (secure 0 0 0) 0x7f8cf00749a0 con 0x7f8ce0003fb0 2023-11-15T03:05:16.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.746+0000 7f8cf96e1700 1 --2- 172.21.15.121:0/4096429619 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8cd805a540 0x7f8cd805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:16.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.746+0000 7f8cf96e1700 1 --2- 172.21.15.121:0/4096429619 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8cd805a540 0x7f8cd805ca00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f8ce4006d50 tx=0x7f8ce401a000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:16.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.750+0000 7f8ce9ffb700 1 -- 172.21.15.121:0/4096429619 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8cf0047d30 con 0x7f8ce0003fb0 2023-11-15T03:05:16.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.950+0000 7f8cfaee4700 1 -- 172.21.15.121:0/4096429619 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cf4065890 con 0x7f8ce0003fb0 2023-11-15T03:05:16.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.950+0000 7f8ce9ffb700 1 -- 172.21.15.121:0/4096429619 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f8cf004b350 con 0x7f8ce0003fb0 2023-11-15T03:05:16.960 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:16.960 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":141,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8199,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2860780116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2860780116},{"type":"v1","addr":"172.21.15.121:6836","nonce":2860780116}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:12.149521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:05:16.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.954+0000 7f8cfaee4700 1 -- 172.21.15.121:0/4096429619 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8cd805a540 msgr2=0x7f8cd805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:16.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.954+0000 7f8cfaee4700 1 --2- 172.21.15.121:0/4096429619 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8cd805a540 0x7f8cd805ca00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f8ce4006d50 tx=0x7f8ce401a000).stop 2023-11-15T03:05:16.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.954+0000 7f8cfaee4700 1 -- 172.21.15.121:0/4096429619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce0003fb0 msgr2=0x7f8cf4146290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:16.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.954+0000 7f8cfaee4700 1 --2- 172.21.15.121:0/4096429619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce0003fb0 0x7f8cf4146290 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f8cf0002a50 tx=0x7f8cf0007bb0).stop 2023-11-15T03:05:16.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.954+0000 7f8cfaee4700 1 -- 172.21.15.121:0/4096429619 shutdown_connections 2023-11-15T03:05:16.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.954+0000 7f8cfaee4700 1 --2- 172.21.15.121:0/4096429619 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8cd805a540 0x7f8cd805ca00 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:16.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.954+0000 7f8cfaee4700 1 --2- 172.21.15.121:0/4096429619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf40a7cc0 0x7f8cf40a8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:16.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.954+0000 7f8cfaee4700 1 --2- 172.21.15.121:0/4096429619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cf414ef60 0x7f8cf414f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:16.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.954+0000 7f8cfaee4700 1 --2- 172.21.15.121:0/4096429619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce0003fb0 0x7f8cf4146290 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:16.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.954+0000 7f8cfaee4700 1 -- 172.21.15.121:0/4096429619 >> 172.21.15.121:0/4096429619 conn(0x7f8cf4062190 msgr2=0x7f8cf40c5850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:16.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.954+0000 7f8cfaee4700 1 -- 172.21.15.121:0/4096429619 shutdown_connections 2023-11-15T03:05:16.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:16.954+0000 7f8cfaee4700 1 -- 172.21.15.121:0/4096429619 wait complete. 2023-11-15T03:05:16.965 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 141 2023-11-15T03:05:16.979 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:05:16.979 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:05:17.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.122+0000 7fe261707700 1 Processor -- start 2023-11-15T03:05:17.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.122+0000 7fe261707700 1 -- start start 2023-11-15T03:05:17.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe261707700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe25c151870 0x7fe25c151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:17.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe261707700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe25c0c30e0 0x7fe25c0c3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:17.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe261707700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe25c0c3a80 con 0x7fe25c152140 2023-11-15T03:05:17.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe261707700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe25c0657f0 con 0x7fe25c0c30e0 2023-11-15T03:05:17.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe261707700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c065ad0 con 0x7fe25c151870 2023-11-15T03:05:17.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25b7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe25c152140 0x7fe25c0a9830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:42964/0 (socket says 172.21.15.121:42964) 2023-11-15T03:05:17.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25b7fe700 1 -- 172.21.15.121:0/1366512834 learned_addr learned my addr 172.21.15.121:0/1366512834 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:17.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25bfff700 1 --2- 172.21.15.121:0/1366512834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe25c151870 0x7fe25c151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:17.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe260f06700 1 --2- 172.21.15.121:0/1366512834 >> v2:172.21.15.190:3300/0 conn(0x7fe25c0c30e0 0x7fe25c0c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:17.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25affd700 1 -- 172.21.15.121:0/1366512834 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2743960664 0 0) 0x7fe24c001490 con 0x7fe25c152140 2023-11-15T03:05:17.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25affd700 1 -- 172.21.15.121:0/1366512834 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2480033f0 con 0x7fe25c152140 2023-11-15T03:05:17.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25affd700 1 -- 172.21.15.121:0/1366512834 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3986114992 0 0) 0x7fe2480033f0 con 0x7fe25c152140 2023-11-15T03:05:17.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25affd700 1 -- 172.21.15.121:0/1366512834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe25c151870 msgr2=0x7fe25c151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:17.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25affd700 1 --2- 172.21.15.121:0/1366512834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe25c151870 0x7fe25c151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:17.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25affd700 1 -- 172.21.15.121:0/1366512834 >> v2:172.21.15.190:3300/0 conn(0x7fe25c0c30e0 msgr2=0x7fe25c0c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:17.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25affd700 1 --2- 172.21.15.121:0/1366512834 >> v2:172.21.15.190:3300/0 conn(0x7fe25c0c30e0 0x7fe25c0c3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:17.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25affd700 1 -- 172.21.15.121:0/1366512834 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25c0aa260 con 0x7fe25c152140 2023-11-15T03:05:17.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25affd700 1 -- 172.21.15.121:0/1366512834 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fe24c001980 con 0x7fe25c152140 2023-11-15T03:05:17.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25affd700 1 -- 172.21.15.121:0/1366512834 >> v1:172.21.15.121:6789/0 conn(0x7fe25c152140 legacy=0x7fe25c0a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:17.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25affd700 1 --2- 172.21.15.121:0/1366512834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2480038e0 0x7fe248003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:17.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25affd700 1 --2- 172.21.15.121:0/1366512834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe248004690 0x7fe248004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:17.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25affd700 1 --2- 172.21.15.121:0/1366512834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe25c151870 0x7fe248005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:17.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25affd700 1 -- 172.21.15.121:0/1366512834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c0657f0 con 0x7fe248004690 2023-11-15T03:05:17.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25affd700 1 -- 172.21.15.121:0/1366512834 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c065ad0 con 0x7fe2480038e0 2023-11-15T03:05:17.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25affd700 1 -- 172.21.15.121:0/1366512834 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24800b8e0 con 0x7fe25c151870 2023-11-15T03:05:17.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe260f06700 1 --2- 172.21.15.121:0/1366512834 >> v2:172.21.15.190:3300/0 conn(0x7fe25c0c30e0 0x7fe25c0c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:05:17.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.126+0000 7fe25b7fe700 1 --2- 172.21.15.121:0/1366512834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe248004690 0x7fe248004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:17.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.130+0000 7fe25bfff700 1 --2- 172.21.15.121:0/1366512834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2480038e0 0x7fe248003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:17.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.130+0000 7fe25b7fe700 1 -- 172.21.15.121:0/1366512834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe25c151870 msgr2=0x7fe248005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:17.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.130+0000 7fe260f06700 1 --2- 172.21.15.121:0/1366512834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe25c151870 0x7fe248005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:17.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.130+0000 7fe25b7fe700 1 --2- 172.21.15.121:0/1366512834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe25c151870 0x7fe248005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:17.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.130+0000 7fe25b7fe700 1 -- 172.21.15.121:0/1366512834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2480038e0 msgr2=0x7fe248003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:17.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.130+0000 7fe25b7fe700 1 --2- 172.21.15.121:0/1366512834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2480038e0 0x7fe248003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:17.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.130+0000 7fe25b7fe700 1 -- 172.21.15.121:0/1366512834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2480033f0 con 0x7fe248004690 2023-11-15T03:05:17.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.130+0000 7fe260f06700 1 --2- 172.21.15.121:0/1366512834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe25c151870 0x7fe248005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:05:17.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.130+0000 7fe25b7fe700 1 --2- 172.21.15.121:0/1366512834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe248004690 0x7fe248004b10 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7fe24c00c6a0 tx=0x7fe24c007c80).ready entity=mon.0 client_cookie=e1b250377c9cda2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:17.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.130+0000 7fe25affd700 1 -- 172.21.15.121:0/1366512834 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c005b20 con 0x7fe248004690 2023-11-15T03:05:17.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.130+0000 7fe25affd700 1 -- 172.21.15.121:0/1366512834 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24c005c80 con 0x7fe248004690 2023-11-15T03:05:17.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.130+0000 7fe25affd700 1 -- 172.21.15.121:0/1366512834 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c007110 con 0x7fe248004690 2023-11-15T03:05:17.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.130+0000 7fe261707700 1 -- 172.21.15.121:0/1366512834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe248004690 msgr2=0x7fe248004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:17.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.130+0000 7fe261707700 1 --2- 172.21.15.121:0/1366512834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe248004690 0x7fe248004b10 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7fe24c00c6a0 tx=0x7fe24c007c80).stop 2023-11-15T03:05:17.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.130+0000 7fe260f06700 1 -- 172.21.15.121:0/1366512834 reap_dead start 2023-11-15T03:05:17.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.130+0000 7fe261707700 1 -- 172.21.15.121:0/1366512834 shutdown_connections 2023-11-15T03:05:17.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.130+0000 7fe261707700 1 -- 172.21.15.121:0/1366512834 >> 172.21.15.121:0/1366512834 conn(0x7fe25c062190 msgr2=0x7fe25c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:17.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.130+0000 7fe261707700 1 -- 172.21.15.121:0/1366512834 shutdown_connections 2023-11-15T03:05:17.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.130+0000 7fe261707700 1 -- 172.21.15.121:0/1366512834 wait complete. 2023-11-15T03:05:17.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe261707700 1 Processor -- start 2023-11-15T03:05:17.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe261707700 1 -- start start 2023-11-15T03:05:17.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe261707700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe248004690 0x7fe25c140400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:17.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe261707700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe25c1408f0 0x7fe25c14fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:17.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe261707700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe25c1500e0 0x7fe25c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:17.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe261707700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c1664a0 con 0x7fe25c1408f0 2023-11-15T03:05:17.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe261707700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c150520 con 0x7fe25c1500e0 2023-11-15T03:05:17.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe261707700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c165890 con 0x7fe248004690 2023-11-15T03:05:17.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe25b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe25c1408f0 0x7fe25c14fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:17.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe25b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe25c1408f0 0x7fe25c14fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39822/0 (socket says 172.21.15.121:39822) 2023-11-15T03:05:17.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe25b7fe700 1 -- 172.21.15.121:0/1470137303 learned_addr learned my addr 172.21.15.121:0/1470137303 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:17.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe260f06700 1 --2- 172.21.15.121:0/1470137303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe25c1500e0 0x7fe25c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:17.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe25bfff700 1 --2- 172.21.15.121:0/1470137303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe248004690 0x7fe25c140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:17.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe25b7fe700 1 -- 172.21.15.121:0/1470137303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe248004690 msgr2=0x7fe25c140400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:17.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe25b7fe700 1 --2- 172.21.15.121:0/1470137303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe248004690 0x7fe25c140400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:17.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe25b7fe700 1 -- 172.21.15.121:0/1470137303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe25c1500e0 msgr2=0x7fe25c0a2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:17.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe25b7fe700 1 --2- 172.21.15.121:0/1470137303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe25c1500e0 0x7fe25c0a2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:17.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe25b7fe700 1 -- 172.21.15.121:0/1470137303 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25c165b70 con 0x7fe25c1408f0 2023-11-15T03:05:17.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe260f06700 1 --2- 172.21.15.121:0/1470137303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe25c1500e0 0x7fe25c0a2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:17.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe25b7fe700 1 --2- 172.21.15.121:0/1470137303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe25c1408f0 0x7fe25c14fbf0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7fe248004480 tx=0x7fe24c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:17.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe253fff700 1 -- 172.21.15.121:0/1470137303 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c0014e0 con 0x7fe25c1408f0 2023-11-15T03:05:17.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe253fff700 1 -- 172.21.15.121:0/1470137303 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24c0070f0 con 0x7fe25c1408f0 2023-11-15T03:05:17.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe253fff700 1 -- 172.21.15.121:0/1470137303 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c007710 con 0x7fe25c1408f0 2023-11-15T03:05:17.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe261707700 1 -- 172.21.15.121:0/1470137303 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe25c165e50 con 0x7fe25c1408f0 2023-11-15T03:05:17.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.134+0000 7fe261707700 1 -- 172.21.15.121:0/1470137303 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe25c162290 con 0x7fe25c1408f0 2023-11-15T03:05:17.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.138+0000 7fe261707700 1 -- 172.21.15.121:0/1470137303 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe25c166610 con 0x7fe25c1408f0 2023-11-15T03:05:17.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.138+0000 7fe253fff700 1 -- 172.21.15.121:0/1470137303 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe24c014410 con 0x7fe25c1408f0 2023-11-15T03:05:17.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.138+0000 7fe253fff700 1 --2- 172.21.15.121:0/1470137303 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe23c05a510 0x7fe23c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:17.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.138+0000 7fe253fff700 1 -- 172.21.15.121:0/1470137303 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7126+0+0 (secure 0 0 0) 0x7fe24c01c040 con 0x7fe25c1408f0 2023-11-15T03:05:17.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.138+0000 7fe25bfff700 1 --2- 172.21.15.121:0/1470137303 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe23c05a510 0x7fe23c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:17.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.142+0000 7fe25bfff700 1 --2- 172.21.15.121:0/1470137303 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe23c05a510 0x7fe23c05c9d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7fe244006c40 tx=0x7fe244008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:17.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.146+0000 7fe253fff700 1 -- 172.21.15.121:0/1470137303 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe24c0497b0 con 0x7fe25c1408f0 2023-11-15T03:05:17.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.346+0000 7fe261707700 1 -- 172.21.15.121:0/1470137303 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe25c0c3a80 con 0x7fe25c1408f0 2023-11-15T03:05:17.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.346+0000 7fe253fff700 1 -- 172.21.15.121:0/1470137303 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+4496 (secure 0 0 0) 0x7fe25c0c3a80 con 0x7fe25c1408f0 2023-11-15T03:05:17.356 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:17.356 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":141,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8199,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2860780116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2860780116},{"type":"v1","addr":"172.21.15.121:6836","nonce":2860780116}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:12.149521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:05:17.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.350+0000 7fe261707700 1 -- 172.21.15.121:0/1470137303 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe23c05a510 msgr2=0x7fe23c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:17.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.350+0000 7fe261707700 1 --2- 172.21.15.121:0/1470137303 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe23c05a510 0x7fe23c05c9d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7fe244006c40 tx=0x7fe244008040).stop 2023-11-15T03:05:17.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.350+0000 7fe261707700 1 -- 172.21.15.121:0/1470137303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe25c1408f0 msgr2=0x7fe25c14fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:17.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.350+0000 7fe261707700 1 --2- 172.21.15.121:0/1470137303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe25c1408f0 0x7fe25c14fbf0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7fe248004480 tx=0x7fe24c015000).stop 2023-11-15T03:05:17.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.350+0000 7fe261707700 1 -- 172.21.15.121:0/1470137303 shutdown_connections 2023-11-15T03:05:17.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.350+0000 7fe261707700 1 --2- 172.21.15.121:0/1470137303 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe23c05a510 0x7fe23c05c9d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:17.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.350+0000 7fe261707700 1 --2- 172.21.15.121:0/1470137303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe25c1500e0 0x7fe25c0a2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:17.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.350+0000 7fe261707700 1 --2- 172.21.15.121:0/1470137303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe25c1408f0 0x7fe25c14fbf0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:17.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.350+0000 7fe261707700 1 --2- 172.21.15.121:0/1470137303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe248004690 0x7fe25c140400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:17.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.350+0000 7fe261707700 1 -- 172.21.15.121:0/1470137303 >> 172.21.15.121:0/1470137303 conn(0x7fe25c062190 msgr2=0x7fe25c0a9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:17.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.354+0000 7fe261707700 1 -- 172.21.15.121:0/1470137303 shutdown_connections 2023-11-15T03:05:17.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.354+0000 7fe261707700 1 -- 172.21.15.121:0/1470137303 wait complete. 2023-11-15T03:05:17.364 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 141 2023-11-15T03:05:17.383 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:05:17.383 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:05:17.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.534+0000 7fd69759e700 1 Processor -- start 2023-11-15T03:05:17.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.534+0000 7fd69759e700 1 -- start start 2023-11-15T03:05:17.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.538+0000 7fd69759e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd6980cabd0 0x7fd6980cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:17.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.538+0000 7fd69759e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd6980acc90 0x7fd698066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:17.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.538+0000 7fd69759e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6980ad650 con 0x7fd6980ad0f0 2023-11-15T03:05:17.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.538+0000 7fd69759e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd698065fc0 con 0x7fd6980acc90 2023-11-15T03:05:17.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.538+0000 7fd69759e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6980662a0 con 0x7fd6980cabd0 2023-11-15T03:05:17.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.538+0000 7fd695d9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd6980acc90 0x7fd698066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:17.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.538+0000 7fd695d9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd6980acc90 0x7fd698066d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39828/0 (socket says 172.21.15.121:39828) 2023-11-15T03:05:17.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.538+0000 7fd695d9b700 1 -- 172.21.15.121:0/3515098277 learned_addr learned my addr 172.21.15.121:0/3515098277 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:17.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.538+0000 7fd69659c700 1 --2- 172.21.15.121:0/3515098277 >> v2:172.21.15.190:3300/0 conn(0x7fd6980cabd0 0x7fd6980cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:17.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.538+0000 7fd695d9b700 1 -- 172.21.15.121:0/3515098277 >> v2:172.21.15.190:3300/0 conn(0x7fd6980cabd0 msgr2=0x7fd6980cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:17.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.538+0000 7fd695d9b700 1 --2- 172.21.15.121:0/3515098277 >> v2:172.21.15.190:3300/0 conn(0x7fd6980cabd0 0x7fd6980cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:17.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.538+0000 7fd695d9b700 1 -- 172.21.15.121:0/3515098277 >> v1:172.21.15.190:6789/0 conn(0x7fd6980ad0f0 legacy=0x7fd6980ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:17.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.538+0000 7fd695d9b700 1 -- 172.21.15.121:0/3515098277 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd698066580 con 0x7fd6980acc90 2023-11-15T03:05:17.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.538+0000 7fd695d9b700 1 --2- 172.21.15.121:0/3515098277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6980acc90 0x7fd698066d80 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7fd698064750 tx=0x7fd68c0090f0).ready entity=mon.0 client_cookie=5f1f664e906f6802 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:17.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.538+0000 7fd69559a700 1 -- 172.21.15.121:0/3515098277 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c00b520 con 0x7fd6980acc90 2023-11-15T03:05:17.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.538+0000 7fd69559a700 1 -- 172.21.15.121:0/3515098277 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c00b680 con 0x7fd6980acc90 2023-11-15T03:05:17.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.538+0000 7fd69559a700 1 -- 172.21.15.121:0/3515098277 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c00ba00 con 0x7fd6980acc90 2023-11-15T03:05:17.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.542+0000 7fd69759e700 1 -- 172.21.15.121:0/3515098277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6980acc90 msgr2=0x7fd698066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:17.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.542+0000 7fd69759e700 1 --2- 172.21.15.121:0/3515098277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6980acc90 0x7fd698066d80 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7fd698064750 tx=0x7fd68c0090f0).stop 2023-11-15T03:05:17.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.542+0000 7fd69759e700 1 -- 172.21.15.121:0/3515098277 shutdown_connections 2023-11-15T03:05:17.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.542+0000 7fd69759e700 1 --2- 172.21.15.121:0/3515098277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6980acc90 0x7fd698066d80 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:17.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.542+0000 7fd69759e700 1 --2- 172.21.15.121:0/3515098277 >> v2:172.21.15.190:3300/0 conn(0x7fd6980cabd0 0x7fd6980cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:17.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.542+0000 7fd69759e700 1 -- 172.21.15.121:0/3515098277 >> 172.21.15.121:0/3515098277 conn(0x7fd698062190 msgr2=0x7fd6980625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:17.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.542+0000 7fd69759e700 1 -- 172.21.15.121:0/3515098277 shutdown_connections 2023-11-15T03:05:17.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.542+0000 7fd69759e700 1 -- 172.21.15.121:0/3515098277 wait complete. 2023-11-15T03:05:17.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd69759e700 1 Processor -- start 2023-11-15T03:05:17.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd69759e700 1 -- start start 2023-11-15T03:05:17.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd69759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6980acc90 0x7fd6980c04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:17.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd69759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6980ad0f0 0x7fd698154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:17.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd69759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6980cabd0 0x7fd698167a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:17.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd69759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69814dee0 con 0x7fd6980ad0f0 2023-11-15T03:05:17.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd69759e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980c1750 con 0x7fd6980acc90 2023-11-15T03:05:17.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd69759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd698152930 con 0x7fd6980cabd0 2023-11-15T03:05:17.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd695d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6980ad0f0 0x7fd698154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:17.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd695d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6980ad0f0 0x7fd698154e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39834/0 (socket says 172.21.15.121:39834) 2023-11-15T03:05:17.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd695d9b700 1 -- 172.21.15.121:0/155751176 learned_addr learned my addr 172.21.15.121:0/155751176 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:17.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd69659c700 1 --2- 172.21.15.121:0/155751176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6980acc90 0x7fd6980c04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:17.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd696d9d700 1 --2- 172.21.15.121:0/155751176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6980cabd0 0x7fd698167a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:17.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd695d9b700 1 -- 172.21.15.121:0/155751176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6980cabd0 msgr2=0x7fd698167a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:17.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd695d9b700 1 --2- 172.21.15.121:0/155751176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6980cabd0 0x7fd698167a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:17.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd695d9b700 1 -- 172.21.15.121:0/155751176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6980acc90 msgr2=0x7fd6980c04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:17.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd695d9b700 1 --2- 172.21.15.121:0/155751176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6980acc90 0x7fd6980c04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:17.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd695d9b700 1 -- 172.21.15.121:0/155751176 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c009370 con 0x7fd6980ad0f0 2023-11-15T03:05:17.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd69659c700 1 --2- 172.21.15.121:0/155751176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6980acc90 0x7fd6980c04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:05:17.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd696d9d700 1 --2- 172.21.15.121:0/155751176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6980cabd0 0x7fd698167a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:05:17.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd695d9b700 1 --2- 172.21.15.121:0/155751176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6980ad0f0 0x7fd698154e80 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7fd68c009130 tx=0x7fd68c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:17.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.546+0000 7fd6867fc700 1 -- 172.21.15.121:0/155751176 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c00be90 con 0x7fd6980ad0f0 2023-11-15T03:05:17.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.550+0000 7fd6867fc700 1 -- 172.21.15.121:0/155751176 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c00b7f0 con 0x7fd6980ad0f0 2023-11-15T03:05:17.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.550+0000 7fd69759e700 1 -- 172.21.15.121:0/155751176 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6980c0bd0 con 0x7fd6980ad0f0 2023-11-15T03:05:17.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.550+0000 7fd6867fc700 1 -- 172.21.15.121:0/155751176 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c01c670 con 0x7fd6980ad0f0 2023-11-15T03:05:17.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.550+0000 7fd69759e700 1 -- 172.21.15.121:0/155751176 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6980c1190 con 0x7fd6980ad0f0 2023-11-15T03:05:17.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.550+0000 7fd69759e700 1 -- 172.21.15.121:0/155751176 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6980cba60 con 0x7fd6980ad0f0 2023-11-15T03:05:17.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.550+0000 7fd6867fc700 1 -- 172.21.15.121:0/155751176 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd68c01ca20 con 0x7fd6980ad0f0 2023-11-15T03:05:17.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.554+0000 7fd6867fc700 1 --2- 172.21.15.121:0/155751176 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd67805a510 0x7fd67805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:17.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.554+0000 7fd6867fc700 1 -- 172.21.15.121:0/155751176 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7126+0+0 (secure 0 0 0) 0x7fd68c076990 con 0x7fd6980ad0f0 2023-11-15T03:05:17.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.554+0000 7fd69659c700 1 --2- 172.21.15.121:0/155751176 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd67805a510 0x7fd67805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:17.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.554+0000 7fd69659c700 1 --2- 172.21.15.121:0/155751176 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd67805a510 0x7fd67805c9d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7fd688006d80 tx=0x7fd688008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:17.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.558+0000 7fd6867fc700 1 -- 172.21.15.121:0/155751176 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd6980cba60 con 0x7fd6980ad0f0 2023-11-15T03:05:17.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.750+0000 7fd69759e700 1 -- 172.21.15.121:0/155751176 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd698066aa0 con 0x7fd6980ad0f0 2023-11-15T03:05:17.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.750+0000 7fd6867fc700 1 -- 172.21.15.121:0/155751176 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+4496 (secure 0 0 0) 0x7fd698066aa0 con 0x7fd6980ad0f0 2023-11-15T03:05:17.761 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:17.761 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":141,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8199,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2860780116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2860780116},{"type":"v1","addr":"172.21.15.121:6836","nonce":2860780116}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:12.149521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:05:17.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.754+0000 7fd69759e700 1 -- 172.21.15.121:0/155751176 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd67805a510 msgr2=0x7fd67805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:17.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.754+0000 7fd69759e700 1 --2- 172.21.15.121:0/155751176 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd67805a510 0x7fd67805c9d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7fd688006d80 tx=0x7fd688008040).stop 2023-11-15T03:05:17.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.754+0000 7fd69759e700 1 -- 172.21.15.121:0/155751176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6980ad0f0 msgr2=0x7fd698154e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:17.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.754+0000 7fd69759e700 1 --2- 172.21.15.121:0/155751176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6980ad0f0 0x7fd698154e80 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7fd68c009130 tx=0x7fd68c017000).stop 2023-11-15T03:05:17.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.758+0000 7fd69759e700 1 -- 172.21.15.121:0/155751176 shutdown_connections 2023-11-15T03:05:17.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.758+0000 7fd69759e700 1 --2- 172.21.15.121:0/155751176 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd67805a510 0x7fd67805c9d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:17.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.758+0000 7fd69759e700 1 --2- 172.21.15.121:0/155751176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6980cabd0 0x7fd698167a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:17.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.758+0000 7fd69759e700 1 --2- 172.21.15.121:0/155751176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6980ad0f0 0x7fd698154e80 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:17.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.758+0000 7fd69759e700 1 --2- 172.21.15.121:0/155751176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6980acc90 0x7fd6980c04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:17.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.758+0000 7fd69759e700 1 -- 172.21.15.121:0/155751176 >> 172.21.15.121:0/155751176 conn(0x7fd698062190 msgr2=0x7fd6980ae470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:17.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.758+0000 7fd69759e700 1 -- 172.21.15.121:0/155751176 shutdown_connections 2023-11-15T03:05:17.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:17.758+0000 7fd69759e700 1 -- 172.21.15.121:0/155751176 wait complete. 2023-11-15T03:05:17.778 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 141 2023-11-15T03:05:17.785 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_inotable' led to DAMAGED state 2023-11-15T03:05:17.786 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 100.00000000' 2023-11-15T03:05:17.786 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:05:17.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:05:17.787 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:05:17.807 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:05:17.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:05:17.809 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:05:18.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.002+0000 7f2607fff700 1 Processor -- start 2023-11-15T03:05:18.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.002+0000 7f2607fff700 1 -- start start 2023-11-15T03:05:18.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.006+0000 7f2607fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26080abaa0 0x7f26080abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:18.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.006+0000 7f2607fff700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26080c2c00 con 0x7f26080bb510 2023-11-15T03:05:18.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.006+0000 7f2607fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26080657b0 con 0x7f26080a8880 2023-11-15T03:05:18.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.006+0000 7f2607fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2608065a90 con 0x7f26080abaa0 2023-11-15T03:05:18.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.006+0000 7f25fffff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f26080bb510 0x7f2608063570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:42970/0 (socket says 172.21.15.121:42970) 2023-11-15T03:05:18.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.006+0000 7f25fffff700 1 -- 172.21.15.121:0/4192647629 learned_addr learned my addr 172.21.15.121:0/4192647629 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:18.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.006+0000 7f2606ffd700 1 --2- 172.21.15.121:0/4192647629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26080abaa0 0x7f26080abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:18.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.006+0000 7f26067fc700 1 -- 172.21.15.121:0/4192647629 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981076356 0 0) 0x7f26080c2c00 con 0x7f26080bb510 2023-11-15T03:05:18.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.006+0000 7f26067fc700 1 -- 172.21.15.121:0/4192647629 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25f40033f0 con 0x7f26080bb510 2023-11-15T03:05:18.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.006+0000 7f26067fc700 1 -- 172.21.15.121:0/4192647629 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918382695 0 0) 0x7f26080657b0 con 0x7f26080a8880 2023-11-15T03:05:18.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.006+0000 7f26067fc700 1 -- 172.21.15.121:0/4192647629 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26080c2c00 con 0x7f26080a8880 2023-11-15T03:05:18.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.006+0000 7f2606ffd700 1 -- 172.21.15.121:0/4192647629 >> v1:172.21.15.190:6789/0 conn(0x7f26080a8880 legacy=0x7f2608151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:18.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.006+0000 7f2606ffd700 1 -- 172.21.15.121:0/4192647629 >> v1:172.21.15.121:6789/0 conn(0x7f26080bb510 legacy=0x7f2608063570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:18.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.006+0000 7f2606ffd700 1 -- 172.21.15.121:0/4192647629 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26080c2890 con 0x7f26080abaa0 2023-11-15T03:05:18.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.006+0000 7f2606ffd700 1 --2- 172.21.15.121:0/4192647629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26080abaa0 0x7f26080abe80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f25f8009790 tx=0x7f25f8009060).ready entity=mon.2 client_cookie=5bae2aa1202a5bd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:18.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.010+0000 7f26067fc700 1 -- 172.21.15.121:0/4192647629 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f800f510 con 0x7f26080abaa0 2023-11-15T03:05:18.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.010+0000 7f26067fc700 1 -- 172.21.15.121:0/4192647629 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f800f670 con 0x7f26080abaa0 2023-11-15T03:05:18.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.010+0000 7f26067fc700 1 -- 172.21.15.121:0/4192647629 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f800f9f0 con 0x7f26080abaa0 2023-11-15T03:05:18.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.010+0000 7f2607fff700 1 -- 172.21.15.121:0/4192647629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26080abaa0 msgr2=0x7f26080abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:18.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.010+0000 7f2607fff700 1 --2- 172.21.15.121:0/4192647629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26080abaa0 0x7f26080abe80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f25f8009790 tx=0x7f25f8009060).stop 2023-11-15T03:05:18.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.010+0000 7f2607fff700 1 -- 172.21.15.121:0/4192647629 shutdown_connections 2023-11-15T03:05:18.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.010+0000 7f2607fff700 1 --2- 172.21.15.121:0/4192647629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26080abaa0 0x7f26080abe80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:18.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.010+0000 7f2607fff700 1 -- 172.21.15.121:0/4192647629 >> 172.21.15.121:0/4192647629 conn(0x7f2608067a60 msgr2=0x7f2608067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:18.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.010+0000 7f2607fff700 1 -- 172.21.15.121:0/4192647629 shutdown_connections 2023-11-15T03:05:18.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.010+0000 7f2607fff700 1 -- 172.21.15.121:0/4192647629 wait complete. 2023-11-15T03:05:18.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.010+0000 7f2607fff700 1 Processor -- start 2023-11-15T03:05:18.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.010+0000 7f2607fff700 1 -- start start 2023-11-15T03:05:18.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f2607fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26080a8880 0x7f26081399b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:18.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f2607fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26080abaa0 0x7f2608153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:18.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f2607fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26080bb510 0x7f2608154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:18.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f2607fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26081547f0 con 0x7f26080bb510 2023-11-15T03:05:18.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f2607fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2608154ad0 con 0x7f26080abaa0 2023-11-15T03:05:18.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f2607fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2608154db0 con 0x7f26080a8880 2023-11-15T03:05:18.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f26077fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26080bb510 0x7f2608154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:18.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f26077fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26080bb510 0x7f2608154200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39850/0 (socket says 172.21.15.121:39850) 2023-11-15T03:05:18.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f26077fe700 1 -- 172.21.15.121:0/1059833161 learned_addr learned my addr 172.21.15.121:0/1059833161 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:18.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f2606ffd700 1 --2- 172.21.15.121:0/1059833161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26080a8880 0x7f26081399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:18.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f25fffff700 1 --2- 172.21.15.121:0/1059833161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26080abaa0 0x7f2608153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:18.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f26077fe700 1 -- 172.21.15.121:0/1059833161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26080a8880 msgr2=0x7f26081399b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:18.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f26077fe700 1 --2- 172.21.15.121:0/1059833161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26080a8880 0x7f26081399b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:18.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f26077fe700 1 -- 172.21.15.121:0/1059833161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26080abaa0 msgr2=0x7f2608153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:18.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f26077fe700 1 --2- 172.21.15.121:0/1059833161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26080abaa0 0x7f2608153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:18.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f26077fe700 1 -- 172.21.15.121:0/1059833161 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f80092a0 con 0x7f26080bb510 2023-11-15T03:05:18.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f26077fe700 1 --2- 172.21.15.121:0/1059833161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26080bb510 0x7f2608154200 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7f260000c7c0 tx=0x7f260000cad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:18.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f25feffd700 1 -- 172.21.15.121:0/1059833161 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600002020 con 0x7f26080bb510 2023-11-15T03:05:18.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f25feffd700 1 -- 172.21.15.121:0/1059833161 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2600002180 con 0x7f26080bb510 2023-11-15T03:05:18.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f25feffd700 1 -- 172.21.15.121:0/1059833161 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600001590 con 0x7f26080bb510 2023-11-15T03:05:18.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f2607fff700 1 -- 172.21.15.121:0/1059833161 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2608155090 con 0x7f26080bb510 2023-11-15T03:05:18.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.014+0000 7f2607fff700 1 -- 172.21.15.121:0/1059833161 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f260814c4c0 con 0x7f26080bb510 2023-11-15T03:05:18.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.018+0000 7f2607fff700 1 -- 172.21.15.121:0/1059833161 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260806bcf0 con 0x7f26080bb510 2023-11-15T03:05:18.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.018+0000 7f25feffd700 1 -- 172.21.15.121:0/1059833161 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2600001980 con 0x7f26080bb510 2023-11-15T03:05:18.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.022+0000 7f25feffd700 1 --2- 172.21.15.121:0/1059833161 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25e805a510 0x7f25e805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:18.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.022+0000 7f25feffd700 1 -- 172.21.15.121:0/1059833161 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7126+0+0 (secure 0 0 0) 0x7f2600074260 con 0x7f26080bb510 2023-11-15T03:05:18.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.022+0000 7f2606ffd700 1 --2- 172.21.15.121:0/1059833161 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25e805a510 0x7f25e805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:18.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.022+0000 7f2606ffd700 1 --2- 172.21.15.121:0/1059833161 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25e805a510 0x7f25e805c9d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f25f8009fd0 tx=0x7f25f8017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:18.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.030+0000 7f25feffd700 1 -- 172.21.15.121:0/1059833161 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2600047530 con 0x7f26080bb510 2023-11-15T03:05:18.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:18.222+0000 7f2607fff700 1 -- 172.21.15.121:0/1059833161 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f26080c2540 con 0x7f26080bb510 2023-11-15T03:05:19.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.194+0000 7f25feffd700 1 -- 172.21.15.121:0/1059833161 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v142) v1 ==== 161+0+0 (secure 0 0 0) 0x7f260004ab50 con 0x7f26080bb510 2023-11-15T03:05:19.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.198+0000 7f2607fff700 1 -- 172.21.15.121:0/1059833161 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25e805a510 msgr2=0x7f25e805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:19.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.198+0000 7f2607fff700 1 --2- 172.21.15.121:0/1059833161 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25e805a510 0x7f25e805c9d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f25f8009fd0 tx=0x7f25f8017000).stop 2023-11-15T03:05:19.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.198+0000 7f2607fff700 1 -- 172.21.15.121:0/1059833161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26080bb510 msgr2=0x7f2608154200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:19.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.198+0000 7f2607fff700 1 --2- 172.21.15.121:0/1059833161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26080bb510 0x7f2608154200 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7f260000c7c0 tx=0x7f260000cad0).stop 2023-11-15T03:05:19.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.198+0000 7f2607fff700 1 -- 172.21.15.121:0/1059833161 shutdown_connections 2023-11-15T03:05:19.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.198+0000 7f2607fff700 1 --2- 172.21.15.121:0/1059833161 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25e805a510 0x7f25e805c9d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:19.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.198+0000 7f2607fff700 1 --2- 172.21.15.121:0/1059833161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26080bb510 0x7f2608154200 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:19.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.198+0000 7f2607fff700 1 --2- 172.21.15.121:0/1059833161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26080abaa0 0x7f2608153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:19.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.198+0000 7f2607fff700 1 --2- 172.21.15.121:0/1059833161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26080a8880 0x7f26081399b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:19.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.198+0000 7f2607fff700 1 -- 172.21.15.121:0/1059833161 >> 172.21.15.121:0/1059833161 conn(0x7f2608067a60 msgr2=0x7f2608066370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:19.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.198+0000 7f2607fff700 1 -- 172.21.15.121:0/1059833161 shutdown_connections 2023-11-15T03:05:19.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.198+0000 7f2607fff700 1 -- 172.21.15.121:0/1059833161 wait complete. 2023-11-15T03:05:19.212 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:05:19.232 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:05:19.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.394+0000 7f2ccd969700 1 Processor -- start 2023-11-15T03:05:19.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.398+0000 7f2ccd969700 1 -- start start 2023-11-15T03:05:19.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.398+0000 7f2ccd969700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cc80c2580 0x7f2cc80c2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:19.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.398+0000 7f2ccd969700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cc8067ee0 con 0x7f2cc80c9170 2023-11-15T03:05:19.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.398+0000 7f2ccd969700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cc80cbf00 con 0x7f2cc80cb1f0 2023-11-15T03:05:19.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.398+0000 7f2ccd969700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc80c6e80 con 0x7f2cc80c2580 2023-11-15T03:05:19.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.398+0000 7f2cc67fc700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f2cc80c9170 0x7f2cc80c6770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:42986/0 (socket says 172.21.15.121:42986) 2023-11-15T03:05:19.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.398+0000 7f2cc67fc700 1 -- 172.21.15.121:0/3407489531 learned_addr learned my addr 172.21.15.121:0/3407489531 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:19.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.398+0000 7f2cc6ffd700 1 --2- 172.21.15.121:0/3407489531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cc80c2580 0x7f2cc80c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:19.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.402+0000 7f2cc6ffd700 1 -- 172.21.15.121:0/3407489531 >> v1:172.21.15.190:6789/0 conn(0x7f2cc80cb1f0 legacy=0x7f2cc8068370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:19.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.402+0000 7f2cc5ffb700 1 -- 172.21.15.121:0/3407489531 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3488380402 0 0) 0x7f2cc80cbf00 con 0x7f2cc80cb1f0 2023-11-15T03:05:19.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.402+0000 7f2cc6ffd700 1 -- 172.21.15.121:0/3407489531 >> v1:172.21.15.121:6789/0 conn(0x7f2cc80c9170 legacy=0x7f2cc80c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:19.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.402+0000 7f2cc6ffd700 1 -- 172.21.15.121:0/3407489531 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc80c95d0 con 0x7f2cc80c2580 2023-11-15T03:05:19.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.402+0000 7f2cc6ffd700 1 --2- 172.21.15.121:0/3407489531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cc80c2580 0x7f2cc80c2960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f2cb8009760 tx=0x7f2cb8006f40).ready entity=mon.2 client_cookie=166af15d346cf62d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:19.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.402+0000 7f2cc5ffb700 1 -- 172.21.15.121:0/3407489531 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8005880 con 0x7f2cc80c2580 2023-11-15T03:05:19.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.402+0000 7f2cc5ffb700 1 -- 172.21.15.121:0/3407489531 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb80059e0 con 0x7f2cc80c2580 2023-11-15T03:05:19.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.402+0000 7f2cc5ffb700 1 -- 172.21.15.121:0/3407489531 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8005d60 con 0x7f2cc80c2580 2023-11-15T03:05:19.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.402+0000 7f2ccd969700 1 -- 172.21.15.121:0/3407489531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cc80c2580 msgr2=0x7f2cc80c2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:19.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.402+0000 7f2ccd969700 1 --2- 172.21.15.121:0/3407489531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cc80c2580 0x7f2cc80c2960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f2cb8009760 tx=0x7f2cb8006f40).stop 2023-11-15T03:05:19.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.402+0000 7f2ccd969700 1 -- 172.21.15.121:0/3407489531 shutdown_connections 2023-11-15T03:05:19.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.402+0000 7f2ccd969700 1 --2- 172.21.15.121:0/3407489531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cc80c2580 0x7f2cc80c2960 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:19.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.402+0000 7f2ccd969700 1 -- 172.21.15.121:0/3407489531 >> 172.21.15.121:0/3407489531 conn(0x7f2cc80957e0 msgr2=0x7f2cc8069820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:19.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.402+0000 7f2ccd969700 1 -- 172.21.15.121:0/3407489531 shutdown_connections 2023-11-15T03:05:19.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.402+0000 7f2ccd969700 1 -- 172.21.15.121:0/3407489531 wait complete. 2023-11-15T03:05:19.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2ccd969700 1 Processor -- start 2023-11-15T03:05:19.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2ccd969700 1 -- start start 2023-11-15T03:05:19.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2ccd969700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cc80c2580 0x7f2cc80ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:19.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2ccd969700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cc80c9170 0x7f2cc80acad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:19.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2ccd969700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cc80cb1f0 0x7f2cc8150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:19.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2ccd969700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc80acfc0 con 0x7f2cc80c2580 2023-11-15T03:05:19.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2ccd969700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc80ae970 con 0x7f2cc80cb1f0 2023-11-15T03:05:19.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2ccd969700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc8155f10 con 0x7f2cc80c9170 2023-11-15T03:05:19.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2cc6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cc80c2580 0x7f2cc80ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:19.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2cc6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cc80c2580 0x7f2cc80ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39852/0 (socket says 172.21.15.121:39852) 2023-11-15T03:05:19.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2cc67fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cc80c9170 0x7f2cc80acad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:19.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2cc6ffd700 1 -- 172.21.15.121:0/3609150569 learned_addr learned my addr 172.21.15.121:0/3609150569 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:19.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2cc7fff700 1 --2- 172.21.15.121:0/3609150569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cc80cb1f0 0x7f2cc8150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:19.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2cc6ffd700 1 -- 172.21.15.121:0/3609150569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cc80c9170 msgr2=0x7f2cc80acad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:19.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2cc6ffd700 1 --2- 172.21.15.121:0/3609150569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cc80c9170 0x7f2cc80acad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:19.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2cc6ffd700 1 -- 172.21.15.121:0/3609150569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cc80cb1f0 msgr2=0x7f2cc8150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:19.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2cc6ffd700 1 --2- 172.21.15.121:0/3609150569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cc80cb1f0 0x7f2cc8150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:19.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2cc6ffd700 1 -- 172.21.15.121:0/3609150569 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb8009250 con 0x7f2cc80c2580 2023-11-15T03:05:19.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2cc7fff700 1 --2- 172.21.15.121:0/3609150569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cc80cb1f0 0x7f2cc8150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:19.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.406+0000 7f2cc6ffd700 1 --2- 172.21.15.121:0/3609150569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cc80c2580 0x7f2cc80ae3b0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7f2cb8009eb0 tx=0x7f2cb8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:19.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.410+0000 7f2caeffd700 1 -- 172.21.15.121:0/3609150569 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb800f7f0 con 0x7f2cc80c2580 2023-11-15T03:05:19.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.410+0000 7f2caeffd700 1 -- 172.21.15.121:0/3609150569 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb800f950 con 0x7f2cc80c2580 2023-11-15T03:05:19.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.410+0000 7f2ccd969700 1 -- 172.21.15.121:0/3609150569 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc81555e0 con 0x7f2cc80c2580 2023-11-15T03:05:19.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.410+0000 7f2caeffd700 1 -- 172.21.15.121:0/3609150569 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb800fd10 con 0x7f2cc80c2580 2023-11-15T03:05:19.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.410+0000 7f2ccd969700 1 -- 172.21.15.121:0/3609150569 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc8155ba0 con 0x7f2cc80c2580 2023-11-15T03:05:19.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.410+0000 7f2ccd969700 1 -- 172.21.15.121:0/3609150569 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc8047060 con 0x7f2cc80c2580 2023-11-15T03:05:19.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.410+0000 7f2caeffd700 1 -- 172.21.15.121:0/3609150569 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2cb8011650 con 0x7f2cc80c2580 2023-11-15T03:05:19.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.414+0000 7f2caeffd700 1 --2- 172.21.15.121:0/3609150569 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2cb405a510 0x7f2cb405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:19.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.414+0000 7f2caeffd700 1 -- 172.21.15.121:0/3609150569 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7126+0+0 (secure 0 0 0) 0x7f2cb8075a80 con 0x7f2cc80c2580 2023-11-15T03:05:19.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.414+0000 7f2cc67fc700 1 --2- 172.21.15.121:0/3609150569 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2cb405a510 0x7f2cb405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:19.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.414+0000 7f2cc67fc700 1 --2- 172.21.15.121:0/3609150569 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2cb405a510 0x7f2cb405c9d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f2cc80689b0 tx=0x7f2cbc009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:19.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.422+0000 7f2caeffd700 1 -- 172.21.15.121:0/3609150569 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2cb8048d50 con 0x7f2cc80c2580 2023-11-15T03:05:19.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:19.642+0000 7f2ccd969700 1 -- 172.21.15.121:0/3609150569 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f2cc806a0c0 con 0x7f2cc80c2580 2023-11-15T03:05:20.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.214+0000 7f2caeffd700 1 -- 172.21.15.121:0/3609150569 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v143) v1 ==== 101+0+0 (secure 0 0 0) 0x7f2cb8022070 con 0x7f2cc80c2580 2023-11-15T03:05:20.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.218+0000 7f2ccd969700 1 -- 172.21.15.121:0/3609150569 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2cb405a510 msgr2=0x7f2cb405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.218+0000 7f2ccd969700 1 --2- 172.21.15.121:0/3609150569 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2cb405a510 0x7f2cb405c9d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f2cc80689b0 tx=0x7f2cbc009040).stop 2023-11-15T03:05:20.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.218+0000 7f2ccd969700 1 -- 172.21.15.121:0/3609150569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cc80c2580 msgr2=0x7f2cc80ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.218+0000 7f2ccd969700 1 --2- 172.21.15.121:0/3609150569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cc80c2580 0x7f2cc80ae3b0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7f2cb8009eb0 tx=0x7f2cb8016000).stop 2023-11-15T03:05:20.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.218+0000 7f2ccd969700 1 -- 172.21.15.121:0/3609150569 shutdown_connections 2023-11-15T03:05:20.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.218+0000 7f2ccd969700 1 --2- 172.21.15.121:0/3609150569 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2cb405a510 0x7f2cb405c9d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.218+0000 7f2ccd969700 1 --2- 172.21.15.121:0/3609150569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cc80cb1f0 0x7f2cc8150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.218+0000 7f2ccd969700 1 --2- 172.21.15.121:0/3609150569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cc80c9170 0x7f2cc80acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.218+0000 7f2ccd969700 1 --2- 172.21.15.121:0/3609150569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cc80c2580 0x7f2cc80ae3b0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.218+0000 7f2ccd969700 1 -- 172.21.15.121:0/3609150569 >> 172.21.15.121:0/3609150569 conn(0x7f2cc80957e0 msgr2=0x7f2cc8153ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:20.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.218+0000 7f2ccd969700 1 -- 172.21.15.121:0/3609150569 shutdown_connections 2023-11-15T03:05:20.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.218+0000 7f2ccd969700 1 -- 172.21.15.121:0/3609150569 wait complete. 2023-11-15T03:05:20.231 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:05:20.247 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:05:20.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.294+0000 7fe8b1cfbd00 1 Processor -- start 2023-11-15T03:05:20.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8b1cfbd00 1 -- start start 2023-11-15T03:05:20.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8b1cfbd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f50fe00bb0 0x55f50fe2fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8b1cfbd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f50fe4ab00 0x55f50fee5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8b1cfbd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f50fedd650 0x55f50fee6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8b1cfbd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f50fd79220 con 0x55f50fe4ab00 2023-11-15T03:05:20.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8b1cfbd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f50fe30860 con 0x55f50fe00bb0 2023-11-15T03:05:20.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8b1cfbd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f50fed3310 con 0x55f50fedd650 2023-11-15T03:05:20.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8a6e44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f50fe4ab00 0x55f50fee5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8a6e44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f50fe4ab00 0x55f50fee5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39878/0 (socket says 172.21.15.121:39878) 2023-11-15T03:05:20.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8a6e44700 1 -- 172.21.15.121:0/1387922725 learned_addr learned my addr 172.21.15.121:0/1387922725 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:20.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8af645700 1 --2- 172.21.15.121:0/1387922725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f50fe00bb0 0x55f50fe2fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8afe46700 1 --2- 172.21.15.121:0/1387922725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f50fedd650 0x55f50fee6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8a6e44700 1 -- 172.21.15.121:0/1387922725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f50fedd650 msgr2=0x55f50fee6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:20.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8a6e44700 1 --2- 172.21.15.121:0/1387922725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f50fedd650 0x55f50fee6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8a6e44700 1 -- 172.21.15.121:0/1387922725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f50fe00bb0 msgr2=0x55f50fe2fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:20.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8a6e44700 1 --2- 172.21.15.121:0/1387922725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f50fe00bb0 0x55f50fe2fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8a6e44700 1 -- 172.21.15.121:0/1387922725 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a8005db0 con 0x55f50fe4ab00 2023-11-15T03:05:20.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8a6e44700 1 --2- 172.21.15.121:0/1387922725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f50fe4ab00 0x55f50fee5f60 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7fe898009950 tx=0x7fe89800c040).ready entity=mon.0 client_cookie=dfe802337234738c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:20.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8ae643700 1 -- 172.21.15.121:0/1387922725 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89800f040 con 0x55f50fe4ab00 2023-11-15T03:05:20.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8ae643700 1 -- 172.21.15.121:0/1387922725 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe898008860 con 0x55f50fe4ab00 2023-11-15T03:05:20.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8ae643700 1 -- 172.21.15.121:0/1387922725 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898008be0 con 0x55f50fe4ab00 2023-11-15T03:05:20.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/1387922725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f50fe4ab00 msgr2=0x55f50fee5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.298+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/1387922725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f50fe4ab00 0x55f50fee5f60 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7fe898009950 tx=0x7fe89800c040).stop 2023-11-15T03:05:20.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/1387922725 shutdown_connections 2023-11-15T03:05:20.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/1387922725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f50fedd650 0x55f50fee6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/1387922725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f50fe4ab00 0x55f50fee5f60 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/1387922725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f50fe00bb0 0x55f50fe2fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/1387922725 >> 172.21.15.121:0/1387922725 conn(0x55f50fe50080 msgr2=0x55f50fe44fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:20.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/1387922725 shutdown_connections 2023-11-15T03:05:20.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/1387922725 wait complete. 2023-11-15T03:05:20.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8b1cfbd00 1 Processor -- start 2023-11-15T03:05:20.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8b1cfbd00 1 -- start start 2023-11-15T03:05:20.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8b1cfbd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f50fe00bb0 0x55f50fecccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8b1cfbd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f50fe4ab00 0x55f50fec8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8b1cfbd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f50fedd650 0x55f50feea1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8b1cfbd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f50fed14e0 con 0x55f50fe4ab00 2023-11-15T03:05:20.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8b1cfbd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f50fed1780 con 0x55f50fe00bb0 2023-11-15T03:05:20.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8b1cfbd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f50fed1a60 con 0x55f50fedd650 2023-11-15T03:05:20.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8a6e44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f50fe4ab00 0x55f50fec8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8a6e44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f50fe4ab00 0x55f50fec8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39884/0 (socket says 172.21.15.121:39884) 2023-11-15T03:05:20.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 learned_addr learned my addr 172.21.15.121:0/3256006648 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:20.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8afe46700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f50fedd650 0x55f50feea1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.302+0000 7fe8af645700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f50fe00bb0 0x55f50fecccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.306+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f50fedd650 msgr2=0x55f50feea1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.306+0000 7fe8a6e44700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f50fedd650 0x55f50feea1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.306+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f50fe00bb0 msgr2=0x55f50fecccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.306+0000 7fe8a6e44700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f50fe00bb0 0x55f50fecccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.306+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe898009580 con 0x55f50fe4ab00 2023-11-15T03:05:20.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.306+0000 7fe8a6e44700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f50fe4ab00 0x55f50fec8250 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x55f50fe30e90 tx=0x7fe898018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:20.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.306+0000 7fe8ace40700 1 -- 172.21.15.121:0/3256006648 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89800f040 con 0x55f50fe4ab00 2023-11-15T03:05:20.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.306+0000 7fe8ace40700 1 -- 172.21.15.121:0/3256006648 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe898014e40 con 0x55f50fe4ab00 2023-11-15T03:05:20.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.306+0000 7fe8ace40700 1 -- 172.21.15.121:0/3256006648 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89801e6b0 con 0x55f50fe4ab00 2023-11-15T03:05:20.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.306+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f50fda2770 con 0x55f50fe4ab00 2023-11-15T03:05:20.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.306+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f50fd71a70 con 0x55f50fe4ab00 2023-11-15T03:05:20.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.306+0000 7fe8ace40700 1 -- 172.21.15.121:0/3256006648 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe89801eae0 con 0x55f50fe4ab00 2023-11-15T03:05:20.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.310+0000 7fe8ace40700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe890058ff0 0x7fe89005b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.310+0000 7fe8ace40700 1 -- 172.21.15.121:0/3256006648 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7126+0+0 (secure 0 0 0) 0x7fe898075290 con 0x55f50fe4ab00 2023-11-15T03:05:20.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.310+0000 7fe8af645700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe890058ff0 0x7fe89005b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.340 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:05:20.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.310+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55f50fec8740 0x55f50fedbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.310+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fe320f0 con 0x55f50fec8740 2023-11-15T03:05:20.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.310+0000 7fe8afe46700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55f50fec8740 0x55f50fedbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.310+0000 7fe8af645700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe890058ff0 0x7fe89005b4b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fe8a0002a80 tx=0x7fe8a0008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:20.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.310+0000 7fe8afe46700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55f50fec8740 0x55f50fedbcc0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:20.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.314+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v104'113 uv106 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80054f0 con 0x55f50fec8740 2023-11-15T03:05:20.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.314+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50fed2540 con 0x55f50fec8740 2023-11-15T03:05:20.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.314+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v104'114 uv114 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50fec8740 2023-11-15T03:05:20.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.314+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fed2920 con 0x55f50fec8740 2023-11-15T03:05:20.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.318+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v104'115 uv115 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50fec8740 2023-11-15T03:05:20.344 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:05:20.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.318+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fef7760 con 0x55f50fec8740 2023-11-15T03:05:20.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.322+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v104'116 uv116 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50fec8740 2023-11-15T03:05:20.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.322+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fef7b10 con 0x55f50fec8740 2023-11-15T03:05:20.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.322+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v104'117 uv117 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50fec8740 2023-11-15T03:05:20.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.322+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fef6750 con 0x55f50fec8740 2023-11-15T03:05:20.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.326+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v104'118 uv118 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50fec8740 2023-11-15T03:05:20.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.326+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fef6b50 con 0x55f50fec8740 2023-11-15T03:05:20.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.326+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v104'119 uv112 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50fec8740 2023-11-15T03:05:20.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.326+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50fef6f50 con 0x55f50fec8740 2023-11-15T03:05:20.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.330+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v104'120 uv120 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50fec8740 2023-11-15T03:05:20.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.330+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fef6f50 con 0x55f50fec8740 2023-11-15T03:05:20.351 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:05:20.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.330+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v104'121 uv121 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50fec8740 2023-11-15T03:05:20.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.334+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fef6f00 con 0x55f50fec8740 2023-11-15T03:05:20.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.334+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v104'122 uv122 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50fec8740 2023-11-15T03:05:20.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.334+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fef6f00 con 0x55f50fec8740 2023-11-15T03:05:20.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.338+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v104'123 uv123 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50fec8740 2023-11-15T03:05:20.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.338+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fef6f00 con 0x55f50fec8740 2023-11-15T03:05:20.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.342+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v104'124 uv124 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50fec8740 2023-11-15T03:05:20.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.342+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55f50fef7300 0x55f50fefa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.342+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fef6f00 con 0x55f50fef7300 2023-11-15T03:05:20.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.342+0000 7fe8af645700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55f50fef7300 0x55f50fefa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.342+0000 7fe8af645700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55f50fef7300 0x55f50fefa3e0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:20.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.346+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v104'176 uv163 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.346+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50fefa8d0 con 0x55f50fef7300 2023-11-15T03:05:20.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.354+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v104'177 uv177 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.354+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fefac80 con 0x55f50fef7300 2023-11-15T03:05:20.367 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:05:20.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.354+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v104'178 uv178 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.354+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fefc040 con 0x55f50fef7300 2023-11-15T03:05:20.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.358+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v104'179 uv179 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.358+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fefc3f0 con 0x55f50fef7300 2023-11-15T03:05:20.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.362+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v104'180 uv180 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.362+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fefb030 con 0x55f50fef7300 2023-11-15T03:05:20.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.366+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v104'181 uv181 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.366+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fefbb90 con 0x55f50fef7300 2023-11-15T03:05:20.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.366+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v104'182 uv182 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.366+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fefc7a0 con 0x55f50fef7300 2023-11-15T03:05:20.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.374+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v104'183 uv169 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.374+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50fefc7a0 con 0x55f50fef7300 2023-11-15T03:05:20.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.374+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v104'184 uv184 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.374+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fefc7a0 con 0x55f50fef7300 2023-11-15T03:05:20.387 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:05:20.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.378+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v104'185 uv185 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.378+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fefd7b0 con 0x55f50fef7300 2023-11-15T03:05:20.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.382+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v104'186 uv186 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.382+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fefd7b0 con 0x55f50fef7300 2023-11-15T03:05:20.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.382+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v104'187 uv187 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.386+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fefc7a0 con 0x55f50fef7300 2023-11-15T03:05:20.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.386+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v104'188 uv188 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.386+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fefc7a0 con 0x55f50fef7300 2023-11-15T03:05:20.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.390+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v104'189 uv175 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.390+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50fefc7a0 con 0x55f50fef7300 2023-11-15T03:05:20.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.394+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v104'190 uv190 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.394+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fefc7a0 con 0x55f50fef7300 2023-11-15T03:05:20.404 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:05:20.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.394+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v104'191 uv191 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.394+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fefd7b0 con 0x55f50fef7300 2023-11-15T03:05:20.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.398+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v104'192 uv192 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.398+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fefd7b0 con 0x55f50fef7300 2023-11-15T03:05:20.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.398+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v104'193 uv193 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.402+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fefc7a0 con 0x55f50fef7300 2023-11-15T03:05:20.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.402+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v104'194 uv194 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.402+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55f50fefccb0 0x55f50feff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.402+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50feff590 con 0x55f50fefccb0 2023-11-15T03:05:20.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.402+0000 7fe8a6e44700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55f50fefccb0 0x55f50feff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.406+0000 7fe8a6e44700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55f50fefccb0 0x55f50feff0a0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:20.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.406+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v104'97 uv91 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50fefccb0 2023-11-15T03:05:20.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.406+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50feffa80 con 0x55f50fefccb0 2023-11-15T03:05:20.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.410+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v104'98 uv98 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50fefccb0 2023-11-15T03:05:20.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.410+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50fefff40 con 0x55f50fefccb0 2023-11-15T03:05:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.414+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v104'99 uv99 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50fefccb0 2023-11-15T03:05:20.423 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:05:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.414+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff00830 con 0x55f50fefccb0 2023-11-15T03:05:20.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.414+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v104'100 uv100 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50fefccb0 2023-11-15T03:05:20.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.414+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff00be0 con 0x55f50fefccb0 2023-11-15T03:05:20.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.418+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v104'101 uv101 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50fefccb0 2023-11-15T03:05:20.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.418+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff00fe0 con 0x55f50fefccb0 2023-11-15T03:05:20.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.422+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v104'102 uv96 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50fefccb0 2023-11-15T03:05:20.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.422+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50ff01390 con 0x55f50fefccb0 2023-11-15T03:05:20.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.426+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v104'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50fefccb0 2023-11-15T03:05:20.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.426+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff01740 con 0x55f50fefccb0 2023-11-15T03:05:20.436 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:05:20.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.426+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v104'104 uv104 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50fefccb0 2023-11-15T03:05:20.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.426+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff01740 con 0x55f50fefccb0 2023-11-15T03:05:20.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.430+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v104'105 uv105 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50fefccb0 2023-11-15T03:05:20.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.430+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff01b40 con 0x55f50fefccb0 2023-11-15T03:05:20.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.434+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v104'106 uv106 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50fefccb0 2023-11-15T03:05:20.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.434+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f50ff02050 0x55f50ff04440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.434+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff04930 con 0x55f50ff02050 2023-11-15T03:05:20.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.434+0000 7fe8afe46700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f50ff02050 0x55f50ff04440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.434+0000 7fe8afe46700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f50ff02050 0x55f50ff04440 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:20.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.438+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v104'122 uv115 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.438+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50ff04930 con 0x55f50ff02050 2023-11-15T03:05:20.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.442+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v104'123 uv123 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.442+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff04930 con 0x55f50ff02050 2023-11-15T03:05:20.452 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:05:20.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.442+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v104'124 uv124 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.442+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff08dc0 con 0x55f50ff02050 2023-11-15T03:05:20.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.446+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v104'125 uv125 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.446+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff08dc0 con 0x55f50ff02050 2023-11-15T03:05:20.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.450+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v104'126 uv126 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.450+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff04d30 con 0x55f50ff02050 2023-11-15T03:05:20.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.454+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v104'127 uv121 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.454+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50ff04d30 con 0x55f50ff02050 2023-11-15T03:05:20.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.458+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v104'128 uv128 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.458+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff04d30 con 0x55f50ff02050 2023-11-15T03:05:20.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.458+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v104'129 uv129 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.469 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:05:20.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.458+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff05d40 con 0x55f50ff02050 2023-11-15T03:05:20.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.462+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v104'130 uv130 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.462+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff05d40 con 0x55f50ff02050 2023-11-15T03:05:20.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.466+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v104'131 uv131 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.466+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff04d30 con 0x55f50ff02050 2023-11-15T03:05:20.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.470+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v104'132 uv132 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.470+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55f50ff05240 0x55f50ff07630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.470+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff07b60 con 0x55f50ff05240 2023-11-15T03:05:20.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.470+0000 7fe8af645700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55f50ff05240 0x55f50ff07630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.470+0000 7fe8af645700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55f50ff05240 0x55f50ff07630 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:20.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.474+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v104'57 uv56 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50ff05240 2023-11-15T03:05:20.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.474+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50ff07f40 con 0x55f50ff05240 2023-11-15T03:05:20.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.474+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v104'58 uv58 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50ff05240 2023-11-15T03:05:20.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.474+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff07f40 con 0x55f50ff05240 2023-11-15T03:05:20.487 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:05:20.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.478+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v104'59 uv59 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50ff05240 2023-11-15T03:05:20.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.478+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff08f50 con 0x55f50ff05240 2023-11-15T03:05:20.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.482+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v104'60 uv60 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50ff05240 2023-11-15T03:05:20.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.482+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff08f50 con 0x55f50ff05240 2023-11-15T03:05:20.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.486+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v104'61 uv61 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50ff05240 2023-11-15T03:05:20.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.486+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff07f40 con 0x55f50ff05240 2023-11-15T03:05:20.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.486+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v104'62 uv62 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50ff05240 2023-11-15T03:05:20.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.486+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55f50ff085c0 0x55f50ff0a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.486+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b0d0 con 0x55f50ff085c0 2023-11-15T03:05:20.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.490+0000 7fe8a6e44700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55f50ff085c0 0x55f50ff0a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.490+0000 7fe8a6e44700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55f50ff085c0 0x55f50ff0a9b0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:20.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.490+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v104'43 uv42 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.494+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50ff085c0 2023-11-15T03:05:20.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.494+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v104'44 uv44 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.494+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50ff085c0 2023-11-15T03:05:20.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.498+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v104'45 uv45 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.507 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:05:20.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.498+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50ff085c0 2023-11-15T03:05:20.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.502+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v104'46 uv46 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.502+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50ff085c0 2023-11-15T03:05:20.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.502+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v104'57 uv56 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.502+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50ff085c0 2023-11-15T03:05:20.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.506+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v104'58 uv58 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.506+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50ff085c0 2023-11-15T03:05:20.518 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:05:20.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.510+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v104'59 uv59 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.510+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0c4c0 con 0x55f50ff085c0 2023-11-15T03:05:20.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.510+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v104'60 uv60 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.510+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0c4c0 con 0x55f50ff085c0 2023-11-15T03:05:20.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.514+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v104'61 uv61 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.514+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50ff085c0 2023-11-15T03:05:20.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.518+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v104'62 uv62 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.518+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50fef7300 2023-11-15T03:05:20.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.522+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v104'105 uv99 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.522+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50fef7300 2023-11-15T03:05:20.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.526+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v104'106 uv106 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.526+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50fef7300 2023-11-15T03:05:20.536 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:05:20.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.526+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v104'107 uv107 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.526+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0c4c0 con 0x55f50fef7300 2023-11-15T03:05:20.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.530+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v104'108 uv108 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.530+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0c4c0 con 0x55f50fef7300 2023-11-15T03:05:20.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.534+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v104'109 uv109 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.534+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50fef7300 2023-11-15T03:05:20.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.538+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v104'110 uv110 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.538+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50fef7300 2023-11-15T03:05:20.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.542+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v104'111 uv104 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.542+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50fef7300 2023-11-15T03:05:20.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.542+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v104'112 uv112 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.542+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50fef7300 2023-11-15T03:05:20.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.546+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v104'113 uv113 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.556 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:05:20.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.546+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50fef7300 2023-11-15T03:05:20.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.550+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v104'114 uv114 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.550+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50fef7300 2023-11-15T03:05:20.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.554+0000 7fe8af645700 1 -- 172.21.15.121:0/3256006648 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v104'115 uv115 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a0010a30 con 0x55f50fef7300 2023-11-15T03:05:20.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.554+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50fefccb0 2023-11-15T03:05:20.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.554+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v104'46 uv45 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50fefccb0 2023-11-15T03:05:20.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.554+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50fefccb0 2023-11-15T03:05:20.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.558+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v104'47 uv47 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50fefccb0 2023-11-15T03:05:20.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.558+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50fefccb0 2023-11-15T03:05:20.570 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:05:20.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.562+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v104'48 uv48 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50fefccb0 2023-11-15T03:05:20.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.562+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50fefccb0 2023-11-15T03:05:20.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.562+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v104'49 uv49 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50fefccb0 2023-11-15T03:05:20.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.566+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50fefccb0 2023-11-15T03:05:20.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.566+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v104'50 uv50 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50fefccb0 2023-11-15T03:05:20.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.566+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50ff085c0 2023-11-15T03:05:20.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.570+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v104'57 uv56 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.570+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50ff085c0 2023-11-15T03:05:20.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.570+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v104'58 uv58 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.570+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50ff085c0 2023-11-15T03:05:20.583 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:05:20.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.574+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v104'59 uv59 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.574+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50ff085c0 2023-11-15T03:05:20.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.578+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v104'60 uv60 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.578+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50ff085c0 2023-11-15T03:05:20.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.578+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v104'61 uv61 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.578+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50ff085c0 2023-11-15T03:05:20.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.582+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v104'62 uv62 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.582+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50ff085c0 2023-11-15T03:05:20.594 INFO:teuthology.orchestra.run.smithi121.stdout:Write #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:05:20.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.586+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v104'49 uv49 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.586+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50ff085c0 2023-11-15T03:05:20.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.590+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [write 0~34] v104'50 uv50 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.590+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b900 con 0x55f50ff085c0 2023-11-15T03:05:20.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.594+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v104'51 uv51 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.594+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b900 con 0x55f50ff085c0 2023-11-15T03:05:20.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.594+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 100.00000000 [create] v104'47 uv46 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.594+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50ff085c0 2023-11-15T03:05:20.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.598+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 100.00000000 [delete] v104'48 uv48 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.598+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b4b0 con 0x55f50ff085c0 2023-11-15T03:05:20.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.602+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v104'49 uv49 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.612 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:05:20.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.602+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b8b0 con 0x55f50ff085c0 2023-11-15T03:05:20.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.606+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [omap-set-header] v104'50 uv50 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.606+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0e620 con 0x55f50ff085c0 2023-11-15T03:05:20.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.606+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v104'51 uv51 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.606+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55f50ff0c2b0 0x55f50ff0e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.606+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0eb90 con 0x55f50ff0c2b0 2023-11-15T03:05:20.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.610+0000 7fe8afe46700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55f50ff0c2b0 0x55f50ff0e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.610+0000 7fe8afe46700 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55f50ff0c2b0 0x55f50ff0e6a0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:20.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.610+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(99 mds0_sessionmap [create] v104'39 uv38 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff0c2b0 2023-11-15T03:05:20.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.614+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50ff0eb90 con 0x55f50ff0c2b0 2023-11-15T03:05:20.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.614+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(100 mds0_sessionmap [delete] v104'40 uv40 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff0c2b0 2023-11-15T03:05:20.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.614+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0eb90 con 0x55f50ff0c2b0 2023-11-15T03:05:20.627 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:05:20.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.618+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(101 mds0_sessionmap [create] v104'41 uv41 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff0c2b0 2023-11-15T03:05:20.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.618+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0eb90 con 0x55f50ff0c2b0 2023-11-15T03:05:20.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.622+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v104'42 uv42 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff0c2b0 2023-11-15T03:05:20.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.622+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0eb90 con 0x55f50ff085c0 2023-11-15T03:05:20.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.622+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(103 609.00000000 [create] v104'57 uv56 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.622+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50ff0eb90 con 0x55f50ff085c0 2023-11-15T03:05:20.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.626+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(104 609.00000000 [delete] v104'58 uv58 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.626+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0eb90 con 0x55f50ff085c0 2023-11-15T03:05:20.639 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:05:20.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.630+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v104'59 uv59 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.630+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0fba0 con 0x55f50ff085c0 2023-11-15T03:05:20.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.634+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v104'60 uv60 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.634+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0fba0 con 0x55f50ff085c0 2023-11-15T03:05:20.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.634+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v104'61 uv61 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.634+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0eb90 con 0x55f50ff085c0 2023-11-15T03:05:20.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.638+0000 7fe8a6e44700 1 -- 172.21.15.121:0/3256006648 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [omap-set-header] v104'62 uv62 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe89804a620 con 0x55f50ff085c0 2023-11-15T03:05:20.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.638+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0eb90 con 0x55f50ff02050 2023-11-15T03:05:20.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.642+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(109 400.00000000 [create] v104'47 uv46 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.642+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50ff0eb90 con 0x55f50ff02050 2023-11-15T03:05:20.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.642+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(110 400.00000000 [delete] v104'48 uv48 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.642+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0eb90 con 0x55f50ff02050 2023-11-15T03:05:20.656 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:05:20.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.646+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(111 400.00000000 [create] v104'49 uv49 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.646+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0eb90 con 0x55f50ff02050 2023-11-15T03:05:20.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.650+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(112 400.00000000 [write 0~22] v104'50 uv50 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.650+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0eb90 con 0x55f50ff02050 2023-11-15T03:05:20.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.650+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(113 400.00000000 [omap-set-header] v104'51 uv51 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.650+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0eb90 con 0x55f50ff02050 2023-11-15T03:05:20.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.654+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(114 100.00000000.inode [create] v104'114 uv106 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.654+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50ff0eb90 con 0x55f50ff02050 2023-11-15T03:05:20.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.658+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(115 100.00000000.inode [delete] v104'115 uv115 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.658+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0eb90 con 0x55f50ff02050 2023-11-15T03:05:20.668 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:05:20.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.658+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(116 100.00000000.inode [create] v104'116 uv116 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.658+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0eb90 con 0x55f50ff02050 2023-11-15T03:05:20.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.662+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v104'117 uv117 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.662+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b930 con 0x55f50ff02050 2023-11-15T03:05:20.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.666+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v104'118 uv118 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.666+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b930 con 0x55f50ff02050 2023-11-15T03:05:20.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.670+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(119 1.00000000 [create] v104'119 uv113 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.670+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55f50ff0b8b0 con 0x55f50ff02050 2023-11-15T03:05:20.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.674+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(120 1.00000000 [delete] v104'120 uv120 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.674+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b8b0 con 0x55f50ff02050 2023-11-15T03:05:20.687 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:05:20.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.678+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(121 1.00000000 [create] v104'121 uv121 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.678+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b8b0 con 0x55f50ff02050 2023-11-15T03:05:20.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.678+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v104'122 uv122 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.682+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0b8b0 con 0x55f50ff02050 2023-11-15T03:05:20.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.682+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v104'123 uv123 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.682+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0f080 con 0x55f50ff02050 2023-11-15T03:05:20.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.686+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v104'124 uv124 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.686+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e104) v8 -- 0x55f50ff0f080 con 0x55f50ff02050 2023-11-15T03:05:20.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v104'125 uv125 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe8a80089d0 con 0x55f50ff02050 2023-11-15T03:05:20.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55f50fefccb0 msgr2=0x55f50feff0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55f50fefccb0 0x55f50feff0a0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55f50fec8740 msgr2=0x55f50fedbcc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55f50fec8740 0x55f50fedbcc0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55f50ff05240 msgr2=0x55f50ff07630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55f50ff05240 0x55f50ff07630 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55f50fef7300 msgr2=0x55f50fefa3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55f50fef7300 0x55f50fefa3e0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55f50ff0c2b0 msgr2=0x55f50ff0e6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55f50ff0c2b0 0x55f50ff0e6a0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f50ff02050 msgr2=0x55f50ff04440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f50ff02050 0x55f50ff04440 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55f50ff085c0 msgr2=0x55f50ff0a9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55f50ff085c0 0x55f50ff0a9b0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 reap_dead start 2023-11-15T03:05:20.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 reap_dead start 2023-11-15T03:05:20.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 reap_dead start 2023-11-15T03:05:20.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8afe46700 1 -- 172.21.15.121:0/3256006648 reap_dead start 2023-11-15T03:05:20.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe890058ff0 msgr2=0x7fe89005b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe890058ff0 0x7fe89005b4b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fe8a0002a80 tx=0x7fe8a0008040).stop 2023-11-15T03:05:20.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f50fe4ab00 msgr2=0x55f50fec8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f50fe4ab00 0x55f50fec8250 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x55f50fe30e90 tx=0x7fe898018000).stop 2023-11-15T03:05:20.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 shutdown_connections 2023-11-15T03:05:20.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe890058ff0 0x7fe89005b4b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 --2- 172.21.15.121:0/3256006648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f50fe4ab00 0x55f50fec8250 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 >> 172.21.15.121:0/3256006648 conn(0x55f50fe50080 msgr2=0x55f50fe2fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:20.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 shutdown_connections 2023-11-15T03:05:20.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.690+0000 7fe8b1cfbd00 1 -- 172.21.15.121:0/3256006648 wait complete. 2023-11-15T03:05:20.706 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100.00000000 2023-11-15T03:05:20.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.798+0000 7fba06b41d00 1 Processor -- start 2023-11-15T03:05:20.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba06b41d00 1 -- start start 2023-11-15T03:05:20.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba06b41d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559307635eb0 0x559307636290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba06b41d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559307634ca0 0x5593076e71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba06b41d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5593076e76d0 0x5593076e7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba06b41d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593075afcc0 con 0x559307634ca0 2023-11-15T03:05:20.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba06b41d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930757a220 con 0x559307635eb0 2023-11-15T03:05:20.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba06b41d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x559307637010 con 0x5593076e76d0 2023-11-15T03:05:20.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba03c44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559307634ca0 0x5593076e71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba03c44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559307634ca0 0x5593076e71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39894/0 (socket says 172.21.15.121:39894) 2023-11-15T03:05:20.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba03c44700 1 -- 172.21.15.121:0/2090921731 learned_addr learned my addr 172.21.15.121:0/2090921731 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:20.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba04445700 1 --2- 172.21.15.121:0/2090921731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559307635eb0 0x559307636290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba04c46700 1 --2- 172.21.15.121:0/2090921731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5593076e76d0 0x5593076e7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba03c44700 1 -- 172.21.15.121:0/2090921731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5593076e76d0 msgr2=0x5593076e7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:20.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba03c44700 1 --2- 172.21.15.121:0/2090921731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5593076e76d0 0x5593076e7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba03c44700 1 -- 172.21.15.121:0/2090921731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559307635eb0 msgr2=0x559307636290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:20.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba03c44700 1 --2- 172.21.15.121:0/2090921731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559307635eb0 0x559307636290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba03c44700 1 -- 172.21.15.121:0/2090921731 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559307572a70 con 0x559307634ca0 2023-11-15T03:05:20.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba03c44700 1 --2- 172.21.15.121:0/2090921731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559307634ca0 0x5593076e71e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7fb9ec00a400 tx=0x7fb9f8016000).ready entity=mon.0 client_cookie=2eba545a5b0764e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:20.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba02c42700 1 -- 172.21.15.121:0/2090921731 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f800f040 con 0x559307634ca0 2023-11-15T03:05:20.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba02c42700 1 -- 172.21.15.121:0/2090921731 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f8008490 con 0x559307634ca0 2023-11-15T03:05:20.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba02c42700 1 -- 172.21.15.121:0/2090921731 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8008a90 con 0x559307634ca0 2023-11-15T03:05:20.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba06b41d00 1 -- 172.21.15.121:0/2090921731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559307634ca0 msgr2=0x5593076e71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba06b41d00 1 --2- 172.21.15.121:0/2090921731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559307634ca0 0x5593076e71e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7fb9ec00a400 tx=0x7fb9f8016000).stop 2023-11-15T03:05:20.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba06b41d00 1 -- 172.21.15.121:0/2090921731 shutdown_connections 2023-11-15T03:05:20.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba06b41d00 1 --2- 172.21.15.121:0/2090921731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5593076e76d0 0x5593076e7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba06b41d00 1 --2- 172.21.15.121:0/2090921731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559307634ca0 0x5593076e71e0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba06b41d00 1 --2- 172.21.15.121:0/2090921731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559307635eb0 0x559307636290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba06b41d00 1 -- 172.21.15.121:0/2090921731 >> 172.21.15.121:0/2090921731 conn(0x559307651080 msgr2=0x5593076355f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:20.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba06b41d00 1 -- 172.21.15.121:0/2090921731 shutdown_connections 2023-11-15T03:05:20.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.802+0000 7fba06b41d00 1 -- 172.21.15.121:0/2090921731 wait complete. 2023-11-15T03:05:20.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba06b41d00 1 Processor -- start 2023-11-15T03:05:20.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba06b41d00 1 -- start start 2023-11-15T03:05:20.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba06b41d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559307634ca0 0x5593076caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba06b41d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559307635eb0 0x5593076d2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba06b41d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593076e76d0 0x5593076d2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba06b41d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593076d0580 con 0x559307635eb0 2023-11-15T03:05:20.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba06b41d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593076cf970 con 0x5593076e76d0 2023-11-15T03:05:20.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba06b41d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593076cfc50 con 0x559307634ca0 2023-11-15T03:05:20.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba03c44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559307635eb0 0x5593076d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba03c44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559307635eb0 0x5593076d2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39906/0 (socket says 172.21.15.121:39906) 2023-11-15T03:05:20.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba03c44700 1 -- 172.21.15.121:0/3705785733 learned_addr learned my addr 172.21.15.121:0/3705785733 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:20.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba04445700 1 --2- 172.21.15.121:0/3705785733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559307634ca0 0x5593076caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba04c46700 1 --2- 172.21.15.121:0/3705785733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593076e76d0 0x5593076d2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba03c44700 1 -- 172.21.15.121:0/3705785733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559307634ca0 msgr2=0x5593076caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba03c44700 1 --2- 172.21.15.121:0/3705785733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559307634ca0 0x5593076caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba03c44700 1 -- 172.21.15.121:0/3705785733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593076e76d0 msgr2=0x5593076d2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba03c44700 1 --2- 172.21.15.121:0/3705785733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593076e76d0 0x5593076d2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba03c44700 1 -- 172.21.15.121:0/3705785733 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec001e10 con 0x559307635eb0 2023-11-15T03:05:20.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba04c46700 1 --2- 172.21.15.121:0/3705785733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593076e76d0 0x5593076d2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:05:20.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.806+0000 7fba03c44700 1 --2- 172.21.15.121:0/3705785733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559307635eb0 0x5593076d2170 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x559307635540 tx=0x7fb9f801d000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:20.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.810+0000 7fba0143f700 1 -- 172.21.15.121:0/3705785733 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f800f040 con 0x559307635eb0 2023-11-15T03:05:20.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.810+0000 7fba0143f700 1 -- 172.21.15.121:0/3705785733 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f8008600 con 0x559307635eb0 2023-11-15T03:05:20.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.810+0000 7fba0143f700 1 -- 172.21.15.121:0/3705785733 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8027690 con 0x559307635eb0 2023-11-15T03:05:20.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.810+0000 7fba06b41d00 1 -- 172.21.15.121:0/3705785733 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55930757e010 con 0x559307635eb0 2023-11-15T03:05:20.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.810+0000 7fba06b41d00 1 -- 172.21.15.121:0/3705785733 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593076d0210 con 0x559307635eb0 2023-11-15T03:05:20.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.810+0000 7fba0143f700 1 -- 172.21.15.121:0/3705785733 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb9f8027cd0 con 0x559307635eb0 2023-11-15T03:05:20.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.814+0000 7fba0143f700 1 --2- 172.21.15.121:0/3705785733 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb9f0058ff0 0x7fb9f005b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.814+0000 7fba0143f700 1 -- 172.21.15.121:0/3705785733 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7126+0+0 (secure 0 0 0) 0x7fb9f80770b0 con 0x559307635eb0 2023-11-15T03:05:20.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.814+0000 7fba04445700 1 --2- 172.21.15.121:0/3705785733 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb9f0058ff0 0x7fb9f005b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.814+0000 7fba06b41d00 1 --2- 172.21.15.121:0/3705785733 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5593076d2660 0x55930764a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:20.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.814+0000 7fba06b41d00 1 -- 172.21.15.121:0/3705785733 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e104) v8 -- 0x55930764a7a0 con 0x5593076d2660 2023-11-15T03:05:20.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.814+0000 7fba04445700 1 --2- 172.21.15.121:0/3705785733 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb9f0058ff0 0x7fb9f005b4b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fb9f40062f0 tx=0x7fb9f4005040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:20.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.814+0000 7fba04c46700 1 --2- 172.21.15.121:0/3705785733 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5593076d2660 0x55930764a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:20.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.814+0000 7fba04c46700 1 --2- 172.21.15.121:0/3705785733 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5593076d2660 0x55930764a2b0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:20.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.818+0000 7fba04c46700 1 -- 172.21.15.121:0/3705785733 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 100.00000000 [delete] v104'52 uv52 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb9fc003a50 con 0x5593076d2660 2023-11-15T03:05:20.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.818+0000 7fba06b41d00 1 -- 172.21.15.121:0/3705785733 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5593076d2660 msgr2=0x55930764a2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.818+0000 7fba06b41d00 1 --2- 172.21.15.121:0/3705785733 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5593076d2660 0x55930764a2b0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:20.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.818+0000 7fba06b41d00 1 -- 172.21.15.121:0/3705785733 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb9f0058ff0 msgr2=0x7fb9f005b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.818+0000 7fba06b41d00 1 --2- 172.21.15.121:0/3705785733 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb9f0058ff0 0x7fb9f005b4b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fb9f40062f0 tx=0x7fb9f4005040).stop 2023-11-15T03:05:20.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.818+0000 7fba06b41d00 1 -- 172.21.15.121:0/3705785733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559307635eb0 msgr2=0x5593076d2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:20.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.818+0000 7fba06b41d00 1 --2- 172.21.15.121:0/3705785733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559307635eb0 0x5593076d2170 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x559307635540 tx=0x7fb9f801d000).stop 2023-11-15T03:05:20.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.818+0000 7fba04c46700 1 -- 172.21.15.121:0/3705785733 reap_dead start 2023-11-15T03:05:20.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.818+0000 7fba06b41d00 1 -- 172.21.15.121:0/3705785733 shutdown_connections 2023-11-15T03:05:20.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.818+0000 7fba06b41d00 1 -- 172.21.15.121:0/3705785733 >> 172.21.15.121:0/3705785733 conn(0x559307651080 msgr2=0x5593076355a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:20.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.818+0000 7fba06b41d00 1 -- 172.21.15.121:0/3705785733 shutdown_connections 2023-11-15T03:05:20.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.818+0000 7fba06b41d00 1 -- 172.21.15.121:0/3705785733 wait complete. 2023-11-15T03:05:20.837 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:05:21.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.998+0000 7fe1f6c1b700 1 Processor -- start 2023-11-15T03:05:21.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.998+0000 7fe1f6c1b700 1 -- start start 2023-11-15T03:05:21.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.998+0000 7fe1f6c1b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe1f0067020 0x7fe1f01582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:21.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.998+0000 7fe1f6c1b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe1f0068170 0x7fe1f00bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:21.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.998+0000 7fe1f6c1b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f0062da0 con 0x7fe1f0157ed0 2023-11-15T03:05:21.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.998+0000 7fe1f6c1b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f00a93a0 con 0x7fe1f0067020 2023-11-15T03:05:21.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:20.998+0000 7fe1f6c1b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f0062190 con 0x7fe1f0068170 2023-11-15T03:05:21.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1f49b7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe1f0157ed0 0x7fe1f0067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:43004/0 (socket says 172.21.15.121:43004) 2023-11-15T03:05:21.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1f49b7700 1 -- 172.21.15.121:0/659953496 learned_addr learned my addr 172.21.15.121:0/659953496 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:21.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1e7fff700 1 --2- 172.21.15.121:0/659953496 >> v2:172.21.15.121:3300/0 conn(0x7fe1f0067020 0x7fe1f01582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:21.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1f51b8700 1 --2- 172.21.15.121:0/659953496 >> v2:172.21.15.190:3300/0 conn(0x7fe1f0068170 0x7fe1f00bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:21.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1e7fff700 1 -- 172.21.15.121:0/659953496 >> v2:172.21.15.190:3300/0 conn(0x7fe1f0068170 msgr2=0x7fe1f00bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:21.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1e7fff700 1 --2- 172.21.15.121:0/659953496 >> v2:172.21.15.190:3300/0 conn(0x7fe1f0068170 0x7fe1f00bc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:21.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1e7fff700 1 -- 172.21.15.121:0/659953496 >> v1:172.21.15.121:6789/0 conn(0x7fe1f0157ed0 legacy=0x7fe1f0067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:21.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1e77fe700 1 -- 172.21.15.121:0/659953496 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445255620 0 0) 0x7fe1f0062da0 con 0x7fe1f0157ed0 2023-11-15T03:05:21.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1e7fff700 1 -- 172.21.15.121:0/659953496 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f0062470 con 0x7fe1f0067020 2023-11-15T03:05:21.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1f51b8700 1 --2- 172.21.15.121:0/659953496 >> v2:172.21.15.190:3300/0 conn(0x7fe1f0068170 0x7fe1f00bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:21.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1e7fff700 1 --2- 172.21.15.121:0/659953496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1f0067020 0x7fe1f01582b0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7fe1f0063690 tx=0x7fe1e0009060).ready entity=mon.0 client_cookie=1b791d316af322a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:21.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1e77fe700 1 -- 172.21.15.121:0/659953496 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e000b580 con 0x7fe1f0067020 2023-11-15T03:05:21.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1e77fe700 1 -- 172.21.15.121:0/659953496 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e000b6e0 con 0x7fe1f0067020 2023-11-15T03:05:21.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1e77fe700 1 -- 172.21.15.121:0/659953496 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e000ba60 con 0x7fe1f0067020 2023-11-15T03:05:21.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1f6c1b700 1 -- 172.21.15.121:0/659953496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1f0067020 msgr2=0x7fe1f01582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:21.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1f6c1b700 1 --2- 172.21.15.121:0/659953496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1f0067020 0x7fe1f01582b0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7fe1f0063690 tx=0x7fe1e0009060).stop 2023-11-15T03:05:21.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1f6c1b700 1 -- 172.21.15.121:0/659953496 shutdown_connections 2023-11-15T03:05:21.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1f6c1b700 1 --2- 172.21.15.121:0/659953496 >> v2:172.21.15.190:3300/0 conn(0x7fe1f0068170 0x7fe1f00bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:21.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1f6c1b700 1 --2- 172.21.15.121:0/659953496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1f0067020 0x7fe1f01582b0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:21.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1f6c1b700 1 -- 172.21.15.121:0/659953496 >> 172.21.15.121:0/659953496 conn(0x7fe1f00591c0 msgr2=0x7fe1f00c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:21.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1f6c1b700 1 -- 172.21.15.121:0/659953496 shutdown_connections 2023-11-15T03:05:21.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.002+0000 7fe1f6c1b700 1 -- 172.21.15.121:0/659953496 wait complete. 2023-11-15T03:05:21.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.006+0000 7fe1f6c1b700 1 Processor -- start 2023-11-15T03:05:21.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.006+0000 7fe1f6c1b700 1 -- start start 2023-11-15T03:05:21.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1f6c1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1f0067020 0x7fe1f0156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:21.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1f6c1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1f0068170 0x7fe1f01688e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:21.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1f6c1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1f0157ed0 0x7fe1f0168dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:21.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1f6c1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f015a500 con 0x7fe1f0157ed0 2023-11-15T03:05:21.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1f6c1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f0169480 con 0x7fe1f0067020 2023-11-15T03:05:21.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1f6c1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f0169760 con 0x7fe1f0068170 2023-11-15T03:05:21.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1f51b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1f0157ed0 0x7fe1f0168dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:21.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1f51b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1f0157ed0 0x7fe1f0168dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39920/0 (socket says 172.21.15.121:39920) 2023-11-15T03:05:21.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1f51b8700 1 -- 172.21.15.121:0/2578681103 learned_addr learned my addr 172.21.15.121:0/2578681103 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:21.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1e7fff700 1 --2- 172.21.15.121:0/2578681103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1f0068170 0x7fe1f01688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:21.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1f49b7700 1 --2- 172.21.15.121:0/2578681103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1f0067020 0x7fe1f0156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:21.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1f51b8700 1 -- 172.21.15.121:0/2578681103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1f0068170 msgr2=0x7fe1f01688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:21.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1f51b8700 1 --2- 172.21.15.121:0/2578681103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1f0068170 0x7fe1f01688e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:21.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1f51b8700 1 -- 172.21.15.121:0/2578681103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1f0067020 msgr2=0x7fe1f0156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:21.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1f51b8700 1 --2- 172.21.15.121:0/2578681103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1f0067020 0x7fe1f0156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:21.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1f51b8700 1 -- 172.21.15.121:0/2578681103 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e00092a0 con 0x7fe1f0157ed0 2023-11-15T03:05:21.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1f49b7700 1 --2- 172.21.15.121:0/2578681103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1f0067020 0x7fe1f0156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:21.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1f51b8700 1 --2- 172.21.15.121:0/2578681103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1f0157ed0 0x7fe1f0168dd0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7fe1e8006c40 tx=0x7fe1e8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:21.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1e4ff9700 1 -- 172.21.15.121:0/2578681103 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8011590 con 0x7fe1f0157ed0 2023-11-15T03:05:21.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1e4ff9700 1 -- 172.21.15.121:0/2578681103 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e80116f0 con 0x7fe1f0157ed0 2023-11-15T03:05:21.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1f6c1b700 1 -- 172.21.15.121:0/2578681103 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f0159eb0 con 0x7fe1f0157ed0 2023-11-15T03:05:21.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.010+0000 7fe1e4ff9700 1 -- 172.21.15.121:0/2578681103 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8011a90 con 0x7fe1f0157ed0 2023-11-15T03:05:21.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.014+0000 7fe1f6c1b700 1 -- 172.21.15.121:0/2578681103 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f00a82a0 con 0x7fe1f0157ed0 2023-11-15T03:05:21.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.014+0000 7fe1f6c1b700 1 -- 172.21.15.121:0/2578681103 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f00bcb50 con 0x7fe1f0157ed0 2023-11-15T03:05:21.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.014+0000 7fe1e4ff9700 1 -- 172.21.15.121:0/2578681103 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe1e801c460 con 0x7fe1f0157ed0 2023-11-15T03:05:21.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.018+0000 7fe1e4ff9700 1 --2- 172.21.15.121:0/2578681103 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe1dc05a510 0x7fe1dc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:21.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.018+0000 7fe1e4ff9700 1 -- 172.21.15.121:0/2578681103 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7126+0+0 (secure 0 0 0) 0x7fe1e8074990 con 0x7fe1f0157ed0 2023-11-15T03:05:21.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.018+0000 7fe1f49b7700 1 --2- 172.21.15.121:0/2578681103 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe1dc05a510 0x7fe1dc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:21.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.018+0000 7fe1f49b7700 1 --2- 172.21.15.121:0/2578681103 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe1dc05a510 0x7fe1dc05c9d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fe1ec001e10 tx=0x7fe1ec009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:21.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.022+0000 7fe1e4ff9700 1 -- 172.21.15.121:0/2578681103 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe1f00bcb50 con 0x7fe1f0157ed0 2023-11-15T03:05:21.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:21.222+0000 7fe1f6c1b700 1 -- 172.21.15.121:0/2578681103 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fe1f00bcb50 con 0x7fe1f0157ed0 2023-11-15T03:05:22.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.222+0000 7fe1e4ff9700 1 -- 172.21.15.121:0/2578681103 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v144) v1 ==== 162+0+0 (secure 0 0 0) 0x7fe1f00bcb50 con 0x7fe1f0157ed0 2023-11-15T03:05:22.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.226+0000 7fe1f6c1b700 1 -- 172.21.15.121:0/2578681103 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe1dc05a510 msgr2=0x7fe1dc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:22.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.226+0000 7fe1f6c1b700 1 --2- 172.21.15.121:0/2578681103 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe1dc05a510 0x7fe1dc05c9d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fe1ec001e10 tx=0x7fe1ec009040).stop 2023-11-15T03:05:22.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.226+0000 7fe1f6c1b700 1 -- 172.21.15.121:0/2578681103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1f0157ed0 msgr2=0x7fe1f0168dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:22.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.226+0000 7fe1f6c1b700 1 --2- 172.21.15.121:0/2578681103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1f0157ed0 0x7fe1f0168dd0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7fe1e8006c40 tx=0x7fe1e8008040).stop 2023-11-15T03:05:22.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.226+0000 7fe1f6c1b700 1 -- 172.21.15.121:0/2578681103 shutdown_connections 2023-11-15T03:05:22.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.226+0000 7fe1f6c1b700 1 --2- 172.21.15.121:0/2578681103 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe1dc05a510 0x7fe1dc05c9d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:22.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.226+0000 7fe1f6c1b700 1 --2- 172.21.15.121:0/2578681103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1f0157ed0 0x7fe1f0168dd0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:22.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.226+0000 7fe1f6c1b700 1 --2- 172.21.15.121:0/2578681103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1f0068170 0x7fe1f01688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:22.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.226+0000 7fe1f6c1b700 1 --2- 172.21.15.121:0/2578681103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1f0067020 0x7fe1f0156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:22.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.226+0000 7fe1f6c1b700 1 -- 172.21.15.121:0/2578681103 >> 172.21.15.121:0/2578681103 conn(0x7fe1f00591c0 msgr2=0x7fe1f00c59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:22.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.230+0000 7fe1f6c1b700 1 -- 172.21.15.121:0/2578681103 shutdown_connections 2023-11-15T03:05:22.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.230+0000 7fe1f6c1b700 1 -- 172.21.15.121:0/2578681103 wait complete. 2023-11-15T03:05:22.240 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:05:22.260 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:05:22.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.414+0000 7f8c7cda9700 1 Processor -- start 2023-11-15T03:05:22.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.418+0000 7f8c7cda9700 1 -- start start 2023-11-15T03:05:22.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.418+0000 7f8c7cda9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8c781562f0 0x7f8c78155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:22.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.418+0000 7f8c7cda9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8c780a57c0 0x7f8c780a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:22.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.418+0000 7f8c7cda9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c780c4c30 con 0x7f8c78154ca0 2023-11-15T03:05:22.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.418+0000 7f8c7cda9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c780a8280 con 0x7f8c780a57c0 2023-11-15T03:05:22.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.418+0000 7f8c7cda9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c780a6d40 con 0x7f8c781562f0 2023-11-15T03:05:22.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.418+0000 7f8c76d9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8c780a57c0 0x7f8c780a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:22.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.418+0000 7f8c7659c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f8c78154ca0 0x7f8c780a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:43008/0 (socket says 172.21.15.121:43008) 2023-11-15T03:05:22.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.418+0000 7f8c7659c700 1 -- 172.21.15.121:0/28911211 learned_addr learned my addr 172.21.15.121:0/28911211 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:22.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.418+0000 7f8c6ffff700 1 --2- 172.21.15.121:0/28911211 >> v2:172.21.15.190:3300/0 conn(0x7f8c781562f0 0x7f8c78155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:22.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.422+0000 7f8c75d9b700 1 -- 172.21.15.121:0/28911211 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4021113881 0 0) 0x7f8c780c4c30 con 0x7f8c78154ca0 2023-11-15T03:05:22.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.422+0000 7f8c75d9b700 1 -- 172.21.15.121:0/28911211 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c640033f0 con 0x7f8c78154ca0 2023-11-15T03:05:22.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.422+0000 7f8c76d9d700 1 -- 172.21.15.121:0/28911211 >> v2:172.21.15.190:3300/0 conn(0x7f8c781562f0 msgr2=0x7f8c78155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:22.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.422+0000 7f8c76d9d700 1 --2- 172.21.15.121:0/28911211 >> v2:172.21.15.190:3300/0 conn(0x7f8c781562f0 0x7f8c78155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:22.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.422+0000 7f8c76d9d700 1 -- 172.21.15.121:0/28911211 >> v1:172.21.15.121:6789/0 conn(0x7f8c78154ca0 legacy=0x7f8c780a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:22.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.422+0000 7f8c76d9d700 1 -- 172.21.15.121:0/28911211 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c780a6100 con 0x7f8c780a57c0 2023-11-15T03:05:22.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.422+0000 7f8c76d9d700 1 --2- 172.21.15.121:0/28911211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c780a57c0 0x7f8c780a5c10 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7f8c780649a0 tx=0x7f8c70009780).ready entity=mon.0 client_cookie=392e9fd0e342e35e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:22.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.422+0000 7f8c75d9b700 1 -- 172.21.15.121:0/28911211 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7000e070 con 0x7f8c780a57c0 2023-11-15T03:05:22.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.422+0000 7f8c75d9b700 1 -- 172.21.15.121:0/28911211 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7000bab0 con 0x7f8c780a57c0 2023-11-15T03:05:22.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.422+0000 7f8c75d9b700 1 -- 172.21.15.121:0/28911211 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7000be30 con 0x7f8c780a57c0 2023-11-15T03:05:22.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.422+0000 7f8c7cda9700 1 -- 172.21.15.121:0/28911211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c780a57c0 msgr2=0x7f8c780a5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:22.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.422+0000 7f8c7cda9700 1 --2- 172.21.15.121:0/28911211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c780a57c0 0x7f8c780a5c10 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7f8c780649a0 tx=0x7f8c70009780).stop 2023-11-15T03:05:22.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.422+0000 7f8c7cda9700 1 -- 172.21.15.121:0/28911211 shutdown_connections 2023-11-15T03:05:22.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.422+0000 7f8c7cda9700 1 --2- 172.21.15.121:0/28911211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c780a57c0 0x7f8c780a5c10 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:22.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.422+0000 7f8c7cda9700 1 --2- 172.21.15.121:0/28911211 >> v2:172.21.15.190:3300/0 conn(0x7f8c781562f0 0x7f8c78155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:22.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.422+0000 7f8c7cda9700 1 -- 172.21.15.121:0/28911211 >> 172.21.15.121:0/28911211 conn(0x7f8c78062190 msgr2=0x7f8c780625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:22.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.422+0000 7f8c7cda9700 1 -- 172.21.15.121:0/28911211 shutdown_connections 2023-11-15T03:05:22.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.422+0000 7f8c7cda9700 1 -- 172.21.15.121:0/28911211 wait complete. 2023-11-15T03:05:22.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.426+0000 7f8c7cda9700 1 Processor -- start 2023-11-15T03:05:22.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.426+0000 7f8c7cda9700 1 -- start start 2023-11-15T03:05:22.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.426+0000 7f8c7cda9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c780a57c0 0x7f8c780a9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:22.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.426+0000 7f8c7cda9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c78154ca0 0x7f8c7814e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:22.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.426+0000 7f8c7cda9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c781562f0 0x7f8c78156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:22.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.426+0000 7f8c7cda9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c78152260 con 0x7f8c781562f0 2023-11-15T03:05:22.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.426+0000 7f8c7cda9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c781574b0 con 0x7f8c78154ca0 2023-11-15T03:05:22.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.426+0000 7f8c7cda9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c78157790 con 0x7f8c780a57c0 2023-11-15T03:05:22.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.426+0000 7f8c76d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c781562f0 0x7f8c78156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:22.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.426+0000 7f8c76d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c781562f0 0x7f8c78156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39942/0 (socket says 172.21.15.121:39942) 2023-11-15T03:05:22.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.426+0000 7f8c76d9d700 1 -- 172.21.15.121:0/696276646 learned_addr learned my addr 172.21.15.121:0/696276646 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:22.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.426+0000 7f8c7659c700 1 --2- 172.21.15.121:0/696276646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c780a57c0 0x7f8c780a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:22.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.426+0000 7f8c6ffff700 1 --2- 172.21.15.121:0/696276646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c78154ca0 0x7f8c7814e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:22.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.426+0000 7f8c76d9d700 1 -- 172.21.15.121:0/696276646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c780a57c0 msgr2=0x7f8c780a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:22.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.426+0000 7f8c76d9d700 1 --2- 172.21.15.121:0/696276646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c780a57c0 0x7f8c780a9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:22.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.426+0000 7f8c76d9d700 1 -- 172.21.15.121:0/696276646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c78154ca0 msgr2=0x7f8c7814e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:22.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.426+0000 7f8c76d9d700 1 --2- 172.21.15.121:0/696276646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c78154ca0 0x7f8c7814e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:22.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.426+0000 7f8c76d9d700 1 -- 172.21.15.121:0/696276646 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c70009a20 con 0x7f8c781562f0 2023-11-15T03:05:22.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.430+0000 7f8c76d9d700 1 --2- 172.21.15.121:0/696276646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c781562f0 0x7f8c78156c60 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7f8c78063d40 tx=0x7f8c780a6290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:22.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.430+0000 7f8c6e7fc700 1 -- 172.21.15.121:0/696276646 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7001d630 con 0x7f8c781562f0 2023-11-15T03:05:22.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.430+0000 7f8c6e7fc700 1 -- 172.21.15.121:0/696276646 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7001d790 con 0x7f8c781562f0 2023-11-15T03:05:22.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.430+0000 7f8c7cda9700 1 -- 172.21.15.121:0/696276646 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c78157a70 con 0x7f8c781562f0 2023-11-15T03:05:22.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.430+0000 7f8c6e7fc700 1 -- 172.21.15.121:0/696276646 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7001db30 con 0x7f8c781562f0 2023-11-15T03:05:22.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.430+0000 7f8c7cda9700 1 -- 172.21.15.121:0/696276646 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7814edb0 con 0x7f8c781562f0 2023-11-15T03:05:22.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.430+0000 7f8c7cda9700 1 -- 172.21.15.121:0/696276646 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c78152030 con 0x7f8c781562f0 2023-11-15T03:05:22.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.430+0000 7f8c6e7fc700 1 -- 172.21.15.121:0/696276646 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8c7001c380 con 0x7f8c781562f0 2023-11-15T03:05:22.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.434+0000 7f8c6e7fc700 1 --2- 172.21.15.121:0/696276646 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8c5805a510 0x7f8c5805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:22.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.434+0000 7f8c6e7fc700 1 -- 172.21.15.121:0/696276646 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7126+0+0 (secure 0 0 0) 0x7f8c70076aa0 con 0x7f8c781562f0 2023-11-15T03:05:22.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.434+0000 7f8c7659c700 1 --2- 172.21.15.121:0/696276646 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8c5805a510 0x7f8c5805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:22.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.434+0000 7f8c7659c700 1 --2- 172.21.15.121:0/696276646 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8c5805a510 0x7f8c5805c9d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f8c68005750 tx=0x7f8c68009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:22.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.442+0000 7f8c6e7fc700 1 -- 172.21.15.121:0/696276646 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8c78152030 con 0x7f8c781562f0 2023-11-15T03:05:22.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.646+0000 7f8c7cda9700 1 -- 172.21.15.121:0/696276646 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c78152030 con 0x7f8c781562f0 2023-11-15T03:05:22.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.646+0000 7f8c6e7fc700 1 -- 172.21.15.121:0/696276646 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 145 v145) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f8c78152030 con 0x7f8c781562f0 2023-11-15T03:05:22.655 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:22.656 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":145,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":145,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:22.229093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8199},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8199":{"gid":8199,"name":"a","rank":0,"incarnation":145,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/2860780116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2860780116},{"type":"v1","addr":"172.21.15.121:6836","nonce":2860780116}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:05:22.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.650+0000 7f8c7cda9700 1 -- 172.21.15.121:0/696276646 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8c5805a510 msgr2=0x7f8c5805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:22.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.650+0000 7f8c7cda9700 1 --2- 172.21.15.121:0/696276646 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8c5805a510 0x7f8c5805c9d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f8c68005750 tx=0x7f8c68009040).stop 2023-11-15T03:05:22.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.650+0000 7f8c7cda9700 1 -- 172.21.15.121:0/696276646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c781562f0 msgr2=0x7f8c78156c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:22.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.650+0000 7f8c7cda9700 1 --2- 172.21.15.121:0/696276646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c781562f0 0x7f8c78156c60 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7f8c78063d40 tx=0x7f8c780a6290).stop 2023-11-15T03:05:22.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.650+0000 7f8c7cda9700 1 -- 172.21.15.121:0/696276646 shutdown_connections 2023-11-15T03:05:22.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.650+0000 7f8c7cda9700 1 --2- 172.21.15.121:0/696276646 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8c5805a510 0x7f8c5805c9d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:22.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.650+0000 7f8c7cda9700 1 --2- 172.21.15.121:0/696276646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c781562f0 0x7f8c78156c60 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:22.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.650+0000 7f8c7cda9700 1 --2- 172.21.15.121:0/696276646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c78154ca0 0x7f8c7814e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:22.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.650+0000 7f8c7cda9700 1 --2- 172.21.15.121:0/696276646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c780a57c0 0x7f8c780a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:22.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.650+0000 7f8c7cda9700 1 -- 172.21.15.121:0/696276646 >> 172.21.15.121:0/696276646 conn(0x7f8c78062190 msgr2=0x7f8c78150660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:22.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.650+0000 7f8c7cda9700 1 -- 172.21.15.121:0/696276646 shutdown_connections 2023-11-15T03:05:22.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:22.650+0000 7f8c7cda9700 1 -- 172.21.15.121:0/696276646 wait complete. 2023-11-15T03:05:22.662 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 145 2023-11-15T03:05:22.680 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:05:25.267 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:05:25.258+0000 7f6f4cfbe700 -1 log_channel(cluster) log [ERR] : dir 0x100 object missing on disk; some files may be lost (~mds0) 2023-11-15T03:05:26.267 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:05:26.258+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:05:26.268 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:05:26.258+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:05:26.281 INFO:tasks.ceph.mds.a.smithi121.stderr: -21> 2023-11-15T03:05:25.258+0000 7f6f4cfbe700 -1 log_channel(cluster) log [ERR] : dir 0x100 object missing on disk; some files may be lost (~mds0) 2023-11-15T03:05:26.321 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:05:27.681 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:05:27.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.838+0000 7ff1a1169700 1 Processor -- start 2023-11-15T03:05:27.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.838+0000 7ff1a1169700 1 -- start start 2023-11-15T03:05:27.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.842+0000 7ff1a1169700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff19c152780 0x7ff19c159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:27.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.842+0000 7ff1a1169700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff19c0c65d0 0x7ff19c0c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:27.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.842+0000 7ff1a1169700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff19c0a5510 0x7ff19c0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:27.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.842+0000 7ff1a1169700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19c0c3fa0 con 0x7ff19c152780 2023-11-15T03:05:27.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.842+0000 7ff1a1169700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19c159630 con 0x7ff19c0c65d0 2023-11-15T03:05:27.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.842+0000 7ff1a1169700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0c30b0 con 0x7ff19c0a5510 2023-11-15T03:05:27.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.842+0000 7ff19ad9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff19c152780 0x7ff19c159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:27.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.842+0000 7ff19b59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff19c0a5510 0x7ff19c0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:27.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.842+0000 7ff19b59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff19c0a5510 0x7ff19c0a59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:41888/0 (socket says 172.21.15.121:41888) 2023-11-15T03:05:27.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.842+0000 7ff19b59e700 1 -- 172.21.15.121:0/2872157288 learned_addr learned my addr 172.21.15.121:0/2872157288 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:27.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.842+0000 7ff19a59c700 1 --2- 172.21.15.121:0/2872157288 >> v2:172.21.15.190:3300/0 conn(0x7ff19c0c65d0 0x7ff19c0c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:27.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.846+0000 7ff19b59e700 1 -- 172.21.15.121:0/2872157288 >> v2:172.21.15.190:3300/0 conn(0x7ff19c0c65d0 msgr2=0x7ff19c0c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:27.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.846+0000 7ff19b59e700 1 --2- 172.21.15.121:0/2872157288 >> v2:172.21.15.190:3300/0 conn(0x7ff19c0c65d0 0x7ff19c0c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:27.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.846+0000 7ff19b59e700 1 -- 172.21.15.121:0/2872157288 >> v2:172.21.15.121:3300/0 conn(0x7ff19c152780 msgr2=0x7ff19c159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:27.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.846+0000 7ff19b59e700 1 --2- 172.21.15.121:0/2872157288 >> v2:172.21.15.121:3300/0 conn(0x7ff19c152780 0x7ff19c159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:27.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.846+0000 7ff19b59e700 1 -- 172.21.15.121:0/2872157288 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c0c3390 con 0x7ff19c0a5510 2023-11-15T03:05:27.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.846+0000 7ff19ad9d700 1 --2- 172.21.15.121:0/2872157288 >> v2:172.21.15.121:3300/0 conn(0x7ff19c152780 0x7ff19c159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:05:27.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.846+0000 7ff19b59e700 1 --2- 172.21.15.121:0/2872157288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff19c0a5510 0x7ff19c0a59a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7ff19c063d40 tx=0x7ff194009840).ready entity=mon.2 client_cookie=32d5f952b07000ca server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:27.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.846+0000 7ff199d9b700 1 -- 172.21.15.121:0/2872157288 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19400e070 con 0x7ff19c0a5510 2023-11-15T03:05:27.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.846+0000 7ff199d9b700 1 -- 172.21.15.121:0/2872157288 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19400b970 con 0x7ff19c0a5510 2023-11-15T03:05:27.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.846+0000 7ff199d9b700 1 -- 172.21.15.121:0/2872157288 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19400bcf0 con 0x7ff19c0a5510 2023-11-15T03:05:27.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.846+0000 7ff1a1169700 1 -- 172.21.15.121:0/2872157288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff19c0a5510 msgr2=0x7ff19c0a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:27.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.846+0000 7ff1a1169700 1 --2- 172.21.15.121:0/2872157288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff19c0a5510 0x7ff19c0a59a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7ff19c063d40 tx=0x7ff194009840).stop 2023-11-15T03:05:27.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.846+0000 7ff1a1169700 1 -- 172.21.15.121:0/2872157288 shutdown_connections 2023-11-15T03:05:27.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.846+0000 7ff1a1169700 1 --2- 172.21.15.121:0/2872157288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff19c0a5510 0x7ff19c0a59a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:27.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.846+0000 7ff1a1169700 1 --2- 172.21.15.121:0/2872157288 >> v2:172.21.15.190:3300/0 conn(0x7ff19c0c65d0 0x7ff19c0c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:27.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.846+0000 7ff1a1169700 1 --2- 172.21.15.121:0/2872157288 >> v2:172.21.15.121:3300/0 conn(0x7ff19c152780 0x7ff19c159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:27.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.846+0000 7ff1a1169700 1 -- 172.21.15.121:0/2872157288 >> 172.21.15.121:0/2872157288 conn(0x7ff19c062190 msgr2=0x7ff19c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:27.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.846+0000 7ff1a1169700 1 -- 172.21.15.121:0/2872157288 shutdown_connections 2023-11-15T03:05:27.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.846+0000 7ff1a1169700 1 -- 172.21.15.121:0/2872157288 wait complete. 2023-11-15T03:05:27.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff1a1169700 1 Processor -- start 2023-11-15T03:05:27.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff1a1169700 1 -- start start 2023-11-15T03:05:27.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff1a1169700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff19c0a5510 0x7ff19c142920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:27.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff1a1169700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff19c0c65d0 0x7ff19c14ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:27.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff1a1169700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff19c152780 0x7ff19c14f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:27.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff1a1169700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c14f700 con 0x7ff19c0c65d0 2023-11-15T03:05:27.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff1a1169700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c169770 con 0x7ff19c152780 2023-11-15T03:05:27.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff1a1169700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c168b60 con 0x7ff19c0a5510 2023-11-15T03:05:27.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff19a59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff19c0c65d0 0x7ff19c14ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:27.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff19a59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff19c0c65d0 0x7ff19c14ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59454/0 (socket says 172.21.15.121:59454) 2023-11-15T03:05:27.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff19a59c700 1 -- 172.21.15.121:0/1561729305 learned_addr learned my addr 172.21.15.121:0/1561729305 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:27.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff19ad9d700 1 --2- 172.21.15.121:0/1561729305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff19c0a5510 0x7ff19c142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:27.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff19b59e700 1 --2- 172.21.15.121:0/1561729305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff19c152780 0x7ff19c14f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:27.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff19a59c700 1 -- 172.21.15.121:0/1561729305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff19c0a5510 msgr2=0x7ff19c142920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:27.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff19a59c700 1 --2- 172.21.15.121:0/1561729305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff19c0a5510 0x7ff19c142920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:27.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff19a59c700 1 -- 172.21.15.121:0/1561729305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff19c152780 msgr2=0x7ff19c14f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:27.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff19a59c700 1 --2- 172.21.15.121:0/1561729305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff19c152780 0x7ff19c14f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:27.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff19a59c700 1 -- 172.21.15.121:0/1561729305 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18c0092a0 con 0x7ff19c0c65d0 2023-11-15T03:05:27.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff19a59c700 1 --2- 172.21.15.121:0/1561729305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff19c0c65d0 0x7ff19c14ed20 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7ff19000e950 tx=0x7ff190009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:27.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff182ffd700 1 -- 172.21.15.121:0/1561729305 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190009530 con 0x7ff19c0c65d0 2023-11-15T03:05:27.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff182ffd700 1 -- 172.21.15.121:0/1561729305 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19000c850 con 0x7ff19c0c65d0 2023-11-15T03:05:27.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.850+0000 7ff1a1169700 1 -- 172.21.15.121:0/1561729305 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff194009ae0 con 0x7ff19c0c65d0 2023-11-15T03:05:27.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.854+0000 7ff182ffd700 1 -- 172.21.15.121:0/1561729305 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19000cbd0 con 0x7ff19c0c65d0 2023-11-15T03:05:27.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.854+0000 7ff1a1169700 1 -- 172.21.15.121:0/1561729305 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19c169120 con 0x7ff19c0c65d0 2023-11-15T03:05:27.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.854+0000 7ff1a1169700 1 -- 172.21.15.121:0/1561729305 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19c06c150 con 0x7ff19c0c65d0 2023-11-15T03:05:27.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.854+0000 7ff182ffd700 1 -- 172.21.15.121:0/1561729305 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff19001b400 con 0x7ff19c0c65d0 2023-11-15T03:05:27.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.858+0000 7ff182ffd700 1 --2- 172.21.15.121:0/1561729305 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff18805a510 0x7ff18805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:27.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.858+0000 7ff182ffd700 1 -- 172.21.15.121:0/1561729305 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7212+0+0 (secure 0 0 0) 0x7ff1900751e0 con 0x7ff19c0c65d0 2023-11-15T03:05:27.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.858+0000 7ff19ad9d700 1 --2- 172.21.15.121:0/1561729305 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff18805a510 0x7ff18805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:27.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.858+0000 7ff19ad9d700 1 --2- 172.21.15.121:0/1561729305 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff18805a510 0x7ff18805c9d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7ff18c009270 tx=0x7ff18c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:27.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:27.862+0000 7ff182ffd700 1 -- 172.21.15.121:0/1561729305 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff190048460 con 0x7ff19c0c65d0 2023-11-15T03:05:28.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.066+0000 7ff1a1169700 1 -- 172.21.15.121:0/1561729305 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff19c0c11b0 con 0x7ff19c0c65d0 2023-11-15T03:05:28.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.066+0000 7ff182ffd700 1 -- 172.21.15.121:0/1561729305 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) v1 ==== 95+0+4496 (secure 0 0 0) 0x7ff19004ba80 con 0x7ff19c0c65d0 2023-11-15T03:05:28.077 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:28.077 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":150,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8271,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1529582418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1529582418},{"type":"v1","addr":"172.21.15.121:6836","nonce":1529582418}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:26.263706+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:05:28.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.070+0000 7ff1a1169700 1 -- 172.21.15.121:0/1561729305 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff18805a510 msgr2=0x7ff18805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:28.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.070+0000 7ff1a1169700 1 --2- 172.21.15.121:0/1561729305 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff18805a510 0x7ff18805c9d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7ff18c009270 tx=0x7ff18c008040).stop 2023-11-15T03:05:28.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.070+0000 7ff1a1169700 1 -- 172.21.15.121:0/1561729305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff19c0c65d0 msgr2=0x7ff19c14ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:28.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.070+0000 7ff1a1169700 1 --2- 172.21.15.121:0/1561729305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff19c0c65d0 0x7ff19c14ed20 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7ff19000e950 tx=0x7ff190009060).stop 2023-11-15T03:05:28.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.070+0000 7ff1a1169700 1 -- 172.21.15.121:0/1561729305 shutdown_connections 2023-11-15T03:05:28.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.070+0000 7ff1a1169700 1 --2- 172.21.15.121:0/1561729305 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff18805a510 0x7ff18805c9d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.070+0000 7ff1a1169700 1 --2- 172.21.15.121:0/1561729305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff19c152780 0x7ff19c14f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.070+0000 7ff1a1169700 1 --2- 172.21.15.121:0/1561729305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff19c0c65d0 0x7ff19c14ed20 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.070+0000 7ff1a1169700 1 --2- 172.21.15.121:0/1561729305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff19c0a5510 0x7ff19c142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.070+0000 7ff1a1169700 1 -- 172.21.15.121:0/1561729305 >> 172.21.15.121:0/1561729305 conn(0x7ff19c062190 msgr2=0x7ff19c066520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:28.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.070+0000 7ff1a1169700 1 -- 172.21.15.121:0/1561729305 shutdown_connections 2023-11-15T03:05:28.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.070+0000 7ff1a1169700 1 -- 172.21.15.121:0/1561729305 wait complete. 2023-11-15T03:05:28.082 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 150 2023-11-15T03:05:28.098 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:05:28.099 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:05:28.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.262+0000 7f765b107700 1 Processor -- start 2023-11-15T03:05:28.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.262+0000 7f765b107700 1 -- start start 2023-11-15T03:05:28.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.266+0000 7f765b107700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7654151870 0x7f7654151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:28.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.266+0000 7f765b107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7654152140 0x7f76540a9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:28.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.266+0000 7f765b107700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76540657f0 con 0x7f76540a9d20 2023-11-15T03:05:28.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.266+0000 7f765b107700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7654065ad0 con 0x7f7654151870 2023-11-15T03:05:28.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.266+0000 7f765b107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76540aa260 con 0x7f7654152140 2023-11-15T03:05:28.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.266+0000 7f7659904700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7654152140 0x7f76540a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:28.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.266+0000 7f765a105700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7654151870 0x7f7654151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:28.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.266+0000 7f765a906700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f76540a9d20 0x7f76540c30e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:48418/0 (socket says 172.21.15.121:48418) 2023-11-15T03:05:28.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.266+0000 7f765a906700 1 -- 172.21.15.121:0/1137570770 learned_addr learned my addr 172.21.15.121:0/1137570770 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:28.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.270+0000 7f765a105700 1 -- 172.21.15.121:0/1137570770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7654152140 msgr2=0x7f76540a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:28.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.270+0000 7f765a105700 1 --2- 172.21.15.121:0/1137570770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7654152140 0x7f76540a9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.270+0000 7f765a105700 1 -- 172.21.15.121:0/1137570770 >> v1:172.21.15.190:6789/0 conn(0x7f76540a9d20 legacy=0x7f76540c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:28.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.270+0000 7f765a105700 1 -- 172.21.15.121:0/1137570770 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76540aa540 con 0x7f7654151870 2023-11-15T03:05:28.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.270+0000 7f765a105700 1 --2- 172.21.15.121:0/1137570770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7654151870 0x7f7654151c50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f7650009760 tx=0x7f7650009060).ready entity=mon.1 client_cookie=726cb90c36205f15 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:28.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.270+0000 7f7659103700 1 -- 172.21.15.121:0/1137570770 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765000f510 con 0x7f7654151870 2023-11-15T03:05:28.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.270+0000 7f7659103700 1 -- 172.21.15.121:0/1137570770 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765000f670 con 0x7f7654151870 2023-11-15T03:05:28.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.270+0000 7f7659103700 1 -- 172.21.15.121:0/1137570770 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765000f9f0 con 0x7f7654151870 2023-11-15T03:05:28.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.270+0000 7f765b107700 1 -- 172.21.15.121:0/1137570770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7654151870 msgr2=0x7f7654151c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:28.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.270+0000 7f765b107700 1 --2- 172.21.15.121:0/1137570770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7654151870 0x7f7654151c50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f7650009760 tx=0x7f7650009060).stop 2023-11-15T03:05:28.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.270+0000 7f765b107700 1 -- 172.21.15.121:0/1137570770 shutdown_connections 2023-11-15T03:05:28.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.270+0000 7f765b107700 1 --2- 172.21.15.121:0/1137570770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7654152140 0x7f76540a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.270+0000 7f765b107700 1 --2- 172.21.15.121:0/1137570770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7654151870 0x7f7654151c50 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.270+0000 7f765b107700 1 -- 172.21.15.121:0/1137570770 >> 172.21.15.121:0/1137570770 conn(0x7f7654062190 msgr2=0x7f7654062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:28.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.270+0000 7f765b107700 1 -- 172.21.15.121:0/1137570770 shutdown_connections 2023-11-15T03:05:28.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.270+0000 7f765b107700 1 -- 172.21.15.121:0/1137570770 wait complete. 2023-11-15T03:05:28.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.274+0000 7f765b107700 1 Processor -- start 2023-11-15T03:05:28.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.274+0000 7f765b107700 1 -- start start 2023-11-15T03:05:28.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f765b107700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76540a9d20 0x7f76541464f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:28.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f765b107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7654151870 0x7f7654158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:28.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f765b107700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7654152140 0x7f765414e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:28.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f765b107700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76541584f0 con 0x7f76540a9d20 2023-11-15T03:05:28.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f765b107700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76541592f0 con 0x7f7654152140 2023-11-15T03:05:28.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f765b107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765414e930 con 0x7f7654151870 2023-11-15T03:05:28.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f765a105700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76540a9d20 0x7f76541464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:28.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f765a105700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76540a9d20 0x7f76541464f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59468/0 (socket says 172.21.15.121:59468) 2023-11-15T03:05:28.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f765a105700 1 -- 172.21.15.121:0/2785567030 learned_addr learned my addr 172.21.15.121:0/2785567030 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:28.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f765a906700 1 --2- 172.21.15.121:0/2785567030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7654152140 0x7f765414e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:28.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f7659904700 1 --2- 172.21.15.121:0/2785567030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7654151870 0x7f7654158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:28.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f765a105700 1 -- 172.21.15.121:0/2785567030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7654151870 msgr2=0x7f7654158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:28.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f765a105700 1 --2- 172.21.15.121:0/2785567030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7654151870 0x7f7654158000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f765a105700 1 -- 172.21.15.121:0/2785567030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7654152140 msgr2=0x7f765414e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:28.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f765a105700 1 --2- 172.21.15.121:0/2785567030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7654152140 0x7f765414e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f765a105700 1 -- 172.21.15.121:0/2785567030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76500092a0 con 0x7f76540a9d20 2023-11-15T03:05:28.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f7659904700 1 --2- 172.21.15.121:0/2785567030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7654151870 0x7f7654158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:05:28.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f765a105700 1 --2- 172.21.15.121:0/2785567030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76540a9d20 0x7f76541464f0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7f7650009e00 tx=0x7f7650017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:28.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f764a7fc700 1 -- 172.21.15.121:0/2785567030 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765000f7e0 con 0x7f76540a9d20 2023-11-15T03:05:28.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f764a7fc700 1 -- 172.21.15.121:0/2785567030 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765000f940 con 0x7f76540a9d20 2023-11-15T03:05:28.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f764a7fc700 1 -- 172.21.15.121:0/2785567030 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765001b660 con 0x7f76540a9d20 2023-11-15T03:05:28.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.278+0000 7f765b107700 1 -- 172.21.15.121:0/2785567030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76541586e0 con 0x7f76540a9d20 2023-11-15T03:05:28.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.282+0000 7f765b107700 1 -- 172.21.15.121:0/2785567030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7654158ca0 con 0x7f76540a9d20 2023-11-15T03:05:28.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.282+0000 7f765b107700 1 -- 172.21.15.121:0/2785567030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765406bcf0 con 0x7f76540a9d20 2023-11-15T03:05:28.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.282+0000 7f764a7fc700 1 -- 172.21.15.121:0/2785567030 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f765001b9d0 con 0x7f76540a9d20 2023-11-15T03:05:28.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.286+0000 7f764a7fc700 1 --2- 172.21.15.121:0/2785567030 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f763805a4e0 0x7f763805c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:28.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.286+0000 7f764a7fc700 1 -- 172.21.15.121:0/2785567030 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7212+0+0 (secure 0 0 0) 0x7f7650076560 con 0x7f76540a9d20 2023-11-15T03:05:28.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.286+0000 7f7659904700 1 --2- 172.21.15.121:0/2785567030 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f763805a4e0 0x7f763805c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:28.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.286+0000 7f7659904700 1 --2- 172.21.15.121:0/2785567030 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f763805a4e0 0x7f763805c9a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f7644006d80 tx=0x7f7644008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:28.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.290+0000 7f764a7fc700 1 -- 172.21.15.121:0/2785567030 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7650049900 con 0x7f76540a9d20 2023-11-15T03:05:28.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.490+0000 7f765b107700 1 -- 172.21.15.121:0/2785567030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76540aa820 con 0x7f76540a9d20 2023-11-15T03:05:28.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.494+0000 7f764a7fc700 1 -- 172.21.15.121:0/2785567030 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f7650029090 con 0x7f76540a9d20 2023-11-15T03:05:28.502 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:28.502 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":150,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8271,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1529582418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1529582418},{"type":"v1","addr":"172.21.15.121:6836","nonce":1529582418}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:26.263706+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:05:28.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.494+0000 7f765b107700 1 -- 172.21.15.121:0/2785567030 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f763805a4e0 msgr2=0x7f763805c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:28.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.494+0000 7f765b107700 1 --2- 172.21.15.121:0/2785567030 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f763805a4e0 0x7f763805c9a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f7644006d80 tx=0x7f7644008040).stop 2023-11-15T03:05:28.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.494+0000 7f765b107700 1 -- 172.21.15.121:0/2785567030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76540a9d20 msgr2=0x7f76541464f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:28.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.494+0000 7f765b107700 1 --2- 172.21.15.121:0/2785567030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76540a9d20 0x7f76541464f0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7f7650009e00 tx=0x7f7650017000).stop 2023-11-15T03:05:28.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.494+0000 7f765b107700 1 -- 172.21.15.121:0/2785567030 shutdown_connections 2023-11-15T03:05:28.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.494+0000 7f765b107700 1 --2- 172.21.15.121:0/2785567030 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f763805a4e0 0x7f763805c9a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.494+0000 7f765b107700 1 --2- 172.21.15.121:0/2785567030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7654152140 0x7f765414e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.494+0000 7f765b107700 1 --2- 172.21.15.121:0/2785567030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7654151870 0x7f7654158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.494+0000 7f765b107700 1 --2- 172.21.15.121:0/2785567030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76540a9d20 0x7f76541464f0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.494+0000 7f765b107700 1 -- 172.21.15.121:0/2785567030 >> 172.21.15.121:0/2785567030 conn(0x7f7654062190 msgr2=0x7f76540bfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:28.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.494+0000 7f765b107700 1 -- 172.21.15.121:0/2785567030 shutdown_connections 2023-11-15T03:05:28.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.494+0000 7f765b107700 1 -- 172.21.15.121:0/2785567030 wait complete. 2023-11-15T03:05:28.507 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 150 2023-11-15T03:05:28.529 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:05:28.529 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:05:28.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.686+0000 7fe24669b700 1 Processor -- start 2023-11-15T03:05:28.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.686+0000 7fe24669b700 1 -- start start 2023-11-15T03:05:28.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.690+0000 7fe24669b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe2401539d0 0x7fe2400c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:28.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.690+0000 7fe24669b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe240153db0 0x7fe2400c2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:28.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.690+0000 7fe24669b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2400c1530 con 0x7fe2400c19c0 2023-11-15T03:05:28.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.690+0000 7fe24669b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2401541f0 con 0x7fe240153db0 2023-11-15T03:05:28.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.690+0000 7fe24669b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2400c2b60 con 0x7fe2401539d0 2023-11-15T03:05:28.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.690+0000 7fe2377fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe240153db0 0x7fe2400c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:28.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.690+0000 7fe2377fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe240153db0 0x7fe2400c2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59480/0 (socket says 172.21.15.121:59480) 2023-11-15T03:05:28.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.690+0000 7fe2377fe700 1 -- 172.21.15.121:0/3852966775 learned_addr learned my addr 172.21.15.121:0/3852966775 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:28.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.690+0000 7fe23ffff700 1 --2- 172.21.15.121:0/3852966775 >> v2:172.21.15.190:3300/0 conn(0x7fe2401539d0 0x7fe2400c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:28.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.690+0000 7fe2377fe700 1 -- 172.21.15.121:0/3852966775 >> v2:172.21.15.190:3300/0 conn(0x7fe2401539d0 msgr2=0x7fe2400c06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:28.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.690+0000 7fe2377fe700 1 --2- 172.21.15.121:0/3852966775 >> v2:172.21.15.190:3300/0 conn(0x7fe2401539d0 0x7fe2400c06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.690+0000 7fe2377fe700 1 -- 172.21.15.121:0/3852966775 >> v1:172.21.15.190:6789/0 conn(0x7fe2400c19c0 legacy=0x7fe2400c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:28.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.690+0000 7fe2377fe700 1 -- 172.21.15.121:0/3852966775 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2400c2d80 con 0x7fe240153db0 2023-11-15T03:05:28.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.694+0000 7fe2377fe700 1 --2- 172.21.15.121:0/3852966775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe240153db0 0x7fe2400c2670 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7fe2400c4270 tx=0x7fe2280090f0).ready entity=mon.0 client_cookie=3baf3673e985640 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:28.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.694+0000 7fe23f7fe700 1 -- 172.21.15.121:0/3852966775 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22800b520 con 0x7fe240153db0 2023-11-15T03:05:28.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.694+0000 7fe23f7fe700 1 -- 172.21.15.121:0/3852966775 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22800b680 con 0x7fe240153db0 2023-11-15T03:05:28.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.694+0000 7fe23f7fe700 1 -- 172.21.15.121:0/3852966775 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22800ba00 con 0x7fe240153db0 2023-11-15T03:05:28.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.694+0000 7fe24669b700 1 -- 172.21.15.121:0/3852966775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe240153db0 msgr2=0x7fe2400c2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:28.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.694+0000 7fe24669b700 1 --2- 172.21.15.121:0/3852966775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe240153db0 0x7fe2400c2670 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7fe2400c4270 tx=0x7fe2280090f0).stop 2023-11-15T03:05:28.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.694+0000 7fe24669b700 1 -- 172.21.15.121:0/3852966775 shutdown_connections 2023-11-15T03:05:28.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.694+0000 7fe24669b700 1 --2- 172.21.15.121:0/3852966775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe240153db0 0x7fe2400c2670 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.694+0000 7fe24669b700 1 --2- 172.21.15.121:0/3852966775 >> v2:172.21.15.190:3300/0 conn(0x7fe2401539d0 0x7fe2400c06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.694+0000 7fe24669b700 1 -- 172.21.15.121:0/3852966775 >> 172.21.15.121:0/3852966775 conn(0x7fe240062190 msgr2=0x7fe2400625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:28.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.694+0000 7fe24669b700 1 -- 172.21.15.121:0/3852966775 shutdown_connections 2023-11-15T03:05:28.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.694+0000 7fe24669b700 1 -- 172.21.15.121:0/3852966775 wait complete. 2023-11-15T03:05:28.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.698+0000 7fe24669b700 1 Processor -- start 2023-11-15T03:05:28.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.698+0000 7fe24669b700 1 -- start start 2023-11-15T03:05:28.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe24669b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2400c19c0 0x7fe240164c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:28.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe24669b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2401539d0 0x7fe240157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:28.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe24669b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe240153db0 0x7fe24014e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:28.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe24669b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2401662a0 con 0x7fe2400c19c0 2023-11-15T03:05:28.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe24669b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe240165130 con 0x7fe240153db0 2023-11-15T03:05:28.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe24669b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2401653b0 con 0x7fe2401539d0 2023-11-15T03:05:28.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe23ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2400c19c0 0x7fe240164c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:28.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe23ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2400c19c0 0x7fe240164c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59494/0 (socket says 172.21.15.121:59494) 2023-11-15T03:05:28.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe23ffff700 1 -- 172.21.15.121:0/2500568926 learned_addr learned my addr 172.21.15.121:0/2500568926 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:28.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe2377fe700 1 --2- 172.21.15.121:0/2500568926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2401539d0 0x7fe240157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:28.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe244c38700 1 --2- 172.21.15.121:0/2500568926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe240153db0 0x7fe24014e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:28.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe23ffff700 1 -- 172.21.15.121:0/2500568926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2401539d0 msgr2=0x7fe240157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:28.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe23ffff700 1 --2- 172.21.15.121:0/2500568926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2401539d0 0x7fe240157dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe23ffff700 1 -- 172.21.15.121:0/2500568926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe240153db0 msgr2=0x7fe24014e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:28.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe23ffff700 1 --2- 172.21.15.121:0/2500568926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe240153db0 0x7fe24014e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe23ffff700 1 -- 172.21.15.121:0/2500568926 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe228009370 con 0x7fe2400c19c0 2023-11-15T03:05:28.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe244c38700 1 --2- 172.21.15.121:0/2500568926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe240153db0 0x7fe24014e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:05:28.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe23ffff700 1 --2- 172.21.15.121:0/2500568926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2400c19c0 0x7fe240164c40 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7fe230009ef0 tx=0x7fe23000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:28.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe23cff9700 1 -- 172.21.15.121:0/2500568926 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23000cc50 con 0x7fe2400c19c0 2023-11-15T03:05:28.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe23cff9700 1 -- 172.21.15.121:0/2500568926 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23000f040 con 0x7fe2400c19c0 2023-11-15T03:05:28.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe24669b700 1 -- 172.21.15.121:0/2500568926 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe240165690 con 0x7fe2400c19c0 2023-11-15T03:05:28.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe23cff9700 1 -- 172.21.15.121:0/2500568926 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230013400 con 0x7fe2400c19c0 2023-11-15T03:05:28.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe24669b700 1 -- 172.21.15.121:0/2500568926 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe240165c50 con 0x7fe2400c19c0 2023-11-15T03:05:28.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.702+0000 7fe24669b700 1 -- 172.21.15.121:0/2500568926 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe24006c150 con 0x7fe2400c19c0 2023-11-15T03:05:28.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.706+0000 7fe23cff9700 1 -- 172.21.15.121:0/2500568926 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe230013770 con 0x7fe2400c19c0 2023-11-15T03:05:28.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.706+0000 7fe23cff9700 1 --2- 172.21.15.121:0/2500568926 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe22005a510 0x7fe22005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:28.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.706+0000 7fe23cff9700 1 -- 172.21.15.121:0/2500568926 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7212+0+0 (secure 0 0 0) 0x7fe230075560 con 0x7fe2400c19c0 2023-11-15T03:05:28.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.706+0000 7fe2377fe700 1 --2- 172.21.15.121:0/2500568926 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe22005a510 0x7fe22005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:28.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.710+0000 7fe2377fe700 1 --2- 172.21.15.121:0/2500568926 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe22005a510 0x7fe22005c9d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fe228009130 tx=0x7fe228017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:28.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.714+0000 7fe23cff9700 1 -- 172.21.15.121:0/2500568926 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe230048900 con 0x7fe2400c19c0 2023-11-15T03:05:28.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.910+0000 7fe24669b700 1 -- 172.21.15.121:0/2500568926 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2400c0ee0 con 0x7fe2400c19c0 2023-11-15T03:05:28.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.914+0000 7fe23cff9700 1 -- 172.21.15.121:0/2500568926 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) v1 ==== 95+0+4496 (secure 0 0 0) 0x7fe23004bf20 con 0x7fe2400c19c0 2023-11-15T03:05:28.922 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:28.922 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":150,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8271,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1529582418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1529582418},{"type":"v1","addr":"172.21.15.121:6836","nonce":1529582418}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:26.263706+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:05:28.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.914+0000 7fe24669b700 1 -- 172.21.15.121:0/2500568926 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe22005a510 msgr2=0x7fe22005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:28.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.914+0000 7fe24669b700 1 --2- 172.21.15.121:0/2500568926 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe22005a510 0x7fe22005c9d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fe228009130 tx=0x7fe228017000).stop 2023-11-15T03:05:28.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.914+0000 7fe24669b700 1 -- 172.21.15.121:0/2500568926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2400c19c0 msgr2=0x7fe240164c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:28.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.914+0000 7fe24669b700 1 --2- 172.21.15.121:0/2500568926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2400c19c0 0x7fe240164c40 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7fe230009ef0 tx=0x7fe23000b040).stop 2023-11-15T03:05:28.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.914+0000 7fe24669b700 1 -- 172.21.15.121:0/2500568926 shutdown_connections 2023-11-15T03:05:28.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.914+0000 7fe24669b700 1 --2- 172.21.15.121:0/2500568926 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe22005a510 0x7fe22005c9d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.914+0000 7fe24669b700 1 --2- 172.21.15.121:0/2500568926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe240153db0 0x7fe24014e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.914+0000 7fe24669b700 1 --2- 172.21.15.121:0/2500568926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2401539d0 0x7fe240157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.914+0000 7fe24669b700 1 --2- 172.21.15.121:0/2500568926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2400c19c0 0x7fe240164c40 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:28.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.914+0000 7fe24669b700 1 -- 172.21.15.121:0/2500568926 >> 172.21.15.121:0/2500568926 conn(0x7fe240062190 msgr2=0x7fe2400c22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:28.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.914+0000 7fe24669b700 1 -- 172.21.15.121:0/2500568926 shutdown_connections 2023-11-15T03:05:28.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:28.914+0000 7fe24669b700 1 -- 172.21.15.121:0/2500568926 wait complete. 2023-11-15T03:05:28.927 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 150 2023-11-15T03:05:28.944 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 100.00000000' led to DAMAGED state 2023-11-15T03:05:28.944 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_sessionmap' 2023-11-15T03:05:28.945 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:05:28.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:05:28.946 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:05:28.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:05:28.966 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:05:28.967 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:05:29.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.166+0000 7fc93f89b700 1 Processor -- start 2023-11-15T03:05:29.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.170+0000 7fc93f89b700 1 -- start start 2023-11-15T03:05:29.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.170+0000 7fc93f89b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc938151920 0x7fc9380bb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:29.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.170+0000 7fc93f89b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc938067150 0x7fc9380aba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:29.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.170+0000 7fc93f89b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc938155de0 con 0x7fc938156270 2023-11-15T03:05:29.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.170+0000 7fc93f89b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9380c4060 con 0x7fc938151920 2023-11-15T03:05:29.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.170+0000 7fc93f89b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380c3aa0 con 0x7fc938067150 2023-11-15T03:05:29.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.170+0000 7fc93ce36700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc938067150 0x7fc9380aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:29.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.170+0000 7fc93de38700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc938156270 0x7fc9380658a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56464/0 (socket says 172.21.15.121:56464) 2023-11-15T03:05:29.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.170+0000 7fc93de38700 1 -- 172.21.15.121:0/1592550335 learned_addr learned my addr 172.21.15.121:0/1592550335 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:29.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.170+0000 7fc93d637700 1 --2- 172.21.15.121:0/1592550335 >> v2:172.21.15.121:3300/0 conn(0x7fc938151920 0x7fc9380bb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:29.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.170+0000 7fc92ffff700 1 -- 172.21.15.121:0/1592550335 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923489383 0 0) 0x7fc938155de0 con 0x7fc938156270 2023-11-15T03:05:29.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.170+0000 7fc93ce36700 1 -- 172.21.15.121:0/1592550335 >> v2:172.21.15.121:3300/0 conn(0x7fc938151920 msgr2=0x7fc9380bb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:29.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.170+0000 7fc93ce36700 1 --2- 172.21.15.121:0/1592550335 >> v2:172.21.15.121:3300/0 conn(0x7fc938151920 0x7fc9380bb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:29.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.170+0000 7fc93ce36700 1 -- 172.21.15.121:0/1592550335 >> v1:172.21.15.121:6789/0 conn(0x7fc938156270 legacy=0x7fc9380658a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:29.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.170+0000 7fc93ce36700 1 -- 172.21.15.121:0/1592550335 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9380c3d80 con 0x7fc938067150 2023-11-15T03:05:29.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.174+0000 7fc93ce36700 1 --2- 172.21.15.121:0/1592550335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc938067150 0x7fc9380aba40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fc928009760 tx=0x7fc928009060).ready entity=mon.2 client_cookie=b54045b3c8420924 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:29.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.174+0000 7fc92ffff700 1 -- 172.21.15.121:0/1592550335 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92800f510 con 0x7fc938067150 2023-11-15T03:05:29.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.174+0000 7fc92ffff700 1 -- 172.21.15.121:0/1592550335 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92800f670 con 0x7fc938067150 2023-11-15T03:05:29.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.174+0000 7fc92ffff700 1 -- 172.21.15.121:0/1592550335 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92800f9f0 con 0x7fc938067150 2023-11-15T03:05:29.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.174+0000 7fc93f89b700 1 -- 172.21.15.121:0/1592550335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc938067150 msgr2=0x7fc9380aba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:29.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.174+0000 7fc93f89b700 1 --2- 172.21.15.121:0/1592550335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc938067150 0x7fc9380aba40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fc928009760 tx=0x7fc928009060).stop 2023-11-15T03:05:29.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.174+0000 7fc93f89b700 1 -- 172.21.15.121:0/1592550335 shutdown_connections 2023-11-15T03:05:29.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.174+0000 7fc93f89b700 1 --2- 172.21.15.121:0/1592550335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc938067150 0x7fc9380aba40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:29.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.174+0000 7fc93f89b700 1 --2- 172.21.15.121:0/1592550335 >> v2:172.21.15.121:3300/0 conn(0x7fc938151920 0x7fc9380bb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:29.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.174+0000 7fc93f89b700 1 -- 172.21.15.121:0/1592550335 >> 172.21.15.121:0/1592550335 conn(0x7fc938067a60 msgr2=0x7fc938067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:29.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.174+0000 7fc93f89b700 1 -- 172.21.15.121:0/1592550335 shutdown_connections 2023-11-15T03:05:29.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.174+0000 7fc93f89b700 1 -- 172.21.15.121:0/1592550335 wait complete. 2023-11-15T03:05:29.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.178+0000 7fc93f89b700 1 Processor -- start 2023-11-15T03:05:29.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.178+0000 7fc93f89b700 1 -- start start 2023-11-15T03:05:29.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.178+0000 7fc93f89b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc938067150 0x7fc9380bd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:29.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.178+0000 7fc93f89b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc938151920 0x7fc938162ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:29.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.178+0000 7fc93f89b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc938156270 0x7fc938149ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:29.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.178+0000 7fc93f89b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc938166530 con 0x7fc938067150 2023-11-15T03:05:29.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.178+0000 7fc93f89b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc938162fb0 con 0x7fc938156270 2023-11-15T03:05:29.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.178+0000 7fc93f89b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc938165640 con 0x7fc938151920 2023-11-15T03:05:29.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.178+0000 7fc93d637700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc938067150 0x7fc9380bd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:29.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.178+0000 7fc93d637700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc938067150 0x7fc9380bd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59512/0 (socket says 172.21.15.121:59512) 2023-11-15T03:05:29.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.178+0000 7fc93d637700 1 -- 172.21.15.121:0/4206991986 learned_addr learned my addr 172.21.15.121:0/4206991986 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:29.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.178+0000 7fc93ce36700 1 --2- 172.21.15.121:0/4206991986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc938151920 0x7fc938162ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:29.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.178+0000 7fc93de38700 1 --2- 172.21.15.121:0/4206991986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc938156270 0x7fc938149ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:29.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.178+0000 7fc93d637700 1 -- 172.21.15.121:0/4206991986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc938151920 msgr2=0x7fc938162ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:29.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.178+0000 7fc93d637700 1 --2- 172.21.15.121:0/4206991986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc938151920 0x7fc938162ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:29.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.178+0000 7fc93d637700 1 -- 172.21.15.121:0/4206991986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc938156270 msgr2=0x7fc938149ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:29.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.178+0000 7fc93d637700 1 --2- 172.21.15.121:0/4206991986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc938156270 0x7fc938149ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:29.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.178+0000 7fc93d637700 1 -- 172.21.15.121:0/4206991986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9280092a0 con 0x7fc938067150 2023-11-15T03:05:29.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.178+0000 7fc93de38700 1 --2- 172.21.15.121:0/4206991986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc938156270 0x7fc938149ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:29.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.182+0000 7fc93d637700 1 --2- 172.21.15.121:0/4206991986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc938067150 0x7fc9380bd740 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7fc934009ef0 tx=0x7fc93400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:29.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.182+0000 7fc92d7fa700 1 -- 172.21.15.121:0/4206991986 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93400cc20 con 0x7fc938067150 2023-11-15T03:05:29.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.182+0000 7fc92d7fa700 1 -- 172.21.15.121:0/4206991986 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93400f040 con 0x7fc938067150 2023-11-15T03:05:29.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.182+0000 7fc92d7fa700 1 -- 172.21.15.121:0/4206991986 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934013400 con 0x7fc938067150 2023-11-15T03:05:29.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.182+0000 7fc93f89b700 1 -- 172.21.15.121:0/4206991986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc938165920 con 0x7fc938067150 2023-11-15T03:05:29.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.182+0000 7fc93f89b700 1 -- 172.21.15.121:0/4206991986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc938165ee0 con 0x7fc938067150 2023-11-15T03:05:29.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.182+0000 7fc93f89b700 1 -- 172.21.15.121:0/4206991986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc938047060 con 0x7fc938067150 2023-11-15T03:05:29.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.182+0000 7fc92d7fa700 1 -- 172.21.15.121:0/4206991986 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc934013770 con 0x7fc938067150 2023-11-15T03:05:29.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.186+0000 7fc92d7fa700 1 --2- 172.21.15.121:0/4206991986 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc91c05a510 0x7fc91c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:29.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.186+0000 7fc92d7fa700 1 -- 172.21.15.121:0/4206991986 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7212+0+0 (secure 0 0 0) 0x7fc934075470 con 0x7fc938067150 2023-11-15T03:05:29.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.186+0000 7fc93ce36700 1 --2- 172.21.15.121:0/4206991986 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc91c05a510 0x7fc91c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:29.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.186+0000 7fc93ce36700 1 --2- 172.21.15.121:0/4206991986 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc91c05a510 0x7fc91c05c9d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7fc938067590 tx=0x7fc928017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:29.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.190+0000 7fc92d7fa700 1 -- 172.21.15.121:0/4206991986 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc934048810 con 0x7fc938067150 2023-11-15T03:05:29.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:29.390+0000 7fc93f89b700 1 -- 172.21.15.121:0/4206991986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc938068b20 con 0x7fc938067150 2023-11-15T03:05:30.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.294+0000 7fc92d7fa700 1 -- 172.21.15.121:0/4206991986 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v151) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc93404be30 con 0x7fc938067150 2023-11-15T03:05:30.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.298+0000 7fc93f89b700 1 -- 172.21.15.121:0/4206991986 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc91c05a510 msgr2=0x7fc91c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:30.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.298+0000 7fc93f89b700 1 --2- 172.21.15.121:0/4206991986 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc91c05a510 0x7fc91c05c9d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7fc938067590 tx=0x7fc928017000).stop 2023-11-15T03:05:30.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.298+0000 7fc93f89b700 1 -- 172.21.15.121:0/4206991986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc938067150 msgr2=0x7fc9380bd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:30.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.298+0000 7fc93f89b700 1 --2- 172.21.15.121:0/4206991986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc938067150 0x7fc9380bd740 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7fc934009ef0 tx=0x7fc93400b040).stop 2023-11-15T03:05:30.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.298+0000 7fc93f89b700 1 -- 172.21.15.121:0/4206991986 shutdown_connections 2023-11-15T03:05:30.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.298+0000 7fc93f89b700 1 --2- 172.21.15.121:0/4206991986 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc91c05a510 0x7fc91c05c9d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:30.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.298+0000 7fc93f89b700 1 --2- 172.21.15.121:0/4206991986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc938156270 0x7fc938149ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:30.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.298+0000 7fc93f89b700 1 --2- 172.21.15.121:0/4206991986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc938151920 0x7fc938162ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:30.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.298+0000 7fc93f89b700 1 --2- 172.21.15.121:0/4206991986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc938067150 0x7fc9380bd740 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:30.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.298+0000 7fc93f89b700 1 -- 172.21.15.121:0/4206991986 >> 172.21.15.121:0/4206991986 conn(0x7fc938067a60 msgr2=0x7fc93814b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:30.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.298+0000 7fc93f89b700 1 -- 172.21.15.121:0/4206991986 shutdown_connections 2023-11-15T03:05:30.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.298+0000 7fc93f89b700 1 -- 172.21.15.121:0/4206991986 wait complete. 2023-11-15T03:05:30.310 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:05:30.330 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:05:30.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.490+0000 7fc0ca969700 1 Processor -- start 2023-11-15T03:05:30.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.490+0000 7fc0ca969700 1 -- start start 2023-11-15T03:05:30.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0ca969700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c40aa130 0x7fc0c4062710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:30.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0ca969700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc0c40ac150 0x7fc0c414ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:30.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0ca969700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c40aac50 con 0x7fc0c4062330 2023-11-15T03:05:30.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0ca969700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c40aa6b0 con 0x7fc0c40ac150 2023-11-15T03:05:30.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0ca969700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c40aa970 con 0x7fc0c40aa130 2023-11-15T03:05:30.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0c8f06700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc0c40ac150 0x7fc0c414ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:30.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0c8f06700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc0c40ac150 0x7fc0c414ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59526/0 (socket says 172.21.15.121:59526) 2023-11-15T03:05:30.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0c8f06700 1 -- 172.21.15.121:0/4291566836 learned_addr learned my addr 172.21.15.121:0/4291566836 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:30.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0bbfff700 1 --2- 172.21.15.121:0/4291566836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c40aa130 0x7fc0c4062710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:30.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0c8f06700 1 -- 172.21.15.121:0/4291566836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c40aa130 msgr2=0x7fc0c4062710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:30.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0c8f06700 1 --2- 172.21.15.121:0/4291566836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c40aa130 0x7fc0c4062710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:30.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0c8f06700 1 -- 172.21.15.121:0/4291566836 >> v1:172.21.15.121:6789/0 conn(0x7fc0c4062330 legacy=0x7fc0c4154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:30.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0c8f06700 1 -- 172.21.15.121:0/4291566836 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c4154410 con 0x7fc0c40ac150 2023-11-15T03:05:30.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0bbfff700 1 --2- 172.21.15.121:0/4291566836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c40aa130 0x7fc0c4062710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:05:30.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0c8f06700 1 --2- 172.21.15.121:0/4291566836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c40ac150 0x7fc0c414ddd0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7fc0bc009f20 tx=0x7fc0bc009840).ready entity=mon.0 client_cookie=e0c2c360a24a9414 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:30.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0c37fe700 1 -- 172.21.15.121:0/4291566836 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc00e070 con 0x7fc0c40ac150 2023-11-15T03:05:30.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0c37fe700 1 -- 172.21.15.121:0/4291566836 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc00ba80 con 0x7fc0c40ac150 2023-11-15T03:05:30.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0c37fe700 1 -- 172.21.15.121:0/4291566836 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc00be00 con 0x7fc0c40ac150 2023-11-15T03:05:30.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0ca969700 1 -- 172.21.15.121:0/4291566836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c40ac150 msgr2=0x7fc0c414ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:30.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.494+0000 7fc0ca969700 1 --2- 172.21.15.121:0/4291566836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c40ac150 0x7fc0c414ddd0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7fc0bc009f20 tx=0x7fc0bc009840).stop 2023-11-15T03:05:30.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.498+0000 7fc0ca969700 1 -- 172.21.15.121:0/4291566836 shutdown_connections 2023-11-15T03:05:30.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.498+0000 7fc0ca969700 1 --2- 172.21.15.121:0/4291566836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c40ac150 0x7fc0c414ddd0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:30.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.498+0000 7fc0ca969700 1 --2- 172.21.15.121:0/4291566836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c40aa130 0x7fc0c4062710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:30.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.498+0000 7fc0ca969700 1 -- 172.21.15.121:0/4291566836 >> 172.21.15.121:0/4291566836 conn(0x7fc0c4067a60 msgr2=0x7fc0c4067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:30.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.498+0000 7fc0ca969700 1 -- 172.21.15.121:0/4291566836 shutdown_connections 2023-11-15T03:05:30.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.498+0000 7fc0ca969700 1 -- 172.21.15.121:0/4291566836 wait complete. 2023-11-15T03:05:30.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.498+0000 7fc0ca969700 1 Processor -- start 2023-11-15T03:05:30.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.502+0000 7fc0ca969700 1 -- start start 2023-11-15T03:05:30.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.502+0000 7fc0ca969700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c4062330 0x7fc0c4149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:30.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.502+0000 7fc0ca969700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c40aa130 0x7fc0c4164c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:30.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.502+0000 7fc0ca969700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c40ac150 0x7fc0c40c2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:30.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.502+0000 7fc0ca969700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c40c1f10 con 0x7fc0c40ac150 2023-11-15T03:05:30.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.502+0000 7fc0ca969700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c414a0c0 con 0x7fc0c40aa130 2023-11-15T03:05:30.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.502+0000 7fc0ca969700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c4165110 con 0x7fc0c4062330 2023-11-15T03:05:30.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.502+0000 7fc0c8f06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c40ac150 0x7fc0c40c2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:30.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.502+0000 7fc0c8f06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c40ac150 0x7fc0c40c2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59536/0 (socket says 172.21.15.121:59536) 2023-11-15T03:05:30.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.502+0000 7fc0c8f06700 1 -- 172.21.15.121:0/1653638554 learned_addr learned my addr 172.21.15.121:0/1653638554 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:30.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.502+0000 7fc0bbfff700 1 --2- 172.21.15.121:0/1653638554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c40aa130 0x7fc0c4164c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:30.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.502+0000 7fc0c3fff700 1 --2- 172.21.15.121:0/1653638554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c4062330 0x7fc0c4149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:30.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.502+0000 7fc0c8f06700 1 -- 172.21.15.121:0/1653638554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c4062330 msgr2=0x7fc0c4149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:30.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.502+0000 7fc0c8f06700 1 --2- 172.21.15.121:0/1653638554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c4062330 0x7fc0c4149bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:30.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.502+0000 7fc0c8f06700 1 -- 172.21.15.121:0/1653638554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c40aa130 msgr2=0x7fc0c4164c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:30.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.502+0000 7fc0c8f06700 1 --2- 172.21.15.121:0/1653638554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c40aa130 0x7fc0c4164c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:30.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.502+0000 7fc0c8f06700 1 -- 172.21.15.121:0/1653638554 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc009ae0 con 0x7fc0c40ac150 2023-11-15T03:05:30.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.502+0000 7fc0c8f06700 1 --2- 172.21.15.121:0/1653638554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c40ac150 0x7fc0c40c2ac0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0x7fc0c4063db0 tx=0x7fc0bc009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:30.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.506+0000 7fc0c0ff9700 1 -- 172.21.15.121:0/1653638554 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc01e480 con 0x7fc0c40ac150 2023-11-15T03:05:30.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.506+0000 7fc0c0ff9700 1 -- 172.21.15.121:0/1653638554 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc01e5e0 con 0x7fc0c40ac150 2023-11-15T03:05:30.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.506+0000 7fc0ca969700 1 -- 172.21.15.121:0/1653638554 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c40c1020 con 0x7fc0c40ac150 2023-11-15T03:05:30.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.506+0000 7fc0c0ff9700 1 -- 172.21.15.121:0/1653638554 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc01e980 con 0x7fc0c40ac150 2023-11-15T03:05:30.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.506+0000 7fc0ca969700 1 -- 172.21.15.121:0/1653638554 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c40c15e0 con 0x7fc0c40ac150 2023-11-15T03:05:30.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.506+0000 7fc0ca969700 1 -- 172.21.15.121:0/1653638554 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c4047060 con 0x7fc0c40ac150 2023-11-15T03:05:30.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.506+0000 7fc0c0ff9700 1 -- 172.21.15.121:0/1653638554 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc0bc0123c0 con 0x7fc0c40ac150 2023-11-15T03:05:30.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.510+0000 7fc0c0ff9700 1 --2- 172.21.15.121:0/1653638554 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc0a405a510 0x7fc0a405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:30.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.510+0000 7fc0c0ff9700 1 -- 172.21.15.121:0/1653638554 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7212+0+0 (secure 0 0 0) 0x7fc0bc0770f0 con 0x7fc0c40ac150 2023-11-15T03:05:30.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.510+0000 7fc0c3fff700 1 --2- 172.21.15.121:0/1653638554 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc0a405a510 0x7fc0a405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:30.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.510+0000 7fc0c3fff700 1 --2- 172.21.15.121:0/1653638554 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc0a405a510 0x7fc0a405c9d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fc0b40036d0 tx=0x7fc0b4009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:30.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.518+0000 7fc0c0ff9700 1 -- 172.21.15.121:0/1653638554 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc0bc04a3b0 con 0x7fc0c40ac150 2023-11-15T03:05:30.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:30.734+0000 7fc0ca969700 1 -- 172.21.15.121:0/1653638554 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fc0c41546f0 con 0x7fc0c40ac150 2023-11-15T03:05:31.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.306+0000 7fc0c0ff9700 1 -- 172.21.15.121:0/1653638554 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v152) v1 ==== 101+0+0 (secure 0 0 0) 0x7fc0bc01edb0 con 0x7fc0c40ac150 2023-11-15T03:05:31.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.310+0000 7fc0ca969700 1 -- 172.21.15.121:0/1653638554 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc0a405a510 msgr2=0x7fc0a405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.310+0000 7fc0ca969700 1 --2- 172.21.15.121:0/1653638554 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc0a405a510 0x7fc0a405c9d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fc0b40036d0 tx=0x7fc0b4009040).stop 2023-11-15T03:05:31.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.310+0000 7fc0ca969700 1 -- 172.21.15.121:0/1653638554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c40ac150 msgr2=0x7fc0c40c2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.310+0000 7fc0ca969700 1 --2- 172.21.15.121:0/1653638554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c40ac150 0x7fc0c40c2ac0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0x7fc0c4063db0 tx=0x7fc0bc009840).stop 2023-11-15T03:05:31.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.314+0000 7fc0ca969700 1 -- 172.21.15.121:0/1653638554 shutdown_connections 2023-11-15T03:05:31.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.314+0000 7fc0ca969700 1 --2- 172.21.15.121:0/1653638554 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc0a405a510 0x7fc0a405c9d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.314+0000 7fc0ca969700 1 --2- 172.21.15.121:0/1653638554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0c40ac150 0x7fc0c40c2ac0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.314+0000 7fc0ca969700 1 --2- 172.21.15.121:0/1653638554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c40aa130 0x7fc0c4164c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.314+0000 7fc0ca969700 1 --2- 172.21.15.121:0/1653638554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c4062330 0x7fc0c4149bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.314+0000 7fc0ca969700 1 -- 172.21.15.121:0/1653638554 >> 172.21.15.121:0/1653638554 conn(0x7fc0c4067a60 msgr2=0x7fc0c40c5290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:31.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.314+0000 7fc0ca969700 1 -- 172.21.15.121:0/1653638554 shutdown_connections 2023-11-15T03:05:31.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.314+0000 7fc0ca969700 1 -- 172.21.15.121:0/1653638554 wait complete. 2023-11-15T03:05:31.324 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:05:31.344 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:05:31.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.390+0000 7ff2c3edad00 1 Processor -- start 2023-11-15T03:05:31.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.390+0000 7ff2c3edad00 1 -- start start 2023-11-15T03:05:31.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2c3edad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f4893b860 0x556f4897e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2c3edad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f48a10ec0 0x556f48981bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2c3edad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556f48a112a0 0x556f489663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2c3edad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f488b1220 con 0x556f48a10ec0 2023-11-15T03:05:31.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2c3edad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f48982970 con 0x556f4893b860 2023-11-15T03:05:31.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2c3edad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f48982e00 con 0x556f48a112a0 2023-11-15T03:05:31.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2b8f11700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f48a10ec0 0x556f48981bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2b8f11700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f48a10ec0 0x556f48981bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59540/0 (socket says 172.21.15.121:59540) 2023-11-15T03:05:31.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2b8f11700 1 -- 172.21.15.121:0/2172962793 learned_addr learned my addr 172.21.15.121:0/2172962793 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:31.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2b8f11700 1 -- 172.21.15.121:0/2172962793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556f48a112a0 msgr2=0x556f489663c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:31.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2b8f11700 1 --2- 172.21.15.121:0/2172962793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556f48a112a0 0x556f489663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:05:31.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2b8f11700 1 -- 172.21.15.121:0/2172962793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f4893b860 msgr2=0x556f4897e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:31.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2b8f11700 1 --2- 172.21.15.121:0/2172962793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f4893b860 0x556f4897e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:05:31.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2b8f11700 1 -- 172.21.15.121:0/2172962793 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b4005cf0 con 0x556f48a10ec0 2023-11-15T03:05:31.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2b8f11700 1 --2- 172.21.15.121:0/2172962793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f48a10ec0 0x556f48981bd0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x556f48925c90 tx=0x7ff2ac015000).ready entity=mon.0 client_cookie=247eee9980ca2d9b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:31.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2bbfff700 1 -- 172.21.15.121:0/2172962793 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac007610 con 0x556f48a10ec0 2023-11-15T03:05:31.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2bbfff700 1 -- 172.21.15.121:0/2172962793 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ac005330 con 0x556f48a10ec0 2023-11-15T03:05:31.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2bbfff700 1 -- 172.21.15.121:0/2172962793 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac0078e0 con 0x556f48a10ec0 2023-11-15T03:05:31.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2c3edad00 1 -- 172.21.15.121:0/2172962793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f48a10ec0 msgr2=0x556f48981bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/2172962793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f48a10ec0 0x556f48981bd0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x556f48925c90 tx=0x7ff2ac015000).stop 2023-11-15T03:05:31.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2c3edad00 1 -- 172.21.15.121:0/2172962793 shutdown_connections 2023-11-15T03:05:31.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/2172962793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556f48a112a0 0x556f489663c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:05:31.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/2172962793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f48a10ec0 0x556f48981bd0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/2172962793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f4893b860 0x556f4897e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:05:31.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2c3edad00 1 -- 172.21.15.121:0/2172962793 >> 172.21.15.121:0/2172962793 conn(0x556f489397d0 msgr2=0x556f48a14510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:31.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2c3edad00 1 -- 172.21.15.121:0/2172962793 shutdown_connections 2023-11-15T03:05:31.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2c3edad00 1 -- 172.21.15.121:0/2172962793 wait complete. 2023-11-15T03:05:31.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2c3edad00 1 Processor -- start 2023-11-15T03:05:31.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.394+0000 7ff2c3edad00 1 -- start start 2023-11-15T03:05:31.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2c3edad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f4893b860 0x556f48a05450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2c3edad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556f48a10ec0 0x556f489807d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2c3edad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f48a112a0 0x556f48a0a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2c3edad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f48a13c70 con 0x556f48a112a0 2023-11-15T03:05:31.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2c3edad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f48a13060 con 0x556f4893b860 2023-11-15T03:05:31.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2c3edad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f48a13340 con 0x556f48a10ec0 2023-11-15T03:05:31.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2c1f13700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f48a112a0 0x556f48a0a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2b8f11700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556f48a10ec0 0x556f489807d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2c1f13700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f48a112a0 0x556f48a0a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59554/0 (socket says 172.21.15.121:59554) 2023-11-15T03:05:31.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2b8f11700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556f48a10ec0 0x556f489807d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:42006/0 (socket says 172.21.15.121:42006) 2023-11-15T03:05:31.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2c1712700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f4893b860 0x556f48a05450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 learned_addr learned my addr 172.21.15.121:0/71612221 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:31.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556f48a10ec0 msgr2=0x556f489807d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2c1f13700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556f48a10ec0 0x556f489807d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f4893b860 msgr2=0x556f48a05450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2c1f13700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f4893b860 0x556f48a05450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f488daa20 con 0x556f48a112a0 2023-11-15T03:05:31.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2c1f13700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f48a112a0 0x556f48a0a870 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0x7ff2bc006f90 tx=0x7ff2bc0068f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:31.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2ba7fc700 1 -- 172.21.15.121:0/71612221 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc0062e0 con 0x556f48a112a0 2023-11-15T03:05:31.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2ba7fc700 1 -- 172.21.15.121:0/71612221 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2bc00ba20 con 0x556f48a112a0 2023-11-15T03:05:31.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2ba7fc700 1 -- 172.21.15.121:0/71612221 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc00bde0 con 0x556f48a112a0 2023-11-15T03:05:31.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ac005cf0 con 0x556f48a112a0 2023-11-15T03:05:31.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.398+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f488da770 con 0x556f48a112a0 2023-11-15T03:05:31.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.402+0000 7ff2ba7fc700 1 -- 172.21.15.121:0/71612221 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff2bc004750 con 0x556f48a112a0 2023-11-15T03:05:31.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.402+0000 7ff2ba7fc700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff2a4058ff0 0x7ff2a405b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.406+0000 7ff2ba7fc700 1 -- 172.21.15.121:0/71612221 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7212+0+0 (secure 0 0 0) 0x7ff2bc072e20 con 0x556f48a112a0 2023-11-15T03:05:31.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.406+0000 7ff2c1712700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff2a4058ff0 0x7ff2a405b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.435 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:05:31.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.406+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x556f48a12af0 0x556f48a0eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.406+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a0efa0 con 0x556f48a12af0 2023-11-15T03:05:31.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.406+0000 7ff2c1712700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff2a4058ff0 0x7ff2a405b4b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7ff2b4005fd0 tx=0x7ff2b400c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:31.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.406+0000 7ff2b8f11700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x556f48a12af0 0x556f48a0eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.406+0000 7ff2b8f11700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x556f48a12af0 0x556f48a0eab0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:31.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.410+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v105'125 uv118 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a12af0 2023-11-15T03:05:31.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.410+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a0f380 con 0x556f48a12af0 2023-11-15T03:05:31.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.410+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v105'126 uv126 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a12af0 2023-11-15T03:05:31.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.410+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a2e220 con 0x556f48a12af0 2023-11-15T03:05:31.438 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:05:31.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.414+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v105'127 uv127 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a12af0 2023-11-15T03:05:31.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.414+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a2f5e0 con 0x556f48a12af0 2023-11-15T03:05:31.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.414+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v105'128 uv128 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a12af0 2023-11-15T03:05:31.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.414+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a2f990 con 0x556f48a12af0 2023-11-15T03:05:31.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.418+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v105'129 uv129 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a12af0 2023-11-15T03:05:31.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.418+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a2e5d0 con 0x556f48a12af0 2023-11-15T03:05:31.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.418+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v105'130 uv130 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a12af0 2023-11-15T03:05:31.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.418+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a2e9d0 con 0x556f48a12af0 2023-11-15T03:05:31.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.422+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v105'131 uv124 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a12af0 2023-11-15T03:05:31.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.422+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a2ed80 con 0x556f48a12af0 2023-11-15T03:05:31.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.422+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v105'132 uv132 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a12af0 2023-11-15T03:05:31.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.422+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a2ed80 con 0x556f48a12af0 2023-11-15T03:05:31.445 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:05:31.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.426+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v105'133 uv133 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a12af0 2023-11-15T03:05:31.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.426+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a2ed80 con 0x556f48a12af0 2023-11-15T03:05:31.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.426+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v105'134 uv134 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a12af0 2023-11-15T03:05:31.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.426+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a2ed80 con 0x556f48a12af0 2023-11-15T03:05:31.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.430+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v105'135 uv135 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a12af0 2023-11-15T03:05:31.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.430+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a2ed80 con 0x556f48a12af0 2023-11-15T03:05:31.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.430+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v105'136 uv136 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a12af0 2023-11-15T03:05:31.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.430+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x556f48a2f180 0x556f48a32260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.430+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a2ed80 con 0x556f48a2f180 2023-11-15T03:05:31.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.434+0000 7ff2c1712700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x556f48a2f180 0x556f48a32260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.434+0000 7ff2c1712700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x556f48a2f180 0x556f48a32260 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:31.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.438+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v105'195 uv182 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.438+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a32750 con 0x556f48a2f180 2023-11-15T03:05:31.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.442+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v105'196 uv196 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.442+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a32b00 con 0x556f48a2f180 2023-11-15T03:05:31.453 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:05:31.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.442+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v105'197 uv197 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.442+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a33ec0 con 0x556f48a2f180 2023-11-15T03:05:31.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.446+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v105'198 uv198 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.446+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a34270 con 0x556f48a2f180 2023-11-15T03:05:31.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.446+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v105'199 uv199 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.446+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a32eb0 con 0x556f48a2f180 2023-11-15T03:05:31.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.450+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v105'200 uv200 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.450+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a33800 con 0x556f48a2f180 2023-11-15T03:05:31.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.454+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v105'201 uv201 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.454+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a34620 con 0x556f48a2f180 2023-11-15T03:05:31.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.454+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v105'202 uv188 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.454+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a34620 con 0x556f48a2f180 2023-11-15T03:05:31.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.458+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v105'203 uv203 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.458+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a34620 con 0x556f48a2f180 2023-11-15T03:05:31.470 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:05:31.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.462+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v105'204 uv204 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.462+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a35630 con 0x556f48a2f180 2023-11-15T03:05:31.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.462+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v105'205 uv205 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.462+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a35630 con 0x556f48a2f180 2023-11-15T03:05:31.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.466+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v105'206 uv206 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.466+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a34620 con 0x556f48a2f180 2023-11-15T03:05:31.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.470+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v105'207 uv207 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.470+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a34620 con 0x556f48a2f180 2023-11-15T03:05:31.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.470+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v105'208 uv194 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.470+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a34620 con 0x556f48a2f180 2023-11-15T03:05:31.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.474+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v105'209 uv209 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.474+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a34620 con 0x556f48a2f180 2023-11-15T03:05:31.486 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:05:31.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.478+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v105'210 uv210 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.478+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a35630 con 0x556f48a2f180 2023-11-15T03:05:31.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.478+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v105'211 uv211 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.478+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a35630 con 0x556f48a2f180 2023-11-15T03:05:31.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.482+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v105'212 uv212 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.482+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a34620 con 0x556f48a2f180 2023-11-15T03:05:31.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.486+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v105'213 uv213 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.486+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x556f48a34b30 0x556f48a36f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.486+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a37410 con 0x556f48a34b30 2023-11-15T03:05:31.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.486+0000 7ff2b8f11700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x556f48a34b30 0x556f48a36f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.486+0000 7ff2b8f11700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x556f48a34b30 0x556f48a36f20 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:31.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.494+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v105'107 uv101 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a34b30 2023-11-15T03:05:31.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.494+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a37450 con 0x556f48a34b30 2023-11-15T03:05:31.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.494+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v105'108 uv108 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a34b30 2023-11-15T03:05:31.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.494+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a37850 con 0x556f48a34b30 2023-11-15T03:05:31.507 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:05:31.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.498+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v105'109 uv109 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a34b30 2023-11-15T03:05:31.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.498+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a37850 con 0x556f48a34b30 2023-11-15T03:05:31.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.502+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v105'110 uv110 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a34b30 2023-11-15T03:05:31.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.502+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a37c50 con 0x556f48a34b30 2023-11-15T03:05:31.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.510+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v105'111 uv111 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a34b30 2023-11-15T03:05:31.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.510+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a37c50 con 0x556f48a34b30 2023-11-15T03:05:31.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.510+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v105'112 uv106 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a34b30 2023-11-15T03:05:31.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.510+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a37850 con 0x556f48a34b30 2023-11-15T03:05:31.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.514+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v105'113 uv113 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a34b30 2023-11-15T03:05:31.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.514+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a37850 con 0x556f48a34b30 2023-11-15T03:05:31.526 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:05:31.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.518+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v105'114 uv114 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a34b30 2023-11-15T03:05:31.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.518+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a37850 con 0x556f48a34b30 2023-11-15T03:05:31.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.522+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v105'115 uv115 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a34b30 2023-11-15T03:05:31.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.522+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a33330 con 0x556f48a34b30 2023-11-15T03:05:31.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.522+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v105'116 uv116 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a34b30 2023-11-15T03:05:31.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.522+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556f48a33330 0x556f48a3a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.522+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a3a820 con 0x556f48a33330 2023-11-15T03:05:31.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.522+0000 7ff2c1f13700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556f48a33330 0x556f48a3a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.526+0000 7ff2c1f13700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556f48a33330 0x556f48a3a260 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:31.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.526+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v105'134 uv126 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.526+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a3ad70 con 0x556f48a33330 2023-11-15T03:05:31.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.530+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v105'135 uv135 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.530+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a3b220 con 0x556f48a33330 2023-11-15T03:05:31.543 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:05:31.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.534+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v105'136 uv136 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.534+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a37930 con 0x556f48a33330 2023-11-15T03:05:31.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.538+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v105'137 uv137 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.538+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a3fca0 con 0x556f48a33330 2023-11-15T03:05:31.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.542+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v105'138 uv138 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.542+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a3f830 con 0x556f48a33330 2023-11-15T03:05:31.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.542+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v105'139 uv132 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.542+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a3bb10 con 0x556f48a33330 2023-11-15T03:05:31.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.546+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v105'140 uv140 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.546+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a3bfd0 con 0x556f48a33330 2023-11-15T03:05:31.560 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:05:31.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.550+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v105'141 uv141 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.550+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a3bf90 con 0x556f48a33330 2023-11-15T03:05:31.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.554+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v105'142 uv142 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.554+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a3d430 con 0x556f48a33330 2023-11-15T03:05:31.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.554+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v105'143 uv143 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.554+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a3c010 con 0x556f48a33330 2023-11-15T03:05:31.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.558+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v105'144 uv144 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.558+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x556f48a3ca90 0x556f48a3ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.558+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a3f370 con 0x556f48a3ca90 2023-11-15T03:05:31.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.558+0000 7ff2c1712700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x556f48a3ca90 0x556f48a3ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.562+0000 7ff2c1712700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x556f48a3ca90 0x556f48a3ee80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:31.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.562+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v105'63 uv62 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a3ca90 2023-11-15T03:05:31.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.562+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a3f370 con 0x556f48a3ca90 2023-11-15T03:05:31.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.566+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v105'64 uv64 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a3ca90 2023-11-15T03:05:31.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.566+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a3f370 con 0x556f48a3ca90 2023-11-15T03:05:31.578 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:05:31.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.570+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v105'65 uv65 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a3ca90 2023-11-15T03:05:31.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.570+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a410b0 con 0x556f48a3ca90 2023-11-15T03:05:31.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.570+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v105'66 uv66 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a3ca90 2023-11-15T03:05:31.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.570+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a410b0 con 0x556f48a3ca90 2023-11-15T03:05:31.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.574+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v105'67 uv67 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a3ca90 2023-11-15T03:05:31.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.574+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a400a0 con 0x556f48a3ca90 2023-11-15T03:05:31.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.574+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v105'68 uv68 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a3ca90 2023-11-15T03:05:31.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.574+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x556f48a405b0 0x556f48a429a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.574+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43040 con 0x556f48a405b0 2023-11-15T03:05:31.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.574+0000 7ff2b8f11700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x556f48a405b0 0x556f48a429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.578+0000 7ff2b8f11700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x556f48a405b0 0x556f48a429a0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:31.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.578+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v105'47 uv46 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.578+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a405b0 2023-11-15T03:05:31.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.582+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v105'48 uv48 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.582+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a405b0 2023-11-15T03:05:31.594 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:05:31.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.586+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v105'49 uv49 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.586+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a405b0 2023-11-15T03:05:31.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.586+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v105'50 uv50 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.590+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a405b0 2023-11-15T03:05:31.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.590+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v105'63 uv62 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.590+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a405b0 2023-11-15T03:05:31.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.594+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v105'64 uv64 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.594+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a405b0 2023-11-15T03:05:31.605 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:05:31.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.598+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v105'65 uv65 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.598+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a44430 con 0x556f48a405b0 2023-11-15T03:05:31.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.598+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v105'66 uv66 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.598+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a44430 con 0x556f48a405b0 2023-11-15T03:05:31.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.602+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v105'67 uv67 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.602+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a405b0 2023-11-15T03:05:31.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.606+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v105'68 uv68 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.606+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a2f180 2023-11-15T03:05:31.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.610+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v105'116 uv110 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.610+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a2f180 2023-11-15T03:05:31.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.610+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v105'117 uv117 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.610+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a2f180 2023-11-15T03:05:31.624 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:05:31.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.614+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v105'118 uv118 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.614+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a44430 con 0x556f48a2f180 2023-11-15T03:05:31.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.618+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v105'119 uv119 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.618+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a44430 con 0x556f48a2f180 2023-11-15T03:05:31.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.622+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v105'120 uv120 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.622+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a2f180 2023-11-15T03:05:31.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.622+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v105'121 uv121 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.622+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a2f180 2023-11-15T03:05:31.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.626+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v105'122 uv115 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.626+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a2f180 2023-11-15T03:05:31.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.630+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v105'123 uv123 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.630+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a2f180 2023-11-15T03:05:31.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.634+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v105'124 uv124 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.642 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:05:31.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.634+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a2f180 2023-11-15T03:05:31.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.634+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v105'125 uv125 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.634+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a2f180 2023-11-15T03:05:31.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.638+0000 7ff2c1712700 1 -- 172.21.15.121:0/71612221 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v105'126 uv126 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2b4012040 con 0x556f48a2f180 2023-11-15T03:05:31.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.638+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a34b30 2023-11-15T03:05:31.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.642+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v105'51 uv50 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a34b30 2023-11-15T03:05:31.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.642+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a34b30 2023-11-15T03:05:31.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.642+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v105'52 uv52 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a34b30 2023-11-15T03:05:31.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.646+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a34b30 2023-11-15T03:05:31.655 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:05:31.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.646+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v105'53 uv53 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a34b30 2023-11-15T03:05:31.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.646+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a34b30 2023-11-15T03:05:31.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.650+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v105'54 uv54 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a34b30 2023-11-15T03:05:31.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.650+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43820 con 0x556f48a34b30 2023-11-15T03:05:31.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.650+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v105'55 uv55 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a34b30 2023-11-15T03:05:31.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.650+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43820 con 0x556f48a405b0 2023-11-15T03:05:31.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.654+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v105'63 uv62 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.654+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a405b0 2023-11-15T03:05:31.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.658+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v105'64 uv64 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.658+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43420 con 0x556f48a405b0 2023-11-15T03:05:31.669 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:05:31.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.662+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v105'65 uv65 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.662+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a44830 con 0x556f48a405b0 2023-11-15T03:05:31.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.662+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v105'66 uv66 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.662+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a44830 con 0x556f48a405b0 2023-11-15T03:05:31.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.666+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v105'67 uv67 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.666+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43820 con 0x556f48a405b0 2023-11-15T03:05:31.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.670+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v105'68 uv68 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.670+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43820 con 0x556f48a405b0 2023-11-15T03:05:31.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.670+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v105'52 uv51 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.674+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a43820 con 0x556f48a405b0 2023-11-15T03:05:31.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.674+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v105'53 uv53 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.674+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43820 con 0x556f48a405b0 2023-11-15T03:05:31.687 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:05:31.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.678+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v105'54 uv54 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.678+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43820 con 0x556f48a405b0 2023-11-15T03:05:31.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.682+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v105'55 uv55 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.682+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43820 con 0x556f48a405b0 2023-11-15T03:05:31.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.686+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v105'56 uv56 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.686+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43820 con 0x556f48a405b0 2023-11-15T03:05:31.696 INFO:teuthology.orchestra.run.smithi121.stdout:Write #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:05:31.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.686+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v105'53 uv53 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.686+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a43820 con 0x556f48a405b0 2023-11-15T03:05:31.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.690+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [omap-set-header] v105'54 uv54 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.690+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a46590 con 0x556f48a405b0 2023-11-15T03:05:31.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.694+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v105'55 uv55 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.694+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x556f48a44220 0x556f48a46610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.694+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a46b00 con 0x556f48a44220 2023-11-15T03:05:31.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.694+0000 7ff2c1f13700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x556f48a44220 0x556f48a46610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.694+0000 7ff2c1f13700 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x556f48a44220 0x556f48a46610 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:31.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.698+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(99 mds0_sessionmap [create] v105'43 uv42 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a44220 2023-11-15T03:05:31.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.698+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a46b00 con 0x556f48a44220 2023-11-15T03:05:31.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.698+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(100 mds0_sessionmap [delete] v105'44 uv44 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a44220 2023-11-15T03:05:31.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.698+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a46b00 con 0x556f48a44220 2023-11-15T03:05:31.712 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:05:31.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.702+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(101 mds0_sessionmap [create] v105'45 uv45 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a44220 2023-11-15T03:05:31.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.702+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a46b00 con 0x556f48a44220 2023-11-15T03:05:31.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.706+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v105'46 uv46 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a44220 2023-11-15T03:05:31.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.706+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a46b00 con 0x556f48a405b0 2023-11-15T03:05:31.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.710+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(103 609.00000000 [create] v105'63 uv62 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.710+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a46b00 con 0x556f48a405b0 2023-11-15T03:05:31.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.710+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(104 609.00000000 [delete] v105'64 uv64 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.710+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a46b00 con 0x556f48a405b0 2023-11-15T03:05:31.723 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:05:31.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.714+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v105'65 uv65 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.714+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a47b10 con 0x556f48a405b0 2023-11-15T03:05:31.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.718+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v105'66 uv66 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.718+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a47b10 con 0x556f48a405b0 2023-11-15T03:05:31.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.718+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v105'67 uv67 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.722+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a46b00 con 0x556f48a405b0 2023-11-15T03:05:31.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.722+0000 7ff2b8f11700 1 -- 172.21.15.121:0/71612221 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [omap-set-header] v105'68 uv68 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2ac000bc0 con 0x556f48a405b0 2023-11-15T03:05:31.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.722+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a46b00 con 0x556f48a33330 2023-11-15T03:05:31.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.726+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(109 400.00000000 [create] v105'52 uv51 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.726+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a46b00 con 0x556f48a33330 2023-11-15T03:05:31.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.730+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(110 400.00000000 [delete] v105'53 uv53 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.730+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a46b00 con 0x556f48a33330 2023-11-15T03:05:31.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.730+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(111 400.00000000 [create] v105'54 uv54 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.740 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:05:31.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.730+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a46b00 con 0x556f48a33330 2023-11-15T03:05:31.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.734+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(112 400.00000000 [write 0~22] v105'55 uv55 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.734+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a46b00 con 0x556f48a33330 2023-11-15T03:05:31.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.738+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(113 400.00000000 [omap-set-header] v105'56 uv56 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.738+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a46b00 con 0x556f48a33330 2023-11-15T03:05:31.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.738+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(114 100.00000000.inode [create] v105'126 uv118 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.738+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a46b00 con 0x556f48a33330 2023-11-15T03:05:31.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.742+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(115 100.00000000.inode [delete] v105'127 uv127 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.742+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a46b00 con 0x556f48a33330 2023-11-15T03:05:31.754 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:05:31.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.746+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(116 100.00000000.inode [create] v105'128 uv128 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.746+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a46b00 con 0x556f48a33330 2023-11-15T03:05:31.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.746+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v105'129 uv129 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.746+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a438a0 con 0x556f48a33330 2023-11-15T03:05:31.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.750+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v105'130 uv130 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.750+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a438a0 con 0x556f48a33330 2023-11-15T03:05:31.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.754+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(119 1.00000000 [create] v105'131 uv125 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.754+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x556f48a43820 con 0x556f48a33330 2023-11-15T03:05:31.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.758+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(120 1.00000000 [delete] v105'132 uv132 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.758+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a46b00 con 0x556f48a33330 2023-11-15T03:05:31.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.758+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(121 1.00000000 [create] v105'133 uv133 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.768 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:05:31.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.758+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a47b30 con 0x556f48a33330 2023-11-15T03:05:31.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.762+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v105'134 uv134 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.762+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a47b30 con 0x556f48a33330 2023-11-15T03:05:31.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.766+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v105'135 uv135 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.766+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a47b30 con 0x556f48a33330 2023-11-15T03:05:31.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.766+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v105'136 uv136 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.766+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e105) v8 -- 0x556f48a46b00 con 0x556f48a33330 2023-11-15T03:05:31.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v105'137 uv137 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff2bc010de0 con 0x556f48a33330 2023-11-15T03:05:31.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x556f48a34b30 msgr2=0x556f48a36f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x556f48a34b30 0x556f48a36f20 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x556f48a12af0 msgr2=0x556f48a0eab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x556f48a12af0 0x556f48a0eab0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x556f48a3ca90 msgr2=0x556f48a3ee80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x556f48a3ca90 0x556f48a3ee80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x556f48a2f180 msgr2=0x556f48a32260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x556f48a2f180 0x556f48a32260 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x556f48a44220 msgr2=0x556f48a46610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 reap_dead start 2023-11-15T03:05:31.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x556f48a44220 0x556f48a46610 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556f48a33330 msgr2=0x556f48a3a260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556f48a33330 0x556f48a3a260 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x556f48a405b0 msgr2=0x556f48a429a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x556f48a405b0 0x556f48a429a0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff2a4058ff0 msgr2=0x7ff2a405b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff2a4058ff0 0x7ff2a405b4b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7ff2b4005fd0 tx=0x7ff2b400c040).stop 2023-11-15T03:05:31.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f48a112a0 msgr2=0x556f48a0a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 --2- 172.21.15.121:0/71612221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f48a112a0 0x556f48a0a870 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0x7ff2bc006f90 tx=0x7ff2bc0068f0).stop 2023-11-15T03:05:31.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c1f13700 1 -- 172.21.15.121:0/71612221 reap_dead start 2023-11-15T03:05:31.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 shutdown_connections 2023-11-15T03:05:31.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 >> 172.21.15.121:0/71612221 conn(0x556f489397d0 msgr2=0x556f48a14130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:31.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 shutdown_connections 2023-11-15T03:05:31.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.770+0000 7ff2c3edad00 1 -- 172.21.15.121:0/71612221 wait complete. 2023-11-15T03:05:31.787 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_sessionmap 2023-11-15T03:05:31.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.830+0000 7f3801f2fd00 1 Processor -- start 2023-11-15T03:05:31.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.830+0000 7f3801f2fd00 1 -- start start 2023-11-15T03:05:31.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.830+0000 7f3801f2fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5618ddf8d740 0x5618de061ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.830+0000 7f3801f2fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618ddfd4850 0x5618ddfd5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.830+0000 7f3801f2fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5618de062e40 0x5618ddfd5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.830+0000 7f3801f2fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ddf05220 con 0x5618de062e40 2023-11-15T03:05:31.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.830+0000 7f3801f2fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ddf900e0 con 0x5618ddfd4850 2023-11-15T03:05:31.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.830+0000 7f3801f2fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618ddfb9410 con 0x5618ddf8d740 2023-11-15T03:05:31.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f37f6d16700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5618de062e40 0x5618ddfd5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f37effff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618ddfd4850 0x5618ddfd5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f37f6d16700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5618de062e40 0x5618ddfd5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59574/0 (socket says 172.21.15.121:59574) 2023-11-15T03:05:31.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f37f6d16700 1 -- 172.21.15.121:0/2680639380 learned_addr learned my addr 172.21.15.121:0/2680639380 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:31.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f37f6515700 1 --2- 172.21.15.121:0/2680639380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5618ddf8d740 0x5618de061ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f37f6d16700 1 -- 172.21.15.121:0/2680639380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5618ddf8d740 msgr2=0x5618de061ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:31.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f37f6d16700 1 --2- 172.21.15.121:0/2680639380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5618ddf8d740 0x5618de061ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f37f6d16700 1 -- 172.21.15.121:0/2680639380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618ddfd4850 msgr2=0x5618ddfd5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:31.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f37f6d16700 1 --2- 172.21.15.121:0/2680639380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618ddfd4850 0x5618ddfd5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f37f6d16700 1 -- 172.21.15.121:0/2680639380 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e0005db0 con 0x5618de062e40 2023-11-15T03:05:31.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f37effff700 1 --2- 172.21.15.121:0/2680639380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618ddfd4850 0x5618ddfd5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:05:31.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f37f6d16700 1 --2- 172.21.15.121:0/2680639380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5618de062e40 0x5618ddfd5830 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f37f0002920 tx=0x7f37f0005d40).ready entity=mon.0 client_cookie=38a1dc25327190bb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:31.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f37f5513700 1 -- 172.21.15.121:0/2680639380 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f000d670 con 0x5618de062e40 2023-11-15T03:05:31.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f37f5513700 1 -- 172.21.15.121:0/2680639380 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f000d7d0 con 0x5618de062e40 2023-11-15T03:05:31.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f37f5513700 1 -- 172.21.15.121:0/2680639380 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f000db70 con 0x5618de062e40 2023-11-15T03:05:31.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f3801f2fd00 1 -- 172.21.15.121:0/2680639380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5618de062e40 msgr2=0x5618ddfd5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f3801f2fd00 1 --2- 172.21.15.121:0/2680639380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5618de062e40 0x5618ddfd5830 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f37f0002920 tx=0x7f37f0005d40).stop 2023-11-15T03:05:31.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f3801f2fd00 1 -- 172.21.15.121:0/2680639380 shutdown_connections 2023-11-15T03:05:31.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f3801f2fd00 1 --2- 172.21.15.121:0/2680639380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5618de062e40 0x5618ddfd5830 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f3801f2fd00 1 --2- 172.21.15.121:0/2680639380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618ddfd4850 0x5618ddfd5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f3801f2fd00 1 --2- 172.21.15.121:0/2680639380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5618ddf8d740 0x5618de061ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f3801f2fd00 1 -- 172.21.15.121:0/2680639380 >> 172.21.15.121:0/2680639380 conn(0x5618ddfd4f60 msgr2=0x5618ddfcdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:31.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f3801f2fd00 1 -- 172.21.15.121:0/2680639380 shutdown_connections 2023-11-15T03:05:31.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f3801f2fd00 1 -- 172.21.15.121:0/2680639380 wait complete. 2023-11-15T03:05:31.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f3801f2fd00 1 Processor -- start 2023-11-15T03:05:31.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.834+0000 7f3801f2fd00 1 -- start start 2023-11-15T03:05:31.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f3801f2fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5618ddf8d740 0x5618de05d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f3801f2fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618ddfd4850 0x5618ddfbbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f3801f2fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5618de062e40 0x5618ddfbc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f3801f2fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618de079550 con 0x5618ddf8d740 2023-11-15T03:05:31.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f3801f2fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ddfbc930 con 0x5618ddfd4850 2023-11-15T03:05:31.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f3801f2fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618ddfb78f0 con 0x5618de062e40 2023-11-15T03:05:31.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f37f6515700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5618ddf8d740 0x5618de05d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f37f6515700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5618ddf8d740 0x5618de05d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59582/0 (socket says 172.21.15.121:59582) 2023-11-15T03:05:31.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f37f6515700 1 -- 172.21.15.121:0/3989038695 learned_addr learned my addr 172.21.15.121:0/3989038695 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:31.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f37f6d16700 1 --2- 172.21.15.121:0/3989038695 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5618de062e40 0x5618ddfbc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f37effff700 1 --2- 172.21.15.121:0/3989038695 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618ddfd4850 0x5618ddfbbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f37f6515700 1 -- 172.21.15.121:0/3989038695 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5618de062e40 msgr2=0x5618ddfbc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f37f6515700 1 --2- 172.21.15.121:0/3989038695 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5618de062e40 0x5618ddfbc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f37f6515700 1 -- 172.21.15.121:0/3989038695 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618ddfd4850 msgr2=0x5618ddfbbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f37f6515700 1 --2- 172.21.15.121:0/3989038695 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618ddfd4850 0x5618ddfbbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f37f6515700 1 -- 172.21.15.121:0/3989038695 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618ddf2ea20 con 0x5618ddf8d740 2023-11-15T03:05:31.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f37f6515700 1 --2- 172.21.15.121:0/3989038695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5618ddf8d740 0x5618de05d0a0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7f37e8002e90 tx=0x7f37e80017a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:31.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f37eeffd700 1 -- 172.21.15.121:0/3989038695 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8002b40 con 0x5618ddf8d740 2023-11-15T03:05:31.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f37eeffd700 1 -- 172.21.15.121:0/3989038695 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e8008070 con 0x5618ddf8d740 2023-11-15T03:05:31.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f3801f2fd00 1 -- 172.21.15.121:0/3989038695 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f0002600 con 0x5618ddf8d740 2023-11-15T03:05:31.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f37eeffd700 1 -- 172.21.15.121:0/3989038695 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e800cd50 con 0x5618ddf8d740 2023-11-15T03:05:31.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.838+0000 7f3801f2fd00 1 -- 172.21.15.121:0/3989038695 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618ddf2e770 con 0x5618ddf8d740 2023-11-15T03:05:31.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.842+0000 7f37eeffd700 1 -- 172.21.15.121:0/3989038695 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f37e8011660 con 0x5618ddf8d740 2023-11-15T03:05:31.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.842+0000 7f37eeffd700 1 --2- 172.21.15.121:0/3989038695 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f37d8058ff0 0x7f37d805b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.842+0000 7f37eeffd700 1 -- 172.21.15.121:0/3989038695 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7212+0+0 (secure 0 0 0) 0x7f37e80735a0 con 0x5618ddf8d740 2023-11-15T03:05:31.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.842+0000 7f37effff700 1 --2- 172.21.15.121:0/3989038695 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f37d8058ff0 0x7f37d805b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.842+0000 7f3801f2fd00 1 --2- 172.21.15.121:0/3989038695 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5618ddfd3530 0x5618de075c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:31.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.842+0000 7f3801f2fd00 1 -- 172.21.15.121:0/3989038695 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e105) v8 -- 0x5618ddfd5970 con 0x5618ddfd3530 2023-11-15T03:05:31.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.842+0000 7f37effff700 1 --2- 172.21.15.121:0/3989038695 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f37d8058ff0 0x7f37d805b4b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x5618ddfd4c60 tx=0x7f37e0016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:31.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.842+0000 7f37f6d16700 1 --2- 172.21.15.121:0/3989038695 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5618ddfd3530 0x5618de075c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:31.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.842+0000 7f37f6d16700 1 --2- 172.21.15.121:0/3989038695 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5618ddfd3530 0x5618de075c00 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:31.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.846+0000 7f37f6d16700 1 -- 172.21.15.121:0/3989038695 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(1 mds0_sessionmap [delete] v105'47 uv47 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f37f0017020 con 0x5618ddfd3530 2023-11-15T03:05:31.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.846+0000 7f3801f2fd00 1 -- 172.21.15.121:0/3989038695 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5618ddfd3530 msgr2=0x5618de075c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.846+0000 7f3801f2fd00 1 --2- 172.21.15.121:0/3989038695 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5618ddfd3530 0x5618de075c00 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:31.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.846+0000 7f3801f2fd00 1 -- 172.21.15.121:0/3989038695 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f37d8058ff0 msgr2=0x7f37d805b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.846+0000 7f3801f2fd00 1 --2- 172.21.15.121:0/3989038695 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f37d8058ff0 0x7f37d805b4b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x5618ddfd4c60 tx=0x7f37e0016000).stop 2023-11-15T03:05:31.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.846+0000 7f3801f2fd00 1 -- 172.21.15.121:0/3989038695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5618ddf8d740 msgr2=0x5618de05d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:31.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.846+0000 7f3801f2fd00 1 --2- 172.21.15.121:0/3989038695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5618ddf8d740 0x5618de05d0a0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7f37e8002e90 tx=0x7f37e80017a0).stop 2023-11-15T03:05:31.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.846+0000 7f37f6d16700 1 -- 172.21.15.121:0/3989038695 reap_dead start 2023-11-15T03:05:31.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.846+0000 7f3801f2fd00 1 -- 172.21.15.121:0/3989038695 shutdown_connections 2023-11-15T03:05:31.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.846+0000 7f3801f2fd00 1 -- 172.21.15.121:0/3989038695 >> 172.21.15.121:0/3989038695 conn(0x5618ddfd4f60 msgr2=0x5618de061ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:31.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.846+0000 7f3801f2fd00 1 -- 172.21.15.121:0/3989038695 shutdown_connections 2023-11-15T03:05:31.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:31.846+0000 7f3801f2fd00 1 -- 172.21.15.121:0/3989038695 wait complete. 2023-11-15T03:05:31.864 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:05:32.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.026+0000 7f65e685b700 1 Processor -- start 2023-11-15T03:05:32.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.026+0000 7f65e685b700 1 -- start start 2023-11-15T03:05:32.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65e685b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f65e00a8b80 0x7f65e00a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:32.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65e685b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65e0068170 0x7f65e00bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:32.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65e685b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e0152150 con 0x7f65e0065850 2023-11-15T03:05:32.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65e685b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e0148f70 con 0x7f65e00a8b80 2023-11-15T03:05:32.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65e685b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e01523a0 con 0x7f65e0068170 2023-11-15T03:05:32.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65d77fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f65e0065850 0x7f65e0067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56474/0 (socket says 172.21.15.121:56474) 2023-11-15T03:05:32.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65d77fe700 1 -- 172.21.15.121:0/4293034036 learned_addr learned my addr 172.21.15.121:0/4293034036 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:32.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65dffff700 1 --2- 172.21.15.121:0/4293034036 >> v2:172.21.15.190:3300/0 conn(0x7f65e00a8b80 0x7f65e00a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:32.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65e4df8700 1 --2- 172.21.15.121:0/4293034036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65e0068170 0x7f65e00bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:32.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65df7fe700 1 -- 172.21.15.121:0/4293034036 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439253693 0 0) 0x7f65e0152150 con 0x7f65e0065850 2023-11-15T03:05:32.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65df7fe700 1 -- 172.21.15.121:0/4293034036 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65cc0033f0 con 0x7f65e0065850 2023-11-15T03:05:32.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65df7fe700 1 -- 172.21.15.121:0/4293034036 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3916742422 0 0) 0x7f65cc0033f0 con 0x7f65e0065850 2023-11-15T03:05:32.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65df7fe700 1 -- 172.21.15.121:0/4293034036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65e0068170 msgr2=0x7f65e00bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:32.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65df7fe700 1 --2- 172.21.15.121:0/4293034036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65e0068170 0x7f65e00bbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65df7fe700 1 -- 172.21.15.121:0/4293034036 >> v2:172.21.15.190:3300/0 conn(0x7f65e00a8b80 msgr2=0x7f65e00a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:32.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65df7fe700 1 --2- 172.21.15.121:0/4293034036 >> v2:172.21.15.190:3300/0 conn(0x7f65e00a8b80 0x7f65e00a8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65df7fe700 1 -- 172.21.15.121:0/4293034036 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e0152680 con 0x7f65e0065850 2023-11-15T03:05:32.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65df7fe700 1 -- 172.21.15.121:0/4293034036 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f65c8001470 con 0x7f65e0065850 2023-11-15T03:05:32.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65df7fe700 1 -- 172.21.15.121:0/4293034036 >> v1:172.21.15.121:6789/0 conn(0x7f65e0065850 legacy=0x7f65e0067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:32.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65df7fe700 1 --2- 172.21.15.121:0/4293034036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65cc0038e0 0x7f65cc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:32.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65df7fe700 1 --2- 172.21.15.121:0/4293034036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65cc004690 0x7f65cc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:32.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65df7fe700 1 --2- 172.21.15.121:0/4293034036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65e0068170 0x7f65cc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:32.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65df7fe700 1 -- 172.21.15.121:0/4293034036 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e0148f70 con 0x7f65cc0038e0 2023-11-15T03:05:32.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65df7fe700 1 -- 172.21.15.121:0/4293034036 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e01523a0 con 0x7f65cc004690 2023-11-15T03:05:32.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65df7fe700 1 -- 172.21.15.121:0/4293034036 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc00b8e0 con 0x7f65e0068170 2023-11-15T03:05:32.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.030+0000 7f65dffff700 1 --2- 172.21.15.121:0/4293034036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65cc0038e0 0x7f65cc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:32.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65d77fe700 1 --2- 172.21.15.121:0/4293034036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65cc004690 0x7f65cc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:32.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65e4df8700 1 --2- 172.21.15.121:0/4293034036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65e0068170 0x7f65cc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:32.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65dffff700 1 -- 172.21.15.121:0/4293034036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65e0068170 msgr2=0x7f65cc005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:32.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65dffff700 1 --2- 172.21.15.121:0/4293034036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65e0068170 0x7f65cc005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65dffff700 1 -- 172.21.15.121:0/4293034036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65cc004690 msgr2=0x7f65cc004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:32.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65dffff700 1 --2- 172.21.15.121:0/4293034036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65cc004690 0x7f65cc004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65dffff700 1 -- 172.21.15.121:0/4293034036 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc0033f0 con 0x7f65cc0038e0 2023-11-15T03:05:32.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65e4df8700 1 --2- 172.21.15.121:0/4293034036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65e0068170 0x7f65cc005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:05:32.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65d77fe700 1 --2- 172.21.15.121:0/4293034036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65cc004690 0x7f65cc004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:32.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65dffff700 1 --2- 172.21.15.121:0/4293034036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65cc0038e0 0x7f65cc003d00 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f65d0009a80 tx=0x7f65d0009060).ready entity=mon.0 client_cookie=4944ed0f733cc6bc server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:32.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65df7fe700 1 -- 172.21.15.121:0/4293034036 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d0009430 con 0x7f65cc0038e0 2023-11-15T03:05:32.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65df7fe700 1 -- 172.21.15.121:0/4293034036 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d000e640 con 0x7f65cc0038e0 2023-11-15T03:05:32.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65df7fe700 1 -- 172.21.15.121:0/4293034036 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d000e9c0 con 0x7f65cc0038e0 2023-11-15T03:05:32.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65e685b700 1 -- 172.21.15.121:0/4293034036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65cc0038e0 msgr2=0x7f65cc003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:32.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65e685b700 1 --2- 172.21.15.121:0/4293034036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65cc0038e0 0x7f65cc003d00 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f65d0009a80 tx=0x7f65d0009060).stop 2023-11-15T03:05:32.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65e4df8700 1 -- 172.21.15.121:0/4293034036 reap_dead start 2023-11-15T03:05:32.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65e685b700 1 -- 172.21.15.121:0/4293034036 shutdown_connections 2023-11-15T03:05:32.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65e685b700 1 -- 172.21.15.121:0/4293034036 >> 172.21.15.121:0/4293034036 conn(0x7f65e00591c0 msgr2=0x7f65e00c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:32.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65e685b700 1 -- 172.21.15.121:0/4293034036 shutdown_connections 2023-11-15T03:05:32.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.034+0000 7f65e685b700 1 -- 172.21.15.121:0/4293034036 wait complete. 2023-11-15T03:05:32.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.038+0000 7f65e685b700 1 Processor -- start 2023-11-15T03:05:32.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.038+0000 7f65e685b700 1 -- start start 2023-11-15T03:05:32.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.038+0000 7f65e685b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65cc0038e0 0x7f65e01481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:32.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.038+0000 7f65e685b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65e0065c60 0x7f65e00660e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:32.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.038+0000 7f65e685b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65e014d840 0x7f65e014dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:32.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.038+0000 7f65e685b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e0149ea0 con 0x7f65e0065c60 2023-11-15T03:05:32.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.038+0000 7f65e685b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e00665d0 con 0x7f65cc0038e0 2023-11-15T03:05:32.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.038+0000 7f65e685b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e014e1b0 con 0x7f65e014d840 2023-11-15T03:05:32.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.038+0000 7f65d77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65e0065c60 0x7f65e00660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:32.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.038+0000 7f65d77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65e0065c60 0x7f65e00660e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59604/0 (socket says 172.21.15.121:59604) 2023-11-15T03:05:32.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.038+0000 7f65d77fe700 1 -- 172.21.15.121:0/1357089510 learned_addr learned my addr 172.21.15.121:0/1357089510 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:32.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.038+0000 7f65dffff700 1 --2- 172.21.15.121:0/1357089510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65cc0038e0 0x7f65e01481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:32.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.042+0000 7f65e4df8700 1 --2- 172.21.15.121:0/1357089510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65e014d840 0x7f65e014dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:32.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.042+0000 7f65d77fe700 1 -- 172.21.15.121:0/1357089510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65e014d840 msgr2=0x7f65e014dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:32.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.042+0000 7f65d77fe700 1 --2- 172.21.15.121:0/1357089510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65e014d840 0x7f65e014dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.042+0000 7f65d77fe700 1 -- 172.21.15.121:0/1357089510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65cc0038e0 msgr2=0x7f65e01481b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:32.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.042+0000 7f65d77fe700 1 --2- 172.21.15.121:0/1357089510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65cc0038e0 0x7f65e01481b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.042+0000 7f65d77fe700 1 -- 172.21.15.121:0/1357089510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e0149290 con 0x7f65e0065c60 2023-11-15T03:05:32.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.042+0000 7f65e4df8700 1 --2- 172.21.15.121:0/1357089510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65e014d840 0x7f65e014dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:05:32.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.042+0000 7f65d77fe700 1 --2- 172.21.15.121:0/1357089510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65e0065c60 0x7f65e00660e0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f65c800e860 tx=0x7f65c8000cc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:32.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.042+0000 7f65dcff9700 1 -- 172.21.15.121:0/1357089510 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c800c540 con 0x7f65e0065c60 2023-11-15T03:05:32.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.042+0000 7f65dcff9700 1 -- 172.21.15.121:0/1357089510 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c800c6a0 con 0x7f65e0065c60 2023-11-15T03:05:32.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.042+0000 7f65e685b700 1 -- 172.21.15.121:0/1357089510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e0149570 con 0x7f65e0065c60 2023-11-15T03:05:32.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.042+0000 7f65dcff9700 1 -- 172.21.15.121:0/1357089510 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c800ca20 con 0x7f65e0065c60 2023-11-15T03:05:32.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.042+0000 7f65e685b700 1 -- 172.21.15.121:0/1357089510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e0149b30 con 0x7f65e0065c60 2023-11-15T03:05:32.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.042+0000 7f65e685b700 1 -- 172.21.15.121:0/1357089510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e0047060 con 0x7f65e0065c60 2023-11-15T03:05:32.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.042+0000 7f65dcff9700 1 -- 172.21.15.121:0/1357089510 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f65c800cd90 con 0x7f65e0065c60 2023-11-15T03:05:32.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.046+0000 7f65dcff9700 1 --2- 172.21.15.121:0/1357089510 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f65c005a540 0x7f65c005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:32.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.046+0000 7f65dcff9700 1 -- 172.21.15.121:0/1357089510 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7212+0+0 (secure 0 0 0) 0x7f65c8074ae0 con 0x7f65e0065c60 2023-11-15T03:05:32.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.046+0000 7f65dffff700 1 --2- 172.21.15.121:0/1357089510 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f65c005a540 0x7f65c005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:32.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.046+0000 7f65dffff700 1 --2- 172.21.15.121:0/1357089510 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f65c005a540 0x7f65c005ca00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f65d0006d50 tx=0x7f65d001a000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:32.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.050+0000 7f65dcff9700 1 -- 172.21.15.121:0/1357089510 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f65c8047e80 con 0x7f65e0065c60 2023-11-15T03:05:32.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.254+0000 7f65e685b700 1 -- 172.21.15.121:0/1357089510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f65e014c470 con 0x7f65e0065c60 2023-11-15T03:05:32.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.314+0000 7f65dcff9700 1 -- 172.21.15.121:0/1357089510 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v153) v1 ==== 162+0+0 (secure 0 0 0) 0x7f65c804b4a0 con 0x7f65e0065c60 2023-11-15T03:05:32.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.322+0000 7f65e685b700 1 -- 172.21.15.121:0/1357089510 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f65c005a540 msgr2=0x7f65c005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:32.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.322+0000 7f65e685b700 1 --2- 172.21.15.121:0/1357089510 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f65c005a540 0x7f65c005ca00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f65d0006d50 tx=0x7f65d001a000).stop 2023-11-15T03:05:32.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.322+0000 7f65e685b700 1 -- 172.21.15.121:0/1357089510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65e0065c60 msgr2=0x7f65e00660e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:32.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.322+0000 7f65e685b700 1 --2- 172.21.15.121:0/1357089510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65e0065c60 0x7f65e00660e0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f65c800e860 tx=0x7f65c8000cc0).stop 2023-11-15T03:05:32.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.322+0000 7f65e685b700 1 -- 172.21.15.121:0/1357089510 shutdown_connections 2023-11-15T03:05:32.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.322+0000 7f65e685b700 1 --2- 172.21.15.121:0/1357089510 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f65c005a540 0x7f65c005ca00 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.322+0000 7f65e685b700 1 --2- 172.21.15.121:0/1357089510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65e014d840 0x7f65e014dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.322+0000 7f65e685b700 1 --2- 172.21.15.121:0/1357089510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65e0065c60 0x7f65e00660e0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.322+0000 7f65e685b700 1 --2- 172.21.15.121:0/1357089510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65cc0038e0 0x7f65e01481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.322+0000 7f65e685b700 1 -- 172.21.15.121:0/1357089510 >> 172.21.15.121:0/1357089510 conn(0x7f65e00591c0 msgr2=0x7f65e00c5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:32.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.322+0000 7f65e685b700 1 -- 172.21.15.121:0/1357089510 shutdown_connections 2023-11-15T03:05:32.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.322+0000 7f65e685b700 1 -- 172.21.15.121:0/1357089510 wait complete. 2023-11-15T03:05:32.333 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:05:32.335 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:05:32.326+0000 7f80797cc700 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2023-11-15T03:05:32.335 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:05:32.326+0000 7f80797cc700 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2023-11-15T03:05:32.353 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:05:32.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.514+0000 7f5d80a1b700 1 Processor -- start 2023-11-15T03:05:32.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.514+0000 7f5d80a1b700 1 -- start start 2023-11-15T03:05:32.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.514+0000 7f5d80a1b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5d7c154ca0 0x7f5d7c155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:32.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.514+0000 7f5d80a1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d7c1562f0 0x7f5d7c0a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:32.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.514+0000 7f5d80a1b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5d7c0a8f40 0x7f5d7c0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:32.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.514+0000 7f5d80a1b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d7c0a8280 con 0x7f5d7c0a8f40 2023-11-15T03:05:32.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.514+0000 7f5d80a1b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d7c0a6d40 con 0x7f5d7c154ca0 2023-11-15T03:05:32.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.514+0000 7f5d80a1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0c4c30 con 0x7f5d7c1562f0 2023-11-15T03:05:32.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.514+0000 7f5d7ad9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5d7c0a8f40 0x7f5d7c0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:32.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.514+0000 7f5d7ad9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5d7c0a8f40 0x7f5d7c0a57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59620/0 (socket says 172.21.15.121:59620) 2023-11-15T03:05:32.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.514+0000 7f5d7ad9d700 1 -- 172.21.15.121:0/3141819313 learned_addr learned my addr 172.21.15.121:0/3141819313 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:32.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.514+0000 7f5d71d9b700 1 --2- 172.21.15.121:0/3141819313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d7c1562f0 0x7f5d7c0a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:32.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.514+0000 7f5d7a59c700 1 --2- 172.21.15.121:0/3141819313 >> v2:172.21.15.190:3300/0 conn(0x7f5d7c154ca0 0x7f5d7c155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:32.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.518+0000 7f5d71d9b700 1 -- 172.21.15.121:0/3141819313 >> v2:172.21.15.190:3300/0 conn(0x7f5d7c154ca0 msgr2=0x7f5d7c155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:32.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.518+0000 7f5d71d9b700 1 --2- 172.21.15.121:0/3141819313 >> v2:172.21.15.190:3300/0 conn(0x7f5d7c154ca0 0x7f5d7c155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.518+0000 7f5d71d9b700 1 -- 172.21.15.121:0/3141819313 >> v2:172.21.15.121:3300/0 conn(0x7f5d7c0a8f40 msgr2=0x7f5d7c0a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:32.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.518+0000 7f5d71d9b700 1 --2- 172.21.15.121:0/3141819313 >> v2:172.21.15.121:3300/0 conn(0x7f5d7c0a8f40 0x7f5d7c0a57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.518+0000 7f5d71d9b700 1 -- 172.21.15.121:0/3141819313 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c0a6f50 con 0x7f5d7c1562f0 2023-11-15T03:05:32.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.518+0000 7f5d71d9b700 1 --2- 172.21.15.121:0/3141819313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d7c1562f0 0x7f5d7c0a8a50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f5d7c064f10 tx=0x7f5d64009060).ready entity=mon.2 client_cookie=d1b89cc8179b116c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:32.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.518+0000 7f5d79d9b700 1 -- 172.21.15.121:0/3141819313 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6400d4a0 con 0x7f5d7c1562f0 2023-11-15T03:05:32.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.518+0000 7f5d79d9b700 1 -- 172.21.15.121:0/3141819313 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6400d600 con 0x7f5d7c1562f0 2023-11-15T03:05:32.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.518+0000 7f5d79d9b700 1 -- 172.21.15.121:0/3141819313 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6400d980 con 0x7f5d7c1562f0 2023-11-15T03:05:32.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.518+0000 7f5d80a1b700 1 -- 172.21.15.121:0/3141819313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d7c1562f0 msgr2=0x7f5d7c0a8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:32.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.518+0000 7f5d80a1b700 1 --2- 172.21.15.121:0/3141819313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d7c1562f0 0x7f5d7c0a8a50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f5d7c064f10 tx=0x7f5d64009060).stop 2023-11-15T03:05:32.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.518+0000 7f5d80a1b700 1 -- 172.21.15.121:0/3141819313 shutdown_connections 2023-11-15T03:05:32.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.518+0000 7f5d80a1b700 1 --2- 172.21.15.121:0/3141819313 >> v2:172.21.15.121:3300/0 conn(0x7f5d7c0a8f40 0x7f5d7c0a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.518+0000 7f5d80a1b700 1 --2- 172.21.15.121:0/3141819313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d7c1562f0 0x7f5d7c0a8a50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.518+0000 7f5d80a1b700 1 --2- 172.21.15.121:0/3141819313 >> v2:172.21.15.190:3300/0 conn(0x7f5d7c154ca0 0x7f5d7c155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.518+0000 7f5d80a1b700 1 -- 172.21.15.121:0/3141819313 >> 172.21.15.121:0/3141819313 conn(0x7f5d7c062190 msgr2=0x7f5d7c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:32.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.518+0000 7f5d80a1b700 1 -- 172.21.15.121:0/3141819313 shutdown_connections 2023-11-15T03:05:32.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.518+0000 7f5d80a1b700 1 -- 172.21.15.121:0/3141819313 wait complete. 2023-11-15T03:05:32.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.522+0000 7f5d80a1b700 1 Processor -- start 2023-11-15T03:05:32.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.522+0000 7f5d80a1b700 1 -- start start 2023-11-15T03:05:32.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.522+0000 7f5d80a1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d7c0a8f40 0x7f5d7c14e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:32.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.522+0000 7f5d80a1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d7c154ca0 0x7f5d7c0bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:32.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.522+0000 7f5d80a1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d7c1562f0 0x7f5d7c0c0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:32.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.522+0000 7f5d80a1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0c0570 con 0x7f5d7c1562f0 2023-11-15T03:05:32.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.522+0000 7f5d80a1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c167b40 con 0x7f5d7c154ca0 2023-11-15T03:05:32.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.522+0000 7f5d80a1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c167d90 con 0x7f5d7c0a8f40 2023-11-15T03:05:32.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.526+0000 7f5d7ad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d7c1562f0 0x7f5d7c0c0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:32.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.526+0000 7f5d7ad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d7c1562f0 0x7f5d7c0c0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59628/0 (socket says 172.21.15.121:59628) 2023-11-15T03:05:32.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.526+0000 7f5d7ad9d700 1 -- 172.21.15.121:0/2123229317 learned_addr learned my addr 172.21.15.121:0/2123229317 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:32.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.526+0000 7f5d7a59c700 1 --2- 172.21.15.121:0/2123229317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d7c0a8f40 0x7f5d7c14e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:32.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.526+0000 7f5d71d9b700 1 --2- 172.21.15.121:0/2123229317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d7c154ca0 0x7f5d7c0bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:32.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.526+0000 7f5d7ad9d700 1 -- 172.21.15.121:0/2123229317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d7c0a8f40 msgr2=0x7f5d7c14e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:32.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.526+0000 7f5d7ad9d700 1 --2- 172.21.15.121:0/2123229317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d7c0a8f40 0x7f5d7c14e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.526+0000 7f5d7ad9d700 1 -- 172.21.15.121:0/2123229317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d7c154ca0 msgr2=0x7f5d7c0bfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:32.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.526+0000 7f5d7ad9d700 1 --2- 172.21.15.121:0/2123229317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d7c154ca0 0x7f5d7c0bfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.526+0000 7f5d7ad9d700 1 -- 172.21.15.121:0/2123229317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d640092a0 con 0x7f5d7c1562f0 2023-11-15T03:05:32.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.526+0000 7f5d7a59c700 1 --2- 172.21.15.121:0/2123229317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d7c0a8f40 0x7f5d7c14e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:32.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.526+0000 7f5d71d9b700 1 --2- 172.21.15.121:0/2123229317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d7c154ca0 0x7f5d7c0bfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:32.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.526+0000 7f5d7ad9d700 1 --2- 172.21.15.121:0/2123229317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d7c1562f0 0x7f5d7c0c0080 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7f5d74009e50 tx=0x7f5d7400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:32.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.526+0000 7f5d72ffd700 1 -- 172.21.15.121:0/2123229317 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7400ca10 con 0x7f5d7c1562f0 2023-11-15T03:05:32.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.526+0000 7f5d72ffd700 1 -- 172.21.15.121:0/2123229317 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7400f040 con 0x7f5d7c1562f0 2023-11-15T03:05:32.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.526+0000 7f5d80a1b700 1 -- 172.21.15.121:0/2123229317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d7c168070 con 0x7f5d7c1562f0 2023-11-15T03:05:32.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.526+0000 7f5d72ffd700 1 -- 172.21.15.121:0/2123229317 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7400cd80 con 0x7f5d7c1562f0 2023-11-15T03:05:32.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.526+0000 7f5d80a1b700 1 -- 172.21.15.121:0/2123229317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7c168630 con 0x7f5d7c1562f0 2023-11-15T03:05:32.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.526+0000 7f5d80a1b700 1 -- 172.21.15.121:0/2123229317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d7c168850 con 0x7f5d7c1562f0 2023-11-15T03:05:32.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.530+0000 7f5d72ffd700 1 -- 172.21.15.121:0/2123229317 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5d74014620 con 0x7f5d7c1562f0 2023-11-15T03:05:32.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.530+0000 7f5d72ffd700 1 --2- 172.21.15.121:0/2123229317 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5d5c05a510 0x7f5d5c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:32.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.530+0000 7f5d72ffd700 1 -- 172.21.15.121:0/2123229317 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7298+0+0 (secure 0 0 0) 0x7f5d74074e30 con 0x7f5d7c1562f0 2023-11-15T03:05:32.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.534+0000 7f5d7a59c700 1 --2- 172.21.15.121:0/2123229317 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5d5c05a510 0x7f5d5c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:32.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.534+0000 7f5d7a59c700 1 --2- 172.21.15.121:0/2123229317 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5d5c05a510 0x7f5d5c05c9d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f5d6c009b70 tx=0x7f5d6c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:32.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.538+0000 7f5d72ffd700 1 -- 172.21.15.121:0/2123229317 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5d74048170 con 0x7f5d7c1562f0 2023-11-15T03:05:32.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.734+0000 7f5d80a1b700 1 -- 172.21.15.121:0/2123229317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d7c0a7230 con 0x7f5d7c1562f0 2023-11-15T03:05:32.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.734+0000 7f5d72ffd700 1 -- 172.21.15.121:0/2123229317 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f5d7404b790 con 0x7f5d7c1562f0 2023-11-15T03:05:32.744 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:32.745 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":154,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:32.321188+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8271":{"gid":8271,"name":"a","rank":0,"incarnation":154,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/1529582418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1529582418},{"type":"v1","addr":"172.21.15.121:6836","nonce":1529582418}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:05:32.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.738+0000 7f5d80a1b700 1 -- 172.21.15.121:0/2123229317 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5d5c05a510 msgr2=0x7f5d5c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:32.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.738+0000 7f5d80a1b700 1 --2- 172.21.15.121:0/2123229317 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5d5c05a510 0x7f5d5c05c9d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f5d6c009b70 tx=0x7f5d6c008040).stop 2023-11-15T03:05:32.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.738+0000 7f5d80a1b700 1 -- 172.21.15.121:0/2123229317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d7c1562f0 msgr2=0x7f5d7c0c0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:32.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.738+0000 7f5d80a1b700 1 --2- 172.21.15.121:0/2123229317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d7c1562f0 0x7f5d7c0c0080 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7f5d74009e50 tx=0x7f5d7400b040).stop 2023-11-15T03:05:32.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.738+0000 7f5d80a1b700 1 -- 172.21.15.121:0/2123229317 shutdown_connections 2023-11-15T03:05:32.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.738+0000 7f5d80a1b700 1 --2- 172.21.15.121:0/2123229317 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5d5c05a510 0x7f5d5c05c9d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.738+0000 7f5d80a1b700 1 --2- 172.21.15.121:0/2123229317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d7c1562f0 0x7f5d7c0c0080 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.738+0000 7f5d80a1b700 1 --2- 172.21.15.121:0/2123229317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d7c154ca0 0x7f5d7c0bfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.738+0000 7f5d80a1b700 1 --2- 172.21.15.121:0/2123229317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d7c0a8f40 0x7f5d7c14e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:32.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.738+0000 7f5d80a1b700 1 -- 172.21.15.121:0/2123229317 >> 172.21.15.121:0/2123229317 conn(0x7f5d7c062190 msgr2=0x7f5d7c066690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:32.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.738+0000 7f5d80a1b700 1 -- 172.21.15.121:0/2123229317 shutdown_connections 2023-11-15T03:05:32.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:32.738+0000 7f5d80a1b700 1 -- 172.21.15.121:0/2123229317 wait complete. 2023-11-15T03:05:32.751 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 154 2023-11-15T03:05:32.769 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:05:33.332 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:05:33.322+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:05:33.332 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:05:33.322+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:05:33.341 INFO:tasks.ceph.mds.a.smithi121.stderr: -28> 2023-11-15T03:05:32.326+0000 7f80797cc700 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2023-11-15T03:05:33.342 INFO:tasks.ceph.mds.a.smithi121.stderr: -27> 2023-11-15T03:05:32.326+0000 7f80797cc700 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2023-11-15T03:05:33.385 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:05:37.770 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:05:37.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.942+0000 7fb345369700 1 Processor -- start 2023-11-15T03:05:37.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.942+0000 7fb345369700 1 -- start start 2023-11-15T03:05:37.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.942+0000 7fb345369700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb3400ac260 0x7fb340066a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:37.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.942+0000 7fb345369700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3400c3820 0x7fb3400c1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:37.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.942+0000 7fb345369700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3400c1e60 con 0x7fb3400ac640 2023-11-15T03:05:37.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb345369700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3400665f0 con 0x7fb3400ac260 2023-11-15T03:05:37.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb345369700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb340065d30 con 0x7fb3400c3820 2023-11-15T03:05:37.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb33effd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb3400ac260 0x7fb340066a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:37.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb33e7fc700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb3400ac640 0x7fb3400c3110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:36994/0 (socket says 172.21.15.121:36994) 2023-11-15T03:05:37.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb33e7fc700 1 -- 172.21.15.121:0/3221524627 learned_addr learned my addr 172.21.15.121:0/3221524627 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:37.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb33f7fe700 1 --2- 172.21.15.121:0/3221524627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3400c3820 0x7fb3400c1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:37.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb33effd700 1 -- 172.21.15.121:0/3221524627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3400c3820 msgr2=0x7fb3400c1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:37.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb33effd700 1 --2- 172.21.15.121:0/3221524627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3400c3820 0x7fb3400c1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:37.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb33effd700 1 -- 172.21.15.121:0/3221524627 >> v1:172.21.15.121:6789/0 conn(0x7fb3400ac640 legacy=0x7fb3400c3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:37.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb33effd700 1 -- 172.21.15.121:0/3221524627 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3400c2140 con 0x7fb3400ac260 2023-11-15T03:05:37.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb33effd700 1 --2- 172.21.15.121:0/3221524627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3400ac260 0x7fb340066a10 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7fb330009760 tx=0x7fb330009060).ready entity=mon.0 client_cookie=f9e6eeca68f67882 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:37.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb33dffb700 1 -- 172.21.15.121:0/3221524627 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33000f510 con 0x7fb3400ac260 2023-11-15T03:05:37.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb33dffb700 1 -- 172.21.15.121:0/3221524627 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33000f670 con 0x7fb3400ac260 2023-11-15T03:05:37.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb33dffb700 1 -- 172.21.15.121:0/3221524627 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33000f9f0 con 0x7fb3400ac260 2023-11-15T03:05:37.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb345369700 1 -- 172.21.15.121:0/3221524627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3400ac260 msgr2=0x7fb340066a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:37.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb345369700 1 --2- 172.21.15.121:0/3221524627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3400ac260 0x7fb340066a10 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7fb330009760 tx=0x7fb330009060).stop 2023-11-15T03:05:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb345369700 1 -- 172.21.15.121:0/3221524627 shutdown_connections 2023-11-15T03:05:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb345369700 1 --2- 172.21.15.121:0/3221524627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3400c3820 0x7fb3400c1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb345369700 1 --2- 172.21.15.121:0/3221524627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3400ac260 0x7fb340066a10 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.946+0000 7fb345369700 1 -- 172.21.15.121:0/3221524627 >> 172.21.15.121:0/3221524627 conn(0x7fb340062190 msgr2=0x7fb340062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:37.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.950+0000 7fb345369700 1 -- 172.21.15.121:0/3221524627 shutdown_connections 2023-11-15T03:05:37.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.950+0000 7fb345369700 1 -- 172.21.15.121:0/3221524627 wait complete. 2023-11-15T03:05:37.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.950+0000 7fb345369700 1 Processor -- start 2023-11-15T03:05:37.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.950+0000 7fb345369700 1 -- start start 2023-11-15T03:05:37.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.950+0000 7fb345369700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3400ac260 0x7fb340156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:37.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.950+0000 7fb345369700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3400c3820 0x7fb3400c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:37.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.950+0000 7fb345369700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3400c4a00 0x7fb3400c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:37.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.950+0000 7fb345369700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3400c4fb0 con 0x7fb3400c3820 2023-11-15T03:05:37.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.950+0000 7fb345369700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3400c5290 con 0x7fb3400ac260 2023-11-15T03:05:37.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.950+0000 7fb345369700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3400c5570 con 0x7fb3400c4a00 2023-11-15T03:05:37.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.954+0000 7fb33e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3400c3820 0x7fb3400c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:37.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.954+0000 7fb33e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3400c3820 0x7fb3400c4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49188/0 (socket says 172.21.15.121:49188) 2023-11-15T03:05:37.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.954+0000 7fb33e7fc700 1 -- 172.21.15.121:0/1397990266 learned_addr learned my addr 172.21.15.121:0/1397990266 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:37.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.954+0000 7fb33f7fe700 1 --2- 172.21.15.121:0/1397990266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3400c4a00 0x7fb3400c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:37.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.954+0000 7fb33effd700 1 --2- 172.21.15.121:0/1397990266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3400ac260 0x7fb340156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:37.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.954+0000 7fb33e7fc700 1 -- 172.21.15.121:0/1397990266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3400c4a00 msgr2=0x7fb3400c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:37.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.954+0000 7fb33e7fc700 1 --2- 172.21.15.121:0/1397990266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3400c4a00 0x7fb3400c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:37.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.954+0000 7fb33e7fc700 1 -- 172.21.15.121:0/1397990266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3400ac260 msgr2=0x7fb340156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:37.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.954+0000 7fb33e7fc700 1 --2- 172.21.15.121:0/1397990266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3400ac260 0x7fb340156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:37.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.954+0000 7fb33e7fc700 1 -- 172.21.15.121:0/1397990266 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3300092a0 con 0x7fb3400c3820 2023-11-15T03:05:37.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.954+0000 7fb33e7fc700 1 --2- 172.21.15.121:0/1397990266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3400c3820 0x7fb3400c4510 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7fb334007fd0 tx=0x7fb334007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:37.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.954+0000 7fb326ffd700 1 -- 172.21.15.121:0/1397990266 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33400ce10 con 0x7fb3400c3820 2023-11-15T03:05:37.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.954+0000 7fb326ffd700 1 -- 172.21.15.121:0/1397990266 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3340072e0 con 0x7fb3400c3820 2023-11-15T03:05:37.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.954+0000 7fb326ffd700 1 -- 172.21.15.121:0/1397990266 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334007660 con 0x7fb3400c3820 2023-11-15T03:05:37.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.954+0000 7fb345369700 1 -- 172.21.15.121:0/1397990266 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3400c5850 con 0x7fb3400c3820 2023-11-15T03:05:37.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.954+0000 7fb345369700 1 -- 172.21.15.121:0/1397990266 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3401571f0 con 0x7fb3400c3820 2023-11-15T03:05:37.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.954+0000 7fb345369700 1 -- 172.21.15.121:0/1397990266 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3400c5d70 con 0x7fb3400c3820 2023-11-15T03:05:37.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.958+0000 7fb326ffd700 1 -- 172.21.15.121:0/1397990266 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb3340059b0 con 0x7fb3400c3820 2023-11-15T03:05:37.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.958+0000 7fb326ffd700 1 --2- 172.21.15.121:0/1397990266 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb32c05a510 0x7fb32c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:37.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.958+0000 7fb326ffd700 1 -- 172.21.15.121:0/1397990266 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7298+0+0 (secure 0 0 0) 0x7fb334074370 con 0x7fb3400c3820 2023-11-15T03:05:37.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.958+0000 7fb33effd700 1 --2- 172.21.15.121:0/1397990266 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb32c05a510 0x7fb32c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:37.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.958+0000 7fb33effd700 1 --2- 172.21.15.121:0/1397990266 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb32c05a510 0x7fb32c05c9d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fb330009fd0 tx=0x7fb330017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:37.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:37.966+0000 7fb326ffd700 1 -- 172.21.15.121:0/1397990266 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb3400c5d70 con 0x7fb3400c3820 2023-11-15T03:05:38.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.170+0000 7fb345369700 1 -- 172.21.15.121:0/1397990266 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb34006bcf0 con 0x7fb3400c3820 2023-11-15T03:05:38.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.170+0000 7fb326ffd700 1 -- 172.21.15.121:0/1397990266 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+4496 (secure 0 0 0) 0x7fb3340476b0 con 0x7fb3400c3820 2023-11-15T03:05:38.181 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:38.181 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":156,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8337,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3454441824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3454441824},{"type":"v1","addr":"172.21.15.121:6836","nonce":3454441824}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:33.328281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:05:38.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.174+0000 7fb345369700 1 -- 172.21.15.121:0/1397990266 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb32c05a510 msgr2=0x7fb32c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:38.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.174+0000 7fb345369700 1 --2- 172.21.15.121:0/1397990266 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb32c05a510 0x7fb32c05c9d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fb330009fd0 tx=0x7fb330017000).stop 2023-11-15T03:05:38.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.174+0000 7fb345369700 1 -- 172.21.15.121:0/1397990266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3400c3820 msgr2=0x7fb3400c4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:38.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.174+0000 7fb345369700 1 --2- 172.21.15.121:0/1397990266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3400c3820 0x7fb3400c4510 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7fb334007fd0 tx=0x7fb334007df0).stop 2023-11-15T03:05:38.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.174+0000 7fb345369700 1 -- 172.21.15.121:0/1397990266 shutdown_connections 2023-11-15T03:05:38.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.174+0000 7fb345369700 1 --2- 172.21.15.121:0/1397990266 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb32c05a510 0x7fb32c05c9d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.174+0000 7fb345369700 1 --2- 172.21.15.121:0/1397990266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3400c4a00 0x7fb3400c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.174+0000 7fb345369700 1 --2- 172.21.15.121:0/1397990266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3400c3820 0x7fb3400c4510 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.174+0000 7fb345369700 1 --2- 172.21.15.121:0/1397990266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3400ac260 0x7fb340156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.174+0000 7fb345369700 1 -- 172.21.15.121:0/1397990266 >> 172.21.15.121:0/1397990266 conn(0x7fb340062190 msgr2=0x7fb3401578e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:38.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.174+0000 7fb345369700 1 -- 172.21.15.121:0/1397990266 shutdown_connections 2023-11-15T03:05:38.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.174+0000 7fb345369700 1 -- 172.21.15.121:0/1397990266 wait complete. 2023-11-15T03:05:38.186 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 156 2023-11-15T03:05:38.205 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:05:38.206 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:05:38.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.350+0000 7f5c43479700 1 Processor -- start 2023-11-15T03:05:38.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.350+0000 7f5c43479700 1 -- start start 2023-11-15T03:05:38.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.354+0000 7f5c43479700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5c3c151870 0x7f5c3c151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:38.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.354+0000 7f5c43479700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c3c0657f0 con 0x7f5c3c0c30e0 2023-11-15T03:05:38.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.354+0000 7f5c43479700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c3c065ad0 con 0x7f5c3c152140 2023-11-15T03:05:38.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.354+0000 7f5c43479700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c3c0aa260 con 0x7f5c3c151870 2023-11-15T03:05:38.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.354+0000 7f5c42c78700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f5c3c0c30e0 0x7f5c3c065f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:37006/0 (socket says 172.21.15.121:37006) 2023-11-15T03:05:38.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.354+0000 7f5c42477700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5c3c151870 0x7f5c3c151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:38.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.354+0000 7f5c42c78700 1 -- 172.21.15.121:0/2333709765 learned_addr learned my addr 172.21.15.121:0/2333709765 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:38.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.354+0000 7f5c42477700 1 -- 172.21.15.121:0/2333709765 >> v1:172.21.15.190:6789/0 conn(0x7f5c3c152140 legacy=0x7f5c3c0a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:38.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.354+0000 7f5c42477700 1 -- 172.21.15.121:0/2333709765 >> v1:172.21.15.121:6789/0 conn(0x7f5c3c0c30e0 legacy=0x7f5c3c065f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:38.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.354+0000 7f5c42477700 1 -- 172.21.15.121:0/2333709765 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3c0aa540 con 0x7f5c3c151870 2023-11-15T03:05:38.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.354+0000 7f5c42477700 1 --2- 172.21.15.121:0/2333709765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c151870 0x7f5c3c151c50 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7f5c38009e60 tx=0x7f5c38009780).ready entity=mon.0 client_cookie=368ba4ac39a29a9d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:38.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.354+0000 7f5c41475700 1 -- 172.21.15.121:0/2333709765 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3800e070 con 0x7f5c3c151870 2023-11-15T03:05:38.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.358+0000 7f5c41475700 1 -- 172.21.15.121:0/2333709765 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3800bbf0 con 0x7f5c3c151870 2023-11-15T03:05:38.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.358+0000 7f5c41475700 1 -- 172.21.15.121:0/2333709765 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c38012410 con 0x7f5c3c151870 2023-11-15T03:05:38.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.358+0000 7f5c43479700 1 -- 172.21.15.121:0/2333709765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c151870 msgr2=0x7f5c3c151c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:38.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.358+0000 7f5c43479700 1 --2- 172.21.15.121:0/2333709765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c151870 0x7f5c3c151c50 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7f5c38009e60 tx=0x7f5c38009780).stop 2023-11-15T03:05:38.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.358+0000 7f5c43479700 1 -- 172.21.15.121:0/2333709765 shutdown_connections 2023-11-15T03:05:38.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.358+0000 7f5c43479700 1 --2- 172.21.15.121:0/2333709765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c151870 0x7f5c3c151c50 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.358+0000 7f5c43479700 1 -- 172.21.15.121:0/2333709765 >> 172.21.15.121:0/2333709765 conn(0x7f5c3c062190 msgr2=0x7f5c3c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:38.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.358+0000 7f5c43479700 1 -- 172.21.15.121:0/2333709765 shutdown_connections 2023-11-15T03:05:38.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.358+0000 7f5c43479700 1 -- 172.21.15.121:0/2333709765 wait complete. 2023-11-15T03:05:38.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.358+0000 7f5c43479700 1 Processor -- start 2023-11-15T03:05:38.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.358+0000 7f5c43479700 1 -- start start 2023-11-15T03:05:38.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c43479700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c0c30e0 0x7f5c3c0bfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:38.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c43479700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c3c151870 0x7f5c3c0c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:38.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c43479700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c3c152140 0x7f5c3c14b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:38.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c43479700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c162230 con 0x7f5c3c0c30e0 2023-11-15T03:05:38.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c43479700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c0c0380 con 0x7f5c3c152140 2023-11-15T03:05:38.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c43479700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c162420 con 0x7f5c3c151870 2023-11-15T03:05:38.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c42477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c0c30e0 0x7f5c3c0bfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:38.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c42477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c0c30e0 0x7f5c3c0bfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49208/0 (socket says 172.21.15.121:49208) 2023-11-15T03:05:38.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c42477700 1 -- 172.21.15.121:0/2870041156 learned_addr learned my addr 172.21.15.121:0/2870041156 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:38.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c41c76700 1 --2- 172.21.15.121:0/2870041156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c3c151870 0x7f5c3c0c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:38.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c42c78700 1 --2- 172.21.15.121:0/2870041156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c3c152140 0x7f5c3c14b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:38.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c42477700 1 -- 172.21.15.121:0/2870041156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c3c151870 msgr2=0x7f5c3c0c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:38.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c42477700 1 --2- 172.21.15.121:0/2870041156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c3c151870 0x7f5c3c0c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c42477700 1 -- 172.21.15.121:0/2870041156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c3c152140 msgr2=0x7f5c3c14b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:38.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c42477700 1 --2- 172.21.15.121:0/2870041156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c3c152140 0x7f5c3c14b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c42477700 1 -- 172.21.15.121:0/2870041156 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c38009a20 con 0x7f5c3c0c30e0 2023-11-15T03:05:38.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c42477700 1 --2- 172.21.15.121:0/2870041156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c0c30e0 0x7f5c3c0bfdc0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7f5c3800b6c0 tx=0x7f5c38009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:38.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c327fc700 1 -- 172.21.15.121:0/2870041156 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3801c660 con 0x7f5c3c0c30e0 2023-11-15T03:05:38.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c327fc700 1 -- 172.21.15.121:0/2870041156 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3801c7c0 con 0x7f5c3c0c30e0 2023-11-15T03:05:38.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c43479700 1 -- 172.21.15.121:0/2870041156 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3c162700 con 0x7f5c3c0c30e0 2023-11-15T03:05:38.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c327fc700 1 -- 172.21.15.121:0/2870041156 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3801cb60 con 0x7f5c3c0c30e0 2023-11-15T03:05:38.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c43479700 1 -- 172.21.15.121:0/2870041156 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3c0c5ff0 con 0x7f5c3c0c30e0 2023-11-15T03:05:38.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.362+0000 7f5c43479700 1 -- 172.21.15.121:0/2870041156 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3c06bcf0 con 0x7f5c3c0c30e0 2023-11-15T03:05:38.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.366+0000 7f5c327fc700 1 -- 172.21.15.121:0/2870041156 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5c3801d420 con 0x7f5c3c0c30e0 2023-11-15T03:05:38.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.366+0000 7f5c327fc700 1 --2- 172.21.15.121:0/2870041156 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5c2005a510 0x7f5c2005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:38.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.366+0000 7f5c327fc700 1 -- 172.21.15.121:0/2870041156 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7298+0+0 (secure 0 0 0) 0x7f5c38076ca0 con 0x7f5c3c0c30e0 2023-11-15T03:05:38.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.366+0000 7f5c41c76700 1 --2- 172.21.15.121:0/2870041156 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5c2005a510 0x7f5c2005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:38.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.370+0000 7f5c41c76700 1 --2- 172.21.15.121:0/2870041156 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5c2005a510 0x7f5c2005c9d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f5c3c0c02b0 tx=0x7f5c2c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:38.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.374+0000 7f5c327fc700 1 -- 172.21.15.121:0/2870041156 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5c38049ec0 con 0x7f5c3c0c30e0 2023-11-15T03:05:38.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.574+0000 7f5c43479700 1 -- 172.21.15.121:0/2870041156 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c3c0aa820 con 0x7f5c3c0c30e0 2023-11-15T03:05:38.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.574+0000 7f5c327fc700 1 -- 172.21.15.121:0/2870041156 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f5c3804d4e0 con 0x7f5c3c0c30e0 2023-11-15T03:05:38.585 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:38.585 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":156,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8337,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3454441824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3454441824},{"type":"v1","addr":"172.21.15.121:6836","nonce":3454441824}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:33.328281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:05:38.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.578+0000 7f5c43479700 1 -- 172.21.15.121:0/2870041156 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5c2005a510 msgr2=0x7f5c2005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:38.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.578+0000 7f5c43479700 1 --2- 172.21.15.121:0/2870041156 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5c2005a510 0x7f5c2005c9d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f5c3c0c02b0 tx=0x7f5c2c009040).stop 2023-11-15T03:05:38.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.578+0000 7f5c43479700 1 -- 172.21.15.121:0/2870041156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c0c30e0 msgr2=0x7f5c3c0bfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:38.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.578+0000 7f5c43479700 1 --2- 172.21.15.121:0/2870041156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c0c30e0 0x7f5c3c0bfdc0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7f5c3800b6c0 tx=0x7f5c38009780).stop 2023-11-15T03:05:38.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.578+0000 7f5c43479700 1 -- 172.21.15.121:0/2870041156 shutdown_connections 2023-11-15T03:05:38.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.578+0000 7f5c43479700 1 --2- 172.21.15.121:0/2870041156 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5c2005a510 0x7f5c2005c9d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.578+0000 7f5c43479700 1 --2- 172.21.15.121:0/2870041156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c3c152140 0x7f5c3c14b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.578+0000 7f5c43479700 1 --2- 172.21.15.121:0/2870041156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c3c151870 0x7f5c3c0c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.578+0000 7f5c43479700 1 --2- 172.21.15.121:0/2870041156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c0c30e0 0x7f5c3c0bfdc0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.578+0000 7f5c43479700 1 -- 172.21.15.121:0/2870041156 >> 172.21.15.121:0/2870041156 conn(0x7f5c3c062190 msgr2=0x7f5c3c0a9760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:38.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.578+0000 7f5c43479700 1 -- 172.21.15.121:0/2870041156 shutdown_connections 2023-11-15T03:05:38.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.578+0000 7f5c43479700 1 -- 172.21.15.121:0/2870041156 wait complete. 2023-11-15T03:05:38.593 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 156 2023-11-15T03:05:38.607 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:05:38.607 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:05:38.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.758+0000 7f23c3edb700 1 Processor -- start 2023-11-15T03:05:38.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.762+0000 7f23c3edb700 1 -- start start 2023-11-15T03:05:38.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.762+0000 7f23c3edb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23bc150ff0 0x7f23bc151470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:38.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.762+0000 7f23c3edb700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f23bc0c6640 0x7f23bc0c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:38.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.762+0000 7f23c3edb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23bc152000 con 0x7f23bc14af80 2023-11-15T03:05:38.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.762+0000 7f23c3edb700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23bc0c1270 con 0x7f23bc0c6640 2023-11-15T03:05:38.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.762+0000 7f23c3edb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0c6fc0 con 0x7f23bc150ff0 2023-11-15T03:05:38.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.762+0000 7f23c1c77700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f23bc14af80 0x7f23bc0c1700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:37012/0 (socket says 172.21.15.121:37012) 2023-11-15T03:05:38.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.762+0000 7f23c1c77700 1 -- 172.21.15.121:0/408846804 learned_addr learned my addr 172.21.15.121:0/408846804 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:38.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.762+0000 7f23c2478700 1 --2- 172.21.15.121:0/408846804 >> v2:172.21.15.121:3300/0 conn(0x7f23bc0c6640 0x7f23bc0c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:38.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.762+0000 7f23c1476700 1 --2- 172.21.15.121:0/408846804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23bc150ff0 0x7f23bc151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:38.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.762+0000 7f23c0c75700 1 -- 172.21.15.121:0/408846804 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821917376 0 0) 0x7f23b8001110 con 0x7f23bc14af80 2023-11-15T03:05:38.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.762+0000 7f23c0c75700 1 -- 172.21.15.121:0/408846804 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23a80033f0 con 0x7f23bc14af80 2023-11-15T03:05:38.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.766+0000 7f23c0c75700 1 -- 172.21.15.121:0/408846804 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2820478594 0 0) 0x7f23a80033f0 con 0x7f23bc14af80 2023-11-15T03:05:38.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.766+0000 7f23c2478700 1 -- 172.21.15.121:0/408846804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23bc150ff0 msgr2=0x7f23bc151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:38.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.766+0000 7f23c2478700 1 --2- 172.21.15.121:0/408846804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23bc150ff0 0x7f23bc151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.766+0000 7f23c2478700 1 -- 172.21.15.121:0/408846804 >> v1:172.21.15.121:6789/0 conn(0x7f23bc14af80 legacy=0x7f23bc0c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:38.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.766+0000 7f23c2478700 1 -- 172.21.15.121:0/408846804 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc0c1e10 con 0x7f23bc0c6640 2023-11-15T03:05:38.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.766+0000 7f23c2478700 1 --2- 172.21.15.121:0/408846804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23bc0c6640 0x7f23bc0c6ad0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7f23b4009e60 tx=0x7f23b4009780).ready entity=mon.0 client_cookie=af0ad93a264e3981 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:38.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.766+0000 7f23c0c75700 1 -- 172.21.15.121:0/408846804 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b400e070 con 0x7f23bc0c6640 2023-11-15T03:05:38.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.766+0000 7f23c0c75700 1 -- 172.21.15.121:0/408846804 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b400b7a0 con 0x7f23bc0c6640 2023-11-15T03:05:38.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.766+0000 7f23c0c75700 1 -- 172.21.15.121:0/408846804 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8001600 con 0x7f23bc0c6640 2023-11-15T03:05:38.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.766+0000 7f23c3edb700 1 -- 172.21.15.121:0/408846804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23bc0c6640 msgr2=0x7f23bc0c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:38.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.766+0000 7f23c3edb700 1 --2- 172.21.15.121:0/408846804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23bc0c6640 0x7f23bc0c6ad0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7f23b4009e60 tx=0x7f23b4009780).stop 2023-11-15T03:05:38.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.766+0000 7f23c3edb700 1 -- 172.21.15.121:0/408846804 shutdown_connections 2023-11-15T03:05:38.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.766+0000 7f23c3edb700 1 --2- 172.21.15.121:0/408846804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23bc0c6640 0x7f23bc0c6ad0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.766+0000 7f23c3edb700 1 --2- 172.21.15.121:0/408846804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23bc150ff0 0x7f23bc151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.766+0000 7f23c3edb700 1 -- 172.21.15.121:0/408846804 >> 172.21.15.121:0/408846804 conn(0x7f23bc062190 msgr2=0x7f23bc0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:38.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.766+0000 7f23c3edb700 1 -- 172.21.15.121:0/408846804 shutdown_connections 2023-11-15T03:05:38.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.766+0000 7f23c3edb700 1 -- 172.21.15.121:0/408846804 wait complete. 2023-11-15T03:05:38.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.770+0000 7f23c3edb700 1 Processor -- start 2023-11-15T03:05:38.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.770+0000 7f23c3edb700 1 -- start start 2023-11-15T03:05:38.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.770+0000 7f23c3edb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23bc0c6640 0x7f23bc139ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:38.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.770+0000 7f23c3edb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23bc14af80 0x7f23bc0aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:38.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.770+0000 7f23c3edb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23bc150ff0 0x7f23bc0ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:38.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.770+0000 7f23c3edb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0ac040 con 0x7f23bc150ff0 2023-11-15T03:05:38.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.770+0000 7f23c3edb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0ab5c0 con 0x7f23bc14af80 2023-11-15T03:05:38.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.770+0000 7f23c3edb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0ab7a0 con 0x7f23bc0c6640 2023-11-15T03:05:38.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.770+0000 7f23c2478700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23bc150ff0 0x7f23bc0ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:38.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.770+0000 7f23c2478700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23bc150ff0 0x7f23bc0ab0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49232/0 (socket says 172.21.15.121:49232) 2023-11-15T03:05:38.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.770+0000 7f23c2478700 1 -- 172.21.15.121:0/615904733 learned_addr learned my addr 172.21.15.121:0/615904733 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:38.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.770+0000 7f23c1476700 1 --2- 172.21.15.121:0/615904733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23bc14af80 0x7f23bc0aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:38.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.770+0000 7f23c1c77700 1 --2- 172.21.15.121:0/615904733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23bc0c6640 0x7f23bc139ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:38.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.774+0000 7f23c2478700 1 -- 172.21.15.121:0/615904733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23bc0c6640 msgr2=0x7f23bc139ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:38.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.774+0000 7f23c2478700 1 --2- 172.21.15.121:0/615904733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23bc0c6640 0x7f23bc139ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.774+0000 7f23c2478700 1 -- 172.21.15.121:0/615904733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23bc14af80 msgr2=0x7f23bc0aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:38.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.774+0000 7f23c2478700 1 --2- 172.21.15.121:0/615904733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23bc14af80 0x7f23bc0aabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.774+0000 7f23c2478700 1 -- 172.21.15.121:0/615904733 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b4009a20 con 0x7f23bc150ff0 2023-11-15T03:05:38.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.774+0000 7f23c2478700 1 --2- 172.21.15.121:0/615904733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23bc150ff0 0x7f23bc0ab0d0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7f23b400b610 tx=0x7f23b4009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:38.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.774+0000 7f23b1ffb700 1 -- 172.21.15.121:0/615904733 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b401d420 con 0x7f23bc150ff0 2023-11-15T03:05:38.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.774+0000 7f23b1ffb700 1 -- 172.21.15.121:0/615904733 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b401d580 con 0x7f23bc150ff0 2023-11-15T03:05:38.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.774+0000 7f23b1ffb700 1 -- 172.21.15.121:0/615904733 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b401d940 con 0x7f23bc150ff0 2023-11-15T03:05:38.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.774+0000 7f23c3edb700 1 -- 172.21.15.121:0/615904733 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23bc0aba80 con 0x7f23bc150ff0 2023-11-15T03:05:38.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.774+0000 7f23c3edb700 1 -- 172.21.15.121:0/615904733 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23bc14b500 con 0x7f23bc150ff0 2023-11-15T03:05:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.774+0000 7f23c3edb700 1 -- 172.21.15.121:0/615904733 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23bc14b360 con 0x7f23bc150ff0 2023-11-15T03:05:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.774+0000 7f23b1ffb700 1 -- 172.21.15.121:0/615904733 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f23b401dd30 con 0x7f23bc150ff0 2023-11-15T03:05:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.778+0000 7f23b1ffb700 1 --2- 172.21.15.121:0/615904733 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f23a005a540 0x7f23a005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.778+0000 7f23b1ffb700 1 -- 172.21.15.121:0/615904733 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7298+0+0 (secure 0 0 0) 0x7f23b4075ff0 con 0x7f23bc150ff0 2023-11-15T03:05:38.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.778+0000 7f23c1c77700 1 --2- 172.21.15.121:0/615904733 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f23a005a540 0x7f23a005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:38.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.778+0000 7f23c1c77700 1 --2- 172.21.15.121:0/615904733 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f23a005a540 0x7f23a005ca00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f23b80017b0 tx=0x7f23b8001dd0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:38.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.782+0000 7f23b1ffb700 1 -- 172.21.15.121:0/615904733 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f23b40a4af0 con 0x7f23bc150ff0 2023-11-15T03:05:38.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.974+0000 7f23c3edb700 1 -- 172.21.15.121:0/615904733 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23bc06c150 con 0x7f23bc150ff0 2023-11-15T03:05:38.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.974+0000 7f23b1ffb700 1 -- 172.21.15.121:0/615904733 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f23bc14b360 con 0x7f23bc150ff0 2023-11-15T03:05:38.985 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:38.985 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":156,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8337,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3454441824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3454441824},{"type":"v1","addr":"172.21.15.121:6836","nonce":3454441824}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:33.328281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:05:38.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.978+0000 7f23c3edb700 1 -- 172.21.15.121:0/615904733 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f23a005a540 msgr2=0x7f23a005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:38.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.978+0000 7f23c3edb700 1 --2- 172.21.15.121:0/615904733 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f23a005a540 0x7f23a005ca00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f23b80017b0 tx=0x7f23b8001dd0).stop 2023-11-15T03:05:38.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.978+0000 7f23c3edb700 1 -- 172.21.15.121:0/615904733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23bc150ff0 msgr2=0x7f23bc0ab0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:38.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.978+0000 7f23c3edb700 1 --2- 172.21.15.121:0/615904733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23bc150ff0 0x7f23bc0ab0d0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7f23b400b610 tx=0x7f23b4009780).stop 2023-11-15T03:05:38.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.978+0000 7f23c3edb700 1 -- 172.21.15.121:0/615904733 shutdown_connections 2023-11-15T03:05:38.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.978+0000 7f23c3edb700 1 --2- 172.21.15.121:0/615904733 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f23a005a540 0x7f23a005ca00 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.978+0000 7f23c3edb700 1 --2- 172.21.15.121:0/615904733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23bc150ff0 0x7f23bc0ab0d0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.978+0000 7f23c3edb700 1 --2- 172.21.15.121:0/615904733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23bc14af80 0x7f23bc0aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.978+0000 7f23c3edb700 1 --2- 172.21.15.121:0/615904733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23bc0c6640 0x7f23bc139ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:38.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.978+0000 7f23c3edb700 1 -- 172.21.15.121:0/615904733 >> 172.21.15.121:0/615904733 conn(0x7f23bc062190 msgr2=0x7f23bc0a8330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:38.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.978+0000 7f23c3edb700 1 -- 172.21.15.121:0/615904733 shutdown_connections 2023-11-15T03:05:38.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:38.978+0000 7f23c3edb700 1 -- 172.21.15.121:0/615904733 wait complete. 2023-11-15T03:05:38.990 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 156 2023-11-15T03:05:39.007 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_sessionmap' led to DAMAGED state 2023-11-15T03:05:39.008 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 400.00000000' 2023-11-15T03:05:39.008 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:05:39.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:05:39.009 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:05:39.025 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:05:39.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:05:39.027 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:05:39.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.218+0000 7f4ddd7f9700 1 Processor -- start 2023-11-15T03:05:39.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.218+0000 7f4ddd7f9700 1 -- start start 2023-11-15T03:05:39.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.218+0000 7f4ddd7f9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4dd80abaa0 0x7f4dd80abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:39.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.218+0000 7f4ddd7f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4dd80bb510 0x7f4dd8063570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:39.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.218+0000 7f4ddd7f9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd80c2c00 con 0x7f4dd80a8880 2023-11-15T03:05:39.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.218+0000 7f4ddd7f9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd8063a60 con 0x7f4dd80abaa0 2023-11-15T03:05:39.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.218+0000 7f4ddd7f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd80657b0 con 0x7f4dd80bb510 2023-11-15T03:05:39.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.218+0000 7f4dd7fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4dd80abaa0 0x7f4dd80abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:39.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.218+0000 7f4dd7fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4dd80abaa0 0x7f4dd80abe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49236/0 (socket says 172.21.15.121:49236) 2023-11-15T03:05:39.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.218+0000 7f4dd7fff700 1 -- 172.21.15.121:0/3923698680 learned_addr learned my addr 172.21.15.121:0/3923698680 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:39.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.218+0000 7f4dcffff700 1 --2- 172.21.15.121:0/3923698680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4dd80bb510 0x7f4dd8063570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:39.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.218+0000 7f4dd7fff700 1 -- 172.21.15.121:0/3923698680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4dd80bb510 msgr2=0x7f4dd8063570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:39.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.222+0000 7f4dd7fff700 1 --2- 172.21.15.121:0/3923698680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4dd80bb510 0x7f4dd8063570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:39.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.222+0000 7f4dd7fff700 1 -- 172.21.15.121:0/3923698680 >> v1:172.21.15.190:6789/0 conn(0x7f4dd80a8880 legacy=0x7f4dd8151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:39.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.222+0000 7f4dd7fff700 1 -- 172.21.15.121:0/3923698680 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd8065a90 con 0x7f4dd80abaa0 2023-11-15T03:05:39.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.222+0000 7f4dd7fff700 1 --2- 172.21.15.121:0/3923698680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dd80abaa0 0x7f4dd80abe80 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f4dc8006fd0 tx=0x7f4dc80090f0).ready entity=mon.0 client_cookie=15364aba69ef60d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:39.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.222+0000 7f4dd77fe700 1 -- 172.21.15.121:0/3923698680 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc800b600 con 0x7f4dd80abaa0 2023-11-15T03:05:39.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.222+0000 7f4dd77fe700 1 -- 172.21.15.121:0/3923698680 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc800b760 con 0x7f4dd80abaa0 2023-11-15T03:05:39.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.222+0000 7f4dd77fe700 1 -- 172.21.15.121:0/3923698680 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc800bb00 con 0x7f4dd80abaa0 2023-11-15T03:05:39.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.222+0000 7f4ddd7f9700 1 -- 172.21.15.121:0/3923698680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dd80abaa0 msgr2=0x7f4dd80abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:39.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.222+0000 7f4ddd7f9700 1 --2- 172.21.15.121:0/3923698680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dd80abaa0 0x7f4dd80abe80 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f4dc8006fd0 tx=0x7f4dc80090f0).stop 2023-11-15T03:05:39.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.222+0000 7f4ddd7f9700 1 -- 172.21.15.121:0/3923698680 shutdown_connections 2023-11-15T03:05:39.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.222+0000 7f4ddd7f9700 1 --2- 172.21.15.121:0/3923698680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4dd80bb510 0x7f4dd8063570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:39.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.222+0000 7f4ddd7f9700 1 --2- 172.21.15.121:0/3923698680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dd80abaa0 0x7f4dd80abe80 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:39.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.222+0000 7f4ddd7f9700 1 -- 172.21.15.121:0/3923698680 >> 172.21.15.121:0/3923698680 conn(0x7f4dd8067a60 msgr2=0x7f4dd8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:39.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.222+0000 7f4ddd7f9700 1 -- 172.21.15.121:0/3923698680 shutdown_connections 2023-11-15T03:05:39.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.222+0000 7f4ddd7f9700 1 -- 172.21.15.121:0/3923698680 wait complete. 2023-11-15T03:05:39.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.226+0000 7f4ddd7f9700 1 Processor -- start 2023-11-15T03:05:39.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.226+0000 7f4ddd7f9700 1 -- start start 2023-11-15T03:05:39.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.226+0000 7f4ddd7f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dd80a8880 0x7f4dd80c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:39.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.226+0000 7f4ddd7f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4dd80abaa0 0x7f4dd80c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:39.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.226+0000 7f4ddd7f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dd80bb510 0x7f4dd8161f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:39.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.226+0000 7f4ddd7f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd8160940 con 0x7f4dd80a8880 2023-11-15T03:05:39.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.226+0000 7f4ddd7f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd80c6970 con 0x7f4dd80bb510 2023-11-15T03:05:39.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.226+0000 7f4ddd7f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd815fd30 con 0x7f4dd80abaa0 2023-11-15T03:05:39.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.230+0000 7f4dd7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dd80a8880 0x7f4dd80c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:39.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.230+0000 7f4dd7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dd80a8880 0x7f4dd80c5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49250/0 (socket says 172.21.15.121:49250) 2023-11-15T03:05:39.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.230+0000 7f4dd7fff700 1 -- 172.21.15.121:0/3740000302 learned_addr learned my addr 172.21.15.121:0/3740000302 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:39.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.230+0000 7f4dcffff700 1 --2- 172.21.15.121:0/3740000302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4dd80abaa0 0x7f4dd80c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:39.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.230+0000 7f4ddcff8700 1 --2- 172.21.15.121:0/3740000302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dd80bb510 0x7f4dd8161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:39.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.230+0000 7f4dd7fff700 1 -- 172.21.15.121:0/3740000302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4dd80abaa0 msgr2=0x7f4dd80c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:39.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.230+0000 7f4dd7fff700 1 --2- 172.21.15.121:0/3740000302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4dd80abaa0 0x7f4dd80c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:39.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.230+0000 7f4dd7fff700 1 -- 172.21.15.121:0/3740000302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dd80bb510 msgr2=0x7f4dd8161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:39.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.230+0000 7f4dd7fff700 1 --2- 172.21.15.121:0/3740000302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dd80bb510 0x7f4dd8161f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:39.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.230+0000 7f4dd7fff700 1 -- 172.21.15.121:0/3740000302 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc8009370 con 0x7f4dd80a8880 2023-11-15T03:05:39.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.230+0000 7f4ddcff8700 1 --2- 172.21.15.121:0/3740000302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dd80bb510 0x7f4dd8161f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:39.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.230+0000 7f4dcffff700 1 --2- 172.21.15.121:0/3740000302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4dd80abaa0 0x7f4dd80c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:39.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.230+0000 7f4dd7fff700 1 --2- 172.21.15.121:0/3740000302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dd80a8880 0x7f4dd80c5f90 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7f4dd8064eb0 tx=0x7f4dc8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:39.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.230+0000 7f4dd4ff9700 1 -- 172.21.15.121:0/3740000302 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc800bec0 con 0x7f4dd80a8880 2023-11-15T03:05:39.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.230+0000 7f4dd4ff9700 1 -- 172.21.15.121:0/3740000302 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc800b8d0 con 0x7f4dd80a8880 2023-11-15T03:05:39.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.230+0000 7f4dd4ff9700 1 -- 172.21.15.121:0/3740000302 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8024470 con 0x7f4dd80a8880 2023-11-15T03:05:39.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.230+0000 7f4ddd7f9700 1 -- 172.21.15.121:0/3740000302 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd8160010 con 0x7f4dd80a8880 2023-11-15T03:05:39.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.230+0000 7f4ddd7f9700 1 -- 172.21.15.121:0/3740000302 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd81605d0 con 0x7f4dd80a8880 2023-11-15T03:05:39.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.230+0000 7f4ddd7f9700 1 -- 172.21.15.121:0/3740000302 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd806bcf0 con 0x7f4dd80a8880 2023-11-15T03:05:39.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.234+0000 7f4dd4ff9700 1 -- 172.21.15.121:0/3740000302 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4dc8021070 con 0x7f4dd80a8880 2023-11-15T03:05:39.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.234+0000 7f4dd4ff9700 1 --2- 172.21.15.121:0/3740000302 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4db805a510 0x7f4db805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:39.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.234+0000 7f4dd4ff9700 1 -- 172.21.15.121:0/3740000302 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7298+0+0 (secure 0 0 0) 0x7f4dc80766b0 con 0x7f4dd80a8880 2023-11-15T03:05:39.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.234+0000 7f4dcffff700 1 --2- 172.21.15.121:0/3740000302 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4db805a510 0x7f4db805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:39.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.234+0000 7f4dcffff700 1 --2- 172.21.15.121:0/3740000302 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4db805a510 0x7f4db805c9d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f4dc0006920 tx=0x7f4dc0008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:39.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.242+0000 7f4dd4ff9700 1 -- 172.21.15.121:0/3740000302 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4dc80499f0 con 0x7f4dd80a8880 2023-11-15T03:05:39.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:39.438+0000 7f4ddd7f9700 1 -- 172.21.15.121:0/3740000302 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4dd80c2890 con 0x7f4dd80a8880 2023-11-15T03:05:40.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.090+0000 7f4dd4ff9700 1 -- 172.21.15.121:0/3740000302 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v157) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4dc804d010 con 0x7f4dd80a8880 2023-11-15T03:05:40.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.098+0000 7f4ddd7f9700 1 -- 172.21.15.121:0/3740000302 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4db805a510 msgr2=0x7f4db805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:40.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.098+0000 7f4ddd7f9700 1 --2- 172.21.15.121:0/3740000302 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4db805a510 0x7f4db805c9d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f4dc0006920 tx=0x7f4dc0008040).stop 2023-11-15T03:05:40.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.098+0000 7f4ddd7f9700 1 -- 172.21.15.121:0/3740000302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dd80a8880 msgr2=0x7f4dd80c5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:40.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.098+0000 7f4ddd7f9700 1 --2- 172.21.15.121:0/3740000302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dd80a8880 0x7f4dd80c5f90 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7f4dd8064eb0 tx=0x7f4dc8017000).stop 2023-11-15T03:05:40.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.098+0000 7f4ddd7f9700 1 -- 172.21.15.121:0/3740000302 shutdown_connections 2023-11-15T03:05:40.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.098+0000 7f4ddd7f9700 1 --2- 172.21.15.121:0/3740000302 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4db805a510 0x7f4db805c9d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:40.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.098+0000 7f4ddd7f9700 1 --2- 172.21.15.121:0/3740000302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dd80bb510 0x7f4dd8161f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:40.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.098+0000 7f4ddd7f9700 1 --2- 172.21.15.121:0/3740000302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4dd80abaa0 0x7f4dd80c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:40.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.098+0000 7f4ddd7f9700 1 --2- 172.21.15.121:0/3740000302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dd80a8880 0x7f4dd80c5f90 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:40.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.098+0000 7f4ddd7f9700 1 -- 172.21.15.121:0/3740000302 >> 172.21.15.121:0/3740000302 conn(0x7f4dd8067a60 msgr2=0x7f4dd814e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:40.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.098+0000 7f4ddd7f9700 1 -- 172.21.15.121:0/3740000302 shutdown_connections 2023-11-15T03:05:40.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.098+0000 7f4ddd7f9700 1 -- 172.21.15.121:0/3740000302 wait complete. 2023-11-15T03:05:40.108 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:05:40.129 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:05:40.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.294+0000 7f9fa0f07700 1 Processor -- start 2023-11-15T03:05:40.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.294+0000 7f9fa0f07700 1 -- start start 2023-11-15T03:05:40.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9fa0f07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f9c153990 0x7f9f9c153d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:40.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9fa0f07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9f9c0bb910 0x7f9f9c1581e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:40.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9fa0f07700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9f9c0c2ec0 0x7f9f9c0c3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:40.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9fa0f07700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f9c0a8b20 con 0x7f9f9c0c2ec0 2023-11-15T03:05:40.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9fa0f07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f9c0c1760 con 0x7f9f9c0bb910 2023-11-15T03:05:40.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9fa0f07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c0a8d50 con 0x7f9f9c153990 2023-11-15T03:05:40.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9f9bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9f9c0c2ec0 0x7f9f9c0c3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:40.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9f9bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9f9c0c2ec0 0x7f9f9c0c3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49258/0 (socket says 172.21.15.121:49258) 2023-11-15T03:05:40.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9f9bfff700 1 -- 172.21.15.121:0/2228714802 learned_addr learned my addr 172.21.15.121:0/2228714802 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:40.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9f9b7fe700 1 --2- 172.21.15.121:0/2228714802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f9c153990 0x7f9f9c153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:40.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9f9bfff700 1 -- 172.21.15.121:0/2228714802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f9c153990 msgr2=0x7f9f9c153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:40.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9f9bfff700 1 --2- 172.21.15.121:0/2228714802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f9c153990 0x7f9f9c153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:40.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9f9bfff700 1 -- 172.21.15.121:0/2228714802 >> v2:172.21.15.190:3300/0 conn(0x7f9f9c0bb910 msgr2=0x7f9f9c1581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:40.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9f9affd700 1 --2- 172.21.15.121:0/2228714802 >> v2:172.21.15.190:3300/0 conn(0x7f9f9c0bb910 0x7f9f9c1581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:40.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9f9bfff700 1 --2- 172.21.15.121:0/2228714802 >> v2:172.21.15.190:3300/0 conn(0x7f9f9c0bb910 0x7f9f9c1581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:40.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9f9bfff700 1 -- 172.21.15.121:0/2228714802 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9c0a9030 con 0x7f9f9c0c2ec0 2023-11-15T03:05:40.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9f9affd700 1 --2- 172.21.15.121:0/2228714802 >> v2:172.21.15.190:3300/0 conn(0x7f9f9c0bb910 0x7f9f9c1581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:05:40.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9f9bfff700 1 --2- 172.21.15.121:0/2228714802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f9c0c2ec0 0x7f9f9c0c3330 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7f9f9c063c90 tx=0x7f9f94009840).ready entity=mon.0 client_cookie=ac663994a5066074 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:40.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9f9a7fc700 1 -- 172.21.15.121:0/2228714802 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400e070 con 0x7f9f9c0c2ec0 2023-11-15T03:05:40.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9f9a7fc700 1 -- 172.21.15.121:0/2228714802 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9400ba20 con 0x7f9f9c0c2ec0 2023-11-15T03:05:40.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.298+0000 7f9f9a7fc700 1 -- 172.21.15.121:0/2228714802 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400bda0 con 0x7f9f9c0c2ec0 2023-11-15T03:05:40.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.302+0000 7f9fa0f07700 1 -- 172.21.15.121:0/2228714802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f9c0c2ec0 msgr2=0x7f9f9c0c3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:40.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.302+0000 7f9fa0f07700 1 --2- 172.21.15.121:0/2228714802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f9c0c2ec0 0x7f9f9c0c3330 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7f9f9c063c90 tx=0x7f9f94009840).stop 2023-11-15T03:05:40.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.302+0000 7f9fa0f07700 1 -- 172.21.15.121:0/2228714802 shutdown_connections 2023-11-15T03:05:40.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.302+0000 7f9fa0f07700 1 --2- 172.21.15.121:0/2228714802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f9c0c2ec0 0x7f9f9c0c3330 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:40.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.302+0000 7f9fa0f07700 1 --2- 172.21.15.121:0/2228714802 >> v2:172.21.15.190:3300/0 conn(0x7f9f9c0bb910 0x7f9f9c1581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:40.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.302+0000 7f9fa0f07700 1 --2- 172.21.15.121:0/2228714802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f9c153990 0x7f9f9c153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:40.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.302+0000 7f9fa0f07700 1 -- 172.21.15.121:0/2228714802 >> 172.21.15.121:0/2228714802 conn(0x7f9f9c067a60 msgr2=0x7f9f9c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:40.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.302+0000 7f9fa0f07700 1 -- 172.21.15.121:0/2228714802 shutdown_connections 2023-11-15T03:05:40.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.302+0000 7f9fa0f07700 1 -- 172.21.15.121:0/2228714802 wait complete. 2023-11-15T03:05:40.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.302+0000 7f9fa0f07700 1 Processor -- start 2023-11-15T03:05:40.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.302+0000 7f9fa0f07700 1 -- start start 2023-11-15T03:05:40.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9fa0f07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f9c0bb910 0x7f9f9c1642d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:40.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9fa0f07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f9c0c2ec0 0x7f9f9c164cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:40.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9fa0f07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f9c153990 0x7f9f9c156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:40.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9fa0f07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c0be400 con 0x7f9f9c0c2ec0 2023-11-15T03:05:40.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9fa0f07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c0aaa90 con 0x7f9f9c0bb910 2023-11-15T03:05:40.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9fa0f07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c0aa740 con 0x7f9f9c153990 2023-11-15T03:05:40.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9f9affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f9c0c2ec0 0x7f9f9c164cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:40.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9f9affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f9c0c2ec0 0x7f9f9c164cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49264/0 (socket says 172.21.15.121:49264) 2023-11-15T03:05:40.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9f9affd700 1 -- 172.21.15.121:0/1673391672 learned_addr learned my addr 172.21.15.121:0/1673391672 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:40.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9f9b7fe700 1 --2- 172.21.15.121:0/1673391672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f9c0bb910 0x7f9f9c1642d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:40.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9f9bfff700 1 --2- 172.21.15.121:0/1673391672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f9c153990 0x7f9f9c156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:40.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9f9affd700 1 -- 172.21.15.121:0/1673391672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f9c153990 msgr2=0x7f9f9c156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:40.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9f9affd700 1 --2- 172.21.15.121:0/1673391672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f9c153990 0x7f9f9c156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:40.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9f9affd700 1 -- 172.21.15.121:0/1673391672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f9c0bb910 msgr2=0x7f9f9c1642d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:40.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9f9affd700 1 --2- 172.21.15.121:0/1673391672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f9c0bb910 0x7f9f9c1642d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:40.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9f9affd700 1 -- 172.21.15.121:0/1673391672 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f94009ae0 con 0x7f9f9c0c2ec0 2023-11-15T03:05:40.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9f9affd700 1 --2- 172.21.15.121:0/1673391672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f9c0c2ec0 0x7f9f9c164cd0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f9f9000c880 tx=0x7f9f90008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:40.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9f837fe700 1 -- 172.21.15.121:0/1673391672 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90010580 con 0x7f9f9c0c2ec0 2023-11-15T03:05:40.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9f837fe700 1 -- 172.21.15.121:0/1673391672 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f900106e0 con 0x7f9f9c0c2ec0 2023-11-15T03:05:40.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9fa0f07700 1 -- 172.21.15.121:0/1673391672 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9c1569e0 con 0x7f9f9c0c2ec0 2023-11-15T03:05:40.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9f837fe700 1 -- 172.21.15.121:0/1673391672 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90010a60 con 0x7f9f9c0c2ec0 2023-11-15T03:05:40.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.306+0000 7f9fa0f07700 1 -- 172.21.15.121:0/1673391672 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9c156fa0 con 0x7f9f9c0c2ec0 2023-11-15T03:05:40.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.310+0000 7f9fa0f07700 1 -- 172.21.15.121:0/1673391672 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9c1535a0 con 0x7f9f9c0c2ec0 2023-11-15T03:05:40.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.310+0000 7f9f837fe700 1 -- 172.21.15.121:0/1673391672 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9f9001b460 con 0x7f9f9c0c2ec0 2023-11-15T03:05:40.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.314+0000 7f9f837fe700 1 --2- 172.21.15.121:0/1673391672 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9f8805a510 0x7f9f8805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:40.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.314+0000 7f9f837fe700 1 -- 172.21.15.121:0/1673391672 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7298+0+0 (secure 0 0 0) 0x7f9f90074c90 con 0x7f9f9c0c2ec0 2023-11-15T03:05:40.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.314+0000 7f9f9b7fe700 1 --2- 172.21.15.121:0/1673391672 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9f8805a510 0x7f9f8805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:40.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.314+0000 7f9f9b7fe700 1 --2- 172.21.15.121:0/1673391672 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9f8805a510 0x7f9f8805c9d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f9f8c009b90 tx=0x7f9f8c009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:40.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.318+0000 7f9f837fe700 1 -- 172.21.15.121:0/1673391672 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9f90047eb0 con 0x7f9f9c0c2ec0 2023-11-15T03:05:40.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:40.534+0000 7f9fa0f07700 1 -- 172.21.15.121:0/1673391672 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f9f9c06bcf0 con 0x7f9f9c0c2ec0 2023-11-15T03:05:41.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.098+0000 7f9f837fe700 1 -- 172.21.15.121:0/1673391672 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v158) v1 ==== 101+0+0 (secure 0 0 0) 0x7f9f9004b4d0 con 0x7f9f9c0c2ec0 2023-11-15T03:05:41.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.102+0000 7f9fa0f07700 1 -- 172.21.15.121:0/1673391672 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9f8805a510 msgr2=0x7f9f8805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.102+0000 7f9fa0f07700 1 --2- 172.21.15.121:0/1673391672 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9f8805a510 0x7f9f8805c9d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f9f8c009b90 tx=0x7f9f8c009060).stop 2023-11-15T03:05:41.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.102+0000 7f9fa0f07700 1 -- 172.21.15.121:0/1673391672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f9c0c2ec0 msgr2=0x7f9f9c164cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.102+0000 7f9fa0f07700 1 --2- 172.21.15.121:0/1673391672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f9c0c2ec0 0x7f9f9c164cd0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f9f9000c880 tx=0x7f9f90008040).stop 2023-11-15T03:05:41.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.102+0000 7f9fa0f07700 1 -- 172.21.15.121:0/1673391672 shutdown_connections 2023-11-15T03:05:41.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.102+0000 7f9fa0f07700 1 --2- 172.21.15.121:0/1673391672 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9f8805a510 0x7f9f8805c9d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.102+0000 7f9fa0f07700 1 --2- 172.21.15.121:0/1673391672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f9c153990 0x7f9f9c156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.102+0000 7f9fa0f07700 1 --2- 172.21.15.121:0/1673391672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f9c0c2ec0 0x7f9f9c164cd0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.102+0000 7f9fa0f07700 1 --2- 172.21.15.121:0/1673391672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f9c0bb910 0x7f9f9c1642d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.102+0000 7f9fa0f07700 1 -- 172.21.15.121:0/1673391672 >> 172.21.15.121:0/1673391672 conn(0x7f9f9c067a60 msgr2=0x7f9f9c0c42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:41.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.102+0000 7f9fa0f07700 1 -- 172.21.15.121:0/1673391672 shutdown_connections 2023-11-15T03:05:41.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.102+0000 7f9fa0f07700 1 -- 172.21.15.121:0/1673391672 wait complete. 2023-11-15T03:05:41.115 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:05:41.137 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:05:41.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.186+0000 7f5449730d00 1 Processor -- start 2023-11-15T03:05:41.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.186+0000 7f5449730d00 1 -- start start 2023-11-15T03:05:41.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f5449730d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56141a6f7dd0 0x56141a743fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f5449730d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56141a741b00 0x56141a7444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f5449730d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56141a7d4650 0x56141a7c9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f5449730d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141a6a5cc0 con 0x56141a741b00 2023-11-15T03:05:41.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f5449730d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141a670220 con 0x56141a7d4650 2023-11-15T03:05:41.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f5449730d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56141a7449c0 con 0x56141a6f7dd0 2023-11-15T03:05:41.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f543557f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56141a741b00 0x56141a7444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f543557f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56141a741b00 0x56141a7444d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49282/0 (socket says 172.21.15.121:49282) 2023-11-15T03:05:41.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f543557f700 1 -- 172.21.15.121:0/101598884 learned_addr learned my addr 172.21.15.121:0/101598884 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:41.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f543dd80700 1 --2- 172.21.15.121:0/101598884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56141a6f7dd0 0x56141a743fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f543e581700 1 --2- 172.21.15.121:0/101598884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56141a7d4650 0x56141a7c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f543557f700 1 -- 172.21.15.121:0/101598884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56141a6f7dd0 msgr2=0x56141a743fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:41.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f543557f700 1 --2- 172.21.15.121:0/101598884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56141a6f7dd0 0x56141a743fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f543557f700 1 -- 172.21.15.121:0/101598884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56141a7d4650 msgr2=0x56141a7c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:41.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f543557f700 1 --2- 172.21.15.121:0/101598884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56141a7d4650 0x56141a7c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f543557f700 1 -- 172.21.15.121:0/101598884 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543800a0e0 con 0x56141a741b00 2023-11-15T03:05:41.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f543e581700 1 --2- 172.21.15.121:0/101598884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56141a7d4650 0x56141a7c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:41.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f543557f700 1 --2- 172.21.15.121:0/101598884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56141a741b00 0x56141a7444d0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7f5428006e50 tx=0x7f5428006900).ready entity=mon.0 client_cookie=576727d70d35e966 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:41.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f543cd7e700 1 -- 172.21.15.121:0/101598884 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542800bb90 con 0x56141a741b00 2023-11-15T03:05:41.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f543cd7e700 1 -- 172.21.15.121:0/101598884 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542800bcf0 con 0x56141a741b00 2023-11-15T03:05:41.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.190+0000 7f543cd7e700 1 -- 172.21.15.121:0/101598884 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542800e680 con 0x56141a741b00 2023-11-15T03:05:41.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f5449730d00 1 -- 172.21.15.121:0/101598884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56141a741b00 msgr2=0x56141a7444d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f5449730d00 1 --2- 172.21.15.121:0/101598884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56141a741b00 0x56141a7444d0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7f5428006e50 tx=0x7f5428006900).stop 2023-11-15T03:05:41.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f5449730d00 1 -- 172.21.15.121:0/101598884 shutdown_connections 2023-11-15T03:05:41.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f5449730d00 1 --2- 172.21.15.121:0/101598884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56141a7d4650 0x56141a7c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f5449730d00 1 --2- 172.21.15.121:0/101598884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56141a741b00 0x56141a7444d0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f5449730d00 1 --2- 172.21.15.121:0/101598884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56141a6f7dd0 0x56141a743fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f5449730d00 1 -- 172.21.15.121:0/101598884 >> 172.21.15.121:0/101598884 conn(0x56141a747080 msgr2=0x56141a7ca510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:41.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f5449730d00 1 -- 172.21.15.121:0/101598884 shutdown_connections 2023-11-15T03:05:41.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f5449730d00 1 -- 172.21.15.121:0/101598884 wait complete. 2023-11-15T03:05:41.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f5449730d00 1 Processor -- start 2023-11-15T03:05:41.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f5449730d00 1 -- start start 2023-11-15T03:05:41.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f5449730d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56141a6f7dd0 0x56141a7e0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f5449730d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56141a7d4650 0x56141a728bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f5449730d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56141a7290b0 0x56141a7e4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f5449730d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141a7298a0 con 0x56141a7d4650 2023-11-15T03:05:41.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f5449730d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141a7401b0 con 0x56141a7290b0 2023-11-15T03:05:41.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f5449730d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56141a72bca0 con 0x56141a6f7dd0 2023-11-15T03:05:41.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f543557f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56141a7d4650 0x56141a728bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f543557f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56141a7d4650 0x56141a728bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49290/0 (socket says 172.21.15.121:49290) 2023-11-15T03:05:41.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 learned_addr learned my addr 172.21.15.121:0/1458201327 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:41.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f543dd80700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56141a6f7dd0 0x56141a7e0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.194+0000 7f543e581700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56141a7290b0 0x56141a7e4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.198+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56141a6f7dd0 msgr2=0x56141a7e0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.198+0000 7f543557f700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56141a6f7dd0 0x56141a7e0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.198+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56141a7290b0 msgr2=0x56141a7e4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.198+0000 7f543557f700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56141a7290b0 0x56141a7e4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.198+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56141a69e7f0 con 0x56141a7d4650 2023-11-15T03:05:41.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.198+0000 7f543e581700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56141a7290b0 0x56141a7e4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:41.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.198+0000 7f543557f700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56141a7d4650 0x56141a728bc0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7f542800f000 tx=0x7f5428006900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:41.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.198+0000 7f5436ffd700 1 -- 172.21.15.121:0/1458201327 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542800bee0 con 0x56141a7d4650 2023-11-15T03:05:41.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.198+0000 7f5436ffd700 1 -- 172.21.15.121:0/1458201327 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542801a630 con 0x56141a7d4650 2023-11-15T03:05:41.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.198+0000 7f5436ffd700 1 -- 172.21.15.121:0/1458201327 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542801a9f0 con 0x56141a7d4650 2023-11-15T03:05:41.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.198+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56141a668a70 con 0x56141a7d4650 2023-11-15T03:05:41.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.198+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56141a674010 con 0x56141a7d4650 2023-11-15T03:05:41.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.198+0000 7f5436ffd700 1 -- 172.21.15.121:0/1458201327 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5428018430 con 0x56141a7d4650 2023-11-15T03:05:41.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.202+0000 7f5436ffd700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5420059070 0x7f542005b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.202+0000 7f5436ffd700 1 -- 172.21.15.121:0/1458201327 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7298+0+0 (secure 0 0 0) 0x7f5428075380 con 0x56141a7d4650 2023-11-15T03:05:41.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.202+0000 7f543dd80700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5420059070 0x7f542005b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.217 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:05:41.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.202+0000 7f5449730d00 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56141a7e4b00 0x56141a7e4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.202+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7c87f0 con 0x56141a7e4b00 2023-11-15T03:05:41.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.202+0000 7f543e581700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56141a7e4b00 0x56141a7e4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.202+0000 7f543dd80700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5420059070 0x7f542005b530 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f543000f770 tx=0x7f5430000bc0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:41.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.202+0000 7f543e581700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56141a7e4b00 0x56141a7e4ee0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:41.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.206+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v106'137 uv130 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7e4b00 2023-11-15T03:05:41.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.206+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a740690 con 0x56141a7e4b00 2023-11-15T03:05:41.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.206+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v106'138 uv138 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7e4b00 2023-11-15T03:05:41.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.206+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7ed8f0 con 0x56141a7e4b00 2023-11-15T03:05:41.221 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:05:41.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.210+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v106'139 uv139 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7e4b00 2023-11-15T03:05:41.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.210+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7eecb0 con 0x56141a7e4b00 2023-11-15T03:05:41.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.210+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v106'140 uv140 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7e4b00 2023-11-15T03:05:41.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.210+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7ef060 con 0x56141a7e4b00 2023-11-15T03:05:41.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.214+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v106'141 uv141 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7e4b00 2023-11-15T03:05:41.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.214+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7edca0 con 0x56141a7e4b00 2023-11-15T03:05:41.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.218+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v106'142 uv142 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7e4b00 2023-11-15T03:05:41.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.218+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7ee0a0 con 0x56141a7e4b00 2023-11-15T03:05:41.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.218+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v106'143 uv136 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7e4b00 2023-11-15T03:05:41.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.218+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a7ee4a0 con 0x56141a7e4b00 2023-11-15T03:05:41.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.222+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v106'144 uv144 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7e4b00 2023-11-15T03:05:41.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.222+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7ee4a0 con 0x56141a7e4b00 2023-11-15T03:05:41.232 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:05:41.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.222+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v106'145 uv145 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7e4b00 2023-11-15T03:05:41.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.222+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7ee450 con 0x56141a7e4b00 2023-11-15T03:05:41.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.226+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v106'146 uv146 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7e4b00 2023-11-15T03:05:41.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.226+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7ee450 con 0x56141a7e4b00 2023-11-15T03:05:41.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.226+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v106'147 uv147 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7e4b00 2023-11-15T03:05:41.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.226+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7ee450 con 0x56141a7e4b00 2023-11-15T03:05:41.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.230+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v106'148 uv148 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7e4b00 2023-11-15T03:05:41.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.230+0000 7f5449730d00 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56141a7ee850 0x56141a7f1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.230+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7ee450 con 0x56141a7ee850 2023-11-15T03:05:41.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.230+0000 7f543dd80700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56141a7ee850 0x56141a7f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.230+0000 7f543dd80700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56141a7ee850 0x56141a7f1930 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:41.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.234+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v106'214 uv201 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.234+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a7f1e20 con 0x56141a7ee850 2023-11-15T03:05:41.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.238+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v106'215 uv215 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.238+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f21d0 con 0x56141a7ee850 2023-11-15T03:05:41.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.242+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v106'216 uv216 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.250 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:05:41.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.242+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f3590 con 0x56141a7ee850 2023-11-15T03:05:41.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.242+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v106'217 uv217 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.242+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f3940 con 0x56141a7ee850 2023-11-15T03:05:41.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.246+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v106'218 uv218 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.246+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f2580 con 0x56141a7ee850 2023-11-15T03:05:41.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.250+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v106'219 uv219 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.250+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f2e70 con 0x56141a7ee850 2023-11-15T03:05:41.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.250+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v106'220 uv220 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.250+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f3cf0 con 0x56141a7ee850 2023-11-15T03:05:41.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.254+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v106'221 uv207 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.254+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a7f3cf0 con 0x56141a7ee850 2023-11-15T03:05:41.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.258+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v106'222 uv222 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.258+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f3cf0 con 0x56141a7ee850 2023-11-15T03:05:41.269 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:05:41.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.258+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v106'223 uv223 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.262+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f4d00 con 0x56141a7ee850 2023-11-15T03:05:41.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.262+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v106'224 uv224 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.262+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f4d00 con 0x56141a7ee850 2023-11-15T03:05:41.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.266+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v106'225 uv225 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.266+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f3cf0 con 0x56141a7ee850 2023-11-15T03:05:41.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.270+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v106'226 uv226 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.270+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f3cf0 con 0x56141a7ee850 2023-11-15T03:05:41.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.274+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v106'227 uv213 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.274+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a7f3cf0 con 0x56141a7ee850 2023-11-15T03:05:41.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.278+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v106'228 uv228 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.278+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f3cf0 con 0x56141a7ee850 2023-11-15T03:05:41.288 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:05:41.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.278+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v106'229 uv229 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.278+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f4d00 con 0x56141a7ee850 2023-11-15T03:05:41.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.282+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v106'230 uv230 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.282+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f4d00 con 0x56141a7ee850 2023-11-15T03:05:41.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.286+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v106'231 uv231 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.286+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f3cf0 con 0x56141a7ee850 2023-11-15T03:05:41.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.286+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v106'232 uv232 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.286+0000 7f5449730d00 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x56141a7f4200 0x56141a7f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.286+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f6ae0 con 0x56141a7f4200 2023-11-15T03:05:41.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.286+0000 7f543557f700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x56141a7f4200 0x56141a7f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.290+0000 7f543557f700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x56141a7f4200 0x56141a7f65f0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:41.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.290+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v106'117 uv111 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7f4200 2023-11-15T03:05:41.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.290+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a7f6e90 con 0x56141a7f4200 2023-11-15T03:05:41.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.294+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v106'118 uv118 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7f4200 2023-11-15T03:05:41.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.294+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f7240 con 0x56141a7f4200 2023-11-15T03:05:41.305 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:05:41.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.298+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v106'119 uv119 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7f4200 2023-11-15T03:05:41.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.298+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f7ae0 con 0x56141a7f4200 2023-11-15T03:05:41.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.298+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v106'120 uv120 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7f4200 2023-11-15T03:05:41.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.298+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f7e90 con 0x56141a7f4200 2023-11-15T03:05:41.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.302+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v106'121 uv121 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7f4200 2023-11-15T03:05:41.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.302+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f8240 con 0x56141a7f4200 2023-11-15T03:05:41.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.302+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v106'122 uv116 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7f4200 2023-11-15T03:05:41.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.302+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a7f85f0 con 0x56141a7f4200 2023-11-15T03:05:41.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.306+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v106'123 uv123 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7f4200 2023-11-15T03:05:41.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.306+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f89a0 con 0x56141a7f4200 2023-11-15T03:05:41.318 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:05:41.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.310+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v106'124 uv124 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7f4200 2023-11-15T03:05:41.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.310+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f89a0 con 0x56141a7f4200 2023-11-15T03:05:41.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.310+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v106'125 uv125 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7f4200 2023-11-15T03:05:41.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.310+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f89a0 con 0x56141a7f4200 2023-11-15T03:05:41.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.314+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v106'126 uv126 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7f4200 2023-11-15T03:05:41.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.314+0000 7f5449730d00 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56141a7f8eb0 0x56141a7fb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.314+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7fb790 con 0x56141a7f8eb0 2023-11-15T03:05:41.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.314+0000 7f543e581700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56141a7f8eb0 0x56141a7fb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.318+0000 7f543e581700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56141a7f8eb0 0x56141a7fb2a0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:41.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.318+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v106'145 uv138 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.318+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a7fb7d0 con 0x56141a7f8eb0 2023-11-15T03:05:41.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.322+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v106'146 uv146 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.322+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7fbc00 con 0x56141a7f8eb0 2023-11-15T03:05:41.336 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:05:41.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.326+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v106'147 uv147 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.326+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7ffc90 con 0x56141a7f8eb0 2023-11-15T03:05:41.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.330+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v106'148 uv148 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.330+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7ffc90 con 0x56141a7f8eb0 2023-11-15T03:05:41.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.334+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v106'149 uv149 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.334+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7fbc00 con 0x56141a7f8eb0 2023-11-15T03:05:41.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.334+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v106'150 uv144 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.334+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a7fbc00 con 0x56141a7f8eb0 2023-11-15T03:05:41.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.338+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v106'151 uv151 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.338+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7fbc00 con 0x56141a7f8eb0 2023-11-15T03:05:41.350 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:05:41.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.342+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v106'152 uv152 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.342+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7fcc10 con 0x56141a7f8eb0 2023-11-15T03:05:41.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.342+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v106'153 uv153 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.342+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7fcc10 con 0x56141a7f8eb0 2023-11-15T03:05:41.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.346+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v106'154 uv154 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.346+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7fbc00 con 0x56141a7f8eb0 2023-11-15T03:05:41.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.350+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v106'155 uv155 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.350+0000 7f5449730d00 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x56141a7fc280 0x56141a7fe670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.350+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7febf0 con 0x56141a7fc280 2023-11-15T03:05:41.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.350+0000 7f543dd80700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x56141a7fc280 0x56141a7fe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.350+0000 7f543dd80700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x56141a7fc280 0x56141a7fe670 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:41.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.354+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v106'69 uv68 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7fc280 2023-11-15T03:05:41.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.354+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a7fed10 con 0x56141a7fc280 2023-11-15T03:05:41.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.354+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v106'70 uv70 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7fc280 2023-11-15T03:05:41.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.354+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7ff140 con 0x56141a7fc280 2023-11-15T03:05:41.367 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:05:41.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.358+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v106'71 uv71 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7fc280 2023-11-15T03:05:41.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.358+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a800150 con 0x56141a7fc280 2023-11-15T03:05:41.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.362+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v106'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7fc280 2023-11-15T03:05:41.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.362+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a800150 con 0x56141a7fc280 2023-11-15T03:05:41.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.362+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v106'73 uv73 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7fc280 2023-11-15T03:05:41.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.362+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7ff140 con 0x56141a7fc280 2023-11-15T03:05:41.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.366+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v106'74 uv74 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7fc280 2023-11-15T03:05:41.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.366+0000 7f5449730d00 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56141a7ff650 0x56141a801a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.366+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a802110 con 0x56141a7ff650 2023-11-15T03:05:41.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.366+0000 7f543557f700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56141a7ff650 0x56141a801a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.366+0000 7f543557f700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56141a7ff650 0x56141a801a40 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:41.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.370+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v106'51 uv50 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.370+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a802270 con 0x56141a7ff650 2023-11-15T03:05:41.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.374+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v106'52 uv52 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.374+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a802270 con 0x56141a7ff650 2023-11-15T03:05:41.386 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:05:41.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.378+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v106'53 uv53 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.378+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ff650 2023-11-15T03:05:41.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.378+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v106'54 uv54 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.378+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ff650 2023-11-15T03:05:41.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.382+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v106'69 uv68 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.382+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ff650 2023-11-15T03:05:41.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.386+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v106'70 uv70 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.386+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ff650 2023-11-15T03:05:41.397 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:05:41.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.386+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v106'71 uv71 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.386+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a803700 con 0x56141a7ff650 2023-11-15T03:05:41.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.390+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v106'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.390+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a803700 con 0x56141a7ff650 2023-11-15T03:05:41.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.394+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v106'73 uv73 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.394+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ff650 2023-11-15T03:05:41.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.394+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v106'74 uv74 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.394+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ee850 2023-11-15T03:05:41.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.398+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v106'127 uv121 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.398+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ee850 2023-11-15T03:05:41.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.402+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v106'128 uv128 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.402+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ee850 2023-11-15T03:05:41.414 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:05:41.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.406+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v106'129 uv129 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.406+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a803700 con 0x56141a7ee850 2023-11-15T03:05:41.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.410+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v106'130 uv130 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.410+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a803700 con 0x56141a7ee850 2023-11-15T03:05:41.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.410+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v106'131 uv131 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.414+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ee850 2023-11-15T03:05:41.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.414+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v106'132 uv132 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.414+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ee850 2023-11-15T03:05:41.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.418+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v106'133 uv126 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.418+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ee850 2023-11-15T03:05:41.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.422+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v106'134 uv134 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.422+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ee850 2023-11-15T03:05:41.433 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:05:41.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.426+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v106'135 uv135 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.426+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ee850 2023-11-15T03:05:41.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.426+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v106'136 uv136 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.430+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ee850 2023-11-15T03:05:41.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.430+0000 7f543dd80700 1 -- 172.21.15.121:0/1458201327 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v106'137 uv137 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f54300113c0 con 0x56141a7ee850 2023-11-15T03:05:41.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.430+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7f4200 2023-11-15T03:05:41.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.434+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v106'56 uv55 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7f4200 2023-11-15T03:05:41.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.434+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7f4200 2023-11-15T03:05:41.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.438+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v106'57 uv57 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7f4200 2023-11-15T03:05:41.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.438+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7f4200 2023-11-15T03:05:41.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.438+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v106'58 uv58 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7f4200 2023-11-15T03:05:41.450 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:05:41.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.438+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7f4200 2023-11-15T03:05:41.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.442+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v106'59 uv59 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7f4200 2023-11-15T03:05:41.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.442+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7f4200 2023-11-15T03:05:41.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.446+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v106'60 uv60 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7f4200 2023-11-15T03:05:41.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.446+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ff650 2023-11-15T03:05:41.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.446+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v106'69 uv68 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.446+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ff650 2023-11-15T03:05:41.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.450+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v106'70 uv70 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.450+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ff650 2023-11-15T03:05:41.462 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:05:41.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.454+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v106'71 uv71 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.454+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a803700 con 0x56141a7ff650 2023-11-15T03:05:41.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.458+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v106'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.458+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a803700 con 0x56141a7ff650 2023-11-15T03:05:41.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.458+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v106'73 uv73 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.458+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ff650 2023-11-15T03:05:41.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.462+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v106'74 uv74 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.462+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ff650 2023-11-15T03:05:41.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.466+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v106'57 uv56 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.466+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ff650 2023-11-15T03:05:41.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.470+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v106'58 uv58 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.470+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ff650 2023-11-15T03:05:41.481 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:05:41.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.470+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v106'59 uv59 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.470+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8026f0 con 0x56141a7ff650 2023-11-15T03:05:41.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.474+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v106'60 uv60 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.474+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f7670 con 0x56141a7ff650 2023-11-15T03:05:41.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.478+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v106'61 uv61 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.478+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f7670 con 0x56141a7ff650 2023-11-15T03:05:41.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.482+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v106'56 uv55 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.482+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a7f75f0 con 0x56141a7ff650 2023-11-15T03:05:41.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.486+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v106'57 uv57 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.486+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f75f0 con 0x56141a7ff650 2023-11-15T03:05:41.496 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:05:41.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.486+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v106'58 uv58 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.486+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a7f75f0 con 0x56141a7ff650 2023-11-15T03:05:41.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.490+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v106'59 uv59 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.490+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8054e0 con 0x56141a7ff650 2023-11-15T03:05:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.494+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v106'60 uv60 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.494+0000 7f5449730d00 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56141a802c00 0x56141a805b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.494+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a804ff0 con 0x56141a802c00 2023-11-15T03:05:41.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.494+0000 7f543e581700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56141a802c00 0x56141a805b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.494+0000 7f543e581700 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56141a802c00 0x56141a805b00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:41.507 INFO:teuthology.orchestra.run.smithi121.stdout:Write #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:05:41.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.498+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v106'48 uv48 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a802c00 2023-11-15T03:05:41.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.498+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a804ff0 con 0x56141a802c00 2023-11-15T03:05:41.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.502+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v106'49 uv49 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a802c00 2023-11-15T03:05:41.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.502+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a804ff0 con 0x56141a7ff650 2023-11-15T03:05:41.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.502+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(103 609.00000000 [create] v106'69 uv68 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.502+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a804ff0 con 0x56141a7ff650 2023-11-15T03:05:41.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.506+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(104 609.00000000 [delete] v106'70 uv70 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.506+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a804ff0 con 0x56141a7ff650 2023-11-15T03:05:41.519 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:05:41.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.510+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(105 609.00000000 [create] v106'71 uv71 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.510+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a804ff0 con 0x56141a7ff650 2023-11-15T03:05:41.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.510+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v106'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.514+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a804ff0 con 0x56141a7ff650 2023-11-15T03:05:41.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.514+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v106'73 uv73 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.514+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a804ff0 con 0x56141a7ff650 2023-11-15T03:05:41.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.518+0000 7f543557f700 1 -- 172.21.15.121:0/1458201327 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(108 609.00000000 [omap-set-header] v106'74 uv74 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5428014020 con 0x56141a7ff650 2023-11-15T03:05:41.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.518+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a804ff0 con 0x56141a7f8eb0 2023-11-15T03:05:41.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.522+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(109 400.00000000 [create] v106'57 uv56 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.522+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a804ff0 con 0x56141a7f8eb0 2023-11-15T03:05:41.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.522+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(110 400.00000000 [delete] v106'58 uv58 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.522+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a804ff0 con 0x56141a7f8eb0 2023-11-15T03:05:41.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.526+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(111 400.00000000 [create] v106'59 uv59 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.535 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:05:41.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.526+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a805ff0 con 0x56141a7f8eb0 2023-11-15T03:05:41.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.530+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(112 400.00000000 [write 0~22] v106'60 uv60 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.530+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a805ff0 con 0x56141a7f8eb0 2023-11-15T03:05:41.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.530+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(113 400.00000000 [omap-set-header] v106'61 uv61 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.530+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a805ff0 con 0x56141a7f8eb0 2023-11-15T03:05:41.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.534+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(114 100.00000000.inode [create] v106'138 uv130 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.534+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a805ff0 con 0x56141a7f8eb0 2023-11-15T03:05:41.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.538+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(115 100.00000000.inode [delete] v106'139 uv139 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.538+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a805ff0 con 0x56141a7f8eb0 2023-11-15T03:05:41.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.538+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(116 100.00000000.inode [create] v106'140 uv140 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.549 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:05:41.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.538+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a805ff0 con 0x56141a7f8eb0 2023-11-15T03:05:41.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.542+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v106'141 uv141 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.542+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a805ff0 con 0x56141a7f8eb0 2023-11-15T03:05:41.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.546+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v106'142 uv142 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.546+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a805ff0 con 0x56141a7f8eb0 2023-11-15T03:05:41.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.550+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(119 1.00000000 [create] v106'143 uv137 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.550+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x56141a805ff0 con 0x56141a7f8eb0 2023-11-15T03:05:41.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.550+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(120 1.00000000 [delete] v106'144 uv144 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.550+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a805ff0 con 0x56141a7f8eb0 2023-11-15T03:05:41.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.554+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(121 1.00000000 [create] v106'145 uv145 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.563 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:05:41.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.554+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a807000 con 0x56141a7f8eb0 2023-11-15T03:05:41.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.558+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v106'146 uv146 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.558+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a807000 con 0x56141a7f8eb0 2023-11-15T03:05:41.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.558+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v106'147 uv147 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.558+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a805ff0 con 0x56141a7f8eb0 2023-11-15T03:05:41.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.562+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v106'148 uv148 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.562+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e106) v8 -- 0x56141a8064e0 con 0x56141a7f8eb0 2023-11-15T03:05:41.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v106'149 uv149 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f543800c8a0 con 0x56141a7f8eb0 2023-11-15T03:05:41.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x56141a7f4200 msgr2=0x56141a7f65f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x56141a7f4200 0x56141a7f65f0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56141a7e4b00 msgr2=0x56141a7e4ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56141a7e4b00 0x56141a7e4ee0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x56141a7fc280 msgr2=0x56141a7fe670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x56141a7fc280 0x56141a7fe670 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56141a7ee850 msgr2=0x56141a7f1930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56141a7ee850 0x56141a7f1930 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56141a802c00 msgr2=0x56141a805b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56141a802c00 0x56141a805b00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56141a7f8eb0 msgr2=0x56141a7fb2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56141a7f8eb0 0x56141a7fb2a0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 reap_dead start 2023-11-15T03:05:41.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56141a7ff650 msgr2=0x56141a801a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 reap_dead start 2023-11-15T03:05:41.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56141a7ff650 0x56141a801a40 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f543e581700 1 -- 172.21.15.121:0/1458201327 reap_dead start 2023-11-15T03:05:41.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5420059070 msgr2=0x7f542005b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5420059070 0x7f542005b530 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f543000f770 tx=0x7f5430000bc0).stop 2023-11-15T03:05:41.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56141a7d4650 msgr2=0x56141a728bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56141a7d4650 0x56141a728bc0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7f542800f000 tx=0x7f5428006900).stop 2023-11-15T03:05:41.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 shutdown_connections 2023-11-15T03:05:41.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5420059070 0x7f542005b530 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 --2- 172.21.15.121:0/1458201327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56141a7d4650 0x56141a728bc0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 >> 172.21.15.121:0/1458201327 conn(0x56141a747080 msgr2=0x56141a743fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:41.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 shutdown_connections 2023-11-15T03:05:41.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.566+0000 7f5449730d00 1 -- 172.21.15.121:0/1458201327 wait complete. 2023-11-15T03:05:41.584 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:05:41.586 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 400.00000000 2023-11-15T03:05:41.635 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:05:41.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.678+0000 7feb9c72fd00 1 Processor -- start 2023-11-15T03:05:41.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.678+0000 7feb9c72fd00 1 -- start start 2023-11-15T03:05:41.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb9c72fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d398fbfbb0 0x55d399091250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb9c72fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d399009b00 0x55d398ff3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb9c72fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d39909c650 0x55d398ff3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb9c72fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d398f6dcc0 con 0x55d399009b00 2023-11-15T03:05:41.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb9c72fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d398f38220 con 0x55d39909c650 2023-11-15T03:05:41.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb9c72fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d39909b7c0 con 0x55d398fbfbb0 2023-11-15T03:05:41.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb885be700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d399009b00 0x55d398ff3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb885be700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d399009b00 0x55d398ff3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49314/0 (socket says 172.21.15.121:49314) 2023-11-15T03:05:41.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb885be700 1 -- 172.21.15.121:0/1225137772 learned_addr learned my addr 172.21.15.121:0/1225137772 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:41.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb90dbf700 1 --2- 172.21.15.121:0/1225137772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d398fbfbb0 0x55d399091250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb915c0700 1 --2- 172.21.15.121:0/1225137772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d39909c650 0x55d398ff3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb885be700 1 -- 172.21.15.121:0/1225137772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d398fbfbb0 msgr2=0x55d399091250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:41.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb885be700 1 --2- 172.21.15.121:0/1225137772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d398fbfbb0 0x55d399091250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb885be700 1 -- 172.21.15.121:0/1225137772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d39909c650 msgr2=0x55d398ff3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:41.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb885be700 1 --2- 172.21.15.121:0/1225137772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d39909c650 0x55d398ff3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb885be700 1 -- 172.21.15.121:0/1225137772 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7800a0e0 con 0x55d399009b00 2023-11-15T03:05:41.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb885be700 1 --2- 172.21.15.121:0/1225137772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d399009b00 0x55d398ff3430 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x55d39900ee50 tx=0x7feb78015000).ready entity=mon.0 client_cookie=ef29e63112eb649f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:41.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb8b7fe700 1 -- 172.21.15.121:0/1225137772 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb78000f40 con 0x55d399009b00 2023-11-15T03:05:41.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb8b7fe700 1 -- 172.21.15.121:0/1225137772 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7800dcc0 con 0x55d399009b00 2023-11-15T03:05:41.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb8b7fe700 1 -- 172.21.15.121:0/1225137772 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb780010a0 con 0x55d399009b00 2023-11-15T03:05:41.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb9c72fd00 1 -- 172.21.15.121:0/1225137772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d399009b00 msgr2=0x55d398ff3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb9c72fd00 1 --2- 172.21.15.121:0/1225137772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d399009b00 0x55d398ff3430 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x55d39900ee50 tx=0x7feb78015000).stop 2023-11-15T03:05:41.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb9c72fd00 1 -- 172.21.15.121:0/1225137772 shutdown_connections 2023-11-15T03:05:41.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb9c72fd00 1 --2- 172.21.15.121:0/1225137772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d39909c650 0x55d398ff3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb9c72fd00 1 --2- 172.21.15.121:0/1225137772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d399009b00 0x55d398ff3430 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb9c72fd00 1 --2- 172.21.15.121:0/1225137772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d398fbfbb0 0x55d399091250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb9c72fd00 1 -- 172.21.15.121:0/1225137772 >> 172.21.15.121:0/1225137772 conn(0x55d39900f080 msgr2=0x55d399003fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:41.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb9c72fd00 1 -- 172.21.15.121:0/1225137772 shutdown_connections 2023-11-15T03:05:41.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.682+0000 7feb9c72fd00 1 -- 172.21.15.121:0/1225137772 wait complete. 2023-11-15T03:05:41.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb9c72fd00 1 Processor -- start 2023-11-15T03:05:41.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb9c72fd00 1 -- start start 2023-11-15T03:05:41.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb9c72fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d398fbfbb0 0x55d39908bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb9c72fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d399009b00 0x55d398ff3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb9c72fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d39909c650 0x55d398ff3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb9c72fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d398ff3bf0 con 0x55d399009b00 2023-11-15T03:05:41.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb9c72fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d399090460 con 0x55d39909c650 2023-11-15T03:05:41.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb9c72fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d39908e7f0 con 0x55d398fbfbb0 2023-11-15T03:05:41.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb885be700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d399009b00 0x55d398ff3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb885be700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d399009b00 0x55d398ff3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49318/0 (socket says 172.21.15.121:49318) 2023-11-15T03:05:41.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb885be700 1 -- 172.21.15.121:0/1409023951 learned_addr learned my addr 172.21.15.121:0/1409023951 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:41.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb915c0700 1 --2- 172.21.15.121:0/1409023951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d39909c650 0x55d398ff3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb90dbf700 1 --2- 172.21.15.121:0/1409023951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d398fbfbb0 0x55d39908bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb885be700 1 -- 172.21.15.121:0/1409023951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d398fbfbb0 msgr2=0x55d39908bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb885be700 1 --2- 172.21.15.121:0/1409023951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d398fbfbb0 0x55d39908bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb885be700 1 -- 172.21.15.121:0/1409023951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d39909c650 msgr2=0x55d398ff3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb885be700 1 --2- 172.21.15.121:0/1409023951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d39909c650 0x55d398ff3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb885be700 1 -- 172.21.15.121:0/1409023951 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d398f61770 con 0x55d399009b00 2023-11-15T03:05:41.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb915c0700 1 --2- 172.21.15.121:0/1409023951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d39909c650 0x55d398ff3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:05:41.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb90dbf700 1 --2- 172.21.15.121:0/1409023951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d398fbfbb0 0x55d39908bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:05:41.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb885be700 1 --2- 172.21.15.121:0/1409023951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d399009b00 0x55d398ff3210 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x55d3990a4e00 tx=0x7feb7801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:41.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb89ffb700 1 -- 172.21.15.121:0/1409023951 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7801a830 con 0x55d399009b00 2023-11-15T03:05:41.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb89ffb700 1 -- 172.21.15.121:0/1409023951 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7801a990 con 0x55d399009b00 2023-11-15T03:05:41.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb9c72fd00 1 -- 172.21.15.121:0/1409023951 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d398f30a70 con 0x55d399009b00 2023-11-15T03:05:41.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.686+0000 7feb89ffb700 1 -- 172.21.15.121:0/1409023951 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7801ad50 con 0x55d399009b00 2023-11-15T03:05:41.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.690+0000 7feb9c72fd00 1 -- 172.21.15.121:0/1409023951 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d398f3c010 con 0x55d399009b00 2023-11-15T03:05:41.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.690+0000 7feb89ffb700 1 -- 172.21.15.121:0/1409023951 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7feb7802a070 con 0x55d399009b00 2023-11-15T03:05:41.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.690+0000 7feb89ffb700 1 --2- 172.21.15.121:0/1409023951 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feb70059020 0x7feb7005b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.690+0000 7feb89ffb700 1 -- 172.21.15.121:0/1409023951 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7298+0+0 (secure 0 0 0) 0x7feb78076660 con 0x55d399009b00 2023-11-15T03:05:41.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.690+0000 7feb90dbf700 1 --2- 172.21.15.121:0/1409023951 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feb70059020 0x7feb7005b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.690+0000 7feb9c72fd00 1 --2- 172.21.15.121:0/1409023951 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55d399092320 0x55d39908dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.690+0000 7feb9c72fd00 1 -- 172.21.15.121:0/1409023951 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106) v8 -- 0x55d39908ec80 con 0x55d399092320 2023-11-15T03:05:41.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.690+0000 7feb90dbf700 1 --2- 172.21.15.121:0/1409023951 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feb70059020 0x7feb7005b4e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7feb80004170 tx=0x7feb80001690).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:41.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.694+0000 7feb915c0700 1 --2- 172.21.15.121:0/1409023951 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55d399092320 0x55d39908dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.694+0000 7feb915c0700 1 --2- 172.21.15.121:0/1409023951 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55d399092320 0x55d39908dda0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:41.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.694+0000 7feb915c0700 1 -- 172.21.15.121:0/1409023951 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 400.00000000 [delete] v106'62 uv62 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feb8c00e400 con 0x55d399092320 2023-11-15T03:05:41.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.694+0000 7feb9c72fd00 1 -- 172.21.15.121:0/1409023951 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55d399092320 msgr2=0x55d39908dda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.694+0000 7feb9c72fd00 1 --2- 172.21.15.121:0/1409023951 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55d399092320 0x55d39908dda0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.694+0000 7feb9c72fd00 1 -- 172.21.15.121:0/1409023951 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feb70059020 msgr2=0x7feb7005b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.694+0000 7feb9c72fd00 1 --2- 172.21.15.121:0/1409023951 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feb70059020 0x7feb7005b4e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7feb80004170 tx=0x7feb80001690).stop 2023-11-15T03:05:41.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.694+0000 7feb9c72fd00 1 -- 172.21.15.121:0/1409023951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d399009b00 msgr2=0x55d398ff3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.694+0000 7feb9c72fd00 1 --2- 172.21.15.121:0/1409023951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d399009b00 0x55d398ff3210 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x55d3990a4e00 tx=0x7feb7801a000).stop 2023-11-15T03:05:41.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.694+0000 7feb915c0700 1 -- 172.21.15.121:0/1409023951 reap_dead start 2023-11-15T03:05:41.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.698+0000 7feb9c72fd00 1 -- 172.21.15.121:0/1409023951 shutdown_connections 2023-11-15T03:05:41.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.698+0000 7feb9c72fd00 1 -- 172.21.15.121:0/1409023951 >> 172.21.15.121:0/1409023951 conn(0x55d39900f080 msgr2=0x55d39900c270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:41.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.698+0000 7feb9c72fd00 1 -- 172.21.15.121:0/1409023951 shutdown_connections 2023-11-15T03:05:41.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.698+0000 7feb9c72fd00 1 -- 172.21.15.121:0/1409023951 wait complete. 2023-11-15T03:05:41.714 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:05:41.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.918+0000 7fb3f6c9b700 1 Processor -- start 2023-11-15T03:05:41.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.918+0000 7fb3f6c9b700 1 -- start start 2023-11-15T03:05:41.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3f6c9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb3f0067ec0 0x7fb3f00682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3f6c9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f00c10b0 0x7fb3f0062c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3f6c9b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f00c0c20 con 0x7fb3f0063120 2023-11-15T03:05:41.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3f6c9b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f0158730 con 0x7fb3f0067ec0 2023-11-15T03:05:41.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3f6c9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f00c02f0 con 0x7fb3f00c10b0 2023-11-15T03:05:41.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3f5238700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb3f0063120 0x7fb3f0153700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:37038/0 (socket says 172.21.15.121:37038) 2023-11-15T03:05:41.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3f5238700 1 -- 172.21.15.121:0/608056351 learned_addr learned my addr 172.21.15.121:0/608056351 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:41.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3f4a37700 1 --2- 172.21.15.121:0/608056351 >> v2:172.21.15.190:3300/0 conn(0x7fb3f0067ec0 0x7fb3f00682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3e7fff700 1 --2- 172.21.15.121:0/608056351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f00c10b0 0x7fb3f0062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3e77fe700 1 -- 172.21.15.121:0/608056351 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713395791 0 0) 0x7fb3f00c0c20 con 0x7fb3f0063120 2023-11-15T03:05:41.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3e77fe700 1 -- 172.21.15.121:0/608056351 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3d80033f0 con 0x7fb3f0063120 2023-11-15T03:05:41.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3f4a37700 1 -- 172.21.15.121:0/608056351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f00c10b0 msgr2=0x7fb3f0062c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:41.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3f4a37700 1 --2- 172.21.15.121:0/608056351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f00c10b0 0x7fb3f0062c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3f4a37700 1 -- 172.21.15.121:0/608056351 >> v1:172.21.15.121:6789/0 conn(0x7fb3f0063120 legacy=0x7fb3f0153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3f4a37700 1 -- 172.21.15.121:0/608056351 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f00c05d0 con 0x7fb3f0067ec0 2023-11-15T03:05:41.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3e7fff700 1 --2- 172.21.15.121:0/608056351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f00c10b0 0x7fb3f0062c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:05:41.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3f4a37700 1 --2- 172.21.15.121:0/608056351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3f0067ec0 0x7fb3f00682a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7fb3ec009e60 tx=0x7fb3ec009780).ready entity=mon.1 client_cookie=4132f965ee5c5150 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:41.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3e77fe700 1 -- 172.21.15.121:0/608056351 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec00e070 con 0x7fb3f0067ec0 2023-11-15T03:05:41.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3e77fe700 1 -- 172.21.15.121:0/608056351 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3ec00bab0 con 0x7fb3f0067ec0 2023-11-15T03:05:41.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.922+0000 7fb3e77fe700 1 -- 172.21.15.121:0/608056351 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec00be30 con 0x7fb3f0067ec0 2023-11-15T03:05:41.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.926+0000 7fb3f6c9b700 1 -- 172.21.15.121:0/608056351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3f0067ec0 msgr2=0x7fb3f00682a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.926+0000 7fb3f6c9b700 1 --2- 172.21.15.121:0/608056351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3f0067ec0 0x7fb3f00682a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7fb3ec009e60 tx=0x7fb3ec009780).stop 2023-11-15T03:05:41.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.926+0000 7fb3f6c9b700 1 -- 172.21.15.121:0/608056351 shutdown_connections 2023-11-15T03:05:41.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.926+0000 7fb3f6c9b700 1 --2- 172.21.15.121:0/608056351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f00c10b0 0x7fb3f0062c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.926+0000 7fb3f6c9b700 1 --2- 172.21.15.121:0/608056351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3f0067ec0 0x7fb3f00682a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.926+0000 7fb3f6c9b700 1 -- 172.21.15.121:0/608056351 >> 172.21.15.121:0/608056351 conn(0x7fb3f00cad40 msgr2=0x7fb3f00cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.926+0000 7fb3f6c9b700 1 -- 172.21.15.121:0/608056351 shutdown_connections 2023-11-15T03:05:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.926+0000 7fb3f6c9b700 1 -- 172.21.15.121:0/608056351 wait complete. 2023-11-15T03:05:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.926+0000 7fb3f6c9b700 1 Processor -- start 2023-11-15T03:05:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.930+0000 7fb3f6c9b700 1 -- start start 2023-11-15T03:05:41.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.930+0000 7fb3f6c9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f0063120 0x7fb3f014d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.930+0000 7fb3f6c9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f0067ec0 0x7fb3f0148180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.930+0000 7fb3f6c9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3f00c10b0 0x7fb3f0148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.930+0000 7fb3f6c9b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f00ad270 con 0x7fb3f0067ec0 2023-11-15T03:05:41.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.930+0000 7fb3f6c9b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f014af80 con 0x7fb3f00c10b0 2023-11-15T03:05:41.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.930+0000 7fb3f6c9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f014a970 con 0x7fb3f0063120 2023-11-15T03:05:41.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.930+0000 7fb3e7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f0067ec0 0x7fb3f0148180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.930+0000 7fb3e7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f0067ec0 0x7fb3f0148180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49322/0 (socket says 172.21.15.121:49322) 2023-11-15T03:05:41.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.930+0000 7fb3e7fff700 1 -- 172.21.15.121:0/3647356393 learned_addr learned my addr 172.21.15.121:0/3647356393 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:41.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.930+0000 7fb3f4a37700 1 --2- 172.21.15.121:0/3647356393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f0063120 0x7fb3f014d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.930+0000 7fb3f5238700 1 --2- 172.21.15.121:0/3647356393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3f00c10b0 0x7fb3f0148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.930+0000 7fb3e7fff700 1 -- 172.21.15.121:0/3647356393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f0063120 msgr2=0x7fb3f014d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.930+0000 7fb3e7fff700 1 --2- 172.21.15.121:0/3647356393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f0063120 0x7fb3f014d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.930+0000 7fb3e7fff700 1 -- 172.21.15.121:0/3647356393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3f00c10b0 msgr2=0x7fb3f0148670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:41.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.930+0000 7fb3e7fff700 1 --2- 172.21.15.121:0/3647356393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3f00c10b0 0x7fb3f0148670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:41.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.930+0000 7fb3e7fff700 1 -- 172.21.15.121:0/3647356393 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e00092a0 con 0x7fb3f0067ec0 2023-11-15T03:05:41.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.930+0000 7fb3e7fff700 1 --2- 172.21.15.121:0/3647356393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f0067ec0 0x7fb3f0148180 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7fb3e000e980 tx=0x7fb3e0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:41.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.934+0000 7fb3e4ff9700 1 -- 172.21.15.121:0/3647356393 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e000ee70 con 0x7fb3f0067ec0 2023-11-15T03:05:41.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.934+0000 7fb3e4ff9700 1 -- 172.21.15.121:0/3647356393 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e000c850 con 0x7fb3f0067ec0 2023-11-15T03:05:41.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.934+0000 7fb3f6c9b700 1 -- 172.21.15.121:0/3647356393 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3ec009a20 con 0x7fb3f0067ec0 2023-11-15T03:05:41.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.934+0000 7fb3e4ff9700 1 -- 172.21.15.121:0/3647356393 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e000cbd0 con 0x7fb3f0067ec0 2023-11-15T03:05:41.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.934+0000 7fb3f6c9b700 1 -- 172.21.15.121:0/3647356393 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f014ac10 con 0x7fb3f0067ec0 2023-11-15T03:05:41.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.934+0000 7fb3f6c9b700 1 -- 172.21.15.121:0/3647356393 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f014a7d0 con 0x7fb3f0067ec0 2023-11-15T03:05:41.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.934+0000 7fb3e4ff9700 1 -- 172.21.15.121:0/3647356393 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb3e001b400 con 0x7fb3f0067ec0 2023-11-15T03:05:41.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.938+0000 7fb3e4ff9700 1 --2- 172.21.15.121:0/3647356393 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3dc05a540 0x7fb3dc05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:41.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.938+0000 7fb3e4ff9700 1 -- 172.21.15.121:0/3647356393 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7298+0+0 (secure 0 0 0) 0x7fb3e0075150 con 0x7fb3f0067ec0 2023-11-15T03:05:41.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.938+0000 7fb3f4a37700 1 --2- 172.21.15.121:0/3647356393 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3dc05a540 0x7fb3dc05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:41.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.938+0000 7fb3f4a37700 1 --2- 172.21.15.121:0/3647356393 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3dc05a540 0x7fb3dc05ca00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fb3ec009e60 tx=0x7fb3ec009780).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:41.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:41.946+0000 7fb3e4ff9700 1 -- 172.21.15.121:0/3647356393 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb3f014a7d0 con 0x7fb3f0067ec0 2023-11-15T03:05:42.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:42.154+0000 7fb3f6c9b700 1 -- 172.21.15.121:0/3647356393 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fb3f014b110 con 0x7fb3f0067ec0 2023-11-15T03:05:43.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.070+0000 7fb3e4ff9700 1 -- 172.21.15.121:0/3647356393 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v159) v1 ==== 162+0+0 (secure 0 0 0) 0x7fb3e0048430 con 0x7fb3f0067ec0 2023-11-15T03:05:43.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.074+0000 7fb3f6c9b700 1 -- 172.21.15.121:0/3647356393 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3dc05a540 msgr2=0x7fb3dc05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:43.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.074+0000 7fb3f6c9b700 1 --2- 172.21.15.121:0/3647356393 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3dc05a540 0x7fb3dc05ca00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fb3ec009e60 tx=0x7fb3ec009780).stop 2023-11-15T03:05:43.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.074+0000 7fb3f6c9b700 1 -- 172.21.15.121:0/3647356393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f0067ec0 msgr2=0x7fb3f0148180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:43.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.074+0000 7fb3f6c9b700 1 --2- 172.21.15.121:0/3647356393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f0067ec0 0x7fb3f0148180 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7fb3e000e980 tx=0x7fb3e0009060).stop 2023-11-15T03:05:43.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.074+0000 7fb3f6c9b700 1 -- 172.21.15.121:0/3647356393 shutdown_connections 2023-11-15T03:05:43.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.074+0000 7fb3f6c9b700 1 --2- 172.21.15.121:0/3647356393 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb3dc05a540 0x7fb3dc05ca00 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:43.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.074+0000 7fb3f6c9b700 1 --2- 172.21.15.121:0/3647356393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3f00c10b0 0x7fb3f0148670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:43.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.074+0000 7fb3f6c9b700 1 --2- 172.21.15.121:0/3647356393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f0067ec0 0x7fb3f0148180 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:43.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.074+0000 7fb3f6c9b700 1 --2- 172.21.15.121:0/3647356393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f0063120 0x7fb3f014d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:43.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.074+0000 7fb3f6c9b700 1 -- 172.21.15.121:0/3647356393 >> 172.21.15.121:0/3647356393 conn(0x7fb3f00cad40 msgr2=0x7fb3f01507b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:43.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.074+0000 7fb3f6c9b700 1 -- 172.21.15.121:0/3647356393 shutdown_connections 2023-11-15T03:05:43.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.074+0000 7fb3f6c9b700 1 -- 172.21.15.121:0/3647356393 wait complete. 2023-11-15T03:05:43.087 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:05:43.507 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:05:43.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.658+0000 7f250dc5b700 1 Processor -- start 2023-11-15T03:05:43.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.658+0000 7f250dc5b700 1 -- start start 2023-11-15T03:05:43.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.662+0000 7f250dc5b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f25080aa240 0x7f2508067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:43.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.662+0000 7f250dc5b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25080cb5c0 con 0x7f2508153bb0 2023-11-15T03:05:43.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.662+0000 7f250dc5b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25080c9ca0 con 0x7f2508067f50 2023-11-15T03:05:43.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.662+0000 7f250dc5b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25080aa6f0 con 0x7f25080aa240 2023-11-15T03:05:43.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.662+0000 7f25077fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f2508153bb0 0x7f25080a9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:37042/0 (socket says 172.21.15.121:37042) 2023-11-15T03:05:43.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.662+0000 7f25077fe700 1 -- 172.21.15.121:0/121031172 learned_addr learned my addr 172.21.15.121:0/121031172 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:43.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.662+0000 7f24fffff700 1 --2- 172.21.15.121:0/121031172 >> v2:172.21.15.121:3300/0 conn(0x7f25080aa240 0x7f2508067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:43.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.666+0000 7f2506ffd700 1 -- 172.21.15.121:0/121031172 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907463841 0 0) 0x7f25080cb5c0 con 0x7f2508153bb0 2023-11-15T03:05:43.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.666+0000 7f2506ffd700 1 -- 172.21.15.121:0/121031172 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24f40033f0 con 0x7f2508153bb0 2023-11-15T03:05:43.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.666+0000 7f24fffff700 1 -- 172.21.15.121:0/121031172 >> v1:172.21.15.190:6789/0 conn(0x7f2508067f50 legacy=0x7f2508065d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:43.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.666+0000 7f24fffff700 1 -- 172.21.15.121:0/121031172 >> v1:172.21.15.121:6789/0 conn(0x7f2508153bb0 legacy=0x7f25080a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:43.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.666+0000 7f24fffff700 1 -- 172.21.15.121:0/121031172 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25080caa20 con 0x7f25080aa240 2023-11-15T03:05:43.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.666+0000 7f24fffff700 1 --2- 172.21.15.121:0/121031172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25080aa240 0x7f2508067a60 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7f24f0009760 tx=0x7f24f0009060).ready entity=mon.0 client_cookie=9925a78c89a1bcbf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:43.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.666+0000 7f2506ffd700 1 -- 172.21.15.121:0/121031172 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f000f510 con 0x7f25080aa240 2023-11-15T03:05:43.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.666+0000 7f2506ffd700 1 -- 172.21.15.121:0/121031172 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f000f670 con 0x7f25080aa240 2023-11-15T03:05:43.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.666+0000 7f2506ffd700 1 -- 172.21.15.121:0/121031172 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f000f9f0 con 0x7f25080aa240 2023-11-15T03:05:43.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.666+0000 7f250dc5b700 1 -- 172.21.15.121:0/121031172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25080aa240 msgr2=0x7f2508067a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:43.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.666+0000 7f250dc5b700 1 --2- 172.21.15.121:0/121031172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25080aa240 0x7f2508067a60 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7f24f0009760 tx=0x7f24f0009060).stop 2023-11-15T03:05:43.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.666+0000 7f250dc5b700 1 -- 172.21.15.121:0/121031172 shutdown_connections 2023-11-15T03:05:43.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.666+0000 7f250dc5b700 1 --2- 172.21.15.121:0/121031172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25080aa240 0x7f2508067a60 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:43.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.666+0000 7f250dc5b700 1 -- 172.21.15.121:0/121031172 >> 172.21.15.121:0/121031172 conn(0x7f2508058c50 msgr2=0x7f2508059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:43.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.666+0000 7f250dc5b700 1 -- 172.21.15.121:0/121031172 shutdown_connections 2023-11-15T03:05:43.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.666+0000 7f250dc5b700 1 -- 172.21.15.121:0/121031172 wait complete. 2023-11-15T03:05:43.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.670+0000 7f250dc5b700 1 Processor -- start 2023-11-15T03:05:43.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.670+0000 7f250dc5b700 1 -- start start 2023-11-15T03:05:43.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.670+0000 7f250dc5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2508067f50 0x7f25081642c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:43.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.670+0000 7f250dc5b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25080aa240 0x7f25080c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:43.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.670+0000 7f250dc5b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2508153bb0 0x7f25080abac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:43.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.670+0000 7f250dc5b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250814f3f0 con 0x7f2508153bb0 2023-11-15T03:05:43.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.670+0000 7f250dc5b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2508167a90 con 0x7f25080aa240 2023-11-15T03:05:43.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.670+0000 7f250dc5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25081647b0 con 0x7f2508067f50 2023-11-15T03:05:43.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.670+0000 7f2507fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2508153bb0 0x7f25080abac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:43.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.670+0000 7f2507fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2508153bb0 0x7f25080abac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49340/0 (socket says 172.21.15.121:49340) 2023-11-15T03:05:43.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.670+0000 7f2507fff700 1 -- 172.21.15.121:0/4158979770 learned_addr learned my addr 172.21.15.121:0/4158979770 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:43.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.670+0000 7f25077fe700 1 --2- 172.21.15.121:0/4158979770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2508067f50 0x7f25081642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:43.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.670+0000 7f24fffff700 1 --2- 172.21.15.121:0/4158979770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25080aa240 0x7f25080c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:43.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.674+0000 7f2507fff700 1 -- 172.21.15.121:0/4158979770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2508067f50 msgr2=0x7f25081642c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:43.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.674+0000 7f2507fff700 1 --2- 172.21.15.121:0/4158979770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2508067f50 0x7f25081642c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:43.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.674+0000 7f2507fff700 1 -- 172.21.15.121:0/4158979770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25080aa240 msgr2=0x7f25080c3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:43.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.674+0000 7f2507fff700 1 --2- 172.21.15.121:0/4158979770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25080aa240 0x7f25080c3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:43.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.674+0000 7f2507fff700 1 -- 172.21.15.121:0/4158979770 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f00092a0 con 0x7f2508153bb0 2023-11-15T03:05:43.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.674+0000 7f2507fff700 1 --2- 172.21.15.121:0/4158979770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2508153bb0 0x7f25080abac0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7f250000c810 tx=0x7f2500002560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:43.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.674+0000 7f24ff7fe700 1 -- 172.21.15.121:0/4158979770 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500010510 con 0x7f2508153bb0 2023-11-15T03:05:43.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.674+0000 7f24ff7fe700 1 -- 172.21.15.121:0/4158979770 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25000106b0 con 0x7f2508153bb0 2023-11-15T03:05:43.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.674+0000 7f250dc5b700 1 -- 172.21.15.121:0/4158979770 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25081652c0 con 0x7f2508153bb0 2023-11-15T03:05:43.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.674+0000 7f24ff7fe700 1 -- 172.21.15.121:0/4158979770 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500010a70 con 0x7f2508153bb0 2023-11-15T03:05:43.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.674+0000 7f250dc5b700 1 -- 172.21.15.121:0/4158979770 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2508164d00 con 0x7f2508153bb0 2023-11-15T03:05:43.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.674+0000 7f250dc5b700 1 -- 172.21.15.121:0/4158979770 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2508167c00 con 0x7f2508153bb0 2023-11-15T03:05:43.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.674+0000 7f24ff7fe700 1 -- 172.21.15.121:0/4158979770 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2500012440 con 0x7f2508153bb0 2023-11-15T03:05:43.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.678+0000 7f24ff7fe700 1 --2- 172.21.15.121:0/4158979770 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f24e805a540 0x7f24e805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:43.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.678+0000 7f24ff7fe700 1 -- 172.21.15.121:0/4158979770 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7298+0+0 (secure 0 0 0) 0x7f2500074580 con 0x7f2508153bb0 2023-11-15T03:05:43.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.678+0000 7f25077fe700 1 --2- 172.21.15.121:0/4158979770 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f24e805a540 0x7f24e805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:43.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.678+0000 7f25077fe700 1 --2- 172.21.15.121:0/4158979770 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f24e805a540 0x7f24e805ca00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f24f8007fd0 tx=0x7f24f8009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:43.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.682+0000 7f24ff7fe700 1 -- 172.21.15.121:0/4158979770 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2508167c00 con 0x7f2508153bb0 2023-11-15T03:05:43.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.882+0000 7f250dc5b700 1 -- 172.21.15.121:0/4158979770 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2508167c00 con 0x7f2508153bb0 2023-11-15T03:05:43.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.882+0000 7f24ff7fe700 1 -- 172.21.15.121:0/4158979770 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 159 v159) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f2508167c00 con 0x7f2508153bb0 2023-11-15T03:05:43.893 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:43.893 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":159,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":159,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:43.067282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8337":{"gid":8337,"name":"a","rank":0,"incarnation":159,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/3454441824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3454441824},{"type":"v1","addr":"172.21.15.121:6836","nonce":3454441824}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:05:43.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.886+0000 7f250dc5b700 1 -- 172.21.15.121:0/4158979770 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f24e805a540 msgr2=0x7f24e805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:43.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.886+0000 7f250dc5b700 1 --2- 172.21.15.121:0/4158979770 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f24e805a540 0x7f24e805ca00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f24f8007fd0 tx=0x7f24f8009040).stop 2023-11-15T03:05:43.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.886+0000 7f250dc5b700 1 -- 172.21.15.121:0/4158979770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2508153bb0 msgr2=0x7f25080abac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:43.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.886+0000 7f250dc5b700 1 --2- 172.21.15.121:0/4158979770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2508153bb0 0x7f25080abac0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7f250000c810 tx=0x7f2500002560).stop 2023-11-15T03:05:43.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.886+0000 7f250dc5b700 1 -- 172.21.15.121:0/4158979770 shutdown_connections 2023-11-15T03:05:43.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.886+0000 7f250dc5b700 1 --2- 172.21.15.121:0/4158979770 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f24e805a540 0x7f24e805ca00 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:43.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.886+0000 7f250dc5b700 1 --2- 172.21.15.121:0/4158979770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2508153bb0 0x7f25080abac0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:43.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.886+0000 7f250dc5b700 1 --2- 172.21.15.121:0/4158979770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25080aa240 0x7f25080c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:43.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.886+0000 7f250dc5b700 1 --2- 172.21.15.121:0/4158979770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2508067f50 0x7f25081642c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:43.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.886+0000 7f250dc5b700 1 -- 172.21.15.121:0/4158979770 >> 172.21.15.121:0/4158979770 conn(0x7f2508058c50 msgr2=0x7f25080b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:43.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.890+0000 7f250dc5b700 1 -- 172.21.15.121:0/4158979770 shutdown_connections 2023-11-15T03:05:43.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:43.890+0000 7f250dc5b700 1 -- 172.21.15.121:0/4158979770 wait complete. 2023-11-15T03:05:43.900 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 159 2023-11-15T03:05:43.918 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 159, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:05:43.067282+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 106, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8337}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8337': {'gid': 8337, 'name': 'a', 'rank': 0, 'incarnation': 159, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.121:6836/3454441824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 3454441824}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 3454441824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:05:43.919 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8337:up:replay 2023-11-15T03:05:43.919 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:05:49.325 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:05:49.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.482+0000 7f997f746700 1 Processor -- start 2023-11-15T03:05:49.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.482+0000 7f997f746700 1 -- start start 2023-11-15T03:05:49.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.486+0000 7f997f746700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9978152780 0x7f9978159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.486+0000 7f997f746700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f99780c65d0 0x7f99780c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.486+0000 7f997f746700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99780c3fa0 con 0x7f99780a5510 2023-11-15T03:05:49.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.486+0000 7f997f746700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9978159630 con 0x7f99780c65d0 2023-11-15T03:05:49.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.486+0000 7f997f746700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99780c30b0 con 0x7f9978152780 2023-11-15T03:05:49.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.486+0000 7f997dce3700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f99780a5510 0x7f99780a59c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:33612/0 (socket says 172.21.15.121:33612) 2023-11-15T03:05:49.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.486+0000 7f997dce3700 1 -- 172.21.15.121:0/123236286 learned_addr learned my addr 172.21.15.121:0/123236286 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:49.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.486+0000 7f997cce1700 1 --2- 172.21.15.121:0/123236286 >> v2:172.21.15.190:3300/0 conn(0x7f99780c65d0 0x7f99780c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.486+0000 7f997d4e2700 1 --2- 172.21.15.121:0/123236286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9978152780 0x7f9978159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.486+0000 7f996ffff700 1 -- 172.21.15.121:0/123236286 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445141290 0 0) 0x7f99780c3fa0 con 0x7f99780a5510 2023-11-15T03:05:49.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.486+0000 7f996ffff700 1 -- 172.21.15.121:0/123236286 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99640033f0 con 0x7f99780a5510 2023-11-15T03:05:49.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.486+0000 7f997cce1700 1 -- 172.21.15.121:0/123236286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9978152780 msgr2=0x7f9978159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:49.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.486+0000 7f997cce1700 1 --2- 172.21.15.121:0/123236286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9978152780 0x7f9978159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.486+0000 7f997cce1700 1 -- 172.21.15.121:0/123236286 >> v1:172.21.15.121:6789/0 conn(0x7f99780a5510 legacy=0x7f99780a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:49.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.486+0000 7f997cce1700 1 -- 172.21.15.121:0/123236286 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99780c3390 con 0x7f99780c65d0 2023-11-15T03:05:49.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.486+0000 7f997d4e2700 1 --2- 172.21.15.121:0/123236286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9978152780 0x7f9978159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:05:49.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.486+0000 7f997cce1700 1 --2- 172.21.15.121:0/123236286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99780c65d0 0x7f99780c6a50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f9968009760 tx=0x7f9968009060).ready entity=mon.1 client_cookie=47ae0248c3791294 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:49.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.490+0000 7f996ffff700 1 -- 172.21.15.121:0/123236286 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996800d4a0 con 0x7f99780c65d0 2023-11-15T03:05:49.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.490+0000 7f996ffff700 1 -- 172.21.15.121:0/123236286 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996800d600 con 0x7f99780c65d0 2023-11-15T03:05:49.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.490+0000 7f996ffff700 1 -- 172.21.15.121:0/123236286 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996800d980 con 0x7f99780c65d0 2023-11-15T03:05:49.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.490+0000 7f997f746700 1 -- 172.21.15.121:0/123236286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99780c65d0 msgr2=0x7f99780c6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:49.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.490+0000 7f997f746700 1 --2- 172.21.15.121:0/123236286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99780c65d0 0x7f99780c6a50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f9968009760 tx=0x7f9968009060).stop 2023-11-15T03:05:49.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.490+0000 7f997f746700 1 -- 172.21.15.121:0/123236286 shutdown_connections 2023-11-15T03:05:49.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.490+0000 7f997f746700 1 --2- 172.21.15.121:0/123236286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99780c65d0 0x7f99780c6a50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.490+0000 7f997f746700 1 --2- 172.21.15.121:0/123236286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9978152780 0x7f9978159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.490+0000 7f997f746700 1 -- 172.21.15.121:0/123236286 >> 172.21.15.121:0/123236286 conn(0x7f9978062190 msgr2=0x7f99780625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:49.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.490+0000 7f997f746700 1 -- 172.21.15.121:0/123236286 shutdown_connections 2023-11-15T03:05:49.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.490+0000 7f997f746700 1 -- 172.21.15.121:0/123236286 wait complete. 2023-11-15T03:05:49.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.490+0000 7f997f746700 1 Processor -- start 2023-11-15T03:05:49.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.490+0000 7f997f746700 1 -- start start 2023-11-15T03:05:49.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.494+0000 7f997f746700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99780a5510 0x7f997816cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.494+0000 7f997f746700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99780c65d0 0x7f99780aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.494+0000 7f997f746700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9978152780 0x7f99781479d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.494+0000 7f997f746700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997816d010 con 0x7f9978152780 2023-11-15T03:05:49.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.494+0000 7f997f746700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99780a7e80 con 0x7f99780c65d0 2023-11-15T03:05:49.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.494+0000 7f997f746700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9978150320 con 0x7f99780a5510 2023-11-15T03:05:49.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.494+0000 7f997dce3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9978152780 0x7f99781479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.494+0000 7f997dce3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9978152780 0x7f99781479d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45648/0 (socket says 172.21.15.121:45648) 2023-11-15T03:05:49.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.494+0000 7f997dce3700 1 -- 172.21.15.121:0/167371914 learned_addr learned my addr 172.21.15.121:0/167371914 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:49.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.494+0000 7f997cce1700 1 --2- 172.21.15.121:0/167371914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99780c65d0 0x7f99780aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.494+0000 7f997d4e2700 1 --2- 172.21.15.121:0/167371914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99780a5510 0x7f997816cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.494+0000 7f997dce3700 1 -- 172.21.15.121:0/167371914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99780a5510 msgr2=0x7f997816cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:49.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.494+0000 7f997dce3700 1 --2- 172.21.15.121:0/167371914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99780a5510 0x7f997816cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.494+0000 7f997dce3700 1 -- 172.21.15.121:0/167371914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99780c65d0 msgr2=0x7f99780aaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:49.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.494+0000 7f997dce3700 1 --2- 172.21.15.121:0/167371914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99780c65d0 0x7f99780aaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.494+0000 7f997dce3700 1 -- 172.21.15.121:0/167371914 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99740092a0 con 0x7f9978152780 2023-11-15T03:05:49.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.494+0000 7f997dce3700 1 --2- 172.21.15.121:0/167371914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9978152780 0x7f99781479d0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 rx=0x7f9970005ad0 tx=0x7f9970001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:49.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.494+0000 7f996d7fa700 1 -- 172.21.15.121:0/167371914 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9970005d30 con 0x7f9978152780 2023-11-15T03:05:49.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.498+0000 7f996d7fa700 1 -- 172.21.15.121:0/167371914 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9970005e90 con 0x7f9978152780 2023-11-15T03:05:49.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.498+0000 7f997f746700 1 -- 172.21.15.121:0/167371914 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99680092a0 con 0x7f9978152780 2023-11-15T03:05:49.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.498+0000 7f996d7fa700 1 -- 172.21.15.121:0/167371914 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9970007320 con 0x7f9978152780 2023-11-15T03:05:49.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.498+0000 7f997f746700 1 -- 172.21.15.121:0/167371914 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99781508e0 con 0x7f9978152780 2023-11-15T03:05:49.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.498+0000 7f997f746700 1 -- 172.21.15.121:0/167371914 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99780a6a00 con 0x7f9978152780 2023-11-15T03:05:49.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.498+0000 7f996d7fa700 1 -- 172.21.15.121:0/167371914 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9970007710 con 0x7f9978152780 2023-11-15T03:05:49.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.502+0000 7f996d7fa700 1 --2- 172.21.15.121:0/167371914 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f995c05a4e0 0x7f995c05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.502+0000 7f996d7fa700 1 -- 172.21.15.121:0/167371914 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7298+0+0 (secure 0 0 0) 0x7f9970074c70 con 0x7f9978152780 2023-11-15T03:05:49.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.502+0000 7f997d4e2700 1 --2- 172.21.15.121:0/167371914 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f995c05a4e0 0x7f995c05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.502+0000 7f997d4e2700 1 --2- 172.21.15.121:0/167371914 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f995c05a4e0 0x7f995c05c9a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f9974009d20 tx=0x7f9974009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:49.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.506+0000 7f996d7fa700 1 -- 172.21.15.121:0/167371914 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f99780a6a00 con 0x7f9978152780 2023-11-15T03:05:49.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.706+0000 7f997f746700 1 -- 172.21.15.121:0/167371914 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99780a6a00 con 0x7f9978152780 2023-11-15T03:05:49.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.706+0000 7f996d7fa700 1 -- 172.21.15.121:0/167371914 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f99780a6a00 con 0x7f9978152780 2023-11-15T03:05:49.715 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:49.715 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":162,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:46.093516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8337":{"gid":8337,"name":"a","rank":0,"incarnation":159,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6836/3454441824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3454441824},{"type":"v1","addr":"172.21.15.121:6836","nonce":3454441824}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:05:49.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.706+0000 7f997f746700 1 -- 172.21.15.121:0/167371914 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f995c05a4e0 msgr2=0x7f995c05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:49.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.706+0000 7f997f746700 1 --2- 172.21.15.121:0/167371914 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f995c05a4e0 0x7f995c05c9a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f9974009d20 tx=0x7f9974009060).stop 2023-11-15T03:05:49.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.706+0000 7f997f746700 1 -- 172.21.15.121:0/167371914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9978152780 msgr2=0x7f99781479d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:49.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.706+0000 7f997f746700 1 --2- 172.21.15.121:0/167371914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9978152780 0x7f99781479d0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 rx=0x7f9970005ad0 tx=0x7f9970001650).stop 2023-11-15T03:05:49.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.710+0000 7f997f746700 1 -- 172.21.15.121:0/167371914 shutdown_connections 2023-11-15T03:05:49.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.710+0000 7f997f746700 1 --2- 172.21.15.121:0/167371914 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f995c05a4e0 0x7f995c05c9a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.710+0000 7f997f746700 1 --2- 172.21.15.121:0/167371914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9978152780 0x7f99781479d0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.710+0000 7f997f746700 1 --2- 172.21.15.121:0/167371914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99780c65d0 0x7f99780aaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.710+0000 7f997f746700 1 --2- 172.21.15.121:0/167371914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99780a5510 0x7f997816cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.710+0000 7f997f746700 1 -- 172.21.15.121:0/167371914 >> 172.21.15.121:0/167371914 conn(0x7f9978062190 msgr2=0x7f99780a59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:49.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.710+0000 7f997f746700 1 -- 172.21.15.121:0/167371914 shutdown_connections 2023-11-15T03:05:49.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.710+0000 7f997f746700 1 -- 172.21.15.121:0/167371914 wait complete. 2023-11-15T03:05:49.721 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 162 2023-11-15T03:05:49.736 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 162, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:05:46.093516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 106, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8337}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8337': {'gid': 8337, 'name': 'a', 'rank': 0, 'incarnation': 159, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.121:6836/3454441824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 3454441824}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 3454441824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:05:49.737 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T03:05:49.737 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-15T03:05:49.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.894+0000 7fb5dd2f1700 1 Processor -- start 2023-11-15T03:05:49.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.894+0000 7fb5dd2f1700 1 -- start start 2023-11-15T03:05:49.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.898+0000 7fb5dd2f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5d80c4920 0x7fb5d81522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.898+0000 7fb5dd2f1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb5d81527c0 0x7fb5d8063560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.898+0000 7fb5dd2f1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d81532d0 con 0x7fb5d814c360 2023-11-15T03:05:49.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.898+0000 7fb5dd2f1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d80630d0 con 0x7fb5d81527c0 2023-11-15T03:05:49.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.898+0000 7fb5dd2f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d8152ff0 con 0x7fb5d80c4920 2023-11-15T03:05:49.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.898+0000 7fb5d6ffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb5d814c360 0x7fb5d80c4210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:33626/0 (socket says 172.21.15.121:33626) 2023-11-15T03:05:49.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.898+0000 7fb5d6ffd700 1 -- 172.21.15.121:0/2195956365 learned_addr learned my addr 172.21.15.121:0/2195956365 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:49.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.898+0000 7fb5d77fe700 1 --2- 172.21.15.121:0/2195956365 >> v2:172.21.15.190:3300/0 conn(0x7fb5d81527c0 0x7fb5d8063560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.898+0000 7fb5cffff700 1 --2- 172.21.15.121:0/2195956365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5d80c4920 0x7fb5d81522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.898+0000 7fb5d67fc700 1 -- 172.21.15.121:0/2195956365 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491866220 0 0) 0x7fb5d81532d0 con 0x7fb5d814c360 2023-11-15T03:05:49.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.898+0000 7fb5d67fc700 1 -- 172.21.15.121:0/2195956365 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5c40033f0 con 0x7fb5d814c360 2023-11-15T03:05:49.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.898+0000 7fb5cffff700 1 -- 172.21.15.121:0/2195956365 >> v2:172.21.15.190:3300/0 conn(0x7fb5d81527c0 msgr2=0x7fb5d8063560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:49.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.898+0000 7fb5cffff700 1 --2- 172.21.15.121:0/2195956365 >> v2:172.21.15.190:3300/0 conn(0x7fb5d81527c0 0x7fb5d8063560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.898+0000 7fb5cffff700 1 -- 172.21.15.121:0/2195956365 >> v1:172.21.15.121:6789/0 conn(0x7fb5d814c360 legacy=0x7fb5d80c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:49.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.898+0000 7fb5cffff700 1 -- 172.21.15.121:0/2195956365 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d8062a80 con 0x7fb5d80c4920 2023-11-15T03:05:49.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.898+0000 7fb5d77fe700 1 --2- 172.21.15.121:0/2195956365 >> v2:172.21.15.190:3300/0 conn(0x7fb5d81527c0 0x7fb5d8063560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:05:49.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.898+0000 7fb5cffff700 1 --2- 172.21.15.121:0/2195956365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5d80c4920 0x7fb5d81522d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fb5c0009760 tx=0x7fb5c0009060).ready entity=mon.2 client_cookie=4dc9d35b437fffb1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:49.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.902+0000 7fb5d67fc700 1 -- 172.21.15.121:0/2195956365 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00090d0 con 0x7fb5d80c4920 2023-11-15T03:05:49.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.902+0000 7fb5d67fc700 1 -- 172.21.15.121:0/2195956365 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c000d4a0 con 0x7fb5d80c4920 2023-11-15T03:05:49.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.902+0000 7fb5d67fc700 1 -- 172.21.15.121:0/2195956365 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c000d820 con 0x7fb5d80c4920 2023-11-15T03:05:49.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.902+0000 7fb5dd2f1700 1 -- 172.21.15.121:0/2195956365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5d80c4920 msgr2=0x7fb5d81522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:49.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.902+0000 7fb5dd2f1700 1 --2- 172.21.15.121:0/2195956365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5d80c4920 0x7fb5d81522d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fb5c0009760 tx=0x7fb5c0009060).stop 2023-11-15T03:05:49.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.902+0000 7fb5dd2f1700 1 -- 172.21.15.121:0/2195956365 shutdown_connections 2023-11-15T03:05:49.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.902+0000 7fb5dd2f1700 1 --2- 172.21.15.121:0/2195956365 >> v2:172.21.15.190:3300/0 conn(0x7fb5d81527c0 0x7fb5d8063560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.902+0000 7fb5dd2f1700 1 --2- 172.21.15.121:0/2195956365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5d80c4920 0x7fb5d81522d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.902+0000 7fb5dd2f1700 1 -- 172.21.15.121:0/2195956365 >> 172.21.15.121:0/2195956365 conn(0x7fb5d8068b70 msgr2=0x7fb5d8068f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:49.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.902+0000 7fb5dd2f1700 1 -- 172.21.15.121:0/2195956365 shutdown_connections 2023-11-15T03:05:49.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.902+0000 7fb5dd2f1700 1 -- 172.21.15.121:0/2195956365 wait complete. 2023-11-15T03:05:49.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.902+0000 7fb5dd2f1700 1 Processor -- start 2023-11-15T03:05:49.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.902+0000 7fb5dd2f1700 1 -- start start 2023-11-15T03:05:49.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5dd2f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5d80c4920 0x7fb5d80ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5dd2f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5d814c360 0x7fb5d8164640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5dd2f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5d81527c0 0x7fb5d80c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5dd2f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d80acba0 con 0x7fb5d81527c0 2023-11-15T03:05:49.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5dd2f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d80abf90 con 0x7fb5d80c4920 2023-11-15T03:05:49.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5dd2f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d80ac270 con 0x7fb5d814c360 2023-11-15T03:05:49.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5d77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5d81527c0 0x7fb5d80c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5d77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5d81527c0 0x7fb5d80c7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45660/0 (socket says 172.21.15.121:45660) 2023-11-15T03:05:49.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5d77fe700 1 -- 172.21.15.121:0/1851379378 learned_addr learned my addr 172.21.15.121:0/1851379378 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:49.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5cffff700 1 --2- 172.21.15.121:0/1851379378 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5d814c360 0x7fb5d8164640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5d6ffd700 1 --2- 172.21.15.121:0/1851379378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5d80c4920 0x7fb5d80ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5d77fe700 1 -- 172.21.15.121:0/1851379378 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5d814c360 msgr2=0x7fb5d8164640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:49.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5d77fe700 1 --2- 172.21.15.121:0/1851379378 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5d814c360 0x7fb5d8164640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5d77fe700 1 -- 172.21.15.121:0/1851379378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5d80c4920 msgr2=0x7fb5d80ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:49.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5d77fe700 1 --2- 172.21.15.121:0/1851379378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5d80c4920 0x7fb5d80ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5d77fe700 1 -- 172.21.15.121:0/1851379378 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d0009a20 con 0x7fb5d81527c0 2023-11-15T03:05:49.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5d77fe700 1 --2- 172.21.15.121:0/1851379378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5d81527c0 0x7fb5d80c7980 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7fb5d0009fc0 tx=0x7fb5d000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:49.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5ceffd700 1 -- 172.21.15.121:0/1851379378 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d000cc80 con 0x7fb5d81527c0 2023-11-15T03:05:49.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5ceffd700 1 -- 172.21.15.121:0/1851379378 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d000f040 con 0x7fb5d81527c0 2023-11-15T03:05:49.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5dd2f1700 1 -- 172.21.15.121:0/1851379378 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c00092a0 con 0x7fb5d81527c0 2023-11-15T03:05:49.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5ceffd700 1 -- 172.21.15.121:0/1851379378 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0013400 con 0x7fb5d81527c0 2023-11-15T03:05:49.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.906+0000 7fb5dd2f1700 1 -- 172.21.15.121:0/1851379378 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d80ac830 con 0x7fb5d81527c0 2023-11-15T03:05:49.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.910+0000 7fb5ceffd700 1 -- 172.21.15.121:0/1851379378 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb5d00137f0 con 0x7fb5d81527c0 2023-11-15T03:05:49.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.910+0000 7fb5ceffd700 1 --2- 172.21.15.121:0/1851379378 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb5b805a510 0x7fb5b805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.910+0000 7fb5ceffd700 1 -- 172.21.15.121:0/1851379378 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7298+0+0 (secure 0 0 0) 0x7fb5d00756f0 con 0x7fb5d81527c0 2023-11-15T03:05:49.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.910+0000 7fb5d6ffd700 1 --2- 172.21.15.121:0/1851379378 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb5b805a510 0x7fb5b805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.910+0000 7fb5e76b1740 1 Processor -- start 2023-11-15T03:05:49.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5e76b1740 1 -- start start 2023-11-15T03:05:49.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5d6ffd700 1 --2- 172.21.15.121:0/1851379378 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb5b805a510 0x7fb5b805c9d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fb5d8062430 tx=0x7fb5c8009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:49.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5e76b1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x164e300 0x16955b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5e76b1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x164d510 0x164e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5e76b1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1693ab0 0x1693f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5e76b1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x15ce410 con 0x164e300 2023-11-15T03:05:49.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5e76b1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x15ce550 con 0x164d510 2023-11-15T03:05:49.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5e76b1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x15cea00 con 0x1693ab0 2023-11-15T03:05:49.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5d6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x164e300 0x16955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5d6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x164e300 0x16955b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45672/0 (socket says 172.21.15.121:45672) 2023-11-15T03:05:49.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5d6ffd700 1 -- 172.21.15.121:0/3019731618 learned_addr learned my addr 172.21.15.121:0/3019731618 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:49.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5cffff700 1 --2- 172.21.15.121:0/3019731618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x164d510 0x164e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5d6ffd700 1 -- 172.21.15.121:0/3019731618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1693ab0 msgr2=0x1693f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:49.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5d77fe700 1 --2- 172.21.15.121:0/3019731618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1693ab0 0x1693f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5d6ffd700 1 --2- 172.21.15.121:0/3019731618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1693ab0 0x1693f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5d6ffd700 1 -- 172.21.15.121:0/3019731618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x164d510 msgr2=0x164e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:49.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5d6ffd700 1 --2- 172.21.15.121:0/3019731618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x164d510 0x164e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5d6ffd700 1 -- 172.21.15.121:0/3019731618 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x15c7730 con 0x164e300 2023-11-15T03:05:49.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5d77fe700 1 --2- 172.21.15.121:0/3019731618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1693ab0 0x1693f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:05:49.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5d6ffd700 1 --2- 172.21.15.121:0/3019731618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x164e300 0x16955b0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0x7fb5c8007390 tx=0x7fb5c8007000).ready entity=mon.0 client_cookie=95f21b3cbbee7bfc server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:49.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5b7fff700 1 -- 172.21.15.121:0/3019731618 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8015410 con 0x164e300 2023-11-15T03:05:49.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5b7fff700 1 -- 172.21.15.121:0/3019731618 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c8015570 con 0x164e300 2023-11-15T03:05:49.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5b7fff700 1 -- 172.21.15.121:0/3019731618 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c80158f0 con 0x164e300 2023-11-15T03:05:49.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5e76b1740 1 -- 172.21.15.121:0/3019731618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x164e300 msgr2=0x16955b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:49.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/3019731618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x164e300 0x16955b0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0x7fb5c8007390 tx=0x7fb5c8007000).stop 2023-11-15T03:05:49.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5e76b1740 1 -- 172.21.15.121:0/3019731618 shutdown_connections 2023-11-15T03:05:49.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/3019731618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1693ab0 0x1693f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/3019731618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x164d510 0x164e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/3019731618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x164e300 0x16955b0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5e76b1740 1 -- 172.21.15.121:0/3019731618 >> 172.21.15.121:0/3019731618 conn(0x142c550 msgr2=0x1694a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:49.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5e76b1740 1 -- 172.21.15.121:0/3019731618 shutdown_connections 2023-11-15T03:05:49.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.914+0000 7fb5e76b1740 1 -- 172.21.15.121:0/3019731618 wait complete. 2023-11-15T03:05:49.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.918+0000 7fb5e76b1740 1 -- start start 2023-11-15T03:05:49.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.918+0000 7fb5e76b1740 1 Processor -- start 2023-11-15T03:05:49.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.918+0000 7fb5e76b1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x164d510 0x16679c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.918+0000 7fb5e76b1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x164e300 0x1674b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.918+0000 7fb5e76b1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1693ab0 0x1674ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.918+0000 7fb5e76b1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1599650 con 0x1693ab0 2023-11-15T03:05:49.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.918+0000 7fb5e76b1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1667eb0 con 0x164e300 2023-11-15T03:05:49.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.918+0000 7fb5e76b1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1664e40 con 0x164d510 2023-11-15T03:05:49.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.918+0000 7fb5e76b1740 10 client.0 ll_register_callbacks cb 0x1606020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:05:49.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.918+0000 7fb5d77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1693ab0 0x1674ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.918+0000 7fb5d77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1693ab0 0x1674ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45674/0 (socket says 172.21.15.121:45674) 2023-11-15T03:05:49.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.918+0000 7fb5d77fe700 1 -- 172.21.15.121:0/1776697343 learned_addr learned my addr 172.21.15.121:0/1776697343 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:49.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.918+0000 7fb5d6ffd700 1 --2- 172.21.15.121:0/1776697343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x164d510 0x16679c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.918+0000 7fb5cffff700 1 --2- 172.21.15.121:0/1776697343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x164e300 0x1674b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5d77fe700 1 -- 172.21.15.121:0/1776697343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x164d510 msgr2=0x16679c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:49.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5d77fe700 1 --2- 172.21.15.121:0/1776697343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x164d510 0x16679c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5d77fe700 1 -- 172.21.15.121:0/1776697343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x164e300 msgr2=0x1674b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:49.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5d77fe700 1 --2- 172.21.15.121:0/1776697343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x164e300 0x1674b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5d77fe700 1 -- 172.21.15.121:0/1776697343 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c800f040 con 0x1693ab0 2023-11-15T03:05:49.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5d77fe700 1 --2- 172.21.15.121:0/1776697343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1693ab0 0x1674ff0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7fb5d00736a0 tx=0x7fb5d0009780).ready entity=mon.0 client_cookie=bc7ecffee11a6b40 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:49.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:05:49.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5b57fa700 1 -- 172.21.15.121:0/1776697343 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0048420 con 0x1693ab0 2023-11-15T03:05:49.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1776697343 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x15c41a0 con 0x1693ab0 2023-11-15T03:05:49.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5b57fa700 1 -- 172.21.15.121:0/1776697343 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 7298+0+0 (secure 0 0 0) 0x7fb5d0048580 con 0x1693ab0 2023-11-15T03:05:49.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5b57fa700 1 -- 172.21.15.121:0/1776697343 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=162) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5d0048740 con 0x1693ab0 2023-11-15T03:05:49.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5e76b1740 10 client.8433 fetch_fsmap learned FSMap version 162 2023-11-15T03:05:49.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1776697343 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=162,monmap=2+}) v3 -- 0x15c2960 con 0x1693ab0 2023-11-15T03:05:49.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5b57fa700 1 -- 172.21.15.121:0/1776697343 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x15c41a0 con 0x1693ab0 2023-11-15T03:05:49.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5b57fa700 1 -- 172.21.15.121:0/1776697343 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 162) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fb5d004b7a0 con 0x1693ab0 2023-11-15T03:05:49.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5e76b1740 10 client.8433 fetch_fsmap finished waiting for FSMap version 162 2023-11-15T03:05:49.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5e76b1740 10 client.8433 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:05:49.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5e76b1740 20 client.8433 populate_metadata read hostname 'smithi121' 2023-11-15T03:05:49.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/1776697343 >> [v2:172.21.15.121:6834/3454441824,v1:172.21.15.121:6836/3454441824] conn(0x1676510 0x16a4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5e76b1740 4 client.8433 mds_command: new command op to 8337 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:05:49.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1776697343 --> [v2:172.21.15.121:6834/3454441824,v1:172.21.15.121:6836/3454441824] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x15ba6f0 con 0x1676510 2023-11-15T03:05:49.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5d6ffd700 1 --2- 172.21.15.121:0/1776697343 >> [v2:172.21.15.121:6834/3454441824,v1:172.21.15.121:6836/3454441824] conn(0x1676510 0x16a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5d6ffd700 1 --2- 172.21.15.121:0/1776697343 >> [v2:172.21.15.121:6834/3454441824,v1:172.21.15.121:6836/3454441824] conn(0x1676510 0x16a4270 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=a2b234f938a16f0d server_cookie=57ac262def4df3d3 in_seq=0 out_seq=0 2023-11-15T03:05:49.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5b57fa700 10 client.8433 ms_handle_connect on v2:172.21.15.121:6834/3454441824 2023-11-15T03:05:49.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5b57fa700 1 -- 172.21.15.121:0/1776697343 <== mds.0 v2:172.21.15.121:6834/3454441824 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x15ba6f0 con 0x1676510 2023-11-15T03:05:49.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.922+0000 7fb5b57fa700 10 client.8433 handle_command_reply: tid=0 2023-11-15T03:05:49.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.926+0000 7fb5e76b1740 1 client.8433 shutdown 2023-11-15T03:05:49.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.926+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1776697343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1693ab0 msgr2=0x1674ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:49.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.926+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/1776697343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1693ab0 0x1674ff0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7fb5d00736a0 tx=0x7fb5d0009780).stop 2023-11-15T03:05:49.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.926+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1776697343 shutdown_connections 2023-11-15T03:05:49.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.926+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/1776697343 >> [v2:172.21.15.121:6834/3454441824,v1:172.21.15.121:6836/3454441824] conn(0x1676510 0x16a4270 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.926+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/1776697343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1693ab0 0x1674ff0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.926+0000 7fb5b57fa700 0 client.8433 ms_handle_reset on v2:172.21.15.121:6834/3454441824 2023-11-15T03:05:49.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.926+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/1776697343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x164e300 0x1674b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.926+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/1776697343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x164d510 0x16679c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.926+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1776697343 >> 172.21.15.121:0/1776697343 conn(0x142c550 msgr2=0x168b230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:49.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.926+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1776697343 shutdown_connections 2023-11-15T03:05:49.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.926+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1776697343 wait complete. 2023-11-15T03:05:49.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.926+0000 7fb5e76b1740 20 client.8433 trim_cache size 0 max 16384 2023-11-15T03:05:49.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.942+0000 7fb5e76b1740 1 Processor -- start 2023-11-15T03:05:49.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.942+0000 7fb5e76b1740 1 -- start start 2023-11-15T03:05:49.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5e76b1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x164d510 0x1649580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5e76b1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x164e300 0x16f1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5e76b1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1693ab0 0x16f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5e76b1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x15a3c60 con 0x164d510 2023-11-15T03:05:49.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5e76b1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x15ba6f0 con 0x1693ab0 2023-11-15T03:05:49.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5e76b1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1692a40 con 0x164e300 2023-11-15T03:05:49.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5d6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x164d510 0x1649580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5d6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x164d510 0x1649580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45686/0 (socket says 172.21.15.121:45686) 2023-11-15T03:05:49.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5d6ffd700 1 -- 172.21.15.121:0/1550697840 learned_addr learned my addr 172.21.15.121:0/1550697840 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:49.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5cffff700 1 --2- 172.21.15.121:0/1550697840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x164e300 0x16f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5d77fe700 1 --2- 172.21.15.121:0/1550697840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1693ab0 0x16f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5d6ffd700 1 -- 172.21.15.121:0/1550697840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x164e300 msgr2=0x16f1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:49.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5d6ffd700 1 --2- 172.21.15.121:0/1550697840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x164e300 0x16f1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5d6ffd700 1 -- 172.21.15.121:0/1550697840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1693ab0 msgr2=0x16f1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:49.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5d6ffd700 1 --2- 172.21.15.121:0/1550697840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1693ab0 0x16f1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5d6ffd700 1 -- 172.21.15.121:0/1550697840 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d80c81d0 con 0x164d510 2023-11-15T03:05:49.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5d77fe700 1 --2- 172.21.15.121:0/1550697840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1693ab0 0x16f1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:05:49.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5d6ffd700 1 --2- 172.21.15.121:0/1550697840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x164d510 0x1649580 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7fb5c8012000 tx=0x7fb5c8025040).ready entity=mon.0 client_cookie=837153227c4f01d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:49.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5ccff9700 1 -- 172.21.15.121:0/1550697840 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8029030 con 0x164d510 2023-11-15T03:05:49.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5ccff9700 1 -- 172.21.15.121:0/1550697840 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c802a040 con 0x164d510 2023-11-15T03:05:49.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5ccff9700 1 -- 172.21.15.121:0/1550697840 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c802d280 con 0x164d510 2023-11-15T03:05:49.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1550697840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x164d510 msgr2=0x1649580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:49.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/1550697840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x164d510 0x1649580 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7fb5c8012000 tx=0x7fb5c8025040).stop 2023-11-15T03:05:49.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1550697840 shutdown_connections 2023-11-15T03:05:49.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/1550697840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1693ab0 0x16f1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/1550697840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x164e300 0x16f1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/1550697840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x164d510 0x1649580 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1550697840 >> 172.21.15.121:0/1550697840 conn(0x142c550 msgr2=0x1673270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:49.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1550697840 shutdown_connections 2023-11-15T03:05:49.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.946+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1550697840 wait complete. 2023-11-15T03:05:49.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.950+0000 7fb5e76b1740 1 -- start start 2023-11-15T03:05:49.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.950+0000 7fb5e76b1740 1 Processor -- start 2023-11-15T03:05:49.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.950+0000 7fb5e76b1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x164d510 0x16f6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5e76b1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x164e300 0x16919a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5e76b1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1693ab0 0x16fa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5e76b1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x16f6a00 con 0x1693ab0 2023-11-15T03:05:49.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5e76b1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x16fabd0 con 0x164e300 2023-11-15T03:05:49.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5e76b1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1664050 con 0x164d510 2023-11-15T03:05:49.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5e76b1740 10 client.0 ll_register_callbacks cb 0x7fb5d0009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:05:49.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5d77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1693ab0 0x16fa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5d77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1693ab0 0x16fa070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45696/0 (socket says 172.21.15.121:45696) 2023-11-15T03:05:49.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5d77fe700 1 -- 172.21.15.121:0/1093351447 learned_addr learned my addr 172.21.15.121:0/1093351447 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:49.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5cffff700 1 --2- 172.21.15.121:0/1093351447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x164e300 0x16919a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5d6ffd700 1 --2- 172.21.15.121:0/1093351447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x164d510 0x16f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5d77fe700 1 -- 172.21.15.121:0/1093351447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x164d510 msgr2=0x16f6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:49.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5d77fe700 1 --2- 172.21.15.121:0/1093351447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x164d510 0x16f6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5d77fe700 1 -- 172.21.15.121:0/1093351447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x164e300 msgr2=0x16919a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:49.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5d77fe700 1 --2- 172.21.15.121:0/1093351447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x164e300 0x16919a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5d77fe700 1 -- 172.21.15.121:0/1093351447 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x15c7730 con 0x1693ab0 2023-11-15T03:05:49.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5d6ffd700 1 --2- 172.21.15.121:0/1093351447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x164d510 0x16f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:49.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5d77fe700 1 --2- 172.21.15.121:0/1093351447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1693ab0 0x16fa070 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 rx=0x7fb5d0048b30 tx=0x7fb5d001e000).ready entity=mon.0 client_cookie=9d8ce5f424690354 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:49.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:05:49.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5b57fa700 1 -- 172.21.15.121:0/1093351447 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d004bb20 con 0x1693ab0 2023-11-15T03:05:49.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5b57fa700 1 -- 172.21.15.121:0/1093351447 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 7298+0+0 (secure 0 0 0) 0x7fb5d0020e10 con 0x1693ab0 2023-11-15T03:05:49.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1093351447 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5c8015570 con 0x1693ab0 2023-11-15T03:05:49.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5b57fa700 1 -- 172.21.15.121:0/1093351447 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=162) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5d0048420 con 0x1693ab0 2023-11-15T03:05:49.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5e76b1740 10 client.8439 fetch_fsmap learned FSMap version 162 2023-11-15T03:05:49.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1093351447 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=162,monmap=2+}) v3 -- 0x15c26b0 con 0x1693ab0 2023-11-15T03:05:49.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5b57fa700 1 -- 172.21.15.121:0/1093351447 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c8015570 con 0x1693ab0 2023-11-15T03:05:49.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5b57fa700 1 -- 172.21.15.121:0/1093351447 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 162) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fb5d0048580 con 0x1693ab0 2023-11-15T03:05:49.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5e76b1740 10 client.8439 fetch_fsmap finished waiting for FSMap version 162 2023-11-15T03:05:49.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5e76b1740 10 client.8439 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:05:49.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5e76b1740 20 client.8439 populate_metadata read hostname 'smithi121' 2023-11-15T03:05:49.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/1093351447 >> [v2:172.21.15.121:6834/3454441824,v1:172.21.15.121:6836/3454441824] conn(0x1649610 0x1678d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:49.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5e76b1740 4 client.8439 mds_command: new command op to 8337 tid=0[{"prefix": "status"}] 2023-11-15T03:05:49.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1093351447 --> [v2:172.21.15.121:6834/3454441824,v1:172.21.15.121:6836/3454441824] -- command(tid 0: {"prefix": "status"}) v1 -- 0x14834e0 con 0x1649610 2023-11-15T03:05:49.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5d6ffd700 1 --2- 172.21.15.121:0/1093351447 >> [v2:172.21.15.121:6834/3454441824,v1:172.21.15.121:6836/3454441824] conn(0x1649610 0x1678d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:49.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5d6ffd700 1 --2- 172.21.15.121:0/1093351447 >> [v2:172.21.15.121:6834/3454441824,v1:172.21.15.121:6836/3454441824] conn(0x1649610 0x1678d20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=a2689f11fba6e4fc server_cookie=92389d6d3cee687 in_seq=0 out_seq=0 2023-11-15T03:05:49.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5b57fa700 10 client.8439 ms_handle_connect on v2:172.21.15.121:6834/3454441824 2023-11-15T03:05:49.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5b57fa700 1 -- 172.21.15.121:0/1093351447 <== mds.0 v2:172.21.15.121:6834/3454441824 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x14834e0 con 0x1649610 2023-11-15T03:05:49.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5b57fa700 10 client.8439 handle_command_reply: tid=0 2023-11-15T03:05:49.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.954+0000 7fb5e76b1740 1 client.8439 shutdown 2023-11-15T03:05:49.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1093351447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1693ab0 msgr2=0x16fa070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:49.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/1093351447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1693ab0 0x16fa070 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 rx=0x7fb5d0048b30 tx=0x7fb5d001e000).stop 2023-11-15T03:05:49.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1093351447 shutdown_connections 2023-11-15T03:05:49.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/1093351447 >> [v2:172.21.15.121:6834/3454441824,v1:172.21.15.121:6836/3454441824] conn(0x1649610 0x1678d20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/1093351447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1693ab0 0x16fa070 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/1093351447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x164e300 0x16919a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5e76b1740 1 --2- 172.21.15.121:0/1093351447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x164d510 0x16f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5b57fa700 0 client.8439 ms_handle_reset on v2:172.21.15.121:6834/3454441824 2023-11-15T03:05:49.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1093351447 >> 172.21.15.121:0/1093351447 conn(0x142c550 msgr2=0x1673270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:49.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1093351447 shutdown_connections 2023-11-15T03:05:49.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5e76b1740 1 -- 172.21.15.121:0/1093351447 wait complete. 2023-11-15T03:05:49.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5e76b1740 20 client.8439 trim_cache size 0 max 16384 2023-11-15T03:05:49.980 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:05:49.980 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T03:05:49.981 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T03:05:49.981 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 8337, 2023-11-15T03:05:49.981 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T03:05:49.981 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T03:05:49.981 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T03:05:49.982 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 6.8888768330000003, 2023-11-15T03:05:49.982 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 162, 2023-11-15T03:05:49.982 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 106, 2023-11-15T03:05:49.983 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 106, 2023-11-15T03:05:49.983 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 16.574563543 2023-11-15T03:05:49.983 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:05:49.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5dd2f1700 1 -- 172.21.15.121:0/1851379378 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb5b805a510 msgr2=0x7fb5b805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:49.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5dd2f1700 1 --2- 172.21.15.121:0/1851379378 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb5b805a510 0x7fb5b805c9d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fb5d8062430 tx=0x7fb5c8009040).stop 2023-11-15T03:05:49.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5dd2f1700 1 -- 172.21.15.121:0/1851379378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5d81527c0 msgr2=0x7fb5d80c7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:49.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5dd2f1700 1 --2- 172.21.15.121:0/1851379378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5d81527c0 0x7fb5d80c7980 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7fb5d0009fc0 tx=0x7fb5d000b040).stop 2023-11-15T03:05:49.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5dd2f1700 1 -- 172.21.15.121:0/1851379378 shutdown_connections 2023-11-15T03:05:49.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5dd2f1700 1 --2- 172.21.15.121:0/1851379378 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb5b805a510 0x7fb5b805c9d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5dd2f1700 1 --2- 172.21.15.121:0/1851379378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5d81527c0 0x7fb5d80c7980 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5dd2f1700 1 --2- 172.21.15.121:0/1851379378 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5d814c360 0x7fb5d8164640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5dd2f1700 1 --2- 172.21.15.121:0/1851379378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5d80c4920 0x7fb5d80ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:49.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5dd2f1700 1 -- 172.21.15.121:0/1851379378 >> 172.21.15.121:0/1851379378 conn(0x7fb5d8068b70 msgr2=0x7fb5d8150fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:49.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5dd2f1700 1 -- 172.21.15.121:0/1851379378 shutdown_connections 2023-11-15T03:05:49.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:49.958+0000 7fb5dd2f1700 1 -- 172.21.15.121:0/1851379378 wait complete. 2023-11-15T03:05:49.989 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:05:49.990 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 400.00000000', proceeding to ls 2023-11-15T03:05:49.990 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T03:05:50.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.126+0000 7fb4a8fb9700 1 Processor -- start 2023-11-15T03:05:50.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.126+0000 7fb4a8fb9700 1 -- start start 2023-11-15T03:05:50.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a8fb9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb4a406a4a0 0x7fb4a4151510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:50.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a8fb9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a4065dc0 con 0x7fb4a406a0c0 2023-11-15T03:05:50.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a8fb9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a41566c0 con 0x7fb4a4151a00 2023-11-15T03:05:50.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a8fb9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a4155b40 con 0x7fb4a406a4a0 2023-11-15T03:05:50.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a37fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb4a406a0c0 0x7fb4a4068330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:33628/0 (socket says 172.21.15.121:33628) 2023-11-15T03:05:50.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a37fe700 1 -- 172.21.15.121:0/3128100341 learned_addr learned my addr 172.21.15.121:0/3128100341 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:50.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a2ffd700 1 -- 172.21.15.121:0/3128100341 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334311135 0 0) 0x7fb4a4065dc0 con 0x7fb4a406a0c0 2023-11-15T03:05:50.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb49affd700 1 --2- 172.21.15.121:0/3128100341 >> v2:172.21.15.190:3300/0 conn(0x7fb4a406a4a0 0x7fb4a4151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:50.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a2ffd700 1 -- 172.21.15.121:0/3128100341 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4900033f0 con 0x7fb4a406a0c0 2023-11-15T03:05:50.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a2ffd700 1 -- 172.21.15.121:0/3128100341 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4059845135 0 0) 0x7fb4900033f0 con 0x7fb4a406a0c0 2023-11-15T03:05:50.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a2ffd700 1 -- 172.21.15.121:0/3128100341 >> v2:172.21.15.190:3300/0 conn(0x7fb4a406a4a0 msgr2=0x7fb4a4151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:50.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a2ffd700 1 --2- 172.21.15.121:0/3128100341 >> v2:172.21.15.190:3300/0 conn(0x7fb4a406a4a0 0x7fb4a4151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:50.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a2ffd700 1 -- 172.21.15.121:0/3128100341 >> v1:172.21.15.190:6789/0 conn(0x7fb4a4151a00 legacy=0x7fb4a40bb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:50.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a2ffd700 1 -- 172.21.15.121:0/3128100341 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a4155e20 con 0x7fb4a406a0c0 2023-11-15T03:05:50.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a2ffd700 1 -- 172.21.15.121:0/3128100341 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fb4940017f0 con 0x7fb4a406a0c0 2023-11-15T03:05:50.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a2ffd700 1 -- 172.21.15.121:0/3128100341 >> v1:172.21.15.121:6789/0 conn(0x7fb4a406a0c0 legacy=0x7fb4a4068330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:50.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a2ffd700 1 --2- 172.21.15.121:0/3128100341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4900038e0 0x7fb490003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:50.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a2ffd700 1 --2- 172.21.15.121:0/3128100341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb490004690 0x7fb490004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:50.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a2ffd700 1 --2- 172.21.15.121:0/3128100341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb490005050 0x7fb49000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:50.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a2ffd700 1 -- 172.21.15.121:0/3128100341 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a4155b40 con 0x7fb4900038e0 2023-11-15T03:05:50.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a2ffd700 1 -- 172.21.15.121:0/3128100341 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49000bfe0 con 0x7fb490004690 2023-11-15T03:05:50.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a2ffd700 1 -- 172.21.15.121:0/3128100341 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49000c1a0 con 0x7fb490005050 2023-11-15T03:05:50.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a37fe700 1 --2- 172.21.15.121:0/3128100341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4900038e0 0x7fb490003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:50.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a3fff700 1 --2- 172.21.15.121:0/3128100341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb490005050 0x7fb49000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:50.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb49affd700 1 --2- 172.21.15.121:0/3128100341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb490004690 0x7fb490004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:50.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a37fe700 1 -- 172.21.15.121:0/3128100341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb490005050 msgr2=0x7fb49000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:50.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a37fe700 1 --2- 172.21.15.121:0/3128100341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb490005050 0x7fb49000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:50.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a37fe700 1 -- 172.21.15.121:0/3128100341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb490004690 msgr2=0x7fb490004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:50.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a37fe700 1 --2- 172.21.15.121:0/3128100341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb490004690 0x7fb490004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:50.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a3fff700 1 -- 172.21.15.121:0/3128100341 reap_dead start 2023-11-15T03:05:50.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.130+0000 7fb4a37fe700 1 -- 172.21.15.121:0/3128100341 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4900033f0 con 0x7fb4900038e0 2023-11-15T03:05:50.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.134+0000 7fb4a37fe700 1 --2- 172.21.15.121:0/3128100341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4900038e0 0x7fb490003d00 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7fb49400c480 tx=0x7fb49400c790).ready entity=mon.0 client_cookie=6cc9a0261f3d74d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:50.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.134+0000 7fb4a2ffd700 1 -- 172.21.15.121:0/3128100341 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb494002030 con 0x7fb4900038e0 2023-11-15T03:05:50.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.134+0000 7fb4a2ffd700 1 -- 172.21.15.121:0/3128100341 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49400cde0 con 0x7fb4900038e0 2023-11-15T03:05:50.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.134+0000 7fb4a2ffd700 1 -- 172.21.15.121:0/3128100341 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb494005d30 con 0x7fb4900038e0 2023-11-15T03:05:50.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.134+0000 7fb4a8fb9700 1 -- 172.21.15.121:0/3128100341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4900038e0 msgr2=0x7fb490003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:50.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.134+0000 7fb4a8fb9700 1 --2- 172.21.15.121:0/3128100341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4900038e0 0x7fb490003d00 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7fb49400c480 tx=0x7fb49400c790).stop 2023-11-15T03:05:50.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.134+0000 7fb4a8fb9700 1 -- 172.21.15.121:0/3128100341 shutdown_connections 2023-11-15T03:05:50.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.134+0000 7fb4a8fb9700 1 --2- 172.21.15.121:0/3128100341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4900038e0 0x7fb490003d00 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:50.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.134+0000 7fb4a8fb9700 1 -- 172.21.15.121:0/3128100341 >> 172.21.15.121:0/3128100341 conn(0x7fb4a40677c0 msgr2=0x7fb4a40c65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:50.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.134+0000 7fb4a8fb9700 1 -- 172.21.15.121:0/3128100341 shutdown_connections 2023-11-15T03:05:50.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.134+0000 7fb4a8fb9700 1 -- 172.21.15.121:0/3128100341 wait complete. 2023-11-15T03:05:50.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.138+0000 7fb4a8fb9700 1 Processor -- start 2023-11-15T03:05:50.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.138+0000 7fb4a8fb9700 1 -- start start 2023-11-15T03:05:50.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.138+0000 7fb4a8fb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4900038e0 0x7fb4a41696c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:50.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.138+0000 7fb4a8fb9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4a414a5a0 0x7fb4a40bf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:50.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.138+0000 7fb4a8fb9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4a41542c0 0x7fb4a4154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:50.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.138+0000 7fb4a8fb9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a40c0720 con 0x7fb4a414a5a0 2023-11-15T03:05:50.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.138+0000 7fb4a8fb9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a4154c30 con 0x7fb4a41542c0 2023-11-15T03:05:50.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.138+0000 7fb4a8fb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a40bfb10 con 0x7fb4900038e0 2023-11-15T03:05:50.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.138+0000 7fb49affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4a414a5a0 0x7fb4a40bf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:50.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.138+0000 7fb49affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4a414a5a0 0x7fb4a40bf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45708/0 (socket says 172.21.15.121:45708) 2023-11-15T03:05:50.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.138+0000 7fb49affd700 1 -- 172.21.15.121:0/1997188045 learned_addr learned my addr 172.21.15.121:0/1997188045 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:50.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.138+0000 7fb4a3fff700 1 --2- 172.21.15.121:0/1997188045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4a41542c0 0x7fb4a4154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:50.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.138+0000 7fb4a37fe700 1 --2- 172.21.15.121:0/1997188045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4900038e0 0x7fb4a41696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:50.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.138+0000 7fb49affd700 1 -- 172.21.15.121:0/1997188045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4900038e0 msgr2=0x7fb4a41696c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:50.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.138+0000 7fb49affd700 1 --2- 172.21.15.121:0/1997188045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4900038e0 0x7fb4a41696c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:50.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.138+0000 7fb49affd700 1 -- 172.21.15.121:0/1997188045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4a41542c0 msgr2=0x7fb4a4154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:50.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.138+0000 7fb49affd700 1 --2- 172.21.15.121:0/1997188045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4a41542c0 0x7fb4a4154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:50.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.138+0000 7fb49affd700 1 -- 172.21.15.121:0/1997188045 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a40bfdf0 con 0x7fb4a414a5a0 2023-11-15T03:05:50.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.138+0000 7fb49affd700 1 --2- 172.21.15.121:0/1997188045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4a414a5a0 0x7fb4a40bf1c0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7fb48c010960 tx=0x7fb48c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:50.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.142+0000 7fb49bfff700 1 -- 172.21.15.121:0/1997188045 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c010ea0 con 0x7fb4a414a5a0 2023-11-15T03:05:50.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.142+0000 7fb49bfff700 1 -- 172.21.15.121:0/1997188045 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48c00e8d0 con 0x7fb4a414a5a0 2023-11-15T03:05:50.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.142+0000 7fb4a8fb9700 1 -- 172.21.15.121:0/1997188045 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a40c00d0 con 0x7fb4a414a5a0 2023-11-15T03:05:50.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.142+0000 7fb49bfff700 1 -- 172.21.15.121:0/1997188045 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c00ec50 con 0x7fb4a414a5a0 2023-11-15T03:05:50.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.142+0000 7fb4a8fb9700 1 -- 172.21.15.121:0/1997188045 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a414aa20 con 0x7fb4a414a5a0 2023-11-15T03:05:50.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.142+0000 7fb4a8fb9700 1 -- 172.21.15.121:0/1997188045 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a4152050 con 0x7fb4a414a5a0 2023-11-15T03:05:50.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.142+0000 7fb49bfff700 1 -- 172.21.15.121:0/1997188045 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb48c014470 con 0x7fb4a414a5a0 2023-11-15T03:05:50.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.146+0000 7fb49bfff700 1 --2- 172.21.15.121:0/1997188045 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb48405a510 0x7fb48405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:50.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.146+0000 7fb49bfff700 1 -- 172.21.15.121:0/1997188045 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7298+0+0 (secure 0 0 0) 0x7fb48c074d10 con 0x7fb4a414a5a0 2023-11-15T03:05:50.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.146+0000 7fb4a37fe700 1 --2- 172.21.15.121:0/1997188045 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb48405a510 0x7fb48405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:50.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.146+0000 7fb4a37fe700 1 --2- 172.21.15.121:0/1997188045 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb48405a510 0x7fb48405c9d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fb49400c7d0 tx=0x7fb494015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:50.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.150+0000 7fb49bfff700 1 -- 172.21.15.121:0/1997188045 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb48c07a020 con 0x7fb4a414a5a0 2023-11-15T03:05:50.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.354+0000 7fb4a8fb9700 1 -- 172.21.15.121:0/1997188045 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb4a4062720 con 0x7fb4a414a5a0 2023-11-15T03:05:50.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.354+0000 7fb49bfff700 1 -- 172.21.15.121:0/1997188045 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v162) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb48c047f30 con 0x7fb4a414a5a0 2023-11-15T03:05:50.365 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T03:05:50.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.358+0000 7fb4a8fb9700 1 -- 172.21.15.121:0/1997188045 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb48405a510 msgr2=0x7fb48405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:50.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.358+0000 7fb4a8fb9700 1 --2- 172.21.15.121:0/1997188045 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb48405a510 0x7fb48405c9d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fb49400c7d0 tx=0x7fb494015000).stop 2023-11-15T03:05:50.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.358+0000 7fb4a8fb9700 1 -- 172.21.15.121:0/1997188045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4a414a5a0 msgr2=0x7fb4a40bf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:50.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.358+0000 7fb4a8fb9700 1 --2- 172.21.15.121:0/1997188045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4a414a5a0 0x7fb4a40bf1c0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7fb48c010960 tx=0x7fb48c008040).stop 2023-11-15T03:05:50.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.358+0000 7fb4a8fb9700 1 -- 172.21.15.121:0/1997188045 shutdown_connections 2023-11-15T03:05:50.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.358+0000 7fb4a8fb9700 1 --2- 172.21.15.121:0/1997188045 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb48405a510 0x7fb48405c9d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:50.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.358+0000 7fb4a8fb9700 1 --2- 172.21.15.121:0/1997188045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4a41542c0 0x7fb4a4154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:50.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.358+0000 7fb4a8fb9700 1 --2- 172.21.15.121:0/1997188045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4a414a5a0 0x7fb4a40bf1c0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:50.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.358+0000 7fb4a8fb9700 1 --2- 172.21.15.121:0/1997188045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4900038e0 0x7fb4a41696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:50.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.358+0000 7fb4a8fb9700 1 -- 172.21.15.121:0/1997188045 >> 172.21.15.121:0/1997188045 conn(0x7fb4a40677c0 msgr2=0x7fb4a41550b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:50.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.358+0000 7fb4a8fb9700 1 -- 172.21.15.121:0/1997188045 shutdown_connections 2023-11-15T03:05:50.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:50.358+0000 7fb4a8fb9700 1 -- 172.21.15.121:0/1997188045 wait complete. 2023-11-15T03:05:50.385 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T03:05:50.385 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T03:05:50.386 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T03:05:50.387 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T03:05:50.387 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T03:05:50.388 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T03:05:50.388 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T03:05:50.388 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T03:05:50.389 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:05:50.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:05:50.390 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T03:05:50.409 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T03:05:50.409 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T03:05:50.409 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T03:05:50.409 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:05:50.410 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T03:05:50.410 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:05:50.410 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:05:50.410 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:05:50.410 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T03:05:50.411 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:05:50.411 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:05:50.411 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T03:05:50.411 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:05:50.411 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:05:50.412 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T03:05:50.412 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:05:50.412 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T03:05:50.412 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 41889sec preferred_lft 41889sec 2023-11-15T03:05:50.412 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T03:05:50.412 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:05:50.413 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T03:05:50.413 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:05:50.413 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T03:05:50.413 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:05:50.413 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T03:05:50.414 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T03:05:50.414 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T03:05:50.414 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:05:50.414 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T03:05:50.414 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:05:50.415 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:05:50.415 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T03:05:50.415 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T03:05:50.415 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:05:50.415 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:05:50.416 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T03:05:50.416 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T03:05:50.416 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:05:50.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:05:50.421 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T03:05:50.498 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T03:05:50.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:05:50.501 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T03:05:50.564 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T03:05:50.565 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T03:05:50.565 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T03:05:50.565 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T03:05:50.565 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T03:05:50.566 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T03:05:50.566 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T03:05:50.566 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T03:05:50.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:05:50.567 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T03:05:50.567 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T03:05:50.568 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T03:05:50.568 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T03:05:50.568 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T03:05:50.569 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T03:05:50.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:05:50.702 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T03:05:50.716 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T03:05:50.716 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T03:05:50.717 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:05:50.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:05:50.718 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T03:05:50.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:05:50.798 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T03:05:50.858 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T03:05:50.859 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:05:50.983 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:05:50.983 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:05:51.007 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:05:51.007 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:05:51.013 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T03:05:51.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:05:51.014 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T03:05:51.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:05:51.047 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2023-11-15T03:05:51.096 INFO:teuthology.orchestra.run:waiting for 20 2023-11-15T03:05:51.113 INFO:teuthology.orchestra.run.smithi121.stdout:.: 2023-11-15T03:05:51.114 INFO:teuthology.orchestra.run.smithi121.stdout:subdir 2023-11-15T03:05:51.114 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:51.114 INFO:teuthology.orchestra.run.smithi121.stdout:./subdir: 2023-11-15T03:05:51.114 INFO:teuthology.orchestra.run.smithi121.stdout:sixmegs 2023-11-15T03:05:51.115 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 400.00000000' did not caused DAMAGED state 2023-11-15T03:05:51.116 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 100.00000000.inode' 2023-11-15T03:05:51.116 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T03:05:51.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:05:51.117 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-15T03:05:51.201 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:05:51.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:05:51.202 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:05:51.272 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:05:51.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.470+0000 7f4a728e4700 1 Processor -- start 2023-11-15T03:05:51.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.474+0000 7f4a728e4700 1 -- start start 2023-11-15T03:05:51.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.474+0000 7f4a728e4700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4a6c157d90 0x7f4a6c158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:51.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.474+0000 7f4a728e4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4a6c0669b0 0x7f4a6c0c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:51.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.474+0000 7f4a728e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a6c0a8750 0x7f4a6c0a8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:51.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.474+0000 7f4a728e4700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a6c0c1fb0 con 0x7f4a6c157d90 2023-11-15T03:05:51.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.474+0000 7f4a728e4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a6c15bbd0 con 0x7f4a6c0669b0 2023-11-15T03:05:51.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.474+0000 7f4a728e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c0a90d0 con 0x7f4a6c0a8750 2023-11-15T03:05:51.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.474+0000 7f4a718e2700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4a6c157d90 0x7f4a6c158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:51.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a718e2700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4a6c157d90 0x7f4a6c158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45722/0 (socket says 172.21.15.121:45722) 2023-11-15T03:05:51.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a718e2700 1 -- 172.21.15.121:0/312973508 learned_addr learned my addr 172.21.15.121:0/312973508 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:51.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a710e1700 1 --2- 172.21.15.121:0/312973508 >> v2:172.21.15.190:3300/0 conn(0x7f4a6c0669b0 0x7f4a6c0c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:51.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a720e3700 1 --2- 172.21.15.121:0/312973508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a6c0a8750 0x7f4a6c0a8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:51.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a718e2700 1 -- 172.21.15.121:0/312973508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a6c0a8750 msgr2=0x7f4a6c0a8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:51.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a718e2700 1 --2- 172.21.15.121:0/312973508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a6c0a8750 0x7f4a6c0a8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:51.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a718e2700 1 -- 172.21.15.121:0/312973508 >> v2:172.21.15.190:3300/0 conn(0x7f4a6c0669b0 msgr2=0x7f4a6c0c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:51.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a718e2700 1 --2- 172.21.15.121:0/312973508 >> v2:172.21.15.190:3300/0 conn(0x7f4a6c0669b0 0x7f4a6c0c1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:51.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a718e2700 1 -- 172.21.15.121:0/312973508 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6c0bb910 con 0x7f4a6c157d90 2023-11-15T03:05:51.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a718e2700 1 --2- 172.21.15.121:0/312973508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a6c157d90 0x7f4a6c158170 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7f4a6c059700 tx=0x7f4a680090f0).ready entity=mon.0 client_cookie=50d9360a0572166b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:51.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a708e0700 1 -- 172.21.15.121:0/312973508 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6800b520 con 0x7f4a6c157d90 2023-11-15T03:05:51.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a708e0700 1 -- 172.21.15.121:0/312973508 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6800b680 con 0x7f4a6c157d90 2023-11-15T03:05:51.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a708e0700 1 -- 172.21.15.121:0/312973508 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6800ba00 con 0x7f4a6c157d90 2023-11-15T03:05:51.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a728e4700 1 -- 172.21.15.121:0/312973508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a6c157d90 msgr2=0x7f4a6c158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:51.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a728e4700 1 --2- 172.21.15.121:0/312973508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a6c157d90 0x7f4a6c158170 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7f4a6c059700 tx=0x7f4a680090f0).stop 2023-11-15T03:05:51.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a728e4700 1 -- 172.21.15.121:0/312973508 shutdown_connections 2023-11-15T03:05:51.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a728e4700 1 --2- 172.21.15.121:0/312973508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a6c0a8750 0x7f4a6c0a8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:51.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a728e4700 1 --2- 172.21.15.121:0/312973508 >> v2:172.21.15.190:3300/0 conn(0x7f4a6c0669b0 0x7f4a6c0c1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:51.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a728e4700 1 --2- 172.21.15.121:0/312973508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a6c157d90 0x7f4a6c158170 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:51.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a728e4700 1 -- 172.21.15.121:0/312973508 >> 172.21.15.121:0/312973508 conn(0x7f4a6c067a60 msgr2=0x7f4a6c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:51.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a728e4700 1 -- 172.21.15.121:0/312973508 shutdown_connections 2023-11-15T03:05:51.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.478+0000 7f4a728e4700 1 -- 172.21.15.121:0/312973508 wait complete. 2023-11-15T03:05:51.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.482+0000 7f4a728e4700 1 Processor -- start 2023-11-15T03:05:51.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.482+0000 7f4a728e4700 1 -- start start 2023-11-15T03:05:51.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.482+0000 7f4a728e4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a6c0669b0 0x7f4a6c15a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:51.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.482+0000 7f4a728e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a6c0a8750 0x7f4a6c15de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:51.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.482+0000 7f4a728e4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a6c157d90 0x7f4a6c15e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:51.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.482+0000 7f4a728e4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c15e800 con 0x7f4a6c157d90 2023-11-15T03:05:51.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.482+0000 7f4a728e4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c0ab280 con 0x7f4a6c0669b0 2023-11-15T03:05:51.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.482+0000 7f4a728e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c0aa670 con 0x7f4a6c0a8750 2023-11-15T03:05:51.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.486+0000 7f4a720e3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a6c157d90 0x7f4a6c15e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:51.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.486+0000 7f4a720e3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a6c157d90 0x7f4a6c15e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45736/0 (socket says 172.21.15.121:45736) 2023-11-15T03:05:51.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.486+0000 7f4a720e3700 1 -- 172.21.15.121:0/1658663337 learned_addr learned my addr 172.21.15.121:0/1658663337 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:51.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.486+0000 7f4a710e1700 1 --2- 172.21.15.121:0/1658663337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a6c0a8750 0x7f4a6c15de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:51.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.486+0000 7f4a718e2700 1 --2- 172.21.15.121:0/1658663337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a6c0669b0 0x7f4a6c15a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:51.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.486+0000 7f4a720e3700 1 -- 172.21.15.121:0/1658663337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a6c0a8750 msgr2=0x7f4a6c15de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:51.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.486+0000 7f4a720e3700 1 --2- 172.21.15.121:0/1658663337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a6c0a8750 0x7f4a6c15de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:51.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.486+0000 7f4a720e3700 1 -- 172.21.15.121:0/1658663337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a6c0669b0 msgr2=0x7f4a6c15a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:51.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.486+0000 7f4a720e3700 1 --2- 172.21.15.121:0/1658663337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a6c0669b0 0x7f4a6c15a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:51.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.486+0000 7f4a720e3700 1 -- 172.21.15.121:0/1658663337 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a68009370 con 0x7f4a6c157d90 2023-11-15T03:05:51.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.486+0000 7f4a710e1700 1 --2- 172.21.15.121:0/1658663337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a6c0a8750 0x7f4a6c15de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:51.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.486+0000 7f4a720e3700 1 --2- 172.21.15.121:0/1658663337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a6c157d90 0x7f4a6c15e310 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7f4a64009fd0 tx=0x7f4a64009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:51.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.486+0000 7f4a61ffb700 1 -- 172.21.15.121:0/1658663337 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6400eca0 con 0x7f4a6c157d90 2023-11-15T03:05:51.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.486+0000 7f4a61ffb700 1 -- 172.21.15.121:0/1658663337 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6400ee00 con 0x7f4a6c157d90 2023-11-15T03:05:51.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.486+0000 7f4a61ffb700 1 -- 172.21.15.121:0/1658663337 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a64014610 con 0x7f4a6c157d90 2023-11-15T03:05:51.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.486+0000 7f4a728e4700 1 -- 172.21.15.121:0/1658663337 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a6c0aa950 con 0x7f4a6c157d90 2023-11-15T03:05:51.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.486+0000 7f4a728e4700 1 -- 172.21.15.121:0/1658663337 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6c0aaf10 con 0x7f4a6c157d90 2023-11-15T03:05:51.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.490+0000 7f4a728e4700 1 -- 172.21.15.121:0/1658663337 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a6c06bcf0 con 0x7f4a6c157d90 2023-11-15T03:05:51.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.490+0000 7f4a61ffb700 1 -- 172.21.15.121:0/1658663337 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4a6401d3b0 con 0x7f4a6c157d90 2023-11-15T03:05:51.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.490+0000 7f4a61ffb700 1 --2- 172.21.15.121:0/1658663337 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a5005a510 0x7f4a5005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:51.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.490+0000 7f4a61ffb700 1 -- 172.21.15.121:0/1658663337 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7298+0+0 (secure 0 0 0) 0x7f4a64075170 con 0x7f4a6c157d90 2023-11-15T03:05:51.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.490+0000 7f4a718e2700 1 --2- 172.21.15.121:0/1658663337 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a5005a510 0x7f4a5005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:51.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.494+0000 7f4a718e2700 1 --2- 172.21.15.121:0/1658663337 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a5005a510 0x7f4a5005c9d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f4a68009130 tx=0x7f4a68017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:51.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.498+0000 7f4a61ffb700 1 -- 172.21.15.121:0/1658663337 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4a64048390 con 0x7f4a6c157d90 2023-11-15T03:05:51.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:51.694+0000 7f4a728e4700 1 -- 172.21.15.121:0/1658663337 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4a6c0c3c70 con 0x7f4a6c157d90 2023-11-15T03:05:52.107 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:05:52.094+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:05:52.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.110+0000 7f4a61ffb700 1 -- 172.21.15.121:0/1658663337 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v163) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4a6404b9b0 con 0x7f4a6c157d90 2023-11-15T03:05:52.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.114+0000 7f4a728e4700 1 -- 172.21.15.121:0/1658663337 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a5005a510 msgr2=0x7f4a5005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:52.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.114+0000 7f4a728e4700 1 --2- 172.21.15.121:0/1658663337 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a5005a510 0x7f4a5005c9d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f4a68009130 tx=0x7f4a68017000).stop 2023-11-15T03:05:52.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.114+0000 7f4a728e4700 1 -- 172.21.15.121:0/1658663337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a6c157d90 msgr2=0x7f4a6c15e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:52.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.114+0000 7f4a728e4700 1 --2- 172.21.15.121:0/1658663337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a6c157d90 0x7f4a6c15e310 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7f4a64009fd0 tx=0x7f4a64009900).stop 2023-11-15T03:05:52.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.114+0000 7f4a728e4700 1 -- 172.21.15.121:0/1658663337 shutdown_connections 2023-11-15T03:05:52.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.114+0000 7f4a728e4700 1 --2- 172.21.15.121:0/1658663337 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a5005a510 0x7f4a5005c9d0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:52.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.114+0000 7f4a728e4700 1 --2- 172.21.15.121:0/1658663337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a6c157d90 0x7f4a6c15e310 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:52.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.114+0000 7f4a728e4700 1 --2- 172.21.15.121:0/1658663337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a6c0a8750 0x7f4a6c15de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:52.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.114+0000 7f4a728e4700 1 --2- 172.21.15.121:0/1658663337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a6c0669b0 0x7f4a6c15a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:52.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.114+0000 7f4a728e4700 1 -- 172.21.15.121:0/1658663337 >> 172.21.15.121:0/1658663337 conn(0x7f4a6c067a60 msgr2=0x7f4a6c0c5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:52.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.114+0000 7f4a728e4700 1 -- 172.21.15.121:0/1658663337 shutdown_connections 2023-11-15T03:05:52.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.114+0000 7f4a728e4700 1 -- 172.21.15.121:0/1658663337 wait complete. 2023-11-15T03:05:52.126 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:05:52.146 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:05:52.169 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:05:52.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.302+0000 7f9157cf1700 1 Processor -- start 2023-11-15T03:05:52.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.302+0000 7f9157cf1700 1 -- start start 2023-11-15T03:05:52.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.302+0000 7f9157cf1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9150157e30 0x7f9150158210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:52.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.302+0000 7f9157cf1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9150062190 0x7f91500625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:52.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.302+0000 7f9157cf1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f91500660e0 0x7f9150066550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:52.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.302+0000 7f9157cf1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91500aaf80 con 0x7f91500660e0 2023-11-15T03:05:52.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.302+0000 7f9157cf1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9150062ae0 con 0x7f9150157e30 2023-11-15T03:05:52.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.302+0000 7f9157cf1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91500aa970 con 0x7f9150062190 2023-11-15T03:05:52.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.302+0000 7f9155a8d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9150157e30 0x7f9150158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:52.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.302+0000 7f9155a8d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9150157e30 0x7f9150158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:44104/0 (socket says 172.21.15.121:44104) 2023-11-15T03:05:52.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.302+0000 7f9155a8d700 1 -- 172.21.15.121:0/2898610700 learned_addr learned my addr 172.21.15.121:0/2898610700 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:52.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.302+0000 7f915628e700 1 --2- 172.21.15.121:0/2898610700 >> v2:172.21.15.121:3300/0 conn(0x7f91500660e0 0x7f9150066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:52.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.302+0000 7f915528c700 1 --2- 172.21.15.121:0/2898610700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9150062190 0x7f91500625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:52.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.302+0000 7f9155a8d700 1 -- 172.21.15.121:0/2898610700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9150062190 msgr2=0x7f91500625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:52.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.302+0000 7f9155a8d700 1 --2- 172.21.15.121:0/2898610700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9150062190 0x7f91500625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:52.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.306+0000 7f9155a8d700 1 -- 172.21.15.121:0/2898610700 >> v2:172.21.15.121:3300/0 conn(0x7f91500660e0 msgr2=0x7f9150066550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:52.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.306+0000 7f9155a8d700 1 --2- 172.21.15.121:0/2898610700 >> v2:172.21.15.121:3300/0 conn(0x7f91500660e0 0x7f9150066550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:52.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.306+0000 7f9155a8d700 1 -- 172.21.15.121:0/2898610700 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9150066e10 con 0x7f9150157e30 2023-11-15T03:05:52.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.306+0000 7f9155a8d700 1 --2- 172.21.15.121:0/2898610700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9150157e30 0x7f9150158210 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f9150063390 tx=0x7f914c0090f0).ready entity=mon.1 client_cookie=1a3682cbf8b08f82 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:52.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.306+0000 7f9154a8b700 1 -- 172.21.15.121:0/2898610700 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c00f510 con 0x7f9150157e30 2023-11-15T03:05:52.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.306+0000 7f9154a8b700 1 -- 172.21.15.121:0/2898610700 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914c00f670 con 0x7f9150157e30 2023-11-15T03:05:52.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.306+0000 7f9154a8b700 1 -- 172.21.15.121:0/2898610700 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c00f9f0 con 0x7f9150157e30 2023-11-15T03:05:52.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.306+0000 7f9157cf1700 1 -- 172.21.15.121:0/2898610700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9150157e30 msgr2=0x7f9150158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:52.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.306+0000 7f9157cf1700 1 --2- 172.21.15.121:0/2898610700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9150157e30 0x7f9150158210 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f9150063390 tx=0x7f914c0090f0).stop 2023-11-15T03:05:52.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.306+0000 7f9157cf1700 1 -- 172.21.15.121:0/2898610700 shutdown_connections 2023-11-15T03:05:52.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.306+0000 7f9157cf1700 1 --2- 172.21.15.121:0/2898610700 >> v2:172.21.15.121:3300/0 conn(0x7f91500660e0 0x7f9150066550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:52.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.306+0000 7f9157cf1700 1 --2- 172.21.15.121:0/2898610700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9150062190 0x7f91500625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:52.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.306+0000 7f9157cf1700 1 --2- 172.21.15.121:0/2898610700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9150157e30 0x7f9150158210 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:52.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.306+0000 7f9157cf1700 1 -- 172.21.15.121:0/2898610700 >> 172.21.15.121:0/2898610700 conn(0x7f9150067a60 msgr2=0x7f9150067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:52.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.306+0000 7f9157cf1700 1 -- 172.21.15.121:0/2898610700 shutdown_connections 2023-11-15T03:05:52.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.306+0000 7f9157cf1700 1 -- 172.21.15.121:0/2898610700 wait complete. 2023-11-15T03:05:52.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.310+0000 7f9157cf1700 1 Processor -- start 2023-11-15T03:05:52.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.310+0000 7f9157cf1700 1 -- start start 2023-11-15T03:05:52.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.310+0000 7f9157cf1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9150062190 0x7f915013e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:52.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.310+0000 7f9157cf1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91500660e0 0x7f9150158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:52.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.310+0000 7f9157cf1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9150157e30 0x7f9150158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:52.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.310+0000 7f9157cf1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9150158f70 con 0x7f9150062190 2023-11-15T03:05:52.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.310+0000 7f9157cf1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9150168510 con 0x7f9150157e30 2023-11-15T03:05:52.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.310+0000 7f9157cf1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9150168da0 con 0x7f91500660e0 2023-11-15T03:05:52.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.310+0000 7f9155a8d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9150062190 0x7f915013e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:52.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.310+0000 7f9155a8d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9150062190 0x7f915013e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45778/0 (socket says 172.21.15.121:45778) 2023-11-15T03:05:52.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.310+0000 7f9155a8d700 1 -- 172.21.15.121:0/3440770349 learned_addr learned my addr 172.21.15.121:0/3440770349 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:52.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.310+0000 7f915528c700 1 --2- 172.21.15.121:0/3440770349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91500660e0 0x7f9150158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:52.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.310+0000 7f915628e700 1 --2- 172.21.15.121:0/3440770349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9150157e30 0x7f9150158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:52.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.314+0000 7f9155a8d700 1 -- 172.21.15.121:0/3440770349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91500660e0 msgr2=0x7f9150158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:52.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.314+0000 7f9155a8d700 1 --2- 172.21.15.121:0/3440770349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91500660e0 0x7f9150158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:52.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.314+0000 7f9155a8d700 1 -- 172.21.15.121:0/3440770349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9150157e30 msgr2=0x7f9150158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:52.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.314+0000 7f9155a8d700 1 --2- 172.21.15.121:0/3440770349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9150157e30 0x7f9150158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:52.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.314+0000 7f9155a8d700 1 -- 172.21.15.121:0/3440770349 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914c009370 con 0x7f9150062190 2023-11-15T03:05:52.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.314+0000 7f9155a8d700 1 --2- 172.21.15.121:0/3440770349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9150062190 0x7f915013e1b0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f914c006fd0 tx=0x7f914c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:52.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.314+0000 7f9145ffb700 1 -- 172.21.15.121:0/3440770349 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c00f7e0 con 0x7f9150062190 2023-11-15T03:05:52.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.314+0000 7f9145ffb700 1 -- 172.21.15.121:0/3440770349 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914c00f940 con 0x7f9150062190 2023-11-15T03:05:52.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.314+0000 7f9157cf1700 1 -- 172.21.15.121:0/3440770349 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9150158c90 con 0x7f9150062190 2023-11-15T03:05:52.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.314+0000 7f9145ffb700 1 -- 172.21.15.121:0/3440770349 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c01c670 con 0x7f9150062190 2023-11-15T03:05:52.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.314+0000 7f9157cf1700 1 -- 172.21.15.121:0/3440770349 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9150168a30 con 0x7f9150062190 2023-11-15T03:05:52.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.314+0000 7f9157cf1700 1 -- 172.21.15.121:0/3440770349 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9150047060 con 0x7f9150062190 2023-11-15T03:05:52.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.318+0000 7f9145ffb700 1 -- 172.21.15.121:0/3440770349 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f914c01ca60 con 0x7f9150062190 2023-11-15T03:05:52.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.318+0000 7f9145ffb700 1 --2- 172.21.15.121:0/3440770349 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f913405a510 0x7f913405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:52.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.318+0000 7f9145ffb700 1 -- 172.21.15.121:0/3440770349 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7384+0+0 (secure 0 0 0) 0x7f914c0765c0 con 0x7f9150062190 2023-11-15T03:05:52.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.318+0000 7f915528c700 1 --2- 172.21.15.121:0/3440770349 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f913405a510 0x7f913405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:52.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.318+0000 7f915528c700 1 --2- 172.21.15.121:0/3440770349 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f913405a510 0x7f913405c9d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f915015d990 tx=0x7f9140009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:52.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.326+0000 7f9145ffb700 1 -- 172.21.15.121:0/3440770349 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f914c0498b0 con 0x7f9150062190 2023-11-15T03:05:52.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:52.546+0000 7f9157cf1700 1 -- 172.21.15.121:0/3440770349 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f9150068b20 con 0x7f9150062190 2023-11-15T03:05:53.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.126+0000 7f9145ffb700 1 -- 172.21.15.121:0/3440770349 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v164) v1 ==== 107+0+0 (secure 0 0 0) 0x7f914c04ced0 con 0x7f9150062190 2023-11-15T03:05:53.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.130+0000 7f9157cf1700 1 -- 172.21.15.121:0/3440770349 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f913405a510 msgr2=0x7f913405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.130+0000 7f9157cf1700 1 --2- 172.21.15.121:0/3440770349 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f913405a510 0x7f913405c9d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f915015d990 tx=0x7f9140009060).stop 2023-11-15T03:05:53.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.130+0000 7f9157cf1700 1 -- 172.21.15.121:0/3440770349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9150062190 msgr2=0x7f915013e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.130+0000 7f9157cf1700 1 --2- 172.21.15.121:0/3440770349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9150062190 0x7f915013e1b0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f914c006fd0 tx=0x7f914c017000).stop 2023-11-15T03:05:53.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.134+0000 7f9157cf1700 1 -- 172.21.15.121:0/3440770349 shutdown_connections 2023-11-15T03:05:53.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.134+0000 7f9157cf1700 1 --2- 172.21.15.121:0/3440770349 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f913405a510 0x7f913405c9d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.134+0000 7f9157cf1700 1 --2- 172.21.15.121:0/3440770349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9150157e30 0x7f9150158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.134+0000 7f9157cf1700 1 --2- 172.21.15.121:0/3440770349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91500660e0 0x7f9150158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.134+0000 7f9157cf1700 1 --2- 172.21.15.121:0/3440770349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9150062190 0x7f915013e1b0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.134+0000 7f9157cf1700 1 -- 172.21.15.121:0/3440770349 >> 172.21.15.121:0/3440770349 conn(0x7f9150067a60 msgr2=0x7f9150152780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:53.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.134+0000 7f9157cf1700 1 -- 172.21.15.121:0/3440770349 shutdown_connections 2023-11-15T03:05:53.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.134+0000 7f9157cf1700 1 -- 172.21.15.121:0/3440770349 wait complete. 2023-11-15T03:05:53.145 INFO:teuthology.orchestra.run.smithi121.stderr:nothing to do: rank is not damaged 2023-11-15T03:05:53.162 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:05:53.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.254+0000 7f6cf33dad00 1 Processor -- start 2023-11-15T03:05:53.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.254+0000 7f6cf33dad00 1 -- start start 2023-11-15T03:05:53.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cf33dad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c97e95bdd0 0x55c97e9a8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cf33dad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c97e9a5b00 0x55c97ea40e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cf33dad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c97ea38650 0x55c97ea41350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cf33dad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c97e909cc0 con 0x55c97e9a5b00 2023-11-15T03:05:53.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cf33dad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c97e8d4220 con 0x55c97ea38650 2023-11-15T03:05:53.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cf33dad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c97e98e070 con 0x55c97e95bdd0 2023-11-15T03:05:53.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6ce880b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c97e9a5b00 0x55c97ea40e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6ce880b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c97e9a5b00 0x55c97ea40e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45790/0 (socket says 172.21.15.121:45790) 2023-11-15T03:05:53.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6ce880b700 1 -- 172.21.15.121:0/2725033747 learned_addr learned my addr 172.21.15.121:0/2725033747 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:53.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cf100c700 1 --2- 172.21.15.121:0/2725033747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c97e95bdd0 0x55c97e9a8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cf180d700 1 --2- 172.21.15.121:0/2725033747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c97ea38650 0x55c97ea41350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6ce880b700 1 -- 172.21.15.121:0/2725033747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c97e95bdd0 msgr2=0x55c97e9a8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:53.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6ce880b700 1 --2- 172.21.15.121:0/2725033747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c97e95bdd0 0x55c97e9a8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6ce880b700 1 -- 172.21.15.121:0/2725033747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c97ea38650 msgr2=0x55c97ea41350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:53.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6ce880b700 1 --2- 172.21.15.121:0/2725033747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c97ea38650 0x55c97ea41350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6ce880b700 1 -- 172.21.15.121:0/2725033747 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec005cf0 con 0x55c97e9a5b00 2023-11-15T03:05:53.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6ce880b700 1 --2- 172.21.15.121:0/2725033747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c97e9a5b00 0x55c97ea40e60 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f6cdc001d00 tx=0x7f6cdc001780).ready entity=mon.0 client_cookie=a9d1abdb618fbb55 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:53.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cebfff700 1 -- 172.21.15.121:0/2725033747 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc00bc80 con 0x55c97e9a5b00 2023-11-15T03:05:53.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cebfff700 1 -- 172.21.15.121:0/2725033747 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc00bde0 con 0x55c97e9a5b00 2023-11-15T03:05:53.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cebfff700 1 -- 172.21.15.121:0/2725033747 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc00e6a0 con 0x55c97e9a5b00 2023-11-15T03:05:53.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cf33dad00 1 -- 172.21.15.121:0/2725033747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c97e9a5b00 msgr2=0x55c97ea40e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/2725033747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c97e9a5b00 0x55c97ea40e60 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f6cdc001d00 tx=0x7f6cdc001780).stop 2023-11-15T03:05:53.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cf33dad00 1 -- 172.21.15.121:0/2725033747 shutdown_connections 2023-11-15T03:05:53.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/2725033747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c97ea38650 0x55c97ea41350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/2725033747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c97e9a5b00 0x55c97ea40e60 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/2725033747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c97e95bdd0 0x55c97e9a8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cf33dad00 1 -- 172.21.15.121:0/2725033747 >> 172.21.15.121:0/2725033747 conn(0x55c97e9ab080 msgr2=0x55c97ea36a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:53.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cf33dad00 1 -- 172.21.15.121:0/2725033747 shutdown_connections 2023-11-15T03:05:53.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.258+0000 7f6cf33dad00 1 -- 172.21.15.121:0/2725033747 wait complete. 2023-11-15T03:05:53.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.262+0000 7f6cf33dad00 1 Processor -- start 2023-11-15T03:05:53.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.262+0000 7f6cf33dad00 1 -- start start 2023-11-15T03:05:53.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.262+0000 7f6cf33dad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c97e95bdd0 0x55c97e9a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.262+0000 7f6cf33dad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c97e9a5b00 0x55c97ea2c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.262+0000 7f6cf33dad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c97ea38650 0x55c97ea2ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.262+0000 7f6cf33dad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c97ea2d350 con 0x55c97ea38650 2023-11-15T03:05:53.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.262+0000 7f6cf33dad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c97ea39590 con 0x55c97e95bdd0 2023-11-15T03:05:53.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.262+0000 7f6cf33dad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c97e98db70 con 0x55c97e9a5b00 2023-11-15T03:05:53.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.262+0000 7f6cf180d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c97ea38650 0x55c97ea2ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.262+0000 7f6cf100c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c97e95bdd0 0x55c97e9a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.262+0000 7f6cf180d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c97ea38650 0x55c97ea2ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45800/0 (socket says 172.21.15.121:45800) 2023-11-15T03:05:53.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.262+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 learned_addr learned my addr 172.21.15.121:0/177414619 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:53.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.262+0000 7f6ce880b700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c97e9a5b00 0x55c97ea2c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.266+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c97e9a5b00 msgr2=0x55c97ea2c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.266+0000 7f6cf180d700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c97e9a5b00 0x55c97ea2c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.266+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c97e95bdd0 msgr2=0x55c97e9a47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.266+0000 7f6cf180d700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c97e95bdd0 0x55c97e9a47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.266+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce0001e10 con 0x55c97ea38650 2023-11-15T03:05:53.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.266+0000 7f6cf180d700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c97ea38650 0x55c97ea2ce60 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 rx=0x7f6cec010000 tx=0x7f6cec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:53.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.266+0000 7f6cea7fc700 1 -- 172.21.15.121:0/177414619 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec008250 con 0x55c97ea38650 2023-11-15T03:05:53.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.266+0000 7f6cea7fc700 1 -- 172.21.15.121:0/177414619 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cec01e480 con 0x55c97ea38650 2023-11-15T03:05:53.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.266+0000 7f6cea7fc700 1 -- 172.21.15.121:0/177414619 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec01e820 con 0x55c97ea38650 2023-11-15T03:05:53.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.266+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c97e8cca70 con 0x55c97ea38650 2023-11-15T03:05:53.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.266+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c97e8d8010 con 0x55c97ea38650 2023-11-15T03:05:53.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.270+0000 7f6cea7fc700 1 -- 172.21.15.121:0/177414619 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6cec01ebd0 con 0x55c97ea38650 2023-11-15T03:05:53.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.270+0000 7f6cea7fc700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6cd4059020 0x7f6cd405b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.270+0000 7f6cea7fc700 1 -- 172.21.15.121:0/177414619 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7384+0+0 (secure 0 0 0) 0x7f6cec0756d0 con 0x55c97ea38650 2023-11-15T03:05:53.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.270+0000 7f6cf100c700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6cd4059020 0x7f6cd405b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.282 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:05:53.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.270+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c97ea39a70 0x55c97ea39e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.270+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea3a360 con 0x55c97ea39a70 2023-11-15T03:05:53.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.270+0000 7f6ce880b700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c97ea39a70 0x55c97ea39e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.270+0000 7f6cf100c700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6cd4059020 0x7f6cd405b4e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f6ce4006fd0 tx=0x7f6ce4008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:53.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.270+0000 7f6ce880b700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c97ea39a70 0x55c97ea39e70 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:53.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.274+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v107'149 uv142 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea39a70 2023-11-15T03:05:53.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.274+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea3a710 con 0x55c97ea39a70 2023-11-15T03:05:53.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.278+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v107'150 uv150 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea39a70 2023-11-15T03:05:53.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.278+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea3aac0 con 0x55c97ea39a70 2023-11-15T03:05:53.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.278+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v107'151 uv151 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea39a70 2023-11-15T03:05:53.289 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:05:53.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.278+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea51e60 con 0x55c97ea39a70 2023-11-15T03:05:53.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.282+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v107'152 uv152 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea39a70 2023-11-15T03:05:53.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.282+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea52210 con 0x55c97ea39a70 2023-11-15T03:05:53.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.286+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v107'153 uv153 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea39a70 2023-11-15T03:05:53.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.286+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea50e50 con 0x55c97ea39a70 2023-11-15T03:05:53.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.286+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v107'154 uv154 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea39a70 2023-11-15T03:05:53.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.286+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea51200 con 0x55c97ea39a70 2023-11-15T03:05:53.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.290+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v107'155 uv148 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea39a70 2023-11-15T03:05:53.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.290+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea51600 con 0x55c97ea39a70 2023-11-15T03:05:53.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.298+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v107'156 uv156 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea39a70 2023-11-15T03:05:53.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.298+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea51600 con 0x55c97ea39a70 2023-11-15T03:05:53.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.298+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v107'157 uv157 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea39a70 2023-11-15T03:05:53.308 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:05:53.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.298+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea51600 con 0x55c97ea39a70 2023-11-15T03:05:53.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.302+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v107'158 uv158 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea39a70 2023-11-15T03:05:53.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.302+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea51600 con 0x55c97ea39a70 2023-11-15T03:05:53.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.302+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v107'159 uv159 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea39a70 2023-11-15T03:05:53.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.302+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea51600 con 0x55c97ea39a70 2023-11-15T03:05:53.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.306+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v107'160 uv160 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea39a70 2023-11-15T03:05:53.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.306+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c97ea51a00 0x55c97ea54ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.306+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea51600 con 0x55c97ea51a00 2023-11-15T03:05:53.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.306+0000 7f6cf100c700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c97ea51a00 0x55c97ea54ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.310+0000 7f6cf100c700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c97ea51a00 0x55c97ea54ae0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:53.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.310+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v107'233 uv220 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.314+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea54fd0 con 0x55c97ea51a00 2023-11-15T03:05:53.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.314+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v107'234 uv234 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.314+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea55380 con 0x55c97ea51a00 2023-11-15T03:05:53.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.318+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v107'235 uv235 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.329 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:05:53.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.318+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea56740 con 0x55c97ea51a00 2023-11-15T03:05:53.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.322+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v107'236 uv236 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.322+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea56af0 con 0x55c97ea51a00 2023-11-15T03:05:53.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.326+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v107'237 uv237 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.326+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea55730 con 0x55c97ea51a00 2023-11-15T03:05:53.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.326+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v107'238 uv238 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.326+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea56040 con 0x55c97ea51a00 2023-11-15T03:05:53.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.330+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v107'239 uv239 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.330+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea56020 con 0x55c97ea51a00 2023-11-15T03:05:53.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.334+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v107'240 uv226 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.334+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea56020 con 0x55c97ea51a00 2023-11-15T03:05:53.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.338+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v107'241 uv241 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.338+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea56020 con 0x55c97ea51a00 2023-11-15T03:05:53.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.338+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v107'242 uv242 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.349 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:05:53.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.338+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea56020 con 0x55c97ea51a00 2023-11-15T03:05:53.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.342+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v107'243 uv243 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.342+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea56020 con 0x55c97ea51a00 2023-11-15T03:05:53.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.346+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v107'244 uv244 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.346+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea56020 con 0x55c97ea51a00 2023-11-15T03:05:53.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.346+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v107'245 uv245 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.346+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea56020 con 0x55c97ea51a00 2023-11-15T03:05:53.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.350+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v107'246 uv232 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.350+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea56020 con 0x55c97ea51a00 2023-11-15T03:05:53.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.354+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v107'247 uv247 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.354+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea56020 con 0x55c97ea51a00 2023-11-15T03:05:53.365 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:05:53.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.354+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v107'248 uv248 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.354+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea56020 con 0x55c97ea51a00 2023-11-15T03:05:53.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.358+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v107'249 uv249 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.358+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea56020 con 0x55c97ea51a00 2023-11-15T03:05:53.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.362+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v107'250 uv250 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.362+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea56020 con 0x55c97ea51a00 2023-11-15T03:05:53.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.366+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v107'251 uv251 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.366+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c97ea573b0 0x55c97ea597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.366+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea56020 con 0x55c97ea573b0 2023-11-15T03:05:53.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.366+0000 7f6ce880b700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c97ea573b0 0x55c97ea597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.366+0000 7f6ce880b700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c97ea573b0 0x55c97ea597a0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:53.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.370+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v107'128 uv121 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea573b0 2023-11-15T03:05:53.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.370+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea59c90 con 0x55c97ea573b0 2023-11-15T03:05:53.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.374+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v107'129 uv129 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea573b0 2023-11-15T03:05:53.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.374+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea59c90 con 0x55c97ea573b0 2023-11-15T03:05:53.385 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:05:53.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.378+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v107'130 uv130 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea573b0 2023-11-15T03:05:53.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.378+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5a180 con 0x55c97ea573b0 2023-11-15T03:05:53.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.378+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v107'131 uv131 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea573b0 2023-11-15T03:05:53.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.378+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5a180 con 0x55c97ea573b0 2023-11-15T03:05:53.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.382+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v107'132 uv132 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea573b0 2023-11-15T03:05:53.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.382+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5a180 con 0x55c97ea573b0 2023-11-15T03:05:53.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.386+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v107'133 uv127 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea573b0 2023-11-15T03:05:53.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.386+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea5a180 con 0x55c97ea573b0 2023-11-15T03:05:53.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.386+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v107'134 uv134 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea573b0 2023-11-15T03:05:53.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.386+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5a180 con 0x55c97ea573b0 2023-11-15T03:05:53.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.390+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v107'135 uv135 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea573b0 2023-11-15T03:05:53.399 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:05:53.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.390+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5a180 con 0x55c97ea573b0 2023-11-15T03:05:53.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.394+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v107'136 uv136 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea573b0 2023-11-15T03:05:53.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.394+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5a180 con 0x55c97ea573b0 2023-11-15T03:05:53.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.394+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v107'137 uv137 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea573b0 2023-11-15T03:05:53.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.394+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c97ea5a690 0x55c97ea5ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.394+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5cf70 con 0x55c97ea5a690 2023-11-15T03:05:53.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.398+0000 7f6cf180d700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c97ea5a690 0x55c97ea5ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.398+0000 7f6cf180d700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c97ea5a690 0x55c97ea5ca80 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:53.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.402+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v107'159 uv158 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.402+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea5d320 con 0x55c97ea5a690 2023-11-15T03:05:53.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.402+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v107'160 uv160 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.402+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5d6d0 con 0x55c97ea5a690 2023-11-15T03:05:53.416 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:05:53.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.406+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v107'161 uv161 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.406+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea59d10 con 0x55c97ea5a690 2023-11-15T03:05:53.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.410+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v107'162 uv162 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.410+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea62100 con 0x55c97ea5a690 2023-11-15T03:05:53.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.414+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v107'163 uv163 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.414+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5da80 con 0x55c97ea5a690 2023-11-15T03:05:53.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.418+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v107'164 uv155 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.418+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea5de80 con 0x55c97ea5a690 2023-11-15T03:05:53.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.418+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v107'165 uv165 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.418+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5e770 con 0x55c97ea5a690 2023-11-15T03:05:53.432 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:05:53.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.422+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v107'166 uv166 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.422+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5fb80 con 0x55c97ea5a690 2023-11-15T03:05:53.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.426+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v107'167 uv167 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.426+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5e720 con 0x55c97ea5a690 2023-11-15T03:05:53.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.430+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v107'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.430+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5eb20 con 0x55c97ea5a690 2023-11-15T03:05:53.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.430+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v107'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.434+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c97ea5f5c0 0x55c97ea619b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.434+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5eb20 con 0x55c97ea5f5c0 2023-11-15T03:05:53.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.434+0000 7f6cf100c700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c97ea5f5c0 0x55c97ea619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.434+0000 7f6cf100c700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c97ea5f5c0 0x55c97ea619b0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:53.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.438+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v107'75 uv74 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea5f5c0 2023-11-15T03:05:53.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.438+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea5eb20 con 0x55c97ea5f5c0 2023-11-15T03:05:53.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.438+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v107'76 uv76 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea5f5c0 2023-11-15T03:05:53.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.438+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5eb20 con 0x55c97ea5f5c0 2023-11-15T03:05:53.451 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:05:53.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.442+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v107'77 uv77 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea5f5c0 2023-11-15T03:05:53.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.442+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5eb20 con 0x55c97ea5f5c0 2023-11-15T03:05:53.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.446+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v107'78 uv78 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea5f5c0 2023-11-15T03:05:53.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.446+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5eb20 con 0x55c97ea5f5c0 2023-11-15T03:05:53.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.446+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v107'79 uv79 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea5f5c0 2023-11-15T03:05:53.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.446+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5eb20 con 0x55c97ea5f5c0 2023-11-15T03:05:53.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.450+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v107'80 uv80 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea5f5c0 2023-11-15T03:05:53.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.450+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c97ea629c0 0x55c97ea64db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.450+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea652c0 con 0x55c97ea629c0 2023-11-15T03:05:53.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.450+0000 7f6ce880b700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c97ea629c0 0x55c97ea64db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.450+0000 7f6ce880b700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c97ea629c0 0x55c97ea64db0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:53.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.454+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v107'56 uv55 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.454+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea656a0 con 0x55c97ea629c0 2023-11-15T03:05:53.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.458+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v107'57 uv57 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.458+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea656a0 con 0x55c97ea629c0 2023-11-15T03:05:53.469 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:05:53.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.462+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v107'58 uv58 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.462+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea656a0 con 0x55c97ea629c0 2023-11-15T03:05:53.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.466+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v107'59 uv59 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.466+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea656a0 con 0x55c97ea629c0 2023-11-15T03:05:53.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.466+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v107'75 uv74 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.466+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea656a0 con 0x55c97ea629c0 2023-11-15T03:05:53.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.474+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v107'76 uv76 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.474+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea656a0 con 0x55c97ea629c0 2023-11-15T03:05:53.484 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:05:53.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.474+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v107'77 uv77 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.474+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea666b0 con 0x55c97ea629c0 2023-11-15T03:05:53.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.478+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v107'78 uv78 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.478+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea666b0 con 0x55c97ea629c0 2023-11-15T03:05:53.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.482+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v107'79 uv79 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.482+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea656a0 con 0x55c97ea629c0 2023-11-15T03:05:53.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.486+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v107'80 uv80 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.486+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea656a0 con 0x55c97ea51a00 2023-11-15T03:05:53.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.486+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v107'138 uv132 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.486+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea656a0 con 0x55c97ea51a00 2023-11-15T03:05:53.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.490+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v107'139 uv139 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.490+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea656a0 con 0x55c97ea51a00 2023-11-15T03:05:53.503 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:05:53.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.494+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v107'140 uv140 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.494+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea666b0 con 0x55c97ea51a00 2023-11-15T03:05:53.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.498+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v107'141 uv141 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.498+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea666b0 con 0x55c97ea51a00 2023-11-15T03:05:53.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.498+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v107'142 uv142 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.498+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea656a0 con 0x55c97ea51a00 2023-11-15T03:05:53.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.502+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v107'143 uv143 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.502+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea656a0 con 0x55c97ea51a00 2023-11-15T03:05:53.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.506+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v107'144 uv137 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.506+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea656a0 con 0x55c97ea51a00 2023-11-15T03:05:53.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.506+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v107'145 uv145 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.506+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea656a0 con 0x55c97ea51a00 2023-11-15T03:05:53.519 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:05:53.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.510+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v107'146 uv146 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.510+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea656a0 con 0x55c97ea51a00 2023-11-15T03:05:53.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.514+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v107'147 uv147 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.514+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5e2b0 con 0x55c97ea51a00 2023-11-15T03:05:53.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.518+0000 7f6cf100c700 1 -- 172.21.15.121:0/177414619 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v107'148 uv148 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6ce4015800 con 0x55c97ea51a00 2023-11-15T03:05:53.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.518+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5e2b0 con 0x55c97ea573b0 2023-11-15T03:05:53.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.518+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v107'61 uv60 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea573b0 2023-11-15T03:05:53.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.518+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea5e2b0 con 0x55c97ea573b0 2023-11-15T03:05:53.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.522+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v107'62 uv62 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea573b0 2023-11-15T03:05:53.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.522+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5e2b0 con 0x55c97ea573b0 2023-11-15T03:05:53.534 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:05:53.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.522+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v107'63 uv63 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea573b0 2023-11-15T03:05:53.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.522+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5e2b0 con 0x55c97ea573b0 2023-11-15T03:05:53.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.526+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v107'64 uv64 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea573b0 2023-11-15T03:05:53.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.526+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5e370 con 0x55c97ea573b0 2023-11-15T03:05:53.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.530+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v107'65 uv65 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea573b0 2023-11-15T03:05:53.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.530+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5e330 con 0x55c97ea629c0 2023-11-15T03:05:53.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.530+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v107'75 uv74 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.530+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea5e2b0 con 0x55c97ea629c0 2023-11-15T03:05:53.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.534+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v107'76 uv76 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.534+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5e2b0 con 0x55c97ea629c0 2023-11-15T03:05:53.547 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:05:53.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.538+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v107'77 uv77 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.538+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5e2b0 con 0x55c97ea629c0 2023-11-15T03:05:53.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.542+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v107'78 uv78 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.542+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5e2b0 con 0x55c97ea629c0 2023-11-15T03:05:53.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.542+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v107'79 uv79 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.542+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5e2b0 con 0x55c97ea629c0 2023-11-15T03:05:53.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.546+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v107'80 uv80 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.546+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5e2b0 con 0x55c97ea629c0 2023-11-15T03:05:53.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.550+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v107'62 uv61 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.550+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea5e2b0 con 0x55c97ea629c0 2023-11-15T03:05:53.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.554+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v107'63 uv63 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.554+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5e2b0 con 0x55c97ea629c0 2023-11-15T03:05:53.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.554+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v107'64 uv64 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.565 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:05:53.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.554+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5e2b0 con 0x55c97ea629c0 2023-11-15T03:05:53.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.558+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v107'65 uv65 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.558+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea65be0 con 0x55c97ea629c0 2023-11-15T03:05:53.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.562+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v107'66 uv66 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.562+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea65be0 con 0x55c97ea629c0 2023-11-15T03:05:53.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.562+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v107'61 uv60 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.562+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea5e2b0 con 0x55c97ea629c0 2023-11-15T03:05:53.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.566+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v107'62 uv62 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.566+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea5e2b0 con 0x55c97ea629c0 2023-11-15T03:05:53.579 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:05:53.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.570+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v107'63 uv63 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.570+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea65b90 con 0x55c97ea629c0 2023-11-15T03:05:53.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.574+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v107'64 uv64 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.574+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea68630 con 0x55c97ea629c0 2023-11-15T03:05:53.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.574+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v107'65 uv65 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.578+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c97ea656a0 0x55c97ea68c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.578+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea67a90 con 0x55c97ea656a0 2023-11-15T03:05:53.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.578+0000 7f6cf180d700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c97ea656a0 0x55c97ea68c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.578+0000 7f6cf180d700 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c97ea656a0 0x55c97ea68c50 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:53.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.582+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v107'50 uv49 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea656a0 2023-11-15T03:05:53.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.582+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea67a90 con 0x55c97ea656a0 2023-11-15T03:05:53.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.582+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v107'51 uv51 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea656a0 2023-11-15T03:05:53.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.582+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea67a90 con 0x55c97ea656a0 2023-11-15T03:05:53.595 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:05:53.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.586+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v107'52 uv52 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea656a0 2023-11-15T03:05:53.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.586+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea67a90 con 0x55c97ea656a0 2023-11-15T03:05:53.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.590+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v107'53 uv53 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea656a0 2023-11-15T03:05:53.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.590+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea67a90 con 0x55c97ea629c0 2023-11-15T03:05:53.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.594+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v107'75 uv74 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.594+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea67a90 con 0x55c97ea629c0 2023-11-15T03:05:53.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.594+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v107'76 uv76 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.594+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea67a90 con 0x55c97ea629c0 2023-11-15T03:05:53.607 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:05:53.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.598+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v107'77 uv77 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.598+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea67a90 con 0x55c97ea629c0 2023-11-15T03:05:53.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.602+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v107'78 uv78 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.602+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea67a90 con 0x55c97ea629c0 2023-11-15T03:05:53.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.602+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v107'79 uv79 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.602+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea67a90 con 0x55c97ea629c0 2023-11-15T03:05:53.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.606+0000 7f6ce880b700 1 -- 172.21.15.121:0/177414619 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v107'80 uv80 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cdc000c30 con 0x55c97ea629c0 2023-11-15T03:05:53.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.606+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea67a90 con 0x55c97ea5a690 2023-11-15T03:05:53.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.610+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v107'64 uv63 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.610+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea67a90 con 0x55c97ea5a690 2023-11-15T03:05:53.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.610+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v107'65 uv65 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.610+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea67a90 con 0x55c97ea5a690 2023-11-15T03:05:53.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.614+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v107'66 uv66 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.623 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:05:53.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.614+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea69140 con 0x55c97ea5a690 2023-11-15T03:05:53.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.618+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v107'67 uv67 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.618+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea69140 con 0x55c97ea5a690 2023-11-15T03:05:53.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.618+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v107'68 uv68 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.618+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea69140 con 0x55c97ea5a690 2023-11-15T03:05:53.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.622+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v107'150 uv142 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.622+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea69140 con 0x55c97ea5a690 2023-11-15T03:05:53.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.626+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v107'151 uv151 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.626+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea69140 con 0x55c97ea5a690 2023-11-15T03:05:53.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.626+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v107'152 uv152 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.637 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:05:53.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.626+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea69140 con 0x55c97ea5a690 2023-11-15T03:05:53.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.630+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v107'153 uv153 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.630+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea69140 con 0x55c97ea5a690 2023-11-15T03:05:53.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.634+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v107'154 uv154 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.634+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea69140 con 0x55c97ea5a690 2023-11-15T03:05:53.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.634+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v107'155 uv149 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.634+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x55c97ea69140 con 0x55c97ea5a690 2023-11-15T03:05:53.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.638+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v107'156 uv156 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.638+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea69140 con 0x55c97ea5a690 2023-11-15T03:05:53.650 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:05:53.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.642+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v107'157 uv157 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.642+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea6a150 con 0x55c97ea5a690 2023-11-15T03:05:53.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.642+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v107'158 uv158 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.642+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea6a150 con 0x55c97ea5a690 2023-11-15T03:05:53.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.646+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v107'159 uv159 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.646+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea69140 con 0x55c97ea5a690 2023-11-15T03:05:53.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.650+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v107'160 uv160 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.650+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e107) v8 -- 0x55c97ea69630 con 0x55c97ea5a690 2023-11-15T03:05:53.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v107'161 uv161 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f6cec024020 con 0x55c97ea5a690 2023-11-15T03:05:53.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c97ea573b0 msgr2=0x55c97ea597a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c97ea573b0 0x55c97ea597a0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c97ea39a70 msgr2=0x55c97ea39e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c97ea39a70 0x55c97ea39e70 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c97ea5f5c0 msgr2=0x55c97ea619b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c97ea5f5c0 0x55c97ea619b0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c97ea51a00 msgr2=0x55c97ea54ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c97ea51a00 0x55c97ea54ae0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c97ea656a0 msgr2=0x55c97ea68c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c97ea656a0 0x55c97ea68c50 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 reap_dead start 2023-11-15T03:05:53.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c97ea5a690 msgr2=0x55c97ea5ca80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c97ea5a690 0x55c97ea5ca80 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c97ea629c0 msgr2=0x55c97ea64db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c97ea629c0 0x55c97ea64db0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6cd4059020 msgr2=0x7f6cd405b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6cd4059020 0x7f6cd405b4e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f6ce4006fd0 tx=0x7f6ce4008040).stop 2023-11-15T03:05:53.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c97ea38650 msgr2=0x55c97ea2ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 --2- 172.21.15.121:0/177414619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c97ea38650 0x55c97ea2ce60 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 rx=0x7f6cec010000 tx=0x7f6cec016000).stop 2023-11-15T03:05:53.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf180d700 1 -- 172.21.15.121:0/177414619 reap_dead start 2023-11-15T03:05:53.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 shutdown_connections 2023-11-15T03:05:53.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 >> 172.21.15.121:0/177414619 conn(0x55c97e9ab080 msgr2=0x55c97ea36a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:53.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 shutdown_connections 2023-11-15T03:05:53.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.654+0000 7f6cf33dad00 1 -- 172.21.15.121:0/177414619 wait complete. 2023-11-15T03:05:53.671 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100.00000000.inode 2023-11-15T03:05:53.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.766+0000 7fe0e0cd8d00 1 Processor -- start 2023-11-15T03:05:53.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.766+0000 7fe0e0cd8d00 1 -- start start 2023-11-15T03:05:53.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0e0cd8d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563267a34dd0 0x563267a34830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0e0cd8d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563267a7eb00 0x563267a7a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0e0cd8d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563267b0d670 0x563267b0c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0e0cd8d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632679ad220 con 0x563267a34dd0 2023-11-15T03:05:53.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0e0cd8d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563267b10c40 con 0x563267b0d670 2023-11-15T03:05:53.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0e0cd8d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x563267b0de00 con 0x563267a7eb00 2023-11-15T03:05:53.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0de510700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563267a34dd0 0x563267a34830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0de510700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563267a34dd0 0x563267a34830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45816/0 (socket says 172.21.15.121:45816) 2023-11-15T03:05:53.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0de510700 1 -- 172.21.15.121:0/2008528427 learned_addr learned my addr 172.21.15.121:0/2008528427 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0ded11700 1 --2- 172.21.15.121:0/2008528427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563267b0d670 0x563267b0c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0d5d0f700 1 --2- 172.21.15.121:0/2008528427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563267a7eb00 0x563267a7a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0de510700 1 -- 172.21.15.121:0/2008528427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563267a7eb00 msgr2=0x563267a7a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:53.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0de510700 1 --2- 172.21.15.121:0/2008528427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563267a7eb00 0x563267a7a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0de510700 1 -- 172.21.15.121:0/2008528427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563267b0d670 msgr2=0x563267b0c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:53.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0de510700 1 --2- 172.21.15.121:0/2008528427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563267b0d670 0x563267b0c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0de510700 1 -- 172.21.15.121:0/2008528427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d000a0e0 con 0x563267a34dd0 2023-11-15T03:05:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0ded11700 1 --2- 172.21.15.121:0/2008528427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563267b0d670 0x563267b0c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:05:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0d5d0f700 1 --2- 172.21.15.121:0/2008528427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563267a7eb00 0x563267a7a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:05:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0de510700 1 --2- 172.21.15.121:0/2008528427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563267a34dd0 0x563267a34830 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x563267b11130 tx=0x7fe0d0015000).ready entity=mon.0 client_cookie=45940a5bde191eea server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0dd50e700 1 -- 172.21.15.121:0/2008528427 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0005750 con 0x563267a34dd0 2023-11-15T03:05:53.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0dd50e700 1 -- 172.21.15.121:0/2008528427 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d000abc0 con 0x563267a34dd0 2023-11-15T03:05:53.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.770+0000 7fe0dd50e700 1 -- 172.21.15.121:0/2008528427 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d000dcc0 con 0x563267a34dd0 2023-11-15T03:05:53.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0e0cd8d00 1 -- 172.21.15.121:0/2008528427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563267a34dd0 msgr2=0x563267a34830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0e0cd8d00 1 --2- 172.21.15.121:0/2008528427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563267a34dd0 0x563267a34830 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x563267b11130 tx=0x7fe0d0015000).stop 2023-11-15T03:05:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0e0cd8d00 1 -- 172.21.15.121:0/2008528427 shutdown_connections 2023-11-15T03:05:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0e0cd8d00 1 --2- 172.21.15.121:0/2008528427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563267b0d670 0x563267b0c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0e0cd8d00 1 --2- 172.21.15.121:0/2008528427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563267a7eb00 0x563267a7a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0e0cd8d00 1 --2- 172.21.15.121:0/2008528427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563267a34dd0 0x563267a34830 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0e0cd8d00 1 -- 172.21.15.121:0/2008528427 >> 172.21.15.121:0/2008528427 conn(0x563267a84080 msgr2=0x563267a66700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:53.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0e0cd8d00 1 -- 172.21.15.121:0/2008528427 shutdown_connections 2023-11-15T03:05:53.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0e0cd8d00 1 -- 172.21.15.121:0/2008528427 wait complete. 2023-11-15T03:05:53.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0e0cd8d00 1 Processor -- start 2023-11-15T03:05:53.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0e0cd8d00 1 -- start start 2023-11-15T03:05:53.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0e0cd8d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563267a34dd0 0x563267b0c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0e0cd8d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563267a7eb00 0x563267a7cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0e0cd8d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563267b0d670 0x563267b21360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0e0cd8d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563267b09eb0 con 0x563267a34dd0 2023-11-15T03:05:53.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0e0cd8d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563267b0a150 con 0x563267a7eb00 2023-11-15T03:05:53.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0e0cd8d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x563267b0a430 con 0x563267b0d670 2023-11-15T03:05:53.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0de510700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563267a34dd0 0x563267b0c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0de510700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563267a34dd0 0x563267b0c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45832/0 (socket says 172.21.15.121:45832) 2023-11-15T03:05:53.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0de510700 1 -- 172.21.15.121:0/1953264655 learned_addr learned my addr 172.21.15.121:0/1953264655 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:53.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0ded11700 1 --2- 172.21.15.121:0/1953264655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563267b0d670 0x563267b21360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0d5d0f700 1 --2- 172.21.15.121:0/1953264655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563267a7eb00 0x563267a7cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0de510700 1 -- 172.21.15.121:0/1953264655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563267b0d670 msgr2=0x563267b21360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0de510700 1 --2- 172.21.15.121:0/1953264655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563267b0d670 0x563267b21360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0de510700 1 -- 172.21.15.121:0/1953264655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563267a7eb00 msgr2=0x563267a7cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0de510700 1 --2- 172.21.15.121:0/1953264655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563267a7eb00 0x563267a7cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0de510700 1 -- 172.21.15.121:0/1953264655 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632679d6a20 con 0x563267a34dd0 2023-11-15T03:05:53.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0ded11700 1 --2- 172.21.15.121:0/1953264655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563267b0d670 0x563267b21360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:53.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.774+0000 7fe0de510700 1 --2- 172.21.15.121:0/1953264655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563267a34dd0 0x563267b0c900 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x563267b169e0 tx=0x7fe0d001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:53.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.778+0000 7fe0d77fe700 1 -- 172.21.15.121:0/1953264655 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0001010 con 0x563267a34dd0 2023-11-15T03:05:53.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.778+0000 7fe0d77fe700 1 -- 172.21.15.121:0/1953264655 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d000dcc0 con 0x563267a34dd0 2023-11-15T03:05:53.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.778+0000 7fe0e0cd8d00 1 -- 172.21.15.121:0/1953264655 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5632679d6770 con 0x563267a34dd0 2023-11-15T03:05:53.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.778+0000 7fe0d77fe700 1 -- 172.21.15.121:0/1953264655 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0024650 con 0x563267a34dd0 2023-11-15T03:05:53.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.778+0000 7fe0e0cd8d00 1 -- 172.21.15.121:0/1953264655 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632679a5a70 con 0x563267a34dd0 2023-11-15T03:05:53.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.778+0000 7fe0d77fe700 1 -- 172.21.15.121:0/1953264655 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe0d0017070 con 0x563267a34dd0 2023-11-15T03:05:53.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.782+0000 7fe0d77fe700 1 --2- 172.21.15.121:0/1953264655 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe0c0058ff0 0x7fe0c005b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.782+0000 7fe0d77fe700 1 -- 172.21.15.121:0/1953264655 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7384+0+0 (secure 0 0 0) 0x7fe0d00762c0 con 0x563267a34dd0 2023-11-15T03:05:53.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.782+0000 7fe0d5d0f700 1 --2- 172.21.15.121:0/1953264655 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe0c0058ff0 0x7fe0c005b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.782+0000 7fe0e0cd8d00 1 --2- 172.21.15.121:0/1953264655 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x563267a34830 0x563267b18990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.782+0000 7fe0e0cd8d00 1 -- 172.21.15.121:0/1953264655 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e107) v8 -- 0x563267b0d220 con 0x563267a34830 2023-11-15T03:05:53.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.782+0000 7fe0d5d0f700 1 --2- 172.21.15.121:0/1953264655 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe0c0058ff0 0x7fe0c005b4b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x563267b0e580 tx=0x7fe0c8000bc0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:53.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.782+0000 7fe0ded11700 1 --2- 172.21.15.121:0/1953264655 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x563267a34830 0x563267b18990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.782+0000 7fe0ded11700 1 --2- 172.21.15.121:0/1953264655 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x563267a34830 0x563267b18990 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:53.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.786+0000 7fe0ded11700 1 -- 172.21.15.121:0/1953264655 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 100.00000000.inode [delete] v107'162 uv162 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe0d80099a0 con 0x563267a34830 2023-11-15T03:05:53.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.786+0000 7fe0e0cd8d00 1 -- 172.21.15.121:0/1953264655 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x563267a34830 msgr2=0x563267b18990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.786+0000 7fe0e0cd8d00 1 --2- 172.21.15.121:0/1953264655 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x563267a34830 0x563267b18990 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.786+0000 7fe0e0cd8d00 1 -- 172.21.15.121:0/1953264655 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe0c0058ff0 msgr2=0x7fe0c005b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.786+0000 7fe0e0cd8d00 1 --2- 172.21.15.121:0/1953264655 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe0c0058ff0 0x7fe0c005b4b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x563267b0e580 tx=0x7fe0c8000bc0).stop 2023-11-15T03:05:53.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.786+0000 7fe0e0cd8d00 1 -- 172.21.15.121:0/1953264655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563267a34dd0 msgr2=0x563267b0c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.786+0000 7fe0e0cd8d00 1 --2- 172.21.15.121:0/1953264655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563267a34dd0 0x563267b0c900 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x563267b169e0 tx=0x7fe0d001a000).stop 2023-11-15T03:05:53.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.786+0000 7fe0ded11700 1 -- 172.21.15.121:0/1953264655 reap_dead start 2023-11-15T03:05:53.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.786+0000 7fe0e0cd8d00 1 -- 172.21.15.121:0/1953264655 shutdown_connections 2023-11-15T03:05:53.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.786+0000 7fe0e0cd8d00 1 -- 172.21.15.121:0/1953264655 >> 172.21.15.121:0/1953264655 conn(0x563267a84080 msgr2=0x563267a81ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:53.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.790+0000 7fe0e0cd8d00 1 -- 172.21.15.121:0/1953264655 shutdown_connections 2023-11-15T03:05:53.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.790+0000 7fe0e0cd8d00 1 -- 172.21.15.121:0/1953264655 wait complete. 2023-11-15T03:05:53.811 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:05:53.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.966+0000 7f97a14e4700 1 Processor -- start 2023-11-15T03:05:53.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.966+0000 7f97a14e4700 1 -- start start 2023-11-15T03:05:53.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.966+0000 7f97a14e4700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f979c14ac10 0x7f979c0c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.966+0000 7f97a14e4700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f979c0657f0 con 0x7f979c067380 2023-11-15T03:05:53.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.966+0000 7f97a14e4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f979c066650 con 0x7f979c14aff0 2023-11-15T03:05:53.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.970+0000 7f97a14e4700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979c065a40 con 0x7f979c14ac10 2023-11-15T03:05:53.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.970+0000 7f979bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f979c14ac10 0x7f979c0c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.970+0000 7f97a0ce3700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f979c067380 0x7f979c0a78a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:33656/0 (socket says 172.21.15.121:33656) 2023-11-15T03:05:53.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.970+0000 7f97a0ce3700 1 -- 172.21.15.121:0/3893756719 learned_addr learned my addr 172.21.15.121:0/3893756719 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:53.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.970+0000 7f979bfff700 1 -- 172.21.15.121:0/3893756719 >> v1:172.21.15.190:6789/0 conn(0x7f979c14aff0 legacy=0x7f979c066c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.970+0000 7f979bfff700 1 -- 172.21.15.121:0/3893756719 >> v1:172.21.15.121:6789/0 conn(0x7f979c067380 legacy=0x7f979c0a78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.970+0000 7f979bfff700 1 -- 172.21.15.121:0/3893756719 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c065d20 con 0x7f979c14ac10 2023-11-15T03:05:53.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.970+0000 7f979affd700 1 -- 172.21.15.121:0/3893756719 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383744716 0 0) 0x7f979c0657f0 con 0x7f979c067380 2023-11-15T03:05:53.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.970+0000 7f979bfff700 1 --2- 172.21.15.121:0/3893756719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f979c14ac10 0x7f979c0c6bc0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7f978c009760 tx=0x7f978c009060).ready entity=mon.0 client_cookie=a4ad3d4dc0751702 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:53.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.970+0000 7f979affd700 1 -- 172.21.15.121:0/3893756719 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c00b580 con 0x7f979c14ac10 2023-11-15T03:05:53.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.970+0000 7f979affd700 1 -- 172.21.15.121:0/3893756719 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978c00b6e0 con 0x7f979c14ac10 2023-11-15T03:05:53.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.970+0000 7f979affd700 1 -- 172.21.15.121:0/3893756719 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c00ba60 con 0x7f979c14ac10 2023-11-15T03:05:53.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.970+0000 7f97a14e4700 1 -- 172.21.15.121:0/3893756719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f979c14ac10 msgr2=0x7f979c0c6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.970+0000 7f97a14e4700 1 --2- 172.21.15.121:0/3893756719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f979c14ac10 0x7f979c0c6bc0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7f978c009760 tx=0x7f978c009060).stop 2023-11-15T03:05:53.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.970+0000 7f97a14e4700 1 -- 172.21.15.121:0/3893756719 shutdown_connections 2023-11-15T03:05:53.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.970+0000 7f97a14e4700 1 --2- 172.21.15.121:0/3893756719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f979c14ac10 0x7f979c0c6bc0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.970+0000 7f97a14e4700 1 -- 172.21.15.121:0/3893756719 >> 172.21.15.121:0/3893756719 conn(0x7f979c067a60 msgr2=0x7f979c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:53.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.970+0000 7f97a14e4700 1 -- 172.21.15.121:0/3893756719 shutdown_connections 2023-11-15T03:05:53.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.970+0000 7f97a14e4700 1 -- 172.21.15.121:0/3893756719 wait complete. 2023-11-15T03:05:53.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.974+0000 7f97a14e4700 1 Processor -- start 2023-11-15T03:05:53.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.974+0000 7f97a14e4700 1 -- start start 2023-11-15T03:05:53.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.974+0000 7f97a14e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c067380 0x7f979c149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.974+0000 7f97a14e4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f979c14ac10 0x7f979c163c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.974+0000 7f97a14e4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f979c14aff0 0x7f979c09f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.974+0000 7f97a14e4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c1494a0 con 0x7f979c14ac10 2023-11-15T03:05:53.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.974+0000 7f97a14e4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c1488e0 con 0x7f979c14aff0 2023-11-15T03:05:53.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.974+0000 7f97a14e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979c148b70 con 0x7f979c067380 2023-11-15T03:05:53.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.978+0000 7f979b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f979c14ac10 0x7f979c163c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.978+0000 7f979b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f979c14ac10 0x7f979c163c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:45856/0 (socket says 172.21.15.121:45856) 2023-11-15T03:05:53.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.978+0000 7f979b7fe700 1 -- 172.21.15.121:0/3170249501 learned_addr learned my addr 172.21.15.121:0/3170249501 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:53.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.978+0000 7f97a0ce3700 1 --2- 172.21.15.121:0/3170249501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f979c14aff0 0x7f979c09f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.978+0000 7f979bfff700 1 --2- 172.21.15.121:0/3170249501 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c067380 0x7f979c149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.978+0000 7f979b7fe700 1 -- 172.21.15.121:0/3170249501 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c067380 msgr2=0x7f979c149b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.978+0000 7f979b7fe700 1 --2- 172.21.15.121:0/3170249501 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c067380 0x7f979c149b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.978+0000 7f979b7fe700 1 -- 172.21.15.121:0/3170249501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f979c14aff0 msgr2=0x7f979c09f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:53.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.978+0000 7f979b7fe700 1 --2- 172.21.15.121:0/3170249501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f979c14aff0 0x7f979c09f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:53.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.978+0000 7f979b7fe700 1 -- 172.21.15.121:0/3170249501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c0092a0 con 0x7f979c14ac10 2023-11-15T03:05:53.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.978+0000 7f979b7fe700 1 --2- 172.21.15.121:0/3170249501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f979c14ac10 0x7f979c163c70 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0x7f979000c440 tx=0x7f9790007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:53.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.978+0000 7f9783fff700 1 -- 172.21.15.121:0/3170249501 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790007020 con 0x7f979c14ac10 2023-11-15T03:05:53.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.978+0000 7f9783fff700 1 -- 172.21.15.121:0/3170249501 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9790007180 con 0x7f979c14ac10 2023-11-15T03:05:53.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.978+0000 7f97a14e4700 1 -- 172.21.15.121:0/3170249501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979c148e50 con 0x7f979c14ac10 2023-11-15T03:05:53.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.978+0000 7f9783fff700 1 -- 172.21.15.121:0/3170249501 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790007540 con 0x7f979c14ac10 2023-11-15T03:05:53.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.978+0000 7f97a14e4700 1 -- 172.21.15.121:0/3170249501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979c164160 con 0x7f979c14ac10 2023-11-15T03:05:53.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.978+0000 7f97a14e4700 1 -- 172.21.15.121:0/3170249501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979c149650 con 0x7f979c14ac10 2023-11-15T03:05:53.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.982+0000 7f9783fff700 1 -- 172.21.15.121:0/3170249501 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f97900068b0 con 0x7f979c14ac10 2023-11-15T03:05:53.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.982+0000 7f9783fff700 1 --2- 172.21.15.121:0/3170249501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f978805a4e0 0x7f978805c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:53.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.982+0000 7f9783fff700 1 -- 172.21.15.121:0/3170249501 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7384+0+0 (secure 0 0 0) 0x7f9790074760 con 0x7f979c14ac10 2023-11-15T03:05:53.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.982+0000 7f979bfff700 1 --2- 172.21.15.121:0/3170249501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f978805a4e0 0x7f978805c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:53.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.982+0000 7f979bfff700 1 --2- 172.21.15.121:0/3170249501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f978805a4e0 0x7f978805c9a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f978c006e00 tx=0x7f978c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:53.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:53.990+0000 7f9783fff700 1 -- 172.21.15.121:0/3170249501 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f979c149650 con 0x7f979c14ac10 2023-11-15T03:05:54.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:54.194+0000 7f97a14e4700 1 -- 172.21.15.121:0/3170249501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f979c1495e0 con 0x7f979c14ac10 2023-11-15T03:05:55.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.150+0000 7f9783fff700 1 -- 172.21.15.121:0/3170249501 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v165) v1 ==== 162+0+0 (secure 0 0 0) 0x7f9790047990 con 0x7f979c14ac10 2023-11-15T03:05:55.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.150+0000 7f97a14e4700 1 -- 172.21.15.121:0/3170249501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f978805a4e0 msgr2=0x7f978805c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:55.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.150+0000 7f97a14e4700 1 --2- 172.21.15.121:0/3170249501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f978805a4e0 0x7f978805c9a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f978c006e00 tx=0x7f978c017000).stop 2023-11-15T03:05:55.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.150+0000 7f97a14e4700 1 -- 172.21.15.121:0/3170249501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f979c14ac10 msgr2=0x7f979c163c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:55.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.150+0000 7f97a14e4700 1 --2- 172.21.15.121:0/3170249501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f979c14ac10 0x7f979c163c70 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0x7f979000c440 tx=0x7f9790007dd0).stop 2023-11-15T03:05:55.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.154+0000 7f97a14e4700 1 -- 172.21.15.121:0/3170249501 shutdown_connections 2023-11-15T03:05:55.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.154+0000 7f97a14e4700 1 --2- 172.21.15.121:0/3170249501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f978805a4e0 0x7f978805c9a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:55.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.154+0000 7f97a14e4700 1 --2- 172.21.15.121:0/3170249501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f979c14aff0 0x7f979c09f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:55.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.154+0000 7f97a14e4700 1 --2- 172.21.15.121:0/3170249501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f979c14ac10 0x7f979c163c70 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:55.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.154+0000 7f97a14e4700 1 --2- 172.21.15.121:0/3170249501 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c067380 0x7f979c149b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:55.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.154+0000 7f97a14e4700 1 -- 172.21.15.121:0/3170249501 >> 172.21.15.121:0/3170249501 conn(0x7f979c067a60 msgr2=0x7f979c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:55.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.154+0000 7f97a14e4700 1 -- 172.21.15.121:0/3170249501 shutdown_connections 2023-11-15T03:05:55.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.154+0000 7f97a14e4700 1 -- 172.21.15.121:0/3170249501 wait complete. 2023-11-15T03:05:55.164 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:05:55.175 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:05:55.166+0000 7f74796a6700 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2023-11-15T03:05:55.175 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:05:55.166+0000 7f74796a6700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2023-11-15T03:05:55.183 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:05:55.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.330+0000 7fe1a385b700 1 Processor -- start 2023-11-15T03:05:55.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.330+0000 7fe1a385b700 1 -- start start 2023-11-15T03:05:55.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.330+0000 7fe1a385b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe19c14af80 0x7fe19c0c1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:55.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.330+0000 7fe1a385b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe19c0c1bf0 0x7fe19c150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:55.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.330+0000 7fe1a385b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe19c152000 con 0x7fe19c1514e0 2023-11-15T03:05:55.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.330+0000 7fe1a385b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe19c0c1270 con 0x7fe19c0c1bf0 2023-11-15T03:05:55.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.330+0000 7fe1a385b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe19c151a40 con 0x7fe19c14af80 2023-11-15T03:05:55.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.330+0000 7fe1a1df8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe19c1514e0 0x7fe19c0c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39510/0 (socket says 172.21.15.121:39510) 2023-11-15T03:05:55.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.330+0000 7fe1a1df8700 1 -- 172.21.15.121:0/787812960 learned_addr learned my addr 172.21.15.121:0/787812960 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:05:55.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.330+0000 7fe1a0df6700 1 --2- 172.21.15.121:0/787812960 >> v2:172.21.15.121:3300/0 conn(0x7fe19c0c1bf0 0x7fe19c150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:55.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.330+0000 7fe1a15f7700 1 --2- 172.21.15.121:0/787812960 >> v2:172.21.15.190:3300/0 conn(0x7fe19c14af80 0x7fe19c0c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:55.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.334+0000 7fe1a0df6700 1 -- 172.21.15.121:0/787812960 >> v2:172.21.15.190:3300/0 conn(0x7fe19c14af80 msgr2=0x7fe19c0c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:05:55.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.334+0000 7fe1a0df6700 1 --2- 172.21.15.121:0/787812960 >> v2:172.21.15.190:3300/0 conn(0x7fe19c14af80 0x7fe19c0c1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:55.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.334+0000 7fe1a0df6700 1 -- 172.21.15.121:0/787812960 >> v1:172.21.15.121:6789/0 conn(0x7fe19c1514e0 legacy=0x7fe19c0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:55.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.334+0000 7fe1a0df6700 1 -- 172.21.15.121:0/787812960 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19c151d20 con 0x7fe19c0c1bf0 2023-11-15T03:05:55.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.334+0000 7fe1a0df6700 1 --2- 172.21.15.121:0/787812960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c0c1bf0 0x7fe19c150ff0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7fe19c0653f0 tx=0x7fe18c009060).ready entity=mon.0 client_cookie=a69cb61eda80f868 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:55.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.334+0000 7fe193fff700 1 -- 172.21.15.121:0/787812960 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c00f510 con 0x7fe19c0c1bf0 2023-11-15T03:05:55.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.334+0000 7fe193fff700 1 -- 172.21.15.121:0/787812960 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18c00f670 con 0x7fe19c0c1bf0 2023-11-15T03:05:55.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.334+0000 7fe193fff700 1 -- 172.21.15.121:0/787812960 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c00f9f0 con 0x7fe19c0c1bf0 2023-11-15T03:05:55.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.334+0000 7fe1a385b700 1 -- 172.21.15.121:0/787812960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c0c1bf0 msgr2=0x7fe19c150ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:55.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.334+0000 7fe1a385b700 1 --2- 172.21.15.121:0/787812960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c0c1bf0 0x7fe19c150ff0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7fe19c0653f0 tx=0x7fe18c009060).stop 2023-11-15T03:05:55.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.334+0000 7fe1a385b700 1 -- 172.21.15.121:0/787812960 shutdown_connections 2023-11-15T03:05:55.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.334+0000 7fe1a385b700 1 --2- 172.21.15.121:0/787812960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c0c1bf0 0x7fe19c150ff0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:55.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.334+0000 7fe1a385b700 1 --2- 172.21.15.121:0/787812960 >> v2:172.21.15.190:3300/0 conn(0x7fe19c14af80 0x7fe19c0c1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:55.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.334+0000 7fe1a385b700 1 -- 172.21.15.121:0/787812960 >> 172.21.15.121:0/787812960 conn(0x7fe19c062190 msgr2=0x7fe19c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:55.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.334+0000 7fe1a385b700 1 -- 172.21.15.121:0/787812960 shutdown_connections 2023-11-15T03:05:55.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.334+0000 7fe1a385b700 1 -- 172.21.15.121:0/787812960 wait complete. 2023-11-15T03:05:55.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a385b700 1 Processor -- start 2023-11-15T03:05:55.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a385b700 1 -- start start 2023-11-15T03:05:55.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a385b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c0c1bf0 0x7fe19c0ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:55.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a385b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe19c14af80 0x7fe19c0bfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:55.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a385b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe19c1514e0 0x7fe19c148e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:55.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a385b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c167960 con 0x7fe19c0c1bf0 2023-11-15T03:05:55.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a385b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c1684e0 con 0x7fe19c14af80 2023-11-15T03:05:55.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a385b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c14c9b0 con 0x7fe19c1514e0 2023-11-15T03:05:55.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a15f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c0c1bf0 0x7fe19c0ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:55.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a15f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c0c1bf0 0x7fe19c0ab0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51842/0 (socket says 172.21.15.121:51842) 2023-11-15T03:05:55.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a15f7700 1 -- 172.21.15.121:0/2439848501 learned_addr learned my addr 172.21.15.121:0/2439848501 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:05:55.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a0df6700 1 --2- 172.21.15.121:0/2439848501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe19c14af80 0x7fe19c0bfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:55.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a1df8700 1 --2- 172.21.15.121:0/2439848501 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe19c1514e0 0x7fe19c148e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:55.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a15f7700 1 -- 172.21.15.121:0/2439848501 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe19c1514e0 msgr2=0x7fe19c148e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:55.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a15f7700 1 --2- 172.21.15.121:0/2439848501 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe19c1514e0 0x7fe19c148e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:55.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a15f7700 1 -- 172.21.15.121:0/2439848501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe19c14af80 msgr2=0x7fe19c0bfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:55.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a15f7700 1 --2- 172.21.15.121:0/2439848501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe19c14af80 0x7fe19c0bfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:55.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a15f7700 1 -- 172.21.15.121:0/2439848501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18c0092a0 con 0x7fe19c0c1bf0 2023-11-15T03:05:55.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a0df6700 1 --2- 172.21.15.121:0/2439848501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe19c14af80 0x7fe19c0bfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:05:55.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a15f7700 1 --2- 172.21.15.121:0/2439848501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c0c1bf0 0x7fe19c0ab0e0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7fe19800e960 tx=0x7fe198009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:55.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1917fa700 1 -- 172.21.15.121:0/2439848501 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19800c850 con 0x7fe19c0c1bf0 2023-11-15T03:05:55.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1917fa700 1 -- 172.21.15.121:0/2439848501 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19800c9b0 con 0x7fe19c0c1bf0 2023-11-15T03:05:55.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.338+0000 7fe1a385b700 1 -- 172.21.15.121:0/2439848501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe19c167bb0 con 0x7fe19c0c1bf0 2023-11-15T03:05:55.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.342+0000 7fe1917fa700 1 -- 172.21.15.121:0/2439848501 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19800cd30 con 0x7fe19c0c1bf0 2023-11-15T03:05:55.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.342+0000 7fe1a385b700 1 -- 172.21.15.121:0/2439848501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe19c168170 con 0x7fe19c0c1bf0 2023-11-15T03:05:55.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.342+0000 7fe1a385b700 1 -- 172.21.15.121:0/2439848501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe19c06c150 con 0x7fe19c0c1bf0 2023-11-15T03:05:55.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.342+0000 7fe1917fa700 1 -- 172.21.15.121:0/2439848501 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe198012600 con 0x7fe19c0c1bf0 2023-11-15T03:05:55.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.346+0000 7fe1917fa700 1 --2- 172.21.15.121:0/2439848501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe18005a510 0x7fe18005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:05:55.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.346+0000 7fe1917fa700 1 -- 172.21.15.121:0/2439848501 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7470+0+0 (secure 0 0 0) 0x7fe1980753a0 con 0x7fe19c0c1bf0 2023-11-15T03:05:55.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.346+0000 7fe1a0df6700 1 --2- 172.21.15.121:0/2439848501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe18005a510 0x7fe18005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:05:55.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.346+0000 7fe1a0df6700 1 --2- 172.21.15.121:0/2439848501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe18005a510 0x7fe18005c9d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7fe19c154250 tx=0x7fe18c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:05:55.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.350+0000 7fe1917fa700 1 -- 172.21.15.121:0/2439848501 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe198048580 con 0x7fe19c0c1bf0 2023-11-15T03:05:55.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.554+0000 7fe1a385b700 1 -- 172.21.15.121:0/2439848501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe19c0c5820 con 0x7fe19c0c1bf0 2023-11-15T03:05:55.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.554+0000 7fe1917fa700 1 -- 172.21.15.121:0/2439848501 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+4502 (secure 0 0 0) 0x7fe19804bba0 con 0x7fe19c0c1bf0 2023-11-15T03:05:55.563 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:05:55.564 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":166,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:55.154017+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8457":{"gid":8457,"name":"a","rank":0,"incarnation":166,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/400230871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":400230871},{"type":"v1","addr":"172.21.15.121:6836","nonce":400230871}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:05:55.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.554+0000 7fe1a385b700 1 -- 172.21.15.121:0/2439848501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe18005a510 msgr2=0x7fe18005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:55.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.554+0000 7fe1a385b700 1 --2- 172.21.15.121:0/2439848501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe18005a510 0x7fe18005c9d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7fe19c154250 tx=0x7fe18c017000).stop 2023-11-15T03:05:55.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.558+0000 7fe1a385b700 1 -- 172.21.15.121:0/2439848501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c0c1bf0 msgr2=0x7fe19c0ab0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:05:55.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.558+0000 7fe1a385b700 1 --2- 172.21.15.121:0/2439848501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c0c1bf0 0x7fe19c0ab0e0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7fe19800e960 tx=0x7fe198009060).stop 2023-11-15T03:05:55.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.558+0000 7fe1a385b700 1 -- 172.21.15.121:0/2439848501 shutdown_connections 2023-11-15T03:05:55.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.558+0000 7fe1a385b700 1 --2- 172.21.15.121:0/2439848501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe18005a510 0x7fe18005c9d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:55.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.558+0000 7fe1a385b700 1 --2- 172.21.15.121:0/2439848501 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe19c1514e0 0x7fe19c148e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:55.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.558+0000 7fe1a385b700 1 --2- 172.21.15.121:0/2439848501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe19c14af80 0x7fe19c0bfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:55.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.558+0000 7fe1a385b700 1 --2- 172.21.15.121:0/2439848501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe19c0c1bf0 0x7fe19c0ab0e0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:05:55.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.558+0000 7fe1a385b700 1 -- 172.21.15.121:0/2439848501 >> 172.21.15.121:0/2439848501 conn(0x7fe19c062190 msgr2=0x7fe19c0c6570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:05:55.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.558+0000 7fe1a385b700 1 -- 172.21.15.121:0/2439848501 shutdown_connections 2023-11-15T03:05:55.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:05:55.558+0000 7fe1a385b700 1 -- 172.21.15.121:0/2439848501 wait complete. 2023-11-15T03:05:55.569 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 166 2023-11-15T03:05:55.582 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:05:56.165 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:05:56.154+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:05:56.165 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:05:56.154+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:05:56.173 INFO:tasks.ceph.mds.a.smithi121.stderr: -25> 2023-11-15T03:05:55.166+0000 7f74796a6700 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2023-11-15T03:05:56.174 INFO:tasks.ceph.mds.a.smithi121.stderr: -15> 2023-11-15T03:05:55.166+0000 7f74796a6700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2023-11-15T03:05:56.216 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:06:00.583 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:00.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.734+0000 7febec81a700 1 Processor -- start 2023-11-15T03:06:00.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.738+0000 7febec81a700 1 -- start start 2023-11-15T03:06:00.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.738+0000 7febec81a700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7febe80c3280 0x7febe80c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:00.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.738+0000 7febec81a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febe815be30 0x7febe815c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:00.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.738+0000 7febec81a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe80c1d50 con 0x7febe8152140 2023-11-15T03:06:00.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.738+0000 7febec81a700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe80c1860 con 0x7febe80c3280 2023-11-15T03:06:00.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.738+0000 7febec81a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe815cb80 con 0x7febe815be30 2023-11-15T03:06:00.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.738+0000 7febe6d9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7febe80c3280 0x7febe80c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:00.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.738+0000 7febe759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febe815be30 0x7febe815c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:00.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.738+0000 7febde59c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7febe8152140 0x7febe81525f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:49950/0 (socket says 172.21.15.121:49950) 2023-11-15T03:06:00.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febde59c700 1 -- 172.21.15.121:0/3326414785 learned_addr learned my addr 172.21.15.121:0/3326414785 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:00.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febe659c700 1 -- 172.21.15.121:0/3326414785 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 347380050 0 0) 0x7febe80c1d50 con 0x7febe8152140 2023-11-15T03:06:00.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febe659c700 1 -- 172.21.15.121:0/3326414785 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febd40033f0 con 0x7febe8152140 2023-11-15T03:06:00.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febe6d9d700 1 -- 172.21.15.121:0/3326414785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febe815be30 msgr2=0x7febe815c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:00.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febe6d9d700 1 --2- 172.21.15.121:0/3326414785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febe815be30 0x7febe815c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:00.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febe6d9d700 1 -- 172.21.15.121:0/3326414785 >> v1:172.21.15.190:6789/0 conn(0x7febe8152140 legacy=0x7febe81525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:00.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febe6d9d700 1 -- 172.21.15.121:0/3326414785 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe80a9240 con 0x7febe80c3280 2023-11-15T03:06:00.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febe759e700 1 --2- 172.21.15.121:0/3326414785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febe815be30 0x7febe815c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:06:00.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febe6d9d700 1 --2- 172.21.15.121:0/3326414785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe80c3280 0x7febe80c3660 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7febd8009e60 tx=0x7febd8009780).ready entity=mon.0 client_cookie=f0815576b4222096 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:00.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febe659c700 1 -- 172.21.15.121:0/3326414785 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd800e070 con 0x7febe80c3280 2023-11-15T03:06:00.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febe659c700 1 -- 172.21.15.121:0/3326414785 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd800bab0 con 0x7febe80c3280 2023-11-15T03:06:00.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febe659c700 1 -- 172.21.15.121:0/3326414785 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd800be30 con 0x7febe80c3280 2023-11-15T03:06:00.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febec81a700 1 -- 172.21.15.121:0/3326414785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe80c3280 msgr2=0x7febe80c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:00.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febec81a700 1 --2- 172.21.15.121:0/3326414785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe80c3280 0x7febe80c3660 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7febd8009e60 tx=0x7febd8009780).stop 2023-11-15T03:06:00.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febec81a700 1 -- 172.21.15.121:0/3326414785 shutdown_connections 2023-11-15T03:06:00.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febec81a700 1 --2- 172.21.15.121:0/3326414785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febe815be30 0x7febe815c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:00.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febec81a700 1 --2- 172.21.15.121:0/3326414785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe80c3280 0x7febe80c3660 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:00.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febec81a700 1 -- 172.21.15.121:0/3326414785 >> 172.21.15.121:0/3326414785 conn(0x7febe8062190 msgr2=0x7febe8062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:00.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febec81a700 1 -- 172.21.15.121:0/3326414785 shutdown_connections 2023-11-15T03:06:00.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.742+0000 7febec81a700 1 -- 172.21.15.121:0/3326414785 wait complete. 2023-11-15T03:06:00.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.746+0000 7febec81a700 1 Processor -- start 2023-11-15T03:06:00.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.746+0000 7febec81a700 1 -- start start 2023-11-15T03:06:00.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febec81a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe8152140 0x7febe80c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:00.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febec81a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febe815be30 0x7febe81525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:00.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febec81a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febe813d870 0x7febe813dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:00.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febec81a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe80c67f0 con 0x7febe8152140 2023-11-15T03:06:00.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febec81a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe80c7170 con 0x7febe815be30 2023-11-15T03:06:00.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febec81a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe8155760 con 0x7febe813d870 2023-11-15T03:06:00.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febe6d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe8152140 0x7febe80c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:00.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febe6d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe8152140 0x7febe80c6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51868/0 (socket says 172.21.15.121:51868) 2023-11-15T03:06:00.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febe6d9d700 1 -- 172.21.15.121:0/182557426 learned_addr learned my addr 172.21.15.121:0/182557426 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:00.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febde59c700 1 --2- 172.21.15.121:0/182557426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febe815be30 0x7febe81525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:00.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febe759e700 1 --2- 172.21.15.121:0/182557426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febe813d870 0x7febe813dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:00.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febe6d9d700 1 -- 172.21.15.121:0/182557426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febe813d870 msgr2=0x7febe813dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:00.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febe6d9d700 1 --2- 172.21.15.121:0/182557426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febe813d870 0x7febe813dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:00.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febe6d9d700 1 -- 172.21.15.121:0/182557426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febe815be30 msgr2=0x7febe81525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:00.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febe6d9d700 1 --2- 172.21.15.121:0/182557426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febe815be30 0x7febe81525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:00.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febe6d9d700 1 -- 172.21.15.121:0/182557426 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe00092a0 con 0x7febe8152140 2023-11-15T03:06:00.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febe6d9d700 1 --2- 172.21.15.121:0/182557426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe8152140 0x7febe80c6c80 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7febe80654e0 tx=0x7febe80c1020).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:00.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febdf7fe700 1 -- 172.21.15.121:0/182557426 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd801d420 con 0x7febe8152140 2023-11-15T03:06:00.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febdf7fe700 1 -- 172.21.15.121:0/182557426 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd801d580 con 0x7febe8152140 2023-11-15T03:06:00.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febec81a700 1 -- 172.21.15.121:0/182557426 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd8009a20 con 0x7febe8152140 2023-11-15T03:06:00.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febdf7fe700 1 -- 172.21.15.121:0/182557426 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd801d900 con 0x7febe8152140 2023-11-15T03:06:00.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.750+0000 7febec81a700 1 -- 172.21.15.121:0/182557426 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe80c6480 con 0x7febe8152140 2023-11-15T03:06:00.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.754+0000 7febec81a700 1 -- 172.21.15.121:0/182557426 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe806bcf0 con 0x7febe8152140 2023-11-15T03:06:00.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.754+0000 7febdf7fe700 1 -- 172.21.15.121:0/182557426 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7febd8003040 con 0x7febe8152140 2023-11-15T03:06:00.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.754+0000 7febdf7fe700 1 --2- 172.21.15.121:0/182557426 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7febc805a510 0x7febc805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:00.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.754+0000 7febdf7fe700 1 -- 172.21.15.121:0/182557426 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7470+0+0 (secure 0 0 0) 0x7febd8076a10 con 0x7febe8152140 2023-11-15T03:06:00.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.754+0000 7febde59c700 1 --2- 172.21.15.121:0/182557426 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7febc805a510 0x7febc805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:00.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.758+0000 7febde59c700 1 --2- 172.21.15.121:0/182557426 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7febc805a510 0x7febc805c9d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7febe8153bf0 tx=0x7febd0009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:00.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.762+0000 7febdf7fe700 1 -- 172.21.15.121:0/182557426 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7febd8049bf0 con 0x7febe8152140 2023-11-15T03:06:00.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.958+0000 7febec81a700 1 -- 172.21.15.121:0/182557426 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febe80c1f80 con 0x7febe8152140 2023-11-15T03:06:00.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.958+0000 7febdf7fe700 1 -- 172.21.15.121:0/182557426 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 168 v168) v1 ==== 95+0+4493 (secure 0 0 0) 0x7febd804d210 con 0x7febe8152140 2023-11-15T03:06:00.967 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:00.968 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":168,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/228280964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":228280964},{"type":"v1","addr":"172.21.15.121:6836","nonce":228280964}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":167,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:56.160979+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:00.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.962+0000 7febec81a700 1 -- 172.21.15.121:0/182557426 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7febc805a510 msgr2=0x7febc805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:00.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.962+0000 7febec81a700 1 --2- 172.21.15.121:0/182557426 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7febc805a510 0x7febc805c9d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7febe8153bf0 tx=0x7febd0009040).stop 2023-11-15T03:06:00.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.962+0000 7febec81a700 1 -- 172.21.15.121:0/182557426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe8152140 msgr2=0x7febe80c6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:00.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.962+0000 7febec81a700 1 --2- 172.21.15.121:0/182557426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe8152140 0x7febe80c6c80 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7febe80654e0 tx=0x7febe80c1020).stop 2023-11-15T03:06:00.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.962+0000 7febec81a700 1 -- 172.21.15.121:0/182557426 shutdown_connections 2023-11-15T03:06:00.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.962+0000 7febec81a700 1 --2- 172.21.15.121:0/182557426 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7febc805a510 0x7febc805c9d0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:00.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.962+0000 7febec81a700 1 --2- 172.21.15.121:0/182557426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febe813d870 0x7febe813dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:00.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.962+0000 7febec81a700 1 --2- 172.21.15.121:0/182557426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febe815be30 0x7febe81525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:00.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.962+0000 7febec81a700 1 --2- 172.21.15.121:0/182557426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe8152140 0x7febe80c6c80 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:00.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.962+0000 7febec81a700 1 -- 172.21.15.121:0/182557426 >> 172.21.15.121:0/182557426 conn(0x7febe8062190 msgr2=0x7febe815ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:00.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.962+0000 7febec81a700 1 -- 172.21.15.121:0/182557426 shutdown_connections 2023-11-15T03:06:00.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:00.962+0000 7febec81a700 1 -- 172.21.15.121:0/182557426 wait complete. 2023-11-15T03:06:00.973 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 168 2023-11-15T03:06:00.991 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:06:00.992 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:01.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.138+0000 7f0e18e5b700 1 Processor -- start 2023-11-15T03:06:01.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.142+0000 7f0e18e5b700 1 -- start start 2023-11-15T03:06:01.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.142+0000 7f0e18e5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e141562f0 0x7f0e14155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:01.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.142+0000 7f0e18e5b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0e140a57c0 0x7f0e140a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:01.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.142+0000 7f0e18e5b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e140c4c30 con 0x7f0e14154ca0 2023-11-15T03:06:01.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.142+0000 7f0e18e5b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e140a8280 con 0x7f0e140a57c0 2023-11-15T03:06:01.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.142+0000 7f0e18e5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e140a6d40 con 0x7f0e141562f0 2023-11-15T03:06:01.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.142+0000 7f0e11d9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e141562f0 0x7f0e14155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:01.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.142+0000 7f0e1259c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f0e14154ca0 0x7f0e140a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:49962/0 (socket says 172.21.15.121:49962) 2023-11-15T03:06:01.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.142+0000 7f0e1259c700 1 -- 172.21.15.121:0/345563224 learned_addr learned my addr 172.21.15.121:0/345563224 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:01.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.142+0000 7f0e12d9d700 1 --2- 172.21.15.121:0/345563224 >> v2:172.21.15.190:3300/0 conn(0x7f0e140a57c0 0x7f0e140a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:01.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.146+0000 7f0e1159a700 1 -- 172.21.15.121:0/345563224 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873013640 0 0) 0x7f0e140c4c30 con 0x7f0e14154ca0 2023-11-15T03:06:01.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.146+0000 7f0e1159a700 1 -- 172.21.15.121:0/345563224 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dfc0033f0 con 0x7f0e14154ca0 2023-11-15T03:06:01.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.146+0000 7f0e11d9b700 1 -- 172.21.15.121:0/345563224 >> v2:172.21.15.190:3300/0 conn(0x7f0e140a57c0 msgr2=0x7f0e140a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:01.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.146+0000 7f0e11d9b700 1 --2- 172.21.15.121:0/345563224 >> v2:172.21.15.190:3300/0 conn(0x7f0e140a57c0 0x7f0e140a5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:01.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.146+0000 7f0e11d9b700 1 -- 172.21.15.121:0/345563224 >> v1:172.21.15.190:6789/0 conn(0x7f0e14154ca0 legacy=0x7f0e140a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:01.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.146+0000 7f0e11d9b700 1 -- 172.21.15.121:0/345563224 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e140a6140 con 0x7f0e141562f0 2023-11-15T03:06:01.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.146+0000 7f0e11d9b700 1 --2- 172.21.15.121:0/345563224 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e141562f0 0x7f0e14155080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f0e08009760 tx=0x7f0e08009060).ready entity=mon.2 client_cookie=1388a8b093d36a35 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:01.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.146+0000 7f0e1159a700 1 -- 172.21.15.121:0/345563224 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0800d4a0 con 0x7f0e141562f0 2023-11-15T03:06:01.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.146+0000 7f0e1159a700 1 -- 172.21.15.121:0/345563224 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0800d600 con 0x7f0e141562f0 2023-11-15T03:06:01.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.146+0000 7f0e1159a700 1 -- 172.21.15.121:0/345563224 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0800d980 con 0x7f0e141562f0 2023-11-15T03:06:01.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.146+0000 7f0e18e5b700 1 -- 172.21.15.121:0/345563224 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e141562f0 msgr2=0x7f0e14155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:01.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.146+0000 7f0e18e5b700 1 --2- 172.21.15.121:0/345563224 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e141562f0 0x7f0e14155080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f0e08009760 tx=0x7f0e08009060).stop 2023-11-15T03:06:01.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.146+0000 7f0e18e5b700 1 -- 172.21.15.121:0/345563224 shutdown_connections 2023-11-15T03:06:01.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.146+0000 7f0e18e5b700 1 --2- 172.21.15.121:0/345563224 >> v2:172.21.15.190:3300/0 conn(0x7f0e140a57c0 0x7f0e140a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:01.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.146+0000 7f0e18e5b700 1 --2- 172.21.15.121:0/345563224 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e141562f0 0x7f0e14155080 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:01.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.146+0000 7f0e18e5b700 1 -- 172.21.15.121:0/345563224 >> 172.21.15.121:0/345563224 conn(0x7f0e14062190 msgr2=0x7f0e140625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:01.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.146+0000 7f0e18e5b700 1 -- 172.21.15.121:0/345563224 shutdown_connections 2023-11-15T03:06:01.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.146+0000 7f0e18e5b700 1 -- 172.21.15.121:0/345563224 wait complete. 2023-11-15T03:06:01.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.150+0000 7f0e18e5b700 1 Processor -- start 2023-11-15T03:06:01.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.150+0000 7f0e18e5b700 1 -- start start 2023-11-15T03:06:01.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e18e5b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e140a57c0 0x7f0e1414dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:01.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e18e5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e14154ca0 0x7f0e14151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:01.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e18e5b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e141562f0 0x7f0e140c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:01.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e18e5b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e140ab880 con 0x7f0e141562f0 2023-11-15T03:06:01.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e18e5b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1414e0b0 con 0x7f0e140a57c0 2023-11-15T03:06:01.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e18e5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e14150e80 con 0x7f0e14154ca0 2023-11-15T03:06:01.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e12d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e141562f0 0x7f0e140c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:01.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e12d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e141562f0 0x7f0e140c54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51870/0 (socket says 172.21.15.121:51870) 2023-11-15T03:06:01.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e12d9d700 1 -- 172.21.15.121:0/1499080589 learned_addr learned my addr 172.21.15.121:0/1499080589 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:01.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e1259c700 1 --2- 172.21.15.121:0/1499080589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e140a57c0 0x7f0e1414dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:01.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e11d9b700 1 --2- 172.21.15.121:0/1499080589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e14154ca0 0x7f0e14151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:01.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e12d9d700 1 -- 172.21.15.121:0/1499080589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e14154ca0 msgr2=0x7f0e14151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:01.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e12d9d700 1 --2- 172.21.15.121:0/1499080589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e14154ca0 0x7f0e14151310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:01.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e12d9d700 1 -- 172.21.15.121:0/1499080589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e140a57c0 msgr2=0x7f0e1414dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:01.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e12d9d700 1 --2- 172.21.15.121:0/1499080589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e140a57c0 0x7f0e1414dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:01.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e12d9d700 1 -- 172.21.15.121:0/1499080589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e080092a0 con 0x7f0e141562f0 2023-11-15T03:06:01.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e12d9d700 1 --2- 172.21.15.121:0/1499080589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e141562f0 0x7f0e140c54c0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7f0e0c00e8d0 tx=0x7f0e0c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:01.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e027fc700 1 -- 172.21.15.121:0/1499080589 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c009530 con 0x7f0e141562f0 2023-11-15T03:06:01.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e027fc700 1 -- 172.21.15.121:0/1499080589 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0c012600 con 0x7f0e141562f0 2023-11-15T03:06:01.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e18e5b700 1 -- 172.21.15.121:0/1499080589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e14150550 con 0x7f0e141562f0 2023-11-15T03:06:01.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.154+0000 7f0e027fc700 1 -- 172.21.15.121:0/1499080589 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c012980 con 0x7f0e141562f0 2023-11-15T03:06:01.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.158+0000 7f0e18e5b700 1 -- 172.21.15.121:0/1499080589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e14150b10 con 0x7f0e141562f0 2023-11-15T03:06:01.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.158+0000 7f0e18e5b700 1 -- 172.21.15.121:0/1499080589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e14150ff0 con 0x7f0e141562f0 2023-11-15T03:06:01.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.158+0000 7f0e027fc700 1 -- 172.21.15.121:0/1499080589 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0e0c012d30 con 0x7f0e141562f0 2023-11-15T03:06:01.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.162+0000 7f0e027fc700 1 --2- 172.21.15.121:0/1499080589 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0df405a510 0x7f0df405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:01.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.162+0000 7f0e027fc700 1 -- 172.21.15.121:0/1499080589 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7470+0+0 (secure 0 0 0) 0x7f0e0c0751a0 con 0x7f0e141562f0 2023-11-15T03:06:01.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.162+0000 7f0e1259c700 1 --2- 172.21.15.121:0/1499080589 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0df405a510 0x7f0df405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:01.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.162+0000 7f0e1259c700 1 --2- 172.21.15.121:0/1499080589 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0df405a510 0x7f0df405c9d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f0e04005ad0 tx=0x7f0e04009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:01.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.166+0000 7f0e027fc700 1 -- 172.21.15.121:0/1499080589 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0e0c048380 con 0x7f0e141562f0 2023-11-15T03:06:01.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.366+0000 7f0e18e5b700 1 -- 172.21.15.121:0/1499080589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e1406c150 con 0x7f0e141562f0 2023-11-15T03:06:01.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.366+0000 7f0e027fc700 1 -- 172.21.15.121:0/1499080589 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 168 v168) v1 ==== 95+0+4493 (secure 0 0 0) 0x7f0e0c04b9a0 con 0x7f0e141562f0 2023-11-15T03:06:01.376 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:01.376 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":168,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/228280964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":228280964},{"type":"v1","addr":"172.21.15.121:6836","nonce":228280964}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":167,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:56.160979+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:01.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.370+0000 7f0e18e5b700 1 -- 172.21.15.121:0/1499080589 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0df405a510 msgr2=0x7f0df405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:01.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.370+0000 7f0e18e5b700 1 --2- 172.21.15.121:0/1499080589 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0df405a510 0x7f0df405c9d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f0e04005ad0 tx=0x7f0e04009040).stop 2023-11-15T03:06:01.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.370+0000 7f0e18e5b700 1 -- 172.21.15.121:0/1499080589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e141562f0 msgr2=0x7f0e140c54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:01.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.370+0000 7f0e18e5b700 1 --2- 172.21.15.121:0/1499080589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e141562f0 0x7f0e140c54c0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7f0e0c00e8d0 tx=0x7f0e0c009060).stop 2023-11-15T03:06:01.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.370+0000 7f0e18e5b700 1 -- 172.21.15.121:0/1499080589 shutdown_connections 2023-11-15T03:06:01.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.370+0000 7f0e18e5b700 1 --2- 172.21.15.121:0/1499080589 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0df405a510 0x7f0df405c9d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:01.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.370+0000 7f0e18e5b700 1 --2- 172.21.15.121:0/1499080589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e141562f0 0x7f0e140c54c0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:01.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.370+0000 7f0e18e5b700 1 --2- 172.21.15.121:0/1499080589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e14154ca0 0x7f0e14151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:01.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.370+0000 7f0e18e5b700 1 --2- 172.21.15.121:0/1499080589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e140a57c0 0x7f0e1414dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:01.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.370+0000 7f0e18e5b700 1 -- 172.21.15.121:0/1499080589 >> 172.21.15.121:0/1499080589 conn(0x7f0e14062190 msgr2=0x7f0e14066b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:01.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.370+0000 7f0e18e5b700 1 -- 172.21.15.121:0/1499080589 shutdown_connections 2023-11-15T03:06:01.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.370+0000 7f0e18e5b700 1 -- 172.21.15.121:0/1499080589 wait complete. 2023-11-15T03:06:01.383 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 168 2023-11-15T03:06:01.400 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:06:01.401 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:01.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.554+0000 7f253ca8f700 1 Processor -- start 2023-11-15T03:06:01.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.554+0000 7f253ca8f700 1 -- start start 2023-11-15T03:06:01.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.558+0000 7f253ca8f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2538066700 0x7f2538154110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:01.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.558+0000 7f253ca8f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25380acf10 0x7f2538062190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:01.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.558+0000 7f253ca8f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25380c73e0 con 0x7f2538066320 2023-11-15T03:06:01.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.558+0000 7f253ca8f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25381546c0 con 0x7f2538066700 2023-11-15T03:06:01.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.558+0000 7f253ca8f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25381549a0 con 0x7f25380acf10 2023-11-15T03:06:01.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.558+0000 7f25377fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f2538066320 0x7f25380656a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39516/0 (socket says 172.21.15.121:39516) 2023-11-15T03:06:01.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.558+0000 7f25377fe700 1 -- 172.21.15.121:0/525154946 learned_addr learned my addr 172.21.15.121:0/525154946 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:01.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.558+0000 7f2536ffd700 1 --2- 172.21.15.121:0/525154946 >> v2:172.21.15.190:3300/0 conn(0x7f2538066700 0x7f2538154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:01.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.558+0000 7f2537fff700 1 --2- 172.21.15.121:0/525154946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25380acf10 0x7f2538062190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:01.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.558+0000 7f25367fc700 1 -- 172.21.15.121:0/525154946 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968005766 0 0) 0x7f25380c73e0 con 0x7f2538066320 2023-11-15T03:06:01.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.558+0000 7f25367fc700 1 -- 172.21.15.121:0/525154946 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25200033f0 con 0x7f2538066320 2023-11-15T03:06:01.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.558+0000 7f2536ffd700 1 -- 172.21.15.121:0/525154946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25380acf10 msgr2=0x7f2538062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:01.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.558+0000 7f2536ffd700 1 --2- 172.21.15.121:0/525154946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25380acf10 0x7f2538062190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:01.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.558+0000 7f2536ffd700 1 -- 172.21.15.121:0/525154946 >> v1:172.21.15.121:6789/0 conn(0x7f2538066320 legacy=0x7f25380656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:01.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.558+0000 7f2536ffd700 1 -- 172.21.15.121:0/525154946 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2538154c80 con 0x7f2538066700 2023-11-15T03:06:01.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.558+0000 7f2536ffd700 1 --2- 172.21.15.121:0/525154946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2538066700 0x7f2538154110 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f252c009760 tx=0x7f252c009060).ready entity=mon.1 client_cookie=7ae82914542323be server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:01.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.558+0000 7f25367fc700 1 -- 172.21.15.121:0/525154946 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c00d4a0 con 0x7f2538066700 2023-11-15T03:06:01.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.558+0000 7f25367fc700 1 -- 172.21.15.121:0/525154946 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252c00d600 con 0x7f2538066700 2023-11-15T03:06:01.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.558+0000 7f25367fc700 1 -- 172.21.15.121:0/525154946 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c00d980 con 0x7f2538066700 2023-11-15T03:06:01.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.562+0000 7f253ca8f700 1 -- 172.21.15.121:0/525154946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2538066700 msgr2=0x7f2538154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:01.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.562+0000 7f253ca8f700 1 --2- 172.21.15.121:0/525154946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2538066700 0x7f2538154110 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f252c009760 tx=0x7f252c009060).stop 2023-11-15T03:06:01.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.562+0000 7f253ca8f700 1 -- 172.21.15.121:0/525154946 shutdown_connections 2023-11-15T03:06:01.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.562+0000 7f253ca8f700 1 --2- 172.21.15.121:0/525154946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25380acf10 0x7f2538062190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:01.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.562+0000 7f253ca8f700 1 --2- 172.21.15.121:0/525154946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2538066700 0x7f2538154110 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:01.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.562+0000 7f253ca8f700 1 -- 172.21.15.121:0/525154946 >> 172.21.15.121:0/525154946 conn(0x7f2538067a60 msgr2=0x7f2538067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:01.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.562+0000 7f253ca8f700 1 -- 172.21.15.121:0/525154946 shutdown_connections 2023-11-15T03:06:01.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.562+0000 7f253ca8f700 1 -- 172.21.15.121:0/525154946 wait complete. 2023-11-15T03:06:01.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.562+0000 7f253ca8f700 1 Processor -- start 2023-11-15T03:06:01.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.562+0000 7f253ca8f700 1 -- start start 2023-11-15T03:06:01.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f253ca8f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2538066320 0x7f25380c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:01.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f253ca8f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2538066700 0x7f25380a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:01.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f253ca8f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25380acf10 0x7f25380a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:01.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f253ca8f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25380a7110 con 0x7f2538066320 2023-11-15T03:06:01.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f253ca8f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25380a6500 con 0x7f2538066700 2023-11-15T03:06:01.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f253ca8f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25380a67e0 con 0x7f25380acf10 2023-11-15T03:06:01.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f25377fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2538066320 0x7f25380c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:01.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f25377fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2538066320 0x7f25380c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51880/0 (socket says 172.21.15.121:51880) 2023-11-15T03:06:01.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f25377fe700 1 -- 172.21.15.121:0/2902917048 learned_addr learned my addr 172.21.15.121:0/2902917048 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:01.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f2536ffd700 1 --2- 172.21.15.121:0/2902917048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2538066700 0x7f25380a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:01.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f2537fff700 1 --2- 172.21.15.121:0/2902917048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25380acf10 0x7f25380a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:01.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f25377fe700 1 -- 172.21.15.121:0/2902917048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25380acf10 msgr2=0x7f25380a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:01.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f25377fe700 1 --2- 172.21.15.121:0/2902917048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25380acf10 0x7f25380a5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:01.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f25377fe700 1 -- 172.21.15.121:0/2902917048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2538066700 msgr2=0x7f25380a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:01.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f25377fe700 1 --2- 172.21.15.121:0/2902917048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2538066700 0x7f25380a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:01.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f25377fe700 1 -- 172.21.15.121:0/2902917048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c0092a0 con 0x7f2538066320 2023-11-15T03:06:01.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f25377fe700 1 --2- 172.21.15.121:0/2902917048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2538066320 0x7f25380c6190 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0x7f252800c590 tx=0x7f2528007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:01.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f251f7fe700 1 -- 172.21.15.121:0/2902917048 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25280072e0 con 0x7f2538066320 2023-11-15T03:06:01.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f251f7fe700 1 -- 172.21.15.121:0/2902917048 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2528007440 con 0x7f2538066320 2023-11-15T03:06:01.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f251f7fe700 1 -- 172.21.15.121:0/2902917048 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25280057a0 con 0x7f2538066320 2023-11-15T03:06:01.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f253ca8f700 1 -- 172.21.15.121:0/2902917048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25380a6ac0 con 0x7f2538066320 2023-11-15T03:06:01.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.566+0000 7f253ca8f700 1 -- 172.21.15.121:0/2902917048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25380a75a0 con 0x7f2538066320 2023-11-15T03:06:01.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.570+0000 7f253ca8f700 1 -- 172.21.15.121:0/2902917048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253806bcf0 con 0x7f2538066320 2023-11-15T03:06:01.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.570+0000 7f251f7fe700 1 -- 172.21.15.121:0/2902917048 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2528005bd0 con 0x7f2538066320 2023-11-15T03:06:01.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.574+0000 7f251f7fe700 1 --2- 172.21.15.121:0/2902917048 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f252405a510 0x7f252405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:01.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.574+0000 7f251f7fe700 1 -- 172.21.15.121:0/2902917048 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7470+0+0 (secure 0 0 0) 0x7f2528074440 con 0x7f2538066320 2023-11-15T03:06:01.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.574+0000 7f2536ffd700 1 --2- 172.21.15.121:0/2902917048 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f252405a510 0x7f252405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:01.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.574+0000 7f2536ffd700 1 --2- 172.21.15.121:0/2902917048 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f252405a510 0x7f252405c9d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f2538066b40 tx=0x7f252c016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:01.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.578+0000 7f251f7fe700 1 -- 172.21.15.121:0/2902917048 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f25280476e0 con 0x7f2538066320 2023-11-15T03:06:01.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.774+0000 7f253ca8f700 1 -- 172.21.15.121:0/2902917048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2538154f60 con 0x7f2538066320 2023-11-15T03:06:01.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.774+0000 7f251f7fe700 1 -- 172.21.15.121:0/2902917048 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 168 v168) v1 ==== 95+0+4493 (secure 0 0 0) 0x7f252804ad00 con 0x7f2538066320 2023-11-15T03:06:01.785 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:01.785 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":168,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/228280964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":228280964},{"type":"v1","addr":"172.21.15.121:6836","nonce":228280964}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":167,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:05:56.160979+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:01.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.778+0000 7f253ca8f700 1 -- 172.21.15.121:0/2902917048 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f252405a510 msgr2=0x7f252405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:01.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.778+0000 7f253ca8f700 1 --2- 172.21.15.121:0/2902917048 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f252405a510 0x7f252405c9d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f2538066b40 tx=0x7f252c016000).stop 2023-11-15T03:06:01.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.778+0000 7f253ca8f700 1 -- 172.21.15.121:0/2902917048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2538066320 msgr2=0x7f25380c6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:01.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.778+0000 7f253ca8f700 1 --2- 172.21.15.121:0/2902917048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2538066320 0x7f25380c6190 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0x7f252800c590 tx=0x7f2528007c80).stop 2023-11-15T03:06:01.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.778+0000 7f253ca8f700 1 -- 172.21.15.121:0/2902917048 shutdown_connections 2023-11-15T03:06:01.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.778+0000 7f253ca8f700 1 --2- 172.21.15.121:0/2902917048 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f252405a510 0x7f252405c9d0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:01.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.778+0000 7f253ca8f700 1 --2- 172.21.15.121:0/2902917048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25380acf10 0x7f25380a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:01.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.778+0000 7f253ca8f700 1 --2- 172.21.15.121:0/2902917048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2538066700 0x7f25380a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:01.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.778+0000 7f253ca8f700 1 --2- 172.21.15.121:0/2902917048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2538066320 0x7f25380c6190 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:01.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.778+0000 7f253ca8f700 1 -- 172.21.15.121:0/2902917048 >> 172.21.15.121:0/2902917048 conn(0x7f2538067a60 msgr2=0x7f25380aaba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:01.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.778+0000 7f253ca8f700 1 -- 172.21.15.121:0/2902917048 shutdown_connections 2023-11-15T03:06:01.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:01.778+0000 7f253ca8f700 1 -- 172.21.15.121:0/2902917048 wait complete. 2023-11-15T03:06:01.790 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 168 2023-11-15T03:06:01.808 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 100.00000000.inode' led to DAMAGED state 2023-11-15T03:06:01.809 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 1.00000000' 2023-11-15T03:06:01.809 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:06:01.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:06:01.810 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:06:01.827 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:06:01.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:06:01.829 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:06:02.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.026+0000 7fa735cb9700 1 Processor -- start 2023-11-15T03:06:02.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.026+0000 7fa735cb9700 1 -- start start 2023-11-15T03:06:02.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.030+0000 7fa735cb9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa730151b00 0x7fa730152b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:02.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.030+0000 7fa735cb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa730068a70 0x7fa730068ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:02.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.030+0000 7fa735cb9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7300bbd50 con 0x7fa730151720 2023-11-15T03:06:02.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa735cb9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7300bc030 con 0x7fa730151b00 2023-11-15T03:06:02.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa735cb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7300693e0 con 0x7fa730068a70 2023-11-15T03:06:02.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa72ffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa730151b00 0x7fa730152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:02.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa7354b8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa730068a70 0x7fa730068ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:02.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa72ffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa730151b00 0x7fa730152b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:50668/0 (socket says 172.21.15.121:50668) 2023-11-15T03:06:02.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa72ffff700 1 -- 172.21.15.121:0/43597044 learned_addr learned my addr 172.21.15.121:0/43597044 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:02.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa72ffff700 1 -- 172.21.15.121:0/43597044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa730068a70 msgr2=0x7fa730068ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:02.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa72ffff700 1 --2- 172.21.15.121:0/43597044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa730068a70 0x7fa730068ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:02.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa72ffff700 1 -- 172.21.15.121:0/43597044 >> v1:172.21.15.190:6789/0 conn(0x7fa730151720 legacy=0x7fa7301554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:02.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa72ffff700 1 -- 172.21.15.121:0/43597044 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa730155d20 con 0x7fa730151b00 2023-11-15T03:06:02.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa72f7fe700 1 -- 172.21.15.121:0/43597044 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1313616126 0 0) 0x7fa7300bbd50 con 0x7fa730151720 2023-11-15T03:06:02.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa72ffff700 1 --2- 172.21.15.121:0/43597044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa730151b00 0x7fa730152b90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7fa71c006fd0 tx=0x7fa71c0090f0).ready entity=mon.1 client_cookie=bced50b48ce8c24e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:02.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa72f7fe700 1 -- 172.21.15.121:0/43597044 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c00b580 con 0x7fa730151b00 2023-11-15T03:06:02.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa72f7fe700 1 -- 172.21.15.121:0/43597044 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c00b6e0 con 0x7fa730151b00 2023-11-15T03:06:02.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa72f7fe700 1 -- 172.21.15.121:0/43597044 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c00ba60 con 0x7fa730151b00 2023-11-15T03:06:02.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa735cb9700 1 -- 172.21.15.121:0/43597044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa730151b00 msgr2=0x7fa730152b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:02.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa735cb9700 1 --2- 172.21.15.121:0/43597044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa730151b00 0x7fa730152b90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7fa71c006fd0 tx=0x7fa71c0090f0).stop 2023-11-15T03:06:02.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa735cb9700 1 -- 172.21.15.121:0/43597044 shutdown_connections 2023-11-15T03:06:02.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa735cb9700 1 --2- 172.21.15.121:0/43597044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa730068a70 0x7fa730068ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:02.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa735cb9700 1 --2- 172.21.15.121:0/43597044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa730151b00 0x7fa730152b90 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:02.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.034+0000 7fa735cb9700 1 -- 172.21.15.121:0/43597044 >> 172.21.15.121:0/43597044 conn(0x7fa730065fa0 msgr2=0x7fa7300663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:02.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.038+0000 7fa735cb9700 1 -- 172.21.15.121:0/43597044 shutdown_connections 2023-11-15T03:06:02.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.038+0000 7fa735cb9700 1 -- 172.21.15.121:0/43597044 wait complete. 2023-11-15T03:06:02.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.038+0000 7fa735cb9700 1 Processor -- start 2023-11-15T03:06:02.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.038+0000 7fa735cb9700 1 -- start start 2023-11-15T03:06:02.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.038+0000 7fa735cb9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa730068a70 0x7fa7300c07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:02.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.038+0000 7fa735cb9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa730151720 0x7fa7300be3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:02.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.038+0000 7fa735cb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa730151b00 0x7fa73014cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:02.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.038+0000 7fa735cb9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7301624e0 con 0x7fa730151720 2023-11-15T03:06:02.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.038+0000 7fa735cb9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7300c6690 con 0x7fa730068a70 2023-11-15T03:06:02.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.038+0000 7fa735cb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7301626d0 con 0x7fa730151b00 2023-11-15T03:06:02.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.042+0000 7fa72ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa730151720 0x7fa7300be3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:02.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.042+0000 7fa72ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa730151720 0x7fa7300be3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51882/0 (socket says 172.21.15.121:51882) 2023-11-15T03:06:02.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.042+0000 7fa72ffff700 1 -- 172.21.15.121:0/1455828873 learned_addr learned my addr 172.21.15.121:0/1455828873 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:02.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.042+0000 7fa734cb7700 1 --2- 172.21.15.121:0/1455828873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa730068a70 0x7fa7300c07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:02.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.042+0000 7fa7354b8700 1 --2- 172.21.15.121:0/1455828873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa730151b00 0x7fa73014cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:02.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.042+0000 7fa72ffff700 1 -- 172.21.15.121:0/1455828873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa730151b00 msgr2=0x7fa73014cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:02.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.042+0000 7fa72ffff700 1 --2- 172.21.15.121:0/1455828873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa730151b00 0x7fa73014cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:02.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.042+0000 7fa72ffff700 1 -- 172.21.15.121:0/1455828873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa730068a70 msgr2=0x7fa7300c07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:02.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.042+0000 7fa72ffff700 1 --2- 172.21.15.121:0/1455828873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa730068a70 0x7fa7300c07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:02.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.042+0000 7fa72ffff700 1 -- 172.21.15.121:0/1455828873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c009370 con 0x7fa730151720 2023-11-15T03:06:02.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.042+0000 7fa7354b8700 1 --2- 172.21.15.121:0/1455828873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa730151b00 0x7fa73014cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:02.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.042+0000 7fa72ffff700 1 --2- 172.21.15.121:0/1455828873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa730151720 0x7fa7300be3c0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7fa71c009130 tx=0x7fa71c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:02.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.042+0000 7fa72cff9700 1 -- 172.21.15.121:0/1455828873 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c00b850 con 0x7fa730151720 2023-11-15T03:06:02.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.042+0000 7fa72cff9700 1 -- 172.21.15.121:0/1455828873 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c00b9b0 con 0x7fa730151720 2023-11-15T03:06:02.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.042+0000 7fa735cb9700 1 -- 172.21.15.121:0/1455828873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7300bea70 con 0x7fa730151720 2023-11-15T03:06:02.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.042+0000 7fa72cff9700 1 -- 172.21.15.121:0/1455828873 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c01b660 con 0x7fa730151720 2023-11-15T03:06:02.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.042+0000 7fa735cb9700 1 -- 172.21.15.121:0/1455828873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7300bf030 con 0x7fa730151720 2023-11-15T03:06:02.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.042+0000 7fa735cb9700 1 -- 172.21.15.121:0/1455828873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7300bdd80 con 0x7fa730151720 2023-11-15T03:06:02.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.046+0000 7fa72cff9700 1 -- 172.21.15.121:0/1455828873 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa71c01ba10 con 0x7fa730151720 2023-11-15T03:06:02.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.046+0000 7fa72cff9700 1 --2- 172.21.15.121:0/1455828873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa71405a510 0x7fa71405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:02.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.046+0000 7fa72cff9700 1 -- 172.21.15.121:0/1455828873 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7470+0+0 (secure 0 0 0) 0x7fa71c076710 con 0x7fa730151720 2023-11-15T03:06:02.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.046+0000 7fa734cb7700 1 --2- 172.21.15.121:0/1455828873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa71405a510 0x7fa71405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:02.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.046+0000 7fa734cb7700 1 --2- 172.21.15.121:0/1455828873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa71405a510 0x7fa71405c9d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7fa7300c0f30 tx=0x7fa724009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:02.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.054+0000 7fa72cff9700 1 -- 172.21.15.121:0/1455828873 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa71c0498f0 con 0x7fa730151720 2023-11-15T03:06:02.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:02.258+0000 7fa735cb9700 1 -- 172.21.15.121:0/1455828873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa730162890 con 0x7fa730151720 2023-11-15T03:06:03.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.214+0000 7fa72cff9700 1 -- 172.21.15.121:0/1455828873 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v169) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa71c029020 con 0x7fa730151720 2023-11-15T03:06:03.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.218+0000 7fa735cb9700 1 -- 172.21.15.121:0/1455828873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa71405a510 msgr2=0x7fa71405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:03.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.218+0000 7fa735cb9700 1 --2- 172.21.15.121:0/1455828873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa71405a510 0x7fa71405c9d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7fa7300c0f30 tx=0x7fa724009040).stop 2023-11-15T03:06:03.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.218+0000 7fa735cb9700 1 -- 172.21.15.121:0/1455828873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa730151720 msgr2=0x7fa7300be3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:03.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.218+0000 7fa735cb9700 1 --2- 172.21.15.121:0/1455828873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa730151720 0x7fa7300be3c0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7fa71c009130 tx=0x7fa71c017000).stop 2023-11-15T03:06:03.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.218+0000 7fa735cb9700 1 -- 172.21.15.121:0/1455828873 shutdown_connections 2023-11-15T03:06:03.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.218+0000 7fa735cb9700 1 --2- 172.21.15.121:0/1455828873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa71405a510 0x7fa71405c9d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:03.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.218+0000 7fa735cb9700 1 --2- 172.21.15.121:0/1455828873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa730151b00 0x7fa73014cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:03.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.218+0000 7fa735cb9700 1 --2- 172.21.15.121:0/1455828873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa730151720 0x7fa7300be3c0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:03.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.218+0000 7fa735cb9700 1 --2- 172.21.15.121:0/1455828873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa730068a70 0x7fa7300c07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:03.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.218+0000 7fa735cb9700 1 -- 172.21.15.121:0/1455828873 >> 172.21.15.121:0/1455828873 conn(0x7fa730065fa0 msgr2=0x7fa7300bc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:03.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.218+0000 7fa735cb9700 1 -- 172.21.15.121:0/1455828873 shutdown_connections 2023-11-15T03:06:03.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.218+0000 7fa735cb9700 1 -- 172.21.15.121:0/1455828873 wait complete. 2023-11-15T03:06:03.230 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:06:03.250 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:06:03.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.402+0000 7f52348f1700 1 Processor -- start 2023-11-15T03:06:03.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.402+0000 7f52348f1700 1 -- start start 2023-11-15T03:06:03.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.402+0000 7f52348f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5230153aa0 0x7f5230066680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:03.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.402+0000 7f52348f1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5230153e80 0x7f52300ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:03.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.402+0000 7f52348f1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52300c5c50 con 0x7f52300c0490 2023-11-15T03:06:03.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.402+0000 7f52348f1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5230066b70 con 0x7f5230153e80 2023-11-15T03:06:03.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.402+0000 7f52348f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52300c5920 con 0x7f5230153aa0 2023-11-15T03:06:03.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f522ed9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f52300c0490 0x7f5230062190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39524/0 (socket says 172.21.15.121:39524) 2023-11-15T03:06:03.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f522ed9d700 1 -- 172.21.15.121:0/3384428519 learned_addr learned my addr 172.21.15.121:0/3384428519 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:03.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f522dd9b700 1 --2- 172.21.15.121:0/3384428519 >> v2:172.21.15.190:3300/0 conn(0x7f5230153e80 0x7f52300ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:03.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f522e59c700 1 --2- 172.21.15.121:0/3384428519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5230153aa0 0x7f5230066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:03.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f522d59a700 1 -- 172.21.15.121:0/3384428519 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268391550 0 0) 0x7f52300c5c50 con 0x7f52300c0490 2023-11-15T03:06:03.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f522d59a700 1 -- 172.21.15.121:0/3384428519 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52180033f0 con 0x7f52300c0490 2023-11-15T03:06:03.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f522dd9b700 1 -- 172.21.15.121:0/3384428519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5230153aa0 msgr2=0x7f5230066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:03.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f522dd9b700 1 --2- 172.21.15.121:0/3384428519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5230153aa0 0x7f5230066680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:03.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f522dd9b700 1 -- 172.21.15.121:0/3384428519 >> v1:172.21.15.121:6789/0 conn(0x7f52300c0490 legacy=0x7f5230062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:03.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f522dd9b700 1 -- 172.21.15.121:0/3384428519 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5230066d50 con 0x7f5230153e80 2023-11-15T03:06:03.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f522dd9b700 1 --2- 172.21.15.121:0/3384428519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5230153e80 0x7f52300ad150 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f5224009760 tx=0x7f5224009060).ready entity=mon.1 client_cookie=44fc630c97a55479 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:03.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f522d59a700 1 -- 172.21.15.121:0/3384428519 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522400d4a0 con 0x7f5230153e80 2023-11-15T03:06:03.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f522d59a700 1 -- 172.21.15.121:0/3384428519 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522400d600 con 0x7f5230153e80 2023-11-15T03:06:03.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f522d59a700 1 -- 172.21.15.121:0/3384428519 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522400d980 con 0x7f5230153e80 2023-11-15T03:06:03.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f52348f1700 1 -- 172.21.15.121:0/3384428519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5230153e80 msgr2=0x7f52300ad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:03.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f52348f1700 1 --2- 172.21.15.121:0/3384428519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5230153e80 0x7f52300ad150 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f5224009760 tx=0x7f5224009060).stop 2023-11-15T03:06:03.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f52348f1700 1 -- 172.21.15.121:0/3384428519 shutdown_connections 2023-11-15T03:06:03.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f52348f1700 1 --2- 172.21.15.121:0/3384428519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5230153e80 0x7f52300ad150 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:03.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f52348f1700 1 --2- 172.21.15.121:0/3384428519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5230153aa0 0x7f5230066680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:03.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f52348f1700 1 -- 172.21.15.121:0/3384428519 >> 172.21.15.121:0/3384428519 conn(0x7f5230067a60 msgr2=0x7f5230067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:03.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f52348f1700 1 -- 172.21.15.121:0/3384428519 shutdown_connections 2023-11-15T03:06:03.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.406+0000 7f52348f1700 1 -- 172.21.15.121:0/3384428519 wait complete. 2023-11-15T03:06:03.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.410+0000 7f52348f1700 1 Processor -- start 2023-11-15T03:06:03.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.410+0000 7f52348f1700 1 -- start start 2023-11-15T03:06:03.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f52348f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52300c0490 0x7f5230152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:03.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f52348f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5230153aa0 0x7f5230167d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:03.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f52348f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5230153e80 0x7f5230163f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:03.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f52348f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5230167870 con 0x7f5230153aa0 2023-11-15T03:06:03.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f52348f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5230164dd0 con 0x7f52300c0490 2023-11-15T03:06:03.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f52348f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5230164810 con 0x7f5230153e80 2023-11-15T03:06:03.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f522dd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5230153aa0 0x7f5230167d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:03.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f522dd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5230153aa0 0x7f5230167d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51894/0 (socket says 172.21.15.121:51894) 2023-11-15T03:06:03.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f522dd9b700 1 -- 172.21.15.121:0/3784539372 learned_addr learned my addr 172.21.15.121:0/3784539372 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:03.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f522e59c700 1 --2- 172.21.15.121:0/3784539372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52300c0490 0x7f5230152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:03.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f522ed9d700 1 --2- 172.21.15.121:0/3784539372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5230153e80 0x7f5230163f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:03.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f522dd9b700 1 -- 172.21.15.121:0/3784539372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5230153e80 msgr2=0x7f5230163f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:03.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f522dd9b700 1 --2- 172.21.15.121:0/3784539372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5230153e80 0x7f5230163f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:03.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f522dd9b700 1 -- 172.21.15.121:0/3784539372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52300c0490 msgr2=0x7f5230152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:03.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f522dd9b700 1 --2- 172.21.15.121:0/3784539372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52300c0490 0x7f5230152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:03.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f522dd9b700 1 -- 172.21.15.121:0/3784539372 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52240092a0 con 0x7f5230153aa0 2023-11-15T03:06:03.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f522dd9b700 1 --2- 172.21.15.121:0/3784539372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5230153aa0 0x7f5230167d00 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7f52300cbb20 tx=0x7f5224016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:03.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f521e7fc700 1 -- 172.21.15.121:0/3784539372 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522400d4a0 con 0x7f5230153aa0 2023-11-15T03:06:03.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f521e7fc700 1 -- 172.21.15.121:0/3784539372 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522400dd30 con 0x7f5230153aa0 2023-11-15T03:06:03.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f521e7fc700 1 -- 172.21.15.121:0/3784539372 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522400dea0 con 0x7f5230153aa0 2023-11-15T03:06:03.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f52348f1700 1 -- 172.21.15.121:0/3784539372 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5230164af0 con 0x7f5230153aa0 2023-11-15T03:06:03.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.414+0000 7f52348f1700 1 -- 172.21.15.121:0/3784539372 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52301681f0 con 0x7f5230153aa0 2023-11-15T03:06:03.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.418+0000 7f52348f1700 1 -- 172.21.15.121:0/3784539372 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52301679e0 con 0x7f5230153aa0 2023-11-15T03:06:03.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.418+0000 7f521e7fc700 1 -- 172.21.15.121:0/3784539372 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f522401d640 con 0x7f5230153aa0 2023-11-15T03:06:03.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.418+0000 7f521e7fc700 1 --2- 172.21.15.121:0/3784539372 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f521005a510 0x7f521005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:03.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.418+0000 7f521e7fc700 1 -- 172.21.15.121:0/3784539372 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7470+0+0 (secure 0 0 0) 0x7f5224076360 con 0x7f5230153aa0 2023-11-15T03:06:03.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.418+0000 7f522e59c700 1 --2- 172.21.15.121:0/3784539372 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f521005a510 0x7f521005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:03.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.418+0000 7f522e59c700 1 --2- 172.21.15.121:0/3784539372 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f521005a510 0x7f521005c9d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f5220009c30 tx=0x7f5220009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:03.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.426+0000 7f521e7fc700 1 -- 172.21.15.121:0/3784539372 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5224049600 con 0x7f5230153aa0 2023-11-15T03:06:03.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:03.646+0000 7f52348f1700 1 -- 172.21.15.121:0/3784539372 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f5230047060 con 0x7f5230153aa0 2023-11-15T03:06:04.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.234+0000 7f521e7fc700 1 -- 172.21.15.121:0/3784539372 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v170) v1 ==== 101+0+0 (secure 0 0 0) 0x7f522400d980 con 0x7f5230153aa0 2023-11-15T03:06:04.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.238+0000 7f52348f1700 1 -- 172.21.15.121:0/3784539372 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f521005a510 msgr2=0x7f521005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.238+0000 7f52348f1700 1 --2- 172.21.15.121:0/3784539372 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f521005a510 0x7f521005c9d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f5220009c30 tx=0x7f5220009060).stop 2023-11-15T03:06:04.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.238+0000 7f52348f1700 1 -- 172.21.15.121:0/3784539372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5230153aa0 msgr2=0x7f5230167d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.238+0000 7f52348f1700 1 --2- 172.21.15.121:0/3784539372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5230153aa0 0x7f5230167d00 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7f52300cbb20 tx=0x7f5224016000).stop 2023-11-15T03:06:04.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.238+0000 7f52348f1700 1 -- 172.21.15.121:0/3784539372 shutdown_connections 2023-11-15T03:06:04.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.238+0000 7f52348f1700 1 --2- 172.21.15.121:0/3784539372 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f521005a510 0x7f521005c9d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.238+0000 7f52348f1700 1 --2- 172.21.15.121:0/3784539372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5230153e80 0x7f5230163f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.238+0000 7f52348f1700 1 --2- 172.21.15.121:0/3784539372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5230153aa0 0x7f5230167d00 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.238+0000 7f52348f1700 1 --2- 172.21.15.121:0/3784539372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52300c0490 0x7f5230152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.238+0000 7f52348f1700 1 -- 172.21.15.121:0/3784539372 >> 172.21.15.121:0/3784539372 conn(0x7f5230067a60 msgr2=0x7f52300c6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:04.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.238+0000 7f52348f1700 1 -- 172.21.15.121:0/3784539372 shutdown_connections 2023-11-15T03:06:04.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.238+0000 7f52348f1700 1 -- 172.21.15.121:0/3784539372 wait complete. 2023-11-15T03:06:04.250 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:06:04.268 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:06:04.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.322+0000 7f51522d8d00 1 Processor -- start 2023-11-15T03:06:04.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.322+0000 7f51522d8d00 1 -- start start 2023-11-15T03:06:04.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f51522d8d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a49236e7e0 0x55a49242b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f51522d8d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a49242c650 0x55a492369ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f51522d8d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a49242ca30 0x55a4923b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f51522d8d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a492318cc0 con 0x55a49242c650 2023-11-15T03:06:04.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f51522d8d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4922e3220 con 0x55a49236e7e0 2023-11-15T03:06:04.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f51522d8d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a49239d680 con 0x55a49242ca30 2023-11-15T03:06:04.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f514f396700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a49242c650 0x55a492369ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f514fb97700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a49236e7e0 0x55a49242b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f514f396700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a49242c650 0x55a492369ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51896/0 (socket says 172.21.15.121:51896) 2023-11-15T03:06:04.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f514f396700 1 -- 172.21.15.121:0/805138254 learned_addr learned my addr 172.21.15.121:0/805138254 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:04.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f5150398700 1 --2- 172.21.15.121:0/805138254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a49242ca30 0x55a4923b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f514fb97700 1 -- 172.21.15.121:0/805138254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a49242ca30 msgr2=0x55a4923b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:04.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f514fb97700 1 --2- 172.21.15.121:0/805138254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a49242ca30 0x55a4923b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f514fb97700 1 -- 172.21.15.121:0/805138254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a49242c650 msgr2=0x55a492369ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:04.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f514fb97700 1 --2- 172.21.15.121:0/805138254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a49242c650 0x55a492369ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f514fb97700 1 -- 172.21.15.121:0/805138254 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5140005cf0 con 0x55a49236e7e0 2023-11-15T03:06:04.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f514f396700 1 --2- 172.21.15.121:0/805138254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a49242c650 0x55a492369ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:06:04.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f514fb97700 1 --2- 172.21.15.121:0/805138254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a49236e7e0 0x55a49242b3b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f5140013000 tx=0x7f5140016000).ready entity=mon.1 client_cookie=e7dcc6a37f7ad5bb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:04.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f514e394700 1 -- 172.21.15.121:0/805138254 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5140008700 con 0x55a49236e7e0 2023-11-15T03:06:04.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f514e394700 1 -- 172.21.15.121:0/805138254 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5140000bc0 con 0x55a49236e7e0 2023-11-15T03:06:04.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.326+0000 7f514e394700 1 -- 172.21.15.121:0/805138254 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514001d470 con 0x55a49236e7e0 2023-11-15T03:06:04.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.330+0000 7f51522d8d00 1 -- 172.21.15.121:0/805138254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a49236e7e0 msgr2=0x55a49242b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.330+0000 7f51522d8d00 1 --2- 172.21.15.121:0/805138254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a49236e7e0 0x55a49242b3b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f5140013000 tx=0x7f5140016000).stop 2023-11-15T03:06:04.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.330+0000 7f51522d8d00 1 -- 172.21.15.121:0/805138254 shutdown_connections 2023-11-15T03:06:04.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.330+0000 7f51522d8d00 1 --2- 172.21.15.121:0/805138254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a49242ca30 0x55a4923b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.330+0000 7f51522d8d00 1 --2- 172.21.15.121:0/805138254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a49242c650 0x55a492369ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.330+0000 7f51522d8d00 1 --2- 172.21.15.121:0/805138254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a49236e7e0 0x55a49242b3b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.330+0000 7f51522d8d00 1 -- 172.21.15.121:0/805138254 >> 172.21.15.121:0/805138254 conn(0x55a4923b48f0 msgr2=0x55a49239d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:04.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.330+0000 7f51522d8d00 1 -- 172.21.15.121:0/805138254 shutdown_connections 2023-11-15T03:06:04.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.330+0000 7f51522d8d00 1 -- 172.21.15.121:0/805138254 wait complete. 2023-11-15T03:06:04.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.330+0000 7f51522d8d00 1 Processor -- start 2023-11-15T03:06:04.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.330+0000 7f51522d8d00 1 -- start start 2023-11-15T03:06:04.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f51522d8d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a49236e7e0 0x55a4923ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f51522d8d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a49242c650 0x55a492429850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f51522d8d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a49242ca30 0x55a4923aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f51522d8d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4923992b0 con 0x55a49242ca30 2023-11-15T03:06:04.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f51522d8d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4923b15f0 con 0x55a49242c650 2023-11-15T03:06:04.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f51522d8d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4923b18d0 con 0x55a49236e7e0 2023-11-15T03:06:04.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f5150398700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a49242ca30 0x55a4923aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f5150398700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a49242ca30 0x55a4923aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51912/0 (socket says 172.21.15.121:51912) 2023-11-15T03:06:04.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 learned_addr learned my addr 172.21.15.121:0/5040815 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:04.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f514f396700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a49242c650 0x55a492429850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f514fb97700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a49236e7e0 0x55a4923ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a49236e7e0 msgr2=0x55a4923ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f5150398700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a49236e7e0 0x55a4923ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a49242c650 msgr2=0x55a492429850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f5150398700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a49242c650 0x55a492429850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514400a040 con 0x55a49242ca30 2023-11-15T03:06:04.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f514f396700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a49242c650 0x55a492429850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:06:04.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f5150398700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a49242ca30 0x55a4923aeef0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0x7f5148006530 tx=0x7f5148006ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:04.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f514cb91700 1 -- 172.21.15.121:0/5040815 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514800ede0 con 0x55a49242ca30 2023-11-15T03:06:04.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f514cb91700 1 -- 172.21.15.121:0/5040815 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5148014410 con 0x55a49242ca30 2023-11-15T03:06:04.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5140017040 con 0x55a49242ca30 2023-11-15T03:06:04.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f514cb91700 1 -- 172.21.15.121:0/5040815 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5148014790 con 0x55a49242ca30 2023-11-15T03:06:04.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.334+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4922dba70 con 0x55a49242ca30 2023-11-15T03:06:04.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.338+0000 7f514cb91700 1 -- 172.21.15.121:0/5040815 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5148014b80 con 0x55a49242ca30 2023-11-15T03:06:04.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.338+0000 7f514cb91700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f513c058ff0 0x7f513c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.338+0000 7f514cb91700 1 -- 172.21.15.121:0/5040815 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7470+0+0 (secure 0 0 0) 0x7f5148073920 con 0x55a49242ca30 2023-11-15T03:06:04.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.342+0000 7f514fb97700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f513c058ff0 0x7f513c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.354 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:06:04.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.342+0000 7f51522d8d00 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55a4923af3e0 0x55a49242b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.342+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4923ad1d0 con 0x55a4923af3e0 2023-11-15T03:06:04.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.342+0000 7f514f396700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55a4923af3e0 0x55a49242b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.342+0000 7f514fb97700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f513c058ff0 0x7f513c05b4b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f514001e000 tx=0x7f514001c000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:04.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.342+0000 7f514f396700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55a4923af3e0 0x55a49242b940 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:04.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.346+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v108'161 uv154 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4923af3e0 2023-11-15T03:06:04.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.346+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a4923ad5b0 con 0x55a4923af3e0 2023-11-15T03:06:04.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.346+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v108'162 uv162 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4923af3e0 2023-11-15T03:06:04.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.346+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492460020 con 0x55a4923af3e0 2023-11-15T03:06:04.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.350+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v108'163 uv163 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4923af3e0 2023-11-15T03:06:04.363 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:06:04.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.350+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4923ad960 con 0x55a4923af3e0 2023-11-15T03:06:04.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.354+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v108'164 uv164 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4923af3e0 2023-11-15T03:06:04.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.354+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4924613e0 con 0x55a4923af3e0 2023-11-15T03:06:04.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.354+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v108'165 uv165 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4923af3e0 2023-11-15T03:06:04.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.354+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4924603d0 con 0x55a4923af3e0 2023-11-15T03:06:04.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.358+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v108'166 uv166 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4923af3e0 2023-11-15T03:06:04.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.358+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492460800 con 0x55a4923af3e0 2023-11-15T03:06:04.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.358+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v108'167 uv160 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4923af3e0 2023-11-15T03:06:04.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.358+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a492460c00 con 0x55a4923af3e0 2023-11-15T03:06:04.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.362+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v108'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4923af3e0 2023-11-15T03:06:04.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.362+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492460c00 con 0x55a4923af3e0 2023-11-15T03:06:04.375 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:06:04.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.366+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v108'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4923af3e0 2023-11-15T03:06:04.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.366+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492461000 con 0x55a4923af3e0 2023-11-15T03:06:04.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.370+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v108'170 uv170 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4923af3e0 2023-11-15T03:06:04.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.370+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492460bb0 con 0x55a4923af3e0 2023-11-15T03:06:04.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.374+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v108'171 uv171 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4923af3e0 2023-11-15T03:06:04.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.374+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492460bb0 con 0x55a4923af3e0 2023-11-15T03:06:04.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.378+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v108'172 uv172 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4923af3e0 2023-11-15T03:06:04.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.378+0000 7f51522d8d00 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55a492460f90 0x55a492463cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.378+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492460bb0 con 0x55a492460f90 2023-11-15T03:06:04.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.378+0000 7f514fb97700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55a492460f90 0x55a492463cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.378+0000 7f514fb97700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55a492460f90 0x55a492463cb0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:04.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.382+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v108'252 uv239 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.382+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a4924641a0 con 0x55a492460f90 2023-11-15T03:06:04.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.386+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v108'253 uv253 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.386+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492464550 con 0x55a492460f90 2023-11-15T03:06:04.397 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:06:04.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.390+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v108'254 uv254 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.390+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492465910 con 0x55a492460f90 2023-11-15T03:06:04.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.390+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v108'255 uv255 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.390+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492465cc0 con 0x55a492460f90 2023-11-15T03:06:04.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.394+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v108'256 uv256 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.394+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492464900 con 0x55a492460f90 2023-11-15T03:06:04.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.398+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v108'257 uv257 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.398+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492465230 con 0x55a492460f90 2023-11-15T03:06:04.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.398+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v108'258 uv258 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.398+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4924651f0 con 0x55a492460f90 2023-11-15T03:06:04.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.402+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v108'259 uv245 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.402+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a4924651f0 con 0x55a492460f90 2023-11-15T03:06:04.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.406+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v108'260 uv260 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.406+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4924651f0 con 0x55a492460f90 2023-11-15T03:06:04.416 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:06:04.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.406+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v108'261 uv261 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.406+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4924651f0 con 0x55a492460f90 2023-11-15T03:06:04.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.410+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v108'262 uv262 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.410+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4924651f0 con 0x55a492460f90 2023-11-15T03:06:04.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.410+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v108'263 uv263 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.410+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4924651f0 con 0x55a492460f90 2023-11-15T03:06:04.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.414+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v108'264 uv264 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.414+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4924651f0 con 0x55a492460f90 2023-11-15T03:06:04.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.418+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v108'265 uv251 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.418+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a4924651f0 con 0x55a492460f90 2023-11-15T03:06:04.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.418+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v108'266 uv266 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.418+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4924651f0 con 0x55a492460f90 2023-11-15T03:06:04.431 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:06:04.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.422+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v108'267 uv267 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.422+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4924651f0 con 0x55a492460f90 2023-11-15T03:06:04.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.426+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v108'268 uv268 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.426+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4924651f0 con 0x55a492460f90 2023-11-15T03:06:04.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.426+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v108'269 uv269 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.426+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4924651f0 con 0x55a492460f90 2023-11-15T03:06:04.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.430+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v108'270 uv270 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.430+0000 7f51522d8d00 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a4924665d0 0x55a4924689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.430+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4924651f0 con 0x55a4924665d0 2023-11-15T03:06:04.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.430+0000 7f514f396700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a4924665d0 0x55a4924689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.430+0000 7f514f396700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a4924665d0 0x55a4924689c0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:04.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.434+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v108'138 uv132 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4924665d0 2023-11-15T03:06:04.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.434+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a492468eb0 con 0x55a4924665d0 2023-11-15T03:06:04.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.438+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v108'139 uv139 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4924665d0 2023-11-15T03:06:04.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.438+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492468eb0 con 0x55a4924665d0 2023-11-15T03:06:04.449 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:06:04.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.438+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v108'140 uv140 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4924665d0 2023-11-15T03:06:04.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.438+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4924693a0 con 0x55a4924665d0 2023-11-15T03:06:04.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.442+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v108'141 uv141 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4924665d0 2023-11-15T03:06:04.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.442+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4924693a0 con 0x55a4924665d0 2023-11-15T03:06:04.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.446+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v108'142 uv142 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4924665d0 2023-11-15T03:06:04.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.446+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4924693a0 con 0x55a4924665d0 2023-11-15T03:06:04.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.446+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v108'143 uv137 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4924665d0 2023-11-15T03:06:04.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.446+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a4924693a0 con 0x55a4924665d0 2023-11-15T03:06:04.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.450+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v108'144 uv144 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4924665d0 2023-11-15T03:06:04.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.450+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4924693a0 con 0x55a4924665d0 2023-11-15T03:06:04.463 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:06:04.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.454+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v108'145 uv145 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4924665d0 2023-11-15T03:06:04.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.454+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4924693a0 con 0x55a4924665d0 2023-11-15T03:06:04.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.458+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v108'146 uv146 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4924665d0 2023-11-15T03:06:04.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.458+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a4924693a0 con 0x55a4924665d0 2023-11-15T03:06:04.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.458+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v108'147 uv147 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4924665d0 2023-11-15T03:06:04.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.458+0000 7f51522d8d00 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55a4924698b0 0x55a49246bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.458+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a49246c190 con 0x55a4924698b0 2023-11-15T03:06:04.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.462+0000 7f5150398700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55a4924698b0 0x55a49246bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.462+0000 7f5150398700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55a4924698b0 0x55a49246bca0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:04.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.462+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v108'170 uv163 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.462+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a49246c540 con 0x55a4924698b0 2023-11-15T03:06:04.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.466+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v108'171 uv171 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.466+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a49246c8f0 con 0x55a4924698b0 2023-11-15T03:06:04.481 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:06:04.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.470+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v108'172 uv172 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.470+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492468f90 con 0x55a4924698b0 2023-11-15T03:06:04.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.474+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v108'173 uv173 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.474+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492471320 con 0x55a4924698b0 2023-11-15T03:06:04.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.478+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v108'174 uv174 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.478+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a49246cca0 con 0x55a4924698b0 2023-11-15T03:06:04.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.482+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v108'175 uv169 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.482+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a49246d0a0 con 0x55a4924698b0 2023-11-15T03:06:04.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.482+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v108'176 uv176 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.482+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a49246da90 con 0x55a4924698b0 2023-11-15T03:06:04.495 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:06:04.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.486+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v108'177 uv177 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.486+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a49246da50 con 0x55a4924698b0 2023-11-15T03:06:04.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.490+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v108'178 uv178 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.490+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a49246eef0 con 0x55a4924698b0 2023-11-15T03:06:04.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.490+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v108'179 uv179 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.490+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a49246dab0 con 0x55a4924698b0 2023-11-15T03:06:04.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.498+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v108'180 uv180 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.498+0000 7f51522d8d00 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55a49246da70 0x55a492470560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.498+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492470c00 con 0x55a49246da70 2023-11-15T03:06:04.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.498+0000 7f514fb97700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55a49246da70 0x55a492470560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.498+0000 7f514fb97700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55a49246da70 0x55a492470560 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:04.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.502+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v108'81 uv80 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a49246da70 2023-11-15T03:06:04.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.502+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a492470c00 con 0x55a49246da70 2023-11-15T03:06:04.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.502+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v108'82 uv82 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a49246da70 2023-11-15T03:06:04.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.502+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492470c00 con 0x55a49246da70 2023-11-15T03:06:04.515 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:06:04.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.506+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v108'83 uv83 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a49246da70 2023-11-15T03:06:04.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.506+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492470c00 con 0x55a49246da70 2023-11-15T03:06:04.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.510+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v108'84 uv84 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a49246da70 2023-11-15T03:06:04.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.510+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492470c00 con 0x55a49246da70 2023-11-15T03:06:04.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.510+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v108'85 uv85 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a49246da70 2023-11-15T03:06:04.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.510+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492470c00 con 0x55a49246da70 2023-11-15T03:06:04.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.514+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v108'86 uv86 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a49246da70 2023-11-15T03:06:04.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.514+0000 7f51522d8d00 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a492471c30 0x55a492472010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.514+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474790 con 0x55a492471c30 2023-11-15T03:06:04.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.514+0000 7f514f396700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a492471c30 0x55a492472010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.514+0000 7f514f396700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a492471c30 0x55a492472010 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:04.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.518+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v108'60 uv59 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.518+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a492474790 con 0x55a492471c30 2023-11-15T03:06:04.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.522+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v108'61 uv61 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.522+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474790 con 0x55a492471c30 2023-11-15T03:06:04.533 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:06:04.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.522+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v108'62 uv62 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.522+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492471c30 2023-11-15T03:06:04.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.526+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v108'63 uv63 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.526+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492471c30 2023-11-15T03:06:04.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.530+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v108'81 uv80 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.530+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492471c30 2023-11-15T03:06:04.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.530+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v108'82 uv82 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.530+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492471c30 2023-11-15T03:06:04.544 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:06:04.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.534+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v108'83 uv83 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.534+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492475ba0 con 0x55a492471c30 2023-11-15T03:06:04.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.538+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v108'84 uv84 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.538+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492475ba0 con 0x55a492471c30 2023-11-15T03:06:04.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.538+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v108'85 uv85 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.538+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492471c30 2023-11-15T03:06:04.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.542+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v108'86 uv86 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.542+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492460f90 2023-11-15T03:06:04.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.546+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v108'149 uv143 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.546+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492460f90 2023-11-15T03:06:04.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.550+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v108'150 uv150 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.550+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492460f90 2023-11-15T03:06:04.561 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:06:04.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.550+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v108'151 uv151 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.554+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492475ba0 con 0x55a492460f90 2023-11-15T03:06:04.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.554+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v108'152 uv152 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.554+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492475ba0 con 0x55a492460f90 2023-11-15T03:06:04.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.558+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v108'153 uv153 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.558+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492460f90 2023-11-15T03:06:04.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.562+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v108'154 uv154 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.562+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492460f90 2023-11-15T03:06:04.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.566+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v108'155 uv148 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.566+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492460f90 2023-11-15T03:06:04.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.570+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v108'156 uv156 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.570+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492460f90 2023-11-15T03:06:04.582 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:06:04.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.574+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v108'157 uv157 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.574+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492460f90 2023-11-15T03:06:04.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.574+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v108'158 uv158 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.574+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492460f90 2023-11-15T03:06:04.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.578+0000 7f514fb97700 1 -- 172.21.15.121:0/5040815 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v108'159 uv159 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5140023020 con 0x55a492460f90 2023-11-15T03:06:04.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.578+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a4924665d0 2023-11-15T03:06:04.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.582+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v108'66 uv65 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4924665d0 2023-11-15T03:06:04.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.582+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a4924665d0 2023-11-15T03:06:04.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.582+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v108'67 uv67 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4924665d0 2023-11-15T03:06:04.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.582+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a4924665d0 2023-11-15T03:06:04.595 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:06:04.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.586+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v108'68 uv68 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4924665d0 2023-11-15T03:06:04.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.586+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a4924665d0 2023-11-15T03:06:04.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.590+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v108'69 uv69 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4924665d0 2023-11-15T03:06:04.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.590+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a4924665d0 2023-11-15T03:06:04.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.590+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v108'70 uv70 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a4924665d0 2023-11-15T03:06:04.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.590+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492471c30 2023-11-15T03:06:04.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.594+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v108'81 uv80 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.594+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492471c30 2023-11-15T03:06:04.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.598+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v108'82 uv82 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.598+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492471c30 2023-11-15T03:06:04.608 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:06:04.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.598+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v108'83 uv83 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.598+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492475ba0 con 0x55a492471c30 2023-11-15T03:06:04.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.602+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v108'84 uv84 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.602+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492475ba0 con 0x55a492471c30 2023-11-15T03:06:04.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.606+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v108'85 uv85 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.606+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492471c30 2023-11-15T03:06:04.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.606+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v108'86 uv86 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.606+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492471c30 2023-11-15T03:06:04.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.610+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v108'67 uv66 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.610+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492471c30 2023-11-15T03:06:04.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.614+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v108'68 uv68 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.614+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492471c30 2023-11-15T03:06:04.626 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:06:04.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.614+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v108'69 uv69 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.614+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492471c30 2023-11-15T03:06:04.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.618+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v108'70 uv70 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.618+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492471c30 2023-11-15T03:06:04.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.622+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v108'71 uv71 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.622+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492471c30 2023-11-15T03:06:04.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.626+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v108'66 uv65 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.626+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492471c30 2023-11-15T03:06:04.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.626+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v108'67 uv67 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.626+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492471c30 2023-11-15T03:06:04.639 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:06:04.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.630+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v108'68 uv68 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.630+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492474b90 con 0x55a492471c30 2023-11-15T03:06:04.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.634+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v108'69 uv69 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.634+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492477900 con 0x55a492471c30 2023-11-15T03:06:04.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.638+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v108'70 uv70 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.638+0000 7f51522d8d00 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55a492475590 0x55a492475970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.638+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492476040 con 0x55a492475590 2023-11-15T03:06:04.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.638+0000 7f5150398700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55a492475590 0x55a492475970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.638+0000 7f5150398700 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55a492475590 0x55a492475970 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:04.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.642+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v108'54 uv53 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a492475590 2023-11-15T03:06:04.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.642+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a492476040 con 0x55a492475590 2023-11-15T03:06:04.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.646+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v108'55 uv55 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a492475590 2023-11-15T03:06:04.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.646+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492476040 con 0x55a492475590 2023-11-15T03:06:04.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.646+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v108'56 uv56 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a492475590 2023-11-15T03:06:04.658 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:06:04.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.650+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492476040 con 0x55a492475590 2023-11-15T03:06:04.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.654+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v108'57 uv57 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a492475590 2023-11-15T03:06:04.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.654+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492476040 con 0x55a492471c30 2023-11-15T03:06:04.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.654+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v108'81 uv80 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.658+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a492476040 con 0x55a492471c30 2023-11-15T03:06:04.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.658+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v108'82 uv82 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.658+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492476040 con 0x55a492471c30 2023-11-15T03:06:04.671 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:06:04.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.662+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v108'83 uv83 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.662+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492476040 con 0x55a492471c30 2023-11-15T03:06:04.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.666+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v108'84 uv84 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.666+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492477450 con 0x55a492471c30 2023-11-15T03:06:04.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.666+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v108'85 uv85 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.666+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492477870 con 0x55a492471c30 2023-11-15T03:06:04.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.674+0000 7f514f396700 1 -- 172.21.15.121:0/5040815 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v108'86 uv86 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f514400ec60 con 0x55a492471c30 2023-11-15T03:06:04.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.674+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492477870 con 0x55a4924698b0 2023-11-15T03:06:04.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.674+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v108'69 uv68 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.674+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a492476040 con 0x55a4924698b0 2023-11-15T03:06:04.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.678+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v108'70 uv70 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.678+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492477870 con 0x55a4924698b0 2023-11-15T03:06:04.690 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:06:04.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.682+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v108'71 uv71 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.682+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492476040 con 0x55a4924698b0 2023-11-15T03:06:04.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.686+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v108'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.686+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492477870 con 0x55a4924698b0 2023-11-15T03:06:04.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.686+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v108'73 uv73 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.686+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492476040 con 0x55a4924698b0 2023-11-15T03:06:04.699 INFO:teuthology.orchestra.run.smithi121.stdout:Write #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:06:04.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.690+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v108'163 uv163 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.690+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492477870 con 0x55a4924698b0 2023-11-15T03:06:04.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.694+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v108'164 uv164 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.694+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492476040 con 0x55a4924698b0 2023-11-15T03:06:04.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.694+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v108'165 uv165 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.694+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492476040 con 0x55a4924698b0 2023-11-15T03:06:04.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.698+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 1.00000000 [create] v108'166 uv161 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.698+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x55a492476040 con 0x55a4924698b0 2023-11-15T03:06:04.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.702+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 1.00000000 [delete] v108'167 uv167 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.702+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492477870 con 0x55a4924698b0 2023-11-15T03:06:04.713 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:06:04.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.702+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v108'168 uv168 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.702+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492476040 con 0x55a4924698b0 2023-11-15T03:06:04.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.706+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v108'169 uv169 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.706+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492477870 con 0x55a4924698b0 2023-11-15T03:06:04.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.710+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v108'170 uv170 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.710+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492477870 con 0x55a4924698b0 2023-11-15T03:06:04.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.714+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [omap-set-header] v108'171 uv171 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.714+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e108) v8 -- 0x55a492476440 con 0x55a4924698b0 2023-11-15T03:06:04.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v108'172 uv172 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5148048b60 con 0x55a4924698b0 2023-11-15T03:06:04.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a4924665d0 msgr2=0x55a4924689c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a4924665d0 0x55a4924689c0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55a4923af3e0 msgr2=0x55a49242b940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55a4923af3e0 0x55a49242b940 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55a49246da70 msgr2=0x55a492470560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55a49246da70 0x55a492470560 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55a492460f90 msgr2=0x55a492463cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55a492460f90 0x55a492463cb0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55a492475590 msgr2=0x55a492475970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55a492475590 0x55a492475970 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55a4924698b0 msgr2=0x55a49246bca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55a4924698b0 0x55a49246bca0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 reap_dead start 2023-11-15T03:06:04.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f5150398700 1 -- 172.21.15.121:0/5040815 reap_dead start 2023-11-15T03:06:04.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a492471c30 msgr2=0x55a492472010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a492471c30 0x55a492472010 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f513c058ff0 msgr2=0x7f513c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f513c058ff0 0x7f513c05b4b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f514001e000 tx=0x7f514001c000).stop 2023-11-15T03:06:04.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a49242ca30 msgr2=0x55a4923aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a49242ca30 0x55a4923aeef0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0x7f5148006530 tx=0x7f5148006ae0).stop 2023-11-15T03:06:04.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 shutdown_connections 2023-11-15T03:06:04.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a492471c30 0x55a492472010 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f513c058ff0 0x7f513c05b4b0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 --2- 172.21.15.121:0/5040815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a49242ca30 0x55a4923aeef0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 >> 172.21.15.121:0/5040815 conn(0x55a4923b48f0 msgr2=0x55a49242a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:04.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 shutdown_connections 2023-11-15T03:06:04.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.718+0000 7f51522d8d00 1 -- 172.21.15.121:0/5040815 wait complete. 2023-11-15T03:06:04.734 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 1.00000000 2023-11-15T03:06:04.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.782+0000 7fa1788dad00 1 Processor -- start 2023-11-15T03:06:04.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.782+0000 7fa1788dad00 1 -- start start 2023-11-15T03:06:04.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.782+0000 7fa1788dad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56035ab01920 0x56035ab001f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.782+0000 7fa1788dad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56035ab492f0 0x56035abd5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.782+0000 7fa1788dad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035abd7a20 0x56035ab2e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.782+0000 7fa1788dad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035aa76220 con 0x56035ab01920 2023-11-15T03:06:04.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.782+0000 7fa1788dad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035abcdf40 con 0x56035abd7a20 2023-11-15T03:06:04.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.782+0000 7fa1788dad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56035ab305b0 con 0x56035ab492f0 2023-11-15T03:06:04.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.782+0000 7fa1761bc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56035ab01920 0x56035ab001f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.782+0000 7fa1761bc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56035ab01920 0x56035ab001f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34360/0 (socket says 172.21.15.121:34360) 2023-11-15T03:06:04.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.782+0000 7fa1761bc700 1 -- 172.21.15.121:0/2756849710 learned_addr learned my addr 172.21.15.121:0/2756849710 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:04.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1769bd700 1 --2- 172.21.15.121:0/2756849710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035abd7a20 0x56035ab2e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa16d9bb700 1 --2- 172.21.15.121:0/2756849710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56035ab492f0 0x56035abd5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1761bc700 1 -- 172.21.15.121:0/2756849710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56035ab492f0 msgr2=0x56035abd5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:04.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1761bc700 1 --2- 172.21.15.121:0/2756849710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56035ab492f0 0x56035abd5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1761bc700 1 -- 172.21.15.121:0/2756849710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035abd7a20 msgr2=0x56035ab2e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:04.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1761bc700 1 --2- 172.21.15.121:0/2756849710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035abd7a20 0x56035ab2e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1761bc700 1 -- 172.21.15.121:0/2756849710 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17000a180 con 0x56035ab01920 2023-11-15T03:06:04.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa16d9bb700 1 --2- 172.21.15.121:0/2756849710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56035ab492f0 0x56035abd5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:04.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1761bc700 1 --2- 172.21.15.121:0/2756849710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56035ab01920 0x56035ab001f0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7fa168009950 tx=0x7fa16800c040).ready entity=mon.0 client_cookie=42b3441283a86fdd server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:04.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1751ba700 1 -- 172.21.15.121:0/2756849710 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16800f040 con 0x56035ab01920 2023-11-15T03:06:04.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1751ba700 1 -- 172.21.15.121:0/2756849710 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa168008ac0 con 0x56035ab01920 2023-11-15T03:06:04.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1751ba700 1 -- 172.21.15.121:0/2756849710 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa168008e40 con 0x56035ab01920 2023-11-15T03:06:04.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1788dad00 1 -- 172.21.15.121:0/2756849710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56035ab01920 msgr2=0x56035ab001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1788dad00 1 --2- 172.21.15.121:0/2756849710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56035ab01920 0x56035ab001f0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7fa168009950 tx=0x7fa16800c040).stop 2023-11-15T03:06:04.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1788dad00 1 -- 172.21.15.121:0/2756849710 shutdown_connections 2023-11-15T03:06:04.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1788dad00 1 --2- 172.21.15.121:0/2756849710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035abd7a20 0x56035ab2e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1788dad00 1 --2- 172.21.15.121:0/2756849710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56035ab492f0 0x56035abd5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1788dad00 1 --2- 172.21.15.121:0/2756849710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56035ab01920 0x56035ab001f0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1788dad00 1 -- 172.21.15.121:0/2756849710 >> 172.21.15.121:0/2756849710 conn(0x56035ab478f0 msgr2=0x56035abd5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:04.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1788dad00 1 -- 172.21.15.121:0/2756849710 shutdown_connections 2023-11-15T03:06:04.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1788dad00 1 -- 172.21.15.121:0/2756849710 wait complete. 2023-11-15T03:06:04.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1788dad00 1 Processor -- start 2023-11-15T03:06:04.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.786+0000 7fa1788dad00 1 -- start start 2023-11-15T03:06:04.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa1788dad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035ab01920 0x56035abd4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa1788dad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56035ab492f0 0x56035abd4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa1788dad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56035abd7a20 0x56035ab41ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa1788dad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035abe2630 con 0x56035ab492f0 2023-11-15T03:06:04.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa1788dad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035abe0c70 con 0x56035ab01920 2023-11-15T03:06:04.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa1788dad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56035abd33c0 con 0x56035abd7a20 2023-11-15T03:06:04.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa16d9bb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56035ab492f0 0x56035abd4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa16d9bb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56035ab492f0 0x56035abd4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34376/0 (socket says 172.21.15.121:34376) 2023-11-15T03:06:04.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa16d9bb700 1 -- 172.21.15.121:0/3780937974 learned_addr learned my addr 172.21.15.121:0/3780937974 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:04.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa1761bc700 1 --2- 172.21.15.121:0/3780937974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035ab01920 0x56035abd4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa1769bd700 1 --2- 172.21.15.121:0/3780937974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56035abd7a20 0x56035ab41ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa16d9bb700 1 -- 172.21.15.121:0/3780937974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56035abd7a20 msgr2=0x56035ab41ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa16d9bb700 1 --2- 172.21.15.121:0/3780937974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56035abd7a20 0x56035ab41ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa16d9bb700 1 -- 172.21.15.121:0/3780937974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035ab01920 msgr2=0x56035abd4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa16d9bb700 1 --2- 172.21.15.121:0/3780937974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035ab01920 0x56035abd4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa16d9bb700 1 -- 172.21.15.121:0/3780937974 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa168009580 con 0x56035ab492f0 2023-11-15T03:06:04.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa16d9bb700 1 --2- 172.21.15.121:0/3780937974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56035ab492f0 0x56035abd4760 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7fa160001e50 tx=0x7fa160001860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:04.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa16f7fe700 1 -- 172.21.15.121:0/3780937974 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16000ad60 con 0x56035ab492f0 2023-11-15T03:06:04.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa16f7fe700 1 -- 172.21.15.121:0/3780937974 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa160008040 con 0x56035ab492f0 2023-11-15T03:06:04.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa1788dad00 1 -- 172.21.15.121:0/3780937974 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56035aa9f770 con 0x56035ab492f0 2023-11-15T03:06:04.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa16f7fe700 1 -- 172.21.15.121:0/3780937974 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160010680 con 0x56035ab492f0 2023-11-15T03:06:04.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.790+0000 7fa1788dad00 1 -- 172.21.15.121:0/3780937974 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56035aa6ea70 con 0x56035ab492f0 2023-11-15T03:06:04.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.794+0000 7fa16f7fe700 1 -- 172.21.15.121:0/3780937974 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa160010ab0 con 0x56035ab492f0 2023-11-15T03:06:04.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.794+0000 7fa16f7fe700 1 --2- 172.21.15.121:0/3780937974 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa158059020 0x7fa15805b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.794+0000 7fa16f7fe700 1 -- 172.21.15.121:0/3780937974 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7470+0+0 (secure 0 0 0) 0x7fa1600739b0 con 0x56035ab492f0 2023-11-15T03:06:04.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.794+0000 7fa1761bc700 1 --2- 172.21.15.121:0/3780937974 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa158059020 0x7fa15805b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.794+0000 7fa1788dad00 1 --2- 172.21.15.121:0/3780937974 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56035abd4c50 0x56035abea0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.794+0000 7fa1788dad00 1 -- 172.21.15.121:0/3780937974 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e108) v8 -- 0x56035abd3850 con 0x56035abd4c50 2023-11-15T03:06:04.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.798+0000 7fa1761bc700 1 --2- 172.21.15.121:0/3780937974 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa158059020 0x7fa15805b4e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7fa168009950 tx=0x7fa168018000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:04.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.798+0000 7fa1769bd700 1 --2- 172.21.15.121:0/3780937974 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56035abd4c50 0x56035abea0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.798+0000 7fa1769bd700 1 --2- 172.21.15.121:0/3780937974 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56035abd4c50 0x56035abea0b0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:04.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.802+0000 7fa1769bd700 1 -- 172.21.15.121:0/3780937974 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 1.00000000 [delete] v108'173 uv173 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa17000cd30 con 0x56035abd4c50 2023-11-15T03:06:04.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.802+0000 7fa1788dad00 1 -- 172.21.15.121:0/3780937974 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56035abd4c50 msgr2=0x56035abea0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.802+0000 7fa1788dad00 1 --2- 172.21.15.121:0/3780937974 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x56035abd4c50 0x56035abea0b0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.802+0000 7fa1788dad00 1 -- 172.21.15.121:0/3780937974 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa158059020 msgr2=0x7fa15805b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.802+0000 7fa1788dad00 1 --2- 172.21.15.121:0/3780937974 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa158059020 0x7fa15805b4e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7fa168009950 tx=0x7fa168018000).stop 2023-11-15T03:06:04.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.802+0000 7fa1788dad00 1 -- 172.21.15.121:0/3780937974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56035ab492f0 msgr2=0x56035abd4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.802+0000 7fa1788dad00 1 --2- 172.21.15.121:0/3780937974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56035ab492f0 0x56035abd4760 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7fa160001e50 tx=0x7fa160001860).stop 2023-11-15T03:06:04.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.802+0000 7fa1769bd700 1 -- 172.21.15.121:0/3780937974 reap_dead start 2023-11-15T03:06:04.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.802+0000 7fa1788dad00 1 -- 172.21.15.121:0/3780937974 shutdown_connections 2023-11-15T03:06:04.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.802+0000 7fa1788dad00 1 -- 172.21.15.121:0/3780937974 >> 172.21.15.121:0/3780937974 conn(0x56035ab478f0 msgr2=0x56035abd5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:04.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.802+0000 7fa1788dad00 1 -- 172.21.15.121:0/3780937974 shutdown_connections 2023-11-15T03:06:04.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.802+0000 7fa1788dad00 1 -- 172.21.15.121:0/3780937974 wait complete. 2023-11-15T03:06:04.816 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:06:04.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.978+0000 7f265728f700 1 Processor -- start 2023-11-15T03:06:04.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.978+0000 7f265728f700 1 -- start start 2023-11-15T03:06:04.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.982+0000 7f265728f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f26500acd00 0x7f26500ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:04.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.982+0000 7f265728f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2650065ea0 con 0x7f2650066330 2023-11-15T03:06:04.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.982+0000 7f265728f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2650066bb0 con 0x7f26500c5c70 2023-11-15T03:06:04.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.982+0000 7f265728f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2650065890 con 0x7f26500acd00 2023-11-15T03:06:04.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.982+0000 7f265628d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f26500acd00 0x7f26500ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:04.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.982+0000 7f2656a8e700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f2650066330 0x7f26501577e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:38778/0 (socket says 172.21.15.121:38778) 2023-11-15T03:06:04.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.982+0000 7f2656a8e700 1 -- 172.21.15.121:0/1630318762 learned_addr learned my addr 172.21.15.121:0/1630318762 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:04.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.982+0000 7f265628d700 1 -- 172.21.15.121:0/1630318762 >> v1:172.21.15.190:6789/0 conn(0x7f26500c5c70 legacy=0x7f26501536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.982+0000 7f265628d700 1 -- 172.21.15.121:0/1630318762 >> v1:172.21.15.121:6789/0 conn(0x7f2650066330 legacy=0x7f26501577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.982+0000 7f265628d700 1 -- 172.21.15.121:0/1630318762 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26500c64a0 con 0x7f26500acd00 2023-11-15T03:06:04.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.986+0000 7f265628d700 1 --2- 172.21.15.121:0/1630318762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26500acd00 0x7f26500ad0e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7f264c006fd0 tx=0x7f264c006f40).ready entity=mon.0 client_cookie=c3442f5455f07fd server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:04.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.986+0000 7f265528b700 1 -- 172.21.15.121:0/1630318762 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c005880 con 0x7f26500acd00 2023-11-15T03:06:04.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.986+0000 7f265528b700 1 -- 172.21.15.121:0/1630318762 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264c0059e0 con 0x7f26500acd00 2023-11-15T03:06:04.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.986+0000 7f265528b700 1 -- 172.21.15.121:0/1630318762 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c005d60 con 0x7f26500acd00 2023-11-15T03:06:04.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.986+0000 7f265728f700 1 -- 172.21.15.121:0/1630318762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26500acd00 msgr2=0x7f26500ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:04.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.986+0000 7f265728f700 1 --2- 172.21.15.121:0/1630318762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26500acd00 0x7f26500ad0e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7f264c006fd0 tx=0x7f264c006f40).stop 2023-11-15T03:06:04.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.986+0000 7f265728f700 1 -- 172.21.15.121:0/1630318762 shutdown_connections 2023-11-15T03:06:04.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.986+0000 7f265728f700 1 --2- 172.21.15.121:0/1630318762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26500acd00 0x7f26500ad0e0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:04.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.986+0000 7f265728f700 1 -- 172.21.15.121:0/1630318762 >> 172.21.15.121:0/1630318762 conn(0x7f2650063fc0 msgr2=0x7f2650067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:04.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.986+0000 7f265728f700 1 -- 172.21.15.121:0/1630318762 shutdown_connections 2023-11-15T03:06:04.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.986+0000 7f265728f700 1 -- 172.21.15.121:0/1630318762 wait complete. 2023-11-15T03:06:04.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f265728f700 1 Processor -- start 2023-11-15T03:06:04.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f265728f700 1 -- start start 2023-11-15T03:06:05.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f265728f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2650066330 0x7f265014e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:05.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f265728f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26500acd00 0x7f2650153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:05.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f265728f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26500c5c70 0x7f2650153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:05.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f265728f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2650150160 con 0x7f26500acd00 2023-11-15T03:06:05.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f265728f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265014ed50 con 0x7f2650066330 2023-11-15T03:06:05.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f265728f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265014f550 con 0x7f26500c5c70 2023-11-15T03:06:05.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f2655a8c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26500acd00 0x7f2650153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:05.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f2655a8c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26500acd00 0x7f2650153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34398/0 (socket says 172.21.15.121:34398) 2023-11-15T03:06:05.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f2655a8c700 1 -- 172.21.15.121:0/2378419458 learned_addr learned my addr 172.21.15.121:0/2378419458 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:05.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f265628d700 1 --2- 172.21.15.121:0/2378419458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2650066330 0x7f265014e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:05.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f2656a8e700 1 --2- 172.21.15.121:0/2378419458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26500c5c70 0x7f2650153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:05.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f2655a8c700 1 -- 172.21.15.121:0/2378419458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26500c5c70 msgr2=0x7f2650153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:05.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f2655a8c700 1 --2- 172.21.15.121:0/2378419458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26500c5c70 0x7f2650153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:05.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f2655a8c700 1 -- 172.21.15.121:0/2378419458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2650066330 msgr2=0x7f265014e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:05.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f2655a8c700 1 --2- 172.21.15.121:0/2378419458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2650066330 0x7f265014e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:05.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f2655a8c700 1 -- 172.21.15.121:0/2378419458 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264c009250 con 0x7f26500acd00 2023-11-15T03:06:05.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f2655a8c700 1 --2- 172.21.15.121:0/2378419458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26500acd00 0x7f2650153490 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7f264000c470 tx=0x7f2640007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:05.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f26467fc700 1 -- 172.21.15.121:0/2378419458 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2640007020 con 0x7f26500acd00 2023-11-15T03:06:05.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f26467fc700 1 -- 172.21.15.121:0/2378419458 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2640007180 con 0x7f26500acd00 2023-11-15T03:06:05.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f265728f700 1 -- 172.21.15.121:0/2378419458 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265014f830 con 0x7f26500acd00 2023-11-15T03:06:05.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f26467fc700 1 -- 172.21.15.121:0/2378419458 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2640007520 con 0x7f26500acd00 2023-11-15T03:06:05.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.990+0000 7f265728f700 1 -- 172.21.15.121:0/2378419458 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265014fdf0 con 0x7f26500acd00 2023-11-15T03:06:05.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.994+0000 7f265728f700 1 -- 172.21.15.121:0/2378419458 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f265006bcf0 con 0x7f26500acd00 2023-11-15T03:06:05.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.994+0000 7f26467fc700 1 -- 172.21.15.121:0/2378419458 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f26400068b0 con 0x7f26500acd00 2023-11-15T03:06:05.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.994+0000 7f26467fc700 1 --2- 172.21.15.121:0/2378419458 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f263405a510 0x7f263405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:05.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.994+0000 7f26467fc700 1 -- 172.21.15.121:0/2378419458 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7470+0+0 (secure 0 0 0) 0x7f2640073f00 con 0x7f26500acd00 2023-11-15T03:06:05.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.998+0000 7f265628d700 1 --2- 172.21.15.121:0/2378419458 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f263405a510 0x7f263405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:05.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:04.998+0000 7f265628d700 1 --2- 172.21.15.121:0/2378419458 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f263405a510 0x7f263405c9d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f264c006fa0 tx=0x7f264c016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:05.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.002+0000 7f26467fc700 1 -- 172.21.15.121:0/2378419458 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f26400471a0 con 0x7f26500acd00 2023-11-15T03:06:05.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.198+0000 7f265728f700 1 -- 172.21.15.121:0/2378419458 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f26500c61c0 con 0x7f26500acd00 2023-11-15T03:06:05.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.234+0000 7f26467fc700 1 -- 172.21.15.121:0/2378419458 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v171) v1 ==== 162+0+0 (secure 0 0 0) 0x7f264004a7c0 con 0x7f26500acd00 2023-11-15T03:06:05.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.238+0000 7f265728f700 1 -- 172.21.15.121:0/2378419458 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f263405a510 msgr2=0x7f263405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:05.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.238+0000 7f265728f700 1 --2- 172.21.15.121:0/2378419458 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f263405a510 0x7f263405c9d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f264c006fa0 tx=0x7f264c016000).stop 2023-11-15T03:06:05.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.238+0000 7f265728f700 1 -- 172.21.15.121:0/2378419458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26500acd00 msgr2=0x7f2650153490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:05.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.238+0000 7f265728f700 1 --2- 172.21.15.121:0/2378419458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26500acd00 0x7f2650153490 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7f264000c470 tx=0x7f2640007e00).stop 2023-11-15T03:06:05.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.238+0000 7f265728f700 1 -- 172.21.15.121:0/2378419458 shutdown_connections 2023-11-15T03:06:05.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.238+0000 7f265728f700 1 --2- 172.21.15.121:0/2378419458 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f263405a510 0x7f263405c9d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:05.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.238+0000 7f265728f700 1 --2- 172.21.15.121:0/2378419458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26500c5c70 0x7f2650153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:05.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.238+0000 7f265728f700 1 --2- 172.21.15.121:0/2378419458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26500acd00 0x7f2650153490 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:05.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.238+0000 7f265728f700 1 --2- 172.21.15.121:0/2378419458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2650066330 0x7f265014e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:05.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.238+0000 7f265728f700 1 -- 172.21.15.121:0/2378419458 >> 172.21.15.121:0/2378419458 conn(0x7f2650063fc0 msgr2=0x7f26500c5070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:05.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.238+0000 7f265728f700 1 -- 172.21.15.121:0/2378419458 shutdown_connections 2023-11-15T03:06:05.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.238+0000 7f265728f700 1 -- 172.21.15.121:0/2378419458 wait complete. 2023-11-15T03:06:05.252 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:06:05.270 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:05.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.422+0000 7f56916f1700 1 Processor -- start 2023-11-15T03:06:05.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.422+0000 7f56916f1700 1 -- start start 2023-11-15T03:06:05.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.426+0000 7f56916f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f568c152780 0x7f568c159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:05.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.426+0000 7f56916f1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f568c0a5510 0x7f568c0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:05.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.426+0000 7f56916f1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568c0c3fa0 con 0x7f568c0c65d0 2023-11-15T03:06:05.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.426+0000 7f56916f1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568c159630 con 0x7f568c0a5510 2023-11-15T03:06:05.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.426+0000 7f56916f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0c30b0 con 0x7f568c152780 2023-11-15T03:06:05.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.426+0000 7f568b7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f568c0a5510 0x7f568c0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:05.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.426+0000 7f568b7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f568c0a5510 0x7f568c0a59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34408/0 (socket says 172.21.15.121:34408) 2023-11-15T03:06:05.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.426+0000 7f568b7fe700 1 -- 172.21.15.121:0/2230301952 learned_addr learned my addr 172.21.15.121:0/2230301952 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:05.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.426+0000 7f568affd700 1 --2- 172.21.15.121:0/2230301952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f568c152780 0x7f568c159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:05.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.426+0000 7f568b7fe700 1 -- 172.21.15.121:0/2230301952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f568c152780 msgr2=0x7f568c159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:05.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.426+0000 7f568b7fe700 1 --2- 172.21.15.121:0/2230301952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f568c152780 0x7f568c159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:05.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.426+0000 7f568b7fe700 1 -- 172.21.15.121:0/2230301952 >> v1:172.21.15.190:6789/0 conn(0x7f568c0c65d0 legacy=0x7f568c0c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:05.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.426+0000 7f568b7fe700 1 -- 172.21.15.121:0/2230301952 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c0c3390 con 0x7f568c0a5510 2023-11-15T03:06:05.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.426+0000 7f568b7fe700 1 --2- 172.21.15.121:0/2230301952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568c0a5510 0x7f568c0a59c0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7f5684009f20 tx=0x7f5684009840).ready entity=mon.0 client_cookie=18c5e6e46e5c069d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:05.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.426+0000 7f5689ffb700 1 -- 172.21.15.121:0/2230301952 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568400e070 con 0x7f568c0a5510 2023-11-15T03:06:05.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.426+0000 7f5689ffb700 1 -- 172.21.15.121:0/2230301952 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568400ba80 con 0x7f568c0a5510 2023-11-15T03:06:05.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.426+0000 7f5689ffb700 1 -- 172.21.15.121:0/2230301952 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568400be00 con 0x7f568c0a5510 2023-11-15T03:06:05.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.426+0000 7f56916f1700 1 -- 172.21.15.121:0/2230301952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568c0a5510 msgr2=0x7f568c0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:05.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.426+0000 7f56916f1700 1 --2- 172.21.15.121:0/2230301952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568c0a5510 0x7f568c0a59c0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7f5684009f20 tx=0x7f5684009840).stop 2023-11-15T03:06:05.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.430+0000 7f56916f1700 1 -- 172.21.15.121:0/2230301952 shutdown_connections 2023-11-15T03:06:05.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.430+0000 7f56916f1700 1 --2- 172.21.15.121:0/2230301952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568c0a5510 0x7f568c0a59c0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:05.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.430+0000 7f56916f1700 1 --2- 172.21.15.121:0/2230301952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f568c152780 0x7f568c159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:05.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.430+0000 7f56916f1700 1 -- 172.21.15.121:0/2230301952 >> 172.21.15.121:0/2230301952 conn(0x7f568c062190 msgr2=0x7f568c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:05.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.430+0000 7f56916f1700 1 -- 172.21.15.121:0/2230301952 shutdown_connections 2023-11-15T03:06:05.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.430+0000 7f56916f1700 1 -- 172.21.15.121:0/2230301952 wait complete. 2023-11-15T03:06:05.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.430+0000 7f56916f1700 1 Processor -- start 2023-11-15T03:06:05.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.430+0000 7f56916f1700 1 -- start start 2023-11-15T03:06:05.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f56916f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568c0a5510 0x7f568c14fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:05.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f56916f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f568c0c65d0 0x7f568c0a7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:05.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f56916f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f568c152780 0x7f568c14dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:05.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f56916f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0a7760 con 0x7f568c0a5510 2023-11-15T03:06:05.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f56916f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0a7a20 con 0x7f568c152780 2023-11-15T03:06:05.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f56916f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0a7d00 con 0x7f568c0c65d0 2023-11-15T03:06:05.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f568affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568c0a5510 0x7f568c14fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:05.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f568affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568c0a5510 0x7f568c14fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34424/0 (socket says 172.21.15.121:34424) 2023-11-15T03:06:05.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f568affd700 1 -- 172.21.15.121:0/1645736058 learned_addr learned my addr 172.21.15.121:0/1645736058 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:05.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f568b7fe700 1 --2- 172.21.15.121:0/1645736058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f568c152780 0x7f568c14dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:05.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f568a7fc700 1 --2- 172.21.15.121:0/1645736058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f568c0c65d0 0x7f568c0a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:05.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f568affd700 1 -- 172.21.15.121:0/1645736058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f568c0c65d0 msgr2=0x7f568c0a7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:05.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f568affd700 1 --2- 172.21.15.121:0/1645736058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f568c0c65d0 0x7f568c0a7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:05.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f568affd700 1 -- 172.21.15.121:0/1645736058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f568c152780 msgr2=0x7f568c14dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:05.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f568affd700 1 --2- 172.21.15.121:0/1645736058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f568c152780 0x7f568c14dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:05.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f568affd700 1 -- 172.21.15.121:0/1645736058 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5684009ae0 con 0x7f568c0a5510 2023-11-15T03:06:05.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f568affd700 1 --2- 172.21.15.121:0/1645736058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568c0a5510 0x7f568c14fbc0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7f567c00d890 tx=0x7f567c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:05.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f5672ffd700 1 -- 172.21.15.121:0/1645736058 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c011590 con 0x7f568c0a5510 2023-11-15T03:06:05.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f5672ffd700 1 -- 172.21.15.121:0/1645736058 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c0116f0 con 0x7f568c0a5510 2023-11-15T03:06:05.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f56916f1700 1 -- 172.21.15.121:0/1645736058 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568c0a7fe0 con 0x7f568c0a5510 2023-11-15T03:06:05.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f5672ffd700 1 -- 172.21.15.121:0/1645736058 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c011a70 con 0x7f568c0a5510 2023-11-15T03:06:05.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.434+0000 7f56916f1700 1 -- 172.21.15.121:0/1645736058 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568c14e240 con 0x7f568c0a5510 2023-11-15T03:06:05.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.438+0000 7f56916f1700 1 -- 172.21.15.121:0/1645736058 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568c06c150 con 0x7f568c0a5510 2023-11-15T03:06:05.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.438+0000 7f5672ffd700 1 -- 172.21.15.121:0/1645736058 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f567c012420 con 0x7f568c0a5510 2023-11-15T03:06:05.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.442+0000 7f5672ffd700 1 --2- 172.21.15.121:0/1645736058 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f567805a540 0x7f567805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:05.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.442+0000 7f5672ffd700 1 -- 172.21.15.121:0/1645736058 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7470+0+0 (secure 0 0 0) 0x7f567c074670 con 0x7f568c0a5510 2023-11-15T03:06:05.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.442+0000 7f568a7fc700 1 --2- 172.21.15.121:0/1645736058 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f567805a540 0x7f567805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:05.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.442+0000 7f568a7fc700 1 --2- 172.21.15.121:0/1645736058 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f567805a540 0x7f567805ca00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f568c0a98e0 tx=0x7f5680007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:05.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.446+0000 7f5672ffd700 1 -- 172.21.15.121:0/1645736058 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f567c0477f0 con 0x7f568c0a5510 2023-11-15T03:06:05.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.646+0000 7f56916f1700 1 -- 172.21.15.121:0/1645736058 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f568c0c3670 con 0x7f568c0a5510 2023-11-15T03:06:05.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.646+0000 7f5672ffd700 1 -- 172.21.15.121:0/1645736058 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) v1 ==== 95+0+4502 (secure 0 0 0) 0x7f567c011e60 con 0x7f568c0a5510 2023-11-15T03:06:05.656 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:05.657 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":172,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:05.241099+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8505":{"gid":8505,"name":"a","rank":0,"incarnation":172,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/228280964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":228280964},{"type":"v1","addr":"172.21.15.121:6836","nonce":228280964}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:05.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.650+0000 7f56916f1700 1 -- 172.21.15.121:0/1645736058 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f567805a540 msgr2=0x7f567805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:05.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.650+0000 7f56916f1700 1 --2- 172.21.15.121:0/1645736058 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f567805a540 0x7f567805ca00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f568c0a98e0 tx=0x7f5680007d40).stop 2023-11-15T03:06:05.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.650+0000 7f56916f1700 1 -- 172.21.15.121:0/1645736058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568c0a5510 msgr2=0x7f568c14fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:05.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.650+0000 7f56916f1700 1 --2- 172.21.15.121:0/1645736058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568c0a5510 0x7f568c14fbc0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7f567c00d890 tx=0x7f567c008040).stop 2023-11-15T03:06:05.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.650+0000 7f56916f1700 1 -- 172.21.15.121:0/1645736058 shutdown_connections 2023-11-15T03:06:05.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.650+0000 7f56916f1700 1 --2- 172.21.15.121:0/1645736058 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f567805a540 0x7f567805ca00 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:05.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.650+0000 7f56916f1700 1 --2- 172.21.15.121:0/1645736058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f568c152780 0x7f568c14dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:05.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.650+0000 7f56916f1700 1 --2- 172.21.15.121:0/1645736058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f568c0c65d0 0x7f568c0a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:05.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.650+0000 7f56916f1700 1 --2- 172.21.15.121:0/1645736058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568c0a5510 0x7f568c14fbc0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:05.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.650+0000 7f56916f1700 1 -- 172.21.15.121:0/1645736058 >> 172.21.15.121:0/1645736058 conn(0x7f568c062190 msgr2=0x7f568c1508c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:05.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.650+0000 7f56916f1700 1 -- 172.21.15.121:0/1645736058 shutdown_connections 2023-11-15T03:06:05.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:05.650+0000 7f56916f1700 1 -- 172.21.15.121:0/1645736058 wait complete. 2023-11-15T03:06:05.662 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 172 2023-11-15T03:06:05.679 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:06:08.286 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:06:08.277+0000 7f1fc1554700 -1 log_channel(cluster) log [ERR] : dir 0x1 object missing on disk; some files may be lost () 2023-11-15T03:06:08.286 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:06:08.277+0000 7f1fc1554700 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2023-11-15T03:06:09.284 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:06:09.273+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:06:09.284 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:06:09.273+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:06:09.295 INFO:tasks.ceph.mds.a.smithi121.stderr: -19> 2023-11-15T03:06:08.277+0000 7f1fc1554700 -1 log_channel(cluster) log [ERR] : dir 0x1 object missing on disk; some files may be lost () 2023-11-15T03:06:09.295 INFO:tasks.ceph.mds.a.smithi121.stderr: -17> 2023-11-15T03:06:08.277+0000 7f1fc1554700 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2023-11-15T03:06:09.333 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:06:10.681 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:10.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.841+0000 7ff5dae1b700 1 Processor -- start 2023-11-15T03:06:10.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.841+0000 7ff5dae1b700 1 -- start start 2023-11-15T03:06:10.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.845+0000 7ff5dae1b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff5d40c1ea0 0x7ff5d415ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:10.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.845+0000 7ff5dae1b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d4156630 con 0x7ff5d4151840 2023-11-15T03:06:10.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.845+0000 7ff5dae1b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d40a6b30 con 0x7ff5d4151c20 2023-11-15T03:06:10.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.845+0000 7ff5dae1b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d4155a20 con 0x7ff5d40c1ea0 2023-11-15T03:06:10.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.845+0000 7ff5d93b8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff5d40c1ea0 0x7ff5d415ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:10.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.845+0000 7ff5d8bb7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff5d4151840 0x7ff5d40a5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:38782/0 (socket says 172.21.15.121:38782) 2023-11-15T03:06:10.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.845+0000 7ff5d8bb7700 1 -- 172.21.15.121:0/8813318 learned_addr learned my addr 172.21.15.121:0/8813318 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:10.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.845+0000 7ff5d37fe700 1 -- 172.21.15.121:0/8813318 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307929691 0 0) 0x7ff5d4156630 con 0x7ff5d4151840 2023-11-15T03:06:10.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.845+0000 7ff5d37fe700 1 -- 172.21.15.121:0/8813318 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5c40033f0 con 0x7ff5d4151840 2023-11-15T03:06:10.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.845+0000 7ff5d93b8700 1 -- 172.21.15.121:0/8813318 >> v1:172.21.15.190:6789/0 conn(0x7ff5d4151c20 legacy=0x7ff5d40c1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:10.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.845+0000 7ff5d93b8700 1 -- 172.21.15.121:0/8813318 >> v1:172.21.15.121:6789/0 conn(0x7ff5d4151840 legacy=0x7ff5d40a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:10.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.845+0000 7ff5d93b8700 1 -- 172.21.15.121:0/8813318 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d4155d00 con 0x7ff5d40c1ea0 2023-11-15T03:06:10.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.845+0000 7ff5d93b8700 1 --2- 172.21.15.121:0/8813318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5d40c1ea0 0x7ff5d415ca00 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7ff5cc009e90 tx=0x7ff5cc009780).ready entity=mon.0 client_cookie=7e81f3359106b99f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:10.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.845+0000 7ff5d37fe700 1 -- 172.21.15.121:0/8813318 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc00e070 con 0x7ff5d40c1ea0 2023-11-15T03:06:10.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.845+0000 7ff5d37fe700 1 -- 172.21.15.121:0/8813318 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5cc00bb20 con 0x7ff5d40c1ea0 2023-11-15T03:06:10.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.845+0000 7ff5d37fe700 1 -- 172.21.15.121:0/8813318 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc00bea0 con 0x7ff5d40c1ea0 2023-11-15T03:06:10.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.849+0000 7ff5dae1b700 1 -- 172.21.15.121:0/8813318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5d40c1ea0 msgr2=0x7ff5d415ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:10.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.849+0000 7ff5dae1b700 1 --2- 172.21.15.121:0/8813318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5d40c1ea0 0x7ff5d415ca00 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7ff5cc009e90 tx=0x7ff5cc009780).stop 2023-11-15T03:06:10.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.849+0000 7ff5dae1b700 1 -- 172.21.15.121:0/8813318 shutdown_connections 2023-11-15T03:06:10.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.849+0000 7ff5dae1b700 1 --2- 172.21.15.121:0/8813318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5d40c1ea0 0x7ff5d415ca00 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:10.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.849+0000 7ff5dae1b700 1 -- 172.21.15.121:0/8813318 >> 172.21.15.121:0/8813318 conn(0x7ff5d4062190 msgr2=0x7ff5d40625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:10.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.849+0000 7ff5dae1b700 1 -- 172.21.15.121:0/8813318 shutdown_connections 2023-11-15T03:06:10.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.849+0000 7ff5dae1b700 1 -- 172.21.15.121:0/8813318 wait complete. 2023-11-15T03:06:10.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.849+0000 7ff5dae1b700 1 Processor -- start 2023-11-15T03:06:10.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.849+0000 7ff5dae1b700 1 -- start start 2023-11-15T03:06:10.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5dae1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5d40c1ea0 0x7ff5d413e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:10.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5dae1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5d4151840 0x7ff5d414cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:10.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5dae1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5d4151c20 0x7ff5d414d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:10.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5dae1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d414d6e0 con 0x7ff5d40c1ea0 2023-11-15T03:06:10.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5dae1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d413e740 con 0x7ff5d4151840 2023-11-15T03:06:10.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5dae1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d40aa560 con 0x7ff5d4151c20 2023-11-15T03:06:10.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5d8bb7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5d40c1ea0 0x7ff5d413e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:10.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5d8bb7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5d40c1ea0 0x7ff5d413e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34452/0 (socket says 172.21.15.121:34452) 2023-11-15T03:06:10.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5d8bb7700 1 -- 172.21.15.121:0/448761108 learned_addr learned my addr 172.21.15.121:0/448761108 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:10.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5d3fff700 1 --2- 172.21.15.121:0/448761108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5d4151840 0x7ff5d414cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:10.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5d93b8700 1 --2- 172.21.15.121:0/448761108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5d4151c20 0x7ff5d414d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:10.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5d8bb7700 1 -- 172.21.15.121:0/448761108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5d4151c20 msgr2=0x7ff5d414d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:10.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5d8bb7700 1 --2- 172.21.15.121:0/448761108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5d4151c20 0x7ff5d414d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:10.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5d8bb7700 1 -- 172.21.15.121:0/448761108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5d4151840 msgr2=0x7ff5d414cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:10.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5d8bb7700 1 --2- 172.21.15.121:0/448761108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5d4151840 0x7ff5d414cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:10.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5d8bb7700 1 -- 172.21.15.121:0/448761108 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5cc009a20 con 0x7ff5d40c1ea0 2023-11-15T03:06:10.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5d3fff700 1 --2- 172.21.15.121:0/448761108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5d4151840 0x7ff5d414cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:06:10.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5d8bb7700 1 --2- 172.21.15.121:0/448761108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5d40c1ea0 0x7ff5d413e250 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7ff5c800c590 tx=0x7ff5c8007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:10.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5d0ff9700 1 -- 172.21.15.121:0/448761108 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80072e0 con 0x7ff5d40c1ea0 2023-11-15T03:06:10.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5d0ff9700 1 -- 172.21.15.121:0/448761108 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c8007440 con 0x7ff5d40c1ea0 2023-11-15T03:06:10.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5d0ff9700 1 -- 172.21.15.121:0/448761108 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80057a0 con 0x7ff5d40c1ea0 2023-11-15T03:06:10.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5dae1b700 1 -- 172.21.15.121:0/448761108 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d40aa840 con 0x7ff5d40c1ea0 2023-11-15T03:06:10.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.853+0000 7ff5dae1b700 1 -- 172.21.15.121:0/448761108 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d40aae00 con 0x7ff5d40c1ea0 2023-11-15T03:06:10.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.857+0000 7ff5dae1b700 1 -- 172.21.15.121:0/448761108 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d406c150 con 0x7ff5d40c1ea0 2023-11-15T03:06:10.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.857+0000 7ff5d0ff9700 1 -- 172.21.15.121:0/448761108 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff5c8005bd0 con 0x7ff5d40c1ea0 2023-11-15T03:06:10.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.857+0000 7ff5d0ff9700 1 --2- 172.21.15.121:0/448761108 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff5b805a4e0 0x7ff5b805c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:10.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.857+0000 7ff5d0ff9700 1 -- 172.21.15.121:0/448761108 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7556+0+0 (secure 0 0 0) 0x7ff5c8074560 con 0x7ff5d40c1ea0 2023-11-15T03:06:10.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.857+0000 7ff5d3fff700 1 --2- 172.21.15.121:0/448761108 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff5b805a4e0 0x7ff5b805c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:10.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.857+0000 7ff5d3fff700 1 --2- 172.21.15.121:0/448761108 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff5b805a4e0 0x7ff5b805c9a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7ff5d414d860 tx=0x7ff5c0009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:10.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:10.865+0000 7ff5d0ff9700 1 -- 172.21.15.121:0/448761108 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff5c80476e0 con 0x7ff5d40c1ea0 2023-11-15T03:06:11.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.061+0000 7ff5dae1b700 1 -- 172.21.15.121:0/448761108 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5d4155fe0 con 0x7ff5d40c1ea0 2023-11-15T03:06:11.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.061+0000 7ff5d0ff9700 1 -- 172.21.15.121:0/448761108 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+4493 (secure 0 0 0) 0x7ff5c804ad00 con 0x7ff5d40c1ea0 2023-11-15T03:06:11.073 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:11.073 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":177,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/564044995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":564044995},{"type":"v1","addr":"172.21.15.121:6836","nonce":564044995}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":177}],"filesystems":[{"mdsmap":{"epoch":176,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:09.279740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:11.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.065+0000 7ff5dae1b700 1 -- 172.21.15.121:0/448761108 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff5b805a4e0 msgr2=0x7ff5b805c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:11.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.065+0000 7ff5dae1b700 1 --2- 172.21.15.121:0/448761108 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff5b805a4e0 0x7ff5b805c9a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7ff5d414d860 tx=0x7ff5c0009040).stop 2023-11-15T03:06:11.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.065+0000 7ff5dae1b700 1 -- 172.21.15.121:0/448761108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5d40c1ea0 msgr2=0x7ff5d413e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:11.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.065+0000 7ff5dae1b700 1 --2- 172.21.15.121:0/448761108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5d40c1ea0 0x7ff5d413e250 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7ff5c800c590 tx=0x7ff5c8007c80).stop 2023-11-15T03:06:11.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.065+0000 7ff5dae1b700 1 -- 172.21.15.121:0/448761108 shutdown_connections 2023-11-15T03:06:11.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.065+0000 7ff5dae1b700 1 --2- 172.21.15.121:0/448761108 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff5b805a4e0 0x7ff5b805c9a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.065+0000 7ff5dae1b700 1 --2- 172.21.15.121:0/448761108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5d4151c20 0x7ff5d414d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.065+0000 7ff5dae1b700 1 --2- 172.21.15.121:0/448761108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5d4151840 0x7ff5d414cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.065+0000 7ff5dae1b700 1 --2- 172.21.15.121:0/448761108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5d40c1ea0 0x7ff5d413e250 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.065+0000 7ff5dae1b700 1 -- 172.21.15.121:0/448761108 >> 172.21.15.121:0/448761108 conn(0x7ff5d4062190 msgr2=0x7ff5d40c66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:11.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.065+0000 7ff5dae1b700 1 -- 172.21.15.121:0/448761108 shutdown_connections 2023-11-15T03:06:11.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.065+0000 7ff5dae1b700 1 -- 172.21.15.121:0/448761108 wait complete. 2023-11-15T03:06:11.078 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 177 2023-11-15T03:06:11.098 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:06:11.098 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:11.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.257+0000 7f1d7f107700 1 Processor -- start 2023-11-15T03:06:11.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.257+0000 7f1d7f107700 1 -- start start 2023-11-15T03:06:11.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.261+0000 7f1d7f107700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1d78066700 0x7f1d78154110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:11.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.261+0000 7f1d7f107700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d780c73e0 con 0x7f1d78066320 2023-11-15T03:06:11.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.261+0000 7f1d7f107700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d781546c0 con 0x7f1d780acf10 2023-11-15T03:06:11.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.261+0000 7f1d7f107700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d781549a0 con 0x7f1d78066700 2023-11-15T03:06:11.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.261+0000 7f1d7e105700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f1d78066320 0x7f1d780656a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:38794/0 (socket says 172.21.15.121:38794) 2023-11-15T03:06:11.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.261+0000 7f1d7e105700 1 -- 172.21.15.121:0/4135277670 learned_addr learned my addr 172.21.15.121:0/4135277670 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:11.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.261+0000 7f1d7d904700 1 --2- 172.21.15.121:0/4135277670 >> v2:172.21.15.190:3300/0 conn(0x7f1d78066700 0x7f1d78154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:11.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.261+0000 7f1d7d103700 1 -- 172.21.15.121:0/4135277670 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944449539 0 0) 0x7f1d780c73e0 con 0x7f1d78066320 2023-11-15T03:06:11.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.261+0000 7f1d7d103700 1 -- 172.21.15.121:0/4135277670 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d640033f0 con 0x7f1d78066320 2023-11-15T03:06:11.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.261+0000 7f1d7d904700 1 -- 172.21.15.121:0/4135277670 >> v1:172.21.15.190:6789/0 conn(0x7f1d780acf10 legacy=0x7f1d78062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:11.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.261+0000 7f1d7d904700 1 -- 172.21.15.121:0/4135277670 >> v1:172.21.15.121:6789/0 conn(0x7f1d78066320 legacy=0x7f1d780656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:11.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.261+0000 7f1d7d904700 1 -- 172.21.15.121:0/4135277670 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d78154c80 con 0x7f1d78066700 2023-11-15T03:06:11.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.261+0000 7f1d7d904700 1 --2- 172.21.15.121:0/4135277670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d78066700 0x7f1d78154110 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f1d68009760 tx=0x7f1d68009060).ready entity=mon.1 client_cookie=f32579d0f62ef4f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:11.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.261+0000 7f1d7d103700 1 -- 172.21.15.121:0/4135277670 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6800f510 con 0x7f1d78066700 2023-11-15T03:06:11.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.261+0000 7f1d7d103700 1 -- 172.21.15.121:0/4135277670 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6800f670 con 0x7f1d78066700 2023-11-15T03:06:11.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.261+0000 7f1d7d103700 1 -- 172.21.15.121:0/4135277670 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6800f9f0 con 0x7f1d78066700 2023-11-15T03:06:11.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.265+0000 7f1d7f107700 1 -- 172.21.15.121:0/4135277670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d78066700 msgr2=0x7f1d78154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:11.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.265+0000 7f1d7f107700 1 --2- 172.21.15.121:0/4135277670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d78066700 0x7f1d78154110 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f1d68009760 tx=0x7f1d68009060).stop 2023-11-15T03:06:11.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.265+0000 7f1d7f107700 1 -- 172.21.15.121:0/4135277670 shutdown_connections 2023-11-15T03:06:11.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.265+0000 7f1d7f107700 1 --2- 172.21.15.121:0/4135277670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d78066700 0x7f1d78154110 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.265+0000 7f1d7f107700 1 -- 172.21.15.121:0/4135277670 >> 172.21.15.121:0/4135277670 conn(0x7f1d78067a60 msgr2=0x7f1d78067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:11.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.265+0000 7f1d7f107700 1 -- 172.21.15.121:0/4135277670 shutdown_connections 2023-11-15T03:06:11.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.265+0000 7f1d7f107700 1 -- 172.21.15.121:0/4135277670 wait complete. 2023-11-15T03:06:11.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.265+0000 7f1d7f107700 1 Processor -- start 2023-11-15T03:06:11.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.265+0000 7f1d7f107700 1 -- start start 2023-11-15T03:06:11.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7f107700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d78066320 0x7f1d78147480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:11.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7f107700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d78066700 0x7f1d78140420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:11.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7f107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d780acf10 0x7f1d78167d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:11.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7f107700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7814b780 con 0x7f1d78066700 2023-11-15T03:06:11.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7f107700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7814db30 con 0x7f1d78066320 2023-11-15T03:06:11.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7f107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7814de10 con 0x7f1d780acf10 2023-11-15T03:06:11.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7d904700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d78066700 0x7f1d78140420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:11.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7d904700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d78066700 0x7f1d78140420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34466/0 (socket says 172.21.15.121:34466) 2023-11-15T03:06:11.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7d904700 1 -- 172.21.15.121:0/371021935 learned_addr learned my addr 172.21.15.121:0/371021935 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:11.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7e105700 1 --2- 172.21.15.121:0/371021935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d78066320 0x7f1d78147480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:11.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7e906700 1 --2- 172.21.15.121:0/371021935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d780acf10 0x7f1d78167d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:11.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7d904700 1 -- 172.21.15.121:0/371021935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d780acf10 msgr2=0x7f1d78167d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:11.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7d904700 1 --2- 172.21.15.121:0/371021935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d780acf10 0x7f1d78167d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7d904700 1 -- 172.21.15.121:0/371021935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d78066320 msgr2=0x7f1d78147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:11.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7d904700 1 --2- 172.21.15.121:0/371021935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d78066320 0x7f1d78147480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7d904700 1 -- 172.21.15.121:0/371021935 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d680092a0 con 0x7f1d78066700 2023-11-15T03:06:11.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7d904700 1 --2- 172.21.15.121:0/371021935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d78066700 0x7f1d78140420 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7f1d78064af0 tx=0x7f1d68017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:11.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d6e7fc700 1 -- 172.21.15.121:0/371021935 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6800f7e0 con 0x7f1d78066700 2023-11-15T03:06:11.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d6e7fc700 1 -- 172.21.15.121:0/371021935 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6800f940 con 0x7f1d78066700 2023-11-15T03:06:11.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d6e7fc700 1 -- 172.21.15.121:0/371021935 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6801b660 con 0x7f1d78066700 2023-11-15T03:06:11.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7f107700 1 -- 172.21.15.121:0/371021935 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7814e0f0 con 0x7f1d78066700 2023-11-15T03:06:11.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7f107700 1 -- 172.21.15.121:0/371021935 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7814ba60 con 0x7f1d78066700 2023-11-15T03:06:11.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.269+0000 7f1d7f107700 1 -- 172.21.15.121:0/371021935 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7814e330 con 0x7f1d78066700 2023-11-15T03:06:11.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.273+0000 7f1d6e7fc700 1 -- 172.21.15.121:0/371021935 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1d6801b9d0 con 0x7f1d78066700 2023-11-15T03:06:11.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.273+0000 7f1d6e7fc700 1 --2- 172.21.15.121:0/371021935 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d5c05a510 0x7f1d5c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:11.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.273+0000 7f1d6e7fc700 1 -- 172.21.15.121:0/371021935 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7556+0+0 (secure 0 0 0) 0x7f1d680768b0 con 0x7f1d78066700 2023-11-15T03:06:11.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.273+0000 7f1d7e105700 1 --2- 172.21.15.121:0/371021935 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d5c05a510 0x7f1d5c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:11.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.277+0000 7f1d7e105700 1 --2- 172.21.15.121:0/371021935 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d5c05a510 0x7f1d5c05c9d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f1d74005750 tx=0x7f1d74009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:11.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.281+0000 7f1d6e7fc700 1 -- 172.21.15.121:0/371021935 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1d7814e330 con 0x7f1d78066700 2023-11-15T03:06:11.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.481+0000 7f1d7f107700 1 -- 172.21.15.121:0/371021935 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d78154f60 con 0x7f1d78066700 2023-11-15T03:06:11.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.481+0000 7f1d6e7fc700 1 -- 172.21.15.121:0/371021935 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+4493 (secure 0 0 0) 0x7f1d68049ae0 con 0x7f1d78066700 2023-11-15T03:06:11.492 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:11.492 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":177,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/564044995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":564044995},{"type":"v1","addr":"172.21.15.121:6836","nonce":564044995}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":177}],"filesystems":[{"mdsmap":{"epoch":176,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:09.279740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:11.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.485+0000 7f1d7f107700 1 -- 172.21.15.121:0/371021935 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d5c05a510 msgr2=0x7f1d5c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:11.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.485+0000 7f1d7f107700 1 --2- 172.21.15.121:0/371021935 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d5c05a510 0x7f1d5c05c9d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f1d74005750 tx=0x7f1d74009040).stop 2023-11-15T03:06:11.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.485+0000 7f1d7f107700 1 -- 172.21.15.121:0/371021935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d78066700 msgr2=0x7f1d78140420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:11.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.485+0000 7f1d7f107700 1 --2- 172.21.15.121:0/371021935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d78066700 0x7f1d78140420 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7f1d78064af0 tx=0x7f1d68017000).stop 2023-11-15T03:06:11.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.485+0000 7f1d7f107700 1 -- 172.21.15.121:0/371021935 shutdown_connections 2023-11-15T03:06:11.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.485+0000 7f1d7f107700 1 --2- 172.21.15.121:0/371021935 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d5c05a510 0x7f1d5c05c9d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.485+0000 7f1d7f107700 1 --2- 172.21.15.121:0/371021935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d780acf10 0x7f1d78167d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.485+0000 7f1d7f107700 1 --2- 172.21.15.121:0/371021935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d78066700 0x7f1d78140420 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.485+0000 7f1d7f107700 1 --2- 172.21.15.121:0/371021935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d78066320 0x7f1d78147480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.485+0000 7f1d7f107700 1 -- 172.21.15.121:0/371021935 >> 172.21.15.121:0/371021935 conn(0x7f1d78067a60 msgr2=0x7f1d780aa110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:11.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.485+0000 7f1d7f107700 1 -- 172.21.15.121:0/371021935 shutdown_connections 2023-11-15T03:06:11.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.485+0000 7f1d7f107700 1 -- 172.21.15.121:0/371021935 wait complete. 2023-11-15T03:06:11.498 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 177 2023-11-15T03:06:11.517 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:06:11.517 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:11.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.665+0000 7f9b4165b700 1 Processor -- start 2023-11-15T03:06:11.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.665+0000 7f9b4165b700 1 -- start start 2023-11-15T03:06:11.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b4165b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9b3c1539d0 0x7f9b3c0c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:11.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b4165b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b3c153db0 0x7f9b3c0c2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:11.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b4165b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b3c0c1530 con 0x7f9b3c0c19c0 2023-11-15T03:06:11.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b4165b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b3c1541f0 con 0x7f9b3c1539d0 2023-11-15T03:06:11.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b4165b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c0c2b60 con 0x7f9b3c153db0 2023-11-15T03:06:11.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b3b7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9b3c0c19c0 0x7f9b3c0c3190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:38798/0 (socket says 172.21.15.121:38798) 2023-11-15T03:06:11.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b3b7fe700 1 -- 172.21.15.121:0/3723459420 learned_addr learned my addr 172.21.15.121:0/3723459420 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:11.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b3affd700 1 --2- 172.21.15.121:0/3723459420 >> v2:172.21.15.121:3300/0 conn(0x7f9b3c1539d0 0x7f9b3c0c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:11.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b33fff700 1 --2- 172.21.15.121:0/3723459420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b3c153db0 0x7f9b3c0c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:11.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b3a7fc700 1 -- 172.21.15.121:0/3723459420 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691594372 0 0) 0x7f9b3c0c1530 con 0x7f9b3c0c19c0 2023-11-15T03:06:11.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b3a7fc700 1 -- 172.21.15.121:0/3723459420 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b280033f0 con 0x7f9b3c0c19c0 2023-11-15T03:06:11.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b3affd700 1 -- 172.21.15.121:0/3723459420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b3c153db0 msgr2=0x7f9b3c0c2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:11.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b3affd700 1 --2- 172.21.15.121:0/3723459420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b3c153db0 0x7f9b3c0c2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b3affd700 1 -- 172.21.15.121:0/3723459420 >> v1:172.21.15.121:6789/0 conn(0x7f9b3c0c19c0 legacy=0x7f9b3c0c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:11.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b3affd700 1 -- 172.21.15.121:0/3723459420 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c0c2d80 con 0x7f9b3c1539d0 2023-11-15T03:06:11.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b3affd700 1 --2- 172.21.15.121:0/3723459420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c1539d0 0x7f9b3c0c06e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7f9b2c009760 tx=0x7f9b2c009060).ready entity=mon.0 client_cookie=7773cafac858e054 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:11.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b3a7fc700 1 -- 172.21.15.121:0/3723459420 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c00d4a0 con 0x7f9b3c1539d0 2023-11-15T03:06:11.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b3a7fc700 1 -- 172.21.15.121:0/3723459420 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2c00d600 con 0x7f9b3c1539d0 2023-11-15T03:06:11.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b3a7fc700 1 -- 172.21.15.121:0/3723459420 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c00d980 con 0x7f9b3c1539d0 2023-11-15T03:06:11.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b4165b700 1 -- 172.21.15.121:0/3723459420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c1539d0 msgr2=0x7f9b3c0c06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:11.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.669+0000 7f9b4165b700 1 --2- 172.21.15.121:0/3723459420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c1539d0 0x7f9b3c0c06e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7f9b2c009760 tx=0x7f9b2c009060).stop 2023-11-15T03:06:11.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.673+0000 7f9b4165b700 1 -- 172.21.15.121:0/3723459420 shutdown_connections 2023-11-15T03:06:11.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.673+0000 7f9b4165b700 1 --2- 172.21.15.121:0/3723459420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b3c153db0 0x7f9b3c0c2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.673+0000 7f9b4165b700 1 --2- 172.21.15.121:0/3723459420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c1539d0 0x7f9b3c0c06e0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.673+0000 7f9b4165b700 1 -- 172.21.15.121:0/3723459420 >> 172.21.15.121:0/3723459420 conn(0x7f9b3c062190 msgr2=0x7f9b3c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:11.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.673+0000 7f9b4165b700 1 -- 172.21.15.121:0/3723459420 shutdown_connections 2023-11-15T03:06:11.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.673+0000 7f9b4165b700 1 -- 172.21.15.121:0/3723459420 wait complete. 2023-11-15T03:06:11.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.673+0000 7f9b4165b700 1 Processor -- start 2023-11-15T03:06:11.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.673+0000 7f9b4165b700 1 -- start start 2023-11-15T03:06:11.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b4165b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b3c0c19c0 0x7f9b3c0aae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:11.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b4165b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c1539d0 0x7f9b3c157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:11.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b4165b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b3c153db0 0x7f9b3c164500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:11.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b4165b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c0a5da0 con 0x7f9b3c1539d0 2023-11-15T03:06:11.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b4165b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c0aa9b0 con 0x7f9b3c0c19c0 2023-11-15T03:06:11.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b4165b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c1563c0 con 0x7f9b3c153db0 2023-11-15T03:06:11.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b33fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c1539d0 0x7f9b3c157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:11.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b33fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c1539d0 0x7f9b3c157010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34488/0 (socket says 172.21.15.121:34488) 2023-11-15T03:06:11.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b33fff700 1 -- 172.21.15.121:0/896731204 learned_addr learned my addr 172.21.15.121:0/896731204 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:11.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b3affd700 1 --2- 172.21.15.121:0/896731204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b3c0c19c0 0x7f9b3c0aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:11.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b3b7fe700 1 --2- 172.21.15.121:0/896731204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b3c153db0 0x7f9b3c164500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:11.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b33fff700 1 -- 172.21.15.121:0/896731204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b3c153db0 msgr2=0x7f9b3c164500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:11.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b33fff700 1 --2- 172.21.15.121:0/896731204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b3c153db0 0x7f9b3c164500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b33fff700 1 -- 172.21.15.121:0/896731204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b3c0c19c0 msgr2=0x7f9b3c0aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:11.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b33fff700 1 --2- 172.21.15.121:0/896731204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b3c0c19c0 0x7f9b3c0aae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b33fff700 1 -- 172.21.15.121:0/896731204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c0092a0 con 0x7f9b3c1539d0 2023-11-15T03:06:11.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b33fff700 1 --2- 172.21.15.121:0/896731204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c1539d0 0x7f9b3c157010 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7f9b2400e930 tx=0x7f9b24009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:11.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b32ffd700 1 -- 172.21.15.121:0/896731204 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2400c640 con 0x7f9b3c1539d0 2023-11-15T03:06:11.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b32ffd700 1 -- 172.21.15.121:0/896731204 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2400c7a0 con 0x7f9b3c1539d0 2023-11-15T03:06:11.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b4165b700 1 -- 172.21.15.121:0/896731204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3c0a9da0 con 0x7f9b3c1539d0 2023-11-15T03:06:11.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b32ffd700 1 -- 172.21.15.121:0/896731204 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2400cb20 con 0x7f9b3c1539d0 2023-11-15T03:06:11.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b4165b700 1 -- 172.21.15.121:0/896731204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3c0aa360 con 0x7f9b3c1539d0 2023-11-15T03:06:11.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.677+0000 7f9b4165b700 1 -- 172.21.15.121:0/896731204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b3c06c150 con 0x7f9b3c1539d0 2023-11-15T03:06:11.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.681+0000 7f9b32ffd700 1 -- 172.21.15.121:0/896731204 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9b2401b400 con 0x7f9b3c1539d0 2023-11-15T03:06:11.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.681+0000 7f9b32ffd700 1 --2- 172.21.15.121:0/896731204 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9b1c05a510 0x7f9b1c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:11.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.681+0000 7f9b32ffd700 1 -- 172.21.15.121:0/896731204 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7556+0+0 (secure 0 0 0) 0x7f9b240751e0 con 0x7f9b3c1539d0 2023-11-15T03:06:11.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.681+0000 7f9b3affd700 1 --2- 172.21.15.121:0/896731204 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9b1c05a510 0x7f9b1c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:11.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.685+0000 7f9b3affd700 1 --2- 172.21.15.121:0/896731204 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9b1c05a510 0x7f9b1c05c9d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f9b2c00d260 tx=0x7f9b2c016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:11.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.689+0000 7f9b32ffd700 1 -- 172.21.15.121:0/896731204 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9b24048300 con 0x7f9b3c1539d0 2023-11-15T03:06:11.717 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:06:11.723 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:06:11.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.885+0000 7f9b4165b700 1 -- 172.21.15.121:0/896731204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b3c0c0ee0 con 0x7f9b3c1539d0 2023-11-15T03:06:11.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.885+0000 7f9b32ffd700 1 -- 172.21.15.121:0/896731204 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+4493 (secure 0 0 0) 0x7f9b24016030 con 0x7f9b3c1539d0 2023-11-15T03:06:11.897 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:11.897 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":177,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/564044995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":564044995},{"type":"v1","addr":"172.21.15.121:6836","nonce":564044995}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":177}],"filesystems":[{"mdsmap":{"epoch":176,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:09.279740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:11.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.889+0000 7f9b4165b700 1 -- 172.21.15.121:0/896731204 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9b1c05a510 msgr2=0x7f9b1c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:11.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.889+0000 7f9b4165b700 1 --2- 172.21.15.121:0/896731204 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9b1c05a510 0x7f9b1c05c9d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f9b2c00d260 tx=0x7f9b2c016000).stop 2023-11-15T03:06:11.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.889+0000 7f9b4165b700 1 -- 172.21.15.121:0/896731204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c1539d0 msgr2=0x7f9b3c157010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:11.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.889+0000 7f9b4165b700 1 --2- 172.21.15.121:0/896731204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c1539d0 0x7f9b3c157010 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7f9b2400e930 tx=0x7f9b24009060).stop 2023-11-15T03:06:11.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.893+0000 7f9b4165b700 1 -- 172.21.15.121:0/896731204 shutdown_connections 2023-11-15T03:06:11.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.893+0000 7f9b4165b700 1 --2- 172.21.15.121:0/896731204 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9b1c05a510 0x7f9b1c05c9d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.893+0000 7f9b4165b700 1 --2- 172.21.15.121:0/896731204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b3c153db0 0x7f9b3c164500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.893+0000 7f9b4165b700 1 --2- 172.21.15.121:0/896731204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c1539d0 0x7f9b3c157010 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.893+0000 7f9b4165b700 1 --2- 172.21.15.121:0/896731204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b3c0c19c0 0x7f9b3c0aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:11.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.893+0000 7f9b4165b700 1 -- 172.21.15.121:0/896731204 >> 172.21.15.121:0/896731204 conn(0x7f9b3c062190 msgr2=0x7f9b3c158a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:11.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.893+0000 7f9b4165b700 1 -- 172.21.15.121:0/896731204 shutdown_connections 2023-11-15T03:06:11.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:11.893+0000 7f9b4165b700 1 -- 172.21.15.121:0/896731204 wait complete. 2023-11-15T03:06:11.906 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 177 2023-11-15T03:06:11.921 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 1.00000000' led to DAMAGED state 2023-11-15T03:06:11.922 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 1.00000000.inode' 2023-11-15T03:06:11.922 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:06:11.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:06:11.923 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:06:11.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:06:11.944 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:06:11.945 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:06:12.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.141+0000 7f0a3759e700 1 Processor -- start 2023-11-15T03:06:12.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.141+0000 7f0a3759e700 1 -- start start 2023-11-15T03:06:12.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.141+0000 7f0a3759e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0a380ab0d0 0x7f0a380ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:12.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.141+0000 7f0a3759e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0a380ad1b0 0x7f0a38065710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:12.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.141+0000 7f0a3759e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a38066c90 con 0x7f0a38065c00 2023-11-15T03:06:12.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.141+0000 7f0a3759e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a38066080 con 0x7f0a380ad1b0 2023-11-15T03:06:12.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.141+0000 7f0a3759e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a38066360 con 0x7f0a380ab0d0 2023-11-15T03:06:12.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a2dd9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0a380ad1b0 0x7f0a38065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:12.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a36d9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f0a38065c00 0x7f0a380c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:38804/0 (socket says 172.21.15.121:38804) 2023-11-15T03:06:12.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a36d9d700 1 -- 172.21.15.121:0/1483811428 learned_addr learned my addr 172.21.15.121:0/1483811428 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:12.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a3659c700 1 --2- 172.21.15.121:0/1483811428 >> v2:172.21.15.190:3300/0 conn(0x7f0a380ab0d0 0x7f0a380ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:12.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a2dd9b700 1 -- 172.21.15.121:0/1483811428 >> v2:172.21.15.190:3300/0 conn(0x7f0a380ab0d0 msgr2=0x7f0a380ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:12.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a2dd9b700 1 --2- 172.21.15.121:0/1483811428 >> v2:172.21.15.190:3300/0 conn(0x7f0a380ab0d0 0x7f0a380ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:12.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a2dd9b700 1 -- 172.21.15.121:0/1483811428 >> v1:172.21.15.121:6789/0 conn(0x7f0a38065c00 legacy=0x7f0a380c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:12.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a2dd9b700 1 -- 172.21.15.121:0/1483811428 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a38066640 con 0x7f0a380ad1b0 2023-11-15T03:06:12.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a2dd9b700 1 --2- 172.21.15.121:0/1483811428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a380ad1b0 0x7f0a38065710 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7f0a38064570 tx=0x7f0a20009060).ready entity=mon.0 client_cookie=20ab747dae2c8996 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:12.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a35d9b700 1 -- 172.21.15.121:0/1483811428 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2000f510 con 0x7f0a380ad1b0 2023-11-15T03:06:12.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a35d9b700 1 -- 172.21.15.121:0/1483811428 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2000f670 con 0x7f0a380ad1b0 2023-11-15T03:06:12.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a35d9b700 1 -- 172.21.15.121:0/1483811428 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2000f9f0 con 0x7f0a380ad1b0 2023-11-15T03:06:12.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a3759e700 1 -- 172.21.15.121:0/1483811428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a380ad1b0 msgr2=0x7f0a38065710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:12.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a3759e700 1 --2- 172.21.15.121:0/1483811428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a380ad1b0 0x7f0a38065710 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7f0a38064570 tx=0x7f0a20009060).stop 2023-11-15T03:06:12.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a3759e700 1 -- 172.21.15.121:0/1483811428 shutdown_connections 2023-11-15T03:06:12.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a3759e700 1 --2- 172.21.15.121:0/1483811428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a380ad1b0 0x7f0a38065710 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:12.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a3759e700 1 --2- 172.21.15.121:0/1483811428 >> v2:172.21.15.190:3300/0 conn(0x7f0a380ab0d0 0x7f0a380ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:12.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a3759e700 1 -- 172.21.15.121:0/1483811428 >> 172.21.15.121:0/1483811428 conn(0x7f0a38067a60 msgr2=0x7f0a38067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:12.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a3759e700 1 -- 172.21.15.121:0/1483811428 shutdown_connections 2023-11-15T03:06:12.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.145+0000 7f0a3759e700 1 -- 172.21.15.121:0/1483811428 wait complete. 2023-11-15T03:06:12.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.149+0000 7f0a3759e700 1 Processor -- start 2023-11-15T03:06:12.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.149+0000 7f0a3759e700 1 -- start start 2023-11-15T03:06:12.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.149+0000 7f0a3759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a38065c00 0x7f0a38163910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:12.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.149+0000 7f0a3759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a380ab0d0 0x7f0a380cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:12.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.149+0000 7f0a3759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a380ad1b0 0x7f0a380cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:12.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.149+0000 7f0a3759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380cb7a0 con 0x7f0a380ab0d0 2023-11-15T03:06:12.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.149+0000 7f0a3759e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380a7df0 con 0x7f0a38065c00 2023-11-15T03:06:12.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.149+0000 7f0a3759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380a80d0 con 0x7f0a380ad1b0 2023-11-15T03:06:12.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.149+0000 7f0a2dd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a380ab0d0 0x7f0a380cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:12.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.149+0000 7f0a2dd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a380ab0d0 0x7f0a380cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34506/0 (socket says 172.21.15.121:34506) 2023-11-15T03:06:12.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.149+0000 7f0a2dd9b700 1 -- 172.21.15.121:0/3906173551 learned_addr learned my addr 172.21.15.121:0/3906173551 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:12.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.149+0000 7f0a36d9d700 1 --2- 172.21.15.121:0/3906173551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a380ad1b0 0x7f0a380cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:12.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.149+0000 7f0a3659c700 1 --2- 172.21.15.121:0/3906173551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a38065c00 0x7f0a38163910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:12.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.153+0000 7f0a2dd9b700 1 -- 172.21.15.121:0/3906173551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a380ad1b0 msgr2=0x7f0a380cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:12.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.153+0000 7f0a2dd9b700 1 --2- 172.21.15.121:0/3906173551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a380ad1b0 0x7f0a380cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:12.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.153+0000 7f0a2dd9b700 1 -- 172.21.15.121:0/3906173551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a38065c00 msgr2=0x7f0a38163910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:12.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.153+0000 7f0a2dd9b700 1 --2- 172.21.15.121:0/3906173551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a38065c00 0x7f0a38163910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:12.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.153+0000 7f0a2dd9b700 1 -- 172.21.15.121:0/3906173551 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a200092a0 con 0x7f0a380ab0d0 2023-11-15T03:06:12.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.153+0000 7f0a3659c700 1 --2- 172.21.15.121:0/3906173551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a38065c00 0x7f0a38163910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:12.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.153+0000 7f0a36d9d700 1 --2- 172.21.15.121:0/3906173551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a380ad1b0 0x7f0a380cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:12.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.153+0000 7f0a2dd9b700 1 --2- 172.21.15.121:0/3906173551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a380ab0d0 0x7f0a380cadc0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7f0a38065120 tx=0x7f0a20017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:12.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.153+0000 7f0a2effd700 1 -- 172.21.15.121:0/3906173551 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2000f9f0 con 0x7f0a380ab0d0 2023-11-15T03:06:12.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.153+0000 7f0a2effd700 1 -- 172.21.15.121:0/3906173551 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a20000c90 con 0x7f0a380ab0d0 2023-11-15T03:06:12.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.153+0000 7f0a3759e700 1 -- 172.21.15.121:0/3906173551 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a380a83b0 con 0x7f0a380ab0d0 2023-11-15T03:06:12.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.153+0000 7f0a2effd700 1 -- 172.21.15.121:0/3906173551 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2001c650 con 0x7f0a380ab0d0 2023-11-15T03:06:12.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.153+0000 7f0a3759e700 1 -- 172.21.15.121:0/3906173551 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a380a8970 con 0x7f0a380ab0d0 2023-11-15T03:06:12.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.153+0000 7f0a3759e700 1 -- 172.21.15.121:0/3906173551 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3806bcf0 con 0x7f0a380ab0d0 2023-11-15T03:06:12.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.157+0000 7f0a2effd700 1 -- 172.21.15.121:0/3906173551 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0a2001ca00 con 0x7f0a380ab0d0 2023-11-15T03:06:12.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.157+0000 7f0a2effd700 1 --2- 172.21.15.121:0/3906173551 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0a1805a540 0x7f0a1805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:12.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.157+0000 7f0a2effd700 1 -- 172.21.15.121:0/3906173551 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7556+0+0 (secure 0 0 0) 0x7f0a20076a60 con 0x7f0a380ab0d0 2023-11-15T03:06:12.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.157+0000 7f0a3659c700 1 --2- 172.21.15.121:0/3906173551 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0a1805a540 0x7f0a1805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:12.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.157+0000 7f0a3659c700 1 --2- 172.21.15.121:0/3906173551 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0a1805a540 0x7f0a1805ca00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f0a28009200 tx=0x7f0a28008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:12.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.165+0000 7f0a2effd700 1 -- 172.21.15.121:0/3906173551 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0a20049be0 con 0x7f0a380ab0d0 2023-11-15T03:06:12.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:12.381+0000 7f0a3759e700 1 -- 172.21.15.121:0/3906173551 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0a380c7160 con 0x7f0a380ab0d0 2023-11-15T03:06:13.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.321+0000 7f0a2effd700 1 -- 172.21.15.121:0/3906173551 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v178) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0a2001ccb0 con 0x7f0a380ab0d0 2023-11-15T03:06:13.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.321+0000 7f0a3759e700 1 -- 172.21.15.121:0/3906173551 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0a1805a540 msgr2=0x7f0a1805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:13.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.321+0000 7f0a3759e700 1 --2- 172.21.15.121:0/3906173551 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0a1805a540 0x7f0a1805ca00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f0a28009200 tx=0x7f0a28008040).stop 2023-11-15T03:06:13.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.321+0000 7f0a3759e700 1 -- 172.21.15.121:0/3906173551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a380ab0d0 msgr2=0x7f0a380cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:13.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.321+0000 7f0a3759e700 1 --2- 172.21.15.121:0/3906173551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a380ab0d0 0x7f0a380cadc0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7f0a38065120 tx=0x7f0a20017000).stop 2023-11-15T03:06:13.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.321+0000 7f0a3759e700 1 -- 172.21.15.121:0/3906173551 shutdown_connections 2023-11-15T03:06:13.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.321+0000 7f0a3759e700 1 --2- 172.21.15.121:0/3906173551 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0a1805a540 0x7f0a1805ca00 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:13.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.321+0000 7f0a3759e700 1 --2- 172.21.15.121:0/3906173551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a380ad1b0 0x7f0a380cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:13.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.321+0000 7f0a3759e700 1 --2- 172.21.15.121:0/3906173551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a380ab0d0 0x7f0a380cadc0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:13.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.321+0000 7f0a3759e700 1 --2- 172.21.15.121:0/3906173551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a38065c00 0x7f0a38163910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:13.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.321+0000 7f0a3759e700 1 -- 172.21.15.121:0/3906173551 >> 172.21.15.121:0/3906173551 conn(0x7f0a38067a60 msgr2=0x7f0a38155f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:13.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.325+0000 7f0a3759e700 1 -- 172.21.15.121:0/3906173551 shutdown_connections 2023-11-15T03:06:13.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.325+0000 7f0a3759e700 1 -- 172.21.15.121:0/3906173551 wait complete. 2023-11-15T03:06:13.336 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:06:13.355 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:06:13.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e4e68f700 1 Processor -- start 2023-11-15T03:06:13.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e4e68f700 1 -- start start 2023-11-15T03:06:13.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e4e68f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3e480669b0 0x7f3e48158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:13.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e4e68f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3e480a8750 0x7f3e480a8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:13.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e4e68f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e480bb910 con 0x7f3e48157d90 2023-11-15T03:06:13.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e4e68f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e480c1fb0 con 0x7f3e480a8750 2023-11-15T03:06:13.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e4e68f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e4815bbd0 con 0x7f3e480669b0 2023-11-15T03:06:13.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e4d68d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3e48157d90 0x7f3e480c1760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:38820/0 (socket says 172.21.15.121:38820) 2023-11-15T03:06:13.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e4d68d700 1 -- 172.21.15.121:0/3897900126 learned_addr learned my addr 172.21.15.121:0/3897900126 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:13.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e4ce8c700 1 --2- 172.21.15.121:0/3897900126 >> v2:172.21.15.190:3300/0 conn(0x7f3e480669b0 0x7f3e48158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:13.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e4de8e700 1 --2- 172.21.15.121:0/3897900126 >> v2:172.21.15.121:3300/0 conn(0x7f3e480a8750 0x7f3e480a8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:13.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e3ffff700 1 -- 172.21.15.121:0/3897900126 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 687425426 0 0) 0x7f3e480bb910 con 0x7f3e48157d90 2023-11-15T03:06:13.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e3ffff700 1 -- 172.21.15.121:0/3897900126 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e340033f0 con 0x7f3e48157d90 2023-11-15T03:06:13.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e4de8e700 1 -- 172.21.15.121:0/3897900126 >> v2:172.21.15.190:3300/0 conn(0x7f3e480669b0 msgr2=0x7f3e48158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:13.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e4de8e700 1 --2- 172.21.15.121:0/3897900126 >> v2:172.21.15.190:3300/0 conn(0x7f3e480669b0 0x7f3e48158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:13.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e4de8e700 1 -- 172.21.15.121:0/3897900126 >> v1:172.21.15.121:6789/0 conn(0x7f3e48157d90 legacy=0x7f3e480c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:13.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e4de8e700 1 -- 172.21.15.121:0/3897900126 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e480a9090 con 0x7f3e480a8750 2023-11-15T03:06:13.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e4ce8c700 1 --2- 172.21.15.121:0/3897900126 >> v2:172.21.15.190:3300/0 conn(0x7f3e480669b0 0x7f3e48158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:06:13.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e4de8e700 1 --2- 172.21.15.121:0/3897900126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e480a8750 0x7f3e480a8ba0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7f3e4815bd10 tx=0x7f3e40009780).ready entity=mon.0 client_cookie=3f5f3b000cbc697 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:13.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e3ffff700 1 -- 172.21.15.121:0/3897900126 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4000e070 con 0x7f3e480a8750 2023-11-15T03:06:13.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e3ffff700 1 -- 172.21.15.121:0/3897900126 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4000ba80 con 0x7f3e480a8750 2023-11-15T03:06:13.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e3ffff700 1 -- 172.21.15.121:0/3897900126 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4000be00 con 0x7f3e480a8750 2023-11-15T03:06:13.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e4e68f700 1 -- 172.21.15.121:0/3897900126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e480a8750 msgr2=0x7f3e480a8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:13.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.509+0000 7f3e4e68f700 1 --2- 172.21.15.121:0/3897900126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e480a8750 0x7f3e480a8ba0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7f3e4815bd10 tx=0x7f3e40009780).stop 2023-11-15T03:06:13.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.513+0000 7f3e4e68f700 1 -- 172.21.15.121:0/3897900126 shutdown_connections 2023-11-15T03:06:13.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.513+0000 7f3e4e68f700 1 --2- 172.21.15.121:0/3897900126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e480a8750 0x7f3e480a8ba0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:13.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.513+0000 7f3e4e68f700 1 --2- 172.21.15.121:0/3897900126 >> v2:172.21.15.190:3300/0 conn(0x7f3e480669b0 0x7f3e48158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:13.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.513+0000 7f3e4e68f700 1 -- 172.21.15.121:0/3897900126 >> 172.21.15.121:0/3897900126 conn(0x7f3e48067a60 msgr2=0x7f3e48067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:13.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.513+0000 7f3e4e68f700 1 -- 172.21.15.121:0/3897900126 shutdown_connections 2023-11-15T03:06:13.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.513+0000 7f3e4e68f700 1 -- 172.21.15.121:0/3897900126 wait complete. 2023-11-15T03:06:13.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.513+0000 7f3e4e68f700 1 Processor -- start 2023-11-15T03:06:13.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.513+0000 7f3e4e68f700 1 -- start start 2023-11-15T03:06:13.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e4e68f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e480669b0 0x7f3e481564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:13.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e4e68f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e480a8750 0x7f3e48149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:13.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e4e68f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e48157d90 0x7f3e48153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:13.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e4e68f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e480c4380 con 0x7f3e48157d90 2023-11-15T03:06:13.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e4e68f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e480c45f0 con 0x7f3e480669b0 2023-11-15T03:06:13.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e4e68f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e481582b0 con 0x7f3e480a8750 2023-11-15T03:06:13.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e4de8e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e48157d90 0x7f3e48153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:13.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e4de8e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e48157d90 0x7f3e48153110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34526/0 (socket says 172.21.15.121:34526) 2023-11-15T03:06:13.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e4de8e700 1 -- 172.21.15.121:0/73524240 learned_addr learned my addr 172.21.15.121:0/73524240 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:13.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e4d68d700 1 --2- 172.21.15.121:0/73524240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e480669b0 0x7f3e481564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:13.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e4ce8c700 1 --2- 172.21.15.121:0/73524240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e480a8750 0x7f3e48149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:13.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e4de8e700 1 -- 172.21.15.121:0/73524240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e480a8750 msgr2=0x7f3e48149e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:13.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e4de8e700 1 --2- 172.21.15.121:0/73524240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e480a8750 0x7f3e48149e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:13.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e4de8e700 1 -- 172.21.15.121:0/73524240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e480669b0 msgr2=0x7f3e481564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:13.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e4de8e700 1 --2- 172.21.15.121:0/73524240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e480669b0 0x7f3e481564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:13.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e4de8e700 1 -- 172.21.15.121:0/73524240 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e38009250 con 0x7f3e48157d90 2023-11-15T03:06:13.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e4de8e700 1 --2- 172.21.15.121:0/73524240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e48157d90 0x7f3e48153110 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f3e400097c0 tx=0x7f3e48063a10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:13.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e3d7fa700 1 -- 172.21.15.121:0/73524240 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4000b610 con 0x7f3e48157d90 2023-11-15T03:06:13.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e3d7fa700 1 -- 172.21.15.121:0/73524240 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4001d630 con 0x7f3e48157d90 2023-11-15T03:06:13.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e4e68f700 1 -- 172.21.15.121:0/73524240 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e40009a20 con 0x7f3e48157d90 2023-11-15T03:06:13.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e3d7fa700 1 -- 172.21.15.121:0/73524240 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4001d9b0 con 0x7f3e48157d90 2023-11-15T03:06:13.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.517+0000 7f3e4e68f700 1 -- 172.21.15.121:0/73524240 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e48156ad0 con 0x7f3e48157d90 2023-11-15T03:06:13.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.521+0000 7f3e4e68f700 1 -- 172.21.15.121:0/73524240 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e4806bcf0 con 0x7f3e48157d90 2023-11-15T03:06:13.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.521+0000 7f3e3d7fa700 1 -- 172.21.15.121:0/73524240 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3e40003040 con 0x7f3e48157d90 2023-11-15T03:06:13.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.521+0000 7f3e3d7fa700 1 --2- 172.21.15.121:0/73524240 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e2c05a510 0x7f3e2c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:13.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.521+0000 7f3e3d7fa700 1 -- 172.21.15.121:0/73524240 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7556+0+0 (secure 0 0 0) 0x7f3e40076c90 con 0x7f3e48157d90 2023-11-15T03:06:13.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.521+0000 7f3e4d68d700 1 --2- 172.21.15.121:0/73524240 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e2c05a510 0x7f3e2c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:13.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.525+0000 7f3e4d68d700 1 --2- 172.21.15.121:0/73524240 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e2c05a510 0x7f3e2c05c9d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f3e44005750 tx=0x7f3e44009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:13.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.525+0000 7f3e3d7fa700 1 -- 172.21.15.121:0/73524240 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3e40049e10 con 0x7f3e48157d90 2023-11-15T03:06:13.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:13.745+0000 7f3e4e68f700 1 -- 172.21.15.121:0/73524240 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f3e48068a70 con 0x7f3e48157d90 2023-11-15T03:06:14.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.337+0000 7f3e3d7fa700 1 -- 172.21.15.121:0/73524240 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v179) v1 ==== 101+0+0 (secure 0 0 0) 0x7f3e4004d430 con 0x7f3e48157d90 2023-11-15T03:06:14.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.337+0000 7f3e4e68f700 1 -- 172.21.15.121:0/73524240 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e2c05a510 msgr2=0x7f3e2c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.337+0000 7f3e4e68f700 1 --2- 172.21.15.121:0/73524240 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e2c05a510 0x7f3e2c05c9d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f3e44005750 tx=0x7f3e44009040).stop 2023-11-15T03:06:14.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.337+0000 7f3e4e68f700 1 -- 172.21.15.121:0/73524240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e48157d90 msgr2=0x7f3e48153110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.337+0000 7f3e4e68f700 1 --2- 172.21.15.121:0/73524240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e48157d90 0x7f3e48153110 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f3e400097c0 tx=0x7f3e48063a10).stop 2023-11-15T03:06:14.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.341+0000 7f3e4e68f700 1 -- 172.21.15.121:0/73524240 shutdown_connections 2023-11-15T03:06:14.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.341+0000 7f3e4e68f700 1 --2- 172.21.15.121:0/73524240 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e2c05a510 0x7f3e2c05c9d0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.341+0000 7f3e4e68f700 1 --2- 172.21.15.121:0/73524240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e48157d90 0x7f3e48153110 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.341+0000 7f3e4e68f700 1 --2- 172.21.15.121:0/73524240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e480a8750 0x7f3e48149e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.341+0000 7f3e4e68f700 1 --2- 172.21.15.121:0/73524240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e480669b0 0x7f3e481564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.341+0000 7f3e4e68f700 1 -- 172.21.15.121:0/73524240 >> 172.21.15.121:0/73524240 conn(0x7f3e48067a60 msgr2=0x7f3e480c6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:14.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.341+0000 7f3e4e68f700 1 -- 172.21.15.121:0/73524240 shutdown_connections 2023-11-15T03:06:14.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.341+0000 7f3e4e68f700 1 -- 172.21.15.121:0/73524240 wait complete. 2023-11-15T03:06:14.349 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:06:14.366 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:06:14.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.421+0000 7f25df4ebd00 1 Processor -- start 2023-11-15T03:06:14.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.421+0000 7f25df4ebd00 1 -- start start 2023-11-15T03:06:14.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25df4ebd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e43f5a30 0x5636e44d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25df4ebd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5636e43f5e10 0x5636e4441d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25df4ebd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636e4444b20 0x5636e44cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25df4ebd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e436e220 con 0x5636e43f5a30 2023-11-15T03:06:14.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25df4ebd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e44cbc70 con 0x5636e4444b20 2023-11-15T03:06:14.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25df4ebd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636e4441090 con 0x5636e43f5e10 2023-11-15T03:06:14.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25dcdcd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e43f5a30 0x5636e44d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25dcdcd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e43f5a30 0x5636e44d0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34542/0 (socket says 172.21.15.121:34542) 2023-11-15T03:06:14.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25dcdcd700 1 -- 172.21.15.121:0/2930168080 learned_addr learned my addr 172.21.15.121:0/2930168080 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:14.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25d7fff700 1 --2- 172.21.15.121:0/2930168080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5636e43f5e10 0x5636e4441d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25dd5ce700 1 --2- 172.21.15.121:0/2930168080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636e4444b20 0x5636e44cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25dcdcd700 1 -- 172.21.15.121:0/2930168080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5636e43f5e10 msgr2=0x5636e4441d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:14.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25dcdcd700 1 --2- 172.21.15.121:0/2930168080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5636e43f5e10 0x5636e4441d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25dcdcd700 1 -- 172.21.15.121:0/2930168080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636e4444b20 msgr2=0x5636e44cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:14.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25dcdcd700 1 --2- 172.21.15.121:0/2930168080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636e4444b20 0x5636e44cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25dcdcd700 1 -- 172.21.15.121:0/2930168080 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d800a0e0 con 0x5636e43f5a30 2023-11-15T03:06:14.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25dd5ce700 1 --2- 172.21.15.121:0/2930168080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636e4444b20 0x5636e44cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:14.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25dcdcd700 1 --2- 172.21.15.121:0/2930168080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e43f5a30 0x5636e44d0680 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7f25d0001d30 tx=0x7f25d0001810).ready entity=mon.0 client_cookie=30f0df0267bf73b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:14.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25d6ffd700 1 -- 172.21.15.121:0/2930168080 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d000bcc0 con 0x5636e43f5a30 2023-11-15T03:06:14.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25d6ffd700 1 -- 172.21.15.121:0/2930168080 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d000be20 con 0x5636e43f5a30 2023-11-15T03:06:14.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25d6ffd700 1 -- 172.21.15.121:0/2930168080 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d000e680 con 0x5636e43f5a30 2023-11-15T03:06:14.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25df4ebd00 1 -- 172.21.15.121:0/2930168080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e43f5a30 msgr2=0x5636e44d0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/2930168080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e43f5a30 0x5636e44d0680 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7f25d0001d30 tx=0x7f25d0001810).stop 2023-11-15T03:06:14.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25df4ebd00 1 -- 172.21.15.121:0/2930168080 shutdown_connections 2023-11-15T03:06:14.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/2930168080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636e4444b20 0x5636e44cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/2930168080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5636e43f5e10 0x5636e4441d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/2930168080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e43f5a30 0x5636e44d0680 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25df4ebd00 1 -- 172.21.15.121:0/2930168080 >> 172.21.15.121:0/2930168080 conn(0x5636e4445080 msgr2=0x5636e44271a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:14.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25df4ebd00 1 -- 172.21.15.121:0/2930168080 shutdown_connections 2023-11-15T03:06:14.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.425+0000 7f25df4ebd00 1 -- 172.21.15.121:0/2930168080 wait complete. 2023-11-15T03:06:14.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.429+0000 7f25df4ebd00 1 Processor -- start 2023-11-15T03:06:14.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.429+0000 7f25df4ebd00 1 -- start start 2023-11-15T03:06:14.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.429+0000 7f25df4ebd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5636e43f5a30 0x5636e44ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.429+0000 7f25df4ebd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636e43f5e10 0x5636e44d6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.429+0000 7f25df4ebd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e4444b20 0x5636e44ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.429+0000 7f25df4ebd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e443f9d0 con 0x5636e4444b20 2023-11-15T03:06:14.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.429+0000 7f25df4ebd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e443fcb0 con 0x5636e43f5e10 2023-11-15T03:06:14.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.429+0000 7f25df4ebd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636e443ff90 con 0x5636e43f5a30 2023-11-15T03:06:14.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.429+0000 7f25dd5ce700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e4444b20 0x5636e44ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.429+0000 7f25dd5ce700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e4444b20 0x5636e44ca110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34546/0 (socket says 172.21.15.121:34546) 2023-11-15T03:06:14.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.429+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 learned_addr learned my addr 172.21.15.121:0/1423333378 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:14.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.429+0000 7f25dcdcd700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5636e43f5a30 0x5636e44ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.429+0000 7f25d7fff700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636e43f5e10 0x5636e44d6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.433+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5636e43f5a30 msgr2=0x5636e44ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.433+0000 7f25dd5ce700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5636e43f5a30 0x5636e44ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.433+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636e43f5e10 msgr2=0x5636e44d6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.433+0000 7f25dd5ce700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636e43f5e10 0x5636e44d6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.433+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c4001e10 con 0x5636e4444b20 2023-11-15T03:06:14.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.433+0000 7f25dd5ce700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e4444b20 0x5636e44ca110 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7f25d800f000 tx=0x7f25d8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:14.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.433+0000 7f25d57fa700 1 -- 172.21.15.121:0/1423333378 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d800d470 con 0x5636e4444b20 2023-11-15T03:06:14.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.433+0000 7f25d57fa700 1 -- 172.21.15.121:0/1423333378 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d800abc0 con 0x5636e4444b20 2023-11-15T03:06:14.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.433+0000 7f25d57fa700 1 -- 172.21.15.121:0/1423333378 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d80058b0 con 0x5636e4444b20 2023-11-15T03:06:14.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.433+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5636e4397770 con 0x5636e4444b20 2023-11-15T03:06:14.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.433+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636e4366a70 con 0x5636e4444b20 2023-11-15T03:06:14.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.437+0000 7f25d57fa700 1 -- 172.21.15.121:0/1423333378 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f25d8003040 con 0x5636e4444b20 2023-11-15T03:06:14.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.437+0000 7f25d57fa700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25c8058ff0 0x7f25c805b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.437+0000 7f25d57fa700 1 -- 172.21.15.121:0/1423333378 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7556+0+0 (secure 0 0 0) 0x7f25d8074ed0 con 0x5636e4444b20 2023-11-15T03:06:14.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.437+0000 7f25dcdcd700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25c8058ff0 0x7f25c805b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.451 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:06:14.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.437+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5636e44d53a0 0x5636e44e2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.437+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44cb3e0 con 0x5636e44d53a0 2023-11-15T03:06:14.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.437+0000 7f25dcdcd700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25c8058ff0 0x7f25c805b4b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f25d0001d30 tx=0x7f25d0014000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:14.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.437+0000 7f25d7fff700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5636e44d53a0 0x5636e44e2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.437+0000 7f25d7fff700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5636e44d53a0 0x5636e44e2ce0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:14.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.441+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v109'173 uv166 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44d53a0 2023-11-15T03:06:14.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.441+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e44e31d0 con 0x5636e44d53a0 2023-11-15T03:06:14.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.445+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v109'174 uv174 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44d53a0 2023-11-15T03:06:14.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.445+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44e3580 con 0x5636e44d53a0 2023-11-15T03:06:14.454 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:06:14.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.449+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v109'175 uv175 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44d53a0 2023-11-15T03:06:14.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.449+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f02c0 con 0x5636e44d53a0 2023-11-15T03:06:14.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.449+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v109'176 uv176 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44d53a0 2023-11-15T03:06:14.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.449+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f0670 con 0x5636e44d53a0 2023-11-15T03:06:14.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.453+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v109'177 uv177 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44d53a0 2023-11-15T03:06:14.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.453+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44ef2b0 con 0x5636e44d53a0 2023-11-15T03:06:14.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.457+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v109'178 uv178 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44d53a0 2023-11-15T03:06:14.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.457+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44ef6b0 con 0x5636e44d53a0 2023-11-15T03:06:14.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.457+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v109'179 uv172 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44d53a0 2023-11-15T03:06:14.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.457+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e44ef6b0 con 0x5636e44d53a0 2023-11-15T03:06:14.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.461+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v109'180 uv180 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44d53a0 2023-11-15T03:06:14.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.461+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44ef6b0 con 0x5636e44d53a0 2023-11-15T03:06:14.469 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:06:14.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.461+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v109'181 uv181 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44d53a0 2023-11-15T03:06:14.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.461+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44ef6b0 con 0x5636e44d53a0 2023-11-15T03:06:14.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.465+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v109'182 uv182 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44d53a0 2023-11-15T03:06:14.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.465+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44ef6b0 con 0x5636e44d53a0 2023-11-15T03:06:14.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.469+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v109'183 uv183 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44d53a0 2023-11-15T03:06:14.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.469+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44ef6b0 con 0x5636e44d53a0 2023-11-15T03:06:14.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.473+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v109'184 uv184 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44d53a0 2023-11-15T03:06:14.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.473+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5636e44efbc0 0x5636e44f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.473+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f2f20 con 0x5636e44efbc0 2023-11-15T03:06:14.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.473+0000 7f25dcdcd700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5636e44efbc0 0x5636e44f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.473+0000 7f25dcdcd700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5636e44efbc0 0x5636e44f2a30 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:14.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.477+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v109'271 uv258 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.477+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e44f32d0 con 0x5636e44efbc0 2023-11-15T03:06:14.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.481+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v109'272 uv272 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.481+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f3680 con 0x5636e44efbc0 2023-11-15T03:06:14.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.485+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v109'273 uv273 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.492 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:06:14.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.485+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f4a40 con 0x5636e44efbc0 2023-11-15T03:06:14.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.489+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v109'274 uv274 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.489+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f4df0 con 0x5636e44efbc0 2023-11-15T03:06:14.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.489+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v109'275 uv275 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.489+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f3a30 con 0x5636e44efbc0 2023-11-15T03:06:14.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.497+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v109'276 uv276 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.497+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f4320 con 0x5636e44efbc0 2023-11-15T03:06:14.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.497+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v109'277 uv277 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.497+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f51a0 con 0x5636e44efbc0 2023-11-15T03:06:14.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.501+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v109'278 uv264 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.501+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e44f51a0 con 0x5636e44efbc0 2023-11-15T03:06:14.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.505+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v109'279 uv279 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.505+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f51a0 con 0x5636e44efbc0 2023-11-15T03:06:14.513 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:06:14.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.509+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v109'280 uv280 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.509+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f61b0 con 0x5636e44efbc0 2023-11-15T03:06:14.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.509+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v109'281 uv281 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.509+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f61b0 con 0x5636e44efbc0 2023-11-15T03:06:14.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.513+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v109'282 uv282 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.513+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f51a0 con 0x5636e44efbc0 2023-11-15T03:06:14.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.517+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v109'283 uv283 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.517+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f51a0 con 0x5636e44efbc0 2023-11-15T03:06:14.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.517+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v109'284 uv270 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.517+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e44f51a0 con 0x5636e44efbc0 2023-11-15T03:06:14.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.521+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v109'285 uv285 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.521+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f51a0 con 0x5636e44efbc0 2023-11-15T03:06:14.530 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:06:14.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.525+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v109'286 uv286 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.525+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f61b0 con 0x5636e44efbc0 2023-11-15T03:06:14.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.525+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v109'287 uv287 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.525+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f61b0 con 0x5636e44efbc0 2023-11-15T03:06:14.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.529+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v109'288 uv288 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.529+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f51a0 con 0x5636e44efbc0 2023-11-15T03:06:14.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.533+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v109'289 uv289 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.533+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5636e44f56b0 0x5636e44f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.533+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f7f90 con 0x5636e44f56b0 2023-11-15T03:06:14.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.533+0000 7f25d7fff700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5636e44f56b0 0x5636e44f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.533+0000 7f25d7fff700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5636e44f56b0 0x5636e44f7aa0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:14.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.537+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v109'148 uv142 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44f56b0 2023-11-15T03:06:14.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.537+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e44f7f90 con 0x5636e44f56b0 2023-11-15T03:06:14.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.541+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v109'149 uv149 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44f56b0 2023-11-15T03:06:14.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.541+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f7f90 con 0x5636e44f56b0 2023-11-15T03:06:14.549 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:06:14.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.545+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v109'150 uv150 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44f56b0 2023-11-15T03:06:14.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.545+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f8480 con 0x5636e44f56b0 2023-11-15T03:06:14.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.545+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v109'151 uv151 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44f56b0 2023-11-15T03:06:14.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.545+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f8480 con 0x5636e44f56b0 2023-11-15T03:06:14.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.549+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v109'152 uv152 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44f56b0 2023-11-15T03:06:14.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.549+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f8480 con 0x5636e44f56b0 2023-11-15T03:06:14.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.553+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v109'153 uv147 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44f56b0 2023-11-15T03:06:14.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.553+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e44f8480 con 0x5636e44f56b0 2023-11-15T03:06:14.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.557+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v109'154 uv154 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44f56b0 2023-11-15T03:06:14.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.557+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f8480 con 0x5636e44f56b0 2023-11-15T03:06:14.566 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:06:14.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.561+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v109'155 uv155 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44f56b0 2023-11-15T03:06:14.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.561+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f8480 con 0x5636e44f56b0 2023-11-15T03:06:14.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.561+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v109'156 uv156 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44f56b0 2023-11-15T03:06:14.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.561+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f8480 con 0x5636e44f56b0 2023-11-15T03:06:14.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.565+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v109'157 uv157 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44f56b0 2023-11-15T03:06:14.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.565+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5636e44f8990 0x5636e44fad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.565+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44fb270 con 0x5636e44f8990 2023-11-15T03:06:14.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.565+0000 7f25dd5ce700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5636e44f8990 0x5636e44fad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.569+0000 7f25dd5ce700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5636e44f8990 0x5636e44fad80 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:14.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.569+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v109'182 uv174 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.569+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e44fb620 con 0x5636e44f8990 2023-11-15T03:06:14.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.573+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v109'183 uv183 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.573+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44fba50 con 0x5636e44f8990 2023-11-15T03:06:14.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.577+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v109'184 uv184 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.584 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:06:14.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.577+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44f8050 con 0x5636e44f8990 2023-11-15T03:06:14.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.581+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v109'185 uv185 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.581+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45004d0 con 0x5636e44f8990 2023-11-15T03:06:14.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.585+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v109'186 uv186 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.585+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44fbe50 con 0x5636e44f8990 2023-11-15T03:06:14.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.585+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v109'187 uv180 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.585+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e44fc250 con 0x5636e44f8990 2023-11-15T03:06:14.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.589+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v109'188 uv188 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.589+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44fcb40 con 0x5636e44f8990 2023-11-15T03:06:14.599 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:06:14.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.593+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v109'189 uv189 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.593+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44fdb50 con 0x5636e44f8990 2023-11-15T03:06:14.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.597+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v109'190 uv190 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.597+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44fd850 con 0x5636e44f8990 2023-11-15T03:06:14.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.597+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v109'191 uv191 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.597+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44fd830 con 0x5636e44f8990 2023-11-15T03:06:14.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.601+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v109'192 uv192 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.601+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5636e44fcc50 0x5636e44fd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.601+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44ffc60 con 0x5636e44fcc50 2023-11-15T03:06:14.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.601+0000 7f25dcdcd700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5636e44fcc50 0x5636e44fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.605+0000 7f25dcdcd700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5636e44fcc50 0x5636e44fd090 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:14.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.605+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v109'87 uv86 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44fcc50 2023-11-15T03:06:14.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.605+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e44ffc60 con 0x5636e44fcc50 2023-11-15T03:06:14.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.609+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v109'88 uv88 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44fcc50 2023-11-15T03:06:14.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.609+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44ffc60 con 0x5636e44fcc50 2023-11-15T03:06:14.619 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:06:14.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.613+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v109'89 uv89 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44fcc50 2023-11-15T03:06:14.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.613+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44ffc60 con 0x5636e44fcc50 2023-11-15T03:06:14.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.613+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v109'90 uv90 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44fcc50 2023-11-15T03:06:14.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.613+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44ffc60 con 0x5636e44fcc50 2023-11-15T03:06:14.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.617+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v109'91 uv91 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44fcc50 2023-11-15T03:06:14.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.617+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e44ffc60 con 0x5636e44fcc50 2023-11-15T03:06:14.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.621+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v109'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44fcc50 2023-11-15T03:06:14.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.621+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5636e4500060 0x5636e4502df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.621+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45032e0 con 0x5636e4500060 2023-11-15T03:06:14.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.621+0000 7f25d7fff700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5636e4500060 0x5636e4502df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.621+0000 7f25d7fff700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5636e4500060 0x5636e4502df0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:14.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.625+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v109'64 uv63 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.625+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e4500060 2023-11-15T03:06:14.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.629+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v109'65 uv65 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.629+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e4500060 2023-11-15T03:06:14.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.629+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v109'66 uv66 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.637 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:06:14.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.629+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4503720 con 0x5636e4500060 2023-11-15T03:06:14.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.633+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v109'67 uv67 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.633+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e4500060 2023-11-15T03:06:14.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.637+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v109'87 uv86 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.637+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e4500060 2023-11-15T03:06:14.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.637+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v109'88 uv88 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.637+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e4500060 2023-11-15T03:06:14.648 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:06:14.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.641+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v109'89 uv89 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.641+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45046f0 con 0x5636e4500060 2023-11-15T03:06:14.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.645+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v109'90 uv90 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.645+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45046f0 con 0x5636e4500060 2023-11-15T03:06:14.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.645+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v109'91 uv91 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.649+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e4500060 2023-11-15T03:06:14.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.649+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v109'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.649+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e44efbc0 2023-11-15T03:06:14.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.653+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v109'160 uv154 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.653+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e44efbc0 2023-11-15T03:06:14.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.657+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v109'161 uv161 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.657+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e44efbc0 2023-11-15T03:06:14.666 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:06:14.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.661+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v109'162 uv162 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.661+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45046f0 con 0x5636e44efbc0 2023-11-15T03:06:14.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.665+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v109'163 uv163 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.665+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45046f0 con 0x5636e44efbc0 2023-11-15T03:06:14.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.665+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v109'164 uv164 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.665+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e44efbc0 2023-11-15T03:06:14.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.669+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v109'165 uv165 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.669+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e44efbc0 2023-11-15T03:06:14.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.673+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v109'166 uv159 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.673+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e44efbc0 2023-11-15T03:06:14.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.677+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v109'167 uv167 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.677+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e44efbc0 2023-11-15T03:06:14.684 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:06:14.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.677+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v109'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.677+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e44efbc0 2023-11-15T03:06:14.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.681+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v109'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.681+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e44efbc0 2023-11-15T03:06:14.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.685+0000 7f25dcdcd700 1 -- 172.21.15.121:0/1423333378 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v109'170 uv170 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d000eac0 con 0x5636e44efbc0 2023-11-15T03:06:14.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.685+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e44f56b0 2023-11-15T03:06:14.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.685+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v109'71 uv70 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44f56b0 2023-11-15T03:06:14.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.685+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e44f56b0 2023-11-15T03:06:14.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.689+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v109'72 uv72 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44f56b0 2023-11-15T03:06:14.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.689+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e44f56b0 2023-11-15T03:06:14.698 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:06:14.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.693+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v109'73 uv73 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44f56b0 2023-11-15T03:06:14.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.693+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e44f56b0 2023-11-15T03:06:14.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.693+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v109'74 uv74 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44f56b0 2023-11-15T03:06:14.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.693+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4503bd0 con 0x5636e44f56b0 2023-11-15T03:06:14.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.697+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v109'75 uv75 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e44f56b0 2023-11-15T03:06:14.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.697+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4503bd0 con 0x5636e4500060 2023-11-15T03:06:14.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.701+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v109'87 uv86 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.701+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e45036e0 con 0x5636e4500060 2023-11-15T03:06:14.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.701+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v109'88 uv88 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.701+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4503b80 con 0x5636e4500060 2023-11-15T03:06:14.712 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:06:14.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.705+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v109'89 uv89 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.705+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504b90 con 0x5636e4500060 2023-11-15T03:06:14.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.709+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v109'90 uv90 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.709+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504b90 con 0x5636e4500060 2023-11-15T03:06:14.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.713+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v109'91 uv91 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.713+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4503b80 con 0x5636e4500060 2023-11-15T03:06:14.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.713+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v109'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.713+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4503b30 con 0x5636e4500060 2023-11-15T03:06:14.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.717+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v109'72 uv71 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.717+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e4503b30 con 0x5636e4500060 2023-11-15T03:06:14.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.721+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v109'73 uv73 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.721+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4503b30 con 0x5636e4500060 2023-11-15T03:06:14.730 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:06:14.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.725+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v109'74 uv74 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.725+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4503b00 con 0x5636e4500060 2023-11-15T03:06:14.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.729+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v109'75 uv75 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.729+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4503ae0 con 0x5636e4500060 2023-11-15T03:06:14.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.729+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v109'76 uv76 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.729+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4503ae0 con 0x5636e4500060 2023-11-15T03:06:14.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.733+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v109'71 uv70 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.733+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e4503ae0 con 0x5636e4500060 2023-11-15T03:06:14.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.737+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v109'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.737+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4503ae0 con 0x5636e4500060 2023-11-15T03:06:14.746 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:06:14.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.741+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v109'73 uv73 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.741+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4503ae0 con 0x5636e4500060 2023-11-15T03:06:14.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.741+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v109'74 uv74 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.741+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4506850 con 0x5636e4500060 2023-11-15T03:06:14.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.745+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v109'75 uv75 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.745+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5636e45044e0 0x5636e45048c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.745+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e45044e0 2023-11-15T03:06:14.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.749+0000 7f25dd5ce700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5636e45044e0 0x5636e45048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.749+0000 7f25dd5ce700 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5636e45044e0 0x5636e45048c0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:14.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.753+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v109'58 uv57 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e45044e0 2023-11-15T03:06:14.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.753+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e45044e0 2023-11-15T03:06:14.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.757+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v109'59 uv59 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e45044e0 2023-11-15T03:06:14.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.757+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e45044e0 2023-11-15T03:06:14.764 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:06:14.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.757+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v109'60 uv60 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e45044e0 2023-11-15T03:06:14.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.757+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e45044e0 2023-11-15T03:06:14.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.761+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v109'61 uv61 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e45044e0 2023-11-15T03:06:14.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.761+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e4500060 2023-11-15T03:06:14.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.765+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v109'87 uv86 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.765+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e4500060 2023-11-15T03:06:14.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.769+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v109'88 uv88 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.769+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e4500060 2023-11-15T03:06:14.776 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:06:14.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.769+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v109'89 uv89 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.769+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e4500060 2023-11-15T03:06:14.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.773+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v109'90 uv90 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.773+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e4500060 2023-11-15T03:06:14.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.777+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v109'91 uv91 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.777+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e4500060 2023-11-15T03:06:14.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.777+0000 7f25d7fff700 1 -- 172.21.15.121:0/1423333378 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v109'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25cc009bb0 con 0x5636e4500060 2023-11-15T03:06:14.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.777+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e44f8990 2023-11-15T03:06:14.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.781+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v109'74 uv73 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.781+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e44f8990 2023-11-15T03:06:14.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.785+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v109'75 uv75 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.785+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e44f8990 2023-11-15T03:06:14.793 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:06:14.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.789+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v109'76 uv76 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.789+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e44f8990 2023-11-15T03:06:14.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.789+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v109'77 uv77 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.789+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e44f8990 2023-11-15T03:06:14.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.793+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v109'78 uv78 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.793+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e44f8990 2023-11-15T03:06:14.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.797+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v109'174 uv165 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.797+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e44f8990 2023-11-15T03:06:14.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.797+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v109'175 uv175 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.797+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e44f8990 2023-11-15T03:06:14.807 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:06:14.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.801+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v109'176 uv176 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.801+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e44f8990 2023-11-15T03:06:14.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.805+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v109'177 uv177 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.805+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e44f8990 2023-11-15T03:06:14.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.805+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v109'178 uv178 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.805+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e44f8990 2023-11-15T03:06:14.816 INFO:teuthology.orchestra.run.smithi121.stdout:Write #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:06:14.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.809+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v109'179 uv179 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.809+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e44f8990 2023-11-15T03:06:14.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.813+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v109'180 uv180 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.813+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4504f90 con 0x5636e44f8990 2023-11-15T03:06:14.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.813+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v109'181 uv181 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.813+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4505390 con 0x5636e44f8990 2023-11-15T03:06:14.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.817+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [omap-set-header] v109'182 uv182 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.817+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x5636e4505790 con 0x5636e44f8990 2023-11-15T03:06:14.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v109'183 uv183 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f25d8021020 con 0x5636e44f8990 2023-11-15T03:06:14.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5636e44f56b0 msgr2=0x5636e44f7aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5636e44f56b0 0x5636e44f7aa0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5636e44d53a0 msgr2=0x5636e44e2ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5636e44d53a0 0x5636e44e2ce0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5636e44fcc50 msgr2=0x5636e44fd090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5636e44fcc50 0x5636e44fd090 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5636e44efbc0 msgr2=0x5636e44f2a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5636e44efbc0 0x5636e44f2a30 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 reap_dead start 2023-11-15T03:06:14.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5636e45044e0 msgr2=0x5636e45048c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5636e45044e0 0x5636e45048c0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5636e44f8990 msgr2=0x5636e44fad80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5636e44f8990 0x5636e44fad80 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5636e4500060 msgr2=0x5636e4502df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5636e4500060 0x5636e4502df0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 reap_dead start 2023-11-15T03:06:14.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 reap_dead start 2023-11-15T03:06:14.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25dd5ce700 1 -- 172.21.15.121:0/1423333378 reap_dead start 2023-11-15T03:06:14.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25c8058ff0 msgr2=0x7f25c805b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25c8058ff0 0x7f25c805b4b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f25d0001d30 tx=0x7f25d0014000).stop 2023-11-15T03:06:14.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e4444b20 msgr2=0x5636e44ca110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e4444b20 0x5636e44ca110 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7f25d800f000 tx=0x7f25d8015000).stop 2023-11-15T03:06:14.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 shutdown_connections 2023-11-15T03:06:14.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25c8058ff0 0x7f25c805b4b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 --2- 172.21.15.121:0/1423333378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e4444b20 0x5636e44ca110 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 >> 172.21.15.121:0/1423333378 conn(0x5636e4445080 msgr2=0x5636e44271a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:14.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 shutdown_connections 2023-11-15T03:06:14.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.821+0000 7f25df4ebd00 1 -- 172.21.15.121:0/1423333378 wait complete. 2023-11-15T03:06:14.837 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 1.00000000.inode - 2023-11-15T03:06:14.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a38eedd00 1 Processor -- start 2023-11-15T03:06:14.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a38eedd00 1 -- start start 2023-11-15T03:06:14.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a38eedd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a24002490 0x55b7da386060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a38eedd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b7da386550 0x55b7da41e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a38eedd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7da41e9f0 0x55b7da41ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a38eedd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7da2ebcc0 con 0x7f1a24002490 2023-11-15T03:06:14.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a38eedd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7da2b6220 con 0x55b7da41e9f0 2023-11-15T03:06:14.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a38eedd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7da40dc80 con 0x55b7da386550 2023-11-15T03:06:14.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a36822700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a24002490 0x55b7da386060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a36822700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a24002490 0x55b7da386060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39620/0 (socket says 172.21.15.121:39620) 2023-11-15T03:06:14.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a36822700 1 -- 172.21.15.121:0/1442252127 learned_addr learned my addr 172.21.15.121:0/1442252127 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:14.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a2e021700 1 --2- 172.21.15.121:0/1442252127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b7da386550 0x55b7da41e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a37023700 1 --2- 172.21.15.121:0/1442252127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7da41e9f0 0x55b7da41ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a36822700 1 -- 172.21.15.121:0/1442252127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b7da386550 msgr2=0x55b7da41e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:14.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a36822700 1 --2- 172.21.15.121:0/1442252127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b7da386550 0x55b7da41e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a36822700 1 -- 172.21.15.121:0/1442252127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7da41e9f0 msgr2=0x55b7da41ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:14.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a36822700 1 --2- 172.21.15.121:0/1442252127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7da41e9f0 0x55b7da41ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a36822700 1 -- 172.21.15.121:0/1442252127 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7da2aea70 con 0x7f1a24002490 2023-11-15T03:06:14.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a37023700 1 --2- 172.21.15.121:0/1442252127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7da41e9f0 0x55b7da41ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:14.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a36822700 1 --2- 172.21.15.121:0/1442252127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a24002490 0x55b7da386060 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7f1a28001990 tx=0x7f1a280018f0).ready entity=mon.0 client_cookie=330915c6e6c5d82c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:14.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a35820700 1 -- 172.21.15.121:0/1442252127 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a280105e0 con 0x7f1a24002490 2023-11-15T03:06:14.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a35820700 1 -- 172.21.15.121:0/1442252127 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a28010740 con 0x7f1a24002490 2023-11-15T03:06:14.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a35820700 1 -- 172.21.15.121:0/1442252127 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a28010ac0 con 0x7f1a24002490 2023-11-15T03:06:14.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.933+0000 7f1a38eedd00 1 -- 172.21.15.121:0/1442252127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a24002490 msgr2=0x55b7da386060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a38eedd00 1 --2- 172.21.15.121:0/1442252127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a24002490 0x55b7da386060 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7f1a28001990 tx=0x7f1a280018f0).stop 2023-11-15T03:06:14.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a38eedd00 1 -- 172.21.15.121:0/1442252127 shutdown_connections 2023-11-15T03:06:14.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a38eedd00 1 --2- 172.21.15.121:0/1442252127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7da41e9f0 0x55b7da41ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a38eedd00 1 --2- 172.21.15.121:0/1442252127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b7da386550 0x55b7da41e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a38eedd00 1 --2- 172.21.15.121:0/1442252127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a24002490 0x55b7da386060 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a38eedd00 1 -- 172.21.15.121:0/1442252127 >> 172.21.15.121:0/1442252127 conn(0x55b7da3878f0 msgr2=0x55b7da418a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:14.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a38eedd00 1 -- 172.21.15.121:0/1442252127 shutdown_connections 2023-11-15T03:06:14.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a38eedd00 1 -- 172.21.15.121:0/1442252127 wait complete. 2023-11-15T03:06:14.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a38eedd00 1 Processor -- start 2023-11-15T03:06:14.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a38eedd00 1 -- start start 2023-11-15T03:06:14.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a38eedd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7da386550 0x55b7da40ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a38eedd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7da41e9f0 0x55b7da3849b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a38eedd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a24002490 0x55b7da3700d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a38eedd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7da3870f0 con 0x55b7da386550 2023-11-15T03:06:14.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a38eedd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7da418a20 con 0x55b7da41e9f0 2023-11-15T03:06:14.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a38eedd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7da384520 con 0x7f1a24002490 2023-11-15T03:06:14.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a36822700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7da386550 0x55b7da40ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a36822700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7da386550 0x55b7da40ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39634/0 (socket says 172.21.15.121:39634) 2023-11-15T03:06:14.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a36822700 1 -- 172.21.15.121:0/2760456648 learned_addr learned my addr 172.21.15.121:0/2760456648 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:14.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a2e021700 1 --2- 172.21.15.121:0/2760456648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7da41e9f0 0x55b7da3849b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.937+0000 7f1a36822700 1 -- 172.21.15.121:0/2760456648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a24002490 msgr2=0x55b7da3700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.941+0000 7f1a36822700 1 --2- 172.21.15.121:0/2760456648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a24002490 0x55b7da3700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:06:14.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.941+0000 7f1a36822700 1 -- 172.21.15.121:0/2760456648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7da41e9f0 msgr2=0x55b7da3849b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.941+0000 7f1a36822700 1 --2- 172.21.15.121:0/2760456648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7da41e9f0 0x55b7da3849b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.941+0000 7f1a36822700 1 -- 172.21.15.121:0/2760456648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a24001e10 con 0x55b7da386550 2023-11-15T03:06:14.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.941+0000 7f1a2e021700 1 --2- 172.21.15.121:0/2760456648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b7da41e9f0 0x55b7da3849b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:14.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.941+0000 7f1a36822700 1 --2- 172.21.15.121:0/2760456648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7da386550 0x55b7da40ad60 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7f1a28002000 tx=0x7f1a2801c000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:14.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.941+0000 7f1a2ffff700 1 -- 172.21.15.121:0/2760456648 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a28025460 con 0x55b7da386550 2023-11-15T03:06:14.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.941+0000 7f1a2ffff700 1 -- 172.21.15.121:0/2760456648 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a280255c0 con 0x55b7da386550 2023-11-15T03:06:14.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.941+0000 7f1a2ffff700 1 -- 172.21.15.121:0/2760456648 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a28025980 con 0x55b7da386550 2023-11-15T03:06:14.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.941+0000 7f1a38eedd00 1 -- 172.21.15.121:0/2760456648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b7da2ba010 con 0x55b7da386550 2023-11-15T03:06:14.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.941+0000 7f1a38eedd00 1 -- 172.21.15.121:0/2760456648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7da418f50 con 0x55b7da386550 2023-11-15T03:06:14.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.941+0000 7f1a2ffff700 1 -- 172.21.15.121:0/2760456648 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1a28016070 con 0x55b7da386550 2023-11-15T03:06:14.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.945+0000 7f1a2ffff700 1 --2- 172.21.15.121:0/2760456648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1a18058ff0 0x7f1a1805b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.945+0000 7f1a2ffff700 1 -- 172.21.15.121:0/2760456648 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7556+0+0 (secure 0 0 0) 0x7f1a28076770 con 0x55b7da386550 2023-11-15T03:06:14.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.945+0000 7f1a2e021700 1 --2- 172.21.15.121:0/2760456648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1a18058ff0 0x7f1a1805b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.945+0000 7f1a38eedd00 1 --2- 172.21.15.121:0/2760456648 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55b7da384ea0 0x55b7da4267c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:14.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.945+0000 7f1a38eedd00 1 -- 172.21.15.121:0/2760456648 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected e109) v8 -- 0x55b7da341eb0 con 0x55b7da384ea0 2023-11-15T03:06:14.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.945+0000 7f1a37023700 1 --2- 172.21.15.121:0/2760456648 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55b7da384ea0 0x55b7da4267c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:14.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.945+0000 7f1a2e021700 1 --2- 172.21.15.121:0/2760456648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1a18058ff0 0x7f1a1805b4b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x55b7da387480 tx=0x7f1a2000c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:14.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.945+0000 7f1a37023700 1 --2- 172.21.15.121:0/2760456648 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55b7da384ea0 0x55b7da4267c0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:14.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.949+0000 7f1a37023700 1 -- 172.21.15.121:0/2760456648 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(1 1.00000000.inode [writefull 0~80] v109'158 uv158 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1a3000f1e0 con 0x55b7da384ea0 2023-11-15T03:06:14.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.949+0000 7f1a38eedd00 1 -- 172.21.15.121:0/2760456648 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55b7da384ea0 msgr2=0x55b7da4267c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.949+0000 7f1a38eedd00 1 --2- 172.21.15.121:0/2760456648 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55b7da384ea0 0x55b7da4267c0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:14.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.949+0000 7f1a38eedd00 1 -- 172.21.15.121:0/2760456648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1a18058ff0 msgr2=0x7f1a1805b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.949+0000 7f1a38eedd00 1 --2- 172.21.15.121:0/2760456648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1a18058ff0 0x7f1a1805b4b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x55b7da387480 tx=0x7f1a2000c040).stop 2023-11-15T03:06:14.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.949+0000 7f1a38eedd00 1 -- 172.21.15.121:0/2760456648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7da386550 msgr2=0x55b7da40ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:14.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.949+0000 7f1a38eedd00 1 --2- 172.21.15.121:0/2760456648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7da386550 0x55b7da40ad60 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7f1a28002000 tx=0x7f1a2801c000).stop 2023-11-15T03:06:14.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.949+0000 7f1a37023700 1 -- 172.21.15.121:0/2760456648 reap_dead start 2023-11-15T03:06:14.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.949+0000 7f1a38eedd00 1 -- 172.21.15.121:0/2760456648 shutdown_connections 2023-11-15T03:06:14.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.949+0000 7f1a38eedd00 1 -- 172.21.15.121:0/2760456648 >> 172.21.15.121:0/2760456648 conn(0x55b7da3878f0 msgr2=0x55b7da40b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:14.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.949+0000 7f1a38eedd00 1 -- 172.21.15.121:0/2760456648 shutdown_connections 2023-11-15T03:06:14.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:14.949+0000 7f1a38eedd00 1 -- 172.21.15.121:0/2760456648 wait complete. 2023-11-15T03:06:14.961 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:06:15.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.165+0000 7f11f94e4700 1 Processor -- start 2023-11-15T03:06:15.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.165+0000 7f11f94e4700 1 -- start start 2023-11-15T03:06:15.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f94e4700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f11f40672e0 0x7f11f40c14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:15.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f94e4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f11f40c3470 0x7f11f40c19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:15.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f94e4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f4065e40 con 0x7f11f414fc60 2023-11-15T03:06:15.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f94e4700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f40c21e0 con 0x7f11f40672e0 2023-11-15T03:06:15.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f94e4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f40c24c0 con 0x7f11f40c3470 2023-11-15T03:06:15.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f37fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f11f40672e0 0x7f11f40c14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:15.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f37fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f11f40672e0 0x7f11f40c14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39642/0 (socket says 172.21.15.121:39642) 2023-11-15T03:06:15.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f37fe700 1 -- 172.21.15.121:0/3235980579 learned_addr learned my addr 172.21.15.121:0/3235980579 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:15.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f37fe700 1 -- 172.21.15.121:0/3235980579 >> v2:172.21.15.190:3300/0 conn(0x7f11f40c3470 msgr2=0x7f11f40c19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:15.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f8ce3700 1 --2- 172.21.15.121:0/3235980579 >> v2:172.21.15.190:3300/0 conn(0x7f11f40c3470 0x7f11f40c19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:15.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f37fe700 1 --2- 172.21.15.121:0/3235980579 >> v2:172.21.15.190:3300/0 conn(0x7f11f40c3470 0x7f11f40c19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:15.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f37fe700 1 -- 172.21.15.121:0/3235980579 >> v1:172.21.15.190:6789/0 conn(0x7f11f414fc60 legacy=0x7f11f40c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:15.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f37fe700 1 -- 172.21.15.121:0/3235980579 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f40657f0 con 0x7f11f40672e0 2023-11-15T03:06:15.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f8ce3700 1 --2- 172.21.15.121:0/3235980579 >> v2:172.21.15.190:3300/0 conn(0x7f11f40c3470 0x7f11f40c19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:15.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f37fe700 1 --2- 172.21.15.121:0/3235980579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11f40672e0 0x7f11f40c14f0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7f11f40a6c40 tx=0x7f11e800b3b0).ready entity=mon.0 client_cookie=14a5f61364e572f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:15.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f2ffd700 1 -- 172.21.15.121:0/3235980579 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e800dad0 con 0x7f11f40672e0 2023-11-15T03:06:15.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f2ffd700 1 -- 172.21.15.121:0/3235980579 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e800dc30 con 0x7f11f40672e0 2023-11-15T03:06:15.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f2ffd700 1 -- 172.21.15.121:0/3235980579 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e8010470 con 0x7f11f40672e0 2023-11-15T03:06:15.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f94e4700 1 -- 172.21.15.121:0/3235980579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11f40672e0 msgr2=0x7f11f40c14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:15.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f94e4700 1 --2- 172.21.15.121:0/3235980579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11f40672e0 0x7f11f40c14f0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7f11f40a6c40 tx=0x7f11e800b3b0).stop 2023-11-15T03:06:15.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f94e4700 1 -- 172.21.15.121:0/3235980579 shutdown_connections 2023-11-15T03:06:15.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f94e4700 1 --2- 172.21.15.121:0/3235980579 >> v2:172.21.15.190:3300/0 conn(0x7f11f40c3470 0x7f11f40c19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:15.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f94e4700 1 --2- 172.21.15.121:0/3235980579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11f40672e0 0x7f11f40c14f0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:15.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.169+0000 7f11f94e4700 1 -- 172.21.15.121:0/3235980579 >> 172.21.15.121:0/3235980579 conn(0x7f11f4067a60 msgr2=0x7f11f4067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:15.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.173+0000 7f11f94e4700 1 -- 172.21.15.121:0/3235980579 shutdown_connections 2023-11-15T03:06:15.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.173+0000 7f11f94e4700 1 -- 172.21.15.121:0/3235980579 wait complete. 2023-11-15T03:06:15.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.173+0000 7f11f94e4700 1 Processor -- start 2023-11-15T03:06:15.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.173+0000 7f11f94e4700 1 -- start start 2023-11-15T03:06:15.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.173+0000 7f11f94e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11f40672e0 0x7f11f414d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:15.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.173+0000 7f11f94e4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11f40c3470 0x7f11f41517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:15.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.173+0000 7f11f94e4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11f414fc60 0x7f11f4151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:15.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.173+0000 7f11f94e4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f41521a0 con 0x7f11f414fc60 2023-11-15T03:06:15.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.173+0000 7f11f94e4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f40ac580 con 0x7f11f40c3470 2023-11-15T03:06:15.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.173+0000 7f11f94e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f414d750 con 0x7f11f40672e0 2023-11-15T03:06:15.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.173+0000 7f11f8ce3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11f414fc60 0x7f11f4151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:15.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.173+0000 7f11f8ce3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11f414fc60 0x7f11f4151cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39652/0 (socket says 172.21.15.121:39652) 2023-11-15T03:06:15.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.173+0000 7f11f8ce3700 1 -- 172.21.15.121:0/4176643768 learned_addr learned my addr 172.21.15.121:0/4176643768 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:15.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.173+0000 7f11f37fe700 1 --2- 172.21.15.121:0/4176643768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11f40c3470 0x7f11f41517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:15.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.177+0000 7f11f3fff700 1 --2- 172.21.15.121:0/4176643768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11f40672e0 0x7f11f414d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:15.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.177+0000 7f11f8ce3700 1 -- 172.21.15.121:0/4176643768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11f40672e0 msgr2=0x7f11f414d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:15.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.177+0000 7f11f8ce3700 1 --2- 172.21.15.121:0/4176643768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11f40672e0 0x7f11f414d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:15.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.177+0000 7f11f8ce3700 1 -- 172.21.15.121:0/4176643768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11f40c3470 msgr2=0x7f11f41517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:15.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.177+0000 7f11f8ce3700 1 --2- 172.21.15.121:0/4176643768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11f40c3470 0x7f11f41517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:15.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.177+0000 7f11f8ce3700 1 -- 172.21.15.121:0/4176643768 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e800b610 con 0x7f11f414fc60 2023-11-15T03:06:15.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.177+0000 7f11f3fff700 1 --2- 172.21.15.121:0/4176643768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11f40672e0 0x7f11f414d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:15.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.177+0000 7f11f37fe700 1 --2- 172.21.15.121:0/4176643768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11f40c3470 0x7f11f41517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:15.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.177+0000 7f11f8ce3700 1 --2- 172.21.15.121:0/4176643768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11f414fc60 0x7f11f4151cb0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7f11ec00a380 tx=0x7f11ec007900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:15.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.177+0000 7f11dbfff700 1 -- 172.21.15.121:0/4176643768 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec00ae10 con 0x7f11f414fc60 2023-11-15T03:06:15.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.177+0000 7f11dbfff700 1 -- 172.21.15.121:0/4176643768 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ec00d650 con 0x7f11f414fc60 2023-11-15T03:06:15.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.177+0000 7f11dbfff700 1 -- 172.21.15.121:0/4176643768 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec00d9d0 con 0x7f11f414fc60 2023-11-15T03:06:15.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.177+0000 7f11f94e4700 1 -- 172.21.15.121:0/4176643768 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f40ab970 con 0x7f11f414fc60 2023-11-15T03:06:15.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.177+0000 7f11f94e4700 1 -- 172.21.15.121:0/4176643768 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f40abf30 con 0x7f11f414fc60 2023-11-15T03:06:15.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.177+0000 7f11f94e4700 1 -- 172.21.15.121:0/4176643768 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f4065ad0 con 0x7f11f414fc60 2023-11-15T03:06:15.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.181+0000 7f11dbfff700 1 -- 172.21.15.121:0/4176643768 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f11ec013070 con 0x7f11f414fc60 2023-11-15T03:06:15.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.181+0000 7f11dbfff700 1 --2- 172.21.15.121:0/4176643768 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f11e005a4e0 0x7f11e005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:15.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.181+0000 7f11dbfff700 1 -- 172.21.15.121:0/4176643768 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7556+0+0 (secure 0 0 0) 0x7f11ec075890 con 0x7f11f414fc60 2023-11-15T03:06:15.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.181+0000 7f11f3fff700 1 --2- 172.21.15.121:0/4176643768 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f11e005a4e0 0x7f11e005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:15.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.181+0000 7f11f3fff700 1 --2- 172.21.15.121:0/4176643768 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f11e005a4e0 0x7f11e005c9a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f11e400a820 tx=0x7f11e4008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:15.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.189+0000 7f11dbfff700 1 -- 172.21.15.121:0/4176643768 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f11ec048ad0 con 0x7f11f414fc60 2023-11-15T03:06:15.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:15.393+0000 7f11f94e4700 1 -- 172.21.15.121:0/4176643768 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f11f4068a70 con 0x7f11f414fc60 2023-11-15T03:06:16.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.349+0000 7f11dbfff700 1 -- 172.21.15.121:0/4176643768 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v180) v1 ==== 162+0+0 (secure 0 0 0) 0x7f11ec04c0f0 con 0x7f11f414fc60 2023-11-15T03:06:16.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.353+0000 7f11f94e4700 1 -- 172.21.15.121:0/4176643768 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f11e005a4e0 msgr2=0x7f11e005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:16.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.353+0000 7f11f94e4700 1 --2- 172.21.15.121:0/4176643768 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f11e005a4e0 0x7f11e005c9a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f11e400a820 tx=0x7f11e4008040).stop 2023-11-15T03:06:16.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.353+0000 7f11f94e4700 1 -- 172.21.15.121:0/4176643768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11f414fc60 msgr2=0x7f11f4151cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:16.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.353+0000 7f11f94e4700 1 --2- 172.21.15.121:0/4176643768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11f414fc60 0x7f11f4151cb0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7f11ec00a380 tx=0x7f11ec007900).stop 2023-11-15T03:06:16.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.357+0000 7f11f94e4700 1 -- 172.21.15.121:0/4176643768 shutdown_connections 2023-11-15T03:06:16.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.357+0000 7f11f94e4700 1 --2- 172.21.15.121:0/4176643768 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f11e005a4e0 0x7f11e005c9a0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:16.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.357+0000 7f11f94e4700 1 --2- 172.21.15.121:0/4176643768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11f414fc60 0x7f11f4151cb0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:16.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.357+0000 7f11f94e4700 1 --2- 172.21.15.121:0/4176643768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11f40c3470 0x7f11f41517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:16.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.357+0000 7f11f94e4700 1 --2- 172.21.15.121:0/4176643768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11f40672e0 0x7f11f414d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:16.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.357+0000 7f11f94e4700 1 -- 172.21.15.121:0/4176643768 >> 172.21.15.121:0/4176643768 conn(0x7f11f4067a60 msgr2=0x7f11f4062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:16.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.357+0000 7f11f94e4700 1 -- 172.21.15.121:0/4176643768 shutdown_connections 2023-11-15T03:06:16.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.357+0000 7f11f94e4700 1 -- 172.21.15.121:0/4176643768 wait complete. 2023-11-15T03:06:16.365 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:06:16.375 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:06:16.369+0000 7f341b097700 -1 mds.0.cache.ino(0x1) Corrupt inode 0x1: End of buffer 2023-11-15T03:06:16.375 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:06:16.369+0000 7f341b097700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2023-11-15T03:06:16.384 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:16.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.537+0000 7fe39189b700 1 Processor -- start 2023-11-15T03:06:16.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.537+0000 7fe39189b700 1 -- start start 2023-11-15T03:06:16.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe39189b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe38c151d00 0x7fe38c0c4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:16.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe39189b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe38c068430 con 0x7fe38c0c4c10 2023-11-15T03:06:16.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe39189b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe38c0c50a0 con 0x7fe38c151920 2023-11-15T03:06:16.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe39189b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c064b50 con 0x7fe38c151d00 2023-11-15T03:06:16.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe38bfff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe38c0c4c10 0x7fe38c0a4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60536/0 (socket says 172.21.15.121:60536) 2023-11-15T03:06:16.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe38bfff700 1 -- 172.21.15.121:0/645179317 learned_addr learned my addr 172.21.15.121:0/645179317 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:16.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe38affd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe38c151920 0x7fe38c063fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:53260/0 (socket says 172.21.15.121:53260) 2023-11-15T03:06:16.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe38a7fc700 1 --2- 172.21.15.121:0/645179317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe38c151d00 0x7fe38c0c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:16.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe389ffb700 1 -- 172.21.15.121:0/645179317 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112854644 0 0) 0x7fe38c068430 con 0x7fe38c0c4c10 2023-11-15T03:06:16.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe389ffb700 1 -- 172.21.15.121:0/645179317 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3740033f0 con 0x7fe38c0c4c10 2023-11-15T03:06:16.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe389ffb700 1 -- 172.21.15.121:0/645179317 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656395007 0 0) 0x7fe38c0c50a0 con 0x7fe38c151920 2023-11-15T03:06:16.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe389ffb700 1 -- 172.21.15.121:0/645179317 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe38c068430 con 0x7fe38c151920 2023-11-15T03:06:16.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe38a7fc700 1 -- 172.21.15.121:0/645179317 >> v1:172.21.15.190:6789/0 conn(0x7fe38c151920 legacy=0x7fe38c063fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:16.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe38a7fc700 1 -- 172.21.15.121:0/645179317 >> v1:172.21.15.121:6789/0 conn(0x7fe38c0c4c10 legacy=0x7fe38c0a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:16.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe38a7fc700 1 -- 172.21.15.121:0/645179317 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38c0bec30 con 0x7fe38c151d00 2023-11-15T03:06:16.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe38a7fc700 1 --2- 172.21.15.121:0/645179317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe38c151d00 0x7fe38c0c4720 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fe380009790 tx=0x7fe380009060).ready entity=mon.2 client_cookie=729007d4d1a60054 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:16.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe389ffb700 1 -- 172.21.15.121:0/645179317 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38000f510 con 0x7fe38c151d00 2023-11-15T03:06:16.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe389ffb700 1 -- 172.21.15.121:0/645179317 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38000f670 con 0x7fe38c151d00 2023-11-15T03:06:16.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe389ffb700 1 -- 172.21.15.121:0/645179317 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38000f9f0 con 0x7fe38c151d00 2023-11-15T03:06:16.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe39189b700 1 -- 172.21.15.121:0/645179317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe38c151d00 msgr2=0x7fe38c0c4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:16.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe39189b700 1 --2- 172.21.15.121:0/645179317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe38c151d00 0x7fe38c0c4720 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fe380009790 tx=0x7fe380009060).stop 2023-11-15T03:06:16.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe39189b700 1 -- 172.21.15.121:0/645179317 shutdown_connections 2023-11-15T03:06:16.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe39189b700 1 --2- 172.21.15.121:0/645179317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe38c151d00 0x7fe38c0c4720 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:16.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe39189b700 1 -- 172.21.15.121:0/645179317 >> 172.21.15.121:0/645179317 conn(0x7fe38c069ac0 msgr2=0x7fe38c090a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:16.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe39189b700 1 -- 172.21.15.121:0/645179317 shutdown_connections 2023-11-15T03:06:16.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.541+0000 7fe39189b700 1 -- 172.21.15.121:0/645179317 wait complete. 2023-11-15T03:06:16.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe39189b700 1 Processor -- start 2023-11-15T03:06:16.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe39189b700 1 -- start start 2023-11-15T03:06:16.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe39189b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe38c0c4c10 0x7fe38c156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:16.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe39189b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe38c151920 0x7fe38c14ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:16.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe39189b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe38c151d00 0x7fe38c0a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:16.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe39189b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c0a76b0 con 0x7fe38c151920 2023-11-15T03:06:16.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe39189b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c0c7680 con 0x7fe38c151d00 2023-11-15T03:06:16.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe39189b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c0c6a70 con 0x7fe38c0c4c10 2023-11-15T03:06:16.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe38a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe38c151920 0x7fe38c14ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:16.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe38a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe38c151920 0x7fe38c14ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39668/0 (socket says 172.21.15.121:39668) 2023-11-15T03:06:16.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe38a7fc700 1 -- 172.21.15.121:0/2766014156 learned_addr learned my addr 172.21.15.121:0/2766014156 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:16.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe38bfff700 1 --2- 172.21.15.121:0/2766014156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe38c151d00 0x7fe38c0a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:16.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe38affd700 1 --2- 172.21.15.121:0/2766014156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe38c0c4c10 0x7fe38c156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:16.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe38a7fc700 1 -- 172.21.15.121:0/2766014156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe38c0c4c10 msgr2=0x7fe38c156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:16.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe38a7fc700 1 --2- 172.21.15.121:0/2766014156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe38c0c4c10 0x7fe38c156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:16.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe38a7fc700 1 -- 172.21.15.121:0/2766014156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe38c151d00 msgr2=0x7fe38c0a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:16.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe38a7fc700 1 --2- 172.21.15.121:0/2766014156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe38c151d00 0x7fe38c0a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:16.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe38a7fc700 1 -- 172.21.15.121:0/2766014156 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3800092a0 con 0x7fe38c151920 2023-11-15T03:06:16.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe38bfff700 1 --2- 172.21.15.121:0/2766014156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe38c151d00 0x7fe38c0a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:16.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe38affd700 1 --2- 172.21.15.121:0/2766014156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe38c0c4c10 0x7fe38c156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:16.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe38a7fc700 1 --2- 172.21.15.121:0/2766014156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe38c151920 0x7fe38c14ebf0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7fe38c059bd0 tx=0x7fe380017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:16.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe372ffd700 1 -- 172.21.15.121:0/2766014156 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38000f7e0 con 0x7fe38c151920 2023-11-15T03:06:16.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe372ffd700 1 -- 172.21.15.121:0/2766014156 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38000f940 con 0x7fe38c151920 2023-11-15T03:06:16.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe39189b700 1 -- 172.21.15.121:0/2766014156 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38c0c6d50 con 0x7fe38c151920 2023-11-15T03:06:16.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe372ffd700 1 -- 172.21.15.121:0/2766014156 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38001b660 con 0x7fe38c151920 2023-11-15T03:06:16.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.545+0000 7fe39189b700 1 -- 172.21.15.121:0/2766014156 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38c0c7310 con 0x7fe38c151920 2023-11-15T03:06:16.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.549+0000 7fe39189b700 1 -- 172.21.15.121:0/2766014156 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe38c047060 con 0x7fe38c151920 2023-11-15T03:06:16.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.549+0000 7fe372ffd700 1 -- 172.21.15.121:0/2766014156 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe38001b9d0 con 0x7fe38c151920 2023-11-15T03:06:16.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.549+0000 7fe372ffd700 1 --2- 172.21.15.121:0/2766014156 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe37805a510 0x7fe37805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:16.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.549+0000 7fe372ffd700 1 -- 172.21.15.121:0/2766014156 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7642+0+0 (secure 0 0 0) 0x7fe3800766e0 con 0x7fe38c151920 2023-11-15T03:06:16.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.549+0000 7fe38affd700 1 --2- 172.21.15.121:0/2766014156 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe37805a510 0x7fe37805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:16.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.553+0000 7fe38affd700 1 --2- 172.21.15.121:0/2766014156 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe37805a510 0x7fe37805c9d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fe37c001010 tx=0x7fe37c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:16.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.557+0000 7fe372ffd700 1 -- 172.21.15.121:0/2766014156 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe3800498d0 con 0x7fe38c151920 2023-11-15T03:06:16.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.753+0000 7fe39189b700 1 -- 172.21.15.121:0/2766014156 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe38c06a100 con 0x7fe38c151920 2023-11-15T03:06:16.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.757+0000 7fe372ffd700 1 -- 172.21.15.121:0/2766014156 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) v1 ==== 95+0+4502 (secure 0 0 0) 0x7fe380029090 con 0x7fe38c151920 2023-11-15T03:06:16.762 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:16.763 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":181,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:16.353037+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8568},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"a","rank":0,"incarnation":181,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/564044995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":564044995},{"type":"v1","addr":"172.21.15.121:6836","nonce":564044995}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:16.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.757+0000 7fe39189b700 1 -- 172.21.15.121:0/2766014156 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe37805a510 msgr2=0x7fe37805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:16.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.757+0000 7fe39189b700 1 --2- 172.21.15.121:0/2766014156 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe37805a510 0x7fe37805c9d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fe37c001010 tx=0x7fe37c009040).stop 2023-11-15T03:06:16.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.757+0000 7fe39189b700 1 -- 172.21.15.121:0/2766014156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe38c151920 msgr2=0x7fe38c14ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:16.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.757+0000 7fe39189b700 1 --2- 172.21.15.121:0/2766014156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe38c151920 0x7fe38c14ebf0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7fe38c059bd0 tx=0x7fe380017000).stop 2023-11-15T03:06:16.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.757+0000 7fe39189b700 1 -- 172.21.15.121:0/2766014156 shutdown_connections 2023-11-15T03:06:16.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.757+0000 7fe39189b700 1 --2- 172.21.15.121:0/2766014156 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe37805a510 0x7fe37805c9d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:16.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.757+0000 7fe39189b700 1 --2- 172.21.15.121:0/2766014156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe38c151d00 0x7fe38c0a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:16.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.757+0000 7fe39189b700 1 --2- 172.21.15.121:0/2766014156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe38c151920 0x7fe38c14ebf0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:16.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.757+0000 7fe39189b700 1 --2- 172.21.15.121:0/2766014156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe38c0c4c10 0x7fe38c156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:16.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.757+0000 7fe39189b700 1 -- 172.21.15.121:0/2766014156 >> 172.21.15.121:0/2766014156 conn(0x7fe38c069ac0 msgr2=0x7fe38c0ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:16.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.757+0000 7fe39189b700 1 -- 172.21.15.121:0/2766014156 shutdown_connections 2023-11-15T03:06:16.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:16.757+0000 7fe39189b700 1 -- 172.21.15.121:0/2766014156 wait complete. 2023-11-15T03:06:16.768 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 181 2023-11-15T03:06:16.783 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:06:17.367 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:06:17.361+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:06:17.368 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:06:17.361+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:06:17.377 INFO:tasks.ceph.mds.a.smithi121.stderr: -18> 2023-11-15T03:06:16.369+0000 7f341b097700 -1 mds.0.cache.ino(0x1) Corrupt inode 0x1: End of buffer 2023-11-15T03:06:17.377 INFO:tasks.ceph.mds.a.smithi121.stderr: -16> 2023-11-15T03:06:16.369+0000 7f341b097700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2023-11-15T03:06:17.413 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:06:21.785 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:21.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.945+0000 7fcfe4546700 1 Processor -- start 2023-11-15T03:06:21.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.945+0000 7fcfe4546700 1 -- start start 2023-11-15T03:06:21.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.949+0000 7fcfe4546700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcfdc1562f0 0x7fcfdc155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:21.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.949+0000 7fcfe4546700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfdc0c4c30 con 0x7fcfdc0a57c0 2023-11-15T03:06:21.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.949+0000 7fcfe4546700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfdc0a6f50 con 0x7fcfdc154ca0 2023-11-15T03:06:21.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.949+0000 7fcfe4546700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfdc0a8280 con 0x7fcfdc1562f0 2023-11-15T03:06:21.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.949+0000 7fcfe1ae1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcfdc1562f0 0x7fcfdc155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:21.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.949+0000 7fcfe2ae3700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fcfdc0a57c0 0x7fcfdc0c50c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60554/0 (socket says 172.21.15.121:60554) 2023-11-15T03:06:21.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.949+0000 7fcfe2ae3700 1 -- 172.21.15.121:0/415153608 learned_addr learned my addr 172.21.15.121:0/415153608 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:21.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.949+0000 7fcfe22e2700 1 --1- 172.21.15.121:0/415153608 >> v1:172.21.15.190:6789/0 conn(0x7fcfdc154ca0 0x7fcfdc0a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:53266/0 (socket says 172.21.15.121:53266) 2023-11-15T03:06:21.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.953+0000 7fcfe1ae1700 1 -- 172.21.15.121:0/415153608 >> v1:172.21.15.190:6789/0 conn(0x7fcfdc154ca0 legacy=0x7fcfdc0a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:21.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.953+0000 7fcfe12e0700 1 -- 172.21.15.121:0/415153608 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 11897056 0 0) 0x7fcfdc0c4c30 con 0x7fcfdc0a57c0 2023-11-15T03:06:21.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.953+0000 7fcfe1ae1700 1 -- 172.21.15.121:0/415153608 >> v1:172.21.15.121:6789/0 conn(0x7fcfdc0a57c0 legacy=0x7fcfdc0c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:21.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.953+0000 7fcfe1ae1700 1 -- 172.21.15.121:0/415153608 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc0a7230 con 0x7fcfdc1562f0 2023-11-15T03:06:21.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.953+0000 7fcfe1ae1700 1 --2- 172.21.15.121:0/415153608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfdc1562f0 0x7fcfdc155080 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7fcfcc0096e0 tx=0x7fcfcc009060).ready entity=mon.0 client_cookie=f2a426eaa3610a4f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:21.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.953+0000 7fcfe12e0700 1 -- 172.21.15.121:0/415153608 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00f510 con 0x7fcfdc1562f0 2023-11-15T03:06:21.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.953+0000 7fcfe12e0700 1 -- 172.21.15.121:0/415153608 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc00f670 con 0x7fcfdc1562f0 2023-11-15T03:06:21.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.953+0000 7fcfe12e0700 1 -- 172.21.15.121:0/415153608 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00f9f0 con 0x7fcfdc1562f0 2023-11-15T03:06:21.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.953+0000 7fcfe4546700 1 -- 172.21.15.121:0/415153608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfdc1562f0 msgr2=0x7fcfdc155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:21.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.953+0000 7fcfe4546700 1 --2- 172.21.15.121:0/415153608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfdc1562f0 0x7fcfdc155080 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7fcfcc0096e0 tx=0x7fcfcc009060).stop 2023-11-15T03:06:21.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.953+0000 7fcfe4546700 1 -- 172.21.15.121:0/415153608 shutdown_connections 2023-11-15T03:06:21.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.953+0000 7fcfe4546700 1 --2- 172.21.15.121:0/415153608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfdc1562f0 0x7fcfdc155080 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:21.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.953+0000 7fcfe4546700 1 -- 172.21.15.121:0/415153608 >> 172.21.15.121:0/415153608 conn(0x7fcfdc062190 msgr2=0x7fcfdc0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:21.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.953+0000 7fcfe4546700 1 -- 172.21.15.121:0/415153608 shutdown_connections 2023-11-15T03:06:21.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.953+0000 7fcfe4546700 1 -- 172.21.15.121:0/415153608 wait complete. 2023-11-15T03:06:21.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.957+0000 7fcfe4546700 1 Processor -- start 2023-11-15T03:06:21.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.957+0000 7fcfe4546700 1 -- start start 2023-11-15T03:06:21.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.957+0000 7fcfe4546700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfdc0a57c0 0x7fcfdc14c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:21.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.957+0000 7fcfe4546700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfdc154ca0 0x7fcfdc1468a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:21.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfe4546700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfdc1562f0 0x7fcfdc1452b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:21.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfe4546700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc146d90 con 0x7fcfdc0a57c0 2023-11-15T03:06:21.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfe4546700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc0c1690 con 0x7fcfdc1562f0 2023-11-15T03:06:21.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfe4546700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc146fa0 con 0x7fcfdc154ca0 2023-11-15T03:06:21.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfe22e2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfdc0a57c0 0x7fcfdc14c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:21.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfe22e2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfdc0a57c0 0x7fcfdc14c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39712/0 (socket says 172.21.15.121:39712) 2023-11-15T03:06:21.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfe22e2700 1 -- 172.21.15.121:0/2003221341 learned_addr learned my addr 172.21.15.121:0/2003221341 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:21.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfe1ae1700 1 --2- 172.21.15.121:0/2003221341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfdc154ca0 0x7fcfdc1468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:21.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfe2ae3700 1 --2- 172.21.15.121:0/2003221341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfdc1562f0 0x7fcfdc1452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:21.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfe1ae1700 1 -- 172.21.15.121:0/2003221341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfdc1562f0 msgr2=0x7fcfdc1452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:21.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfe1ae1700 1 --2- 172.21.15.121:0/2003221341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfdc1562f0 0x7fcfdc1452b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:21.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfe1ae1700 1 -- 172.21.15.121:0/2003221341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfdc0a57c0 msgr2=0x7fcfdc14c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:21.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfe1ae1700 1 --2- 172.21.15.121:0/2003221341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfdc0a57c0 0x7fcfdc14c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:21.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfe1ae1700 1 -- 172.21.15.121:0/2003221341 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc0092a0 con 0x7fcfdc154ca0 2023-11-15T03:06:21.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfe1ae1700 1 --2- 172.21.15.121:0/2003221341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfdc154ca0 0x7fcfdc1468a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fcfcc014000 tx=0x7fcfcc017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:21.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfd27fc700 1 -- 172.21.15.121:0/2003221341 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00f7e0 con 0x7fcfdc154ca0 2023-11-15T03:06:21.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfd27fc700 1 -- 172.21.15.121:0/2003221341 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc00f940 con 0x7fcfdc154ca0 2023-11-15T03:06:21.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfe4546700 1 -- 172.21.15.121:0/2003221341 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfdc0c0d60 con 0x7fcfdc154ca0 2023-11-15T03:06:21.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfd27fc700 1 -- 172.21.15.121:0/2003221341 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc01b640 con 0x7fcfdc154ca0 2023-11-15T03:06:21.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.961+0000 7fcfe4546700 1 -- 172.21.15.121:0/2003221341 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfdc0c1320 con 0x7fcfdc154ca0 2023-11-15T03:06:21.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.965+0000 7fcfe4546700 1 -- 172.21.15.121:0/2003221341 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfdc0a9080 con 0x7fcfdc154ca0 2023-11-15T03:06:21.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.965+0000 7fcfd27fc700 1 -- 172.21.15.121:0/2003221341 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fcfcc01b9b0 con 0x7fcfdc154ca0 2023-11-15T03:06:21.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.965+0000 7fcfd27fc700 1 --2- 172.21.15.121:0/2003221341 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcfc005a510 0x7fcfc005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:21.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.965+0000 7fcfd27fc700 1 -- 172.21.15.121:0/2003221341 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7642+0+0 (secure 0 0 0) 0x7fcfcc076430 con 0x7fcfdc154ca0 2023-11-15T03:06:21.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.965+0000 7fcfe22e2700 1 --2- 172.21.15.121:0/2003221341 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcfc005a510 0x7fcfc005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:21.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.965+0000 7fcfe22e2700 1 --2- 172.21.15.121:0/2003221341 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcfc005a510 0x7fcfc005c9d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7fcfd8000c00 tx=0x7fcfd8007d70).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:21.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:21.969+0000 7fcfd27fc700 1 -- 172.21.15.121:0/2003221341 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcfcc049560 con 0x7fcfdc154ca0 2023-11-15T03:06:22.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.169+0000 7fcfe4546700 1 -- 172.21.15.121:0/2003221341 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfdc06c150 con 0x7fcfdc154ca0 2023-11-15T03:06:22.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.169+0000 7fcfd27fc700 1 -- 172.21.15.121:0/2003221341 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 183 v183) v1 ==== 95+0+4496 (secure 0 0 0) 0x7fcfcc04cb80 con 0x7fcfdc154ca0 2023-11-15T03:06:22.180 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:22.180 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":183,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2127410277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2127410277},{"type":"v1","addr":"172.21.15.121:6836","nonce":2127410277}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":183}],"filesystems":[{"mdsmap":{"epoch":182,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:17.363737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:22.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.173+0000 7fcfe4546700 1 -- 172.21.15.121:0/2003221341 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcfc005a510 msgr2=0x7fcfc005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:22.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.173+0000 7fcfe4546700 1 --2- 172.21.15.121:0/2003221341 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcfc005a510 0x7fcfc005c9d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7fcfd8000c00 tx=0x7fcfd8007d70).stop 2023-11-15T03:06:22.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.173+0000 7fcfe4546700 1 -- 172.21.15.121:0/2003221341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfdc154ca0 msgr2=0x7fcfdc1468a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:22.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.173+0000 7fcfe4546700 1 --2- 172.21.15.121:0/2003221341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfdc154ca0 0x7fcfdc1468a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fcfcc014000 tx=0x7fcfcc017000).stop 2023-11-15T03:06:22.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.173+0000 7fcfe4546700 1 -- 172.21.15.121:0/2003221341 shutdown_connections 2023-11-15T03:06:22.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.173+0000 7fcfe4546700 1 --2- 172.21.15.121:0/2003221341 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcfc005a510 0x7fcfc005c9d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.173+0000 7fcfe4546700 1 --2- 172.21.15.121:0/2003221341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfdc1562f0 0x7fcfdc1452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.173+0000 7fcfe4546700 1 --2- 172.21.15.121:0/2003221341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfdc154ca0 0x7fcfdc1468a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.173+0000 7fcfe4546700 1 --2- 172.21.15.121:0/2003221341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfdc0a57c0 0x7fcfdc14c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.173+0000 7fcfe4546700 1 -- 172.21.15.121:0/2003221341 >> 172.21.15.121:0/2003221341 conn(0x7fcfdc062190 msgr2=0x7fcfdc0c52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:22.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.173+0000 7fcfe4546700 1 -- 172.21.15.121:0/2003221341 shutdown_connections 2023-11-15T03:06:22.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.173+0000 7fcfe4546700 1 -- 172.21.15.121:0/2003221341 wait complete. 2023-11-15T03:06:22.184 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 183 2023-11-15T03:06:22.201 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:06:22.201 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:22.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.349+0000 7ff14e5f9700 1 Processor -- start 2023-11-15T03:06:22.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.349+0000 7ff14e5f9700 1 -- start start 2023-11-15T03:06:22.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14e5f9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff148066d60 0x7ff148067140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:22.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14e5f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff148066370 0x7ff1481577e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:22.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14e5f9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff148157cd0 0x7ff1481535c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:22.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14e5f9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1480c00f0 con 0x7ff148066d60 2023-11-15T03:06:22.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14e5f9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff148158160 con 0x7ff148157cd0 2023-11-15T03:06:22.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14e5f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff148066790 con 0x7ff148066370 2023-11-15T03:06:22.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14d5f7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff148066d60 0x7ff148067140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:22.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14d5f7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff148066d60 0x7ff148067140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39724/0 (socket says 172.21.15.121:39724) 2023-11-15T03:06:22.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14d5f7700 1 -- 172.21.15.121:0/284681143 learned_addr learned my addr 172.21.15.121:0/284681143 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:22.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14ddf8700 1 --2- 172.21.15.121:0/284681143 >> v2:172.21.15.190:3300/0 conn(0x7ff148157cd0 0x7ff1481535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:22.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14cdf6700 1 --2- 172.21.15.121:0/284681143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff148066370 0x7ff1481577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:22.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14d5f7700 1 -- 172.21.15.121:0/284681143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff148066370 msgr2=0x7ff1481577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:22.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14d5f7700 1 --2- 172.21.15.121:0/284681143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff148066370 0x7ff1481577e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14d5f7700 1 -- 172.21.15.121:0/284681143 >> v2:172.21.15.190:3300/0 conn(0x7ff148157cd0 msgr2=0x7ff1481535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:22.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14d5f7700 1 --2- 172.21.15.121:0/284681143 >> v2:172.21.15.190:3300/0 conn(0x7ff148157cd0 0x7ff1481535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14d5f7700 1 -- 172.21.15.121:0/284681143 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1481583f0 con 0x7ff148066d60 2023-11-15T03:06:22.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14cdf6700 1 --2- 172.21.15.121:0/284681143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff148066370 0x7ff1481577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:22.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14ddf8700 1 --2- 172.21.15.121:0/284681143 >> v2:172.21.15.190:3300/0 conn(0x7ff148157cd0 0x7ff1481535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:22.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14d5f7700 1 --2- 172.21.15.121:0/284681143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff148066d60 0x7ff148067140 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7ff1480c0fd0 tx=0x7ff1440090f0).ready entity=mon.0 client_cookie=7b5e44506636b78f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:22.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff13ffff700 1 -- 172.21.15.121:0/284681143 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14400b600 con 0x7ff148066d60 2023-11-15T03:06:22.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff13ffff700 1 -- 172.21.15.121:0/284681143 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14400b760 con 0x7ff148066d60 2023-11-15T03:06:22.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff13ffff700 1 -- 172.21.15.121:0/284681143 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14400bae0 con 0x7ff148066d60 2023-11-15T03:06:22.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14e5f9700 1 -- 172.21.15.121:0/284681143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff148066d60 msgr2=0x7ff148067140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:22.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14e5f9700 1 --2- 172.21.15.121:0/284681143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff148066d60 0x7ff148067140 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7ff1480c0fd0 tx=0x7ff1440090f0).stop 2023-11-15T03:06:22.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14e5f9700 1 -- 172.21.15.121:0/284681143 shutdown_connections 2023-11-15T03:06:22.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14e5f9700 1 --2- 172.21.15.121:0/284681143 >> v2:172.21.15.190:3300/0 conn(0x7ff148157cd0 0x7ff1481535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14e5f9700 1 --2- 172.21.15.121:0/284681143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff148066370 0x7ff1481577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14e5f9700 1 --2- 172.21.15.121:0/284681143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff148066d60 0x7ff148067140 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.353+0000 7ff14e5f9700 1 -- 172.21.15.121:0/284681143 >> 172.21.15.121:0/284681143 conn(0x7ff148062190 msgr2=0x7ff148062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:22.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.357+0000 7ff14e5f9700 1 -- 172.21.15.121:0/284681143 shutdown_connections 2023-11-15T03:06:22.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.357+0000 7ff14e5f9700 1 -- 172.21.15.121:0/284681143 wait complete. 2023-11-15T03:06:22.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.357+0000 7ff14e5f9700 1 Processor -- start 2023-11-15T03:06:22.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.357+0000 7ff14e5f9700 1 -- start start 2023-11-15T03:06:22.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff14e5f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff148066370 0x7ff14813e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:22.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff14e5f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff148066d60 0x7ff14814f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:22.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff14e5f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff148157cd0 0x7ff1481558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:22.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff14e5f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff148155450 con 0x7ff148066d60 2023-11-15T03:06:22.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff14e5f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff148154840 con 0x7ff148157cd0 2023-11-15T03:06:22.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff14e5f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff148154b20 con 0x7ff148066370 2023-11-15T03:06:22.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff14cdf6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff148066d60 0x7ff14814f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:22.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff14cdf6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff148066d60 0x7ff14814f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39728/0 (socket says 172.21.15.121:39728) 2023-11-15T03:06:22.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff14cdf6700 1 -- 172.21.15.121:0/4201543476 learned_addr learned my addr 172.21.15.121:0/4201543476 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:22.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff14d5f7700 1 --2- 172.21.15.121:0/4201543476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff148066370 0x7ff14813e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:22.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff14ddf8700 1 --2- 172.21.15.121:0/4201543476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff148157cd0 0x7ff1481558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:22.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff14cdf6700 1 -- 172.21.15.121:0/4201543476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff148066370 msgr2=0x7ff14813e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:22.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff14cdf6700 1 --2- 172.21.15.121:0/4201543476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff148066370 0x7ff14813e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff14cdf6700 1 -- 172.21.15.121:0/4201543476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff148157cd0 msgr2=0x7ff1481558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:22.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff14cdf6700 1 --2- 172.21.15.121:0/4201543476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff148157cd0 0x7ff1481558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff14cdf6700 1 -- 172.21.15.121:0/4201543476 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff144009370 con 0x7ff148066d60 2023-11-15T03:06:22.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff14cdf6700 1 --2- 172.21.15.121:0/4201543476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff148066d60 0x7ff14814f2c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7ff13800c880 tx=0x7ff138008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:22.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff13d7fa700 1 -- 172.21.15.121:0/4201543476 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138010580 con 0x7ff148066d60 2023-11-15T03:06:22.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff13d7fa700 1 -- 172.21.15.121:0/4201543476 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1380106e0 con 0x7ff148066d60 2023-11-15T03:06:22.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff14e5f9700 1 -- 172.21.15.121:0/4201543476 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff148154e00 con 0x7ff148066d60 2023-11-15T03:06:22.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff13d7fa700 1 -- 172.21.15.121:0/4201543476 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138010a80 con 0x7ff148066d60 2023-11-15T03:06:22.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.361+0000 7ff14e5f9700 1 -- 172.21.15.121:0/4201543476 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff14814f7b0 con 0x7ff148066d60 2023-11-15T03:06:22.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.365+0000 7ff14e5f9700 1 -- 172.21.15.121:0/4201543476 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff148155280 con 0x7ff148066d60 2023-11-15T03:06:22.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.365+0000 7ff13d7fa700 1 -- 172.21.15.121:0/4201543476 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff13801b460 con 0x7ff148066d60 2023-11-15T03:06:22.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.369+0000 7ff13d7fa700 1 --2- 172.21.15.121:0/4201543476 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff12c05a510 0x7ff12c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:22.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.369+0000 7ff13d7fa700 1 -- 172.21.15.121:0/4201543476 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7642+0+0 (secure 0 0 0) 0x7ff138074da0 con 0x7ff148066d60 2023-11-15T03:06:22.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.369+0000 7ff14d5f7700 1 --2- 172.21.15.121:0/4201543476 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff12c05a510 0x7ff12c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:22.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.369+0000 7ff14d5f7700 1 --2- 172.21.15.121:0/4201543476 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff12c05a510 0x7ff12c05c9d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7ff144009130 tx=0x7ff144017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:22.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.373+0000 7ff13d7fa700 1 -- 172.21.15.121:0/4201543476 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff138047e70 con 0x7ff148066d60 2023-11-15T03:06:22.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.573+0000 7ff14e5f9700 1 -- 172.21.15.121:0/4201543476 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1481586d0 con 0x7ff148066d60 2023-11-15T03:06:22.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.573+0000 7ff13d7fa700 1 -- 172.21.15.121:0/4201543476 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 183 v183) v1 ==== 95+0+4496 (secure 0 0 0) 0x7ff13804b490 con 0x7ff148066d60 2023-11-15T03:06:22.580 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:22.580 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":183,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2127410277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2127410277},{"type":"v1","addr":"172.21.15.121:6836","nonce":2127410277}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":183}],"filesystems":[{"mdsmap":{"epoch":182,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:17.363737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:22.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.577+0000 7ff14e5f9700 1 -- 172.21.15.121:0/4201543476 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff12c05a510 msgr2=0x7ff12c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:22.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.577+0000 7ff14e5f9700 1 --2- 172.21.15.121:0/4201543476 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff12c05a510 0x7ff12c05c9d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7ff144009130 tx=0x7ff144017000).stop 2023-11-15T03:06:22.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.577+0000 7ff14e5f9700 1 -- 172.21.15.121:0/4201543476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff148066d60 msgr2=0x7ff14814f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:22.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.577+0000 7ff14e5f9700 1 --2- 172.21.15.121:0/4201543476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff148066d60 0x7ff14814f2c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7ff13800c880 tx=0x7ff138008040).stop 2023-11-15T03:06:22.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.577+0000 7ff14e5f9700 1 -- 172.21.15.121:0/4201543476 shutdown_connections 2023-11-15T03:06:22.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.577+0000 7ff14e5f9700 1 --2- 172.21.15.121:0/4201543476 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff12c05a510 0x7ff12c05c9d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.577+0000 7ff14e5f9700 1 --2- 172.21.15.121:0/4201543476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff148157cd0 0x7ff1481558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.577+0000 7ff14e5f9700 1 --2- 172.21.15.121:0/4201543476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff148066d60 0x7ff14814f2c0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.577+0000 7ff14e5f9700 1 --2- 172.21.15.121:0/4201543476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff148066370 0x7ff14813e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.577+0000 7ff14e5f9700 1 -- 172.21.15.121:0/4201543476 >> 172.21.15.121:0/4201543476 conn(0x7ff148062190 msgr2=0x7ff1480ac170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:22.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.577+0000 7ff14e5f9700 1 -- 172.21.15.121:0/4201543476 shutdown_connections 2023-11-15T03:06:22.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.577+0000 7ff14e5f9700 1 -- 172.21.15.121:0/4201543476 wait complete. 2023-11-15T03:06:22.588 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 183 2023-11-15T03:06:22.603 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:06:22.604 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:22.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.765+0000 7f8d1c81b700 1 Processor -- start 2023-11-15T03:06:22.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.765+0000 7f8d1c81b700 1 -- start start 2023-11-15T03:06:22.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.765+0000 7f8d1c81b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d18151830 0x7f8d180a87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:22.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.765+0000 7f8d1c81b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8d180ab260 0x7f8d18155640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:22.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.765+0000 7f8d1c81b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8d18151c10 0x7f8d18155b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:22.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.765+0000 7f8d1c81b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d18156020 con 0x7f8d180ab260 2023-11-15T03:06:22.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.765+0000 7f8d1c81b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d18065d00 con 0x7f8d18151c10 2023-11-15T03:06:22.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.765+0000 7f8d1c81b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d180c6b90 con 0x7f8d18151830 2023-11-15T03:06:22.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.765+0000 7f8d0ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8d180ab260 0x7f8d18155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:22.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.765+0000 7f8d0ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8d180ab260 0x7f8d18155640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39740/0 (socket says 172.21.15.121:39740) 2023-11-15T03:06:22.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.765+0000 7f8d0ffff700 1 -- 172.21.15.121:0/2837207680 learned_addr learned my addr 172.21.15.121:0/2837207680 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:22.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.765+0000 7f8d16d9d700 1 --2- 172.21.15.121:0/2837207680 >> v2:172.21.15.190:3300/0 conn(0x7f8d18151c10 0x7f8d18155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:22.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.765+0000 7f8d1659c700 1 --2- 172.21.15.121:0/2837207680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d18151830 0x7f8d180a87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:22.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.765+0000 7f8d0ffff700 1 -- 172.21.15.121:0/2837207680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d18151830 msgr2=0x7f8d180a87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:22.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.765+0000 7f8d0ffff700 1 --2- 172.21.15.121:0/2837207680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d18151830 0x7f8d180a87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.765+0000 7f8d0ffff700 1 -- 172.21.15.121:0/2837207680 >> v2:172.21.15.190:3300/0 conn(0x7f8d18151c10 msgr2=0x7f8d18155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:22.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.765+0000 7f8d0ffff700 1 --2- 172.21.15.121:0/2837207680 >> v2:172.21.15.190:3300/0 conn(0x7f8d18151c10 0x7f8d18155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.765+0000 7f8d0ffff700 1 -- 172.21.15.121:0/2837207680 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d180c6e30 con 0x7f8d180ab260 2023-11-15T03:06:22.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.769+0000 7f8d0ffff700 1 --2- 172.21.15.121:0/2837207680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d180ab260 0x7f8d18155640 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7f8d180ab030 tx=0x7f8d000090f0).ready entity=mon.0 client_cookie=b4182da66e5c2b8d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:22.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.769+0000 7f8d15d9b700 1 -- 172.21.15.121:0/2837207680 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0000b520 con 0x7f8d180ab260 2023-11-15T03:06:22.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.769+0000 7f8d15d9b700 1 -- 172.21.15.121:0/2837207680 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0000b680 con 0x7f8d180ab260 2023-11-15T03:06:22.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.769+0000 7f8d15d9b700 1 -- 172.21.15.121:0/2837207680 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0000ba00 con 0x7f8d180ab260 2023-11-15T03:06:22.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.769+0000 7f8d1c81b700 1 -- 172.21.15.121:0/2837207680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d180ab260 msgr2=0x7f8d18155640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:22.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.769+0000 7f8d1c81b700 1 --2- 172.21.15.121:0/2837207680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d180ab260 0x7f8d18155640 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7f8d180ab030 tx=0x7f8d000090f0).stop 2023-11-15T03:06:22.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.769+0000 7f8d1c81b700 1 -- 172.21.15.121:0/2837207680 shutdown_connections 2023-11-15T03:06:22.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.769+0000 7f8d1c81b700 1 --2- 172.21.15.121:0/2837207680 >> v2:172.21.15.190:3300/0 conn(0x7f8d18151c10 0x7f8d18155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.769+0000 7f8d1c81b700 1 --2- 172.21.15.121:0/2837207680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d180ab260 0x7f8d18155640 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.769+0000 7f8d1c81b700 1 --2- 172.21.15.121:0/2837207680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d18151830 0x7f8d180a87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.769+0000 7f8d1c81b700 1 -- 172.21.15.121:0/2837207680 >> 172.21.15.121:0/2837207680 conn(0x7f8d180c8960 msgr2=0x7f8d180c8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:22.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.769+0000 7f8d1c81b700 1 -- 172.21.15.121:0/2837207680 shutdown_connections 2023-11-15T03:06:22.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.769+0000 7f8d1c81b700 1 -- 172.21.15.121:0/2837207680 wait complete. 2023-11-15T03:06:22.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d1c81b700 1 Processor -- start 2023-11-15T03:06:22.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d1c81b700 1 -- start start 2023-11-15T03:06:22.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d1c81b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d180ab260 0x7f8d1814b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:22.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d1c81b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d18151830 0x7f8d1814abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:22.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d1c81b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d18151c10 0x7f8d180c0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:22.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d1c81b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1814ba40 con 0x7f8d18151830 2023-11-15T03:06:22.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d1c81b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d18168e90 con 0x7f8d18151c10 2023-11-15T03:06:22.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d1c81b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d18168280 con 0x7f8d180ab260 2023-11-15T03:06:22.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d0ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d18151830 0x7f8d1814abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:22.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d0ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d18151830 0x7f8d1814abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39754/0 (socket says 172.21.15.121:39754) 2023-11-15T03:06:22.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d0ffff700 1 -- 172.21.15.121:0/3536885818 learned_addr learned my addr 172.21.15.121:0/3536885818 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:22.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d1659c700 1 --2- 172.21.15.121:0/3536885818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d180ab260 0x7f8d1814b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:22.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d16d9d700 1 --2- 172.21.15.121:0/3536885818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d18151c10 0x7f8d180c0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:22.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d0ffff700 1 -- 172.21.15.121:0/3536885818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d180ab260 msgr2=0x7f8d1814b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:22.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d0ffff700 1 --2- 172.21.15.121:0/3536885818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d180ab260 0x7f8d1814b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d0ffff700 1 -- 172.21.15.121:0/3536885818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d18151c10 msgr2=0x7f8d180c0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:22.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d0ffff700 1 --2- 172.21.15.121:0/3536885818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d18151c10 0x7f8d180c0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d0ffff700 1 -- 172.21.15.121:0/3536885818 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d00009370 con 0x7f8d18151830 2023-11-15T03:06:22.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d0ffff700 1 --2- 172.21.15.121:0/3536885818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d18151830 0x7f8d1814abd0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f8d00011000 tx=0x7f8d00017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:22.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d0e7fc700 1 -- 172.21.15.121:0/3536885818 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0000b7f0 con 0x7f8d18151830 2023-11-15T03:06:22.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d0e7fc700 1 -- 172.21.15.121:0/3536885818 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0000b950 con 0x7f8d18151830 2023-11-15T03:06:22.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d0e7fc700 1 -- 172.21.15.121:0/3536885818 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0001b660 con 0x7f8d18151830 2023-11-15T03:06:22.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d1c81b700 1 -- 172.21.15.121:0/3536885818 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d18168560 con 0x7f8d18151830 2023-11-15T03:06:22.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.773+0000 7f8d1c81b700 1 -- 172.21.15.121:0/3536885818 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d18168b20 con 0x7f8d18151830 2023-11-15T03:06:22.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.777+0000 7f8d1c81b700 1 -- 172.21.15.121:0/3536885818 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d18169020 con 0x7f8d18151830 2023-11-15T03:06:22.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.777+0000 7f8d0e7fc700 1 -- 172.21.15.121:0/3536885818 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8d0001ba10 con 0x7f8d18151830 2023-11-15T03:06:22.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.781+0000 7f8d0e7fc700 1 --2- 172.21.15.121:0/3536885818 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8cf805a510 0x7f8cf805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:22.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.781+0000 7f8d0e7fc700 1 -- 172.21.15.121:0/3536885818 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7642+0+0 (secure 0 0 0) 0x7f8d000766a0 con 0x7f8d18151830 2023-11-15T03:06:22.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.781+0000 7f8d1659c700 1 --2- 172.21.15.121:0/3536885818 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8cf805a510 0x7f8cf805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:22.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.781+0000 7f8d1659c700 1 --2- 172.21.15.121:0/3536885818 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8cf805a510 0x7f8cf805c9d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f8d08006fd0 tx=0x7f8d08008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:22.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.785+0000 7f8d0e7fc700 1 -- 172.21.15.121:0/3536885818 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8d00049890 con 0x7f8d18151830 2023-11-15T03:06:22.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.985+0000 7f8d1c81b700 1 -- 172.21.15.121:0/3536885818 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d1806c150 con 0x7f8d18151830 2023-11-15T03:06:22.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.985+0000 7f8d0e7fc700 1 -- 172.21.15.121:0/3536885818 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 183 v183) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f8d00029020 con 0x7f8d18151830 2023-11-15T03:06:22.994 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:22.994 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":183,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2127410277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2127410277},{"type":"v1","addr":"172.21.15.121:6836","nonce":2127410277}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":183}],"filesystems":[{"mdsmap":{"epoch":182,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:17.363737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:22.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.989+0000 7f8d1c81b700 1 -- 172.21.15.121:0/3536885818 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8cf805a510 msgr2=0x7f8cf805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:22.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.989+0000 7f8d1c81b700 1 --2- 172.21.15.121:0/3536885818 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8cf805a510 0x7f8cf805c9d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f8d08006fd0 tx=0x7f8d08008040).stop 2023-11-15T03:06:22.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.989+0000 7f8d1c81b700 1 -- 172.21.15.121:0/3536885818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d18151830 msgr2=0x7f8d1814abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:22.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.989+0000 7f8d1c81b700 1 --2- 172.21.15.121:0/3536885818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d18151830 0x7f8d1814abd0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f8d00011000 tx=0x7f8d00017000).stop 2023-11-15T03:06:22.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.989+0000 7f8d1c81b700 1 -- 172.21.15.121:0/3536885818 shutdown_connections 2023-11-15T03:06:22.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.989+0000 7f8d1c81b700 1 --2- 172.21.15.121:0/3536885818 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8cf805a510 0x7f8cf805c9d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.989+0000 7f8d1c81b700 1 --2- 172.21.15.121:0/3536885818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d18151c10 0x7f8d180c0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.989+0000 7f8d1c81b700 1 --2- 172.21.15.121:0/3536885818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d18151830 0x7f8d1814abd0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.989+0000 7f8d1c81b700 1 --2- 172.21.15.121:0/3536885818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d180ab260 0x7f8d1814b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:22.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.989+0000 7f8d1c81b700 1 -- 172.21.15.121:0/3536885818 >> 172.21.15.121:0/3536885818 conn(0x7f8d180c8960 msgr2=0x7f8d180c8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:22.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.989+0000 7f8d1c81b700 1 -- 172.21.15.121:0/3536885818 shutdown_connections 2023-11-15T03:06:23.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:22.993+0000 7f8d1c81b700 1 -- 172.21.15.121:0/3536885818 wait complete. 2023-11-15T03:06:23.000 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 183 2023-11-15T03:06:23.020 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 1.00000000.inode' led to DAMAGED state 2023-11-15T03:06:23.020 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 200.00000000' 2023-11-15T03:06:23.021 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:06:23.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:06:23.021 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:06:23.040 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:06:23.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:06:23.042 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:06:23.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.249+0000 7f5e21639700 1 Processor -- start 2023-11-15T03:06:23.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.253+0000 7f5e21639700 1 -- start start 2023-11-15T03:06:23.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.253+0000 7f5e21639700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e1c0669b0 0x7f5e1c158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:23.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.253+0000 7f5e21639700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e1c0bb910 con 0x7f5e1c0a8750 2023-11-15T03:06:23.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.253+0000 7f5e21639700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e1c15bde0 con 0x7f5e1c157d90 2023-11-15T03:06:23.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.253+0000 7f5e21639700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c0c1fb0 con 0x7f5e1c0669b0 2023-11-15T03:06:23.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.253+0000 7f5e20e38700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f5e1c0a8750 0x7f5e1c065680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60566/0 (socket says 172.21.15.121:60566) 2023-11-15T03:06:23.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.253+0000 7f5e20e38700 1 -- 172.21.15.121:0/155765141 learned_addr learned my addr 172.21.15.121:0/155765141 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:23.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.253+0000 7f5e13fff700 1 --2- 172.21.15.121:0/155765141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e1c0669b0 0x7f5e1c158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:23.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.253+0000 7f5e1b7fe700 1 -- 172.21.15.121:0/155765141 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1886835565 0 0) 0x7f5e1c0bb910 con 0x7f5e1c0a8750 2023-11-15T03:06:23.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.257+0000 7f5e1b7fe700 1 -- 172.21.15.121:0/155765141 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e080033f0 con 0x7f5e1c0a8750 2023-11-15T03:06:23.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.257+0000 7f5e1b7fe700 1 -- 172.21.15.121:0/155765141 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2129234658 0 0) 0x7f5e1c15bde0 con 0x7f5e1c157d90 2023-11-15T03:06:23.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.257+0000 7f5e1b7fe700 1 -- 172.21.15.121:0/155765141 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e1c0bb910 con 0x7f5e1c157d90 2023-11-15T03:06:23.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.257+0000 7f5e13fff700 1 -- 172.21.15.121:0/155765141 >> v1:172.21.15.190:6789/0 conn(0x7f5e1c157d90 legacy=0x7f5e1c0c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:23.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.257+0000 7f5e13fff700 1 -- 172.21.15.121:0/155765141 >> v1:172.21.15.121:6789/0 conn(0x7f5e1c0a8750 legacy=0x7f5e1c065680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:23.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.257+0000 7f5e13fff700 1 -- 172.21.15.121:0/155765141 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1c15c0c0 con 0x7f5e1c0669b0 2023-11-15T03:06:23.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.257+0000 7f5e13fff700 1 --2- 172.21.15.121:0/155765141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e1c0669b0 0x7f5e1c158170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f5e04009760 tx=0x7f5e04006f40).ready entity=mon.2 client_cookie=d9468f6a69cc63a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:23.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.257+0000 7f5e1b7fe700 1 -- 172.21.15.121:0/155765141 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e04005880 con 0x7f5e1c0669b0 2023-11-15T03:06:23.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.257+0000 7f5e1b7fe700 1 -- 172.21.15.121:0/155765141 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e040059e0 con 0x7f5e1c0669b0 2023-11-15T03:06:23.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.257+0000 7f5e1b7fe700 1 -- 172.21.15.121:0/155765141 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e04005d60 con 0x7f5e1c0669b0 2023-11-15T03:06:23.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.257+0000 7f5e21639700 1 -- 172.21.15.121:0/155765141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e1c0669b0 msgr2=0x7f5e1c158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:23.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.257+0000 7f5e21639700 1 --2- 172.21.15.121:0/155765141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e1c0669b0 0x7f5e1c158170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f5e04009760 tx=0x7f5e04006f40).stop 2023-11-15T03:06:23.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.257+0000 7f5e21639700 1 -- 172.21.15.121:0/155765141 shutdown_connections 2023-11-15T03:06:23.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.257+0000 7f5e21639700 1 --2- 172.21.15.121:0/155765141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e1c0669b0 0x7f5e1c158170 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:23.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.257+0000 7f5e21639700 1 -- 172.21.15.121:0/155765141 >> 172.21.15.121:0/155765141 conn(0x7f5e1c067a60 msgr2=0x7f5e1c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:23.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.257+0000 7f5e21639700 1 -- 172.21.15.121:0/155765141 shutdown_connections 2023-11-15T03:06:23.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.257+0000 7f5e21639700 1 -- 172.21.15.121:0/155765141 wait complete. 2023-11-15T03:06:23.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.261+0000 7f5e21639700 1 Processor -- start 2023-11-15T03:06:23.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.261+0000 7f5e21639700 1 -- start start 2023-11-15T03:06:23.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.261+0000 7f5e21639700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e1c0669b0 0x7f5e1c159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:23.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.261+0000 7f5e21639700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e1c0a8750 0x7f5e1c0ac030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:23.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.261+0000 7f5e21639700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e1c157d90 0x7f5e1c0bc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:23.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.261+0000 7f5e21639700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c159140 con 0x7f5e1c0a8750 2023-11-15T03:06:23.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.261+0000 7f5e21639700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c0a9f10 con 0x7f5e1c157d90 2023-11-15T03:06:23.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.261+0000 7f5e21639700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c159a20 con 0x7f5e1c0669b0 2023-11-15T03:06:23.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.261+0000 7f5e13fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e1c0a8750 0x7f5e1c0ac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:23.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.265+0000 7f5e13fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e1c0a8750 0x7f5e1c0ac030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39766/0 (socket says 172.21.15.121:39766) 2023-11-15T03:06:23.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.265+0000 7f5e13fff700 1 -- 172.21.15.121:0/2138907802 learned_addr learned my addr 172.21.15.121:0/2138907802 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:23.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.265+0000 7f5e1bfff700 1 --2- 172.21.15.121:0/2138907802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e1c0669b0 0x7f5e1c159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:23.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.265+0000 7f5e20e38700 1 --2- 172.21.15.121:0/2138907802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e1c157d90 0x7f5e1c0bc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:23.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.265+0000 7f5e13fff700 1 -- 172.21.15.121:0/2138907802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e1c0669b0 msgr2=0x7f5e1c159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:23.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.265+0000 7f5e13fff700 1 --2- 172.21.15.121:0/2138907802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e1c0669b0 0x7f5e1c159530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:23.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.265+0000 7f5e13fff700 1 -- 172.21.15.121:0/2138907802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e1c157d90 msgr2=0x7f5e1c0bc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:23.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.265+0000 7f5e13fff700 1 --2- 172.21.15.121:0/2138907802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e1c157d90 0x7f5e1c0bc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:23.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.265+0000 7f5e13fff700 1 -- 172.21.15.121:0/2138907802 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e04009250 con 0x7f5e1c0a8750 2023-11-15T03:06:23.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.265+0000 7f5e20e38700 1 --2- 172.21.15.121:0/2138907802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e1c157d90 0x7f5e1c0bc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:23.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.265+0000 7f5e1bfff700 1 --2- 172.21.15.121:0/2138907802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e1c0669b0 0x7f5e1c159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:23.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.265+0000 7f5e13fff700 1 --2- 172.21.15.121:0/2138907802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e1c0a8750 0x7f5e1c0ac030 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7f5e1c064eb0 tx=0x7f5e04015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:23.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.265+0000 7f5e18ff9700 1 -- 172.21.15.121:0/2138907802 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0400d6b0 con 0x7f5e1c0a8750 2023-11-15T03:06:23.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.265+0000 7f5e18ff9700 1 -- 172.21.15.121:0/2138907802 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e0400d810 con 0x7f5e1c0a8750 2023-11-15T03:06:23.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.265+0000 7f5e21639700 1 -- 172.21.15.121:0/2138907802 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1c158530 con 0x7f5e1c0a8750 2023-11-15T03:06:23.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.265+0000 7f5e18ff9700 1 -- 172.21.15.121:0/2138907802 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0400db90 con 0x7f5e1c0a8750 2023-11-15T03:06:23.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.265+0000 7f5e21639700 1 -- 172.21.15.121:0/2138907802 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e1c158af0 con 0x7f5e1c0a8750 2023-11-15T03:06:23.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.265+0000 7f5e21639700 1 -- 172.21.15.121:0/2138907802 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e1c0a9180 con 0x7f5e1c0a8750 2023-11-15T03:06:23.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.269+0000 7f5e18ff9700 1 -- 172.21.15.121:0/2138907802 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5e0401d450 con 0x7f5e1c0a8750 2023-11-15T03:06:23.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.269+0000 7f5e18ff9700 1 --2- 172.21.15.121:0/2138907802 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5dfc05a4e0 0x7f5dfc05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:23.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.269+0000 7f5e18ff9700 1 -- 172.21.15.121:0/2138907802 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7642+0+0 (secure 0 0 0) 0x7f5e040761e0 con 0x7f5e1c0a8750 2023-11-15T03:06:23.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.269+0000 7f5e1bfff700 1 --2- 172.21.15.121:0/2138907802 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5dfc05a4e0 0x7f5dfc05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:23.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.269+0000 7f5e1bfff700 1 --2- 172.21.15.121:0/2138907802 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5dfc05a4e0 0x7f5dfc05c9a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f5e0c007fd0 tx=0x7f5e0c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:23.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.277+0000 7f5e18ff9700 1 -- 172.21.15.121:0/2138907802 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5e1c0a9180 con 0x7f5e1c0a8750 2023-11-15T03:06:23.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:23.473+0000 7f5e21639700 1 -- 172.21.15.121:0/2138907802 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5e1c0a9180 con 0x7f5e1c0a8750 2023-11-15T03:06:24.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.245+0000 7f5e18ff9700 1 -- 172.21.15.121:0/2138907802 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v184) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5e1c0a9180 con 0x7f5e1c0a8750 2023-11-15T03:06:24.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.249+0000 7f5e21639700 1 -- 172.21.15.121:0/2138907802 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5dfc05a4e0 msgr2=0x7f5dfc05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:24.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.249+0000 7f5e21639700 1 --2- 172.21.15.121:0/2138907802 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5dfc05a4e0 0x7f5dfc05c9a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f5e0c007fd0 tx=0x7f5e0c009040).stop 2023-11-15T03:06:24.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.249+0000 7f5e21639700 1 -- 172.21.15.121:0/2138907802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e1c0a8750 msgr2=0x7f5e1c0ac030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:24.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.249+0000 7f5e21639700 1 --2- 172.21.15.121:0/2138907802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e1c0a8750 0x7f5e1c0ac030 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7f5e1c064eb0 tx=0x7f5e04015000).stop 2023-11-15T03:06:24.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.249+0000 7f5e21639700 1 -- 172.21.15.121:0/2138907802 shutdown_connections 2023-11-15T03:06:24.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.249+0000 7f5e21639700 1 --2- 172.21.15.121:0/2138907802 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5dfc05a4e0 0x7f5dfc05c9a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:24.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.249+0000 7f5e21639700 1 --2- 172.21.15.121:0/2138907802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e1c157d90 0x7f5e1c0bc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:24.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.249+0000 7f5e21639700 1 --2- 172.21.15.121:0/2138907802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e1c0a8750 0x7f5e1c0ac030 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:24.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.249+0000 7f5e21639700 1 --2- 172.21.15.121:0/2138907802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e1c0669b0 0x7f5e1c159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:24.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.249+0000 7f5e21639700 1 -- 172.21.15.121:0/2138907802 >> 172.21.15.121:0/2138907802 conn(0x7f5e1c067a60 msgr2=0x7f5e1c0c5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:24.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.249+0000 7f5e21639700 1 -- 172.21.15.121:0/2138907802 shutdown_connections 2023-11-15T03:06:24.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.249+0000 7f5e21639700 1 -- 172.21.15.121:0/2138907802 wait complete. 2023-11-15T03:06:24.258 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:06:24.279 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:06:24.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.441+0000 7fce6ffff700 1 Processor -- start 2023-11-15T03:06:24.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.441+0000 7fce6ffff700 1 -- start start 2023-11-15T03:06:24.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.445+0000 7fce6ffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fce700bb510 0x7fce700abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:24.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.445+0000 7fce6ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fce700a8880 0x7fce70151400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:24.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.445+0000 7fce6ffff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce700c2c00 con 0x7fce700abaa0 2023-11-15T03:06:24.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.445+0000 7fce6ffff700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce700657b0 con 0x7fce700a8880 2023-11-15T03:06:24.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.445+0000 7fce6ffff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce70065a90 con 0x7fce700bb510 2023-11-15T03:06:24.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.445+0000 7fce6f7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fce700a8880 0x7fce70151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:24.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.445+0000 7fce67fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fce700bb510 0x7fce700abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:24.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.445+0000 7fce6f7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fce700a8880 0x7fce70151400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39770/0 (socket says 172.21.15.121:39770) 2023-11-15T03:06:24.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.445+0000 7fce6f7fe700 1 -- 172.21.15.121:0/3020957386 learned_addr learned my addr 172.21.15.121:0/3020957386 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:24.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.445+0000 7fce6effd700 1 --1- 172.21.15.121:0/3020957386 >> v1:172.21.15.190:6789/0 conn(0x7fce700abaa0 0x7fce70063570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:53284/0 (socket says 172.21.15.121:53284) 2023-11-15T03:06:24.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.445+0000 7fce67fff700 1 -- 172.21.15.121:0/3020957386 >> v2:172.21.15.121:3300/0 conn(0x7fce700a8880 msgr2=0x7fce70151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:24.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.445+0000 7fce67fff700 1 --2- 172.21.15.121:0/3020957386 >> v2:172.21.15.121:3300/0 conn(0x7fce700a8880 0x7fce70151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:24.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.445+0000 7fce67fff700 1 -- 172.21.15.121:0/3020957386 >> v1:172.21.15.190:6789/0 conn(0x7fce700abaa0 legacy=0x7fce70063570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:24.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.445+0000 7fce67fff700 1 -- 172.21.15.121:0/3020957386 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce700c2890 con 0x7fce700bb510 2023-11-15T03:06:24.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.445+0000 7fce67fff700 1 --2- 172.21.15.121:0/3020957386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce700bb510 0x7fce700abe80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fce701554d0 tx=0x7fce58006f40).ready entity=mon.1 client_cookie=3bad9fe39902a46c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:24.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.449+0000 7fce6e7fc700 1 -- 172.21.15.121:0/3020957386 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58005880 con 0x7fce700bb510 2023-11-15T03:06:24.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.449+0000 7fce6e7fc700 1 -- 172.21.15.121:0/3020957386 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce580059e0 con 0x7fce700bb510 2023-11-15T03:06:24.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.449+0000 7fce6e7fc700 1 -- 172.21.15.121:0/3020957386 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58005d60 con 0x7fce700bb510 2023-11-15T03:06:24.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.449+0000 7fce6ffff700 1 -- 172.21.15.121:0/3020957386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce700bb510 msgr2=0x7fce700abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:24.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.449+0000 7fce6ffff700 1 --2- 172.21.15.121:0/3020957386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce700bb510 0x7fce700abe80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fce701554d0 tx=0x7fce58006f40).stop 2023-11-15T03:06:24.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.449+0000 7fce6ffff700 1 -- 172.21.15.121:0/3020957386 shutdown_connections 2023-11-15T03:06:24.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.449+0000 7fce6ffff700 1 --2- 172.21.15.121:0/3020957386 >> v2:172.21.15.121:3300/0 conn(0x7fce700a8880 0x7fce70151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:24.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.449+0000 7fce6ffff700 1 --2- 172.21.15.121:0/3020957386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce700bb510 0x7fce700abe80 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:24.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.449+0000 7fce6ffff700 1 -- 172.21.15.121:0/3020957386 >> 172.21.15.121:0/3020957386 conn(0x7fce70067a60 msgr2=0x7fce70067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:24.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.449+0000 7fce6ffff700 1 -- 172.21.15.121:0/3020957386 shutdown_connections 2023-11-15T03:06:24.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.449+0000 7fce6ffff700 1 -- 172.21.15.121:0/3020957386 wait complete. 2023-11-15T03:06:24.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce6ffff700 1 Processor -- start 2023-11-15T03:06:24.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce6ffff700 1 -- start start 2023-11-15T03:06:24.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce6ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce700a8880 0x7fce70140df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:24.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce6ffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce700abaa0 0x7fce70065fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:24.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce6ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce700bb510 0x7fce700664c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:24.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce6ffff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce700669b0 con 0x7fce700bb510 2023-11-15T03:06:24.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce6ffff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce700c2e50 con 0x7fce700a8880 2023-11-15T03:06:24.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce6ffff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce700c3130 con 0x7fce700abaa0 2023-11-15T03:06:24.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce6f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce700bb510 0x7fce700664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:24.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce6f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce700bb510 0x7fce700664c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39772/0 (socket says 172.21.15.121:39772) 2023-11-15T03:06:24.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce6f7fe700 1 -- 172.21.15.121:0/1448127287 learned_addr learned my addr 172.21.15.121:0/1448127287 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:24.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce6effd700 1 --2- 172.21.15.121:0/1448127287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce700a8880 0x7fce70140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:24.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce67fff700 1 --2- 172.21.15.121:0/1448127287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce700abaa0 0x7fce70065fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:24.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce6f7fe700 1 -- 172.21.15.121:0/1448127287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce700abaa0 msgr2=0x7fce70065fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:24.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce6f7fe700 1 --2- 172.21.15.121:0/1448127287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce700abaa0 0x7fce70065fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:24.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce6f7fe700 1 -- 172.21.15.121:0/1448127287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce700a8880 msgr2=0x7fce70140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:24.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce6f7fe700 1 --2- 172.21.15.121:0/1448127287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce700a8880 0x7fce70140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:24.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce6f7fe700 1 -- 172.21.15.121:0/1448127287 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce58009250 con 0x7fce700bb510 2023-11-15T03:06:24.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce67fff700 1 --2- 172.21.15.121:0/1448127287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce700abaa0 0x7fce70065fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:24.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce6f7fe700 1 --2- 172.21.15.121:0/1448127287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce700bb510 0x7fce700664c0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7fce68009db0 tx=0x7fce6800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:24.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce66ffd700 1 -- 172.21.15.121:0/1448127287 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6800c940 con 0x7fce700bb510 2023-11-15T03:06:24.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce66ffd700 1 -- 172.21.15.121:0/1448127287 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6800f040 con 0x7fce700bb510 2023-11-15T03:06:24.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.453+0000 7fce6ffff700 1 -- 172.21.15.121:0/1448127287 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce700c3410 con 0x7fce700bb510 2023-11-15T03:06:24.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.457+0000 7fce66ffd700 1 -- 172.21.15.121:0/1448127287 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6800ccd0 con 0x7fce700bb510 2023-11-15T03:06:24.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.457+0000 7fce6ffff700 1 -- 172.21.15.121:0/1448127287 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce700c39d0 con 0x7fce700bb510 2023-11-15T03:06:24.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.457+0000 7fce6ffff700 1 -- 172.21.15.121:0/1448127287 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7006bcf0 con 0x7fce700bb510 2023-11-15T03:06:24.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.457+0000 7fce66ffd700 1 -- 172.21.15.121:0/1448127287 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fce68014620 con 0x7fce700bb510 2023-11-15T03:06:24.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.461+0000 7fce66ffd700 1 --2- 172.21.15.121:0/1448127287 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fce5005a510 0x7fce5005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:24.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.461+0000 7fce66ffd700 1 -- 172.21.15.121:0/1448127287 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7642+0+0 (secure 0 0 0) 0x7fce68074ef0 con 0x7fce700bb510 2023-11-15T03:06:24.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.461+0000 7fce6effd700 1 --2- 172.21.15.121:0/1448127287 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fce5005a510 0x7fce5005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:24.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.461+0000 7fce6effd700 1 --2- 172.21.15.121:0/1448127287 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fce5005a510 0x7fce5005c9d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7fce60007fd0 tx=0x7fce60007d30).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:24.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.465+0000 7fce66ffd700 1 -- 172.21.15.121:0/1448127287 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fce68048020 con 0x7fce700bb510 2023-11-15T03:06:24.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:24.681+0000 7fce6ffff700 1 -- 172.21.15.121:0/1448127287 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fce700c2540 con 0x7fce700bb510 2023-11-15T03:06:25.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.249+0000 7fce66ffd700 1 -- 172.21.15.121:0/1448127287 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v185) v1 ==== 101+0+0 (secure 0 0 0) 0x7fce6804b640 con 0x7fce700bb510 2023-11-15T03:06:25.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.253+0000 7fce6ffff700 1 -- 172.21.15.121:0/1448127287 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fce5005a510 msgr2=0x7fce5005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.253+0000 7fce6ffff700 1 --2- 172.21.15.121:0/1448127287 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fce5005a510 0x7fce5005c9d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7fce60007fd0 tx=0x7fce60007d30).stop 2023-11-15T03:06:25.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.253+0000 7fce6ffff700 1 -- 172.21.15.121:0/1448127287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce700bb510 msgr2=0x7fce700664c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.253+0000 7fce6ffff700 1 --2- 172.21.15.121:0/1448127287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce700bb510 0x7fce700664c0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7fce68009db0 tx=0x7fce6800b040).stop 2023-11-15T03:06:25.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.253+0000 7fce6ffff700 1 -- 172.21.15.121:0/1448127287 shutdown_connections 2023-11-15T03:06:25.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.253+0000 7fce6ffff700 1 --2- 172.21.15.121:0/1448127287 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fce5005a510 0x7fce5005c9d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.253+0000 7fce6ffff700 1 --2- 172.21.15.121:0/1448127287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce700bb510 0x7fce700664c0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.253+0000 7fce6ffff700 1 --2- 172.21.15.121:0/1448127287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce700abaa0 0x7fce70065fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.253+0000 7fce6ffff700 1 --2- 172.21.15.121:0/1448127287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce700a8880 0x7fce70140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.253+0000 7fce6ffff700 1 -- 172.21.15.121:0/1448127287 >> 172.21.15.121:0/1448127287 conn(0x7fce70067a60 msgr2=0x7fce7014ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:25.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.253+0000 7fce6ffff700 1 -- 172.21.15.121:0/1448127287 shutdown_connections 2023-11-15T03:06:25.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.253+0000 7fce6ffff700 1 -- 172.21.15.121:0/1448127287 wait complete. 2023-11-15T03:06:25.262 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:06:25.279 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:06:25.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.329+0000 7f9b160d8d00 1 Processor -- start 2023-11-15T03:06:25.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.329+0000 7f9b160d8d00 1 -- start start 2023-11-15T03:06:25.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b160d8d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ebd7f26de0 0x55ebd7f293c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b160d8d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ebd7f58420 0x55ebd8009ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b160d8d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebd7ffc9f0 0x55ebd800a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b160d8d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd7e9f220 con 0x55ebd7f58420 2023-11-15T03:06:25.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b160d8d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd7ff92d0 con 0x55ebd7ffc9f0 2023-11-15T03:06:25.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b160d8d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ebd7ffc1c0 con 0x55ebd7f26de0 2023-11-15T03:06:25.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b0b12b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ebd7f58420 0x55ebd8009ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b0b12b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ebd7f58420 0x55ebd8009ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55020/0 (socket says 172.21.15.121:55020) 2023-11-15T03:06:25.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b0b12b700 1 -- 172.21.15.121:0/3324396731 learned_addr learned my addr 172.21.15.121:0/3324396731 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:25.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b1412d700 1 --2- 172.21.15.121:0/3324396731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebd7ffc9f0 0x55ebd800a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b1392c700 1 --2- 172.21.15.121:0/3324396731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ebd7f26de0 0x55ebd7f293c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b0b12b700 1 -- 172.21.15.121:0/3324396731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ebd7f26de0 msgr2=0x55ebd7f293c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:25.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b0b12b700 1 --2- 172.21.15.121:0/3324396731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ebd7f26de0 0x55ebd7f293c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b0b12b700 1 -- 172.21.15.121:0/3324396731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebd7ffc9f0 msgr2=0x55ebd800a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:25.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b0b12b700 1 --2- 172.21.15.121:0/3324396731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebd7ffc9f0 0x55ebd800a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b0b12b700 1 -- 172.21.15.121:0/3324396731 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc005db0 con 0x55ebd7f58420 2023-11-15T03:06:25.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b0b12b700 1 --2- 172.21.15.121:0/3324396731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ebd7f58420 0x55ebd8009ef0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7f9afc00afd0 tx=0x7f9afc016000).ready entity=mon.0 client_cookie=767757ddd077c1ef server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:25.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b1292a700 1 -- 172.21.15.121:0/3324396731 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc0055c0 con 0x55ebd7f58420 2023-11-15T03:06:25.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b1292a700 1 -- 172.21.15.121:0/3324396731 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9afc000bc0 con 0x55ebd7f58420 2023-11-15T03:06:25.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b1292a700 1 -- 172.21.15.121:0/3324396731 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc0089d0 con 0x55ebd7f58420 2023-11-15T03:06:25.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b160d8d00 1 -- 172.21.15.121:0/3324396731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ebd7f58420 msgr2=0x55ebd8009ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/3324396731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ebd7f58420 0x55ebd8009ef0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7f9afc00afd0 tx=0x7f9afc016000).stop 2023-11-15T03:06:25.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b160d8d00 1 -- 172.21.15.121:0/3324396731 shutdown_connections 2023-11-15T03:06:25.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/3324396731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebd7ffc9f0 0x55ebd800a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/3324396731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ebd7f58420 0x55ebd8009ef0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/3324396731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ebd7f26de0 0x55ebd7f293c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b160d8d00 1 -- 172.21.15.121:0/3324396731 >> 172.21.15.121:0/3324396731 conn(0x55ebd7f277d0 msgr2=0x55ebd7ffce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:25.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b160d8d00 1 -- 172.21.15.121:0/3324396731 shutdown_connections 2023-11-15T03:06:25.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.333+0000 7f9b160d8d00 1 -- 172.21.15.121:0/3324396731 wait complete. 2023-11-15T03:06:25.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b160d8d00 1 Processor -- start 2023-11-15T03:06:25.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b160d8d00 1 -- start start 2023-11-15T03:06:25.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b160d8d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ebd7f26de0 0x55ebd7ffd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b160d8d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ebd7f58420 0x55ebd7ff4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b160d8d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebd7ffc9f0 0x55ebd7ffda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b160d8d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd7f694a0 con 0x55ebd7f58420 2023-11-15T03:06:25.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b160d8d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd80049e0 con 0x55ebd7ffc9f0 2023-11-15T03:06:25.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b160d8d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ebd800dc80 con 0x55ebd7f26de0 2023-11-15T03:06:25.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b0b12b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ebd7f58420 0x55ebd7ff4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b0b12b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ebd7f58420 0x55ebd7ff4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55036/0 (socket says 172.21.15.121:55036) 2023-11-15T03:06:25.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 learned_addr learned my addr 172.21.15.121:0/1545569035 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:25.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b1412d700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebd7ffc9f0 0x55ebd7ffda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b1392c700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ebd7f26de0 0x55ebd7ffd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ebd7f26de0 msgr2=0x55ebd7ffd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b0b12b700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ebd7f26de0 0x55ebd7ffd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebd7ffc9f0 msgr2=0x55ebd7ffda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b0b12b700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebd7ffc9f0 0x55ebd7ffda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebd7ec8a20 con 0x55ebd7f58420 2023-11-15T03:06:25.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b1412d700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ebd7ffc9f0 0x55ebd7ffda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:25.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.337+0000 7f9b0b12b700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ebd7f58420 0x55ebd7ff4ff0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7f9afc00afd0 tx=0x7f9afc01c000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:25.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.341+0000 7f9b11127700 1 -- 172.21.15.121:0/1545569035 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc0089d0 con 0x55ebd7f58420 2023-11-15T03:06:25.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.341+0000 7f9b11127700 1 -- 172.21.15.121:0/1545569035 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9afc013c60 con 0x55ebd7f58420 2023-11-15T03:06:25.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.341+0000 7f9b11127700 1 -- 172.21.15.121:0/1545569035 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc030470 con 0x55ebd7f58420 2023-11-15T03:06:25.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.341+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ebd7ec8770 con 0x55ebd7f58420 2023-11-15T03:06:25.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.341+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebd7e97a70 con 0x55ebd7f58420 2023-11-15T03:06:25.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.341+0000 7f9b11127700 1 -- 172.21.15.121:0/1545569035 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9afc02d070 con 0x55ebd7f58420 2023-11-15T03:06:25.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.341+0000 7f9b11127700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9af4058ff0 0x7f9af405b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.345+0000 7f9b11127700 1 -- 172.21.15.121:0/1545569035 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7642+0+0 (secure 0 0 0) 0x7f9afc077020 con 0x55ebd7f58420 2023-11-15T03:06:25.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.345+0000 7f9b1392c700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9af4058ff0 0x7f9af405b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.356 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:06:25.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.345+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55ebd800f120 0x55ebd8003060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.345+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8003550 con 0x55ebd800f120 2023-11-15T03:06:25.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.345+0000 7f9b1412d700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55ebd800f120 0x55ebd8003060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.345+0000 7f9b1392c700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9af4058ff0 0x7f9af405b4b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f9b04003c70 tx=0x7f9b04000bc0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:25.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.345+0000 7f9b1412d700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55ebd800f120 0x55ebd8003060 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:25.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.349+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v110'185 uv178 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd800f120 2023-11-15T03:06:25.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.349+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd8003930 con 0x55ebd800f120 2023-11-15T03:06:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.349+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v110'186 uv186 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd800f120 2023-11-15T03:06:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.349+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8003ce0 con 0x55ebd800f120 2023-11-15T03:06:25.359 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:06:25.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.353+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v110'187 uv187 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd800f120 2023-11-15T03:06:25.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.353+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd801b090 con 0x55ebd800f120 2023-11-15T03:06:25.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.357+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v110'188 uv188 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd800f120 2023-11-15T03:06:25.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.357+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd801b440 con 0x55ebd800f120 2023-11-15T03:06:25.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.357+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v110'189 uv189 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd800f120 2023-11-15T03:06:25.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.357+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd801a080 con 0x55ebd800f120 2023-11-15T03:06:25.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.361+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v110'190 uv190 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd800f120 2023-11-15T03:06:25.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.361+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd801a4b0 con 0x55ebd800f120 2023-11-15T03:06:25.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.361+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v110'191 uv184 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd800f120 2023-11-15T03:06:25.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.361+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd801a8b0 con 0x55ebd800f120 2023-11-15T03:06:25.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.365+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v110'192 uv192 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd800f120 2023-11-15T03:06:25.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.365+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd801ac60 con 0x55ebd800f120 2023-11-15T03:06:25.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.369+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v110'193 uv193 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd800f120 2023-11-15T03:06:25.375 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:06:25.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.369+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd801ac60 con 0x55ebd800f120 2023-11-15T03:06:25.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.369+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v110'194 uv194 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd800f120 2023-11-15T03:06:25.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.369+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd801ac60 con 0x55ebd800f120 2023-11-15T03:06:25.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.377+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v110'195 uv195 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd800f120 2023-11-15T03:06:25.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.377+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd801ac60 con 0x55ebd800f120 2023-11-15T03:06:25.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.377+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v110'196 uv196 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd800f120 2023-11-15T03:06:25.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.377+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55ebd801bd00 0x55ebd801e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.377+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd801ac60 con 0x55ebd801bd00 2023-11-15T03:06:25.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.377+0000 7f9b1392c700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55ebd801bd00 0x55ebd801e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.381+0000 7f9b1392c700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55ebd801bd00 0x55ebd801e0f0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:25.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.381+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v110'290 uv277 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.381+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd801e5e0 con 0x55ebd801bd00 2023-11-15T03:06:25.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.385+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v110'291 uv291 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.385+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd801e990 con 0x55ebd801bd00 2023-11-15T03:06:25.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.389+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v110'292 uv292 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.402 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:06:25.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.389+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd801fd50 con 0x55ebd801bd00 2023-11-15T03:06:25.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.393+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v110'293 uv293 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.393+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8020100 con 0x55ebd801bd00 2023-11-15T03:06:25.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.393+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v110'294 uv294 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.393+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd801ed40 con 0x55ebd801bd00 2023-11-15T03:06:25.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.397+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v110'295 uv295 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.397+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd801f610 con 0x55ebd801bd00 2023-11-15T03:06:25.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.401+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v110'296 uv296 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.401+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd80204b0 con 0x55ebd801bd00 2023-11-15T03:06:25.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.401+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v110'297 uv283 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.401+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd80204b0 con 0x55ebd801bd00 2023-11-15T03:06:25.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.405+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v110'298 uv298 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.405+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd80204b0 con 0x55ebd801bd00 2023-11-15T03:06:25.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.409+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v110'299 uv299 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.415 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:06:25.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.409+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd80214c0 con 0x55ebd801bd00 2023-11-15T03:06:25.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.409+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v110'300 uv300 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.409+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd80214c0 con 0x55ebd801bd00 2023-11-15T03:06:25.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.413+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v110'301 uv301 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.413+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd80204b0 con 0x55ebd801bd00 2023-11-15T03:06:25.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.417+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v110'302 uv302 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.417+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd80204b0 con 0x55ebd801bd00 2023-11-15T03:06:25.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.417+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v110'303 uv289 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.417+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd80204b0 con 0x55ebd801bd00 2023-11-15T03:06:25.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.421+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v110'304 uv304 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.421+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd80204b0 con 0x55ebd801bd00 2023-11-15T03:06:25.429 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:06:25.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.425+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v110'305 uv305 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.425+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd80214c0 con 0x55ebd801bd00 2023-11-15T03:06:25.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.425+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v110'306 uv306 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.425+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd80214c0 con 0x55ebd801bd00 2023-11-15T03:06:25.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.429+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v110'307 uv307 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.429+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd80204b0 con 0x55ebd801bd00 2023-11-15T03:06:25.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.429+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v110'308 uv308 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.433+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55ebd80209c0 0x55ebd8022db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.433+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd80232a0 con 0x55ebd80209c0 2023-11-15T03:06:25.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.433+0000 7f9b0b12b700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55ebd80209c0 0x55ebd8022db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.433+0000 7f9b0b12b700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55ebd80209c0 0x55ebd8022db0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:25.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.437+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v110'159 uv158 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd80209c0 2023-11-15T03:06:25.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.437+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd8023650 con 0x55ebd80209c0 2023-11-15T03:06:25.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.437+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v110'160 uv160 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd80209c0 2023-11-15T03:06:25.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.437+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8023a00 con 0x55ebd80209c0 2023-11-15T03:06:25.448 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:06:25.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.441+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v110'161 uv161 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd80209c0 2023-11-15T03:06:25.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.441+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd80242a0 con 0x55ebd80209c0 2023-11-15T03:06:25.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.445+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v110'162 uv162 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd80209c0 2023-11-15T03:06:25.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.445+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8024650 con 0x55ebd80209c0 2023-11-15T03:06:25.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.449+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v110'163 uv163 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd80209c0 2023-11-15T03:06:25.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.449+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8024a00 con 0x55ebd80209c0 2023-11-15T03:06:25.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.453+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v110'164 uv157 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd80209c0 2023-11-15T03:06:25.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.453+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd8024a00 con 0x55ebd80209c0 2023-11-15T03:06:25.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.453+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v110'165 uv165 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd80209c0 2023-11-15T03:06:25.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.453+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8024a00 con 0x55ebd80209c0 2023-11-15T03:06:25.463 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:06:25.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.457+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v110'166 uv166 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd80209c0 2023-11-15T03:06:25.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.457+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8024a00 con 0x55ebd80209c0 2023-11-15T03:06:25.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.461+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v110'167 uv167 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd80209c0 2023-11-15T03:06:25.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.461+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8023e30 con 0x55ebd80209c0 2023-11-15T03:06:25.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.461+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v110'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd80209c0 2023-11-15T03:06:25.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.461+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55ebd8025400 0x55ebd80277f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.465+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8027ce0 con 0x55ebd8025400 2023-11-15T03:06:25.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.465+0000 7f9b1412d700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55ebd8025400 0x55ebd80277f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.465+0000 7f9b1412d700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55ebd8025400 0x55ebd80277f0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:25.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.469+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v110'193 uv186 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.469+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd8027ce0 con 0x55ebd8025400 2023-11-15T03:06:25.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.473+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v110'194 uv194 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.473+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8027ce0 con 0x55ebd8025400 2023-11-15T03:06:25.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.473+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v110'195 uv195 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.480 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:06:25.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.473+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802bd70 con 0x55ebd8025400 2023-11-15T03:06:25.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.481+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v110'196 uv196 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.481+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802bd70 con 0x55ebd8025400 2023-11-15T03:06:25.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.485+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v110'197 uv197 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.485+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8027ce0 con 0x55ebd8025400 2023-11-15T03:06:25.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.489+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v110'198 uv192 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.489+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd8027ce0 con 0x55ebd8025400 2023-11-15T03:06:25.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.493+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v110'199 uv199 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.493+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8027ce0 con 0x55ebd8025400 2023-11-15T03:06:25.503 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:06:25.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.497+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v110'200 uv200 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.497+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8028cf0 con 0x55ebd8025400 2023-11-15T03:06:25.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.497+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v110'201 uv201 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.497+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8028cf0 con 0x55ebd8025400 2023-11-15T03:06:25.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.501+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v110'202 uv202 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.501+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8027ce0 con 0x55ebd8025400 2023-11-15T03:06:25.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.505+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v110'203 uv203 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.505+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55ebd8028360 0x55ebd802a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.505+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802ad70 con 0x55ebd8028360 2023-11-15T03:06:25.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.505+0000 7f9b1392c700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55ebd8028360 0x55ebd802a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.505+0000 7f9b1392c700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55ebd8028360 0x55ebd802a750 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:25.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.509+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v110'93 uv92 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd8028360 2023-11-15T03:06:25.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.509+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd802aed0 con 0x55ebd8028360 2023-11-15T03:06:25.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.513+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v110'94 uv94 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd8028360 2023-11-15T03:06:25.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.513+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802b300 con 0x55ebd8028360 2023-11-15T03:06:25.524 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:06:25.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.513+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v110'95 uv95 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd8028360 2023-11-15T03:06:25.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.513+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802c310 con 0x55ebd8028360 2023-11-15T03:06:25.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.517+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v110'96 uv96 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd8028360 2023-11-15T03:06:25.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.517+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802c310 con 0x55ebd8028360 2023-11-15T03:06:25.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.517+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v110'97 uv97 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd8028360 2023-11-15T03:06:25.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.517+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802b300 con 0x55ebd8028360 2023-11-15T03:06:25.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.521+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v110'98 uv98 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd8028360 2023-11-15T03:06:25.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.521+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55ebd802b810 0x55ebd802dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.521+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802e320 con 0x55ebd802b810 2023-11-15T03:06:25.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.521+0000 7f9b0b12b700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55ebd802b810 0x55ebd802dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.525+0000 7f9b0b12b700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55ebd802b810 0x55ebd802dc00 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:25.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.525+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v110'68 uv67 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.525+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd802e700 con 0x55ebd802b810 2023-11-15T03:06:25.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.533+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v110'69 uv69 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.533+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802e700 con 0x55ebd802b810 2023-11-15T03:06:25.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.533+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v110'70 uv70 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.542 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:06:25.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.533+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802e700 con 0x55ebd802b810 2023-11-15T03:06:25.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.537+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v110'71 uv71 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.537+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802e700 con 0x55ebd802b810 2023-11-15T03:06:25.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.541+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v110'93 uv92 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.541+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd802e700 con 0x55ebd802b810 2023-11-15T03:06:25.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.545+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v110'94 uv94 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.545+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802e700 con 0x55ebd802b810 2023-11-15T03:06:25.556 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:06:25.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.545+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v110'95 uv95 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.545+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802f710 con 0x55ebd802b810 2023-11-15T03:06:25.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.549+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v110'96 uv96 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.549+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802f710 con 0x55ebd802b810 2023-11-15T03:06:25.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.549+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v110'97 uv97 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.553+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802e700 con 0x55ebd802b810 2023-11-15T03:06:25.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.557+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v110'98 uv98 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.557+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802e700 con 0x55ebd801bd00 2023-11-15T03:06:25.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.561+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v110'171 uv165 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.561+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd802e700 con 0x55ebd801bd00 2023-11-15T03:06:25.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.565+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v110'172 uv172 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.565+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802e700 con 0x55ebd801bd00 2023-11-15T03:06:25.574 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:06:25.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.569+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v110'173 uv173 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.569+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802f710 con 0x55ebd801bd00 2023-11-15T03:06:25.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.569+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v110'174 uv174 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.569+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802f710 con 0x55ebd801bd00 2023-11-15T03:06:25.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.573+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v110'175 uv175 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.573+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802e700 con 0x55ebd801bd00 2023-11-15T03:06:25.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.577+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v110'176 uv176 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.577+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802e700 con 0x55ebd801bd00 2023-11-15T03:06:25.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.577+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v110'177 uv170 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.577+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd802e700 con 0x55ebd801bd00 2023-11-15T03:06:25.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.581+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v110'178 uv178 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.581+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802e700 con 0x55ebd801bd00 2023-11-15T03:06:25.592 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:06:25.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.585+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v110'179 uv179 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.585+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802e700 con 0x55ebd801bd00 2023-11-15T03:06:25.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.589+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v110'180 uv180 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.589+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802eb00 con 0x55ebd801bd00 2023-11-15T03:06:25.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.593+0000 7f9b1392c700 1 -- 172.21.15.121:0/1545569035 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v110'181 uv181 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b04014500 con 0x55ebd801bd00 2023-11-15T03:06:25.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.593+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802eb00 con 0x55ebd80209c0 2023-11-15T03:06:25.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.593+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v110'76 uv75 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd80209c0 2023-11-15T03:06:25.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.593+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd802eb00 con 0x55ebd80209c0 2023-11-15T03:06:25.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.597+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v110'77 uv77 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd80209c0 2023-11-15T03:06:25.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.597+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802eb00 con 0x55ebd80209c0 2023-11-15T03:06:25.606 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:06:25.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.601+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v110'78 uv78 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd80209c0 2023-11-15T03:06:25.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.601+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802eb00 con 0x55ebd80209c0 2023-11-15T03:06:25.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.601+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v110'79 uv79 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd80209c0 2023-11-15T03:06:25.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.601+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802eb00 con 0x55ebd80209c0 2023-11-15T03:06:25.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.605+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v110'80 uv80 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd80209c0 2023-11-15T03:06:25.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.605+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802eb00 con 0x55ebd802b810 2023-11-15T03:06:25.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.609+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v110'93 uv92 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.609+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd802eb00 con 0x55ebd802b810 2023-11-15T03:06:25.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.609+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v110'94 uv94 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.609+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802eb00 con 0x55ebd802b810 2023-11-15T03:06:25.619 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:06:25.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.613+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v110'95 uv95 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.613+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802fb10 con 0x55ebd802b810 2023-11-15T03:06:25.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.617+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v110'96 uv96 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.617+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802fb10 con 0x55ebd802b810 2023-11-15T03:06:25.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.617+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v110'97 uv97 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.621+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802eb00 con 0x55ebd802b810 2023-11-15T03:06:25.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.625+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v110'98 uv98 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.625+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802eb00 con 0x55ebd802b810 2023-11-15T03:06:25.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.625+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v110'77 uv76 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.625+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd802eb00 con 0x55ebd802b810 2023-11-15T03:06:25.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.629+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v110'78 uv78 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.629+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802eb00 con 0x55ebd802b810 2023-11-15T03:06:25.641 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:06:25.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.633+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v110'79 uv79 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.633+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802eb00 con 0x55ebd802b810 2023-11-15T03:06:25.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.637+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v110'80 uv80 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.637+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802eb00 con 0x55ebd802b810 2023-11-15T03:06:25.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.641+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v110'81 uv81 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.641+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802eb00 con 0x55ebd802b810 2023-11-15T03:06:25.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.641+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v110'76 uv75 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.641+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd802eb00 con 0x55ebd802b810 2023-11-15T03:06:25.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.645+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v110'77 uv77 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.645+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802eb00 con 0x55ebd802b810 2023-11-15T03:06:25.654 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:06:25.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.649+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v110'78 uv78 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.649+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd802eb00 con 0x55ebd802b810 2023-11-15T03:06:25.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.653+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v110'79 uv79 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.653+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8031870 con 0x55ebd802b810 2023-11-15T03:06:25.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.657+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v110'80 uv80 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.657+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55ebd802f500 0x55ebd802f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.657+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd802f500 2023-11-15T03:06:25.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.657+0000 7f9b1412d700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55ebd802f500 0x55ebd802f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.657+0000 7f9b1412d700 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55ebd802f500 0x55ebd802f8e0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:25.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.661+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v110'62 uv61 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd802f500 2023-11-15T03:06:25.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.661+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd802f500 2023-11-15T03:06:25.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.665+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v110'63 uv63 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd802f500 2023-11-15T03:06:25.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.665+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd802f500 2023-11-15T03:06:25.673 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:06:25.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.665+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v110'64 uv64 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd802f500 2023-11-15T03:06:25.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.665+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd802f500 2023-11-15T03:06:25.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.669+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v110'65 uv65 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd802f500 2023-11-15T03:06:25.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.669+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd802b810 2023-11-15T03:06:25.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.673+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v110'93 uv92 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.673+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd802b810 2023-11-15T03:06:25.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.677+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v110'94 uv94 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.677+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd802b810 2023-11-15T03:06:25.684 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:06:25.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.677+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v110'95 uv95 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.677+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8032fd0 con 0x55ebd802b810 2023-11-15T03:06:25.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.681+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v110'96 uv96 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.681+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8032fd0 con 0x55ebd802b810 2023-11-15T03:06:25.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.685+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v110'97 uv97 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.685+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd802b810 2023-11-15T03:06:25.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.685+0000 7f9b0b12b700 1 -- 172.21.15.121:0/1545569035 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v110'98 uv98 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9afc02c030 con 0x55ebd802b810 2023-11-15T03:06:25.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.685+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd8025400 2023-11-15T03:06:25.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.689+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v110'79 uv78 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.689+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd8025400 2023-11-15T03:06:25.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.693+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v110'80 uv80 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.693+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd8025400 2023-11-15T03:06:25.702 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:06:25.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.697+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v110'81 uv81 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.697+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd8025400 2023-11-15T03:06:25.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.697+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v110'82 uv82 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.697+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd8025400 2023-11-15T03:06:25.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.701+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v110'83 uv83 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.701+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd8025400 2023-11-15T03:06:25.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.705+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v110'184 uv178 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.705+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd8025400 2023-11-15T03:06:25.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.705+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v110'185 uv185 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.705+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd8025400 2023-11-15T03:06:25.716 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:06:25.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.709+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v110'186 uv186 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.709+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd8025400 2023-11-15T03:06:25.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.713+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v110'187 uv187 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.713+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd8025400 2023-11-15T03:06:25.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.717+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v110'188 uv188 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.717+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd8025400 2023-11-15T03:06:25.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.717+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v110'189 uv183 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.717+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd8025400 2023-11-15T03:06:25.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.721+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v110'190 uv190 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.721+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8031fc0 con 0x55ebd8025400 2023-11-15T03:06:25.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.725+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v110'191 uv191 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.731 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:06:25.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.725+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8032fd0 con 0x55ebd8025400 2023-11-15T03:06:25.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.725+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v110'192 uv192 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.725+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd8032fd0 con 0x55ebd8025400 2023-11-15T03:06:25.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.729+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v110'193 uv193 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.729+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd80324b0 con 0x55ebd8025400 2023-11-15T03:06:25.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.733+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v110'194 uv194 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.733+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x55ebd80324b0 con 0x55ebd8025400 2023-11-15T03:06:25.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.733+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v110'195 uv195 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9b0c00db30 con 0x55ebd8025400 2023-11-15T03:06:25.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.733+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55ebd80209c0 msgr2=0x55ebd8022db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.733+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55ebd80209c0 0x55ebd8022db0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.733+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55ebd800f120 msgr2=0x55ebd8003060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.733+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55ebd800f120 0x55ebd8003060 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.733+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55ebd8028360 msgr2=0x55ebd802a750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.733+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55ebd8028360 0x55ebd802a750 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.733+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55ebd801bd00 msgr2=0x55ebd801e0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.733+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55ebd801bd00 0x55ebd801e0f0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.733+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55ebd802f500 msgr2=0x55ebd802f8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.733+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55ebd802f500 0x55ebd802f8e0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.733+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55ebd8025400 msgr2=0x55ebd80277f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.733+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55ebd8025400 0x55ebd80277f0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.733+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55ebd802b810 msgr2=0x55ebd802dc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.733+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55ebd802b810 0x55ebd802dc00 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.733+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 reap_dead start 2023-11-15T03:06:25.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.733+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 reap_dead start 2023-11-15T03:06:25.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.737+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 reap_dead start 2023-11-15T03:06:25.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.737+0000 7f9b1412d700 1 -- 172.21.15.121:0/1545569035 reap_dead start 2023-11-15T03:06:25.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.737+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9af4058ff0 msgr2=0x7f9af405b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.737+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9af4058ff0 0x7f9af405b4b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f9b04003c70 tx=0x7f9b04000bc0).stop 2023-11-15T03:06:25.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.737+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ebd7f58420 msgr2=0x55ebd7ff4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.737+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ebd7f58420 0x55ebd7ff4ff0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7f9afc00afd0 tx=0x7f9afc01c000).stop 2023-11-15T03:06:25.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.737+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 shutdown_connections 2023-11-15T03:06:25.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.737+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9af4058ff0 0x7f9af405b4b0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.737+0000 7f9b160d8d00 1 --2- 172.21.15.121:0/1545569035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ebd7f58420 0x55ebd7ff4ff0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.737+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 >> 172.21.15.121:0/1545569035 conn(0x55ebd7f277d0 msgr2=0x55ebd7f293c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:25.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.737+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 shutdown_connections 2023-11-15T03:06:25.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.737+0000 7f9b160d8d00 1 -- 172.21.15.121:0/1545569035 wait complete. 2023-11-15T03:06:25.752 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 200.00000000 - 2023-11-15T03:06:25.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.801+0000 7fd2d2d3ed00 1 Processor -- start 2023-11-15T03:06:25.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2d2d3ed00 1 -- start start 2023-11-15T03:06:25.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2d2d3ed00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632847ecdd0 0x5632848331a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2d2d3ed00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563284836b00 0x5632848c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2d2d3ed00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632848c5670 0x563284839190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2d2d3ed00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563284765220 con 0x5632848c5670 2023-11-15T03:06:25.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2d2d3ed00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632848c85b0 con 0x563284836b00 2023-11-15T03:06:25.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2d2d3ed00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632848c7f60 con 0x5632847ecdd0 2023-11-15T03:06:25.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2c7a46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632848c5670 0x563284839190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2c7245700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632847ecdd0 0x5632848331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2c7a46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632848c5670 0x563284839190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55052/0 (socket says 172.21.15.121:55052) 2023-11-15T03:06:25.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2c7a46700 1 -- 172.21.15.121:0/2941544868 learned_addr learned my addr 172.21.15.121:0/2941544868 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:25.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2bea44700 1 --2- 172.21.15.121:0/2941544868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563284836b00 0x5632848c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2c7a46700 1 -- 172.21.15.121:0/2941544868 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632847ecdd0 msgr2=0x5632848331a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:25.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2c7a46700 1 --2- 172.21.15.121:0/2941544868 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632847ecdd0 0x5632848331a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2c7a46700 1 -- 172.21.15.121:0/2941544868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563284836b00 msgr2=0x5632848c2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:25.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2c7a46700 1 --2- 172.21.15.121:0/2941544868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563284836b00 0x5632848c2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2c7a46700 1 -- 172.21.15.121:0/2941544868 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c000a180 con 0x5632848c5670 2023-11-15T03:06:25.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2c7a46700 1 --2- 172.21.15.121:0/2941544868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632848c5670 0x563284839190 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x5632847d9e40 tx=0x7fd2c0014000).ready entity=mon.0 client_cookie=e54d29e8c4c3d40c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:25.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2c6243700 1 -- 172.21.15.121:0/2941544868 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c000c4a0 con 0x5632848c5670 2023-11-15T03:06:25.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2c6243700 1 -- 172.21.15.121:0/2941544868 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c000ccf0 con 0x5632848c5670 2023-11-15T03:06:25.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2c6243700 1 -- 172.21.15.121:0/2941544868 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c000ce60 con 0x5632848c5670 2023-11-15T03:06:25.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2d2d3ed00 1 -- 172.21.15.121:0/2941544868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632848c5670 msgr2=0x563284839190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2d2d3ed00 1 --2- 172.21.15.121:0/2941544868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632848c5670 0x563284839190 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x5632847d9e40 tx=0x7fd2c0014000).stop 2023-11-15T03:06:25.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2d2d3ed00 1 -- 172.21.15.121:0/2941544868 shutdown_connections 2023-11-15T03:06:25.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2d2d3ed00 1 --2- 172.21.15.121:0/2941544868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632848c5670 0x563284839190 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2d2d3ed00 1 --2- 172.21.15.121:0/2941544868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563284836b00 0x5632848c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.805+0000 7fd2d2d3ed00 1 --2- 172.21.15.121:0/2941544868 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632847ecdd0 0x5632848331a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.809+0000 7fd2d2d3ed00 1 -- 172.21.15.121:0/2941544868 >> 172.21.15.121:0/2941544868 conn(0x56328483c080 msgr2=0x56328483c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:25.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.809+0000 7fd2d2d3ed00 1 -- 172.21.15.121:0/2941544868 shutdown_connections 2023-11-15T03:06:25.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.809+0000 7fd2d2d3ed00 1 -- 172.21.15.121:0/2941544868 wait complete. 2023-11-15T03:06:25.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.809+0000 7fd2d2d3ed00 1 Processor -- start 2023-11-15T03:06:25.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.809+0000 7fd2d2d3ed00 1 -- start start 2023-11-15T03:06:25.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.809+0000 7fd2d2d3ed00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632847ecdd0 0x563284839190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.809+0000 7fd2d2d3ed00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563284836b00 0x5632848b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.809+0000 7fd2d2d3ed00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632848c5670 0x5632848c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.809+0000 7fd2d2d3ed00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632848331a0 con 0x5632847ecdd0 2023-11-15T03:06:25.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.809+0000 7fd2d2d3ed00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632848b9590 con 0x5632848c5670 2023-11-15T03:06:25.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.809+0000 7fd2d2d3ed00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632848b9870 con 0x563284836b00 2023-11-15T03:06:25.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.809+0000 7fd2c7245700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632847ecdd0 0x563284839190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.809+0000 7fd2c7245700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632847ecdd0 0x563284839190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55066/0 (socket says 172.21.15.121:55066) 2023-11-15T03:06:25.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.809+0000 7fd2c7245700 1 -- 172.21.15.121:0/911881559 learned_addr learned my addr 172.21.15.121:0/911881559 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:25.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.813+0000 7fd2bea44700 1 --2- 172.21.15.121:0/911881559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563284836b00 0x5632848b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.813+0000 7fd2c7a46700 1 --2- 172.21.15.121:0/911881559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632848c5670 0x5632848c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.813+0000 7fd2c7245700 1 -- 172.21.15.121:0/911881559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563284836b00 msgr2=0x5632848b8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.813+0000 7fd2c7245700 1 --2- 172.21.15.121:0/911881559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563284836b00 0x5632848b8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.813+0000 7fd2c7245700 1 -- 172.21.15.121:0/911881559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632848c5670 msgr2=0x5632848c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.813+0000 7fd2c7245700 1 --2- 172.21.15.121:0/911881559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632848c5670 0x5632848c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.813+0000 7fd2c7245700 1 -- 172.21.15.121:0/911881559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56328478ea20 con 0x5632847ecdd0 2023-11-15T03:06:25.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.813+0000 7fd2bea44700 1 --2- 172.21.15.121:0/911881559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563284836b00 0x5632848b8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:25.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.813+0000 7fd2c7245700 1 --2- 172.21.15.121:0/911881559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632847ecdd0 0x563284839190 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7fd2b8005dc0 tx=0x7fd2b800c040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:25.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.813+0000 7fd2c4a40700 1 -- 172.21.15.121:0/911881559 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b800f040 con 0x5632847ecdd0 2023-11-15T03:06:25.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.813+0000 7fd2d2d3ed00 1 -- 172.21.15.121:0/911881559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56328478e770 con 0x5632847ecdd0 2023-11-15T03:06:25.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.813+0000 7fd2c4a40700 1 -- 172.21.15.121:0/911881559 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b8009430 con 0x5632847ecdd0 2023-11-15T03:06:25.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.813+0000 7fd2c4a40700 1 -- 172.21.15.121:0/911881559 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8010ac0 con 0x5632847ecdd0 2023-11-15T03:06:25.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.813+0000 7fd2d2d3ed00 1 -- 172.21.15.121:0/911881559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56328475da70 con 0x5632847ecdd0 2023-11-15T03:06:25.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.813+0000 7fd2c4a40700 1 -- 172.21.15.121:0/911881559 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd2b8015470 con 0x5632847ecdd0 2023-11-15T03:06:25.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.817+0000 7fd2c4a40700 1 --2- 172.21.15.121:0/911881559 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd2a8059020 0x7fd2a805b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.817+0000 7fd2c4a40700 1 -- 172.21.15.121:0/911881559 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7642+0+0 (secure 0 0 0) 0x7fd2b8074770 con 0x5632847ecdd0 2023-11-15T03:06:25.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.817+0000 7fd2bea44700 1 --2- 172.21.15.121:0/911881559 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd2a8059020 0x7fd2a805b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.817+0000 7fd2d2d3ed00 1 --2- 172.21.15.121:0/911881559 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5632848bf020 0x5632848ba860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:25.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.817+0000 7fd2d2d3ed00 1 -- 172.21.15.121:0/911881559 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected e110) v8 -- 0x5632848b8f00 con 0x5632848bf020 2023-11-15T03:06:25.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.817+0000 7fd2c7a46700 1 --2- 172.21.15.121:0/911881559 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5632848bf020 0x5632848ba860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:25.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.817+0000 7fd2bea44700 1 --2- 172.21.15.121:0/911881559 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd2a8059020 0x7fd2a805b4e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x5632848c73b0 tx=0x7fd2b0001740).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:25.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.817+0000 7fd2c7a46700 1 --2- 172.21.15.121:0/911881559 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5632848bf020 0x5632848ba860 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:25.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.821+0000 7fd2c7a46700 1 -- 172.21.15.121:0/911881559 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(1 200.00000000 [writefull 0~80] v110'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd2c000c770 con 0x5632848bf020 2023-11-15T03:06:25.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.821+0000 7fd2d2d3ed00 1 -- 172.21.15.121:0/911881559 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5632848bf020 msgr2=0x5632848ba860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.821+0000 7fd2d2d3ed00 1 --2- 172.21.15.121:0/911881559 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5632848bf020 0x5632848ba860 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:25.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.821+0000 7fd2d2d3ed00 1 -- 172.21.15.121:0/911881559 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd2a8059020 msgr2=0x7fd2a805b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.821+0000 7fd2d2d3ed00 1 --2- 172.21.15.121:0/911881559 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd2a8059020 0x7fd2a805b4e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x5632848c73b0 tx=0x7fd2b0001740).stop 2023-11-15T03:06:25.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.821+0000 7fd2d2d3ed00 1 -- 172.21.15.121:0/911881559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632847ecdd0 msgr2=0x563284839190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:25.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.821+0000 7fd2d2d3ed00 1 --2- 172.21.15.121:0/911881559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632847ecdd0 0x563284839190 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7fd2b8005dc0 tx=0x7fd2b800c040).stop 2023-11-15T03:06:25.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.821+0000 7fd2c7a46700 1 -- 172.21.15.121:0/911881559 reap_dead start 2023-11-15T03:06:25.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.821+0000 7fd2d2d3ed00 1 -- 172.21.15.121:0/911881559 shutdown_connections 2023-11-15T03:06:25.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.821+0000 7fd2d2d3ed00 1 -- 172.21.15.121:0/911881559 >> 172.21.15.121:0/911881559 conn(0x56328483c080 msgr2=0x5632848c5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:25.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.821+0000 7fd2d2d3ed00 1 -- 172.21.15.121:0/911881559 shutdown_connections 2023-11-15T03:06:25.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:25.821+0000 7fd2d2d3ed00 1 -- 172.21.15.121:0/911881559 wait complete. 2023-11-15T03:06:25.839 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:06:26.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.029+0000 7fe8a54db700 1 Processor -- start 2023-11-15T03:06:26.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.033+0000 7fe8a54db700 1 -- start start 2023-11-15T03:06:26.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.033+0000 7fe8a54db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe8a0068e40 0x7fe8a0062820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:26.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.033+0000 7fe8a54db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8a0069220 0x7fe8a0150f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:26.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.033+0000 7fe8a54db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a01547d0 con 0x7fe8a0154c60 2023-11-15T03:06:26.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.033+0000 7fe8a54db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a014b460 con 0x7fe8a0068e40 2023-11-15T03:06:26.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.033+0000 7fe8a54db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a0154530 con 0x7fe8a0069220 2023-11-15T03:06:26.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.033+0000 7fe89f7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe8a0154c60 0x7fe8a00c15d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44700/0 (socket says 172.21.15.121:44700) 2023-11-15T03:06:26.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.033+0000 7fe89f7fe700 1 -- 172.21.15.121:0/359885219 learned_addr learned my addr 172.21.15.121:0/359885219 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:26.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.033+0000 7fe89effd700 1 --2- 172.21.15.121:0/359885219 >> v2:172.21.15.121:3300/0 conn(0x7fe8a0068e40 0x7fe8a0062820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:26.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.033+0000 7fe89e7fc700 1 --2- 172.21.15.121:0/359885219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8a0069220 0x7fe8a0150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:26.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.033+0000 7fe89dffb700 1 -- 172.21.15.121:0/359885219 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315115778 0 0) 0x7fe8a01547d0 con 0x7fe8a0154c60 2023-11-15T03:06:26.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.033+0000 7fe89dffb700 1 -- 172.21.15.121:0/359885219 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8880033f0 con 0x7fe8a0154c60 2023-11-15T03:06:26.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.033+0000 7fe89effd700 1 -- 172.21.15.121:0/359885219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8a0069220 msgr2=0x7fe8a0150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:26.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.033+0000 7fe89effd700 1 --2- 172.21.15.121:0/359885219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8a0069220 0x7fe8a0150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:26.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.033+0000 7fe89effd700 1 -- 172.21.15.121:0/359885219 >> v1:172.21.15.121:6789/0 conn(0x7fe8a0154c60 legacy=0x7fe8a00c15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:26.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.033+0000 7fe89effd700 1 -- 172.21.15.121:0/359885219 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a0062d10 con 0x7fe8a0068e40 2023-11-15T03:06:26.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.037+0000 7fe89effd700 1 --2- 172.21.15.121:0/359885219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8a0068e40 0x7fe8a0062820 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7fe890009760 tx=0x7fe890009060).ready entity=mon.0 client_cookie=d0d30a9f5b74174c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:26.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.037+0000 7fe89dffb700 1 -- 172.21.15.121:0/359885219 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89000d4a0 con 0x7fe8a0068e40 2023-11-15T03:06:26.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.037+0000 7fe89dffb700 1 -- 172.21.15.121:0/359885219 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89000d600 con 0x7fe8a0068e40 2023-11-15T03:06:26.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.037+0000 7fe89dffb700 1 -- 172.21.15.121:0/359885219 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89000d980 con 0x7fe8a0068e40 2023-11-15T03:06:26.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.037+0000 7fe8a54db700 1 -- 172.21.15.121:0/359885219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8a0068e40 msgr2=0x7fe8a0062820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:26.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.037+0000 7fe8a54db700 1 --2- 172.21.15.121:0/359885219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8a0068e40 0x7fe8a0062820 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7fe890009760 tx=0x7fe890009060).stop 2023-11-15T03:06:26.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.037+0000 7fe8a54db700 1 -- 172.21.15.121:0/359885219 shutdown_connections 2023-11-15T03:06:26.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.037+0000 7fe8a54db700 1 --2- 172.21.15.121:0/359885219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8a0069220 0x7fe8a0150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:26.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.037+0000 7fe8a54db700 1 --2- 172.21.15.121:0/359885219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8a0068e40 0x7fe8a0062820 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:26.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.037+0000 7fe8a54db700 1 -- 172.21.15.121:0/359885219 >> 172.21.15.121:0/359885219 conn(0x7fe8a00c4540 msgr2=0x7fe8a0066180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:26.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.037+0000 7fe8a54db700 1 -- 172.21.15.121:0/359885219 shutdown_connections 2023-11-15T03:06:26.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.037+0000 7fe8a54db700 1 -- 172.21.15.121:0/359885219 wait complete. 2023-11-15T03:06:26.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.041+0000 7fe8a54db700 1 Processor -- start 2023-11-15T03:06:26.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.041+0000 7fe8a54db700 1 -- start start 2023-11-15T03:06:26.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.041+0000 7fe8a54db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8a0068e40 0x7fe8a0132170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:26.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.041+0000 7fe8a54db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8a0069220 0x7fe8a0134790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:26.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.041+0000 7fe8a54db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8a0154c60 0x7fe8a014a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:26.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.041+0000 7fe8a54db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a0134c80 con 0x7fe8a0069220 2023-11-15T03:06:26.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.041+0000 7fe8a54db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a0135aa0 con 0x7fe8a0154c60 2023-11-15T03:06:26.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.041+0000 7fe8a54db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a012f5d0 con 0x7fe8a0068e40 2023-11-15T03:06:26.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.041+0000 7fe89e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8a0069220 0x7fe8a0134790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:26.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.041+0000 7fe89e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8a0069220 0x7fe8a0134790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55078/0 (socket says 172.21.15.121:55078) 2023-11-15T03:06:26.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.041+0000 7fe89e7fc700 1 -- 172.21.15.121:0/2066043940 learned_addr learned my addr 172.21.15.121:0/2066043940 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:26.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.041+0000 7fe89f7fe700 1 --2- 172.21.15.121:0/2066043940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8a0154c60 0x7fe8a014a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:26.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.041+0000 7fe89effd700 1 --2- 172.21.15.121:0/2066043940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8a0068e40 0x7fe8a0132170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:26.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.041+0000 7fe89e7fc700 1 -- 172.21.15.121:0/2066043940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8a0068e40 msgr2=0x7fe8a0132170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:26.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.041+0000 7fe89e7fc700 1 --2- 172.21.15.121:0/2066043940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8a0068e40 0x7fe8a0132170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:26.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.041+0000 7fe89e7fc700 1 -- 172.21.15.121:0/2066043940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8a0154c60 msgr2=0x7fe8a014a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:26.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.041+0000 7fe89e7fc700 1 --2- 172.21.15.121:0/2066043940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8a0154c60 0x7fe8a014a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:26.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.041+0000 7fe89e7fc700 1 -- 172.21.15.121:0/2066043940 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8900092a0 con 0x7fe8a0069220 2023-11-15T03:06:26.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.041+0000 7fe89f7fe700 1 --2- 172.21.15.121:0/2066043940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8a0154c60 0x7fe8a014a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:26.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.045+0000 7fe89e7fc700 1 --2- 172.21.15.121:0/2066043940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8a0069220 0x7fe8a0134790 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7fe89400e930 tx=0x7fe894009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:26.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.045+0000 7fe886ffd700 1 -- 172.21.15.121:0/2066043940 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89400c670 con 0x7fe8a0069220 2023-11-15T03:06:26.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.045+0000 7fe886ffd700 1 -- 172.21.15.121:0/2066043940 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89400c7d0 con 0x7fe8a0069220 2023-11-15T03:06:26.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.045+0000 7fe886ffd700 1 -- 172.21.15.121:0/2066043940 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89400cb50 con 0x7fe8a0069220 2023-11-15T03:06:26.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.045+0000 7fe8a54db700 1 -- 172.21.15.121:0/2066043940 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a0134e90 con 0x7fe8a0069220 2023-11-15T03:06:26.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.045+0000 7fe8a54db700 1 -- 172.21.15.121:0/2066043940 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a0135450 con 0x7fe8a0069220 2023-11-15T03:06:26.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.045+0000 7fe8a54db700 1 -- 172.21.15.121:0/2066043940 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a0047060 con 0x7fe8a0069220 2023-11-15T03:06:26.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.045+0000 7fe886ffd700 1 -- 172.21.15.121:0/2066043940 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe89401b400 con 0x7fe8a0069220 2023-11-15T03:06:26.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.049+0000 7fe886ffd700 1 --2- 172.21.15.121:0/2066043940 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe88c05a540 0x7fe88c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:26.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.049+0000 7fe886ffd700 1 -- 172.21.15.121:0/2066043940 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7642+0+0 (secure 0 0 0) 0x7fe894075270 con 0x7fe8a0069220 2023-11-15T03:06:26.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.049+0000 7fe89effd700 1 --2- 172.21.15.121:0/2066043940 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe88c05a540 0x7fe88c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:26.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.049+0000 7fe89effd700 1 --2- 172.21.15.121:0/2066043940 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe88c05a540 0x7fe88c05ca00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fe89000d260 tx=0x7fe890016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:26.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.053+0000 7fe886ffd700 1 -- 172.21.15.121:0/2066043940 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe894048400 con 0x7fe8a0069220 2023-11-15T03:06:26.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:26.249+0000 7fe8a54db700 1 -- 172.21.15.121:0/2066043940 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fe8a0151720 con 0x7fe8a0069220 2023-11-15T03:06:27.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.265+0000 7fe886ffd700 1 -- 172.21.15.121:0/2066043940 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v186) v1 ==== 162+0+0 (secure 0 0 0) 0x7fe89404ba20 con 0x7fe8a0069220 2023-11-15T03:06:27.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.269+0000 7fe8a54db700 1 -- 172.21.15.121:0/2066043940 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe88c05a540 msgr2=0x7fe88c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:27.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.269+0000 7fe8a54db700 1 --2- 172.21.15.121:0/2066043940 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe88c05a540 0x7fe88c05ca00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fe89000d260 tx=0x7fe890016000).stop 2023-11-15T03:06:27.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.269+0000 7fe8a54db700 1 -- 172.21.15.121:0/2066043940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8a0069220 msgr2=0x7fe8a0134790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:27.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.269+0000 7fe8a54db700 1 --2- 172.21.15.121:0/2066043940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8a0069220 0x7fe8a0134790 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7fe89400e930 tx=0x7fe894009060).stop 2023-11-15T03:06:27.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.269+0000 7fe8a54db700 1 -- 172.21.15.121:0/2066043940 shutdown_connections 2023-11-15T03:06:27.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.269+0000 7fe8a54db700 1 --2- 172.21.15.121:0/2066043940 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe88c05a540 0x7fe88c05ca00 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:27.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.269+0000 7fe8a54db700 1 --2- 172.21.15.121:0/2066043940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8a0154c60 0x7fe8a014a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:27.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.269+0000 7fe8a54db700 1 --2- 172.21.15.121:0/2066043940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8a0069220 0x7fe8a0134790 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:27.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.269+0000 7fe8a54db700 1 --2- 172.21.15.121:0/2066043940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8a0068e40 0x7fe8a0132170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:27.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.269+0000 7fe8a54db700 1 -- 172.21.15.121:0/2066043940 >> 172.21.15.121:0/2066043940 conn(0x7fe8a00c4540 msgr2=0x7fe8a01376c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:27.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.269+0000 7fe8a54db700 1 -- 172.21.15.121:0/2066043940 shutdown_connections 2023-11-15T03:06:27.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.269+0000 7fe8a54db700 1 -- 172.21.15.121:0/2066043940 wait complete. 2023-11-15T03:06:27.278 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:06:27.282 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:06:27.277+0000 7f4511337700 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (22) Invalid argument 2023-11-15T03:06:27.299 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:27.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.453+0000 7f43338f1700 1 Processor -- start 2023-11-15T03:06:27.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.453+0000 7f43338f1700 1 -- start start 2023-11-15T03:06:27.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.457+0000 7f43338f1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f432c1562f0 0x7f432c155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:27.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.457+0000 7f43338f1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f432c0a57c0 0x7f432c0a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:27.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.457+0000 7f43338f1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432c0c4c30 con 0x7f432c154ca0 2023-11-15T03:06:27.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.457+0000 7f43338f1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432c0a8280 con 0x7f432c1562f0 2023-11-15T03:06:27.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.457+0000 7f43338f1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432c0a6d40 con 0x7f432c0a57c0 2023-11-15T03:06:27.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.457+0000 7f433168d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f432c154ca0 0x7f432c0a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44710/0 (socket says 172.21.15.121:44710) 2023-11-15T03:06:27.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.457+0000 7f433168d700 1 -- 172.21.15.121:0/2303385919 learned_addr learned my addr 172.21.15.121:0/2303385919 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:27.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.457+0000 7f4330e8c700 1 --2- 172.21.15.121:0/2303385919 >> v2:172.21.15.121:3300/0 conn(0x7f432c1562f0 0x7f432c155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:27.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.457+0000 7f4331e8e700 1 --2- 172.21.15.121:0/2303385919 >> v2:172.21.15.190:3300/0 conn(0x7f432c0a57c0 0x7f432c0a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:27.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.457+0000 7f4323fff700 1 -- 172.21.15.121:0/2303385919 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449431111 0 0) 0x7f432c0c4c30 con 0x7f432c154ca0 2023-11-15T03:06:27.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.457+0000 7f4323fff700 1 -- 172.21.15.121:0/2303385919 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43180033f0 con 0x7f432c154ca0 2023-11-15T03:06:27.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.457+0000 7f4330e8c700 1 -- 172.21.15.121:0/2303385919 >> v2:172.21.15.190:3300/0 conn(0x7f432c0a57c0 msgr2=0x7f432c0a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:27.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.457+0000 7f4330e8c700 1 --2- 172.21.15.121:0/2303385919 >> v2:172.21.15.190:3300/0 conn(0x7f432c0a57c0 0x7f432c0a5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:27.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.457+0000 7f4330e8c700 1 -- 172.21.15.121:0/2303385919 >> v1:172.21.15.121:6789/0 conn(0x7f432c154ca0 legacy=0x7f432c0a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:27.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.457+0000 7f4330e8c700 1 -- 172.21.15.121:0/2303385919 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432c0a6100 con 0x7f432c1562f0 2023-11-15T03:06:27.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.461+0000 7f4330e8c700 1 --2- 172.21.15.121:0/2303385919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f432c1562f0 0x7f432c155080 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f432c0a8820 tx=0x7f431c009060).ready entity=mon.0 client_cookie=c5f05344b2177ad3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:27.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.461+0000 7f4323fff700 1 -- 172.21.15.121:0/2303385919 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c00d4a0 con 0x7f432c1562f0 2023-11-15T03:06:27.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.461+0000 7f4323fff700 1 -- 172.21.15.121:0/2303385919 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c00d600 con 0x7f432c1562f0 2023-11-15T03:06:27.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.461+0000 7f4323fff700 1 -- 172.21.15.121:0/2303385919 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c00d980 con 0x7f432c1562f0 2023-11-15T03:06:27.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.461+0000 7f43338f1700 1 -- 172.21.15.121:0/2303385919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f432c1562f0 msgr2=0x7f432c155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:27.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.461+0000 7f43338f1700 1 --2- 172.21.15.121:0/2303385919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f432c1562f0 0x7f432c155080 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f432c0a8820 tx=0x7f431c009060).stop 2023-11-15T03:06:27.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.461+0000 7f43338f1700 1 -- 172.21.15.121:0/2303385919 shutdown_connections 2023-11-15T03:06:27.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.461+0000 7f43338f1700 1 --2- 172.21.15.121:0/2303385919 >> v2:172.21.15.190:3300/0 conn(0x7f432c0a57c0 0x7f432c0a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:27.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.461+0000 7f43338f1700 1 --2- 172.21.15.121:0/2303385919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f432c1562f0 0x7f432c155080 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:27.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.461+0000 7f43338f1700 1 -- 172.21.15.121:0/2303385919 >> 172.21.15.121:0/2303385919 conn(0x7f432c062190 msgr2=0x7f432c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:27.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.461+0000 7f43338f1700 1 -- 172.21.15.121:0/2303385919 shutdown_connections 2023-11-15T03:06:27.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.461+0000 7f43338f1700 1 -- 172.21.15.121:0/2303385919 wait complete. 2023-11-15T03:06:27.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f43338f1700 1 Processor -- start 2023-11-15T03:06:27.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f43338f1700 1 -- start start 2023-11-15T03:06:27.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f43338f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f432c0a57c0 0x7f432c14c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:27.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f43338f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f432c154ca0 0x7f432c165120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:27.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f43338f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f432c1562f0 0x7f432c0c1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:27.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f43338f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c0ab860 con 0x7f432c154ca0 2023-11-15T03:06:27.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f43338f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c14c520 con 0x7f432c1562f0 2023-11-15T03:06:27.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f43338f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432c0c0f00 con 0x7f432c0a57c0 2023-11-15T03:06:27.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f4330e8c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f432c154ca0 0x7f432c165120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:27.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f4330e8c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f432c154ca0 0x7f432c165120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55092/0 (socket says 172.21.15.121:55092) 2023-11-15T03:06:27.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f4330e8c700 1 -- 172.21.15.121:0/2477850740 learned_addr learned my addr 172.21.15.121:0/2477850740 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:27.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f4331e8e700 1 --2- 172.21.15.121:0/2477850740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f432c1562f0 0x7f432c0c1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:27.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f433168d700 1 --2- 172.21.15.121:0/2477850740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f432c0a57c0 0x7f432c14c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:27.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f4330e8c700 1 -- 172.21.15.121:0/2477850740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f432c0a57c0 msgr2=0x7f432c14c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:27.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f4330e8c700 1 --2- 172.21.15.121:0/2477850740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f432c0a57c0 0x7f432c14c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:27.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f4330e8c700 1 -- 172.21.15.121:0/2477850740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f432c1562f0 msgr2=0x7f432c0c1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:27.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f4330e8c700 1 --2- 172.21.15.121:0/2477850740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f432c1562f0 0x7f432c0c1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:27.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f4330e8c700 1 -- 172.21.15.121:0/2477850740 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c0092a0 con 0x7f432c154ca0 2023-11-15T03:06:27.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f4330e8c700 1 --2- 172.21.15.121:0/2477850740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f432c154ca0 0x7f432c165120 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 rx=0x7f431c010000 tx=0x7f431c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:27.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f43217fa700 1 -- 172.21.15.121:0/2477850740 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c00dd40 con 0x7f432c154ca0 2023-11-15T03:06:27.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f43217fa700 1 -- 172.21.15.121:0/2477850740 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c00d770 con 0x7f432c154ca0 2023-11-15T03:06:27.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f43217fa700 1 -- 172.21.15.121:0/2477850740 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c00dea0 con 0x7f432c154ca0 2023-11-15T03:06:27.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f43338f1700 1 -- 172.21.15.121:0/2477850740 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432c0c11e0 con 0x7f432c154ca0 2023-11-15T03:06:27.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.465+0000 7f43338f1700 1 -- 172.21.15.121:0/2477850740 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432c0ab1d0 con 0x7f432c154ca0 2023-11-15T03:06:27.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.469+0000 7f43338f1700 1 -- 172.21.15.121:0/2477850740 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f432c0ab9d0 con 0x7f432c154ca0 2023-11-15T03:06:27.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.469+0000 7f43217fa700 1 -- 172.21.15.121:0/2477850740 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f431c01d8b0 con 0x7f432c154ca0 2023-11-15T03:06:27.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.469+0000 7f43217fa700 1 --2- 172.21.15.121:0/2477850740 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f431005a510 0x7f431005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:27.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.469+0000 7f43217fa700 1 -- 172.21.15.121:0/2477850740 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 7728+0+0 (secure 0 0 0) 0x7f431c0767c0 con 0x7f432c154ca0 2023-11-15T03:06:27.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.469+0000 7f433168d700 1 --2- 172.21.15.121:0/2477850740 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f431005a510 0x7f431005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:27.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.469+0000 7f433168d700 1 --2- 172.21.15.121:0/2477850740 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f431005a510 0x7f431005c9d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f432c1493d0 tx=0x7f4328009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:27.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.477+0000 7f43217fa700 1 -- 172.21.15.121:0/2477850740 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f431c0498c0 con 0x7f432c154ca0 2023-11-15T03:06:27.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.677+0000 7f43338f1700 1 -- 172.21.15.121:0/2477850740 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f432c0a6f50 con 0x7f432c154ca0 2023-11-15T03:06:27.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.681+0000 7f43217fa700 1 -- 172.21.15.121:0/2477850740 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 187 v187) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f431c04cee0 con 0x7f432c154ca0 2023-11-15T03:06:27.687 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:27.687 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":187,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":187,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:27.266733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8649":{"gid":8649,"name":"a","rank":0,"incarnation":187,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/2127410277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2127410277},{"type":"v1","addr":"172.21.15.121:6836","nonce":2127410277}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:27.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.681+0000 7f43338f1700 1 -- 172.21.15.121:0/2477850740 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f431005a510 msgr2=0x7f431005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:27.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.681+0000 7f43338f1700 1 --2- 172.21.15.121:0/2477850740 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f431005a510 0x7f431005c9d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f432c1493d0 tx=0x7f4328009040).stop 2023-11-15T03:06:27.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.681+0000 7f43338f1700 1 -- 172.21.15.121:0/2477850740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f432c154ca0 msgr2=0x7f432c165120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:27.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.681+0000 7f43338f1700 1 --2- 172.21.15.121:0/2477850740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f432c154ca0 0x7f432c165120 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 rx=0x7f431c010000 tx=0x7f431c016000).stop 2023-11-15T03:06:27.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.681+0000 7f43338f1700 1 -- 172.21.15.121:0/2477850740 shutdown_connections 2023-11-15T03:06:27.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.681+0000 7f43338f1700 1 --2- 172.21.15.121:0/2477850740 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f431005a510 0x7f431005c9d0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:27.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.681+0000 7f43338f1700 1 --2- 172.21.15.121:0/2477850740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f432c1562f0 0x7f432c0c1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:27.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.681+0000 7f43338f1700 1 --2- 172.21.15.121:0/2477850740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f432c154ca0 0x7f432c165120 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:27.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.681+0000 7f43338f1700 1 --2- 172.21.15.121:0/2477850740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f432c0a57c0 0x7f432c14c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:27.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.681+0000 7f43338f1700 1 -- 172.21.15.121:0/2477850740 >> 172.21.15.121:0/2477850740 conn(0x7f432c062190 msgr2=0x7f432c065830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:27.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.681+0000 7f43338f1700 1 -- 172.21.15.121:0/2477850740 shutdown_connections 2023-11-15T03:06:27.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:27.681+0000 7f43338f1700 1 -- 172.21.15.121:0/2477850740 wait complete. 2023-11-15T03:06:27.692 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 187 2023-11-15T03:06:27.708 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:06:28.277 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:06:28.269+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:06:28.277 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:06:28.269+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:06:28.288 INFO:tasks.ceph.mds.a.smithi121.stderr: -19> 2023-11-15T03:06:27.277+0000 7f4511337700 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (22) Invalid argument 2023-11-15T03:06:28.323 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:06:32.710 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:32.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.869+0000 7fc69cedb700 1 Processor -- start 2023-11-15T03:06:32.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.869+0000 7fc69cedb700 1 -- start start 2023-11-15T03:06:32.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.873+0000 7fc69cedb700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc698152780 0x7fc698159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:32.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.873+0000 7fc69cedb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6980c3fa0 con 0x7fc6980c65d0 2023-11-15T03:06:32.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.873+0000 7fc69cedb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc698159630 con 0x7fc6980a5510 2023-11-15T03:06:32.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.873+0000 7fc69cedb700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6980c30b0 con 0x7fc698152780 2023-11-15T03:06:32.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.873+0000 7fc695d9b700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc6980c65d0 0x7fc6980c6a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44716/0 (socket says 172.21.15.121:44716) 2023-11-15T03:06:32.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.873+0000 7fc695d9b700 1 -- 172.21.15.121:0/1395255028 learned_addr learned my addr 172.21.15.121:0/1395255028 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:32.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.873+0000 7fc69659c700 1 --2- 172.21.15.121:0/1395255028 >> v2:172.21.15.121:3300/0 conn(0x7fc698152780 0x7fc698159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:32.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.873+0000 7fc69559a700 1 -- 172.21.15.121:0/1395255028 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 757799373 0 0) 0x7fc6980c3fa0 con 0x7fc6980c65d0 2023-11-15T03:06:32.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.873+0000 7fc69559a700 1 -- 172.21.15.121:0/1395255028 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6800033f0 con 0x7fc6980c65d0 2023-11-15T03:06:32.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.873+0000 7fc69659c700 1 -- 172.21.15.121:0/1395255028 >> v1:172.21.15.190:6789/0 conn(0x7fc6980a5510 legacy=0x7fc6980a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:32.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.873+0000 7fc69659c700 1 -- 172.21.15.121:0/1395255028 >> v1:172.21.15.121:6789/0 conn(0x7fc6980c65d0 legacy=0x7fc6980c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:32.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.873+0000 7fc69659c700 1 -- 172.21.15.121:0/1395255028 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6980c3390 con 0x7fc698152780 2023-11-15T03:06:32.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.873+0000 7fc69659c700 1 --2- 172.21.15.121:0/1395255028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc698152780 0x7fc698159140 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7fc688009760 tx=0x7fc688009060).ready entity=mon.0 client_cookie=9ec6909e01691614 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:32.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.873+0000 7fc69559a700 1 -- 172.21.15.121:0/1395255028 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68800f510 con 0x7fc698152780 2023-11-15T03:06:32.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.873+0000 7fc69559a700 1 -- 172.21.15.121:0/1395255028 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68800f670 con 0x7fc698152780 2023-11-15T03:06:32.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.873+0000 7fc69559a700 1 -- 172.21.15.121:0/1395255028 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68800f9f0 con 0x7fc698152780 2023-11-15T03:06:32.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.877+0000 7fc69cedb700 1 -- 172.21.15.121:0/1395255028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc698152780 msgr2=0x7fc698159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:32.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.877+0000 7fc69cedb700 1 --2- 172.21.15.121:0/1395255028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc698152780 0x7fc698159140 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7fc688009760 tx=0x7fc688009060).stop 2023-11-15T03:06:32.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.877+0000 7fc69cedb700 1 -- 172.21.15.121:0/1395255028 shutdown_connections 2023-11-15T03:06:32.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.877+0000 7fc69cedb700 1 --2- 172.21.15.121:0/1395255028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc698152780 0x7fc698159140 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:32.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.877+0000 7fc69cedb700 1 -- 172.21.15.121:0/1395255028 >> 172.21.15.121:0/1395255028 conn(0x7fc698062190 msgr2=0x7fc6980625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:32.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.877+0000 7fc69cedb700 1 -- 172.21.15.121:0/1395255028 shutdown_connections 2023-11-15T03:06:32.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.877+0000 7fc69cedb700 1 -- 172.21.15.121:0/1395255028 wait complete. 2023-11-15T03:06:32.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.877+0000 7fc69cedb700 1 Processor -- start 2023-11-15T03:06:32.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.877+0000 7fc69cedb700 1 -- start start 2023-11-15T03:06:32.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc69cedb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6980a5510 0x7fc698150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:32.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc69cedb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6980c65d0 0x7fc6980bff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:32.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc69cedb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc698152780 0x7fc6980ab980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:32.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc69cedb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69816cf20 con 0x7fc6980c65d0 2023-11-15T03:06:32.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc69cedb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6981540b0 con 0x7fc698152780 2023-11-15T03:06:32.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc69cedb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69816c310 con 0x7fc6980a5510 2023-11-15T03:06:32.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc695d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6980c65d0 0x7fc6980bff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:32.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc695d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6980c65d0 0x7fc6980bff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55128/0 (socket says 172.21.15.121:55128) 2023-11-15T03:06:32.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc695d9b700 1 -- 172.21.15.121:0/425531397 learned_addr learned my addr 172.21.15.121:0/425531397 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:32.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc696d9d700 1 --2- 172.21.15.121:0/425531397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc698152780 0x7fc6980ab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:32.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc69659c700 1 --2- 172.21.15.121:0/425531397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6980a5510 0x7fc698150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:32.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc695d9b700 1 -- 172.21.15.121:0/425531397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6980a5510 msgr2=0x7fc698150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:32.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc695d9b700 1 --2- 172.21.15.121:0/425531397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6980a5510 0x7fc698150540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:32.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc695d9b700 1 -- 172.21.15.121:0/425531397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc698152780 msgr2=0x7fc6980ab980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:32.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc695d9b700 1 --2- 172.21.15.121:0/425531397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc698152780 0x7fc6980ab980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:32.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc695d9b700 1 -- 172.21.15.121:0/425531397 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6880092a0 con 0x7fc6980c65d0 2023-11-15T03:06:32.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc695d9b700 1 --2- 172.21.15.121:0/425531397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6980c65d0 0x7fc6980bff50 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7fc68c00c590 tx=0x7fc68c007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:32.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc6867fc700 1 -- 172.21.15.121:0/425531397 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0072e0 con 0x7fc6980c65d0 2023-11-15T03:06:32.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc6867fc700 1 -- 172.21.15.121:0/425531397 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c007440 con 0x7fc6980c65d0 2023-11-15T03:06:32.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc6867fc700 1 -- 172.21.15.121:0/425531397 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0057a0 con 0x7fc6980c65d0 2023-11-15T03:06:32.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc69cedb700 1 -- 172.21.15.121:0/425531397 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69816c5f0 con 0x7fc6980c65d0 2023-11-15T03:06:32.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.881+0000 7fc69cedb700 1 -- 172.21.15.121:0/425531397 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69816cbb0 con 0x7fc6980c65d0 2023-11-15T03:06:32.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.885+0000 7fc69cedb700 1 -- 172.21.15.121:0/425531397 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc698154610 con 0x7fc6980c65d0 2023-11-15T03:06:32.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.885+0000 7fc6867fc700 1 -- 172.21.15.121:0/425531397 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc68c005b90 con 0x7fc6980c65d0 2023-11-15T03:06:32.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.885+0000 7fc6867fc700 1 --2- 172.21.15.121:0/425531397 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc67805a510 0x7fc67805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:32.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.885+0000 7fc6867fc700 1 -- 172.21.15.121:0/425531397 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 7728+0+0 (secure 0 0 0) 0x7fc68c0745f0 con 0x7fc6980c65d0 2023-11-15T03:06:32.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.885+0000 7fc69659c700 1 --2- 172.21.15.121:0/425531397 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc67805a510 0x7fc67805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:32.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.889+0000 7fc69659c700 1 --2- 172.21.15.121:0/425531397 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc67805a510 0x7fc67805c9d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fc68800f640 tx=0x7fc688017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:32.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:32.893+0000 7fc6867fc700 1 -- 172.21.15.121:0/425531397 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc698154610 con 0x7fc6980c65d0 2023-11-15T03:06:33.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.089+0000 7fc69cedb700 1 -- 172.21.15.121:0/425531397 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc69806c150 con 0x7fc6980c65d0 2023-11-15T03:06:33.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.089+0000 7fc6867fc700 1 -- 172.21.15.121:0/425531397 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 189 v189) v1 ==== 95+0+4493 (secure 0 0 0) 0x7fc68c047790 con 0x7fc6980c65d0 2023-11-15T03:06:33.096 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:33.096 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":189,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8724,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/909619389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":909619389},{"type":"v1","addr":"172.21.15.121:6836","nonce":909619389}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189}],"filesystems":[{"mdsmap":{"epoch":188,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:28.273064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:33.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.093+0000 7fc69cedb700 1 -- 172.21.15.121:0/425531397 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc67805a510 msgr2=0x7fc67805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:33.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.093+0000 7fc69cedb700 1 --2- 172.21.15.121:0/425531397 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc67805a510 0x7fc67805c9d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fc68800f640 tx=0x7fc688017000).stop 2023-11-15T03:06:33.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.093+0000 7fc69cedb700 1 -- 172.21.15.121:0/425531397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6980c65d0 msgr2=0x7fc6980bff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:33.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.093+0000 7fc69cedb700 1 --2- 172.21.15.121:0/425531397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6980c65d0 0x7fc6980bff50 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7fc68c00c590 tx=0x7fc68c007c80).stop 2023-11-15T03:06:33.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.093+0000 7fc69cedb700 1 -- 172.21.15.121:0/425531397 shutdown_connections 2023-11-15T03:06:33.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.093+0000 7fc69cedb700 1 --2- 172.21.15.121:0/425531397 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc67805a510 0x7fc67805c9d0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:33.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.093+0000 7fc69cedb700 1 --2- 172.21.15.121:0/425531397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc698152780 0x7fc6980ab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:33.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.093+0000 7fc69cedb700 1 --2- 172.21.15.121:0/425531397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6980c65d0 0x7fc6980bff50 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:33.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.093+0000 7fc69cedb700 1 --2- 172.21.15.121:0/425531397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6980a5510 0x7fc698150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:33.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.093+0000 7fc69cedb700 1 -- 172.21.15.121:0/425531397 >> 172.21.15.121:0/425531397 conn(0x7fc698062190 msgr2=0x7fc698062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:33.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.093+0000 7fc69cedb700 1 -- 172.21.15.121:0/425531397 shutdown_connections 2023-11-15T03:06:33.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.093+0000 7fc69cedb700 1 -- 172.21.15.121:0/425531397 wait complete. 2023-11-15T03:06:33.105 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 189 2023-11-15T03:06:33.120 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:06:33.120 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:33.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.281+0000 7fcb8359e700 1 Processor -- start 2023-11-15T03:06:33.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.281+0000 7fcb8359e700 1 -- start start 2023-11-15T03:06:33.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.285+0000 7fcb8359e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcb84153cd0 0x7fcb84067140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:33.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.285+0000 7fcb8359e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb841583f0 con 0x7fcb84066d60 2023-11-15T03:06:33.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.285+0000 7fcb8359e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb841586d0 con 0x7fcb84157ef0 2023-11-15T03:06:33.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.285+0000 7fcb8359e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb840c00f0 con 0x7fcb84153cd0 2023-11-15T03:06:33.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.285+0000 7fcb8259c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fcb84066d60 0x7fcb841577e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44724/0 (socket says 172.21.15.121:44724) 2023-11-15T03:06:33.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.285+0000 7fcb8259c700 1 -- 172.21.15.121:0/1640984295 learned_addr learned my addr 172.21.15.121:0/1640984295 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:33.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.285+0000 7fcb82d9d700 1 --2- 172.21.15.121:0/1640984295 >> v2:172.21.15.121:3300/0 conn(0x7fcb84153cd0 0x7fcb84067140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:33.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.285+0000 7fcb81d9b700 1 -- 172.21.15.121:0/1640984295 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578851177 0 0) 0x7fcb841586d0 con 0x7fcb84157ef0 2023-11-15T03:06:33.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.285+0000 7fcb81d9b700 1 -- 172.21.15.121:0/1640984295 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb700033f0 con 0x7fcb84157ef0 2023-11-15T03:06:33.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.285+0000 7fcb81d9b700 1 -- 172.21.15.121:0/1640984295 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1229847068 0 0) 0x7fcb841583f0 con 0x7fcb84066d60 2023-11-15T03:06:33.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.285+0000 7fcb81d9b700 1 -- 172.21.15.121:0/1640984295 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb841586d0 con 0x7fcb84066d60 2023-11-15T03:06:33.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.285+0000 7fcb82d9d700 1 -- 172.21.15.121:0/1640984295 >> v1:172.21.15.190:6789/0 conn(0x7fcb84157ef0 legacy=0x7fcb841535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:33.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.285+0000 7fcb82d9d700 1 -- 172.21.15.121:0/1640984295 >> v1:172.21.15.121:6789/0 conn(0x7fcb84066d60 legacy=0x7fcb841577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:33.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.285+0000 7fcb82d9d700 1 -- 172.21.15.121:0/1640984295 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb84065890 con 0x7fcb84153cd0 2023-11-15T03:06:33.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.289+0000 7fcb82d9d700 1 --2- 172.21.15.121:0/1640984295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb84153cd0 0x7fcb84067140 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7fcb7c009790 tx=0x7fcb7c009060).ready entity=mon.0 client_cookie=fdfa594b861348fa server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:33.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.289+0000 7fcb81d9b700 1 -- 172.21.15.121:0/1640984295 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c00d4a0 con 0x7fcb84153cd0 2023-11-15T03:06:33.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.289+0000 7fcb81d9b700 1 -- 172.21.15.121:0/1640984295 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c00d600 con 0x7fcb84153cd0 2023-11-15T03:06:33.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.289+0000 7fcb81d9b700 1 -- 172.21.15.121:0/1640984295 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c00d980 con 0x7fcb84153cd0 2023-11-15T03:06:33.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.289+0000 7fcb8359e700 1 -- 172.21.15.121:0/1640984295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb84153cd0 msgr2=0x7fcb84067140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:33.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.289+0000 7fcb8359e700 1 --2- 172.21.15.121:0/1640984295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb84153cd0 0x7fcb84067140 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7fcb7c009790 tx=0x7fcb7c009060).stop 2023-11-15T03:06:33.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.289+0000 7fcb8359e700 1 -- 172.21.15.121:0/1640984295 shutdown_connections 2023-11-15T03:06:33.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.289+0000 7fcb8359e700 1 --2- 172.21.15.121:0/1640984295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb84153cd0 0x7fcb84067140 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:33.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.289+0000 7fcb8359e700 1 -- 172.21.15.121:0/1640984295 >> 172.21.15.121:0/1640984295 conn(0x7fcb84062190 msgr2=0x7fcb84062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:33.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.289+0000 7fcb8359e700 1 -- 172.21.15.121:0/1640984295 shutdown_connections 2023-11-15T03:06:33.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.289+0000 7fcb8359e700 1 -- 172.21.15.121:0/1640984295 wait complete. 2023-11-15T03:06:33.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.293+0000 7fcb8359e700 1 Processor -- start 2023-11-15T03:06:33.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.293+0000 7fcb8359e700 1 -- start start 2023-11-15T03:06:33.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.293+0000 7fcb8359e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb84066d60 0x7fcb84168220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:33.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.293+0000 7fcb8359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb84153cd0 0x7fcb840c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:33.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.293+0000 7fcb8359e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb84157ef0 0x7fcb84154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:33.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.293+0000 7fcb8359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb840ab420 con 0x7fcb84153cd0 2023-11-15T03:06:33.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.293+0000 7fcb8359e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb840c6f00 con 0x7fcb84066d60 2023-11-15T03:06:33.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.293+0000 7fcb8359e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb840c71b0 con 0x7fcb84157ef0 2023-11-15T03:06:33.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.293+0000 7fcb79d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb84153cd0 0x7fcb840c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:33.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.293+0000 7fcb82d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb84157ef0 0x7fcb84154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:33.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.293+0000 7fcb8259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb84066d60 0x7fcb84168220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:33.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.293+0000 7fcb79d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb84153cd0 0x7fcb840c6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55156/0 (socket says 172.21.15.121:55156) 2023-11-15T03:06:33.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.293+0000 7fcb79d9b700 1 -- 172.21.15.121:0/777367491 learned_addr learned my addr 172.21.15.121:0/777367491 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:33.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.297+0000 7fcb79d9b700 1 -- 172.21.15.121:0/777367491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb84157ef0 msgr2=0x7fcb84154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:33.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.297+0000 7fcb79d9b700 1 --2- 172.21.15.121:0/777367491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb84157ef0 0x7fcb84154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:33.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.297+0000 7fcb79d9b700 1 -- 172.21.15.121:0/777367491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb84066d60 msgr2=0x7fcb84168220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:33.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.297+0000 7fcb79d9b700 1 --2- 172.21.15.121:0/777367491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb84066d60 0x7fcb84168220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:33.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.297+0000 7fcb79d9b700 1 -- 172.21.15.121:0/777367491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c0092a0 con 0x7fcb84153cd0 2023-11-15T03:06:33.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.297+0000 7fcb79d9b700 1 --2- 172.21.15.121:0/777367491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb84153cd0 0x7fcb840c6a10 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7fcb6c00c3c0 tx=0x7fcb6c007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:33.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.297+0000 7fcb7affd700 1 -- 172.21.15.121:0/777367491 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c012400 con 0x7fcb84153cd0 2023-11-15T03:06:33.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.297+0000 7fcb7affd700 1 -- 172.21.15.121:0/777367491 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6c012560 con 0x7fcb84153cd0 2023-11-15T03:06:33.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.297+0000 7fcb7affd700 1 -- 172.21.15.121:0/777367491 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c012900 con 0x7fcb84153cd0 2023-11-15T03:06:33.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.297+0000 7fcb8359e700 1 -- 172.21.15.121:0/777367491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb840aadd0 con 0x7fcb84153cd0 2023-11-15T03:06:33.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.297+0000 7fcb8359e700 1 -- 172.21.15.121:0/777367491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb84154770 con 0x7fcb84153cd0 2023-11-15T03:06:33.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.297+0000 7fcb8359e700 1 -- 172.21.15.121:0/777367491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8406bcf0 con 0x7fcb84153cd0 2023-11-15T03:06:33.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.301+0000 7fcb7affd700 1 -- 172.21.15.121:0/777367491 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fcb6c012d30 con 0x7fcb84153cd0 2023-11-15T03:06:33.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.301+0000 7fcb7affd700 1 --2- 172.21.15.121:0/777367491 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcb6405a510 0x7fcb6405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:33.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.301+0000 7fcb7affd700 1 -- 172.21.15.121:0/777367491 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 7728+0+0 (secure 0 0 0) 0x7fcb6c074290 con 0x7fcb84153cd0 2023-11-15T03:06:33.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.301+0000 7fcb8259c700 1 --2- 172.21.15.121:0/777367491 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcb6405a510 0x7fcb6405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:33.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.301+0000 7fcb8259c700 1 --2- 172.21.15.121:0/777367491 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcb6405a510 0x7fcb6405c9d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7fcb74005ad0 tx=0x7fcb74009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:33.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.309+0000 7fcb7affd700 1 -- 172.21.15.121:0/777367491 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcb6c047430 con 0x7fcb84153cd0 2023-11-15T03:06:33.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.505+0000 7fcb8359e700 1 -- 172.21.15.121:0/777367491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb840ab250 con 0x7fcb84153cd0 2023-11-15T03:06:33.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.509+0000 7fcb7affd700 1 -- 172.21.15.121:0/777367491 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 189 v189) v1 ==== 95+0+4493 (secure 0 0 0) 0x7fcb6c04aa50 con 0x7fcb84153cd0 2023-11-15T03:06:33.514 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:33.515 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":189,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8724,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/909619389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":909619389},{"type":"v1","addr":"172.21.15.121:6836","nonce":909619389}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189}],"filesystems":[{"mdsmap":{"epoch":188,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:28.273064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:33.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.509+0000 7fcb8359e700 1 -- 172.21.15.121:0/777367491 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcb6405a510 msgr2=0x7fcb6405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:33.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.509+0000 7fcb8359e700 1 --2- 172.21.15.121:0/777367491 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcb6405a510 0x7fcb6405c9d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7fcb74005ad0 tx=0x7fcb74009040).stop 2023-11-15T03:06:33.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.509+0000 7fcb8359e700 1 -- 172.21.15.121:0/777367491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb84153cd0 msgr2=0x7fcb840c6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:33.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.509+0000 7fcb8359e700 1 --2- 172.21.15.121:0/777367491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb84153cd0 0x7fcb840c6a10 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7fcb6c00c3c0 tx=0x7fcb6c007c80).stop 2023-11-15T03:06:33.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.509+0000 7fcb8359e700 1 -- 172.21.15.121:0/777367491 shutdown_connections 2023-11-15T03:06:33.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.509+0000 7fcb8359e700 1 --2- 172.21.15.121:0/777367491 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcb6405a510 0x7fcb6405c9d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:33.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.509+0000 7fcb8359e700 1 --2- 172.21.15.121:0/777367491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb84157ef0 0x7fcb84154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:33.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.509+0000 7fcb8359e700 1 --2- 172.21.15.121:0/777367491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb84153cd0 0x7fcb840c6a10 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:33.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.509+0000 7fcb8359e700 1 --2- 172.21.15.121:0/777367491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb84066d60 0x7fcb84168220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:33.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.509+0000 7fcb8359e700 1 -- 172.21.15.121:0/777367491 >> 172.21.15.121:0/777367491 conn(0x7fcb84062190 msgr2=0x7fcb8414cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:33.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.509+0000 7fcb8359e700 1 -- 172.21.15.121:0/777367491 shutdown_connections 2023-11-15T03:06:33.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.509+0000 7fcb8359e700 1 -- 172.21.15.121:0/777367491 wait complete. 2023-11-15T03:06:33.520 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 189 2023-11-15T03:06:33.537 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:06:33.537 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:33.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.697+0000 7f3ab5507700 1 Processor -- start 2023-11-15T03:06:33.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.697+0000 7f3ab5507700 1 -- start start 2023-11-15T03:06:33.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.701+0000 7f3ab5507700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3ab0066700 0x7f3ab0154110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:33.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.701+0000 7f3ab5507700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab00c73e0 con 0x7f3ab0066320 2023-11-15T03:06:33.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.701+0000 7f3ab5507700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab01546c0 con 0x7f3ab00acf10 2023-11-15T03:06:33.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.701+0000 7f3ab5507700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab01549a0 con 0x7f3ab0066700 2023-11-15T03:06:33.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.701+0000 7f3aaffff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3ab0066320 0x7f3ab00656a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44740/0 (socket says 172.21.15.121:44740) 2023-11-15T03:06:33.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.701+0000 7f3aaffff700 1 -- 172.21.15.121:0/1045112925 learned_addr learned my addr 172.21.15.121:0/1045112925 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:33.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.701+0000 7f3aaf7fe700 1 --2- 172.21.15.121:0/1045112925 >> v2:172.21.15.190:3300/0 conn(0x7f3ab0066700 0x7f3ab0154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:33.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.705+0000 7f3aaeffd700 1 -- 172.21.15.121:0/1045112925 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2926098483 0 0) 0x7f3ab00c73e0 con 0x7f3ab0066320 2023-11-15T03:06:33.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.705+0000 7f3aaeffd700 1 -- 172.21.15.121:0/1045112925 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a980033f0 con 0x7f3ab0066320 2023-11-15T03:06:33.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.705+0000 7f3aaeffd700 1 -- 172.21.15.121:0/1045112925 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308719569 0 0) 0x7f3ab01546c0 con 0x7f3ab00acf10 2023-11-15T03:06:33.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.705+0000 7f3aaeffd700 1 -- 172.21.15.121:0/1045112925 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ab00c73e0 con 0x7f3ab00acf10 2023-11-15T03:06:33.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.705+0000 7f3aaf7fe700 1 -- 172.21.15.121:0/1045112925 >> v1:172.21.15.190:6789/0 conn(0x7f3ab00acf10 legacy=0x7f3ab0062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:33.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.705+0000 7f3aaf7fe700 1 -- 172.21.15.121:0/1045112925 >> v1:172.21.15.121:6789/0 conn(0x7f3ab0066320 legacy=0x7f3ab00656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:33.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.705+0000 7f3aaf7fe700 1 -- 172.21.15.121:0/1045112925 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab0154c80 con 0x7f3ab0066700 2023-11-15T03:06:33.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.705+0000 7f3aaf7fe700 1 --2- 172.21.15.121:0/1045112925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ab0066700 0x7f3ab0154110 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f3aa4009760 tx=0x7f3aa4009060).ready entity=mon.1 client_cookie=c5cb762cca4c49bd server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:33.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.705+0000 7f3aaeffd700 1 -- 172.21.15.121:0/1045112925 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa400d4a0 con 0x7f3ab0066700 2023-11-15T03:06:33.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.705+0000 7f3aaeffd700 1 -- 172.21.15.121:0/1045112925 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa400d600 con 0x7f3ab0066700 2023-11-15T03:06:33.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.705+0000 7f3aaeffd700 1 -- 172.21.15.121:0/1045112925 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa400d980 con 0x7f3ab0066700 2023-11-15T03:06:33.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.705+0000 7f3ab5507700 1 -- 172.21.15.121:0/1045112925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ab0066700 msgr2=0x7f3ab0154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:33.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.705+0000 7f3ab5507700 1 --2- 172.21.15.121:0/1045112925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ab0066700 0x7f3ab0154110 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f3aa4009760 tx=0x7f3aa4009060).stop 2023-11-15T03:06:33.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.705+0000 7f3ab5507700 1 -- 172.21.15.121:0/1045112925 shutdown_connections 2023-11-15T03:06:33.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.705+0000 7f3ab5507700 1 --2- 172.21.15.121:0/1045112925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ab0066700 0x7f3ab0154110 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:33.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.705+0000 7f3ab5507700 1 -- 172.21.15.121:0/1045112925 >> 172.21.15.121:0/1045112925 conn(0x7f3ab0067a60 msgr2=0x7f3ab0067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:33.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.705+0000 7f3ab5507700 1 -- 172.21.15.121:0/1045112925 shutdown_connections 2023-11-15T03:06:33.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.705+0000 7f3ab5507700 1 -- 172.21.15.121:0/1045112925 wait complete. 2023-11-15T03:06:33.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3ab5507700 1 Processor -- start 2023-11-15T03:06:33.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3ab5507700 1 -- start start 2023-11-15T03:06:33.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3ab5507700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ab0066320 0x7f3ab00c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:33.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3ab5507700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ab0066700 0x7f3ab00a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:33.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3ab5507700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ab00acf10 0x7f3ab00a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:33.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3ab5507700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab00a6610 con 0x7f3ab0066320 2023-11-15T03:06:33.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3ab5507700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab00a5470 con 0x7f3ab0066700 2023-11-15T03:06:33.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3ab5507700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab00a4bd0 con 0x7f3ab00acf10 2023-11-15T03:06:33.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3aaffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ab0066320 0x7f3ab00c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:33.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3aaffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ab0066320 0x7f3ab00c6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55166/0 (socket says 172.21.15.121:55166) 2023-11-15T03:06:33.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3aaffff700 1 -- 172.21.15.121:0/3584608225 learned_addr learned my addr 172.21.15.121:0/3584608225 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:33.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3ab4d06700 1 --2- 172.21.15.121:0/3584608225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ab00acf10 0x7f3ab00a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:33.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3aaf7fe700 1 --2- 172.21.15.121:0/3584608225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ab0066700 0x7f3ab00a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:33.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3aaffff700 1 -- 172.21.15.121:0/3584608225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ab00acf10 msgr2=0x7f3ab00a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:33.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3aaffff700 1 --2- 172.21.15.121:0/3584608225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ab00acf10 0x7f3ab00a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:33.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3aaffff700 1 -- 172.21.15.121:0/3584608225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ab0066700 msgr2=0x7f3ab00a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:33.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3aaffff700 1 --2- 172.21.15.121:0/3584608225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ab0066700 0x7f3ab00a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:33.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3aaffff700 1 -- 172.21.15.121:0/3584608225 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa40092a0 con 0x7f3ab0066320 2023-11-15T03:06:33.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3aaf7fe700 1 --2- 172.21.15.121:0/3584608225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ab0066700 0x7f3ab00a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:33.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3aaffff700 1 --2- 172.21.15.121:0/3584608225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ab0066320 0x7f3ab00c6bb0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7f3aa000c590 tx=0x7f3aa0007d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:33.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3a97fff700 1 -- 172.21.15.121:0/3584608225 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa00072e0 con 0x7f3ab0066320 2023-11-15T03:06:33.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.709+0000 7f3a97fff700 1 -- 172.21.15.121:0/3584608225 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa0007440 con 0x7f3ab0066320 2023-11-15T03:06:33.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.713+0000 7f3ab5507700 1 -- 172.21.15.121:0/3584608225 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab00a4eb0 con 0x7f3ab0066320 2023-11-15T03:06:33.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.713+0000 7f3a97fff700 1 -- 172.21.15.121:0/3584608225 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa00057a0 con 0x7f3ab0066320 2023-11-15T03:06:33.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.713+0000 7f3ab5507700 1 -- 172.21.15.121:0/3584608225 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab00c70a0 con 0x7f3ab0066320 2023-11-15T03:06:33.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.713+0000 7f3ab5507700 1 -- 172.21.15.121:0/3584608225 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab00680f0 con 0x7f3ab0066320 2023-11-15T03:06:33.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.713+0000 7f3a97fff700 1 -- 172.21.15.121:0/3584608225 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3aa0005b90 con 0x7f3ab0066320 2023-11-15T03:06:33.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.717+0000 7f3a97fff700 1 --2- 172.21.15.121:0/3584608225 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3a9c05a510 0x7f3a9c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:33.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.717+0000 7f3a97fff700 1 -- 172.21.15.121:0/3584608225 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 7728+0+0 (secure 0 0 0) 0x7f3aa00747e0 con 0x7f3ab0066320 2023-11-15T03:06:33.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.717+0000 7f3aaf7fe700 1 --2- 172.21.15.121:0/3584608225 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3a9c05a510 0x7f3a9c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:33.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.717+0000 7f3aaf7fe700 1 --2- 172.21.15.121:0/3584608225 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3a9c05a510 0x7f3a9c05c9d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f3ab0066b40 tx=0x7f3aa4016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:33.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.721+0000 7f3a97fff700 1 -- 172.21.15.121:0/3584608225 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3aa00478e0 con 0x7f3ab0066320 2023-11-15T03:06:33.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.925+0000 7f3ab5507700 1 -- 172.21.15.121:0/3584608225 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ab0154f60 con 0x7f3ab0066320 2023-11-15T03:06:33.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.925+0000 7f3a97fff700 1 -- 172.21.15.121:0/3584608225 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 189 v189) v1 ==== 95+0+4493 (secure 0 0 0) 0x7f3aa0018030 con 0x7f3ab0066320 2023-11-15T03:06:33.934 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:33.934 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":189,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8724,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/909619389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":909619389},{"type":"v1","addr":"172.21.15.121:6836","nonce":909619389}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189}],"filesystems":[{"mdsmap":{"epoch":188,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:28.273064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:33.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.933+0000 7f3ab5507700 1 -- 172.21.15.121:0/3584608225 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3a9c05a510 msgr2=0x7f3a9c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:33.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.933+0000 7f3ab5507700 1 --2- 172.21.15.121:0/3584608225 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3a9c05a510 0x7f3a9c05c9d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f3ab0066b40 tx=0x7f3aa4016000).stop 2023-11-15T03:06:33.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.933+0000 7f3ab5507700 1 -- 172.21.15.121:0/3584608225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ab0066320 msgr2=0x7f3ab00c6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:33.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.933+0000 7f3ab5507700 1 --2- 172.21.15.121:0/3584608225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ab0066320 0x7f3ab00c6bb0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7f3aa000c590 tx=0x7f3aa0007d20).stop 2023-11-15T03:06:33.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.933+0000 7f3ab5507700 1 -- 172.21.15.121:0/3584608225 shutdown_connections 2023-11-15T03:06:33.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.933+0000 7f3ab5507700 1 --2- 172.21.15.121:0/3584608225 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3a9c05a510 0x7f3a9c05c9d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:33.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.933+0000 7f3ab5507700 1 --2- 172.21.15.121:0/3584608225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ab00acf10 0x7f3ab00a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:33.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.933+0000 7f3ab5507700 1 --2- 172.21.15.121:0/3584608225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ab0066700 0x7f3ab00a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:33.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.933+0000 7f3ab5507700 1 --2- 172.21.15.121:0/3584608225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ab0066320 0x7f3ab00c6bb0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:33.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.933+0000 7f3ab5507700 1 -- 172.21.15.121:0/3584608225 >> 172.21.15.121:0/3584608225 conn(0x7f3ab0067a60 msgr2=0x7f3ab0153210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:33.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.933+0000 7f3ab5507700 1 -- 172.21.15.121:0/3584608225 shutdown_connections 2023-11-15T03:06:33.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:33.933+0000 7f3ab5507700 1 -- 172.21.15.121:0/3584608225 wait complete. 2023-11-15T03:06:33.942 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 189 2023-11-15T03:06:33.957 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 200.00000000' led to DAMAGED state 2023-11-15T03:06:33.958 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 200.00000001' 2023-11-15T03:06:33.958 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:06:33.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:06:33.959 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:06:33.980 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:06:33.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:06:33.982 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:06:34.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.185+0000 7f66f9d79700 1 Processor -- start 2023-11-15T03:06:34.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.189+0000 7f66f9d79700 1 -- start start 2023-11-15T03:06:34.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.189+0000 7f66f9d79700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f66f41511f0 0x7f66f4066570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:34.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.189+0000 7f66f9d79700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f4066a60 con 0x7f66f40a8750 2023-11-15T03:06:34.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.189+0000 7f66f9d79700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f40660e0 con 0x7f66f40bcab0 2023-11-15T03:06:34.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.189+0000 7f66f9d79700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f40bd390 con 0x7f66f41511f0 2023-11-15T03:06:34.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.193+0000 7f66f9578700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f66f40a8750 0x7f66f40a8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44756/0 (socket says 172.21.15.121:44756) 2023-11-15T03:06:34.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.193+0000 7f66f9578700 1 -- 172.21.15.121:0/2037703365 learned_addr learned my addr 172.21.15.121:0/2037703365 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:34.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.193+0000 7f66f8d77700 1 --2- 172.21.15.121:0/2037703365 >> v2:172.21.15.190:3300/0 conn(0x7f66f41511f0 0x7f66f4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:34.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.193+0000 7f66f37fe700 1 -- 172.21.15.121:0/2037703365 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4063760642 0 0) 0x7f66f4066a60 con 0x7f66f40a8750 2023-11-15T03:06:34.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.193+0000 7f66f37fe700 1 -- 172.21.15.121:0/2037703365 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66e40033f0 con 0x7f66f40a8750 2023-11-15T03:06:34.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.193+0000 7f66f8d77700 1 -- 172.21.15.121:0/2037703365 >> v1:172.21.15.190:6789/0 conn(0x7f66f40bcab0 legacy=0x7f66f41515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:34.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.193+0000 7f66f8d77700 1 -- 172.21.15.121:0/2037703365 >> v1:172.21.15.121:6789/0 conn(0x7f66f40a8750 legacy=0x7f66f40a8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:34.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.193+0000 7f66f8d77700 1 -- 172.21.15.121:0/2037703365 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f40657b0 con 0x7f66f41511f0 2023-11-15T03:06:34.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.193+0000 7f66f8d77700 1 --2- 172.21.15.121:0/2037703365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66f41511f0 0x7f66f4066570 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f66e8009e60 tx=0x7f66e8009780).ready entity=mon.1 client_cookie=5c75ab017086985f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:34.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.193+0000 7f66f37fe700 1 -- 172.21.15.121:0/2037703365 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e800e070 con 0x7f66f41511f0 2023-11-15T03:06:34.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.193+0000 7f66f37fe700 1 -- 172.21.15.121:0/2037703365 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e800bb60 con 0x7f66f41511f0 2023-11-15T03:06:34.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.193+0000 7f66f37fe700 1 -- 172.21.15.121:0/2037703365 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8012410 con 0x7f66f41511f0 2023-11-15T03:06:34.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.193+0000 7f66f9d79700 1 -- 172.21.15.121:0/2037703365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66f41511f0 msgr2=0x7f66f4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:34.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.193+0000 7f66f9d79700 1 --2- 172.21.15.121:0/2037703365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66f41511f0 0x7f66f4066570 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f66e8009e60 tx=0x7f66e8009780).stop 2023-11-15T03:06:34.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.193+0000 7f66f9d79700 1 -- 172.21.15.121:0/2037703365 shutdown_connections 2023-11-15T03:06:34.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.193+0000 7f66f9d79700 1 --2- 172.21.15.121:0/2037703365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66f41511f0 0x7f66f4066570 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:34.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.193+0000 7f66f9d79700 1 -- 172.21.15.121:0/2037703365 >> 172.21.15.121:0/2037703365 conn(0x7f66f4067a60 msgr2=0x7f66f4067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:34.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.193+0000 7f66f9d79700 1 -- 172.21.15.121:0/2037703365 shutdown_connections 2023-11-15T03:06:34.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.193+0000 7f66f9d79700 1 -- 172.21.15.121:0/2037703365 wait complete. 2023-11-15T03:06:34.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.197+0000 7f66f9d79700 1 Processor -- start 2023-11-15T03:06:34.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.197+0000 7f66f9d79700 1 -- start start 2023-11-15T03:06:34.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.197+0000 7f66f9d79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66f40a8750 0x7f66f40aa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:34.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.197+0000 7f66f9d79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66f40bcab0 0x7f66f4152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:34.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.197+0000 7f66f9d79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66f41511f0 0x7f66f41647b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:34.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.197+0000 7f66f9d79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40aaca0 con 0x7f66f40bcab0 2023-11-15T03:06:34.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.197+0000 7f66f9d79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f4152ca0 con 0x7f66f41511f0 2023-11-15T03:06:34.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.197+0000 7f66f9d79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f4153540 con 0x7f66f40a8750 2023-11-15T03:06:34.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.197+0000 7f66f3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66f40bcab0 0x7f66f4152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:34.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.197+0000 7f66f3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66f40bcab0 0x7f66f4152010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55176/0 (socket says 172.21.15.121:55176) 2023-11-15T03:06:34.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.197+0000 7f66f3fff700 1 -- 172.21.15.121:0/150869069 learned_addr learned my addr 172.21.15.121:0/150869069 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:34.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.197+0000 7f66f8d77700 1 --2- 172.21.15.121:0/150869069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66f40a8750 0x7f66f40aa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:34.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.197+0000 7f66f9578700 1 --2- 172.21.15.121:0/150869069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66f41511f0 0x7f66f41647b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:34.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.197+0000 7f66f3fff700 1 -- 172.21.15.121:0/150869069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66f40a8750 msgr2=0x7f66f40aa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:34.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.201+0000 7f66f3fff700 1 --2- 172.21.15.121:0/150869069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66f40a8750 0x7f66f40aa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:34.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.201+0000 7f66f3fff700 1 -- 172.21.15.121:0/150869069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66f41511f0 msgr2=0x7f66f41647b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:34.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.201+0000 7f66f3fff700 1 --2- 172.21.15.121:0/150869069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66f41511f0 0x7f66f41647b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:34.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.201+0000 7f66f3fff700 1 -- 172.21.15.121:0/150869069 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e8009a20 con 0x7f66f40bcab0 2023-11-15T03:06:34.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.201+0000 7f66f9578700 1 --2- 172.21.15.121:0/150869069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66f41511f0 0x7f66f41647b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:34.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.201+0000 7f66f3fff700 1 --2- 172.21.15.121:0/150869069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66f40bcab0 0x7f66f4152010 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7f66e000c470 tx=0x7f66e0007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:34.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.201+0000 7f66f0ff9700 1 -- 172.21.15.121:0/150869069 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e000ce90 con 0x7f66f40bcab0 2023-11-15T03:06:34.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.201+0000 7f66f0ff9700 1 -- 172.21.15.121:0/150869069 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e000f470 con 0x7f66f40bcab0 2023-11-15T03:06:34.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.201+0000 7f66f9d79700 1 -- 172.21.15.121:0/150869069 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f4152500 con 0x7f66f40bcab0 2023-11-15T03:06:34.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.201+0000 7f66f0ff9700 1 -- 172.21.15.121:0/150869069 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e000f810 con 0x7f66f40bcab0 2023-11-15T03:06:34.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.201+0000 7f66f9d79700 1 -- 172.21.15.121:0/150869069 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f4152ef0 con 0x7f66f40bcab0 2023-11-15T03:06:34.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.205+0000 7f66f9d79700 1 -- 172.21.15.121:0/150869069 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f406bcf0 con 0x7f66f40bcab0 2023-11-15T03:06:34.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.205+0000 7f66f0ff9700 1 -- 172.21.15.121:0/150869069 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f66e000fc40 con 0x7f66f40bcab0 2023-11-15T03:06:34.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.205+0000 7f66f0ff9700 1 --2- 172.21.15.121:0/150869069 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f66d805a590 0x7f66d805ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:34.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.205+0000 7f66f0ff9700 1 -- 172.21.15.121:0/150869069 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 7728+0+0 (secure 0 0 0) 0x7f66e00741b0 con 0x7f66f40bcab0 2023-11-15T03:06:34.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.209+0000 7f66f8d77700 1 --2- 172.21.15.121:0/150869069 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f66d805a590 0x7f66d805ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:34.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.209+0000 7f66f8d77700 1 --2- 172.21.15.121:0/150869069 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f66d805a590 0x7f66d805ca50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f66e80094d0 tx=0x7f66e8009780).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:34.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.213+0000 7f66f0ff9700 1 -- 172.21.15.121:0/150869069 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f66e00474c0 con 0x7f66f40bcab0 2023-11-15T03:06:34.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:34.409+0000 7f66f9d79700 1 -- 172.21.15.121:0/150869069 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f66f40c3ce0 con 0x7f66f40bcab0 2023-11-15T03:06:35.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.221+0000 7f66f0ff9700 1 -- 172.21.15.121:0/150869069 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v190) v1 ==== 161+0+0 (secure 0 0 0) 0x7f66e004aae0 con 0x7f66f40bcab0 2023-11-15T03:06:35.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.225+0000 7f66f9d79700 1 -- 172.21.15.121:0/150869069 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f66d805a590 msgr2=0x7f66d805ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:35.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.225+0000 7f66f9d79700 1 --2- 172.21.15.121:0/150869069 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f66d805a590 0x7f66d805ca50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f66e80094d0 tx=0x7f66e8009780).stop 2023-11-15T03:06:35.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.225+0000 7f66f9d79700 1 -- 172.21.15.121:0/150869069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66f40bcab0 msgr2=0x7f66f4152010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:35.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.225+0000 7f66f9d79700 1 --2- 172.21.15.121:0/150869069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66f40bcab0 0x7f66f4152010 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7f66e000c470 tx=0x7f66e0007db0).stop 2023-11-15T03:06:35.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.225+0000 7f66f9d79700 1 -- 172.21.15.121:0/150869069 shutdown_connections 2023-11-15T03:06:35.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.225+0000 7f66f9d79700 1 --2- 172.21.15.121:0/150869069 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f66d805a590 0x7f66d805ca50 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:35.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.225+0000 7f66f9d79700 1 --2- 172.21.15.121:0/150869069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66f41511f0 0x7f66f41647b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:35.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.225+0000 7f66f9d79700 1 --2- 172.21.15.121:0/150869069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66f40bcab0 0x7f66f4152010 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:35.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.225+0000 7f66f9d79700 1 --2- 172.21.15.121:0/150869069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66f40a8750 0x7f66f40aa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:35.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.225+0000 7f66f9d79700 1 -- 172.21.15.121:0/150869069 >> 172.21.15.121:0/150869069 conn(0x7f66f4067a60 msgr2=0x7f66f414bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:35.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.225+0000 7f66f9d79700 1 -- 172.21.15.121:0/150869069 shutdown_connections 2023-11-15T03:06:35.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.229+0000 7f66f9d79700 1 -- 172.21.15.121:0/150869069 wait complete. 2023-11-15T03:06:35.237 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:06:35.256 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:06:35.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.425+0000 7fc33de9b700 1 Processor -- start 2023-11-15T03:06:35.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.425+0000 7fc33de9b700 1 -- start start 2023-11-15T03:06:35.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc33de9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3380c5000 0x7fc3380c5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:35.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc33de9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc3380a9370 0x7fc3380a9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:35.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc33de9b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3380a8ee0 con 0x7fc3380aafe0 2023-11-15T03:06:35.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc33de9b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc338152010 con 0x7fc3380a9370 2023-11-15T03:06:35.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc33de9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3380bf730 con 0x7fc3380c5000 2023-11-15T03:06:35.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc337fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc3380a9370 0x7fc3380a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:35.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc3377fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc3380aafe0 0x7fc3380be8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40504/0 (socket says 172.21.15.121:40504) 2023-11-15T03:06:35.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc3377fe700 1 -- 172.21.15.121:0/3733210302 learned_addr learned my addr 172.21.15.121:0/3733210302 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:35.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc336ffd700 1 --2- 172.21.15.121:0/3733210302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3380c5000 0x7fc3380c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:35.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc3367fc700 1 -- 172.21.15.121:0/3733210302 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739193387 0 0) 0x7fc3380a8ee0 con 0x7fc3380aafe0 2023-11-15T03:06:35.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc3367fc700 1 -- 172.21.15.121:0/3733210302 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3200033f0 con 0x7fc3380aafe0 2023-11-15T03:06:35.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc337fff700 1 -- 172.21.15.121:0/3733210302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3380c5000 msgr2=0x7fc3380c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:35.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc337fff700 1 --2- 172.21.15.121:0/3733210302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3380c5000 0x7fc3380c5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:35.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc337fff700 1 -- 172.21.15.121:0/3733210302 >> v1:172.21.15.121:6789/0 conn(0x7fc3380aafe0 legacy=0x7fc3380be8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:35.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc337fff700 1 -- 172.21.15.121:0/3733210302 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3380c3b30 con 0x7fc3380a9370 2023-11-15T03:06:35.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc336ffd700 1 --2- 172.21.15.121:0/3733210302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3380c5000 0x7fc3380c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:35.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc337fff700 1 --2- 172.21.15.121:0/3733210302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3380a9370 0x7fc3380a9800 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7fc330009e60 tx=0x7fc330009780).ready entity=mon.0 client_cookie=69d486a4d856cdd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:35.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc3367fc700 1 -- 172.21.15.121:0/3733210302 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33000e070 con 0x7fc3380a9370 2023-11-15T03:06:35.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc3367fc700 1 -- 172.21.15.121:0/3733210302 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33000bb60 con 0x7fc3380a9370 2023-11-15T03:06:35.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.429+0000 7fc3367fc700 1 -- 172.21.15.121:0/3733210302 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc330012410 con 0x7fc3380a9370 2023-11-15T03:06:35.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.433+0000 7fc33de9b700 1 -- 172.21.15.121:0/3733210302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3380a9370 msgr2=0x7fc3380a9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:35.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.433+0000 7fc33de9b700 1 --2- 172.21.15.121:0/3733210302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3380a9370 0x7fc3380a9800 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7fc330009e60 tx=0x7fc330009780).stop 2023-11-15T03:06:35.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.433+0000 7fc33de9b700 1 -- 172.21.15.121:0/3733210302 shutdown_connections 2023-11-15T03:06:35.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.433+0000 7fc33de9b700 1 --2- 172.21.15.121:0/3733210302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3380a9370 0x7fc3380a9800 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:35.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.433+0000 7fc33de9b700 1 --2- 172.21.15.121:0/3733210302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3380c5000 0x7fc3380c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:35.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.433+0000 7fc33de9b700 1 -- 172.21.15.121:0/3733210302 >> 172.21.15.121:0/3733210302 conn(0x7fc338067a60 msgr2=0x7fc338067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:35.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.433+0000 7fc33de9b700 1 -- 172.21.15.121:0/3733210302 shutdown_connections 2023-11-15T03:06:35.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.433+0000 7fc33de9b700 1 -- 172.21.15.121:0/3733210302 wait complete. 2023-11-15T03:06:35.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc33de9b700 1 Processor -- start 2023-11-15T03:06:35.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc33de9b700 1 -- start start 2023-11-15T03:06:35.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc33de9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3380a9370 0x7fc3380bf970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:35.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc33de9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3380aafe0 0x7fc338164930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:35.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc33de9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3380c5000 0x7fc3381431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:35.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc33de9b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc338167e20 con 0x7fc3380aafe0 2023-11-15T03:06:35.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc33de9b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc338164e20 con 0x7fc3380c5000 2023-11-15T03:06:35.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc33de9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3380bffe0 con 0x7fc3380a9370 2023-11-15T03:06:35.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc336ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3380aafe0 0x7fc338164930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:35.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc336ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3380aafe0 0x7fc338164930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50626/0 (socket says 172.21.15.121:50626) 2023-11-15T03:06:35.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc336ffd700 1 -- 172.21.15.121:0/3287572762 learned_addr learned my addr 172.21.15.121:0/3287572762 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:35.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc3377fe700 1 --2- 172.21.15.121:0/3287572762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3380a9370 0x7fc3380bf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:35.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc337fff700 1 --2- 172.21.15.121:0/3287572762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3380c5000 0x7fc3381431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:35.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc336ffd700 1 -- 172.21.15.121:0/3287572762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3380a9370 msgr2=0x7fc3380bf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:35.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc336ffd700 1 --2- 172.21.15.121:0/3287572762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3380a9370 0x7fc3380bf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:35.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc336ffd700 1 -- 172.21.15.121:0/3287572762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3380c5000 msgr2=0x7fc3381431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:35.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc336ffd700 1 --2- 172.21.15.121:0/3287572762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3380c5000 0x7fc3381431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:35.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc336ffd700 1 -- 172.21.15.121:0/3287572762 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc330009a20 con 0x7fc3380aafe0 2023-11-15T03:06:35.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc337fff700 1 --2- 172.21.15.121:0/3287572762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3380c5000 0x7fc3381431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:35.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.437+0000 7fc336ffd700 1 --2- 172.21.15.121:0/3287572762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3380aafe0 0x7fc338164930 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7fc32c00d890 tx=0x7fc32c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:35.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.441+0000 7fc31f7fe700 1 -- 172.21.15.121:0/3287572762 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c011590 con 0x7fc3380aafe0 2023-11-15T03:06:35.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.441+0000 7fc31f7fe700 1 -- 172.21.15.121:0/3287572762 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32c0116f0 con 0x7fc3380aafe0 2023-11-15T03:06:35.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.441+0000 7fc33de9b700 1 -- 172.21.15.121:0/3287572762 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc338167480 con 0x7fc3380aafe0 2023-11-15T03:06:35.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.441+0000 7fc31f7fe700 1 -- 172.21.15.121:0/3287572762 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c011a70 con 0x7fc3380aafe0 2023-11-15T03:06:35.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.441+0000 7fc33de9b700 1 -- 172.21.15.121:0/3287572762 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc338167a40 con 0x7fc3380aafe0 2023-11-15T03:06:35.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.441+0000 7fc33de9b700 1 -- 172.21.15.121:0/3287572762 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc338167f60 con 0x7fc3380aafe0 2023-11-15T03:06:35.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.441+0000 7fc31f7fe700 1 -- 172.21.15.121:0/3287572762 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc32c01c460 con 0x7fc3380aafe0 2023-11-15T03:06:35.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.445+0000 7fc31f7fe700 1 --2- 172.21.15.121:0/3287572762 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc32405a560 0x7fc32405ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:35.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.445+0000 7fc31f7fe700 1 -- 172.21.15.121:0/3287572762 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 7728+0+0 (secure 0 0 0) 0x7fc32c074b00 con 0x7fc3380aafe0 2023-11-15T03:06:35.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.445+0000 7fc3377fe700 1 --2- 172.21.15.121:0/3287572762 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc32405a560 0x7fc32405ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:35.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.445+0000 7fc3377fe700 1 --2- 172.21.15.121:0/3287572762 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc32405a560 0x7fc32405ca20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7fc328005750 tx=0x7fc328009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:35.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.449+0000 7fc31f7fe700 1 -- 172.21.15.121:0/3287572762 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc338167f60 con 0x7fc3380aafe0 2023-11-15T03:06:35.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:35.669+0000 7fc33de9b700 1 -- 172.21.15.121:0/3287572762 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fc338167f60 con 0x7fc3380aafe0 2023-11-15T03:06:36.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.233+0000 7fc31f7fe700 1 -- 172.21.15.121:0/3287572762 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v191) v1 ==== 101+0+0 (secure 0 0 0) 0x7fc338167f60 con 0x7fc3380aafe0 2023-11-15T03:06:36.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.237+0000 7fc33de9b700 1 -- 172.21.15.121:0/3287572762 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc32405a560 msgr2=0x7fc32405ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.237+0000 7fc33de9b700 1 --2- 172.21.15.121:0/3287572762 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc32405a560 0x7fc32405ca20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7fc328005750 tx=0x7fc328009040).stop 2023-11-15T03:06:36.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.237+0000 7fc33de9b700 1 -- 172.21.15.121:0/3287572762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3380aafe0 msgr2=0x7fc338164930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.237+0000 7fc33de9b700 1 --2- 172.21.15.121:0/3287572762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3380aafe0 0x7fc338164930 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7fc32c00d890 tx=0x7fc32c008040).stop 2023-11-15T03:06:36.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.237+0000 7fc33de9b700 1 -- 172.21.15.121:0/3287572762 shutdown_connections 2023-11-15T03:06:36.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.237+0000 7fc33de9b700 1 --2- 172.21.15.121:0/3287572762 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc32405a560 0x7fc32405ca20 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.237+0000 7fc33de9b700 1 --2- 172.21.15.121:0/3287572762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3380c5000 0x7fc3381431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.237+0000 7fc33de9b700 1 --2- 172.21.15.121:0/3287572762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3380aafe0 0x7fc338164930 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.237+0000 7fc33de9b700 1 --2- 172.21.15.121:0/3287572762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3380a9370 0x7fc3380bf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.237+0000 7fc33de9b700 1 -- 172.21.15.121:0/3287572762 >> 172.21.15.121:0/3287572762 conn(0x7fc338067a60 msgr2=0x7fc3380aa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:36.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.237+0000 7fc33de9b700 1 -- 172.21.15.121:0/3287572762 shutdown_connections 2023-11-15T03:06:36.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.237+0000 7fc33de9b700 1 -- 172.21.15.121:0/3287572762 wait complete. 2023-11-15T03:06:36.247 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:06:36.268 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:06:36.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.317+0000 7fe04e334d00 1 Processor -- start 2023-11-15T03:06:36.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.317+0000 7fe04e334d00 1 -- start start 2023-11-15T03:06:36.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe04e334d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567a02b88d0 0x5567a032d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe04e334d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5567a032e650 0x5567a02b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe04e334d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567a032ea30 0x5567a02b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe04e334d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567a02aff50 con 0x5567a02b88d0 2023-11-15T03:06:36.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe04e334d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567a032db80 con 0x5567a032ea30 2023-11-15T03:06:36.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe04e334d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567a029ffa0 con 0x5567a032e650 2023-11-15T03:06:36.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe042a3c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567a02b88d0 0x5567a032d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe042a3c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567a02b88d0 0x5567a032d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50640/0 (socket says 172.21.15.121:50640) 2023-11-15T03:06:36.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe042a3c700 1 -- 172.21.15.121:0/2501735671 learned_addr learned my addr 172.21.15.121:0/2501735671 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:36.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe04323d700 1 --2- 172.21.15.121:0/2501735671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567a032ea30 0x5567a02b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe03a23b700 1 --2- 172.21.15.121:0/2501735671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5567a032e650 0x5567a02b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe042a3c700 1 -- 172.21.15.121:0/2501735671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5567a032e650 msgr2=0x5567a02b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:36.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe042a3c700 1 --2- 172.21.15.121:0/2501735671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5567a032e650 0x5567a02b03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe042a3c700 1 -- 172.21.15.121:0/2501735671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567a032ea30 msgr2=0x5567a02b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:36.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe042a3c700 1 --2- 172.21.15.121:0/2501735671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567a032ea30 0x5567a02b7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe042a3c700 1 -- 172.21.15.121:0/2501735671 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c00a180 con 0x5567a02b88d0 2023-11-15T03:06:36.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe042a3c700 1 --2- 172.21.15.121:0/2501735671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567a02b88d0 0x5567a032d500 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 rx=0x7fe0340097d0 tx=0x7fe03400c040).ready entity=mon.0 client_cookie=8bd9d5dee56b4a75 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:36.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe041a3a700 1 -- 172.21.15.121:0/2501735671 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03400f040 con 0x5567a02b88d0 2023-11-15T03:06:36.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe041a3a700 1 -- 172.21.15.121:0/2501735671 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe034008860 con 0x5567a02b88d0 2023-11-15T03:06:36.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe041a3a700 1 -- 172.21.15.121:0/2501735671 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034008be0 con 0x5567a02b88d0 2023-11-15T03:06:36.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe04e334d00 1 -- 172.21.15.121:0/2501735671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567a02b88d0 msgr2=0x5567a032d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe04e334d00 1 --2- 172.21.15.121:0/2501735671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567a02b88d0 0x5567a032d500 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 rx=0x7fe0340097d0 tx=0x7fe03400c040).stop 2023-11-15T03:06:36.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe04e334d00 1 -- 172.21.15.121:0/2501735671 shutdown_connections 2023-11-15T03:06:36.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe04e334d00 1 --2- 172.21.15.121:0/2501735671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567a032ea30 0x5567a02b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe04e334d00 1 --2- 172.21.15.121:0/2501735671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5567a032e650 0x5567a02b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe04e334d00 1 --2- 172.21.15.121:0/2501735671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567a02b88d0 0x5567a032d500 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe04e334d00 1 -- 172.21.15.121:0/2501735671 >> 172.21.15.121:0/2501735671 conn(0x5567a02b68f0 msgr2=0x5567a026e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:36.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe04e334d00 1 -- 172.21.15.121:0/2501735671 shutdown_connections 2023-11-15T03:06:36.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.321+0000 7fe04e334d00 1 -- 172.21.15.121:0/2501735671 wait complete. 2023-11-15T03:06:36.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe04e334d00 1 Processor -- start 2023-11-15T03:06:36.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe04e334d00 1 -- start start 2023-11-15T03:06:36.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe04e334d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567a02b88d0 0x5567a0355eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe04e334d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567a032e650 0x5567a026cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe04e334d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5567a032ea30 0x5567a03586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe04e334d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567a029ace0 con 0x5567a02b88d0 2023-11-15T03:06:36.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe04e334d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567a029afc0 con 0x5567a032e650 2023-11-15T03:06:36.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe04e334d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567a029b2a0 con 0x5567a032ea30 2023-11-15T03:06:36.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe042a3c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567a02b88d0 0x5567a0355eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe042a3c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567a02b88d0 0x5567a0355eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50656/0 (socket says 172.21.15.121:50656) 2023-11-15T03:06:36.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 learned_addr learned my addr 172.21.15.121:0/1395593081 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:36.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe03a23b700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567a032e650 0x5567a026cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe04323d700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5567a032ea30 0x5567a03586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5567a032ea30 msgr2=0x5567a03586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe042a3c700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5567a032ea30 0x5567a03586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567a032e650 msgr2=0x5567a026cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe042a3c700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567a032e650 0x5567a026cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe034009430 con 0x5567a02b88d0 2023-11-15T03:06:36.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe03a23b700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567a032e650 0x5567a026cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:36.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.325+0000 7fe042a3c700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567a02b88d0 0x5567a0355eb0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x5567a02b0090 tx=0x7fe034018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:36.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.329+0000 7fe03bfff700 1 -- 172.21.15.121:0/1395593081 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03400f040 con 0x5567a02b88d0 2023-11-15T03:06:36.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.329+0000 7fe03bfff700 1 -- 172.21.15.121:0/1395593081 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe034008be0 con 0x5567a02b88d0 2023-11-15T03:06:36.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.329+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02c005cf0 con 0x5567a02b88d0 2023-11-15T03:06:36.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.329+0000 7fe03bfff700 1 -- 172.21.15.121:0/1395593081 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03401e640 con 0x5567a02b88d0 2023-11-15T03:06:36.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.329+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567a020ea20 con 0x5567a02b88d0 2023-11-15T03:06:36.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.329+0000 7fe03bfff700 1 -- 172.21.15.121:0/1395593081 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe03401ea70 con 0x5567a02b88d0 2023-11-15T03:06:36.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.333+0000 7fe03bfff700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe024058ff0 0x7fe02405b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.333+0000 7fe03bfff700 1 -- 172.21.15.121:0/1395593081 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 7728+0+0 (secure 0 0 0) 0x7fe034075350 con 0x5567a02b88d0 2023-11-15T03:06:36.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.333+0000 7fe03a23b700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe024058ff0 0x7fe02405b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.350 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:06:36.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.333+0000 7fe04e334d00 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5567a026d2b0 0x5567a029c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.333+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0326c70 con 0x5567a026d2b0 2023-11-15T03:06:36.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.333+0000 7fe04323d700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5567a026d2b0 0x5567a029c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.333+0000 7fe03a23b700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe024058ff0 0x7fe02405b4b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x5567a025cc40 tx=0x7fe02c015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:36.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.333+0000 7fe04323d700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5567a026d2b0 0x5567a029c290 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:36.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.337+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v111'197 uv190 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a026d2b0 2023-11-15T03:06:36.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.337+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a02b61e0 con 0x5567a026d2b0 2023-11-15T03:06:36.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.337+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v111'198 uv198 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a026d2b0 2023-11-15T03:06:36.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.337+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a02707c0 con 0x5567a026d2b0 2023-11-15T03:06:36.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.341+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v111'199 uv199 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a026d2b0 2023-11-15T03:06:36.354 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:06:36.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.341+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0358be0 con 0x5567a026d2b0 2023-11-15T03:06:36.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.341+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v111'200 uv200 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a026d2b0 2023-11-15T03:06:36.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.341+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0358f90 con 0x5567a026d2b0 2023-11-15T03:06:36.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.345+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v111'201 uv201 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a026d2b0 2023-11-15T03:06:36.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.345+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0362940 con 0x5567a026d2b0 2023-11-15T03:06:36.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.345+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v111'202 uv202 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a026d2b0 2023-11-15T03:06:36.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.345+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0362cf0 con 0x5567a026d2b0 2023-11-15T03:06:36.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.349+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v111'203 uv196 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a026d2b0 2023-11-15T03:06:36.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.349+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a03630a0 con 0x5567a026d2b0 2023-11-15T03:06:36.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.349+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v111'204 uv204 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a026d2b0 2023-11-15T03:06:36.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.349+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03630a0 con 0x5567a026d2b0 2023-11-15T03:06:36.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.353+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v111'205 uv205 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a026d2b0 2023-11-15T03:06:36.361 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:06:36.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.353+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03640b0 con 0x5567a026d2b0 2023-11-15T03:06:36.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.357+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v111'206 uv206 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a026d2b0 2023-11-15T03:06:36.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.357+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03640b0 con 0x5567a026d2b0 2023-11-15T03:06:36.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.357+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v111'207 uv207 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a026d2b0 2023-11-15T03:06:36.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.357+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03630a0 con 0x5567a026d2b0 2023-11-15T03:06:36.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.361+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v111'208 uv208 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a026d2b0 2023-11-15T03:06:36.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.361+0000 7fe04e334d00 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5567a03635b0 0x5567a03659a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.361+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0365e90 con 0x5567a03635b0 2023-11-15T03:06:36.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.361+0000 7fe042a3c700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5567a03635b0 0x5567a03659a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.361+0000 7fe042a3c700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5567a03635b0 0x5567a03659a0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:36.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.365+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v111'309 uv296 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.365+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a0366240 con 0x5567a03635b0 2023-11-15T03:06:36.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.369+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v111'310 uv310 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.369+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0366670 con 0x5567a03635b0 2023-11-15T03:06:36.379 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:06:36.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.373+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v111'311 uv311 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.373+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0362570 con 0x5567a03635b0 2023-11-15T03:06:36.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.377+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v111'312 uv312 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.377+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0367ff0 con 0x5567a03635b0 2023-11-15T03:06:36.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.377+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v111'313 uv313 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.377+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0366f90 con 0x5567a03635b0 2023-11-15T03:06:36.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.381+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v111'314 uv314 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.381+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03678e0 con 0x5567a03635b0 2023-11-15T03:06:36.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.385+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v111'315 uv315 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.385+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0368420 con 0x5567a03635b0 2023-11-15T03:06:36.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.385+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v111'316 uv302 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.385+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a0368420 con 0x5567a03635b0 2023-11-15T03:06:36.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.389+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v111'317 uv317 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.389+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0368420 con 0x5567a03635b0 2023-11-15T03:06:36.398 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:06:36.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.393+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v111'318 uv318 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.393+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0369430 con 0x5567a03635b0 2023-11-15T03:06:36.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.393+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v111'319 uv319 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.393+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0369430 con 0x5567a03635b0 2023-11-15T03:06:36.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.397+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v111'320 uv320 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.397+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0368420 con 0x5567a03635b0 2023-11-15T03:06:36.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.397+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v111'321 uv321 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.401+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0368420 con 0x5567a03635b0 2023-11-15T03:06:36.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.401+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v111'322 uv308 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.401+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a0368420 con 0x5567a03635b0 2023-11-15T03:06:36.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.405+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v111'323 uv323 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.405+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0368420 con 0x5567a03635b0 2023-11-15T03:06:36.414 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:06:36.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.409+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v111'324 uv324 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.409+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0369430 con 0x5567a03635b0 2023-11-15T03:06:36.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.409+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v111'325 uv325 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.409+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0369430 con 0x5567a03635b0 2023-11-15T03:06:36.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.413+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v111'326 uv326 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.413+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0368420 con 0x5567a03635b0 2023-11-15T03:06:36.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.417+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v111'327 uv327 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.417+0000 7fe04e334d00 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5567a0368930 0x5567a036ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.417+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a036b250 con 0x5567a0368930 2023-11-15T03:06:36.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.417+0000 7fe03a23b700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5567a0368930 0x5567a036ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.417+0000 7fe03a23b700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5567a0368930 0x5567a036ad20 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:36.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.421+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v111'170 uv163 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0368930 2023-11-15T03:06:36.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.421+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a036b780 con 0x5567a0368930 2023-11-15T03:06:36.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.425+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v111'171 uv171 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0368930 2023-11-15T03:06:36.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.425+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a036bc70 con 0x5567a0368930 2023-11-15T03:06:36.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.425+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v111'172 uv172 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0368930 2023-11-15T03:06:36.438 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:06:36.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.425+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a036c560 con 0x5567a0368930 2023-11-15T03:06:36.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.429+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v111'173 uv173 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0368930 2023-11-15T03:06:36.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.429+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a036c910 con 0x5567a0368930 2023-11-15T03:06:36.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.433+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v111'174 uv174 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0368930 2023-11-15T03:06:36.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.433+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a036ccc0 con 0x5567a0368930 2023-11-15T03:06:36.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.437+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v111'175 uv169 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0368930 2023-11-15T03:06:36.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.437+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a036d070 con 0x5567a0368930 2023-11-15T03:06:36.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.437+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v111'176 uv176 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0368930 2023-11-15T03:06:36.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.437+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a036d420 con 0x5567a0368930 2023-11-15T03:06:36.449 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:06:36.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.441+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v111'177 uv177 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0368930 2023-11-15T03:06:36.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.441+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a036d420 con 0x5567a0368930 2023-11-15T03:06:36.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.445+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v111'178 uv178 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0368930 2023-11-15T03:06:36.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.445+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a036d420 con 0x5567a0368930 2023-11-15T03:06:36.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.449+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v111'179 uv179 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0368930 2023-11-15T03:06:36.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.449+0000 7fe04e334d00 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5567a036d930 0x5567a036fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.449+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0370330 con 0x5567a036d930 2023-11-15T03:06:36.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.449+0000 7fe04323d700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5567a036d930 0x5567a036fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.449+0000 7fe04323d700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5567a036d930 0x5567a036fd20 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:36.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.453+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v111'204 uv197 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.453+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a0370710 con 0x5567a036d930 2023-11-15T03:06:36.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.457+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v111'205 uv205 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.457+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0370710 con 0x5567a036d930 2023-11-15T03:06:36.467 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:06:36.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.461+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v111'206 uv206 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.461+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03747a0 con 0x5567a036d930 2023-11-15T03:06:36.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.461+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v111'207 uv207 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.465+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03747a0 con 0x5567a036d930 2023-11-15T03:06:36.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.465+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v111'208 uv208 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.465+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0370710 con 0x5567a036d930 2023-11-15T03:06:36.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.469+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v111'209 uv203 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.469+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a0370710 con 0x5567a036d930 2023-11-15T03:06:36.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.473+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v111'210 uv210 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.473+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0370710 con 0x5567a036d930 2023-11-15T03:06:36.487 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:06:36.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.473+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v111'211 uv211 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.473+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0371720 con 0x5567a036d930 2023-11-15T03:06:36.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.477+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v111'212 uv212 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.477+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0371720 con 0x5567a036d930 2023-11-15T03:06:36.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.481+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v111'213 uv213 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.481+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0370710 con 0x5567a036d930 2023-11-15T03:06:36.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.485+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v111'214 uv214 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.485+0000 7fe04e334d00 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5567a0370d90 0x5567a0373180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.485+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0373700 con 0x5567a0370d90 2023-11-15T03:06:36.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.485+0000 7fe042a3c700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5567a0370d90 0x5567a0373180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.485+0000 7fe042a3c700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5567a0370d90 0x5567a0373180 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:36.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.485+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v111'99 uv98 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a0370d90 2023-11-15T03:06:36.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.485+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a0373ae0 con 0x5567a0370d90 2023-11-15T03:06:36.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.489+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v111'100 uv100 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a0370d90 2023-11-15T03:06:36.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.489+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0373ae0 con 0x5567a0370d90 2023-11-15T03:06:36.499 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:06:36.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.493+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v111'101 uv101 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a0370d90 2023-11-15T03:06:36.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.493+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0374af0 con 0x5567a0370d90 2023-11-15T03:06:36.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.493+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v111'102 uv102 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a0370d90 2023-11-15T03:06:36.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.493+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0374af0 con 0x5567a0370d90 2023-11-15T03:06:36.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.497+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v111'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a0370d90 2023-11-15T03:06:36.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.497+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0373ae0 con 0x5567a0370d90 2023-11-15T03:06:36.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.497+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v111'104 uv104 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a0370d90 2023-11-15T03:06:36.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.501+0000 7fe04e334d00 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5567a0374160 0x5567a0376550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.501+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0376c70 con 0x5567a0374160 2023-11-15T03:06:36.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.501+0000 7fe03a23b700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5567a0374160 0x5567a0376550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.501+0000 7fe03a23b700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5567a0374160 0x5567a0376550 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:36.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.505+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v111'72 uv71 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.505+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0374160 2023-11-15T03:06:36.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.505+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v111'73 uv73 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.505+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0374160 2023-11-15T03:06:36.517 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:06:36.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.509+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v111'74 uv74 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.509+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0374160 2023-11-15T03:06:36.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.513+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v111'75 uv75 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.513+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0374160 2023-11-15T03:06:36.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.517+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v111'99 uv98 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.517+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0374160 2023-11-15T03:06:36.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.521+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v111'100 uv100 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.521+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0374160 2023-11-15T03:06:36.529 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:06:36.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.521+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v111'101 uv101 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.521+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03780b0 con 0x5567a0374160 2023-11-15T03:06:36.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.525+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v111'102 uv102 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.525+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03780b0 con 0x5567a0374160 2023-11-15T03:06:36.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.529+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v111'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.529+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0374160 2023-11-15T03:06:36.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.533+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v111'104 uv104 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.533+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a03635b0 2023-11-15T03:06:36.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.537+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v111'182 uv176 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.537+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a03635b0 2023-11-15T03:06:36.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.541+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v111'183 uv183 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.541+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a03635b0 2023-11-15T03:06:36.550 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:06:36.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.545+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v111'184 uv184 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.545+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03780b0 con 0x5567a03635b0 2023-11-15T03:06:36.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.545+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v111'185 uv185 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.545+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03780b0 con 0x5567a03635b0 2023-11-15T03:06:36.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.549+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v111'186 uv186 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.549+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a03635b0 2023-11-15T03:06:36.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.553+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v111'187 uv187 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.553+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a03635b0 2023-11-15T03:06:36.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.553+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v111'188 uv181 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.553+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a03635b0 2023-11-15T03:06:36.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.557+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v111'189 uv189 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.561+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a03635b0 2023-11-15T03:06:36.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.561+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v111'190 uv190 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.569 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:06:36.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.561+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a03635b0 2023-11-15T03:06:36.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.565+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v111'191 uv191 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.565+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770e0 con 0x5567a03635b0 2023-11-15T03:06:36.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.569+0000 7fe042a3c700 1 -- 172.21.15.121:0/1395593081 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v111'192 uv192 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03404a610 con 0x5567a03635b0 2023-11-15T03:06:36.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.569+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0368930 2023-11-15T03:06:36.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.573+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v111'81 uv80 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0368930 2023-11-15T03:06:36.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.573+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0368930 2023-11-15T03:06:36.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.573+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v111'82 uv82 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0368930 2023-11-15T03:06:36.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.573+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0368930 2023-11-15T03:06:36.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.577+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v111'83 uv83 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0368930 2023-11-15T03:06:36.584 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:06:36.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.577+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0368930 2023-11-15T03:06:36.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.581+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v111'84 uv84 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0368930 2023-11-15T03:06:36.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.581+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03774e0 con 0x5567a0368930 2023-11-15T03:06:36.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.581+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v111'85 uv85 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0368930 2023-11-15T03:06:36.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.581+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03774e0 con 0x5567a0374160 2023-11-15T03:06:36.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.585+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v111'99 uv98 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.585+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0374160 2023-11-15T03:06:36.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.589+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v111'100 uv100 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.589+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0374160 2023-11-15T03:06:36.598 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:06:36.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.593+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v111'101 uv101 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.593+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770e0 con 0x5567a0374160 2023-11-15T03:06:36.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.593+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v111'102 uv102 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.593+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0374160 2023-11-15T03:06:36.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.597+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v111'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.597+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0374160 2023-11-15T03:06:36.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.601+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v111'104 uv104 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.601+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0374160 2023-11-15T03:06:36.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.605+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v111'82 uv81 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.605+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0374160 2023-11-15T03:06:36.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.605+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v111'83 uv83 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.609+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0374160 2023-11-15T03:06:36.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.609+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v111'84 uv84 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.616 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:06:36.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.609+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0374160 2023-11-15T03:06:36.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.613+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v111'85 uv85 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.613+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0377530 con 0x5567a0374160 2023-11-15T03:06:36.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.617+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v111'86 uv86 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.617+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0377530 con 0x5567a0374160 2023-11-15T03:06:36.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.617+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v111'81 uv80 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.617+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a03774e0 con 0x5567a0374160 2023-11-15T03:06:36.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.621+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v111'82 uv82 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.621+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a03770a0 con 0x5567a0374160 2023-11-15T03:06:36.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.625+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v111'83 uv83 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.632 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:06:36.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.625+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a0377530 con 0x5567a0374160 2023-11-15T03:06:36.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.629+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v111'84 uv84 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.629+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037a2a0 con 0x5567a0374160 2023-11-15T03:06:36.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.633+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v111'85 uv85 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.633+0000 7fe04e334d00 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5567a0377ee0 0x5567a037a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.633+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037a7c0 con 0x5567a0377ee0 2023-11-15T03:06:36.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.633+0000 7fe04323d700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5567a0377ee0 0x5567a037a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.633+0000 7fe04323d700 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5567a0377ee0 0x5567a037a2d0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:36.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.637+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v111'66 uv65 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a0377ee0 2023-11-15T03:06:36.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.637+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a037a7c0 con 0x5567a0377ee0 2023-11-15T03:06:36.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.641+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v111'67 uv67 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a0377ee0 2023-11-15T03:06:36.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.641+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037a7c0 con 0x5567a0377ee0 2023-11-15T03:06:36.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.645+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v111'68 uv68 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a0377ee0 2023-11-15T03:06:36.652 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:06:36.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.645+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037a7c0 con 0x5567a0377ee0 2023-11-15T03:06:36.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.649+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v111'69 uv69 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a0377ee0 2023-11-15T03:06:36.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.649+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037a7c0 con 0x5567a0374160 2023-11-15T03:06:36.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.649+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v111'99 uv98 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.649+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a037a7c0 con 0x5567a0374160 2023-11-15T03:06:36.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.653+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v111'100 uv100 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.653+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037a7c0 con 0x5567a0374160 2023-11-15T03:06:36.665 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:06:36.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.657+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v111'101 uv101 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.657+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037b7d0 con 0x5567a0374160 2023-11-15T03:06:36.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.661+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v111'102 uv102 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.661+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037b7d0 con 0x5567a0374160 2023-11-15T03:06:36.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.661+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v111'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.661+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037a7c0 con 0x5567a0374160 2023-11-15T03:06:36.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.665+0000 7fe03a23b700 1 -- 172.21.15.121:0/1395593081 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v111'104 uv104 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe02c0078e0 con 0x5567a0374160 2023-11-15T03:06:36.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.665+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037a7c0 con 0x5567a036d930 2023-11-15T03:06:36.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.665+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v111'84 uv83 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.669+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a037a7c0 con 0x5567a036d930 2023-11-15T03:06:36.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.669+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v111'85 uv85 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.669+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037a7c0 con 0x5567a036d930 2023-11-15T03:06:36.680 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:06:36.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.673+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v111'86 uv86 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.673+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037a7c0 con 0x5567a036d930 2023-11-15T03:06:36.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.677+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v111'87 uv87 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.677+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037a7c0 con 0x5567a036d930 2023-11-15T03:06:36.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.677+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v111'88 uv88 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.677+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037a7c0 con 0x5567a036d930 2023-11-15T03:06:36.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.681+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v111'196 uv188 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.681+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a037a7c0 con 0x5567a036d930 2023-11-15T03:06:36.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.685+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v111'197 uv197 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.685+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037a7c0 con 0x5567a036d930 2023-11-15T03:06:36.693 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:06:36.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.685+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v111'198 uv198 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.685+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037a7c0 con 0x5567a036d930 2023-11-15T03:06:36.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.689+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v111'199 uv199 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.689+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037ac10 con 0x5567a036d930 2023-11-15T03:06:36.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.693+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v111'200 uv200 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.693+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037ac10 con 0x5567a036d930 2023-11-15T03:06:36.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.697+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v111'201 uv195 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.697+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e111) v8 -- 0x5567a037abc0 con 0x5567a036d930 2023-11-15T03:06:36.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.701+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v111'202 uv202 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.701+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037a890 con 0x5567a036d930 2023-11-15T03:06:36.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.705+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v111'203 uv203 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.712 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:06:36.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.705+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037b8a0 con 0x5567a036d930 2023-11-15T03:06:36.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.709+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v111'204 uv204 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.709+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037b8a0 con 0x5567a036d930 2023-11-15T03:06:36.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.713+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v111'205 uv205 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.713+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037a840 con 0x5567a036d930 2023-11-15T03:06:36.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.717+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v111'206 uv206 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.717+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x5567a037ac20 con 0x5567a036d930 2023-11-15T03:06:36.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v111'207 uv207 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe03c00ac20 con 0x5567a036d930 2023-11-15T03:06:36.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5567a0368930 msgr2=0x5567a036ad20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5567a0368930 0x5567a036ad20 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5567a026d2b0 msgr2=0x5567a029c290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5567a026d2b0 0x5567a029c290 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5567a0370d90 msgr2=0x5567a0373180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5567a0370d90 0x5567a0373180 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5567a03635b0 msgr2=0x5567a03659a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5567a03635b0 0x5567a03659a0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5567a0377ee0 msgr2=0x5567a037a2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5567a0377ee0 0x5567a037a2d0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5567a036d930 msgr2=0x5567a036fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5567a036d930 0x5567a036fd20 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5567a0374160 msgr2=0x5567a0376550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5567a0374160 0x5567a0376550 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 reap_dead start 2023-11-15T03:06:36.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 reap_dead start 2023-11-15T03:06:36.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 reap_dead start 2023-11-15T03:06:36.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe024058ff0 msgr2=0x7fe02405b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe024058ff0 0x7fe02405b4b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x5567a025cc40 tx=0x7fe02c015000).stop 2023-11-15T03:06:36.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567a02b88d0 msgr2=0x5567a0355eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 --2- 172.21.15.121:0/1395593081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567a02b88d0 0x5567a0355eb0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x5567a02b0090 tx=0x7fe034018000).stop 2023-11-15T03:06:36.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04323d700 1 -- 172.21.15.121:0/1395593081 reap_dead start 2023-11-15T03:06:36.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 shutdown_connections 2023-11-15T03:06:36.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 >> 172.21.15.121:0/1395593081 conn(0x5567a02b68f0 msgr2=0x5567a026e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:36.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 shutdown_connections 2023-11-15T03:06:36.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.721+0000 7fe04e334d00 1 -- 172.21.15.121:0/1395593081 wait complete. 2023-11-15T03:06:36.736 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 200.00000001 - 2023-11-15T03:06:36.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.829+0000 7f5b8613dd00 1 Processor -- start 2023-11-15T03:06:36.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.829+0000 7f5b8613dd00 1 -- start start 2023-11-15T03:06:36.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b8613dd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cefe06920 0x559cefed2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b8613dd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cefe4dbe0 0x559cefe4af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b8613dd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cefedd130 0x559cefe4b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b8613dd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cefd7b220 con 0x559cefedd130 2023-11-15T03:06:36.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b8613dd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cefe33660 con 0x559cefe4dbe0 2023-11-15T03:06:36.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b8613dd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cefe06d00 con 0x559cefe06920 2023-11-15T03:06:36.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b7af1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cefedd130 0x559cefe4b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b7af1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cefedd130 0x559cefe4b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50676/0 (socket says 172.21.15.121:50676) 2023-11-15T03:06:36.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b7af1b700 1 -- 172.21.15.121:0/3198847607 learned_addr learned my addr 172.21.15.121:0/3198847607 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:36.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b7a71a700 1 --2- 172.21.15.121:0/3198847607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cefe06920 0x559cefed2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b71f19700 1 --2- 172.21.15.121:0/3198847607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cefe4dbe0 0x559cefe4af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b7af1b700 1 -- 172.21.15.121:0/3198847607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cefe06920 msgr2=0x559cefed2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:36.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b7af1b700 1 --2- 172.21.15.121:0/3198847607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cefe06920 0x559cefed2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b7af1b700 1 -- 172.21.15.121:0/3198847607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cefe4dbe0 msgr2=0x559cefe4af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:36.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b7af1b700 1 --2- 172.21.15.121:0/3198847607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cefe4dbe0 0x559cefe4af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b7af1b700 1 -- 172.21.15.121:0/3198847607 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7400a0e0 con 0x559cefedd130 2023-11-15T03:06:36.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b71f19700 1 --2- 172.21.15.121:0/3198847607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cefe4dbe0 0x559cefe4af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:36.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b7af1b700 1 --2- 172.21.15.121:0/3198847607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cefedd130 0x559cefe4b430 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7f5b7400ab90 tx=0x7f5b74015000).ready entity=mon.0 client_cookie=16dbe29b20852ae7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:36.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b79718700 1 -- 172.21.15.121:0/3198847607 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7400d470 con 0x559cefedd130 2023-11-15T03:06:36.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b79718700 1 -- 172.21.15.121:0/3198847607 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7400dd00 con 0x559cefedd130 2023-11-15T03:06:36.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b79718700 1 -- 172.21.15.121:0/3198847607 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7400d970 con 0x559cefedd130 2023-11-15T03:06:36.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b8613dd00 1 -- 172.21.15.121:0/3198847607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cefedd130 msgr2=0x559cefe4b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b8613dd00 1 --2- 172.21.15.121:0/3198847607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cefedd130 0x559cefe4b430 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7f5b7400ab90 tx=0x7f5b74015000).stop 2023-11-15T03:06:36.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b8613dd00 1 -- 172.21.15.121:0/3198847607 shutdown_connections 2023-11-15T03:06:36.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b8613dd00 1 --2- 172.21.15.121:0/3198847607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cefedd130 0x559cefe4b430 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b8613dd00 1 --2- 172.21.15.121:0/3198847607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cefe4dbe0 0x559cefe4af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b8613dd00 1 --2- 172.21.15.121:0/3198847607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cefe06920 0x559cefed2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b8613dd00 1 -- 172.21.15.121:0/3198847607 >> 172.21.15.121:0/3198847607 conn(0x559cefe4c8f0 msgr2=0x559cefe32530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:36.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b8613dd00 1 -- 172.21.15.121:0/3198847607 shutdown_connections 2023-11-15T03:06:36.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.833+0000 7f5b8613dd00 1 -- 172.21.15.121:0/3198847607 wait complete. 2023-11-15T03:06:36.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.837+0000 7f5b8613dd00 1 Processor -- start 2023-11-15T03:06:36.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.837+0000 7f5b8613dd00 1 -- start start 2023-11-15T03:06:36.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.837+0000 7f5b8613dd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cefe06920 0x559cefed0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.837+0000 7f5b8613dd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cefe4dbe0 0x559cefe320b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.837+0000 7f5b8613dd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cefedd130 0x559cefe325a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.837+0000 7f5b8613dd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cefe32a90 con 0x559cefedd130 2023-11-15T03:06:36.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.837+0000 7f5b8613dd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cefed88f0 con 0x559cefe4dbe0 2023-11-15T03:06:36.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.837+0000 7f5b8613dd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cefed4a90 con 0x559cefe06920 2023-11-15T03:06:36.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.837+0000 7f5b7af1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cefedd130 0x559cefe325a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.837+0000 7f5b7af1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cefedd130 0x559cefe325a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50684/0 (socket says 172.21.15.121:50684) 2023-11-15T03:06:36.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.837+0000 7f5b7af1b700 1 -- 172.21.15.121:0/2638903903 learned_addr learned my addr 172.21.15.121:0/2638903903 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:36.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.837+0000 7f5b7a71a700 1 --2- 172.21.15.121:0/2638903903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cefe06920 0x559cefed0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.837+0000 7f5b71f19700 1 --2- 172.21.15.121:0/2638903903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cefe4dbe0 0x559cefe320b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.837+0000 7f5b7af1b700 1 -- 172.21.15.121:0/2638903903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cefe06920 msgr2=0x559cefed0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.837+0000 7f5b7af1b700 1 --2- 172.21.15.121:0/2638903903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559cefe06920 0x559cefed0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.837+0000 7f5b7af1b700 1 -- 172.21.15.121:0/2638903903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cefe4dbe0 msgr2=0x559cefe320b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.837+0000 7f5b7af1b700 1 --2- 172.21.15.121:0/2638903903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cefe4dbe0 0x559cefe320b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.837+0000 7f5b7af1b700 1 -- 172.21.15.121:0/2638903903 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cefda4a20 con 0x559cefedd130 2023-11-15T03:06:36.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.841+0000 7f5b7af1b700 1 --2- 172.21.15.121:0/2638903903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cefedd130 0x559cefe325a0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7f5b7400a470 tx=0x7f5b7401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:36.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.841+0000 7f5b737fe700 1 -- 172.21.15.121:0/2638903903 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7400de70 con 0x559cefedd130 2023-11-15T03:06:36.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.841+0000 7f5b737fe700 1 -- 172.21.15.121:0/2638903903 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7400d950 con 0x559cefedd130 2023-11-15T03:06:36.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.841+0000 7f5b8613dd00 1 -- 172.21.15.121:0/2638903903 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559cefda4770 con 0x559cefedd130 2023-11-15T03:06:36.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.841+0000 7f5b737fe700 1 -- 172.21.15.121:0/2638903903 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b74024660 con 0x559cefedd130 2023-11-15T03:06:36.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.841+0000 7f5b8613dd00 1 -- 172.21.15.121:0/2638903903 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559cefd73a70 con 0x559cefedd130 2023-11-15T03:06:36.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.841+0000 7f5b737fe700 1 -- 172.21.15.121:0/2638903903 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5b74021030 con 0x559cefedd130 2023-11-15T03:06:36.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.845+0000 7f5b737fe700 1 --2- 172.21.15.121:0/2638903903 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b5c058ff0 0x7f5b5c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.845+0000 7f5b737fe700 1 -- 172.21.15.121:0/2638903903 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 7728+0+0 (secure 0 0 0) 0x7f5b74075750 con 0x559cefedd130 2023-11-15T03:06:36.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.845+0000 7f5b7a71a700 1 --2- 172.21.15.121:0/2638903903 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b5c058ff0 0x7f5b5c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.845+0000 7f5b8613dd00 1 --2- 172.21.15.121:0/2638903903 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x559cefed7000 0x559cefed5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:36.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.845+0000 7f5b8613dd00 1 -- 172.21.15.121:0/2638903903 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.34 13:2e2fa760:::200.00000001:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected e111) v8 -- 0x559cefe03d50 con 0x559cefed7000 2023-11-15T03:06:36.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.845+0000 7f5b7a71a700 1 --2- 172.21.15.121:0/2638903903 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b5c058ff0 0x7f5b5c05b4b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f5b6c004170 tx=0x7f5b6c0018e0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:36.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.845+0000 7f5b71f19700 1 --2- 172.21.15.121:0/2638903903 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x559cefed7000 0x559cefed5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:36.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.845+0000 7f5b71f19700 1 --2- 172.21.15.121:0/2638903903 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x559cefed7000 0x559cefed5980 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:36.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.849+0000 7f5b71f19700 1 -- 172.21.15.121:0/2638903903 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 200.00000001 [writefull 0~80] v111'215 uv215 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b64009ca0 con 0x559cefed7000 2023-11-15T03:06:36.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.849+0000 7f5b8613dd00 1 -- 172.21.15.121:0/2638903903 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x559cefed7000 msgr2=0x559cefed5980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.849+0000 7f5b8613dd00 1 --2- 172.21.15.121:0/2638903903 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x559cefed7000 0x559cefed5980 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:36.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.849+0000 7f5b8613dd00 1 -- 172.21.15.121:0/2638903903 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b5c058ff0 msgr2=0x7f5b5c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.849+0000 7f5b8613dd00 1 --2- 172.21.15.121:0/2638903903 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b5c058ff0 0x7f5b5c05b4b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f5b6c004170 tx=0x7f5b6c0018e0).stop 2023-11-15T03:06:36.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.849+0000 7f5b8613dd00 1 -- 172.21.15.121:0/2638903903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cefedd130 msgr2=0x559cefe325a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:36.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.849+0000 7f5b8613dd00 1 --2- 172.21.15.121:0/2638903903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cefedd130 0x559cefe325a0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7f5b7400a470 tx=0x7f5b7401a000).stop 2023-11-15T03:06:36.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.849+0000 7f5b7af1b700 1 -- 172.21.15.121:0/2638903903 reap_dead start 2023-11-15T03:06:36.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.849+0000 7f5b8613dd00 1 -- 172.21.15.121:0/2638903903 shutdown_connections 2023-11-15T03:06:36.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.849+0000 7f5b8613dd00 1 -- 172.21.15.121:0/2638903903 >> 172.21.15.121:0/2638903903 conn(0x559cefe4c8f0 msgr2=0x559cefe02c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:36.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.849+0000 7f5b8613dd00 1 -- 172.21.15.121:0/2638903903 shutdown_connections 2023-11-15T03:06:36.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:36.849+0000 7f5b8613dd00 1 -- 172.21.15.121:0/2638903903 wait complete. 2023-11-15T03:06:36.864 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:06:37.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.077+0000 7f10bd2db700 1 Processor -- start 2023-11-15T03:06:37.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.077+0000 7f10bd2db700 1 -- start start 2023-11-15T03:06:37.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.077+0000 7f10bd2db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f10b8068c10 0x7f10b8068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:37.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.077+0000 7f10bd2db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b8063230 0x7f10b8063670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:37.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.077+0000 7f10bd2db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10b8063f40 con 0x7f10b8153740 2023-11-15T03:06:37.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.077+0000 7f10bd2db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b8064220 con 0x7f10b8068c10 2023-11-15T03:06:37.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.077+0000 7f10bd2db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b8064500 con 0x7f10b8063230 2023-11-15T03:06:37.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.077+0000 7f10b67fc700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f10b8153740 0x7f10b8153b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40510/0 (socket says 172.21.15.121:40510) 2023-11-15T03:06:37.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10b67fc700 1 -- 172.21.15.121:0/2729137993 learned_addr learned my addr 172.21.15.121:0/2729137993 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:37.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.077+0000 7f10b6ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f10b8068c10 0x7f10b8068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:37.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10b77fe700 1 --2- 172.21.15.121:0/2729137993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b8063230 0x7f10b8063670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:37.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10b5ffb700 1 -- 172.21.15.121:0/2729137993 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942930458 0 0) 0x7f10b8063f40 con 0x7f10b8153740 2023-11-15T03:06:37.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10b5ffb700 1 -- 172.21.15.121:0/2729137993 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10a00033f0 con 0x7f10b8153740 2023-11-15T03:06:37.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10b6ffd700 1 -- 172.21.15.121:0/2729137993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b8063230 msgr2=0x7f10b8063670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:37.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10b6ffd700 1 --2- 172.21.15.121:0/2729137993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b8063230 0x7f10b8063670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:37.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10b6ffd700 1 -- 172.21.15.121:0/2729137993 >> v1:172.21.15.121:6789/0 conn(0x7f10b8153740 legacy=0x7f10b8153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:37.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10b6ffd700 1 -- 172.21.15.121:0/2729137993 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b80647e0 con 0x7f10b8068c10 2023-11-15T03:06:37.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10b6ffd700 1 --2- 172.21.15.121:0/2729137993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b8068c10 0x7f10b8068ff0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7f10a8009760 tx=0x7f10a8009060).ready entity=mon.0 client_cookie=f4d6cbd33067f2eb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:37.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10b5ffb700 1 -- 172.21.15.121:0/2729137993 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a800d4a0 con 0x7f10b8068c10 2023-11-15T03:06:37.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10b5ffb700 1 -- 172.21.15.121:0/2729137993 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a800d600 con 0x7f10b8068c10 2023-11-15T03:06:37.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10b5ffb700 1 -- 172.21.15.121:0/2729137993 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a800d980 con 0x7f10b8068c10 2023-11-15T03:06:37.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10bd2db700 1 -- 172.21.15.121:0/2729137993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b8068c10 msgr2=0x7f10b8068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:37.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10bd2db700 1 --2- 172.21.15.121:0/2729137993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b8068c10 0x7f10b8068ff0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7f10a8009760 tx=0x7f10a8009060).stop 2023-11-15T03:06:37.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10bd2db700 1 -- 172.21.15.121:0/2729137993 shutdown_connections 2023-11-15T03:06:37.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10bd2db700 1 --2- 172.21.15.121:0/2729137993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b8063230 0x7f10b8063670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:37.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10bd2db700 1 --2- 172.21.15.121:0/2729137993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b8068c10 0x7f10b8068ff0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:37.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10bd2db700 1 -- 172.21.15.121:0/2729137993 >> 172.21.15.121:0/2729137993 conn(0x7f10b80c4540 msgr2=0x7f10b8066180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:37.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10bd2db700 1 -- 172.21.15.121:0/2729137993 shutdown_connections 2023-11-15T03:06:37.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.081+0000 7f10bd2db700 1 -- 172.21.15.121:0/2729137993 wait complete. 2023-11-15T03:06:37.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.085+0000 7f10bd2db700 1 Processor -- start 2023-11-15T03:06:37.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.085+0000 7f10bd2db700 1 -- start start 2023-11-15T03:06:37.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.085+0000 7f10bd2db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b8063230 0x7f10b80bf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:37.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.085+0000 7f10bd2db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b8068c10 0x7f10b80aaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:37.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.085+0000 7f10bd2db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10b8153740 0x7f10b814fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:37.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.085+0000 7f10bd2db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b8155d60 con 0x7f10b8063230 2023-11-15T03:06:37.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.085+0000 7f10bd2db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b80ab320 con 0x7f10b8153740 2023-11-15T03:06:37.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.085+0000 7f10bd2db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b80ab600 con 0x7f10b8068c10 2023-11-15T03:06:37.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.085+0000 7f10b6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b8063230 0x7f10b80bf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:37.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.085+0000 7f10b6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b8063230 0x7f10b80bf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50698/0 (socket says 172.21.15.121:50698) 2023-11-15T03:06:37.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.085+0000 7f10b6ffd700 1 -- 172.21.15.121:0/437684648 learned_addr learned my addr 172.21.15.121:0/437684648 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:37.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.085+0000 7f10b67fc700 1 --2- 172.21.15.121:0/437684648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b8068c10 0x7f10b80aaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:37.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.089+0000 7f10b6ffd700 1 -- 172.21.15.121:0/437684648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b8068c10 msgr2=0x7f10b80aaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:37.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.089+0000 7f10b6ffd700 1 --2- 172.21.15.121:0/437684648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b8068c10 0x7f10b80aaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:37.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.089+0000 7f10b6ffd700 1 -- 172.21.15.121:0/437684648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10b8153740 msgr2=0x7f10b814fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:37.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.089+0000 7f10b77fe700 1 --2- 172.21.15.121:0/437684648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10b8153740 0x7f10b814fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:37.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.089+0000 7f10b6ffd700 1 --2- 172.21.15.121:0/437684648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10b8153740 0x7f10b814fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:37.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.089+0000 7f10b6ffd700 1 -- 172.21.15.121:0/437684648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a80092a0 con 0x7f10b8063230 2023-11-15T03:06:37.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.089+0000 7f10b67fc700 1 --2- 172.21.15.121:0/437684648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b8068c10 0x7f10b80aaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:37.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.089+0000 7f10b77fe700 1 --2- 172.21.15.121:0/437684648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10b8153740 0x7f10b814fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:37.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.089+0000 7f10b6ffd700 1 --2- 172.21.15.121:0/437684648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b8063230 0x7f10b80bf600 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f10a8009d50 tx=0x7f10a8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:37.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.089+0000 7f109effd700 1 -- 172.21.15.121:0/437684648 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a800dd40 con 0x7f10b8063230 2023-11-15T03:06:37.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.089+0000 7f109effd700 1 -- 172.21.15.121:0/437684648 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a800d770 con 0x7f10b8063230 2023-11-15T03:06:37.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.089+0000 7f109effd700 1 -- 172.21.15.121:0/437684648 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a800dea0 con 0x7f10b8063230 2023-11-15T03:06:37.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.089+0000 7f10bd2db700 1 -- 172.21.15.121:0/437684648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b80ab8e0 con 0x7f10b8063230 2023-11-15T03:06:37.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.089+0000 7f10bd2db700 1 -- 172.21.15.121:0/437684648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b8155980 con 0x7f10b8063230 2023-11-15T03:06:37.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.089+0000 7f10bd2db700 1 -- 172.21.15.121:0/437684648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b8047060 con 0x7f10b8063230 2023-11-15T03:06:37.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.089+0000 7f109effd700 1 -- 172.21.15.121:0/437684648 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f10a8018070 con 0x7f10b8063230 2023-11-15T03:06:37.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.093+0000 7f109effd700 1 --2- 172.21.15.121:0/437684648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f10a405a510 0x7f10a405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:37.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.093+0000 7f109effd700 1 -- 172.21.15.121:0/437684648 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 7728+0+0 (secure 0 0 0) 0x7f10a8076720 con 0x7f10b8063230 2023-11-15T03:06:37.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.093+0000 7f10b67fc700 1 --2- 172.21.15.121:0/437684648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f10a405a510 0x7f10a405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:37.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.093+0000 7f10b67fc700 1 --2- 172.21.15.121:0/437684648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f10a405a510 0x7f10a405c9d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f10b81588d0 tx=0x7f10ac009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:37.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.101+0000 7f109effd700 1 -- 172.21.15.121:0/437684648 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f10a80498a0 con 0x7f10b8063230 2023-11-15T03:06:37.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:37.305+0000 7f10bd2db700 1 -- 172.21.15.121:0/437684648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f10b8064ac0 con 0x7f10b8063230 2023-11-15T03:06:38.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.097+0000 7f109effd700 1 -- 172.21.15.121:0/437684648 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v192) v1 ==== 162+0+0 (secure 0 0 0) 0x7f10a804cec0 con 0x7f10b8063230 2023-11-15T03:06:38.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.105+0000 7f10bd2db700 1 -- 172.21.15.121:0/437684648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f10a405a510 msgr2=0x7f10a405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:38.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.105+0000 7f10bd2db700 1 --2- 172.21.15.121:0/437684648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f10a405a510 0x7f10a405c9d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f10b81588d0 tx=0x7f10ac009040).stop 2023-11-15T03:06:38.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.105+0000 7f10bd2db700 1 -- 172.21.15.121:0/437684648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b8063230 msgr2=0x7f10b80bf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:38.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.105+0000 7f10bd2db700 1 --2- 172.21.15.121:0/437684648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b8063230 0x7f10b80bf600 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f10a8009d50 tx=0x7f10a8016000).stop 2023-11-15T03:06:38.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.105+0000 7f10bd2db700 1 -- 172.21.15.121:0/437684648 shutdown_connections 2023-11-15T03:06:38.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.105+0000 7f10bd2db700 1 --2- 172.21.15.121:0/437684648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f10a405a510 0x7f10a405c9d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:38.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.105+0000 7f10bd2db700 1 --2- 172.21.15.121:0/437684648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10b8153740 0x7f10b814fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:38.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.105+0000 7f10bd2db700 1 --2- 172.21.15.121:0/437684648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b8068c10 0x7f10b80aaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:38.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.105+0000 7f10bd2db700 1 --2- 172.21.15.121:0/437684648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b8063230 0x7f10b80bf600 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:38.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.105+0000 7f10bd2db700 1 -- 172.21.15.121:0/437684648 >> 172.21.15.121:0/437684648 conn(0x7f10b80c4540 msgr2=0x7f10b8152b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:38.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.105+0000 7f10bd2db700 1 -- 172.21.15.121:0/437684648 shutdown_connections 2023-11-15T03:06:38.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.105+0000 7f10bd2db700 1 -- 172.21.15.121:0/437684648 wait complete. 2023-11-15T03:06:38.113 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:06:38.115 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:06:38.109+0000 7f7fd43e7700 -1 log_channel(cluster) log [ERR] : missing journal object 2023-11-15T03:06:38.132 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:38.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.293+0000 7fb17759e700 1 Processor -- start 2023-11-15T03:06:38.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.293+0000 7fb17759e700 1 -- start start 2023-11-15T03:06:38.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.293+0000 7fb17759e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb1780a9830 0x7fb1780a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:38.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.293+0000 7fb17759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1780c30e0 0x7fb1780c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:38.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.293+0000 7fb17759e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1780657f0 con 0x7fb178151870 2023-11-15T03:06:38.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.293+0000 7fb17759e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb178065ad0 con 0x7fb1780a9830 2023-11-15T03:06:38.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.293+0000 7fb17759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1780c3a60 con 0x7fb1780c30e0 2023-11-15T03:06:38.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.293+0000 7fb175d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb1780a9830 0x7fb1780a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:38.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.293+0000 7fb176d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1780c30e0 0x7fb1780c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:38.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.293+0000 7fb175d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb1780a9830 0x7fb1780a9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:38862/0 (socket says 172.21.15.121:38862) 2023-11-15T03:06:38.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.297+0000 7fb175d9b700 1 -- 172.21.15.121:0/164676707 learned_addr learned my addr 172.21.15.121:0/164676707 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:38.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.293+0000 7fb17659c700 1 --1- 172.21.15.121:0/164676707 >> v1:172.21.15.190:6789/0 conn(0x7fb178151870 0x7fb178151c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:34756/0 (socket says 172.21.15.121:34756) 2023-11-15T03:06:38.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.297+0000 7fb175d9b700 1 -- 172.21.15.121:0/164676707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1780c30e0 msgr2=0x7fb1780c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:38.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.297+0000 7fb175d9b700 1 --2- 172.21.15.121:0/164676707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1780c30e0 0x7fb1780c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:38.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.297+0000 7fb175d9b700 1 -- 172.21.15.121:0/164676707 >> v1:172.21.15.190:6789/0 conn(0x7fb178151870 legacy=0x7fb178151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:38.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.297+0000 7fb175d9b700 1 -- 172.21.15.121:0/164676707 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1780aa260 con 0x7fb1780a9830 2023-11-15T03:06:38.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.297+0000 7fb175d9b700 1 --2- 172.21.15.121:0/164676707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1780a9830 0x7fb1780a9c70 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fb16c006fd0 tx=0x7fb16c0090f0).ready entity=mon.1 client_cookie=57c64a63d765143 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:38.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.297+0000 7fb17559a700 1 -- 172.21.15.121:0/164676707 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c00b580 con 0x7fb1780a9830 2023-11-15T03:06:38.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.297+0000 7fb17559a700 1 -- 172.21.15.121:0/164676707 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16c00b6e0 con 0x7fb1780a9830 2023-11-15T03:06:38.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.297+0000 7fb17559a700 1 -- 172.21.15.121:0/164676707 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c00ba60 con 0x7fb1780a9830 2023-11-15T03:06:38.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.297+0000 7fb17759e700 1 -- 172.21.15.121:0/164676707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1780a9830 msgr2=0x7fb1780a9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:38.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.297+0000 7fb17759e700 1 --2- 172.21.15.121:0/164676707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1780a9830 0x7fb1780a9c70 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fb16c006fd0 tx=0x7fb16c0090f0).stop 2023-11-15T03:06:38.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.297+0000 7fb17759e700 1 -- 172.21.15.121:0/164676707 shutdown_connections 2023-11-15T03:06:38.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.297+0000 7fb17759e700 1 --2- 172.21.15.121:0/164676707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1780c30e0 0x7fb1780c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:38.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.297+0000 7fb17759e700 1 --2- 172.21.15.121:0/164676707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1780a9830 0x7fb1780a9c70 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:38.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.297+0000 7fb17759e700 1 -- 172.21.15.121:0/164676707 >> 172.21.15.121:0/164676707 conn(0x7fb178062190 msgr2=0x7fb178062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:38.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.297+0000 7fb17759e700 1 -- 172.21.15.121:0/164676707 shutdown_connections 2023-11-15T03:06:38.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.301+0000 7fb17759e700 1 -- 172.21.15.121:0/164676707 wait complete. 2023-11-15T03:06:38.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.301+0000 7fb17759e700 1 Processor -- start 2023-11-15T03:06:38.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.301+0000 7fb17759e700 1 -- start start 2023-11-15T03:06:38.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.305+0000 7fb17759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1780a9830 0x7fb17814d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:38.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.305+0000 7fb17759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1780c30e0 0x7fb17809f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:38.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.305+0000 7fb17759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb178151870 0x7fb17809f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:38.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.305+0000 7fb17759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1780bd8a0 con 0x7fb1780a9830 2023-11-15T03:06:38.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.305+0000 7fb17759e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1780bdb80 con 0x7fb178151870 2023-11-15T03:06:38.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.305+0000 7fb17759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1780bde60 con 0x7fb1780c30e0 2023-11-15T03:06:38.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.305+0000 7fb17659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1780a9830 0x7fb17814d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:38.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.305+0000 7fb17659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1780a9830 0x7fb17814d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50706/0 (socket says 172.21.15.121:50706) 2023-11-15T03:06:38.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.305+0000 7fb17659c700 1 -- 172.21.15.121:0/3735636808 learned_addr learned my addr 172.21.15.121:0/3735636808 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:38.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.305+0000 7fb175d9b700 1 --2- 172.21.15.121:0/3735636808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1780c30e0 0x7fb17809f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:38.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.305+0000 7fb176d9d700 1 --2- 172.21.15.121:0/3735636808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb178151870 0x7fb17809f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:38.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.305+0000 7fb176d9d700 1 -- 172.21.15.121:0/3735636808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1780c30e0 msgr2=0x7fb17809f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:38.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.305+0000 7fb176d9d700 1 --2- 172.21.15.121:0/3735636808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1780c30e0 0x7fb17809f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:38.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.305+0000 7fb176d9d700 1 -- 172.21.15.121:0/3735636808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1780a9830 msgr2=0x7fb17814d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:38.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.305+0000 7fb176d9d700 1 --2- 172.21.15.121:0/3735636808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1780a9830 0x7fb17814d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:38.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.305+0000 7fb176d9d700 1 -- 172.21.15.121:0/3735636808 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16c009370 con 0x7fb178151870 2023-11-15T03:06:38.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.305+0000 7fb176d9d700 1 --2- 172.21.15.121:0/3735636808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb178151870 0x7fb17809f9a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fb1700124f0 tx=0x7fb17000f040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:38.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.309+0000 7fb1667fc700 1 -- 172.21.15.121:0/3735636808 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1700100c0 con 0x7fb178151870 2023-11-15T03:06:38.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.309+0000 7fb1667fc700 1 -- 172.21.15.121:0/3735636808 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb170013040 con 0x7fb178151870 2023-11-15T03:06:38.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.309+0000 7fb1667fc700 1 -- 172.21.15.121:0/3735636808 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17000d300 con 0x7fb178151870 2023-11-15T03:06:38.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.309+0000 7fb17759e700 1 -- 172.21.15.121:0/3735636808 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1780be140 con 0x7fb178151870 2023-11-15T03:06:38.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.309+0000 7fb17759e700 1 -- 172.21.15.121:0/3735636808 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1780c0370 con 0x7fb178151870 2023-11-15T03:06:38.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.309+0000 7fb17759e700 1 -- 172.21.15.121:0/3735636808 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1780c0a80 con 0x7fb178151870 2023-11-15T03:06:38.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.309+0000 7fb1667fc700 1 -- 172.21.15.121:0/3735636808 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb17000b750 con 0x7fb178151870 2023-11-15T03:06:38.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.309+0000 7fb1667fc700 1 --2- 172.21.15.121:0/3735636808 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb15805a510 0x7fb15805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:38.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.313+0000 7fb1667fc700 1 -- 172.21.15.121:0/3735636808 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7814+0+0 (secure 0 0 0) 0x7fb1700791b0 con 0x7fb178151870 2023-11-15T03:06:38.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.313+0000 7fb17659c700 1 --2- 172.21.15.121:0/3735636808 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb15805a510 0x7fb15805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:38.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.313+0000 7fb17659c700 1 --2- 172.21.15.121:0/3735636808 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb15805a510 0x7fb15805c9d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7fb168007fd0 tx=0x7fb168007df0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:38.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.321+0000 7fb1667fc700 1 -- 172.21.15.121:0/3735636808 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb1780c0a80 con 0x7fb178151870 2023-11-15T03:06:38.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.517+0000 7fb17759e700 1 -- 172.21.15.121:0/3735636808 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1780c0a80 con 0x7fb178151870 2023-11-15T03:06:38.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.521+0000 7fb1667fc700 1 -- 172.21.15.121:0/3735636808 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) v1 ==== 95+0+4502 (secure 0 0 0) 0x7fb1780c0a80 con 0x7fb178151870 2023-11-15T03:06:38.526 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:38.526 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":192,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":192,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:38.091498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8724},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8724":{"gid":8724,"name":"a","rank":0,"incarnation":192,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/909619389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":909619389},{"type":"v1","addr":"172.21.15.121:6836","nonce":909619389}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:38.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.521+0000 7fb17759e700 1 -- 172.21.15.121:0/3735636808 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb15805a510 msgr2=0x7fb15805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:38.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.521+0000 7fb17759e700 1 --2- 172.21.15.121:0/3735636808 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb15805a510 0x7fb15805c9d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7fb168007fd0 tx=0x7fb168007df0).stop 2023-11-15T03:06:38.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.521+0000 7fb17759e700 1 -- 172.21.15.121:0/3735636808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb178151870 msgr2=0x7fb17809f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:38.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.521+0000 7fb17759e700 1 --2- 172.21.15.121:0/3735636808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb178151870 0x7fb17809f9a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fb1700124f0 tx=0x7fb17000f040).stop 2023-11-15T03:06:38.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.521+0000 7fb17759e700 1 -- 172.21.15.121:0/3735636808 shutdown_connections 2023-11-15T03:06:38.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.521+0000 7fb17759e700 1 --2- 172.21.15.121:0/3735636808 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb15805a510 0x7fb15805c9d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:38.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.521+0000 7fb17759e700 1 --2- 172.21.15.121:0/3735636808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb178151870 0x7fb17809f9a0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:38.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.521+0000 7fb17759e700 1 --2- 172.21.15.121:0/3735636808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1780c30e0 0x7fb17809f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:38.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.521+0000 7fb17759e700 1 --2- 172.21.15.121:0/3735636808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1780a9830 0x7fb17814d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:38.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.521+0000 7fb17759e700 1 -- 172.21.15.121:0/3735636808 >> 172.21.15.121:0/3735636808 conn(0x7fb178062190 msgr2=0x7fb1780c0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:38.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.521+0000 7fb17759e700 1 -- 172.21.15.121:0/3735636808 shutdown_connections 2023-11-15T03:06:38.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:38.521+0000 7fb17759e700 1 -- 172.21.15.121:0/3735636808 wait complete. 2023-11-15T03:06:38.531 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 192 2023-11-15T03:06:38.551 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:06:39.108 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:06:39.101+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:06:39.108 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:06:39.101+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:06:39.121 INFO:tasks.ceph.mds.a.smithi121.stderr: -17> 2023-11-15T03:06:38.109+0000 7f7fd43e7700 -1 log_channel(cluster) log [ERR] : missing journal object 2023-11-15T03:06:39.159 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:06:41.828 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:06:41.835 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:06:43.552 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:43.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.713+0000 7fdec941b700 1 Processor -- start 2023-11-15T03:06:43.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.717+0000 7fdec941b700 1 -- start start 2023-11-15T03:06:43.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.717+0000 7fdec941b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdec414af80 0x7fdec40c1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:43.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.717+0000 7fdec941b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdec40c6640 0x7fdec40c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:43.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.717+0000 7fdec941b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec4152000 con 0x7fdec40c1bf0 2023-11-15T03:06:43.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.717+0000 7fdec941b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec40c1270 con 0x7fdec40c6640 2023-11-15T03:06:43.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.717+0000 7fdec941b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec40c6fc0 con 0x7fdec414af80 2023-11-15T03:06:43.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.717+0000 7fdec37fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdec40c6640 0x7fdec40c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:43.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.717+0000 7fdec2ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdec414af80 0x7fdec40c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:43.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.717+0000 7fdec37fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdec40c6640 0x7fdec40c6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50728/0 (socket says 172.21.15.121:50728) 2023-11-15T03:06:43.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.717+0000 7fdec37fe700 1 -- 172.21.15.121:0/1657821778 learned_addr learned my addr 172.21.15.121:0/1657821778 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:43.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.717+0000 7fdebbfff700 1 --1- 172.21.15.121:0/1657821778 >> v1:172.21.15.190:6789/0 conn(0x7fdec40c1bf0 0x7fdec4150ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:34768/0 (socket says 172.21.15.121:34768) 2023-11-15T03:06:43.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.717+0000 7fdec2ffd700 1 -- 172.21.15.121:0/1657821778 >> v2:172.21.15.121:3300/0 conn(0x7fdec40c6640 msgr2=0x7fdec40c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:43.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.717+0000 7fdec2ffd700 1 --2- 172.21.15.121:0/1657821778 >> v2:172.21.15.121:3300/0 conn(0x7fdec40c6640 0x7fdec40c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:43.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.717+0000 7fdec2ffd700 1 -- 172.21.15.121:0/1657821778 >> v1:172.21.15.190:6789/0 conn(0x7fdec40c1bf0 legacy=0x7fdec4150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:43.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.717+0000 7fdec2ffd700 1 -- 172.21.15.121:0/1657821778 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec4151a40 con 0x7fdec414af80 2023-11-15T03:06:43.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.721+0000 7fdec2ffd700 1 --2- 172.21.15.121:0/1657821778 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdec414af80 0x7fdec40c1700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fdeb400ba00 tx=0x7fdeb400b300).ready entity=mon.2 client_cookie=12f74e9888b516b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:43.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.721+0000 7fdec27fc700 1 -- 172.21.15.121:0/1657821778 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb400da70 con 0x7fdec414af80 2023-11-15T03:06:43.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.721+0000 7fdec27fc700 1 -- 172.21.15.121:0/1657821778 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb400dbd0 con 0x7fdec414af80 2023-11-15T03:06:43.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.721+0000 7fdec27fc700 1 -- 172.21.15.121:0/1657821778 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4010470 con 0x7fdec414af80 2023-11-15T03:06:43.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.721+0000 7fdec941b700 1 -- 172.21.15.121:0/1657821778 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdec414af80 msgr2=0x7fdec40c1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:43.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.721+0000 7fdec941b700 1 --2- 172.21.15.121:0/1657821778 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdec414af80 0x7fdec40c1700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fdeb400ba00 tx=0x7fdeb400b300).stop 2023-11-15T03:06:43.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.721+0000 7fdec941b700 1 -- 172.21.15.121:0/1657821778 shutdown_connections 2023-11-15T03:06:43.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.721+0000 7fdec941b700 1 --2- 172.21.15.121:0/1657821778 >> v2:172.21.15.121:3300/0 conn(0x7fdec40c6640 0x7fdec40c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:43.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.721+0000 7fdec941b700 1 --2- 172.21.15.121:0/1657821778 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdec414af80 0x7fdec40c1700 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:43.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.721+0000 7fdec941b700 1 -- 172.21.15.121:0/1657821778 >> 172.21.15.121:0/1657821778 conn(0x7fdec4062190 msgr2=0x7fdec40625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:43.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.721+0000 7fdec941b700 1 -- 172.21.15.121:0/1657821778 shutdown_connections 2023-11-15T03:06:43.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.721+0000 7fdec941b700 1 -- 172.21.15.121:0/1657821778 wait complete. 2023-11-15T03:06:43.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdec941b700 1 Processor -- start 2023-11-15T03:06:43.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdec941b700 1 -- start start 2023-11-15T03:06:43.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdec941b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdec40c1bf0 0x7fdec4148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:43.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdec941b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdec40c6640 0x7fdec4152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:43.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdec941b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdec414af80 0x7fdec414c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:43.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdec941b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec4165270 con 0x7fdec40c6640 2023-11-15T03:06:43.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdec941b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec414b360 con 0x7fdec40c1bf0 2023-11-15T03:06:43.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdec941b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec414b640 con 0x7fdec414af80 2023-11-15T03:06:43.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdebbfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdec40c6640 0x7fdec4152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:43.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdebbfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdec40c6640 0x7fdec4152970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50732/0 (socket says 172.21.15.121:50732) 2023-11-15T03:06:43.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdebbfff700 1 -- 172.21.15.121:0/4143797853 learned_addr learned my addr 172.21.15.121:0/4143797853 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:43.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdec2ffd700 1 --2- 172.21.15.121:0/4143797853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdec40c1bf0 0x7fdec4148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:43.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdec37fe700 1 --2- 172.21.15.121:0/4143797853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdec414af80 0x7fdec414c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:43.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdebbfff700 1 -- 172.21.15.121:0/4143797853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdec414af80 msgr2=0x7fdec414c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:43.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdebbfff700 1 --2- 172.21.15.121:0/4143797853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdec414af80 0x7fdec414c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:43.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdebbfff700 1 -- 172.21.15.121:0/4143797853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdec40c1bf0 msgr2=0x7fdec4148480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:43.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdebbfff700 1 --2- 172.21.15.121:0/4143797853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdec40c1bf0 0x7fdec4148480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:43.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdebbfff700 1 -- 172.21.15.121:0/4143797853 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb400b540 con 0x7fdec40c6640 2023-11-15T03:06:43.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdebbfff700 1 --2- 172.21.15.121:0/4143797853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdec40c6640 0x7fdec4152970 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7fdeac00c440 tx=0x7fdeac007d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:43.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdebaffd700 1 -- 172.21.15.121:0/4143797853 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac0070d0 con 0x7fdec40c6640 2023-11-15T03:06:43.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdebaffd700 1 -- 172.21.15.121:0/4143797853 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeac007230 con 0x7fdec40c6640 2023-11-15T03:06:43.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.725+0000 7fdec941b700 1 -- 172.21.15.121:0/4143797853 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec414b920 con 0x7fdec40c6640 2023-11-15T03:06:43.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.729+0000 7fdebaffd700 1 -- 172.21.15.121:0/4143797853 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac0056e0 con 0x7fdec40c6640 2023-11-15T03:06:43.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.729+0000 7fdec941b700 1 -- 172.21.15.121:0/4143797853 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec414bee0 con 0x7fdec40c6640 2023-11-15T03:06:43.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.729+0000 7fdec941b700 1 -- 172.21.15.121:0/4143797853 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec406c150 con 0x7fdec40c6640 2023-11-15T03:06:43.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.729+0000 7fdebaffd700 1 -- 172.21.15.121:0/4143797853 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fdeac005a50 con 0x7fdec40c6640 2023-11-15T03:06:43.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.733+0000 7fdebaffd700 1 --2- 172.21.15.121:0/4143797853 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdea405a510 0x7fdea405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:43.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.733+0000 7fdebaffd700 1 -- 172.21.15.121:0/4143797853 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7814+0+0 (secure 0 0 0) 0x7fdeac0741a0 con 0x7fdec40c6640 2023-11-15T03:06:43.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.733+0000 7fdec2ffd700 1 --2- 172.21.15.121:0/4143797853 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdea405a510 0x7fdea405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:43.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.733+0000 7fdec2ffd700 1 --2- 172.21.15.121:0/4143797853 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdea405a510 0x7fdea405c9d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7fdeb400ba00 tx=0x7fdeb400b300).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:43.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.737+0000 7fdebaffd700 1 -- 172.21.15.121:0/4143797853 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdeac047220 con 0x7fdec40c6640 2023-11-15T03:06:43.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.933+0000 7fdec941b700 1 -- 172.21.15.121:0/4143797853 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdec4152000 con 0x7fdec40c6640 2023-11-15T03:06:43.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.933+0000 7fdebaffd700 1 -- 172.21.15.121:0/4143797853 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 194 v194) v1 ==== 95+0+4496 (secure 0 0 0) 0x7fdec4152000 con 0x7fdec40c6640 2023-11-15T03:06:43.941 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:43.942 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":194,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2289093120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2289093120},{"type":"v1","addr":"172.21.15.121:6836","nonce":2289093120}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:39.103503+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:43.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.937+0000 7fdec941b700 1 -- 172.21.15.121:0/4143797853 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdea405a510 msgr2=0x7fdea405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:43.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.937+0000 7fdec941b700 1 --2- 172.21.15.121:0/4143797853 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdea405a510 0x7fdea405c9d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7fdeb400ba00 tx=0x7fdeb400b300).stop 2023-11-15T03:06:43.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.937+0000 7fdec941b700 1 -- 172.21.15.121:0/4143797853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdec40c6640 msgr2=0x7fdec4152970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:43.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.937+0000 7fdec941b700 1 --2- 172.21.15.121:0/4143797853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdec40c6640 0x7fdec4152970 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7fdeac00c440 tx=0x7fdeac007d50).stop 2023-11-15T03:06:43.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.937+0000 7fdec941b700 1 -- 172.21.15.121:0/4143797853 shutdown_connections 2023-11-15T03:06:43.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.937+0000 7fdec941b700 1 --2- 172.21.15.121:0/4143797853 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdea405a510 0x7fdea405c9d0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:43.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.937+0000 7fdec941b700 1 --2- 172.21.15.121:0/4143797853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdec414af80 0x7fdec414c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:43.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.937+0000 7fdec941b700 1 --2- 172.21.15.121:0/4143797853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdec40c6640 0x7fdec4152970 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:43.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.937+0000 7fdec941b700 1 --2- 172.21.15.121:0/4143797853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdec40c1bf0 0x7fdec4148480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:43.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.937+0000 7fdec941b700 1 -- 172.21.15.121:0/4143797853 >> 172.21.15.121:0/4143797853 conn(0x7fdec4062190 msgr2=0x7fdec40a7990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:43.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.937+0000 7fdec941b700 1 -- 172.21.15.121:0/4143797853 shutdown_connections 2023-11-15T03:06:43.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:43.937+0000 7fdec941b700 1 -- 172.21.15.121:0/4143797853 wait complete. 2023-11-15T03:06:43.949 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 194 2023-11-15T03:06:43.962 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:06:43.963 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:44.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.117+0000 7f3a8021b700 1 Processor -- start 2023-11-15T03:06:44.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.117+0000 7f3a8021b700 1 -- start start 2023-11-15T03:06:44.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.121+0000 7f3a8021b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a781539d0 0x7f3a780c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:44.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.121+0000 7f3a8021b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a780c1530 con 0x7f3a78153db0 2023-11-15T03:06:44.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.121+0000 7f3a8021b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a780c2b20 con 0x7f3a780c2670 2023-11-15T03:06:44.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.121+0000 7f3a8021b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a781541f0 con 0x7f3a781539d0 2023-11-15T03:06:44.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.121+0000 7f3a7d7b6700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3a78153db0 0x7f3a780c19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40526/0 (socket says 172.21.15.121:40526) 2023-11-15T03:06:44.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.121+0000 7f3a7d7b6700 1 -- 172.21.15.121:0/1428858899 learned_addr learned my addr 172.21.15.121:0/1428858899 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:44.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.121+0000 7f3a7dfb7700 1 --2- 172.21.15.121:0/1428858899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a781539d0 0x7f3a780c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:44.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.121+0000 7f3a7cfb5700 1 -- 172.21.15.121:0/1428858899 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2698543160 0 0) 0x7f3a780c1530 con 0x7f3a78153db0 2023-11-15T03:06:44.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.121+0000 7f3a7cfb5700 1 -- 172.21.15.121:0/1428858899 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a640033f0 con 0x7f3a78153db0 2023-11-15T03:06:44.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.121+0000 7f3a7cfb5700 1 -- 172.21.15.121:0/1428858899 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 698462276 0 0) 0x7f3a780c2b20 con 0x7f3a780c2670 2023-11-15T03:06:44.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7cfb5700 1 -- 172.21.15.121:0/1428858899 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a780c1530 con 0x7f3a780c2670 2023-11-15T03:06:44.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7cfb5700 1 -- 172.21.15.121:0/1428858899 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3730393482 0 0) 0x7f3a640033f0 con 0x7f3a78153db0 2023-11-15T03:06:44.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7cfb5700 1 -- 172.21.15.121:0/1428858899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a781539d0 msgr2=0x7f3a780c06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:44.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7cfb5700 1 --2- 172.21.15.121:0/1428858899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a781539d0 0x7f3a780c06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:44.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7cfb5700 1 -- 172.21.15.121:0/1428858899 >> v1:172.21.15.190:6789/0 conn(0x7f3a780c2670 legacy=0x7f3a780c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:44.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7cfb5700 1 -- 172.21.15.121:0/1428858899 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a780c2d80 con 0x7f3a78153db0 2023-11-15T03:06:44.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7cfb5700 1 -- 172.21.15.121:0/1428858899 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f3a68001470 con 0x7f3a78153db0 2023-11-15T03:06:44.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7cfb5700 1 -- 172.21.15.121:0/1428858899 >> v1:172.21.15.121:6789/0 conn(0x7f3a78153db0 legacy=0x7f3a780c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:44.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7cfb5700 1 --2- 172.21.15.121:0/1428858899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a64003aa0 0x7f3a64003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:44.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7cfb5700 1 --2- 172.21.15.121:0/1428858899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a781539d0 0x7f3a64004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:44.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7cfb5700 1 --2- 172.21.15.121:0/1428858899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a64004b80 0x7f3a64005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:44.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7cfb5700 1 -- 172.21.15.121:0/1428858899 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a781541f0 con 0x7f3a64004b80 2023-11-15T03:06:44.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7cfb5700 1 -- 172.21.15.121:0/1428858899 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6400b910 con 0x7f3a64003aa0 2023-11-15T03:06:44.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7cfb5700 1 -- 172.21.15.121:0/1428858899 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6400bad0 con 0x7f3a781539d0 2023-11-15T03:06:44.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7e7b8700 1 --2- 172.21.15.121:0/1428858899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a64004b80 0x7f3a64005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:44.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7d7b6700 1 --2- 172.21.15.121:0/1428858899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a781539d0 0x7f3a64004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:44.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7dfb7700 1 --2- 172.21.15.121:0/1428858899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a64003aa0 0x7f3a64003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:44.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7e7b8700 1 -- 172.21.15.121:0/1428858899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a781539d0 msgr2=0x7f3a64004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:44.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7e7b8700 1 --2- 172.21.15.121:0/1428858899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a781539d0 0x7f3a64004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:44.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7e7b8700 1 -- 172.21.15.121:0/1428858899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a64003aa0 msgr2=0x7f3a64003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:44.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7e7b8700 1 --2- 172.21.15.121:0/1428858899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a64003aa0 0x7f3a64003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:44.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7e7b8700 1 -- 172.21.15.121:0/1428858899 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a640033f0 con 0x7f3a64004b80 2023-11-15T03:06:44.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7e7b8700 1 --2- 172.21.15.121:0/1428858899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a64004b80 0x7f3a64005000 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 rx=0x7f3a70001890 tx=0x7f3a70007bb0).ready entity=mon.0 client_cookie=ff817c991e346918 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:44.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7cfb5700 1 -- 172.21.15.121:0/1428858899 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70007020 con 0x7f3a64004b80 2023-11-15T03:06:44.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7cfb5700 1 -- 172.21.15.121:0/1428858899 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a70007180 con 0x7f3a64004b80 2023-11-15T03:06:44.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.125+0000 7f3a7cfb5700 1 -- 172.21.15.121:0/1428858899 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70007500 con 0x7f3a64004b80 2023-11-15T03:06:44.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.129+0000 7f3a8021b700 1 -- 172.21.15.121:0/1428858899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a64004b80 msgr2=0x7f3a64005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:44.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.129+0000 7f3a8021b700 1 --2- 172.21.15.121:0/1428858899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a64004b80 0x7f3a64005000 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 rx=0x7f3a70001890 tx=0x7f3a70007bb0).stop 2023-11-15T03:06:44.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.129+0000 7f3a7e7b8700 1 -- 172.21.15.121:0/1428858899 reap_dead start 2023-11-15T03:06:44.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.129+0000 7f3a8021b700 1 -- 172.21.15.121:0/1428858899 shutdown_connections 2023-11-15T03:06:44.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.129+0000 7f3a8021b700 1 -- 172.21.15.121:0/1428858899 >> 172.21.15.121:0/1428858899 conn(0x7f3a78062190 msgr2=0x7f3a780625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:44.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.129+0000 7f3a8021b700 1 -- 172.21.15.121:0/1428858899 shutdown_connections 2023-11-15T03:06:44.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.129+0000 7f3a8021b700 1 -- 172.21.15.121:0/1428858899 wait complete. 2023-11-15T03:06:44.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.129+0000 7f3a8021b700 1 Processor -- start 2023-11-15T03:06:44.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.129+0000 7f3a8021b700 1 -- start start 2023-11-15T03:06:44.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a8021b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a64004b80 0x7f3a7814ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:44.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a8021b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a78168510 0x7f3a780abb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:44.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a8021b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a780ac010 0x7f3a7814c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:44.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a8021b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a78156620 con 0x7f3a64004b80 2023-11-15T03:06:44.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a8021b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a78156890 con 0x7f3a78168510 2023-11-15T03:06:44.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a8021b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a78155f40 con 0x7f3a780ac010 2023-11-15T03:06:44.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a7dfb7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a64004b80 0x7f3a7814ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:44.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a7dfb7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a64004b80 0x7f3a7814ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50754/0 (socket says 172.21.15.121:50754) 2023-11-15T03:06:44.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a7dfb7700 1 -- 172.21.15.121:0/3137254970 learned_addr learned my addr 172.21.15.121:0/3137254970 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:44.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a7d7b6700 1 --2- 172.21.15.121:0/3137254970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a78168510 0x7f3a780abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:44.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a7dfb7700 1 -- 172.21.15.121:0/3137254970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a780ac010 msgr2=0x7f3a7814c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:44.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a7dfb7700 1 --2- 172.21.15.121:0/3137254970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a780ac010 0x7f3a7814c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:06:44.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a7dfb7700 1 -- 172.21.15.121:0/3137254970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a78168510 msgr2=0x7f3a780abb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:44.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a7dfb7700 1 --2- 172.21.15.121:0/3137254970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a78168510 0x7f3a780abb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:44.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a7dfb7700 1 -- 172.21.15.121:0/3137254970 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a78156220 con 0x7f3a64004b80 2023-11-15T03:06:44.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a7dfb7700 1 --2- 172.21.15.121:0/3137254970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a64004b80 0x7f3a7814ec20 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7f3a74011f00 tx=0x7f3a7400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:44.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a6dffb700 1 -- 172.21.15.121:0/3137254970 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7400e040 con 0x7f3a64004b80 2023-11-15T03:06:44.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a6dffb700 1 -- 172.21.15.121:0/3137254970 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7400f040 con 0x7f3a64004b80 2023-11-15T03:06:44.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a6dffb700 1 -- 172.21.15.121:0/3137254970 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a74015450 con 0x7f3a64004b80 2023-11-15T03:06:44.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a8021b700 1 -- 172.21.15.121:0/3137254970 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a78168990 con 0x7f3a64004b80 2023-11-15T03:06:44.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.133+0000 7f3a8021b700 1 -- 172.21.15.121:0/3137254970 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7814cb00 con 0x7f3a64004b80 2023-11-15T03:06:44.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.137+0000 7f3a8021b700 1 -- 172.21.15.121:0/3137254970 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7814cd50 con 0x7f3a64004b80 2023-11-15T03:06:44.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.137+0000 7f3a6dffb700 1 -- 172.21.15.121:0/3137254970 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3a74015880 con 0x7f3a64004b80 2023-11-15T03:06:44.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.137+0000 7f3a6dffb700 1 --2- 172.21.15.121:0/3137254970 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3a5c05a510 0x7f3a5c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:44.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.137+0000 7f3a6dffb700 1 -- 172.21.15.121:0/3137254970 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7814+0+0 (secure 0 0 0) 0x7f3a74075c10 con 0x7f3a64004b80 2023-11-15T03:06:44.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.137+0000 7f3a7d7b6700 1 --2- 172.21.15.121:0/3137254970 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3a5c05a510 0x7f3a5c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:44.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.141+0000 7f3a7d7b6700 1 --2- 172.21.15.121:0/3137254970 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3a5c05a510 0x7f3a5c05c9d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f3a68001c80 tx=0x7f3a68009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:44.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.145+0000 7f3a6dffb700 1 -- 172.21.15.121:0/3137254970 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3a74048d50 con 0x7f3a64004b80 2023-11-15T03:06:44.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.337+0000 7f3a8021b700 1 -- 172.21.15.121:0/3137254970 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a78156410 con 0x7f3a64004b80 2023-11-15T03:06:44.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.337+0000 7f3a6dffb700 1 -- 172.21.15.121:0/3137254970 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 194 v194) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f3a7404c370 con 0x7f3a64004b80 2023-11-15T03:06:44.346 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:44.346 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":194,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2289093120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2289093120},{"type":"v1","addr":"172.21.15.121:6836","nonce":2289093120}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:39.103503+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:44.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.341+0000 7f3a8021b700 1 -- 172.21.15.121:0/3137254970 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3a5c05a510 msgr2=0x7f3a5c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:44.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.341+0000 7f3a8021b700 1 --2- 172.21.15.121:0/3137254970 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3a5c05a510 0x7f3a5c05c9d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f3a68001c80 tx=0x7f3a68009040).stop 2023-11-15T03:06:44.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.341+0000 7f3a8021b700 1 -- 172.21.15.121:0/3137254970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a64004b80 msgr2=0x7f3a7814ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:44.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.341+0000 7f3a8021b700 1 --2- 172.21.15.121:0/3137254970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a64004b80 0x7f3a7814ec20 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7f3a74011f00 tx=0x7f3a7400b040).stop 2023-11-15T03:06:44.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.341+0000 7f3a8021b700 1 -- 172.21.15.121:0/3137254970 shutdown_connections 2023-11-15T03:06:44.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.341+0000 7f3a8021b700 1 --2- 172.21.15.121:0/3137254970 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3a5c05a510 0x7f3a5c05c9d0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:44.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.341+0000 7f3a8021b700 1 --2- 172.21.15.121:0/3137254970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a780ac010 0x7f3a7814c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:06:44.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.341+0000 7f3a8021b700 1 --2- 172.21.15.121:0/3137254970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a78168510 0x7f3a780abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:44.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.341+0000 7f3a8021b700 1 --2- 172.21.15.121:0/3137254970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a64004b80 0x7f3a7814ec20 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:44.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.341+0000 7f3a8021b700 1 -- 172.21.15.121:0/3137254970 >> 172.21.15.121:0/3137254970 conn(0x7f3a78062190 msgr2=0x7f3a780c2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:44.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.341+0000 7f3a8021b700 1 -- 172.21.15.121:0/3137254970 shutdown_connections 2023-11-15T03:06:44.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.341+0000 7f3a8021b700 1 -- 172.21.15.121:0/3137254970 wait complete. 2023-11-15T03:06:44.351 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 194 2023-11-15T03:06:44.369 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:06:44.369 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:44.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.521+0000 7f06f8a8f700 1 Processor -- start 2023-11-15T03:06:44.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.521+0000 7f06f8a8f700 1 -- start start 2023-11-15T03:06:44.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.521+0000 7f06f8a8f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06f4066700 0x7f06f4154110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:44.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.521+0000 7f06f8a8f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f40c73e0 con 0x7f06f4066320 2023-11-15T03:06:44.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.521+0000 7f06f8a8f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f41546c0 con 0x7f06f40acf10 2023-11-15T03:06:44.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.521+0000 7f06f8a8f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f41549a0 con 0x7f06f4066700 2023-11-15T03:06:44.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.521+0000 7f06f37fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f06f4066320 0x7f06f40656a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40534/0 (socket says 172.21.15.121:40534) 2023-11-15T03:06:44.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.521+0000 7f06f37fe700 1 -- 172.21.15.121:0/2294715825 learned_addr learned my addr 172.21.15.121:0/2294715825 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:44.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.521+0000 7f06f2ffd700 1 --2- 172.21.15.121:0/2294715825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06f4066700 0x7f06f4154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:44.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.521+0000 7f06f27fc700 1 -- 172.21.15.121:0/2294715825 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714418722 0 0) 0x7f06f40c73e0 con 0x7f06f4066320 2023-11-15T03:06:44.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.525+0000 7f06f27fc700 1 -- 172.21.15.121:0/2294715825 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06dc0033f0 con 0x7f06f4066320 2023-11-15T03:06:44.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.525+0000 7f06f2ffd700 1 -- 172.21.15.121:0/2294715825 >> v1:172.21.15.190:6789/0 conn(0x7f06f40acf10 legacy=0x7f06f4062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:44.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.525+0000 7f06f27fc700 1 -- 172.21.15.121:0/2294715825 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1955041100 0 0) 0x7f06f41546c0 con 0x7f06f40acf10 2023-11-15T03:06:44.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.525+0000 7f06f2ffd700 1 -- 172.21.15.121:0/2294715825 >> v1:172.21.15.121:6789/0 conn(0x7f06f4066320 legacy=0x7f06f40656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:44.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.525+0000 7f06f2ffd700 1 -- 172.21.15.121:0/2294715825 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f4154c80 con 0x7f06f4066700 2023-11-15T03:06:44.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.525+0000 7f06f2ffd700 1 --2- 172.21.15.121:0/2294715825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06f4066700 0x7f06f4154110 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f06e8009790 tx=0x7f06e8009060).ready entity=mon.2 client_cookie=fd3bc11b471aba57 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:44.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.525+0000 7f06f27fc700 1 -- 172.21.15.121:0/2294715825 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e4001470 con 0x7f06f4066700 2023-11-15T03:06:44.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.525+0000 7f06f27fc700 1 -- 172.21.15.121:0/2294715825 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e8012410 con 0x7f06f4066700 2023-11-15T03:06:44.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.525+0000 7f06f27fc700 1 -- 172.21.15.121:0/2294715825 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8012790 con 0x7f06f4066700 2023-11-15T03:06:44.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.525+0000 7f06f8a8f700 1 -- 172.21.15.121:0/2294715825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06f4066700 msgr2=0x7f06f4154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:44.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.525+0000 7f06f8a8f700 1 --2- 172.21.15.121:0/2294715825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06f4066700 0x7f06f4154110 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f06e8009790 tx=0x7f06e8009060).stop 2023-11-15T03:06:44.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.525+0000 7f06f8a8f700 1 -- 172.21.15.121:0/2294715825 shutdown_connections 2023-11-15T03:06:44.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.525+0000 7f06f8a8f700 1 --2- 172.21.15.121:0/2294715825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06f4066700 0x7f06f4154110 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:44.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.525+0000 7f06f8a8f700 1 -- 172.21.15.121:0/2294715825 >> 172.21.15.121:0/2294715825 conn(0x7f06f4067a60 msgr2=0x7f06f4067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:44.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.525+0000 7f06f8a8f700 1 -- 172.21.15.121:0/2294715825 shutdown_connections 2023-11-15T03:06:44.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.525+0000 7f06f8a8f700 1 -- 172.21.15.121:0/2294715825 wait complete. 2023-11-15T03:06:44.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.529+0000 7f06f8a8f700 1 Processor -- start 2023-11-15T03:06:44.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.529+0000 7f06f8a8f700 1 -- start start 2023-11-15T03:06:44.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f8a8f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06f4066320 0x7f06f4143450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:44.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f8a8f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06f4066700 0x7f06f4158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:44.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f8a8f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06f40acf10 0x7f06f4156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:44.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f8a8f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f4158500 con 0x7f06f4066700 2023-11-15T03:06:44.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f8a8f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f41578f0 con 0x7f06f4066320 2023-11-15T03:06:44.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f8a8f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f4157bd0 con 0x7f06f40acf10 2023-11-15T03:06:44.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f2ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06f4066700 0x7f06f4158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:44.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f2ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06f4066700 0x7f06f4158990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50764/0 (socket says 172.21.15.121:50764) 2023-11-15T03:06:44.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f2ffd700 1 -- 172.21.15.121:0/3831015997 learned_addr learned my addr 172.21.15.121:0/3831015997 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:44.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f37fe700 1 --2- 172.21.15.121:0/3831015997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06f4066320 0x7f06f4143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:44.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f3fff700 1 --2- 172.21.15.121:0/3831015997 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06f40acf10 0x7f06f4156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:44.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f2ffd700 1 -- 172.21.15.121:0/3831015997 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06f40acf10 msgr2=0x7f06f4156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:44.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f2ffd700 1 --2- 172.21.15.121:0/3831015997 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06f40acf10 0x7f06f4156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:44.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f2ffd700 1 -- 172.21.15.121:0/3831015997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06f4066320 msgr2=0x7f06f4143450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:44.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f2ffd700 1 --2- 172.21.15.121:0/3831015997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06f4066320 0x7f06f4143450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:44.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f2ffd700 1 -- 172.21.15.121:0/3831015997 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e80092a0 con 0x7f06f4066700 2023-11-15T03:06:44.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f37fe700 1 --2- 172.21.15.121:0/3831015997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06f4066320 0x7f06f4143450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:44.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f2ffd700 1 --2- 172.21.15.121:0/3831015997 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06f4066700 0x7f06f4158990 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7f06e800f000 tx=0x7f06e8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:44.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06db7fe700 1 -- 172.21.15.121:0/3831015997 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8011900 con 0x7f06f4066700 2023-11-15T03:06:44.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06db7fe700 1 -- 172.21.15.121:0/3831015997 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e8011a60 con 0x7f06f4066700 2023-11-15T03:06:44.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f8a8f700 1 -- 172.21.15.121:0/3831015997 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f4157eb0 con 0x7f06f4066700 2023-11-15T03:06:44.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06db7fe700 1 -- 172.21.15.121:0/3831015997 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8011de0 con 0x7f06f4066700 2023-11-15T03:06:44.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.533+0000 7f06f8a8f700 1 -- 172.21.15.121:0/3831015997 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f4156900 con 0x7f06f4066700 2023-11-15T03:06:44.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.537+0000 7f06f8a8f700 1 -- 172.21.15.121:0/3831015997 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f406bcf0 con 0x7f06f4066700 2023-11-15T03:06:44.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.537+0000 7f06db7fe700 1 -- 172.21.15.121:0/3831015997 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f06e801d600 con 0x7f06f4066700 2023-11-15T03:06:44.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.541+0000 7f06db7fe700 1 --2- 172.21.15.121:0/3831015997 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f06e005a540 0x7f06e005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:44.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.541+0000 7f06db7fe700 1 -- 172.21.15.121:0/3831015997 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7814+0+0 (secure 0 0 0) 0x7f06e8076820 con 0x7f06f4066700 2023-11-15T03:06:44.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.541+0000 7f06f37fe700 1 --2- 172.21.15.121:0/3831015997 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f06e005a540 0x7f06e005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:44.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.541+0000 7f06f37fe700 1 --2- 172.21.15.121:0/3831015997 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f06e005a540 0x7f06e005ca00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f06e4000fb0 tx=0x7f06e4001ab0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:44.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.545+0000 7f06db7fe700 1 -- 172.21.15.121:0/3831015997 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f06e80498a0 con 0x7f06f4066700 2023-11-15T03:06:44.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.741+0000 7f06f8a8f700 1 -- 172.21.15.121:0/3831015997 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06f406bcf0 con 0x7f06f4066700 2023-11-15T03:06:44.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.741+0000 7f06db7fe700 1 -- 172.21.15.121:0/3831015997 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 194 v194) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f06e804cec0 con 0x7f06f4066700 2023-11-15T03:06:44.748 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:44.749 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":194,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2289093120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2289093120},{"type":"v1","addr":"172.21.15.121:6836","nonce":2289093120}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:39.103503+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:44.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.745+0000 7f06f8a8f700 1 -- 172.21.15.121:0/3831015997 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f06e005a540 msgr2=0x7f06e005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:44.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.745+0000 7f06f8a8f700 1 --2- 172.21.15.121:0/3831015997 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f06e005a540 0x7f06e005ca00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f06e4000fb0 tx=0x7f06e4001ab0).stop 2023-11-15T03:06:44.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.745+0000 7f06f8a8f700 1 -- 172.21.15.121:0/3831015997 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06f4066700 msgr2=0x7f06f4158990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:44.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.745+0000 7f06f8a8f700 1 --2- 172.21.15.121:0/3831015997 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06f4066700 0x7f06f4158990 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7f06e800f000 tx=0x7f06e8009060).stop 2023-11-15T03:06:44.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.745+0000 7f06f8a8f700 1 -- 172.21.15.121:0/3831015997 shutdown_connections 2023-11-15T03:06:44.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.745+0000 7f06f8a8f700 1 --2- 172.21.15.121:0/3831015997 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f06e005a540 0x7f06e005ca00 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:44.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.745+0000 7f06f8a8f700 1 --2- 172.21.15.121:0/3831015997 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06f40acf10 0x7f06f4156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:44.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.745+0000 7f06f8a8f700 1 --2- 172.21.15.121:0/3831015997 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06f4066700 0x7f06f4158990 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:44.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.745+0000 7f06f8a8f700 1 --2- 172.21.15.121:0/3831015997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06f4066320 0x7f06f4143450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:44.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.745+0000 7f06f8a8f700 1 -- 172.21.15.121:0/3831015997 >> 172.21.15.121:0/3831015997 conn(0x7f06f4067a60 msgr2=0x7f06f41514d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:44.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.745+0000 7f06f8a8f700 1 -- 172.21.15.121:0/3831015997 shutdown_connections 2023-11-15T03:06:44.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.745+0000 7f06f8a8f700 1 -- 172.21.15.121:0/3831015997 wait complete. 2023-11-15T03:06:44.754 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 194 2023-11-15T03:06:44.771 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 200.00000001' led to DAMAGED state 2023-11-15T03:06:44.771 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 500.00000000' 2023-11-15T03:06:44.772 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:06:44.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:06:44.773 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:06:44.792 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:06:44.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:06:44.793 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:06:45.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.993+0000 7ff8c2039700 1 Processor -- start 2023-11-15T03:06:45.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.993+0000 7ff8c2039700 1 -- start start 2023-11-15T03:06:45.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.997+0000 7ff8c2039700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff8bc0669b0 0x7ff8bc158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:45.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.997+0000 7ff8c2039700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8bc0a8750 0x7ff8bc0a8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:45.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.997+0000 7ff8c2039700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8bc0a90d0 con 0x7ff8bc157d90 2023-11-15T03:06:45.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.997+0000 7ff8c2039700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8bc0c1fb0 con 0x7ff8bc0669b0 2023-11-15T03:06:45.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.997+0000 7ff8c2039700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc15bbd0 con 0x7ff8bc0a8750 2023-11-15T03:06:45.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.997+0000 7ff8c0836700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff8bc0669b0 0x7ff8bc158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:45.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.997+0000 7ff8c0836700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff8bc0669b0 0x7ff8bc158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38030/0 (socket says 172.21.15.121:38030) 2023-11-15T03:06:45.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.997+0000 7ff8c0836700 1 -- 172.21.15.121:0/2144288421 learned_addr learned my addr 172.21.15.121:0/2144288421 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:45.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.997+0000 7ff8c1838700 1 --2- 172.21.15.121:0/2144288421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8bc0a8750 0x7ff8bc0a8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:45.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.997+0000 7ff8c0836700 1 -- 172.21.15.121:0/2144288421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8bc0a8750 msgr2=0x7ff8bc0a8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:45.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.997+0000 7ff8c0836700 1 --2- 172.21.15.121:0/2144288421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8bc0a8750 0x7ff8bc0a8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:45.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.997+0000 7ff8c0836700 1 -- 172.21.15.121:0/2144288421 >> v1:172.21.15.190:6789/0 conn(0x7ff8bc157d90 legacy=0x7ff8bc0c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:45.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.997+0000 7ff8c0836700 1 -- 172.21.15.121:0/2144288421 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc0bb910 con 0x7ff8bc0669b0 2023-11-15T03:06:45.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.997+0000 7ff8b3fff700 1 -- 172.21.15.121:0/2144288421 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1279494010 0 0) 0x7ff8bc0a90d0 con 0x7ff8bc157d90 2023-11-15T03:06:45.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.997+0000 7ff8c0836700 1 --2- 172.21.15.121:0/2144288421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8bc0669b0 0x7ff8bc158170 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7ff8ac006fd0 tx=0x7ff8ac0090f0).ready entity=mon.0 client_cookie=5ceb3441b7bc46fd server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:45.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.997+0000 7ff8b3fff700 1 -- 172.21.15.121:0/2144288421 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac00b580 con 0x7ff8bc0669b0 2023-11-15T03:06:45.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.997+0000 7ff8b3fff700 1 -- 172.21.15.121:0/2144288421 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ac00b6e0 con 0x7ff8bc0669b0 2023-11-15T03:06:45.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:44.997+0000 7ff8b3fff700 1 -- 172.21.15.121:0/2144288421 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac00ba80 con 0x7ff8bc0669b0 2023-11-15T03:06:45.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.001+0000 7ff8c2039700 1 -- 172.21.15.121:0/2144288421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8bc0669b0 msgr2=0x7ff8bc158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:45.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.001+0000 7ff8c2039700 1 --2- 172.21.15.121:0/2144288421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8bc0669b0 0x7ff8bc158170 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7ff8ac006fd0 tx=0x7ff8ac0090f0).stop 2023-11-15T03:06:45.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.001+0000 7ff8c2039700 1 -- 172.21.15.121:0/2144288421 shutdown_connections 2023-11-15T03:06:45.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.001+0000 7ff8c2039700 1 --2- 172.21.15.121:0/2144288421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8bc0a8750 0x7ff8bc0a8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:45.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.001+0000 7ff8c2039700 1 --2- 172.21.15.121:0/2144288421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8bc0669b0 0x7ff8bc158170 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:45.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.001+0000 7ff8c2039700 1 -- 172.21.15.121:0/2144288421 >> 172.21.15.121:0/2144288421 conn(0x7ff8bc067a60 msgr2=0x7ff8bc067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:45.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.001+0000 7ff8c2039700 1 -- 172.21.15.121:0/2144288421 shutdown_connections 2023-11-15T03:06:45.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.001+0000 7ff8c2039700 1 -- 172.21.15.121:0/2144288421 wait complete. 2023-11-15T03:06:45.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.001+0000 7ff8c2039700 1 Processor -- start 2023-11-15T03:06:45.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.001+0000 7ff8c2039700 1 -- start start 2023-11-15T03:06:45.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8c2039700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8bc0669b0 0x7ff8bc148970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:45.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8c2039700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8bc0a8750 0x7ff8bc150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:45.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8c2039700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8bc157d90 0x7ff8bc151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:45.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8c2039700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc151560 con 0x7ff8bc0a8750 2023-11-15T03:06:45.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8c2039700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc155470 con 0x7ff8bc157d90 2023-11-15T03:06:45.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8c2039700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc1556c0 con 0x7ff8bc0669b0 2023-11-15T03:06:45.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8c0836700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8bc0a8750 0x7ff8bc150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:45.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8c0836700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8bc0a8750 0x7ff8bc150b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38036/0 (socket says 172.21.15.121:38036) 2023-11-15T03:06:45.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8c0836700 1 -- 172.21.15.121:0/2065810740 learned_addr learned my addr 172.21.15.121:0/2065810740 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:45.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8c1037700 1 --2- 172.21.15.121:0/2065810740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8bc0669b0 0x7ff8bc148970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:45.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8c1838700 1 --2- 172.21.15.121:0/2065810740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8bc157d90 0x7ff8bc151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:45.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8c0836700 1 -- 172.21.15.121:0/2065810740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8bc0669b0 msgr2=0x7ff8bc148970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:45.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8c0836700 1 --2- 172.21.15.121:0/2065810740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8bc0669b0 0x7ff8bc148970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:45.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8c0836700 1 -- 172.21.15.121:0/2065810740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8bc157d90 msgr2=0x7ff8bc151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:45.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8c0836700 1 --2- 172.21.15.121:0/2065810740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8bc157d90 0x7ff8bc151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:45.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8c0836700 1 -- 172.21.15.121:0/2065810740 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ac009310 con 0x7ff8bc0a8750 2023-11-15T03:06:45.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8c0836700 1 --2- 172.21.15.121:0/2065810740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8bc0a8750 0x7ff8bc150b80 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7ff8ac009e90 tx=0x7ff8ac017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:45.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8b17fa700 1 -- 172.21.15.121:0/2065810740 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac00be40 con 0x7ff8bc0a8750 2023-11-15T03:06:45.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8b17fa700 1 -- 172.21.15.121:0/2065810740 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ac00b850 con 0x7ff8bc0a8750 2023-11-15T03:06:45.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8b17fa700 1 -- 172.21.15.121:0/2065810740 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac01c420 con 0x7ff8bc0a8750 2023-11-15T03:06:45.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8c2039700 1 -- 172.21.15.121:0/2065810740 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8bc1559a0 con 0x7ff8bc0a8750 2023-11-15T03:06:45.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.005+0000 7ff8c2039700 1 -- 172.21.15.121:0/2065810740 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8bc155f60 con 0x7ff8bc0a8750 2023-11-15T03:06:45.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.009+0000 7ff8c2039700 1 -- 172.21.15.121:0/2065810740 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8bc06bcf0 con 0x7ff8bc0a8750 2023-11-15T03:06:45.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.009+0000 7ff8b17fa700 1 -- 172.21.15.121:0/2065810740 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff8ac01c810 con 0x7ff8bc0a8750 2023-11-15T03:06:45.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.013+0000 7ff8b17fa700 1 --2- 172.21.15.121:0/2065810740 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff8a005a4e0 0x7ff8a005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:45.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.013+0000 7ff8b17fa700 1 -- 172.21.15.121:0/2065810740 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7814+0+0 (secure 0 0 0) 0x7ff8ac076b80 con 0x7ff8bc0a8750 2023-11-15T03:06:45.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.013+0000 7ff8c1037700 1 --2- 172.21.15.121:0/2065810740 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff8a005a4e0 0x7ff8a005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:45.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.013+0000 7ff8c1037700 1 --2- 172.21.15.121:0/2065810740 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff8a005a4e0 0x7ff8a005c9a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7ff8b8007fd0 tx=0x7ff8b8007bb0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:45.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.017+0000 7ff8b17fa700 1 -- 172.21.15.121:0/2065810740 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff8ac049c00 con 0x7ff8bc0a8750 2023-11-15T03:06:45.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.213+0000 7ff8c2039700 1 -- 172.21.15.121:0/2065810740 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff8bc0c3c70 con 0x7ff8bc0a8750 2023-11-15T03:06:45.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.273+0000 7ff8b17fa700 1 -- 172.21.15.121:0/2065810740 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v195) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff8ac076e60 con 0x7ff8bc0a8750 2023-11-15T03:06:45.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.277+0000 7ff8c2039700 1 -- 172.21.15.121:0/2065810740 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff8a005a4e0 msgr2=0x7ff8a005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:45.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.277+0000 7ff8c2039700 1 --2- 172.21.15.121:0/2065810740 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff8a005a4e0 0x7ff8a005c9a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7ff8b8007fd0 tx=0x7ff8b8007bb0).stop 2023-11-15T03:06:45.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.277+0000 7ff8c2039700 1 -- 172.21.15.121:0/2065810740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8bc0a8750 msgr2=0x7ff8bc150b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:45.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.277+0000 7ff8c2039700 1 --2- 172.21.15.121:0/2065810740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8bc0a8750 0x7ff8bc150b80 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7ff8ac009e90 tx=0x7ff8ac017000).stop 2023-11-15T03:06:45.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.277+0000 7ff8c2039700 1 -- 172.21.15.121:0/2065810740 shutdown_connections 2023-11-15T03:06:45.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.277+0000 7ff8c2039700 1 --2- 172.21.15.121:0/2065810740 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff8a005a4e0 0x7ff8a005c9a0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:45.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.277+0000 7ff8c2039700 1 --2- 172.21.15.121:0/2065810740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8bc157d90 0x7ff8bc151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:45.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.277+0000 7ff8c2039700 1 --2- 172.21.15.121:0/2065810740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8bc0a8750 0x7ff8bc150b80 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:45.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.277+0000 7ff8c2039700 1 --2- 172.21.15.121:0/2065810740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8bc0669b0 0x7ff8bc148970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:45.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.277+0000 7ff8c2039700 1 -- 172.21.15.121:0/2065810740 >> 172.21.15.121:0/2065810740 conn(0x7ff8bc067a60 msgr2=0x7ff8bc0bbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:45.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.277+0000 7ff8c2039700 1 -- 172.21.15.121:0/2065810740 shutdown_connections 2023-11-15T03:06:45.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.277+0000 7ff8c2039700 1 -- 172.21.15.121:0/2065810740 wait complete. 2023-11-15T03:06:45.287 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:06:45.309 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:06:45.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.457+0000 7f1a3b7a9700 1 Processor -- start 2023-11-15T03:06:45.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.457+0000 7f1a3b7a9700 1 -- start start 2023-11-15T03:06:45.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.461+0000 7f1a3b7a9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1a34153b80 0x7f1a34094e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:45.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.461+0000 7f1a3b7a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a340a6e70 0x7f1a340c4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:45.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.461+0000 7f1a3b7a9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1a340c4520 0x7f1a34067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:45.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.461+0000 7f1a3b7a9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a340a6a20 con 0x7f1a34153b80 2023-11-15T03:06:45.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.461+0000 7f1a3b7a9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a34095350 con 0x7f1a340c4520 2023-11-15T03:06:45.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.461+0000 7f1a3b7a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340c49d0 con 0x7f1a340a6e70 2023-11-15T03:06:45.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.461+0000 7f1a39545700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1a34153b80 0x7f1a34094e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:45.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.461+0000 7f1a39545700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1a34153b80 0x7f1a34094e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38040/0 (socket says 172.21.15.121:38040) 2023-11-15T03:06:45.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.461+0000 7f1a39545700 1 -- 172.21.15.121:0/344486985 learned_addr learned my addr 172.21.15.121:0/344486985 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:45.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.461+0000 7f1a38d44700 1 --2- 172.21.15.121:0/344486985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a340a6e70 0x7f1a340c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:45.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.461+0000 7f1a39d46700 1 --2- 172.21.15.121:0/344486985 >> v2:172.21.15.190:3300/0 conn(0x7f1a340c4520 0x7f1a34067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:45.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.461+0000 7f1a39545700 1 -- 172.21.15.121:0/344486985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a340a6e70 msgr2=0x7f1a340c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:45.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.461+0000 7f1a39545700 1 --2- 172.21.15.121:0/344486985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a340a6e70 0x7f1a340c4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:45.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.461+0000 7f1a39545700 1 -- 172.21.15.121:0/344486985 >> v2:172.21.15.190:3300/0 conn(0x7f1a340c4520 msgr2=0x7f1a34067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:45.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.461+0000 7f1a39545700 1 --2- 172.21.15.121:0/344486985 >> v2:172.21.15.190:3300/0 conn(0x7f1a340c4520 0x7f1a34067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:45.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.461+0000 7f1a39545700 1 -- 172.21.15.121:0/344486985 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a340be5a0 con 0x7f1a34153b80 2023-11-15T03:06:45.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.461+0000 7f1a39d46700 1 --2- 172.21.15.121:0/344486985 >> v2:172.21.15.190:3300/0 conn(0x7f1a340c4520 0x7f1a34067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:45.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.461+0000 7f1a39545700 1 --2- 172.21.15.121:0/344486985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a34153b80 0x7f1a34094e60 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7f1a34095800 tx=0x7f1a300090f0).ready entity=mon.0 client_cookie=5a7dacfd77db1d32 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:45.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.461+0000 7f1a2bfff700 1 -- 172.21.15.121:0/344486985 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3000b520 con 0x7f1a34153b80 2023-11-15T03:06:45.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.465+0000 7f1a2bfff700 1 -- 172.21.15.121:0/344486985 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3000b680 con 0x7f1a34153b80 2023-11-15T03:06:45.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.465+0000 7f1a2bfff700 1 -- 172.21.15.121:0/344486985 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3000ba00 con 0x7f1a34153b80 2023-11-15T03:06:45.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.465+0000 7f1a3b7a9700 1 -- 172.21.15.121:0/344486985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a34153b80 msgr2=0x7f1a34094e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:45.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.465+0000 7f1a3b7a9700 1 --2- 172.21.15.121:0/344486985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a34153b80 0x7f1a34094e60 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7f1a34095800 tx=0x7f1a300090f0).stop 2023-11-15T03:06:45.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.465+0000 7f1a3b7a9700 1 -- 172.21.15.121:0/344486985 shutdown_connections 2023-11-15T03:06:45.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.465+0000 7f1a3b7a9700 1 --2- 172.21.15.121:0/344486985 >> v2:172.21.15.190:3300/0 conn(0x7f1a340c4520 0x7f1a34067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:45.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.465+0000 7f1a3b7a9700 1 --2- 172.21.15.121:0/344486985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a340a6e70 0x7f1a340c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:45.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.465+0000 7f1a3b7a9700 1 --2- 172.21.15.121:0/344486985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a34153b80 0x7f1a34094e60 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:45.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.465+0000 7f1a3b7a9700 1 -- 172.21.15.121:0/344486985 >> 172.21.15.121:0/344486985 conn(0x7f1a34067480 msgr2=0x7f1a34068b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:45.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.465+0000 7f1a3b7a9700 1 -- 172.21.15.121:0/344486985 shutdown_connections 2023-11-15T03:06:45.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.465+0000 7f1a3b7a9700 1 -- 172.21.15.121:0/344486985 wait complete. 2023-11-15T03:06:45.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.469+0000 7f1a3b7a9700 1 Processor -- start 2023-11-15T03:06:45.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.469+0000 7f1a3b7a9700 1 -- start start 2023-11-15T03:06:45.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.469+0000 7f1a3b7a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a340a6e70 0x7f1a340c1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:45.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.469+0000 7f1a3b7a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a340c4520 0x7f1a3414d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:45.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.469+0000 7f1a3b7a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a34153b80 0x7f1a340bf410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:45.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.473+0000 7f1a3b7a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340bff60 con 0x7f1a34153b80 2023-11-15T03:06:45.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.473+0000 7f1a3b7a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340c0dc0 con 0x7f1a340c4520 2023-11-15T03:06:45.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.473+0000 7f1a3b7a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340c01b0 con 0x7f1a340a6e70 2023-11-15T03:06:45.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.473+0000 7f1a39d46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a34153b80 0x7f1a340bf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:45.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.473+0000 7f1a39d46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a34153b80 0x7f1a340bf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38054/0 (socket says 172.21.15.121:38054) 2023-11-15T03:06:45.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.473+0000 7f1a39d46700 1 -- 172.21.15.121:0/852386283 learned_addr learned my addr 172.21.15.121:0/852386283 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:45.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.473+0000 7f1a38d44700 1 --2- 172.21.15.121:0/852386283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a340c4520 0x7f1a3414d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:45.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.473+0000 7f1a39545700 1 --2- 172.21.15.121:0/852386283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a340a6e70 0x7f1a340c1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:45.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.473+0000 7f1a39d46700 1 -- 172.21.15.121:0/852386283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a340a6e70 msgr2=0x7f1a340c1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:45.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.473+0000 7f1a39d46700 1 --2- 172.21.15.121:0/852386283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a340a6e70 0x7f1a340c1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:45.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.473+0000 7f1a39d46700 1 -- 172.21.15.121:0/852386283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a340c4520 msgr2=0x7f1a3414d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:45.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.473+0000 7f1a39d46700 1 --2- 172.21.15.121:0/852386283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a340c4520 0x7f1a3414d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:45.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.473+0000 7f1a39d46700 1 -- 172.21.15.121:0/852386283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a30009370 con 0x7f1a34153b80 2023-11-15T03:06:45.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.473+0000 7f1a39d46700 1 --2- 172.21.15.121:0/852386283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a34153b80 0x7f1a340bf410 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f1a2c00d1b0 tx=0x7f1a2c009a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:45.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.473+0000 7f1a297fa700 1 -- 172.21.15.121:0/852386283 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c00dcf0 con 0x7f1a34153b80 2023-11-15T03:06:45.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.473+0000 7f1a297fa700 1 -- 172.21.15.121:0/852386283 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2c00f040 con 0x7f1a34153b80 2023-11-15T03:06:45.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.473+0000 7f1a3b7a9700 1 -- 172.21.15.121:0/852386283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a340c0490 con 0x7f1a34153b80 2023-11-15T03:06:45.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.473+0000 7f1a297fa700 1 -- 172.21.15.121:0/852386283 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c013600 con 0x7f1a34153b80 2023-11-15T03:06:45.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.473+0000 7f1a3b7a9700 1 -- 172.21.15.121:0/852386283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a340c0a50 con 0x7f1a34153b80 2023-11-15T03:06:45.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.477+0000 7f1a3b7a9700 1 -- 172.21.15.121:0/852386283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a34064860 con 0x7f1a34153b80 2023-11-15T03:06:45.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.477+0000 7f1a297fa700 1 -- 172.21.15.121:0/852386283 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1a2c0139f0 con 0x7f1a34153b80 2023-11-15T03:06:45.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.477+0000 7f1a297fa700 1 --2- 172.21.15.121:0/852386283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1a1805a510 0x7f1a1805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:45.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.477+0000 7f1a297fa700 1 -- 172.21.15.121:0/852386283 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7814+0+0 (secure 0 0 0) 0x7f1a2c075440 con 0x7f1a34153b80 2023-11-15T03:06:45.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.477+0000 7f1a39545700 1 --2- 172.21.15.121:0/852386283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1a1805a510 0x7f1a1805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:45.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.481+0000 7f1a39545700 1 --2- 172.21.15.121:0/852386283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1a1805a510 0x7f1a1805c9d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f1a30009130 tx=0x7f1a30017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:45.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.481+0000 7f1a297fa700 1 -- 172.21.15.121:0/852386283 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1a34064860 con 0x7f1a34153b80 2023-11-15T03:06:45.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:45.697+0000 7f1a3b7a9700 1 -- 172.21.15.121:0/852386283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f1a340c0f50 con 0x7f1a34153b80 2023-11-15T03:06:46.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.281+0000 7f1a297fa700 1 -- 172.21.15.121:0/852386283 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v196) v1 ==== 101+0+0 (secure 0 0 0) 0x7f1a2c048520 con 0x7f1a34153b80 2023-11-15T03:06:46.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.285+0000 7f1a3b7a9700 1 -- 172.21.15.121:0/852386283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1a1805a510 msgr2=0x7f1a1805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.285+0000 7f1a3b7a9700 1 --2- 172.21.15.121:0/852386283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1a1805a510 0x7f1a1805c9d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f1a30009130 tx=0x7f1a30017000).stop 2023-11-15T03:06:46.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.285+0000 7f1a3b7a9700 1 -- 172.21.15.121:0/852386283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a34153b80 msgr2=0x7f1a340bf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.285+0000 7f1a3b7a9700 1 --2- 172.21.15.121:0/852386283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a34153b80 0x7f1a340bf410 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f1a2c00d1b0 tx=0x7f1a2c009a50).stop 2023-11-15T03:06:46.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.285+0000 7f1a3b7a9700 1 -- 172.21.15.121:0/852386283 shutdown_connections 2023-11-15T03:06:46.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.285+0000 7f1a3b7a9700 1 --2- 172.21.15.121:0/852386283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1a1805a510 0x7f1a1805c9d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.285+0000 7f1a3b7a9700 1 --2- 172.21.15.121:0/852386283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a34153b80 0x7f1a340bf410 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.285+0000 7f1a3b7a9700 1 --2- 172.21.15.121:0/852386283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a340c4520 0x7f1a3414d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.285+0000 7f1a3b7a9700 1 --2- 172.21.15.121:0/852386283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a340a6e70 0x7f1a340c1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.285+0000 7f1a3b7a9700 1 -- 172.21.15.121:0/852386283 >> 172.21.15.121:0/852386283 conn(0x7f1a34067480 msgr2=0x7f1a340bd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:46.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.285+0000 7f1a3b7a9700 1 -- 172.21.15.121:0/852386283 shutdown_connections 2023-11-15T03:06:46.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.285+0000 7f1a3b7a9700 1 -- 172.21.15.121:0/852386283 wait complete. 2023-11-15T03:06:46.298 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:06:46.315 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:06:46.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.369+0000 7f56e62dad00 1 Processor -- start 2023-11-15T03:06:46.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.369+0000 7f56e62dad00 1 -- start start 2023-11-15T03:06:46.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.369+0000 7f56e62dad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603296c0dc0 0x560329674dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.369+0000 7f56e62dad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560329736650 0x560329759cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.369+0000 7f56e62dad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560329736a30 0x56032975a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.369+0000 7f56e62dad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560329622cc0 con 0x560329736650 2023-11-15T03:06:46.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.369+0000 7f56e62dad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603295ed220 con 0x5603296c0dc0 2023-11-15T03:06:46.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.369+0000 7f56e62dad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603296a5790 con 0x560329736a30 2023-11-15T03:06:46.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.369+0000 7f56db3bb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560329736650 0x560329759cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.369+0000 7f56db3bb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560329736650 0x560329759cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38068/0 (socket says 172.21.15.121:38068) 2023-11-15T03:06:46.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.369+0000 7f56db3bb700 1 -- 172.21.15.121:0/2652689154 learned_addr learned my addr 172.21.15.121:0/2652689154 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:46.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.369+0000 7f56e3bbc700 1 --2- 172.21.15.121:0/2652689154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603296c0dc0 0x560329674dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.369+0000 7f56e43bd700 1 --2- 172.21.15.121:0/2652689154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560329736a30 0x56032975a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.369+0000 7f56db3bb700 1 -- 172.21.15.121:0/2652689154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560329736a30 msgr2=0x56032975a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:46.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.369+0000 7f56db3bb700 1 --2- 172.21.15.121:0/2652689154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560329736a30 0x56032975a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.369+0000 7f56db3bb700 1 -- 172.21.15.121:0/2652689154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603296c0dc0 msgr2=0x560329674dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:46.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.369+0000 7f56db3bb700 1 --2- 172.21.15.121:0/2652689154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603296c0dc0 0x560329674dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.369+0000 7f56db3bb700 1 -- 172.21.15.121:0/2652689154 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc005cf0 con 0x560329736650 2023-11-15T03:06:46.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.373+0000 7f56db3bb700 1 --2- 172.21.15.121:0/2652689154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560329736650 0x560329759cf0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7f56cc00f000 tx=0x7f56cc015000).ready entity=mon.0 client_cookie=23c62ae4bf3bfb8b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:46.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.373+0000 7f56e2bba700 1 -- 172.21.15.121:0/2652689154 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc007610 con 0x560329736650 2023-11-15T03:06:46.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.373+0000 7f56e2bba700 1 -- 172.21.15.121:0/2652689154 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56cc007e60 con 0x560329736650 2023-11-15T03:06:46.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.373+0000 7f56e2bba700 1 -- 172.21.15.121:0/2652689154 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc0078e0 con 0x560329736650 2023-11-15T03:06:46.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.373+0000 7f56e62dad00 1 -- 172.21.15.121:0/2652689154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560329736650 msgr2=0x560329759cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.373+0000 7f56e62dad00 1 --2- 172.21.15.121:0/2652689154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560329736650 0x560329759cf0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7f56cc00f000 tx=0x7f56cc015000).stop 2023-11-15T03:06:46.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.373+0000 7f56e62dad00 1 -- 172.21.15.121:0/2652689154 shutdown_connections 2023-11-15T03:06:46.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.373+0000 7f56e62dad00 1 --2- 172.21.15.121:0/2652689154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560329736a30 0x56032975a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.373+0000 7f56e62dad00 1 --2- 172.21.15.121:0/2652689154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560329736650 0x560329759cf0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.373+0000 7f56e62dad00 1 --2- 172.21.15.121:0/2652689154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603296c0dc0 0x560329674dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.373+0000 7f56e62dad00 1 -- 172.21.15.121:0/2652689154 >> 172.21.15.121:0/2652689154 conn(0x5603296be8f0 msgr2=0x5603296b86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:46.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.373+0000 7f56e62dad00 1 -- 172.21.15.121:0/2652689154 shutdown_connections 2023-11-15T03:06:46.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.373+0000 7f56e62dad00 1 -- 172.21.15.121:0/2652689154 wait complete. 2023-11-15T03:06:46.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.373+0000 7f56e62dad00 1 Processor -- start 2023-11-15T03:06:46.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.373+0000 7f56e62dad00 1 -- start start 2023-11-15T03:06:46.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56e62dad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5603296c0dc0 0x56032974f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56e62dad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560329736650 0x560329731730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56e62dad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560329736a30 0x560329731c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56e62dad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56032974c950 con 0x560329736650 2023-11-15T03:06:46.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56e62dad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560329750920 con 0x560329736a30 2023-11-15T03:06:46.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56e62dad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56032974fda0 con 0x5603296c0dc0 2023-11-15T03:06:46.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56db3bb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560329736650 0x560329731730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56db3bb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560329736650 0x560329731730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38076/0 (socket says 172.21.15.121:38076) 2023-11-15T03:06:46.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 learned_addr learned my addr 172.21.15.121:0/768387232 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:46.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56e43bd700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560329736a30 0x560329731c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56e3bbc700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5603296c0dc0 0x56032974f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5603296c0dc0 msgr2=0x56032974f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56db3bb700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5603296c0dc0 0x56032974f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560329736a30 msgr2=0x560329731c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56db3bb700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560329736a30 0x560329731c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560329616770 con 0x560329736650 2023-11-15T03:06:46.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56e3bbc700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5603296c0dc0 0x56032974f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:46.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56db3bb700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560329736650 0x560329731730 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7f56cc00c010 tx=0x7f56cc01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:46.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56e13b7700 1 -- 172.21.15.121:0/768387232 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc0053b0 con 0x560329736650 2023-11-15T03:06:46.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56e13b7700 1 -- 172.21.15.121:0/768387232 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56cc007400 con 0x560329736650 2023-11-15T03:06:46.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56e13b7700 1 -- 172.21.15.121:0/768387232 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc027690 con 0x560329736650 2023-11-15T03:06:46.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5603295e5a70 con 0x560329736650 2023-11-15T03:06:46.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.377+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603295f1010 con 0x560329736650 2023-11-15T03:06:46.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.381+0000 7f56e13b7700 1 -- 172.21.15.121:0/768387232 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f56cc027a80 con 0x560329736650 2023-11-15T03:06:46.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.381+0000 7f56e13b7700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f56c4059020 0x7f56c405b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.381+0000 7f56e13b7700 1 -- 172.21.15.121:0/768387232 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7814+0+0 (secure 0 0 0) 0x7f56cc076a30 con 0x560329736650 2023-11-15T03:06:46.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.381+0000 7f56e3bbc700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f56c4059020 0x7f56c405b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.397 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:06:46.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.381+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56032974f860 0x560329732f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.381+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032974d6f0 con 0x56032974f860 2023-11-15T03:06:46.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.385+0000 7f56e3bbc700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f56c4059020 0x7f56c405b4e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f56d4004170 tx=0x7f56d4000bc0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:46.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.385+0000 7f56e43bd700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56032974f860 0x560329732f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.385+0000 7f56e43bd700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56032974f860 0x560329732f10 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:46.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.385+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v112'209 uv202 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x56032974f860 2023-11-15T03:06:46.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.385+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x560329733400 con 0x56032974f860 2023-11-15T03:06:46.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.389+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v112'210 uv210 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x56032974f860 2023-11-15T03:06:46.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.389+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x5603297337e0 con 0x56032974f860 2023-11-15T03:06:46.400 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:06:46.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.393+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v112'211 uv211 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x56032974f860 2023-11-15T03:06:46.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.393+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032976ba50 con 0x56032974f860 2023-11-15T03:06:46.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.393+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v112'212 uv212 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x56032974f860 2023-11-15T03:06:46.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.393+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032976be00 con 0x56032974f860 2023-11-15T03:06:46.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.397+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v112'213 uv213 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x56032974f860 2023-11-15T03:06:46.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.397+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032976aa40 con 0x56032974f860 2023-11-15T03:06:46.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.401+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v112'214 uv214 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x56032974f860 2023-11-15T03:06:46.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.401+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032976ae70 con 0x56032974f860 2023-11-15T03:06:46.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.401+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v112'215 uv208 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x56032974f860 2023-11-15T03:06:46.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.401+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x56032976b270 con 0x56032974f860 2023-11-15T03:06:46.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.405+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v112'216 uv216 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x56032974f860 2023-11-15T03:06:46.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.405+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032976b270 con 0x56032974f860 2023-11-15T03:06:46.413 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:06:46.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.405+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v112'217 uv217 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x56032974f860 2023-11-15T03:06:46.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.405+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032976b670 con 0x56032974f860 2023-11-15T03:06:46.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.409+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v112'218 uv218 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x56032974f860 2023-11-15T03:06:46.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.409+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032976b220 con 0x56032974f860 2023-11-15T03:06:46.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.413+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v112'219 uv219 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x56032974f860 2023-11-15T03:06:46.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.413+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032976b220 con 0x56032974f860 2023-11-15T03:06:46.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.413+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v112'220 uv220 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x56032974f860 2023-11-15T03:06:46.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.413+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56032976b620 0x56032976e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.413+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032976b220 con 0x56032976b620 2023-11-15T03:06:46.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.417+0000 7f56e3bbc700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56032976b620 0x56032976e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.417+0000 7f56e3bbc700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56032976b620 0x56032976e6d0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:46.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.421+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v112'328 uv315 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.421+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x56032976ebc0 con 0x56032976b620 2023-11-15T03:06:46.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.425+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v112'329 uv329 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.425+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032976ef70 con 0x56032976b620 2023-11-15T03:06:46.433 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:06:46.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.425+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v112'330 uv330 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.425+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329770330 con 0x56032976b620 2023-11-15T03:06:46.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.429+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v112'331 uv331 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.429+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x5603297706e0 con 0x56032976b620 2023-11-15T03:06:46.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.433+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v112'332 uv332 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.433+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032976f320 con 0x56032976b620 2023-11-15T03:06:46.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.433+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v112'333 uv333 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.433+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032976fc10 con 0x56032976b620 2023-11-15T03:06:46.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.437+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v112'334 uv334 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.437+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329770a90 con 0x56032976b620 2023-11-15T03:06:46.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.441+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v112'335 uv321 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.441+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x560329770a90 con 0x56032976b620 2023-11-15T03:06:46.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.445+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v112'336 uv336 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.445+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329770a90 con 0x56032976b620 2023-11-15T03:06:46.452 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:06:46.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.445+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v112'337 uv337 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.445+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329771aa0 con 0x56032976b620 2023-11-15T03:06:46.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.449+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v112'338 uv338 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.449+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329771aa0 con 0x56032976b620 2023-11-15T03:06:46.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.453+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v112'339 uv339 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.453+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329770a90 con 0x56032976b620 2023-11-15T03:06:46.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.453+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v112'340 uv340 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.453+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329770a90 con 0x56032976b620 2023-11-15T03:06:46.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.457+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v112'341 uv327 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.457+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x560329770a90 con 0x56032976b620 2023-11-15T03:06:46.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.461+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v112'342 uv342 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.461+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329770a90 con 0x56032976b620 2023-11-15T03:06:46.468 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:06:46.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.461+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v112'343 uv343 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.461+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329771aa0 con 0x56032976b620 2023-11-15T03:06:46.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.465+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v112'344 uv344 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.465+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329771aa0 con 0x56032976b620 2023-11-15T03:06:46.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.465+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v112'345 uv345 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.469+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329770a90 con 0x56032976b620 2023-11-15T03:06:46.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.469+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v112'346 uv346 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.469+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x560329770fa0 0x560329773390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.469+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329773880 con 0x560329770fa0 2023-11-15T03:06:46.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.469+0000 7f56db3bb700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x560329770fa0 0x560329773390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.473+0000 7f56db3bb700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x560329770fa0 0x560329773390 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:46.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.473+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v112'180 uv174 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x560329770fa0 2023-11-15T03:06:46.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.473+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x560329773c30 con 0x560329770fa0 2023-11-15T03:06:46.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.477+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v112'181 uv181 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x560329770fa0 2023-11-15T03:06:46.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.477+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329773fe0 con 0x560329770fa0 2023-11-15T03:06:46.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.481+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v112'182 uv182 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x560329770fa0 2023-11-15T03:06:46.487 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:06:46.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.481+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329774880 con 0x560329770fa0 2023-11-15T03:06:46.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.485+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v112'183 uv183 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x560329770fa0 2023-11-15T03:06:46.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.485+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329774c30 con 0x560329770fa0 2023-11-15T03:06:46.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.485+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v112'184 uv184 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x560329770fa0 2023-11-15T03:06:46.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.485+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329774fe0 con 0x560329770fa0 2023-11-15T03:06:46.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.489+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v112'185 uv179 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x560329770fa0 2023-11-15T03:06:46.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.489+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x560329775390 con 0x560329770fa0 2023-11-15T03:06:46.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.489+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v112'186 uv186 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x560329770fa0 2023-11-15T03:06:46.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.489+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329775740 con 0x560329770fa0 2023-11-15T03:06:46.499 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:06:46.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.493+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v112'187 uv187 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x560329770fa0 2023-11-15T03:06:46.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.493+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329775740 con 0x560329770fa0 2023-11-15T03:06:46.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.497+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v112'188 uv188 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x560329770fa0 2023-11-15T03:06:46.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.497+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329775740 con 0x560329770fa0 2023-11-15T03:06:46.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.497+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v112'189 uv189 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x560329770fa0 2023-11-15T03:06:46.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.497+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x560329775c50 0x560329778040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.497+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329778530 con 0x560329775c50 2023-11-15T03:06:46.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.501+0000 7f56e43bd700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x560329775c50 0x560329778040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.501+0000 7f56e43bd700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x560329775c50 0x560329778040 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:46.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.501+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v112'216 uv215 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.501+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x560329778910 con 0x560329775c50 2023-11-15T03:06:46.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.505+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v112'217 uv217 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.505+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329778910 con 0x560329775c50 2023-11-15T03:06:46.515 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:06:46.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.509+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v112'218 uv218 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.509+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977c9a0 con 0x560329775c50 2023-11-15T03:06:46.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.513+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v112'219 uv219 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.513+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977c9a0 con 0x560329775c50 2023-11-15T03:06:46.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.517+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v112'220 uv220 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.517+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329778910 con 0x560329775c50 2023-11-15T03:06:46.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.517+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v112'221 uv214 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.517+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x560329778910 con 0x560329775c50 2023-11-15T03:06:46.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.521+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v112'222 uv222 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.521+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329778910 con 0x560329775c50 2023-11-15T03:06:46.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.525+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v112'223 uv223 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.530 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:06:46.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.525+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329779920 con 0x560329775c50 2023-11-15T03:06:46.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.525+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v112'224 uv224 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.525+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329779920 con 0x560329775c50 2023-11-15T03:06:46.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.529+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v112'225 uv225 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.529+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329778910 con 0x560329775c50 2023-11-15T03:06:46.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.533+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v112'226 uv226 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.533+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x560329778f90 0x56032977b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.533+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977b9a0 con 0x560329778f90 2023-11-15T03:06:46.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.533+0000 7f56e3bbc700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x560329778f90 0x56032977b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.533+0000 7f56e3bbc700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x560329778f90 0x56032977b380 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:46.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.537+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v112'105 uv104 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x560329778f90 2023-11-15T03:06:46.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.537+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x56032977bae0 con 0x560329778f90 2023-11-15T03:06:46.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.537+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v112'106 uv106 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x560329778f90 2023-11-15T03:06:46.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.537+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977bf10 con 0x560329778f90 2023-11-15T03:06:46.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.541+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v112'107 uv107 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x560329778f90 2023-11-15T03:06:46.546 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:06:46.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.541+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977cf20 con 0x560329778f90 2023-11-15T03:06:46.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.541+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v112'108 uv108 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x560329778f90 2023-11-15T03:06:46.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.541+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977cf20 con 0x560329778f90 2023-11-15T03:06:46.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.545+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v112'109 uv109 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x560329778f90 2023-11-15T03:06:46.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.545+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977bf10 con 0x560329778f90 2023-11-15T03:06:46.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.545+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v112'110 uv110 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x560329778f90 2023-11-15T03:06:46.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.545+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56032977c420 0x56032977e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.545+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977eec0 con 0x56032977c420 2023-11-15T03:06:46.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.545+0000 7f56db3bb700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56032977c420 0x56032977e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.549+0000 7f56db3bb700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56032977c420 0x56032977e810 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:46.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.549+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v112'76 uv75 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.549+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x56032977f2a0 con 0x56032977c420 2023-11-15T03:06:46.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.553+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v112'77 uv77 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.553+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977f2a0 con 0x56032977c420 2023-11-15T03:06:46.562 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:06:46.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.557+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v112'78 uv78 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.557+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977f2a0 con 0x56032977c420 2023-11-15T03:06:46.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.557+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v112'79 uv79 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.557+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977f2a0 con 0x56032977c420 2023-11-15T03:06:46.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.561+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v112'105 uv104 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.561+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x56032977f2a0 con 0x56032977c420 2023-11-15T03:06:46.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.565+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v112'106 uv106 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.565+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977f2a0 con 0x56032977c420 2023-11-15T03:06:46.574 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:06:46.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.569+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v112'107 uv107 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.569+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x5603297802b0 con 0x56032977c420 2023-11-15T03:06:46.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.569+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v112'108 uv108 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.569+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x5603297802b0 con 0x56032977c420 2023-11-15T03:06:46.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.573+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v112'109 uv109 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.577+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977f2a0 con 0x56032977c420 2023-11-15T03:06:46.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.577+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v112'110 uv110 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.577+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977f2a0 con 0x56032976b620 2023-11-15T03:06:46.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.581+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v112'193 uv187 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.581+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x56032977f2a0 con 0x56032976b620 2023-11-15T03:06:46.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.585+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v112'194 uv194 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.585+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977f2a0 con 0x56032976b620 2023-11-15T03:06:46.593 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:06:46.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.585+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v112'195 uv195 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.585+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x5603297802f0 con 0x56032976b620 2023-11-15T03:06:46.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.589+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v112'196 uv196 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.589+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x5603297802b0 con 0x56032976b620 2023-11-15T03:06:46.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.593+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v112'197 uv197 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.593+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977f2a0 con 0x56032976b620 2023-11-15T03:06:46.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.597+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v112'198 uv198 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.597+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977f2a0 con 0x56032976b620 2023-11-15T03:06:46.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.601+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v112'199 uv192 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.601+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x56032977f2a0 con 0x56032976b620 2023-11-15T03:06:46.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.605+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v112'200 uv200 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.605+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977f2a0 con 0x56032976b620 2023-11-15T03:06:46.614 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:06:46.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.605+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v112'201 uv201 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.605+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977f2a0 con 0x56032976b620 2023-11-15T03:06:46.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.609+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v112'202 uv202 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.609+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329774410 con 0x56032976b620 2023-11-15T03:06:46.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.613+0000 7f56e3bbc700 1 -- 172.21.15.121:0/768387232 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v112'203 uv203 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56d4010c50 con 0x56032976b620 2023-11-15T03:06:46.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.613+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329774410 con 0x560329770fa0 2023-11-15T03:06:46.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.613+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v112'86 uv85 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x560329770fa0 2023-11-15T03:06:46.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.613+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x560329774410 con 0x560329770fa0 2023-11-15T03:06:46.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.617+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v112'87 uv87 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x560329770fa0 2023-11-15T03:06:46.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.617+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329774410 con 0x560329770fa0 2023-11-15T03:06:46.626 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:06:46.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.621+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v112'88 uv88 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x560329770fa0 2023-11-15T03:06:46.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.621+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329774410 con 0x560329770fa0 2023-11-15T03:06:46.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.621+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v112'89 uv89 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x560329770fa0 2023-11-15T03:06:46.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.621+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329774410 con 0x560329770fa0 2023-11-15T03:06:46.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.625+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v112'90 uv90 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x560329770fa0 2023-11-15T03:06:46.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.625+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329774410 con 0x56032977c420 2023-11-15T03:06:46.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.629+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v112'105 uv104 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.629+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x560329774410 con 0x56032977c420 2023-11-15T03:06:46.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.633+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v112'106 uv106 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.633+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329774410 con 0x56032977c420 2023-11-15T03:06:46.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.633+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v112'107 uv107 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.641 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:06:46.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.633+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329774410 con 0x56032977c420 2023-11-15T03:06:46.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.637+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v112'108 uv108 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.637+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329774410 con 0x56032977c420 2023-11-15T03:06:46.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.641+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v112'109 uv109 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.641+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329774410 con 0x56032977c420 2023-11-15T03:06:46.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.641+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v112'110 uv110 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.641+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329774410 con 0x56032977c420 2023-11-15T03:06:46.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.645+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v112'87 uv86 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.649+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x560329774410 con 0x56032977c420 2023-11-15T03:06:46.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.649+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v112'88 uv88 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.649+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329774410 con 0x56032977c420 2023-11-15T03:06:46.660 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:06:46.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.653+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v112'89 uv89 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.653+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329774410 con 0x56032977c420 2023-11-15T03:06:46.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.657+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v112'90 uv90 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.657+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329774410 con 0x56032977c420 2023-11-15T03:06:46.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.661+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v112'91 uv91 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.661+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329774410 con 0x56032977c420 2023-11-15T03:06:46.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.665+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v112'86 uv85 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.665+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x560329774410 con 0x56032977c420 2023-11-15T03:06:46.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.669+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v112'87 uv87 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.669+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329774410 con 0x56032977c420 2023-11-15T03:06:46.677 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:06:46.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.669+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v112'88 uv88 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.669+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x56032977f790 con 0x56032977c420 2023-11-15T03:06:46.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.673+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v112'89 uv89 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.673+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x5603297821b0 con 0x56032977c420 2023-11-15T03:06:46.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.677+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v112'90 uv90 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.677+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56032977f2a0 0x560329782800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.677+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329781690 con 0x56032977f2a0 2023-11-15T03:06:46.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.677+0000 7f56e43bd700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56032977f2a0 0x560329782800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.677+0000 7f56e43bd700 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56032977f2a0 0x560329782800 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:46.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.681+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v112'70 uv69 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x56032977f2a0 2023-11-15T03:06:46.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.681+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x560329781690 con 0x56032977f2a0 2023-11-15T03:06:46.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.685+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v112'71 uv71 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x56032977f2a0 2023-11-15T03:06:46.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.685+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329781690 con 0x56032977f2a0 2023-11-15T03:06:46.695 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:06:46.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.689+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v112'72 uv72 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x56032977f2a0 2023-11-15T03:06:46.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.689+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329781690 con 0x56032977f2a0 2023-11-15T03:06:46.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.689+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v112'73 uv73 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x56032977f2a0 2023-11-15T03:06:46.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.689+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329781690 con 0x56032977c420 2023-11-15T03:06:46.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.693+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v112'105 uv104 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.693+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x560329781690 con 0x56032977c420 2023-11-15T03:06:46.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.697+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v112'106 uv106 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.697+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329781690 con 0x56032977c420 2023-11-15T03:06:46.705 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:06:46.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.697+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v112'107 uv107 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.697+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329781690 con 0x56032977c420 2023-11-15T03:06:46.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.701+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v112'108 uv108 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.701+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329781690 con 0x56032977c420 2023-11-15T03:06:46.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.705+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v112'109 uv109 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.705+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329781690 con 0x56032977c420 2023-11-15T03:06:46.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.705+0000 7f56db3bb700 1 -- 172.21.15.121:0/768387232 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v112'110 uv110 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56cc02d020 con 0x56032977c420 2023-11-15T03:06:46.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.705+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329781690 con 0x560329775c50 2023-11-15T03:06:46.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.709+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v112'89 uv88 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.709+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x560329781690 con 0x560329775c50 2023-11-15T03:06:46.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.713+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v112'90 uv90 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.713+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329781690 con 0x560329775c50 2023-11-15T03:06:46.721 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:06:46.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.713+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v112'91 uv91 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.713+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329781690 con 0x560329775c50 2023-11-15T03:06:46.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.717+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v112'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.717+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329782cf0 con 0x560329775c50 2023-11-15T03:06:46.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.721+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v112'93 uv93 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.721+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329782cf0 con 0x560329775c50 2023-11-15T03:06:46.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.721+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v112'208 uv200 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.725+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x560329782cf0 con 0x560329775c50 2023-11-15T03:06:46.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.725+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v112'209 uv209 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.725+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329782cf0 con 0x560329775c50 2023-11-15T03:06:46.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.729+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v112'210 uv210 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.735 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:06:46.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.729+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329782cf0 con 0x560329775c50 2023-11-15T03:06:46.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.733+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v112'211 uv211 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.733+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329782cf0 con 0x560329775c50 2023-11-15T03:06:46.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.733+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v112'212 uv212 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.733+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329782cf0 con 0x560329775c50 2023-11-15T03:06:46.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.737+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v112'213 uv207 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.737+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e112) v8 -- 0x560329782cf0 con 0x560329775c50 2023-11-15T03:06:46.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.741+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v112'214 uv214 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.741+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329781690 con 0x560329775c50 2023-11-15T03:06:46.748 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:06:46.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.741+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v112'215 uv215 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.741+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329781690 con 0x560329775c50 2023-11-15T03:06:46.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.745+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v112'216 uv216 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.745+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329783d50 con 0x560329775c50 2023-11-15T03:06:46.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.745+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v112'217 uv217 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.745+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x560329782cf0 con 0x560329775c50 2023-11-15T03:06:46.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.749+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v112'218 uv218 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.749+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x5603297831e0 con 0x560329775c50 2023-11-15T03:06:46.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v112'219 uv219 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f56dc002180 con 0x560329775c50 2023-11-15T03:06:46.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x560329770fa0 msgr2=0x560329773390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x560329770fa0 0x560329773390 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56032974f860 msgr2=0x560329732f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x56032974f860 0x560329732f10 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x560329778f90 msgr2=0x56032977b380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x560329778f90 0x56032977b380 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56032976b620 msgr2=0x56032976e6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x56032976b620 0x56032976e6d0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56032977f2a0 msgr2=0x560329782800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56032977f2a0 0x560329782800 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x560329775c50 msgr2=0x560329778040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x560329775c50 0x560329778040 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 reap_dead start 2023-11-15T03:06:46.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e43bd700 1 -- 172.21.15.121:0/768387232 reap_dead start 2023-11-15T03:06:46.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56032977c420 msgr2=0x56032977e810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56032977c420 0x56032977e810 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f56c4059020 msgr2=0x7f56c405b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f56c4059020 0x7f56c405b4e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f56d4004170 tx=0x7f56d4000bc0).stop 2023-11-15T03:06:46.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560329736650 msgr2=0x560329731730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560329736650 0x560329731730 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7f56cc00c010 tx=0x7f56cc01a000).stop 2023-11-15T03:06:46.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 shutdown_connections 2023-11-15T03:06:46.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x56032977c420 0x56032977e810 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x560329775c50 0x560329778040 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f56c4059020 0x7f56c405b4e0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 --2- 172.21.15.121:0/768387232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560329736650 0x560329731730 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 >> 172.21.15.121:0/768387232 conn(0x5603296be8f0 msgr2=0x5603296bdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:46.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 shutdown_connections 2023-11-15T03:06:46.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.753+0000 7f56e62dad00 1 -- 172.21.15.121:0/768387232 wait complete. 2023-11-15T03:06:46.767 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 500.00000000 - 2023-11-15T03:06:46.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.821+0000 7ffbabcffd00 1 Processor -- start 2023-11-15T03:06:46.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.821+0000 7ffbabcffd00 1 -- start start 2023-11-15T03:06:46.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffbabcffd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56442309e6c0 0x5644230e44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffbabcffd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5644230e8b00 0x564423170080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffbabcffd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564423177670 0x564423170570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffbabcffd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564423017220 con 0x56442309e6c0 2023-11-15T03:06:46.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffbabcffd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564423170a60 con 0x564423177670 2023-11-15T03:06:46.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffbabcffd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564423178260 con 0x5644230e8b00 2023-11-15T03:06:46.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffba9645700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56442309e6c0 0x5644230e44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffba9645700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56442309e6c0 0x5644230e44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38080/0 (socket says 172.21.15.121:38080) 2023-11-15T03:06:46.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffba9645700 1 -- 172.21.15.121:0/1378999689 learned_addr learned my addr 172.21.15.121:0/1378999689 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:46.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffba0e44700 1 --2- 172.21.15.121:0/1378999689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5644230e8b00 0x564423170080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffba9e46700 1 --2- 172.21.15.121:0/1378999689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564423177670 0x564423170570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffba9645700 1 -- 172.21.15.121:0/1378999689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5644230e8b00 msgr2=0x564423170080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:46.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffba9645700 1 --2- 172.21.15.121:0/1378999689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5644230e8b00 0x564423170080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffba9645700 1 -- 172.21.15.121:0/1378999689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564423177670 msgr2=0x564423170570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:46.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffba9645700 1 --2- 172.21.15.121:0/1378999689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564423177670 0x564423170570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffba9645700 1 -- 172.21.15.121:0/1378999689 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c005db0 con 0x56442309e6c0 2023-11-15T03:06:46.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffba0e44700 1 --2- 172.21.15.121:0/1378999689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5644230e8b00 0x564423170080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:46.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffba9e46700 1 --2- 172.21.15.121:0/1378999689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564423177670 0x564423170570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:46.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffba9645700 1 --2- 172.21.15.121:0/1378999689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56442309e6c0 0x5644230e44c0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7ffb9c00ae20 tx=0x7ffb9c016000).ready entity=mon.0 client_cookie=2a311e08d491b688 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:46.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffba3fff700 1 -- 172.21.15.121:0/1378999689 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c005350 con 0x56442309e6c0 2023-11-15T03:06:46.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffba3fff700 1 -- 172.21.15.121:0/1378999689 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9c000bc0 con 0x56442309e6c0 2023-11-15T03:06:46.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffba3fff700 1 -- 172.21.15.121:0/1378999689 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c008ab0 con 0x56442309e6c0 2023-11-15T03:06:46.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffbabcffd00 1 -- 172.21.15.121:0/1378999689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56442309e6c0 msgr2=0x5644230e44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffbabcffd00 1 --2- 172.21.15.121:0/1378999689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56442309e6c0 0x5644230e44c0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7ffb9c00ae20 tx=0x7ffb9c016000).stop 2023-11-15T03:06:46.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffbabcffd00 1 -- 172.21.15.121:0/1378999689 shutdown_connections 2023-11-15T03:06:46.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffbabcffd00 1 --2- 172.21.15.121:0/1378999689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564423177670 0x564423170570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffbabcffd00 1 --2- 172.21.15.121:0/1378999689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5644230e8b00 0x564423170080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffbabcffd00 1 --2- 172.21.15.121:0/1378999689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56442309e6c0 0x5644230e44c0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffbabcffd00 1 -- 172.21.15.121:0/1378999689 >> 172.21.15.121:0/1378999689 conn(0x5644230ee080 msgr2=0x5644231799e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:46.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffbabcffd00 1 -- 172.21.15.121:0/1378999689 shutdown_connections 2023-11-15T03:06:46.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.825+0000 7ffbabcffd00 1 -- 172.21.15.121:0/1378999689 wait complete. 2023-11-15T03:06:46.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffbabcffd00 1 Processor -- start 2023-11-15T03:06:46.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffbabcffd00 1 -- start start 2023-11-15T03:06:46.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffbabcffd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644230e8b00 0x5644230e51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffbabcffd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564423177670 0x564423170270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffbabcffd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56442317a8e0 0x5644230d3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffbabcffd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564423169650 con 0x5644230e8b00 2023-11-15T03:06:46.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffbabcffd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644230e5690 con 0x56442317a8e0 2023-11-15T03:06:46.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffbabcffd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56442316bea0 con 0x564423177670 2023-11-15T03:06:46.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffba9645700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644230e8b00 0x5644230e51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffba9645700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644230e8b00 0x5644230e51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38092/0 (socket says 172.21.15.121:38092) 2023-11-15T03:06:46.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffba9645700 1 -- 172.21.15.121:0/1150340438 learned_addr learned my addr 172.21.15.121:0/1150340438 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:46.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffba0e44700 1 --2- 172.21.15.121:0/1150340438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564423177670 0x564423170270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffba9e46700 1 --2- 172.21.15.121:0/1150340438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56442317a8e0 0x5644230d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffba9645700 1 -- 172.21.15.121:0/1150340438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564423177670 msgr2=0x564423170270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffba9645700 1 --2- 172.21.15.121:0/1150340438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564423177670 0x564423170270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffba9645700 1 -- 172.21.15.121:0/1150340438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56442317a8e0 msgr2=0x5644230d3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffba9645700 1 --2- 172.21.15.121:0/1150340438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56442317a8e0 0x5644230d3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffba9645700 1 -- 172.21.15.121:0/1150340438 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564423040a20 con 0x5644230e8b00 2023-11-15T03:06:46.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffba9e46700 1 --2- 172.21.15.121:0/1150340438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56442317a8e0 0x5644230d3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:46.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffba9645700 1 --2- 172.21.15.121:0/1150340438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644230e8b00 0x5644230e51a0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 rx=0x7ffb9c003000 tx=0x7ffb9c01b000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:46.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffba27fc700 1 -- 172.21.15.121:0/1150340438 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c008ab0 con 0x5644230e8b00 2023-11-15T03:06:46.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffba27fc700 1 -- 172.21.15.121:0/1150340438 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9c005350 con 0x5644230e8b00 2023-11-15T03:06:46.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffba27fc700 1 -- 172.21.15.121:0/1150340438 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c013dc0 con 0x5644230e8b00 2023-11-15T03:06:46.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffbabcffd00 1 -- 172.21.15.121:0/1150340438 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564423040770 con 0x5644230e8b00 2023-11-15T03:06:46.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.829+0000 7ffbabcffd00 1 -- 172.21.15.121:0/1150340438 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56442300fa70 con 0x5644230e8b00 2023-11-15T03:06:46.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.833+0000 7ffba27fc700 1 -- 172.21.15.121:0/1150340438 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ffb9c01c610 con 0x5644230e8b00 2023-11-15T03:06:46.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.833+0000 7ffba27fc700 1 --2- 172.21.15.121:0/1150340438 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffb8c058ff0 0x7ffb8c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.833+0000 7ffba27fc700 1 -- 172.21.15.121:0/1150340438 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7814+0+0 (secure 0 0 0) 0x7ffb9c0766c0 con 0x5644230e8b00 2023-11-15T03:06:46.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.833+0000 7ffba0e44700 1 --2- 172.21.15.121:0/1150340438 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffb8c058ff0 0x7ffb8c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.837+0000 7ffbabcffd00 1 --2- 172.21.15.121:0/1150340438 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5644230e7130 0x5644230e7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:46.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.837+0000 7ffbabcffd00 1 -- 172.21.15.121:0/1150340438 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected e112) v8 -- 0x564423169ae0 con 0x5644230e7130 2023-11-15T03:06:46.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.837+0000 7ffba0e44700 1 --2- 172.21.15.121:0/1150340438 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffb8c058ff0 0x7ffb8c05b4b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x56442316bac0 tx=0x7ffb9400d040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:46.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.837+0000 7ffba9e46700 1 --2- 172.21.15.121:0/1150340438 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5644230e7130 0x5644230e7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:46.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.837+0000 7ffba9e46700 1 --2- 172.21.15.121:0/1150340438 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5644230e7130 0x5644230e7510 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:46.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.841+0000 7ffba9e46700 1 -- 172.21.15.121:0/1150340438 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(1 500.00000000 [writefull 0~80] v112'204 uv204 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ffba400c6f0 con 0x5644230e7130 2023-11-15T03:06:46.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.841+0000 7ffbabcffd00 1 -- 172.21.15.121:0/1150340438 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5644230e7130 msgr2=0x5644230e7510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.841+0000 7ffbabcffd00 1 --2- 172.21.15.121:0/1150340438 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5644230e7130 0x5644230e7510 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:46.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.841+0000 7ffbabcffd00 1 -- 172.21.15.121:0/1150340438 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffb8c058ff0 msgr2=0x7ffb8c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.841+0000 7ffbabcffd00 1 --2- 172.21.15.121:0/1150340438 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffb8c058ff0 0x7ffb8c05b4b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x56442316bac0 tx=0x7ffb9400d040).stop 2023-11-15T03:06:46.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.841+0000 7ffbabcffd00 1 -- 172.21.15.121:0/1150340438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644230e8b00 msgr2=0x5644230e51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:46.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.841+0000 7ffbabcffd00 1 --2- 172.21.15.121:0/1150340438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644230e8b00 0x5644230e51a0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 rx=0x7ffb9c003000 tx=0x7ffb9c01b000).stop 2023-11-15T03:06:46.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.841+0000 7ffba9e46700 1 -- 172.21.15.121:0/1150340438 reap_dead start 2023-11-15T03:06:46.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.841+0000 7ffbabcffd00 1 -- 172.21.15.121:0/1150340438 shutdown_connections 2023-11-15T03:06:46.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.841+0000 7ffbabcffd00 1 -- 172.21.15.121:0/1150340438 >> 172.21.15.121:0/1150340438 conn(0x5644230ee080 msgr2=0x5644230eb270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:46.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.841+0000 7ffbabcffd00 1 -- 172.21.15.121:0/1150340438 shutdown_connections 2023-11-15T03:06:46.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:46.841+0000 7ffbabcffd00 1 -- 172.21.15.121:0/1150340438 wait complete. 2023-11-15T03:06:46.875 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:06:47.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.073+0000 7f8e19679700 1 Processor -- start 2023-11-15T03:06:47.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.073+0000 7f8e19679700 1 -- start start 2023-11-15T03:06:47.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.077+0000 7f8e19679700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e140acb60 0x7f8e140acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:47.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.077+0000 7f8e19679700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e140c7e90 con 0x7f8e14065680 2023-11-15T03:06:47.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.077+0000 7f8e19679700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e140ad4d0 con 0x7f8e1414e380 2023-11-15T03:06:47.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.077+0000 7f8e19679700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140c8320 con 0x7f8e140acb60 2023-11-15T03:06:47.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.077+0000 7f8e13fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f8e1414e380 0x7f8e140c5b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:50130/0 (socket says 172.21.15.121:50130) 2023-11-15T03:06:47.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.077+0000 7f8e13fff700 1 -- 172.21.15.121:0/4220786674 learned_addr learned my addr 172.21.15.121:0/4220786674 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:47.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.077+0000 7f8e0b7fe700 1 --2- 172.21.15.121:0/4220786674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e140acb60 0x7f8e140acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:47.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.077+0000 7f8e18e78700 1 --1- 172.21.15.121:0/4220786674 >> v1:172.21.15.121:6789/0 conn(0x7f8e14065680 0x7f8e14065b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39792/0 (socket says 172.21.15.121:39792) 2023-11-15T03:06:47.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.077+0000 7f8e137fe700 1 -- 172.21.15.121:0/4220786674 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 107668324 0 0) 0x7f8e140c7e90 con 0x7f8e14065680 2023-11-15T03:06:47.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.081+0000 7f8e137fe700 1 -- 172.21.15.121:0/4220786674 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e000033f0 con 0x7f8e14065680 2023-11-15T03:06:47.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.081+0000 7f8e137fe700 1 -- 172.21.15.121:0/4220786674 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196502638 0 0) 0x7f8e140ad4d0 con 0x7f8e1414e380 2023-11-15T03:06:47.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.081+0000 7f8e137fe700 1 -- 172.21.15.121:0/4220786674 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e140c7e90 con 0x7f8e1414e380 2023-11-15T03:06:47.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.081+0000 7f8e0b7fe700 1 -- 172.21.15.121:0/4220786674 >> v1:172.21.15.190:6789/0 conn(0x7f8e1414e380 legacy=0x7f8e140c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:47.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.081+0000 7f8e0b7fe700 1 -- 172.21.15.121:0/4220786674 >> v1:172.21.15.121:6789/0 conn(0x7f8e14065680 legacy=0x7f8e14065b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:47.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.081+0000 7f8e0b7fe700 1 -- 172.21.15.121:0/4220786674 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1414e760 con 0x7f8e140acb60 2023-11-15T03:06:47.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.081+0000 7f8e0b7fe700 1 --2- 172.21.15.121:0/4220786674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e140acb60 0x7f8e140acfe0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f8dfc009790 tx=0x7f8dfc009060).ready entity=mon.2 client_cookie=98c593a543acc697 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:47.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.081+0000 7f8e137fe700 1 -- 172.21.15.121:0/4220786674 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc00f510 con 0x7f8e140acb60 2023-11-15T03:06:47.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.081+0000 7f8e137fe700 1 -- 172.21.15.121:0/4220786674 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dfc00f670 con 0x7f8e140acb60 2023-11-15T03:06:47.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.081+0000 7f8e137fe700 1 -- 172.21.15.121:0/4220786674 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc00f9f0 con 0x7f8e140acb60 2023-11-15T03:06:47.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.081+0000 7f8e19679700 1 -- 172.21.15.121:0/4220786674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e140acb60 msgr2=0x7f8e140acfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:47.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.081+0000 7f8e19679700 1 --2- 172.21.15.121:0/4220786674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e140acb60 0x7f8e140acfe0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f8dfc009790 tx=0x7f8dfc009060).stop 2023-11-15T03:06:47.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.081+0000 7f8e19679700 1 -- 172.21.15.121:0/4220786674 shutdown_connections 2023-11-15T03:06:47.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.081+0000 7f8e19679700 1 --2- 172.21.15.121:0/4220786674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e140acb60 0x7f8e140acfe0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:47.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.081+0000 7f8e19679700 1 -- 172.21.15.121:0/4220786674 >> 172.21.15.121:0/4220786674 conn(0x7f8e14067a60 msgr2=0x7f8e14067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:47.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.081+0000 7f8e19679700 1 -- 172.21.15.121:0/4220786674 shutdown_connections 2023-11-15T03:06:47.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.081+0000 7f8e19679700 1 -- 172.21.15.121:0/4220786674 wait complete. 2023-11-15T03:06:47.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.085+0000 7f8e19679700 1 Processor -- start 2023-11-15T03:06:47.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.085+0000 7f8e19679700 1 -- start start 2023-11-15T03:06:47.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.085+0000 7f8e19679700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e14065680 0x7f8e14155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:47.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.085+0000 7f8e19679700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e140acb60 0x7f8e14149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:47.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.085+0000 7f8e19679700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e1414e380 0x7f8e1414b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:47.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.085+0000 7f8e19679700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140adb60 con 0x7f8e14065680 2023-11-15T03:06:47.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.085+0000 7f8e19679700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e141560b0 con 0x7f8e1414e380 2023-11-15T03:06:47.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.085+0000 7f8e19679700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e14156390 con 0x7f8e140acb60 2023-11-15T03:06:47.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.089+0000 7f8e13fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e14065680 0x7f8e14155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:47.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.089+0000 7f8e13fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e14065680 0x7f8e14155ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38100/0 (socket says 172.21.15.121:38100) 2023-11-15T03:06:47.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.089+0000 7f8e13fff700 1 -- 172.21.15.121:0/3129864461 learned_addr learned my addr 172.21.15.121:0/3129864461 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:47.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.089+0000 7f8e0b7fe700 1 --2- 172.21.15.121:0/3129864461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e140acb60 0x7f8e14149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:47.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.089+0000 7f8e18e78700 1 --2- 172.21.15.121:0/3129864461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e1414e380 0x7f8e1414b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:47.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.089+0000 7f8e13fff700 1 -- 172.21.15.121:0/3129864461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e140acb60 msgr2=0x7f8e14149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:47.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.089+0000 7f8e13fff700 1 --2- 172.21.15.121:0/3129864461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e140acb60 0x7f8e14149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:47.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.089+0000 7f8e13fff700 1 -- 172.21.15.121:0/3129864461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e1414e380 msgr2=0x7f8e1414b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:47.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.089+0000 7f8e13fff700 1 --2- 172.21.15.121:0/3129864461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e1414e380 0x7f8e1414b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:47.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.089+0000 7f8e13fff700 1 -- 172.21.15.121:0/3129864461 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dfc0092a0 con 0x7f8e14065680 2023-11-15T03:06:47.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.089+0000 7f8e18e78700 1 --2- 172.21.15.121:0/3129864461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e1414e380 0x7f8e1414b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:47.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.089+0000 7f8e13fff700 1 --2- 172.21.15.121:0/3129864461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e14065680 0x7f8e14155ad0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7f8e0400c820 tx=0x7f8e04001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:47.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.089+0000 7f8e10ff9700 1 -- 172.21.15.121:0/3129864461 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04005d30 con 0x7f8e14065680 2023-11-15T03:06:47.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.089+0000 7f8e10ff9700 1 -- 172.21.15.121:0/3129864461 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e04005e90 con 0x7f8e14065680 2023-11-15T03:06:47.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.089+0000 7f8e19679700 1 -- 172.21.15.121:0/3129864461 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e14156670 con 0x7f8e14065680 2023-11-15T03:06:47.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.089+0000 7f8e10ff9700 1 -- 172.21.15.121:0/3129864461 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04007340 con 0x7f8e14065680 2023-11-15T03:06:47.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.089+0000 7f8e19679700 1 -- 172.21.15.121:0/3129864461 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e14149a60 con 0x7f8e14065680 2023-11-15T03:06:47.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.093+0000 7f8e19679700 1 -- 172.21.15.121:0/3129864461 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e1406bcf0 con 0x7f8e14065680 2023-11-15T03:06:47.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.093+0000 7f8e10ff9700 1 -- 172.21.15.121:0/3129864461 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8e04007770 con 0x7f8e14065680 2023-11-15T03:06:47.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.093+0000 7f8e10ff9700 1 --2- 172.21.15.121:0/3129864461 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8df405a4e0 0x7f8df405c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:47.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.093+0000 7f8e10ff9700 1 -- 172.21.15.121:0/3129864461 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7814+0+0 (secure 0 0 0) 0x7f8e04075070 con 0x7f8e14065680 2023-11-15T03:06:47.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.093+0000 7f8e0b7fe700 1 --2- 172.21.15.121:0/3129864461 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8df405a4e0 0x7f8df405c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:47.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.097+0000 7f8e0b7fe700 1 --2- 172.21.15.121:0/3129864461 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8df405a4e0 0x7f8df405c9a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f8dfc009fd0 tx=0x7f8dfc017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:47.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.101+0000 7f8e10ff9700 1 -- 172.21.15.121:0/3129864461 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8e04048090 con 0x7f8e14065680 2023-11-15T03:06:47.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:47.301+0000 7f8e19679700 1 -- 172.21.15.121:0/3129864461 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f8e14068a70 con 0x7f8e14065680 2023-11-15T03:06:48.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.105+0000 7f8e10ff9700 1 -- 172.21.15.121:0/3129864461 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v197) v1 ==== 162+0+0 (secure 0 0 0) 0x7f8e0404b6b0 con 0x7f8e14065680 2023-11-15T03:06:48.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.109+0000 7f8e19679700 1 -- 172.21.15.121:0/3129864461 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8df405a4e0 msgr2=0x7f8df405c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:48.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.109+0000 7f8e19679700 1 --2- 172.21.15.121:0/3129864461 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8df405a4e0 0x7f8df405c9a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f8dfc009fd0 tx=0x7f8dfc017000).stop 2023-11-15T03:06:48.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.109+0000 7f8e19679700 1 -- 172.21.15.121:0/3129864461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e14065680 msgr2=0x7f8e14155ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:48.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.109+0000 7f8e19679700 1 --2- 172.21.15.121:0/3129864461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e14065680 0x7f8e14155ad0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7f8e0400c820 tx=0x7f8e04001650).stop 2023-11-15T03:06:48.116 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:06:48.109+0000 7f73d1525700 -1 mds.0.purge_queue operator(): Error -22 loading Journaler 2023-11-15T03:06:48.116 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:06:48.109+0000 7f73d1525700 -1 mds.0.197 unhandled write error (22) Invalid argument, force readonly... 2023-11-15T03:06:48.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.109+0000 7f8e19679700 1 -- 172.21.15.121:0/3129864461 shutdown_connections 2023-11-15T03:06:48.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.109+0000 7f8e19679700 1 --2- 172.21.15.121:0/3129864461 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8df405a4e0 0x7f8df405c9a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:48.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.109+0000 7f8e19679700 1 --2- 172.21.15.121:0/3129864461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e1414e380 0x7f8e1414b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:48.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.109+0000 7f8e19679700 1 --2- 172.21.15.121:0/3129864461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e140acb60 0x7f8e14149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:48.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.109+0000 7f8e19679700 1 --2- 172.21.15.121:0/3129864461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e14065680 0x7f8e14155ad0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:48.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.109+0000 7f8e19679700 1 -- 172.21.15.121:0/3129864461 >> 172.21.15.121:0/3129864461 conn(0x7f8e14067a60 msgr2=0x7f8e14149df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:48.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.109+0000 7f8e19679700 1 -- 172.21.15.121:0/3129864461 shutdown_connections 2023-11-15T03:06:48.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.109+0000 7f8e19679700 1 -- 172.21.15.121:0/3129864461 wait complete. 2023-11-15T03:06:48.119 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:06:48.138 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:48.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.305+0000 7fd67f1b9700 1 Processor -- start 2023-11-15T03:06:48.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.305+0000 7fd67f1b9700 1 -- start start 2023-11-15T03:06:48.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.305+0000 7fd67f1b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd6780cafb0 0x7fd678066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:48.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.305+0000 7fd67f1b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6780ad3a0 con 0x7fd678067270 2023-11-15T03:06:48.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.305+0000 7fd67f1b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6780ad650 con 0x7fd6780cabd0 2023-11-15T03:06:48.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.305+0000 7fd67f1b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6780cb410 con 0x7fd6780cafb0 2023-11-15T03:06:48.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.305+0000 7fd67d9b6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd6780cafb0 0x7fd678066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:48.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.305+0000 7fd67d9b6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd6780cafb0 0x7fd678066d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38108/0 (socket says 172.21.15.121:38108) 2023-11-15T03:06:48.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.305+0000 7fd67d9b6700 1 -- 172.21.15.121:0/1074030760 learned_addr learned my addr 172.21.15.121:0/1074030760 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:48.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.305+0000 7fd67d9b6700 1 -- 172.21.15.121:0/1074030760 >> v1:172.21.15.190:6789/0 conn(0x7fd6780cabd0 legacy=0x7fd6780acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:48.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.305+0000 7fd67d9b6700 1 -- 172.21.15.121:0/1074030760 >> v1:172.21.15.121:6789/0 conn(0x7fd678067270 legacy=0x7fd6780ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:48.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.305+0000 7fd67d9b6700 1 -- 172.21.15.121:0/1074030760 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd678065fc0 con 0x7fd6780cafb0 2023-11-15T03:06:48.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.305+0000 7fd67d9b6700 1 --2- 172.21.15.121:0/1074030760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6780cafb0 0x7fd678066d80 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7fd668006fd0 tx=0x7fd6680090f0).ready entity=mon.0 client_cookie=23f89350993d6ef7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:48.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.309+0000 7fd67d1b5700 1 -- 172.21.15.121:0/1074030760 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66800b600 con 0x7fd6780cafb0 2023-11-15T03:06:48.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.309+0000 7fd67d1b5700 1 -- 172.21.15.121:0/1074030760 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66800b760 con 0x7fd6780cafb0 2023-11-15T03:06:48.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.309+0000 7fd67d1b5700 1 -- 172.21.15.121:0/1074030760 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66800bae0 con 0x7fd6780cafb0 2023-11-15T03:06:48.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.309+0000 7fd67f1b9700 1 -- 172.21.15.121:0/1074030760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6780cafb0 msgr2=0x7fd678066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:48.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.309+0000 7fd67f1b9700 1 --2- 172.21.15.121:0/1074030760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6780cafb0 0x7fd678066d80 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7fd668006fd0 tx=0x7fd6680090f0).stop 2023-11-15T03:06:48.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.309+0000 7fd67f1b9700 1 -- 172.21.15.121:0/1074030760 shutdown_connections 2023-11-15T03:06:48.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.309+0000 7fd67f1b9700 1 --2- 172.21.15.121:0/1074030760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6780cafb0 0x7fd678066d80 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:48.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.309+0000 7fd67f1b9700 1 -- 172.21.15.121:0/1074030760 >> 172.21.15.121:0/1074030760 conn(0x7fd678062190 msgr2=0x7fd6780625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:48.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.309+0000 7fd67f1b9700 1 -- 172.21.15.121:0/1074030760 shutdown_connections 2023-11-15T03:06:48.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.309+0000 7fd67f1b9700 1 -- 172.21.15.121:0/1074030760 wait complete. 2023-11-15T03:06:48.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.309+0000 7fd67f1b9700 1 Processor -- start 2023-11-15T03:06:48.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67f1b9700 1 -- start start 2023-11-15T03:06:48.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67f1b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd678067270 0x7fd678151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:48.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67f1b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6780cabd0 0x7fd678164e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:48.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67f1b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6780cafb0 0x7fd67813d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:48.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67f1b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd678151c00 con 0x7fd678067270 2023-11-15T03:06:48.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67f1b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6780ae1e0 con 0x7fd6780cabd0 2023-11-15T03:06:48.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67f1b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6780aea40 con 0x7fd6780cafb0 2023-11-15T03:06:48.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67e1b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd678067270 0x7fd678151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:48.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67e1b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd678067270 0x7fd678151710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38112/0 (socket says 172.21.15.121:38112) 2023-11-15T03:06:48.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67e1b7700 1 -- 172.21.15.121:0/1935147307 learned_addr learned my addr 172.21.15.121:0/1935147307 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:48.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67d9b6700 1 --2- 172.21.15.121:0/1935147307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6780cabd0 0x7fd678164e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:48.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67e9b8700 1 --2- 172.21.15.121:0/1935147307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6780cafb0 0x7fd67813d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:48.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67e1b7700 1 -- 172.21.15.121:0/1935147307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6780cafb0 msgr2=0x7fd67813d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:48.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67e1b7700 1 --2- 172.21.15.121:0/1935147307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6780cafb0 0x7fd67813d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:48.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67e1b7700 1 -- 172.21.15.121:0/1935147307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6780cabd0 msgr2=0x7fd678164e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:48.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67e1b7700 1 --2- 172.21.15.121:0/1935147307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6780cabd0 0x7fd678164e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:48.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67e1b7700 1 -- 172.21.15.121:0/1935147307 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd668009370 con 0x7fd678067270 2023-11-15T03:06:48.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67d9b6700 1 --2- 172.21.15.121:0/1935147307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6780cabd0 0x7fd678164e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:48.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67e1b7700 1 --2- 172.21.15.121:0/1935147307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd678067270 0x7fd678151710 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7fd67400c840 tx=0x7fd674001660).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:48.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd66e7fc700 1 -- 172.21.15.121:0/1935147307 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd674006a20 con 0x7fd678067270 2023-11-15T03:06:48.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd66e7fc700 1 -- 172.21.15.121:0/1935147307 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd674006b80 con 0x7fd678067270 2023-11-15T03:06:48.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67f1b9700 1 -- 172.21.15.121:0/1935147307 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd678153ad0 con 0x7fd678067270 2023-11-15T03:06:48.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd66e7fc700 1 -- 172.21.15.121:0/1935147307 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd674007020 con 0x7fd678067270 2023-11-15T03:06:48.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.313+0000 7fd67f1b9700 1 -- 172.21.15.121:0/1935147307 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6780ae6d0 con 0x7fd678067270 2023-11-15T03:06:48.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.317+0000 7fd67f1b9700 1 -- 172.21.15.121:0/1935147307 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd67806bcf0 con 0x7fd678067270 2023-11-15T03:06:48.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.317+0000 7fd66e7fc700 1 -- 172.21.15.121:0/1935147307 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd674007410 con 0x7fd678067270 2023-11-15T03:06:48.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.317+0000 7fd66e7fc700 1 --2- 172.21.15.121:0/1935147307 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd65c05a510 0x7fd65c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:48.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.317+0000 7fd66e7fc700 1 -- 172.21.15.121:0/1935147307 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7814+0+0 (secure 0 0 0) 0x7fd674074cb0 con 0x7fd678067270 2023-11-15T03:06:48.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.317+0000 7fd67d9b6700 1 --2- 172.21.15.121:0/1935147307 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd65c05a510 0x7fd65c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:48.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.321+0000 7fd67d9b6700 1 --2- 172.21.15.121:0/1935147307 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd65c05a510 0x7fd65c05c9d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fd668009130 tx=0x7fd668017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:48.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.325+0000 7fd66e7fc700 1 -- 172.21.15.121:0/1935147307 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd674047d30 con 0x7fd678067270 2023-11-15T03:06:48.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.529+0000 7fd67f1b9700 1 -- 172.21.15.121:0/1935147307 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6780662a0 con 0x7fd678067270 2023-11-15T03:06:48.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.529+0000 7fd66e7fc700 1 -- 172.21.15.121:0/1935147307 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 197 v197) v1 ==== 95+0+4505 (secure 0 0 0) 0x7fd67404b350 con 0x7fd678067270 2023-11-15T03:06:48.536 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:48.536 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":197,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":197,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:48.095916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8802},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8802":{"gid":8802,"name":"a","rank":0,"incarnation":197,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/2289093120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2289093120},{"type":"v1","addr":"172.21.15.121:6836","nonce":2289093120}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:48.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.533+0000 7fd67f1b9700 1 -- 172.21.15.121:0/1935147307 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd65c05a510 msgr2=0x7fd65c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:48.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.533+0000 7fd67f1b9700 1 --2- 172.21.15.121:0/1935147307 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd65c05a510 0x7fd65c05c9d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fd668009130 tx=0x7fd668017000).stop 2023-11-15T03:06:48.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.533+0000 7fd67f1b9700 1 -- 172.21.15.121:0/1935147307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd678067270 msgr2=0x7fd678151710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:48.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.533+0000 7fd67f1b9700 1 --2- 172.21.15.121:0/1935147307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd678067270 0x7fd678151710 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7fd67400c840 tx=0x7fd674001660).stop 2023-11-15T03:06:48.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.533+0000 7fd67f1b9700 1 -- 172.21.15.121:0/1935147307 shutdown_connections 2023-11-15T03:06:48.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.533+0000 7fd67f1b9700 1 --2- 172.21.15.121:0/1935147307 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd65c05a510 0x7fd65c05c9d0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:48.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.533+0000 7fd67f1b9700 1 --2- 172.21.15.121:0/1935147307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6780cafb0 0x7fd67813d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:48.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.533+0000 7fd67f1b9700 1 --2- 172.21.15.121:0/1935147307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6780cabd0 0x7fd678164e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:48.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.533+0000 7fd67f1b9700 1 --2- 172.21.15.121:0/1935147307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd678067270 0x7fd678151710 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:48.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.533+0000 7fd67f1b9700 1 -- 172.21.15.121:0/1935147307 >> 172.21.15.121:0/1935147307 conn(0x7fd678062190 msgr2=0x7fd6780c90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:48.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.533+0000 7fd67f1b9700 1 -- 172.21.15.121:0/1935147307 shutdown_connections 2023-11-15T03:06:48.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:48.533+0000 7fd67f1b9700 1 -- 172.21.15.121:0/1935147307 wait complete. 2023-11-15T03:06:48.544 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 197 2023-11-15T03:06:48.965 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:49.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.125+0000 7f44193b9700 1 Processor -- start 2023-11-15T03:06:49.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.125+0000 7f44193b9700 1 -- start start 2023-11-15T03:06:49.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.125+0000 7f44193b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f44140cabd0 0x7f44140cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:49.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.125+0000 7f44193b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44140ad0f0 0x7f44140ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:49.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.125+0000 7f44193b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44140ad650 con 0x7f44140acc90 2023-11-15T03:06:49.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.125+0000 7f44193b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4414065fc0 con 0x7f44140cabd0 2023-11-15T03:06:49.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.125+0000 7f44193b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44140662a0 con 0x7f44140ad0f0 2023-11-15T03:06:49.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.125+0000 7f440bfff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f44140acc90 0x7f4414066d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39816/0 (socket says 172.21.15.121:39816) 2023-11-15T03:06:49.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.125+0000 7f440bfff700 1 -- 172.21.15.121:0/3108257738 learned_addr learned my addr 172.21.15.121:0/3108257738 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:49.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.125+0000 7f4413fff700 1 --2- 172.21.15.121:0/3108257738 >> v2:172.21.15.121:3300/0 conn(0x7f44140cabd0 0x7f44140cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:49.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.129+0000 7f4418bb8700 1 --2- 172.21.15.121:0/3108257738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44140ad0f0 0x7f44140ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:49.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.129+0000 7f44137fe700 1 -- 172.21.15.121:0/3108257738 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3600221962 0 0) 0x7f44140ad650 con 0x7f44140acc90 2023-11-15T03:06:49.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.129+0000 7f44137fe700 1 -- 172.21.15.121:0/3108257738 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44000033f0 con 0x7f44140acc90 2023-11-15T03:06:49.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.129+0000 7f4413fff700 1 -- 172.21.15.121:0/3108257738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44140ad0f0 msgr2=0x7f44140ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:49.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.129+0000 7f4413fff700 1 --2- 172.21.15.121:0/3108257738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44140ad0f0 0x7f44140ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:49.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.129+0000 7f4413fff700 1 -- 172.21.15.121:0/3108257738 >> v1:172.21.15.121:6789/0 conn(0x7f44140acc90 legacy=0x7f4414066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:49.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.129+0000 7f4413fff700 1 -- 172.21.15.121:0/3108257738 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4414066580 con 0x7f44140cabd0 2023-11-15T03:06:49.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.129+0000 7f4413fff700 1 --2- 172.21.15.121:0/3108257738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44140cabd0 0x7f44140cafb0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7f4404009760 tx=0x7f4404009060).ready entity=mon.0 client_cookie=eb8dc6c723b70e47 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:49.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.129+0000 7f44137fe700 1 -- 172.21.15.121:0/3108257738 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440400d4a0 con 0x7f44140cabd0 2023-11-15T03:06:49.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.129+0000 7f44137fe700 1 -- 172.21.15.121:0/3108257738 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440400d600 con 0x7f44140cabd0 2023-11-15T03:06:49.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.129+0000 7f44137fe700 1 -- 172.21.15.121:0/3108257738 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440400d980 con 0x7f44140cabd0 2023-11-15T03:06:49.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.129+0000 7f44193b9700 1 -- 172.21.15.121:0/3108257738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44140cabd0 msgr2=0x7f44140cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:49.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.129+0000 7f44193b9700 1 --2- 172.21.15.121:0/3108257738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44140cabd0 0x7f44140cafb0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7f4404009760 tx=0x7f4404009060).stop 2023-11-15T03:06:49.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.129+0000 7f44193b9700 1 -- 172.21.15.121:0/3108257738 shutdown_connections 2023-11-15T03:06:49.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.129+0000 7f44193b9700 1 --2- 172.21.15.121:0/3108257738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44140ad0f0 0x7f44140ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:49.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.129+0000 7f44193b9700 1 --2- 172.21.15.121:0/3108257738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44140cabd0 0x7f44140cafb0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:49.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.129+0000 7f44193b9700 1 -- 172.21.15.121:0/3108257738 >> 172.21.15.121:0/3108257738 conn(0x7f4414062190 msgr2=0x7f44140625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:49.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.129+0000 7f44193b9700 1 -- 172.21.15.121:0/3108257738 shutdown_connections 2023-11-15T03:06:49.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.129+0000 7f44193b9700 1 -- 172.21.15.121:0/3108257738 wait complete. 2023-11-15T03:06:49.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.133+0000 7f44193b9700 1 Processor -- start 2023-11-15T03:06:49.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.133+0000 7f44193b9700 1 -- start start 2023-11-15T03:06:49.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.133+0000 7f44193b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44140acc90 0x7f4414164f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:49.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.133+0000 7f44193b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44140ad0f0 0x7f44140ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:49.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.133+0000 7f44193b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44140cabd0 0x7f441413c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:49.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.133+0000 7f44193b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44140af230 con 0x7f44140acc90 2023-11-15T03:06:49.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.133+0000 7f44193b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44140af510 con 0x7f44140ad0f0 2023-11-15T03:06:49.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.133+0000 7f44193b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44140af7f0 con 0x7f44140cabd0 2023-11-15T03:06:49.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.137+0000 7f4413fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44140acc90 0x7f4414164f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:49.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.137+0000 7f4413fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44140acc90 0x7f4414164f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38124/0 (socket says 172.21.15.121:38124) 2023-11-15T03:06:49.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.137+0000 7f4413fff700 1 -- 172.21.15.121:0/542487327 learned_addr learned my addr 172.21.15.121:0/542487327 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:49.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.137+0000 7f440bfff700 1 --2- 172.21.15.121:0/542487327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44140ad0f0 0x7f44140ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:49.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.137+0000 7f4418bb8700 1 --2- 172.21.15.121:0/542487327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44140cabd0 0x7f441413c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:49.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.137+0000 7f4413fff700 1 -- 172.21.15.121:0/542487327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44140cabd0 msgr2=0x7f441413c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:49.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.137+0000 7f4413fff700 1 --2- 172.21.15.121:0/542487327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44140cabd0 0x7f441413c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:49.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.137+0000 7f4413fff700 1 -- 172.21.15.121:0/542487327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44140ad0f0 msgr2=0x7f44140ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:49.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.137+0000 7f4413fff700 1 --2- 172.21.15.121:0/542487327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44140ad0f0 0x7f44140ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:49.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.137+0000 7f4413fff700 1 -- 172.21.15.121:0/542487327 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44040092a0 con 0x7f44140acc90 2023-11-15T03:06:49.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.137+0000 7f4413fff700 1 --2- 172.21.15.121:0/542487327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44140acc90 0x7f4414164f90 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f4404009d50 tx=0x7f4404016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:49.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.137+0000 7f4410ff9700 1 -- 172.21.15.121:0/542487327 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4404000bc0 con 0x7f44140acc90 2023-11-15T03:06:49.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.137+0000 7f4410ff9700 1 -- 172.21.15.121:0/542487327 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440400d770 con 0x7f44140acc90 2023-11-15T03:06:49.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.137+0000 7f44193b9700 1 -- 172.21.15.121:0/542487327 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44140afad0 con 0x7f44140acc90 2023-11-15T03:06:49.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.137+0000 7f4410ff9700 1 -- 172.21.15.121:0/542487327 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4404000d20 con 0x7f44140acc90 2023-11-15T03:06:49.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.137+0000 7f44193b9700 1 -- 172.21.15.121:0/542487327 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44140aeed0 con 0x7f44140acc90 2023-11-15T03:06:49.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.137+0000 7f44193b9700 1 -- 172.21.15.121:0/542487327 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44140affd0 con 0x7f44140acc90 2023-11-15T03:06:49.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.141+0000 7f4410ff9700 1 -- 172.21.15.121:0/542487327 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f44040263b0 con 0x7f44140acc90 2023-11-15T03:06:49.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.141+0000 7f4410ff9700 1 --2- 172.21.15.121:0/542487327 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f43f405a510 0x7f43f405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:49.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.141+0000 7f4410ff9700 1 -- 172.21.15.121:0/542487327 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7814+0+0 (secure 0 0 0) 0x7f4404075e80 con 0x7f44140acc90 2023-11-15T03:06:49.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.141+0000 7f440bfff700 1 --2- 172.21.15.121:0/542487327 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f43f405a510 0x7f43f405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:49.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.141+0000 7f440bfff700 1 --2- 172.21.15.121:0/542487327 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f43f405a510 0x7f43f405c9d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f4414157340 tx=0x7f43fc009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:49.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.145+0000 7f4410ff9700 1 -- 172.21.15.121:0/542487327 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4404048f50 con 0x7f44140acc90 2023-11-15T03:06:49.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.341+0000 7f44193b9700 1 -- 172.21.15.121:0/542487327 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4414066860 con 0x7f44140acc90 2023-11-15T03:06:49.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.345+0000 7f4410ff9700 1 -- 172.21.15.121:0/542487327 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 198 v198) v1 ==== 95+0+4508 (secure 0 0 0) 0x7f4404022030 con 0x7f44140acc90 2023-11-15T03:06:49.350 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:49.350 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":198,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":198,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:49.109724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8802},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8802":{"gid":8802,"name":"a","rank":0,"incarnation":197,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.121:6836/2289093120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2289093120},{"type":"v1","addr":"172.21.15.121:6836","nonce":2289093120}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:49.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.345+0000 7f44193b9700 1 -- 172.21.15.121:0/542487327 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f43f405a510 msgr2=0x7f43f405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:49.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.345+0000 7f44193b9700 1 --2- 172.21.15.121:0/542487327 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f43f405a510 0x7f43f405c9d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f4414157340 tx=0x7f43fc009040).stop 2023-11-15T03:06:49.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.345+0000 7f44193b9700 1 -- 172.21.15.121:0/542487327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44140acc90 msgr2=0x7f4414164f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:49.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.345+0000 7f44193b9700 1 --2- 172.21.15.121:0/542487327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44140acc90 0x7f4414164f90 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f4404009d50 tx=0x7f4404016000).stop 2023-11-15T03:06:49.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.345+0000 7f44193b9700 1 -- 172.21.15.121:0/542487327 shutdown_connections 2023-11-15T03:06:49.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.345+0000 7f44193b9700 1 --2- 172.21.15.121:0/542487327 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f43f405a510 0x7f43f405c9d0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:49.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.345+0000 7f44193b9700 1 --2- 172.21.15.121:0/542487327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44140cabd0 0x7f441413c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:49.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.345+0000 7f44193b9700 1 --2- 172.21.15.121:0/542487327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44140ad0f0 0x7f44140ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:49.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.345+0000 7f44193b9700 1 --2- 172.21.15.121:0/542487327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44140acc90 0x7f4414164f90 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:49.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.345+0000 7f44193b9700 1 -- 172.21.15.121:0/542487327 >> 172.21.15.121:0/542487327 conn(0x7f4414062190 msgr2=0x7f441414f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:49.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.345+0000 7f44193b9700 1 -- 172.21.15.121:0/542487327 shutdown_connections 2023-11-15T03:06:49.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:49.345+0000 7f44193b9700 1 -- 172.21.15.121:0/542487327 wait complete. 2023-11-15T03:06:49.356 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 198 2023-11-15T03:06:49.373 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 198, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:06:49.109724+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8802}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8802': {'gid': 8802, 'name': 'a', 'rank': 0, 'incarnation': 197, 'state': 'up:reconnect', 'state_seq': 4, 'addr': '172.21.15.121:6836/2289093120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 2289093120}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 2289093120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:06:49.374 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8802:up:reconnect 2023-11-15T03:06:49.374 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:06:54.376 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:54.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.533+0000 7f358ec79700 1 Processor -- start 2023-11-15T03:06:54.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.533+0000 7f358ec79700 1 -- start start 2023-11-15T03:06:54.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358ec79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35880c3280 0x7f35880c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:54.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358ec79700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3588152140 0x7f35881525c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:54.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358ec79700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35880c1d50 con 0x7f358815be30 2023-11-15T03:06:54.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358ec79700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35880c1860 con 0x7f3588152140 2023-11-15T03:06:54.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358ec79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358815cb80 con 0x7f35880c3280 2023-11-15T03:06:54.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358e478700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f358815be30 0x7f358815c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:39822/0 (socket says 172.21.15.121:39822) 2023-11-15T03:06:54.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358e478700 1 -- 172.21.15.121:0/259345645 learned_addr learned my addr 172.21.15.121:0/259345645 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:54.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358d476700 1 --2- 172.21.15.121:0/259345645 >> v2:172.21.15.190:3300/0 conn(0x7f3588152140 0x7f35881525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:54.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358dc77700 1 --2- 172.21.15.121:0/259345645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35880c3280 0x7f35880c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:54.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358cc75700 1 -- 172.21.15.121:0/259345645 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218147201 0 0) 0x7f35880c1d50 con 0x7f358815be30 2023-11-15T03:06:54.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358cc75700 1 -- 172.21.15.121:0/259345645 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35740033f0 con 0x7f358815be30 2023-11-15T03:06:54.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358cc75700 1 -- 172.21.15.121:0/259345645 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1096809259 0 0) 0x7f35740033f0 con 0x7f358815be30 2023-11-15T03:06:54.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358cc75700 1 -- 172.21.15.121:0/259345645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35880c3280 msgr2=0x7f35880c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:54.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358cc75700 1 --2- 172.21.15.121:0/259345645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35880c3280 0x7f35880c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:54.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358cc75700 1 -- 172.21.15.121:0/259345645 >> v2:172.21.15.190:3300/0 conn(0x7f3588152140 msgr2=0x7f35881525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:54.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358cc75700 1 --2- 172.21.15.121:0/259345645 >> v2:172.21.15.190:3300/0 conn(0x7f3588152140 0x7f35881525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:54.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358cc75700 1 -- 172.21.15.121:0/259345645 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35880a9240 con 0x7f358815be30 2023-11-15T03:06:54.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358cc75700 1 -- 172.21.15.121:0/259345645 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f35800017f0 con 0x7f358815be30 2023-11-15T03:06:54.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358cc75700 1 -- 172.21.15.121:0/259345645 >> v1:172.21.15.121:6789/0 conn(0x7f358815be30 legacy=0x7f358815c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:54.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358cc75700 1 --2- 172.21.15.121:0/259345645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35740038e0 0x7f3574003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:54.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358cc75700 1 --2- 172.21.15.121:0/259345645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35880c3280 0x7f3574004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:54.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358cc75700 1 --2- 172.21.15.121:0/259345645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3574004b80 0x7f3574005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:54.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358cc75700 1 -- 172.21.15.121:0/259345645 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35880c1860 con 0x7f3574004b80 2023-11-15T03:06:54.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358cc75700 1 -- 172.21.15.121:0/259345645 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358815cb80 con 0x7f35740038e0 2023-11-15T03:06:54.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.537+0000 7f358cc75700 1 -- 172.21.15.121:0/259345645 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357400b8e0 con 0x7f35880c3280 2023-11-15T03:06:54.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.541+0000 7f358e478700 1 --2- 172.21.15.121:0/259345645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3574004b80 0x7f3574005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:54.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.541+0000 7f358dc77700 1 --2- 172.21.15.121:0/259345645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35740038e0 0x7f3574003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:54.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.541+0000 7f358d476700 1 --2- 172.21.15.121:0/259345645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35880c3280 0x7f3574004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:54.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.541+0000 7f358e478700 1 -- 172.21.15.121:0/259345645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35880c3280 msgr2=0x7f3574004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:54.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.541+0000 7f358e478700 1 --2- 172.21.15.121:0/259345645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35880c3280 0x7f3574004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:54.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.541+0000 7f358e478700 1 -- 172.21.15.121:0/259345645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35740038e0 msgr2=0x7f3574003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:54.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.541+0000 7f358e478700 1 --2- 172.21.15.121:0/259345645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35740038e0 0x7f3574003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:54.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.541+0000 7f358e478700 1 -- 172.21.15.121:0/259345645 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35740033f0 con 0x7f3574004b80 2023-11-15T03:06:54.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.541+0000 7f358e478700 1 --2- 172.21.15.121:0/259345645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3574004b80 0x7f3574005000 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7f358000c530 tx=0x7f35880c1020).ready entity=mon.0 client_cookie=a322bfbbf02b1339 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:54.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.541+0000 7f358cc75700 1 -- 172.21.15.121:0/259345645 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580007de0 con 0x7f3574004b80 2023-11-15T03:06:54.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.541+0000 7f358cc75700 1 -- 172.21.15.121:0/259345645 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358000ce60 con 0x7f3574004b80 2023-11-15T03:06:54.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.541+0000 7f358cc75700 1 -- 172.21.15.121:0/259345645 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580005d30 con 0x7f3574004b80 2023-11-15T03:06:54.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.541+0000 7f358ec79700 1 -- 172.21.15.121:0/259345645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3574004b80 msgr2=0x7f3574005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:54.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.541+0000 7f358ec79700 1 --2- 172.21.15.121:0/259345645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3574004b80 0x7f3574005000 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7f358000c530 tx=0x7f35880c1020).stop 2023-11-15T03:06:54.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.541+0000 7f358e478700 1 -- 172.21.15.121:0/259345645 reap_dead start 2023-11-15T03:06:54.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.541+0000 7f358ec79700 1 -- 172.21.15.121:0/259345645 shutdown_connections 2023-11-15T03:06:54.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.541+0000 7f358ec79700 1 -- 172.21.15.121:0/259345645 >> 172.21.15.121:0/259345645 conn(0x7f3588062190 msgr2=0x7f3588062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:54.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.541+0000 7f358ec79700 1 -- 172.21.15.121:0/259345645 shutdown_connections 2023-11-15T03:06:54.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.541+0000 7f358ec79700 1 -- 172.21.15.121:0/259345645 wait complete. 2023-11-15T03:06:54.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358ec79700 1 Processor -- start 2023-11-15T03:06:54.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358ec79700 1 -- start start 2023-11-15T03:06:54.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358ec79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3574004b80 0x7f3588153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:54.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358ec79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3588152820 0x7f35881420e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:54.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358ec79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35880c5660 0x7f35880c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:54.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358ec79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35881576f0 con 0x7f3588152820 2023-11-15T03:06:54.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358ec79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35880c5fd0 con 0x7f35880c5660 2023-11-15T03:06:54.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358ec79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35881536c0 con 0x7f3574004b80 2023-11-15T03:06:54.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358d476700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3588152820 0x7f35881420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:54.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358d476700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3588152820 0x7f35881420e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38142/0 (socket says 172.21.15.121:38142) 2023-11-15T03:06:54.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358d476700 1 -- 172.21.15.121:0/1824621957 learned_addr learned my addr 172.21.15.121:0/1824621957 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:54.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358dc77700 1 --2- 172.21.15.121:0/1824621957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3574004b80 0x7f3588153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:54.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358e478700 1 --2- 172.21.15.121:0/1824621957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35880c5660 0x7f35880c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:54.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358d476700 1 -- 172.21.15.121:0/1824621957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3574004b80 msgr2=0x7f3588153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:54.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358d476700 1 --2- 172.21.15.121:0/1824621957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3574004b80 0x7f3588153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:54.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358d476700 1 -- 172.21.15.121:0/1824621957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35880c5660 msgr2=0x7f35880c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:54.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358d476700 1 --2- 172.21.15.121:0/1824621957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35880c5660 0x7f35880c5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:54.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358d476700 1 -- 172.21.15.121:0/1824621957 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35881539a0 con 0x7f3588152820 2023-11-15T03:06:54.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358dc77700 1 --2- 172.21.15.121:0/1824621957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3574004b80 0x7f3588153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:54.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.545+0000 7f358e478700 1 --2- 172.21.15.121:0/1824621957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35880c5660 0x7f35880c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:54.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.549+0000 7f358d476700 1 --2- 172.21.15.121:0/1824621957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3588152820 0x7f35881420e0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7f3578011940 tx=0x7f3578009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:54.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.549+0000 7f357dffb700 1 -- 172.21.15.121:0/1824621957 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3578011ee0 con 0x7f3588152820 2023-11-15T03:06:54.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.549+0000 7f357dffb700 1 -- 172.21.15.121:0/1824621957 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357800f940 con 0x7f3588152820 2023-11-15T03:06:54.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.549+0000 7f358ec79700 1 -- 172.21.15.121:0/1824621957 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3588157380 con 0x7f3588152820 2023-11-15T03:06:54.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.549+0000 7f357dffb700 1 -- 172.21.15.121:0/1824621957 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357800fcc0 con 0x7f3588152820 2023-11-15T03:06:54.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.549+0000 7f358ec79700 1 -- 172.21.15.121:0/1824621957 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3588153c80 con 0x7f3588152820 2023-11-15T03:06:54.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.549+0000 7f358ec79700 1 -- 172.21.15.121:0/1824621957 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35881578a0 con 0x7f3588152820 2023-11-15T03:06:54.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.549+0000 7f357dffb700 1 -- 172.21.15.121:0/1824621957 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3578014650 con 0x7f3588152820 2023-11-15T03:06:54.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.553+0000 7f357dffb700 1 --2- 172.21.15.121:0/1824621957 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f356c05a540 0x7f356c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:54.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.553+0000 7f357dffb700 1 -- 172.21.15.121:0/1824621957 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7814+0+0 (secure 0 0 0) 0x7f35780757e0 con 0x7f3588152820 2023-11-15T03:06:54.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.553+0000 7f358dc77700 1 --2- 172.21.15.121:0/1824621957 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f356c05a540 0x7f356c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:54.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.553+0000 7f358dc77700 1 --2- 172.21.15.121:0/1824621957 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f356c05a540 0x7f356c05ca00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f3584006d30 tx=0x7f3584008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:54.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.557+0000 7f357dffb700 1 -- 172.21.15.121:0/1824621957 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f35881578a0 con 0x7f3588152820 2023-11-15T03:06:54.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.753+0000 7f358ec79700 1 -- 172.21.15.121:0/1824621957 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f358806bcf0 con 0x7f3588152820 2023-11-15T03:06:54.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.753+0000 7f357dffb700 1 -- 172.21.15.121:0/1824621957 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f3578022020 con 0x7f3588152820 2023-11-15T03:06:54.761 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:54.761 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":200,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:51.125630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8802},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8802":{"gid":8802,"name":"a","rank":0,"incarnation":197,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6836/2289093120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2289093120},{"type":"v1","addr":"172.21.15.121:6836","nonce":2289093120}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:54.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.757+0000 7f358ec79700 1 -- 172.21.15.121:0/1824621957 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f356c05a540 msgr2=0x7f356c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:54.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.757+0000 7f358ec79700 1 --2- 172.21.15.121:0/1824621957 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f356c05a540 0x7f356c05ca00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f3584006d30 tx=0x7f3584008040).stop 2023-11-15T03:06:54.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.757+0000 7f358ec79700 1 -- 172.21.15.121:0/1824621957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3588152820 msgr2=0x7f35881420e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:54.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.757+0000 7f358ec79700 1 --2- 172.21.15.121:0/1824621957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3588152820 0x7f35881420e0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7f3578011940 tx=0x7f3578009060).stop 2023-11-15T03:06:54.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.757+0000 7f358ec79700 1 -- 172.21.15.121:0/1824621957 shutdown_connections 2023-11-15T03:06:54.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.757+0000 7f358ec79700 1 --2- 172.21.15.121:0/1824621957 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f356c05a540 0x7f356c05ca00 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:54.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.757+0000 7f358ec79700 1 --2- 172.21.15.121:0/1824621957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35880c5660 0x7f35880c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:54.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.757+0000 7f358ec79700 1 --2- 172.21.15.121:0/1824621957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3588152820 0x7f35881420e0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:54.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.757+0000 7f358ec79700 1 --2- 172.21.15.121:0/1824621957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3574004b80 0x7f3588153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:54.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.757+0000 7f358ec79700 1 -- 172.21.15.121:0/1824621957 >> 172.21.15.121:0/1824621957 conn(0x7f3588062190 msgr2=0x7f3588155970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:54.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.757+0000 7f358ec79700 1 -- 172.21.15.121:0/1824621957 shutdown_connections 2023-11-15T03:06:54.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:54.757+0000 7f358ec79700 1 -- 172.21.15.121:0/1824621957 wait complete. 2023-11-15T03:06:54.766 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 200 2023-11-15T03:06:55.141 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:06:55.133+0000 7f73d3d2a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:06:55.185 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:55.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.341+0000 7f1f6359e700 1 Processor -- start 2023-11-15T03:06:55.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.341+0000 7f1f6359e700 1 -- start start 2023-11-15T03:06:55.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.341+0000 7f1f6359e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f64152850 0x7f1f640c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.341+0000 7f1f6359e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1f6415be30 0x7f1f6415c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.341+0000 7f1f6359e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f640c1d50 con 0x7f1f640c3280 2023-11-15T03:06:55.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.341+0000 7f1f6359e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f640c1860 con 0x7f1f6415be30 2023-11-15T03:06:55.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.341+0000 7f1f6359e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6415cb80 con 0x7f1f64152850 2023-11-15T03:06:55.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f6259c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f1f640c3280 0x7f1f64152140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44038/0 (socket says 172.21.15.121:44038) 2023-11-15T03:06:55.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f6259c700 1 -- 172.21.15.121:0/2729259704 learned_addr learned my addr 172.21.15.121:0/2729259704 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:55.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f59d9b700 1 --2- 172.21.15.121:0/2729259704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f64152850 0x7f1f640c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f62d9d700 1 --2- 172.21.15.121:0/2729259704 >> v2:172.21.15.190:3300/0 conn(0x7f1f6415be30 0x7f1f6415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f61d9b700 1 -- 172.21.15.121:0/2729259704 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368625416 0 0) 0x7f1f640c1d50 con 0x7f1f640c3280 2023-11-15T03:06:55.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f61d9b700 1 -- 172.21.15.121:0/2729259704 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f500033f0 con 0x7f1f640c3280 2023-11-15T03:06:55.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f59d9b700 1 -- 172.21.15.121:0/2729259704 >> v2:172.21.15.190:3300/0 conn(0x7f1f6415be30 msgr2=0x7f1f6415c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:55.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f59d9b700 1 --2- 172.21.15.121:0/2729259704 >> v2:172.21.15.190:3300/0 conn(0x7f1f6415be30 0x7f1f6415c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f59d9b700 1 -- 172.21.15.121:0/2729259704 >> v1:172.21.15.121:6789/0 conn(0x7f1f640c3280 legacy=0x7f1f64152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:55.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f59d9b700 1 -- 172.21.15.121:0/2729259704 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f640a9240 con 0x7f1f64152850 2023-11-15T03:06:55.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f59d9b700 1 --2- 172.21.15.121:0/2729259704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f64152850 0x7f1f640c3660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f1f4c009760 tx=0x7f1f4c009060).ready entity=mon.2 client_cookie=34f6d9fba0e76ed3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:55.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f61d9b700 1 -- 172.21.15.121:0/2729259704 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00d4a0 con 0x7f1f64152850 2023-11-15T03:06:55.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f61d9b700 1 -- 172.21.15.121:0/2729259704 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c00d600 con 0x7f1f64152850 2023-11-15T03:06:55.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f61d9b700 1 -- 172.21.15.121:0/2729259704 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00d980 con 0x7f1f64152850 2023-11-15T03:06:55.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f6359e700 1 -- 172.21.15.121:0/2729259704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f64152850 msgr2=0x7f1f640c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:55.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f6359e700 1 --2- 172.21.15.121:0/2729259704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f64152850 0x7f1f640c3660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f1f4c009760 tx=0x7f1f4c009060).stop 2023-11-15T03:06:55.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f6359e700 1 -- 172.21.15.121:0/2729259704 shutdown_connections 2023-11-15T03:06:55.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f6359e700 1 --2- 172.21.15.121:0/2729259704 >> v2:172.21.15.190:3300/0 conn(0x7f1f6415be30 0x7f1f6415c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f6359e700 1 --2- 172.21.15.121:0/2729259704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f64152850 0x7f1f640c3660 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f6359e700 1 -- 172.21.15.121:0/2729259704 >> 172.21.15.121:0/2729259704 conn(0x7f1f64062190 msgr2=0x7f1f64062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:55.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.345+0000 7f1f6359e700 1 -- 172.21.15.121:0/2729259704 shutdown_connections 2023-11-15T03:06:55.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.349+0000 7f1f6359e700 1 -- 172.21.15.121:0/2729259704 wait complete. 2023-11-15T03:06:55.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.349+0000 7f1f6359e700 1 Processor -- start 2023-11-15T03:06:55.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f6359e700 1 -- start start 2023-11-15T03:06:55.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f6359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f640c3280 0x7f1f64140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f6359e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f64152850 0x7f1f64159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f6359e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f6415be30 0x7f1f6415a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f6359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f64140720 con 0x7f1f640c3280 2023-11-15T03:06:55.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f6359e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f641687e0 con 0x7f1f6415be30 2023-11-15T03:06:55.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f6359e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f64168ac0 con 0x7f1f64152850 2023-11-15T03:06:55.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f6259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f640c3280 0x7f1f64140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f6259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f640c3280 0x7f1f64140230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54590/0 (socket says 172.21.15.121:54590) 2023-11-15T03:06:55.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f6259c700 1 -- 172.21.15.121:0/3710905541 learned_addr learned my addr 172.21.15.121:0/3710905541 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:55.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f62d9d700 1 --2- 172.21.15.121:0/3710905541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f6415be30 0x7f1f6415a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f59d9b700 1 --2- 172.21.15.121:0/3710905541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f64152850 0x7f1f64159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f6259c700 1 -- 172.21.15.121:0/3710905541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f64152850 msgr2=0x7f1f64159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:55.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f6259c700 1 --2- 172.21.15.121:0/3710905541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f64152850 0x7f1f64159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f6259c700 1 -- 172.21.15.121:0/3710905541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f6415be30 msgr2=0x7f1f6415a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:55.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f6259c700 1 --2- 172.21.15.121:0/3710905541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f6415be30 0x7f1f6415a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f6259c700 1 -- 172.21.15.121:0/3710905541 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c0092a0 con 0x7f1f640c3280 2023-11-15T03:06:55.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f6259c700 1 --2- 172.21.15.121:0/3710905541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f640c3280 0x7f1f64140230 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 rx=0x7f1f5400c410 tx=0x7f1f54007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:55.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f5affd700 1 -- 172.21.15.121:0/3710905541 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f540070d0 con 0x7f1f640c3280 2023-11-15T03:06:55.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f5affd700 1 -- 172.21.15.121:0/3710905541 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f54007230 con 0x7f1f640c3280 2023-11-15T03:06:55.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f6359e700 1 -- 172.21.15.121:0/3710905541 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f64168da0 con 0x7f1f640c3280 2023-11-15T03:06:55.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f5affd700 1 -- 172.21.15.121:0/3710905541 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f540075d0 con 0x7f1f640c3280 2023-11-15T03:06:55.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.353+0000 7f1f6359e700 1 -- 172.21.15.121:0/3710905541 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f64169540 con 0x7f1f640c3280 2023-11-15T03:06:55.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.357+0000 7f1f6359e700 1 -- 172.21.15.121:0/3710905541 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6406bcf0 con 0x7f1f640c3280 2023-11-15T03:06:55.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.357+0000 7f1f5affd700 1 -- 172.21.15.121:0/3710905541 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1f540059b0 con 0x7f1f640c3280 2023-11-15T03:06:55.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.357+0000 7f1f5affd700 1 --2- 172.21.15.121:0/3710905541 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1f4405a510 0x7f1f4405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.361+0000 7f1f5affd700 1 -- 172.21.15.121:0/3710905541 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7814+0+0 (secure 0 0 0) 0x7f1f54074390 con 0x7f1f640c3280 2023-11-15T03:06:55.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.361+0000 7f1f59d9b700 1 --2- 172.21.15.121:0/3710905541 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1f4405a510 0x7f1f4405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.361+0000 7f1f59d9b700 1 --2- 172.21.15.121:0/3710905541 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1f4405a510 0x7f1f4405c9d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f1f4c00d260 tx=0x7f1f4c016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:55.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.365+0000 7f1f5affd700 1 -- 172.21.15.121:0/3710905541 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1f540474d0 con 0x7f1f640c3280 2023-11-15T03:06:55.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.561+0000 7f1f6359e700 1 -- 172.21.15.121:0/3710905541 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f640a7cc0 con 0x7f1f640c3280 2023-11-15T03:06:55.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.561+0000 7f1f5affd700 1 -- 172.21.15.121:0/3710905541 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f1f5404aaf0 con 0x7f1f640c3280 2023-11-15T03:06:55.568 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:55.568 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":200,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:51.125630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8802},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8802":{"gid":8802,"name":"a","rank":0,"incarnation":197,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6836/2289093120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2289093120},{"type":"v1","addr":"172.21.15.121:6836","nonce":2289093120}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:55.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.565+0000 7f1f6359e700 1 -- 172.21.15.121:0/3710905541 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1f4405a510 msgr2=0x7f1f4405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:55.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.565+0000 7f1f6359e700 1 --2- 172.21.15.121:0/3710905541 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1f4405a510 0x7f1f4405c9d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f1f4c00d260 tx=0x7f1f4c016000).stop 2023-11-15T03:06:55.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.565+0000 7f1f6359e700 1 -- 172.21.15.121:0/3710905541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f640c3280 msgr2=0x7f1f64140230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:55.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.565+0000 7f1f6359e700 1 --2- 172.21.15.121:0/3710905541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f640c3280 0x7f1f64140230 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 rx=0x7f1f5400c410 tx=0x7f1f54007c70).stop 2023-11-15T03:06:55.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.565+0000 7f1f6359e700 1 -- 172.21.15.121:0/3710905541 shutdown_connections 2023-11-15T03:06:55.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.565+0000 7f1f6359e700 1 --2- 172.21.15.121:0/3710905541 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1f4405a510 0x7f1f4405c9d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.565+0000 7f1f6359e700 1 --2- 172.21.15.121:0/3710905541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f6415be30 0x7f1f6415a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.565+0000 7f1f6359e700 1 --2- 172.21.15.121:0/3710905541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f64152850 0x7f1f64159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.565+0000 7f1f6359e700 1 --2- 172.21.15.121:0/3710905541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f640c3280 0x7f1f64140230 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.565+0000 7f1f6359e700 1 -- 172.21.15.121:0/3710905541 >> 172.21.15.121:0/3710905541 conn(0x7f1f64062190 msgr2=0x7f1f64062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:55.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.565+0000 7f1f6359e700 1 -- 172.21.15.121:0/3710905541 shutdown_connections 2023-11-15T03:06:55.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.565+0000 7f1f6359e700 1 -- 172.21.15.121:0/3710905541 wait complete. 2023-11-15T03:06:55.574 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 200 2023-11-15T03:06:55.589 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 200, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:06:51.125630+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8802}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8802': {'gid': 8802, 'name': 'a', 'rank': 0, 'incarnation': 197, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.121:6836/2289093120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 2289093120}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 2289093120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:06:55.590 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T03:06:55.590 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-15T03:06:55.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.741+0000 7f55c6179700 1 Processor -- start 2023-11-15T03:06:55.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.745+0000 7f55c6179700 1 -- start start 2023-11-15T03:06:55.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.745+0000 7f55c6179700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f55c00abaa0 0x7f55c00abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.745+0000 7f55c6179700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f55c00bb510 0x7f55c0063570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.745+0000 7f55c6179700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c00c2c00 con 0x7f55c00a8880 2023-11-15T03:06:55.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55c6179700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c0063a60 con 0x7f55c00bb510 2023-11-15T03:06:55.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55c6179700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c00657b0 con 0x7f55c00abaa0 2023-11-15T03:06:55.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55c4976700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f55c00bb510 0x7f55c0063570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55c4976700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f55c00bb510 0x7f55c0063570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54602/0 (socket says 172.21.15.121:54602) 2023-11-15T03:06:55.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55c5177700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f55c00abaa0 0x7f55c00abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55c4976700 1 -- 172.21.15.121:0/1390417038 learned_addr learned my addr 172.21.15.121:0/1390417038 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:55.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55c4976700 1 -- 172.21.15.121:0/1390417038 >> v2:172.21.15.190:3300/0 conn(0x7f55c00abaa0 msgr2=0x7f55c00abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:55.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55c4976700 1 --2- 172.21.15.121:0/1390417038 >> v2:172.21.15.190:3300/0 conn(0x7f55c00abaa0 0x7f55c00abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55c4976700 1 -- 172.21.15.121:0/1390417038 >> v1:172.21.15.190:6789/0 conn(0x7f55c00a8880 legacy=0x7f55c0151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:55.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55c4976700 1 -- 172.21.15.121:0/1390417038 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c0065a90 con 0x7f55c00bb510 2023-11-15T03:06:55.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55c4976700 1 --2- 172.21.15.121:0/1390417038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c00bb510 0x7f55c0063570 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 rx=0x7f55c00599c0 tx=0x7f55b0009060).ready entity=mon.0 client_cookie=c6d226030235e24c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:55.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55b7fff700 1 -- 172.21.15.121:0/1390417038 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b0005880 con 0x7f55c00bb510 2023-11-15T03:06:55.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55b7fff700 1 -- 172.21.15.121:0/1390417038 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b00059e0 con 0x7f55c00bb510 2023-11-15T03:06:55.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55b7fff700 1 -- 172.21.15.121:0/1390417038 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b0005d60 con 0x7f55c00bb510 2023-11-15T03:06:55.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55c6179700 1 -- 172.21.15.121:0/1390417038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c00bb510 msgr2=0x7f55c0063570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:55.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55c6179700 1 --2- 172.21.15.121:0/1390417038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c00bb510 0x7f55c0063570 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 rx=0x7f55c00599c0 tx=0x7f55b0009060).stop 2023-11-15T03:06:55.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55c6179700 1 -- 172.21.15.121:0/1390417038 shutdown_connections 2023-11-15T03:06:55.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55c6179700 1 --2- 172.21.15.121:0/1390417038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c00bb510 0x7f55c0063570 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55c6179700 1 --2- 172.21.15.121:0/1390417038 >> v2:172.21.15.190:3300/0 conn(0x7f55c00abaa0 0x7f55c00abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55c6179700 1 -- 172.21.15.121:0/1390417038 >> 172.21.15.121:0/1390417038 conn(0x7f55c0067a60 msgr2=0x7f55c0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:55.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55c6179700 1 -- 172.21.15.121:0/1390417038 shutdown_connections 2023-11-15T03:06:55.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.749+0000 7f55c6179700 1 -- 172.21.15.121:0/1390417038 wait complete. 2023-11-15T03:06:55.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.753+0000 7f55c6179700 1 Processor -- start 2023-11-15T03:06:55.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.753+0000 7f55c6179700 1 -- start start 2023-11-15T03:06:55.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55c6179700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c00a8880 0x7f55c014da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55c6179700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55c00abaa0 0x7f55c0158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55c6179700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c00bb510 0x7f55c00c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55c6179700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c0157920 con 0x7f55c00a8880 2023-11-15T03:06:55.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55c6179700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c0158780 con 0x7f55c00abaa0 2023-11-15T03:06:55.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55c6179700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c014df30 con 0x7f55c00bb510 2023-11-15T03:06:55.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55c5177700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c00a8880 0x7f55c014da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55c5177700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c00a8880 0x7f55c014da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54606/0 (socket says 172.21.15.121:54606) 2023-11-15T03:06:55.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55c5177700 1 -- 172.21.15.121:0/1538967940 learned_addr learned my addr 172.21.15.121:0/1538967940 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:55.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55c4976700 1 --2- 172.21.15.121:0/1538967940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55c00abaa0 0x7f55c0158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55c5978700 1 --2- 172.21.15.121:0/1538967940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c00bb510 0x7f55c00c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55c5177700 1 -- 172.21.15.121:0/1538967940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c00bb510 msgr2=0x7f55c00c4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:55.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55c5177700 1 --2- 172.21.15.121:0/1538967940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c00bb510 0x7f55c00c4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55c5177700 1 -- 172.21.15.121:0/1538967940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55c00abaa0 msgr2=0x7f55c0158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:55.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55c5177700 1 --2- 172.21.15.121:0/1538967940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55c00abaa0 0x7f55c0158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55c5177700 1 -- 172.21.15.121:0/1538967940 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b0009300 con 0x7f55c00a8880 2023-11-15T03:06:55.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55c5177700 1 --2- 172.21.15.121:0/1538967940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c00a8880 0x7f55c014da40 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7f55bc00e960 tx=0x7f55bc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:55.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55b57fa700 1 -- 172.21.15.121:0/1538967940 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc00c6a0 con 0x7f55c00a8880 2023-11-15T03:06:55.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55b57fa700 1 -- 172.21.15.121:0/1538967940 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55bc00c800 con 0x7f55c00a8880 2023-11-15T03:06:55.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55c6179700 1 -- 172.21.15.121:0/1538967940 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c0157b70 con 0x7f55c00a8880 2023-11-15T03:06:55.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55b57fa700 1 -- 172.21.15.121:0/1538967940 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc00cb80 con 0x7f55c00a8880 2023-11-15T03:06:55.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.757+0000 7f55c6179700 1 -- 172.21.15.121:0/1538967940 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c0158130 con 0x7f55c00a8880 2023-11-15T03:06:55.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.761+0000 7f55b57fa700 1 -- 172.21.15.121:0/1538967940 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f55bc01b400 con 0x7f55c00a8880 2023-11-15T03:06:55.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.761+0000 7f55b57fa700 1 --2- 172.21.15.121:0/1538967940 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f55a405a510 0x7f55a405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.761+0000 7f55b57fa700 1 -- 172.21.15.121:0/1538967940 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7814+0+0 (secure 0 0 0) 0x7f55bc075320 con 0x7f55c00a8880 2023-11-15T03:06:55.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.765+0000 7f55c4976700 1 --2- 172.21.15.121:0/1538967940 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f55a405a510 0x7f55a405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.765+0000 7f55c4976700 1 --2- 172.21.15.121:0/1538967940 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f55a405a510 0x7f55a405c9d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f55b0009fd0 tx=0x7f55b0016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:55.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.765+0000 7f55d176d740 1 Processor -- start 2023-11-15T03:06:55.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.765+0000 7f55d176d740 1 -- start start 2023-11-15T03:06:55.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55d176d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 0x1f86090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55d176d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 0x1fac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55d176d740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1fac630 0x1facab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55d176d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ee0410 con 0x1f5fad0 2023-11-15T03:06:55.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55d176d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ee0550 con 0x1f5feb0 2023-11-15T03:06:55.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55d176d740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ee0a00 con 0x1fac630 2023-11-15T03:06:55.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55c5177700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 0x1f86090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55c5177700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 0x1f86090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54616/0 (socket says 172.21.15.121:54616) 2023-11-15T03:06:55.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55c5177700 1 -- 172.21.15.121:0/2231298455 learned_addr learned my addr 172.21.15.121:0/2231298455 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:55.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55c4976700 1 --2- 172.21.15.121:0/2231298455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 0x1fac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55c5978700 1 --2- 172.21.15.121:0/2231298455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1fac630 0x1facab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55c5177700 1 -- 172.21.15.121:0/2231298455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1fac630 msgr2=0x1facab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:55.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55c5177700 1 --2- 172.21.15.121:0/2231298455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1fac630 0x1facab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55c5177700 1 -- 172.21.15.121:0/2231298455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 msgr2=0x1fac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:55.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55c5177700 1 --2- 172.21.15.121:0/2231298455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 0x1fac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55c5177700 1 -- 172.21.15.121:0/2231298455 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1ed9730 con 0x1f5fad0 2023-11-15T03:06:55.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55c5978700 1 --2- 172.21.15.121:0/2231298455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1fac630 0x1facab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:55.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55c4976700 1 --2- 172.21.15.121:0/2231298455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 0x1fac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:55.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55c5177700 1 --2- 172.21.15.121:0/2231298455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 0x1f86090 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 rx=0x7f55bc04bfe0 tx=0x7f55bc011000).ready entity=mon.0 client_cookie=f52452a8f2d02d71 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:55.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55aa7fc700 1 -- 172.21.15.121:0/2231298455 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc0750d0 con 0x1f5fad0 2023-11-15T03:06:55.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55aa7fc700 1 -- 172.21.15.121:0/2231298455 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55bc04a510 con 0x1f5fad0 2023-11-15T03:06:55.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55aa7fc700 1 -- 172.21.15.121:0/2231298455 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc048880 con 0x1f5fad0 2023-11-15T03:06:55.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55d176d740 1 -- 172.21.15.121:0/2231298455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 msgr2=0x1f86090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:55.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55d176d740 1 --2- 172.21.15.121:0/2231298455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 0x1f86090 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 rx=0x7f55bc04bfe0 tx=0x7f55bc011000).stop 2023-11-15T03:06:55.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55d176d740 1 -- 172.21.15.121:0/2231298455 shutdown_connections 2023-11-15T03:06:55.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55d176d740 1 --2- 172.21.15.121:0/2231298455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1fac630 0x1facab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55d176d740 1 --2- 172.21.15.121:0/2231298455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 0x1fac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55d176d740 1 --2- 172.21.15.121:0/2231298455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 0x1f86090 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55d176d740 1 -- 172.21.15.121:0/2231298455 >> 172.21.15.121:0/2231298455 conn(0x1d3e550 msgr2=0x1f5e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:55.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55d176d740 1 -- 172.21.15.121:0/2231298455 shutdown_connections 2023-11-15T03:06:55.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.769+0000 7f55d176d740 1 -- 172.21.15.121:0/2231298455 wait complete. 2023-11-15T03:06:55.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55d176d740 1 -- start start 2023-11-15T03:06:55.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55d176d740 1 Processor -- start 2023-11-15T03:06:55.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55d176d740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1f5fad0 0x1f89b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55d176d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 0x1f8c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55d176d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fac630 0x1f8c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55d176d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1eab650 con 0x1fac630 2023-11-15T03:06:55.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55d176d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f8ac90 con 0x1f5feb0 2023-11-15T03:06:55.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55d176d740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f8a020 con 0x1f5fad0 2023-11-15T03:06:55.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55d176d740 10 client.0 ll_register_callbacks cb 0x1f18020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:06:55.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55c5978700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fac630 0x1f8c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55c5978700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fac630 0x1f8c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54628/0 (socket says 172.21.15.121:54628) 2023-11-15T03:06:55.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55c5978700 1 -- 172.21.15.121:0/3633319661 learned_addr learned my addr 172.21.15.121:0/3633319661 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:55.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55c4976700 1 --2- 172.21.15.121:0/3633319661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 0x1f8c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55c5177700 1 --2- 172.21.15.121:0/3633319661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1f5fad0 0x1f89b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55c5978700 1 -- 172.21.15.121:0/3633319661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1f5fad0 msgr2=0x1f89b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:55.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55c5978700 1 --2- 172.21.15.121:0/3633319661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1f5fad0 0x1f89b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55c5978700 1 -- 172.21.15.121:0/3633319661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 msgr2=0x1f8c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:55.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55c5978700 1 --2- 172.21.15.121:0/3633319661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 0x1f8c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55c5978700 1 -- 172.21.15.121:0/3633319661 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c0158130 con 0x1fac630 2023-11-15T03:06:55.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55c4976700 1 --2- 172.21.15.121:0/3633319661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 0x1f8c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:55.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f55c5978700 1 --2- 172.21.15.121:0/3633319661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fac630 0x1f8c710 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 rx=0x7f55b800f820 tx=0x7f55b8009040).ready entity=mon.0 client_cookie=8eb82e0600ccbdac server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:55.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.773+0000 7f559f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:06:55.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f559f7fe700 1 -- 172.21.15.121:0/3633319661 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b80143f0 con 0x1fac630 2023-11-15T03:06:55.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f559f7fe700 1 -- 172.21.15.121:0/3633319661 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 7814+0+0 (secure 0 0 0) 0x7f55b8014550 con 0x1fac630 2023-11-15T03:06:55.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f55d176d740 1 -- 172.21.15.121:0/3633319661 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1ed61a0 con 0x1fac630 2023-11-15T03:06:55.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f559f7fe700 1 -- 172.21.15.121:0/3633319661 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=200) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55b8014970 con 0x1fac630 2023-11-15T03:06:55.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f55d176d740 10 client.8901 fetch_fsmap learned FSMap version 200 2023-11-15T03:06:55.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f55d176d740 1 -- 172.21.15.121:0/3633319661 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=200,monmap=2+}) v3 -- 0x1ed4960 con 0x1fac630 2023-11-15T03:06:55.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f559f7fe700 1 -- 172.21.15.121:0/3633319661 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1ed61a0 con 0x1fac630 2023-11-15T03:06:55.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f559f7fe700 1 -- 172.21.15.121:0/3633319661 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 200) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f55b8022410 con 0x1fac630 2023-11-15T03:06:55.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f55d176d740 10 client.8901 fetch_fsmap finished waiting for FSMap version 200 2023-11-15T03:06:55.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f55d176d740 10 client.8901 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:06:55.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f55d176d740 20 client.8901 populate_metadata read hostname 'smithi121' 2023-11-15T03:06:55.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f55d176d740 1 --2- 172.21.15.121:0/3633319661 >> [v2:172.21.15.121:6834/2289093120,v1:172.21.15.121:6836/2289093120] conn(0x1f9f770 0x1f9fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f55d176d740 4 client.8901 mds_command: new command op to 8802 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:06:55.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f55d176d740 1 -- 172.21.15.121:0/3633319661 --> [v2:172.21.15.121:6834/2289093120,v1:172.21.15.121:6836/2289093120] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1ecc6f0 con 0x1f9f770 2023-11-15T03:06:55.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f55c5177700 1 --2- 172.21.15.121:0/3633319661 >> [v2:172.21.15.121:6834/2289093120,v1:172.21.15.121:6836/2289093120] conn(0x1f9f770 0x1f9fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f559f7fe700 10 client.8901 ms_handle_connect on v2:172.21.15.121:6834/2289093120 2023-11-15T03:06:55.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f55c5177700 1 --2- 172.21.15.121:0/3633319661 >> [v2:172.21.15.121:6834/2289093120,v1:172.21.15.121:6836/2289093120] conn(0x1f9f770 0x1f9fb50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=f74fd6ddd80b85fd server_cookie=587a29a1969f0e89 in_seq=0 out_seq=0 2023-11-15T03:06:55.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f559f7fe700 1 -- 172.21.15.121:0/3633319661 <== mds.0 v2:172.21.15.121:6834/2289093120 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x1ecc6f0 con 0x1f9f770 2023-11-15T03:06:55.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f559f7fe700 10 client.8901 handle_command_reply: tid=0 2023-11-15T03:06:55.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.777+0000 7f55d176d740 1 client.8901 shutdown 2023-11-15T03:06:55.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.781+0000 7f55d176d740 1 -- 172.21.15.121:0/3633319661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fac630 msgr2=0x1f8c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:55.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.781+0000 7f55d176d740 1 --2- 172.21.15.121:0/3633319661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fac630 0x1f8c710 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 rx=0x7f55b800f820 tx=0x7f55b8009040).stop 2023-11-15T03:06:55.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.781+0000 7f55d176d740 1 -- 172.21.15.121:0/3633319661 shutdown_connections 2023-11-15T03:06:55.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.781+0000 7f55d176d740 1 --2- 172.21.15.121:0/3633319661 >> [v2:172.21.15.121:6834/2289093120,v1:172.21.15.121:6836/2289093120] conn(0x1f9f770 0x1f9fb50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.781+0000 7f55d176d740 1 --2- 172.21.15.121:0/3633319661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1fac630 0x1f8c710 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.781+0000 7f55d176d740 1 --2- 172.21.15.121:0/3633319661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 0x1f8c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.781+0000 7f55d176d740 1 --2- 172.21.15.121:0/3633319661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1f5fad0 0x1f89b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.781+0000 7f55d176d740 1 -- 172.21.15.121:0/3633319661 >> 172.21.15.121:0/3633319661 conn(0x1d3e550 msgr2=0x1fa74b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:55.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.781+0000 7f559f7fe700 0 client.8901 ms_handle_reset on v2:172.21.15.121:6834/2289093120 2023-11-15T03:06:55.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.781+0000 7f55d176d740 1 -- 172.21.15.121:0/3633319661 shutdown_connections 2023-11-15T03:06:55.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.781+0000 7f55d176d740 1 -- 172.21.15.121:0/3633319661 wait complete. 2023-11-15T03:06:55.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.781+0000 7f55d176d740 20 client.8901 trim_cache size 0 max 16384 2023-11-15T03:06:55.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.801+0000 7f55d176d740 1 Processor -- start 2023-11-15T03:06:55.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.801+0000 7f55d176d740 1 -- start start 2023-11-15T03:06:55.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.801+0000 7f55d176d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 0x1f8c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.801+0000 7f55d176d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 0x1f8c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.801+0000 7f55d176d740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1fac630 0x1f9d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.801+0000 7f55d176d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f8a4b0 con 0x1f5fad0 2023-11-15T03:06:55.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.801+0000 7f55d176d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f895c0 con 0x1f5feb0 2023-11-15T03:06:55.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.801+0000 7f55d176d740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f8cc00 con 0x1fac630 2023-11-15T03:06:55.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.801+0000 7f55c5177700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 0x1f8c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.801+0000 7f55c5177700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 0x1f8c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54642/0 (socket says 172.21.15.121:54642) 2023-11-15T03:06:55.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.801+0000 7f55c5177700 1 -- 172.21.15.121:0/1617858497 learned_addr learned my addr 172.21.15.121:0/1617858497 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:55.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.801+0000 7f55c4976700 1 --2- 172.21.15.121:0/1617858497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 0x1f8c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.801+0000 7f55c5978700 1 --2- 172.21.15.121:0/1617858497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1fac630 0x1f9d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.801+0000 7f55c5177700 1 -- 172.21.15.121:0/1617858497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1fac630 msgr2=0x1f9d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:55.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.801+0000 7f55c5177700 1 --2- 172.21.15.121:0/1617858497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1fac630 0x1f9d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.801+0000 7f55c5177700 1 -- 172.21.15.121:0/1617858497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 msgr2=0x1f8c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:55.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.801+0000 7f55c5177700 1 --2- 172.21.15.121:0/1617858497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 0x1f8c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.801+0000 7f55c5177700 1 -- 172.21.15.121:0/1617858497 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c0159460 con 0x1f5fad0 2023-11-15T03:06:55.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.801+0000 7f55c5177700 1 --2- 172.21.15.121:0/1617858497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 0x1f8c220 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7f55bc048670 tx=0x7f55bc01d000).ready entity=mon.0 client_cookie=3e2ae1be69ea7fa2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:55.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55aaffd700 1 -- 172.21.15.121:0/1617858497 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc05d110 con 0x1f5fad0 2023-11-15T03:06:55.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55aaffd700 1 -- 172.21.15.121:0/1617858497 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55bc0515a0 con 0x1f5fad0 2023-11-15T03:06:55.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55aaffd700 1 -- 172.21.15.121:0/1617858497 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc00c490 con 0x1f5fad0 2023-11-15T03:06:55.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55d176d740 1 -- 172.21.15.121:0/1617858497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 msgr2=0x1f8c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:55.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55d176d740 1 --2- 172.21.15.121:0/1617858497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 0x1f8c220 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7f55bc048670 tx=0x7f55bc01d000).stop 2023-11-15T03:06:55.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55d176d740 1 -- 172.21.15.121:0/1617858497 shutdown_connections 2023-11-15T03:06:55.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55d176d740 1 --2- 172.21.15.121:0/1617858497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1fac630 0x1f9d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55d176d740 1 --2- 172.21.15.121:0/1617858497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 0x1f8c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55d176d740 1 --2- 172.21.15.121:0/1617858497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 0x1f8c220 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55d176d740 1 -- 172.21.15.121:0/1617858497 >> 172.21.15.121:0/1617858497 conn(0x1d3e550 msgr2=0x1f8b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:55.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55d176d740 1 -- 172.21.15.121:0/1617858497 shutdown_connections 2023-11-15T03:06:55.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55d176d740 1 -- 172.21.15.121:0/1617858497 wait complete. 2023-11-15T03:06:55.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55d176d740 1 -- start start 2023-11-15T03:06:55.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55d176d740 1 Processor -- start 2023-11-15T03:06:55.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55d176d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 0x2007b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55d176d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 0x1f8c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55d176d740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1fac630 0x1f8c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55d176d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f8cbe0 con 0x1f5fad0 2023-11-15T03:06:55.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55d176d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fab270 con 0x1f5feb0 2023-11-15T03:06:55.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55d176d740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f5ee20 con 0x1fac630 2023-11-15T03:06:55.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.805+0000 7f55d176d740 10 client.0 ll_register_callbacks cb 0x7f55b800f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:06:55.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55c5177700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 0x2007b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55c5177700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 0x2007b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54652/0 (socket says 172.21.15.121:54652) 2023-11-15T03:06:55.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55c5177700 1 -- 172.21.15.121:0/1661917926 learned_addr learned my addr 172.21.15.121:0/1661917926 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:55.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55c4976700 1 --2- 172.21.15.121:0/1661917926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 0x1f8c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55c5978700 1 --2- 172.21.15.121:0/1661917926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1fac630 0x1f8c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55c5177700 1 -- 172.21.15.121:0/1661917926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1fac630 msgr2=0x1f8c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:55.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55c5177700 1 --2- 172.21.15.121:0/1661917926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1fac630 0x1f8c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55c5177700 1 -- 172.21.15.121:0/1661917926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 msgr2=0x1f8c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:55.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55c5177700 1 --2- 172.21.15.121:0/1661917926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 0x1f8c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55c5177700 1 -- 172.21.15.121:0/1661917926 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1ed9730 con 0x1f5fad0 2023-11-15T03:06:55.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55c5177700 1 --2- 172.21.15.121:0/1661917926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 0x2007b10 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7f55bc0096d0 tx=0x7f55bc05c000).ready entity=mon.0 client_cookie=9ffd44bb5964405a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:55.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f559f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:06:55.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f559f7fe700 1 -- 172.21.15.121:0/1661917926 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc069070 con 0x1f5fad0 2023-11-15T03:06:55.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f559f7fe700 1 -- 172.21.15.121:0/1661917926 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 7814+0+0 (secure 0 0 0) 0x7f55bc072dd0 con 0x1f5fad0 2023-11-15T03:06:55.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55d176d740 1 -- 172.21.15.121:0/1661917926 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55bc04a510 con 0x1f5fad0 2023-11-15T03:06:55.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f559f7fe700 1 -- 172.21.15.121:0/1661917926 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=200) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55bc072f90 con 0x1f5fad0 2023-11-15T03:06:55.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55d176d740 10 client.8907 fetch_fsmap learned FSMap version 200 2023-11-15T03:06:55.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55d176d740 1 -- 172.21.15.121:0/1661917926 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=200,monmap=2+}) v3 -- 0x1ed46b0 con 0x1f5fad0 2023-11-15T03:06:55.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f559f7fe700 1 -- 172.21.15.121:0/1661917926 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55bc04a510 con 0x1f5fad0 2023-11-15T03:06:55.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f559f7fe700 1 -- 172.21.15.121:0/1661917926 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 200) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f55bc070020 con 0x1f5fad0 2023-11-15T03:06:55.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55d176d740 10 client.8907 fetch_fsmap finished waiting for FSMap version 200 2023-11-15T03:06:55.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55d176d740 10 client.8907 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:06:55.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55d176d740 20 client.8907 populate_metadata read hostname 'smithi121' 2023-11-15T03:06:55.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55d176d740 1 --2- 172.21.15.121:0/1661917926 >> [v2:172.21.15.121:6834/2289093120,v1:172.21.15.121:6836/2289093120] conn(0x1f5a8a0 0x1f828c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:55.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55d176d740 4 client.8907 mds_command: new command op to 8802 tid=0[{"prefix": "status"}] 2023-11-15T03:06:55.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55d176d740 1 -- 172.21.15.121:0/1661917926 --> [v2:172.21.15.121:6834/2289093120,v1:172.21.15.121:6836/2289093120] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1d954e0 con 0x1f5a8a0 2023-11-15T03:06:55.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.809+0000 7f55c4976700 1 --2- 172.21.15.121:0/1661917926 >> [v2:172.21.15.121:6834/2289093120,v1:172.21.15.121:6836/2289093120] conn(0x1f5a8a0 0x1f828c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:55.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f559f7fe700 10 client.8907 ms_handle_connect on v2:172.21.15.121:6834/2289093120 2023-11-15T03:06:55.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55c4976700 1 --2- 172.21.15.121:0/1661917926 >> [v2:172.21.15.121:6834/2289093120,v1:172.21.15.121:6836/2289093120] conn(0x1f5a8a0 0x1f828c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=e7d5d588f018948b server_cookie=2a423af31b5011e3 in_seq=0 out_seq=0 2023-11-15T03:06:55.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f559f7fe700 1 -- 172.21.15.121:0/1661917926 <== mds.0 v2:172.21.15.121:6834/2289093120 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x1d954e0 con 0x1f5a8a0 2023-11-15T03:06:55.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f559f7fe700 10 client.8907 handle_command_reply: tid=0 2023-11-15T03:06:55.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55d176d740 1 client.8907 shutdown 2023-11-15T03:06:55.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55d176d740 1 -- 172.21.15.121:0/1661917926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 msgr2=0x2007b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:55.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55d176d740 1 --2- 172.21.15.121:0/1661917926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 0x2007b10 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7f55bc0096d0 tx=0x7f55bc05c000).stop 2023-11-15T03:06:55.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55d176d740 1 -- 172.21.15.121:0/1661917926 shutdown_connections 2023-11-15T03:06:55.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55d176d740 1 --2- 172.21.15.121:0/1661917926 >> [v2:172.21.15.121:6834/2289093120,v1:172.21.15.121:6836/2289093120] conn(0x1f5a8a0 0x1f828c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55d176d740 1 --2- 172.21.15.121:0/1661917926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1fac630 0x1f8c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f559f7fe700 0 client.8907 ms_handle_reset on v2:172.21.15.121:6834/2289093120 2023-11-15T03:06:55.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55d176d740 1 --2- 172.21.15.121:0/1661917926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f5feb0 0x1f8c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55d176d740 1 --2- 172.21.15.121:0/1661917926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f5fad0 0x2007b10 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55d176d740 1 -- 172.21.15.121:0/1661917926 >> 172.21.15.121:0/1661917926 conn(0x1d3e550 msgr2=0x1f8b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:55.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55d176d740 1 -- 172.21.15.121:0/1661917926 shutdown_connections 2023-11-15T03:06:55.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55d176d740 1 -- 172.21.15.121:0/1661917926 wait complete. 2023-11-15T03:06:55.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55d176d740 20 client.8907 trim_cache size 0 max 16384 2023-11-15T03:06:55.831 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:06:55.831 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T03:06:55.831 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T03:06:55.832 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 8802, 2023-11-15T03:06:55.832 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T03:06:55.832 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T03:06:55.832 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T03:06:55.832 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 7.7124970639999999, 2023-11-15T03:06:55.833 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 200, 2023-11-15T03:06:55.833 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 112, 2023-11-15T03:06:55.833 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 112, 2023-11-15T03:06:55.833 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 16.654695540999999 2023-11-15T03:06:55.833 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:06:55.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55c6179700 1 -- 172.21.15.121:0/1538967940 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f55a405a510 msgr2=0x7f55a405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:55.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55c6179700 1 --2- 172.21.15.121:0/1538967940 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f55a405a510 0x7f55a405c9d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f55b0009fd0 tx=0x7f55b0016000).stop 2023-11-15T03:06:55.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55c6179700 1 -- 172.21.15.121:0/1538967940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c00a8880 msgr2=0x7f55c014da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:55.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55c6179700 1 --2- 172.21.15.121:0/1538967940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c00a8880 0x7f55c014da40 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7f55bc00e960 tx=0x7f55bc009060).stop 2023-11-15T03:06:55.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55c6179700 1 -- 172.21.15.121:0/1538967940 shutdown_connections 2023-11-15T03:06:55.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55c6179700 1 --2- 172.21.15.121:0/1538967940 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f55a405a510 0x7f55a405c9d0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55c6179700 1 --2- 172.21.15.121:0/1538967940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c00bb510 0x7f55c00c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55c6179700 1 --2- 172.21.15.121:0/1538967940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55c00abaa0 0x7f55c0158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55c6179700 1 --2- 172.21.15.121:0/1538967940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c00a8880 0x7f55c014da40 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:55.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55c6179700 1 -- 172.21.15.121:0/1538967940 >> 172.21.15.121:0/1538967940 conn(0x7f55c0067a60 msgr2=0x7f55c006b440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:55.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55c6179700 1 -- 172.21.15.121:0/1538967940 shutdown_connections 2023-11-15T03:06:55.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:55.813+0000 7f55c6179700 1 -- 172.21.15.121:0/1538967940 wait complete. 2023-11-15T03:06:55.842 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:06:55.843 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:06:56.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.017+0000 7fa071279700 1 Processor -- start 2023-11-15T03:06:56.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.017+0000 7fa071279700 1 -- start start 2023-11-15T03:06:56.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.017+0000 7fa071279700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa06c066320 0x7fa06c154110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:56.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.017+0000 7fa071279700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa06c0acf10 0x7fa06c062190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:56.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.017+0000 7fa071279700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa06c0c73e0 con 0x7fa06c066700 2023-11-15T03:06:56.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.017+0000 7fa071279700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa06c1546c0 con 0x7fa06c0acf10 2023-11-15T03:06:56.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.017+0000 7fa071279700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa06c1549a0 con 0x7fa06c066320 2023-11-15T03:06:56.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.017+0000 7fa070a78700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa06c0acf10 0x7fa06c062190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:56.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.017+0000 7fa0637fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa06c066700 0x7fa06c0656a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44046/0 (socket says 172.21.15.121:44046) 2023-11-15T03:06:56.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.017+0000 7fa0637fe700 1 -- 172.21.15.121:0/4248766354 learned_addr learned my addr 172.21.15.121:0/4248766354 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:06:56.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.017+0000 7fa06bfff700 1 --2- 172.21.15.121:0/4248766354 >> v2:172.21.15.190:3300/0 conn(0x7fa06c066320 0x7fa06c154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:56.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.017+0000 7fa070a78700 1 -- 172.21.15.121:0/4248766354 >> v2:172.21.15.190:3300/0 conn(0x7fa06c066320 msgr2=0x7fa06c154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:56.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.017+0000 7fa070a78700 1 --2- 172.21.15.121:0/4248766354 >> v2:172.21.15.190:3300/0 conn(0x7fa06c066320 0x7fa06c154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:56.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.017+0000 7fa070a78700 1 -- 172.21.15.121:0/4248766354 >> v1:172.21.15.121:6789/0 conn(0x7fa06c066700 legacy=0x7fa06c0656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:56.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.017+0000 7fa070a78700 1 -- 172.21.15.121:0/4248766354 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c154c80 con 0x7fa06c0acf10 2023-11-15T03:06:56.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.021+0000 7fa070a78700 1 --2- 172.21.15.121:0/4248766354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa06c0acf10 0x7fa06c062190 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7fa06c0cba90 tx=0x7fa064009780).ready entity=mon.0 client_cookie=e7a32f2f7654b1f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:56.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.021+0000 7fa06b7fe700 1 -- 172.21.15.121:0/4248766354 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06400e070 con 0x7fa06c0acf10 2023-11-15T03:06:56.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.021+0000 7fa06b7fe700 1 -- 172.21.15.121:0/4248766354 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06400bc30 con 0x7fa06c0acf10 2023-11-15T03:06:56.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.021+0000 7fa06b7fe700 1 -- 172.21.15.121:0/4248766354 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064012410 con 0x7fa06c0acf10 2023-11-15T03:06:56.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.021+0000 7fa071279700 1 -- 172.21.15.121:0/4248766354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa06c0acf10 msgr2=0x7fa06c062190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:56.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.021+0000 7fa071279700 1 --2- 172.21.15.121:0/4248766354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa06c0acf10 0x7fa06c062190 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7fa06c0cba90 tx=0x7fa064009780).stop 2023-11-15T03:06:56.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.021+0000 7fa071279700 1 -- 172.21.15.121:0/4248766354 shutdown_connections 2023-11-15T03:06:56.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.021+0000 7fa071279700 1 --2- 172.21.15.121:0/4248766354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa06c0acf10 0x7fa06c062190 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:56.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.021+0000 7fa071279700 1 --2- 172.21.15.121:0/4248766354 >> v2:172.21.15.190:3300/0 conn(0x7fa06c066320 0x7fa06c154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:56.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.021+0000 7fa071279700 1 -- 172.21.15.121:0/4248766354 >> 172.21.15.121:0/4248766354 conn(0x7fa06c067a60 msgr2=0x7fa06c067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:56.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.021+0000 7fa071279700 1 -- 172.21.15.121:0/4248766354 shutdown_connections 2023-11-15T03:06:56.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.021+0000 7fa071279700 1 -- 172.21.15.121:0/4248766354 wait complete. 2023-11-15T03:06:56.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.025+0000 7fa071279700 1 Processor -- start 2023-11-15T03:06:56.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.025+0000 7fa071279700 1 -- start start 2023-11-15T03:06:56.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.025+0000 7fa071279700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa06c066320 0x7fa06c14d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:56.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.025+0000 7fa071279700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa06c066700 0x7fa06c14b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:56.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.025+0000 7fa071279700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa06c0acf10 0x7fa06c1676b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:56.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.025+0000 7fa071279700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c14dd10 con 0x7fa06c066700 2023-11-15T03:06:56.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.025+0000 7fa071279700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c14def0 con 0x7fa06c0acf10 2023-11-15T03:06:56.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.025+0000 7fa071279700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c14e1d0 con 0x7fa06c066320 2023-11-15T03:06:56.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.025+0000 7fa0637fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa06c066700 0x7fa06c14b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:56.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.025+0000 7fa0637fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa06c066700 0x7fa06c14b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54664/0 (socket says 172.21.15.121:54664) 2023-11-15T03:06:56.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.025+0000 7fa0637fe700 1 -- 172.21.15.121:0/3770769650 learned_addr learned my addr 172.21.15.121:0/3770769650 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:56.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.025+0000 7fa06bfff700 1 --2- 172.21.15.121:0/3770769650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa06c066320 0x7fa06c14d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:56.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.025+0000 7fa070a78700 1 --2- 172.21.15.121:0/3770769650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa06c0acf10 0x7fa06c1676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:56.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.025+0000 7fa0637fe700 1 -- 172.21.15.121:0/3770769650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa06c066320 msgr2=0x7fa06c14d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:56.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.025+0000 7fa0637fe700 1 --2- 172.21.15.121:0/3770769650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa06c066320 0x7fa06c14d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:56.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.025+0000 7fa0637fe700 1 -- 172.21.15.121:0/3770769650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa06c0acf10 msgr2=0x7fa06c1676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:56.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.025+0000 7fa0637fe700 1 --2- 172.21.15.121:0/3770769650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa06c0acf10 0x7fa06c1676b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:56.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.025+0000 7fa0637fe700 1 -- 172.21.15.121:0/3770769650 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa064009a20 con 0x7fa06c066700 2023-11-15T03:06:56.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.029+0000 7fa0637fe700 1 --2- 172.21.15.121:0/3770769650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa06c066700 0x7fa06c14b380 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7fa05400c4c0 tx=0x7fa054007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:56.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.029+0000 7fa068ff9700 1 -- 172.21.15.121:0/3770769650 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0540072e0 con 0x7fa06c066700 2023-11-15T03:06:56.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.029+0000 7fa068ff9700 1 -- 172.21.15.121:0/3770769650 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa054007440 con 0x7fa06c066700 2023-11-15T03:06:56.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.029+0000 7fa071279700 1 -- 172.21.15.121:0/3770769650 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06c14e4b0 con 0x7fa06c066700 2023-11-15T03:06:56.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.029+0000 7fa068ff9700 1 -- 172.21.15.121:0/3770769650 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0540057a0 con 0x7fa06c066700 2023-11-15T03:06:56.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.029+0000 7fa071279700 1 -- 172.21.15.121:0/3770769650 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06c14b870 con 0x7fa06c066700 2023-11-15T03:06:56.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.029+0000 7fa071279700 1 -- 172.21.15.121:0/3770769650 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06c14e6f0 con 0x7fa06c066700 2023-11-15T03:06:56.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.029+0000 7fa068ff9700 1 -- 172.21.15.121:0/3770769650 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa054005bd0 con 0x7fa06c066700 2023-11-15T03:06:56.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.033+0000 7fa068ff9700 1 --2- 172.21.15.121:0/3770769650 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa04c05a510 0x7fa04c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:56.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.033+0000 7fa068ff9700 1 -- 172.21.15.121:0/3770769650 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7814+0+0 (secure 0 0 0) 0x7fa0540744b0 con 0x7fa06c066700 2023-11-15T03:06:56.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.033+0000 7fa06bfff700 1 --2- 172.21.15.121:0/3770769650 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa04c05a510 0x7fa04c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:56.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.033+0000 7fa06bfff700 1 --2- 172.21.15.121:0/3770769650 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa04c05a510 0x7fa04c05c9d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fa05c00bb20 tx=0x7fa05c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:56.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.037+0000 7fa068ff9700 1 -- 172.21.15.121:0/3770769650 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa06c14e6f0 con 0x7fa06c066700 2023-11-15T03:06:56.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.241+0000 7fa071279700 1 -- 172.21.15.121:0/3770769650 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa06c0aea70 con 0x7fa06c066700 2023-11-15T03:06:56.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.245+0000 7fa068ff9700 1 -- 172.21.15.121:0/3770769650 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) v1 ==== 95+0+4505 (secure 0 0 0) 0x7fa06c0aea70 con 0x7fa06c066700 2023-11-15T03:06:56.250 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:56.251 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":200,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:06:51.125630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8802},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8802":{"gid":8802,"name":"a","rank":0,"incarnation":197,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6836/2289093120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2289093120},{"type":"v1","addr":"172.21.15.121:6836","nonce":2289093120}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:06:56.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.245+0000 7fa071279700 1 -- 172.21.15.121:0/3770769650 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa04c05a510 msgr2=0x7fa04c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:56.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.245+0000 7fa071279700 1 --2- 172.21.15.121:0/3770769650 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa04c05a510 0x7fa04c05c9d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fa05c00bb20 tx=0x7fa05c008040).stop 2023-11-15T03:06:56.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.245+0000 7fa071279700 1 -- 172.21.15.121:0/3770769650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa06c066700 msgr2=0x7fa06c14b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:56.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.245+0000 7fa071279700 1 --2- 172.21.15.121:0/3770769650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa06c066700 0x7fa06c14b380 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7fa05400c4c0 tx=0x7fa054007dc0).stop 2023-11-15T03:06:56.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.245+0000 7fa071279700 1 -- 172.21.15.121:0/3770769650 shutdown_connections 2023-11-15T03:06:56.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.245+0000 7fa071279700 1 --2- 172.21.15.121:0/3770769650 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa04c05a510 0x7fa04c05c9d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:56.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.245+0000 7fa071279700 1 --2- 172.21.15.121:0/3770769650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa06c0acf10 0x7fa06c1676b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:56.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.245+0000 7fa071279700 1 --2- 172.21.15.121:0/3770769650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa06c066700 0x7fa06c14b380 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:56.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.245+0000 7fa071279700 1 --2- 172.21.15.121:0/3770769650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa06c066320 0x7fa06c14d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:56.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.245+0000 7fa071279700 1 -- 172.21.15.121:0/3770769650 >> 172.21.15.121:0/3770769650 conn(0x7fa06c067a60 msgr2=0x7fa06c0ca9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:56.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.245+0000 7fa071279700 1 -- 172.21.15.121:0/3770769650 shutdown_connections 2023-11-15T03:06:56.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.245+0000 7fa071279700 1 -- 172.21.15.121:0/3770769650 wait complete. 2023-11-15T03:06:56.257 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 200 2023-11-15T03:06:56.271 INFO:tasks.cephfs.test_damage:Mutation 'Corrupt 500.00000000' did not prevent MDS startup, attempting ls... 2023-11-15T03:06:56.272 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T03:06:56.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.397+0000 7f754d8ff700 1 Processor -- start 2023-11-15T03:06:56.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.397+0000 7f754d8ff700 1 -- start start 2023-11-15T03:06:56.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.397+0000 7f754d8ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75480c9ea0 0x7f75480ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:56.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.397+0000 7f754d8ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7548154b30 0x7f7548065180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:56.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.397+0000 7f754d8ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75480cb650 con 0x7f7548065cb0 2023-11-15T03:06:56.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.397+0000 7f754d8ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75480c8220 con 0x7f7548154b30 2023-11-15T03:06:56.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.397+0000 7f754d8ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75480cb8f0 con 0x7f75480c9ea0 2023-11-15T03:06:56.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f75467fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7548154b30 0x7f7548065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:56.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f75467fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7548154b30 0x7f7548065180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:48116/0 (socket says 172.21.15.121:48116) 2023-11-15T03:06:56.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f75467fc700 1 -- 172.21.15.121:0/1347350706 learned_addr learned my addr 172.21.15.121:0/1347350706 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:56.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f7547fff700 1 --1- 172.21.15.121:0/1347350706 >> v1:172.21.15.190:6789/0 conn(0x7f7548065cb0 0x7f75480c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:49082/0 (socket says 172.21.15.121:49082) 2023-11-15T03:06:56.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f7546ffd700 1 --2- 172.21.15.121:0/1347350706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75480c9ea0 0x7f75480ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:56.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f7545ffb700 1 -- 172.21.15.121:0/1347350706 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199634716 0 0) 0x7f75480cb650 con 0x7f7548065cb0 2023-11-15T03:06:56.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f7545ffb700 1 -- 172.21.15.121:0/1347350706 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75300033f0 con 0x7f7548065cb0 2023-11-15T03:06:56.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f75467fc700 1 -- 172.21.15.121:0/1347350706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75480c9ea0 msgr2=0x7f75480ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:56.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f75467fc700 1 --2- 172.21.15.121:0/1347350706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75480c9ea0 0x7f75480ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:56.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f75467fc700 1 -- 172.21.15.121:0/1347350706 >> v1:172.21.15.190:6789/0 conn(0x7f7548065cb0 legacy=0x7f75480c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:56.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f75467fc700 1 -- 172.21.15.121:0/1347350706 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75480cbbd0 con 0x7f7548154b30 2023-11-15T03:06:56.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f75467fc700 1 --2- 172.21.15.121:0/1347350706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7548154b30 0x7f7548065180 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f753c006fd0 tx=0x7f753c0090f0).ready entity=mon.1 client_cookie=a5ebf32233a16536 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:56.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f7545ffb700 1 -- 172.21.15.121:0/1347350706 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c00f510 con 0x7f7548154b30 2023-11-15T03:06:56.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f7545ffb700 1 -- 172.21.15.121:0/1347350706 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753c00f670 con 0x7f7548154b30 2023-11-15T03:06:56.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f7545ffb700 1 -- 172.21.15.121:0/1347350706 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c00f9f0 con 0x7f7548154b30 2023-11-15T03:06:56.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f754d8ff700 1 -- 172.21.15.121:0/1347350706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7548154b30 msgr2=0x7f7548065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:56.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f754d8ff700 1 --2- 172.21.15.121:0/1347350706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7548154b30 0x7f7548065180 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f753c006fd0 tx=0x7f753c0090f0).stop 2023-11-15T03:06:56.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f754d8ff700 1 -- 172.21.15.121:0/1347350706 shutdown_connections 2023-11-15T03:06:56.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f754d8ff700 1 --2- 172.21.15.121:0/1347350706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7548154b30 0x7f7548065180 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:56.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f754d8ff700 1 --2- 172.21.15.121:0/1347350706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75480c9ea0 0x7f75480ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:56.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f754d8ff700 1 -- 172.21.15.121:0/1347350706 >> 172.21.15.121:0/1347350706 conn(0x7f7548069960 msgr2=0x7f7548069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:56.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f754d8ff700 1 -- 172.21.15.121:0/1347350706 shutdown_connections 2023-11-15T03:06:56.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.401+0000 7f754d8ff700 1 -- 172.21.15.121:0/1347350706 wait complete. 2023-11-15T03:06:56.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.405+0000 7f754d8ff700 1 Processor -- start 2023-11-15T03:06:56.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.405+0000 7f754d8ff700 1 -- start start 2023-11-15T03:06:56.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f754d8ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7548065cb0 0x7f7548159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:56.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f754d8ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75480c9ea0 0x7f75480c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:56.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f754d8ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7548154b30 0x7f75480ac520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:56.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f754d8ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75480b21b0 con 0x7f75480c9ea0 2023-11-15T03:06:56.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f754d8ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75480aed10 con 0x7f7548065cb0 2023-11-15T03:06:56.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f754d8ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75480aea30 con 0x7f7548154b30 2023-11-15T03:06:56.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f75467fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75480c9ea0 0x7f75480c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:56.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f75467fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75480c9ea0 0x7f75480c5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54666/0 (socket says 172.21.15.121:54666) 2023-11-15T03:06:56.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f75467fc700 1 -- 172.21.15.121:0/3829400478 learned_addr learned my addr 172.21.15.121:0/3829400478 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:56.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f7547fff700 1 --2- 172.21.15.121:0/3829400478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7548154b30 0x7f75480ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:56.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f7546ffd700 1 --2- 172.21.15.121:0/3829400478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7548065cb0 0x7f7548159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:56.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f75467fc700 1 -- 172.21.15.121:0/3829400478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7548154b30 msgr2=0x7f75480ac520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:56.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f75467fc700 1 --2- 172.21.15.121:0/3829400478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7548154b30 0x7f75480ac520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:56.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f75467fc700 1 -- 172.21.15.121:0/3829400478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7548065cb0 msgr2=0x7f7548159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:56.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f75467fc700 1 --2- 172.21.15.121:0/3829400478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7548065cb0 0x7f7548159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:56.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f75467fc700 1 -- 172.21.15.121:0/3829400478 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753c009370 con 0x7f75480c9ea0 2023-11-15T03:06:56.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f7546ffd700 1 --2- 172.21.15.121:0/3829400478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7548065cb0 0x7f7548159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:56.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f75467fc700 1 --2- 172.21.15.121:0/3829400478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75480c9ea0 0x7f75480c5610 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 rx=0x7f75480645d0 tx=0x7f753c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:56.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f752effd700 1 -- 172.21.15.121:0/3829400478 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c00f7e0 con 0x7f75480c9ea0 2023-11-15T03:06:56.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f752effd700 1 -- 172.21.15.121:0/3829400478 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753c00f940 con 0x7f75480c9ea0 2023-11-15T03:06:56.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f754d8ff700 1 -- 172.21.15.121:0/3829400478 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75480b1b60 con 0x7f75480c9ea0 2023-11-15T03:06:56.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f752effd700 1 -- 172.21.15.121:0/3829400478 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c01c670 con 0x7f75480c9ea0 2023-11-15T03:06:56.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f754d8ff700 1 -- 172.21.15.121:0/3829400478 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75481594f0 con 0x7f75480c9ea0 2023-11-15T03:06:56.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.409+0000 7f754d8ff700 1 -- 172.21.15.121:0/3829400478 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75480af0f0 con 0x7f75480c9ea0 2023-11-15T03:06:56.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.413+0000 7f752effd700 1 -- 172.21.15.121:0/3829400478 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f753c01ca20 con 0x7f75480c9ea0 2023-11-15T03:06:56.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.413+0000 7f752effd700 1 --2- 172.21.15.121:0/3829400478 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f753405a510 0x7f753405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:56.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.413+0000 7f752effd700 1 -- 172.21.15.121:0/3829400478 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7814+0+0 (secure 0 0 0) 0x7f753c0769d0 con 0x7f75480c9ea0 2023-11-15T03:06:56.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.413+0000 7f7546ffd700 1 --2- 172.21.15.121:0/3829400478 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f753405a510 0x7f753405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:56.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.413+0000 7f7546ffd700 1 --2- 172.21.15.121:0/3829400478 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f753405a510 0x7f753405c9d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f7538006d80 tx=0x7f7538008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:56.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.421+0000 7f752effd700 1 -- 172.21.15.121:0/3829400478 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f75480af0f0 con 0x7f75480c9ea0 2023-11-15T03:06:56.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.621+0000 7f754d8ff700 1 -- 172.21.15.121:0/3829400478 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f75480af0f0 con 0x7f75480c9ea0 2023-11-15T03:06:56.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.621+0000 7f752effd700 1 -- 172.21.15.121:0/3829400478 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v200) v1 ==== 53+0+73 (secure 0 0 0) 0x7f75480af0f0 con 0x7f75480c9ea0 2023-11-15T03:06:56.628 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T03:06:56.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.625+0000 7f754d8ff700 1 -- 172.21.15.121:0/3829400478 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f753405a510 msgr2=0x7f753405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:56.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.625+0000 7f754d8ff700 1 --2- 172.21.15.121:0/3829400478 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f753405a510 0x7f753405c9d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f7538006d80 tx=0x7f7538008040).stop 2023-11-15T03:06:56.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.625+0000 7f754d8ff700 1 -- 172.21.15.121:0/3829400478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75480c9ea0 msgr2=0x7f75480c5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:56.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.625+0000 7f754d8ff700 1 --2- 172.21.15.121:0/3829400478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75480c9ea0 0x7f75480c5610 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 rx=0x7f75480645d0 tx=0x7f753c017000).stop 2023-11-15T03:06:56.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.625+0000 7f754d8ff700 1 -- 172.21.15.121:0/3829400478 shutdown_connections 2023-11-15T03:06:56.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.625+0000 7f754d8ff700 1 --2- 172.21.15.121:0/3829400478 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f753405a510 0x7f753405c9d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:56.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.625+0000 7f754d8ff700 1 --2- 172.21.15.121:0/3829400478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7548154b30 0x7f75480ac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:56.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.625+0000 7f754d8ff700 1 --2- 172.21.15.121:0/3829400478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75480c9ea0 0x7f75480c5610 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:56.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.625+0000 7f754d8ff700 1 --2- 172.21.15.121:0/3829400478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7548065cb0 0x7f7548159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:56.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.625+0000 7f754d8ff700 1 -- 172.21.15.121:0/3829400478 >> 172.21.15.121:0/3829400478 conn(0x7f7548069960 msgr2=0x7f7548155120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:56.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.625+0000 7f754d8ff700 1 -- 172.21.15.121:0/3829400478 shutdown_connections 2023-11-15T03:06:56.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:56.625+0000 7f754d8ff700 1 -- 172.21.15.121:0/3829400478 wait complete. 2023-11-15T03:06:56.647 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T03:06:56.648 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T03:06:56.648 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T03:06:56.648 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T03:06:56.649 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T03:06:56.649 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T03:06:56.649 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T03:06:56.650 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T03:06:56.650 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:06:56.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:06:56.651 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T03:06:56.670 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T03:06:56.671 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T03:06:56.671 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T03:06:56.671 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:06:56.672 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T03:06:56.672 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:06:56.672 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:06:56.672 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:06:56.672 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T03:06:56.673 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:06:56.673 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:06:56.673 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T03:06:56.673 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:06:56.673 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:06:56.674 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T03:06:56.674 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:06:56.674 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T03:06:56.674 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 41823sec preferred_lft 41823sec 2023-11-15T03:06:56.674 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T03:06:56.675 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:06:56.675 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T03:06:56.675 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:06:56.675 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T03:06:56.675 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:06:56.676 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T03:06:56.676 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T03:06:56.676 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T03:06:56.676 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:06:56.676 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T03:06:56.677 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:06:56.677 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:06:56.677 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T03:06:56.677 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T03:06:56.678 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:06:56.678 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:06:56.678 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T03:06:56.678 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T03:06:56.679 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:06:56.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:06:56.680 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T03:06:56.758 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T03:06:56.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:06:56.764 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T03:06:56.826 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T03:06:56.826 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T03:06:56.826 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T03:06:56.827 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T03:06:56.827 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T03:06:56.827 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T03:06:56.827 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T03:06:56.827 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T03:06:56.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:06:56.828 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T03:06:56.829 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T03:06:56.829 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T03:06:56.830 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T03:06:56.830 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T03:06:56.830 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T03:06:56.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:06:56.969 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T03:06:56.988 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T03:06:56.988 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T03:06:56.989 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:06:56.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:06:56.990 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T03:06:57.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:06:57.065 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T03:06:57.127 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T03:06:57.127 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:06:57.247 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:06:57.248 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:06:57.274 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:06:57.275 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:06:57.279 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T03:06:57.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:06:57.280 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T03:06:57.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:06:57.313 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2023-11-15T03:06:57.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:06:57.367 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir foo) 2023-11-15T03:06:57.375 INFO:teuthology.orchestra.run.smithi121.stdout:.: 2023-11-15T03:06:57.375 INFO:teuthology.orchestra.run.smithi121.stdout:subdir 2023-11-15T03:06:57.375 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:06:57.375 INFO:teuthology.orchestra.run.smithi121.stdout:./subdir: 2023-11-15T03:06:57.376 INFO:teuthology.orchestra.run.smithi121.stdout:sixmegs 2023-11-15T03:06:57.389 INFO:teuthology.orchestra.run.smithi121.stderr:mkdir: cannot create directory 'foo': Read-only file system 2023-11-15T03:06:57.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:06:57.392 INFO:tasks.cephfs.test_damage:mkdir: cannot create directory 'foo': Read-only file system 2023-11-15T03:06:57.392 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt mds_snaptable' 2023-11-15T03:06:57.392 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T03:06:57.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:06:57.393 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-15T03:06:57.479 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:06:57.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:06:57.480 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:06:57.546 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:06:57.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.749+0000 7f73c8968700 1 Processor -- start 2023-11-15T03:06:57.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.749+0000 7f73c8968700 1 -- start start 2023-11-15T03:06:57.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.753+0000 7f73c8968700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f73c40abaa0 0x7f73c40abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:57.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.753+0000 7f73c8968700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f73c40bb510 0x7f73c4063570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:57.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.753+0000 7f73c8968700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73c40c2c00 con 0x7f73c40a8880 2023-11-15T03:06:57.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.753+0000 7f73c8968700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c4063a60 con 0x7f73c40bb510 2023-11-15T03:06:57.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.753+0000 7f73c8968700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c40657b0 con 0x7f73c40abaa0 2023-11-15T03:06:57.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.753+0000 7f73bbfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f73c40bb510 0x7f73c4063570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:57.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.753+0000 7f73bbfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f73c40bb510 0x7f73c4063570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54676/0 (socket says 172.21.15.121:54676) 2023-11-15T03:06:57.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.753+0000 7f73bbfff700 1 -- 172.21.15.121:0/2939696086 learned_addr learned my addr 172.21.15.121:0/2939696086 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:57.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.753+0000 7f73c2d9d700 1 --2- 172.21.15.121:0/2939696086 >> v2:172.21.15.190:3300/0 conn(0x7f73c40abaa0 0x7f73c40abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:57.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.753+0000 7f73bbfff700 1 -- 172.21.15.121:0/2939696086 >> v2:172.21.15.190:3300/0 conn(0x7f73c40abaa0 msgr2=0x7f73c40abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:57.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.753+0000 7f73bbfff700 1 --2- 172.21.15.121:0/2939696086 >> v2:172.21.15.190:3300/0 conn(0x7f73c40abaa0 0x7f73c40abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:57.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.753+0000 7f73bbfff700 1 -- 172.21.15.121:0/2939696086 >> v1:172.21.15.121:6789/0 conn(0x7f73c40a8880 legacy=0x7f73c4151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:57.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.753+0000 7f73bbfff700 1 -- 172.21.15.121:0/2939696086 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c4065a90 con 0x7f73c40bb510 2023-11-15T03:06:57.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.753+0000 7f73c2d9d700 1 --2- 172.21.15.121:0/2939696086 >> v2:172.21.15.190:3300/0 conn(0x7f73c40abaa0 0x7f73c40abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:57.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.753+0000 7f73bbfff700 1 --2- 172.21.15.121:0/2939696086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73c40bb510 0x7f73c4063570 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7f73c40599c0 tx=0x7f73ac009060).ready entity=mon.0 client_cookie=f3ef4a40f62e8624 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:57.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.753+0000 7f73c259c700 1 -- 172.21.15.121:0/2939696086 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac005880 con 0x7f73c40bb510 2023-11-15T03:06:57.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.753+0000 7f73c259c700 1 -- 172.21.15.121:0/2939696086 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ac0059e0 con 0x7f73c40bb510 2023-11-15T03:06:57.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.753+0000 7f73c259c700 1 -- 172.21.15.121:0/2939696086 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac005d60 con 0x7f73c40bb510 2023-11-15T03:06:57.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.753+0000 7f73c8968700 1 -- 172.21.15.121:0/2939696086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73c40bb510 msgr2=0x7f73c4063570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:57.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.757+0000 7f73c8968700 1 --2- 172.21.15.121:0/2939696086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73c40bb510 0x7f73c4063570 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7f73c40599c0 tx=0x7f73ac009060).stop 2023-11-15T03:06:57.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.757+0000 7f73c8968700 1 -- 172.21.15.121:0/2939696086 shutdown_connections 2023-11-15T03:06:57.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.757+0000 7f73c8968700 1 --2- 172.21.15.121:0/2939696086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73c40bb510 0x7f73c4063570 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:57.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.757+0000 7f73c8968700 1 --2- 172.21.15.121:0/2939696086 >> v2:172.21.15.190:3300/0 conn(0x7f73c40abaa0 0x7f73c40abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:57.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.757+0000 7f73c8968700 1 -- 172.21.15.121:0/2939696086 >> 172.21.15.121:0/2939696086 conn(0x7f73c4067a60 msgr2=0x7f73c4067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:57.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.757+0000 7f73c8968700 1 -- 172.21.15.121:0/2939696086 shutdown_connections 2023-11-15T03:06:57.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.757+0000 7f73c8968700 1 -- 172.21.15.121:0/2939696086 wait complete. 2023-11-15T03:06:57.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.757+0000 7f73c8968700 1 Processor -- start 2023-11-15T03:06:57.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.757+0000 7f73c8968700 1 -- start start 2023-11-15T03:06:57.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73c8968700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73c40a8880 0x7f73c40c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:57.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73c8968700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73c40abaa0 0x7f73c41675d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:57.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73c8968700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73c40bb510 0x7f73c414bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:57.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73c8968700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c4155fb0 con 0x7f73c40bb510 2023-11-15T03:06:57.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73c8968700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c4156220 con 0x7f73c40a8880 2023-11-15T03:06:57.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73c8968700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c40aa810 con 0x7f73c40abaa0 2023-11-15T03:06:57.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73c359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73c40bb510 0x7f73c414bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:57.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73c359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73c40bb510 0x7f73c414bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54682/0 (socket says 172.21.15.121:54682) 2023-11-15T03:06:57.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73c359e700 1 -- 172.21.15.121:0/1522881632 learned_addr learned my addr 172.21.15.121:0/1522881632 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:57.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73bbfff700 1 --2- 172.21.15.121:0/1522881632 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73c40abaa0 0x7f73c41675d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:57.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73c2d9d700 1 --2- 172.21.15.121:0/1522881632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73c40a8880 0x7f73c40c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:57.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73c359e700 1 -- 172.21.15.121:0/1522881632 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73c40abaa0 msgr2=0x7f73c41675d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:57.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73c359e700 1 --2- 172.21.15.121:0/1522881632 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73c40abaa0 0x7f73c41675d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:57.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73c359e700 1 -- 172.21.15.121:0/1522881632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73c40a8880 msgr2=0x7f73c40c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:57.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73c359e700 1 --2- 172.21.15.121:0/1522881632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73c40a8880 0x7f73c40c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:57.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73c359e700 1 -- 172.21.15.121:0/1522881632 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac009300 con 0x7f73c40bb510 2023-11-15T03:06:57.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73c2d9d700 1 --2- 172.21.15.121:0/1522881632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73c40a8880 0x7f73c40c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:57.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73c359e700 1 --2- 172.21.15.121:0/1522881632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73c40bb510 0x7f73c414bd40 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7f73bc00c7f0 tx=0x7f73bc001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:57.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73baffd700 1 -- 172.21.15.121:0/1522881632 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc010510 con 0x7f73c40bb510 2023-11-15T03:06:57.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73baffd700 1 -- 172.21.15.121:0/1522881632 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73bc010670 con 0x7f73c40bb510 2023-11-15T03:06:57.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73baffd700 1 -- 172.21.15.121:0/1522881632 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc010a10 con 0x7f73c40bb510 2023-11-15T03:06:57.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.761+0000 7f73c8968700 1 -- 172.21.15.121:0/1522881632 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c40aaa80 con 0x7f73c40bb510 2023-11-15T03:06:57.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.765+0000 7f73c8968700 1 -- 172.21.15.121:0/1522881632 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c4168050 con 0x7f73c40bb510 2023-11-15T03:06:57.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.765+0000 7f73c8968700 1 -- 172.21.15.121:0/1522881632 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c406bcf0 con 0x7f73c40bb510 2023-11-15T03:06:57.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.765+0000 7f73baffd700 1 -- 172.21.15.121:0/1522881632 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f73bc01b400 con 0x7f73c40bb510 2023-11-15T03:06:57.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.765+0000 7f73baffd700 1 --2- 172.21.15.121:0/1522881632 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f73a405a4e0 0x7f73a405c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:57.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.765+0000 7f73baffd700 1 -- 172.21.15.121:0/1522881632 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7814+0+0 (secure 0 0 0) 0x7f73bc074bb0 con 0x7f73c40bb510 2023-11-15T03:06:57.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.765+0000 7f73c2d9d700 1 --2- 172.21.15.121:0/1522881632 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f73a405a4e0 0x7f73a405c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:57.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.769+0000 7f73c2d9d700 1 --2- 172.21.15.121:0/1522881632 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f73a405a4e0 0x7f73a405c9a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f73b4008990 tx=0x7f73b4008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:57.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.769+0000 7f73baffd700 1 -- 172.21.15.121:0/1522881632 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f73bc047c90 con 0x7f73c40bb510 2023-11-15T03:06:57.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:57.973+0000 7f73c8968700 1 -- 172.21.15.121:0/1522881632 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f73c40c2890 con 0x7f73c40bb510 2023-11-15T03:06:58.230 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:06:58.221+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:06:58.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.237+0000 7f73baffd700 1 -- 172.21.15.121:0/1522881632 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v201) v1 ==== 161+0+0 (secure 0 0 0) 0x7f73bc04b2b0 con 0x7f73c40bb510 2023-11-15T03:06:58.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.241+0000 7f73c8968700 1 -- 172.21.15.121:0/1522881632 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f73a405a4e0 msgr2=0x7f73a405c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:58.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.241+0000 7f73c8968700 1 --2- 172.21.15.121:0/1522881632 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f73a405a4e0 0x7f73a405c9a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f73b4008990 tx=0x7f73b4008040).stop 2023-11-15T03:06:58.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.241+0000 7f73c8968700 1 -- 172.21.15.121:0/1522881632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73c40bb510 msgr2=0x7f73c414bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:58.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.241+0000 7f73c8968700 1 --2- 172.21.15.121:0/1522881632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73c40bb510 0x7f73c414bd40 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7f73bc00c7f0 tx=0x7f73bc001510).stop 2023-11-15T03:06:58.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.241+0000 7f73c8968700 1 -- 172.21.15.121:0/1522881632 shutdown_connections 2023-11-15T03:06:58.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.241+0000 7f73c8968700 1 --2- 172.21.15.121:0/1522881632 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f73a405a4e0 0x7f73a405c9a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:58.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.241+0000 7f73c8968700 1 --2- 172.21.15.121:0/1522881632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73c40bb510 0x7f73c414bd40 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:58.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.241+0000 7f73c8968700 1 --2- 172.21.15.121:0/1522881632 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73c40abaa0 0x7f73c41675d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:58.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.241+0000 7f73c8968700 1 --2- 172.21.15.121:0/1522881632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73c40a8880 0x7f73c40c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:58.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.241+0000 7f73c8968700 1 -- 172.21.15.121:0/1522881632 >> 172.21.15.121:0/1522881632 conn(0x7f73c4067a60 msgr2=0x7f73c40c3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:58.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.241+0000 7f73c8968700 1 -- 172.21.15.121:0/1522881632 shutdown_connections 2023-11-15T03:06:58.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.241+0000 7f73c8968700 1 -- 172.21.15.121:0/1522881632 wait complete. 2023-11-15T03:06:58.250 INFO:tasks.ceph.mds.a.smithi121.stderr: -1493> 2023-11-15T03:06:48.109+0000 7f73d1525700 -1 mds.0.purge_queue operator(): Error -22 loading Journaler 2023-11-15T03:06:58.250 INFO:tasks.ceph.mds.a.smithi121.stderr: -1490> 2023-11-15T03:06:48.109+0000 7f73d1525700 -1 mds.0.197 unhandled write error (22) Invalid argument, force readonly... 2023-11-15T03:06:58.250 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:06:58.253 INFO:tasks.ceph.mds.a.smithi121.stderr: -449> 2023-11-15T03:06:55.133+0000 7f73d3d2a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:06:58.270 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:06:58.306 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:06:58.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.425+0000 7f550f59e700 1 Processor -- start 2023-11-15T03:06:58.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.429+0000 7f550f59e700 1 -- start start 2023-11-15T03:06:58.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.429+0000 7f550f59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f55100abaa0 0x7f55100abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:58.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.429+0000 7f550f59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f55100bb510 0x7f5510063570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:58.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.429+0000 7f550f59e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55100c2c00 con 0x7f55100a8880 2023-11-15T03:06:58.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.429+0000 7f550f59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5510063a60 con 0x7f55100bb510 2023-11-15T03:06:58.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.429+0000 7f550f59e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55100657b0 con 0x7f55100abaa0 2023-11-15T03:06:58.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.429+0000 7f550dd9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f55100bb510 0x7f5510063570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:58.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.429+0000 7f550dd9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f55100bb510 0x7f5510063570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54722/0 (socket says 172.21.15.121:54722) 2023-11-15T03:06:58.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.429+0000 7f550dd9b700 1 -- 172.21.15.121:0/1261076474 learned_addr learned my addr 172.21.15.121:0/1261076474 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:58.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.429+0000 7f550ed9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f55100a8880 0x7f5510151400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44070/0 (socket says 172.21.15.121:44070) 2023-11-15T03:06:58.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.429+0000 7f550e59c700 1 --2- 172.21.15.121:0/1261076474 >> v2:172.21.15.190:3300/0 conn(0x7f55100abaa0 0x7f55100abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:58.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.429+0000 7f550dd9b700 1 -- 172.21.15.121:0/1261076474 >> v2:172.21.15.190:3300/0 conn(0x7f55100abaa0 msgr2=0x7f55100abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:58.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.429+0000 7f550dd9b700 1 --2- 172.21.15.121:0/1261076474 >> v2:172.21.15.190:3300/0 conn(0x7f55100abaa0 0x7f55100abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:58.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.433+0000 7f550dd9b700 1 -- 172.21.15.121:0/1261076474 >> v1:172.21.15.121:6789/0 conn(0x7f55100a8880 legacy=0x7f5510151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:58.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.433+0000 7f550dd9b700 1 -- 172.21.15.121:0/1261076474 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5510065a90 con 0x7f55100bb510 2023-11-15T03:06:58.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.433+0000 7f550dd9b700 1 --2- 172.21.15.121:0/1261076474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55100bb510 0x7f5510063570 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 rx=0x7f55100599c0 tx=0x7f55040090f0).ready entity=mon.0 client_cookie=e869fb5fea488a49 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:58.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.433+0000 7f550d59a700 1 -- 172.21.15.121:0/1261076474 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550400b600 con 0x7f55100bb510 2023-11-15T03:06:58.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.433+0000 7f550d59a700 1 -- 172.21.15.121:0/1261076474 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550400b760 con 0x7f55100bb510 2023-11-15T03:06:58.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.433+0000 7f550d59a700 1 -- 172.21.15.121:0/1261076474 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550400bae0 con 0x7f55100bb510 2023-11-15T03:06:58.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.433+0000 7f550f59e700 1 -- 172.21.15.121:0/1261076474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55100bb510 msgr2=0x7f5510063570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:58.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.433+0000 7f550f59e700 1 --2- 172.21.15.121:0/1261076474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55100bb510 0x7f5510063570 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 rx=0x7f55100599c0 tx=0x7f55040090f0).stop 2023-11-15T03:06:58.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.433+0000 7f550f59e700 1 -- 172.21.15.121:0/1261076474 shutdown_connections 2023-11-15T03:06:58.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.433+0000 7f550f59e700 1 --2- 172.21.15.121:0/1261076474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55100bb510 0x7f5510063570 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:58.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.433+0000 7f550f59e700 1 --2- 172.21.15.121:0/1261076474 >> v2:172.21.15.190:3300/0 conn(0x7f55100abaa0 0x7f55100abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:58.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.433+0000 7f550f59e700 1 -- 172.21.15.121:0/1261076474 >> 172.21.15.121:0/1261076474 conn(0x7f5510067a60 msgr2=0x7f5510067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:58.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.433+0000 7f550f59e700 1 -- 172.21.15.121:0/1261076474 shutdown_connections 2023-11-15T03:06:58.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.433+0000 7f550f59e700 1 -- 172.21.15.121:0/1261076474 wait complete. 2023-11-15T03:06:58.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550f59e700 1 Processor -- start 2023-11-15T03:06:58.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550f59e700 1 -- start start 2023-11-15T03:06:58.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550f59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55100a8880 0x7f55100c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:58.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550f59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55100abaa0 0x7f55100bf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:58.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550f59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55100bb510 0x7f551014dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:58.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550f59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55100c65e0 con 0x7f55100bb510 2023-11-15T03:06:58.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550f59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55100c68c0 con 0x7f55100abaa0 2023-11-15T03:06:58.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550f59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55100c6ba0 con 0x7f55100a8880 2023-11-15T03:06:58.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550ed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55100bb510 0x7f551014dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:58.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550ed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55100bb510 0x7f551014dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54728/0 (socket says 172.21.15.121:54728) 2023-11-15T03:06:58.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550ed9d700 1 -- 172.21.15.121:0/2539464873 learned_addr learned my addr 172.21.15.121:0/2539464873 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:58.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550e59c700 1 --2- 172.21.15.121:0/2539464873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55100a8880 0x7f55100c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:58.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550dd9b700 1 --2- 172.21.15.121:0/2539464873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55100abaa0 0x7f55100bf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:58.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550ed9d700 1 -- 172.21.15.121:0/2539464873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55100a8880 msgr2=0x7f55100c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:58.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550ed9d700 1 --2- 172.21.15.121:0/2539464873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55100a8880 0x7f55100c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:58.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550ed9d700 1 -- 172.21.15.121:0/2539464873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55100abaa0 msgr2=0x7f55100bf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:58.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550ed9d700 1 --2- 172.21.15.121:0/2539464873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55100abaa0 0x7f55100bf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:58.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550ed9d700 1 -- 172.21.15.121:0/2539464873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5504009310 con 0x7f55100bb510 2023-11-15T03:06:58.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550dd9b700 1 --2- 172.21.15.121:0/2539464873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55100abaa0 0x7f55100bf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:06:58.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550ed9d700 1 --2- 172.21.15.121:0/2539464873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55100bb510 0x7f551014dcf0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7f550800c7c0 tx=0x7f5508001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:58.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f54fe7fc700 1 -- 172.21.15.121:0/2539464873 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508005c70 con 0x7f55100bb510 2023-11-15T03:06:58.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f54fe7fc700 1 -- 172.21.15.121:0/2539464873 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5508005dd0 con 0x7f55100bb510 2023-11-15T03:06:58.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550f59e700 1 -- 172.21.15.121:0/2539464873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55100c6e80 con 0x7f55100bb510 2023-11-15T03:06:58.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f54fe7fc700 1 -- 172.21.15.121:0/2539464873 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508007320 con 0x7f55100bb510 2023-11-15T03:06:58.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.437+0000 7f550f59e700 1 -- 172.21.15.121:0/2539464873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55100c6160 con 0x7f55100bb510 2023-11-15T03:06:58.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.441+0000 7f550f59e700 1 -- 172.21.15.121:0/2539464873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55100c63b0 con 0x7f55100bb510 2023-11-15T03:06:58.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.441+0000 7f54fe7fc700 1 -- 172.21.15.121:0/2539464873 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5508007710 con 0x7f55100bb510 2023-11-15T03:06:58.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.441+0000 7f54fe7fc700 1 --2- 172.21.15.121:0/2539464873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f54f005a510 0x7f54f005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:58.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.444+0000 7f54fe7fc700 1 -- 172.21.15.121:0/2539464873 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 7900+0+0 (secure 0 0 0) 0x7f5508074e00 con 0x7f55100bb510 2023-11-15T03:06:58.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.444+0000 7f550e59c700 1 --2- 172.21.15.121:0/2539464873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f54f005a510 0x7f54f005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:58.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.444+0000 7f550e59c700 1 --2- 172.21.15.121:0/2539464873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f54f005a510 0x7f54f005c9d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f550000bb20 tx=0x7f5500008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:58.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.448+0000 7f54fe7fc700 1 -- 172.21.15.121:0/2539464873 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5508047ef0 con 0x7f55100bb510 2023-11-15T03:06:58.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:58.660+0000 7f550f59e700 1 -- 172.21.15.121:0/2539464873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f5510164590 con 0x7f55100bb510 2023-11-15T03:06:59.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.244+0000 7f54fe7fc700 1 -- 172.21.15.121:0/2539464873 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v202) v1 ==== 107+0+0 (secure 0 0 0) 0x7f5510164590 con 0x7f55100bb510 2023-11-15T03:06:59.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.248+0000 7f550f59e700 1 -- 172.21.15.121:0/2539464873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f54f005a510 msgr2=0x7f54f005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.248+0000 7f550f59e700 1 --2- 172.21.15.121:0/2539464873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f54f005a510 0x7f54f005c9d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f550000bb20 tx=0x7f5500008040).stop 2023-11-15T03:06:59.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.248+0000 7f550f59e700 1 -- 172.21.15.121:0/2539464873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55100bb510 msgr2=0x7f551014dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.248+0000 7f550f59e700 1 --2- 172.21.15.121:0/2539464873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55100bb510 0x7f551014dcf0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7f550800c7c0 tx=0x7f5508001510).stop 2023-11-15T03:06:59.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.248+0000 7f550f59e700 1 -- 172.21.15.121:0/2539464873 shutdown_connections 2023-11-15T03:06:59.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.248+0000 7f550f59e700 1 --2- 172.21.15.121:0/2539464873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f54f005a510 0x7f54f005c9d0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.248+0000 7f550f59e700 1 --2- 172.21.15.121:0/2539464873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55100bb510 0x7f551014dcf0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.248+0000 7f550f59e700 1 --2- 172.21.15.121:0/2539464873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55100abaa0 0x7f55100bf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.248+0000 7f550f59e700 1 --2- 172.21.15.121:0/2539464873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55100a8880 0x7f55100c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.248+0000 7f550f59e700 1 -- 172.21.15.121:0/2539464873 >> 172.21.15.121:0/2539464873 conn(0x7f5510067a60 msgr2=0x7f55100c38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:59.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.248+0000 7f550f59e700 1 -- 172.21.15.121:0/2539464873 shutdown_connections 2023-11-15T03:06:59.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.248+0000 7f550f59e700 1 -- 172.21.15.121:0/2539464873 wait complete. 2023-11-15T03:06:59.258 INFO:teuthology.orchestra.run.smithi121.stderr:nothing to do: rank is not damaged 2023-11-15T03:06:59.276 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:06:59.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.372+0000 7f2419144d00 1 Processor -- start 2023-11-15T03:06:59.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.372+0000 7f2419144d00 1 -- start start 2023-11-15T03:06:59.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f2419144d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3c55fb6c0 0x55d3c56d71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f2419144d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d3c5645b00 0x55d3c56411a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f2419144d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3c56d4670 0x55d3c56cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f2419144d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c5574220 con 0x55d3c55fb6c0 2023-11-15T03:06:59.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f2419144d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c564b7e0 con 0x55d3c56d4670 2023-11-15T03:06:59.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f2419144d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3c562e230 con 0x55d3c5645b00 2023-11-15T03:06:59.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f240d7aa700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3c55fb6c0 0x55d3c56d71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f240d7aa700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3c55fb6c0 0x55d3c56d71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54756/0 (socket says 172.21.15.121:54756) 2023-11-15T03:06:59.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f240d7aa700 1 -- 172.21.15.121:0/714492810 learned_addr learned my addr 172.21.15.121:0/714492810 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:59.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f240cfa9700 1 --2- 172.21.15.121:0/714492810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d3c5645b00 0x55d3c56411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f240d7aa700 1 -- 172.21.15.121:0/714492810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d3c5645b00 msgr2=0x55d3c56411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:59.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f240d7aa700 1 --2- 172.21.15.121:0/714492810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d3c5645b00 0x55d3c56411a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f240d7aa700 1 -- 172.21.15.121:0/714492810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3c56d4670 msgr2=0x55d3c56cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:59.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f240dfab700 1 --2- 172.21.15.121:0/714492810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3c56d4670 0x55d3c56cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f240d7aa700 1 --2- 172.21.15.121:0/714492810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3c56d4670 0x55d3c56cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f240d7aa700 1 -- 172.21.15.121:0/714492810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2404005db0 con 0x55d3c55fb6c0 2023-11-15T03:06:59.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f240dfab700 1 --2- 172.21.15.121:0/714492810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3c56d4670 0x55d3c56cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:59.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f240d7aa700 1 --2- 172.21.15.121:0/714492810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3c55fb6c0 0x55d3c56d71c0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7f2400009980 tx=0x7f240000c040).ready entity=mon.0 client_cookie=475a894e74702bfb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:59.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f23ff7fe700 1 -- 172.21.15.121:0/714492810 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240000f040 con 0x55d3c55fb6c0 2023-11-15T03:06:59.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f23ff7fe700 1 -- 172.21.15.121:0/714492810 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2400008ac0 con 0x55d3c55fb6c0 2023-11-15T03:06:59.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.376+0000 7f23ff7fe700 1 -- 172.21.15.121:0/714492810 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2400008e40 con 0x55d3c55fb6c0 2023-11-15T03:06:59.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f2419144d00 1 -- 172.21.15.121:0/714492810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3c55fb6c0 msgr2=0x55d3c56d71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f2419144d00 1 --2- 172.21.15.121:0/714492810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3c55fb6c0 0x55d3c56d71c0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7f2400009980 tx=0x7f240000c040).stop 2023-11-15T03:06:59.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f2419144d00 1 -- 172.21.15.121:0/714492810 shutdown_connections 2023-11-15T03:06:59.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f2419144d00 1 --2- 172.21.15.121:0/714492810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3c56d4670 0x55d3c56cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f2419144d00 1 --2- 172.21.15.121:0/714492810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d3c5645b00 0x55d3c56411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f2419144d00 1 --2- 172.21.15.121:0/714492810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3c55fb6c0 0x55d3c56d71c0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f2419144d00 1 -- 172.21.15.121:0/714492810 >> 172.21.15.121:0/714492810 conn(0x55d3c564b080 msgr2=0x55d3c56d3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:59.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f2419144d00 1 -- 172.21.15.121:0/714492810 shutdown_connections 2023-11-15T03:06:59.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f2419144d00 1 -- 172.21.15.121:0/714492810 wait complete. 2023-11-15T03:06:59.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f2419144d00 1 Processor -- start 2023-11-15T03:06:59.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f2419144d00 1 -- start start 2023-11-15T03:06:59.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f2419144d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d3c55fb6c0 0x55d3c56cd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f2419144d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3c5645b00 0x55d3c56d6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f2419144d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3c56d4670 0x55d3c56ddf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f2419144d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c56de9d0 con 0x55d3c56d4670 2023-11-15T03:06:59.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f2419144d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c56cc820 con 0x55d3c5645b00 2023-11-15T03:06:59.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f2419144d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3c56ccb00 con 0x55d3c55fb6c0 2023-11-15T03:06:59.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f240dfab700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3c56d4670 0x55d3c56ddf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f240dfab700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3c56d4670 0x55d3c56ddf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54766/0 (socket says 172.21.15.121:54766) 2023-11-15T03:06:59.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 learned_addr learned my addr 172.21.15.121:0/2947319791 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:59.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f240cfa9700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3c5645b00 0x55d3c56d6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.380+0000 7f240d7aa700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d3c55fb6c0 0x55d3c56cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.384+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d3c55fb6c0 msgr2=0x55d3c56cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.384+0000 7f240dfab700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d3c55fb6c0 0x55d3c56cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.384+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3c5645b00 msgr2=0x55d3c56d6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.384+0000 7f240dfab700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3c5645b00 0x55d3c56d6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.384+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2400009580 con 0x55d3c56d4670 2023-11-15T03:06:59.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.384+0000 7f240dfab700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3c56d4670 0x55d3c56ddf10 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 rx=0x7f24080016e0 tx=0x7f2408008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:59.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.384+0000 7f23fdffb700 1 -- 172.21.15.121:0/2947319791 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408006170 con 0x55d3c56d4670 2023-11-15T03:06:59.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.384+0000 7f23fdffb700 1 -- 172.21.15.121:0/2947319791 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24080062d0 con 0x55d3c56d4670 2023-11-15T03:06:59.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.384+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d3c559d770 con 0x55d3c56d4670 2023-11-15T03:06:59.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.384+0000 7f23fdffb700 1 -- 172.21.15.121:0/2947319791 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408003680 con 0x55d3c56d4670 2023-11-15T03:06:59.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.384+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3c556ca70 con 0x55d3c56d4670 2023-11-15T03:06:59.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.384+0000 7f23fdffb700 1 -- 172.21.15.121:0/2947319791 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2408003a30 con 0x55d3c56d4670 2023-11-15T03:06:59.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.388+0000 7f23fdffb700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f23f0059020 0x7f23f005b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.388+0000 7f23fdffb700 1 -- 172.21.15.121:0/2947319791 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 7900+0+0 (secure 0 0 0) 0x7f2408073560 con 0x55d3c56d4670 2023-11-15T03:06:59.401 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:06:59.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.388+0000 7f240d7aa700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f23f0059020 0x7f23f005b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.388+0000 7f2419144d00 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55d3c56d7330 0x55d3c56cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.388+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56e50a0 con 0x55d3c56d7330 2023-11-15T03:06:59.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.388+0000 7f240cfa9700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55d3c56d7330 0x55d3c56cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.388+0000 7f240d7aa700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f23f0059020 0x7f23f005b4e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f2400009980 tx=0x7f2400018000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:59.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.388+0000 7f240cfa9700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55d3c56d7330 0x55d3c56cae60 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:59.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.392+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v113'221 uv214 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56d7330 2023-11-15T03:06:59.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.392+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c56e5480 con 0x55d3c56d7330 2023-11-15T03:06:59.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.392+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v113'222 uv222 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56d7330 2023-11-15T03:06:59.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.392+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56e5860 con 0x55d3c56d7330 2023-11-15T03:06:59.408 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:06:59.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.396+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v113'223 uv223 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56d7330 2023-11-15T03:06:59.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.396+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f2650 con 0x55d3c56d7330 2023-11-15T03:06:59.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.396+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v113'224 uv224 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56d7330 2023-11-15T03:06:59.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.396+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f2a00 con 0x55d3c56d7330 2023-11-15T03:06:59.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.400+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v113'225 uv225 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56d7330 2023-11-15T03:06:59.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.400+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f1640 con 0x55d3c56d7330 2023-11-15T03:06:59.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.400+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v113'226 uv226 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56d7330 2023-11-15T03:06:59.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.400+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f1a40 con 0x55d3c56d7330 2023-11-15T03:06:59.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.404+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v113'227 uv220 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56d7330 2023-11-15T03:06:59.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.404+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c56f1df0 con 0x55d3c56d7330 2023-11-15T03:06:59.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.408+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v113'228 uv228 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56d7330 2023-11-15T03:06:59.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.408+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f1df0 con 0x55d3c56d7330 2023-11-15T03:06:59.416 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:06:59.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.408+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v113'229 uv229 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56d7330 2023-11-15T03:06:59.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.408+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f1df0 con 0x55d3c56d7330 2023-11-15T03:06:59.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.412+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v113'230 uv230 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56d7330 2023-11-15T03:06:59.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.412+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f1df0 con 0x55d3c56d7330 2023-11-15T03:06:59.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.412+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v113'231 uv231 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56d7330 2023-11-15T03:06:59.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.412+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f1df0 con 0x55d3c56d7330 2023-11-15T03:06:59.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.416+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v113'232 uv232 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56d7330 2023-11-15T03:06:59.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.416+0000 7f2419144d00 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55d3c56f2db0 0x55d3c56f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.416+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f5690 con 0x55d3c56f2db0 2023-11-15T03:06:59.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.416+0000 7f240d7aa700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55d3c56f2db0 0x55d3c56f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.416+0000 7f240d7aa700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55d3c56f2db0 0x55d3c56f51a0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:59.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.420+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v113'347 uv334 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.420+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c56f5a40 con 0x55d3c56f2db0 2023-11-15T03:06:59.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.424+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v113'348 uv348 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.424+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f5df0 con 0x55d3c56f2db0 2023-11-15T03:06:59.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.428+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v113'349 uv349 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.435 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:06:59.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.428+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f71b0 con 0x55d3c56f2db0 2023-11-15T03:06:59.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.428+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v113'350 uv350 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.428+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f7560 con 0x55d3c56f2db0 2023-11-15T03:06:59.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.432+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v113'351 uv351 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.432+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f61a0 con 0x55d3c56f2db0 2023-11-15T03:06:59.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.436+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v113'352 uv352 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.436+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f6a90 con 0x55d3c56f2db0 2023-11-15T03:06:59.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.440+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v113'353 uv353 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.440+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f7910 con 0x55d3c56f2db0 2023-11-15T03:06:59.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.440+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v113'354 uv340 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.440+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c56f7910 con 0x55d3c56f2db0 2023-11-15T03:06:59.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.444+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v113'355 uv355 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.444+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f7910 con 0x55d3c56f2db0 2023-11-15T03:06:59.454 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:06:59.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.448+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v113'356 uv356 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.448+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f8920 con 0x55d3c56f2db0 2023-11-15T03:06:59.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.448+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v113'357 uv357 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.448+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f8920 con 0x55d3c56f2db0 2023-11-15T03:06:59.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.452+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v113'358 uv358 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.452+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f7910 con 0x55d3c56f2db0 2023-11-15T03:06:59.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.456+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v113'359 uv359 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.456+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f7910 con 0x55d3c56f2db0 2023-11-15T03:06:59.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.456+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v113'360 uv346 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.456+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c56f7910 con 0x55d3c56f2db0 2023-11-15T03:06:59.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.460+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v113'361 uv361 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.460+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f7910 con 0x55d3c56f2db0 2023-11-15T03:06:59.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.464+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v113'362 uv362 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.471 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:06:59.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.464+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f8920 con 0x55d3c56f2db0 2023-11-15T03:06:59.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.464+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v113'363 uv363 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.464+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f8920 con 0x55d3c56f2db0 2023-11-15T03:06:59.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.468+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v113'364 uv364 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.468+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56f7910 con 0x55d3c56f2db0 2023-11-15T03:06:59.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.472+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v113'365 uv365 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.472+0000 7f2419144d00 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55d3c56f7e20 0x55d3c56fa210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.472+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fa700 con 0x55d3c56f7e20 2023-11-15T03:06:59.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.472+0000 7f240cfa9700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55d3c56f7e20 0x55d3c56fa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.472+0000 7f240cfa9700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55d3c56f7e20 0x55d3c56fa210 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:59.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.476+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v113'191 uv184 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56f7e20 2023-11-15T03:06:59.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.476+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c56fa700 con 0x55d3c56f7e20 2023-11-15T03:06:59.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.480+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v113'192 uv192 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56f7e20 2023-11-15T03:06:59.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.480+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fa700 con 0x55d3c56f7e20 2023-11-15T03:06:59.490 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:06:59.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.484+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v113'193 uv193 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56f7e20 2023-11-15T03:06:59.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.484+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fabf0 con 0x55d3c56f7e20 2023-11-15T03:06:59.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.484+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v113'194 uv194 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56f7e20 2023-11-15T03:06:59.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.484+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fabf0 con 0x55d3c56f7e20 2023-11-15T03:06:59.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.488+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v113'195 uv195 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56f7e20 2023-11-15T03:06:59.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.488+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fabf0 con 0x55d3c56f7e20 2023-11-15T03:06:59.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.492+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v113'196 uv190 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56f7e20 2023-11-15T03:06:59.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.492+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c56fabf0 con 0x55d3c56f7e20 2023-11-15T03:06:59.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.496+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v113'197 uv197 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56f7e20 2023-11-15T03:06:59.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.496+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fabf0 con 0x55d3c56f7e20 2023-11-15T03:06:59.505 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:06:59.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.496+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v113'198 uv198 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56f7e20 2023-11-15T03:06:59.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.496+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fabf0 con 0x55d3c56f7e20 2023-11-15T03:06:59.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.500+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v113'199 uv199 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56f7e20 2023-11-15T03:06:59.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.500+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fabf0 con 0x55d3c56f7e20 2023-11-15T03:06:59.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.504+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v113'200 uv200 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56f7e20 2023-11-15T03:06:59.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.504+0000 7f2419144d00 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55d3c56fb100 0x55d3c56fd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.504+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fd9e0 con 0x55d3c56fb100 2023-11-15T03:06:59.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.504+0000 7f240dfab700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55d3c56fb100 0x55d3c56fd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.504+0000 7f240dfab700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55d3c56fb100 0x55d3c56fd4f0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:59.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.508+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v113'230 uv229 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.508+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c56fde10 con 0x55d3c56fb100 2023-11-15T03:06:59.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.512+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v113'231 uv231 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.512+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fe2f0 con 0x55d3c56fb100 2023-11-15T03:06:59.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.516+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v113'232 uv232 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.524 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:06:59.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.516+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fa7e0 con 0x55d3c56fb100 2023-11-15T03:06:59.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.520+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v113'233 uv233 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.520+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5702d70 con 0x55d3c56fb100 2023-11-15T03:06:59.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.524+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v113'234 uv234 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.524+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fe6f0 con 0x55d3c56fb100 2023-11-15T03:06:59.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.524+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v113'235 uv226 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.524+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c56feaf0 con 0x55d3c56fb100 2023-11-15T03:06:59.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.528+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v113'236 uv236 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.528+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56ff4e0 con 0x55d3c56fb100 2023-11-15T03:06:59.539 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:06:59.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.532+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v113'237 uv237 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.532+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56ff4a0 con 0x55d3c56fb100 2023-11-15T03:06:59.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.536+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v113'238 uv238 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.536+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c57008f0 con 0x55d3c56fb100 2023-11-15T03:06:59.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.536+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v113'239 uv239 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.536+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56ff500 con 0x55d3c56fb100 2023-11-15T03:06:59.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.540+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v113'240 uv240 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.540+0000 7f2419144d00 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55d3c56ffe10 0x55d3c57001f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.540+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c57006e0 con 0x55d3c56ffe10 2023-11-15T03:06:59.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.540+0000 7f240d7aa700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55d3c56ffe10 0x55d3c57001f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.540+0000 7f240d7aa700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55d3c56ffe10 0x55d3c57001f0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:59.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.544+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v113'111 uv110 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56ffe10 2023-11-15T03:06:59.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.544+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c57006e0 con 0x55d3c56ffe10 2023-11-15T03:06:59.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.548+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v113'112 uv112 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56ffe10 2023-11-15T03:06:59.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.548+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c57006e0 con 0x55d3c56ffe10 2023-11-15T03:06:59.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.552+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v113'113 uv113 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56ffe10 2023-11-15T03:06:59.559 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:06:59.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.552+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c57006e0 con 0x55d3c56ffe10 2023-11-15T03:06:59.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.552+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v113'114 uv114 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56ffe10 2023-11-15T03:06:59.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.552+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c57006e0 con 0x55d3c56ffe10 2023-11-15T03:06:59.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.556+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v113'115 uv115 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56ffe10 2023-11-15T03:06:59.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.556+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c57006e0 con 0x55d3c56ffe10 2023-11-15T03:06:59.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.556+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v113'116 uv116 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56ffe10 2023-11-15T03:06:59.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.556+0000 7f2419144d00 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55d3c57036d0 0x55d3c5705ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.556+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5706040 con 0x55d3c57036d0 2023-11-15T03:06:59.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.556+0000 7f240cfa9700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55d3c57036d0 0x55d3c5705ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.560+0000 7f240cfa9700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55d3c57036d0 0x55d3c5705ac0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:59.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.560+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v113'80 uv79 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.560+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c5706420 con 0x55d3c57036d0 2023-11-15T03:06:59.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.564+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v113'81 uv81 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.564+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5706420 con 0x55d3c57036d0 2023-11-15T03:06:59.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.568+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v113'82 uv82 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.575 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:06:59.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.568+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5706420 con 0x55d3c57036d0 2023-11-15T03:06:59.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.568+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v113'83 uv83 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.568+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5706420 con 0x55d3c57036d0 2023-11-15T03:06:59.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.572+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v113'111 uv110 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.572+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c5706420 con 0x55d3c57036d0 2023-11-15T03:06:59.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.576+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v113'112 uv112 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.576+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5706420 con 0x55d3c57036d0 2023-11-15T03:06:59.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.580+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v113'113 uv113 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.586 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:06:59.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.580+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5707430 con 0x55d3c57036d0 2023-11-15T03:06:59.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.580+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v113'114 uv114 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.580+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5707430 con 0x55d3c57036d0 2023-11-15T03:06:59.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.584+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v113'115 uv115 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.584+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5706420 con 0x55d3c57036d0 2023-11-15T03:06:59.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.588+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v113'116 uv116 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.588+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5706420 con 0x55d3c56f2db0 2023-11-15T03:06:59.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.588+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v113'205 uv198 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.588+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c5706420 con 0x55d3c56f2db0 2023-11-15T03:06:59.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.592+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v113'206 uv206 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.592+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5706420 con 0x55d3c56f2db0 2023-11-15T03:06:59.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.596+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v113'207 uv207 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.605 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:06:59.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.596+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5707430 con 0x55d3c56f2db0 2023-11-15T03:06:59.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.600+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v113'208 uv208 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.600+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5707430 con 0x55d3c56f2db0 2023-11-15T03:06:59.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.604+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v113'209 uv209 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.604+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5706420 con 0x55d3c56f2db0 2023-11-15T03:06:59.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.604+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v113'210 uv210 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.604+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5706420 con 0x55d3c56f2db0 2023-11-15T03:06:59.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.608+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v113'211 uv204 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.608+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c5706420 con 0x55d3c56f2db0 2023-11-15T03:06:59.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.612+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v113'212 uv212 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.612+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5706420 con 0x55d3c56f2db0 2023-11-15T03:06:59.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.616+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v113'213 uv213 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.622 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:06:59.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.616+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5706420 con 0x55d3c56f2db0 2023-11-15T03:06:59.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.620+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v113'214 uv214 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.620+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c56f2db0 2023-11-15T03:06:59.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.620+0000 7f240d7aa700 1 -- 172.21.15.121:0/2947319791 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v113'215 uv215 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2400014640 con 0x55d3c56f2db0 2023-11-15T03:06:59.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.620+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c56f7e20 2023-11-15T03:06:59.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.624+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v113'91 uv90 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56f7e20 2023-11-15T03:06:59.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.624+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c56f7e20 2023-11-15T03:06:59.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.628+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v113'92 uv92 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56f7e20 2023-11-15T03:06:59.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.628+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c56f7e20 2023-11-15T03:06:59.637 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:06:59.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.628+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v113'93 uv93 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56f7e20 2023-11-15T03:06:59.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.628+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c56f7e20 2023-11-15T03:06:59.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.632+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v113'94 uv94 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56f7e20 2023-11-15T03:06:59.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.632+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c56f7e20 2023-11-15T03:06:59.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.636+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v113'95 uv95 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c56f7e20 2023-11-15T03:06:59.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.636+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c57036d0 2023-11-15T03:06:59.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.636+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v113'111 uv110 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.636+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c57036d0 2023-11-15T03:06:59.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.640+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v113'112 uv112 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.640+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c57036d0 2023-11-15T03:06:59.650 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:06:59.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.644+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v113'113 uv113 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.644+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c57036d0 2023-11-15T03:06:59.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.648+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v113'114 uv114 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.648+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c57036d0 2023-11-15T03:06:59.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.652+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v113'115 uv115 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.652+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c57036d0 2023-11-15T03:06:59.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.652+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v113'116 uv116 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.652+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c57036d0 2023-11-15T03:06:59.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.656+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v113'92 uv91 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.656+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c57036d0 2023-11-15T03:06:59.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.660+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v113'93 uv93 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.660+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c57036d0 2023-11-15T03:06:59.671 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:06:59.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.664+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v113'94 uv94 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.664+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c57036d0 2023-11-15T03:06:59.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.668+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v113'95 uv95 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.668+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c57036d0 2023-11-15T03:06:59.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.668+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v113'96 uv96 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.668+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c57036d0 2023-11-15T03:06:59.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.672+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v113'91 uv90 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.672+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c57036d0 2023-11-15T03:06:59.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.676+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v113'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.676+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c57036d0 2023-11-15T03:06:59.686 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:06:59.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.676+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v113'93 uv93 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.680+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c56fef70 con 0x55d3c57036d0 2023-11-15T03:06:59.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.684+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v113'94 uv94 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.684+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c57092b0 con 0x55d3c57036d0 2023-11-15T03:06:59.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.688+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v113'95 uv95 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.688+0000 7f2419144d00 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55d3c5706aa0 0x55d3c57098d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.688+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c5706aa0 2023-11-15T03:06:59.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.688+0000 7f240dfab700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55d3c5706aa0 0x55d3c57098d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.688+0000 7f240dfab700 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55d3c5706aa0 0x55d3c57098d0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:59.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.692+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v113'74 uv73 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c5706aa0 2023-11-15T03:06:59.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.692+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c5706aa0 2023-11-15T03:06:59.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.696+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v113'75 uv75 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c5706aa0 2023-11-15T03:06:59.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.696+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c5706aa0 2023-11-15T03:06:59.705 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:06:59.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.696+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v113'76 uv76 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c5706aa0 2023-11-15T03:06:59.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.696+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c5706aa0 2023-11-15T03:06:59.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.700+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v113'77 uv77 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c5706aa0 2023-11-15T03:06:59.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.700+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c57036d0 2023-11-15T03:06:59.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.704+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v113'111 uv110 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.704+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c57036d0 2023-11-15T03:06:59.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.708+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v113'112 uv112 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.708+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c57036d0 2023-11-15T03:06:59.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.712+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v113'113 uv113 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.719 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:06:59.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.712+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c570add0 con 0x55d3c57036d0 2023-11-15T03:06:59.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.712+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v113'114 uv114 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.712+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c570add0 con 0x55d3c57036d0 2023-11-15T03:06:59.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.716+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v113'115 uv115 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.716+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c57036d0 2023-11-15T03:06:59.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.720+0000 7f240cfa9700 1 -- 172.21.15.121:0/2947319791 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v113'116 uv116 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2404000bc0 con 0x55d3c57036d0 2023-11-15T03:06:59.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.720+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c56fb100 2023-11-15T03:06:59.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.724+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v113'94 uv93 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.724+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c56fb100 2023-11-15T03:06:59.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.724+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v113'95 uv95 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.724+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c56fb100 2023-11-15T03:06:59.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.728+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v113'96 uv96 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.736 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:06:59.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.728+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c56fb100 2023-11-15T03:06:59.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.732+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v113'97 uv97 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.732+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c56fb100 2023-11-15T03:06:59.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.736+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v113'98 uv98 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.736+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c56fb100 2023-11-15T03:06:59.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.736+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v113'220 uv212 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.736+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c56fb100 2023-11-15T03:06:59.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.740+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v113'221 uv221 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.740+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c56fb100 2023-11-15T03:06:59.750 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:06:59.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.744+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v113'222 uv222 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.744+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c56fb100 2023-11-15T03:06:59.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.744+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v113'223 uv223 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.744+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c56fb100 2023-11-15T03:06:59.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.748+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v113'224 uv224 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.748+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c56fb100 2023-11-15T03:06:59.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.752+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v113'225 uv219 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.752+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c56fb100 2023-11-15T03:06:59.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.752+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v113'226 uv226 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.752+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c56fb100 2023-11-15T03:06:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.756+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v113'227 uv227 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.764 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:06:59.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.756+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c570add0 con 0x55d3c56fb100 2023-11-15T03:06:59.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.760+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v113'228 uv228 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.760+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c570add0 con 0x55d3c56fb100 2023-11-15T03:06:59.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.760+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v113'229 uv229 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.760+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c5709dc0 con 0x55d3c56fb100 2023-11-15T03:06:59.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.764+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v113'230 uv230 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.764+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x55d3c570a2b0 con 0x55d3c56fb100 2023-11-15T03:06:59.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.764+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v113'231 uv231 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f24080485f0 con 0x55d3c56fb100 2023-11-15T03:06:59.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55d3c56f7e20 msgr2=0x55d3c56fa210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55d3c56f7e20 0x55d3c56fa210 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55d3c56d7330 msgr2=0x55d3c56cae60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55d3c56d7330 0x55d3c56cae60 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55d3c56ffe10 msgr2=0x55d3c57001f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55d3c56ffe10 0x55d3c57001f0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55d3c56f2db0 msgr2=0x55d3c56f51a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55d3c56f2db0 0x55d3c56f51a0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55d3c5706aa0 msgr2=0x55d3c57098d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55d3c5706aa0 0x55d3c57098d0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 reap_dead start 2023-11-15T03:06:59.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55d3c56fb100 msgr2=0x55d3c56fd4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55d3c56fb100 0x55d3c56fd4f0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f240dfab700 1 -- 172.21.15.121:0/2947319791 reap_dead start 2023-11-15T03:06:59.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55d3c57036d0 msgr2=0x55d3c5705ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55d3c57036d0 0x55d3c5705ac0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f23f0059020 msgr2=0x7f23f005b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f23f0059020 0x7f23f005b4e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f2400009980 tx=0x7f2400018000).stop 2023-11-15T03:06:59.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3c56d4670 msgr2=0x55d3c56ddf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3c56d4670 0x55d3c56ddf10 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 rx=0x7f24080016e0 tx=0x7f2408008040).stop 2023-11-15T03:06:59.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 shutdown_connections 2023-11-15T03:06:59.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55d3c57036d0 0x55d3c5705ac0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f23f0059020 0x7f23f005b4e0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 --2- 172.21.15.121:0/2947319791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3c56d4670 0x55d3c56ddf10 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 >> 172.21.15.121:0/2947319791 conn(0x55d3c564b080 msgr2=0x55d3c56411a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:59.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 shutdown_connections 2023-11-15T03:06:59.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.768+0000 7f2419144d00 1 -- 172.21.15.121:0/2947319791 wait complete. 2023-11-15T03:06:59.784 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds_snaptable - 2023-11-15T03:06:59.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.876+0000 7fda8f132d00 1 Processor -- start 2023-11-15T03:06:59.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.876+0000 7fda8f132d00 1 -- start start 2023-11-15T03:06:59.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.876+0000 7fda8f132d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568614bba30 0x5568615916d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.876+0000 7fda8f132d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556861506e70 0x556861591bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.876+0000 7fda8f132d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55686159c000 0x5568615074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.876+0000 7fda8f132d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556861469cc0 con 0x556861506e70 2023-11-15T03:06:59.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.876+0000 7fda8f132d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556861434220 con 0x5568614bba30 2023-11-15T03:06:59.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.876+0000 7fda8f132d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568614ee240 con 0x55686159c000 2023-11-15T03:06:59.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.876+0000 7fda7aeb6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556861506e70 0x556861591bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.876+0000 7fda7aeb6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556861506e70 0x556861591bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54796/0 (socket says 172.21.15.121:54796) 2023-11-15T03:06:59.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.876+0000 7fda7aeb6700 1 -- 172.21.15.121:0/4286493144 learned_addr learned my addr 172.21.15.121:0/4286493144 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:59.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.876+0000 7fda836b7700 1 --2- 172.21.15.121:0/4286493144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568614bba30 0x5568615916d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.876+0000 7fda7aeb6700 1 -- 172.21.15.121:0/4286493144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55686159c000 msgr2=0x5568615074b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:59.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.876+0000 7fda7aeb6700 1 --2- 172.21.15.121:0/4286493144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55686159c000 0x5568615074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:06:59.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.876+0000 7fda7aeb6700 1 -- 172.21.15.121:0/4286493144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568614bba30 msgr2=0x5568615916d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:06:59.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.876+0000 7fda7aeb6700 1 --2- 172.21.15.121:0/4286493144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568614bba30 0x5568615916d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.880+0000 7fda7aeb6700 1 -- 172.21.15.121:0/4286493144 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7400a180 con 0x556861506e70 2023-11-15T03:06:59.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.880+0000 7fda7aeb6700 1 --2- 172.21.15.121:0/4286493144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556861506e70 0x556861591bc0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 rx=0x7fda6c001d90 tx=0x7fda6c001860).ready entity=mon.0 client_cookie=c83a0aad94413c86 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:59.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.880+0000 7fda826b5700 1 -- 172.21.15.121:0/4286493144 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c00bb50 con 0x556861506e70 2023-11-15T03:06:59.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.880+0000 7fda826b5700 1 -- 172.21.15.121:0/4286493144 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6c008070 con 0x556861506e70 2023-11-15T03:06:59.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.880+0000 7fda826b5700 1 -- 172.21.15.121:0/4286493144 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c00bcb0 con 0x556861506e70 2023-11-15T03:06:59.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.880+0000 7fda8f132d00 1 -- 172.21.15.121:0/4286493144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556861506e70 msgr2=0x556861591bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.880+0000 7fda8f132d00 1 --2- 172.21.15.121:0/4286493144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556861506e70 0x556861591bc0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 rx=0x7fda6c001d90 tx=0x7fda6c001860).stop 2023-11-15T03:06:59.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.880+0000 7fda8f132d00 1 -- 172.21.15.121:0/4286493144 shutdown_connections 2023-11-15T03:06:59.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.880+0000 7fda8f132d00 1 --2- 172.21.15.121:0/4286493144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55686159c000 0x5568615074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:06:59.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.880+0000 7fda8f132d00 1 --2- 172.21.15.121:0/4286493144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556861506e70 0x556861591bc0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.880+0000 7fda8f132d00 1 --2- 172.21.15.121:0/4286493144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568614bba30 0x5568615916d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.880+0000 7fda8f132d00 1 -- 172.21.15.121:0/4286493144 >> 172.21.15.121:0/4286493144 conn(0x55686150b080 msgr2=0x5568615083b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:59.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.880+0000 7fda8f132d00 1 -- 172.21.15.121:0/4286493144 shutdown_connections 2023-11-15T03:06:59.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.880+0000 7fda8f132d00 1 -- 172.21.15.121:0/4286493144 wait complete. 2023-11-15T03:06:59.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.880+0000 7fda8f132d00 1 Processor -- start 2023-11-15T03:06:59.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.880+0000 7fda8f132d00 1 -- start start 2023-11-15T03:06:59.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda8f132d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568614bba30 0x55686159a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda8f132d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556861506e70 0x5568615013d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda8f132d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55686159c000 0x5568615ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda8f132d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55686158fce0 con 0x5568614bba30 2023-11-15T03:06:59.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda8f132d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556861593980 con 0x556861506e70 2023-11-15T03:06:59.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda8f132d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55686158f990 con 0x55686159c000 2023-11-15T03:06:59.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda836b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568614bba30 0x55686159a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda836b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568614bba30 0x55686159a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54812/0 (socket says 172.21.15.121:54812) 2023-11-15T03:06:59.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda836b7700 1 -- 172.21.15.121:0/4063264546 learned_addr learned my addr 172.21.15.121:0/4063264546 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:06:59.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda7aeb6700 1 --2- 172.21.15.121:0/4063264546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556861506e70 0x5568615013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda836b7700 1 -- 172.21.15.121:0/4063264546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55686159c000 msgr2=0x5568615ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda836b7700 1 --2- 172.21.15.121:0/4063264546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55686159c000 0x5568615ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:06:59.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda836b7700 1 -- 172.21.15.121:0/4063264546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556861506e70 msgr2=0x5568615013d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda836b7700 1 --2- 172.21.15.121:0/4063264546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556861506e70 0x5568615013d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda836b7700 1 -- 172.21.15.121:0/4063264546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c00a040 con 0x5568614bba30 2023-11-15T03:06:59.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda7aeb6700 1 --2- 172.21.15.121:0/4063264546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556861506e70 0x5568615013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:06:59.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda836b7700 1 --2- 172.21.15.121:0/4063264546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568614bba30 0x55686159a620 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7fda7400f000 tx=0x7fda74015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:59.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda80eb2700 1 -- 172.21.15.121:0/4063264546 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74005950 con 0x5568614bba30 2023-11-15T03:06:59.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda80eb2700 1 -- 172.21.15.121:0/4063264546 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7400ccf0 con 0x5568614bba30 2023-11-15T03:06:59.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda8f132d00 1 -- 172.21.15.121:0/4063264546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55686142ca70 con 0x5568614bba30 2023-11-15T03:06:59.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda80eb2700 1 -- 172.21.15.121:0/4063264546 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7400c980 con 0x5568614bba30 2023-11-15T03:06:59.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.884+0000 7fda8f132d00 1 -- 172.21.15.121:0/4063264546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556861438010 con 0x5568614bba30 2023-11-15T03:06:59.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.888+0000 7fda80eb2700 1 -- 172.21.15.121:0/4063264546 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fda74003040 con 0x5568614bba30 2023-11-15T03:06:59.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.888+0000 7fda80eb2700 1 --2- 172.21.15.121:0/4063264546 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fda64058ff0 0x7fda6405b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.888+0000 7fda80eb2700 1 -- 172.21.15.121:0/4063264546 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 7900+0+0 (secure 0 0 0) 0x7fda74075b10 con 0x5568614bba30 2023-11-15T03:06:59.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.888+0000 7fda7aeb6700 1 --2- 172.21.15.121:0/4063264546 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fda64058ff0 0x7fda6405b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.888+0000 7fda8f132d00 1 --2- 172.21.15.121:0/4063264546 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5568615913e0 0x556861594dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:06:59.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.888+0000 7fda8f132d00 1 -- 172.21.15.121:0/4063264546 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:1 13.2d 13:b50e409b:::mds_snaptable:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected e113) v8 -- 0x5568615018c0 con 0x5568615913e0 2023-11-15T03:06:59.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.888+0000 7fda83eb8700 1 --2- 172.21.15.121:0/4063264546 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5568615913e0 0x556861594dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:06:59.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.888+0000 7fda7aeb6700 1 --2- 172.21.15.121:0/4063264546 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fda64058ff0 0x7fda6405b4b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x556861596600 tx=0x7fda6c015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:59.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.892+0000 7fda83eb8700 1 --2- 172.21.15.121:0/4063264546 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5568615913e0 0x556861594dc0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:06:59.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.892+0000 7fda83eb8700 1 -- 172.21.15.121:0/4063264546 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~80] v113'96 uv96 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fda7c00b2c0 con 0x5568615913e0 2023-11-15T03:06:59.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.892+0000 7fda8f132d00 1 -- 172.21.15.121:0/4063264546 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5568615913e0 msgr2=0x556861594dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.892+0000 7fda8f132d00 1 --2- 172.21.15.121:0/4063264546 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5568615913e0 0x556861594dc0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:06:59.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.892+0000 7fda8f132d00 1 -- 172.21.15.121:0/4063264546 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fda64058ff0 msgr2=0x7fda6405b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.892+0000 7fda8f132d00 1 --2- 172.21.15.121:0/4063264546 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fda64058ff0 0x7fda6405b4b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x556861596600 tx=0x7fda6c015000).stop 2023-11-15T03:06:59.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.892+0000 7fda8f132d00 1 -- 172.21.15.121:0/4063264546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568614bba30 msgr2=0x55686159a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:06:59.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.892+0000 7fda8f132d00 1 --2- 172.21.15.121:0/4063264546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568614bba30 0x55686159a620 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7fda7400f000 tx=0x7fda74015000).stop 2023-11-15T03:06:59.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.892+0000 7fda83eb8700 1 -- 172.21.15.121:0/4063264546 reap_dead start 2023-11-15T03:06:59.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.892+0000 7fda8f132d00 1 -- 172.21.15.121:0/4063264546 shutdown_connections 2023-11-15T03:06:59.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.892+0000 7fda8f132d00 1 -- 172.21.15.121:0/4063264546 >> 172.21.15.121:0/4063264546 conn(0x55686150b080 msgr2=0x556861507620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:06:59.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.896+0000 7fda8f132d00 1 -- 172.21.15.121:0/4063264546 shutdown_connections 2023-11-15T03:06:59.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:06:59.896+0000 7fda8f132d00 1 -- 172.21.15.121:0/4063264546 wait complete. 2023-11-15T03:06:59.909 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:07:00.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.116+0000 7f42b16b9700 1 Processor -- start 2023-11-15T03:07:00.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.116+0000 7f42b16b9700 1 -- start start 2023-11-15T03:07:00.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.120+0000 7f42b16b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f42ac14e380 0x7f42ac0c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:00.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.120+0000 7f42b16b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42ac0ad270 0x7f42ac065680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:00.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.120+0000 7f42b16b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ac0c7e90 con 0x7f42ac0c6040 2023-11-15T03:07:00.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.120+0000 7f42b16b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ac0c8320 con 0x7f42ac14e380 2023-11-15T03:07:00.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.120+0000 7f42b16b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac0c7080 con 0x7f42ac0ad270 2023-11-15T03:07:00.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.120+0000 7f42abfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f42ac14e380 0x7f42ac0c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:00.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.120+0000 7f42a3fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f42ac0c6040 0x7f42ac0acb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44096/0 (socket says 172.21.15.121:44096) 2023-11-15T03:07:00.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.120+0000 7f42a3fff700 1 -- 172.21.15.121:0/2690341542 learned_addr learned my addr 172.21.15.121:0/2690341542 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:00.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.120+0000 7f42b0eb8700 1 --2- 172.21.15.121:0/2690341542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42ac0ad270 0x7f42ac065680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:00.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.120+0000 7f42ab7fe700 1 -- 172.21.15.121:0/2690341542 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034104659 0 0) 0x7f42ac0c7e90 con 0x7f42ac0c6040 2023-11-15T03:07:00.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.120+0000 7f42abfff700 1 -- 172.21.15.121:0/2690341542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42ac0ad270 msgr2=0x7f42ac065680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:00.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.120+0000 7f42abfff700 1 --2- 172.21.15.121:0/2690341542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42ac0ad270 0x7f42ac065680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:00.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.120+0000 7f42abfff700 1 -- 172.21.15.121:0/2690341542 >> v1:172.21.15.121:6789/0 conn(0x7f42ac0c6040 legacy=0x7f42ac0acb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:00.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.120+0000 7f42abfff700 1 -- 172.21.15.121:0/2690341542 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac14e760 con 0x7f42ac14e380 2023-11-15T03:07:00.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.120+0000 7f42abfff700 1 --2- 172.21.15.121:0/2690341542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42ac14e380 0x7f42ac0c5b50 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7f429c009760 tx=0x7f429c009060).ready entity=mon.0 client_cookie=9f4b3484879fac63 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:00.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.120+0000 7f42ab7fe700 1 -- 172.21.15.121:0/2690341542 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c00f510 con 0x7f42ac14e380 2023-11-15T03:07:00.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.120+0000 7f42ab7fe700 1 -- 172.21.15.121:0/2690341542 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429c00f670 con 0x7f42ac14e380 2023-11-15T03:07:00.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.124+0000 7f42ab7fe700 1 -- 172.21.15.121:0/2690341542 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c00f9f0 con 0x7f42ac14e380 2023-11-15T03:07:00.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.124+0000 7f42b16b9700 1 -- 172.21.15.121:0/2690341542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42ac14e380 msgr2=0x7f42ac0c5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:00.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.124+0000 7f42b16b9700 1 --2- 172.21.15.121:0/2690341542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42ac14e380 0x7f42ac0c5b50 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7f429c009760 tx=0x7f429c009060).stop 2023-11-15T03:07:00.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.124+0000 7f42b16b9700 1 -- 172.21.15.121:0/2690341542 shutdown_connections 2023-11-15T03:07:00.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.124+0000 7f42b16b9700 1 --2- 172.21.15.121:0/2690341542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42ac0ad270 0x7f42ac065680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:00.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.124+0000 7f42b16b9700 1 --2- 172.21.15.121:0/2690341542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42ac14e380 0x7f42ac0c5b50 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:00.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.124+0000 7f42b16b9700 1 -- 172.21.15.121:0/2690341542 >> 172.21.15.121:0/2690341542 conn(0x7f42ac067a60 msgr2=0x7f42ac067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:00.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.124+0000 7f42b16b9700 1 -- 172.21.15.121:0/2690341542 shutdown_connections 2023-11-15T03:07:00.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.124+0000 7f42b16b9700 1 -- 172.21.15.121:0/2690341542 wait complete. 2023-11-15T03:07:00.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.124+0000 7f42b16b9700 1 Processor -- start 2023-11-15T03:07:00.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.124+0000 7f42b16b9700 1 -- start start 2023-11-15T03:07:00.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42b16b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42ac0ad270 0x7f42ac14a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:00.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42b16b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42ac0c6040 0x7f42ac0c19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:00.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42b16b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42ac14e380 0x7f42ac0c1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:00.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42b16b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac0aeae0 con 0x7f42ac0c6040 2023-11-15T03:07:00.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42b16b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac14a8f0 con 0x7f42ac0ad270 2023-11-15T03:07:00.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42b16b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac0c2d00 con 0x7f42ac14e380 2023-11-15T03:07:00.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42a3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42ac0c6040 0x7f42ac0c19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:00.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42a3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42ac0c6040 0x7f42ac0c19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54830/0 (socket says 172.21.15.121:54830) 2023-11-15T03:07:00.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42a3fff700 1 -- 172.21.15.121:0/1782265426 learned_addr learned my addr 172.21.15.121:0/1782265426 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:00.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42b0eb8700 1 --2- 172.21.15.121:0/1782265426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42ac14e380 0x7f42ac0c1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:00.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42abfff700 1 --2- 172.21.15.121:0/1782265426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42ac0ad270 0x7f42ac14a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:00.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42a3fff700 1 -- 172.21.15.121:0/1782265426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42ac14e380 msgr2=0x7f42ac0c1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:00.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42a3fff700 1 --2- 172.21.15.121:0/1782265426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42ac14e380 0x7f42ac0c1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:00.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42a3fff700 1 -- 172.21.15.121:0/1782265426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42ac0ad270 msgr2=0x7f42ac14a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:00.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42a3fff700 1 --2- 172.21.15.121:0/1782265426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42ac0ad270 0x7f42ac14a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:00.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42a3fff700 1 -- 172.21.15.121:0/1782265426 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429c0092a0 con 0x7f42ac0c6040 2023-11-15T03:07:00.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42a3fff700 1 --2- 172.21.15.121:0/1782265426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42ac0c6040 0x7f42ac0c19e0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7f429400c510 tx=0x7f4294007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:00.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42a8ff9700 1 -- 172.21.15.121:0/1782265426 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429400ce80 con 0x7f42ac0c6040 2023-11-15T03:07:00.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42a8ff9700 1 -- 172.21.15.121:0/1782265426 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42940072e0 con 0x7f42ac0c6040 2023-11-15T03:07:00.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42b16b9700 1 -- 172.21.15.121:0/1782265426 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ac0aeda0 con 0x7f42ac0c6040 2023-11-15T03:07:00.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42a8ff9700 1 -- 172.21.15.121:0/1782265426 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294007660 con 0x7f42ac0c6040 2023-11-15T03:07:00.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.128+0000 7f42b16b9700 1 -- 172.21.15.121:0/1782265426 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ac0c2740 con 0x7f42ac0c6040 2023-11-15T03:07:00.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.132+0000 7f42b16b9700 1 -- 172.21.15.121:0/1782265426 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ac069f60 con 0x7f42ac0c6040 2023-11-15T03:07:00.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.132+0000 7f42a8ff9700 1 -- 172.21.15.121:0/1782265426 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f42940059b0 con 0x7f42ac0c6040 2023-11-15T03:07:00.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.132+0000 7f42a8ff9700 1 --2- 172.21.15.121:0/1782265426 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f428c05a4e0 0x7f428c05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:00.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.132+0000 7f42a8ff9700 1 -- 172.21.15.121:0/1782265426 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 7900+0+0 (secure 0 0 0) 0x7f4294074710 con 0x7f42ac0c6040 2023-11-15T03:07:00.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.132+0000 7f42abfff700 1 --2- 172.21.15.121:0/1782265426 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f428c05a4e0 0x7f428c05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:00.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.136+0000 7f42abfff700 1 --2- 172.21.15.121:0/1782265426 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f428c05a4e0 0x7f428c05c9a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f42ac158860 tx=0x7f429c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:00.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.140+0000 7f42a8ff9700 1 -- 172.21.15.121:0/1782265426 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f42ac069f60 con 0x7f42ac0c6040 2023-11-15T03:07:00.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:00.336+0000 7f42b16b9700 1 -- 172.21.15.121:0/1782265426 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f42ac0aef30 con 0x7f42ac0c6040 2023-11-15T03:07:01.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.260+0000 7f42a8ff9700 1 -- 172.21.15.121:0/1782265426 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v203) v1 ==== 162+0+0 (secure 0 0 0) 0x7f4294018020 con 0x7f42ac0c6040 2023-11-15T03:07:01.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.264+0000 7f42b16b9700 1 -- 172.21.15.121:0/1782265426 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f428c05a4e0 msgr2=0x7f428c05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:01.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.264+0000 7f42b16b9700 1 --2- 172.21.15.121:0/1782265426 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f428c05a4e0 0x7f428c05c9a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f42ac158860 tx=0x7f429c017000).stop 2023-11-15T03:07:01.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.264+0000 7f42b16b9700 1 -- 172.21.15.121:0/1782265426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42ac0c6040 msgr2=0x7f42ac0c19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:01.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.264+0000 7f42b16b9700 1 --2- 172.21.15.121:0/1782265426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42ac0c6040 0x7f42ac0c19e0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7f429400c510 tx=0x7f4294007c70).stop 2023-11-15T03:07:01.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.264+0000 7f42b16b9700 1 -- 172.21.15.121:0/1782265426 shutdown_connections 2023-11-15T03:07:01.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.264+0000 7f42b16b9700 1 --2- 172.21.15.121:0/1782265426 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f428c05a4e0 0x7f428c05c9a0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:01.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.264+0000 7f42b16b9700 1 --2- 172.21.15.121:0/1782265426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42ac14e380 0x7f42ac0c1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:01.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.264+0000 7f42b16b9700 1 --2- 172.21.15.121:0/1782265426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42ac0c6040 0x7f42ac0c19e0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:01.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.264+0000 7f42b16b9700 1 --2- 172.21.15.121:0/1782265426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42ac0ad270 0x7f42ac14a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:01.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.264+0000 7f42b16b9700 1 -- 172.21.15.121:0/1782265426 >> 172.21.15.121:0/1782265426 conn(0x7f42ac067a60 msgr2=0x7f42ac0c0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:01.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.264+0000 7f42b16b9700 1 -- 172.21.15.121:0/1782265426 shutdown_connections 2023-11-15T03:07:01.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.264+0000 7f42b16b9700 1 -- 172.21.15.121:0/1782265426 wait complete. 2023-11-15T03:07:01.276 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:07:01.280 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:07:01.272+0000 7fdf17ed5700 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': virtual void SnapServer::decode_server_state(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 5 < 101: Malformed input 2023-11-15T03:07:01.297 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:01.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.448+0000 7f9d5b21b700 1 Processor -- start 2023-11-15T03:07:01.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.448+0000 7f9d5b21b700 1 -- start start 2023-11-15T03:07:01.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.452+0000 7f9d5b21b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9d541562f0 0x7f9d54155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:01.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.452+0000 7f9d5b21b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d540c4c30 con 0x7f9d54154ca0 2023-11-15T03:07:01.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.452+0000 7f9d5b21b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d540a6f50 con 0x7f9d540a57c0 2023-11-15T03:07:01.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.452+0000 7f9d5b21b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d540a8280 con 0x7f9d541562f0 2023-11-15T03:07:01.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.452+0000 7f9d58fb7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9d54154ca0 0x7f9d540a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44112/0 (socket says 172.21.15.121:44112) 2023-11-15T03:07:01.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.452+0000 7f9d58fb7700 1 -- 172.21.15.121:0/1468696487 learned_addr learned my addr 172.21.15.121:0/1468696487 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:01.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.452+0000 7f9d53fff700 1 --2- 172.21.15.121:0/1468696487 >> v2:172.21.15.190:3300/0 conn(0x7f9d541562f0 0x7f9d54155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:01.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.456+0000 7f9d537fe700 1 -- 172.21.15.121:0/1468696487 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313516952 0 0) 0x7f9d540c4c30 con 0x7f9d54154ca0 2023-11-15T03:07:01.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.456+0000 7f9d537fe700 1 -- 172.21.15.121:0/1468696487 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d440033f0 con 0x7f9d54154ca0 2023-11-15T03:07:01.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.456+0000 7f9d53fff700 1 -- 172.21.15.121:0/1468696487 >> v1:172.21.15.190:6789/0 conn(0x7f9d540a57c0 legacy=0x7f9d540c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:01.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.456+0000 7f9d53fff700 1 -- 172.21.15.121:0/1468696487 >> v1:172.21.15.121:6789/0 conn(0x7f9d54154ca0 legacy=0x7f9d540a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:01.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.456+0000 7f9d53fff700 1 -- 172.21.15.121:0/1468696487 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d540a7230 con 0x7f9d541562f0 2023-11-15T03:07:01.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.456+0000 7f9d53fff700 1 --2- 172.21.15.121:0/1468696487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d541562f0 0x7f9d54155080 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f9d40009760 tx=0x7f9d40009060).ready entity=mon.1 client_cookie=585329ed96800859 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:01.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.456+0000 7f9d537fe700 1 -- 172.21.15.121:0/1468696487 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4000f510 con 0x7f9d541562f0 2023-11-15T03:07:01.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.456+0000 7f9d537fe700 1 -- 172.21.15.121:0/1468696487 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4000f670 con 0x7f9d541562f0 2023-11-15T03:07:01.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.456+0000 7f9d537fe700 1 -- 172.21.15.121:0/1468696487 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4000f9f0 con 0x7f9d541562f0 2023-11-15T03:07:01.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.456+0000 7f9d5b21b700 1 -- 172.21.15.121:0/1468696487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d541562f0 msgr2=0x7f9d54155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:01.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.456+0000 7f9d5b21b700 1 --2- 172.21.15.121:0/1468696487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d541562f0 0x7f9d54155080 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f9d40009760 tx=0x7f9d40009060).stop 2023-11-15T03:07:01.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.456+0000 7f9d5b21b700 1 -- 172.21.15.121:0/1468696487 shutdown_connections 2023-11-15T03:07:01.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.456+0000 7f9d5b21b700 1 --2- 172.21.15.121:0/1468696487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d541562f0 0x7f9d54155080 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:01.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.456+0000 7f9d5b21b700 1 -- 172.21.15.121:0/1468696487 >> 172.21.15.121:0/1468696487 conn(0x7f9d54062190 msgr2=0x7f9d540625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:01.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.460+0000 7f9d5b21b700 1 -- 172.21.15.121:0/1468696487 shutdown_connections 2023-11-15T03:07:01.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.460+0000 7f9d5b21b700 1 -- 172.21.15.121:0/1468696487 wait complete. 2023-11-15T03:07:01.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.460+0000 7f9d5b21b700 1 Processor -- start 2023-11-15T03:07:01.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.460+0000 7f9d5b21b700 1 -- start start 2023-11-15T03:07:01.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d5b21b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d540a57c0 0x7f9d541539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:01.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d5b21b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d54154ca0 0x7f9d5414f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:01.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d5b21b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d541562f0 0x7f9d54066c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:01.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d5b21b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5414f400 con 0x7f9d541562f0 2023-11-15T03:07:01.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d5b21b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540aa270 con 0x7f9d540a57c0 2023-11-15T03:07:01.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d5b21b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d54160c90 con 0x7f9d54154ca0 2023-11-15T03:07:01.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d597b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d541562f0 0x7f9d54066c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:01.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d597b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d541562f0 0x7f9d54066c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54846/0 (socket says 172.21.15.121:54846) 2023-11-15T03:07:01.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d597b8700 1 -- 172.21.15.121:0/3099848071 learned_addr learned my addr 172.21.15.121:0/3099848071 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:01.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d58fb7700 1 --2- 172.21.15.121:0/3099848071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d540a57c0 0x7f9d541539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:01.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d53fff700 1 --2- 172.21.15.121:0/3099848071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d54154ca0 0x7f9d5414f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:01.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d597b8700 1 -- 172.21.15.121:0/3099848071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d54154ca0 msgr2=0x7f9d5414f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:01.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d597b8700 1 --2- 172.21.15.121:0/3099848071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d54154ca0 0x7f9d5414f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:01.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d597b8700 1 -- 172.21.15.121:0/3099848071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d540a57c0 msgr2=0x7f9d541539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:01.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d597b8700 1 --2- 172.21.15.121:0/3099848071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d540a57c0 0x7f9d541539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:01.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d597b8700 1 -- 172.21.15.121:0/3099848071 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d400092a0 con 0x7f9d541562f0 2023-11-15T03:07:01.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d597b8700 1 --2- 172.21.15.121:0/3099848071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d541562f0 0x7f9d54066c70 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7f9d4c00c790 tx=0x7f9d4c0015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:01.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d50ff9700 1 -- 172.21.15.121:0/3099848071 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c00e4a0 con 0x7f9d541562f0 2023-11-15T03:07:01.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d50ff9700 1 -- 172.21.15.121:0/3099848071 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4c00e600 con 0x7f9d541562f0 2023-11-15T03:07:01.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d5b21b700 1 -- 172.21.15.121:0/3099848071 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d54160f70 con 0x7f9d541562f0 2023-11-15T03:07:01.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d50ff9700 1 -- 172.21.15.121:0/3099848071 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c00e980 con 0x7f9d541562f0 2023-11-15T03:07:01.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d5b21b700 1 -- 172.21.15.121:0/3099848071 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d54153eb0 con 0x7f9d541562f0 2023-11-15T03:07:01.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.464+0000 7f9d5b21b700 1 -- 172.21.15.121:0/3099848071 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d5414f5b0 con 0x7f9d541562f0 2023-11-15T03:07:01.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.468+0000 7f9d50ff9700 1 -- 172.21.15.121:0/3099848071 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9d4c01b3b0 con 0x7f9d541562f0 2023-11-15T03:07:01.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.468+0000 7f9d50ff9700 1 --2- 172.21.15.121:0/3099848071 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9d3805a510 0x7f9d3805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:01.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.468+0000 7f9d50ff9700 1 -- 172.21.15.121:0/3099848071 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 7986+0+0 (secure 0 0 0) 0x7f9d4c074630 con 0x7f9d541562f0 2023-11-15T03:07:01.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.468+0000 7f9d58fb7700 1 --2- 172.21.15.121:0/3099848071 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9d3805a510 0x7f9d3805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:01.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.468+0000 7f9d58fb7700 1 --2- 172.21.15.121:0/3099848071 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9d3805a510 0x7f9d3805c9d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f9d48007fd0 tx=0x7f9d48009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:01.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.472+0000 7f9d50ff9700 1 -- 172.21.15.121:0/3099848071 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9d5414f5b0 con 0x7f9d541562f0 2023-11-15T03:07:01.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.664+0000 7f9d5b21b700 1 -- 172.21.15.121:0/3099848071 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d5414f5b0 con 0x7f9d541562f0 2023-11-15T03:07:01.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.668+0000 7f9d50ff9700 1 -- 172.21.15.121:0/3099848071 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 204 v204) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f9d5414f5b0 con 0x7f9d541562f0 2023-11-15T03:07:01.675 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:01.675 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":204,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":204,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:01.265010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7607":{"gid":7607,"name":"a","rank":0,"incarnation":204,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/1704127295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1704127295},{"type":"v1","addr":"172.21.15.121:6836","nonce":1704127295}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:01.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.672+0000 7f9d5b21b700 1 -- 172.21.15.121:0/3099848071 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9d3805a510 msgr2=0x7f9d3805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:01.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.672+0000 7f9d5b21b700 1 --2- 172.21.15.121:0/3099848071 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9d3805a510 0x7f9d3805c9d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f9d48007fd0 tx=0x7f9d48009040).stop 2023-11-15T03:07:01.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.672+0000 7f9d5b21b700 1 -- 172.21.15.121:0/3099848071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d541562f0 msgr2=0x7f9d54066c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:01.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.672+0000 7f9d5b21b700 1 --2- 172.21.15.121:0/3099848071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d541562f0 0x7f9d54066c70 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7f9d4c00c790 tx=0x7f9d4c0015e0).stop 2023-11-15T03:07:01.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.672+0000 7f9d5b21b700 1 -- 172.21.15.121:0/3099848071 shutdown_connections 2023-11-15T03:07:01.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.672+0000 7f9d5b21b700 1 --2- 172.21.15.121:0/3099848071 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9d3805a510 0x7f9d3805c9d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:01.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.672+0000 7f9d5b21b700 1 --2- 172.21.15.121:0/3099848071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d541562f0 0x7f9d54066c70 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:01.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.672+0000 7f9d5b21b700 1 --2- 172.21.15.121:0/3099848071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d54154ca0 0x7f9d5414f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:01.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.672+0000 7f9d5b21b700 1 --2- 172.21.15.121:0/3099848071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d540a57c0 0x7f9d541539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:01.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.672+0000 7f9d5b21b700 1 -- 172.21.15.121:0/3099848071 >> 172.21.15.121:0/3099848071 conn(0x7f9d54062190 msgr2=0x7f9d540c1810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:01.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.672+0000 7f9d5b21b700 1 -- 172.21.15.121:0/3099848071 shutdown_connections 2023-11-15T03:07:01.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:01.672+0000 7f9d5b21b700 1 -- 172.21.15.121:0/3099848071 wait complete. 2023-11-15T03:07:01.682 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 204 2023-11-15T03:07:01.701 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:07:02.275 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:07:02.268+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:07:02.275 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:07:02.268+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:07:02.286 INFO:tasks.ceph.mds.a.smithi121.stderr: -26> 2023-11-15T03:07:01.272+0000 7fdf17ed5700 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': virtual void SnapServer::decode_server_state(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 5 < 101: Malformed input 2023-11-15T03:07:02.325 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:07:06.703 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:06.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.864+0000 7fc9453a9700 1 Processor -- start 2023-11-15T03:07:06.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.864+0000 7fc9453a9700 1 -- start start 2023-11-15T03:07:06.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.864+0000 7fc9453a9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc940152780 0x7fc940159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:06.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.864+0000 7fc9453a9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc9400c65d0 0x7fc9400c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:06.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.864+0000 7fc9453a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9400a5510 0x7fc9400a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:06.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.864+0000 7fc9453a9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9400c3fa0 con 0x7fc9400c65d0 2023-11-15T03:07:06.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.864+0000 7fc9453a9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc940159630 con 0x7fc940152780 2023-11-15T03:07:06.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.864+0000 7fc9453a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9400c30b0 con 0x7fc9400a5510 2023-11-15T03:07:06.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc93e7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc9400c65d0 0x7fc9400c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:06.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc93f7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9400a5510 0x7fc9400a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:06.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc93f7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9400a5510 0x7fc9400a59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:49248/0 (socket says 172.21.15.121:49248) 2023-11-15T03:07:06.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc93f7fe700 1 -- 172.21.15.121:0/3053458207 learned_addr learned my addr 172.21.15.121:0/3053458207 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:06.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc93effd700 1 --2- 172.21.15.121:0/3053458207 >> v2:172.21.15.190:3300/0 conn(0x7fc940152780 0x7fc940159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:06.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc93e7fc700 1 -- 172.21.15.121:0/3053458207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9400a5510 msgr2=0x7fc9400a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:06.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc93e7fc700 1 --2- 172.21.15.121:0/3053458207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9400a5510 0x7fc9400a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:06.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc93e7fc700 1 -- 172.21.15.121:0/3053458207 >> v2:172.21.15.190:3300/0 conn(0x7fc940152780 msgr2=0x7fc940159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:06.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc93e7fc700 1 --2- 172.21.15.121:0/3053458207 >> v2:172.21.15.190:3300/0 conn(0x7fc940152780 0x7fc940159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:06.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc93e7fc700 1 -- 172.21.15.121:0/3053458207 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9400c3390 con 0x7fc9400c65d0 2023-11-15T03:07:06.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc93e7fc700 1 --2- 172.21.15.121:0/3053458207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9400c65d0 0x7fc9400c6a50 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 rx=0x7fc9400653f0 tx=0x7fc934009060).ready entity=mon.0 client_cookie=5f1f49108172caa server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:06.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc93dffb700 1 -- 172.21.15.121:0/3053458207 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93400d4a0 con 0x7fc9400c65d0 2023-11-15T03:07:06.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc93dffb700 1 -- 172.21.15.121:0/3053458207 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93400d600 con 0x7fc9400c65d0 2023-11-15T03:07:06.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc93dffb700 1 -- 172.21.15.121:0/3053458207 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93400d980 con 0x7fc9400c65d0 2023-11-15T03:07:06.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc9453a9700 1 -- 172.21.15.121:0/3053458207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9400c65d0 msgr2=0x7fc9400c6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:06.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc9453a9700 1 --2- 172.21.15.121:0/3053458207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9400c65d0 0x7fc9400c6a50 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 rx=0x7fc9400653f0 tx=0x7fc934009060).stop 2023-11-15T03:07:06.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc9453a9700 1 -- 172.21.15.121:0/3053458207 shutdown_connections 2023-11-15T03:07:06.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc9453a9700 1 --2- 172.21.15.121:0/3053458207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9400a5510 0x7fc9400a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:06.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc9453a9700 1 --2- 172.21.15.121:0/3053458207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9400c65d0 0x7fc9400c6a50 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:06.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc9453a9700 1 --2- 172.21.15.121:0/3053458207 >> v2:172.21.15.190:3300/0 conn(0x7fc940152780 0x7fc940159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:06.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc9453a9700 1 -- 172.21.15.121:0/3053458207 >> 172.21.15.121:0/3053458207 conn(0x7fc940062190 msgr2=0x7fc9400625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:06.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc9453a9700 1 -- 172.21.15.121:0/3053458207 shutdown_connections 2023-11-15T03:07:06.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.868+0000 7fc9453a9700 1 -- 172.21.15.121:0/3053458207 wait complete. 2023-11-15T03:07:06.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.872+0000 7fc9453a9700 1 Processor -- start 2023-11-15T03:07:06.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.872+0000 7fc9453a9700 1 -- start start 2023-11-15T03:07:06.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc9453a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9400a5510 0x7fc940168a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:06.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc9453a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9400c65d0 0x7fc940150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:06.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc9453a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc940152780 0x7fc940149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:06.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc9453a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94016c400 con 0x7fc940152780 2023-11-15T03:07:06.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc9453a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94016c6e0 con 0x7fc9400c65d0 2023-11-15T03:07:06.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc9453a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94016c9c0 con 0x7fc9400a5510 2023-11-15T03:07:06.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc93f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc940152780 0x7fc940149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:06.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc93f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc940152780 0x7fc940149900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53268/0 (socket says 172.21.15.121:53268) 2023-11-15T03:07:06.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc93f7fe700 1 -- 172.21.15.121:0/2094276810 learned_addr learned my addr 172.21.15.121:0/2094276810 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:06.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc93effd700 1 --2- 172.21.15.121:0/2094276810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9400a5510 0x7fc940168a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:06.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc93e7fc700 1 --2- 172.21.15.121:0/2094276810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9400c65d0 0x7fc940150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:06.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc93f7fe700 1 -- 172.21.15.121:0/2094276810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9400a5510 msgr2=0x7fc940168a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:06.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc93f7fe700 1 --2- 172.21.15.121:0/2094276810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9400a5510 0x7fc940168a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:06.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc93f7fe700 1 -- 172.21.15.121:0/2094276810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9400c65d0 msgr2=0x7fc940150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:06.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc93f7fe700 1 --2- 172.21.15.121:0/2094276810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9400c65d0 0x7fc940150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:06.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc93f7fe700 1 -- 172.21.15.121:0/2094276810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9340092a0 con 0x7fc940152780 2023-11-15T03:07:06.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc93f7fe700 1 --2- 172.21.15.121:0/2094276810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc940152780 0x7fc940149900 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7fc938009e50 tx=0x7fc93800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:06.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc926ffd700 1 -- 172.21.15.121:0/2094276810 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938009c30 con 0x7fc940152780 2023-11-15T03:07:06.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc926ffd700 1 -- 172.21.15.121:0/2094276810 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93800f040 con 0x7fc940152780 2023-11-15T03:07:06.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc926ffd700 1 -- 172.21.15.121:0/2094276810 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93800cb50 con 0x7fc940152780 2023-11-15T03:07:06.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc9453a9700 1 -- 172.21.15.121:0/2094276810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94016cca0 con 0x7fc940152780 2023-11-15T03:07:06.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.876+0000 7fc9453a9700 1 -- 172.21.15.121:0/2094276810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc940149df0 con 0x7fc940152780 2023-11-15T03:07:06.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.880+0000 7fc9453a9700 1 -- 172.21.15.121:0/2094276810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9400ab2d0 con 0x7fc940152780 2023-11-15T03:07:06.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.880+0000 7fc926ffd700 1 -- 172.21.15.121:0/2094276810 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc938014410 con 0x7fc940152780 2023-11-15T03:07:06.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.880+0000 7fc926ffd700 1 --2- 172.21.15.121:0/2094276810 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc92c05a510 0x7fc92c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:06.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.880+0000 7fc926ffd700 1 -- 172.21.15.121:0/2094276810 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 7986+0+0 (secure 0 0 0) 0x7fc938074fe0 con 0x7fc940152780 2023-11-15T03:07:06.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.880+0000 7fc93effd700 1 --2- 172.21.15.121:0/2094276810 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc92c05a510 0x7fc92c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:06.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.880+0000 7fc93effd700 1 --2- 172.21.15.121:0/2094276810 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc92c05a510 0x7fc92c05c9d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7fc930009b00 tx=0x7fc930009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:06.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:06.888+0000 7fc926ffd700 1 -- 172.21.15.121:0/2094276810 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc9380a3af0 con 0x7fc940152780 2023-11-15T03:07:07.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.080+0000 7fc9453a9700 1 -- 172.21.15.121:0/2094276810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc94006c150 con 0x7fc940152780 2023-11-15T03:07:07.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.084+0000 7fc926ffd700 1 -- 172.21.15.121:0/2094276810 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 206 v206) v1 ==== 95+0+4496 (secure 0 0 0) 0x7fc9400ab2d0 con 0x7fc940152780 2023-11-15T03:07:07.091 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:07.091 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":206,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1007300937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1007300937},{"type":"v1","addr":"172.21.15.121:6836","nonce":1007300937}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":206}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:02.271022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:07.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.084+0000 7fc9453a9700 1 -- 172.21.15.121:0/2094276810 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc92c05a510 msgr2=0x7fc92c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:07.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.084+0000 7fc9453a9700 1 --2- 172.21.15.121:0/2094276810 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc92c05a510 0x7fc92c05c9d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7fc930009b00 tx=0x7fc930009060).stop 2023-11-15T03:07:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.084+0000 7fc9453a9700 1 -- 172.21.15.121:0/2094276810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc940152780 msgr2=0x7fc940149900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.084+0000 7fc9453a9700 1 --2- 172.21.15.121:0/2094276810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc940152780 0x7fc940149900 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7fc938009e50 tx=0x7fc93800b040).stop 2023-11-15T03:07:07.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.084+0000 7fc9453a9700 1 -- 172.21.15.121:0/2094276810 shutdown_connections 2023-11-15T03:07:07.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.084+0000 7fc9453a9700 1 --2- 172.21.15.121:0/2094276810 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc92c05a510 0x7fc92c05c9d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.084+0000 7fc9453a9700 1 --2- 172.21.15.121:0/2094276810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc940152780 0x7fc940149900 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.084+0000 7fc9453a9700 1 --2- 172.21.15.121:0/2094276810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9400c65d0 0x7fc940150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.084+0000 7fc9453a9700 1 --2- 172.21.15.121:0/2094276810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9400a5510 0x7fc940168a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.084+0000 7fc9453a9700 1 -- 172.21.15.121:0/2094276810 >> 172.21.15.121:0/2094276810 conn(0x7fc940062190 msgr2=0x7fc9400a8970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:07.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.084+0000 7fc9453a9700 1 -- 172.21.15.121:0/2094276810 shutdown_connections 2023-11-15T03:07:07.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.084+0000 7fc9453a9700 1 -- 172.21.15.121:0/2094276810 wait complete. 2023-11-15T03:07:07.103 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 206 2023-11-15T03:07:07.113 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:07:07.113 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:07.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.256+0000 7f82727b9700 1 Processor -- start 2023-11-15T03:07:07.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.256+0000 7f82727b9700 1 -- start start 2023-11-15T03:07:07.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.260+0000 7f82727b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f826c152850 0x7f826c0c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:07.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.260+0000 7f82727b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f826c15be30 0x7f826c15c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:07.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.260+0000 7f82727b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826c0c1d50 con 0x7f826c0c3280 2023-11-15T03:07:07.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.260+0000 7f82727b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826c0c1860 con 0x7f826c152850 2023-11-15T03:07:07.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.260+0000 7f82727b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826c15cb80 con 0x7f826c15be30 2023-11-15T03:07:07.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.260+0000 7f8271fb8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f826c15be30 0x7f826c15c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:07.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.260+0000 7f8270fb6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f826c152850 0x7f826c0c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:07.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.260+0000 7f8271fb8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f826c15be30 0x7f826c15c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:41510/0 (socket says 172.21.15.121:41510) 2023-11-15T03:07:07.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.260+0000 7f8271fb8700 1 -- 172.21.15.121:0/1974172643 learned_addr learned my addr 172.21.15.121:0/1974172643 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:07.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.260+0000 7f8271fb8700 1 -- 172.21.15.121:0/1974172643 >> v2:172.21.15.121:3300/0 conn(0x7f826c152850 msgr2=0x7f826c0c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:07.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.260+0000 7f8271fb8700 1 --2- 172.21.15.121:0/1974172643 >> v2:172.21.15.121:3300/0 conn(0x7f826c152850 0x7f826c0c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.260+0000 7f8271fb8700 1 -- 172.21.15.121:0/1974172643 >> v1:172.21.15.190:6789/0 conn(0x7f826c0c3280 legacy=0x7f826c152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:07.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.260+0000 7f8271fb8700 1 -- 172.21.15.121:0/1974172643 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c0a9240 con 0x7f826c15be30 2023-11-15T03:07:07.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.260+0000 7f8271fb8700 1 --2- 172.21.15.121:0/1974172643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f826c15be30 0x7f826c15c280 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f826c0c0d80 tx=0x7f8264009840).ready entity=mon.1 client_cookie=e5aa18e96dee6db6 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:07.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.260+0000 7f8263fff700 1 -- 172.21.15.121:0/1974172643 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826400e070 con 0x7f826c15be30 2023-11-15T03:07:07.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.260+0000 7f8263fff700 1 -- 172.21.15.121:0/1974172643 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826400ba20 con 0x7f826c15be30 2023-11-15T03:07:07.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.260+0000 7f8263fff700 1 -- 172.21.15.121:0/1974172643 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826400bda0 con 0x7f826c15be30 2023-11-15T03:07:07.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.264+0000 7f82727b9700 1 -- 172.21.15.121:0/1974172643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f826c15be30 msgr2=0x7f826c15c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:07.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.264+0000 7f82727b9700 1 --2- 172.21.15.121:0/1974172643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f826c15be30 0x7f826c15c280 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f826c0c0d80 tx=0x7f8264009840).stop 2023-11-15T03:07:07.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.264+0000 7f82727b9700 1 -- 172.21.15.121:0/1974172643 shutdown_connections 2023-11-15T03:07:07.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.264+0000 7f82727b9700 1 --2- 172.21.15.121:0/1974172643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f826c15be30 0x7f826c15c280 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.264+0000 7f82727b9700 1 --2- 172.21.15.121:0/1974172643 >> v2:172.21.15.121:3300/0 conn(0x7f826c152850 0x7f826c0c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.264+0000 7f82727b9700 1 -- 172.21.15.121:0/1974172643 >> 172.21.15.121:0/1974172643 conn(0x7f826c062190 msgr2=0x7f826c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:07.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.264+0000 7f82727b9700 1 -- 172.21.15.121:0/1974172643 shutdown_connections 2023-11-15T03:07:07.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.264+0000 7f82727b9700 1 -- 172.21.15.121:0/1974172643 wait complete. 2023-11-15T03:07:07.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.268+0000 7f82727b9700 1 Processor -- start 2023-11-15T03:07:07.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.268+0000 7f82727b9700 1 -- start start 2023-11-15T03:07:07.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.268+0000 7f82727b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f826c0c3280 0x7f826c1690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:07.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.268+0000 7f82727b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f826c152850 0x7f826c14cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:07.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.268+0000 7f82727b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f826c15be30 0x7f826c14ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:07.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.268+0000 7f82727b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c0c6ad0 con 0x7f826c152850 2023-11-15T03:07:07.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.268+0000 7f82727b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c16c460 con 0x7f826c0c3280 2023-11-15T03:07:07.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.268+0000 7f82727b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826c1695d0 con 0x7f826c15be30 2023-11-15T03:07:07.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.268+0000 7f8270fb6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f826c152850 0x7f826c14cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:07.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.268+0000 7f8270fb6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f826c152850 0x7f826c14cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53282/0 (socket says 172.21.15.121:53282) 2023-11-15T03:07:07.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.268+0000 7f8270fb6700 1 -- 172.21.15.121:0/1440911633 learned_addr learned my addr 172.21.15.121:0/1440911633 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:07.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.268+0000 7f82717b7700 1 --2- 172.21.15.121:0/1440911633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f826c0c3280 0x7f826c1690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:07.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.268+0000 7f8271fb8700 1 --2- 172.21.15.121:0/1440911633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f826c15be30 0x7f826c14ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:07.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.268+0000 7f8270fb6700 1 -- 172.21.15.121:0/1440911633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f826c15be30 msgr2=0x7f826c14ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:07.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.268+0000 7f8270fb6700 1 --2- 172.21.15.121:0/1440911633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f826c15be30 0x7f826c14ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.268+0000 7f8270fb6700 1 -- 172.21.15.121:0/1440911633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f826c0c3280 msgr2=0x7f826c1690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:07.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.268+0000 7f8270fb6700 1 --2- 172.21.15.121:0/1440911633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f826c0c3280 0x7f826c1690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.268+0000 7f8270fb6700 1 -- 172.21.15.121:0/1440911633 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8264009ae0 con 0x7f826c152850 2023-11-15T03:07:07.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.268+0000 7f82717b7700 1 --2- 172.21.15.121:0/1440911633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f826c0c3280 0x7f826c1690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:07:07.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.272+0000 7f8270fb6700 1 --2- 172.21.15.121:0/1440911633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f826c152850 0x7f826c14cf60 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 rx=0x7f825c00e930 tx=0x7f825c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:07.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.272+0000 7f82617fa700 1 -- 172.21.15.121:0/1440911633 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c00eed0 con 0x7f826c152850 2023-11-15T03:07:07.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.272+0000 7f82617fa700 1 -- 172.21.15.121:0/1440911633 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c00c850 con 0x7f826c152850 2023-11-15T03:07:07.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.272+0000 7f82727b9700 1 -- 172.21.15.121:0/1440911633 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826c0c67f0 con 0x7f826c152850 2023-11-15T03:07:07.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.272+0000 7f82617fa700 1 -- 172.21.15.121:0/1440911633 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c00cbd0 con 0x7f826c152850 2023-11-15T03:07:07.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.272+0000 7f82727b9700 1 -- 172.21.15.121:0/1440911633 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826c16c0f0 con 0x7f826c152850 2023-11-15T03:07:07.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.272+0000 7f82727b9700 1 -- 172.21.15.121:0/1440911633 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826c16bc70 con 0x7f826c152850 2023-11-15T03:07:07.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.272+0000 7f82617fa700 1 -- 172.21.15.121:0/1440911633 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f825c01b400 con 0x7f826c152850 2023-11-15T03:07:07.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.276+0000 7f82617fa700 1 --2- 172.21.15.121:0/1440911633 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f825005a510 0x7f825005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:07.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.276+0000 7f82617fa700 1 -- 172.21.15.121:0/1440911633 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 7986+0+0 (secure 0 0 0) 0x7f825c075430 con 0x7f826c152850 2023-11-15T03:07:07.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.276+0000 7f82717b7700 1 --2- 172.21.15.121:0/1440911633 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f825005a510 0x7f825005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:07.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.276+0000 7f82717b7700 1 --2- 172.21.15.121:0/1440911633 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f825005a510 0x7f825005c9d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f8268001690 tx=0x7f8268009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:07.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.280+0000 7f82617fa700 1 -- 172.21.15.121:0/1440911633 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f826c16bc70 con 0x7f826c152850 2023-11-15T03:07:07.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.484+0000 7f82727b9700 1 -- 172.21.15.121:0/1440911633 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f826c16bc70 con 0x7f826c152850 2023-11-15T03:07:07.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.484+0000 7f82617fa700 1 -- 172.21.15.121:0/1440911633 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 206 v206) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f826c16bc70 con 0x7f826c152850 2023-11-15T03:07:07.492 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:07.492 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":206,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1007300937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1007300937},{"type":"v1","addr":"172.21.15.121:6836","nonce":1007300937}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":206}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:02.271022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:07.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.488+0000 7f82727b9700 1 -- 172.21.15.121:0/1440911633 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f825005a510 msgr2=0x7f825005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:07.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.488+0000 7f82727b9700 1 --2- 172.21.15.121:0/1440911633 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f825005a510 0x7f825005c9d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f8268001690 tx=0x7f8268009040).stop 2023-11-15T03:07:07.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.488+0000 7f82727b9700 1 -- 172.21.15.121:0/1440911633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f826c152850 msgr2=0x7f826c14cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:07.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.488+0000 7f82727b9700 1 --2- 172.21.15.121:0/1440911633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f826c152850 0x7f826c14cf60 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 rx=0x7f825c00e930 tx=0x7f825c009060).stop 2023-11-15T03:07:07.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.488+0000 7f82727b9700 1 -- 172.21.15.121:0/1440911633 shutdown_connections 2023-11-15T03:07:07.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.488+0000 7f82727b9700 1 --2- 172.21.15.121:0/1440911633 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f825005a510 0x7f825005c9d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.488+0000 7f82727b9700 1 --2- 172.21.15.121:0/1440911633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f826c15be30 0x7f826c14ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.488+0000 7f82727b9700 1 --2- 172.21.15.121:0/1440911633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f826c152850 0x7f826c14cf60 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.488+0000 7f82727b9700 1 --2- 172.21.15.121:0/1440911633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f826c0c3280 0x7f826c1690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.488+0000 7f82727b9700 1 -- 172.21.15.121:0/1440911633 >> 172.21.15.121:0/1440911633 conn(0x7f826c062190 msgr2=0x7f826c14e340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:07.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.488+0000 7f82727b9700 1 -- 172.21.15.121:0/1440911633 shutdown_connections 2023-11-15T03:07:07.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.488+0000 7f82727b9700 1 -- 172.21.15.121:0/1440911633 wait complete. 2023-11-15T03:07:07.498 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 206 2023-11-15T03:07:07.517 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:07:07.519 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:07.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.676+0000 7f8450c1b700 1 Processor -- start 2023-11-15T03:07:07.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.676+0000 7f8450c1b700 1 -- start start 2023-11-15T03:07:07.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.676+0000 7f8450c1b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f844c0c28d0 0x7f844c159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:07.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.676+0000 7f8450c1b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f844c0c3fa0 con 0x7f844c0c6ce0 2023-11-15T03:07:07.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.676+0000 7f8450c1b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f844c159630 con 0x7f844c152780 2023-11-15T03:07:07.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.676+0000 7f8450c1b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f844c0c30b0 con 0x7f844c0c28d0 2023-11-15T03:07:07.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.676+0000 7f8441d9b700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f844c0c6ce0 0x7f844c0a5510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:54150/0 (socket says 172.21.15.121:54150) 2023-11-15T03:07:07.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.676+0000 7f8441d9b700 1 -- 172.21.15.121:0/1666694715 learned_addr learned my addr 172.21.15.121:0/1666694715 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:07.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.676+0000 7f844ad9d700 1 --2- 172.21.15.121:0/1666694715 >> v2:172.21.15.121:3300/0 conn(0x7f844c0c28d0 0x7f844c159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:07.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.680+0000 7f8449d9b700 1 -- 172.21.15.121:0/1666694715 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670669913 0 0) 0x7f844c0c3fa0 con 0x7f844c0c6ce0 2023-11-15T03:07:07.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.680+0000 7f8449d9b700 1 -- 172.21.15.121:0/1666694715 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84380033f0 con 0x7f844c0c6ce0 2023-11-15T03:07:07.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.680+0000 7f8449d9b700 1 -- 172.21.15.121:0/1666694715 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055819915 0 0) 0x7f844c159630 con 0x7f844c152780 2023-11-15T03:07:07.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.680+0000 7f844ad9d700 1 -- 172.21.15.121:0/1666694715 >> v1:172.21.15.190:6789/0 conn(0x7f844c152780 legacy=0x7f844c0c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:07.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.680+0000 7f844ad9d700 1 -- 172.21.15.121:0/1666694715 >> v1:172.21.15.121:6789/0 conn(0x7f844c0c6ce0 legacy=0x7f844c0a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:07.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.680+0000 7f844ad9d700 1 -- 172.21.15.121:0/1666694715 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844c0c3390 con 0x7f844c0c28d0 2023-11-15T03:07:07.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.680+0000 7f844ad9d700 1 --2- 172.21.15.121:0/1666694715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f844c0c28d0 0x7f844c159140 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 rx=0x7f8444009e60 tx=0x7f8444009780).ready entity=mon.0 client_cookie=5005cd0399976631 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:07.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.680+0000 7f8449d9b700 1 -- 172.21.15.121:0/1666694715 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844400e070 con 0x7f844c0c28d0 2023-11-15T03:07:07.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.680+0000 7f8449d9b700 1 -- 172.21.15.121:0/1666694715 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844400bb60 con 0x7f844c0c28d0 2023-11-15T03:07:07.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.680+0000 7f8449d9b700 1 -- 172.21.15.121:0/1666694715 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844400bee0 con 0x7f844c0c28d0 2023-11-15T03:07:07.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.680+0000 7f8450c1b700 1 -- 172.21.15.121:0/1666694715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f844c0c28d0 msgr2=0x7f844c159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:07.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.680+0000 7f8450c1b700 1 --2- 172.21.15.121:0/1666694715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f844c0c28d0 0x7f844c159140 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 rx=0x7f8444009e60 tx=0x7f8444009780).stop 2023-11-15T03:07:07.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.680+0000 7f8450c1b700 1 -- 172.21.15.121:0/1666694715 shutdown_connections 2023-11-15T03:07:07.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.680+0000 7f8450c1b700 1 --2- 172.21.15.121:0/1666694715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f844c0c28d0 0x7f844c159140 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.680+0000 7f8450c1b700 1 -- 172.21.15.121:0/1666694715 >> 172.21.15.121:0/1666694715 conn(0x7f844c062190 msgr2=0x7f844c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:07.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.680+0000 7f8450c1b700 1 -- 172.21.15.121:0/1666694715 shutdown_connections 2023-11-15T03:07:07.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.680+0000 7f8450c1b700 1 -- 172.21.15.121:0/1666694715 wait complete. 2023-11-15T03:07:07.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.684+0000 7f8450c1b700 1 Processor -- start 2023-11-15T03:07:07.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.684+0000 7f8450c1b700 1 -- start start 2023-11-15T03:07:07.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.684+0000 7f8450c1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f844c0c28d0 0x7f844c152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:07.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.684+0000 7f8450c1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f844c0c6ce0 0x7f844c15b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:07.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.684+0000 7f8450c1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f844c152780 0x7f844c143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:07.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.684+0000 7f8450c1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844c15d240 con 0x7f844c0c28d0 2023-11-15T03:07:07.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.684+0000 7f8450c1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844c15cdb0 con 0x7f844c0c6ce0 2023-11-15T03:07:07.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.684+0000 7f8450c1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844c15c1a0 con 0x7f844c152780 2023-11-15T03:07:07.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.684+0000 7f844a59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f844c0c28d0 0x7f844c152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:07.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.684+0000 7f844a59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f844c0c28d0 0x7f844c152d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53308/0 (socket says 172.21.15.121:53308) 2023-11-15T03:07:07.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.684+0000 7f844a59c700 1 -- 172.21.15.121:0/1180814444 learned_addr learned my addr 172.21.15.121:0/1180814444 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:07.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.684+0000 7f844ad9d700 1 --2- 172.21.15.121:0/1180814444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f844c152780 0x7f844c143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:07.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.684+0000 7f8441d9b700 1 --2- 172.21.15.121:0/1180814444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f844c0c6ce0 0x7f844c15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:07.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.684+0000 7f844a59c700 1 -- 172.21.15.121:0/1180814444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f844c152780 msgr2=0x7f844c143c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:07.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.684+0000 7f844a59c700 1 --2- 172.21.15.121:0/1180814444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f844c152780 0x7f844c143c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.684+0000 7f844a59c700 1 -- 172.21.15.121:0/1180814444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f844c0c6ce0 msgr2=0x7f844c15b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:07.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.684+0000 7f844a59c700 1 --2- 172.21.15.121:0/1180814444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f844c0c6ce0 0x7f844c15b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.684+0000 7f844a59c700 1 -- 172.21.15.121:0/1180814444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8444009a20 con 0x7f844c0c28d0 2023-11-15T03:07:07.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.684+0000 7f844a59c700 1 --2- 172.21.15.121:0/1180814444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f844c0c28d0 0x7f844c152d90 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7f843c00c410 tx=0x7f843c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:07.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.688+0000 7f8442ffd700 1 -- 172.21.15.121:0/1180814444 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c007020 con 0x7f844c0c28d0 2023-11-15T03:07:07.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.688+0000 7f8442ffd700 1 -- 172.21.15.121:0/1180814444 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843c0071a0 con 0x7f844c0c28d0 2023-11-15T03:07:07.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.688+0000 7f8450c1b700 1 -- 172.21.15.121:0/1180814444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f844c15c480 con 0x7f844c0c28d0 2023-11-15T03:07:07.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.688+0000 7f8442ffd700 1 -- 172.21.15.121:0/1180814444 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c007560 con 0x7f844c0c28d0 2023-11-15T03:07:07.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.688+0000 7f8450c1b700 1 -- 172.21.15.121:0/1180814444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f844c15ca40 con 0x7f844c0c28d0 2023-11-15T03:07:07.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.688+0000 7f8450c1b700 1 -- 172.21.15.121:0/1180814444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f844c06c150 con 0x7f844c0c28d0 2023-11-15T03:07:07.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.688+0000 7f8442ffd700 1 -- 172.21.15.121:0/1180814444 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f843c0068b0 con 0x7f844c0c28d0 2023-11-15T03:07:07.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.692+0000 7f8442ffd700 1 --2- 172.21.15.121:0/1180814444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f842c05a540 0x7f842c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:07.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.692+0000 7f8442ffd700 1 -- 172.21.15.121:0/1180814444 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 7986+0+0 (secure 0 0 0) 0x7f843c0743e0 con 0x7f844c0c28d0 2023-11-15T03:07:07.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.692+0000 7f8441d9b700 1 --2- 172.21.15.121:0/1180814444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f842c05a540 0x7f842c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:07.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.692+0000 7f8441d9b700 1 --2- 172.21.15.121:0/1180814444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f842c05a540 0x7f842c05ca00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f844c0a8cf0 tx=0x7f8434009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:07.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.696+0000 7f8442ffd700 1 -- 172.21.15.121:0/1180814444 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f843c0473b0 con 0x7f844c0c28d0 2023-11-15T03:07:07.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.892+0000 7f8450c1b700 1 -- 172.21.15.121:0/1180814444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f844c0c3670 con 0x7f844c0c28d0 2023-11-15T03:07:07.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.892+0000 7f8442ffd700 1 -- 172.21.15.121:0/1180814444 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 206 v206) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f843c04a9d0 con 0x7f844c0c28d0 2023-11-15T03:07:07.900 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:07.900 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":206,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1007300937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1007300937},{"type":"v1","addr":"172.21.15.121:6836","nonce":1007300937}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":206}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:02.271022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:07.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.896+0000 7f8450c1b700 1 -- 172.21.15.121:0/1180814444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f842c05a540 msgr2=0x7f842c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:07.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.896+0000 7f8450c1b700 1 --2- 172.21.15.121:0/1180814444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f842c05a540 0x7f842c05ca00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f844c0a8cf0 tx=0x7f8434009040).stop 2023-11-15T03:07:07.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.896+0000 7f8450c1b700 1 -- 172.21.15.121:0/1180814444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f844c0c28d0 msgr2=0x7f844c152d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:07.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.896+0000 7f8450c1b700 1 --2- 172.21.15.121:0/1180814444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f844c0c28d0 0x7f844c152d90 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7f843c00c410 tx=0x7f843c007e00).stop 2023-11-15T03:07:07.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.896+0000 7f8450c1b700 1 -- 172.21.15.121:0/1180814444 shutdown_connections 2023-11-15T03:07:07.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.896+0000 7f8450c1b700 1 --2- 172.21.15.121:0/1180814444 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f842c05a540 0x7f842c05ca00 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.896+0000 7f8450c1b700 1 --2- 172.21.15.121:0/1180814444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f844c152780 0x7f844c143c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.896+0000 7f8450c1b700 1 --2- 172.21.15.121:0/1180814444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f844c0c6ce0 0x7f844c15b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.896+0000 7f8450c1b700 1 --2- 172.21.15.121:0/1180814444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f844c0c28d0 0x7f844c152d90 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:07.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.896+0000 7f8450c1b700 1 -- 172.21.15.121:0/1180814444 >> 172.21.15.121:0/1180814444 conn(0x7f844c062190 msgr2=0x7f844c0c6500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:07.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.896+0000 7f8450c1b700 1 -- 172.21.15.121:0/1180814444 shutdown_connections 2023-11-15T03:07:07.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:07.896+0000 7f8450c1b700 1 -- 172.21.15.121:0/1180814444 wait complete. 2023-11-15T03:07:07.905 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 206 2023-11-15T03:07:07.925 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt mds_snaptable' led to DAMAGED state 2023-11-15T03:07:07.925 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt mds0_inotable' 2023-11-15T03:07:07.925 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:07:07.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:07:07.926 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:07:07.944 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:07:07.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:07:07.946 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:07:08.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.148+0000 7f2274d07700 1 Processor -- start 2023-11-15T03:07:08.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.148+0000 7f2274d07700 1 -- start start 2023-11-15T03:07:08.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.148+0000 7f2274d07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f22700bb510 0x7f22700abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:08.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.148+0000 7f2274d07700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22700c2c00 con 0x7f22700abaa0 2023-11-15T03:07:08.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.148+0000 7f2274d07700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22700657b0 con 0x7f22700a8880 2023-11-15T03:07:08.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.148+0000 7f2274d07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2270065a90 con 0x7f22700bb510 2023-11-15T03:07:08.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.148+0000 7f226f7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f22700abaa0 0x7f2270063570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:54152/0 (socket says 172.21.15.121:54152) 2023-11-15T03:07:08.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.148+0000 7f226f7fe700 1 -- 172.21.15.121:0/3791102241 learned_addr learned my addr 172.21.15.121:0/3791102241 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:08.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.148+0000 7f226effd700 1 --2- 172.21.15.121:0/3791102241 >> v2:172.21.15.190:3300/0 conn(0x7f22700bb510 0x7f22700abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:08.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.148+0000 7f226e7fc700 1 -- 172.21.15.121:0/3791102241 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213102724 0 0) 0x7f22700c2c00 con 0x7f22700abaa0 2023-11-15T03:07:08.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.148+0000 7f226e7fc700 1 -- 172.21.15.121:0/3791102241 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22580033f0 con 0x7f22700abaa0 2023-11-15T03:07:08.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.148+0000 7f226effd700 1 -- 172.21.15.121:0/3791102241 >> v1:172.21.15.190:6789/0 conn(0x7f22700a8880 legacy=0x7f2270151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:08.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.148+0000 7f226effd700 1 -- 172.21.15.121:0/3791102241 >> v1:172.21.15.121:6789/0 conn(0x7f22700abaa0 legacy=0x7f2270063570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:08.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.152+0000 7f226effd700 1 -- 172.21.15.121:0/3791102241 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22700c2890 con 0x7f22700bb510 2023-11-15T03:07:08.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.152+0000 7f226effd700 1 --2- 172.21.15.121:0/3791102241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22700bb510 0x7f22700abe80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f2264006fd0 tx=0x7f2264006f40).ready entity=mon.1 client_cookie=37ef7e99b9042331 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:08.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.152+0000 7f226e7fc700 1 -- 172.21.15.121:0/3791102241 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264005880 con 0x7f22700bb510 2023-11-15T03:07:08.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.152+0000 7f226e7fc700 1 -- 172.21.15.121:0/3791102241 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22640059e0 con 0x7f22700bb510 2023-11-15T03:07:08.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.152+0000 7f226e7fc700 1 -- 172.21.15.121:0/3791102241 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264005d60 con 0x7f22700bb510 2023-11-15T03:07:08.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.152+0000 7f2274d07700 1 -- 172.21.15.121:0/3791102241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22700bb510 msgr2=0x7f22700abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:08.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.152+0000 7f2274d07700 1 --2- 172.21.15.121:0/3791102241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22700bb510 0x7f22700abe80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f2264006fd0 tx=0x7f2264006f40).stop 2023-11-15T03:07:08.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.152+0000 7f2274d07700 1 -- 172.21.15.121:0/3791102241 shutdown_connections 2023-11-15T03:07:08.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.152+0000 7f2274d07700 1 --2- 172.21.15.121:0/3791102241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22700bb510 0x7f22700abe80 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:08.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.152+0000 7f2274d07700 1 -- 172.21.15.121:0/3791102241 >> 172.21.15.121:0/3791102241 conn(0x7f2270067a60 msgr2=0x7f2270067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:08.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.152+0000 7f2274d07700 1 -- 172.21.15.121:0/3791102241 shutdown_connections 2023-11-15T03:07:08.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.152+0000 7f2274d07700 1 -- 172.21.15.121:0/3791102241 wait complete. 2023-11-15T03:07:08.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f2274d07700 1 Processor -- start 2023-11-15T03:07:08.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f2274d07700 1 -- start start 2023-11-15T03:07:08.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f2274d07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22700a8880 0x7f227014d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:08.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f2274d07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22700abaa0 0x7f22700c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:08.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f2274d07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22700bb510 0x7f22700c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:08.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f2274d07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22700c3790 con 0x7f22700abaa0 2023-11-15T03:07:08.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f2274d07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2270155550 con 0x7f22700a8880 2023-11-15T03:07:08.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f2274d07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22700c4da0 con 0x7f22700bb510 2023-11-15T03:07:08.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f226effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22700abaa0 0x7f22700c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:08.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f226f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22700a8880 0x7f227014d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:08.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f226effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22700abaa0 0x7f22700c2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53312/0 (socket says 172.21.15.121:53312) 2023-11-15T03:07:08.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f226effd700 1 -- 172.21.15.121:0/2711619481 learned_addr learned my addr 172.21.15.121:0/2711619481 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:08.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f226ffff700 1 --2- 172.21.15.121:0/2711619481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22700bb510 0x7f22700c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:08.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f226f7fe700 1 -- 172.21.15.121:0/2711619481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22700bb510 msgr2=0x7f22700c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:08.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f226f7fe700 1 --2- 172.21.15.121:0/2711619481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22700bb510 0x7f22700c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:08.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f226f7fe700 1 -- 172.21.15.121:0/2711619481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22700abaa0 msgr2=0x7f22700c2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:08.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f226f7fe700 1 --2- 172.21.15.121:0/2711619481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22700abaa0 0x7f22700c2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:08.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f226f7fe700 1 -- 172.21.15.121:0/2711619481 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2264009250 con 0x7f22700a8880 2023-11-15T03:07:08.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f226ffff700 1 --2- 172.21.15.121:0/2711619481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22700bb510 0x7f22700c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:07:08.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.156+0000 7f226effd700 1 --2- 172.21.15.121:0/2711619481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22700abaa0 0x7f22700c2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:07:08.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.160+0000 7f226f7fe700 1 --2- 172.21.15.121:0/2711619481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22700a8880 0x7f227014d9b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f226000c410 tx=0x7f2260007bd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:08.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.160+0000 7f22577fe700 1 -- 172.21.15.121:0/2711619481 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226000ce30 con 0x7f22700a8880 2023-11-15T03:07:08.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.160+0000 7f22577fe700 1 -- 172.21.15.121:0/2711619481 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22600072e0 con 0x7f22700a8880 2023-11-15T03:07:08.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.160+0000 7f2274d07700 1 -- 172.21.15.121:0/2711619481 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22700ab070 con 0x7f22700a8880 2023-11-15T03:07:08.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.160+0000 7f22577fe700 1 -- 172.21.15.121:0/2711619481 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22600057a0 con 0x7f22700a8880 2023-11-15T03:07:08.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.160+0000 7f2274d07700 1 -- 172.21.15.121:0/2711619481 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22700c4fd0 con 0x7f22700a8880 2023-11-15T03:07:08.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.160+0000 7f2274d07700 1 -- 172.21.15.121:0/2711619481 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22700c5730 con 0x7f22700a8880 2023-11-15T03:07:08.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.164+0000 7f22577fe700 1 -- 172.21.15.121:0/2711619481 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2260005b90 con 0x7f22700a8880 2023-11-15T03:07:08.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.164+0000 7f22577fe700 1 --2- 172.21.15.121:0/2711619481 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f225c05a510 0x7f225c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:08.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.164+0000 7f22577fe700 1 -- 172.21.15.121:0/2711619481 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 7986+0+0 (secure 0 0 0) 0x7f22600746b0 con 0x7f22700a8880 2023-11-15T03:07:08.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.164+0000 7f226effd700 1 --2- 172.21.15.121:0/2711619481 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f225c05a510 0x7f225c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:08.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.164+0000 7f226effd700 1 --2- 172.21.15.121:0/2711619481 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f225c05a510 0x7f225c05c9d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f2270155500 tx=0x7f2264016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:08.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.172+0000 7f22577fe700 1 -- 172.21.15.121:0/2711619481 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2260047680 con 0x7f22700a8880 2023-11-15T03:07:08.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:08.376+0000 7f2274d07700 1 -- 172.21.15.121:0/2711619481 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f22700c6c30 con 0x7f22700a8880 2023-11-15T03:07:09.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.224+0000 7f22577fe700 1 -- 172.21.15.121:0/2711619481 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v207) v1 ==== 161+0+0 (secure 0 0 0) 0x7f226004aca0 con 0x7f22700a8880 2023-11-15T03:07:09.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.224+0000 7f2274d07700 1 -- 172.21.15.121:0/2711619481 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f225c05a510 msgr2=0x7f225c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:09.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.224+0000 7f2274d07700 1 --2- 172.21.15.121:0/2711619481 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f225c05a510 0x7f225c05c9d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f2270155500 tx=0x7f2264016000).stop 2023-11-15T03:07:09.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.224+0000 7f2274d07700 1 -- 172.21.15.121:0/2711619481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22700a8880 msgr2=0x7f227014d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:09.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.224+0000 7f2274d07700 1 --2- 172.21.15.121:0/2711619481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22700a8880 0x7f227014d9b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f226000c410 tx=0x7f2260007bd0).stop 2023-11-15T03:07:09.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.224+0000 7f2274d07700 1 -- 172.21.15.121:0/2711619481 shutdown_connections 2023-11-15T03:07:09.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.224+0000 7f2274d07700 1 --2- 172.21.15.121:0/2711619481 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f225c05a510 0x7f225c05c9d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:09.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.224+0000 7f2274d07700 1 --2- 172.21.15.121:0/2711619481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22700bb510 0x7f22700c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:09.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.224+0000 7f2274d07700 1 --2- 172.21.15.121:0/2711619481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22700abaa0 0x7f22700c2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:09.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.224+0000 7f2274d07700 1 --2- 172.21.15.121:0/2711619481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22700a8880 0x7f227014d9b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:09.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.224+0000 7f2274d07700 1 -- 172.21.15.121:0/2711619481 >> 172.21.15.121:0/2711619481 conn(0x7f2270067a60 msgr2=0x7f22700bfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:09.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.224+0000 7f2274d07700 1 -- 172.21.15.121:0/2711619481 shutdown_connections 2023-11-15T03:07:09.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.224+0000 7f2274d07700 1 -- 172.21.15.121:0/2711619481 wait complete. 2023-11-15T03:07:09.237 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:07:09.255 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:07:09.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.412+0000 7f77fdfa9700 1 Processor -- start 2023-11-15T03:07:09.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.412+0000 7f77fdfa9700 1 -- start start 2023-11-15T03:07:09.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.412+0000 7f77fdfa9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77f8151920 0x7f77f80bb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:09.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.412+0000 7f77fdfa9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f77f80ac150 0x7f77f8062190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:09.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.412+0000 7f77fdfa9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f8155de0 con 0x7f77f8067150 2023-11-15T03:07:09.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.412+0000 7f77fdfa9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f80c4060 con 0x7f77f80ac150 2023-11-15T03:07:09.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.412+0000 7f77fdfa9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f80c3aa0 con 0x7f77f8151920 2023-11-15T03:07:09.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.412+0000 7f77f7fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f77f80ac150 0x7f77f8062190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:09.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77f7fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f77f80ac150 0x7f77f8062190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:41552/0 (socket says 172.21.15.121:41552) 2023-11-15T03:07:09.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77f7fff700 1 -- 172.21.15.121:0/1501002088 learned_addr learned my addr 172.21.15.121:0/1501002088 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:09.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77effff700 1 --1- 172.21.15.121:0/1501002088 >> v1:172.21.15.190:6789/0 conn(0x7f77f8067150 0x7f77f80aba40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:37780/0 (socket says 172.21.15.121:37780) 2023-11-15T03:07:09.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77f77fe700 1 --2- 172.21.15.121:0/1501002088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77f8151920 0x7f77f80bb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:09.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77f7fff700 1 -- 172.21.15.121:0/1501002088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77f8151920 msgr2=0x7f77f80bb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:09.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77f7fff700 1 --2- 172.21.15.121:0/1501002088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77f8151920 0x7f77f80bb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:09.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77f7fff700 1 -- 172.21.15.121:0/1501002088 >> v1:172.21.15.190:6789/0 conn(0x7f77f8067150 legacy=0x7f77f80aba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:09.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77f6ffd700 1 -- 172.21.15.121:0/1501002088 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894411109 0 0) 0x7f77f8155de0 con 0x7f77f8067150 2023-11-15T03:07:09.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77f7fff700 1 -- 172.21.15.121:0/1501002088 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f80c3d80 con 0x7f77f80ac150 2023-11-15T03:07:09.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77f7fff700 1 --2- 172.21.15.121:0/1501002088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f80ac150 0x7f77f8062190 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f77f0009f20 tx=0x7f77f0009840).ready entity=mon.1 client_cookie=c7eec12277dbd90b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:09.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77f6ffd700 1 -- 172.21.15.121:0/1501002088 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f000e070 con 0x7f77f80ac150 2023-11-15T03:07:09.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77f6ffd700 1 -- 172.21.15.121:0/1501002088 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f000b9d0 con 0x7f77f80ac150 2023-11-15T03:07:09.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77f6ffd700 1 -- 172.21.15.121:0/1501002088 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f000bd50 con 0x7f77f80ac150 2023-11-15T03:07:09.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77fdfa9700 1 -- 172.21.15.121:0/1501002088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f80ac150 msgr2=0x7f77f8062190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:09.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77fdfa9700 1 --2- 172.21.15.121:0/1501002088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f80ac150 0x7f77f8062190 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f77f0009f20 tx=0x7f77f0009840).stop 2023-11-15T03:07:09.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77fdfa9700 1 -- 172.21.15.121:0/1501002088 shutdown_connections 2023-11-15T03:07:09.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77fdfa9700 1 --2- 172.21.15.121:0/1501002088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f80ac150 0x7f77f8062190 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:09.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77fdfa9700 1 --2- 172.21.15.121:0/1501002088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77f8151920 0x7f77f80bb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:09.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77fdfa9700 1 -- 172.21.15.121:0/1501002088 >> 172.21.15.121:0/1501002088 conn(0x7f77f8067a60 msgr2=0x7f77f8067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:09.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77fdfa9700 1 -- 172.21.15.121:0/1501002088 shutdown_connections 2023-11-15T03:07:09.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.416+0000 7f77fdfa9700 1 -- 172.21.15.121:0/1501002088 wait complete. 2023-11-15T03:07:09.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.420+0000 7f77fdfa9700 1 Processor -- start 2023-11-15T03:07:09.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.420+0000 7f77fdfa9700 1 -- start start 2023-11-15T03:07:09.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.420+0000 7f77fdfa9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77f8067150 0x7f77f81496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:09.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.420+0000 7f77fdfa9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f80ac150 0x7f77f80a7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:09.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.420+0000 7f77fdfa9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77f8151920 0x7f77f80c1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:09.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.420+0000 7f77fdfa9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f814ab20 con 0x7f77f8067150 2023-11-15T03:07:09.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.420+0000 7f77fdfa9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f8149f50 con 0x7f77f80ac150 2023-11-15T03:07:09.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.420+0000 7f77fdfa9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f8149b90 con 0x7f77f8151920 2023-11-15T03:07:09.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.424+0000 7f77f77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77f8067150 0x7f77f81496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:09.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.424+0000 7f77f77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77f8067150 0x7f77f81496a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53320/0 (socket says 172.21.15.121:53320) 2023-11-15T03:07:09.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.424+0000 7f77f77fe700 1 -- 172.21.15.121:0/111329098 learned_addr learned my addr 172.21.15.121:0/111329098 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:09.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.424+0000 7f77f7fff700 1 --2- 172.21.15.121:0/111329098 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77f8151920 0x7f77f80c1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:09.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.424+0000 7f77effff700 1 --2- 172.21.15.121:0/111329098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f80ac150 0x7f77f80a7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:09.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.424+0000 7f77f77fe700 1 -- 172.21.15.121:0/111329098 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77f8151920 msgr2=0x7f77f80c1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:09.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.424+0000 7f77f77fe700 1 --2- 172.21.15.121:0/111329098 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77f8151920 0x7f77f80c1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:09.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.424+0000 7f77f77fe700 1 -- 172.21.15.121:0/111329098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f80ac150 msgr2=0x7f77f80a7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:09.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.424+0000 7f77f77fe700 1 --2- 172.21.15.121:0/111329098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f80ac150 0x7f77f80a7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:09.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.424+0000 7f77f77fe700 1 -- 172.21.15.121:0/111329098 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f0009ae0 con 0x7f77f8067150 2023-11-15T03:07:09.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.424+0000 7f77f77fe700 1 --2- 172.21.15.121:0/111329098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77f8067150 0x7f77f81496a0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 rx=0x7f77e800e900 tx=0x7f77e8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:09.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.424+0000 7f77ef7fe700 1 -- 172.21.15.121:0/111329098 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8012600 con 0x7f77f8067150 2023-11-15T03:07:09.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.424+0000 7f77ef7fe700 1 -- 172.21.15.121:0/111329098 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e8012760 con 0x7f77f8067150 2023-11-15T03:07:09.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.424+0000 7f77fdfa9700 1 -- 172.21.15.121:0/111329098 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f814a1f0 con 0x7f77f8067150 2023-11-15T03:07:09.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.424+0000 7f77ef7fe700 1 -- 172.21.15.121:0/111329098 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8012ae0 con 0x7f77f8067150 2023-11-15T03:07:09.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.424+0000 7f77fdfa9700 1 -- 172.21.15.121:0/111329098 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f814a7b0 con 0x7f77f8067150 2023-11-15T03:07:09.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.424+0000 7f77fdfa9700 1 -- 172.21.15.121:0/111329098 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f8047060 con 0x7f77f8067150 2023-11-15T03:07:09.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.428+0000 7f77ef7fe700 1 -- 172.21.15.121:0/111329098 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f77e801b400 con 0x7f77f8067150 2023-11-15T03:07:09.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.428+0000 7f77ef7fe700 1 --2- 172.21.15.121:0/111329098 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f77d805a510 0x7f77d805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:09.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.428+0000 7f77ef7fe700 1 -- 172.21.15.121:0/111329098 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 7986+0+0 (secure 0 0 0) 0x7f77e80751d0 con 0x7f77f8067150 2023-11-15T03:07:09.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.428+0000 7f77effff700 1 --2- 172.21.15.121:0/111329098 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f77d805a510 0x7f77d805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:09.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.432+0000 7f77effff700 1 --2- 172.21.15.121:0/111329098 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f77d805a510 0x7f77d805c9d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f77f814bc20 tx=0x7f77e0007bd0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:09.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.436+0000 7f77ef7fe700 1 -- 172.21.15.121:0/111329098 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f77e8048200 con 0x7f77f8067150 2023-11-15T03:07:09.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:09.652+0000 7f77fdfa9700 1 -- 172.21.15.121:0/111329098 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f77f80c35c0 con 0x7f77f8067150 2023-11-15T03:07:10.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.240+0000 7f77ef7fe700 1 -- 172.21.15.121:0/111329098 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v208) v1 ==== 101+0+0 (secure 0 0 0) 0x7f77e804b820 con 0x7f77f8067150 2023-11-15T03:07:10.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.244+0000 7f77fdfa9700 1 -- 172.21.15.121:0/111329098 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f77d805a510 msgr2=0x7f77d805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.244+0000 7f77fdfa9700 1 --2- 172.21.15.121:0/111329098 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f77d805a510 0x7f77d805c9d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f77f814bc20 tx=0x7f77e0007bd0).stop 2023-11-15T03:07:10.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.244+0000 7f77fdfa9700 1 -- 172.21.15.121:0/111329098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77f8067150 msgr2=0x7f77f81496a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.244+0000 7f77fdfa9700 1 --2- 172.21.15.121:0/111329098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77f8067150 0x7f77f81496a0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 rx=0x7f77e800e900 tx=0x7f77e8009060).stop 2023-11-15T03:07:10.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.244+0000 7f77fdfa9700 1 -- 172.21.15.121:0/111329098 shutdown_connections 2023-11-15T03:07:10.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.244+0000 7f77fdfa9700 1 --2- 172.21.15.121:0/111329098 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f77d805a510 0x7f77d805c9d0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.244+0000 7f77fdfa9700 1 --2- 172.21.15.121:0/111329098 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77f8151920 0x7f77f80c1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.244+0000 7f77fdfa9700 1 --2- 172.21.15.121:0/111329098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f80ac150 0x7f77f80a7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.244+0000 7f77fdfa9700 1 --2- 172.21.15.121:0/111329098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77f8067150 0x7f77f81496a0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.244+0000 7f77fdfa9700 1 -- 172.21.15.121:0/111329098 >> 172.21.15.121:0/111329098 conn(0x7f77f8067a60 msgr2=0x7f77f80c5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:10.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.244+0000 7f77fdfa9700 1 -- 172.21.15.121:0/111329098 shutdown_connections 2023-11-15T03:07:10.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.248+0000 7f77fdfa9700 1 -- 172.21.15.121:0/111329098 wait complete. 2023-11-15T03:07:10.256 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:07:10.277 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:07:10.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.332+0000 7f3c7093ad00 1 Processor -- start 2023-11-15T03:07:10.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.332+0000 7f3c7093ad00 1 -- start start 2023-11-15T03:07:10.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.332+0000 7f3c7093ad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d62263bbb0 0x55d62270d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.332+0000 7f3c7093ad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d622685b00 0x55d62270d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.332+0000 7f3c7093ad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d622718650 0x55d622683a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.332+0000 7f3c7093ad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6225e9cc0 con 0x55d622718650 2023-11-15T03:07:10.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.332+0000 7f3c7093ad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6225b4220 con 0x55d62263bbb0 2023-11-15T03:07:10.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.332+0000 7f3c7093ad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d62270ded0 con 0x55d622685b00 2023-11-15T03:07:10.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.332+0000 7f3c656b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d622718650 0x55d622683a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.332+0000 7f3c656b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d622718650 0x55d622683a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53332/0 (socket says 172.21.15.121:53332) 2023-11-15T03:07:10.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.332+0000 7f3c656b8700 1 -- 172.21.15.121:0/2567469896 learned_addr learned my addr 172.21.15.121:0/2567469896 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:10.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.332+0000 7f3c64eb7700 1 --2- 172.21.15.121:0/2567469896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d62263bbb0 0x55d62270d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.332+0000 7f3c5c6b6700 1 --2- 172.21.15.121:0/2567469896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d622685b00 0x55d62270d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.332+0000 7f3c656b8700 1 -- 172.21.15.121:0/2567469896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d622685b00 msgr2=0x55d62270d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:10.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c656b8700 1 --2- 172.21.15.121:0/2567469896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d622685b00 0x55d62270d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c656b8700 1 -- 172.21.15.121:0/2567469896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d62263bbb0 msgr2=0x55d62270d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:10.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c656b8700 1 --2- 172.21.15.121:0/2567469896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d62263bbb0 0x55d62270d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c656b8700 1 -- 172.21.15.121:0/2567469896 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4c005cf0 con 0x55d622718650 2023-11-15T03:07:10.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c5c6b6700 1 --2- 172.21.15.121:0/2567469896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d622685b00 0x55d62270d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:07:10.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c656b8700 1 --2- 172.21.15.121:0/2567469896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d622718650 0x55d622683a10 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 rx=0x7f3c60006f40 tx=0x7f3c600069f0).ready entity=mon.0 client_cookie=6c113cbc8f11bb15 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:10.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c5f7fe700 1 -- 172.21.15.121:0/2567469896 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c60006110 con 0x55d622718650 2023-11-15T03:07:10.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c5f7fe700 1 -- 172.21.15.121:0/2567469896 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c60006270 con 0x55d622718650 2023-11-15T03:07:10.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c5f7fe700 1 -- 172.21.15.121:0/2567469896 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c60004750 con 0x55d622718650 2023-11-15T03:07:10.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c7093ad00 1 -- 172.21.15.121:0/2567469896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d622718650 msgr2=0x55d622683a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/2567469896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d622718650 0x55d622683a10 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 rx=0x7f3c60006f40 tx=0x7f3c600069f0).stop 2023-11-15T03:07:10.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c7093ad00 1 -- 172.21.15.121:0/2567469896 shutdown_connections 2023-11-15T03:07:10.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/2567469896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d622718650 0x55d622683a10 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/2567469896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d622685b00 0x55d62270d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/2567469896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d62263bbb0 0x55d62270d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c7093ad00 1 -- 172.21.15.121:0/2567469896 >> 172.21.15.121:0/2567469896 conn(0x55d62268b080 msgr2=0x55d62267ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:10.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c7093ad00 1 -- 172.21.15.121:0/2567469896 shutdown_connections 2023-11-15T03:07:10.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c7093ad00 1 -- 172.21.15.121:0/2567469896 wait complete. 2023-11-15T03:07:10.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c7093ad00 1 Processor -- start 2023-11-15T03:07:10.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.336+0000 7f3c7093ad00 1 -- start start 2023-11-15T03:07:10.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c7093ad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d62263bbb0 0x55d622709b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c7093ad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d622685b00 0x55d622709460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c7093ad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d622718650 0x55d62266b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c7093ad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d62266abe0 con 0x55d622718650 2023-11-15T03:07:10.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c7093ad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d622709950 con 0x55d62263bbb0 2023-11-15T03:07:10.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c7093ad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d622682750 con 0x55d622685b00 2023-11-15T03:07:10.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c656b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d622718650 0x55d62266b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c656b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d622718650 0x55d62266b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53342/0 (socket says 172.21.15.121:53342) 2023-11-15T03:07:10.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 learned_addr learned my addr 172.21.15.121:0/3134244921 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:10.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c5c6b6700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d622685b00 0x55d622709460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c64eb7700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d62263bbb0 0x55d622709b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d622685b00 msgr2=0x55d622709460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c656b8700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d622685b00 0x55d622709460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d62263bbb0 msgr2=0x55d622709b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c656b8700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d62263bbb0 0x55d622709b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c50001e10 con 0x55d622718650 2023-11-15T03:07:10.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c656b8700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d622718650 0x55d62266b050 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7f3c60000f80 tx=0x7f3c60014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:10.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c5dffb700 1 -- 172.21.15.121:0/3134244921 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c60006070 con 0x55d622718650 2023-11-15T03:07:10.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c5dffb700 1 -- 172.21.15.121:0/3134244921 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c60017070 con 0x55d622718650 2023-11-15T03:07:10.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d6225aca70 con 0x55d622718650 2023-11-15T03:07:10.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c5dffb700 1 -- 172.21.15.121:0/3134244921 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c60020460 con 0x55d622718650 2023-11-15T03:07:10.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.340+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d6225b8010 con 0x55d622718650 2023-11-15T03:07:10.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.344+0000 7f3c5dffb700 1 -- 172.21.15.121:0/3134244921 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3c60020810 con 0x55d622718650 2023-11-15T03:07:10.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.344+0000 7f3c5dffb700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3c44058ff0 0x7f3c4405b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.344+0000 7f3c5dffb700 1 -- 172.21.15.121:0/3134244921 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 7986+0+0 (secure 0 0 0) 0x7f3c60074c80 con 0x55d622718650 2023-11-15T03:07:10.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.344+0000 7f3c64eb7700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3c44058ff0 0x7f3c4405b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.360 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:07:10.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.344+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55d62270a070 0x55d62270bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.348+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622670030 con 0x55d62270a070 2023-11-15T03:07:10.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.348+0000 7f3c5c6b6700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55d62270a070 0x55d62270bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.348+0000 7f3c64eb7700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3c44058ff0 0x7f3c4405b4b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f3c54000c00 tx=0x7f3c540039b0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:10.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.348+0000 7f3c5c6b6700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55d62270a070 0x55d62270bfb0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:10.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.348+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v114'233 uv226 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d62270a070 2023-11-15T03:07:10.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.348+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d6226703e0 con 0x55d62270a070 2023-11-15T03:07:10.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.352+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v114'234 uv234 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d62270a070 2023-11-15T03:07:10.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.352+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622670790 con 0x55d62270a070 2023-11-15T03:07:10.363 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:07:10.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.356+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v114'235 uv235 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d62270a070 2023-11-15T03:07:10.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.356+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227327b0 con 0x55d62270a070 2023-11-15T03:07:10.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.356+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v114'236 uv236 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d62270a070 2023-11-15T03:07:10.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.356+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622732b60 con 0x55d62270a070 2023-11-15T03:07:10.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.360+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v114'237 uv237 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d62270a070 2023-11-15T03:07:10.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.360+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227317a0 con 0x55d62270a070 2023-11-15T03:07:10.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.360+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v114'238 uv238 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d62270a070 2023-11-15T03:07:10.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.360+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622731ba0 con 0x55d62270a070 2023-11-15T03:07:10.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.364+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v114'239 uv232 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d62270a070 2023-11-15T03:07:10.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.364+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d622731fa0 con 0x55d62270a070 2023-11-15T03:07:10.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.368+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v114'240 uv240 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d62270a070 2023-11-15T03:07:10.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.368+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622731fa0 con 0x55d62270a070 2023-11-15T03:07:10.376 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:07:10.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.368+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v114'241 uv241 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d62270a070 2023-11-15T03:07:10.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.368+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622731fa0 con 0x55d62270a070 2023-11-15T03:07:10.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.372+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v114'242 uv242 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d62270a070 2023-11-15T03:07:10.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.372+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622731fa0 con 0x55d62270a070 2023-11-15T03:07:10.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.372+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v114'243 uv243 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d62270a070 2023-11-15T03:07:10.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.372+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622731fa0 con 0x55d62270a070 2023-11-15T03:07:10.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.376+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v114'244 uv244 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d62270a070 2023-11-15T03:07:10.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.376+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55d6227323a0 0x55d622735430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.376+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622731fa0 con 0x55d6227323a0 2023-11-15T03:07:10.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.376+0000 7f3c64eb7700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55d6227323a0 0x55d622735430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.376+0000 7f3c64eb7700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55d6227323a0 0x55d622735430 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:10.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.380+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v114'366 uv353 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.380+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d622735920 con 0x55d6227323a0 2023-11-15T03:07:10.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.384+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v114'367 uv367 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.384+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622735cd0 con 0x55d6227323a0 2023-11-15T03:07:10.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.388+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v114'368 uv368 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.396 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:07:10.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.388+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622737090 con 0x55d6227323a0 2023-11-15T03:07:10.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.392+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v114'369 uv369 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.392+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622737490 con 0x55d6227323a0 2023-11-15T03:07:10.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.392+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v114'370 uv370 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.392+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622736080 con 0x55d6227323a0 2023-11-15T03:07:10.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.396+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v114'371 uv371 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.396+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622736970 con 0x55d6227323a0 2023-11-15T03:07:10.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.400+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v114'372 uv372 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.400+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622736970 con 0x55d6227323a0 2023-11-15T03:07:10.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.400+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v114'373 uv359 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.400+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d622736970 con 0x55d6227323a0 2023-11-15T03:07:10.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.404+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v114'374 uv374 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.404+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622736970 con 0x55d6227323a0 2023-11-15T03:07:10.415 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:07:10.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.408+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v114'375 uv375 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.408+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622736970 con 0x55d6227323a0 2023-11-15T03:07:10.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.412+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v114'376 uv376 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.412+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622736970 con 0x55d6227323a0 2023-11-15T03:07:10.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.412+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v114'377 uv377 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.412+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622736970 con 0x55d6227323a0 2023-11-15T03:07:10.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.416+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v114'378 uv378 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.416+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622736970 con 0x55d6227323a0 2023-11-15T03:07:10.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.420+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v114'379 uv365 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.420+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d622736970 con 0x55d6227323a0 2023-11-15T03:07:10.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.420+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v114'380 uv380 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.420+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622736970 con 0x55d6227323a0 2023-11-15T03:07:10.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.424+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v114'381 uv381 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.433 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:07:10.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.424+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622736970 con 0x55d6227323a0 2023-11-15T03:07:10.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.428+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v114'382 uv382 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.428+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622736970 con 0x55d6227323a0 2023-11-15T03:07:10.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.428+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v114'383 uv383 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.428+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622736970 con 0x55d6227323a0 2023-11-15T03:07:10.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.432+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v114'384 uv384 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.432+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55d622737da0 0x55d62273a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.432+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622736970 con 0x55d622737da0 2023-11-15T03:07:10.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.432+0000 7f3c5c6b6700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55d622737da0 0x55d62273a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.436+0000 7f3c5c6b6700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55d622737da0 0x55d62273a190 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:10.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.436+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v114'201 uv195 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d622737da0 2023-11-15T03:07:10.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.436+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d62273a680 con 0x55d622737da0 2023-11-15T03:07:10.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.440+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v114'202 uv202 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d622737da0 2023-11-15T03:07:10.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.440+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d62273a680 con 0x55d622737da0 2023-11-15T03:07:10.451 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:07:10.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.444+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v114'203 uv203 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d622737da0 2023-11-15T03:07:10.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.444+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d62273ab70 con 0x55d622737da0 2023-11-15T03:07:10.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.448+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v114'204 uv204 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d622737da0 2023-11-15T03:07:10.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.448+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d62273ab70 con 0x55d622737da0 2023-11-15T03:07:10.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.448+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v114'205 uv205 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d622737da0 2023-11-15T03:07:10.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.448+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d62273ab70 con 0x55d622737da0 2023-11-15T03:07:10.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.452+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v114'206 uv200 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d622737da0 2023-11-15T03:07:10.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.452+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d62273ab70 con 0x55d622737da0 2023-11-15T03:07:10.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.456+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v114'207 uv207 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d622737da0 2023-11-15T03:07:10.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.456+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d62273ab70 con 0x55d622737da0 2023-11-15T03:07:10.465 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:07:10.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.456+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v114'208 uv208 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d622737da0 2023-11-15T03:07:10.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.456+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d62273ab70 con 0x55d622737da0 2023-11-15T03:07:10.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.460+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v114'209 uv209 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d622737da0 2023-11-15T03:07:10.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.460+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d62273ab70 con 0x55d622737da0 2023-11-15T03:07:10.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.464+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v114'210 uv210 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d622737da0 2023-11-15T03:07:10.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.464+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55d62273b080 0x55d62273d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.464+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d62273d960 con 0x55d62273b080 2023-11-15T03:07:10.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.464+0000 7f3c656b8700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55d62273b080 0x55d62273d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.464+0000 7f3c656b8700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55d62273b080 0x55d62273d470 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:10.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.468+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v114'241 uv234 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.468+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d62273dd10 con 0x55d62273b080 2023-11-15T03:07:10.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.472+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v114'242 uv242 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.472+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d62273e140 con 0x55d62273b080 2023-11-15T03:07:10.482 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:07:10.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.476+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v114'243 uv243 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.476+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d62273a740 con 0x55d62273b080 2023-11-15T03:07:10.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.480+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v114'244 uv244 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.480+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622742bc0 con 0x55d62273b080 2023-11-15T03:07:10.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.480+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v114'245 uv245 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.480+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d62273e540 con 0x55d62273b080 2023-11-15T03:07:10.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.484+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v114'246 uv240 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.484+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d62273e940 con 0x55d62273b080 2023-11-15T03:07:10.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.488+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v114'247 uv247 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.488+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d62273f2b0 con 0x55d62273b080 2023-11-15T03:07:10.498 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:07:10.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.492+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v114'248 uv248 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.492+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d62273f2b0 con 0x55d62273b080 2023-11-15T03:07:10.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.492+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v114'249 uv249 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.492+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227406c0 con 0x55d62273b080 2023-11-15T03:07:10.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.496+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v114'250 uv250 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.496+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d62273f2e0 con 0x55d62273b080 2023-11-15T03:07:10.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.500+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v114'251 uv251 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.500+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55d62273fbd0 0x55d622741fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.500+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227424d0 con 0x55d62273fbd0 2023-11-15T03:07:10.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.500+0000 7f3c64eb7700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55d62273fbd0 0x55d622741fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.500+0000 7f3c64eb7700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55d62273fbd0 0x55d622741fe0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:10.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.504+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v114'117 uv116 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d62273fbd0 2023-11-15T03:07:10.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.504+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d6227424d0 con 0x55d62273fbd0 2023-11-15T03:07:10.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.504+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v114'118 uv118 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d62273fbd0 2023-11-15T03:07:10.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.504+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227424d0 con 0x55d62273fbd0 2023-11-15T03:07:10.516 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:07:10.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.508+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v114'119 uv119 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d62273fbd0 2023-11-15T03:07:10.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.508+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227424d0 con 0x55d62273fbd0 2023-11-15T03:07:10.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.512+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v114'120 uv120 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d62273fbd0 2023-11-15T03:07:10.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.512+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227424d0 con 0x55d62273fbd0 2023-11-15T03:07:10.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.512+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v114'121 uv121 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d62273fbd0 2023-11-15T03:07:10.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.512+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227424d0 con 0x55d62273fbd0 2023-11-15T03:07:10.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.516+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v114'122 uv122 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d62273fbd0 2023-11-15T03:07:10.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.516+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55d6227434d0 0x55d6227458c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.516+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622745db0 con 0x55d6227434d0 2023-11-15T03:07:10.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.516+0000 7f3c5c6b6700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55d6227434d0 0x55d6227458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.516+0000 7f3c5c6b6700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55d6227434d0 0x55d6227458c0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:10.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.520+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v114'84 uv83 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.520+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d622745df0 con 0x55d6227434d0 2023-11-15T03:07:10.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.524+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v114'85 uv85 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.524+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227434d0 2023-11-15T03:07:10.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.524+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v114'86 uv86 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.533 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:07:10.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.524+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227434d0 2023-11-15T03:07:10.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.528+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v114'87 uv87 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.528+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227434d0 2023-11-15T03:07:10.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.532+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v114'117 uv116 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.532+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227434d0 2023-11-15T03:07:10.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.536+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v114'118 uv118 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.536+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227434d0 2023-11-15T03:07:10.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.536+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v114'119 uv119 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.546 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:07:10.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.536+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622747200 con 0x55d6227434d0 2023-11-15T03:07:10.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.540+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v114'120 uv120 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.540+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622747200 con 0x55d6227434d0 2023-11-15T03:07:10.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.544+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v114'121 uv121 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.544+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227434d0 2023-11-15T03:07:10.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.548+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v114'122 uv122 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.548+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227323a0 2023-11-15T03:07:10.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.548+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v114'216 uv210 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.548+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227323a0 2023-11-15T03:07:10.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.552+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v114'217 uv217 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.552+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227323a0 2023-11-15T03:07:10.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.556+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v114'218 uv218 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.564 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:07:10.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.556+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622747200 con 0x55d6227323a0 2023-11-15T03:07:10.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.560+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v114'219 uv219 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.560+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622747200 con 0x55d6227323a0 2023-11-15T03:07:10.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.564+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v114'220 uv220 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.564+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227323a0 2023-11-15T03:07:10.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.564+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v114'221 uv221 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.568+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227323a0 2023-11-15T03:07:10.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.568+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v114'222 uv215 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.568+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227323a0 2023-11-15T03:07:10.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.572+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v114'223 uv223 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.572+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227323a0 2023-11-15T03:07:10.583 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:07:10.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.576+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v114'224 uv224 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.576+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227323a0 2023-11-15T03:07:10.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.580+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v114'225 uv225 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.580+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227323a0 2023-11-15T03:07:10.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.584+0000 7f3c64eb7700 1 -- 172.21.15.121:0/3134244921 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v114'226 uv226 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c54012880 con 0x55d6227323a0 2023-11-15T03:07:10.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.584+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d622737da0 2023-11-15T03:07:10.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.584+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v114'97 uv96 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d622737da0 2023-11-15T03:07:10.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.584+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d622737da0 2023-11-15T03:07:10.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.588+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v114'98 uv98 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d622737da0 2023-11-15T03:07:10.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.588+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d622737da0 2023-11-15T03:07:10.598 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:07:10.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.588+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v114'99 uv99 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d622737da0 2023-11-15T03:07:10.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.588+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d622737da0 2023-11-15T03:07:10.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.592+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v114'100 uv100 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d622737da0 2023-11-15T03:07:10.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.592+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d622737da0 2023-11-15T03:07:10.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.596+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v114'101 uv101 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d622737da0 2023-11-15T03:07:10.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.596+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227434d0 2023-11-15T03:07:10.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.600+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v114'117 uv116 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.600+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227434d0 2023-11-15T03:07:10.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.600+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v114'118 uv118 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.600+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227434d0 2023-11-15T03:07:10.612 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:07:10.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.604+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v114'119 uv119 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.604+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622747200 con 0x55d6227434d0 2023-11-15T03:07:10.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.608+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v114'120 uv120 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.608+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622747200 con 0x55d6227434d0 2023-11-15T03:07:10.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.608+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v114'121 uv121 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.608+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227434d0 2023-11-15T03:07:10.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.612+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v114'122 uv122 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.612+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227434d0 2023-11-15T03:07:10.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.616+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v114'97 uv96 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.616+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227434d0 2023-11-15T03:07:10.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.620+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v114'98 uv98 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.620+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227434d0 2023-11-15T03:07:10.629 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:07:10.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.620+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v114'99 uv99 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.620+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227434d0 2023-11-15T03:07:10.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.624+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v114'100 uv100 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.624+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227434d0 2023-11-15T03:07:10.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.628+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v114'101 uv101 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.628+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227434d0 2023-11-15T03:07:10.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.628+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v114'96 uv95 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.632+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227434d0 2023-11-15T03:07:10.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.632+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v114'97 uv97 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.632+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227434d0 2023-11-15T03:07:10.643 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:07:10.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.636+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v114'98 uv98 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.636+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227461f0 con 0x55d6227434d0 2023-11-15T03:07:10.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.640+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v114'99 uv99 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.640+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622748f60 con 0x55d6227434d0 2023-11-15T03:07:10.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.640+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v114'100 uv100 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.640+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55d622746d60 0x55d622747140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.640+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227476a0 con 0x55d622746d60 2023-11-15T03:07:10.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.644+0000 7f3c656b8700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55d622746d60 0x55d622747140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.644+0000 7f3c656b8700 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55d622746d60 0x55d622747140 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:10.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.644+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v114'78 uv77 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d622746d60 2023-11-15T03:07:10.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.644+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d6227476a0 con 0x55d622746d60 2023-11-15T03:07:10.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.648+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v114'79 uv79 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d622746d60 2023-11-15T03:07:10.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.648+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227476a0 con 0x55d622746d60 2023-11-15T03:07:10.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.652+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v114'80 uv80 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d622746d60 2023-11-15T03:07:10.660 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:07:10.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.652+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227476a0 con 0x55d622746d60 2023-11-15T03:07:10.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.656+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v114'81 uv81 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d622746d60 2023-11-15T03:07:10.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.656+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227476a0 con 0x55d6227434d0 2023-11-15T03:07:10.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.656+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v114'117 uv116 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.656+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d6227476a0 con 0x55d6227434d0 2023-11-15T03:07:10.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.660+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v114'118 uv118 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.660+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227476a0 con 0x55d6227434d0 2023-11-15T03:07:10.670 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:07:10.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.664+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v114'119 uv119 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.664+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227476a0 con 0x55d6227434d0 2023-11-15T03:07:10.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.664+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v114'120 uv120 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.664+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227476a0 con 0x55d6227434d0 2023-11-15T03:07:10.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.668+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v114'121 uv121 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.668+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227476a0 con 0x55d6227434d0 2023-11-15T03:07:10.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.672+0000 7f3c5c6b6700 1 -- 172.21.15.121:0/3134244921 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v114'122 uv122 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c4c000bc0 con 0x55d6227434d0 2023-11-15T03:07:10.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.672+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227476a0 con 0x55d62273b080 2023-11-15T03:07:10.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.672+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v114'99 uv98 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.672+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d6227476a0 con 0x55d62273b080 2023-11-15T03:07:10.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.676+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v114'100 uv100 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.676+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227476a0 con 0x55d62273b080 2023-11-15T03:07:10.686 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:07:10.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.680+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v114'101 uv101 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.680+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227476a0 con 0x55d62273b080 2023-11-15T03:07:10.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.680+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v114'102 uv102 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.680+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622747af0 con 0x55d62273b080 2023-11-15T03:07:10.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.684+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v114'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.684+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622747af0 con 0x55d62273b080 2023-11-15T03:07:10.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.688+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v114'232 uv224 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.688+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d6227476a0 con 0x55d62273b080 2023-11-15T03:07:10.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.692+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v114'233 uv233 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.692+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622747af0 con 0x55d62273b080 2023-11-15T03:07:10.701 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:07:10.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.692+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v114'234 uv234 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.692+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d6227476a0 con 0x55d62273b080 2023-11-15T03:07:10.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.696+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v114'235 uv235 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.696+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622747ba0 con 0x55d62273b080 2023-11-15T03:07:10.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.700+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v114'236 uv236 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.700+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622747ba0 con 0x55d62273b080 2023-11-15T03:07:10.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.700+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v114'237 uv231 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.700+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e114) v8 -- 0x55d6227476a0 con 0x55d62273b080 2023-11-15T03:07:10.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.704+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v114'238 uv238 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.704+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622747aa0 con 0x55d62273b080 2023-11-15T03:07:10.715 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:07:10.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.708+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v114'239 uv239 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.708+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622748ab0 con 0x55d62273b080 2023-11-15T03:07:10.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.708+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v114'240 uv240 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.708+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622748ab0 con 0x55d62273b080 2023-11-15T03:07:10.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.712+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v114'241 uv241 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.712+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622748eb0 con 0x55d62273b080 2023-11-15T03:07:10.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.716+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v114'242 uv242 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.716+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x55d622747aa0 con 0x55d62273b080 2023-11-15T03:07:10.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v114'243 uv243 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3c60049cb0 con 0x55d62273b080 2023-11-15T03:07:10.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55d622737da0 msgr2=0x55d62273a190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55d622737da0 0x55d62273a190 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55d62270a070 msgr2=0x55d62270bfb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55d62270a070 0x55d62270bfb0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55d62273fbd0 msgr2=0x55d622741fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55d62273fbd0 0x55d622741fe0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55d6227323a0 msgr2=0x55d622735430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55d6227323a0 0x55d622735430 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55d622746d60 msgr2=0x55d622747140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55d622746d60 0x55d622747140 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55d62273b080 msgr2=0x55d62273d470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55d62273b080 0x55d62273d470 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55d6227434d0 msgr2=0x55d6227458c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55d6227434d0 0x55d6227458c0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 reap_dead start 2023-11-15T03:07:10.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 reap_dead start 2023-11-15T03:07:10.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 reap_dead start 2023-11-15T03:07:10.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c656b8700 1 -- 172.21.15.121:0/3134244921 reap_dead start 2023-11-15T03:07:10.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3c44058ff0 msgr2=0x7f3c4405b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3c44058ff0 0x7f3c4405b4b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f3c54000c00 tx=0x7f3c540039b0).stop 2023-11-15T03:07:10.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d622718650 msgr2=0x55d62266b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d622718650 0x55d62266b050 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7f3c60000f80 tx=0x7f3c60014000).stop 2023-11-15T03:07:10.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 shutdown_connections 2023-11-15T03:07:10.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3c44058ff0 0x7f3c4405b4b0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 --2- 172.21.15.121:0/3134244921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d622718650 0x55d62266b050 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 >> 172.21.15.121:0/3134244921 conn(0x55d62268b080 msgr2=0x55d62270d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:10.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 shutdown_connections 2023-11-15T03:07:10.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.720+0000 7f3c7093ad00 1 -- 172.21.15.121:0/3134244921 wait complete. 2023-11-15T03:07:10.736 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2023-11-15T03:07:10.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.832+0000 7f0a5eb2ed00 1 Processor -- start 2023-11-15T03:07:10.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.832+0000 7f0a5eb2ed00 1 -- start start 2023-11-15T03:07:10.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.832+0000 7f0a5eb2ed00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5584d36cc6c0 0x5584d37a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.832+0000 7f0a5eb2ed00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584d3716b00 0x5584d37139b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.832+0000 7f0a5eb2ed00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5584d37a5670 0x5584d3713ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.832+0000 7f0a5eb2ed00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d3645220 con 0x5584d3716b00 2023-11-15T03:07:10.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.832+0000 7f0a5eb2ed00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d37974b0 con 0x5584d37a5670 2023-11-15T03:07:10.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.832+0000 7f0a5eb2ed00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584d36ffa10 con 0x5584d36cc6c0 2023-11-15T03:07:10.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a52a3b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584d3716b00 0x5584d37139b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a52a3b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584d3716b00 0x5584d37139b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53360/0 (socket says 172.21.15.121:53360) 2023-11-15T03:07:10.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a52a3b700 1 -- 172.21.15.121:0/4066062275 learned_addr learned my addr 172.21.15.121:0/4066062275 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:10.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a5323c700 1 --2- 172.21.15.121:0/4066062275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5584d36cc6c0 0x5584d37a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a53a3d700 1 --2- 172.21.15.121:0/4066062275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5584d37a5670 0x5584d3713ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a52a3b700 1 -- 172.21.15.121:0/4066062275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5584d36cc6c0 msgr2=0x5584d37a6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:10.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a52a3b700 1 --2- 172.21.15.121:0/4066062275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5584d36cc6c0 0x5584d37a6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a52a3b700 1 -- 172.21.15.121:0/4066062275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5584d37a5670 msgr2=0x5584d3713ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:10.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a52a3b700 1 --2- 172.21.15.121:0/4066062275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5584d37a5670 0x5584d3713ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a52a3b700 1 -- 172.21.15.121:0/4066062275 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a48005db0 con 0x5584d3716b00 2023-11-15T03:07:10.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a53a3d700 1 --2- 172.21.15.121:0/4066062275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5584d37a5670 0x5584d3713ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:07:10.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a52a3b700 1 --2- 172.21.15.121:0/4066062275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584d3716b00 0x5584d37139b0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 rx=0x7f0a48010000 tx=0x7f0a48016000).ready entity=mon.0 client_cookie=530fb1e10a937a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:10.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a51a39700 1 -- 172.21.15.121:0/4066062275 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a480054f0 con 0x5584d3716b00 2023-11-15T03:07:10.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a51a39700 1 -- 172.21.15.121:0/4066062275 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a48000bc0 con 0x5584d3716b00 2023-11-15T03:07:10.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a51a39700 1 -- 172.21.15.121:0/4066062275 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48008ab0 con 0x5584d3716b00 2023-11-15T03:07:10.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a5eb2ed00 1 -- 172.21.15.121:0/4066062275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584d3716b00 msgr2=0x5584d37139b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a5eb2ed00 1 --2- 172.21.15.121:0/4066062275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584d3716b00 0x5584d37139b0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 rx=0x7f0a48010000 tx=0x7f0a48016000).stop 2023-11-15T03:07:10.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a5eb2ed00 1 -- 172.21.15.121:0/4066062275 shutdown_connections 2023-11-15T03:07:10.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a5eb2ed00 1 --2- 172.21.15.121:0/4066062275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5584d37a5670 0x5584d3713ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a5eb2ed00 1 --2- 172.21.15.121:0/4066062275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584d3716b00 0x5584d37139b0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a5eb2ed00 1 --2- 172.21.15.121:0/4066062275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5584d36cc6c0 0x5584d37a6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a5eb2ed00 1 -- 172.21.15.121:0/4066062275 >> 172.21.15.121:0/4066062275 conn(0x5584d371c080 msgr2=0x5584d37a4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:10.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a5eb2ed00 1 -- 172.21.15.121:0/4066062275 shutdown_connections 2023-11-15T03:07:10.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.836+0000 7f0a5eb2ed00 1 -- 172.21.15.121:0/4066062275 wait complete. 2023-11-15T03:07:10.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a5eb2ed00 1 Processor -- start 2023-11-15T03:07:10.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a5eb2ed00 1 -- start start 2023-11-15T03:07:10.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a5eb2ed00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584d36cc6c0 0x5584d37b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a5eb2ed00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5584d3716b00 0x5584d37a9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a5eb2ed00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5584d37a5670 0x5584d37a9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a5eb2ed00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d37aa010 con 0x5584d36cc6c0 2023-11-15T03:07:10.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a5eb2ed00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d36d56d0 con 0x5584d37a5670 2023-11-15T03:07:10.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a5eb2ed00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584d37b7410 con 0x5584d3716b00 2023-11-15T03:07:10.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a5323c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584d36cc6c0 0x5584d37b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a5323c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584d36cc6c0 0x5584d37b8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53376/0 (socket says 172.21.15.121:53376) 2023-11-15T03:07:10.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a5323c700 1 -- 172.21.15.121:0/2361434938 learned_addr learned my addr 172.21.15.121:0/2361434938 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:10.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a53a3d700 1 --2- 172.21.15.121:0/2361434938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5584d37a5670 0x5584d37a9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a52a3b700 1 --2- 172.21.15.121:0/2361434938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5584d3716b00 0x5584d37a9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a5323c700 1 -- 172.21.15.121:0/2361434938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5584d3716b00 msgr2=0x5584d37a9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a5323c700 1 --2- 172.21.15.121:0/2361434938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5584d3716b00 0x5584d37a9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a5323c700 1 -- 172.21.15.121:0/2361434938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5584d37a5670 msgr2=0x5584d37a9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a5323c700 1 --2- 172.21.15.121:0/2361434938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5584d37a5670 0x5584d37a9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a5323c700 1 -- 172.21.15.121:0/2361434938 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584d366ea20 con 0x5584d36cc6c0 2023-11-15T03:07:10.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a52a3b700 1 --2- 172.21.15.121:0/2361434938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5584d3716b00 0x5584d37a9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:07:10.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.840+0000 7f0a5323c700 1 --2- 172.21.15.121:0/2361434938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584d36cc6c0 0x5584d37b8c60 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 rx=0x7f0a44005940 tx=0x7f0a4400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:10.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.844+0000 7f0a3bfff700 1 -- 172.21.15.121:0/2361434938 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a44006e30 con 0x5584d36cc6c0 2023-11-15T03:07:10.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.844+0000 7f0a3bfff700 1 -- 172.21.15.121:0/2361434938 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4400f040 con 0x5584d36cc6c0 2023-11-15T03:07:10.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.844+0000 7f0a3bfff700 1 -- 172.21.15.121:0/2361434938 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4400e630 con 0x5584d36cc6c0 2023-11-15T03:07:10.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.844+0000 7f0a5eb2ed00 1 -- 172.21.15.121:0/2361434938 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5584d366e770 con 0x5584d36cc6c0 2023-11-15T03:07:10.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.844+0000 7f0a5eb2ed00 1 -- 172.21.15.121:0/2361434938 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584d363da70 con 0x5584d36cc6c0 2023-11-15T03:07:10.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.844+0000 7f0a3bfff700 1 -- 172.21.15.121:0/2361434938 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0a4400ea20 con 0x5584d36cc6c0 2023-11-15T03:07:10.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.844+0000 7f0a3bfff700 1 --2- 172.21.15.121:0/2361434938 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0a40059020 0x7f0a4005b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.848+0000 7f0a3bfff700 1 -- 172.21.15.121:0/2361434938 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 7986+0+0 (secure 0 0 0) 0x7f0a44073a60 con 0x5584d36cc6c0 2023-11-15T03:07:10.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.848+0000 7f0a52a3b700 1 --2- 172.21.15.121:0/2361434938 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0a40059020 0x7f0a4005b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.848+0000 7f0a5eb2ed00 1 --2- 172.21.15.121:0/2361434938 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5584d3796d20 0x5584d37b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:10.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.848+0000 7f0a5eb2ed00 1 -- 172.21.15.121:0/2361434938 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected e114) v8 -- 0x5584d37b9150 con 0x5584d3796d20 2023-11-15T03:07:10.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.848+0000 7f0a52a3b700 1 --2- 172.21.15.121:0/2361434938 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0a40059020 0x7f0a4005b4e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f0a48021000 tx=0x7f0a4801c000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:10.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.848+0000 7f0a53a3d700 1 --2- 172.21.15.121:0/2361434938 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5584d3796d20 0x5584d37b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:10.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.848+0000 7f0a53a3d700 1 --2- 172.21.15.121:0/2361434938 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5584d3796d20 0x5584d37b69c0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:10.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.852+0000 7f0a53a3d700 1 -- 172.21.15.121:0/2361434938 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~80] v114'102 uv102 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0a4c004610 con 0x5584d3796d20 2023-11-15T03:07:10.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.852+0000 7f0a5eb2ed00 1 -- 172.21.15.121:0/2361434938 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5584d3796d20 msgr2=0x5584d37b69c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.852+0000 7f0a5eb2ed00 1 --2- 172.21.15.121:0/2361434938 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5584d3796d20 0x5584d37b69c0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:10.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.852+0000 7f0a5eb2ed00 1 -- 172.21.15.121:0/2361434938 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0a40059020 msgr2=0x7f0a4005b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.852+0000 7f0a5eb2ed00 1 --2- 172.21.15.121:0/2361434938 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0a40059020 0x7f0a4005b4e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f0a48021000 tx=0x7f0a4801c000).stop 2023-11-15T03:07:10.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.852+0000 7f0a5eb2ed00 1 -- 172.21.15.121:0/2361434938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584d36cc6c0 msgr2=0x5584d37b8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:10.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.852+0000 7f0a5eb2ed00 1 --2- 172.21.15.121:0/2361434938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584d36cc6c0 0x5584d37b8c60 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 rx=0x7f0a44005940 tx=0x7f0a4400b040).stop 2023-11-15T03:07:10.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.852+0000 7f0a53a3d700 1 -- 172.21.15.121:0/2361434938 reap_dead start 2023-11-15T03:07:10.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.852+0000 7f0a5eb2ed00 1 -- 172.21.15.121:0/2361434938 shutdown_connections 2023-11-15T03:07:10.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.852+0000 7f0a5eb2ed00 1 -- 172.21.15.121:0/2361434938 >> 172.21.15.121:0/2361434938 conn(0x5584d371c080 msgr2=0x5584d3710fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:10.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.852+0000 7f0a5eb2ed00 1 -- 172.21.15.121:0/2361434938 shutdown_connections 2023-11-15T03:07:10.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:10.852+0000 7f0a5eb2ed00 1 -- 172.21.15.121:0/2361434938 wait complete. 2023-11-15T03:07:10.867 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:07:11.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.068+0000 7fe51dee4700 1 Processor -- start 2023-11-15T03:07:11.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.072+0000 7fe51dee4700 1 -- start start 2023-11-15T03:07:11.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.072+0000 7fe51dee4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5181539c0 0x7fe5180c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:11.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.072+0000 7fe51dee4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe5180c22b0 0x7fe5180c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:11.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.072+0000 7fe51dee4700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5180a38c0 con 0x7fe518153da0 2023-11-15T03:07:11.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.072+0000 7fe51dee4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5180c2740 con 0x7fe5180c22b0 2023-11-15T03:07:11.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.072+0000 7fe51dee4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5180c2a20 con 0x7fe5181539c0 2023-11-15T03:07:11.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe51d6e3700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe518153da0 0x7fe518066080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:54178/0 (socket says 172.21.15.121:54178) 2023-11-15T03:07:11.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe51d6e3700 1 -- 172.21.15.121:0/3707826210 learned_addr learned my addr 172.21.15.121:0/3707826210 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:11.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe51cee2700 1 --2- 172.21.15.121:0/3707826210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5181539c0 0x7fe5180c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:11.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe50ffff700 1 --2- 172.21.15.121:0/3707826210 >> v2:172.21.15.190:3300/0 conn(0x7fe5180c22b0 0x7fe5180c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:11.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe51cee2700 1 -- 172.21.15.121:0/3707826210 >> v2:172.21.15.190:3300/0 conn(0x7fe5180c22b0 msgr2=0x7fe5180c4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:11.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe50f7fe700 1 -- 172.21.15.121:0/3707826210 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313777717 0 0) 0x7fe5180a38c0 con 0x7fe518153da0 2023-11-15T03:07:11.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe51cee2700 1 --2- 172.21.15.121:0/3707826210 >> v2:172.21.15.190:3300/0 conn(0x7fe5180c22b0 0x7fe5180c4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:11.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe51cee2700 1 -- 172.21.15.121:0/3707826210 >> v1:172.21.15.121:6789/0 conn(0x7fe518153da0 legacy=0x7fe518066080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:11.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe51cee2700 1 -- 172.21.15.121:0/3707826210 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5180c2d00 con 0x7fe5181539c0 2023-11-15T03:07:11.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe51cee2700 1 --2- 172.21.15.121:0/3707826210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5181539c0 0x7fe5180c49b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fe514009760 tx=0x7fe514009060).ready entity=mon.2 client_cookie=4cc972f5480fef2e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:11.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe50f7fe700 1 -- 172.21.15.121:0/3707826210 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51400f510 con 0x7fe5181539c0 2023-11-15T03:07:11.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe50f7fe700 1 -- 172.21.15.121:0/3707826210 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51400f670 con 0x7fe5181539c0 2023-11-15T03:07:11.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe50f7fe700 1 -- 172.21.15.121:0/3707826210 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51400f9f0 con 0x7fe5181539c0 2023-11-15T03:07:11.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe51dee4700 1 -- 172.21.15.121:0/3707826210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5181539c0 msgr2=0x7fe5180c49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:11.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe51dee4700 1 --2- 172.21.15.121:0/3707826210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5181539c0 0x7fe5180c49b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fe514009760 tx=0x7fe514009060).stop 2023-11-15T03:07:11.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe51dee4700 1 -- 172.21.15.121:0/3707826210 shutdown_connections 2023-11-15T03:07:11.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe51dee4700 1 --2- 172.21.15.121:0/3707826210 >> v2:172.21.15.190:3300/0 conn(0x7fe5180c22b0 0x7fe5180c4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:11.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe51dee4700 1 --2- 172.21.15.121:0/3707826210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5181539c0 0x7fe5180c49b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:11.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe51dee4700 1 -- 172.21.15.121:0/3707826210 >> 172.21.15.121:0/3707826210 conn(0x7fe518062c00 msgr2=0x7fe5180c6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:11.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe51dee4700 1 -- 172.21.15.121:0/3707826210 shutdown_connections 2023-11-15T03:07:11.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.076+0000 7fe51dee4700 1 -- 172.21.15.121:0/3707826210 wait complete. 2023-11-15T03:07:11.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe51dee4700 1 Processor -- start 2023-11-15T03:07:11.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe51dee4700 1 -- start start 2023-11-15T03:07:11.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe51dee4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5181539c0 0x7fe5180abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:11.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe51dee4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe518153da0 0x7fe518066080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:11.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe51dee4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe518066570 0x7fe518167980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:11.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe51dee4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5180acd20 con 0x7fe518066570 2023-11-15T03:07:11.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe51dee4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5180669f0 con 0x7fe5181539c0 2023-11-15T03:07:11.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe51dee4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5180ac110 con 0x7fe518153da0 2023-11-15T03:07:11.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe51d6e3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe518066570 0x7fe518167980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:11.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe51d6e3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe518066570 0x7fe518167980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53390/0 (socket says 172.21.15.121:53390) 2023-11-15T03:07:11.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe51d6e3700 1 -- 172.21.15.121:0/1537939960 learned_addr learned my addr 172.21.15.121:0/1537939960 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:11.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe51cee2700 1 --2- 172.21.15.121:0/1537939960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5181539c0 0x7fe5180abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:11.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe50ffff700 1 --2- 172.21.15.121:0/1537939960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe518153da0 0x7fe518066080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:11.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe51d6e3700 1 -- 172.21.15.121:0/1537939960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe518153da0 msgr2=0x7fe518066080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:11.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe51d6e3700 1 --2- 172.21.15.121:0/1537939960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe518153da0 0x7fe518066080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:11.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe51d6e3700 1 -- 172.21.15.121:0/1537939960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5181539c0 msgr2=0x7fe5180abaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:11.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe51d6e3700 1 --2- 172.21.15.121:0/1537939960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5181539c0 0x7fe5180abaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:11.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe51d6e3700 1 -- 172.21.15.121:0/1537939960 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5140092a0 con 0x7fe518066570 2023-11-15T03:07:11.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe51d6e3700 1 --2- 172.21.15.121:0/1537939960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe518066570 0x7fe518167980 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 rx=0x7fe51000c810 tx=0x7fe5100014d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:11.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.080+0000 7fe50cff9700 1 -- 172.21.15.121:0/1537939960 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510005d30 con 0x7fe518066570 2023-11-15T03:07:11.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.084+0000 7fe50cff9700 1 -- 172.21.15.121:0/1537939960 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe510005e90 con 0x7fe518066570 2023-11-15T03:07:11.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.084+0000 7fe51dee4700 1 -- 172.21.15.121:0/1537939960 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5180ac3f0 con 0x7fe518066570 2023-11-15T03:07:11.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.084+0000 7fe50cff9700 1 -- 172.21.15.121:0/1537939960 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510007340 con 0x7fe518066570 2023-11-15T03:07:11.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.084+0000 7fe51dee4700 1 -- 172.21.15.121:0/1537939960 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5180ac9b0 con 0x7fe518066570 2023-11-15T03:07:11.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.084+0000 7fe51dee4700 1 -- 172.21.15.121:0/1537939960 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51806bcf0 con 0x7fe518066570 2023-11-15T03:07:11.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.084+0000 7fe50cff9700 1 -- 172.21.15.121:0/1537939960 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe510007730 con 0x7fe518066570 2023-11-15T03:07:11.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.088+0000 7fe50cff9700 1 --2- 172.21.15.121:0/1537939960 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe50405a510 0x7fe50405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:11.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.088+0000 7fe50cff9700 1 -- 172.21.15.121:0/1537939960 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 7986+0+0 (secure 0 0 0) 0x7fe510074fe0 con 0x7fe518066570 2023-11-15T03:07:11.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.088+0000 7fe51cee2700 1 --2- 172.21.15.121:0/1537939960 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe50405a510 0x7fe50405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:11.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.088+0000 7fe51cee2700 1 --2- 172.21.15.121:0/1537939960 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe50405a510 0x7fe50405c9d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7fe514009fd0 tx=0x7fe514017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:11.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.092+0000 7fe50cff9700 1 -- 172.21.15.121:0/1537939960 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe510047ff0 con 0x7fe518066570 2023-11-15T03:07:11.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:11.288+0000 7fe51dee4700 1 -- 172.21.15.121:0/1537939960 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fe518067b00 con 0x7fe518066570 2023-11-15T03:07:11.902 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:07:11.910 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:07:12.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.256+0000 7fe50cff9700 1 -- 172.21.15.121:0/1537939960 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v209) v1 ==== 162+0+0 (secure 0 0 0) 0x7fe51004b610 con 0x7fe518066570 2023-11-15T03:07:12.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.260+0000 7fe51dee4700 1 -- 172.21.15.121:0/1537939960 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe50405a510 msgr2=0x7fe50405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:12.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.260+0000 7fe51dee4700 1 --2- 172.21.15.121:0/1537939960 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe50405a510 0x7fe50405c9d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7fe514009fd0 tx=0x7fe514017000).stop 2023-11-15T03:07:12.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.260+0000 7fe51dee4700 1 -- 172.21.15.121:0/1537939960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe518066570 msgr2=0x7fe518167980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:12.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.260+0000 7fe51dee4700 1 --2- 172.21.15.121:0/1537939960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe518066570 0x7fe518167980 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 rx=0x7fe51000c810 tx=0x7fe5100014d0).stop 2023-11-15T03:07:12.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.260+0000 7fe51dee4700 1 -- 172.21.15.121:0/1537939960 shutdown_connections 2023-11-15T03:07:12.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.260+0000 7fe51dee4700 1 --2- 172.21.15.121:0/1537939960 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe50405a510 0x7fe50405c9d0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:12.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.260+0000 7fe51dee4700 1 --2- 172.21.15.121:0/1537939960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe518066570 0x7fe518167980 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:12.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.260+0000 7fe51dee4700 1 --2- 172.21.15.121:0/1537939960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe518153da0 0x7fe518066080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:12.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.260+0000 7fe51dee4700 1 --2- 172.21.15.121:0/1537939960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5181539c0 0x7fe5180abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:12.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.260+0000 7fe51dee4700 1 -- 172.21.15.121:0/1537939960 >> 172.21.15.121:0/1537939960 conn(0x7fe518062c00 msgr2=0x7fe5180c3990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:12.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.260+0000 7fe51dee4700 1 -- 172.21.15.121:0/1537939960 shutdown_connections 2023-11-15T03:07:12.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.260+0000 7fe51dee4700 1 -- 172.21.15.121:0/1537939960 wait complete. 2023-11-15T03:07:12.272 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:07:12.276 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:07:12.268+0000 7f9ce09b5700 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': virtual void InoTable::decode_state(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 2 < 101: Malformed input 2023-11-15T03:07:12.292 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:12.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.448+0000 7f86665db700 1 Processor -- start 2023-11-15T03:07:12.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.448+0000 7f86665db700 1 -- start start 2023-11-15T03:07:12.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.448+0000 7f86665db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8660153bb0 0x7f86600a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:12.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.448+0000 7f86665db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86600aa020 0x7f8660067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:12.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.448+0000 7f86665db700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86600cb5c0 con 0x7f8660067f50 2023-11-15T03:07:12.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.448+0000 7f86665db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86600aa6f0 con 0x7f8660153bb0 2023-11-15T03:07:12.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.448+0000 7f86665db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86600c5900 con 0x7f86600aa020 2023-11-15T03:07:12.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.452+0000 7f865ffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8660153bb0 0x7f86600a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:12.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.452+0000 7f86577fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86600aa020 0x7f8660067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:12.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.452+0000 7f8664b78700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f8660067f50 0x7f86601578a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:37794/0 (socket says 172.21.15.121:37794) 2023-11-15T03:07:12.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.452+0000 7f865ffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8660153bb0 0x7f86600a9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:41624/0 (socket says 172.21.15.121:41624) 2023-11-15T03:07:12.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.452+0000 7f8664b78700 1 -- 172.21.15.121:0/1198943645 learned_addr learned my addr 172.21.15.121:0/1198943645 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:12.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.452+0000 7f865f7fe700 1 -- 172.21.15.121:0/1198943645 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395926163 0 0) 0x7f86600cb5c0 con 0x7f8660067f50 2023-11-15T03:07:12.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.452+0000 7f865f7fe700 1 -- 172.21.15.121:0/1198943645 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f864c0033f0 con 0x7f8660067f50 2023-11-15T03:07:12.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.452+0000 7f865ffff700 1 -- 172.21.15.121:0/1198943645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86600aa020 msgr2=0x7f8660067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:12.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.452+0000 7f865ffff700 1 --2- 172.21.15.121:0/1198943645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86600aa020 0x7f8660067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:12.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.452+0000 7f865ffff700 1 -- 172.21.15.121:0/1198943645 >> v1:172.21.15.190:6789/0 conn(0x7f8660067f50 legacy=0x7f86601578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:12.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.452+0000 7f865ffff700 1 -- 172.21.15.121:0/1198943645 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86600683e0 con 0x7f8660153bb0 2023-11-15T03:07:12.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.452+0000 7f86577fe700 1 --2- 172.21.15.121:0/1198943645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86600aa020 0x7f8660067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:07:12.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.452+0000 7f865ffff700 1 --2- 172.21.15.121:0/1198943645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8660153bb0 0x7f86600a9b30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f8650009760 tx=0x7f8650009060).ready entity=mon.1 client_cookie=136fb0724593ee30 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:12.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.452+0000 7f865f7fe700 1 -- 172.21.15.121:0/1198943645 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865000d4a0 con 0x7f8660153bb0 2023-11-15T03:07:12.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.452+0000 7f865f7fe700 1 -- 172.21.15.121:0/1198943645 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865000d600 con 0x7f8660153bb0 2023-11-15T03:07:12.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.452+0000 7f865f7fe700 1 -- 172.21.15.121:0/1198943645 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865000d980 con 0x7f8660153bb0 2023-11-15T03:07:12.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.452+0000 7f86665db700 1 -- 172.21.15.121:0/1198943645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8660153bb0 msgr2=0x7f86600a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:12.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.452+0000 7f86665db700 1 --2- 172.21.15.121:0/1198943645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8660153bb0 0x7f86600a9b30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f8650009760 tx=0x7f8650009060).stop 2023-11-15T03:07:12.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.456+0000 7f86665db700 1 -- 172.21.15.121:0/1198943645 shutdown_connections 2023-11-15T03:07:12.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.456+0000 7f86665db700 1 --2- 172.21.15.121:0/1198943645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86600aa020 0x7f8660067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:12.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.456+0000 7f86665db700 1 --2- 172.21.15.121:0/1198943645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8660153bb0 0x7f86600a9b30 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:12.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.456+0000 7f86665db700 1 -- 172.21.15.121:0/1198943645 >> 172.21.15.121:0/1198943645 conn(0x7f8660058c50 msgr2=0x7f8660059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:12.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.456+0000 7f86665db700 1 -- 172.21.15.121:0/1198943645 shutdown_connections 2023-11-15T03:07:12.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.456+0000 7f86665db700 1 -- 172.21.15.121:0/1198943645 wait complete. 2023-11-15T03:07:12.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.456+0000 7f86665db700 1 Processor -- start 2023-11-15T03:07:12.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.456+0000 7f86665db700 1 -- start start 2023-11-15T03:07:12.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f86665db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8660067f50 0x7f86601648e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:12.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f86665db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86600aa020 0x7f8660157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:12.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f86665db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8660153bb0 0x7f86600c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:12.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f86665db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86600ac600 con 0x7f86600aa020 2023-11-15T03:07:12.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f86665db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86600aacd0 con 0x7f8660153bb0 2023-11-15T03:07:12.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f86665db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86600aafb0 con 0x7f8660067f50 2023-11-15T03:07:12.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f86577fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86600aa020 0x7f8660157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:12.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f86577fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86600aa020 0x7f8660157510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53396/0 (socket says 172.21.15.121:53396) 2023-11-15T03:07:12.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f86577fe700 1 -- 172.21.15.121:0/1947975065 learned_addr learned my addr 172.21.15.121:0/1947975065 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:12.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f865ffff700 1 --2- 172.21.15.121:0/1947975065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8660067f50 0x7f86601648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:12.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f8664b78700 1 --2- 172.21.15.121:0/1947975065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8660153bb0 0x7f86600c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:12.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f86577fe700 1 -- 172.21.15.121:0/1947975065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8660067f50 msgr2=0x7f86601648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:12.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f86577fe700 1 --2- 172.21.15.121:0/1947975065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8660067f50 0x7f86601648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:12.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f86577fe700 1 -- 172.21.15.121:0/1947975065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8660153bb0 msgr2=0x7f86600c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:12.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f86577fe700 1 --2- 172.21.15.121:0/1947975065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8660153bb0 0x7f86600c3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:12.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f86577fe700 1 -- 172.21.15.121:0/1947975065 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86480092a0 con 0x7f86600aa020 2023-11-15T03:07:12.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f865ffff700 1 --2- 172.21.15.121:0/1947975065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8660067f50 0x7f86601648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:07:12.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f86577fe700 1 --2- 172.21.15.121:0/1947975065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86600aa020 0x7f8660157510 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 rx=0x7f864800ea20 tx=0x7f8648009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:12.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f865cff9700 1 -- 172.21.15.121:0/1947975065 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864800ee60 con 0x7f86600aa020 2023-11-15T03:07:12.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f865cff9700 1 -- 172.21.15.121:0/1947975065 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864800c850 con 0x7f86600aa020 2023-11-15T03:07:12.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f865cff9700 1 -- 172.21.15.121:0/1947975065 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864800cbd0 con 0x7f86600aa020 2023-11-15T03:07:12.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f86665db700 1 -- 172.21.15.121:0/1947975065 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86500092a0 con 0x7f86600aa020 2023-11-15T03:07:12.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.460+0000 7f86665db700 1 -- 172.21.15.121:0/1947975065 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86600ab570 con 0x7f86600aa020 2023-11-15T03:07:12.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.464+0000 7f86665db700 1 -- 172.21.15.121:0/1947975065 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866006c150 con 0x7f86600aa020 2023-11-15T03:07:12.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.464+0000 7f865cff9700 1 -- 172.21.15.121:0/1947975065 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f864801b400 con 0x7f86600aa020 2023-11-15T03:07:12.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.464+0000 7f865cff9700 1 --2- 172.21.15.121:0/1947975065 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f864005a510 0x7f864005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:12.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.464+0000 7f865cff9700 1 -- 172.21.15.121:0/1947975065 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8072+0+0 (secure 0 0 0) 0x7f8648075420 con 0x7f86600aa020 2023-11-15T03:07:12.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.464+0000 7f865ffff700 1 --2- 172.21.15.121:0/1947975065 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f864005a510 0x7f864005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:12.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.468+0000 7f865ffff700 1 --2- 172.21.15.121:0/1947975065 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f864005a510 0x7f864005c9d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f86600ada70 tx=0x7f8650016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:12.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.472+0000 7f865cff9700 1 -- 172.21.15.121:0/1947975065 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8648048400 con 0x7f86600aa020 2023-11-15T03:07:12.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.672+0000 7f86665db700 1 -- 172.21.15.121:0/1947975065 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86600c83c0 con 0x7f86600aa020 2023-11-15T03:07:12.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.672+0000 7f865cff9700 1 -- 172.21.15.121:0/1947975065 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 210 v210) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f864804ba20 con 0x7f86600aa020 2023-11-15T03:07:12.680 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:12.680 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":210,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":210,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:12.259975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8988},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8988":{"gid":8988,"name":"a","rank":0,"incarnation":210,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/1007300937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1007300937},{"type":"v1","addr":"172.21.15.121:6836","nonce":1007300937}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:12.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.676+0000 7f86665db700 1 -- 172.21.15.121:0/1947975065 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f864005a510 msgr2=0x7f864005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:12.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.676+0000 7f86665db700 1 --2- 172.21.15.121:0/1947975065 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f864005a510 0x7f864005c9d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f86600ada70 tx=0x7f8650016000).stop 2023-11-15T03:07:12.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.676+0000 7f86665db700 1 -- 172.21.15.121:0/1947975065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86600aa020 msgr2=0x7f8660157510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:12.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.676+0000 7f86665db700 1 --2- 172.21.15.121:0/1947975065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86600aa020 0x7f8660157510 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 rx=0x7f864800ea20 tx=0x7f8648009060).stop 2023-11-15T03:07:12.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.676+0000 7f86665db700 1 -- 172.21.15.121:0/1947975065 shutdown_connections 2023-11-15T03:07:12.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.676+0000 7f86665db700 1 --2- 172.21.15.121:0/1947975065 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f864005a510 0x7f864005c9d0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:12.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.676+0000 7f86665db700 1 --2- 172.21.15.121:0/1947975065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8660153bb0 0x7f86600c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:12.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.676+0000 7f86665db700 1 --2- 172.21.15.121:0/1947975065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86600aa020 0x7f8660157510 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:12.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.676+0000 7f86665db700 1 --2- 172.21.15.121:0/1947975065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8660067f50 0x7f86601648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:12.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.676+0000 7f86665db700 1 -- 172.21.15.121:0/1947975065 >> 172.21.15.121:0/1947975065 conn(0x7f8660058c50 msgr2=0x7f86600c8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:12.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.676+0000 7f86665db700 1 -- 172.21.15.121:0/1947975065 shutdown_connections 2023-11-15T03:07:12.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:12.676+0000 7f86665db700 1 -- 172.21.15.121:0/1947975065 wait complete. 2023-11-15T03:07:12.685 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 210 2023-11-15T03:07:12.702 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:07:13.269 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:07:13.260+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:07:13.270 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:07:13.260+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:07:13.282 INFO:tasks.ceph.mds.a.smithi121.stderr: -22> 2023-11-15T03:07:12.268+0000 7f9ce09b5700 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': virtual void InoTable::decode_state(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 2 < 101: Malformed input 2023-11-15T03:07:13.324 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:07:17.703 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:17.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.864+0000 7f67d2a9b700 1 Processor -- start 2023-11-15T03:07:17.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.864+0000 7f67d2a9b700 1 -- start start 2023-11-15T03:07:17.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.868+0000 7f67d2a9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67cc14af80 0x7f67cc0c1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:17.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.868+0000 7f67d2a9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f67cc0c1bf0 0x7f67cc150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:17.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.868+0000 7f67d2a9b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67cc152000 con 0x7f67cc1514e0 2023-11-15T03:07:17.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.868+0000 7f67d2a9b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67cc0c1270 con 0x7f67cc0c1bf0 2023-11-15T03:07:17.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.868+0000 7f67d2a9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc151a40 con 0x7f67cc14af80 2023-11-15T03:07:17.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.868+0000 7f67cbfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f67cc0c1bf0 0x7f67cc150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:17.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.868+0000 7f67cbfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f67cc0c1bf0 0x7f67cc150ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56218/0 (socket says 172.21.15.121:56218) 2023-11-15T03:07:17.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.868+0000 7f67cbfff700 1 -- 172.21.15.121:0/3589633766 learned_addr learned my addr 172.21.15.121:0/3589633766 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:17.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.868+0000 7f67d0837700 1 --2- 172.21.15.121:0/3589633766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67cc14af80 0x7f67cc0c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:17.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.868+0000 7f67cbfff700 1 -- 172.21.15.121:0/3589633766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67cc14af80 msgr2=0x7f67cc0c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:17.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.868+0000 7f67cbfff700 1 --2- 172.21.15.121:0/3589633766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67cc14af80 0x7f67cc0c1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:17.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.868+0000 7f67cbfff700 1 -- 172.21.15.121:0/3589633766 >> v1:172.21.15.190:6789/0 conn(0x7f67cc1514e0 legacy=0x7f67cc0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:17.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.868+0000 7f67cbfff700 1 -- 172.21.15.121:0/3589633766 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc151d20 con 0x7f67cc0c1bf0 2023-11-15T03:07:17.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.868+0000 7f67cb7fe700 1 -- 172.21.15.121:0/3589633766 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296662980 0 0) 0x7f67c4001490 con 0x7f67cc1514e0 2023-11-15T03:07:17.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.868+0000 7f67cbfff700 1 --2- 172.21.15.121:0/3589633766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67cc0c1bf0 0x7f67cc150ff0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7f67b8006fd0 tx=0x7f67b80090f0).ready entity=mon.0 client_cookie=ea00dcab152f0c98 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:17.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.868+0000 7f67cb7fe700 1 -- 172.21.15.121:0/3589633766 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b800b580 con 0x7f67cc0c1bf0 2023-11-15T03:07:17.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.868+0000 7f67cb7fe700 1 -- 172.21.15.121:0/3589633766 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b800b6e0 con 0x7f67cc0c1bf0 2023-11-15T03:07:17.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.868+0000 7f67cb7fe700 1 -- 172.21.15.121:0/3589633766 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b800ba60 con 0x7f67cc0c1bf0 2023-11-15T03:07:17.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.872+0000 7f67d2a9b700 1 -- 172.21.15.121:0/3589633766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67cc0c1bf0 msgr2=0x7f67cc150ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:17.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.872+0000 7f67d2a9b700 1 --2- 172.21.15.121:0/3589633766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67cc0c1bf0 0x7f67cc150ff0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7f67b8006fd0 tx=0x7f67b80090f0).stop 2023-11-15T03:07:17.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.872+0000 7f67d2a9b700 1 -- 172.21.15.121:0/3589633766 shutdown_connections 2023-11-15T03:07:17.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.872+0000 7f67d2a9b700 1 --2- 172.21.15.121:0/3589633766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67cc0c1bf0 0x7f67cc150ff0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:17.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.872+0000 7f67d2a9b700 1 --2- 172.21.15.121:0/3589633766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67cc14af80 0x7f67cc0c1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:17.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.872+0000 7f67d2a9b700 1 -- 172.21.15.121:0/3589633766 >> 172.21.15.121:0/3589633766 conn(0x7f67cc062190 msgr2=0x7f67cc0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:17.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.872+0000 7f67d2a9b700 1 -- 172.21.15.121:0/3589633766 shutdown_connections 2023-11-15T03:07:17.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.872+0000 7f67d2a9b700 1 -- 172.21.15.121:0/3589633766 wait complete. 2023-11-15T03:07:17.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.872+0000 7f67d2a9b700 1 Processor -- start 2023-11-15T03:07:17.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.872+0000 7f67d2a9b700 1 -- start start 2023-11-15T03:07:17.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67d2a9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67cc0c1bf0 0x7f67cc161d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:17.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67d2a9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67cc14af80 0x7f67cc1521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:17.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67d2a9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67cc1514e0 0x7f67cc152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:17.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67d2a9b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc1621f0 con 0x7f67cc14af80 2023-11-15T03:07:17.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67d2a9b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc152c20 con 0x7f67cc1514e0 2023-11-15T03:07:17.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67d2a9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc152f00 con 0x7f67cc0c1bf0 2023-11-15T03:07:17.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67cbfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67cc14af80 0x7f67cc1521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:17.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67cbfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67cc14af80 0x7f67cc1521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56222/0 (socket says 172.21.15.121:56222) 2023-11-15T03:07:17.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67cbfff700 1 -- 172.21.15.121:0/19202726 learned_addr learned my addr 172.21.15.121:0/19202726 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:17.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67d0837700 1 --2- 172.21.15.121:0/19202726 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67cc0c1bf0 0x7f67cc161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:17.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67d1038700 1 --2- 172.21.15.121:0/19202726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67cc1514e0 0x7f67cc152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:17.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67cbfff700 1 -- 172.21.15.121:0/19202726 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67cc0c1bf0 msgr2=0x7f67cc161d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:17.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67cbfff700 1 --2- 172.21.15.121:0/19202726 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67cc0c1bf0 0x7f67cc161d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:17.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67cbfff700 1 -- 172.21.15.121:0/19202726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67cc1514e0 msgr2=0x7f67cc152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:17.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67cbfff700 1 --2- 172.21.15.121:0/19202726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67cc1514e0 0x7f67cc152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:17.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67cbfff700 1 -- 172.21.15.121:0/19202726 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b8009370 con 0x7f67cc14af80 2023-11-15T03:07:17.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67cbfff700 1 --2- 172.21.15.121:0/19202726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67cc14af80 0x7f67cc1521a0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 rx=0x7f67b8009130 tx=0x7f67b8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:17.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67c8ff9700 1 -- 172.21.15.121:0/19202726 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8000bc0 con 0x7f67cc14af80 2023-11-15T03:07:17.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67c8ff9700 1 -- 172.21.15.121:0/19202726 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b800b850 con 0x7f67cc14af80 2023-11-15T03:07:17.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67d2a9b700 1 -- 172.21.15.121:0/19202726 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67cc1531e0 con 0x7f67cc14af80 2023-11-15T03:07:17.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67c8ff9700 1 -- 172.21.15.121:0/19202726 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8000d20 con 0x7f67cc14af80 2023-11-15T03:07:17.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.876+0000 7f67d2a9b700 1 -- 172.21.15.121:0/19202726 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67cc1649b0 con 0x7f67cc14af80 2023-11-15T03:07:17.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.880+0000 7f67d2a9b700 1 -- 172.21.15.121:0/19202726 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67cc06c150 con 0x7f67cc14af80 2023-11-15T03:07:17.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.880+0000 7f67c8ff9700 1 -- 172.21.15.121:0/19202726 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f67b801cad0 con 0x7f67cc14af80 2023-11-15T03:07:17.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.880+0000 7f67c8ff9700 1 --2- 172.21.15.121:0/19202726 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f67b005a510 0x7f67b005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:17.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.880+0000 7f67c8ff9700 1 -- 172.21.15.121:0/19202726 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8072+0+0 (secure 0 0 0) 0x7f67b8076d60 con 0x7f67cc14af80 2023-11-15T03:07:17.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.880+0000 7f67d0837700 1 --2- 172.21.15.121:0/19202726 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f67b005a510 0x7f67b005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:17.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.880+0000 7f67d0837700 1 --2- 172.21.15.121:0/19202726 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f67b005a510 0x7f67b005c9d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f67cc0c2fd0 tx=0x7f67c0009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:17.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:17.888+0000 7f67c8ff9700 1 -- 172.21.15.121:0/19202726 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f67b8049da0 con 0x7f67cc14af80 2023-11-15T03:07:18.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.084+0000 7f67d2a9b700 1 -- 172.21.15.121:0/19202726 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67cc0c0f00 con 0x7f67cc14af80 2023-11-15T03:07:18.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.088+0000 7f67c8ff9700 1 -- 172.21.15.121:0/19202726 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 212 v212) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f67b802a030 con 0x7f67cc14af80 2023-11-15T03:07:18.095 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:18.095 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":212,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2131958479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2131958479},{"type":"v1","addr":"172.21.15.121:6836","nonce":2131958479}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":212}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:13.265231+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:18.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.088+0000 7f67d2a9b700 1 -- 172.21.15.121:0/19202726 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f67b005a510 msgr2=0x7f67b005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:18.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.088+0000 7f67d2a9b700 1 --2- 172.21.15.121:0/19202726 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f67b005a510 0x7f67b005c9d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f67cc0c2fd0 tx=0x7f67c0009060).stop 2023-11-15T03:07:18.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.088+0000 7f67d2a9b700 1 -- 172.21.15.121:0/19202726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67cc14af80 msgr2=0x7f67cc1521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:18.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.088+0000 7f67d2a9b700 1 --2- 172.21.15.121:0/19202726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67cc14af80 0x7f67cc1521a0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 rx=0x7f67b8009130 tx=0x7f67b8017000).stop 2023-11-15T03:07:18.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.088+0000 7f67d2a9b700 1 -- 172.21.15.121:0/19202726 shutdown_connections 2023-11-15T03:07:18.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.088+0000 7f67d2a9b700 1 --2- 172.21.15.121:0/19202726 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f67b005a510 0x7f67b005c9d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.088+0000 7f67d2a9b700 1 --2- 172.21.15.121:0/19202726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67cc1514e0 0x7f67cc152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.088+0000 7f67d2a9b700 1 --2- 172.21.15.121:0/19202726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67cc14af80 0x7f67cc1521a0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.088+0000 7f67d2a9b700 1 --2- 172.21.15.121:0/19202726 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67cc0c1bf0 0x7f67cc161d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.088+0000 7f67d2a9b700 1 -- 172.21.15.121:0/19202726 >> 172.21.15.121:0/19202726 conn(0x7f67cc062190 msgr2=0x7f67cc0c2820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:18.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.088+0000 7f67d2a9b700 1 -- 172.21.15.121:0/19202726 shutdown_connections 2023-11-15T03:07:18.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.088+0000 7f67d2a9b700 1 -- 172.21.15.121:0/19202726 wait complete. 2023-11-15T03:07:18.100 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 212 2023-11-15T03:07:18.119 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:07:18.120 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:18.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.276+0000 7f4a51879700 1 Processor -- start 2023-11-15T03:07:18.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.276+0000 7f4a51879700 1 -- start start 2023-11-15T03:07:18.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.280+0000 7f4a51879700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4a4c0c3770 0x7f4a4c0c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:18.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.280+0000 7f4a51879700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a4c0bdd70 con 0x7f4a4c0a7c20 2023-11-15T03:07:18.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.280+0000 7f4a51879700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a4c1529f0 con 0x7f4a4c0be200 2023-11-15T03:07:18.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.280+0000 7f4a51879700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a4c152cd0 con 0x7f4a4c0c3770 2023-11-15T03:07:18.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.280+0000 7f4a50877700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4a4c0a7c20 0x7f4a4c151e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59170/0 (socket says 172.21.15.121:59170) 2023-11-15T03:07:18.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.280+0000 7f4a50877700 1 -- 172.21.15.121:0/764337924 learned_addr learned my addr 172.21.15.121:0/764337924 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:18.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.280+0000 7f4a43fff700 1 --2- 172.21.15.121:0/764337924 >> v2:172.21.15.121:3300/0 conn(0x7f4a4c0c3770 0x7f4a4c0c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:18.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.280+0000 7f4a437fe700 1 -- 172.21.15.121:0/764337924 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228039155 0 0) 0x7f4a4c0bdd70 con 0x7f4a4c0a7c20 2023-11-15T03:07:18.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.280+0000 7f4a437fe700 1 -- 172.21.15.121:0/764337924 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a340033f0 con 0x7f4a4c0a7c20 2023-11-15T03:07:18.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.280+0000 7f4a437fe700 1 -- 172.21.15.121:0/764337924 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1710948265 0 0) 0x7f4a4c1529f0 con 0x7f4a4c0be200 2023-11-15T03:07:18.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.280+0000 7f4a43fff700 1 -- 172.21.15.121:0/764337924 >> v1:172.21.15.190:6789/0 conn(0x7f4a4c0be200 legacy=0x7f4a4c0be690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:18.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.280+0000 7f4a43fff700 1 -- 172.21.15.121:0/764337924 >> v1:172.21.15.121:6789/0 conn(0x7f4a4c0a7c20 legacy=0x7f4a4c151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:18.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.280+0000 7f4a43fff700 1 -- 172.21.15.121:0/764337924 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c0bd720 con 0x7f4a4c0c3770 2023-11-15T03:07:18.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.284+0000 7f4a43fff700 1 --2- 172.21.15.121:0/764337924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a4c0c3770 0x7f4a4c0c3bd0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 rx=0x7f4a48009e60 tx=0x7f4a48009780).ready entity=mon.0 client_cookie=ef5d40b769166335 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:18.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.284+0000 7f4a437fe700 1 -- 172.21.15.121:0/764337924 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4800e070 con 0x7f4a4c0c3770 2023-11-15T03:07:18.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.284+0000 7f4a437fe700 1 -- 172.21.15.121:0/764337924 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4800bb60 con 0x7f4a4c0c3770 2023-11-15T03:07:18.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.284+0000 7f4a437fe700 1 -- 172.21.15.121:0/764337924 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4800bee0 con 0x7f4a4c0c3770 2023-11-15T03:07:18.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.284+0000 7f4a51879700 1 -- 172.21.15.121:0/764337924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a4c0c3770 msgr2=0x7f4a4c0c3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:18.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.284+0000 7f4a51879700 1 --2- 172.21.15.121:0/764337924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a4c0c3770 0x7f4a4c0c3bd0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 rx=0x7f4a48009e60 tx=0x7f4a48009780).stop 2023-11-15T03:07:18.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.284+0000 7f4a51879700 1 -- 172.21.15.121:0/764337924 shutdown_connections 2023-11-15T03:07:18.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.284+0000 7f4a51879700 1 --2- 172.21.15.121:0/764337924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a4c0c3770 0x7f4a4c0c3bd0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.284+0000 7f4a51879700 1 -- 172.21.15.121:0/764337924 >> 172.21.15.121:0/764337924 conn(0x7f4a4c068530 msgr2=0x7f4a4c0c87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:18.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.284+0000 7f4a51879700 1 -- 172.21.15.121:0/764337924 shutdown_connections 2023-11-15T03:07:18.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.284+0000 7f4a51879700 1 -- 172.21.15.121:0/764337924 wait complete. 2023-11-15T03:07:18.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a51879700 1 Processor -- start 2023-11-15T03:07:18.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a51879700 1 -- start start 2023-11-15T03:07:18.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a51879700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a4c0a7c20 0x7f4a4c151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:18.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a51879700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a4c0be200 0x7f4a4c0ab770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:18.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a51879700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4c0c3770 0x7f4a4c150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:18.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a51879700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c151830 con 0x7f4a4c0a7c20 2023-11-15T03:07:18.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a51879700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c0abe00 con 0x7f4a4c0be200 2023-11-15T03:07:18.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a51879700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c0ac0e0 con 0x7f4a4c0c3770 2023-11-15T03:07:18.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a50877700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a4c0a7c20 0x7f4a4c151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:18.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a50877700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a4c0a7c20 0x7f4a4c151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56244/0 (socket says 172.21.15.121:56244) 2023-11-15T03:07:18.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a50877700 1 -- 172.21.15.121:0/1136995796 learned_addr learned my addr 172.21.15.121:0/1136995796 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:18.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a43fff700 1 --2- 172.21.15.121:0/1136995796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a4c0be200 0x7f4a4c0ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:18.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a51078700 1 --2- 172.21.15.121:0/1136995796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4c0c3770 0x7f4a4c150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:18.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a50877700 1 -- 172.21.15.121:0/1136995796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4c0c3770 msgr2=0x7f4a4c150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:18.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a50877700 1 --2- 172.21.15.121:0/1136995796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4c0c3770 0x7f4a4c150770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a50877700 1 -- 172.21.15.121:0/1136995796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a4c0be200 msgr2=0x7f4a4c0ab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:18.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a50877700 1 --2- 172.21.15.121:0/1136995796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a4c0be200 0x7f4a4c0ab770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a50877700 1 -- 172.21.15.121:0/1136995796 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a48009a20 con 0x7f4a4c0a7c20 2023-11-15T03:07:18.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a50877700 1 --2- 172.21.15.121:0/1136995796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a4c0a7c20 0x7f4a4c151020 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 rx=0x7f4a3c001ed0 tx=0x7f4a3c007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:18.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a40ff9700 1 -- 172.21.15.121:0/1136995796 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c0072e0 con 0x7f4a4c0a7c20 2023-11-15T03:07:18.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a40ff9700 1 -- 172.21.15.121:0/1136995796 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3c007440 con 0x7f4a4c0a7c20 2023-11-15T03:07:18.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a51879700 1 -- 172.21.15.121:0/1136995796 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4c0ac3c0 con 0x7f4a4c0a7c20 2023-11-15T03:07:18.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a40ff9700 1 -- 172.21.15.121:0/1136995796 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c0057a0 con 0x7f4a4c0a7c20 2023-11-15T03:07:18.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.288+0000 7f4a51879700 1 -- 172.21.15.121:0/1136995796 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c0ac980 con 0x7f4a4c0a7c20 2023-11-15T03:07:18.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.292+0000 7f4a51879700 1 -- 172.21.15.121:0/1136995796 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4c06bcf0 con 0x7f4a4c0a7c20 2023-11-15T03:07:18.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.292+0000 7f4a40ff9700 1 -- 172.21.15.121:0/1136995796 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4a3c005b50 con 0x7f4a4c0a7c20 2023-11-15T03:07:18.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.292+0000 7f4a40ff9700 1 --2- 172.21.15.121:0/1136995796 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a3805a510 0x7f4a3805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:18.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.292+0000 7f4a40ff9700 1 -- 172.21.15.121:0/1136995796 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8072+0+0 (secure 0 0 0) 0x7f4a3c074610 con 0x7f4a4c0a7c20 2023-11-15T03:07:18.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.292+0000 7f4a43fff700 1 --2- 172.21.15.121:0/1136995796 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a3805a510 0x7f4a3805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:18.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.296+0000 7f4a43fff700 1 --2- 172.21.15.121:0/1136995796 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a3805a510 0x7f4a3805c9d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f4a480094d0 tx=0x7f4a48009780).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:18.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.300+0000 7f4a40ff9700 1 -- 172.21.15.121:0/1136995796 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4a3c047650 con 0x7f4a4c0a7c20 2023-11-15T03:07:18.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.496+0000 7f4a51879700 1 -- 172.21.15.121:0/1136995796 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a4c064fe0 con 0x7f4a4c0a7c20 2023-11-15T03:07:18.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.496+0000 7f4a40ff9700 1 -- 172.21.15.121:0/1136995796 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 212 v212) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f4a3c04ac70 con 0x7f4a4c0a7c20 2023-11-15T03:07:18.505 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:18.505 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":212,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2131958479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2131958479},{"type":"v1","addr":"172.21.15.121:6836","nonce":2131958479}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":212}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:13.265231+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:18.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.496+0000 7f4a51879700 1 -- 172.21.15.121:0/1136995796 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a3805a510 msgr2=0x7f4a3805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:18.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.496+0000 7f4a51879700 1 --2- 172.21.15.121:0/1136995796 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a3805a510 0x7f4a3805c9d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f4a480094d0 tx=0x7f4a48009780).stop 2023-11-15T03:07:18.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.500+0000 7f4a51879700 1 -- 172.21.15.121:0/1136995796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a4c0a7c20 msgr2=0x7f4a4c151020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:18.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.500+0000 7f4a51879700 1 --2- 172.21.15.121:0/1136995796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a4c0a7c20 0x7f4a4c151020 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 rx=0x7f4a3c001ed0 tx=0x7f4a3c007c70).stop 2023-11-15T03:07:18.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.500+0000 7f4a51879700 1 -- 172.21.15.121:0/1136995796 shutdown_connections 2023-11-15T03:07:18.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.500+0000 7f4a51879700 1 --2- 172.21.15.121:0/1136995796 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4a3805a510 0x7f4a3805c9d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.500+0000 7f4a51879700 1 --2- 172.21.15.121:0/1136995796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4c0c3770 0x7f4a4c150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.500+0000 7f4a51879700 1 --2- 172.21.15.121:0/1136995796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a4c0be200 0x7f4a4c0ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.500+0000 7f4a51879700 1 --2- 172.21.15.121:0/1136995796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a4c0a7c20 0x7f4a4c151020 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.500+0000 7f4a51879700 1 -- 172.21.15.121:0/1136995796 >> 172.21.15.121:0/1136995796 conn(0x7f4a4c068530 msgr2=0x7f4a4c155460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:18.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.500+0000 7f4a51879700 1 -- 172.21.15.121:0/1136995796 shutdown_connections 2023-11-15T03:07:18.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.500+0000 7f4a51879700 1 -- 172.21.15.121:0/1136995796 wait complete. 2023-11-15T03:07:18.510 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 212 2023-11-15T03:07:18.525 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:07:18.526 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:18.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.676+0000 7f807cb69700 1 Processor -- start 2023-11-15T03:07:18.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.676+0000 7f807cb69700 1 -- start start 2023-11-15T03:07:18.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f807cb69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80781562f0 0x7f8078155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:18.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f807cb69700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f80780a57c0 0x7f80780a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:18.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f807cb69700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80780c4c30 con 0x7f8078154ca0 2023-11-15T03:07:18.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f807cb69700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80780a8280 con 0x7f80780a57c0 2023-11-15T03:07:18.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f807cb69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80780a6d40 con 0x7f80781562f0 2023-11-15T03:07:18.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f8076d9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f80780a57c0 0x7f80780a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:18.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f8076d9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f80780a57c0 0x7f80780a5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56260/0 (socket says 172.21.15.121:56260) 2023-11-15T03:07:18.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f8076d9d700 1 -- 172.21.15.121:0/2708985941 learned_addr learned my addr 172.21.15.121:0/2708985941 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:18.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f806dd9b700 1 --2- 172.21.15.121:0/2708985941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80781562f0 0x7f8078155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:18.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f8076d9d700 1 -- 172.21.15.121:0/2708985941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80781562f0 msgr2=0x7f8078155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:18.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f8076d9d700 1 --2- 172.21.15.121:0/2708985941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80781562f0 0x7f8078155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f8076d9d700 1 -- 172.21.15.121:0/2708985941 >> v1:172.21.15.190:6789/0 conn(0x7f8078154ca0 legacy=0x7f80780a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:18.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f8076d9d700 1 -- 172.21.15.121:0/2708985941 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80780a6140 con 0x7f80780a57c0 2023-11-15T03:07:18.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f8076d9d700 1 --2- 172.21.15.121:0/2708985941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80780a57c0 0x7f80780a5c50 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 rx=0x7f8070009f20 tx=0x7f8070009840).ready entity=mon.0 client_cookie=813e54dfc1753738 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:18.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f8075d9b700 1 -- 172.21.15.121:0/2708985941 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807000e070 con 0x7f80780a57c0 2023-11-15T03:07:18.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f8075d9b700 1 -- 172.21.15.121:0/2708985941 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807000ba40 con 0x7f80780a57c0 2023-11-15T03:07:18.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f8075d9b700 1 -- 172.21.15.121:0/2708985941 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807000bdc0 con 0x7f80780a57c0 2023-11-15T03:07:18.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f807cb69700 1 -- 172.21.15.121:0/2708985941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80780a57c0 msgr2=0x7f80780a5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:18.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f807cb69700 1 --2- 172.21.15.121:0/2708985941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80780a57c0 0x7f80780a5c50 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 rx=0x7f8070009f20 tx=0x7f8070009840).stop 2023-11-15T03:07:18.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f807cb69700 1 -- 172.21.15.121:0/2708985941 shutdown_connections 2023-11-15T03:07:18.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f807cb69700 1 --2- 172.21.15.121:0/2708985941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80780a57c0 0x7f80780a5c50 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f807cb69700 1 --2- 172.21.15.121:0/2708985941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80781562f0 0x7f8078155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f807cb69700 1 -- 172.21.15.121:0/2708985941 >> 172.21.15.121:0/2708985941 conn(0x7f8078062190 msgr2=0x7f80780625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:18.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f807cb69700 1 -- 172.21.15.121:0/2708985941 shutdown_connections 2023-11-15T03:07:18.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.680+0000 7f807cb69700 1 -- 172.21.15.121:0/2708985941 wait complete. 2023-11-15T03:07:18.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.684+0000 7f807cb69700 1 Processor -- start 2023-11-15T03:07:18.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.684+0000 7f807cb69700 1 -- start start 2023-11-15T03:07:18.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.684+0000 7f807cb69700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80780a57c0 0x7f8078151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:18.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.684+0000 7f807cb69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8078154ca0 0x7f80780a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:18.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.684+0000 7f807cb69700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80781562f0 0x7f8078066430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:18.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.684+0000 7f807cb69700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80780ab9f0 con 0x7f80781562f0 2023-11-15T03:07:18.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.684+0000 7f807cb69700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8078152990 con 0x7f80780a57c0 2023-11-15T03:07:18.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.684+0000 7f807cb69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80780aa100 con 0x7f8078154ca0 2023-11-15T03:07:18.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.688+0000 7f8076d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80781562f0 0x7f8078066430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:18.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.688+0000 7f8076d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80781562f0 0x7f8078066430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56266/0 (socket says 172.21.15.121:56266) 2023-11-15T03:07:18.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.688+0000 7f8076d9d700 1 -- 172.21.15.121:0/413703629 learned_addr learned my addr 172.21.15.121:0/413703629 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:18.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.688+0000 7f807659c700 1 --2- 172.21.15.121:0/413703629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80780a57c0 0x7f8078151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:18.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.688+0000 7f806dd9b700 1 --2- 172.21.15.121:0/413703629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8078154ca0 0x7f80780a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:18.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.688+0000 7f8076d9d700 1 -- 172.21.15.121:0/413703629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8078154ca0 msgr2=0x7f80780a9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:18.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.688+0000 7f8076d9d700 1 --2- 172.21.15.121:0/413703629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8078154ca0 0x7f80780a9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.688+0000 7f8076d9d700 1 -- 172.21.15.121:0/413703629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80780a57c0 msgr2=0x7f8078151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:18.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.688+0000 7f8076d9d700 1 --2- 172.21.15.121:0/413703629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80780a57c0 0x7f8078151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.688+0000 7f8076d9d700 1 -- 172.21.15.121:0/413703629 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8070009ae0 con 0x7f80781562f0 2023-11-15T03:07:18.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.688+0000 7f8076d9d700 1 --2- 172.21.15.121:0/413703629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80781562f0 0x7f8078066430 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 rx=0x7f8078063d40 tx=0x7f80780c4ba0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:18.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.688+0000 7f806effd700 1 -- 172.21.15.121:0/413703629 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80700092e0 con 0x7f80781562f0 2023-11-15T03:07:18.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.688+0000 7f806effd700 1 -- 172.21.15.121:0/413703629 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807001e470 con 0x7f80781562f0 2023-11-15T03:07:18.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.688+0000 7f806effd700 1 -- 172.21.15.121:0/413703629 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807001e830 con 0x7f80781562f0 2023-11-15T03:07:18.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.688+0000 7f807cb69700 1 -- 172.21.15.121:0/413703629 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80780aae70 con 0x7f80781562f0 2023-11-15T03:07:18.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.688+0000 7f807cb69700 1 -- 172.21.15.121:0/413703629 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80780ab430 con 0x7f80781562f0 2023-11-15T03:07:18.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.688+0000 7f807cb69700 1 -- 172.21.15.121:0/413703629 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8078152b90 con 0x7f80781562f0 2023-11-15T03:07:18.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.692+0000 7f806effd700 1 -- 172.21.15.121:0/413703629 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8070003040 con 0x7f80781562f0 2023-11-15T03:07:18.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.692+0000 7f806effd700 1 --2- 172.21.15.121:0/413703629 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f805c05a510 0x7f805c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:18.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.692+0000 7f806effd700 1 -- 172.21.15.121:0/413703629 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8072+0+0 (secure 0 0 0) 0x7f8070076660 con 0x7f80781562f0 2023-11-15T03:07:18.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.692+0000 7f807659c700 1 --2- 172.21.15.121:0/413703629 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f805c05a510 0x7f805c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:18.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.696+0000 7f807659c700 1 --2- 172.21.15.121:0/413703629 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f805c05a510 0x7f805c05c9d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f8068003900 tx=0x7f8068007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:18.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.700+0000 7f806effd700 1 -- 172.21.15.121:0/413703629 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8078152b90 con 0x7f80781562f0 2023-11-15T03:07:18.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.896+0000 7f807cb69700 1 -- 172.21.15.121:0/413703629 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f807806c150 con 0x7f80781562f0 2023-11-15T03:07:18.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.896+0000 7f806effd700 1 -- 172.21.15.121:0/413703629 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 212 v212) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f8070023020 con 0x7f80781562f0 2023-11-15T03:07:18.903 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:18.904 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":212,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2131958479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2131958479},{"type":"v1","addr":"172.21.15.121:6836","nonce":2131958479}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":212}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:13.265231+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:18.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.896+0000 7f807cb69700 1 -- 172.21.15.121:0/413703629 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f805c05a510 msgr2=0x7f805c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:18.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.896+0000 7f807cb69700 1 --2- 172.21.15.121:0/413703629 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f805c05a510 0x7f805c05c9d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f8068003900 tx=0x7f8068007d40).stop 2023-11-15T03:07:18.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.896+0000 7f807cb69700 1 -- 172.21.15.121:0/413703629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80781562f0 msgr2=0x7f8078066430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:18.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.896+0000 7f807cb69700 1 --2- 172.21.15.121:0/413703629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80781562f0 0x7f8078066430 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 rx=0x7f8078063d40 tx=0x7f80780c4ba0).stop 2023-11-15T03:07:18.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.900+0000 7f807cb69700 1 -- 172.21.15.121:0/413703629 shutdown_connections 2023-11-15T03:07:18.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.900+0000 7f807cb69700 1 --2- 172.21.15.121:0/413703629 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f805c05a510 0x7f805c05c9d0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.900+0000 7f807cb69700 1 --2- 172.21.15.121:0/413703629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80781562f0 0x7f8078066430 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.900+0000 7f807cb69700 1 --2- 172.21.15.121:0/413703629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8078154ca0 0x7f80780a9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.900+0000 7f807cb69700 1 --2- 172.21.15.121:0/413703629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80780a57c0 0x7f8078151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:18.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.900+0000 7f807cb69700 1 -- 172.21.15.121:0/413703629 >> 172.21.15.121:0/413703629 conn(0x7f8078062190 msgr2=0x7f80780c2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:18.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.900+0000 7f807cb69700 1 -- 172.21.15.121:0/413703629 shutdown_connections 2023-11-15T03:07:18.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:18.900+0000 7f807cb69700 1 -- 172.21.15.121:0/413703629 wait complete. 2023-11-15T03:07:18.909 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 212 2023-11-15T03:07:18.925 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt mds0_inotable' led to DAMAGED state 2023-11-15T03:07:18.925 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 400.00000000' 2023-11-15T03:07:18.926 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:07:18.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:07:18.926 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:07:18.947 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:07:18.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:07:18.949 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:07:19.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.140+0000 7fe8c0f07700 1 Processor -- start 2023-11-15T03:07:19.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.140+0000 7fe8c0f07700 1 -- start start 2023-11-15T03:07:19.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.144+0000 7fe8c0f07700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe8bc0a67b0 0x7fe8bc0c2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:19.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.144+0000 7fe8c0f07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe8bc0bd1f0 0x7fe8bc0a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:19.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.144+0000 7fe8c0f07700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8bc0c2b40 con 0x7fe8bc0c46d0 2023-11-15T03:07:19.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.144+0000 7fe8c0f07700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8bc1586a0 con 0x7fe8bc0a67b0 2023-11-15T03:07:19.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.144+0000 7fe8c0f07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8bc0c2210 con 0x7fe8bc0bd1f0 2023-11-15T03:07:19.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.144+0000 7fe8bb7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe8bc0a67b0 0x7fe8bc0c2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:19.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.144+0000 7fe8bbfff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe8bc0c46d0 0x7fe8bc067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59184/0 (socket says 172.21.15.121:59184) 2023-11-15T03:07:19.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.144+0000 7fe8bbfff700 1 -- 172.21.15.121:0/1354210625 learned_addr learned my addr 172.21.15.121:0/1354210625 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:19.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.144+0000 7fe8baffd700 1 --2- 172.21.15.121:0/1354210625 >> v2:172.21.15.190:3300/0 conn(0x7fe8bc0bd1f0 0x7fe8bc0a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:19.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.144+0000 7fe8bb7fe700 1 -- 172.21.15.121:0/1354210625 >> v2:172.21.15.190:3300/0 conn(0x7fe8bc0bd1f0 msgr2=0x7fe8bc0a6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:19.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.144+0000 7fe8bb7fe700 1 --2- 172.21.15.121:0/1354210625 >> v2:172.21.15.190:3300/0 conn(0x7fe8bc0bd1f0 0x7fe8bc0a6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:19.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.144+0000 7fe8bb7fe700 1 -- 172.21.15.121:0/1354210625 >> v1:172.21.15.121:6789/0 conn(0x7fe8bc0c46d0 legacy=0x7fe8bc067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:19.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.144+0000 7fe8bb7fe700 1 -- 172.21.15.121:0/1354210625 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc0c24f0 con 0x7fe8bc0a67b0 2023-11-15T03:07:19.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.144+0000 7fe8bb7fe700 1 --2- 172.21.15.121:0/1354210625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8bc0a67b0 0x7fe8bc0c2fd0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 rx=0x7fe8bc0c75f0 tx=0x7fe8ac009060).ready entity=mon.0 client_cookie=884d6e5405d937a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:19.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.148+0000 7fe8ba7fc700 1 -- 172.21.15.121:0/1354210625 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac00f510 con 0x7fe8bc0a67b0 2023-11-15T03:07:19.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.148+0000 7fe8ba7fc700 1 -- 172.21.15.121:0/1354210625 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ac00f670 con 0x7fe8bc0a67b0 2023-11-15T03:07:19.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.148+0000 7fe8ba7fc700 1 -- 172.21.15.121:0/1354210625 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac00f9f0 con 0x7fe8bc0a67b0 2023-11-15T03:07:19.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.148+0000 7fe8c0f07700 1 -- 172.21.15.121:0/1354210625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8bc0a67b0 msgr2=0x7fe8bc0c2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:19.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.148+0000 7fe8c0f07700 1 --2- 172.21.15.121:0/1354210625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8bc0a67b0 0x7fe8bc0c2fd0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 rx=0x7fe8bc0c75f0 tx=0x7fe8ac009060).stop 2023-11-15T03:07:19.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.148+0000 7fe8c0f07700 1 -- 172.21.15.121:0/1354210625 shutdown_connections 2023-11-15T03:07:19.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.148+0000 7fe8c0f07700 1 --2- 172.21.15.121:0/1354210625 >> v2:172.21.15.190:3300/0 conn(0x7fe8bc0bd1f0 0x7fe8bc0a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:19.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.148+0000 7fe8c0f07700 1 --2- 172.21.15.121:0/1354210625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8bc0a67b0 0x7fe8bc0c2fd0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:19.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.148+0000 7fe8c0f07700 1 -- 172.21.15.121:0/1354210625 >> 172.21.15.121:0/1354210625 conn(0x7fe8bc069640 msgr2=0x7fe8bc066860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:19.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.148+0000 7fe8c0f07700 1 -- 172.21.15.121:0/1354210625 shutdown_connections 2023-11-15T03:07:19.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.148+0000 7fe8c0f07700 1 -- 172.21.15.121:0/1354210625 wait complete. 2023-11-15T03:07:19.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.152+0000 7fe8c0f07700 1 Processor -- start 2023-11-15T03:07:19.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.152+0000 7fe8c0f07700 1 -- start start 2023-11-15T03:07:19.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.152+0000 7fe8c0f07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8bc0a67b0 0x7fe8bc0a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:19.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.152+0000 7fe8c0f07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8bc0bd1f0 0x7fe8bc0c0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:19.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.152+0000 7fe8c0f07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8bc0c46d0 0x7fe8bc16c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:19.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.152+0000 7fe8c0f07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0a9980 con 0x7fe8bc0c46d0 2023-11-15T03:07:19.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.152+0000 7fe8c0f07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0a8d70 con 0x7fe8bc0a67b0 2023-11-15T03:07:19.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.152+0000 7fe8c0f07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0a9050 con 0x7fe8bc0bd1f0 2023-11-15T03:07:19.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.152+0000 7fe8bbfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8bc0c46d0 0x7fe8bc16c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:19.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.152+0000 7fe8bb7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8bc0a67b0 0x7fe8bc0a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:19.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.152+0000 7fe8bb7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8bc0a67b0 0x7fe8bc0a9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:60108/0 (socket says 172.21.15.121:60108) 2023-11-15T03:07:19.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.152+0000 7fe8bb7fe700 1 -- 172.21.15.121:0/240199642 learned_addr learned my addr 172.21.15.121:0/240199642 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:19.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.152+0000 7fe8baffd700 1 --2- 172.21.15.121:0/240199642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8bc0bd1f0 0x7fe8bc0c0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:19.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.152+0000 7fe8bb7fe700 1 -- 172.21.15.121:0/240199642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8bc0bd1f0 msgr2=0x7fe8bc0c0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:19.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.152+0000 7fe8bb7fe700 1 --2- 172.21.15.121:0/240199642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8bc0bd1f0 0x7fe8bc0c0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:19.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.152+0000 7fe8bb7fe700 1 -- 172.21.15.121:0/240199642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8bc0c46d0 msgr2=0x7fe8bc16c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:19.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.152+0000 7fe8bb7fe700 1 --2- 172.21.15.121:0/240199642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8bc0c46d0 0x7fe8bc16c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:19.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.152+0000 7fe8bb7fe700 1 -- 172.21.15.121:0/240199642 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac0092a0 con 0x7fe8bc0a67b0 2023-11-15T03:07:19.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.152+0000 7fe8bbfff700 1 --2- 172.21.15.121:0/240199642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8bc0c46d0 0x7fe8bc16c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:07:19.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.156+0000 7fe8bb7fe700 1 --2- 172.21.15.121:0/240199642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8bc0a67b0 0x7fe8bc0a9e10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fe8ac009dd0 tx=0x7fe8ac017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:19.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.156+0000 7fe8a37fe700 1 -- 172.21.15.121:0/240199642 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac023480 con 0x7fe8bc0a67b0 2023-11-15T03:07:19.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.156+0000 7fe8a37fe700 1 -- 172.21.15.121:0/240199642 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ac01e070 con 0x7fe8bc0a67b0 2023-11-15T03:07:19.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.156+0000 7fe8c0f07700 1 -- 172.21.15.121:0/240199642 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8bc0a9330 con 0x7fe8bc0a67b0 2023-11-15T03:07:19.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.156+0000 7fe8a37fe700 1 -- 172.21.15.121:0/240199642 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac023830 con 0x7fe8bc0a67b0 2023-11-15T03:07:19.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.156+0000 7fe8c0f07700 1 -- 172.21.15.121:0/240199642 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8bc0c0fc0 con 0x7fe8bc0a67b0 2023-11-15T03:07:19.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.156+0000 7fe8c0f07700 1 -- 172.21.15.121:0/240199642 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8bc06bcf0 con 0x7fe8bc0a67b0 2023-11-15T03:07:19.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.156+0000 7fe8a37fe700 1 -- 172.21.15.121:0/240199642 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe8ac020030 con 0x7fe8bc0a67b0 2023-11-15T03:07:19.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.160+0000 7fe8a37fe700 1 --2- 172.21.15.121:0/240199642 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe8a805a510 0x7fe8a805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:19.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.160+0000 7fe8a37fe700 1 -- 172.21.15.121:0/240199642 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8072+0+0 (secure 0 0 0) 0x7fe8ac075f50 con 0x7fe8bc0a67b0 2023-11-15T03:07:19.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.160+0000 7fe8baffd700 1 --2- 172.21.15.121:0/240199642 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe8a805a510 0x7fe8a805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:19.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.160+0000 7fe8baffd700 1 --2- 172.21.15.121:0/240199642 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe8a805a510 0x7fe8a805c9d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7fe8bc15cdc0 tx=0x7fe8b0009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:19.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.164+0000 7fe8a37fe700 1 -- 172.21.15.121:0/240199642 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe8ac048ed0 con 0x7fe8bc0a67b0 2023-11-15T03:07:19.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:19.360+0000 7fe8c0f07700 1 -- 172.21.15.121:0/240199642 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe8bc0c27d0 con 0x7fe8bc0a67b0 2023-11-15T03:07:20.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.168+0000 7fe8a37fe700 1 -- 172.21.15.121:0/240199642 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v213) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe8ac04c4f0 con 0x7fe8bc0a67b0 2023-11-15T03:07:20.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.172+0000 7fe8c0f07700 1 -- 172.21.15.121:0/240199642 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe8a805a510 msgr2=0x7fe8a805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:20.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.172+0000 7fe8c0f07700 1 --2- 172.21.15.121:0/240199642 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe8a805a510 0x7fe8a805c9d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7fe8bc15cdc0 tx=0x7fe8b0009060).stop 2023-11-15T03:07:20.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.172+0000 7fe8c0f07700 1 -- 172.21.15.121:0/240199642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8bc0a67b0 msgr2=0x7fe8bc0a9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:20.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.172+0000 7fe8c0f07700 1 --2- 172.21.15.121:0/240199642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8bc0a67b0 0x7fe8bc0a9e10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fe8ac009dd0 tx=0x7fe8ac017000).stop 2023-11-15T03:07:20.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.172+0000 7fe8c0f07700 1 -- 172.21.15.121:0/240199642 shutdown_connections 2023-11-15T03:07:20.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.172+0000 7fe8c0f07700 1 --2- 172.21.15.121:0/240199642 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe8a805a510 0x7fe8a805c9d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:20.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.172+0000 7fe8c0f07700 1 --2- 172.21.15.121:0/240199642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8bc0c46d0 0x7fe8bc16c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:20.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.172+0000 7fe8c0f07700 1 --2- 172.21.15.121:0/240199642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8bc0bd1f0 0x7fe8bc0c0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:20.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.172+0000 7fe8c0f07700 1 --2- 172.21.15.121:0/240199642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8bc0a67b0 0x7fe8bc0a9e10 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:20.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.172+0000 7fe8c0f07700 1 -- 172.21.15.121:0/240199642 >> 172.21.15.121:0/240199642 conn(0x7fe8bc069640 msgr2=0x7fe8bc0c61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:20.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.172+0000 7fe8c0f07700 1 -- 172.21.15.121:0/240199642 shutdown_connections 2023-11-15T03:07:20.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.172+0000 7fe8c0f07700 1 -- 172.21.15.121:0/240199642 wait complete. 2023-11-15T03:07:20.183 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:07:20.202 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:07:20.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.364+0000 7f18c549b700 1 Processor -- start 2023-11-15T03:07:20.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.364+0000 7f18c549b700 1 -- start start 2023-11-15T03:07:20.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.368+0000 7f18c549b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f18c00c04d0 0x7f18c00c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:20.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.368+0000 7f18c549b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f18c0153670 0x7f18c0153b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:20.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.368+0000 7f18c549b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c00c5720 con 0x7f18c00c8fb0 2023-11-15T03:07:20.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.368+0000 7f18c549b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c00c0da0 con 0x7f18c00c04d0 2023-11-15T03:07:20.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.368+0000 7f18c549b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c00ca8a0 con 0x7f18c0153670 2023-11-15T03:07:20.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.368+0000 7f18beffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f18c00c04d0 0x7f18c00c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:20.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.368+0000 7f18bf7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f18c00c8fb0 0x7f18c00c9440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59196/0 (socket says 172.21.15.121:59196) 2023-11-15T03:07:20.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.368+0000 7f18bf7fe700 1 -- 172.21.15.121:0/3948330685 learned_addr learned my addr 172.21.15.121:0/3948330685 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:20.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.368+0000 7f18b7fff700 1 --2- 172.21.15.121:0/3948330685 >> v2:172.21.15.190:3300/0 conn(0x7f18c0153670 0x7f18c0153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:20.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.368+0000 7f18beffd700 1 -- 172.21.15.121:0/3948330685 >> v2:172.21.15.190:3300/0 conn(0x7f18c0153670 msgr2=0x7f18c0153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:20.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.368+0000 7f18beffd700 1 --2- 172.21.15.121:0/3948330685 >> v2:172.21.15.190:3300/0 conn(0x7f18c0153670 0x7f18c0153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:20.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.368+0000 7f18beffd700 1 -- 172.21.15.121:0/3948330685 >> v1:172.21.15.121:6789/0 conn(0x7f18c00c8fb0 legacy=0x7f18c00c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:20.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.368+0000 7f18be7fc700 1 -- 172.21.15.121:0/3948330685 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887142467 0 0) 0x7f18c00c5720 con 0x7f18c00c8fb0 2023-11-15T03:07:20.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.368+0000 7f18beffd700 1 -- 172.21.15.121:0/3948330685 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c00c9d20 con 0x7f18c00c04d0 2023-11-15T03:07:20.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.372+0000 7f18beffd700 1 --2- 172.21.15.121:0/3948330685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18c00c04d0 0x7f18c00c08b0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 rx=0x7f18c01577b0 tx=0x7f18b0009060).ready entity=mon.0 client_cookie=c37bf7c5f1a30898 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:20.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.372+0000 7f18be7fc700 1 -- 172.21.15.121:0/3948330685 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b00090d0 con 0x7f18c00c04d0 2023-11-15T03:07:20.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.372+0000 7f18be7fc700 1 -- 172.21.15.121:0/3948330685 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b000f510 con 0x7f18c00c04d0 2023-11-15T03:07:20.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.372+0000 7f18be7fc700 1 -- 172.21.15.121:0/3948330685 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b000f890 con 0x7f18c00c04d0 2023-11-15T03:07:20.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.372+0000 7f18c549b700 1 -- 172.21.15.121:0/3948330685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18c00c04d0 msgr2=0x7f18c00c08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:20.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.372+0000 7f18c549b700 1 --2- 172.21.15.121:0/3948330685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18c00c04d0 0x7f18c00c08b0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 rx=0x7f18c01577b0 tx=0x7f18b0009060).stop 2023-11-15T03:07:20.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.372+0000 7f18c549b700 1 -- 172.21.15.121:0/3948330685 shutdown_connections 2023-11-15T03:07:20.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.372+0000 7f18c549b700 1 --2- 172.21.15.121:0/3948330685 >> v2:172.21.15.190:3300/0 conn(0x7f18c0153670 0x7f18c0153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:20.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.372+0000 7f18c549b700 1 --2- 172.21.15.121:0/3948330685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18c00c04d0 0x7f18c00c08b0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.372+0000 7f18c549b700 1 -- 172.21.15.121:0/3948330685 >> 172.21.15.121:0/3948330685 conn(0x7f18c0067a60 msgr2=0x7f18c0067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.372+0000 7f18c549b700 1 -- 172.21.15.121:0/3948330685 shutdown_connections 2023-11-15T03:07:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.372+0000 7f18c549b700 1 -- 172.21.15.121:0/3948330685 wait complete. 2023-11-15T03:07:20.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.372+0000 7f18c549b700 1 Processor -- start 2023-11-15T03:07:20.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18c549b700 1 -- start start 2023-11-15T03:07:20.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18c549b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18c00c04d0 0x7f18c014f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:20.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18c549b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18c00c8fb0 0x7f18c0149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:20.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18c549b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18c0153670 0x7f18c00ad070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:20.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18c549b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c01679c0 con 0x7f18c00c04d0 2023-11-15T03:07:20.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18c549b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c0168820 con 0x7f18c00c8fb0 2023-11-15T03:07:20.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18c549b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c0167c10 con 0x7f18c0153670 2023-11-15T03:07:20.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18beffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18c00c04d0 0x7f18c014f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:20.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18beffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18c00c04d0 0x7f18c014f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56314/0 (socket says 172.21.15.121:56314) 2023-11-15T03:07:20.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18beffd700 1 -- 172.21.15.121:0/3104486480 learned_addr learned my addr 172.21.15.121:0/3104486480 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:20.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18b7fff700 1 --2- 172.21.15.121:0/3104486480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18c00c8fb0 0x7f18c0149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:20.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18bf7fe700 1 --2- 172.21.15.121:0/3104486480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18c0153670 0x7f18c00ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:20.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18beffd700 1 -- 172.21.15.121:0/3104486480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18c0153670 msgr2=0x7f18c00ad070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:20.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18beffd700 1 --2- 172.21.15.121:0/3104486480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18c0153670 0x7f18c00ad070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:20.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18beffd700 1 -- 172.21.15.121:0/3104486480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18c00c8fb0 msgr2=0x7f18c0149390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:20.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18beffd700 1 --2- 172.21.15.121:0/3104486480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18c00c8fb0 0x7f18c0149390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:20.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18beffd700 1 -- 172.21.15.121:0/3104486480 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b00092a0 con 0x7f18c00c04d0 2023-11-15T03:07:20.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18bf7fe700 1 --2- 172.21.15.121:0/3104486480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18c0153670 0x7f18c00ad070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:07:20.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18b7fff700 1 --2- 172.21.15.121:0/3104486480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18c00c8fb0 0x7f18c0149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:07:20.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18beffd700 1 --2- 172.21.15.121:0/3104486480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18c00c04d0 0x7f18c014f730 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 rx=0x7f18b0009250 tx=0x7f18b0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:20.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18b6ffd700 1 -- 172.21.15.121:0/3104486480 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b000fc50 con 0x7f18c00c04d0 2023-11-15T03:07:20.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18b6ffd700 1 -- 172.21.15.121:0/3104486480 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b000fdb0 con 0x7f18c00c04d0 2023-11-15T03:07:20.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18b6ffd700 1 -- 172.21.15.121:0/3104486480 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b001b650 con 0x7f18c00c04d0 2023-11-15T03:07:20.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18c549b700 1 -- 172.21.15.121:0/3104486480 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c0167ef0 con 0x7f18c00c04d0 2023-11-15T03:07:20.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.376+0000 7f18c549b700 1 -- 172.21.15.121:0/3104486480 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c01684b0 con 0x7f18c00c04d0 2023-11-15T03:07:20.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.380+0000 7f18c549b700 1 -- 172.21.15.121:0/3104486480 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c0047060 con 0x7f18c00c04d0 2023-11-15T03:07:20.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.380+0000 7f18b6ffd700 1 -- 172.21.15.121:0/3104486480 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f18b001ba40 con 0x7f18c00c04d0 2023-11-15T03:07:20.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.380+0000 7f18b6ffd700 1 --2- 172.21.15.121:0/3104486480 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f18a005a510 0x7f18a005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:20.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.380+0000 7f18b6ffd700 1 -- 172.21.15.121:0/3104486480 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8072+0+0 (secure 0 0 0) 0x7f18b0076740 con 0x7f18c00c04d0 2023-11-15T03:07:20.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.380+0000 7f18b7fff700 1 --2- 172.21.15.121:0/3104486480 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f18a005a510 0x7f18a005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:20.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.384+0000 7f18b7fff700 1 --2- 172.21.15.121:0/3104486480 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f18a005a510 0x7f18a005c9d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f18c00b0810 tx=0x7f18a8008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:20.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.388+0000 7f18b6ffd700 1 -- 172.21.15.121:0/3104486480 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f18b0049780 con 0x7f18c00c04d0 2023-11-15T03:07:20.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:20.608+0000 7f18c549b700 1 -- 172.21.15.121:0/3104486480 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f18c00c8070 con 0x7f18c00c04d0 2023-11-15T03:07:21.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.172+0000 7f18b6ffd700 1 -- 172.21.15.121:0/3104486480 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v214) v1 ==== 101+0+0 (secure 0 0 0) 0x7f18b004cda0 con 0x7f18c00c04d0 2023-11-15T03:07:21.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.176+0000 7f18c549b700 1 -- 172.21.15.121:0/3104486480 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f18a005a510 msgr2=0x7f18a005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.176+0000 7f18c549b700 1 --2- 172.21.15.121:0/3104486480 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f18a005a510 0x7f18a005c9d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f18c00b0810 tx=0x7f18a8008040).stop 2023-11-15T03:07:21.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.176+0000 7f18c549b700 1 -- 172.21.15.121:0/3104486480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18c00c04d0 msgr2=0x7f18c014f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.176+0000 7f18c549b700 1 --2- 172.21.15.121:0/3104486480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18c00c04d0 0x7f18c014f730 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 rx=0x7f18b0009250 tx=0x7f18b0017000).stop 2023-11-15T03:07:21.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.176+0000 7f18c549b700 1 -- 172.21.15.121:0/3104486480 shutdown_connections 2023-11-15T03:07:21.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.176+0000 7f18c549b700 1 --2- 172.21.15.121:0/3104486480 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f18a005a510 0x7f18a005c9d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.176+0000 7f18c549b700 1 --2- 172.21.15.121:0/3104486480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18c0153670 0x7f18c00ad070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.176+0000 7f18c549b700 1 --2- 172.21.15.121:0/3104486480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18c00c8fb0 0x7f18c0149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.176+0000 7f18c549b700 1 --2- 172.21.15.121:0/3104486480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18c00c04d0 0x7f18c014f730 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.176+0000 7f18c549b700 1 -- 172.21.15.121:0/3104486480 >> 172.21.15.121:0/3104486480 conn(0x7f18c0067a60 msgr2=0x7f18c00c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:21.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.176+0000 7f18c549b700 1 -- 172.21.15.121:0/3104486480 shutdown_connections 2023-11-15T03:07:21.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.176+0000 7f18c549b700 1 -- 172.21.15.121:0/3104486480 wait complete. 2023-11-15T03:07:21.187 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:07:21.208 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:07:21.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa41a735d00 1 Processor -- start 2023-11-15T03:07:21.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa41a735d00 1 -- start start 2023-11-15T03:07:21.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa41a735d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644e3e0edd0 0x5644e3e541a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa41a735d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5644e3e58b00 0x5644e3e56e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa41a735d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5644e3ee7670 0x5644e3e57350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa41a735d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e3d87220 con 0x5644e3e0edd0 2023-11-15T03:07:21.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa41a735d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e3eea3c0 con 0x5644e3ee7670 2023-11-15T03:07:21.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa41a735d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5644e3e413e0 con 0x5644e3e58b00 2023-11-15T03:07:21.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa40ec45700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644e3e0edd0 0x5644e3e541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa40ec45700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644e3e0edd0 0x5644e3e541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56320/0 (socket says 172.21.15.121:56320) 2023-11-15T03:07:21.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa40ec45700 1 -- 172.21.15.121:0/2233880307 learned_addr learned my addr 172.21.15.121:0/2233880307 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:21.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa406444700 1 --2- 172.21.15.121:0/2233880307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5644e3e58b00 0x5644e3e56e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa40f446700 1 --2- 172.21.15.121:0/2233880307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5644e3ee7670 0x5644e3e57350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa40ec45700 1 -- 172.21.15.121:0/2233880307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5644e3e58b00 msgr2=0x5644e3e56e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:21.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa40ec45700 1 --2- 172.21.15.121:0/2233880307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5644e3e58b00 0x5644e3e56e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa40ec45700 1 -- 172.21.15.121:0/2233880307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5644e3ee7670 msgr2=0x5644e3e57350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:21.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa40ec45700 1 --2- 172.21.15.121:0/2233880307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5644e3ee7670 0x5644e3e57350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa40ec45700 1 -- 172.21.15.121:0/2233880307 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa400005cf0 con 0x5644e3e0edd0 2023-11-15T03:07:21.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa40ec45700 1 --2- 172.21.15.121:0/2233880307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644e3e0edd0 0x5644e3e541a0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 rx=0x7fa40000ac40 tx=0x7fa400015000).ready entity=mon.0 client_cookie=af715d46b982cb07 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:21.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa40dc43700 1 -- 172.21.15.121:0/2233880307 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa400007610 con 0x5644e3e0edd0 2023-11-15T03:07:21.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa40dc43700 1 -- 172.21.15.121:0/2233880307 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa400005330 con 0x5644e3e0edd0 2023-11-15T03:07:21.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.260+0000 7fa40dc43700 1 -- 172.21.15.121:0/2233880307 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4000078e0 con 0x5644e3e0edd0 2023-11-15T03:07:21.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.264+0000 7fa41a735d00 1 -- 172.21.15.121:0/2233880307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644e3e0edd0 msgr2=0x5644e3e541a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.264+0000 7fa41a735d00 1 --2- 172.21.15.121:0/2233880307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644e3e0edd0 0x5644e3e541a0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 rx=0x7fa40000ac40 tx=0x7fa400015000).stop 2023-11-15T03:07:21.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.264+0000 7fa41a735d00 1 -- 172.21.15.121:0/2233880307 shutdown_connections 2023-11-15T03:07:21.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.264+0000 7fa41a735d00 1 --2- 172.21.15.121:0/2233880307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5644e3ee7670 0x5644e3e57350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.264+0000 7fa41a735d00 1 --2- 172.21.15.121:0/2233880307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5644e3e58b00 0x5644e3e56e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.264+0000 7fa41a735d00 1 --2- 172.21.15.121:0/2233880307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644e3e0edd0 0x5644e3e541a0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.264+0000 7fa41a735d00 1 -- 172.21.15.121:0/2233880307 >> 172.21.15.121:0/2233880307 conn(0x5644e3e5e080 msgr2=0x5644e3e5e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:21.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.264+0000 7fa41a735d00 1 -- 172.21.15.121:0/2233880307 shutdown_connections 2023-11-15T03:07:21.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.264+0000 7fa41a735d00 1 -- 172.21.15.121:0/2233880307 wait complete. 2023-11-15T03:07:21.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.264+0000 7fa41a735d00 1 Processor -- start 2023-11-15T03:07:21.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.264+0000 7fa41a735d00 1 -- start start 2023-11-15T03:07:21.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa41a735d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5644e3e0edd0 0x5644e3e5bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa41a735d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5644e3e58b00 0x5644e3ef8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa41a735d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644e3ee7670 0x5644e3e3d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa41a735d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e3edef60 con 0x5644e3ee7670 2023-11-15T03:07:21.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa41a735d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e3ef8ff0 con 0x5644e3e58b00 2023-11-15T03:07:21.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa41a735d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5644e3e5c400 con 0x5644e3e0edd0 2023-11-15T03:07:21.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa40f446700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644e3ee7670 0x5644e3e3d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa40f446700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644e3ee7670 0x5644e3e3d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56332/0 (socket says 172.21.15.121:56332) 2023-11-15T03:07:21.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 learned_addr learned my addr 172.21.15.121:0/1154058209 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:21.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa40ec45700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5644e3e0edd0 0x5644e3e5bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa406444700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5644e3e58b00 0x5644e3ef8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5644e3e0edd0 msgr2=0x5644e3e5bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa40f446700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5644e3e0edd0 0x5644e3e5bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5644e3e58b00 msgr2=0x5644e3ef8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa40f446700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5644e3e58b00 0x5644e3ef8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644e3db0a20 con 0x5644e3ee7670 2023-11-15T03:07:21.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa40f446700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644e3ee7670 0x5644e3e3d9e0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 rx=0x7fa4080087e0 tx=0x7fa40800c040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:21.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa407fff700 1 -- 172.21.15.121:0/1154058209 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40800f030 con 0x5644e3ee7670 2023-11-15T03:07:21.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa407fff700 1 -- 172.21.15.121:0/1154058209 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa408010040 con 0x5644e3ee7670 2023-11-15T03:07:21.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa407fff700 1 -- 172.21.15.121:0/1154058209 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4080068f0 con 0x5644e3ee7670 2023-11-15T03:07:21.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5644e3db0770 con 0x5644e3ee7670 2023-11-15T03:07:21.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.268+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644e3d7fa70 con 0x5644e3ee7670 2023-11-15T03:07:21.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.272+0000 7fa407fff700 1 -- 172.21.15.121:0/1154058209 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa408006ce0 con 0x5644e3ee7670 2023-11-15T03:07:21.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.272+0000 7fa407fff700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa3f0059020 0x7fa3f005b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.272+0000 7fa407fff700 1 -- 172.21.15.121:0/1154058209 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8072+0+0 (secure 0 0 0) 0x7fa408073a60 con 0x5644e3ee7670 2023-11-15T03:07:21.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.272+0000 7fa40ec45700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa3f0059020 0x7fa3f005b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.294 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:07:21.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.272+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5644e3e534f0 0x5644e3edf640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.272+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3edfb30 con 0x5644e3e534f0 2023-11-15T03:07:21.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.276+0000 7fa406444700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5644e3e534f0 0x5644e3edf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.276+0000 7fa40ec45700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa3f0059020 0x7fa3f005b4e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x5644e3e5d2d0 tx=0x7fa40001a000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:21.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.276+0000 7fa406444700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5644e3e534f0 0x5644e3edf640 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:21.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.276+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v115'245 uv238 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3e534f0 2023-11-15T03:07:21.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.276+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3edb2a0 con 0x5644e3e534f0 2023-11-15T03:07:21.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.280+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v115'246 uv246 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3e534f0 2023-11-15T03:07:21.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.280+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3edb650 con 0x5644e3e534f0 2023-11-15T03:07:21.297 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:07:21.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.284+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v115'247 uv247 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3e534f0 2023-11-15T03:07:21.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.284+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f056a0 con 0x5644e3e534f0 2023-11-15T03:07:21.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.284+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v115'248 uv248 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3e534f0 2023-11-15T03:07:21.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.284+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f05a50 con 0x5644e3e534f0 2023-11-15T03:07:21.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.288+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v115'249 uv249 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3e534f0 2023-11-15T03:07:21.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.288+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f04690 con 0x5644e3e534f0 2023-11-15T03:07:21.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.288+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v115'250 uv250 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3e534f0 2023-11-15T03:07:21.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.288+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f04a90 con 0x5644e3e534f0 2023-11-15T03:07:21.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.292+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v115'251 uv244 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3e534f0 2023-11-15T03:07:21.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.292+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f04e90 con 0x5644e3e534f0 2023-11-15T03:07:21.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.292+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v115'252 uv252 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3e534f0 2023-11-15T03:07:21.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.292+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f04e90 con 0x5644e3e534f0 2023-11-15T03:07:21.306 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:07:21.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.296+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v115'253 uv253 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3e534f0 2023-11-15T03:07:21.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.296+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f04e40 con 0x5644e3e534f0 2023-11-15T03:07:21.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.300+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v115'254 uv254 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3e534f0 2023-11-15T03:07:21.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.300+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f04e40 con 0x5644e3e534f0 2023-11-15T03:07:21.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.300+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v115'255 uv255 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3e534f0 2023-11-15T03:07:21.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.300+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f04e40 con 0x5644e3e534f0 2023-11-15T03:07:21.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.304+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v115'256 uv256 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3e534f0 2023-11-15T03:07:21.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.304+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5644e3f05e00 0x5644e3f081f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.304+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f086e0 con 0x5644e3f05e00 2023-11-15T03:07:21.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.304+0000 7fa40ec45700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5644e3f05e00 0x5644e3f081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.304+0000 7fa40ec45700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5644e3f05e00 0x5644e3f081f0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:21.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.308+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v115'385 uv372 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.308+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f08a90 con 0x5644e3f05e00 2023-11-15T03:07:21.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.312+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v115'386 uv386 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.312+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f08e40 con 0x5644e3f05e00 2023-11-15T03:07:21.322 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:07:21.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.316+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v115'387 uv387 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.316+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0a200 con 0x5644e3f05e00 2023-11-15T03:07:21.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.316+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v115'388 uv388 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.316+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0a5b0 con 0x5644e3f05e00 2023-11-15T03:07:21.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.320+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v115'389 uv389 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.320+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f091f0 con 0x5644e3f05e00 2023-11-15T03:07:21.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.324+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v115'390 uv390 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.324+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f09ae0 con 0x5644e3f05e00 2023-11-15T03:07:21.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.324+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v115'391 uv391 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.324+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0a960 con 0x5644e3f05e00 2023-11-15T03:07:21.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.328+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v115'392 uv378 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.328+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f0a960 con 0x5644e3f05e00 2023-11-15T03:07:21.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.332+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v115'393 uv393 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.332+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0a960 con 0x5644e3f05e00 2023-11-15T03:07:21.342 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:07:21.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.336+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v115'394 uv394 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.336+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0b970 con 0x5644e3f05e00 2023-11-15T03:07:21.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.336+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v115'395 uv395 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.336+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0b970 con 0x5644e3f05e00 2023-11-15T03:07:21.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.340+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v115'396 uv396 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.340+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0a960 con 0x5644e3f05e00 2023-11-15T03:07:21.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.344+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v115'397 uv397 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.344+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0a960 con 0x5644e3f05e00 2023-11-15T03:07:21.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.344+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v115'398 uv384 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.344+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f0a960 con 0x5644e3f05e00 2023-11-15T03:07:21.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.348+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v115'399 uv399 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.348+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0a960 con 0x5644e3f05e00 2023-11-15T03:07:21.358 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:07:21.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.348+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v115'400 uv400 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.352+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0b970 con 0x5644e3f05e00 2023-11-15T03:07:21.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.352+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v115'401 uv401 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.352+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0b970 con 0x5644e3f05e00 2023-11-15T03:07:21.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.356+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v115'402 uv402 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.356+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0a960 con 0x5644e3f05e00 2023-11-15T03:07:21.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.360+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v115'403 uv403 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.360+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5644e3f0ae70 0x5644e3f0d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.360+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0d750 con 0x5644e3f0ae70 2023-11-15T03:07:21.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.360+0000 7fa406444700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5644e3f0ae70 0x5644e3f0d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.360+0000 7fa406444700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5644e3f0ae70 0x5644e3f0d260 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:21.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.364+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v115'211 uv205 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f0ae70 2023-11-15T03:07:21.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.364+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f0d750 con 0x5644e3f0ae70 2023-11-15T03:07:21.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.364+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v115'212 uv212 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f0ae70 2023-11-15T03:07:21.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.364+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0d750 con 0x5644e3f0ae70 2023-11-15T03:07:21.375 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:07:21.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.368+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v115'213 uv213 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f0ae70 2023-11-15T03:07:21.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.368+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0dc40 con 0x5644e3f0ae70 2023-11-15T03:07:21.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.372+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v115'214 uv214 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f0ae70 2023-11-15T03:07:21.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.372+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0dc40 con 0x5644e3f0ae70 2023-11-15T03:07:21.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.372+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v115'215 uv215 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f0ae70 2023-11-15T03:07:21.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.372+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0dc40 con 0x5644e3f0ae70 2023-11-15T03:07:21.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.376+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v115'216 uv210 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f0ae70 2023-11-15T03:07:21.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.376+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f0dc40 con 0x5644e3f0ae70 2023-11-15T03:07:21.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.380+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v115'217 uv217 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f0ae70 2023-11-15T03:07:21.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.380+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0dc40 con 0x5644e3f0ae70 2023-11-15T03:07:21.389 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:07:21.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.380+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v115'218 uv218 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f0ae70 2023-11-15T03:07:21.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.380+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0dc40 con 0x5644e3f0ae70 2023-11-15T03:07:21.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.384+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v115'219 uv219 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f0ae70 2023-11-15T03:07:21.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.384+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0dc40 con 0x5644e3f0ae70 2023-11-15T03:07:21.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.388+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v115'220 uv220 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f0ae70 2023-11-15T03:07:21.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.388+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5644e3f0e150 0x5644e3f10540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.388+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f10a30 con 0x5644e3f0e150 2023-11-15T03:07:21.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.388+0000 7fa40f446700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5644e3f0e150 0x5644e3f10540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.388+0000 7fa40f446700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5644e3f0e150 0x5644e3f10540 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:21.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.392+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v115'252 uv245 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.392+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f10e20 con 0x5644e3f0e150 2023-11-15T03:07:21.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.396+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v115'253 uv253 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.396+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f112e0 con 0x5644e3f0e150 2023-11-15T03:07:21.405 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:07:21.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.396+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v115'254 uv254 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.396+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f0d810 con 0x5644e3f0e150 2023-11-15T03:07:21.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.400+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v115'255 uv255 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.400+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f15d60 con 0x5644e3f0e150 2023-11-15T03:07:21.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.404+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v115'256 uv256 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.404+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f116e0 con 0x5644e3f0e150 2023-11-15T03:07:21.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.408+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v115'257 uv251 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.408+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f11ae0 con 0x5644e3f0e150 2023-11-15T03:07:21.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.408+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v115'258 uv258 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.408+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f12450 con 0x5644e3f0e150 2023-11-15T03:07:21.419 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:07:21.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.412+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v115'259 uv259 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.412+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f12450 con 0x5644e3f0e150 2023-11-15T03:07:21.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.416+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v115'260 uv260 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.416+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f138b0 con 0x5644e3f0e150 2023-11-15T03:07:21.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.420+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v115'261 uv261 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.420+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f138b0 con 0x5644e3f0e150 2023-11-15T03:07:21.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.424+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v115'262 uv262 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.424+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5644e3f12b10 0x5644e3f12ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.424+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f134c0 con 0x5644e3f12b10 2023-11-15T03:07:21.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.424+0000 7fa40ec45700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5644e3f12b10 0x5644e3f12ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.424+0000 7fa40ec45700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5644e3f12b10 0x5644e3f12ef0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:21.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.428+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v115'123 uv122 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f12b10 2023-11-15T03:07:21.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.428+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f138a0 con 0x5644e3f12b10 2023-11-15T03:07:21.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.432+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v115'124 uv124 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f12b10 2023-11-15T03:07:21.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.432+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f16160 con 0x5644e3f12b10 2023-11-15T03:07:21.440 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:07:21.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.432+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v115'125 uv125 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f12b10 2023-11-15T03:07:21.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.432+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f17170 con 0x5644e3f12b10 2023-11-15T03:07:21.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.436+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v115'126 uv126 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f12b10 2023-11-15T03:07:21.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.436+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f17170 con 0x5644e3f12b10 2023-11-15T03:07:21.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.436+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v115'127 uv127 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f12b10 2023-11-15T03:07:21.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.436+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f16160 con 0x5644e3f12b10 2023-11-15T03:07:21.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.440+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v115'128 uv128 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f12b10 2023-11-15T03:07:21.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.440+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5644e3f167e0 0x5644e3f18bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.440+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f192f0 con 0x5644e3f167e0 2023-11-15T03:07:21.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.440+0000 7fa406444700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5644e3f167e0 0x5644e3f18bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.440+0000 7fa406444700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5644e3f167e0 0x5644e3f18bd0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:21.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.444+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v115'88 uv87 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.444+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f167e0 2023-11-15T03:07:21.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.448+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v115'89 uv89 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.448+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f167e0 2023-11-15T03:07:21.456 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:07:21.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.448+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v115'90 uv90 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.448+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f167e0 2023-11-15T03:07:21.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.452+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v115'91 uv91 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.452+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f167e0 2023-11-15T03:07:21.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.456+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v115'123 uv122 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.456+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f167e0 2023-11-15T03:07:21.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.456+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v115'124 uv124 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.456+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f167e0 2023-11-15T03:07:21.467 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:07:21.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.460+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v115'125 uv125 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.460+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1a6e0 con 0x5644e3f167e0 2023-11-15T03:07:21.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.464+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v115'126 uv126 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.464+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1a6e0 con 0x5644e3f167e0 2023-11-15T03:07:21.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.464+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v115'127 uv127 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.464+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f167e0 2023-11-15T03:07:21.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.468+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v115'128 uv128 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.468+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f05e00 2023-11-15T03:07:21.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.472+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v115'227 uv221 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.472+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f05e00 2023-11-15T03:07:21.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.476+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v115'228 uv228 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.476+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f05e00 2023-11-15T03:07:21.486 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:07:21.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.480+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v115'229 uv229 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.480+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1a6e0 con 0x5644e3f05e00 2023-11-15T03:07:21.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.480+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v115'230 uv230 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.480+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1a6e0 con 0x5644e3f05e00 2023-11-15T03:07:21.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.484+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v115'231 uv231 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.484+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f05e00 2023-11-15T03:07:21.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.488+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v115'232 uv232 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.488+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f05e00 2023-11-15T03:07:21.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.488+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v115'233 uv226 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.488+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f05e00 2023-11-15T03:07:21.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.492+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v115'234 uv234 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.492+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f05e00 2023-11-15T03:07:21.503 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:07:21.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.496+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v115'235 uv235 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.496+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f05e00 2023-11-15T03:07:21.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.500+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v115'236 uv236 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.500+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f05e00 2023-11-15T03:07:21.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.504+0000 7fa40ec45700 1 -- 172.21.15.121:0/1154058209 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v115'237 uv237 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa400013440 con 0x5644e3f05e00 2023-11-15T03:07:21.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.504+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f0ae70 2023-11-15T03:07:21.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.504+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v115'102 uv101 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f0ae70 2023-11-15T03:07:21.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.504+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f0ae70 2023-11-15T03:07:21.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.508+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v115'103 uv103 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f0ae70 2023-11-15T03:07:21.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.508+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f0ae70 2023-11-15T03:07:21.518 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:07:21.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.512+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v115'104 uv104 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f0ae70 2023-11-15T03:07:21.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.512+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f0ae70 2023-11-15T03:07:21.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.512+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v115'105 uv105 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f0ae70 2023-11-15T03:07:21.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.512+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f11f10 con 0x5644e3f0ae70 2023-11-15T03:07:21.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.516+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v115'106 uv106 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f0ae70 2023-11-15T03:07:21.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.516+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f11f10 con 0x5644e3f167e0 2023-11-15T03:07:21.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.520+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v115'123 uv122 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.520+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f167e0 2023-11-15T03:07:21.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.520+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v115'124 uv124 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.520+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f167e0 2023-11-15T03:07:21.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.524+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v115'125 uv125 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.532 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:07:21.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.524+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1a6e0 con 0x5644e3f167e0 2023-11-15T03:07:21.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.528+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v115'126 uv126 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.528+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1a6e0 con 0x5644e3f167e0 2023-11-15T03:07:21.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.532+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v115'127 uv127 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.532+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f167e0 2023-11-15T03:07:21.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.536+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v115'128 uv128 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.536+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f167e0 2023-11-15T03:07:21.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.536+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v115'103 uv102 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.536+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f167e0 2023-11-15T03:07:21.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.540+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v115'104 uv104 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.540+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f167e0 2023-11-15T03:07:21.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.544+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v115'105 uv105 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.551 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:07:21.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.544+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f196d0 con 0x5644e3f167e0 2023-11-15T03:07:21.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.548+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v115'106 uv106 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.548+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f19af0 con 0x5644e3f167e0 2023-11-15T03:07:21.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.552+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v115'107 uv107 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.552+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f19af0 con 0x5644e3f167e0 2023-11-15T03:07:21.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.556+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v115'101 uv100 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.556+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f19af0 con 0x5644e3f167e0 2023-11-15T03:07:21.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.560+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v115'102 uv102 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.560+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f19af0 con 0x5644e3f167e0 2023-11-15T03:07:21.569 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:07:21.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.560+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v115'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.560+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f19af0 con 0x5644e3f167e0 2023-11-15T03:07:21.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.564+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v115'104 uv104 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.564+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1c860 con 0x5644e3f167e0 2023-11-15T03:07:21.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.568+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v115'105 uv105 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.568+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5644e3f1a4f0 0x5644e3f1c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.568+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1cdd0 con 0x5644e3f1a4f0 2023-11-15T03:07:21.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.568+0000 7fa40f446700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5644e3f1a4f0 0x5644e3f1c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.572+0000 7fa40f446700 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5644e3f1a4f0 0x5644e3f1c8e0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:21.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.572+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v115'82 uv81 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f1a4f0 2023-11-15T03:07:21.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.572+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f1cdd0 con 0x5644e3f1a4f0 2023-11-15T03:07:21.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.576+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v115'83 uv83 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f1a4f0 2023-11-15T03:07:21.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.576+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1cdd0 con 0x5644e3f1a4f0 2023-11-15T03:07:21.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.580+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v115'84 uv84 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f1a4f0 2023-11-15T03:07:21.588 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:07:21.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.580+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1cdd0 con 0x5644e3f1a4f0 2023-11-15T03:07:21.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.584+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v115'85 uv85 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f1a4f0 2023-11-15T03:07:21.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.584+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1cdd0 con 0x5644e3f167e0 2023-11-15T03:07:21.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.588+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v115'123 uv122 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.588+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f1cdd0 con 0x5644e3f167e0 2023-11-15T03:07:21.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.588+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v115'124 uv124 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.588+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1cdd0 con 0x5644e3f167e0 2023-11-15T03:07:21.599 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:07:21.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.592+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v115'125 uv125 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.592+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1dde0 con 0x5644e3f167e0 2023-11-15T03:07:21.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.596+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v115'126 uv126 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.596+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1dde0 con 0x5644e3f167e0 2023-11-15T03:07:21.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.596+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v115'127 uv127 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.600+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1cdd0 con 0x5644e3f167e0 2023-11-15T03:07:21.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.600+0000 7fa406444700 1 -- 172.21.15.121:0/1154058209 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v115'128 uv128 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa3f8001ac0 con 0x5644e3f167e0 2023-11-15T03:07:21.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.600+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1cdd0 con 0x5644e3f0e150 2023-11-15T03:07:21.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.604+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v115'104 uv103 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.604+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f1cdd0 con 0x5644e3f0e150 2023-11-15T03:07:21.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.608+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v115'105 uv105 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.608+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1cdd0 con 0x5644e3f0e150 2023-11-15T03:07:21.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.608+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v115'106 uv106 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.617 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:07:21.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.608+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1d1d0 con 0x5644e3f0e150 2023-11-15T03:07:21.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.612+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v115'107 uv107 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.612+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1d1d0 con 0x5644e3f0e150 2023-11-15T03:07:21.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.616+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v115'108 uv108 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.616+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1cf50 con 0x5644e3f0e150 2023-11-15T03:07:21.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.620+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v115'244 uv236 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.620+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f1cf50 con 0x5644e3f0e150 2023-11-15T03:07:21.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.620+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v115'245 uv245 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.620+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1cf50 con 0x5644e3f0e150 2023-11-15T03:07:21.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.624+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v115'246 uv246 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.632 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:07:21.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.624+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1cf50 con 0x5644e3f0e150 2023-11-15T03:07:21.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.628+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v115'247 uv247 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.628+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1d350 con 0x5644e3f0e150 2023-11-15T03:07:21.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.632+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v115'248 uv248 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.632+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1d350 con 0x5644e3f0e150 2023-11-15T03:07:21.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.632+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v115'249 uv243 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.632+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e115) v8 -- 0x5644e3f1cf50 con 0x5644e3f0e150 2023-11-15T03:07:21.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.636+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v115'250 uv250 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.636+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1cf50 con 0x5644e3f0e150 2023-11-15T03:07:21.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.640+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v115'251 uv251 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.647 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:07:21.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.640+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1cf30 con 0x5644e3f0e150 2023-11-15T03:07:21.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.640+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v115'252 uv252 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.640+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1df40 con 0x5644e3f0e150 2023-11-15T03:07:21.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.644+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v115'253 uv253 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.644+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1d420 con 0x5644e3f0e150 2023-11-15T03:07:21.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.648+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v115'254 uv254 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.648+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x5644e3f1d420 con 0x5644e3f0e150 2023-11-15T03:07:21.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.648+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v115'255 uv255 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa408012070 con 0x5644e3f0e150 2023-11-15T03:07:21.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.648+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5644e3f0ae70 msgr2=0x5644e3f0d260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.648+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5644e3f0ae70 0x5644e3f0d260 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.648+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5644e3e534f0 msgr2=0x5644e3edf640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.648+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5644e3e534f0 0x5644e3edf640 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.648+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5644e3f12b10 msgr2=0x5644e3f12ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.648+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5644e3f12b10 0x5644e3f12ef0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.648+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5644e3f05e00 msgr2=0x5644e3f081f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.648+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5644e3f05e00 0x5644e3f081f0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.648+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 reap_dead start 2023-11-15T03:07:21.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.648+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5644e3f1a4f0 msgr2=0x5644e3f1c8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.648+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5644e3f1a4f0 0x5644e3f1c8e0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.648+0000 7fa40f446700 1 -- 172.21.15.121:0/1154058209 reap_dead start 2023-11-15T03:07:21.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.648+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5644e3f0e150 msgr2=0x5644e3f10540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.648+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5644e3f0e150 0x5644e3f10540 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.648+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5644e3f167e0 msgr2=0x5644e3f18bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.648+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5644e3f167e0 0x5644e3f18bd0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.652+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa3f0059020 msgr2=0x7fa3f005b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.652+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa3f0059020 0x7fa3f005b4e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x5644e3e5d2d0 tx=0x7fa40001a000).stop 2023-11-15T03:07:21.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.652+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644e3ee7670 msgr2=0x5644e3e3d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.652+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644e3ee7670 0x5644e3e3d9e0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 rx=0x7fa4080087e0 tx=0x7fa40800c040).stop 2023-11-15T03:07:21.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.652+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 shutdown_connections 2023-11-15T03:07:21.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.652+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5644e3f167e0 0x5644e3f18bd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.652+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5644e3f0e150 0x5644e3f10540 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.652+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa3f0059020 0x7fa3f005b4e0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.652+0000 7fa41a735d00 1 --2- 172.21.15.121:0/1154058209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5644e3ee7670 0x5644e3e3d9e0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.652+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 >> 172.21.15.121:0/1154058209 conn(0x5644e3e5e080 msgr2=0x5644e3e43c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:21.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.652+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 shutdown_connections 2023-11-15T03:07:21.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.652+0000 7fa41a735d00 1 -- 172.21.15.121:0/1154058209 wait complete. 2023-11-15T03:07:21.667 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 400.00000000 - 2023-11-15T03:07:21.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.760+0000 7fc9e6b31d00 1 Processor -- start 2023-11-15T03:07:21.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.760+0000 7fc9e6b31d00 1 -- start start 2023-11-15T03:07:21.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.760+0000 7fc9e6b31d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cea7cd3dc0 0x55cea7cd0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.760+0000 7fc9e6b31d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cea7d49650 0x55cea7cca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.760+0000 7fc9e6b31d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cea7d49a30 0x55cea7cca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.760+0000 7fc9e6b31d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea7c00220 con 0x55cea7d49650 2023-11-15T03:07:21.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.760+0000 7fc9e6b31d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea7c88990 con 0x55cea7d49a30 2023-11-15T03:07:21.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.760+0000 7fc9e6b31d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cea7cb8210 con 0x55cea7cd3dc0 2023-11-15T03:07:21.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.760+0000 7fc9d3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cea7d49650 0x55cea7cca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.760+0000 7fc9d3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cea7d49650 0x55cea7cca390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56344/0 (socket says 172.21.15.121:56344) 2023-11-15T03:07:21.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.760+0000 7fc9d3fff700 1 -- 172.21.15.121:0/1521041799 learned_addr learned my addr 172.21.15.121:0/1521041799 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:21.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.760+0000 7fc9dba3d700 1 --2- 172.21.15.121:0/1521041799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cea7d49a30 0x55cea7cca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.760+0000 7fc9db23c700 1 --2- 172.21.15.121:0/1521041799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cea7cd3dc0 0x55cea7cd0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.760+0000 7fc9d3fff700 1 -- 172.21.15.121:0/1521041799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cea7cd3dc0 msgr2=0x55cea7cd0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:21.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.760+0000 7fc9d3fff700 1 --2- 172.21.15.121:0/1521041799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cea7cd3dc0 0x55cea7cd0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.760+0000 7fc9d3fff700 1 -- 172.21.15.121:0/1521041799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cea7d49a30 msgr2=0x55cea7cca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:21.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.760+0000 7fc9d3fff700 1 --2- 172.21.15.121:0/1521041799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cea7d49a30 0x55cea7cca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.760+0000 7fc9d3fff700 1 -- 172.21.15.121:0/1521041799 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c4005cf0 con 0x55cea7d49650 2023-11-15T03:07:21.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.764+0000 7fc9d3fff700 1 --2- 172.21.15.121:0/1521041799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cea7d49650 0x55cea7cca390 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 rx=0x55cea7cbb1b0 tx=0x7fc9c4015000).ready entity=mon.0 client_cookie=4a47328567703de1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:21.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.764+0000 7fc9da23a700 1 -- 172.21.15.121:0/1521041799 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4007610 con 0x55cea7d49650 2023-11-15T03:07:21.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.764+0000 7fc9da23a700 1 -- 172.21.15.121:0/1521041799 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c4000bc0 con 0x55cea7d49650 2023-11-15T03:07:21.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.764+0000 7fc9da23a700 1 -- 172.21.15.121:0/1521041799 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4007b10 con 0x55cea7d49650 2023-11-15T03:07:21.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.764+0000 7fc9e6b31d00 1 -- 172.21.15.121:0/1521041799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cea7d49650 msgr2=0x55cea7cca390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.764+0000 7fc9e6b31d00 1 --2- 172.21.15.121:0/1521041799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cea7d49650 0x55cea7cca390 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 rx=0x55cea7cbb1b0 tx=0x7fc9c4015000).stop 2023-11-15T03:07:21.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.764+0000 7fc9e6b31d00 1 -- 172.21.15.121:0/1521041799 shutdown_connections 2023-11-15T03:07:21.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.764+0000 7fc9e6b31d00 1 --2- 172.21.15.121:0/1521041799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cea7d49a30 0x55cea7cca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.764+0000 7fc9e6b31d00 1 --2- 172.21.15.121:0/1521041799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cea7d49650 0x55cea7cca390 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.764+0000 7fc9e6b31d00 1 --2- 172.21.15.121:0/1521041799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cea7cd3dc0 0x55cea7cd0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.764+0000 7fc9e6b31d00 1 -- 172.21.15.121:0/1521041799 >> 172.21.15.121:0/1521041799 conn(0x55cea7cd18f0 msgr2=0x55cea7c8c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:21.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.764+0000 7fc9e6b31d00 1 -- 172.21.15.121:0/1521041799 shutdown_connections 2023-11-15T03:07:21.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.764+0000 7fc9e6b31d00 1 -- 172.21.15.121:0/1521041799 wait complete. 2023-11-15T03:07:21.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.764+0000 7fc9e6b31d00 1 Processor -- start 2023-11-15T03:07:21.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.764+0000 7fc9e6b31d00 1 -- start start 2023-11-15T03:07:21.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9e6b31d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cea7cd3dc0 0x55cea7cb0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9e6b31d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cea7d49650 0x55cea7d476a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9e6b31d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cea7d49a30 0x55cea7d47b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9e6b31d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea7cb0b70 con 0x55cea7d49650 2023-11-15T03:07:21.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9e6b31d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea7d481f0 con 0x55cea7d49a30 2023-11-15T03:07:21.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9e6b31d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cea7cb5d20 con 0x55cea7cd3dc0 2023-11-15T03:07:21.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9d3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cea7d49650 0x55cea7d476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9d3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cea7d49650 0x55cea7d476a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56346/0 (socket says 172.21.15.121:56346) 2023-11-15T03:07:21.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9d3fff700 1 -- 172.21.15.121:0/854438333 learned_addr learned my addr 172.21.15.121:0/854438333 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:21.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9db23c700 1 --2- 172.21.15.121:0/854438333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cea7cd3dc0 0x55cea7cb0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9dba3d700 1 --2- 172.21.15.121:0/854438333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cea7d49a30 0x55cea7d47b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9d3fff700 1 -- 172.21.15.121:0/854438333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cea7cd3dc0 msgr2=0x55cea7cb0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9d3fff700 1 --2- 172.21.15.121:0/854438333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cea7cd3dc0 0x55cea7cb0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9d3fff700 1 -- 172.21.15.121:0/854438333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cea7d49a30 msgr2=0x55cea7d47b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9d3fff700 1 --2- 172.21.15.121:0/854438333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cea7d49a30 0x55cea7d47b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9d3fff700 1 -- 172.21.15.121:0/854438333 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cea7c29a20 con 0x55cea7d49650 2023-11-15T03:07:21.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9d3fff700 1 --2- 172.21.15.121:0/854438333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cea7d49650 0x55cea7d476a0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 rx=0x7fc9c4020040 tx=0x7fc9c401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:21.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9d8a37700 1 -- 172.21.15.121:0/854438333 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4005630 con 0x55cea7d49650 2023-11-15T03:07:21.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9d8a37700 1 -- 172.21.15.121:0/854438333 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c4007af0 con 0x55cea7d49650 2023-11-15T03:07:21.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9e6b31d00 1 -- 172.21.15.121:0/854438333 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cea7c29770 con 0x55cea7d49650 2023-11-15T03:07:21.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9d8a37700 1 -- 172.21.15.121:0/854438333 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4027690 con 0x55cea7d49650 2023-11-15T03:07:21.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.768+0000 7fc9e6b31d00 1 -- 172.21.15.121:0/854438333 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cea7bf8a70 con 0x55cea7d49650 2023-11-15T03:07:21.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.772+0000 7fc9d8a37700 1 -- 172.21.15.121:0/854438333 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc9c4027a00 con 0x55cea7d49650 2023-11-15T03:07:21.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.772+0000 7fc9d8a37700 1 --2- 172.21.15.121:0/854438333 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc9bc059020 0x7fc9bc05b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.772+0000 7fc9d8a37700 1 -- 172.21.15.121:0/854438333 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8072+0+0 (secure 0 0 0) 0x7fc9c4076b70 con 0x55cea7d49650 2023-11-15T03:07:21.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.772+0000 7fc9db23c700 1 --2- 172.21.15.121:0/854438333 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc9bc059020 0x7fc9bc05b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.772+0000 7fc9e6b31d00 1 --2- 172.21.15.121:0/854438333 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55cea7c895d0 0x55cea7cb4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:21.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.772+0000 7fc9e6b31d00 1 -- 172.21.15.121:0/854438333 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected e115) v8 -- 0x55cea7cb4fb0 con 0x55cea7c895d0 2023-11-15T03:07:21.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.776+0000 7fc9db23c700 1 --2- 172.21.15.121:0/854438333 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc9bc059020 0x7fc9bc05b4e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7fc9cc001ed0 tx=0x7fc9cc000bc0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:21.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.776+0000 7fc9dba3d700 1 --2- 172.21.15.121:0/854438333 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55cea7c895d0 0x55cea7cb4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:21.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.776+0000 7fc9dba3d700 1 --2- 172.21.15.121:0/854438333 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55cea7c895d0 0x55cea7cb4ac0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:21.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.780+0000 7fc9dba3d700 1 -- 172.21.15.121:0/854438333 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 400.00000000 [writefull 0~80] v115'109 uv109 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc9d400bc10 con 0x55cea7c895d0 2023-11-15T03:07:21.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.780+0000 7fc9e6b31d00 1 -- 172.21.15.121:0/854438333 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55cea7c895d0 msgr2=0x55cea7cb4ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.780+0000 7fc9e6b31d00 1 --2- 172.21.15.121:0/854438333 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55cea7c895d0 0x55cea7cb4ac0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:21.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.780+0000 7fc9e6b31d00 1 -- 172.21.15.121:0/854438333 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc9bc059020 msgr2=0x7fc9bc05b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.780+0000 7fc9e6b31d00 1 --2- 172.21.15.121:0/854438333 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc9bc059020 0x7fc9bc05b4e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7fc9cc001ed0 tx=0x7fc9cc000bc0).stop 2023-11-15T03:07:21.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.780+0000 7fc9e6b31d00 1 -- 172.21.15.121:0/854438333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cea7d49650 msgr2=0x55cea7d476a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:21.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.780+0000 7fc9e6b31d00 1 --2- 172.21.15.121:0/854438333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cea7d49650 0x55cea7d476a0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 rx=0x7fc9c4020040 tx=0x7fc9c401a000).stop 2023-11-15T03:07:21.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.780+0000 7fc9dba3d700 1 -- 172.21.15.121:0/854438333 reap_dead start 2023-11-15T03:07:21.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.780+0000 7fc9e6b31d00 1 -- 172.21.15.121:0/854438333 shutdown_connections 2023-11-15T03:07:21.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.780+0000 7fc9e6b31d00 1 -- 172.21.15.121:0/854438333 >> 172.21.15.121:0/854438333 conn(0x55cea7cd18f0 msgr2=0x55cea7cd0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:21.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.780+0000 7fc9e6b31d00 1 -- 172.21.15.121:0/854438333 shutdown_connections 2023-11-15T03:07:21.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:21.780+0000 7fc9e6b31d00 1 -- 172.21.15.121:0/854438333 wait complete. 2023-11-15T03:07:21.795 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:07:22.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.004+0000 7f823d579700 1 Processor -- start 2023-11-15T03:07:22.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.004+0000 7f823d579700 1 -- start start 2023-11-15T03:07:22.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.008+0000 7f823d579700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8238151a50 0x7f82380bc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:22.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.008+0000 7f823d579700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82380c1ba0 0x7f82380c1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:22.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.008+0000 7f823d579700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8238065bf0 con 0x7f82380bc400 2023-11-15T03:07:22.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.008+0000 7f823d579700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82380c2910 con 0x7f8238151a50 2023-11-15T03:07:22.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.008+0000 7f823d579700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82380c2bf0 con 0x7f82380c1ba0 2023-11-15T03:07:22.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.008+0000 7f82377fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8238151a50 0x7f82380bc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:22.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.008+0000 7f82377fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8238151a50 0x7f82380bc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56350/0 (socket says 172.21.15.121:56350) 2023-11-15T03:07:22.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.008+0000 7f82377fe700 1 -- 172.21.15.121:0/2090012266 learned_addr learned my addr 172.21.15.121:0/2090012266 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:22.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.008+0000 7f823cd78700 1 --2- 172.21.15.121:0/2090012266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82380c1ba0 0x7f82380c1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:22.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.008+0000 7f82377fe700 1 -- 172.21.15.121:0/2090012266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82380c1ba0 msgr2=0x7f82380c1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:22.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.008+0000 7f82377fe700 1 --2- 172.21.15.121:0/2090012266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82380c1ba0 0x7f82380c1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:22.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.008+0000 7f82377fe700 1 -- 172.21.15.121:0/2090012266 >> v1:172.21.15.190:6789/0 conn(0x7f82380bc400 legacy=0x7f8238151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:22.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.008+0000 7f82377fe700 1 -- 172.21.15.121:0/2090012266 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82380c2ed0 con 0x7f8238151a50 2023-11-15T03:07:22.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.012+0000 7f82377fe700 1 --2- 172.21.15.121:0/2090012266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8238151a50 0x7f82380bc7e0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 rx=0x7f822c006fd0 tx=0x7f822c0090f0).ready entity=mon.0 client_cookie=e966293dbea625d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:22.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.012+0000 7f8236ffd700 1 -- 172.21.15.121:0/2090012266 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c00b580 con 0x7f8238151a50 2023-11-15T03:07:22.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.012+0000 7f8236ffd700 1 -- 172.21.15.121:0/2090012266 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c00b6e0 con 0x7f8238151a50 2023-11-15T03:07:22.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.012+0000 7f8236ffd700 1 -- 172.21.15.121:0/2090012266 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c00ba60 con 0x7f8238151a50 2023-11-15T03:07:22.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.012+0000 7f823d579700 1 -- 172.21.15.121:0/2090012266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8238151a50 msgr2=0x7f82380bc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:22.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.012+0000 7f823d579700 1 --2- 172.21.15.121:0/2090012266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8238151a50 0x7f82380bc7e0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 rx=0x7f822c006fd0 tx=0x7f822c0090f0).stop 2023-11-15T03:07:22.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.012+0000 7f823d579700 1 -- 172.21.15.121:0/2090012266 shutdown_connections 2023-11-15T03:07:22.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.012+0000 7f823d579700 1 --2- 172.21.15.121:0/2090012266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82380c1ba0 0x7f82380c1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:22.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.012+0000 7f823d579700 1 --2- 172.21.15.121:0/2090012266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8238151a50 0x7f82380bc7e0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:22.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.012+0000 7f823d579700 1 -- 172.21.15.121:0/2090012266 >> 172.21.15.121:0/2090012266 conn(0x7f82380c3460 msgr2=0x7f82380c3870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:22.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.012+0000 7f823d579700 1 -- 172.21.15.121:0/2090012266 shutdown_connections 2023-11-15T03:07:22.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.012+0000 7f823d579700 1 -- 172.21.15.121:0/2090012266 wait complete. 2023-11-15T03:07:22.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.016+0000 7f823d579700 1 Processor -- start 2023-11-15T03:07:22.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.016+0000 7f823d579700 1 -- start start 2023-11-15T03:07:22.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.016+0000 7f823d579700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82380bc400 0x7f82380a9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:22.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.016+0000 7f823d579700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82380c1ba0 0x7f82380a8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:22.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.016+0000 7f823d579700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8238151a50 0x7f8238167860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:22.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.016+0000 7f823d579700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82380aa100 con 0x7f8238151a50 2023-11-15T03:07:22.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.016+0000 7f823d579700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82380bf440 con 0x7f82380c1ba0 2023-11-15T03:07:22.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.016+0000 7f823d579700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82380bff80 con 0x7f82380bc400 2023-11-15T03:07:22.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.020+0000 7f823cd78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8238151a50 0x7f8238167860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:22.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.020+0000 7f823cd78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8238151a50 0x7f8238167860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56358/0 (socket says 172.21.15.121:56358) 2023-11-15T03:07:22.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.020+0000 7f823cd78700 1 -- 172.21.15.121:0/662330476 learned_addr learned my addr 172.21.15.121:0/662330476 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:22.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.020+0000 7f82377fe700 1 --2- 172.21.15.121:0/662330476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82380c1ba0 0x7f82380a8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:22.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.020+0000 7f8237fff700 1 --2- 172.21.15.121:0/662330476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82380bc400 0x7f82380a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:22.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.020+0000 7f823cd78700 1 -- 172.21.15.121:0/662330476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82380bc400 msgr2=0x7f82380a9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:22.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.020+0000 7f823cd78700 1 --2- 172.21.15.121:0/662330476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82380bc400 0x7f82380a9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:22.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.020+0000 7f823cd78700 1 -- 172.21.15.121:0/662330476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82380c1ba0 msgr2=0x7f82380a8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:22.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.020+0000 7f823cd78700 1 --2- 172.21.15.121:0/662330476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82380c1ba0 0x7f82380a8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:22.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.020+0000 7f823cd78700 1 -- 172.21.15.121:0/662330476 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c009370 con 0x7f8238151a50 2023-11-15T03:07:22.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.020+0000 7f823cd78700 1 --2- 172.21.15.121:0/662330476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8238151a50 0x7f8238167860 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7f8230009e90 tx=0x7f823000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:22.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.020+0000 7f821ffff700 1 -- 172.21.15.121:0/662330476 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823000caa0 con 0x7f8238151a50 2023-11-15T03:07:22.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.020+0000 7f821ffff700 1 -- 172.21.15.121:0/662330476 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823000f040 con 0x7f8238151a50 2023-11-15T03:07:22.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.020+0000 7f823d579700 1 -- 172.21.15.121:0/662330476 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82380c0410 con 0x7f8238151a50 2023-11-15T03:07:22.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.020+0000 7f821ffff700 1 -- 172.21.15.121:0/662330476 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823000ce10 con 0x7f8238151a50 2023-11-15T03:07:22.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.020+0000 7f823d579700 1 -- 172.21.15.121:0/662330476 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82380bf930 con 0x7f8238151a50 2023-11-15T03:07:22.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.020+0000 7f823d579700 1 -- 172.21.15.121:0/662330476 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82380c0110 con 0x7f8238151a50 2023-11-15T03:07:22.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.024+0000 7f821ffff700 1 -- 172.21.15.121:0/662330476 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8230013610 con 0x7f8238151a50 2023-11-15T03:07:22.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.024+0000 7f821ffff700 1 --2- 172.21.15.121:0/662330476 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f822405a540 0x7f822405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:22.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.024+0000 7f821ffff700 1 -- 172.21.15.121:0/662330476 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8072+0+0 (secure 0 0 0) 0x7f8230075870 con 0x7f8238151a50 2023-11-15T03:07:22.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.024+0000 7f8237fff700 1 --2- 172.21.15.121:0/662330476 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f822405a540 0x7f822405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:22.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.028+0000 7f8237fff700 1 --2- 172.21.15.121:0/662330476 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f822405a540 0x7f822405ca00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f8228003870 tx=0x7f8228007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:22.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.032+0000 7f821ffff700 1 -- 172.21.15.121:0/662330476 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f82300487f0 con 0x7f8238151a50 2023-11-15T03:07:22.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:22.244+0000 7f823d579700 1 -- 172.21.15.121:0/662330476 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f823806bcf0 con 0x7f8238151a50 2023-11-15T03:07:23.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.116+0000 7f821ffff700 1 -- 172.21.15.121:0/662330476 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v215) v1 ==== 162+0+0 (secure 0 0 0) 0x7f823004be10 con 0x7f8238151a50 2023-11-15T03:07:23.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.120+0000 7f823d579700 1 -- 172.21.15.121:0/662330476 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f822405a540 msgr2=0x7f822405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:23.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.120+0000 7f823d579700 1 --2- 172.21.15.121:0/662330476 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f822405a540 0x7f822405ca00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f8228003870 tx=0x7f8228007d40).stop 2023-11-15T03:07:23.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.120+0000 7f823d579700 1 -- 172.21.15.121:0/662330476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8238151a50 msgr2=0x7f8238167860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:23.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.120+0000 7f823d579700 1 --2- 172.21.15.121:0/662330476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8238151a50 0x7f8238167860 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7f8230009e90 tx=0x7f823000b040).stop 2023-11-15T03:07:23.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.120+0000 7f823d579700 1 -- 172.21.15.121:0/662330476 shutdown_connections 2023-11-15T03:07:23.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.120+0000 7f823d579700 1 --2- 172.21.15.121:0/662330476 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f822405a540 0x7f822405ca00 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:23.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.120+0000 7f823d579700 1 --2- 172.21.15.121:0/662330476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8238151a50 0x7f8238167860 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:23.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.120+0000 7f823d579700 1 --2- 172.21.15.121:0/662330476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82380c1ba0 0x7f82380a8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:23.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.120+0000 7f823d579700 1 --2- 172.21.15.121:0/662330476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82380bc400 0x7f82380a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:23.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.120+0000 7f823d579700 1 -- 172.21.15.121:0/662330476 >> 172.21.15.121:0/662330476 conn(0x7f82380c3460 msgr2=0x7f8238065f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:23.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.120+0000 7f823d579700 1 -- 172.21.15.121:0/662330476 shutdown_connections 2023-11-15T03:07:23.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.120+0000 7f823d579700 1 -- 172.21.15.121:0/662330476 wait complete. 2023-11-15T03:07:23.132 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:07:23.124+0000 7ffa90d3c700 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2023-11-15T03:07:23.133 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:07:23.152 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:23.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.312+0000 7f6dadc5b700 1 Processor -- start 2023-11-15T03:07:23.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.312+0000 7f6dadc5b700 1 -- start start 2023-11-15T03:07:23.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.316+0000 7f6dadc5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6da8068170 0x7f6da815ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:23.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.316+0000 7f6dadc5b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da80cb5c0 con 0x7f6da80aa240 2023-11-15T03:07:23.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.316+0000 7f6dadc5b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da80c9ca0 con 0x7f6da8153bb0 2023-11-15T03:07:23.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.316+0000 7f6dadc5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da80aa6f0 con 0x7f6da8068170 2023-11-15T03:07:23.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.316+0000 7f6da6ffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6da80aa240 0x7f6da8067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59228/0 (socket says 172.21.15.121:59228) 2023-11-15T03:07:23.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.316+0000 7f6da6ffd700 1 -- 172.21.15.121:0/602526709 learned_addr learned my addr 172.21.15.121:0/602526709 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:23.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.316+0000 7f6da7fff700 1 --2- 172.21.15.121:0/602526709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6da8068170 0x7f6da815ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:23.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.316+0000 7f6da67fc700 1 -- 172.21.15.121:0/602526709 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936484801 0 0) 0x7f6da80cb5c0 con 0x7f6da80aa240 2023-11-15T03:07:23.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.316+0000 7f6da67fc700 1 -- 172.21.15.121:0/602526709 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d900033f0 con 0x7f6da80aa240 2023-11-15T03:07:23.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.316+0000 7f6da67fc700 1 -- 172.21.15.121:0/602526709 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718909737 0 0) 0x7f6da80c9ca0 con 0x7f6da8153bb0 2023-11-15T03:07:23.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.316+0000 7f6da7fff700 1 -- 172.21.15.121:0/602526709 >> v1:172.21.15.190:6789/0 conn(0x7f6da8153bb0 legacy=0x7f6da80a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:23.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.316+0000 7f6da7fff700 1 -- 172.21.15.121:0/602526709 >> v1:172.21.15.121:6789/0 conn(0x7f6da80aa240 legacy=0x7f6da8067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:23.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.316+0000 7f6da7fff700 1 -- 172.21.15.121:0/602526709 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da80caa20 con 0x7f6da8068170 2023-11-15T03:07:23.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.316+0000 7f6da7fff700 1 --2- 172.21.15.121:0/602526709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6da8068170 0x7f6da815ec40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f6da0009e90 tx=0x7f6da0009780).ready entity=mon.2 client_cookie=33174772ae4af64d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:23.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.316+0000 7f6da67fc700 1 -- 172.21.15.121:0/602526709 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da000e070 con 0x7f6da8068170 2023-11-15T03:07:23.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.316+0000 7f6da67fc700 1 -- 172.21.15.121:0/602526709 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da000bb60 con 0x7f6da8068170 2023-11-15T03:07:23.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.316+0000 7f6da67fc700 1 -- 172.21.15.121:0/602526709 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da000bee0 con 0x7f6da8068170 2023-11-15T03:07:23.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.320+0000 7f6dadc5b700 1 -- 172.21.15.121:0/602526709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6da8068170 msgr2=0x7f6da815ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:23.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.320+0000 7f6dadc5b700 1 --2- 172.21.15.121:0/602526709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6da8068170 0x7f6da815ec40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f6da0009e90 tx=0x7f6da0009780).stop 2023-11-15T03:07:23.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.320+0000 7f6dadc5b700 1 -- 172.21.15.121:0/602526709 shutdown_connections 2023-11-15T03:07:23.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.320+0000 7f6dadc5b700 1 --2- 172.21.15.121:0/602526709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6da8068170 0x7f6da815ec40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:23.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.320+0000 7f6dadc5b700 1 -- 172.21.15.121:0/602526709 >> 172.21.15.121:0/602526709 conn(0x7f6da8058c50 msgr2=0x7f6da8059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:23.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.320+0000 7f6dadc5b700 1 -- 172.21.15.121:0/602526709 shutdown_connections 2023-11-15T03:07:23.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.320+0000 7f6dadc5b700 1 -- 172.21.15.121:0/602526709 wait complete. 2023-11-15T03:07:23.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.320+0000 7f6dadc5b700 1 Processor -- start 2023-11-15T03:07:23.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.320+0000 7f6dadc5b700 1 -- start start 2023-11-15T03:07:23.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6dadc5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6da8068170 0x7f6da8168760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:23.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6dadc5b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6da80aa240 0x7f6da814e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:23.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6dadc5b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6da8153bb0 0x7f6da80c3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:23.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6dadc5b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da80c4290 con 0x7f6da80aa240 2023-11-15T03:07:23.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6dadc5b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da81655f0 con 0x7f6da8153bb0 2023-11-15T03:07:23.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6dadc5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da8164630 con 0x7f6da8068170 2023-11-15T03:07:23.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6da6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6da80aa240 0x7f6da814e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:23.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6da6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6da80aa240 0x7f6da814e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56366/0 (socket says 172.21.15.121:56366) 2023-11-15T03:07:23.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6da6ffd700 1 -- 172.21.15.121:0/1136029169 learned_addr learned my addr 172.21.15.121:0/1136029169 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:23.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6da7fff700 1 --2- 172.21.15.121:0/1136029169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6da8153bb0 0x7f6da80c3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:23.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6da77fe700 1 --2- 172.21.15.121:0/1136029169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6da8068170 0x7f6da8168760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:23.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6da6ffd700 1 -- 172.21.15.121:0/1136029169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6da8068170 msgr2=0x7f6da8168760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:23.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6da6ffd700 1 --2- 172.21.15.121:0/1136029169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6da8068170 0x7f6da8168760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:23.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6da6ffd700 1 -- 172.21.15.121:0/1136029169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6da8153bb0 msgr2=0x7f6da80c3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:23.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6da6ffd700 1 --2- 172.21.15.121:0/1136029169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6da8153bb0 0x7f6da80c3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:23.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6da6ffd700 1 -- 172.21.15.121:0/1136029169 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da0009a20 con 0x7f6da80aa240 2023-11-15T03:07:23.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6da77fe700 1 --2- 172.21.15.121:0/1136029169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6da8068170 0x7f6da8168760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:07:23.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6da7fff700 1 --2- 172.21.15.121:0/1136029169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6da8153bb0 0x7f6da80c3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:07:23.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6da6ffd700 1 --2- 172.21.15.121:0/1136029169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6da80aa240 0x7f6da814e4d0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 rx=0x7f6d9c00c4c0 tx=0x7f6d9c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:23.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6d8f7fe700 1 -- 172.21.15.121:0/1136029169 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c0072e0 con 0x7f6da80aa240 2023-11-15T03:07:23.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6d8f7fe700 1 -- 172.21.15.121:0/1136029169 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9c007440 con 0x7f6da80aa240 2023-11-15T03:07:23.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6dadc5b700 1 -- 172.21.15.121:0/1136029169 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da8165160 con 0x7f6da80aa240 2023-11-15T03:07:23.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6d8f7fe700 1 -- 172.21.15.121:0/1136029169 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c0057a0 con 0x7f6da80aa240 2023-11-15T03:07:23.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.324+0000 7f6dadc5b700 1 -- 172.21.15.121:0/1136029169 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da8164b10 con 0x7f6da80aa240 2023-11-15T03:07:23.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.328+0000 7f6dadc5b700 1 -- 172.21.15.121:0/1136029169 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da81652f0 con 0x7f6da80aa240 2023-11-15T03:07:23.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.328+0000 7f6d8f7fe700 1 -- 172.21.15.121:0/1136029169 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6d9c005bd0 con 0x7f6da80aa240 2023-11-15T03:07:23.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.332+0000 7f6d8f7fe700 1 --2- 172.21.15.121:0/1136029169 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6d9405a510 0x7f6d9405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:23.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.332+0000 7f6d8f7fe700 1 -- 172.21.15.121:0/1136029169 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8158+0+0 (secure 0 0 0) 0x7f6d9c074890 con 0x7f6da80aa240 2023-11-15T03:07:23.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.332+0000 7f6da77fe700 1 --2- 172.21.15.121:0/1136029169 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6d9405a510 0x7f6d9405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:23.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.332+0000 7f6da77fe700 1 --2- 172.21.15.121:0/1136029169 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6d9405a510 0x7f6d9405c9d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f6d98001700 tx=0x7f6d98009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:23.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.336+0000 7f6d8f7fe700 1 -- 172.21.15.121:0/1136029169 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6d9c0477c0 con 0x7f6da80aa240 2023-11-15T03:07:23.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.536+0000 7f6dadc5b700 1 -- 172.21.15.121:0/1136029169 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6da806c150 con 0x7f6da80aa240 2023-11-15T03:07:23.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.536+0000 7f6d8f7fe700 1 -- 172.21.15.121:0/1136029169 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f6d9c018020 con 0x7f6da80aa240 2023-11-15T03:07:23.545 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:23.545 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":215,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":215,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:23.111652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7679":{"gid":7679,"name":"a","rank":0,"incarnation":215,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/2131958479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2131958479},{"type":"v1","addr":"172.21.15.121:6836","nonce":2131958479}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:23.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.540+0000 7f6dadc5b700 1 -- 172.21.15.121:0/1136029169 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6d9405a510 msgr2=0x7f6d9405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:23.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.540+0000 7f6dadc5b700 1 --2- 172.21.15.121:0/1136029169 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6d9405a510 0x7f6d9405c9d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f6d98001700 tx=0x7f6d98009040).stop 2023-11-15T03:07:23.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.540+0000 7f6dadc5b700 1 -- 172.21.15.121:0/1136029169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6da80aa240 msgr2=0x7f6da814e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:23.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.540+0000 7f6dadc5b700 1 --2- 172.21.15.121:0/1136029169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6da80aa240 0x7f6da814e4d0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 rx=0x7f6d9c00c4c0 tx=0x7f6d9c007c50).stop 2023-11-15T03:07:23.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.540+0000 7f6dadc5b700 1 -- 172.21.15.121:0/1136029169 shutdown_connections 2023-11-15T03:07:23.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.540+0000 7f6dadc5b700 1 --2- 172.21.15.121:0/1136029169 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6d9405a510 0x7f6d9405c9d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:23.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.540+0000 7f6dadc5b700 1 --2- 172.21.15.121:0/1136029169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6da8153bb0 0x7f6da80c3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:23.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.540+0000 7f6dadc5b700 1 --2- 172.21.15.121:0/1136029169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6da80aa240 0x7f6da814e4d0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:23.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.540+0000 7f6dadc5b700 1 --2- 172.21.15.121:0/1136029169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6da8068170 0x7f6da8168760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:23.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.540+0000 7f6dadc5b700 1 -- 172.21.15.121:0/1136029169 >> 172.21.15.121:0/1136029169 conn(0x7f6da8058c50 msgr2=0x7f6da80c0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:23.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.540+0000 7f6dadc5b700 1 -- 172.21.15.121:0/1136029169 shutdown_connections 2023-11-15T03:07:23.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:23.540+0000 7f6dadc5b700 1 -- 172.21.15.121:0/1136029169 wait complete. 2023-11-15T03:07:23.550 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 215 2023-11-15T03:07:23.567 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:07:24.128 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:07:24.120+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:07:24.128 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:07:24.120+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:07:24.142 INFO:tasks.ceph.mds.a.smithi121.stderr: -26> 2023-11-15T03:07:23.124+0000 7ffa90d3c700 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2023-11-15T03:07:24.182 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:07:28.569 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:28.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.728+0000 7f95daf69700 1 Processor -- start 2023-11-15T03:07:28.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.728+0000 7f95daf69700 1 -- start start 2023-11-15T03:07:28.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95daf69700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f95d4153bb0 0x7f95d40a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:28.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95daf69700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f95d40aa020 0x7f95d4067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:28.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95daf69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d4067f50 0x7f95d41578a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:28.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95daf69700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d40aa6f0 con 0x7f95d4153bb0 2023-11-15T03:07:28.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95daf69700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d40c5900 con 0x7f95d40aa020 2023-11-15T03:07:28.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95daf69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d40cb5c0 con 0x7f95d4067f50 2023-11-15T03:07:28.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95d8d05700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f95d4153bb0 0x7f95d40a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:28.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95d8d05700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f95d4153bb0 0x7f95d40a9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50182/0 (socket says 172.21.15.121:50182) 2023-11-15T03:07:28.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95d8d05700 1 -- 172.21.15.121:0/1813960742 learned_addr learned my addr 172.21.15.121:0/1813960742 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:28.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95d3fff700 1 --2- 172.21.15.121:0/1813960742 >> v2:172.21.15.190:3300/0 conn(0x7f95d40aa020 0x7f95d4067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:28.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95d9506700 1 --2- 172.21.15.121:0/1813960742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d4067f50 0x7f95d41578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:28.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95d8d05700 1 -- 172.21.15.121:0/1813960742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d4067f50 msgr2=0x7f95d41578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:28.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95d8d05700 1 --2- 172.21.15.121:0/1813960742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d4067f50 0x7f95d41578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:28.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95d8d05700 1 -- 172.21.15.121:0/1813960742 >> v2:172.21.15.190:3300/0 conn(0x7f95d40aa020 msgr2=0x7f95d4067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:28.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95d8d05700 1 --2- 172.21.15.121:0/1813960742 >> v2:172.21.15.190:3300/0 conn(0x7f95d40aa020 0x7f95d4067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:28.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95d8d05700 1 -- 172.21.15.121:0/1813960742 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d40683e0 con 0x7f95d4153bb0 2023-11-15T03:07:28.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95d8d05700 1 --2- 172.21.15.121:0/1813960742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95d4153bb0 0x7f95d40a9b30 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 rx=0x7f95d40c77f0 tx=0x7f95c80090f0).ready entity=mon.0 client_cookie=f57ba4943af95e5e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:28.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95d37fe700 1 -- 172.21.15.121:0/1813960742 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c800b600 con 0x7f95d4153bb0 2023-11-15T03:07:28.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95d37fe700 1 -- 172.21.15.121:0/1813960742 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c800b760 con 0x7f95d4153bb0 2023-11-15T03:07:28.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.732+0000 7f95d37fe700 1 -- 172.21.15.121:0/1813960742 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c800bae0 con 0x7f95d4153bb0 2023-11-15T03:07:28.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.736+0000 7f95daf69700 1 -- 172.21.15.121:0/1813960742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95d4153bb0 msgr2=0x7f95d40a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:28.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.736+0000 7f95daf69700 1 --2- 172.21.15.121:0/1813960742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95d4153bb0 0x7f95d40a9b30 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 rx=0x7f95d40c77f0 tx=0x7f95c80090f0).stop 2023-11-15T03:07:28.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.736+0000 7f95daf69700 1 -- 172.21.15.121:0/1813960742 shutdown_connections 2023-11-15T03:07:28.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.736+0000 7f95daf69700 1 --2- 172.21.15.121:0/1813960742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d4067f50 0x7f95d41578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:28.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.736+0000 7f95daf69700 1 --2- 172.21.15.121:0/1813960742 >> v2:172.21.15.190:3300/0 conn(0x7f95d40aa020 0x7f95d4067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:28.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.736+0000 7f95daf69700 1 --2- 172.21.15.121:0/1813960742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95d4153bb0 0x7f95d40a9b30 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:28.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.736+0000 7f95daf69700 1 -- 172.21.15.121:0/1813960742 >> 172.21.15.121:0/1813960742 conn(0x7f95d4058c50 msgr2=0x7f95d4059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:28.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.736+0000 7f95daf69700 1 -- 172.21.15.121:0/1813960742 shutdown_connections 2023-11-15T03:07:28.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.736+0000 7f95daf69700 1 -- 172.21.15.121:0/1813960742 wait complete. 2023-11-15T03:07:28.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.736+0000 7f95daf69700 1 Processor -- start 2023-11-15T03:07:28.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.736+0000 7f95daf69700 1 -- start start 2023-11-15T03:07:28.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95daf69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d4067f50 0x7f95d40c2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:28.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95daf69700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95d40aa020 0x7f95d40c33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:28.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95daf69700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95d4153bb0 0x7f95d4150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:28.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95daf69700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d40aae20 con 0x7f95d40aa020 2023-11-15T03:07:28.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95daf69700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d40c3bf0 con 0x7f95d4153bb0 2023-11-15T03:07:28.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95daf69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d40c3ed0 con 0x7f95d4067f50 2023-11-15T03:07:28.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95d3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95d40aa020 0x7f95d40c33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:28.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95d3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95d40aa020 0x7f95d40c33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50184/0 (socket says 172.21.15.121:50184) 2023-11-15T03:07:28.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95d3fff700 1 -- 172.21.15.121:0/2628378614 learned_addr learned my addr 172.21.15.121:0/2628378614 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:28.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95d8d05700 1 --2- 172.21.15.121:0/2628378614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d4067f50 0x7f95d40c2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:28.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95d9506700 1 --2- 172.21.15.121:0/2628378614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95d4153bb0 0x7f95d4150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:28.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95d3fff700 1 -- 172.21.15.121:0/2628378614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d4067f50 msgr2=0x7f95d40c2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:28.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95d3fff700 1 --2- 172.21.15.121:0/2628378614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d4067f50 0x7f95d40c2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:28.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95d3fff700 1 -- 172.21.15.121:0/2628378614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95d4153bb0 msgr2=0x7f95d4150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:28.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95d3fff700 1 --2- 172.21.15.121:0/2628378614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95d4153bb0 0x7f95d4150130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:28.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95d3fff700 1 -- 172.21.15.121:0/2628378614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c8009370 con 0x7f95d40aa020 2023-11-15T03:07:28.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95d3fff700 1 --2- 172.21.15.121:0/2628378614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95d40aa020 0x7f95d40c33b0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 rx=0x7f95c000d890 tx=0x7f95c0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:28.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95d0ff9700 1 -- 172.21.15.121:0/2628378614 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c00125f0 con 0x7f95d40aa020 2023-11-15T03:07:28.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95d0ff9700 1 -- 172.21.15.121:0/2628378614 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c0012750 con 0x7f95d40aa020 2023-11-15T03:07:28.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95daf69700 1 -- 172.21.15.121:0/2628378614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d40c41b0 con 0x7f95d40aa020 2023-11-15T03:07:28.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95d0ff9700 1 -- 172.21.15.121:0/2628378614 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0012ad0 con 0x7f95d40aa020 2023-11-15T03:07:28.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.740+0000 7f95daf69700 1 -- 172.21.15.121:0/2628378614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d413a5c0 con 0x7f95d40aa020 2023-11-15T03:07:28.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.744+0000 7f95daf69700 1 -- 172.21.15.121:0/2628378614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d40aafd0 con 0x7f95d40aa020 2023-11-15T03:07:28.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.744+0000 7f95d0ff9700 1 -- 172.21.15.121:0/2628378614 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f95c0013480 con 0x7f95d40aa020 2023-11-15T03:07:28.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.744+0000 7f95d0ff9700 1 --2- 172.21.15.121:0/2628378614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f95b805a510 0x7f95b805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:28.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.744+0000 7f95d0ff9700 1 -- 172.21.15.121:0/2628378614 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8158+0+0 (secure 0 0 0) 0x7f95c00751c0 con 0x7f95d40aa020 2023-11-15T03:07:28.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.744+0000 7f95d8d05700 1 --2- 172.21.15.121:0/2628378614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f95b805a510 0x7f95b805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:28.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.744+0000 7f95d8d05700 1 --2- 172.21.15.121:0/2628378614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f95b805a510 0x7f95b805c9d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f95c8009130 tx=0x7f95c8017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:28.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.752+0000 7f95d0ff9700 1 -- 172.21.15.121:0/2628378614 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f95d40aafd0 con 0x7f95d40aa020 2023-11-15T03:07:28.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.952+0000 7f95daf69700 1 -- 172.21.15.121:0/2628378614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95d40aaf60 con 0x7f95d40aa020 2023-11-15T03:07:28.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.952+0000 7f95d0ff9700 1 -- 172.21.15.121:0/2628378614 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+4493 (secure 0 0 0) 0x7f95c0048090 con 0x7f95d40aa020 2023-11-15T03:07:28.960 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:28.960 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":217,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9129,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/115896047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":115896047},{"type":"v1","addr":"172.21.15.121:6836","nonce":115896047}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":217}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:24.123941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:28.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.956+0000 7f95daf69700 1 -- 172.21.15.121:0/2628378614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f95b805a510 msgr2=0x7f95b805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:28.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.956+0000 7f95daf69700 1 --2- 172.21.15.121:0/2628378614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f95b805a510 0x7f95b805c9d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f95c8009130 tx=0x7f95c8017000).stop 2023-11-15T03:07:28.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.956+0000 7f95daf69700 1 -- 172.21.15.121:0/2628378614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95d40aa020 msgr2=0x7f95d40c33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:28.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.956+0000 7f95daf69700 1 --2- 172.21.15.121:0/2628378614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95d40aa020 0x7f95d40c33b0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 rx=0x7f95c000d890 tx=0x7f95c0008040).stop 2023-11-15T03:07:28.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.956+0000 7f95daf69700 1 -- 172.21.15.121:0/2628378614 shutdown_connections 2023-11-15T03:07:28.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.956+0000 7f95daf69700 1 --2- 172.21.15.121:0/2628378614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f95b805a510 0x7f95b805c9d0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:28.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.956+0000 7f95daf69700 1 --2- 172.21.15.121:0/2628378614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95d4153bb0 0x7f95d4150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:28.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.956+0000 7f95daf69700 1 --2- 172.21.15.121:0/2628378614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95d40aa020 0x7f95d40c33b0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:28.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.956+0000 7f95daf69700 1 --2- 172.21.15.121:0/2628378614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d4067f50 0x7f95d40c2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:28.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.956+0000 7f95daf69700 1 -- 172.21.15.121:0/2628378614 >> 172.21.15.121:0/2628378614 conn(0x7f95d4058c50 msgr2=0x7f95d40bfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:28.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.956+0000 7f95daf69700 1 -- 172.21.15.121:0/2628378614 shutdown_connections 2023-11-15T03:07:28.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:28.956+0000 7f95daf69700 1 -- 172.21.15.121:0/2628378614 wait complete. 2023-11-15T03:07:28.966 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 217 2023-11-15T03:07:28.983 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:07:28.984 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:29.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.132+0000 7f824a3f9700 1 Processor -- start 2023-11-15T03:07:29.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.132+0000 7f824a3f9700 1 -- start start 2023-11-15T03:07:29.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.132+0000 7f824a3f9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f82440cafb0 0x7f8244066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:29.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.132+0000 7f824a3f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8244067270 0x7f82440ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:29.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.132+0000 7f824a3f9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82440ad3a0 con 0x7f82440cabd0 2023-11-15T03:07:29.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.132+0000 7f824a3f9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82440cb410 con 0x7f82440cafb0 2023-11-15T03:07:29.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.132+0000 7f824a3f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82440ad650 con 0x7f8244067270 2023-11-15T03:07:29.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.132+0000 7f82493f7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f82440cabd0 0x7f82440acc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47890/0 (socket says 172.21.15.121:47890) 2023-11-15T03:07:29.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.132+0000 7f82493f7700 1 -- 172.21.15.121:0/820281859 learned_addr learned my addr 172.21.15.121:0/820281859 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:29.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.132+0000 7f8248bf6700 1 --2- 172.21.15.121:0/820281859 >> v2:172.21.15.190:3300/0 conn(0x7f82440cafb0 0x7f8244066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:29.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.132+0000 7f8249bf8700 1 --2- 172.21.15.121:0/820281859 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8244067270 0x7f82440ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:29.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.132+0000 7f823bfff700 1 -- 172.21.15.121:0/820281859 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602324378 0 0) 0x7f82440ad3a0 con 0x7f82440cabd0 2023-11-15T03:07:29.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.132+0000 7f823bfff700 1 -- 172.21.15.121:0/820281859 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82300033f0 con 0x7f82440cabd0 2023-11-15T03:07:29.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f823bfff700 1 -- 172.21.15.121:0/820281859 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3859137395 0 0) 0x7f82300033f0 con 0x7f82440cabd0 2023-11-15T03:07:29.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f823bfff700 1 -- 172.21.15.121:0/820281859 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8244067270 msgr2=0x7f82440ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:29.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f823bfff700 1 --2- 172.21.15.121:0/820281859 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8244067270 0x7f82440ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:29.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f823bfff700 1 -- 172.21.15.121:0/820281859 >> v2:172.21.15.190:3300/0 conn(0x7f82440cafb0 msgr2=0x7f8244066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:29.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f823bfff700 1 --2- 172.21.15.121:0/820281859 >> v2:172.21.15.190:3300/0 conn(0x7f82440cafb0 0x7f8244066d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:29.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f823bfff700 1 -- 172.21.15.121:0/820281859 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8244065fc0 con 0x7f82440cabd0 2023-11-15T03:07:29.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f823bfff700 1 -- 172.21.15.121:0/820281859 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f8240001470 con 0x7f82440cabd0 2023-11-15T03:07:29.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f823bfff700 1 -- 172.21.15.121:0/820281859 >> v1:172.21.15.121:6789/0 conn(0x7f82440cabd0 legacy=0x7f82440acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:29.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f823bfff700 1 --2- 172.21.15.121:0/820281859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82300038e0 0x7f8230003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:29.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f823bfff700 1 --2- 172.21.15.121:0/820281859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8230004690 0x7f8230004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:29.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f823bfff700 1 --2- 172.21.15.121:0/820281859 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8244067270 0x7f8230005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:29.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f823bfff700 1 -- 172.21.15.121:0/820281859 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82440cb410 con 0x7f82300038e0 2023-11-15T03:07:29.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f823bfff700 1 -- 172.21.15.121:0/820281859 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82440ad650 con 0x7f8230004690 2023-11-15T03:07:29.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f823bfff700 1 -- 172.21.15.121:0/820281859 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823000b8e0 con 0x7f8244067270 2023-11-15T03:07:29.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f8248bf6700 1 --2- 172.21.15.121:0/820281859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8230004690 0x7f8230004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:29.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f82493f7700 1 --2- 172.21.15.121:0/820281859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82300038e0 0x7f8230003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:29.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f8249bf8700 1 --2- 172.21.15.121:0/820281859 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8244067270 0x7f8230005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:29.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f8248bf6700 1 -- 172.21.15.121:0/820281859 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8244067270 msgr2=0x7f8230005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:29.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f8248bf6700 1 --2- 172.21.15.121:0/820281859 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8244067270 0x7f8230005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:29.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f8248bf6700 1 -- 172.21.15.121:0/820281859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82300038e0 msgr2=0x7f8230003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:29.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f8248bf6700 1 --2- 172.21.15.121:0/820281859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82300038e0 0x7f8230003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:29.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f8248bf6700 1 -- 172.21.15.121:0/820281859 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82300033f0 con 0x7f8230004690 2023-11-15T03:07:29.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f8248bf6700 1 --2- 172.21.15.121:0/820281859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8230004690 0x7f8230004b10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f8234009a80 tx=0x7f8234009060).ready entity=mon.1 client_cookie=2792c26f89ebee9f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:29.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f823bfff700 1 -- 172.21.15.121:0/820281859 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234009430 con 0x7f8230004690 2023-11-15T03:07:29.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f823bfff700 1 -- 172.21.15.121:0/820281859 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823400e640 con 0x7f8230004690 2023-11-15T03:07:29.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f823bfff700 1 -- 172.21.15.121:0/820281859 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823400e9c0 con 0x7f8230004690 2023-11-15T03:07:29.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f824a3f9700 1 -- 172.21.15.121:0/820281859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8230004690 msgr2=0x7f8230004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:29.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f824a3f9700 1 --2- 172.21.15.121:0/820281859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8230004690 0x7f8230004b10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f8234009a80 tx=0x7f8234009060).stop 2023-11-15T03:07:29.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f8249bf8700 1 -- 172.21.15.121:0/820281859 reap_dead start 2023-11-15T03:07:29.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f824a3f9700 1 -- 172.21.15.121:0/820281859 shutdown_connections 2023-11-15T03:07:29.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f824a3f9700 1 -- 172.21.15.121:0/820281859 >> 172.21.15.121:0/820281859 conn(0x7f8244062190 msgr2=0x7f82440625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:29.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f824a3f9700 1 -- 172.21.15.121:0/820281859 shutdown_connections 2023-11-15T03:07:29.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.136+0000 7f824a3f9700 1 -- 172.21.15.121:0/820281859 wait complete. 2023-11-15T03:07:29.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.140+0000 7f824a3f9700 1 Processor -- start 2023-11-15T03:07:29.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.140+0000 7f824a3f9700 1 -- start start 2023-11-15T03:07:29.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.140+0000 7f824a3f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8230004690 0x7f82441549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:29.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.140+0000 7f824a3f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8244140600 0x7f82441550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:29.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.140+0000 7f824a3f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82440af830 0x7f82440afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:29.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.140+0000 7f824a3f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82441564c0 con 0x7f8244140600 2023-11-15T03:07:29.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.140+0000 7f824a3f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82440b01a0 con 0x7f8230004690 2023-11-15T03:07:29.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.140+0000 7f824a3f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8244140a40 con 0x7f82440af830 2023-11-15T03:07:29.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.140+0000 7f8248bf6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8244140600 0x7f82441550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:29.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.140+0000 7f8248bf6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8244140600 0x7f82441550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50204/0 (socket says 172.21.15.121:50204) 2023-11-15T03:07:29.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.140+0000 7f8248bf6700 1 -- 172.21.15.121:0/1823697175 learned_addr learned my addr 172.21.15.121:0/1823697175 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:29.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.140+0000 7f82493f7700 1 --2- 172.21.15.121:0/1823697175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8230004690 0x7f82441549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:29.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.140+0000 7f8249bf8700 1 --2- 172.21.15.121:0/1823697175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82440af830 0x7f82440afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:29.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.140+0000 7f8248bf6700 1 -- 172.21.15.121:0/1823697175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82440af830 msgr2=0x7f82440afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:29.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.140+0000 7f8248bf6700 1 --2- 172.21.15.121:0/1823697175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82440af830 0x7f82440afcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:29.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.140+0000 7f8248bf6700 1 -- 172.21.15.121:0/1823697175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8230004690 msgr2=0x7f82441549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:29.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.140+0000 7f8248bf6700 1 --2- 172.21.15.121:0/1823697175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8230004690 0x7f82441549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:29.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.140+0000 7f8248bf6700 1 -- 172.21.15.121:0/1823697175 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82441555e0 con 0x7f8244140600 2023-11-15T03:07:29.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.144+0000 7f8248bf6700 1 --2- 172.21.15.121:0/1823697175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8244140600 0x7f82441550f0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 rx=0x7f82340030b0 tx=0x7f823401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:29.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.144+0000 7f82397fa700 1 -- 172.21.15.121:0/1823697175 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823400ed30 con 0x7f8244140600 2023-11-15T03:07:29.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.144+0000 7f82397fa700 1 -- 172.21.15.121:0/1823697175 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823400ee90 con 0x7f8244140600 2023-11-15T03:07:29.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.144+0000 7f824a3f9700 1 -- 172.21.15.121:0/1823697175 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82441558b0 con 0x7f8244140600 2023-11-15T03:07:29.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.144+0000 7f82397fa700 1 -- 172.21.15.121:0/1823697175 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234026440 con 0x7f8244140600 2023-11-15T03:07:29.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.144+0000 7f824a3f9700 1 -- 172.21.15.121:0/1823697175 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8244155e70 con 0x7f8244140600 2023-11-15T03:07:29.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.144+0000 7f824a3f9700 1 -- 172.21.15.121:0/1823697175 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f824406bcf0 con 0x7f8244140600 2023-11-15T03:07:29.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.144+0000 7f82397fa700 1 -- 172.21.15.121:0/1823697175 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f823402f440 con 0x7f8244140600 2023-11-15T03:07:29.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.148+0000 7f82397fa700 1 --2- 172.21.15.121:0/1823697175 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f822805a540 0x7f822805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:29.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.148+0000 7f82397fa700 1 -- 172.21.15.121:0/1823697175 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8158+0+0 (secure 0 0 0) 0x7f8234076ec0 con 0x7f8244140600 2023-11-15T03:07:29.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.148+0000 7f82493f7700 1 --2- 172.21.15.121:0/1823697175 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f822805a540 0x7f822805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:29.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.148+0000 7f82493f7700 1 --2- 172.21.15.121:0/1823697175 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f822805a540 0x7f822805ca00 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f8240007c50 tx=0x7f8240009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:29.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.152+0000 7f82397fa700 1 -- 172.21.15.121:0/1823697175 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8234049d90 con 0x7f8244140600 2023-11-15T03:07:29.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.352+0000 7f824a3f9700 1 -- 172.21.15.121:0/1823697175 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82440662a0 con 0x7f8244140600 2023-11-15T03:07:29.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.352+0000 7f82397fa700 1 -- 172.21.15.121:0/1823697175 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+4493 (secure 0 0 0) 0x7f8234016030 con 0x7f8244140600 2023-11-15T03:07:29.361 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:29.361 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":217,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9129,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/115896047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":115896047},{"type":"v1","addr":"172.21.15.121:6836","nonce":115896047}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":217}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:24.123941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:29.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.360+0000 7f824a3f9700 1 -- 172.21.15.121:0/1823697175 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f822805a540 msgr2=0x7f822805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:29.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.360+0000 7f824a3f9700 1 --2- 172.21.15.121:0/1823697175 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f822805a540 0x7f822805ca00 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f8240007c50 tx=0x7f8240009040).stop 2023-11-15T03:07:29.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.360+0000 7f824a3f9700 1 -- 172.21.15.121:0/1823697175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8244140600 msgr2=0x7f82441550f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:29.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.360+0000 7f824a3f9700 1 --2- 172.21.15.121:0/1823697175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8244140600 0x7f82441550f0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 rx=0x7f82340030b0 tx=0x7f823401a000).stop 2023-11-15T03:07:29.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.360+0000 7f824a3f9700 1 -- 172.21.15.121:0/1823697175 shutdown_connections 2023-11-15T03:07:29.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.360+0000 7f824a3f9700 1 --2- 172.21.15.121:0/1823697175 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f822805a540 0x7f822805ca00 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:29.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.360+0000 7f824a3f9700 1 --2- 172.21.15.121:0/1823697175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82440af830 0x7f82440afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:29.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.360+0000 7f824a3f9700 1 --2- 172.21.15.121:0/1823697175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8244140600 0x7f82441550f0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:29.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.360+0000 7f824a3f9700 1 --2- 172.21.15.121:0/1823697175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8230004690 0x7f82441549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:29.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.360+0000 7f824a3f9700 1 -- 172.21.15.121:0/1823697175 >> 172.21.15.121:0/1823697175 conn(0x7f8244062190 msgr2=0x7f82441524d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:29.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.360+0000 7f824a3f9700 1 -- 172.21.15.121:0/1823697175 shutdown_connections 2023-11-15T03:07:29.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.360+0000 7f824a3f9700 1 -- 172.21.15.121:0/1823697175 wait complete. 2023-11-15T03:07:29.370 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 217 2023-11-15T03:07:29.383 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:07:29.384 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:29.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.536+0000 7f008f59e700 1 Processor -- start 2023-11-15T03:07:29.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.536+0000 7f008f59e700 1 -- start start 2023-11-15T03:07:29.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.536+0000 7f008f59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0090153aa0 0x7f0090153e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:29.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.536+0000 7f008f59e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00900c6490 con 0x7f00900caa00 2023-11-15T03:07:29.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.536+0000 7f008f59e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0090157e10 con 0x7f00901578a0 2023-11-15T03:07:29.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.536+0000 7f008f59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00901580f0 con 0x7f0090153aa0 2023-11-15T03:07:29.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.540+0000 7f008e59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0090153aa0 0x7f0090153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:29.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.540+0000 7f008ed9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f00900caa00 0x7f0090065c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47900/0 (socket says 172.21.15.121:47900) 2023-11-15T03:07:29.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.540+0000 7f008ed9d700 1 -- 172.21.15.121:0/7442042 learned_addr learned my addr 172.21.15.121:0/7442042 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:29.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.540+0000 7f008e59c700 1 -- 172.21.15.121:0/7442042 >> v1:172.21.15.190:6789/0 conn(0x7f00901578a0 legacy=0x7f00900c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:29.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.540+0000 7f008e59c700 1 -- 172.21.15.121:0/7442042 >> v1:172.21.15.121:6789/0 conn(0x7f00900caa00 legacy=0x7f0090065c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:29.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.540+0000 7f008e59c700 1 -- 172.21.15.121:0/7442042 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00901583d0 con 0x7f0090153aa0 2023-11-15T03:07:29.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.540+0000 7f008d59a700 1 -- 172.21.15.121:0/7442042 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200390913 0 0) 0x7f00900c6490 con 0x7f00900caa00 2023-11-15T03:07:29.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.540+0000 7f008e59c700 1 --2- 172.21.15.121:0/7442042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0090153aa0 0x7f0090153e80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f0080009790 tx=0x7f0080009060).ready entity=mon.2 client_cookie=d580036b38c46b10 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:29.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.540+0000 7f008d59a700 1 -- 172.21.15.121:0/7442042 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008000b5a0 con 0x7f0090153aa0 2023-11-15T03:07:29.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.540+0000 7f008d59a700 1 -- 172.21.15.121:0/7442042 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008000b700 con 0x7f0090153aa0 2023-11-15T03:07:29.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.540+0000 7f008d59a700 1 -- 172.21.15.121:0/7442042 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008000ba80 con 0x7f0090153aa0 2023-11-15T03:07:29.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.540+0000 7f008f59e700 1 -- 172.21.15.121:0/7442042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0090153aa0 msgr2=0x7f0090153e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:29.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.540+0000 7f008f59e700 1 --2- 172.21.15.121:0/7442042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0090153aa0 0x7f0090153e80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f0080009790 tx=0x7f0080009060).stop 2023-11-15T03:07:29.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.540+0000 7f008f59e700 1 -- 172.21.15.121:0/7442042 shutdown_connections 2023-11-15T03:07:29.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.540+0000 7f008f59e700 1 --2- 172.21.15.121:0/7442042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0090153aa0 0x7f0090153e80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:29.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.540+0000 7f008f59e700 1 -- 172.21.15.121:0/7442042 >> 172.21.15.121:0/7442042 conn(0x7f0090063650 msgr2=0x7f0090063a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:29.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.540+0000 7f008f59e700 1 -- 172.21.15.121:0/7442042 shutdown_connections 2023-11-15T03:07:29.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.540+0000 7f008f59e700 1 -- 172.21.15.121:0/7442042 wait complete. 2023-11-15T03:07:29.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008f59e700 1 Processor -- start 2023-11-15T03:07:29.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008f59e700 1 -- start start 2023-11-15T03:07:29.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008f59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00900caa00 0x7f00900aa040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:29.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008f59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0090153aa0 0x7f0090165290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:29.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008f59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00901578a0 0x7f00900ac300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:29.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008f59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0090165780 con 0x7f00900caa00 2023-11-15T03:07:29.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008f59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00900ae7e0 con 0x7f0090153aa0 2023-11-15T03:07:29.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008f59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00900adbd0 con 0x7f00901578a0 2023-11-15T03:07:29.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008e59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00900caa00 0x7f00900aa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:29.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008e59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00900caa00 0x7f00900aa040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50210/0 (socket says 172.21.15.121:50210) 2023-11-15T03:07:29.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008e59c700 1 -- 172.21.15.121:0/1691344489 learned_addr learned my addr 172.21.15.121:0/1691344489 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:29.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008ed9d700 1 --2- 172.21.15.121:0/1691344489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00901578a0 0x7f00900ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:29.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008dd9b700 1 --2- 172.21.15.121:0/1691344489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0090153aa0 0x7f0090165290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:29.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008e59c700 1 -- 172.21.15.121:0/1691344489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00901578a0 msgr2=0x7f00900ac300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:29.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008e59c700 1 --2- 172.21.15.121:0/1691344489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00901578a0 0x7f00900ac300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:29.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008e59c700 1 -- 172.21.15.121:0/1691344489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0090153aa0 msgr2=0x7f0090165290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:29.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008e59c700 1 --2- 172.21.15.121:0/1691344489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0090153aa0 0x7f0090165290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:29.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008e59c700 1 -- 172.21.15.121:0/1691344489 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00800092a0 con 0x7f00900caa00 2023-11-15T03:07:29.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008dd9b700 1 --2- 172.21.15.121:0/1691344489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0090153aa0 0x7f0090165290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:07:29.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f008e59c700 1 --2- 172.21.15.121:0/1691344489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00900caa00 0x7f00900aa040 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 rx=0x7f0080009ee0 tx=0x7f0080015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:29.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.544+0000 7f007e7fc700 1 -- 172.21.15.121:0/1691344489 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008000ba80 con 0x7f00900caa00 2023-11-15T03:07:29.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.548+0000 7f007e7fc700 1 -- 172.21.15.121:0/1691344489 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0080000c90 con 0x7f00900caa00 2023-11-15T03:07:29.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.548+0000 7f008f59e700 1 -- 172.21.15.121:0/1691344489 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00900adeb0 con 0x7f00900caa00 2023-11-15T03:07:29.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.548+0000 7f007e7fc700 1 -- 172.21.15.121:0/1691344489 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008001b660 con 0x7f00900caa00 2023-11-15T03:07:29.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.548+0000 7f008f59e700 1 -- 172.21.15.121:0/1691344489 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00900ae470 con 0x7f00900caa00 2023-11-15T03:07:29.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.548+0000 7f008f59e700 1 -- 172.21.15.121:0/1691344489 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00900cb8b0 con 0x7f00900caa00 2023-11-15T03:07:29.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.548+0000 7f007e7fc700 1 -- 172.21.15.121:0/1691344489 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f008001ba50 con 0x7f00900caa00 2023-11-15T03:07:29.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.552+0000 7f007e7fc700 1 --2- 172.21.15.121:0/1691344489 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f007005a510 0x7f007005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:29.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.552+0000 7f007e7fc700 1 -- 172.21.15.121:0/1691344489 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8158+0+0 (secure 0 0 0) 0x7f0080076a90 con 0x7f00900caa00 2023-11-15T03:07:29.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.552+0000 7f008dd9b700 1 --2- 172.21.15.121:0/1691344489 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f007005a510 0x7f007005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:29.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.552+0000 7f008dd9b700 1 --2- 172.21.15.121:0/1691344489 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f007005a510 0x7f007005c9d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f0090066670 tx=0x7f0084009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:29.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.556+0000 7f007e7fc700 1 -- 172.21.15.121:0/1691344489 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f00900cb8b0 con 0x7f00900caa00 2023-11-15T03:07:29.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.756+0000 7f008f59e700 1 -- 172.21.15.121:0/1691344489 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00900cb8b0 con 0x7f00900caa00 2023-11-15T03:07:29.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.756+0000 7f007e7fc700 1 -- 172.21.15.121:0/1691344489 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+4493 (secure 0 0 0) 0x7f00900cb8b0 con 0x7f00900caa00 2023-11-15T03:07:29.766 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:29.767 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":217,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9129,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/115896047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":115896047},{"type":"v1","addr":"172.21.15.121:6836","nonce":115896047}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":217}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:24.123941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:29.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.760+0000 7f008f59e700 1 -- 172.21.15.121:0/1691344489 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f007005a510 msgr2=0x7f007005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:29.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.760+0000 7f008f59e700 1 --2- 172.21.15.121:0/1691344489 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f007005a510 0x7f007005c9d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f0090066670 tx=0x7f0084009040).stop 2023-11-15T03:07:29.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.760+0000 7f008f59e700 1 -- 172.21.15.121:0/1691344489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00900caa00 msgr2=0x7f00900aa040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:29.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.760+0000 7f008f59e700 1 --2- 172.21.15.121:0/1691344489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00900caa00 0x7f00900aa040 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 rx=0x7f0080009ee0 tx=0x7f0080015000).stop 2023-11-15T03:07:29.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.760+0000 7f008f59e700 1 -- 172.21.15.121:0/1691344489 shutdown_connections 2023-11-15T03:07:29.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.760+0000 7f008f59e700 1 --2- 172.21.15.121:0/1691344489 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f007005a510 0x7f007005c9d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:29.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.760+0000 7f008f59e700 1 --2- 172.21.15.121:0/1691344489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00901578a0 0x7f00900ac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:29.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.760+0000 7f008f59e700 1 --2- 172.21.15.121:0/1691344489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0090153aa0 0x7f0090165290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:29.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.760+0000 7f008f59e700 1 --2- 172.21.15.121:0/1691344489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00900caa00 0x7f00900aa040 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:29.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.760+0000 7f008f59e700 1 -- 172.21.15.121:0/1691344489 >> 172.21.15.121:0/1691344489 conn(0x7f0090063650 msgr2=0x7f0090065b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:29.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.760+0000 7f008f59e700 1 -- 172.21.15.121:0/1691344489 shutdown_connections 2023-11-15T03:07:29.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:29.760+0000 7f008f59e700 1 -- 172.21.15.121:0/1691344489 wait complete. 2023-11-15T03:07:29.771 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 217 2023-11-15T03:07:29.788 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 400.00000000' led to DAMAGED state 2023-11-15T03:07:29.788 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 100.00000000.inode' 2023-11-15T03:07:29.789 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:07:29.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:07:29.789 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:07:29.814 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:07:29.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:07:29.816 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:07:30.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.020+0000 7fb1246db700 1 Processor -- start 2023-11-15T03:07:30.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.020+0000 7fb1246db700 1 -- start start 2023-11-15T03:07:30.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.020+0000 7fb1246db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb11c151920 0x7fb11c0bb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:30.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.020+0000 7fb1246db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb11c067150 0x7fb11c0aba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:30.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.020+0000 7fb1246db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11c155de0 con 0x7fb11c156270 2023-11-15T03:07:30.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.020+0000 7fb1246db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11c0c4060 con 0x7fb11c067150 2023-11-15T03:07:30.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.020+0000 7fb1246db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11c0c3aa0 con 0x7fb11c151920 2023-11-15T03:07:30.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.020+0000 7fb121c76700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb11c067150 0x7fb11c0aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:30.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.020+0000 7fb122c78700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb11c156270 0x7fb11c0658a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47904/0 (socket says 172.21.15.121:47904) 2023-11-15T03:07:30.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.020+0000 7fb122c78700 1 -- 172.21.15.121:0/3763882100 learned_addr learned my addr 172.21.15.121:0/3763882100 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:30.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.020+0000 7fb122477700 1 --2- 172.21.15.121:0/3763882100 >> v2:172.21.15.190:3300/0 conn(0x7fb11c151920 0x7fb11c0bb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:30.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.024+0000 7fb121c76700 1 -- 172.21.15.121:0/3763882100 >> v2:172.21.15.190:3300/0 conn(0x7fb11c151920 msgr2=0x7fb11c0bb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:30.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.024+0000 7fb121c76700 1 --2- 172.21.15.121:0/3763882100 >> v2:172.21.15.190:3300/0 conn(0x7fb11c151920 0x7fb11c0bb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:30.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.024+0000 7fb121c76700 1 -- 172.21.15.121:0/3763882100 >> v1:172.21.15.121:6789/0 conn(0x7fb11c156270 legacy=0x7fb11c0658a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:30.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.024+0000 7fb121c76700 1 -- 172.21.15.121:0/3763882100 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c0c3d80 con 0x7fb11c067150 2023-11-15T03:07:30.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.024+0000 7fb121c76700 1 --2- 172.21.15.121:0/3763882100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb11c067150 0x7fb11c0aba40 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 rx=0x7fb11c063d30 tx=0x7fb10c009060).ready entity=mon.0 client_cookie=bfbc892de6b50329 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:30.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.024+0000 7fb121475700 1 -- 172.21.15.121:0/3763882100 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c00f510 con 0x7fb11c067150 2023-11-15T03:07:30.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.024+0000 7fb121475700 1 -- 172.21.15.121:0/3763882100 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c00f670 con 0x7fb11c067150 2023-11-15T03:07:30.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.024+0000 7fb121475700 1 -- 172.21.15.121:0/3763882100 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c00f9f0 con 0x7fb11c067150 2023-11-15T03:07:30.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.024+0000 7fb1246db700 1 -- 172.21.15.121:0/3763882100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb11c067150 msgr2=0x7fb11c0aba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:30.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.024+0000 7fb1246db700 1 --2- 172.21.15.121:0/3763882100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb11c067150 0x7fb11c0aba40 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 rx=0x7fb11c063d30 tx=0x7fb10c009060).stop 2023-11-15T03:07:30.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.024+0000 7fb1246db700 1 -- 172.21.15.121:0/3763882100 shutdown_connections 2023-11-15T03:07:30.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.024+0000 7fb1246db700 1 --2- 172.21.15.121:0/3763882100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb11c067150 0x7fb11c0aba40 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:30.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.024+0000 7fb1246db700 1 --2- 172.21.15.121:0/3763882100 >> v2:172.21.15.190:3300/0 conn(0x7fb11c151920 0x7fb11c0bb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:30.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.024+0000 7fb1246db700 1 -- 172.21.15.121:0/3763882100 >> 172.21.15.121:0/3763882100 conn(0x7fb11c067a60 msgr2=0x7fb11c067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:30.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.024+0000 7fb1246db700 1 -- 172.21.15.121:0/3763882100 shutdown_connections 2023-11-15T03:07:30.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.024+0000 7fb1246db700 1 -- 172.21.15.121:0/3763882100 wait complete. 2023-11-15T03:07:30.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.028+0000 7fb1246db700 1 Processor -- start 2023-11-15T03:07:30.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.028+0000 7fb1246db700 1 -- start start 2023-11-15T03:07:30.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.028+0000 7fb1246db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb11c067150 0x7fb11c136d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:30.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.028+0000 7fb1246db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb11c151920 0x7fb11c0aae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:30.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.028+0000 7fb1246db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb11c156270 0x7fb11c0ab350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:30.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.028+0000 7fb1246db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c152280 con 0x7fb11c156270 2023-11-15T03:07:30.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.028+0000 7fb1246db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c1530e0 con 0x7fb11c067150 2023-11-15T03:07:30.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.028+0000 7fb1246db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c137220 con 0x7fb11c151920 2023-11-15T03:07:30.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.028+0000 7fb122c78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb11c156270 0x7fb11c0ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:30.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.028+0000 7fb122c78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb11c156270 0x7fb11c0ab350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50228/0 (socket says 172.21.15.121:50228) 2023-11-15T03:07:30.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.028+0000 7fb122c78700 1 -- 172.21.15.121:0/2943352888 learned_addr learned my addr 172.21.15.121:0/2943352888 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:30.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.028+0000 7fb122477700 1 --2- 172.21.15.121:0/2943352888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb11c067150 0x7fb11c136d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:30.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.028+0000 7fb121c76700 1 --2- 172.21.15.121:0/2943352888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb11c151920 0x7fb11c0aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:30.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.028+0000 7fb122c78700 1 -- 172.21.15.121:0/2943352888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb11c151920 msgr2=0x7fb11c0aae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:30.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.028+0000 7fb122c78700 1 --2- 172.21.15.121:0/2943352888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb11c151920 0x7fb11c0aae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:30.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.028+0000 7fb122c78700 1 -- 172.21.15.121:0/2943352888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb11c067150 msgr2=0x7fb11c136d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:30.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.028+0000 7fb122c78700 1 --2- 172.21.15.121:0/2943352888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb11c067150 0x7fb11c136d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:30.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.028+0000 7fb122c78700 1 -- 172.21.15.121:0/2943352888 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c0092a0 con 0x7fb11c156270 2023-11-15T03:07:30.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.028+0000 7fb122c78700 1 --2- 172.21.15.121:0/2943352888 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb11c156270 0x7fb11c0ab350 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 rx=0x7fb11400c7f0 tx=0x7fb114001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:30.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.032+0000 7fb1127fc700 1 -- 172.21.15.121:0/2943352888 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb114005d30 con 0x7fb11c156270 2023-11-15T03:07:30.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.032+0000 7fb1127fc700 1 -- 172.21.15.121:0/2943352888 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb114005e90 con 0x7fb11c156270 2023-11-15T03:07:30.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.032+0000 7fb1127fc700 1 -- 172.21.15.121:0/2943352888 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1140072e0 con 0x7fb11c156270 2023-11-15T03:07:30.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.032+0000 7fb1246db700 1 -- 172.21.15.121:0/2943352888 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11c1524d0 con 0x7fb11c156270 2023-11-15T03:07:30.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.032+0000 7fb1246db700 1 -- 172.21.15.121:0/2943352888 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11c152a90 con 0x7fb11c156270 2023-11-15T03:07:30.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.032+0000 7fb1246db700 1 -- 172.21.15.121:0/2943352888 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11c047060 con 0x7fb11c156270 2023-11-15T03:07:30.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.032+0000 7fb1127fc700 1 -- 172.21.15.121:0/2943352888 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb114007710 con 0x7fb11c156270 2023-11-15T03:07:30.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.036+0000 7fb1127fc700 1 --2- 172.21.15.121:0/2943352888 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb10005a510 0x7fb10005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:30.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.036+0000 7fb1127fc700 1 -- 172.21.15.121:0/2943352888 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8158+0+0 (secure 0 0 0) 0x7fb114074990 con 0x7fb11c156270 2023-11-15T03:07:30.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.036+0000 7fb122477700 1 --2- 172.21.15.121:0/2943352888 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb10005a510 0x7fb10005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:30.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.036+0000 7fb122477700 1 --2- 172.21.15.121:0/2943352888 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb10005a510 0x7fb10005c9d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7fb118009c30 tx=0x7fb118009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:30.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.040+0000 7fb1127fc700 1 -- 172.21.15.121:0/2943352888 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb1140478c0 con 0x7fb11c156270 2023-11-15T03:07:30.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:30.248+0000 7fb1246db700 1 -- 172.21.15.121:0/2943352888 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb11c0c35c0 con 0x7fb11c156270 2023-11-15T03:07:31.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.148+0000 7fb1127fc700 1 -- 172.21.15.121:0/2943352888 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v218) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb11404aee0 con 0x7fb11c156270 2023-11-15T03:07:31.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.152+0000 7fb1246db700 1 -- 172.21.15.121:0/2943352888 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb10005a510 msgr2=0x7fb10005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:31.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.152+0000 7fb1246db700 1 --2- 172.21.15.121:0/2943352888 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb10005a510 0x7fb10005c9d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7fb118009c30 tx=0x7fb118009060).stop 2023-11-15T03:07:31.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.152+0000 7fb1246db700 1 -- 172.21.15.121:0/2943352888 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb11c156270 msgr2=0x7fb11c0ab350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:31.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.152+0000 7fb1246db700 1 --2- 172.21.15.121:0/2943352888 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb11c156270 0x7fb11c0ab350 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 rx=0x7fb11400c7f0 tx=0x7fb114001510).stop 2023-11-15T03:07:31.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.152+0000 7fb1246db700 1 -- 172.21.15.121:0/2943352888 shutdown_connections 2023-11-15T03:07:31.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.152+0000 7fb1246db700 1 --2- 172.21.15.121:0/2943352888 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb10005a510 0x7fb10005c9d0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:31.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.152+0000 7fb1246db700 1 --2- 172.21.15.121:0/2943352888 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb11c156270 0x7fb11c0ab350 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:31.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.152+0000 7fb1246db700 1 --2- 172.21.15.121:0/2943352888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb11c151920 0x7fb11c0aae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:31.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.152+0000 7fb1246db700 1 --2- 172.21.15.121:0/2943352888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb11c067150 0x7fb11c136d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:31.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.152+0000 7fb1246db700 1 -- 172.21.15.121:0/2943352888 >> 172.21.15.121:0/2943352888 conn(0x7fb11c067a60 msgr2=0x7fb11c14d510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:31.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.152+0000 7fb1246db700 1 -- 172.21.15.121:0/2943352888 shutdown_connections 2023-11-15T03:07:31.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.152+0000 7fb1246db700 1 -- 172.21.15.121:0/2943352888 wait complete. 2023-11-15T03:07:31.163 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:07:31.182 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:07:31.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.336+0000 7f73e09db700 1 Processor -- start 2023-11-15T03:07:31.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.336+0000 7f73e09db700 1 -- start start 2023-11-15T03:07:31.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.340+0000 7f73e09db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73dc0a8de0 0x7f73dc066420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:31.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.340+0000 7f73e09db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f73dc150f90 0x7f73dc151410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:31.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.340+0000 7f73e09db700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73dc065f90 con 0x7f73dc062190 2023-11-15T03:07:31.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.340+0000 7f73e09db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73dc151900 con 0x7f73dc150f90 2023-11-15T03:07:31.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.340+0000 7f73e09db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc147ea0 con 0x7f73dc0a8de0 2023-11-15T03:07:31.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.340+0000 7f73d9d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f73dc150f90 0x7f73dc151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:31.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.340+0000 7f73da59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73dc0a8de0 0x7f73dc066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:31.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.340+0000 7f73d9d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f73dc150f90 0x7f73dc151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:54692/0 (socket says 172.21.15.121:54692) 2023-11-15T03:07:31.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.340+0000 7f73d9d9b700 1 -- 172.21.15.121:0/516541127 learned_addr learned my addr 172.21.15.121:0/516541127 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:31.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.340+0000 7f73d9d9b700 1 -- 172.21.15.121:0/516541127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73dc0a8de0 msgr2=0x7f73dc066420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:31.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.340+0000 7f73d9d9b700 1 --2- 172.21.15.121:0/516541127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73dc0a8de0 0x7f73dc066420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:31.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.340+0000 7f73d9d9b700 1 -- 172.21.15.121:0/516541127 >> v1:172.21.15.190:6789/0 conn(0x7f73dc062190 legacy=0x7f73dc152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:31.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.340+0000 7f73d9d9b700 1 -- 172.21.15.121:0/516541127 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73dc151e30 con 0x7f73dc150f90 2023-11-15T03:07:31.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.340+0000 7f73d9d9b700 1 --2- 172.21.15.121:0/516541127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73dc150f90 0x7f73dc151410 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f73d0006fd0 tx=0x7f73d000b3b0).ready entity=mon.1 client_cookie=75350ea407d5e7e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:31.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.340+0000 7f73d959a700 1 -- 172.21.15.121:0/516541127 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d000da70 con 0x7f73dc150f90 2023-11-15T03:07:31.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.340+0000 7f73d959a700 1 -- 172.21.15.121:0/516541127 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d000dbd0 con 0x7f73dc150f90 2023-11-15T03:07:31.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.340+0000 7f73d959a700 1 -- 172.21.15.121:0/516541127 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d0010470 con 0x7f73dc150f90 2023-11-15T03:07:31.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.344+0000 7f73e09db700 1 -- 172.21.15.121:0/516541127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73dc150f90 msgr2=0x7f73dc151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:31.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.344+0000 7f73e09db700 1 --2- 172.21.15.121:0/516541127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73dc150f90 0x7f73dc151410 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f73d0006fd0 tx=0x7f73d000b3b0).stop 2023-11-15T03:07:31.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.344+0000 7f73e09db700 1 -- 172.21.15.121:0/516541127 shutdown_connections 2023-11-15T03:07:31.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.344+0000 7f73e09db700 1 --2- 172.21.15.121:0/516541127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73dc150f90 0x7f73dc151410 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:31.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.344+0000 7f73e09db700 1 --2- 172.21.15.121:0/516541127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73dc0a8de0 0x7f73dc066420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:31.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.344+0000 7f73e09db700 1 -- 172.21.15.121:0/516541127 >> 172.21.15.121:0/516541127 conn(0x7f73dc067a60 msgr2=0x7f73dc067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:31.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.344+0000 7f73e09db700 1 -- 172.21.15.121:0/516541127 shutdown_connections 2023-11-15T03:07:31.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.344+0000 7f73e09db700 1 -- 172.21.15.121:0/516541127 wait complete. 2023-11-15T03:07:31.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.348+0000 7f73e09db700 1 Processor -- start 2023-11-15T03:07:31.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.348+0000 7f73e09db700 1 -- start start 2023-11-15T03:07:31.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.348+0000 7f73e09db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73dc062190 0x7f73dc14b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:31.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.348+0000 7f73e09db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73dc0a8de0 0x7f73dc146a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:31.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.348+0000 7f73e09db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73dc150f90 0x7f73dc14cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:31.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.348+0000 7f73e09db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc16c080 con 0x7f73dc062190 2023-11-15T03:07:31.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.348+0000 7f73e09db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc16c360 con 0x7f73dc150f90 2023-11-15T03:07:31.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.348+0000 7f73e09db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc16c640 con 0x7f73dc0a8de0 2023-11-15T03:07:31.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.348+0000 7f73da59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73dc062190 0x7f73dc14b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:31.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.348+0000 7f73da59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73dc062190 0x7f73dc14b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50234/0 (socket says 172.21.15.121:50234) 2023-11-15T03:07:31.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.348+0000 7f73da59c700 1 -- 172.21.15.121:0/4065986873 learned_addr learned my addr 172.21.15.121:0/4065986873 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:31.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.348+0000 7f73dad9d700 1 --2- 172.21.15.121:0/4065986873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73dc150f90 0x7f73dc14cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:31.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.348+0000 7f73d9d9b700 1 --2- 172.21.15.121:0/4065986873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73dc0a8de0 0x7f73dc146a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:31.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.348+0000 7f73da59c700 1 -- 172.21.15.121:0/4065986873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73dc0a8de0 msgr2=0x7f73dc146a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:31.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.348+0000 7f73da59c700 1 --2- 172.21.15.121:0/4065986873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73dc0a8de0 0x7f73dc146a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:31.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.348+0000 7f73da59c700 1 -- 172.21.15.121:0/4065986873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73dc150f90 msgr2=0x7f73dc14cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:31.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.348+0000 7f73da59c700 1 --2- 172.21.15.121:0/4065986873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73dc150f90 0x7f73dc14cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:31.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.348+0000 7f73da59c700 1 -- 172.21.15.121:0/4065986873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d000b610 con 0x7f73dc062190 2023-11-15T03:07:31.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.348+0000 7f73dad9d700 1 --2- 172.21.15.121:0/4065986873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73dc150f90 0x7f73dc14cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:07:31.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.352+0000 7f73da59c700 1 --2- 172.21.15.121:0/4065986873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73dc062190 0x7f73dc14b5e0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 rx=0x7f73cc00e930 tx=0x7f73cc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:31.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.352+0000 7f73ca7fc700 1 -- 172.21.15.121:0/4065986873 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc012600 con 0x7f73dc062190 2023-11-15T03:07:31.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.352+0000 7f73ca7fc700 1 -- 172.21.15.121:0/4065986873 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73cc012760 con 0x7f73dc062190 2023-11-15T03:07:31.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.352+0000 7f73ca7fc700 1 -- 172.21.15.121:0/4065986873 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc012ae0 con 0x7f73dc062190 2023-11-15T03:07:31.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.352+0000 7f73e09db700 1 -- 172.21.15.121:0/4065986873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73dc16c920 con 0x7f73dc062190 2023-11-15T03:07:31.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.352+0000 7f73e09db700 1 -- 172.21.15.121:0/4065986873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73dc146f30 con 0x7f73dc062190 2023-11-15T03:07:31.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.352+0000 7f73e09db700 1 -- 172.21.15.121:0/4065986873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73dc1471b0 con 0x7f73dc062190 2023-11-15T03:07:31.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.352+0000 7f73ca7fc700 1 -- 172.21.15.121:0/4065986873 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f73cc01b400 con 0x7f73dc062190 2023-11-15T03:07:31.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.356+0000 7f73ca7fc700 1 --2- 172.21.15.121:0/4065986873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f73bc05a510 0x7f73bc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:31.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.356+0000 7f73ca7fc700 1 -- 172.21.15.121:0/4065986873 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8158+0+0 (secure 0 0 0) 0x7f73cc0754a0 con 0x7f73dc062190 2023-11-15T03:07:31.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.356+0000 7f73d9d9b700 1 --2- 172.21.15.121:0/4065986873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f73bc05a510 0x7f73bc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:31.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.356+0000 7f73d9d9b700 1 --2- 172.21.15.121:0/4065986873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f73bc05a510 0x7f73bc05c9d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f73d0006fd0 tx=0x7f73dc065f00).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:31.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.364+0000 7f73ca7fc700 1 -- 172.21.15.121:0/4065986873 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f73cc048430 con 0x7f73dc062190 2023-11-15T03:07:31.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:31.588+0000 7f73e09db700 1 -- 172.21.15.121:0/4065986873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f73dc047060 con 0x7f73dc062190 2023-11-15T03:07:32.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.156+0000 7f73ca7fc700 1 -- 172.21.15.121:0/4065986873 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v219) v1 ==== 101+0+0 (secure 0 0 0) 0x7f73cc04ba50 con 0x7f73dc062190 2023-11-15T03:07:32.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.164+0000 7f73e09db700 1 -- 172.21.15.121:0/4065986873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f73bc05a510 msgr2=0x7f73bc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.164+0000 7f73e09db700 1 --2- 172.21.15.121:0/4065986873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f73bc05a510 0x7f73bc05c9d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f73d0006fd0 tx=0x7f73dc065f00).stop 2023-11-15T03:07:32.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.164+0000 7f73e09db700 1 -- 172.21.15.121:0/4065986873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73dc062190 msgr2=0x7f73dc14b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.164+0000 7f73e09db700 1 --2- 172.21.15.121:0/4065986873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73dc062190 0x7f73dc14b5e0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 rx=0x7f73cc00e930 tx=0x7f73cc009060).stop 2023-11-15T03:07:32.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.164+0000 7f73e09db700 1 -- 172.21.15.121:0/4065986873 shutdown_connections 2023-11-15T03:07:32.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.164+0000 7f73e09db700 1 --2- 172.21.15.121:0/4065986873 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f73bc05a510 0x7f73bc05c9d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.164+0000 7f73e09db700 1 --2- 172.21.15.121:0/4065986873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73dc150f90 0x7f73dc14cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.164+0000 7f73e09db700 1 --2- 172.21.15.121:0/4065986873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73dc0a8de0 0x7f73dc146a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.164+0000 7f73e09db700 1 --2- 172.21.15.121:0/4065986873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73dc062190 0x7f73dc14b5e0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.164+0000 7f73e09db700 1 -- 172.21.15.121:0/4065986873 >> 172.21.15.121:0/4065986873 conn(0x7f73dc067a60 msgr2=0x7f73dc153810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:32.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.164+0000 7f73e09db700 1 -- 172.21.15.121:0/4065986873 shutdown_connections 2023-11-15T03:07:32.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.164+0000 7f73e09db700 1 -- 172.21.15.121:0/4065986873 wait complete. 2023-11-15T03:07:32.173 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:07:32.190 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:07:32.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.240+0000 7f16e893cd00 1 Processor -- start 2023-11-15T03:07:32.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.240+0000 7f16e893cd00 1 -- start start 2023-11-15T03:07:32.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16e893cd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559ce16ce6c0 0x559ce17a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16e893cd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ce1718b00 0x559ce16fdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16e893cd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ce17a7670 0x559ce16ffb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16e893cd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce1647220 con 0x559ce16ce6c0 2023-11-15T03:07:32.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16e893cd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce17187f0 con 0x559ce17a7670 2023-11-15T03:07:32.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16e893cd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ce1704040 con 0x559ce1718b00 2023-11-15T03:07:32.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16dd019700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559ce16ce6c0 0x559ce17a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16dd019700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559ce16ce6c0 0x559ce17a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50252/0 (socket says 172.21.15.121:50252) 2023-11-15T03:07:32.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16dd019700 1 -- 172.21.15.121:0/1153830618 learned_addr learned my addr 172.21.15.121:0/1153830618 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:32.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16d4818700 1 --2- 172.21.15.121:0/1153830618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ce1718b00 0x559ce16fdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16dd81a700 1 --2- 172.21.15.121:0/1153830618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ce17a7670 0x559ce16ffb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16dd019700 1 -- 172.21.15.121:0/1153830618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ce1718b00 msgr2=0x559ce16fdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:32.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16dd019700 1 --2- 172.21.15.121:0/1153830618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ce1718b00 0x559ce16fdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16dd019700 1 -- 172.21.15.121:0/1153830618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ce17a7670 msgr2=0x559ce16ffb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:32.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16dd019700 1 --2- 172.21.15.121:0/1153830618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ce17a7670 0x559ce16ffb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16dd019700 1 -- 172.21.15.121:0/1153830618 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c8005cf0 con 0x559ce16ce6c0 2023-11-15T03:07:32.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16dd019700 1 --2- 172.21.15.121:0/1153830618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559ce16ce6c0 0x559ce17a0ca0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 rx=0x7f16d000ab90 tx=0x7f16d0015000).ready entity=mon.0 client_cookie=14769af51b948076 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:32.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16d7fff700 1 -- 172.21.15.121:0/1153830618 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d000d470 con 0x559ce16ce6c0 2023-11-15T03:07:32.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16d7fff700 1 -- 172.21.15.121:0/1153830618 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d00056d0 con 0x559ce16ce6c0 2023-11-15T03:07:32.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16d7fff700 1 -- 172.21.15.121:0/1153830618 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d000dd00 con 0x559ce16ce6c0 2023-11-15T03:07:32.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16e893cd00 1 -- 172.21.15.121:0/1153830618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559ce16ce6c0 msgr2=0x559ce17a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1153830618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559ce16ce6c0 0x559ce17a0ca0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 rx=0x7f16d000ab90 tx=0x7f16d0015000).stop 2023-11-15T03:07:32.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16e893cd00 1 -- 172.21.15.121:0/1153830618 shutdown_connections 2023-11-15T03:07:32.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1153830618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ce17a7670 0x559ce16ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1153830618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ce1718b00 0x559ce16fdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1153830618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559ce16ce6c0 0x559ce17a0ca0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16e893cd00 1 -- 172.21.15.121:0/1153830618 >> 172.21.15.121:0/1153830618 conn(0x559ce171e080 msgr2=0x559ce1712fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:32.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16e893cd00 1 -- 172.21.15.121:0/1153830618 shutdown_connections 2023-11-15T03:07:32.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.244+0000 7f16e893cd00 1 -- 172.21.15.121:0/1153830618 wait complete. 2023-11-15T03:07:32.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16e893cd00 1 Processor -- start 2023-11-15T03:07:32.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16e893cd00 1 -- start start 2023-11-15T03:07:32.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16e893cd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559ce16ce6c0 0x559ce179fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16e893cd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ce1718b00 0x559ce1716f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16e893cd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ce17a7670 0x559ce17173f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16e893cd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce1716650 con 0x559ce16ce6c0 2023-11-15T03:07:32.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16e893cd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce16fe1b0 con 0x559ce17a7670 2023-11-15T03:07:32.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16e893cd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ce1716ae0 con 0x559ce1718b00 2023-11-15T03:07:32.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16dd019700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559ce16ce6c0 0x559ce179fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16dd019700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559ce16ce6c0 0x559ce179fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50262/0 (socket says 172.21.15.121:50262) 2023-11-15T03:07:32.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 learned_addr learned my addr 172.21.15.121:0/1300448110 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:32.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16d4818700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ce1718b00 0x559ce1716f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16dd81a700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ce17a7670 0x559ce17173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ce1718b00 msgr2=0x559ce1716f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16dd019700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ce1718b00 0x559ce1716f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ce17a7670 msgr2=0x559ce17173f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16dd019700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ce17a7670 0x559ce17173f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ce1670a20 con 0x559ce16ce6c0 2023-11-15T03:07:32.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16dd019700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559ce16ce6c0 0x559ce179fc10 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 rx=0x7f16d0005e30 tx=0x7f16d001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:32.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16d67fc700 1 -- 172.21.15.121:0/1300448110 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d000d260 con 0x559ce16ce6c0 2023-11-15T03:07:32.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16d67fc700 1 -- 172.21.15.121:0/1300448110 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d000d3c0 con 0x559ce16ce6c0 2023-11-15T03:07:32.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16d67fc700 1 -- 172.21.15.121:0/1300448110 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d0024660 con 0x559ce16ce6c0 2023-11-15T03:07:32.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d000a0e0 con 0x559ce16ce6c0 2023-11-15T03:07:32.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.248+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ce1670770 con 0x559ce16ce6c0 2023-11-15T03:07:32.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.252+0000 7f16d67fc700 1 -- 172.21.15.121:0/1300448110 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f16d0021030 con 0x559ce16ce6c0 2023-11-15T03:07:32.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.252+0000 7f16d67fc700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f16c0058ff0 0x7f16c005b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.252+0000 7f16d67fc700 1 -- 172.21.15.121:0/1300448110 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8158+0+0 (secure 0 0 0) 0x7f16d00755a0 con 0x559ce16ce6c0 2023-11-15T03:07:32.274 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:07:32.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.252+0000 7f16d4818700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f16c0058ff0 0x7f16c005b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.256+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x559ce17038d0 0x559ce17b9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.256+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17b99f0 con 0x559ce17038d0 2023-11-15T03:07:32.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.256+0000 7f16dd81a700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x559ce17038d0 0x559ce17b9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.256+0000 7f16d4818700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f16c0058ff0 0x7f16c005b4b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x559ce17bb570 tx=0x7f16c800c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:32.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.256+0000 7f16dd81a700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x559ce17038d0 0x559ce17b9500 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:32.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.260+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v116'257 uv250 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17038d0 2023-11-15T03:07:32.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.260+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17c4660 con 0x559ce17038d0 2023-11-15T03:07:32.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.260+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v116'258 uv258 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17038d0 2023-11-15T03:07:32.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.260+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17c4a10 con 0x559ce17038d0 2023-11-15T03:07:32.277 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:07:32.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.264+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v116'259 uv259 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17038d0 2023-11-15T03:07:32.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.264+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17c5dd0 con 0x559ce17038d0 2023-11-15T03:07:32.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.264+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v116'260 uv260 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17038d0 2023-11-15T03:07:32.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.264+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17c6180 con 0x559ce17038d0 2023-11-15T03:07:32.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.268+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v116'261 uv261 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17038d0 2023-11-15T03:07:32.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.268+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17c4dc0 con 0x559ce17038d0 2023-11-15T03:07:32.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.272+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v116'262 uv262 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17038d0 2023-11-15T03:07:32.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.272+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17c5170 con 0x559ce17038d0 2023-11-15T03:07:32.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.272+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v116'263 uv256 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17038d0 2023-11-15T03:07:32.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.272+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17c5520 con 0x559ce17038d0 2023-11-15T03:07:32.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.276+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v116'264 uv264 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17038d0 2023-11-15T03:07:32.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.276+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17c5520 con 0x559ce17038d0 2023-11-15T03:07:32.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.276+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v116'265 uv265 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17038d0 2023-11-15T03:07:32.286 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:07:32.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.276+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17c5520 con 0x559ce17038d0 2023-11-15T03:07:32.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.280+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v116'266 uv266 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17038d0 2023-11-15T03:07:32.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.280+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17c5520 con 0x559ce17038d0 2023-11-15T03:07:32.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.280+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v116'267 uv267 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17038d0 2023-11-15T03:07:32.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.280+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17c5520 con 0x559ce17038d0 2023-11-15T03:07:32.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.284+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v116'268 uv268 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17038d0 2023-11-15T03:07:32.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.284+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x559ce17c6530 0x559ce17c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.284+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17c8e10 con 0x559ce17c6530 2023-11-15T03:07:32.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.284+0000 7f16dd019700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x559ce17c6530 0x559ce17c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.284+0000 7f16dd019700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x559ce17c6530 0x559ce17c8920 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:32.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.288+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v116'404 uv391 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.288+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17c91c0 con 0x559ce17c6530 2023-11-15T03:07:32.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.292+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v116'405 uv405 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.292+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17c9570 con 0x559ce17c6530 2023-11-15T03:07:32.304 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:07:32.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.296+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v116'406 uv406 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.296+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ca930 con 0x559ce17c6530 2023-11-15T03:07:32.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.300+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v116'407 uv407 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.300+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17cace0 con 0x559ce17c6530 2023-11-15T03:07:32.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.300+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v116'408 uv408 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.304+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17c9920 con 0x559ce17c6530 2023-11-15T03:07:32.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.304+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v116'409 uv409 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.304+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17c9d20 con 0x559ce17c6530 2023-11-15T03:07:32.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.308+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v116'410 uv410 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.308+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ca140 con 0x559ce17c6530 2023-11-15T03:07:32.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.308+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v116'411 uv397 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.308+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17ca140 con 0x559ce17c6530 2023-11-15T03:07:32.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.312+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v116'412 uv412 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.312+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ca540 con 0x559ce17c6530 2023-11-15T03:07:32.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.316+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v116'413 uv413 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.323 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:07:32.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.316+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ca540 con 0x559ce17c6530 2023-11-15T03:07:32.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.320+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v116'414 uv414 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.320+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ca0f0 con 0x559ce17c6530 2023-11-15T03:07:32.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.320+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v116'415 uv415 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.320+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ca0f0 con 0x559ce17c6530 2023-11-15T03:07:32.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.324+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v116'416 uv416 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.324+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ca0f0 con 0x559ce17c6530 2023-11-15T03:07:32.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.328+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v116'417 uv403 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.328+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17ca0f0 con 0x559ce17c6530 2023-11-15T03:07:32.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.328+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v116'418 uv418 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.328+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ca0f0 con 0x559ce17c6530 2023-11-15T03:07:32.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.332+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v116'419 uv419 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.340 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:07:32.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.332+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ca0d0 con 0x559ce17c6530 2023-11-15T03:07:32.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.336+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v116'420 uv420 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.336+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ca0d0 con 0x559ce17c6530 2023-11-15T03:07:32.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.336+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v116'421 uv421 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.336+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ca0d0 con 0x559ce17c6530 2023-11-15T03:07:32.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.340+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v116'422 uv422 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.340+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x559ce17cb090 0x559ce17cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.340+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17cd970 con 0x559ce17cb090 2023-11-15T03:07:32.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.340+0000 7f16d4818700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x559ce17cb090 0x559ce17cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.344+0000 7f16d4818700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x559ce17cb090 0x559ce17cd480 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:32.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.344+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v116'221 uv215 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17cb090 2023-11-15T03:07:32.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.344+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17cdd20 con 0x559ce17cb090 2023-11-15T03:07:32.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.348+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v116'222 uv222 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17cb090 2023-11-15T03:07:32.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.348+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ce0d0 con 0x559ce17cb090 2023-11-15T03:07:32.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.352+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v116'223 uv223 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17cb090 2023-11-15T03:07:32.360 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:07:32.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.352+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ce970 con 0x559ce17cb090 2023-11-15T03:07:32.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.356+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v116'224 uv224 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17cb090 2023-11-15T03:07:32.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.356+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ced20 con 0x559ce17cb090 2023-11-15T03:07:32.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.360+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v116'225 uv225 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17cb090 2023-11-15T03:07:32.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.360+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17cf0d0 con 0x559ce17cb090 2023-11-15T03:07:32.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.360+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v116'226 uv220 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17cb090 2023-11-15T03:07:32.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.360+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17cf480 con 0x559ce17cb090 2023-11-15T03:07:32.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.364+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v116'227 uv227 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17cb090 2023-11-15T03:07:32.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.364+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17cf480 con 0x559ce17cb090 2023-11-15T03:07:32.374 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:07:32.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.368+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v116'228 uv228 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17cb090 2023-11-15T03:07:32.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.368+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17cf480 con 0x559ce17cb090 2023-11-15T03:07:32.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.368+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v116'229 uv229 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17cb090 2023-11-15T03:07:32.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.368+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17cf480 con 0x559ce17cb090 2023-11-15T03:07:32.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.372+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v116'230 uv230 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17cb090 2023-11-15T03:07:32.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.372+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x559ce17cf990 0x559ce17d1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.372+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d2270 con 0x559ce17cf990 2023-11-15T03:07:32.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.372+0000 7f16dd81a700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x559ce17cf990 0x559ce17d1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.372+0000 7f16dd81a700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x559ce17cf990 0x559ce17d1d80 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:32.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.376+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v116'263 uv256 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.376+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17d2330 con 0x559ce17cf990 2023-11-15T03:07:32.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.380+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v116'264 uv264 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.380+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d2730 con 0x559ce17cf990 2023-11-15T03:07:32.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.384+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v116'265 uv265 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.392 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:07:32.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.384+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d67c0 con 0x559ce17cf990 2023-11-15T03:07:32.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.388+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v116'266 uv266 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.388+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d67c0 con 0x559ce17cf990 2023-11-15T03:07:32.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.392+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v116'267 uv267 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.392+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d2730 con 0x559ce17cf990 2023-11-15T03:07:32.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.392+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v116'268 uv262 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.396+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17d2730 con 0x559ce17cf990 2023-11-15T03:07:32.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.396+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v116'269 uv269 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.396+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d2730 con 0x559ce17cf990 2023-11-15T03:07:32.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.400+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v116'270 uv270 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.409 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:07:32.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.400+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d3740 con 0x559ce17cf990 2023-11-15T03:07:32.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.404+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v116'271 uv271 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.404+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d3740 con 0x559ce17cf990 2023-11-15T03:07:32.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.408+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v116'272 uv272 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.408+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d2730 con 0x559ce17cf990 2023-11-15T03:07:32.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.408+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v116'273 uv273 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.408+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x559ce17d2db0 0x559ce17d51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.408+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d5720 con 0x559ce17d2db0 2023-11-15T03:07:32.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.408+0000 7f16dd019700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x559ce17d2db0 0x559ce17d51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.412+0000 7f16dd019700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x559ce17d2db0 0x559ce17d51a0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:32.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.412+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v116'129 uv128 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17d2db0 2023-11-15T03:07:32.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.412+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17d57a0 con 0x559ce17d2db0 2023-11-15T03:07:32.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.416+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v116'130 uv130 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17d2db0 2023-11-15T03:07:32.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.416+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d5bd0 con 0x559ce17d2db0 2023-11-15T03:07:32.426 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:07:32.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.420+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v116'131 uv131 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17d2db0 2023-11-15T03:07:32.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.420+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d6be0 con 0x559ce17d2db0 2023-11-15T03:07:32.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.420+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v116'132 uv132 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17d2db0 2023-11-15T03:07:32.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.420+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d6be0 con 0x559ce17d2db0 2023-11-15T03:07:32.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.424+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v116'133 uv133 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17d2db0 2023-11-15T03:07:32.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.424+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d5bd0 con 0x559ce17d2db0 2023-11-15T03:07:32.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.424+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v116'134 uv134 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17d2db0 2023-11-15T03:07:32.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.424+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x559ce17d60e0 0x559ce17d8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.424+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d8c00 con 0x559ce17d60e0 2023-11-15T03:07:32.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.428+0000 7f16d4818700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x559ce17d60e0 0x559ce17d8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.428+0000 7f16d4818700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x559ce17d60e0 0x559ce17d8500 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:32.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.428+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v116'92 uv91 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.432+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17d8fe0 con 0x559ce17d60e0 2023-11-15T03:07:32.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.432+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v116'93 uv93 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.432+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d8fe0 con 0x559ce17d60e0 2023-11-15T03:07:32.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.436+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v116'94 uv94 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.444 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:07:32.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.436+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d8fe0 con 0x559ce17d60e0 2023-11-15T03:07:32.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.440+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v116'95 uv95 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.440+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d8fe0 con 0x559ce17d60e0 2023-11-15T03:07:32.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.440+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v116'129 uv128 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.440+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17d8fe0 con 0x559ce17d60e0 2023-11-15T03:07:32.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.444+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v116'130 uv130 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.444+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d8fe0 con 0x559ce17d60e0 2023-11-15T03:07:32.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.448+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v116'131 uv131 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.455 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:07:32.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.448+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d9ff0 con 0x559ce17d60e0 2023-11-15T03:07:32.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.448+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v116'132 uv132 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.448+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d9ff0 con 0x559ce17d60e0 2023-11-15T03:07:32.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.452+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v116'133 uv133 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.452+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d8fe0 con 0x559ce17d60e0 2023-11-15T03:07:32.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.456+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v116'134 uv134 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.456+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d8fe0 con 0x559ce17c6530 2023-11-15T03:07:32.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.460+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v116'238 uv232 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.460+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17d8fe0 con 0x559ce17c6530 2023-11-15T03:07:32.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.464+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v116'239 uv239 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.464+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d8fe0 con 0x559ce17c6530 2023-11-15T03:07:32.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.464+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v116'240 uv240 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.475 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:07:32.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.464+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d9ff0 con 0x559ce17c6530 2023-11-15T03:07:32.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.468+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v116'241 uv241 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.468+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d9ff0 con 0x559ce17c6530 2023-11-15T03:07:32.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.472+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v116'242 uv242 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.472+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d8fe0 con 0x559ce17c6530 2023-11-15T03:07:32.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.476+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v116'243 uv243 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.476+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d8fe0 con 0x559ce17c6530 2023-11-15T03:07:32.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.476+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v116'244 uv237 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.476+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17d8fe0 con 0x559ce17c6530 2023-11-15T03:07:32.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.480+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v116'245 uv245 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.480+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d8fe0 con 0x559ce17c6530 2023-11-15T03:07:32.491 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:07:32.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.484+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v116'246 uv246 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.484+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17d8fe0 con 0x559ce17c6530 2023-11-15T03:07:32.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.488+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v116'247 uv247 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.488+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17c6530 2023-11-15T03:07:32.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.492+0000 7f16dd019700 1 -- 172.21.15.121:0/1300448110 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v116'248 uv248 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d0072e40 con 0x559ce17c6530 2023-11-15T03:07:32.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.492+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17cb090 2023-11-15T03:07:32.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.492+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v116'107 uv106 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17cb090 2023-11-15T03:07:32.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.492+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17cb090 2023-11-15T03:07:32.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.496+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v116'108 uv108 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17cb090 2023-11-15T03:07:32.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.496+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17cb090 2023-11-15T03:07:32.506 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:07:32.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.500+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v116'109 uv109 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17cb090 2023-11-15T03:07:32.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.500+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17cb090 2023-11-15T03:07:32.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.500+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v116'110 uv110 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17cb090 2023-11-15T03:07:32.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.500+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17cb090 2023-11-15T03:07:32.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.504+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v116'111 uv111 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17cb090 2023-11-15T03:07:32.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.504+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17d60e0 2023-11-15T03:07:32.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.508+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v116'129 uv128 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.508+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17d60e0 2023-11-15T03:07:32.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.508+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v116'130 uv130 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.508+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17d60e0 2023-11-15T03:07:32.519 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:07:32.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.512+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v116'131 uv131 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.512+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17d60e0 2023-11-15T03:07:32.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.516+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v116'132 uv132 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.516+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17d60e0 2023-11-15T03:07:32.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.520+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v116'133 uv133 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.520+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17d60e0 2023-11-15T03:07:32.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.520+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v116'134 uv134 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.520+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17d60e0 2023-11-15T03:07:32.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.524+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v116'108 uv107 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.524+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17d60e0 2023-11-15T03:07:32.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.528+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v116'109 uv109 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.528+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17d60e0 2023-11-15T03:07:32.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.528+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v116'110 uv110 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.538 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:07:32.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.528+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17d60e0 2023-11-15T03:07:32.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.532+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v116'111 uv111 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.532+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17d60e0 2023-11-15T03:07:32.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.536+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v116'112 uv112 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.536+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17d60e0 2023-11-15T03:07:32.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.540+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v116'106 uv105 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.540+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17d60e0 2023-11-15T03:07:32.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.540+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v116'107 uv107 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.540+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ce500 con 0x559ce17d60e0 2023-11-15T03:07:32.552 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:07:32.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.544+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v116'108 uv108 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.544+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17ce540 con 0x559ce17d60e0 2023-11-15T03:07:32.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.548+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v116'109 uv109 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.548+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17dc290 con 0x559ce17d60e0 2023-11-15T03:07:32.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.552+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v116'110 uv110 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.552+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x559ce17da090 0x559ce17da470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.552+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17da090 2023-11-15T03:07:32.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.552+0000 7f16dd81a700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x559ce17da090 0x559ce17da470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.552+0000 7f16dd81a700 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x559ce17da090 0x559ce17da470 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:32.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.556+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v116'86 uv85 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17da090 2023-11-15T03:07:32.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.556+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17da090 2023-11-15T03:07:32.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.560+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v116'87 uv87 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17da090 2023-11-15T03:07:32.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.560+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17da090 2023-11-15T03:07:32.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.560+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v116'88 uv88 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17da090 2023-11-15T03:07:32.570 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:07:32.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.560+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17da090 2023-11-15T03:07:32.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.564+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v116'89 uv89 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17da090 2023-11-15T03:07:32.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.564+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17d60e0 2023-11-15T03:07:32.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.568+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v116'129 uv128 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.568+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17d60e0 2023-11-15T03:07:32.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.572+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v116'130 uv130 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.572+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17d60e0 2023-11-15T03:07:32.581 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:07:32.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.572+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v116'131 uv131 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.572+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17d60e0 2023-11-15T03:07:32.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.576+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v116'132 uv132 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.576+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17d60e0 2023-11-15T03:07:32.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.580+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v116'133 uv133 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.580+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17d60e0 2023-11-15T03:07:32.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.580+0000 7f16d4818700 1 -- 172.21.15.121:0/1300448110 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v116'134 uv134 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16c8012040 con 0x559ce17d60e0 2023-11-15T03:07:32.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.580+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17cf990 2023-11-15T03:07:32.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.584+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v116'110 uv109 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.584+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17cf990 2023-11-15T03:07:32.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.588+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v116'111 uv111 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.588+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17cf990 2023-11-15T03:07:32.597 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:07:32.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.588+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v116'112 uv112 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.588+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17cf990 2023-11-15T03:07:32.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.592+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v116'113 uv113 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.592+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17dbe30 con 0x559ce17cf990 2023-11-15T03:07:32.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.596+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v116'114 uv114 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.596+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17dbe30 con 0x559ce17cf990 2023-11-15T03:07:32.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.600+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v116'256 uv248 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.600+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17cf990 2023-11-15T03:07:32.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.600+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v116'257 uv257 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.600+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17cf990 2023-11-15T03:07:32.611 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:07:32.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.604+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v116'258 uv258 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.604+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17dbec0 con 0x559ce17cf990 2023-11-15T03:07:32.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.608+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v116'259 uv259 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.608+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17dbe70 con 0x559ce17cf990 2023-11-15T03:07:32.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.608+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v116'260 uv260 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.612+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17dbe70 con 0x559ce17cf990 2023-11-15T03:07:32.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.612+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v116'261 uv255 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.612+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17cf990 2023-11-15T03:07:32.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.616+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v116'262 uv262 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.616+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17cf990 2023-11-15T03:07:32.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.620+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v116'263 uv263 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.626 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:07:32.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.620+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17dbe80 con 0x559ce17cf990 2023-11-15T03:07:32.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.620+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v116'264 uv264 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.620+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17dbe80 con 0x559ce17cf990 2023-11-15T03:07:32.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.624+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v116'265 uv265 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.624+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17cf990 2023-11-15T03:07:32.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.628+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v116'266 uv266 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.628+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x559ce17da9d0 con 0x559ce17cf990 2023-11-15T03:07:32.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v116'267 uv267 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f16d800d3d0 con 0x559ce17cf990 2023-11-15T03:07:32.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x559ce17cb090 msgr2=0x559ce17cd480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x559ce17cb090 0x559ce17cd480 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x559ce17038d0 msgr2=0x559ce17b9500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x559ce17038d0 0x559ce17b9500 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x559ce17d2db0 msgr2=0x559ce17d51a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x559ce17d2db0 0x559ce17d51a0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x559ce17c6530 msgr2=0x559ce17c8920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x559ce17c6530 0x559ce17c8920 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x559ce17da090 msgr2=0x559ce17da470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x559ce17da090 0x559ce17da470 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x559ce17cf990 msgr2=0x559ce17d1d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x559ce17cf990 0x559ce17d1d80 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x559ce17d60e0 msgr2=0x559ce17d8500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x559ce17d60e0 0x559ce17d8500 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 reap_dead start 2023-11-15T03:07:32.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 reap_dead start 2023-11-15T03:07:32.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 reap_dead start 2023-11-15T03:07:32.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16dd81a700 1 -- 172.21.15.121:0/1300448110 reap_dead start 2023-11-15T03:07:32.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f16c0058ff0 msgr2=0x7f16c005b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f16c0058ff0 0x7f16c005b4b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x559ce17bb570 tx=0x7f16c800c040).stop 2023-11-15T03:07:32.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559ce16ce6c0 msgr2=0x559ce179fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559ce16ce6c0 0x559ce179fc10 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 rx=0x7f16d0005e30 tx=0x7f16d001a000).stop 2023-11-15T03:07:32.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 shutdown_connections 2023-11-15T03:07:32.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f16c0058ff0 0x7f16c005b4b0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 --2- 172.21.15.121:0/1300448110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559ce16ce6c0 0x559ce179fc10 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 >> 172.21.15.121:0/1300448110 conn(0x559ce171e080 msgr2=0x559ce171b270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:32.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 shutdown_connections 2023-11-15T03:07:32.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.632+0000 7f16e893cd00 1 -- 172.21.15.121:0/1300448110 wait complete. 2023-11-15T03:07:32.649 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 100.00000000.inode - 2023-11-15T03:07:32.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.740+0000 7f04c4b07d00 1 Processor -- start 2023-11-15T03:07:32.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.740+0000 7f04c4b07d00 1 -- start start 2023-11-15T03:07:32.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c4b07d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1dc4e860 0x556b1dc7a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c4b07d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b1dd23ec0 0x556b1dc7bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c4b07d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b1dd242a0 0x556b1dc950e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c4b07d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b1dc78c80 con 0x556b1dd242a0 2023-11-15T03:07:32.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c4b07d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b1dd24c00 con 0x556b1dc4e860 2023-11-15T03:07:32.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c4b07d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b1dc77590 con 0x556b1dd23ec0 2023-11-15T03:07:32.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c2c46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b1dd242a0 0x556b1dc950e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c2c46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b1dd242a0 0x556b1dc950e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50282/0 (socket says 172.21.15.121:50282) 2023-11-15T03:07:32.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c2c46700 1 -- 172.21.15.121:0/604246592 learned_addr learned my addr 172.21.15.121:0/604246592 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:32.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c2445700 1 --2- 172.21.15.121:0/604246592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1dc4e860 0x556b1dc7a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c1c44700 1 --2- 172.21.15.121:0/604246592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b1dd23ec0 0x556b1dc7bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c2c46700 1 -- 172.21.15.121:0/604246592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b1dd23ec0 msgr2=0x556b1dc7bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:32.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c2c46700 1 --2- 172.21.15.121:0/604246592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b1dd23ec0 0x556b1dc7bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c2c46700 1 -- 172.21.15.121:0/604246592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1dc4e860 msgr2=0x556b1dc7a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:32.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c2c46700 1 --2- 172.21.15.121:0/604246592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1dc4e860 0x556b1dc7a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c2c46700 1 -- 172.21.15.121:0/604246592 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b4005db0 con 0x556b1dd242a0 2023-11-15T03:07:32.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c2c46700 1 --2- 172.21.15.121:0/604246592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b1dd242a0 0x556b1dc950e0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 rx=0x7f04bc00a530 tx=0x7f04bc005cf0).ready entity=mon.0 client_cookie=4bb9aafa662af02f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:32.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c0c42700 1 -- 172.21.15.121:0/604246592 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc001460 con 0x556b1dd242a0 2023-11-15T03:07:32.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c0c42700 1 -- 172.21.15.121:0/604246592 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc0015c0 con 0x556b1dd242a0 2023-11-15T03:07:32.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c0c42700 1 -- 172.21.15.121:0/604246592 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc001960 con 0x556b1dd242a0 2023-11-15T03:07:32.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c4b07d00 1 -- 172.21.15.121:0/604246592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b1dd242a0 msgr2=0x556b1dc950e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c4b07d00 1 --2- 172.21.15.121:0/604246592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b1dd242a0 0x556b1dc950e0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 rx=0x7f04bc00a530 tx=0x7f04bc005cf0).stop 2023-11-15T03:07:32.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c4b07d00 1 -- 172.21.15.121:0/604246592 shutdown_connections 2023-11-15T03:07:32.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c4b07d00 1 --2- 172.21.15.121:0/604246592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b1dd242a0 0x556b1dc950e0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c4b07d00 1 --2- 172.21.15.121:0/604246592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b1dd23ec0 0x556b1dc7bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c4b07d00 1 --2- 172.21.15.121:0/604246592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1dc4e860 0x556b1dc7a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c4b07d00 1 -- 172.21.15.121:0/604246592 >> 172.21.15.121:0/604246592 conn(0x556b1dc4c7d0 msgr2=0x556b1dc79580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:32.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c4b07d00 1 -- 172.21.15.121:0/604246592 shutdown_connections 2023-11-15T03:07:32.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.744+0000 7f04c4b07d00 1 -- 172.21.15.121:0/604246592 wait complete. 2023-11-15T03:07:32.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.748+0000 7f04c4b07d00 1 Processor -- start 2023-11-15T03:07:32.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.748+0000 7f04c4b07d00 1 -- start start 2023-11-15T03:07:32.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.748+0000 7f04c4b07d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1dc4e860 0x556b1dc965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.748+0000 7f04c4b07d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b1dd23ec0 0x556b1dc95060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.748+0000 7f04c4b07d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b1dd242a0 0x556b1dd1c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.748+0000 7f04c4b07d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b1dd21d30 con 0x556b1dd242a0 2023-11-15T03:07:32.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.748+0000 7f04c4b07d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b1dc96ae0 con 0x556b1dc4e860 2023-11-15T03:07:32.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.748+0000 7f04c4b07d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b1dd20e60 con 0x556b1dd23ec0 2023-11-15T03:07:32.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.752+0000 7f04c2c46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b1dd242a0 0x556b1dd1c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.752+0000 7f04c2445700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1dc4e860 0x556b1dc965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.752+0000 7f04c2445700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1dc4e860 0x556b1dc965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:54720/0 (socket says 172.21.15.121:54720) 2023-11-15T03:07:32.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.752+0000 7f04c1c44700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b1dd23ec0 0x556b1dc95060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.752+0000 7f04c2445700 1 -- 172.21.15.121:0/22637920 learned_addr learned my addr 172.21.15.121:0/22637920 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:32.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.752+0000 7f04c2c46700 1 -- 172.21.15.121:0/22637920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b1dd23ec0 msgr2=0x556b1dc95060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.752+0000 7f04c2c46700 1 --2- 172.21.15.121:0/22637920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b1dd23ec0 0x556b1dc95060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.752+0000 7f04c2c46700 1 -- 172.21.15.121:0/22637920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1dc4e860 msgr2=0x556b1dc965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.752+0000 7f04c2c46700 1 --2- 172.21.15.121:0/22637920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1dc4e860 0x556b1dc965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.752+0000 7f04c2c46700 1 -- 172.21.15.121:0/22637920 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04bc00a0e0 con 0x556b1dd242a0 2023-11-15T03:07:32.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.752+0000 7f04c2c46700 1 --2- 172.21.15.121:0/22637920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b1dd242a0 0x556b1dd1c890 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 rx=0x7f04bc011000 tx=0x7f04bc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:32.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.752+0000 7f04b2ffd700 1 -- 172.21.15.121:0/22637920 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0116f0 con 0x556b1dd242a0 2023-11-15T03:07:32.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.752+0000 7f04b2ffd700 1 -- 172.21.15.121:0/22637920 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc011850 con 0x556b1dd242a0 2023-11-15T03:07:32.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.752+0000 7f04b2ffd700 1 -- 172.21.15.121:0/22637920 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc011c10 con 0x556b1dd242a0 2023-11-15T03:07:32.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.752+0000 7f04c4b07d00 1 -- 172.21.15.121:0/22637920 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556b1dbeda20 con 0x556b1dd242a0 2023-11-15T03:07:32.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.752+0000 7f04c4b07d00 1 -- 172.21.15.121:0/22637920 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b1dbed770 con 0x556b1dd242a0 2023-11-15T03:07:32.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.756+0000 7f04b2ffd700 1 -- 172.21.15.121:0/22637920 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f04bc01c3f0 con 0x556b1dd242a0 2023-11-15T03:07:32.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.756+0000 7f04b2ffd700 1 --2- 172.21.15.121:0/22637920 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f04a4058ff0 0x7f04a405b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.756+0000 7f04b2ffd700 1 -- 172.21.15.121:0/22637920 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8158+0+0 (secure 0 0 0) 0x7f04bc074b30 con 0x556b1dd242a0 2023-11-15T03:07:32.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.756+0000 7f04c2445700 1 --2- 172.21.15.121:0/22637920 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f04a4058ff0 0x7f04a405b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.756+0000 7f04c4b07d00 1 --2- 172.21.15.121:0/22637920 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556b1dd273b0 0x556b1dd38b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.756+0000 7f04c4b07d00 1 -- 172.21.15.121:0/22637920 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.3f 13:fe07bba1:::100.00000000.inode:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected e116) v8 -- 0x556b1dc95550 con 0x556b1dd273b0 2023-11-15T03:07:32.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.756+0000 7f04c1c44700 1 --2- 172.21.15.121:0/22637920 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556b1dd273b0 0x556b1dd38b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.756+0000 7f04c2445700 1 --2- 172.21.15.121:0/22637920 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f04a4058ff0 0x7f04a405b4b0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f04b4003080 tx=0x7f04b4016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:32.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.760+0000 7f04c1c44700 1 --2- 172.21.15.121:0/22637920 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556b1dd273b0 0x556b1dd38b40 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:32.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.764+0000 7f04c1c44700 1 -- 172.21.15.121:0/22637920 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 100.00000000.inode [writefull 0~80] v116'268 uv268 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f04b8012040 con 0x556b1dd273b0 2023-11-15T03:07:32.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.764+0000 7f04c4b07d00 1 -- 172.21.15.121:0/22637920 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556b1dd273b0 msgr2=0x556b1dd38b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.764+0000 7f04c4b07d00 1 --2- 172.21.15.121:0/22637920 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x556b1dd273b0 0x556b1dd38b40 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.764+0000 7f04c4b07d00 1 -- 172.21.15.121:0/22637920 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f04a4058ff0 msgr2=0x7f04a405b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.764+0000 7f04c4b07d00 1 --2- 172.21.15.121:0/22637920 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f04a4058ff0 0x7f04a405b4b0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f04b4003080 tx=0x7f04b4016000).stop 2023-11-15T03:07:32.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.764+0000 7f04c4b07d00 1 -- 172.21.15.121:0/22637920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b1dd242a0 msgr2=0x556b1dd1c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.764+0000 7f04c4b07d00 1 --2- 172.21.15.121:0/22637920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b1dd242a0 0x556b1dd1c890 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 rx=0x7f04bc011000 tx=0x7f04bc014000).stop 2023-11-15T03:07:32.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.764+0000 7f04c2c46700 1 -- 172.21.15.121:0/22637920 reap_dead start 2023-11-15T03:07:32.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.764+0000 7f04c4b07d00 1 -- 172.21.15.121:0/22637920 shutdown_connections 2023-11-15T03:07:32.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.764+0000 7f04c4b07d00 1 -- 172.21.15.121:0/22637920 >> 172.21.15.121:0/22637920 conn(0x556b1dc4c7d0 msgr2=0x556b1dc7bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:32.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.764+0000 7f04c4b07d00 1 -- 172.21.15.121:0/22637920 shutdown_connections 2023-11-15T03:07:32.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.764+0000 7f04c4b07d00 1 -- 172.21.15.121:0/22637920 wait complete. 2023-11-15T03:07:32.776 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:07:32.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.972+0000 7ff3eb59e700 1 Processor -- start 2023-11-15T03:07:32.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.972+0000 7ff3eb59e700 1 -- start start 2023-11-15T03:07:32.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.972+0000 7ff3eb59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3ec0acb60 0x7ff3ec0acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.972+0000 7ff3eb59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff3ec065680 0x7ff3ec065b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.972+0000 7ff3eb59e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ec0c7e90 con 0x7ff3ec14e380 2023-11-15T03:07:32.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.972+0000 7ff3eb59e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ec0c8320 con 0x7ff3ec065680 2023-11-15T03:07:32.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.972+0000 7ff3eb59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec0c7080 con 0x7ff3ec0acb60 2023-11-15T03:07:32.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.976+0000 7ff3ead9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff3ec065680 0x7ff3ec065b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.976+0000 7ff3e9d9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3ec0acb60 0x7ff3ec0acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.976+0000 7ff3ead9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff3ec065680 0x7ff3ec065b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:54728/0 (socket says 172.21.15.121:54728) 2023-11-15T03:07:32.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.976+0000 7ff3ead9d700 1 -- 172.21.15.121:0/751496622 learned_addr learned my addr 172.21.15.121:0/751496622 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:32.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.976+0000 7ff3ead9d700 1 -- 172.21.15.121:0/751496622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3ec0acb60 msgr2=0x7ff3ec0acfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:32.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.976+0000 7ff3ead9d700 1 --2- 172.21.15.121:0/751496622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3ec0acb60 0x7ff3ec0acfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.976+0000 7ff3ead9d700 1 -- 172.21.15.121:0/751496622 >> v1:172.21.15.190:6789/0 conn(0x7ff3ec14e380 legacy=0x7ff3ec0c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.976+0000 7ff3ead9d700 1 -- 172.21.15.121:0/751496622 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec0ad4d0 con 0x7ff3ec065680 2023-11-15T03:07:32.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.976+0000 7ff3ead9d700 1 --2- 172.21.15.121:0/751496622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ec065680 0x7ff3ec065b10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7ff3e4009f20 tx=0x7ff3e4009840).ready entity=mon.1 client_cookie=6888e980c711f41d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:32.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.976+0000 7ff3e959a700 1 -- 172.21.15.121:0/751496622 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e400e070 con 0x7ff3ec065680 2023-11-15T03:07:32.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.976+0000 7ff3e959a700 1 -- 172.21.15.121:0/751496622 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e400b9d0 con 0x7ff3ec065680 2023-11-15T03:07:32.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.976+0000 7ff3e959a700 1 -- 172.21.15.121:0/751496622 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e400bd50 con 0x7ff3ec065680 2023-11-15T03:07:32.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.976+0000 7ff3eb59e700 1 -- 172.21.15.121:0/751496622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ec065680 msgr2=0x7ff3ec065b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.976+0000 7ff3eb59e700 1 --2- 172.21.15.121:0/751496622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ec065680 0x7ff3ec065b10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7ff3e4009f20 tx=0x7ff3e4009840).stop 2023-11-15T03:07:32.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.980+0000 7ff3eb59e700 1 -- 172.21.15.121:0/751496622 shutdown_connections 2023-11-15T03:07:32.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.980+0000 7ff3eb59e700 1 --2- 172.21.15.121:0/751496622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ec065680 0x7ff3ec065b10 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.980+0000 7ff3eb59e700 1 --2- 172.21.15.121:0/751496622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3ec0acb60 0x7ff3ec0acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.980+0000 7ff3eb59e700 1 -- 172.21.15.121:0/751496622 >> 172.21.15.121:0/751496622 conn(0x7ff3ec067a60 msgr2=0x7ff3ec067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:32.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.980+0000 7ff3eb59e700 1 -- 172.21.15.121:0/751496622 shutdown_connections 2023-11-15T03:07:32.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.980+0000 7ff3eb59e700 1 -- 172.21.15.121:0/751496622 wait complete. 2023-11-15T03:07:32.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.980+0000 7ff3eb59e700 1 Processor -- start 2023-11-15T03:07:32.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.980+0000 7ff3eb59e700 1 -- start start 2023-11-15T03:07:32.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3eb59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3ec065680 0x7ff3ec141670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3eb59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ec0acb60 0x7ff3ec156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3eb59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3ec14e380 0x7ff3ec1569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3eb59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec156ee0 con 0x7ff3ec065680 2023-11-15T03:07:32.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3eb59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec155e20 con 0x7ff3ec0acb60 2023-11-15T03:07:32.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3eb59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec1562b0 con 0x7ff3ec14e380 2023-11-15T03:07:32.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3ea59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3ec065680 0x7ff3ec141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3ea59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3ec065680 0x7ff3ec141670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50298/0 (socket says 172.21.15.121:50298) 2023-11-15T03:07:32.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3ea59c700 1 -- 172.21.15.121:0/846841283 learned_addr learned my addr 172.21.15.121:0/846841283 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:32.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3ead9d700 1 --2- 172.21.15.121:0/846841283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3ec14e380 0x7ff3ec1569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3e9d9b700 1 --2- 172.21.15.121:0/846841283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ec0acb60 0x7ff3ec156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:32.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3ea59c700 1 -- 172.21.15.121:0/846841283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3ec14e380 msgr2=0x7ff3ec1569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3ea59c700 1 --2- 172.21.15.121:0/846841283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3ec14e380 0x7ff3ec1569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3ea59c700 1 -- 172.21.15.121:0/846841283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ec0acb60 msgr2=0x7ff3ec156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:32.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3ea59c700 1 --2- 172.21.15.121:0/846841283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ec0acb60 0x7ff3ec156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:32.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3ea59c700 1 -- 172.21.15.121:0/846841283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e4009ae0 con 0x7ff3ec065680 2023-11-15T03:07:32.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3ea59c700 1 --2- 172.21.15.121:0/846841283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3ec065680 0x7ff3ec141670 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 rx=0x7ff3dc00c440 tx=0x7ff3dc007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:32.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3da7fc700 1 -- 172.21.15.121:0/846841283 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc007020 con 0x7ff3ec065680 2023-11-15T03:07:32.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3da7fc700 1 -- 172.21.15.121:0/846841283 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc007180 con 0x7ff3ec065680 2023-11-15T03:07:32.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3eb59e700 1 -- 172.21.15.121:0/846841283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ec155210 con 0x7ff3ec065680 2023-11-15T03:07:32.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3da7fc700 1 -- 172.21.15.121:0/846841283 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc007520 con 0x7ff3ec065680 2023-11-15T03:07:32.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.984+0000 7ff3eb59e700 1 -- 172.21.15.121:0/846841283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ec1557d0 con 0x7ff3ec065680 2023-11-15T03:07:32.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.988+0000 7ff3eb59e700 1 -- 172.21.15.121:0/846841283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ec06bcf0 con 0x7ff3ec065680 2023-11-15T03:07:32.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.988+0000 7ff3da7fc700 1 -- 172.21.15.121:0/846841283 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff3dc0068b0 con 0x7ff3ec065680 2023-11-15T03:07:32.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.988+0000 7ff3da7fc700 1 --2- 172.21.15.121:0/846841283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff3cc05a510 0x7ff3cc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:32.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.992+0000 7ff3da7fc700 1 -- 172.21.15.121:0/846841283 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8158+0+0 (secure 0 0 0) 0x7ff3dc0742a0 con 0x7ff3ec065680 2023-11-15T03:07:32.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.992+0000 7ff3e9d9b700 1 --2- 172.21.15.121:0/846841283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff3cc05a510 0x7ff3cc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:33.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.992+0000 7ff3e9d9b700 1 --2- 172.21.15.121:0/846841283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff3cc05a510 0x7ff3cc05c9d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7ff3ec14adf0 tx=0x7ff3e0009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:33.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:32.996+0000 7ff3da7fc700 1 -- 172.21.15.121:0/846841283 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff3dc0471d0 con 0x7ff3ec065680 2023-11-15T03:07:33.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:33.192+0000 7ff3eb59e700 1 -- 172.21.15.121:0/846841283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7ff3ec068a70 con 0x7ff3ec065680 2023-11-15T03:07:34.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.176+0000 7ff3da7fc700 1 -- 172.21.15.121:0/846841283 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v220) v1 ==== 162+0+0 (secure 0 0 0) 0x7ff3dc04a7f0 con 0x7ff3ec065680 2023-11-15T03:07:34.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.180+0000 7ff3eb59e700 1 -- 172.21.15.121:0/846841283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff3cc05a510 msgr2=0x7ff3cc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:34.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.180+0000 7ff3eb59e700 1 --2- 172.21.15.121:0/846841283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff3cc05a510 0x7ff3cc05c9d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7ff3ec14adf0 tx=0x7ff3e0009060).stop 2023-11-15T03:07:34.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.180+0000 7ff3eb59e700 1 -- 172.21.15.121:0/846841283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3ec065680 msgr2=0x7ff3ec141670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:34.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.180+0000 7ff3eb59e700 1 --2- 172.21.15.121:0/846841283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3ec065680 0x7ff3ec141670 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 rx=0x7ff3dc00c440 tx=0x7ff3dc007e00).stop 2023-11-15T03:07:34.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.180+0000 7ff3eb59e700 1 -- 172.21.15.121:0/846841283 shutdown_connections 2023-11-15T03:07:34.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.180+0000 7ff3eb59e700 1 --2- 172.21.15.121:0/846841283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff3cc05a510 0x7ff3cc05c9d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:34.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.180+0000 7ff3eb59e700 1 --2- 172.21.15.121:0/846841283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3ec14e380 0x7ff3ec1569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:34.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.180+0000 7ff3eb59e700 1 --2- 172.21.15.121:0/846841283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ec0acb60 0x7ff3ec156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:34.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.180+0000 7ff3eb59e700 1 --2- 172.21.15.121:0/846841283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3ec065680 0x7ff3ec141670 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:34.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.180+0000 7ff3eb59e700 1 -- 172.21.15.121:0/846841283 >> 172.21.15.121:0/846841283 conn(0x7ff3ec067a60 msgr2=0x7ff3ec14d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:34.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.180+0000 7ff3eb59e700 1 -- 172.21.15.121:0/846841283 shutdown_connections 2023-11-15T03:07:34.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.180+0000 7ff3eb59e700 1 -- 172.21.15.121:0/846841283 wait complete. 2023-11-15T03:07:34.190 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:07:34.198 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:07:34.192+0000 7fa0b55a8700 -1 mds.0.cache.ino(0x100) Corrupt inode 0x100: End of buffer 2023-11-15T03:07:34.198 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:07:34.192+0000 7fa0b55a8700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2023-11-15T03:07:34.209 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:34.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.360+0000 7f0ceb71b700 1 Processor -- start 2023-11-15T03:07:34.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.360+0000 7f0ceb71b700 1 -- start start 2023-11-15T03:07:34.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.360+0000 7f0ceb71b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce40aa240 0x7f0ce4067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:34.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.360+0000 7f0ceb71b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0ce4067f50 0x7f0ce4065d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:34.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.360+0000 7f0ceb71b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce40cb5c0 con 0x7f0ce4153bb0 2023-11-15T03:07:34.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.360+0000 7f0ceb71b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce40aa6f0 con 0x7f0ce4067f50 2023-11-15T03:07:34.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.360+0000 7f0ceb71b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce40c5900 con 0x7f0ce40aa240 2023-11-15T03:07:34.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0ce9cb8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0ce4067f50 0x7f0ce4065d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:34.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0ce8cb6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce40aa240 0x7f0ce4067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:34.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0ce9cb8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0ce4067f50 0x7f0ce4065d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50304/0 (socket says 172.21.15.121:50304) 2023-11-15T03:07:34.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0ce9cb8700 1 -- 172.21.15.121:0/4184405282 learned_addr learned my addr 172.21.15.121:0/4184405282 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:34.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0ce94b7700 1 --1- 172.21.15.121:0/4184405282 >> v1:172.21.15.190:6789/0 conn(0x7f0ce4153bb0 0x7f0ce40a9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:58118/0 (socket says 172.21.15.121:58118) 2023-11-15T03:07:34.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0ce8cb6700 1 -- 172.21.15.121:0/4184405282 >> v2:172.21.15.121:3300/0 conn(0x7f0ce4067f50 msgr2=0x7f0ce4065d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:34.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0ce8cb6700 1 --2- 172.21.15.121:0/4184405282 >> v2:172.21.15.121:3300/0 conn(0x7f0ce4067f50 0x7f0ce4065d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:34.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0ce8cb6700 1 -- 172.21.15.121:0/4184405282 >> v1:172.21.15.190:6789/0 conn(0x7f0ce4153bb0 legacy=0x7f0ce40a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:34.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0ce8cb6700 1 -- 172.21.15.121:0/4184405282 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce40683e0 con 0x7f0ce40aa240 2023-11-15T03:07:34.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0ce8cb6700 1 --2- 172.21.15.121:0/4184405282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce40aa240 0x7f0ce4067a60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f0cd4009760 tx=0x7f0cd4009060).ready entity=mon.2 client_cookie=d413e26f7aa1ecc2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:34.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0cdbfff700 1 -- 172.21.15.121:0/4184405282 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd400f510 con 0x7f0ce40aa240 2023-11-15T03:07:34.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0cdbfff700 1 -- 172.21.15.121:0/4184405282 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd400f670 con 0x7f0ce40aa240 2023-11-15T03:07:34.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0cdbfff700 1 -- 172.21.15.121:0/4184405282 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd400f9f0 con 0x7f0ce40aa240 2023-11-15T03:07:34.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0ceb71b700 1 -- 172.21.15.121:0/4184405282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce40aa240 msgr2=0x7f0ce4067a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:34.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0ceb71b700 1 --2- 172.21.15.121:0/4184405282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce40aa240 0x7f0ce4067a60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f0cd4009760 tx=0x7f0cd4009060).stop 2023-11-15T03:07:34.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0ceb71b700 1 -- 172.21.15.121:0/4184405282 shutdown_connections 2023-11-15T03:07:34.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0ceb71b700 1 --2- 172.21.15.121:0/4184405282 >> v2:172.21.15.121:3300/0 conn(0x7f0ce4067f50 0x7f0ce4065d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:34.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0ceb71b700 1 --2- 172.21.15.121:0/4184405282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce40aa240 0x7f0ce4067a60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:34.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0ceb71b700 1 -- 172.21.15.121:0/4184405282 >> 172.21.15.121:0/4184405282 conn(0x7f0ce4058c50 msgr2=0x7f0ce4059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:34.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0ceb71b700 1 -- 172.21.15.121:0/4184405282 shutdown_connections 2023-11-15T03:07:34.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.364+0000 7f0ceb71b700 1 -- 172.21.15.121:0/4184405282 wait complete. 2023-11-15T03:07:34.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.368+0000 7f0ceb71b700 1 Processor -- start 2023-11-15T03:07:34.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.368+0000 7f0ceb71b700 1 -- start start 2023-11-15T03:07:34.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0ceb71b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce4067f50 0x7f0ce4154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:34.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0ceb71b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ce40aa240 0x7f0ce40acec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:34.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0ceb71b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce4153bb0 0x7f0ce40c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:34.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0ceb71b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce414f670 con 0x7f0ce40aa240 2023-11-15T03:07:34.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0ceb71b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce41504d0 con 0x7f0ce4067f50 2023-11-15T03:07:34.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0ceb71b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce414f8c0 con 0x7f0ce4153bb0 2023-11-15T03:07:34.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0ce8cb6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ce40aa240 0x7f0ce40acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:34.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0ce8cb6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ce40aa240 0x7f0ce40acec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50318/0 (socket says 172.21.15.121:50318) 2023-11-15T03:07:34.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0ce8cb6700 1 -- 172.21.15.121:0/4222229966 learned_addr learned my addr 172.21.15.121:0/4222229966 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:34.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0ce94b7700 1 --2- 172.21.15.121:0/4222229966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce4067f50 0x7f0ce4154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:34.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0ce9cb8700 1 --2- 172.21.15.121:0/4222229966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce4153bb0 0x7f0ce40c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:34.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0ce8cb6700 1 -- 172.21.15.121:0/4222229966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce4153bb0 msgr2=0x7f0ce40c31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:34.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0ce8cb6700 1 --2- 172.21.15.121:0/4222229966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce4153bb0 0x7f0ce40c31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:34.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0ce8cb6700 1 -- 172.21.15.121:0/4222229966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce4067f50 msgr2=0x7f0ce4154b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:34.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0ce8cb6700 1 --2- 172.21.15.121:0/4222229966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce4067f50 0x7f0ce4154b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:34.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0ce8cb6700 1 -- 172.21.15.121:0/4222229966 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd40092a0 con 0x7f0ce40aa240 2023-11-15T03:07:34.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0ce8cb6700 1 --2- 172.21.15.121:0/4222229966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ce40aa240 0x7f0ce40acec0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 rx=0x7f0ce4069c10 tx=0x7f0cd4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:34.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0cd97fa700 1 -- 172.21.15.121:0/4222229966 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd400f7e0 con 0x7f0ce40aa240 2023-11-15T03:07:34.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0cd97fa700 1 -- 172.21.15.121:0/4222229966 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd400f940 con 0x7f0ce40aa240 2023-11-15T03:07:34.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0ceb71b700 1 -- 172.21.15.121:0/4222229966 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce414fba0 con 0x7f0ce40aa240 2023-11-15T03:07:34.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0cd97fa700 1 -- 172.21.15.121:0/4222229966 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd401b660 con 0x7f0ce40aa240 2023-11-15T03:07:34.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.372+0000 7f0ceb71b700 1 -- 172.21.15.121:0/4222229966 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce4150160 con 0x7f0ce40aa240 2023-11-15T03:07:34.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.376+0000 7f0ceb71b700 1 -- 172.21.15.121:0/4222229966 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce4150660 con 0x7f0ce40aa240 2023-11-15T03:07:34.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.376+0000 7f0cd97fa700 1 -- 172.21.15.121:0/4222229966 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0cd401ba50 con 0x7f0ce40aa240 2023-11-15T03:07:34.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.380+0000 7f0cd97fa700 1 --2- 172.21.15.121:0/4222229966 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0cc805a510 0x7f0cc805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:34.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.380+0000 7f0cd97fa700 1 -- 172.21.15.121:0/4222229966 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8244+0+0 (secure 0 0 0) 0x7f0cd4076c60 con 0x7f0ce40aa240 2023-11-15T03:07:34.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.380+0000 7f0ce94b7700 1 --2- 172.21.15.121:0/4222229966 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0cc805a510 0x7f0cc805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:34.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.380+0000 7f0ce94b7700 1 --2- 172.21.15.121:0/4222229966 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0cc805a510 0x7f0cc805c9d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f0ce00076b0 tx=0x7f0ce0007bf0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:34.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.384+0000 7f0cd97fa700 1 -- 172.21.15.121:0/4222229966 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0cd4049bb0 con 0x7f0ce40aa240 2023-11-15T03:07:34.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.584+0000 7f0ceb71b700 1 -- 172.21.15.121:0/4222229966 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ce40c83c0 con 0x7f0ce40aa240 2023-11-15T03:07:34.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.584+0000 7f0cd97fa700 1 -- 172.21.15.121:0/4222229966 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 221 v221) v1 ==== 95+0+4502 (secure 0 0 0) 0x7f0cd404d1d0 con 0x7f0ce40aa240 2023-11-15T03:07:34.595 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:34.595 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":221,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":221,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:34.178477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9129},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9129":{"gid":9129,"name":"a","rank":0,"incarnation":221,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/115896047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":115896047},{"type":"v1","addr":"172.21.15.121:6836","nonce":115896047}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:34.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.588+0000 7f0ceb71b700 1 -- 172.21.15.121:0/4222229966 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0cc805a510 msgr2=0x7f0cc805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:34.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.588+0000 7f0ceb71b700 1 --2- 172.21.15.121:0/4222229966 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0cc805a510 0x7f0cc805c9d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f0ce00076b0 tx=0x7f0ce0007bf0).stop 2023-11-15T03:07:34.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.588+0000 7f0ceb71b700 1 -- 172.21.15.121:0/4222229966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ce40aa240 msgr2=0x7f0ce40acec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:34.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.588+0000 7f0ceb71b700 1 --2- 172.21.15.121:0/4222229966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ce40aa240 0x7f0ce40acec0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 rx=0x7f0ce4069c10 tx=0x7f0cd4017000).stop 2023-11-15T03:07:34.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.588+0000 7f0ceb71b700 1 -- 172.21.15.121:0/4222229966 shutdown_connections 2023-11-15T03:07:34.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.588+0000 7f0ceb71b700 1 --2- 172.21.15.121:0/4222229966 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0cc805a510 0x7f0cc805c9d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:34.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.588+0000 7f0ceb71b700 1 --2- 172.21.15.121:0/4222229966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ce4153bb0 0x7f0ce40c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:34.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.588+0000 7f0ceb71b700 1 --2- 172.21.15.121:0/4222229966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ce40aa240 0x7f0ce40acec0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:34.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.588+0000 7f0ceb71b700 1 --2- 172.21.15.121:0/4222229966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce4067f50 0x7f0ce4154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:34.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.588+0000 7f0ceb71b700 1 -- 172.21.15.121:0/4222229966 >> 172.21.15.121:0/4222229966 conn(0x7f0ce4058c50 msgr2=0x7f0ce4152a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:34.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.588+0000 7f0ceb71b700 1 -- 172.21.15.121:0/4222229966 shutdown_connections 2023-11-15T03:07:34.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:34.588+0000 7f0ceb71b700 1 -- 172.21.15.121:0/4222229966 wait complete. 2023-11-15T03:07:34.602 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 221 2023-11-15T03:07:34.615 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:07:35.189 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:07:35.180+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:07:35.190 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:07:35.180+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:07:35.198 INFO:tasks.ceph.mds.a.smithi121.stderr: -25> 2023-11-15T03:07:34.192+0000 7fa0b55a8700 -1 mds.0.cache.ino(0x100) Corrupt inode 0x100: End of buffer 2023-11-15T03:07:35.198 INFO:tasks.ceph.mds.a.smithi121.stderr: -16> 2023-11-15T03:07:34.192+0000 7fa0b55a8700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2023-11-15T03:07:35.239 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:07:39.617 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:39.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.772+0000 7fcd695a9700 1 Processor -- start 2023-11-15T03:07:39.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.772+0000 7fcd695a9700 1 -- start start 2023-11-15T03:07:39.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd695a9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcd640c6ce0 0x7fcd64159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:39.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd695a9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcd640a5510 0x7fcd640a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:39.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd695a9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd640c3fa0 con 0x7fcd64152780 2023-11-15T03:07:39.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd695a9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd64159630 con 0x7fcd640a5510 2023-11-15T03:07:39.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd695a9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd640c30b0 con 0x7fcd640c6ce0 2023-11-15T03:07:39.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd62ffd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fcd64152780 0x7fcd640c65d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:42672/0 (socket says 172.21.15.121:42672) 2023-11-15T03:07:39.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd62ffd700 1 -- 172.21.15.121:0/1030107981 learned_addr learned my addr 172.21.15.121:0/1030107981 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:39.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd5a7fc700 1 --2- 172.21.15.121:0/1030107981 >> v2:172.21.15.190:3300/0 conn(0x7fcd640c6ce0 0x7fcd64159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:39.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd637fe700 1 --2- 172.21.15.121:0/1030107981 >> v2:172.21.15.121:3300/0 conn(0x7fcd640a5510 0x7fcd640a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:39.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd627fc700 1 -- 172.21.15.121:0/1030107981 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100098391 0 0) 0x7fcd640c3fa0 con 0x7fcd64152780 2023-11-15T03:07:39.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd627fc700 1 -- 172.21.15.121:0/1030107981 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd500033f0 con 0x7fcd64152780 2023-11-15T03:07:39.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd637fe700 1 -- 172.21.15.121:0/1030107981 >> v2:172.21.15.190:3300/0 conn(0x7fcd640c6ce0 msgr2=0x7fcd64159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:39.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd637fe700 1 --2- 172.21.15.121:0/1030107981 >> v2:172.21.15.190:3300/0 conn(0x7fcd640c6ce0 0x7fcd64159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:39.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd637fe700 1 -- 172.21.15.121:0/1030107981 >> v1:172.21.15.190:6789/0 conn(0x7fcd64152780 legacy=0x7fcd640c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:39.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd637fe700 1 -- 172.21.15.121:0/1030107981 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd640c3390 con 0x7fcd640a5510 2023-11-15T03:07:39.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd637fe700 1 --2- 172.21.15.121:0/1030107981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd640a5510 0x7fcd640a59a0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 rx=0x7fcd64063d40 tx=0x7fcd5c009060).ready entity=mon.0 client_cookie=c2762ab0ae04007a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:39.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd627fc700 1 -- 172.21.15.121:0/1030107981 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c00d4a0 con 0x7fcd640a5510 2023-11-15T03:07:39.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd627fc700 1 -- 172.21.15.121:0/1030107981 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5c00d600 con 0x7fcd640a5510 2023-11-15T03:07:39.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd627fc700 1 -- 172.21.15.121:0/1030107981 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c00d980 con 0x7fcd640a5510 2023-11-15T03:07:39.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd695a9700 1 -- 172.21.15.121:0/1030107981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd640a5510 msgr2=0x7fcd640a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:39.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.776+0000 7fcd695a9700 1 --2- 172.21.15.121:0/1030107981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd640a5510 0x7fcd640a59a0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 rx=0x7fcd64063d40 tx=0x7fcd5c009060).stop 2023-11-15T03:07:39.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.780+0000 7fcd695a9700 1 -- 172.21.15.121:0/1030107981 shutdown_connections 2023-11-15T03:07:39.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.780+0000 7fcd695a9700 1 --2- 172.21.15.121:0/1030107981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd640a5510 0x7fcd640a59a0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:39.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.780+0000 7fcd695a9700 1 --2- 172.21.15.121:0/1030107981 >> v2:172.21.15.190:3300/0 conn(0x7fcd640c6ce0 0x7fcd64159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:39.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.780+0000 7fcd695a9700 1 -- 172.21.15.121:0/1030107981 >> 172.21.15.121:0/1030107981 conn(0x7fcd64062190 msgr2=0x7fcd640625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:39.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.780+0000 7fcd695a9700 1 -- 172.21.15.121:0/1030107981 shutdown_connections 2023-11-15T03:07:39.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.780+0000 7fcd695a9700 1 -- 172.21.15.121:0/1030107981 wait complete. 2023-11-15T03:07:39.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.780+0000 7fcd695a9700 1 Processor -- start 2023-11-15T03:07:39.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.780+0000 7fcd695a9700 1 -- start start 2023-11-15T03:07:39.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd695a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd640a5510 0x7fcd6413c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:39.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd695a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd640c6ce0 0x7fcd64150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:39.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd695a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd64152780 0x7fcd64151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:39.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd695a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd64151840 con 0x7fcd64152780 2023-11-15T03:07:39.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd695a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6414a3a0 con 0x7fcd640a5510 2023-11-15T03:07:39.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd695a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6416ba60 con 0x7fcd640c6ce0 2023-11-15T03:07:39.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd637fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd64152780 0x7fcd64151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:39.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd637fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd64152780 0x7fcd64151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44074/0 (socket says 172.21.15.121:44074) 2023-11-15T03:07:39.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd637fe700 1 -- 172.21.15.121:0/1170594058 learned_addr learned my addr 172.21.15.121:0/1170594058 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:39.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd5a7fc700 1 --2- 172.21.15.121:0/1170594058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd640c6ce0 0x7fcd64150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:39.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd62ffd700 1 --2- 172.21.15.121:0/1170594058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd640a5510 0x7fcd6413c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:39.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd637fe700 1 -- 172.21.15.121:0/1170594058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd640c6ce0 msgr2=0x7fcd64150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:39.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd637fe700 1 --2- 172.21.15.121:0/1170594058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd640c6ce0 0x7fcd64150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:39.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd637fe700 1 -- 172.21.15.121:0/1170594058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd640a5510 msgr2=0x7fcd6413c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:39.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd637fe700 1 --2- 172.21.15.121:0/1170594058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd640a5510 0x7fcd6413c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:39.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd637fe700 1 -- 172.21.15.121:0/1170594058 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5c0092a0 con 0x7fcd64152780 2023-11-15T03:07:39.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd637fe700 1 --2- 172.21.15.121:0/1170594058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd64152780 0x7fcd64151350 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 rx=0x7fcd64064f10 tx=0x7fcd5c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:39.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd5b7fe700 1 -- 172.21.15.121:0/1170594058 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c021070 con 0x7fcd64152780 2023-11-15T03:07:39.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd5b7fe700 1 -- 172.21.15.121:0/1170594058 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5c00d770 con 0x7fcd64152780 2023-11-15T03:07:39.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd695a9700 1 -- 172.21.15.121:0/1170594058 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd6416bd40 con 0x7fcd64152780 2023-11-15T03:07:39.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd5b7fe700 1 -- 172.21.15.121:0/1170594058 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c01d430 con 0x7fcd64152780 2023-11-15T03:07:39.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.784+0000 7fcd695a9700 1 -- 172.21.15.121:0/1170594058 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd6416c300 con 0x7fcd64152780 2023-11-15T03:07:39.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.788+0000 7fcd695a9700 1 -- 172.21.15.121:0/1170594058 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd6406c150 con 0x7fcd64152780 2023-11-15T03:07:39.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.788+0000 7fcd5b7fe700 1 -- 172.21.15.121:0/1170594058 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fcd5c01d820 con 0x7fcd64152780 2023-11-15T03:07:39.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.788+0000 7fcd5b7fe700 1 --2- 172.21.15.121:0/1170594058 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd4405a510 0x7fcd4405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:39.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.788+0000 7fcd5b7fe700 1 -- 172.21.15.121:0/1170594058 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8244+0+0 (secure 0 0 0) 0x7fcd5c0764f0 con 0x7fcd64152780 2023-11-15T03:07:39.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.788+0000 7fcd62ffd700 1 --2- 172.21.15.121:0/1170594058 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd4405a510 0x7fcd4405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:39.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.792+0000 7fcd62ffd700 1 --2- 172.21.15.121:0/1170594058 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd4405a510 0x7fcd4405c9d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7fcd54007fd0 tx=0x7fcd54009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:39.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.796+0000 7fcd5b7fe700 1 -- 172.21.15.121:0/1170594058 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcd5c0493e0 con 0x7fcd64152780 2023-11-15T03:07:39.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.992+0000 7fcd695a9700 1 -- 172.21.15.121:0/1170594058 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd640c11b0 con 0x7fcd64152780 2023-11-15T03:07:39.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.992+0000 7fcd5b7fe700 1 -- 172.21.15.121:0/1170594058 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) v1 ==== 95+0+4496 (secure 0 0 0) 0x7fcd5c04ca00 con 0x7fcd64152780 2023-11-15T03:07:39.999 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:40.000 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":223,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9198,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2544923351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2544923351},{"type":"v1","addr":"172.21.15.121:6836","nonce":2544923351}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":223}],"filesystems":[{"mdsmap":{"epoch":222,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:35.185332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:40.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.996+0000 7fcd695a9700 1 -- 172.21.15.121:0/1170594058 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd4405a510 msgr2=0x7fcd4405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:40.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.996+0000 7fcd695a9700 1 --2- 172.21.15.121:0/1170594058 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd4405a510 0x7fcd4405c9d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7fcd54007fd0 tx=0x7fcd54009040).stop 2023-11-15T03:07:40.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.996+0000 7fcd695a9700 1 -- 172.21.15.121:0/1170594058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd64152780 msgr2=0x7fcd64151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:40.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.996+0000 7fcd695a9700 1 --2- 172.21.15.121:0/1170594058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd64152780 0x7fcd64151350 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 rx=0x7fcd64064f10 tx=0x7fcd5c016000).stop 2023-11-15T03:07:40.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.996+0000 7fcd695a9700 1 -- 172.21.15.121:0/1170594058 shutdown_connections 2023-11-15T03:07:40.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.996+0000 7fcd695a9700 1 --2- 172.21.15.121:0/1170594058 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcd4405a510 0x7fcd4405c9d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.996+0000 7fcd695a9700 1 --2- 172.21.15.121:0/1170594058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd64152780 0x7fcd64151350 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.996+0000 7fcd695a9700 1 --2- 172.21.15.121:0/1170594058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd640c6ce0 0x7fcd64150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.996+0000 7fcd695a9700 1 --2- 172.21.15.121:0/1170594058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd640a5510 0x7fcd6413c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.996+0000 7fcd695a9700 1 -- 172.21.15.121:0/1170594058 >> 172.21.15.121:0/1170594058 conn(0x7fcd64062190 msgr2=0x7fcd640a8980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:40.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.996+0000 7fcd695a9700 1 -- 172.21.15.121:0/1170594058 shutdown_connections 2023-11-15T03:07:40.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:39.996+0000 7fcd695a9700 1 -- 172.21.15.121:0/1170594058 wait complete. 2023-11-15T03:07:40.007 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 223 2023-11-15T03:07:40.026 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:07:40.027 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:40.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.180+0000 7f71bc9b9700 1 Processor -- start 2023-11-15T03:07:40.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.180+0000 7f71bc9b9700 1 -- start start 2023-11-15T03:07:40.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.180+0000 7f71bc9b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f71b80c3280 0x7f71b80c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:40.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.180+0000 7f71bc9b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71b815be30 0x7f71b815c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:40.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.180+0000 7f71bc9b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b80c1d50 con 0x7f71b8152140 2023-11-15T03:07:40.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.180+0000 7f71bc9b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b80c1860 con 0x7f71b80c3280 2023-11-15T03:07:40.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.180+0000 7f71bc9b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b815cb80 con 0x7f71b815be30 2023-11-15T03:07:40.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.180+0000 7f71affff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f71b8152140 0x7f71b81525f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50582/0 (socket says 172.21.15.121:50582) 2023-11-15T03:07:40.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71affff700 1 -- 172.21.15.121:0/2722873758 learned_addr learned my addr 172.21.15.121:0/2722873758 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:40.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.180+0000 7f71b77fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f71b80c3280 0x7f71b80c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:40.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71b7fff700 1 --2- 172.21.15.121:0/2722873758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71b815be30 0x7f71b815c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:40.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71b6ffd700 1 -- 172.21.15.121:0/2722873758 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873724183 0 0) 0x7f71b80c1d50 con 0x7f71b8152140 2023-11-15T03:07:40.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71b6ffd700 1 -- 172.21.15.121:0/2722873758 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71a40033f0 con 0x7f71b8152140 2023-11-15T03:07:40.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71b77fe700 1 -- 172.21.15.121:0/2722873758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71b815be30 msgr2=0x7f71b815c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:40.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71b77fe700 1 --2- 172.21.15.121:0/2722873758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71b815be30 0x7f71b815c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71b77fe700 1 -- 172.21.15.121:0/2722873758 >> v1:172.21.15.121:6789/0 conn(0x7f71b8152140 legacy=0x7f71b81525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:40.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71b77fe700 1 -- 172.21.15.121:0/2722873758 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b80a9240 con 0x7f71b80c3280 2023-11-15T03:07:40.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71b77fe700 1 --2- 172.21.15.121:0/2722873758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71b80c3280 0x7f71b80c3660 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 rx=0x7f71a8009760 tx=0x7f71a8009060).ready entity=mon.0 client_cookie=5951603775c398c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:40.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71b6ffd700 1 -- 172.21.15.121:0/2722873758 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a800f510 con 0x7f71b80c3280 2023-11-15T03:07:40.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71b6ffd700 1 -- 172.21.15.121:0/2722873758 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a800f670 con 0x7f71b80c3280 2023-11-15T03:07:40.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71b6ffd700 1 -- 172.21.15.121:0/2722873758 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a800f9f0 con 0x7f71b80c3280 2023-11-15T03:07:40.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71bc9b9700 1 -- 172.21.15.121:0/2722873758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71b80c3280 msgr2=0x7f71b80c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:40.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71bc9b9700 1 --2- 172.21.15.121:0/2722873758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71b80c3280 0x7f71b80c3660 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 rx=0x7f71a8009760 tx=0x7f71a8009060).stop 2023-11-15T03:07:40.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71bc9b9700 1 -- 172.21.15.121:0/2722873758 shutdown_connections 2023-11-15T03:07:40.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71bc9b9700 1 --2- 172.21.15.121:0/2722873758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71b815be30 0x7f71b815c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71bc9b9700 1 --2- 172.21.15.121:0/2722873758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71b80c3280 0x7f71b80c3660 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71bc9b9700 1 -- 172.21.15.121:0/2722873758 >> 172.21.15.121:0/2722873758 conn(0x7f71b8062190 msgr2=0x7f71b8062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:40.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71bc9b9700 1 -- 172.21.15.121:0/2722873758 shutdown_connections 2023-11-15T03:07:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.184+0000 7f71bc9b9700 1 -- 172.21.15.121:0/2722873758 wait complete. 2023-11-15T03:07:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.188+0000 7f71bc9b9700 1 Processor -- start 2023-11-15T03:07:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.188+0000 7f71bc9b9700 1 -- start start 2023-11-15T03:07:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.188+0000 7f71bc9b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71b80c3280 0x7f71b813d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.188+0000 7f71bc9b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71b8152140 0x7f71b8158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:40.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.188+0000 7f71bc9b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71b815be30 0x7f71b816baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:40.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.188+0000 7f71bc9b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b8156bb0 con 0x7f71b8152140 2023-11-15T03:07:40.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.188+0000 7f71bc9b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b8157450 con 0x7f71b80c3280 2023-11-15T03:07:40.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.188+0000 7f71bc9b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b8153ee0 con 0x7f71b815be30 2023-11-15T03:07:40.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.188+0000 7f71affff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71b8152140 0x7f71b8158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:40.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.188+0000 7f71affff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71b8152140 0x7f71b8158d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44090/0 (socket says 172.21.15.121:44090) 2023-11-15T03:07:40.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.188+0000 7f71affff700 1 -- 172.21.15.121:0/2025452817 learned_addr learned my addr 172.21.15.121:0/2025452817 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:40.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.188+0000 7f71b77fe700 1 --2- 172.21.15.121:0/2025452817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71b80c3280 0x7f71b813d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:40.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.188+0000 7f71b7fff700 1 --2- 172.21.15.121:0/2025452817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71b815be30 0x7f71b816baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:40.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.188+0000 7f71affff700 1 -- 172.21.15.121:0/2025452817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71b815be30 msgr2=0x7f71b816baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:40.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.188+0000 7f71affff700 1 --2- 172.21.15.121:0/2025452817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71b815be30 0x7f71b816baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.188+0000 7f71affff700 1 -- 172.21.15.121:0/2025452817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71b80c3280 msgr2=0x7f71b813d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:40.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.188+0000 7f71affff700 1 --2- 172.21.15.121:0/2025452817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71b80c3280 0x7f71b813d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.188+0000 7f71affff700 1 -- 172.21.15.121:0/2025452817 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a80092a0 con 0x7f71b8152140 2023-11-15T03:07:40.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.188+0000 7f71affff700 1 --2- 172.21.15.121:0/2025452817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71b8152140 0x7f71b8158d20 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 rx=0x7f71a000c5c0 tx=0x7f71a0007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:40.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.192+0000 7f71af7fe700 1 -- 172.21.15.121:0/2025452817 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0007e40 con 0x7f71b8152140 2023-11-15T03:07:40.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.192+0000 7f71af7fe700 1 -- 172.21.15.121:0/2025452817 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a00072e0 con 0x7f71b8152140 2023-11-15T03:07:40.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.192+0000 7f71bc9b9700 1 -- 172.21.15.121:0/2025452817 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b81567f0 con 0x7f71b8152140 2023-11-15T03:07:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.192+0000 7f71af7fe700 1 -- 172.21.15.121:0/2025452817 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a00057a0 con 0x7f71b8152140 2023-11-15T03:07:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.192+0000 7f71bc9b9700 1 -- 172.21.15.121:0/2025452817 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b81570e0 con 0x7f71b8152140 2023-11-15T03:07:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.192+0000 7f71bc9b9700 1 -- 172.21.15.121:0/2025452817 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b8157600 con 0x7f71b8152140 2023-11-15T03:07:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.192+0000 7f71af7fe700 1 -- 172.21.15.121:0/2025452817 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f71a0005b90 con 0x7f71b8152140 2023-11-15T03:07:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.196+0000 7f71af7fe700 1 --2- 172.21.15.121:0/2025452817 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f719805a540 0x7f719805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:40.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.196+0000 7f71af7fe700 1 -- 172.21.15.121:0/2025452817 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8244+0+0 (secure 0 0 0) 0x7f71a00748f0 con 0x7f71b8152140 2023-11-15T03:07:40.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.196+0000 7f71b77fe700 1 --2- 172.21.15.121:0/2025452817 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f719805a540 0x7f719805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:40.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.196+0000 7f71b77fe700 1 --2- 172.21.15.121:0/2025452817 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f719805a540 0x7f719805ca00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f71a8009760 tx=0x7f71a8017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:40.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.200+0000 7f71af7fe700 1 -- 172.21.15.121:0/2025452817 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f71b8157600 con 0x7f71b8152140 2023-11-15T03:07:40.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.404+0000 7f71bc9b9700 1 -- 172.21.15.121:0/2025452817 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71b81575c0 con 0x7f71b8152140 2023-11-15T03:07:40.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.404+0000 7f71af7fe700 1 -- 172.21.15.121:0/2025452817 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f71a0047840 con 0x7f71b8152140 2023-11-15T03:07:40.412 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:40.413 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":223,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9198,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2544923351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2544923351},{"type":"v1","addr":"172.21.15.121:6836","nonce":2544923351}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":223}],"filesystems":[{"mdsmap":{"epoch":222,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:35.185332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:40.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.408+0000 7f71bc9b9700 1 -- 172.21.15.121:0/2025452817 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f719805a540 msgr2=0x7f719805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:40.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.408+0000 7f71bc9b9700 1 --2- 172.21.15.121:0/2025452817 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f719805a540 0x7f719805ca00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f71a8009760 tx=0x7f71a8017000).stop 2023-11-15T03:07:40.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.408+0000 7f71bc9b9700 1 -- 172.21.15.121:0/2025452817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71b8152140 msgr2=0x7f71b8158d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:40.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.408+0000 7f71bc9b9700 1 --2- 172.21.15.121:0/2025452817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71b8152140 0x7f71b8158d20 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 rx=0x7f71a000c5c0 tx=0x7f71a0007c80).stop 2023-11-15T03:07:40.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.408+0000 7f71bc9b9700 1 -- 172.21.15.121:0/2025452817 shutdown_connections 2023-11-15T03:07:40.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.408+0000 7f71bc9b9700 1 --2- 172.21.15.121:0/2025452817 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f719805a540 0x7f719805ca00 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.408+0000 7f71bc9b9700 1 --2- 172.21.15.121:0/2025452817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71b815be30 0x7f71b816baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.408+0000 7f71bc9b9700 1 --2- 172.21.15.121:0/2025452817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71b8152140 0x7f71b8158d20 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.408+0000 7f71bc9b9700 1 --2- 172.21.15.121:0/2025452817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71b80c3280 0x7f71b813d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.408+0000 7f71bc9b9700 1 -- 172.21.15.121:0/2025452817 >> 172.21.15.121:0/2025452817 conn(0x7f71b8062190 msgr2=0x7f71b8066ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:40.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.408+0000 7f71bc9b9700 1 -- 172.21.15.121:0/2025452817 shutdown_connections 2023-11-15T03:07:40.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.408+0000 7f71bc9b9700 1 -- 172.21.15.121:0/2025452817 wait complete. 2023-11-15T03:07:40.420 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 223 2023-11-15T03:07:40.439 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:07:40.440 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:40.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.592+0000 7f9cf09a9700 1 Processor -- start 2023-11-15T03:07:40.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.596+0000 7f9cf09a9700 1 -- start start 2023-11-15T03:07:40.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.596+0000 7f9cf09a9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9cec152780 0x7f9cec159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:40.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.596+0000 7f9cf09a9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9cec0a5510 0x7f9cec0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:40.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.596+0000 7f9cf09a9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cec0c3fa0 con 0x7f9cec0c65d0 2023-11-15T03:07:40.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.596+0000 7f9cf09a9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cec159630 con 0x7f9cec0a5510 2023-11-15T03:07:40.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.596+0000 7f9cf09a9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cec0c30b0 con 0x7f9cec152780 2023-11-15T03:07:40.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.596+0000 7f9ce3fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9cec0c65d0 0x7f9cec0c6a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50584/0 (socket says 172.21.15.121:50584) 2023-11-15T03:07:40.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.596+0000 7f9ce3fff700 1 -- 172.21.15.121:0/368585803 learned_addr learned my addr 172.21.15.121:0/368585803 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:40.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.596+0000 7f9cead9d700 1 --2- 172.21.15.121:0/368585803 >> v2:172.21.15.121:3300/0 conn(0x7f9cec0a5510 0x7f9cec0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:40.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.596+0000 7f9cea59c700 1 --2- 172.21.15.121:0/368585803 >> v2:172.21.15.190:3300/0 conn(0x7f9cec152780 0x7f9cec159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:40.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.596+0000 7f9ce9d9b700 1 -- 172.21.15.121:0/368585803 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2031631887 0 0) 0x7f9cec0c3fa0 con 0x7f9cec0c65d0 2023-11-15T03:07:40.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.596+0000 7f9ce9d9b700 1 -- 172.21.15.121:0/368585803 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cd80033f0 con 0x7f9cec0c65d0 2023-11-15T03:07:40.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.596+0000 7f9cead9d700 1 -- 172.21.15.121:0/368585803 >> v2:172.21.15.190:3300/0 conn(0x7f9cec152780 msgr2=0x7f9cec159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:40.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.596+0000 7f9cead9d700 1 --2- 172.21.15.121:0/368585803 >> v2:172.21.15.190:3300/0 conn(0x7f9cec152780 0x7f9cec159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.596+0000 7f9cead9d700 1 -- 172.21.15.121:0/368585803 >> v1:172.21.15.121:6789/0 conn(0x7f9cec0c65d0 legacy=0x7f9cec0c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:40.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.600+0000 7f9cead9d700 1 -- 172.21.15.121:0/368585803 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec0c3390 con 0x7f9cec0a5510 2023-11-15T03:07:40.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.600+0000 7f9cead9d700 1 --2- 172.21.15.121:0/368585803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cec0a5510 0x7f9cec0a59a0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 rx=0x7f9cec0653f0 tx=0x7f9ce4009780).ready entity=mon.0 client_cookie=63a2ca4e9a9654a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:40.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.600+0000 7f9ce9d9b700 1 -- 172.21.15.121:0/368585803 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce400e070 con 0x7f9cec0a5510 2023-11-15T03:07:40.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.600+0000 7f9ce9d9b700 1 -- 172.21.15.121:0/368585803 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce400bab0 con 0x7f9cec0a5510 2023-11-15T03:07:40.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.600+0000 7f9ce9d9b700 1 -- 172.21.15.121:0/368585803 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce400be30 con 0x7f9cec0a5510 2023-11-15T03:07:40.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.600+0000 7f9cf09a9700 1 -- 172.21.15.121:0/368585803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cec0a5510 msgr2=0x7f9cec0a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:40.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.600+0000 7f9cf09a9700 1 --2- 172.21.15.121:0/368585803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cec0a5510 0x7f9cec0a59a0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 rx=0x7f9cec0653f0 tx=0x7f9ce4009780).stop 2023-11-15T03:07:40.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.600+0000 7f9cf09a9700 1 -- 172.21.15.121:0/368585803 shutdown_connections 2023-11-15T03:07:40.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.600+0000 7f9cf09a9700 1 --2- 172.21.15.121:0/368585803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cec0a5510 0x7f9cec0a59a0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.600+0000 7f9cf09a9700 1 --2- 172.21.15.121:0/368585803 >> v2:172.21.15.190:3300/0 conn(0x7f9cec152780 0x7f9cec159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.600+0000 7f9cf09a9700 1 -- 172.21.15.121:0/368585803 >> 172.21.15.121:0/368585803 conn(0x7f9cec062190 msgr2=0x7f9cec0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:40.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.600+0000 7f9cf09a9700 1 -- 172.21.15.121:0/368585803 shutdown_connections 2023-11-15T03:07:40.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.600+0000 7f9cf09a9700 1 -- 172.21.15.121:0/368585803 wait complete. 2023-11-15T03:07:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.604+0000 7f9cf09a9700 1 Processor -- start 2023-11-15T03:07:40.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.604+0000 7f9cf09a9700 1 -- start start 2023-11-15T03:07:40.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.604+0000 7f9cf09a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cec0a5510 0x7f9cec150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:40.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.604+0000 7f9cf09a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cec0c65d0 0x7f9cec14d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:40.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.604+0000 7f9cf09a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cec152780 0x7f9cec0c1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:40.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.604+0000 7f9cf09a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec150b20 con 0x7f9cec0a5510 2023-11-15T03:07:40.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.604+0000 7f9cf09a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec14d3a0 con 0x7f9cec152780 2023-11-15T03:07:40.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.604+0000 7f9cf09a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec0a6f10 con 0x7f9cec0c65d0 2023-11-15T03:07:40.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.604+0000 7f9cea59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cec0a5510 0x7f9cec150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:40.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.604+0000 7f9cea59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cec0a5510 0x7f9cec150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44116/0 (socket says 172.21.15.121:44116) 2023-11-15T03:07:40.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.604+0000 7f9cea59c700 1 -- 172.21.15.121:0/4125572504 learned_addr learned my addr 172.21.15.121:0/4125572504 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:40.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.604+0000 7f9ce3fff700 1 --2- 172.21.15.121:0/4125572504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cec0c65d0 0x7f9cec14d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:40.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.604+0000 7f9cea59c700 1 -- 172.21.15.121:0/4125572504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cec0c65d0 msgr2=0x7f9cec14d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:40.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.604+0000 7f9cea59c700 1 --2- 172.21.15.121:0/4125572504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cec0c65d0 0x7f9cec14d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.604+0000 7f9cead9d700 1 --2- 172.21.15.121:0/4125572504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cec152780 0x7f9cec0c1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:40.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.604+0000 7f9cea59c700 1 -- 172.21.15.121:0/4125572504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cec152780 msgr2=0x7f9cec0c1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:40.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.604+0000 7f9cea59c700 1 --2- 172.21.15.121:0/4125572504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cec152780 0x7f9cec0c1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.604+0000 7f9cea59c700 1 -- 172.21.15.121:0/4125572504 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce4009a20 con 0x7f9cec0a5510 2023-11-15T03:07:40.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.604+0000 7f9cea59c700 1 --2- 172.21.15.121:0/4125572504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cec0a5510 0x7f9cec150630 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 rx=0x7f9cdc00e900 tx=0x7f9cdc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:40.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.608+0000 7f9ce27fc700 1 -- 172.21.15.121:0/4125572504 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc00c6d0 con 0x7f9cec0a5510 2023-11-15T03:07:40.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.608+0000 7f9ce27fc700 1 -- 172.21.15.121:0/4125572504 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc00c830 con 0x7f9cec0a5510 2023-11-15T03:07:40.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.608+0000 7f9ce27fc700 1 -- 172.21.15.121:0/4125572504 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc00cbb0 con 0x7f9cec0a5510 2023-11-15T03:07:40.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.608+0000 7f9cf09a9700 1 -- 172.21.15.121:0/4125572504 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cec0a74a0 con 0x7f9cec0a5510 2023-11-15T03:07:40.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.608+0000 7f9cf09a9700 1 -- 172.21.15.121:0/4125572504 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cec0a7a60 con 0x7f9cec0a5510 2023-11-15T03:07:40.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.608+0000 7f9cf09a9700 1 -- 172.21.15.121:0/4125572504 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cec06c150 con 0x7f9cec0a5510 2023-11-15T03:07:40.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.608+0000 7f9ce27fc700 1 -- 172.21.15.121:0/4125572504 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9cdc01b400 con 0x7f9cec0a5510 2023-11-15T03:07:40.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.612+0000 7f9ce27fc700 1 --2- 172.21.15.121:0/4125572504 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ccc05a4e0 0x7f9ccc05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:40.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.612+0000 7f9ce27fc700 1 -- 172.21.15.121:0/4125572504 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8244+0+0 (secure 0 0 0) 0x7f9cdc075490 con 0x7f9cec0a5510 2023-11-15T03:07:40.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.612+0000 7f9ce3fff700 1 --2- 172.21.15.121:0/4125572504 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ccc05a4e0 0x7f9ccc05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:40.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.612+0000 7f9ce3fff700 1 --2- 172.21.15.121:0/4125572504 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ccc05a4e0 0x7f9ccc05c9a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f9cec0c2660 tx=0x7f9cd4009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:40.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.616+0000 7f9ce27fc700 1 -- 172.21.15.121:0/4125572504 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9cdc048430 con 0x7f9cec0a5510 2023-11-15T03:07:40.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.812+0000 7f9cf09a9700 1 -- 172.21.15.121:0/4125572504 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cec0c11b0 con 0x7f9cec0a5510 2023-11-15T03:07:40.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.812+0000 7f9ce27fc700 1 -- 172.21.15.121:0/4125572504 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f9cdc016020 con 0x7f9cec0a5510 2023-11-15T03:07:40.822 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:40.822 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":223,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9198,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2544923351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2544923351},{"type":"v1","addr":"172.21.15.121:6836","nonce":2544923351}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":223}],"filesystems":[{"mdsmap":{"epoch":222,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:35.185332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.816+0000 7f9cf09a9700 1 -- 172.21.15.121:0/4125572504 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ccc05a4e0 msgr2=0x7f9ccc05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:40.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.816+0000 7f9cf09a9700 1 --2- 172.21.15.121:0/4125572504 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ccc05a4e0 0x7f9ccc05c9a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f9cec0c2660 tx=0x7f9cd4009040).stop 2023-11-15T03:07:40.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.816+0000 7f9cf09a9700 1 -- 172.21.15.121:0/4125572504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cec0a5510 msgr2=0x7f9cec150630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:40.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.816+0000 7f9cf09a9700 1 --2- 172.21.15.121:0/4125572504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cec0a5510 0x7f9cec150630 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 rx=0x7f9cdc00e900 tx=0x7f9cdc009060).stop 2023-11-15T03:07:40.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.816+0000 7f9cf09a9700 1 -- 172.21.15.121:0/4125572504 shutdown_connections 2023-11-15T03:07:40.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.816+0000 7f9cf09a9700 1 --2- 172.21.15.121:0/4125572504 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ccc05a4e0 0x7f9ccc05c9a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.816+0000 7f9cf09a9700 1 --2- 172.21.15.121:0/4125572504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cec152780 0x7f9cec0c1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.816+0000 7f9cf09a9700 1 --2- 172.21.15.121:0/4125572504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cec0c65d0 0x7f9cec14d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.816+0000 7f9cf09a9700 1 --2- 172.21.15.121:0/4125572504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cec0a5510 0x7f9cec150630 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:40.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.816+0000 7f9cf09a9700 1 -- 172.21.15.121:0/4125572504 >> 172.21.15.121:0/4125572504 conn(0x7f9cec062190 msgr2=0x7f9cec0668a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:40.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.816+0000 7f9cf09a9700 1 -- 172.21.15.121:0/4125572504 shutdown_connections 2023-11-15T03:07:40.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:40.816+0000 7f9cf09a9700 1 -- 172.21.15.121:0/4125572504 wait complete. 2023-11-15T03:07:40.829 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 223 2023-11-15T03:07:40.847 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 100.00000000.inode' led to DAMAGED state 2023-11-15T03:07:40.847 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 1.00000000.inode' 2023-11-15T03:07:40.847 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:07:40.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:07:40.848 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:07:40.868 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:07:40.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:07:40.871 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:07:41.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.064+0000 7f41e4cf1700 1 Processor -- start 2023-11-15T03:07:41.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.064+0000 7f41e4cf1700 1 -- start start 2023-11-15T03:07:41.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.068+0000 7f41e4cf1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f41e00a8de0 0x7f41e0066420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:41.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.068+0000 7f41e4cf1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e0065f90 con 0x7f41e0150f90 2023-11-15T03:07:41.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.068+0000 7f41e4cf1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e0147ea0 con 0x7f41e00628a0 2023-11-15T03:07:41.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.068+0000 7f41e4cf1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e0151e30 con 0x7f41e00a8de0 2023-11-15T03:07:41.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.068+0000 7f41ddd9b700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f41e0150f90 0x7f41e0062190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50588/0 (socket says 172.21.15.121:50588) 2023-11-15T03:07:41.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.068+0000 7f41ddd9b700 1 -- 172.21.15.121:0/2217296733 learned_addr learned my addr 172.21.15.121:0/2217296733 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:41.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.068+0000 7f41de59c700 1 --2- 172.21.15.121:0/2217296733 >> v2:172.21.15.121:3300/0 conn(0x7f41e00a8de0 0x7f41e0066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:41.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.068+0000 7f41dd59a700 1 -- 172.21.15.121:0/2217296733 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604678971 0 0) 0x7f41e0065f90 con 0x7f41e0150f90 2023-11-15T03:07:41.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.068+0000 7f41dd59a700 1 -- 172.21.15.121:0/2217296733 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41c80033f0 con 0x7f41e0150f90 2023-11-15T03:07:41.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.068+0000 7f41de59c700 1 -- 172.21.15.121:0/2217296733 >> v1:172.21.15.190:6789/0 conn(0x7f41e00628a0 legacy=0x7f41e0152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:41.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.068+0000 7f41de59c700 1 -- 172.21.15.121:0/2217296733 >> v1:172.21.15.121:6789/0 conn(0x7f41e0150f90 legacy=0x7f41e0062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:41.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.068+0000 7f41de59c700 1 -- 172.21.15.121:0/2217296733 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e00671f0 con 0x7f41e00a8de0 2023-11-15T03:07:41.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.068+0000 7f41de59c700 1 --2- 172.21.15.121:0/2217296733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41e00a8de0 0x7f41e0066420 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 rx=0x7f41d0009760 tx=0x7f41d0009060).ready entity=mon.0 client_cookie=41ce8b1e36e5dac5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:41.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.068+0000 7f41dd59a700 1 -- 172.21.15.121:0/2217296733 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d000f510 con 0x7f41e00a8de0 2023-11-15T03:07:41.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.068+0000 7f41dd59a700 1 -- 172.21.15.121:0/2217296733 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d000f670 con 0x7f41e00a8de0 2023-11-15T03:07:41.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.068+0000 7f41dd59a700 1 -- 172.21.15.121:0/2217296733 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d000f9f0 con 0x7f41e00a8de0 2023-11-15T03:07:41.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.072+0000 7f41e4cf1700 1 -- 172.21.15.121:0/2217296733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41e00a8de0 msgr2=0x7f41e0066420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:41.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.072+0000 7f41e4cf1700 1 --2- 172.21.15.121:0/2217296733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41e00a8de0 0x7f41e0066420 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 rx=0x7f41d0009760 tx=0x7f41d0009060).stop 2023-11-15T03:07:41.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.072+0000 7f41e4cf1700 1 -- 172.21.15.121:0/2217296733 shutdown_connections 2023-11-15T03:07:41.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.072+0000 7f41e4cf1700 1 --2- 172.21.15.121:0/2217296733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41e00a8de0 0x7f41e0066420 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:41.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.072+0000 7f41e4cf1700 1 -- 172.21.15.121:0/2217296733 >> 172.21.15.121:0/2217296733 conn(0x7f41e0067a60 msgr2=0x7f41e0067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:41.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.072+0000 7f41e4cf1700 1 -- 172.21.15.121:0/2217296733 shutdown_connections 2023-11-15T03:07:41.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.072+0000 7f41e4cf1700 1 -- 172.21.15.121:0/2217296733 wait complete. 2023-11-15T03:07:41.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.072+0000 7f41e4cf1700 1 Processor -- start 2023-11-15T03:07:41.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.072+0000 7f41e4cf1700 1 -- start start 2023-11-15T03:07:41.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41e4cf1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41e00628a0 0x7f41e01648b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:41.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41e4cf1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41e00a8de0 0x7f41e01468e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:41.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41e4cf1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41e0150f90 0x7f41e014c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:41.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41e4cf1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e0164da0 con 0x7f41e00a8de0 2023-11-15T03:07:41.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41e4cf1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e01483d0 con 0x7f41e00628a0 2023-11-15T03:07:41.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41e4cf1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e01486b0 con 0x7f41e0150f90 2023-11-15T03:07:41.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41ddd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41e00a8de0 0x7f41e01468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:41.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41ddd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41e00a8de0 0x7f41e01468e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44128/0 (socket says 172.21.15.121:44128) 2023-11-15T03:07:41.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41ddd9b700 1 -- 172.21.15.121:0/1636350454 learned_addr learned my addr 172.21.15.121:0/1636350454 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:41.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41ded9d700 1 --2- 172.21.15.121:0/1636350454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41e0150f90 0x7f41e014c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:41.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41de59c700 1 --2- 172.21.15.121:0/1636350454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41e00628a0 0x7f41e01648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:41.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41ddd9b700 1 -- 172.21.15.121:0/1636350454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41e0150f90 msgr2=0x7f41e014c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:41.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41ddd9b700 1 --2- 172.21.15.121:0/1636350454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41e0150f90 0x7f41e014c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:41.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41ddd9b700 1 -- 172.21.15.121:0/1636350454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41e00628a0 msgr2=0x7f41e01648b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:41.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41ddd9b700 1 --2- 172.21.15.121:0/1636350454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41e00628a0 0x7f41e01648b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:41.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41ddd9b700 1 -- 172.21.15.121:0/1636350454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d00092a0 con 0x7f41e00a8de0 2023-11-15T03:07:41.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41ddd9b700 1 --2- 172.21.15.121:0/1636350454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41e00a8de0 0x7f41e01468e0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 rx=0x7f41d400c590 tx=0x7f41d4007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:41.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41ce7fc700 1 -- 172.21.15.121:0/1636350454 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d40072e0 con 0x7f41e00a8de0 2023-11-15T03:07:41.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41ce7fc700 1 -- 172.21.15.121:0/1636350454 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d4007440 con 0x7f41e00a8de0 2023-11-15T03:07:41.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41e4cf1700 1 -- 172.21.15.121:0/1636350454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e0148990 con 0x7f41e00a8de0 2023-11-15T03:07:41.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41ce7fc700 1 -- 172.21.15.121:0/1636350454 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d40057a0 con 0x7f41e00a8de0 2023-11-15T03:07:41.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.076+0000 7f41e4cf1700 1 -- 172.21.15.121:0/1636350454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e0148060 con 0x7f41e00a8de0 2023-11-15T03:07:41.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.080+0000 7f41e4cf1700 1 -- 172.21.15.121:0/1636350454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e0148e90 con 0x7f41e00a8de0 2023-11-15T03:07:41.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.080+0000 7f41ce7fc700 1 -- 172.21.15.121:0/1636350454 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f41d4005bd0 con 0x7f41e00a8de0 2023-11-15T03:07:41.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.080+0000 7f41ce7fc700 1 --2- 172.21.15.121:0/1636350454 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41c005a510 0x7f41c005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:41.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.080+0000 7f41ce7fc700 1 -- 172.21.15.121:0/1636350454 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8244+0+0 (secure 0 0 0) 0x7f41d40744f0 con 0x7f41e00a8de0 2023-11-15T03:07:41.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.080+0000 7f41de59c700 1 --2- 172.21.15.121:0/1636350454 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41c005a510 0x7f41c005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:41.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.080+0000 7f41de59c700 1 --2- 172.21.15.121:0/1636350454 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41c005a510 0x7f41c005c9d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f41d0009760 tx=0x7f41d0017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:41.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.088+0000 7f41ce7fc700 1 -- 172.21.15.121:0/1636350454 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f41d40473c0 con 0x7f41e00a8de0 2023-11-15T03:07:41.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:41.288+0000 7f41e4cf1700 1 -- 172.21.15.121:0/1636350454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f41e0047060 con 0x7f41e00a8de0 2023-11-15T03:07:41.990 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:07:41.998 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:07:42.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.160+0000 7f41ce7fc700 1 -- 172.21.15.121:0/1636350454 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v224) v1 ==== 161+0+0 (secure 0 0 0) 0x7f41d4018030 con 0x7f41e00a8de0 2023-11-15T03:07:42.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.164+0000 7f41e4cf1700 1 -- 172.21.15.121:0/1636350454 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41c005a510 msgr2=0x7f41c005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:42.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.164+0000 7f41e4cf1700 1 --2- 172.21.15.121:0/1636350454 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41c005a510 0x7f41c005c9d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f41d0009760 tx=0x7f41d0017000).stop 2023-11-15T03:07:42.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.164+0000 7f41e4cf1700 1 -- 172.21.15.121:0/1636350454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41e00a8de0 msgr2=0x7f41e01468e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:42.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.164+0000 7f41e4cf1700 1 --2- 172.21.15.121:0/1636350454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41e00a8de0 0x7f41e01468e0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 rx=0x7f41d400c590 tx=0x7f41d4007c80).stop 2023-11-15T03:07:42.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.164+0000 7f41e4cf1700 1 -- 172.21.15.121:0/1636350454 shutdown_connections 2023-11-15T03:07:42.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.164+0000 7f41e4cf1700 1 --2- 172.21.15.121:0/1636350454 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41c005a510 0x7f41c005c9d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:42.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.164+0000 7f41e4cf1700 1 --2- 172.21.15.121:0/1636350454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41e0150f90 0x7f41e014c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:42.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.164+0000 7f41e4cf1700 1 --2- 172.21.15.121:0/1636350454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41e00a8de0 0x7f41e01468e0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:42.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.164+0000 7f41e4cf1700 1 --2- 172.21.15.121:0/1636350454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41e00628a0 0x7f41e01648b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:42.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.164+0000 7f41e4cf1700 1 -- 172.21.15.121:0/1636350454 >> 172.21.15.121:0/1636350454 conn(0x7f41e0067a60 msgr2=0x7f41e0152b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:42.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.164+0000 7f41e4cf1700 1 -- 172.21.15.121:0/1636350454 shutdown_connections 2023-11-15T03:07:42.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.164+0000 7f41e4cf1700 1 -- 172.21.15.121:0/1636350454 wait complete. 2023-11-15T03:07:42.175 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:07:42.193 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:07:42.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.352+0000 7f9348769700 1 Processor -- start 2023-11-15T03:07:42.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.352+0000 7f9348769700 1 -- start start 2023-11-15T03:07:42.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.356+0000 7f9348769700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93400c2ed0 0x7f93401580e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:42.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.356+0000 7f9348769700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f93400bd830 0x7f93400bdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:42.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.356+0000 7f9348769700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93401551f0 con 0x7f93400becf0 2023-11-15T03:07:42.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.356+0000 7f9348769700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93400a9170 con 0x7f93400bd830 2023-11-15T03:07:42.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.356+0000 7f9348769700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9340157c50 con 0x7f93400c2ed0 2023-11-15T03:07:42.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.356+0000 7f9345d04700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f93400becf0 0x7f9340154140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:42696/0 (socket says 172.21.15.121:42696) 2023-11-15T03:07:42.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.356+0000 7f9345d04700 1 -- 172.21.15.121:0/1707913338 learned_addr learned my addr 172.21.15.121:0/1707913338 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:42.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.356+0000 7f9346505700 1 --2- 172.21.15.121:0/1707913338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93400c2ed0 0x7f93401580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:42.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.356+0000 7f9346d06700 1 --2- 172.21.15.121:0/1707913338 >> v2:172.21.15.190:3300/0 conn(0x7f93400bd830 0x7f93400bdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.356+0000 7f9345503700 1 -- 172.21.15.121:0/1707913338 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1797956449 0 0) 0x7f93401551f0 con 0x7f93400becf0 2023-11-15T03:07:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.356+0000 7f9345503700 1 -- 172.21.15.121:0/1707913338 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f932c0033f0 con 0x7f93400becf0 2023-11-15T03:07:42.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.356+0000 7f9346505700 1 -- 172.21.15.121:0/1707913338 >> v2:172.21.15.190:3300/0 conn(0x7f93400bd830 msgr2=0x7f93400bdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:42.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.356+0000 7f9346505700 1 --2- 172.21.15.121:0/1707913338 >> v2:172.21.15.190:3300/0 conn(0x7f93400bd830 0x7f93400bdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:42.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.356+0000 7f9346505700 1 -- 172.21.15.121:0/1707913338 >> v1:172.21.15.190:6789/0 conn(0x7f93400becf0 legacy=0x7f9340154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:42.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.356+0000 7f9346505700 1 -- 172.21.15.121:0/1707913338 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9340154c30 con 0x7f93400c2ed0 2023-11-15T03:07:42.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.356+0000 7f9346d06700 1 --2- 172.21.15.121:0/1707913338 >> v2:172.21.15.190:3300/0 conn(0x7f93400bd830 0x7f93400bdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:07:42.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.360+0000 7f9346505700 1 --2- 172.21.15.121:0/1707913338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93400c2ed0 0x7f93401580e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f933c009760 tx=0x7f933c009060).ready entity=mon.2 client_cookie=643701fa1deb2f4e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:42.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.360+0000 7f9345503700 1 -- 172.21.15.121:0/1707913338 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c00d4a0 con 0x7f93400c2ed0 2023-11-15T03:07:42.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.360+0000 7f9345503700 1 -- 172.21.15.121:0/1707913338 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933c00d600 con 0x7f93400c2ed0 2023-11-15T03:07:42.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.360+0000 7f9345503700 1 -- 172.21.15.121:0/1707913338 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c00d980 con 0x7f93400c2ed0 2023-11-15T03:07:42.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.360+0000 7f9348769700 1 -- 172.21.15.121:0/1707913338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93400c2ed0 msgr2=0x7f93401580e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:42.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.360+0000 7f9348769700 1 --2- 172.21.15.121:0/1707913338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93400c2ed0 0x7f93401580e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f933c009760 tx=0x7f933c009060).stop 2023-11-15T03:07:42.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.360+0000 7f9348769700 1 -- 172.21.15.121:0/1707913338 shutdown_connections 2023-11-15T03:07:42.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.360+0000 7f9348769700 1 --2- 172.21.15.121:0/1707913338 >> v2:172.21.15.190:3300/0 conn(0x7f93400bd830 0x7f93400bdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:42.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.360+0000 7f9348769700 1 --2- 172.21.15.121:0/1707913338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93400c2ed0 0x7f93401580e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:42.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.360+0000 7f9348769700 1 -- 172.21.15.121:0/1707913338 >> 172.21.15.121:0/1707913338 conn(0x7f9340067a60 msgr2=0x7f9340067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:42.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.360+0000 7f9348769700 1 -- 172.21.15.121:0/1707913338 shutdown_connections 2023-11-15T03:07:42.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.360+0000 7f9348769700 1 -- 172.21.15.121:0/1707913338 wait complete. 2023-11-15T03:07:42.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.364+0000 7f9348769700 1 Processor -- start 2023-11-15T03:07:42.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.364+0000 7f9348769700 1 -- start start 2023-11-15T03:07:42.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.364+0000 7f9348769700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93400bd830 0x7f9340146650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:42.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.364+0000 7f9348769700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93400becf0 0x7f93400635a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:42.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.364+0000 7f9348769700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93400c2ed0 0x7f93400bf970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:42.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.364+0000 7f9348769700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9340063110 con 0x7f93400becf0 2023-11-15T03:07:42.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.364+0000 7f9348769700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9340062500 con 0x7f93400c2ed0 2023-11-15T03:07:42.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.364+0000 7f9348769700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93400627e0 con 0x7f93400bd830 2023-11-15T03:07:42.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.368+0000 7f9345d04700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93400becf0 0x7f93400635a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:42.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.368+0000 7f9345d04700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93400becf0 0x7f93400635a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44140/0 (socket says 172.21.15.121:44140) 2023-11-15T03:07:42.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.368+0000 7f9345d04700 1 -- 172.21.15.121:0/1137036861 learned_addr learned my addr 172.21.15.121:0/1137036861 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:42.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.368+0000 7f9346d06700 1 --2- 172.21.15.121:0/1137036861 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93400c2ed0 0x7f93400bf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:42.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.368+0000 7f9345d04700 1 -- 172.21.15.121:0/1137036861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93400bd830 msgr2=0x7f9340146650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:42.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.368+0000 7f9345d04700 1 --2- 172.21.15.121:0/1137036861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93400bd830 0x7f9340146650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:07:42.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.368+0000 7f9345d04700 1 -- 172.21.15.121:0/1137036861 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93400c2ed0 msgr2=0x7f93400bf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:42.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.368+0000 7f9345d04700 1 --2- 172.21.15.121:0/1137036861 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93400c2ed0 0x7f93400bf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:42.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.368+0000 7f9345d04700 1 -- 172.21.15.121:0/1137036861 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9338009a20 con 0x7f93400becf0 2023-11-15T03:07:42.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.368+0000 7f9345d04700 1 --2- 172.21.15.121:0/1137036861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93400becf0 0x7f93400635a0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 rx=0x7f9330001ed0 tx=0x7f9330007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:42.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.368+0000 7f93367fc700 1 -- 172.21.15.121:0/1137036861 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93300072e0 con 0x7f93400becf0 2023-11-15T03:07:42.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.368+0000 7f93367fc700 1 -- 172.21.15.121:0/1137036861 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9330007440 con 0x7f93400becf0 2023-11-15T03:07:42.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.368+0000 7f93367fc700 1 -- 172.21.15.121:0/1137036861 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93300057a0 con 0x7f93400becf0 2023-11-15T03:07:42.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.368+0000 7f9348769700 1 -- 172.21.15.121:0/1137036861 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933c0092a0 con 0x7f93400becf0 2023-11-15T03:07:42.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.368+0000 7f9348769700 1 -- 172.21.15.121:0/1137036861 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9340062da0 con 0x7f93400becf0 2023-11-15T03:07:42.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.368+0000 7f9348769700 1 -- 172.21.15.121:0/1137036861 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9340047060 con 0x7f93400becf0 2023-11-15T03:07:42.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.372+0000 7f93367fc700 1 -- 172.21.15.121:0/1137036861 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9330005b50 con 0x7f93400becf0 2023-11-15T03:07:42.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.372+0000 7f93367fc700 1 --2- 172.21.15.121:0/1137036861 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f932405a510 0x7f932405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:42.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.372+0000 7f93367fc700 1 -- 172.21.15.121:0/1137036861 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8244+0+0 (secure 0 0 0) 0x7f93300747a0 con 0x7f93400becf0 2023-11-15T03:07:42.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.372+0000 7f9346505700 1 --2- 172.21.15.121:0/1137036861 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f932405a510 0x7f932405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:42.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.372+0000 7f9346505700 1 --2- 172.21.15.121:0/1137036861 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f932405a510 0x7f932405c9d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f933c00d260 tx=0x7f933c016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:42.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.380+0000 7f93367fc700 1 -- 172.21.15.121:0/1137036861 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f93300476f0 con 0x7f93400becf0 2023-11-15T03:07:42.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:42.596+0000 7f9348769700 1 -- 172.21.15.121:0/1137036861 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f9340154f10 con 0x7f93400becf0 2023-11-15T03:07:43.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.172+0000 7f93367fc700 1 -- 172.21.15.121:0/1137036861 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v225) v1 ==== 101+0+0 (secure 0 0 0) 0x7f933004ad10 con 0x7f93400becf0 2023-11-15T03:07:43.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.176+0000 7f9348769700 1 -- 172.21.15.121:0/1137036861 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f932405a510 msgr2=0x7f932405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.176+0000 7f9348769700 1 --2- 172.21.15.121:0/1137036861 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f932405a510 0x7f932405c9d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f933c00d260 tx=0x7f933c016000).stop 2023-11-15T03:07:43.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.176+0000 7f9348769700 1 -- 172.21.15.121:0/1137036861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93400becf0 msgr2=0x7f93400635a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.176+0000 7f9348769700 1 --2- 172.21.15.121:0/1137036861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93400becf0 0x7f93400635a0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 rx=0x7f9330001ed0 tx=0x7f9330007bd0).stop 2023-11-15T03:07:43.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.176+0000 7f9348769700 1 -- 172.21.15.121:0/1137036861 shutdown_connections 2023-11-15T03:07:43.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.176+0000 7f9348769700 1 --2- 172.21.15.121:0/1137036861 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f932405a510 0x7f932405c9d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.176+0000 7f9348769700 1 --2- 172.21.15.121:0/1137036861 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93400c2ed0 0x7f93400bf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.176+0000 7f9348769700 1 --2- 172.21.15.121:0/1137036861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93400becf0 0x7f93400635a0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.176+0000 7f9348769700 1 --2- 172.21.15.121:0/1137036861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93400bd830 0x7f9340146650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:07:43.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.176+0000 7f9348769700 1 -- 172.21.15.121:0/1137036861 >> 172.21.15.121:0/1137036861 conn(0x7f9340067a60 msgr2=0x7f93400aaca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:43.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.176+0000 7f9348769700 1 -- 172.21.15.121:0/1137036861 shutdown_connections 2023-11-15T03:07:43.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.176+0000 7f9348769700 1 -- 172.21.15.121:0/1137036861 wait complete. 2023-11-15T03:07:43.186 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:07:43.203 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:07:43.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.252+0000 7f1f0a530d00 1 Processor -- start 2023-11-15T03:07:43.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1f0a530d00 1 -- start start 2023-11-15T03:07:43.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1f0a530d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f64b8ce940 0x55f64b841530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1f0a530d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f64b7f4660 0x55f64b8ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1f0a530d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f64b827680 0x55f64b827b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1f0a530d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64b7a2cc0 con 0x55f64b827680 2023-11-15T03:07:43.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1f0a530d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64b76d220 con 0x55f64b8ce940 2023-11-15T03:07:43.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1f0a530d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f64b827230 con 0x55f64b7f4660 2023-11-15T03:07:43.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1eff343700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f64b827680 0x55f64b827b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1eff343700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f64b827680 0x55f64b827b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44160/0 (socket says 172.21.15.121:44160) 2023-11-15T03:07:43.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1eff343700 1 -- 172.21.15.121:0/389722886 learned_addr learned my addr 172.21.15.121:0/389722886 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:43.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1efe341700 1 --2- 172.21.15.121:0/389722886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f64b7f4660 0x55f64b8ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1efeb42700 1 --2- 172.21.15.121:0/389722886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f64b8ce940 0x55f64b841530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1eff343700 1 -- 172.21.15.121:0/389722886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f64b7f4660 msgr2=0x55f64b8ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:43.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1eff343700 1 --2- 172.21.15.121:0/389722886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f64b7f4660 0x55f64b8ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1eff343700 1 -- 172.21.15.121:0/389722886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f64b8ce940 msgr2=0x55f64b841530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:43.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1eff343700 1 --2- 172.21.15.121:0/389722886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f64b8ce940 0x55f64b841530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1eff343700 1 -- 172.21.15.121:0/389722886 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f64b765a70 con 0x55f64b827680 2023-11-15T03:07:43.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1eff343700 1 --2- 172.21.15.121:0/389722886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f64b827680 0x55f64b827b00 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 rx=0x7f1ef80020c0 tx=0x55f64b8d1130).ready entity=mon.0 client_cookie=4cc4430d1f0c5e19 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:43.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1efd33f700 1 -- 172.21.15.121:0/389722886 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef800f450 con 0x55f64b827680 2023-11-15T03:07:43.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1efd33f700 1 -- 172.21.15.121:0/389722886 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef800f5d0 con 0x55f64b827680 2023-11-15T03:07:43.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.256+0000 7f1efd33f700 1 -- 172.21.15.121:0/389722886 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef800f990 con 0x55f64b827680 2023-11-15T03:07:43.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1f0a530d00 1 -- 172.21.15.121:0/389722886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f64b827680 msgr2=0x55f64b827b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/389722886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f64b827680 0x55f64b827b00 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 rx=0x7f1ef80020c0 tx=0x55f64b8d1130).stop 2023-11-15T03:07:43.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1f0a530d00 1 -- 172.21.15.121:0/389722886 shutdown_connections 2023-11-15T03:07:43.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/389722886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f64b827680 0x55f64b827b00 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/389722886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f64b7f4660 0x55f64b8ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/389722886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f64b8ce940 0x55f64b841530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1f0a530d00 1 -- 172.21.15.121:0/389722886 >> 172.21.15.121:0/389722886 conn(0x55f64b844080 msgr2=0x55f64b8cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:43.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1f0a530d00 1 -- 172.21.15.121:0/389722886 shutdown_connections 2023-11-15T03:07:43.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1f0a530d00 1 -- 172.21.15.121:0/389722886 wait complete. 2023-11-15T03:07:43.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1f0a530d00 1 Processor -- start 2023-11-15T03:07:43.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1f0a530d00 1 -- start start 2023-11-15T03:07:43.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1f0a530d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f64b7f4660 0x55f64b8bee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1f0a530d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f64b827680 0x55f64b8cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1f0a530d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f64b8ce940 0x55f64b8291b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1f0a530d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64b8bf380 con 0x55f64b8ce940 2023-11-15T03:07:43.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1f0a530d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64b8bfad0 con 0x55f64b7f4660 2023-11-15T03:07:43.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1f0a530d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f64b8bfdb0 con 0x55f64b827680 2023-11-15T03:07:43.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1eff343700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f64b8ce940 0x55f64b8291b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1eff343700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f64b8ce940 0x55f64b8291b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44166/0 (socket says 172.21.15.121:44166) 2023-11-15T03:07:43.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 learned_addr learned my addr 172.21.15.121:0/42818720 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:43.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1efe341700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f64b827680 0x55f64b8cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1efeb42700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f64b7f4660 0x55f64b8bee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f64b827680 msgr2=0x55f64b8cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1eff343700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f64b827680 0x55f64b8cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f64b7f4660 msgr2=0x55f64b8bee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1eff343700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f64b7f4660 0x55f64b8bee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.260+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef800e040 con 0x55f64b8ce940 2023-11-15T03:07:43.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.264+0000 7f1eff343700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f64b8ce940 0x55f64b8291b0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 rx=0x55f64b844c90 tx=0x7f1ef8009000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:43.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.264+0000 7f1eef7fe700 1 -- 172.21.15.121:0/42818720 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef801b070 con 0x55f64b8ce940 2023-11-15T03:07:43.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.264+0000 7f1eef7fe700 1 -- 172.21.15.121:0/42818720 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef800a070 con 0x55f64b8ce940 2023-11-15T03:07:43.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.264+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f64b771010 con 0x55f64b8ce940 2023-11-15T03:07:43.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.264+0000 7f1eef7fe700 1 -- 172.21.15.121:0/42818720 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef800f450 con 0x55f64b8ce940 2023-11-15T03:07:43.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.264+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f64b8c0370 con 0x55f64b8ce940 2023-11-15T03:07:43.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.264+0000 7f1eef7fe700 1 -- 172.21.15.121:0/42818720 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1ef801e070 con 0x55f64b8ce940 2023-11-15T03:07:43.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.268+0000 7f1eef7fe700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1ee4059020 0x7f1ee405b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.268+0000 7f1eef7fe700 1 -- 172.21.15.121:0/42818720 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8244+0+0 (secure 0 0 0) 0x7f1ef80745a0 con 0x55f64b8ce940 2023-11-15T03:07:43.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.268+0000 7f1efeb42700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1ee4059020 0x7f1ee405b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.283 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:07:43.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.268+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55f64b8bf5c0 0x55f64b8c1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.268+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8c1570 con 0x55f64b8bf5c0 2023-11-15T03:07:43.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.268+0000 7f1efe341700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55f64b8bf5c0 0x55f64b8c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.268+0000 7f1efeb42700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1ee4059020 0x7f1ee405b4e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f1ef00010e0 tx=0x7f1ef00018f0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:43.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.268+0000 7f1efe341700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55f64b8bf5c0 0x55f64b8c1080 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:43.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.272+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v117'269 uv262 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8bf5c0 2023-11-15T03:07:43.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.272+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b8c1920 con 0x55f64b8bf5c0 2023-11-15T03:07:43.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.272+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v117'270 uv270 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8bf5c0 2023-11-15T03:07:43.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.272+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8eb150 con 0x55f64b8bf5c0 2023-11-15T03:07:43.286 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:07:43.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.276+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v117'271 uv271 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8bf5c0 2023-11-15T03:07:43.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.276+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8ec510 con 0x55f64b8bf5c0 2023-11-15T03:07:43.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.276+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v117'272 uv272 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8bf5c0 2023-11-15T03:07:43.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.276+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8ec8c0 con 0x55f64b8bf5c0 2023-11-15T03:07:43.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.280+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v117'273 uv273 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8bf5c0 2023-11-15T03:07:43.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.280+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8eb500 con 0x55f64b8bf5c0 2023-11-15T03:07:43.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.280+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v117'274 uv274 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8bf5c0 2023-11-15T03:07:43.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.280+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8eb900 con 0x55f64b8bf5c0 2023-11-15T03:07:43.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.284+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v117'275 uv268 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8bf5c0 2023-11-15T03:07:43.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.284+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b8ebd00 con 0x55f64b8bf5c0 2023-11-15T03:07:43.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.288+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v117'276 uv276 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8bf5c0 2023-11-15T03:07:43.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.288+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8ebd00 con 0x55f64b8bf5c0 2023-11-15T03:07:43.296 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:07:43.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.288+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v117'277 uv277 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8bf5c0 2023-11-15T03:07:43.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.288+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8ebd00 con 0x55f64b8bf5c0 2023-11-15T03:07:43.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.292+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v117'278 uv278 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8bf5c0 2023-11-15T03:07:43.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.292+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8ebd00 con 0x55f64b8bf5c0 2023-11-15T03:07:43.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.292+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v117'279 uv279 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8bf5c0 2023-11-15T03:07:43.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.292+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8ebd00 con 0x55f64b8bf5c0 2023-11-15T03:07:43.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.296+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v117'280 uv280 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8bf5c0 2023-11-15T03:07:43.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.296+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55f64b8ec100 0x55f64b8ef190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.296+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8ebd00 con 0x55f64b8ec100 2023-11-15T03:07:43.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.296+0000 7f1efeb42700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55f64b8ec100 0x55f64b8ef190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.296+0000 7f1efeb42700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55f64b8ec100 0x55f64b8ef190 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:43.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.300+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v117'423 uv410 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.300+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b8ef680 con 0x55f64b8ec100 2023-11-15T03:07:43.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.304+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v117'424 uv424 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.304+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8efa30 con 0x55f64b8ec100 2023-11-15T03:07:43.315 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:07:43.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.308+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v117'425 uv425 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.308+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f0df0 con 0x55f64b8ec100 2023-11-15T03:07:43.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.312+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v117'426 uv426 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.312+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f11a0 con 0x55f64b8ec100 2023-11-15T03:07:43.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.312+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v117'427 uv427 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.312+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8efde0 con 0x55f64b8ec100 2023-11-15T03:07:43.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.316+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v117'428 uv428 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.316+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f06d0 con 0x55f64b8ec100 2023-11-15T03:07:43.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.320+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v117'429 uv429 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.320+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f1550 con 0x55f64b8ec100 2023-11-15T03:07:43.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.320+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v117'430 uv416 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.320+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b8f1550 con 0x55f64b8ec100 2023-11-15T03:07:43.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.324+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v117'431 uv431 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.324+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f1550 con 0x55f64b8ec100 2023-11-15T03:07:43.337 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:07:43.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.328+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v117'432 uv432 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.328+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f2560 con 0x55f64b8ec100 2023-11-15T03:07:43.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.328+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v117'433 uv433 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.328+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f2560 con 0x55f64b8ec100 2023-11-15T03:07:43.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.332+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v117'434 uv434 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.332+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f1550 con 0x55f64b8ec100 2023-11-15T03:07:43.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.336+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v117'435 uv435 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.336+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f1550 con 0x55f64b8ec100 2023-11-15T03:07:43.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.336+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v117'436 uv422 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.336+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b8f1550 con 0x55f64b8ec100 2023-11-15T03:07:43.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.340+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v117'437 uv437 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.340+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f1550 con 0x55f64b8ec100 2023-11-15T03:07:43.351 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:07:43.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.344+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v117'438 uv438 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.344+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f2560 con 0x55f64b8ec100 2023-11-15T03:07:43.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.348+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v117'439 uv439 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.348+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f2560 con 0x55f64b8ec100 2023-11-15T03:07:43.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.348+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v117'440 uv440 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.348+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f1550 con 0x55f64b8ec100 2023-11-15T03:07:43.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.352+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v117'441 uv441 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.352+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55f64b8f1a60 0x55f64b8f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.352+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f4340 con 0x55f64b8f1a60 2023-11-15T03:07:43.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.352+0000 7f1efe341700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55f64b8f1a60 0x55f64b8f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.352+0000 7f1efe341700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55f64b8f1a60 0x55f64b8f3e50 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:43.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.356+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v117'231 uv225 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8f1a60 2023-11-15T03:07:43.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.356+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b8f4340 con 0x55f64b8f1a60 2023-11-15T03:07:43.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.360+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v117'232 uv232 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8f1a60 2023-11-15T03:07:43.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.360+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f4340 con 0x55f64b8f1a60 2023-11-15T03:07:43.370 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:07:43.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.364+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v117'233 uv233 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8f1a60 2023-11-15T03:07:43.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.364+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f47c0 con 0x55f64b8f1a60 2023-11-15T03:07:43.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.364+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v117'234 uv234 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8f1a60 2023-11-15T03:07:43.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.364+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f0260 con 0x55f64b8f1a60 2023-11-15T03:07:43.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.368+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v117'235 uv235 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8f1a60 2023-11-15T03:07:43.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.368+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f0260 con 0x55f64b8f1a60 2023-11-15T03:07:43.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.372+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v117'236 uv230 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8f1a60 2023-11-15T03:07:43.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.372+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b8f01e0 con 0x55f64b8f1a60 2023-11-15T03:07:43.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.372+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v117'237 uv237 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8f1a60 2023-11-15T03:07:43.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.372+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f01e0 con 0x55f64b8f1a60 2023-11-15T03:07:43.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.376+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v117'238 uv238 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8f1a60 2023-11-15T03:07:43.384 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:07:43.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.376+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f01e0 con 0x55f64b8f1a60 2023-11-15T03:07:43.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.380+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v117'239 uv239 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8f1a60 2023-11-15T03:07:43.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.380+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f01e0 con 0x55f64b8f1a60 2023-11-15T03:07:43.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.384+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v117'240 uv240 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8f1a60 2023-11-15T03:07:43.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.384+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f64b8f51c0 0x55f64b8f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.384+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f01e0 con 0x55f64b8f51c0 2023-11-15T03:07:43.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.384+0000 7f1eff343700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f64b8f51c0 0x55f64b8f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.388+0000 7f1eff343700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f64b8f51c0 0x55f64b8f75b0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:43.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.392+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v117'274 uv267 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.392+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b8f7ba0 con 0x55f64b8f51c0 2023-11-15T03:07:43.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.392+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v117'275 uv275 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.392+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f8050 con 0x55f64b8f51c0 2023-11-15T03:07:43.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.396+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v117'276 uv276 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.404 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:07:43.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.396+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f48a0 con 0x55f64b8f51c0 2023-11-15T03:07:43.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.400+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v117'277 uv277 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.400+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8fcad0 con 0x55f64b8f51c0 2023-11-15T03:07:43.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.404+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v117'278 uv278 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.404+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f8450 con 0x55f64b8f51c0 2023-11-15T03:07:43.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.404+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v117'279 uv273 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.404+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b8f8850 con 0x55f64b8f51c0 2023-11-15T03:07:43.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.408+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v117'280 uv280 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.408+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f9200 con 0x55f64b8f51c0 2023-11-15T03:07:43.419 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:07:43.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.412+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v117'281 uv281 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.412+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8fa230 con 0x55f64b8f51c0 2023-11-15T03:07:43.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.416+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v117'282 uv282 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.416+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8fa220 con 0x55f64b8f51c0 2023-11-15T03:07:43.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.420+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v117'283 uv283 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.420+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f90f0 con 0x55f64b8f51c0 2023-11-15T03:07:43.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.420+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v117'284 uv284 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.420+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55f64b8f9770 0x55f64b8f9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.420+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8fced0 con 0x55f64b8f9770 2023-11-15T03:07:43.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.424+0000 7f1efeb42700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55f64b8f9770 0x55f64b8f9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.424+0000 7f1efeb42700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55f64b8f9770 0x55f64b8f9b50 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:43.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.428+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v117'135 uv134 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8f9770 2023-11-15T03:07:43.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.428+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b8fced0 con 0x55f64b8f9770 2023-11-15T03:07:43.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.428+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v117'136 uv136 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8f9770 2023-11-15T03:07:43.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.428+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8fced0 con 0x55f64b8f9770 2023-11-15T03:07:43.439 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:07:43.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.432+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v117'137 uv137 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8f9770 2023-11-15T03:07:43.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.432+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8fdee0 con 0x55f64b8f9770 2023-11-15T03:07:43.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.432+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v117'138 uv138 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8f9770 2023-11-15T03:07:43.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.432+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8fdee0 con 0x55f64b8f9770 2023-11-15T03:07:43.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.436+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v117'139 uv139 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8f9770 2023-11-15T03:07:43.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.436+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8fced0 con 0x55f64b8f9770 2023-11-15T03:07:43.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.436+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v117'140 uv140 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8f9770 2023-11-15T03:07:43.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.436+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55f64b8fd7e0 0x55f64b8fdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.436+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9002f0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.440+0000 7f1efe341700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55f64b8fd7e0 0x55f64b8fdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.440+0000 7f1efe341700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55f64b8fd7e0 0x55f64b8fdbf0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:43.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.440+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v117'96 uv95 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.444+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b9002f0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.444+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v117'97 uv97 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.444+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9002f0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.448+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v117'98 uv98 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.457 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:07:43.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.448+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9007e0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.452+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v117'99 uv99 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.452+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9007e0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.452+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v117'135 uv134 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.452+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b9007e0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.456+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v117'136 uv136 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.456+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f8d20 con 0x55f64b8fd7e0 2023-11-15T03:07:43.467 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:07:43.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.460+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v117'137 uv137 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.460+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f8d20 con 0x55f64b8fd7e0 2023-11-15T03:07:43.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.464+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v117'138 uv138 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.464+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f8d20 con 0x55f64b8fd7e0 2023-11-15T03:07:43.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.464+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v117'139 uv139 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.464+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f8d20 con 0x55f64b8fd7e0 2023-11-15T03:07:43.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.472+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v117'140 uv140 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.472+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f8d20 con 0x55f64b8ec100 2023-11-15T03:07:43.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.472+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v117'249 uv243 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.472+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b8f8d20 con 0x55f64b8ec100 2023-11-15T03:07:43.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.476+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v117'250 uv250 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.476+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f8d20 con 0x55f64b8ec100 2023-11-15T03:07:43.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.480+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v117'251 uv251 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.489 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:07:43.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.480+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f8d20 con 0x55f64b8ec100 2023-11-15T03:07:43.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.484+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v117'252 uv252 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.484+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f8d20 con 0x55f64b8ec100 2023-11-15T03:07:43.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.488+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v117'253 uv253 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.488+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f8d20 con 0x55f64b8ec100 2023-11-15T03:07:43.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.488+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v117'254 uv254 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.488+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f8d20 con 0x55f64b8ec100 2023-11-15T03:07:43.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.492+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v117'255 uv248 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.492+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b8f8d20 con 0x55f64b8ec100 2023-11-15T03:07:43.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.496+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v117'256 uv256 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.496+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f8d20 con 0x55f64b8ec100 2023-11-15T03:07:43.507 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:07:43.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.500+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v117'257 uv257 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.500+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b8f8d20 con 0x55f64b8ec100 2023-11-15T03:07:43.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.504+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v117'258 uv258 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.504+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9008d0 con 0x55f64b8ec100 2023-11-15T03:07:43.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.508+0000 7f1efeb42700 1 -- 172.21.15.121:0/42818720 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v117'259 uv259 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef00139b0 con 0x55f64b8ec100 2023-11-15T03:07:43.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.508+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9008d0 con 0x55f64b8f1a60 2023-11-15T03:07:43.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.508+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v117'112 uv111 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8f1a60 2023-11-15T03:07:43.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.508+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b9008d0 con 0x55f64b8f1a60 2023-11-15T03:07:43.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.512+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v117'113 uv113 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8f1a60 2023-11-15T03:07:43.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.512+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9008d0 con 0x55f64b8f1a60 2023-11-15T03:07:43.522 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:07:43.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.516+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v117'114 uv114 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8f1a60 2023-11-15T03:07:43.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.516+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9008d0 con 0x55f64b8f1a60 2023-11-15T03:07:43.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.516+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v117'115 uv115 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8f1a60 2023-11-15T03:07:43.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.516+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9008d0 con 0x55f64b8f1a60 2023-11-15T03:07:43.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.520+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v117'116 uv116 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8f1a60 2023-11-15T03:07:43.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.520+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9008d0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.524+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v117'135 uv134 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.524+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b9008d0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.524+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v117'136 uv136 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.524+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9008d0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.536 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:07:43.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.528+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v117'137 uv137 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.528+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9018e0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.532+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v117'138 uv138 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.532+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9018e0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.536+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v117'139 uv139 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.536+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9008d0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.540+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v117'140 uv140 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.540+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9008d0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.544+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v117'113 uv112 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.544+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b9008d0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.548+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v117'114 uv114 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.548+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9008d0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.559 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:07:43.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.552+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v117'115 uv115 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.552+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9008d0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.556+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v117'116 uv116 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.556+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9008d0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.560+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v117'117 uv117 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.560+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9008d0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.560+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v117'111 uv110 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.560+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b9008d0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.564+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v117'112 uv112 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.564+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9008d0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.575 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:07:43.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.568+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v117'113 uv113 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.568+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9008d0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.568+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v117'114 uv114 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.572+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b903640 con 0x55f64b8fd7e0 2023-11-15T03:07:43.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.572+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v117'115 uv115 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.572+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55f64b9012d0 0x55f64b9036c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.572+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b903bb0 con 0x55f64b9012d0 2023-11-15T03:07:43.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.572+0000 7f1eff343700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55f64b9012d0 0x55f64b9036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.576+0000 7f1eff343700 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55f64b9012d0 0x55f64b9036c0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:43.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.576+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v117'90 uv89 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b9012d0 2023-11-15T03:07:43.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.576+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b903bb0 con 0x55f64b9012d0 2023-11-15T03:07:43.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.580+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v117'91 uv91 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b9012d0 2023-11-15T03:07:43.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.580+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b903bb0 con 0x55f64b9012d0 2023-11-15T03:07:43.591 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:07:43.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.584+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v117'92 uv92 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b9012d0 2023-11-15T03:07:43.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.584+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b903bb0 con 0x55f64b9012d0 2023-11-15T03:07:43.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.588+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v117'93 uv93 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b9012d0 2023-11-15T03:07:43.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.588+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b903bb0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.588+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v117'135 uv134 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.588+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b903bb0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.592+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v117'136 uv136 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.592+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b903bb0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.603 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:07:43.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.596+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v117'137 uv137 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.596+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b904bc0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.600+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v117'138 uv138 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.600+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b904bc0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.600+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v117'139 uv139 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.600+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b903bb0 con 0x55f64b8fd7e0 2023-11-15T03:07:43.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.604+0000 7f1efe341700 1 -- 172.21.15.121:0/42818720 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v117'140 uv140 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef4005580 con 0x55f64b8fd7e0 2023-11-15T03:07:43.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.604+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b903bb0 con 0x55f64b8f51c0 2023-11-15T03:07:43.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.608+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v117'115 uv114 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.608+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b903bb0 con 0x55f64b8f51c0 2023-11-15T03:07:43.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.608+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v117'116 uv116 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.608+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b903bb0 con 0x55f64b8f51c0 2023-11-15T03:07:43.619 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:07:43.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.612+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v117'117 uv117 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.612+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b903bb0 con 0x55f64b8f51c0 2023-11-15T03:07:43.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.616+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v117'118 uv118 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.616+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b903bb0 con 0x55f64b8f51c0 2023-11-15T03:07:43.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.616+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v117'119 uv119 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.616+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b903bb0 con 0x55f64b8f51c0 2023-11-15T03:07:43.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.620+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v117'269 uv268 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.620+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b903bb0 con 0x55f64b8f51c0 2023-11-15T03:07:43.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.624+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v117'270 uv270 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.624+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b903bb0 con 0x55f64b8f51c0 2023-11-15T03:07:43.632 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:07:43.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.624+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v117'271 uv271 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.624+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b903bb0 con 0x55f64b8f51c0 2023-11-15T03:07:43.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.628+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v117'272 uv272 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.628+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b903fb0 con 0x55f64b8f51c0 2023-11-15T03:07:43.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.628+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v117'273 uv273 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.628+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b903fb0 con 0x55f64b8f51c0 2023-11-15T03:07:43.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.632+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v117'274 uv267 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.632+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e117) v8 -- 0x55f64b903bb0 con 0x55f64b8f51c0 2023-11-15T03:07:43.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.632+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v117'275 uv275 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.636+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b903bb0 con 0x55f64b8f51c0 2023-11-15T03:07:43.644 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:07:43.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.636+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v117'276 uv276 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.636+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b904bc0 con 0x55f64b8f51c0 2023-11-15T03:07:43.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.640+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v117'277 uv277 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.640+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b904bc0 con 0x55f64b8f51c0 2023-11-15T03:07:43.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.640+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v117'278 uv278 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.640+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9040a0 con 0x55f64b8f51c0 2023-11-15T03:07:43.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.644+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v117'279 uv279 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.644+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x55f64b9040a0 con 0x55f64b8f51c0 2023-11-15T03:07:43.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v117'280 uv280 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1ef801d030 con 0x55f64b8f51c0 2023-11-15T03:07:43.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55f64b8f1a60 msgr2=0x55f64b8f3e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55f64b8f1a60 0x55f64b8f3e50 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55f64b8bf5c0 msgr2=0x55f64b8c1080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55f64b8bf5c0 0x55f64b8c1080 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55f64b8f9770 msgr2=0x55f64b8f9b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55f64b8f9770 0x55f64b8f9b50 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55f64b8ec100 msgr2=0x55f64b8ef190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55f64b8ec100 0x55f64b8ef190 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55f64b9012d0 msgr2=0x55f64b9036c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55f64b9012d0 0x55f64b9036c0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f64b8f51c0 msgr2=0x55f64b8f75b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 reap_dead start 2023-11-15T03:07:43.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f64b8f51c0 0x55f64b8f75b0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1eff343700 1 -- 172.21.15.121:0/42818720 reap_dead start 2023-11-15T03:07:43.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55f64b8fd7e0 msgr2=0x55f64b8fdbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55f64b8fd7e0 0x55f64b8fdbf0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1ee4059020 msgr2=0x7f1ee405b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1ee4059020 0x7f1ee405b4e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f1ef00010e0 tx=0x7f1ef00018f0).stop 2023-11-15T03:07:43.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f64b8ce940 msgr2=0x55f64b8291b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f64b8ce940 0x55f64b8291b0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 rx=0x55f64b844c90 tx=0x7f1ef8009000).stop 2023-11-15T03:07:43.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 shutdown_connections 2023-11-15T03:07:43.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55f64b8fd7e0 0x55f64b8fdbf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f64b8f51c0 0x55f64b8f75b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1ee4059020 0x7f1ee405b4e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 --2- 172.21.15.121:0/42818720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f64b8ce940 0x55f64b8291b0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 >> 172.21.15.121:0/42818720 conn(0x55f64b844080 msgr2=0x55f64b83a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:43.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 shutdown_connections 2023-11-15T03:07:43.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.648+0000 7f1f0a530d00 1 -- 172.21.15.121:0/42818720 wait complete. 2023-11-15T03:07:43.663 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 1.00000000.inode 0 2023-11-15T03:07:43.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.716+0000 7fc2b72dad00 1 Processor -- start 2023-11-15T03:07:43.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.716+0000 7fc2b72dad00 1 -- start start 2023-11-15T03:07:43.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2b72dad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606acebddd0 0x5606acf8fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2b72dad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5606acf07b00 0x5606acf02ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2b72dad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5606acf96670 0x5606acf034e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2b72dad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ace36220 con 0x5606acf07b00 2023-11-15T03:07:43.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2b72dad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606acf068b0 con 0x5606acebddd0 2023-11-15T03:07:43.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2b72dad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606acf039d0 con 0x5606acf96670 2023-11-15T03:07:43.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2ac398700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5606acf07b00 0x5606acf02ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2ac398700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5606acf07b00 0x5606acf02ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44188/0 (socket says 172.21.15.121:44188) 2023-11-15T03:07:43.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2ac398700 1 -- 172.21.15.121:0/729944672 learned_addr learned my addr 172.21.15.121:0/729944672 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:43.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2b4b99700 1 --2- 172.21.15.121:0/729944672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606acebddd0 0x5606acf8fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2b539a700 1 --2- 172.21.15.121:0/729944672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5606acf96670 0x5606acf034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2ac398700 1 -- 172.21.15.121:0/729944672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5606acf96670 msgr2=0x5606acf034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:43.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2ac398700 1 --2- 172.21.15.121:0/729944672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5606acf96670 0x5606acf034e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2ac398700 1 -- 172.21.15.121:0/729944672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606acebddd0 msgr2=0x5606acf8fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:43.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2ac398700 1 --2- 172.21.15.121:0/729944672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606acebddd0 0x5606acf8fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2ac398700 1 -- 172.21.15.121:0/729944672 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a400a0e0 con 0x5606acf07b00 2023-11-15T03:07:43.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2ac398700 1 --2- 172.21.15.121:0/729944672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5606acf07b00 0x5606acf02ff0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 rx=0x7fc29c006e10 tx=0x7fc29c0068c0).ready entity=mon.0 client_cookie=a39623191d5ea537 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:43.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2af7fe700 1 -- 172.21.15.121:0/729944672 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c00bcc0 con 0x5606acf07b00 2023-11-15T03:07:43.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2af7fe700 1 -- 172.21.15.121:0/729944672 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29c00be20 con 0x5606acf07b00 2023-11-15T03:07:43.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2af7fe700 1 -- 172.21.15.121:0/729944672 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c00e6c0 con 0x5606acf07b00 2023-11-15T03:07:43.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2b72dad00 1 -- 172.21.15.121:0/729944672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5606acf07b00 msgr2=0x5606acf02ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2b72dad00 1 --2- 172.21.15.121:0/729944672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5606acf07b00 0x5606acf02ff0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 rx=0x7fc29c006e10 tx=0x7fc29c0068c0).stop 2023-11-15T03:07:43.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2b72dad00 1 -- 172.21.15.121:0/729944672 shutdown_connections 2023-11-15T03:07:43.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2b72dad00 1 --2- 172.21.15.121:0/729944672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5606acf96670 0x5606acf034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2b72dad00 1 --2- 172.21.15.121:0/729944672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5606acf07b00 0x5606acf02ff0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2b72dad00 1 --2- 172.21.15.121:0/729944672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606acebddd0 0x5606acf8fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2b72dad00 1 -- 172.21.15.121:0/729944672 >> 172.21.15.121:0/729944672 conn(0x5606acf0d080 msgr2=0x5606acf0b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:43.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2b72dad00 1 -- 172.21.15.121:0/729944672 shutdown_connections 2023-11-15T03:07:43.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.720+0000 7fc2b72dad00 1 -- 172.21.15.121:0/729944672 wait complete. 2023-11-15T03:07:43.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.724+0000 7fc2b72dad00 1 Processor -- start 2023-11-15T03:07:43.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.724+0000 7fc2b72dad00 1 -- start start 2023-11-15T03:07:43.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.724+0000 7fc2b72dad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606acebddd0 0x5606acf8e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.724+0000 7fc2b72dad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5606acf07b00 0x5606acf981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.724+0000 7fc2b72dad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5606acf96670 0x5606acf986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.724+0000 7fc2b72dad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606aceecbc0 con 0x5606acf07b00 2023-11-15T03:07:43.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.724+0000 7fc2b72dad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606aceed740 con 0x5606acebddd0 2023-11-15T03:07:43.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.724+0000 7fc2b72dad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606aceece10 con 0x5606acf96670 2023-11-15T03:07:43.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.724+0000 7fc2ac398700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5606acf07b00 0x5606acf981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.724+0000 7fc2ac398700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5606acf07b00 0x5606acf981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44202/0 (socket says 172.21.15.121:44202) 2023-11-15T03:07:43.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.724+0000 7fc2ac398700 1 -- 172.21.15.121:0/2776870828 learned_addr learned my addr 172.21.15.121:0/2776870828 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:43.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.728+0000 7fc2b4b99700 1 --2- 172.21.15.121:0/2776870828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606acebddd0 0x5606acf8e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.728+0000 7fc2b539a700 1 --2- 172.21.15.121:0/2776870828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5606acf96670 0x5606acf986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.728+0000 7fc2ac398700 1 -- 172.21.15.121:0/2776870828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5606acf96670 msgr2=0x5606acf986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.728+0000 7fc2ac398700 1 --2- 172.21.15.121:0/2776870828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5606acf96670 0x5606acf986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.728+0000 7fc2ac398700 1 -- 172.21.15.121:0/2776870828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606acebddd0 msgr2=0x5606acf8e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.728+0000 7fc2ac398700 1 --2- 172.21.15.121:0/2776870828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606acebddd0 0x5606acf8e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.728+0000 7fc2ac398700 1 -- 172.21.15.121:0/2776870828 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606ace647f0 con 0x5606acf07b00 2023-11-15T03:07:43.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.728+0000 7fc2ac398700 1 --2- 172.21.15.121:0/2776870828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5606acf07b00 0x5606acf981f0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 rx=0x5606aceefe00 tx=0x7fc29c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:43.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.728+0000 7fc2adffb700 1 -- 172.21.15.121:0/2776870828 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c01a420 con 0x5606acf07b00 2023-11-15T03:07:43.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.728+0000 7fc2adffb700 1 -- 172.21.15.121:0/2776870828 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29c01a580 con 0x5606acf07b00 2023-11-15T03:07:43.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.728+0000 7fc2adffb700 1 -- 172.21.15.121:0/2776870828 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c01a940 con 0x5606acf07b00 2023-11-15T03:07:43.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.728+0000 7fc2b72dad00 1 -- 172.21.15.121:0/2776870828 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5606ace5f770 con 0x5606acf07b00 2023-11-15T03:07:43.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.728+0000 7fc2b72dad00 1 -- 172.21.15.121:0/2776870828 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606ace2ea70 con 0x5606acf07b00 2023-11-15T03:07:43.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.732+0000 7fc2adffb700 1 -- 172.21.15.121:0/2776870828 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc29c01acb0 con 0x5606acf07b00 2023-11-15T03:07:43.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.732+0000 7fc2adffb700 1 --2- 172.21.15.121:0/2776870828 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc294059020 0x7fc29405b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.732+0000 7fc2adffb700 1 -- 172.21.15.121:0/2776870828 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8244+0+0 (secure 0 0 0) 0x7fc29c075bc0 con 0x5606acf07b00 2023-11-15T03:07:43.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.732+0000 7fc2b4b99700 1 --2- 172.21.15.121:0/2776870828 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc294059020 0x7fc29405b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.732+0000 7fc2b72dad00 1 --2- 172.21.15.121:0/2776870828 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5606aceef880 0x5606acfaaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.732+0000 7fc2b72dad00 1 -- 172.21.15.121:0/2776870828 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x5606acf8f480 con 0x5606aceef880 2023-11-15T03:07:43.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.732+0000 7fc2b4b99700 1 --2- 172.21.15.121:0/2776870828 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc294059020 0x7fc29405b4e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7fc2a400a470 tx=0x7fc2a4015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:43.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.732+0000 7fc2b539a700 1 --2- 172.21.15.121:0/2776870828 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5606aceef880 0x5606acfaaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.732+0000 7fc2b539a700 1 --2- 172.21.15.121:0/2776870828 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5606aceef880 0x5606acfaaff0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:43.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.736+0000 7fc2b539a700 1 -- 172.21.15.121:0/2776870828 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(1 1.00000000.inode [truncate 0] v117'241 uv241 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fc2b0007ab0 con 0x5606aceef880 2023-11-15T03:07:43.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.736+0000 7fc2b72dad00 1 -- 172.21.15.121:0/2776870828 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5606aceef880 msgr2=0x5606acfaaff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.736+0000 7fc2b72dad00 1 --2- 172.21.15.121:0/2776870828 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5606aceef880 0x5606acfaaff0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.736+0000 7fc2b72dad00 1 -- 172.21.15.121:0/2776870828 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc294059020 msgr2=0x7fc29405b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.736+0000 7fc2b72dad00 1 --2- 172.21.15.121:0/2776870828 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc294059020 0x7fc29405b4e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7fc2a400a470 tx=0x7fc2a4015000).stop 2023-11-15T03:07:43.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.736+0000 7fc2b72dad00 1 -- 172.21.15.121:0/2776870828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5606acf07b00 msgr2=0x5606acf981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.736+0000 7fc2b72dad00 1 --2- 172.21.15.121:0/2776870828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5606acf07b00 0x5606acf981f0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 rx=0x5606aceefe00 tx=0x7fc29c015000).stop 2023-11-15T03:07:43.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.736+0000 7fc2b539a700 1 -- 172.21.15.121:0/2776870828 reap_dead start 2023-11-15T03:07:43.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.736+0000 7fc2b72dad00 1 -- 172.21.15.121:0/2776870828 shutdown_connections 2023-11-15T03:07:43.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.736+0000 7fc2b72dad00 1 -- 172.21.15.121:0/2776870828 >> 172.21.15.121:0/2776870828 conn(0x5606acf0d080 msgr2=0x5606acf977e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:43.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.736+0000 7fc2b72dad00 1 -- 172.21.15.121:0/2776870828 shutdown_connections 2023-11-15T03:07:43.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.736+0000 7fc2b72dad00 1 -- 172.21.15.121:0/2776870828 wait complete. 2023-11-15T03:07:43.752 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:07:43.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.928+0000 7f25b229b700 1 Processor -- start 2023-11-15T03:07:43.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.928+0000 7f25b229b700 1 -- start start 2023-11-15T03:07:43.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.932+0000 7f25b229b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f25ac067ec0 0x7f25ac0682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.932+0000 7f25b229b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f25ac0c10b0 0x7f25ac062c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.932+0000 7f25b229b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25ac0c0c20 con 0x7f25ac063120 2023-11-15T03:07:43.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.932+0000 7f25b229b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25ac063590 con 0x7f25ac067ec0 2023-11-15T03:07:43.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.932+0000 7f25b229b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25ac158730 con 0x7f25ac0c10b0 2023-11-15T03:07:43.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.932+0000 7f25abfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f25ac067ec0 0x7f25ac0682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.932+0000 7f25abfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f25ac067ec0 0x7f25ac0682a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44218/0 (socket says 172.21.15.121:44218) 2023-11-15T03:07:43.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.932+0000 7f25abfff700 1 -- 172.21.15.121:0/3307971234 learned_addr learned my addr 172.21.15.121:0/3307971234 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:43.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.932+0000 7f25ab7fe700 1 --2- 172.21.15.121:0/3307971234 >> v2:172.21.15.190:3300/0 conn(0x7f25ac0c10b0 0x7f25ac062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.932+0000 7f25ab7fe700 1 -- 172.21.15.121:0/3307971234 >> v2:172.21.15.121:3300/0 conn(0x7f25ac067ec0 msgr2=0x7f25ac0682a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:43.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.932+0000 7f25ab7fe700 1 --2- 172.21.15.121:0/3307971234 >> v2:172.21.15.121:3300/0 conn(0x7f25ac067ec0 0x7f25ac0682a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.932+0000 7f25ab7fe700 1 -- 172.21.15.121:0/3307971234 >> v1:172.21.15.190:6789/0 conn(0x7f25ac063120 legacy=0x7f25ac153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.932+0000 7f25ab7fe700 1 -- 172.21.15.121:0/3307971234 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac0c02f0 con 0x7f25ac0c10b0 2023-11-15T03:07:43.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.932+0000 7f25ab7fe700 1 --2- 172.21.15.121:0/3307971234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25ac0c10b0 0x7f25ac062c30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f25ac068a10 tx=0x7f25a0009060).ready entity=mon.1 client_cookie=796759887ab27fbb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:43.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.932+0000 7f25aaffd700 1 -- 172.21.15.121:0/3307971234 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a000f510 con 0x7f25ac0c10b0 2023-11-15T03:07:43.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.936+0000 7f25aaffd700 1 -- 172.21.15.121:0/3307971234 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a000f670 con 0x7f25ac0c10b0 2023-11-15T03:07:43.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.936+0000 7f25aaffd700 1 -- 172.21.15.121:0/3307971234 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a000f9f0 con 0x7f25ac0c10b0 2023-11-15T03:07:43.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.936+0000 7f25b229b700 1 -- 172.21.15.121:0/3307971234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25ac0c10b0 msgr2=0x7f25ac062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.936+0000 7f25b229b700 1 --2- 172.21.15.121:0/3307971234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25ac0c10b0 0x7f25ac062c30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f25ac068a10 tx=0x7f25a0009060).stop 2023-11-15T03:07:43.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.936+0000 7f25b229b700 1 -- 172.21.15.121:0/3307971234 shutdown_connections 2023-11-15T03:07:43.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.936+0000 7f25b229b700 1 --2- 172.21.15.121:0/3307971234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25ac0c10b0 0x7f25ac062c30 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.936+0000 7f25b229b700 1 --2- 172.21.15.121:0/3307971234 >> v2:172.21.15.121:3300/0 conn(0x7f25ac067ec0 0x7f25ac0682a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.936+0000 7f25b229b700 1 -- 172.21.15.121:0/3307971234 >> 172.21.15.121:0/3307971234 conn(0x7f25ac0cad40 msgr2=0x7f25ac0cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:43.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.936+0000 7f25b229b700 1 -- 172.21.15.121:0/3307971234 shutdown_connections 2023-11-15T03:07:43.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.936+0000 7f25b229b700 1 -- 172.21.15.121:0/3307971234 wait complete. 2023-11-15T03:07:43.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.936+0000 7f25b229b700 1 Processor -- start 2023-11-15T03:07:43.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.936+0000 7f25b229b700 1 -- start start 2023-11-15T03:07:43.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25b229b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25ac063120 0x7f25ac0ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25b229b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25ac067ec0 0x7f25ac1679d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25b229b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25ac0c10b0 0x7f25ac167ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25b229b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac13f160 con 0x7f25ac067ec0 2023-11-15T03:07:43.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25b229b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac1683b0 con 0x7f25ac0c10b0 2023-11-15T03:07:43.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25b229b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac1655d0 con 0x7f25ac063120 2023-11-15T03:07:43.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25ab7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25ac067ec0 0x7f25ac1679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25ab7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25ac067ec0 0x7f25ac1679d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44228/0 (socket says 172.21.15.121:44228) 2023-11-15T03:07:43.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25ab7fe700 1 -- 172.21.15.121:0/1743017320 learned_addr learned my addr 172.21.15.121:0/1743017320 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:43.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25abfff700 1 --2- 172.21.15.121:0/1743017320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25ac063120 0x7f25ac0ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25b0838700 1 --2- 172.21.15.121:0/1743017320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25ac0c10b0 0x7f25ac167ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25ab7fe700 1 -- 172.21.15.121:0/1743017320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25ac063120 msgr2=0x7f25ac0ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25ab7fe700 1 --2- 172.21.15.121:0/1743017320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25ac063120 0x7f25ac0ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25ab7fe700 1 -- 172.21.15.121:0/1743017320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25ac0c10b0 msgr2=0x7f25ac167ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:43.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25ab7fe700 1 --2- 172.21.15.121:0/1743017320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25ac0c10b0 0x7f25ac167ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:43.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25ab7fe700 1 -- 172.21.15.121:0/1743017320 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a00092a0 con 0x7f25ac067ec0 2023-11-15T03:07:43.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25abfff700 1 --2- 172.21.15.121:0/1743017320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25ac063120 0x7f25ac0ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:07:43.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25ab7fe700 1 --2- 172.21.15.121:0/1743017320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25ac067ec0 0x7f25ac1679d0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 rx=0x7f25a0009dd0 tx=0x7f25a0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:43.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f2593fff700 1 -- 172.21.15.121:0/1743017320 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a000f7e0 con 0x7f25ac067ec0 2023-11-15T03:07:43.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f2593fff700 1 -- 172.21.15.121:0/1743017320 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a000f940 con 0x7f25ac067ec0 2023-11-15T03:07:43.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f2593fff700 1 -- 172.21.15.121:0/1743017320 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a001b660 con 0x7f25ac067ec0 2023-11-15T03:07:43.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25b229b700 1 -- 172.21.15.121:0/1743017320 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ac166150 con 0x7f25ac067ec0 2023-11-15T03:07:43.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.940+0000 7f25b229b700 1 -- 172.21.15.121:0/1743017320 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ac165b00 con 0x7f25ac067ec0 2023-11-15T03:07:43.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.944+0000 7f25b229b700 1 -- 172.21.15.121:0/1743017320 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ac047060 con 0x7f25ac067ec0 2023-11-15T03:07:43.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.944+0000 7f2593fff700 1 -- 172.21.15.121:0/1743017320 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f25a001ba10 con 0x7f25ac067ec0 2023-11-15T03:07:43.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.948+0000 7f2593fff700 1 --2- 172.21.15.121:0/1743017320 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f259805a510 0x7f259805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:43.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.948+0000 7f2593fff700 1 -- 172.21.15.121:0/1743017320 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8244+0+0 (secure 0 0 0) 0x7f25a0076930 con 0x7f25ac067ec0 2023-11-15T03:07:43.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.948+0000 7f25abfff700 1 --2- 172.21.15.121:0/1743017320 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f259805a510 0x7f259805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:43.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.948+0000 7f25abfff700 1 --2- 172.21.15.121:0/1743017320 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f259805a510 0x7f259805c9d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f259c006bc0 tx=0x7f259c0090f0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:43.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:43.952+0000 7f2593fff700 1 -- 172.21.15.121:0/1743017320 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f25a0049910 con 0x7f25ac067ec0 2023-11-15T03:07:44.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.148+0000 7f25b229b700 1 -- 172.21.15.121:0/1743017320 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f25ac0c05d0 con 0x7f25ac067ec0 2023-11-15T03:07:44.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.180+0000 7f2593fff700 1 -- 172.21.15.121:0/1743017320 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v226) v1 ==== 162+0+0 (secure 0 0 0) 0x7f25a001f070 con 0x7f25ac067ec0 2023-11-15T03:07:44.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.184+0000 7f25b229b700 1 -- 172.21.15.121:0/1743017320 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f259805a510 msgr2=0x7f259805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:44.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.184+0000 7f25b229b700 1 --2- 172.21.15.121:0/1743017320 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f259805a510 0x7f259805c9d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f259c006bc0 tx=0x7f259c0090f0).stop 2023-11-15T03:07:44.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.184+0000 7f25b229b700 1 -- 172.21.15.121:0/1743017320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25ac067ec0 msgr2=0x7f25ac1679d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:44.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.184+0000 7f25b229b700 1 --2- 172.21.15.121:0/1743017320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25ac067ec0 0x7f25ac1679d0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 rx=0x7f25a0009dd0 tx=0x7f25a0017000).stop 2023-11-15T03:07:44.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.184+0000 7f25b229b700 1 -- 172.21.15.121:0/1743017320 shutdown_connections 2023-11-15T03:07:44.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.184+0000 7f25b229b700 1 --2- 172.21.15.121:0/1743017320 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f259805a510 0x7f259805c9d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:44.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.184+0000 7f25b229b700 1 --2- 172.21.15.121:0/1743017320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25ac0c10b0 0x7f25ac167ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:44.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.184+0000 7f25b229b700 1 --2- 172.21.15.121:0/1743017320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25ac067ec0 0x7f25ac1679d0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:44.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.184+0000 7f25b229b700 1 --2- 172.21.15.121:0/1743017320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25ac063120 0x7f25ac0ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:44.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.184+0000 7f25b229b700 1 -- 172.21.15.121:0/1743017320 >> 172.21.15.121:0/1743017320 conn(0x7f25ac0cad40 msgr2=0x7f25ac0adb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:44.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.184+0000 7f25b229b700 1 -- 172.21.15.121:0/1743017320 shutdown_connections 2023-11-15T03:07:44.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.184+0000 7f25b229b700 1 -- 172.21.15.121:0/1743017320 wait complete. 2023-11-15T03:07:44.198 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:07:44.202 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:07:44.196+0000 7f4989492700 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2023-11-15T03:07:44.202 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:07:44.196+0000 7f4989492700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2023-11-15T03:07:44.215 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:44.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.364+0000 7fe83041b700 1 Processor -- start 2023-11-15T03:07:44.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.368+0000 7fe83041b700 1 -- start start 2023-11-15T03:07:44.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe83041b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe828157f70 0x7fe82815ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:44.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe83041b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8280c3320 con 0x7fe828153da0 2023-11-15T03:07:44.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe83041b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8280c2710 con 0x7fe8280c5e10 2023-11-15T03:07:44.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe83041b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe828155190 con 0x7fe828157f70 2023-11-15T03:07:44.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82e1b7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe828153da0 0x7fe8280c1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50626/0 (socket says 172.21.15.121:50626) 2023-11-15T03:07:44.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82e1b7700 1 -- 172.21.15.121:0/3584232454 learned_addr learned my addr 172.21.15.121:0/3584232454 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:44.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82e9b8700 1 --2- 172.21.15.121:0/3584232454 >> v2:172.21.15.190:3300/0 conn(0x7fe828157f70 0x7fe82815ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:44.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82d1b5700 1 -- 172.21.15.121:0/3584232454 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1770652420 0 0) 0x7fe8280c3320 con 0x7fe828153da0 2023-11-15T03:07:44.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82d1b5700 1 -- 172.21.15.121:0/3584232454 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8140033f0 con 0x7fe828153da0 2023-11-15T03:07:44.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82d1b5700 1 -- 172.21.15.121:0/3584232454 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175271371 0 0) 0x7fe8280c2710 con 0x7fe8280c5e10 2023-11-15T03:07:44.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82d1b5700 1 -- 172.21.15.121:0/3584232454 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8280c3320 con 0x7fe8280c5e10 2023-11-15T03:07:44.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82d1b5700 1 -- 172.21.15.121:0/3584232454 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4020008629 0 0) 0x7fe8140033f0 con 0x7fe828153da0 2023-11-15T03:07:44.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82d1b5700 1 -- 172.21.15.121:0/3584232454 >> v2:172.21.15.190:3300/0 conn(0x7fe828157f70 msgr2=0x7fe82815ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:44.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82d1b5700 1 --2- 172.21.15.121:0/3584232454 >> v2:172.21.15.190:3300/0 conn(0x7fe828157f70 0x7fe82815ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:44.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82d1b5700 1 -- 172.21.15.121:0/3584232454 >> v1:172.21.15.190:6789/0 conn(0x7fe8280c5e10 legacy=0x7fe828157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:44.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82d1b5700 1 -- 172.21.15.121:0/3584232454 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8280c29f0 con 0x7fe828153da0 2023-11-15T03:07:44.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82d1b5700 1 -- 172.21.15.121:0/3584232454 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fe824001470 con 0x7fe828153da0 2023-11-15T03:07:44.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82d1b5700 1 -- 172.21.15.121:0/3584232454 >> v1:172.21.15.121:6789/0 conn(0x7fe828153da0 legacy=0x7fe8280c1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:44.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82d1b5700 1 --2- 172.21.15.121:0/3584232454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814004ad0 0x7fe814004ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:44.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82d1b5700 1 --2- 172.21.15.121:0/3584232454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe814003630 0x7fe814003ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:44.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82d1b5700 1 --2- 172.21.15.121:0/3584232454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe814003fa0 0x7fe81400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:44.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82d1b5700 1 -- 172.21.15.121:0/3584232454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe828155190 con 0x7fe814004ad0 2023-11-15T03:07:44.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82d1b5700 1 -- 172.21.15.121:0/3584232454 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81400be80 con 0x7fe814003630 2023-11-15T03:07:44.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82d1b5700 1 -- 172.21.15.121:0/3584232454 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81400c040 con 0x7fe814003fa0 2023-11-15T03:07:44.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82d9b6700 1 --2- 172.21.15.121:0/3584232454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe814003630 0x7fe814003ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:44.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82e1b7700 1 --2- 172.21.15.121:0/3584232454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814004ad0 0x7fe814004ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:44.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82e9b8700 1 --2- 172.21.15.121:0/3584232454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe814003fa0 0x7fe81400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:44.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82e1b7700 1 -- 172.21.15.121:0/3584232454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe814003fa0 msgr2=0x7fe81400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:44.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82e1b7700 1 --2- 172.21.15.121:0/3584232454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe814003fa0 0x7fe81400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:44.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82e1b7700 1 -- 172.21.15.121:0/3584232454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe814003630 msgr2=0x7fe814003ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:44.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82e1b7700 1 --2- 172.21.15.121:0/3584232454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe814003630 0x7fe814003ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:44.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.372+0000 7fe82e1b7700 1 -- 172.21.15.121:0/3584232454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8140033f0 con 0x7fe814004ad0 2023-11-15T03:07:44.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.376+0000 7fe82e9b8700 1 -- 172.21.15.121:0/3584232454 reap_dead start 2023-11-15T03:07:44.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.376+0000 7fe82e1b7700 1 --2- 172.21.15.121:0/3584232454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814004ad0 0x7fe814004ed0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 rx=0x7fe82400c0e0 tx=0x7fe824001df0).ready entity=mon.0 client_cookie=f4987795b9482064 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:44.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.376+0000 7fe82d1b5700 1 -- 172.21.15.121:0/3584232454 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82400ca10 con 0x7fe814004ad0 2023-11-15T03:07:44.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.376+0000 7fe82d1b5700 1 -- 172.21.15.121:0/3584232454 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82400cb70 con 0x7fe814004ad0 2023-11-15T03:07:44.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.376+0000 7fe82d1b5700 1 -- 172.21.15.121:0/3584232454 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8240070d0 con 0x7fe814004ad0 2023-11-15T03:07:44.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.376+0000 7fe83041b700 1 -- 172.21.15.121:0/3584232454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814004ad0 msgr2=0x7fe814004ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:44.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.376+0000 7fe83041b700 1 --2- 172.21.15.121:0/3584232454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814004ad0 0x7fe814004ed0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 rx=0x7fe82400c0e0 tx=0x7fe824001df0).stop 2023-11-15T03:07:44.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.376+0000 7fe83041b700 1 -- 172.21.15.121:0/3584232454 shutdown_connections 2023-11-15T03:07:44.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.376+0000 7fe83041b700 1 --2- 172.21.15.121:0/3584232454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814004ad0 0x7fe814004ed0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:44.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.376+0000 7fe83041b700 1 -- 172.21.15.121:0/3584232454 >> 172.21.15.121:0/3584232454 conn(0x7fe8280c7ac0 msgr2=0x7fe8280c3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:44.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.376+0000 7fe83041b700 1 -- 172.21.15.121:0/3584232454 shutdown_connections 2023-11-15T03:07:44.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.376+0000 7fe83041b700 1 -- 172.21.15.121:0/3584232454 wait complete. 2023-11-15T03:07:44.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.380+0000 7fe83041b700 1 Processor -- start 2023-11-15T03:07:44.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.380+0000 7fe83041b700 1 -- start start 2023-11-15T03:07:44.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.380+0000 7fe83041b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814004ad0 0x7fe82813d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:44.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.380+0000 7fe83041b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8281656e0 0x7fe8280a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:44.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.380+0000 7fe83041b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe828165b60 0x7fe8280aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:44.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.380+0000 7fe83041b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8280aa980 con 0x7fe814004ad0 2023-11-15T03:07:44.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.380+0000 7fe83041b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82813d690 con 0x7fe8281656e0 2023-11-15T03:07:44.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.380+0000 7fe83041b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8280ad0a0 con 0x7fe828165b60 2023-11-15T03:07:44.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.380+0000 7fe82e1b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814004ad0 0x7fe82813d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:44.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.380+0000 7fe82e1b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814004ad0 0x7fe82813d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44250/0 (socket says 172.21.15.121:44250) 2023-11-15T03:07:44.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.380+0000 7fe82e1b7700 1 -- 172.21.15.121:0/4064080123 learned_addr learned my addr 172.21.15.121:0/4064080123 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:44.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.380+0000 7fe82d9b6700 1 --2- 172.21.15.121:0/4064080123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8281656e0 0x7fe8280a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:44.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.380+0000 7fe82e9b8700 1 --2- 172.21.15.121:0/4064080123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe828165b60 0x7fe8280aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:44.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.380+0000 7fe82e1b7700 1 -- 172.21.15.121:0/4064080123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe828165b60 msgr2=0x7fe8280aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:44.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.380+0000 7fe82e1b7700 1 --2- 172.21.15.121:0/4064080123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe828165b60 0x7fe8280aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:44.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.380+0000 7fe82e1b7700 1 -- 172.21.15.121:0/4064080123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8281656e0 msgr2=0x7fe8280a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:44.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.380+0000 7fe82e1b7700 1 --2- 172.21.15.121:0/4064080123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8281656e0 0x7fe8280a9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:44.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.380+0000 7fe82e1b7700 1 -- 172.21.15.121:0/4064080123 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8280a52f0 con 0x7fe814004ad0 2023-11-15T03:07:44.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.384+0000 7fe82e1b7700 1 --2- 172.21.15.121:0/4064080123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814004ad0 0x7fe82813d1a0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 rx=0x7fe824001ec0 tx=0x7fe824015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:44.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.384+0000 7fe81e7fc700 1 -- 172.21.15.121:0/4064080123 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe824001640 con 0x7fe814004ad0 2023-11-15T03:07:44.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.384+0000 7fe81e7fc700 1 -- 172.21.15.121:0/4064080123 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8240046b0 con 0x7fe814004ad0 2023-11-15T03:07:44.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.384+0000 7fe83041b700 1 -- 172.21.15.121:0/4064080123 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8280a49c0 con 0x7fe814004ad0 2023-11-15T03:07:44.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.384+0000 7fe81e7fc700 1 -- 172.21.15.121:0/4064080123 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe824004a50 con 0x7fe814004ad0 2023-11-15T03:07:44.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.384+0000 7fe83041b700 1 -- 172.21.15.121:0/4064080123 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8280a4f80 con 0x7fe814004ad0 2023-11-15T03:07:44.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.384+0000 7fe83041b700 1 -- 172.21.15.121:0/4064080123 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe82806c150 con 0x7fe814004ad0 2023-11-15T03:07:44.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.384+0000 7fe81e7fc700 1 -- 172.21.15.121:0/4064080123 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe824013450 con 0x7fe814004ad0 2023-11-15T03:07:44.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.388+0000 7fe81e7fc700 1 --2- 172.21.15.121:0/4064080123 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe80c05a510 0x7fe80c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:44.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.388+0000 7fe81e7fc700 1 -- 172.21.15.121:0/4064080123 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8330+0+0 (secure 0 0 0) 0x7fe824075a50 con 0x7fe814004ad0 2023-11-15T03:07:44.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.388+0000 7fe82d9b6700 1 --2- 172.21.15.121:0/4064080123 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe80c05a510 0x7fe80c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:44.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.388+0000 7fe82d9b6700 1 --2- 172.21.15.121:0/4064080123 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe80c05a510 0x7fe80c05c9d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7fe8180010e0 tx=0x7fe818009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:44.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.392+0000 7fe81e7fc700 1 -- 172.21.15.121:0/4064080123 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe824048970 con 0x7fe814004ad0 2023-11-15T03:07:44.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.592+0000 7fe83041b700 1 -- 172.21.15.121:0/4064080123 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8280c2cd0 con 0x7fe814004ad0 2023-11-15T03:07:44.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.596+0000 7fe81e7fc700 1 -- 172.21.15.121:0/4064080123 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 227 v227) v1 ==== 95+0+4505 (secure 0 0 0) 0x7fe824025020 con 0x7fe814004ad0 2023-11-15T03:07:44.603 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:44.603 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":227,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":227,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:44.184904+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9198":{"gid":9198,"name":"a","rank":0,"incarnation":227,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/2544923351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2544923351},{"type":"v1","addr":"172.21.15.121:6836","nonce":2544923351}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:44.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.596+0000 7fe83041b700 1 -- 172.21.15.121:0/4064080123 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe80c05a510 msgr2=0x7fe80c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:44.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.596+0000 7fe83041b700 1 --2- 172.21.15.121:0/4064080123 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe80c05a510 0x7fe80c05c9d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7fe8180010e0 tx=0x7fe818009040).stop 2023-11-15T03:07:44.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.596+0000 7fe83041b700 1 -- 172.21.15.121:0/4064080123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814004ad0 msgr2=0x7fe82813d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:44.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.596+0000 7fe83041b700 1 --2- 172.21.15.121:0/4064080123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814004ad0 0x7fe82813d1a0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 rx=0x7fe824001ec0 tx=0x7fe824015000).stop 2023-11-15T03:07:44.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.596+0000 7fe83041b700 1 -- 172.21.15.121:0/4064080123 shutdown_connections 2023-11-15T03:07:44.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.596+0000 7fe83041b700 1 --2- 172.21.15.121:0/4064080123 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe80c05a510 0x7fe80c05c9d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:44.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.596+0000 7fe83041b700 1 --2- 172.21.15.121:0/4064080123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe828165b60 0x7fe8280aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:44.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.596+0000 7fe83041b700 1 --2- 172.21.15.121:0/4064080123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8281656e0 0x7fe8280a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:44.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.596+0000 7fe83041b700 1 --2- 172.21.15.121:0/4064080123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814004ad0 0x7fe82813d1a0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:44.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.596+0000 7fe83041b700 1 -- 172.21.15.121:0/4064080123 >> 172.21.15.121:0/4064080123 conn(0x7fe8280c7ac0 msgr2=0x7fe828067ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:44.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.596+0000 7fe83041b700 1 -- 172.21.15.121:0/4064080123 shutdown_connections 2023-11-15T03:07:44.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:44.596+0000 7fe83041b700 1 -- 172.21.15.121:0/4064080123 wait complete. 2023-11-15T03:07:44.610 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 227 2023-11-15T03:07:44.626 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:07:45.197 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:07:45.188+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:07:45.198 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:07:45.188+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:07:45.206 INFO:tasks.ceph.mds.a.smithi121.stderr: -17> 2023-11-15T03:07:44.196+0000 7f4989492700 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2023-11-15T03:07:45.206 INFO:tasks.ceph.mds.a.smithi121.stderr: -15> 2023-11-15T03:07:44.196+0000 7f4989492700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2023-11-15T03:07:45.244 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:07:49.628 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:49.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.788+0000 7f9efcfb9700 1 Processor -- start 2023-11-15T03:07:49.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.788+0000 7f9efcfb9700 1 -- start start 2023-11-15T03:07:49.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.788+0000 7f9efcfb9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9ef8151870 0x7f9ef8151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:49.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.788+0000 7f9efcfb9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9ef80c30e0 0x7f9ef80c3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:49.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.788+0000 7f9efcfb9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef80657f0 con 0x7f9ef8152140 2023-11-15T03:07:49.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.788+0000 7f9efcfb9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef8065ad0 con 0x7f9ef80c30e0 2023-11-15T03:07:49.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.788+0000 7f9efcfb9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef80c3a50 con 0x7f9ef8151870 2023-11-15T03:07:49.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.788+0000 7f9eeeffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9ef8152140 0x7f9ef80a9830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60376/0 (socket says 172.21.15.121:60376) 2023-11-15T03:07:49.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.788+0000 7f9eeeffd700 1 -- 172.21.15.121:0/766844633 learned_addr learned my addr 172.21.15.121:0/766844633 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:49.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.788+0000 7f9ef7fff700 1 --2- 172.21.15.121:0/766844633 >> v2:172.21.15.121:3300/0 conn(0x7f9ef80c30e0 0x7f9ef80c3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:49.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.788+0000 7f9ef77fe700 1 --2- 172.21.15.121:0/766844633 >> v2:172.21.15.190:3300/0 conn(0x7f9ef8151870 0x7f9ef8151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:49.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.792+0000 7f9ef6ffd700 1 -- 172.21.15.121:0/766844633 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3209380063 0 0) 0x7f9ef80657f0 con 0x7f9ef8152140 2023-11-15T03:07:49.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.792+0000 7f9ef6ffd700 1 -- 172.21.15.121:0/766844633 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ee40033f0 con 0x7f9ef8152140 2023-11-15T03:07:49.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.792+0000 7f9ef7fff700 1 -- 172.21.15.121:0/766844633 >> v2:172.21.15.190:3300/0 conn(0x7f9ef8151870 msgr2=0x7f9ef8151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:49.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.792+0000 7f9ef7fff700 1 --2- 172.21.15.121:0/766844633 >> v2:172.21.15.190:3300/0 conn(0x7f9ef8151870 0x7f9ef8151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:49.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.792+0000 7f9ef7fff700 1 -- 172.21.15.121:0/766844633 >> v1:172.21.15.121:6789/0 conn(0x7f9ef8152140 legacy=0x7f9ef80a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:49.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.792+0000 7f9ef7fff700 1 -- 172.21.15.121:0/766844633 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef80aa260 con 0x7f9ef80c30e0 2023-11-15T03:07:49.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.792+0000 7f9ef7fff700 1 --2- 172.21.15.121:0/766844633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ef80c30e0 0x7f9ef80c3560 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 rx=0x7f9ef8067720 tx=0x7f9ef0009780).ready entity=mon.0 client_cookie=f140aec35624589d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:49.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.792+0000 7f9ef6ffd700 1 -- 172.21.15.121:0/766844633 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef000e070 con 0x7f9ef80c30e0 2023-11-15T03:07:49.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.792+0000 7f9ef6ffd700 1 -- 172.21.15.121:0/766844633 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef000bb60 con 0x7f9ef80c30e0 2023-11-15T03:07:49.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.792+0000 7f9ef6ffd700 1 -- 172.21.15.121:0/766844633 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0012410 con 0x7f9ef80c30e0 2023-11-15T03:07:49.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.792+0000 7f9efcfb9700 1 -- 172.21.15.121:0/766844633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ef80c30e0 msgr2=0x7f9ef80c3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:49.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.792+0000 7f9efcfb9700 1 --2- 172.21.15.121:0/766844633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ef80c30e0 0x7f9ef80c3560 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 rx=0x7f9ef8067720 tx=0x7f9ef0009780).stop 2023-11-15T03:07:49.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.792+0000 7f9efcfb9700 1 -- 172.21.15.121:0/766844633 shutdown_connections 2023-11-15T03:07:49.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.792+0000 7f9efcfb9700 1 --2- 172.21.15.121:0/766844633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ef80c30e0 0x7f9ef80c3560 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:49.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.792+0000 7f9efcfb9700 1 --2- 172.21.15.121:0/766844633 >> v2:172.21.15.190:3300/0 conn(0x7f9ef8151870 0x7f9ef8151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:49.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.792+0000 7f9efcfb9700 1 -- 172.21.15.121:0/766844633 >> 172.21.15.121:0/766844633 conn(0x7f9ef8062190 msgr2=0x7f9ef8062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:49.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.792+0000 7f9efcfb9700 1 -- 172.21.15.121:0/766844633 shutdown_connections 2023-11-15T03:07:49.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.792+0000 7f9efcfb9700 1 -- 172.21.15.121:0/766844633 wait complete. 2023-11-15T03:07:49.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.796+0000 7f9efcfb9700 1 Processor -- start 2023-11-15T03:07:49.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.796+0000 7f9efcfb9700 1 -- start start 2023-11-15T03:07:49.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9efcfb9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ef80c30e0 0x7f9ef81652c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:49.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9efcfb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ef8151870 0x7f9ef81657b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:49.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9efcfb9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ef8152140 0x7f9ef81569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:49.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9efcfb9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef8165ca0 con 0x7f9ef8152140 2023-11-15T03:07:49.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9efcfb9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef8163060 con 0x7f9ef80c30e0 2023-11-15T03:07:49.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9efcfb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef8163340 con 0x7f9ef8151870 2023-11-15T03:07:49.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9ef7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ef8152140 0x7f9ef81569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:49.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9ef7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ef8152140 0x7f9ef81569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38194/0 (socket says 172.21.15.121:38194) 2023-11-15T03:07:49.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9ef7fff700 1 -- 172.21.15.121:0/506573546 learned_addr learned my addr 172.21.15.121:0/506573546 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:49.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9ef77fe700 1 --2- 172.21.15.121:0/506573546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ef80c30e0 0x7f9ef81652c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:49.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9eeeffd700 1 --2- 172.21.15.121:0/506573546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ef8151870 0x7f9ef81657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:49.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9ef7fff700 1 -- 172.21.15.121:0/506573546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ef8151870 msgr2=0x7f9ef81657b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:49.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9ef7fff700 1 --2- 172.21.15.121:0/506573546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ef8151870 0x7f9ef81657b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:49.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9ef7fff700 1 -- 172.21.15.121:0/506573546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ef80c30e0 msgr2=0x7f9ef81652c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:49.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9ef7fff700 1 --2- 172.21.15.121:0/506573546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ef80c30e0 0x7f9ef81652c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:49.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9ef7fff700 1 -- 172.21.15.121:0/506573546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef0009a20 con 0x7f9ef8152140 2023-11-15T03:07:49.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9ef7fff700 1 --2- 172.21.15.121:0/506573546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ef8152140 0x7f9ef81569f0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 rx=0x7f9ef80c3000 tx=0x7f9ef8065a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:49.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9eeffff700 1 -- 172.21.15.121:0/506573546 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef001d630 con 0x7f9ef8152140 2023-11-15T03:07:49.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9eeffff700 1 -- 172.21.15.121:0/506573546 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef001d790 con 0x7f9ef8152140 2023-11-15T03:07:49.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9eeffff700 1 -- 172.21.15.121:0/506573546 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef001db30 con 0x7f9ef8152140 2023-11-15T03:07:49.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9efcfb9700 1 -- 172.21.15.121:0/506573546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef8163620 con 0x7f9ef8152140 2023-11-15T03:07:49.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9efcfb9700 1 -- 172.21.15.121:0/506573546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef8163be0 con 0x7f9ef8152140 2023-11-15T03:07:49.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.800+0000 7f9efcfb9700 1 -- 172.21.15.121:0/506573546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef8163e00 con 0x7f9ef8152140 2023-11-15T03:07:49.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.804+0000 7f9eeffff700 1 -- 172.21.15.121:0/506573546 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9ef0027460 con 0x7f9ef8152140 2023-11-15T03:07:49.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.804+0000 7f9eeffff700 1 --2- 172.21.15.121:0/506573546 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ed805a510 0x7f9ed805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:49.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.804+0000 7f9eeffff700 1 -- 172.21.15.121:0/506573546 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8330+0+0 (secure 0 0 0) 0x7f9ef0076f60 con 0x7f9ef8152140 2023-11-15T03:07:49.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.804+0000 7f9ef77fe700 1 --2- 172.21.15.121:0/506573546 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ed805a510 0x7f9ed805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:49.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.804+0000 7f9ef77fe700 1 --2- 172.21.15.121:0/506573546 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ed805a510 0x7f9ed805c9d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f9ef80a9070 tx=0x7f9ee8008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:49.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:49.812+0000 7f9eeffff700 1 -- 172.21.15.121:0/506573546 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9ef0049d80 con 0x7f9ef8152140 2023-11-15T03:07:50.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.008+0000 7f9efcfb9700 1 -- 172.21.15.121:0/506573546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ef806bcf0 con 0x7f9ef8152140 2023-11-15T03:07:50.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.008+0000 7f9eeffff700 1 -- 172.21.15.121:0/506573546 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 229 v229) v1 ==== 95+0+4493 (secure 0 0 0) 0x7f9ef004d3a0 con 0x7f9ef8152140 2023-11-15T03:07:50.015 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:50.016 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":229,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9273,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/461059977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":461059977},{"type":"v1","addr":"172.21.15.121:6836","nonce":461059977}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":229}],"filesystems":[{"mdsmap":{"epoch":228,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:45.192820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:50.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.008+0000 7f9efcfb9700 1 -- 172.21.15.121:0/506573546 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ed805a510 msgr2=0x7f9ed805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:50.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.008+0000 7f9efcfb9700 1 --2- 172.21.15.121:0/506573546 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ed805a510 0x7f9ed805c9d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f9ef80a9070 tx=0x7f9ee8008040).stop 2023-11-15T03:07:50.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.008+0000 7f9efcfb9700 1 -- 172.21.15.121:0/506573546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ef8152140 msgr2=0x7f9ef81569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:50.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.008+0000 7f9efcfb9700 1 --2- 172.21.15.121:0/506573546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ef8152140 0x7f9ef81569f0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 rx=0x7f9ef80c3000 tx=0x7f9ef8065a40).stop 2023-11-15T03:07:50.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.012+0000 7f9efcfb9700 1 -- 172.21.15.121:0/506573546 shutdown_connections 2023-11-15T03:07:50.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.012+0000 7f9efcfb9700 1 --2- 172.21.15.121:0/506573546 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ed805a510 0x7f9ed805c9d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.012+0000 7f9efcfb9700 1 --2- 172.21.15.121:0/506573546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ef8152140 0x7f9ef81569f0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.012+0000 7f9efcfb9700 1 --2- 172.21.15.121:0/506573546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ef8151870 0x7f9ef81657b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.012+0000 7f9efcfb9700 1 --2- 172.21.15.121:0/506573546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ef80c30e0 0x7f9ef81652c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.012+0000 7f9efcfb9700 1 -- 172.21.15.121:0/506573546 >> 172.21.15.121:0/506573546 conn(0x7f9ef8062190 msgr2=0x7f9ef81508c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:50.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.012+0000 7f9efcfb9700 1 -- 172.21.15.121:0/506573546 shutdown_connections 2023-11-15T03:07:50.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.012+0000 7f9efcfb9700 1 -- 172.21.15.121:0/506573546 wait complete. 2023-11-15T03:07:50.021 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 229 2023-11-15T03:07:50.037 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:07:50.037 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:50.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.184+0000 7f58d961b700 1 Processor -- start 2023-11-15T03:07:50.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.184+0000 7f58d961b700 1 -- start start 2023-11-15T03:07:50.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.188+0000 7f58d961b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f58d40c06e0 0x7f58d4153db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:50.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.188+0000 7f58d961b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d40c2670 0x7f58d40c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:50.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.188+0000 7f58d961b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d40c1530 con 0x7f58d41539d0 2023-11-15T03:07:50.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.188+0000 7f58d961b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d40c2b00 con 0x7f58d40c06e0 2023-11-15T03:07:50.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.188+0000 7f58d961b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d40c2d80 con 0x7f58d40c2670 2023-11-15T03:07:50.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.188+0000 7f58d27fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f58d40c06e0 0x7f58d4153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:50.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.188+0000 7f58d27fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f58d40c06e0 0x7f58d4153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:47950/0 (socket says 172.21.15.121:47950) 2023-11-15T03:07:50.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.188+0000 7f58d27fc700 1 -- 172.21.15.121:0/1729637627 learned_addr learned my addr 172.21.15.121:0/1729637627 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:50.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.188+0000 7f58d37fe700 1 --2- 172.21.15.121:0/1729637627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d40c2670 0x7f58d40c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:50.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.188+0000 7f58d2ffd700 1 --1- 172.21.15.121:0/1729637627 >> v1:172.21.15.190:6789/0 conn(0x7f58d41539d0 0x7f58d40c19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:55630/0 (socket says 172.21.15.121:55630) 2023-11-15T03:07:50.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.188+0000 7f58d27fc700 1 -- 172.21.15.121:0/1729637627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d40c2670 msgr2=0x7f58d40c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:50.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.188+0000 7f58d27fc700 1 --2- 172.21.15.121:0/1729637627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d40c2670 0x7f58d40c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.188+0000 7f58d27fc700 1 -- 172.21.15.121:0/1729637627 >> v1:172.21.15.190:6789/0 conn(0x7f58d41539d0 legacy=0x7f58d40c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:50.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.188+0000 7f58d27fc700 1 -- 172.21.15.121:0/1729637627 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d40c0ee0 con 0x7f58d40c06e0 2023-11-15T03:07:50.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.188+0000 7f58d1ffb700 1 -- 172.21.15.121:0/1729637627 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309187707 0 0) 0x7f58d40c1530 con 0x7f58d41539d0 2023-11-15T03:07:50.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.188+0000 7f58d27fc700 1 --2- 172.21.15.121:0/1729637627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d40c06e0 0x7f58d4153db0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f58c8006fd0 tx=0x7f58c80090f0).ready entity=mon.1 client_cookie=28a68d9dfa9d76ab server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:50.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.188+0000 7f58d1ffb700 1 -- 172.21.15.121:0/1729637627 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c800b580 con 0x7f58d40c06e0 2023-11-15T03:07:50.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.188+0000 7f58d1ffb700 1 -- 172.21.15.121:0/1729637627 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c800b6e0 con 0x7f58d40c06e0 2023-11-15T03:07:50.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.188+0000 7f58d1ffb700 1 -- 172.21.15.121:0/1729637627 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c800ba60 con 0x7f58d40c06e0 2023-11-15T03:07:50.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.192+0000 7f58d961b700 1 -- 172.21.15.121:0/1729637627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d40c06e0 msgr2=0x7f58d4153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:50.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.192+0000 7f58d961b700 1 --2- 172.21.15.121:0/1729637627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d40c06e0 0x7f58d4153db0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f58c8006fd0 tx=0x7f58c80090f0).stop 2023-11-15T03:07:50.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.192+0000 7f58d961b700 1 -- 172.21.15.121:0/1729637627 shutdown_connections 2023-11-15T03:07:50.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.192+0000 7f58d961b700 1 --2- 172.21.15.121:0/1729637627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d40c2670 0x7f58d40c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.192+0000 7f58d961b700 1 --2- 172.21.15.121:0/1729637627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d40c06e0 0x7f58d4153db0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.192+0000 7f58d961b700 1 -- 172.21.15.121:0/1729637627 >> 172.21.15.121:0/1729637627 conn(0x7f58d4062190 msgr2=0x7f58d40625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:50.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.192+0000 7f58d961b700 1 -- 172.21.15.121:0/1729637627 shutdown_connections 2023-11-15T03:07:50.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.192+0000 7f58d961b700 1 -- 172.21.15.121:0/1729637627 wait complete. 2023-11-15T03:07:50.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.196+0000 7f58d961b700 1 Processor -- start 2023-11-15T03:07:50.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.196+0000 7f58d961b700 1 -- start start 2023-11-15T03:07:50.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.196+0000 7f58d961b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d40c06e0 0x7f58d40aa340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:50.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.196+0000 7f58d961b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d40c2670 0x7f58d414d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:50.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.196+0000 7f58d961b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d41539d0 0x7f58d41585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:50.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.196+0000 7f58d961b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d40a9eb0 con 0x7f58d40c06e0 2023-11-15T03:07:50.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.196+0000 7f58d961b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d40a98d0 con 0x7f58d40c2670 2023-11-15T03:07:50.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.196+0000 7f58d961b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d4158f00 con 0x7f58d41539d0 2023-11-15T03:07:50.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.196+0000 7f58d2ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d40c06e0 0x7f58d40aa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:50.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.196+0000 7f58d2ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d40c06e0 0x7f58d40aa340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38200/0 (socket says 172.21.15.121:38200) 2023-11-15T03:07:50.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.196+0000 7f58d2ffd700 1 -- 172.21.15.121:0/2449742944 learned_addr learned my addr 172.21.15.121:0/2449742944 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:50.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.196+0000 7f58d27fc700 1 --2- 172.21.15.121:0/2449742944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d40c2670 0x7f58d414d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:50.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.196+0000 7f58d37fe700 1 --2- 172.21.15.121:0/2449742944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d41539d0 0x7f58d41585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:50.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.196+0000 7f58d2ffd700 1 -- 172.21.15.121:0/2449742944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d41539d0 msgr2=0x7f58d41585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:50.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.196+0000 7f58d2ffd700 1 --2- 172.21.15.121:0/2449742944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d41539d0 0x7f58d41585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.196+0000 7f58d2ffd700 1 -- 172.21.15.121:0/2449742944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d40c2670 msgr2=0x7f58d414d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:50.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.196+0000 7f58d2ffd700 1 --2- 172.21.15.121:0/2449742944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d40c2670 0x7f58d414d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.200+0000 7f58d2ffd700 1 -- 172.21.15.121:0/2449742944 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c8009370 con 0x7f58d40c06e0 2023-11-15T03:07:50.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.200+0000 7f58d2ffd700 1 --2- 172.21.15.121:0/2449742944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d40c06e0 0x7f58d40aa340 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 rx=0x7f58c400c470 tx=0x7f58c4007e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:50.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.200+0000 7f58baffd700 1 -- 172.21.15.121:0/2449742944 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c40072e0 con 0x7f58d40c06e0 2023-11-15T03:07:50.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.200+0000 7f58d961b700 1 -- 172.21.15.121:0/2449742944 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d40a9b40 con 0x7f58d40c06e0 2023-11-15T03:07:50.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.200+0000 7f58baffd700 1 -- 172.21.15.121:0/2449742944 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c4007440 con 0x7f58d40c06e0 2023-11-15T03:07:50.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.200+0000 7f58baffd700 1 -- 172.21.15.121:0/2449742944 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c4005910 con 0x7f58d40c06e0 2023-11-15T03:07:50.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.200+0000 7f58d961b700 1 -- 172.21.15.121:0/2449742944 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d41591e0 con 0x7f58d40c06e0 2023-11-15T03:07:50.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.200+0000 7f58baffd700 1 -- 172.21.15.121:0/2449742944 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f58c4005d00 con 0x7f58d40c06e0 2023-11-15T03:07:50.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.200+0000 7f58d961b700 1 -- 172.21.15.121:0/2449742944 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d406c150 con 0x7f58d40c06e0 2023-11-15T03:07:50.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.204+0000 7f58baffd700 1 --2- 172.21.15.121:0/2449742944 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58c005a4e0 0x7f58c005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:50.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.204+0000 7f58baffd700 1 -- 172.21.15.121:0/2449742944 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8330+0+0 (secure 0 0 0) 0x7f58c40748c0 con 0x7f58d40c06e0 2023-11-15T03:07:50.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.204+0000 7f58d27fc700 1 --2- 172.21.15.121:0/2449742944 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58c005a4e0 0x7f58c005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:50.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.204+0000 7f58d27fc700 1 --2- 172.21.15.121:0/2449742944 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58c005a4e0 0x7f58c005c9a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f58c8009fd0 tx=0x7f58c8017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:50.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.208+0000 7f58baffd700 1 -- 172.21.15.121:0/2449742944 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f58c4047740 con 0x7f58d40c06e0 2023-11-15T03:07:50.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.400+0000 7f58d961b700 1 -- 172.21.15.121:0/2449742944 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58d4068ae0 con 0x7f58d40c06e0 2023-11-15T03:07:50.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.404+0000 7f58baffd700 1 -- 172.21.15.121:0/2449742944 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 229 v229) v1 ==== 95+0+4493 (secure 0 0 0) 0x7f58c4018020 con 0x7f58d40c06e0 2023-11-15T03:07:50.411 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:50.411 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":229,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9273,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/461059977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":461059977},{"type":"v1","addr":"172.21.15.121:6836","nonce":461059977}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":229}],"filesystems":[{"mdsmap":{"epoch":228,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:45.192820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:50.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.404+0000 7f58d961b700 1 -- 172.21.15.121:0/2449742944 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58c005a4e0 msgr2=0x7f58c005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:50.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.404+0000 7f58d961b700 1 --2- 172.21.15.121:0/2449742944 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58c005a4e0 0x7f58c005c9a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f58c8009fd0 tx=0x7f58c8017000).stop 2023-11-15T03:07:50.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.404+0000 7f58d961b700 1 -- 172.21.15.121:0/2449742944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d40c06e0 msgr2=0x7f58d40aa340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:50.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.404+0000 7f58d961b700 1 --2- 172.21.15.121:0/2449742944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d40c06e0 0x7f58d40aa340 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 rx=0x7f58c400c470 tx=0x7f58c4007e30).stop 2023-11-15T03:07:50.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.404+0000 7f58d961b700 1 -- 172.21.15.121:0/2449742944 shutdown_connections 2023-11-15T03:07:50.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.404+0000 7f58d961b700 1 --2- 172.21.15.121:0/2449742944 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58c005a4e0 0x7f58c005c9a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.404+0000 7f58d961b700 1 --2- 172.21.15.121:0/2449742944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d41539d0 0x7f58d41585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.404+0000 7f58d961b700 1 --2- 172.21.15.121:0/2449742944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d40c2670 0x7f58d414d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.404+0000 7f58d961b700 1 --2- 172.21.15.121:0/2449742944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d40c06e0 0x7f58d40aa340 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.404+0000 7f58d961b700 1 -- 172.21.15.121:0/2449742944 >> 172.21.15.121:0/2449742944 conn(0x7f58d4062190 msgr2=0x7f58d414f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:50.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.404+0000 7f58d961b700 1 -- 172.21.15.121:0/2449742944 shutdown_connections 2023-11-15T03:07:50.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.408+0000 7f58d961b700 1 -- 172.21.15.121:0/2449742944 wait complete. 2023-11-15T03:07:50.416 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 229 2023-11-15T03:07:50.434 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:07:50.434 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:50.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.584+0000 7fdb8ae8f700 1 Processor -- start 2023-11-15T03:07:50.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.584+0000 7fdb8ae8f700 1 -- start start 2023-11-15T03:07:50.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.584+0000 7fdb8ae8f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb840c3280 0x7fdb840c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:50.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.584+0000 7fdb8ae8f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdb8415be30 0x7fdb8415c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:50.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.584+0000 7fdb8ae8f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb840c1d50 con 0x7fdb84152140 2023-11-15T03:07:50.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.584+0000 7fdb8ae8f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb840c1860 con 0x7fdb8415be30 2023-11-15T03:07:50.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.584+0000 7fdb8ae8f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8415cb80 con 0x7fdb840c3280 2023-11-15T03:07:50.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb8a68e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdb8415be30 0x7fdb8415c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:50.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb8968c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fdb84152140 0x7fdb841525c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:55638/0 (socket says 172.21.15.121:55638) 2023-11-15T03:07:50.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb8968c700 1 -- 172.21.15.121:0/1523335533 learned_addr learned my addr 172.21.15.121:0/1523335533 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:50.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb8a68e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdb8415be30 0x7fdb8415c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38216/0 (socket says 172.21.15.121:38216) 2023-11-15T03:07:50.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb89e8d700 1 --2- 172.21.15.121:0/1523335533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb840c3280 0x7fdb840c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:50.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb88e8b700 1 -- 172.21.15.121:0/1523335533 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191264494 0 0) 0x7fdb840c1d50 con 0x7fdb84152140 2023-11-15T03:07:50.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb88e8b700 1 -- 172.21.15.121:0/1523335533 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb700033f0 con 0x7fdb84152140 2023-11-15T03:07:50.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb8a68e700 1 -- 172.21.15.121:0/1523335533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb840c3280 msgr2=0x7fdb840c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:50.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb8a68e700 1 --2- 172.21.15.121:0/1523335533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb840c3280 0x7fdb840c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb8a68e700 1 -- 172.21.15.121:0/1523335533 >> v1:172.21.15.190:6789/0 conn(0x7fdb84152140 legacy=0x7fdb841525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:50.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb8a68e700 1 -- 172.21.15.121:0/1523335533 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb840a9240 con 0x7fdb8415be30 2023-11-15T03:07:50.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb89e8d700 1 --2- 172.21.15.121:0/1523335533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb840c3280 0x7fdb840c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:07:50.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb8a68e700 1 --2- 172.21.15.121:0/1523335533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8415be30 0x7fdb8415c2a0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 rx=0x7fdb7c009e60 tx=0x7fdb7c009780).ready entity=mon.0 client_cookie=c8b6fc37a7f8acea server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:50.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb88e8b700 1 -- 172.21.15.121:0/1523335533 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c00e070 con 0x7fdb8415be30 2023-11-15T03:07:50.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb88e8b700 1 -- 172.21.15.121:0/1523335533 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7c00bb60 con 0x7fdb8415be30 2023-11-15T03:07:50.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb88e8b700 1 -- 172.21.15.121:0/1523335533 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c012410 con 0x7fdb8415be30 2023-11-15T03:07:50.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb8ae8f700 1 -- 172.21.15.121:0/1523335533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8415be30 msgr2=0x7fdb8415c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:50.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb8ae8f700 1 --2- 172.21.15.121:0/1523335533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8415be30 0x7fdb8415c2a0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 rx=0x7fdb7c009e60 tx=0x7fdb7c009780).stop 2023-11-15T03:07:50.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb8ae8f700 1 -- 172.21.15.121:0/1523335533 shutdown_connections 2023-11-15T03:07:50.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb8ae8f700 1 --2- 172.21.15.121:0/1523335533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8415be30 0x7fdb8415c2a0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb8ae8f700 1 --2- 172.21.15.121:0/1523335533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb840c3280 0x7fdb840c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb8ae8f700 1 -- 172.21.15.121:0/1523335533 >> 172.21.15.121:0/1523335533 conn(0x7fdb84062190 msgr2=0x7fdb84062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:50.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb8ae8f700 1 -- 172.21.15.121:0/1523335533 shutdown_connections 2023-11-15T03:07:50.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.588+0000 7fdb8ae8f700 1 -- 172.21.15.121:0/1523335533 wait complete. 2023-11-15T03:07:50.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.592+0000 7fdb8ae8f700 1 Processor -- start 2023-11-15T03:07:50.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.592+0000 7fdb8ae8f700 1 -- start start 2023-11-15T03:07:50.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.592+0000 7fdb8ae8f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb840c3280 0x7fdb8414da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:50.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.592+0000 7fdb8ae8f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb84152140 0x7fdb840bc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:50.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.592+0000 7fdb8ae8f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8415be30 0x7fdb840bff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:50.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.592+0000 7fdb8ae8f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb840bfa70 con 0x7fdb8415be30 2023-11-15T03:07:50.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.592+0000 7fdb8ae8f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb841513a0 con 0x7fdb840c3280 2023-11-15T03:07:50.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.592+0000 7fdb8ae8f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8414e5e0 con 0x7fdb84152140 2023-11-15T03:07:50.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.596+0000 7fdb8a68e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8415be30 0x7fdb840bff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:50.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.596+0000 7fdb8a68e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8415be30 0x7fdb840bff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38226/0 (socket says 172.21.15.121:38226) 2023-11-15T03:07:50.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.596+0000 7fdb8a68e700 1 -- 172.21.15.121:0/4057730144 learned_addr learned my addr 172.21.15.121:0/4057730144 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:50.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.596+0000 7fdb89e8d700 1 --2- 172.21.15.121:0/4057730144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb840c3280 0x7fdb8414da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:50.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.596+0000 7fdb8968c700 1 --2- 172.21.15.121:0/4057730144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb84152140 0x7fdb840bc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:50.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.596+0000 7fdb8a68e700 1 -- 172.21.15.121:0/4057730144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb84152140 msgr2=0x7fdb840bc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:50.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.596+0000 7fdb8a68e700 1 --2- 172.21.15.121:0/4057730144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb84152140 0x7fdb840bc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.596+0000 7fdb8a68e700 1 -- 172.21.15.121:0/4057730144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb840c3280 msgr2=0x7fdb8414da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:50.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.596+0000 7fdb8a68e700 1 --2- 172.21.15.121:0/4057730144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb840c3280 0x7fdb8414da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.596+0000 7fdb8a68e700 1 -- 172.21.15.121:0/4057730144 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c009a20 con 0x7fdb8415be30 2023-11-15T03:07:50.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.596+0000 7fdb8a68e700 1 --2- 172.21.15.121:0/4057730144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8415be30 0x7fdb840bff00 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 rx=0x7fdb7c00b6c0 tx=0x7fdb7c009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:50.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.596+0000 7fdb79ffb700 1 -- 172.21.15.121:0/4057730144 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c01d630 con 0x7fdb8415be30 2023-11-15T03:07:50.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.596+0000 7fdb79ffb700 1 -- 172.21.15.121:0/4057730144 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7c01d790 con 0x7fdb8415be30 2023-11-15T03:07:50.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.596+0000 7fdb79ffb700 1 -- 172.21.15.121:0/4057730144 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c01db50 con 0x7fdb8415be30 2023-11-15T03:07:50.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.596+0000 7fdb8ae8f700 1 -- 172.21.15.121:0/4057730144 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8414e8c0 con 0x7fdb8415be30 2023-11-15T03:07:50.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.596+0000 7fdb8ae8f700 1 -- 172.21.15.121:0/4057730144 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8414e270 con 0x7fdb8415be30 2023-11-15T03:07:50.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.596+0000 7fdb8ae8f700 1 -- 172.21.15.121:0/4057730144 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb840bfbe0 con 0x7fdb8415be30 2023-11-15T03:07:50.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.600+0000 7fdb79ffb700 1 -- 172.21.15.121:0/4057730144 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fdb7c003040 con 0x7fdb8415be30 2023-11-15T03:07:50.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.600+0000 7fdb79ffb700 1 --2- 172.21.15.121:0/4057730144 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdb6805a510 0x7fdb6805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:50.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.600+0000 7fdb79ffb700 1 -- 172.21.15.121:0/4057730144 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8330+0+0 (secure 0 0 0) 0x7fdb7c077110 con 0x7fdb8415be30 2023-11-15T03:07:50.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.600+0000 7fdb89e8d700 1 --2- 172.21.15.121:0/4057730144 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdb6805a510 0x7fdb6805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:50.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.600+0000 7fdb89e8d700 1 --2- 172.21.15.121:0/4057730144 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdb6805a510 0x7fdb6805c9d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fdb80006fd0 tx=0x7fdb80008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:50.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.608+0000 7fdb79ffb700 1 -- 172.21.15.121:0/4057730144 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdb7c04a010 con 0x7fdb8415be30 2023-11-15T03:07:50.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.800+0000 7fdb8ae8f700 1 -- 172.21.15.121:0/4057730144 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb840a7cc0 con 0x7fdb8415be30 2023-11-15T03:07:50.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.800+0000 7fdb79ffb700 1 -- 172.21.15.121:0/4057730144 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 229 v229) v1 ==== 95+0+4493 (secure 0 0 0) 0x7fdb7c04d630 con 0x7fdb8415be30 2023-11-15T03:07:50.809 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:50.809 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":229,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9273,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/461059977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":461059977},{"type":"v1","addr":"172.21.15.121:6836","nonce":461059977}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":229}],"filesystems":[{"mdsmap":{"epoch":228,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:45.192820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:50.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.804+0000 7fdb8ae8f700 1 -- 172.21.15.121:0/4057730144 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdb6805a510 msgr2=0x7fdb6805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:50.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.804+0000 7fdb8ae8f700 1 --2- 172.21.15.121:0/4057730144 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdb6805a510 0x7fdb6805c9d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fdb80006fd0 tx=0x7fdb80008040).stop 2023-11-15T03:07:50.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.804+0000 7fdb8ae8f700 1 -- 172.21.15.121:0/4057730144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8415be30 msgr2=0x7fdb840bff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:50.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.804+0000 7fdb8ae8f700 1 --2- 172.21.15.121:0/4057730144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8415be30 0x7fdb840bff00 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 rx=0x7fdb7c00b6c0 tx=0x7fdb7c009780).stop 2023-11-15T03:07:50.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.804+0000 7fdb8ae8f700 1 -- 172.21.15.121:0/4057730144 shutdown_connections 2023-11-15T03:07:50.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.804+0000 7fdb8ae8f700 1 --2- 172.21.15.121:0/4057730144 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdb6805a510 0x7fdb6805c9d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.804+0000 7fdb8ae8f700 1 --2- 172.21.15.121:0/4057730144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8415be30 0x7fdb840bff00 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.804+0000 7fdb8ae8f700 1 --2- 172.21.15.121:0/4057730144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb84152140 0x7fdb840bc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.804+0000 7fdb8ae8f700 1 --2- 172.21.15.121:0/4057730144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb840c3280 0x7fdb8414da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:50.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.804+0000 7fdb8ae8f700 1 -- 172.21.15.121:0/4057730144 >> 172.21.15.121:0/4057730144 conn(0x7fdb84062190 msgr2=0x7fdb8414f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:50.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.804+0000 7fdb8ae8f700 1 -- 172.21.15.121:0/4057730144 shutdown_connections 2023-11-15T03:07:50.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:50.804+0000 7fdb8ae8f700 1 -- 172.21.15.121:0/4057730144 wait complete. 2023-11-15T03:07:50.815 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 229 2023-11-15T03:07:50.832 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 1.00000000.inode' led to DAMAGED state 2023-11-15T03:07:50.832 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 200.00000000' 2023-11-15T03:07:50.832 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:07:50.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:07:50.833 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:07:50.849 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:07:50.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:07:50.851 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:07:51.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.060+0000 7f5caf169700 1 Processor -- start 2023-11-15T03:07:51.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.060+0000 7f5caf169700 1 -- start start 2023-11-15T03:07:51.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.064+0000 7f5caf169700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5ca8153aa0 0x7f5ca8066680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:51.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.064+0000 7f5caf169700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ca8153e80 0x7f5ca80ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:51.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.064+0000 7f5caf169700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca80c5c50 con 0x7f5ca80c0490 2023-11-15T03:07:51.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.064+0000 7f5caf169700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca8066b70 con 0x7f5ca8153aa0 2023-11-15T03:07:51.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.064+0000 7f5caf169700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca80c5920 con 0x7f5ca8153e80 2023-11-15T03:07:51.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.064+0000 7f5cacf05700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5ca8153aa0 0x7f5ca8066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:51.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.064+0000 7f5cad706700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f5ca80c0490 0x7f5ca8062190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60388/0 (socket says 172.21.15.121:60388) 2023-11-15T03:07:51.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.064+0000 7f5cad706700 1 -- 172.21.15.121:0/2422362649 learned_addr learned my addr 172.21.15.121:0/2422362649 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:51.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.064+0000 7f5ca7fff700 1 --2- 172.21.15.121:0/2422362649 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ca8153e80 0x7f5ca80ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:51.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.064+0000 7f5cacf05700 1 -- 172.21.15.121:0/2422362649 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ca8153e80 msgr2=0x7f5ca80ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:51.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.064+0000 7f5cacf05700 1 --2- 172.21.15.121:0/2422362649 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ca8153e80 0x7f5ca80ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:51.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.064+0000 7f5cacf05700 1 -- 172.21.15.121:0/2422362649 >> v1:172.21.15.121:6789/0 conn(0x7f5ca80c0490 legacy=0x7f5ca8062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:51.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.064+0000 7f5ca77fe700 1 -- 172.21.15.121:0/2422362649 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968296147 0 0) 0x7f5ca80c5c50 con 0x7f5ca80c0490 2023-11-15T03:07:51.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.064+0000 7f5cacf05700 1 -- 172.21.15.121:0/2422362649 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca8066d50 con 0x7f5ca8153aa0 2023-11-15T03:07:51.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.064+0000 7f5cacf05700 1 --2- 172.21.15.121:0/2422362649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ca8153aa0 0x7f5ca8066680 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 rx=0x7f5c9c009760 tx=0x7f5c9c009060).ready entity=mon.0 client_cookie=205e3c6d942a24a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:51.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.064+0000 7f5ca77fe700 1 -- 172.21.15.121:0/2422362649 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c00f510 con 0x7f5ca8153aa0 2023-11-15T03:07:51.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.064+0000 7f5ca77fe700 1 -- 172.21.15.121:0/2422362649 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9c00f670 con 0x7f5ca8153aa0 2023-11-15T03:07:51.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.064+0000 7f5ca77fe700 1 -- 172.21.15.121:0/2422362649 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c00f9f0 con 0x7f5ca8153aa0 2023-11-15T03:07:51.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.068+0000 7f5caf169700 1 -- 172.21.15.121:0/2422362649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ca8153aa0 msgr2=0x7f5ca8066680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:51.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.068+0000 7f5caf169700 1 --2- 172.21.15.121:0/2422362649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ca8153aa0 0x7f5ca8066680 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 rx=0x7f5c9c009760 tx=0x7f5c9c009060).stop 2023-11-15T03:07:51.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.068+0000 7f5caf169700 1 -- 172.21.15.121:0/2422362649 shutdown_connections 2023-11-15T03:07:51.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.068+0000 7f5caf169700 1 --2- 172.21.15.121:0/2422362649 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ca8153e80 0x7f5ca80ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:51.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.068+0000 7f5caf169700 1 --2- 172.21.15.121:0/2422362649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ca8153aa0 0x7f5ca8066680 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:51.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.068+0000 7f5caf169700 1 -- 172.21.15.121:0/2422362649 >> 172.21.15.121:0/2422362649 conn(0x7f5ca8067a60 msgr2=0x7f5ca8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:51.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.068+0000 7f5caf169700 1 -- 172.21.15.121:0/2422362649 shutdown_connections 2023-11-15T03:07:51.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.068+0000 7f5caf169700 1 -- 172.21.15.121:0/2422362649 wait complete. 2023-11-15T03:07:51.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5caf169700 1 Processor -- start 2023-11-15T03:07:51.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5caf169700 1 -- start start 2023-11-15T03:07:51.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5caf169700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ca80c0490 0x7f5ca8152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:51.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5caf169700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ca8153aa0 0x7f5ca8150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:51.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5caf169700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ca8153e80 0x7f5ca8156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:51.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5caf169700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca8152b80 con 0x7f5ca8153aa0 2023-11-15T03:07:51.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5caf169700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca814a050 con 0x7f5ca80c0490 2023-11-15T03:07:51.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5caf169700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca8167b00 con 0x7f5ca8153e80 2023-11-15T03:07:51.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5ca7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ca8153aa0 0x7f5ca8150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:51.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5ca7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ca8153aa0 0x7f5ca8150ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38248/0 (socket says 172.21.15.121:38248) 2023-11-15T03:07:51.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5ca7fff700 1 -- 172.21.15.121:0/2775374964 learned_addr learned my addr 172.21.15.121:0/2775374964 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:51.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5cacf05700 1 --2- 172.21.15.121:0/2775374964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ca80c0490 0x7f5ca8152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:51.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5cad706700 1 --2- 172.21.15.121:0/2775374964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ca8153e80 0x7f5ca8156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:51.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5ca7fff700 1 -- 172.21.15.121:0/2775374964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ca8153e80 msgr2=0x7f5ca8156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:51.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5ca7fff700 1 --2- 172.21.15.121:0/2775374964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ca8153e80 0x7f5ca8156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:51.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5ca7fff700 1 -- 172.21.15.121:0/2775374964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ca80c0490 msgr2=0x7f5ca8152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:51.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5ca7fff700 1 --2- 172.21.15.121:0/2775374964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ca80c0490 0x7f5ca8152fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:51.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5ca7fff700 1 -- 172.21.15.121:0/2775374964 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c0092a0 con 0x7f5ca8153aa0 2023-11-15T03:07:51.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5ca7fff700 1 --2- 172.21.15.121:0/2775374964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ca8153aa0 0x7f5ca8150ca0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 rx=0x7f5c9400e930 tx=0x7f5c94009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:51.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.072+0000 7f5ca4ff9700 1 -- 172.21.15.121:0/2775374964 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94012600 con 0x7f5ca8153aa0 2023-11-15T03:07:51.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.076+0000 7f5ca4ff9700 1 -- 172.21.15.121:0/2775374964 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c94012760 con 0x7f5ca8153aa0 2023-11-15T03:07:51.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.076+0000 7f5caf169700 1 -- 172.21.15.121:0/2775374964 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca8167de0 con 0x7f5ca8153aa0 2023-11-15T03:07:51.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.076+0000 7f5ca4ff9700 1 -- 172.21.15.121:0/2775374964 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94012ae0 con 0x7f5ca8153aa0 2023-11-15T03:07:51.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.076+0000 7f5caf169700 1 -- 172.21.15.121:0/2775374964 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca8151610 con 0x7f5ca8153aa0 2023-11-15T03:07:51.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.076+0000 7f5caf169700 1 -- 172.21.15.121:0/2775374964 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ca8047060 con 0x7f5ca8153aa0 2023-11-15T03:07:51.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.076+0000 7f5ca4ff9700 1 -- 172.21.15.121:0/2775374964 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5c9401b400 con 0x7f5ca8153aa0 2023-11-15T03:07:51.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.080+0000 7f5ca4ff9700 1 --2- 172.21.15.121:0/2775374964 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5c8c05a510 0x7f5c8c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:51.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.080+0000 7f5ca4ff9700 1 -- 172.21.15.121:0/2775374964 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8330+0+0 (secure 0 0 0) 0x7f5c94075500 con 0x7f5ca8153aa0 2023-11-15T03:07:51.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.080+0000 7f5cacf05700 1 --2- 172.21.15.121:0/2775374964 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5c8c05a510 0x7f5c8c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:51.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.080+0000 7f5cacf05700 1 --2- 172.21.15.121:0/2775374964 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5c8c05a510 0x7f5c8c05c9d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f5c9c009fd0 tx=0x7f5c9c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:51.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.084+0000 7f5ca4ff9700 1 -- 172.21.15.121:0/2775374964 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5c94048320 con 0x7f5ca8153aa0 2023-11-15T03:07:51.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:51.288+0000 7f5caf169700 1 -- 172.21.15.121:0/2775374964 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5ca8067030 con 0x7f5ca8153aa0 2023-11-15T03:07:52.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.168+0000 7f5ca4ff9700 1 -- 172.21.15.121:0/2775374964 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v230) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5c9404b940 con 0x7f5ca8153aa0 2023-11-15T03:07:52.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.172+0000 7f5caf169700 1 -- 172.21.15.121:0/2775374964 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5c8c05a510 msgr2=0x7f5c8c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:52.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.172+0000 7f5caf169700 1 --2- 172.21.15.121:0/2775374964 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5c8c05a510 0x7f5c8c05c9d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f5c9c009fd0 tx=0x7f5c9c017000).stop 2023-11-15T03:07:52.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.172+0000 7f5caf169700 1 -- 172.21.15.121:0/2775374964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ca8153aa0 msgr2=0x7f5ca8150ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:52.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.172+0000 7f5caf169700 1 --2- 172.21.15.121:0/2775374964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ca8153aa0 0x7f5ca8150ca0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 rx=0x7f5c9400e930 tx=0x7f5c94009060).stop 2023-11-15T03:07:52.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.172+0000 7f5caf169700 1 -- 172.21.15.121:0/2775374964 shutdown_connections 2023-11-15T03:07:52.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.172+0000 7f5caf169700 1 --2- 172.21.15.121:0/2775374964 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5c8c05a510 0x7f5c8c05c9d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:52.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.172+0000 7f5caf169700 1 --2- 172.21.15.121:0/2775374964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ca8153e80 0x7f5ca8156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:52.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.172+0000 7f5caf169700 1 --2- 172.21.15.121:0/2775374964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ca8153aa0 0x7f5ca8150ca0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:52.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.172+0000 7f5caf169700 1 --2- 172.21.15.121:0/2775374964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ca80c0490 0x7f5ca8152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:52.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.172+0000 7f5caf169700 1 -- 172.21.15.121:0/2775374964 >> 172.21.15.121:0/2775374964 conn(0x7f5ca8067a60 msgr2=0x7f5ca80ca330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:52.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.176+0000 7f5caf169700 1 -- 172.21.15.121:0/2775374964 shutdown_connections 2023-11-15T03:07:52.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.176+0000 7f5caf169700 1 -- 172.21.15.121:0/2775374964 wait complete. 2023-11-15T03:07:52.185 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:07:52.204 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:07:52.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.364+0000 7f4375bdb700 1 Processor -- start 2023-11-15T03:07:52.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.364+0000 7f4375bdb700 1 -- start start 2023-11-15T03:07:52.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f4375bdb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4370153aa0 0x7f4370066680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:52.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f4375bdb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43700c5c50 con 0x7f4370153e80 2023-11-15T03:07:52.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f4375bdb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43700c5920 con 0x7f43700c0490 2023-11-15T03:07:52.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f4375bdb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4370066b70 con 0x7f4370153aa0 2023-11-15T03:07:52.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f4367fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4370153e80 0x7f43700ad150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60400/0 (socket says 172.21.15.121:60400) 2023-11-15T03:07:52.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f4367fff700 1 -- 172.21.15.121:0/2309145653 learned_addr learned my addr 172.21.15.121:0/2309145653 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:52.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f436f7fe700 1 --2- 172.21.15.121:0/2309145653 >> v2:172.21.15.190:3300/0 conn(0x7f4370153aa0 0x7f4370066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:52.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f436effd700 1 -- 172.21.15.121:0/2309145653 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630351414 0 0) 0x7f43700c5c50 con 0x7f4370153e80 2023-11-15T03:07:52.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f436effd700 1 -- 172.21.15.121:0/2309145653 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f435c0033f0 con 0x7f4370153e80 2023-11-15T03:07:52.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f436effd700 1 -- 172.21.15.121:0/2309145653 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1569048205 0 0) 0x7f435c0033f0 con 0x7f4370153e80 2023-11-15T03:07:52.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f436effd700 1 -- 172.21.15.121:0/2309145653 >> v2:172.21.15.190:3300/0 conn(0x7f4370153aa0 msgr2=0x7f4370066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:52.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f436effd700 1 --2- 172.21.15.121:0/2309145653 >> v2:172.21.15.190:3300/0 conn(0x7f4370153aa0 0x7f4370066680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:52.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f436effd700 1 -- 172.21.15.121:0/2309145653 >> v1:172.21.15.190:6789/0 conn(0x7f43700c0490 legacy=0x7f43700c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:52.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f436effd700 1 -- 172.21.15.121:0/2309145653 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4370066d50 con 0x7f4370153e80 2023-11-15T03:07:52.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f436effd700 1 -- 172.21.15.121:0/2309145653 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f4358001470 con 0x7f4370153e80 2023-11-15T03:07:52.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f436effd700 1 -- 172.21.15.121:0/2309145653 >> v1:172.21.15.121:6789/0 conn(0x7f4370153e80 legacy=0x7f43700ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:52.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f436effd700 1 --2- 172.21.15.121:0/2309145653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f435c0038e0 0x7f435c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:52.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f436effd700 1 --2- 172.21.15.121:0/2309145653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c004690 0x7f435c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:52.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f436effd700 1 --2- 172.21.15.121:0/2309145653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f435c005000 0x7f435c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:52.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f436effd700 1 -- 172.21.15.121:0/2309145653 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4370066b70 con 0x7f435c0038e0 2023-11-15T03:07:52.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f436effd700 1 -- 172.21.15.121:0/2309145653 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c00bf60 con 0x7f435c005000 2023-11-15T03:07:52.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.368+0000 7f436effd700 1 -- 172.21.15.121:0/2309145653 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435c00c0f0 con 0x7f435c004690 2023-11-15T03:07:52.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f436f7fe700 1 --2- 172.21.15.121:0/2309145653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f435c0038e0 0x7f435c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:52.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f4367fff700 1 --2- 172.21.15.121:0/2309145653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c004690 0x7f435c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:52.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f436ffff700 1 --2- 172.21.15.121:0/2309145653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f435c005000 0x7f435c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:52.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f4367fff700 1 -- 172.21.15.121:0/2309145653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f435c005000 msgr2=0x7f435c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:52.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f4367fff700 1 --2- 172.21.15.121:0/2309145653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f435c005000 0x7f435c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:52.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f4367fff700 1 -- 172.21.15.121:0/2309145653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f435c0038e0 msgr2=0x7f435c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:52.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f4367fff700 1 --2- 172.21.15.121:0/2309145653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f435c0038e0 0x7f435c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:52.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f4367fff700 1 -- 172.21.15.121:0/2309145653 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c0033f0 con 0x7f435c004690 2023-11-15T03:07:52.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f436ffff700 1 -- 172.21.15.121:0/2309145653 reap_dead start 2023-11-15T03:07:52.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f4367fff700 1 --2- 172.21.15.121:0/2309145653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c004690 0x7f435c004b10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f435800c160 tx=0x7f4358001e50).ready entity=mon.2 client_cookie=8421ba707429c919 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:52.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f436effd700 1 -- 172.21.15.121:0/2309145653 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358007a20 con 0x7f435c004690 2023-11-15T03:07:52.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f436effd700 1 -- 172.21.15.121:0/2309145653 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435800ca90 con 0x7f435c004690 2023-11-15T03:07:52.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f436effd700 1 -- 172.21.15.121:0/2309145653 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435800ce10 con 0x7f435c004690 2023-11-15T03:07:52.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f4375bdb700 1 -- 172.21.15.121:0/2309145653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c004690 msgr2=0x7f435c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:52.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f4375bdb700 1 --2- 172.21.15.121:0/2309145653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c004690 0x7f435c004b10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f435800c160 tx=0x7f4358001e50).stop 2023-11-15T03:07:52.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f4375bdb700 1 -- 172.21.15.121:0/2309145653 shutdown_connections 2023-11-15T03:07:52.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f4375bdb700 1 --2- 172.21.15.121:0/2309145653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c004690 0x7f435c004b10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:52.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f4375bdb700 1 -- 172.21.15.121:0/2309145653 >> 172.21.15.121:0/2309145653 conn(0x7f4370067a60 msgr2=0x7f4370067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:52.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f4375bdb700 1 -- 172.21.15.121:0/2309145653 shutdown_connections 2023-11-15T03:07:52.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.372+0000 7f4375bdb700 1 -- 172.21.15.121:0/2309145653 wait complete. 2023-11-15T03:07:52.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.376+0000 7f4375bdb700 1 Processor -- start 2023-11-15T03:07:52.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.376+0000 7f4375bdb700 1 -- start start 2023-11-15T03:07:52.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f4375bdb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c004690 0x7f43700acea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:52.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f4375bdb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f437014f260 0x7f43700af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:52.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f4375bdb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f437014f6e0 0x7f43700afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:52.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f4375bdb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437014ff10 con 0x7f437014f6e0 2023-11-15T03:07:52.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f4375bdb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43700b0120 con 0x7f437014f260 2023-11-15T03:07:52.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f4375bdb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4370164380 con 0x7f435c004690 2023-11-15T03:07:52.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f436ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f437014f6e0 0x7f43700afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:52.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f436ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f437014f6e0 0x7f43700afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38272/0 (socket says 172.21.15.121:38272) 2023-11-15T03:07:52.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f436ffff700 1 -- 172.21.15.121:0/4001856026 learned_addr learned my addr 172.21.15.121:0/4001856026 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:52.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f436f7fe700 1 --2- 172.21.15.121:0/4001856026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c004690 0x7f43700acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:52.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f4367fff700 1 --2- 172.21.15.121:0/4001856026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f437014f260 0x7f43700af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:52.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f436ffff700 1 -- 172.21.15.121:0/4001856026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c004690 msgr2=0x7f43700acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:52.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f436ffff700 1 --2- 172.21.15.121:0/4001856026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c004690 0x7f43700acea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:52.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f436ffff700 1 -- 172.21.15.121:0/4001856026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f437014f260 msgr2=0x7f43700af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:52.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f436ffff700 1 --2- 172.21.15.121:0/4001856026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f437014f260 0x7f43700af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:52.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f436ffff700 1 -- 172.21.15.121:0/4001856026 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4370164660 con 0x7f437014f6e0 2023-11-15T03:07:52.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f436ffff700 1 --2- 172.21.15.121:0/4001856026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f437014f6e0 0x7f43700afc30 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 rx=0x7f4370065870 tx=0x7f4368009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:52.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f43677fe700 1 -- 172.21.15.121:0/4001856026 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4368003030 con 0x7f437014f6e0 2023-11-15T03:07:52.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f43677fe700 1 -- 172.21.15.121:0/4001856026 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4368003190 con 0x7f437014f6e0 2023-11-15T03:07:52.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f43677fe700 1 -- 172.21.15.121:0/4001856026 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4368001790 con 0x7f437014f6e0 2023-11-15T03:07:52.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f4375bdb700 1 -- 172.21.15.121:0/4001856026 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4370164940 con 0x7f437014f6e0 2023-11-15T03:07:52.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f4375bdb700 1 -- 172.21.15.121:0/4001856026 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4370163f50 con 0x7f437014f6e0 2023-11-15T03:07:52.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.380+0000 7f4375bdb700 1 -- 172.21.15.121:0/4001856026 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4370047060 con 0x7f437014f6e0 2023-11-15T03:07:52.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.384+0000 7f43677fe700 1 -- 172.21.15.121:0/4001856026 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4368001b40 con 0x7f437014f6e0 2023-11-15T03:07:52.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.384+0000 7f43677fe700 1 --2- 172.21.15.121:0/4001856026 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f435005a510 0x7f435005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:52.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.384+0000 7f43677fe700 1 -- 172.21.15.121:0/4001856026 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8330+0+0 (secure 0 0 0) 0x7f43680741e0 con 0x7f437014f6e0 2023-11-15T03:07:52.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.384+0000 7f436f7fe700 1 --2- 172.21.15.121:0/4001856026 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f435005a510 0x7f435005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:52.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.388+0000 7f436f7fe700 1 --2- 172.21.15.121:0/4001856026 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f435005a510 0x7f435005c9d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f4360006d80 tx=0x7f4360008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:52.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.392+0000 7f43677fe700 1 -- 172.21.15.121:0/4001856026 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f43680470e0 con 0x7f437014f6e0 2023-11-15T03:07:52.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:52.612+0000 7f4375bdb700 1 -- 172.21.15.121:0/4001856026 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f4370067030 con 0x7f437014f6e0 2023-11-15T03:07:53.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.184+0000 7f43677fe700 1 -- 172.21.15.121:0/4001856026 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v231) v1 ==== 101+0+0 (secure 0 0 0) 0x7f436801c020 con 0x7f437014f6e0 2023-11-15T03:07:53.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.188+0000 7f4375bdb700 1 -- 172.21.15.121:0/4001856026 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f435005a510 msgr2=0x7f435005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.188+0000 7f4375bdb700 1 --2- 172.21.15.121:0/4001856026 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f435005a510 0x7f435005c9d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f4360006d80 tx=0x7f4360008040).stop 2023-11-15T03:07:53.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.188+0000 7f4375bdb700 1 -- 172.21.15.121:0/4001856026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f437014f6e0 msgr2=0x7f43700afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.188+0000 7f4375bdb700 1 --2- 172.21.15.121:0/4001856026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f437014f6e0 0x7f43700afc30 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 rx=0x7f4370065870 tx=0x7f4368009040).stop 2023-11-15T03:07:53.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.188+0000 7f4375bdb700 1 -- 172.21.15.121:0/4001856026 shutdown_connections 2023-11-15T03:07:53.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.188+0000 7f4375bdb700 1 --2- 172.21.15.121:0/4001856026 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f435005a510 0x7f435005c9d0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.188+0000 7f4375bdb700 1 --2- 172.21.15.121:0/4001856026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f437014f6e0 0x7f43700afc30 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.188+0000 7f4375bdb700 1 --2- 172.21.15.121:0/4001856026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f437014f260 0x7f43700af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.188+0000 7f4375bdb700 1 --2- 172.21.15.121:0/4001856026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c004690 0x7f43700acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.188+0000 7f4375bdb700 1 -- 172.21.15.121:0/4001856026 >> 172.21.15.121:0/4001856026 conn(0x7f4370067a60 msgr2=0x7f4370152b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:53.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.188+0000 7f4375bdb700 1 -- 172.21.15.121:0/4001856026 shutdown_connections 2023-11-15T03:07:53.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.188+0000 7f4375bdb700 1 -- 172.21.15.121:0/4001856026 wait complete. 2023-11-15T03:07:53.200 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:07:53.222 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:07:53.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17e8332d00 1 Processor -- start 2023-11-15T03:07:53.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17e8332d00 1 -- start start 2023-11-15T03:07:53.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17e8332d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56436844ddc0 0x5643684c1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17e8332d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5643684c3650 0x564368401c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17e8332d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5643684c3a30 0x564368402100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17e8332d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56436837a220 con 0x56436844ddc0 2023-11-15T03:07:53.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17e8332d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643684025f0 con 0x5643684c3a30 2023-11-15T03:07:53.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17e8332d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564368430080 con 0x5643684c3650 2023-11-15T03:07:53.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17dc934700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56436844ddc0 0x5643684c1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17dc934700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56436844ddc0 0x5643684c1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38280/0 (socket says 172.21.15.121:38280) 2023-11-15T03:07:53.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17dc934700 1 -- 172.21.15.121:0/2876933380 learned_addr learned my addr 172.21.15.121:0/2876933380 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:53.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17d4133700 1 --2- 172.21.15.121:0/2876933380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5643684c3650 0x564368401c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17dd135700 1 --2- 172.21.15.121:0/2876933380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5643684c3a30 0x564368402100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17dc934700 1 -- 172.21.15.121:0/2876933380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5643684c3650 msgr2=0x564368401c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:53.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17dc934700 1 --2- 172.21.15.121:0/2876933380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5643684c3650 0x564368401c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17dc934700 1 -- 172.21.15.121:0/2876933380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5643684c3a30 msgr2=0x564368402100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:53.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17dc934700 1 --2- 172.21.15.121:0/2876933380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5643684c3a30 0x564368402100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17dc934700 1 -- 172.21.15.121:0/2876933380 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c4005cf0 con 0x56436844ddc0 2023-11-15T03:07:53.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17d4133700 1 --2- 172.21.15.121:0/2876933380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5643684c3650 0x564368401c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:07:53.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.268+0000 7f17dc934700 1 --2- 172.21.15.121:0/2876933380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56436844ddc0 0x5643684c1ea0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 rx=0x7f17cc006f60 tx=0x7f17cc006a10).ready entity=mon.0 client_cookie=39c69d8e560a3541 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:53.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.272+0000 7f17d77fe700 1 -- 172.21.15.121:0/2876933380 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc006110 con 0x56436844ddc0 2023-11-15T03:07:53.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.272+0000 7f17d77fe700 1 -- 172.21.15.121:0/2876933380 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17cc006270 con 0x56436844ddc0 2023-11-15T03:07:53.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.272+0000 7f17d77fe700 1 -- 172.21.15.121:0/2876933380 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc004750 con 0x56436844ddc0 2023-11-15T03:07:53.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.272+0000 7f17e8332d00 1 -- 172.21.15.121:0/2876933380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56436844ddc0 msgr2=0x5643684c1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.272+0000 7f17e8332d00 1 --2- 172.21.15.121:0/2876933380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56436844ddc0 0x5643684c1ea0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 rx=0x7f17cc006f60 tx=0x7f17cc006a10).stop 2023-11-15T03:07:53.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.272+0000 7f17e8332d00 1 -- 172.21.15.121:0/2876933380 shutdown_connections 2023-11-15T03:07:53.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.272+0000 7f17e8332d00 1 --2- 172.21.15.121:0/2876933380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5643684c3a30 0x564368402100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.272+0000 7f17e8332d00 1 --2- 172.21.15.121:0/2876933380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5643684c3650 0x564368401c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.272+0000 7f17e8332d00 1 --2- 172.21.15.121:0/2876933380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56436844ddc0 0x5643684c1ea0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.272+0000 7f17e8332d00 1 -- 172.21.15.121:0/2876933380 >> 172.21.15.121:0/2876933380 conn(0x56436844b8f0 msgr2=0x5643684c2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:53.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.272+0000 7f17e8332d00 1 -- 172.21.15.121:0/2876933380 shutdown_connections 2023-11-15T03:07:53.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.272+0000 7f17e8332d00 1 -- 172.21.15.121:0/2876933380 wait complete. 2023-11-15T03:07:53.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.272+0000 7f17e8332d00 1 Processor -- start 2023-11-15T03:07:53.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.272+0000 7f17e8332d00 1 -- start start 2023-11-15T03:07:53.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17e8332d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56436844ddc0 0x5643684bee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17e8332d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5643684c3650 0x564368445740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17e8332d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5643684c3a30 0x564368434760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17e8332d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643684342d0 con 0x56436844ddc0 2023-11-15T03:07:53.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17e8332d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564368433cc0 con 0x5643684c3650 2023-11-15T03:07:53.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17e8332d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564368448eb0 con 0x5643684c3a30 2023-11-15T03:07:53.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17dc934700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56436844ddc0 0x5643684bee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17dc934700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56436844ddc0 0x5643684bee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38286/0 (socket says 172.21.15.121:38286) 2023-11-15T03:07:53.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 learned_addr learned my addr 172.21.15.121:0/37227324 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:53.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17dd135700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5643684c3a30 0x564368434760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17d4133700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5643684c3650 0x564368445740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5643684c3a30 msgr2=0x564368434760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17dc934700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5643684c3a30 0x564368434760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5643684c3650 msgr2=0x564368445740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17dc934700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5643684c3650 0x564368445740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c8001e10 con 0x56436844ddc0 2023-11-15T03:07:53.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17d4133700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5643684c3650 0x564368445740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:07:53.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17dc934700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56436844ddc0 0x5643684bee70 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 rx=0x5643684e3450 tx=0x7f17cc009000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:53.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17d5ffb700 1 -- 172.21.15.121:0/37227324 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc013620 con 0x56436844ddc0 2023-11-15T03:07:53.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17d5ffb700 1 -- 172.21.15.121:0/37227324 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17cc013780 con 0x56436844ddc0 2023-11-15T03:07:53.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5643683a3770 con 0x56436844ddc0 2023-11-15T03:07:53.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17d5ffb700 1 -- 172.21.15.121:0/37227324 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc013b40 con 0x56436844ddc0 2023-11-15T03:07:53.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.276+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564368372a70 con 0x56436844ddc0 2023-11-15T03:07:53.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.280+0000 7f17d5ffb700 1 -- 172.21.15.121:0/37227324 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f17cc01f3b0 con 0x56436844ddc0 2023-11-15T03:07:53.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.280+0000 7f17d5ffb700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f17bc058ff0 0x7f17bc05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.280+0000 7f17d5ffb700 1 -- 172.21.15.121:0/37227324 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8330+0+0 (secure 0 0 0) 0x7f17cc074aa0 con 0x56436844ddc0 2023-11-15T03:07:53.294 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:07:53.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.280+0000 7f17d4133700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f17bc058ff0 0x7f17bc05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.280+0000 7f17e8332d00 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5643684318c0 0x5643684daa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.280+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x564368445ef0 con 0x5643684318c0 2023-11-15T03:07:53.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.280+0000 7f17dd135700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5643684318c0 0x5643684daa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.284+0000 7f17d4133700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f17bc058ff0 0x7f17bc05b4b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x5643684c3e10 tx=0x7f17c4015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:53.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.284+0000 7f17dd135700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5643684318c0 0x5643684daa90 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:53.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.284+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v118'281 uv274 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x5643684318c0 2023-11-15T03:07:53.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.284+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x5643684462a0 con 0x5643684318c0 2023-11-15T03:07:53.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.288+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v118'282 uv282 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x5643684318c0 2023-11-15T03:07:53.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.288+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x564368446680 con 0x5643684318c0 2023-11-15T03:07:53.298 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:07:53.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.292+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v118'283 uv283 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x5643684318c0 2023-11-15T03:07:53.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.292+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684f81c0 con 0x5643684318c0 2023-11-15T03:07:53.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.292+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v118'284 uv284 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x5643684318c0 2023-11-15T03:07:53.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.292+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684f8570 con 0x5643684318c0 2023-11-15T03:07:53.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.296+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v118'285 uv285 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x5643684318c0 2023-11-15T03:07:53.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.296+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684f71b0 con 0x5643684318c0 2023-11-15T03:07:53.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.296+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v118'286 uv286 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x5643684318c0 2023-11-15T03:07:53.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.296+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684f7560 con 0x5643684318c0 2023-11-15T03:07:53.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.300+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v118'287 uv280 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x5643684318c0 2023-11-15T03:07:53.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.300+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x5643684f7560 con 0x5643684318c0 2023-11-15T03:07:53.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.300+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v118'288 uv288 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x5643684318c0 2023-11-15T03:07:53.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.300+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684f7560 con 0x5643684318c0 2023-11-15T03:07:53.311 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:07:53.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.304+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v118'289 uv289 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x5643684318c0 2023-11-15T03:07:53.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.304+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684f7560 con 0x5643684318c0 2023-11-15T03:07:53.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.304+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v118'290 uv290 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x5643684318c0 2023-11-15T03:07:53.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.304+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684f7560 con 0x5643684318c0 2023-11-15T03:07:53.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.308+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v118'291 uv291 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x5643684318c0 2023-11-15T03:07:53.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.308+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684f7560 con 0x5643684318c0 2023-11-15T03:07:53.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.312+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v118'292 uv292 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x5643684318c0 2023-11-15T03:07:53.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.312+0000 7f17e8332d00 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5643684f7a70 0x5643684fa930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.312+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684fae20 con 0x5643684f7a70 2023-11-15T03:07:53.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.312+0000 7f17dc934700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5643684f7a70 0x5643684fa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.312+0000 7f17dc934700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5643684f7a70 0x5643684fa930 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:53.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.316+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v118'442 uv429 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.316+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x5643684fb1d0 con 0x5643684f7a70 2023-11-15T03:07:53.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.320+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v118'443 uv443 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.320+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684fb580 con 0x5643684f7a70 2023-11-15T03:07:53.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.324+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v118'444 uv444 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.331 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:07:53.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.324+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684fc940 con 0x5643684f7a70 2023-11-15T03:07:53.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.324+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v118'445 uv445 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.324+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684fccf0 con 0x5643684f7a70 2023-11-15T03:07:53.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.328+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v118'446 uv446 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.328+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684fb930 con 0x5643684f7a70 2023-11-15T03:07:53.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.332+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v118'447 uv447 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.332+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684fc220 con 0x5643684f7a70 2023-11-15T03:07:53.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.336+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v118'448 uv448 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.336+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684fd0a0 con 0x5643684f7a70 2023-11-15T03:07:53.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.336+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v118'449 uv435 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.336+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x5643684fd0a0 con 0x5643684f7a70 2023-11-15T03:07:53.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.340+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v118'450 uv450 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.340+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684fd0a0 con 0x5643684f7a70 2023-11-15T03:07:53.351 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:07:53.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.344+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v118'451 uv451 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.344+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684fe0b0 con 0x5643684f7a70 2023-11-15T03:07:53.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.344+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v118'452 uv452 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.344+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684fe0b0 con 0x5643684f7a70 2023-11-15T03:07:53.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.348+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v118'453 uv453 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.348+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684fd0a0 con 0x5643684f7a70 2023-11-15T03:07:53.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.352+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v118'454 uv454 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.352+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684fd0a0 con 0x5643684f7a70 2023-11-15T03:07:53.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.356+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v118'455 uv441 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.356+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x5643684fd0a0 con 0x5643684f7a70 2023-11-15T03:07:53.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.360+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v118'456 uv456 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.360+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684fd0a0 con 0x5643684f7a70 2023-11-15T03:07:53.369 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:07:53.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.360+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v118'457 uv457 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.360+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684fe0b0 con 0x5643684f7a70 2023-11-15T03:07:53.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.364+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v118'458 uv458 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.364+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684fe0b0 con 0x5643684f7a70 2023-11-15T03:07:53.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.364+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v118'459 uv459 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.364+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684fd0a0 con 0x5643684f7a70 2023-11-15T03:07:53.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.368+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v118'460 uv460 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.368+0000 7f17e8332d00 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5643684fd5b0 0x5643684ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.368+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643684ffe90 con 0x5643684fd5b0 2023-11-15T03:07:53.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.368+0000 7f17d4133700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5643684fd5b0 0x5643684ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.368+0000 7f17d4133700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5643684fd5b0 0x5643684ff9a0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:53.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.372+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v118'242 uv241 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x5643684fd5b0 2023-11-15T03:07:53.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.372+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x564368500300 con 0x5643684fd5b0 2023-11-15T03:07:53.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.372+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v118'243 uv243 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x5643684fd5b0 2023-11-15T03:07:53.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.372+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643685007f0 con 0x5643684fd5b0 2023-11-15T03:07:53.383 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:07:53.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.376+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v118'244 uv244 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x5643684fd5b0 2023-11-15T03:07:53.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.376+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643685010e0 con 0x5643684fd5b0 2023-11-15T03:07:53.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.380+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v118'245 uv245 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x5643684fd5b0 2023-11-15T03:07:53.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.380+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x564368501490 con 0x5643684fd5b0 2023-11-15T03:07:53.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.380+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v118'246 uv246 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x5643684fd5b0 2023-11-15T03:07:53.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.384+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x564368501890 con 0x5643684fd5b0 2023-11-15T03:07:53.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.384+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v118'247 uv240 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x5643684fd5b0 2023-11-15T03:07:53.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.384+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x564368501c40 con 0x5643684fd5b0 2023-11-15T03:07:53.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.388+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v118'248 uv248 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x5643684fd5b0 2023-11-15T03:07:53.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.388+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x564368501ff0 con 0x5643684fd5b0 2023-11-15T03:07:53.399 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:07:53.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.392+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v118'249 uv249 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x5643684fd5b0 2023-11-15T03:07:53.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.392+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x564368501ff0 con 0x5643684fd5b0 2023-11-15T03:07:53.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.396+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v118'250 uv250 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x5643684fd5b0 2023-11-15T03:07:53.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.396+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x564368501ff0 con 0x5643684fd5b0 2023-11-15T03:07:53.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.396+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v118'251 uv251 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x5643684fd5b0 2023-11-15T03:07:53.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.396+0000 7f17e8332d00 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x564368502500 0x5643685048f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.396+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x564368504f00 con 0x564368502500 2023-11-15T03:07:53.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.400+0000 7f17dd135700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x564368502500 0x5643685048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.400+0000 7f17dd135700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x564368502500 0x5643685048f0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:53.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.400+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v118'285 uv278 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.400+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x5643685052e0 con 0x564368502500 2023-11-15T03:07:53.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.404+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v118'286 uv286 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.404+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643685052e0 con 0x564368502500 2023-11-15T03:07:53.415 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:07:53.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.408+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v118'287 uv287 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.408+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x564368509370 con 0x564368502500 2023-11-15T03:07:53.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.412+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v118'288 uv288 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.412+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x564368509370 con 0x564368502500 2023-11-15T03:07:53.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.416+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v118'289 uv289 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.416+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643685052e0 con 0x564368502500 2023-11-15T03:07:53.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.416+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v118'290 uv284 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.416+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x5643685052e0 con 0x564368502500 2023-11-15T03:07:53.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.420+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v118'291 uv291 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.420+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643685052e0 con 0x564368502500 2023-11-15T03:07:53.431 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:07:53.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.424+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v118'292 uv292 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.424+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643685062f0 con 0x564368502500 2023-11-15T03:07:53.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.428+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v118'293 uv293 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.428+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643685062f0 con 0x564368502500 2023-11-15T03:07:53.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.428+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v118'294 uv294 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.428+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x5643685052e0 con 0x564368502500 2023-11-15T03:07:53.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.432+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v118'295 uv295 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.432+0000 7f17e8332d00 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x564368505960 0x564368507d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.432+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x564368508370 con 0x564368505960 2023-11-15T03:07:53.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.432+0000 7f17dc934700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x564368505960 0x564368507d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.432+0000 7f17dc934700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x564368505960 0x564368507d50 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:53.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.436+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v118'141 uv140 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x564368505960 2023-11-15T03:07:53.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.436+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x564368508750 con 0x564368505960 2023-11-15T03:07:53.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.440+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v118'142 uv142 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x564368505960 2023-11-15T03:07:53.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.440+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x564368508750 con 0x564368505960 2023-11-15T03:07:53.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.440+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v118'143 uv143 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x564368505960 2023-11-15T03:07:53.449 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:07:53.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.440+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x564368509760 con 0x564368505960 2023-11-15T03:07:53.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.444+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v118'144 uv144 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x564368505960 2023-11-15T03:07:53.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.444+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x564368509760 con 0x564368505960 2023-11-15T03:07:53.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.444+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v118'145 uv145 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x564368505960 2023-11-15T03:07:53.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.444+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x564368508750 con 0x564368505960 2023-11-15T03:07:53.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.448+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v118'146 uv146 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x564368505960 2023-11-15T03:07:53.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.448+0000 7f17e8332d00 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x564368508dd0 0x56436850b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.448+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850b8b0 con 0x564368508dd0 2023-11-15T03:07:53.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.448+0000 7f17d4133700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x564368508dd0 0x56436850b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.452+0000 7f17d4133700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x564368508dd0 0x56436850b1c0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:53.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.452+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v118'100 uv99 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.452+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x564368508dd0 2023-11-15T03:07:53.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.456+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v118'101 uv101 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.456+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x564368508dd0 2023-11-15T03:07:53.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.460+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v118'102 uv102 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.467 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:07:53.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.460+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x564368508dd0 2023-11-15T03:07:53.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.460+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v118'103 uv103 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.460+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x564368508dd0 2023-11-15T03:07:53.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.464+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v118'141 uv140 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.464+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x564368508dd0 2023-11-15T03:07:53.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.468+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v118'142 uv142 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.468+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x564368508dd0 2023-11-15T03:07:53.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.468+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v118'143 uv143 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.477 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:07:53.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.468+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850cca0 con 0x564368508dd0 2023-11-15T03:07:53.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.472+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v118'144 uv144 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.472+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850cca0 con 0x564368508dd0 2023-11-15T03:07:53.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.472+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v118'145 uv145 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.476+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x564368508dd0 2023-11-15T03:07:53.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.476+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v118'146 uv146 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.476+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x5643684f7a70 2023-11-15T03:07:53.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.480+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v118'260 uv254 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.480+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x5643684f7a70 2023-11-15T03:07:53.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.484+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v118'261 uv261 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.484+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x5643684f7a70 2023-11-15T03:07:53.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.488+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v118'262 uv262 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.495 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:07:53.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.488+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850cce0 con 0x5643684f7a70 2023-11-15T03:07:53.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.488+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v118'263 uv263 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.488+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850ccc0 con 0x5643684f7a70 2023-11-15T03:07:53.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.492+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v118'264 uv264 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.492+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x5643684f7a70 2023-11-15T03:07:53.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.496+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v118'265 uv265 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.496+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x5643684f7a70 2023-11-15T03:07:53.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.500+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v118'266 uv259 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.500+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x5643684f7a70 2023-11-15T03:07:53.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.504+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v118'267 uv267 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.504+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x5643684f7a70 2023-11-15T03:07:53.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.504+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v118'268 uv268 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.513 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:07:53.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.504+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x5643684f7a70 2023-11-15T03:07:53.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.508+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v118'269 uv269 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.508+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x5643684f7a70 2023-11-15T03:07:53.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.512+0000 7f17dc934700 1 -- 172.21.15.121:0/37227324 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v118'270 uv270 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17cc049920 con 0x5643684f7a70 2023-11-15T03:07:53.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.512+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x5643684fd5b0 2023-11-15T03:07:53.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.516+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v118'117 uv116 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x5643684fd5b0 2023-11-15T03:07:53.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.516+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x5643684fd5b0 2023-11-15T03:07:53.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.516+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v118'118 uv118 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x5643684fd5b0 2023-11-15T03:07:53.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.516+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x5643684fd5b0 2023-11-15T03:07:53.528 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:07:53.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.520+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v118'119 uv119 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x5643684fd5b0 2023-11-15T03:07:53.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.520+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x5643684fd5b0 2023-11-15T03:07:53.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.524+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v118'120 uv120 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x5643684fd5b0 2023-11-15T03:07:53.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.524+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850c090 con 0x5643684fd5b0 2023-11-15T03:07:53.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.524+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v118'121 uv121 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x5643684fd5b0 2023-11-15T03:07:53.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.524+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850c090 con 0x564368508dd0 2023-11-15T03:07:53.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.528+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v118'141 uv140 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.528+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x564368508dd0 2023-11-15T03:07:53.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.532+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v118'142 uv142 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.532+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x564368508dd0 2023-11-15T03:07:53.542 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:07:53.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.536+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v118'143 uv143 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.536+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x564368508dd0 2023-11-15T03:07:53.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.536+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v118'144 uv144 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.536+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x564368508dd0 2023-11-15T03:07:53.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.540+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v118'145 uv145 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.540+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x564368508dd0 2023-11-15T03:07:53.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.544+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v118'146 uv146 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.544+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x564368508dd0 2023-11-15T03:07:53.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.544+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v118'118 uv117 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.544+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x564368508dd0 2023-11-15T03:07:53.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.548+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v118'119 uv119 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.548+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x564368508dd0 2023-11-15T03:07:53.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.552+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v118'120 uv120 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.559 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:07:53.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.552+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x564368508dd0 2023-11-15T03:07:53.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.556+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v118'121 uv121 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.556+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850c0e0 con 0x564368508dd0 2023-11-15T03:07:53.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.556+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v118'122 uv122 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.556+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850c0e0 con 0x564368508dd0 2023-11-15T03:07:53.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.560+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v118'116 uv115 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.560+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x56436850bcd0 con 0x564368508dd0 2023-11-15T03:07:53.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.564+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v118'117 uv117 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.564+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850bc90 con 0x564368508dd0 2023-11-15T03:07:53.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.568+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v118'118 uv118 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.574 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:07:53.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.568+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850c0d0 con 0x564368508dd0 2023-11-15T03:07:53.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.568+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v118'119 uv119 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.568+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850eb90 con 0x564368508dd0 2023-11-15T03:07:53.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.576+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v118'120 uv120 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.576+0000 7f17e8332d00 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56436850c2f0 0x56436850f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.576+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x56436850c2f0 2023-11-15T03:07:53.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.576+0000 7f17dd135700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56436850c2f0 0x56436850f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.576+0000 7f17dd135700 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56436850c2f0 0x56436850f1b0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:53.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.580+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v118'94 uv93 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x56436850c2f0 2023-11-15T03:07:53.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.580+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x56436850c2f0 2023-11-15T03:07:53.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.584+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v118'95 uv95 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x56436850c2f0 2023-11-15T03:07:53.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.584+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x56436850c2f0 2023-11-15T03:07:53.593 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:07:53.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.584+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v118'96 uv96 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x56436850c2f0 2023-11-15T03:07:53.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.584+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x56436850c2f0 2023-11-15T03:07:53.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.588+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v118'97 uv97 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x56436850c2f0 2023-11-15T03:07:53.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.588+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x564368508dd0 2023-11-15T03:07:53.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.592+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v118'141 uv140 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.592+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x564368508dd0 2023-11-15T03:07:53.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.596+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v118'142 uv142 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.596+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x564368508dd0 2023-11-15T03:07:53.604 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:07:53.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.596+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v118'143 uv143 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.596+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x564368508dd0 2023-11-15T03:07:53.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.600+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v118'144 uv144 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.600+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x564368508dd0 2023-11-15T03:07:53.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.604+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v118'145 uv145 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.604+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x564368508dd0 2023-11-15T03:07:53.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.604+0000 7f17d4133700 1 -- 172.21.15.121:0/37227324 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v118'146 uv146 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17c4007a10 con 0x564368508dd0 2023-11-15T03:07:53.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.604+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x564368502500 2023-11-15T03:07:53.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.608+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v118'120 uv119 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.608+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x564368502500 2023-11-15T03:07:53.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.612+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v118'121 uv121 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.612+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x564368502500 2023-11-15T03:07:53.621 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:07:53.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.612+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v118'122 uv122 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.612+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x564368502500 2023-11-15T03:07:53.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.616+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v118'123 uv123 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.616+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850f6a0 con 0x564368502500 2023-11-15T03:07:53.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.620+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v118'124 uv124 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.620+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850f6a0 con 0x564368502500 2023-11-15T03:07:53.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.624+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v118'281 uv273 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.624+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x56436850f6a0 con 0x564368502500 2023-11-15T03:07:53.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.624+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v118'282 uv282 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.624+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850f6a0 con 0x564368502500 2023-11-15T03:07:53.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.628+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v118'283 uv283 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.636 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:07:53.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.628+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x564368502500 2023-11-15T03:07:53.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.632+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v118'284 uv284 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.632+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x564368502500 2023-11-15T03:07:53.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.636+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v118'285 uv285 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.636+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x564368502500 2023-11-15T03:07:53.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.636+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v118'286 uv280 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.636+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x564368502500 2023-11-15T03:07:53.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.640+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v118'287 uv287 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.640+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x564368502500 2023-11-15T03:07:53.650 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:07:53.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.644+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v118'288 uv288 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.644+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x564368502500 2023-11-15T03:07:53.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.644+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v118'289 uv289 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.644+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850e6e0 con 0x564368502500 2023-11-15T03:07:53.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.648+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v118'290 uv290 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.648+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850f750 con 0x564368502500 2023-11-15T03:07:53.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v118'291 uv291 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x56436850fbf0 con 0x564368502500 2023-11-15T03:07:53.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v118'292 uv292 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f17d8012040 con 0x564368502500 2023-11-15T03:07:53.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5643684fd5b0 msgr2=0x5643684ff9a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17e8332d00 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5643684fd5b0 0x5643684ff9a0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5643684318c0 msgr2=0x5643684daa90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17e8332d00 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5643684318c0 0x5643684daa90 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x564368505960 msgr2=0x564368507d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17e8332d00 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x564368505960 0x564368507d50 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5643684f7a70 msgr2=0x5643684fa930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17e8332d00 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5643684f7a70 0x5643684fa930 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56436850c2f0 msgr2=0x56436850f1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17e8332d00 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x56436850c2f0 0x56436850f1b0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x564368502500 msgr2=0x5643685048f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17e8332d00 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x564368502500 0x5643685048f0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x564368508dd0 msgr2=0x56436850b1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17e8332d00 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x564368508dd0 0x56436850b1c0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 reap_dead start 2023-11-15T03:07:53.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 reap_dead start 2023-11-15T03:07:53.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.652+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 reap_dead start 2023-11-15T03:07:53.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.656+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 reap_dead start 2023-11-15T03:07:53.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.656+0000 7f17dd135700 1 -- 172.21.15.121:0/37227324 reap_dead start 2023-11-15T03:07:53.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.656+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f17bc058ff0 msgr2=0x7f17bc05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.656+0000 7f17e8332d00 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f17bc058ff0 0x7f17bc05b4b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x5643684c3e10 tx=0x7f17c4015000).stop 2023-11-15T03:07:53.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.656+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56436844ddc0 msgr2=0x5643684bee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.656+0000 7f17e8332d00 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56436844ddc0 0x5643684bee70 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 rx=0x5643684e3450 tx=0x7f17cc009000).stop 2023-11-15T03:07:53.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.656+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 shutdown_connections 2023-11-15T03:07:53.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.656+0000 7f17e8332d00 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f17bc058ff0 0x7f17bc05b4b0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.656+0000 7f17e8332d00 1 --2- 172.21.15.121:0/37227324 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56436844ddc0 0x5643684bee70 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.656+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 >> 172.21.15.121:0/37227324 conn(0x56436844b8f0 msgr2=0x564368401dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:53.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.656+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 shutdown_connections 2023-11-15T03:07:53.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.656+0000 7f17e8332d00 1 -- 172.21.15.121:0/37227324 wait complete. 2023-11-15T03:07:53.671 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 200.00000000 0 2023-11-15T03:07:53.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.764+0000 7f646bb41d00 1 Processor -- start 2023-11-15T03:07:53.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.764+0000 7f646bb41d00 1 -- start start 2023-11-15T03:07:53.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.764+0000 7f646bb41d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f67c64a7e0 0x55f67c706e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.764+0000 7f646bb41d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f67c708650 0x55f67c679c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.764+0000 7f646bb41d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f67c708a30 0x55f67c678380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.764+0000 7f646bb41d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67c5bf220 con 0x55f67c708650 2023-11-15T03:07:53.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.764+0000 7f646bb41d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67c707b20 con 0x55f67c708a30 2023-11-15T03:07:53.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.764+0000 7f646bb41d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f67c646060 con 0x55f67c64a7e0 2023-11-15T03:07:53.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.764+0000 7f6460c44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f67c708650 0x55f67c679c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.764+0000 7f6460c44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f67c708650 0x55f67c679c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38308/0 (socket says 172.21.15.121:38308) 2023-11-15T03:07:53.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.764+0000 7f6460c44700 1 -- 172.21.15.121:0/3260733511 learned_addr learned my addr 172.21.15.121:0/3260733511 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:53.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.764+0000 7f6469445700 1 --2- 172.21.15.121:0/3260733511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f67c64a7e0 0x55f67c706e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.764+0000 7f6469c46700 1 --2- 172.21.15.121:0/3260733511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f67c708a30 0x55f67c678380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.764+0000 7f6460c44700 1 -- 172.21.15.121:0/3260733511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f67c64a7e0 msgr2=0x55f67c706e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:53.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.764+0000 7f6460c44700 1 --2- 172.21.15.121:0/3260733511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f67c64a7e0 0x55f67c706e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.764+0000 7f6460c44700 1 -- 172.21.15.121:0/3260733511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f67c708a30 msgr2=0x55f67c678380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:07:53.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.768+0000 7f6460c44700 1 --2- 172.21.15.121:0/3260733511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f67c708a30 0x55f67c678380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.768+0000 7f6460c44700 1 -- 172.21.15.121:0/3260733511 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c00a180 con 0x55f67c708650 2023-11-15T03:07:53.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.768+0000 7f6469c46700 1 --2- 172.21.15.121:0/3260733511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f67c708a30 0x55f67c678380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:07:53.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.768+0000 7f6460c44700 1 --2- 172.21.15.121:0/3260733511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f67c708650 0x55f67c679c80 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 rx=0x7f6454001dc0 tx=0x7f6454001890).ready entity=mon.0 client_cookie=d85b22632cf449ae server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:53.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.768+0000 7f6463fff700 1 -- 172.21.15.121:0/3260733511 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645400bbd0 con 0x55f67c708650 2023-11-15T03:07:53.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.768+0000 7f6463fff700 1 -- 172.21.15.121:0/3260733511 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6454008070 con 0x55f67c708650 2023-11-15T03:07:53.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.768+0000 7f6463fff700 1 -- 172.21.15.121:0/3260733511 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645400bd70 con 0x55f67c708650 2023-11-15T03:07:53.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.768+0000 7f646bb41d00 1 -- 172.21.15.121:0/3260733511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f67c708650 msgr2=0x55f67c679c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.768+0000 7f646bb41d00 1 --2- 172.21.15.121:0/3260733511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f67c708650 0x55f67c679c80 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 rx=0x7f6454001dc0 tx=0x7f6454001890).stop 2023-11-15T03:07:53.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.768+0000 7f646bb41d00 1 -- 172.21.15.121:0/3260733511 shutdown_connections 2023-11-15T03:07:53.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.768+0000 7f646bb41d00 1 --2- 172.21.15.121:0/3260733511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f67c708a30 0x55f67c678380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.768+0000 7f646bb41d00 1 --2- 172.21.15.121:0/3260733511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f67c708650 0x55f67c679c80 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.768+0000 7f646bb41d00 1 --2- 172.21.15.121:0/3260733511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f67c64a7e0 0x55f67c706e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.768+0000 7f646bb41d00 1 -- 172.21.15.121:0/3260733511 >> 172.21.15.121:0/3260733511 conn(0x55f67c6908f0 msgr2=0x55f67c68fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:53.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.768+0000 7f646bb41d00 1 -- 172.21.15.121:0/3260733511 shutdown_connections 2023-11-15T03:07:53.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.768+0000 7f646bb41d00 1 -- 172.21.15.121:0/3260733511 wait complete. 2023-11-15T03:07:53.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.768+0000 7f646bb41d00 1 Processor -- start 2023-11-15T03:07:53.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.768+0000 7f646bb41d00 1 -- start start 2023-11-15T03:07:53.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f646bb41d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f67c64a7e0 0x55f67c71d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f646bb41d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f67c708650 0x55f67c703e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f646bb41d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f67c708a30 0x55f67c677130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f646bb41d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67c702b10 con 0x55f67c708650 2023-11-15T03:07:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f646bb41d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f67c701f00 con 0x55f67c64a7e0 2023-11-15T03:07:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f646bb41d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f67c7021e0 con 0x55f67c708a30 2023-11-15T03:07:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f6460c44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f67c708650 0x55f67c703e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f6460c44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f67c708650 0x55f67c703e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38324/0 (socket says 172.21.15.121:38324) 2023-11-15T03:07:53.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f6460c44700 1 -- 172.21.15.121:0/1942358070 learned_addr learned my addr 172.21.15.121:0/1942358070 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:53.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f6469445700 1 --2- 172.21.15.121:0/1942358070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f67c64a7e0 0x55f67c71d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f6469c46700 1 --2- 172.21.15.121:0/1942358070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f67c708a30 0x55f67c677130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f6460c44700 1 -- 172.21.15.121:0/1942358070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f67c708a30 msgr2=0x55f67c677130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f6460c44700 1 --2- 172.21.15.121:0/1942358070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f67c708a30 0x55f67c677130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f6460c44700 1 -- 172.21.15.121:0/1942358070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f67c64a7e0 msgr2=0x55f67c71d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f6460c44700 1 --2- 172.21.15.121:0/1942358070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f67c64a7e0 0x55f67c71d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f6460c44700 1 -- 172.21.15.121:0/1942358070 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645400a040 con 0x55f67c708650 2023-11-15T03:07:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f6469c46700 1 --2- 172.21.15.121:0/1942358070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f67c708a30 0x55f67c677130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:07:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f6460c44700 1 --2- 172.21.15.121:0/1942358070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f67c708650 0x55f67c703e10 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 rx=0x7f645400f000 tx=0x7f6454015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f64627fc700 1 -- 172.21.15.121:0/1942358070 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454001ea0 con 0x55f67c708650 2023-11-15T03:07:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f64627fc700 1 -- 172.21.15.121:0/1942358070 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645401c400 con 0x55f67c708650 2023-11-15T03:07:53.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f646bb41d00 1 -- 172.21.15.121:0/1942358070 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f67c5e8770 con 0x55f67c708650 2023-11-15T03:07:53.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f64627fc700 1 -- 172.21.15.121:0/1942358070 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645401c7c0 con 0x55f67c708650 2023-11-15T03:07:53.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.772+0000 7f646bb41d00 1 -- 172.21.15.121:0/1942358070 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f67c5b7a70 con 0x55f67c708650 2023-11-15T03:07:53.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.776+0000 7f64627fc700 1 -- 172.21.15.121:0/1942358070 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6454017070 con 0x55f67c708650 2023-11-15T03:07:53.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.776+0000 7f64627fc700 1 --2- 172.21.15.121:0/1942358070 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f644c058ff0 0x7f644c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.776+0000 7f64627fc700 1 -- 172.21.15.121:0/1942358070 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8330+0+0 (secure 0 0 0) 0x7f6454075410 con 0x55f67c708650 2023-11-15T03:07:53.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.776+0000 7f6469445700 1 --2- 172.21.15.121:0/1942358070 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f644c058ff0 0x7f644c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.776+0000 7f646bb41d00 1 --2- 172.21.15.121:0/1942358070 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55f67c7034b0 0x55f67c701190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.776+0000 7f646bb41d00 1 -- 172.21.15.121:0/1942358070 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x55f67c701680 con 0x55f67c7034b0 2023-11-15T03:07:53.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.776+0000 7f6469c46700 1 --2- 172.21.15.121:0/1942358070 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55f67c7034b0 0x55f67c701190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.776+0000 7f6469445700 1 --2- 172.21.15.121:0/1942358070 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f644c058ff0 0x7f644c05b4b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f645c006fd0 tx=0x7f645c015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:53.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.780+0000 7f6469c46700 1 --2- 172.21.15.121:0/1942358070 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55f67c7034b0 0x55f67c701190 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:53.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.780+0000 7f6469c46700 1 -- 172.21.15.121:0/1942358070 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(1 200.00000000 [truncate 0] v118'252 uv252 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6464009890 con 0x55f67c7034b0 2023-11-15T03:07:53.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.780+0000 7f646bb41d00 1 -- 172.21.15.121:0/1942358070 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55f67c7034b0 msgr2=0x55f67c701190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.780+0000 7f646bb41d00 1 --2- 172.21.15.121:0/1942358070 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55f67c7034b0 0x55f67c701190 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.784+0000 7f646bb41d00 1 -- 172.21.15.121:0/1942358070 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f644c058ff0 msgr2=0x7f644c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.784+0000 7f646bb41d00 1 --2- 172.21.15.121:0/1942358070 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f644c058ff0 0x7f644c05b4b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f645c006fd0 tx=0x7f645c015000).stop 2023-11-15T03:07:53.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.784+0000 7f646bb41d00 1 -- 172.21.15.121:0/1942358070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f67c708650 msgr2=0x55f67c703e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.784+0000 7f646bb41d00 1 --2- 172.21.15.121:0/1942358070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f67c708650 0x55f67c703e10 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 rx=0x7f645400f000 tx=0x7f6454015000).stop 2023-11-15T03:07:53.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.784+0000 7f6469c46700 1 -- 172.21.15.121:0/1942358070 reap_dead start 2023-11-15T03:07:53.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.784+0000 7f646bb41d00 1 -- 172.21.15.121:0/1942358070 shutdown_connections 2023-11-15T03:07:53.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.784+0000 7f646bb41d00 1 -- 172.21.15.121:0/1942358070 >> 172.21.15.121:0/1942358070 conn(0x55f67c6908f0 msgr2=0x55f67c678380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:53.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.784+0000 7f646bb41d00 1 -- 172.21.15.121:0/1942358070 shutdown_connections 2023-11-15T03:07:53.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.784+0000 7f646bb41d00 1 -- 172.21.15.121:0/1942358070 wait complete. 2023-11-15T03:07:53.799 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:07:53.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.956+0000 7f21d49f9700 1 Processor -- start 2023-11-15T03:07:53.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.956+0000 7f21d49f9700 1 -- start start 2023-11-15T03:07:53.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.960+0000 7f21d49f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d00acb60 0x7f21d00acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.960+0000 7f21d49f9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d00c7e90 con 0x7f21d0065680 2023-11-15T03:07:53.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.960+0000 7f21d49f9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d00ad4d0 con 0x7f21d014e380 2023-11-15T03:07:53.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.960+0000 7f21d49f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d00c8320 con 0x7f21d00acb60 2023-11-15T03:07:53.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.960+0000 7f21cffff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f21d0065680 0x7f21d0065b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60426/0 (socket says 172.21.15.121:60426) 2023-11-15T03:07:53.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.960+0000 7f21cffff700 1 -- 172.21.15.121:0/1029505546 learned_addr learned my addr 172.21.15.121:0/1029505546 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:53.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.960+0000 7f21ceffd700 1 --2- 172.21.15.121:0/1029505546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d00acb60 0x7f21d00acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.960+0000 7f21cf7fe700 1 --1- 172.21.15.121:0/1029505546 >> v1:172.21.15.190:6789/0 conn(0x7f21d014e380 0x7f21d00c5b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:55650/0 (socket says 172.21.15.121:55650) 2023-11-15T03:07:53.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.960+0000 7f21ceffd700 1 -- 172.21.15.121:0/1029505546 >> v1:172.21.15.190:6789/0 conn(0x7f21d014e380 legacy=0x7f21d00c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.960+0000 7f21ceffd700 1 -- 172.21.15.121:0/1029505546 >> v1:172.21.15.121:6789/0 conn(0x7f21d0065680 legacy=0x7f21d0065b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.960+0000 7f21ce7fc700 1 -- 172.21.15.121:0/1029505546 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3347296607 0 0) 0x7f21d00c7e90 con 0x7f21d0065680 2023-11-15T03:07:53.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.960+0000 7f21ceffd700 1 -- 172.21.15.121:0/1029505546 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d014e760 con 0x7f21d00acb60 2023-11-15T03:07:53.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.960+0000 7f21ceffd700 1 --2- 172.21.15.121:0/1029505546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d00acb60 0x7f21d00acfe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f21c4009710 tx=0x7f21c4009060).ready entity=mon.2 client_cookie=85bd6e79184c2eb4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:53.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.960+0000 7f21ce7fc700 1 -- 172.21.15.121:0/1029505546 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c400f510 con 0x7f21d00acb60 2023-11-15T03:07:53.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.964+0000 7f21ce7fc700 1 -- 172.21.15.121:0/1029505546 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c400f670 con 0x7f21d00acb60 2023-11-15T03:07:53.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.964+0000 7f21ce7fc700 1 -- 172.21.15.121:0/1029505546 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c400f9f0 con 0x7f21d00acb60 2023-11-15T03:07:53.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.964+0000 7f21d49f9700 1 -- 172.21.15.121:0/1029505546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d00acb60 msgr2=0x7f21d00acfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.964+0000 7f21d49f9700 1 --2- 172.21.15.121:0/1029505546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d00acb60 0x7f21d00acfe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f21c4009710 tx=0x7f21c4009060).stop 2023-11-15T03:07:53.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.964+0000 7f21d49f9700 1 -- 172.21.15.121:0/1029505546 shutdown_connections 2023-11-15T03:07:53.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.964+0000 7f21d49f9700 1 --2- 172.21.15.121:0/1029505546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d00acb60 0x7f21d00acfe0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.964+0000 7f21d49f9700 1 -- 172.21.15.121:0/1029505546 >> 172.21.15.121:0/1029505546 conn(0x7f21d0067a60 msgr2=0x7f21d0067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:53.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.964+0000 7f21d49f9700 1 -- 172.21.15.121:0/1029505546 shutdown_connections 2023-11-15T03:07:53.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.964+0000 7f21d49f9700 1 -- 172.21.15.121:0/1029505546 wait complete. 2023-11-15T03:07:53.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.964+0000 7f21d49f9700 1 Processor -- start 2023-11-15T03:07:53.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.964+0000 7f21d49f9700 1 -- start start 2023-11-15T03:07:53.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21d49f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21d0065680 0x7f21d0164720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21d49f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21d00acb60 0x7f21d014d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21d49f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d014e380 0x7f21d00c3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21d49f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d0164da0 con 0x7f21d00acb60 2023-11-15T03:07:53.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21d49f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d0165080 con 0x7f21d0065680 2023-11-15T03:07:53.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21d49f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d0165360 con 0x7f21d014e380 2023-11-15T03:07:53.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21ceffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21d00acb60 0x7f21d014d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21ceffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21d00acb60 0x7f21d014d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38334/0 (socket says 172.21.15.121:38334) 2023-11-15T03:07:53.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21ceffd700 1 -- 172.21.15.121:0/1068730341 learned_addr learned my addr 172.21.15.121:0/1068730341 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:53.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21cffff700 1 --2- 172.21.15.121:0/1068730341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d014e380 0x7f21d00c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21cf7fe700 1 --2- 172.21.15.121:0/1068730341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21d0065680 0x7f21d0164720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21ceffd700 1 -- 172.21.15.121:0/1068730341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d014e380 msgr2=0x7f21d00c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21ceffd700 1 --2- 172.21.15.121:0/1068730341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d014e380 0x7f21d00c3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21ceffd700 1 -- 172.21.15.121:0/1068730341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21d0065680 msgr2=0x7f21d0164720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:53.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21ceffd700 1 --2- 172.21.15.121:0/1068730341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21d0065680 0x7f21d0164720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:53.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21ceffd700 1 -- 172.21.15.121:0/1068730341 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c40092a0 con 0x7f21d00acb60 2023-11-15T03:07:53.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21cf7fe700 1 --2- 172.21.15.121:0/1068730341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21d0065680 0x7f21d0164720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:07:53.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21ceffd700 1 --2- 172.21.15.121:0/1068730341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21d00acb60 0x7f21d014d250 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 rx=0x7f21d0064eb0 tx=0x7f21c4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:53.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21b77fe700 1 -- 172.21.15.121:0/1068730341 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c400f510 con 0x7f21d00acb60 2023-11-15T03:07:53.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21b77fe700 1 -- 172.21.15.121:0/1068730341 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c4000d10 con 0x7f21d00acb60 2023-11-15T03:07:53.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21d49f9700 1 -- 172.21.15.121:0/1068730341 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d0165640 con 0x7f21d00acb60 2023-11-15T03:07:53.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21b77fe700 1 -- 172.21.15.121:0/1068730341 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c401b620 con 0x7f21d00acb60 2023-11-15T03:07:53.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.968+0000 7f21d49f9700 1 -- 172.21.15.121:0/1068730341 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d014d740 con 0x7f21d00acb60 2023-11-15T03:07:53.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.972+0000 7f21d49f9700 1 -- 172.21.15.121:0/1068730341 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d006bcf0 con 0x7f21d00acb60 2023-11-15T03:07:53.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.972+0000 7f21b77fe700 1 -- 172.21.15.121:0/1068730341 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f21c401ba50 con 0x7f21d00acb60 2023-11-15T03:07:53.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.976+0000 7f21b77fe700 1 --2- 172.21.15.121:0/1068730341 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f21bc05a4e0 0x7f21bc05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:53.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.976+0000 7f21b77fe700 1 -- 172.21.15.121:0/1068730341 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8330+0+0 (secure 0 0 0) 0x7f21c40768b0 con 0x7f21d00acb60 2023-11-15T03:07:53.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.976+0000 7f21cf7fe700 1 --2- 172.21.15.121:0/1068730341 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f21bc05a4e0 0x7f21bc05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:53.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.976+0000 7f21cf7fe700 1 --2- 172.21.15.121:0/1068730341 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f21bc05a4e0 0x7f21bc05c9a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7f21c0007fd0 tx=0x7f21c0007c10).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:53.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:53.980+0000 7f21b77fe700 1 -- 172.21.15.121:0/1068730341 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f21c4049790 con 0x7f21d00acb60 2023-11-15T03:07:54.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:54.188+0000 7f21d49f9700 1 -- 172.21.15.121:0/1068730341 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f21d00c62e0 con 0x7f21d00acb60 2023-11-15T03:07:55.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.200+0000 7f21b77fe700 1 -- 172.21.15.121:0/1068730341 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v232) v1 ==== 162+0+0 (secure 0 0 0) 0x7f21c400f9f0 con 0x7f21d00acb60 2023-11-15T03:07:55.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.204+0000 7f21d49f9700 1 -- 172.21.15.121:0/1068730341 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f21bc05a4e0 msgr2=0x7f21bc05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:55.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.204+0000 7f21d49f9700 1 --2- 172.21.15.121:0/1068730341 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f21bc05a4e0 0x7f21bc05c9a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7f21c0007fd0 tx=0x7f21c0007c10).stop 2023-11-15T03:07:55.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.204+0000 7f21d49f9700 1 -- 172.21.15.121:0/1068730341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21d00acb60 msgr2=0x7f21d014d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:55.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.204+0000 7f21d49f9700 1 --2- 172.21.15.121:0/1068730341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21d00acb60 0x7f21d014d250 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 rx=0x7f21d0064eb0 tx=0x7f21c4015000).stop 2023-11-15T03:07:55.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.204+0000 7f21d49f9700 1 -- 172.21.15.121:0/1068730341 shutdown_connections 2023-11-15T03:07:55.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.204+0000 7f21d49f9700 1 --2- 172.21.15.121:0/1068730341 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f21bc05a4e0 0x7f21bc05c9a0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:55.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.204+0000 7f21d49f9700 1 --2- 172.21.15.121:0/1068730341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d014e380 0x7f21d00c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:55.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.204+0000 7f21d49f9700 1 --2- 172.21.15.121:0/1068730341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21d00acb60 0x7f21d014d250 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:55.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.204+0000 7f21d49f9700 1 --2- 172.21.15.121:0/1068730341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21d0065680 0x7f21d0164720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:55.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.204+0000 7f21d49f9700 1 -- 172.21.15.121:0/1068730341 >> 172.21.15.121:0/1068730341 conn(0x7f21d0067a60 msgr2=0x7f21d0149e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:55.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.204+0000 7f21d49f9700 1 -- 172.21.15.121:0/1068730341 shutdown_connections 2023-11-15T03:07:55.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.204+0000 7f21d49f9700 1 -- 172.21.15.121:0/1068730341 wait complete. 2023-11-15T03:07:55.215 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:07:55.221 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:07:55.212+0000 7f5346291700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2023-11-15T03:07:55.236 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:07:55.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.392+0000 7efbff49b700 1 Processor -- start 2023-11-15T03:07:55.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.396+0000 7efbff49b700 1 -- start start 2023-11-15T03:07:55.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.396+0000 7efbff49b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efbf8153d20 0x7efbf8150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:55.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.396+0000 7efbff49b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbf80c6350 con 0x7efbf8150870 2023-11-15T03:07:55.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.396+0000 7efbff49b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbf81541b0 con 0x7efbf80c38a0 2023-11-15T03:07:55.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.396+0000 7efbff49b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf8154490 con 0x7efbf8153d20 2023-11-15T03:07:55.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.396+0000 7efbfd237700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7efbf8150870 0x7efbf80c3190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56892/0 (socket says 172.21.15.121:56892) 2023-11-15T03:07:55.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.400+0000 7efbfd237700 1 -- 172.21.15.121:0/376872635 learned_addr learned my addr 172.21.15.121:0/376872635 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:07:55.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.400+0000 7efbfca36700 1 --2- 172.21.15.121:0/376872635 >> v2:172.21.15.121:3300/0 conn(0x7efbf8153d20 0x7efbf8150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:55.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.400+0000 7efbeffff700 1 -- 172.21.15.121:0/376872635 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437542827 0 0) 0x7efbf80c6350 con 0x7efbf8150870 2023-11-15T03:07:55.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.400+0000 7efbeffff700 1 -- 172.21.15.121:0/376872635 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbe40033f0 con 0x7efbf8150870 2023-11-15T03:07:55.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.400+0000 7efbfca36700 1 -- 172.21.15.121:0/376872635 >> v1:172.21.15.190:6789/0 conn(0x7efbf80c38a0 legacy=0x7efbf8151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:55.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.400+0000 7efbfca36700 1 -- 172.21.15.121:0/376872635 >> v1:172.21.15.121:6789/0 conn(0x7efbf8150870 legacy=0x7efbf80c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:55.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.400+0000 7efbfca36700 1 -- 172.21.15.121:0/376872635 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf8154770 con 0x7efbf8153d20 2023-11-15T03:07:55.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.400+0000 7efbfca36700 1 --2- 172.21.15.121:0/376872635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efbf8153d20 0x7efbf8150c50 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 rx=0x7efbe800a900 tx=0x7efbe800a200).ready entity=mon.0 client_cookie=2bf0dc5ff465841f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:55.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.400+0000 7efbeffff700 1 -- 172.21.15.121:0/376872635 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe800f750 con 0x7efbf8153d20 2023-11-15T03:07:55.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.400+0000 7efbeffff700 1 -- 172.21.15.121:0/376872635 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe800f8b0 con 0x7efbf8153d20 2023-11-15T03:07:55.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.400+0000 7efbeffff700 1 -- 172.21.15.121:0/376872635 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe800fc30 con 0x7efbf8153d20 2023-11-15T03:07:55.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.400+0000 7efbff49b700 1 -- 172.21.15.121:0/376872635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efbf8153d20 msgr2=0x7efbf8150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:55.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.400+0000 7efbff49b700 1 --2- 172.21.15.121:0/376872635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efbf8153d20 0x7efbf8150c50 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 rx=0x7efbe800a900 tx=0x7efbe800a200).stop 2023-11-15T03:07:55.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.400+0000 7efbff49b700 1 -- 172.21.15.121:0/376872635 shutdown_connections 2023-11-15T03:07:55.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.400+0000 7efbff49b700 1 --2- 172.21.15.121:0/376872635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efbf8153d20 0x7efbf8150c50 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:55.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.400+0000 7efbff49b700 1 -- 172.21.15.121:0/376872635 >> 172.21.15.121:0/376872635 conn(0x7efbf8062190 msgr2=0x7efbf80625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:55.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.400+0000 7efbff49b700 1 -- 172.21.15.121:0/376872635 shutdown_connections 2023-11-15T03:07:55.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.400+0000 7efbff49b700 1 -- 172.21.15.121:0/376872635 wait complete. 2023-11-15T03:07:55.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.404+0000 7efbff49b700 1 Processor -- start 2023-11-15T03:07:55.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.404+0000 7efbff49b700 1 -- start start 2023-11-15T03:07:55.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.404+0000 7efbff49b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efbf80c38a0 0x7efbf814faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:55.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.404+0000 7efbff49b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efbf8150870 0x7efbf80ab350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:55.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.404+0000 7efbff49b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efbf80ab840 0x7efbf80abca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:55.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.404+0000 7efbff49b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf815abb0 con 0x7efbf80c38a0 2023-11-15T03:07:55.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.404+0000 7efbff49b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf8159d50 con 0x7efbf8150870 2023-11-15T03:07:55.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.404+0000 7efbff49b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf80ac190 con 0x7efbf80ab840 2023-11-15T03:07:55.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.404+0000 7efbfd237700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efbf80c38a0 0x7efbf814faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:55.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.404+0000 7efbfd237700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efbf80c38a0 0x7efbf814faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:57554/0 (socket says 172.21.15.121:57554) 2023-11-15T03:07:55.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.404+0000 7efbfd237700 1 -- 172.21.15.121:0/2461990085 learned_addr learned my addr 172.21.15.121:0/2461990085 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:07:55.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.408+0000 7efbfda38700 1 --2- 172.21.15.121:0/2461990085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efbf80ab840 0x7efbf80abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:55.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.408+0000 7efbfd237700 1 -- 172.21.15.121:0/2461990085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efbf80ab840 msgr2=0x7efbf80abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:55.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.408+0000 7efbfca36700 1 --2- 172.21.15.121:0/2461990085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efbf8150870 0x7efbf80ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:55.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.408+0000 7efbfd237700 1 --2- 172.21.15.121:0/2461990085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efbf80ab840 0x7efbf80abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:55.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.408+0000 7efbfd237700 1 -- 172.21.15.121:0/2461990085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efbf8150870 msgr2=0x7efbf80ab350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:55.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.408+0000 7efbfd237700 1 --2- 172.21.15.121:0/2461990085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efbf8150870 0x7efbf80ab350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:55.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.408+0000 7efbfd237700 1 -- 172.21.15.121:0/2461990085 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbe800a440 con 0x7efbf80c38a0 2023-11-15T03:07:55.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.408+0000 7efbfda38700 1 --2- 172.21.15.121:0/2461990085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efbf80ab840 0x7efbf80abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:07:55.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.408+0000 7efbfca36700 1 --2- 172.21.15.121:0/2461990085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efbf8150870 0x7efbf80ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:07:55.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.408+0000 7efbfd237700 1 --2- 172.21.15.121:0/2461990085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efbf80c38a0 0x7efbf814faf0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 rx=0x7efbf400c5c0 tx=0x7efbf4007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:55.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.408+0000 7efbed7fa700 1 -- 172.21.15.121:0/2461990085 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4007e40 con 0x7efbf80c38a0 2023-11-15T03:07:55.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.408+0000 7efbed7fa700 1 -- 172.21.15.121:0/2461990085 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf40072e0 con 0x7efbf80c38a0 2023-11-15T03:07:55.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.408+0000 7efbff49b700 1 -- 172.21.15.121:0/2461990085 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf80ac660 con 0x7efbf80c38a0 2023-11-15T03:07:55.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.408+0000 7efbed7fa700 1 -- 172.21.15.121:0/2461990085 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf40057a0 con 0x7efbf80c38a0 2023-11-15T03:07:55.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.408+0000 7efbff49b700 1 -- 172.21.15.121:0/2461990085 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf815a280 con 0x7efbf80c38a0 2023-11-15T03:07:55.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.408+0000 7efbff49b700 1 -- 172.21.15.121:0/2461990085 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf806c150 con 0x7efbf80c38a0 2023-11-15T03:07:55.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.408+0000 7efbed7fa700 1 -- 172.21.15.121:0/2461990085 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7efbf4005b90 con 0x7efbf80c38a0 2023-11-15T03:07:55.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.412+0000 7efbed7fa700 1 --2- 172.21.15.121:0/2461990085 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efbdc05a510 0x7efbdc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:07:55.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.412+0000 7efbed7fa700 1 -- 172.21.15.121:0/2461990085 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8416+0+0 (secure 0 0 0) 0x7efbf4074980 con 0x7efbf80c38a0 2023-11-15T03:07:55.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.412+0000 7efbfca36700 1 --2- 172.21.15.121:0/2461990085 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efbdc05a510 0x7efbdc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:07:55.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.412+0000 7efbfca36700 1 --2- 172.21.15.121:0/2461990085 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efbdc05a510 0x7efbdc05c9d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7efbe800afd0 tx=0x7efbe8019000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:07:55.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.416+0000 7efbed7fa700 1 -- 172.21.15.121:0/2461990085 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efbf4047830 con 0x7efbf80c38a0 2023-11-15T03:07:55.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.608+0000 7efbff49b700 1 -- 172.21.15.121:0/2461990085 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efbf8068ae0 con 0x7efbf80c38a0 2023-11-15T03:07:55.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.608+0000 7efbed7fa700 1 -- 172.21.15.121:0/2461990085 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 233 v233) v1 ==== 95+0+4502 (secure 0 0 0) 0x7efbf404ae50 con 0x7efbf80c38a0 2023-11-15T03:07:55.617 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:07:55.617 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":233,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":233,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:55.203824+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9273":{"gid":9273,"name":"a","rank":0,"incarnation":233,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/461059977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":461059977},{"type":"v1","addr":"172.21.15.121:6836","nonce":461059977}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:07:55.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.612+0000 7efbff49b700 1 -- 172.21.15.121:0/2461990085 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efbdc05a510 msgr2=0x7efbdc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:55.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.612+0000 7efbff49b700 1 --2- 172.21.15.121:0/2461990085 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efbdc05a510 0x7efbdc05c9d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7efbe800afd0 tx=0x7efbe8019000).stop 2023-11-15T03:07:55.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.612+0000 7efbff49b700 1 -- 172.21.15.121:0/2461990085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efbf80c38a0 msgr2=0x7efbf814faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:07:55.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.612+0000 7efbff49b700 1 --2- 172.21.15.121:0/2461990085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efbf80c38a0 0x7efbf814faf0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 rx=0x7efbf400c5c0 tx=0x7efbf4007c80).stop 2023-11-15T03:07:55.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.612+0000 7efbff49b700 1 -- 172.21.15.121:0/2461990085 shutdown_connections 2023-11-15T03:07:55.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.612+0000 7efbff49b700 1 --2- 172.21.15.121:0/2461990085 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efbdc05a510 0x7efbdc05c9d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:55.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.612+0000 7efbff49b700 1 --2- 172.21.15.121:0/2461990085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efbf80ab840 0x7efbf80abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:55.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.612+0000 7efbff49b700 1 --2- 172.21.15.121:0/2461990085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efbf8150870 0x7efbf80ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:55.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.612+0000 7efbff49b700 1 --2- 172.21.15.121:0/2461990085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efbf80c38a0 0x7efbf814faf0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:07:55.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.612+0000 7efbff49b700 1 -- 172.21.15.121:0/2461990085 >> 172.21.15.121:0/2461990085 conn(0x7efbf8062190 msgr2=0x7efbf80c6560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:07:55.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.616+0000 7efbff49b700 1 -- 172.21.15.121:0/2461990085 shutdown_connections 2023-11-15T03:07:55.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:07:55.616+0000 7efbff49b700 1 -- 172.21.15.121:0/2461990085 wait complete. 2023-11-15T03:07:55.624 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 233 2023-11-15T03:07:55.646 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:07:56.215 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:07:56.208+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:07:56.216 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:07:56.208+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:07:56.223 INFO:tasks.ceph.mds.a.smithi121.stderr: -21> 2023-11-15T03:07:55.212+0000 7f5346291700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2023-11-15T03:07:56.270 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:08:00.647 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:00.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.800+0000 7f4eeab07700 1 Processor -- start 2023-11-15T03:08:00.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.800+0000 7f4eeab07700 1 -- start start 2023-11-15T03:08:00.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.804+0000 7f4eeab07700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4ee415c540 0x7f4ee40c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:00.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.804+0000 7f4eeab07700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee40c1d50 con 0x7f4ee4152850 2023-11-15T03:08:00.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.804+0000 7f4eeab07700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee415cb80 con 0x7f4ee40c3280 2023-11-15T03:08:00.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.804+0000 7f4eeab07700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee40c1860 con 0x7f4ee415c540 2023-11-15T03:08:00.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.804+0000 7f4ee9304700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4ee4152850 0x7f4ee415be30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56898/0 (socket says 172.21.15.121:56898) 2023-11-15T03:08:00.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.804+0000 7f4ee9304700 1 -- 172.21.15.121:0/2853924218 learned_addr learned my addr 172.21.15.121:0/2853924218 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:00.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.804+0000 7f4eea306700 1 --2- 172.21.15.121:0/2853924218 >> v2:172.21.15.121:3300/0 conn(0x7f4ee415c540 0x7f4ee40c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:00.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.804+0000 7f4ee8b03700 1 -- 172.21.15.121:0/2853924218 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3770214501 0 0) 0x7f4ee40c1d50 con 0x7f4ee4152850 2023-11-15T03:08:00.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.804+0000 7f4ee8b03700 1 -- 172.21.15.121:0/2853924218 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ed00033f0 con 0x7f4ee4152850 2023-11-15T03:08:00.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.804+0000 7f4eea306700 1 -- 172.21.15.121:0/2853924218 >> v1:172.21.15.190:6789/0 conn(0x7f4ee40c3280 legacy=0x7f4ee4152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:00.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.804+0000 7f4eea306700 1 -- 172.21.15.121:0/2853924218 >> v1:172.21.15.121:6789/0 conn(0x7f4ee4152850 legacy=0x7f4ee415be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:00.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.804+0000 7f4eea306700 1 -- 172.21.15.121:0/2853924218 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee40a9240 con 0x7f4ee415c540 2023-11-15T03:08:00.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.804+0000 7f4eea306700 1 --2- 172.21.15.121:0/2853924218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee415c540 0x7f4ee40c3660 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 rx=0x7f4edc009e60 tx=0x7f4edc009780).ready entity=mon.0 client_cookie=981717cdc7ddb319 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:00.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.804+0000 7f4ee8b03700 1 -- 172.21.15.121:0/2853924218 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc00e070 con 0x7f4ee415c540 2023-11-15T03:08:00.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.804+0000 7f4ee8b03700 1 -- 172.21.15.121:0/2853924218 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4edc00bb60 con 0x7f4ee415c540 2023-11-15T03:08:00.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.804+0000 7f4ee8b03700 1 -- 172.21.15.121:0/2853924218 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc012410 con 0x7f4ee415c540 2023-11-15T03:08:00.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.804+0000 7f4eeab07700 1 -- 172.21.15.121:0/2853924218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee415c540 msgr2=0x7f4ee40c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:00.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.804+0000 7f4eeab07700 1 --2- 172.21.15.121:0/2853924218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee415c540 0x7f4ee40c3660 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 rx=0x7f4edc009e60 tx=0x7f4edc009780).stop 2023-11-15T03:08:00.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.808+0000 7f4eeab07700 1 -- 172.21.15.121:0/2853924218 shutdown_connections 2023-11-15T03:08:00.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.808+0000 7f4eeab07700 1 --2- 172.21.15.121:0/2853924218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee415c540 0x7f4ee40c3660 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:00.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.808+0000 7f4eeab07700 1 -- 172.21.15.121:0/2853924218 >> 172.21.15.121:0/2853924218 conn(0x7f4ee4062190 msgr2=0x7f4ee4062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:00.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.808+0000 7f4eeab07700 1 -- 172.21.15.121:0/2853924218 shutdown_connections 2023-11-15T03:08:00.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.808+0000 7f4eeab07700 1 -- 172.21.15.121:0/2853924218 wait complete. 2023-11-15T03:08:00.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.808+0000 7f4eeab07700 1 Processor -- start 2023-11-15T03:08:00.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.808+0000 7f4eeab07700 1 -- start start 2023-11-15T03:08:00.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.812+0000 7f4eeab07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee40c3280 0x7f4ee415a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:00.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.812+0000 7f4eeab07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee4152850 0x7f4ee4157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:00.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.812+0000 7f4eeab07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee415c540 0x7f4ee4157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:00.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.812+0000 7f4eeab07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee416c980 con 0x7f4ee4152850 2023-11-15T03:08:00.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.812+0000 7f4eeab07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee4156ed0 con 0x7f4ee415c540 2023-11-15T03:08:00.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.812+0000 7f4eeab07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee41565a0 con 0x7f4ee40c3280 2023-11-15T03:08:00.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.812+0000 7f4ee9304700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee4152850 0x7f4ee4157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:00.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.812+0000 7f4ee9304700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee4152850 0x7f4ee4157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:57580/0 (socket says 172.21.15.121:57580) 2023-11-15T03:08:00.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.812+0000 7f4ee9304700 1 -- 172.21.15.121:0/2089203076 learned_addr learned my addr 172.21.15.121:0/2089203076 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:00.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.812+0000 7f4ee9b05700 1 --2- 172.21.15.121:0/2089203076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee40c3280 0x7f4ee415a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:00.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.812+0000 7f4eea306700 1 --2- 172.21.15.121:0/2089203076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee415c540 0x7f4ee4157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:00.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.812+0000 7f4ee9304700 1 -- 172.21.15.121:0/2089203076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee40c3280 msgr2=0x7f4ee415a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:00.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.812+0000 7f4ee9304700 1 --2- 172.21.15.121:0/2089203076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee40c3280 0x7f4ee415a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:00.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.812+0000 7f4ee9304700 1 -- 172.21.15.121:0/2089203076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee415c540 msgr2=0x7f4ee4157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:00.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.812+0000 7f4ee9304700 1 --2- 172.21.15.121:0/2089203076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee415c540 0x7f4ee4157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:00.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.812+0000 7f4ee9304700 1 -- 172.21.15.121:0/2089203076 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc009a20 con 0x7f4ee4152850 2023-11-15T03:08:00.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.812+0000 7f4ee9b05700 1 --2- 172.21.15.121:0/2089203076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee40c3280 0x7f4ee415a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:00.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.812+0000 7f4ee9304700 1 --2- 172.21.15.121:0/2089203076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee4152850 0x7f4ee4157360 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 rx=0x7f4ed400c5c0 tx=0x7f4ed4007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:00.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.812+0000 7f4ed9ffb700 1 -- 172.21.15.121:0/2089203076 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4007e40 con 0x7f4ee4152850 2023-11-15T03:08:00.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.816+0000 7f4ed9ffb700 1 -- 172.21.15.121:0/2089203076 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed40072e0 con 0x7f4ee4152850 2023-11-15T03:08:00.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.816+0000 7f4ed9ffb700 1 -- 172.21.15.121:0/2089203076 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed40057a0 con 0x7f4ee4152850 2023-11-15T03:08:00.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.816+0000 7f4eeab07700 1 -- 172.21.15.121:0/2089203076 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee4156880 con 0x7f4ee4152850 2023-11-15T03:08:00.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.816+0000 7f4eeab07700 1 -- 172.21.15.121:0/2089203076 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee415a8b0 con 0x7f4ee4152850 2023-11-15T03:08:00.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.816+0000 7f4eeab07700 1 -- 172.21.15.121:0/2089203076 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee4157060 con 0x7f4ee4152850 2023-11-15T03:08:00.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.816+0000 7f4ed9ffb700 1 -- 172.21.15.121:0/2089203076 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4ed4005b90 con 0x7f4ee4152850 2023-11-15T03:08:00.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.820+0000 7f4ed9ffb700 1 --2- 172.21.15.121:0/2089203076 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4ec805a540 0x7f4ec805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:00.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.820+0000 7f4ed9ffb700 1 -- 172.21.15.121:0/2089203076 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8416+0+0 (secure 0 0 0) 0x7f4ed4074980 con 0x7f4ee4152850 2023-11-15T03:08:00.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.820+0000 7f4ee9b05700 1 --2- 172.21.15.121:0/2089203076 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4ec805a540 0x7f4ec805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:00.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.820+0000 7f4ee9b05700 1 --2- 172.21.15.121:0/2089203076 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4ec805a540 0x7f4ec805ca00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f4ee0001760 tx=0x7f4ee0009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:00.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:00.824+0000 7f4ed9ffb700 1 -- 172.21.15.121:0/2089203076 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4ed4047830 con 0x7f4ee4152850 2023-11-15T03:08:01.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.020+0000 7f4eeab07700 1 -- 172.21.15.121:0/2089203076 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ee406bcf0 con 0x7f4ee4152850 2023-11-15T03:08:01.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.020+0000 7f4ed9ffb700 1 -- 172.21.15.121:0/2089203076 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 235 v235) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f4ed404ae50 con 0x7f4ee4152850 2023-11-15T03:08:01.029 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:01.030 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":235,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9345,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2873318212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2873318212},{"type":"v1","addr":"172.21.15.121:6836","nonce":2873318212}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":235}],"filesystems":[{"mdsmap":{"epoch":234,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:56.211086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:01.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.024+0000 7f4eeab07700 1 -- 172.21.15.121:0/2089203076 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4ec805a540 msgr2=0x7f4ec805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:01.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.024+0000 7f4eeab07700 1 --2- 172.21.15.121:0/2089203076 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4ec805a540 0x7f4ec805ca00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f4ee0001760 tx=0x7f4ee0009040).stop 2023-11-15T03:08:01.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.024+0000 7f4eeab07700 1 -- 172.21.15.121:0/2089203076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee4152850 msgr2=0x7f4ee4157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:01.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.024+0000 7f4eeab07700 1 --2- 172.21.15.121:0/2089203076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee4152850 0x7f4ee4157360 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 rx=0x7f4ed400c5c0 tx=0x7f4ed4007c80).stop 2023-11-15T03:08:01.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.024+0000 7f4eeab07700 1 -- 172.21.15.121:0/2089203076 shutdown_connections 2023-11-15T03:08:01.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.024+0000 7f4eeab07700 1 --2- 172.21.15.121:0/2089203076 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4ec805a540 0x7f4ec805ca00 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.024+0000 7f4eeab07700 1 --2- 172.21.15.121:0/2089203076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee415c540 0x7f4ee4157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.024+0000 7f4eeab07700 1 --2- 172.21.15.121:0/2089203076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ee4152850 0x7f4ee4157360 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.024+0000 7f4eeab07700 1 --2- 172.21.15.121:0/2089203076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee40c3280 0x7f4ee415a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.024+0000 7f4eeab07700 1 -- 172.21.15.121:0/2089203076 >> 172.21.15.121:0/2089203076 conn(0x7f4ee4062190 msgr2=0x7f4ee40669d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:01.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.024+0000 7f4eeab07700 1 -- 172.21.15.121:0/2089203076 shutdown_connections 2023-11-15T03:08:01.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.024+0000 7f4eeab07700 1 -- 172.21.15.121:0/2089203076 wait complete. 2023-11-15T03:08:01.035 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 235 2023-11-15T03:08:01.055 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:08:01.056 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:01.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.204+0000 7f1665c5b700 1 Processor -- start 2023-11-15T03:08:01.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.204+0000 7f1665c5b700 1 -- start start 2023-11-15T03:08:01.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.208+0000 7f1665c5b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f16600c6ce0 0x7f1660159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:01.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.208+0000 7f1665c5b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f16600a5510 0x7f16600a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:01.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.208+0000 7f1665c5b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16600c3fa0 con 0x7f1660152780 2023-11-15T03:08:01.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.208+0000 7f1665c5b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1660159630 con 0x7f16600a5510 2023-11-15T03:08:01.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.208+0000 7f1665c5b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16600c30b0 con 0x7f16600c6ce0 2023-11-15T03:08:01.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.208+0000 7f165ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f16600a5510 0x7f16600a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:01.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.208+0000 7f165ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f16600a5510 0x7f16600a59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:57594/0 (socket says 172.21.15.121:57594) 2023-11-15T03:08:01.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.208+0000 7f165ffff700 1 -- 172.21.15.121:0/4194615837 learned_addr learned my addr 172.21.15.121:0/4194615837 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:01.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.208+0000 7f1656ffd700 1 --2- 172.21.15.121:0/4194615837 >> v2:172.21.15.190:3300/0 conn(0x7f16600c6ce0 0x7f1660159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:01.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.208+0000 7f165ffff700 1 -- 172.21.15.121:0/4194615837 >> v2:172.21.15.190:3300/0 conn(0x7f16600c6ce0 msgr2=0x7f1660159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:01.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.208+0000 7f165ffff700 1 --2- 172.21.15.121:0/4194615837 >> v2:172.21.15.190:3300/0 conn(0x7f16600c6ce0 0x7f1660159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.208+0000 7f165ffff700 1 -- 172.21.15.121:0/4194615837 >> v1:172.21.15.190:6789/0 conn(0x7f1660152780 legacy=0x7f16600c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:01.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.208+0000 7f165ffff700 1 -- 172.21.15.121:0/4194615837 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16600c3390 con 0x7f16600a5510 2023-11-15T03:08:01.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.208+0000 7f165ffff700 1 --2- 172.21.15.121:0/4194615837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16600a5510 0x7f16600a59a0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 rx=0x7f1660063d40 tx=0x7f1658009840).ready entity=mon.0 client_cookie=333f8821d30777b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:01.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.208+0000 7f165effd700 1 -- 172.21.15.121:0/4194615837 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165800e070 con 0x7f16600a5510 2023-11-15T03:08:01.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.208+0000 7f165effd700 1 -- 172.21.15.121:0/4194615837 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165800ba20 con 0x7f16600a5510 2023-11-15T03:08:01.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.208+0000 7f165effd700 1 -- 172.21.15.121:0/4194615837 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165800bda0 con 0x7f16600a5510 2023-11-15T03:08:01.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.212+0000 7f1665c5b700 1 -- 172.21.15.121:0/4194615837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16600a5510 msgr2=0x7f16600a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:01.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.212+0000 7f1665c5b700 1 --2- 172.21.15.121:0/4194615837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16600a5510 0x7f16600a59a0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 rx=0x7f1660063d40 tx=0x7f1658009840).stop 2023-11-15T03:08:01.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.212+0000 7f1665c5b700 1 -- 172.21.15.121:0/4194615837 shutdown_connections 2023-11-15T03:08:01.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.212+0000 7f1665c5b700 1 --2- 172.21.15.121:0/4194615837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16600a5510 0x7f16600a59a0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.212+0000 7f1665c5b700 1 --2- 172.21.15.121:0/4194615837 >> v2:172.21.15.190:3300/0 conn(0x7f16600c6ce0 0x7f1660159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.212+0000 7f1665c5b700 1 -- 172.21.15.121:0/4194615837 >> 172.21.15.121:0/4194615837 conn(0x7f1660062190 msgr2=0x7f16600625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:01.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.212+0000 7f1665c5b700 1 -- 172.21.15.121:0/4194615837 shutdown_connections 2023-11-15T03:08:01.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.212+0000 7f1665c5b700 1 -- 172.21.15.121:0/4194615837 wait complete. 2023-11-15T03:08:01.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.212+0000 7f1665c5b700 1 Processor -- start 2023-11-15T03:08:01.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f1665c5b700 1 -- start start 2023-11-15T03:08:01.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f1665c5b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16600a5510 0x7f16600a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:01.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f1665c5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16600c6ce0 0x7f16600aa200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:01.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f1665c5b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1660152780 0x7f1660149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:01.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f1665c5b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16600a8950 con 0x7f16600a5510 2023-11-15T03:08:01.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f1665c5b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16600aa6f0 con 0x7f1660152780 2023-11-15T03:08:01.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f1665c5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16600aa940 con 0x7f16600c6ce0 2023-11-15T03:08:01.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f165f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16600a5510 0x7f16600a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:01.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f165f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16600a5510 0x7f16600a8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:57604/0 (socket says 172.21.15.121:57604) 2023-11-15T03:08:01.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f165f7fe700 1 -- 172.21.15.121:0/3675497129 learned_addr learned my addr 172.21.15.121:0/3675497129 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:01.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f1656ffd700 1 --2- 172.21.15.121:0/3675497129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16600c6ce0 0x7f16600aa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:01.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f165ffff700 1 --2- 172.21.15.121:0/3675497129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1660152780 0x7f1660149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:01.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f165f7fe700 1 -- 172.21.15.121:0/3675497129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16600c6ce0 msgr2=0x7f16600aa200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:01.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f165f7fe700 1 --2- 172.21.15.121:0/3675497129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16600c6ce0 0x7f16600aa200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f165f7fe700 1 -- 172.21.15.121:0/3675497129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1660152780 msgr2=0x7f1660149990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:01.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f165f7fe700 1 --2- 172.21.15.121:0/3675497129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1660152780 0x7f1660149990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f165f7fe700 1 -- 172.21.15.121:0/3675497129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1658009ae0 con 0x7f16600a5510 2023-11-15T03:08:01.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f165f7fe700 1 --2- 172.21.15.121:0/3675497129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16600a5510 0x7f16600a8460 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 rx=0x7f165000c440 tx=0x7f1650007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:01.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f1657fff700 1 -- 172.21.15.121:0/3675497129 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1650007020 con 0x7f16600a5510 2023-11-15T03:08:01.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f1657fff700 1 -- 172.21.15.121:0/3675497129 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1650007180 con 0x7f16600a5510 2023-11-15T03:08:01.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f1665c5b700 1 -- 172.21.15.121:0/3675497129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16600aac20 con 0x7f16600a5510 2023-11-15T03:08:01.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f1657fff700 1 -- 172.21.15.121:0/3675497129 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1650007520 con 0x7f16600a5510 2023-11-15T03:08:01.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.216+0000 7f1665c5b700 1 -- 172.21.15.121:0/3675497129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16600ab1e0 con 0x7f16600a5510 2023-11-15T03:08:01.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.220+0000 7f1665c5b700 1 -- 172.21.15.121:0/3675497129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f166006c150 con 0x7f16600a5510 2023-11-15T03:08:01.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.220+0000 7f1657fff700 1 -- 172.21.15.121:0/3675497129 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f16500068b0 con 0x7f16600a5510 2023-11-15T03:08:01.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.220+0000 7f1657fff700 1 --2- 172.21.15.121:0/3675497129 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f164005a510 0x7f164005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:01.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.220+0000 7f1657fff700 1 -- 172.21.15.121:0/3675497129 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8416+0+0 (secure 0 0 0) 0x7f16500744b0 con 0x7f16600a5510 2023-11-15T03:08:01.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.220+0000 7f1656ffd700 1 --2- 172.21.15.121:0/3675497129 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f164005a510 0x7f164005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:01.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.224+0000 7f1656ffd700 1 --2- 172.21.15.121:0/3675497129 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f164005a510 0x7f164005c9d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f16600a7330 tx=0x7f1648008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:01.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.228+0000 7f1657fff700 1 -- 172.21.15.121:0/3675497129 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1650047300 con 0x7f16600a5510 2023-11-15T03:08:01.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.428+0000 7f1665c5b700 1 -- 172.21.15.121:0/3675497129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16600c3670 con 0x7f16600a5510 2023-11-15T03:08:01.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.428+0000 7f1657fff700 1 -- 172.21.15.121:0/3675497129 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 235 v235) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f165004a920 con 0x7f16600a5510 2023-11-15T03:08:01.438 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:01.438 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":235,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9345,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2873318212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2873318212},{"type":"v1","addr":"172.21.15.121:6836","nonce":2873318212}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":235}],"filesystems":[{"mdsmap":{"epoch":234,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:56.211086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:01.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.432+0000 7f1665c5b700 1 -- 172.21.15.121:0/3675497129 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f164005a510 msgr2=0x7f164005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:01.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.432+0000 7f1665c5b700 1 --2- 172.21.15.121:0/3675497129 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f164005a510 0x7f164005c9d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f16600a7330 tx=0x7f1648008040).stop 2023-11-15T03:08:01.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.432+0000 7f1665c5b700 1 -- 172.21.15.121:0/3675497129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16600a5510 msgr2=0x7f16600a8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:01.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.432+0000 7f1665c5b700 1 --2- 172.21.15.121:0/3675497129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16600a5510 0x7f16600a8460 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 rx=0x7f165000c440 tx=0x7f1650007e00).stop 2023-11-15T03:08:01.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.436+0000 7f1665c5b700 1 -- 172.21.15.121:0/3675497129 shutdown_connections 2023-11-15T03:08:01.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.436+0000 7f1665c5b700 1 --2- 172.21.15.121:0/3675497129 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f164005a510 0x7f164005c9d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.436+0000 7f1665c5b700 1 --2- 172.21.15.121:0/3675497129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1660152780 0x7f1660149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.436+0000 7f1665c5b700 1 --2- 172.21.15.121:0/3675497129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16600c6ce0 0x7f16600aa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.436+0000 7f1665c5b700 1 --2- 172.21.15.121:0/3675497129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16600a5510 0x7f16600a8460 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.436+0000 7f1665c5b700 1 -- 172.21.15.121:0/3675497129 >> 172.21.15.121:0/3675497129 conn(0x7f1660062190 msgr2=0x7f16600c6500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:01.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.436+0000 7f1665c5b700 1 -- 172.21.15.121:0/3675497129 shutdown_connections 2023-11-15T03:08:01.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.436+0000 7f1665c5b700 1 -- 172.21.15.121:0/3675497129 wait complete. 2023-11-15T03:08:01.446 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 235 2023-11-15T03:08:01.464 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:08:01.465 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:01.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.612+0000 7fee8d769700 1 Processor -- start 2023-11-15T03:08:01.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.616+0000 7fee8d769700 1 -- start start 2023-11-15T03:08:01.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.616+0000 7fee8d769700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee880c1660 0x7fee880c1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:01.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.616+0000 7fee8d769700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee88066950 con 0x7fee880ac1f0 2023-11-15T03:08:01.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.616+0000 7fee8d769700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee880674f0 con 0x7fee880ac5d0 2023-11-15T03:08:01.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.616+0000 7fee8d769700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee88158790 con 0x7fee880c1660 2023-11-15T03:08:01.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.616+0000 7fee86ffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fee880ac1f0 0x7fee88066de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56914/0 (socket says 172.21.15.121:56914) 2023-11-15T03:08:01.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.616+0000 7fee86ffd700 1 -- 172.21.15.121:0/2081763614 learned_addr learned my addr 172.21.15.121:0/2081763614 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:01.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.616+0000 7fee877fe700 1 --2- 172.21.15.121:0/2081763614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee880c1660 0x7fee880c1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:01.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.616+0000 7fee867fc700 1 -- 172.21.15.121:0/2081763614 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898440986 0 0) 0x7fee88066950 con 0x7fee880ac1f0 2023-11-15T03:08:01.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.616+0000 7fee867fc700 1 -- 172.21.15.121:0/2081763614 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee740033f0 con 0x7fee880ac1f0 2023-11-15T03:08:01.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.616+0000 7fee867fc700 1 -- 172.21.15.121:0/2081763614 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573278803 0 0) 0x7fee880674f0 con 0x7fee880ac5d0 2023-11-15T03:08:01.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.616+0000 7fee867fc700 1 -- 172.21.15.121:0/2081763614 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee88066950 con 0x7fee880ac5d0 2023-11-15T03:08:01.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee867fc700 1 -- 172.21.15.121:0/2081763614 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4090944596 0 0) 0x7fee740033f0 con 0x7fee880ac1f0 2023-11-15T03:08:01.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee867fc700 1 -- 172.21.15.121:0/2081763614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee880c1660 msgr2=0x7fee880c1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:01.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee867fc700 1 --2- 172.21.15.121:0/2081763614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee880c1660 0x7fee880c1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee867fc700 1 -- 172.21.15.121:0/2081763614 >> v1:172.21.15.190:6789/0 conn(0x7fee880ac5d0 legacy=0x7fee880c34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:01.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee867fc700 1 -- 172.21.15.121:0/2081763614 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee880c1fe0 con 0x7fee880ac1f0 2023-11-15T03:08:01.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee867fc700 1 -- 172.21.15.121:0/2081763614 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fee78001470 con 0x7fee880ac1f0 2023-11-15T03:08:01.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee877fe700 1 --2- 172.21.15.121:0/2081763614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee880c1660 0x7fee880c1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:08:01.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee867fc700 1 -- 172.21.15.121:0/2081763614 >> v1:172.21.15.121:6789/0 conn(0x7fee880ac1f0 legacy=0x7fee88066de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:01.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee867fc700 1 --2- 172.21.15.121:0/2081763614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee880c1660 0x7fee74004b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:01.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee867fc700 1 --2- 172.21.15.121:0/2081763614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee74004ff0 0x7fee74003630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:01.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee867fc700 1 --2- 172.21.15.121:0/2081763614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee74003b20 0x7fee74003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:01.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee867fc700 1 -- 172.21.15.121:0/2081763614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7400b910 con 0x7fee74004ff0 2023-11-15T03:08:01.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee867fc700 1 -- 172.21.15.121:0/2081763614 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7400ba80 con 0x7fee74003b20 2023-11-15T03:08:01.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee867fc700 1 -- 172.21.15.121:0/2081763614 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7400bc40 con 0x7fee880c1660 2023-11-15T03:08:01.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee7ffff700 1 --2- 172.21.15.121:0/2081763614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee74004ff0 0x7fee74003630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:01.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee86ffd700 1 --2- 172.21.15.121:0/2081763614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee880c1660 0x7fee74004b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:01.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee877fe700 1 --2- 172.21.15.121:0/2081763614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee74003b20 0x7fee74003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:01.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee7ffff700 1 -- 172.21.15.121:0/2081763614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee880c1660 msgr2=0x7fee74004b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:01.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee7ffff700 1 --2- 172.21.15.121:0/2081763614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee880c1660 0x7fee74004b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee7ffff700 1 -- 172.21.15.121:0/2081763614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee74003b20 msgr2=0x7fee74003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:01.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee7ffff700 1 --2- 172.21.15.121:0/2081763614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee74003b20 0x7fee74003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee7ffff700 1 -- 172.21.15.121:0/2081763614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee80009a20 con 0x7fee74004ff0 2023-11-15T03:08:01.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee877fe700 1 --2- 172.21.15.121:0/2081763614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee74003b20 0x7fee74003fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:01.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee7ffff700 1 --2- 172.21.15.121:0/2081763614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee74004ff0 0x7fee74003630 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 rx=0x7fee70001890 tx=0x7fee70007bb0).ready entity=mon.0 client_cookie=53f45c724ff2ad50 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:01.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee867fc700 1 -- 172.21.15.121:0/2081763614 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70007020 con 0x7fee74004ff0 2023-11-15T03:08:01.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee867fc700 1 -- 172.21.15.121:0/2081763614 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee70007180 con 0x7fee74004ff0 2023-11-15T03:08:01.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee867fc700 1 -- 172.21.15.121:0/2081763614 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70007500 con 0x7fee74004ff0 2023-11-15T03:08:01.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee8d769700 1 -- 172.21.15.121:0/2081763614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee74004ff0 msgr2=0x7fee74003630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:01.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee8d769700 1 --2- 172.21.15.121:0/2081763614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee74004ff0 0x7fee74003630 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 rx=0x7fee70001890 tx=0x7fee70007bb0).stop 2023-11-15T03:08:01.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee877fe700 1 -- 172.21.15.121:0/2081763614 reap_dead start 2023-11-15T03:08:01.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee8d769700 1 -- 172.21.15.121:0/2081763614 shutdown_connections 2023-11-15T03:08:01.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee8d769700 1 -- 172.21.15.121:0/2081763614 >> 172.21.15.121:0/2081763614 conn(0x7fee88062190 msgr2=0x7fee88062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:01.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee8d769700 1 -- 172.21.15.121:0/2081763614 shutdown_connections 2023-11-15T03:08:01.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.620+0000 7fee8d769700 1 -- 172.21.15.121:0/2081763614 wait complete. 2023-11-15T03:08:01.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.624+0000 7fee8d769700 1 Processor -- start 2023-11-15T03:08:01.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.624+0000 7fee8d769700 1 -- start start 2023-11-15T03:08:01.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.624+0000 7fee8d769700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee74004ff0 0x7fee880bef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:01.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.624+0000 7fee8d769700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee88156d20 0x7fee88156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:01.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.624+0000 7fee8d769700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee88156870 0x7fee8815bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:01.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.624+0000 7fee8d769700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee880c0110 con 0x7fee74004ff0 2023-11-15T03:08:01.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.624+0000 7fee8d769700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee881571a0 con 0x7fee88156870 2023-11-15T03:08:01.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.624+0000 7fee8d769700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee880c05a0 con 0x7fee88156d20 2023-11-15T03:08:01.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.624+0000 7fee86ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee74004ff0 0x7fee880bef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:01.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.624+0000 7fee86ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee74004ff0 0x7fee880bef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:57618/0 (socket says 172.21.15.121:57618) 2023-11-15T03:08:01.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.624+0000 7fee86ffd700 1 -- 172.21.15.121:0/1625075719 learned_addr learned my addr 172.21.15.121:0/1625075719 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:01.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.624+0000 7fee877fe700 1 --2- 172.21.15.121:0/1625075719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee88156870 0x7fee8815bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:01.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.624+0000 7fee7ffff700 1 --2- 172.21.15.121:0/1625075719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee88156d20 0x7fee88156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:01.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.628+0000 7fee86ffd700 1 -- 172.21.15.121:0/1625075719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee88156d20 msgr2=0x7fee88156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:01.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.628+0000 7fee86ffd700 1 --2- 172.21.15.121:0/1625075719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee88156d20 0x7fee88156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.628+0000 7fee86ffd700 1 -- 172.21.15.121:0/1625075719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee88156870 msgr2=0x7fee8815bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:01.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.628+0000 7fee86ffd700 1 --2- 172.21.15.121:0/1625075719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee88156870 0x7fee8815bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.628+0000 7fee86ffd700 1 -- 172.21.15.121:0/1625075719 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee880bf500 con 0x7fee74004ff0 2023-11-15T03:08:01.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.628+0000 7fee877fe700 1 --2- 172.21.15.121:0/1625075719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee88156870 0x7fee8815bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:01.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.628+0000 7fee7ffff700 1 --2- 172.21.15.121:0/1625075719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee88156d20 0x7fee88156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:01.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.628+0000 7fee86ffd700 1 --2- 172.21.15.121:0/1625075719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee74004ff0 0x7fee880bef20 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 rx=0x7fee780011d0 tx=0x7fee78001df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:01.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.628+0000 7fee7effd700 1 -- 172.21.15.121:0/1625075719 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee78011590 con 0x7fee74004ff0 2023-11-15T03:08:01.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.628+0000 7fee7effd700 1 -- 172.21.15.121:0/1625075719 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee780116f0 con 0x7fee74004ff0 2023-11-15T03:08:01.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.628+0000 7fee8d769700 1 -- 172.21.15.121:0/1625075719 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee880bf7e0 con 0x7fee74004ff0 2023-11-15T03:08:01.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.628+0000 7fee7effd700 1 -- 172.21.15.121:0/1625075719 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee78011a70 con 0x7fee74004ff0 2023-11-15T03:08:01.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.628+0000 7fee8d769700 1 -- 172.21.15.121:0/1625075719 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee880bfda0 con 0x7fee74004ff0 2023-11-15T03:08:01.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.628+0000 7fee8d769700 1 -- 172.21.15.121:0/1625075719 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee8806bcf0 con 0x7fee74004ff0 2023-11-15T03:08:01.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.632+0000 7fee7effd700 1 -- 172.21.15.121:0/1625075719 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fee7801d3b0 con 0x7fee74004ff0 2023-11-15T03:08:01.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.632+0000 7fee7effd700 1 --2- 172.21.15.121:0/1625075719 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fee6805a540 0x7fee6805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:01.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.632+0000 7fee7effd700 1 -- 172.21.15.121:0/1625075719 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8416+0+0 (secure 0 0 0) 0x7fee78074c80 con 0x7fee74004ff0 2023-11-15T03:08:01.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.632+0000 7fee7ffff700 1 --2- 172.21.15.121:0/1625075719 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fee6805a540 0x7fee6805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:01.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.632+0000 7fee7ffff700 1 --2- 172.21.15.121:0/1625075719 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fee6805a540 0x7fee6805ca00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fee88157420 tx=0x7fee70016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:01.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.636+0000 7fee7effd700 1 -- 172.21.15.121:0/1625075719 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fee78047ad0 con 0x7fee74004ff0 2023-11-15T03:08:01.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.852+0000 7fee8d769700 1 -- 172.21.15.121:0/1625075719 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee88065850 con 0x7fee74004ff0 2023-11-15T03:08:01.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.852+0000 7fee7effd700 1 -- 172.21.15.121:0/1625075719 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 235 v235) v1 ==== 95+0+4496 (secure 0 0 0) 0x7fee78011e60 con 0x7fee74004ff0 2023-11-15T03:08:01.860 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:01.860 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":235,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9345,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2873318212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2873318212},{"type":"v1","addr":"172.21.15.121:6836","nonce":2873318212}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":235}],"filesystems":[{"mdsmap":{"epoch":234,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:07:56.211086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:01.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.856+0000 7fee8d769700 1 -- 172.21.15.121:0/1625075719 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fee6805a540 msgr2=0x7fee6805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:01.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.856+0000 7fee8d769700 1 --2- 172.21.15.121:0/1625075719 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fee6805a540 0x7fee6805ca00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fee88157420 tx=0x7fee70016000).stop 2023-11-15T03:08:01.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.856+0000 7fee8d769700 1 -- 172.21.15.121:0/1625075719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee74004ff0 msgr2=0x7fee880bef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:01.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.856+0000 7fee8d769700 1 --2- 172.21.15.121:0/1625075719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee74004ff0 0x7fee880bef20 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 rx=0x7fee780011d0 tx=0x7fee78001df0).stop 2023-11-15T03:08:01.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.856+0000 7fee8d769700 1 -- 172.21.15.121:0/1625075719 shutdown_connections 2023-11-15T03:08:01.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.856+0000 7fee8d769700 1 --2- 172.21.15.121:0/1625075719 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fee6805a540 0x7fee6805ca00 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.856+0000 7fee8d769700 1 --2- 172.21.15.121:0/1625075719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee88156870 0x7fee8815bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.856+0000 7fee8d769700 1 --2- 172.21.15.121:0/1625075719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee88156d20 0x7fee88156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.856+0000 7fee8d769700 1 --2- 172.21.15.121:0/1625075719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee74004ff0 0x7fee880bef20 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:01.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.856+0000 7fee8d769700 1 -- 172.21.15.121:0/1625075719 >> 172.21.15.121:0/1625075719 conn(0x7fee88062190 msgr2=0x7fee88066c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:01.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.856+0000 7fee8d769700 1 -- 172.21.15.121:0/1625075719 shutdown_connections 2023-11-15T03:08:01.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:01.856+0000 7fee8d769700 1 -- 172.21.15.121:0/1625075719 wait complete. 2023-11-15T03:08:01.866 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 235 2023-11-15T03:08:01.882 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 200.00000000' led to DAMAGED state 2023-11-15T03:08:01.882 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 200.00000001' 2023-11-15T03:08:01.883 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:08:01.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:08:01.883 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:08:01.901 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:08:01.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:08:01.903 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:08:02.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.104+0000 7fb013569700 1 Processor -- start 2023-11-15T03:08:02.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.104+0000 7fb013569700 1 -- start start 2023-11-15T03:08:02.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.108+0000 7fb013569700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb00c153aa0 0x7fb00c066680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:02.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.108+0000 7fb013569700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb00c153e80 0x7fb00c0ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:02.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.108+0000 7fb013569700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb00c0c5c50 con 0x7fb00c0c0490 2023-11-15T03:08:02.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.108+0000 7fb013569700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb00c066b70 con 0x7fb00c153aa0 2023-11-15T03:08:02.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.108+0000 7fb013569700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb00c0c5920 con 0x7fb00c153e80 2023-11-15T03:08:02.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.108+0000 7fb011305700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb00c153aa0 0x7fb00c066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:02.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.108+0000 7fb011b06700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb00c0c0490 0x7fb00c062190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56928/0 (socket says 172.21.15.121:56928) 2023-11-15T03:08:02.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.108+0000 7fb011b06700 1 -- 172.21.15.121:0/1625918077 learned_addr learned my addr 172.21.15.121:0/1625918077 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:02.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.108+0000 7fb010b04700 1 --2- 172.21.15.121:0/1625918077 >> v2:172.21.15.190:3300/0 conn(0x7fb00c153e80 0x7fb00c0ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:02.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.108+0000 7fb011305700 1 -- 172.21.15.121:0/1625918077 >> v2:172.21.15.190:3300/0 conn(0x7fb00c153e80 msgr2=0x7fb00c0ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:02.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.108+0000 7fb011305700 1 --2- 172.21.15.121:0/1625918077 >> v2:172.21.15.190:3300/0 conn(0x7fb00c153e80 0x7fb00c0ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:02.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.108+0000 7fb011305700 1 -- 172.21.15.121:0/1625918077 >> v1:172.21.15.121:6789/0 conn(0x7fb00c0c0490 legacy=0x7fb00c062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:02.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.108+0000 7fb011305700 1 -- 172.21.15.121:0/1625918077 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c066d50 con 0x7fb00c153aa0 2023-11-15T03:08:02.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.108+0000 7fb003fff700 1 -- 172.21.15.121:0/1625918077 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613149256 0 0) 0x7fb00c0c5c50 con 0x7fb00c0c0490 2023-11-15T03:08:02.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.108+0000 7fb010b04700 1 --2- 172.21.15.121:0/1625918077 >> v2:172.21.15.190:3300/0 conn(0x7fb00c153e80 0x7fb00c0ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:02.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.108+0000 7fb011305700 1 --2- 172.21.15.121:0/1625918077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c153aa0 0x7fb00c066680 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 rx=0x7fb00c0caa40 tx=0x7fb008009060).ready entity=mon.0 client_cookie=5d440b08c75231de server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:02.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.108+0000 7fb003fff700 1 -- 172.21.15.121:0/1625918077 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00800b580 con 0x7fb00c153aa0 2023-11-15T03:08:02.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.108+0000 7fb003fff700 1 -- 172.21.15.121:0/1625918077 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00800b6e0 con 0x7fb00c153aa0 2023-11-15T03:08:02.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.108+0000 7fb003fff700 1 -- 172.21.15.121:0/1625918077 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00800ba60 con 0x7fb00c153aa0 2023-11-15T03:08:02.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.112+0000 7fb013569700 1 -- 172.21.15.121:0/1625918077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c153aa0 msgr2=0x7fb00c066680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:02.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.112+0000 7fb013569700 1 --2- 172.21.15.121:0/1625918077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c153aa0 0x7fb00c066680 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 rx=0x7fb00c0caa40 tx=0x7fb008009060).stop 2023-11-15T03:08:02.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.112+0000 7fb013569700 1 -- 172.21.15.121:0/1625918077 shutdown_connections 2023-11-15T03:08:02.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.112+0000 7fb013569700 1 --2- 172.21.15.121:0/1625918077 >> v2:172.21.15.190:3300/0 conn(0x7fb00c153e80 0x7fb00c0ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:02.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.112+0000 7fb013569700 1 --2- 172.21.15.121:0/1625918077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c153aa0 0x7fb00c066680 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:02.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.112+0000 7fb013569700 1 -- 172.21.15.121:0/1625918077 >> 172.21.15.121:0/1625918077 conn(0x7fb00c067a60 msgr2=0x7fb00c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:02.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.112+0000 7fb013569700 1 -- 172.21.15.121:0/1625918077 shutdown_connections 2023-11-15T03:08:02.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.112+0000 7fb013569700 1 -- 172.21.15.121:0/1625918077 wait complete. 2023-11-15T03:08:02.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.112+0000 7fb013569700 1 Processor -- start 2023-11-15T03:08:02.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.112+0000 7fb013569700 1 -- start start 2023-11-15T03:08:02.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb013569700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb00c0c0490 0x7fb00c0c1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:02.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb013569700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb00c153aa0 0x7fb00c0c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:02.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb013569700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c153e80 0x7fb00c0b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:02.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb013569700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c0c4a30 con 0x7fb00c153e80 2023-11-15T03:08:02.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb013569700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c0ae130 con 0x7fb00c0c0490 2023-11-15T03:08:02.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb013569700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c0b0740 con 0x7fb00c153aa0 2023-11-15T03:08:02.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb011b06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c153e80 0x7fb00c0b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:02.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb011b06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c153e80 0x7fb00c0b0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:57630/0 (socket says 172.21.15.121:57630) 2023-11-15T03:08:02.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb011b06700 1 -- 172.21.15.121:0/1748318540 learned_addr learned my addr 172.21.15.121:0/1748318540 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:02.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb010b04700 1 --2- 172.21.15.121:0/1748318540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb00c153aa0 0x7fb00c0c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:02.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb011305700 1 --2- 172.21.15.121:0/1748318540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb00c0c0490 0x7fb00c0c1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:02.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb011b06700 1 -- 172.21.15.121:0/1748318540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb00c153aa0 msgr2=0x7fb00c0c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:02.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb011b06700 1 --2- 172.21.15.121:0/1748318540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb00c153aa0 0x7fb00c0c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:02.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb011b06700 1 -- 172.21.15.121:0/1748318540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb00c0c0490 msgr2=0x7fb00c0c1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:02.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb011b06700 1 --2- 172.21.15.121:0/1748318540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb00c0c0490 0x7fb00c0c1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:02.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb011b06700 1 -- 172.21.15.121:0/1748318540 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0080092a0 con 0x7fb00c153e80 2023-11-15T03:08:02.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb011305700 1 --2- 172.21.15.121:0/1748318540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb00c0c0490 0x7fb00c0c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:02.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb011b06700 1 --2- 172.21.15.121:0/1748318540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c153e80 0x7fb00c0b0250 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 rx=0x7fb00400c7e0 tx=0x7fb004007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:02.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb0017fa700 1 -- 172.21.15.121:0/1748318540 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004005d30 con 0x7fb00c153e80 2023-11-15T03:08:02.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb0017fa700 1 -- 172.21.15.121:0/1748318540 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb004005e90 con 0x7fb00c153e80 2023-11-15T03:08:02.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb013569700 1 -- 172.21.15.121:0/1748318540 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00c0c2210 con 0x7fb00c153e80 2023-11-15T03:08:02.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb0017fa700 1 -- 172.21.15.121:0/1748318540 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004007320 con 0x7fb00c153e80 2023-11-15T03:08:02.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.116+0000 7fb013569700 1 -- 172.21.15.121:0/1748318540 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb00c0c27b0 con 0x7fb00c153e80 2023-11-15T03:08:02.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.120+0000 7fb013569700 1 -- 172.21.15.121:0/1748318540 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00c0ae2a0 con 0x7fb00c153e80 2023-11-15T03:08:02.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.120+0000 7fb0017fa700 1 -- 172.21.15.121:0/1748318540 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb0040076d0 con 0x7fb00c153e80 2023-11-15T03:08:02.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.120+0000 7fb0017fa700 1 --2- 172.21.15.121:0/1748318540 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faff005a4e0 0x7faff005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:02.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.120+0000 7fb0017fa700 1 -- 172.21.15.121:0/1748318540 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8416+0+0 (secure 0 0 0) 0x7fb004075170 con 0x7fb00c153e80 2023-11-15T03:08:02.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.124+0000 7fb011305700 1 --2- 172.21.15.121:0/1748318540 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faff005a4e0 0x7faff005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:02.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.124+0000 7fb011305700 1 --2- 172.21.15.121:0/1748318540 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faff005a4e0 0x7faff005c9a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fb008009fd0 tx=0x7fb008017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:02.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.128+0000 7fb0017fa700 1 -- 172.21.15.121:0/1748318540 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb004048020 con 0x7fb00c153e80 2023-11-15T03:08:02.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:02.320+0000 7fb013569700 1 -- 172.21.15.121:0/1748318540 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb00c067030 con 0x7fb00c153e80 2023-11-15T03:08:03.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.264+0000 7fb0017fa700 1 -- 172.21.15.121:0/1748318540 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v236) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb00404b640 con 0x7fb00c153e80 2023-11-15T03:08:03.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.268+0000 7fb013569700 1 -- 172.21.15.121:0/1748318540 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faff005a4e0 msgr2=0x7faff005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:03.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.268+0000 7fb013569700 1 --2- 172.21.15.121:0/1748318540 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faff005a4e0 0x7faff005c9a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fb008009fd0 tx=0x7fb008017000).stop 2023-11-15T03:08:03.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.268+0000 7fb013569700 1 -- 172.21.15.121:0/1748318540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c153e80 msgr2=0x7fb00c0b0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:03.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.268+0000 7fb013569700 1 --2- 172.21.15.121:0/1748318540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c153e80 0x7fb00c0b0250 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 rx=0x7fb00400c7e0 tx=0x7fb004007f90).stop 2023-11-15T03:08:03.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.268+0000 7fb013569700 1 -- 172.21.15.121:0/1748318540 shutdown_connections 2023-11-15T03:08:03.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.268+0000 7fb013569700 1 --2- 172.21.15.121:0/1748318540 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faff005a4e0 0x7faff005c9a0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:03.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.268+0000 7fb013569700 1 --2- 172.21.15.121:0/1748318540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c153e80 0x7fb00c0b0250 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:03.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.268+0000 7fb013569700 1 --2- 172.21.15.121:0/1748318540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb00c153aa0 0x7fb00c0c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:03.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.268+0000 7fb013569700 1 --2- 172.21.15.121:0/1748318540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb00c0c0490 0x7fb00c0c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:03.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.268+0000 7fb013569700 1 -- 172.21.15.121:0/1748318540 >> 172.21.15.121:0/1748318540 conn(0x7fb00c067a60 msgr2=0x7fb00c150230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:03.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.268+0000 7fb013569700 1 -- 172.21.15.121:0/1748318540 shutdown_connections 2023-11-15T03:08:03.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.268+0000 7fb013569700 1 -- 172.21.15.121:0/1748318540 wait complete. 2023-11-15T03:08:03.280 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:08:03.297 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:08:03.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.456+0000 7fd8effff700 1 Processor -- start 2023-11-15T03:08:03.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.460+0000 7fd8effff700 1 -- start start 2023-11-15T03:08:03.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.460+0000 7fd8effff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8f00ab0d0 0x7fd8f00ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:03.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.460+0000 7fd8effff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd8f00ad1b0 0x7fd8f0065710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:03.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.460+0000 7fd8effff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f0066c90 con 0x7fd8f0065c00 2023-11-15T03:08:03.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.460+0000 7fd8effff700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f0066080 con 0x7fd8f00ad1b0 2023-11-15T03:08:03.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.460+0000 7fd8effff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f0066360 con 0x7fd8f00ab0d0 2023-11-15T03:08:03.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.460+0000 7fd8ee7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd8f00ad1b0 0x7fd8f0065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:03.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.460+0000 7fd8ee7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd8f00ad1b0 0x7fd8f0065710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:57632/0 (socket says 172.21.15.121:57632) 2023-11-15T03:08:03.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.460+0000 7fd8ee7fc700 1 -- 172.21.15.121:0/814022509 learned_addr learned my addr 172.21.15.121:0/814022509 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:03.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.464+0000 7fd8eeffd700 1 --2- 172.21.15.121:0/814022509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8f00ab0d0 0x7fd8f00ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:03.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.464+0000 7fd8ee7fc700 1 -- 172.21.15.121:0/814022509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8f00ab0d0 msgr2=0x7fd8f00ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:03.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.464+0000 7fd8ee7fc700 1 --2- 172.21.15.121:0/814022509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8f00ab0d0 0x7fd8f00ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:03.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.464+0000 7fd8ee7fc700 1 -- 172.21.15.121:0/814022509 >> v1:172.21.15.190:6789/0 conn(0x7fd8f0065c00 legacy=0x7fd8f00c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:03.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.464+0000 7fd8ee7fc700 1 -- 172.21.15.121:0/814022509 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f0066640 con 0x7fd8f00ad1b0 2023-11-15T03:08:03.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.464+0000 7fd8ee7fc700 1 --2- 172.21.15.121:0/814022509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8f00ad1b0 0x7fd8f0065710 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 rx=0x7fd8e4006fd0 tx=0x7fd8e40090f0).ready entity=mon.0 client_cookie=d785677f908b628 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:03.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.464+0000 7fd8edffb700 1 -- 172.21.15.121:0/814022509 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e400b580 con 0x7fd8f00ad1b0 2023-11-15T03:08:03.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.464+0000 7fd8edffb700 1 -- 172.21.15.121:0/814022509 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e400b6e0 con 0x7fd8f00ad1b0 2023-11-15T03:08:03.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.464+0000 7fd8edffb700 1 -- 172.21.15.121:0/814022509 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e400ba60 con 0x7fd8f00ad1b0 2023-11-15T03:08:03.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.464+0000 7fd8effff700 1 -- 172.21.15.121:0/814022509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8f00ad1b0 msgr2=0x7fd8f0065710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:03.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.464+0000 7fd8effff700 1 --2- 172.21.15.121:0/814022509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8f00ad1b0 0x7fd8f0065710 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 rx=0x7fd8e4006fd0 tx=0x7fd8e40090f0).stop 2023-11-15T03:08:03.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.464+0000 7fd8effff700 1 -- 172.21.15.121:0/814022509 shutdown_connections 2023-11-15T03:08:03.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.464+0000 7fd8effff700 1 --2- 172.21.15.121:0/814022509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8f00ad1b0 0x7fd8f0065710 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:03.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.464+0000 7fd8effff700 1 --2- 172.21.15.121:0/814022509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8f00ab0d0 0x7fd8f00ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:03.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.464+0000 7fd8effff700 1 -- 172.21.15.121:0/814022509 >> 172.21.15.121:0/814022509 conn(0x7fd8f0067a60 msgr2=0x7fd8f0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:03.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.464+0000 7fd8effff700 1 -- 172.21.15.121:0/814022509 shutdown_connections 2023-11-15T03:08:03.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.464+0000 7fd8effff700 1 -- 172.21.15.121:0/814022509 wait complete. 2023-11-15T03:08:03.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.468+0000 7fd8effff700 1 Processor -- start 2023-11-15T03:08:03.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.468+0000 7fd8effff700 1 -- start start 2023-11-15T03:08:03.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.468+0000 7fd8effff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8f0065c00 0x7fd8f0152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:03.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.468+0000 7fd8effff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8f00ab0d0 0x7fd8f00a5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:03.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.468+0000 7fd8effff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8f00ad1b0 0x7fd8f01653b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:03.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.468+0000 7fd8effff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f00a6ad0 con 0x7fd8f00ab0d0 2023-11-15T03:08:03.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.468+0000 7fd8effff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f00a6d70 con 0x7fd8f0065c00 2023-11-15T03:08:03.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.468+0000 7fd8effff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f00a7050 con 0x7fd8f00ad1b0 2023-11-15T03:08:03.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.468+0000 7fd8ee7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8f00ab0d0 0x7fd8f00a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:03.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.468+0000 7fd8ee7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8f00ab0d0 0x7fd8f00a5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:57634/0 (socket says 172.21.15.121:57634) 2023-11-15T03:08:03.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.468+0000 7fd8ee7fc700 1 -- 172.21.15.121:0/3598632175 learned_addr learned my addr 172.21.15.121:0/3598632175 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:03.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.472+0000 7fd8eeffd700 1 --2- 172.21.15.121:0/3598632175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8f0065c00 0x7fd8f0152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:03.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.472+0000 7fd8ef7fe700 1 --2- 172.21.15.121:0/3598632175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8f00ad1b0 0x7fd8f01653b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:03.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.472+0000 7fd8ee7fc700 1 -- 172.21.15.121:0/3598632175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8f00ad1b0 msgr2=0x7fd8f01653b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:03.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.472+0000 7fd8ee7fc700 1 --2- 172.21.15.121:0/3598632175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8f00ad1b0 0x7fd8f01653b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:03.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.472+0000 7fd8ee7fc700 1 -- 172.21.15.121:0/3598632175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8f0065c00 msgr2=0x7fd8f0152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:03.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.472+0000 7fd8ee7fc700 1 --2- 172.21.15.121:0/3598632175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8f0065c00 0x7fd8f0152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:03.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.472+0000 7fd8ee7fc700 1 -- 172.21.15.121:0/3598632175 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e4009370 con 0x7fd8f00ab0d0 2023-11-15T03:08:03.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.472+0000 7fd8ee7fc700 1 --2- 172.21.15.121:0/3598632175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8f00ab0d0 0x7fd8f00a5010 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 rx=0x7fd8e4011000 tx=0x7fd8e4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:03.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.472+0000 7fd8d6ffd700 1 -- 172.21.15.121:0/3598632175 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e400b850 con 0x7fd8f00ab0d0 2023-11-15T03:08:03.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.472+0000 7fd8d6ffd700 1 -- 172.21.15.121:0/3598632175 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e400b9b0 con 0x7fd8f00ab0d0 2023-11-15T03:08:03.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.472+0000 7fd8d6ffd700 1 -- 172.21.15.121:0/3598632175 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e401d680 con 0x7fd8f00ab0d0 2023-11-15T03:08:03.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.472+0000 7fd8effff700 1 -- 172.21.15.121:0/3598632175 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f00a7330 con 0x7fd8f00ab0d0 2023-11-15T03:08:03.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.472+0000 7fd8effff700 1 -- 172.21.15.121:0/3598632175 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f00a49b0 con 0x7fd8f00ab0d0 2023-11-15T03:08:03.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.472+0000 7fd8effff700 1 -- 172.21.15.121:0/3598632175 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f00a4ba0 con 0x7fd8f00ab0d0 2023-11-15T03:08:03.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.476+0000 7fd8d6ffd700 1 -- 172.21.15.121:0/3598632175 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd8e401d9f0 con 0x7fd8f00ab0d0 2023-11-15T03:08:03.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.476+0000 7fd8d6ffd700 1 --2- 172.21.15.121:0/3598632175 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd8dc05a510 0x7fd8dc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:03.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.476+0000 7fd8d6ffd700 1 -- 172.21.15.121:0/3598632175 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8416+0+0 (secure 0 0 0) 0x7fd8e4077010 con 0x7fd8f00ab0d0 2023-11-15T03:08:03.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.476+0000 7fd8eeffd700 1 --2- 172.21.15.121:0/3598632175 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd8dc05a510 0x7fd8dc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:03.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.476+0000 7fd8eeffd700 1 --2- 172.21.15.121:0/3598632175 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd8dc05a510 0x7fd8dc05c9d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fd8e0009200 tx=0x7fd8e0008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:03.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.484+0000 7fd8d6ffd700 1 -- 172.21.15.121:0/3598632175 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd8e4049ec0 con 0x7fd8f00ab0d0 2023-11-15T03:08:03.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:03.700+0000 7fd8effff700 1 -- 172.21.15.121:0/3598632175 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fd8f006bcf0 con 0x7fd8f00ab0d0 2023-11-15T03:08:04.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.280+0000 7fd8d6ffd700 1 -- 172.21.15.121:0/3598632175 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v237) v1 ==== 101+0+0 (secure 0 0 0) 0x7fd8e404d4e0 con 0x7fd8f00ab0d0 2023-11-15T03:08:04.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.280+0000 7fd8effff700 1 -- 172.21.15.121:0/3598632175 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd8dc05a510 msgr2=0x7fd8dc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.280+0000 7fd8effff700 1 --2- 172.21.15.121:0/3598632175 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd8dc05a510 0x7fd8dc05c9d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fd8e0009200 tx=0x7fd8e0008040).stop 2023-11-15T03:08:04.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.280+0000 7fd8effff700 1 -- 172.21.15.121:0/3598632175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8f00ab0d0 msgr2=0x7fd8f00a5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.280+0000 7fd8effff700 1 --2- 172.21.15.121:0/3598632175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8f00ab0d0 0x7fd8f00a5010 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 rx=0x7fd8e4011000 tx=0x7fd8e4017000).stop 2023-11-15T03:08:04.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.280+0000 7fd8effff700 1 -- 172.21.15.121:0/3598632175 shutdown_connections 2023-11-15T03:08:04.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.280+0000 7fd8effff700 1 --2- 172.21.15.121:0/3598632175 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd8dc05a510 0x7fd8dc05c9d0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.280+0000 7fd8effff700 1 --2- 172.21.15.121:0/3598632175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8f00ad1b0 0x7fd8f01653b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.280+0000 7fd8effff700 1 --2- 172.21.15.121:0/3598632175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8f00ab0d0 0x7fd8f00a5010 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.280+0000 7fd8effff700 1 --2- 172.21.15.121:0/3598632175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8f0065c00 0x7fd8f0152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.280+0000 7fd8effff700 1 -- 172.21.15.121:0/3598632175 >> 172.21.15.121:0/3598632175 conn(0x7fd8f0067a60 msgr2=0x7fd8f00ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:04.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.280+0000 7fd8effff700 1 -- 172.21.15.121:0/3598632175 shutdown_connections 2023-11-15T03:08:04.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.280+0000 7fd8effff700 1 -- 172.21.15.121:0/3598632175 wait complete. 2023-11-15T03:08:04.292 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:08:04.309 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:08:04.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.360+0000 7fcb2e73cd00 1 Processor -- start 2023-11-15T03:08:04.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.360+0000 7fcb2e73cd00 1 -- start start 2023-11-15T03:08:04.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.360+0000 7fcb2e73cd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c5fe3e8a30 0x55c5fe4bebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.360+0000 7fcb2e73cd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5fe433e70 0x55c5fe42e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.360+0000 7fcb2e73cd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5fe4c9000 0x55c5fe3e8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.360+0000 7fcb2e73cd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fe361220 con 0x55c5fe433e70 2023-11-15T03:08:04.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.360+0000 7fcb2e73cd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fe4bf0d0 con 0x55c5fe4c9000 2023-11-15T03:08:04.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.360+0000 7fcb2e73cd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5fe4bf3c0 con 0x55c5fe3e8a30 2023-11-15T03:08:04.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb1a444700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5fe433e70 0x55c5fe42e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb22c45700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c5fe3e8a30 0x55c5fe4bebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb1a444700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5fe433e70 0x55c5fe42e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:57650/0 (socket says 172.21.15.121:57650) 2023-11-15T03:08:04.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb1a444700 1 -- 172.21.15.121:0/3227708594 learned_addr learned my addr 172.21.15.121:0/3227708594 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:04.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb23446700 1 --2- 172.21.15.121:0/3227708594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5fe4c9000 0x55c5fe3e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb1a444700 1 -- 172.21.15.121:0/3227708594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c5fe3e8a30 msgr2=0x55c5fe4bebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:04.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb1a444700 1 --2- 172.21.15.121:0/3227708594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c5fe3e8a30 0x55c5fe4bebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb1a444700 1 -- 172.21.15.121:0/3227708594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5fe4c9000 msgr2=0x55c5fe3e8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:04.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb1a444700 1 --2- 172.21.15.121:0/3227708594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5fe4c9000 0x55c5fe3e8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb1a444700 1 -- 172.21.15.121:0/3227708594 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1c00a180 con 0x55c5fe433e70 2023-11-15T03:08:04.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb1a444700 1 --2- 172.21.15.121:0/3227708594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5fe433e70 0x55c5fe42e640 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 rx=0x7fcb0c00a450 tx=0x7fcb0c0059d0).ready entity=mon.0 client_cookie=283d81331ae51e51 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:04.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb21c43700 1 -- 172.21.15.121:0/3227708594 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c00ae00 con 0x55c5fe433e70 2023-11-15T03:08:04.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb21c43700 1 -- 172.21.15.121:0/3227708594 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0c00c610 con 0x55c5fe433e70 2023-11-15T03:08:04.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb21c43700 1 -- 172.21.15.121:0/3227708594 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c00c9d0 con 0x55c5fe433e70 2023-11-15T03:08:04.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/3227708594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5fe433e70 msgr2=0x55c5fe42e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/3227708594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5fe433e70 0x55c5fe42e640 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 rx=0x7fcb0c00a450 tx=0x7fcb0c0059d0).stop 2023-11-15T03:08:04.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/3227708594 shutdown_connections 2023-11-15T03:08:04.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/3227708594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5fe4c9000 0x55c5fe3e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/3227708594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5fe433e70 0x55c5fe42e640 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/3227708594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c5fe3e8a30 0x55c5fe4bebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/3227708594 >> 172.21.15.121:0/3227708594 conn(0x55c5fe438080 msgr2=0x55c5fe437760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:04.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/3227708594 shutdown_connections 2023-11-15T03:08:04.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/3227708594 wait complete. 2023-11-15T03:08:04.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.364+0000 7fcb2e73cd00 1 Processor -- start 2023-11-15T03:08:04.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb2e73cd00 1 -- start start 2023-11-15T03:08:04.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb2e73cd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c5fe3e8a30 0x55c5fe4bc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb2e73cd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5fe433e70 0x55c5fe4bc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb2e73cd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5fe4c9000 0x55c5fe4ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb2e73cd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fe4ca320 con 0x55c5fe4c9000 2023-11-15T03:08:04.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb2e73cd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fe433300 con 0x55c5fe433e70 2023-11-15T03:08:04.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb2e73cd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5fe432a60 con 0x55c5fe3e8a30 2023-11-15T03:08:04.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb23446700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5fe4c9000 0x55c5fe4ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb23446700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5fe4c9000 0x55c5fe4ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:57658/0 (socket says 172.21.15.121:57658) 2023-11-15T03:08:04.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 learned_addr learned my addr 172.21.15.121:0/1690000609 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:04.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb22c45700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c5fe3e8a30 0x55c5fe4bc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb1a444700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5fe433e70 0x55c5fe4bc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c5fe3e8a30 msgr2=0x55c5fe4bc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb23446700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c5fe3e8a30 0x55c5fe4bc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5fe433e70 msgr2=0x55c5fe4bc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb23446700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5fe433e70 0x55c5fe4bc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c005c90 con 0x55c5fe4c9000 2023-11-15T03:08:04.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb22c45700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c5fe3e8a30 0x55c5fe4bc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:08:04.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb23446700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5fe4c9000 0x55c5fe4ca770 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 rx=0x7fcb1c00e000 tx=0x7fcb1c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:04.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb1bfff700 1 -- 172.21.15.121:0/1690000609 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c00cd30 con 0x55c5fe4c9000 2023-11-15T03:08:04.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb1bfff700 1 -- 172.21.15.121:0/1690000609 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1c00ce90 con 0x55c5fe4c9000 2023-11-15T03:08:04.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb1bfff700 1 -- 172.21.15.121:0/1690000609 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c00c980 con 0x55c5fe4c9000 2023-11-15T03:08:04.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.368+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c5fe38a770 con 0x55c5fe4c9000 2023-11-15T03:08:04.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.372+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5fe359a70 con 0x55c5fe4c9000 2023-11-15T03:08:04.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.372+0000 7fcb1bfff700 1 -- 172.21.15.121:0/1690000609 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fcb1c003040 con 0x55c5fe4c9000 2023-11-15T03:08:04.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.372+0000 7fcb1bfff700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcb04059020 0x7fcb0405b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.372+0000 7fcb1bfff700 1 -- 172.21.15.121:0/1690000609 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8416+0+0 (secure 0 0 0) 0x7fcb1c0763e0 con 0x55c5fe4c9000 2023-11-15T03:08:04.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.376+0000 7fcb22c45700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcb04059020 0x7fcb0405b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.388 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:08:04.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.376+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c5fe432520 0x55c5fe4cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.376+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4c72d0 con 0x55c5fe432520 2023-11-15T03:08:04.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.376+0000 7fcb1a444700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c5fe432520 0x55c5fe4cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.376+0000 7fcb22c45700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcb04059020 0x7fcb0405b4e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7fcb14005fd0 tx=0x7fcb14005bb0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:04.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.376+0000 7fcb1a444700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c5fe432520 0x55c5fe4cac60 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:04.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.380+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v119'293 uv286 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c00c780 con 0x55c5fe432520 2023-11-15T03:08:04.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.380+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4cb150 con 0x55c5fe432520 2023-11-15T03:08:04.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.380+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v119'294 uv294 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe432520 2023-11-15T03:08:04.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.380+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4cb500 con 0x55c5fe432520 2023-11-15T03:08:04.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.384+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v119'295 uv295 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe432520 2023-11-15T03:08:04.392 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:08:04.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.384+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e3760 con 0x55c5fe432520 2023-11-15T03:08:04.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.384+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v119'296 uv296 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe432520 2023-11-15T03:08:04.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.384+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e3b10 con 0x55c5fe432520 2023-11-15T03:08:04.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.388+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v119'297 uv297 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe432520 2023-11-15T03:08:04.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.388+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e2750 con 0x55c5fe432520 2023-11-15T03:08:04.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.388+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v119'298 uv298 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe432520 2023-11-15T03:08:04.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.392+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e2b50 con 0x55c5fe432520 2023-11-15T03:08:04.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.392+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v119'299 uv292 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe432520 2023-11-15T03:08:04.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.392+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4e2f50 con 0x55c5fe432520 2023-11-15T03:08:04.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.396+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v119'300 uv300 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe432520 2023-11-15T03:08:04.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.396+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e2f50 con 0x55c5fe432520 2023-11-15T03:08:04.406 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:08:04.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.396+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v119'301 uv301 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe432520 2023-11-15T03:08:04.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.396+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e2f50 con 0x55c5fe432520 2023-11-15T03:08:04.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.400+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v119'302 uv302 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe432520 2023-11-15T03:08:04.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.400+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e2f50 con 0x55c5fe432520 2023-11-15T03:08:04.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.400+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v119'303 uv303 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe432520 2023-11-15T03:08:04.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.400+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e2f50 con 0x55c5fe432520 2023-11-15T03:08:04.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.404+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v119'304 uv304 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe432520 2023-11-15T03:08:04.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.404+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c5fe4e3350 0x55c5fe4e63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.404+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e2f50 con 0x55c5fe4e3350 2023-11-15T03:08:04.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.404+0000 7fcb22c45700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c5fe4e3350 0x55c5fe4e63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.404+0000 7fcb22c45700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c5fe4e3350 0x55c5fe4e63e0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:04.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.408+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v119'461 uv448 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.408+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4e68d0 con 0x55c5fe4e3350 2023-11-15T03:08:04.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.412+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v119'462 uv462 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.412+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e6c80 con 0x55c5fe4e3350 2023-11-15T03:08:04.423 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:08:04.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.416+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v119'463 uv463 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.416+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e8040 con 0x55c5fe4e3350 2023-11-15T03:08:04.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.420+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v119'464 uv464 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.420+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e83f0 con 0x55c5fe4e3350 2023-11-15T03:08:04.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.420+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v119'465 uv465 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.420+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e7030 con 0x55c5fe4e3350 2023-11-15T03:08:04.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.424+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v119'466 uv466 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.424+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e7920 con 0x55c5fe4e3350 2023-11-15T03:08:04.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.428+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v119'467 uv467 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.428+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e87a0 con 0x55c5fe4e3350 2023-11-15T03:08:04.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.428+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v119'468 uv454 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.428+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4e87a0 con 0x55c5fe4e3350 2023-11-15T03:08:04.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.432+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v119'469 uv469 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.432+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e87a0 con 0x55c5fe4e3350 2023-11-15T03:08:04.443 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:08:04.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.436+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v119'470 uv470 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.436+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e97b0 con 0x55c5fe4e3350 2023-11-15T03:08:04.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.436+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v119'471 uv471 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.436+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e97b0 con 0x55c5fe4e3350 2023-11-15T03:08:04.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.440+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v119'472 uv472 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.440+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e87a0 con 0x55c5fe4e3350 2023-11-15T03:08:04.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.444+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v119'473 uv473 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.444+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e87a0 con 0x55c5fe4e3350 2023-11-15T03:08:04.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.448+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v119'474 uv460 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.448+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4e87a0 con 0x55c5fe4e3350 2023-11-15T03:08:04.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.448+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v119'475 uv475 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.448+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e87a0 con 0x55c5fe4e3350 2023-11-15T03:08:04.459 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:08:04.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.452+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v119'476 uv476 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.452+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e97b0 con 0x55c5fe4e3350 2023-11-15T03:08:04.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.456+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v119'477 uv477 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.456+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e97b0 con 0x55c5fe4e3350 2023-11-15T03:08:04.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.456+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v119'478 uv478 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.456+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4e87a0 con 0x55c5fe4e3350 2023-11-15T03:08:04.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.460+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v119'479 uv479 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.460+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c5fe4e8cb0 0x55c5fe4eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.460+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4eb590 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.460+0000 7fcb1a444700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c5fe4e8cb0 0x55c5fe4eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.464+0000 7fcb1a444700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c5fe4e8cb0 0x55c5fe4eb0a0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:04.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.464+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v119'253 uv246 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.464+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4eb590 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.468+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v119'254 uv254 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.468+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4eb590 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.479 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:08:04.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.472+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v119'255 uv255 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.472+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4eba80 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.476+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v119'256 uv256 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.476+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4eba80 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.476+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v119'257 uv257 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.476+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4eba80 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.480+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v119'258 uv252 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.480+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4eba80 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.484+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v119'259 uv259 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.484+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4eba80 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.494 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:08:04.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.484+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v119'260 uv260 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.488+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4eba80 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.488+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v119'261 uv261 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.488+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4eba80 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.492+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v119'262 uv262 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.492+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c5fe4ebf90 0x55c5fe4ee380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.492+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4ee870 con 0x55c5fe4ebf90 2023-11-15T03:08:04.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.492+0000 7fcb23446700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c5fe4ebf90 0x55c5fe4ee380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.492+0000 7fcb23446700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c5fe4ebf90 0x55c5fe4ee380 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:04.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.496+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v119'296 uv289 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.496+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4eece0 con 0x55c5fe4ebf90 2023-11-15T03:08:04.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.500+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v119'297 uv297 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.500+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4ef190 con 0x55c5fe4ebf90 2023-11-15T03:08:04.510 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:08:04.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.504+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v119'298 uv298 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.504+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4eb670 con 0x55c5fe4ebf90 2023-11-15T03:08:04.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.504+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v119'299 uv299 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.508+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f3c10 con 0x55c5fe4ebf90 2023-11-15T03:08:04.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.508+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v119'300 uv300 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.508+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4ef590 con 0x55c5fe4ebf90 2023-11-15T03:08:04.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.512+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v119'301 uv295 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.512+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4ef990 con 0x55c5fe4ebf90 2023-11-15T03:08:04.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.516+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v119'302 uv302 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.516+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f0340 con 0x55c5fe4ebf90 2023-11-15T03:08:04.526 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:08:04.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.516+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v119'303 uv303 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.520+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f0340 con 0x55c5fe4ebf90 2023-11-15T03:08:04.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.520+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v119'304 uv304 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.520+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f17a0 con 0x55c5fe4ebf90 2023-11-15T03:08:04.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.524+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v119'305 uv305 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.524+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f0300 con 0x55c5fe4ebf90 2023-11-15T03:08:04.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.528+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v119'306 uv306 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.528+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c5fe4f0da0 0x55c5fe4f1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.528+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f0300 con 0x55c5fe4f0da0 2023-11-15T03:08:04.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.528+0000 7fcb22c45700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c5fe4f0da0 0x55c5fe4f1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.528+0000 7fcb22c45700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c5fe4f0da0 0x55c5fe4f1180 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:04.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.532+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v119'147 uv146 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4f0da0 2023-11-15T03:08:04.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.532+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4f0300 con 0x55c5fe4f0da0 2023-11-15T03:08:04.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.532+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v119'148 uv148 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4f0da0 2023-11-15T03:08:04.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.532+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f0280 con 0x55c5fe4f0da0 2023-11-15T03:08:04.543 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:08:04.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.536+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v119'149 uv149 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4f0da0 2023-11-15T03:08:04.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.536+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f5020 con 0x55c5fe4f0da0 2023-11-15T03:08:04.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.540+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v119'150 uv150 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4f0da0 2023-11-15T03:08:04.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.540+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f5020 con 0x55c5fe4f0da0 2023-11-15T03:08:04.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.540+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v119'151 uv151 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4f0da0 2023-11-15T03:08:04.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.540+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f4010 con 0x55c5fe4f0da0 2023-11-15T03:08:04.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.544+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v119'152 uv152 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4f0da0 2023-11-15T03:08:04.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.544+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c5fe4f4520 0x55c5fe4f6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.544+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f6ef0 con 0x55c5fe4f4520 2023-11-15T03:08:04.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.544+0000 7fcb1a444700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c5fe4f4520 0x55c5fe4f6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.544+0000 7fcb1a444700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c5fe4f4520 0x55c5fe4f6910 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:04.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.548+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v119'104 uv103 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.548+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4f7030 con 0x55c5fe4f4520 2023-11-15T03:08:04.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.552+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v119'105 uv105 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.552+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f7030 con 0x55c5fe4f4520 2023-11-15T03:08:04.562 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:08:04.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.556+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v119'106 uv106 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.556+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4f4520 2023-11-15T03:08:04.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.556+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v119'107 uv107 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.556+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4f4520 2023-11-15T03:08:04.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.560+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v119'147 uv146 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.560+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4f4520 2023-11-15T03:08:04.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.564+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v119'148 uv148 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.564+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4f4520 2023-11-15T03:08:04.574 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:08:04.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.568+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v119'149 uv149 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.568+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f84c0 con 0x55c5fe4f4520 2023-11-15T03:08:04.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.568+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v119'150 uv150 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.568+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f84c0 con 0x55c5fe4f4520 2023-11-15T03:08:04.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.572+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v119'151 uv151 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.572+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4f4520 2023-11-15T03:08:04.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.576+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v119'152 uv152 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.576+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4e3350 2023-11-15T03:08:04.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.580+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v119'271 uv265 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.580+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4e3350 2023-11-15T03:08:04.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.584+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v119'272 uv272 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.584+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4e3350 2023-11-15T03:08:04.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.588+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v119'273 uv273 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.594 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:08:04.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.588+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f84c0 con 0x55c5fe4e3350 2023-11-15T03:08:04.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.588+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v119'274 uv274 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.588+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f84c0 con 0x55c5fe4e3350 2023-11-15T03:08:04.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.592+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v119'275 uv275 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.592+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4e3350 2023-11-15T03:08:04.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.596+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v119'276 uv276 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.596+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4e3350 2023-11-15T03:08:04.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.596+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v119'277 uv270 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.596+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4e3350 2023-11-15T03:08:04.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.600+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v119'278 uv278 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.600+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4e3350 2023-11-15T03:08:04.612 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:08:04.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.604+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v119'279 uv279 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.604+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4e3350 2023-11-15T03:08:04.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.608+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v119'280 uv280 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.608+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4e3350 2023-11-15T03:08:04.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.612+0000 7fcb22c45700 1 -- 172.21.15.121:0/1690000609 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v119'281 uv281 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb14016840 con 0x55c5fe4e3350 2023-11-15T03:08:04.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.612+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.612+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v119'122 uv121 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.612+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.616+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v119'123 uv123 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.616+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.626 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:08:04.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.620+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v119'124 uv124 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.620+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.620+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v119'125 uv125 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.620+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.624+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v119'126 uv126 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4e8cb0 2023-11-15T03:08:04.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.624+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4f4520 2023-11-15T03:08:04.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.628+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v119'147 uv146 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.628+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4f4520 2023-11-15T03:08:04.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.628+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v119'148 uv148 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.628+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4f4520 2023-11-15T03:08:04.640 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:08:04.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.632+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v119'149 uv149 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.632+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f84c0 con 0x55c5fe4f4520 2023-11-15T03:08:04.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.636+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v119'150 uv150 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.636+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f84c0 con 0x55c5fe4f4520 2023-11-15T03:08:04.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.640+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v119'151 uv151 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.640+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4f4520 2023-11-15T03:08:04.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.640+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v119'152 uv152 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.640+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4f4520 2023-11-15T03:08:04.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.644+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v119'123 uv122 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.644+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4f4520 2023-11-15T03:08:04.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.648+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v119'124 uv124 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.648+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4f4520 2023-11-15T03:08:04.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.648+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v119'125 uv125 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.658 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:08:04.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.652+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4f4520 2023-11-15T03:08:04.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.652+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v119'126 uv126 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.652+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4f4520 2023-11-15T03:08:04.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.656+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v119'127 uv127 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.656+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4f4520 2023-11-15T03:08:04.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.660+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v119'121 uv120 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.660+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4f4520 2023-11-15T03:08:04.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.660+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v119'122 uv122 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.660+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4f4520 2023-11-15T03:08:04.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.664+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v119'123 uv123 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.672 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:08:04.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.664+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f74b0 con 0x55c5fe4f4520 2023-11-15T03:08:04.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.668+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v119'124 uv124 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.668+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4fa220 con 0x55c5fe4f4520 2023-11-15T03:08:04.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.672+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v119'125 uv125 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.672+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c5fe4f8020 0x55c5fe4f8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.672+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8960 con 0x55c5fe4f8020 2023-11-15T03:08:04.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.672+0000 7fcb23446700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c5fe4f8020 0x55c5fe4f8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.672+0000 7fcb23446700 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c5fe4f8020 0x55c5fe4f8400 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:04.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.676+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v119'98 uv97 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4f8020 2023-11-15T03:08:04.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.676+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4f8960 con 0x55c5fe4f8020 2023-11-15T03:08:04.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.676+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v119'99 uv99 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4f8020 2023-11-15T03:08:04.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.680+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8960 con 0x55c5fe4f8020 2023-11-15T03:08:04.689 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:08:04.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.680+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v119'100 uv100 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4f8020 2023-11-15T03:08:04.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.680+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8960 con 0x55c5fe4f8020 2023-11-15T03:08:04.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.684+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v119'101 uv101 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4f8020 2023-11-15T03:08:04.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.684+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8960 con 0x55c5fe4f4520 2023-11-15T03:08:04.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.688+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v119'147 uv146 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.688+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4f8960 con 0x55c5fe4f4520 2023-11-15T03:08:04.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.688+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v119'148 uv148 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.692+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8960 con 0x55c5fe4f4520 2023-11-15T03:08:04.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.692+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v119'149 uv149 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.701 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:08:04.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.692+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8960 con 0x55c5fe4f4520 2023-11-15T03:08:04.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.696+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v119'150 uv150 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.696+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8960 con 0x55c5fe4f4520 2023-11-15T03:08:04.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.700+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v119'151 uv151 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.700+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8960 con 0x55c5fe4f4520 2023-11-15T03:08:04.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.700+0000 7fcb1a444700 1 -- 172.21.15.121:0/1690000609 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v119'152 uv152 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb0c01f040 con 0x55c5fe4f4520 2023-11-15T03:08:04.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.700+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8960 con 0x55c5fe4ebf90 2023-11-15T03:08:04.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.704+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v119'125 uv124 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.704+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4f8960 con 0x55c5fe4ebf90 2023-11-15T03:08:04.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.708+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v119'126 uv126 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.708+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8960 con 0x55c5fe4ebf90 2023-11-15T03:08:04.717 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:08:04.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.708+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v119'127 uv127 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.708+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8960 con 0x55c5fe4ebf90 2023-11-15T03:08:04.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.712+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v119'128 uv128 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.712+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8d60 con 0x55c5fe4ebf90 2023-11-15T03:08:04.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.716+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v119'129 uv129 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.716+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8d60 con 0x55c5fe4ebf90 2023-11-15T03:08:04.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.720+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v119'293 uv285 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.720+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4f8d60 con 0x55c5fe4ebf90 2023-11-15T03:08:04.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.720+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v119'294 uv294 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.720+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8d60 con 0x55c5fe4ebf90 2023-11-15T03:08:04.731 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:08:04.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.724+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v119'295 uv295 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.724+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8d60 con 0x55c5fe4ebf90 2023-11-15T03:08:04.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.724+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v119'296 uv296 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.724+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8d60 con 0x55c5fe4ebf90 2023-11-15T03:08:04.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.728+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v119'297 uv297 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.728+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8d60 con 0x55c5fe4ebf90 2023-11-15T03:08:04.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.732+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v119'298 uv292 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.732+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e119) v8 -- 0x55c5fe4f8d60 con 0x55c5fe4ebf90 2023-11-15T03:08:04.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.736+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v119'299 uv299 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.736+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8d60 con 0x55c5fe4ebf90 2023-11-15T03:08:04.745 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:08:04.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.736+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v119'300 uv300 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.736+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8d60 con 0x55c5fe4ebf90 2023-11-15T03:08:04.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.740+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v119'301 uv301 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.740+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8d60 con 0x55c5fe4ebf90 2023-11-15T03:08:04.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.744+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v119'302 uv302 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.744+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f8d60 con 0x55c5fe4ebf90 2023-11-15T03:08:04.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.744+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v119'303 uv303 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.744+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x55c5fe4f9160 con 0x55c5fe4ebf90 2023-11-15T03:08:04.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v119'304 uv304 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcb1c024020 con 0x55c5fe4ebf90 2023-11-15T03:08:04.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c5fe4e8cb0 msgr2=0x55c5fe4eb0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55c5fe4e8cb0 0x55c5fe4eb0a0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c5fe432520 msgr2=0x55c5fe4cac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55c5fe432520 0x55c5fe4cac60 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c5fe4f0da0 msgr2=0x55c5fe4f1180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55c5fe4f0da0 0x55c5fe4f1180 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c5fe4e3350 msgr2=0x55c5fe4e63e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55c5fe4e3350 0x55c5fe4e63e0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c5fe4f8020 msgr2=0x55c5fe4f8400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55c5fe4f8020 0x55c5fe4f8400 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c5fe4ebf90 msgr2=0x55c5fe4ee380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c5fe4ebf90 0x55c5fe4ee380 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c5fe4f4520 msgr2=0x55c5fe4f6910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c5fe4f4520 0x55c5fe4f6910 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 reap_dead start 2023-11-15T03:08:04.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 reap_dead start 2023-11-15T03:08:04.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 reap_dead start 2023-11-15T03:08:04.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb23446700 1 -- 172.21.15.121:0/1690000609 reap_dead start 2023-11-15T03:08:04.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcb04059020 msgr2=0x7fcb0405b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcb04059020 0x7fcb0405b4e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7fcb14005fd0 tx=0x7fcb14005bb0).stop 2023-11-15T03:08:04.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5fe4c9000 msgr2=0x55c5fe4ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5fe4c9000 0x55c5fe4ca770 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 rx=0x7fcb1c00e000 tx=0x7fcb1c014000).stop 2023-11-15T03:08:04.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 shutdown_connections 2023-11-15T03:08:04.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcb04059020 0x7fcb0405b4e0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 --2- 172.21.15.121:0/1690000609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5fe4c9000 0x55c5fe4ca770 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 >> 172.21.15.121:0/1690000609 conn(0x55c5fe438080 msgr2=0x55c5fe42e640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:04.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.748+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 shutdown_connections 2023-11-15T03:08:04.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.752+0000 7fcb2e73cd00 1 -- 172.21.15.121:0/1690000609 wait complete. 2023-11-15T03:08:04.766 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 200.00000001 0 2023-11-15T03:08:04.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.860+0000 7fb3570fbd00 1 Processor -- start 2023-11-15T03:08:04.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.860+0000 7fb3570fbd00 1 -- start start 2023-11-15T03:08:04.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb3570fbd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5636e661c440 0x5636e6616b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb3570fbd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636e6636c60 0x5636e6634d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb3570fbd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e66a92e0 0x5636e66c14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb3570fbd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e6563220 con 0x5636e66a92e0 2023-11-15T03:08:04.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb3570fbd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e65eb1d0 con 0x5636e6636c60 2023-11-15T03:08:04.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb3570fbd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636e66c2ff0 con 0x5636e661c440 2023-11-15T03:08:04.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb355246700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e66a92e0 0x5636e66c14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb355246700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e66a92e0 0x5636e66c14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48838/0 (socket says 172.21.15.121:48838) 2023-11-15T03:08:04.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb355246700 1 -- 172.21.15.121:0/3191775157 learned_addr learned my addr 172.21.15.121:0/3191775157 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:04.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb34c244700 1 --2- 172.21.15.121:0/3191775157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636e6636c60 0x5636e6634d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb354a45700 1 --2- 172.21.15.121:0/3191775157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5636e661c440 0x5636e6616b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb355246700 1 -- 172.21.15.121:0/3191775157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5636e661c440 msgr2=0x5636e6616b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:04.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb355246700 1 --2- 172.21.15.121:0/3191775157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5636e661c440 0x5636e6616b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb355246700 1 -- 172.21.15.121:0/3191775157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636e6636c60 msgr2=0x5636e6634d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:04.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb355246700 1 --2- 172.21.15.121:0/3191775157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636e6636c60 0x5636e6634d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb355246700 1 -- 172.21.15.121:0/3191775157 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb344005cf0 con 0x5636e66a92e0 2023-11-15T03:08:04.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb355246700 1 --2- 172.21.15.121:0/3191775157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e66a92e0 0x5636e66c14f0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 rx=0x7fb350001460 tx=0x7fb3500069f0).ready entity=mon.0 client_cookie=9d88a3aef5444ab8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:04.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb34f7fe700 1 -- 172.21.15.121:0/3191775157 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350002080 con 0x5636e66a92e0 2023-11-15T03:08:04.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb34f7fe700 1 -- 172.21.15.121:0/3191775157 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb350006110 con 0x5636e66a92e0 2023-11-15T03:08:04.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb34f7fe700 1 -- 172.21.15.121:0/3191775157 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350004540 con 0x5636e66a92e0 2023-11-15T03:08:04.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb3570fbd00 1 -- 172.21.15.121:0/3191775157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e66a92e0 msgr2=0x5636e66c14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb3570fbd00 1 --2- 172.21.15.121:0/3191775157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e66a92e0 0x5636e66c14f0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 rx=0x7fb350001460 tx=0x7fb3500069f0).stop 2023-11-15T03:08:04.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb3570fbd00 1 -- 172.21.15.121:0/3191775157 shutdown_connections 2023-11-15T03:08:04.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb3570fbd00 1 --2- 172.21.15.121:0/3191775157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e66a92e0 0x5636e66c14f0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb3570fbd00 1 --2- 172.21.15.121:0/3191775157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636e6636c60 0x5636e6634d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb3570fbd00 1 --2- 172.21.15.121:0/3191775157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5636e661c440 0x5636e6616b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb3570fbd00 1 -- 172.21.15.121:0/3191775157 >> 172.21.15.121:0/3191775157 conn(0x5636e65eb7d0 msgr2=0x5636e661b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:04.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb3570fbd00 1 -- 172.21.15.121:0/3191775157 shutdown_connections 2023-11-15T03:08:04.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.864+0000 7fb3570fbd00 1 -- 172.21.15.121:0/3191775157 wait complete. 2023-11-15T03:08:04.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb3570fbd00 1 Processor -- start 2023-11-15T03:08:04.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb3570fbd00 1 -- start start 2023-11-15T03:08:04.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb3570fbd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5636e661c440 0x5636e661b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb3570fbd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636e6636c60 0x5636e6630430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb3570fbd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e66a92e0 0x5636e66a6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb3570fbd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e662cf80 con 0x5636e66a92e0 2023-11-15T03:08:04.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb3570fbd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e661b610 con 0x5636e6636c60 2023-11-15T03:08:04.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb3570fbd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636e662c370 con 0x5636e661c440 2023-11-15T03:08:04.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb355246700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e66a92e0 0x5636e66a6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb355246700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e66a92e0 0x5636e66a6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48846/0 (socket says 172.21.15.121:48846) 2023-11-15T03:08:04.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb355246700 1 -- 172.21.15.121:0/741860748 learned_addr learned my addr 172.21.15.121:0/741860748 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:04.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb354a45700 1 --2- 172.21.15.121:0/741860748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5636e661c440 0x5636e661b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb34c244700 1 --2- 172.21.15.121:0/741860748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636e6636c60 0x5636e6630430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb355246700 1 -- 172.21.15.121:0/741860748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5636e661c440 msgr2=0x5636e661b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb355246700 1 --2- 172.21.15.121:0/741860748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5636e661c440 0x5636e661b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb355246700 1 -- 172.21.15.121:0/741860748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636e6636c60 msgr2=0x5636e6630430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb355246700 1 --2- 172.21.15.121:0/741860748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636e6636c60 0x5636e6630430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb355246700 1 -- 172.21.15.121:0/741860748 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb340001e10 con 0x5636e66a92e0 2023-11-15T03:08:04.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb355246700 1 --2- 172.21.15.121:0/741860748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e66a92e0 0x5636e66a6ed0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 rx=0x5636e66c3f40 tx=0x7fb350014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:04.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb34dffb700 1 -- 172.21.15.121:0/741860748 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350004240 con 0x5636e66a92e0 2023-11-15T03:08:04.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb34dffb700 1 -- 172.21.15.121:0/741860748 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb350017070 con 0x5636e66a92e0 2023-11-15T03:08:04.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb3570fbd00 1 -- 172.21.15.121:0/741860748 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33c005cf0 con 0x5636e66a92e0 2023-11-15T03:08:04.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb34dffb700 1 -- 172.21.15.121:0/741860748 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350004c80 con 0x5636e66a92e0 2023-11-15T03:08:04.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.868+0000 7fb3570fbd00 1 -- 172.21.15.121:0/741860748 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636e658c770 con 0x5636e66a92e0 2023-11-15T03:08:04.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.872+0000 7fb34dffb700 1 -- 172.21.15.121:0/741860748 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb350003450 con 0x5636e66a92e0 2023-11-15T03:08:04.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.872+0000 7fb34dffb700 1 --2- 172.21.15.121:0/741860748 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb334059020 0x7fb33405b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.872+0000 7fb34dffb700 1 -- 172.21.15.121:0/741860748 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8416+0+0 (secure 0 0 0) 0x7fb350074da0 con 0x5636e66a92e0 2023-11-15T03:08:04.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.872+0000 7fb354a45700 1 --2- 172.21.15.121:0/741860748 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb334059020 0x7fb33405b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.876+0000 7fb3570fbd00 1 --2- 172.21.15.121:0/741860748 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5636e662bee0 0x5636e662d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:04.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.876+0000 7fb3570fbd00 1 -- 172.21.15.121:0/741860748 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.34 13:2e2fa760:::200.00000001:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected e119) v8 -- 0x5636e662d900 con 0x5636e662bee0 2023-11-15T03:08:04.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.876+0000 7fb354a45700 1 --2- 172.21.15.121:0/741860748 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb334059020 0x7fb33405b4e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7fb344005c60 tx=0x7fb34400c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:04.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.876+0000 7fb34c244700 1 --2- 172.21.15.121:0/741860748 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5636e662bee0 0x5636e662d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:04.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.876+0000 7fb34c244700 1 --2- 172.21.15.121:0/741860748 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5636e662bee0 0x5636e662d410 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:04.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.880+0000 7fb34c244700 1 -- 172.21.15.121:0/741860748 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 200.00000001 [truncate 0] v119'307 uv307 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb33c000bc0 con 0x5636e662bee0 2023-11-15T03:08:04.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.880+0000 7fb3570fbd00 1 -- 172.21.15.121:0/741860748 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5636e662bee0 msgr2=0x5636e662d410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.880+0000 7fb3570fbd00 1 --2- 172.21.15.121:0/741860748 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5636e662bee0 0x5636e662d410 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:04.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.880+0000 7fb3570fbd00 1 -- 172.21.15.121:0/741860748 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb334059020 msgr2=0x7fb33405b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.880+0000 7fb3570fbd00 1 --2- 172.21.15.121:0/741860748 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb334059020 0x7fb33405b4e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7fb344005c60 tx=0x7fb34400c040).stop 2023-11-15T03:08:04.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.880+0000 7fb3570fbd00 1 -- 172.21.15.121:0/741860748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e66a92e0 msgr2=0x5636e66a6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:04.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.880+0000 7fb3570fbd00 1 --2- 172.21.15.121:0/741860748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636e66a92e0 0x5636e66a6ed0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 rx=0x5636e66c3f40 tx=0x7fb350014000).stop 2023-11-15T03:08:04.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.880+0000 7fb355246700 1 -- 172.21.15.121:0/741860748 reap_dead start 2023-11-15T03:08:04.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.880+0000 7fb3570fbd00 1 -- 172.21.15.121:0/741860748 shutdown_connections 2023-11-15T03:08:04.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.880+0000 7fb3570fbd00 1 -- 172.21.15.121:0/741860748 >> 172.21.15.121:0/741860748 conn(0x5636e65eb7d0 msgr2=0x5636e6637060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:04.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.880+0000 7fb3570fbd00 1 -- 172.21.15.121:0/741860748 shutdown_connections 2023-11-15T03:08:04.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:04.880+0000 7fb3570fbd00 1 -- 172.21.15.121:0/741860748 wait complete. 2023-11-15T03:08:04.898 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:08:05.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.060+0000 7f4831f47700 1 Processor -- start 2023-11-15T03:08:05.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.060+0000 7f4831f47700 1 -- start start 2023-11-15T03:08:05.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.064+0000 7f4831f47700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f482c0acd00 0x7f482c0ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:05.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.064+0000 7f4831f47700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f482c0c5c70 0x7f482c1536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:05.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.064+0000 7f4831f47700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482c065ea0 con 0x7f482c066330 2023-11-15T03:08:05.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.064+0000 7f4831f47700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f482c066bb0 con 0x7f482c0c5c70 2023-11-15T03:08:05.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.064+0000 7f4831f47700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f482c065890 con 0x7f482c0acd00 2023-11-15T03:08:05.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.064+0000 7f482bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f482c0c5c70 0x7f482c1536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:05.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.064+0000 7f482bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f482c0c5c70 0x7f482c1536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48848/0 (socket says 172.21.15.121:48848) 2023-11-15T03:08:05.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.064+0000 7f482bfff700 1 -- 172.21.15.121:0/1044519401 learned_addr learned my addr 172.21.15.121:0/1044519401 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:05.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.064+0000 7f4830f45700 1 --2- 172.21.15.121:0/1044519401 >> v2:172.21.15.190:3300/0 conn(0x7f482c0acd00 0x7f482c0ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:05.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.064+0000 7f482bfff700 1 -- 172.21.15.121:0/1044519401 >> v2:172.21.15.190:3300/0 conn(0x7f482c0acd00 msgr2=0x7f482c0ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:05.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.064+0000 7f482bfff700 1 --2- 172.21.15.121:0/1044519401 >> v2:172.21.15.190:3300/0 conn(0x7f482c0acd00 0x7f482c0ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:05.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.064+0000 7f482bfff700 1 -- 172.21.15.121:0/1044519401 >> v1:172.21.15.190:6789/0 conn(0x7f482c066330 legacy=0x7f482c153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:05.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.064+0000 7f482bfff700 1 -- 172.21.15.121:0/1044519401 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c0c64a0 con 0x7f482c0c5c70 2023-11-15T03:08:05.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.068+0000 7f482bfff700 1 --2- 172.21.15.121:0/1044519401 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f482c0c5c70 0x7f482c1536c0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 rx=0x7f482c06c280 tx=0x7f48180090f0).ready entity=mon.0 client_cookie=3c4ef9e0d8b31f33 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:05.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.068+0000 7f482b7fe700 1 -- 172.21.15.121:0/1044519401 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481800b520 con 0x7f482c0c5c70 2023-11-15T03:08:05.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.068+0000 7f482b7fe700 1 -- 172.21.15.121:0/1044519401 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481800b680 con 0x7f482c0c5c70 2023-11-15T03:08:05.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.068+0000 7f482b7fe700 1 -- 172.21.15.121:0/1044519401 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481800ba00 con 0x7f482c0c5c70 2023-11-15T03:08:05.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.068+0000 7f4831f47700 1 -- 172.21.15.121:0/1044519401 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f482c0c5c70 msgr2=0x7f482c1536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:05.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.068+0000 7f4831f47700 1 --2- 172.21.15.121:0/1044519401 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f482c0c5c70 0x7f482c1536c0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 rx=0x7f482c06c280 tx=0x7f48180090f0).stop 2023-11-15T03:08:05.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.068+0000 7f4831f47700 1 -- 172.21.15.121:0/1044519401 shutdown_connections 2023-11-15T03:08:05.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.068+0000 7f4831f47700 1 --2- 172.21.15.121:0/1044519401 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f482c0c5c70 0x7f482c1536c0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:05.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.068+0000 7f4831f47700 1 --2- 172.21.15.121:0/1044519401 >> v2:172.21.15.190:3300/0 conn(0x7f482c0acd00 0x7f482c0ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:05.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.068+0000 7f4831f47700 1 -- 172.21.15.121:0/1044519401 >> 172.21.15.121:0/1044519401 conn(0x7f482c063fc0 msgr2=0x7f482c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:05.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.068+0000 7f4831f47700 1 -- 172.21.15.121:0/1044519401 shutdown_connections 2023-11-15T03:08:05.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.068+0000 7f4831f47700 1 -- 172.21.15.121:0/1044519401 wait complete. 2023-11-15T03:08:05.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.068+0000 7f4831f47700 1 Processor -- start 2023-11-15T03:08:05.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.068+0000 7f4831f47700 1 -- start start 2023-11-15T03:08:05.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f4831f47700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f482c066330 0x7f482c14aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:05.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f4831f47700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f482c0acd00 0x7f482c14bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:05.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f4831f47700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f482c0c5c70 0x7f482c0c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:05.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f4831f47700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c152000 con 0x7f482c0acd00 2023-11-15T03:08:05.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f4831f47700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c14c460 con 0x7f482c066330 2023-11-15T03:08:05.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f4831f47700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482c14c730 con 0x7f482c0c5c70 2023-11-15T03:08:05.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f482bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f482c0acd00 0x7f482c14bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:05.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f482bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f482c0acd00 0x7f482c14bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48854/0 (socket says 172.21.15.121:48854) 2023-11-15T03:08:05.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f482bfff700 1 -- 172.21.15.121:0/2094902283 learned_addr learned my addr 172.21.15.121:0/2094902283 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:05.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f4830f45700 1 --2- 172.21.15.121:0/2094902283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f482c066330 0x7f482c14aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:05.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f4831746700 1 --2- 172.21.15.121:0/2094902283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f482c0c5c70 0x7f482c0c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:05.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f482bfff700 1 -- 172.21.15.121:0/2094902283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f482c0c5c70 msgr2=0x7f482c0c2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:05.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f482bfff700 1 --2- 172.21.15.121:0/2094902283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f482c0c5c70 0x7f482c0c2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:05.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f482bfff700 1 -- 172.21.15.121:0/2094902283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f482c066330 msgr2=0x7f482c14aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:05.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f482bfff700 1 --2- 172.21.15.121:0/2094902283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f482c066330 0x7f482c14aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:05.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f482bfff700 1 -- 172.21.15.121:0/2094902283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4818009370 con 0x7f482c0acd00 2023-11-15T03:08:05.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f482bfff700 1 --2- 172.21.15.121:0/2094902283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f482c0acd00 0x7f482c14bf70 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 rx=0x7f482c064f20 tx=0x7f4818017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:05.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f4828ff9700 1 -- 172.21.15.121:0/2094902283 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481800be90 con 0x7f482c0acd00 2023-11-15T03:08:05.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f4828ff9700 1 -- 172.21.15.121:0/2094902283 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481800b7f0 con 0x7f482c0acd00 2023-11-15T03:08:05.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f4831f47700 1 -- 172.21.15.121:0/2094902283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f482c1519b0 con 0x7f482c0acd00 2023-11-15T03:08:05.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f4828ff9700 1 -- 172.21.15.121:0/2094902283 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481801c670 con 0x7f482c0acd00 2023-11-15T03:08:05.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.072+0000 7f4831f47700 1 -- 172.21.15.121:0/2094902283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f482c151660 con 0x7f482c0acd00 2023-11-15T03:08:05.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.076+0000 7f4831f47700 1 -- 172.21.15.121:0/2094902283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482c14b3c0 con 0x7f482c0acd00 2023-11-15T03:08:05.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.076+0000 7f4828ff9700 1 -- 172.21.15.121:0/2094902283 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f481801c9e0 con 0x7f482c0acd00 2023-11-15T03:08:05.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.076+0000 7f4828ff9700 1 --2- 172.21.15.121:0/2094902283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f481005a510 0x7f481005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:05.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.076+0000 7f4828ff9700 1 -- 172.21.15.121:0/2094902283 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8416+0+0 (secure 0 0 0) 0x7f4818076ec0 con 0x7f482c0acd00 2023-11-15T03:08:05.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.080+0000 7f4830f45700 1 --2- 172.21.15.121:0/2094902283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f481005a510 0x7f481005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:05.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.080+0000 7f4830f45700 1 --2- 172.21.15.121:0/2094902283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f481005a510 0x7f481005c9d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f4820006fd0 tx=0x7f4820006f40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:05.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.084+0000 7f4828ff9700 1 -- 172.21.15.121:0/2094902283 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f482c14b3c0 con 0x7f482c0acd00 2023-11-15T03:08:05.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:05.288+0000 7f4831f47700 1 -- 172.21.15.121:0/2094902283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f482c14b3c0 con 0x7f482c0acd00 2023-11-15T03:08:06.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.291+0000 7f4828ff9700 1 -- 172.21.15.121:0/2094902283 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v238) v1 ==== 162+0+0 (secure 0 0 0) 0x7f482c14b3c0 con 0x7f482c0acd00 2023-11-15T03:08:06.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.295+0000 7f4831f47700 1 -- 172.21.15.121:0/2094902283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f481005a510 msgr2=0x7f481005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:06.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.295+0000 7f4831f47700 1 --2- 172.21.15.121:0/2094902283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f481005a510 0x7f481005c9d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f4820006fd0 tx=0x7f4820006f40).stop 2023-11-15T03:08:06.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.295+0000 7f4831f47700 1 -- 172.21.15.121:0/2094902283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f482c0acd00 msgr2=0x7f482c14bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:06.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.295+0000 7f4831f47700 1 --2- 172.21.15.121:0/2094902283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f482c0acd00 0x7f482c14bf70 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 rx=0x7f482c064f20 tx=0x7f4818017000).stop 2023-11-15T03:08:06.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.299+0000 7f4831f47700 1 -- 172.21.15.121:0/2094902283 shutdown_connections 2023-11-15T03:08:06.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.299+0000 7f4831f47700 1 --2- 172.21.15.121:0/2094902283 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f481005a510 0x7f481005c9d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:06.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.299+0000 7f4831f47700 1 --2- 172.21.15.121:0/2094902283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f482c0c5c70 0x7f482c0c2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:06.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.299+0000 7f4831f47700 1 --2- 172.21.15.121:0/2094902283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f482c0acd00 0x7f482c14bf70 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:06.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.299+0000 7f4831f47700 1 --2- 172.21.15.121:0/2094902283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f482c066330 0x7f482c14aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:06.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.299+0000 7f4831f47700 1 -- 172.21.15.121:0/2094902283 >> 172.21.15.121:0/2094902283 conn(0x7f482c063fc0 msgr2=0x7f482c0cac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:06.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.299+0000 7f4831f47700 1 -- 172.21.15.121:0/2094902283 shutdown_connections 2023-11-15T03:08:06.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.299+0000 7f4831f47700 1 -- 172.21.15.121:0/2094902283 wait complete. 2023-11-15T03:08:06.311 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:08:06.319 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:08:06.311+0000 7f6a08a8f700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2023-11-15T03:08:06.328 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:06.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.467+0000 7fa9848da700 1 Processor -- start 2023-11-15T03:08:06.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.467+0000 7fa9848da700 1 -- start start 2023-11-15T03:08:06.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa9848da700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa9800cabd0 0x7fa9800cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:06.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa9848da700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa9800acc90 0x7fa980066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:06.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa9848da700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9800ad650 con 0x7fa9800ad0f0 2023-11-15T03:08:06.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa9848da700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa980065fc0 con 0x7fa9800acc90 2023-11-15T03:08:06.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa9848da700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9800662a0 con 0x7fa9800cabd0 2023-11-15T03:08:06.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa97e59c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa9800acc90 0x7fa980066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:06.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa97e59c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa9800acc90 0x7fa980066d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48864/0 (socket says 172.21.15.121:48864) 2023-11-15T03:08:06.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa97e59c700 1 -- 172.21.15.121:0/1893445230 learned_addr learned my addr 172.21.15.121:0/1893445230 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:06.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa97ed9d700 1 --2- 172.21.15.121:0/1893445230 >> v2:172.21.15.190:3300/0 conn(0x7fa9800cabd0 0x7fa9800cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:06.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa97e59c700 1 -- 172.21.15.121:0/1893445230 >> v2:172.21.15.190:3300/0 conn(0x7fa9800cabd0 msgr2=0x7fa9800cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:06.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa97e59c700 1 --2- 172.21.15.121:0/1893445230 >> v2:172.21.15.190:3300/0 conn(0x7fa9800cabd0 0x7fa9800cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:06.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa97e59c700 1 -- 172.21.15.121:0/1893445230 >> v1:172.21.15.190:6789/0 conn(0x7fa9800ad0f0 legacy=0x7fa9800ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:06.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa97e59c700 1 -- 172.21.15.121:0/1893445230 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa980066580 con 0x7fa9800acc90 2023-11-15T03:08:06.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa97e59c700 1 --2- 172.21.15.121:0/1893445230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9800acc90 0x7fa980066d80 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 rx=0x7fa980064750 tx=0x7fa9740090f0).ready entity=mon.0 client_cookie=889ba902247d383f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:06.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa97dd9b700 1 -- 172.21.15.121:0/1893445230 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97400b520 con 0x7fa9800acc90 2023-11-15T03:08:06.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa97dd9b700 1 -- 172.21.15.121:0/1893445230 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97400b680 con 0x7fa9800acc90 2023-11-15T03:08:06.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa97dd9b700 1 -- 172.21.15.121:0/1893445230 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97400ba00 con 0x7fa9800acc90 2023-11-15T03:08:06.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa9848da700 1 -- 172.21.15.121:0/1893445230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9800acc90 msgr2=0x7fa980066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:06.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa9848da700 1 --2- 172.21.15.121:0/1893445230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9800acc90 0x7fa980066d80 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 rx=0x7fa980064750 tx=0x7fa9740090f0).stop 2023-11-15T03:08:06.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa9848da700 1 -- 172.21.15.121:0/1893445230 shutdown_connections 2023-11-15T03:08:06.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa9848da700 1 --2- 172.21.15.121:0/1893445230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9800acc90 0x7fa980066d80 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:06.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa9848da700 1 --2- 172.21.15.121:0/1893445230 >> v2:172.21.15.190:3300/0 conn(0x7fa9800cabd0 0x7fa9800cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:06.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa9848da700 1 -- 172.21.15.121:0/1893445230 >> 172.21.15.121:0/1893445230 conn(0x7fa980062190 msgr2=0x7fa9800625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:06.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa9848da700 1 -- 172.21.15.121:0/1893445230 shutdown_connections 2023-11-15T03:08:06.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.471+0000 7fa9848da700 1 -- 172.21.15.121:0/1893445230 wait complete. 2023-11-15T03:08:06.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.475+0000 7fa9848da700 1 Processor -- start 2023-11-15T03:08:06.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.475+0000 7fa9848da700 1 -- start start 2023-11-15T03:08:06.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.475+0000 7fa9848da700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9800acc90 0x7fa9800c04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:06.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.475+0000 7fa9848da700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9800ad0f0 0x7fa980154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:06.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.475+0000 7fa9848da700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9800cabd0 0x7fa980167a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:06.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.475+0000 7fa9848da700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98014dee0 con 0x7fa9800acc90 2023-11-15T03:08:06.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.475+0000 7fa9848da700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9800c1750 con 0x7fa9800ad0f0 2023-11-15T03:08:06.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.475+0000 7fa9848da700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa980152930 con 0x7fa9800cabd0 2023-11-15T03:08:06.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.479+0000 7fa97ed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9800acc90 0x7fa9800c04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:06.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.479+0000 7fa97ed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9800acc90 0x7fa9800c04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48872/0 (socket says 172.21.15.121:48872) 2023-11-15T03:08:06.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.479+0000 7fa97ed9d700 1 -- 172.21.15.121:0/3305335667 learned_addr learned my addr 172.21.15.121:0/3305335667 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:06.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.479+0000 7fa97f59e700 1 --2- 172.21.15.121:0/3305335667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9800cabd0 0x7fa980167a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:06.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.479+0000 7fa97e59c700 1 --2- 172.21.15.121:0/3305335667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9800ad0f0 0x7fa980154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:06.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.479+0000 7fa97ed9d700 1 -- 172.21.15.121:0/3305335667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9800cabd0 msgr2=0x7fa980167a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:06.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.479+0000 7fa97ed9d700 1 --2- 172.21.15.121:0/3305335667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9800cabd0 0x7fa980167a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:06.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.479+0000 7fa97ed9d700 1 -- 172.21.15.121:0/3305335667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9800ad0f0 msgr2=0x7fa980154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:06.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.479+0000 7fa97ed9d700 1 --2- 172.21.15.121:0/3305335667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9800ad0f0 0x7fa980154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:06.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.479+0000 7fa97ed9d700 1 -- 172.21.15.121:0/3305335667 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa974009370 con 0x7fa9800acc90 2023-11-15T03:08:06.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.479+0000 7fa97ed9d700 1 --2- 172.21.15.121:0/3305335667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9800acc90 0x7fa9800c04d0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 rx=0x7fa97000e900 tx=0x7fa970009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:06.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.479+0000 7fa966ffd700 1 -- 172.21.15.121:0/3305335667 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa970012600 con 0x7fa9800acc90 2023-11-15T03:08:06.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.479+0000 7fa966ffd700 1 -- 172.21.15.121:0/3305335667 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa970012760 con 0x7fa9800acc90 2023-11-15T03:08:06.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.479+0000 7fa966ffd700 1 -- 172.21.15.121:0/3305335667 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa970012ae0 con 0x7fa9800acc90 2023-11-15T03:08:06.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.479+0000 7fa9848da700 1 -- 172.21.15.121:0/3305335667 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9800c0bd0 con 0x7fa9800acc90 2023-11-15T03:08:06.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.479+0000 7fa9848da700 1 -- 172.21.15.121:0/3305335667 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9800c1190 con 0x7fa9800acc90 2023-11-15T03:08:06.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.479+0000 7fa9848da700 1 -- 172.21.15.121:0/3305335667 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9800cba60 con 0x7fa9800acc90 2023-11-15T03:08:06.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.483+0000 7fa966ffd700 1 -- 172.21.15.121:0/3305335667 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa97001b400 con 0x7fa9800acc90 2023-11-15T03:08:06.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.483+0000 7fa966ffd700 1 --2- 172.21.15.121:0/3305335667 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa96c05a510 0x7fa96c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:06.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.483+0000 7fa966ffd700 1 -- 172.21.15.121:0/3305335667 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8502+0+0 (secure 0 0 0) 0x7fa9700753d0 con 0x7fa9800acc90 2023-11-15T03:08:06.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.483+0000 7fa97e59c700 1 --2- 172.21.15.121:0/3305335667 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa96c05a510 0x7fa96c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:06.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.483+0000 7fa97e59c700 1 --2- 172.21.15.121:0/3305335667 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa96c05a510 0x7fa96c05c9d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7fa974009130 tx=0x7fa974017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:06.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.491+0000 7fa966ffd700 1 -- 172.21.15.121:0/3305335667 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa9800cba60 con 0x7fa9800acc90 2023-11-15T03:08:06.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.703+0000 7fa9848da700 1 -- 172.21.15.121:0/3305335667 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa980066aa0 con 0x7fa9800acc90 2023-11-15T03:08:06.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.703+0000 7fa966ffd700 1 -- 172.21.15.121:0/3305335667 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 239 v239) v1 ==== 95+0+4505 (secure 0 0 0) 0x7fa980066aa0 con 0x7fa9800acc90 2023-11-15T03:08:06.712 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:06.713 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":239,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":239,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:08:06.297191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9345},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9345":{"gid":9345,"name":"a","rank":0,"incarnation":239,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/2873318212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2873318212},{"type":"v1","addr":"172.21.15.121:6836","nonce":2873318212}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:06.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.707+0000 7fa9848da700 1 -- 172.21.15.121:0/3305335667 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa96c05a510 msgr2=0x7fa96c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:06.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.707+0000 7fa9848da700 1 --2- 172.21.15.121:0/3305335667 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa96c05a510 0x7fa96c05c9d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7fa974009130 tx=0x7fa974017000).stop 2023-11-15T03:08:06.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.707+0000 7fa9848da700 1 -- 172.21.15.121:0/3305335667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9800acc90 msgr2=0x7fa9800c04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:06.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.707+0000 7fa9848da700 1 --2- 172.21.15.121:0/3305335667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9800acc90 0x7fa9800c04d0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 rx=0x7fa97000e900 tx=0x7fa970009060).stop 2023-11-15T03:08:06.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.707+0000 7fa9848da700 1 -- 172.21.15.121:0/3305335667 shutdown_connections 2023-11-15T03:08:06.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.707+0000 7fa9848da700 1 --2- 172.21.15.121:0/3305335667 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa96c05a510 0x7fa96c05c9d0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:06.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.707+0000 7fa9848da700 1 --2- 172.21.15.121:0/3305335667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9800cabd0 0x7fa980167a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:06.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.707+0000 7fa9848da700 1 --2- 172.21.15.121:0/3305335667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9800ad0f0 0x7fa980154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:06.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.707+0000 7fa9848da700 1 --2- 172.21.15.121:0/3305335667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9800acc90 0x7fa9800c04d0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:06.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.707+0000 7fa9848da700 1 -- 172.21.15.121:0/3305335667 >> 172.21.15.121:0/3305335667 conn(0x7fa980062190 msgr2=0x7fa9800ae470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:06.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.707+0000 7fa9848da700 1 -- 172.21.15.121:0/3305335667 shutdown_connections 2023-11-15T03:08:06.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:06.707+0000 7fa9848da700 1 -- 172.21.15.121:0/3305335667 wait complete. 2023-11-15T03:08:06.718 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 239 2023-11-15T03:08:06.732 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:08:07.309 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:08:07.299+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:08:07.309 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:08:07.299+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:08:07.321 INFO:tasks.ceph.mds.a.smithi121.stderr: -16> 2023-11-15T03:08:06.311+0000 7f6a08a8f700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2023-11-15T03:08:07.363 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:08:11.733 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:11.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.887+0000 7f200e5a9700 1 Processor -- start 2023-11-15T03:08:11.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.887+0000 7f200e5a9700 1 -- start start 2023-11-15T03:08:11.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.891+0000 7f200e5a9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2008153bb0 0x7f20080a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:11.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.891+0000 7f200e5a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2008068170 0x7f2008065d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:11.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.891+0000 7f200e5a9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20080cb5c0 con 0x7f20080aa020 2023-11-15T03:08:11.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.891+0000 7f200e5a9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20080aa6f0 con 0x7f2008153bb0 2023-11-15T03:08:11.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.891+0000 7f200e5a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20080c5900 con 0x7f2008068170 2023-11-15T03:08:11.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.891+0000 7f2007fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2008153bb0 0x7f20080a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:11.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.891+0000 7f200cb46700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2008068170 0x7f2008065d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:11.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.891+0000 7f1ffffff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f20080aa020 0x7f2008067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:54748/0 (socket says 172.21.15.121:54748) 2023-11-15T03:08:11.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.891+0000 7f1ffffff700 1 -- 172.21.15.121:0/1392921439 learned_addr learned my addr 172.21.15.121:0/1392921439 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:11.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.895+0000 7f200cb46700 1 -- 172.21.15.121:0/1392921439 >> v2:172.21.15.190:3300/0 conn(0x7f2008153bb0 msgr2=0x7f20080a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:11.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.895+0000 7f200cb46700 1 --2- 172.21.15.121:0/1392921439 >> v2:172.21.15.190:3300/0 conn(0x7f2008153bb0 0x7f20080a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:11.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.895+0000 7f200cb46700 1 -- 172.21.15.121:0/1392921439 >> v1:172.21.15.190:6789/0 conn(0x7f20080aa020 legacy=0x7f2008067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:11.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.895+0000 7f200cb46700 1 -- 172.21.15.121:0/1392921439 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20080c9ca0 con 0x7f2008068170 2023-11-15T03:08:11.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.895+0000 7f200cb46700 1 --2- 172.21.15.121:0/1392921439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2008068170 0x7f2008065d30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f2000009e60 tx=0x7f2000009780).ready entity=mon.2 client_cookie=87400d36df490b87 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:11.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.895+0000 7f20077fe700 1 -- 172.21.15.121:0/1392921439 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200000e070 con 0x7f2008068170 2023-11-15T03:08:11.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.895+0000 7f20077fe700 1 -- 172.21.15.121:0/1392921439 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f200000bb60 con 0x7f2008068170 2023-11-15T03:08:11.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.895+0000 7f20077fe700 1 -- 172.21.15.121:0/1392921439 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2000012410 con 0x7f2008068170 2023-11-15T03:08:11.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.895+0000 7f200e5a9700 1 -- 172.21.15.121:0/1392921439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2008068170 msgr2=0x7f2008065d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:11.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.895+0000 7f200e5a9700 1 --2- 172.21.15.121:0/1392921439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2008068170 0x7f2008065d30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f2000009e60 tx=0x7f2000009780).stop 2023-11-15T03:08:11.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.895+0000 7f200e5a9700 1 -- 172.21.15.121:0/1392921439 shutdown_connections 2023-11-15T03:08:11.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.895+0000 7f200e5a9700 1 --2- 172.21.15.121:0/1392921439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2008068170 0x7f2008065d30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:11.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.895+0000 7f200e5a9700 1 --2- 172.21.15.121:0/1392921439 >> v2:172.21.15.190:3300/0 conn(0x7f2008153bb0 0x7f20080a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:11.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.895+0000 7f200e5a9700 1 -- 172.21.15.121:0/1392921439 >> 172.21.15.121:0/1392921439 conn(0x7f2008058c50 msgr2=0x7f2008059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:11.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.895+0000 7f200e5a9700 1 -- 172.21.15.121:0/1392921439 shutdown_connections 2023-11-15T03:08:11.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.895+0000 7f200e5a9700 1 -- 172.21.15.121:0/1392921439 wait complete. 2023-11-15T03:08:11.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.899+0000 7f200e5a9700 1 Processor -- start 2023-11-15T03:08:11.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.899+0000 7f200e5a9700 1 -- start start 2023-11-15T03:08:11.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.899+0000 7f200e5a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2008068170 0x7f20081415d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:11.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f200e5a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20080aa020 0x7f20080ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:11.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f200e5a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2008153bb0 0x7f20080c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:11.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f200e5a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2008152cb0 con 0x7f2008068170 2023-11-15T03:08:11.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f200e5a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20081515b0 con 0x7f2008153bb0 2023-11-15T03:08:11.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f200e5a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2008150710 con 0x7f20080aa020 2023-11-15T03:08:11.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f2007fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2008068170 0x7f20081415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:11.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f2007fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2008068170 0x7f20081415d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48896/0 (socket says 172.21.15.121:48896) 2023-11-15T03:08:11.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f2007fff700 1 -- 172.21.15.121:0/1900727334 learned_addr learned my addr 172.21.15.121:0/1900727334 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:11.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f1ffffff700 1 --2- 172.21.15.121:0/1900727334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20080aa020 0x7f20080ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:11.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f200cb46700 1 --2- 172.21.15.121:0/1900727334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2008153bb0 0x7f20080c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:11.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f2007fff700 1 -- 172.21.15.121:0/1900727334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20080aa020 msgr2=0x7f20080ac5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:11.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f2007fff700 1 --2- 172.21.15.121:0/1900727334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20080aa020 0x7f20080ac5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:11.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f2007fff700 1 -- 172.21.15.121:0/1900727334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2008153bb0 msgr2=0x7f20080c3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:11.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f2007fff700 1 --2- 172.21.15.121:0/1900727334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2008153bb0 0x7f20080c3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:11.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f2007fff700 1 -- 172.21.15.121:0/1900727334 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2000009a20 con 0x7f2008068170 2023-11-15T03:08:11.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f200cb46700 1 --2- 172.21.15.121:0/1900727334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2008153bb0 0x7f20080c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:08:11.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f1ffffff700 1 --2- 172.21.15.121:0/1900727334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20080aa020 0x7f20080ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:08:11.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f2007fff700 1 --2- 172.21.15.121:0/1900727334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2008068170 0x7f20081415d0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 rx=0x7f1ff800e930 tx=0x7f1ff8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:11.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f2004ff9700 1 -- 172.21.15.121:0/1900727334 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800eed0 con 0x7f2008068170 2023-11-15T03:08:11.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f2004ff9700 1 -- 172.21.15.121:0/1900727334 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff800c850 con 0x7f2008068170 2023-11-15T03:08:11.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f200e5a9700 1 -- 172.21.15.121:0/1900727334 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20081509a0 con 0x7f2008068170 2023-11-15T03:08:11.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f2004ff9700 1 -- 172.21.15.121:0/1900727334 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800cbd0 con 0x7f2008068170 2023-11-15T03:08:11.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f200e5a9700 1 -- 172.21.15.121:0/1900727334 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2008150f60 con 0x7f2008068170 2023-11-15T03:08:11.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.903+0000 7f200e5a9700 1 -- 172.21.15.121:0/1900727334 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f200806c150 con 0x7f2008068170 2023-11-15T03:08:11.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.907+0000 7f2004ff9700 1 -- 172.21.15.121:0/1900727334 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1ff801b400 con 0x7f2008068170 2023-11-15T03:08:11.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.907+0000 7f2004ff9700 1 --2- 172.21.15.121:0/1900727334 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1fe805a510 0x7f1fe805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:11.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.907+0000 7f2004ff9700 1 -- 172.21.15.121:0/1900727334 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8502+0+0 (secure 0 0 0) 0x7f1ff8075780 con 0x7f2008068170 2023-11-15T03:08:11.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.907+0000 7f1ffffff700 1 --2- 172.21.15.121:0/1900727334 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1fe805a510 0x7f1fe805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:11.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.907+0000 7f1ffffff700 1 --2- 172.21.15.121:0/1900727334 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1fe805a510 0x7f1fe805c9d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f20080ada50 tx=0x7f1ff0009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:11.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:11.911+0000 7f2004ff9700 1 -- 172.21.15.121:0/1900727334 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1ff80484f0 con 0x7f2008068170 2023-11-15T03:08:12.072 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:08:12.080 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:08:12.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.107+0000 7f200e5a9700 1 -- 172.21.15.121:0/1900727334 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20080c83c0 con 0x7f2008068170 2023-11-15T03:08:12.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.107+0000 7f2004ff9700 1 -- 172.21.15.121:0/1900727334 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 241 v241) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f1ff804bb10 con 0x7f2008068170 2023-11-15T03:08:12.116 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:12.116 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":241,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1958064439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1958064439},{"type":"v1","addr":"172.21.15.121:6836","nonce":1958064439}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":241}],"filesystems":[{"mdsmap":{"epoch":240,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:08:07.304547+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:12.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.111+0000 7f200e5a9700 1 -- 172.21.15.121:0/1900727334 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1fe805a510 msgr2=0x7f1fe805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:12.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.111+0000 7f200e5a9700 1 --2- 172.21.15.121:0/1900727334 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1fe805a510 0x7f1fe805c9d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f20080ada50 tx=0x7f1ff0009040).stop 2023-11-15T03:08:12.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.111+0000 7f200e5a9700 1 -- 172.21.15.121:0/1900727334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2008068170 msgr2=0x7f20081415d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:12.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.111+0000 7f200e5a9700 1 --2- 172.21.15.121:0/1900727334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2008068170 0x7f20081415d0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 rx=0x7f1ff800e930 tx=0x7f1ff8009060).stop 2023-11-15T03:08:12.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.111+0000 7f200e5a9700 1 -- 172.21.15.121:0/1900727334 shutdown_connections 2023-11-15T03:08:12.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.111+0000 7f200e5a9700 1 --2- 172.21.15.121:0/1900727334 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1fe805a510 0x7f1fe805c9d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.111+0000 7f200e5a9700 1 --2- 172.21.15.121:0/1900727334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2008153bb0 0x7f20080c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.111+0000 7f200e5a9700 1 --2- 172.21.15.121:0/1900727334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20080aa020 0x7f20080ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.111+0000 7f200e5a9700 1 --2- 172.21.15.121:0/1900727334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2008068170 0x7f20081415d0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.111+0000 7f200e5a9700 1 -- 172.21.15.121:0/1900727334 >> 172.21.15.121:0/1900727334 conn(0x7f2008058c50 msgr2=0x7f20080bfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:12.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.111+0000 7f200e5a9700 1 -- 172.21.15.121:0/1900727334 shutdown_connections 2023-11-15T03:08:12.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.111+0000 7f200e5a9700 1 -- 172.21.15.121:0/1900727334 wait complete. 2023-11-15T03:08:12.121 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 241 2023-11-15T03:08:12.135 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:08:12.136 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:12.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.355+0000 7fc7358db700 1 Processor -- start 2023-11-15T03:08:12.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.355+0000 7fc7358db700 1 -- start start 2023-11-15T03:08:12.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.359+0000 7fc7358db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc730151720 0x7fc730157810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:12.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.359+0000 7fc7358db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc7300c53a0 0x7fc730154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:12.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.359+0000 7fc7358db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc730067a60 0x7fc730067ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:12.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.359+0000 7fc7358db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7300c65d0 con 0x7fc7300c53a0 2023-11-15T03:08:12.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.359+0000 7fc7358db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7300683c0 con 0x7fc730151720 2023-11-15T03:08:12.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.359+0000 7fc7358db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc730158060 con 0x7fc730067a60 2023-11-15T03:08:12.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.359+0000 7fc72e7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc7300c53a0 0x7fc730154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:12.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.359+0000 7fc72e7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc7300c53a0 0x7fc730154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48900/0 (socket says 172.21.15.121:48900) 2023-11-15T03:08:12.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.359+0000 7fc72f7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc730067a60 0x7fc730067ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:12.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.359+0000 7fc72e7fc700 1 -- 172.21.15.121:0/808029027 learned_addr learned my addr 172.21.15.121:0/808029027 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:12.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.359+0000 7fc72effd700 1 --2- 172.21.15.121:0/808029027 >> v2:172.21.15.190:3300/0 conn(0x7fc730151720 0x7fc730157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:12.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.359+0000 7fc72e7fc700 1 -- 172.21.15.121:0/808029027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc730067a60 msgr2=0x7fc730067ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:12.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.359+0000 7fc72e7fc700 1 --2- 172.21.15.121:0/808029027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc730067a60 0x7fc730067ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.359+0000 7fc72e7fc700 1 -- 172.21.15.121:0/808029027 >> v2:172.21.15.190:3300/0 conn(0x7fc730151720 msgr2=0x7fc730157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:12.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.359+0000 7fc72e7fc700 1 --2- 172.21.15.121:0/808029027 >> v2:172.21.15.190:3300/0 conn(0x7fc730151720 0x7fc730157810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.359+0000 7fc72e7fc700 1 -- 172.21.15.121:0/808029027 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc730158340 con 0x7fc7300c53a0 2023-11-15T03:08:12.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.363+0000 7fc72e7fc700 1 --2- 172.21.15.121:0/808029027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7300c53a0 0x7fc730154ab0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 rx=0x7fc7300cbb60 tx=0x7fc7240090f0).ready entity=mon.0 client_cookie=a61b70b80ea1820 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:12.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.363+0000 7fc72dffb700 1 -- 172.21.15.121:0/808029027 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72400f510 con 0x7fc7300c53a0 2023-11-15T03:08:12.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.363+0000 7fc72dffb700 1 -- 172.21.15.121:0/808029027 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72400f670 con 0x7fc7300c53a0 2023-11-15T03:08:12.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.363+0000 7fc72dffb700 1 -- 172.21.15.121:0/808029027 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72400f9f0 con 0x7fc7300c53a0 2023-11-15T03:08:12.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.363+0000 7fc7358db700 1 -- 172.21.15.121:0/808029027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7300c53a0 msgr2=0x7fc730154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:12.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.363+0000 7fc7358db700 1 --2- 172.21.15.121:0/808029027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7300c53a0 0x7fc730154ab0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 rx=0x7fc7300cbb60 tx=0x7fc7240090f0).stop 2023-11-15T03:08:12.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.363+0000 7fc7358db700 1 -- 172.21.15.121:0/808029027 shutdown_connections 2023-11-15T03:08:12.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.363+0000 7fc7358db700 1 --2- 172.21.15.121:0/808029027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc730067a60 0x7fc730067ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.363+0000 7fc7358db700 1 --2- 172.21.15.121:0/808029027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7300c53a0 0x7fc730154ab0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.363+0000 7fc7358db700 1 --2- 172.21.15.121:0/808029027 >> v2:172.21.15.190:3300/0 conn(0x7fc730151720 0x7fc730157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.363+0000 7fc7358db700 1 -- 172.21.15.121:0/808029027 >> 172.21.15.121:0/808029027 conn(0x7fc730058c50 msgr2=0x7fc730059030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:12.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.363+0000 7fc7358db700 1 -- 172.21.15.121:0/808029027 shutdown_connections 2023-11-15T03:08:12.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.363+0000 7fc7358db700 1 -- 172.21.15.121:0/808029027 wait complete. 2023-11-15T03:08:12.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.367+0000 7fc7358db700 1 Processor -- start 2023-11-15T03:08:12.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.367+0000 7fc7358db700 1 -- start start 2023-11-15T03:08:12.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc7358db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc730067a60 0x7fc73014eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:12.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc7358db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7300c53a0 0x7fc7300c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:12.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc7358db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc730151720 0x7fc730153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:12.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc7358db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc730164530 con 0x7fc730067a60 2023-11-15T03:08:12.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc7358db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7300bffb0 con 0x7fc730151720 2023-11-15T03:08:12.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc7358db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73014efb0 con 0x7fc7300c53a0 2023-11-15T03:08:12.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc72effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc730067a60 0x7fc73014eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:12.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc72effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc730067a60 0x7fc73014eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48906/0 (socket says 172.21.15.121:48906) 2023-11-15T03:08:12.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc72effd700 1 -- 172.21.15.121:0/1049746495 learned_addr learned my addr 172.21.15.121:0/1049746495 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:12.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc72e7fc700 1 --2- 172.21.15.121:0/1049746495 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7300c53a0 0x7fc7300c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:12.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc72f7fe700 1 --2- 172.21.15.121:0/1049746495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc730151720 0x7fc730153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:12.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc72effd700 1 -- 172.21.15.121:0/1049746495 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7300c53a0 msgr2=0x7fc7300c35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:12.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc72effd700 1 --2- 172.21.15.121:0/1049746495 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7300c53a0 0x7fc7300c35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc72effd700 1 -- 172.21.15.121:0/1049746495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc730151720 msgr2=0x7fc730153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:12.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc72effd700 1 --2- 172.21.15.121:0/1049746495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc730151720 0x7fc730153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc72effd700 1 -- 172.21.15.121:0/1049746495 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc724009370 con 0x7fc730067a60 2023-11-15T03:08:12.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc72f7fe700 1 --2- 172.21.15.121:0/1049746495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc730151720 0x7fc730153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:12.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc72effd700 1 --2- 172.21.15.121:0/1049746495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc730067a60 0x7fc73014eac0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 rx=0x7fc72000e950 tx=0x7fc720009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:12.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc716ffd700 1 -- 172.21.15.121:0/1049746495 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72000c6a0 con 0x7fc730067a60 2023-11-15T03:08:12.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc716ffd700 1 -- 172.21.15.121:0/1049746495 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72000c800 con 0x7fc730067a60 2023-11-15T03:08:12.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc716ffd700 1 -- 172.21.15.121:0/1049746495 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72000cb80 con 0x7fc730067a60 2023-11-15T03:08:12.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc7358db700 1 -- 172.21.15.121:0/1049746495 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7300bf3a0 con 0x7fc730067a60 2023-11-15T03:08:12.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.371+0000 7fc7358db700 1 -- 172.21.15.121:0/1049746495 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7300bf960 con 0x7fc730067a60 2023-11-15T03:08:12.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.375+0000 7fc7358db700 1 -- 172.21.15.121:0/1049746495 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc73006bcf0 con 0x7fc730067a60 2023-11-15T03:08:12.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.375+0000 7fc716ffd700 1 -- 172.21.15.121:0/1049746495 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc72001b400 con 0x7fc730067a60 2023-11-15T03:08:12.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.375+0000 7fc716ffd700 1 --2- 172.21.15.121:0/1049746495 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc71c05a540 0x7fc71c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:12.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.375+0000 7fc716ffd700 1 -- 172.21.15.121:0/1049746495 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8502+0+0 (secure 0 0 0) 0x7fc720075640 con 0x7fc730067a60 2023-11-15T03:08:12.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.375+0000 7fc72e7fc700 1 --2- 172.21.15.121:0/1049746495 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc71c05a540 0x7fc71c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:12.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.379+0000 7fc72e7fc700 1 --2- 172.21.15.121:0/1049746495 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc71c05a540 0x7fc71c05ca00 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7fc724009130 tx=0x7fc724017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:12.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.383+0000 7fc716ffd700 1 -- 172.21.15.121:0/1049746495 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc720048430 con 0x7fc730067a60 2023-11-15T03:08:12.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.579+0000 7fc7358db700 1 -- 172.21.15.121:0/1049746495 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc730158620 con 0x7fc730067a60 2023-11-15T03:08:12.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.579+0000 7fc716ffd700 1 -- 172.21.15.121:0/1049746495 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 241 v241) v1 ==== 95+0+4496 (secure 0 0 0) 0x7fc720016030 con 0x7fc730067a60 2023-11-15T03:08:12.589 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:12.590 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":241,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1958064439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1958064439},{"type":"v1","addr":"172.21.15.121:6836","nonce":1958064439}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":241}],"filesystems":[{"mdsmap":{"epoch":240,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:08:07.304547+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:12.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.587+0000 7fc7358db700 1 -- 172.21.15.121:0/1049746495 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc71c05a540 msgr2=0x7fc71c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:12.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.587+0000 7fc7358db700 1 --2- 172.21.15.121:0/1049746495 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc71c05a540 0x7fc71c05ca00 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7fc724009130 tx=0x7fc724017000).stop 2023-11-15T03:08:12.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.587+0000 7fc7358db700 1 -- 172.21.15.121:0/1049746495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc730067a60 msgr2=0x7fc73014eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:12.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.587+0000 7fc7358db700 1 --2- 172.21.15.121:0/1049746495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc730067a60 0x7fc73014eac0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 rx=0x7fc72000e950 tx=0x7fc720009060).stop 2023-11-15T03:08:12.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.587+0000 7fc7358db700 1 -- 172.21.15.121:0/1049746495 shutdown_connections 2023-11-15T03:08:12.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.587+0000 7fc7358db700 1 --2- 172.21.15.121:0/1049746495 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc71c05a540 0x7fc71c05ca00 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.587+0000 7fc7358db700 1 --2- 172.21.15.121:0/1049746495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc730151720 0x7fc730153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.587+0000 7fc7358db700 1 --2- 172.21.15.121:0/1049746495 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7300c53a0 0x7fc7300c35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.587+0000 7fc7358db700 1 --2- 172.21.15.121:0/1049746495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc730067a60 0x7fc73014eac0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.587+0000 7fc7358db700 1 -- 172.21.15.121:0/1049746495 >> 172.21.15.121:0/1049746495 conn(0x7fc730058c50 msgr2=0x7fc730062190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:12.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.587+0000 7fc7358db700 1 -- 172.21.15.121:0/1049746495 shutdown_connections 2023-11-15T03:08:12.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.587+0000 7fc7358db700 1 -- 172.21.15.121:0/1049746495 wait complete. 2023-11-15T03:08:12.597 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 241 2023-11-15T03:08:12.620 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:08:12.621 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:12.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.763+0000 7fd149ee4700 1 Processor -- start 2023-11-15T03:08:12.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.763+0000 7fd149ee4700 1 -- start start 2023-11-15T03:08:12.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd149ee4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd144151870 0x7fd144151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:12.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd149ee4700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd1440c30e0 0x7fd1440c3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:12.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd149ee4700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1440657f0 con 0x7fd144152140 2023-11-15T03:08:12.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd149ee4700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd144065ad0 con 0x7fd1440c30e0 2023-11-15T03:08:12.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd149ee4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1440c3a50 con 0x7fd144151870 2023-11-15T03:08:12.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd13bfff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fd144152140 0x7fd1440a9830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:46764/0 (socket says 172.21.15.121:46764) 2023-11-15T03:08:12.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd13bfff700 1 -- 172.21.15.121:0/4021511356 learned_addr learned my addr 172.21.15.121:0/4021511356 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:12.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd1496e3700 1 --2- 172.21.15.121:0/4021511356 >> v2:172.21.15.121:3300/0 conn(0x7fd1440c30e0 0x7fd1440c3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:12.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd148ee2700 1 --2- 172.21.15.121:0/4021511356 >> v2:172.21.15.190:3300/0 conn(0x7fd144151870 0x7fd144151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:12.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd13b7fe700 1 -- 172.21.15.121:0/4021511356 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736655246 0 0) 0x7fd1440657f0 con 0x7fd144152140 2023-11-15T03:08:12.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd13b7fe700 1 -- 172.21.15.121:0/4021511356 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd12c0033f0 con 0x7fd144152140 2023-11-15T03:08:12.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd1496e3700 1 -- 172.21.15.121:0/4021511356 >> v2:172.21.15.190:3300/0 conn(0x7fd144151870 msgr2=0x7fd144151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:12.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd1496e3700 1 --2- 172.21.15.121:0/4021511356 >> v2:172.21.15.190:3300/0 conn(0x7fd144151870 0x7fd144151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd1496e3700 1 -- 172.21.15.121:0/4021511356 >> v1:172.21.15.121:6789/0 conn(0x7fd144152140 legacy=0x7fd1440a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:12.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd1496e3700 1 -- 172.21.15.121:0/4021511356 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1440aa260 con 0x7fd1440c30e0 2023-11-15T03:08:12.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd148ee2700 1 --2- 172.21.15.121:0/4021511356 >> v2:172.21.15.190:3300/0 conn(0x7fd144151870 0x7fd144151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:08:12.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd1496e3700 1 --2- 172.21.15.121:0/4021511356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1440c30e0 0x7fd1440c3560 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 rx=0x7fd144067720 tx=0x7fd13c009060).ready entity=mon.0 client_cookie=6f7062f33920adcb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:12.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd13b7fe700 1 -- 172.21.15.121:0/4021511356 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c00d4a0 con 0x7fd1440c30e0 2023-11-15T03:08:12.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd13b7fe700 1 -- 172.21.15.121:0/4021511356 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13c00d600 con 0x7fd1440c30e0 2023-11-15T03:08:12.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.767+0000 7fd13b7fe700 1 -- 172.21.15.121:0/4021511356 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c00d980 con 0x7fd1440c30e0 2023-11-15T03:08:12.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.771+0000 7fd149ee4700 1 -- 172.21.15.121:0/4021511356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1440c30e0 msgr2=0x7fd1440c3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:12.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.771+0000 7fd149ee4700 1 --2- 172.21.15.121:0/4021511356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1440c30e0 0x7fd1440c3560 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 rx=0x7fd144067720 tx=0x7fd13c009060).stop 2023-11-15T03:08:12.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.771+0000 7fd149ee4700 1 -- 172.21.15.121:0/4021511356 shutdown_connections 2023-11-15T03:08:12.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.771+0000 7fd149ee4700 1 --2- 172.21.15.121:0/4021511356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1440c30e0 0x7fd1440c3560 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.771+0000 7fd149ee4700 1 --2- 172.21.15.121:0/4021511356 >> v2:172.21.15.190:3300/0 conn(0x7fd144151870 0x7fd144151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.771+0000 7fd149ee4700 1 -- 172.21.15.121:0/4021511356 >> 172.21.15.121:0/4021511356 conn(0x7fd144062190 msgr2=0x7fd144062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:12.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.771+0000 7fd149ee4700 1 -- 172.21.15.121:0/4021511356 shutdown_connections 2023-11-15T03:08:12.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.771+0000 7fd149ee4700 1 -- 172.21.15.121:0/4021511356 wait complete. 2023-11-15T03:08:12.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.771+0000 7fd149ee4700 1 Processor -- start 2023-11-15T03:08:12.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.775+0000 7fd149ee4700 1 -- start start 2023-11-15T03:08:12.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.775+0000 7fd149ee4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1440c30e0 0x7fd14414a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:12.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.775+0000 7fd149ee4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd144151870 0x7fd1440c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:12.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.775+0000 7fd149ee4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd144152140 0x7fd144143130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:12.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.775+0000 7fd149ee4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1440c2190 con 0x7fd144151870 2023-11-15T03:08:12.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.775+0000 7fd149ee4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1440c2620 con 0x7fd144152140 2023-11-15T03:08:12.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.775+0000 7fd149ee4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1440c1580 con 0x7fd1440c30e0 2023-11-15T03:08:12.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.775+0000 7fd13bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd144151870 0x7fd1440c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:12.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.775+0000 7fd13bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd144151870 0x7fd1440c0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48928/0 (socket says 172.21.15.121:48928) 2023-11-15T03:08:12.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.775+0000 7fd13bfff700 1 -- 172.21.15.121:0/3852802797 learned_addr learned my addr 172.21.15.121:0/3852802797 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:12.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.775+0000 7fd148ee2700 1 --2- 172.21.15.121:0/3852802797 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1440c30e0 0x7fd14414a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:12.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.775+0000 7fd1496e3700 1 --2- 172.21.15.121:0/3852802797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd144152140 0x7fd144143130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:12.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.775+0000 7fd13bfff700 1 -- 172.21.15.121:0/3852802797 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1440c30e0 msgr2=0x7fd14414a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:12.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.775+0000 7fd13bfff700 1 --2- 172.21.15.121:0/3852802797 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1440c30e0 0x7fd14414a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.775+0000 7fd13bfff700 1 -- 172.21.15.121:0/3852802797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd144152140 msgr2=0x7fd144143130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:12.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.775+0000 7fd13bfff700 1 --2- 172.21.15.121:0/3852802797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd144152140 0x7fd144143130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:12.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.775+0000 7fd13bfff700 1 -- 172.21.15.121:0/3852802797 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd140009a20 con 0x7fd144151870 2023-11-15T03:08:12.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.775+0000 7fd148ee2700 1 --2- 172.21.15.121:0/3852802797 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1440c30e0 0x7fd14414a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:12.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.775+0000 7fd13bfff700 1 --2- 172.21.15.121:0/3852802797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd144151870 0x7fd1440c0d50 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 rx=0x7fd13400c420 tx=0x7fd134007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:12.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.779+0000 7fd138ff9700 1 -- 172.21.15.121:0/3852802797 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1340072e0 con 0x7fd144151870 2023-11-15T03:08:12.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.779+0000 7fd138ff9700 1 -- 172.21.15.121:0/3852802797 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd134007440 con 0x7fd144151870 2023-11-15T03:08:12.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.779+0000 7fd138ff9700 1 -- 172.21.15.121:0/3852802797 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1340057a0 con 0x7fd144151870 2023-11-15T03:08:12.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.779+0000 7fd149ee4700 1 -- 172.21.15.121:0/3852802797 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd13c0092a0 con 0x7fd144151870 2023-11-15T03:08:12.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.779+0000 7fd149ee4700 1 -- 172.21.15.121:0/3852802797 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1440c1b40 con 0x7fd144151870 2023-11-15T03:08:12.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.779+0000 7fd149ee4700 1 -- 172.21.15.121:0/3852802797 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14406bcf0 con 0x7fd144151870 2023-11-15T03:08:12.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.783+0000 7fd138ff9700 1 -- 172.21.15.121:0/3852802797 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd134005bd0 con 0x7fd144151870 2023-11-15T03:08:12.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.783+0000 7fd138ff9700 1 --2- 172.21.15.121:0/3852802797 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd13005a4e0 0x7fd13005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:12.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.783+0000 7fd138ff9700 1 -- 172.21.15.121:0/3852802797 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8502+0+0 (secure 0 0 0) 0x7fd134074970 con 0x7fd144151870 2023-11-15T03:08:12.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.783+0000 7fd148ee2700 1 --2- 172.21.15.121:0/3852802797 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd13005a4e0 0x7fd13005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:12.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.783+0000 7fd148ee2700 1 --2- 172.21.15.121:0/3852802797 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd13005a4e0 0x7fd13005c9a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7fd1400099f0 tx=0x7fd14000b040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:12.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.791+0000 7fd138ff9700 1 -- 172.21.15.121:0/3852802797 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd134047740 con 0x7fd144151870 2023-11-15T03:08:12.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.987+0000 7fd149ee4700 1 -- 172.21.15.121:0/3852802797 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1440aa540 con 0x7fd144151870 2023-11-15T03:08:12.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.987+0000 7fd138ff9700 1 -- 172.21.15.121:0/3852802797 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 241 v241) v1 ==== 95+0+4496 (secure 0 0 0) 0x7fd134007d90 con 0x7fd144151870 2023-11-15T03:08:12.997 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:12.997 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":241,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1958064439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1958064439},{"type":"v1","addr":"172.21.15.121:6836","nonce":1958064439}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":241}],"filesystems":[{"mdsmap":{"epoch":240,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:08:07.304547+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:13.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.991+0000 7fd149ee4700 1 -- 172.21.15.121:0/3852802797 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd13005a4e0 msgr2=0x7fd13005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:13.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.991+0000 7fd149ee4700 1 --2- 172.21.15.121:0/3852802797 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd13005a4e0 0x7fd13005c9a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7fd1400099f0 tx=0x7fd14000b040).stop 2023-11-15T03:08:13.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.991+0000 7fd149ee4700 1 -- 172.21.15.121:0/3852802797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd144151870 msgr2=0x7fd1440c0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:13.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.991+0000 7fd149ee4700 1 --2- 172.21.15.121:0/3852802797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd144151870 0x7fd1440c0d50 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 rx=0x7fd13400c420 tx=0x7fd134007c50).stop 2023-11-15T03:08:13.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.991+0000 7fd149ee4700 1 -- 172.21.15.121:0/3852802797 shutdown_connections 2023-11-15T03:08:13.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.991+0000 7fd149ee4700 1 --2- 172.21.15.121:0/3852802797 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd13005a4e0 0x7fd13005c9a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:13.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.991+0000 7fd149ee4700 1 --2- 172.21.15.121:0/3852802797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd144152140 0x7fd144143130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:13.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.991+0000 7fd149ee4700 1 --2- 172.21.15.121:0/3852802797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd144151870 0x7fd1440c0d50 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:13.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.991+0000 7fd149ee4700 1 --2- 172.21.15.121:0/3852802797 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1440c30e0 0x7fd14414a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:13.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.991+0000 7fd149ee4700 1 -- 172.21.15.121:0/3852802797 >> 172.21.15.121:0/3852802797 conn(0x7fd144062190 msgr2=0x7fd1440c4900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:13.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.991+0000 7fd149ee4700 1 -- 172.21.15.121:0/3852802797 shutdown_connections 2023-11-15T03:08:13.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:12.991+0000 7fd149ee4700 1 -- 172.21.15.121:0/3852802797 wait complete. 2023-11-15T03:08:13.004 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 241 2023-11-15T03:08:13.024 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 200.00000001' led to DAMAGED state 2023-11-15T03:08:13.024 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 500.00000000' 2023-11-15T03:08:13.025 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:08:13.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:08:13.026 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:08:13.046 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:08:13.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:08:13.048 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:08:13.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.271+0000 7fdc0209b700 1 Processor -- start 2023-11-15T03:08:13.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.271+0000 7fdc0209b700 1 -- start start 2023-11-15T03:08:13.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.275+0000 7fdc0209b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdbfc151920 0x7fdbfc0bb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:13.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.275+0000 7fdc0209b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdbfc0ac150 0x7fdbfc062190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:13.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.275+0000 7fdc0209b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbfc155de0 con 0x7fdbfc067150 2023-11-15T03:08:13.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.275+0000 7fdc0209b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbfc0c4060 con 0x7fdbfc151920 2023-11-15T03:08:13.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.275+0000 7fdc0209b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbfc0c3aa0 con 0x7fdbfc0ac150 2023-11-15T03:08:13.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.275+0000 7fdbfb7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdbfc151920 0x7fdbfc0bb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:13.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.275+0000 7fdbfbfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdbfc0ac150 0x7fdbfc062190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:13.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.275+0000 7fdbfbfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdbfc0ac150 0x7fdbfc062190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:51440/0 (socket says 172.21.15.121:51440) 2023-11-15T03:08:13.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.275+0000 7fdbfbfff700 1 -- 172.21.15.121:0/3059702853 learned_addr learned my addr 172.21.15.121:0/3059702853 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:13.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.275+0000 7fdbfb7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdbfc151920 0x7fdbfc0bb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48940/0 (socket says 172.21.15.121:48940) 2023-11-15T03:08:13.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.275+0000 7fdbfb7fe700 1 -- 172.21.15.121:0/3059702853 >> v2:172.21.15.190:3300/0 conn(0x7fdbfc0ac150 msgr2=0x7fdbfc062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:13.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.275+0000 7fdbfb7fe700 1 --2- 172.21.15.121:0/3059702853 >> v2:172.21.15.190:3300/0 conn(0x7fdbfc0ac150 0x7fdbfc062190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:13.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.275+0000 7fdbfb7fe700 1 -- 172.21.15.121:0/3059702853 >> v1:172.21.15.190:6789/0 conn(0x7fdbfc067150 legacy=0x7fdbfc0aba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:13.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.275+0000 7fdbfb7fe700 1 -- 172.21.15.121:0/3059702853 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc0c3d80 con 0x7fdbfc151920 2023-11-15T03:08:13.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.275+0000 7fdbfb7fe700 1 --2- 172.21.15.121:0/3059702853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbfc151920 0x7fdbfc0bb6b0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 rx=0x7fdbfc067010 tx=0x7fdbec009060).ready entity=mon.0 client_cookie=4b65d67c6833096d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:13.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.275+0000 7fdbfa7fc700 1 -- 172.21.15.121:0/3059702853 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec00f510 con 0x7fdbfc151920 2023-11-15T03:08:13.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.275+0000 7fdbfa7fc700 1 -- 172.21.15.121:0/3059702853 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbec00f670 con 0x7fdbfc151920 2023-11-15T03:08:13.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.275+0000 7fdbfa7fc700 1 -- 172.21.15.121:0/3059702853 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec00f9f0 con 0x7fdbfc151920 2023-11-15T03:08:13.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.279+0000 7fdc0209b700 1 -- 172.21.15.121:0/3059702853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbfc151920 msgr2=0x7fdbfc0bb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:13.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.279+0000 7fdc0209b700 1 --2- 172.21.15.121:0/3059702853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbfc151920 0x7fdbfc0bb6b0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 rx=0x7fdbfc067010 tx=0x7fdbec009060).stop 2023-11-15T03:08:13.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.279+0000 7fdc0209b700 1 -- 172.21.15.121:0/3059702853 shutdown_connections 2023-11-15T03:08:13.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.279+0000 7fdc0209b700 1 --2- 172.21.15.121:0/3059702853 >> v2:172.21.15.190:3300/0 conn(0x7fdbfc0ac150 0x7fdbfc062190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:13.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.279+0000 7fdc0209b700 1 --2- 172.21.15.121:0/3059702853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbfc151920 0x7fdbfc0bb6b0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:13.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.279+0000 7fdc0209b700 1 -- 172.21.15.121:0/3059702853 >> 172.21.15.121:0/3059702853 conn(0x7fdbfc067a60 msgr2=0x7fdbfc067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:13.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.279+0000 7fdc0209b700 1 -- 172.21.15.121:0/3059702853 shutdown_connections 2023-11-15T03:08:13.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.279+0000 7fdc0209b700 1 -- 172.21.15.121:0/3059702853 wait complete. 2023-11-15T03:08:13.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.279+0000 7fdc0209b700 1 Processor -- start 2023-11-15T03:08:13.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.279+0000 7fdc0209b700 1 -- start start 2023-11-15T03:08:13.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.279+0000 7fdc0209b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbfc067150 0x7fdbfc14e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:13.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.279+0000 7fdc0209b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbfc0ac150 0x7fdbfc147f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:13.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdc0209b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbfc151920 0x7fdbfc164770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:13.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdc0209b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc153020 con 0x7fdbfc0ac150 2023-11-15T03:08:13.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdc0209b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc1508d0 con 0x7fdbfc067150 2023-11-15T03:08:13.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdc0209b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc147d00 con 0x7fdbfc151920 2023-11-15T03:08:13.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdbfaffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbfc0ac150 0x7fdbfc147f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:13.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdbfaffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbfc0ac150 0x7fdbfc147f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48956/0 (socket says 172.21.15.121:48956) 2023-11-15T03:08:13.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdbfaffd700 1 -- 172.21.15.121:0/3362039938 learned_addr learned my addr 172.21.15.121:0/3362039938 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:13.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdbfb7fe700 1 --2- 172.21.15.121:0/3362039938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbfc067150 0x7fdbfc14e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:13.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdbfbfff700 1 --2- 172.21.15.121:0/3362039938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbfc151920 0x7fdbfc164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:13.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdbfaffd700 1 -- 172.21.15.121:0/3362039938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbfc151920 msgr2=0x7fdbfc164770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:13.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdbfaffd700 1 --2- 172.21.15.121:0/3362039938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbfc151920 0x7fdbfc164770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:13.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdbfaffd700 1 -- 172.21.15.121:0/3362039938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbfc067150 msgr2=0x7fdbfc14e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:13.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdbfaffd700 1 --2- 172.21.15.121:0/3362039938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbfc067150 0x7fdbfc14e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:13.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdbfaffd700 1 -- 172.21.15.121:0/3362039938 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbec0092a0 con 0x7fdbfc0ac150 2023-11-15T03:08:13.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdbfaffd700 1 --2- 172.21.15.121:0/3362039938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbfc0ac150 0x7fdbfc147f20 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 rx=0x7fdbf000c440 tx=0x7fdbf0007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:13.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdbe37fe700 1 -- 172.21.15.121:0/3362039938 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0007020 con 0x7fdbfc0ac150 2023-11-15T03:08:13.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdbe37fe700 1 -- 172.21.15.121:0/3362039938 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf0007180 con 0x7fdbfc0ac150 2023-11-15T03:08:13.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdc0209b700 1 -- 172.21.15.121:0/3362039938 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbfc150b60 con 0x7fdbfc0ac150 2023-11-15T03:08:13.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdbe37fe700 1 -- 172.21.15.121:0/3362039938 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0007520 con 0x7fdbfc0ac150 2023-11-15T03:08:13.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdc0209b700 1 -- 172.21.15.121:0/3362039938 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbfc151120 con 0x7fdbfc0ac150 2023-11-15T03:08:13.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.283+0000 7fdc0209b700 1 -- 172.21.15.121:0/3362039938 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbfc047060 con 0x7fdbfc0ac150 2023-11-15T03:08:13.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.287+0000 7fdbe37fe700 1 -- 172.21.15.121:0/3362039938 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fdbf00068b0 con 0x7fdbfc0ac150 2023-11-15T03:08:13.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.287+0000 7fdbe37fe700 1 --2- 172.21.15.121:0/3362039938 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdbe805a510 0x7fdbe805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:13.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.287+0000 7fdbe37fe700 1 -- 172.21.15.121:0/3362039938 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8502+0+0 (secure 0 0 0) 0x7fdbf00744d0 con 0x7fdbfc0ac150 2023-11-15T03:08:13.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.287+0000 7fdbfb7fe700 1 --2- 172.21.15.121:0/3362039938 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdbe805a510 0x7fdbe805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:13.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.287+0000 7fdbfb7fe700 1 --2- 172.21.15.121:0/3362039938 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdbe805a510 0x7fdbe805c9d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7fdbec00ffd0 tx=0x7fdbec017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:13.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.295+0000 7fdbe37fe700 1 -- 172.21.15.121:0/3362039938 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdbf0047320 con 0x7fdbfc0ac150 2023-11-15T03:08:13.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:13.491+0000 7fdc0209b700 1 -- 172.21.15.121:0/3362039938 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdbfc068b20 con 0x7fdbfc0ac150 2023-11-15T03:08:14.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.251+0000 7fdbe37fe700 1 -- 172.21.15.121:0/3362039938 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v242) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdbf004a940 con 0x7fdbfc0ac150 2023-11-15T03:08:14.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.255+0000 7fdc0209b700 1 -- 172.21.15.121:0/3362039938 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdbe805a510 msgr2=0x7fdbe805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:14.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.255+0000 7fdc0209b700 1 --2- 172.21.15.121:0/3362039938 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdbe805a510 0x7fdbe805c9d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7fdbec00ffd0 tx=0x7fdbec017000).stop 2023-11-15T03:08:14.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.255+0000 7fdc0209b700 1 -- 172.21.15.121:0/3362039938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbfc0ac150 msgr2=0x7fdbfc147f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:14.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.255+0000 7fdc0209b700 1 --2- 172.21.15.121:0/3362039938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbfc0ac150 0x7fdbfc147f20 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 rx=0x7fdbf000c440 tx=0x7fdbf0007e00).stop 2023-11-15T03:08:14.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.255+0000 7fdc0209b700 1 -- 172.21.15.121:0/3362039938 shutdown_connections 2023-11-15T03:08:14.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.255+0000 7fdc0209b700 1 --2- 172.21.15.121:0/3362039938 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdbe805a510 0x7fdbe805c9d0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:14.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.255+0000 7fdc0209b700 1 --2- 172.21.15.121:0/3362039938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbfc151920 0x7fdbfc164770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:14.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.255+0000 7fdc0209b700 1 --2- 172.21.15.121:0/3362039938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbfc0ac150 0x7fdbfc147f20 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:14.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.255+0000 7fdc0209b700 1 --2- 172.21.15.121:0/3362039938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbfc067150 0x7fdbfc14e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:14.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.255+0000 7fdc0209b700 1 -- 172.21.15.121:0/3362039938 >> 172.21.15.121:0/3362039938 conn(0x7fdbfc067a60 msgr2=0x7fdbfc0658a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:14.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.255+0000 7fdc0209b700 1 -- 172.21.15.121:0/3362039938 shutdown_connections 2023-11-15T03:08:14.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.255+0000 7fdc0209b700 1 -- 172.21.15.121:0/3362039938 wait complete. 2023-11-15T03:08:14.267 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:08:14.289 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:08:14.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.447+0000 7f96d5b69700 1 Processor -- start 2023-11-15T03:08:14.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.447+0000 7f96d5b69700 1 -- start start 2023-11-15T03:08:14.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96d5b69700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f96d0153aa0 0x7f96d0066680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:14.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96d5b69700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f96d00c0490 0x7f96d00c0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:14.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96d5b69700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d00c5c50 con 0x7f96d0153e80 2023-11-15T03:08:14.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96d5b69700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d0066b70 con 0x7f96d00c0490 2023-11-15T03:08:14.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96d5b69700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d00c0e30 con 0x7f96d0153aa0 2023-11-15T03:08:14.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96ceffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f96d0153e80 0x7f96d00ad150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:46772/0 (socket says 172.21.15.121:46772) 2023-11-15T03:08:14.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96ceffd700 1 -- 172.21.15.121:0/2933916235 learned_addr learned my addr 172.21.15.121:0/2933916235 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:14.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96cffff700 1 --2- 172.21.15.121:0/2933916235 >> v2:172.21.15.121:3300/0 conn(0x7f96d00c0490 0x7f96d00c0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:14.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96cf7fe700 1 --2- 172.21.15.121:0/2933916235 >> v2:172.21.15.190:3300/0 conn(0x7f96d0153aa0 0x7f96d0066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:14.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96ce7fc700 1 -- 172.21.15.121:0/2933916235 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1114149873 0 0) 0x7f96d00c5c50 con 0x7f96d0153e80 2023-11-15T03:08:14.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96ce7fc700 1 -- 172.21.15.121:0/2933916235 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96b80033f0 con 0x7f96d0153e80 2023-11-15T03:08:14.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96cffff700 1 -- 172.21.15.121:0/2933916235 >> v2:172.21.15.190:3300/0 conn(0x7f96d0153aa0 msgr2=0x7f96d0066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:14.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96cffff700 1 --2- 172.21.15.121:0/2933916235 >> v2:172.21.15.190:3300/0 conn(0x7f96d0153aa0 0x7f96d0066680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:14.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96cffff700 1 -- 172.21.15.121:0/2933916235 >> v1:172.21.15.121:6789/0 conn(0x7f96d0153e80 legacy=0x7f96d00ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:14.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96cffff700 1 -- 172.21.15.121:0/2933916235 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d00c5920 con 0x7f96d00c0490 2023-11-15T03:08:14.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96cffff700 1 --2- 172.21.15.121:0/2933916235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96d00c0490 0x7f96d00c0940 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 rx=0x7f96d00cbaf0 tx=0x7f96c8009780).ready entity=mon.0 client_cookie=691896537609e2af server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:14.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96ce7fc700 1 -- 172.21.15.121:0/2933916235 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c800e070 con 0x7f96d00c0490 2023-11-15T03:08:14.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96ce7fc700 1 -- 172.21.15.121:0/2933916235 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c800bab0 con 0x7f96d00c0490 2023-11-15T03:08:14.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96ce7fc700 1 -- 172.21.15.121:0/2933916235 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c800be30 con 0x7f96d00c0490 2023-11-15T03:08:14.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96d5b69700 1 -- 172.21.15.121:0/2933916235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96d00c0490 msgr2=0x7f96d00c0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:14.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96d5b69700 1 --2- 172.21.15.121:0/2933916235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96d00c0490 0x7f96d00c0940 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 rx=0x7f96d00cbaf0 tx=0x7f96c8009780).stop 2023-11-15T03:08:14.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96d5b69700 1 -- 172.21.15.121:0/2933916235 shutdown_connections 2023-11-15T03:08:14.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96d5b69700 1 --2- 172.21.15.121:0/2933916235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96d00c0490 0x7f96d00c0940 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:14.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96d5b69700 1 --2- 172.21.15.121:0/2933916235 >> v2:172.21.15.190:3300/0 conn(0x7f96d0153aa0 0x7f96d0066680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:14.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.451+0000 7f96d5b69700 1 -- 172.21.15.121:0/2933916235 >> 172.21.15.121:0/2933916235 conn(0x7f96d0067a60 msgr2=0x7f96d0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:14.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.455+0000 7f96d5b69700 1 -- 172.21.15.121:0/2933916235 shutdown_connections 2023-11-15T03:08:14.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.455+0000 7f96d5b69700 1 -- 172.21.15.121:0/2933916235 wait complete. 2023-11-15T03:08:14.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.455+0000 7f96d5b69700 1 Processor -- start 2023-11-15T03:08:14.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.455+0000 7f96d5b69700 1 -- start start 2023-11-15T03:08:14.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96d5b69700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96d00c0490 0x7f96d0149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:14.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96d5b69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96d0153aa0 0x7f96d00aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:14.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96d5b69700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d0153e80 0x7f96d00af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:14.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96d5b69700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d00af8a0 con 0x7f96d00c0490 2023-11-15T03:08:14.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96d5b69700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d0155ed0 con 0x7f96d0153e80 2023-11-15T03:08:14.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96d5b69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d00ae640 con 0x7f96d0153aa0 2023-11-15T03:08:14.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96cf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96d00c0490 0x7f96d0149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:14.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96cf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96d00c0490 0x7f96d0149710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48968/0 (socket says 172.21.15.121:48968) 2023-11-15T03:08:14.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96cf7fe700 1 -- 172.21.15.121:0/1613139399 learned_addr learned my addr 172.21.15.121:0/1613139399 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:14.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96cffff700 1 --2- 172.21.15.121:0/1613139399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d0153e80 0x7f96d00af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:14.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96ceffd700 1 --2- 172.21.15.121:0/1613139399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96d0153aa0 0x7f96d00aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:14.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96cf7fe700 1 -- 172.21.15.121:0/1613139399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96d0153aa0 msgr2=0x7f96d00aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:14.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96cf7fe700 1 --2- 172.21.15.121:0/1613139399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96d0153aa0 0x7f96d00aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:14.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96cf7fe700 1 -- 172.21.15.121:0/1613139399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d0153e80 msgr2=0x7f96d00af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:14.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96cf7fe700 1 --2- 172.21.15.121:0/1613139399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d0153e80 0x7f96d00af220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:14.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96cf7fe700 1 -- 172.21.15.121:0/1613139399 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c8009a20 con 0x7f96d00c0490 2023-11-15T03:08:14.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96cf7fe700 1 --2- 172.21.15.121:0/1613139399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96d00c0490 0x7f96d0149710 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 rx=0x7f96c000e930 tx=0x7f96c0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:14.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96b77fe700 1 -- 172.21.15.121:0/1613139399 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c000c670 con 0x7f96d00c0490 2023-11-15T03:08:14.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96b77fe700 1 -- 172.21.15.121:0/1613139399 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c000c7d0 con 0x7f96d00c0490 2023-11-15T03:08:14.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96d5b69700 1 -- 172.21.15.121:0/1613139399 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d00aead0 con 0x7f96d00c0490 2023-11-15T03:08:14.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96b77fe700 1 -- 172.21.15.121:0/1613139399 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c000cb50 con 0x7f96d00c0490 2023-11-15T03:08:14.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96d5b69700 1 -- 172.21.15.121:0/1613139399 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d00adff0 con 0x7f96d00c0490 2023-11-15T03:08:14.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.459+0000 7f96d5b69700 1 -- 172.21.15.121:0/1613139399 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d0047060 con 0x7f96d00c0490 2023-11-15T03:08:14.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.463+0000 7f96b77fe700 1 -- 172.21.15.121:0/1613139399 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f96c001b400 con 0x7f96d00c0490 2023-11-15T03:08:14.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.463+0000 7f96b77fe700 1 --2- 172.21.15.121:0/1613139399 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f96bc05a540 0x7f96bc05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:14.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.463+0000 7f96b77fe700 1 -- 172.21.15.121:0/1613139399 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8502+0+0 (secure 0 0 0) 0x7f96c0075650 con 0x7f96d00c0490 2023-11-15T03:08:14.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.463+0000 7f96ceffd700 1 --2- 172.21.15.121:0/1613139399 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f96bc05a540 0x7f96bc05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:14.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.463+0000 7f96ceffd700 1 --2- 172.21.15.121:0/1613139399 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f96bc05a540 0x7f96bc05ca00 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f96d00acb80 tx=0x7f96c4009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:14.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.471+0000 7f96b77fe700 1 -- 172.21.15.121:0/1613139399 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f96c00483c0 con 0x7f96d00c0490 2023-11-15T03:08:14.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:14.687+0000 7f96d5b69700 1 -- 172.21.15.121:0/1613139399 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f96d0068b20 con 0x7f96d00c0490 2023-11-15T03:08:15.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.263+0000 7f96b77fe700 1 -- 172.21.15.121:0/1613139399 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v243) v1 ==== 101+0+0 (secure 0 0 0) 0x7f96c004b9e0 con 0x7f96d00c0490 2023-11-15T03:08:15.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.267+0000 7f96d5b69700 1 -- 172.21.15.121:0/1613139399 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f96bc05a540 msgr2=0x7f96bc05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.267+0000 7f96d5b69700 1 --2- 172.21.15.121:0/1613139399 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f96bc05a540 0x7f96bc05ca00 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f96d00acb80 tx=0x7f96c4009040).stop 2023-11-15T03:08:15.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.267+0000 7f96d5b69700 1 -- 172.21.15.121:0/1613139399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96d00c0490 msgr2=0x7f96d0149710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.267+0000 7f96d5b69700 1 --2- 172.21.15.121:0/1613139399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96d00c0490 0x7f96d0149710 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 rx=0x7f96c000e930 tx=0x7f96c0009060).stop 2023-11-15T03:08:15.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.267+0000 7f96d5b69700 1 -- 172.21.15.121:0/1613139399 shutdown_connections 2023-11-15T03:08:15.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.267+0000 7f96d5b69700 1 --2- 172.21.15.121:0/1613139399 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f96bc05a540 0x7f96bc05ca00 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.267+0000 7f96d5b69700 1 --2- 172.21.15.121:0/1613139399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d0153e80 0x7f96d00af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.267+0000 7f96d5b69700 1 --2- 172.21.15.121:0/1613139399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96d0153aa0 0x7f96d00aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.267+0000 7f96d5b69700 1 --2- 172.21.15.121:0/1613139399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96d00c0490 0x7f96d0149710 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.267+0000 7f96d5b69700 1 -- 172.21.15.121:0/1613139399 >> 172.21.15.121:0/1613139399 conn(0x7f96d0067a60 msgr2=0x7f96d00658c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:15.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.267+0000 7f96d5b69700 1 -- 172.21.15.121:0/1613139399 shutdown_connections 2023-11-15T03:08:15.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.267+0000 7f96d5b69700 1 -- 172.21.15.121:0/1613139399 wait complete. 2023-11-15T03:08:15.279 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:08:15.297 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:08:15.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.347+0000 7f8c8a32fd00 1 Processor -- start 2023-11-15T03:08:15.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c8a32fd00 1 -- start start 2023-11-15T03:08:15.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c8a32fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5e929da30 0x55a5e92e31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c8a32fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5e92ecb20 0x55a5e937bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c8a32fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5e937e000 0x55a5e937c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c8a32fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e92ccf20 con 0x55a5e92ecb20 2023-11-15T03:08:15.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c8a32fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e92ea430 con 0x55a5e929da30 2023-11-15T03:08:15.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c8a32fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5e9374680 con 0x55a5e937e000 2023-11-15T03:08:15.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c77fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5e92ecb20 0x55a5e937bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c77fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5e92ecb20 0x55a5e937bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51870/0 (socket says 172.21.15.121:51870) 2023-11-15T03:08:15.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c77fff700 1 -- 172.21.15.121:0/267546949 learned_addr learned my addr 172.21.15.121:0/267546949 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:15.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c7e915700 1 --2- 172.21.15.121:0/267546949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5e929da30 0x55a5e92e31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c7f116700 1 --2- 172.21.15.121:0/267546949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5e937e000 0x55a5e937c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c77fff700 1 -- 172.21.15.121:0/267546949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5e937e000 msgr2=0x55a5e937c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:15.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c77fff700 1 --2- 172.21.15.121:0/267546949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5e937e000 0x55a5e937c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c77fff700 1 -- 172.21.15.121:0/267546949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5e929da30 msgr2=0x55a5e92e31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:15.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c77fff700 1 --2- 172.21.15.121:0/267546949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5e929da30 0x55a5e92e31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c77fff700 1 -- 172.21.15.121:0/267546949 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7800a180 con 0x55a5e92ecb20 2023-11-15T03:08:15.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c7f116700 1 --2- 172.21.15.121:0/267546949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5e937e000 0x55a5e937c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:08:15.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c77fff700 1 --2- 172.21.15.121:0/267546949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5e92ecb20 0x55a5e937bc70 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 rx=0x7f8c6800a810 tx=0x7f8c68005be0).ready entity=mon.0 client_cookie=9f79308c9ce29ee2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:15.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c7d913700 1 -- 172.21.15.121:0/267546949 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6800f070 con 0x55a5e92ecb20 2023-11-15T03:08:15.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c7d913700 1 -- 172.21.15.121:0/267546949 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c68008760 con 0x55a5e92ecb20 2023-11-15T03:08:15.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c7d913700 1 -- 172.21.15.121:0/267546949 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c68008ae0 con 0x55a5e92ecb20 2023-11-15T03:08:15.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/267546949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5e92ecb20 msgr2=0x55a5e937bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/267546949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5e92ecb20 0x55a5e937bc70 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 rx=0x7f8c6800a810 tx=0x7f8c68005be0).stop 2023-11-15T03:08:15.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/267546949 shutdown_connections 2023-11-15T03:08:15.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/267546949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5e937e000 0x55a5e937c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/267546949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5e92ecb20 0x55a5e937bc70 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/267546949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5e929da30 0x55a5e92e31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/267546949 >> 172.21.15.121:0/267546949 conn(0x55a5e92ed080 msgr2=0x55a5e92e9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:15.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/267546949 shutdown_connections 2023-11-15T03:08:15.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.351+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/267546949 wait complete. 2023-11-15T03:08:15.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c8a32fd00 1 Processor -- start 2023-11-15T03:08:15.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c8a32fd00 1 -- start start 2023-11-15T03:08:15.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c8a32fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5e929da30 0x55a5e93750e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c8a32fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5e92ecb20 0x55a5e92e6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c8a32fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5e937e000 0x55a5e93728e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c8a32fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e9380750 con 0x55a5e937e000 2023-11-15T03:08:15.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c8a32fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e9380d10 con 0x55a5e929da30 2023-11-15T03:08:15.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c8a32fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5e92e82d0 con 0x55a5e92ecb20 2023-11-15T03:08:15.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c7f116700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5e937e000 0x55a5e93728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c7f116700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5e937e000 0x55a5e93728e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51884/0 (socket says 172.21.15.121:51884) 2023-11-15T03:08:15.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 learned_addr learned my addr 172.21.15.121:0/2196808373 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:15.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c7e915700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5e929da30 0x55a5e93750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c77fff700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5e92ecb20 0x55a5e92e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5e92ecb20 msgr2=0x55a5e92e6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c7f116700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5e92ecb20 0x55a5e92e6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5e929da30 msgr2=0x55a5e93750e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c7f116700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5e929da30 0x55a5e93750e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6800a440 con 0x55a5e937e000 2023-11-15T03:08:15.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c77fff700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5e92ecb20 0x55a5e92e6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:15.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.355+0000 7f8c7f116700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5e937e000 0x55a5e93728e0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 rx=0x7f8c7800acb0 tx=0x7f8c78014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:15.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.359+0000 7f8c777fe700 1 -- 172.21.15.121:0/2196808373 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78005b30 con 0x55a5e937e000 2023-11-15T03:08:15.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.359+0000 7f8c777fe700 1 -- 172.21.15.121:0/2196808373 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7800cd30 con 0x55a5e937e000 2023-11-15T03:08:15.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.359+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a5e923fa20 con 0x55a5e937e000 2023-11-15T03:08:15.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.359+0000 7f8c777fe700 1 -- 172.21.15.121:0/2196808373 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800c9a0 con 0x55a5e937e000 2023-11-15T03:08:15.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.359+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5e923f770 con 0x55a5e937e000 2023-11-15T03:08:15.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.359+0000 7f8c777fe700 1 -- 172.21.15.121:0/2196808373 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8c78003040 con 0x55a5e937e000 2023-11-15T03:08:15.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.363+0000 7f8c777fe700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8c60059040 0x7f8c6005b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.363+0000 7f8c777fe700 1 -- 172.21.15.121:0/2196808373 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8502+0+0 (secure 0 0 0) 0x7f8c78075c30 con 0x55a5e937e000 2023-11-15T03:08:15.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.363+0000 7f8c7e915700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8c60059040 0x7f8c6005b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.383 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:08:15.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.363+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55a5e93811a0 0x55a5e9381580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.363+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e9381a70 con 0x55a5e93811a0 2023-11-15T03:08:15.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.363+0000 7f8c7e915700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8c60059040 0x7f8c6005b500 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f8c70009c20 tx=0x7f8c7000c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:15.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.363+0000 7f8c77fff700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55a5e93811a0 0x55a5e9381580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.363+0000 7f8c77fff700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55a5e93811a0 0x55a5e9381580 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:15.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.367+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v120'305 uv298 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93811a0 2023-11-15T03:08:15.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.367+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93976e0 con 0x55a5e93811a0 2023-11-15T03:08:15.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.371+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v120'306 uv306 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93811a0 2023-11-15T03:08:15.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.371+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e9397a90 con 0x55a5e93811a0 2023-11-15T03:08:15.386 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:08:15.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.375+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v120'307 uv307 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93811a0 2023-11-15T03:08:15.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.375+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e9398e50 con 0x55a5e93811a0 2023-11-15T03:08:15.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.375+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v120'308 uv308 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93811a0 2023-11-15T03:08:15.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.375+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e9399200 con 0x55a5e93811a0 2023-11-15T03:08:15.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.379+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v120'309 uv309 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93811a0 2023-11-15T03:08:15.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.379+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e9397e40 con 0x55a5e93811a0 2023-11-15T03:08:15.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.383+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v120'310 uv310 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93811a0 2023-11-15T03:08:15.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.383+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93981f0 con 0x55a5e93811a0 2023-11-15T03:08:15.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.383+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v120'311 uv304 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93811a0 2023-11-15T03:08:15.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.383+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93985f0 con 0x55a5e93811a0 2023-11-15T03:08:15.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.387+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v120'312 uv312 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93811a0 2023-11-15T03:08:15.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.387+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93985f0 con 0x55a5e93811a0 2023-11-15T03:08:15.398 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:08:15.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.391+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v120'313 uv313 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93811a0 2023-11-15T03:08:15.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.391+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93985a0 con 0x55a5e93811a0 2023-11-15T03:08:15.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.391+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v120'314 uv314 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93811a0 2023-11-15T03:08:15.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.391+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93985a0 con 0x55a5e93811a0 2023-11-15T03:08:15.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.395+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v120'315 uv315 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93811a0 2023-11-15T03:08:15.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.395+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93985a0 con 0x55a5e93811a0 2023-11-15T03:08:15.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.395+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v120'316 uv316 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93811a0 2023-11-15T03:08:15.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.395+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55a5e93995b0 0x55a5e939b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.395+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e939be90 con 0x55a5e93995b0 2023-11-15T03:08:15.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.399+0000 7f8c7e915700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55a5e93995b0 0x55a5e939b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.399+0000 7f8c7e915700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55a5e93995b0 0x55a5e939b9a0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:15.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.403+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v120'480 uv467 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.403+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e939c240 con 0x55a5e93995b0 2023-11-15T03:08:15.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.407+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v120'481 uv481 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.407+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e939c5f0 con 0x55a5e93995b0 2023-11-15T03:08:15.419 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:08:15.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.411+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v120'482 uv482 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.411+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e939d9b0 con 0x55a5e93995b0 2023-11-15T03:08:15.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.411+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v120'483 uv483 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.411+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e939dd60 con 0x55a5e93995b0 2023-11-15T03:08:15.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.415+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v120'484 uv484 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.415+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e939c9a0 con 0x55a5e93995b0 2023-11-15T03:08:15.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.419+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v120'485 uv485 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.419+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e939cda0 con 0x55a5e93995b0 2023-11-15T03:08:15.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.423+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v120'486 uv486 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.423+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e939d1c0 con 0x55a5e93995b0 2023-11-15T03:08:15.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.423+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v120'487 uv473 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.423+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e939d1c0 con 0x55a5e93995b0 2023-11-15T03:08:15.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.427+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v120'488 uv488 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.427+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e939d5c0 con 0x55a5e93995b0 2023-11-15T03:08:15.439 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:08:15.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.431+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v120'489 uv489 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.431+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e939d150 con 0x55a5e93995b0 2023-11-15T03:08:15.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.431+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v120'490 uv490 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.431+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e939d150 con 0x55a5e93995b0 2023-11-15T03:08:15.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.435+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v120'491 uv491 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.435+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e939d150 con 0x55a5e93995b0 2023-11-15T03:08:15.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.439+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v120'492 uv492 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.439+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e939d150 con 0x55a5e93995b0 2023-11-15T03:08:15.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.443+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v120'493 uv479 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.443+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e939d150 con 0x55a5e93995b0 2023-11-15T03:08:15.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.443+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v120'494 uv494 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.443+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e939d150 con 0x55a5e93995b0 2023-11-15T03:08:15.456 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:08:15.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.447+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v120'495 uv495 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.447+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e939d150 con 0x55a5e93995b0 2023-11-15T03:08:15.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.451+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v120'496 uv496 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.451+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e939d150 con 0x55a5e93995b0 2023-11-15T03:08:15.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.455+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v120'497 uv497 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.455+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e939d150 con 0x55a5e93995b0 2023-11-15T03:08:15.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.455+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v120'498 uv498 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.455+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a5e939e110 0x55a5e93a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.455+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a09f0 con 0x55a5e939e110 2023-11-15T03:08:15.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.459+0000 7f8c77fff700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a5e939e110 0x55a5e93a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.459+0000 7f8c77fff700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a5e939e110 0x55a5e93a0500 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:15.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.463+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v120'263 uv257 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e939e110 2023-11-15T03:08:15.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.463+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93a09f0 con 0x55a5e939e110 2023-11-15T03:08:15.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.463+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v120'264 uv264 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e939e110 2023-11-15T03:08:15.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.463+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a09f0 con 0x55a5e939e110 2023-11-15T03:08:15.476 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:08:15.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.467+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v120'265 uv265 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e939e110 2023-11-15T03:08:15.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.467+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a09f0 con 0x55a5e939e110 2023-11-15T03:08:15.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.471+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v120'266 uv266 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e939e110 2023-11-15T03:08:15.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.471+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a09f0 con 0x55a5e939e110 2023-11-15T03:08:15.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.475+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v120'267 uv267 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e939e110 2023-11-15T03:08:15.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.475+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a09f0 con 0x55a5e939e110 2023-11-15T03:08:15.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.479+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v120'268 uv262 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e939e110 2023-11-15T03:08:15.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.479+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93a09f0 con 0x55a5e939e110 2023-11-15T03:08:15.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.479+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v120'269 uv269 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e939e110 2023-11-15T03:08:15.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.479+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a09f0 con 0x55a5e939e110 2023-11-15T03:08:15.492 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:08:15.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.483+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v120'270 uv270 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e939e110 2023-11-15T03:08:15.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.483+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a09f0 con 0x55a5e939e110 2023-11-15T03:08:15.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.487+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v120'271 uv271 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e939e110 2023-11-15T03:08:15.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.487+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a09f0 con 0x55a5e939e110 2023-11-15T03:08:15.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.491+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v120'272 uv272 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e939e110 2023-11-15T03:08:15.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.491+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55a5e93a0f00 0x55a5e93a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.491+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a37e0 con 0x55a5e93a0f00 2023-11-15T03:08:15.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.491+0000 7f8c7f116700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55a5e93a0f00 0x55a5e93a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.491+0000 7f8c7f116700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55a5e93a0f00 0x55a5e93a32f0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:15.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.495+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v120'308 uv307 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.495+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93a3b90 con 0x55a5e93a0f00 2023-11-15T03:08:15.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.499+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v120'309 uv309 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.499+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a3f40 con 0x55a5e93a0f00 2023-11-15T03:08:15.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.499+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v120'310 uv310 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.510 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:08:15.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.499+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e92e8d40 con 0x55a5e93a0f00 2023-11-15T03:08:15.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.503+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v120'311 uv311 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.503+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a89c0 con 0x55a5e93a0f00 2023-11-15T03:08:15.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.507+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v120'312 uv312 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.507+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a8550 con 0x55a5e93a0f00 2023-11-15T03:08:15.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.511+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v120'313 uv306 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.511+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93a4830 con 0x55a5e93a0f00 2023-11-15T03:08:15.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.515+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v120'314 uv314 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.515+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a4cb0 con 0x55a5e93a0f00 2023-11-15T03:08:15.525 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:08:15.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.515+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v120'315 uv315 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.515+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a4cb0 con 0x55a5e93a0f00 2023-11-15T03:08:15.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.519+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v120'316 uv316 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.519+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a60c0 con 0x55a5e93a0f00 2023-11-15T03:08:15.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.523+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v120'317 uv317 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.523+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a4cc0 con 0x55a5e93a0f00 2023-11-15T03:08:15.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.527+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v120'318 uv318 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.527+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55a5e93a55d0 0x55a5e93a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.527+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a5ea0 con 0x55a5e93a55d0 2023-11-15T03:08:15.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.527+0000 7f8c7e915700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55a5e93a55d0 0x55a5e93a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.527+0000 7f8c7e915700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55a5e93a55d0 0x55a5e93a59b0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:15.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.531+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v120'153 uv152 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93a55d0 2023-11-15T03:08:15.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.531+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93a5ea0 con 0x55a5e93a55d0 2023-11-15T03:08:15.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.531+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v120'154 uv154 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93a55d0 2023-11-15T03:08:15.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.531+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a5ea0 con 0x55a5e93a55d0 2023-11-15T03:08:15.546 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:08:15.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.535+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v120'155 uv155 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93a55d0 2023-11-15T03:08:15.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.535+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a9dd0 con 0x55a5e93a55d0 2023-11-15T03:08:15.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.539+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v120'156 uv156 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93a55d0 2023-11-15T03:08:15.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.539+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a9dd0 con 0x55a5e93a55d0 2023-11-15T03:08:15.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.539+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v120'157 uv157 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93a55d0 2023-11-15T03:08:15.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.539+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93a8dc0 con 0x55a5e93a55d0 2023-11-15T03:08:15.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.547+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v120'158 uv158 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93a55d0 2023-11-15T03:08:15.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.547+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a5e93a92d0 0x55a5e93ab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.547+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93abcc0 con 0x55a5e93a92d0 2023-11-15T03:08:15.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.547+0000 7f8c77fff700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a5e93a92d0 0x55a5e93ab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.547+0000 7f8c77fff700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a5e93a92d0 0x55a5e93ab6c0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:15.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.551+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v120'108 uv107 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.551+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93ac0a0 con 0x55a5e93a92d0 2023-11-15T03:08:15.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.551+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v120'109 uv109 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.551+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac0a0 con 0x55a5e93a92d0 2023-11-15T03:08:15.563 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:08:15.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.555+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v120'110 uv110 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.555+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac0a0 con 0x55a5e93a92d0 2023-11-15T03:08:15.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.555+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v120'111 uv111 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.555+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93a92d0 2023-11-15T03:08:15.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.559+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v120'153 uv152 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.559+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93a92d0 2023-11-15T03:08:15.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.563+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v120'154 uv154 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.563+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93a92d0 2023-11-15T03:08:15.574 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:08:15.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.567+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v120'155 uv155 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.567+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ad020 con 0x55a5e93a92d0 2023-11-15T03:08:15.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.567+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v120'156 uv156 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.567+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ad020 con 0x55a5e93a92d0 2023-11-15T03:08:15.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.571+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v120'157 uv157 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.571+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93a92d0 2023-11-15T03:08:15.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.575+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v120'158 uv158 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.575+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93995b0 2023-11-15T03:08:15.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.575+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v120'282 uv276 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.575+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93995b0 2023-11-15T03:08:15.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.579+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v120'283 uv283 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.579+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93995b0 2023-11-15T03:08:15.593 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:08:15.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.583+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v120'284 uv284 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.583+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93995b0 2023-11-15T03:08:15.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.587+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v120'285 uv285 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.587+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93995b0 2023-11-15T03:08:15.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.591+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v120'286 uv286 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.591+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93995b0 2023-11-15T03:08:15.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.595+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v120'287 uv287 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.595+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93995b0 2023-11-15T03:08:15.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.595+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v120'288 uv281 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.595+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93995b0 2023-11-15T03:08:15.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.599+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v120'289 uv289 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.599+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93995b0 2023-11-15T03:08:15.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.603+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v120'290 uv290 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.612 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:08:15.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.603+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93995b0 2023-11-15T03:08:15.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.607+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v120'291 uv291 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.607+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93995b0 2023-11-15T03:08:15.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.611+0000 7f8c7e915700 1 -- 172.21.15.121:0/2196808373 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v120'292 uv292 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c70016450 con 0x55a5e93995b0 2023-11-15T03:08:15.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.611+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e939e110 2023-11-15T03:08:15.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.615+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v120'127 uv126 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e939e110 2023-11-15T03:08:15.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.615+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e939e110 2023-11-15T03:08:15.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.615+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v120'128 uv128 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e939e110 2023-11-15T03:08:15.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.615+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e939e110 2023-11-15T03:08:15.628 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:08:15.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.619+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v120'129 uv129 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e939e110 2023-11-15T03:08:15.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.619+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e939e110 2023-11-15T03:08:15.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.623+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v120'130 uv130 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e939e110 2023-11-15T03:08:15.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.623+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e939e110 2023-11-15T03:08:15.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.623+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v120'131 uv131 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e939e110 2023-11-15T03:08:15.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.627+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93a92d0 2023-11-15T03:08:15.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.627+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v120'153 uv152 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.627+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93a92d0 2023-11-15T03:08:15.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.631+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v120'154 uv154 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.631+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93a92d0 2023-11-15T03:08:15.642 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:08:15.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.635+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v120'155 uv155 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.635+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93a92d0 2023-11-15T03:08:15.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.635+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v120'156 uv156 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.635+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93a92d0 2023-11-15T03:08:15.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.639+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v120'157 uv157 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.639+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93a92d0 2023-11-15T03:08:15.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.643+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v120'158 uv158 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.643+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93a92d0 2023-11-15T03:08:15.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.643+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v120'128 uv127 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.643+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93a92d0 2023-11-15T03:08:15.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.647+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v120'129 uv129 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.647+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93a92d0 2023-11-15T03:08:15.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.651+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v120'130 uv130 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.660 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:08:15.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.651+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93a92d0 2023-11-15T03:08:15.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.655+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v120'131 uv131 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.655+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac430 con 0x55a5e93a92d0 2023-11-15T03:08:15.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.655+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v120'132 uv132 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.659+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac430 con 0x55a5e93a92d0 2023-11-15T03:08:15.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.659+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v120'126 uv125 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.659+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93a92d0 2023-11-15T03:08:15.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.663+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v120'127 uv127 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.663+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac010 con 0x55a5e93a92d0 2023-11-15T03:08:15.674 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:08:15.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.667+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v120'128 uv128 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.667+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93ac480 con 0x55a5e93a92d0 2023-11-15T03:08:15.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.667+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v120'129 uv129 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.667+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93af1f0 con 0x55a5e93a92d0 2023-11-15T03:08:15.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.671+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v120'130 uv130 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.671+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55a5e93ace30 0x55a5e93af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.671+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93ace30 2023-11-15T03:08:15.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.671+0000 7f8c7f116700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55a5e93ace30 0x55a5e93af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.675+0000 7f8c7f116700 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55a5e93ace30 0x55a5e93af220 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:15.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.675+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v120'102 uv101 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93ace30 2023-11-15T03:08:15.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.675+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93ace30 2023-11-15T03:08:15.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.679+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v120'103 uv103 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93ace30 2023-11-15T03:08:15.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.679+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93ace30 2023-11-15T03:08:15.691 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:08:15.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.683+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v120'104 uv104 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93ace30 2023-11-15T03:08:15.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.683+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93ace30 2023-11-15T03:08:15.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.687+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v120'105 uv105 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93ace30 2023-11-15T03:08:15.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.687+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93a92d0 2023-11-15T03:08:15.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.687+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v120'153 uv152 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.687+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93a92d0 2023-11-15T03:08:15.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.691+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v120'154 uv154 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.691+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93a92d0 2023-11-15T03:08:15.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.695+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v120'155 uv155 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.703 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:08:15.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.695+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93b0720 con 0x55a5e93a92d0 2023-11-15T03:08:15.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.695+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v120'156 uv156 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.699+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93b0720 con 0x55a5e93a92d0 2023-11-15T03:08:15.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.699+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v120'157 uv157 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.699+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93a92d0 2023-11-15T03:08:15.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.703+0000 7f8c77fff700 1 -- 172.21.15.121:0/2196808373 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v120'158 uv158 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c68008950 con 0x55a5e93a92d0 2023-11-15T03:08:15.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.703+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93a0f00 2023-11-15T03:08:15.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.707+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v120'130 uv129 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.707+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93a0f00 2023-11-15T03:08:15.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.711+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v120'131 uv131 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.711+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93a0f00 2023-11-15T03:08:15.720 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:08:15.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.711+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v120'132 uv132 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.711+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93a0f00 2023-11-15T03:08:15.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.715+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v120'133 uv133 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.715+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93a0f00 2023-11-15T03:08:15.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.719+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v120'134 uv134 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.719+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93a0f00 2023-11-15T03:08:15.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.719+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v120'305 uv297 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.719+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93a0f00 2023-11-15T03:08:15.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.723+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v120'306 uv306 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.723+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93a0f00 2023-11-15T03:08:15.735 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:08:15.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.727+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v120'307 uv307 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.727+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93a0f00 2023-11-15T03:08:15.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.727+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v120'308 uv308 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.727+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93a0f00 2023-11-15T03:08:15.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.731+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v120'309 uv309 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.731+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93a0f00 2023-11-15T03:08:15.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.735+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v120'310 uv304 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.735+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93a0f00 2023-11-15T03:08:15.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.739+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v120'311 uv311 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.739+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93a0f00 2023-11-15T03:08:15.749 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:08:15.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.739+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v120'312 uv312 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.739+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93b0720 con 0x55a5e93a0f00 2023-11-15T03:08:15.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.743+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v120'313 uv313 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.743+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93b0720 con 0x55a5e93a0f00 2023-11-15T03:08:15.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.747+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v120'314 uv314 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.747+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93af710 con 0x55a5e93a0f00 2023-11-15T03:08:15.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.747+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v120'315 uv315 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.747+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55a5e93afc00 con 0x55a5e93a0f00 2023-11-15T03:08:15.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v120'316 uv316 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8c78022020 con 0x55a5e93a0f00 2023-11-15T03:08:15.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a5e939e110 msgr2=0x55a5e93a0500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a5e939e110 0x55a5e93a0500 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55a5e93811a0 msgr2=0x55a5e9381580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55a5e93811a0 0x55a5e9381580 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55a5e93a55d0 msgr2=0x55a5e93a59b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55a5e93a55d0 0x55a5e93a59b0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55a5e93995b0 msgr2=0x55a5e939b9a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55a5e93995b0 0x55a5e939b9a0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55a5e93ace30 msgr2=0x55a5e93af220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55a5e93ace30 0x55a5e93af220 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 reap_dead start 2023-11-15T03:08:15.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55a5e93a0f00 msgr2=0x55a5e93a32f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55a5e93a0f00 0x55a5e93a32f0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a5e93a92d0 msgr2=0x55a5e93ab6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c7f116700 1 -- 172.21.15.121:0/2196808373 reap_dead start 2023-11-15T03:08:15.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a5e93a92d0 0x55a5e93ab6c0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8c60059040 msgr2=0x7f8c6005b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8c60059040 0x7f8c6005b500 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f8c70009c20 tx=0x7f8c7000c040).stop 2023-11-15T03:08:15.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5e937e000 msgr2=0x55a5e93728e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5e937e000 0x55a5e93728e0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 rx=0x7f8c7800acb0 tx=0x7f8c78014000).stop 2023-11-15T03:08:15.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 shutdown_connections 2023-11-15T03:08:15.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55a5e93a92d0 0x55a5e93ab6c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8c60059040 0x7f8c6005b500 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 --2- 172.21.15.121:0/2196808373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5e937e000 0x55a5e93728e0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 >> 172.21.15.121:0/2196808373 conn(0x55a5e92ed080 msgr2=0x55a5e92e31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:15.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 shutdown_connections 2023-11-15T03:08:15.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.751+0000 7f8c8a32fd00 1 -- 172.21.15.121:0/2196808373 wait complete. 2023-11-15T03:08:15.768 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 500.00000000 0 2023-11-15T03:08:15.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.863+0000 7f9cc4b32d00 1 Processor -- start 2023-11-15T03:08:15.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.863+0000 7f9cc4b32d00 1 -- start start 2023-11-15T03:08:15.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.863+0000 7f9cc4b32d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b487c93dd0 0x55b487d66260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.863+0000 7f9cc4b32d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b487cddb00 0x55b487cd8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.863+0000 7f9cc4b32d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b487d6c670 0x55b487cd94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.863+0000 7f9cc4b32d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b487c0c220 con 0x55b487c93dd0 2023-11-15T03:08:15.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.863+0000 7f9cc4b32d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b487cd99d0 con 0x55b487cddb00 2023-11-15T03:08:15.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.863+0000 7f9cc4b32d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b487ce16c0 con 0x55b487d6c670 2023-11-15T03:08:15.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.863+0000 7f9cb923c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b487c93dd0 0x55b487d66260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.863+0000 7f9cb923c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b487c93dd0 0x55b487d66260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51898/0 (socket says 172.21.15.121:51898) 2023-11-15T03:08:15.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.863+0000 7f9cb923c700 1 -- 172.21.15.121:0/1346314136 learned_addr learned my addr 172.21.15.121:0/1346314136 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:15.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.863+0000 7f9cb9a3d700 1 --2- 172.21.15.121:0/1346314136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b487d6c670 0x55b487cd94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.863+0000 7f9cb0a3b700 1 --2- 172.21.15.121:0/1346314136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b487cddb00 0x55b487cd8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cb923c700 1 -- 172.21.15.121:0/1346314136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b487d6c670 msgr2=0x55b487cd94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:15.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cb923c700 1 --2- 172.21.15.121:0/1346314136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b487d6c670 0x55b487cd94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cb923c700 1 -- 172.21.15.121:0/1346314136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b487cddb00 msgr2=0x55b487cd8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:15.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cb923c700 1 --2- 172.21.15.121:0/1346314136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b487cddb00 0x55b487cd8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cb923c700 1 -- 172.21.15.121:0/1346314136 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac005cf0 con 0x55b487c93dd0 2023-11-15T03:08:15.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cb923c700 1 --2- 172.21.15.121:0/1346314136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b487c93dd0 0x55b487d66260 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 rx=0x55b487c80e40 tx=0x7f9cac016000).ready entity=mon.0 client_cookie=2425627f14361e16 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:15.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cb3fff700 1 -- 172.21.15.121:0/1346314136 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac0086a0 con 0x55b487c93dd0 2023-11-15T03:08:15.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cb3fff700 1 -- 172.21.15.121:0/1346314136 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cac000bc0 con 0x55b487c93dd0 2023-11-15T03:08:15.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cb3fff700 1 -- 172.21.15.121:0/1346314136 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac008970 con 0x55b487c93dd0 2023-11-15T03:08:15.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cc4b32d00 1 -- 172.21.15.121:0/1346314136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b487c93dd0 msgr2=0x55b487d66260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cc4b32d00 1 --2- 172.21.15.121:0/1346314136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b487c93dd0 0x55b487d66260 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 rx=0x55b487c80e40 tx=0x7f9cac016000).stop 2023-11-15T03:08:15.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cc4b32d00 1 -- 172.21.15.121:0/1346314136 shutdown_connections 2023-11-15T03:08:15.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cc4b32d00 1 --2- 172.21.15.121:0/1346314136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b487d6c670 0x55b487cd94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cc4b32d00 1 --2- 172.21.15.121:0/1346314136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b487cddb00 0x55b487cd8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cc4b32d00 1 --2- 172.21.15.121:0/1346314136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b487c93dd0 0x55b487d66260 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cc4b32d00 1 -- 172.21.15.121:0/1346314136 >> 172.21.15.121:0/1346314136 conn(0x55b487ce3080 msgr2=0x55b487c935f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:15.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cc4b32d00 1 -- 172.21.15.121:0/1346314136 shutdown_connections 2023-11-15T03:08:15.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cc4b32d00 1 -- 172.21.15.121:0/1346314136 wait complete. 2023-11-15T03:08:15.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cc4b32d00 1 Processor -- start 2023-11-15T03:08:15.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.867+0000 7f9cc4b32d00 1 -- start start 2023-11-15T03:08:15.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cc4b32d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b487c93dd0 0x55b487d62a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cc4b32d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b487cddb00 0x55b487cdc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cc4b32d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b487d6c670 0x55b487cdc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cc4b32d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b487d6c0b0 con 0x55b487c93dd0 2023-11-15T03:08:15.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cc4b32d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b487cdce30 con 0x55b487d6c670 2023-11-15T03:08:15.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cc4b32d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b487cc33b0 con 0x55b487cddb00 2023-11-15T03:08:15.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cb923c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b487c93dd0 0x55b487d62a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cb923c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b487c93dd0 0x55b487d62a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51904/0 (socket says 172.21.15.121:51904) 2023-11-15T03:08:15.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cb923c700 1 -- 172.21.15.121:0/1484176702 learned_addr learned my addr 172.21.15.121:0/1484176702 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:15.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cb9a3d700 1 --2- 172.21.15.121:0/1484176702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b487d6c670 0x55b487cdc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cb0a3b700 1 --2- 172.21.15.121:0/1484176702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b487cddb00 0x55b487cdc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cb923c700 1 -- 172.21.15.121:0/1484176702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b487cddb00 msgr2=0x55b487cdc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cb923c700 1 --2- 172.21.15.121:0/1484176702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b487cddb00 0x55b487cdc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cb923c700 1 -- 172.21.15.121:0/1484176702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b487d6c670 msgr2=0x55b487cdc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cb923c700 1 --2- 172.21.15.121:0/1484176702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b487d6c670 0x55b487cdc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cb923c700 1 -- 172.21.15.121:0/1484176702 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b487c35a20 con 0x55b487c93dd0 2023-11-15T03:08:15.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cb9a3d700 1 --2- 172.21.15.121:0/1484176702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b487d6c670 0x55b487cdc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:15.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cb923c700 1 --2- 172.21.15.121:0/1484176702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b487c93dd0 0x55b487d62a80 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 rx=0x7f9cac00c010 tx=0x7f9cac01c000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:15.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cb27fc700 1 -- 172.21.15.121:0/1484176702 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac013720 con 0x55b487c93dd0 2023-11-15T03:08:15.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cb27fc700 1 -- 172.21.15.121:0/1484176702 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cac013880 con 0x55b487c93dd0 2023-11-15T03:08:15.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cc4b32d00 1 -- 172.21.15.121:0/1484176702 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b487c35770 con 0x55b487c93dd0 2023-11-15T03:08:15.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cb27fc700 1 -- 172.21.15.121:0/1484176702 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac013c40 con 0x55b487c93dd0 2023-11-15T03:08:15.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.871+0000 7f9cc4b32d00 1 -- 172.21.15.121:0/1484176702 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b487c04a70 con 0x55b487c93dd0 2023-11-15T03:08:15.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.875+0000 7f9cb27fc700 1 -- 172.21.15.121:0/1484176702 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9cac02c070 con 0x55b487c93dd0 2023-11-15T03:08:15.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.879+0000 7f9cb27fc700 1 --2- 172.21.15.121:0/1484176702 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9c9c059020 0x7f9c9c05b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.879+0000 7f9cb27fc700 1 -- 172.21.15.121:0/1484176702 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8502+0+0 (secure 0 0 0) 0x7f9cac076f10 con 0x55b487c93dd0 2023-11-15T03:08:15.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.879+0000 7f9cb0a3b700 1 --2- 172.21.15.121:0/1484176702 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9c9c059020 0x7f9c9c05b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.879+0000 7f9cc4b32d00 1 --2- 172.21.15.121:0/1484176702 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55b487d66770 0x55b487cc8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:15.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.879+0000 7f9cc4b32d00 1 -- 172.21.15.121:0/1484176702 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected e120) v8 -- 0x55b487d62f70 con 0x55b487d66770 2023-11-15T03:08:15.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.879+0000 7f9cb0a3b700 1 --2- 172.21.15.121:0/1484176702 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9c9c059020 0x7f9c9c05b4e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x55b487ce1e30 tx=0x7f9ca40017d0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:15.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.879+0000 7f9cb9a3d700 1 --2- 172.21.15.121:0/1484176702 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55b487d66770 0x55b487cc8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:15.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.879+0000 7f9cb9a3d700 1 --2- 172.21.15.121:0/1484176702 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55b487d66770 0x55b487cc8820 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:15.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.883+0000 7f9cb9a3d700 1 -- 172.21.15.121:0/1484176702 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(1 500.00000000 [truncate 0] v120'293 uv293 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9cb400daf0 con 0x55b487d66770 2023-11-15T03:08:15.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.883+0000 7f9cc4b32d00 1 -- 172.21.15.121:0/1484176702 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55b487d66770 msgr2=0x55b487cc8820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.883+0000 7f9cc4b32d00 1 --2- 172.21.15.121:0/1484176702 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55b487d66770 0x55b487cc8820 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:15.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.883+0000 7f9cc4b32d00 1 -- 172.21.15.121:0/1484176702 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9c9c059020 msgr2=0x7f9c9c05b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.883+0000 7f9cc4b32d00 1 --2- 172.21.15.121:0/1484176702 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9c9c059020 0x7f9c9c05b4e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x55b487ce1e30 tx=0x7f9ca40017d0).stop 2023-11-15T03:08:15.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.883+0000 7f9cc4b32d00 1 -- 172.21.15.121:0/1484176702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b487c93dd0 msgr2=0x55b487d62a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:15.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.883+0000 7f9cc4b32d00 1 --2- 172.21.15.121:0/1484176702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b487c93dd0 0x55b487d62a80 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 rx=0x7f9cac00c010 tx=0x7f9cac01c000).stop 2023-11-15T03:08:15.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.883+0000 7f9cb9a3d700 1 -- 172.21.15.121:0/1484176702 reap_dead start 2023-11-15T03:08:15.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.883+0000 7f9cc4b32d00 1 -- 172.21.15.121:0/1484176702 shutdown_connections 2023-11-15T03:08:15.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.883+0000 7f9cc4b32d00 1 -- 172.21.15.121:0/1484176702 >> 172.21.15.121:0/1484176702 conn(0x55b487ce3080 msgr2=0x55b487c935f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:15.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.883+0000 7f9cc4b32d00 1 -- 172.21.15.121:0/1484176702 shutdown_connections 2023-11-15T03:08:15.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:15.883+0000 7f9cc4b32d00 1 -- 172.21.15.121:0/1484176702 wait complete. 2023-11-15T03:08:15.899 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:08:16.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.063+0000 7f92df71b700 1 Processor -- start 2023-11-15T03:08:16.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.063+0000 7f92df71b700 1 -- start start 2023-11-15T03:08:16.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.063+0000 7f92df71b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d8067ec0 0x7f92d80c3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:16.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.067+0000 7f92df71b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f92d8065e00 0x7f92d8154b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:16.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.067+0000 7f92df71b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d8064d60 con 0x7f92d80682a0 2023-11-15T03:08:16.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.067+0000 7f92df71b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92d80c4af0 con 0x7f92d8065e00 2023-11-15T03:08:16.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.067+0000 7f92df71b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d80686e0 con 0x7f92d8067ec0 2023-11-15T03:08:16.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.067+0000 7f92dd4b7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d8067ec0 0x7f92d80c3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:16.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.067+0000 7f92ddcb8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f92d8065e00 0x7f92d8154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:16.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.067+0000 7f92dccb6700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f92d80682a0 0x7f92d8156110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:33678/0 (socket says 172.21.15.121:33678) 2023-11-15T03:08:16.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.067+0000 7f92dccb6700 1 -- 172.21.15.121:0/1307650711 learned_addr learned my addr 172.21.15.121:0/1307650711 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:16.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.067+0000 7f92dd4b7700 1 -- 172.21.15.121:0/1307650711 >> v2:172.21.15.121:3300/0 conn(0x7f92d8065e00 msgr2=0x7f92d8154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:16.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.067+0000 7f92dd4b7700 1 --2- 172.21.15.121:0/1307650711 >> v2:172.21.15.121:3300/0 conn(0x7f92d8065e00 0x7f92d8154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:16.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.067+0000 7f92dd4b7700 1 -- 172.21.15.121:0/1307650711 >> v1:172.21.15.121:6789/0 conn(0x7f92d80682a0 legacy=0x7f92d8156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:16.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.067+0000 7f92dd4b7700 1 -- 172.21.15.121:0/1307650711 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d8065040 con 0x7f92d8067ec0 2023-11-15T03:08:16.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.067+0000 7f92dd4b7700 1 --2- 172.21.15.121:0/1307650711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d8067ec0 0x7f92d80c3e30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f92d4009760 tx=0x7f92d4009060).ready entity=mon.2 client_cookie=279037b93c34725b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:16.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.067+0000 7f92cffff700 1 -- 172.21.15.121:0/1307650711 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d400f510 con 0x7f92d8067ec0 2023-11-15T03:08:16.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.067+0000 7f92cffff700 1 -- 172.21.15.121:0/1307650711 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d400f670 con 0x7f92d8067ec0 2023-11-15T03:08:16.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.067+0000 7f92cffff700 1 -- 172.21.15.121:0/1307650711 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d400f9f0 con 0x7f92d8067ec0 2023-11-15T03:08:16.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.071+0000 7f92df71b700 1 -- 172.21.15.121:0/1307650711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d8067ec0 msgr2=0x7f92d80c3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:16.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.071+0000 7f92df71b700 1 --2- 172.21.15.121:0/1307650711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d8067ec0 0x7f92d80c3e30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f92d4009760 tx=0x7f92d4009060).stop 2023-11-15T03:08:16.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.071+0000 7f92df71b700 1 -- 172.21.15.121:0/1307650711 shutdown_connections 2023-11-15T03:08:16.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.071+0000 7f92df71b700 1 --2- 172.21.15.121:0/1307650711 >> v2:172.21.15.121:3300/0 conn(0x7f92d8065e00 0x7f92d8154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:16.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.071+0000 7f92df71b700 1 --2- 172.21.15.121:0/1307650711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d8067ec0 0x7f92d80c3e30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:16.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.071+0000 7f92df71b700 1 -- 172.21.15.121:0/1307650711 >> 172.21.15.121:0/1307650711 conn(0x7f92d80c8b50 msgr2=0x7f92d80c8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:16.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.071+0000 7f92df71b700 1 -- 172.21.15.121:0/1307650711 shutdown_connections 2023-11-15T03:08:16.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.071+0000 7f92df71b700 1 -- 172.21.15.121:0/1307650711 wait complete. 2023-11-15T03:08:16.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.071+0000 7f92df71b700 1 Processor -- start 2023-11-15T03:08:16.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.071+0000 7f92df71b700 1 -- start start 2023-11-15T03:08:16.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92df71b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d8065e00 0x7f92d8164d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:16.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92df71b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92d8067ec0 0x7f92d80a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:16.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92df71b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92d80682a0 0x7f92d80a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:16.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92df71b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d80a4500 con 0x7f92d8067ec0 2023-11-15T03:08:16.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92df71b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d80a5af0 con 0x7f92d80682a0 2023-11-15T03:08:16.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92df71b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d80a9e30 con 0x7f92d8065e00 2023-11-15T03:08:16.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92dccb6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92d8067ec0 0x7f92d80a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:16.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92dccb6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92d8067ec0 0x7f92d80a3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51920/0 (socket says 172.21.15.121:51920) 2023-11-15T03:08:16.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92dccb6700 1 -- 172.21.15.121:0/580008170 learned_addr learned my addr 172.21.15.121:0/580008170 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:16.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92dd4b7700 1 --2- 172.21.15.121:0/580008170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d8065e00 0x7f92d8164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:16.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92ddcb8700 1 --2- 172.21.15.121:0/580008170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92d80682a0 0x7f92d80a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:16.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92dccb6700 1 -- 172.21.15.121:0/580008170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d8065e00 msgr2=0x7f92d8164d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:16.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92dccb6700 1 --2- 172.21.15.121:0/580008170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d8065e00 0x7f92d8164d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:16.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92dccb6700 1 -- 172.21.15.121:0/580008170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92d80682a0 msgr2=0x7f92d80a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:16.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92dccb6700 1 --2- 172.21.15.121:0/580008170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92d80682a0 0x7f92d80a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:16.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92dccb6700 1 -- 172.21.15.121:0/580008170 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d40092a0 con 0x7f92d8067ec0 2023-11-15T03:08:16.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92dccb6700 1 --2- 172.21.15.121:0/580008170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92d8067ec0 0x7f92d80a3b20 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 rx=0x7f92c800c440 tx=0x7f92c8007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:16.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92cd7fa700 1 -- 172.21.15.121:0/580008170 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c800ce40 con 0x7f92d8067ec0 2023-11-15T03:08:16.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92cd7fa700 1 -- 172.21.15.121:0/580008170 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c80072e0 con 0x7f92d8067ec0 2023-11-15T03:08:16.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92df71b700 1 -- 172.21.15.121:0/580008170 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d80aa110 con 0x7f92d8067ec0 2023-11-15T03:08:16.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92cd7fa700 1 -- 172.21.15.121:0/580008170 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c8007660 con 0x7f92d8067ec0 2023-11-15T03:08:16.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.075+0000 7f92df71b700 1 -- 172.21.15.121:0/580008170 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d80aa6d0 con 0x7f92d8067ec0 2023-11-15T03:08:16.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.079+0000 7f92df71b700 1 -- 172.21.15.121:0/580008170 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d80aeaf0 con 0x7f92d8067ec0 2023-11-15T03:08:16.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.079+0000 7f92cd7fa700 1 -- 172.21.15.121:0/580008170 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f92c80059b0 con 0x7f92d8067ec0 2023-11-15T03:08:16.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.079+0000 7f92cd7fa700 1 --2- 172.21.15.121:0/580008170 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f92bc05a540 0x7f92bc05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:16.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.083+0000 7f92cd7fa700 1 -- 172.21.15.121:0/580008170 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8502+0+0 (secure 0 0 0) 0x7f92c8074860 con 0x7f92d8067ec0 2023-11-15T03:08:16.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.083+0000 7f92dd4b7700 1 --2- 172.21.15.121:0/580008170 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f92bc05a540 0x7f92bc05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:16.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.083+0000 7f92dd4b7700 1 --2- 172.21.15.121:0/580008170 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f92bc05a540 0x7f92bc05ca00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f92d80aacd0 tx=0x7f92d4017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:16.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.087+0000 7f92cd7fa700 1 -- 172.21.15.121:0/580008170 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f92d80aeaf0 con 0x7f92d8067ec0 2023-11-15T03:08:16.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:16.291+0000 7f92df71b700 1 -- 172.21.15.121:0/580008170 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f92d80aeaf0 con 0x7f92d8067ec0 2023-11-15T03:08:17.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.271+0000 7f92cd7fa700 1 -- 172.21.15.121:0/580008170 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v244) v1 ==== 162+0+0 (secure 0 0 0) 0x7f92d80aeaf0 con 0x7f92d8067ec0 2023-11-15T03:08:17.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.275+0000 7f92df71b700 1 -- 172.21.15.121:0/580008170 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f92bc05a540 msgr2=0x7f92bc05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:17.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.275+0000 7f92df71b700 1 --2- 172.21.15.121:0/580008170 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f92bc05a540 0x7f92bc05ca00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f92d80aacd0 tx=0x7f92d4017000).stop 2023-11-15T03:08:17.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.275+0000 7f92df71b700 1 -- 172.21.15.121:0/580008170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92d8067ec0 msgr2=0x7f92d80a3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:17.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.275+0000 7f92df71b700 1 --2- 172.21.15.121:0/580008170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92d8067ec0 0x7f92d80a3b20 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 rx=0x7f92c800c440 tx=0x7f92c8007e20).stop 2023-11-15T03:08:17.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.275+0000 7f92df71b700 1 -- 172.21.15.121:0/580008170 shutdown_connections 2023-11-15T03:08:17.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.275+0000 7f92df71b700 1 --2- 172.21.15.121:0/580008170 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f92bc05a540 0x7f92bc05ca00 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:17.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.275+0000 7f92df71b700 1 --2- 172.21.15.121:0/580008170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92d80682a0 0x7f92d80a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:17.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.275+0000 7f92df71b700 1 --2- 172.21.15.121:0/580008170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92d8067ec0 0x7f92d80a3b20 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:17.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.275+0000 7f92df71b700 1 --2- 172.21.15.121:0/580008170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d8065e00 0x7f92d8164d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:17.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.275+0000 7f92df71b700 1 -- 172.21.15.121:0/580008170 >> 172.21.15.121:0/580008170 conn(0x7f92d80c8b50 msgr2=0x7f92d8153e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:17.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.275+0000 7f92df71b700 1 -- 172.21.15.121:0/580008170 shutdown_connections 2023-11-15T03:08:17.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.275+0000 7f92df71b700 1 -- 172.21.15.121:0/580008170 wait complete. 2023-11-15T03:08:17.288 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:08:17.711 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:17.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.859+0000 7fd98228f700 1 Processor -- start 2023-11-15T03:08:17.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.859+0000 7fd98228f700 1 -- start start 2023-11-15T03:08:17.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.863+0000 7fd98228f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd97c0a9830 0x7fd97c0a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:17.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.863+0000 7fd98228f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd97c0c30e0 0x7fd97c0c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:17.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.863+0000 7fd98228f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97c0657f0 con 0x7fd97c151870 2023-11-15T03:08:17.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.863+0000 7fd98228f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97c065ad0 con 0x7fd97c0c30e0 2023-11-15T03:08:17.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.863+0000 7fd98228f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0c3a60 con 0x7fd97c0a9830 2023-11-15T03:08:17.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.863+0000 7fd981a8e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd97c0c30e0 0x7fd97c0c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:17.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.863+0000 7fd98128d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fd97c151870 0x7fd97c151c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:33684/0 (socket says 172.21.15.121:33684) 2023-11-15T03:08:17.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.863+0000 7fd98128d700 1 -- 172.21.15.121:0/4224882693 learned_addr learned my addr 172.21.15.121:0/4224882693 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:17.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.863+0000 7fd980a8c700 1 --2- 172.21.15.121:0/4224882693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd97c0a9830 0x7fd97c0a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:17.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.863+0000 7fd973fff700 1 -- 172.21.15.121:0/4224882693 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000752390 0 0) 0x7fd97c0657f0 con 0x7fd97c151870 2023-11-15T03:08:17.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.863+0000 7fd981a8e700 1 -- 172.21.15.121:0/4224882693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd97c0a9830 msgr2=0x7fd97c0a9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:17.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.863+0000 7fd981a8e700 1 --2- 172.21.15.121:0/4224882693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd97c0a9830 0x7fd97c0a9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:17.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.863+0000 7fd981a8e700 1 -- 172.21.15.121:0/4224882693 >> v1:172.21.15.121:6789/0 conn(0x7fd97c151870 legacy=0x7fd97c151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:17.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.863+0000 7fd981a8e700 1 -- 172.21.15.121:0/4224882693 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97c0aa260 con 0x7fd97c0c30e0 2023-11-15T03:08:17.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.863+0000 7fd981a8e700 1 --2- 172.21.15.121:0/4224882693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd97c0c30e0 0x7fd97c0c3570 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 rx=0x7fd974009e60 tx=0x7fd974009780).ready entity=mon.0 client_cookie=1acc4ede4b6b885 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:17.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.867+0000 7fd973fff700 1 -- 172.21.15.121:0/4224882693 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97400e070 con 0x7fd97c0c30e0 2023-11-15T03:08:17.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.867+0000 7fd973fff700 1 -- 172.21.15.121:0/4224882693 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97400bb60 con 0x7fd97c0c30e0 2023-11-15T03:08:17.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.867+0000 7fd973fff700 1 -- 172.21.15.121:0/4224882693 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd974012410 con 0x7fd97c0c30e0 2023-11-15T03:08:17.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.867+0000 7fd98228f700 1 -- 172.21.15.121:0/4224882693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd97c0c30e0 msgr2=0x7fd97c0c3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:17.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.867+0000 7fd98228f700 1 --2- 172.21.15.121:0/4224882693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd97c0c30e0 0x7fd97c0c3570 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 rx=0x7fd974009e60 tx=0x7fd974009780).stop 2023-11-15T03:08:17.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.867+0000 7fd98228f700 1 -- 172.21.15.121:0/4224882693 shutdown_connections 2023-11-15T03:08:17.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.867+0000 7fd98228f700 1 --2- 172.21.15.121:0/4224882693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd97c0c30e0 0x7fd97c0c3570 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:17.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.867+0000 7fd98228f700 1 --2- 172.21.15.121:0/4224882693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd97c0a9830 0x7fd97c0a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:17.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.867+0000 7fd98228f700 1 -- 172.21.15.121:0/4224882693 >> 172.21.15.121:0/4224882693 conn(0x7fd97c062190 msgr2=0x7fd97c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:17.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.867+0000 7fd98228f700 1 -- 172.21.15.121:0/4224882693 shutdown_connections 2023-11-15T03:08:17.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.867+0000 7fd98228f700 1 -- 172.21.15.121:0/4224882693 wait complete. 2023-11-15T03:08:17.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.867+0000 7fd98228f700 1 Processor -- start 2023-11-15T03:08:17.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd98228f700 1 -- start start 2023-11-15T03:08:17.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd98228f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd97c0a9830 0x7fd97c156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:17.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd98228f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd97c0c30e0 0x7fd97c0bd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:17.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd98228f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd97c151870 0x7fd97c1399d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:17.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd98228f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c157110 con 0x7fd97c0a9830 2023-11-15T03:08:17.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd98228f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0c00c0 con 0x7fd97c0c30e0 2023-11-15T03:08:17.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd98228f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0bde10 con 0x7fd97c151870 2023-11-15T03:08:17.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd98128d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd97c0a9830 0x7fd97c156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:17.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd98128d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd97c0a9830 0x7fd97c156c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51938/0 (socket says 172.21.15.121:51938) 2023-11-15T03:08:17.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd98128d700 1 -- 172.21.15.121:0/2986719182 learned_addr learned my addr 172.21.15.121:0/2986719182 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:17.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd981a8e700 1 --2- 172.21.15.121:0/2986719182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd97c151870 0x7fd97c1399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:17.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd980a8c700 1 --2- 172.21.15.121:0/2986719182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd97c0c30e0 0x7fd97c0bd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:17.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd98128d700 1 -- 172.21.15.121:0/2986719182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd97c151870 msgr2=0x7fd97c1399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:17.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd98128d700 1 --2- 172.21.15.121:0/2986719182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd97c151870 0x7fd97c1399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:17.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd98128d700 1 -- 172.21.15.121:0/2986719182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd97c0c30e0 msgr2=0x7fd97c0bd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:17.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd98128d700 1 --2- 172.21.15.121:0/2986719182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd97c0c30e0 0x7fd97c0bd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:17.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd98128d700 1 -- 172.21.15.121:0/2986719182 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd974009a20 con 0x7fd97c0a9830 2023-11-15T03:08:17.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd981a8e700 1 --2- 172.21.15.121:0/2986719182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd97c151870 0x7fd97c1399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:17.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd98128d700 1 --2- 172.21.15.121:0/2986719182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd97c0a9830 0x7fd97c156c20 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 rx=0x7fd978001eb0 tx=0x7fd978007e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:17.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd9717fa700 1 -- 172.21.15.121:0/2986719182 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9780072e0 con 0x7fd97c0a9830 2023-11-15T03:08:17.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd9717fa700 1 -- 172.21.15.121:0/2986719182 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd978007440 con 0x7fd97c0a9830 2023-11-15T03:08:17.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd98228f700 1 -- 172.21.15.121:0/2986719182 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97c0bfab0 con 0x7fd97c0a9830 2023-11-15T03:08:17.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.871+0000 7fd9717fa700 1 -- 172.21.15.121:0/2986719182 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9780057a0 con 0x7fd97c0a9830 2023-11-15T03:08:17.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.875+0000 7fd98228f700 1 -- 172.21.15.121:0/2986719182 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97c0be090 con 0x7fd97c0a9830 2023-11-15T03:08:17.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.875+0000 7fd98228f700 1 -- 172.21.15.121:0/2986719182 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97c06bcf0 con 0x7fd97c0a9830 2023-11-15T03:08:17.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.875+0000 7fd9717fa700 1 -- 172.21.15.121:0/2986719182 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd978005bd0 con 0x7fd97c0a9830 2023-11-15T03:08:17.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.879+0000 7fd9717fa700 1 --2- 172.21.15.121:0/2986719182 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd96005a540 0x7fd96005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:17.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.879+0000 7fd9717fa700 1 -- 172.21.15.121:0/2986719182 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8502+0+0 (secure 0 0 0) 0x7fd9780748e0 con 0x7fd97c0a9830 2023-11-15T03:08:17.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.879+0000 7fd980a8c700 1 --2- 172.21.15.121:0/2986719182 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd96005a540 0x7fd96005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:17.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.879+0000 7fd980a8c700 1 --2- 172.21.15.121:0/2986719182 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd96005a540 0x7fd96005ca00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7fd97c14e170 tx=0x7fd96c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:17.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:17.883+0000 7fd9717fa700 1 -- 172.21.15.121:0/2986719182 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd9780476b0 con 0x7fd97c0a9830 2023-11-15T03:08:18.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:18.079+0000 7fd98228f700 1 -- 172.21.15.121:0/2986719182 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd97c068a70 con 0x7fd97c0a9830 2023-11-15T03:08:18.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:18.083+0000 7fd9717fa700 1 -- 172.21.15.121:0/2986719182 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 245 v245) v1 ==== 95+0+4505 (secure 0 0 0) 0x7fd97804acd0 con 0x7fd97c0a9830 2023-11-15T03:08:18.090 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:18.091 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":245,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":245,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:08:17.276959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9417},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9417":{"gid":9417,"name":"a","rank":0,"incarnation":245,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/1958064439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1958064439},{"type":"v1","addr":"172.21.15.121:6836","nonce":1958064439}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:18.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:18.083+0000 7fd98228f700 1 -- 172.21.15.121:0/2986719182 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd96005a540 msgr2=0x7fd96005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:18.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:18.083+0000 7fd98228f700 1 --2- 172.21.15.121:0/2986719182 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd96005a540 0x7fd96005ca00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7fd97c14e170 tx=0x7fd96c008040).stop 2023-11-15T03:08:18.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:18.083+0000 7fd98228f700 1 -- 172.21.15.121:0/2986719182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd97c0a9830 msgr2=0x7fd97c156c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:18.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:18.083+0000 7fd98228f700 1 --2- 172.21.15.121:0/2986719182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd97c0a9830 0x7fd97c156c20 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 rx=0x7fd978001eb0 tx=0x7fd978007e90).stop 2023-11-15T03:08:18.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:18.083+0000 7fd98228f700 1 -- 172.21.15.121:0/2986719182 shutdown_connections 2023-11-15T03:08:18.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:18.083+0000 7fd98228f700 1 --2- 172.21.15.121:0/2986719182 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd96005a540 0x7fd96005ca00 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:18.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:18.083+0000 7fd98228f700 1 --2- 172.21.15.121:0/2986719182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd97c151870 0x7fd97c1399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:18.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:18.083+0000 7fd98228f700 1 --2- 172.21.15.121:0/2986719182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd97c0c30e0 0x7fd97c0bd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:18.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:18.083+0000 7fd98228f700 1 --2- 172.21.15.121:0/2986719182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd97c0a9830 0x7fd97c156c20 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:18.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:18.083+0000 7fd98228f700 1 -- 172.21.15.121:0/2986719182 >> 172.21.15.121:0/2986719182 conn(0x7fd97c062190 msgr2=0x7fd97c0a89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:18.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:18.083+0000 7fd98228f700 1 -- 172.21.15.121:0/2986719182 shutdown_connections 2023-11-15T03:08:18.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:18.083+0000 7fd98228f700 1 -- 172.21.15.121:0/2986719182 wait complete. 2023-11-15T03:08:18.096 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 245 2023-11-15T03:08:18.112 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 245, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:08:17.276959+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 120, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9417}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9417': {'gid': 9417, 'name': 'a', 'rank': 0, 'incarnation': 245, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.121:6836/1958064439', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 1958064439}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 1958064439}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:08:18.113 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9417:up:replay 2023-11-15T03:08:18.113 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:08:23.328 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:08:23.319+0000 7fc16ad99700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:08:23.517 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:23.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.683+0000 7fb47bfdb700 1 Processor -- start 2023-11-15T03:08:23.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.683+0000 7fb47bfdb700 1 -- start start 2023-11-15T03:08:23.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.687+0000 7fb47bfdb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb4740c6ce0 0x7fb474159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:23.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.687+0000 7fb47bfdb700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb4740a5510 0x7fb4740a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:23.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.687+0000 7fb47bfdb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4740c3fa0 con 0x7fb474152780 2023-11-15T03:08:23.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.687+0000 7fb47bfdb700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb474159630 con 0x7fb4740a5510 2023-11-15T03:08:23.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.687+0000 7fb47bfdb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4740c30b0 con 0x7fb4740c6ce0 2023-11-15T03:08:23.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.687+0000 7fb479d77700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb474152780 0x7fb4740c65d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:33688/0 (socket says 172.21.15.121:33688) 2023-11-15T03:08:23.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.687+0000 7fb479d77700 1 -- 172.21.15.121:0/1040854898 learned_addr learned my addr 172.21.15.121:0/1040854898 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:23.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.687+0000 7fb47a578700 1 --2- 172.21.15.121:0/1040854898 >> v2:172.21.15.121:3300/0 conn(0x7fb4740a5510 0x7fb4740a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:23.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.687+0000 7fb479576700 1 --2- 172.21.15.121:0/1040854898 >> v2:172.21.15.190:3300/0 conn(0x7fb4740c6ce0 0x7fb474159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:23.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.687+0000 7fb478d75700 1 -- 172.21.15.121:0/1040854898 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623515421 0 0) 0x7fb4740c3fa0 con 0x7fb474152780 2023-11-15T03:08:23.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.687+0000 7fb478d75700 1 -- 172.21.15.121:0/1040854898 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4600033f0 con 0x7fb474152780 2023-11-15T03:08:23.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.687+0000 7fb47a578700 1 -- 172.21.15.121:0/1040854898 >> v2:172.21.15.190:3300/0 conn(0x7fb4740c6ce0 msgr2=0x7fb474159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:23.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.687+0000 7fb47a578700 1 --2- 172.21.15.121:0/1040854898 >> v2:172.21.15.190:3300/0 conn(0x7fb4740c6ce0 0x7fb474159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:23.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.687+0000 7fb47a578700 1 -- 172.21.15.121:0/1040854898 >> v1:172.21.15.121:6789/0 conn(0x7fb474152780 legacy=0x7fb4740c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:23.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.687+0000 7fb47a578700 1 -- 172.21.15.121:0/1040854898 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4740c3390 con 0x7fb4740a5510 2023-11-15T03:08:23.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.687+0000 7fb47a578700 1 --2- 172.21.15.121:0/1040854898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4740a5510 0x7fb4740a59a0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 rx=0x7fb474063d40 tx=0x7fb46c009780).ready entity=mon.0 client_cookie=4f98c5472f1db3c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:23.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.687+0000 7fb478d75700 1 -- 172.21.15.121:0/1040854898 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c00e070 con 0x7fb4740a5510 2023-11-15T03:08:23.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.687+0000 7fb478d75700 1 -- 172.21.15.121:0/1040854898 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46c00bab0 con 0x7fb4740a5510 2023-11-15T03:08:23.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.687+0000 7fb478d75700 1 -- 172.21.15.121:0/1040854898 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c00be30 con 0x7fb4740a5510 2023-11-15T03:08:23.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.691+0000 7fb47bfdb700 1 -- 172.21.15.121:0/1040854898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4740a5510 msgr2=0x7fb4740a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:23.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.691+0000 7fb47bfdb700 1 --2- 172.21.15.121:0/1040854898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4740a5510 0x7fb4740a59a0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 rx=0x7fb474063d40 tx=0x7fb46c009780).stop 2023-11-15T03:08:23.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.691+0000 7fb47bfdb700 1 -- 172.21.15.121:0/1040854898 shutdown_connections 2023-11-15T03:08:23.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.691+0000 7fb47bfdb700 1 --2- 172.21.15.121:0/1040854898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4740a5510 0x7fb4740a59a0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:23.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.691+0000 7fb47bfdb700 1 --2- 172.21.15.121:0/1040854898 >> v2:172.21.15.190:3300/0 conn(0x7fb4740c6ce0 0x7fb474159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:23.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.691+0000 7fb47bfdb700 1 -- 172.21.15.121:0/1040854898 >> 172.21.15.121:0/1040854898 conn(0x7fb474062190 msgr2=0x7fb4740625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:23.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.691+0000 7fb47bfdb700 1 -- 172.21.15.121:0/1040854898 shutdown_connections 2023-11-15T03:08:23.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.691+0000 7fb47bfdb700 1 -- 172.21.15.121:0/1040854898 wait complete. 2023-11-15T03:08:23.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.695+0000 7fb47bfdb700 1 Processor -- start 2023-11-15T03:08:23.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.695+0000 7fb47bfdb700 1 -- start start 2023-11-15T03:08:23.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.695+0000 7fb47bfdb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4740a5510 0x7fb47415aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:23.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.695+0000 7fb47bfdb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4740c6ce0 0x7fb4740ac260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:23.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.695+0000 7fb47bfdb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb474152780 0x7fb47415c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:23.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.695+0000 7fb47bfdb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4740a8080 con 0x7fb4740c6ce0 2023-11-15T03:08:23.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.695+0000 7fb47bfdb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4740a74b0 con 0x7fb474152780 2023-11-15T03:08:23.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.695+0000 7fb47bfdb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4740a7750 con 0x7fb4740a5510 2023-11-15T03:08:23.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.695+0000 7fb479576700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4740c6ce0 0x7fb4740ac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:23.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.695+0000 7fb479576700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4740c6ce0 0x7fb4740ac260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51952/0 (socket says 172.21.15.121:51952) 2023-11-15T03:08:23.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.695+0000 7fb479576700 1 -- 172.21.15.121:0/2255626216 learned_addr learned my addr 172.21.15.121:0/2255626216 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:23.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.695+0000 7fb479d77700 1 --2- 172.21.15.121:0/2255626216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4740a5510 0x7fb47415aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:23.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.695+0000 7fb47a578700 1 --2- 172.21.15.121:0/2255626216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb474152780 0x7fb47415c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:23.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.695+0000 7fb479576700 1 -- 172.21.15.121:0/2255626216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4740a5510 msgr2=0x7fb47415aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:23.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.695+0000 7fb479576700 1 --2- 172.21.15.121:0/2255626216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4740a5510 0x7fb47415aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:23.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.695+0000 7fb479576700 1 -- 172.21.15.121:0/2255626216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb474152780 msgr2=0x7fb47415c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:23.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.695+0000 7fb479576700 1 --2- 172.21.15.121:0/2255626216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb474152780 0x7fb47415c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:23.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.695+0000 7fb479576700 1 -- 172.21.15.121:0/2255626216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c009a20 con 0x7fb4740c6ce0 2023-11-15T03:08:23.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.695+0000 7fb479576700 1 --2- 172.21.15.121:0/2255626216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4740c6ce0 0x7fb4740ac260 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 rx=0x7fb46400e930 tx=0x7fb464009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:23.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.699+0000 7fb469ffb700 1 -- 172.21.15.121:0/2255626216 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46400eed0 con 0x7fb4740c6ce0 2023-11-15T03:08:23.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.699+0000 7fb469ffb700 1 -- 172.21.15.121:0/2255626216 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46400c850 con 0x7fb4740c6ce0 2023-11-15T03:08:23.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.699+0000 7fb47bfdb700 1 -- 172.21.15.121:0/2255626216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4740a7a30 con 0x7fb4740c6ce0 2023-11-15T03:08:23.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.699+0000 7fb469ffb700 1 -- 172.21.15.121:0/2255626216 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46400cbd0 con 0x7fb4740c6ce0 2023-11-15T03:08:23.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.699+0000 7fb47bfdb700 1 -- 172.21.15.121:0/2255626216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4740ac750 con 0x7fb4740c6ce0 2023-11-15T03:08:23.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.699+0000 7fb47bfdb700 1 -- 172.21.15.121:0/2255626216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb47406c150 con 0x7fb4740c6ce0 2023-11-15T03:08:23.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.699+0000 7fb469ffb700 1 -- 172.21.15.121:0/2255626216 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb46401b400 con 0x7fb4740c6ce0 2023-11-15T03:08:23.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.703+0000 7fb469ffb700 1 --2- 172.21.15.121:0/2255626216 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb45805a510 0x7fb45805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:23.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.703+0000 7fb469ffb700 1 -- 172.21.15.121:0/2255626216 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8502+0+0 (secure 0 0 0) 0x7fb464075670 con 0x7fb4740c6ce0 2023-11-15T03:08:23.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.703+0000 7fb479d77700 1 --2- 172.21.15.121:0/2255626216 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb45805a510 0x7fb45805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:23.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.703+0000 7fb479d77700 1 --2- 172.21.15.121:0/2255626216 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb45805a510 0x7fb45805c9d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fb470005750 tx=0x7fb470009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:23.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.707+0000 7fb469ffb700 1 -- 172.21.15.121:0/2255626216 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb464048550 con 0x7fb4740c6ce0 2023-11-15T03:08:23.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.927+0000 7fb47bfdb700 1 -- 172.21.15.121:0/2255626216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4740c11b0 con 0x7fb4740c6ce0 2023-11-15T03:08:23.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.927+0000 7fb469ffb700 1 -- 172.21.15.121:0/2255626216 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 248 v248) v1 ==== 95+0+4505 (secure 0 0 0) 0x7fb46404bb30 con 0x7fb4740c6ce0 2023-11-15T03:08:23.936 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:23.936 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":248,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":248,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:08:20.298894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9417},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9417":{"gid":9417,"name":"a","rank":0,"incarnation":245,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6836/1958064439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1958064439},{"type":"v1","addr":"172.21.15.121:6836","nonce":1958064439}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:23.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.931+0000 7fb47bfdb700 1 -- 172.21.15.121:0/2255626216 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb45805a510 msgr2=0x7fb45805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:23.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.931+0000 7fb47bfdb700 1 --2- 172.21.15.121:0/2255626216 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb45805a510 0x7fb45805c9d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fb470005750 tx=0x7fb470009040).stop 2023-11-15T03:08:23.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.931+0000 7fb47bfdb700 1 -- 172.21.15.121:0/2255626216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4740c6ce0 msgr2=0x7fb4740ac260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:23.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.931+0000 7fb47bfdb700 1 --2- 172.21.15.121:0/2255626216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4740c6ce0 0x7fb4740ac260 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 rx=0x7fb46400e930 tx=0x7fb464009060).stop 2023-11-15T03:08:23.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.931+0000 7fb47bfdb700 1 -- 172.21.15.121:0/2255626216 shutdown_connections 2023-11-15T03:08:23.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.931+0000 7fb47bfdb700 1 --2- 172.21.15.121:0/2255626216 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb45805a510 0x7fb45805c9d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:23.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.931+0000 7fb47bfdb700 1 --2- 172.21.15.121:0/2255626216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb474152780 0x7fb47415c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:23.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.931+0000 7fb47bfdb700 1 --2- 172.21.15.121:0/2255626216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4740c6ce0 0x7fb4740ac260 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:23.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.931+0000 7fb47bfdb700 1 --2- 172.21.15.121:0/2255626216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4740a5510 0x7fb47415aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:23.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.931+0000 7fb47bfdb700 1 -- 172.21.15.121:0/2255626216 >> 172.21.15.121:0/2255626216 conn(0x7fb474062190 msgr2=0x7fb47415e700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:23.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.931+0000 7fb47bfdb700 1 -- 172.21.15.121:0/2255626216 shutdown_connections 2023-11-15T03:08:23.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:23.931+0000 7fb47bfdb700 1 -- 172.21.15.121:0/2255626216 wait complete. 2023-11-15T03:08:23.945 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 248 2023-11-15T03:08:23.959 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 248, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:08:20.298894+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 120, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9417}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9417': {'gid': 9417, 'name': 'a', 'rank': 0, 'incarnation': 245, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.121:6836/1958064439', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 1958064439}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 1958064439}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:08:23.960 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T03:08:23.961 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-15T03:08:24.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.111+0000 7fa5eb59e700 1 Processor -- start 2023-11-15T03:08:24.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.111+0000 7fa5eb59e700 1 -- start start 2023-11-15T03:08:24.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5eb59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa5ec0ad1b0 0x7fa5ec065710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5eb59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5ec065c00 0x7fa5ec0add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5eb59e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ec066c90 con 0x7fa5ec0ab0d0 2023-11-15T03:08:24.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5eb59e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ec066080 con 0x7fa5ec0ad1b0 2023-11-15T03:08:24.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5eb59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec066360 con 0x7fa5ec065c00 2023-11-15T03:08:24.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5ea59c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa5ec0ab0d0 0x7fa5ec0ab4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:33702/0 (socket says 172.21.15.121:33702) 2023-11-15T03:08:24.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5ea59c700 1 -- 172.21.15.121:0/4047513773 learned_addr learned my addr 172.21.15.121:0/4047513773 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:24.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5ead9d700 1 --2- 172.21.15.121:0/4047513773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5ec065c00 0x7fa5ec0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e9d9b700 1 --2- 172.21.15.121:0/4047513773 >> v2:172.21.15.190:3300/0 conn(0x7fa5ec0ad1b0 0x7fa5ec065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e959a700 1 -- 172.21.15.121:0/4047513773 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 815343996 0 0) 0x7fa5ec066c90 con 0x7fa5ec0ab0d0 2023-11-15T03:08:24.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e959a700 1 -- 172.21.15.121:0/4047513773 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5d40033f0 con 0x7fa5ec0ab0d0 2023-11-15T03:08:24.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e959a700 1 -- 172.21.15.121:0/4047513773 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3203181249 0 0) 0x7fa5d40033f0 con 0x7fa5ec0ab0d0 2023-11-15T03:08:24.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e959a700 1 -- 172.21.15.121:0/4047513773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5ec065c00 msgr2=0x7fa5ec0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:24.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e959a700 1 --2- 172.21.15.121:0/4047513773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5ec065c00 0x7fa5ec0add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e959a700 1 -- 172.21.15.121:0/4047513773 >> v2:172.21.15.190:3300/0 conn(0x7fa5ec0ad1b0 msgr2=0x7fa5ec065710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:24.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e959a700 1 --2- 172.21.15.121:0/4047513773 >> v2:172.21.15.190:3300/0 conn(0x7fa5ec0ad1b0 0x7fa5ec065710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e959a700 1 -- 172.21.15.121:0/4047513773 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ec066640 con 0x7fa5ec0ab0d0 2023-11-15T03:08:24.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e959a700 1 -- 172.21.15.121:0/4047513773 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fa5dc0015f0 con 0x7fa5ec0ab0d0 2023-11-15T03:08:24.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e959a700 1 -- 172.21.15.121:0/4047513773 >> v1:172.21.15.121:6789/0 conn(0x7fa5ec0ab0d0 legacy=0x7fa5ec0ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:24.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e959a700 1 --2- 172.21.15.121:0/4047513773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5ec065c00 0x7fa5d40038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e959a700 1 --2- 172.21.15.121:0/4047513773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5d4003e10 0x7fa5d4004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e959a700 1 --2- 172.21.15.121:0/4047513773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5d4004b80 0x7fa5d4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e959a700 1 -- 172.21.15.121:0/4047513773 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec066080 con 0x7fa5d4004b80 2023-11-15T03:08:24.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e959a700 1 -- 172.21.15.121:0/4047513773 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec066360 con 0x7fa5d4003e10 2023-11-15T03:08:24.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e959a700 1 -- 172.21.15.121:0/4047513773 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d400b8e0 con 0x7fa5ec065c00 2023-11-15T03:08:24.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5ead9d700 1 --2- 172.21.15.121:0/4047513773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5d4004b80 0x7fa5d4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5ea59c700 1 --2- 172.21.15.121:0/4047513773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5ec065c00 0x7fa5d40038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e9d9b700 1 --2- 172.21.15.121:0/4047513773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5d4003e10 0x7fa5d4004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5ead9d700 1 -- 172.21.15.121:0/4047513773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5ec065c00 msgr2=0x7fa5d40038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:24.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5ead9d700 1 --2- 172.21.15.121:0/4047513773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5ec065c00 0x7fa5d40038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5ead9d700 1 -- 172.21.15.121:0/4047513773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5d4003e10 msgr2=0x7fa5d4004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:24.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5ead9d700 1 --2- 172.21.15.121:0/4047513773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5d4003e10 0x7fa5d4004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5ead9d700 1 -- 172.21.15.121:0/4047513773 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d40033f0 con 0x7fa5d4004b80 2023-11-15T03:08:24.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e9d9b700 1 --2- 172.21.15.121:0/4047513773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5d4003e10 0x7fa5d4004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:24.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5ead9d700 1 --2- 172.21.15.121:0/4047513773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5d4004b80 0x7fa5d4005000 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 rx=0x7fa5ec068cc0 tx=0x7fa5e400b040).ready entity=mon.0 client_cookie=f29107c754eb211e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:24.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.115+0000 7fa5e959a700 1 -- 172.21.15.121:0/4047513773 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4009ac0 con 0x7fa5d4004b80 2023-11-15T03:08:24.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.119+0000 7fa5e959a700 1 -- 172.21.15.121:0/4047513773 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e400f040 con 0x7fa5d4004b80 2023-11-15T03:08:24.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.119+0000 7fa5e959a700 1 -- 172.21.15.121:0/4047513773 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e400ec10 con 0x7fa5d4004b80 2023-11-15T03:08:24.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.119+0000 7fa5eb59e700 1 -- 172.21.15.121:0/4047513773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5d4004b80 msgr2=0x7fa5d4005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:24.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.119+0000 7fa5eb59e700 1 --2- 172.21.15.121:0/4047513773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5d4004b80 0x7fa5d4005000 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 rx=0x7fa5ec068cc0 tx=0x7fa5e400b040).stop 2023-11-15T03:08:24.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.119+0000 7fa5ead9d700 1 -- 172.21.15.121:0/4047513773 reap_dead start 2023-11-15T03:08:24.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.119+0000 7fa5eb59e700 1 -- 172.21.15.121:0/4047513773 shutdown_connections 2023-11-15T03:08:24.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.119+0000 7fa5eb59e700 1 -- 172.21.15.121:0/4047513773 >> 172.21.15.121:0/4047513773 conn(0x7fa5ec067a60 msgr2=0x7fa5ec067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:24.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.119+0000 7fa5eb59e700 1 -- 172.21.15.121:0/4047513773 shutdown_connections 2023-11-15T03:08:24.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.119+0000 7fa5eb59e700 1 -- 172.21.15.121:0/4047513773 wait complete. 2023-11-15T03:08:24.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5eb59e700 1 Processor -- start 2023-11-15T03:08:24.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5eb59e700 1 -- start start 2023-11-15T03:08:24.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5eb59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5d4004b80 0x7fa5ec156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5eb59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5ec150580 0x7fa5ec0c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5eb59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5ec150a00 0x7fa5ec0ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5eb59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec0ca530 con 0x7fa5ec150a00 2023-11-15T03:08:24.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5eb59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec1684c0 con 0x7fa5ec150580 2023-11-15T03:08:24.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5eb59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec156f80 con 0x7fa5d4004b80 2023-11-15T03:08:24.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5ead9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5ec150a00 0x7fa5ec0ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5ead9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5ec150a00 0x7fa5ec0ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51964/0 (socket says 172.21.15.121:51964) 2023-11-15T03:08:24.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5ead9d700 1 -- 172.21.15.121:0/3155159678 learned_addr learned my addr 172.21.15.121:0/3155159678 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:24.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5ea59c700 1 --2- 172.21.15.121:0/3155159678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5d4004b80 0x7fa5ec156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5e9d9b700 1 --2- 172.21.15.121:0/3155159678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5ec150580 0x7fa5ec0c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5ead9d700 1 -- 172.21.15.121:0/3155159678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5d4004b80 msgr2=0x7fa5ec156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:24.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5ead9d700 1 --2- 172.21.15.121:0/3155159678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5d4004b80 0x7fa5ec156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5ead9d700 1 -- 172.21.15.121:0/3155159678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5ec150580 msgr2=0x7fa5ec0c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:24.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5ead9d700 1 --2- 172.21.15.121:0/3155159678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5ec150580 0x7fa5ec0c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5ead9d700 1 -- 172.21.15.121:0/3155159678 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ec1678b0 con 0x7fa5ec150a00 2023-11-15T03:08:24.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5ea59c700 1 --2- 172.21.15.121:0/3155159678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5d4004b80 0x7fa5ec156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:24.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.123+0000 7fa5e9d9b700 1 --2- 172.21.15.121:0/3155159678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5ec150580 0x7fa5ec0c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:24.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.127+0000 7fa5ead9d700 1 --2- 172.21.15.121:0/3155159678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5ec150a00 0x7fa5ec0ca040 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 rx=0x7fa5d40045a0 tx=0x7fa5e401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:24.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.127+0000 7fa5da7fc700 1 -- 172.21.15.121:0/3155159678 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4021030 con 0x7fa5ec150a00 2023-11-15T03:08:24.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.127+0000 7fa5da7fc700 1 -- 172.21.15.121:0/3155159678 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e4009ac0 con 0x7fa5ec150a00 2023-11-15T03:08:24.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.127+0000 7fa5eb59e700 1 -- 172.21.15.121:0/3155159678 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ec167b90 con 0x7fa5ec150a00 2023-11-15T03:08:24.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.127+0000 7fa5da7fc700 1 -- 172.21.15.121:0/3155159678 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4013e70 con 0x7fa5ec150a00 2023-11-15T03:08:24.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.127+0000 7fa5eb59e700 1 -- 172.21.15.121:0/3155159678 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ec168150 con 0x7fa5ec150a00 2023-11-15T03:08:24.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.127+0000 7fa5da7fc700 1 -- 172.21.15.121:0/3155159678 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa5e4024070 con 0x7fa5ec150a00 2023-11-15T03:08:24.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.131+0000 7fa5da7fc700 1 --2- 172.21.15.121:0/3155159678 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa5cc05a540 0x7fa5cc05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.131+0000 7fa5da7fc700 1 -- 172.21.15.121:0/3155159678 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8502+0+0 (secure 0 0 0) 0x7fa5e4022070 con 0x7fa5ec150a00 2023-11-15T03:08:24.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.131+0000 7fa5ea59c700 1 --2- 172.21.15.121:0/3155159678 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa5cc05a540 0x7fa5cc05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.131+0000 7fa5ea59c700 1 --2- 172.21.15.121:0/3155159678 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa5cc05a540 0x7fa5cc05ca00 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7fa5dc000ef0 tx=0x7fa5dc009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:24.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.131+0000 7fa5fade1740 1 Processor -- start 2023-11-15T03:08:24.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.131+0000 7fa5fade1740 1 -- start start 2023-11-15T03:08:24.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5fade1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13ad0 0x1e3a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5fade1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e13eb0 0x1e60140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5fade1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e60630 0x1e60ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5fade1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d94410 con 0x1e13ad0 2023-11-15T03:08:24.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5fade1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d94550 con 0x1e60630 2023-11-15T03:08:24.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5fade1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d94a00 con 0x1e13eb0 2023-11-15T03:08:24.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5ea59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13ad0 0x1e3a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5ea59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13ad0 0x1e3a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51972/0 (socket says 172.21.15.121:51972) 2023-11-15T03:08:24.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5ea59c700 1 -- 172.21.15.121:0/3370917466 learned_addr learned my addr 172.21.15.121:0/3370917466 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:24.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5e9d9b700 1 --2- 172.21.15.121:0/3370917466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e13eb0 0x1e60140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5ead9d700 1 --2- 172.21.15.121:0/3370917466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e60630 0x1e60ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5ea59c700 1 -- 172.21.15.121:0/3370917466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e13eb0 msgr2=0x1e60140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:24.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5ea59c700 1 --2- 172.21.15.121:0/3370917466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e13eb0 0x1e60140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5ea59c700 1 -- 172.21.15.121:0/3370917466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e60630 msgr2=0x1e60ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:24.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5ea59c700 1 --2- 172.21.15.121:0/3370917466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e60630 0x1e60ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5ea59c700 1 -- 172.21.15.121:0/3370917466 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1d8d730 con 0x1e13ad0 2023-11-15T03:08:24.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5ead9d700 1 --2- 172.21.15.121:0/3370917466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e60630 0x1e60ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:24.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5ea59c700 1 --2- 172.21.15.121:0/3370917466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13ad0 0x1e3a090 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 rx=0x7fa5dc005990 tx=0x7fa5dc00a000).ready entity=mon.0 client_cookie=f09e1efbf793b1cb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:24.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5cb7fe700 1 -- 172.21.15.121:0/3370917466 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc005ed0 con 0x1e13ad0 2023-11-15T03:08:24.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5cb7fe700 1 -- 172.21.15.121:0/3370917466 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5dc016a90 con 0x1e13ad0 2023-11-15T03:08:24.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5cb7fe700 1 -- 172.21.15.121:0/3370917466 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc016c00 con 0x1e13ad0 2023-11-15T03:08:24.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5fade1740 1 -- 172.21.15.121:0/3370917466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13ad0 msgr2=0x1e3a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:24.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5fade1740 1 --2- 172.21.15.121:0/3370917466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13ad0 0x1e3a090 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 rx=0x7fa5dc005990 tx=0x7fa5dc00a000).stop 2023-11-15T03:08:24.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5fade1740 1 -- 172.21.15.121:0/3370917466 shutdown_connections 2023-11-15T03:08:24.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5fade1740 1 --2- 172.21.15.121:0/3370917466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e60630 0x1e60ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5fade1740 1 --2- 172.21.15.121:0/3370917466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e13eb0 0x1e60140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5fade1740 1 --2- 172.21.15.121:0/3370917466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13ad0 0x1e3a090 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5fade1740 1 -- 172.21.15.121:0/3370917466 >> 172.21.15.121:0/3370917466 conn(0x1bf2550 msgr2=0x1e127e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:24.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5fade1740 1 -- 172.21.15.121:0/3370917466 shutdown_connections 2023-11-15T03:08:24.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.135+0000 7fa5fade1740 1 -- 172.21.15.121:0/3370917466 wait complete. 2023-11-15T03:08:24.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.139+0000 7fa5fade1740 1 -- start start 2023-11-15T03:08:24.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.139+0000 7fa5fade1740 1 Processor -- start 2023-11-15T03:08:24.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.139+0000 7fa5fade1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e13ad0 0x1e3d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.139+0000 7fa5fade1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 0x1e39560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.139+0000 7fa5fade1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e60630 0x1e54080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.139+0000 7fa5fade1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d5f650 con 0x1e13eb0 2023-11-15T03:08:24.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.139+0000 7fa5fade1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e3f490 con 0x1e13ad0 2023-11-15T03:08:24.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.139+0000 7fa5fade1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e5f1d0 con 0x1e60630 2023-11-15T03:08:24.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.139+0000 7fa5fade1740 10 client.0 ll_register_callbacks cb 0x1dcc020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:08:24.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.139+0000 7fa5e9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 0x1e39560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.139+0000 7fa5e9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 0x1e39560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51982/0 (socket says 172.21.15.121:51982) 2023-11-15T03:08:24.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.139+0000 7fa5e9d9b700 1 -- 172.21.15.121:0/979602654 learned_addr learned my addr 172.21.15.121:0/979602654 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:24.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.139+0000 7fa5ead9d700 1 --2- 172.21.15.121:0/979602654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e60630 0x1e54080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.139+0000 7fa5ea59c700 1 --2- 172.21.15.121:0/979602654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e13ad0 0x1e3d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5e9d9b700 1 -- 172.21.15.121:0/979602654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e60630 msgr2=0x1e54080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:24.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5e9d9b700 1 --2- 172.21.15.121:0/979602654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e60630 0x1e54080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5e9d9b700 1 -- 172.21.15.121:0/979602654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e13ad0 msgr2=0x1e3d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:24.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5e9d9b700 1 --2- 172.21.15.121:0/979602654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e13ad0 0x1e3d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5e9d9b700 1 -- 172.21.15.121:0/979602654 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ec066640 con 0x1e13eb0 2023-11-15T03:08:24.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5ea59c700 1 --2- 172.21.15.121:0/979602654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e13ad0 0x1e3d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:24.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5e9d9b700 1 --2- 172.21.15.121:0/979602654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 0x1e39560 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 rx=0x7fa5e0009df0 tx=0x7fa5e0008040).ready entity=mon.0 client_cookie=f8737f1deb72e9a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:24.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:08:24.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5c8ff9700 1 -- 172.21.15.121:0/979602654 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0015df0 con 0x1e13eb0 2023-11-15T03:08:24.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5c8ff9700 1 -- 172.21.15.121:0/979602654 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8502+0+0 (secure 0 0 0) 0x7fa5e0017440 con 0x1e13eb0 2023-11-15T03:08:24.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5fade1740 1 -- 172.21.15.121:0/979602654 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1d8a1a0 con 0x1e13eb0 2023-11-15T03:08:24.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5c8ff9700 1 -- 172.21.15.121:0/979602654 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=248) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5e0017810 con 0x1e13eb0 2023-11-15T03:08:24.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5fade1740 10 client.9507 fetch_fsmap learned FSMap version 248 2023-11-15T03:08:24.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5fade1740 1 -- 172.21.15.121:0/979602654 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=248,monmap=2+}) v3 -- 0x1d88960 con 0x1e13eb0 2023-11-15T03:08:24.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5c8ff9700 1 -- 172.21.15.121:0/979602654 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1d8a1a0 con 0x1e13eb0 2023-11-15T03:08:24.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5c8ff9700 1 -- 172.21.15.121:0/979602654 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 248) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fa5e0017970 con 0x1e13eb0 2023-11-15T03:08:24.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5fade1740 10 client.9507 fetch_fsmap finished waiting for FSMap version 248 2023-11-15T03:08:24.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5fade1740 10 client.9507 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:08:24.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5fade1740 20 client.9507 populate_metadata read hostname 'smithi121' 2023-11-15T03:08:24.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5fade1740 1 --2- 172.21.15.121:0/979602654 >> [v2:172.21.15.121:6834/1958064439,v1:172.21.15.121:6836/1958064439] conn(0x1e5ece0 0x1e5da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5fade1740 4 client.9507 mds_command: new command op to 9417 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:08:24.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5fade1740 1 -- 172.21.15.121:0/979602654 --> [v2:172.21.15.121:6834/1958064439,v1:172.21.15.121:6836/1958064439] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1d806f0 con 0x1e5ece0 2023-11-15T03:08:24.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.143+0000 7fa5ea59c700 1 --2- 172.21.15.121:0/979602654 >> [v2:172.21.15.121:6834/1958064439,v1:172.21.15.121:6836/1958064439] conn(0x1e5ece0 0x1e5da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.147+0000 7fa5c8ff9700 10 client.9507 ms_handle_connect on v2:172.21.15.121:6834/1958064439 2023-11-15T03:08:24.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.147+0000 7fa5ea59c700 1 --2- 172.21.15.121:0/979602654 >> [v2:172.21.15.121:6834/1958064439,v1:172.21.15.121:6836/1958064439] conn(0x1e5ece0 0x1e5da90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=4df0c738bd8cd660 server_cookie=f71bdd7bb0684974 in_seq=0 out_seq=0 2023-11-15T03:08:24.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.147+0000 7fa5c8ff9700 1 -- 172.21.15.121:0/979602654 <== mds.0 v2:172.21.15.121:6834/1958064439 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x1d806f0 con 0x1e5ece0 2023-11-15T03:08:24.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.147+0000 7fa5c8ff9700 10 client.9507 handle_command_reply: tid=0 2023-11-15T03:08:24.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.147+0000 7fa5fade1740 1 client.9507 shutdown 2023-11-15T03:08:24.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.147+0000 7fa5fade1740 1 -- 172.21.15.121:0/979602654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 msgr2=0x1e39560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:24.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.147+0000 7fa5fade1740 1 --2- 172.21.15.121:0/979602654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 0x1e39560 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 rx=0x7fa5e0009df0 tx=0x7fa5e0008040).stop 2023-11-15T03:08:24.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.147+0000 7fa5fade1740 1 -- 172.21.15.121:0/979602654 shutdown_connections 2023-11-15T03:08:24.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.147+0000 7fa5fade1740 1 --2- 172.21.15.121:0/979602654 >> [v2:172.21.15.121:6834/1958064439,v1:172.21.15.121:6836/1958064439] conn(0x1e5ece0 0x1e5da90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.147+0000 7fa5fade1740 1 --2- 172.21.15.121:0/979602654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e60630 0x1e54080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.147+0000 7fa5fade1740 1 --2- 172.21.15.121:0/979602654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 0x1e39560 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.147+0000 7fa5fade1740 1 --2- 172.21.15.121:0/979602654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e13ad0 0x1e3d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.147+0000 7fa5c8ff9700 0 client.9507 ms_handle_reset on v2:172.21.15.121:6834/1958064439 2023-11-15T03:08:24.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.147+0000 7fa5fade1740 1 -- 172.21.15.121:0/979602654 >> 172.21.15.121:0/979602654 conn(0x1bf2550 msgr2=0x1e117e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:24.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.147+0000 7fa5fade1740 1 -- 172.21.15.121:0/979602654 shutdown_connections 2023-11-15T03:08:24.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.147+0000 7fa5fade1740 1 -- 172.21.15.121:0/979602654 wait complete. 2023-11-15T03:08:24.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.147+0000 7fa5fade1740 20 client.9507 trim_cache size 0 max 16384 2023-11-15T03:08:24.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5fade1740 1 Processor -- start 2023-11-15T03:08:24.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5fade1740 1 -- start start 2023-11-15T03:08:24.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5fade1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e13ad0 0x1e6fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5fade1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 0x1e70270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5fade1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e60630 0x1e37d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5fade1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e5c820 con 0x1e13eb0 2023-11-15T03:08:24.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5fade1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e5eb60 con 0x1e13ad0 2023-11-15T03:08:24.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5fade1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e0ebb0 con 0x1e60630 2023-11-15T03:08:24.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5e9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 0x1e70270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5e9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 0x1e70270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51988/0 (socket says 172.21.15.121:51988) 2023-11-15T03:08:24.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5e9d9b700 1 -- 172.21.15.121:0/2621547210 learned_addr learned my addr 172.21.15.121:0/2621547210 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:24.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5ea59c700 1 --2- 172.21.15.121:0/2621547210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e13ad0 0x1e6fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5ead9d700 1 --2- 172.21.15.121:0/2621547210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e60630 0x1e37d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5e9d9b700 1 -- 172.21.15.121:0/2621547210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e60630 msgr2=0x1e37d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:24.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5e9d9b700 1 --2- 172.21.15.121:0/2621547210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e60630 0x1e37d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5e9d9b700 1 -- 172.21.15.121:0/2621547210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e13ad0 msgr2=0x1e6fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:24.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5e9d9b700 1 --2- 172.21.15.121:0/2621547210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e13ad0 0x1e6fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5e9d9b700 1 -- 172.21.15.121:0/2621547210 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1e0fa90 con 0x1e13eb0 2023-11-15T03:08:24.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5ea59c700 1 --2- 172.21.15.121:0/2621547210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e13ad0 0x1e6fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:24.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5ead9d700 1 --2- 172.21.15.121:0/2621547210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e60630 0x1e37d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:08:24.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.167+0000 7fa5e9d9b700 1 --2- 172.21.15.121:0/2621547210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 0x1e70270 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 rx=0x7fa5e0006e00 tx=0x7fa5e0024000).ready entity=mon.0 client_cookie=7bb209e7dc8d2e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:24.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.171+0000 7fa5cbfff700 1 -- 172.21.15.121:0/2621547210 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0009340 con 0x1e13eb0 2023-11-15T03:08:24.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.171+0000 7fa5cbfff700 1 -- 172.21.15.121:0/2621547210 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e0015df0 con 0x1e13eb0 2023-11-15T03:08:24.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.171+0000 7fa5cbfff700 1 -- 172.21.15.121:0/2621547210 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0017a20 con 0x1e13eb0 2023-11-15T03:08:24.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.171+0000 7fa5fade1740 1 -- 172.21.15.121:0/2621547210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 msgr2=0x1e70270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:24.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.171+0000 7fa5fade1740 1 --2- 172.21.15.121:0/2621547210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 0x1e70270 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 rx=0x7fa5e0006e00 tx=0x7fa5e0024000).stop 2023-11-15T03:08:24.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.171+0000 7fa5fade1740 1 -- 172.21.15.121:0/2621547210 shutdown_connections 2023-11-15T03:08:24.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.171+0000 7fa5fade1740 1 --2- 172.21.15.121:0/2621547210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e60630 0x1e37d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.171+0000 7fa5fade1740 1 --2- 172.21.15.121:0/2621547210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 0x1e70270 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.171+0000 7fa5fade1740 1 --2- 172.21.15.121:0/2621547210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e13ad0 0x1e6fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.171+0000 7fa5fade1740 1 -- 172.21.15.121:0/2621547210 >> 172.21.15.121:0/2621547210 conn(0x1bf2550 msgr2=0x1e11980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:24.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.171+0000 7fa5fade1740 1 -- 172.21.15.121:0/2621547210 shutdown_connections 2023-11-15T03:08:24.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.171+0000 7fa5fade1740 1 -- 172.21.15.121:0/2621547210 wait complete. 2023-11-15T03:08:24.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.171+0000 7fa5fade1740 1 -- start start 2023-11-15T03:08:24.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.171+0000 7fa5fade1740 1 Processor -- start 2023-11-15T03:08:24.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5fade1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e13ad0 0x1e3c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5fade1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 0x1ec5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5fade1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e60630 0x1e6fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5fade1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e597f0 con 0x1e13eb0 2023-11-15T03:08:24.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5fade1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ec5850 con 0x1e13ad0 2023-11-15T03:08:24.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5fade1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e57fd0 con 0x1e60630 2023-11-15T03:08:24.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5fade1740 10 client.0 ll_register_callbacks cb 0x7fa5e0009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:08:24.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5e9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 0x1ec5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5e9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 0x1ec5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:52000/0 (socket says 172.21.15.121:52000) 2023-11-15T03:08:24.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5e9d9b700 1 -- 172.21.15.121:0/2054798758 learned_addr learned my addr 172.21.15.121:0/2054798758 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:24.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5ea59c700 1 --2- 172.21.15.121:0/2054798758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e13ad0 0x1e3c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5ead9d700 1 --2- 172.21.15.121:0/2054798758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e60630 0x1e6fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5e9d9b700 1 -- 172.21.15.121:0/2054798758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e60630 msgr2=0x1e6fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:24.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5e9d9b700 1 --2- 172.21.15.121:0/2054798758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e60630 0x1e6fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5e9d9b700 1 -- 172.21.15.121:0/2054798758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e13ad0 msgr2=0x1e3c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:24.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5e9d9b700 1 --2- 172.21.15.121:0/2054798758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e13ad0 0x1e3c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5e9d9b700 1 -- 172.21.15.121:0/2054798758 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1d88960 con 0x1e13eb0 2023-11-15T03:08:24.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5e9d9b700 1 --2- 172.21.15.121:0/2054798758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 0x1ec5360 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 rx=0x7fa5e002d040 tx=0x7fa5e0022040).ready entity=mon.0 client_cookie=cd643b50a956b9f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:24.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:08:24.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5c8ff9700 1 -- 172.21.15.121:0/2054798758 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0037070 con 0x1e13eb0 2023-11-15T03:08:24.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5c8ff9700 1 -- 172.21.15.121:0/2054798758 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8502+0+0 (secure 0 0 0) 0x7fa5e0000e40 con 0x1e13eb0 2023-11-15T03:08:24.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.175+0000 7fa5fade1740 1 -- 172.21.15.121:0/2054798758 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1e5a990 con 0x1e13eb0 2023-11-15T03:08:24.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5c8ff9700 1 -- 172.21.15.121:0/2054798758 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=248) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5e0009340 con 0x1e13eb0 2023-11-15T03:08:24.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 10 client.9513 fetch_fsmap learned FSMap version 248 2023-11-15T03:08:24.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 1 -- 172.21.15.121:0/2054798758 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=248,monmap=2+}) v3 -- 0x7fa5ec0c74c0 con 0x1e13eb0 2023-11-15T03:08:24.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5c8ff9700 1 -- 172.21.15.121:0/2054798758 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1e5a990 con 0x1e13eb0 2023-11-15T03:08:24.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5c8ff9700 1 -- 172.21.15.121:0/2054798758 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 248) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fa5e003a020 con 0x1e13eb0 2023-11-15T03:08:24.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 10 client.9513 fetch_fsmap finished waiting for FSMap version 248 2023-11-15T03:08:24.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 10 client.9513 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:08:24.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 20 client.9513 populate_metadata read hostname 'smithi121' 2023-11-15T03:08:24.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 1 --2- 172.21.15.121:0/2054798758 >> [v2:172.21.15.121:6834/1958064439,v1:172.21.15.121:6836/1958064439] conn(0x1e144c0 0x1e148a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 4 client.9513 mds_command: new command op to 9417 tid=0[{"prefix": "status"}] 2023-11-15T03:08:24.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 1 -- 172.21.15.121:0/2054798758 --> [v2:172.21.15.121:6834/1958064439,v1:172.21.15.121:6836/1958064439] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fa5dc016a90 con 0x1e144c0 2023-11-15T03:08:24.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5ea59c700 1 --2- 172.21.15.121:0/2054798758 >> [v2:172.21.15.121:6834/1958064439,v1:172.21.15.121:6836/1958064439] conn(0x1e144c0 0x1e148a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5ea59c700 1 --2- 172.21.15.121:0/2054798758 >> [v2:172.21.15.121:6834/1958064439,v1:172.21.15.121:6836/1958064439] conn(0x1e144c0 0x1e148a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=10334ec970263504 server_cookie=51bbc26cb4c5134c in_seq=0 out_seq=0 2023-11-15T03:08:24.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5c8ff9700 10 client.9513 ms_handle_connect on v2:172.21.15.121:6834/1958064439 2023-11-15T03:08:24.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5c8ff9700 1 -- 172.21.15.121:0/2054798758 <== mds.0 v2:172.21.15.121:6834/1958064439 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x7fa5dc016a90 con 0x1e144c0 2023-11-15T03:08:24.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5c8ff9700 10 client.9513 handle_command_reply: tid=0 2023-11-15T03:08:24.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 1 client.9513 shutdown 2023-11-15T03:08:24.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 1 -- 172.21.15.121:0/2054798758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 msgr2=0x1ec5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:24.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 1 --2- 172.21.15.121:0/2054798758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 0x1ec5360 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 rx=0x7fa5e002d040 tx=0x7fa5e0022040).stop 2023-11-15T03:08:24.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 1 -- 172.21.15.121:0/2054798758 shutdown_connections 2023-11-15T03:08:24.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 1 --2- 172.21.15.121:0/2054798758 >> [v2:172.21.15.121:6834/1958064439,v1:172.21.15.121:6836/1958064439] conn(0x1e144c0 0x1e148a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 1 --2- 172.21.15.121:0/2054798758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e60630 0x1e6fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5c8ff9700 0 client.9513 ms_handle_reset on v2:172.21.15.121:6834/1958064439 2023-11-15T03:08:24.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 1 --2- 172.21.15.121:0/2054798758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e13eb0 0x1ec5360 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 1 --2- 172.21.15.121:0/2054798758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e13ad0 0x1e3c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 1 -- 172.21.15.121:0/2054798758 >> 172.21.15.121:0/2054798758 conn(0x1bf2550 msgr2=0x1e51150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:24.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 1 -- 172.21.15.121:0/2054798758 shutdown_connections 2023-11-15T03:08:24.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 1 -- 172.21.15.121:0/2054798758 wait complete. 2023-11-15T03:08:24.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.179+0000 7fa5fade1740 20 client.9513 trim_cache size 0 max 16384 2023-11-15T03:08:24.199 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:08:24.199 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T03:08:24.199 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T03:08:24.200 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 9417, 2023-11-15T03:08:24.200 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T03:08:24.200 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T03:08:24.200 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T03:08:24.200 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 6.9004463810000001, 2023-11-15T03:08:24.201 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 248, 2023-11-15T03:08:24.201 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 120, 2023-11-15T03:08:24.201 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 120, 2023-11-15T03:08:24.201 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 16.820130273 2023-11-15T03:08:24.201 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:08:24.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.183+0000 7fa5eb59e700 1 -- 172.21.15.121:0/3155159678 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa5cc05a540 msgr2=0x7fa5cc05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:24.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.183+0000 7fa5eb59e700 1 --2- 172.21.15.121:0/3155159678 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa5cc05a540 0x7fa5cc05ca00 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7fa5dc000ef0 tx=0x7fa5dc009040).stop 2023-11-15T03:08:24.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.183+0000 7fa5eb59e700 1 -- 172.21.15.121:0/3155159678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5ec150a00 msgr2=0x7fa5ec0ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:24.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.183+0000 7fa5eb59e700 1 --2- 172.21.15.121:0/3155159678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5ec150a00 0x7fa5ec0ca040 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 rx=0x7fa5d40045a0 tx=0x7fa5e401a000).stop 2023-11-15T03:08:24.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.183+0000 7fa5eb59e700 1 -- 172.21.15.121:0/3155159678 shutdown_connections 2023-11-15T03:08:24.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.183+0000 7fa5eb59e700 1 --2- 172.21.15.121:0/3155159678 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa5cc05a540 0x7fa5cc05ca00 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.183+0000 7fa5eb59e700 1 --2- 172.21.15.121:0/3155159678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5ec150a00 0x7fa5ec0ca040 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.183+0000 7fa5eb59e700 1 --2- 172.21.15.121:0/3155159678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5ec150580 0x7fa5ec0c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.183+0000 7fa5eb59e700 1 --2- 172.21.15.121:0/3155159678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5d4004b80 0x7fa5ec156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.183+0000 7fa5eb59e700 1 -- 172.21.15.121:0/3155159678 >> 172.21.15.121:0/3155159678 conn(0x7fa5ec067a60 msgr2=0x7fa5ec0add90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:24.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.183+0000 7fa5eb59e700 1 -- 172.21.15.121:0/3155159678 shutdown_connections 2023-11-15T03:08:24.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.183+0000 7fa5eb59e700 1 -- 172.21.15.121:0/3155159678 wait complete. 2023-11-15T03:08:24.211 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:08:24.212 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Truncate 500.00000000', proceeding to ls 2023-11-15T03:08:24.212 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T03:08:24.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.339+0000 7f3bc8f69700 1 Processor -- start 2023-11-15T03:08:24.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.339+0000 7f3bc8f69700 1 -- start start 2023-11-15T03:08:24.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.343+0000 7f3bc8f69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bc40c7520 0x7f3bc40c1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.343+0000 7f3bc8f69700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3bc40aa980 0x7f3bc40aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.343+0000 7f3bc8f69700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc40c0f70 con 0x7f3bc40c7900 2023-11-15T03:08:24.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.343+0000 7f3bc8f69700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc40c8470 con 0x7f3bc40aa980 2023-11-15T03:08:24.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.343+0000 7f3bc8f69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc40c5db0 con 0x7f3bc40c7520 2023-11-15T03:08:24.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.343+0000 7f3bc1d9b700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3bc40c7900 0x7f3bc40c7d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:33708/0 (socket says 172.21.15.121:33708) 2023-11-15T03:08:24.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.343+0000 7f3bc1d9b700 1 -- 172.21.15.121:0/3550849520 learned_addr learned my addr 172.21.15.121:0/3550849520 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.343+0000 7f3bc259c700 1 --2- 172.21.15.121:0/3550849520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bc40c7520 0x7f3bc40c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.343+0000 7f3bc2d9d700 1 --2- 172.21.15.121:0/3550849520 >> v2:172.21.15.190:3300/0 conn(0x7f3bc40aa980 0x7f3bc40aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.343+0000 7f3bc259c700 1 -- 172.21.15.121:0/3550849520 >> v2:172.21.15.190:3300/0 conn(0x7f3bc40aa980 msgr2=0x7f3bc40aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:24.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.343+0000 7f3bc259c700 1 --2- 172.21.15.121:0/3550849520 >> v2:172.21.15.190:3300/0 conn(0x7f3bc40aa980 0x7f3bc40aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.343+0000 7f3bc259c700 1 -- 172.21.15.121:0/3550849520 >> v1:172.21.15.121:6789/0 conn(0x7f3bc40c7900 legacy=0x7f3bc40c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:24.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.343+0000 7f3bc159a700 1 -- 172.21.15.121:0/3550849520 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3672127082 0 0) 0x7f3bc40c0f70 con 0x7f3bc40c7900 2023-11-15T03:08:24.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.343+0000 7f3bc259c700 1 -- 172.21.15.121:0/3550849520 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc40c6090 con 0x7f3bc40c7520 2023-11-15T03:08:24.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.343+0000 7f3bc259c700 1 --2- 172.21.15.121:0/3550849520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bc40c7520 0x7f3bc40c1400 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f3bb4009760 tx=0x7f3bb4009060).ready entity=mon.2 client_cookie=8951d5ac1201201e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:24.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.343+0000 7f3bc159a700 1 -- 172.21.15.121:0/3550849520 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb400f510 con 0x7f3bc40c7520 2023-11-15T03:08:24.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.343+0000 7f3bc159a700 1 -- 172.21.15.121:0/3550849520 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb400f670 con 0x7f3bc40c7520 2023-11-15T03:08:24.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.343+0000 7f3bc159a700 1 -- 172.21.15.121:0/3550849520 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb400f9f0 con 0x7f3bc40c7520 2023-11-15T03:08:24.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.347+0000 7f3bc8f69700 1 -- 172.21.15.121:0/3550849520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bc40c7520 msgr2=0x7f3bc40c1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:24.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.347+0000 7f3bc8f69700 1 --2- 172.21.15.121:0/3550849520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bc40c7520 0x7f3bc40c1400 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f3bb4009760 tx=0x7f3bb4009060).stop 2023-11-15T03:08:24.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.347+0000 7f3bc8f69700 1 -- 172.21.15.121:0/3550849520 shutdown_connections 2023-11-15T03:08:24.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.347+0000 7f3bc8f69700 1 --2- 172.21.15.121:0/3550849520 >> v2:172.21.15.190:3300/0 conn(0x7f3bc40aa980 0x7f3bc40aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.347+0000 7f3bc8f69700 1 --2- 172.21.15.121:0/3550849520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bc40c7520 0x7f3bc40c1400 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.347+0000 7f3bc8f69700 1 -- 172.21.15.121:0/3550849520 >> 172.21.15.121:0/3550849520 conn(0x7f3bc4067780 msgr2=0x7f3bc40681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:24.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.347+0000 7f3bc8f69700 1 -- 172.21.15.121:0/3550849520 shutdown_connections 2023-11-15T03:08:24.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.347+0000 7f3bc8f69700 1 -- 172.21.15.121:0/3550849520 wait complete. 2023-11-15T03:08:24.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.347+0000 7f3bc8f69700 1 Processor -- start 2023-11-15T03:08:24.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.351+0000 7f3bc8f69700 1 -- start start 2023-11-15T03:08:24.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.351+0000 7f3bc8f69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bc40aa980 0x7f3bc413f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.351+0000 7f3bc8f69700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3bc40c7520 0x7f3bc4151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.351+0000 7f3bc8f69700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3bc40c7900 0x7f3bc40aca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.351+0000 7f3bc8f69700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc4153150 con 0x7f3bc40c7520 2023-11-15T03:08:24.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.351+0000 7f3bc8f69700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc4152540 con 0x7f3bc40c7900 2023-11-15T03:08:24.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.351+0000 7f3bc8f69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc4152820 con 0x7f3bc40aa980 2023-11-15T03:08:24.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.351+0000 7f3bc1d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3bc40c7520 0x7f3bc4151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.351+0000 7f3bc1d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3bc40c7520 0x7f3bc4151bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:52012/0 (socket says 172.21.15.121:52012) 2023-11-15T03:08:24.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.351+0000 7f3bc1d9b700 1 -- 172.21.15.121:0/2773688274 learned_addr learned my addr 172.21.15.121:0/2773688274 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:24.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.351+0000 7f3bc259c700 1 --2- 172.21.15.121:0/2773688274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bc40aa980 0x7f3bc413f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.351+0000 7f3bc2d9d700 1 --2- 172.21.15.121:0/2773688274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3bc40c7900 0x7f3bc40aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.351+0000 7f3bc2d9d700 1 -- 172.21.15.121:0/2773688274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bc40aa980 msgr2=0x7f3bc413f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:24.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.351+0000 7f3bc2d9d700 1 --2- 172.21.15.121:0/2773688274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bc40aa980 0x7f3bc413f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.351+0000 7f3bc2d9d700 1 -- 172.21.15.121:0/2773688274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3bc40c7520 msgr2=0x7f3bc4151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:24.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.351+0000 7f3bc2d9d700 1 --2- 172.21.15.121:0/2773688274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3bc40c7520 0x7f3bc4151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.351+0000 7f3bc2d9d700 1 -- 172.21.15.121:0/2773688274 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb40092a0 con 0x7f3bc40c7900 2023-11-15T03:08:24.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.351+0000 7f3bc1d9b700 1 --2- 172.21.15.121:0/2773688274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3bc40c7520 0x7f3bc4151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:08:24.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.351+0000 7f3bc2d9d700 1 --2- 172.21.15.121:0/2773688274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3bc40c7900 0x7f3bc40aca10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f3bbc009e50 tx=0x7f3bbc00b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:24.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.355+0000 7f3bb27fc700 1 -- 172.21.15.121:0/2773688274 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc00caa0 con 0x7f3bc40c7900 2023-11-15T03:08:24.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.355+0000 7f3bb27fc700 1 -- 172.21.15.121:0/2773688274 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc00f040 con 0x7f3bc40c7900 2023-11-15T03:08:24.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.355+0000 7f3bb27fc700 1 -- 172.21.15.121:0/2773688274 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc00ce10 con 0x7f3bc40c7900 2023-11-15T03:08:24.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.355+0000 7f3bc8f69700 1 -- 172.21.15.121:0/2773688274 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc4152b00 con 0x7f3bc40c7900 2023-11-15T03:08:24.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.355+0000 7f3bc8f69700 1 -- 172.21.15.121:0/2773688274 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc413fbb0 con 0x7f3bc40c7900 2023-11-15T03:08:24.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.355+0000 7f3bc8f69700 1 -- 172.21.15.121:0/2773688274 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc405eeb0 con 0x7f3bc40c7900 2023-11-15T03:08:24.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.355+0000 7f3bb27fc700 1 -- 172.21.15.121:0/2773688274 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3bbc013610 con 0x7f3bc40c7900 2023-11-15T03:08:24.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.359+0000 7f3bb27fc700 1 --2- 172.21.15.121:0/2773688274 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ba405a510 0x7f3ba405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:24.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.359+0000 7f3bb27fc700 1 -- 172.21.15.121:0/2773688274 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8502+0+0 (secure 0 0 0) 0x7f3bbc075950 con 0x7f3bc40c7900 2023-11-15T03:08:24.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.359+0000 7f3bc259c700 1 --2- 172.21.15.121:0/2773688274 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ba405a510 0x7f3ba405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:24.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.359+0000 7f3bc259c700 1 --2- 172.21.15.121:0/2773688274 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ba405a510 0x7f3ba405c9d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f3bb4009fd0 tx=0x7f3bb4017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:24.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.367+0000 7f3bb27fc700 1 -- 172.21.15.121:0/2773688274 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3bbc048720 con 0x7f3bc40c7900 2023-11-15T03:08:24.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.563+0000 7f3bc8f69700 1 -- 172.21.15.121:0/2773688274 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3bc40c0c00 con 0x7f3bc40c7900 2023-11-15T03:08:24.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.563+0000 7f3bb27fc700 1 -- 172.21.15.121:0/2773688274 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v248) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3bbc04bd40 con 0x7f3bc40c7900 2023-11-15T03:08:24.573 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T03:08:24.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.567+0000 7f3bc8f69700 1 -- 172.21.15.121:0/2773688274 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ba405a510 msgr2=0x7f3ba405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:24.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.567+0000 7f3bc8f69700 1 --2- 172.21.15.121:0/2773688274 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ba405a510 0x7f3ba405c9d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f3bb4009fd0 tx=0x7f3bb4017000).stop 2023-11-15T03:08:24.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.567+0000 7f3bc8f69700 1 -- 172.21.15.121:0/2773688274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3bc40c7900 msgr2=0x7f3bc40aca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:24.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.567+0000 7f3bc8f69700 1 --2- 172.21.15.121:0/2773688274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3bc40c7900 0x7f3bc40aca10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f3bbc009e50 tx=0x7f3bbc00b040).stop 2023-11-15T03:08:24.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.567+0000 7f3bc8f69700 1 -- 172.21.15.121:0/2773688274 shutdown_connections 2023-11-15T03:08:24.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.567+0000 7f3bc8f69700 1 --2- 172.21.15.121:0/2773688274 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ba405a510 0x7f3ba405c9d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.567+0000 7f3bc8f69700 1 --2- 172.21.15.121:0/2773688274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3bc40c7900 0x7f3bc40aca10 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.567+0000 7f3bc8f69700 1 --2- 172.21.15.121:0/2773688274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3bc40c7520 0x7f3bc4151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.567+0000 7f3bc8f69700 1 --2- 172.21.15.121:0/2773688274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bc40aa980 0x7f3bc413f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:24.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.567+0000 7f3bc8f69700 1 -- 172.21.15.121:0/2773688274 >> 172.21.15.121:0/2773688274 conn(0x7f3bc4067780 msgr2=0x7f3bc40aadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:24.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.567+0000 7f3bc8f69700 1 -- 172.21.15.121:0/2773688274 shutdown_connections 2023-11-15T03:08:24.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:24.567+0000 7f3bc8f69700 1 -- 172.21.15.121:0/2773688274 wait complete. 2023-11-15T03:08:24.593 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T03:08:24.594 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T03:08:24.594 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T03:08:24.594 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T03:08:24.595 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T03:08:24.595 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T03:08:24.595 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T03:08:24.596 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T03:08:24.596 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:08:24.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:08:24.597 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T03:08:24.619 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T03:08:24.619 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T03:08:24.619 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T03:08:24.620 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:08:24.620 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T03:08:24.620 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:08:24.620 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:08:24.620 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:08:24.621 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T03:08:24.621 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:08:24.621 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:08:24.621 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T03:08:24.621 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:08:24.622 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:08:24.622 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T03:08:24.622 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:08:24.622 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T03:08:24.623 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 41735sec preferred_lft 41735sec 2023-11-15T03:08:24.623 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T03:08:24.623 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:08:24.623 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T03:08:24.623 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:08:24.624 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T03:08:24.624 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:08:24.624 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T03:08:24.624 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T03:08:24.626 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T03:08:24.626 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:08:24.626 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T03:08:24.626 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:08:24.626 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:08:24.627 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T03:08:24.627 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T03:08:24.627 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:08:24.627 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:08:24.627 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T03:08:24.628 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T03:08:24.628 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:08:24.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:08:24.630 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T03:08:24.712 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T03:08:24.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:08:24.715 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T03:08:24.778 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T03:08:24.778 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T03:08:24.778 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T03:08:24.779 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T03:08:24.779 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T03:08:24.779 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T03:08:24.779 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T03:08:24.779 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T03:08:24.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:08:24.780 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T03:08:24.781 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T03:08:24.781 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T03:08:24.781 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T03:08:24.782 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T03:08:24.782 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T03:08:24.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:08:24.919 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T03:08:24.936 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T03:08:24.937 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T03:08:24.938 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:08:24.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:08:24.939 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T03:08:25.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:08:25.020 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T03:08:25.084 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T03:08:25.085 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:08:25.200 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:08:25.201 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:08:25.225 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:08:25.225 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:08:25.230 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T03:08:25.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:08:25.231 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T03:08:25.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:08:25.263 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2023-11-15T03:08:25.316 INFO:teuthology.orchestra.run:waiting for 20 2023-11-15T03:08:25.333 INFO:teuthology.orchestra.run.smithi121.stdout:.: 2023-11-15T03:08:25.334 INFO:teuthology.orchestra.run.smithi121.stdout:subdir 2023-11-15T03:08:25.334 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:25.334 INFO:teuthology.orchestra.run.smithi121.stdout:./subdir: 2023-11-15T03:08:25.334 INFO:teuthology.orchestra.run.smithi121.stdout:sixmegs 2023-11-15T03:08:25.335 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 500.00000000' did not caused DAMAGED state 2023-11-15T03:08:25.336 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate mds_snaptable' 2023-11-15T03:08:25.336 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T03:08:25.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:08:25.337 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-15T03:08:25.424 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:08:25.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:08:25.425 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:08:25.487 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:08:25.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.695+0000 7fe772f69700 1 Processor -- start 2023-11-15T03:08:25.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.695+0000 7fe772f69700 1 -- start start 2023-11-15T03:08:25.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.699+0000 7fe772f69700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe76c153e80 0x7fe76c066680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:25.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.699+0000 7fe772f69700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe76c0c0490 0x7fe76c0c0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:25.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.699+0000 7fe772f69700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe76c0c5c50 con 0x7fe76c153aa0 2023-11-15T03:08:25.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.699+0000 7fe772f69700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe76c066b70 con 0x7fe76c153e80 2023-11-15T03:08:25.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.699+0000 7fe772f69700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe76c0c0e10 con 0x7fe76c0c0490 2023-11-15T03:08:25.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.699+0000 7fe763fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe76c153e80 0x7fe76c066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:25.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.699+0000 7fe763fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe76c153e80 0x7fe76c066680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53806/0 (socket says 172.21.15.121:53806) 2023-11-15T03:08:25.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.699+0000 7fe763fff700 1 -- 172.21.15.121:0/1466783792 learned_addr learned my addr 172.21.15.121:0/1466783792 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:25.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.699+0000 7fe771506700 1 --2- 172.21.15.121:0/1466783792 >> v2:172.21.15.190:3300/0 conn(0x7fe76c0c0490 0x7fe76c0c0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:25.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.699+0000 7fe763fff700 1 -- 172.21.15.121:0/1466783792 >> v2:172.21.15.190:3300/0 conn(0x7fe76c0c0490 msgr2=0x7fe76c0c0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:25.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.699+0000 7fe763fff700 1 --2- 172.21.15.121:0/1466783792 >> v2:172.21.15.190:3300/0 conn(0x7fe76c0c0490 0x7fe76c0c0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:25.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.699+0000 7fe763fff700 1 -- 172.21.15.121:0/1466783792 >> v1:172.21.15.190:6789/0 conn(0x7fe76c153aa0 legacy=0x7fe76c0ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:25.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.699+0000 7fe763fff700 1 -- 172.21.15.121:0/1466783792 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76c0c5920 con 0x7fe76c153e80 2023-11-15T03:08:25.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.699+0000 7fe763fff700 1 --2- 172.21.15.121:0/1466783792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe76c153e80 0x7fe76c066680 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 rx=0x7fe76c0c5bc0 tx=0x7fe75c0090f0).ready entity=mon.0 client_cookie=c41d92aa529cbccb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:25.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.699+0000 7fe7637fe700 1 -- 172.21.15.121:0/1466783792 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00b600 con 0x7fe76c153e80 2023-11-15T03:08:25.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.699+0000 7fe7637fe700 1 -- 172.21.15.121:0/1466783792 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c00b760 con 0x7fe76c153e80 2023-11-15T03:08:25.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.699+0000 7fe7637fe700 1 -- 172.21.15.121:0/1466783792 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00bae0 con 0x7fe76c153e80 2023-11-15T03:08:25.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.703+0000 7fe772f69700 1 -- 172.21.15.121:0/1466783792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe76c153e80 msgr2=0x7fe76c066680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:25.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.703+0000 7fe772f69700 1 --2- 172.21.15.121:0/1466783792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe76c153e80 0x7fe76c066680 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 rx=0x7fe76c0c5bc0 tx=0x7fe75c0090f0).stop 2023-11-15T03:08:25.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.703+0000 7fe772f69700 1 -- 172.21.15.121:0/1466783792 shutdown_connections 2023-11-15T03:08:25.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.703+0000 7fe772f69700 1 --2- 172.21.15.121:0/1466783792 >> v2:172.21.15.190:3300/0 conn(0x7fe76c0c0490 0x7fe76c0c0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:25.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.703+0000 7fe772f69700 1 --2- 172.21.15.121:0/1466783792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe76c153e80 0x7fe76c066680 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:25.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.703+0000 7fe772f69700 1 -- 172.21.15.121:0/1466783792 >> 172.21.15.121:0/1466783792 conn(0x7fe76c067a60 msgr2=0x7fe76c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:25.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.703+0000 7fe772f69700 1 -- 172.21.15.121:0/1466783792 shutdown_connections 2023-11-15T03:08:25.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.703+0000 7fe772f69700 1 -- 172.21.15.121:0/1466783792 wait complete. 2023-11-15T03:08:25.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe772f69700 1 Processor -- start 2023-11-15T03:08:25.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe772f69700 1 -- start start 2023-11-15T03:08:25.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe772f69700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe76c0c0490 0x7fe76c13f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:25.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe772f69700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe76c153aa0 0x7fe76c0c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:25.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe772f69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe76c153e80 0x7fe76c0c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:25.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe772f69700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c154fe0 con 0x7fe76c153aa0 2023-11-15T03:08:25.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe772f69700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c0c3cb0 con 0x7fe76c0c0490 2023-11-15T03:08:25.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe772f69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c154360 con 0x7fe76c153e80 2023-11-15T03:08:25.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe763fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe76c153aa0 0x7fe76c0c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:25.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe763fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe76c153aa0 0x7fe76c0c32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53808/0 (socket says 172.21.15.121:53808) 2023-11-15T03:08:25.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe763fff700 1 -- 172.21.15.121:0/1458681202 learned_addr learned my addr 172.21.15.121:0/1458681202 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:25.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe770d05700 1 --2- 172.21.15.121:0/1458681202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe76c0c0490 0x7fe76c13f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:25.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe771506700 1 --2- 172.21.15.121:0/1458681202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe76c153e80 0x7fe76c0c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:25.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe763fff700 1 -- 172.21.15.121:0/1458681202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe76c153e80 msgr2=0x7fe76c0c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:25.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe763fff700 1 --2- 172.21.15.121:0/1458681202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe76c153e80 0x7fe76c0c37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:25.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe763fff700 1 -- 172.21.15.121:0/1458681202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe76c0c0490 msgr2=0x7fe76c13f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:25.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe763fff700 1 --2- 172.21.15.121:0/1458681202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe76c0c0490 0x7fe76c13f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:25.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe763fff700 1 -- 172.21.15.121:0/1458681202 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c009370 con 0x7fe76c153aa0 2023-11-15T03:08:25.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe770d05700 1 --2- 172.21.15.121:0/1458681202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe76c0c0490 0x7fe76c13f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:25.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.707+0000 7fe771506700 1 --2- 172.21.15.121:0/1458681202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe76c153e80 0x7fe76c0c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:25.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.711+0000 7fe763fff700 1 --2- 172.21.15.121:0/1458681202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe76c153aa0 0x7fe76c0c32d0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 rx=0x7fe75c009130 tx=0x7fe75c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:25.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.711+0000 7fe760ff9700 1 -- 172.21.15.121:0/1458681202 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00b8d0 con 0x7fe76c153aa0 2023-11-15T03:08:25.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.711+0000 7fe760ff9700 1 -- 172.21.15.121:0/1458681202 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c00ba30 con 0x7fe76c153aa0 2023-11-15T03:08:25.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.711+0000 7fe772f69700 1 -- 172.21.15.121:0/1458681202 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe76c154640 con 0x7fe76c153aa0 2023-11-15T03:08:25.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.711+0000 7fe760ff9700 1 -- 172.21.15.121:0/1458681202 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c01b660 con 0x7fe76c153aa0 2023-11-15T03:08:25.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.711+0000 7fe772f69700 1 -- 172.21.15.121:0/1458681202 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76c154c00 con 0x7fe76c153aa0 2023-11-15T03:08:25.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.711+0000 7fe772f69700 1 -- 172.21.15.121:0/1458681202 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe76c047060 con 0x7fe76c153aa0 2023-11-15T03:08:25.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.711+0000 7fe760ff9700 1 -- 172.21.15.121:0/1458681202 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe75c01b9d0 con 0x7fe76c153aa0 2023-11-15T03:08:25.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.715+0000 7fe760ff9700 1 --2- 172.21.15.121:0/1458681202 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe75805a4e0 0x7fe75805c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:25.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.715+0000 7fe760ff9700 1 -- 172.21.15.121:0/1458681202 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8502+0+0 (secure 0 0 0) 0x7fe75c076ab0 con 0x7fe76c153aa0 2023-11-15T03:08:25.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.715+0000 7fe770d05700 1 --2- 172.21.15.121:0/1458681202 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe75805a4e0 0x7fe75805c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:25.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.715+0000 7fe770d05700 1 --2- 172.21.15.121:0/1458681202 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe75805a4e0 0x7fe75805c9a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7fe768001730 tx=0x7fe768009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:25.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.719+0000 7fe760ff9700 1 -- 172.21.15.121:0/1458681202 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe75c049880 con 0x7fe76c153aa0 2023-11-15T03:08:25.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:25.919+0000 7fe772f69700 1 -- 172.21.15.121:0/1458681202 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe76c068b20 con 0x7fe76c153aa0 2023-11-15T03:08:26.161 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:08:26.151+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:08:26.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.167+0000 7fe760ff9700 1 -- 172.21.15.121:0/1458681202 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v249) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe75c04cea0 con 0x7fe76c153aa0 2023-11-15T03:08:26.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.171+0000 7fe772f69700 1 -- 172.21.15.121:0/1458681202 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe75805a4e0 msgr2=0x7fe75805c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:26.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.171+0000 7fe772f69700 1 --2- 172.21.15.121:0/1458681202 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe75805a4e0 0x7fe75805c9a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7fe768001730 tx=0x7fe768009040).stop 2023-11-15T03:08:26.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.171+0000 7fe772f69700 1 -- 172.21.15.121:0/1458681202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe76c153aa0 msgr2=0x7fe76c0c32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:26.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.171+0000 7fe772f69700 1 --2- 172.21.15.121:0/1458681202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe76c153aa0 0x7fe76c0c32d0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 rx=0x7fe75c009130 tx=0x7fe75c017000).stop 2023-11-15T03:08:26.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.171+0000 7fe772f69700 1 -- 172.21.15.121:0/1458681202 shutdown_connections 2023-11-15T03:08:26.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.171+0000 7fe772f69700 1 --2- 172.21.15.121:0/1458681202 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe75805a4e0 0x7fe75805c9a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:26.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.171+0000 7fe772f69700 1 --2- 172.21.15.121:0/1458681202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe76c153e80 0x7fe76c0c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:26.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.171+0000 7fe772f69700 1 --2- 172.21.15.121:0/1458681202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe76c153aa0 0x7fe76c0c32d0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:26.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.171+0000 7fe772f69700 1 --2- 172.21.15.121:0/1458681202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe76c0c0490 0x7fe76c13f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:26.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.171+0000 7fe772f69700 1 -- 172.21.15.121:0/1458681202 >> 172.21.15.121:0/1458681202 conn(0x7fe76c067a60 msgr2=0x7fe76c14f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:26.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.171+0000 7fe772f69700 1 -- 172.21.15.121:0/1458681202 shutdown_connections 2023-11-15T03:08:26.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.171+0000 7fe772f69700 1 -- 172.21.15.121:0/1458681202 wait complete. 2023-11-15T03:08:26.184 INFO:tasks.ceph.mds.a.smithi121.stderr: -398> 2023-11-15T03:08:23.319+0000 7fc16ad99700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:08:26.185 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:08:26.202 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:08:26.226 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:08:26.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.355+0000 7fbe3759e700 1 Processor -- start 2023-11-15T03:08:26.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.355+0000 7fbe3759e700 1 -- start start 2023-11-15T03:08:26.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.355+0000 7fbe3759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe38062330 0x7fbe38062710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:26.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.355+0000 7fbe3759e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbe380bd140 0x7fbe38150330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:26.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.355+0000 7fbe3759e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe380c14d0 con 0x7fbe380bcce0 2023-11-15T03:08:26.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.355+0000 7fbe3759e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe380c2330 con 0x7fbe380bd140 2023-11-15T03:08:26.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.355+0000 7fbe3759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe38063800 con 0x7fbe38062330 2023-11-15T03:08:26.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.359+0000 7fbe36d9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbe380bd140 0x7fbe38150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:26.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.359+0000 7fbe3659c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe38062330 0x7fbe38062710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:26.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.359+0000 7fbe36d9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbe380bd140 0x7fbe38150330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53834/0 (socket says 172.21.15.121:53834) 2023-11-15T03:08:26.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.359+0000 7fbe36d9d700 1 -- 172.21.15.121:0/2510393748 learned_addr learned my addr 172.21.15.121:0/2510393748 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:26.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.359+0000 7fbe2ffff700 1 --1- 172.21.15.121:0/2510393748 >> v1:172.21.15.190:6789/0 conn(0x7fbe380bcce0 0x7fbe3814fc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:39600/0 (socket says 172.21.15.121:39600) 2023-11-15T03:08:26.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.359+0000 7fbe3659c700 1 -- 172.21.15.121:0/2510393748 >> v2:172.21.15.121:3300/0 conn(0x7fbe380bd140 msgr2=0x7fbe38150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:26.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.359+0000 7fbe3659c700 1 --2- 172.21.15.121:0/2510393748 >> v2:172.21.15.121:3300/0 conn(0x7fbe380bd140 0x7fbe38150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:26.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.359+0000 7fbe3659c700 1 -- 172.21.15.121:0/2510393748 >> v1:172.21.15.190:6789/0 conn(0x7fbe380bcce0 legacy=0x7fbe3814fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:26.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.359+0000 7fbe3659c700 1 -- 172.21.15.121:0/2510393748 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe380c1720 con 0x7fbe38062330 2023-11-15T03:08:26.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.359+0000 7fbe3659c700 1 --2- 172.21.15.121:0/2510393748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe38062330 0x7fbe38062710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fbe28009760 tx=0x7fbe28009060).ready entity=mon.2 client_cookie=5659fd3f829c1207 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:26.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.359+0000 7fbe35d9b700 1 -- 172.21.15.121:0/2510393748 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2800f510 con 0x7fbe38062330 2023-11-15T03:08:26.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.359+0000 7fbe35d9b700 1 -- 172.21.15.121:0/2510393748 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2800f670 con 0x7fbe38062330 2023-11-15T03:08:26.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.359+0000 7fbe35d9b700 1 -- 172.21.15.121:0/2510393748 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2800f9f0 con 0x7fbe38062330 2023-11-15T03:08:26.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.359+0000 7fbe3759e700 1 -- 172.21.15.121:0/2510393748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe38062330 msgr2=0x7fbe38062710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:26.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.359+0000 7fbe3759e700 1 --2- 172.21.15.121:0/2510393748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe38062330 0x7fbe38062710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fbe28009760 tx=0x7fbe28009060).stop 2023-11-15T03:08:26.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.359+0000 7fbe3759e700 1 -- 172.21.15.121:0/2510393748 shutdown_connections 2023-11-15T03:08:26.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.359+0000 7fbe3759e700 1 --2- 172.21.15.121:0/2510393748 >> v2:172.21.15.121:3300/0 conn(0x7fbe380bd140 0x7fbe38150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:26.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.359+0000 7fbe3759e700 1 --2- 172.21.15.121:0/2510393748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe38062330 0x7fbe38062710 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:26.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.359+0000 7fbe3759e700 1 -- 172.21.15.121:0/2510393748 >> 172.21.15.121:0/2510393748 conn(0x7fbe38067a60 msgr2=0x7fbe38067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:26.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.363+0000 7fbe3759e700 1 -- 172.21.15.121:0/2510393748 shutdown_connections 2023-11-15T03:08:26.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.363+0000 7fbe3759e700 1 -- 172.21.15.121:0/2510393748 wait complete. 2023-11-15T03:08:26.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.363+0000 7fbe3759e700 1 Processor -- start 2023-11-15T03:08:26.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.363+0000 7fbe3759e700 1 -- start start 2023-11-15T03:08:26.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe3759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe38062330 0x7fbe3814cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:26.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe3759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe380bcce0 0x7fbe380be5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:26.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe3759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe380bd140 0x7fbe38157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:26.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe3759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380befa0 con 0x7fbe380bcce0 2023-11-15T03:08:26.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe3759e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380bf280 con 0x7fbe380bd140 2023-11-15T03:08:26.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe3759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380bf560 con 0x7fbe38062330 2023-11-15T03:08:26.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe380bcce0 0x7fbe380be5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:26.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe380bcce0 0x7fbe380be5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53850/0 (socket says 172.21.15.121:53850) 2023-11-15T03:08:26.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe2ffff700 1 -- 172.21.15.121:0/2706484631 learned_addr learned my addr 172.21.15.121:0/2706484631 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:26.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe3659c700 1 --2- 172.21.15.121:0/2706484631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe38062330 0x7fbe3814cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:26.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe36d9d700 1 --2- 172.21.15.121:0/2706484631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe380bd140 0x7fbe38157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:26.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe2ffff700 1 -- 172.21.15.121:0/2706484631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe38062330 msgr2=0x7fbe3814cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:26.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe2ffff700 1 --2- 172.21.15.121:0/2706484631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe38062330 0x7fbe3814cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:26.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe2ffff700 1 -- 172.21.15.121:0/2706484631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe380bd140 msgr2=0x7fbe38157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:26.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe2ffff700 1 --2- 172.21.15.121:0/2706484631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe380bd140 0x7fbe38157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:26.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe2ffff700 1 -- 172.21.15.121:0/2706484631 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe280092a0 con 0x7fbe380bcce0 2023-11-15T03:08:26.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe36d9d700 1 --2- 172.21.15.121:0/2706484631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe380bd140 0x7fbe38157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:26.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe2ffff700 1 --2- 172.21.15.121:0/2706484631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe380bcce0 0x7fbe380be5c0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 rx=0x7fbe2000c410 tx=0x7fbe20007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:26.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe2e7fc700 1 -- 172.21.15.121:0/2706484631 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2000ce10 con 0x7fbe380bcce0 2023-11-15T03:08:26.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe2e7fc700 1 -- 172.21.15.121:0/2706484631 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe200072e0 con 0x7fbe380bcce0 2023-11-15T03:08:26.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe2e7fc700 1 -- 172.21.15.121:0/2706484631 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20005730 con 0x7fbe380bcce0 2023-11-15T03:08:26.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe3759e700 1 -- 172.21.15.121:0/2706484631 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe380bf840 con 0x7fbe380bcce0 2023-11-15T03:08:26.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.367+0000 7fbe3759e700 1 -- 172.21.15.121:0/2706484631 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe380c0040 con 0x7fbe380bcce0 2023-11-15T03:08:26.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.371+0000 7fbe3759e700 1 -- 172.21.15.121:0/2706484631 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe380ac0f0 con 0x7fbe380bcce0 2023-11-15T03:08:26.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.371+0000 7fbe2e7fc700 1 -- 172.21.15.121:0/2706484631 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fbe20005b20 con 0x7fbe380bcce0 2023-11-15T03:08:26.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.371+0000 7fbe2e7fc700 1 --2- 172.21.15.121:0/2706484631 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbe1c05a510 0x7fbe1c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:26.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.371+0000 7fbe2e7fc700 1 -- 172.21.15.121:0/2706484631 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8588+0+0 (secure 0 0 0) 0x7fbe20074670 con 0x7fbe380bcce0 2023-11-15T03:08:26.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.371+0000 7fbe3659c700 1 --2- 172.21.15.121:0/2706484631 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbe1c05a510 0x7fbe1c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:26.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.375+0000 7fbe3659c700 1 --2- 172.21.15.121:0/2706484631 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbe1c05a510 0x7fbe1c05c9d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7fbe28009fd0 tx=0x7fbe28017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:26.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.379+0000 7fbe2e7fc700 1 -- 172.21.15.121:0/2706484631 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbe380ac0f0 con 0x7fbe380bcce0 2023-11-15T03:08:26.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:26.595+0000 7fbe3759e700 1 -- 172.21.15.121:0/2706484631 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fbe380ac0f0 con 0x7fbe380bcce0 2023-11-15T03:08:27.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.179+0000 7fbe2e7fc700 1 -- 172.21.15.121:0/2706484631 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v250) v1 ==== 107+0+0 (secure 0 0 0) 0x7fbe380ac0f0 con 0x7fbe380bcce0 2023-11-15T03:08:27.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.183+0000 7fbe3759e700 1 -- 172.21.15.121:0/2706484631 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbe1c05a510 msgr2=0x7fbe1c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.183+0000 7fbe3759e700 1 --2- 172.21.15.121:0/2706484631 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbe1c05a510 0x7fbe1c05c9d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7fbe28009fd0 tx=0x7fbe28017000).stop 2023-11-15T03:08:27.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.183+0000 7fbe3759e700 1 -- 172.21.15.121:0/2706484631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe380bcce0 msgr2=0x7fbe380be5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.183+0000 7fbe3759e700 1 --2- 172.21.15.121:0/2706484631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe380bcce0 0x7fbe380be5c0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 rx=0x7fbe2000c410 tx=0x7fbe20007e10).stop 2023-11-15T03:08:27.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.183+0000 7fbe3759e700 1 -- 172.21.15.121:0/2706484631 shutdown_connections 2023-11-15T03:08:27.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.183+0000 7fbe3759e700 1 --2- 172.21.15.121:0/2706484631 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbe1c05a510 0x7fbe1c05c9d0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.183+0000 7fbe3759e700 1 --2- 172.21.15.121:0/2706484631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe380bd140 0x7fbe38157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.183+0000 7fbe3759e700 1 --2- 172.21.15.121:0/2706484631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe380bcce0 0x7fbe380be5c0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.183+0000 7fbe3759e700 1 --2- 172.21.15.121:0/2706484631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe38062330 0x7fbe3814cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.183+0000 7fbe3759e700 1 -- 172.21.15.121:0/2706484631 >> 172.21.15.121:0/2706484631 conn(0x7fbe38067a60 msgr2=0x7fbe3814f170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:27.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.183+0000 7fbe3759e700 1 -- 172.21.15.121:0/2706484631 shutdown_connections 2023-11-15T03:08:27.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.183+0000 7fbe3759e700 1 -- 172.21.15.121:0/2706484631 wait complete. 2023-11-15T03:08:27.196 INFO:teuthology.orchestra.run.smithi121.stderr:nothing to do: rank is not damaged 2023-11-15T03:08:27.214 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:08:27.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.303+0000 7f9e1b736d00 1 Processor -- start 2023-11-15T03:08:27.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.303+0000 7f9e1b736d00 1 -- start start 2023-11-15T03:08:27.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.303+0000 7f9e1b736d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e9e436440 0x557e9e4d4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.303+0000 7f9e1b736d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557e9e450c60 0x557e9e44f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.303+0000 7f9e1b736d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e9e4c32e0 0x557e9e44f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.303+0000 7f9e1b736d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9e37d220 con 0x557e9e4c32e0 2023-11-15T03:08:27.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.303+0000 7f9e1b736d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9e4db790 con 0x557e9e436440 2023-11-15T03:08:27.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.303+0000 7f9e1b736d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e9e44fd60 con 0x557e9e450c60 2023-11-15T03:08:27.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.303+0000 7f9e105a2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e9e4c32e0 0x557e9e44f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.303+0000 7f9e0fda1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e9e436440 0x557e9e4d4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.303+0000 7f9e105a2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e9e4c32e0 0x557e9e44f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53882/0 (socket says 172.21.15.121:53882) 2023-11-15T03:08:27.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.303+0000 7f9e105a2700 1 -- 172.21.15.121:0/932979188 learned_addr learned my addr 172.21.15.121:0/932979188 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:27.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.303+0000 7f9e075a0700 1 --2- 172.21.15.121:0/932979188 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557e9e450c60 0x557e9e44f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e105a2700 1 -- 172.21.15.121:0/932979188 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557e9e450c60 msgr2=0x557e9e44f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:27.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e105a2700 1 --2- 172.21.15.121:0/932979188 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557e9e450c60 0x557e9e44f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e105a2700 1 -- 172.21.15.121:0/932979188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e9e436440 msgr2=0x557e9e4d4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:27.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e105a2700 1 --2- 172.21.15.121:0/932979188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e9e436440 0x557e9e4d4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e105a2700 1 -- 172.21.15.121:0/932979188 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0800a180 con 0x557e9e4c32e0 2023-11-15T03:08:27.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e105a2700 1 --2- 172.21.15.121:0/932979188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e9e4c32e0 0x557e9e44f870 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 rx=0x7f9e0800ad90 tx=0x7f9e08017000).ready entity=mon.0 client_cookie=624ff301d477b005 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:27.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e0ed9f700 1 -- 172.21.15.121:0/932979188 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0800e6b0 con 0x557e9e4c32e0 2023-11-15T03:08:27.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e0ed9f700 1 -- 172.21.15.121:0/932979188 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0800e980 con 0x557e9e4c32e0 2023-11-15T03:08:27.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e0ed9f700 1 -- 172.21.15.121:0/932979188 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0800f680 con 0x557e9e4c32e0 2023-11-15T03:08:27.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e1b736d00 1 -- 172.21.15.121:0/932979188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e9e4c32e0 msgr2=0x557e9e44f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/932979188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e9e4c32e0 0x557e9e44f870 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 rx=0x7f9e0800ad90 tx=0x7f9e08017000).stop 2023-11-15T03:08:27.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e1b736d00 1 -- 172.21.15.121:0/932979188 shutdown_connections 2023-11-15T03:08:27.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/932979188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e9e4c32e0 0x557e9e44f870 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/932979188 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557e9e450c60 0x557e9e44f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/932979188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e9e436440 0x557e9e4d4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e1b736d00 1 -- 172.21.15.121:0/932979188 >> 172.21.15.121:0/932979188 conn(0x557e9e4057d0 msgr2=0x557e9e433470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:27.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e1b736d00 1 -- 172.21.15.121:0/932979188 shutdown_connections 2023-11-15T03:08:27.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e1b736d00 1 -- 172.21.15.121:0/932979188 wait complete. 2023-11-15T03:08:27.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e1b736d00 1 Processor -- start 2023-11-15T03:08:27.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.307+0000 7f9e1b736d00 1 -- start start 2023-11-15T03:08:27.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e1b736d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557e9e436440 0x557e9e4d97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e1b736d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e9e450c60 0x557e9e4d9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e1b736d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e9e4c32e0 0x557e9e4d7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e1b736d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9e4da190 con 0x557e9e4c32e0 2023-11-15T03:08:27.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e1b736d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9e4d3ed0 con 0x557e9e450c60 2023-11-15T03:08:27.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e1b736d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e9e4d4490 con 0x557e9e436440 2023-11-15T03:08:27.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e105a2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e9e4c32e0 0x557e9e4d7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e105a2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e9e4c32e0 0x557e9e4d7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53894/0 (socket says 172.21.15.121:53894) 2023-11-15T03:08:27.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 learned_addr learned my addr 172.21.15.121:0/1629733340 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:27.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e0fda1700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557e9e436440 0x557e9e4d97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e075a0700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e9e450c60 0x557e9e4d9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557e9e436440 msgr2=0x557e9e4d97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e105a2700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557e9e436440 0x557e9e4d97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e9e450c60 msgr2=0x557e9e4d9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e105a2700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557e9e450c60 0x557e9e4d9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e9e3a6a20 con 0x557e9e4c32e0 2023-11-15T03:08:27.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e105a2700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e9e4c32e0 0x557e9e4d7550 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 rx=0x7f9e08005ef0 tx=0x7f9e0801b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:27.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e0d59c700 1 -- 172.21.15.121:0/1629733340 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e08027070 con 0x557e9e4c32e0 2023-11-15T03:08:27.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e0d59c700 1 -- 172.21.15.121:0/1629733340 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0800e6a0 con 0x557e9e4c32e0 2023-11-15T03:08:27.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557e9e3a6770 con 0x557e9e4c32e0 2023-11-15T03:08:27.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e0d59c700 1 -- 172.21.15.121:0/1629733340 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e08005b30 con 0x557e9e4c32e0 2023-11-15T03:08:27.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.311+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e9e375a70 con 0x557e9e4c32e0 2023-11-15T03:08:27.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.315+0000 7f9e0d59c700 1 -- 172.21.15.121:0/1629733340 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9e08003040 con 0x557e9e4c32e0 2023-11-15T03:08:27.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.315+0000 7f9e0d59c700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9df0058ff0 0x7f9df005b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.315+0000 7f9e0d59c700 1 -- 172.21.15.121:0/1629733340 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8588+0+0 (secure 0 0 0) 0x7f9e08076340 con 0x557e9e4c32e0 2023-11-15T03:08:27.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.315+0000 7f9e0fda1700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9df0058ff0 0x7f9df005b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.331 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:08:27.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.315+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x557e9e4c03e0 0x557e9e4d8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.315+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e4d3800 con 0x557e9e4c03e0 2023-11-15T03:08:27.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.315+0000 7f9e0fda1700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9df0058ff0 0x7f9df005b4b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7f9e00009ae0 tx=0x7f9e00005bb0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:27.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.315+0000 7f9e075a0700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x557e9e4c03e0 0x557e9e4d8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.319+0000 7f9e075a0700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x557e9e4c03e0 0x557e9e4d8440 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:27.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.319+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v121'317 uv310 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e4c03e0 2023-11-15T03:08:27.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.319+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e4f9ff0 con 0x557e9e4c03e0 2023-11-15T03:08:27.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.323+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v121'318 uv318 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e4c03e0 2023-11-15T03:08:27.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.323+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e4fa3a0 con 0x557e9e4c03e0 2023-11-15T03:08:27.334 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:08:27.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.323+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v121'319 uv319 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e4c03e0 2023-11-15T03:08:27.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.323+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e4fb760 con 0x557e9e4c03e0 2023-11-15T03:08:27.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.327+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v121'320 uv320 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e4c03e0 2023-11-15T03:08:27.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.327+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e4fbb10 con 0x557e9e4c03e0 2023-11-15T03:08:27.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.331+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v121'321 uv321 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e4c03e0 2023-11-15T03:08:27.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.331+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e4fa750 con 0x557e9e4c03e0 2023-11-15T03:08:27.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.335+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v121'322 uv322 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e4c03e0 2023-11-15T03:08:27.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.335+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e4fab50 con 0x557e9e4c03e0 2023-11-15T03:08:27.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.339+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v121'323 uv316 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e4c03e0 2023-11-15T03:08:27.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.339+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e4faf50 con 0x557e9e4c03e0 2023-11-15T03:08:27.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.339+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v121'324 uv324 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e4c03e0 2023-11-15T03:08:27.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.339+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e4faf50 con 0x557e9e4c03e0 2023-11-15T03:08:27.352 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:08:27.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.343+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v121'325 uv325 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e4c03e0 2023-11-15T03:08:27.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.343+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e4faf50 con 0x557e9e4c03e0 2023-11-15T03:08:27.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.347+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v121'326 uv326 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e4c03e0 2023-11-15T03:08:27.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.347+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e4faf50 con 0x557e9e4c03e0 2023-11-15T03:08:27.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.347+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v121'327 uv327 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e4c03e0 2023-11-15T03:08:27.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.347+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e4faf00 con 0x557e9e4c03e0 2023-11-15T03:08:27.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.351+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v121'328 uv328 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e4c03e0 2023-11-15T03:08:27.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.351+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x557e9e4fb2e0 0x557e9e4fe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.351+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e4faf00 con 0x557e9e4fb2e0 2023-11-15T03:08:27.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.351+0000 7f9e0fda1700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x557e9e4fb2e0 0x557e9e4fe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.351+0000 7f9e0fda1700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x557e9e4fb2e0 0x557e9e4fe3e0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:27.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.355+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v121'499 uv486 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.355+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e4fe8d0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.359+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v121'500 uv500 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.359+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e4fec80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.372 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:08:27.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.363+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v121'501 uv501 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.363+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e500040 con 0x557e9e4fb2e0 2023-11-15T03:08:27.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.367+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v121'502 uv502 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.367+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5003f0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.367+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v121'503 uv503 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.371+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e4ff030 con 0x557e9e4fb2e0 2023-11-15T03:08:27.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.375+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v121'504 uv504 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.375+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e4ff920 con 0x557e9e4fb2e0 2023-11-15T03:08:27.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.375+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v121'505 uv505 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.375+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5007a0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.379+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v121'506 uv492 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.379+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e5007a0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.383+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v121'507 uv507 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.383+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5007a0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.394 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:08:27.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.387+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v121'508 uv508 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.387+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5017b0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.387+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v121'509 uv509 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.387+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5017b0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.391+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v121'510 uv510 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.391+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5007a0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.395+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v121'511 uv511 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.395+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5007a0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.395+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v121'512 uv498 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.395+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e5007a0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.403+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v121'513 uv513 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.403+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5007a0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.414 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:08:27.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.407+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v121'514 uv514 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.407+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5017b0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.407+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v121'515 uv515 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.407+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5017b0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.411+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v121'516 uv516 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.411+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5007a0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.415+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v121'517 uv517 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.415+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x557e9e500cb0 0x557e9e5030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.415+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e503590 con 0x557e9e500cb0 2023-11-15T03:08:27.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.415+0000 7f9e075a0700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x557e9e500cb0 0x557e9e5030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.415+0000 7f9e075a0700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x557e9e500cb0 0x557e9e5030a0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:27.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.419+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v121'274 uv267 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e500cb0 2023-11-15T03:08:27.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.419+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e503590 con 0x557e9e500cb0 2023-11-15T03:08:27.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.423+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v121'275 uv275 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e500cb0 2023-11-15T03:08:27.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.423+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e503590 con 0x557e9e500cb0 2023-11-15T03:08:27.433 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:08:27.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.423+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v121'276 uv276 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e500cb0 2023-11-15T03:08:27.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.423+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e503a80 con 0x557e9e500cb0 2023-11-15T03:08:27.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.431+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v121'277 uv277 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e500cb0 2023-11-15T03:08:27.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.431+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e503a80 con 0x557e9e500cb0 2023-11-15T03:08:27.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.431+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v121'278 uv278 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e500cb0 2023-11-15T03:08:27.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.431+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e503a80 con 0x557e9e500cb0 2023-11-15T03:08:27.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.435+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v121'279 uv273 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e500cb0 2023-11-15T03:08:27.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.435+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e503a80 con 0x557e9e500cb0 2023-11-15T03:08:27.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.439+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v121'280 uv280 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e500cb0 2023-11-15T03:08:27.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.439+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e503a80 con 0x557e9e500cb0 2023-11-15T03:08:27.450 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:08:27.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.443+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v121'281 uv281 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e500cb0 2023-11-15T03:08:27.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.443+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e503a80 con 0x557e9e500cb0 2023-11-15T03:08:27.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.443+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v121'282 uv282 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e500cb0 2023-11-15T03:08:27.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.443+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e503a80 con 0x557e9e500cb0 2023-11-15T03:08:27.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.447+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v121'283 uv283 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e500cb0 2023-11-15T03:08:27.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.447+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x557e9e503f90 0x557e9e506380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.447+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e506870 con 0x557e9e503f90 2023-11-15T03:08:27.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.447+0000 7f9e105a2700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x557e9e503f90 0x557e9e506380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.451+0000 7f9e105a2700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x557e9e503f90 0x557e9e506380 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:27.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.455+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v121'322 uv321 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.455+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e506c20 con 0x557e9e503f90 2023-11-15T03:08:27.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.455+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v121'323 uv323 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.455+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e506fd0 con 0x557e9e503f90 2023-11-15T03:08:27.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.459+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v121'324 uv324 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.469 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:08:27.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.459+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e503650 con 0x557e9e503f90 2023-11-15T03:08:27.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.463+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v121'325 uv325 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.463+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50ba50 con 0x557e9e503f90 2023-11-15T03:08:27.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.467+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v121'326 uv326 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.467+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5073d0 con 0x557e9e503f90 2023-11-15T03:08:27.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.471+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v121'327 uv318 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.471+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e5077d0 con 0x557e9e503f90 2023-11-15T03:08:27.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.471+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v121'328 uv328 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.471+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e508140 con 0x557e9e503f90 2023-11-15T03:08:27.484 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:08:27.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.475+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v121'329 uv329 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.475+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5091d0 con 0x557e9e503f90 2023-11-15T03:08:27.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.479+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v121'330 uv330 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.479+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e509170 con 0x557e9e503f90 2023-11-15T03:08:27.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.483+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v121'331 uv331 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.483+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e508070 con 0x557e9e503f90 2023-11-15T03:08:27.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.483+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v121'332 uv332 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.487+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x557e9e508980 0x557e9e50be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.487+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e508070 con 0x557e9e508980 2023-11-15T03:08:27.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.487+0000 7f9e0fda1700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x557e9e508980 0x557e9e50be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.487+0000 7f9e0fda1700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x557e9e508980 0x557e9e50be50 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:27.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.491+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v121'159 uv158 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e508980 2023-11-15T03:08:27.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.491+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e508070 con 0x557e9e508980 2023-11-15T03:08:27.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.491+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v121'160 uv160 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e508980 2023-11-15T03:08:27.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.491+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e508070 con 0x557e9e508980 2023-11-15T03:08:27.503 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:08:27.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.495+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v121'161 uv161 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e508980 2023-11-15T03:08:27.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.495+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e508070 con 0x557e9e508980 2023-11-15T03:08:27.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.495+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v121'162 uv162 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e508980 2023-11-15T03:08:27.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.495+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e508070 con 0x557e9e508980 2023-11-15T03:08:27.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.499+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v121'163 uv163 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e508980 2023-11-15T03:08:27.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.499+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e508070 con 0x557e9e508980 2023-11-15T03:08:27.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.503+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v121'164 uv164 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e508980 2023-11-15T03:08:27.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.503+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x557e9e50b590 0x557e9e50e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.503+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50ed50 con 0x557e9e50b590 2023-11-15T03:08:27.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.503+0000 7f9e075a0700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x557e9e50b590 0x557e9e50e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.503+0000 7f9e075a0700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x557e9e50b590 0x557e9e50e860 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:27.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.507+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v121'113 uv112 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.507+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e50edf0 con 0x557e9e50b590 2023-11-15T03:08:27.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.511+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v121'114 uv114 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.511+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e50b590 2023-11-15T03:08:27.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.511+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v121'115 uv115 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.521 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:08:27.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.511+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e50b590 2023-11-15T03:08:27.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.515+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v121'116 uv116 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.515+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e50b590 2023-11-15T03:08:27.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.519+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v121'159 uv158 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.519+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e50b590 2023-11-15T03:08:27.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.523+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v121'160 uv160 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.523+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e50b590 2023-11-15T03:08:27.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.523+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v121'161 uv161 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.534 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:08:27.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.523+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e510200 con 0x557e9e50b590 2023-11-15T03:08:27.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.527+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v121'162 uv162 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.527+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e510200 con 0x557e9e50b590 2023-11-15T03:08:27.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.531+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v121'163 uv163 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.531+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e50b590 2023-11-15T03:08:27.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.531+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v121'164 uv164 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.531+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.535+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v121'294 uv287 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.535+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.539+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v121'295 uv295 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.539+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.543+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v121'296 uv296 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.553 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:08:27.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.543+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e510200 con 0x557e9e4fb2e0 2023-11-15T03:08:27.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.547+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v121'297 uv297 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.547+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e510200 con 0x557e9e4fb2e0 2023-11-15T03:08:27.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.551+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v121'298 uv298 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.551+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.555+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v121'299 uv299 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.555+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.555+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v121'300 uv293 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.555+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.559+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v121'301 uv301 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.559+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.563+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v121'302 uv302 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.573 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:08:27.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.563+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.567+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v121'303 uv303 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.567+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e4fb2e0 2023-11-15T03:08:27.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.571+0000 7f9e0fda1700 1 -- 172.21.15.121:0/1629733340 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v121'304 uv304 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e00007c80 con 0x557e9e4fb2e0 2023-11-15T03:08:27.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.571+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e500cb0 2023-11-15T03:08:27.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.575+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v121'132 uv131 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e500cb0 2023-11-15T03:08:27.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.575+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e500cb0 2023-11-15T03:08:27.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.575+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v121'133 uv133 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e500cb0 2023-11-15T03:08:27.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.575+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e500cb0 2023-11-15T03:08:27.587 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:08:27.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.579+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v121'134 uv134 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e500cb0 2023-11-15T03:08:27.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.579+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e500cb0 2023-11-15T03:08:27.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.583+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v121'135 uv135 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e500cb0 2023-11-15T03:08:27.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.583+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e500cb0 2023-11-15T03:08:27.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.583+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v121'136 uv136 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e500cb0 2023-11-15T03:08:27.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.583+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e50b590 2023-11-15T03:08:27.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.587+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v121'159 uv158 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.587+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e50b590 2023-11-15T03:08:27.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.591+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v121'160 uv160 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.591+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e50b590 2023-11-15T03:08:27.600 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:08:27.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.591+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v121'161 uv161 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.591+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e510200 con 0x557e9e50b590 2023-11-15T03:08:27.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.595+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v121'162 uv162 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.595+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e510200 con 0x557e9e50b590 2023-11-15T03:08:27.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.599+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v121'163 uv163 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.599+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e50b590 2023-11-15T03:08:27.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.599+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v121'164 uv164 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.599+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e50b590 2023-11-15T03:08:27.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.603+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v121'133 uv132 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.603+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e50b590 2023-11-15T03:08:27.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.607+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v121'134 uv134 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.607+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e50b590 2023-11-15T03:08:27.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.611+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v121'135 uv135 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.619 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:08:27.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.611+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e50b590 2023-11-15T03:08:27.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.611+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v121'136 uv136 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.611+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e50b590 2023-11-15T03:08:27.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.615+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v121'137 uv137 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.615+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e50b590 2023-11-15T03:08:27.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.619+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v121'131 uv130 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.619+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e50b590 2023-11-15T03:08:27.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.623+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v121'132 uv132 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.623+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e50b590 2023-11-15T03:08:27.635 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:08:27.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.627+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v121'133 uv133 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.627+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e50f1f0 con 0x557e9e50b590 2023-11-15T03:08:27.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.631+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v121'134 uv134 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.631+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e511f60 con 0x557e9e50b590 2023-11-15T03:08:27.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.631+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v121'135 uv135 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.631+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x557e9e50fbf0 0x557e9e50ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.631+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e50fbf0 2023-11-15T03:08:27.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.635+0000 7f9e105a2700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x557e9e50fbf0 0x557e9e50ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.635+0000 7f9e105a2700 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x557e9e50fbf0 0x557e9e50ffd0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:27.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.639+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v121'106 uv105 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e50fbf0 2023-11-15T03:08:27.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.639+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e50fbf0 2023-11-15T03:08:27.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.643+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v121'107 uv107 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e50fbf0 2023-11-15T03:08:27.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.643+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e50fbf0 2023-11-15T03:08:27.653 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:08:27.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.643+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v121'108 uv108 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e50fbf0 2023-11-15T03:08:27.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.643+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e50fbf0 2023-11-15T03:08:27.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.647+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v121'109 uv109 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e50fbf0 2023-11-15T03:08:27.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.647+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e50b590 2023-11-15T03:08:27.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.651+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v121'159 uv158 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.651+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e50b590 2023-11-15T03:08:27.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.655+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v121'160 uv160 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.655+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e50b590 2023-11-15T03:08:27.666 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:08:27.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.659+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v121'161 uv161 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.659+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e50b590 2023-11-15T03:08:27.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.659+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v121'162 uv162 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.659+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e50b590 2023-11-15T03:08:27.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.663+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v121'163 uv163 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.663+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e50b590 2023-11-15T03:08:27.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.667+0000 7f9e075a0700 1 -- 172.21.15.121:0/1629733340 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v121'164 uv164 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9df800f7d0 con 0x557e9e50b590 2023-11-15T03:08:27.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.667+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e503f90 2023-11-15T03:08:27.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.667+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v121'135 uv134 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.667+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e503f90 2023-11-15T03:08:27.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.671+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v121'136 uv136 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.671+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e503f90 2023-11-15T03:08:27.683 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:08:27.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.675+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v121'137 uv137 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.675+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e503f90 2023-11-15T03:08:27.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.679+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v121'138 uv138 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.679+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e510720 con 0x557e9e503f90 2023-11-15T03:08:27.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.679+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v121'139 uv139 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.679+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e510720 con 0x557e9e503f90 2023-11-15T03:08:27.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.683+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v121'317 uv309 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.683+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e503f90 2023-11-15T03:08:27.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.687+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v121'318 uv318 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.687+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e503f90 2023-11-15T03:08:27.698 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:08:27.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.691+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v121'319 uv319 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.691+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e503f90 2023-11-15T03:08:27.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.691+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v121'320 uv320 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.691+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e510af0 con 0x557e9e503f90 2023-11-15T03:08:27.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.695+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v121'321 uv321 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.695+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e510af0 con 0x557e9e503f90 2023-11-15T03:08:27.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.699+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v121'322 uv316 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.699+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e503f90 2023-11-15T03:08:27.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.703+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v121'323 uv323 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.703+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e503f90 2023-11-15T03:08:27.712 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:08:27.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.703+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v121'324 uv324 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.703+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5106a0 con 0x557e9e503f90 2023-11-15T03:08:27.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.707+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v121'325 uv325 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.707+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e5116f0 con 0x557e9e503f90 2023-11-15T03:08:27.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.707+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v121'326 uv326 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.707+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e512060 con 0x557e9e503f90 2023-11-15T03:08:27.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.711+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v121'327 uv327 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.711+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x557e9e512060 con 0x557e9e503f90 2023-11-15T03:08:27.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v121'328 uv328 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9e08015070 con 0x557e9e503f90 2023-11-15T03:08:27.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x557e9e500cb0 msgr2=0x557e9e5030a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x557e9e500cb0 0x557e9e5030a0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x557e9e4c03e0 msgr2=0x557e9e4d8440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x557e9e4c03e0 0x557e9e4d8440 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x557e9e508980 msgr2=0x557e9e50be50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x557e9e508980 0x557e9e50be50 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x557e9e4fb2e0 msgr2=0x557e9e4fe3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x557e9e4fb2e0 0x557e9e4fe3e0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x557e9e50fbf0 msgr2=0x557e9e50ffd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x557e9e50fbf0 0x557e9e50ffd0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x557e9e503f90 msgr2=0x557e9e506380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x557e9e503f90 0x557e9e506380 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x557e9e50b590 msgr2=0x557e9e50e860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x557e9e50b590 0x557e9e50e860 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 reap_dead start 2023-11-15T03:08:27.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 reap_dead start 2023-11-15T03:08:27.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 reap_dead start 2023-11-15T03:08:27.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e105a2700 1 -- 172.21.15.121:0/1629733340 reap_dead start 2023-11-15T03:08:27.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9df0058ff0 msgr2=0x7f9df005b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9df0058ff0 0x7f9df005b4b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7f9e00009ae0 tx=0x7f9e00005bb0).stop 2023-11-15T03:08:27.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e9e4c32e0 msgr2=0x557e9e4d7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e9e4c32e0 0x557e9e4d7550 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 rx=0x7f9e08005ef0 tx=0x7f9e0801b040).stop 2023-11-15T03:08:27.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 shutdown_connections 2023-11-15T03:08:27.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9df0058ff0 0x7f9df005b4b0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 --2- 172.21.15.121:0/1629733340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e9e4c32e0 0x557e9e4d7550 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 >> 172.21.15.121:0/1629733340 conn(0x557e9e4057d0 msgr2=0x557e9e4b60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:27.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 shutdown_connections 2023-11-15T03:08:27.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.715+0000 7f9e1b736d00 1 -- 172.21.15.121:0/1629733340 wait complete. 2023-11-15T03:08:27.735 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate mds_snaptable 0 2023-11-15T03:08:27.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.827+0000 7fee8e733d00 1 Processor -- start 2023-11-15T03:08:27.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.827+0000 7fee8e733d00 1 -- start start 2023-11-15T03:08:27.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8e733d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c041fd920 0x562c042c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8e733d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562c04242f50 0x562c042d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8e733d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562c042d4130 0x562c041f9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8e733d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c041a7cc0 con 0x562c042d4130 2023-11-15T03:08:27.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8e733d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c04172220 con 0x562c041fd920 2023-11-15T03:08:27.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8e733d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c042433d0 con 0x562c04242f50 2023-11-15T03:08:27.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8363d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562c042d4130 0x562c041f9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8363d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562c042d4130 0x562c041f9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53914/0 (socket says 172.21.15.121:53914) 2023-11-15T03:08:27.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8363d700 1 -- 172.21.15.121:0/1880942442 learned_addr learned my addr 172.21.15.121:0/1880942442 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:27.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee82e3c700 1 --2- 172.21.15.121:0/1880942442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c041fd920 0x562c042c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee7a63b700 1 --2- 172.21.15.121:0/1880942442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562c04242f50 0x562c042d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8363d700 1 -- 172.21.15.121:0/1880942442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562c04242f50 msgr2=0x562c042d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:27.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8363d700 1 --2- 172.21.15.121:0/1880942442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562c04242f50 0x562c042d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8363d700 1 -- 172.21.15.121:0/1880942442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c041fd920 msgr2=0x562c042c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:27.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8363d700 1 --2- 172.21.15.121:0/1880942442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c041fd920 0x562c042c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8363d700 1 -- 172.21.15.121:0/1880942442 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c005cf0 con 0x562c042d4130 2023-11-15T03:08:27.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8363d700 1 --2- 172.21.15.121:0/1880942442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562c042d4130 0x562c041f9870 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 rx=0x562c041e8c20 tx=0x7fee7c014000).ready entity=mon.0 client_cookie=a7c0704477163a90 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:27.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee81e3a700 1 -- 172.21.15.121:0/1880942442 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c0075b0 con 0x562c042d4130 2023-11-15T03:08:27.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee81e3a700 1 -- 172.21.15.121:0/1880942442 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7c007a20 con 0x562c042d4130 2023-11-15T03:08:27.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee81e3a700 1 -- 172.21.15.121:0/1880942442 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c007b90 con 0x562c042d4130 2023-11-15T03:08:27.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8e733d00 1 -- 172.21.15.121:0/1880942442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562c042d4130 msgr2=0x562c041f9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8e733d00 1 --2- 172.21.15.121:0/1880942442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562c042d4130 0x562c041f9870 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 rx=0x562c041e8c20 tx=0x7fee7c014000).stop 2023-11-15T03:08:27.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8e733d00 1 -- 172.21.15.121:0/1880942442 shutdown_connections 2023-11-15T03:08:27.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8e733d00 1 --2- 172.21.15.121:0/1880942442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562c042d4130 0x562c041f9870 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8e733d00 1 --2- 172.21.15.121:0/1880942442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562c04242f50 0x562c042d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8e733d00 1 --2- 172.21.15.121:0/1880942442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c041fd920 0x562c042c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8e733d00 1 -- 172.21.15.121:0/1880942442 >> 172.21.15.121:0/1880942442 conn(0x562c042438f0 msgr2=0x562c042d1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:27.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8e733d00 1 -- 172.21.15.121:0/1880942442 shutdown_connections 2023-11-15T03:08:27.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.831+0000 7fee8e733d00 1 -- 172.21.15.121:0/1880942442 wait complete. 2023-11-15T03:08:27.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee8e733d00 1 Processor -- start 2023-11-15T03:08:27.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee8e733d00 1 -- start start 2023-11-15T03:08:27.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee8e733d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c041fd920 0x562c041f9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee8e733d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562c04242f50 0x562c042c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee8e733d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562c042d4130 0x562c042cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee8e733d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c041fa130 con 0x562c042d4130 2023-11-15T03:08:27.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee8e733d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c041faf80 con 0x562c041fd920 2023-11-15T03:08:27.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee8e733d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c041fa970 con 0x562c04242f50 2023-11-15T03:08:27.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee8363d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562c042d4130 0x562c042cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee8363d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562c042d4130 0x562c042cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53922/0 (socket says 172.21.15.121:53922) 2023-11-15T03:08:27.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee8363d700 1 -- 172.21.15.121:0/3497145648 learned_addr learned my addr 172.21.15.121:0/3497145648 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:27.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee82e3c700 1 --2- 172.21.15.121:0/3497145648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c041fd920 0x562c041f9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee7a63b700 1 --2- 172.21.15.121:0/3497145648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562c04242f50 0x562c042c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee8363d700 1 -- 172.21.15.121:0/3497145648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562c04242f50 msgr2=0x562c042c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee8363d700 1 --2- 172.21.15.121:0/3497145648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562c04242f50 0x562c042c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee8363d700 1 -- 172.21.15.121:0/3497145648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c041fd920 msgr2=0x562c041f9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee8363d700 1 --2- 172.21.15.121:0/3497145648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c041fd920 0x562c041f9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee8363d700 1 -- 172.21.15.121:0/3497145648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c0419b770 con 0x562c042d4130 2023-11-15T03:08:27.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee8363d700 1 --2- 172.21.15.121:0/3497145648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562c042d4130 0x562c042cce40 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 rx=0x7fee7c01dca0 tx=0x7fee7c01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:27.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee7bfff700 1 -- 172.21.15.121:0/3497145648 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c007b90 con 0x562c042d4130 2023-11-15T03:08:27.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee7bfff700 1 -- 172.21.15.121:0/3497145648 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7c005330 con 0x562c042d4130 2023-11-15T03:08:27.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.835+0000 7fee7bfff700 1 -- 172.21.15.121:0/3497145648 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c026670 con 0x562c042d4130 2023-11-15T03:08:27.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.839+0000 7fee8e733d00 1 -- 172.21.15.121:0/3497145648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562c0416aa70 con 0x562c042d4130 2023-11-15T03:08:27.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.839+0000 7fee8e733d00 1 -- 172.21.15.121:0/3497145648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c04176010 con 0x562c042d4130 2023-11-15T03:08:27.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.839+0000 7fee7bfff700 1 -- 172.21.15.121:0/3497145648 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fee7c024070 con 0x562c042d4130 2023-11-15T03:08:27.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.839+0000 7fee7bfff700 1 --2- 172.21.15.121:0/3497145648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fee64059040 0x7fee6405b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.843+0000 7fee7bfff700 1 -- 172.21.15.121:0/3497145648 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8588+0+0 (secure 0 0 0) 0x7fee7c075cf0 con 0x562c042d4130 2023-11-15T03:08:27.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.843+0000 7fee82e3c700 1 --2- 172.21.15.121:0/3497145648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fee64059040 0x7fee6405b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.843+0000 7fee8e733d00 1 --2- 172.21.15.121:0/3497145648 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x562c042c8ed0 0x562c042407a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:27.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.843+0000 7fee8e733d00 1 -- 172.21.15.121:0/3497145648 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:1 13.2d 13:b50e409b:::mds_snaptable:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected e121) v8 -- 0x562c041fb410 con 0x562c042c8ed0 2023-11-15T03:08:27.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.843+0000 7fee7a63b700 1 --2- 172.21.15.121:0/3497145648 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x562c042c8ed0 0x562c042407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:27.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.843+0000 7fee82e3c700 1 --2- 172.21.15.121:0/3497145648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fee64059040 0x7fee6405b500 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7fee74001660 tx=0x7fee74000d90).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:27.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.843+0000 7fee7a63b700 1 --2- 172.21.15.121:0/3497145648 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x562c042c8ed0 0x562c042407a0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:27.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.847+0000 7fee7a63b700 1 -- 172.21.15.121:0/3497145648 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(1 mds_snaptable [truncate 0] v121'137 uv137 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fee6c0016c0 con 0x562c042c8ed0 2023-11-15T03:08:27.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.847+0000 7fee8e733d00 1 -- 172.21.15.121:0/3497145648 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x562c042c8ed0 msgr2=0x562c042407a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.847+0000 7fee8e733d00 1 --2- 172.21.15.121:0/3497145648 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x562c042c8ed0 0x562c042407a0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:27.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.847+0000 7fee8e733d00 1 -- 172.21.15.121:0/3497145648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fee64059040 msgr2=0x7fee6405b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.847+0000 7fee8e733d00 1 --2- 172.21.15.121:0/3497145648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fee64059040 0x7fee6405b500 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7fee74001660 tx=0x7fee74000d90).stop 2023-11-15T03:08:27.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.847+0000 7fee8e733d00 1 -- 172.21.15.121:0/3497145648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562c042d4130 msgr2=0x562c042cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:27.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.847+0000 7fee8e733d00 1 --2- 172.21.15.121:0/3497145648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562c042d4130 0x562c042cce40 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 rx=0x7fee7c01dca0 tx=0x7fee7c01a000).stop 2023-11-15T03:08:27.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.847+0000 7fee8363d700 1 -- 172.21.15.121:0/3497145648 reap_dead start 2023-11-15T03:08:27.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.847+0000 7fee8e733d00 1 -- 172.21.15.121:0/3497145648 shutdown_connections 2023-11-15T03:08:27.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.847+0000 7fee8e733d00 1 -- 172.21.15.121:0/3497145648 >> 172.21.15.121:0/3497145648 conn(0x562c042438f0 msgr2=0x562c042d1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:27.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.847+0000 7fee8e733d00 1 -- 172.21.15.121:0/3497145648 shutdown_connections 2023-11-15T03:08:27.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:27.847+0000 7fee8e733d00 1 -- 172.21.15.121:0/3497145648 wait complete. 2023-11-15T03:08:27.862 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:08:28.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.015+0000 7fd171cf1700 1 Processor -- start 2023-11-15T03:08:28.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.015+0000 7fd171cf1700 1 -- start start 2023-11-15T03:08:28.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.015+0000 7fd171cf1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd16c0a8b80 0x7fd16c0a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:28.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.015+0000 7fd171cf1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd16c065850 0x7fd16c067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:28.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.015+0000 7fd171cf1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd16c152150 con 0x7fd16c067f50 2023-11-15T03:08:28.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.015+0000 7fd171cf1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd16c148f70 con 0x7fd16c0a8b80 2023-11-15T03:08:28.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.015+0000 7fd171cf1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c0683d0 con 0x7fd16c065850 2023-11-15T03:08:28.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd16b7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd16c0a8b80 0x7fd16c0a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:28.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd16b7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd16c0a8b80 0x7fd16c0a8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53926/0 (socket says 172.21.15.121:53926) 2023-11-15T03:08:28.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd16b7fe700 1 -- 172.21.15.121:0/1332882712 learned_addr learned my addr 172.21.15.121:0/1332882712 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:28.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd16bfff700 1 --1- 172.21.15.121:0/1332882712 >> v1:172.21.15.121:6789/0 conn(0x7fd16c067f50 0x7fd16c0bbe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50292/0 (socket says 172.21.15.121:50292) 2023-11-15T03:08:28.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd16affd700 1 --2- 172.21.15.121:0/1332882712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd16c065850 0x7fd16c067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:28.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd16b7fe700 1 -- 172.21.15.121:0/1332882712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd16c065850 msgr2=0x7fd16c067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:28.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd16b7fe700 1 --2- 172.21.15.121:0/1332882712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd16c065850 0x7fd16c067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:28.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd16b7fe700 1 -- 172.21.15.121:0/1332882712 >> v1:172.21.15.121:6789/0 conn(0x7fd16c067f50 legacy=0x7fd16c0bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:28.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd16b7fe700 1 -- 172.21.15.121:0/1332882712 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16c1523a0 con 0x7fd16c0a8b80 2023-11-15T03:08:28.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd16b7fe700 1 --2- 172.21.15.121:0/1332882712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd16c0a8b80 0x7fd16c0a8f60 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 rx=0x7fd15c006fd0 tx=0x7fd15c0090f0).ready entity=mon.0 client_cookie=976b28cf9ae0f1e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:28.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd16a7fc700 1 -- 172.21.15.121:0/1332882712 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c00b580 con 0x7fd16c0a8b80 2023-11-15T03:08:28.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd16a7fc700 1 -- 172.21.15.121:0/1332882712 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15c00b6e0 con 0x7fd16c0a8b80 2023-11-15T03:08:28.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd16a7fc700 1 -- 172.21.15.121:0/1332882712 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c00ba60 con 0x7fd16c0a8b80 2023-11-15T03:08:28.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd171cf1700 1 -- 172.21.15.121:0/1332882712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd16c0a8b80 msgr2=0x7fd16c0a8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:28.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd171cf1700 1 --2- 172.21.15.121:0/1332882712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd16c0a8b80 0x7fd16c0a8f60 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 rx=0x7fd15c006fd0 tx=0x7fd15c0090f0).stop 2023-11-15T03:08:28.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd171cf1700 1 -- 172.21.15.121:0/1332882712 shutdown_connections 2023-11-15T03:08:28.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd171cf1700 1 --2- 172.21.15.121:0/1332882712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd16c065850 0x7fd16c067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:28.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd171cf1700 1 --2- 172.21.15.121:0/1332882712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd16c0a8b80 0x7fd16c0a8f60 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:28.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd171cf1700 1 -- 172.21.15.121:0/1332882712 >> 172.21.15.121:0/1332882712 conn(0x7fd16c0591c0 msgr2=0x7fd16c0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:28.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd171cf1700 1 -- 172.21.15.121:0/1332882712 shutdown_connections 2023-11-15T03:08:28.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.019+0000 7fd171cf1700 1 -- 172.21.15.121:0/1332882712 wait complete. 2023-11-15T03:08:28.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.023+0000 7fd171cf1700 1 Processor -- start 2023-11-15T03:08:28.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.023+0000 7fd171cf1700 1 -- start start 2023-11-15T03:08:28.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.023+0000 7fd171cf1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd16c065850 0x7fd16c153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:28.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.023+0000 7fd171cf1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd16c067f50 0x7fd16c155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:28.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.023+0000 7fd171cf1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd16c0a8b80 0x7fd16c0bf990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:28.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.023+0000 7fd171cf1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c153640 con 0x7fd16c065850 2023-11-15T03:08:28.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.023+0000 7fd171cf1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c0c1cd0 con 0x7fd16c067f50 2023-11-15T03:08:28.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.023+0000 7fd171cf1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c0c1770 con 0x7fd16c0a8b80 2023-11-15T03:08:28.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.023+0000 7fd16b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd16c065850 0x7fd16c153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:28.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.023+0000 7fd16b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd16c065850 0x7fd16c153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53940/0 (socket says 172.21.15.121:53940) 2023-11-15T03:08:28.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.023+0000 7fd16b7fe700 1 -- 172.21.15.121:0/3160612398 learned_addr learned my addr 172.21.15.121:0/3160612398 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:28.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.027+0000 7fd16affd700 1 --2- 172.21.15.121:0/3160612398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd16c067f50 0x7fd16c155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:28.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.027+0000 7fd16bfff700 1 --2- 172.21.15.121:0/3160612398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd16c0a8b80 0x7fd16c0bf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:28.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.027+0000 7fd16b7fe700 1 -- 172.21.15.121:0/3160612398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd16c0a8b80 msgr2=0x7fd16c0bf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:28.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.027+0000 7fd16b7fe700 1 --2- 172.21.15.121:0/3160612398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd16c0a8b80 0x7fd16c0bf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:28.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.027+0000 7fd16b7fe700 1 -- 172.21.15.121:0/3160612398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd16c067f50 msgr2=0x7fd16c155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:28.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.027+0000 7fd16b7fe700 1 --2- 172.21.15.121:0/3160612398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd16c067f50 0x7fd16c155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:28.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.027+0000 7fd16b7fe700 1 -- 172.21.15.121:0/3160612398 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15c009370 con 0x7fd16c065850 2023-11-15T03:08:28.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.027+0000 7fd16b7fe700 1 --2- 172.21.15.121:0/3160612398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd16c065850 0x7fd16c153150 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 rx=0x7fd15c009fd0 tx=0x7fd15c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:28.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.027+0000 7fd1537fe700 1 -- 172.21.15.121:0/3160612398 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c00b850 con 0x7fd16c065850 2023-11-15T03:08:28.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.027+0000 7fd1537fe700 1 -- 172.21.15.121:0/3160612398 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15c00b9b0 con 0x7fd16c065850 2023-11-15T03:08:28.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.027+0000 7fd171cf1700 1 -- 172.21.15.121:0/3160612398 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16c1563e0 con 0x7fd16c065850 2023-11-15T03:08:28.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.027+0000 7fd1537fe700 1 -- 172.21.15.121:0/3160612398 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c01b660 con 0x7fd16c065850 2023-11-15T03:08:28.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.027+0000 7fd171cf1700 1 -- 172.21.15.121:0/3160612398 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd16c0beeb0 con 0x7fd16c065850 2023-11-15T03:08:28.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.027+0000 7fd171cf1700 1 -- 172.21.15.121:0/3160612398 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd16c047060 con 0x7fd16c065850 2023-11-15T03:08:28.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.031+0000 7fd1537fe700 1 -- 172.21.15.121:0/3160612398 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd15c01ba50 con 0x7fd16c065850 2023-11-15T03:08:28.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.031+0000 7fd1537fe700 1 --2- 172.21.15.121:0/3160612398 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd15805a540 0x7fd15805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:28.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.031+0000 7fd1537fe700 1 -- 172.21.15.121:0/3160612398 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8588+0+0 (secure 0 0 0) 0x7fd15c076b20 con 0x7fd16c065850 2023-11-15T03:08:28.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.031+0000 7fd16affd700 1 --2- 172.21.15.121:0/3160612398 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd15805a540 0x7fd15805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:28.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.031+0000 7fd16affd700 1 --2- 172.21.15.121:0/3160612398 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd15805a540 0x7fd15805ca00 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7fd16c0c5710 tx=0x7fd160008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:28.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.035+0000 7fd1537fe700 1 -- 172.21.15.121:0/3160612398 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd15c049920 con 0x7fd16c065850 2023-11-15T03:08:28.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:28.235+0000 7fd171cf1700 1 -- 172.21.15.121:0/3160612398 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fd16c063b60 con 0x7fd16c065850 2023-11-15T03:08:29.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.195+0000 7fd1537fe700 1 -- 172.21.15.121:0/3160612398 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v251) v1 ==== 162+0+0 (secure 0 0 0) 0x7fd15c029030 con 0x7fd16c065850 2023-11-15T03:08:29.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.199+0000 7fd171cf1700 1 -- 172.21.15.121:0/3160612398 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd15805a540 msgr2=0x7fd15805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:29.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.199+0000 7fd171cf1700 1 --2- 172.21.15.121:0/3160612398 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd15805a540 0x7fd15805ca00 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7fd16c0c5710 tx=0x7fd160008040).stop 2023-11-15T03:08:29.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.199+0000 7fd171cf1700 1 -- 172.21.15.121:0/3160612398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd16c065850 msgr2=0x7fd16c153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:29.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.199+0000 7fd171cf1700 1 --2- 172.21.15.121:0/3160612398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd16c065850 0x7fd16c153150 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 rx=0x7fd15c009fd0 tx=0x7fd15c017000).stop 2023-11-15T03:08:29.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.199+0000 7fd171cf1700 1 -- 172.21.15.121:0/3160612398 shutdown_connections 2023-11-15T03:08:29.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.199+0000 7fd171cf1700 1 --2- 172.21.15.121:0/3160612398 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd15805a540 0x7fd15805ca00 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:29.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.199+0000 7fd171cf1700 1 --2- 172.21.15.121:0/3160612398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd16c0a8b80 0x7fd16c0bf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:29.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.199+0000 7fd171cf1700 1 --2- 172.21.15.121:0/3160612398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd16c067f50 0x7fd16c155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:29.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.199+0000 7fd171cf1700 1 --2- 172.21.15.121:0/3160612398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd16c065850 0x7fd16c153150 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:29.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.199+0000 7fd171cf1700 1 -- 172.21.15.121:0/3160612398 >> 172.21.15.121:0/3160612398 conn(0x7fd16c0591c0 msgr2=0x7fd16c0c4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:29.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.199+0000 7fd171cf1700 1 -- 172.21.15.121:0/3160612398 shutdown_connections 2023-11-15T03:08:29.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.199+0000 7fd171cf1700 1 -- 172.21.15.121:0/3160612398 wait complete. 2023-11-15T03:08:29.212 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:08:29.218 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:08:29.211+0000 7f952fd72700 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': End of buffer 2023-11-15T03:08:29.233 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:29.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.383+0000 7f79da879700 1 Processor -- start 2023-11-15T03:08:29.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.383+0000 7f79da879700 1 -- start start 2023-11-15T03:08:29.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.387+0000 7f79da879700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79d4151fd0 0x7f79d40657f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:29.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.387+0000 7f79da879700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f79d40c30e0 0x7f79d40c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:29.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.387+0000 7f79da879700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d40c3a60 con 0x7f79d414bfb0 2023-11-15T03:08:29.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.387+0000 7f79da879700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d40ac7c0 con 0x7f79d40c30e0 2023-11-15T03:08:29.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.387+0000 7f79da879700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d41524b0 con 0x7f79d4151fd0 2023-11-15T03:08:29.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79d9877700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f79d414bfb0 0x7f79d40ac0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:39638/0 (socket says 172.21.15.121:39638) 2023-11-15T03:08:29.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79d9877700 1 -- 172.21.15.121:0/4217028830 learned_addr learned my addr 172.21.15.121:0/4217028830 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:29.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79da078700 1 --2- 172.21.15.121:0/4217028830 >> v2:172.21.15.121:3300/0 conn(0x7f79d40c30e0 0x7f79d40c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:29.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79d9076700 1 --2- 172.21.15.121:0/4217028830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79d4151fd0 0x7f79d40657f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:29.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79d8875700 1 -- 172.21.15.121:0/4217028830 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228396475 0 0) 0x7f79d40c3a60 con 0x7f79d414bfb0 2023-11-15T03:08:29.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79d8875700 1 -- 172.21.15.121:0/4217028830 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79c00033f0 con 0x7f79d414bfb0 2023-11-15T03:08:29.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79da078700 1 -- 172.21.15.121:0/4217028830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79d4151fd0 msgr2=0x7f79d40657f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:29.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79da078700 1 --2- 172.21.15.121:0/4217028830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79d4151fd0 0x7f79d40657f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:29.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79da078700 1 -- 172.21.15.121:0/4217028830 >> v1:172.21.15.190:6789/0 conn(0x7f79d414bfb0 legacy=0x7f79d40ac0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:29.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79da078700 1 -- 172.21.15.121:0/4217028830 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d4152790 con 0x7f79d40c30e0 2023-11-15T03:08:29.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79da078700 1 --2- 172.21.15.121:0/4217028830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d40c30e0 0x7f79d40c3570 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 rx=0x7f79cc009e60 tx=0x7f79cc009780).ready entity=mon.0 client_cookie=9df90e591c585ff5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:29.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79d8875700 1 -- 172.21.15.121:0/4217028830 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc00e070 con 0x7f79d40c30e0 2023-11-15T03:08:29.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79d8875700 1 -- 172.21.15.121:0/4217028830 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79cc00bab0 con 0x7f79d40c30e0 2023-11-15T03:08:29.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79d8875700 1 -- 172.21.15.121:0/4217028830 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc00be30 con 0x7f79d40c30e0 2023-11-15T03:08:29.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79da879700 1 -- 172.21.15.121:0/4217028830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d40c30e0 msgr2=0x7f79d40c3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:29.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79da879700 1 --2- 172.21.15.121:0/4217028830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d40c30e0 0x7f79d40c3570 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 rx=0x7f79cc009e60 tx=0x7f79cc009780).stop 2023-11-15T03:08:29.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79da879700 1 -- 172.21.15.121:0/4217028830 shutdown_connections 2023-11-15T03:08:29.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79da879700 1 --2- 172.21.15.121:0/4217028830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d40c30e0 0x7f79d40c3570 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:29.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79da879700 1 --2- 172.21.15.121:0/4217028830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79d4151fd0 0x7f79d40657f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:29.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.391+0000 7f79da879700 1 -- 172.21.15.121:0/4217028830 >> 172.21.15.121:0/4217028830 conn(0x7f79d4062190 msgr2=0x7f79d4062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:29.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.395+0000 7f79da879700 1 -- 172.21.15.121:0/4217028830 shutdown_connections 2023-11-15T03:08:29.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.395+0000 7f79da879700 1 -- 172.21.15.121:0/4217028830 wait complete. 2023-11-15T03:08:29.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.395+0000 7f79da879700 1 Processor -- start 2023-11-15T03:08:29.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.395+0000 7f79da879700 1 -- start start 2023-11-15T03:08:29.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.395+0000 7f79da879700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79d40c30e0 0x7f79d4138f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:29.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.395+0000 7f79da879700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79d414bfb0 0x7f79d414e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:29.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.395+0000 7f79da879700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d4151fd0 0x7f79d4157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:29.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.395+0000 7f79da879700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40c5100 con 0x7f79d4151fd0 2023-11-15T03:08:29.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.395+0000 7f79da879700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d414e0a0 con 0x7f79d414bfb0 2023-11-15T03:08:29.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.395+0000 7f79da879700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d414d490 con 0x7f79d40c30e0 2023-11-15T03:08:29.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.395+0000 7f79da078700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d4151fd0 0x7f79d4157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:29.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.399+0000 7f79d9076700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79d414bfb0 0x7f79d414e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:29.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.399+0000 7f79da078700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d4151fd0 0x7f79d4157660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53946/0 (socket says 172.21.15.121:53946) 2023-11-15T03:08:29.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.399+0000 7f79da078700 1 -- 172.21.15.121:0/3191666352 learned_addr learned my addr 172.21.15.121:0/3191666352 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:29.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.399+0000 7f79d9877700 1 --2- 172.21.15.121:0/3191666352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79d40c30e0 0x7f79d4138f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:29.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.399+0000 7f79da078700 1 -- 172.21.15.121:0/3191666352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79d40c30e0 msgr2=0x7f79d4138f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:29.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.399+0000 7f79da078700 1 --2- 172.21.15.121:0/3191666352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79d40c30e0 0x7f79d4138f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:29.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.399+0000 7f79da078700 1 -- 172.21.15.121:0/3191666352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79d414bfb0 msgr2=0x7f79d414e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:29.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.399+0000 7f79da078700 1 --2- 172.21.15.121:0/3191666352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79d414bfb0 0x7f79d414e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:29.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.399+0000 7f79da078700 1 -- 172.21.15.121:0/3191666352 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc009a20 con 0x7f79d4151fd0 2023-11-15T03:08:29.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.399+0000 7f79da078700 1 --2- 172.21.15.121:0/3191666352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d4151fd0 0x7f79d4157660 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 rx=0x7f79cc00b610 tx=0x7f79cc009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:29.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.399+0000 7f79c9ffb700 1 -- 172.21.15.121:0/3191666352 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc013e70 con 0x7f79d4151fd0 2023-11-15T03:08:29.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.399+0000 7f79c9ffb700 1 -- 172.21.15.121:0/3191666352 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79cc01d630 con 0x7f79d4151fd0 2023-11-15T03:08:29.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.399+0000 7f79c9ffb700 1 -- 172.21.15.121:0/3191666352 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc01d9d0 con 0x7f79d4151fd0 2023-11-15T03:08:29.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.399+0000 7f79da879700 1 -- 172.21.15.121:0/3191666352 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d414d770 con 0x7f79d4151fd0 2023-11-15T03:08:29.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.399+0000 7f79da879700 1 -- 172.21.15.121:0/3191666352 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d414dd30 con 0x7f79d4151fd0 2023-11-15T03:08:29.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.399+0000 7f79da879700 1 -- 172.21.15.121:0/3191666352 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d414e210 con 0x7f79d4151fd0 2023-11-15T03:08:29.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.403+0000 7f79c9ffb700 1 -- 172.21.15.121:0/3191666352 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f79cc003040 con 0x7f79d4151fd0 2023-11-15T03:08:29.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.403+0000 7f79c9ffb700 1 --2- 172.21.15.121:0/3191666352 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f79b805a510 0x7f79b805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:29.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.403+0000 7f79c9ffb700 1 -- 172.21.15.121:0/3191666352 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8674+0+0 (secure 0 0 0) 0x7f79cc076e60 con 0x7f79d4151fd0 2023-11-15T03:08:29.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.403+0000 7f79d9877700 1 --2- 172.21.15.121:0/3191666352 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f79b805a510 0x7f79b805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:29.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.403+0000 7f79d9877700 1 --2- 172.21.15.121:0/3191666352 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f79b805a510 0x7f79b805c9d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f79d0005750 tx=0x7f79d0009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:29.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.411+0000 7f79c9ffb700 1 -- 172.21.15.121:0/3191666352 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f79cc049b80 con 0x7f79d4151fd0 2023-11-15T03:08:29.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.611+0000 7f79da879700 1 -- 172.21.15.121:0/3191666352 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79d4152a70 con 0x7f79d4151fd0 2023-11-15T03:08:29.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.611+0000 7f79c9ffb700 1 -- 172.21.15.121:0/3191666352 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 252 v252) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f79cc04d1a0 con 0x7f79d4151fd0 2023-11-15T03:08:29.619 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:29.620 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":252,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":252,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:08:29.201510+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9531":{"gid":9531,"name":"a","rank":0,"incarnation":252,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/1244489663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1244489663},{"type":"v1","addr":"172.21.15.121:6836","nonce":1244489663}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:29.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.615+0000 7f79da879700 1 -- 172.21.15.121:0/3191666352 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f79b805a510 msgr2=0x7f79b805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:29.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.615+0000 7f79da879700 1 --2- 172.21.15.121:0/3191666352 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f79b805a510 0x7f79b805c9d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f79d0005750 tx=0x7f79d0009040).stop 2023-11-15T03:08:29.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.615+0000 7f79da879700 1 -- 172.21.15.121:0/3191666352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d4151fd0 msgr2=0x7f79d4157660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:29.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.615+0000 7f79da879700 1 --2- 172.21.15.121:0/3191666352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d4151fd0 0x7f79d4157660 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 rx=0x7f79cc00b610 tx=0x7f79cc009780).stop 2023-11-15T03:08:29.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.615+0000 7f79da879700 1 -- 172.21.15.121:0/3191666352 shutdown_connections 2023-11-15T03:08:29.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.615+0000 7f79da879700 1 --2- 172.21.15.121:0/3191666352 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f79b805a510 0x7f79b805c9d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:29.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.615+0000 7f79da879700 1 --2- 172.21.15.121:0/3191666352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d4151fd0 0x7f79d4157660 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:29.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.615+0000 7f79da879700 1 --2- 172.21.15.121:0/3191666352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79d414bfb0 0x7f79d414e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:29.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.615+0000 7f79da879700 1 --2- 172.21.15.121:0/3191666352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79d40c30e0 0x7f79d4138f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:29.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.615+0000 7f79da879700 1 -- 172.21.15.121:0/3191666352 >> 172.21.15.121:0/3191666352 conn(0x7f79d4062190 msgr2=0x7f79d4066330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:29.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.615+0000 7f79da879700 1 -- 172.21.15.121:0/3191666352 shutdown_connections 2023-11-15T03:08:29.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:29.615+0000 7f79da879700 1 -- 172.21.15.121:0/3191666352 wait complete. 2023-11-15T03:08:29.626 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 252 2023-11-15T03:08:29.643 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:08:30.213 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:08:30.203+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:08:30.213 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:08:30.203+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:08:30.222 INFO:tasks.ceph.mds.a.smithi121.stderr: -26> 2023-11-15T03:08:29.211+0000 7f952fd72700 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': End of buffer 2023-11-15T03:08:30.263 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:08:34.644 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:34.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.803+0000 7fc18581b700 1 Processor -- start 2023-11-15T03:08:34.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.803+0000 7fc18581b700 1 -- start start 2023-11-15T03:08:34.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc18581b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc180154ca0 0x7fc180155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:34.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc18581b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc1801562f0 0x7fc1800a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:34.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc18581b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1800c4c30 con 0x7fc1800a8f40 2023-11-15T03:08:34.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc18581b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1800a8280 con 0x7fc180154ca0 2023-11-15T03:08:34.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc18581b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1800a6d40 con 0x7fc1801562f0 2023-11-15T03:08:34.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc17effd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc180154ca0 0x7fc180155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:34.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc17effd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc180154ca0 0x7fc180155080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56728/0 (socket says 172.21.15.121:56728) 2023-11-15T03:08:34.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc17effd700 1 -- 172.21.15.121:0/3588530170 learned_addr learned my addr 172.21.15.121:0/3588530170 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:34.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc17e7fc700 1 --2- 172.21.15.121:0/3588530170 >> v2:172.21.15.190:3300/0 conn(0x7fc1801562f0 0x7fc1800a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:34.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc17effd700 1 -- 172.21.15.121:0/3588530170 >> v2:172.21.15.190:3300/0 conn(0x7fc1801562f0 msgr2=0x7fc1800a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:34.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc17effd700 1 --2- 172.21.15.121:0/3588530170 >> v2:172.21.15.190:3300/0 conn(0x7fc1801562f0 0x7fc1800a8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:34.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc17effd700 1 -- 172.21.15.121:0/3588530170 >> v1:172.21.15.190:6789/0 conn(0x7fc1800a8f40 legacy=0x7fc1800a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:34.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc17effd700 1 -- 172.21.15.121:0/3588530170 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1800a6f50 con 0x7fc180154ca0 2023-11-15T03:08:34.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc17effd700 1 --2- 172.21.15.121:0/3588530170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc180154ca0 0x7fc180155080 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 rx=0x7fc1800a6e80 tx=0x7fc1700090f0).ready entity=mon.0 client_cookie=89abfde84ea775d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:34.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc17dffb700 1 -- 172.21.15.121:0/3588530170 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17000b520 con 0x7fc180154ca0 2023-11-15T03:08:34.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc17dffb700 1 -- 172.21.15.121:0/3588530170 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17000b680 con 0x7fc180154ca0 2023-11-15T03:08:34.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc17dffb700 1 -- 172.21.15.121:0/3588530170 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17000ba00 con 0x7fc180154ca0 2023-11-15T03:08:34.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc18581b700 1 -- 172.21.15.121:0/3588530170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc180154ca0 msgr2=0x7fc180155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:34.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc18581b700 1 --2- 172.21.15.121:0/3588530170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc180154ca0 0x7fc180155080 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 rx=0x7fc1800a6e80 tx=0x7fc1700090f0).stop 2023-11-15T03:08:34.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc18581b700 1 -- 172.21.15.121:0/3588530170 shutdown_connections 2023-11-15T03:08:34.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc18581b700 1 --2- 172.21.15.121:0/3588530170 >> v2:172.21.15.190:3300/0 conn(0x7fc1801562f0 0x7fc1800a8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:34.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc18581b700 1 --2- 172.21.15.121:0/3588530170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc180154ca0 0x7fc180155080 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:34.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc18581b700 1 -- 172.21.15.121:0/3588530170 >> 172.21.15.121:0/3588530170 conn(0x7fc180062190 msgr2=0x7fc1800625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:34.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc18581b700 1 -- 172.21.15.121:0/3588530170 shutdown_connections 2023-11-15T03:08:34.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.807+0000 7fc18581b700 1 -- 172.21.15.121:0/3588530170 wait complete. 2023-11-15T03:08:34.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.811+0000 7fc18581b700 1 Processor -- start 2023-11-15T03:08:34.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc18581b700 1 -- start start 2023-11-15T03:08:34.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc18581b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1800a8f40 0x7fc180153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:34.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc18581b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc180154ca0 0x7fc180157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:34.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc18581b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1801562f0 0x7fc1800c0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:34.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc18581b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc180153f50 con 0x7fc180154ca0 2023-11-15T03:08:34.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc18581b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1800aa5e0 con 0x7fc1801562f0 2023-11-15T03:08:34.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc18581b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1800aaa70 con 0x7fc1800a8f40 2023-11-15T03:08:34.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc17e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc180154ca0 0x7fc180157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:34.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc17e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc180154ca0 0x7fc180157160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56736/0 (socket says 172.21.15.121:56736) 2023-11-15T03:08:34.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc17e7fc700 1 -- 172.21.15.121:0/106543992 learned_addr learned my addr 172.21.15.121:0/106543992 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:34.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc17f7fe700 1 --2- 172.21.15.121:0/106543992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1801562f0 0x7fc1800c0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:34.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc17effd700 1 --2- 172.21.15.121:0/106543992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1800a8f40 0x7fc180153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:34.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc17e7fc700 1 -- 172.21.15.121:0/106543992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1800a8f40 msgr2=0x7fc180153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:34.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc17e7fc700 1 --2- 172.21.15.121:0/106543992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1800a8f40 0x7fc180153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:34.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc17e7fc700 1 -- 172.21.15.121:0/106543992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1801562f0 msgr2=0x7fc1800c0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:34.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc17e7fc700 1 --2- 172.21.15.121:0/106543992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1801562f0 0x7fc1800c0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:34.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc17e7fc700 1 -- 172.21.15.121:0/106543992 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc170009370 con 0x7fc180154ca0 2023-11-15T03:08:34.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc17e7fc700 1 --2- 172.21.15.121:0/106543992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc180154ca0 0x7fc180157160 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 rx=0x7fc17400e900 tx=0x7fc174009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:34.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc166ffd700 1 -- 172.21.15.121:0/106543992 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17400c640 con 0x7fc180154ca0 2023-11-15T03:08:34.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc166ffd700 1 -- 172.21.15.121:0/106543992 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17400c7a0 con 0x7fc180154ca0 2023-11-15T03:08:34.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc18581b700 1 -- 172.21.15.121:0/106543992 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1800a99d0 con 0x7fc180154ca0 2023-11-15T03:08:34.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc166ffd700 1 -- 172.21.15.121:0/106543992 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17400cb20 con 0x7fc180154ca0 2023-11-15T03:08:34.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.815+0000 7fc18581b700 1 -- 172.21.15.121:0/106543992 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1800a9f90 con 0x7fc180154ca0 2023-11-15T03:08:34.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.819+0000 7fc18581b700 1 -- 172.21.15.121:0/106543992 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1801548b0 con 0x7fc180154ca0 2023-11-15T03:08:34.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.819+0000 7fc166ffd700 1 -- 172.21.15.121:0/106543992 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc17401b400 con 0x7fc180154ca0 2023-11-15T03:08:34.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.823+0000 7fc166ffd700 1 --2- 172.21.15.121:0/106543992 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc16c05a510 0x7fc16c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:34.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.823+0000 7fc166ffd700 1 -- 172.21.15.121:0/106543992 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8674+0+0 (secure 0 0 0) 0x7fc1740756f0 con 0x7fc180154ca0 2023-11-15T03:08:34.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.823+0000 7fc17effd700 1 --2- 172.21.15.121:0/106543992 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc16c05a510 0x7fc16c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:34.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.823+0000 7fc17effd700 1 --2- 172.21.15.121:0/106543992 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc16c05a510 0x7fc16c05c9d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7fc1800a6e80 tx=0x7fc170017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:34.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:34.827+0000 7fc166ffd700 1 -- 172.21.15.121:0/106543992 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc174048460 con 0x7fc180154ca0 2023-11-15T03:08:35.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.023+0000 7fc18581b700 1 -- 172.21.15.121:0/106543992 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc18006c150 con 0x7fc180154ca0 2023-11-15T03:08:35.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.027+0000 7fc166ffd700 1 -- 172.21.15.121:0/106543992 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 254 v254) v1 ==== 95+0+4496 (secure 0 0 0) 0x7fc174016030 con 0x7fc180154ca0 2023-11-15T03:08:35.035 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:35.035 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":254,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1978069160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1978069160},{"type":"v1","addr":"172.21.15.121:6836","nonce":1978069160}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":254}],"filesystems":[{"mdsmap":{"epoch":253,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:08:30.208591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:35.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.027+0000 7fc18581b700 1 -- 172.21.15.121:0/106543992 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc16c05a510 msgr2=0x7fc16c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:35.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.027+0000 7fc18581b700 1 --2- 172.21.15.121:0/106543992 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc16c05a510 0x7fc16c05c9d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7fc1800a6e80 tx=0x7fc170017000).stop 2023-11-15T03:08:35.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.027+0000 7fc18581b700 1 -- 172.21.15.121:0/106543992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc180154ca0 msgr2=0x7fc180157160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:35.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.027+0000 7fc18581b700 1 --2- 172.21.15.121:0/106543992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc180154ca0 0x7fc180157160 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 rx=0x7fc17400e900 tx=0x7fc174009060).stop 2023-11-15T03:08:35.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.027+0000 7fc18581b700 1 -- 172.21.15.121:0/106543992 shutdown_connections 2023-11-15T03:08:35.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.027+0000 7fc18581b700 1 --2- 172.21.15.121:0/106543992 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc16c05a510 0x7fc16c05c9d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.027+0000 7fc18581b700 1 --2- 172.21.15.121:0/106543992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1801562f0 0x7fc1800c0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.027+0000 7fc18581b700 1 --2- 172.21.15.121:0/106543992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc180154ca0 0x7fc180157160 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.027+0000 7fc18581b700 1 --2- 172.21.15.121:0/106543992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1800a8f40 0x7fc180153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.027+0000 7fc18581b700 1 -- 172.21.15.121:0/106543992 >> 172.21.15.121:0/106543992 conn(0x7fc180062190 msgr2=0x7fc1800c1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:35.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.027+0000 7fc18581b700 1 -- 172.21.15.121:0/106543992 shutdown_connections 2023-11-15T03:08:35.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.027+0000 7fc18581b700 1 -- 172.21.15.121:0/106543992 wait complete. 2023-11-15T03:08:35.040 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 254 2023-11-15T03:08:35.058 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:08:35.058 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:35.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.207+0000 7fcc27569700 1 Processor -- start 2023-11-15T03:08:35.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.207+0000 7fcc27569700 1 -- start start 2023-11-15T03:08:35.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.211+0000 7fcc27569700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc20152780 0x7fcc20159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:35.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.211+0000 7fcc27569700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcc200c65d0 0x7fcc200c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:35.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.211+0000 7fcc27569700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc200c3fa0 con 0x7fcc200a5510 2023-11-15T03:08:35.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.211+0000 7fcc27569700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc20159630 con 0x7fcc200c65d0 2023-11-15T03:08:35.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.211+0000 7fcc27569700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc200c30b0 con 0x7fcc20152780 2023-11-15T03:08:35.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.211+0000 7fcc25b06700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fcc200a5510 0x7fcc200a59c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:52276/0 (socket says 172.21.15.121:52276) 2023-11-15T03:08:35.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.211+0000 7fcc25b06700 1 -- 172.21.15.121:0/3265351758 learned_addr learned my addr 172.21.15.121:0/3265351758 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:35.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.211+0000 7fcc25305700 1 --2- 172.21.15.121:0/3265351758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc20152780 0x7fcc20159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:35.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.211+0000 7fcc24b04700 1 --2- 172.21.15.121:0/3265351758 >> v2:172.21.15.190:3300/0 conn(0x7fcc200c65d0 0x7fcc200c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:35.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.215+0000 7fcc17fff700 1 -- 172.21.15.121:0/3265351758 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926469390 0 0) 0x7fcc200c3fa0 con 0x7fcc200a5510 2023-11-15T03:08:35.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.215+0000 7fcc17fff700 1 -- 172.21.15.121:0/3265351758 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc0c0033f0 con 0x7fcc200a5510 2023-11-15T03:08:35.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.215+0000 7fcc24b04700 1 -- 172.21.15.121:0/3265351758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc20152780 msgr2=0x7fcc20159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:35.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.215+0000 7fcc24b04700 1 --2- 172.21.15.121:0/3265351758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc20152780 0x7fcc20159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.215+0000 7fcc24b04700 1 -- 172.21.15.121:0/3265351758 >> v1:172.21.15.121:6789/0 conn(0x7fcc200a5510 legacy=0x7fcc200a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:35.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.215+0000 7fcc24b04700 1 -- 172.21.15.121:0/3265351758 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc200c3390 con 0x7fcc200c65d0 2023-11-15T03:08:35.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.215+0000 7fcc25305700 1 --2- 172.21.15.121:0/3265351758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc20152780 0x7fcc20159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:08:35.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.215+0000 7fcc24b04700 1 --2- 172.21.15.121:0/3265351758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc200c65d0 0x7fcc200c6a50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7fcc10009760 tx=0x7fcc10009060).ready entity=mon.1 client_cookie=fdba8b61cd4346ba server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:35.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.215+0000 7fcc17fff700 1 -- 172.21.15.121:0/3265351758 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1000d4a0 con 0x7fcc200c65d0 2023-11-15T03:08:35.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.215+0000 7fcc17fff700 1 -- 172.21.15.121:0/3265351758 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1000d600 con 0x7fcc200c65d0 2023-11-15T03:08:35.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.215+0000 7fcc17fff700 1 -- 172.21.15.121:0/3265351758 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1000d980 con 0x7fcc200c65d0 2023-11-15T03:08:35.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.215+0000 7fcc27569700 1 -- 172.21.15.121:0/3265351758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc200c65d0 msgr2=0x7fcc200c6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:35.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.215+0000 7fcc27569700 1 --2- 172.21.15.121:0/3265351758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc200c65d0 0x7fcc200c6a50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7fcc10009760 tx=0x7fcc10009060).stop 2023-11-15T03:08:35.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.215+0000 7fcc27569700 1 -- 172.21.15.121:0/3265351758 shutdown_connections 2023-11-15T03:08:35.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.215+0000 7fcc27569700 1 --2- 172.21.15.121:0/3265351758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc200c65d0 0x7fcc200c6a50 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.215+0000 7fcc27569700 1 --2- 172.21.15.121:0/3265351758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc20152780 0x7fcc20159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.215+0000 7fcc27569700 1 -- 172.21.15.121:0/3265351758 >> 172.21.15.121:0/3265351758 conn(0x7fcc20062190 msgr2=0x7fcc200625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:35.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.215+0000 7fcc27569700 1 -- 172.21.15.121:0/3265351758 shutdown_connections 2023-11-15T03:08:35.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.215+0000 7fcc27569700 1 -- 172.21.15.121:0/3265351758 wait complete. 2023-11-15T03:08:35.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc27569700 1 Processor -- start 2023-11-15T03:08:35.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc27569700 1 -- start start 2023-11-15T03:08:35.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc27569700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc200a5510 0x7fcc200ac270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:35.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc27569700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc200c65d0 0x7fcc2016cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:35.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc27569700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc20152780 0x7fcc2014fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:35.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc27569700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc200ab4f0 con 0x7fcc200a5510 2023-11-15T03:08:35.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc27569700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc200a9da0 con 0x7fcc20152780 2023-11-15T03:08:35.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc27569700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc200ab790 con 0x7fcc200c65d0 2023-11-15T03:08:35.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc25305700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc200a5510 0x7fcc200ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:35.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc25305700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc200a5510 0x7fcc200ac270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56738/0 (socket says 172.21.15.121:56738) 2023-11-15T03:08:35.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc25305700 1 -- 172.21.15.121:0/3067015800 learned_addr learned my addr 172.21.15.121:0/3067015800 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:35.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc25b06700 1 --2- 172.21.15.121:0/3067015800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc20152780 0x7fcc2014fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:35.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc24b04700 1 --2- 172.21.15.121:0/3067015800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc200c65d0 0x7fcc2016cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:35.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc25305700 1 -- 172.21.15.121:0/3067015800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc200c65d0 msgr2=0x7fcc2016cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:35.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc25305700 1 --2- 172.21.15.121:0/3067015800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc200c65d0 0x7fcc2016cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc25305700 1 -- 172.21.15.121:0/3067015800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc20152780 msgr2=0x7fcc2014fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:35.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc25305700 1 --2- 172.21.15.121:0/3067015800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc20152780 0x7fcc2014fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc25305700 1 -- 172.21.15.121:0/3067015800 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c0092a0 con 0x7fcc200a5510 2023-11-15T03:08:35.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc25b06700 1 --2- 172.21.15.121:0/3067015800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc20152780 0x7fcc2014fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:08:35.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.219+0000 7fcc24b04700 1 --2- 172.21.15.121:0/3067015800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc200c65d0 0x7fcc2016cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:08:35.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.223+0000 7fcc25305700 1 --2- 172.21.15.121:0/3067015800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc200a5510 0x7fcc200ac270 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 rx=0x7fcc1c00e980 tx=0x7fcc1c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:35.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.223+0000 7fcc157fa700 1 -- 172.21.15.121:0/3067015800 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c00c850 con 0x7fcc200a5510 2023-11-15T03:08:35.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.223+0000 7fcc157fa700 1 -- 172.21.15.121:0/3067015800 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c00c9b0 con 0x7fcc200a5510 2023-11-15T03:08:35.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.223+0000 7fcc27569700 1 -- 172.21.15.121:0/3067015800 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc100092a0 con 0x7fcc200a5510 2023-11-15T03:08:35.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.223+0000 7fcc157fa700 1 -- 172.21.15.121:0/3067015800 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c00cd30 con 0x7fcc200a5510 2023-11-15T03:08:35.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.223+0000 7fcc27569700 1 -- 172.21.15.121:0/3067015800 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc200abd50 con 0x7fcc200a5510 2023-11-15T03:08:35.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.223+0000 7fcc27569700 1 -- 172.21.15.121:0/3067015800 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2006c150 con 0x7fcc200a5510 2023-11-15T03:08:35.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.223+0000 7fcc157fa700 1 -- 172.21.15.121:0/3067015800 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fcc1c012600 con 0x7fcc200a5510 2023-11-15T03:08:35.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.227+0000 7fcc157fa700 1 --2- 172.21.15.121:0/3067015800 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcc0405a4e0 0x7fcc0405c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:35.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.227+0000 7fcc157fa700 1 -- 172.21.15.121:0/3067015800 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8674+0+0 (secure 0 0 0) 0x7fcc1c075840 con 0x7fcc200a5510 2023-11-15T03:08:35.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.227+0000 7fcc24b04700 1 --2- 172.21.15.121:0/3067015800 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcc0405a4e0 0x7fcc0405c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:35.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.227+0000 7fcc24b04700 1 --2- 172.21.15.121:0/3067015800 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcc0405a4e0 0x7fcc0405c9a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7fcc10009760 tx=0x7fcc10016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:35.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.231+0000 7fcc157fa700 1 -- 172.21.15.121:0/3067015800 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcc1c048560 con 0x7fcc200a5510 2023-11-15T03:08:35.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.427+0000 7fcc27569700 1 -- 172.21.15.121:0/3067015800 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc200c3670 con 0x7fcc200a5510 2023-11-15T03:08:35.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.427+0000 7fcc157fa700 1 -- 172.21.15.121:0/3067015800 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 254 v254) v1 ==== 95+0+4496 (secure 0 0 0) 0x7fcc1c04bb80 con 0x7fcc200a5510 2023-11-15T03:08:35.436 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:35.436 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":254,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1978069160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1978069160},{"type":"v1","addr":"172.21.15.121:6836","nonce":1978069160}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":254}],"filesystems":[{"mdsmap":{"epoch":253,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:08:30.208591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:35.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.431+0000 7fcc27569700 1 -- 172.21.15.121:0/3067015800 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcc0405a4e0 msgr2=0x7fcc0405c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:35.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.431+0000 7fcc27569700 1 --2- 172.21.15.121:0/3067015800 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcc0405a4e0 0x7fcc0405c9a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7fcc10009760 tx=0x7fcc10016000).stop 2023-11-15T03:08:35.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.431+0000 7fcc27569700 1 -- 172.21.15.121:0/3067015800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc200a5510 msgr2=0x7fcc200ac270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:35.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.431+0000 7fcc27569700 1 --2- 172.21.15.121:0/3067015800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc200a5510 0x7fcc200ac270 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 rx=0x7fcc1c00e980 tx=0x7fcc1c009060).stop 2023-11-15T03:08:35.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.431+0000 7fcc27569700 1 -- 172.21.15.121:0/3067015800 shutdown_connections 2023-11-15T03:08:35.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.431+0000 7fcc27569700 1 --2- 172.21.15.121:0/3067015800 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcc0405a4e0 0x7fcc0405c9a0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.431+0000 7fcc27569700 1 --2- 172.21.15.121:0/3067015800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc20152780 0x7fcc2014fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.431+0000 7fcc27569700 1 --2- 172.21.15.121:0/3067015800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc200c65d0 0x7fcc2016cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.431+0000 7fcc27569700 1 --2- 172.21.15.121:0/3067015800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc200a5510 0x7fcc200ac270 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.431+0000 7fcc27569700 1 -- 172.21.15.121:0/3067015800 >> 172.21.15.121:0/3067015800 conn(0x7fcc20062190 msgr2=0x7fcc200c18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:35.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.431+0000 7fcc27569700 1 -- 172.21.15.121:0/3067015800 shutdown_connections 2023-11-15T03:08:35.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.431+0000 7fcc27569700 1 -- 172.21.15.121:0/3067015800 wait complete. 2023-11-15T03:08:35.441 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 254 2023-11-15T03:08:35.457 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:08:35.457 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:35.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.611+0000 7fae3fc5b700 1 Processor -- start 2023-11-15T03:08:35.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.611+0000 7fae3fc5b700 1 -- start start 2023-11-15T03:08:35.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.615+0000 7fae3fc5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae38153bb0 0x7fae380a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:35.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.615+0000 7fae3fc5b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fae380aa020 0x7fae38067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:35.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.615+0000 7fae3fc5b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae38068400 con 0x7fae38067f50 2023-11-15T03:08:35.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.615+0000 7fae3fc5b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae380aa6f0 con 0x7fae380aa020 2023-11-15T03:08:35.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.615+0000 7fae3fc5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae380c5900 con 0x7fae38153bb0 2023-11-15T03:08:35.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.615+0000 7fae3d1f6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fae380aa020 0x7fae38067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:35.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.615+0000 7fae3d9f7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae38153bb0 0x7fae380a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:35.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.615+0000 7fae3e1f8700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fae38067f50 0x7fae381578a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:43060/0 (socket says 172.21.15.121:43060) 2023-11-15T03:08:35.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.615+0000 7fae3e1f8700 1 -- 172.21.15.121:0/3063643142 learned_addr learned my addr 172.21.15.121:0/3063643142 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:35.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.615+0000 7fae3d1f6700 1 -- 172.21.15.121:0/3063643142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae38153bb0 msgr2=0x7fae380a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:35.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.615+0000 7fae3d1f6700 1 --2- 172.21.15.121:0/3063643142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae38153bb0 0x7fae380a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.615+0000 7fae3d1f6700 1 -- 172.21.15.121:0/3063643142 >> v1:172.21.15.190:6789/0 conn(0x7fae38067f50 legacy=0x7fae381578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:35.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.615+0000 7fae3d1f6700 1 -- 172.21.15.121:0/3063643142 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae380cb5c0 con 0x7fae380aa020 2023-11-15T03:08:35.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.615+0000 7fae3c9f5700 1 -- 172.21.15.121:0/3063643142 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360877958 0 0) 0x7fae30001490 con 0x7fae38067f50 2023-11-15T03:08:35.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.615+0000 7fae3d1f6700 1 --2- 172.21.15.121:0/3063643142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae380aa020 0x7fae38067a60 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fae28009760 tx=0x7fae28009060).ready entity=mon.1 client_cookie=d98b6a00b98e96ed server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:35.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.615+0000 7fae3c9f5700 1 -- 172.21.15.121:0/3063643142 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2800f510 con 0x7fae380aa020 2023-11-15T03:08:35.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.615+0000 7fae3c9f5700 1 -- 172.21.15.121:0/3063643142 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2800f670 con 0x7fae380aa020 2023-11-15T03:08:35.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.615+0000 7fae3c9f5700 1 -- 172.21.15.121:0/3063643142 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2800f9f0 con 0x7fae380aa020 2023-11-15T03:08:35.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.619+0000 7fae3fc5b700 1 -- 172.21.15.121:0/3063643142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae380aa020 msgr2=0x7fae38067a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:35.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.619+0000 7fae3fc5b700 1 --2- 172.21.15.121:0/3063643142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae380aa020 0x7fae38067a60 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fae28009760 tx=0x7fae28009060).stop 2023-11-15T03:08:35.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.619+0000 7fae3fc5b700 1 -- 172.21.15.121:0/3063643142 shutdown_connections 2023-11-15T03:08:35.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.619+0000 7fae3fc5b700 1 --2- 172.21.15.121:0/3063643142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae380aa020 0x7fae38067a60 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.619+0000 7fae3fc5b700 1 --2- 172.21.15.121:0/3063643142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae38153bb0 0x7fae380a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.619+0000 7fae3fc5b700 1 -- 172.21.15.121:0/3063643142 >> 172.21.15.121:0/3063643142 conn(0x7fae38058c50 msgr2=0x7fae38059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:35.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.619+0000 7fae3fc5b700 1 -- 172.21.15.121:0/3063643142 shutdown_connections 2023-11-15T03:08:35.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.619+0000 7fae3fc5b700 1 -- 172.21.15.121:0/3063643142 wait complete. 2023-11-15T03:08:35.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.619+0000 7fae3fc5b700 1 Processor -- start 2023-11-15T03:08:35.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.619+0000 7fae3fc5b700 1 -- start start 2023-11-15T03:08:35.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3fc5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae38067f50 0x7fae38065ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:35.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3fc5b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae380aa020 0x7fae380aa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:35.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3fc5b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae38153bb0 0x7fae380aad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:35.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3fc5b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae380ab2e0 con 0x7fae38153bb0 2023-11-15T03:08:35.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3fc5b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae380ab5c0 con 0x7fae380aa020 2023-11-15T03:08:35.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3fc5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae380ab8a0 con 0x7fae38067f50 2023-11-15T03:08:35.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3d1f6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae380aa020 0x7fae380aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:35.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3e1f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae38153bb0 0x7fae380aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:35.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3d9f7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae38067f50 0x7fae38065ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:35.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3d9f7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae38067f50 0x7fae38065ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:35256/0 (socket says 172.21.15.121:35256) 2023-11-15T03:08:35.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3d1f6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae380aa020 0x7fae380aa840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:56336/0 (socket says 172.21.15.121:56336) 2023-11-15T03:08:35.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3e1f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae38153bb0 0x7fae380aad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56744/0 (socket says 172.21.15.121:56744) 2023-11-15T03:08:35.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3d9f7700 1 -- 172.21.15.121:0/3914947351 learned_addr learned my addr 172.21.15.121:0/3914947351 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:35.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3d9f7700 1 -- 172.21.15.121:0/3914947351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae380aa020 msgr2=0x7fae380aa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:35.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3d9f7700 1 --2- 172.21.15.121:0/3914947351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae380aa020 0x7fae380aa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3d9f7700 1 -- 172.21.15.121:0/3914947351 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae38153bb0 msgr2=0x7fae380aad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:35.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3d9f7700 1 --2- 172.21.15.121:0/3914947351 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae38153bb0 0x7fae380aad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3d9f7700 1 -- 172.21.15.121:0/3914947351 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae280092a0 con 0x7fae38067f50 2023-11-15T03:08:35.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3e1f8700 1 --2- 172.21.15.121:0/3914947351 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae38153bb0 0x7fae380aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:08:35.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3d9f7700 1 --2- 172.21.15.121:0/3914947351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae38067f50 0x7fae38065ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fae3400e900 tx=0x7fae34009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:35.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae2dffb700 1 -- 172.21.15.121:0/3914947351 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34012600 con 0x7fae38067f50 2023-11-15T03:08:35.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae3fc5b700 1 -- 172.21.15.121:0/3914947351 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae380abb80 con 0x7fae38067f50 2023-11-15T03:08:35.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae2dffb700 1 -- 172.21.15.121:0/3914947351 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae34012760 con 0x7fae38067f50 2023-11-15T03:08:35.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.623+0000 7fae2dffb700 1 -- 172.21.15.121:0/3914947351 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34012ae0 con 0x7fae38067f50 2023-11-15T03:08:35.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.627+0000 7fae3fc5b700 1 -- 172.21.15.121:0/3914947351 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae380c4980 con 0x7fae38067f50 2023-11-15T03:08:35.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.627+0000 7fae3fc5b700 1 -- 172.21.15.121:0/3914947351 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3806c150 con 0x7fae38067f50 2023-11-15T03:08:35.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.627+0000 7fae2dffb700 1 -- 172.21.15.121:0/3914947351 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fae3401b400 con 0x7fae38067f50 2023-11-15T03:08:35.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.631+0000 7fae2dffb700 1 --2- 172.21.15.121:0/3914947351 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fae1c05a560 0x7fae1c05ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:35.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.631+0000 7fae2dffb700 1 -- 172.21.15.121:0/3914947351 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8674+0+0 (secure 0 0 0) 0x7fae340754f0 con 0x7fae38067f50 2023-11-15T03:08:35.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.631+0000 7fae3d1f6700 1 --2- 172.21.15.121:0/3914947351 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fae1c05a560 0x7fae1c05ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:35.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.631+0000 7fae3d1f6700 1 --2- 172.21.15.121:0/3914947351 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fae1c05a560 0x7fae1c05ca20 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7fae28009760 tx=0x7fae28017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:35.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.639+0000 7fae2dffb700 1 -- 172.21.15.121:0/3914947351 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fae34048230 con 0x7fae38067f50 2023-11-15T03:08:35.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.839+0000 7fae3fc5b700 1 -- 172.21.15.121:0/3914947351 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae380ab9e0 con 0x7fae38067f50 2023-11-15T03:08:35.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.839+0000 7fae2dffb700 1 -- 172.21.15.121:0/3914947351 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 254 v254) v1 ==== 95+0+4496 (secure 0 0 0) 0x7fae380ab9e0 con 0x7fae38067f50 2023-11-15T03:08:35.850 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:35.850 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":254,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1978069160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1978069160},{"type":"v1","addr":"172.21.15.121:6836","nonce":1978069160}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":254}],"filesystems":[{"mdsmap":{"epoch":253,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:08:30.208591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:35.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.843+0000 7fae3fc5b700 1 -- 172.21.15.121:0/3914947351 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fae1c05a560 msgr2=0x7fae1c05ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:35.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.843+0000 7fae3fc5b700 1 --2- 172.21.15.121:0/3914947351 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fae1c05a560 0x7fae1c05ca20 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7fae28009760 tx=0x7fae28017000).stop 2023-11-15T03:08:35.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.843+0000 7fae3fc5b700 1 -- 172.21.15.121:0/3914947351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae38067f50 msgr2=0x7fae38065ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:35.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.843+0000 7fae3fc5b700 1 --2- 172.21.15.121:0/3914947351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae38067f50 0x7fae38065ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fae3400e900 tx=0x7fae34009060).stop 2023-11-15T03:08:35.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.843+0000 7fae3fc5b700 1 -- 172.21.15.121:0/3914947351 shutdown_connections 2023-11-15T03:08:35.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.843+0000 7fae3fc5b700 1 --2- 172.21.15.121:0/3914947351 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fae1c05a560 0x7fae1c05ca20 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.843+0000 7fae3fc5b700 1 --2- 172.21.15.121:0/3914947351 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae38153bb0 0x7fae380aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.843+0000 7fae3fc5b700 1 --2- 172.21.15.121:0/3914947351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae380aa020 0x7fae380aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.843+0000 7fae3fc5b700 1 --2- 172.21.15.121:0/3914947351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae38067f50 0x7fae38065ed0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:35.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.843+0000 7fae3fc5b700 1 -- 172.21.15.121:0/3914947351 >> 172.21.15.121:0/3914947351 conn(0x7fae38058c50 msgr2=0x7fae381581e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:35.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.843+0000 7fae3fc5b700 1 -- 172.21.15.121:0/3914947351 shutdown_connections 2023-11-15T03:08:35.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:35.843+0000 7fae3fc5b700 1 -- 172.21.15.121:0/3914947351 wait complete. 2023-11-15T03:08:35.856 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 254 2023-11-15T03:08:35.874 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate mds_snaptable' led to DAMAGED state 2023-11-15T03:08:35.875 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate mds0_inotable' 2023-11-15T03:08:35.875 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:08:35.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:08:35.876 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:08:35.895 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:08:35.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:08:35.897 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:08:36.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.099+0000 7f55a628f700 1 Processor -- start 2023-11-15T03:08:36.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.099+0000 7f55a628f700 1 -- start start 2023-11-15T03:08:36.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.099+0000 7f55a628f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f55a00abaa0 0x7f55a00abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:36.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.099+0000 7f55a628f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a00bb510 0x7f55a0063570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:36.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.099+0000 7f55a628f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a00c2c00 con 0x7f55a00a8880 2023-11-15T03:08:36.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.099+0000 7f55a628f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a0063a60 con 0x7f55a00abaa0 2023-11-15T03:08:36.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.099+0000 7f55a628f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a00657b0 con 0x7f55a00bb510 2023-11-15T03:08:36.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f55a5a8e700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f55a00a8880 0x7f55a0151400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:52290/0 (socket says 172.21.15.121:52290) 2023-11-15T03:08:36.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f55a5a8e700 1 -- 172.21.15.121:0/1287102878 learned_addr learned my addr 172.21.15.121:0/1287102878 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:36.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f55a528d700 1 --2- 172.21.15.121:0/1287102878 >> v2:172.21.15.190:3300/0 conn(0x7f55a00abaa0 0x7f55a00abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:36.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f55a4a8c700 1 --2- 172.21.15.121:0/1287102878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a00bb510 0x7f55a0063570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:36.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f55a528d700 1 -- 172.21.15.121:0/1287102878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a00bb510 msgr2=0x7f55a0063570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:36.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f5597fff700 1 -- 172.21.15.121:0/1287102878 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661715353 0 0) 0x7f55a00c2c00 con 0x7f55a00a8880 2023-11-15T03:08:36.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f55a528d700 1 --2- 172.21.15.121:0/1287102878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a00bb510 0x7f55a0063570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:36.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f55a528d700 1 -- 172.21.15.121:0/1287102878 >> v1:172.21.15.121:6789/0 conn(0x7f55a00a8880 legacy=0x7f55a0151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:36.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f55a528d700 1 -- 172.21.15.121:0/1287102878 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a0065a90 con 0x7f55a00abaa0 2023-11-15T03:08:36.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f55a528d700 1 --2- 172.21.15.121:0/1287102878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a00abaa0 0x7f55a00abe80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f559c009760 tx=0x7f559c009060).ready entity=mon.1 client_cookie=106ff3d08a207e48 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:36.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f5597fff700 1 -- 172.21.15.121:0/1287102878 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559c0090d0 con 0x7f55a00abaa0 2023-11-15T03:08:36.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f5597fff700 1 -- 172.21.15.121:0/1287102878 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559c00f510 con 0x7f55a00abaa0 2023-11-15T03:08:36.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f5597fff700 1 -- 172.21.15.121:0/1287102878 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559c00f890 con 0x7f55a00abaa0 2023-11-15T03:08:36.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f55a628f700 1 -- 172.21.15.121:0/1287102878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a00abaa0 msgr2=0x7f55a00abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:36.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f55a628f700 1 --2- 172.21.15.121:0/1287102878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a00abaa0 0x7f55a00abe80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f559c009760 tx=0x7f559c009060).stop 2023-11-15T03:08:36.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f55a628f700 1 -- 172.21.15.121:0/1287102878 shutdown_connections 2023-11-15T03:08:36.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f55a628f700 1 --2- 172.21.15.121:0/1287102878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a00bb510 0x7f55a0063570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:36.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f55a628f700 1 --2- 172.21.15.121:0/1287102878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a00abaa0 0x7f55a00abe80 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:36.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f55a628f700 1 -- 172.21.15.121:0/1287102878 >> 172.21.15.121:0/1287102878 conn(0x7f55a0067a60 msgr2=0x7f55a0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:36.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f55a628f700 1 -- 172.21.15.121:0/1287102878 shutdown_connections 2023-11-15T03:08:36.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.103+0000 7f55a628f700 1 -- 172.21.15.121:0/1287102878 wait complete. 2023-11-15T03:08:36.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.107+0000 7f55a628f700 1 Processor -- start 2023-11-15T03:08:36.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.107+0000 7f55a628f700 1 -- start start 2023-11-15T03:08:36.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.107+0000 7f55a628f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a00a8880 0x7f55a014c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:36.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.107+0000 7f55a628f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a00abaa0 0x7f55a00bfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:36.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.107+0000 7f55a628f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a00bb510 0x7f55a00c0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:36.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.107+0000 7f55a628f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a00c0530 con 0x7f55a00bb510 2023-11-15T03:08:36.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.107+0000 7f55a628f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a00c4a20 con 0x7f55a00abaa0 2023-11-15T03:08:36.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.107+0000 7f55a628f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a014c870 con 0x7f55a00a8880 2023-11-15T03:08:36.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.111+0000 7f55a5a8e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a00bb510 0x7f55a00c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:36.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.111+0000 7f55a5a8e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a00bb510 0x7f55a00c0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56748/0 (socket says 172.21.15.121:56748) 2023-11-15T03:08:36.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.111+0000 7f55a5a8e700 1 -- 172.21.15.121:0/2830671000 learned_addr learned my addr 172.21.15.121:0/2830671000 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:36.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.111+0000 7f55a4a8c700 1 --2- 172.21.15.121:0/2830671000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a00abaa0 0x7f55a00bfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:36.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.111+0000 7f55a528d700 1 --2- 172.21.15.121:0/2830671000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a00a8880 0x7f55a014c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:36.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.111+0000 7f55a5a8e700 1 -- 172.21.15.121:0/2830671000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a00a8880 msgr2=0x7f55a014c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:36.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.111+0000 7f55a5a8e700 1 --2- 172.21.15.121:0/2830671000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a00a8880 0x7f55a014c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:36.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.111+0000 7f55a5a8e700 1 -- 172.21.15.121:0/2830671000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a00abaa0 msgr2=0x7f55a00bfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:36.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.111+0000 7f55a5a8e700 1 --2- 172.21.15.121:0/2830671000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a00abaa0 0x7f55a00bfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:36.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.111+0000 7f55a5a8e700 1 -- 172.21.15.121:0/2830671000 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c0092a0 con 0x7f55a00bb510 2023-11-15T03:08:36.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.111+0000 7f55a5a8e700 1 --2- 172.21.15.121:0/2830671000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a00bb510 0x7f55a00c0040 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 rx=0x7f559800c730 tx=0x7f5598001520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:36.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.111+0000 7f55957fa700 1 -- 172.21.15.121:0/2830671000 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598005d30 con 0x7f55a00bb510 2023-11-15T03:08:36.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.111+0000 7f55957fa700 1 -- 172.21.15.121:0/2830671000 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5598005e90 con 0x7f55a00bb510 2023-11-15T03:08:36.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.111+0000 7f55a628f700 1 -- 172.21.15.121:0/2830671000 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a00a7590 con 0x7f55a00bb510 2023-11-15T03:08:36.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.111+0000 7f55957fa700 1 -- 172.21.15.121:0/2830671000 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598007300 con 0x7f55a00bb510 2023-11-15T03:08:36.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.111+0000 7f55a628f700 1 -- 172.21.15.121:0/2830671000 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a00c43d0 con 0x7f55a00bb510 2023-11-15T03:08:36.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.111+0000 7f55a628f700 1 -- 172.21.15.121:0/2830671000 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a00c4bb0 con 0x7f55a00bb510 2023-11-15T03:08:36.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.115+0000 7f55957fa700 1 -- 172.21.15.121:0/2830671000 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f55980076b0 con 0x7f55a00bb510 2023-11-15T03:08:36.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.115+0000 7f55957fa700 1 --2- 172.21.15.121:0/2830671000 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f558405a510 0x7f558405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:36.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.115+0000 7f55957fa700 1 -- 172.21.15.121:0/2830671000 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8674+0+0 (secure 0 0 0) 0x7f5598074ba0 con 0x7f55a00bb510 2023-11-15T03:08:36.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.115+0000 7f55a528d700 1 --2- 172.21.15.121:0/2830671000 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f558405a510 0x7f558405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:36.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.115+0000 7f55a528d700 1 --2- 172.21.15.121:0/2830671000 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f558405a510 0x7f558405c9d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f559c009fd0 tx=0x7f559c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:36.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.123+0000 7f55957fa700 1 -- 172.21.15.121:0/2830671000 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f55980478c0 con 0x7f55a00bb510 2023-11-15T03:08:36.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:36.319+0000 7f55a628f700 1 -- 172.21.15.121:0/2830671000 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f55a00c2890 con 0x7f55a00bb510 2023-11-15T03:08:37.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.267+0000 7f55957fa700 1 -- 172.21.15.121:0/2830671000 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v255) v1 ==== 161+0+0 (secure 0 0 0) 0x7f559804aee0 con 0x7f55a00bb510 2023-11-15T03:08:37.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.271+0000 7f55a628f700 1 -- 172.21.15.121:0/2830671000 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f558405a510 msgr2=0x7f558405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:37.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.271+0000 7f55a628f700 1 --2- 172.21.15.121:0/2830671000 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f558405a510 0x7f558405c9d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f559c009fd0 tx=0x7f559c017000).stop 2023-11-15T03:08:37.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.271+0000 7f55a628f700 1 -- 172.21.15.121:0/2830671000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a00bb510 msgr2=0x7f55a00c0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:37.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.271+0000 7f55a628f700 1 --2- 172.21.15.121:0/2830671000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a00bb510 0x7f55a00c0040 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 rx=0x7f559800c730 tx=0x7f5598001520).stop 2023-11-15T03:08:37.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.271+0000 7f55a628f700 1 -- 172.21.15.121:0/2830671000 shutdown_connections 2023-11-15T03:08:37.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.271+0000 7f55a628f700 1 --2- 172.21.15.121:0/2830671000 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f558405a510 0x7f558405c9d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:37.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.271+0000 7f55a628f700 1 --2- 172.21.15.121:0/2830671000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a00bb510 0x7f55a00c0040 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:37.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.271+0000 7f55a628f700 1 --2- 172.21.15.121:0/2830671000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a00abaa0 0x7f55a00bfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:37.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.271+0000 7f55a628f700 1 --2- 172.21.15.121:0/2830671000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a00a8880 0x7f55a014c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:37.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.271+0000 7f55a628f700 1 -- 172.21.15.121:0/2830671000 >> 172.21.15.121:0/2830671000 conn(0x7f55a0067a60 msgr2=0x7f55a00bb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:37.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.271+0000 7f55a628f700 1 -- 172.21.15.121:0/2830671000 shutdown_connections 2023-11-15T03:08:37.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.271+0000 7f55a628f700 1 -- 172.21.15.121:0/2830671000 wait complete. 2023-11-15T03:08:37.283 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:08:37.300 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:08:37.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.455+0000 7f9bf09a8700 1 Processor -- start 2023-11-15T03:08:37.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.455+0000 7f9bf09a8700 1 -- start start 2023-11-15T03:08:37.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.459+0000 7f9bf09a8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9bec0bb510 0x7f9bec0abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:37.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.459+0000 7f9bf09a8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bec0a8880 0x7f9bec151400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:37.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.459+0000 7f9bf09a8700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bec0c2c00 con 0x7f9bec0abaa0 2023-11-15T03:08:37.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.459+0000 7f9bf09a8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bec0657b0 con 0x7f9bec0bb510 2023-11-15T03:08:37.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.459+0000 7f9bf09a8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec065a90 con 0x7f9bec0a8880 2023-11-15T03:08:37.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.459+0000 7f9bead9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9bec0abaa0 0x7f9bec063570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:52298/0 (socket says 172.21.15.121:52298) 2023-11-15T03:08:37.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.459+0000 7f9bead9d700 1 -- 172.21.15.121:0/2605915015 learned_addr learned my addr 172.21.15.121:0/2605915015 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:37.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.459+0000 7f9be259c700 1 --2- 172.21.15.121:0/2605915015 >> v2:172.21.15.190:3300/0 conn(0x7f9bec0bb510 0x7f9bec0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:37.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.459+0000 7f9beb59e700 1 --2- 172.21.15.121:0/2605915015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bec0a8880 0x7f9bec151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:37.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.459+0000 7f9bea59c700 1 -- 172.21.15.121:0/2605915015 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 463255395 0 0) 0x7f9bec0c2c00 con 0x7f9bec0abaa0 2023-11-15T03:08:37.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.459+0000 7f9bea59c700 1 -- 172.21.15.121:0/2605915015 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bd80033f0 con 0x7f9bec0abaa0 2023-11-15T03:08:37.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.459+0000 7f9be259c700 1 -- 172.21.15.121:0/2605915015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bec0a8880 msgr2=0x7f9bec151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:37.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.459+0000 7f9be259c700 1 --2- 172.21.15.121:0/2605915015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bec0a8880 0x7f9bec151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:37.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.459+0000 7f9be259c700 1 -- 172.21.15.121:0/2605915015 >> v1:172.21.15.121:6789/0 conn(0x7f9bec0abaa0 legacy=0x7f9bec063570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:37.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.459+0000 7f9be259c700 1 -- 172.21.15.121:0/2605915015 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec0c2890 con 0x7f9bec0bb510 2023-11-15T03:08:37.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.459+0000 7f9beb59e700 1 --2- 172.21.15.121:0/2605915015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bec0a8880 0x7f9bec151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:08:37.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.459+0000 7f9be259c700 1 --2- 172.21.15.121:0/2605915015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bec0bb510 0x7f9bec0abe80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f9bd4006fd0 tx=0x7f9bd4006f40).ready entity=mon.1 client_cookie=8b88ce0c56fe4fc9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:37.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.463+0000 7f9bea59c700 1 -- 172.21.15.121:0/2605915015 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4005880 con 0x7f9bec0bb510 2023-11-15T03:08:37.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.463+0000 7f9bea59c700 1 -- 172.21.15.121:0/2605915015 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd40059e0 con 0x7f9bec0bb510 2023-11-15T03:08:37.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.463+0000 7f9bea59c700 1 -- 172.21.15.121:0/2605915015 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4005d60 con 0x7f9bec0bb510 2023-11-15T03:08:37.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.463+0000 7f9bf09a8700 1 -- 172.21.15.121:0/2605915015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bec0bb510 msgr2=0x7f9bec0abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:37.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.463+0000 7f9bf09a8700 1 --2- 172.21.15.121:0/2605915015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bec0bb510 0x7f9bec0abe80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f9bd4006fd0 tx=0x7f9bd4006f40).stop 2023-11-15T03:08:37.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.463+0000 7f9bf09a8700 1 -- 172.21.15.121:0/2605915015 shutdown_connections 2023-11-15T03:08:37.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.463+0000 7f9bf09a8700 1 --2- 172.21.15.121:0/2605915015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bec0a8880 0x7f9bec151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:37.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.463+0000 7f9bf09a8700 1 --2- 172.21.15.121:0/2605915015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bec0bb510 0x7f9bec0abe80 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:37.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.463+0000 7f9bf09a8700 1 -- 172.21.15.121:0/2605915015 >> 172.21.15.121:0/2605915015 conn(0x7f9bec067a60 msgr2=0x7f9bec067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:37.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.463+0000 7f9bf09a8700 1 -- 172.21.15.121:0/2605915015 shutdown_connections 2023-11-15T03:08:37.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.463+0000 7f9bf09a8700 1 -- 172.21.15.121:0/2605915015 wait complete. 2023-11-15T03:08:37.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.467+0000 7f9bf09a8700 1 Processor -- start 2023-11-15T03:08:37.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.467+0000 7f9bf09a8700 1 -- start start 2023-11-15T03:08:37.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.467+0000 7f9bf09a8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bec0a8880 0x7f9bec143690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:37.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.467+0000 7f9bf09a8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bec0abaa0 0x7f9bec0becb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:37.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.467+0000 7f9bf09a8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bec0bb510 0x7f9bec0bf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:37.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.467+0000 7f9bf09a8700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec0bf690 con 0x7f9bec0bb510 2023-11-15T03:08:37.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.467+0000 7f9bf09a8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec140b10 con 0x7f9bec0a8880 2023-11-15T03:08:37.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.467+0000 7f9bf09a8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec0aa160 con 0x7f9bec0abaa0 2023-11-15T03:08:37.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.467+0000 7f9beb59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bec0bb510 0x7f9bec0bf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:37.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.467+0000 7f9beb59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bec0bb510 0x7f9bec0bf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56752/0 (socket says 172.21.15.121:56752) 2023-11-15T03:08:37.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.467+0000 7f9beb59e700 1 -- 172.21.15.121:0/547895834 learned_addr learned my addr 172.21.15.121:0/547895834 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:37.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.467+0000 7f9bead9d700 1 --2- 172.21.15.121:0/547895834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bec0a8880 0x7f9bec143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:37.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.467+0000 7f9be259c700 1 --2- 172.21.15.121:0/547895834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bec0abaa0 0x7f9bec0becb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:37.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.467+0000 7f9beb59e700 1 -- 172.21.15.121:0/547895834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bec0abaa0 msgr2=0x7f9bec0becb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:37.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.467+0000 7f9beb59e700 1 --2- 172.21.15.121:0/547895834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bec0abaa0 0x7f9bec0becb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:37.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.467+0000 7f9beb59e700 1 -- 172.21.15.121:0/547895834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bec0a8880 msgr2=0x7f9bec143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:37.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.467+0000 7f9beb59e700 1 --2- 172.21.15.121:0/547895834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bec0a8880 0x7f9bec143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:37.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.467+0000 7f9beb59e700 1 -- 172.21.15.121:0/547895834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be40092a0 con 0x7f9bec0bb510 2023-11-15T03:08:37.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.467+0000 7f9beb59e700 1 --2- 172.21.15.121:0/547895834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bec0bb510 0x7f9bec0bf1a0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 rx=0x7f9be400e900 tx=0x7f9be4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:37.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.471+0000 7f9be37fe700 1 -- 172.21.15.121:0/547895834 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be4012600 con 0x7f9bec0bb510 2023-11-15T03:08:37.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.471+0000 7f9be37fe700 1 -- 172.21.15.121:0/547895834 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be4012760 con 0x7f9bec0bb510 2023-11-15T03:08:37.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.471+0000 7f9be37fe700 1 -- 172.21.15.121:0/547895834 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be4012b00 con 0x7f9bec0bb510 2023-11-15T03:08:37.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.471+0000 7f9bf09a8700 1 -- 172.21.15.121:0/547895834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd4009250 con 0x7f9bec0bb510 2023-11-15T03:08:37.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.471+0000 7f9bf09a8700 1 -- 172.21.15.121:0/547895834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bec0a9830 con 0x7f9bec0bb510 2023-11-15T03:08:37.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.471+0000 7f9bf09a8700 1 -- 172.21.15.121:0/547895834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bec0aa2d0 con 0x7f9bec0bb510 2023-11-15T03:08:37.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.471+0000 7f9be37fe700 1 -- 172.21.15.121:0/547895834 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9be401b400 con 0x7f9bec0bb510 2023-11-15T03:08:37.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.475+0000 7f9be37fe700 1 --2- 172.21.15.121:0/547895834 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9bcc05a510 0x7f9bcc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:37.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.475+0000 7f9be37fe700 1 -- 172.21.15.121:0/547895834 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8674+0+0 (secure 0 0 0) 0x7f9be40756a0 con 0x7f9bec0bb510 2023-11-15T03:08:37.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.475+0000 7f9bead9d700 1 --2- 172.21.15.121:0/547895834 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9bcc05a510 0x7f9bcc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:37.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.475+0000 7f9bead9d700 1 --2- 172.21.15.121:0/547895834 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9bcc05a510 0x7f9bcc05c9d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f9bdc005ad0 tx=0x7f9bdc009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:37.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.479+0000 7f9be37fe700 1 -- 172.21.15.121:0/547895834 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9be4048360 con 0x7f9bec0bb510 2023-11-15T03:08:37.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:37.703+0000 7f9bf09a8700 1 -- 172.21.15.121:0/547895834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f9bec06bcf0 con 0x7f9bec0bb510 2023-11-15T03:08:38.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.279+0000 7f9be37fe700 1 -- 172.21.15.121:0/547895834 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v256) v1 ==== 101+0+0 (secure 0 0 0) 0x7f9be4016030 con 0x7f9bec0bb510 2023-11-15T03:08:38.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.283+0000 7f9bf09a8700 1 -- 172.21.15.121:0/547895834 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9bcc05a510 msgr2=0x7f9bcc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.283+0000 7f9bf09a8700 1 --2- 172.21.15.121:0/547895834 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9bcc05a510 0x7f9bcc05c9d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f9bdc005ad0 tx=0x7f9bdc009040).stop 2023-11-15T03:08:38.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.283+0000 7f9bf09a8700 1 -- 172.21.15.121:0/547895834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bec0bb510 msgr2=0x7f9bec0bf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.283+0000 7f9bf09a8700 1 --2- 172.21.15.121:0/547895834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bec0bb510 0x7f9bec0bf1a0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 rx=0x7f9be400e900 tx=0x7f9be4009060).stop 2023-11-15T03:08:38.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.287+0000 7f9bf09a8700 1 -- 172.21.15.121:0/547895834 shutdown_connections 2023-11-15T03:08:38.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.287+0000 7f9bf09a8700 1 --2- 172.21.15.121:0/547895834 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9bcc05a510 0x7f9bcc05c9d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.287+0000 7f9bf09a8700 1 --2- 172.21.15.121:0/547895834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bec0bb510 0x7f9bec0bf1a0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.287+0000 7f9bf09a8700 1 --2- 172.21.15.121:0/547895834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bec0abaa0 0x7f9bec0becb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.287+0000 7f9bf09a8700 1 --2- 172.21.15.121:0/547895834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bec0a8880 0x7f9bec143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.287+0000 7f9bf09a8700 1 -- 172.21.15.121:0/547895834 >> 172.21.15.121:0/547895834 conn(0x7f9bec067a60 msgr2=0x7f9bec066370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:38.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.287+0000 7f9bf09a8700 1 -- 172.21.15.121:0/547895834 shutdown_connections 2023-11-15T03:08:38.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.287+0000 7f9bf09a8700 1 -- 172.21.15.121:0/547895834 wait complete. 2023-11-15T03:08:38.297 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:08:38.316 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:08:38.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.371+0000 7f875e9a6d00 1 Processor -- start 2023-11-15T03:08:38.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.371+0000 7f875e9a6d00 1 -- start start 2023-11-15T03:08:38.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.371+0000 7f875e9a6d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640285cabb0 0x5640285fd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.371+0000 7f875e9a6d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564028614b00 0x564028612df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.371+0000 7f875e9a6d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640286a7650 0x5640286a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.371+0000 7f875e9a6d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564028543220 con 0x5640286a7650 2023-11-15T03:08:38.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.371+0000 7f875e9a6d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564028610a60 con 0x5640285cabb0 2023-11-15T03:08:38.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.371+0000 7f875e9a6d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564028618b40 con 0x564028614b00 2023-11-15T03:08:38.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.371+0000 7f875cab8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640286a7650 0x5640286a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.371+0000 7f875cab8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640286a7650 0x5640286a5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56758/0 (socket says 172.21.15.121:56758) 2023-11-15T03:08:38.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.371+0000 7f875cab8700 1 -- 172.21.15.121:0/3621004720 learned_addr learned my addr 172.21.15.121:0/3621004720 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:38.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.371+0000 7f8753ab6700 1 --2- 172.21.15.121:0/3621004720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564028614b00 0x564028612df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.371+0000 7f875c2b7700 1 --2- 172.21.15.121:0/3621004720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640285cabb0 0x5640285fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.375+0000 7f875cab8700 1 -- 172.21.15.121:0/3621004720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564028614b00 msgr2=0x564028612df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:38.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.375+0000 7f875cab8700 1 --2- 172.21.15.121:0/3621004720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564028614b00 0x564028612df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.375+0000 7f875cab8700 1 -- 172.21.15.121:0/3621004720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640285cabb0 msgr2=0x5640285fd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:38.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.375+0000 7f875cab8700 1 --2- 172.21.15.121:0/3621004720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640285cabb0 0x5640285fd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.375+0000 7f875cab8700 1 -- 172.21.15.121:0/3621004720 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8744005db0 con 0x5640286a7650 2023-11-15T03:08:38.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.375+0000 7f875cab8700 1 --2- 172.21.15.121:0/3621004720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640286a7650 0x5640286a5ea0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 rx=0x7f875400b780 tx=0x7f8754005d40).ready entity=mon.0 client_cookie=931f878c297c34ec server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:38.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.375+0000 7f875b2b5700 1 -- 172.21.15.121:0/3621004720 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875400f590 con 0x5640286a7650 2023-11-15T03:08:38.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.375+0000 7f875b2b5700 1 -- 172.21.15.121:0/3621004720 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875400f6f0 con 0x5640286a7650 2023-11-15T03:08:38.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.375+0000 7f875b2b5700 1 -- 172.21.15.121:0/3621004720 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875400fa70 con 0x5640286a7650 2023-11-15T03:08:38.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.375+0000 7f875e9a6d00 1 -- 172.21.15.121:0/3621004720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640286a7650 msgr2=0x5640286a5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.375+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/3621004720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640286a7650 0x5640286a5ea0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 rx=0x7f875400b780 tx=0x7f8754005d40).stop 2023-11-15T03:08:38.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.375+0000 7f875e9a6d00 1 -- 172.21.15.121:0/3621004720 shutdown_connections 2023-11-15T03:08:38.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.375+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/3621004720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640286a7650 0x5640286a5ea0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.375+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/3621004720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564028614b00 0x564028612df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.375+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/3621004720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640285cabb0 0x5640285fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.375+0000 7f875e9a6d00 1 -- 172.21.15.121:0/3621004720 >> 172.21.15.121:0/3621004720 conn(0x56402861a080 msgr2=0x56402869d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:38.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.375+0000 7f875e9a6d00 1 -- 172.21.15.121:0/3621004720 shutdown_connections 2023-11-15T03:08:38.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.375+0000 7f875e9a6d00 1 -- 172.21.15.121:0/3621004720 wait complete. 2023-11-15T03:08:38.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875e9a6d00 1 Processor -- start 2023-11-15T03:08:38.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875e9a6d00 1 -- start start 2023-11-15T03:08:38.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875e9a6d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640285cabb0 0x564028698a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875e9a6d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564028614b00 0x56402860f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875e9a6d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5640286a7650 0x56402860f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875e9a6d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640285caa70 con 0x5640285cabb0 2023-11-15T03:08:38.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875e9a6d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640285fa140 con 0x564028614b00 2023-11-15T03:08:38.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875e9a6d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56402860fed0 con 0x5640286a7650 2023-11-15T03:08:38.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875c2b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640285cabb0 0x564028698a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875c2b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640285cabb0 0x564028698a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56760/0 (socket says 172.21.15.121:56760) 2023-11-15T03:08:38.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 learned_addr learned my addr 172.21.15.121:0/124709130 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:38.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875cab8700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5640286a7650 0x56402860f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f8753ab6700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564028614b00 0x56402860f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5640286a7650 msgr2=0x56402860f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875c2b7700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5640286a7650 0x56402860f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564028614b00 msgr2=0x56402860f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875c2b7700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564028614b00 0x56402860f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875400b430 con 0x5640285cabb0 2023-11-15T03:08:38.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875c2b7700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640285cabb0 0x564028698a70 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 rx=0x7f874c00efd0 tx=0x7f874c0017f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:38.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f8759ab2700 1 -- 172.21.15.121:0/124709130 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c015070 con 0x5640285cabb0 2023-11-15T03:08:38.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f8759ab2700 1 -- 172.21.15.121:0/124709130 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874c005030 con 0x5640285cabb0 2023-11-15T03:08:38.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56402856c770 con 0x5640285cabb0 2023-11-15T03:08:38.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f8759ab2700 1 -- 172.21.15.121:0/124709130 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c010820 con 0x5640285cabb0 2023-11-15T03:08:38.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.379+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56402853ba70 con 0x5640285cabb0 2023-11-15T03:08:38.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.383+0000 7f8759ab2700 1 -- 172.21.15.121:0/124709130 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f874c010b90 con 0x5640285cabb0 2023-11-15T03:08:38.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.383+0000 7f8759ab2700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f873c058ff0 0x7f873c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.383+0000 7f8759ab2700 1 -- 172.21.15.121:0/124709130 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8674+0+0 (secure 0 0 0) 0x7f874c008070 con 0x5640285cabb0 2023-11-15T03:08:38.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.383+0000 7f8753ab6700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f873c058ff0 0x7f873c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.398 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:08:38.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.387+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5640285fc390 0x5640286c0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.387+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286123e0 con 0x5640285fc390 2023-11-15T03:08:38.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.387+0000 7f875cab8700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5640285fc390 0x5640286c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.387+0000 7f8753ab6700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f873c058ff0 0x7f873c05b4b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f874400c010 tx=0x7f8744016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:38.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.387+0000 7f875cab8700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5640285fc390 0x5640286c0750 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:38.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.387+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v122'329 uv322 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640285fc390 2023-11-15T03:08:38.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.387+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640285fc770 con 0x5640285fc390 2023-11-15T03:08:38.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.391+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v122'330 uv330 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640285fc390 2023-11-15T03:08:38.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.391+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x564028615a70 con 0x5640285fc390 2023-11-15T03:08:38.403 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:08:38.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.395+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v122'331 uv331 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640285fc390 2023-11-15T03:08:38.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.395+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x564028698f60 con 0x5640285fc390 2023-11-15T03:08:38.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.395+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v122'332 uv332 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640285fc390 2023-11-15T03:08:38.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.395+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c1c50 con 0x5640285fc390 2023-11-15T03:08:38.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.399+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v122'333 uv333 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640285fc390 2023-11-15T03:08:38.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.399+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c0c40 con 0x5640285fc390 2023-11-15T03:08:38.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.403+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v122'334 uv334 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640285fc390 2023-11-15T03:08:38.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.403+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c0ff0 con 0x5640285fc390 2023-11-15T03:08:38.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.403+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v122'335 uv328 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640285fc390 2023-11-15T03:08:38.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.403+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286c13a0 con 0x5640285fc390 2023-11-15T03:08:38.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.407+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v122'336 uv336 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640285fc390 2023-11-15T03:08:38.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.407+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c1750 con 0x5640285fc390 2023-11-15T03:08:38.416 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:08:38.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.407+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v122'337 uv337 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640285fc390 2023-11-15T03:08:38.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.407+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c1750 con 0x5640285fc390 2023-11-15T03:08:38.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.411+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v122'338 uv338 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640285fc390 2023-11-15T03:08:38.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.411+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c1750 con 0x5640285fc390 2023-11-15T03:08:38.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.411+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v122'339 uv339 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640285fc390 2023-11-15T03:08:38.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.411+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c1750 con 0x5640285fc390 2023-11-15T03:08:38.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.415+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v122'340 uv340 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640285fc390 2023-11-15T03:08:38.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.415+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5640286c2510 0x5640286c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.415+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c1750 con 0x5640286c2510 2023-11-15T03:08:38.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.415+0000 7f875c2b7700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5640286c2510 0x5640286c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.415+0000 7f875c2b7700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5640286c2510 0x5640286c4900 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:38.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.419+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v122'518 uv505 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.419+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286c4df0 con 0x5640286c2510 2023-11-15T03:08:38.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.423+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v122'519 uv519 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.423+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c51a0 con 0x5640286c2510 2023-11-15T03:08:38.435 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:08:38.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.427+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v122'520 uv520 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.427+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c6560 con 0x5640286c2510 2023-11-15T03:08:38.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.427+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v122'521 uv521 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.431+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c6910 con 0x5640286c2510 2023-11-15T03:08:38.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.431+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v122'522 uv522 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.431+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c5550 con 0x5640286c2510 2023-11-15T03:08:38.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.435+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v122'523 uv523 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.435+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c5e40 con 0x5640286c2510 2023-11-15T03:08:38.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.435+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v122'524 uv524 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.439+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c6cc0 con 0x5640286c2510 2023-11-15T03:08:38.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.439+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v122'525 uv511 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.439+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286c6cc0 con 0x5640286c2510 2023-11-15T03:08:38.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.443+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v122'526 uv526 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.443+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c6cc0 con 0x5640286c2510 2023-11-15T03:08:38.454 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:08:38.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.447+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v122'527 uv527 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.447+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c7cd0 con 0x5640286c2510 2023-11-15T03:08:38.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.447+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v122'528 uv528 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.447+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c7cd0 con 0x5640286c2510 2023-11-15T03:08:38.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.451+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v122'529 uv529 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.451+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c6cc0 con 0x5640286c2510 2023-11-15T03:08:38.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.455+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v122'530 uv530 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.455+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c6cc0 con 0x5640286c2510 2023-11-15T03:08:38.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.455+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v122'531 uv517 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.455+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286c6cc0 con 0x5640286c2510 2023-11-15T03:08:38.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.459+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v122'532 uv532 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.459+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c6cc0 con 0x5640286c2510 2023-11-15T03:08:38.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.463+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v122'533 uv533 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.471 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:08:38.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.463+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c7cd0 con 0x5640286c2510 2023-11-15T03:08:38.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.463+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v122'534 uv534 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.463+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c7cd0 con 0x5640286c2510 2023-11-15T03:08:38.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.467+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v122'535 uv535 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.467+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c6cc0 con 0x5640286c2510 2023-11-15T03:08:38.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.471+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v122'536 uv536 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.471+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5640286c71d0 0x5640286c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.471+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286c9ab0 con 0x5640286c71d0 2023-11-15T03:08:38.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.471+0000 7f8753ab6700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5640286c71d0 0x5640286c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.471+0000 7f8753ab6700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5640286c71d0 0x5640286c95c0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:38.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.475+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v122'284 uv278 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286c71d0 2023-11-15T03:08:38.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.475+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286c9e60 con 0x5640286c71d0 2023-11-15T03:08:38.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.479+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v122'285 uv285 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286c71d0 2023-11-15T03:08:38.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.479+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286ca210 con 0x5640286c71d0 2023-11-15T03:08:38.491 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:08:38.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.483+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v122'286 uv286 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286c71d0 2023-11-15T03:08:38.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.483+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286caab0 con 0x5640286c71d0 2023-11-15T03:08:38.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.487+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v122'287 uv287 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286c71d0 2023-11-15T03:08:38.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.487+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286cae60 con 0x5640286c71d0 2023-11-15T03:08:38.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.491+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v122'288 uv288 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286c71d0 2023-11-15T03:08:38.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.491+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286cb210 con 0x5640286c71d0 2023-11-15T03:08:38.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.491+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v122'289 uv283 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286c71d0 2023-11-15T03:08:38.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.491+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286cb5c0 con 0x5640286c71d0 2023-11-15T03:08:38.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.495+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v122'290 uv290 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286c71d0 2023-11-15T03:08:38.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.495+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286cb970 con 0x5640286c71d0 2023-11-15T03:08:38.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.499+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v122'291 uv291 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286c71d0 2023-11-15T03:08:38.507 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:08:38.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.499+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286cb970 con 0x5640286c71d0 2023-11-15T03:08:38.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.499+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v122'292 uv292 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286c71d0 2023-11-15T03:08:38.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.499+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286cb970 con 0x5640286c71d0 2023-11-15T03:08:38.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.503+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v122'293 uv293 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286c71d0 2023-11-15T03:08:38.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.503+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5640286cbe80 0x5640286ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.503+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286ce760 con 0x5640286cbe80 2023-11-15T03:08:38.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.503+0000 7f875cab8700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5640286cbe80 0x5640286ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.507+0000 7f875cab8700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5640286cbe80 0x5640286ce270 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:38.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.507+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v122'333 uv326 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.507+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286ce7a0 con 0x5640286cbe80 2023-11-15T03:08:38.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.511+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v122'334 uv334 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.511+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286ce7a0 con 0x5640286cbe80 2023-11-15T03:08:38.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.515+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v122'335 uv335 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.524 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:08:38.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.515+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d2cb0 con 0x5640286cbe80 2023-11-15T03:08:38.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.519+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v122'336 uv336 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.519+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d2cb0 con 0x5640286cbe80 2023-11-15T03:08:38.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.519+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v122'337 uv337 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.519+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286cec20 con 0x5640286cbe80 2023-11-15T03:08:38.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.523+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v122'338 uv332 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.523+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286cec20 con 0x5640286cbe80 2023-11-15T03:08:38.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.527+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v122'339 uv339 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.527+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286cec20 con 0x5640286cbe80 2023-11-15T03:08:38.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.531+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v122'340 uv340 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.543 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:08:38.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.531+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286cfc30 con 0x5640286cbe80 2023-11-15T03:08:38.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.535+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v122'341 uv341 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.535+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286cfc30 con 0x5640286cbe80 2023-11-15T03:08:38.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.539+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v122'342 uv342 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.539+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286cec20 con 0x5640286cbe80 2023-11-15T03:08:38.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.539+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v122'343 uv343 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.539+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5640286cf130 0x5640286d1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.539+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d1ae0 con 0x5640286cf130 2023-11-15T03:08:38.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.539+0000 7f875c2b7700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5640286cf130 0x5640286d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.543+0000 7f875c2b7700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5640286cf130 0x5640286d1520 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:38.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.543+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v122'165 uv164 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286cf130 2023-11-15T03:08:38.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.543+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286d1ec0 con 0x5640286cf130 2023-11-15T03:08:38.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.547+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v122'166 uv166 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286cf130 2023-11-15T03:08:38.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.547+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d1ec0 con 0x5640286cf130 2023-11-15T03:08:38.558 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:08:38.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.551+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v122'167 uv167 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286cf130 2023-11-15T03:08:38.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.551+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d2ed0 con 0x5640286cf130 2023-11-15T03:08:38.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.551+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v122'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286cf130 2023-11-15T03:08:38.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.551+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d2ed0 con 0x5640286cf130 2023-11-15T03:08:38.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.555+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v122'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286cf130 2023-11-15T03:08:38.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.555+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d1ec0 con 0x5640286cf130 2023-11-15T03:08:38.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.555+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v122'170 uv170 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286cf130 2023-11-15T03:08:38.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.555+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5640286d2540 0x5640286d4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.555+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d4fb0 con 0x5640286d2540 2023-11-15T03:08:38.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.559+0000 7f8753ab6700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5640286d2540 0x5640286d4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.559+0000 7f8753ab6700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5640286d2540 0x5640286d4930 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:38.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.559+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v122'117 uv116 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.559+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286d5390 con 0x5640286d2540 2023-11-15T03:08:38.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.563+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v122'118 uv118 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.563+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5390 con 0x5640286d2540 2023-11-15T03:08:38.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.567+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v122'119 uv119 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.576 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:08:38.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.567+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5390 con 0x5640286d2540 2023-11-15T03:08:38.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.571+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v122'120 uv120 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.571+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5390 con 0x5640286d2540 2023-11-15T03:08:38.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.571+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v122'165 uv164 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.571+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286d5390 con 0x5640286d2540 2023-11-15T03:08:38.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.575+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v122'166 uv166 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.575+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5390 con 0x5640286d2540 2023-11-15T03:08:38.587 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:08:38.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.579+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v122'167 uv167 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.579+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d63a0 con 0x5640286d2540 2023-11-15T03:08:38.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.579+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v122'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.583+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d63a0 con 0x5640286d2540 2023-11-15T03:08:38.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.583+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v122'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.583+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5390 con 0x5640286d2540 2023-11-15T03:08:38.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.587+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v122'170 uv170 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.587+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5390 con 0x5640286c2510 2023-11-15T03:08:38.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.591+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v122'305 uv299 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.591+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286d5390 con 0x5640286c2510 2023-11-15T03:08:38.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.595+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v122'306 uv306 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.595+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5390 con 0x5640286c2510 2023-11-15T03:08:38.609 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:08:38.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.599+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v122'307 uv307 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.599+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d63a0 con 0x5640286c2510 2023-11-15T03:08:38.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.603+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v122'308 uv308 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.603+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d63a0 con 0x5640286c2510 2023-11-15T03:08:38.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.607+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v122'309 uv309 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.607+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5390 con 0x5640286c2510 2023-11-15T03:08:38.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.607+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v122'310 uv310 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.607+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5390 con 0x5640286c2510 2023-11-15T03:08:38.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.611+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v122'311 uv304 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.611+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286d5390 con 0x5640286c2510 2023-11-15T03:08:38.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.615+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v122'312 uv312 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.615+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5390 con 0x5640286c2510 2023-11-15T03:08:38.628 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:08:38.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.619+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v122'313 uv313 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.619+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5390 con 0x5640286c2510 2023-11-15T03:08:38.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.623+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v122'314 uv314 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.623+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286ca640 con 0x5640286c2510 2023-11-15T03:08:38.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.627+0000 7f875c2b7700 1 -- 172.21.15.121:0/124709130 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v122'315 uv315 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f874c017070 con 0x5640286c2510 2023-11-15T03:08:38.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.627+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286ca640 con 0x5640286c71d0 2023-11-15T03:08:38.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.627+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v122'138 uv137 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286c71d0 2023-11-15T03:08:38.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.627+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286ca640 con 0x5640286c71d0 2023-11-15T03:08:38.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.631+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v122'139 uv139 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286c71d0 2023-11-15T03:08:38.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.631+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286ca640 con 0x5640286c71d0 2023-11-15T03:08:38.643 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:08:38.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.635+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v122'140 uv140 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286c71d0 2023-11-15T03:08:38.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.635+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286ca640 con 0x5640286c71d0 2023-11-15T03:08:38.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.635+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v122'141 uv141 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286c71d0 2023-11-15T03:08:38.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.635+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286ca640 con 0x5640286c71d0 2023-11-15T03:08:38.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.639+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v122'142 uv142 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286c71d0 2023-11-15T03:08:38.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.639+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286ca640 con 0x5640286d2540 2023-11-15T03:08:38.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.643+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v122'165 uv164 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.643+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286ca640 con 0x5640286d2540 2023-11-15T03:08:38.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.643+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v122'166 uv166 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.643+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286ca640 con 0x5640286d2540 2023-11-15T03:08:38.656 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:08:38.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.647+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v122'167 uv167 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.647+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286ca640 con 0x5640286d2540 2023-11-15T03:08:38.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.651+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v122'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.651+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286ca680 con 0x5640286d2540 2023-11-15T03:08:38.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.655+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v122'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.655+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5880 con 0x5640286d2540 2023-11-15T03:08:38.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.655+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v122'170 uv170 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.655+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5880 con 0x5640286d2540 2023-11-15T03:08:38.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.659+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v122'138 uv137 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.659+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286d5880 con 0x5640286d2540 2023-11-15T03:08:38.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.663+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v122'139 uv139 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.663+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5880 con 0x5640286d2540 2023-11-15T03:08:38.674 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:08:38.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.667+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v122'140 uv140 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.667+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5880 con 0x5640286d2540 2023-11-15T03:08:38.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.667+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v122'141 uv141 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.667+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5900 con 0x5640286d2540 2023-11-15T03:08:38.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.671+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v122'142 uv142 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.671+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5900 con 0x5640286d2540 2023-11-15T03:08:38.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.675+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v122'136 uv135 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.675+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286d5880 con 0x5640286d2540 2023-11-15T03:08:38.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.675+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v122'137 uv137 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.675+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5880 con 0x5640286d2540 2023-11-15T03:08:38.687 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:08:38.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.679+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v122'138 uv138 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.679+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5900 con 0x5640286d2540 2023-11-15T03:08:38.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.683+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v122'139 uv139 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.683+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d8650 con 0x5640286d2540 2023-11-15T03:08:38.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.683+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v122'140 uv140 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.683+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5640286d5410 0x5640286d57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.683+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5ce0 con 0x5640286d5410 2023-11-15T03:08:38.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.687+0000 7f875cab8700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5640286d5410 0x5640286d57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.687+0000 7f875cab8700 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5640286d5410 0x5640286d57f0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:38.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.691+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v122'110 uv109 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286d5410 2023-11-15T03:08:38.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.691+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286d5ce0 con 0x5640286d5410 2023-11-15T03:08:38.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.691+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v122'111 uv111 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286d5410 2023-11-15T03:08:38.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.691+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5ce0 con 0x5640286d5410 2023-11-15T03:08:38.704 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:08:38.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.695+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v122'112 uv112 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286d5410 2023-11-15T03:08:38.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.695+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5ce0 con 0x5640286d5410 2023-11-15T03:08:38.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.699+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v122'113 uv113 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286d5410 2023-11-15T03:08:38.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.699+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5ce0 con 0x5640286d2540 2023-11-15T03:08:38.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.703+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v122'165 uv164 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.703+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286d5ce0 con 0x5640286d2540 2023-11-15T03:08:38.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.707+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v122'166 uv166 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.707+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5ce0 con 0x5640286d2540 2023-11-15T03:08:38.718 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:08:38.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.711+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v122'167 uv167 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.711+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5ce0 con 0x5640286d2540 2023-11-15T03:08:38.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.711+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v122'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.711+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5ce0 con 0x5640286d2540 2023-11-15T03:08:38.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.715+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v122'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.715+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5ce0 con 0x5640286d2540 2023-11-15T03:08:38.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.719+0000 7f8753ab6700 1 -- 172.21.15.121:0/124709130 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v122'170 uv170 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8744020470 con 0x5640286d2540 2023-11-15T03:08:38.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.719+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5ce0 con 0x5640286cbe80 2023-11-15T03:08:38.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.723+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v122'140 uv139 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.723+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286d5ce0 con 0x5640286cbe80 2023-11-15T03:08:38.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.723+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v122'141 uv141 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.723+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d5ce0 con 0x5640286cbe80 2023-11-15T03:08:38.736 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:08:38.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.727+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v122'142 uv142 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.727+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d6130 con 0x5640286cbe80 2023-11-15T03:08:38.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.731+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v122'143 uv143 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.731+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d60e0 con 0x5640286cbe80 2023-11-15T03:08:38.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.731+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v122'144 uv144 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.731+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d60e0 con 0x5640286cbe80 2023-11-15T03:08:38.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.735+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v122'329 uv321 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.735+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286d60e0 con 0x5640286cbe80 2023-11-15T03:08:38.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.739+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v122'330 uv330 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.739+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d60e0 con 0x5640286cbe80 2023-11-15T03:08:38.749 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:08:38.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.739+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v122'331 uv331 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.739+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d60e0 con 0x5640286cbe80 2023-11-15T03:08:38.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.743+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v122'332 uv332 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.743+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d60e0 con 0x5640286cbe80 2023-11-15T03:08:38.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.747+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v122'333 uv333 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.747+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d60e0 con 0x5640286cbe80 2023-11-15T03:08:38.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.747+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v122'334 uv328 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.747+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e122) v8 -- 0x5640286d60e0 con 0x5640286cbe80 2023-11-15T03:08:38.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.751+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v122'335 uv335 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.751+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d60e0 con 0x5640286cbe80 2023-11-15T03:08:38.763 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:08:38.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.755+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v122'336 uv336 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.755+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d79b0 con 0x5640286cbe80 2023-11-15T03:08:38.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.755+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v122'337 uv337 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.759+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d79b0 con 0x5640286cbe80 2023-11-15T03:08:38.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.759+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v122'338 uv338 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.759+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d60e0 con 0x5640286cbe80 2023-11-15T03:08:38.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.763+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v122'339 uv339 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.763+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x5640286d69a0 con 0x5640286cbe80 2023-11-15T03:08:38.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v122'340 uv340 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8754019040 con 0x5640286cbe80 2023-11-15T03:08:38.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5640286c71d0 msgr2=0x5640286c95c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5640286c71d0 0x5640286c95c0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5640285fc390 msgr2=0x5640286c0750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5640285fc390 0x5640286c0750 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5640286cf130 msgr2=0x5640286d1520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5640286cf130 0x5640286d1520 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5640286c2510 msgr2=0x5640286c4900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5640286c2510 0x5640286c4900 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5640286d5410 msgr2=0x5640286d57f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5640286d5410 0x5640286d57f0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5640286cbe80 msgr2=0x5640286ce270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5640286cbe80 0x5640286ce270 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5640286d2540 msgr2=0x5640286d4930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5640286d2540 0x5640286d4930 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 reap_dead start 2023-11-15T03:08:38.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 reap_dead start 2023-11-15T03:08:38.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875cab8700 1 -- 172.21.15.121:0/124709130 reap_dead start 2023-11-15T03:08:38.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f873c058ff0 msgr2=0x7f873c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f873c058ff0 0x7f873c05b4b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f874400c010 tx=0x7f8744016000).stop 2023-11-15T03:08:38.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640285cabb0 msgr2=0x564028698a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640285cabb0 0x564028698a70 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 rx=0x7f874c00efd0 tx=0x7f874c0017f0).stop 2023-11-15T03:08:38.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 shutdown_connections 2023-11-15T03:08:38.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f873c058ff0 0x7f873c05b4b0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 --2- 172.21.15.121:0/124709130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640285cabb0 0x564028698a70 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 >> 172.21.15.121:0/124709130 conn(0x56402861a080 msgr2=0x5640286a61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:38.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 shutdown_connections 2023-11-15T03:08:38.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.767+0000 7f875e9a6d00 1 -- 172.21.15.121:0/124709130 wait complete. 2023-11-15T03:08:38.784 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate mds0_inotable 0 2023-11-15T03:08:38.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.879+0000 7f656f980d00 1 Processor -- start 2023-11-15T03:08:38.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.879+0000 7f656f980d00 1 -- start start 2023-11-15T03:08:38.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.879+0000 7f656f980d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c50d68c010 0x55c50d68c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.879+0000 7f656f980d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c50d5f4fe0 0x55c50d5f5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.879+0000 7f656f980d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c50d5f5950 0x55c50d695f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.879+0000 7f656f980d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50d529220 con 0x55c50d5f5950 2023-11-15T03:08:38.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.879+0000 7f656f980d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50d5e5d20 con 0x55c50d5f4fe0 2023-11-15T03:08:38.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.879+0000 7f656f980d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c50d6895c0 con 0x55c50d68c010 2023-11-15T03:08:38.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.879+0000 7f656dab8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c50d5f5950 0x55c50d695f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.879+0000 7f656dab8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c50d5f5950 0x55c50d695f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56782/0 (socket says 172.21.15.121:56782) 2023-11-15T03:08:38.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.879+0000 7f656dab8700 1 -- 172.21.15.121:0/2132366851 learned_addr learned my addr 172.21.15.121:0/2132366851 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:38.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.879+0000 7f656cab6700 1 --2- 172.21.15.121:0/2132366851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c50d5f4fe0 0x55c50d5f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.879+0000 7f656d2b7700 1 --2- 172.21.15.121:0/2132366851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c50d68c010 0x55c50d68c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.879+0000 7f656dab8700 1 -- 172.21.15.121:0/2132366851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c50d68c010 msgr2=0x55c50d68c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:38.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.879+0000 7f656dab8700 1 --2- 172.21.15.121:0/2132366851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c50d68c010 0x55c50d68c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.879+0000 7f656dab8700 1 -- 172.21.15.121:0/2132366851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c50d5f4fe0 msgr2=0x55c50d5f5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:38.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.879+0000 7f656dab8700 1 --2- 172.21.15.121:0/2132366851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c50d5f4fe0 0x55c50d5f5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.879+0000 7f656dab8700 1 -- 172.21.15.121:0/2132366851 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c50d552770 con 0x55c50d5f5950 2023-11-15T03:08:38.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.879+0000 7f656dab8700 1 --2- 172.21.15.121:0/2132366851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c50d5f5950 0x55c50d695f90 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 rx=0x7f6568002650 tx=0x7f6568007040).ready entity=mon.0 client_cookie=a6e5accbc5d2cdb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:38.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.883+0000 7f655f7fe700 1 -- 172.21.15.121:0/2132366851 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65680022a0 con 0x55c50d5f5950 2023-11-15T03:08:38.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.883+0000 7f655f7fe700 1 -- 172.21.15.121:0/2132366851 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6568011540 con 0x55c50d5f5950 2023-11-15T03:08:38.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.883+0000 7f655f7fe700 1 -- 172.21.15.121:0/2132366851 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65680118e0 con 0x55c50d5f5950 2023-11-15T03:08:38.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.883+0000 7f656f980d00 1 -- 172.21.15.121:0/2132366851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c50d5f5950 msgr2=0x55c50d695f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.883+0000 7f656f980d00 1 --2- 172.21.15.121:0/2132366851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c50d5f5950 0x55c50d695f90 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 rx=0x7f6568002650 tx=0x7f6568007040).stop 2023-11-15T03:08:38.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.883+0000 7f656f980d00 1 -- 172.21.15.121:0/2132366851 shutdown_connections 2023-11-15T03:08:38.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.883+0000 7f656f980d00 1 --2- 172.21.15.121:0/2132366851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c50d5f5950 0x55c50d695f90 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.883+0000 7f656f980d00 1 --2- 172.21.15.121:0/2132366851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c50d5f4fe0 0x55c50d5f5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.883+0000 7f656f980d00 1 --2- 172.21.15.121:0/2132366851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c50d68c010 0x55c50d68c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.883+0000 7f656f980d00 1 -- 172.21.15.121:0/2132366851 >> 172.21.15.121:0/2132366851 conn(0x55c50d600080 msgr2=0x55c50d5b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:38.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.883+0000 7f656f980d00 1 -- 172.21.15.121:0/2132366851 shutdown_connections 2023-11-15T03:08:38.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.883+0000 7f656f980d00 1 -- 172.21.15.121:0/2132366851 wait complete. 2023-11-15T03:08:38.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.883+0000 7f656f980d00 1 Processor -- start 2023-11-15T03:08:38.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.883+0000 7f656f980d00 1 -- start start 2023-11-15T03:08:38.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656f980d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c50d5f4fe0 0x55c50d5f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656f980d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c50d5f5950 0x55c50d688080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656f980d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c50d68c010 0x55c50d688570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656f980d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50d687bf0 con 0x55c50d68c010 2023-11-15T03:08:38.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656f980d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50d686fe0 con 0x55c50d5f4fe0 2023-11-15T03:08:38.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656f980d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c50d6872c0 con 0x55c50d5f5950 2023-11-15T03:08:38.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656dab8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c50d68c010 0x55c50d688570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656dab8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c50d68c010 0x55c50d688570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56798/0 (socket says 172.21.15.121:56798) 2023-11-15T03:08:38.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656dab8700 1 -- 172.21.15.121:0/474695786 learned_addr learned my addr 172.21.15.121:0/474695786 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:38.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656cab6700 1 --2- 172.21.15.121:0/474695786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c50d5f5950 0x55c50d688080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656d2b7700 1 --2- 172.21.15.121:0/474695786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c50d5f4fe0 0x55c50d5f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656dab8700 1 -- 172.21.15.121:0/474695786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c50d5f5950 msgr2=0x55c50d688080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656dab8700 1 --2- 172.21.15.121:0/474695786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c50d5f5950 0x55c50d688080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656dab8700 1 -- 172.21.15.121:0/474695786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c50d5f4fe0 msgr2=0x55c50d5f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656dab8700 1 --2- 172.21.15.121:0/474695786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c50d5f4fe0 0x55c50d5f7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656dab8700 1 -- 172.21.15.121:0/474695786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6568002400 con 0x55c50d68c010 2023-11-15T03:08:38.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656cab6700 1 --2- 172.21.15.121:0/474695786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c50d5f5950 0x55c50d688080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:38.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656d2b7700 1 --2- 172.21.15.121:0/474695786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c50d5f4fe0 0x55c50d5f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:38.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656dab8700 1 --2- 172.21.15.121:0/474695786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c50d68c010 0x55c50d688570 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 rx=0x7f6568003010 tx=0x7f6568019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:38.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f655dffb700 1 -- 172.21.15.121:0/474695786 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568013c50 con 0x55c50d68c010 2023-11-15T03:08:38.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f655dffb700 1 -- 172.21.15.121:0/474695786 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6568013db0 con 0x55c50d68c010 2023-11-15T03:08:38.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656f980d00 1 -- 172.21.15.121:0/474695786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c50d521a70 con 0x55c50d68c010 2023-11-15T03:08:38.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f655dffb700 1 -- 172.21.15.121:0/474695786 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656801d660 con 0x55c50d68c010 2023-11-15T03:08:38.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.887+0000 7f656f980d00 1 -- 172.21.15.121:0/474695786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c50d52d010 con 0x55c50d68c010 2023-11-15T03:08:38.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.891+0000 7f655dffb700 1 -- 172.21.15.121:0/474695786 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6568003040 con 0x55c50d68c010 2023-11-15T03:08:38.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.891+0000 7f655dffb700 1 --2- 172.21.15.121:0/474695786 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6550058ff0 0x7f655005b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.891+0000 7f655dffb700 1 -- 172.21.15.121:0/474695786 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8674+0+0 (secure 0 0 0) 0x7f65680754d0 con 0x55c50d68c010 2023-11-15T03:08:38.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.891+0000 7f656f980d00 1 --2- 172.21.15.121:0/474695786 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c50d5f7af0 0x55c50d5dfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:38.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.891+0000 7f656d2b7700 1 --2- 172.21.15.121:0/474695786 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6550058ff0 0x7f655005b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.891+0000 7f656f980d00 1 -- 172.21.15.121:0/474695786 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x55c50d5e0160 con 0x55c50d5f7af0 2023-11-15T03:08:38.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.895+0000 7f656cab6700 1 --2- 172.21.15.121:0/474695786 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c50d5f7af0 0x55c50d5dfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:38.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.895+0000 7f656d2b7700 1 --2- 172.21.15.121:0/474695786 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6550058ff0 0x7f655005b4b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f6560000f60 tx=0x7f6560006040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:38.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.895+0000 7f656cab6700 1 --2- 172.21.15.121:0/474695786 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c50d5f7af0 0x55c50d5dfc70 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:38.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.899+0000 7f656cab6700 1 -- 172.21.15.121:0/474695786 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 mds0_inotable [truncate 0] v122'143 uv143 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6564008560 con 0x55c50d5f7af0 2023-11-15T03:08:38.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.899+0000 7f656f980d00 1 -- 172.21.15.121:0/474695786 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c50d5f7af0 msgr2=0x55c50d5dfc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.899+0000 7f656f980d00 1 --2- 172.21.15.121:0/474695786 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55c50d5f7af0 0x55c50d5dfc70 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:38.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.899+0000 7f656f980d00 1 -- 172.21.15.121:0/474695786 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6550058ff0 msgr2=0x7f655005b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.899+0000 7f656f980d00 1 --2- 172.21.15.121:0/474695786 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6550058ff0 0x7f655005b4b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f6560000f60 tx=0x7f6560006040).stop 2023-11-15T03:08:38.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.899+0000 7f656f980d00 1 -- 172.21.15.121:0/474695786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c50d68c010 msgr2=0x55c50d688570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:38.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.899+0000 7f656f980d00 1 --2- 172.21.15.121:0/474695786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c50d68c010 0x55c50d688570 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 rx=0x7f6568003010 tx=0x7f6568019000).stop 2023-11-15T03:08:38.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.899+0000 7f656dab8700 1 -- 172.21.15.121:0/474695786 reap_dead start 2023-11-15T03:08:38.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.899+0000 7f656f980d00 1 -- 172.21.15.121:0/474695786 shutdown_connections 2023-11-15T03:08:38.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.899+0000 7f656f980d00 1 -- 172.21.15.121:0/474695786 >> 172.21.15.121:0/474695786 conn(0x55c50d600080 msgr2=0x55c50d5f5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:38.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.899+0000 7f656f980d00 1 -- 172.21.15.121:0/474695786 shutdown_connections 2023-11-15T03:08:38.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:38.899+0000 7f656f980d00 1 -- 172.21.15.121:0/474695786 wait complete. 2023-11-15T03:08:38.920 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:08:39.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.091+0000 7fed6041b700 1 Processor -- start 2023-11-15T03:08:39.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.091+0000 7fed6041b700 1 -- start start 2023-11-15T03:08:39.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.095+0000 7fed6041b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed580a6b80 0x7fed580bd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:39.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.095+0000 7fed6041b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fed580a6f60 0x7fed580bd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:39.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.095+0000 7fed6041b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fed580c6130 0x7fed580c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:39.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.095+0000 7fed6041b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed580bdae0 con 0x7fed580a6f60 2023-11-15T03:08:39.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.095+0000 7fed6041b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed58150f00 con 0x7fed580c6130 2023-11-15T03:08:39.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.095+0000 7fed6041b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed580c2730 con 0x7fed580a6b80 2023-11-15T03:08:39.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.095+0000 7fed5d9b6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fed580a6f60 0x7fed580bd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:39.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.095+0000 7fed5e9b8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fed580c6130 0x7fed580c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:39.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.095+0000 7fed5d9b6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fed580a6f60 0x7fed580bd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56812/0 (socket says 172.21.15.121:56812) 2023-11-15T03:08:39.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.095+0000 7fed5d9b6700 1 -- 172.21.15.121:0/3683592740 learned_addr learned my addr 172.21.15.121:0/3683592740 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:39.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.095+0000 7fed5e1b7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed580a6b80 0x7fed580bd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:39.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.095+0000 7fed5d9b6700 1 -- 172.21.15.121:0/3683592740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed580a6b80 msgr2=0x7fed580bd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:39.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.095+0000 7fed5d9b6700 1 --2- 172.21.15.121:0/3683592740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed580a6b80 0x7fed580bd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:39.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.099+0000 7fed5d9b6700 1 -- 172.21.15.121:0/3683592740 >> v2:172.21.15.190:3300/0 conn(0x7fed580c6130 msgr2=0x7fed580c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:39.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.099+0000 7fed5d9b6700 1 --2- 172.21.15.121:0/3683592740 >> v2:172.21.15.190:3300/0 conn(0x7fed580c6130 0x7fed580c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:39.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.099+0000 7fed5d9b6700 1 -- 172.21.15.121:0/3683592740 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed580c75f0 con 0x7fed580a6f60 2023-11-15T03:08:39.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.099+0000 7fed5e9b8700 1 --2- 172.21.15.121:0/3683592740 >> v2:172.21.15.190:3300/0 conn(0x7fed580c6130 0x7fed580c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:08:39.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.099+0000 7fed5d9b6700 1 --2- 172.21.15.121:0/3683592740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed580a6f60 0x7fed580bd5f0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 rx=0x7fed580623e0 tx=0x7fed4800b3b0).ready entity=mon.0 client_cookie=461ede6e90c025f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:39.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.099+0000 7fed5d1b5700 1 -- 172.21.15.121:0/3683592740 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4800da30 con 0x7fed580a6f60 2023-11-15T03:08:39.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.099+0000 7fed5d1b5700 1 -- 172.21.15.121:0/3683592740 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4800db90 con 0x7fed580a6f60 2023-11-15T03:08:39.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.099+0000 7fed5d1b5700 1 -- 172.21.15.121:0/3683592740 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48010470 con 0x7fed580a6f60 2023-11-15T03:08:39.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.099+0000 7fed6041b700 1 -- 172.21.15.121:0/3683592740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed580a6f60 msgr2=0x7fed580bd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:39.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.099+0000 7fed6041b700 1 --2- 172.21.15.121:0/3683592740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed580a6f60 0x7fed580bd5f0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 rx=0x7fed580623e0 tx=0x7fed4800b3b0).stop 2023-11-15T03:08:39.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.099+0000 7fed6041b700 1 -- 172.21.15.121:0/3683592740 shutdown_connections 2023-11-15T03:08:39.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.099+0000 7fed6041b700 1 --2- 172.21.15.121:0/3683592740 >> v2:172.21.15.190:3300/0 conn(0x7fed580c6130 0x7fed580c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:39.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.099+0000 7fed6041b700 1 --2- 172.21.15.121:0/3683592740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed580a6f60 0x7fed580bd5f0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:39.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.099+0000 7fed6041b700 1 --2- 172.21.15.121:0/3683592740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed580a6b80 0x7fed580bd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:39.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.099+0000 7fed6041b700 1 -- 172.21.15.121:0/3683592740 >> 172.21.15.121:0/3683592740 conn(0x7fed58066f50 msgr2=0x7fed58090f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:39.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.099+0000 7fed6041b700 1 -- 172.21.15.121:0/3683592740 shutdown_connections 2023-11-15T03:08:39.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.099+0000 7fed6041b700 1 -- 172.21.15.121:0/3683592740 wait complete. 2023-11-15T03:08:39.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed6041b700 1 Processor -- start 2023-11-15T03:08:39.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed6041b700 1 -- start start 2023-11-15T03:08:39.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed6041b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed580a6b80 0x7fed5814b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:39.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed6041b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed580a6f60 0x7fed580acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:39.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed6041b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed580c6130 0x7fed580ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:39.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed6041b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed580c1e00 con 0x7fed580a6f60 2023-11-15T03:08:39.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed6041b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed580ad840 con 0x7fed580c6130 2023-11-15T03:08:39.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed6041b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed5814b760 con 0x7fed580a6b80 2023-11-15T03:08:39.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed5d9b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed580a6f60 0x7fed580acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:39.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed5d9b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed580a6f60 0x7fed580acd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56822/0 (socket says 172.21.15.121:56822) 2023-11-15T03:08:39.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed5d9b6700 1 -- 172.21.15.121:0/210759810 learned_addr learned my addr 172.21.15.121:0/210759810 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:39.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed5e1b7700 1 --2- 172.21.15.121:0/210759810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed580a6b80 0x7fed5814b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:39.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed5e9b8700 1 --2- 172.21.15.121:0/210759810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed580c6130 0x7fed580ab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:39.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed5d9b6700 1 -- 172.21.15.121:0/210759810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed580a6b80 msgr2=0x7fed5814b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:39.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed5d9b6700 1 --2- 172.21.15.121:0/210759810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed580a6b80 0x7fed5814b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:39.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed5d9b6700 1 -- 172.21.15.121:0/210759810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed580c6130 msgr2=0x7fed580ab3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:39.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed5d9b6700 1 --2- 172.21.15.121:0/210759810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed580c6130 0x7fed580ab3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:39.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed5d9b6700 1 -- 172.21.15.121:0/210759810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed50009a20 con 0x7fed580a6f60 2023-11-15T03:08:39.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed5e9b8700 1 --2- 172.21.15.121:0/210759810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed580c6130 0x7fed580ab3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:39.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed5e1b7700 1 --2- 172.21.15.121:0/210759810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed580a6b80 0x7fed5814b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:39.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.103+0000 7fed5d9b6700 1 --2- 172.21.15.121:0/210759810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed580a6f60 0x7fed580acd80 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 rx=0x7fed58066140 tx=0x7fed4800b3b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:39.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.107+0000 7fed4e7fc700 1 -- 172.21.15.121:0/210759810 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48006d90 con 0x7fed580a6f60 2023-11-15T03:08:39.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.107+0000 7fed4e7fc700 1 -- 172.21.15.121:0/210759810 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed48019450 con 0x7fed580a6f60 2023-11-15T03:08:39.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.107+0000 7fed6041b700 1 -- 172.21.15.121:0/210759810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4800b610 con 0x7fed580a6f60 2023-11-15T03:08:39.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.107+0000 7fed4e7fc700 1 -- 172.21.15.121:0/210759810 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48019810 con 0x7fed580a6f60 2023-11-15T03:08:39.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.107+0000 7fed6041b700 1 -- 172.21.15.121:0/210759810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed58164750 con 0x7fed580a6f60 2023-11-15T03:08:39.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.107+0000 7fed6041b700 1 -- 172.21.15.121:0/210759810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed58047060 con 0x7fed580a6f60 2023-11-15T03:08:39.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.107+0000 7fed4e7fc700 1 -- 172.21.15.121:0/210759810 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fed48019c00 con 0x7fed580a6f60 2023-11-15T03:08:39.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.111+0000 7fed4e7fc700 1 --2- 172.21.15.121:0/210759810 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fed3c05a540 0x7fed3c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:39.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.111+0000 7fed4e7fc700 1 -- 172.21.15.121:0/210759810 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8674+0+0 (secure 0 0 0) 0x7fed48076950 con 0x7fed580a6f60 2023-11-15T03:08:39.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.111+0000 7fed5e1b7700 1 --2- 172.21.15.121:0/210759810 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fed3c05a540 0x7fed3c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:39.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.111+0000 7fed5e1b7700 1 --2- 172.21.15.121:0/210759810 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fed3c05a540 0x7fed3c05ca00 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7fed580acb70 tx=0x7fed54008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:39.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.115+0000 7fed4e7fc700 1 -- 172.21.15.121:0/210759810 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fed480496f0 con 0x7fed580a6f60 2023-11-15T03:08:39.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:39.311+0000 7fed6041b700 1 -- 172.21.15.121:0/210759810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fed58059020 con 0x7fed580a6f60 2023-11-15T03:08:40.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.303+0000 7fed4e7fc700 1 -- 172.21.15.121:0/210759810 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v257) v1 ==== 162+0+0 (secure 0 0 0) 0x7fed48028030 con 0x7fed580a6f60 2023-11-15T03:08:40.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.307+0000 7fed6041b700 1 -- 172.21.15.121:0/210759810 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fed3c05a540 msgr2=0x7fed3c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:40.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.307+0000 7fed6041b700 1 --2- 172.21.15.121:0/210759810 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fed3c05a540 0x7fed3c05ca00 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7fed580acb70 tx=0x7fed54008040).stop 2023-11-15T03:08:40.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.307+0000 7fed6041b700 1 -- 172.21.15.121:0/210759810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed580a6f60 msgr2=0x7fed580acd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:40.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.307+0000 7fed6041b700 1 --2- 172.21.15.121:0/210759810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed580a6f60 0x7fed580acd80 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 rx=0x7fed58066140 tx=0x7fed4800b3b0).stop 2023-11-15T03:08:40.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.307+0000 7fed6041b700 1 -- 172.21.15.121:0/210759810 shutdown_connections 2023-11-15T03:08:40.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.307+0000 7fed6041b700 1 --2- 172.21.15.121:0/210759810 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fed3c05a540 0x7fed3c05ca00 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:40.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.307+0000 7fed6041b700 1 --2- 172.21.15.121:0/210759810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed580c6130 0x7fed580ab3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:40.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.307+0000 7fed6041b700 1 --2- 172.21.15.121:0/210759810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed580a6f60 0x7fed580acd80 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:40.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.307+0000 7fed6041b700 1 --2- 172.21.15.121:0/210759810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed580a6b80 0x7fed5814b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:40.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.307+0000 7fed6041b700 1 -- 172.21.15.121:0/210759810 >> 172.21.15.121:0/210759810 conn(0x7fed58066f50 msgr2=0x7fed580a4630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:40.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.307+0000 7fed6041b700 1 -- 172.21.15.121:0/210759810 shutdown_connections 2023-11-15T03:08:40.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.307+0000 7fed6041b700 1 -- 172.21.15.121:0/210759810 wait complete. 2023-11-15T03:08:40.319 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:08:40.325 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:08:40.315+0000 7fb41ab46700 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': End of buffer 2023-11-15T03:08:40.339 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:40.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.495+0000 7fef8b59e700 1 Processor -- start 2023-11-15T03:08:40.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.495+0000 7fef8b59e700 1 -- start start 2023-11-15T03:08:40.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.495+0000 7fef8b59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fef8c066700 0x7fef8c154110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:40.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.495+0000 7fef8b59e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef8c0c73e0 con 0x7fef8c0acf10 2023-11-15T03:08:40.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.495+0000 7fef8b59e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef8c1546c0 con 0x7fef8c066320 2023-11-15T03:08:40.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.495+0000 7fef8b59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef8c1549a0 con 0x7fef8c066700 2023-11-15T03:08:40.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.499+0000 7fef8ad9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fef8c0acf10 0x7fef8c062190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:52312/0 (socket says 172.21.15.121:52312) 2023-11-15T03:08:40.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.499+0000 7fef8ad9d700 1 -- 172.21.15.121:0/941963254 learned_addr learned my addr 172.21.15.121:0/941963254 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:40.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.499+0000 7fef81d9b700 1 --2- 172.21.15.121:0/941963254 >> v2:172.21.15.121:3300/0 conn(0x7fef8c066700 0x7fef8c154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:40.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.499+0000 7fef89d9b700 1 -- 172.21.15.121:0/941963254 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3421106780 0 0) 0x7fef8c0c73e0 con 0x7fef8c0acf10 2023-11-15T03:08:40.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.499+0000 7fef81d9b700 1 -- 172.21.15.121:0/941963254 >> v1:172.21.15.190:6789/0 conn(0x7fef8c066320 legacy=0x7fef8c0656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:40.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.499+0000 7fef81d9b700 1 -- 172.21.15.121:0/941963254 >> v1:172.21.15.121:6789/0 conn(0x7fef8c0acf10 legacy=0x7fef8c062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:40.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.499+0000 7fef81d9b700 1 -- 172.21.15.121:0/941963254 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c154c80 con 0x7fef8c066700 2023-11-15T03:08:40.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.499+0000 7fef81d9b700 1 --2- 172.21.15.121:0/941963254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef8c066700 0x7fef8c154110 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 rx=0x7fef74009760 tx=0x7fef74009060).ready entity=mon.0 client_cookie=5f0f9384f2ff1b8d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:40.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.499+0000 7fef89d9b700 1 -- 172.21.15.121:0/941963254 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7400b580 con 0x7fef8c066700 2023-11-15T03:08:40.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.499+0000 7fef89d9b700 1 -- 172.21.15.121:0/941963254 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7400b6e0 con 0x7fef8c066700 2023-11-15T03:08:40.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.499+0000 7fef89d9b700 1 -- 172.21.15.121:0/941963254 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7400ba60 con 0x7fef8c066700 2023-11-15T03:08:40.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.499+0000 7fef8b59e700 1 -- 172.21.15.121:0/941963254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef8c066700 msgr2=0x7fef8c154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:40.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.499+0000 7fef8b59e700 1 --2- 172.21.15.121:0/941963254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef8c066700 0x7fef8c154110 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 rx=0x7fef74009760 tx=0x7fef74009060).stop 2023-11-15T03:08:40.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.499+0000 7fef8b59e700 1 -- 172.21.15.121:0/941963254 shutdown_connections 2023-11-15T03:08:40.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.499+0000 7fef8b59e700 1 --2- 172.21.15.121:0/941963254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef8c066700 0x7fef8c154110 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:40.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.499+0000 7fef8b59e700 1 -- 172.21.15.121:0/941963254 >> 172.21.15.121:0/941963254 conn(0x7fef8c067a60 msgr2=0x7fef8c067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:40.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.499+0000 7fef8b59e700 1 -- 172.21.15.121:0/941963254 shutdown_connections 2023-11-15T03:08:40.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.499+0000 7fef8b59e700 1 -- 172.21.15.121:0/941963254 wait complete. 2023-11-15T03:08:40.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.503+0000 7fef8b59e700 1 Processor -- start 2023-11-15T03:08:40.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.503+0000 7fef8b59e700 1 -- start start 2023-11-15T03:08:40.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef8b59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef8c066320 0x7fef8c147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:40.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef8b59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef8c066700 0x7fef8c1485b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:40.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef8b59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef8c0acf10 0x7fef8c14a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:40.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef8b59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0afcb0 con 0x7fef8c0acf10 2023-11-15T03:08:40.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef8b59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0aff90 con 0x7fef8c066700 2023-11-15T03:08:40.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef8b59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0b0270 con 0x7fef8c066320 2023-11-15T03:08:40.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef8ad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef8c0acf10 0x7fef8c14a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:40.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef8ad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef8c0acf10 0x7fef8c14a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56846/0 (socket says 172.21.15.121:56846) 2023-11-15T03:08:40.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef8ad9d700 1 -- 172.21.15.121:0/3223201138 learned_addr learned my addr 172.21.15.121:0/3223201138 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:40.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef81d9b700 1 --2- 172.21.15.121:0/3223201138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef8c066700 0x7fef8c1485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:40.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef8a59c700 1 --2- 172.21.15.121:0/3223201138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef8c066320 0x7fef8c147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:40.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef8ad9d700 1 -- 172.21.15.121:0/3223201138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef8c066320 msgr2=0x7fef8c147500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:40.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef8ad9d700 1 --2- 172.21.15.121:0/3223201138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef8c066320 0x7fef8c147500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:40.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef8ad9d700 1 -- 172.21.15.121:0/3223201138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef8c066700 msgr2=0x7fef8c1485b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:40.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef8ad9d700 1 --2- 172.21.15.121:0/3223201138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef8c066700 0x7fef8c1485b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:40.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef8ad9d700 1 -- 172.21.15.121:0/3223201138 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef740092a0 con 0x7fef8c0acf10 2023-11-15T03:08:40.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef8ad9d700 1 --2- 172.21.15.121:0/3223201138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef8c0acf10 0x7fef8c14a1d0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 rx=0x7fef8400c810 tx=0x7fef840014d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:40.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef82ffd700 1 -- 172.21.15.121:0/3223201138 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84005d30 con 0x7fef8c0acf10 2023-11-15T03:08:40.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef82ffd700 1 -- 172.21.15.121:0/3223201138 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef84005e90 con 0x7fef8c0acf10 2023-11-15T03:08:40.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef82ffd700 1 -- 172.21.15.121:0/3223201138 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840072e0 con 0x7fef8c0acf10 2023-11-15T03:08:40.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef8b59e700 1 -- 172.21.15.121:0/3223201138 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8c0b0550 con 0x7fef8c0acf10 2023-11-15T03:08:40.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.507+0000 7fef8b59e700 1 -- 172.21.15.121:0/3223201138 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8c1479f0 con 0x7fef8c0acf10 2023-11-15T03:08:40.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.511+0000 7fef8b59e700 1 -- 172.21.15.121:0/3223201138 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef8c14cad0 con 0x7fef8c0acf10 2023-11-15T03:08:40.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.511+0000 7fef82ffd700 1 -- 172.21.15.121:0/3223201138 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fef84007710 con 0x7fef8c0acf10 2023-11-15T03:08:40.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.511+0000 7fef82ffd700 1 --2- 172.21.15.121:0/3223201138 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef6c05a510 0x7fef6c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:40.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.511+0000 7fef82ffd700 1 -- 172.21.15.121:0/3223201138 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8760+0+0 (secure 0 0 0) 0x7fef84074b40 con 0x7fef8c0acf10 2023-11-15T03:08:40.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.511+0000 7fef8a59c700 1 --2- 172.21.15.121:0/3223201138 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef6c05a510 0x7fef6c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:40.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.515+0000 7fef8a59c700 1 --2- 172.21.15.121:0/3223201138 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef6c05a510 0x7fef6c05c9d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7fef7c003900 tx=0x7fef7c007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:40.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.519+0000 7fef82ffd700 1 -- 172.21.15.121:0/3223201138 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fef84047890 con 0x7fef8c0acf10 2023-11-15T03:08:40.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.711+0000 7fef8b59e700 1 -- 172.21.15.121:0/3223201138 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef8c06bcf0 con 0x7fef8c0acf10 2023-11-15T03:08:40.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.715+0000 7fef82ffd700 1 -- 172.21.15.121:0/3223201138 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 258 v258) v1 ==== 95+0+4505 (secure 0 0 0) 0x7fef8404aeb0 con 0x7fef8c0acf10 2023-11-15T03:08:40.723 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:40.723 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":258,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":258,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:08:40.308342+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9582},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9582":{"gid":9582,"name":"a","rank":0,"incarnation":258,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/1978069160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1978069160},{"type":"v1","addr":"172.21.15.121:6836","nonce":1978069160}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:40.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.715+0000 7fef8b59e700 1 -- 172.21.15.121:0/3223201138 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef6c05a510 msgr2=0x7fef6c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:40.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.715+0000 7fef8b59e700 1 --2- 172.21.15.121:0/3223201138 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef6c05a510 0x7fef6c05c9d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7fef7c003900 tx=0x7fef7c007d40).stop 2023-11-15T03:08:40.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.715+0000 7fef8b59e700 1 -- 172.21.15.121:0/3223201138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef8c0acf10 msgr2=0x7fef8c14a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:40.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.715+0000 7fef8b59e700 1 --2- 172.21.15.121:0/3223201138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef8c0acf10 0x7fef8c14a1d0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 rx=0x7fef8400c810 tx=0x7fef840014d0).stop 2023-11-15T03:08:40.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.715+0000 7fef8b59e700 1 -- 172.21.15.121:0/3223201138 shutdown_connections 2023-11-15T03:08:40.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.715+0000 7fef8b59e700 1 --2- 172.21.15.121:0/3223201138 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef6c05a510 0x7fef6c05c9d0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:40.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.715+0000 7fef8b59e700 1 --2- 172.21.15.121:0/3223201138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef8c0acf10 0x7fef8c14a1d0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:40.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.715+0000 7fef8b59e700 1 --2- 172.21.15.121:0/3223201138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef8c066700 0x7fef8c1485b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:40.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.715+0000 7fef8b59e700 1 --2- 172.21.15.121:0/3223201138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef8c066320 0x7fef8c147500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:40.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.715+0000 7fef8b59e700 1 -- 172.21.15.121:0/3223201138 >> 172.21.15.121:0/3223201138 conn(0x7fef8c067a60 msgr2=0x7fef8c0aaba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:40.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.715+0000 7fef8b59e700 1 -- 172.21.15.121:0/3223201138 shutdown_connections 2023-11-15T03:08:40.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:40.715+0000 7fef8b59e700 1 -- 172.21.15.121:0/3223201138 wait complete. 2023-11-15T03:08:40.729 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 258 2023-11-15T03:08:40.747 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:08:41.320 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:08:41.311+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:08:41.320 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:08:41.311+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:08:41.332 INFO:tasks.ceph.mds.a.smithi121.stderr: -30> 2023-11-15T03:08:40.315+0000 7fb41ab46700 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': End of buffer 2023-11-15T03:08:41.373 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:08:42.153 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:08:42.161 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:08:45.748 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:45.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.907+0000 7ff535f47700 1 Processor -- start 2023-11-15T03:08:45.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.907+0000 7ff535f47700 1 -- start start 2023-11-15T03:08:45.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.911+0000 7ff535f47700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff530151870 0x7ff530151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:45.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.911+0000 7ff535f47700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff530152140 0x7ff5300a9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:45.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.911+0000 7ff535f47700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff5300a9d20 0x7ff5300c30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:45.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.911+0000 7ff535f47700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5300657f0 con 0x7ff530152140 2023-11-15T03:08:45.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.911+0000 7ff535f47700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff530065ad0 con 0x7ff5300a9d20 2023-11-15T03:08:45.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.911+0000 7ff535f47700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5300aa260 con 0x7ff530151870 2023-11-15T03:08:45.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.911+0000 7ff52ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff530152140 0x7ff5300a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:45.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.911+0000 7ff52ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff530152140 0x7ff5300a9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36464/0 (socket says 172.21.15.121:36464) 2023-11-15T03:08:45.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.911+0000 7ff52ffff700 1 -- 172.21.15.121:0/3758630257 learned_addr learned my addr 172.21.15.121:0/3758630257 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:45.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.911+0000 7ff534f45700 1 --2- 172.21.15.121:0/3758630257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff530151870 0x7ff530151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:45.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.911+0000 7ff535746700 1 --2- 172.21.15.121:0/3758630257 >> v2:172.21.15.190:3300/0 conn(0x7ff5300a9d20 0x7ff5300c30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:45.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.915+0000 7ff52ffff700 1 -- 172.21.15.121:0/3758630257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff530151870 msgr2=0x7ff530151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:45.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.915+0000 7ff52ffff700 1 --2- 172.21.15.121:0/3758630257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff530151870 0x7ff530151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:45.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.915+0000 7ff52ffff700 1 -- 172.21.15.121:0/3758630257 >> v2:172.21.15.190:3300/0 conn(0x7ff5300a9d20 msgr2=0x7ff5300c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:45.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.915+0000 7ff52ffff700 1 --2- 172.21.15.121:0/3758630257 >> v2:172.21.15.190:3300/0 conn(0x7ff5300a9d20 0x7ff5300c30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:45.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.915+0000 7ff52ffff700 1 -- 172.21.15.121:0/3758630257 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5300aa540 con 0x7ff530152140 2023-11-15T03:08:45.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.915+0000 7ff535746700 1 --2- 172.21.15.121:0/3758630257 >> v2:172.21.15.190:3300/0 conn(0x7ff5300a9d20 0x7ff5300c30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:45.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.915+0000 7ff52ffff700 1 --2- 172.21.15.121:0/3758630257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff530152140 0x7ff5300a9830 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 rx=0x7ff53006c380 tx=0x7ff51c0090f0).ready entity=mon.0 client_cookie=9764b385cafbdd92 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:45.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.915+0000 7ff52f7fe700 1 -- 172.21.15.121:0/3758630257 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c00b520 con 0x7ff530152140 2023-11-15T03:08:45.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.915+0000 7ff52f7fe700 1 -- 172.21.15.121:0/3758630257 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51c00b680 con 0x7ff530152140 2023-11-15T03:08:45.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.915+0000 7ff52f7fe700 1 -- 172.21.15.121:0/3758630257 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c00ba00 con 0x7ff530152140 2023-11-15T03:08:45.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.915+0000 7ff535f47700 1 -- 172.21.15.121:0/3758630257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff530152140 msgr2=0x7ff5300a9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:45.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.915+0000 7ff535f47700 1 --2- 172.21.15.121:0/3758630257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff530152140 0x7ff5300a9830 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 rx=0x7ff53006c380 tx=0x7ff51c0090f0).stop 2023-11-15T03:08:45.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.915+0000 7ff535f47700 1 -- 172.21.15.121:0/3758630257 shutdown_connections 2023-11-15T03:08:45.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.915+0000 7ff535f47700 1 --2- 172.21.15.121:0/3758630257 >> v2:172.21.15.190:3300/0 conn(0x7ff5300a9d20 0x7ff5300c30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:45.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.915+0000 7ff535f47700 1 --2- 172.21.15.121:0/3758630257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff530152140 0x7ff5300a9830 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:45.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.915+0000 7ff535f47700 1 --2- 172.21.15.121:0/3758630257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff530151870 0x7ff530151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:45.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.915+0000 7ff535f47700 1 -- 172.21.15.121:0/3758630257 >> 172.21.15.121:0/3758630257 conn(0x7ff530062190 msgr2=0x7ff530062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:45.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.915+0000 7ff535f47700 1 -- 172.21.15.121:0/3758630257 shutdown_connections 2023-11-15T03:08:45.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.915+0000 7ff535f47700 1 -- 172.21.15.121:0/3758630257 wait complete. 2023-11-15T03:08:45.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.919+0000 7ff535f47700 1 Processor -- start 2023-11-15T03:08:45.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.919+0000 7ff535f47700 1 -- start start 2023-11-15T03:08:45.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.919+0000 7ff535f47700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5300a9d20 0x7ff530143ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:45.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.919+0000 7ff535f47700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff530151870 0x7ff5300c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:45.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.919+0000 7ff535f47700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff530152140 0x7ff53014cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:45.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.919+0000 7ff535f47700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5300c5020 con 0x7ff530152140 2023-11-15T03:08:45.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.919+0000 7ff535f47700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5300c6180 con 0x7ff5300a9d20 2023-11-15T03:08:45.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.919+0000 7ff535f47700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff530144190 con 0x7ff530151870 2023-11-15T03:08:45.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.919+0000 7ff535746700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff530152140 0x7ff53014cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:45.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.919+0000 7ff535746700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff530152140 0x7ff53014cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36468/0 (socket says 172.21.15.121:36468) 2023-11-15T03:08:45.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.919+0000 7ff535746700 1 -- 172.21.15.121:0/183974020 learned_addr learned my addr 172.21.15.121:0/183974020 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:45.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.919+0000 7ff534f45700 1 --2- 172.21.15.121:0/183974020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5300a9d20 0x7ff530143ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:45.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.919+0000 7ff52ffff700 1 --2- 172.21.15.121:0/183974020 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff530151870 0x7ff5300c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:45.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.923+0000 7ff535746700 1 -- 172.21.15.121:0/183974020 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff530151870 msgr2=0x7ff5300c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:45.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.923+0000 7ff535746700 1 --2- 172.21.15.121:0/183974020 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff530151870 0x7ff5300c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:45.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.923+0000 7ff535746700 1 -- 172.21.15.121:0/183974020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5300a9d20 msgr2=0x7ff530143ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:45.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.923+0000 7ff535746700 1 --2- 172.21.15.121:0/183974020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5300a9d20 0x7ff530143ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:45.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.923+0000 7ff535746700 1 -- 172.21.15.121:0/183974020 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c009370 con 0x7ff530152140 2023-11-15T03:08:45.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.923+0000 7ff52ffff700 1 --2- 172.21.15.121:0/183974020 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff530151870 0x7ff5300c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:45.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.923+0000 7ff535746700 1 --2- 172.21.15.121:0/183974020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff530152140 0x7ff53014cbf0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 rx=0x7ff528009fc0 tx=0x7ff5280099b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:45.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.923+0000 7ff52cff9700 1 -- 172.21.15.121:0/183974020 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52800dc90 con 0x7ff530152140 2023-11-15T03:08:45.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.923+0000 7ff52cff9700 1 -- 172.21.15.121:0/183974020 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52800f040 con 0x7ff530152140 2023-11-15T03:08:45.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.923+0000 7ff52cff9700 1 -- 172.21.15.121:0/183974020 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5280133f0 con 0x7ff530152140 2023-11-15T03:08:45.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.923+0000 7ff535f47700 1 -- 172.21.15.121:0/183974020 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53014d5d0 con 0x7ff530152140 2023-11-15T03:08:45.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.923+0000 7ff535f47700 1 -- 172.21.15.121:0/183974020 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5300c5e10 con 0x7ff530152140 2023-11-15T03:08:45.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.923+0000 7ff535f47700 1 -- 172.21.15.121:0/183974020 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5300c6310 con 0x7ff530152140 2023-11-15T03:08:45.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.927+0000 7ff52cff9700 1 -- 172.21.15.121:0/183974020 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff5280137a0 con 0x7ff530152140 2023-11-15T03:08:45.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.927+0000 7ff52cff9700 1 --2- 172.21.15.121:0/183974020 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff51405a510 0x7ff51405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:45.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.927+0000 7ff52cff9700 1 -- 172.21.15.121:0/183974020 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8760+0+0 (secure 0 0 0) 0x7ff5280758d0 con 0x7ff530152140 2023-11-15T03:08:45.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.927+0000 7ff534f45700 1 --2- 172.21.15.121:0/183974020 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff51405a510 0x7ff51405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:45.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.927+0000 7ff534f45700 1 --2- 172.21.15.121:0/183974020 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff51405a510 0x7ff51405c9d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7ff524009ba0 tx=0x7ff524009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:45.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:45.935+0000 7ff52cff9700 1 -- 172.21.15.121:0/183974020 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff5280485a0 con 0x7ff530152140 2023-11-15T03:08:46.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.131+0000 7ff535f47700 1 -- 172.21.15.121:0/183974020 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff53006bcf0 con 0x7ff530152140 2023-11-15T03:08:46.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.131+0000 7ff52cff9700 1 -- 172.21.15.121:0/183974020 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 260 v260) v1 ==== 95+0+4496 (secure 0 0 0) 0x7ff52804bbc0 con 0x7ff530152140 2023-11-15T03:08:46.140 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:46.140 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":260,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3050548459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3050548459},{"type":"v1","addr":"172.21.15.121:6836","nonce":3050548459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":260}],"filesystems":[{"mdsmap":{"epoch":259,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:08:41.315469+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:46.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.135+0000 7ff535f47700 1 -- 172.21.15.121:0/183974020 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff51405a510 msgr2=0x7ff51405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:46.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.135+0000 7ff535f47700 1 --2- 172.21.15.121:0/183974020 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff51405a510 0x7ff51405c9d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7ff524009ba0 tx=0x7ff524009060).stop 2023-11-15T03:08:46.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.135+0000 7ff535f47700 1 -- 172.21.15.121:0/183974020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff530152140 msgr2=0x7ff53014cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:46.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.135+0000 7ff535f47700 1 --2- 172.21.15.121:0/183974020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff530152140 0x7ff53014cbf0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 rx=0x7ff528009fc0 tx=0x7ff5280099b0).stop 2023-11-15T03:08:46.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.135+0000 7ff535f47700 1 -- 172.21.15.121:0/183974020 shutdown_connections 2023-11-15T03:08:46.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.135+0000 7ff535f47700 1 --2- 172.21.15.121:0/183974020 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff51405a510 0x7ff51405c9d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:46.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.135+0000 7ff535f47700 1 --2- 172.21.15.121:0/183974020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff530152140 0x7ff53014cbf0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:46.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.135+0000 7ff535f47700 1 --2- 172.21.15.121:0/183974020 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff530151870 0x7ff5300c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:46.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.135+0000 7ff535f47700 1 --2- 172.21.15.121:0/183974020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5300a9d20 0x7ff530143ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:46.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.135+0000 7ff535f47700 1 -- 172.21.15.121:0/183974020 >> 172.21.15.121:0/183974020 conn(0x7ff530062190 msgr2=0x7ff5300a8980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:46.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.135+0000 7ff535f47700 1 -- 172.21.15.121:0/183974020 shutdown_connections 2023-11-15T03:08:46.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.135+0000 7ff535f47700 1 -- 172.21.15.121:0/183974020 wait complete. 2023-11-15T03:08:46.147 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 260 2023-11-15T03:08:46.166 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:08:46.167 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:46.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.311+0000 7f3501907700 1 Processor -- start 2023-11-15T03:08:46.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.311+0000 7f3501907700 1 -- start start 2023-11-15T03:08:46.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.315+0000 7f3501907700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34fc0a87f0 0x7f34fc067150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:46.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.315+0000 7f3501907700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34fc0a9350 con 0x7f34fc0c29a0 2023-11-15T03:08:46.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.315+0000 7f3501907700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34fc151420 con 0x7f34fc066d70 2023-11-15T03:08:46.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.315+0000 7f3501907700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc150ed0 con 0x7f34fc0a87f0 2023-11-15T03:08:46.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.315+0000 7f3501106700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f34fc0c29a0 0x7f34fc14e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49982/0 (socket says 172.21.15.121:49982) 2023-11-15T03:08:46.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.315+0000 7f3501106700 1 -- 172.21.15.121:0/3424497279 learned_addr learned my addr 172.21.15.121:0/3424497279 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:46.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.315+0000 7f34f3fff700 1 --2- 172.21.15.121:0/3424497279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34fc0a87f0 0x7f34fc067150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:46.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.315+0000 7f34f37fe700 1 -- 172.21.15.121:0/3424497279 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3281022714 0 0) 0x7f34fc151420 con 0x7f34fc066d70 2023-11-15T03:08:46.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.315+0000 7f34f37fe700 1 -- 172.21.15.121:0/3424497279 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34e40033f0 con 0x7f34fc066d70 2023-11-15T03:08:46.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.315+0000 7f34f37fe700 1 -- 172.21.15.121:0/3424497279 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2932304484 0 0) 0x7f34fc0a9350 con 0x7f34fc0c29a0 2023-11-15T03:08:46.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.315+0000 7f34f37fe700 1 -- 172.21.15.121:0/3424497279 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34fc151420 con 0x7f34fc0c29a0 2023-11-15T03:08:46.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.315+0000 7f34f3fff700 1 -- 172.21.15.121:0/3424497279 >> v1:172.21.15.190:6789/0 conn(0x7f34fc066d70 legacy=0x7f34fc1518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:46.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.315+0000 7f34f3fff700 1 -- 172.21.15.121:0/3424497279 >> v1:172.21.15.121:6789/0 conn(0x7f34fc0c29a0 legacy=0x7f34fc14e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:46.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.315+0000 7f34f3fff700 1 -- 172.21.15.121:0/3424497279 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34fc0a8d90 con 0x7f34fc0a87f0 2023-11-15T03:08:46.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.319+0000 7f34f3fff700 1 --2- 172.21.15.121:0/3424497279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34fc0a87f0 0x7f34fc067150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f34ec009790 tx=0x7f34ec009060).ready entity=mon.2 client_cookie=8fd44a1b354c60e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:46.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.319+0000 7f34f37fe700 1 -- 172.21.15.121:0/3424497279 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec00f510 con 0x7f34fc0a87f0 2023-11-15T03:08:46.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.319+0000 7f34f37fe700 1 -- 172.21.15.121:0/3424497279 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ec00f670 con 0x7f34fc0a87f0 2023-11-15T03:08:46.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.319+0000 7f34f37fe700 1 -- 172.21.15.121:0/3424497279 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec00f9f0 con 0x7f34fc0a87f0 2023-11-15T03:08:46.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.319+0000 7f3501907700 1 -- 172.21.15.121:0/3424497279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34fc0a87f0 msgr2=0x7f34fc067150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:46.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.319+0000 7f3501907700 1 --2- 172.21.15.121:0/3424497279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34fc0a87f0 0x7f34fc067150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f34ec009790 tx=0x7f34ec009060).stop 2023-11-15T03:08:46.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.319+0000 7f3501907700 1 -- 172.21.15.121:0/3424497279 shutdown_connections 2023-11-15T03:08:46.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.319+0000 7f3501907700 1 --2- 172.21.15.121:0/3424497279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34fc0a87f0 0x7f34fc067150 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:46.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.319+0000 7f3501907700 1 -- 172.21.15.121:0/3424497279 >> 172.21.15.121:0/3424497279 conn(0x7f34fc062190 msgr2=0x7f34fc062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:46.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.319+0000 7f3501907700 1 -- 172.21.15.121:0/3424497279 shutdown_connections 2023-11-15T03:08:46.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.319+0000 7f3501907700 1 -- 172.21.15.121:0/3424497279 wait complete. 2023-11-15T03:08:46.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.323+0000 7f3501907700 1 Processor -- start 2023-11-15T03:08:46.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.323+0000 7f3501907700 1 -- start start 2023-11-15T03:08:46.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.323+0000 7f3501907700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34fc066d70 0x7f34fc13a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:46.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.323+0000 7f3501907700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34fc0a87f0 0x7f34fc0c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:46.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.323+0000 7f3501907700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34fc0c29a0 0x7f34fc148030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:46.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.323+0000 7f3501907700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc161e10 con 0x7f34fc0c29a0 2023-11-15T03:08:46.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.323+0000 7f3501907700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc162bf0 con 0x7f34fc0a87f0 2023-11-15T03:08:46.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.323+0000 7f3501907700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc162020 con 0x7f34fc066d70 2023-11-15T03:08:46.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.323+0000 7f3501106700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34fc0c29a0 0x7f34fc148030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:46.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.327+0000 7f3501106700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34fc0c29a0 0x7f34fc148030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36476/0 (socket says 172.21.15.121:36476) 2023-11-15T03:08:46.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.327+0000 7f3501106700 1 -- 172.21.15.121:0/1166407999 learned_addr learned my addr 172.21.15.121:0/1166407999 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:46.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.327+0000 7f3500905700 1 --2- 172.21.15.121:0/1166407999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34fc066d70 0x7f34fc13a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:46.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.327+0000 7f34f3fff700 1 --2- 172.21.15.121:0/1166407999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34fc0a87f0 0x7f34fc0c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:46.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.327+0000 7f3501106700 1 -- 172.21.15.121:0/1166407999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34fc066d70 msgr2=0x7f34fc13a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:46.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.327+0000 7f3501106700 1 --2- 172.21.15.121:0/1166407999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34fc066d70 0x7f34fc13a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:46.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.327+0000 7f3501106700 1 -- 172.21.15.121:0/1166407999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34fc0a87f0 msgr2=0x7f34fc0c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:46.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.327+0000 7f3501106700 1 --2- 172.21.15.121:0/1166407999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34fc0a87f0 0x7f34fc0c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:46.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.327+0000 7f3501106700 1 -- 172.21.15.121:0/1166407999 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec0092a0 con 0x7f34fc0c29a0 2023-11-15T03:08:46.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.327+0000 7f3501106700 1 --2- 172.21.15.121:0/1166407999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34fc0c29a0 0x7f34fc148030 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 rx=0x7f34f4005ad0 tx=0x7f34f40015b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:46.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.327+0000 7f34f0ff9700 1 -- 172.21.15.121:0/1166407999 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4005d30 con 0x7f34fc0c29a0 2023-11-15T03:08:46.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.327+0000 7f34f0ff9700 1 -- 172.21.15.121:0/1166407999 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f4005e90 con 0x7f34fc0c29a0 2023-11-15T03:08:46.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.327+0000 7f3501907700 1 -- 172.21.15.121:0/1166407999 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34fc13a620 con 0x7f34fc0c29a0 2023-11-15T03:08:46.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.327+0000 7f34f0ff9700 1 -- 172.21.15.121:0/1166407999 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f40072e0 con 0x7f34fc0c29a0 2023-11-15T03:08:46.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.327+0000 7f3501907700 1 -- 172.21.15.121:0/1166407999 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34fc1625a0 con 0x7f34fc0c29a0 2023-11-15T03:08:46.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.327+0000 7f3501907700 1 -- 172.21.15.121:0/1166407999 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34fc162d60 con 0x7f34fc0c29a0 2023-11-15T03:08:46.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.331+0000 7f34f0ff9700 1 -- 172.21.15.121:0/1166407999 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f34f40076d0 con 0x7f34fc0c29a0 2023-11-15T03:08:46.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.331+0000 7f34f0ff9700 1 --2- 172.21.15.121:0/1166407999 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f34e805a4e0 0x7f34e805c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:46.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.331+0000 7f34f0ff9700 1 -- 172.21.15.121:0/1166407999 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8760+0+0 (secure 0 0 0) 0x7f34f4074ae0 con 0x7f34fc0c29a0 2023-11-15T03:08:46.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.331+0000 7f3500905700 1 --2- 172.21.15.121:0/1166407999 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f34e805a4e0 0x7f34e805c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:46.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.335+0000 7f3500905700 1 --2- 172.21.15.121:0/1166407999 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f34e805a4e0 0x7f34e805c9a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f34f8003930 tx=0x7f34f8007bf0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:46.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.339+0000 7f34f0ff9700 1 -- 172.21.15.121:0/1166407999 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f34f40477d0 con 0x7f34fc0c29a0 2023-11-15T03:08:46.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.539+0000 7f3501907700 1 -- 172.21.15.121:0/1166407999 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34fc06bcf0 con 0x7f34fc0c29a0 2023-11-15T03:08:46.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.539+0000 7f34f0ff9700 1 -- 172.21.15.121:0/1166407999 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 260 v260) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f34f4018020 con 0x7f34fc0c29a0 2023-11-15T03:08:46.548 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:46.548 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":260,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3050548459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3050548459},{"type":"v1","addr":"172.21.15.121:6836","nonce":3050548459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":260}],"filesystems":[{"mdsmap":{"epoch":259,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:08:41.315469+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:46.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.543+0000 7f3501907700 1 -- 172.21.15.121:0/1166407999 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f34e805a4e0 msgr2=0x7f34e805c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:46.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.543+0000 7f3501907700 1 --2- 172.21.15.121:0/1166407999 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f34e805a4e0 0x7f34e805c9a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f34f8003930 tx=0x7f34f8007bf0).stop 2023-11-15T03:08:46.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.543+0000 7f3501907700 1 -- 172.21.15.121:0/1166407999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34fc0c29a0 msgr2=0x7f34fc148030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:46.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.543+0000 7f3501907700 1 --2- 172.21.15.121:0/1166407999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34fc0c29a0 0x7f34fc148030 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 rx=0x7f34f4005ad0 tx=0x7f34f40015b0).stop 2023-11-15T03:08:46.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.543+0000 7f3501907700 1 -- 172.21.15.121:0/1166407999 shutdown_connections 2023-11-15T03:08:46.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.543+0000 7f3501907700 1 --2- 172.21.15.121:0/1166407999 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f34e805a4e0 0x7f34e805c9a0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:46.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.543+0000 7f3501907700 1 --2- 172.21.15.121:0/1166407999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34fc0c29a0 0x7f34fc148030 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:46.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.543+0000 7f3501907700 1 --2- 172.21.15.121:0/1166407999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34fc0a87f0 0x7f34fc0c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:46.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.543+0000 7f3501907700 1 --2- 172.21.15.121:0/1166407999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34fc066d70 0x7f34fc13a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:46.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.543+0000 7f3501907700 1 -- 172.21.15.121:0/1166407999 >> 172.21.15.121:0/1166407999 conn(0x7f34fc062190 msgr2=0x7f34fc157770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:46.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.543+0000 7f3501907700 1 -- 172.21.15.121:0/1166407999 shutdown_connections 2023-11-15T03:08:46.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.543+0000 7f3501907700 1 -- 172.21.15.121:0/1166407999 wait complete. 2023-11-15T03:08:46.558 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 260 2023-11-15T03:08:46.573 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:08:46.573 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:46.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.719+0000 7f2078839700 1 Processor -- start 2023-11-15T03:08:46.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.719+0000 7f2078839700 1 -- start start 2023-11-15T03:08:46.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.723+0000 7f2078839700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2074152850 0x7f20740c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:46.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.723+0000 7f2078839700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20740c1d50 con 0x7f207415be30 2023-11-15T03:08:46.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.723+0000 7f2078839700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f207415cb80 con 0x7f20740c3280 2023-11-15T03:08:46.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.723+0000 7f2078839700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20740c1860 con 0x7f2074152850 2023-11-15T03:08:46.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.723+0000 7f2073fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f207415be30 0x7f207415c280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49986/0 (socket says 172.21.15.121:49986) 2023-11-15T03:08:46.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.723+0000 7f206affd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2074152850 0x7f20740c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:46.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.723+0000 7f2073fff700 1 -- 172.21.15.121:0/869797451 learned_addr learned my addr 172.21.15.121:0/869797451 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:46.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.723+0000 7f2072ffd700 1 -- 172.21.15.121:0/869797451 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978311171 0 0) 0x7f207415cb80 con 0x7f20740c3280 2023-11-15T03:08:46.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.723+0000 7f2072ffd700 1 -- 172.21.15.121:0/869797451 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20600033f0 con 0x7f20740c3280 2023-11-15T03:08:46.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.723+0000 7f2072ffd700 1 -- 172.21.15.121:0/869797451 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934419402 0 0) 0x7f20740c1d50 con 0x7f207415be30 2023-11-15T03:08:46.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.723+0000 7f2072ffd700 1 -- 172.21.15.121:0/869797451 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f207415cb80 con 0x7f207415be30 2023-11-15T03:08:46.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.723+0000 7f206affd700 1 -- 172.21.15.121:0/869797451 >> v1:172.21.15.190:6789/0 conn(0x7f20740c3280 legacy=0x7f2074152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:46.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.723+0000 7f206affd700 1 -- 172.21.15.121:0/869797451 >> v1:172.21.15.121:6789/0 conn(0x7f207415be30 legacy=0x7f207415c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:46.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.723+0000 7f206affd700 1 -- 172.21.15.121:0/869797451 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20740a9240 con 0x7f2074152850 2023-11-15T03:08:46.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.723+0000 7f206affd700 1 --2- 172.21.15.121:0/869797451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2074152850 0x7f20740c3660 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f205c009760 tx=0x7f205c009060).ready entity=mon.1 client_cookie=fe90f6e8c444719c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:46.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.723+0000 7f2072ffd700 1 -- 172.21.15.121:0/869797451 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c00d4a0 con 0x7f2074152850 2023-11-15T03:08:46.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.723+0000 7f2072ffd700 1 -- 172.21.15.121:0/869797451 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205c00d600 con 0x7f2074152850 2023-11-15T03:08:46.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.723+0000 7f2072ffd700 1 -- 172.21.15.121:0/869797451 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c00d980 con 0x7f2074152850 2023-11-15T03:08:46.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.727+0000 7f2078839700 1 -- 172.21.15.121:0/869797451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2074152850 msgr2=0x7f20740c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:46.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.727+0000 7f2078839700 1 --2- 172.21.15.121:0/869797451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2074152850 0x7f20740c3660 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f205c009760 tx=0x7f205c009060).stop 2023-11-15T03:08:46.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.727+0000 7f2078839700 1 -- 172.21.15.121:0/869797451 shutdown_connections 2023-11-15T03:08:46.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.727+0000 7f2078839700 1 --2- 172.21.15.121:0/869797451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2074152850 0x7f20740c3660 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:46.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.727+0000 7f2078839700 1 -- 172.21.15.121:0/869797451 >> 172.21.15.121:0/869797451 conn(0x7f2074062190 msgr2=0x7f2074062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:46.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.727+0000 7f2078839700 1 -- 172.21.15.121:0/869797451 shutdown_connections 2023-11-15T03:08:46.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.727+0000 7f2078839700 1 -- 172.21.15.121:0/869797451 wait complete. 2023-11-15T03:08:46.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.731+0000 7f2078839700 1 Processor -- start 2023-11-15T03:08:46.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.731+0000 7f2078839700 1 -- start start 2023-11-15T03:08:46.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.731+0000 7f2078839700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740c3280 0x7f207414eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:46.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.731+0000 7f2078839700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2074152850 0x7f2074168de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:46.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.731+0000 7f2078839700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f207415be30 0x7f2074155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:46.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.731+0000 7f2078839700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2074168950 con 0x7f2074152850 2023-11-15T03:08:46.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.731+0000 7f2078839700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2074168600 con 0x7f207415be30 2023-11-15T03:08:46.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.731+0000 7f2078839700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20741585a0 con 0x7f20740c3280 2023-11-15T03:08:46.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.731+0000 7f206affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2074152850 0x7f2074168de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:46.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.731+0000 7f206affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2074152850 0x7f2074168de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36490/0 (socket says 172.21.15.121:36490) 2023-11-15T03:08:46.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.731+0000 7f206affd700 1 -- 172.21.15.121:0/3545968829 learned_addr learned my addr 172.21.15.121:0/3545968829 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:46.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.735+0000 7f20737fe700 1 --2- 172.21.15.121:0/3545968829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740c3280 0x7f207414eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:46.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.735+0000 7f2073fff700 1 --2- 172.21.15.121:0/3545968829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f207415be30 0x7f2074155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:46.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.735+0000 7f206affd700 1 -- 172.21.15.121:0/3545968829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740c3280 msgr2=0x7f207414eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:46.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.735+0000 7f206affd700 1 --2- 172.21.15.121:0/3545968829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740c3280 0x7f207414eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:46.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.735+0000 7f206affd700 1 -- 172.21.15.121:0/3545968829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f207415be30 msgr2=0x7f2074155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:46.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.735+0000 7f206affd700 1 --2- 172.21.15.121:0/3545968829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f207415be30 0x7f2074155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:46.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.735+0000 7f206affd700 1 -- 172.21.15.121:0/3545968829 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c0092a0 con 0x7f2074152850 2023-11-15T03:08:46.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.735+0000 7f2073fff700 1 --2- 172.21.15.121:0/3545968829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f207415be30 0x7f2074155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:46.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.735+0000 7f206affd700 1 --2- 172.21.15.121:0/3545968829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2074152850 0x7f2074168de0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 rx=0x7f20740654e0 tx=0x7f205c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:46.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.735+0000 7f206bfff700 1 -- 172.21.15.121:0/3545968829 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c00dcf0 con 0x7f2074152850 2023-11-15T03:08:46.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.735+0000 7f2078839700 1 -- 172.21.15.121:0/3545968829 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2074158880 con 0x7f2074152850 2023-11-15T03:08:46.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.735+0000 7f206bfff700 1 -- 172.21.15.121:0/3545968829 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205c00de50 con 0x7f2074152850 2023-11-15T03:08:46.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.735+0000 7f206bfff700 1 -- 172.21.15.121:0/3545968829 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c01d430 con 0x7f2074152850 2023-11-15T03:08:46.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.735+0000 7f2078839700 1 -- 172.21.15.121:0/3545968829 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207414f010 con 0x7f2074152850 2023-11-15T03:08:46.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.735+0000 7f2078839700 1 -- 172.21.15.121:0/3545968829 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f207406bcf0 con 0x7f2074152850 2023-11-15T03:08:46.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.739+0000 7f206bfff700 1 -- 172.21.15.121:0/3545968829 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f205c01d7a0 con 0x7f2074152850 2023-11-15T03:08:46.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.739+0000 7f206bfff700 1 --2- 172.21.15.121:0/3545968829 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f205405a510 0x7f205405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:46.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.739+0000 7f206bfff700 1 -- 172.21.15.121:0/3545968829 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8760+0+0 (secure 0 0 0) 0x7f205c077550 con 0x7f2074152850 2023-11-15T03:08:46.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.739+0000 7f20737fe700 1 --2- 172.21.15.121:0/3545968829 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f205405a510 0x7f205405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:46.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.739+0000 7f20737fe700 1 --2- 172.21.15.121:0/3545968829 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f205405a510 0x7f205405c9d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7f2074151d90 tx=0x7f2064007bc0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:46.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.743+0000 7f206bfff700 1 -- 172.21.15.121:0/3545968829 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f205c04a220 con 0x7f2074152850 2023-11-15T03:08:46.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.939+0000 7f2078839700 1 -- 172.21.15.121:0/3545968829 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20740c1f80 con 0x7f2074152850 2023-11-15T03:08:46.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.939+0000 7f206bfff700 1 -- 172.21.15.121:0/3545968829 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 260 v260) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f205c04d840 con 0x7f2074152850 2023-11-15T03:08:46.949 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:46.949 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":260,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3050548459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3050548459},{"type":"v1","addr":"172.21.15.121:6836","nonce":3050548459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":260}],"filesystems":[{"mdsmap":{"epoch":259,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:08:41.315469+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:46.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.943+0000 7f2078839700 1 -- 172.21.15.121:0/3545968829 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f205405a510 msgr2=0x7f205405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:46.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.943+0000 7f2078839700 1 --2- 172.21.15.121:0/3545968829 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f205405a510 0x7f205405c9d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7f2074151d90 tx=0x7f2064007bc0).stop 2023-11-15T03:08:46.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.943+0000 7f2078839700 1 -- 172.21.15.121:0/3545968829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2074152850 msgr2=0x7f2074168de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:46.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.943+0000 7f2078839700 1 --2- 172.21.15.121:0/3545968829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2074152850 0x7f2074168de0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 rx=0x7f20740654e0 tx=0x7f205c016000).stop 2023-11-15T03:08:46.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.943+0000 7f2078839700 1 -- 172.21.15.121:0/3545968829 shutdown_connections 2023-11-15T03:08:46.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.943+0000 7f2078839700 1 --2- 172.21.15.121:0/3545968829 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f205405a510 0x7f205405c9d0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:46.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.943+0000 7f2078839700 1 --2- 172.21.15.121:0/3545968829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f207415be30 0x7f2074155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:46.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.943+0000 7f2078839700 1 --2- 172.21.15.121:0/3545968829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2074152850 0x7f2074168de0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:46.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.943+0000 7f2078839700 1 --2- 172.21.15.121:0/3545968829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740c3280 0x7f207414eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:46.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.943+0000 7f2078839700 1 -- 172.21.15.121:0/3545968829 >> 172.21.15.121:0/3545968829 conn(0x7f2074062190 msgr2=0x7f207414fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:46.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.943+0000 7f2078839700 1 -- 172.21.15.121:0/3545968829 shutdown_connections 2023-11-15T03:08:46.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:46.943+0000 7f2078839700 1 -- 172.21.15.121:0/3545968829 wait complete. 2023-11-15T03:08:46.956 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 260 2023-11-15T03:08:46.975 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate mds0_inotable' led to DAMAGED state 2023-11-15T03:08:46.975 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 400.00000000' 2023-11-15T03:08:46.976 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:08:46.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:08:46.976 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:08:46.992 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:08:46.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:08:46.993 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:08:47.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.195+0000 7ff7fd3b9700 1 Processor -- start 2023-11-15T03:08:47.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.195+0000 7ff7fd3b9700 1 -- start start 2023-11-15T03:08:47.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.195+0000 7ff7fd3b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7f80c5a00 0x7ff7f8158260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:47.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.195+0000 7ff7fd3b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff7f80c7a90 0x7ff7f8154b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:47.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.195+0000 7ff7fd3b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f80c6170 con 0x7ff7f80a9f80 2023-11-15T03:08:47.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.195+0000 7ff7fd3b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f8154000 con 0x7ff7f80c7a90 2023-11-15T03:08:47.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.195+0000 7ff7fd3b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f8157dd0 con 0x7ff7f80c5a00 2023-11-15T03:08:47.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.195+0000 7ff7fcbb8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff7f80c7a90 0x7ff7f8154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:47.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.195+0000 7ff7f7fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ff7f80a9f80 0x7ff7f80c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:56370/0 (socket says 172.21.15.121:56370) 2023-11-15T03:08:47.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.195+0000 7ff7f7fff700 1 -- 172.21.15.121:0/2196816620 learned_addr learned my addr 172.21.15.121:0/2196816620 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:47.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.195+0000 7ff7f77fe700 1 --2- 172.21.15.121:0/2196816620 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7f80c5a00 0x7ff7f8158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:47.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.195+0000 7ff7fcbb8700 1 -- 172.21.15.121:0/2196816620 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7f80c5a00 msgr2=0x7ff7f8158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:47.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.195+0000 7ff7fcbb8700 1 --2- 172.21.15.121:0/2196816620 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7f80c5a00 0x7ff7f8158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:47.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.195+0000 7ff7fcbb8700 1 -- 172.21.15.121:0/2196816620 >> v1:172.21.15.190:6789/0 conn(0x7ff7f80a9f80 legacy=0x7ff7f80c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:47.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.195+0000 7ff7fcbb8700 1 -- 172.21.15.121:0/2196816620 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f8155080 con 0x7ff7f80c7a90 2023-11-15T03:08:47.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.199+0000 7ff7fcbb8700 1 --2- 172.21.15.121:0/2196816620 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f80c7a90 0x7ff7f8154b90 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 rx=0x7ff7f0009e60 tx=0x7ff7f0009780).ready entity=mon.0 client_cookie=86768e4c09185b70 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:47.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.199+0000 7ff7f6ffd700 1 -- 172.21.15.121:0/2196816620 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f000e070 con 0x7ff7f80c7a90 2023-11-15T03:08:47.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.199+0000 7ff7f6ffd700 1 -- 172.21.15.121:0/2196816620 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f000bb60 con 0x7ff7f80c7a90 2023-11-15T03:08:47.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.199+0000 7ff7f6ffd700 1 -- 172.21.15.121:0/2196816620 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0012410 con 0x7ff7f80c7a90 2023-11-15T03:08:47.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.203+0000 7ff7fd3b9700 1 -- 172.21.15.121:0/2196816620 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f80c7a90 msgr2=0x7ff7f8154b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:47.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.203+0000 7ff7fd3b9700 1 --2- 172.21.15.121:0/2196816620 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f80c7a90 0x7ff7f8154b90 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 rx=0x7ff7f0009e60 tx=0x7ff7f0009780).stop 2023-11-15T03:08:47.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.203+0000 7ff7fd3b9700 1 -- 172.21.15.121:0/2196816620 shutdown_connections 2023-11-15T03:08:47.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.203+0000 7ff7fd3b9700 1 --2- 172.21.15.121:0/2196816620 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f80c7a90 0x7ff7f8154b90 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:47.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.203+0000 7ff7fd3b9700 1 --2- 172.21.15.121:0/2196816620 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7f80c5a00 0x7ff7f8158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:47.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.203+0000 7ff7fd3b9700 1 -- 172.21.15.121:0/2196816620 >> 172.21.15.121:0/2196816620 conn(0x7ff7f8067a60 msgr2=0x7ff7f8067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:47.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.203+0000 7ff7fd3b9700 1 -- 172.21.15.121:0/2196816620 shutdown_connections 2023-11-15T03:08:47.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.203+0000 7ff7fd3b9700 1 -- 172.21.15.121:0/2196816620 wait complete. 2023-11-15T03:08:47.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.203+0000 7ff7fd3b9700 1 Processor -- start 2023-11-15T03:08:47.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.203+0000 7ff7fd3b9700 1 -- start start 2023-11-15T03:08:47.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7fd3b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7f80a9f80 0x7ff7f80b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:47.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7fd3b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7f80c7a90 0x7ff7f814aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:47.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7fd3b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f80b0640 0x7ff7f80ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:47.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7fd3b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f814b090 con 0x7ff7f80b0640 2023-11-15T03:08:47.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7fd3b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f80ca0d0 con 0x7ff7f80c7a90 2023-11-15T03:08:47.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7fd3b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f80ca3b0 con 0x7ff7f80a9f80 2023-11-15T03:08:47.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7fcbb8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f80b0640 0x7ff7f80ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:47.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7fcbb8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f80b0640 0x7ff7f80ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36518/0 (socket says 172.21.15.121:36518) 2023-11-15T03:08:47.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7fcbb8700 1 -- 172.21.15.121:0/667992936 learned_addr learned my addr 172.21.15.121:0/667992936 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:47.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7f7fff700 1 --2- 172.21.15.121:0/667992936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7f80a9f80 0x7ff7f80b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:47.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7f77fe700 1 --2- 172.21.15.121:0/667992936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7f80c7a90 0x7ff7f814aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:47.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7fcbb8700 1 -- 172.21.15.121:0/667992936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7f80a9f80 msgr2=0x7ff7f80b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:47.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7fcbb8700 1 --2- 172.21.15.121:0/667992936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7f80a9f80 0x7ff7f80b0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:47.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7fcbb8700 1 -- 172.21.15.121:0/667992936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7f80c7a90 msgr2=0x7ff7f814aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:47.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7fcbb8700 1 --2- 172.21.15.121:0/667992936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7f80c7a90 0x7ff7f814aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:47.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7fcbb8700 1 -- 172.21.15.121:0/667992936 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f0009a20 con 0x7ff7f80b0640 2023-11-15T03:08:47.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7f77fe700 1 --2- 172.21.15.121:0/667992936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7f80c7a90 0x7ff7f814aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:47.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7f7fff700 1 --2- 172.21.15.121:0/667992936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7f80a9f80 0x7ff7f80b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:47.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7fcbb8700 1 --2- 172.21.15.121:0/667992936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f80b0640 0x7ff7f80ca840 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 rx=0x7ff7f8064b60 tx=0x7ff7f0009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:47.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7dffff700 1 -- 172.21.15.121:0/667992936 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f001d700 con 0x7ff7f80b0640 2023-11-15T03:08:47.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7dffff700 1 -- 172.21.15.121:0/667992936 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f001d860 con 0x7ff7f80b0640 2023-11-15T03:08:47.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7fd3b9700 1 -- 172.21.15.121:0/667992936 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f80add30 con 0x7ff7f80b0640 2023-11-15T03:08:47.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7dffff700 1 -- 172.21.15.121:0/667992936 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f001dc20 con 0x7ff7f80b0640 2023-11-15T03:08:47.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.207+0000 7ff7fd3b9700 1 -- 172.21.15.121:0/667992936 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f80ae280 con 0x7ff7f80b0640 2023-11-15T03:08:47.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.211+0000 7ff7fd3b9700 1 -- 172.21.15.121:0/667992936 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f806bcf0 con 0x7ff7f80b0640 2023-11-15T03:08:47.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.211+0000 7ff7dffff700 1 -- 172.21.15.121:0/667992936 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff7f0003040 con 0x7ff7f80b0640 2023-11-15T03:08:47.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.211+0000 7ff7dffff700 1 --2- 172.21.15.121:0/667992936 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff7e405a510 0x7ff7e405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:47.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.211+0000 7ff7dffff700 1 -- 172.21.15.121:0/667992936 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8760+0+0 (secure 0 0 0) 0x7ff7f00772b0 con 0x7ff7f80b0640 2023-11-15T03:08:47.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.211+0000 7ff7f7fff700 1 --2- 172.21.15.121:0/667992936 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff7e405a510 0x7ff7e405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:47.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.211+0000 7ff7f7fff700 1 --2- 172.21.15.121:0/667992936 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff7e405a510 0x7ff7e405c9d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7ff7e8001740 tx=0x7ff7e8009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:47.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.215+0000 7ff7dffff700 1 -- 172.21.15.121:0/667992936 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff7f0049f80 con 0x7ff7f80b0640 2023-11-15T03:08:47.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:47.407+0000 7ff7fd3b9700 1 -- 172.21.15.121:0/667992936 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff7f80c5e90 con 0x7ff7f80b0640 2023-11-15T03:08:48.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.215+0000 7ff7dffff700 1 -- 172.21.15.121:0/667992936 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v261) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff7f004d5a0 con 0x7ff7f80b0640 2023-11-15T03:08:48.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.219+0000 7ff7fd3b9700 1 -- 172.21.15.121:0/667992936 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff7e405a510 msgr2=0x7ff7e405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:48.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.219+0000 7ff7fd3b9700 1 --2- 172.21.15.121:0/667992936 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff7e405a510 0x7ff7e405c9d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7ff7e8001740 tx=0x7ff7e8009040).stop 2023-11-15T03:08:48.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.219+0000 7ff7fd3b9700 1 -- 172.21.15.121:0/667992936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f80b0640 msgr2=0x7ff7f80ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:48.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.219+0000 7ff7fd3b9700 1 --2- 172.21.15.121:0/667992936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f80b0640 0x7ff7f80ca840 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 rx=0x7ff7f8064b60 tx=0x7ff7f0009780).stop 2023-11-15T03:08:48.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.219+0000 7ff7fd3b9700 1 -- 172.21.15.121:0/667992936 shutdown_connections 2023-11-15T03:08:48.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.219+0000 7ff7fd3b9700 1 --2- 172.21.15.121:0/667992936 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff7e405a510 0x7ff7e405c9d0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:48.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.219+0000 7ff7fd3b9700 1 --2- 172.21.15.121:0/667992936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f80b0640 0x7ff7f80ca840 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:48.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.219+0000 7ff7fd3b9700 1 --2- 172.21.15.121:0/667992936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7f80c7a90 0x7ff7f814aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:48.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.219+0000 7ff7fd3b9700 1 --2- 172.21.15.121:0/667992936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7f80a9f80 0x7ff7f80b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:48.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.219+0000 7ff7fd3b9700 1 -- 172.21.15.121:0/667992936 >> 172.21.15.121:0/667992936 conn(0x7ff7f8067a60 msgr2=0x7ff7f80ad0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:48.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.219+0000 7ff7fd3b9700 1 -- 172.21.15.121:0/667992936 shutdown_connections 2023-11-15T03:08:48.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.219+0000 7ff7fd3b9700 1 -- 172.21.15.121:0/667992936 wait complete. 2023-11-15T03:08:48.231 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:08:48.251 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:08:48.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.403+0000 7fc856c79700 1 Processor -- start 2023-11-15T03:08:48.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.403+0000 7fc856c79700 1 -- start start 2023-11-15T03:08:48.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.407+0000 7fc856c79700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc8500ab0d0 0x7fc8500ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:48.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.407+0000 7fc856c79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8500ad1b0 0x7fc850065710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:48.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.407+0000 7fc856c79700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc850066c90 con 0x7fc850065c00 2023-11-15T03:08:48.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.407+0000 7fc856c79700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc850066080 con 0x7fc8500ab0d0 2023-11-15T03:08:48.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.407+0000 7fc856c79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc850066360 con 0x7fc8500ad1b0 2023-11-15T03:08:48.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.407+0000 7fc856478700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc850065c00 0x7fc8500c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49988/0 (socket says 172.21.15.121:49988) 2023-11-15T03:08:48.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.407+0000 7fc856478700 1 -- 172.21.15.121:0/428616169 learned_addr learned my addr 172.21.15.121:0/428616169 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:48.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.407+0000 7fc855c77700 1 --2- 172.21.15.121:0/428616169 >> v2:172.21.15.121:3300/0 conn(0x7fc8500ab0d0 0x7fc8500ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:48.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.407+0000 7fc855476700 1 --2- 172.21.15.121:0/428616169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8500ad1b0 0x7fc850065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:48.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.407+0000 7fc854c75700 1 -- 172.21.15.121:0/428616169 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 563504745 0 0) 0x7fc850066c90 con 0x7fc850065c00 2023-11-15T03:08:48.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.407+0000 7fc854c75700 1 -- 172.21.15.121:0/428616169 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc83c0033f0 con 0x7fc850065c00 2023-11-15T03:08:48.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.407+0000 7fc855476700 1 -- 172.21.15.121:0/428616169 >> v2:172.21.15.121:3300/0 conn(0x7fc8500ab0d0 msgr2=0x7fc8500ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:48.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.407+0000 7fc855476700 1 --2- 172.21.15.121:0/428616169 >> v2:172.21.15.121:3300/0 conn(0x7fc8500ab0d0 0x7fc8500ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:48.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.407+0000 7fc855476700 1 -- 172.21.15.121:0/428616169 >> v1:172.21.15.121:6789/0 conn(0x7fc850065c00 legacy=0x7fc8500c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:48.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.407+0000 7fc855476700 1 -- 172.21.15.121:0/428616169 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc850066640 con 0x7fc8500ad1b0 2023-11-15T03:08:48.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.407+0000 7fc855476700 1 --2- 172.21.15.121:0/428616169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8500ad1b0 0x7fc850065710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fc840009760 tx=0x7fc840009060).ready entity=mon.2 client_cookie=e719b9877763b6db server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:48.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.411+0000 7fc854c75700 1 -- 172.21.15.121:0/428616169 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84000d4a0 con 0x7fc8500ad1b0 2023-11-15T03:08:48.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.411+0000 7fc854c75700 1 -- 172.21.15.121:0/428616169 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84000d600 con 0x7fc8500ad1b0 2023-11-15T03:08:48.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.411+0000 7fc854c75700 1 -- 172.21.15.121:0/428616169 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84000d980 con 0x7fc8500ad1b0 2023-11-15T03:08:48.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.411+0000 7fc856c79700 1 -- 172.21.15.121:0/428616169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8500ad1b0 msgr2=0x7fc850065710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:48.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.411+0000 7fc856c79700 1 --2- 172.21.15.121:0/428616169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8500ad1b0 0x7fc850065710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fc840009760 tx=0x7fc840009060).stop 2023-11-15T03:08:48.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.411+0000 7fc856c79700 1 -- 172.21.15.121:0/428616169 shutdown_connections 2023-11-15T03:08:48.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.411+0000 7fc856c79700 1 --2- 172.21.15.121:0/428616169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8500ad1b0 0x7fc850065710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:48.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.411+0000 7fc856c79700 1 --2- 172.21.15.121:0/428616169 >> v2:172.21.15.121:3300/0 conn(0x7fc8500ab0d0 0x7fc8500ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:48.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.411+0000 7fc856c79700 1 -- 172.21.15.121:0/428616169 >> 172.21.15.121:0/428616169 conn(0x7fc850067a60 msgr2=0x7fc850067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:48.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.411+0000 7fc856c79700 1 -- 172.21.15.121:0/428616169 shutdown_connections 2023-11-15T03:08:48.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.411+0000 7fc856c79700 1 -- 172.21.15.121:0/428616169 wait complete. 2023-11-15T03:08:48.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.415+0000 7fc856c79700 1 Processor -- start 2023-11-15T03:08:48.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.415+0000 7fc856c79700 1 -- start start 2023-11-15T03:08:48.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.415+0000 7fc856c79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc850065c00 0x7fc85014da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:48.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.415+0000 7fc856c79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8500ab0d0 0x7fc850167710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:48.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.415+0000 7fc856c79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8500ad1b0 0x7fc850167c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:48.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.415+0000 7fc856c79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85014df60 con 0x7fc8500ab0d0 2023-11-15T03:08:48.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.415+0000 7fc856c79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85014ed60 con 0x7fc850065c00 2023-11-15T03:08:48.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.415+0000 7fc856c79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc850144e10 con 0x7fc8500ad1b0 2023-11-15T03:08:48.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.415+0000 7fc855476700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8500ab0d0 0x7fc850167710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:48.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.415+0000 7fc855476700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8500ab0d0 0x7fc850167710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36536/0 (socket says 172.21.15.121:36536) 2023-11-15T03:08:48.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.415+0000 7fc855476700 1 -- 172.21.15.121:0/154004700 learned_addr learned my addr 172.21.15.121:0/154004700 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:48.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.419+0000 7fc855c77700 1 --2- 172.21.15.121:0/154004700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc850065c00 0x7fc85014da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:48.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.419+0000 7fc856478700 1 --2- 172.21.15.121:0/154004700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8500ad1b0 0x7fc850167c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:48.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.419+0000 7fc855476700 1 -- 172.21.15.121:0/154004700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8500ad1b0 msgr2=0x7fc850167c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:48.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.419+0000 7fc855476700 1 --2- 172.21.15.121:0/154004700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8500ad1b0 0x7fc850167c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:48.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.419+0000 7fc855476700 1 -- 172.21.15.121:0/154004700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc850065c00 msgr2=0x7fc85014da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:48.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.419+0000 7fc855476700 1 --2- 172.21.15.121:0/154004700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc850065c00 0x7fc85014da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:48.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.419+0000 7fc855476700 1 -- 172.21.15.121:0/154004700 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8400092a0 con 0x7fc8500ab0d0 2023-11-15T03:08:48.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.419+0000 7fc856478700 1 --2- 172.21.15.121:0/154004700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8500ad1b0 0x7fc850167c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:48.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.419+0000 7fc855476700 1 --2- 172.21.15.121:0/154004700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8500ab0d0 0x7fc850167710 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 rx=0x7fc840010000 tx=0x7fc840016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:48.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.419+0000 7fc845ffb700 1 -- 172.21.15.121:0/154004700 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84000dd30 con 0x7fc8500ab0d0 2023-11-15T03:08:48.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.419+0000 7fc845ffb700 1 -- 172.21.15.121:0/154004700 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84000de90 con 0x7fc8500ab0d0 2023-11-15T03:08:48.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.419+0000 7fc856c79700 1 -- 172.21.15.121:0/154004700 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8501450f0 con 0x7fc8500ab0d0 2023-11-15T03:08:48.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.419+0000 7fc845ffb700 1 -- 172.21.15.121:0/154004700 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84000d980 con 0x7fc8500ab0d0 2023-11-15T03:08:48.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.419+0000 7fc856c79700 1 -- 172.21.15.121:0/154004700 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85014f290 con 0x7fc8500ab0d0 2023-11-15T03:08:48.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.419+0000 7fc856c79700 1 -- 172.21.15.121:0/154004700 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85006bcf0 con 0x7fc8500ab0d0 2023-11-15T03:08:48.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.423+0000 7fc845ffb700 1 -- 172.21.15.121:0/154004700 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc84001d640 con 0x7fc8500ab0d0 2023-11-15T03:08:48.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.423+0000 7fc845ffb700 1 --2- 172.21.15.121:0/154004700 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc83405a510 0x7fc83405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:48.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.423+0000 7fc845ffb700 1 -- 172.21.15.121:0/154004700 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8760+0+0 (secure 0 0 0) 0x7fc8400769a0 con 0x7fc8500ab0d0 2023-11-15T03:08:48.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.423+0000 7fc855c77700 1 --2- 172.21.15.121:0/154004700 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc83405a510 0x7fc83405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:48.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.423+0000 7fc855c77700 1 --2- 172.21.15.121:0/154004700 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc83405a510 0x7fc83405c9d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7fc84c009b80 tx=0x7fc84c009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:48.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.427+0000 7fc845ffb700 1 -- 172.21.15.121:0/154004700 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc840049670 con 0x7fc8500ab0d0 2023-11-15T03:08:48.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:48.643+0000 7fc856c79700 1 -- 172.21.15.121:0/154004700 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fc850068a70 con 0x7fc8500ab0d0 2023-11-15T03:08:49.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.223+0000 7fc845ffb700 1 -- 172.21.15.121:0/154004700 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v262) v1 ==== 101+0+0 (secure 0 0 0) 0x7fc84004cc90 con 0x7fc8500ab0d0 2023-11-15T03:08:49.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.227+0000 7fc856c79700 1 -- 172.21.15.121:0/154004700 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc83405a510 msgr2=0x7fc83405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.227+0000 7fc856c79700 1 --2- 172.21.15.121:0/154004700 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc83405a510 0x7fc83405c9d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7fc84c009b80 tx=0x7fc84c009060).stop 2023-11-15T03:08:49.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.227+0000 7fc856c79700 1 -- 172.21.15.121:0/154004700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8500ab0d0 msgr2=0x7fc850167710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.227+0000 7fc856c79700 1 --2- 172.21.15.121:0/154004700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8500ab0d0 0x7fc850167710 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 rx=0x7fc840010000 tx=0x7fc840016000).stop 2023-11-15T03:08:49.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.231+0000 7fc856c79700 1 -- 172.21.15.121:0/154004700 shutdown_connections 2023-11-15T03:08:49.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.231+0000 7fc856c79700 1 --2- 172.21.15.121:0/154004700 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc83405a510 0x7fc83405c9d0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.231+0000 7fc856c79700 1 --2- 172.21.15.121:0/154004700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8500ad1b0 0x7fc850167c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.231+0000 7fc856c79700 1 --2- 172.21.15.121:0/154004700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8500ab0d0 0x7fc850167710 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.231+0000 7fc856c79700 1 --2- 172.21.15.121:0/154004700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc850065c00 0x7fc85014da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.231+0000 7fc856c79700 1 -- 172.21.15.121:0/154004700 >> 172.21.15.121:0/154004700 conn(0x7fc850067a60 msgr2=0x7fc850155250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:49.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.231+0000 7fc856c79700 1 -- 172.21.15.121:0/154004700 shutdown_connections 2023-11-15T03:08:49.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.231+0000 7fc856c79700 1 -- 172.21.15.121:0/154004700 wait complete. 2023-11-15T03:08:49.241 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:08:49.261 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:08:49.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.307+0000 7fbeaeb37d00 1 Processor -- start 2023-11-15T03:08:49.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.307+0000 7fbeaeb37d00 1 -- start start 2023-11-15T03:08:49.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.307+0000 7fbeaeb37d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564ded427440 0x564ded3f5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.307+0000 7fbeaeb37d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ded441c60 0x564ded4cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.307+0000 7fbeaeb37d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ded4b42e0 0x564ded4ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.307+0000 7fbeaeb37d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ded36e220 con 0x564ded441c60 2023-11-15T03:08:49.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.307+0000 7fbeaeb37d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ded440ea0 con 0x564ded4b42e0 2023-11-15T03:08:49.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.307+0000 7fbeaeb37d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564ded4ce590 con 0x564ded427440 2023-11-15T03:08:49.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.307+0000 7fbe9bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ded441c60 0x564ded4cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.307+0000 7fbea3846700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ded4b42e0 0x564ded4ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.307+0000 7fbe9bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ded441c60 0x564ded4cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36562/0 (socket says 172.21.15.121:36562) 2023-11-15T03:08:49.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.307+0000 7fbea3045700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564ded427440 0x564ded3f5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.307+0000 7fbe9bfff700 1 -- 172.21.15.121:0/1049280174 learned_addr learned my addr 172.21.15.121:0/1049280174 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:49.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbe9bfff700 1 -- 172.21.15.121:0/1049280174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564ded427440 msgr2=0x564ded3f5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:49.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbe9bfff700 1 --2- 172.21.15.121:0/1049280174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564ded427440 0x564ded3f5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbe9bfff700 1 -- 172.21.15.121:0/1049280174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ded4b42e0 msgr2=0x564ded4ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:49.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbe9bfff700 1 --2- 172.21.15.121:0/1049280174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ded4b42e0 0x564ded4ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbe9bfff700 1 -- 172.21.15.121:0/1049280174 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c00a180 con 0x564ded441c60 2023-11-15T03:08:49.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbe9bfff700 1 --2- 172.21.15.121:0/1049280174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ded441c60 0x564ded4cdbb0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 rx=0x7fbe8c009980 tx=0x7fbe8c00c040).ready entity=mon.0 client_cookie=3bbb3fb395ba469a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:49.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbea2043700 1 -- 172.21.15.121:0/1049280174 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c00f040 con 0x564ded441c60 2023-11-15T03:08:49.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbea2043700 1 -- 172.21.15.121:0/1049280174 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8c008820 con 0x564ded441c60 2023-11-15T03:08:49.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbea2043700 1 -- 172.21.15.121:0/1049280174 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c008ba0 con 0x564ded441c60 2023-11-15T03:08:49.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/1049280174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ded441c60 msgr2=0x564ded4cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/1049280174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ded441c60 0x564ded4cdbb0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 rx=0x7fbe8c009980 tx=0x7fbe8c00c040).stop 2023-11-15T03:08:49.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/1049280174 shutdown_connections 2023-11-15T03:08:49.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/1049280174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ded4b42e0 0x564ded4ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/1049280174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ded441c60 0x564ded4cdbb0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/1049280174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564ded427440 0x564ded3f5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/1049280174 >> 172.21.15.121:0/1049280174 conn(0x564ded3f67d0 msgr2=0x564ded4265e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:49.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/1049280174 shutdown_connections 2023-11-15T03:08:49.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/1049280174 wait complete. 2023-11-15T03:08:49.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbeaeb37d00 1 Processor -- start 2023-11-15T03:08:49.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.311+0000 7fbeaeb37d00 1 -- start start 2023-11-15T03:08:49.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbeaeb37d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564ded427440 0x564ded4405d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbeaeb37d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ded441c60 0x564ded4ac570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbeaeb37d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ded4b42e0 0x564ded425140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbeaeb37d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ded4b27d0 con 0x564ded4b42e0 2023-11-15T03:08:49.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbeaeb37d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ded4b1bc0 con 0x564ded441c60 2023-11-15T03:08:49.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbeaeb37d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564ded4b1ea0 con 0x564ded427440 2023-11-15T03:08:49.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbea3846700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ded4b42e0 0x564ded425140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbea3846700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ded4b42e0 0x564ded425140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36576/0 (socket says 172.21.15.121:36576) 2023-11-15T03:08:49.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 learned_addr learned my addr 172.21.15.121:0/2941313912 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:49.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbea3045700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564ded427440 0x564ded4405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbe9bfff700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ded441c60 0x564ded4ac570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564ded427440 msgr2=0x564ded4405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbea3846700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564ded427440 0x564ded4405d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ded441c60 msgr2=0x564ded4ac570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbea3846700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ded441c60 0x564ded4ac570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c009580 con 0x564ded4b42e0 2023-11-15T03:08:49.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbea3045700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564ded427440 0x564ded4405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:49.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbea3846700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ded4b42e0 0x564ded425140 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 rx=0x7fbe9c00b010 tx=0x7fbe9c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:49.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbea0840700 1 -- 172.21.15.121:0/2941313912 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c005940 con 0x564ded4b42e0 2023-11-15T03:08:49.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbea0840700 1 -- 172.21.15.121:0/2941313912 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9c00c770 con 0x564ded4b42e0 2023-11-15T03:08:49.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbea0840700 1 -- 172.21.15.121:0/2941313912 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c01e6a0 con 0x564ded4b42e0 2023-11-15T03:08:49.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564ded397770 con 0x564ded4b42e0 2023-11-15T03:08:49.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.315+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ded366a70 con 0x564ded4b42e0 2023-11-15T03:08:49.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.319+0000 7fbea0840700 1 -- 172.21.15.121:0/2941313912 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fbe9c003040 con 0x564ded4b42e0 2023-11-15T03:08:49.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.319+0000 7fbea0840700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbe84058ff0 0x7fbe8405b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.319+0000 7fbea0840700 1 -- 172.21.15.121:0/2941313912 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8760+0+0 (secure 0 0 0) 0x7fbe9c076550 con 0x564ded4b42e0 2023-11-15T03:08:49.334 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:08:49.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.319+0000 7fbea3045700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbe84058ff0 0x7fbe8405b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.319+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x564ded425630 0x564ded4b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.319+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4b3150 con 0x564ded425630 2023-11-15T03:08:49.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.319+0000 7fbe9bfff700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x564ded425630 0x564ded4b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.319+0000 7fbea3045700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbe84058ff0 0x7fbe8405b4b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7fbe94000c00 tx=0x7fbe94001740).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:49.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.319+0000 7fbe9bfff700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x564ded425630 0x564ded4b2c60 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:49.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.323+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v123'341 uv334 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded425630 2023-11-15T03:08:49.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.323+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded4e23a0 con 0x564ded425630 2023-11-15T03:08:49.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.323+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v123'342 uv342 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded425630 2023-11-15T03:08:49.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.327+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4e2750 con 0x564ded425630 2023-11-15T03:08:49.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.327+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v123'343 uv343 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded425630 2023-11-15T03:08:49.338 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:08:49.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.327+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4e2b00 con 0x564ded425630 2023-11-15T03:08:49.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.331+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v123'344 uv344 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded425630 2023-11-15T03:08:49.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.331+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4e2f10 con 0x564ded425630 2023-11-15T03:08:49.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.331+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v123'345 uv345 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded425630 2023-11-15T03:08:49.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.331+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4eb680 con 0x564ded425630 2023-11-15T03:08:49.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.335+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v123'346 uv346 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded425630 2023-11-15T03:08:49.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.335+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4eba30 con 0x564ded425630 2023-11-15T03:08:49.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.335+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v123'347 uv340 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded425630 2023-11-15T03:08:49.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.335+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded4ebde0 con 0x564ded425630 2023-11-15T03:08:49.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.343+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v123'348 uv348 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded425630 2023-11-15T03:08:49.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.343+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4ebde0 con 0x564ded425630 2023-11-15T03:08:49.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.343+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v123'349 uv349 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded425630 2023-11-15T03:08:49.353 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:08:49.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.343+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4ecdf0 con 0x564ded425630 2023-11-15T03:08:49.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.347+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v123'350 uv350 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded425630 2023-11-15T03:08:49.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.347+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4ecdf0 con 0x564ded425630 2023-11-15T03:08:49.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.347+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v123'351 uv351 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded425630 2023-11-15T03:08:49.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.347+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4ebde0 con 0x564ded425630 2023-11-15T03:08:49.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.351+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v123'352 uv352 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded425630 2023-11-15T03:08:49.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.351+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x564ded4ec2f0 0x564ded4ee6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.351+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4eebd0 con 0x564ded4ec2f0 2023-11-15T03:08:49.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.351+0000 7fbea3045700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x564ded4ec2f0 0x564ded4ee6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.355+0000 7fbea3045700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x564ded4ec2f0 0x564ded4ee6e0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:49.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.359+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v123'537 uv524 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.359+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded4eef80 con 0x564ded4ec2f0 2023-11-15T03:08:49.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.363+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v123'538 uv538 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.363+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4ef330 con 0x564ded4ec2f0 2023-11-15T03:08:49.374 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:08:49.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.367+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v123'539 uv539 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.367+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f0750 con 0x564ded4ec2f0 2023-11-15T03:08:49.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.367+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v123'540 uv540 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.367+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f0bf0 con 0x564ded4ec2f0 2023-11-15T03:08:49.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.367+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v123'541 uv541 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.371+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4c6a70 con 0x564ded4ec2f0 2023-11-15T03:08:49.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.371+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v123'542 uv542 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.371+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f00b0 con 0x564ded4ec2f0 2023-11-15T03:08:49.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.375+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v123'543 uv543 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.375+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f0ff0 con 0x564ded4ec2f0 2023-11-15T03:08:49.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.379+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v123'544 uv530 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.379+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded4f0ff0 con 0x564ded4ec2f0 2023-11-15T03:08:49.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.379+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v123'545 uv545 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.379+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f0ff0 con 0x564ded4ec2f0 2023-11-15T03:08:49.391 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:08:49.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.383+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v123'546 uv546 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.383+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f2000 con 0x564ded4ec2f0 2023-11-15T03:08:49.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.387+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v123'547 uv547 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.387+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f2000 con 0x564ded4ec2f0 2023-11-15T03:08:49.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.387+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v123'548 uv548 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.387+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f0ff0 con 0x564ded4ec2f0 2023-11-15T03:08:49.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.391+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v123'549 uv549 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.391+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f0ff0 con 0x564ded4ec2f0 2023-11-15T03:08:49.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.395+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v123'550 uv536 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.395+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded4f0ff0 con 0x564ded4ec2f0 2023-11-15T03:08:49.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.395+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v123'551 uv551 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.395+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f0ff0 con 0x564ded4ec2f0 2023-11-15T03:08:49.408 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:08:49.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.399+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v123'552 uv552 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.399+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f2000 con 0x564ded4ec2f0 2023-11-15T03:08:49.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.403+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v123'553 uv553 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.403+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f2000 con 0x564ded4ec2f0 2023-11-15T03:08:49.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.407+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v123'554 uv554 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.407+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f0ff0 con 0x564ded4ec2f0 2023-11-15T03:08:49.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.407+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v123'555 uv555 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.407+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x564ded4f1500 0x564ded4f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.407+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f3de0 con 0x564ded4f1500 2023-11-15T03:08:49.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.407+0000 7fbe9bfff700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x564ded4f1500 0x564ded4f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.411+0000 7fbe9bfff700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x564ded4f1500 0x564ded4f38f0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:49.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.411+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v123'294 uv288 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4f1500 2023-11-15T03:08:49.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.415+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded4f3ec0 con 0x564ded4f1500 2023-11-15T03:08:49.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.415+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v123'295 uv295 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4f1500 2023-11-15T03:08:49.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.415+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f42f0 con 0x564ded4f1500 2023-11-15T03:08:49.428 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:08:49.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.419+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v123'296 uv296 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4f1500 2023-11-15T03:08:49.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.419+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f42f0 con 0x564ded4f1500 2023-11-15T03:08:49.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.423+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v123'297 uv297 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4f1500 2023-11-15T03:08:49.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.423+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f46f0 con 0x564ded4f1500 2023-11-15T03:08:49.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.427+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v123'298 uv298 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4f1500 2023-11-15T03:08:49.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.427+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f46f0 con 0x564ded4f1500 2023-11-15T03:08:49.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.427+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v123'299 uv293 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4f1500 2023-11-15T03:08:49.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.427+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded4f42f0 con 0x564ded4f1500 2023-11-15T03:08:49.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.431+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v123'300 uv300 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4f1500 2023-11-15T03:08:49.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.431+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f42f0 con 0x564ded4f1500 2023-11-15T03:08:49.442 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:08:49.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.435+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v123'301 uv301 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4f1500 2023-11-15T03:08:49.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.435+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f42f0 con 0x564ded4f1500 2023-11-15T03:08:49.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.435+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v123'302 uv302 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4f1500 2023-11-15T03:08:49.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.435+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4efc40 con 0x564ded4f1500 2023-11-15T03:08:49.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.439+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v123'303 uv303 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4f1500 2023-11-15T03:08:49.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.439+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x564ded4efc40 0x564ded4f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.439+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f7230 con 0x564ded4efc40 2023-11-15T03:08:49.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.439+0000 7fbea3846700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x564ded4efc40 0x564ded4f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.439+0000 7fbea3846700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x564ded4efc40 0x564ded4f6d00 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:49.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.443+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v123'344 uv337 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.443+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded4f7730 con 0x564ded4efc40 2023-11-15T03:08:49.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.447+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v123'345 uv345 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.447+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f7c20 con 0x564ded4efc40 2023-11-15T03:08:49.457 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:08:49.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.447+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v123'346 uv346 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.447+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f43b0 con 0x564ded4efc40 2023-11-15T03:08:49.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.451+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v123'347 uv347 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.451+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fc6a0 con 0x564ded4efc40 2023-11-15T03:08:49.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.455+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v123'348 uv348 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.455+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fc230 con 0x564ded4efc40 2023-11-15T03:08:49.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.459+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v123'349 uv343 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.459+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded4f8510 con 0x564ded4efc40 2023-11-15T03:08:49.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.459+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v123'350 uv350 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.459+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f89d0 con 0x564ded4efc40 2023-11-15T03:08:49.471 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:08:49.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.463+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v123'351 uv351 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.463+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f89d0 con 0x564ded4efc40 2023-11-15T03:08:49.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.467+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v123'352 uv352 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.467+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f9de0 con 0x564ded4efc40 2023-11-15T03:08:49.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.467+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v123'353 uv353 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.467+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f89a0 con 0x564ded4efc40 2023-11-15T03:08:49.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.471+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v123'354 uv354 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.471+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x564ded4f92b0 0x564ded4f96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.471+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f9bc0 con 0x564ded4f92b0 2023-11-15T03:08:49.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.471+0000 7fbea3045700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x564ded4f92b0 0x564ded4f96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.471+0000 7fbea3045700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x564ded4f92b0 0x564ded4f96d0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:49.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.475+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v123'171 uv170 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4f92b0 2023-11-15T03:08:49.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.475+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded4f9bc0 con 0x564ded4f92b0 2023-11-15T03:08:49.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.479+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v123'172 uv172 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4f92b0 2023-11-15T03:08:49.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.479+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f9bc0 con 0x564ded4f92b0 2023-11-15T03:08:49.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.479+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v123'173 uv173 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4f92b0 2023-11-15T03:08:49.489 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:08:49.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.479+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f9bc0 con 0x564ded4f92b0 2023-11-15T03:08:49.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.483+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v123'174 uv174 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4f92b0 2023-11-15T03:08:49.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.483+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f9bc0 con 0x564ded4f92b0 2023-11-15T03:08:49.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.487+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v123'175 uv175 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4f92b0 2023-11-15T03:08:49.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.487+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f9bc0 con 0x564ded4f92b0 2023-11-15T03:08:49.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.487+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v123'176 uv176 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4f92b0 2023-11-15T03:08:49.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.487+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x564ded4fb750 0x564ded4fbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.487+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa1e0 con 0x564ded4fb750 2023-11-15T03:08:49.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.487+0000 7fbe9bfff700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x564ded4fb750 0x564ded4fbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.491+0000 7fbe9bfff700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x564ded4fb750 0x564ded4fbb80 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:49.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.491+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v123'121 uv120 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.491+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded4fa1e0 con 0x564ded4fb750 2023-11-15T03:08:49.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.495+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v123'122 uv122 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.495+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa1e0 con 0x564ded4fb750 2023-11-15T03:08:49.506 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:08:49.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.499+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v123'123 uv123 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.499+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.499+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v123'124 uv124 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.499+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.503+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v123'171 uv170 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.503+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.507+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v123'172 uv172 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.507+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.507+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v123'173 uv173 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.518 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:08:49.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.507+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.511+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v123'174 uv174 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.511+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.515+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v123'175 uv175 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.515+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.515+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v123'176 uv176 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.515+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4ec2f0 2023-11-15T03:08:49.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.519+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v123'316 uv310 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.519+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4ec2f0 2023-11-15T03:08:49.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.523+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v123'317 uv317 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.523+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4ec2f0 2023-11-15T03:08:49.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.527+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v123'318 uv318 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.535 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:08:49.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.527+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4ec2f0 2023-11-15T03:08:49.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.531+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v123'319 uv319 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.531+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4ec2f0 2023-11-15T03:08:49.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.531+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v123'320 uv320 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.531+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4ec2f0 2023-11-15T03:08:49.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.535+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v123'321 uv321 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.535+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4ec2f0 2023-11-15T03:08:49.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.539+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v123'322 uv315 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.539+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4ec2f0 2023-11-15T03:08:49.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.543+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v123'323 uv323 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.543+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4ec2f0 2023-11-15T03:08:49.553 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:08:49.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.543+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v123'324 uv324 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.543+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4ec2f0 2023-11-15T03:08:49.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.547+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v123'325 uv325 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.547+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4ec2f0 2023-11-15T03:08:49.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.551+0000 7fbea3045700 1 -- 172.21.15.121:0/2941313912 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v123'326 uv326 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9400b030 con 0x564ded4ec2f0 2023-11-15T03:08:49.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.551+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4f1500 2023-11-15T03:08:49.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.555+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v123'143 uv142 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4f1500 2023-11-15T03:08:49.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.555+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4f1500 2023-11-15T03:08:49.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.555+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v123'144 uv144 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4f1500 2023-11-15T03:08:49.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.555+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4f1500 2023-11-15T03:08:49.567 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:08:49.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.559+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v123'145 uv145 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4f1500 2023-11-15T03:08:49.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.559+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4f1500 2023-11-15T03:08:49.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.563+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v123'146 uv146 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4f1500 2023-11-15T03:08:49.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.563+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4f1500 2023-11-15T03:08:49.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.563+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v123'147 uv147 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4f1500 2023-11-15T03:08:49.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.563+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.567+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v123'171 uv170 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.567+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.571+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v123'172 uv172 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.571+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.582 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:08:49.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.575+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v123'173 uv173 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.575+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.575+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v123'174 uv174 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.575+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.579+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v123'175 uv175 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.579+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.583+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v123'176 uv176 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.583+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.587+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v123'144 uv143 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.587+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.591+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v123'145 uv145 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.591+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.591+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v123'146 uv146 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.601 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:08:49.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.591+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.595+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v123'147 uv147 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.595+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.599+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v123'148 uv148 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.599+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.603+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v123'141 uv140 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.603+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.603+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v123'142 uv142 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.603+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa660 con 0x564ded4fb750 2023-11-15T03:08:49.616 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:08:49.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.607+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v123'143 uv143 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.607+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4fa6e0 con 0x564ded4fb750 2023-11-15T03:08:49.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.611+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v123'144 uv144 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.611+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded4f8060 con 0x564ded4fb750 2023-11-15T03:08:49.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.615+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v123'145 uv145 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.615+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x564ded4f80d0 0x564ded503390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.615+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded503880 con 0x564ded4f80d0 2023-11-15T03:08:49.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.615+0000 7fbea3846700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x564ded4f80d0 0x564ded503390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.615+0000 7fbea3846700 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x564ded4f80d0 0x564ded503390 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:49.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.619+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v123'114 uv113 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4f80d0 2023-11-15T03:08:49.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.619+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded503880 con 0x564ded4f80d0 2023-11-15T03:08:49.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.623+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v123'115 uv115 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4f80d0 2023-11-15T03:08:49.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.623+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded503880 con 0x564ded4f80d0 2023-11-15T03:08:49.634 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:08:49.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.627+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v123'116 uv116 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4f80d0 2023-11-15T03:08:49.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.627+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded503880 con 0x564ded4f80d0 2023-11-15T03:08:49.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.627+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v123'117 uv117 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4f80d0 2023-11-15T03:08:49.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.627+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded503880 con 0x564ded4fb750 2023-11-15T03:08:49.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.631+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v123'171 uv170 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.631+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded503880 con 0x564ded4fb750 2023-11-15T03:08:49.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.635+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v123'172 uv172 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.635+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded502fe0 con 0x564ded4fb750 2023-11-15T03:08:49.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.635+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v123'173 uv173 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.646 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:08:49.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.639+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded502fe0 con 0x564ded4fb750 2023-11-15T03:08:49.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.639+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v123'174 uv174 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.639+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded502fe0 con 0x564ded4fb750 2023-11-15T03:08:49.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.643+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v123'175 uv175 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.643+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded502fe0 con 0x564ded4fb750 2023-11-15T03:08:49.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.647+0000 7fbe9bfff700 1 -- 172.21.15.121:0/2941313912 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v123'176 uv176 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe8c008990 con 0x564ded4fb750 2023-11-15T03:08:49.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.647+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded502fe0 con 0x564ded4efc40 2023-11-15T03:08:49.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.647+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v123'145 uv144 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.647+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded502fe0 con 0x564ded4efc40 2023-11-15T03:08:49.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.651+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v123'146 uv146 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.651+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded502fe0 con 0x564ded4efc40 2023-11-15T03:08:49.663 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:08:49.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.655+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v123'147 uv147 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.655+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded503880 con 0x564ded4efc40 2023-11-15T03:08:49.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.655+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v123'148 uv148 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.655+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded503880 con 0x564ded4efc40 2023-11-15T03:08:49.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.659+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v123'149 uv149 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.659+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded503880 con 0x564ded4efc40 2023-11-15T03:08:49.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.663+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v123'341 uv333 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.663+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded503880 con 0x564ded4efc40 2023-11-15T03:08:49.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.667+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v123'342 uv342 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.667+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded503880 con 0x564ded4efc40 2023-11-15T03:08:49.677 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:08:49.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.667+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v123'343 uv343 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.667+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded503880 con 0x564ded4efc40 2023-11-15T03:08:49.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.671+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v123'344 uv344 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.671+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded503880 con 0x564ded4efc40 2023-11-15T03:08:49.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.675+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v123'345 uv345 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.675+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded503880 con 0x564ded4efc40 2023-11-15T03:08:49.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.675+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v123'346 uv340 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.675+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e123) v8 -- 0x564ded503880 con 0x564ded4efc40 2023-11-15T03:08:49.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.679+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v123'347 uv347 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.679+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded503880 con 0x564ded4efc40 2023-11-15T03:08:49.691 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:08:49.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.683+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v123'348 uv348 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.683+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded504890 con 0x564ded4efc40 2023-11-15T03:08:49.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.683+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v123'349 uv349 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.683+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded504890 con 0x564ded4efc40 2023-11-15T03:08:49.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.687+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v123'350 uv350 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.687+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded503880 con 0x564ded4efc40 2023-11-15T03:08:49.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.691+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v123'351 uv351 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.691+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x564ded503d70 con 0x564ded4efc40 2023-11-15T03:08:49.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.691+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v123'352 uv352 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fbe9c024020 con 0x564ded4efc40 2023-11-15T03:08:49.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.691+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x564ded4f1500 msgr2=0x564ded4f38f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x564ded4f1500 0x564ded4f38f0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x564ded425630 msgr2=0x564ded4b2c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x564ded425630 0x564ded4b2c60 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x564ded4f92b0 msgr2=0x564ded4f96d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x564ded4f92b0 0x564ded4f96d0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x564ded4ec2f0 msgr2=0x564ded4ee6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x564ded4ec2f0 0x564ded4ee6e0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x564ded4f80d0 msgr2=0x564ded503390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x564ded4f80d0 0x564ded503390 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x564ded4efc40 msgr2=0x564ded4f6d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 reap_dead start 2023-11-15T03:08:49.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x564ded4efc40 0x564ded4f6d00 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbea3846700 1 -- 172.21.15.121:0/2941313912 reap_dead start 2023-11-15T03:08:49.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x564ded4fb750 msgr2=0x564ded4fbb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x564ded4fb750 0x564ded4fbb80 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbe84058ff0 msgr2=0x7fbe8405b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbe84058ff0 0x7fbe8405b4b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7fbe94000c00 tx=0x7fbe94001740).stop 2023-11-15T03:08:49.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ded4b42e0 msgr2=0x564ded425140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ded4b42e0 0x564ded425140 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 rx=0x7fbe9c00b010 tx=0x7fbe9c014000).stop 2023-11-15T03:08:49.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 shutdown_connections 2023-11-15T03:08:49.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x564ded4fb750 0x564ded4fbb80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x564ded4efc40 0x564ded4f6d00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbe84058ff0 0x7fbe8405b4b0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 --2- 172.21.15.121:0/2941313912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ded4b42e0 0x564ded425140 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 >> 172.21.15.121:0/2941313912 conn(0x564ded3f67d0 msgr2=0x564ded426970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:49.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 shutdown_connections 2023-11-15T03:08:49.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.695+0000 7fbeaeb37d00 1 -- 172.21.15.121:0/2941313912 wait complete. 2023-11-15T03:08:49.710 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 400.00000000 0 2023-11-15T03:08:49.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.803+0000 7f5b596f0d00 1 Processor -- start 2023-11-15T03:08:49.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.803+0000 7f5b596f0d00 1 -- start start 2023-11-15T03:08:49.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.803+0000 7f5b596f0d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c0c1cb1440 0x55c0c1d58050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.803+0000 7f5b596f0d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0c1ccbc60 0x55c0c1cc9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b596f0d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0c1d3e2e0 0x55c0c1cca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b596f0d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c1c2dcc0 con 0x55c0c1ccbc60 2023-11-15T03:08:49.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b596f0d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c1bf8220 con 0x55c0c1d3e2e0 2023-11-15T03:08:49.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b596f0d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c0c1cca540 con 0x55c0c1cb1440 2023-11-15T03:08:49.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b4e821700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0c1ccbc60 0x55c0c1cc9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b4e821700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0c1ccbc60 0x55c0c1cc9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36586/0 (socket says 172.21.15.121:36586) 2023-11-15T03:08:49.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b4e821700 1 -- 172.21.15.121:0/4272541187 learned_addr learned my addr 172.21.15.121:0/4272541187 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:49.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b57022700 1 --2- 172.21.15.121:0/4272541187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c0c1cb1440 0x55c0c1d58050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b57823700 1 --2- 172.21.15.121:0/4272541187 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0c1d3e2e0 0x55c0c1cca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b4e821700 1 -- 172.21.15.121:0/4272541187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c0c1cb1440 msgr2=0x55c0c1d58050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:49.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b4e821700 1 --2- 172.21.15.121:0/4272541187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c0c1cb1440 0x55c0c1d58050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b4e821700 1 -- 172.21.15.121:0/4272541187 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0c1d3e2e0 msgr2=0x55c0c1cca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:49.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b4e821700 1 --2- 172.21.15.121:0/4272541187 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0c1d3e2e0 0x55c0c1cca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b4e821700 1 -- 172.21.15.121:0/4272541187 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b48005cf0 con 0x55c0c1ccbc60 2023-11-15T03:08:49.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b4e821700 1 --2- 172.21.15.121:0/4272541187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0c1ccbc60 0x55c0c1cc9b60 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 rx=0x7f5b40001e50 tx=0x7f5b400018b0).ready entity=mon.0 client_cookie=420896ddfbb08081 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:49.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b56020700 1 -- 172.21.15.121:0/4272541187 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4000bc20 con 0x55c0c1ccbc60 2023-11-15T03:08:49.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b56020700 1 -- 172.21.15.121:0/4272541187 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b40008070 con 0x55c0c1ccbc60 2023-11-15T03:08:49.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b56020700 1 -- 172.21.15.121:0/4272541187 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4000bdc0 con 0x55c0c1ccbc60 2023-11-15T03:08:49.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b596f0d00 1 -- 172.21.15.121:0/4272541187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0c1ccbc60 msgr2=0x55c0c1cc9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b596f0d00 1 --2- 172.21.15.121:0/4272541187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0c1ccbc60 0x55c0c1cc9b60 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 rx=0x7f5b40001e50 tx=0x7f5b400018b0).stop 2023-11-15T03:08:49.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b596f0d00 1 -- 172.21.15.121:0/4272541187 shutdown_connections 2023-11-15T03:08:49.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b596f0d00 1 --2- 172.21.15.121:0/4272541187 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0c1d3e2e0 0x55c0c1cca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b596f0d00 1 --2- 172.21.15.121:0/4272541187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0c1ccbc60 0x55c0c1cc9b60 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b596f0d00 1 --2- 172.21.15.121:0/4272541187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c0c1cb1440 0x55c0c1d58050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b596f0d00 1 -- 172.21.15.121:0/4272541187 >> 172.21.15.121:0/4272541187 conn(0x55c0c1c807d0 msgr2=0x55c0c1ccc270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:49.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b596f0d00 1 -- 172.21.15.121:0/4272541187 shutdown_connections 2023-11-15T03:08:49.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.807+0000 7f5b596f0d00 1 -- 172.21.15.121:0/4272541187 wait complete. 2023-11-15T03:08:49.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b596f0d00 1 Processor -- start 2023-11-15T03:08:49.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b596f0d00 1 -- start start 2023-11-15T03:08:49.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b596f0d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0c1cb1440 0x55c0c1cc7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b596f0d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0c1ccbc60 0x55c0c1d549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b596f0d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c0c1d3e2e0 0x55c0c1cb0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b596f0d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c1cae940 con 0x55c0c1cb1440 2023-11-15T03:08:49.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b596f0d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c1cad900 con 0x55c0c1ccbc60 2023-11-15T03:08:49.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b596f0d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c0c1cadba0 con 0x55c0c1d3e2e0 2023-11-15T03:08:49.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b57022700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0c1cb1440 0x55c0c1cc7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b57022700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0c1cb1440 0x55c0c1cc7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36588/0 (socket says 172.21.15.121:36588) 2023-11-15T03:08:49.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b57022700 1 -- 172.21.15.121:0/1429284360 learned_addr learned my addr 172.21.15.121:0/1429284360 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:49.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b4e821700 1 --2- 172.21.15.121:0/1429284360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0c1ccbc60 0x55c0c1d549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b57823700 1 --2- 172.21.15.121:0/1429284360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c0c1d3e2e0 0x55c0c1cb0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b57022700 1 -- 172.21.15.121:0/1429284360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c0c1d3e2e0 msgr2=0x55c0c1cb0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b57022700 1 --2- 172.21.15.121:0/1429284360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c0c1d3e2e0 0x55c0c1cb0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b57022700 1 -- 172.21.15.121:0/1429284360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0c1ccbc60 msgr2=0x55c0c1d549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b57022700 1 --2- 172.21.15.121:0/1429284360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0c1ccbc60 0x55c0c1d549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b57022700 1 -- 172.21.15.121:0/1429284360 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4000a040 con 0x55c0c1cb1440 2023-11-15T03:08:49.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b57823700 1 --2- 172.21.15.121:0/1429284360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c0c1d3e2e0 0x55c0c1cb0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:08:49.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b57022700 1 --2- 172.21.15.121:0/1429284360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0c1cb1440 0x55c0c1cc7ca0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 rx=0x55c0c1c6cc50 tx=0x7f5b48016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:49.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b5481d700 1 -- 172.21.15.121:0/1429284360 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b48008250 con 0x55c0c1cb1440 2023-11-15T03:08:49.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b5481d700 1 -- 172.21.15.121:0/1429284360 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4801e480 con 0x55c0c1cb1440 2023-11-15T03:08:49.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b596f0d00 1 -- 172.21.15.121:0/1429284360 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c0c1bf0a70 con 0x55c0c1cb1440 2023-11-15T03:08:49.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b5481d700 1 -- 172.21.15.121:0/1429284360 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4801e840 con 0x55c0c1cb1440 2023-11-15T03:08:49.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.811+0000 7f5b596f0d00 1 -- 172.21.15.121:0/1429284360 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0c1bfc010 con 0x55c0c1cb1440 2023-11-15T03:08:49.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.815+0000 7f5b5481d700 1 -- 172.21.15.121:0/1429284360 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5b4801ec70 con 0x55c0c1cb1440 2023-11-15T03:08:49.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.815+0000 7f5b5481d700 1 --2- 172.21.15.121:0/1429284360 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b38059020 0x7f5b3805b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.815+0000 7f5b5481d700 1 -- 172.21.15.121:0/1429284360 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8760+0+0 (secure 0 0 0) 0x7f5b48075b00 con 0x55c0c1cb1440 2023-11-15T03:08:49.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.815+0000 7f5b4e821700 1 --2- 172.21.15.121:0/1429284360 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b38059020 0x7f5b3805b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.819+0000 7f5b596f0d00 1 --2- 172.21.15.121:0/1429284360 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c0c1d54e90 0x55c0c1d3b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:49.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.819+0000 7f5b596f0d00 1 -- 172.21.15.121:0/1429284360 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected e123) v8 -- 0x55c0c1d3d000 con 0x55c0c1d54e90 2023-11-15T03:08:49.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.819+0000 7f5b4e821700 1 --2- 172.21.15.121:0/1429284360 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b38059020 0x7f5b3805b4e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x55c0c1cc9020 tx=0x7f5b400018b0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:49.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.819+0000 7f5b57823700 1 --2- 172.21.15.121:0/1429284360 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c0c1d54e90 0x55c0c1d3b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:49.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.819+0000 7f5b57823700 1 --2- 172.21.15.121:0/1429284360 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c0c1d54e90 0x55c0c1d3b510 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:49.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.823+0000 7f5b57823700 1 -- 172.21.15.121:0/1429284360 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 400.00000000 [truncate 0] v123'150 uv150 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b50006cd0 con 0x55c0c1d54e90 2023-11-15T03:08:49.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.823+0000 7f5b596f0d00 1 -- 172.21.15.121:0/1429284360 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c0c1d54e90 msgr2=0x55c0c1d3b510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.823+0000 7f5b596f0d00 1 --2- 172.21.15.121:0/1429284360 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55c0c1d54e90 0x55c0c1d3b510 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:49.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.823+0000 7f5b596f0d00 1 -- 172.21.15.121:0/1429284360 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b38059020 msgr2=0x7f5b3805b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.823+0000 7f5b596f0d00 1 --2- 172.21.15.121:0/1429284360 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b38059020 0x7f5b3805b4e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x55c0c1cc9020 tx=0x7f5b400018b0).stop 2023-11-15T03:08:49.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.823+0000 7f5b596f0d00 1 -- 172.21.15.121:0/1429284360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0c1cb1440 msgr2=0x55c0c1cc7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:49.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.823+0000 7f5b596f0d00 1 --2- 172.21.15.121:0/1429284360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0c1cb1440 0x55c0c1cc7ca0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 rx=0x55c0c1c6cc50 tx=0x7f5b48016000).stop 2023-11-15T03:08:49.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.823+0000 7f5b57823700 1 -- 172.21.15.121:0/1429284360 reap_dead start 2023-11-15T03:08:49.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.823+0000 7f5b596f0d00 1 -- 172.21.15.121:0/1429284360 shutdown_connections 2023-11-15T03:08:49.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.823+0000 7f5b596f0d00 1 -- 172.21.15.121:0/1429284360 >> 172.21.15.121:0/1429284360 conn(0x55c0c1c807d0 msgr2=0x55c0c1cc9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:49.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.823+0000 7f5b596f0d00 1 -- 172.21.15.121:0/1429284360 shutdown_connections 2023-11-15T03:08:49.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.823+0000 7f5b596f0d00 1 -- 172.21.15.121:0/1429284360 wait complete. 2023-11-15T03:08:49.840 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:08:50.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.995+0000 7fa2c94db700 1 Processor -- start 2023-11-15T03:08:50.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.995+0000 7fa2c94db700 1 -- start start 2023-11-15T03:08:50.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c94db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa2c4067ec0 0x7fa2c40682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:50.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c94db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2c4153700 0x7fa2c4153b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:50.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c94db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c40c0c20 con 0x7fa2c40c10b0 2023-11-15T03:08:50.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c94db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c4158730 con 0x7fa2c4067ec0 2023-11-15T03:08:50.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c94db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40c02f0 con 0x7fa2c4153700 2023-11-15T03:08:50.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c27fc700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa2c40c10b0 0x7fa2c4062c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50002/0 (socket says 172.21.15.121:50002) 2023-11-15T03:08:50.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c27fc700 1 -- 172.21.15.121:0/249025995 learned_addr learned my addr 172.21.15.121:0/249025995 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:50.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c2ffd700 1 --2- 172.21.15.121:0/249025995 >> v2:172.21.15.121:3300/0 conn(0x7fa2c4067ec0 0x7fa2c40682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:50.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c37fe700 1 --2- 172.21.15.121:0/249025995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2c4153700 0x7fa2c4153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:50.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c1ffb700 1 -- 172.21.15.121:0/249025995 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307835905 0 0) 0x7fa2c40c0c20 con 0x7fa2c40c10b0 2023-11-15T03:08:50.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c1ffb700 1 -- 172.21.15.121:0/249025995 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2ac0033f0 con 0x7fa2c40c10b0 2023-11-15T03:08:50.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c1ffb700 1 -- 172.21.15.121:0/249025995 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 221836653 0 0) 0x7fa2ac0033f0 con 0x7fa2c40c10b0 2023-11-15T03:08:50.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c1ffb700 1 -- 172.21.15.121:0/249025995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2c4153700 msgr2=0x7fa2c4153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:50.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c1ffb700 1 --2- 172.21.15.121:0/249025995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2c4153700 0x7fa2c4153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:50.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c1ffb700 1 -- 172.21.15.121:0/249025995 >> v2:172.21.15.121:3300/0 conn(0x7fa2c4067ec0 msgr2=0x7fa2c40682a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:50.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c1ffb700 1 --2- 172.21.15.121:0/249025995 >> v2:172.21.15.121:3300/0 conn(0x7fa2c4067ec0 0x7fa2c40682a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:50.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c1ffb700 1 -- 172.21.15.121:0/249025995 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c40c05d0 con 0x7fa2c40c10b0 2023-11-15T03:08:50.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c1ffb700 1 -- 172.21.15.121:0/249025995 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fa2b8001470 con 0x7fa2c40c10b0 2023-11-15T03:08:50.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c1ffb700 1 -- 172.21.15.121:0/249025995 >> v1:172.21.15.121:6789/0 conn(0x7fa2c40c10b0 legacy=0x7fa2c4062c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:50.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c1ffb700 1 --2- 172.21.15.121:0/249025995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2c4153700 0x7fa2ac0038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:50.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c1ffb700 1 --2- 172.21.15.121:0/249025995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2ac003e10 0x7fa2ac004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:50.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c1ffb700 1 --2- 172.21.15.121:0/249025995 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2ac004b80 0x7fa2ac005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:50.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c1ffb700 1 -- 172.21.15.121:0/249025995 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c4158730 con 0x7fa2ac003e10 2023-11-15T03:08:50.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c1ffb700 1 -- 172.21.15.121:0/249025995 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40c02f0 con 0x7fa2ac004b80 2023-11-15T03:08:50.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c1ffb700 1 -- 172.21.15.121:0/249025995 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac00b8e0 con 0x7fa2c4153700 2023-11-15T03:08:50.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c27fc700 1 --2- 172.21.15.121:0/249025995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2ac003e10 0x7fa2ac004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:50.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c2ffd700 1 --2- 172.21.15.121:0/249025995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2c4153700 0x7fa2ac0038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:50.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c37fe700 1 --2- 172.21.15.121:0/249025995 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2ac004b80 0x7fa2ac005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:50.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c27fc700 1 -- 172.21.15.121:0/249025995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2c4153700 msgr2=0x7fa2ac0038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:50.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c27fc700 1 --2- 172.21.15.121:0/249025995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2c4153700 0x7fa2ac0038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:50.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c27fc700 1 -- 172.21.15.121:0/249025995 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2ac004b80 msgr2=0x7fa2ac005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:50.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c27fc700 1 --2- 172.21.15.121:0/249025995 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2ac004b80 0x7fa2ac005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:50.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c27fc700 1 -- 172.21.15.121:0/249025995 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac0033f0 con 0x7fa2ac003e10 2023-11-15T03:08:50.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:49.999+0000 7fa2c2ffd700 1 --2- 172.21.15.121:0/249025995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2c4153700 0x7fa2ac0038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:08:50.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.003+0000 7fa2c27fc700 1 --2- 172.21.15.121:0/249025995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2ac003e10 0x7fa2ac004690 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 rx=0x7fa2b800c240 tx=0x7fa2c40c0b90).ready entity=mon.0 client_cookie=5cfcbfca37357604 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:50.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.003+0000 7fa2c1ffb700 1 -- 172.21.15.121:0/249025995 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8007a60 con 0x7fa2ac003e10 2023-11-15T03:08:50.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.003+0000 7fa2c1ffb700 1 -- 172.21.15.121:0/249025995 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b800cad0 con 0x7fa2ac003e10 2023-11-15T03:08:50.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.003+0000 7fa2c1ffb700 1 -- 172.21.15.121:0/249025995 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b800ce70 con 0x7fa2ac003e10 2023-11-15T03:08:50.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.003+0000 7fa2c94db700 1 -- 172.21.15.121:0/249025995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2ac003e10 msgr2=0x7fa2ac004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:50.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.003+0000 7fa2c94db700 1 --2- 172.21.15.121:0/249025995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2ac003e10 0x7fa2ac004690 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 rx=0x7fa2b800c240 tx=0x7fa2c40c0b90).stop 2023-11-15T03:08:50.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.003+0000 7fa2c37fe700 1 -- 172.21.15.121:0/249025995 reap_dead start 2023-11-15T03:08:50.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.003+0000 7fa2c94db700 1 -- 172.21.15.121:0/249025995 shutdown_connections 2023-11-15T03:08:50.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.003+0000 7fa2c94db700 1 -- 172.21.15.121:0/249025995 >> 172.21.15.121:0/249025995 conn(0x7fa2c40cad40 msgr2=0x7fa2c40cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:50.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.003+0000 7fa2c94db700 1 -- 172.21.15.121:0/249025995 shutdown_connections 2023-11-15T03:08:50.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.003+0000 7fa2c94db700 1 -- 172.21.15.121:0/249025995 wait complete. 2023-11-15T03:08:50.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.007+0000 7fa2c94db700 1 Processor -- start 2023-11-15T03:08:50.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.007+0000 7fa2c94db700 1 -- start start 2023-11-15T03:08:50.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.007+0000 7fa2c94db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2c4143440 0x7fa2c4143820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:50.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.007+0000 7fa2c94db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2c40c3e50 0x7fa2c41545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:50.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.007+0000 7fa2c94db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2c4154ab0 0x7fa2c40c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:50.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.007+0000 7fa2c94db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41562e0 con 0x7fa2c40c3e50 2023-11-15T03:08:50.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.007+0000 7fa2c94db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40c43c0 con 0x7fa2c4154ab0 2023-11-15T03:08:50.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.007+0000 7fa2c94db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40c46a0 con 0x7fa2c4143440 2023-11-15T03:08:50.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.007+0000 7fa2c27fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2c40c3e50 0x7fa2c41545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:50.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.007+0000 7fa2c27fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2c40c3e50 0x7fa2c41545c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36610/0 (socket says 172.21.15.121:36610) 2023-11-15T03:08:50.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.007+0000 7fa2c27fc700 1 -- 172.21.15.121:0/1451061344 learned_addr learned my addr 172.21.15.121:0/1451061344 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:50.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.007+0000 7fa2c37fe700 1 --2- 172.21.15.121:0/1451061344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2c4154ab0 0x7fa2c40c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:50.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.007+0000 7fa2c2ffd700 1 --2- 172.21.15.121:0/1451061344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2c4143440 0x7fa2c4143820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:50.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.007+0000 7fa2c27fc700 1 -- 172.21.15.121:0/1451061344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2c4143440 msgr2=0x7fa2c4143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:50.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.007+0000 7fa2c27fc700 1 --2- 172.21.15.121:0/1451061344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2c4143440 0x7fa2c4143820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:50.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.007+0000 7fa2c27fc700 1 -- 172.21.15.121:0/1451061344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2c4154ab0 msgr2=0x7fa2c40c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:50.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.007+0000 7fa2c27fc700 1 --2- 172.21.15.121:0/1451061344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2c4154ab0 0x7fa2c40c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:50.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.007+0000 7fa2c27fc700 1 -- 172.21.15.121:0/1451061344 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c40c4980 con 0x7fa2c40c3e50 2023-11-15T03:08:50.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.007+0000 7fa2c37fe700 1 --2- 172.21.15.121:0/1451061344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2c4154ab0 0x7fa2c40c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:50.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.011+0000 7fa2c27fc700 1 --2- 172.21.15.121:0/1451061344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2c40c3e50 0x7fa2c41545c0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 rx=0x7fa2b800a000 tx=0x7fa2b8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:50.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.011+0000 7fa2aaffd700 1 -- 172.21.15.121:0/1451061344 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8007640 con 0x7fa2c40c3e50 2023-11-15T03:08:50.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.011+0000 7fa2aaffd700 1 -- 172.21.15.121:0/1451061344 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b8018070 con 0x7fa2c40c3e50 2023-11-15T03:08:50.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.011+0000 7fa2c94db700 1 -- 172.21.15.121:0/1451061344 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c40c4c60 con 0x7fa2c40c3e50 2023-11-15T03:08:50.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.011+0000 7fa2aaffd700 1 -- 172.21.15.121:0/1451061344 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8004bd0 con 0x7fa2c40c3e50 2023-11-15T03:08:50.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.011+0000 7fa2c94db700 1 -- 172.21.15.121:0/1451061344 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c40c6010 con 0x7fa2c40c3e50 2023-11-15T03:08:50.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.011+0000 7fa2c94db700 1 -- 172.21.15.121:0/1451061344 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c4156120 con 0x7fa2c40c3e50 2023-11-15T03:08:50.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.011+0000 7fa2aaffd700 1 -- 172.21.15.121:0/1451061344 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa2b8013450 con 0x7fa2c40c3e50 2023-11-15T03:08:50.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.011+0000 7fa2aaffd700 1 --2- 172.21.15.121:0/1451061344 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa2b005a540 0x7fa2b005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:50.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.011+0000 7fa2aaffd700 1 -- 172.21.15.121:0/1451061344 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8760+0+0 (secure 0 0 0) 0x7fa2b8075ec0 con 0x7fa2c40c3e50 2023-11-15T03:08:50.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.015+0000 7fa2c2ffd700 1 --2- 172.21.15.121:0/1451061344 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa2b005a540 0x7fa2b005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:50.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.015+0000 7fa2c2ffd700 1 --2- 172.21.15.121:0/1451061344 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa2b005a540 0x7fa2b005ca00 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7fa2b4006d80 tx=0x7fa2b4008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:50.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.019+0000 7fa2aaffd700 1 -- 172.21.15.121:0/1451061344 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa2c4156120 con 0x7fa2c40c3e50 2023-11-15T03:08:50.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.215+0000 7fa2c94db700 1 -- 172.21.15.121:0/1451061344 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fa2c4156120 con 0x7fa2c40c3e50 2023-11-15T03:08:50.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.235+0000 7fa2aaffd700 1 -- 172.21.15.121:0/1451061344 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v263) v1 ==== 162+0+0 (secure 0 0 0) 0x7fa2c4156120 con 0x7fa2c40c3e50 2023-11-15T03:08:50.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.239+0000 7fa2c94db700 1 -- 172.21.15.121:0/1451061344 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa2b005a540 msgr2=0x7fa2b005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:50.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.239+0000 7fa2c94db700 1 --2- 172.21.15.121:0/1451061344 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa2b005a540 0x7fa2b005ca00 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7fa2b4006d80 tx=0x7fa2b4008040).stop 2023-11-15T03:08:50.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.239+0000 7fa2c94db700 1 -- 172.21.15.121:0/1451061344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2c40c3e50 msgr2=0x7fa2c41545c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:50.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.239+0000 7fa2c94db700 1 --2- 172.21.15.121:0/1451061344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2c40c3e50 0x7fa2c41545c0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 rx=0x7fa2b800a000 tx=0x7fa2b8015000).stop 2023-11-15T03:08:50.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.239+0000 7fa2c94db700 1 -- 172.21.15.121:0/1451061344 shutdown_connections 2023-11-15T03:08:50.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.239+0000 7fa2c94db700 1 --2- 172.21.15.121:0/1451061344 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa2b005a540 0x7fa2b005ca00 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:50.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.239+0000 7fa2c94db700 1 --2- 172.21.15.121:0/1451061344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2c4154ab0 0x7fa2c40c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:50.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.239+0000 7fa2c94db700 1 --2- 172.21.15.121:0/1451061344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2c40c3e50 0x7fa2c41545c0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:50.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.239+0000 7fa2c94db700 1 --2- 172.21.15.121:0/1451061344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2c4143440 0x7fa2c4143820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:50.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.239+0000 7fa2c94db700 1 -- 172.21.15.121:0/1451061344 >> 172.21.15.121:0/1451061344 conn(0x7fa2c40cad40 msgr2=0x7fa2c40c6810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:50.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.239+0000 7fa2c94db700 1 -- 172.21.15.121:0/1451061344 shutdown_connections 2023-11-15T03:08:50.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.239+0000 7fa2c94db700 1 -- 172.21.15.121:0/1451061344 wait complete. 2023-11-15T03:08:50.255 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:08:50.256 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:08:50.247+0000 7fe4e4f45700 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2023-11-15T03:08:50.270 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:50.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.415+0000 7f152c9f9700 1 Processor -- start 2023-11-15T03:08:50.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.415+0000 7f152c9f9700 1 -- start start 2023-11-15T03:08:50.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.419+0000 7f152c9f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15280cabd0 0x7f15280cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:50.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.419+0000 7f152c9f9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f15280acc90 0x7f1528066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:50.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.419+0000 7f152c9f9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15280ad650 con 0x7f15280ad0f0 2023-11-15T03:08:50.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.419+0000 7f152c9f9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1528065fc0 con 0x7f15280acc90 2023-11-15T03:08:50.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.419+0000 7f152c9f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15280662a0 con 0x7f15280cabd0 2023-11-15T03:08:50.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f15277fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15280cabd0 0x7f15280cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:50.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f1527fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f15280ad0f0 0x7f15280ae190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50004/0 (socket says 172.21.15.121:50004) 2023-11-15T03:08:50.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f1527fff700 1 -- 172.21.15.121:0/3581599022 learned_addr learned my addr 172.21.15.121:0/3581599022 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:50.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f1526ffd700 1 --2- 172.21.15.121:0/3581599022 >> v2:172.21.15.121:3300/0 conn(0x7f15280acc90 0x7f1528066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:50.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f1526ffd700 1 -- 172.21.15.121:0/3581599022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15280cabd0 msgr2=0x7f15280cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:50.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f1526ffd700 1 --2- 172.21.15.121:0/3581599022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15280cabd0 0x7f15280cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:50.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f1526ffd700 1 -- 172.21.15.121:0/3581599022 >> v1:172.21.15.121:6789/0 conn(0x7f15280ad0f0 legacy=0x7f15280ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:50.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f15267fc700 1 -- 172.21.15.121:0/3581599022 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631280812 0 0) 0x7f15280ad650 con 0x7f15280ad0f0 2023-11-15T03:08:50.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f1526ffd700 1 -- 172.21.15.121:0/3581599022 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1528066580 con 0x7f15280acc90 2023-11-15T03:08:50.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f15277fe700 1 --2- 172.21.15.121:0/3581599022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15280cabd0 0x7f15280cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:08:50.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f1526ffd700 1 --2- 172.21.15.121:0/3581599022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15280acc90 0x7f1528066d80 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 rx=0x7f151c009790 tx=0x7f151c009060).ready entity=mon.0 client_cookie=41d2bb84dce04bea server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:50.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f15267fc700 1 -- 172.21.15.121:0/3581599022 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c00f510 con 0x7f15280acc90 2023-11-15T03:08:50.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f15267fc700 1 -- 172.21.15.121:0/3581599022 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151c00f670 con 0x7f15280acc90 2023-11-15T03:08:50.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f15267fc700 1 -- 172.21.15.121:0/3581599022 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c00f9f0 con 0x7f15280acc90 2023-11-15T03:08:50.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f152c9f9700 1 -- 172.21.15.121:0/3581599022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15280acc90 msgr2=0x7f1528066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:50.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f152c9f9700 1 --2- 172.21.15.121:0/3581599022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15280acc90 0x7f1528066d80 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 rx=0x7f151c009790 tx=0x7f151c009060).stop 2023-11-15T03:08:50.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f152c9f9700 1 -- 172.21.15.121:0/3581599022 shutdown_connections 2023-11-15T03:08:50.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f152c9f9700 1 --2- 172.21.15.121:0/3581599022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15280acc90 0x7f1528066d80 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:50.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f152c9f9700 1 --2- 172.21.15.121:0/3581599022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15280cabd0 0x7f15280cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:50.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f152c9f9700 1 -- 172.21.15.121:0/3581599022 >> 172.21.15.121:0/3581599022 conn(0x7f1528062190 msgr2=0x7f15280625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:50.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f152c9f9700 1 -- 172.21.15.121:0/3581599022 shutdown_connections 2023-11-15T03:08:50.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.423+0000 7f152c9f9700 1 -- 172.21.15.121:0/3581599022 wait complete. 2023-11-15T03:08:50.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.427+0000 7f152c9f9700 1 Processor -- start 2023-11-15T03:08:50.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.427+0000 7f152c9f9700 1 -- start start 2023-11-15T03:08:50.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f152c9f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15280acc90 0x7f15280c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:50.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f152c9f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15280ad0f0 0x7f152814f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:50.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f152c9f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15280cabd0 0x7f152814f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:50.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f152c9f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152816cc10 con 0x7f15280cabd0 2023-11-15T03:08:50.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f152c9f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15281691e0 con 0x7f15280acc90 2023-11-15T03:08:50.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f152c9f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152814feb0 con 0x7f15280ad0f0 2023-11-15T03:08:50.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f1527fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15280cabd0 0x7f152814f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:50.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f1527fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15280cabd0 0x7f152814f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36630/0 (socket says 172.21.15.121:36630) 2023-11-15T03:08:50.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f1527fff700 1 -- 172.21.15.121:0/585111270 learned_addr learned my addr 172.21.15.121:0/585111270 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:50.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f1526ffd700 1 --2- 172.21.15.121:0/585111270 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15280ad0f0 0x7f152814f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:50.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f15277fe700 1 --2- 172.21.15.121:0/585111270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15280acc90 0x7f15280c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:50.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f1527fff700 1 -- 172.21.15.121:0/585111270 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15280ad0f0 msgr2=0x7f152814f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:50.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f1527fff700 1 --2- 172.21.15.121:0/585111270 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15280ad0f0 0x7f152814f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:50.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f1527fff700 1 -- 172.21.15.121:0/585111270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15280acc90 msgr2=0x7f15280c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:50.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f1527fff700 1 --2- 172.21.15.121:0/585111270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15280acc90 0x7f15280c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:50.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f1527fff700 1 -- 172.21.15.121:0/585111270 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15180092a0 con 0x7f15280cabd0 2023-11-15T03:08:50.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f1527fff700 1 --2- 172.21.15.121:0/585111270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15280cabd0 0x7f152814f9c0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 rx=0x7f152000c840 tx=0x7f1520007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:50.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f150f7fe700 1 -- 172.21.15.121:0/585111270 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520005d30 con 0x7f15280cabd0 2023-11-15T03:08:50.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f150f7fe700 1 -- 172.21.15.121:0/585111270 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1520005e90 con 0x7f15280cabd0 2023-11-15T03:08:50.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f152c9f9700 1 -- 172.21.15.121:0/585111270 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151c0092a0 con 0x7f15280cabd0 2023-11-15T03:08:50.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f150f7fe700 1 -- 172.21.15.121:0/585111270 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520007360 con 0x7f15280cabd0 2023-11-15T03:08:50.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.431+0000 7f152c9f9700 1 -- 172.21.15.121:0/585111270 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1528168930 con 0x7f15280cabd0 2023-11-15T03:08:50.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.435+0000 7f152c9f9700 1 -- 172.21.15.121:0/585111270 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152806bcf0 con 0x7f15280cabd0 2023-11-15T03:08:50.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.435+0000 7f150f7fe700 1 -- 172.21.15.121:0/585111270 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1520007710 con 0x7f15280cabd0 2023-11-15T03:08:50.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.435+0000 7f150f7fe700 1 --2- 172.21.15.121:0/585111270 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f151405a510 0x7f151405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:50.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.435+0000 7f150f7fe700 1 -- 172.21.15.121:0/585111270 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8846+0+0 (secure 0 0 0) 0x7f1520075300 con 0x7f15280cabd0 2023-11-15T03:08:50.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.435+0000 7f15277fe700 1 --2- 172.21.15.121:0/585111270 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f151405a510 0x7f151405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:50.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.439+0000 7f15277fe700 1 --2- 172.21.15.121:0/585111270 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f151405a510 0x7f151405c9d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f1518009270 tx=0x7f1518008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:50.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.439+0000 7f150f7fe700 1 -- 172.21.15.121:0/585111270 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1520047f20 con 0x7f15280cabd0 2023-11-15T03:08:50.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.631+0000 7f152c9f9700 1 -- 172.21.15.121:0/585111270 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1528068a70 con 0x7f15280cabd0 2023-11-15T03:08:50.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.631+0000 7f150f7fe700 1 -- 172.21.15.121:0/585111270 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 264 v264) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f152004b540 con 0x7f15280cabd0 2023-11-15T03:08:50.642 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:50.642 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":264,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":264,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:08:50.239251+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9648},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9648":{"gid":9648,"name":"a","rank":0,"incarnation":264,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/3050548459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3050548459},{"type":"v1","addr":"172.21.15.121:6836","nonce":3050548459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:50.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.635+0000 7f152c9f9700 1 -- 172.21.15.121:0/585111270 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f151405a510 msgr2=0x7f151405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:50.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.635+0000 7f152c9f9700 1 --2- 172.21.15.121:0/585111270 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f151405a510 0x7f151405c9d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f1518009270 tx=0x7f1518008040).stop 2023-11-15T03:08:50.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.635+0000 7f152c9f9700 1 -- 172.21.15.121:0/585111270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15280cabd0 msgr2=0x7f152814f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:50.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.635+0000 7f152c9f9700 1 --2- 172.21.15.121:0/585111270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15280cabd0 0x7f152814f9c0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 rx=0x7f152000c840 tx=0x7f1520007f30).stop 2023-11-15T03:08:50.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.635+0000 7f152c9f9700 1 -- 172.21.15.121:0/585111270 shutdown_connections 2023-11-15T03:08:50.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.635+0000 7f152c9f9700 1 --2- 172.21.15.121:0/585111270 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f151405a510 0x7f151405c9d0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:50.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.635+0000 7f152c9f9700 1 --2- 172.21.15.121:0/585111270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15280cabd0 0x7f152814f9c0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:50.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.635+0000 7f152c9f9700 1 --2- 172.21.15.121:0/585111270 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15280ad0f0 0x7f152814f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:50.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.635+0000 7f152c9f9700 1 --2- 172.21.15.121:0/585111270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15280acc90 0x7f15280c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:50.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.635+0000 7f152c9f9700 1 -- 172.21.15.121:0/585111270 >> 172.21.15.121:0/585111270 conn(0x7f1528062190 msgr2=0x7f1528151700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:50.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.635+0000 7f152c9f9700 1 -- 172.21.15.121:0/585111270 shutdown_connections 2023-11-15T03:08:50.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:50.635+0000 7f152c9f9700 1 -- 172.21.15.121:0/585111270 wait complete. 2023-11-15T03:08:50.648 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 264 2023-11-15T03:08:50.663 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:08:51.250 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:08:51.243+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:08:51.251 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:08:51.243+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:08:51.268 INFO:tasks.ceph.mds.a.smithi121.stderr: -26> 2023-11-15T03:08:50.247+0000 7fe4e4f45700 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2023-11-15T03:08:51.305 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:08:55.665 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:55.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.823+0000 7ff38b439700 1 Processor -- start 2023-11-15T03:08:55.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.823+0000 7ff38b439700 1 -- start start 2023-11-15T03:08:55.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.827+0000 7ff38b439700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff384151870 0x7ff384151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:55.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.827+0000 7ff38b439700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3840c30e0 0x7ff3840c3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:55.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.827+0000 7ff38b439700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3840c3a80 con 0x7ff384152140 2023-11-15T03:08:55.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.827+0000 7ff38b439700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3840657f0 con 0x7ff384151870 2023-11-15T03:08:55.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.827+0000 7ff38b439700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff384065ad0 con 0x7ff3840c30e0 2023-11-15T03:08:55.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.827+0000 7ff38a437700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff384151870 0x7ff384151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:55.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.827+0000 7ff389c36700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff384152140 0x7ff3840a9830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34656/0 (socket says 172.21.15.121:34656) 2023-11-15T03:08:55.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.827+0000 7ff389c36700 1 -- 172.21.15.121:0/2328074939 learned_addr learned my addr 172.21.15.121:0/2328074939 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:55.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.827+0000 7ff38ac38700 1 --2- 172.21.15.121:0/2328074939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3840c30e0 0x7ff3840c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:55.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.827+0000 7ff38a437700 1 -- 172.21.15.121:0/2328074939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3840c30e0 msgr2=0x7ff3840c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:55.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.827+0000 7ff38a437700 1 --2- 172.21.15.121:0/2328074939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3840c30e0 0x7ff3840c3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:55.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.827+0000 7ff38a437700 1 -- 172.21.15.121:0/2328074939 >> v1:172.21.15.121:6789/0 conn(0x7ff384152140 legacy=0x7ff3840a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:55.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.827+0000 7ff38a437700 1 -- 172.21.15.121:0/2328074939 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3840aa260 con 0x7ff384151870 2023-11-15T03:08:55.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.827+0000 7ff389435700 1 -- 172.21.15.121:0/2328074939 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2048865798 0 0) 0x7ff374001110 con 0x7ff384152140 2023-11-15T03:08:55.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.827+0000 7ff38a437700 1 --2- 172.21.15.121:0/2328074939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff384151870 0x7ff384151c50 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 rx=0x7ff380009e60 tx=0x7ff380009780).ready entity=mon.0 client_cookie=4e96b68f56fd8b31 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:55.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.827+0000 7ff389435700 1 -- 172.21.15.121:0/2328074939 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38000e070 con 0x7ff384151870 2023-11-15T03:08:55.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.827+0000 7ff389435700 1 -- 172.21.15.121:0/2328074939 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38000bb60 con 0x7ff384151870 2023-11-15T03:08:55.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.827+0000 7ff389435700 1 -- 172.21.15.121:0/2328074939 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380012410 con 0x7ff384151870 2023-11-15T03:08:55.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.831+0000 7ff38b439700 1 -- 172.21.15.121:0/2328074939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff384151870 msgr2=0x7ff384151c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:55.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.831+0000 7ff38b439700 1 --2- 172.21.15.121:0/2328074939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff384151870 0x7ff384151c50 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 rx=0x7ff380009e60 tx=0x7ff380009780).stop 2023-11-15T03:08:55.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.831+0000 7ff38b439700 1 -- 172.21.15.121:0/2328074939 shutdown_connections 2023-11-15T03:08:55.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.831+0000 7ff38b439700 1 --2- 172.21.15.121:0/2328074939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3840c30e0 0x7ff3840c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:55.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.831+0000 7ff38b439700 1 --2- 172.21.15.121:0/2328074939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff384151870 0x7ff384151c50 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:55.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.831+0000 7ff38b439700 1 -- 172.21.15.121:0/2328074939 >> 172.21.15.121:0/2328074939 conn(0x7ff384062190 msgr2=0x7ff384062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:55.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.831+0000 7ff38b439700 1 -- 172.21.15.121:0/2328074939 shutdown_connections 2023-11-15T03:08:55.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.831+0000 7ff38b439700 1 -- 172.21.15.121:0/2328074939 wait complete. 2023-11-15T03:08:55.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.831+0000 7ff38b439700 1 Processor -- start 2023-11-15T03:08:55.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.831+0000 7ff38b439700 1 -- start start 2023-11-15T03:08:55.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff38b439700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3840c30e0 0x7ff384156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:55.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff38b439700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff384151870 0x7ff3841678d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:55.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff38b439700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff384152140 0x7ff384167dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:55.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff38b439700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3841682b0 con 0x7ff3840c30e0 2023-11-15T03:08:55.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff38b439700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff384165a80 con 0x7ff384152140 2023-11-15T03:08:55.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff38b439700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff384157050 con 0x7ff384151870 2023-11-15T03:08:55.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff38a437700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3840c30e0 0x7ff384156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:55.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff38a437700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3840c30e0 0x7ff384156b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58392/0 (socket says 172.21.15.121:58392) 2023-11-15T03:08:55.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff38a437700 1 -- 172.21.15.121:0/2384697614 learned_addr learned my addr 172.21.15.121:0/2384697614 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:55.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff389c36700 1 --2- 172.21.15.121:0/2384697614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff384151870 0x7ff3841678d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:55.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff38ac38700 1 --2- 172.21.15.121:0/2384697614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff384152140 0x7ff384167dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:55.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff38a437700 1 -- 172.21.15.121:0/2384697614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff384151870 msgr2=0x7ff3841678d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:55.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff38a437700 1 --2- 172.21.15.121:0/2384697614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff384151870 0x7ff3841678d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:55.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff38a437700 1 -- 172.21.15.121:0/2384697614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff384152140 msgr2=0x7ff384167dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:55.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff38a437700 1 --2- 172.21.15.121:0/2384697614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff384152140 0x7ff384167dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:55.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff38a437700 1 -- 172.21.15.121:0/2384697614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff380009a20 con 0x7ff3840c30e0 2023-11-15T03:08:55.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff389c36700 1 --2- 172.21.15.121:0/2384697614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff384151870 0x7ff3841678d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:55.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff38ac38700 1 --2- 172.21.15.121:0/2384697614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff384152140 0x7ff384167dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:55.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff38a437700 1 --2- 172.21.15.121:0/2384697614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3840c30e0 0x7ff384156b60 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 rx=0x7ff38000b6c0 tx=0x7ff380009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:55.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff37a7fc700 1 -- 172.21.15.121:0/2384697614 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380009440 con 0x7ff3840c30e0 2023-11-15T03:08:55.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff37a7fc700 1 -- 172.21.15.121:0/2384697614 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38000bea0 con 0x7ff3840c30e0 2023-11-15T03:08:55.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff38b439700 1 -- 172.21.15.121:0/2384697614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38414a210 con 0x7ff3840c30e0 2023-11-15T03:08:55.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff37a7fc700 1 -- 172.21.15.121:0/2384697614 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38001d880 con 0x7ff3840c30e0 2023-11-15T03:08:55.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.835+0000 7ff38b439700 1 -- 172.21.15.121:0/2384697614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff384165710 con 0x7ff3840c30e0 2023-11-15T03:08:55.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.839+0000 7ff38b439700 1 -- 172.21.15.121:0/2384697614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38406bcf0 con 0x7ff3840c30e0 2023-11-15T03:08:55.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.839+0000 7ff37a7fc700 1 -- 172.21.15.121:0/2384697614 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff380003040 con 0x7ff3840c30e0 2023-11-15T03:08:55.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.839+0000 7ff37a7fc700 1 --2- 172.21.15.121:0/2384697614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff36805a510 0x7ff36805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:55.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.839+0000 7ff37a7fc700 1 -- 172.21.15.121:0/2384697614 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8846+0+0 (secure 0 0 0) 0x7ff380076ff0 con 0x7ff3840c30e0 2023-11-15T03:08:55.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.839+0000 7ff389c36700 1 --2- 172.21.15.121:0/2384697614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff36805a510 0x7ff36805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:55.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.843+0000 7ff389c36700 1 --2- 172.21.15.121:0/2384697614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff36805a510 0x7ff36805c9d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7ff374007fd0 tx=0x7ff374009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:55.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:55.847+0000 7ff37a7fc700 1 -- 172.21.15.121:0/2384697614 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff380049cf0 con 0x7ff3840c30e0 2023-11-15T03:08:56.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.051+0000 7ff38b439700 1 -- 172.21.15.121:0/2384697614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3840aa540 con 0x7ff3840c30e0 2023-11-15T03:08:56.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.051+0000 7ff37a7fc700 1 -- 172.21.15.121:0/2384697614 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 266 v266) v1 ==== 95+0+4496 (secure 0 0 0) 0x7ff38004d310 con 0x7ff3840c30e0 2023-11-15T03:08:56.059 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:56.060 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":266,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9726,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2666431227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2666431227},{"type":"v1","addr":"172.21.15.121:6836","nonce":2666431227}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":266}],"filesystems":[{"mdsmap":{"epoch":265,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:08:51.246149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:56.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.051+0000 7ff38b439700 1 -- 172.21.15.121:0/2384697614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff36805a510 msgr2=0x7ff36805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:56.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.051+0000 7ff38b439700 1 --2- 172.21.15.121:0/2384697614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff36805a510 0x7ff36805c9d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7ff374007fd0 tx=0x7ff374009040).stop 2023-11-15T03:08:56.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.051+0000 7ff38b439700 1 -- 172.21.15.121:0/2384697614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3840c30e0 msgr2=0x7ff384156b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:56.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.051+0000 7ff38b439700 1 --2- 172.21.15.121:0/2384697614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3840c30e0 0x7ff384156b60 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 rx=0x7ff38000b6c0 tx=0x7ff380009780).stop 2023-11-15T03:08:56.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.051+0000 7ff38b439700 1 -- 172.21.15.121:0/2384697614 shutdown_connections 2023-11-15T03:08:56.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.051+0000 7ff38b439700 1 --2- 172.21.15.121:0/2384697614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff36805a510 0x7ff36805c9d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:56.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.051+0000 7ff38b439700 1 --2- 172.21.15.121:0/2384697614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff384152140 0x7ff384167dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:56.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.051+0000 7ff38b439700 1 --2- 172.21.15.121:0/2384697614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff384151870 0x7ff3841678d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:56.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.051+0000 7ff38b439700 1 --2- 172.21.15.121:0/2384697614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3840c30e0 0x7ff384156b60 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:56.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.051+0000 7ff38b439700 1 -- 172.21.15.121:0/2384697614 >> 172.21.15.121:0/2384697614 conn(0x7ff384062190 msgr2=0x7ff3840bfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:56.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.055+0000 7ff38b439700 1 -- 172.21.15.121:0/2384697614 shutdown_connections 2023-11-15T03:08:56.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.055+0000 7ff38b439700 1 -- 172.21.15.121:0/2384697614 wait complete. 2023-11-15T03:08:56.065 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 266 2023-11-15T03:08:56.081 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:08:56.081 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:56.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.231+0000 7fc0711a9700 1 Processor -- start 2023-11-15T03:08:56.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.231+0000 7fc0711a9700 1 -- start start 2023-11-15T03:08:56.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.231+0000 7fc0711a9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc06c154ca0 0x7fc06c155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:56.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.231+0000 7fc0711a9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc06c0c4c30 con 0x7fc06c1562f0 2023-11-15T03:08:56.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.231+0000 7fc0711a9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc06c0a6f50 con 0x7fc06c0a57c0 2023-11-15T03:08:56.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.231+0000 7fc0711a9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc06c0a8280 con 0x7fc06c154ca0 2023-11-15T03:08:56.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.231+0000 7fc06ad9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc06c154ca0 0x7fc06c155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:56.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.231+0000 7fc06259c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc06c1562f0 0x7fc06c0a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34672/0 (socket says 172.21.15.121:34672) 2023-11-15T03:08:56.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.231+0000 7fc06259c700 1 -- 172.21.15.121:0/145052746 learned_addr learned my addr 172.21.15.121:0/145052746 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:56.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.231+0000 7fc06a59c700 1 -- 172.21.15.121:0/145052746 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305202324 0 0) 0x7fc06c0c4c30 con 0x7fc06c1562f0 2023-11-15T03:08:56.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.235+0000 7fc06a59c700 1 -- 172.21.15.121:0/145052746 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0580033f0 con 0x7fc06c1562f0 2023-11-15T03:08:56.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.235+0000 7fc06ad9d700 1 -- 172.21.15.121:0/145052746 >> v1:172.21.15.190:6789/0 conn(0x7fc06c0a57c0 legacy=0x7fc06c0c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:56.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.235+0000 7fc06ad9d700 1 -- 172.21.15.121:0/145052746 >> v1:172.21.15.121:6789/0 conn(0x7fc06c1562f0 legacy=0x7fc06c0a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:56.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.235+0000 7fc06ad9d700 1 -- 172.21.15.121:0/145052746 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06c0a7230 con 0x7fc06c154ca0 2023-11-15T03:08:56.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.235+0000 7fc06ad9d700 1 --2- 172.21.15.121:0/145052746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc06c154ca0 0x7fc06c155080 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 rx=0x7fc054009760 tx=0x7fc054009060).ready entity=mon.0 client_cookie=84441ded170340b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:56.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.235+0000 7fc06a59c700 1 -- 172.21.15.121:0/145052746 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05400f510 con 0x7fc06c154ca0 2023-11-15T03:08:56.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.235+0000 7fc06a59c700 1 -- 172.21.15.121:0/145052746 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05400f670 con 0x7fc06c154ca0 2023-11-15T03:08:56.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.235+0000 7fc06a59c700 1 -- 172.21.15.121:0/145052746 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05400f9f0 con 0x7fc06c154ca0 2023-11-15T03:08:56.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.235+0000 7fc0711a9700 1 -- 172.21.15.121:0/145052746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc06c154ca0 msgr2=0x7fc06c155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:56.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.235+0000 7fc0711a9700 1 --2- 172.21.15.121:0/145052746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc06c154ca0 0x7fc06c155080 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 rx=0x7fc054009760 tx=0x7fc054009060).stop 2023-11-15T03:08:56.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.235+0000 7fc0711a9700 1 -- 172.21.15.121:0/145052746 shutdown_connections 2023-11-15T03:08:56.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.235+0000 7fc0711a9700 1 --2- 172.21.15.121:0/145052746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc06c154ca0 0x7fc06c155080 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:56.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.235+0000 7fc0711a9700 1 -- 172.21.15.121:0/145052746 >> 172.21.15.121:0/145052746 conn(0x7fc06c062190 msgr2=0x7fc06c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:56.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.235+0000 7fc0711a9700 1 -- 172.21.15.121:0/145052746 shutdown_connections 2023-11-15T03:08:56.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.235+0000 7fc0711a9700 1 -- 172.21.15.121:0/145052746 wait complete. 2023-11-15T03:08:56.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.239+0000 7fc0711a9700 1 Processor -- start 2023-11-15T03:08:56.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.239+0000 7fc0711a9700 1 -- start start 2023-11-15T03:08:56.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.239+0000 7fc0711a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc06c0a57c0 0x7fc06c0ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:56.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.239+0000 7fc0711a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc06c154ca0 0x7fc06c0a9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:56.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.239+0000 7fc0711a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc06c1562f0 0x7fc06c146dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:56.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.239+0000 7fc0711a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c146930 con 0x7fc06c1562f0 2023-11-15T03:08:56.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.239+0000 7fc0711a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c0aa130 con 0x7fc06c0a57c0 2023-11-15T03:08:56.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.239+0000 7fc0711a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c0aa410 con 0x7fc06c154ca0 2023-11-15T03:08:56.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.243+0000 7fc06b59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc06c1562f0 0x7fc06c146dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:56.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.243+0000 7fc06b59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc06c1562f0 0x7fc06c146dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58416/0 (socket says 172.21.15.121:58416) 2023-11-15T03:08:56.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.243+0000 7fc06b59e700 1 -- 172.21.15.121:0/2787712547 learned_addr learned my addr 172.21.15.121:0/2787712547 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:56.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.243+0000 7fc06ad9d700 1 --2- 172.21.15.121:0/2787712547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc06c0a57c0 0x7fc06c0ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:56.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.243+0000 7fc06259c700 1 --2- 172.21.15.121:0/2787712547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc06c154ca0 0x7fc06c0a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:56.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.243+0000 7fc06b59e700 1 -- 172.21.15.121:0/2787712547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc06c154ca0 msgr2=0x7fc06c0a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:56.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.243+0000 7fc06b59e700 1 --2- 172.21.15.121:0/2787712547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc06c154ca0 0x7fc06c0a9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:56.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.243+0000 7fc06b59e700 1 -- 172.21.15.121:0/2787712547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc06c0a57c0 msgr2=0x7fc06c0ac0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:56.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.243+0000 7fc06b59e700 1 --2- 172.21.15.121:0/2787712547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc06c0a57c0 0x7fc06c0ac0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:56.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.243+0000 7fc06b59e700 1 -- 172.21.15.121:0/2787712547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0540092a0 con 0x7fc06c1562f0 2023-11-15T03:08:56.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.243+0000 7fc06ad9d700 1 --2- 172.21.15.121:0/2787712547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc06c0a57c0 0x7fc06c0ac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:56.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.243+0000 7fc06259c700 1 --2- 172.21.15.121:0/2787712547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc06c154ca0 0x7fc06c0a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:56.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.243+0000 7fc06b59e700 1 --2- 172.21.15.121:0/2787712547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc06c1562f0 0x7fc06c146dc0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 rx=0x7fc064002230 tx=0x7fc0640015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:56.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.243+0000 7fc0637fe700 1 -- 172.21.15.121:0/2787712547 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc064010510 con 0x7fc06c1562f0 2023-11-15T03:08:56.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.243+0000 7fc0637fe700 1 -- 172.21.15.121:0/2787712547 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc064010670 con 0x7fc06c1562f0 2023-11-15T03:08:56.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.243+0000 7fc0637fe700 1 -- 172.21.15.121:0/2787712547 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0640109f0 con 0x7fc06c1562f0 2023-11-15T03:08:56.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.243+0000 7fc0711a9700 1 -- 172.21.15.121:0/2787712547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc06c1465c0 con 0x7fc06c1562f0 2023-11-15T03:08:56.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.243+0000 7fc0711a9700 1 -- 172.21.15.121:0/2787712547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc06c0a9d20 con 0x7fc06c1562f0 2023-11-15T03:08:56.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.243+0000 7fc0711a9700 1 -- 172.21.15.121:0/2787712547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06c146ae0 con 0x7fc06c1562f0 2023-11-15T03:08:56.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.247+0000 7fc0637fe700 1 -- 172.21.15.121:0/2787712547 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc06401b400 con 0x7fc06c1562f0 2023-11-15T03:08:56.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.247+0000 7fc0637fe700 1 --2- 172.21.15.121:0/2787712547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc04c05a510 0x7fc04c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:56.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.247+0000 7fc0637fe700 1 -- 172.21.15.121:0/2787712547 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8846+0+0 (secure 0 0 0) 0x7fc064075080 con 0x7fc06c1562f0 2023-11-15T03:08:56.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.247+0000 7fc06ad9d700 1 --2- 172.21.15.121:0/2787712547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc04c05a510 0x7fc04c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:56.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.247+0000 7fc06ad9d700 1 --2- 172.21.15.121:0/2787712547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc04c05a510 0x7fc04c05c9d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7fc06c0662a0 tx=0x7fc054017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:56.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.255+0000 7fc0637fe700 1 -- 172.21.15.121:0/2787712547 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc06c146ae0 con 0x7fc06c1562f0 2023-11-15T03:08:56.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.451+0000 7fc0711a9700 1 -- 172.21.15.121:0/2787712547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc06c146a70 con 0x7fc06c1562f0 2023-11-15T03:08:56.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.451+0000 7fc0637fe700 1 -- 172.21.15.121:0/2787712547 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 266 v266) v1 ==== 95+0+4496 (secure 0 0 0) 0x7fc064047d20 con 0x7fc06c1562f0 2023-11-15T03:08:56.460 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:56.460 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":266,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9726,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2666431227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2666431227},{"type":"v1","addr":"172.21.15.121:6836","nonce":2666431227}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":266}],"filesystems":[{"mdsmap":{"epoch":265,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:08:51.246149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:56.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.451+0000 7fc0711a9700 1 -- 172.21.15.121:0/2787712547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc04c05a510 msgr2=0x7fc04c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:56.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.451+0000 7fc0711a9700 1 --2- 172.21.15.121:0/2787712547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc04c05a510 0x7fc04c05c9d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7fc06c0662a0 tx=0x7fc054017000).stop 2023-11-15T03:08:56.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.451+0000 7fc0711a9700 1 -- 172.21.15.121:0/2787712547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc06c1562f0 msgr2=0x7fc06c146dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:56.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.451+0000 7fc0711a9700 1 --2- 172.21.15.121:0/2787712547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc06c1562f0 0x7fc06c146dc0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 rx=0x7fc064002230 tx=0x7fc0640015e0).stop 2023-11-15T03:08:56.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.455+0000 7fc0711a9700 1 -- 172.21.15.121:0/2787712547 shutdown_connections 2023-11-15T03:08:56.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.455+0000 7fc0711a9700 1 --2- 172.21.15.121:0/2787712547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc04c05a510 0x7fc04c05c9d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:56.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.455+0000 7fc0711a9700 1 --2- 172.21.15.121:0/2787712547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc06c1562f0 0x7fc06c146dc0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:56.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.455+0000 7fc0711a9700 1 --2- 172.21.15.121:0/2787712547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc06c154ca0 0x7fc06c0a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:56.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.455+0000 7fc0711a9700 1 --2- 172.21.15.121:0/2787712547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc06c0a57c0 0x7fc06c0ac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:56.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.455+0000 7fc0711a9700 1 -- 172.21.15.121:0/2787712547 >> 172.21.15.121:0/2787712547 conn(0x7fc06c062190 msgr2=0x7fc06c155610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:56.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.455+0000 7fc0711a9700 1 -- 172.21.15.121:0/2787712547 shutdown_connections 2023-11-15T03:08:56.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.455+0000 7fc0711a9700 1 -- 172.21.15.121:0/2787712547 wait complete. 2023-11-15T03:08:56.465 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 266 2023-11-15T03:08:56.481 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:08:56.482 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:08:56.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.643+0000 7f27ca0f1700 1 Processor -- start 2023-11-15T03:08:56.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.643+0000 7f27ca0f1700 1 -- start start 2023-11-15T03:08:56.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.647+0000 7f27ca0f1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f27c4068170 0x7f27c415ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:56.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.647+0000 7f27ca0f1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c40cb5c0 con 0x7f27c4153bb0 2023-11-15T03:08:56.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.647+0000 7f27ca0f1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c40c9ca0 con 0x7f27c40aa240 2023-11-15T03:08:56.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.647+0000 7f27ca0f1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c40aa6f0 con 0x7f27c4068170 2023-11-15T03:08:56.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.647+0000 7f27c37fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f27c4153bb0 0x7f27c40a9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34684/0 (socket says 172.21.15.121:34684) 2023-11-15T03:08:56.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.647+0000 7f27c37fe700 1 -- 172.21.15.121:0/3989215786 learned_addr learned my addr 172.21.15.121:0/3989215786 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:56.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.647+0000 7f27c3fff700 1 --2- 172.21.15.121:0/3989215786 >> v2:172.21.15.121:3300/0 conn(0x7f27c4068170 0x7f27c415ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:56.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.651+0000 7f27c27fc700 1 -- 172.21.15.121:0/3989215786 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461282078 0 0) 0x7f27c40cb5c0 con 0x7f27c4153bb0 2023-11-15T03:08:56.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.651+0000 7f27c27fc700 1 -- 172.21.15.121:0/3989215786 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27ac0033f0 con 0x7f27c4153bb0 2023-11-15T03:08:56.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.651+0000 7f27c27fc700 1 -- 172.21.15.121:0/3989215786 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714759312 0 0) 0x7f27c40c9ca0 con 0x7f27c40aa240 2023-11-15T03:08:56.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.651+0000 7f27c27fc700 1 -- 172.21.15.121:0/3989215786 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27c40cb5c0 con 0x7f27c40aa240 2023-11-15T03:08:56.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.651+0000 7f27c3fff700 1 -- 172.21.15.121:0/3989215786 >> v1:172.21.15.190:6789/0 conn(0x7f27c40aa240 legacy=0x7f27c4067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:56.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.651+0000 7f27c3fff700 1 -- 172.21.15.121:0/3989215786 >> v1:172.21.15.121:6789/0 conn(0x7f27c4153bb0 legacy=0x7f27c40a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:56.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.651+0000 7f27c3fff700 1 -- 172.21.15.121:0/3989215786 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c40caa20 con 0x7f27c4068170 2023-11-15T03:08:56.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.651+0000 7f27c3fff700 1 --2- 172.21.15.121:0/3989215786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27c4068170 0x7f27c415ec40 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 rx=0x7f27bc009e60 tx=0x7f27bc009780).ready entity=mon.0 client_cookie=cb572d086343075 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:56.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.651+0000 7f27c27fc700 1 -- 172.21.15.121:0/3989215786 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc00e070 con 0x7f27c4068170 2023-11-15T03:08:56.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.651+0000 7f27c27fc700 1 -- 172.21.15.121:0/3989215786 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27bc00bab0 con 0x7f27c4068170 2023-11-15T03:08:56.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.651+0000 7f27c27fc700 1 -- 172.21.15.121:0/3989215786 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc00be30 con 0x7f27c4068170 2023-11-15T03:08:56.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.651+0000 7f27ca0f1700 1 -- 172.21.15.121:0/3989215786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27c4068170 msgr2=0x7f27c415ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:56.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.651+0000 7f27ca0f1700 1 --2- 172.21.15.121:0/3989215786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27c4068170 0x7f27c415ec40 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 rx=0x7f27bc009e60 tx=0x7f27bc009780).stop 2023-11-15T03:08:56.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.651+0000 7f27ca0f1700 1 -- 172.21.15.121:0/3989215786 shutdown_connections 2023-11-15T03:08:56.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.651+0000 7f27ca0f1700 1 --2- 172.21.15.121:0/3989215786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27c4068170 0x7f27c415ec40 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:56.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.651+0000 7f27ca0f1700 1 -- 172.21.15.121:0/3989215786 >> 172.21.15.121:0/3989215786 conn(0x7f27c4058c50 msgr2=0x7f27c4059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:56.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.651+0000 7f27ca0f1700 1 -- 172.21.15.121:0/3989215786 shutdown_connections 2023-11-15T03:08:56.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.651+0000 7f27ca0f1700 1 -- 172.21.15.121:0/3989215786 wait complete. 2023-11-15T03:08:56.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27ca0f1700 1 Processor -- start 2023-11-15T03:08:56.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27ca0f1700 1 -- start start 2023-11-15T03:08:56.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27ca0f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27c4068170 0x7f27c40c2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:56.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27ca0f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27c40aa240 0x7f27c414eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:56.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27ca0f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27c4153bb0 0x7f27c414d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:56.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27ca0f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c40c4090 con 0x7f27c4153bb0 2023-11-15T03:08:56.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27ca0f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c40c3480 con 0x7f27c40aa240 2023-11-15T03:08:56.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27ca0f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c40c3760 con 0x7f27c4068170 2023-11-15T03:08:56.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27c3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27c4153bb0 0x7f27c414d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:56.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27c3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27c4153bb0 0x7f27c414d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58420/0 (socket says 172.21.15.121:58420) 2023-11-15T03:08:56.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27c3fff700 1 -- 172.21.15.121:0/1615789864 learned_addr learned my addr 172.21.15.121:0/1615789864 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:56.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27c37fe700 1 --2- 172.21.15.121:0/1615789864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27c4068170 0x7f27c40c2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:56.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27c2ffd700 1 --2- 172.21.15.121:0/1615789864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27c40aa240 0x7f27c414eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:56.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27c3fff700 1 -- 172.21.15.121:0/1615789864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27c4068170 msgr2=0x7f27c40c2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:56.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27c3fff700 1 --2- 172.21.15.121:0/1615789864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27c4068170 0x7f27c40c2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:56.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27c3fff700 1 -- 172.21.15.121:0/1615789864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27c40aa240 msgr2=0x7f27c414eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:56.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27c3fff700 1 --2- 172.21.15.121:0/1615789864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27c40aa240 0x7f27c414eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:56.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27c3fff700 1 -- 172.21.15.121:0/1615789864 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27bc009a20 con 0x7f27c4153bb0 2023-11-15T03:08:56.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27c2ffd700 1 --2- 172.21.15.121:0/1615789864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27c40aa240 0x7f27c414eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:08:56.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.655+0000 7f27c37fe700 1 --2- 172.21.15.121:0/1615789864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27c4068170 0x7f27c40c2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:08:56.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.659+0000 7f27c3fff700 1 --2- 172.21.15.121:0/1615789864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27c4153bb0 0x7f27c414d800 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 rx=0x7f27c4065560 tx=0x7f27c40cb530).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:56.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.659+0000 7f27ab7fe700 1 -- 172.21.15.121:0/1615789864 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc01d630 con 0x7f27c4153bb0 2023-11-15T03:08:56.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.659+0000 7f27ab7fe700 1 -- 172.21.15.121:0/1615789864 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27bc01d790 con 0x7f27c4153bb0 2023-11-15T03:08:56.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.659+0000 7f27ab7fe700 1 -- 172.21.15.121:0/1615789864 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc01db10 con 0x7f27c4153bb0 2023-11-15T03:08:56.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.659+0000 7f27ca0f1700 1 -- 172.21.15.121:0/1615789864 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c40c3a40 con 0x7f27c4153bb0 2023-11-15T03:08:56.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.659+0000 7f27ca0f1700 1 -- 172.21.15.121:0/1615789864 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c414efd0 con 0x7f27c4153bb0 2023-11-15T03:08:56.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.659+0000 7f27ca0f1700 1 -- 172.21.15.121:0/1615789864 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c406c150 con 0x7f27c4153bb0 2023-11-15T03:08:56.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.659+0000 7f27ab7fe700 1 -- 172.21.15.121:0/1615789864 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f27bc027460 con 0x7f27c4153bb0 2023-11-15T03:08:56.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.663+0000 7f27ab7fe700 1 --2- 172.21.15.121:0/1615789864 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f27b005a4e0 0x7f27b005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:56.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.663+0000 7f27ab7fe700 1 -- 172.21.15.121:0/1615789864 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8846+0+0 (secure 0 0 0) 0x7f27bc077190 con 0x7f27c4153bb0 2023-11-15T03:08:56.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.663+0000 7f27c37fe700 1 --2- 172.21.15.121:0/1615789864 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f27b005a4e0 0x7f27b005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:56.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.663+0000 7f27c37fe700 1 --2- 172.21.15.121:0/1615789864 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f27b005a4e0 0x7f27b005c9a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f27b40016a0 tx=0x7f27b4009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:56.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.667+0000 7f27ab7fe700 1 -- 172.21.15.121:0/1615789864 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f27bc049e30 con 0x7f27c4153bb0 2023-11-15T03:08:56.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.863+0000 7f27ca0f1700 1 -- 172.21.15.121:0/1615789864 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27c4068ae0 con 0x7f27c4153bb0 2023-11-15T03:08:56.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.863+0000 7f27ab7fe700 1 -- 172.21.15.121:0/1615789864 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 266 v266) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f27bc04d450 con 0x7f27c4153bb0 2023-11-15T03:08:56.872 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:08:56.872 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":266,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9726,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2666431227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2666431227},{"type":"v1","addr":"172.21.15.121:6836","nonce":2666431227}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":266}],"filesystems":[{"mdsmap":{"epoch":265,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:08:51.246149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:08:56.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.863+0000 7f27ca0f1700 1 -- 172.21.15.121:0/1615789864 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f27b005a4e0 msgr2=0x7f27b005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:56.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.863+0000 7f27ca0f1700 1 --2- 172.21.15.121:0/1615789864 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f27b005a4e0 0x7f27b005c9a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f27b40016a0 tx=0x7f27b4009040).stop 2023-11-15T03:08:56.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.863+0000 7f27ca0f1700 1 -- 172.21.15.121:0/1615789864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27c4153bb0 msgr2=0x7f27c414d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:56.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.863+0000 7f27ca0f1700 1 --2- 172.21.15.121:0/1615789864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27c4153bb0 0x7f27c414d800 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 rx=0x7f27c4065560 tx=0x7f27c40cb530).stop 2023-11-15T03:08:56.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.867+0000 7f27ca0f1700 1 -- 172.21.15.121:0/1615789864 shutdown_connections 2023-11-15T03:08:56.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.867+0000 7f27ca0f1700 1 --2- 172.21.15.121:0/1615789864 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f27b005a4e0 0x7f27b005c9a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:56.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.867+0000 7f27ca0f1700 1 --2- 172.21.15.121:0/1615789864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27c4153bb0 0x7f27c414d800 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:56.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.867+0000 7f27ca0f1700 1 --2- 172.21.15.121:0/1615789864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27c40aa240 0x7f27c414eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:56.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.867+0000 7f27ca0f1700 1 --2- 172.21.15.121:0/1615789864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27c4068170 0x7f27c40c2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:56.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.867+0000 7f27ca0f1700 1 -- 172.21.15.121:0/1615789864 >> 172.21.15.121:0/1615789864 conn(0x7f27c4058c50 msgr2=0x7f27c40c14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:56.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.867+0000 7f27ca0f1700 1 -- 172.21.15.121:0/1615789864 shutdown_connections 2023-11-15T03:08:56.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:56.867+0000 7f27ca0f1700 1 -- 172.21.15.121:0/1615789864 wait complete. 2023-11-15T03:08:56.878 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 266 2023-11-15T03:08:56.894 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 400.00000000' led to DAMAGED state 2023-11-15T03:08:56.895 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 100.00000000.inode' 2023-11-15T03:08:56.895 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:08:56.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:08:56.896 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:08:56.914 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:08:56.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:08:56.916 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:08:57.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.107+0000 7fb34c945700 1 Processor -- start 2023-11-15T03:08:57.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.107+0000 7fb34c945700 1 -- start start 2023-11-15T03:08:57.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.107+0000 7fb34c945700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb348154b10 0x7fb3480c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:57.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.107+0000 7fb34c945700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3480a6f60 con 0x7fb3480a6710 2023-11-15T03:08:57.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.107+0000 7fb34c945700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3480c3730 con 0x7fb3480a6af0 2023-11-15T03:08:57.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.107+0000 7fb34c945700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3480c3a10 con 0x7fb348154b10 2023-11-15T03:08:57.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.111+0000 7fb346d9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb3480a6710 0x7fb348154400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34690/0 (socket says 172.21.15.121:34690) 2023-11-15T03:08:57.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.111+0000 7fb346d9d700 1 -- 172.21.15.121:0/2274986805 learned_addr learned my addr 172.21.15.121:0/2274986805 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:57.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.111+0000 7fb34659c700 1 --2- 172.21.15.121:0/2274986805 >> v2:172.21.15.190:3300/0 conn(0x7fb348154b10 0x7fb3480c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:57.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.111+0000 7fb34659c700 1 -- 172.21.15.121:0/2274986805 >> v1:172.21.15.190:6789/0 conn(0x7fb3480a6af0 legacy=0x7fb3480bc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:57.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.111+0000 7fb34659c700 1 -- 172.21.15.121:0/2274986805 >> v1:172.21.15.121:6789/0 conn(0x7fb3480a6710 legacy=0x7fb348154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:57.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.111+0000 7fb34659c700 1 -- 172.21.15.121:0/2274986805 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3480c3cf0 con 0x7fb348154b10 2023-11-15T03:08:57.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.111+0000 7fb345d9b700 1 -- 172.21.15.121:0/2274986805 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475443096 0 0) 0x7fb3480a6f60 con 0x7fb3480a6710 2023-11-15T03:08:57.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.111+0000 7fb34659c700 1 --2- 172.21.15.121:0/2274986805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb348154b10 0x7fb3480c47d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fb33c009760 tx=0x7fb33c009060).ready entity=mon.1 client_cookie=8e5e2cb1581a1d63 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:57.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.115+0000 7fb345d9b700 1 -- 172.21.15.121:0/2274986805 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c00b580 con 0x7fb348154b10 2023-11-15T03:08:57.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.115+0000 7fb345d9b700 1 -- 172.21.15.121:0/2274986805 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c00b6e0 con 0x7fb348154b10 2023-11-15T03:08:57.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.115+0000 7fb345d9b700 1 -- 172.21.15.121:0/2274986805 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c00ba60 con 0x7fb348154b10 2023-11-15T03:08:57.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.115+0000 7fb34c945700 1 -- 172.21.15.121:0/2274986805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb348154b10 msgr2=0x7fb3480c47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:57.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.115+0000 7fb34c945700 1 --2- 172.21.15.121:0/2274986805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb348154b10 0x7fb3480c47d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fb33c009760 tx=0x7fb33c009060).stop 2023-11-15T03:08:57.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.115+0000 7fb34c945700 1 -- 172.21.15.121:0/2274986805 shutdown_connections 2023-11-15T03:08:57.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.115+0000 7fb34c945700 1 --2- 172.21.15.121:0/2274986805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb348154b10 0x7fb3480c47d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:57.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.115+0000 7fb34c945700 1 -- 172.21.15.121:0/2274986805 >> 172.21.15.121:0/2274986805 conn(0x7fb348069640 msgr2=0x7fb348066860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:57.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.115+0000 7fb34c945700 1 -- 172.21.15.121:0/2274986805 shutdown_connections 2023-11-15T03:08:57.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.115+0000 7fb34c945700 1 -- 172.21.15.121:0/2274986805 wait complete. 2023-11-15T03:08:57.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.119+0000 7fb34c945700 1 Processor -- start 2023-11-15T03:08:57.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.119+0000 7fb34c945700 1 -- start start 2023-11-15T03:08:57.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.119+0000 7fb34c945700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3480a6710 0x7fb34814f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:57.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.119+0000 7fb34c945700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3480a6af0 0x7fb3480a8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:57.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.119+0000 7fb34c945700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb348154b10 0x7fb348163e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:57.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.119+0000 7fb34c945700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb348149950 con 0x7fb3480a6af0 2023-11-15T03:08:57.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.119+0000 7fb34c945700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb348149620 con 0x7fb348154b10 2023-11-15T03:08:57.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.119+0000 7fb34c945700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34814ff40 con 0x7fb3480a6710 2023-11-15T03:08:57.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.119+0000 7fb346d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3480a6710 0x7fb34814f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:57.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.119+0000 7fb346d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3480a6710 0x7fb34814f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:50482/0 (socket says 172.21.15.121:50482) 2023-11-15T03:08:57.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.119+0000 7fb346d9d700 1 -- 172.21.15.121:0/580386628 learned_addr learned my addr 172.21.15.121:0/580386628 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:57.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.119+0000 7fb34659c700 1 --2- 172.21.15.121:0/580386628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3480a6af0 0x7fb3480a8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:57.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.119+0000 7fb34759e700 1 --2- 172.21.15.121:0/580386628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb348154b10 0x7fb348163e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:57.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.123+0000 7fb346d9d700 1 -- 172.21.15.121:0/580386628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb348154b10 msgr2=0x7fb348163e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:57.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.123+0000 7fb346d9d700 1 --2- 172.21.15.121:0/580386628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb348154b10 0x7fb348163e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:57.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.123+0000 7fb346d9d700 1 -- 172.21.15.121:0/580386628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3480a6af0 msgr2=0x7fb3480a8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:57.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.123+0000 7fb346d9d700 1 --2- 172.21.15.121:0/580386628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3480a6af0 0x7fb3480a8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:57.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.123+0000 7fb346d9d700 1 -- 172.21.15.121:0/580386628 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c0092a0 con 0x7fb3480a6710 2023-11-15T03:08:57.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.123+0000 7fb346d9d700 1 --2- 172.21.15.121:0/580386628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3480a6710 0x7fb34814f8b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fb33800c490 tx=0x7fb338007e40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:57.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.123+0000 7fb32effd700 1 -- 172.21.15.121:0/580386628 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3380070d0 con 0x7fb3480a6710 2023-11-15T03:08:57.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.123+0000 7fb32effd700 1 -- 172.21.15.121:0/580386628 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb338007230 con 0x7fb3480a6710 2023-11-15T03:08:57.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.123+0000 7fb34c945700 1 -- 172.21.15.121:0/580386628 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb348150220 con 0x7fb3480a6710 2023-11-15T03:08:57.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.123+0000 7fb32effd700 1 -- 172.21.15.121:0/580386628 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3380075f0 con 0x7fb3480a6710 2023-11-15T03:08:57.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.123+0000 7fb34c945700 1 -- 172.21.15.121:0/580386628 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3480a90f0 con 0x7fb3480a6710 2023-11-15T03:08:57.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.123+0000 7fb34c945700 1 -- 172.21.15.121:0/580386628 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34806bcf0 con 0x7fb3480a6710 2023-11-15T03:08:57.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.123+0000 7fb32effd700 1 -- 172.21.15.121:0/580386628 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb338005bc0 con 0x7fb3480a6710 2023-11-15T03:08:57.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.127+0000 7fb32effd700 1 --2- 172.21.15.121:0/580386628 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb33405a510 0x7fb33405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:57.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.127+0000 7fb32effd700 1 -- 172.21.15.121:0/580386628 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8846+0+0 (secure 0 0 0) 0x7fb338074c30 con 0x7fb3480a6710 2023-11-15T03:08:57.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.127+0000 7fb34659c700 1 --2- 172.21.15.121:0/580386628 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb33405a510 0x7fb33405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:57.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.127+0000 7fb34659c700 1 --2- 172.21.15.121:0/580386628 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb33405a510 0x7fb33405c9d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7fb33c009760 tx=0x7fb33c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:57.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.131+0000 7fb32effd700 1 -- 172.21.15.121:0/580386628 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb3380478b0 con 0x7fb3480a6710 2023-11-15T03:08:57.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:57.327+0000 7fb34c945700 1 -- 172.21.15.121:0/580386628 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb34814aff0 con 0x7fb3480a6710 2023-11-15T03:08:58.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.135+0000 7fb32effd700 1 -- 172.21.15.121:0/580386628 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v267) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb338018020 con 0x7fb3480a6710 2023-11-15T03:08:58.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.135+0000 7fb34c945700 1 -- 172.21.15.121:0/580386628 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb33405a510 msgr2=0x7fb33405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:58.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.135+0000 7fb34c945700 1 --2- 172.21.15.121:0/580386628 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb33405a510 0x7fb33405c9d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7fb33c009760 tx=0x7fb33c017000).stop 2023-11-15T03:08:58.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.135+0000 7fb34c945700 1 -- 172.21.15.121:0/580386628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3480a6710 msgr2=0x7fb34814f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:58.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.135+0000 7fb34c945700 1 --2- 172.21.15.121:0/580386628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3480a6710 0x7fb34814f8b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fb33800c490 tx=0x7fb338007e40).stop 2023-11-15T03:08:58.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.139+0000 7fb34c945700 1 -- 172.21.15.121:0/580386628 shutdown_connections 2023-11-15T03:08:58.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.139+0000 7fb34c945700 1 --2- 172.21.15.121:0/580386628 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb33405a510 0x7fb33405c9d0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:58.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.139+0000 7fb34c945700 1 --2- 172.21.15.121:0/580386628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb348154b10 0x7fb348163e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:58.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.139+0000 7fb34c945700 1 --2- 172.21.15.121:0/580386628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3480a6af0 0x7fb3480a8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:58.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.139+0000 7fb34c945700 1 --2- 172.21.15.121:0/580386628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3480a6710 0x7fb34814f8b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:58.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.139+0000 7fb34c945700 1 -- 172.21.15.121:0/580386628 >> 172.21.15.121:0/580386628 conn(0x7fb348069640 msgr2=0x7fb348154020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:58.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.139+0000 7fb34c945700 1 -- 172.21.15.121:0/580386628 shutdown_connections 2023-11-15T03:08:58.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.139+0000 7fb34c945700 1 -- 172.21.15.121:0/580386628 wait complete. 2023-11-15T03:08:58.150 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:08:58.167 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:08:58.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.319+0000 7f3e919f9700 1 Processor -- start 2023-11-15T03:08:58.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.319+0000 7f3e919f9700 1 -- start start 2023-11-15T03:08:58.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.323+0000 7f3e919f9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3e8c153d70 0x7f3e8c0bb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:58.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.323+0000 7f3e919f9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3e8c0c2ec0 0x7f3e8c0c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:58.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.323+0000 7f3e919f9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8c0a8b20 con 0x7f3e8c153990 2023-11-15T03:08:58.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.323+0000 7f3e919f9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e8c0c1760 con 0x7f3e8c0c2ec0 2023-11-15T03:08:58.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.323+0000 7f3e919f9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e8c0a8d50 con 0x7f3e8c153d70 2023-11-15T03:08:58.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.323+0000 7f3e909f7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3e8c153990 0x7f3e8c1581e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34696/0 (socket says 172.21.15.121:34696) 2023-11-15T03:08:58.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.323+0000 7f3e909f7700 1 -- 172.21.15.121:0/3201227786 learned_addr learned my addr 172.21.15.121:0/3201227786 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:58.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.323+0000 7f3e911f8700 1 --2- 172.21.15.121:0/3201227786 >> v2:172.21.15.121:3300/0 conn(0x7f3e8c0c2ec0 0x7f3e8c0c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:58.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.323+0000 7f3e83fff700 1 --2- 172.21.15.121:0/3201227786 >> v2:172.21.15.190:3300/0 conn(0x7f3e8c153d70 0x7f3e8c0bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:58.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.323+0000 7f3e837fe700 1 -- 172.21.15.121:0/3201227786 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458476307 0 0) 0x7f3e8c0a8b20 con 0x7f3e8c153990 2023-11-15T03:08:58.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.323+0000 7f3e837fe700 1 -- 172.21.15.121:0/3201227786 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e740033f0 con 0x7f3e8c153990 2023-11-15T03:08:58.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.323+0000 7f3e911f8700 1 -- 172.21.15.121:0/3201227786 >> v2:172.21.15.190:3300/0 conn(0x7f3e8c153d70 msgr2=0x7f3e8c0bb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:58.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.323+0000 7f3e911f8700 1 --2- 172.21.15.121:0/3201227786 >> v2:172.21.15.190:3300/0 conn(0x7f3e8c153d70 0x7f3e8c0bb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:58.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.323+0000 7f3e911f8700 1 -- 172.21.15.121:0/3201227786 >> v1:172.21.15.121:6789/0 conn(0x7f3e8c153990 legacy=0x7f3e8c1581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:58.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.323+0000 7f3e911f8700 1 -- 172.21.15.121:0/3201227786 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c0a9030 con 0x7f3e8c0c2ec0 2023-11-15T03:08:58.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.323+0000 7f3e911f8700 1 --2- 172.21.15.121:0/3201227786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e8c0c2ec0 0x7f3e8c0c32f0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 rx=0x7f3e8c064880 tx=0x7f3e84009780).ready entity=mon.0 client_cookie=69d7b09ef7bc1fc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:58.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.323+0000 7f3e837fe700 1 -- 172.21.15.121:0/3201227786 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8400e070 con 0x7f3e8c0c2ec0 2023-11-15T03:08:58.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.323+0000 7f3e837fe700 1 -- 172.21.15.121:0/3201227786 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8400bab0 con 0x7f3e8c0c2ec0 2023-11-15T03:08:58.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.323+0000 7f3e837fe700 1 -- 172.21.15.121:0/3201227786 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8400be30 con 0x7f3e8c0c2ec0 2023-11-15T03:08:58.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.327+0000 7f3e919f9700 1 -- 172.21.15.121:0/3201227786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e8c0c2ec0 msgr2=0x7f3e8c0c32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:58.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.327+0000 7f3e919f9700 1 --2- 172.21.15.121:0/3201227786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e8c0c2ec0 0x7f3e8c0c32f0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 rx=0x7f3e8c064880 tx=0x7f3e84009780).stop 2023-11-15T03:08:58.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.327+0000 7f3e919f9700 1 -- 172.21.15.121:0/3201227786 shutdown_connections 2023-11-15T03:08:58.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.327+0000 7f3e919f9700 1 --2- 172.21.15.121:0/3201227786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e8c0c2ec0 0x7f3e8c0c32f0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:58.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.327+0000 7f3e919f9700 1 --2- 172.21.15.121:0/3201227786 >> v2:172.21.15.190:3300/0 conn(0x7f3e8c153d70 0x7f3e8c0bb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:58.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.327+0000 7f3e919f9700 1 -- 172.21.15.121:0/3201227786 >> 172.21.15.121:0/3201227786 conn(0x7f3e8c067a60 msgr2=0x7f3e8c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:58.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.327+0000 7f3e919f9700 1 -- 172.21.15.121:0/3201227786 shutdown_connections 2023-11-15T03:08:58.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.327+0000 7f3e919f9700 1 -- 172.21.15.121:0/3201227786 wait complete. 2023-11-15T03:08:58.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.327+0000 7f3e919f9700 1 Processor -- start 2023-11-15T03:08:58.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.327+0000 7f3e919f9700 1 -- start start 2023-11-15T03:08:58.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e919f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e8c0c2ec0 0x7f3e8c0abb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:58.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e919f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e8c153990 0x7f3e8c149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:58.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e919f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e8c153d70 0x7f3e8c167bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:58.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e919f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0aaad0 con 0x7f3e8c0c2ec0 2023-11-15T03:08:58.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e919f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0aadb0 con 0x7f3e8c153d70 2023-11-15T03:08:58.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e919f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0ab090 con 0x7f3e8c153990 2023-11-15T03:08:58.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e909f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e8c0c2ec0 0x7f3e8c0abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:58.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e909f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e8c0c2ec0 0x7f3e8c0abb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58448/0 (socket says 172.21.15.121:58448) 2023-11-15T03:08:58.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e909f7700 1 -- 172.21.15.121:0/2899143829 learned_addr learned my addr 172.21.15.121:0/2899143829 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:58.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e83fff700 1 --2- 172.21.15.121:0/2899143829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e8c153990 0x7f3e8c149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:58.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e911f8700 1 --2- 172.21.15.121:0/2899143829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e8c153d70 0x7f3e8c167bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:58.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e909f7700 1 -- 172.21.15.121:0/2899143829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e8c153990 msgr2=0x7f3e8c149050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:58.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e909f7700 1 --2- 172.21.15.121:0/2899143829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e8c153990 0x7f3e8c149050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:58.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e909f7700 1 -- 172.21.15.121:0/2899143829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e8c153d70 msgr2=0x7f3e8c167bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:58.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e909f7700 1 --2- 172.21.15.121:0/2899143829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e8c153d70 0x7f3e8c167bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:58.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e909f7700 1 -- 172.21.15.121:0/2899143829 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e84009a20 con 0x7f3e8c0c2ec0 2023-11-15T03:08:58.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e83fff700 1 --2- 172.21.15.121:0/2899143829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e8c153990 0x7f3e8c149050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:58.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e911f8700 1 --2- 172.21.15.121:0/2899143829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e8c153d70 0x7f3e8c167bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:08:58.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e909f7700 1 --2- 172.21.15.121:0/2899143829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e8c0c2ec0 0x7f3e8c0abb70 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 rx=0x7f3e8800c590 tx=0x7f3e88007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:58.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e80ff9700 1 -- 172.21.15.121:0/2899143829 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e880072e0 con 0x7f3e8c0c2ec0 2023-11-15T03:08:58.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e80ff9700 1 -- 172.21.15.121:0/2899143829 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e88007440 con 0x7f3e8c0c2ec0 2023-11-15T03:08:58.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e919f9700 1 -- 172.21.15.121:0/2899143829 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8c0ab370 con 0x7f3e8c0c2ec0 2023-11-15T03:08:58.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e80ff9700 1 -- 172.21.15.121:0/2899143829 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e880057a0 con 0x7f3e8c0c2ec0 2023-11-15T03:08:58.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.331+0000 7f3e919f9700 1 -- 172.21.15.121:0/2899143829 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8c0ac060 con 0x7f3e8c0c2ec0 2023-11-15T03:08:58.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.335+0000 7f3e919f9700 1 -- 172.21.15.121:0/2899143829 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e8c06bcf0 con 0x7f3e8c0c2ec0 2023-11-15T03:08:58.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.335+0000 7f3e80ff9700 1 -- 172.21.15.121:0/2899143829 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3e88005bd0 con 0x7f3e8c0c2ec0 2023-11-15T03:08:58.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.335+0000 7f3e80ff9700 1 --2- 172.21.15.121:0/2899143829 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e7805a510 0x7f3e7805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:58.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.339+0000 7f3e80ff9700 1 -- 172.21.15.121:0/2899143829 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8846+0+0 (secure 0 0 0) 0x7f3e88074890 con 0x7f3e8c0c2ec0 2023-11-15T03:08:58.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.339+0000 7f3e83fff700 1 --2- 172.21.15.121:0/2899143829 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e7805a510 0x7f3e7805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:58.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.339+0000 7f3e83fff700 1 --2- 172.21.15.121:0/2899143829 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e7805a510 0x7f3e7805c9d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f3e7c009200 tx=0x7f3e7c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:58.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.343+0000 7f3e80ff9700 1 -- 172.21.15.121:0/2899143829 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3e88047590 con 0x7f3e8c0c2ec0 2023-11-15T03:08:58.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:58.559+0000 7f3e919f9700 1 -- 172.21.15.121:0/2899143829 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f3e8c065360 con 0x7f3e8c0c2ec0 2023-11-15T03:08:59.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.163+0000 7f3e80ff9700 1 -- 172.21.15.121:0/2899143829 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v268) v1 ==== 101+0+0 (secure 0 0 0) 0x7f3e88018030 con 0x7f3e8c0c2ec0 2023-11-15T03:08:59.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.167+0000 7f3e919f9700 1 -- 172.21.15.121:0/2899143829 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e7805a510 msgr2=0x7f3e7805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.167+0000 7f3e919f9700 1 --2- 172.21.15.121:0/2899143829 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e7805a510 0x7f3e7805c9d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f3e7c009200 tx=0x7f3e7c008040).stop 2023-11-15T03:08:59.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.167+0000 7f3e919f9700 1 -- 172.21.15.121:0/2899143829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e8c0c2ec0 msgr2=0x7f3e8c0abb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.167+0000 7f3e919f9700 1 --2- 172.21.15.121:0/2899143829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e8c0c2ec0 0x7f3e8c0abb70 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 rx=0x7f3e8800c590 tx=0x7f3e88007c80).stop 2023-11-15T03:08:59.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.167+0000 7f3e919f9700 1 -- 172.21.15.121:0/2899143829 shutdown_connections 2023-11-15T03:08:59.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.167+0000 7f3e919f9700 1 --2- 172.21.15.121:0/2899143829 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e7805a510 0x7f3e7805c9d0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.167+0000 7f3e919f9700 1 --2- 172.21.15.121:0/2899143829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e8c153d70 0x7f3e8c167bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.167+0000 7f3e919f9700 1 --2- 172.21.15.121:0/2899143829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e8c153990 0x7f3e8c149050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.167+0000 7f3e919f9700 1 --2- 172.21.15.121:0/2899143829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e8c0c2ec0 0x7f3e8c0abb70 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.167+0000 7f3e919f9700 1 -- 172.21.15.121:0/2899143829 >> 172.21.15.121:0/2899143829 conn(0x7f3e8c067a60 msgr2=0x7f3e8c065780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:59.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.171+0000 7f3e919f9700 1 -- 172.21.15.121:0/2899143829 shutdown_connections 2023-11-15T03:08:59.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.171+0000 7f3e919f9700 1 -- 172.21.15.121:0/2899143829 wait complete. 2023-11-15T03:08:59.183 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:08:59.200 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:08:59.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.255+0000 7fab24137d00 1 Processor -- start 2023-11-15T03:08:59.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.255+0000 7fab24137d00 1 -- start start 2023-11-15T03:08:59.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab24137d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563eb73409b0 0x563eb72f5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab24137d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563eb7342030 0x563eb73228c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab24137d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563eb73d2220 0x563eb7322db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab24137d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eb726e220 con 0x563eb7342030 2023-11-15T03:08:59.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab24137d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eb73d1b60 con 0x563eb73409b0 2023-11-15T03:08:59.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab24137d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x563eb7328250 con 0x563eb73d2220 2023-11-15T03:08:59.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab0ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563eb7342030 0x563eb73228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab187cd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563eb73409b0 0x563eb72f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab0ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563eb7342030 0x563eb73228c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58454/0 (socket says 172.21.15.121:58454) 2023-11-15T03:08:59.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab0ffff700 1 -- 172.21.15.121:0/2983262155 learned_addr learned my addr 172.21.15.121:0/2983262155 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:59.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab18fce700 1 --2- 172.21.15.121:0/2983262155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563eb73d2220 0x563eb7322db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab187cd700 1 -- 172.21.15.121:0/2983262155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563eb73d2220 msgr2=0x563eb7322db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:59.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab187cd700 1 --2- 172.21.15.121:0/2983262155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563eb73d2220 0x563eb7322db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab187cd700 1 -- 172.21.15.121:0/2983262155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563eb7342030 msgr2=0x563eb73228c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:59.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab187cd700 1 --2- 172.21.15.121:0/2983262155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563eb7342030 0x563eb73228c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab187cd700 1 -- 172.21.15.121:0/2983262155 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab08005cf0 con 0x563eb73409b0 2023-11-15T03:08:59.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab0ffff700 1 --2- 172.21.15.121:0/2983262155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563eb7342030 0x563eb73228c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:08:59.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab187cd700 1 --2- 172.21.15.121:0/2983262155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563eb73409b0 0x563eb72f5050 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7fab080098d0 tx=0x7fab0800c040).ready entity=mon.1 client_cookie=bacfd7b85b7a91a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:59.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab177cb700 1 -- 172.21.15.121:0/2983262155 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0800f040 con 0x563eb73409b0 2023-11-15T03:08:59.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab177cb700 1 -- 172.21.15.121:0/2983262155 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab080087e0 con 0x563eb73409b0 2023-11-15T03:08:59.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab177cb700 1 -- 172.21.15.121:0/2983262155 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08008b60 con 0x563eb73409b0 2023-11-15T03:08:59.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab24137d00 1 -- 172.21.15.121:0/2983262155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563eb73409b0 msgr2=0x563eb72f5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab24137d00 1 --2- 172.21.15.121:0/2983262155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563eb73409b0 0x563eb72f5050 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7fab080098d0 tx=0x7fab0800c040).stop 2023-11-15T03:08:59.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab24137d00 1 -- 172.21.15.121:0/2983262155 shutdown_connections 2023-11-15T03:08:59.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab24137d00 1 --2- 172.21.15.121:0/2983262155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563eb73d2220 0x563eb7322db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab24137d00 1 --2- 172.21.15.121:0/2983262155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563eb7342030 0x563eb73228c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab24137d00 1 --2- 172.21.15.121:0/2983262155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563eb73409b0 0x563eb72f5050 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab24137d00 1 -- 172.21.15.121:0/2983262155 >> 172.21.15.121:0/2983262155 conn(0x563eb733f8f0 msgr2=0x563eb72f96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:59.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab24137d00 1 -- 172.21.15.121:0/2983262155 shutdown_connections 2023-11-15T03:08:59.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.259+0000 7fab24137d00 1 -- 172.21.15.121:0/2983262155 wait complete. 2023-11-15T03:08:59.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.263+0000 7fab24137d00 1 Processor -- start 2023-11-15T03:08:59.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.263+0000 7fab24137d00 1 -- start start 2023-11-15T03:08:59.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.263+0000 7fab24137d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563eb73409b0 0x563eb73cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.263+0000 7fab24137d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563eb7342030 0x563eb73dabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.263+0000 7fab24137d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563eb73d2220 0x563eb73d85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.263+0000 7fab24137d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eb7323220 con 0x563eb7342030 2023-11-15T03:08:59.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.263+0000 7fab24137d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eb7322960 con 0x563eb73409b0 2023-11-15T03:08:59.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.263+0000 7fab24137d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x563eb73db0b0 con 0x563eb73d2220 2023-11-15T03:08:59.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.263+0000 7fab0ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563eb7342030 0x563eb73dabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.263+0000 7fab187cd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563eb73409b0 0x563eb73cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.263+0000 7fab18fce700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563eb73d2220 0x563eb73d85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.263+0000 7fab187cd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563eb73409b0 0x563eb73cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:41024/0 (socket says 172.21.15.121:41024) 2023-11-15T03:08:59.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.263+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 learned_addr learned my addr 172.21.15.121:0/520871401 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:59.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.263+0000 7fab18fce700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563eb73d2220 0x563eb73d85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:50496/0 (socket says 172.21.15.121:50496) 2023-11-15T03:08:59.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.263+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563eb73409b0 msgr2=0x563eb73cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.263+0000 7fab18fce700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563eb73409b0 0x563eb73cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.263+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563eb7342030 msgr2=0x563eb73dabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.263+0000 7fab18fce700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563eb7342030 0x563eb73dabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.263+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0000a040 con 0x563eb73d2220 2023-11-15T03:08:59.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.267+0000 7fab18fce700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563eb73d2220 0x563eb73d85e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fab1000a640 tx=0x7fab10014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:59.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.267+0000 7fab15fc8700 1 -- 172.21.15.121:0/520871401 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1000ccf0 con 0x563eb73d2220 2023-11-15T03:08:59.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.267+0000 7fab15fc8700 1 -- 172.21.15.121:0/520871401 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1000ce50 con 0x563eb73d2220 2023-11-15T03:08:59.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.267+0000 7fab15fc8700 1 -- 172.21.15.121:0/520871401 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1000c980 con 0x563eb73d2220 2023-11-15T03:08:59.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.267+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab080094d0 con 0x563eb73d2220 2023-11-15T03:08:59.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.267+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1000a180 con 0x563eb73d2220 2023-11-15T03:08:59.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.271+0000 7fab15fc8700 1 -- 172.21.15.121:0/520871401 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fab1001b040 con 0x563eb73d2220 2023-11-15T03:08:59.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.271+0000 7fab15fc8700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faaf8058ff0 0x7faaf805b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.271+0000 7fab15fc8700 1 -- 172.21.15.121:0/520871401 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8846+0+0 (secure 0 0 0) 0x7fab10075d20 con 0x563eb73d2220 2023-11-15T03:08:59.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.271+0000 7fab187cd700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faaf8058ff0 0x7faaf805b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.285 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:08:59.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.271+0000 7fab24137d00 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x563eb73274e0 0x563eb73236b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.271+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7323ba0 con 0x563eb73274e0 2023-11-15T03:08:59.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.271+0000 7fab0ffff700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x563eb73274e0 0x563eb73236b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.271+0000 7fab187cd700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faaf8058ff0 0x7faaf805b4b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x563eb73d7db0 tx=0x7fab08018000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:59.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.271+0000 7fab0ffff700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x563eb73274e0 0x563eb73236b0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:59.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.275+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v124'353 uv346 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73274e0 2023-11-15T03:08:59.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.275+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb7323f80 con 0x563eb73274e0 2023-11-15T03:08:59.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.279+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v124'354 uv354 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73274e0 2023-11-15T03:08:59.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.279+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73ef640 con 0x563eb73274e0 2023-11-15T03:08:59.290 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:08:59.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.283+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v124'355 uv355 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73274e0 2023-11-15T03:08:59.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.283+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f0a00 con 0x563eb73274e0 2023-11-15T03:08:59.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.283+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v124'356 uv356 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73274e0 2023-11-15T03:08:59.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.283+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f0db0 con 0x563eb73274e0 2023-11-15T03:08:59.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.287+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v124'357 uv357 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73274e0 2023-11-15T03:08:59.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.287+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73ef9f0 con 0x563eb73274e0 2023-11-15T03:08:59.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.291+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v124'358 uv358 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73274e0 2023-11-15T03:08:59.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.291+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73efdf0 con 0x563eb73274e0 2023-11-15T03:08:59.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.291+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v124'359 uv352 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73274e0 2023-11-15T03:08:59.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.291+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb73c9a40 con 0x563eb73274e0 2023-11-15T03:08:59.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.295+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v124'360 uv360 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73274e0 2023-11-15T03:08:59.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.295+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73c9a40 con 0x563eb73274e0 2023-11-15T03:08:59.306 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:08:59.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.299+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v124'361 uv361 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73274e0 2023-11-15T03:08:59.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.299+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73c9a40 con 0x563eb73274e0 2023-11-15T03:08:59.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.299+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v124'362 uv362 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73274e0 2023-11-15T03:08:59.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.299+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73c9a40 con 0x563eb73274e0 2023-11-15T03:08:59.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.303+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v124'363 uv363 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73274e0 2023-11-15T03:08:59.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.303+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73c9a40 con 0x563eb73274e0 2023-11-15T03:08:59.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.307+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v124'364 uv364 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73274e0 2023-11-15T03:08:59.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.307+0000 7fab24137d00 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x563eb73c9e40 0x563eb73f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.307+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73c9a40 con 0x563eb73c9e40 2023-11-15T03:08:59.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.307+0000 7fab187cd700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x563eb73c9e40 0x563eb73f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.307+0000 7fab187cd700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x563eb73c9e40 0x563eb73f3680 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:59.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.311+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v124'556 uv543 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.311+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb73f3b70 con 0x563eb73c9e40 2023-11-15T03:08:59.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.315+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v124'557 uv557 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.315+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f3f20 con 0x563eb73c9e40 2023-11-15T03:08:59.326 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:08:59.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.319+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v124'558 uv558 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.319+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f52e0 con 0x563eb73c9e40 2023-11-15T03:08:59.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.319+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v124'559 uv559 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.319+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f5690 con 0x563eb73c9e40 2023-11-15T03:08:59.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.323+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v124'560 uv560 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.323+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f42d0 con 0x563eb73c9e40 2023-11-15T03:08:59.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.327+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v124'561 uv561 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.327+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f4bc0 con 0x563eb73c9e40 2023-11-15T03:08:59.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.327+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v124'562 uv562 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.327+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f5a40 con 0x563eb73c9e40 2023-11-15T03:08:59.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.331+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v124'563 uv549 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.331+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb73f5a40 con 0x563eb73c9e40 2023-11-15T03:08:59.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.335+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v124'564 uv564 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.335+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f5a40 con 0x563eb73c9e40 2023-11-15T03:08:59.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.335+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v124'565 uv565 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.346 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:08:59.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.335+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f6a50 con 0x563eb73c9e40 2023-11-15T03:08:59.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.339+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v124'566 uv566 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.339+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f6a50 con 0x563eb73c9e40 2023-11-15T03:08:59.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.343+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v124'567 uv567 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.343+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f5a40 con 0x563eb73c9e40 2023-11-15T03:08:59.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.343+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v124'568 uv568 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.343+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f5a40 con 0x563eb73c9e40 2023-11-15T03:08:59.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.347+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v124'569 uv555 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.347+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb73f5a40 con 0x563eb73c9e40 2023-11-15T03:08:59.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.351+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v124'570 uv570 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.351+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f5a40 con 0x563eb73c9e40 2023-11-15T03:08:59.362 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:08:59.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.355+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v124'571 uv571 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.355+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f6a50 con 0x563eb73c9e40 2023-11-15T03:08:59.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.355+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v124'572 uv572 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.355+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f6a50 con 0x563eb73c9e40 2023-11-15T03:08:59.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.359+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v124'573 uv573 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.359+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f5a40 con 0x563eb73c9e40 2023-11-15T03:08:59.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.363+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v124'574 uv574 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.363+0000 7fab24137d00 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x563eb73f5f50 0x563eb73f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.363+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f8830 con 0x563eb73f5f50 2023-11-15T03:08:59.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.363+0000 7fab0ffff700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x563eb73f5f50 0x563eb73f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.363+0000 7fab0ffff700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x563eb73f5f50 0x563eb73f8340 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:59.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.367+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v124'304 uv298 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73f5f50 2023-11-15T03:08:59.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.367+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb73f8830 con 0x563eb73f5f50 2023-11-15T03:08:59.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.371+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v124'305 uv305 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73f5f50 2023-11-15T03:08:59.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.371+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f8830 con 0x563eb73f5f50 2023-11-15T03:08:59.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.371+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v124'306 uv306 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73f5f50 2023-11-15T03:08:59.383 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:08:59.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.375+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f8870 con 0x563eb73f5f50 2023-11-15T03:08:59.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.375+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v124'307 uv307 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73f5f50 2023-11-15T03:08:59.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.375+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f8c70 con 0x563eb73f5f50 2023-11-15T03:08:59.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.379+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v124'308 uv308 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73f5f50 2023-11-15T03:08:59.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.379+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f8c70 con 0x563eb73f5f50 2023-11-15T03:08:59.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.383+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v124'309 uv303 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73f5f50 2023-11-15T03:08:59.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.383+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb73f8830 con 0x563eb73f5f50 2023-11-15T03:08:59.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.387+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v124'310 uv310 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73f5f50 2023-11-15T03:08:59.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.387+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f8830 con 0x563eb73f5f50 2023-11-15T03:08:59.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.387+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v124'311 uv311 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73f5f50 2023-11-15T03:08:59.397 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:08:59.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.387+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f8830 con 0x563eb73f5f50 2023-11-15T03:08:59.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.391+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v124'312 uv312 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73f5f50 2023-11-15T03:08:59.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.391+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f4750 con 0x563eb73f5f50 2023-11-15T03:08:59.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.395+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v124'313 uv313 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73f5f50 2023-11-15T03:08:59.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.395+0000 7fab24137d00 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x563eb73f4750 0x563eb73fb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.395+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73fb800 con 0x563eb73f4750 2023-11-15T03:08:59.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.395+0000 7fab18fce700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x563eb73f4750 0x563eb73fb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.395+0000 7fab18fce700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x563eb73f4750 0x563eb73fb240 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:59.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.399+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v124'355 uv348 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.399+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb73fbcb0 con 0x563eb73f4750 2023-11-15T03:08:59.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.403+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v124'356 uv356 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.403+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73fc1a0 con 0x563eb73f4750 2023-11-15T03:08:59.415 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:08:59.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.407+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v124'357 uv357 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.407+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73f8910 con 0x563eb73f4750 2023-11-15T03:08:59.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.411+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v124'358 uv358 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.411+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7400c20 con 0x563eb73f4750 2023-11-15T03:08:59.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.415+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v124'359 uv359 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.415+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb74007b0 con 0x563eb73f4750 2023-11-15T03:08:59.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.415+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v124'360 uv354 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.419+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb73fca90 con 0x563eb73f4750 2023-11-15T03:08:59.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.419+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v124'361 uv361 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.419+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73fcf50 con 0x563eb73f4750 2023-11-15T03:08:59.431 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:08:59.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.423+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v124'362 uv362 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.423+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73fcf10 con 0x563eb73f4750 2023-11-15T03:08:59.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.427+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v124'363 uv363 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.427+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73fe360 con 0x563eb73f4750 2023-11-15T03:08:59.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.427+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v124'364 uv364 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.427+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb73fcf50 con 0x563eb73f4750 2023-11-15T03:08:59.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.431+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v124'365 uv365 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.431+0000 7fab24137d00 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x563eb73fd860 0x563eb73ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.431+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7400160 con 0x563eb73fd860 2023-11-15T03:08:59.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.431+0000 7fab187cd700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x563eb73fd860 0x563eb73ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.435+0000 7fab187cd700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x563eb73fd860 0x563eb73ffc70 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:59.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.435+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v124'177 uv176 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73fd860 2023-11-15T03:08:59.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.435+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb7400160 con 0x563eb73fd860 2023-11-15T03:08:59.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.439+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v124'178 uv178 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73fd860 2023-11-15T03:08:59.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.439+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7400160 con 0x563eb73fd860 2023-11-15T03:08:59.450 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:08:59.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.443+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v124'179 uv179 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73fd860 2023-11-15T03:08:59.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.443+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7402080 con 0x563eb73fd860 2023-11-15T03:08:59.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.443+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v124'180 uv180 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73fd860 2023-11-15T03:08:59.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.443+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7402080 con 0x563eb73fd860 2023-11-15T03:08:59.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.447+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v124'181 uv181 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73fd860 2023-11-15T03:08:59.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.447+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7401070 con 0x563eb73fd860 2023-11-15T03:08:59.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.451+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v124'182 uv182 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73fd860 2023-11-15T03:08:59.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.451+0000 7fab24137d00 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x563eb7401580 0x563eb7403970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.451+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404000 con 0x563eb7401580 2023-11-15T03:08:59.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.451+0000 7fab0ffff700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x563eb7401580 0x563eb7403970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.451+0000 7fab0ffff700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x563eb7401580 0x563eb7403970 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:59.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.455+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v124'125 uv124 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.455+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb74040e0 con 0x563eb7401580 2023-11-15T03:08:59.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.459+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v124'126 uv126 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.459+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb74040e0 con 0x563eb7401580 2023-11-15T03:08:59.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.459+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v124'127 uv127 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.470 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:08:59.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.459+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb7401580 2023-11-15T03:08:59.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.463+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v124'128 uv128 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.463+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb7401580 2023-11-15T03:08:59.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.467+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v124'177 uv176 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.467+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb7401580 2023-11-15T03:08:59.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.471+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v124'178 uv178 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.471+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb7401580 2023-11-15T03:08:59.484 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:08:59.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.475+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v124'179 uv179 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.475+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405570 con 0x563eb7401580 2023-11-15T03:08:59.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.479+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v124'180 uv180 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.479+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405570 con 0x563eb7401580 2023-11-15T03:08:59.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.479+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v124'181 uv181 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.479+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb7401580 2023-11-15T03:08:59.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.483+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v124'182 uv182 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.483+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb73c9e40 2023-11-15T03:08:59.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.487+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v124'327 uv321 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.487+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb73c9e40 2023-11-15T03:08:59.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.491+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v124'328 uv328 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.491+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb73c9e40 2023-11-15T03:08:59.502 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:08:59.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.495+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v124'329 uv329 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.495+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405570 con 0x563eb73c9e40 2023-11-15T03:08:59.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.495+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v124'330 uv330 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.495+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405570 con 0x563eb73c9e40 2023-11-15T03:08:59.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.499+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v124'331 uv331 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.499+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb73c9e40 2023-11-15T03:08:59.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.503+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v124'332 uv332 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.503+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb73c9e40 2023-11-15T03:08:59.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.507+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v124'333 uv326 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.507+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb73c9e40 2023-11-15T03:08:59.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.511+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v124'334 uv334 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.511+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb73c9e40 2023-11-15T03:08:59.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.515+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v124'335 uv335 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.522 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:08:59.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.515+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb73c9e40 2023-11-15T03:08:59.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.515+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v124'336 uv336 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.515+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb73c9e40 2023-11-15T03:08:59.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.519+0000 7fab187cd700 1 -- 172.21.15.121:0/520871401 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v124'337 uv337 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab08014430 con 0x563eb73c9e40 2023-11-15T03:08:59.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.519+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb73f5f50 2023-11-15T03:08:59.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.523+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v124'148 uv147 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73f5f50 2023-11-15T03:08:59.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.523+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb73f5f50 2023-11-15T03:08:59.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.527+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v124'149 uv149 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73f5f50 2023-11-15T03:08:59.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.527+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb73f5f50 2023-11-15T03:08:59.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.527+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v124'150 uv150 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73f5f50 2023-11-15T03:08:59.537 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:08:59.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.527+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb73f5f50 2023-11-15T03:08:59.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.531+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v124'151 uv151 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73f5f50 2023-11-15T03:08:59.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.531+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb73f5f50 2023-11-15T03:08:59.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.531+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v124'152 uv152 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb73f5f50 2023-11-15T03:08:59.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.535+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb7401580 2023-11-15T03:08:59.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.535+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v124'177 uv176 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.535+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb7401580 2023-11-15T03:08:59.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.539+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v124'178 uv178 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.539+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb7401580 2023-11-15T03:08:59.553 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:08:59.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.543+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v124'179 uv179 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.543+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405570 con 0x563eb7401580 2023-11-15T03:08:59.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.547+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v124'180 uv180 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.547+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405570 con 0x563eb7401580 2023-11-15T03:08:59.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.551+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v124'181 uv181 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.551+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb7401580 2023-11-15T03:08:59.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.555+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v124'182 uv182 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.555+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb7401580 2023-11-15T03:08:59.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.559+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v124'149 uv148 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.559+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb7401580 2023-11-15T03:08:59.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.563+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v124'150 uv150 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.563+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb7401580 2023-11-15T03:08:59.575 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:08:59.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.567+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v124'151 uv151 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.567+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb7401580 2023-11-15T03:08:59.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.567+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v124'152 uv152 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.571+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb7401580 2023-11-15T03:08:59.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.571+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v124'153 uv153 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.571+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb7401580 2023-11-15T03:08:59.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.575+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v124'146 uv145 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.575+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb7401580 2023-11-15T03:08:59.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.579+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v124'147 uv147 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.579+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb7401580 2023-11-15T03:08:59.589 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:08:59.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.579+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v124'148 uv148 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.579+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7404560 con 0x563eb7401580 2023-11-15T03:08:59.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.583+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v124'149 uv149 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.583+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb74072d0 con 0x563eb7401580 2023-11-15T03:08:59.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.587+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v124'150 uv150 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.587+0000 7fab24137d00 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x563eb74050d0 0x563eb74054b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.587+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb74050d0 2023-11-15T03:08:59.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.587+0000 7fab18fce700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x563eb74050d0 0x563eb74054b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.591+0000 7fab18fce700 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x563eb74050d0 0x563eb74054b0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:59.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.591+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v124'118 uv117 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb74050d0 2023-11-15T03:08:59.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.591+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb74050d0 2023-11-15T03:08:59.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.595+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v124'119 uv119 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb74050d0 2023-11-15T03:08:59.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.595+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb74050d0 2023-11-15T03:08:59.608 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:08:59.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.599+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v124'120 uv120 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb74050d0 2023-11-15T03:08:59.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.599+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb74050d0 2023-11-15T03:08:59.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.603+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v124'121 uv121 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb74050d0 2023-11-15T03:08:59.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.603+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb7401580 2023-11-15T03:08:59.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.607+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v124'177 uv176 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.607+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb7401580 2023-11-15T03:08:59.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.607+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v124'178 uv178 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.607+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb7401580 2023-11-15T03:08:59.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.611+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v124'179 uv179 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.622 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:08:59.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.611+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb7401580 2023-11-15T03:08:59.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.615+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v124'180 uv180 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.615+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb7401580 2023-11-15T03:08:59.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.619+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v124'181 uv181 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.619+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb7401580 2023-11-15T03:08:59.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.619+0000 7fab0ffff700 1 -- 172.21.15.121:0/520871401 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v124'182 uv182 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab0000f770 con 0x563eb7401580 2023-11-15T03:08:59.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.619+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb73f4750 2023-11-15T03:08:59.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.623+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v124'151 uv150 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.623+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb73f4750 2023-11-15T03:08:59.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.623+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v124'152 uv152 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.627+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb73f4750 2023-11-15T03:08:59.637 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:08:59.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.627+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v124'153 uv153 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.627+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb73f4750 2023-11-15T03:08:59.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.631+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v124'154 uv154 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.631+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb73f4750 2023-11-15T03:08:59.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.635+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v124'155 uv155 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.635+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb73f4750 2023-11-15T03:08:59.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.635+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v124'353 uv345 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.635+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb73f4750 2023-11-15T03:08:59.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.639+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v124'354 uv354 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.639+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb73f4750 2023-11-15T03:08:59.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.643+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v124'355 uv355 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.654 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:08:59.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.643+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb73f4750 2023-11-15T03:08:59.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.647+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v124'356 uv356 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.647+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405e10 con 0x563eb73f4750 2023-11-15T03:08:59.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.647+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v124'357 uv357 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.647+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405e10 con 0x563eb73f4750 2023-11-15T03:08:59.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.651+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v124'358 uv352 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.651+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e124) v8 -- 0x563eb7405e10 con 0x563eb73f4750 2023-11-15T03:08:59.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.655+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v124'359 uv359 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.655+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb73f4750 2023-11-15T03:08:59.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.655+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v124'360 uv360 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.665 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:08:59.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.655+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb73f4750 2023-11-15T03:08:59.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.659+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v124'361 uv361 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.659+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb73f4750 2023-11-15T03:08:59.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.659+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v124'362 uv362 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.663+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405a10 con 0x563eb73f4750 2023-11-15T03:08:59.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.663+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v124'363 uv363 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.663+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x563eb7405e10 con 0x563eb73f4750 2023-11-15T03:08:59.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v124'364 uv364 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fab10023020 con 0x563eb73f4750 2023-11-15T03:08:59.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x563eb73f5f50 msgr2=0x563eb73f8340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x563eb73f5f50 0x563eb73f8340 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x563eb73274e0 msgr2=0x563eb73236b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x563eb73274e0 0x563eb73236b0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x563eb73fd860 msgr2=0x563eb73ffc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x563eb73fd860 0x563eb73ffc70 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x563eb73c9e40 msgr2=0x563eb73f3680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x563eb73c9e40 0x563eb73f3680 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x563eb74050d0 msgr2=0x563eb74054b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x563eb74050d0 0x563eb74054b0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 reap_dead start 2023-11-15T03:08:59.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x563eb73f4750 msgr2=0x563eb73fb240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x563eb73f4750 0x563eb73fb240 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab18fce700 1 -- 172.21.15.121:0/520871401 reap_dead start 2023-11-15T03:08:59.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x563eb7401580 msgr2=0x563eb7403970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x563eb7401580 0x563eb7403970 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faaf8058ff0 msgr2=0x7faaf805b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faaf8058ff0 0x7faaf805b4b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x563eb73d7db0 tx=0x7fab08018000).stop 2023-11-15T03:08:59.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563eb73d2220 msgr2=0x563eb73d85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563eb73d2220 0x563eb73d85e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fab1000a640 tx=0x7fab10014000).stop 2023-11-15T03:08:59.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 shutdown_connections 2023-11-15T03:08:59.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x563eb7401580 0x563eb7403970 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faaf8058ff0 0x7faaf805b4b0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 --2- 172.21.15.121:0/520871401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563eb73d2220 0x563eb73d85e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 >> 172.21.15.121:0/520871401 conn(0x563eb733f8f0 msgr2=0x563eb72fa8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:59.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 shutdown_connections 2023-11-15T03:08:59.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.667+0000 7fab24137d00 1 -- 172.21.15.121:0/520871401 wait complete. 2023-11-15T03:08:59.686 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 100.00000000.inode 0 2023-11-15T03:08:59.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.787+0000 7fd24c92ad00 1 Processor -- start 2023-11-15T03:08:59.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.787+0000 7fd24c92ad00 1 -- start start 2023-11-15T03:08:59.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.787+0000 7fd24c92ad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f2792b7dd0 0x55f279304190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd24c92ad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f279301b00 0x55f2792ff6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd24c92ad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f279394650 0x55f2792ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd24c92ad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f279265cc0 con 0x55f279301b00 2023-11-15T03:08:59.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd24c92ad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f279230220 con 0x55f279394650 2023-11-15T03:08:59.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd24c92ad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f27938a140 con 0x55f2792b7dd0 2023-11-15T03:08:59.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd243fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f279301b00 0x55f2792ff6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd243fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f279301b00 0x55f2792ff6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58474/0 (socket says 172.21.15.121:58474) 2023-11-15T03:08:59.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd243fff700 1 -- 172.21.15.121:0/2628667938 learned_addr learned my addr 172.21.15.121:0/2628667938 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:59.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd24aa46700 1 --2- 172.21.15.121:0/2628667938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f279394650 0x55f2792ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd24a245700 1 --2- 172.21.15.121:0/2628667938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f2792b7dd0 0x55f279304190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd243fff700 1 -- 172.21.15.121:0/2628667938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f2792b7dd0 msgr2=0x55f279304190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:59.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd243fff700 1 --2- 172.21.15.121:0/2628667938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f2792b7dd0 0x55f279304190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd243fff700 1 -- 172.21.15.121:0/2628667938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f279394650 msgr2=0x55f2792ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:08:59.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd243fff700 1 --2- 172.21.15.121:0/2628667938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f279394650 0x55f2792ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd243fff700 1 -- 172.21.15.121:0/2628667938 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24400a0e0 con 0x55f279301b00 2023-11-15T03:08:59.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd243fff700 1 --2- 172.21.15.121:0/2628667938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f279301b00 0x55f2792ff6c0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 rx=0x7fd234002dd0 tx=0x7fd234002890).ready entity=mon.0 client_cookie=2099dfebaa10e150 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:59.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd249243700 1 -- 172.21.15.121:0/2628667938 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23400bac0 con 0x55f279301b00 2023-11-15T03:08:59.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd249243700 1 -- 172.21.15.121:0/2628667938 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd234008070 con 0x55f279301b00 2023-11-15T03:08:59.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd249243700 1 -- 172.21.15.121:0/2628667938 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23400bc40 con 0x55f279301b00 2023-11-15T03:08:59.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd24c92ad00 1 -- 172.21.15.121:0/2628667938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f279301b00 msgr2=0x55f2792ff6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd24c92ad00 1 --2- 172.21.15.121:0/2628667938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f279301b00 0x55f2792ff6c0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 rx=0x7fd234002dd0 tx=0x7fd234002890).stop 2023-11-15T03:08:59.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd24c92ad00 1 -- 172.21.15.121:0/2628667938 shutdown_connections 2023-11-15T03:08:59.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd24c92ad00 1 --2- 172.21.15.121:0/2628667938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f279394650 0x55f2792ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd24c92ad00 1 --2- 172.21.15.121:0/2628667938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f279301b00 0x55f2792ff6c0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd24c92ad00 1 --2- 172.21.15.121:0/2628667938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f2792b7dd0 0x55f279304190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd24c92ad00 1 -- 172.21.15.121:0/2628667938 >> 172.21.15.121:0/2628667938 conn(0x55f279307080 msgr2=0x55f2792fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:59.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd24c92ad00 1 -- 172.21.15.121:0/2628667938 shutdown_connections 2023-11-15T03:08:59.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd24c92ad00 1 -- 172.21.15.121:0/2628667938 wait complete. 2023-11-15T03:08:59.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.791+0000 7fd24c92ad00 1 Processor -- start 2023-11-15T03:08:59.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd24c92ad00 1 -- start start 2023-11-15T03:08:59.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd24c92ad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f2792b7dd0 0x55f2792ec830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd24c92ad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f279394650 0x55f2793890c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd24c92ad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f27937ba50 0x55f2792fd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd24c92ad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f27938a600 con 0x55f27937ba50 2023-11-15T03:08:59.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd24c92ad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f279300d70 con 0x55f279394650 2023-11-15T03:08:59.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd24c92ad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2793895b0 con 0x55f2792b7dd0 2023-11-15T03:08:59.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd24aa46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f27937ba50 0x55f2792fd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd24a245700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f2792b7dd0 0x55f2792ec830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd243fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f279394650 0x55f2793890c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd24aa46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f27937ba50 0x55f2792fd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58476/0 (socket says 172.21.15.121:58476) 2023-11-15T03:08:59.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd24aa46700 1 -- 172.21.15.121:0/3029443930 learned_addr learned my addr 172.21.15.121:0/3029443930 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:08:59.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd24aa46700 1 -- 172.21.15.121:0/3029443930 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f2792b7dd0 msgr2=0x55f2792ec830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd24aa46700 1 --2- 172.21.15.121:0/3029443930 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f2792b7dd0 0x55f2792ec830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd24aa46700 1 -- 172.21.15.121:0/3029443930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f279394650 msgr2=0x55f2793890c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd24aa46700 1 --2- 172.21.15.121:0/3029443930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f279394650 0x55f2793890c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd24aa46700 1 -- 172.21.15.121:0/3029443930 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23400a040 con 0x55f27937ba50 2023-11-15T03:08:59.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd243fff700 1 --2- 172.21.15.121:0/3029443930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f279394650 0x55f2793890c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:08:59.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd24aa46700 1 --2- 172.21.15.121:0/3029443930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f27937ba50 0x55f2792fd4a0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 rx=0x7fd244011ef0 tx=0x7fd244015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:59.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd242ffd700 1 -- 172.21.15.121:0/3029443930 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd244005a70 con 0x55f27937ba50 2023-11-15T03:08:59.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd242ffd700 1 -- 172.21.15.121:0/3029443930 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24401f070 con 0x55f27937ba50 2023-11-15T03:08:59.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd24c92ad00 1 -- 172.21.15.121:0/3029443930 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f279228a70 con 0x55f27937ba50 2023-11-15T03:08:59.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd242ffd700 1 -- 172.21.15.121:0/3029443930 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd244005870 con 0x55f27937ba50 2023-11-15T03:08:59.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.795+0000 7fd24c92ad00 1 -- 172.21.15.121:0/3029443930 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f279234010 con 0x55f27937ba50 2023-11-15T03:08:59.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.799+0000 7fd242ffd700 1 -- 172.21.15.121:0/3029443930 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd244017070 con 0x55f27937ba50 2023-11-15T03:08:59.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.799+0000 7fd242ffd700 1 --2- 172.21.15.121:0/3029443930 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd22c058ff0 0x7fd22c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.799+0000 7fd242ffd700 1 -- 172.21.15.121:0/3029443930 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8846+0+0 (secure 0 0 0) 0x7fd2440752e0 con 0x55f27937ba50 2023-11-15T03:08:59.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.799+0000 7fd24a245700 1 --2- 172.21.15.121:0/3029443930 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd22c058ff0 0x7fd22c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.799+0000 7fd24c92ad00 1 --2- 172.21.15.121:0/3029443930 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f27937bed0 0x55f2792ffb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.803+0000 7fd24c92ad00 1 -- 172.21.15.121:0/3029443930 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.3f 13:fe07bba1:::100.00000000.inode:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x55f279392a60 con 0x55f27937bed0 2023-11-15T03:08:59.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.803+0000 7fd24a245700 1 --2- 172.21.15.121:0/3029443930 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd22c058ff0 0x7fd22c05b4b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7fd23c001ed0 tx=0x7fd23c001760).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:59.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.803+0000 7fd243fff700 1 --2- 172.21.15.121:0/3029443930 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f27937bed0 0x55f2792ffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.803+0000 7fd243fff700 1 --2- 172.21.15.121:0/3029443930 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f27937bed0 0x55f2792ffb90 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:59.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.807+0000 7fd243fff700 1 -- 172.21.15.121:0/3029443930 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 100.00000000.inode [truncate 0] v124'365 uv365 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fd234016040 con 0x55f27937bed0 2023-11-15T03:08:59.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.807+0000 7fd24c92ad00 1 -- 172.21.15.121:0/3029443930 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f27937bed0 msgr2=0x55f2792ffb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.807+0000 7fd24c92ad00 1 --2- 172.21.15.121:0/3029443930 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55f27937bed0 0x55f2792ffb90 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:08:59.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.807+0000 7fd24c92ad00 1 -- 172.21.15.121:0/3029443930 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd22c058ff0 msgr2=0x7fd22c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.807+0000 7fd24c92ad00 1 --2- 172.21.15.121:0/3029443930 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd22c058ff0 0x7fd22c05b4b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7fd23c001ed0 tx=0x7fd23c001760).stop 2023-11-15T03:08:59.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.807+0000 7fd24c92ad00 1 -- 172.21.15.121:0/3029443930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f27937ba50 msgr2=0x55f2792fd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.807+0000 7fd24c92ad00 1 --2- 172.21.15.121:0/3029443930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f27937ba50 0x55f2792fd4a0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 rx=0x7fd244011ef0 tx=0x7fd244015000).stop 2023-11-15T03:08:59.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.807+0000 7fd24aa46700 1 -- 172.21.15.121:0/3029443930 reap_dead start 2023-11-15T03:08:59.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.807+0000 7fd24c92ad00 1 -- 172.21.15.121:0/3029443930 shutdown_connections 2023-11-15T03:08:59.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.807+0000 7fd24c92ad00 1 -- 172.21.15.121:0/3029443930 >> 172.21.15.121:0/3029443930 conn(0x55f279307080 msgr2=0x55f279304190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:08:59.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.807+0000 7fd24c92ad00 1 -- 172.21.15.121:0/3029443930 shutdown_connections 2023-11-15T03:08:59.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.807+0000 7fd24c92ad00 1 -- 172.21.15.121:0/3029443930 wait complete. 2023-11-15T03:08:59.822 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:08:59.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.983+0000 7fb74621b700 1 Processor -- start 2023-11-15T03:08:59.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.983+0000 7fb74621b700 1 -- start start 2023-11-15T03:08:59.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.983+0000 7fb74621b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb7400639b0 0x7fb7400bcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:08:59.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.983+0000 7fb74621b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7400a9250 con 0x7fb740153c10 2023-11-15T03:08:59.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.983+0000 7fb74621b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7400bd5a0 con 0x7fb7400bb6f0 2023-11-15T03:08:59.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.983+0000 7fb74621b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7400a8920 con 0x7fb7400639b0 2023-11-15T03:08:59.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.983+0000 7fb73f7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb740153c10 0x7fb7400632a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34722/0 (socket says 172.21.15.121:34722) 2023-11-15T03:08:59.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.983+0000 7fb73f7fe700 1 -- 172.21.15.121:0/1435094633 learned_addr learned my addr 172.21.15.121:0/1435094633 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:08:59.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.983+0000 7fb73effd700 1 --2- 172.21.15.121:0/1435094633 >> v2:172.21.15.190:3300/0 conn(0x7fb7400639b0 0x7fb7400bcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:08:59.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.987+0000 7fb73e7fc700 1 -- 172.21.15.121:0/1435094633 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473283775 0 0) 0x7fb7400a9250 con 0x7fb740153c10 2023-11-15T03:08:59.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.987+0000 7fb73e7fc700 1 -- 172.21.15.121:0/1435094633 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7280033f0 con 0x7fb740153c10 2023-11-15T03:08:59.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.987+0000 7fb73e7fc700 1 -- 172.21.15.121:0/1435094633 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4269755343 0 0) 0x7fb7400bd5a0 con 0x7fb7400bb6f0 2023-11-15T03:08:59.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.987+0000 7fb73e7fc700 1 -- 172.21.15.121:0/1435094633 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7400a9250 con 0x7fb7400bb6f0 2023-11-15T03:08:59.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.987+0000 7fb73effd700 1 -- 172.21.15.121:0/1435094633 >> v1:172.21.15.190:6789/0 conn(0x7fb7400bb6f0 legacy=0x7fb740154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.987+0000 7fb73effd700 1 -- 172.21.15.121:0/1435094633 >> v1:172.21.15.121:6789/0 conn(0x7fb740153c10 legacy=0x7fb7400632a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.987+0000 7fb73effd700 1 -- 172.21.15.121:0/1435094633 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7400a8c00 con 0x7fb7400639b0 2023-11-15T03:08:59.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.987+0000 7fb73effd700 1 --2- 172.21.15.121:0/1435094633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7400639b0 0x7fb7400bcfe0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7fb734009760 tx=0x7fb734009060).ready entity=mon.1 client_cookie=5ed034ad73256a81 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:08:59.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.987+0000 7fb73e7fc700 1 -- 172.21.15.121:0/1435094633 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73400d4a0 con 0x7fb7400639b0 2023-11-15T03:08:59.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.987+0000 7fb73e7fc700 1 -- 172.21.15.121:0/1435094633 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73400d600 con 0x7fb7400639b0 2023-11-15T03:08:59.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.987+0000 7fb73e7fc700 1 -- 172.21.15.121:0/1435094633 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73400d980 con 0x7fb7400639b0 2023-11-15T03:08:59.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.987+0000 7fb74621b700 1 -- 172.21.15.121:0/1435094633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7400639b0 msgr2=0x7fb7400bcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:08:59.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.987+0000 7fb74621b700 1 --2- 172.21.15.121:0/1435094633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7400639b0 0x7fb7400bcfe0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7fb734009760 tx=0x7fb734009060).stop 2023-11-15T03:09:00.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.987+0000 7fb74621b700 1 -- 172.21.15.121:0/1435094633 shutdown_connections 2023-11-15T03:09:00.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.987+0000 7fb74621b700 1 --2- 172.21.15.121:0/1435094633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7400639b0 0x7fb7400bcfe0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:00.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.987+0000 7fb74621b700 1 -- 172.21.15.121:0/1435094633 >> 172.21.15.121:0/1435094633 conn(0x7fb7400591c0 msgr2=0x7fb7400c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:00.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.987+0000 7fb74621b700 1 -- 172.21.15.121:0/1435094633 shutdown_connections 2023-11-15T03:09:00.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.987+0000 7fb74621b700 1 -- 172.21.15.121:0/1435094633 wait complete. 2023-11-15T03:09:00.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.991+0000 7fb74621b700 1 Processor -- start 2023-11-15T03:09:00.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.991+0000 7fb74621b700 1 -- start start 2023-11-15T03:09:00.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb74621b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7400639b0 0x7fb74013deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:00.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb74621b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7400bb6f0 0x7fb7400abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:00.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb74621b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb740153c10 0x7fb7401469a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:00.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb74621b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400be250 con 0x7fb740153c10 2023-11-15T03:09:00.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb74621b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400be530 con 0x7fb7400bb6f0 2023-11-15T03:09:00.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb74621b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400be810 con 0x7fb7400639b0 2023-11-15T03:09:00.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb73ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb740153c10 0x7fb7401469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:00.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb73f7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7400639b0 0x7fb74013deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:00.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb73effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7400bb6f0 0x7fb7400abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:00.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb73ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb740153c10 0x7fb7401469a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58478/0 (socket says 172.21.15.121:58478) 2023-11-15T03:09:00.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb73effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7400bb6f0 0x7fb7400abde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:41056/0 (socket says 172.21.15.121:41056) 2023-11-15T03:09:00.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb73ffff700 1 -- 172.21.15.121:0/31559495 learned_addr learned my addr 172.21.15.121:0/31559495 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:00.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb73f7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7400639b0 0x7fb74013deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:50532/0 (socket says 172.21.15.121:50532) 2023-11-15T03:09:00.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb73ffff700 1 -- 172.21.15.121:0/31559495 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7400639b0 msgr2=0x7fb74013deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:00.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb73ffff700 1 --2- 172.21.15.121:0/31559495 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7400639b0 0x7fb74013deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:00.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb73ffff700 1 -- 172.21.15.121:0/31559495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7400bb6f0 msgr2=0x7fb7400abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:00.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb73ffff700 1 --2- 172.21.15.121:0/31559495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7400bb6f0 0x7fb7400abde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:00.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb73ffff700 1 -- 172.21.15.121:0/31559495 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7340092a0 con 0x7fb740153c10 2023-11-15T03:09:00.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb73effd700 1 --2- 172.21.15.121:0/31559495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7400bb6f0 0x7fb7400abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:09:00.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb73ffff700 1 --2- 172.21.15.121:0/31559495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb740153c10 0x7fb7401469a0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 rx=0x7fb73800c790 tx=0x7fb738007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:00.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb7277fe700 1 -- 172.21.15.121:0/31559495 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb738010510 con 0x7fb740153c10 2023-11-15T03:09:00.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb7277fe700 1 -- 172.21.15.121:0/31559495 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb738010670 con 0x7fb740153c10 2023-11-15T03:09:00.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb7277fe700 1 -- 172.21.15.121:0/31559495 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7380109f0 con 0x7fb740153c10 2023-11-15T03:09:00.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.995+0000 7fb74621b700 1 -- 172.21.15.121:0/31559495 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7400beaf0 con 0x7fb740153c10 2023-11-15T03:09:00.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.999+0000 7fb74621b700 1 -- 172.21.15.121:0/31559495 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7400bf0b0 con 0x7fb740153c10 2023-11-15T03:09:00.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.999+0000 7fb74621b700 1 -- 172.21.15.121:0/31559495 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7400bf2a0 con 0x7fb740153c10 2023-11-15T03:09:00.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:08:59.999+0000 7fb7277fe700 1 -- 172.21.15.121:0/31559495 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb73801b400 con 0x7fb740153c10 2023-11-15T03:09:00.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:00.003+0000 7fb7277fe700 1 --2- 172.21.15.121:0/31559495 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb72c05a510 0x7fb72c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:00.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:00.003+0000 7fb7277fe700 1 -- 172.21.15.121:0/31559495 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8846+0+0 (secure 0 0 0) 0x7fb738074ff0 con 0x7fb740153c10 2023-11-15T03:09:00.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:00.003+0000 7fb73f7fe700 1 --2- 172.21.15.121:0/31559495 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb72c05a510 0x7fb72c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:00.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:00.003+0000 7fb73f7fe700 1 --2- 172.21.15.121:0/31559495 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb72c05a510 0x7fb72c05c9d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7fb730005750 tx=0x7fb730009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:00.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:00.011+0000 7fb7277fe700 1 -- 172.21.15.121:0/31559495 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb738047d10 con 0x7fb740153c10 2023-11-15T03:09:00.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:00.283+0000 7fb74621b700 1 -- 172.21.15.121:0/31559495 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fb740047060 con 0x7fb740153c10 2023-11-15T03:09:01.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.151+0000 7fb7277fe700 1 -- 172.21.15.121:0/31559495 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v269) v1 ==== 162+0+0 (secure 0 0 0) 0x7fb73804b330 con 0x7fb740153c10 2023-11-15T03:09:01.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.159+0000 7fb74621b700 1 -- 172.21.15.121:0/31559495 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb72c05a510 msgr2=0x7fb72c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:01.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.159+0000 7fb74621b700 1 --2- 172.21.15.121:0/31559495 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb72c05a510 0x7fb72c05c9d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7fb730005750 tx=0x7fb730009040).stop 2023-11-15T03:09:01.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.159+0000 7fb74621b700 1 -- 172.21.15.121:0/31559495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb740153c10 msgr2=0x7fb7401469a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:01.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.159+0000 7fb74621b700 1 --2- 172.21.15.121:0/31559495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb740153c10 0x7fb7401469a0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 rx=0x7fb73800c790 tx=0x7fb738007f40).stop 2023-11-15T03:09:01.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.159+0000 7fb74621b700 1 -- 172.21.15.121:0/31559495 shutdown_connections 2023-11-15T03:09:01.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.159+0000 7fb74621b700 1 --2- 172.21.15.121:0/31559495 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb72c05a510 0x7fb72c05c9d0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:01.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.159+0000 7fb74621b700 1 --2- 172.21.15.121:0/31559495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb740153c10 0x7fb7401469a0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:01.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.159+0000 7fb74621b700 1 --2- 172.21.15.121:0/31559495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7400bb6f0 0x7fb7400abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:01.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.159+0000 7fb74621b700 1 --2- 172.21.15.121:0/31559495 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7400639b0 0x7fb74013deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:01.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.159+0000 7fb74621b700 1 -- 172.21.15.121:0/31559495 >> 172.21.15.121:0/31559495 conn(0x7fb7400591c0 msgr2=0x7fb7400c39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:01.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.159+0000 7fb74621b700 1 -- 172.21.15.121:0/31559495 shutdown_connections 2023-11-15T03:09:01.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.159+0000 7fb74621b700 1 -- 172.21.15.121:0/31559495 wait complete. 2023-11-15T03:09:01.170 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:09:01.178 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:09:01.171+0000 7fb405ccb700 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2023-11-15T03:09:01.179 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:09:01.171+0000 7fb405ccb700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2023-11-15T03:09:01.191 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:09:01.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.343+0000 7f7a5ae1b700 1 Processor -- start 2023-11-15T03:09:01.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.347+0000 7f7a5ae1b700 1 -- start start 2023-11-15T03:09:01.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.347+0000 7f7a5ae1b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7a540c6280 0x7f7a54153c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:01.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.347+0000 7f7a5ae1b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a540c59b0 con 0x7f7a541500d0 2023-11-15T03:09:01.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.347+0000 7f7a5ae1b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a540c6710 con 0x7f7a540c5e40 2023-11-15T03:09:01.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.347+0000 7f7a5ae1b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a540c4dc0 con 0x7f7a540c6280 2023-11-15T03:09:01.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a58bb7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f7a541500d0 0x7f7a540bd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34726/0 (socket says 172.21.15.121:34726) 2023-11-15T03:09:01.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a58bb7700 1 -- 172.21.15.121:0/1068176587 learned_addr learned my addr 172.21.15.121:0/1068176587 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:09:01.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a593b8700 1 --2- 172.21.15.121:0/1068176587 >> v2:172.21.15.121:3300/0 conn(0x7f7a540c6280 0x7f7a54153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:01.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a537fe700 1 -- 172.21.15.121:0/1068176587 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3400285555 0 0) 0x7f7a540c59b0 con 0x7f7a541500d0 2023-11-15T03:09:01.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a537fe700 1 -- 172.21.15.121:0/1068176587 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a440033f0 con 0x7f7a541500d0 2023-11-15T03:09:01.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a593b8700 1 -- 172.21.15.121:0/1068176587 >> v1:172.21.15.190:6789/0 conn(0x7f7a540c5e40 legacy=0x7f7a54067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:01.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a537fe700 1 -- 172.21.15.121:0/1068176587 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252212671 0 0) 0x7f7a540c6710 con 0x7f7a540c5e40 2023-11-15T03:09:01.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a593b8700 1 -- 172.21.15.121:0/1068176587 >> v1:172.21.15.121:6789/0 conn(0x7f7a541500d0 legacy=0x7f7a540bd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:01.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a593b8700 1 -- 172.21.15.121:0/1068176587 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a540c5080 con 0x7f7a540c6280 2023-11-15T03:09:01.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a593b8700 1 --2- 172.21.15.121:0/1068176587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a540c6280 0x7f7a54153c10 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 rx=0x7f7a4c009e60 tx=0x7f7a4c009780).ready entity=mon.0 client_cookie=3bea204aa0347770 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:01.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a537fe700 1 -- 172.21.15.121:0/1068176587 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c00e070 con 0x7f7a540c6280 2023-11-15T03:09:01.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a537fe700 1 -- 172.21.15.121:0/1068176587 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c00bb60 con 0x7f7a540c6280 2023-11-15T03:09:01.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a537fe700 1 -- 172.21.15.121:0/1068176587 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c00bee0 con 0x7f7a540c6280 2023-11-15T03:09:01.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a5ae1b700 1 -- 172.21.15.121:0/1068176587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a540c6280 msgr2=0x7f7a54153c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:01.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a5ae1b700 1 --2- 172.21.15.121:0/1068176587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a540c6280 0x7f7a54153c10 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 rx=0x7f7a4c009e60 tx=0x7f7a4c009780).stop 2023-11-15T03:09:01.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a5ae1b700 1 -- 172.21.15.121:0/1068176587 shutdown_connections 2023-11-15T03:09:01.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a5ae1b700 1 --2- 172.21.15.121:0/1068176587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a540c6280 0x7f7a54153c10 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:01.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a5ae1b700 1 -- 172.21.15.121:0/1068176587 >> 172.21.15.121:0/1068176587 conn(0x7f7a540c3af0 msgr2=0x7f7a540c3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:01.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a5ae1b700 1 -- 172.21.15.121:0/1068176587 shutdown_connections 2023-11-15T03:09:01.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.351+0000 7f7a5ae1b700 1 -- 172.21.15.121:0/1068176587 wait complete. 2023-11-15T03:09:01.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.355+0000 7f7a5ae1b700 1 Processor -- start 2023-11-15T03:09:01.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.355+0000 7f7a5ae1b700 1 -- start start 2023-11-15T03:09:01.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.355+0000 7f7a5ae1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a540c5e40 0x7f7a540aade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:01.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.355+0000 7f7a5ae1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a540c6280 0x7f7a540a8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:01.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.355+0000 7f7a5ae1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a541500d0 0x7f7a540a93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:01.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.355+0000 7f7a5ae1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a54166250 con 0x7f7a540c6280 2023-11-15T03:09:01.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.355+0000 7f7a5ae1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a540a8a60 con 0x7f7a541500d0 2023-11-15T03:09:01.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.355+0000 7f7a5ae1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a540a7e50 con 0x7f7a540c5e40 2023-11-15T03:09:01.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.355+0000 7f7a53fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a540c6280 0x7f7a540a8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:01.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.355+0000 7f7a53fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a540c6280 0x7f7a540a8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58498/0 (socket says 172.21.15.121:58498) 2023-11-15T03:09:01.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.355+0000 7f7a53fff700 1 -- 172.21.15.121:0/914228077 learned_addr learned my addr 172.21.15.121:0/914228077 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:01.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.355+0000 7f7a593b8700 1 --2- 172.21.15.121:0/914228077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a541500d0 0x7f7a540a93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:01.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.355+0000 7f7a58bb7700 1 --2- 172.21.15.121:0/914228077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a540c5e40 0x7f7a540aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:01.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.355+0000 7f7a53fff700 1 -- 172.21.15.121:0/914228077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a540c5e40 msgr2=0x7f7a540aade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:01.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.355+0000 7f7a53fff700 1 --2- 172.21.15.121:0/914228077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a540c5e40 0x7f7a540aade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:01.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.355+0000 7f7a53fff700 1 -- 172.21.15.121:0/914228077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a541500d0 msgr2=0x7f7a540a93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:01.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.355+0000 7f7a53fff700 1 --2- 172.21.15.121:0/914228077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a541500d0 0x7f7a540a93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:01.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.355+0000 7f7a53fff700 1 -- 172.21.15.121:0/914228077 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c009a20 con 0x7f7a540c6280 2023-11-15T03:09:01.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.355+0000 7f7a58bb7700 1 --2- 172.21.15.121:0/914228077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a540c5e40 0x7f7a540aade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:01.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.359+0000 7f7a53fff700 1 --2- 172.21.15.121:0/914228077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a540c6280 0x7f7a540a8ef0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 rx=0x7f7a40001eb0 tx=0x7f7a40007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:01.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.359+0000 7f7a50ff9700 1 -- 172.21.15.121:0/914228077 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4000f470 con 0x7f7a540c6280 2023-11-15T03:09:01.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.359+0000 7f7a50ff9700 1 -- 172.21.15.121:0/914228077 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4000f5f0 con 0x7f7a540c6280 2023-11-15T03:09:01.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.359+0000 7f7a5ae1b700 1 -- 172.21.15.121:0/914228077 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a540a8130 con 0x7f7a540c6280 2023-11-15T03:09:01.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.359+0000 7f7a50ff9700 1 -- 172.21.15.121:0/914228077 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4000f9b0 con 0x7f7a540c6280 2023-11-15T03:09:01.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.359+0000 7f7a5ae1b700 1 -- 172.21.15.121:0/914228077 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a540a86f0 con 0x7f7a540c6280 2023-11-15T03:09:01.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.359+0000 7f7a5ae1b700 1 -- 172.21.15.121:0/914228077 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5406c150 con 0x7f7a540c6280 2023-11-15T03:09:01.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.359+0000 7f7a50ff9700 1 -- 172.21.15.121:0/914228077 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7a4000fd20 con 0x7f7a540c6280 2023-11-15T03:09:01.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.363+0000 7f7a50ff9700 1 --2- 172.21.15.121:0/914228077 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7a3805a510 0x7f7a3805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:01.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.363+0000 7f7a50ff9700 1 -- 172.21.15.121:0/914228077 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7f7a400749c0 con 0x7f7a540c6280 2023-11-15T03:09:01.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.363+0000 7f7a58bb7700 1 --2- 172.21.15.121:0/914228077 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7a3805a510 0x7f7a3805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:01.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.363+0000 7f7a58bb7700 1 --2- 172.21.15.121:0/914228077 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7a3805a510 0x7f7a3805c9d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f7a48005750 tx=0x7f7a48009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:01.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.371+0000 7f7a50ff9700 1 -- 172.21.15.121:0/914228077 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7a400475e0 con 0x7f7a540c6280 2023-11-15T03:09:01.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.571+0000 7f7a5ae1b700 1 -- 172.21.15.121:0/914228077 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a540bc880 con 0x7f7a540c6280 2023-11-15T03:09:01.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.571+0000 7f7a50ff9700 1 -- 172.21.15.121:0/914228077 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 270 v270) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f7a4004ac00 con 0x7f7a540c6280 2023-11-15T03:09:01.580 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:09:01.581 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":270,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":270,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:09:01.157616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9726},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9726":{"gid":9726,"name":"a","rank":0,"incarnation":270,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/2666431227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2666431227},{"type":"v1","addr":"172.21.15.121:6836","nonce":2666431227}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:09:01.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.575+0000 7f7a5ae1b700 1 -- 172.21.15.121:0/914228077 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7a3805a510 msgr2=0x7f7a3805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:01.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.575+0000 7f7a5ae1b700 1 --2- 172.21.15.121:0/914228077 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7a3805a510 0x7f7a3805c9d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f7a48005750 tx=0x7f7a48009040).stop 2023-11-15T03:09:01.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.575+0000 7f7a5ae1b700 1 -- 172.21.15.121:0/914228077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a540c6280 msgr2=0x7f7a540a8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:01.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.575+0000 7f7a5ae1b700 1 --2- 172.21.15.121:0/914228077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a540c6280 0x7f7a540a8ef0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 rx=0x7f7a40001eb0 tx=0x7f7a40007c70).stop 2023-11-15T03:09:01.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.575+0000 7f7a5ae1b700 1 -- 172.21.15.121:0/914228077 shutdown_connections 2023-11-15T03:09:01.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.575+0000 7f7a5ae1b700 1 --2- 172.21.15.121:0/914228077 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7a3805a510 0x7f7a3805c9d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:01.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.575+0000 7f7a5ae1b700 1 --2- 172.21.15.121:0/914228077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a541500d0 0x7f7a540a93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:01.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.575+0000 7f7a5ae1b700 1 --2- 172.21.15.121:0/914228077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a540c6280 0x7f7a540a8ef0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:01.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.575+0000 7f7a5ae1b700 1 --2- 172.21.15.121:0/914228077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a540c5e40 0x7f7a540aade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:01.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.575+0000 7f7a5ae1b700 1 -- 172.21.15.121:0/914228077 >> 172.21.15.121:0/914228077 conn(0x7f7a540c3af0 msgr2=0x7f7a54062190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:01.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.575+0000 7f7a5ae1b700 1 -- 172.21.15.121:0/914228077 shutdown_connections 2023-11-15T03:09:01.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:01.575+0000 7f7a5ae1b700 1 -- 172.21.15.121:0/914228077 wait complete. 2023-11-15T03:09:01.585 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 270 2023-11-15T03:09:01.602 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:09:02.169 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:09:02.159+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:09:02.170 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:09:02.159+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:09:02.180 INFO:tasks.ceph.mds.a.smithi121.stderr: -25> 2023-11-15T03:09:01.171+0000 7fb405ccb700 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2023-11-15T03:09:02.180 INFO:tasks.ceph.mds.a.smithi121.stderr: -15> 2023-11-15T03:09:01.171+0000 7fb405ccb700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2023-11-15T03:09:02.217 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:09:06.603 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:09:06.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.754+0000 7f2f354f1700 1 Processor -- start 2023-11-15T03:09:06.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.754+0000 7f2f354f1700 1 -- start start 2023-11-15T03:09:06.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f354f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3014af80 0x7f2f300c1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:06.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f354f1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2f300c6640 0x7f2f300c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:06.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f354f1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f30152000 con 0x7f2f300c1bf0 2023-11-15T03:09:06.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f354f1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f300c1270 con 0x7f2f300c6640 2023-11-15T03:09:06.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f354f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f300c6fc0 con 0x7f2f3014af80 2023-11-15T03:09:06.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f2f7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2f300c6640 0x7f2f300c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:06.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f2effd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3014af80 0x7f2f300c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:06.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f2effd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3014af80 0x7f2f300c1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:47824/0 (socket says 172.21.15.121:47824) 2023-11-15T03:09:06.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f2f7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2f300c6640 0x7f2f300c6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:34312/0 (socket says 172.21.15.121:34312) 2023-11-15T03:09:06.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f2effd700 1 -- 172.21.15.121:0/1040747276 learned_addr learned my addr 172.21.15.121:0/1040747276 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:06.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f267fc700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f2f300c1bf0 0x7f2f30150ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:53452/0 (socket says 172.21.15.121:53452) 2023-11-15T03:09:06.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f2e7fc700 1 -- 172.21.15.121:0/1040747276 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 961937895 0 0) 0x7f2f18001110 con 0x7f2f300c1bf0 2023-11-15T03:09:06.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f2e7fc700 1 -- 172.21.15.121:0/1040747276 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f1c0033f0 con 0x7f2f300c1bf0 2023-11-15T03:09:06.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f2f7fe700 1 -- 172.21.15.121:0/1040747276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3014af80 msgr2=0x7f2f300c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:06.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f2f7fe700 1 --2- 172.21.15.121:0/1040747276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3014af80 0x7f2f300c1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:06.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f2f7fe700 1 -- 172.21.15.121:0/1040747276 >> v1:172.21.15.190:6789/0 conn(0x7f2f300c1bf0 legacy=0x7f2f30150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:06.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f2f7fe700 1 -- 172.21.15.121:0/1040747276 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f30151a40 con 0x7f2f300c6640 2023-11-15T03:09:06.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f2effd700 1 --2- 172.21.15.121:0/1040747276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3014af80 0x7f2f300c1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:09:06.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f2f7fe700 1 --2- 172.21.15.121:0/1040747276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f300c6640 0x7f2f300c6ad0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f2f2800d0d0 tx=0x7f2f2800b990).ready entity=mon.1 client_cookie=1941d217fb895e7f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:06.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f2e7fc700 1 -- 172.21.15.121:0/1040747276 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28010070 con 0x7f2f300c6640 2023-11-15T03:09:06.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f2e7fc700 1 -- 172.21.15.121:0/1040747276 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2800b2e0 con 0x7f2f300c6640 2023-11-15T03:09:06.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.758+0000 7f2f2e7fc700 1 -- 172.21.15.121:0/1040747276 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2800dc90 con 0x7f2f300c6640 2023-11-15T03:09:06.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.762+0000 7f2f354f1700 1 -- 172.21.15.121:0/1040747276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f300c6640 msgr2=0x7f2f300c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:06.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.762+0000 7f2f354f1700 1 --2- 172.21.15.121:0/1040747276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f300c6640 0x7f2f300c6ad0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f2f2800d0d0 tx=0x7f2f2800b990).stop 2023-11-15T03:09:06.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.762+0000 7f2f354f1700 1 -- 172.21.15.121:0/1040747276 shutdown_connections 2023-11-15T03:09:06.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.762+0000 7f2f354f1700 1 --2- 172.21.15.121:0/1040747276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f300c6640 0x7f2f300c6ad0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:06.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.762+0000 7f2f354f1700 1 --2- 172.21.15.121:0/1040747276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3014af80 0x7f2f300c1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:06.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.762+0000 7f2f354f1700 1 -- 172.21.15.121:0/1040747276 >> 172.21.15.121:0/1040747276 conn(0x7f2f30062190 msgr2=0x7f2f300625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:06.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.762+0000 7f2f354f1700 1 -- 172.21.15.121:0/1040747276 shutdown_connections 2023-11-15T03:09:06.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.762+0000 7f2f354f1700 1 -- 172.21.15.121:0/1040747276 wait complete. 2023-11-15T03:09:06.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.762+0000 7f2f354f1700 1 Processor -- start 2023-11-15T03:09:06.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.762+0000 7f2f354f1700 1 -- start start 2023-11-15T03:09:06.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f354f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f300c1bf0 0x7f2f300a8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:06.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f354f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f300c6640 0x7f2f30167eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:06.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f354f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f3014af80 0x7f2f301521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:06.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f354f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f301676f0 con 0x7f2f300c1bf0 2023-11-15T03:09:06.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f354f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f30164950 con 0x7f2f3014af80 2023-11-15T03:09:06.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f354f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f30164c30 con 0x7f2f300c6640 2023-11-15T03:09:06.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f2effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f300c1bf0 0x7f2f300a8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:06.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f2effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f300c1bf0 0x7f2f300a8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50640/0 (socket says 172.21.15.121:50640) 2023-11-15T03:09:06.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f2effd700 1 -- 172.21.15.121:0/107326628 learned_addr learned my addr 172.21.15.121:0/107326628 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:06.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f2f7fe700 1 --2- 172.21.15.121:0/107326628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f3014af80 0x7f2f301521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:06.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f267fc700 1 --2- 172.21.15.121:0/107326628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f300c6640 0x7f2f30167eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:06.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f2effd700 1 -- 172.21.15.121:0/107326628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f300c6640 msgr2=0x7f2f30167eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:06.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f2effd700 1 --2- 172.21.15.121:0/107326628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f300c6640 0x7f2f30167eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:06.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f2effd700 1 -- 172.21.15.121:0/107326628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f3014af80 msgr2=0x7f2f301521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:06.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f2effd700 1 --2- 172.21.15.121:0/107326628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f3014af80 0x7f2f301521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:06.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f2effd700 1 -- 172.21.15.121:0/107326628 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f20009370 con 0x7f2f300c1bf0 2023-11-15T03:09:06.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f267fc700 1 --2- 172.21.15.121:0/107326628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f300c6640 0x7f2f30167eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:06.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f2effd700 1 --2- 172.21.15.121:0/107326628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f300c1bf0 0x7f2f300a8410 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 rx=0x7f2f2000e9d0 tx=0x7f2f200090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:06.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f277fe700 1 -- 172.21.15.121:0/107326628 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2000c850 con 0x7f2f300c1bf0 2023-11-15T03:09:06.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f277fe700 1 -- 172.21.15.121:0/107326628 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2000c9b0 con 0x7f2f300c1bf0 2023-11-15T03:09:06.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f354f1700 1 -- 172.21.15.121:0/107326628 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2800bc30 con 0x7f2f300c1bf0 2023-11-15T03:09:06.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f277fe700 1 -- 172.21.15.121:0/107326628 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2000cd30 con 0x7f2f300c1bf0 2023-11-15T03:09:06.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.766+0000 7f2f354f1700 1 -- 172.21.15.121:0/107326628 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f30167990 con 0x7f2f300c1bf0 2023-11-15T03:09:06.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.770+0000 7f2f354f1700 1 -- 172.21.15.121:0/107326628 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f3006c150 con 0x7f2f300c1bf0 2023-11-15T03:09:06.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.770+0000 7f2f277fe700 1 -- 172.21.15.121:0/107326628 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2f20012600 con 0x7f2f300c1bf0 2023-11-15T03:09:06.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.770+0000 7f2f277fe700 1 --2- 172.21.15.121:0/107326628 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2f1005a4e0 0x7f2f1005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:06.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.770+0000 7f2f277fe700 1 -- 172.21.15.121:0/107326628 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7f2f200759e0 con 0x7f2f300c1bf0 2023-11-15T03:09:06.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.774+0000 7f2f267fc700 1 --2- 172.21.15.121:0/107326628 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2f1005a4e0 0x7f2f1005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:06.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.774+0000 7f2f267fc700 1 --2- 172.21.15.121:0/107326628 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2f1005a4e0 0x7f2f1005c9a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f2f300ac990 tx=0x7f2f18009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:06.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.778+0000 7f2f277fe700 1 -- 172.21.15.121:0/107326628 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2f20048680 con 0x7f2f300c1bf0 2023-11-15T03:09:06.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.974+0000 7f2f354f1700 1 -- 172.21.15.121:0/107326628 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f300c5820 con 0x7f2f300c1bf0 2023-11-15T03:09:06.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.974+0000 7f2f277fe700 1 -- 172.21.15.121:0/107326628 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 272 v272) v1 ==== 95+0+4493 (secure 0 0 0) 0x7f2f20016070 con 0x7f2f300c1bf0 2023-11-15T03:09:06.984 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:09:06.984 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":272,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9807,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/142843299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":142843299},{"type":"v1","addr":"172.21.15.121:6836","nonce":142843299}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":272}],"filesystems":[{"mdsmap":{"epoch":271,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:09:02.164913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:09:06.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.978+0000 7f2f354f1700 1 -- 172.21.15.121:0/107326628 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2f1005a4e0 msgr2=0x7f2f1005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:06.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.978+0000 7f2f354f1700 1 --2- 172.21.15.121:0/107326628 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2f1005a4e0 0x7f2f1005c9a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f2f300ac990 tx=0x7f2f18009040).stop 2023-11-15T03:09:06.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.978+0000 7f2f354f1700 1 -- 172.21.15.121:0/107326628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f300c1bf0 msgr2=0x7f2f300a8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:06.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.978+0000 7f2f354f1700 1 --2- 172.21.15.121:0/107326628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f300c1bf0 0x7f2f300a8410 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 rx=0x7f2f2000e9d0 tx=0x7f2f200090f0).stop 2023-11-15T03:09:06.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.978+0000 7f2f354f1700 1 -- 172.21.15.121:0/107326628 shutdown_connections 2023-11-15T03:09:06.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.978+0000 7f2f354f1700 1 --2- 172.21.15.121:0/107326628 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2f1005a4e0 0x7f2f1005c9a0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:06.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.978+0000 7f2f354f1700 1 --2- 172.21.15.121:0/107326628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f3014af80 0x7f2f301521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:06.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.978+0000 7f2f354f1700 1 --2- 172.21.15.121:0/107326628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f300c6640 0x7f2f30167eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:06.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.978+0000 7f2f354f1700 1 --2- 172.21.15.121:0/107326628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f300c1bf0 0x7f2f300a8410 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:06.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.978+0000 7f2f354f1700 1 -- 172.21.15.121:0/107326628 >> 172.21.15.121:0/107326628 conn(0x7f2f30062190 msgr2=0x7f2f300c2750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:06.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.978+0000 7f2f354f1700 1 -- 172.21.15.121:0/107326628 shutdown_connections 2023-11-15T03:09:06.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:06.978+0000 7f2f354f1700 1 -- 172.21.15.121:0/107326628 wait complete. 2023-11-15T03:09:06.991 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 272 2023-11-15T03:09:07.009 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:09:07.010 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:09:07.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.166+0000 7f7e5ce9b700 1 Processor -- start 2023-11-15T03:09:07.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.166+0000 7f7e5ce9b700 1 -- start start 2023-11-15T03:09:07.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.170+0000 7f7e5ce9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7e580aa240 0x7f7e58067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:07.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.170+0000 7f7e5ce9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7e58067f50 0x7f7e58065d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:07.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.170+0000 7f7e5ce9b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e580cb5c0 con 0x7f7e58153bb0 2023-11-15T03:09:07.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.170+0000 7f7e5ce9b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e580aa6f0 con 0x7f7e58067f50 2023-11-15T03:09:07.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.170+0000 7f7e5ce9b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e580c5900 con 0x7f7e580aa240 2023-11-15T03:09:07.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.170+0000 7f7e56d9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7e58067f50 0x7f7e58065d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:07.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.170+0000 7f7e56d9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7e58067f50 0x7f7e58065d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50642/0 (socket says 172.21.15.121:50642) 2023-11-15T03:09:07.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.170+0000 7f7e56d9d700 1 -- 172.21.15.121:0/2940590297 learned_addr learned my addr 172.21.15.121:0/2940590297 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:07.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.170+0000 7f7e4dd9b700 1 --2- 172.21.15.121:0/2940590297 >> v2:172.21.15.190:3300/0 conn(0x7f7e580aa240 0x7f7e58067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:07.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.170+0000 7f7e56d9d700 1 -- 172.21.15.121:0/2940590297 >> v2:172.21.15.190:3300/0 conn(0x7f7e580aa240 msgr2=0x7f7e58067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:07.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.170+0000 7f7e56d9d700 1 --2- 172.21.15.121:0/2940590297 >> v2:172.21.15.190:3300/0 conn(0x7f7e580aa240 0x7f7e58067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:07.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.170+0000 7f7e56d9d700 1 -- 172.21.15.121:0/2940590297 >> v1:172.21.15.190:6789/0 conn(0x7f7e58153bb0 legacy=0x7f7e580a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:07.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.170+0000 7f7e56d9d700 1 -- 172.21.15.121:0/2940590297 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e580c9ca0 con 0x7f7e58067f50 2023-11-15T03:09:07.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.170+0000 7f7e56d9d700 1 --2- 172.21.15.121:0/2940590297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e58067f50 0x7f7e58065d30 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 rx=0x7f7e580c77f0 tx=0x7f7e50009840).ready entity=mon.0 client_cookie=c8f0ea8d5de5be server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:07.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.170+0000 7f7e55d9b700 1 -- 172.21.15.121:0/2940590297 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000e070 con 0x7f7e58067f50 2023-11-15T03:09:07.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.174+0000 7f7e55d9b700 1 -- 172.21.15.121:0/2940590297 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5000ba20 con 0x7f7e58067f50 2023-11-15T03:09:07.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.174+0000 7f7e55d9b700 1 -- 172.21.15.121:0/2940590297 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000bda0 con 0x7f7e58067f50 2023-11-15T03:09:07.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.174+0000 7f7e5ce9b700 1 -- 172.21.15.121:0/2940590297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e58067f50 msgr2=0x7f7e58065d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:07.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.174+0000 7f7e5ce9b700 1 --2- 172.21.15.121:0/2940590297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e58067f50 0x7f7e58065d30 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 rx=0x7f7e580c77f0 tx=0x7f7e50009840).stop 2023-11-15T03:09:07.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.174+0000 7f7e5ce9b700 1 -- 172.21.15.121:0/2940590297 shutdown_connections 2023-11-15T03:09:07.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.174+0000 7f7e5ce9b700 1 --2- 172.21.15.121:0/2940590297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e58067f50 0x7f7e58065d30 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:07.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.174+0000 7f7e5ce9b700 1 --2- 172.21.15.121:0/2940590297 >> v2:172.21.15.190:3300/0 conn(0x7f7e580aa240 0x7f7e58067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:07.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.174+0000 7f7e5ce9b700 1 -- 172.21.15.121:0/2940590297 >> 172.21.15.121:0/2940590297 conn(0x7f7e58058c50 msgr2=0x7f7e58059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:07.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.174+0000 7f7e5ce9b700 1 -- 172.21.15.121:0/2940590297 shutdown_connections 2023-11-15T03:09:07.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.174+0000 7f7e5ce9b700 1 -- 172.21.15.121:0/2940590297 wait complete. 2023-11-15T03:09:07.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.178+0000 7f7e5ce9b700 1 Processor -- start 2023-11-15T03:09:07.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.178+0000 7f7e5ce9b700 1 -- start start 2023-11-15T03:09:07.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.178+0000 7f7e5ce9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e58067f50 0x7f7e58151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:07.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.178+0000 7f7e5ce9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e580aa240 0x7f7e581523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:07.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.178+0000 7f7e5ce9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e58153bb0 0x7f7e58164c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:07.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.178+0000 7f7e5ce9b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e581528b0 con 0x7f7e580aa240 2023-11-15T03:09:07.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.178+0000 7f7e5ce9b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e58150f90 con 0x7f7e58067f50 2023-11-15T03:09:07.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.178+0000 7f7e5ce9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e58150380 con 0x7f7e58153bb0 2023-11-15T03:09:07.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.178+0000 7f7e4dd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e580aa240 0x7f7e581523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:07.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.178+0000 7f7e4dd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e580aa240 0x7f7e581523c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50650/0 (socket says 172.21.15.121:50650) 2023-11-15T03:09:07.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.178+0000 7f7e4dd9b700 1 -- 172.21.15.121:0/3213956050 learned_addr learned my addr 172.21.15.121:0/3213956050 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:07.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.178+0000 7f7e56d9d700 1 --2- 172.21.15.121:0/3213956050 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e58153bb0 0x7f7e58164c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:07.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.178+0000 7f7e5659c700 1 --2- 172.21.15.121:0/3213956050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e58067f50 0x7f7e58151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:07.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.182+0000 7f7e4dd9b700 1 -- 172.21.15.121:0/3213956050 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e58153bb0 msgr2=0x7f7e58164c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:07.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.182+0000 7f7e4dd9b700 1 --2- 172.21.15.121:0/3213956050 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e58153bb0 0x7f7e58164c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:07.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.182+0000 7f7e4dd9b700 1 -- 172.21.15.121:0/3213956050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e58067f50 msgr2=0x7f7e58151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:07.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.182+0000 7f7e4dd9b700 1 --2- 172.21.15.121:0/3213956050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e58067f50 0x7f7e58151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:07.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.182+0000 7f7e4dd9b700 1 -- 172.21.15.121:0/3213956050 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e50009ae0 con 0x7f7e580aa240 2023-11-15T03:09:07.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.182+0000 7f7e4dd9b700 1 --2- 172.21.15.121:0/3213956050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e580aa240 0x7f7e581523c0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 rx=0x7f7e4000e900 tx=0x7f7e40009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:07.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.182+0000 7f7e4effd700 1 -- 172.21.15.121:0/3213956050 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4000eea0 con 0x7f7e580aa240 2023-11-15T03:09:07.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.182+0000 7f7e4effd700 1 -- 172.21.15.121:0/3213956050 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4000c850 con 0x7f7e580aa240 2023-11-15T03:09:07.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.182+0000 7f7e4effd700 1 -- 172.21.15.121:0/3213956050 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4000cbd0 con 0x7f7e580aa240 2023-11-15T03:09:07.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.182+0000 7f7e5ce9b700 1 -- 172.21.15.121:0/3213956050 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e58150660 con 0x7f7e580aa240 2023-11-15T03:09:07.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.182+0000 7f7e5ce9b700 1 -- 172.21.15.121:0/3213956050 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e58150c20 con 0x7f7e580aa240 2023-11-15T03:09:07.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.182+0000 7f7e5ce9b700 1 -- 172.21.15.121:0/3213956050 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e5806c150 con 0x7f7e580aa240 2023-11-15T03:09:07.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.186+0000 7f7e4effd700 1 -- 172.21.15.121:0/3213956050 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7e4001b400 con 0x7f7e580aa240 2023-11-15T03:09:07.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.186+0000 7f7e4effd700 1 --2- 172.21.15.121:0/3213956050 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7e3805a590 0x7f7e3805ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:07.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.186+0000 7f7e4effd700 1 -- 172.21.15.121:0/3213956050 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7f7e40075800 con 0x7f7e580aa240 2023-11-15T03:09:07.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.186+0000 7f7e5659c700 1 --2- 172.21.15.121:0/3213956050 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7e3805a590 0x7f7e3805ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:07.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.186+0000 7f7e5659c700 1 --2- 172.21.15.121:0/3213956050 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7e3805a590 0x7f7e3805ca50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f7e581659d0 tx=0x7f7e48009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:07.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.190+0000 7f7e4effd700 1 -- 172.21.15.121:0/3213956050 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7e400484c0 con 0x7f7e580aa240 2023-11-15T03:09:07.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.390+0000 7f7e5ce9b700 1 -- 172.21.15.121:0/3213956050 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e5805f3d0 con 0x7f7e580aa240 2023-11-15T03:09:07.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.390+0000 7f7e4effd700 1 -- 172.21.15.121:0/3213956050 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 272 v272) v1 ==== 95+0+4493 (secure 0 0 0) 0x7f7e40016020 con 0x7f7e580aa240 2023-11-15T03:09:07.402 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:09:07.402 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":272,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9807,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/142843299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":142843299},{"type":"v1","addr":"172.21.15.121:6836","nonce":142843299}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":272}],"filesystems":[{"mdsmap":{"epoch":271,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:09:02.164913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:09:07.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.394+0000 7f7e5ce9b700 1 -- 172.21.15.121:0/3213956050 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7e3805a590 msgr2=0x7f7e3805ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:07.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.394+0000 7f7e5ce9b700 1 --2- 172.21.15.121:0/3213956050 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7e3805a590 0x7f7e3805ca50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f7e581659d0 tx=0x7f7e48009040).stop 2023-11-15T03:09:07.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.394+0000 7f7e5ce9b700 1 -- 172.21.15.121:0/3213956050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e580aa240 msgr2=0x7f7e581523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:07.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.394+0000 7f7e5ce9b700 1 --2- 172.21.15.121:0/3213956050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e580aa240 0x7f7e581523c0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 rx=0x7f7e4000e900 tx=0x7f7e40009060).stop 2023-11-15T03:09:07.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.394+0000 7f7e5ce9b700 1 -- 172.21.15.121:0/3213956050 shutdown_connections 2023-11-15T03:09:07.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.394+0000 7f7e5ce9b700 1 --2- 172.21.15.121:0/3213956050 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7e3805a590 0x7f7e3805ca50 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:07.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.394+0000 7f7e5ce9b700 1 --2- 172.21.15.121:0/3213956050 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e58153bb0 0x7f7e58164c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:07.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.394+0000 7f7e5ce9b700 1 --2- 172.21.15.121:0/3213956050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e580aa240 0x7f7e581523c0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:07.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.394+0000 7f7e5ce9b700 1 --2- 172.21.15.121:0/3213956050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e58067f50 0x7f7e58151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:07.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.394+0000 7f7e5ce9b700 1 -- 172.21.15.121:0/3213956050 >> 172.21.15.121:0/3213956050 conn(0x7f7e58058c50 msgr2=0x7f7e58065d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:07.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.394+0000 7f7e5ce9b700 1 -- 172.21.15.121:0/3213956050 shutdown_connections 2023-11-15T03:09:07.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.394+0000 7f7e5ce9b700 1 -- 172.21.15.121:0/3213956050 wait complete. 2023-11-15T03:09:07.407 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 272 2023-11-15T03:09:07.424 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:09:07.425 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:09:07.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.574+0000 7f19c0a79700 1 Processor -- start 2023-11-15T03:09:07.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.574+0000 7f19c0a79700 1 -- start start 2023-11-15T03:09:07.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.574+0000 7f19c0a79700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f19bc066d60 0x7f19bc067140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:07.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.574+0000 7f19c0a79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19bc066370 0x7f19bc1577e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:07.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19c0a79700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19bc158160 con 0x7f19bc157cd0 2023-11-15T03:09:07.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19c0a79700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19bc0c00f0 con 0x7f19bc066d60 2023-11-15T03:09:07.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19c0a79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc066790 con 0x7f19bc066370 2023-11-15T03:09:07.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19bb7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f19bc066d60 0x7f19bc067140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:07.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19bb7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f19bc066d60 0x7f19bc067140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50652/0 (socket says 172.21.15.121:50652) 2023-11-15T03:09:07.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19bb7fe700 1 -- 172.21.15.121:0/3920523514 learned_addr learned my addr 172.21.15.121:0/3920523514 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:07.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19bb7fe700 1 -- 172.21.15.121:0/3920523514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19bc066370 msgr2=0x7f19bc1577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:07.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19bb7fe700 1 --2- 172.21.15.121:0/3920523514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19bc066370 0x7f19bc1577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:09:07.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19bb7fe700 1 -- 172.21.15.121:0/3920523514 >> v1:172.21.15.121:6789/0 conn(0x7f19bc157cd0 legacy=0x7f19bc1535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:07.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19bb7fe700 1 -- 172.21.15.121:0/3920523514 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19bc1583f0 con 0x7f19bc066d60 2023-11-15T03:09:07.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19bb7fe700 1 --2- 172.21.15.121:0/3920523514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19bc066d60 0x7f19bc067140 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 rx=0x7f19ac006fd0 tx=0x7f19ac0090f0).ready entity=mon.0 client_cookie=ce8f96282f5b43ee server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:07.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19ba7fc700 1 -- 172.21.15.121:0/3920523514 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac00b600 con 0x7f19bc066d60 2023-11-15T03:09:07.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19ba7fc700 1 -- 172.21.15.121:0/3920523514 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ac00b760 con 0x7f19bc066d60 2023-11-15T03:09:07.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19ba7fc700 1 -- 172.21.15.121:0/3920523514 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac00bae0 con 0x7f19bc066d60 2023-11-15T03:09:07.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19c0a79700 1 -- 172.21.15.121:0/3920523514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19bc066d60 msgr2=0x7f19bc067140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:07.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19c0a79700 1 --2- 172.21.15.121:0/3920523514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19bc066d60 0x7f19bc067140 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 rx=0x7f19ac006fd0 tx=0x7f19ac0090f0).stop 2023-11-15T03:09:07.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19c0a79700 1 -- 172.21.15.121:0/3920523514 shutdown_connections 2023-11-15T03:09:07.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19c0a79700 1 --2- 172.21.15.121:0/3920523514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19bc066370 0x7f19bc1577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:09:07.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19c0a79700 1 --2- 172.21.15.121:0/3920523514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19bc066d60 0x7f19bc067140 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:07.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19c0a79700 1 -- 172.21.15.121:0/3920523514 >> 172.21.15.121:0/3920523514 conn(0x7f19bc062190 msgr2=0x7f19bc062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:07.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19c0a79700 1 -- 172.21.15.121:0/3920523514 shutdown_connections 2023-11-15T03:09:07.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.578+0000 7f19c0a79700 1 -- 172.21.15.121:0/3920523514 wait complete. 2023-11-15T03:09:07.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.582+0000 7f19c0a79700 1 Processor -- start 2023-11-15T03:09:07.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.582+0000 7f19c0a79700 1 -- start start 2023-11-15T03:09:07.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.582+0000 7f19c0a79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19bc066370 0x7f19bc152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:07.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.582+0000 7f19c0a79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19bc066d60 0x7f19bc151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:07.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.582+0000 7f19c0a79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19bc157cd0 0x7f19bc14cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:07.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.582+0000 7f19c0a79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc150ef0 con 0x7f19bc066d60 2023-11-15T03:09:07.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.582+0000 7f19c0a79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc150290 con 0x7f19bc157cd0 2023-11-15T03:09:07.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.582+0000 7f19c0a79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc151870 con 0x7f19bc066370 2023-11-15T03:09:07.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.582+0000 7f19baffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19bc066d60 0x7f19bc151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:07.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.582+0000 7f19baffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19bc066d60 0x7f19bc151380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50666/0 (socket says 172.21.15.121:50666) 2023-11-15T03:09:07.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.582+0000 7f19baffd700 1 -- 172.21.15.121:0/1888687624 learned_addr learned my addr 172.21.15.121:0/1888687624 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:07.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.582+0000 7f19bb7fe700 1 --2- 172.21.15.121:0/1888687624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19bc066370 0x7f19bc152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:07.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.582+0000 7f19bbfff700 1 --2- 172.21.15.121:0/1888687624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19bc157cd0 0x7f19bc14cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:07.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.582+0000 7f19baffd700 1 -- 172.21.15.121:0/1888687624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19bc066370 msgr2=0x7f19bc152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:07.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.582+0000 7f19baffd700 1 --2- 172.21.15.121:0/1888687624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19bc066370 0x7f19bc152920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:07.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.582+0000 7f19baffd700 1 -- 172.21.15.121:0/1888687624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19bc157cd0 msgr2=0x7f19bc14cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:07.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.582+0000 7f19baffd700 1 --2- 172.21.15.121:0/1888687624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19bc157cd0 0x7f19bc14cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:07.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.582+0000 7f19baffd700 1 -- 172.21.15.121:0/1888687624 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac009370 con 0x7f19bc066d60 2023-11-15T03:09:07.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.586+0000 7f19baffd700 1 --2- 172.21.15.121:0/1888687624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19bc066d60 0x7f19bc151380 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 rx=0x7f19b000b560 tx=0x7f19b00090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:07.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.586+0000 7f19a37fe700 1 -- 172.21.15.121:0/1888687624 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b000d4a0 con 0x7f19bc066d60 2023-11-15T03:09:07.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.586+0000 7f19a37fe700 1 -- 172.21.15.121:0/1888687624 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b000d620 con 0x7f19bc066d60 2023-11-15T03:09:07.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.586+0000 7f19c0a79700 1 -- 172.21.15.121:0/1888687624 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19bc14d2c0 con 0x7f19bc066d60 2023-11-15T03:09:07.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.586+0000 7f19a37fe700 1 -- 172.21.15.121:0/1888687624 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b000d9e0 con 0x7f19bc066d60 2023-11-15T03:09:07.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.586+0000 7f19c0a79700 1 -- 172.21.15.121:0/1888687624 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19bc1508a0 con 0x7f19bc066d60 2023-11-15T03:09:07.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.586+0000 7f19c0a79700 1 -- 172.21.15.121:0/1888687624 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19bc06bcf0 con 0x7f19bc066d60 2023-11-15T03:09:07.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.586+0000 7f19a37fe700 1 -- 172.21.15.121:0/1888687624 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f19b0012410 con 0x7f19bc066d60 2023-11-15T03:09:07.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.590+0000 7f19a37fe700 1 --2- 172.21.15.121:0/1888687624 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f19a805a510 0x7f19a805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:07.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.590+0000 7f19a37fe700 1 -- 172.21.15.121:0/1888687624 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7f19b00751d0 con 0x7f19bc066d60 2023-11-15T03:09:07.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.590+0000 7f19bb7fe700 1 --2- 172.21.15.121:0/1888687624 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f19a805a510 0x7f19a805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:07.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.590+0000 7f19bb7fe700 1 --2- 172.21.15.121:0/1888687624 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f19a805a510 0x7f19a805c9d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f19ac009130 tx=0x7f19ac017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:07.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.594+0000 7f19a37fe700 1 -- 172.21.15.121:0/1888687624 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f19b0047e10 con 0x7f19bc066d60 2023-11-15T03:09:07.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.794+0000 7f19c0a79700 1 -- 172.21.15.121:0/1888687624 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19bc062e90 con 0x7f19bc066d60 2023-11-15T03:09:07.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.794+0000 7f19a37fe700 1 -- 172.21.15.121:0/1888687624 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 272 v272) v1 ==== 95+0+4493 (secure 0 0 0) 0x7f19b004b430 con 0x7f19bc066d60 2023-11-15T03:09:07.806 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:09:07.806 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":272,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9807,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/142843299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":142843299},{"type":"v1","addr":"172.21.15.121:6836","nonce":142843299}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":272}],"filesystems":[{"mdsmap":{"epoch":271,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:09:02.164913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:09:07.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.798+0000 7f19c0a79700 1 -- 172.21.15.121:0/1888687624 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f19a805a510 msgr2=0x7f19a805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:07.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.798+0000 7f19c0a79700 1 --2- 172.21.15.121:0/1888687624 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f19a805a510 0x7f19a805c9d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f19ac009130 tx=0x7f19ac017000).stop 2023-11-15T03:09:07.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.798+0000 7f19c0a79700 1 -- 172.21.15.121:0/1888687624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19bc066d60 msgr2=0x7f19bc151380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:07.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.798+0000 7f19c0a79700 1 --2- 172.21.15.121:0/1888687624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19bc066d60 0x7f19bc151380 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 rx=0x7f19b000b560 tx=0x7f19b00090f0).stop 2023-11-15T03:09:07.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.798+0000 7f19c0a79700 1 -- 172.21.15.121:0/1888687624 shutdown_connections 2023-11-15T03:09:07.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.798+0000 7f19c0a79700 1 --2- 172.21.15.121:0/1888687624 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f19a805a510 0x7f19a805c9d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:07.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.798+0000 7f19c0a79700 1 --2- 172.21.15.121:0/1888687624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19bc157cd0 0x7f19bc14cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:07.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.798+0000 7f19c0a79700 1 --2- 172.21.15.121:0/1888687624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19bc066d60 0x7f19bc151380 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:07.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.798+0000 7f19c0a79700 1 --2- 172.21.15.121:0/1888687624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19bc066370 0x7f19bc152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:07.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.798+0000 7f19c0a79700 1 -- 172.21.15.121:0/1888687624 >> 172.21.15.121:0/1888687624 conn(0x7f19bc062190 msgr2=0x7f19bc0c6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:07.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.798+0000 7f19c0a79700 1 -- 172.21.15.121:0/1888687624 shutdown_connections 2023-11-15T03:09:07.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:07.798+0000 7f19c0a79700 1 -- 172.21.15.121:0/1888687624 wait complete. 2023-11-15T03:09:07.811 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 272 2023-11-15T03:09:07.829 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 100.00000000.inode' led to DAMAGED state 2023-11-15T03:09:07.830 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 10000000000.00000000:sixmegs_head' 2023-11-15T03:09:07.830 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:09:07.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:07.831 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:09:07.851 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:09:07.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:09:07.853 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:09:08.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.050+0000 7f11e7146700 1 Processor -- start 2023-11-15T03:09:08.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.050+0000 7f11e7146700 1 -- start start 2023-11-15T03:09:08.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.050+0000 7f11e7146700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f11e0151920 0x7f11e00bb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:08.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.050+0000 7f11e7146700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f11e0067150 0x7f11e00aba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:08.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.050+0000 7f11e7146700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e0155de0 con 0x7f11e0156270 2023-11-15T03:09:08.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.050+0000 7f11e7146700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e00c4060 con 0x7f11e0067150 2023-11-15T03:09:08.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.050+0000 7f11e7146700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e00c3aa0 con 0x7f11e0151920 2023-11-15T03:09:08.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11dffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f11e0067150 0x7f11e00aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:08.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11dffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f11e0067150 0x7f11e00aba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50682/0 (socket says 172.21.15.121:50682) 2023-11-15T03:09:08.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11dffff700 1 -- 172.21.15.121:0/394090439 learned_addr learned my addr 172.21.15.121:0/394090439 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:08.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11e4ee2700 1 --2- 172.21.15.121:0/394090439 >> v2:172.21.15.190:3300/0 conn(0x7f11e0151920 0x7f11e00bb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:08.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11dffff700 1 -- 172.21.15.121:0/394090439 >> v2:172.21.15.190:3300/0 conn(0x7f11e0151920 msgr2=0x7f11e00bb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:08.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11dffff700 1 --2- 172.21.15.121:0/394090439 >> v2:172.21.15.190:3300/0 conn(0x7f11e0151920 0x7f11e00bb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:08.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11dffff700 1 -- 172.21.15.121:0/394090439 >> v1:172.21.15.190:6789/0 conn(0x7f11e0156270 legacy=0x7f11e00658a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:08.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11dffff700 1 -- 172.21.15.121:0/394090439 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e00c3d80 con 0x7f11e0067150 2023-11-15T03:09:08.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11dffff700 1 --2- 172.21.15.121:0/394090439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11e0067150 0x7f11e00aba40 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 rx=0x7f11e0063d30 tx=0x7f11cc0090f0).ready entity=mon.0 client_cookie=4cd57cdff49e0e29 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:08.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11df7fe700 1 -- 172.21.15.121:0/394090439 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc00b520 con 0x7f11e0067150 2023-11-15T03:09:08.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11df7fe700 1 -- 172.21.15.121:0/394090439 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11cc00b680 con 0x7f11e0067150 2023-11-15T03:09:08.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11df7fe700 1 -- 172.21.15.121:0/394090439 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc00ba00 con 0x7f11e0067150 2023-11-15T03:09:08.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11e7146700 1 -- 172.21.15.121:0/394090439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11e0067150 msgr2=0x7f11e00aba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:08.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11e7146700 1 --2- 172.21.15.121:0/394090439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11e0067150 0x7f11e00aba40 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 rx=0x7f11e0063d30 tx=0x7f11cc0090f0).stop 2023-11-15T03:09:08.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11e7146700 1 -- 172.21.15.121:0/394090439 shutdown_connections 2023-11-15T03:09:08.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11e7146700 1 --2- 172.21.15.121:0/394090439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11e0067150 0x7f11e00aba40 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:08.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11e7146700 1 --2- 172.21.15.121:0/394090439 >> v2:172.21.15.190:3300/0 conn(0x7f11e0151920 0x7f11e00bb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:08.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11e7146700 1 -- 172.21.15.121:0/394090439 >> 172.21.15.121:0/394090439 conn(0x7f11e0067a60 msgr2=0x7f11e0067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:08.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11e7146700 1 -- 172.21.15.121:0/394090439 shutdown_connections 2023-11-15T03:09:08.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.054+0000 7f11e7146700 1 -- 172.21.15.121:0/394090439 wait complete. 2023-11-15T03:09:08.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.058+0000 7f11e7146700 1 Processor -- start 2023-11-15T03:09:08.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.058+0000 7f11e7146700 1 -- start start 2023-11-15T03:09:08.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11e7146700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11e0067150 0x7f11e0136d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:08.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11e7146700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11e0151920 0x7f11e00aae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:08.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11e7146700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11e0156270 0x7f11e00ab350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:08.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11e7146700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e0152280 con 0x7f11e0156270 2023-11-15T03:09:08.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11e7146700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e01530e0 con 0x7f11e0067150 2023-11-15T03:09:08.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11e7146700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e0137220 con 0x7f11e0151920 2023-11-15T03:09:08.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11e56e3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11e0156270 0x7f11e00ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:08.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11e56e3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11e0156270 0x7f11e00ab350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50696/0 (socket says 172.21.15.121:50696) 2023-11-15T03:09:08.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11e56e3700 1 -- 172.21.15.121:0/3341704850 learned_addr learned my addr 172.21.15.121:0/3341704850 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:08.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11e4ee2700 1 --2- 172.21.15.121:0/3341704850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11e0067150 0x7f11e0136d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:08.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11dffff700 1 --2- 172.21.15.121:0/3341704850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11e0151920 0x7f11e00aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:08.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11e56e3700 1 -- 172.21.15.121:0/3341704850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11e0151920 msgr2=0x7f11e00aae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:08.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11e56e3700 1 --2- 172.21.15.121:0/3341704850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11e0151920 0x7f11e00aae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:08.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11e56e3700 1 -- 172.21.15.121:0/3341704850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11e0067150 msgr2=0x7f11e0136d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:08.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11e56e3700 1 --2- 172.21.15.121:0/3341704850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11e0067150 0x7f11e0136d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:08.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11e56e3700 1 -- 172.21.15.121:0/3341704850 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11cc009370 con 0x7f11e0156270 2023-11-15T03:09:08.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11e56e3700 1 --2- 172.21.15.121:0/3341704850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11e0156270 0x7f11e00ab350 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 rx=0x7f11d800c7f0 tx=0x7f11d8001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:08.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11dcff9700 1 -- 172.21.15.121:0/3341704850 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d800e4a0 con 0x7f11e0156270 2023-11-15T03:09:08.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11dcff9700 1 -- 172.21.15.121:0/3341704850 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d800e600 con 0x7f11e0156270 2023-11-15T03:09:08.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11dcff9700 1 -- 172.21.15.121:0/3341704850 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d800e980 con 0x7f11e0156270 2023-11-15T03:09:08.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11e7146700 1 -- 172.21.15.121:0/3341704850 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e01524d0 con 0x7f11e0156270 2023-11-15T03:09:08.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11e7146700 1 -- 172.21.15.121:0/3341704850 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e0152a90 con 0x7f11e0156270 2023-11-15T03:09:08.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.062+0000 7f11e7146700 1 -- 172.21.15.121:0/3341704850 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e0047060 con 0x7f11e0156270 2023-11-15T03:09:08.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.066+0000 7f11dcff9700 1 -- 172.21.15.121:0/3341704850 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f11d80123f0 con 0x7f11e0156270 2023-11-15T03:09:08.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.066+0000 7f11dcff9700 1 --2- 172.21.15.121:0/3341704850 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f11c405a510 0x7f11c405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:08.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.066+0000 7f11dcff9700 1 -- 172.21.15.121:0/3341704850 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7f11d8074b60 con 0x7f11e0156270 2023-11-15T03:09:08.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.066+0000 7f11e4ee2700 1 --2- 172.21.15.121:0/3341704850 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f11c405a510 0x7f11c405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:08.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.070+0000 7f11e4ee2700 1 --2- 172.21.15.121:0/3341704850 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f11c405a510 0x7f11c405c9d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f11d4009c30 tx=0x7f11d4009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:08.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.074+0000 7f11dcff9700 1 -- 172.21.15.121:0/3341704850 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f11d8047720 con 0x7f11e0156270 2023-11-15T03:09:08.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:08.278+0000 7f11e7146700 1 -- 172.21.15.121:0/3341704850 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f11e0153220 con 0x7f11e0156270 2023-11-15T03:09:09.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.270+0000 7f11dcff9700 1 -- 172.21.15.121:0/3341704850 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v273) v1 ==== 161+0+0 (secure 0 0 0) 0x7f11d804ad40 con 0x7f11e0156270 2023-11-15T03:09:09.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.274+0000 7f11e7146700 1 -- 172.21.15.121:0/3341704850 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f11c405a510 msgr2=0x7f11c405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:09.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.274+0000 7f11e7146700 1 --2- 172.21.15.121:0/3341704850 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f11c405a510 0x7f11c405c9d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f11d4009c30 tx=0x7f11d4009060).stop 2023-11-15T03:09:09.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.274+0000 7f11e7146700 1 -- 172.21.15.121:0/3341704850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11e0156270 msgr2=0x7f11e00ab350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:09.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.274+0000 7f11e7146700 1 --2- 172.21.15.121:0/3341704850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11e0156270 0x7f11e00ab350 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 rx=0x7f11d800c7f0 tx=0x7f11d8001510).stop 2023-11-15T03:09:09.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.274+0000 7f11e7146700 1 -- 172.21.15.121:0/3341704850 shutdown_connections 2023-11-15T03:09:09.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.274+0000 7f11e7146700 1 --2- 172.21.15.121:0/3341704850 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f11c405a510 0x7f11c405c9d0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:09.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.274+0000 7f11e7146700 1 --2- 172.21.15.121:0/3341704850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11e0156270 0x7f11e00ab350 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:09.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.274+0000 7f11e7146700 1 --2- 172.21.15.121:0/3341704850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11e0151920 0x7f11e00aae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:09.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.274+0000 7f11e7146700 1 --2- 172.21.15.121:0/3341704850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11e0067150 0x7f11e0136d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:09.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.274+0000 7f11e7146700 1 -- 172.21.15.121:0/3341704850 >> 172.21.15.121:0/3341704850 conn(0x7f11e0067a60 msgr2=0x7f11e014d510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:09.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.274+0000 7f11e7146700 1 -- 172.21.15.121:0/3341704850 shutdown_connections 2023-11-15T03:09:09.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.274+0000 7f11e7146700 1 -- 172.21.15.121:0/3341704850 wait complete. 2023-11-15T03:09:09.288 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:09:09.308 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:09:09.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.466+0000 7f7cef746700 1 Processor -- start 2023-11-15T03:09:09.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.466+0000 7f7cef746700 1 -- start start 2023-11-15T03:09:09.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.470+0000 7f7cef746700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce8153aa0 0x7f7ce8066680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:09.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.470+0000 7f7cef746700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7ce8153e80 0x7f7ce80ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:09.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.470+0000 7f7cef746700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce80c5c50 con 0x7f7ce80c0490 2023-11-15T03:09:09.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.470+0000 7f7cef746700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce8066b70 con 0x7f7ce8153e80 2023-11-15T03:09:09.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.470+0000 7f7cef746700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce80c5920 con 0x7f7ce8153aa0 2023-11-15T03:09:09.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.470+0000 7f7cecce1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7ce8153e80 0x7f7ce80ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:09.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.470+0000 7f7cecce1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7ce8153e80 0x7f7ce80ad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:34366/0 (socket says 172.21.15.121:34366) 2023-11-15T03:09:09.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.470+0000 7f7cecce1700 1 -- 172.21.15.121:0/1220628367 learned_addr learned my addr 172.21.15.121:0/1220628367 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:09.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.470+0000 7f7ced4e2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce8153aa0 0x7f7ce8066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:09.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.470+0000 7f7cecce1700 1 -- 172.21.15.121:0/1220628367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce8153aa0 msgr2=0x7f7ce8066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:09.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.470+0000 7f7cecce1700 1 --2- 172.21.15.121:0/1220628367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce8153aa0 0x7f7ce8066680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:09.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.470+0000 7f7cecce1700 1 -- 172.21.15.121:0/1220628367 >> v1:172.21.15.190:6789/0 conn(0x7f7ce80c0490 legacy=0x7f7ce8062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:09.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.470+0000 7f7cecce1700 1 -- 172.21.15.121:0/1220628367 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce8066d50 con 0x7f7ce8153e80 2023-11-15T03:09:09.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.470+0000 7f7cecce1700 1 --2- 172.21.15.121:0/1220628367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8153e80 0x7f7ce80ad150 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f7cd8006fd0 tx=0x7f7cd80090f0).ready entity=mon.1 client_cookie=66ab25a2c1bcba07 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:09.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.474+0000 7f7cdffff700 1 -- 172.21.15.121:0/1220628367 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd800b580 con 0x7f7ce8153e80 2023-11-15T03:09:09.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.474+0000 7f7cdffff700 1 -- 172.21.15.121:0/1220628367 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd800b6e0 con 0x7f7ce8153e80 2023-11-15T03:09:09.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.474+0000 7f7cdffff700 1 -- 172.21.15.121:0/1220628367 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd800ba60 con 0x7f7ce8153e80 2023-11-15T03:09:09.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.474+0000 7f7cef746700 1 -- 172.21.15.121:0/1220628367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8153e80 msgr2=0x7f7ce80ad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:09.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.474+0000 7f7cef746700 1 --2- 172.21.15.121:0/1220628367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8153e80 0x7f7ce80ad150 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f7cd8006fd0 tx=0x7f7cd80090f0).stop 2023-11-15T03:09:09.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.474+0000 7f7cef746700 1 -- 172.21.15.121:0/1220628367 shutdown_connections 2023-11-15T03:09:09.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.474+0000 7f7cef746700 1 --2- 172.21.15.121:0/1220628367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8153e80 0x7f7ce80ad150 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:09.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.474+0000 7f7cef746700 1 --2- 172.21.15.121:0/1220628367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce8153aa0 0x7f7ce8066680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:09.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.474+0000 7f7cef746700 1 -- 172.21.15.121:0/1220628367 >> 172.21.15.121:0/1220628367 conn(0x7f7ce8067a60 msgr2=0x7f7ce8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:09.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.474+0000 7f7cef746700 1 -- 172.21.15.121:0/1220628367 shutdown_connections 2023-11-15T03:09:09.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.474+0000 7f7cef746700 1 -- 172.21.15.121:0/1220628367 wait complete. 2023-11-15T03:09:09.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.478+0000 7f7cef746700 1 Processor -- start 2023-11-15T03:09:09.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.478+0000 7f7cef746700 1 -- start start 2023-11-15T03:09:09.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.478+0000 7f7cef746700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce80c0490 0x7f7ce80c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:09.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.478+0000 7f7cef746700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8153aa0 0x7f7ce80c2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:09.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.478+0000 7f7cef746700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ce8153e80 0x7f7ce80c2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:09.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.478+0000 7f7cef746700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce80c36d0 con 0x7f7ce8153e80 2023-11-15T03:09:09.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.478+0000 7f7cef746700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce80c38e0 con 0x7f7ce8153aa0 2023-11-15T03:09:09.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.478+0000 7f7cef746700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce80c3bc0 con 0x7f7ce80c0490 2023-11-15T03:09:09.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.482+0000 7f7cedce3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ce8153e80 0x7f7ce80c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:09.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.482+0000 7f7cecce1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8153aa0 0x7f7ce80c2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:09.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.482+0000 7f7cecce1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8153aa0 0x7f7ce80c2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:34368/0 (socket says 172.21.15.121:34368) 2023-11-15T03:09:09.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.482+0000 7f7cecce1700 1 -- 172.21.15.121:0/2659003546 learned_addr learned my addr 172.21.15.121:0/2659003546 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:09.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.482+0000 7f7ced4e2700 1 --2- 172.21.15.121:0/2659003546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce80c0490 0x7f7ce80c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:09.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.482+0000 7f7cecce1700 1 -- 172.21.15.121:0/2659003546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce80c0490 msgr2=0x7f7ce80c31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:09.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.482+0000 7f7cecce1700 1 --2- 172.21.15.121:0/2659003546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce80c0490 0x7f7ce80c31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:09.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.482+0000 7f7cecce1700 1 -- 172.21.15.121:0/2659003546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ce8153e80 msgr2=0x7f7ce80c2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:09.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.482+0000 7f7cecce1700 1 --2- 172.21.15.121:0/2659003546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ce8153e80 0x7f7ce80c2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:09.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.482+0000 7f7cecce1700 1 -- 172.21.15.121:0/2659003546 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd8009370 con 0x7f7ce8153aa0 2023-11-15T03:09:09.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.482+0000 7f7cecce1700 1 --2- 172.21.15.121:0/2659003546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8153aa0 0x7f7ce80c2440 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f7cd8011000 tx=0x7f7cd8017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:09.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.482+0000 7f7cdd7fa700 1 -- 172.21.15.121:0/2659003546 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8000bc0 con 0x7f7ce8153aa0 2023-11-15T03:09:09.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.482+0000 7f7cdd7fa700 1 -- 172.21.15.121:0/2659003546 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd800b850 con 0x7f7ce8153aa0 2023-11-15T03:09:09.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.482+0000 7f7cef746700 1 -- 172.21.15.121:0/2659003546 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce80c3ea0 con 0x7f7ce8153aa0 2023-11-15T03:09:09.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.482+0000 7f7cdd7fa700 1 -- 172.21.15.121:0/2659003546 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8000d20 con 0x7f7ce8153aa0 2023-11-15T03:09:09.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.482+0000 7f7cef746700 1 -- 172.21.15.121:0/2659003546 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce80c4460 con 0x7f7ce8153aa0 2023-11-15T03:09:09.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.482+0000 7f7cef746700 1 -- 172.21.15.121:0/2659003546 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce80c4680 con 0x7f7ce8153aa0 2023-11-15T03:09:09.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.486+0000 7f7cdd7fa700 1 -- 172.21.15.121:0/2659003546 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7cd801b850 con 0x7f7ce8153aa0 2023-11-15T03:09:09.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.486+0000 7f7cdd7fa700 1 --2- 172.21.15.121:0/2659003546 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7ccc05a510 0x7f7ccc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:09.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.486+0000 7f7cdd7fa700 1 -- 172.21.15.121:0/2659003546 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7f7cd8076b90 con 0x7f7ce8153aa0 2023-11-15T03:09:09.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.486+0000 7f7ced4e2700 1 --2- 172.21.15.121:0/2659003546 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7ccc05a510 0x7f7ccc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:09.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.486+0000 7f7ced4e2700 1 --2- 172.21.15.121:0/2659003546 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7ccc05a510 0x7f7ccc05c9d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f7ce400f570 tx=0x7f7ce400b040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:09.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.494+0000 7f7cdd7fa700 1 -- 172.21.15.121:0/2659003546 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7cd80497b0 con 0x7f7ce8153aa0 2023-11-15T03:09:09.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:09.714+0000 7f7cef746700 1 -- 172.21.15.121:0/2659003546 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f7ce8067030 con 0x7f7ce8153aa0 2023-11-15T03:09:10.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.290+0000 7f7cdd7fa700 1 -- 172.21.15.121:0/2659003546 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v274) v1 ==== 101+0+0 (secure 0 0 0) 0x7f7cd8029030 con 0x7f7ce8153aa0 2023-11-15T03:09:10.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.294+0000 7f7cef746700 1 -- 172.21.15.121:0/2659003546 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7ccc05a510 msgr2=0x7f7ccc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.294+0000 7f7cef746700 1 --2- 172.21.15.121:0/2659003546 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7ccc05a510 0x7f7ccc05c9d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f7ce400f570 tx=0x7f7ce400b040).stop 2023-11-15T03:09:10.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.294+0000 7f7cef746700 1 -- 172.21.15.121:0/2659003546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8153aa0 msgr2=0x7f7ce80c2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.294+0000 7f7cef746700 1 --2- 172.21.15.121:0/2659003546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8153aa0 0x7f7ce80c2440 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f7cd8011000 tx=0x7f7cd8017000).stop 2023-11-15T03:09:10.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.294+0000 7f7cef746700 1 -- 172.21.15.121:0/2659003546 shutdown_connections 2023-11-15T03:09:10.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.294+0000 7f7cef746700 1 --2- 172.21.15.121:0/2659003546 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7ccc05a510 0x7f7ccc05c9d0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.294+0000 7f7cef746700 1 --2- 172.21.15.121:0/2659003546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ce8153e80 0x7f7ce80c2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.294+0000 7f7cef746700 1 --2- 172.21.15.121:0/2659003546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8153aa0 0x7f7ce80c2440 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.294+0000 7f7cef746700 1 --2- 172.21.15.121:0/2659003546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce80c0490 0x7f7ce80c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.294+0000 7f7cef746700 1 -- 172.21.15.121:0/2659003546 >> 172.21.15.121:0/2659003546 conn(0x7f7ce8067a60 msgr2=0x7f7ce80ad160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:10.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.294+0000 7f7cef746700 1 -- 172.21.15.121:0/2659003546 shutdown_connections 2023-11-15T03:09:10.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.294+0000 7f7cef746700 1 -- 172.21.15.121:0/2659003546 wait complete. 2023-11-15T03:09:10.305 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:09:10.326 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:09:10.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.374+0000 7f6e22b42d00 1 Processor -- start 2023-11-15T03:09:10.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.374+0000 7f6e22b42d00 1 -- start start 2023-11-15T03:09:10.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e22b42d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e0a6e12920 0x55e0a6ede020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e22b42d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0a6e59be0 0x55e0a6e56eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e22b42d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e0a6ede510 0x55e0a6e573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e22b42d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a6e57890 con 0x55e0a6e12920 2023-11-15T03:09:10.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e22b42d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a6e57b10 con 0x55e0a6e59be0 2023-11-15T03:09:10.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e22b42d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0a6e586a0 con 0x55e0a6ede510 2023-11-15T03:09:10.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e171aa700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e0a6e12920 0x55e0a6ede020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e171aa700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e0a6e12920 0x55e0a6ede020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50732/0 (socket says 172.21.15.121:50732) 2023-11-15T03:09:10.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e171aa700 1 -- 172.21.15.121:0/2734250698 learned_addr learned my addr 172.21.15.121:0/2734250698 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:10.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e179ab700 1 --2- 172.21.15.121:0/2734250698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e0a6ede510 0x55e0a6e573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e0e9a9700 1 --2- 172.21.15.121:0/2734250698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0a6e59be0 0x55e0a6e56eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e171aa700 1 -- 172.21.15.121:0/2734250698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e0a6ede510 msgr2=0x55e0a6e573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:10.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e171aa700 1 --2- 172.21.15.121:0/2734250698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e0a6ede510 0x55e0a6e573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e171aa700 1 -- 172.21.15.121:0/2734250698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0a6e59be0 msgr2=0x55e0a6e56eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:10.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e171aa700 1 --2- 172.21.15.121:0/2734250698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0a6e59be0 0x55e0a6e56eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e171aa700 1 -- 172.21.15.121:0/2734250698 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e08005cf0 con 0x55e0a6e12920 2023-11-15T03:09:10.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e0e9a9700 1 --2- 172.21.15.121:0/2734250698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0a6e59be0 0x55e0a6e56eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:09:10.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e171aa700 1 --2- 172.21.15.121:0/2734250698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e0a6e12920 0x55e0a6ede020 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 rx=0x7f6e08009a40 tx=0x7f6e0800c040).ready entity=mon.0 client_cookie=83ce1f5993a22f1d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:10.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e161a8700 1 -- 172.21.15.121:0/2734250698 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0800f040 con 0x55e0a6e12920 2023-11-15T03:09:10.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e161a8700 1 -- 172.21.15.121:0/2734250698 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e08008ac0 con 0x55e0a6e12920 2023-11-15T03:09:10.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e161a8700 1 -- 172.21.15.121:0/2734250698 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08008e40 con 0x55e0a6e12920 2023-11-15T03:09:10.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2734250698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e0a6e12920 msgr2=0x55e0a6ede020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2734250698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e0a6e12920 0x55e0a6ede020 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 rx=0x7f6e08009a40 tx=0x7f6e0800c040).stop 2023-11-15T03:09:10.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2734250698 shutdown_connections 2023-11-15T03:09:10.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2734250698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e0a6ede510 0x55e0a6e573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2734250698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0a6e59be0 0x55e0a6e56eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2734250698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e0a6e12920 0x55e0a6ede020 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.378+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2734250698 >> 172.21.15.121:0/2734250698 conn(0x55e0a6e588f0 msgr2=0x55e0a6e419b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:10.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.382+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2734250698 shutdown_connections 2023-11-15T03:09:10.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.382+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2734250698 wait complete. 2023-11-15T03:09:10.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.382+0000 7f6e22b42d00 1 Processor -- start 2023-11-15T03:09:10.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.382+0000 7f6e22b42d00 1 -- start start 2023-11-15T03:09:10.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.382+0000 7f6e22b42d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e0a6e12920 0x55e0a6eda990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.382+0000 7f6e22b42d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e0a6e59be0 0x55e0a6e52ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.382+0000 7f6e22b42d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0a6e51400 0x55e0a6e533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.382+0000 7f6e22b42d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a6e53ff0 con 0x55e0a6e12920 2023-11-15T03:09:10.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.382+0000 7f6e22b42d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a6e54480 con 0x55e0a6e51400 2023-11-15T03:09:10.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.382+0000 7f6e22b42d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0a6edd2c0 con 0x55e0a6e59be0 2023-11-15T03:09:10.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.382+0000 7f6e171aa700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e0a6e12920 0x55e0a6eda990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.382+0000 7f6e171aa700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e0a6e12920 0x55e0a6eda990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50744/0 (socket says 172.21.15.121:50744) 2023-11-15T03:09:10.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.382+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 learned_addr learned my addr 172.21.15.121:0/2915141469 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:10.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.382+0000 7f6e0e9a9700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e0a6e59be0 0x55e0a6e52ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.382+0000 7f6e179ab700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0a6e51400 0x55e0a6e533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.386+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e0a6e59be0 msgr2=0x55e0a6e52ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.386+0000 7f6e171aa700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e0a6e59be0 0x55e0a6e52ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.386+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0a6e51400 msgr2=0x55e0a6e533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.386+0000 7f6e171aa700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0a6e51400 0x55e0a6e533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.386+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e08009670 con 0x55e0a6e12920 2023-11-15T03:09:10.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.386+0000 7f6e171aa700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e0a6e12920 0x55e0a6eda990 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 rx=0x7f6e080086a0 tx=0x7f6e08018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:10.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.386+0000 7f6e149a5700 1 -- 172.21.15.121:0/2915141469 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0800f070 con 0x55e0a6e12920 2023-11-15T03:09:10.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.386+0000 7f6e149a5700 1 -- 172.21.15.121:0/2915141469 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e080056f0 con 0x55e0a6e12920 2023-11-15T03:09:10.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.386+0000 7f6e149a5700 1 -- 172.21.15.121:0/2915141469 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08013800 con 0x55e0a6e12920 2023-11-15T03:09:10.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.386+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e00005db0 con 0x55e0a6e12920 2023-11-15T03:09:10.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.386+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0a6db0a20 con 0x55e0a6e12920 2023-11-15T03:09:10.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.386+0000 7f6e149a5700 1 -- 172.21.15.121:0/2915141469 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6e08013c30 con 0x55e0a6e12920 2023-11-15T03:09:10.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.390+0000 7f6e149a5700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6df8058ff0 0x7f6df805b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.390+0000 7f6e149a5700 1 -- 172.21.15.121:0/2915141469 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7f6e080758f0 con 0x55e0a6e12920 2023-11-15T03:09:10.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.390+0000 7f6e0e9a9700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6df8058ff0 0x7f6df805b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.403 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:09:10.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.390+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55e0a6edb130 0x55e0a6ee39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.390+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6ee3eb0 con 0x55e0a6edb130 2023-11-15T03:09:10.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.390+0000 7f6e179ab700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55e0a6edb130 0x55e0a6ee39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.390+0000 7f6e0e9a9700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6df8058ff0 0x7f6df805b4b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x55e0a6ee8980 tx=0x7f6e00016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:10.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.390+0000 7f6e179ab700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55e0a6edb130 0x55e0a6ee39c0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:10.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.394+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v125'365 uv358 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6edb130 2023-11-15T03:09:10.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.394+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f048f0 con 0x55e0a6edb130 2023-11-15T03:09:10.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.398+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v125'366 uv366 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6edb130 2023-11-15T03:09:10.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.398+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f04ca0 con 0x55e0a6edb130 2023-11-15T03:09:10.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.402+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v125'367 uv367 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6edb130 2023-11-15T03:09:10.412 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:09:10.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.402+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f06060 con 0x55e0a6edb130 2023-11-15T03:09:10.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.402+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v125'368 uv368 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6edb130 2023-11-15T03:09:10.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.402+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f06410 con 0x55e0a6edb130 2023-11-15T03:09:10.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.406+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v125'369 uv369 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6edb130 2023-11-15T03:09:10.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.406+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f05050 con 0x55e0a6edb130 2023-11-15T03:09:10.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.410+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v125'370 uv370 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6edb130 2023-11-15T03:09:10.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.410+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f05450 con 0x55e0a6edb130 2023-11-15T03:09:10.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.410+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v125'371 uv364 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6edb130 2023-11-15T03:09:10.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.410+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f05850 con 0x55e0a6edb130 2023-11-15T03:09:10.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.414+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v125'372 uv372 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6edb130 2023-11-15T03:09:10.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.414+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f05850 con 0x55e0a6edb130 2023-11-15T03:09:10.427 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:09:10.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.418+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v125'373 uv373 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6edb130 2023-11-15T03:09:10.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.418+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f05800 con 0x55e0a6edb130 2023-11-15T03:09:10.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.418+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v125'374 uv374 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6edb130 2023-11-15T03:09:10.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.418+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f05800 con 0x55e0a6edb130 2023-11-15T03:09:10.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.422+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v125'375 uv375 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6edb130 2023-11-15T03:09:10.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.422+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f05800 con 0x55e0a6edb130 2023-11-15T03:09:10.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.426+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v125'376 uv376 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6edb130 2023-11-15T03:09:10.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.426+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e0a6f05c30 0x55e0a6f08ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.426+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f05800 con 0x55e0a6f05c30 2023-11-15T03:09:10.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.426+0000 7f6e171aa700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e0a6f05c30 0x55e0a6f08ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.426+0000 7f6e171aa700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e0a6f05c30 0x55e0a6f08ce0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:10.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.430+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v125'575 uv562 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.430+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f091d0 con 0x55e0a6f05c30 2023-11-15T03:09:10.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.434+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v125'576 uv576 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.434+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f09580 con 0x55e0a6f05c30 2023-11-15T03:09:10.447 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:09:10.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.438+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v125'577 uv577 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.438+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0a940 con 0x55e0a6f05c30 2023-11-15T03:09:10.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.438+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v125'578 uv578 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.438+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0acf0 con 0x55e0a6f05c30 2023-11-15T03:09:10.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.442+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v125'579 uv579 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.442+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f09930 con 0x55e0a6f05c30 2023-11-15T03:09:10.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.446+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v125'580 uv580 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.446+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0a220 con 0x55e0a6f05c30 2023-11-15T03:09:10.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.446+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v125'581 uv581 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.446+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0a220 con 0x55e0a6f05c30 2023-11-15T03:09:10.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.450+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v125'582 uv568 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.450+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f0a220 con 0x55e0a6f05c30 2023-11-15T03:09:10.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.454+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v125'583 uv583 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.454+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0a220 con 0x55e0a6f05c30 2023-11-15T03:09:10.466 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:09:10.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.458+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v125'584 uv584 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.458+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0a220 con 0x55e0a6f05c30 2023-11-15T03:09:10.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.458+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v125'585 uv585 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.458+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0a220 con 0x55e0a6f05c30 2023-11-15T03:09:10.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.462+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v125'586 uv586 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.462+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0a220 con 0x55e0a6f05c30 2023-11-15T03:09:10.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.466+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v125'587 uv587 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.466+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0a220 con 0x55e0a6f05c30 2023-11-15T03:09:10.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.470+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v125'588 uv574 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.470+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f0a220 con 0x55e0a6f05c30 2023-11-15T03:09:10.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.470+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v125'589 uv589 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.470+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0a220 con 0x55e0a6f05c30 2023-11-15T03:09:10.484 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:09:10.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.474+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v125'590 uv590 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.474+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0a220 con 0x55e0a6f05c30 2023-11-15T03:09:10.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.478+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v125'591 uv591 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.478+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0a220 con 0x55e0a6f05c30 2023-11-15T03:09:10.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.478+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v125'592 uv592 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.478+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0a220 con 0x55e0a6f05c30 2023-11-15T03:09:10.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.482+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v125'593 uv593 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.482+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55e0a6f0b0a0 0x55e0a6f0d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.482+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0d980 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.482+0000 7f6e0e9a9700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55e0a6f0b0a0 0x55e0a6f0d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.482+0000 7f6e0e9a9700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55e0a6f0b0a0 0x55e0a6f0d490 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:10.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.486+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v125'314 uv308 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.486+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f0dd30 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.490+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v125'315 uv315 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.490+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0e0e0 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.503 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:09:10.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.494+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v125'316 uv316 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.494+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0e980 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.498+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v125'317 uv317 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.498+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0ed30 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.498+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v125'318 uv318 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.498+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0f130 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.502+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v125'319 uv313 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.502+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f0f4e0 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.506+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v125'320 uv320 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.506+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0f890 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.518 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:09:10.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.510+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v125'321 uv321 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.510+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0f890 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.510+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v125'322 uv322 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.510+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f0f890 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.514+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v125'323 uv323 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.514+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e0a6f0fda0 0x55e0a6f121f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.514+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f12820 con 0x55e0a6f0fda0 2023-11-15T03:09:10.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.514+0000 7f6e179ab700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e0a6f0fda0 0x55e0a6f121f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.514+0000 7f6e179ab700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e0a6f0fda0 0x55e0a6f121f0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:10.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.518+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v125'366 uv359 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.518+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f128c0 con 0x55e0a6f0fda0 2023-11-15T03:09:10.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.522+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v125'367 uv367 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.522+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f12cc0 con 0x55e0a6f0fda0 2023-11-15T03:09:10.536 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:09:10.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.526+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v125'368 uv368 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.526+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f16d50 con 0x55e0a6f0fda0 2023-11-15T03:09:10.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.530+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v125'369 uv369 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.530+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f16d50 con 0x55e0a6f0fda0 2023-11-15T03:09:10.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.534+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v125'370 uv370 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.534+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f12cc0 con 0x55e0a6f0fda0 2023-11-15T03:09:10.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.534+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v125'371 uv365 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.534+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f12cc0 con 0x55e0a6f0fda0 2023-11-15T03:09:10.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.538+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v125'372 uv372 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.538+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f12cc0 con 0x55e0a6f0fda0 2023-11-15T03:09:10.551 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:09:10.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.542+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v125'373 uv373 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.542+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f13cd0 con 0x55e0a6f0fda0 2023-11-15T03:09:10.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.542+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v125'374 uv374 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.542+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f13cd0 con 0x55e0a6f0fda0 2023-11-15T03:09:10.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.546+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v125'375 uv375 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.546+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f12cc0 con 0x55e0a6f0fda0 2023-11-15T03:09:10.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.550+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v125'376 uv376 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.550+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55e0a6f13340 0x55e0a6f15730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.550+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f15cf0 con 0x55e0a6f13340 2023-11-15T03:09:10.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.550+0000 7f6e171aa700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55e0a6f13340 0x55e0a6f15730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.550+0000 7f6e171aa700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55e0a6f13340 0x55e0a6f15730 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:10.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.554+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v125'183 uv182 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f13340 2023-11-15T03:09:10.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.554+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f15e10 con 0x55e0a6f13340 2023-11-15T03:09:10.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.558+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v125'184 uv184 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f13340 2023-11-15T03:09:10.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.558+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f16210 con 0x55e0a6f13340 2023-11-15T03:09:10.569 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:09:10.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.558+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v125'185 uv185 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f13340 2023-11-15T03:09:10.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.558+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f17220 con 0x55e0a6f13340 2023-11-15T03:09:10.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.562+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v125'186 uv186 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f13340 2023-11-15T03:09:10.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.562+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f17220 con 0x55e0a6f13340 2023-11-15T03:09:10.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.566+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v125'187 uv187 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f13340 2023-11-15T03:09:10.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.566+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f16210 con 0x55e0a6f13340 2023-11-15T03:09:10.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.566+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v125'188 uv188 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f13340 2023-11-15T03:09:10.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.566+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e0a6f16720 0x55e0a6f18b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.566+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f191e0 con 0x55e0a6f16720 2023-11-15T03:09:10.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.566+0000 7f6e0e9a9700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e0a6f16720 0x55e0a6f18b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.570+0000 7f6e0e9a9700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e0a6f16720 0x55e0a6f18b10 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:10.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.570+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v125'129 uv128 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.570+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f19340 con 0x55e0a6f16720 2023-11-15T03:09:10.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.574+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v125'130 uv130 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.574+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f16720 2023-11-15T03:09:10.590 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:09:10.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.578+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v125'131 uv131 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.578+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f16720 2023-11-15T03:09:10.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.582+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v125'132 uv132 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.582+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f16720 2023-11-15T03:09:10.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.582+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v125'183 uv182 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.582+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f16720 2023-11-15T03:09:10.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.586+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v125'184 uv184 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.586+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f16720 2023-11-15T03:09:10.598 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:09:10.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.590+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v125'185 uv185 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.590+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1a780 con 0x55e0a6f16720 2023-11-15T03:09:10.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.590+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v125'186 uv186 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.590+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1a780 con 0x55e0a6f16720 2023-11-15T03:09:10.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.594+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v125'187 uv187 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.594+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f16720 2023-11-15T03:09:10.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.598+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v125'188 uv188 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.598+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f05c30 2023-11-15T03:09:10.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.602+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v125'338 uv332 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.602+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f05c30 2023-11-15T03:09:10.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.602+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v125'339 uv339 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.602+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f05c30 2023-11-15T03:09:10.617 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:09:10.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.606+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v125'340 uv340 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.606+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1a780 con 0x55e0a6f05c30 2023-11-15T03:09:10.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.610+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v125'341 uv341 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.610+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1a780 con 0x55e0a6f05c30 2023-11-15T03:09:10.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.614+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v125'342 uv342 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.614+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f05c30 2023-11-15T03:09:10.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.614+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v125'343 uv343 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.618+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f05c30 2023-11-15T03:09:10.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.618+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v125'344 uv337 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.618+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f05c30 2023-11-15T03:09:10.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.622+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v125'345 uv345 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.622+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f05c30 2023-11-15T03:09:10.636 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:09:10.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.626+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v125'346 uv346 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.626+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f05c30 2023-11-15T03:09:10.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.630+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v125'347 uv347 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.630+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f05c30 2023-11-15T03:09:10.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.634+0000 7f6e171aa700 1 -- 172.21.15.121:0/2915141469 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v125'348 uv348 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e0804a6a0 con 0x55e0a6f05c30 2023-11-15T03:09:10.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.634+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.634+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v125'153 uv152 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.634+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.638+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v125'154 uv154 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.638+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.650 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:09:10.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.642+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v125'155 uv155 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.642+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.642+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v125'156 uv156 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.642+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.646+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v125'157 uv157 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f0b0a0 2023-11-15T03:09:10.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.646+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f16720 2023-11-15T03:09:10.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.650+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v125'183 uv182 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.650+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f16720 2023-11-15T03:09:10.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.654+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v125'184 uv184 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.654+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f16720 2023-11-15T03:09:10.665 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:09:10.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.654+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v125'185 uv185 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.654+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1a780 con 0x55e0a6f16720 2023-11-15T03:09:10.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.658+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v125'186 uv186 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.658+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1a780 con 0x55e0a6f16720 2023-11-15T03:09:10.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.662+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v125'187 uv187 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.662+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f16720 2023-11-15T03:09:10.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.666+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v125'188 uv188 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.666+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f16720 2023-11-15T03:09:10.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.670+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v125'154 uv153 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.670+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f16720 2023-11-15T03:09:10.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.674+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v125'155 uv155 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.674+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f16720 2023-11-15T03:09:10.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.674+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v125'156 uv156 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.685 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:09:10.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.674+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f16720 2023-11-15T03:09:10.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.678+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v125'157 uv157 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.678+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f16720 2023-11-15T03:09:10.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.682+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v125'158 uv158 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.682+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f16720 2023-11-15T03:09:10.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.686+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v125'151 uv150 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.686+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f16720 2023-11-15T03:09:10.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.686+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v125'152 uv152 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.690+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f16720 2023-11-15T03:09:10.700 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:09:10.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.690+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v125'153 uv153 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.690+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f19770 con 0x55e0a6f16720 2023-11-15T03:09:10.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.694+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v125'154 uv154 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.694+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1c4e0 con 0x55e0a6f16720 2023-11-15T03:09:10.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.698+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v125'155 uv155 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.698+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55e0a6f1a2e0 0x55e0a6f1a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.698+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f1a2e0 2023-11-15T03:09:10.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.698+0000 7f6e179ab700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55e0a6f1a2e0 0x55e0a6f1a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.698+0000 7f6e179ab700 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55e0a6f1a2e0 0x55e0a6f1a6c0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:10.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.702+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v125'122 uv121 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f1a2e0 2023-11-15T03:09:10.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.702+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f1a2e0 2023-11-15T03:09:10.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.706+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v125'123 uv123 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f1a2e0 2023-11-15T03:09:10.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.706+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f1a2e0 2023-11-15T03:09:10.718 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:09:10.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.710+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v125'124 uv124 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f1a2e0 2023-11-15T03:09:10.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.710+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f1a2e0 2023-11-15T03:09:10.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.714+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v125'125 uv125 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f1a2e0 2023-11-15T03:09:10.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.714+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f16720 2023-11-15T03:09:10.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.714+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v125'183 uv182 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.714+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f16720 2023-11-15T03:09:10.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.718+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v125'184 uv184 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.718+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f16720 2023-11-15T03:09:10.731 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:09:10.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.722+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v125'185 uv185 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.722+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f16720 2023-11-15T03:09:10.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.726+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v125'186 uv186 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.726+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f16720 2023-11-15T03:09:10.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.726+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v125'187 uv187 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.726+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f16720 2023-11-15T03:09:10.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.730+0000 7f6e0e9a9700 1 -- 172.21.15.121:0/2915141469 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v125'188 uv188 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e00008970 con 0x55e0a6f16720 2023-11-15T03:09:10.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.730+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f0fda0 2023-11-15T03:09:10.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.734+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v125'156 uv155 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.734+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f0fda0 2023-11-15T03:09:10.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.734+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v125'157 uv157 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.734+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f0fda0 2023-11-15T03:09:10.748 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:09:10.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.738+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v125'158 uv158 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.738+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f0fda0 2023-11-15T03:09:10.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.742+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v125'159 uv159 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.742+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f0fda0 2023-11-15T03:09:10.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.742+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v125'160 uv160 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.746+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f0fda0 2023-11-15T03:09:10.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.746+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v125'366 uv365 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.746+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f0fda0 2023-11-15T03:09:10.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.750+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v125'367 uv367 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.750+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f0fda0 2023-11-15T03:09:10.762 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:09:10.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.754+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v125'368 uv368 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.754+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f0fda0 2023-11-15T03:09:10.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.754+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v125'369 uv369 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.754+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1b070 con 0x55e0a6f0fda0 2023-11-15T03:09:10.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.758+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v125'370 uv370 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.758+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1b070 con 0x55e0a6f0fda0 2023-11-15T03:09:10.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.762+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v125'371 uv364 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.762+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f0fda0 2023-11-15T03:09:10.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.762+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v125'372 uv372 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.762+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f0fda0 2023-11-15T03:09:10.775 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:09:10.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.766+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v125'373 uv373 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.766+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f0fda0 2023-11-15T03:09:10.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.770+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v125'374 uv374 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.770+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f0fda0 2023-11-15T03:09:10.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.774+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v125'375 uv375 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.774+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1ac20 con 0x55e0a6f0fda0 2023-11-15T03:09:10.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.774+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v125'376 uv376 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.774+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e0a6f1b020 con 0x55e0a6f0fda0 2023-11-15T03:09:10.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v125'377 uv377 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f6e1000db80 con 0x55e0a6f0fda0 2023-11-15T03:09:10.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55e0a6f0b0a0 msgr2=0x55e0a6f0d490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55e0a6f0b0a0 0x55e0a6f0d490 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55e0a6edb130 msgr2=0x55e0a6ee39c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55e0a6edb130 0x55e0a6ee39c0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55e0a6f13340 msgr2=0x55e0a6f15730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55e0a6f13340 0x55e0a6f15730 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e0a6f05c30 msgr2=0x55e0a6f08ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e0a6f05c30 0x55e0a6f08ce0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55e0a6f1a2e0 msgr2=0x55e0a6f1a6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55e0a6f1a2e0 0x55e0a6f1a6c0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e0a6f0fda0 msgr2=0x55e0a6f121f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e0a6f0fda0 0x55e0a6f121f0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e0a6f16720 msgr2=0x55e0a6f18b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e0a6f16720 0x55e0a6f18b10 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 reap_dead start 2023-11-15T03:09:10.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 reap_dead start 2023-11-15T03:09:10.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 reap_dead start 2023-11-15T03:09:10.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e179ab700 1 -- 172.21.15.121:0/2915141469 reap_dead start 2023-11-15T03:09:10.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6df8058ff0 msgr2=0x7f6df805b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6df8058ff0 0x7f6df805b4b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x55e0a6ee8980 tx=0x7f6e00016000).stop 2023-11-15T03:09:10.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e0a6e12920 msgr2=0x55e0a6eda990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.778+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e0a6e12920 0x55e0a6eda990 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 rx=0x7f6e080086a0 tx=0x7f6e08018000).stop 2023-11-15T03:09:10.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.782+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 shutdown_connections 2023-11-15T03:09:10.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.782+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6df8058ff0 0x7f6df805b4b0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.782+0000 7f6e22b42d00 1 --2- 172.21.15.121:0/2915141469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e0a6e12920 0x55e0a6eda990 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.782+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 >> 172.21.15.121:0/2915141469 conn(0x55e0a6e588f0 msgr2=0x55e0a6e57120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:10.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.782+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 shutdown_connections 2023-11-15T03:09:10.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.782+0000 7f6e22b42d00 1 -- 172.21.15.121:0/2915141469 wait complete. 2023-11-15T03:09:10.797 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 sixmegs_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-11-15T03:09:10.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa6a9b38d00 1 Processor -- start 2023-11-15T03:09:10.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa6a9b38d00 1 -- start start 2023-11-15T03:09:10.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa6a9b38d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e2a807ede0 0x55e2a8151010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa6a9b38d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e2a8157680 0x55e2a80c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa6a9b38d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2a815a9c0 0x55e2a80c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa6a9b38d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a7ff7220 con 0x55e2a807ede0 2023-11-15T03:09:10.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa6a9b38d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a80b1520 con 0x55e2a815a9c0 2023-11-15T03:09:10.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa6a9b38d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2a8150990 con 0x55e2a8157680 2023-11-15T03:09:10.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa69e185700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e2a807ede0 0x55e2a8151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa69e185700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e2a807ede0 0x55e2a8151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50760/0 (socket says 172.21.15.121:50760) 2023-11-15T03:09:10.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa69e185700 1 -- 172.21.15.121:0/1788955445 learned_addr learned my addr 172.21.15.121:0/1788955445 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:10.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa695984700 1 --2- 172.21.15.121:0/1788955445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e2a8157680 0x55e2a80c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa69e986700 1 --2- 172.21.15.121:0/1788955445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2a815a9c0 0x55e2a80c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa69e185700 1 -- 172.21.15.121:0/1788955445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e2a8157680 msgr2=0x55e2a80c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:10.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa69e185700 1 --2- 172.21.15.121:0/1788955445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e2a8157680 0x55e2a80c6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa69e185700 1 -- 172.21.15.121:0/1788955445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2a815a9c0 msgr2=0x55e2a80c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:10.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa69e185700 1 --2- 172.21.15.121:0/1788955445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2a815a9c0 0x55e2a80c70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa69e185700 1 -- 172.21.15.121:0/1788955445 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa690005db0 con 0x55e2a807ede0 2023-11-15T03:09:10.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa695984700 1 --2- 172.21.15.121:0/1788955445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e2a8157680 0x55e2a80c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:09:10.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa69e185700 1 --2- 172.21.15.121:0/1788955445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e2a807ede0 0x55e2a8151010 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 rx=0x55e2a806be50 tx=0x7fa690016000).ready entity=mon.0 client_cookie=e3aaf4c25070 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:10.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa69d183700 1 -- 172.21.15.121:0/1788955445 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa690000bc0 con 0x55e2a807ede0 2023-11-15T03:09:10.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa69d183700 1 -- 172.21.15.121:0/1788955445 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6900089d0 con 0x55e2a807ede0 2023-11-15T03:09:10.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.890+0000 7fa69d183700 1 -- 172.21.15.121:0/1788955445 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa690000d20 con 0x55e2a807ede0 2023-11-15T03:09:10.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.894+0000 7fa6a9b38d00 1 -- 172.21.15.121:0/1788955445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e2a807ede0 msgr2=0x55e2a8151010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.894+0000 7fa6a9b38d00 1 --2- 172.21.15.121:0/1788955445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e2a807ede0 0x55e2a8151010 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 rx=0x55e2a806be50 tx=0x7fa690016000).stop 2023-11-15T03:09:10.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.894+0000 7fa6a9b38d00 1 -- 172.21.15.121:0/1788955445 shutdown_connections 2023-11-15T03:09:10.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.894+0000 7fa6a9b38d00 1 --2- 172.21.15.121:0/1788955445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2a815a9c0 0x55e2a80c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.894+0000 7fa6a9b38d00 1 --2- 172.21.15.121:0/1788955445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e2a8157680 0x55e2a80c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.894+0000 7fa6a9b38d00 1 --2- 172.21.15.121:0/1788955445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e2a807ede0 0x55e2a8151010 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.894+0000 7fa6a9b38d00 1 -- 172.21.15.121:0/1788955445 >> 172.21.15.121:0/1788955445 conn(0x55e2a80ce090 msgr2=0x55e2a8159920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:10.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.894+0000 7fa6a9b38d00 1 -- 172.21.15.121:0/1788955445 shutdown_connections 2023-11-15T03:09:10.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.894+0000 7fa6a9b38d00 1 -- 172.21.15.121:0/1788955445 wait complete. 2023-11-15T03:09:10.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.894+0000 7fa6a9b38d00 1 Processor -- start 2023-11-15T03:09:10.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.894+0000 7fa6a9b38d00 1 -- start start 2023-11-15T03:09:10.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa6a9b38d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2a807ede0 0x55e2a80b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa6a9b38d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e2a8157680 0x55e2a814fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa6a9b38d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e2a815a9c0 0x55e2a80c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa6a9b38d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a81571b0 con 0x55e2a815a9c0 2023-11-15T03:09:10.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa6a9b38d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a80c42c0 con 0x55e2a807ede0 2023-11-15T03:09:10.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa6a9b38d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2a81569a0 con 0x55e2a8157680 2023-11-15T03:09:10.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa69e986700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e2a815a9c0 0x55e2a80c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa69e986700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e2a815a9c0 0x55e2a80c6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50770/0 (socket says 172.21.15.121:50770) 2023-11-15T03:09:10.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa69e986700 1 -- 172.21.15.121:0/166868531 learned_addr learned my addr 172.21.15.121:0/166868531 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:10.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa69e185700 1 --2- 172.21.15.121:0/166868531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2a807ede0 0x55e2a80b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa695984700 1 --2- 172.21.15.121:0/166868531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e2a8157680 0x55e2a814fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa69e986700 1 -- 172.21.15.121:0/166868531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e2a8157680 msgr2=0x55e2a814fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa69e986700 1 --2- 172.21.15.121:0/166868531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e2a8157680 0x55e2a814fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa69e986700 1 -- 172.21.15.121:0/166868531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2a807ede0 msgr2=0x55e2a80b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa69e986700 1 --2- 172.21.15.121:0/166868531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2a807ede0 0x55e2a80b0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa69e986700 1 -- 172.21.15.121:0/166868531 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2a8020a20 con 0x55e2a815a9c0 2023-11-15T03:09:10.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa69e986700 1 --2- 172.21.15.121:0/166868531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e2a815a9c0 0x55e2a80c6bd0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 rx=0x7fa69800f930 tx=0x7fa698007040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:10.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa6977fe700 1 -- 172.21.15.121:0/166868531 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698002460 con 0x55e2a815a9c0 2023-11-15T03:09:10.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa6977fe700 1 -- 172.21.15.121:0/166868531 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa698011610 con 0x55e2a815a9c0 2023-11-15T03:09:10.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa6977fe700 1 -- 172.21.15.121:0/166868531 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698011990 con 0x55e2a815a9c0 2023-11-15T03:09:10.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa6a9b38d00 1 -- 172.21.15.121:0/166868531 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e2a8020770 con 0x55e2a815a9c0 2023-11-15T03:09:10.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.898+0000 7fa6a9b38d00 1 -- 172.21.15.121:0/166868531 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2a7fefa70 con 0x55e2a815a9c0 2023-11-15T03:09:10.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.902+0000 7fa6977fe700 1 -- 172.21.15.121:0/166868531 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa698003040 con 0x55e2a815a9c0 2023-11-15T03:09:10.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.902+0000 7fa6977fe700 1 --2- 172.21.15.121:0/166868531 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa680059020 0x7fa68005b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.906+0000 7fa6977fe700 1 -- 172.21.15.121:0/166868531 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7fa6980745f0 con 0x55e2a815a9c0 2023-11-15T03:09:10.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.906+0000 7fa69e185700 1 --2- 172.21.15.121:0/166868531 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa680059020 0x7fa68005b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.906+0000 7fa6a9b38d00 1 --2- 172.21.15.121:0/166868531 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e2a80c7950 0x55e2a8168540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:10.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.906+0000 7fa6a9b38d00 1 -- 172.21.15.121:0/166868531 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:1 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=104b] snapc 0=[] ondisk+write+known_if_redirected e125) v8 -- 0x55e2a8150fc0 con 0x55e2a80c7950 2023-11-15T03:09:10.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.906+0000 7fa69e185700 1 --2- 172.21.15.121:0/166868531 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa680059020 0x7fa68005b4e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7fa69001f040 tx=0x7fa69001c000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:10.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.906+0000 7fa695984700 1 --2- 172.21.15.121:0/166868531 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e2a80c7950 0x55e2a8168540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:10.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.906+0000 7fa695984700 1 --2- 172.21.15.121:0/166868531 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e2a80c7950 0x55e2a8168540 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:10.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.910+0000 7fa695984700 1 -- 172.21.15.121:0/166868531 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v125'594 uv594 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa68800e960 con 0x55e2a80c7950 2023-11-15T03:09:10.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.910+0000 7fa6a9b38d00 1 -- 172.21.15.121:0/166868531 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e2a80c7950 msgr2=0x55e2a8168540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.910+0000 7fa6a9b38d00 1 --2- 172.21.15.121:0/166868531 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e2a80c7950 0x55e2a8168540 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:10.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.910+0000 7fa6a9b38d00 1 -- 172.21.15.121:0/166868531 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa680059020 msgr2=0x7fa68005b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.910+0000 7fa6a9b38d00 1 --2- 172.21.15.121:0/166868531 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa680059020 0x7fa68005b4e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7fa69001f040 tx=0x7fa69001c000).stop 2023-11-15T03:09:10.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.910+0000 7fa6a9b38d00 1 -- 172.21.15.121:0/166868531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e2a815a9c0 msgr2=0x55e2a80c6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:10.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.910+0000 7fa6a9b38d00 1 --2- 172.21.15.121:0/166868531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e2a815a9c0 0x55e2a80c6bd0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 rx=0x7fa69800f930 tx=0x7fa698007040).stop 2023-11-15T03:09:10.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.910+0000 7fa69e986700 1 -- 172.21.15.121:0/166868531 reap_dead start 2023-11-15T03:09:10.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.910+0000 7fa6a9b38d00 1 -- 172.21.15.121:0/166868531 shutdown_connections 2023-11-15T03:09:10.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.910+0000 7fa6a9b38d00 1 -- 172.21.15.121:0/166868531 >> 172.21.15.121:0/166868531 conn(0x55e2a80ce090 msgr2=0x55e2a8159920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:10.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.910+0000 7fa6a9b38d00 1 -- 172.21.15.121:0/166868531 shutdown_connections 2023-11-15T03:09:10.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:10.910+0000 7fa6a9b38d00 1 -- 172.21.15.121:0/166868531 wait complete. 2023-11-15T03:09:10.926 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:09:11.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.082+0000 7f0c7b59e700 1 Processor -- start 2023-11-15T03:09:11.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.082+0000 7f0c7b59e700 1 -- start start 2023-11-15T03:09:11.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c7b59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c7c0a7070 0x7f0c7c0c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:11.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c7b59e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c7c1528b0 con 0x7f0c7c0c3430 2023-11-15T03:09:11.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c7b59e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c7c0a74d0 con 0x7f0c7c151af0 2023-11-15T03:09:11.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c7b59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c151ff0 con 0x7f0c7c0a7070 2023-11-15T03:09:11.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c7ad9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f0c7c0c3430 0x7f0c7c065d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34258/0 (socket says 172.21.15.121:34258) 2023-11-15T03:09:11.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c7ad9d700 1 -- 172.21.15.121:0/3493287252 learned_addr learned my addr 172.21.15.121:0/3493287252 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:09:11.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c71d9b700 1 --2- 172.21.15.121:0/3493287252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c7c0a7070 0x7f0c7c0c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:11.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c79d9b700 1 -- 172.21.15.121:0/3493287252 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1435743214 0 0) 0x7f0c7c1528b0 con 0x7f0c7c0c3430 2023-11-15T03:09:11.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c79d9b700 1 -- 172.21.15.121:0/3493287252 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c5c0033f0 con 0x7f0c7c0c3430 2023-11-15T03:09:11.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c79d9b700 1 -- 172.21.15.121:0/3493287252 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072106465 0 0) 0x7f0c7c0a74d0 con 0x7f0c7c151af0 2023-11-15T03:09:11.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c79d9b700 1 -- 172.21.15.121:0/3493287252 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c7c1528b0 con 0x7f0c7c151af0 2023-11-15T03:09:11.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c71d9b700 1 -- 172.21.15.121:0/3493287252 >> v1:172.21.15.190:6789/0 conn(0x7f0c7c151af0 legacy=0x7f0c7c0a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:11.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c71d9b700 1 -- 172.21.15.121:0/3493287252 >> v1:172.21.15.121:6789/0 conn(0x7f0c7c0c3430 legacy=0x7f0c7c065d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:11.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c71d9b700 1 -- 172.21.15.121:0/3493287252 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c0c3890 con 0x7f0c7c0a7070 2023-11-15T03:09:11.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c71d9b700 1 --2- 172.21.15.121:0/3493287252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c7c0a7070 0x7f0c7c0c2f40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f0c64009790 tx=0x7f0c64009060).ready entity=mon.2 client_cookie=d778e94da859c80f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:11.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c79d9b700 1 -- 172.21.15.121:0/3493287252 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6400f510 con 0x7f0c7c0a7070 2023-11-15T03:09:11.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c79d9b700 1 -- 172.21.15.121:0/3493287252 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6400f670 con 0x7f0c7c0a7070 2023-11-15T03:09:11.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c79d9b700 1 -- 172.21.15.121:0/3493287252 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6400f9f0 con 0x7f0c7c0a7070 2023-11-15T03:09:11.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c7b59e700 1 -- 172.21.15.121:0/3493287252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c7c0a7070 msgr2=0x7f0c7c0c2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:11.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.086+0000 7f0c7b59e700 1 --2- 172.21.15.121:0/3493287252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c7c0a7070 0x7f0c7c0c2f40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f0c64009790 tx=0x7f0c64009060).stop 2023-11-15T03:09:11.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.090+0000 7f0c7b59e700 1 -- 172.21.15.121:0/3493287252 shutdown_connections 2023-11-15T03:09:11.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.090+0000 7f0c7b59e700 1 --2- 172.21.15.121:0/3493287252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c7c0a7070 0x7f0c7c0c2f40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:11.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.090+0000 7f0c7b59e700 1 -- 172.21.15.121:0/3493287252 >> 172.21.15.121:0/3493287252 conn(0x7f0c7c067a60 msgr2=0x7f0c7c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:11.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.090+0000 7f0c7b59e700 1 -- 172.21.15.121:0/3493287252 shutdown_connections 2023-11-15T03:09:11.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.090+0000 7f0c7b59e700 1 -- 172.21.15.121:0/3493287252 wait complete. 2023-11-15T03:09:11.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.090+0000 7f0c7b59e700 1 Processor -- start 2023-11-15T03:09:11.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.090+0000 7f0c7b59e700 1 -- start start 2023-11-15T03:09:11.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c7b59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c7c0a7070 0x7f0c7c142f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:11.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c7b59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c7c0c3430 0x7f0c7c0bc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:11.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c7b59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c7c0bfe40 0x7f0c7c0bcb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:11.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c7b59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c0beda0 con 0x7f0c7c0bfe40 2023-11-15T03:09:11.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c7b59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c0bf080 con 0x7f0c7c0c3430 2023-11-15T03:09:11.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c7b59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c0bf360 con 0x7f0c7c0a7070 2023-11-15T03:09:11.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c7ad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c7c0bfe40 0x7f0c7c0bcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:11.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c7ad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c7c0bfe40 0x7f0c7c0bcb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50774/0 (socket says 172.21.15.121:50774) 2023-11-15T03:09:11.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c7ad9d700 1 -- 172.21.15.121:0/3173072621 learned_addr learned my addr 172.21.15.121:0/3173072621 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:11.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c71d9b700 1 --2- 172.21.15.121:0/3173072621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c7c0c3430 0x7f0c7c0bc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:11.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c7a59c700 1 --2- 172.21.15.121:0/3173072621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c7c0a7070 0x7f0c7c142f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:11.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c7ad9d700 1 -- 172.21.15.121:0/3173072621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c7c0a7070 msgr2=0x7f0c7c142f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:11.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c7ad9d700 1 --2- 172.21.15.121:0/3173072621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c7c0a7070 0x7f0c7c142f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:11.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c7ad9d700 1 -- 172.21.15.121:0/3173072621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c7c0c3430 msgr2=0x7f0c7c0bc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:11.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c7ad9d700 1 --2- 172.21.15.121:0/3173072621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c7c0c3430 0x7f0c7c0bc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:11.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c7ad9d700 1 -- 172.21.15.121:0/3173072621 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c640092a0 con 0x7f0c7c0bfe40 2023-11-15T03:09:11.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c7ad9d700 1 --2- 172.21.15.121:0/3173072621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c7c0bfe40 0x7f0c7c0bcb50 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 rx=0x7f0c74001560 tx=0x7f0c74001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:11.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c72ffd700 1 -- 172.21.15.121:0/3173072621 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74005d30 con 0x7f0c7c0bfe40 2023-11-15T03:09:11.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c72ffd700 1 -- 172.21.15.121:0/3173072621 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c74005e90 con 0x7f0c7c0bfe40 2023-11-15T03:09:11.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c72ffd700 1 -- 172.21.15.121:0/3173072621 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74007340 con 0x7f0c7c0bfe40 2023-11-15T03:09:11.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c7b59e700 1 -- 172.21.15.121:0/3173072621 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7c0bf640 con 0x7f0c7c0bfe40 2023-11-15T03:09:11.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.094+0000 7f0c7b59e700 1 -- 172.21.15.121:0/3173072621 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7c0bea50 con 0x7f0c7c0bfe40 2023-11-15T03:09:11.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.098+0000 7f0c7b59e700 1 -- 172.21.15.121:0/3173072621 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7c06bcf0 con 0x7f0c7c0bfe40 2023-11-15T03:09:11.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.098+0000 7f0c72ffd700 1 -- 172.21.15.121:0/3173072621 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0c740076b0 con 0x7f0c7c0bfe40 2023-11-15T03:09:11.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.098+0000 7f0c72ffd700 1 --2- 172.21.15.121:0/3173072621 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0c6005a510 0x7f0c6005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:11.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.098+0000 7f0c72ffd700 1 -- 172.21.15.121:0/3173072621 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7f0c740753b0 con 0x7f0c7c0bfe40 2023-11-15T03:09:11.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.098+0000 7f0c7a59c700 1 --2- 172.21.15.121:0/3173072621 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0c6005a510 0x7f0c6005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:11.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.102+0000 7f0c7a59c700 1 --2- 172.21.15.121:0/3173072621 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0c6005a510 0x7f0c6005c9d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f0c6c003660 tx=0x7f0c6c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:11.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.102+0000 7f0c72ffd700 1 -- 172.21.15.121:0/3173072621 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0c74047ff0 con 0x7f0c7c0bfe40 2023-11-15T03:09:11.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:11.306+0000 7f0c7b59e700 1 -- 172.21.15.121:0/3173072621 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f0c7c0a7980 con 0x7f0c7c0bfe40 2023-11-15T03:09:12.233 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:09:12.240 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:09:12.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.298+0000 7f0c72ffd700 1 -- 172.21.15.121:0/3173072621 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v275) v1 ==== 162+0+0 (secure 0 0 0) 0x7f0c7404b610 con 0x7f0c7c0bfe40 2023-11-15T03:09:12.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.302+0000 7f0c7b59e700 1 -- 172.21.15.121:0/3173072621 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0c6005a510 msgr2=0x7f0c6005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:12.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.302+0000 7f0c7b59e700 1 --2- 172.21.15.121:0/3173072621 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0c6005a510 0x7f0c6005c9d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f0c6c003660 tx=0x7f0c6c009040).stop 2023-11-15T03:09:12.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.302+0000 7f0c7b59e700 1 -- 172.21.15.121:0/3173072621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c7c0bfe40 msgr2=0x7f0c7c0bcb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:12.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.302+0000 7f0c7b59e700 1 --2- 172.21.15.121:0/3173072621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c7c0bfe40 0x7f0c7c0bcb50 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 rx=0x7f0c74001560 tx=0x7f0c74001650).stop 2023-11-15T03:09:12.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.302+0000 7f0c7b59e700 1 -- 172.21.15.121:0/3173072621 shutdown_connections 2023-11-15T03:09:12.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.302+0000 7f0c7b59e700 1 --2- 172.21.15.121:0/3173072621 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0c6005a510 0x7f0c6005c9d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:12.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.302+0000 7f0c7b59e700 1 --2- 172.21.15.121:0/3173072621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c7c0bfe40 0x7f0c7c0bcb50 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:12.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.302+0000 7f0c7b59e700 1 --2- 172.21.15.121:0/3173072621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c7c0c3430 0x7f0c7c0bc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:12.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.302+0000 7f0c7b59e700 1 --2- 172.21.15.121:0/3173072621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c7c0a7070 0x7f0c7c142f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:12.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.302+0000 7f0c7b59e700 1 -- 172.21.15.121:0/3173072621 >> 172.21.15.121:0/3173072621 conn(0x7f0c7c067a60 msgr2=0x7f0c7c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:12.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.302+0000 7f0c7b59e700 1 -- 172.21.15.121:0/3173072621 shutdown_connections 2023-11-15T03:09:12.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.302+0000 7f0c7b59e700 1 -- 172.21.15.121:0/3173072621 wait complete. 2023-11-15T03:09:12.315 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:09:12.735 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:09:12.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.894+0000 7f668d439700 1 Processor -- start 2023-11-15T03:09:12.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.894+0000 7f668d439700 1 -- start start 2023-11-15T03:09:12.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.898+0000 7f668d439700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6688066320 0x7f6688154110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:12.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.898+0000 7f668d439700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f66880acf10 0x7f6688062190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:12.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.898+0000 7f668d439700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66880c73e0 con 0x7f6688066700 2023-11-15T03:09:12.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.898+0000 7f668d439700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66881546c0 con 0x7f66880acf10 2023-11-15T03:09:12.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.898+0000 7f668d439700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66881549a0 con 0x7f6688066320 2023-11-15T03:09:12.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.898+0000 7f668cc38700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f66880acf10 0x7f6688062190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:12.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.898+0000 7f668cc38700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f66880acf10 0x7f6688062190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50782/0 (socket says 172.21.15.121:50782) 2023-11-15T03:09:12.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.898+0000 7f668cc38700 1 -- 172.21.15.121:0/3797523773 learned_addr learned my addr 172.21.15.121:0/3797523773 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:12.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.898+0000 7f6687fff700 1 --2- 172.21.15.121:0/3797523773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6688066320 0x7f6688154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:12.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.898+0000 7f668cc38700 1 -- 172.21.15.121:0/3797523773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6688066320 msgr2=0x7f6688154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:12.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.898+0000 7f668cc38700 1 --2- 172.21.15.121:0/3797523773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6688066320 0x7f6688154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:12.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.898+0000 7f668cc38700 1 -- 172.21.15.121:0/3797523773 >> v1:172.21.15.190:6789/0 conn(0x7f6688066700 legacy=0x7f66880656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:12.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.898+0000 7f668cc38700 1 -- 172.21.15.121:0/3797523773 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6688154c80 con 0x7f66880acf10 2023-11-15T03:09:12.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.898+0000 7f668cc38700 1 --2- 172.21.15.121:0/3797523773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66880acf10 0x7f6688062190 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 rx=0x7f6680009f20 tx=0x7f6680009840).ready entity=mon.0 client_cookie=f1f32d0560589e68 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:12.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.898+0000 7f66877fe700 1 -- 172.21.15.121:0/3797523773 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668000e070 con 0x7f66880acf10 2023-11-15T03:09:12.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.898+0000 7f66877fe700 1 -- 172.21.15.121:0/3797523773 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668000b9d0 con 0x7f66880acf10 2023-11-15T03:09:12.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.898+0000 7f66877fe700 1 -- 172.21.15.121:0/3797523773 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668000bd50 con 0x7f66880acf10 2023-11-15T03:09:12.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.902+0000 7f668d439700 1 -- 172.21.15.121:0/3797523773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66880acf10 msgr2=0x7f6688062190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:12.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.902+0000 7f668d439700 1 --2- 172.21.15.121:0/3797523773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66880acf10 0x7f6688062190 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 rx=0x7f6680009f20 tx=0x7f6680009840).stop 2023-11-15T03:09:12.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.902+0000 7f668d439700 1 -- 172.21.15.121:0/3797523773 shutdown_connections 2023-11-15T03:09:12.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.902+0000 7f668d439700 1 --2- 172.21.15.121:0/3797523773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66880acf10 0x7f6688062190 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:12.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.902+0000 7f668d439700 1 --2- 172.21.15.121:0/3797523773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6688066320 0x7f6688154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:12.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.902+0000 7f668d439700 1 -- 172.21.15.121:0/3797523773 >> 172.21.15.121:0/3797523773 conn(0x7f6688067a60 msgr2=0x7f6688067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:12.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.902+0000 7f668d439700 1 -- 172.21.15.121:0/3797523773 shutdown_connections 2023-11-15T03:09:12.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.902+0000 7f668d439700 1 -- 172.21.15.121:0/3797523773 wait complete. 2023-11-15T03:09:12.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.902+0000 7f668d439700 1 Processor -- start 2023-11-15T03:09:12.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f668d439700 1 -- start start 2023-11-15T03:09:12.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f668d439700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6688066320 0x7f668814a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:12.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f668d439700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6688066700 0x7f66880ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:12.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f668d439700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66880acf10 0x7f6688150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:12.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f668d439700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66880af220 con 0x7f6688066700 2023-11-15T03:09:12.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f668d439700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668814edf0 con 0x7f66880acf10 2023-11-15T03:09:12.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f668d439700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66880ae980 con 0x7f6688066320 2023-11-15T03:09:12.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f667ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6688066700 0x7f66880ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:12.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f667ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6688066700 0x7f66880ae0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:50796/0 (socket says 172.21.15.121:50796) 2023-11-15T03:09:12.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f667ffff700 1 -- 172.21.15.121:0/4286636455 learned_addr learned my addr 172.21.15.121:0/4286636455 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:12.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f668cc38700 1 --2- 172.21.15.121:0/4286636455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66880acf10 0x7f6688150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:12.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f6687fff700 1 --2- 172.21.15.121:0/4286636455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6688066320 0x7f668814a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:12.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f667ffff700 1 -- 172.21.15.121:0/4286636455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6688066320 msgr2=0x7f668814a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:12.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f667ffff700 1 --2- 172.21.15.121:0/4286636455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6688066320 0x7f668814a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:12.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f667ffff700 1 -- 172.21.15.121:0/4286636455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66880acf10 msgr2=0x7f6688150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:12.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f667ffff700 1 --2- 172.21.15.121:0/4286636455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66880acf10 0x7f6688150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:12.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f667ffff700 1 -- 172.21.15.121:0/4286636455 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6680009ae0 con 0x7f6688066700 2023-11-15T03:09:12.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f667ffff700 1 --2- 172.21.15.121:0/4286636455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6688066700 0x7f66880ae0a0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 rx=0x7f667000c4f0 tx=0x7f6670007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:12.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f6684ff9700 1 -- 172.21.15.121:0/4286636455 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670007020 con 0x7f6688066700 2023-11-15T03:09:12.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f6684ff9700 1 -- 172.21.15.121:0/4286636455 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6670007180 con 0x7f6688066700 2023-11-15T03:09:12.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f6684ff9700 1 -- 172.21.15.121:0/4286636455 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670007540 con 0x7f6688066700 2023-11-15T03:09:12.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.906+0000 7f668d439700 1 -- 172.21.15.121:0/4286636455 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66880aec60 con 0x7f6688066700 2023-11-15T03:09:12.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.910+0000 7f668d439700 1 -- 172.21.15.121:0/4286636455 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668814ad20 con 0x7f6688066700 2023-11-15T03:09:12.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.910+0000 7f668d439700 1 -- 172.21.15.121:0/4286636455 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668806bcf0 con 0x7f6688066700 2023-11-15T03:09:12.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.910+0000 7f6684ff9700 1 -- 172.21.15.121:0/4286636455 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f66700068b0 con 0x7f6688066700 2023-11-15T03:09:12.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.914+0000 7f6684ff9700 1 --2- 172.21.15.121:0/4286636455 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f666805a510 0x7f666805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:12.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.914+0000 7f6684ff9700 1 -- 172.21.15.121:0/4286636455 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7f6670074770 con 0x7f6688066700 2023-11-15T03:09:12.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.914+0000 7f6687fff700 1 --2- 172.21.15.121:0/4286636455 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f666805a510 0x7f666805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:12.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.914+0000 7f6687fff700 1 --2- 172.21.15.121:0/4286636455 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f666805a510 0x7f666805c9d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f6678009ba0 tx=0x7f6678008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:12.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:12.918+0000 7f6684ff9700 1 -- 172.21.15.121:0/4286636455 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6670047390 con 0x7f6688066700 2023-11-15T03:09:13.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:13.110+0000 7f668d439700 1 -- 172.21.15.121:0/4286636455 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6688062cd0 con 0x7f6688066700 2023-11-15T03:09:13.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:13.110+0000 7f6684ff9700 1 -- 172.21.15.121:0/4286636455 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 276 v276) v1 ==== 95+0+4502 (secure 0 0 0) 0x7f667004a9b0 con 0x7f6688066700 2023-11-15T03:09:13.122 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:09:13.122 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":276,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":276,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:09:12.302660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9807":{"gid":9807,"name":"a","rank":0,"incarnation":276,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/142843299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":142843299},{"type":"v1","addr":"172.21.15.121:6836","nonce":142843299}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:09:13.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:13.114+0000 7f668d439700 1 -- 172.21.15.121:0/4286636455 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f666805a510 msgr2=0x7f666805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:13.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:13.114+0000 7f668d439700 1 --2- 172.21.15.121:0/4286636455 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f666805a510 0x7f666805c9d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f6678009ba0 tx=0x7f6678008040).stop 2023-11-15T03:09:13.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:13.114+0000 7f668d439700 1 -- 172.21.15.121:0/4286636455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6688066700 msgr2=0x7f66880ae0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:13.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:13.114+0000 7f668d439700 1 --2- 172.21.15.121:0/4286636455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6688066700 0x7f66880ae0a0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 rx=0x7f667000c4f0 tx=0x7f6670007dd0).stop 2023-11-15T03:09:13.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:13.114+0000 7f668d439700 1 -- 172.21.15.121:0/4286636455 shutdown_connections 2023-11-15T03:09:13.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:13.118+0000 7f668d439700 1 --2- 172.21.15.121:0/4286636455 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f666805a510 0x7f666805c9d0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:13.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:13.118+0000 7f668d439700 1 --2- 172.21.15.121:0/4286636455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66880acf10 0x7f6688150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:13.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:13.118+0000 7f668d439700 1 --2- 172.21.15.121:0/4286636455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6688066700 0x7f66880ae0a0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:13.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:13.118+0000 7f668d439700 1 --2- 172.21.15.121:0/4286636455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6688066320 0x7f668814a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:13.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:13.118+0000 7f668d439700 1 -- 172.21.15.121:0/4286636455 >> 172.21.15.121:0/4286636455 conn(0x7f6688067a60 msgr2=0x7f66880ab3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:13.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:13.118+0000 7f668d439700 1 -- 172.21.15.121:0/4286636455 shutdown_connections 2023-11-15T03:09:13.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:13.118+0000 7f668d439700 1 -- 172.21.15.121:0/4286636455 wait complete. 2023-11-15T03:09:13.129 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 276 2023-11-15T03:09:13.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 276, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:09:12.302660+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 125, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9807}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9807': {'gid': 9807, 'name': 'a', 'rank': 0, 'incarnation': 276, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.121:6836/142843299', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 142843299}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 142843299}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:09:13.148 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9807:up:replay 2023-11-15T03:09:13.148 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:09:18.553 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:09:18.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.706+0000 7f9f63479700 1 Processor -- start 2023-11-15T03:09:18.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.710+0000 7f9f63479700 1 -- start start 2023-11-15T03:09:18.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.714+0000 7f9f63479700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9f5c0c3280 0x7f9f5c0c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:18.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.714+0000 7f9f63479700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9f5c152140 0x7f9f5c1525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:18.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.714+0000 7f9f63479700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5c0c1d50 con 0x7f9f5c15be30 2023-11-15T03:09:18.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.714+0000 7f9f63479700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f5c0c1860 con 0x7f9f5c152140 2023-11-15T03:09:18.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.714+0000 7f9f63479700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f5c15c990 con 0x7f9f5c0c3280 2023-11-15T03:09:18.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.714+0000 7f9f61c76700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9f5c152140 0x7f9f5c1525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:18.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.714+0000 7f9f62c78700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9f5c15be30 0x7f9f5c15c280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40164/0 (socket says 172.21.15.121:40164) 2023-11-15T03:09:18.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.714+0000 7f9f62c78700 1 -- 172.21.15.121:0/3597311335 learned_addr learned my addr 172.21.15.121:0/3597311335 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:09:18.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.714+0000 7f9f62477700 1 --2- 172.21.15.121:0/3597311335 >> v2:172.21.15.190:3300/0 conn(0x7f9f5c0c3280 0x7f9f5c0c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:18.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.714+0000 7f9f61c76700 1 -- 172.21.15.121:0/3597311335 >> v2:172.21.15.190:3300/0 conn(0x7f9f5c0c3280 msgr2=0x7f9f5c0c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:18.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.714+0000 7f9f61475700 1 -- 172.21.15.121:0/3597311335 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268174864 0 0) 0x7f9f5c0c1d50 con 0x7f9f5c15be30 2023-11-15T03:09:18.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.714+0000 7f9f61c76700 1 --2- 172.21.15.121:0/3597311335 >> v2:172.21.15.190:3300/0 conn(0x7f9f5c0c3280 0x7f9f5c0c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:18.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.714+0000 7f9f61c76700 1 -- 172.21.15.121:0/3597311335 >> v1:172.21.15.121:6789/0 conn(0x7f9f5c15be30 legacy=0x7f9f5c15c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:18.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.714+0000 7f9f61c76700 1 -- 172.21.15.121:0/3597311335 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5c15cb80 con 0x7f9f5c152140 2023-11-15T03:09:18.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.714+0000 7f9f61c76700 1 --2- 172.21.15.121:0/3597311335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f5c152140 0x7f9f5c1525f0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 rx=0x7f9f5c0c7720 tx=0x7f9f4c009060).ready entity=mon.0 client_cookie=5c7608b320950c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:18.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.714+0000 7f9f61475700 1 -- 172.21.15.121:0/3597311335 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c00f510 con 0x7f9f5c152140 2023-11-15T03:09:18.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.714+0000 7f9f61475700 1 -- 172.21.15.121:0/3597311335 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c00f670 con 0x7f9f5c152140 2023-11-15T03:09:18.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.714+0000 7f9f61475700 1 -- 172.21.15.121:0/3597311335 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c00f9f0 con 0x7f9f5c152140 2023-11-15T03:09:18.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.718+0000 7f9f63479700 1 -- 172.21.15.121:0/3597311335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f5c152140 msgr2=0x7f9f5c1525f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:18.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.718+0000 7f9f63479700 1 --2- 172.21.15.121:0/3597311335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f5c152140 0x7f9f5c1525f0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 rx=0x7f9f5c0c7720 tx=0x7f9f4c009060).stop 2023-11-15T03:09:18.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.718+0000 7f9f63479700 1 -- 172.21.15.121:0/3597311335 shutdown_connections 2023-11-15T03:09:18.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.718+0000 7f9f63479700 1 --2- 172.21.15.121:0/3597311335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f5c152140 0x7f9f5c1525f0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:18.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.718+0000 7f9f63479700 1 --2- 172.21.15.121:0/3597311335 >> v2:172.21.15.190:3300/0 conn(0x7f9f5c0c3280 0x7f9f5c0c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:18.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.718+0000 7f9f63479700 1 -- 172.21.15.121:0/3597311335 >> 172.21.15.121:0/3597311335 conn(0x7f9f5c062190 msgr2=0x7f9f5c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:18.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.718+0000 7f9f63479700 1 -- 172.21.15.121:0/3597311335 shutdown_connections 2023-11-15T03:09:18.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.718+0000 7f9f63479700 1 -- 172.21.15.121:0/3597311335 wait complete. 2023-11-15T03:09:18.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.718+0000 7f9f63479700 1 Processor -- start 2023-11-15T03:09:18.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.718+0000 7f9f63479700 1 -- start start 2023-11-15T03:09:18.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f63479700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f5c0c3280 0x7f9f5c16b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:18.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f63479700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f5c152140 0x7f9f5c16bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:18.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f63479700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f5c15be30 0x7f9f5c13bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:18.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f63479700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c16c390 con 0x7f9f5c15be30 2023-11-15T03:09:18.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f63479700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c13d420 con 0x7f9f5c152140 2023-11-15T03:09:18.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f63479700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c13c810 con 0x7f9f5c0c3280 2023-11-15T03:09:18.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f62c78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f5c15be30 0x7f9f5c13bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:18.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f62c78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f5c15be30 0x7f9f5c13bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40516/0 (socket says 172.21.15.121:40516) 2023-11-15T03:09:18.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f62c78700 1 -- 172.21.15.121:0/2210368103 learned_addr learned my addr 172.21.15.121:0/2210368103 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:18.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f62477700 1 --2- 172.21.15.121:0/2210368103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f5c0c3280 0x7f9f5c16b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:18.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f61c76700 1 --2- 172.21.15.121:0/2210368103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f5c152140 0x7f9f5c16bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:18.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f62c78700 1 -- 172.21.15.121:0/2210368103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f5c0c3280 msgr2=0x7f9f5c16b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:18.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f62c78700 1 --2- 172.21.15.121:0/2210368103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f5c0c3280 0x7f9f5c16b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:18.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f62c78700 1 -- 172.21.15.121:0/2210368103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f5c152140 msgr2=0x7f9f5c16bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:18.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f62c78700 1 --2- 172.21.15.121:0/2210368103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f5c152140 0x7f9f5c16bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:18.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f62c78700 1 -- 172.21.15.121:0/2210368103 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4c0092a0 con 0x7f9f5c15be30 2023-11-15T03:09:18.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f62477700 1 --2- 172.21.15.121:0/2210368103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f5c0c3280 0x7f9f5c16b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:09:18.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f62c78700 1 --2- 172.21.15.121:0/2210368103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f5c15be30 0x7f9f5c13bae0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 rx=0x7f9f5400c7e0 tx=0x7f9f54007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:18.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f527fc700 1 -- 172.21.15.121:0/2210368103 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54005d30 con 0x7f9f5c15be30 2023-11-15T03:09:18.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f527fc700 1 -- 172.21.15.121:0/2210368103 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f54005e90 con 0x7f9f5c15be30 2023-11-15T03:09:18.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f63479700 1 -- 172.21.15.121:0/2210368103 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5c13caf0 con 0x7f9f5c15be30 2023-11-15T03:09:18.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.722+0000 7f9f527fc700 1 -- 172.21.15.121:0/2210368103 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54007320 con 0x7f9f5c15be30 2023-11-15T03:09:18.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.726+0000 7f9f63479700 1 -- 172.21.15.121:0/2210368103 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5c13d0b0 con 0x7f9f5c15be30 2023-11-15T03:09:18.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.726+0000 7f9f63479700 1 -- 172.21.15.121:0/2210368103 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5c06bcf0 con 0x7f9f5c15be30 2023-11-15T03:09:18.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.726+0000 7f9f527fc700 1 -- 172.21.15.121:0/2210368103 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9f54007710 con 0x7f9f5c15be30 2023-11-15T03:09:18.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.730+0000 7f9f527fc700 1 --2- 172.21.15.121:0/2210368103 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9f4005a510 0x7f9f4005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:18.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.730+0000 7f9f527fc700 1 -- 172.21.15.121:0/2210368103 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7f9f54075410 con 0x7f9f5c15be30 2023-11-15T03:09:18.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.730+0000 7f9f62477700 1 --2- 172.21.15.121:0/2210368103 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9f4005a510 0x7f9f4005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:18.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.730+0000 7f9f62477700 1 --2- 172.21.15.121:0/2210368103 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9f4005a510 0x7f9f4005c9d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f9f58009b60 tx=0x7f9f58008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:18.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.734+0000 7f9f527fc700 1 -- 172.21.15.121:0/2210368103 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9f54048030 con 0x7f9f5c15be30 2023-11-15T03:09:18.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.930+0000 7f9f63479700 1 -- 172.21.15.121:0/2210368103 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f5c068a70 con 0x7f9f5c15be30 2023-11-15T03:09:18.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.930+0000 7f9f527fc700 1 -- 172.21.15.121:0/2210368103 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) v1 ==== 95+0+4502 (secure 0 0 0) 0x7f9f5404b650 con 0x7f9f5c15be30 2023-11-15T03:09:18.941 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:09:18.942 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":279,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:09:15.325719+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9807":{"gid":9807,"name":"a","rank":0,"incarnation":276,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6836/142843299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":142843299},{"type":"v1","addr":"172.21.15.121:6836","nonce":142843299}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:09:18.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.934+0000 7f9f63479700 1 -- 172.21.15.121:0/2210368103 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9f4005a510 msgr2=0x7f9f4005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:18.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.934+0000 7f9f63479700 1 --2- 172.21.15.121:0/2210368103 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9f4005a510 0x7f9f4005c9d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f9f58009b60 tx=0x7f9f58008040).stop 2023-11-15T03:09:18.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.934+0000 7f9f63479700 1 -- 172.21.15.121:0/2210368103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f5c15be30 msgr2=0x7f9f5c13bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:18.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.934+0000 7f9f63479700 1 --2- 172.21.15.121:0/2210368103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f5c15be30 0x7f9f5c13bae0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 rx=0x7f9f5400c7e0 tx=0x7f9f54007f90).stop 2023-11-15T03:09:18.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.934+0000 7f9f63479700 1 -- 172.21.15.121:0/2210368103 shutdown_connections 2023-11-15T03:09:18.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.934+0000 7f9f63479700 1 --2- 172.21.15.121:0/2210368103 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9f4005a510 0x7f9f4005c9d0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:18.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.934+0000 7f9f63479700 1 --2- 172.21.15.121:0/2210368103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f5c15be30 0x7f9f5c13bae0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:18.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.934+0000 7f9f63479700 1 --2- 172.21.15.121:0/2210368103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f5c152140 0x7f9f5c16bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:18.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.934+0000 7f9f63479700 1 --2- 172.21.15.121:0/2210368103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f5c0c3280 0x7f9f5c16b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:18.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.934+0000 7f9f63479700 1 -- 172.21.15.121:0/2210368103 >> 172.21.15.121:0/2210368103 conn(0x7f9f5c062190 msgr2=0x7f9f5c0aa710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:18.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.934+0000 7f9f63479700 1 -- 172.21.15.121:0/2210368103 shutdown_connections 2023-11-15T03:09:18.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:18.934+0000 7f9f63479700 1 -- 172.21.15.121:0/2210368103 wait complete. 2023-11-15T03:09:18.947 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 279 2023-11-15T03:09:18.966 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:09:15.325719+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 125, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9807}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9807': {'gid': 9807, 'name': 'a', 'rank': 0, 'incarnation': 276, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.121:6836/142843299', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 142843299}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 142843299}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:09:18.967 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T03:09:18.969 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-15T03:09:19.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.126+0000 7fa2ad747700 1 Processor -- start 2023-11-15T03:09:19.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.126+0000 7fa2ad747700 1 -- start start 2023-11-15T03:09:19.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.126+0000 7fa2ad747700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa2a8153990 0x7fa2a8158100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.126+0000 7fa2ad747700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa2a8153d70 0x7fa2a80be090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.126+0000 7fa2ad747700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a81586c0 con 0x7fa2a8067a60 2023-11-15T03:09:19.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.126+0000 7fa2ad747700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a80a6850 con 0x7fa2a8153990 2023-11-15T03:09:19.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.126+0000 7fa2ad747700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a80a6b30 con 0x7fa2a8153d70 2023-11-15T03:09:19.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2a7fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa2a8153990 0x7fa2a8158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2acf46700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa2a8067a60 0x7fa2a80c4070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40170/0 (socket says 172.21.15.121:40170) 2023-11-15T03:09:19.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2a7fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa2a8153990 0x7fa2a8158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40526/0 (socket says 172.21.15.121:40526) 2023-11-15T03:09:19.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2acf46700 1 -- 172.21.15.121:0/803110090 learned_addr learned my addr 172.21.15.121:0/803110090 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:09:19.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2a77fe700 1 --2- 172.21.15.121:0/803110090 >> v2:172.21.15.190:3300/0 conn(0x7fa2a8153d70 0x7fa2a80be090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2a7fff700 1 -- 172.21.15.121:0/803110090 >> v2:172.21.15.190:3300/0 conn(0x7fa2a8153d70 msgr2=0x7fa2a80be090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:19.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2a7fff700 1 --2- 172.21.15.121:0/803110090 >> v2:172.21.15.190:3300/0 conn(0x7fa2a8153d70 0x7fa2a80be090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2a7fff700 1 -- 172.21.15.121:0/803110090 >> v1:172.21.15.121:6789/0 conn(0x7fa2a8067a60 legacy=0x7fa2a80c4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:19.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2a7fff700 1 -- 172.21.15.121:0/803110090 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a80be580 con 0x7fa2a8153990 2023-11-15T03:09:19.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2a7fff700 1 --2- 172.21.15.121:0/803110090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2a8153990 0x7fa2a8158100 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 rx=0x7fa2a8065f90 tx=0x7fa2a0009060).ready entity=mon.0 client_cookie=71f7815f98c10140 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:19.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2a6ffd700 1 -- 172.21.15.121:0/803110090 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a00090d0 con 0x7fa2a8153990 2023-11-15T03:09:19.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2a6ffd700 1 -- 172.21.15.121:0/803110090 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a000b580 con 0x7fa2a8153990 2023-11-15T03:09:19.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2a6ffd700 1 -- 172.21.15.121:0/803110090 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a000b900 con 0x7fa2a8153990 2023-11-15T03:09:19.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2ad747700 1 -- 172.21.15.121:0/803110090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2a8153990 msgr2=0x7fa2a8158100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:19.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2ad747700 1 --2- 172.21.15.121:0/803110090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2a8153990 0x7fa2a8158100 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 rx=0x7fa2a8065f90 tx=0x7fa2a0009060).stop 2023-11-15T03:09:19.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2ad747700 1 -- 172.21.15.121:0/803110090 shutdown_connections 2023-11-15T03:09:19.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2ad747700 1 --2- 172.21.15.121:0/803110090 >> v2:172.21.15.190:3300/0 conn(0x7fa2a8153d70 0x7fa2a80be090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2ad747700 1 --2- 172.21.15.121:0/803110090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2a8153990 0x7fa2a8158100 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2ad747700 1 -- 172.21.15.121:0/803110090 >> 172.21.15.121:0/803110090 conn(0x7fa2a8069640 msgr2=0x7fa2a8066860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:19.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2ad747700 1 -- 172.21.15.121:0/803110090 shutdown_connections 2023-11-15T03:09:19.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.130+0000 7fa2ad747700 1 -- 172.21.15.121:0/803110090 wait complete. 2023-11-15T03:09:19.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.134+0000 7fa2ad747700 1 Processor -- start 2023-11-15T03:09:19.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.134+0000 7fa2ad747700 1 -- start start 2023-11-15T03:09:19.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.134+0000 7fa2ad747700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2a8067a60 0x7fa2a8154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.134+0000 7fa2ad747700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2a8153990 0x7fa2a81568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.134+0000 7fa2ad747700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2a8153d70 0x7fa2a80ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.134+0000 7fa2ad747700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a8164120 con 0x7fa2a8067a60 2023-11-15T03:09:19.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.134+0000 7fa2ad747700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a8157320 con 0x7fa2a8153990 2023-11-15T03:09:19.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.134+0000 7fa2ad747700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a81643e0 con 0x7fa2a8153d70 2023-11-15T03:09:19.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.134+0000 7fa2a7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2a8067a60 0x7fa2a8154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.134+0000 7fa2a7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2a8067a60 0x7fa2a8154d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40536/0 (socket says 172.21.15.121:40536) 2023-11-15T03:09:19.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.134+0000 7fa2a7fff700 1 -- 172.21.15.121:0/4011375290 learned_addr learned my addr 172.21.15.121:0/4011375290 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:19.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.134+0000 7fa2acf46700 1 --2- 172.21.15.121:0/4011375290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2a8153d70 0x7fa2a80ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.134+0000 7fa2a77fe700 1 --2- 172.21.15.121:0/4011375290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2a8153990 0x7fa2a81568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.138+0000 7fa2a7fff700 1 -- 172.21.15.121:0/4011375290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2a8153d70 msgr2=0x7fa2a80ab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:19.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.138+0000 7fa2a7fff700 1 --2- 172.21.15.121:0/4011375290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2a8153d70 0x7fa2a80ab5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.138+0000 7fa2a7fff700 1 -- 172.21.15.121:0/4011375290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2a8153990 msgr2=0x7fa2a81568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:19.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.138+0000 7fa2a7fff700 1 --2- 172.21.15.121:0/4011375290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2a8153990 0x7fa2a81568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.138+0000 7fa2a7fff700 1 -- 172.21.15.121:0/4011375290 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a00092a0 con 0x7fa2a8067a60 2023-11-15T03:09:19.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.138+0000 7fa2a7fff700 1 --2- 172.21.15.121:0/4011375290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2a8067a60 0x7fa2a8154d20 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 rx=0x7fa2a806c350 tx=0x7fa2a0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:19.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.138+0000 7fa29ffff700 1 -- 172.21.15.121:0/4011375290 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0006970 con 0x7fa2a8067a60 2023-11-15T03:09:19.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.138+0000 7fa29ffff700 1 -- 172.21.15.121:0/4011375290 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a0000d10 con 0x7fa2a8067a60 2023-11-15T03:09:19.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.138+0000 7fa2ad747700 1 -- 172.21.15.121:0/4011375290 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a81646c0 con 0x7fa2a8067a60 2023-11-15T03:09:19.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.138+0000 7fa29ffff700 1 -- 172.21.15.121:0/4011375290 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a000b940 con 0x7fa2a8067a60 2023-11-15T03:09:19.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.138+0000 7fa2ad747700 1 -- 172.21.15.121:0/4011375290 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a8164ea0 con 0x7fa2a8067a60 2023-11-15T03:09:19.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.142+0000 7fa29ffff700 1 -- 172.21.15.121:0/4011375290 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa2a001b620 con 0x7fa2a8067a60 2023-11-15T03:09:19.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.142+0000 7fa29ffff700 1 --2- 172.21.15.121:0/4011375290 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa28805a510 0x7fa28805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.142+0000 7fa29ffff700 1 -- 172.21.15.121:0/4011375290 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7fa2a0076c70 con 0x7fa2a8067a60 2023-11-15T03:09:19.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.142+0000 7fa2a77fe700 1 --2- 172.21.15.121:0/4011375290 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa28805a510 0x7fa28805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.142+0000 7fa2a77fe700 1 --2- 172.21.15.121:0/4011375290 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa28805a510 0x7fa28805c9d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7fa2a80a49f0 tx=0x7fa2980097a0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:19.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.142+0000 7fa2b8d5e740 1 Processor -- start 2023-11-15T03:09:19.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.142+0000 7fa2b8d5e740 1 -- start start 2023-11-15T03:09:19.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa2b8d5e740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1ad0 0x1ef8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa2b8d5e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1eb0 0x1f1e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa2b8d5e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f1e630 0x1f1eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa2b8d5e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e52410 con 0x1ed1eb0 2023-11-15T03:09:19.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa2b8d5e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e52550 con 0x1f1e630 2023-11-15T03:09:19.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa2b8d5e740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e52a00 con 0x1ed1ad0 2023-11-15T03:09:19.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa2a77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1eb0 0x1f1e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa2a77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1eb0 0x1f1e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40542/0 (socket says 172.21.15.121:40542) 2023-11-15T03:09:19.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa2a77fe700 1 -- 172.21.15.121:0/1807414671 learned_addr learned my addr 172.21.15.121:0/1807414671 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:19.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa2a7fff700 1 --2- 172.21.15.121:0/1807414671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1ad0 0x1ef8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa2acf46700 1 --2- 172.21.15.121:0/1807414671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f1e630 0x1f1eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa2a77fe700 1 -- 172.21.15.121:0/1807414671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1ad0 msgr2=0x1ef8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:19.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa2a77fe700 1 --2- 172.21.15.121:0/1807414671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1ad0 0x1ef8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa2a77fe700 1 -- 172.21.15.121:0/1807414671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f1e630 msgr2=0x1f1eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:19.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa2a77fe700 1 --2- 172.21.15.121:0/1807414671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f1e630 0x1f1eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa2a77fe700 1 -- 172.21.15.121:0/1807414671 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1e4b730 con 0x1ed1eb0 2023-11-15T03:09:19.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa2a77fe700 1 --2- 172.21.15.121:0/1807414671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1eb0 0x1f1e140 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 rx=0x7fa298014780 tx=0x7fa298014000).ready entity=mon.0 client_cookie=d3b91776ea861710 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:19.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa29d7fa700 1 -- 172.21.15.121:0/1807414671 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2980136d0 con 0x1ed1eb0 2023-11-15T03:09:19.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa29d7fa700 1 -- 172.21.15.121:0/1807414671 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa298018070 con 0x1ed1eb0 2023-11-15T03:09:19.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa29d7fa700 1 -- 172.21.15.121:0/1807414671 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298013830 con 0x1ed1eb0 2023-11-15T03:09:19.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/1807414671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1eb0 msgr2=0x1f1e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:19.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.146+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/1807414671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1eb0 0x1f1e140 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 rx=0x7fa298014780 tx=0x7fa298014000).stop 2023-11-15T03:09:19.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/1807414671 shutdown_connections 2023-11-15T03:09:19.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/1807414671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f1e630 0x1f1eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/1807414671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1eb0 0x1f1e140 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/1807414671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1ad0 0x1ef8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/1807414671 >> 172.21.15.121:0/1807414671 conn(0x1cb0550 msgr2=0x1ed07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:19.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/1807414671 shutdown_connections 2023-11-15T03:09:19.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/1807414671 wait complete. 2023-11-15T03:09:19.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2b8d5e740 1 -- start start 2023-11-15T03:09:19.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2b8d5e740 1 Processor -- start 2023-11-15T03:09:19.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2b8d5e740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1ad0 0x1f1c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2b8d5e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1eb0 0x1ef9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2b8d5e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f1e630 0x1f1d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2b8d5e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e1d650 con 0x1ed1eb0 2023-11-15T03:09:19.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2b8d5e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ef7600 con 0x1f1e630 2023-11-15T03:09:19.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2b8d5e740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1efd270 con 0x1ed1ad0 2023-11-15T03:09:19.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2b8d5e740 10 client.0 ll_register_callbacks cb 0x1e8a020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:09:19.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2a77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1eb0 0x1ef9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2a77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1eb0 0x1ef9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40554/0 (socket says 172.21.15.121:40554) 2023-11-15T03:09:19.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2a77fe700 1 -- 172.21.15.121:0/1135144853 learned_addr learned my addr 172.21.15.121:0/1135144853 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:19.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2acf46700 1 --2- 172.21.15.121:0/1135144853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f1e630 0x1f1d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2a7fff700 1 --2- 172.21.15.121:0/1135144853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1ad0 0x1f1c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2a77fe700 1 -- 172.21.15.121:0/1135144853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1ad0 msgr2=0x1f1c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:19.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2a77fe700 1 --2- 172.21.15.121:0/1135144853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1ad0 0x1f1c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2a77fe700 1 -- 172.21.15.121:0/1135144853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f1e630 msgr2=0x1f1d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:19.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2a77fe700 1 --2- 172.21.15.121:0/1135144853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f1e630 0x1f1d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.150+0000 7fa2a77fe700 1 -- 172.21.15.121:0/1135144853 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa298014430 con 0x1ed1eb0 2023-11-15T03:09:19.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.154+0000 7fa2a77fe700 1 --2- 172.21.15.121:0/1135144853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1eb0 0x1ef9020 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 rx=0x7fa298014ed0 tx=0x7fa29801d000).ready entity=mon.0 client_cookie=c85f605fb9e61820 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:19.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.154+0000 7fa2867fc700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:09:19.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.154+0000 7fa2867fc700 1 -- 172.21.15.121:0/1135144853 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298024030 con 0x1ed1eb0 2023-11-15T03:09:19.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.154+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/1135144853 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1e481a0 con 0x1ed1eb0 2023-11-15T03:09:19.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.154+0000 7fa2867fc700 1 -- 172.21.15.121:0/1135144853 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7fa298020660 con 0x1ed1eb0 2023-11-15T03:09:19.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.154+0000 7fa2867fc700 1 -- 172.21.15.121:0/1135144853 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=279) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa298009960 con 0x1ed1eb0 2023-11-15T03:09:19.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.154+0000 7fa2b8d5e740 10 client.9888 fetch_fsmap learned FSMap version 279 2023-11-15T03:09:19.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.154+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/1135144853 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=279,monmap=2+}) v3 -- 0x1e46960 con 0x1ed1eb0 2023-11-15T03:09:19.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.154+0000 7fa2867fc700 1 -- 172.21.15.121:0/1135144853 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1e481a0 con 0x1ed1eb0 2023-11-15T03:09:19.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.154+0000 7fa2867fc700 1 -- 172.21.15.121:0/1135144853 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 279) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fa298020a30 con 0x1ed1eb0 2023-11-15T03:09:19.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.154+0000 7fa2b8d5e740 10 client.9888 fetch_fsmap finished waiting for FSMap version 279 2023-11-15T03:09:19.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.154+0000 7fa2b8d5e740 10 client.9888 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:09:19.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.154+0000 7fa2b8d5e740 20 client.9888 populate_metadata read hostname 'smithi121' 2023-11-15T03:09:19.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.154+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/1135144853 >> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] conn(0x1f1a440 0x1efe560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.154+0000 7fa2b8d5e740 4 client.9888 mds_command: new command op to 9807 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:09:19.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.154+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/1135144853 --> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1e3e6f0 con 0x1f1a440 2023-11-15T03:09:19.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.154+0000 7fa2a7fff700 1 --2- 172.21.15.121:0/1135144853 >> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] conn(0x1f1a440 0x1efe560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.154+0000 7fa2867fc700 10 client.9888 ms_handle_connect on v2:172.21.15.121:6834/142843299 2023-11-15T03:09:19.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.154+0000 7fa2a7fff700 1 --2- 172.21.15.121:0/1135144853 >> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] conn(0x1f1a440 0x1efe560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=2be7e01dd8b6d302 server_cookie=a35bb6a926fdfc9b in_seq=0 out_seq=0 2023-11-15T03:09:19.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.158+0000 7fa2867fc700 1 -- 172.21.15.121:0/1135144853 <== mds.0 v2:172.21.15.121:6834/142843299 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x1e3e6f0 con 0x1f1a440 2023-11-15T03:09:19.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.158+0000 7fa2867fc700 10 client.9888 handle_command_reply: tid=0 2023-11-15T03:09:19.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.158+0000 7fa2b8d5e740 1 client.9888 shutdown 2023-11-15T03:09:19.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.158+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/1135144853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1eb0 msgr2=0x1ef9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:19.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.158+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/1135144853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1eb0 0x1ef9020 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 rx=0x7fa298014ed0 tx=0x7fa29801d000).stop 2023-11-15T03:09:19.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.158+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/1135144853 shutdown_connections 2023-11-15T03:09:19.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.158+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/1135144853 >> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] conn(0x1f1a440 0x1efe560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.158+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/1135144853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f1e630 0x1f1d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.158+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/1135144853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1eb0 0x1ef9020 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.158+0000 7fa2867fc700 0 client.9888 ms_handle_reset on v2:172.21.15.121:6834/142843299 2023-11-15T03:09:19.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.158+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/1135144853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1ad0 0x1f1c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.158+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/1135144853 >> 172.21.15.121:0/1135144853 conn(0x1cb0550 msgr2=0x1f12080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:19.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.158+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/1135144853 shutdown_connections 2023-11-15T03:09:19.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.158+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/1135144853 wait complete. 2023-11-15T03:09:19.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.158+0000 7fa2b8d5e740 20 client.9888 trim_cache size 0 max 16384 2023-11-15T03:09:19.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.178+0000 7fa2b8d5e740 1 Processor -- start 2023-11-15T03:09:19.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.178+0000 7fa2b8d5e740 1 -- start start 2023-11-15T03:09:19.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.178+0000 7fa2b8d5e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1ad0 0x1f6e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.178+0000 7fa2b8d5e740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1eb0 0x1ef6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.178+0000 7fa2b8d5e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f1e630 0x1ecb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.178+0000 7fa2b8d5e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e27c60 con 0x1ed1ad0 2023-11-15T03:09:19.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.178+0000 7fa2b8d5e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e481a0 con 0x1f1e630 2023-11-15T03:09:19.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.178+0000 7fa2b8d5e740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1efe060 con 0x1ed1eb0 2023-11-15T03:09:19.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.178+0000 7fa2a7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1ad0 0x1f6e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.178+0000 7fa2a7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1ad0 0x1f6e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40560/0 (socket says 172.21.15.121:40560) 2023-11-15T03:09:19.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.178+0000 7fa2a7fff700 1 -- 172.21.15.121:0/1837100752 learned_addr learned my addr 172.21.15.121:0/1837100752 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:19.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.178+0000 7fa2acf46700 1 --2- 172.21.15.121:0/1837100752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f1e630 0x1ecb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.178+0000 7fa2a77fe700 1 --2- 172.21.15.121:0/1837100752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1eb0 0x1ef6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa2a7fff700 1 -- 172.21.15.121:0/1837100752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1eb0 msgr2=0x1ef6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:19.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa2a7fff700 1 --2- 172.21.15.121:0/1837100752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1eb0 0x1ef6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa2a7fff700 1 -- 172.21.15.121:0/1837100752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f1e630 msgr2=0x1ecb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:19.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa2a7fff700 1 --2- 172.21.15.121:0/1837100752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f1e630 0x1ecb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa2a7fff700 1 -- 172.21.15.121:0/1837100752 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a80abe10 con 0x1ed1ad0 2023-11-15T03:09:19.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa2a77fe700 1 --2- 172.21.15.121:0/1837100752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1eb0 0x1ef6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:09:19.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa2a7fff700 1 --2- 172.21.15.121:0/1837100752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1ad0 0x1f6e6c0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 rx=0x7fa2a0049a20 tx=0x7fa2a0033280).ready entity=mon.0 client_cookie=305891274be1e0cf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:19.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa29dffb700 1 -- 172.21.15.121:0/1837100752 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a004b6b0 con 0x1ed1ad0 2023-11-15T03:09:19.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa29dffb700 1 -- 172.21.15.121:0/1837100752 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a004b910 con 0x1ed1ad0 2023-11-15T03:09:19.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa29dffb700 1 -- 172.21.15.121:0/1837100752 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0074280 con 0x1ed1ad0 2023-11-15T03:09:19.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/1837100752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1ad0 msgr2=0x1f6e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:19.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/1837100752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1ad0 0x1f6e6c0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 rx=0x7fa2a0049a20 tx=0x7fa2a0033280).stop 2023-11-15T03:09:19.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/1837100752 shutdown_connections 2023-11-15T03:09:19.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/1837100752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1f1e630 0x1ecb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/1837100752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1eb0 0x1ef6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/1837100752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed1ad0 0x1f6e6c0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/1837100752 >> 172.21.15.121:0/1837100752 conn(0x1cb0550 msgr2=0x1efe410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:19.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/1837100752 shutdown_connections 2023-11-15T03:09:19.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/1837100752 wait complete. 2023-11-15T03:09:19.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa2b8d5e740 1 -- start start 2023-11-15T03:09:19.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.182+0000 7fa2b8d5e740 1 Processor -- start 2023-11-15T03:09:19.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2b8d5e740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1ad0 0x1f1c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2b8d5e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ed1eb0 0x1f6d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2b8d5e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f1e630 0x1f6d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2b8d5e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f718a0 con 0x1f1e630 2023-11-15T03:09:19.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2b8d5e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f6daa0 con 0x1ed1eb0 2023-11-15T03:09:19.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2b8d5e740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f73030 con 0x1ed1ad0 2023-11-15T03:09:19.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2b8d5e740 10 client.0 ll_register_callbacks cb 0x7fa298009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:09:19.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2acf46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f1e630 0x1f6d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2acf46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f1e630 0x1f6d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40566/0 (socket says 172.21.15.121:40566) 2023-11-15T03:09:19.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2acf46700 1 -- 172.21.15.121:0/136675766 learned_addr learned my addr 172.21.15.121:0/136675766 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:19.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2a77fe700 1 --2- 172.21.15.121:0/136675766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ed1eb0 0x1f6d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2a7fff700 1 --2- 172.21.15.121:0/136675766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1ad0 0x1f1c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2acf46700 1 -- 172.21.15.121:0/136675766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1ad0 msgr2=0x1f1c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:19.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2acf46700 1 --2- 172.21.15.121:0/136675766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1ad0 0x1f1c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2acf46700 1 -- 172.21.15.121:0/136675766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ed1eb0 msgr2=0x1f6d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:19.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2acf46700 1 --2- 172.21.15.121:0/136675766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ed1eb0 0x1f6d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2acf46700 1 -- 172.21.15.121:0/136675766 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a8164ea0 con 0x1f1e630 2023-11-15T03:09:19.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2a77fe700 1 --2- 172.21.15.121:0/136675766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ed1eb0 0x1f6d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:09:19.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2acf46700 1 --2- 172.21.15.121:0/136675766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f1e630 0x1f6d5b0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 rx=0x7fa2900176e0 tx=0x7fa290009040).ready entity=mon.0 client_cookie=b1b3192403a573df server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:19.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2867fc700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:09:19.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2867fc700 1 -- 172.21.15.121:0/136675766 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa290006e50 con 0x1f1e630 2023-11-15T03:09:19.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/136675766 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa298018070 con 0x1f1e630 2023-11-15T03:09:19.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2867fc700 1 -- 172.21.15.121:0/136675766 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7fa2900060d0 con 0x1f1e630 2023-11-15T03:09:19.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2867fc700 1 -- 172.21.15.121:0/136675766 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=279) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2900064a0 con 0x1f1e630 2023-11-15T03:09:19.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2b8d5e740 10 client.9894 fetch_fsmap learned FSMap version 279 2023-11-15T03:09:19.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.186+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/136675766 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=279,monmap=2+}) v3 -- 0x1e466b0 con 0x1f1e630 2023-11-15T03:09:19.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2867fc700 1 -- 172.21.15.121:0/136675766 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa298018070 con 0x1f1e630 2023-11-15T03:09:19.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2867fc700 1 -- 172.21.15.121:0/136675766 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 279) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fa2900046b0 con 0x1f1e630 2023-11-15T03:09:19.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2b8d5e740 10 client.9894 fetch_fsmap finished waiting for FSMap version 279 2023-11-15T03:09:19.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2b8d5e740 10 client.9894 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:09:19.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2b8d5e740 20 client.9894 populate_metadata read hostname 'smithi121' 2023-11-15T03:09:19.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/136675766 >> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] conn(0x1ef8950 0x1ef8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2b8d5e740 4 client.9894 mds_command: new command op to 9807 tid=0[{"prefix": "status"}] 2023-11-15T03:09:19.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/136675766 --> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1d074e0 con 0x1ef8950 2023-11-15T03:09:19.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2a7fff700 1 --2- 172.21.15.121:0/136675766 >> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] conn(0x1ef8950 0x1ef8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2a7fff700 1 --2- 172.21.15.121:0/136675766 >> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] conn(0x1ef8950 0x1ef8d30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=eb8a9535006d0410 server_cookie=c8944758db624750 in_seq=0 out_seq=0 2023-11-15T03:09:19.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2867fc700 10 client.9894 ms_handle_connect on v2:172.21.15.121:6834/142843299 2023-11-15T03:09:19.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2867fc700 1 -- 172.21.15.121:0/136675766 <== mds.0 v2:172.21.15.121:6834/142843299 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x1d074e0 con 0x1ef8950 2023-11-15T03:09:19.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2867fc700 10 client.9894 handle_command_reply: tid=0 2023-11-15T03:09:19.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2b8d5e740 1 client.9894 shutdown 2023-11-15T03:09:19.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/136675766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f1e630 msgr2=0x1f6d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:19.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/136675766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f1e630 0x1f6d5b0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 rx=0x7fa2900176e0 tx=0x7fa290009040).stop 2023-11-15T03:09:19.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/136675766 shutdown_connections 2023-11-15T03:09:19.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/136675766 >> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] conn(0x1ef8950 0x1ef8d30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/136675766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1f1e630 0x1f6d5b0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/136675766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ed1eb0 0x1f6d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2b8d5e740 1 --2- 172.21.15.121:0/136675766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed1ad0 0x1f1c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2867fc700 0 client.9894 ms_handle_reset on v2:172.21.15.121:6834/142843299 2023-11-15T03:09:19.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/136675766 >> 172.21.15.121:0/136675766 conn(0x1cb0550 msgr2=0x1ecb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:19.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/136675766 shutdown_connections 2023-11-15T03:09:19.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2b8d5e740 1 -- 172.21.15.121:0/136675766 wait complete. 2023-11-15T03:09:19.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2b8d5e740 20 client.9894 trim_cache size 0 max 16384 2023-11-15T03:09:19.222 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:09:19.222 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T03:09:19.222 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T03:09:19.222 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 9807, 2023-11-15T03:09:19.222 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T03:09:19.223 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T03:09:19.223 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T03:09:19.223 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 6.8866760649999996, 2023-11-15T03:09:19.223 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 279, 2023-11-15T03:09:19.223 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 125, 2023-11-15T03:09:19.224 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 125, 2023-11-15T03:09:19.224 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 16.976745276999999 2023-11-15T03:09:19.224 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:09:19.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2ad747700 1 -- 172.21.15.121:0/4011375290 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa28805a510 msgr2=0x7fa28805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:19.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2ad747700 1 --2- 172.21.15.121:0/4011375290 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa28805a510 0x7fa28805c9d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7fa2a80a49f0 tx=0x7fa2980097a0).stop 2023-11-15T03:09:19.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2ad747700 1 -- 172.21.15.121:0/4011375290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2a8067a60 msgr2=0x7fa2a8154d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:19.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2ad747700 1 --2- 172.21.15.121:0/4011375290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2a8067a60 0x7fa2a8154d20 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 rx=0x7fa2a806c350 tx=0x7fa2a0017000).stop 2023-11-15T03:09:19.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2ad747700 1 -- 172.21.15.121:0/4011375290 shutdown_connections 2023-11-15T03:09:19.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2ad747700 1 --2- 172.21.15.121:0/4011375290 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa28805a510 0x7fa28805c9d0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2ad747700 1 --2- 172.21.15.121:0/4011375290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2a8153d70 0x7fa2a80ab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2ad747700 1 --2- 172.21.15.121:0/4011375290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2a8153990 0x7fa2a81568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2ad747700 1 --2- 172.21.15.121:0/4011375290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2a8067a60 0x7fa2a8154d20 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.190+0000 7fa2ad747700 1 -- 172.21.15.121:0/4011375290 >> 172.21.15.121:0/4011375290 conn(0x7fa2a8069640 msgr2=0x7fa2a8062bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:19.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.194+0000 7fa2ad747700 1 -- 172.21.15.121:0/4011375290 shutdown_connections 2023-11-15T03:09:19.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.194+0000 7fa2ad747700 1 -- 172.21.15.121:0/4011375290 wait complete. 2023-11-15T03:09:19.228 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:09:19.228 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap key 10000000000.00000000:sixmegs_head', proceeding to ls 2023-11-15T03:09:19.228 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T03:09:19.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.346+0000 7f04fb239700 1 Processor -- start 2023-11-15T03:09:19.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.346+0000 7f04fb239700 1 -- start start 2023-11-15T03:09:19.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.350+0000 7f04fb239700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f4153c50 0x7f04f40c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.350+0000 7f04fb239700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f04f4154030 0x7f04f40a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.350+0000 7f04fb239700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04f4064f50 con 0x7f04f40653e0 2023-11-15T03:09:19.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.350+0000 7f04fb239700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04f40a5e20 con 0x7f04f4154030 2023-11-15T03:09:19.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.350+0000 7f04fb239700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f4064340 con 0x7f04f4153c50 2023-11-15T03:09:19.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.350+0000 7f04faa38700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f04f40653e0 0x7f04f40bb510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:49562/0 (socket says 172.21.15.121:49562) 2023-11-15T03:09:19.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.350+0000 7f04faa38700 1 -- 172.21.15.121:0/24078248 learned_addr learned my addr 172.21.15.121:0/24078248 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:09:19.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.350+0000 7f04fa237700 1 --2- 172.21.15.121:0/24078248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f4153c50 0x7f04f40c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.350+0000 7f04f9a36700 1 --2- 172.21.15.121:0/24078248 >> v2:172.21.15.121:3300/0 conn(0x7f04f4154030 0x7f04f40a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.350+0000 7f04f9235700 1 -- 172.21.15.121:0/24078248 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861031441 0 0) 0x7f04f4064f50 con 0x7f04f40653e0 2023-11-15T03:09:19.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.350+0000 7f04fa237700 1 -- 172.21.15.121:0/24078248 >> v2:172.21.15.121:3300/0 conn(0x7f04f4154030 msgr2=0x7f04f40a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:19.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.350+0000 7f04fa237700 1 --2- 172.21.15.121:0/24078248 >> v2:172.21.15.121:3300/0 conn(0x7f04f4154030 0x7f04f40a48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.350+0000 7f04fa237700 1 -- 172.21.15.121:0/24078248 >> v1:172.21.15.190:6789/0 conn(0x7f04f40653e0 legacy=0x7f04f40bb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:19.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.354+0000 7f04fa237700 1 -- 172.21.15.121:0/24078248 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f4064620 con 0x7f04f4153c50 2023-11-15T03:09:19.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.354+0000 7f04fa237700 1 --2- 172.21.15.121:0/24078248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f4153c50 0x7f04f40c1870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f04f0009760 tx=0x7f04f0009060).ready entity=mon.2 client_cookie=efbbe8f074882f10 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:19.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.354+0000 7f04f9235700 1 -- 172.21.15.121:0/24078248 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f000f510 con 0x7f04f4153c50 2023-11-15T03:09:19.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.354+0000 7f04f9235700 1 -- 172.21.15.121:0/24078248 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f000f670 con 0x7f04f4153c50 2023-11-15T03:09:19.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.354+0000 7f04f9235700 1 -- 172.21.15.121:0/24078248 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f000f9f0 con 0x7f04f4153c50 2023-11-15T03:09:19.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.354+0000 7f04fb239700 1 -- 172.21.15.121:0/24078248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f4153c50 msgr2=0x7f04f40c1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:19.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.354+0000 7f04fb239700 1 --2- 172.21.15.121:0/24078248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f4153c50 0x7f04f40c1870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f04f0009760 tx=0x7f04f0009060).stop 2023-11-15T03:09:19.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.354+0000 7f04fb239700 1 -- 172.21.15.121:0/24078248 shutdown_connections 2023-11-15T03:09:19.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.354+0000 7f04fb239700 1 --2- 172.21.15.121:0/24078248 >> v2:172.21.15.121:3300/0 conn(0x7f04f4154030 0x7f04f40a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.354+0000 7f04fb239700 1 --2- 172.21.15.121:0/24078248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f4153c50 0x7f04f40c1870 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.354+0000 7f04fb239700 1 -- 172.21.15.121:0/24078248 >> 172.21.15.121:0/24078248 conn(0x7f04f40677c0 msgr2=0x7f04f40c65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:19.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.354+0000 7f04fb239700 1 -- 172.21.15.121:0/24078248 shutdown_connections 2023-11-15T03:09:19.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.354+0000 7f04fb239700 1 -- 172.21.15.121:0/24078248 wait complete. 2023-11-15T03:09:19.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.358+0000 7f04fb239700 1 Processor -- start 2023-11-15T03:09:19.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.358+0000 7f04fb239700 1 -- start start 2023-11-15T03:09:19.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.358+0000 7f04fb239700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f40653e0 0x7f04f40a6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.358+0000 7f04fb239700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f4153c50 0x7f04f414c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.358+0000 7f04fb239700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f4154030 0x7f04f40bfce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.358+0000 7f04fb239700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f4168510 con 0x7f04f4153c50 2023-11-15T03:09:19.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.358+0000 7f04fb239700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f4167900 con 0x7f04f40653e0 2023-11-15T03:09:19.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.358+0000 7f04fb239700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f4167be0 con 0x7f04f4154030 2023-11-15T03:09:19.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.362+0000 7f04f9a36700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f4153c50 0x7f04f414c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.362+0000 7f04f9a36700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f4153c50 0x7f04f414c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40592/0 (socket says 172.21.15.121:40592) 2023-11-15T03:09:19.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.362+0000 7f04f9a36700 1 -- 172.21.15.121:0/1679071708 learned_addr learned my addr 172.21.15.121:0/1679071708 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:19.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.362+0000 7f04fa237700 1 --2- 172.21.15.121:0/1679071708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f40653e0 0x7f04f40a6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.362+0000 7f04faa38700 1 --2- 172.21.15.121:0/1679071708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f4154030 0x7f04f40bfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.362+0000 7f04f9a36700 1 -- 172.21.15.121:0/1679071708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f4154030 msgr2=0x7f04f40bfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:19.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.362+0000 7f04f9a36700 1 --2- 172.21.15.121:0/1679071708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f4154030 0x7f04f40bfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.362+0000 7f04f9a36700 1 -- 172.21.15.121:0/1679071708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f40653e0 msgr2=0x7f04f40a6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:19.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.362+0000 7f04f9a36700 1 --2- 172.21.15.121:0/1679071708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f40653e0 0x7f04f40a6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.362+0000 7f04f9a36700 1 -- 172.21.15.121:0/1679071708 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f00092a0 con 0x7f04f4153c50 2023-11-15T03:09:19.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.362+0000 7f04f9a36700 1 --2- 172.21.15.121:0/1679071708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f4153c50 0x7f04f414c2f0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 rx=0x7f04e400e8d0 tx=0x7f04e4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:19.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.362+0000 7f04ea7fc700 1 -- 172.21.15.121:0/1679071708 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4012600 con 0x7f04f4153c50 2023-11-15T03:09:19.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.362+0000 7f04ea7fc700 1 -- 172.21.15.121:0/1679071708 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e4012760 con 0x7f04f4153c50 2023-11-15T03:09:19.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.362+0000 7f04fb239700 1 -- 172.21.15.121:0/1679071708 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f4167ec0 con 0x7f04f4153c50 2023-11-15T03:09:19.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.362+0000 7f04ea7fc700 1 -- 172.21.15.121:0/1679071708 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4012ae0 con 0x7f04f4153c50 2023-11-15T03:09:19.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.362+0000 7f04fb239700 1 -- 172.21.15.121:0/1679071708 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f414cd00 con 0x7f04f4153c50 2023-11-15T03:09:19.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.362+0000 7f04fb239700 1 -- 172.21.15.121:0/1679071708 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f4062720 con 0x7f04f4153c50 2023-11-15T03:09:19.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.366+0000 7f04ea7fc700 1 -- 172.21.15.121:0/1679071708 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f04e401b400 con 0x7f04f4153c50 2023-11-15T03:09:19.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.366+0000 7f04ea7fc700 1 --2- 172.21.15.121:0/1679071708 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f04d805a510 0x7f04d805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:19.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.366+0000 7f04ea7fc700 1 -- 172.21.15.121:0/1679071708 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7f04e40757c0 con 0x7f04f4153c50 2023-11-15T03:09:19.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.370+0000 7f04fa237700 1 --2- 172.21.15.121:0/1679071708 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f04d805a510 0x7f04d805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:19.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.370+0000 7f04fa237700 1 --2- 172.21.15.121:0/1679071708 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f04d805a510 0x7f04d805c9d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f04f0009fd0 tx=0x7f04f0017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:19.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.374+0000 7f04ea7fc700 1 -- 172.21.15.121:0/1679071708 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f04e4048400 con 0x7f04f4153c50 2023-11-15T03:09:19.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.574+0000 7f04fb239700 1 -- 172.21.15.121:0/1679071708 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f04f4168650 con 0x7f04f4153c50 2023-11-15T03:09:19.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.574+0000 7f04ea7fc700 1 -- 172.21.15.121:0/1679071708 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v279) v1 ==== 53+0+73 (secure 0 0 0) 0x7f04f4168650 con 0x7f04f4153c50 2023-11-15T03:09:19.584 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T03:09:19.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.578+0000 7f04fb239700 1 -- 172.21.15.121:0/1679071708 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f04d805a510 msgr2=0x7f04d805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:19.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.578+0000 7f04fb239700 1 --2- 172.21.15.121:0/1679071708 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f04d805a510 0x7f04d805c9d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f04f0009fd0 tx=0x7f04f0017000).stop 2023-11-15T03:09:19.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.578+0000 7f04fb239700 1 -- 172.21.15.121:0/1679071708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f4153c50 msgr2=0x7f04f414c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:19.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.578+0000 7f04fb239700 1 --2- 172.21.15.121:0/1679071708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f4153c50 0x7f04f414c2f0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 rx=0x7f04e400e8d0 tx=0x7f04e4009060).stop 2023-11-15T03:09:19.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.578+0000 7f04fb239700 1 -- 172.21.15.121:0/1679071708 shutdown_connections 2023-11-15T03:09:19.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.578+0000 7f04fb239700 1 --2- 172.21.15.121:0/1679071708 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f04d805a510 0x7f04d805c9d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.578+0000 7f04fb239700 1 --2- 172.21.15.121:0/1679071708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f4154030 0x7f04f40bfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.578+0000 7f04fb239700 1 --2- 172.21.15.121:0/1679071708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f4153c50 0x7f04f414c2f0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.578+0000 7f04fb239700 1 --2- 172.21.15.121:0/1679071708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04f40653e0 0x7f04f40a6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:19.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.578+0000 7f04fb239700 1 -- 172.21.15.121:0/1679071708 >> 172.21.15.121:0/1679071708 conn(0x7f04f40677c0 msgr2=0x7f04f414d760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:19.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.578+0000 7f04fb239700 1 -- 172.21.15.121:0/1679071708 shutdown_connections 2023-11-15T03:09:19.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:19.578+0000 7f04fb239700 1 -- 172.21.15.121:0/1679071708 wait complete. 2023-11-15T03:09:19.605 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T03:09:19.605 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T03:09:19.606 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T03:09:19.606 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T03:09:19.606 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T03:09:19.607 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T03:09:19.607 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T03:09:19.607 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T03:09:19.608 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:09:19.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:19.608 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T03:09:19.630 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T03:09:19.631 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T03:09:19.631 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T03:09:19.632 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:09:19.632 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T03:09:19.632 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:09:19.632 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:09:19.632 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:09:19.633 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T03:09:19.633 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:09:19.633 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:09:19.633 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T03:09:19.633 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:09:19.634 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:09:19.634 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T03:09:19.634 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:09:19.634 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T03:09:19.634 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 41680sec preferred_lft 41680sec 2023-11-15T03:09:19.635 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T03:09:19.635 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:09:19.635 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T03:09:19.635 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:09:19.635 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T03:09:19.636 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:09:19.636 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T03:09:19.636 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T03:09:19.636 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T03:09:19.637 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:09:19.637 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T03:09:19.637 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:09:19.637 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:09:19.637 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T03:09:19.638 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T03:09:19.638 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:09:19.638 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:09:19.638 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T03:09:19.639 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T03:09:19.639 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:09:19.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:19.640 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T03:09:19.717 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T03:09:19.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:19.721 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T03:09:19.784 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T03:09:19.784 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T03:09:19.784 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T03:09:19.784 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T03:09:19.785 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T03:09:19.785 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T03:09:19.785 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T03:09:19.785 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T03:09:19.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:19.786 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T03:09:19.786 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T03:09:19.787 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T03:09:19.787 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T03:09:19.788 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T03:09:19.788 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T03:09:19.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:19.923 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T03:09:19.943 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T03:09:19.943 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T03:09:19.944 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:09:19.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:19.945 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T03:09:20.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:20.030 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T03:09:20.091 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T03:09:20.092 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:09:20.212 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:09:20.212 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:09:20.236 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:09:20.236 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:09:20.242 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T03:09:20.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:20.243 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T03:09:20.276 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-15T03:09:20.276 DEBUG:teuthology.orchestra.run.smithi121:> import os 2023-11-15T03:09:20.276 DEBUG:teuthology.orchestra.run.smithi121:> import stat 2023-11-15T03:09:20.277 DEBUG:teuthology.orchestra.run.smithi121:> import json 2023-11-15T03:09:20.277 DEBUG:teuthology.orchestra.run.smithi121:> import sys 2023-11-15T03:09:20.277 DEBUG:teuthology.orchestra.run.smithi121:> 2023-11-15T03:09:20.278 DEBUG:teuthology.orchestra.run.smithi121:> try: 2023-11-15T03:09:20.278 DEBUG:teuthology.orchestra.run.smithi121:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/./subdir/sixmegs'"'"') 2023-11-15T03:09:20.279 DEBUG:teuthology.orchestra.run.smithi121:> except OSError as e: 2023-11-15T03:09:20.279 DEBUG:teuthology.orchestra.run.smithi121:> sys.exit(e.errno) 2023-11-15T03:09:20.279 DEBUG:teuthology.orchestra.run.smithi121:> 2023-11-15T03:09:20.280 DEBUG:teuthology.orchestra.run.smithi121:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-11-15T03:09:20.280 DEBUG:teuthology.orchestra.run.smithi121:> print(json.dumps( 2023-11-15T03:09:20.280 DEBUG:teuthology.orchestra.run.smithi121:> dict([(a, getattr(s, a)) for a in attrs]), 2023-11-15T03:09:20.281 DEBUG:teuthology.orchestra.run.smithi121:> indent=2)) 2023-11-15T03:09:20.281 DEBUG:teuthology.orchestra.run.smithi121:> ' 2023-11-15T03:09:20.326 INFO:teuthology.orchestra.run:waiting for 20 2023-11-15T03:09:20.593 INFO:teuthology.orchestra.run.smithi121.stderr:daemon-helper: command failed with exit status 5 2023-11-15T03:09:23.200 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:09:23.190+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-11-15T03:09:26.431 DEBUG:teuthology.orchestra.run:got remote process result: 5 2023-11-15T03:09:26.431 INFO:tasks.cephfs.test_damage:Result: EIO on client 2023-11-15T03:09:26.432 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:09:26.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.590+0000 7f9879ba9700 1 Processor -- start 2023-11-15T03:09:26.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.590+0000 7f9879ba9700 1 -- start start 2023-11-15T03:09:26.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.594+0000 7f9879ba9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9874153bb0 0x7f98740a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:26.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.594+0000 7f9879ba9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9874068170 0x7f9874065d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:26.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.594+0000 7f9879ba9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98740cb5c0 con 0x7f98740aa020 2023-11-15T03:09:26.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.594+0000 7f9879ba9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98740aa6f0 con 0x7f9874153bb0 2023-11-15T03:09:26.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.594+0000 7f9879ba9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98740c5900 con 0x7f9874068170 2023-11-15T03:09:26.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.594+0000 7f98737fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9874153bb0 0x7f98740a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:26.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.594+0000 7f98737fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9874153bb0 0x7f98740a9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51740/0 (socket says 172.21.15.121:51740) 2023-11-15T03:09:26.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.594+0000 7f98737fe700 1 -- 172.21.15.121:0/1747627680 learned_addr learned my addr 172.21.15.121:0/1747627680 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:26.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.594+0000 7f9873fff700 1 --2- 172.21.15.121:0/1747627680 >> v2:172.21.15.190:3300/0 conn(0x7f9874068170 0x7f9874065d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:26.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.594+0000 7f98737fe700 1 -- 172.21.15.121:0/1747627680 >> v2:172.21.15.190:3300/0 conn(0x7f9874068170 msgr2=0x7f9874065d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:26.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.594+0000 7f98737fe700 1 --2- 172.21.15.121:0/1747627680 >> v2:172.21.15.190:3300/0 conn(0x7f9874068170 0x7f9874065d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:26.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.594+0000 7f98737fe700 1 -- 172.21.15.121:0/1747627680 >> v1:172.21.15.121:6789/0 conn(0x7f98740aa020 legacy=0x7f9874067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:26.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.594+0000 7f98737fe700 1 -- 172.21.15.121:0/1747627680 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98740c9ca0 con 0x7f9874153bb0 2023-11-15T03:09:26.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.594+0000 7f9873fff700 1 --2- 172.21.15.121:0/1747627680 >> v2:172.21.15.190:3300/0 conn(0x7f9874068170 0x7f9874065d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:26.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.594+0000 7f98737fe700 1 --2- 172.21.15.121:0/1747627680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9874153bb0 0x7f98740a9b30 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 rx=0x7f98740c7440 tx=0x7f98640090f0).ready entity=mon.0 client_cookie=153bfd7c560e52ec server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:26.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.598+0000 7f9872ffd700 1 -- 172.21.15.121:0/1747627680 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986400b600 con 0x7f9874153bb0 2023-11-15T03:09:26.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.598+0000 7f9872ffd700 1 -- 172.21.15.121:0/1747627680 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986400b760 con 0x7f9874153bb0 2023-11-15T03:09:26.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.598+0000 7f9872ffd700 1 -- 172.21.15.121:0/1747627680 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986400bae0 con 0x7f9874153bb0 2023-11-15T03:09:26.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.598+0000 7f9879ba9700 1 -- 172.21.15.121:0/1747627680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9874153bb0 msgr2=0x7f98740a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:26.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.598+0000 7f9879ba9700 1 --2- 172.21.15.121:0/1747627680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9874153bb0 0x7f98740a9b30 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 rx=0x7f98740c7440 tx=0x7f98640090f0).stop 2023-11-15T03:09:26.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.598+0000 7f9879ba9700 1 -- 172.21.15.121:0/1747627680 shutdown_connections 2023-11-15T03:09:26.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.598+0000 7f9879ba9700 1 --2- 172.21.15.121:0/1747627680 >> v2:172.21.15.190:3300/0 conn(0x7f9874068170 0x7f9874065d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:26.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.598+0000 7f9879ba9700 1 --2- 172.21.15.121:0/1747627680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9874153bb0 0x7f98740a9b30 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:26.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.598+0000 7f9879ba9700 1 -- 172.21.15.121:0/1747627680 >> 172.21.15.121:0/1747627680 conn(0x7f9874058c50 msgr2=0x7f9874059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:26.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.598+0000 7f9879ba9700 1 -- 172.21.15.121:0/1747627680 shutdown_connections 2023-11-15T03:09:26.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.598+0000 7f9879ba9700 1 -- 172.21.15.121:0/1747627680 wait complete. 2023-11-15T03:09:26.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.598+0000 7f9879ba9700 1 Processor -- start 2023-11-15T03:09:26.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.598+0000 7f9879ba9700 1 -- start start 2023-11-15T03:09:26.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f9879ba9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9874068170 0x7f98740ad410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:26.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f9879ba9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98740aa020 0x7f9874152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:26.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f9879ba9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9874153bb0 0x7f98740aa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:26.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f9879ba9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9874153040 con 0x7f9874068170 2023-11-15T03:09:26.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f9879ba9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98740aad30 con 0x7f9874153bb0 2023-11-15T03:09:26.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f9879ba9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98740aaf90 con 0x7f98740aa020 2023-11-15T03:09:26.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f98737fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9874068170 0x7f98740ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:26.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f98737fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9874068170 0x7f98740ad410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51756/0 (socket says 172.21.15.121:51756) 2023-11-15T03:09:26.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f98737fe700 1 -- 172.21.15.121:0/4153493601 learned_addr learned my addr 172.21.15.121:0/4153493601 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:26.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f986bfff700 1 --2- 172.21.15.121:0/4153493601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98740aa020 0x7f9874152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:26.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f9873fff700 1 --2- 172.21.15.121:0/4153493601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9874153bb0 0x7f98740aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:26.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f98737fe700 1 -- 172.21.15.121:0/4153493601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98740aa020 msgr2=0x7f9874152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:26.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f98737fe700 1 --2- 172.21.15.121:0/4153493601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98740aa020 0x7f9874152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:26.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f98737fe700 1 -- 172.21.15.121:0/4153493601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9874153bb0 msgr2=0x7f98740aa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:26.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f98737fe700 1 --2- 172.21.15.121:0/4153493601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9874153bb0 0x7f98740aa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:26.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f98737fe700 1 -- 172.21.15.121:0/4153493601 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9864009370 con 0x7f9874068170 2023-11-15T03:09:26.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f986bfff700 1 --2- 172.21.15.121:0/4153493601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98740aa020 0x7f9874152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:26.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f9873fff700 1 --2- 172.21.15.121:0/4153493601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9874153bb0 0x7f98740aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:26.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f98737fe700 1 --2- 172.21.15.121:0/4153493601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9874068170 0x7f98740ad410 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 rx=0x7f9874065560 tx=0x7f9864017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:26.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f986b7fe700 1 -- 172.21.15.121:0/4153493601 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986400bae0 con 0x7f9874068170 2023-11-15T03:09:26.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f986b7fe700 1 -- 172.21.15.121:0/4153493601 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9864000c90 con 0x7f9874068170 2023-11-15T03:09:26.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f9879ba9700 1 -- 172.21.15.121:0/4153493601 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98740ab270 con 0x7f9874068170 2023-11-15T03:09:26.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f986b7fe700 1 -- 172.21.15.121:0/4153493601 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986401b640 con 0x7f9874068170 2023-11-15T03:09:26.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.602+0000 7f9879ba9700 1 -- 172.21.15.121:0/4153493601 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98740ab830 con 0x7f9874068170 2023-11-15T03:09:26.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.606+0000 7f9879ba9700 1 -- 172.21.15.121:0/4153493601 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f987406c150 con 0x7f9874068170 2023-11-15T03:09:26.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.606+0000 7f986b7fe700 1 -- 172.21.15.121:0/4153493601 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f986401ba30 con 0x7f9874068170 2023-11-15T03:09:26.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.606+0000 7f986b7fe700 1 --2- 172.21.15.121:0/4153493601 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f985405a510 0x7f985405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:26.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.606+0000 7f986b7fe700 1 -- 172.21.15.121:0/4153493601 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7f9864076fa0 con 0x7f9874068170 2023-11-15T03:09:26.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.606+0000 7f986bfff700 1 --2- 172.21.15.121:0/4153493601 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f985405a510 0x7f985405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:26.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.610+0000 7f986bfff700 1 --2- 172.21.15.121:0/4153493601 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f985405a510 0x7f985405c9d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f98740c4cc0 tx=0x7f985c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:26.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.614+0000 7f986b7fe700 1 -- 172.21.15.121:0/4153493601 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9864049cc0 con 0x7f9874068170 2023-11-15T03:09:26.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.806+0000 7f9879ba9700 1 -- 172.21.15.121:0/4153493601 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98740c9f70 con 0x7f9874068170 2023-11-15T03:09:26.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.810+0000 7f986b7fe700 1 -- 172.21.15.121:0/4153493601 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 280 v280) v1 ==== 95+0+4502 (secure 0 0 0) 0x7f986404d2e0 con 0x7f9874068170 2023-11-15T03:09:26.819 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:09:26.819 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":280,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":280,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:09:23.195724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9807":{"gid":9807,"name":"a","rank":0,"incarnation":276,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6836/142843299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":142843299},{"type":"v1","addr":"172.21.15.121:6836","nonce":142843299}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:09:26.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.814+0000 7f9879ba9700 1 -- 172.21.15.121:0/4153493601 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f985405a510 msgr2=0x7f985405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:26.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.814+0000 7f9879ba9700 1 --2- 172.21.15.121:0/4153493601 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f985405a510 0x7f985405c9d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f98740c4cc0 tx=0x7f985c009040).stop 2023-11-15T03:09:26.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.814+0000 7f9879ba9700 1 -- 172.21.15.121:0/4153493601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9874068170 msgr2=0x7f98740ad410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:26.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.814+0000 7f9879ba9700 1 --2- 172.21.15.121:0/4153493601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9874068170 0x7f98740ad410 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 rx=0x7f9874065560 tx=0x7f9864017000).stop 2023-11-15T03:09:26.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.814+0000 7f9879ba9700 1 -- 172.21.15.121:0/4153493601 shutdown_connections 2023-11-15T03:09:26.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.814+0000 7f9879ba9700 1 --2- 172.21.15.121:0/4153493601 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f985405a510 0x7f985405c9d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:26.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.814+0000 7f9879ba9700 1 --2- 172.21.15.121:0/4153493601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9874153bb0 0x7f98740aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:26.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.814+0000 7f9879ba9700 1 --2- 172.21.15.121:0/4153493601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98740aa020 0x7f9874152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:26.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.814+0000 7f9879ba9700 1 --2- 172.21.15.121:0/4153493601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9874068170 0x7f98740ad410 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:26.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.814+0000 7f9879ba9700 1 -- 172.21.15.121:0/4153493601 >> 172.21.15.121:0/4153493601 conn(0x7f9874058c50 msgr2=0x7f98740b03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:26.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.814+0000 7f9879ba9700 1 -- 172.21.15.121:0/4153493601 shutdown_connections 2023-11-15T03:09:26.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.814+0000 7f9879ba9700 1 -- 172.21.15.121:0/4153493601 wait complete. 2023-11-15T03:09:26.826 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 280 2023-11-15T03:09:26.842 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2023-11-15T03:09:27.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.994+0000 7f6e599a9700 1 Processor -- start 2023-11-15T03:09:27.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.994+0000 7f6e599a9700 1 -- start start 2023-11-15T03:09:27.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.998+0000 7f6e599a9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6e540a4100 0x7f6e540a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.998+0000 7f6e599a9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e540a4aa0 con 0x7f6e540661d0 2023-11-15T03:09:27.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.998+0000 7f6e599a9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e54065780 con 0x7f6e5414abc0 2023-11-15T03:09:27.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.998+0000 7f6e599a9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e54065d40 con 0x7f6e540a4100 2023-11-15T03:09:27.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.998+0000 7f6e537fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6e540661d0 0x7f6e54066660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45812/0 (socket says 172.21.15.121:45812) 2023-11-15T03:09:27.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.998+0000 7f6e537fe700 1 -- 172.21.15.121:0/1858949010 learned_addr learned my addr 172.21.15.121:0/1858949010 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:09:27.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.998+0000 7f6e527fc700 1 --2- 172.21.15.121:0/1858949010 >> v2:172.21.15.190:3300/0 conn(0x7f6e540a4100 0x7f6e540a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.998+0000 7f6e51ffb700 1 -- 172.21.15.121:0/1858949010 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750611575 0 0) 0x7f6e4c001490 con 0x7f6e540661d0 2023-11-15T03:09:27.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.998+0000 7f6e51ffb700 1 -- 172.21.15.121:0/1858949010 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e3c0033f0 con 0x7f6e540661d0 2023-11-15T03:09:27.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.998+0000 7f6e527fc700 1 -- 172.21.15.121:0/1858949010 >> v1:172.21.15.190:6789/0 conn(0x7f6e5414abc0 legacy=0x7f6e541540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:27.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.998+0000 7f6e527fc700 1 -- 172.21.15.121:0/1858949010 >> v1:172.21.15.121:6789/0 conn(0x7f6e540661d0 legacy=0x7f6e54066660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:27.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.998+0000 7f6e527fc700 1 -- 172.21.15.121:0/1858949010 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5414afa0 con 0x7f6e540a4100 2023-11-15T03:09:27.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.998+0000 7f6e527fc700 1 --2- 172.21.15.121:0/1858949010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e540a4100 0x7f6e540a45b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f6e48009760 tx=0x7f6e48009060).ready entity=mon.1 client_cookie=6c7131cf45370a50 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:27.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.998+0000 7f6e51ffb700 1 -- 172.21.15.121:0/1858949010 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4800f510 con 0x7f6e540a4100 2023-11-15T03:09:27.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.998+0000 7f6e51ffb700 1 -- 172.21.15.121:0/1858949010 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4800f670 con 0x7f6e540a4100 2023-11-15T03:09:27.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:26.998+0000 7f6e51ffb700 1 -- 172.21.15.121:0/1858949010 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4800f9f0 con 0x7f6e540a4100 2023-11-15T03:09:27.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.002+0000 7f6e599a9700 1 -- 172.21.15.121:0/1858949010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e540a4100 msgr2=0x7f6e540a45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:27.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.002+0000 7f6e599a9700 1 --2- 172.21.15.121:0/1858949010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e540a4100 0x7f6e540a45b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f6e48009760 tx=0x7f6e48009060).stop 2023-11-15T03:09:27.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.002+0000 7f6e599a9700 1 -- 172.21.15.121:0/1858949010 shutdown_connections 2023-11-15T03:09:27.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.002+0000 7f6e599a9700 1 --2- 172.21.15.121:0/1858949010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e540a4100 0x7f6e540a45b0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.002+0000 7f6e599a9700 1 -- 172.21.15.121:0/1858949010 >> 172.21.15.121:0/1858949010 conn(0x7f6e54068530 msgr2=0x7f6e540c6560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:27.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.002+0000 7f6e599a9700 1 -- 172.21.15.121:0/1858949010 shutdown_connections 2023-11-15T03:09:27.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.002+0000 7f6e599a9700 1 -- 172.21.15.121:0/1858949010 wait complete. 2023-11-15T03:09:27.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e599a9700 1 Processor -- start 2023-11-15T03:09:27.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e599a9700 1 -- start start 2023-11-15T03:09:27.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e599a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e540661d0 0x7f6e54167e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e599a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e540a4100 0x7f6e540abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e599a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e5414abc0 0x7f6e54141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e599a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e54164800 con 0x7f6e540661d0 2023-11-15T03:09:27.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e599a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5414f870 con 0x7f6e540a4100 2023-11-15T03:09:27.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e599a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e541683b0 con 0x7f6e5414abc0 2023-11-15T03:09:27.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e52ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e540661d0 0x7f6e54167e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e52ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e540661d0 0x7f6e54167e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51772/0 (socket says 172.21.15.121:51772) 2023-11-15T03:09:27.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e52ffd700 1 -- 172.21.15.121:0/143266224 learned_addr learned my addr 172.21.15.121:0/143266224 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:27.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e527fc700 1 --2- 172.21.15.121:0/143266224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e540a4100 0x7f6e540abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e537fe700 1 --2- 172.21.15.121:0/143266224 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e5414abc0 0x7f6e54141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e52ffd700 1 -- 172.21.15.121:0/143266224 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e5414abc0 msgr2=0x7f6e54141fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:27.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e52ffd700 1 --2- 172.21.15.121:0/143266224 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e5414abc0 0x7f6e54141fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e52ffd700 1 -- 172.21.15.121:0/143266224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e540a4100 msgr2=0x7f6e540abfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:27.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e52ffd700 1 --2- 172.21.15.121:0/143266224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e540a4100 0x7f6e540abfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e52ffd700 1 -- 172.21.15.121:0/143266224 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e480092a0 con 0x7f6e540661d0 2023-11-15T03:09:27.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e527fc700 1 --2- 172.21.15.121:0/143266224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e540a4100 0x7f6e540abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:09:27.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.006+0000 7f6e52ffd700 1 --2- 172.21.15.121:0/143266224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e540661d0 0x7f6e54167e10 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 rx=0x7f6e4400c440 tx=0x7f6e44007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:27.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.010+0000 7f6e3affd700 1 -- 172.21.15.121:0/143266224 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44007020 con 0x7f6e540661d0 2023-11-15T03:09:27.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.010+0000 7f6e3affd700 1 -- 172.21.15.121:0/143266224 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e44007180 con 0x7f6e540661d0 2023-11-15T03:09:27.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.010+0000 7f6e3affd700 1 -- 172.21.15.121:0/143266224 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44007520 con 0x7f6e540661d0 2023-11-15T03:09:27.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.010+0000 7f6e599a9700 1 -- 172.21.15.121:0/143266224 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e54168690 con 0x7f6e540661d0 2023-11-15T03:09:27.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.010+0000 7f6e599a9700 1 -- 172.21.15.121:0/143266224 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e54168c50 con 0x7f6e540661d0 2023-11-15T03:09:27.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.010+0000 7f6e3affd700 1 -- 172.21.15.121:0/143266224 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6e440068b0 con 0x7f6e540661d0 2023-11-15T03:09:27.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.014+0000 7f6e3affd700 1 --2- 172.21.15.121:0/143266224 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e4005a510 0x7f6e4005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.014+0000 7f6e63d67740 1 Processor -- start 2023-11-15T03:09:27.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.014+0000 7f6e3affd700 1 -- 172.21.15.121:0/143266224 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7f6e440746e0 con 0x7f6e540661d0 2023-11-15T03:09:27.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.014+0000 7f6e527fc700 1 --2- 172.21.15.121:0/143266224 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e4005a510 0x7f6e4005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.014+0000 7f6e63d67740 1 -- start start 2023-11-15T03:09:27.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.014+0000 7f6e527fc700 1 --2- 172.21.15.121:0/143266224 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e4005a510 0x7f6e4005c9d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f6e5414e1c0 tx=0x7f6e48017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:27.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.014+0000 7f6e63d67740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27ed810 0x27edbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.014+0000 7f6e63d67740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 0x27a9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.014+0000 7f6e63d67740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27a4110 0x27a4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.014+0000 7f6e63d67740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x26e53f0 con 0x27f1810 2023-11-15T03:09:27.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.014+0000 7f6e63d67740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x2708580 con 0x27ed810 2023-11-15T03:09:27.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.014+0000 7f6e63d67740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x26d1d50 con 0x27a4110 2023-11-15T03:09:27.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.014+0000 7f6e527fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 0x27a9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.014+0000 7f6e527fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 0x27a9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51778/0 (socket says 172.21.15.121:51778) 2023-11-15T03:09:27.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.014+0000 7f6e527fc700 1 -- 172.21.15.121:0/1658071099 learned_addr learned my addr 172.21.15.121:0/1658071099 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:27.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.014+0000 7f6e52ffd700 1 --2- 172.21.15.121:0/1658071099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27ed810 0x27edbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.014+0000 7f6e537fe700 1 --2- 172.21.15.121:0/1658071099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27a4110 0x27a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e527fc700 1 -- 172.21.15.121:0/1658071099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27a4110 msgr2=0x27a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:27.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e527fc700 1 --2- 172.21.15.121:0/1658071099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27a4110 0x27a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e527fc700 1 -- 172.21.15.121:0/1658071099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27ed810 msgr2=0x27edbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:27.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e527fc700 1 --2- 172.21.15.121:0/1658071099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27ed810 0x27edbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e527fc700 1 -- 172.21.15.121:0/1658071099 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2725830 con 0x27f1810 2023-11-15T03:09:27.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e527fc700 1 --2- 172.21.15.121:0/1658071099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 0x27a9310 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 rx=0x7f6e540695a0 tx=0x7f6e4801d040).ready entity=mon.0 client_cookie=8982e4363a9ff234 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:27.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e38ff9700 1 -- 172.21.15.121:0/1658071099 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4801b870 con 0x27f1810 2023-11-15T03:09:27.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e38ff9700 1 -- 172.21.15.121:0/1658071099 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4801b9d0 con 0x27f1810 2023-11-15T03:09:27.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e38ff9700 1 -- 172.21.15.121:0/1658071099 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4801bd70 con 0x27f1810 2023-11-15T03:09:27.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e63d67740 1 -- 172.21.15.121:0/1658071099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 msgr2=0x27a9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:27.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e63d67740 1 --2- 172.21.15.121:0/1658071099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 0x27a9310 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 rx=0x7f6e540695a0 tx=0x7f6e4801d040).stop 2023-11-15T03:09:27.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e63d67740 1 -- 172.21.15.121:0/1658071099 shutdown_connections 2023-11-15T03:09:27.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e63d67740 1 --2- 172.21.15.121:0/1658071099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27a4110 0x27a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e63d67740 1 --2- 172.21.15.121:0/1658071099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 0x27a9310 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e63d67740 1 --2- 172.21.15.121:0/1658071099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27ed810 0x27edbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e63d67740 1 -- 172.21.15.121:0/1658071099 >> 172.21.15.121:0/1658071099 conn(0x258a790 msgr2=0x27f1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:27.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e63d67740 1 -- 172.21.15.121:0/1658071099 shutdown_connections 2023-11-15T03:09:27.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e63d67740 1 -- 172.21.15.121:0/1658071099 wait complete. 2023-11-15T03:09:27.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e63d67740 1 -- start start 2023-11-15T03:09:27.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.018+0000 7f6e63d67740 1 Processor -- start 2023-11-15T03:09:27.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e63d67740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27a4110 0x27e8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e63d67740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27ed810 0x27a4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e63d67740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 0x27a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e63d67740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x27a4f70 con 0x27f1810 2023-11-15T03:09:27.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e63d67740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x27f0e30 con 0x27ed810 2023-11-15T03:09:27.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e63d67740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x27e94b0 con 0x27a4110 2023-11-15T03:09:27.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e63d67740 10 client.0 ll_register_callbacks cb 0x27226b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:09:27.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e537fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 0x27a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e537fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 0x27a4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51792/0 (socket says 172.21.15.121:51792) 2023-11-15T03:09:27.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e537fe700 1 -- 172.21.15.121:0/2229818332 learned_addr learned my addr 172.21.15.121:0/2229818332 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:27.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e527fc700 1 --2- 172.21.15.121:0/2229818332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27ed810 0x27a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e52ffd700 1 --2- 172.21.15.121:0/2229818332 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27a4110 0x27e8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e537fe700 1 -- 172.21.15.121:0/2229818332 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27a4110 msgr2=0x27e8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:27.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e537fe700 1 --2- 172.21.15.121:0/2229818332 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27a4110 0x27e8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e537fe700 1 -- 172.21.15.121:0/2229818332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27ed810 msgr2=0x27a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:27.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e537fe700 1 --2- 172.21.15.121:0/2229818332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27ed810 0x27a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e537fe700 1 -- 172.21.15.121:0/2229818332 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5414afa0 con 0x27f1810 2023-11-15T03:09:27.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e537fe700 1 --2- 172.21.15.121:0/2229818332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 0x27a4a80 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 rx=0x7f6e4c002100 tx=0x7f6e4c009040).ready entity=mon.0 client_cookie=b5f106346ddb707e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:27.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:09:27.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e31ffb700 1 -- 172.21.15.121:0/2229818332 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c005d70 con 0x27f1810 2023-11-15T03:09:27.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e31ffb700 1 -- 172.21.15.121:0/2229818332 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7f6e4c014400 con 0x27f1810 2023-11-15T03:09:27.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e63d67740 1 -- 172.21.15.121:0/2229818332 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x27222a0 con 0x27f1810 2023-11-15T03:09:27.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e31ffb700 1 -- 172.21.15.121:0/2229818332 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=280) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e4c0147d0 con 0x27f1810 2023-11-15T03:09:27.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e63d67740 10 client.9921 fetch_fsmap learned FSMap version 280 2023-11-15T03:09:27.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e63d67740 1 -- 172.21.15.121:0/2229818332 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=280,monmap=2+}) v3 -- 0x2720a60 con 0x27f1810 2023-11-15T03:09:27.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e31ffb700 1 -- 172.21.15.121:0/2229818332 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x27222a0 con 0x27f1810 2023-11-15T03:09:27.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e31ffb700 1 -- 172.21.15.121:0/2229818332 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 280) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6e4c014930 con 0x27f1810 2023-11-15T03:09:27.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e63d67740 10 client.9921 fetch_fsmap finished waiting for FSMap version 280 2023-11-15T03:09:27.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e63d67740 10 client.9921 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:09:27.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e63d67740 20 client.9921 populate_metadata read hostname 'smithi121' 2023-11-15T03:09:27.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e63d67740 1 --2- 172.21.15.121:0/2229818332 >> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] conn(0x27ce930 0x280b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e63d67740 4 client.9921 mds_command: new command op to 9807 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:09:27.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.022+0000 7f6e63d67740 1 -- 172.21.15.121:0/2229818332 --> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x27187f0 con 0x27ce930 2023-11-15T03:09:27.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.026+0000 7f6e52ffd700 1 --2- 172.21.15.121:0/2229818332 >> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] conn(0x27ce930 0x280b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.026+0000 7f6e52ffd700 1 --2- 172.21.15.121:0/2229818332 >> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] conn(0x27ce930 0x280b090 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=8ca6ebf89302bb26 server_cookie=599033582df6838d in_seq=0 out_seq=0 2023-11-15T03:09:27.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.026+0000 7f6e31ffb700 10 client.9921 ms_handle_connect on v2:172.21.15.121:6834/142843299 2023-11-15T03:09:27.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.026+0000 7f6e31ffb700 1 -- 172.21.15.121:0/2229818332 <== mds.0 v2:172.21.15.121:6834/142843299 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x27187f0 con 0x27ce930 2023-11-15T03:09:27.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.026+0000 7f6e31ffb700 10 client.9921 handle_command_reply: tid=0 2023-11-15T03:09:27.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.026+0000 7f6e63d67740 1 client.9921 shutdown 2023-11-15T03:09:27.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.026+0000 7f6e63d67740 1 -- 172.21.15.121:0/2229818332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 msgr2=0x27a4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:27.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.026+0000 7f6e63d67740 1 --2- 172.21.15.121:0/2229818332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 0x27a4a80 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 rx=0x7f6e4c002100 tx=0x7f6e4c009040).stop 2023-11-15T03:09:27.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.026+0000 7f6e63d67740 1 -- 172.21.15.121:0/2229818332 shutdown_connections 2023-11-15T03:09:27.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.026+0000 7f6e63d67740 1 --2- 172.21.15.121:0/2229818332 >> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] conn(0x27ce930 0x280b090 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.026+0000 7f6e63d67740 1 --2- 172.21.15.121:0/2229818332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 0x27a4a80 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.026+0000 7f6e63d67740 1 --2- 172.21.15.121:0/2229818332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27ed810 0x27a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.026+0000 7f6e63d67740 1 --2- 172.21.15.121:0/2229818332 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27a4110 0x27e8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.026+0000 7f6e31ffb700 0 client.9921 ms_handle_reset on v2:172.21.15.121:6834/142843299 2023-11-15T03:09:27.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.026+0000 7f6e63d67740 1 -- 172.21.15.121:0/2229818332 >> 172.21.15.121:0/2229818332 conn(0x258a790 msgr2=0x27d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:27.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.026+0000 7f6e63d67740 1 -- 172.21.15.121:0/2229818332 shutdown_connections 2023-11-15T03:09:27.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.026+0000 7f6e63d67740 1 -- 172.21.15.121:0/2229818332 wait complete. 2023-11-15T03:09:27.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.026+0000 7f6e63d67740 20 client.9921 trim_cache size 0 max 16384 2023-11-15T03:09:27.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.042+0000 7f6e63d67740 1 Processor -- start 2023-11-15T03:09:27.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.042+0000 7f6e63d67740 1 -- start start 2023-11-15T03:09:27.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.042+0000 7f6e63d67740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27a4110 0x27a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.042+0000 7f6e63d67740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27ed810 0x27f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.042+0000 7f6e63d67740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 0x27f2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.042+0000 7f6e63d67740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e54069400 con 0x27f1810 2023-11-15T03:09:27.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.042+0000 7f6e63d67740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x279ff30 con 0x27a4110 2023-11-15T03:09:27.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.042+0000 7f6e63d67740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x26e3f40 con 0x27ed810 2023-11-15T03:09:27.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.042+0000 7f6e537fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 0x27f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.042+0000 7f6e537fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 0x27f2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51800/0 (socket says 172.21.15.121:51800) 2023-11-15T03:09:27.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.042+0000 7f6e537fe700 1 -- 172.21.15.121:0/3979656006 learned_addr learned my addr 172.21.15.121:0/3979656006 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:27.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.042+0000 7f6e527fc700 1 --2- 172.21.15.121:0/3979656006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27ed810 0x27f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.042+0000 7f6e52ffd700 1 --2- 172.21.15.121:0/3979656006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27a4110 0x27a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.042+0000 7f6e537fe700 1 -- 172.21.15.121:0/3979656006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27ed810 msgr2=0x27f1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:27.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.042+0000 7f6e537fe700 1 --2- 172.21.15.121:0/3979656006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27ed810 0x27f1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.042+0000 7f6e537fe700 1 -- 172.21.15.121:0/3979656006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27a4110 msgr2=0x27a48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:27.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.042+0000 7f6e537fe700 1 --2- 172.21.15.121:0/3979656006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27a4110 0x27a48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.046+0000 7f6e537fe700 1 -- 172.21.15.121:0/3979656006 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e540c4680 con 0x27f1810 2023-11-15T03:09:27.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.046+0000 7f6e52ffd700 1 --2- 172.21.15.121:0/3979656006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27a4110 0x27a48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:09:27.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.046+0000 7f6e537fe700 1 --2- 172.21.15.121:0/3979656006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 0x27f2100 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 rx=0x7f6e4c007e20 tx=0x7f6e4c021000).ready entity=mon.0 client_cookie=c56a2820e33b95e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:27.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.046+0000 7f6e397fa700 1 -- 172.21.15.121:0/3979656006 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c0076c0 con 0x27f1810 2023-11-15T03:09:27.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.046+0000 7f6e397fa700 1 -- 172.21.15.121:0/3979656006 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c014e90 con 0x27f1810 2023-11-15T03:09:27.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.046+0000 7f6e397fa700 1 -- 172.21.15.121:0/3979656006 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c0149e0 con 0x27f1810 2023-11-15T03:09:27.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.046+0000 7f6e63d67740 1 -- 172.21.15.121:0/3979656006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 msgr2=0x27f2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:27.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.046+0000 7f6e63d67740 1 --2- 172.21.15.121:0/3979656006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 0x27f2100 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 rx=0x7f6e4c007e20 tx=0x7f6e4c021000).stop 2023-11-15T03:09:27.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.046+0000 7f6e63d67740 1 -- 172.21.15.121:0/3979656006 shutdown_connections 2023-11-15T03:09:27.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.046+0000 7f6e63d67740 1 --2- 172.21.15.121:0/3979656006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27f1810 0x27f2100 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.046+0000 7f6e63d67740 1 --2- 172.21.15.121:0/3979656006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27ed810 0x27f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.046+0000 7f6e63d67740 1 --2- 172.21.15.121:0/3979656006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27a4110 0x27a48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.046+0000 7f6e63d67740 1 -- 172.21.15.121:0/3979656006 >> 172.21.15.121:0/3979656006 conn(0x258a790 msgr2=0x280f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:27.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.046+0000 7f6e63d67740 1 -- 172.21.15.121:0/3979656006 shutdown_connections 2023-11-15T03:09:27.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.046+0000 7f6e63d67740 1 -- 172.21.15.121:0/3979656006 wait complete. 2023-11-15T03:09:27.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.046+0000 7f6e63d67740 1 -- start start 2023-11-15T03:09:27.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.046+0000 7f6e63d67740 1 Processor -- start 2023-11-15T03:09:27.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e63d67740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27a4110 0x27a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e63d67740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27ed810 0x27a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e63d67740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27f1810 0x285cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e63d67740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x27cf350 con 0x27ed810 2023-11-15T03:09:27.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e63d67740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x27c47d0 con 0x27a4110 2023-11-15T03:09:27.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e63d67740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x27a5a90 con 0x27f1810 2023-11-15T03:09:27.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e63d67740 10 client.0 ll_register_callbacks cb 0x7f6e4c001c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:09:27.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e527fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27ed810 0x27a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e527fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27ed810 0x27a55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51802/0 (socket says 172.21.15.121:51802) 2023-11-15T03:09:27.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e527fc700 1 -- 172.21.15.121:0/751172241 learned_addr learned my addr 172.21.15.121:0/751172241 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:27.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e52ffd700 1 --2- 172.21.15.121:0/751172241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27a4110 0x27a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e537fe700 1 --2- 172.21.15.121:0/751172241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27f1810 0x285cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e527fc700 1 -- 172.21.15.121:0/751172241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27f1810 msgr2=0x285cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:27.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e527fc700 1 --2- 172.21.15.121:0/751172241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27f1810 0x285cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e527fc700 1 -- 172.21.15.121:0/751172241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27a4110 msgr2=0x27a50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:27.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e527fc700 1 --2- 172.21.15.121:0/751172241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27a4110 0x27a50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e527fc700 1 -- 172.21.15.121:0/751172241 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2720a60 con 0x27ed810 2023-11-15T03:09:27.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e537fe700 1 --2- 172.21.15.121:0/751172241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27f1810 0x285cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:09:27.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e527fc700 1 --2- 172.21.15.121:0/751172241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27ed810 0x27a55a0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 rx=0x7f6e48028000 tx=0x7f6e48026000).ready entity=mon.0 client_cookie=9625fc225702ef08 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:27.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:09:27.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e31ffb700 1 -- 172.21.15.121:0/751172241 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48006990 con 0x27ed810 2023-11-15T03:09:27.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e31ffb700 1 -- 172.21.15.121:0/751172241 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7f6e48025820 con 0x27ed810 2023-11-15T03:09:27.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e63d67740 1 -- 172.21.15.121:0/751172241 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e4801b9d0 con 0x27ed810 2023-11-15T03:09:27.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e31ffb700 1 -- 172.21.15.121:0/751172241 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=280) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e48025bf0 con 0x27ed810 2023-11-15T03:09:27.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e63d67740 10 client.9927 fetch_fsmap learned FSMap version 280 2023-11-15T03:09:27.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e63d67740 1 -- 172.21.15.121:0/751172241 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=280,monmap=2+}) v3 -- 0x27207b0 con 0x27ed810 2023-11-15T03:09:27.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e31ffb700 1 -- 172.21.15.121:0/751172241 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4801b9d0 con 0x27ed810 2023-11-15T03:09:27.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e31ffb700 1 -- 172.21.15.121:0/751172241 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 280) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6e4802e450 con 0x27ed810 2023-11-15T03:09:27.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e63d67740 10 client.9927 fetch_fsmap finished waiting for FSMap version 280 2023-11-15T03:09:27.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e63d67740 10 client.9927 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:09:27.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e63d67740 20 client.9927 populate_metadata read hostname 'smithi121' 2023-11-15T03:09:27.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e63d67740 1 --2- 172.21.15.121:0/751172241 >> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] conn(0x285c800 0x285d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e63d67740 4 client.9927 mds_command: new command op to 9807 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-11-15T03:09:27.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.050+0000 7f6e63d67740 1 -- 172.21.15.121:0/751172241 --> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x25e1720 con 0x285c800 2023-11-15T03:09:27.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e52ffd700 1 --2- 172.21.15.121:0/751172241 >> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] conn(0x285c800 0x285d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e31ffb700 10 client.9927 ms_handle_connect on v2:172.21.15.121:6834/142843299 2023-11-15T03:09:27.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e52ffd700 1 --2- 172.21.15.121:0/751172241 >> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] conn(0x285c800 0x285d400 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=be1100b4a449a379 server_cookie=b71a81507dd85bc7 in_seq=0 out_seq=0 2023-11-15T03:09:27.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e31ffb700 1 -- 172.21.15.121:0/751172241 <== mds.0 v2:172.21.15.121:6834/142843299 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+215 (crc 0 0 0) 0x25e1720 con 0x285c800 2023-11-15T03:09:27.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e31ffb700 10 client.9927 handle_command_reply: tid=0 2023-11-15T03:09:27.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e63d67740 1 client.9927 shutdown 2023-11-15T03:09:27.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e63d67740 1 -- 172.21.15.121:0/751172241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27ed810 msgr2=0x27a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:27.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e63d67740 1 --2- 172.21.15.121:0/751172241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27ed810 0x27a55a0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 rx=0x7f6e48028000 tx=0x7f6e48026000).stop 2023-11-15T03:09:27.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e63d67740 1 -- 172.21.15.121:0/751172241 shutdown_connections 2023-11-15T03:09:27.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e63d67740 1 --2- 172.21.15.121:0/751172241 >> [v2:172.21.15.121:6834/142843299,v1:172.21.15.121:6836/142843299] conn(0x285c800 0x285d400 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e63d67740 1 --2- 172.21.15.121:0/751172241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x27f1810 0x285cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e63d67740 1 --2- 172.21.15.121:0/751172241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x27ed810 0x27a55a0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e63d67740 1 --2- 172.21.15.121:0/751172241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x27a4110 0x27a50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e31ffb700 0 client.9927 ms_handle_reset on v2:172.21.15.121:6834/142843299 2023-11-15T03:09:27.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e63d67740 1 -- 172.21.15.121:0/751172241 >> 172.21.15.121:0/751172241 conn(0x258a790 msgr2=0x27a83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:27.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e63d67740 1 -- 172.21.15.121:0/751172241 shutdown_connections 2023-11-15T03:09:27.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e63d67740 1 -- 172.21.15.121:0/751172241 wait complete. 2023-11-15T03:09:27.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e63d67740 20 client.9927 trim_cache size 0 max 16384 2023-11-15T03:09:27.079 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:09:27.080 INFO:teuthology.orchestra.run.smithi121.stdout:[ 2023-11-15T03:09:27.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:09:27.080 INFO:teuthology.orchestra.run.smithi121.stdout: "damage_type": "dentry", 2023-11-15T03:09:27.080 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 2356772222, 2023-11-15T03:09:27.080 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 1099511627776, 2023-11-15T03:09:27.080 INFO:teuthology.orchestra.run.smithi121.stdout: "frag": "*", 2023-11-15T03:09:27.081 INFO:teuthology.orchestra.run.smithi121.stdout: "dname": "sixmegs", 2023-11-15T03:09:27.081 INFO:teuthology.orchestra.run.smithi121.stdout: "snap_id": "head", 2023-11-15T03:09:27.081 INFO:teuthology.orchestra.run.smithi121.stdout: "path": "/subdir/sixmegs" 2023-11-15T03:09:27.081 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:09:27.081 INFO:teuthology.orchestra.run.smithi121.stdout:] 2023-11-15T03:09:27.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e599a9700 1 -- 172.21.15.121:0/143266224 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e4005a510 msgr2=0x7f6e4005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:27.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e599a9700 1 --2- 172.21.15.121:0/143266224 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e4005a510 0x7f6e4005c9d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f6e5414e1c0 tx=0x7f6e48017000).stop 2023-11-15T03:09:27.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e599a9700 1 -- 172.21.15.121:0/143266224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e540661d0 msgr2=0x7f6e54167e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:27.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e599a9700 1 --2- 172.21.15.121:0/143266224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e540661d0 0x7f6e54167e10 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 rx=0x7f6e4400c440 tx=0x7f6e44007e00).stop 2023-11-15T03:09:27.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e599a9700 1 -- 172.21.15.121:0/143266224 shutdown_connections 2023-11-15T03:09:27.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e599a9700 1 --2- 172.21.15.121:0/143266224 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e4005a510 0x7f6e4005c9d0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e599a9700 1 --2- 172.21.15.121:0/143266224 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e5414abc0 0x7f6e54141fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e599a9700 1 --2- 172.21.15.121:0/143266224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e540a4100 0x7f6e540abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e599a9700 1 --2- 172.21.15.121:0/143266224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e540661d0 0x7f6e54167e10 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e599a9700 1 -- 172.21.15.121:0/143266224 >> 172.21.15.121:0/143266224 conn(0x7f6e54068530 msgr2=0x7f6e540a7650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:27.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e599a9700 1 -- 172.21.15.121:0/143266224 shutdown_connections 2023-11-15T03:09:27.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.054+0000 7f6e599a9700 1 -- 172.21.15.121:0/143266224 wait complete. 2023-11-15T03:09:27.086 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 100.00000000:stray0_head' 2023-11-15T03:09:27.086 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T03:09:27.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:09:27.087 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-15T03:09:27.127 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:09:27.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:27.129 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:09:27.193 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:09:27.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.398+0000 7f5b7cc8f700 1 Processor -- start 2023-11-15T03:09:27.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.398+0000 7f5b7cc8f700 1 -- start start 2023-11-15T03:09:27.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.398+0000 7f5b7cc8f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5b780ad1b0 0x7f5b78065710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.398+0000 7f5b7cc8f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5b78065c00 0x7f5b780add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.398+0000 7f5b7cc8f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b78066c90 con 0x7f5b780ab0d0 2023-11-15T03:09:27.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.398+0000 7f5b7cc8f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b78066080 con 0x7f5b78065c00 2023-11-15T03:09:27.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.398+0000 7f5b7cc8f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b78066360 con 0x7f5b780ad1b0 2023-11-15T03:09:27.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.398+0000 7f5b77fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5b78065c00 0x7f5b780add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.398+0000 7f5b77fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5b78065c00 0x7f5b780add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51806/0 (socket says 172.21.15.121:51806) 2023-11-15T03:09:27.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.398+0000 7f5b77fff700 1 -- 172.21.15.121:0/933359081 learned_addr learned my addr 172.21.15.121:0/933359081 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:27.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.398+0000 7f5b777fe700 1 --1- 172.21.15.121:0/933359081 >> v1:172.21.15.121:6789/0 conn(0x7f5b780ab0d0 0x7f5b780ab4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45814/0 (socket says 172.21.15.121:45814) 2023-11-15T03:09:27.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.398+0000 7f5b76ffd700 1 --2- 172.21.15.121:0/933359081 >> v2:172.21.15.190:3300/0 conn(0x7f5b780ad1b0 0x7f5b78065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.398+0000 7f5b77fff700 1 -- 172.21.15.121:0/933359081 >> v2:172.21.15.190:3300/0 conn(0x7f5b780ad1b0 msgr2=0x7f5b78065710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:27.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.398+0000 7f5b77fff700 1 --2- 172.21.15.121:0/933359081 >> v2:172.21.15.190:3300/0 conn(0x7f5b780ad1b0 0x7f5b78065710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.398+0000 7f5b77fff700 1 -- 172.21.15.121:0/933359081 >> v1:172.21.15.121:6789/0 conn(0x7f5b780ab0d0 legacy=0x7f5b780ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:27.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.398+0000 7f5b77fff700 1 -- 172.21.15.121:0/933359081 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b78066640 con 0x7f5b78065c00 2023-11-15T03:09:27.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.398+0000 7f5b76ffd700 1 --2- 172.21.15.121:0/933359081 >> v2:172.21.15.190:3300/0 conn(0x7f5b780ad1b0 0x7f5b78065710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:27.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.398+0000 7f5b77fff700 1 --2- 172.21.15.121:0/933359081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b78065c00 0x7f5b780add40 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 rx=0x7f5b7806a180 tx=0x7f5b70009840).ready entity=mon.0 client_cookie=ea775d585b4907b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:27.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.402+0000 7f5b767fc700 1 -- 172.21.15.121:0/933359081 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7000e070 con 0x7f5b78065c00 2023-11-15T03:09:27.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.402+0000 7f5b767fc700 1 -- 172.21.15.121:0/933359081 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7000ba50 con 0x7f5b78065c00 2023-11-15T03:09:27.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.402+0000 7f5b767fc700 1 -- 172.21.15.121:0/933359081 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7000bdd0 con 0x7f5b78065c00 2023-11-15T03:09:27.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.402+0000 7f5b7cc8f700 1 -- 172.21.15.121:0/933359081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b78065c00 msgr2=0x7f5b780add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:27.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.402+0000 7f5b7cc8f700 1 --2- 172.21.15.121:0/933359081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b78065c00 0x7f5b780add40 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 rx=0x7f5b7806a180 tx=0x7f5b70009840).stop 2023-11-15T03:09:27.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.402+0000 7f5b7cc8f700 1 -- 172.21.15.121:0/933359081 shutdown_connections 2023-11-15T03:09:27.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.402+0000 7f5b7cc8f700 1 --2- 172.21.15.121:0/933359081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b78065c00 0x7f5b780add40 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.402+0000 7f5b7cc8f700 1 --2- 172.21.15.121:0/933359081 >> v2:172.21.15.190:3300/0 conn(0x7f5b780ad1b0 0x7f5b78065710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.402+0000 7f5b7cc8f700 1 -- 172.21.15.121:0/933359081 >> 172.21.15.121:0/933359081 conn(0x7f5b78067a60 msgr2=0x7f5b78067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:27.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.402+0000 7f5b7cc8f700 1 -- 172.21.15.121:0/933359081 shutdown_connections 2023-11-15T03:09:27.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.402+0000 7f5b7cc8f700 1 -- 172.21.15.121:0/933359081 wait complete. 2023-11-15T03:09:27.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.402+0000 7f5b7cc8f700 1 Processor -- start 2023-11-15T03:09:27.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b7cc8f700 1 -- start start 2023-11-15T03:09:27.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b7cc8f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b78065c00 0x7f5b78152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b7cc8f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b780ab0d0 0x7f5b780a82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b7cc8f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b780ad1b0 0x7f5b780a87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b7cc8f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b780a8c90 con 0x7f5b780ad1b0 2023-11-15T03:09:27.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b7cc8f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b780a66b0 con 0x7f5b780ab0d0 2023-11-15T03:09:27.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b7cc8f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b781643e0 con 0x7f5b78065c00 2023-11-15T03:09:27.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b77fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b780ad1b0 0x7f5b780a87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b77fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b780ad1b0 0x7f5b780a87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51816/0 (socket says 172.21.15.121:51816) 2023-11-15T03:09:27.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b77fff700 1 -- 172.21.15.121:0/1693474013 learned_addr learned my addr 172.21.15.121:0/1693474013 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:27.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b777fe700 1 --2- 172.21.15.121:0/1693474013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b78065c00 0x7f5b78152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b76ffd700 1 --2- 172.21.15.121:0/1693474013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b780ab0d0 0x7f5b780a82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b77fff700 1 -- 172.21.15.121:0/1693474013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b78065c00 msgr2=0x7f5b78152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:27.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b77fff700 1 --2- 172.21.15.121:0/1693474013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b78065c00 0x7f5b78152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b77fff700 1 -- 172.21.15.121:0/1693474013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b780ab0d0 msgr2=0x7f5b780a82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:27.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b77fff700 1 --2- 172.21.15.121:0/1693474013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b780ab0d0 0x7f5b780a82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:27.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b77fff700 1 -- 172.21.15.121:0/1693474013 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b70009ae0 con 0x7f5b780ad1b0 2023-11-15T03:09:27.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b77fff700 1 --2- 172.21.15.121:0/1693474013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b780ad1b0 0x7f5b780a87a0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 rx=0x7f5b70010000 tx=0x7f5b78066c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:27.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b5f7fe700 1 -- 172.21.15.121:0/1693474013 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7001c400 con 0x7f5b780ad1b0 2023-11-15T03:09:27.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b5f7fe700 1 -- 172.21.15.121:0/1693474013 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7001c560 con 0x7f5b780ad1b0 2023-11-15T03:09:27.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b7cc8f700 1 -- 172.21.15.121:0/1693474013 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b78152570 con 0x7f5b780ad1b0 2023-11-15T03:09:27.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b5f7fe700 1 -- 172.21.15.121:0/1693474013 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7001c8e0 con 0x7f5b780ad1b0 2023-11-15T03:09:27.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.406+0000 7f5b7cc8f700 1 -- 172.21.15.121:0/1693474013 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b780a63d0 con 0x7f5b780ad1b0 2023-11-15T03:09:27.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.410+0000 7f5b7cc8f700 1 -- 172.21.15.121:0/1693474013 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b78164550 con 0x7f5b780ad1b0 2023-11-15T03:09:27.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.410+0000 7f5b5f7fe700 1 -- 172.21.15.121:0/1693474013 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5b7001cd10 con 0x7f5b780ad1b0 2023-11-15T03:09:27.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.410+0000 7f5b5f7fe700 1 --2- 172.21.15.121:0/1693474013 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b6405a540 0x7f5b6405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:27.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.410+0000 7f5b5f7fe700 1 -- 172.21.15.121:0/1693474013 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8932+0+0 (secure 0 0 0) 0x7f5b70076ec0 con 0x7f5b780ad1b0 2023-11-15T03:09:27.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.414+0000 7f5b777fe700 1 --2- 172.21.15.121:0/1693474013 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b6405a540 0x7f5b6405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:27.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.414+0000 7f5b777fe700 1 --2- 172.21.15.121:0/1693474013 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b6405a540 0x7f5b6405ca00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f5b680026a0 tx=0x7f5b68009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:27.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.418+0000 7f5b5f7fe700 1 -- 172.21.15.121:0/1693474013 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5b70049ae0 con 0x7f5b780ad1b0 2023-11-15T03:09:27.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:27.614+0000 7f5b7cc8f700 1 -- 172.21.15.121:0/1693474013 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5b7806bcf0 con 0x7f5b780ad1b0 2023-11-15T03:09:28.236 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:09:28.226+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:09:28.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.238+0000 7f5b5f7fe700 1 -- 172.21.15.121:0/1693474013 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v281) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5b7004d100 con 0x7f5b780ad1b0 2023-11-15T03:09:28.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.242+0000 7f5b7cc8f700 1 -- 172.21.15.121:0/1693474013 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b6405a540 msgr2=0x7f5b6405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:28.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.242+0000 7f5b7cc8f700 1 --2- 172.21.15.121:0/1693474013 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b6405a540 0x7f5b6405ca00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f5b680026a0 tx=0x7f5b68009040).stop 2023-11-15T03:09:28.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.242+0000 7f5b7cc8f700 1 -- 172.21.15.121:0/1693474013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b780ad1b0 msgr2=0x7f5b780a87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:28.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.242+0000 7f5b7cc8f700 1 --2- 172.21.15.121:0/1693474013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b780ad1b0 0x7f5b780a87a0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 rx=0x7f5b70010000 tx=0x7f5b78066c00).stop 2023-11-15T03:09:28.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.242+0000 7f5b7cc8f700 1 -- 172.21.15.121:0/1693474013 shutdown_connections 2023-11-15T03:09:28.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.242+0000 7f5b7cc8f700 1 --2- 172.21.15.121:0/1693474013 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b6405a540 0x7f5b6405ca00 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:28.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.242+0000 7f5b7cc8f700 1 --2- 172.21.15.121:0/1693474013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b780ad1b0 0x7f5b780a87a0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:28.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.242+0000 7f5b7cc8f700 1 --2- 172.21.15.121:0/1693474013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b780ab0d0 0x7f5b780a82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:28.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.242+0000 7f5b7cc8f700 1 --2- 172.21.15.121:0/1693474013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b78065c00 0x7f5b78152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:28.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.242+0000 7f5b7cc8f700 1 -- 172.21.15.121:0/1693474013 >> 172.21.15.121:0/1693474013 conn(0x7f5b78067a60 msgr2=0x7f5b781568c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:28.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.242+0000 7f5b7cc8f700 1 -- 172.21.15.121:0/1693474013 shutdown_connections 2023-11-15T03:09:28.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.242+0000 7f5b7cc8f700 1 -- 172.21.15.121:0/1693474013 wait complete. 2023-11-15T03:09:28.255 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:09:28.276 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:09:28.298 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:09:28.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.426+0000 7ff28d1f9700 1 Processor -- start 2023-11-15T03:09:28.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.426+0000 7ff28d1f9700 1 -- start start 2023-11-15T03:09:28.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff28d1f9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff2880ab0d0 0x7ff2880ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:28.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff28d1f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2880add40 0x7ff2880ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:28.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff28d1f9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff288066c90 con 0x7ff2880ad1b0 2023-11-15T03:09:28.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff28d1f9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff288065e20 con 0x7ff2880ab0d0 2023-11-15T03:09:28.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff28d1f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff288066080 con 0x7ff2880add40 2023-11-15T03:09:28.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff287fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff2880ab0d0 0x7ff2880ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:28.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff2877fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff2880ad1b0 0x7ff288065710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45822/0 (socket says 172.21.15.121:45822) 2023-11-15T03:09:28.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff2877fe700 1 -- 172.21.15.121:0/3675772413 learned_addr learned my addr 172.21.15.121:0/3675772413 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:09:28.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff28c9f8700 1 --2- 172.21.15.121:0/3675772413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2880add40 0x7ff2880ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:28.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff287fff700 1 -- 172.21.15.121:0/3675772413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2880add40 msgr2=0x7ff2880ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:28.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff287fff700 1 --2- 172.21.15.121:0/3675772413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2880add40 0x7ff2880ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:28.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff287fff700 1 -- 172.21.15.121:0/3675772413 >> v1:172.21.15.121:6789/0 conn(0x7ff2880ad1b0 legacy=0x7ff288065710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:28.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff287fff700 1 -- 172.21.15.121:0/3675772413 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff288066360 con 0x7ff2880ab0d0 2023-11-15T03:09:28.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff287fff700 1 --2- 172.21.15.121:0/3675772413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2880ab0d0 0x7ff2880ab4b0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 rx=0x7ff278006fd0 tx=0x7ff278006f40).ready entity=mon.0 client_cookie=fa1f0432ac212271 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:28.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff286ffd700 1 -- 172.21.15.121:0/3675772413 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278005880 con 0x7ff2880ab0d0 2023-11-15T03:09:28.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff286ffd700 1 -- 172.21.15.121:0/3675772413 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2780059e0 con 0x7ff2880ab0d0 2023-11-15T03:09:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff286ffd700 1 -- 172.21.15.121:0/3675772413 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278005d60 con 0x7ff2880ab0d0 2023-11-15T03:09:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff28d1f9700 1 -- 172.21.15.121:0/3675772413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2880ab0d0 msgr2=0x7ff2880ab4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff28d1f9700 1 --2- 172.21.15.121:0/3675772413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2880ab0d0 0x7ff2880ab4b0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 rx=0x7ff278006fd0 tx=0x7ff278006f40).stop 2023-11-15T03:09:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff28d1f9700 1 -- 172.21.15.121:0/3675772413 shutdown_connections 2023-11-15T03:09:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff28d1f9700 1 --2- 172.21.15.121:0/3675772413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2880add40 0x7ff2880ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:28.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff28d1f9700 1 --2- 172.21.15.121:0/3675772413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2880ab0d0 0x7ff2880ab4b0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:28.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff28d1f9700 1 -- 172.21.15.121:0/3675772413 >> 172.21.15.121:0/3675772413 conn(0x7ff288067a60 msgr2=0x7ff288067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:28.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.430+0000 7ff28d1f9700 1 -- 172.21.15.121:0/3675772413 shutdown_connections 2023-11-15T03:09:28.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.434+0000 7ff28d1f9700 1 -- 172.21.15.121:0/3675772413 wait complete. 2023-11-15T03:09:28.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.434+0000 7ff28d1f9700 1 Processor -- start 2023-11-15T03:09:28.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.434+0000 7ff28d1f9700 1 -- start start 2023-11-15T03:09:28.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff28d1f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2880ab0d0 0x7ff288141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:28.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff28d1f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2880ad1b0 0x7ff2880a8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:28.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff28d1f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2880add40 0x7ff2880a90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:28.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff28d1f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2880a95e0 con 0x7ff2880add40 2023-11-15T03:09:28.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff28d1f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2880a8770 con 0x7ff2880ad1b0 2023-11-15T03:09:28.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff28d1f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2880a7b60 con 0x7ff2880ab0d0 2023-11-15T03:09:28.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff28c9f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2880add40 0x7ff2880a90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:28.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff28c9f8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2880add40 0x7ff2880a90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51858/0 (socket says 172.21.15.121:51858) 2023-11-15T03:09:28.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff28c9f8700 1 -- 172.21.15.121:0/2948958648 learned_addr learned my addr 172.21.15.121:0/2948958648 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:28.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff287fff700 1 --2- 172.21.15.121:0/2948958648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2880ab0d0 0x7ff288141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:28.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff2877fe700 1 --2- 172.21.15.121:0/2948958648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2880ad1b0 0x7ff2880a8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:28.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff28c9f8700 1 -- 172.21.15.121:0/2948958648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2880ab0d0 msgr2=0x7ff288141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:28.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff28c9f8700 1 --2- 172.21.15.121:0/2948958648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2880ab0d0 0x7ff288141cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:28.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff28c9f8700 1 -- 172.21.15.121:0/2948958648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2880ad1b0 msgr2=0x7ff2880a8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:28.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff28c9f8700 1 --2- 172.21.15.121:0/2948958648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2880ad1b0 0x7ff2880a8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:28.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff28c9f8700 1 -- 172.21.15.121:0/2948958648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff278009250 con 0x7ff2880add40 2023-11-15T03:09:28.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff2877fe700 1 --2- 172.21.15.121:0/2948958648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2880ad1b0 0x7ff2880a8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:28.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff28c9f8700 1 --2- 172.21.15.121:0/2948958648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2880add40 0x7ff2880a90f0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 rx=0x7ff280009fa0 tx=0x7ff280009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:28.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff26ffff700 1 -- 172.21.15.121:0/2948958648 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28000ed20 con 0x7ff2880add40 2023-11-15T03:09:28.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff26ffff700 1 -- 172.21.15.121:0/2948958648 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28000ee80 con 0x7ff2880add40 2023-11-15T03:09:28.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff28d1f9700 1 -- 172.21.15.121:0/2948958648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2880a7e40 con 0x7ff2880add40 2023-11-15T03:09:28.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff26ffff700 1 -- 172.21.15.121:0/2948958648 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280014610 con 0x7ff2880add40 2023-11-15T03:09:28.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.438+0000 7ff28d1f9700 1 -- 172.21.15.121:0/2948958648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2880a8400 con 0x7ff2880add40 2023-11-15T03:09:28.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.442+0000 7ff28d1f9700 1 -- 172.21.15.121:0/2948958648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff28806bcf0 con 0x7ff2880add40 2023-11-15T03:09:28.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.442+0000 7ff26ffff700 1 -- 172.21.15.121:0/2948958648 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff28001d3b0 con 0x7ff2880add40 2023-11-15T03:09:28.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.442+0000 7ff26ffff700 1 --2- 172.21.15.121:0/2948958648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff27405a510 0x7ff27405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:28.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.442+0000 7ff26ffff700 1 -- 172.21.15.121:0/2948958648 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9018+0+0 (secure 0 0 0) 0x7ff280075a00 con 0x7ff2880add40 2023-11-15T03:09:28.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.442+0000 7ff287fff700 1 --2- 172.21.15.121:0/2948958648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff27405a510 0x7ff27405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:28.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.442+0000 7ff287fff700 1 --2- 172.21.15.121:0/2948958648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff27405a510 0x7ff27405c9d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7ff278006f80 tx=0x7ff278016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:28.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.450+0000 7ff26ffff700 1 -- 172.21.15.121:0/2948958648 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff280048570 con 0x7ff2880add40 2023-11-15T03:09:28.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:28.674+0000 7ff28d1f9700 1 -- 172.21.15.121:0/2948958648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7ff288068a70 con 0x7ff2880add40 2023-11-15T03:09:29.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.246+0000 7ff26ffff700 1 -- 172.21.15.121:0/2948958648 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v282) v1 ==== 107+0+0 (secure 0 0 0) 0x7ff28004bb90 con 0x7ff2880add40 2023-11-15T03:09:29.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.246+0000 7ff28d1f9700 1 -- 172.21.15.121:0/2948958648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff27405a510 msgr2=0x7ff27405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.246+0000 7ff28d1f9700 1 --2- 172.21.15.121:0/2948958648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff27405a510 0x7ff27405c9d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7ff278006f80 tx=0x7ff278016000).stop 2023-11-15T03:09:29.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.246+0000 7ff28d1f9700 1 -- 172.21.15.121:0/2948958648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2880add40 msgr2=0x7ff2880a90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.246+0000 7ff28d1f9700 1 --2- 172.21.15.121:0/2948958648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2880add40 0x7ff2880a90f0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 rx=0x7ff280009fa0 tx=0x7ff280009900).stop 2023-11-15T03:09:29.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.246+0000 7ff28d1f9700 1 -- 172.21.15.121:0/2948958648 shutdown_connections 2023-11-15T03:09:29.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.250+0000 7ff28d1f9700 1 --2- 172.21.15.121:0/2948958648 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff27405a510 0x7ff27405c9d0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.250+0000 7ff28d1f9700 1 --2- 172.21.15.121:0/2948958648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2880add40 0x7ff2880a90f0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.250+0000 7ff28d1f9700 1 --2- 172.21.15.121:0/2948958648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2880ad1b0 0x7ff2880a8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.250+0000 7ff28d1f9700 1 --2- 172.21.15.121:0/2948958648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2880ab0d0 0x7ff288141cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.250+0000 7ff28d1f9700 1 -- 172.21.15.121:0/2948958648 >> 172.21.15.121:0/2948958648 conn(0x7ff288067a60 msgr2=0x7ff2880b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:29.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.250+0000 7ff28d1f9700 1 -- 172.21.15.121:0/2948958648 shutdown_connections 2023-11-15T03:09:29.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.250+0000 7ff28d1f9700 1 -- 172.21.15.121:0/2948958648 wait complete. 2023-11-15T03:09:29.261 INFO:teuthology.orchestra.run.smithi121.stderr:nothing to do: rank is not damaged 2023-11-15T03:09:29.280 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:09:29.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.378+0000 7f3f7693fd00 1 Processor -- start 2023-11-15T03:09:29.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.378+0000 7f3f7693fd00 1 -- start start 2023-11-15T03:09:29.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.378+0000 7f3f7693fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b80616de0 0x557b80642cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.378+0000 7f3f7693fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b80648420 0x557b806e48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.378+0000 7f3f7693fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b806ec9f0 0x557b806e4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.378+0000 7f3f7693fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b8058f220 con 0x557b80616de0 2023-11-15T03:09:29.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.378+0000 7f3f7693fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b80645b80 con 0x557b80648420 2023-11-15T03:09:29.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.378+0000 7f3f7693fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b806e52c0 con 0x557b806ec9f0 2023-11-15T03:09:29.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.378+0000 7f3f6af85700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b80616de0 0x557b80642cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.378+0000 7f3f6af85700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b80616de0 0x557b80642cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51878/0 (socket says 172.21.15.121:51878) 2023-11-15T03:09:29.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.378+0000 7f3f6af85700 1 -- 172.21.15.121:0/169937138 learned_addr learned my addr 172.21.15.121:0/169937138 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:29.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.378+0000 7f3f6a784700 1 --2- 172.21.15.121:0/169937138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b80648420 0x557b806e48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.378+0000 7f3f6b786700 1 --2- 172.21.15.121:0/169937138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b806ec9f0 0x557b806e4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f6af85700 1 -- 172.21.15.121:0/169937138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b806ec9f0 msgr2=0x557b806e4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:29.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f6af85700 1 --2- 172.21.15.121:0/169937138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b806ec9f0 0x557b806e4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f6af85700 1 -- 172.21.15.121:0/169937138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b80648420 msgr2=0x557b806e48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:29.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f6af85700 1 --2- 172.21.15.121:0/169937138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b80648420 0x557b806e48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f6af85700 1 -- 172.21.15.121:0/169937138 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6400a0e0 con 0x557b80616de0 2023-11-15T03:09:29.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f6a784700 1 --2- 172.21.15.121:0/169937138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b80648420 0x557b806e48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:29.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f6b786700 1 --2- 172.21.15.121:0/169937138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b806ec9f0 0x557b806e4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:29.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f6af85700 1 --2- 172.21.15.121:0/169937138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b80616de0 0x557b80642cd0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 rx=0x7f3f5c00b2f0 tx=0x7f3f5c001790).ready entity=mon.0 client_cookie=8b3d38c193bfc6cd server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:29.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f69782700 1 -- 172.21.15.121:0/169937138 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c00be00 con 0x557b80616de0 2023-11-15T03:09:29.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f69782700 1 -- 172.21.15.121:0/169937138 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5c00e470 con 0x557b80616de0 2023-11-15T03:09:29.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f69782700 1 -- 172.21.15.121:0/169937138 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c00e810 con 0x557b80616de0 2023-11-15T03:09:29.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f7693fd00 1 -- 172.21.15.121:0/169937138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b80616de0 msgr2=0x557b80642cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/169937138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b80616de0 0x557b80642cd0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 rx=0x7f3f5c00b2f0 tx=0x7f3f5c001790).stop 2023-11-15T03:09:29.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f7693fd00 1 -- 172.21.15.121:0/169937138 shutdown_connections 2023-11-15T03:09:29.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/169937138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b806ec9f0 0x557b806e4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/169937138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b80648420 0x557b806e48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/169937138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b80616de0 0x557b80642cd0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f7693fd00 1 -- 172.21.15.121:0/169937138 >> 172.21.15.121:0/169937138 conn(0x557b806177d0 msgr2=0x557b80619210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:29.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f7693fd00 1 -- 172.21.15.121:0/169937138 shutdown_connections 2023-11-15T03:09:29.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f7693fd00 1 -- 172.21.15.121:0/169937138 wait complete. 2023-11-15T03:09:29.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f7693fd00 1 Processor -- start 2023-11-15T03:09:29.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.382+0000 7f3f7693fd00 1 -- start start 2023-11-15T03:09:29.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f7693fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b80616de0 0x557b806e8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f7693fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b80648420 0x557b806eabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f7693fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b806ec9f0 0x557b8065e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f7693fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b806eb2b0 con 0x557b80616de0 2023-11-15T03:09:29.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f7693fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b806e8800 con 0x557b806ec9f0 2023-11-15T03:09:29.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f7693fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b806eb0a0 con 0x557b80648420 2023-11-15T03:09:29.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f6af85700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b80616de0 0x557b806e8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f6af85700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b80616de0 0x557b806e8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51890/0 (socket says 172.21.15.121:51890) 2023-11-15T03:09:29.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 learned_addr learned my addr 172.21.15.121:0/714934501 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:29.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f6b786700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b806ec9f0 0x557b8065e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f6a784700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b80648420 0x557b806eabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b80648420 msgr2=0x557b806eabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f6af85700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b80648420 0x557b806eabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b806ec9f0 msgr2=0x557b8065e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f6af85700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b806ec9f0 0x557b8065e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c001a70 con 0x557b80616de0 2023-11-15T03:09:29.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f6b786700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b806ec9f0 0x557b8065e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:09:29.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f6af85700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b80616de0 0x557b806e8310 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 rx=0x7f3f5c00ba40 tx=0x7f3f5c001790).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:29.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f5b7fe700 1 -- 172.21.15.121:0/714934501 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c00d920 con 0x557b80616de0 2023-11-15T03:09:29.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f5b7fe700 1 -- 172.21.15.121:0/714934501 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5c00da80 con 0x557b80616de0 2023-11-15T03:09:29.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557b805b8770 con 0x557b80616de0 2023-11-15T03:09:29.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f5b7fe700 1 -- 172.21.15.121:0/714934501 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c00de40 con 0x557b80616de0 2023-11-15T03:09:29.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.386+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b80587a70 con 0x557b80616de0 2023-11-15T03:09:29.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.390+0000 7f3f5b7fe700 1 -- 172.21.15.121:0/714934501 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3f5c01a640 con 0x557b80616de0 2023-11-15T03:09:29.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.390+0000 7f3f5b7fe700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3f50059020 0x7f3f5005b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.390+0000 7f3f5b7fe700 1 -- 172.21.15.121:0/714934501 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9018+0+0 (secure 0 0 0) 0x7f3f5c076cb0 con 0x557b80616de0 2023-11-15T03:09:29.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.390+0000 7f3f6a784700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3f50059020 0x7f3f5005b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.406 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:09:29.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.390+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x557b806e2990 0x557b806e67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.390+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b806e6cc0 con 0x557b806e2990 2023-11-15T03:09:29.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.394+0000 7f3f6a784700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3f50059020 0x7f3f5005b4e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x557b806425a0 tx=0x7f3f60005a50).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:29.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.394+0000 7f3f6b786700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x557b806e2990 0x557b806e67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.394+0000 7f3f6b786700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x557b806e2990 0x557b806e67d0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:29.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.394+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v126'377 uv370 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b806e2990 2023-11-15T03:09:29.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.394+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b806e7070 con 0x557b806e2990 2023-11-15T03:09:29.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.398+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v126'378 uv378 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b806e2990 2023-11-15T03:09:29.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.398+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b806e7450 con 0x557b806e2990 2023-11-15T03:09:29.411 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:09:29.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.402+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v126'379 uv379 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b806e2990 2023-11-15T03:09:29.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.402+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b806e7800 con 0x557b806e2990 2023-11-15T03:09:29.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.402+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v126'380 uv380 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b806e2990 2023-11-15T03:09:29.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.402+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8070ce40 con 0x557b806e2990 2023-11-15T03:09:29.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.406+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v126'381 uv381 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b806e2990 2023-11-15T03:09:29.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.406+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8070be30 con 0x557b806e2990 2023-11-15T03:09:29.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.410+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v126'382 uv382 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b806e2990 2023-11-15T03:09:29.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.410+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8070c1e0 con 0x557b806e2990 2023-11-15T03:09:29.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.410+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v126'383 uv376 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b806e2990 2023-11-15T03:09:29.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.410+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b8070c590 con 0x557b806e2990 2023-11-15T03:09:29.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.414+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v126'384 uv384 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b806e2990 2023-11-15T03:09:29.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.414+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8070c590 con 0x557b806e2990 2023-11-15T03:09:29.425 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:09:29.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.414+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v126'385 uv385 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b806e2990 2023-11-15T03:09:29.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.414+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8070c590 con 0x557b806e2990 2023-11-15T03:09:29.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.418+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v126'386 uv386 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b806e2990 2023-11-15T03:09:29.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.418+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8070c590 con 0x557b806e2990 2023-11-15T03:09:29.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.422+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v126'387 uv387 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b806e2990 2023-11-15T03:09:29.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.422+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8070c990 con 0x557b806e2990 2023-11-15T03:09:29.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.422+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v126'388 uv388 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b806e2990 2023-11-15T03:09:29.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.422+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x557b8070d700 0x557b8070faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.422+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8070c990 con 0x557b8070d700 2023-11-15T03:09:29.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.422+0000 7f3f6af85700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x557b8070d700 0x557b8070faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.426+0000 7f3f6af85700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x557b8070d700 0x557b8070faf0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:29.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.430+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v126'595 uv594 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.430+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b8070ffe0 con 0x557b8070d700 2023-11-15T03:09:29.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.434+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v126'596 uv596 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.434+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80710390 con 0x557b8070d700 2023-11-15T03:09:29.446 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:09:29.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.434+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v126'597 uv597 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.434+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80711750 con 0x557b8070d700 2023-11-15T03:09:29.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.438+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v126'598 uv598 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.438+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80711b40 con 0x557b8070d700 2023-11-15T03:09:29.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.442+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v126'599 uv599 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.442+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80710740 con 0x557b8070d700 2023-11-15T03:09:29.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.442+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v126'600 uv600 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.442+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80711090 con 0x557b8070d700 2023-11-15T03:09:29.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.446+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v126'601 uv601 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.446+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80711f60 con 0x557b8070d700 2023-11-15T03:09:29.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.450+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v126'602 uv587 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.450+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b80711f60 con 0x557b8070d700 2023-11-15T03:09:29.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.454+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v126'603 uv603 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.454+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80711f60 con 0x557b8070d700 2023-11-15T03:09:29.466 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:09:29.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.458+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v126'604 uv604 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.458+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80712f70 con 0x557b8070d700 2023-11-15T03:09:29.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.458+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v126'605 uv605 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.458+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80712f70 con 0x557b8070d700 2023-11-15T03:09:29.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.462+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v126'606 uv606 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.462+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80711f60 con 0x557b8070d700 2023-11-15T03:09:29.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.466+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v126'607 uv607 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.466+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80711f60 con 0x557b8070d700 2023-11-15T03:09:29.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.466+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v126'608 uv593 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.466+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b80711f60 con 0x557b8070d700 2023-11-15T03:09:29.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.470+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v126'609 uv609 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.470+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80711f60 con 0x557b8070d700 2023-11-15T03:09:29.483 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:09:29.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.474+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v126'610 uv610 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.474+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80712f70 con 0x557b8070d700 2023-11-15T03:09:29.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.474+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v126'611 uv611 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.474+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80712f70 con 0x557b8070d700 2023-11-15T03:09:29.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.478+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v126'612 uv612 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.478+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80711f60 con 0x557b8070d700 2023-11-15T03:09:29.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.482+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v126'613 uv613 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.482+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x557b80712470 0x557b80714860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.482+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80714d50 con 0x557b80712470 2023-11-15T03:09:29.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.482+0000 7f3f6a784700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x557b80712470 0x557b80714860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.482+0000 7f3f6a784700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x557b80712470 0x557b80714860 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:29.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.486+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v126'325 uv318 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b80712470 2023-11-15T03:09:29.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.486+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b80715200 con 0x557b80712470 2023-11-15T03:09:29.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.490+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v126'326 uv326 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b80712470 2023-11-15T03:09:29.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.490+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807156f0 con 0x557b80712470 2023-11-15T03:09:29.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.490+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v126'327 uv327 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b80712470 2023-11-15T03:09:29.502 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:09:29.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.490+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80715fe0 con 0x557b80712470 2023-11-15T03:09:29.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.494+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v126'328 uv328 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b80712470 2023-11-15T03:09:29.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.494+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80716390 con 0x557b80712470 2023-11-15T03:09:29.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.498+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v126'329 uv329 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b80712470 2023-11-15T03:09:29.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.498+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80716740 con 0x557b80712470 2023-11-15T03:09:29.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.502+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v126'330 uv324 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b80712470 2023-11-15T03:09:29.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.502+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b80716af0 con 0x557b80712470 2023-11-15T03:09:29.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.502+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v126'331 uv331 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b80712470 2023-11-15T03:09:29.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.502+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80716ea0 con 0x557b80712470 2023-11-15T03:09:29.517 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:09:29.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.506+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v126'332 uv332 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b80712470 2023-11-15T03:09:29.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.506+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80716ea0 con 0x557b80712470 2023-11-15T03:09:29.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.510+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v126'333 uv333 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b80712470 2023-11-15T03:09:29.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.510+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80715b70 con 0x557b80712470 2023-11-15T03:09:29.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.514+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v126'334 uv334 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b80712470 2023-11-15T03:09:29.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.514+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x557b80715b70 0x557b807198b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.514+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80719e70 con 0x557b80715b70 2023-11-15T03:09:29.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.514+0000 7f3f6b786700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x557b80715b70 0x557b807198b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.514+0000 7f3f6b786700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x557b80715b70 0x557b807198b0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:29.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.518+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v126'380 uv379 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.518+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b80719f30 con 0x557b80715b70 2023-11-15T03:09:29.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.522+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v126'381 uv381 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.522+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8071a360 con 0x557b80715b70 2023-11-15T03:09:29.535 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:09:29.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.526+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v126'382 uv382 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.526+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8071e3f0 con 0x557b80715b70 2023-11-15T03:09:29.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.530+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v126'383 uv383 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.530+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8071e3f0 con 0x557b80715b70 2023-11-15T03:09:29.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.530+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v126'384 uv384 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.530+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8071a360 con 0x557b80715b70 2023-11-15T03:09:29.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.534+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v126'385 uv376 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.534+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b8071a360 con 0x557b80715b70 2023-11-15T03:09:29.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.538+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v126'386 uv386 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.538+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8071a360 con 0x557b80715b70 2023-11-15T03:09:29.549 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:09:29.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.538+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v126'387 uv387 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.538+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8071b370 con 0x557b80715b70 2023-11-15T03:09:29.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.542+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v126'388 uv388 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.542+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8071b370 con 0x557b80715b70 2023-11-15T03:09:29.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.546+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v126'389 uv389 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.546+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8071a360 con 0x557b80715b70 2023-11-15T03:09:29.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.550+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v126'390 uv390 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.550+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x557b8071a870 0x557b8071cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.550+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8071d280 con 0x557b8071a870 2023-11-15T03:09:29.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.550+0000 7f3f6af85700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x557b8071a870 0x557b8071cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.550+0000 7f3f6af85700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x557b8071a870 0x557b8071cc60 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:29.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.554+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v126'189 uv188 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8071a870 2023-11-15T03:09:29.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.554+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b8071d660 con 0x557b8071a870 2023-11-15T03:09:29.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.554+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v126'190 uv190 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8071a870 2023-11-15T03:09:29.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.554+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8071d660 con 0x557b8071a870 2023-11-15T03:09:29.568 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:09:29.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.558+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v126'191 uv191 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8071a870 2023-11-15T03:09:29.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.558+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8071e670 con 0x557b8071a870 2023-11-15T03:09:29.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.562+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v126'192 uv192 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8071a870 2023-11-15T03:09:29.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.562+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8071e670 con 0x557b8071a870 2023-11-15T03:09:29.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.562+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v126'193 uv193 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8071a870 2023-11-15T03:09:29.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.562+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b8071d660 con 0x557b8071a870 2023-11-15T03:09:29.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.566+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v126'194 uv194 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8071a870 2023-11-15T03:09:29.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.566+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x557b8071dce0 0x557b807200d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.566+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807207f0 con 0x557b8071dce0 2023-11-15T03:09:29.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.566+0000 7f3f6a784700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x557b8071dce0 0x557b807200d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.570+0000 7f3f6a784700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x557b8071dce0 0x557b807200d0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:29.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.570+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v126'136 uv135 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.570+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b80720bd0 con 0x557b8071dce0 2023-11-15T03:09:29.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.574+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v126'137 uv137 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.574+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80720bd0 con 0x557b8071dce0 2023-11-15T03:09:29.586 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:09:29.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.578+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v126'138 uv138 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.578+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80720bd0 con 0x557b8071dce0 2023-11-15T03:09:29.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.578+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v126'139 uv139 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.578+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80720bd0 con 0x557b8071dce0 2023-11-15T03:09:29.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.582+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v126'189 uv188 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.582+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b80720bd0 con 0x557b8071dce0 2023-11-15T03:09:29.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.586+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v126'190 uv190 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.586+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80720bd0 con 0x557b8071dce0 2023-11-15T03:09:29.597 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:09:29.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.586+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v126'191 uv191 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.586+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80721be0 con 0x557b8071dce0 2023-11-15T03:09:29.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.590+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v126'192 uv192 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.590+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80721be0 con 0x557b8071dce0 2023-11-15T03:09:29.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.594+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v126'193 uv193 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.594+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80720bd0 con 0x557b8071dce0 2023-11-15T03:09:29.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.594+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v126'194 uv194 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.594+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80720bd0 con 0x557b8070d700 2023-11-15T03:09:29.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.598+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v126'349 uv343 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.598+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b80720bd0 con 0x557b8070d700 2023-11-15T03:09:29.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.602+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v126'350 uv350 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.602+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80720bd0 con 0x557b8070d700 2023-11-15T03:09:29.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.606+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v126'351 uv351 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.617 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:09:29.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.606+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80721be0 con 0x557b8070d700 2023-11-15T03:09:29.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.610+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v126'352 uv352 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.610+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80721be0 con 0x557b8070d700 2023-11-15T03:09:29.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.614+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v126'353 uv353 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.614+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80720bd0 con 0x557b8070d700 2023-11-15T03:09:29.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.618+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v126'354 uv354 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.618+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80720bd0 con 0x557b8070d700 2023-11-15T03:09:29.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.622+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v126'355 uv348 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.622+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b80720bd0 con 0x557b8070d700 2023-11-15T03:09:29.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.626+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v126'356 uv356 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.626+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80720bd0 con 0x557b8070d700 2023-11-15T03:09:29.637 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:09:29.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.626+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v126'357 uv357 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.626+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80720bd0 con 0x557b8070d700 2023-11-15T03:09:29.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.630+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v126'358 uv358 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.630+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80716f20 con 0x557b8070d700 2023-11-15T03:09:29.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.634+0000 7f3f6af85700 1 -- 172.21.15.121:0/714934501 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v126'359 uv359 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f5c04b8e0 con 0x557b8070d700 2023-11-15T03:09:29.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.634+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80716f20 con 0x557b80712470 2023-11-15T03:09:29.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.638+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v126'158 uv157 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b80712470 2023-11-15T03:09:29.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.638+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b80716f20 con 0x557b80712470 2023-11-15T03:09:29.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.638+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v126'159 uv159 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b80712470 2023-11-15T03:09:29.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.638+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80716f20 con 0x557b80712470 2023-11-15T03:09:29.652 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:09:29.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.642+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v126'160 uv160 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b80712470 2023-11-15T03:09:29.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.642+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80716f20 con 0x557b80712470 2023-11-15T03:09:29.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.646+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v126'161 uv161 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b80712470 2023-11-15T03:09:29.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.646+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80716f60 con 0x557b80712470 2023-11-15T03:09:29.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.646+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v126'162 uv162 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b80712470 2023-11-15T03:09:29.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.646+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80716f20 con 0x557b8071dce0 2023-11-15T03:09:29.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.650+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v126'189 uv188 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.650+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b80716f20 con 0x557b8071dce0 2023-11-15T03:09:29.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.654+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v126'190 uv190 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.654+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80716f20 con 0x557b8071dce0 2023-11-15T03:09:29.668 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:09:29.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.658+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v126'191 uv191 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.658+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80716f20 con 0x557b8071dce0 2023-11-15T03:09:29.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.658+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v126'192 uv192 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.658+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80716f20 con 0x557b8071dce0 2023-11-15T03:09:29.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.662+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v126'193 uv193 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.662+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80716f20 con 0x557b8071dce0 2023-11-15T03:09:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.666+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v126'194 uv194 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.666+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80716f20 con 0x557b8071dce0 2023-11-15T03:09:29.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.670+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v126'159 uv158 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.670+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b80716f20 con 0x557b8071dce0 2023-11-15T03:09:29.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.670+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v126'160 uv160 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.670+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80716f20 con 0x557b8071dce0 2023-11-15T03:09:29.684 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:09:29.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.674+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v126'161 uv161 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.674+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80716f20 con 0x557b8071dce0 2023-11-15T03:09:29.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.678+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v126'162 uv162 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.678+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807210c0 con 0x557b8071dce0 2023-11-15T03:09:29.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.682+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v126'163 uv163 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.682+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807210c0 con 0x557b8071dce0 2023-11-15T03:09:29.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.682+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v126'156 uv155 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.682+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b807210c0 con 0x557b8071dce0 2023-11-15T03:09:29.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.686+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v126'157 uv157 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.686+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807210c0 con 0x557b8071dce0 2023-11-15T03:09:29.699 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:09:29.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.690+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v126'158 uv158 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.690+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807210c0 con 0x557b8071dce0 2023-11-15T03:09:29.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.694+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v126'159 uv159 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.694+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80723ae0 con 0x557b8071dce0 2023-11-15T03:09:29.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.694+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v126'160 uv160 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.694+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x557b80723cc0 0x557b80724100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.694+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b80723cc0 2023-11-15T03:09:29.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.698+0000 7f3f6b786700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x557b80723cc0 0x557b80724100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.698+0000 7f3f6b786700 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x557b80723cc0 0x557b80724100 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:29.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.702+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v126'126 uv125 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80723cc0 2023-11-15T03:09:29.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.702+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b80723cc0 2023-11-15T03:09:29.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.702+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v126'127 uv127 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80723cc0 2023-11-15T03:09:29.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.702+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b80723cc0 2023-11-15T03:09:29.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.706+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v126'128 uv128 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80723cc0 2023-11-15T03:09:29.717 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:09:29.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.706+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b80723cc0 2023-11-15T03:09:29.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.710+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v126'129 uv129 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80723cc0 2023-11-15T03:09:29.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.710+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b8071dce0 2023-11-15T03:09:29.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.714+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v126'189 uv188 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.714+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b8071dce0 2023-11-15T03:09:29.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.718+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v126'190 uv190 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.718+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b8071dce0 2023-11-15T03:09:29.729 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:09:29.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.718+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v126'191 uv191 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.718+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80725600 con 0x557b8071dce0 2023-11-15T03:09:29.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.722+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v126'192 uv192 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.722+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80725600 con 0x557b8071dce0 2023-11-15T03:09:29.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.726+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v126'193 uv193 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.726+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b8071dce0 2023-11-15T03:09:29.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.726+0000 7f3f6a784700 1 -- 172.21.15.121:0/714934501 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v126'194 uv194 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f60007c30 con 0x557b8071dce0 2023-11-15T03:09:29.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.726+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b80715b70 2023-11-15T03:09:29.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.730+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v126'161 uv160 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.730+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b80715b70 2023-11-15T03:09:29.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.734+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v126'162 uv162 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.734+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b80715b70 2023-11-15T03:09:29.745 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:09:29.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.734+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v126'163 uv163 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.734+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b80715b70 2023-11-15T03:09:29.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.738+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v126'164 uv164 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.738+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b80715b70 2023-11-15T03:09:29.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.742+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v126'165 uv165 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.742+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b80715b70 2023-11-15T03:09:29.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.742+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v126'378 uv370 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.742+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b80715b70 2023-11-15T03:09:29.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.746+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v126'379 uv379 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.746+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b80715b70 2023-11-15T03:09:29.760 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:09:29.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.750+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v126'380 uv380 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.750+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b80715b70 2023-11-15T03:09:29.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.754+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v126'381 uv381 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.754+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b80715b70 2023-11-15T03:09:29.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.754+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v126'382 uv382 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.754+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b80715b70 2023-11-15T03:09:29.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.758+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v126'383 uv377 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.758+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b80715b70 2023-11-15T03:09:29.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.762+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v126'384 uv384 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.762+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b80715b70 2023-11-15T03:09:29.774 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:09:29.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.762+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v126'385 uv385 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.762+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80725600 con 0x557b80715b70 2023-11-15T03:09:29.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.766+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v126'386 uv386 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.766+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80725600 con 0x557b80715b70 2023-11-15T03:09:29.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.770+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v126'387 uv387 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.770+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b807245f0 con 0x557b80715b70 2023-11-15T03:09:29.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.774+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v126'388 uv388 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.774+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x557b80724ae0 con 0x557b80715b70 2023-11-15T03:09:29.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.774+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v126'389 uv389 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3f64017040 con 0x557b80715b70 2023-11-15T03:09:29.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.774+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x557b80712470 msgr2=0x557b80714860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.774+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x557b80712470 0x557b80714860 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.774+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x557b806e2990 msgr2=0x557b806e67d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.774+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x557b806e2990 0x557b806e67d0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.774+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x557b8071a870 msgr2=0x557b8071cc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.774+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x557b8071a870 0x557b8071cc60 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.774+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x557b8070d700 msgr2=0x557b8070faf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.774+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x557b8070d700 0x557b8070faf0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.774+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x557b80723cc0 msgr2=0x557b80724100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.774+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x557b80723cc0 0x557b80724100 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.774+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x557b80715b70 msgr2=0x557b807198b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.774+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x557b80715b70 0x557b807198b0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.774+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x557b8071dce0 msgr2=0x557b807200d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.774+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x557b8071dce0 0x557b807200d0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.778+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 reap_dead start 2023-11-15T03:09:29.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.778+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 reap_dead start 2023-11-15T03:09:29.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.778+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3f50059020 msgr2=0x7f3f5005b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.778+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3f50059020 0x7f3f5005b4e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x557b806425a0 tx=0x7f3f60005a50).stop 2023-11-15T03:09:29.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.778+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b80616de0 msgr2=0x557b806e8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.778+0000 7f3f7693fd00 1 --2- 172.21.15.121:0/714934501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b80616de0 0x557b806e8310 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 rx=0x7f3f5c00ba40 tx=0x7f3f5c001790).stop 2023-11-15T03:09:29.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.778+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 reap_dead start 2023-11-15T03:09:29.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.778+0000 7f3f6b786700 1 -- 172.21.15.121:0/714934501 reap_dead start 2023-11-15T03:09:29.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.778+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 shutdown_connections 2023-11-15T03:09:29.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.778+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 >> 172.21.15.121:0/714934501 conn(0x557b806177d0 msgr2=0x557b806446f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:29.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.778+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 shutdown_connections 2023-11-15T03:09:29.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.778+0000 7f3f7693fd00 1 -- 172.21.15.121:0/714934501 wait complete. 2023-11-15T03:09:29.800 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 100.00000000 stray0_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-11-15T03:09:29.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f8799736d00 1 Processor -- start 2023-11-15T03:09:29.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f8799736d00 1 -- start start 2023-11-15T03:09:29.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f8799736d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b5468eade0 0x55b54691d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f8799736d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b546934b10 0x55b5469301b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f8799736d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5469c3680 0x55b5469c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f8799736d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b546898cc0 con 0x55b5469c3680 2023-11-15T03:09:29.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f8799736d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b546863220 con 0x55b546934b10 2023-11-15T03:09:29.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f8799736d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5469c3f90 con 0x55b5468eade0 2023-11-15T03:09:29.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f878e615700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5469c3680 0x55b5469c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f878e615700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5469c3680 0x55b5469c47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51896/0 (socket says 172.21.15.121:51896) 2023-11-15T03:09:29.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f878e615700 1 -- 172.21.15.121:0/460488303 learned_addr learned my addr 172.21.15.121:0/460488303 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:29.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f8785613700 1 --2- 172.21.15.121:0/460488303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b546934b10 0x55b5469301b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f878de14700 1 --2- 172.21.15.121:0/460488303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b5468eade0 0x55b54691d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f878e615700 1 -- 172.21.15.121:0/460488303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b5468eade0 msgr2=0x55b54691d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:29.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f878e615700 1 --2- 172.21.15.121:0/460488303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b5468eade0 0x55b54691d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f878e615700 1 -- 172.21.15.121:0/460488303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b546934b10 msgr2=0x55b5469301b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:29.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f878e615700 1 --2- 172.21.15.121:0/460488303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b546934b10 0x55b5469301b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f878e615700 1 -- 172.21.15.121:0/460488303 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8780005cf0 con 0x55b5469c3680 2023-11-15T03:09:29.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f878e615700 1 --2- 172.21.15.121:0/460488303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5469c3680 0x55b5469c47c0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 rx=0x7f8788006fd0 tx=0x7f8788006950).ready entity=mon.0 client_cookie=ec334cfd073d7c8d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:29.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f878ce12700 1 -- 172.21.15.121:0/460488303 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788006320 con 0x55b5469c3680 2023-11-15T03:09:29.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f878ce12700 1 -- 172.21.15.121:0/460488303 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8788004540 con 0x55b5469c3680 2023-11-15T03:09:29.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.878+0000 7f878ce12700 1 -- 172.21.15.121:0/460488303 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788004900 con 0x55b5469c3680 2023-11-15T03:09:29.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f8799736d00 1 -- 172.21.15.121:0/460488303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5469c3680 msgr2=0x55b5469c47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f8799736d00 1 --2- 172.21.15.121:0/460488303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5469c3680 0x55b5469c47c0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 rx=0x7f8788006fd0 tx=0x7f8788006950).stop 2023-11-15T03:09:29.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f8799736d00 1 -- 172.21.15.121:0/460488303 shutdown_connections 2023-11-15T03:09:29.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f8799736d00 1 --2- 172.21.15.121:0/460488303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5469c3680 0x55b5469c47c0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f8799736d00 1 --2- 172.21.15.121:0/460488303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b546934b10 0x55b5469301b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f8799736d00 1 --2- 172.21.15.121:0/460488303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b5468eade0 0x55b54691d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f8799736d00 1 -- 172.21.15.121:0/460488303 >> 172.21.15.121:0/460488303 conn(0x55b54693a090 msgr2=0x55b54693a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:29.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f8799736d00 1 -- 172.21.15.121:0/460488303 shutdown_connections 2023-11-15T03:09:29.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f8799736d00 1 -- 172.21.15.121:0/460488303 wait complete. 2023-11-15T03:09:29.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f8799736d00 1 Processor -- start 2023-11-15T03:09:29.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f8799736d00 1 -- start start 2023-11-15T03:09:29.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f8799736d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5468eade0 0x55b5469b8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f8799736d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b546934b10 0x55b5469b9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f8799736d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5469c3680 0x55b5469ba880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f8799736d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5469bbda0 con 0x55b5469c3680 2023-11-15T03:09:29.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f8799736d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5469bb190 con 0x55b5468eade0 2023-11-15T03:09:29.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f8799736d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5469bb470 con 0x55b546934b10 2023-11-15T03:09:29.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f878e615700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5469c3680 0x55b5469ba880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f878e615700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5469c3680 0x55b5469ba880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51912/0 (socket says 172.21.15.121:51912) 2023-11-15T03:09:29.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f878e615700 1 -- 172.21.15.121:0/3816792436 learned_addr learned my addr 172.21.15.121:0/3816792436 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:29.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f878de14700 1 --2- 172.21.15.121:0/3816792436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5468eade0 0x55b5469b8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f8785613700 1 --2- 172.21.15.121:0/3816792436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b546934b10 0x55b5469b9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f878e615700 1 -- 172.21.15.121:0/3816792436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b546934b10 msgr2=0x55b5469b9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f878e615700 1 --2- 172.21.15.121:0/3816792436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b546934b10 0x55b5469b9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f878e615700 1 -- 172.21.15.121:0/3816792436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5468eade0 msgr2=0x55b5469b8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f878e615700 1 --2- 172.21.15.121:0/3816792436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5468eade0 0x55b5469b8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f878e615700 1 -- 172.21.15.121:0/3816792436 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8788006c30 con 0x55b5469c3680 2023-11-15T03:09:29.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.882+0000 7f878e615700 1 --2- 172.21.15.121:0/3816792436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5469c3680 0x55b5469ba880 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 rx=0x7f8788000f80 tx=0x7f8788014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:29.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.886+0000 7f8786ffd700 1 -- 172.21.15.121:0/3816792436 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87880048c0 con 0x55b5469c3680 2023-11-15T03:09:29.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.886+0000 7f8786ffd700 1 -- 172.21.15.121:0/3816792436 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8788004cb0 con 0x55b5469c3680 2023-11-15T03:09:29.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.886+0000 7f8786ffd700 1 -- 172.21.15.121:0/3816792436 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87880216a0 con 0x55b5469c3680 2023-11-15T03:09:29.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.886+0000 7f8799736d00 1 -- 172.21.15.121:0/3816792436 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b54685ba70 con 0x55b5469c3680 2023-11-15T03:09:29.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.886+0000 7f8799736d00 1 -- 172.21.15.121:0/3816792436 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b546867010 con 0x55b5469c3680 2023-11-15T03:09:29.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.886+0000 7f8786ffd700 1 -- 172.21.15.121:0/3816792436 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8788006530 con 0x55b5469c3680 2023-11-15T03:09:29.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.886+0000 7f8786ffd700 1 --2- 172.21.15.121:0/3816792436 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8770059020 0x7f877005b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.886+0000 7f8786ffd700 1 -- 172.21.15.121:0/3816792436 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9018+0+0 (secure 0 0 0) 0x7f878801b040 con 0x55b5469c3680 2023-11-15T03:09:29.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.890+0000 7f878de14700 1 --2- 172.21.15.121:0/3816792436 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8770059020 0x7f877005b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.890+0000 7f8799736d00 1 --2- 172.21.15.121:0/3816792436 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55b546919d10 0x55b54691a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:29.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.890+0000 7f8799736d00 1 -- 172.21.15.121:0/3816792436 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=103b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x55b54691a5e0 con 0x55b546919d10 2023-11-15T03:09:29.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.890+0000 7f878de14700 1 --2- 172.21.15.121:0/3816792436 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8770059020 0x7f877005b4e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f878000afd0 tx=0x7f8780016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:29.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.890+0000 7f8785613700 1 --2- 172.21.15.121:0/3816792436 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55b546919d10 0x55b54691a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:29.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.890+0000 7f8785613700 1 --2- 172.21.15.121:0/3816792436 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55b546919d10 0x55b54691a0f0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:29.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.894+0000 7f8785613700 1 -- 172.21.15.121:0/3816792436 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 100.00000000 [omap-set-vals] v126'161 uv161 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8778003cb0 con 0x55b546919d10 2023-11-15T03:09:29.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.894+0000 7f8799736d00 1 -- 172.21.15.121:0/3816792436 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55b546919d10 msgr2=0x55b54691a0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.894+0000 7f8799736d00 1 --2- 172.21.15.121:0/3816792436 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55b546919d10 0x55b54691a0f0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:29.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.894+0000 7f8799736d00 1 -- 172.21.15.121:0/3816792436 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8770059020 msgr2=0x7f877005b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.894+0000 7f8799736d00 1 --2- 172.21.15.121:0/3816792436 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8770059020 0x7f877005b4e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f878000afd0 tx=0x7f8780016000).stop 2023-11-15T03:09:29.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.894+0000 7f8799736d00 1 -- 172.21.15.121:0/3816792436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5469c3680 msgr2=0x55b5469ba880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:29.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.894+0000 7f8799736d00 1 --2- 172.21.15.121:0/3816792436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5469c3680 0x55b5469ba880 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 rx=0x7f8788000f80 tx=0x7f8788014000).stop 2023-11-15T03:09:29.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.894+0000 7f878e615700 1 -- 172.21.15.121:0/3816792436 reap_dead start 2023-11-15T03:09:29.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.894+0000 7f8799736d00 1 -- 172.21.15.121:0/3816792436 shutdown_connections 2023-11-15T03:09:29.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.894+0000 7f8799736d00 1 -- 172.21.15.121:0/3816792436 >> 172.21.15.121:0/3816792436 conn(0x55b54693a090 msgr2=0x55b5469301b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:29.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.894+0000 7f8799736d00 1 -- 172.21.15.121:0/3816792436 shutdown_connections 2023-11-15T03:09:29.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:29.894+0000 7f8799736d00 1 -- 172.21.15.121:0/3816792436 wait complete. 2023-11-15T03:09:29.909 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:09:30.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.074+0000 7f1e54c9d700 1 Processor -- start 2023-11-15T03:09:30.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.074+0000 7f1e54c9d700 1 -- start start 2023-11-15T03:09:30.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.078+0000 7f1e54c9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1e5014aff0 0x7f1e50066c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:30.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.078+0000 7f1e54c9d700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e500657f0 con 0x7f1e5014ac10 2023-11-15T03:09:30.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.078+0000 7f1e54c9d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e50066650 con 0x7f1e50067160 2023-11-15T03:09:30.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.078+0000 7f1e54c9d700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e500675b0 con 0x7f1e5014aff0 2023-11-15T03:09:30.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.078+0000 7f1e4f7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f1e5014ac10 0x7f1e500c6bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:45848/0 (socket says 172.21.15.121:45848) 2023-11-15T03:09:30.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.078+0000 7f1e4f7fe700 1 -- 172.21.15.121:0/4277249158 learned_addr learned my addr 172.21.15.121:0/4277249158 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:09:30.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.078+0000 7f1e4effd700 1 --2- 172.21.15.121:0/4277249158 >> v2:172.21.15.121:3300/0 conn(0x7f1e5014aff0 0x7f1e50066c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:30.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.078+0000 7f1e4e7fc700 1 -- 172.21.15.121:0/4277249158 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2524869988 0 0) 0x7f1e500657f0 con 0x7f1e5014ac10 2023-11-15T03:09:30.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.078+0000 7f1e4e7fc700 1 -- 172.21.15.121:0/4277249158 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e380033f0 con 0x7f1e5014ac10 2023-11-15T03:09:30.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.078+0000 7f1e4effd700 1 -- 172.21.15.121:0/4277249158 >> v1:172.21.15.190:6789/0 conn(0x7f1e50067160 legacy=0x7f1e500a78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:30.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.078+0000 7f1e4effd700 1 -- 172.21.15.121:0/4277249158 >> v1:172.21.15.121:6789/0 conn(0x7f1e5014ac10 legacy=0x7f1e500c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:30.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.078+0000 7f1e4effd700 1 -- 172.21.15.121:0/4277249158 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e50065a40 con 0x7f1e5014aff0 2023-11-15T03:09:30.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.078+0000 7f1e4effd700 1 --2- 172.21.15.121:0/4277249158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e5014aff0 0x7f1e50066c70 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 rx=0x7f1e440096e0 tx=0x7f1e44009060).ready entity=mon.0 client_cookie=7616c56ac99c934e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:30.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.078+0000 7f1e4e7fc700 1 -- 172.21.15.121:0/4277249158 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44009130 con 0x7f1e5014aff0 2023-11-15T03:09:30.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.078+0000 7f1e4e7fc700 1 -- 172.21.15.121:0/4277249158 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4400d4a0 con 0x7f1e5014aff0 2023-11-15T03:09:30.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.078+0000 7f1e4e7fc700 1 -- 172.21.15.121:0/4277249158 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4400d820 con 0x7f1e5014aff0 2023-11-15T03:09:30.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.082+0000 7f1e54c9d700 1 -- 172.21.15.121:0/4277249158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e5014aff0 msgr2=0x7f1e50066c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:30.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.082+0000 7f1e54c9d700 1 --2- 172.21.15.121:0/4277249158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e5014aff0 0x7f1e50066c70 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 rx=0x7f1e440096e0 tx=0x7f1e44009060).stop 2023-11-15T03:09:30.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.082+0000 7f1e54c9d700 1 -- 172.21.15.121:0/4277249158 shutdown_connections 2023-11-15T03:09:30.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.082+0000 7f1e54c9d700 1 --2- 172.21.15.121:0/4277249158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e5014aff0 0x7f1e50066c70 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:30.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.082+0000 7f1e54c9d700 1 -- 172.21.15.121:0/4277249158 >> 172.21.15.121:0/4277249158 conn(0x7f1e50067a60 msgr2=0x7f1e50067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:30.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.082+0000 7f1e54c9d700 1 -- 172.21.15.121:0/4277249158 shutdown_connections 2023-11-15T03:09:30.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.082+0000 7f1e54c9d700 1 -- 172.21.15.121:0/4277249158 wait complete. 2023-11-15T03:09:30.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.082+0000 7f1e54c9d700 1 Processor -- start 2023-11-15T03:09:30.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.082+0000 7f1e54c9d700 1 -- start start 2023-11-15T03:09:30.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e54c9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e50067160 0x7f1e50146960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:30.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e54c9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e5014ac10 0x7f1e500a9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:30.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e54c9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5014aff0 0x7f1e5013f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:30.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e54c9d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e500aaa80 con 0x7f1e5014ac10 2023-11-15T03:09:30.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e54c9d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e500a9e70 con 0x7f1e5014aff0 2023-11-15T03:09:30.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e54c9d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e500aa150 con 0x7f1e50067160 2023-11-15T03:09:30.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e4effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e5014ac10 0x7f1e500a9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:30.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e4effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e5014ac10 0x7f1e500a9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51928/0 (socket says 172.21.15.121:51928) 2023-11-15T03:09:30.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e4effd700 1 -- 172.21.15.121:0/105955628 learned_addr learned my addr 172.21.15.121:0/105955628 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:30.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e4effd700 1 -- 172.21.15.121:0/105955628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e50067160 msgr2=0x7f1e50146960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:30.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e4f7fe700 1 --2- 172.21.15.121:0/105955628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e50067160 0x7f1e50146960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:30.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e4effd700 1 --2- 172.21.15.121:0/105955628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e50067160 0x7f1e50146960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:30.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e4effd700 1 -- 172.21.15.121:0/105955628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5014aff0 msgr2=0x7f1e5013f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:30.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e4effd700 1 --2- 172.21.15.121:0/105955628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5014aff0 0x7f1e5013f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:09:30.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e4effd700 1 -- 172.21.15.121:0/105955628 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e440092a0 con 0x7f1e5014ac10 2023-11-15T03:09:30.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e4f7fe700 1 --2- 172.21.15.121:0/105955628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e50067160 0x7f1e50146960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:30.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e4effd700 1 --2- 172.21.15.121:0/105955628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e5014ac10 0x7f1e500a9620 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 rx=0x7f1e44010000 tx=0x7f1e44014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:30.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e377fe700 1 -- 172.21.15.121:0/105955628 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44011980 con 0x7f1e5014ac10 2023-11-15T03:09:30.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e377fe700 1 -- 172.21.15.121:0/105955628 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e44011ae0 con 0x7f1e5014ac10 2023-11-15T03:09:30.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e54c9d700 1 -- 172.21.15.121:0/105955628 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e500aa430 con 0x7f1e5014ac10 2023-11-15T03:09:30.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e377fe700 1 -- 172.21.15.121:0/105955628 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44011e80 con 0x7f1e5014ac10 2023-11-15T03:09:30.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.086+0000 7f1e54c9d700 1 -- 172.21.15.121:0/105955628 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e500a9b10 con 0x7f1e5014ac10 2023-11-15T03:09:30.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.090+0000 7f1e54c9d700 1 -- 172.21.15.121:0/105955628 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e50065d20 con 0x7f1e5014ac10 2023-11-15T03:09:30.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.090+0000 7f1e377fe700 1 -- 172.21.15.121:0/105955628 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1e4401d640 con 0x7f1e5014ac10 2023-11-15T03:09:30.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.090+0000 7f1e377fe700 1 --2- 172.21.15.121:0/105955628 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1e3c05a510 0x7f1e3c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:30.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.090+0000 7f1e377fe700 1 -- 172.21.15.121:0/105955628 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9018+0+0 (secure 0 0 0) 0x7f1e44076b90 con 0x7f1e5014ac10 2023-11-15T03:09:30.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.090+0000 7f1e4f7fe700 1 --2- 172.21.15.121:0/105955628 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1e3c05a510 0x7f1e3c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:30.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.090+0000 7f1e4f7fe700 1 --2- 172.21.15.121:0/105955628 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1e3c05a510 0x7f1e3c05c9d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f1e40005750 tx=0x7f1e40009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:30.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.098+0000 7f1e377fe700 1 -- 172.21.15.121:0/105955628 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1e44049760 con 0x7f1e5014ac10 2023-11-15T03:09:30.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:30.302+0000 7f1e54c9d700 1 -- 172.21.15.121:0/105955628 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f1e50141fb0 con 0x7f1e5014ac10 2023-11-15T03:09:31.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.266+0000 7f1e377fe700 1 -- 172.21.15.121:0/105955628 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v283) v1 ==== 162+0+0 (secure 0 0 0) 0x7f1e44016020 con 0x7f1e5014ac10 2023-11-15T03:09:31.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.270+0000 7f1e54c9d700 1 -- 172.21.15.121:0/105955628 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1e3c05a510 msgr2=0x7f1e3c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:31.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.270+0000 7f1e54c9d700 1 --2- 172.21.15.121:0/105955628 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1e3c05a510 0x7f1e3c05c9d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f1e40005750 tx=0x7f1e40009040).stop 2023-11-15T03:09:31.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.270+0000 7f1e54c9d700 1 -- 172.21.15.121:0/105955628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e5014ac10 msgr2=0x7f1e500a9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:31.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.270+0000 7f1e54c9d700 1 --2- 172.21.15.121:0/105955628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e5014ac10 0x7f1e500a9620 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 rx=0x7f1e44010000 tx=0x7f1e44014000).stop 2023-11-15T03:09:31.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.270+0000 7f1e54c9d700 1 -- 172.21.15.121:0/105955628 shutdown_connections 2023-11-15T03:09:31.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.270+0000 7f1e54c9d700 1 --2- 172.21.15.121:0/105955628 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1e3c05a510 0x7f1e3c05c9d0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:31.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.270+0000 7f1e54c9d700 1 --2- 172.21.15.121:0/105955628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5014aff0 0x7f1e5013f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:09:31.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.270+0000 7f1e54c9d700 1 --2- 172.21.15.121:0/105955628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e5014ac10 0x7f1e500a9620 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:31.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.270+0000 7f1e54c9d700 1 --2- 172.21.15.121:0/105955628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e50067160 0x7f1e50146960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:31.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.270+0000 7f1e54c9d700 1 -- 172.21.15.121:0/105955628 >> 172.21.15.121:0/105955628 conn(0x7f1e50067a60 msgr2=0x7f1e500c47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:31.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.270+0000 7f1e54c9d700 1 -- 172.21.15.121:0/105955628 shutdown_connections 2023-11-15T03:09:31.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.270+0000 7f1e54c9d700 1 -- 172.21.15.121:0/105955628 wait complete. 2023-11-15T03:09:31.283 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:09:31.303 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:09:31.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.454+0000 7fbe0ea9b700 1 Processor -- start 2023-11-15T03:09:31.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.454+0000 7fbe0ea9b700 1 -- start start 2023-11-15T03:09:31.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.458+0000 7fbe0ea9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe0813a6f0 0x7fbe08063e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:31.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.458+0000 7fbe0ea9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbe080c5ff0 0x7fbe08064340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:31.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.458+0000 7fbe0ea9b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe080c17d0 con 0x7fbe080a67a0 2023-11-15T03:09:31.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.458+0000 7fbe0ea9b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe08062330 con 0x7fbe080c5ff0 2023-11-15T03:09:31.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.458+0000 7fbe0ea9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe080c14a0 con 0x7fbe0813a6f0 2023-11-15T03:09:31.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.458+0000 7fbe0c837700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe0813a6f0 0x7fbe08063e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:31.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.458+0000 7fbe0d038700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbe080c5ff0 0x7fbe08064340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:31.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.458+0000 7fbe07fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fbe080a67a0 0x7fbe080c58e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:55770/0 (socket says 172.21.15.121:55770) 2023-11-15T03:09:31.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.458+0000 7fbe07fff700 1 -- 172.21.15.121:0/575709647 learned_addr learned my addr 172.21.15.121:0/575709647 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:09:31.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.458+0000 7fbe077fe700 1 -- 172.21.15.121:0/575709647 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2026429810 0 0) 0x7fbe080c17d0 con 0x7fbe080a67a0 2023-11-15T03:09:31.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.458+0000 7fbe077fe700 1 -- 172.21.15.121:0/575709647 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdf80033f0 con 0x7fbe080a67a0 2023-11-15T03:09:31.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.458+0000 7fbe0c837700 1 -- 172.21.15.121:0/575709647 >> v2:172.21.15.190:3300/0 conn(0x7fbe080c5ff0 msgr2=0x7fbe08064340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:31.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.458+0000 7fbe0c837700 1 --2- 172.21.15.121:0/575709647 >> v2:172.21.15.190:3300/0 conn(0x7fbe080c5ff0 0x7fbe08064340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:31.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.458+0000 7fbe0c837700 1 -- 172.21.15.121:0/575709647 >> v1:172.21.15.190:6789/0 conn(0x7fbe080a67a0 legacy=0x7fbe080c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:31.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.458+0000 7fbe0c837700 1 -- 172.21.15.121:0/575709647 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe080625d0 con 0x7fbe0813a6f0 2023-11-15T03:09:31.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.458+0000 7fbe0d038700 1 --2- 172.21.15.121:0/575709647 >> v2:172.21.15.190:3300/0 conn(0x7fbe080c5ff0 0x7fbe08064340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:09:31.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.462+0000 7fbe0c837700 1 --2- 172.21.15.121:0/575709647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe0813a6f0 0x7fbe08063e50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fbdfc009760 tx=0x7fbdfc009060).ready entity=mon.2 client_cookie=7d63f6b2e624b19b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:31.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.462+0000 7fbe077fe700 1 -- 172.21.15.121:0/575709647 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc00d4a0 con 0x7fbe0813a6f0 2023-11-15T03:09:31.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.462+0000 7fbe077fe700 1 -- 172.21.15.121:0/575709647 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdfc00d600 con 0x7fbe0813a6f0 2023-11-15T03:09:31.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.462+0000 7fbe077fe700 1 -- 172.21.15.121:0/575709647 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc00d980 con 0x7fbe0813a6f0 2023-11-15T03:09:31.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.462+0000 7fbe0ea9b700 1 -- 172.21.15.121:0/575709647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe0813a6f0 msgr2=0x7fbe08063e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:31.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.462+0000 7fbe0ea9b700 1 --2- 172.21.15.121:0/575709647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe0813a6f0 0x7fbe08063e50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fbdfc009760 tx=0x7fbdfc009060).stop 2023-11-15T03:09:31.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.462+0000 7fbe0ea9b700 1 -- 172.21.15.121:0/575709647 shutdown_connections 2023-11-15T03:09:31.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.462+0000 7fbe0ea9b700 1 --2- 172.21.15.121:0/575709647 >> v2:172.21.15.190:3300/0 conn(0x7fbe080c5ff0 0x7fbe08064340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:31.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.462+0000 7fbe0ea9b700 1 --2- 172.21.15.121:0/575709647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe0813a6f0 0x7fbe08063e50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:31.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.462+0000 7fbe0ea9b700 1 -- 172.21.15.121:0/575709647 >> 172.21.15.121:0/575709647 conn(0x7fbe080c6bd0 msgr2=0x7fbe080c6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:31.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.462+0000 7fbe0ea9b700 1 -- 172.21.15.121:0/575709647 shutdown_connections 2023-11-15T03:09:31.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.462+0000 7fbe0ea9b700 1 -- 172.21.15.121:0/575709647 wait complete. 2023-11-15T03:09:31.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe0ea9b700 1 Processor -- start 2023-11-15T03:09:31.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe0ea9b700 1 -- start start 2023-11-15T03:09:31.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe0ea9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe080a67a0 0x7fbe080bdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:31.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe0ea9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe080c5ff0 0x7fbe08150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:31.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe0ea9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe0813a6f0 0x7fbe08165a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:31.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe0ea9b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe080bf200 con 0x7fbe080c5ff0 2023-11-15T03:09:31.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe0ea9b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe080be5f0 con 0x7fbe080a67a0 2023-11-15T03:09:31.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe0ea9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe080be8d0 con 0x7fbe0813a6f0 2023-11-15T03:09:31.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe07fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe080c5ff0 0x7fbe08150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:31.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe07fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe080c5ff0 0x7fbe08150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51932/0 (socket says 172.21.15.121:51932) 2023-11-15T03:09:31.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe07fff700 1 -- 172.21.15.121:0/3545405095 learned_addr learned my addr 172.21.15.121:0/3545405095 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:31.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe0d038700 1 --2- 172.21.15.121:0/3545405095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe0813a6f0 0x7fbe08165a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:31.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe0c837700 1 --2- 172.21.15.121:0/3545405095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe080a67a0 0x7fbe080bdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:31.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe07fff700 1 -- 172.21.15.121:0/3545405095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe0813a6f0 msgr2=0x7fbe08165a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:31.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe07fff700 1 --2- 172.21.15.121:0/3545405095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe0813a6f0 0x7fbe08165a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:31.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe07fff700 1 -- 172.21.15.121:0/3545405095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe080a67a0 msgr2=0x7fbe080bdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:31.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe07fff700 1 --2- 172.21.15.121:0/3545405095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe080a67a0 0x7fbe080bdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:31.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe07fff700 1 -- 172.21.15.121:0/3545405095 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe00009a20 con 0x7fbe080c5ff0 2023-11-15T03:09:31.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe07fff700 1 --2- 172.21.15.121:0/3545405095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe080c5ff0 0x7fbe08150210 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 rx=0x7fbdf400c4c0 tx=0x7fbdf4007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:31.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe04ff9700 1 -- 172.21.15.121:0/3545405095 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf40072e0 con 0x7fbe080c5ff0 2023-11-15T03:09:31.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe04ff9700 1 -- 172.21.15.121:0/3545405095 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf4007440 con 0x7fbe080c5ff0 2023-11-15T03:09:31.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe0ea9b700 1 -- 172.21.15.121:0/3545405095 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdfc0092a0 con 0x7fbe080c5ff0 2023-11-15T03:09:31.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.466+0000 7fbe04ff9700 1 -- 172.21.15.121:0/3545405095 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf40057a0 con 0x7fbe080c5ff0 2023-11-15T03:09:31.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.470+0000 7fbe0ea9b700 1 -- 172.21.15.121:0/3545405095 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe080bee90 con 0x7fbe080c5ff0 2023-11-15T03:09:31.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.470+0000 7fbe0ea9b700 1 -- 172.21.15.121:0/3545405095 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe080bea10 con 0x7fbe080c5ff0 2023-11-15T03:09:31.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.470+0000 7fbe04ff9700 1 -- 172.21.15.121:0/3545405095 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fbdf4005bd0 con 0x7fbe080c5ff0 2023-11-15T03:09:31.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.470+0000 7fbe04ff9700 1 --2- 172.21.15.121:0/3545405095 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbdec05a540 0x7fbdec05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:31.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.470+0000 7fbe04ff9700 1 -- 172.21.15.121:0/3545405095 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9018+0+0 (secure 0 0 0) 0x7fbdf4074bb0 con 0x7fbe080c5ff0 2023-11-15T03:09:31.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.474+0000 7fbe0c837700 1 --2- 172.21.15.121:0/3545405095 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbdec05a540 0x7fbdec05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:31.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.474+0000 7fbe0c837700 1 --2- 172.21.15.121:0/3545405095 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbdec05a540 0x7fbdec05ca00 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7fbdfc00d260 tx=0x7fbdfc016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:31.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.478+0000 7fbe04ff9700 1 -- 172.21.15.121:0/3545405095 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbe080bea10 con 0x7fbe080c5ff0 2023-11-15T03:09:31.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.674+0000 7fbe0ea9b700 1 -- 172.21.15.121:0/3545405095 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe080bea10 con 0x7fbe080c5ff0 2023-11-15T03:09:31.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.674+0000 7fbe04ff9700 1 -- 172.21.15.121:0/3545405095 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 284 v284) v1 ==== 95+0+4502 (secure 0 0 0) 0x7fbe080bea10 con 0x7fbe080c5ff0 2023-11-15T03:09:31.684 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:09:31.685 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":284,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":284,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:09:31.270396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9942":{"gid":9942,"name":"a","rank":0,"incarnation":284,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/621064675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":621064675},{"type":"v1","addr":"172.21.15.121:6836","nonce":621064675}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:09:31.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.678+0000 7fbe0ea9b700 1 -- 172.21.15.121:0/3545405095 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbdec05a540 msgr2=0x7fbdec05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:31.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.678+0000 7fbe0ea9b700 1 --2- 172.21.15.121:0/3545405095 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbdec05a540 0x7fbdec05ca00 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7fbdfc00d260 tx=0x7fbdfc016000).stop 2023-11-15T03:09:31.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.678+0000 7fbe0ea9b700 1 -- 172.21.15.121:0/3545405095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe080c5ff0 msgr2=0x7fbe08150210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:31.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.678+0000 7fbe0ea9b700 1 --2- 172.21.15.121:0/3545405095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe080c5ff0 0x7fbe08150210 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 rx=0x7fbdf400c4c0 tx=0x7fbdf4007c50).stop 2023-11-15T03:09:31.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.678+0000 7fbe0ea9b700 1 -- 172.21.15.121:0/3545405095 shutdown_connections 2023-11-15T03:09:31.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.678+0000 7fbe0ea9b700 1 --2- 172.21.15.121:0/3545405095 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbdec05a540 0x7fbdec05ca00 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:31.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.678+0000 7fbe0ea9b700 1 --2- 172.21.15.121:0/3545405095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe0813a6f0 0x7fbe08165a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:31.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.678+0000 7fbe0ea9b700 1 --2- 172.21.15.121:0/3545405095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe080c5ff0 0x7fbe08150210 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:31.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.678+0000 7fbe0ea9b700 1 --2- 172.21.15.121:0/3545405095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe080a67a0 0x7fbe080bdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:31.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.678+0000 7fbe0ea9b700 1 -- 172.21.15.121:0/3545405095 >> 172.21.15.121:0/3545405095 conn(0x7fbe080c6bd0 msgr2=0x7fbe08138560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:31.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.678+0000 7fbe0ea9b700 1 -- 172.21.15.121:0/3545405095 shutdown_connections 2023-11-15T03:09:31.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:31.678+0000 7fbe0ea9b700 1 -- 172.21.15.121:0/3545405095 wait complete. 2023-11-15T03:09:31.693 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 284 2023-11-15T03:09:31.709 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:09:34.313 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:09:34.302+0000 7fd43adb5700 -1 mds.0.damage notify_dentry Damage to dentries in fragment * of ino 0x100is fatal because it is a system directory for this rank 2023-11-15T03:09:35.309 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:09:35.298+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:09:35.309 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:09:35.298+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:09:35.320 INFO:tasks.ceph.mds.a.smithi121.stderr: -15> 2023-11-15T03:09:34.302+0000 7fd43adb5700 -1 mds.0.damage notify_dentry Damage to dentries in fragment * of ino 0x100is fatal because it is a system directory for this rank 2023-11-15T03:09:35.357 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:09:36.711 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:09:36.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.866+0000 7fd7f16e4700 1 Processor -- start 2023-11-15T03:09:36.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.866+0000 7fd7f16e4700 1 -- start start 2023-11-15T03:09:36.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.866+0000 7fd7f16e4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd7ec0c2010 0x7fd7ec0c23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:36.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.866+0000 7fd7f16e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7ec153030 0x7fd7ec153480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:36.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.870+0000 7fd7f16e4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7ec154b90 con 0x7fd7ec154750 2023-11-15T03:09:36.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.870+0000 7fd7f16e4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7ec153c90 con 0x7fd7ec0c2010 2023-11-15T03:09:36.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.870+0000 7fd7f16e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec0c1810 con 0x7fd7ec153030 2023-11-15T03:09:36.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.870+0000 7fd7ebfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd7ec0c2010 0x7fd7ec0c23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:36.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.870+0000 7fd7ebfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd7ec0c2010 0x7fd7ec0c23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:33080/0 (socket says 172.21.15.121:33080) 2023-11-15T03:09:36.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.870+0000 7fd7ebfff700 1 -- 172.21.15.121:0/2780687993 learned_addr learned my addr 172.21.15.121:0/2780687993 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:36.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.870+0000 7fd7f0ee3700 1 --2- 172.21.15.121:0/2780687993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7ec153030 0x7fd7ec153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:36.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.870+0000 7fd7ebfff700 1 -- 172.21.15.121:0/2780687993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7ec153030 msgr2=0x7fd7ec153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:36.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.870+0000 7fd7ebfff700 1 --2- 172.21.15.121:0/2780687993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7ec153030 0x7fd7ec153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:36.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.870+0000 7fd7ebfff700 1 -- 172.21.15.121:0/2780687993 >> v1:172.21.15.190:6789/0 conn(0x7fd7ec154750 legacy=0x7fd7ec0c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:36.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.870+0000 7fd7ebfff700 1 -- 172.21.15.121:0/2780687993 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ec0c1af0 con 0x7fd7ec0c2010 2023-11-15T03:09:36.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.870+0000 7fd7ebfff700 1 --2- 172.21.15.121:0/2780687993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7ec0c2010 0x7fd7ec0c23f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7fd7dc006fd0 tx=0x7fd7dc00b3b0).ready entity=mon.1 client_cookie=976e2577a305a390 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:36.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.870+0000 7fd7eb7fe700 1 -- 172.21.15.121:0/2780687993 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc00da70 con 0x7fd7ec0c2010 2023-11-15T03:09:36.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.870+0000 7fd7eb7fe700 1 -- 172.21.15.121:0/2780687993 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7dc00dbd0 con 0x7fd7ec0c2010 2023-11-15T03:09:36.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.870+0000 7fd7eb7fe700 1 -- 172.21.15.121:0/2780687993 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc010470 con 0x7fd7ec0c2010 2023-11-15T03:09:36.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.874+0000 7fd7f16e4700 1 -- 172.21.15.121:0/2780687993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7ec0c2010 msgr2=0x7fd7ec0c23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:36.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.874+0000 7fd7f16e4700 1 --2- 172.21.15.121:0/2780687993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7ec0c2010 0x7fd7ec0c23f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7fd7dc006fd0 tx=0x7fd7dc00b3b0).stop 2023-11-15T03:09:36.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.874+0000 7fd7f16e4700 1 -- 172.21.15.121:0/2780687993 shutdown_connections 2023-11-15T03:09:36.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.874+0000 7fd7f16e4700 1 --2- 172.21.15.121:0/2780687993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7ec153030 0x7fd7ec153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:36.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.874+0000 7fd7f16e4700 1 --2- 172.21.15.121:0/2780687993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7ec0c2010 0x7fd7ec0c23f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:36.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.874+0000 7fd7f16e4700 1 -- 172.21.15.121:0/2780687993 >> 172.21.15.121:0/2780687993 conn(0x7fd7ec062190 msgr2=0x7fd7ec062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:36.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.874+0000 7fd7f16e4700 1 -- 172.21.15.121:0/2780687993 shutdown_connections 2023-11-15T03:09:36.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.874+0000 7fd7f16e4700 1 -- 172.21.15.121:0/2780687993 wait complete. 2023-11-15T03:09:36.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.874+0000 7fd7f16e4700 1 Processor -- start 2023-11-15T03:09:36.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.874+0000 7fd7f16e4700 1 -- start start 2023-11-15T03:09:36.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7f16e4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7ec0c2010 0x7fd7ec141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:36.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7f16e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7ec153030 0x7fd7ec160b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:36.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7f16e4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7ec154750 0x7fd7ec13ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:36.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7f16e4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec161020 con 0x7fd7ec154750 2023-11-15T03:09:36.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7f16e4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec140610 con 0x7fd7ec0c2010 2023-11-15T03:09:36.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7f16e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec141190 con 0x7fd7ec153030 2023-11-15T03:09:36.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7f0ee3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7ec154750 0x7fd7ec13ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:36.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7f0ee3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7ec154750 0x7fd7ec13ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53472/0 (socket says 172.21.15.121:53472) 2023-11-15T03:09:36.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7f0ee3700 1 -- 172.21.15.121:0/3308296211 learned_addr learned my addr 172.21.15.121:0/3308296211 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:36.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7ebfff700 1 --2- 172.21.15.121:0/3308296211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7ec0c2010 0x7fd7ec141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:36.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7e37fe700 1 --2- 172.21.15.121:0/3308296211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7ec153030 0x7fd7ec160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:36.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7f0ee3700 1 -- 172.21.15.121:0/3308296211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7ec153030 msgr2=0x7fd7ec160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:36.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7f0ee3700 1 --2- 172.21.15.121:0/3308296211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7ec153030 0x7fd7ec160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:36.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7f0ee3700 1 -- 172.21.15.121:0/3308296211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7ec0c2010 msgr2=0x7fd7ec141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:36.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7f0ee3700 1 --2- 172.21.15.121:0/3308296211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7ec0c2010 0x7fd7ec141720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:36.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7f0ee3700 1 -- 172.21.15.121:0/3308296211 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7dc00b610 con 0x7fd7ec154750 2023-11-15T03:09:36.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7e37fe700 1 --2- 172.21.15.121:0/3308296211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7ec153030 0x7fd7ec160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:36.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7f0ee3700 1 --2- 172.21.15.121:0/3308296211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7ec154750 0x7fd7ec13ccd0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 rx=0x7fd7e4009e90 tx=0x7fd7e400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:36.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7e8ff9700 1 -- 172.21.15.121:0/3308296211 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e400cc20 con 0x7fd7ec154750 2023-11-15T03:09:36.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7e8ff9700 1 -- 172.21.15.121:0/3308296211 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e400f040 con 0x7fd7ec154750 2023-11-15T03:09:36.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7f16e4700 1 -- 172.21.15.121:0/3308296211 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7ec141470 con 0x7fd7ec154750 2023-11-15T03:09:36.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7e8ff9700 1 -- 172.21.15.121:0/3308296211 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4013400 con 0x7fd7ec154750 2023-11-15T03:09:36.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.878+0000 7fd7f16e4700 1 -- 172.21.15.121:0/3308296211 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ec1402a0 con 0x7fd7ec154750 2023-11-15T03:09:36.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.882+0000 7fd7f16e4700 1 -- 172.21.15.121:0/3308296211 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7ec140750 con 0x7fd7ec154750 2023-11-15T03:09:36.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.882+0000 7fd7e8ff9700 1 -- 172.21.15.121:0/3308296211 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd7e40137b0 con 0x7fd7ec154750 2023-11-15T03:09:36.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.882+0000 7fd7e8ff9700 1 --2- 172.21.15.121:0/3308296211 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd7d005a510 0x7fd7d005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:36.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.882+0000 7fd7e8ff9700 1 -- 172.21.15.121:0/3308296211 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9104+0+0 (secure 0 0 0) 0x7fd7e4075c70 con 0x7fd7ec154750 2023-11-15T03:09:36.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.882+0000 7fd7ebfff700 1 --2- 172.21.15.121:0/3308296211 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd7d005a510 0x7fd7d005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:36.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.886+0000 7fd7ebfff700 1 --2- 172.21.15.121:0/3308296211 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd7d005a510 0x7fd7d005c9d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7fd7ec13feb0 tx=0x7fd7dc00b3b0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:36.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:36.890+0000 7fd7e8ff9700 1 -- 172.21.15.121:0/3308296211 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd7ec140750 con 0x7fd7ec154750 2023-11-15T03:09:37.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.090+0000 7fd7f16e4700 1 -- 172.21.15.121:0/3308296211 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7ec140750 con 0x7fd7ec154750 2023-11-15T03:09:37.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.090+0000 7fd7e8ff9700 1 -- 172.21.15.121:0/3308296211 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 289 v289) v1 ==== 95+0+4496 (secure 0 0 0) 0x7fd7ec140750 con 0x7fd7ec154750 2023-11-15T03:09:37.100 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:09:37.100 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":289,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1175031592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1175031592},{"type":"v1","addr":"172.21.15.121:6836","nonce":1175031592}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":289}],"filesystems":[{"mdsmap":{"epoch":288,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:09:35.304666+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:09:37.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.090+0000 7fd7f16e4700 1 -- 172.21.15.121:0/3308296211 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd7d005a510 msgr2=0x7fd7d005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:37.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.090+0000 7fd7f16e4700 1 --2- 172.21.15.121:0/3308296211 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd7d005a510 0x7fd7d005c9d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7fd7ec13feb0 tx=0x7fd7dc00b3b0).stop 2023-11-15T03:09:37.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.094+0000 7fd7f16e4700 1 -- 172.21.15.121:0/3308296211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7ec154750 msgr2=0x7fd7ec13ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:37.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.094+0000 7fd7f16e4700 1 --2- 172.21.15.121:0/3308296211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7ec154750 0x7fd7ec13ccd0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 rx=0x7fd7e4009e90 tx=0x7fd7e400b040).stop 2023-11-15T03:09:37.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.094+0000 7fd7f16e4700 1 -- 172.21.15.121:0/3308296211 shutdown_connections 2023-11-15T03:09:37.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.094+0000 7fd7f16e4700 1 --2- 172.21.15.121:0/3308296211 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd7d005a510 0x7fd7d005c9d0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.094+0000 7fd7f16e4700 1 --2- 172.21.15.121:0/3308296211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7ec154750 0x7fd7ec13ccd0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.094+0000 7fd7f16e4700 1 --2- 172.21.15.121:0/3308296211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7ec153030 0x7fd7ec160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.094+0000 7fd7f16e4700 1 --2- 172.21.15.121:0/3308296211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7ec0c2010 0x7fd7ec141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.094+0000 7fd7f16e4700 1 -- 172.21.15.121:0/3308296211 >> 172.21.15.121:0/3308296211 conn(0x7fd7ec062190 msgr2=0x7fd7ec147120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:37.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.094+0000 7fd7f16e4700 1 -- 172.21.15.121:0/3308296211 shutdown_connections 2023-11-15T03:09:37.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.094+0000 7fd7f16e4700 1 -- 172.21.15.121:0/3308296211 wait complete. 2023-11-15T03:09:37.105 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 289 2023-11-15T03:09:37.122 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:09:37.122 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:09:37.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.286+0000 7f2bffedb700 1 Processor -- start 2023-11-15T03:09:37.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.290+0000 7f2bffedb700 1 -- start start 2023-11-15T03:09:37.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.290+0000 7f2bffedb700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2bf8153bb0 0x7f2bf80a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:37.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.290+0000 7f2bffedb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf8068170 0x7f2bf8065d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:37.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.290+0000 7f2bffedb700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bf80cb5c0 con 0x7f2bf80aa020 2023-11-15T03:09:37.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.290+0000 7f2bffedb700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf80aa6f0 con 0x7f2bf8153bb0 2023-11-15T03:09:37.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.290+0000 7f2bffedb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf80c5900 con 0x7f2bf8068170 2023-11-15T03:09:37.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.290+0000 7f2bfdc77700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2bf8153bb0 0x7f2bf80a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:37.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.290+0000 7f2bfd476700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f2bf80aa020 0x7f2bf8067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34882/0 (socket says 172.21.15.121:34882) 2023-11-15T03:09:37.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.290+0000 7f2bfdc77700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2bf8153bb0 0x7f2bf80a9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53488/0 (socket says 172.21.15.121:53488) 2023-11-15T03:09:37.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.290+0000 7f2bfd476700 1 -- 172.21.15.121:0/1289532350 learned_addr learned my addr 172.21.15.121:0/1289532350 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:09:37.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.290+0000 7f2bfe478700 1 --2- 172.21.15.121:0/1289532350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf8068170 0x7f2bf8065d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:37.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.290+0000 7f2bfdc77700 1 -- 172.21.15.121:0/1289532350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf8068170 msgr2=0x7f2bf8065d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:37.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.294+0000 7f2bfdc77700 1 --2- 172.21.15.121:0/1289532350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf8068170 0x7f2bf8065d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.294+0000 7f2bfdc77700 1 -- 172.21.15.121:0/1289532350 >> v1:172.21.15.121:6789/0 conn(0x7f2bf80aa020 legacy=0x7f2bf8067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:37.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.294+0000 7f2bfdc77700 1 -- 172.21.15.121:0/1289532350 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf80c9ca0 con 0x7f2bf8153bb0 2023-11-15T03:09:37.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.294+0000 7f2bfdc77700 1 --2- 172.21.15.121:0/1289532350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bf8153bb0 0x7f2bf80a9b30 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 rx=0x7f2bf4009760 tx=0x7f2bf4009060).ready entity=mon.0 client_cookie=586674f97b7adde5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:37.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.294+0000 7f2bfcc75700 1 -- 172.21.15.121:0/1289532350 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf400b580 con 0x7f2bf8153bb0 2023-11-15T03:09:37.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.294+0000 7f2bfcc75700 1 -- 172.21.15.121:0/1289532350 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf400b6e0 con 0x7f2bf8153bb0 2023-11-15T03:09:37.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.294+0000 7f2bfcc75700 1 -- 172.21.15.121:0/1289532350 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf400ba60 con 0x7f2bf8153bb0 2023-11-15T03:09:37.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.294+0000 7f2bffedb700 1 -- 172.21.15.121:0/1289532350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bf8153bb0 msgr2=0x7f2bf80a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:37.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.294+0000 7f2bffedb700 1 --2- 172.21.15.121:0/1289532350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bf8153bb0 0x7f2bf80a9b30 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 rx=0x7f2bf4009760 tx=0x7f2bf4009060).stop 2023-11-15T03:09:37.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.294+0000 7f2bffedb700 1 -- 172.21.15.121:0/1289532350 shutdown_connections 2023-11-15T03:09:37.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.294+0000 7f2bffedb700 1 --2- 172.21.15.121:0/1289532350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf8068170 0x7f2bf8065d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.294+0000 7f2bffedb700 1 --2- 172.21.15.121:0/1289532350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bf8153bb0 0x7f2bf80a9b30 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.294+0000 7f2bffedb700 1 -- 172.21.15.121:0/1289532350 >> 172.21.15.121:0/1289532350 conn(0x7f2bf8058c50 msgr2=0x7f2bf8059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:37.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.294+0000 7f2bffedb700 1 -- 172.21.15.121:0/1289532350 shutdown_connections 2023-11-15T03:09:37.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.294+0000 7f2bffedb700 1 -- 172.21.15.121:0/1289532350 wait complete. 2023-11-15T03:09:37.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.298+0000 7f2bffedb700 1 Processor -- start 2023-11-15T03:09:37.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.298+0000 7f2bffedb700 1 -- start start 2023-11-15T03:09:37.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.298+0000 7f2bffedb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf8068170 0x7f2bf814c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:37.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.298+0000 7f2bffedb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bf80aa020 0x7f2bf80c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:37.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.298+0000 7f2bffedb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bf8153bb0 0x7f2bf80c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:37.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.298+0000 7f2bffedb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf80c26e0 con 0x7f2bf80aa020 2023-11-15T03:09:37.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.298+0000 7f2bffedb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf80c1db0 con 0x7f2bf8153bb0 2023-11-15T03:09:37.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.298+0000 7f2bffedb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf80c2090 con 0x7f2bf8068170 2023-11-15T03:09:37.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.302+0000 7f2bfd476700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bf80aa020 0x7f2bf80c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:37.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.302+0000 7f2bfdc77700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf8068170 0x7f2bf814c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:37.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.302+0000 7f2bfd476700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bf80aa020 0x7f2bf80c39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53490/0 (socket says 172.21.15.121:53490) 2023-11-15T03:09:37.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.302+0000 7f2bfd476700 1 -- 172.21.15.121:0/1150734253 learned_addr learned my addr 172.21.15.121:0/1150734253 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:37.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.302+0000 7f2bfe478700 1 --2- 172.21.15.121:0/1150734253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bf8153bb0 0x7f2bf80c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:37.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.302+0000 7f2bfd476700 1 -- 172.21.15.121:0/1150734253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf8068170 msgr2=0x7f2bf814c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:37.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.302+0000 7f2bfd476700 1 --2- 172.21.15.121:0/1150734253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf8068170 0x7f2bf814c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.302+0000 7f2bfd476700 1 -- 172.21.15.121:0/1150734253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bf8153bb0 msgr2=0x7f2bf80c3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:37.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.302+0000 7f2bfd476700 1 --2- 172.21.15.121:0/1150734253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bf8153bb0 0x7f2bf80c3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.302+0000 7f2bfd476700 1 -- 172.21.15.121:0/1150734253 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf40092a0 con 0x7f2bf80aa020 2023-11-15T03:09:37.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.302+0000 7f2bfdc77700 1 --2- 172.21.15.121:0/1150734253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf8068170 0x7f2bf814c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:09:37.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.302+0000 7f2bfd476700 1 --2- 172.21.15.121:0/1150734253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bf80aa020 0x7f2bf80c39b0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 rx=0x7f2be8007fd0 tx=0x7f2be8007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:37.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.302+0000 7f2bedffb700 1 -- 172.21.15.121:0/1150734253 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be800ce10 con 0x7f2bf80aa020 2023-11-15T03:09:37.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.302+0000 7f2bedffb700 1 -- 172.21.15.121:0/1150734253 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be80072e0 con 0x7f2bf80aa020 2023-11-15T03:09:37.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.302+0000 7f2bffedb700 1 -- 172.21.15.121:0/1150734253 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf80c2370 con 0x7f2bf80aa020 2023-11-15T03:09:37.315 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.302+0000 7f2bedffb700 1 -- 172.21.15.121:0/1150734253 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be8007660 con 0x7f2bf80aa020 2023-11-15T03:09:37.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.302+0000 7f2bffedb700 1 -- 172.21.15.121:0/1150734253 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf80c4470 con 0x7f2bf80aa020 2023-11-15T03:09:37.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.306+0000 7f2bffedb700 1 -- 172.21.15.121:0/1150734253 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf806c150 con 0x7f2bf80aa020 2023-11-15T03:09:37.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.306+0000 7f2bedffb700 1 -- 172.21.15.121:0/1150734253 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2be80059b0 con 0x7f2bf80aa020 2023-11-15T03:09:37.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.310+0000 7f2bedffb700 1 --2- 172.21.15.121:0/1150734253 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2bdc05a510 0x7f2bdc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:37.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.310+0000 7f2bedffb700 1 -- 172.21.15.121:0/1150734253 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9104+0+0 (secure 0 0 0) 0x7f2be8074b60 con 0x7f2bf80aa020 2023-11-15T03:09:37.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.310+0000 7f2bfdc77700 1 --2- 172.21.15.121:0/1150734253 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2bdc05a510 0x7f2bdc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:37.320 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.310+0000 7f2bfdc77700 1 --2- 172.21.15.121:0/1150734253 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2bdc05a510 0x7f2bdc05c9d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7f2bf400b340 tx=0x7f2bf4017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:37.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.314+0000 7f2bedffb700 1 -- 172.21.15.121:0/1150734253 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2be8047700 con 0x7f2bf80aa020 2023-11-15T03:09:37.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.514+0000 7f2bffedb700 1 -- 172.21.15.121:0/1150734253 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bf80c3790 con 0x7f2bf80aa020 2023-11-15T03:09:37.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.514+0000 7f2bedffb700 1 -- 172.21.15.121:0/1150734253 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 289 v289) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f2be804ad20 con 0x7f2bf80aa020 2023-11-15T03:09:37.526 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:09:37.526 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":289,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1175031592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1175031592},{"type":"v1","addr":"172.21.15.121:6836","nonce":1175031592}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":289}],"filesystems":[{"mdsmap":{"epoch":288,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:09:35.304666+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:09:37.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.518+0000 7f2bffedb700 1 -- 172.21.15.121:0/1150734253 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2bdc05a510 msgr2=0x7f2bdc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:37.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.518+0000 7f2bffedb700 1 --2- 172.21.15.121:0/1150734253 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2bdc05a510 0x7f2bdc05c9d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7f2bf400b340 tx=0x7f2bf4017000).stop 2023-11-15T03:09:37.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.518+0000 7f2bffedb700 1 -- 172.21.15.121:0/1150734253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bf80aa020 msgr2=0x7f2bf80c39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:37.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.518+0000 7f2bffedb700 1 --2- 172.21.15.121:0/1150734253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bf80aa020 0x7f2bf80c39b0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 rx=0x7f2be8007fd0 tx=0x7f2be8007df0).stop 2023-11-15T03:09:37.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.518+0000 7f2bffedb700 1 -- 172.21.15.121:0/1150734253 shutdown_connections 2023-11-15T03:09:37.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.518+0000 7f2bffedb700 1 --2- 172.21.15.121:0/1150734253 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2bdc05a510 0x7f2bdc05c9d0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.518+0000 7f2bffedb700 1 --2- 172.21.15.121:0/1150734253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bf8153bb0 0x7f2bf80c3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.518+0000 7f2bffedb700 1 --2- 172.21.15.121:0/1150734253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bf80aa020 0x7f2bf80c39b0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.518+0000 7f2bffedb700 1 --2- 172.21.15.121:0/1150734253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf8068170 0x7f2bf814c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.518+0000 7f2bffedb700 1 -- 172.21.15.121:0/1150734253 >> 172.21.15.121:0/1150734253 conn(0x7f2bf8058c50 msgr2=0x7f2bf80c5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:37.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.518+0000 7f2bffedb700 1 -- 172.21.15.121:0/1150734253 shutdown_connections 2023-11-15T03:09:37.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.518+0000 7f2bffedb700 1 -- 172.21.15.121:0/1150734253 wait complete. 2023-11-15T03:09:37.532 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 289 2023-11-15T03:09:37.547 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:09:37.548 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:09:37.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.694+0000 7f58be439700 1 Processor -- start 2023-11-15T03:09:37.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.694+0000 7f58be439700 1 -- start start 2023-11-15T03:09:37.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.698+0000 7f58be439700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b8151870 0x7f58b8151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:37.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.698+0000 7f58be439700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f58b8152140 0x7f58b80a9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:37.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.698+0000 7f58be439700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b80657f0 con 0x7f58b80a9d20 2023-11-15T03:09:37.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.698+0000 7f58be439700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b8065ad0 con 0x7f58b8152140 2023-11-15T03:09:37.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.698+0000 7f58be439700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80aa260 con 0x7f58b8151870 2023-11-15T03:09:37.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.698+0000 7f58bcc36700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f58b8152140 0x7f58b80a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:37.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.698+0000 7f58bd437700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b8151870 0x7f58b8151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:37.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.698+0000 7f58bd437700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b8151870 0x7f58b8151c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:42476/0 (socket says 172.21.15.121:42476) 2023-11-15T03:09:37.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.698+0000 7f58bd437700 1 -- 172.21.15.121:0/3889380812 learned_addr learned my addr 172.21.15.121:0/3889380812 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:37.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.698+0000 7f58bcc36700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f58b8152140 0x7f58b80a9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53494/0 (socket says 172.21.15.121:53494) 2023-11-15T03:09:37.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.698+0000 7f58bdc38700 1 --1- 172.21.15.121:0/3889380812 >> v1:172.21.15.190:6789/0 conn(0x7f58b80a9d20 0x7f58b80c30e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:56616/0 (socket says 172.21.15.121:56616) 2023-11-15T03:09:37.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.698+0000 7f58bd437700 1 -- 172.21.15.121:0/3889380812 >> v2:172.21.15.121:3300/0 conn(0x7f58b8152140 msgr2=0x7f58b80a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:37.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.698+0000 7f58bd437700 1 --2- 172.21.15.121:0/3889380812 >> v2:172.21.15.121:3300/0 conn(0x7f58b8152140 0x7f58b80a9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.698+0000 7f58bd437700 1 -- 172.21.15.121:0/3889380812 >> v1:172.21.15.190:6789/0 conn(0x7f58b80a9d20 legacy=0x7f58b80c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:37.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.698+0000 7f58bd437700 1 -- 172.21.15.121:0/3889380812 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b80aa540 con 0x7f58b8151870 2023-11-15T03:09:37.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.702+0000 7f58bd437700 1 --2- 172.21.15.121:0/3889380812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b8151870 0x7f58b8151c50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f58b4006fd0 tx=0x7f58b40090f0).ready entity=mon.2 client_cookie=e604a8e1dfc7f3c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:37.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.702+0000 7f58affff700 1 -- 172.21.15.121:0/3889380812 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b400b580 con 0x7f58b8151870 2023-11-15T03:09:37.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.702+0000 7f58affff700 1 -- 172.21.15.121:0/3889380812 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b400b6e0 con 0x7f58b8151870 2023-11-15T03:09:37.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.702+0000 7f58affff700 1 -- 172.21.15.121:0/3889380812 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b400ba60 con 0x7f58b8151870 2023-11-15T03:09:37.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.702+0000 7f58be439700 1 -- 172.21.15.121:0/3889380812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b8151870 msgr2=0x7f58b8151c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:37.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.702+0000 7f58be439700 1 --2- 172.21.15.121:0/3889380812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b8151870 0x7f58b8151c50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f58b4006fd0 tx=0x7f58b40090f0).stop 2023-11-15T03:09:37.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.702+0000 7f58be439700 1 -- 172.21.15.121:0/3889380812 shutdown_connections 2023-11-15T03:09:37.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.702+0000 7f58be439700 1 --2- 172.21.15.121:0/3889380812 >> v2:172.21.15.121:3300/0 conn(0x7f58b8152140 0x7f58b80a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.702+0000 7f58be439700 1 --2- 172.21.15.121:0/3889380812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b8151870 0x7f58b8151c50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.702+0000 7f58be439700 1 -- 172.21.15.121:0/3889380812 >> 172.21.15.121:0/3889380812 conn(0x7f58b8062190 msgr2=0x7f58b8062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:37.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.702+0000 7f58be439700 1 -- 172.21.15.121:0/3889380812 shutdown_connections 2023-11-15T03:09:37.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.702+0000 7f58be439700 1 -- 172.21.15.121:0/3889380812 wait complete. 2023-11-15T03:09:37.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.706+0000 7f58be439700 1 Processor -- start 2023-11-15T03:09:37.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.706+0000 7f58be439700 1 -- start start 2023-11-15T03:09:37.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.706+0000 7f58be439700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58b80a9d20 0x7f58b8164fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:37.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.706+0000 7f58be439700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b8151870 0x7f58b8157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:37.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.706+0000 7f58be439700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58b8152140 0x7f58b8145740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:37.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.706+0000 7f58be439700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80c6e10 con 0x7f58b80a9d20 2023-11-15T03:09:37.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.706+0000 7f58be439700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b8165490 con 0x7f58b8152140 2023-11-15T03:09:37.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.706+0000 7f58be439700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b8165740 con 0x7f58b8151870 2023-11-15T03:09:37.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.706+0000 7f58bd437700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58b80a9d20 0x7f58b8164fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:37.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.710+0000 7f58bd437700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58b80a9d20 0x7f58b8164fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53510/0 (socket says 172.21.15.121:53510) 2023-11-15T03:09:37.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.710+0000 7f58bd437700 1 -- 172.21.15.121:0/1171661449 learned_addr learned my addr 172.21.15.121:0/1171661449 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:37.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.710+0000 7f58bcc36700 1 --2- 172.21.15.121:0/1171661449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b8151870 0x7f58b8157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:37.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.710+0000 7f58bdc38700 1 --2- 172.21.15.121:0/1171661449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58b8152140 0x7f58b8145740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:37.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.710+0000 7f58bd437700 1 -- 172.21.15.121:0/1171661449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b8151870 msgr2=0x7f58b8157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:37.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.710+0000 7f58bd437700 1 --2- 172.21.15.121:0/1171661449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b8151870 0x7f58b8157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.710+0000 7f58bd437700 1 -- 172.21.15.121:0/1171661449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58b8152140 msgr2=0x7f58b8145740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:37.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.710+0000 7f58bd437700 1 --2- 172.21.15.121:0/1171661449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58b8152140 0x7f58b8145740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.710+0000 7f58bd437700 1 -- 172.21.15.121:0/1171661449 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b4009370 con 0x7f58b80a9d20 2023-11-15T03:09:37.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.710+0000 7f58bd437700 1 --2- 172.21.15.121:0/1171661449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58b80a9d20 0x7f58b8164fa0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 rx=0x7f58b4009fd0 tx=0x7f58b4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:37.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.710+0000 7f58ad7fa700 1 -- 172.21.15.121:0/1171661449 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b4006a40 con 0x7f58b80a9d20 2023-11-15T03:09:37.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.710+0000 7f58ad7fa700 1 -- 172.21.15.121:0/1171661449 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b4000bc0 con 0x7f58b80a9d20 2023-11-15T03:09:37.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.710+0000 7f58be439700 1 -- 172.21.15.121:0/1171661449 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b8165a20 con 0x7f58b80a9d20 2023-11-15T03:09:37.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.710+0000 7f58ad7fa700 1 -- 172.21.15.121:0/1171661449 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b401c420 con 0x7f58b80a9d20 2023-11-15T03:09:37.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.710+0000 7f58be439700 1 -- 172.21.15.121:0/1171661449 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b80c44e0 con 0x7f58b80a9d20 2023-11-15T03:09:37.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.714+0000 7f58be439700 1 -- 172.21.15.121:0/1171661449 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b806bcf0 con 0x7f58b80a9d20 2023-11-15T03:09:37.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.714+0000 7f58ad7fa700 1 -- 172.21.15.121:0/1171661449 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f58b401c810 con 0x7f58b80a9d20 2023-11-15T03:09:37.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.714+0000 7f58ad7fa700 1 --2- 172.21.15.121:0/1171661449 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f589c05a510 0x7f589c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:37.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.714+0000 7f58ad7fa700 1 -- 172.21.15.121:0/1171661449 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9104+0+0 (secure 0 0 0) 0x7f58b4076f80 con 0x7f58b80a9d20 2023-11-15T03:09:37.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.714+0000 7f58bcc36700 1 --2- 172.21.15.121:0/1171661449 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f589c05a510 0x7f589c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:37.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.718+0000 7f58bcc36700 1 --2- 172.21.15.121:0/1171661449 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f589c05a510 0x7f589c05c9d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f58b80c7600 tx=0x7f58a8009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:37.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.722+0000 7f58ad7fa700 1 -- 172.21.15.121:0/1171661449 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f58b4049ac0 con 0x7f58b80a9d20 2023-11-15T03:09:37.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.918+0000 7f58be439700 1 -- 172.21.15.121:0/1171661449 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58b8068a70 con 0x7f58b80a9d20 2023-11-15T03:09:37.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.918+0000 7f58ad7fa700 1 -- 172.21.15.121:0/1171661449 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 289 v289) v1 ==== 95+0+4496 (secure 0 0 0) 0x7f58b4020030 con 0x7f58b80a9d20 2023-11-15T03:09:37.929 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:09:37.929 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":289,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1175031592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1175031592},{"type":"v1","addr":"172.21.15.121:6836","nonce":1175031592}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":289}],"filesystems":[{"mdsmap":{"epoch":288,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:09:35.304666+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:09:37.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.922+0000 7f58be439700 1 -- 172.21.15.121:0/1171661449 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f589c05a510 msgr2=0x7f589c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:37.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.922+0000 7f58be439700 1 --2- 172.21.15.121:0/1171661449 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f589c05a510 0x7f589c05c9d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f58b80c7600 tx=0x7f58a8009060).stop 2023-11-15T03:09:37.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.922+0000 7f58be439700 1 -- 172.21.15.121:0/1171661449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58b80a9d20 msgr2=0x7f58b8164fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:37.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.922+0000 7f58be439700 1 --2- 172.21.15.121:0/1171661449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58b80a9d20 0x7f58b8164fa0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 rx=0x7f58b4009fd0 tx=0x7f58b4017000).stop 2023-11-15T03:09:37.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.922+0000 7f58be439700 1 -- 172.21.15.121:0/1171661449 shutdown_connections 2023-11-15T03:09:37.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.922+0000 7f58be439700 1 --2- 172.21.15.121:0/1171661449 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f589c05a510 0x7f589c05c9d0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.922+0000 7f58be439700 1 --2- 172.21.15.121:0/1171661449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58b8152140 0x7f58b8145740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.922+0000 7f58be439700 1 --2- 172.21.15.121:0/1171661449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b8151870 0x7f58b8157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.922+0000 7f58be439700 1 --2- 172.21.15.121:0/1171661449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58b80a9d20 0x7f58b8164fa0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:37.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.922+0000 7f58be439700 1 -- 172.21.15.121:0/1171661449 >> 172.21.15.121:0/1171661449 conn(0x7f58b8062190 msgr2=0x7f58b8150c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:37.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.922+0000 7f58be439700 1 -- 172.21.15.121:0/1171661449 shutdown_connections 2023-11-15T03:09:37.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:37.922+0000 7f58be439700 1 -- 172.21.15.121:0/1171661449 wait complete. 2023-11-15T03:09:37.934 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 289 2023-11-15T03:09:37.953 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap key 100.00000000:stray0_head' led to DAMAGED state 2023-11-15T03:09:37.953 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 1.00000000:subdir_head' 2023-11-15T03:09:37.953 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:09:37.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:37.954 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:09:37.974 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:09:37.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:09:37.976 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:09:38.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.182+0000 7f8c063a9700 1 Processor -- start 2023-11-15T03:09:38.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.182+0000 7f8c063a9700 1 -- start start 2023-11-15T03:09:38.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.186+0000 7f8c063a9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8c000c04d0 0x7f8c000c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:38.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.186+0000 7f8c063a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c00153670 0x7f8c00153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:38.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.186+0000 7f8c063a9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c000c5720 con 0x7f8c000c8fb0 2023-11-15T03:09:38.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.186+0000 7f8c063a9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c000c0da0 con 0x7f8c000c04d0 2023-11-15T03:09:38.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.186+0000 7f8c063a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c000ca8a0 con 0x7f8c00153670 2023-11-15T03:09:38.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.186+0000 7f8bfffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8c000c04d0 0x7f8c000c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:38.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.186+0000 7f8bfffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8c000c04d0 0x7f8c000c08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:33114/0 (socket says 172.21.15.121:33114) 2023-11-15T03:09:38.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.186+0000 7f8bfffff700 1 -- 172.21.15.121:0/3707963046 learned_addr learned my addr 172.21.15.121:0/3707963046 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:38.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.186+0000 7f8bf77fe700 1 --2- 172.21.15.121:0/3707963046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c00153670 0x7f8c00153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:38.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.186+0000 7f8bfffff700 1 -- 172.21.15.121:0/3707963046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c00153670 msgr2=0x7f8c00153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:38.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.186+0000 7f8bff7fe700 1 -- 172.21.15.121:0/3707963046 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4117096118 0 0) 0x7f8c000c5720 con 0x7f8c000c8fb0 2023-11-15T03:09:38.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.186+0000 7f8bfffff700 1 --2- 172.21.15.121:0/3707963046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c00153670 0x7f8c00153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:38.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.186+0000 7f8bfffff700 1 -- 172.21.15.121:0/3707963046 >> v1:172.21.15.121:6789/0 conn(0x7f8c000c8fb0 legacy=0x7f8c000c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:38.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.186+0000 7f8bfffff700 1 -- 172.21.15.121:0/3707963046 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c000c9d20 con 0x7f8c000c04d0 2023-11-15T03:09:38.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.186+0000 7f8bf77fe700 1 --2- 172.21.15.121:0/3707963046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c00153670 0x7f8c00153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:09:38.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.186+0000 7f8bfffff700 1 --2- 172.21.15.121:0/3707963046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c000c04d0 0x7f8c000c08b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f8bf0009f20 tx=0x7f8bf0009840).ready entity=mon.1 client_cookie=cc79fd4d87e54322 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:38.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.190+0000 7f8bff7fe700 1 -- 172.21.15.121:0/3707963046 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf000e070 con 0x7f8c000c04d0 2023-11-15T03:09:38.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.190+0000 7f8bff7fe700 1 -- 172.21.15.121:0/3707963046 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf000b9d0 con 0x7f8c000c04d0 2023-11-15T03:09:38.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.190+0000 7f8bff7fe700 1 -- 172.21.15.121:0/3707963046 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf000bd50 con 0x7f8c000c04d0 2023-11-15T03:09:38.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.190+0000 7f8c063a9700 1 -- 172.21.15.121:0/3707963046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c000c04d0 msgr2=0x7f8c000c08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:38.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.190+0000 7f8c063a9700 1 --2- 172.21.15.121:0/3707963046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c000c04d0 0x7f8c000c08b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f8bf0009f20 tx=0x7f8bf0009840).stop 2023-11-15T03:09:38.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.190+0000 7f8c063a9700 1 -- 172.21.15.121:0/3707963046 shutdown_connections 2023-11-15T03:09:38.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.190+0000 7f8c063a9700 1 --2- 172.21.15.121:0/3707963046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c00153670 0x7f8c00153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:38.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.190+0000 7f8c063a9700 1 --2- 172.21.15.121:0/3707963046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c000c04d0 0x7f8c000c08b0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:38.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.190+0000 7f8c063a9700 1 -- 172.21.15.121:0/3707963046 >> 172.21.15.121:0/3707963046 conn(0x7f8c00067a60 msgr2=0x7f8c00067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:38.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.190+0000 7f8c063a9700 1 -- 172.21.15.121:0/3707963046 shutdown_connections 2023-11-15T03:09:38.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.190+0000 7f8c063a9700 1 -- 172.21.15.121:0/3707963046 wait complete. 2023-11-15T03:09:38.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.190+0000 7f8c063a9700 1 Processor -- start 2023-11-15T03:09:38.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.190+0000 7f8c063a9700 1 -- start start 2023-11-15T03:09:38.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8c063a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c000c04d0 0x7f8c0013c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:38.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8c063a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c000c8fb0 0x7f8c001527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:38.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8c063a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c00153670 0x7f8c00152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:38.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8c063a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c00151db0 con 0x7f8c000c04d0 2023-11-15T03:09:38.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8c063a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c00152350 con 0x7f8c000c8fb0 2023-11-15T03:09:38.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8c063a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c00150320 con 0x7f8c00153670 2023-11-15T03:09:38.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8bfffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c000c04d0 0x7f8c0013c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:38.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8bfffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c000c04d0 0x7f8c0013c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53516/0 (socket says 172.21.15.121:53516) 2023-11-15T03:09:38.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8bfffff700 1 -- 172.21.15.121:0/2083964643 learned_addr learned my addr 172.21.15.121:0/2083964643 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:38.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8c04946700 1 --2- 172.21.15.121:0/2083964643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c00153670 0x7f8c00152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:38.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8bf77fe700 1 --2- 172.21.15.121:0/2083964643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c000c8fb0 0x7f8c001527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:38.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8bfffff700 1 -- 172.21.15.121:0/2083964643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c00153670 msgr2=0x7f8c00152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:38.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8bfffff700 1 --2- 172.21.15.121:0/2083964643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c00153670 0x7f8c00152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:38.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8bfffff700 1 -- 172.21.15.121:0/2083964643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c000c8fb0 msgr2=0x7f8c001527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:38.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8bfffff700 1 --2- 172.21.15.121:0/2083964643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c000c8fb0 0x7f8c001527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:38.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8bfffff700 1 -- 172.21.15.121:0/2083964643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be80092a0 con 0x7f8c000c04d0 2023-11-15T03:09:38.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8bfffff700 1 --2- 172.21.15.121:0/2083964643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c000c04d0 0x7f8c0013c440 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 rx=0x7f8bf000b610 tx=0x7f8bf0009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:38.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8bfcff9700 1 -- 172.21.15.121:0/2083964643 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf00092e0 con 0x7f8c000c04d0 2023-11-15T03:09:38.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8bfcff9700 1 -- 172.21.15.121:0/2083964643 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf000bb40 con 0x7f8c000c04d0 2023-11-15T03:09:38.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8c063a9700 1 -- 172.21.15.121:0/2083964643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf0009ae0 con 0x7f8c000c04d0 2023-11-15T03:09:38.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8bfcff9700 1 -- 172.21.15.121:0/2083964643 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf001e680 con 0x7f8c000c04d0 2023-11-15T03:09:38.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.194+0000 7f8c063a9700 1 -- 172.21.15.121:0/2083964643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c00151fe0 con 0x7f8c000c04d0 2023-11-15T03:09:38.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.198+0000 7f8c063a9700 1 -- 172.21.15.121:0/2083964643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c00047060 con 0x7f8c000c04d0 2023-11-15T03:09:38.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.198+0000 7f8bfcff9700 1 -- 172.21.15.121:0/2083964643 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8bf0003040 con 0x7f8c000c04d0 2023-11-15T03:09:38.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.198+0000 7f8bfcff9700 1 --2- 172.21.15.121:0/2083964643 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8be005a510 0x7f8be005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:38.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.198+0000 7f8bfcff9700 1 -- 172.21.15.121:0/2083964643 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9104+0+0 (secure 0 0 0) 0x7f8bf0076550 con 0x7f8c000c04d0 2023-11-15T03:09:38.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.202+0000 7f8bf77fe700 1 --2- 172.21.15.121:0/2083964643 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8be005a510 0x7f8be005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:38.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.202+0000 7f8bf77fe700 1 --2- 172.21.15.121:0/2083964643 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8be005a510 0x7f8be005c9d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f8c001574d0 tx=0x7f8be8009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:38.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.206+0000 7f8bfcff9700 1 -- 172.21.15.121:0/2083964643 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8bf0049200 con 0x7f8c000c04d0 2023-11-15T03:09:38.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:38.406+0000 7f8c063a9700 1 -- 172.21.15.121:0/2083964643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8c000c8070 con 0x7f8c000c04d0 2023-11-15T03:09:39.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.338+0000 7f8bfcff9700 1 -- 172.21.15.121:0/2083964643 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v290) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8bf0023020 con 0x7f8c000c04d0 2023-11-15T03:09:39.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.342+0000 7f8c063a9700 1 -- 172.21.15.121:0/2083964643 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8be005a510 msgr2=0x7f8be005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:39.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.342+0000 7f8c063a9700 1 --2- 172.21.15.121:0/2083964643 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8be005a510 0x7f8be005c9d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f8c001574d0 tx=0x7f8be8009060).stop 2023-11-15T03:09:39.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.342+0000 7f8c063a9700 1 -- 172.21.15.121:0/2083964643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c000c04d0 msgr2=0x7f8c0013c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:39.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.342+0000 7f8c063a9700 1 --2- 172.21.15.121:0/2083964643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c000c04d0 0x7f8c0013c440 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 rx=0x7f8bf000b610 tx=0x7f8bf0009840).stop 2023-11-15T03:09:39.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.342+0000 7f8c063a9700 1 -- 172.21.15.121:0/2083964643 shutdown_connections 2023-11-15T03:09:39.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.342+0000 7f8c063a9700 1 --2- 172.21.15.121:0/2083964643 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8be005a510 0x7f8be005c9d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:39.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.342+0000 7f8c063a9700 1 --2- 172.21.15.121:0/2083964643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c00153670 0x7f8c00152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:39.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.342+0000 7f8c063a9700 1 --2- 172.21.15.121:0/2083964643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c000c8fb0 0x7f8c001527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:39.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.342+0000 7f8c063a9700 1 --2- 172.21.15.121:0/2083964643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c000c04d0 0x7f8c0013c440 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:39.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.342+0000 7f8c063a9700 1 -- 172.21.15.121:0/2083964643 >> 172.21.15.121:0/2083964643 conn(0x7f8c00067a60 msgr2=0x7f8c000c6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:39.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.342+0000 7f8c063a9700 1 -- 172.21.15.121:0/2083964643 shutdown_connections 2023-11-15T03:09:39.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.342+0000 7f8c063a9700 1 -- 172.21.15.121:0/2083964643 wait complete. 2023-11-15T03:09:39.355 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:09:39.373 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:09:39.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.530+0000 7f3738f79700 1 Processor -- start 2023-11-15T03:09:39.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.534+0000 7f3738f79700 1 -- start start 2023-11-15T03:09:39.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.534+0000 7f3738f79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37340669b0 0x7f3734158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:39.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.534+0000 7f3738f79700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f37340a8750 0x7f37340a8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:39.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.534+0000 7f3738f79700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37340a90d0 con 0x7f3734157d90 2023-11-15T03:09:39.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.534+0000 7f3738f79700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37340c1fb0 con 0x7f37340a8750 2023-11-15T03:09:39.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.534+0000 7f3738f79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373415bbd0 con 0x7f37340669b0 2023-11-15T03:09:39.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.534+0000 7f372affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37340669b0 0x7f3734158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:39.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f3733fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f37340a8750 0x7f37340a8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:39.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f372affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37340669b0 0x7f3734158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:42516/0 (socket says 172.21.15.121:42516) 2023-11-15T03:09:39.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f372affd700 1 -- 172.21.15.121:0/201818605 learned_addr learned my addr 172.21.15.121:0/201818605 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:39.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f372affd700 1 -- 172.21.15.121:0/201818605 >> v2:172.21.15.190:3300/0 conn(0x7f37340a8750 msgr2=0x7f37340a8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:39.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f372affd700 1 --2- 172.21.15.121:0/201818605 >> v2:172.21.15.190:3300/0 conn(0x7f37340a8750 0x7f37340a8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:39.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f372affd700 1 -- 172.21.15.121:0/201818605 >> v1:172.21.15.190:6789/0 conn(0x7f3734157d90 legacy=0x7f37340c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:39.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f372affd700 1 -- 172.21.15.121:0/201818605 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37340bb910 con 0x7f37340669b0 2023-11-15T03:09:39.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f3733fff700 1 --2- 172.21.15.121:0/201818605 >> v2:172.21.15.190:3300/0 conn(0x7f37340a8750 0x7f37340a8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:09:39.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f3732ffd700 1 -- 172.21.15.121:0/201818605 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686314514 0 0) 0x7f37340a90d0 con 0x7f3734157d90 2023-11-15T03:09:39.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f372affd700 1 --2- 172.21.15.121:0/201818605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37340669b0 0x7f3734158170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f371c009810 tx=0x7f371c009060).ready entity=mon.2 client_cookie=cce439580353fe18 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:39.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f3732ffd700 1 -- 172.21.15.121:0/201818605 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c005880 con 0x7f37340669b0 2023-11-15T03:09:39.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f3732ffd700 1 -- 172.21.15.121:0/201818605 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371c0059e0 con 0x7f37340669b0 2023-11-15T03:09:39.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f3732ffd700 1 -- 172.21.15.121:0/201818605 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c005d60 con 0x7f37340669b0 2023-11-15T03:09:39.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f3738f79700 1 -- 172.21.15.121:0/201818605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37340669b0 msgr2=0x7f3734158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:39.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f3738f79700 1 --2- 172.21.15.121:0/201818605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37340669b0 0x7f3734158170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f371c009810 tx=0x7f371c009060).stop 2023-11-15T03:09:39.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f3738f79700 1 -- 172.21.15.121:0/201818605 shutdown_connections 2023-11-15T03:09:39.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f3738f79700 1 --2- 172.21.15.121:0/201818605 >> v2:172.21.15.190:3300/0 conn(0x7f37340a8750 0x7f37340a8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:39.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f3738f79700 1 --2- 172.21.15.121:0/201818605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37340669b0 0x7f3734158170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:39.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f3738f79700 1 -- 172.21.15.121:0/201818605 >> 172.21.15.121:0/201818605 conn(0x7f3734067a60 msgr2=0x7f3734067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:39.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f3738f79700 1 -- 172.21.15.121:0/201818605 shutdown_connections 2023-11-15T03:09:39.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.538+0000 7f3738f79700 1 -- 172.21.15.121:0/201818605 wait complete. 2023-11-15T03:09:39.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.542+0000 7f3738f79700 1 Processor -- start 2023-11-15T03:09:39.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.542+0000 7f3738f79700 1 -- start start 2023-11-15T03:09:39.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f3738f79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37340669b0 0x7f3734158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:39.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f3738f79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37340a8750 0x7f373415a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:39.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f3738f79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3734157d90 0x7f37340beb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:39.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f3738f79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373415a430 con 0x7f37340669b0 2023-11-15T03:09:39.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f3738f79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3734159e00 con 0x7f37340a8750 2023-11-15T03:09:39.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f3738f79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3734168e70 con 0x7f3734157d90 2023-11-15T03:09:39.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f37337fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37340669b0 0x7f3734158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:39.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f37337fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37340669b0 0x7f3734158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53518/0 (socket says 172.21.15.121:53518) 2023-11-15T03:09:39.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f37337fe700 1 -- 172.21.15.121:0/645043580 learned_addr learned my addr 172.21.15.121:0/645043580 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:39.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f372affd700 1 --2- 172.21.15.121:0/645043580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37340a8750 0x7f373415a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:39.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f3733fff700 1 --2- 172.21.15.121:0/645043580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3734157d90 0x7f37340beb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:39.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f37337fe700 1 -- 172.21.15.121:0/645043580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3734157d90 msgr2=0x7f37340beb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:39.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f37337fe700 1 --2- 172.21.15.121:0/645043580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3734157d90 0x7f37340beb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:39.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f37337fe700 1 -- 172.21.15.121:0/645043580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37340a8750 msgr2=0x7f373415a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:39.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f37337fe700 1 --2- 172.21.15.121:0/645043580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37340a8750 0x7f373415a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:39.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f37337fe700 1 -- 172.21.15.121:0/645043580 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f372c009a20 con 0x7f37340669b0 2023-11-15T03:09:39.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f37337fe700 1 --2- 172.21.15.121:0/645043580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37340669b0 0x7f3734158170 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 rx=0x7f3724001eb0 tx=0x7f3724007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:39.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f372bfff700 1 -- 172.21.15.121:0/645043580 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372400f470 con 0x7f37340669b0 2023-11-15T03:09:39.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f372bfff700 1 -- 172.21.15.121:0/645043580 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372400f5f0 con 0x7f37340669b0 2023-11-15T03:09:39.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f3738f79700 1 -- 172.21.15.121:0/645043580 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371c009300 con 0x7f37340669b0 2023-11-15T03:09:39.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f372bfff700 1 -- 172.21.15.121:0/645043580 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372400f9b0 con 0x7f37340669b0 2023-11-15T03:09:39.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.546+0000 7f3738f79700 1 -- 172.21.15.121:0/645043580 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3734168b40 con 0x7f37340669b0 2023-11-15T03:09:39.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.550+0000 7f3738f79700 1 -- 172.21.15.121:0/645043580 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f373406bcf0 con 0x7f37340669b0 2023-11-15T03:09:39.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.550+0000 7f372bfff700 1 -- 172.21.15.121:0/645043580 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f372400fd20 con 0x7f37340669b0 2023-11-15T03:09:39.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.554+0000 7f372bfff700 1 --2- 172.21.15.121:0/645043580 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f371405a510 0x7f371405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:39.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.554+0000 7f372bfff700 1 -- 172.21.15.121:0/645043580 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9104+0+0 (secure 0 0 0) 0x7f37240749f0 con 0x7f37340669b0 2023-11-15T03:09:39.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.554+0000 7f372affd700 1 --2- 172.21.15.121:0/645043580 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f371405a510 0x7f371405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:39.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.554+0000 7f372affd700 1 --2- 172.21.15.121:0/645043580 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f371405a510 0x7f371405c9d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f371c00b340 tx=0x7f371c016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:39.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.554+0000 7f372bfff700 1 -- 172.21.15.121:0/645043580 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3724047590 con 0x7f37340669b0 2023-11-15T03:09:39.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:39.770+0000 7f3738f79700 1 -- 172.21.15.121:0/645043580 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f37340c0400 con 0x7f37340669b0 2023-11-15T03:09:40.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.350+0000 7f372bfff700 1 -- 172.21.15.121:0/645043580 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v291) v1 ==== 101+0+0 (secure 0 0 0) 0x7f372404abb0 con 0x7f37340669b0 2023-11-15T03:09:40.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.354+0000 7f3738f79700 1 -- 172.21.15.121:0/645043580 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f371405a510 msgr2=0x7f371405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:40.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.354+0000 7f3738f79700 1 --2- 172.21.15.121:0/645043580 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f371405a510 0x7f371405c9d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f371c00b340 tx=0x7f371c016000).stop 2023-11-15T03:09:40.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.354+0000 7f3738f79700 1 -- 172.21.15.121:0/645043580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37340669b0 msgr2=0x7f3734158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:40.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.354+0000 7f3738f79700 1 --2- 172.21.15.121:0/645043580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37340669b0 0x7f3734158170 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 rx=0x7f3724001eb0 tx=0x7f3724007c70).stop 2023-11-15T03:09:40.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.354+0000 7f3738f79700 1 -- 172.21.15.121:0/645043580 shutdown_connections 2023-11-15T03:09:40.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.354+0000 7f3738f79700 1 --2- 172.21.15.121:0/645043580 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f371405a510 0x7f371405c9d0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.354+0000 7f3738f79700 1 --2- 172.21.15.121:0/645043580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3734157d90 0x7f37340beb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.354+0000 7f3738f79700 1 --2- 172.21.15.121:0/645043580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37340a8750 0x7f373415a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.354+0000 7f3738f79700 1 --2- 172.21.15.121:0/645043580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37340669b0 0x7f3734158170 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.354+0000 7f3738f79700 1 -- 172.21.15.121:0/645043580 >> 172.21.15.121:0/645043580 conn(0x7f3734067a60 msgr2=0x7f37341517d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:40.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.358+0000 7f3738f79700 1 -- 172.21.15.121:0/645043580 shutdown_connections 2023-11-15T03:09:40.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.358+0000 7f3738f79700 1 -- 172.21.15.121:0/645043580 wait complete. 2023-11-15T03:09:40.369 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:09:40.390 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:09:40.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f1456f35d00 1 Processor -- start 2023-11-15T03:09:40.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f1456f35d00 1 -- start start 2023-11-15T03:09:40.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f1456f35d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1d3bff6c0 0x55e1d3c451a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f1456f35d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e1d3c49b00 0x55e1d3c45690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f1456f35d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1d3cd8670 0x55e1d3cd97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f1456f35d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1d3b78220 con 0x55e1d3cd8670 2023-11-15T03:09:40.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f1456f35d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1d3c32540 con 0x55e1d3bff6c0 2023-11-15T03:09:40.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f1456f35d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e1d3cdb8c0 con 0x55e1d3c49b00 2023-11-15T03:09:40.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f144bc46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1d3cd8670 0x55e1d3cd97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f144bc46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1d3cd8670 0x55e1d3cd97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53530/0 (socket says 172.21.15.121:53530) 2023-11-15T03:09:40.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f144bc46700 1 -- 172.21.15.121:0/2131031982 learned_addr learned my addr 172.21.15.121:0/2131031982 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:40.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f1442c44700 1 --2- 172.21.15.121:0/2131031982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e1d3c49b00 0x55e1d3c45690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f144b445700 1 --2- 172.21.15.121:0/2131031982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1d3bff6c0 0x55e1d3c451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f144bc46700 1 -- 172.21.15.121:0/2131031982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e1d3c49b00 msgr2=0x55e1d3c45690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:40.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f144bc46700 1 --2- 172.21.15.121:0/2131031982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e1d3c49b00 0x55e1d3c45690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f144bc46700 1 -- 172.21.15.121:0/2131031982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1d3bff6c0 msgr2=0x55e1d3c451a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:40.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f144bc46700 1 --2- 172.21.15.121:0/2131031982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1d3bff6c0 0x55e1d3c451a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f144bc46700 1 -- 172.21.15.121:0/2131031982 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1434005cf0 con 0x55e1d3cd8670 2023-11-15T03:09:40.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f144b445700 1 --2- 172.21.15.121:0/2131031982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1d3bff6c0 0x55e1d3c451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:40.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f144bc46700 1 --2- 172.21.15.121:0/2131031982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1d3cd8670 0x55e1d3cd97e0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 rx=0x7f1444001620 tx=0x7f1444006b70).ready entity=mon.0 client_cookie=1c3fc032c9c8065f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:40.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f144a443700 1 -- 172.21.15.121:0/2131031982 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444006320 con 0x55e1d3cd8670 2023-11-15T03:09:40.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.442+0000 7f144a443700 1 -- 172.21.15.121:0/2131031982 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1444004540 con 0x55e1d3cd8670 2023-11-15T03:09:40.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.446+0000 7f144a443700 1 -- 172.21.15.121:0/2131031982 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14440048c0 con 0x55e1d3cd8670 2023-11-15T03:09:40.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.446+0000 7f1456f35d00 1 -- 172.21.15.121:0/2131031982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1d3cd8670 msgr2=0x55e1d3cd97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:40.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.446+0000 7f1456f35d00 1 --2- 172.21.15.121:0/2131031982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1d3cd8670 0x55e1d3cd97e0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 rx=0x7f1444001620 tx=0x7f1444006b70).stop 2023-11-15T03:09:40.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.446+0000 7f1456f35d00 1 -- 172.21.15.121:0/2131031982 shutdown_connections 2023-11-15T03:09:40.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.446+0000 7f1456f35d00 1 --2- 172.21.15.121:0/2131031982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1d3cd8670 0x55e1d3cd97e0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.446+0000 7f1456f35d00 1 --2- 172.21.15.121:0/2131031982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e1d3c49b00 0x55e1d3c45690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.446+0000 7f1456f35d00 1 --2- 172.21.15.121:0/2131031982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1d3bff6c0 0x55e1d3c451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.446+0000 7f1456f35d00 1 -- 172.21.15.121:0/2131031982 >> 172.21.15.121:0/2131031982 conn(0x55e1d3c4f080 msgr2=0x55e1d3cd7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:40.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.446+0000 7f1456f35d00 1 -- 172.21.15.121:0/2131031982 shutdown_connections 2023-11-15T03:09:40.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.446+0000 7f1456f35d00 1 -- 172.21.15.121:0/2131031982 wait complete. 2023-11-15T03:09:40.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.446+0000 7f1456f35d00 1 Processor -- start 2023-11-15T03:09:40.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.446+0000 7f1456f35d00 1 -- start start 2023-11-15T03:09:40.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.446+0000 7f1456f35d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1d3bff6c0 0x55e1d3cd22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.446+0000 7f1456f35d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1d3c49b00 0x55e1d3cdec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.446+0000 7f1456f35d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e1d3cd8670 0x55e1d3cdf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.446+0000 7f1456f35d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1d3ce14b0 con 0x55e1d3c49b00 2023-11-15T03:09:40.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.446+0000 7f1456f35d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1d3c492a0 con 0x55e1d3bff6c0 2023-11-15T03:09:40.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.446+0000 7f1456f35d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e1d3ce1b30 con 0x55e1d3cd8670 2023-11-15T03:09:40.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.450+0000 7f1442c44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1d3c49b00 0x55e1d3cdec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.450+0000 7f1442c44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1d3c49b00 0x55e1d3cdec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53544/0 (socket says 172.21.15.121:53544) 2023-11-15T03:09:40.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.450+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 learned_addr learned my addr 172.21.15.121:0/70760488 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:40.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.450+0000 7f144bc46700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e1d3cd8670 0x55e1d3cdf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.450+0000 7f144b445700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1d3bff6c0 0x55e1d3cd22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.450+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e1d3cd8670 msgr2=0x55e1d3cdf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:40.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.450+0000 7f1442c44700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e1d3cd8670 0x55e1d3cdf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.450+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1d3bff6c0 msgr2=0x55e1d3cd22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:40.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.450+0000 7f1442c44700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1d3bff6c0 0x55e1d3cd22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.450+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1444006e50 con 0x55e1d3c49b00 2023-11-15T03:09:40.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.450+0000 7f144b445700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1d3bff6c0 0x55e1d3cd22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:09:40.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.450+0000 7f1442c44700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1d3c49b00 0x55e1d3cdec60 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 rx=0x7f143400afd0 tx=0x7f1434016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:40.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.450+0000 7f1448c40700 1 -- 172.21.15.121:0/70760488 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434013650 con 0x55e1d3c49b00 2023-11-15T03:09:40.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.450+0000 7f1448c40700 1 -- 172.21.15.121:0/70760488 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14340137d0 con 0x55e1d3c49b00 2023-11-15T03:09:40.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.450+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e1d3ba1770 con 0x55e1d3c49b00 2023-11-15T03:09:40.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.450+0000 7f1448c40700 1 -- 172.21.15.121:0/70760488 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434013b90 con 0x55e1d3c49b00 2023-11-15T03:09:40.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.450+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1d3b70a70 con 0x55e1d3c49b00 2023-11-15T03:09:40.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.454+0000 7f1448c40700 1 -- 172.21.15.121:0/70760488 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f143401d3e0 con 0x55e1d3c49b00 2023-11-15T03:09:40.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.454+0000 7f1448c40700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f142c058ff0 0x7f142c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.454+0000 7f1448c40700 1 -- 172.21.15.121:0/70760488 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9104+0+0 (secure 0 0 0) 0x7f1434075490 con 0x55e1d3c49b00 2023-11-15T03:09:40.470 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:09:40.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.454+0000 7f144b445700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f142c058ff0 0x7f142c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.454+0000 7f1456f35d00 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55e1d3ce0740 0x55e1d3ce2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.454+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3ce2a10 con 0x55e1d3ce0740 2023-11-15T03:09:40.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.454+0000 7f144bc46700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55e1d3ce0740 0x55e1d3ce2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.454+0000 7f144b445700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f142c058ff0 0x7f142c05b4b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f143c009b80 tx=0x7f143c005cd0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:40.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.454+0000 7f144bc46700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55e1d3ce0740 0x55e1d3ce2520 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:40.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.458+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v127'389 uv382 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3ce0740 2023-11-15T03:09:40.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.458+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3cdf640 con 0x55e1d3ce0740 2023-11-15T03:09:40.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.462+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v127'390 uv390 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3ce0740 2023-11-15T03:09:40.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.462+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cf5590 con 0x55e1d3ce0740 2023-11-15T03:09:40.476 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:09:40.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.462+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v127'391 uv391 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3ce0740 2023-11-15T03:09:40.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.462+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cf6950 con 0x55e1d3ce0740 2023-11-15T03:09:40.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.466+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v127'392 uv392 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3ce0740 2023-11-15T03:09:40.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.466+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cf6d00 con 0x55e1d3ce0740 2023-11-15T03:09:40.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.466+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v127'393 uv393 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3ce0740 2023-11-15T03:09:40.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.466+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cf5940 con 0x55e1d3ce0740 2023-11-15T03:09:40.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.470+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v127'394 uv394 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3ce0740 2023-11-15T03:09:40.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.470+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cf5d40 con 0x55e1d3ce0740 2023-11-15T03:09:40.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.474+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v127'395 uv388 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3ce0740 2023-11-15T03:09:40.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.474+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3cf6140 con 0x55e1d3ce0740 2023-11-15T03:09:40.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.474+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v127'396 uv396 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3ce0740 2023-11-15T03:09:40.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.474+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cf6140 con 0x55e1d3ce0740 2023-11-15T03:09:40.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.478+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v127'397 uv397 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3ce0740 2023-11-15T03:09:40.490 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:09:40.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.478+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cf60f0 con 0x55e1d3ce0740 2023-11-15T03:09:40.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.482+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v127'398 uv398 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3ce0740 2023-11-15T03:09:40.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.482+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cf60f0 con 0x55e1d3ce0740 2023-11-15T03:09:40.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.486+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v127'399 uv399 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3ce0740 2023-11-15T03:09:40.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.486+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cf60f0 con 0x55e1d3ce0740 2023-11-15T03:09:40.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.490+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v127'400 uv400 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3ce0740 2023-11-15T03:09:40.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.490+0000 7f1456f35d00 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e1d3cf64f0 0x55e1d3cf95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.490+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cf60f0 con 0x55e1d3cf64f0 2023-11-15T03:09:40.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.490+0000 7f144b445700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e1d3cf64f0 0x55e1d3cf95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.490+0000 7f144b445700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e1d3cf64f0 0x55e1d3cf95d0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:40.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.494+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v127'614 uv601 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.494+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3cf9ac0 con 0x55e1d3cf64f0 2023-11-15T03:09:40.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.498+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v127'615 uv615 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.498+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cf9e70 con 0x55e1d3cf64f0 2023-11-15T03:09:40.511 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:09:40.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.502+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v127'616 uv616 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.502+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cfb230 con 0x55e1d3cf64f0 2023-11-15T03:09:40.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.506+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v127'617 uv617 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.506+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cfb5e0 con 0x55e1d3cf64f0 2023-11-15T03:09:40.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.506+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v127'618 uv618 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.506+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cfa220 con 0x55e1d3cf64f0 2023-11-15T03:09:40.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.510+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v127'619 uv619 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.510+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cfab10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.514+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v127'620 uv620 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.514+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cfb990 con 0x55e1d3cf64f0 2023-11-15T03:09:40.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.518+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v127'621 uv607 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.518+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3cfb990 con 0x55e1d3cf64f0 2023-11-15T03:09:40.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.518+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v127'622 uv622 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.518+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cfb990 con 0x55e1d3cf64f0 2023-11-15T03:09:40.533 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:09:40.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.522+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v127'623 uv623 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.522+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cfc9a0 con 0x55e1d3cf64f0 2023-11-15T03:09:40.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.526+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v127'624 uv624 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.526+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cfc9a0 con 0x55e1d3cf64f0 2023-11-15T03:09:40.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.530+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v127'625 uv625 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.530+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cfb990 con 0x55e1d3cf64f0 2023-11-15T03:09:40.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.530+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v127'626 uv626 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.530+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cfb990 con 0x55e1d3cf64f0 2023-11-15T03:09:40.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.534+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v127'627 uv613 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.534+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3cfb990 con 0x55e1d3cf64f0 2023-11-15T03:09:40.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.538+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v127'628 uv628 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.538+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cfb990 con 0x55e1d3cf64f0 2023-11-15T03:09:40.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.542+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v127'629 uv629 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.551 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:09:40.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.542+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cfc9a0 con 0x55e1d3cf64f0 2023-11-15T03:09:40.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.542+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v127'630 uv630 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.542+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cfc9a0 con 0x55e1d3cf64f0 2023-11-15T03:09:40.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.546+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v127'631 uv631 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.546+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cfb990 con 0x55e1d3cf64f0 2023-11-15T03:09:40.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.550+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v127'632 uv632 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.550+0000 7f1456f35d00 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55e1d3cfbea0 0x55e1d3cfe290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.550+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cfe780 con 0x55e1d3cfbea0 2023-11-15T03:09:40.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.550+0000 7f1442c44700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55e1d3cfbea0 0x55e1d3cfe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.550+0000 7f1442c44700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55e1d3cfbea0 0x55e1d3cfe290 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:40.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.554+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v127'335 uv329 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.554+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3cfeb30 con 0x55e1d3cfbea0 2023-11-15T03:09:40.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.558+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v127'336 uv336 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.558+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cfeee0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.570 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:09:40.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.558+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v127'337 uv337 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.562+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cff780 con 0x55e1d3cfbea0 2023-11-15T03:09:40.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.562+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v127'338 uv338 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.562+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cffb30 con 0x55e1d3cfbea0 2023-11-15T03:09:40.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.566+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v127'339 uv339 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.566+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cffee0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.570+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v127'340 uv334 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.570+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3d00290 con 0x55e1d3cfbea0 2023-11-15T03:09:40.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.570+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v127'341 uv341 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.574+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d00640 con 0x55e1d3cfbea0 2023-11-15T03:09:40.585 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:09:40.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.574+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v127'342 uv342 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.574+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d00640 con 0x55e1d3cfbea0 2023-11-15T03:09:40.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.578+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v127'343 uv343 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.578+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d00640 con 0x55e1d3cfbea0 2023-11-15T03:09:40.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.582+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v127'344 uv344 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.582+0000 7f1456f35d00 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e1d3d00b50 0x55e1d3d02f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.582+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d03550 con 0x55e1d3d00b50 2023-11-15T03:09:40.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.582+0000 7f144bc46700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e1d3d00b50 0x55e1d3d02f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.582+0000 7f144bc46700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e1d3d00b50 0x55e1d3d02f40 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:40.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.586+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v127'392 uv384 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.586+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3d03930 con 0x55e1d3d00b50 2023-11-15T03:09:40.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.590+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v127'393 uv393 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.590+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d03930 con 0x55e1d3d00b50 2023-11-15T03:09:40.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.590+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v127'394 uv394 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.602 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:09:40.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.594+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d079c0 con 0x55e1d3d00b50 2023-11-15T03:09:40.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.598+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v127'395 uv395 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.598+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d079c0 con 0x55e1d3d00b50 2023-11-15T03:09:40.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.598+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v127'396 uv396 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.598+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d03930 con 0x55e1d3d00b50 2023-11-15T03:09:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.602+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v127'397 uv390 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.602+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3d03930 con 0x55e1d3d00b50 2023-11-15T03:09:40.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.606+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v127'398 uv398 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.606+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d03930 con 0x55e1d3d00b50 2023-11-15T03:09:40.618 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:09:40.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.606+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v127'399 uv399 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.606+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d04940 con 0x55e1d3d00b50 2023-11-15T03:09:40.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.610+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v127'400 uv400 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.610+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d04940 con 0x55e1d3d00b50 2023-11-15T03:09:40.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.614+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v127'401 uv401 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.614+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d03930 con 0x55e1d3d00b50 2023-11-15T03:09:40.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.614+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v127'402 uv402 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.614+0000 7f1456f35d00 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55e1d3d03fb0 0x55e1d3d063a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.614+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d06960 con 0x55e1d3d03fb0 2023-11-15T03:09:40.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.618+0000 7f144b445700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55e1d3d03fb0 0x55e1d3d063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.618+0000 7f144b445700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55e1d3d03fb0 0x55e1d3d063a0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:40.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.618+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v127'195 uv194 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3d03fb0 2023-11-15T03:09:40.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.618+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3d06d40 con 0x55e1d3d03fb0 2023-11-15T03:09:40.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.622+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v127'196 uv196 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3d03fb0 2023-11-15T03:09:40.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.622+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d06d40 con 0x55e1d3d03fb0 2023-11-15T03:09:40.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.626+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v127'197 uv197 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3d03fb0 2023-11-15T03:09:40.635 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:09:40.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.626+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d07d50 con 0x55e1d3d03fb0 2023-11-15T03:09:40.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.626+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v127'198 uv198 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3d03fb0 2023-11-15T03:09:40.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.626+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d07d50 con 0x55e1d3d03fb0 2023-11-15T03:09:40.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.630+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v127'199 uv199 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3d03fb0 2023-11-15T03:09:40.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.630+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d06d40 con 0x55e1d3d03fb0 2023-11-15T03:09:40.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.634+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v127'200 uv200 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3d03fb0 2023-11-15T03:09:40.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.634+0000 7f1456f35d00 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e1d3d073c0 0x55e1d3d097b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.634+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d09ef0 con 0x55e1d3d073c0 2023-11-15T03:09:40.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.634+0000 7f1442c44700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e1d3d073c0 0x55e1d3d097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.634+0000 7f1442c44700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e1d3d073c0 0x55e1d3d097b0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:40.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.638+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v127'140 uv139 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.638+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3d073c0 2023-11-15T03:09:40.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.642+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v127'141 uv141 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.642+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3d073c0 2023-11-15T03:09:40.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.642+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v127'142 uv142 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.654 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:09:40.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.642+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3d073c0 2023-11-15T03:09:40.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.646+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v127'143 uv143 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.646+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3d073c0 2023-11-15T03:09:40.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.650+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v127'195 uv194 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.650+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3d073c0 2023-11-15T03:09:40.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.654+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v127'196 uv196 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.654+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3d073c0 2023-11-15T03:09:40.669 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:09:40.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.658+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v127'197 uv197 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.658+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0b2e0 con 0x55e1d3d073c0 2023-11-15T03:09:40.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.662+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v127'198 uv198 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.662+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0b2e0 con 0x55e1d3d073c0 2023-11-15T03:09:40.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.666+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v127'199 uv199 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.666+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3d073c0 2023-11-15T03:09:40.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.666+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v127'200 uv200 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.666+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3cf64f0 2023-11-15T03:09:40.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.670+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v127'360 uv354 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.674+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3cf64f0 2023-11-15T03:09:40.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.678+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v127'361 uv361 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.678+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3cf64f0 2023-11-15T03:09:40.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.678+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v127'362 uv362 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.690 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:09:40.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.682+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3cf64f0 2023-11-15T03:09:40.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.682+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v127'363 uv363 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.682+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3cf64f0 2023-11-15T03:09:40.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.686+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v127'364 uv364 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.686+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3cf64f0 2023-11-15T03:09:40.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.690+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v127'365 uv365 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.690+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3cf64f0 2023-11-15T03:09:40.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.694+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v127'366 uv359 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.694+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3cf64f0 2023-11-15T03:09:40.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.694+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v127'367 uv367 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.694+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3cf64f0 2023-11-15T03:09:40.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.698+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v127'368 uv368 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.709 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:09:40.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.698+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3cf64f0 2023-11-15T03:09:40.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.702+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v127'369 uv369 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.702+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3cf64f0 2023-11-15T03:09:40.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.702+0000 7f144b445700 1 -- 172.21.15.121:0/70760488 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v127'370 uv370 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143c007c10 con 0x55e1d3cf64f0 2023-11-15T03:09:40.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.702+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.706+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v127'163 uv162 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.706+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.710+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v127'164 uv164 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.710+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.710+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v127'165 uv165 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.722 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:09:40.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.710+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0a2d0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.714+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v127'166 uv166 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.714+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cff310 con 0x55e1d3cfbea0 2023-11-15T03:09:40.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.718+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v127'167 uv167 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3cfbea0 2023-11-15T03:09:40.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.718+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cff310 con 0x55e1d3d073c0 2023-11-15T03:09:40.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.718+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v127'195 uv194 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.718+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3cff290 con 0x55e1d3d073c0 2023-11-15T03:09:40.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.722+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v127'196 uv196 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.722+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cff290 con 0x55e1d3d073c0 2023-11-15T03:09:40.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.726+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v127'197 uv197 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.737 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:09:40.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.726+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cff290 con 0x55e1d3d073c0 2023-11-15T03:09:40.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.730+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v127'198 uv198 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.730+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cff290 con 0x55e1d3d073c0 2023-11-15T03:09:40.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.730+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v127'199 uv199 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.730+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cff290 con 0x55e1d3d073c0 2023-11-15T03:09:40.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.734+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v127'200 uv200 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.734+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cff290 con 0x55e1d3d073c0 2023-11-15T03:09:40.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.738+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v127'164 uv163 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.738+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3cff290 con 0x55e1d3d073c0 2023-11-15T03:09:40.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.742+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v127'165 uv165 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.742+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cff290 con 0x55e1d3d073c0 2023-11-15T03:09:40.753 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:09:40.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.742+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v127'166 uv166 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.742+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cff290 con 0x55e1d3d073c0 2023-11-15T03:09:40.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.746+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v127'167 uv167 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.746+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cff290 con 0x55e1d3d073c0 2023-11-15T03:09:40.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.750+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v127'168 uv168 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.750+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cff290 con 0x55e1d3d073c0 2023-11-15T03:09:40.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.754+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v127'162 uv161 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.754+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3cff290 con 0x55e1d3d073c0 2023-11-15T03:09:40.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.754+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v127'163 uv163 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.754+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3cff290 con 0x55e1d3d073c0 2023-11-15T03:09:40.768 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:09:40.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.758+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v127'164 uv164 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.758+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0a7c0 con 0x55e1d3d073c0 2023-11-15T03:09:40.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.762+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v127'165 uv165 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.762+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0d1a0 con 0x55e1d3d073c0 2023-11-15T03:09:40.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.766+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v127'166 uv166 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.766+0000 7f1456f35d00 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55e1d3d0d380 0x55e1d3d0d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.766+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d0d380 2023-11-15T03:09:40.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.770+0000 7f144bc46700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55e1d3d0d380 0x55e1d3d0d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.770+0000 7f144bc46700 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55e1d3d0d380 0x55e1d3d0d7f0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:40.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.774+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v127'130 uv129 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d0d380 2023-11-15T03:09:40.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.774+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d0d380 2023-11-15T03:09:40.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.774+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v127'131 uv131 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d0d380 2023-11-15T03:09:40.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.774+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d0d380 2023-11-15T03:09:40.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.778+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v127'132 uv132 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d0d380 2023-11-15T03:09:40.790 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:09:40.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.778+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d0d380 2023-11-15T03:09:40.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.782+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v127'133 uv133 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d0d380 2023-11-15T03:09:40.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.782+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d073c0 2023-11-15T03:09:40.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.786+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v127'195 uv194 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.786+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d073c0 2023-11-15T03:09:40.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.790+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v127'196 uv196 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.790+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d073c0 2023-11-15T03:09:40.802 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:09:40.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.790+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v127'197 uv197 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.790+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0ecf0 con 0x55e1d3d073c0 2023-11-15T03:09:40.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.794+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v127'198 uv198 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.794+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0ecf0 con 0x55e1d3d073c0 2023-11-15T03:09:40.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.798+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v127'199 uv199 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.798+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d073c0 2023-11-15T03:09:40.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.802+0000 7f1442c44700 1 -- 172.21.15.121:0/70760488 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v127'200 uv200 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f143404a0b0 con 0x55e1d3d073c0 2023-11-15T03:09:40.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.802+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d00b50 2023-11-15T03:09:40.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.802+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v127'166 uv165 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.802+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d00b50 2023-11-15T03:09:40.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.806+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v127'167 uv167 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.806+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d00b50 2023-11-15T03:09:40.818 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:09:40.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.810+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v127'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.810+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d00b50 2023-11-15T03:09:40.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.810+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v127'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.810+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d00b50 2023-11-15T03:09:40.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.814+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v127'170 uv170 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.814+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d00b50 2023-11-15T03:09:40.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.818+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v127'390 uv382 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.818+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d00b50 2023-11-15T03:09:40.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.822+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v127'391 uv391 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.822+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d00b50 2023-11-15T03:09:40.833 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:09:40.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.822+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v127'392 uv392 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.822+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d00b50 2023-11-15T03:09:40.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.826+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v127'393 uv393 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.826+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d00b50 2023-11-15T03:09:40.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.830+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v127'394 uv394 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.830+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d00b50 2023-11-15T03:09:40.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.830+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v127'395 uv389 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.830+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d00b50 2023-11-15T03:09:40.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.834+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v127'396 uv396 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.834+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d00b50 2023-11-15T03:09:40.847 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:09:40.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.838+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v127'397 uv397 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.838+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0ecf0 con 0x55e1d3d00b50 2023-11-15T03:09:40.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.838+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v127'398 uv398 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.838+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0ecf0 con 0x55e1d3d00b50 2023-11-15T03:09:40.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.842+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v127'399 uv399 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.842+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0dce0 con 0x55e1d3d00b50 2023-11-15T03:09:40.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.846+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v127'400 uv400 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.846+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55e1d3d0e1d0 con 0x55e1d3d00b50 2023-11-15T03:09:40.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v127'401 uv401 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f14440065f0 con 0x55e1d3d00b50 2023-11-15T03:09:40.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55e1d3cfbea0 msgr2=0x55e1d3cfe290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:40.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55e1d3cfbea0 0x55e1d3cfe290 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55e1d3ce0740 msgr2=0x55e1d3ce2520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:40.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55e1d3ce0740 0x55e1d3ce2520 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55e1d3d03fb0 msgr2=0x55e1d3d063a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:40.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55e1d3d03fb0 0x55e1d3d063a0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e1d3cf64f0 msgr2=0x55e1d3cf95d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:40.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55e1d3cf64f0 0x55e1d3cf95d0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55e1d3d0d380 msgr2=0x55e1d3d0d7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:40.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55e1d3d0d380 0x55e1d3d0d7f0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e1d3d00b50 msgr2=0x55e1d3d02f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:40.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55e1d3d00b50 0x55e1d3d02f40 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e1d3d073c0 msgr2=0x55e1d3d097b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:40.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55e1d3d073c0 0x55e1d3d097b0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 reap_dead start 2023-11-15T03:09:40.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 reap_dead start 2023-11-15T03:09:40.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 reap_dead start 2023-11-15T03:09:40.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f142c058ff0 msgr2=0x7f142c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:40.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f142c058ff0 0x7f142c05b4b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f143c009b80 tx=0x7f143c005cd0).stop 2023-11-15T03:09:40.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f144bc46700 1 -- 172.21.15.121:0/70760488 reap_dead start 2023-11-15T03:09:40.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1d3c49b00 msgr2=0x55e1d3cdec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:40.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1d3c49b00 0x55e1d3cdec60 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 rx=0x7f143400afd0 tx=0x7f1434016000).stop 2023-11-15T03:09:40.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 shutdown_connections 2023-11-15T03:09:40.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 --2- 172.21.15.121:0/70760488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1d3c49b00 0x55e1d3cdec60 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 >> 172.21.15.121:0/70760488 conn(0x55e1d3c4f080 msgr2=0x55e1d3c43fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:40.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 shutdown_connections 2023-11-15T03:09:40.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.850+0000 7f1456f35d00 1 -- 172.21.15.121:0/70760488 wait complete. 2023-11-15T03:09:40.867 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 1.00000000 subdir_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-11-15T03:09:40.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbda5331d00 1 Processor -- start 2023-11-15T03:09:40.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbda5331d00 1 -- start start 2023-11-15T03:09:40.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbda5331d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eba2483930 0x55eba2558d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbda5331d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eba24c8f60 0x55eba2554010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbda5331d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55eba255a140 0x55eba2554500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbda5331d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba242dcc0 con 0x55eba2483930 2023-11-15T03:09:40.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbda5331d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba23f8220 con 0x55eba24c8f60 2023-11-15T03:09:40.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbda5331d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eba24cbc90 con 0x55eba255a140 2023-11-15T03:09:40.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbd998b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eba2483930 0x55eba2558d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbd998b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eba2483930 0x55eba2558d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53552/0 (socket says 172.21.15.121:53552) 2023-11-15T03:09:40.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbd998b7700 1 -- 172.21.15.121:0/2354106058 learned_addr learned my addr 172.21.15.121:0/2354106058 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:40.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbd9a0b8700 1 --2- 172.21.15.121:0/2354106058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55eba255a140 0x55eba2554500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbd990b6700 1 --2- 172.21.15.121:0/2354106058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eba24c8f60 0x55eba2554010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbd998b7700 1 -- 172.21.15.121:0/2354106058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55eba255a140 msgr2=0x55eba2554500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:40.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbd998b7700 1 --2- 172.21.15.121:0/2354106058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55eba255a140 0x55eba2554500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbd998b7700 1 -- 172.21.15.121:0/2354106058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eba24c8f60 msgr2=0x55eba2554010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:40.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbd998b7700 1 --2- 172.21.15.121:0/2354106058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eba24c8f60 0x55eba2554010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbd998b7700 1 -- 172.21.15.121:0/2354106058 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9400a0e0 con 0x55eba2483930 2023-11-15T03:09:40.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbd990b6700 1 --2- 172.21.15.121:0/2354106058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eba24c8f60 0x55eba2554010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:40.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbd998b7700 1 --2- 172.21.15.121:0/2354106058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eba2483930 0x55eba2558d50 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 rx=0x7fbd8c001d70 tx=0x7fbd8c001820).ready entity=mon.0 client_cookie=a9f860dabb5e8a32 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:40.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.970+0000 7fbd8bfff700 1 -- 172.21.15.121:0/2354106058 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c00bc70 con 0x55eba2483930 2023-11-15T03:09:40.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.974+0000 7fbd8bfff700 1 -- 172.21.15.121:0/2354106058 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c00bdd0 con 0x55eba2483930 2023-11-15T03:09:40.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.974+0000 7fbd8bfff700 1 -- 172.21.15.121:0/2354106058 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c00e680 con 0x55eba2483930 2023-11-15T03:09:40.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.974+0000 7fbda5331d00 1 -- 172.21.15.121:0/2354106058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eba2483930 msgr2=0x55eba2558d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:40.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.974+0000 7fbda5331d00 1 --2- 172.21.15.121:0/2354106058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eba2483930 0x55eba2558d50 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 rx=0x7fbd8c001d70 tx=0x7fbd8c001820).stop 2023-11-15T03:09:40.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.974+0000 7fbda5331d00 1 -- 172.21.15.121:0/2354106058 shutdown_connections 2023-11-15T03:09:40.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.974+0000 7fbda5331d00 1 --2- 172.21.15.121:0/2354106058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55eba255a140 0x55eba2554500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.974+0000 7fbda5331d00 1 --2- 172.21.15.121:0/2354106058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eba24c8f60 0x55eba2554010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.974+0000 7fbda5331d00 1 --2- 172.21.15.121:0/2354106058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eba2483930 0x55eba2558d50 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.974+0000 7fbda5331d00 1 -- 172.21.15.121:0/2354106058 >> 172.21.15.121:0/2354106058 conn(0x55eba24c9900 msgr2=0x55eba247f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:40.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.974+0000 7fbda5331d00 1 -- 172.21.15.121:0/2354106058 shutdown_connections 2023-11-15T03:09:40.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.974+0000 7fbda5331d00 1 -- 172.21.15.121:0/2354106058 wait complete. 2023-11-15T03:09:40.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.974+0000 7fbda5331d00 1 Processor -- start 2023-11-15T03:09:40.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.974+0000 7fbda5331d00 1 -- start start 2023-11-15T03:09:40.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbda5331d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55eba2483930 0x55eba253b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbda5331d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eba24c8f60 0x55eba253c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbda5331d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eba255a140 0x55eba2481d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbda5331d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba24818f0 con 0x55eba24c8f60 2023-11-15T03:09:40.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbda5331d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba2480ce0 con 0x55eba255a140 2023-11-15T03:09:40.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbda5331d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eba2480fc0 con 0x55eba2483930 2023-11-15T03:09:40.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbd990b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eba24c8f60 0x55eba253c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbd990b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eba24c8f60 0x55eba253c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53558/0 (socket says 172.21.15.121:53558) 2023-11-15T03:09:40.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbd990b6700 1 -- 172.21.15.121:0/3989799285 learned_addr learned my addr 172.21.15.121:0/3989799285 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:40.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbd9a0b8700 1 --2- 172.21.15.121:0/3989799285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eba255a140 0x55eba2481d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbd998b7700 1 --2- 172.21.15.121:0/3989799285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55eba2483930 0x55eba253b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbd990b6700 1 -- 172.21.15.121:0/3989799285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55eba2483930 msgr2=0x55eba253b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:40.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbd990b6700 1 --2- 172.21.15.121:0/3989799285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55eba2483930 0x55eba253b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbd990b6700 1 -- 172.21.15.121:0/3989799285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eba255a140 msgr2=0x55eba2481d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:40.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbd990b6700 1 --2- 172.21.15.121:0/3989799285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eba255a140 0x55eba2481d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:40.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbd990b6700 1 -- 172.21.15.121:0/3989799285 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd84001e10 con 0x55eba24c8f60 2023-11-15T03:09:40.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbd990b6700 1 --2- 172.21.15.121:0/3989799285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eba24c8f60 0x55eba253c330 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 rx=0x7fbd9000cef0 tx=0x7fbd90001810).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:40.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbd8a7fc700 1 -- 172.21.15.121:0/3989799285 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000abf0 con 0x55eba24c8f60 2023-11-15T03:09:40.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbd8a7fc700 1 -- 172.21.15.121:0/3989799285 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd90008040 con 0x55eba24c8f60 2023-11-15T03:09:40.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbda5331d00 1 -- 172.21.15.121:0/3989799285 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eba23f0a70 con 0x55eba24c8f60 2023-11-15T03:09:40.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbd8a7fc700 1 -- 172.21.15.121:0/3989799285 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90010470 con 0x55eba24c8f60 2023-11-15T03:09:40.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.978+0000 7fbda5331d00 1 -- 172.21.15.121:0/3989799285 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eba23fc010 con 0x55eba24c8f60 2023-11-15T03:09:40.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.982+0000 7fbd8a7fc700 1 -- 172.21.15.121:0/3989799285 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fbd900108a0 con 0x55eba24c8f60 2023-11-15T03:09:40.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.982+0000 7fbd8a7fc700 1 --2- 172.21.15.121:0/3989799285 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbd7c059020 0x7fbd7c05b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.982+0000 7fbd8a7fc700 1 -- 172.21.15.121:0/3989799285 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9104+0+0 (secure 0 0 0) 0x7fbd90073e10 con 0x55eba24c8f60 2023-11-15T03:09:40.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.982+0000 7fbd998b7700 1 --2- 172.21.15.121:0/3989799285 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbd7c059020 0x7fbd7c05b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.982+0000 7fbda5331d00 1 --2- 172.21.15.121:0/3989799285 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55eba24807e0 0x55eba2561e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:40.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.982+0000 7fbda5331d00 1 -- 172.21.15.121:0/3989799285 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=103b] snapc 0=[] ondisk+write+known_if_redirected e127) v8 -- 0x55eba247f550 con 0x55eba24807e0 2023-11-15T03:09:40.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.986+0000 7fbd9a0b8700 1 --2- 172.21.15.121:0/3989799285 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55eba24807e0 0x55eba2561e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:40.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.986+0000 7fbd998b7700 1 --2- 172.21.15.121:0/3989799285 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbd7c059020 0x7fbd7c05b4e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7fbd8c0012d0 tx=0x7fbd8c001820).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:40.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.986+0000 7fbd9a0b8700 1 --2- 172.21.15.121:0/3989799285 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55eba24807e0 0x55eba2561e80 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:41.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.990+0000 7fbd9a0b8700 1 -- 172.21.15.121:0/3989799285 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 1.00000000 [omap-set-vals] v127'402 uv402 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fbd94017040 con 0x55eba24807e0 2023-11-15T03:09:41.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.990+0000 7fbda5331d00 1 -- 172.21.15.121:0/3989799285 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55eba24807e0 msgr2=0x55eba2561e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:41.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.990+0000 7fbda5331d00 1 --2- 172.21.15.121:0/3989799285 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55eba24807e0 0x55eba2561e80 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:41.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.990+0000 7fbda5331d00 1 -- 172.21.15.121:0/3989799285 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbd7c059020 msgr2=0x7fbd7c05b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:41.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.990+0000 7fbda5331d00 1 --2- 172.21.15.121:0/3989799285 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbd7c059020 0x7fbd7c05b4e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7fbd8c0012d0 tx=0x7fbd8c001820).stop 2023-11-15T03:09:41.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.990+0000 7fbda5331d00 1 -- 172.21.15.121:0/3989799285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eba24c8f60 msgr2=0x55eba253c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:41.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.990+0000 7fbda5331d00 1 --2- 172.21.15.121:0/3989799285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eba24c8f60 0x55eba253c330 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 rx=0x7fbd9000cef0 tx=0x7fbd90001810).stop 2023-11-15T03:09:41.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.990+0000 7fbd9a0b8700 1 -- 172.21.15.121:0/3989799285 reap_dead start 2023-11-15T03:09:41.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.990+0000 7fbda5331d00 1 -- 172.21.15.121:0/3989799285 shutdown_connections 2023-11-15T03:09:41.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.990+0000 7fbda5331d00 1 -- 172.21.15.121:0/3989799285 >> 172.21.15.121:0/3989799285 conn(0x55eba24c9900 msgr2=0x55eba24b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:41.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.990+0000 7fbda5331d00 1 -- 172.21.15.121:0/3989799285 shutdown_connections 2023-11-15T03:09:41.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:40.990+0000 7fbda5331d00 1 -- 172.21.15.121:0/3989799285 wait complete. 2023-11-15T03:09:41.006 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:09:41.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.170+0000 7f6a80e79700 1 Processor -- start 2023-11-15T03:09:41.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.170+0000 7f6a80e79700 1 -- start start 2023-11-15T03:09:41.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.174+0000 7f6a80e79700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6a7c0acb60 0x7f6a7c0acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:41.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.174+0000 7f6a80e79700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c0c7e90 con 0x7f6a7c065680 2023-11-15T03:09:41.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.174+0000 7f6a80e79700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c0ad4d0 con 0x7f6a7c14e380 2023-11-15T03:09:41.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.174+0000 7f6a80e79700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a7c0c8320 con 0x7f6a7c0acb60 2023-11-15T03:09:41.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.174+0000 7f6a7bfff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6a7c065680 0x7f6a7c065af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34910/0 (socket says 172.21.15.121:34910) 2023-11-15T03:09:41.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.174+0000 7f6a7bfff700 1 -- 172.21.15.121:0/2737173118 learned_addr learned my addr 172.21.15.121:0/2737173118 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:09:41.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.174+0000 7f6a7affd700 1 --2- 172.21.15.121:0/2737173118 >> v2:172.21.15.190:3300/0 conn(0x7f6a7c0acb60 0x7f6a7c0acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:41.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.174+0000 7f6a7a7fc700 1 -- 172.21.15.121:0/2737173118 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130880447 0 0) 0x7f6a7c0c7e90 con 0x7f6a7c065680 2023-11-15T03:09:41.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.174+0000 7f6a7a7fc700 1 -- 172.21.15.121:0/2737173118 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a640033f0 con 0x7f6a7c065680 2023-11-15T03:09:41.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.174+0000 7f6a7a7fc700 1 -- 172.21.15.121:0/2737173118 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787543216 0 0) 0x7f6a7c0ad4d0 con 0x7f6a7c14e380 2023-11-15T03:09:41.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.174+0000 7f6a7a7fc700 1 -- 172.21.15.121:0/2737173118 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a7c0c7e90 con 0x7f6a7c14e380 2023-11-15T03:09:41.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.174+0000 7f6a7affd700 1 -- 172.21.15.121:0/2737173118 >> v1:172.21.15.190:6789/0 conn(0x7f6a7c14e380 legacy=0x7f6a7c0c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:41.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.174+0000 7f6a7affd700 1 -- 172.21.15.121:0/2737173118 >> v1:172.21.15.121:6789/0 conn(0x7f6a7c065680 legacy=0x7f6a7c065af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:41.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.174+0000 7f6a7affd700 1 -- 172.21.15.121:0/2737173118 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c14e760 con 0x7f6a7c0acb60 2023-11-15T03:09:41.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.174+0000 7f6a7affd700 1 --2- 172.21.15.121:0/2737173118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a7c0acb60 0x7f6a7c0acfe0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f6a70009760 tx=0x7f6a70009060).ready entity=mon.1 client_cookie=2337561a7f0f1a95 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:41.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.174+0000 7f6a7a7fc700 1 -- 172.21.15.121:0/2737173118 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7000d4a0 con 0x7f6a7c0acb60 2023-11-15T03:09:41.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.178+0000 7f6a7a7fc700 1 -- 172.21.15.121:0/2737173118 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7000d600 con 0x7f6a7c0acb60 2023-11-15T03:09:41.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.178+0000 7f6a7a7fc700 1 -- 172.21.15.121:0/2737173118 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7000d980 con 0x7f6a7c0acb60 2023-11-15T03:09:41.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.178+0000 7f6a80e79700 1 -- 172.21.15.121:0/2737173118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a7c0acb60 msgr2=0x7f6a7c0acfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:41.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.178+0000 7f6a80e79700 1 --2- 172.21.15.121:0/2737173118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a7c0acb60 0x7f6a7c0acfe0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f6a70009760 tx=0x7f6a70009060).stop 2023-11-15T03:09:41.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.178+0000 7f6a80e79700 1 -- 172.21.15.121:0/2737173118 shutdown_connections 2023-11-15T03:09:41.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.178+0000 7f6a80e79700 1 --2- 172.21.15.121:0/2737173118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a7c0acb60 0x7f6a7c0acfe0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:41.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.178+0000 7f6a80e79700 1 -- 172.21.15.121:0/2737173118 >> 172.21.15.121:0/2737173118 conn(0x7f6a7c067a60 msgr2=0x7f6a7c067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:41.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.178+0000 7f6a80e79700 1 -- 172.21.15.121:0/2737173118 shutdown_connections 2023-11-15T03:09:41.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.178+0000 7f6a80e79700 1 -- 172.21.15.121:0/2737173118 wait complete. 2023-11-15T03:09:41.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.178+0000 7f6a80e79700 1 Processor -- start 2023-11-15T03:09:41.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.178+0000 7f6a80e79700 1 -- start start 2023-11-15T03:09:41.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a80e79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a7c065680 0x7f6a7c0ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:41.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a80e79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a7c0acb60 0x7f6a7c1641d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:41.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a80e79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a7c14e380 0x7f6a7c152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:41.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a80e79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0c23f0 con 0x7f6a7c14e380 2023-11-15T03:09:41.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a80e79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c164830 con 0x7f6a7c0acb60 2023-11-15T03:09:41.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a80e79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c164b10 con 0x7f6a7c065680 2023-11-15T03:09:41.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a7bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a7c14e380 0x7f6a7c152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:41.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a7bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a7c14e380 0x7f6a7c152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53564/0 (socket says 172.21.15.121:53564) 2023-11-15T03:09:41.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a7bfff700 1 -- 172.21.15.121:0/3578443953 learned_addr learned my addr 172.21.15.121:0/3578443953 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:41.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a7affd700 1 --2- 172.21.15.121:0/3578443953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a7c0acb60 0x7f6a7c1641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:41.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a7b7fe700 1 --2- 172.21.15.121:0/3578443953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a7c065680 0x7f6a7c0ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:41.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a7bfff700 1 -- 172.21.15.121:0/3578443953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a7c065680 msgr2=0x7f6a7c0ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:41.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a7bfff700 1 --2- 172.21.15.121:0/3578443953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a7c065680 0x7f6a7c0ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:41.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a7bfff700 1 -- 172.21.15.121:0/3578443953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a7c0acb60 msgr2=0x7f6a7c1641d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:41.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a7bfff700 1 --2- 172.21.15.121:0/3578443953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a7c0acb60 0x7f6a7c1641d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:41.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a7bfff700 1 -- 172.21.15.121:0/3578443953 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a700092a0 con 0x7f6a7c14e380 2023-11-15T03:09:41.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a7bfff700 1 --2- 172.21.15.121:0/3578443953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a7c14e380 0x7f6a7c152ef0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 rx=0x7f6a74001560 tx=0x7f6a740015b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:41.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a637fe700 1 -- 172.21.15.121:0/3578443953 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74005d30 con 0x7f6a7c14e380 2023-11-15T03:09:41.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a637fe700 1 -- 172.21.15.121:0/3578443953 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a74005e90 con 0x7f6a7c14e380 2023-11-15T03:09:41.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a80e79700 1 -- 172.21.15.121:0/3578443953 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c164df0 con 0x7f6a7c14e380 2023-11-15T03:09:41.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a637fe700 1 -- 172.21.15.121:0/3578443953 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74007340 con 0x7f6a7c14e380 2023-11-15T03:09:41.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.182+0000 7f6a80e79700 1 -- 172.21.15.121:0/3578443953 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c1533e0 con 0x7f6a7c14e380 2023-11-15T03:09:41.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.186+0000 7f6a80e79700 1 -- 172.21.15.121:0/3578443953 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7c0c25a0 con 0x7f6a7c14e380 2023-11-15T03:09:41.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.186+0000 7f6a637fe700 1 -- 172.21.15.121:0/3578443953 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6a740076b0 con 0x7f6a7c14e380 2023-11-15T03:09:41.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.186+0000 7f6a637fe700 1 --2- 172.21.15.121:0/3578443953 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6a6805a510 0x7f6a6805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:41.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.186+0000 7f6a637fe700 1 -- 172.21.15.121:0/3578443953 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9104+0+0 (secure 0 0 0) 0x7f6a740753b0 con 0x7f6a7c14e380 2023-11-15T03:09:41.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.186+0000 7f6a7b7fe700 1 --2- 172.21.15.121:0/3578443953 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6a6805a510 0x7f6a6805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:41.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.190+0000 7f6a7b7fe700 1 --2- 172.21.15.121:0/3578443953 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6a6805a510 0x7f6a6805c9d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f6a7c0c4290 tx=0x7f6a6c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:41.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.194+0000 7f6a637fe700 1 -- 172.21.15.121:0/3578443953 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6a7c0c25a0 con 0x7f6a7c14e380 2023-11-15T03:09:41.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:41.394+0000 7f6a80e79700 1 -- 172.21.15.121:0/3578443953 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f6a7c0c25a0 con 0x7f6a7c14e380 2023-11-15T03:09:42.316 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:09:42.325 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:09:42.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.366+0000 7f6a637fe700 1 -- 172.21.15.121:0/3578443953 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v292) v1 ==== 162+0+0 (secure 0 0 0) 0x7f6a7c0c25a0 con 0x7f6a7c14e380 2023-11-15T03:09:42.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.374+0000 7f6a80e79700 1 -- 172.21.15.121:0/3578443953 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6a6805a510 msgr2=0x7f6a6805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:42.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.374+0000 7f6a80e79700 1 --2- 172.21.15.121:0/3578443953 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6a6805a510 0x7f6a6805c9d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f6a7c0c4290 tx=0x7f6a6c009040).stop 2023-11-15T03:09:42.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.374+0000 7f6a80e79700 1 -- 172.21.15.121:0/3578443953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a7c14e380 msgr2=0x7f6a7c152ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:42.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.374+0000 7f6a80e79700 1 --2- 172.21.15.121:0/3578443953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a7c14e380 0x7f6a7c152ef0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 rx=0x7f6a74001560 tx=0x7f6a740015b0).stop 2023-11-15T03:09:42.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.374+0000 7f6a80e79700 1 -- 172.21.15.121:0/3578443953 shutdown_connections 2023-11-15T03:09:42.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.374+0000 7f6a80e79700 1 --2- 172.21.15.121:0/3578443953 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6a6805a510 0x7f6a6805c9d0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:42.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.374+0000 7f6a80e79700 1 --2- 172.21.15.121:0/3578443953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a7c14e380 0x7f6a7c152ef0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:42.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.374+0000 7f6a80e79700 1 --2- 172.21.15.121:0/3578443953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a7c0acb60 0x7f6a7c1641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:42.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.374+0000 7f6a80e79700 1 --2- 172.21.15.121:0/3578443953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a7c065680 0x7f6a7c0ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:42.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.374+0000 7f6a80e79700 1 -- 172.21.15.121:0/3578443953 >> 172.21.15.121:0/3578443953 conn(0x7f6a7c067a60 msgr2=0x7f6a7c0bfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:42.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.374+0000 7f6a80e79700 1 -- 172.21.15.121:0/3578443953 shutdown_connections 2023-11-15T03:09:42.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.374+0000 7f6a80e79700 1 -- 172.21.15.121:0/3578443953 wait complete. 2023-11-15T03:09:42.386 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:09:42.808 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:09:42.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.958+0000 7ff9c2879700 1 Processor -- start 2023-11-15T03:09:42.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.958+0000 7ff9c2879700 1 -- start start 2023-11-15T03:09:42.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.962+0000 7ff9c2879700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9bc0c3280 0x7ff9bc0c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:42.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.962+0000 7ff9c2879700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff9bc15be30 0x7ff9bc15c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:42.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.962+0000 7ff9c2879700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9bc0c1d50 con 0x7ff9bc152140 2023-11-15T03:09:42.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.962+0000 7ff9c2879700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9bc0c1860 con 0x7ff9bc15be30 2023-11-15T03:09:42.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.962+0000 7ff9c2879700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9bc15cb80 con 0x7ff9bc0c3280 2023-11-15T03:09:42.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.962+0000 7ff9c2078700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff9bc15be30 0x7ff9bc15c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:42.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.962+0000 7ff9c1877700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9bc0c3280 0x7ff9bc0c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:42.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.962+0000 7ff9c2078700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff9bc15be30 0x7ff9bc15c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53576/0 (socket says 172.21.15.121:53576) 2023-11-15T03:09:42.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.962+0000 7ff9c1877700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9bc0c3280 0x7ff9bc0c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:33210/0 (socket says 172.21.15.121:33210) 2023-11-15T03:09:42.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.962+0000 7ff9c2078700 1 -- 172.21.15.121:0/1482824719 learned_addr learned my addr 172.21.15.121:0/1482824719 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:42.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.962+0000 7ff9c2078700 1 -- 172.21.15.121:0/1482824719 >> v2:172.21.15.190:3300/0 conn(0x7ff9bc0c3280 msgr2=0x7ff9bc0c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:42.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.962+0000 7ff9c2078700 1 --2- 172.21.15.121:0/1482824719 >> v2:172.21.15.190:3300/0 conn(0x7ff9bc0c3280 0x7ff9bc0c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:42.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.962+0000 7ff9c2078700 1 -- 172.21.15.121:0/1482824719 >> v1:172.21.15.190:6789/0 conn(0x7ff9bc152140 legacy=0x7ff9bc1525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:42.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.962+0000 7ff9c2078700 1 -- 172.21.15.121:0/1482824719 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9bc0a9240 con 0x7ff9bc15be30 2023-11-15T03:09:42.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.962+0000 7ff9c2078700 1 --2- 172.21.15.121:0/1482824719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9bc15be30 0x7ff9bc15c280 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 rx=0x7ff9bc0c7720 tx=0x7ff9b4009840).ready entity=mon.0 client_cookie=c95f377eeeef66c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:42.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.966+0000 7ff9c0875700 1 -- 172.21.15.121:0/1482824719 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b400e070 con 0x7ff9bc15be30 2023-11-15T03:09:42.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.966+0000 7ff9c0875700 1 -- 172.21.15.121:0/1482824719 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b400ba20 con 0x7ff9bc15be30 2023-11-15T03:09:42.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.966+0000 7ff9c0875700 1 -- 172.21.15.121:0/1482824719 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b400bda0 con 0x7ff9bc15be30 2023-11-15T03:09:42.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.966+0000 7ff9c2879700 1 -- 172.21.15.121:0/1482824719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9bc15be30 msgr2=0x7ff9bc15c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:42.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.966+0000 7ff9c2879700 1 --2- 172.21.15.121:0/1482824719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9bc15be30 0x7ff9bc15c280 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 rx=0x7ff9bc0c7720 tx=0x7ff9b4009840).stop 2023-11-15T03:09:42.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.966+0000 7ff9c2879700 1 -- 172.21.15.121:0/1482824719 shutdown_connections 2023-11-15T03:09:42.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.966+0000 7ff9c2879700 1 --2- 172.21.15.121:0/1482824719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9bc15be30 0x7ff9bc15c280 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:42.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.966+0000 7ff9c2879700 1 --2- 172.21.15.121:0/1482824719 >> v2:172.21.15.190:3300/0 conn(0x7ff9bc0c3280 0x7ff9bc0c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:42.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.966+0000 7ff9c2879700 1 -- 172.21.15.121:0/1482824719 >> 172.21.15.121:0/1482824719 conn(0x7ff9bc062190 msgr2=0x7ff9bc062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:42.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.966+0000 7ff9c2879700 1 -- 172.21.15.121:0/1482824719 shutdown_connections 2023-11-15T03:09:42.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.966+0000 7ff9c2879700 1 -- 172.21.15.121:0/1482824719 wait complete. 2023-11-15T03:09:42.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c2879700 1 Processor -- start 2023-11-15T03:09:42.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c2879700 1 -- start start 2023-11-15T03:09:42.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c2879700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9bc0c3280 0x7ff9bc14e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:42.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c2879700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9bc152140 0x7ff9bc1430d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:42.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c2879700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9bc15be30 0x7ff9bc158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:42.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c2879700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc14ebb0 con 0x7ff9bc0c3280 2023-11-15T03:09:42.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c2879700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc0c6060 con 0x7ff9bc15be30 2023-11-15T03:09:42.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c2879700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc0c6340 con 0x7ff9bc152140 2023-11-15T03:09:42.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c1877700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9bc0c3280 0x7ff9bc14e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:42.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c1877700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9bc0c3280 0x7ff9bc14e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:53582/0 (socket says 172.21.15.121:53582) 2023-11-15T03:09:42.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c1877700 1 -- 172.21.15.121:0/53950547 learned_addr learned my addr 172.21.15.121:0/53950547 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:42.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c1877700 1 -- 172.21.15.121:0/53950547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9bc152140 msgr2=0x7ff9bc1430d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:42.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c2078700 1 --2- 172.21.15.121:0/53950547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9bc15be30 0x7ff9bc158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:42.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c1076700 1 --2- 172.21.15.121:0/53950547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9bc152140 0x7ff9bc1430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:42.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c1877700 1 --2- 172.21.15.121:0/53950547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9bc152140 0x7ff9bc1430d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:42.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c1877700 1 -- 172.21.15.121:0/53950547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9bc15be30 msgr2=0x7ff9bc158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:42.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c1877700 1 --2- 172.21.15.121:0/53950547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9bc15be30 0x7ff9bc158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:42.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c1877700 1 -- 172.21.15.121:0/53950547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b4009ae0 con 0x7ff9bc0c3280 2023-11-15T03:09:42.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c2078700 1 --2- 172.21.15.121:0/53950547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9bc15be30 0x7ff9bc158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:42.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c1076700 1 --2- 172.21.15.121:0/53950547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9bc152140 0x7ff9bc1430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:42.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.970+0000 7ff9c1877700 1 --2- 172.21.15.121:0/53950547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9bc0c3280 0x7ff9bc14e6c0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 rx=0x7ff9b800e900 tx=0x7ff9b8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:42.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.974+0000 7ff9b1ffb700 1 -- 172.21.15.121:0/53950547 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b800c640 con 0x7ff9bc0c3280 2023-11-15T03:09:42.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.974+0000 7ff9b1ffb700 1 -- 172.21.15.121:0/53950547 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b800c7a0 con 0x7ff9bc0c3280 2023-11-15T03:09:42.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.974+0000 7ff9c2879700 1 -- 172.21.15.121:0/53950547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9bc0c6620 con 0x7ff9bc0c3280 2023-11-15T03:09:42.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.974+0000 7ff9b1ffb700 1 -- 172.21.15.121:0/53950547 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b800cb20 con 0x7ff9bc0c3280 2023-11-15T03:09:42.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.974+0000 7ff9c2879700 1 -- 172.21.15.121:0/53950547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9bc1435c0 con 0x7ff9bc0c3280 2023-11-15T03:09:42.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.974+0000 7ff9c2879700 1 -- 172.21.15.121:0/53950547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9bc06bcf0 con 0x7ff9bc0c3280 2023-11-15T03:09:42.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.974+0000 7ff9b1ffb700 1 -- 172.21.15.121:0/53950547 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff9b801b400 con 0x7ff9bc0c3280 2023-11-15T03:09:42.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.978+0000 7ff9b1ffb700 1 --2- 172.21.15.121:0/53950547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff9a005a510 0x7ff9a005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:42.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.978+0000 7ff9b1ffb700 1 -- 172.21.15.121:0/53950547 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9104+0+0 (secure 0 0 0) 0x7ff9b8075860 con 0x7ff9bc0c3280 2023-11-15T03:09:42.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.978+0000 7ff9c1076700 1 --2- 172.21.15.121:0/53950547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff9a005a510 0x7ff9a005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:42.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.978+0000 7ff9c1076700 1 --2- 172.21.15.121:0/53950547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff9a005a510 0x7ff9a005c9d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7ff9bc159740 tx=0x7ff9ac009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:42.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:42.982+0000 7ff9b1ffb700 1 -- 172.21.15.121:0/53950547 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff9b80483a0 con 0x7ff9bc0c3280 2023-11-15T03:09:43.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:43.178+0000 7ff9c2879700 1 -- 172.21.15.121:0/53950547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9bc068a70 con 0x7ff9bc0c3280 2023-11-15T03:09:43.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:43.178+0000 7ff9b1ffb700 1 -- 172.21.15.121:0/53950547 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 293 v293) v1 ==== 95+0+4505 (secure 0 0 0) 0x7ff9b8016030 con 0x7ff9bc0c3280 2023-11-15T03:09:43.190 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:09:43.190 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":293,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":293,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:09:42.373524+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9990":{"gid":9990,"name":"a","rank":0,"incarnation":293,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/1175031592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1175031592},{"type":"v1","addr":"172.21.15.121:6836","nonce":1175031592}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:09:43.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:43.182+0000 7ff9c2879700 1 -- 172.21.15.121:0/53950547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff9a005a510 msgr2=0x7ff9a005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:43.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:43.182+0000 7ff9c2879700 1 --2- 172.21.15.121:0/53950547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff9a005a510 0x7ff9a005c9d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7ff9bc159740 tx=0x7ff9ac009040).stop 2023-11-15T03:09:43.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:43.182+0000 7ff9c2879700 1 -- 172.21.15.121:0/53950547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9bc0c3280 msgr2=0x7ff9bc14e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:43.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:43.182+0000 7ff9c2879700 1 --2- 172.21.15.121:0/53950547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9bc0c3280 0x7ff9bc14e6c0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 rx=0x7ff9b800e900 tx=0x7ff9b8009060).stop 2023-11-15T03:09:43.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:43.182+0000 7ff9c2879700 1 -- 172.21.15.121:0/53950547 shutdown_connections 2023-11-15T03:09:43.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:43.182+0000 7ff9c2879700 1 --2- 172.21.15.121:0/53950547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff9a005a510 0x7ff9a005c9d0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:43.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:43.182+0000 7ff9c2879700 1 --2- 172.21.15.121:0/53950547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9bc15be30 0x7ff9bc158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:43.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:43.182+0000 7ff9c2879700 1 --2- 172.21.15.121:0/53950547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9bc152140 0x7ff9bc1430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:43.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:43.182+0000 7ff9c2879700 1 --2- 172.21.15.121:0/53950547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9bc0c3280 0x7ff9bc14e6c0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:43.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:43.182+0000 7ff9c2879700 1 -- 172.21.15.121:0/53950547 >> 172.21.15.121:0/53950547 conn(0x7ff9bc062190 msgr2=0x7ff9bc14fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:43.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:43.182+0000 7ff9c2879700 1 -- 172.21.15.121:0/53950547 shutdown_connections 2023-11-15T03:09:43.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:43.182+0000 7ff9c2879700 1 -- 172.21.15.121:0/53950547 wait complete. 2023-11-15T03:09:43.197 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 293 2023-11-15T03:09:43.214 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 293, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:09:42.373524+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 127, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9990}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9990': {'gid': 9990, 'name': 'a', 'rank': 0, 'incarnation': 293, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.121:6836/1175031592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 1175031592}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 1175031592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:09:43.214 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9990:up:replay 2023-11-15T03:09:43.215 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:09:48.620 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:09:48.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.774+0000 7ffa9a7b9700 1 Processor -- start 2023-11-15T03:09:48.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.774+0000 7ffa9a7b9700 1 -- start start 2023-11-15T03:09:48.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.778+0000 7ffa9a7b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa940cabd0 0x7ffa940cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:48.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.778+0000 7ffa9a7b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffa940acc90 0x7ffa94066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:48.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.778+0000 7ffa9a7b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa940ad0f0 0x7ffa940ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:48.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.778+0000 7ffa9a7b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa940ad650 con 0x7ffa940ad0f0 2023-11-15T03:09:48.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.778+0000 7ffa9a7b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa94065fc0 con 0x7ffa940acc90 2023-11-15T03:09:48.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.778+0000 7ffa9a7b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa940662a0 con 0x7ffa940cabd0 2023-11-15T03:09:48.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.778+0000 7ffa99fb8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa940ad0f0 0x7ffa940ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:48.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.778+0000 7ffa99fb8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa940ad0f0 0x7ffa940ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46872/0 (socket says 172.21.15.121:46872) 2023-11-15T03:09:48.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.778+0000 7ffa997b7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa940cabd0 0x7ffa940cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:48.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.778+0000 7ffa99fb8700 1 -- 172.21.15.121:0/4125630220 learned_addr learned my addr 172.21.15.121:0/4125630220 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:48.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.778+0000 7ffa98fb6700 1 --2- 172.21.15.121:0/4125630220 >> v2:172.21.15.190:3300/0 conn(0x7ffa940acc90 0x7ffa94066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:48.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.778+0000 7ffa99fb8700 1 -- 172.21.15.121:0/4125630220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa940cabd0 msgr2=0x7ffa940cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:48.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.778+0000 7ffa99fb8700 1 --2- 172.21.15.121:0/4125630220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa940cabd0 0x7ffa940cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:48.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.778+0000 7ffa99fb8700 1 -- 172.21.15.121:0/4125630220 >> v2:172.21.15.190:3300/0 conn(0x7ffa940acc90 msgr2=0x7ffa94066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:48.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.778+0000 7ffa99fb8700 1 --2- 172.21.15.121:0/4125630220 >> v2:172.21.15.190:3300/0 conn(0x7ffa940acc90 0x7ffa94066d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:48.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.778+0000 7ffa99fb8700 1 -- 172.21.15.121:0/4125630220 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa94066580 con 0x7ffa940ad0f0 2023-11-15T03:09:48.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.778+0000 7ffa98fb6700 1 --2- 172.21.15.121:0/4125630220 >> v2:172.21.15.190:3300/0 conn(0x7ffa940acc90 0x7ffa94066d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:09:48.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.778+0000 7ffa997b7700 1 --2- 172.21.15.121:0/4125630220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa940cabd0 0x7ffa940cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:09:48.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.778+0000 7ffa99fb8700 1 --2- 172.21.15.121:0/4125630220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa940ad0f0 0x7ffa940ae190 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 rx=0x7ffa8c009f20 tx=0x7ffa8c009840).ready entity=mon.0 client_cookie=513125869e6a0642 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:48.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.782+0000 7ffa8bfff700 1 -- 172.21.15.121:0/4125630220 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c00e070 con 0x7ffa940ad0f0 2023-11-15T03:09:48.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.782+0000 7ffa8bfff700 1 -- 172.21.15.121:0/4125630220 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8c00b930 con 0x7ffa940ad0f0 2023-11-15T03:09:48.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.782+0000 7ffa8bfff700 1 -- 172.21.15.121:0/4125630220 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c00bcb0 con 0x7ffa940ad0f0 2023-11-15T03:09:48.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.782+0000 7ffa9a7b9700 1 -- 172.21.15.121:0/4125630220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa940ad0f0 msgr2=0x7ffa940ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:48.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.782+0000 7ffa9a7b9700 1 --2- 172.21.15.121:0/4125630220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa940ad0f0 0x7ffa940ae190 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 rx=0x7ffa8c009f20 tx=0x7ffa8c009840).stop 2023-11-15T03:09:48.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.782+0000 7ffa9a7b9700 1 -- 172.21.15.121:0/4125630220 shutdown_connections 2023-11-15T03:09:48.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.782+0000 7ffa9a7b9700 1 --2- 172.21.15.121:0/4125630220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa940ad0f0 0x7ffa940ae190 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:48.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.782+0000 7ffa9a7b9700 1 --2- 172.21.15.121:0/4125630220 >> v2:172.21.15.190:3300/0 conn(0x7ffa940acc90 0x7ffa94066d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:48.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.782+0000 7ffa9a7b9700 1 --2- 172.21.15.121:0/4125630220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa940cabd0 0x7ffa940cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:48.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.782+0000 7ffa9a7b9700 1 -- 172.21.15.121:0/4125630220 >> 172.21.15.121:0/4125630220 conn(0x7ffa94062190 msgr2=0x7ffa940625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:48.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.782+0000 7ffa9a7b9700 1 -- 172.21.15.121:0/4125630220 shutdown_connections 2023-11-15T03:09:48.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.782+0000 7ffa9a7b9700 1 -- 172.21.15.121:0/4125630220 wait complete. 2023-11-15T03:09:48.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.782+0000 7ffa9a7b9700 1 Processor -- start 2023-11-15T03:09:48.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.782+0000 7ffa9a7b9700 1 -- start start 2023-11-15T03:09:48.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa9a7b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa940acc90 0x7ffa9414e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:48.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa9a7b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa940ad0f0 0x7ffa94155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:48.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa9a7b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa940cabd0 0x7ffa94154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:48.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa9a7b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa941542f0 con 0x7ffa940acc90 2023-11-15T03:09:48.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa9a7b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa941536e0 con 0x7ffa940cabd0 2023-11-15T03:09:48.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa9a7b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa941539c0 con 0x7ffa940ad0f0 2023-11-15T03:09:48.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa997b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa940acc90 0x7ffa9414e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:48.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa997b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa940acc90 0x7ffa9414e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46880/0 (socket says 172.21.15.121:46880) 2023-11-15T03:09:48.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa997b7700 1 -- 172.21.15.121:0/2940515301 learned_addr learned my addr 172.21.15.121:0/2940515301 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:48.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa99fb8700 1 --2- 172.21.15.121:0/2940515301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa940cabd0 0x7ffa94154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:48.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa997b7700 1 -- 172.21.15.121:0/2940515301 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa940ad0f0 msgr2=0x7ffa94155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:48.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa98fb6700 1 --2- 172.21.15.121:0/2940515301 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa940ad0f0 0x7ffa94155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:48.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa997b7700 1 --2- 172.21.15.121:0/2940515301 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa940ad0f0 0x7ffa94155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:48.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa997b7700 1 -- 172.21.15.121:0/2940515301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa940cabd0 msgr2=0x7ffa94154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:48.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa997b7700 1 --2- 172.21.15.121:0/2940515301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa940cabd0 0x7ffa94154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:48.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa997b7700 1 -- 172.21.15.121:0/2940515301 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa840092a0 con 0x7ffa940acc90 2023-11-15T03:09:48.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa99fb8700 1 --2- 172.21.15.121:0/2940515301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa940cabd0 0x7ffa94154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:48.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa98fb6700 1 --2- 172.21.15.121:0/2940515301 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa940ad0f0 0x7ffa94155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:48.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa997b7700 1 --2- 172.21.15.121:0/2940515301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa940acc90 0x7ffa9414e550 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 rx=0x7ffa9000e980 tx=0x7ffa90009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:48.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa897fa700 1 -- 172.21.15.121:0/2940515301 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9000c850 con 0x7ffa940acc90 2023-11-15T03:09:48.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa897fa700 1 -- 172.21.15.121:0/2940515301 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9000c9b0 con 0x7ffa940acc90 2023-11-15T03:09:48.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa9a7b9700 1 -- 172.21.15.121:0/2940515301 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa900092a0 con 0x7ffa940acc90 2023-11-15T03:09:48.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa897fa700 1 -- 172.21.15.121:0/2940515301 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9000cd30 con 0x7ffa940acc90 2023-11-15T03:09:48.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.786+0000 7ffa9a7b9700 1 -- 172.21.15.121:0/2940515301 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa8c009ae0 con 0x7ffa940acc90 2023-11-15T03:09:48.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.790+0000 7ffa9a7b9700 1 -- 172.21.15.121:0/2940515301 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa9406bcf0 con 0x7ffa940acc90 2023-11-15T03:09:48.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.790+0000 7ffa897fa700 1 -- 172.21.15.121:0/2940515301 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ffa90012600 con 0x7ffa940acc90 2023-11-15T03:09:48.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.790+0000 7ffa897fa700 1 --2- 172.21.15.121:0/2940515301 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa7805a540 0x7ffa7805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:48.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.790+0000 7ffa897fa700 1 -- 172.21.15.121:0/2940515301 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9104+0+0 (secure 0 0 0) 0x7ffa90075980 con 0x7ffa940acc90 2023-11-15T03:09:48.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.790+0000 7ffa98fb6700 1 --2- 172.21.15.121:0/2940515301 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa7805a540 0x7ffa7805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:48.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.794+0000 7ffa98fb6700 1 --2- 172.21.15.121:0/2940515301 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa7805a540 0x7ffa7805ca00 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7ffa84009270 tx=0x7ffa84008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:48.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.798+0000 7ffa897fa700 1 -- 172.21.15.121:0/2940515301 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ffa90048520 con 0x7ffa940acc90 2023-11-15T03:09:49.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.994+0000 7ffa9a7b9700 1 -- 172.21.15.121:0/2940515301 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa94066860 con 0x7ffa940acc90 2023-11-15T03:09:49.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.994+0000 7ffa897fa700 1 -- 172.21.15.121:0/2940515301 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 296 v296) v1 ==== 95+0+4505 (secure 0 0 0) 0x7ffa9004bb40 con 0x7ffa940acc90 2023-11-15T03:09:49.005 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:09:49.005 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":296,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":296,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:09:45.395385+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9990":{"gid":9990,"name":"a","rank":0,"incarnation":293,"state":"up:active","state_seq":5,"addr":"172.21.15.121:6836/1175031592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1175031592},{"type":"v1","addr":"172.21.15.121:6836","nonce":1175031592}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:09:49.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.998+0000 7ffa9a7b9700 1 -- 172.21.15.121:0/2940515301 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa7805a540 msgr2=0x7ffa7805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:49.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.998+0000 7ffa9a7b9700 1 --2- 172.21.15.121:0/2940515301 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa7805a540 0x7ffa7805ca00 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7ffa84009270 tx=0x7ffa84008040).stop 2023-11-15T03:09:49.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.998+0000 7ffa9a7b9700 1 -- 172.21.15.121:0/2940515301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa940acc90 msgr2=0x7ffa9414e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:49.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.998+0000 7ffa9a7b9700 1 --2- 172.21.15.121:0/2940515301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa940acc90 0x7ffa9414e550 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 rx=0x7ffa9000e980 tx=0x7ffa90009060).stop 2023-11-15T03:09:49.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.998+0000 7ffa9a7b9700 1 -- 172.21.15.121:0/2940515301 shutdown_connections 2023-11-15T03:09:49.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.998+0000 7ffa9a7b9700 1 --2- 172.21.15.121:0/2940515301 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa7805a540 0x7ffa7805ca00 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.998+0000 7ffa9a7b9700 1 --2- 172.21.15.121:0/2940515301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa940cabd0 0x7ffa94154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.998+0000 7ffa9a7b9700 1 --2- 172.21.15.121:0/2940515301 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa940ad0f0 0x7ffa94155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.998+0000 7ffa9a7b9700 1 --2- 172.21.15.121:0/2940515301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa940acc90 0x7ffa9414e550 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.998+0000 7ffa9a7b9700 1 -- 172.21.15.121:0/2940515301 >> 172.21.15.121:0/2940515301 conn(0x7ffa94062190 msgr2=0x7ffa940c7060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:49.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.998+0000 7ffa9a7b9700 1 -- 172.21.15.121:0/2940515301 shutdown_connections 2023-11-15T03:09:49.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:48.998+0000 7ffa9a7b9700 1 -- 172.21.15.121:0/2940515301 wait complete. 2023-11-15T03:09:49.010 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 296 2023-11-15T03:09:49.027 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 296, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:09:45.395385+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 127, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9990}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9990': {'gid': 9990, 'name': 'a', 'rank': 0, 'incarnation': 293, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.121:6836/1175031592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 1175031592}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 1175031592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:09:49.027 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T03:09:49.028 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-15T03:09:49.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.174+0000 7f05b08f0700 1 Processor -- start 2023-11-15T03:09:49.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.174+0000 7f05b08f0700 1 -- start start 2023-11-15T03:09:49.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.174+0000 7f05b08f0700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f05ac153990 0x7f05ac153d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.174+0000 7f05b08f0700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f05ac0c2ec0 0x7f05ac0c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.174+0000 7f05b08f0700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ac0a8b20 con 0x7f05ac0bb910 2023-11-15T03:09:49.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.174+0000 7f05b08f0700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ac0c1760 con 0x7f05ac153990 2023-11-15T03:09:49.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.174+0000 7f05b08f0700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ac0a8d50 con 0x7f05ac0c2ec0 2023-11-15T03:09:49.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.178+0000 7f05aad9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f05ac153990 0x7f05ac153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.178+0000 7f05ab59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f05ac0c2ec0 0x7f05ac0c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.178+0000 7f05ab59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f05ac0c2ec0 0x7f05ac0c32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:33296/0 (socket says 172.21.15.121:33296) 2023-11-15T03:09:49.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.178+0000 7f05ab59e700 1 -- 172.21.15.121:0/2148312024 learned_addr learned my addr 172.21.15.121:0/2148312024 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:49.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.178+0000 7f05aad9d700 1 --2- 172.21.15.121:0/2148312024 >> v2:172.21.15.121:3300/0 conn(0x7f05ac153990 0x7f05ac153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46892/0 (socket says 172.21.15.121:46892) 2023-11-15T03:09:49.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.178+0000 7f05aa59c700 1 --1- 172.21.15.121:0/2148312024 >> v1:172.21.15.190:6789/0 conn(0x7f05ac0bb910 0x7f05ac1581e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:57494/0 (socket says 172.21.15.121:57494) 2023-11-15T03:09:49.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.178+0000 7f05aad9d700 1 -- 172.21.15.121:0/2148312024 >> v2:172.21.15.190:3300/0 conn(0x7f05ac0c2ec0 msgr2=0x7f05ac0c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:49.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.178+0000 7f05aad9d700 1 --2- 172.21.15.121:0/2148312024 >> v2:172.21.15.190:3300/0 conn(0x7f05ac0c2ec0 0x7f05ac0c32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.178+0000 7f05aad9d700 1 -- 172.21.15.121:0/2148312024 >> v1:172.21.15.190:6789/0 conn(0x7f05ac0bb910 legacy=0x7f05ac1581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:49.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.178+0000 7f05aad9d700 1 -- 172.21.15.121:0/2148312024 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac0a9030 con 0x7f05ac153990 2023-11-15T03:09:49.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.178+0000 7f05ab59e700 1 --2- 172.21.15.121:0/2148312024 >> v2:172.21.15.190:3300/0 conn(0x7f05ac0c2ec0 0x7f05ac0c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:09:49.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.178+0000 7f05aad9d700 1 --2- 172.21.15.121:0/2148312024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05ac153990 0x7f05ac153d70 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 rx=0x7f05ac0632c0 tx=0x7f059c009060).ready entity=mon.0 client_cookie=5b313e860f3c08df server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:49.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.178+0000 7f05a9d9b700 1 -- 172.21.15.121:0/2148312024 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c00f510 con 0x7f05ac153990 2023-11-15T03:09:49.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.182+0000 7f05a9d9b700 1 -- 172.21.15.121:0/2148312024 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059c00f670 con 0x7f05ac153990 2023-11-15T03:09:49.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.182+0000 7f05a9d9b700 1 -- 172.21.15.121:0/2148312024 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c00f9f0 con 0x7f05ac153990 2023-11-15T03:09:49.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.182+0000 7f05b08f0700 1 -- 172.21.15.121:0/2148312024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05ac153990 msgr2=0x7f05ac153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:49.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.182+0000 7f05b08f0700 1 --2- 172.21.15.121:0/2148312024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05ac153990 0x7f05ac153d70 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 rx=0x7f05ac0632c0 tx=0x7f059c009060).stop 2023-11-15T03:09:49.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.182+0000 7f05b08f0700 1 -- 172.21.15.121:0/2148312024 shutdown_connections 2023-11-15T03:09:49.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.182+0000 7f05b08f0700 1 --2- 172.21.15.121:0/2148312024 >> v2:172.21.15.190:3300/0 conn(0x7f05ac0c2ec0 0x7f05ac0c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.182+0000 7f05b08f0700 1 --2- 172.21.15.121:0/2148312024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05ac153990 0x7f05ac153d70 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.182+0000 7f05b08f0700 1 -- 172.21.15.121:0/2148312024 >> 172.21.15.121:0/2148312024 conn(0x7f05ac067a60 msgr2=0x7f05ac067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:49.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.182+0000 7f05b08f0700 1 -- 172.21.15.121:0/2148312024 shutdown_connections 2023-11-15T03:09:49.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.182+0000 7f05b08f0700 1 -- 172.21.15.121:0/2148312024 wait complete. 2023-11-15T03:09:49.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.182+0000 7f05b08f0700 1 Processor -- start 2023-11-15T03:09:49.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.182+0000 7f05b08f0700 1 -- start start 2023-11-15T03:09:49.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f05b08f0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05ac0bb910 0x7f05ac146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f05b08f0700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05ac0c2ec0 0x7f05ac0be5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f05b08f0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05ac153990 0x7f05ac1682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f05b08f0700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0bfbb0 con 0x7f05ac0c2ec0 2023-11-15T03:09:49.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f05b08f0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac146a20 con 0x7f05ac153990 2023-11-15T03:09:49.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f05b08f0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0befa0 con 0x7f05ac0bb910 2023-11-15T03:09:49.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f05aa59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05ac0c2ec0 0x7f05ac0be5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f05aa59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05ac0c2ec0 0x7f05ac0be5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46908/0 (socket says 172.21.15.121:46908) 2023-11-15T03:09:49.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f05aa59c700 1 -- 172.21.15.121:0/1151807210 learned_addr learned my addr 172.21.15.121:0/1151807210 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:49.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f05aad9d700 1 --2- 172.21.15.121:0/1151807210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05ac0bb910 0x7f05ac146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f05ab59e700 1 --2- 172.21.15.121:0/1151807210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05ac153990 0x7f05ac1682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f05aa59c700 1 -- 172.21.15.121:0/1151807210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05ac0bb910 msgr2=0x7f05ac146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:49.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f05aa59c700 1 --2- 172.21.15.121:0/1151807210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05ac0bb910 0x7f05ac146530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f05aa59c700 1 -- 172.21.15.121:0/1151807210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05ac153990 msgr2=0x7f05ac1682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:49.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f05aa59c700 1 --2- 172.21.15.121:0/1151807210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05ac153990 0x7f05ac1682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f05aa59c700 1 -- 172.21.15.121:0/1151807210 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a4009ae0 con 0x7f05ac0c2ec0 2023-11-15T03:09:49.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f05aa59c700 1 --2- 172.21.15.121:0/1151807210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05ac0c2ec0 0x7f05ac0be5c0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 rx=0x7f05a000c4c0 tx=0x7f05a0007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:49.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f0592ffd700 1 -- 172.21.15.121:0/1151807210 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a000ce70 con 0x7f05ac0c2ec0 2023-11-15T03:09:49.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f0592ffd700 1 -- 172.21.15.121:0/1151807210 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a00072e0 con 0x7f05ac0c2ec0 2023-11-15T03:09:49.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f0592ffd700 1 -- 172.21.15.121:0/1151807210 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a0005730 con 0x7f05ac0c2ec0 2023-11-15T03:09:49.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f05b08f0700 1 -- 172.21.15.121:0/1151807210 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059c0092a0 con 0x7f05ac0c2ec0 2023-11-15T03:09:49.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.186+0000 7f05b08f0700 1 -- 172.21.15.121:0/1151807210 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ac0bf560 con 0x7f05ac0c2ec0 2023-11-15T03:09:49.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.190+0000 7f0592ffd700 1 -- 172.21.15.121:0/1151807210 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f05a0005b60 con 0x7f05ac0c2ec0 2023-11-15T03:09:49.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.190+0000 7f0592ffd700 1 --2- 172.21.15.121:0/1151807210 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f059805a510 0x7f059805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.190+0000 7f0592ffd700 1 -- 172.21.15.121:0/1151807210 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9104+0+0 (secure 0 0 0) 0x7f05a0074ac0 con 0x7f05ac0c2ec0 2023-11-15T03:09:49.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.190+0000 7f05aad9d700 1 --2- 172.21.15.121:0/1151807210 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f059805a510 0x7f059805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05bb4b3740 1 Processor -- start 2023-11-15T03:09:49.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05aad9d700 1 --2- 172.21.15.121:0/1151807210 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f059805a510 0x7f059805c9d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f05ac15b5b0 tx=0x7f059c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:49.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05bb4b3740 1 -- start start 2023-11-15T03:09:49.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05bb4b3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe10ad0 0xe37090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05bb4b3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 0xe5d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05bb4b3740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe5d630 0xe5dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05bb4b3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0xd91410 con 0xe10eb0 2023-11-15T03:09:49.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05bb4b3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0xd91550 con 0xe10ad0 2023-11-15T03:09:49.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05bb4b3740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0xd91a00 con 0xe5d630 2023-11-15T03:09:49.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05aa59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 0xe5d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05aa59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 0xe5d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46910/0 (socket says 172.21.15.121:46910) 2023-11-15T03:09:49.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05aa59c700 1 -- 172.21.15.121:0/601755476 learned_addr learned my addr 172.21.15.121:0/601755476 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:49.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05ab59e700 1 --2- 172.21.15.121:0/601755476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe5d630 0xe5dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05aad9d700 1 --2- 172.21.15.121:0/601755476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe10ad0 0xe37090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05aa59c700 1 -- 172.21.15.121:0/601755476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe5d630 msgr2=0xe5dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:49.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05aa59c700 1 --2- 172.21.15.121:0/601755476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe5d630 0xe5dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05aa59c700 1 -- 172.21.15.121:0/601755476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe10ad0 msgr2=0xe37090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:49.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05aa59c700 1 --2- 172.21.15.121:0/601755476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe10ad0 0xe37090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05aa59c700 1 -- 172.21.15.121:0/601755476 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xd8a730 con 0xe10eb0 2023-11-15T03:09:49.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05aad9d700 1 --2- 172.21.15.121:0/601755476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe10ad0 0xe37090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:49.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f05aa59c700 1 --2- 172.21.15.121:0/601755476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 0xe5d140 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 rx=0x7f05a0047790 tx=0x7f05a000f000).ready entity=mon.0 client_cookie=54fd69413e1e4f36 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:49.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f058bfff700 1 -- 172.21.15.121:0/601755476 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a004b2c0 con 0xe10eb0 2023-11-15T03:09:49.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f058bfff700 1 -- 172.21.15.121:0/601755476 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a0055030 con 0xe10eb0 2023-11-15T03:09:49.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.194+0000 7f058bfff700 1 -- 172.21.15.121:0/601755476 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a0005e10 con 0xe10eb0 2023-11-15T03:09:49.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.198+0000 7f05bb4b3740 1 -- 172.21.15.121:0/601755476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 msgr2=0xe5d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:49.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.198+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/601755476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 0xe5d140 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 rx=0x7f05a0047790 tx=0x7f05a000f000).stop 2023-11-15T03:09:49.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.198+0000 7f05bb4b3740 1 -- 172.21.15.121:0/601755476 shutdown_connections 2023-11-15T03:09:49.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.198+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/601755476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe5d630 0xe5dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.198+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/601755476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 0xe5d140 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.198+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/601755476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe10ad0 0xe37090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.198+0000 7f05bb4b3740 1 -- 172.21.15.121:0/601755476 >> 172.21.15.121:0/601755476 conn(0xbef550 msgr2=0xe0f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:49.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.198+0000 7f05bb4b3740 1 -- 172.21.15.121:0/601755476 shutdown_connections 2023-11-15T03:09:49.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.198+0000 7f05bb4b3740 1 -- 172.21.15.121:0/601755476 wait complete. 2023-11-15T03:09:49.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.198+0000 7f05bb4b3740 1 -- start start 2023-11-15T03:09:49.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.198+0000 7f05bb4b3740 1 Processor -- start 2023-11-15T03:09:49.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.198+0000 7f05bb4b3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10ad0 0xe4f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.198+0000 7f05bb4b3740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe10eb0 0xe30760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.198+0000 7f05bb4b3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe5d630 0xe63b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.198+0000 7f05bb4b3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0xd5c650 con 0xe10ad0 2023-11-15T03:09:49.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.198+0000 7f05bb4b3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0xe64d00 con 0xe5d630 2023-11-15T03:09:49.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.198+0000 7f05bb4b3740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0xe4ffa0 con 0xe10eb0 2023-11-15T03:09:49.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.198+0000 7f05bb4b3740 10 client.0 ll_register_callbacks cb 0xdc9020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:09:49.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05aad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10ad0 0xe4f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05aad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10ad0 0xe4f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46922/0 (socket says 172.21.15.121:46922) 2023-11-15T03:09:49.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05aad9d700 1 -- 172.21.15.121:0/3800084764 learned_addr learned my addr 172.21.15.121:0/3800084764 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:49.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05aa59c700 1 --2- 172.21.15.121:0/3800084764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe10eb0 0xe30760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05ab59e700 1 --2- 172.21.15.121:0/3800084764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe5d630 0xe63b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05aad9d700 1 -- 172.21.15.121:0/3800084764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe10eb0 msgr2=0xe30760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:49.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05aad9d700 1 --2- 172.21.15.121:0/3800084764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe10eb0 0xe30760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05aad9d700 1 -- 172.21.15.121:0/3800084764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe5d630 msgr2=0xe63b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:49.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05aad9d700 1 --2- 172.21.15.121:0/3800084764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe5d630 0xe63b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05aad9d700 1 -- 172.21.15.121:0/3800084764 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ac0bf560 con 0xe10ad0 2023-11-15T03:09:49.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05aad9d700 1 --2- 172.21.15.121:0/3800084764 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10ad0 0xe4f850 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 rx=0x7f059c009dd0 tx=0x7f059c01e070).ready entity=mon.0 client_cookie=7168acc0d8aa6f5d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:49.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05897fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:09:49.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05897fa700 1 -- 172.21.15.121:0/3800084764 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c02f470 con 0xe10ad0 2023-11-15T03:09:49.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05bb4b3740 1 -- 172.21.15.121:0/3800084764 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xd871a0 con 0xe10ad0 2023-11-15T03:09:49.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05897fa700 1 -- 172.21.15.121:0/3800084764 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 9104+0+0 (secure 0 0 0) 0x7f059c02f5d0 con 0xe10ad0 2023-11-15T03:09:49.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05897fa700 1 -- 172.21.15.121:0/3800084764 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=296) v2 ==== 24+0+0 (secure 0 0 0) 0x7f059c02f9e0 con 0xe10ad0 2023-11-15T03:09:49.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05bb4b3740 10 client.10068 fetch_fsmap learned FSMap version 296 2023-11-15T03:09:49.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05bb4b3740 1 -- 172.21.15.121:0/3800084764 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=296,monmap=2+}) v3 -- 0xd85960 con 0xe10ad0 2023-11-15T03:09:49.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05897fa700 1 -- 172.21.15.121:0/3800084764 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xd871a0 con 0xe10ad0 2023-11-15T03:09:49.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05897fa700 1 -- 172.21.15.121:0/3800084764 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 296) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f059c02fb40 con 0xe10ad0 2023-11-15T03:09:49.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05bb4b3740 10 client.10068 fetch_fsmap finished waiting for FSMap version 296 2023-11-15T03:09:49.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05bb4b3740 10 client.10068 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:09:49.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05bb4b3740 20 client.10068 populate_metadata read hostname 'smithi121' 2023-11-15T03:09:49.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/3800084764 >> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] conn(0xe0dee0 0xe0e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05bb4b3740 4 client.10068 mds_command: new command op to 9990 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:09:49.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05bb4b3740 1 -- 172.21.15.121:0/3800084764 --> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xd7d6f0 con 0xe0dee0 2023-11-15T03:09:49.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05aa59c700 1 --2- 172.21.15.121:0/3800084764 >> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] conn(0xe0dee0 0xe0e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05897fa700 10 client.10068 ms_handle_connect on v2:172.21.15.121:6834/1175031592 2023-11-15T03:09:49.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.202+0000 7f05aa59c700 1 --2- 172.21.15.121:0/3800084764 >> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] conn(0xe0dee0 0xe0e2c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=5dceb3e563cfd7a9 server_cookie=413444ac984168d8 in_seq=0 out_seq=0 2023-11-15T03:09:49.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.206+0000 7f05897fa700 1 -- 172.21.15.121:0/3800084764 <== mds.0 v2:172.21.15.121:6834/1175031592 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0xd7d6f0 con 0xe0dee0 2023-11-15T03:09:49.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.206+0000 7f05897fa700 10 client.10068 handle_command_reply: tid=0 2023-11-15T03:09:49.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.206+0000 7f05bb4b3740 1 client.10068 shutdown 2023-11-15T03:09:49.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.206+0000 7f05bb4b3740 1 -- 172.21.15.121:0/3800084764 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10ad0 msgr2=0xe4f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:49.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.206+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/3800084764 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10ad0 0xe4f850 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 rx=0x7f059c009dd0 tx=0x7f059c01e070).stop 2023-11-15T03:09:49.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.206+0000 7f05bb4b3740 1 -- 172.21.15.121:0/3800084764 shutdown_connections 2023-11-15T03:09:49.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.206+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/3800084764 >> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] conn(0xe0dee0 0xe0e2c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.206+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/3800084764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe5d630 0xe63b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.206+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/3800084764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe10eb0 0xe30760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.206+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/3800084764 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10ad0 0xe4f850 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.206+0000 7f05897fa700 0 client.10068 ms_handle_reset on v2:172.21.15.121:6834/1175031592 2023-11-15T03:09:49.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.206+0000 7f05bb4b3740 1 -- 172.21.15.121:0/3800084764 >> 172.21.15.121:0/3800084764 conn(0xbef550 msgr2=0xe584b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:49.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.206+0000 7f05bb4b3740 1 -- 172.21.15.121:0/3800084764 shutdown_connections 2023-11-15T03:09:49.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.206+0000 7f05bb4b3740 1 -- 172.21.15.121:0/3800084764 wait complete. 2023-11-15T03:09:49.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.206+0000 7f05bb4b3740 20 client.10068 trim_cache size 0 max 16384 2023-11-15T03:09:49.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f05bb4b3740 1 Processor -- start 2023-11-15T03:09:49.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f05bb4b3740 1 -- start start 2023-11-15T03:09:49.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f05bb4b3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe10ad0 0xe0e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f05bb4b3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 0xe0e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f05bb4b3740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe5d630 0xe5c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f05bb4b3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0xdd4e60 con 0xe10eb0 2023-11-15T03:09:49.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f05bb4b3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0xd63540 con 0xe10ad0 2023-11-15T03:09:49.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f05bb4b3740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0xe0eca0 con 0xe5d630 2023-11-15T03:09:49.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f05aa59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 0xe0e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f05ab59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe5d630 0xe5c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f05aa59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 0xe0e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46934/0 (socket says 172.21.15.121:46934) 2023-11-15T03:09:49.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f05aa59c700 1 -- 172.21.15.121:0/437744877 learned_addr learned my addr 172.21.15.121:0/437744877 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:49.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f05aad9d700 1 --2- 172.21.15.121:0/437744877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe10ad0 0xe0e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f05ab59e700 1 -- 172.21.15.121:0/437744877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe10ad0 msgr2=0xe0e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:49.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f05ab59e700 1 --2- 172.21.15.121:0/437744877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe10ad0 0xe0e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f05ab59e700 1 -- 172.21.15.121:0/437744877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 msgr2=0xe0e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:49.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f05ab59e700 1 --2- 172.21.15.121:0/437744877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 0xe0e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f05ab59e700 1 -- 172.21.15.121:0/437744877 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac168b40 con 0xe5d630 2023-11-15T03:09:49.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f05ab59e700 1 --2- 172.21.15.121:0/437744877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe5d630 0xe5c2d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0xe581d0 tx=0x7f05a4009040).ready entity=mon.2 client_cookie=e1740ae88d23da1a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:49.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f0590ff9700 1 -- 172.21.15.121:0/437744877 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a400e040 con 0xe5d630 2023-11-15T03:09:49.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f0590ff9700 1 -- 172.21.15.121:0/437744877 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a400f040 con 0xe5d630 2023-11-15T03:09:49.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.226+0000 7f0590ff9700 1 -- 172.21.15.121:0/437744877 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4017620 con 0xe5d630 2023-11-15T03:09:49.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05bb4b3740 1 -- 172.21.15.121:0/437744877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe5d630 msgr2=0xe5c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:49.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/437744877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe5d630 0xe5c2d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0xe581d0 tx=0x7f05a4009040).stop 2023-11-15T03:09:49.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05bb4b3740 1 -- 172.21.15.121:0/437744877 shutdown_connections 2023-11-15T03:09:49.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/437744877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe5d630 0xe5c2d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/437744877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 0xe0e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/437744877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe10ad0 0xe0e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05bb4b3740 1 -- 172.21.15.121:0/437744877 >> 172.21.15.121:0/437744877 conn(0xbef550 msgr2=0xe0bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:49.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05bb4b3740 1 -- 172.21.15.121:0/437744877 shutdown_connections 2023-11-15T03:09:49.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05bb4b3740 1 -- 172.21.15.121:0/437744877 wait complete. 2023-11-15T03:09:49.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05bb4b3740 1 -- start start 2023-11-15T03:09:49.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05bb4b3740 1 Processor -- start 2023-11-15T03:09:49.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05bb4b3740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe10ad0 0xe336f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05bb4b3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 0xe33be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05bb4b3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe5d630 0xec23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05bb4b3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0xe5c8a0 con 0xe10eb0 2023-11-15T03:09:49.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05bb4b3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0xe340d0 con 0xe5d630 2023-11-15T03:09:49.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05bb4b3740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0xec5b20 con 0xe10ad0 2023-11-15T03:09:49.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05bb4b3740 10 client.0 ll_register_callbacks cb 0x7f059c01c420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:09:49.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05aa59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 0xe33be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05aa59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 0xe33be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46936/0 (socket says 172.21.15.121:46936) 2023-11-15T03:09:49.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05aa59c700 1 -- 172.21.15.121:0/529001852 learned_addr learned my addr 172.21.15.121:0/529001852 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:49.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05ab59e700 1 --2- 172.21.15.121:0/529001852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe5d630 0xec23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05aad9d700 1 --2- 172.21.15.121:0/529001852 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe10ad0 0xe336f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05aa59c700 1 -- 172.21.15.121:0/529001852 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe10ad0 msgr2=0xe336f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:49.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05aa59c700 1 --2- 172.21.15.121:0/529001852 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe10ad0 0xe336f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05aa59c700 1 -- 172.21.15.121:0/529001852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe5d630 msgr2=0xec23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:49.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05aa59c700 1 --2- 172.21.15.121:0/529001852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe5d630 0xec23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.230+0000 7f05aa59c700 1 -- 172.21.15.121:0/529001852 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a4007400 con 0xe10eb0 2023-11-15T03:09:49.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05aa59c700 1 --2- 172.21.15.121:0/529001852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 0xe33be0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 rx=0x7f05a000cab0 tx=0x7f05a004d000).ready entity=mon.0 client_cookie=9983002a68f73b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:49.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05897fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:09:49.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05897fa700 1 -- 172.21.15.121:0/529001852 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a0062030 con 0xe10eb0 2023-11-15T03:09:49.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05bb4b3740 1 -- 172.21.15.121:0/529001852 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05a0055030 con 0xe10eb0 2023-11-15T03:09:49.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05897fa700 1 -- 172.21.15.121:0/529001852 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 9104+0+0 (secure 0 0 0) 0x7f05a004b2c0 con 0xe10eb0 2023-11-15T03:09:49.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05897fa700 1 -- 172.21.15.121:0/529001852 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=296) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05a0066280 con 0xe10eb0 2023-11-15T03:09:49.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05bb4b3740 10 client.10074 fetch_fsmap learned FSMap version 296 2023-11-15T03:09:49.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05bb4b3740 1 -- 172.21.15.121:0/529001852 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=296,monmap=2+}) v3 -- 0xd856b0 con 0xe10eb0 2023-11-15T03:09:49.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05897fa700 1 -- 172.21.15.121:0/529001852 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a0055030 con 0xe10eb0 2023-11-15T03:09:49.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05897fa700 1 -- 172.21.15.121:0/529001852 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 296) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f05a001ab70 con 0xe10eb0 2023-11-15T03:09:49.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05bb4b3740 10 client.10074 fetch_fsmap finished waiting for FSMap version 296 2023-11-15T03:09:49.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05bb4b3740 10 client.10074 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:09:49.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05bb4b3740 20 client.10074 populate_metadata read hostname 'smithi121' 2023-11-15T03:09:49.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/529001852 >> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] conn(0xe34ec0 0xec28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05bb4b3740 4 client.10074 mds_command: new command op to 9990 tid=0[{"prefix": "status"}] 2023-11-15T03:09:49.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05bb4b3740 1 -- 172.21.15.121:0/529001852 --> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] -- command(tid 0: {"prefix": "status"}) v1 -- 0xc464e0 con 0xe34ec0 2023-11-15T03:09:49.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05aad9d700 1 --2- 172.21.15.121:0/529001852 >> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] conn(0xe34ec0 0xec28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05897fa700 10 client.10074 ms_handle_connect on v2:172.21.15.121:6834/1175031592 2023-11-15T03:09:49.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05aad9d700 1 --2- 172.21.15.121:0/529001852 >> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] conn(0xe34ec0 0xec28b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=c44f6c40635d645a server_cookie=5d75ba31c250ceeb in_seq=0 out_seq=0 2023-11-15T03:09:49.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05897fa700 1 -- 172.21.15.121:0/529001852 <== mds.0 v2:172.21.15.121:6834/1175031592 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0xc464e0 con 0xe34ec0 2023-11-15T03:09:49.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05897fa700 10 client.10074 handle_command_reply: tid=0 2023-11-15T03:09:49.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05bb4b3740 1 client.10074 shutdown 2023-11-15T03:09:49.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05bb4b3740 1 -- 172.21.15.121:0/529001852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 msgr2=0xe33be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:49.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/529001852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 0xe33be0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 rx=0x7f05a000cab0 tx=0x7f05a004d000).stop 2023-11-15T03:09:49.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05bb4b3740 1 -- 172.21.15.121:0/529001852 shutdown_connections 2023-11-15T03:09:49.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.234+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/529001852 >> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] conn(0xe34ec0 0xec28b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/529001852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xe5d630 0xec23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/529001852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xe10eb0 0xe33be0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05bb4b3740 1 --2- 172.21.15.121:0/529001852 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xe10ad0 0xe336f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05bb4b3740 1 -- 172.21.15.121:0/529001852 >> 172.21.15.121:0/529001852 conn(0xbef550 msgr2=0xe0bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:49.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05897fa700 0 client.10074 ms_handle_reset on v2:172.21.15.121:6834/1175031592 2023-11-15T03:09:49.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05bb4b3740 1 -- 172.21.15.121:0/529001852 shutdown_connections 2023-11-15T03:09:49.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05bb4b3740 1 -- 172.21.15.121:0/529001852 wait complete. 2023-11-15T03:09:49.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05bb4b3740 20 client.10074 trim_cache size 0 max 16384 2023-11-15T03:09:49.260 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:09:49.260 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T03:09:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T03:09:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 9990, 2023-11-15T03:09:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T03:09:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T03:09:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T03:09:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 6.8643058589999999, 2023-11-15T03:09:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 296, 2023-11-15T03:09:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 127, 2023-11-15T03:09:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 127, 2023-11-15T03:09:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 13.883629274 2023-11-15T03:09:49.263 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:09:49.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05b08f0700 1 -- 172.21.15.121:0/1151807210 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f059805a510 msgr2=0x7f059805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:49.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05b08f0700 1 --2- 172.21.15.121:0/1151807210 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f059805a510 0x7f059805c9d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f05ac15b5b0 tx=0x7f059c017000).stop 2023-11-15T03:09:49.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05b08f0700 1 -- 172.21.15.121:0/1151807210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05ac0c2ec0 msgr2=0x7f05ac0be5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:49.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05b08f0700 1 --2- 172.21.15.121:0/1151807210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05ac0c2ec0 0x7f05ac0be5c0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 rx=0x7f05a000c4c0 tx=0x7f05a0007de0).stop 2023-11-15T03:09:49.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05b08f0700 1 -- 172.21.15.121:0/1151807210 shutdown_connections 2023-11-15T03:09:49.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05b08f0700 1 --2- 172.21.15.121:0/1151807210 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f059805a510 0x7f059805c9d0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05b08f0700 1 --2- 172.21.15.121:0/1151807210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05ac153990 0x7f05ac1682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05b08f0700 1 --2- 172.21.15.121:0/1151807210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05ac0c2ec0 0x7f05ac0be5c0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05b08f0700 1 --2- 172.21.15.121:0/1151807210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05ac0bb910 0x7f05ac146530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05b08f0700 1 -- 172.21.15.121:0/1151807210 >> 172.21.15.121:0/1151807210 conn(0x7f05ac067a60 msgr2=0x7f05ac1574b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:49.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05b08f0700 1 -- 172.21.15.121:0/1151807210 shutdown_connections 2023-11-15T03:09:49.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.238+0000 7f05b08f0700 1 -- 172.21.15.121:0/1151807210 wait complete. 2023-11-15T03:09:49.268 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:09:49.268 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap key 1.00000000:subdir_head', proceeding to ls 2023-11-15T03:09:49.268 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T03:09:49.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.394+0000 7f3c6d969700 1 Processor -- start 2023-11-15T03:09:49.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.394+0000 7f3c6d969700 1 -- start start 2023-11-15T03:09:49.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c6d969700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3c680c5280 0x7f3c680a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c6d969700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c68153ef0 0x7f3c680c2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c6d969700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c680c1bf0 con 0x7f3c68153b10 2023-11-15T03:09:49.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c6d969700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c680c2ee0 con 0x7f3c680c5280 2023-11-15T03:09:49.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c6d969700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c680c12c0 con 0x7f3c68153ef0 2023-11-15T03:09:49.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c66ffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3c68153b10 0x7f3c680c4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56052/0 (socket says 172.21.15.121:56052) 2023-11-15T03:09:49.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c66ffd700 1 -- 172.21.15.121:0/1546372049 learned_addr learned my addr 172.21.15.121:0/1546372049 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:09:49.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c667fc700 1 --2- 172.21.15.121:0/1546372049 >> v2:172.21.15.190:3300/0 conn(0x7f3c680c5280 0x7f3c680a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c677fe700 1 --2- 172.21.15.121:0/1546372049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c68153ef0 0x7f3c680c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c65ffb700 1 -- 172.21.15.121:0/1546372049 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4072269025 0 0) 0x7f3c680c1bf0 con 0x7f3c68153b10 2023-11-15T03:09:49.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c65ffb700 1 -- 172.21.15.121:0/1546372049 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c500033f0 con 0x7f3c68153b10 2023-11-15T03:09:49.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c65ffb700 1 -- 172.21.15.121:0/1546372049 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3106728076 0 0) 0x7f3c500033f0 con 0x7f3c68153b10 2023-11-15T03:09:49.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c65ffb700 1 -- 172.21.15.121:0/1546372049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c68153ef0 msgr2=0x7f3c680c2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:49.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c65ffb700 1 --2- 172.21.15.121:0/1546372049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c68153ef0 0x7f3c680c2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c65ffb700 1 -- 172.21.15.121:0/1546372049 >> v2:172.21.15.190:3300/0 conn(0x7f3c680c5280 msgr2=0x7f3c680a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:49.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c65ffb700 1 --2- 172.21.15.121:0/1546372049 >> v2:172.21.15.190:3300/0 conn(0x7f3c680c5280 0x7f3c680a58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c65ffb700 1 -- 172.21.15.121:0/1546372049 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c680c15a0 con 0x7f3c68153b10 2023-11-15T03:09:49.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c65ffb700 1 -- 172.21.15.121:0/1546372049 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f3c58001470 con 0x7f3c68153b10 2023-11-15T03:09:49.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c667fc700 1 --2- 172.21.15.121:0/1546372049 >> v2:172.21.15.190:3300/0 conn(0x7f3c680c5280 0x7f3c680a58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:09:49.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c65ffb700 1 -- 172.21.15.121:0/1546372049 >> v1:172.21.15.121:6789/0 conn(0x7f3c68153b10 legacy=0x7f3c680c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:49.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c65ffb700 1 --2- 172.21.15.121:0/1546372049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c68153ef0 0x7f3c500038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c65ffb700 1 --2- 172.21.15.121:0/1546372049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c50003e10 0x7f3c50004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c65ffb700 1 --2- 172.21.15.121:0/1546372049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c50004b80 0x7f3c50005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c65ffb700 1 -- 172.21.15.121:0/1546372049 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c680c2ee0 con 0x7f3c50004b80 2023-11-15T03:09:49.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c65ffb700 1 -- 172.21.15.121:0/1546372049 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c680c12c0 con 0x7f3c50003e10 2023-11-15T03:09:49.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c65ffb700 1 -- 172.21.15.121:0/1546372049 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5000b8e0 con 0x7f3c68153ef0 2023-11-15T03:09:49.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c677fe700 1 --2- 172.21.15.121:0/1546372049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c50004b80 0x7f3c50005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c66ffd700 1 --2- 172.21.15.121:0/1546372049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c68153ef0 0x7f3c500038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.398+0000 7f3c667fc700 1 --2- 172.21.15.121:0/1546372049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c50003e10 0x7f3c50004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.402+0000 7f3c677fe700 1 -- 172.21.15.121:0/1546372049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c68153ef0 msgr2=0x7f3c500038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:49.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.402+0000 7f3c677fe700 1 --2- 172.21.15.121:0/1546372049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c68153ef0 0x7f3c500038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.402+0000 7f3c677fe700 1 -- 172.21.15.121:0/1546372049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c50003e10 msgr2=0x7f3c50004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:49.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.402+0000 7f3c677fe700 1 --2- 172.21.15.121:0/1546372049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c50003e10 0x7f3c50004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.402+0000 7f3c677fe700 1 -- 172.21.15.121:0/1546372049 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c500033f0 con 0x7f3c50004b80 2023-11-15T03:09:49.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.402+0000 7f3c677fe700 1 --2- 172.21.15.121:0/1546372049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c50004b80 0x7f3c50005000 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 rx=0x7f3c6000e210 tx=0x7f3c6000b040).ready entity=mon.0 client_cookie=b08ddef5a21cd5c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:49.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.402+0000 7f3c65ffb700 1 -- 172.21.15.121:0/1546372049 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c60009ac0 con 0x7f3c50004b80 2023-11-15T03:09:49.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.402+0000 7f3c65ffb700 1 -- 172.21.15.121:0/1546372049 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6000f040 con 0x7f3c50004b80 2023-11-15T03:09:49.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.402+0000 7f3c65ffb700 1 -- 172.21.15.121:0/1546372049 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6000ecd0 con 0x7f3c50004b80 2023-11-15T03:09:49.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.402+0000 7f3c6d969700 1 -- 172.21.15.121:0/1546372049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c50004b80 msgr2=0x7f3c50005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:49.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.402+0000 7f3c6d969700 1 --2- 172.21.15.121:0/1546372049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c50004b80 0x7f3c50005000 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 rx=0x7f3c6000e210 tx=0x7f3c6000b040).stop 2023-11-15T03:09:49.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.402+0000 7f3c677fe700 1 -- 172.21.15.121:0/1546372049 reap_dead start 2023-11-15T03:09:49.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.402+0000 7f3c6d969700 1 -- 172.21.15.121:0/1546372049 shutdown_connections 2023-11-15T03:09:49.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.402+0000 7f3c6d969700 1 -- 172.21.15.121:0/1546372049 >> 172.21.15.121:0/1546372049 conn(0x7f3c680c66d0 msgr2=0x7f3c680c6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:49.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.402+0000 7f3c6d969700 1 -- 172.21.15.121:0/1546372049 shutdown_connections 2023-11-15T03:09:49.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.402+0000 7f3c6d969700 1 -- 172.21.15.121:0/1546372049 wait complete. 2023-11-15T03:09:49.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c6d969700 1 Processor -- start 2023-11-15T03:09:49.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c6d969700 1 -- start start 2023-11-15T03:09:49.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c6d969700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c50004b80 0x7f3c680a7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c6d969700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c680a6ca0 0x7f3c6814ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c6d969700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c6814cf10 0x7f3c68163950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c6d969700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c681646e0 con 0x7f3c6814cf10 2023-11-15T03:09:49.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c6d969700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6814d370 con 0x7f3c50004b80 2023-11-15T03:09:49.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c6d969700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c680a5ee0 con 0x7f3c680a6ca0 2023-11-15T03:09:49.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c677fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c6814cf10 0x7f3c68163950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c677fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c6814cf10 0x7f3c68163950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46944/0 (socket says 172.21.15.121:46944) 2023-11-15T03:09:49.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c677fe700 1 -- 172.21.15.121:0/2113035603 learned_addr learned my addr 172.21.15.121:0/2113035603 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:49.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c667fc700 1 --2- 172.21.15.121:0/2113035603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c680a6ca0 0x7f3c6814ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c66ffd700 1 --2- 172.21.15.121:0/2113035603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c50004b80 0x7f3c680a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c677fe700 1 -- 172.21.15.121:0/2113035603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c680a6ca0 msgr2=0x7f3c6814ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:49.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c677fe700 1 --2- 172.21.15.121:0/2113035603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c680a6ca0 0x7f3c6814ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c677fe700 1 -- 172.21.15.121:0/2113035603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c50004b80 msgr2=0x7f3c680a7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:49.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c677fe700 1 --2- 172.21.15.121:0/2113035603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c50004b80 0x7f3c680a7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c677fe700 1 -- 172.21.15.121:0/2113035603 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c680a61c0 con 0x7f3c6814cf10 2023-11-15T03:09:49.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c66ffd700 1 --2- 172.21.15.121:0/2113035603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c50004b80 0x7f3c680a7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:09:49.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c667fc700 1 --2- 172.21.15.121:0/2113035603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c680a6ca0 0x7f3c6814ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:09:49.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c677fe700 1 --2- 172.21.15.121:0/2113035603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c6814cf10 0x7f3c68163950 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 rx=0x7f3c60014000 tx=0x7f3c6001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:49.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c4effd700 1 -- 172.21.15.121:0/2113035603 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c60021030 con 0x7f3c6814cf10 2023-11-15T03:09:49.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c4effd700 1 -- 172.21.15.121:0/2113035603 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c60009ac0 con 0x7f3c6814cf10 2023-11-15T03:09:49.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c4effd700 1 -- 172.21.15.121:0/2113035603 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c60013e70 con 0x7f3c6814cf10 2023-11-15T03:09:49.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.406+0000 7f3c6d969700 1 -- 172.21.15.121:0/2113035603 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c680a64a0 con 0x7f3c6814cf10 2023-11-15T03:09:49.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.410+0000 7f3c6d969700 1 -- 172.21.15.121:0/2113035603 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c680a7bf0 con 0x7f3c6814cf10 2023-11-15T03:09:49.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.410+0000 7f3c6d969700 1 -- 172.21.15.121:0/2113035603 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6805eeb0 con 0x7f3c6814cf10 2023-11-15T03:09:49.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.410+0000 7f3c4effd700 1 -- 172.21.15.121:0/2113035603 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3c60024070 con 0x7f3c6814cf10 2023-11-15T03:09:49.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.410+0000 7f3c4effd700 1 --2- 172.21.15.121:0/2113035603 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3c5405a540 0x7f3c5405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:49.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.410+0000 7f3c4effd700 1 -- 172.21.15.121:0/2113035603 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9104+0+0 (secure 0 0 0) 0x7f3c60022070 con 0x7f3c6814cf10 2023-11-15T03:09:49.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.414+0000 7f3c66ffd700 1 --2- 172.21.15.121:0/2113035603 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3c5405a540 0x7f3c5405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:49.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.414+0000 7f3c66ffd700 1 --2- 172.21.15.121:0/2113035603 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3c5405a540 0x7f3c5405ca00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f3c58000e10 tx=0x7f3c58009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:49.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.418+0000 7f3c4effd700 1 -- 172.21.15.121:0/2113035603 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3c60078e60 con 0x7f3c6814cf10 2023-11-15T03:09:49.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.614+0000 7f3c6d969700 1 -- 172.21.15.121:0/2113035603 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3c680c1880 con 0x7f3c6814cf10 2023-11-15T03:09:49.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.614+0000 7f3c4effd700 1 -- 172.21.15.121:0/2113035603 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v296) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3c60049840 con 0x7f3c6814cf10 2023-11-15T03:09:49.626 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T03:09:49.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.618+0000 7f3c6d969700 1 -- 172.21.15.121:0/2113035603 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3c5405a540 msgr2=0x7f3c5405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:49.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.618+0000 7f3c6d969700 1 --2- 172.21.15.121:0/2113035603 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3c5405a540 0x7f3c5405ca00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f3c58000e10 tx=0x7f3c58009040).stop 2023-11-15T03:09:49.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.618+0000 7f3c6d969700 1 -- 172.21.15.121:0/2113035603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c6814cf10 msgr2=0x7f3c68163950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:49.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.618+0000 7f3c6d969700 1 --2- 172.21.15.121:0/2113035603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c6814cf10 0x7f3c68163950 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 rx=0x7f3c60014000 tx=0x7f3c6001a000).stop 2023-11-15T03:09:49.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.618+0000 7f3c6d969700 1 -- 172.21.15.121:0/2113035603 shutdown_connections 2023-11-15T03:09:49.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.618+0000 7f3c6d969700 1 --2- 172.21.15.121:0/2113035603 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3c5405a540 0x7f3c5405ca00 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.618+0000 7f3c6d969700 1 --2- 172.21.15.121:0/2113035603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c6814cf10 0x7f3c68163950 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.618+0000 7f3c6d969700 1 --2- 172.21.15.121:0/2113035603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c680a6ca0 0x7f3c6814ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.618+0000 7f3c6d969700 1 --2- 172.21.15.121:0/2113035603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c50004b80 0x7f3c680a7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:49.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.618+0000 7f3c6d969700 1 -- 172.21.15.121:0/2113035603 >> 172.21.15.121:0/2113035603 conn(0x7f3c680c66d0 msgr2=0x7f3c680c2080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:49.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.618+0000 7f3c6d969700 1 -- 172.21.15.121:0/2113035603 shutdown_connections 2023-11-15T03:09:49.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:49.618+0000 7f3c6d969700 1 -- 172.21.15.121:0/2113035603 wait complete. 2023-11-15T03:09:49.645 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T03:09:49.646 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T03:09:49.646 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T03:09:49.647 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T03:09:49.647 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T03:09:49.647 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T03:09:49.648 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T03:09:49.648 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T03:09:49.649 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:09:49.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:49.649 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T03:09:49.670 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T03:09:49.670 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T03:09:49.671 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T03:09:49.671 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:09:49.671 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T03:09:49.671 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:09:49.671 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:09:49.672 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:09:49.672 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T03:09:49.672 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:09:49.672 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:09:49.672 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T03:09:49.673 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:09:49.673 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:09:49.673 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T03:09:49.673 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:09:49.673 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T03:09:49.674 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 41650sec preferred_lft 41650sec 2023-11-15T03:09:49.674 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T03:09:49.674 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:09:49.674 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T03:09:49.674 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:09:49.675 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T03:09:49.675 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:09:49.675 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T03:09:49.675 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T03:09:49.675 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T03:09:49.676 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:09:49.676 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T03:09:49.676 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:09:49.676 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:09:49.676 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T03:09:49.677 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T03:09:49.677 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:09:49.677 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:09:49.677 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T03:09:49.677 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T03:09:49.678 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:09:49.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:49.679 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T03:09:49.765 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T03:09:49.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:49.770 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T03:09:49.830 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T03:09:49.831 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T03:09:49.831 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T03:09:49.831 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T03:09:49.831 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T03:09:49.832 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T03:09:49.832 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T03:09:49.832 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T03:09:49.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:49.833 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T03:09:49.833 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T03:09:49.834 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T03:09:49.834 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T03:09:49.834 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T03:09:49.835 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T03:09:49.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:49.974 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T03:09:49.992 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T03:09:49.992 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T03:09:49.993 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:09:49.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:49.994 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T03:09:50.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:50.070 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T03:09:50.127 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T03:09:50.128 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:09:50.247 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:09:50.248 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:09:50.271 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:09:50.272 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:09:50.276 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T03:09:50.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:50.277 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T03:09:50.309 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-15T03:09:50.310 DEBUG:teuthology.orchestra.run.smithi121:> import os 2023-11-15T03:09:50.310 DEBUG:teuthology.orchestra.run.smithi121:> import stat 2023-11-15T03:09:50.311 DEBUG:teuthology.orchestra.run.smithi121:> import json 2023-11-15T03:09:50.312 DEBUG:teuthology.orchestra.run.smithi121:> import sys 2023-11-15T03:09:50.312 DEBUG:teuthology.orchestra.run.smithi121:> 2023-11-15T03:09:50.313 DEBUG:teuthology.orchestra.run.smithi121:> try: 2023-11-15T03:09:50.313 DEBUG:teuthology.orchestra.run.smithi121:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/./subdir'"'"') 2023-11-15T03:09:50.314 DEBUG:teuthology.orchestra.run.smithi121:> except OSError as e: 2023-11-15T03:09:50.314 DEBUG:teuthology.orchestra.run.smithi121:> sys.exit(e.errno) 2023-11-15T03:09:50.314 DEBUG:teuthology.orchestra.run.smithi121:> 2023-11-15T03:09:50.315 DEBUG:teuthology.orchestra.run.smithi121:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-11-15T03:09:50.315 DEBUG:teuthology.orchestra.run.smithi121:> print(json.dumps( 2023-11-15T03:09:50.315 DEBUG:teuthology.orchestra.run.smithi121:> dict([(a, getattr(s, a)) for a in attrs]), 2023-11-15T03:09:50.316 DEBUG:teuthology.orchestra.run.smithi121:> indent=2)) 2023-11-15T03:09:50.316 DEBUG:teuthology.orchestra.run.smithi121:> ' 2023-11-15T03:09:50.357 INFO:teuthology.orchestra.run:waiting for 20 2023-11-15T03:09:50.625 INFO:teuthology.orchestra.run.smithi121.stderr:daemon-helper: command failed with exit status 5 2023-11-15T03:09:53.283 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:09:53.274+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-11-15T03:09:56.460 DEBUG:teuthology.orchestra.run:got remote process result: 5 2023-11-15T03:09:56.460 INFO:tasks.cephfs.test_damage:Result: EIO on client 2023-11-15T03:09:56.461 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:09:56.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.614+0000 7f2aaee8f700 1 Processor -- start 2023-11-15T03:09:56.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.614+0000 7f2aaee8f700 1 -- start start 2023-11-15T03:09:56.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.618+0000 7f2aaee8f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2aa8153cd0 0x7f2aa8067140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:56.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.618+0000 7f2aaee8f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa81583f0 con 0x7f2aa8157ef0 2023-11-15T03:09:56.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.618+0000 7f2aaee8f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa81586d0 con 0x7f2aa8066d60 2023-11-15T03:09:56.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.618+0000 7f2aaee8f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa80c00f0 con 0x7f2aa8153cd0 2023-11-15T03:09:56.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.618+0000 7f2aad68c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f2aa8157ef0 0x7f2aa81535c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:46378/0 (socket says 172.21.15.121:46378) 2023-11-15T03:09:56.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.618+0000 7f2aad68c700 1 -- 172.21.15.121:0/3223605656 learned_addr learned my addr 172.21.15.121:0/3223605656 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:09:56.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.618+0000 7f2aae68e700 1 --2- 172.21.15.121:0/3223605656 >> v2:172.21.15.121:3300/0 conn(0x7f2aa8153cd0 0x7f2aa8067140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:56.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.622+0000 7f2aace8b700 1 -- 172.21.15.121:0/3223605656 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791348350 0 0) 0x7f2aa81583f0 con 0x7f2aa8157ef0 2023-11-15T03:09:56.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.622+0000 7f2aace8b700 1 -- 172.21.15.121:0/3223605656 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a940033f0 con 0x7f2aa8157ef0 2023-11-15T03:09:56.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.622+0000 7f2aace8b700 1 -- 172.21.15.121:0/3223605656 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524710039 0 0) 0x7f2aa81586d0 con 0x7f2aa8066d60 2023-11-15T03:09:56.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.622+0000 7f2aace8b700 1 -- 172.21.15.121:0/3223605656 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2aa81583f0 con 0x7f2aa8066d60 2023-11-15T03:09:56.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.622+0000 7f2aae68e700 1 -- 172.21.15.121:0/3223605656 >> v1:172.21.15.190:6789/0 conn(0x7f2aa8066d60 legacy=0x7f2aa81577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:56.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.622+0000 7f2aae68e700 1 -- 172.21.15.121:0/3223605656 >> v1:172.21.15.121:6789/0 conn(0x7f2aa8157ef0 legacy=0x7f2aa81535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:56.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.622+0000 7f2aae68e700 1 -- 172.21.15.121:0/3223605656 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa8065890 con 0x7f2aa8153cd0 2023-11-15T03:09:56.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.622+0000 7f2aae68e700 1 --2- 172.21.15.121:0/3223605656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aa8153cd0 0x7f2aa8067140 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 rx=0x7f2aa0009e90 tx=0x7f2aa0009780).ready entity=mon.0 client_cookie=9921804968571a5d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:56.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.622+0000 7f2aace8b700 1 -- 172.21.15.121:0/3223605656 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa000e070 con 0x7f2aa8153cd0 2023-11-15T03:09:56.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.622+0000 7f2aace8b700 1 -- 172.21.15.121:0/3223605656 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa000ba70 con 0x7f2aa8153cd0 2023-11-15T03:09:56.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.622+0000 7f2aace8b700 1 -- 172.21.15.121:0/3223605656 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa000bdf0 con 0x7f2aa8153cd0 2023-11-15T03:09:56.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.622+0000 7f2aaee8f700 1 -- 172.21.15.121:0/3223605656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aa8153cd0 msgr2=0x7f2aa8067140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:56.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.622+0000 7f2aaee8f700 1 --2- 172.21.15.121:0/3223605656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aa8153cd0 0x7f2aa8067140 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 rx=0x7f2aa0009e90 tx=0x7f2aa0009780).stop 2023-11-15T03:09:56.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.622+0000 7f2aaee8f700 1 -- 172.21.15.121:0/3223605656 shutdown_connections 2023-11-15T03:09:56.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.622+0000 7f2aaee8f700 1 --2- 172.21.15.121:0/3223605656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aa8153cd0 0x7f2aa8067140 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:56.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.622+0000 7f2aaee8f700 1 -- 172.21.15.121:0/3223605656 >> 172.21.15.121:0/3223605656 conn(0x7f2aa8062190 msgr2=0x7f2aa8062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:56.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.622+0000 7f2aaee8f700 1 -- 172.21.15.121:0/3223605656 shutdown_connections 2023-11-15T03:09:56.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.622+0000 7f2aaee8f700 1 -- 172.21.15.121:0/3223605656 wait complete. 2023-11-15T03:09:56.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.626+0000 7f2aaee8f700 1 Processor -- start 2023-11-15T03:09:56.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.626+0000 7f2aaee8f700 1 -- start start 2023-11-15T03:09:56.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.626+0000 7f2aaee8f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aa8066d60 0x7f2aa80c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:56.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.626+0000 7f2aaee8f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2aa8153cd0 0x7f2aa80c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:56.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.626+0000 7f2aaee8f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2aa8157ef0 0x7f2aa814ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:56.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.626+0000 7f2aaee8f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa80c49d0 con 0x7f2aa8066d60 2023-11-15T03:09:56.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.626+0000 7f2aaee8f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa80c70a0 con 0x7f2aa8157ef0 2023-11-15T03:09:56.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.626+0000 7f2aaee8f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa80c4c10 con 0x7f2aa8153cd0 2023-11-15T03:09:56.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.626+0000 7f2aade8d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aa8066d60 0x7f2aa80c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:56.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.626+0000 7f2aae68e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2aa8157ef0 0x7f2aa814ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:56.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.630+0000 7f2aae68e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2aa8157ef0 0x7f2aa814ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:40136/0 (socket says 172.21.15.121:40136) 2023-11-15T03:09:56.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.630+0000 7f2aae68e700 1 -- 172.21.15.121:0/3861131605 learned_addr learned my addr 172.21.15.121:0/3861131605 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:56.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.630+0000 7f2aade8d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aa8066d60 0x7f2aa80c44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40090/0 (socket says 172.21.15.121:40090) 2023-11-15T03:09:56.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.630+0000 7f2aad68c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2aa8153cd0 0x7f2aa80c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:56.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.630+0000 7f2aade8d700 1 -- 172.21.15.121:0/3861131605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2aa8153cd0 msgr2=0x7f2aa80c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:56.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.630+0000 7f2aade8d700 1 --2- 172.21.15.121:0/3861131605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2aa8153cd0 0x7f2aa80c6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:56.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.630+0000 7f2aade8d700 1 -- 172.21.15.121:0/3861131605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2aa8157ef0 msgr2=0x7f2aa814ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:56.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.630+0000 7f2aade8d700 1 --2- 172.21.15.121:0/3861131605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2aa8157ef0 0x7f2aa814ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:56.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.630+0000 7f2aade8d700 1 -- 172.21.15.121:0/3861131605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa0009a20 con 0x7f2aa8066d60 2023-11-15T03:09:56.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.630+0000 7f2aae68e700 1 --2- 172.21.15.121:0/3861131605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2aa8157ef0 0x7f2aa814ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:09:56.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.630+0000 7f2aade8d700 1 --2- 172.21.15.121:0/3861131605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aa8066d60 0x7f2aa80c44e0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 rx=0x7f2aa400c420 tx=0x7f2aa4007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:56.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.630+0000 7f2a9dffb700 1 -- 172.21.15.121:0/3861131605 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4012400 con 0x7f2aa8066d60 2023-11-15T03:09:56.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.630+0000 7f2a9dffb700 1 -- 172.21.15.121:0/3861131605 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa4012560 con 0x7f2aa8066d60 2023-11-15T03:09:56.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.630+0000 7f2a9dffb700 1 -- 172.21.15.121:0/3861131605 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa40128e0 con 0x7f2aa8066d60 2023-11-15T03:09:56.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.630+0000 7f2aaee8f700 1 -- 172.21.15.121:0/3861131605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa80c4f50 con 0x7f2aa8066d60 2023-11-15T03:09:56.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.630+0000 7f2aaee8f700 1 -- 172.21.15.121:0/3861131605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa80aa700 con 0x7f2aa8066d60 2023-11-15T03:09:56.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.630+0000 7f2aaee8f700 1 -- 172.21.15.121:0/3861131605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa80c12b0 con 0x7f2aa8066d60 2023-11-15T03:09:56.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.634+0000 7f2a9dffb700 1 -- 172.21.15.121:0/3861131605 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2aa4012c90 con 0x7f2aa8066d60 2023-11-15T03:09:56.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.634+0000 7f2a9dffb700 1 --2- 172.21.15.121:0/3861131605 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2a8c05a540 0x7f2a8c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:56.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.634+0000 7f2a9dffb700 1 -- 172.21.15.121:0/3861131605 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9104+0+0 (secure 0 0 0) 0x7f2aa4074700 con 0x7f2aa8066d60 2023-11-15T03:09:56.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.634+0000 7f2aad68c700 1 --2- 172.21.15.121:0/3861131605 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2a8c05a540 0x7f2a8c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:56.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.634+0000 7f2aad68c700 1 --2- 172.21.15.121:0/3861131605 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2a8c05a540 0x7f2a8c05ca00 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f2aa8156e70 tx=0x7f2a98009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:56.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.642+0000 7f2a9dffb700 1 -- 172.21.15.121:0/3861131605 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2aa80c12b0 con 0x7f2aa8066d60 2023-11-15T03:09:56.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.842+0000 7f2aaee8f700 1 -- 172.21.15.121:0/3861131605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2aa806bcf0 con 0x7f2aa8066d60 2023-11-15T03:09:56.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.842+0000 7f2a9dffb700 1 -- 172.21.15.121:0/3861131605 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 297 v297) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f2aa4047350 con 0x7f2aa8066d60 2023-11-15T03:09:56.853 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:09:56.854 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":297,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":297,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:09:53.278188+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9990":{"gid":9990,"name":"a","rank":0,"incarnation":293,"state":"up:active","state_seq":7,"addr":"172.21.15.121:6836/1175031592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1175031592},{"type":"v1","addr":"172.21.15.121:6836","nonce":1175031592}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:09:56.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.846+0000 7f2aaee8f700 1 -- 172.21.15.121:0/3861131605 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2a8c05a540 msgr2=0x7f2a8c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:56.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.846+0000 7f2aaee8f700 1 --2- 172.21.15.121:0/3861131605 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2a8c05a540 0x7f2a8c05ca00 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f2aa8156e70 tx=0x7f2a98009040).stop 2023-11-15T03:09:56.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.846+0000 7f2aaee8f700 1 -- 172.21.15.121:0/3861131605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aa8066d60 msgr2=0x7f2aa80c44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:56.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.846+0000 7f2aaee8f700 1 --2- 172.21.15.121:0/3861131605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aa8066d60 0x7f2aa80c44e0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 rx=0x7f2aa400c420 tx=0x7f2aa4007bd0).stop 2023-11-15T03:09:56.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.846+0000 7f2aaee8f700 1 -- 172.21.15.121:0/3861131605 shutdown_connections 2023-11-15T03:09:56.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.846+0000 7f2aaee8f700 1 --2- 172.21.15.121:0/3861131605 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2a8c05a540 0x7f2a8c05ca00 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:56.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.846+0000 7f2aaee8f700 1 --2- 172.21.15.121:0/3861131605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2aa8157ef0 0x7f2aa814ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:56.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.846+0000 7f2aaee8f700 1 --2- 172.21.15.121:0/3861131605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2aa8153cd0 0x7f2aa80c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:56.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.846+0000 7f2aaee8f700 1 --2- 172.21.15.121:0/3861131605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aa8066d60 0x7f2aa80c44e0 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:56.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.846+0000 7f2aaee8f700 1 -- 172.21.15.121:0/3861131605 >> 172.21.15.121:0/3861131605 conn(0x7f2aa8062190 msgr2=0x7f2aa8151ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:56.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.846+0000 7f2aaee8f700 1 -- 172.21.15.121:0/3861131605 shutdown_connections 2023-11-15T03:09:56.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:56.846+0000 7f2aaee8f700 1 -- 172.21.15.121:0/3861131605 wait complete. 2023-11-15T03:09:56.860 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 297 2023-11-15T03:09:56.880 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2023-11-15T03:09:57.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.034+0000 7f3b26d07700 1 Processor -- start 2023-11-15T03:09:57.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.034+0000 7f3b26d07700 1 -- start start 2023-11-15T03:09:57.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.038+0000 7f3b26d07700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3b200acd00 0x7f3b200ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.038+0000 7f3b26d07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3b20066330 0x7f3b20153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.038+0000 7f3b26d07700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b20065ea0 con 0x7f3b200c5c70 2023-11-15T03:09:57.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.038+0000 7f3b26d07700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b20066bb0 con 0x7f3b200acd00 2023-11-15T03:09:57.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.038+0000 7f3b26d07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b20065890 con 0x7f3b20066330 2023-11-15T03:09:57.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.038+0000 7f3b25d05700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3b200acd00 0x7f3b200ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.038+0000 7f3b25d05700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3b200acd00 0x7f3b200ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40094/0 (socket says 172.21.15.121:40094) 2023-11-15T03:09:57.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.038+0000 7f3b25d05700 1 -- 172.21.15.121:0/3192086846 learned_addr learned my addr 172.21.15.121:0/3192086846 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:57.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.038+0000 7f3b26506700 1 --2- 172.21.15.121:0/3192086846 >> v2:172.21.15.190:3300/0 conn(0x7f3b20066330 0x7f3b20153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.038+0000 7f3b25d05700 1 -- 172.21.15.121:0/3192086846 >> v2:172.21.15.190:3300/0 conn(0x7f3b20066330 msgr2=0x7f3b20153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:57.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.038+0000 7f3b25d05700 1 --2- 172.21.15.121:0/3192086846 >> v2:172.21.15.190:3300/0 conn(0x7f3b20066330 0x7f3b20153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.038+0000 7f3b25d05700 1 -- 172.21.15.121:0/3192086846 >> v1:172.21.15.190:6789/0 conn(0x7f3b200c5c70 legacy=0x7f3b201536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:57.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.038+0000 7f3b25d05700 1 -- 172.21.15.121:0/3192086846 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b200c64a0 con 0x7f3b200acd00 2023-11-15T03:09:57.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.038+0000 7f3b25d05700 1 --2- 172.21.15.121:0/3192086846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b200acd00 0x7f3b200ad0e0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 rx=0x7f3b200c60d0 tx=0x7f3b1c009060).ready entity=mon.0 client_cookie=3f347c7820de6877 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:57.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.038+0000 7f3b24d03700 1 -- 172.21.15.121:0/3192086846 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c005880 con 0x7f3b200acd00 2023-11-15T03:09:57.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.042+0000 7f3b24d03700 1 -- 172.21.15.121:0/3192086846 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1c0059e0 con 0x7f3b200acd00 2023-11-15T03:09:57.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.042+0000 7f3b24d03700 1 -- 172.21.15.121:0/3192086846 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c005d60 con 0x7f3b200acd00 2023-11-15T03:09:57.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.042+0000 7f3b26d07700 1 -- 172.21.15.121:0/3192086846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b200acd00 msgr2=0x7f3b200ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:57.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.042+0000 7f3b26d07700 1 --2- 172.21.15.121:0/3192086846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b200acd00 0x7f3b200ad0e0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 rx=0x7f3b200c60d0 tx=0x7f3b1c009060).stop 2023-11-15T03:09:57.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.042+0000 7f3b26d07700 1 -- 172.21.15.121:0/3192086846 shutdown_connections 2023-11-15T03:09:57.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.042+0000 7f3b26d07700 1 --2- 172.21.15.121:0/3192086846 >> v2:172.21.15.190:3300/0 conn(0x7f3b20066330 0x7f3b20153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.042+0000 7f3b26d07700 1 --2- 172.21.15.121:0/3192086846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b200acd00 0x7f3b200ad0e0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.042+0000 7f3b26d07700 1 -- 172.21.15.121:0/3192086846 >> 172.21.15.121:0/3192086846 conn(0x7f3b20063fc0 msgr2=0x7f3b20067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:57.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.042+0000 7f3b26d07700 1 -- 172.21.15.121:0/3192086846 shutdown_connections 2023-11-15T03:09:57.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.042+0000 7f3b26d07700 1 -- 172.21.15.121:0/3192086846 wait complete. 2023-11-15T03:09:57.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.046+0000 7f3b26d07700 1 Processor -- start 2023-11-15T03:09:57.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.046+0000 7f3b26d07700 1 -- start start 2023-11-15T03:09:57.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.046+0000 7f3b26d07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b20066330 0x7f3b20153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.046+0000 7f3b26d07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b200acd00 0x7f3b200c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.046+0000 7f3b26d07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b200c5c70 0x7f3b200c3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.046+0000 7f3b26d07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b20155c80 con 0x7f3b200acd00 2023-11-15T03:09:57.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.046+0000 7f3b26d07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b201567e0 con 0x7f3b20066330 2023-11-15T03:09:57.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.046+0000 7f3b26d07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b20155eb0 con 0x7f3b200c5c70 2023-11-15T03:09:57.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.046+0000 7f3b25504700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b200acd00 0x7f3b200c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.046+0000 7f3b25504700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b200acd00 0x7f3b200c9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40110/0 (socket says 172.21.15.121:40110) 2023-11-15T03:09:57.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.046+0000 7f3b25504700 1 -- 172.21.15.121:0/3605825935 learned_addr learned my addr 172.21.15.121:0/3605825935 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:57.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.046+0000 7f3b25d05700 1 --2- 172.21.15.121:0/3605825935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b20066330 0x7f3b20153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.046+0000 7f3b26506700 1 --2- 172.21.15.121:0/3605825935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b200c5c70 0x7f3b200c3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.046+0000 7f3b25504700 1 -- 172.21.15.121:0/3605825935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b200c5c70 msgr2=0x7f3b200c3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:57.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.046+0000 7f3b25504700 1 --2- 172.21.15.121:0/3605825935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b200c5c70 0x7f3b200c3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.046+0000 7f3b25504700 1 -- 172.21.15.121:0/3605825935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b20066330 msgr2=0x7f3b20153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:57.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.046+0000 7f3b25504700 1 --2- 172.21.15.121:0/3605825935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b20066330 0x7f3b20153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.046+0000 7f3b25504700 1 -- 172.21.15.121:0/3605825935 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c009300 con 0x7f3b200acd00 2023-11-15T03:09:57.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.046+0000 7f3b25504700 1 --2- 172.21.15.121:0/3605825935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b200acd00 0x7f3b200c9970 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 rx=0x7f3b1000c440 tx=0x7f3b10007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:57.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.050+0000 7f3b15ffb700 1 -- 172.21.15.121:0/3605825935 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1000ce40 con 0x7f3b200acd00 2023-11-15T03:09:57.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.050+0000 7f3b15ffb700 1 -- 172.21.15.121:0/3605825935 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b10007020 con 0x7f3b200acd00 2023-11-15T03:09:57.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.050+0000 7f3b26d07700 1 -- 172.21.15.121:0/3605825935 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b20156190 con 0x7f3b200acd00 2023-11-15T03:09:57.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.050+0000 7f3b15ffb700 1 -- 172.21.15.121:0/3605825935 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b100073c0 con 0x7f3b200acd00 2023-11-15T03:09:57.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.050+0000 7f3b26d07700 1 -- 172.21.15.121:0/3605825935 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b20154330 con 0x7f3b200acd00 2023-11-15T03:09:57.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.050+0000 7f3b15ffb700 1 -- 172.21.15.121:0/3605825935 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3b100068b0 con 0x7f3b200acd00 2023-11-15T03:09:57.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.050+0000 7f3b15ffb700 1 --2- 172.21.15.121:0/3605825935 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3b0405a510 0x7f3b0405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.050+0000 7f3b15ffb700 1 -- 172.21.15.121:0/3605825935 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9104+0+0 (secure 0 0 0) 0x7f3b10074660 con 0x7f3b200acd00 2023-11-15T03:09:57.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.054+0000 7f3b25d05700 1 --2- 172.21.15.121:0/3605825935 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3b0405a510 0x7f3b0405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.054+0000 7f3b25d05700 1 --2- 172.21.15.121:0/3605825935 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3b0405a510 0x7f3b0405c9d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f3b1c00b3c0 tx=0x7f3b1c016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:57.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.054+0000 7f3b32342740 1 Processor -- start 2023-11-15T03:09:57.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.054+0000 7f3b32342740 1 -- start start 2023-11-15T03:09:57.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.054+0000 7f3b32342740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a94810 0x1a94bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.054+0000 7f3b32342740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a98810 0x1a50310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.054+0000 7f3b32342740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a4b110 0x1a4b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.054+0000 7f3b32342740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x198c3f0 con 0x1a4b110 2023-11-15T03:09:57.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.054+0000 7f3b32342740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x19af580 con 0x1a94810 2023-11-15T03:09:57.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.054+0000 7f3b32342740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1978d50 con 0x1a98810 2023-11-15T03:09:57.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b26506700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a4b110 0x1a4b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b26506700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a4b110 0x1a4b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40124/0 (socket says 172.21.15.121:40124) 2023-11-15T03:09:57.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b26506700 1 -- 172.21.15.121:0/202010001 learned_addr learned my addr 172.21.15.121:0/202010001 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:57.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b25d05700 1 --2- 172.21.15.121:0/202010001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a94810 0x1a94bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b25504700 1 --2- 172.21.15.121:0/202010001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a98810 0x1a50310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b26506700 1 -- 172.21.15.121:0/202010001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a98810 msgr2=0x1a50310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:57.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b26506700 1 --2- 172.21.15.121:0/202010001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a98810 0x1a50310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b26506700 1 -- 172.21.15.121:0/202010001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a94810 msgr2=0x1a94bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:57.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b26506700 1 --2- 172.21.15.121:0/202010001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a94810 0x1a94bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b26506700 1 -- 172.21.15.121:0/202010001 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x19cc830 con 0x1a4b110 2023-11-15T03:09:57.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b25504700 1 --2- 172.21.15.121:0/202010001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a98810 0x1a50310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:09:57.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b25d05700 1 --2- 172.21.15.121:0/202010001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a94810 0x1a94bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:09:57.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b26506700 1 --2- 172.21.15.121:0/202010001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a4b110 0x1a4b590 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 rx=0x7f3b180073f0 tx=0x7f3b1800b040).ready entity=mon.0 client_cookie=bb0ae782a5a1c1eb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:57.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b0affd700 1 -- 172.21.15.121:0/202010001 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b18007e80 con 0x1a4b110 2023-11-15T03:09:57.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b0affd700 1 -- 172.21.15.121:0/202010001 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1800f030 con 0x1a4b110 2023-11-15T03:09:57.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b0affd700 1 -- 172.21.15.121:0/202010001 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b180126f0 con 0x1a4b110 2023-11-15T03:09:57.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b32342740 1 -- 172.21.15.121:0/202010001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a4b110 msgr2=0x1a4b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:57.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b32342740 1 --2- 172.21.15.121:0/202010001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a4b110 0x1a4b590 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 rx=0x7f3b180073f0 tx=0x7f3b1800b040).stop 2023-11-15T03:09:57.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b32342740 1 -- 172.21.15.121:0/202010001 shutdown_connections 2023-11-15T03:09:57.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b32342740 1 --2- 172.21.15.121:0/202010001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a4b110 0x1a4b590 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b32342740 1 --2- 172.21.15.121:0/202010001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a98810 0x1a50310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b32342740 1 --2- 172.21.15.121:0/202010001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a94810 0x1a94bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b32342740 1 -- 172.21.15.121:0/202010001 >> 172.21.15.121:0/202010001 conn(0x1831790 msgr2=0x1a98ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:57.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b32342740 1 -- 172.21.15.121:0/202010001 shutdown_connections 2023-11-15T03:09:57.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.058+0000 7f3b32342740 1 -- 172.21.15.121:0/202010001 wait complete. 2023-11-15T03:09:57.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.062+0000 7f3b32342740 1 -- start start 2023-11-15T03:09:57.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.062+0000 7f3b32342740 1 Processor -- start 2023-11-15T03:09:57.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.062+0000 7f3b32342740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a4b110 0x1a7b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.062+0000 7f3b32342740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a94810 0x1a4b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.062+0000 7f3b32342740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a98810 0x1a924f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.062+0000 7f3b32342740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a7b740 con 0x1a94810 2023-11-15T03:09:57.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.062+0000 7f3b32342740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a7e4b0 con 0x1a98810 2023-11-15T03:09:57.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.062+0000 7f3b32342740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a7bc50 con 0x1a4b110 2023-11-15T03:09:57.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.062+0000 7f3b32342740 10 client.0 ll_register_callbacks cb 0x19c96b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:09:57.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.062+0000 7f3b25504700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a94810 0x1a4b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.062+0000 7f3b25504700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a94810 0x1a4b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40132/0 (socket says 172.21.15.121:40132) 2023-11-15T03:09:57.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.062+0000 7f3b25504700 1 -- 172.21.15.121:0/2917554223 learned_addr learned my addr 172.21.15.121:0/2917554223 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:57.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3b25d05700 1 --2- 172.21.15.121:0/2917554223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a4b110 0x1a7b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3b26506700 1 --2- 172.21.15.121:0/2917554223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a98810 0x1a924f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3b25504700 1 -- 172.21.15.121:0/2917554223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a4b110 msgr2=0x1a7b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:57.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3b25504700 1 --2- 172.21.15.121:0/2917554223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a4b110 0x1a7b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3b25504700 1 -- 172.21.15.121:0/2917554223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a98810 msgr2=0x1a924f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:57.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3b25504700 1 --2- 172.21.15.121:0/2917554223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a98810 0x1a924f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3b25504700 1 -- 172.21.15.121:0/2917554223 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b18009d60 con 0x1a94810 2023-11-15T03:09:57.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3b25504700 1 --2- 172.21.15.121:0/2917554223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a94810 0x1a4b930 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 rx=0x7f3b10071ff0 tx=0x7f3b1001d040).ready entity=mon.0 client_cookie=364b77d95025ac5d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:57.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3afffff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:09:57.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3afffff700 1 -- 172.21.15.121:0/2917554223 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1004a410 con 0x1a94810 2023-11-15T03:09:57.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3afffff700 1 -- 172.21.15.121:0/2917554223 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 9104+0+0 (secure 0 0 0) 0x7f3b1004a570 con 0x1a94810 2023-11-15T03:09:57.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3b32342740 1 -- 172.21.15.121:0/2917554223 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x19c92a0 con 0x1a94810 2023-11-15T03:09:57.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3afffff700 1 -- 172.21.15.121:0/2917554223 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=297) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b10050560 con 0x1a94810 2023-11-15T03:09:57.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3b32342740 10 client.10101 fetch_fsmap learned FSMap version 297 2023-11-15T03:09:57.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3b32342740 1 -- 172.21.15.121:0/2917554223 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=297,monmap=2+}) v3 -- 0x19c7a60 con 0x1a94810 2023-11-15T03:09:57.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3afffff700 1 -- 172.21.15.121:0/2917554223 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x19c92a0 con 0x1a94810 2023-11-15T03:09:57.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3afffff700 1 -- 172.21.15.121:0/2917554223 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 297) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3b10050070 con 0x1a94810 2023-11-15T03:09:57.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3b32342740 10 client.10101 fetch_fsmap finished waiting for FSMap version 297 2023-11-15T03:09:57.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3b32342740 10 client.10101 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:09:57.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3b32342740 20 client.10101 populate_metadata read hostname 'smithi121' 2023-11-15T03:09:57.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3b32342740 1 --2- 172.21.15.121:0/2917554223 >> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] conn(0x1aa89a0 0x1a7d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3b32342740 4 client.10101 mds_command: new command op to 9990 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:09:57.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3b32342740 1 -- 172.21.15.121:0/2917554223 --> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x19bf7f0 con 0x1aa89a0 2023-11-15T03:09:57.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3b25d05700 1 --2- 172.21.15.121:0/2917554223 >> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] conn(0x1aa89a0 0x1a7d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3b25d05700 1 --2- 172.21.15.121:0/2917554223 >> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] conn(0x1aa89a0 0x1a7d1b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=90dabae44cdf53ae server_cookie=eaa69f28179f3c64 in_seq=0 out_seq=0 2023-11-15T03:09:57.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.066+0000 7f3afffff700 10 client.10101 ms_handle_connect on v2:172.21.15.121:6834/1175031592 2023-11-15T03:09:57.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.070+0000 7f3afffff700 1 -- 172.21.15.121:0/2917554223 <== mds.0 v2:172.21.15.121:6834/1175031592 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x19bf7f0 con 0x1aa89a0 2023-11-15T03:09:57.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.070+0000 7f3afffff700 10 client.10101 handle_command_reply: tid=0 2023-11-15T03:09:57.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.070+0000 7f3b32342740 1 client.10101 shutdown 2023-11-15T03:09:57.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.070+0000 7f3b32342740 1 -- 172.21.15.121:0/2917554223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a94810 msgr2=0x1a4b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:57.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.070+0000 7f3b32342740 1 --2- 172.21.15.121:0/2917554223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a94810 0x1a4b930 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 rx=0x7f3b10071ff0 tx=0x7f3b1001d040).stop 2023-11-15T03:09:57.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.070+0000 7f3b32342740 1 -- 172.21.15.121:0/2917554223 shutdown_connections 2023-11-15T03:09:57.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.070+0000 7f3b32342740 1 --2- 172.21.15.121:0/2917554223 >> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] conn(0x1aa89a0 0x1a7d1b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.070+0000 7f3b32342740 1 --2- 172.21.15.121:0/2917554223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a98810 0x1a924f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.070+0000 7f3b32342740 1 --2- 172.21.15.121:0/2917554223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a94810 0x1a4b930 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.070+0000 7f3afffff700 0 client.10101 ms_handle_reset on v2:172.21.15.121:6834/1175031592 2023-11-15T03:09:57.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.070+0000 7f3b32342740 1 --2- 172.21.15.121:0/2917554223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a4b110 0x1a7b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.070+0000 7f3b32342740 1 -- 172.21.15.121:0/2917554223 >> 172.21.15.121:0/2917554223 conn(0x1831790 msgr2=0x1a77450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:57.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.070+0000 7f3b32342740 1 -- 172.21.15.121:0/2917554223 shutdown_connections 2023-11-15T03:09:57.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.070+0000 7f3b32342740 1 -- 172.21.15.121:0/2917554223 wait complete. 2023-11-15T03:09:57.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.070+0000 7f3b32342740 20 client.10101 trim_cache size 0 max 16384 2023-11-15T03:09:57.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b32342740 1 Processor -- start 2023-11-15T03:09:57.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b32342740 1 -- start start 2023-11-15T03:09:57.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b32342740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a4b110 0x1a4f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b32342740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a94810 0x1a78c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b32342740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a98810 0x1a74e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b32342740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a95370 con 0x1a94810 2023-11-15T03:09:57.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b32342740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1986560 con 0x1a98810 2023-11-15T03:09:57.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b32342740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a94bf0 con 0x1a4b110 2023-11-15T03:09:57.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b25504700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a94810 0x1a78c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b25504700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a94810 0x1a78c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40148/0 (socket says 172.21.15.121:40148) 2023-11-15T03:09:57.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b25504700 1 -- 172.21.15.121:0/2276859942 learned_addr learned my addr 172.21.15.121:0/2276859942 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:57.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b26506700 1 --2- 172.21.15.121:0/2276859942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a98810 0x1a74e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b25d05700 1 --2- 172.21.15.121:0/2276859942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a4b110 0x1a4f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b25504700 1 -- 172.21.15.121:0/2276859942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a4b110 msgr2=0x1a4f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:57.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b25504700 1 --2- 172.21.15.121:0/2276859942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a4b110 0x1a4f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b25504700 1 -- 172.21.15.121:0/2276859942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a98810 msgr2=0x1a74e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:57.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b25504700 1 --2- 172.21.15.121:0/2276859942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a98810 0x1a74e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b25504700 1 -- 172.21.15.121:0/2276859942 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1a7ebb0 con 0x1a94810 2023-11-15T03:09:57.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b25d05700 1 --2- 172.21.15.121:0/2276859942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a4b110 0x1a4f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:09:57.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.086+0000 7f3b26506700 1 --2- 172.21.15.121:0/2276859942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a98810 0x1a74e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:09:57.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.090+0000 7f3b25504700 1 --2- 172.21.15.121:0/2276859942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a94810 0x1a78c70 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 rx=0x7f3b1004ae00 tx=0x7f3b1005b000).ready entity=mon.0 client_cookie=b0081e3d89d4f808 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:57.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.090+0000 7f3b0b7fe700 1 -- 172.21.15.121:0/2276859942 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b100066a0 con 0x1a94810 2023-11-15T03:09:57.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.090+0000 7f3b0b7fe700 1 -- 172.21.15.121:0/2276859942 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b10050560 con 0x1a94810 2023-11-15T03:09:57.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.090+0000 7f3b0b7fe700 1 -- 172.21.15.121:0/2276859942 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1004a410 con 0x1a94810 2023-11-15T03:09:57.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.090+0000 7f3b32342740 1 -- 172.21.15.121:0/2276859942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a94810 msgr2=0x1a78c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:57.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.090+0000 7f3b32342740 1 --2- 172.21.15.121:0/2276859942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a94810 0x1a78c70 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 rx=0x7f3b1004ae00 tx=0x7f3b1005b000).stop 2023-11-15T03:09:57.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.090+0000 7f3b32342740 1 -- 172.21.15.121:0/2276859942 shutdown_connections 2023-11-15T03:09:57.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.090+0000 7f3b32342740 1 --2- 172.21.15.121:0/2276859942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a98810 0x1a74e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.090+0000 7f3b32342740 1 --2- 172.21.15.121:0/2276859942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a94810 0x1a78c70 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.090+0000 7f3b32342740 1 --2- 172.21.15.121:0/2276859942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a4b110 0x1a4f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.090+0000 7f3b32342740 1 -- 172.21.15.121:0/2276859942 >> 172.21.15.121:0/2276859942 conn(0x1831790 msgr2=0x1a77450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:57.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.090+0000 7f3b32342740 1 -- 172.21.15.121:0/2276859942 shutdown_connections 2023-11-15T03:09:57.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.090+0000 7f3b32342740 1 -- 172.21.15.121:0/2276859942 wait complete. 2023-11-15T03:09:57.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.090+0000 7f3b32342740 1 -- start start 2023-11-15T03:09:57.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.090+0000 7f3b32342740 1 Processor -- start 2023-11-15T03:09:57.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.094+0000 7f3b32342740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a4b110 0x1afcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.094+0000 7f3b32342740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a94810 0x1afd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.094+0000 7f3b32342740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a98810 0x1a91220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.094+0000 7f3b32342740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1afd580 con 0x1a98810 2023-11-15T03:09:57.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.094+0000 7f3b32342740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a98630 con 0x1a4b110 2023-11-15T03:09:57.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.094+0000 7f3b32342740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b04cb0 con 0x1a94810 2023-11-15T03:09:57.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.094+0000 7f3b32342740 10 client.0 ll_register_callbacks cb 0x7f3b1004ad50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:09:57.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.094+0000 7f3b26506700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a98810 0x1a91220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.094+0000 7f3b25d05700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a4b110 0x1afcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.094+0000 7f3b26506700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a98810 0x1a91220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40154/0 (socket says 172.21.15.121:40154) 2023-11-15T03:09:57.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.094+0000 7f3b26506700 1 -- 172.21.15.121:0/1623961758 learned_addr learned my addr 172.21.15.121:0/1623961758 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:57.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.094+0000 7f3b25504700 1 --2- 172.21.15.121:0/1623961758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a94810 0x1afd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.094+0000 7f3b25d05700 1 -- 172.21.15.121:0/1623961758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a94810 msgr2=0x1afd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:57.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.094+0000 7f3b25d05700 1 --2- 172.21.15.121:0/1623961758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a94810 0x1afd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.094+0000 7f3b25d05700 1 -- 172.21.15.121:0/1623961758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a98810 msgr2=0x1a91220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:57.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.094+0000 7f3b25d05700 1 --2- 172.21.15.121:0/1623961758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a98810 0x1a91220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.094+0000 7f3b25d05700 1 -- 172.21.15.121:0/1623961758 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x19c7a60 con 0x1a4b110 2023-11-15T03:09:57.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.094+0000 7f3b26506700 1 --2- 172.21.15.121:0/1623961758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a98810 0x1a91220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:09:57.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b25d05700 1 --2- 172.21.15.121:0/1623961758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a4b110 0x1afcba0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7f3b200cb9c0 tx=0x7f3b1c003040).ready entity=mon.1 client_cookie=58bacea73dc0d488 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:57.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3afffff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-11-15T03:09:57.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3afffff700 1 -- 172.21.15.121:0/1623961758 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c02b040 con 0x1a4b110 2023-11-15T03:09:57.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b32342740 1 -- 172.21.15.121:0/1623961758 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b1800f030 con 0x1a4b110 2023-11-15T03:09:57.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3afffff700 1 -- 172.21.15.121:0/1623961758 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 9104+0+0 (secure 0 0 0) 0x7f3b1c030480 con 0x1a4b110 2023-11-15T03:09:57.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3afffff700 1 -- 172.21.15.121:0/1623961758 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=297) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b1c02f280 con 0x1a4b110 2023-11-15T03:09:57.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b32342740 10 client.9727 fetch_fsmap learned FSMap version 297 2023-11-15T03:09:57.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b32342740 1 -- 172.21.15.121:0/1623961758 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=297,monmap=2+}) v3 -- 0x7f3b200c3cc0 con 0x1a4b110 2023-11-15T03:09:57.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3afffff700 1 -- 172.21.15.121:0/1623961758 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1800f030 con 0x1a4b110 2023-11-15T03:09:57.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3afffff700 1 -- 172.21.15.121:0/1623961758 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 297) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3b1c011440 con 0x1a4b110 2023-11-15T03:09:57.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b32342740 10 client.9727 fetch_fsmap finished waiting for FSMap version 297 2023-11-15T03:09:57.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b32342740 10 client.9727 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:09:57.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b32342740 20 client.9727 populate_metadata read hostname 'smithi121' 2023-11-15T03:09:57.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b32342740 1 --2- 172.21.15.121:0/1623961758 >> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] conn(0x1a91920 0x1a91d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b32342740 4 client.9727 mds_command: new command op to 9990 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-11-15T03:09:57.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b32342740 1 -- 172.21.15.121:0/1623961758 --> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x1888720 con 0x1a91920 2023-11-15T03:09:57.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b25504700 1 --2- 172.21.15.121:0/1623961758 >> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] conn(0x1a91920 0x1a91d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b25504700 1 --2- 172.21.15.121:0/1623961758 >> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] conn(0x1a91920 0x1a91d00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=362f9e0b87a44cb4 server_cookie=e503ae03151f92ea in_seq=0 out_seq=0 2023-11-15T03:09:57.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3afffff700 10 client.9727 ms_handle_connect on v2:172.21.15.121:6834/1175031592 2023-11-15T03:09:57.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3afffff700 1 -- 172.21.15.121:0/1623961758 <== mds.0 v2:172.21.15.121:6834/1175031592 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+194 (crc 0 0 0) 0x1888720 con 0x1a91920 2023-11-15T03:09:57.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3afffff700 10 client.9727 handle_command_reply: tid=0 2023-11-15T03:09:57.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b32342740 1 client.9727 shutdown 2023-11-15T03:09:57.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b32342740 1 -- 172.21.15.121:0/1623961758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a4b110 msgr2=0x1afcba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:57.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b32342740 1 --2- 172.21.15.121:0/1623961758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a4b110 0x1afcba0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7f3b200cb9c0 tx=0x7f3b1c003040).stop 2023-11-15T03:09:57.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b32342740 1 -- 172.21.15.121:0/1623961758 shutdown_connections 2023-11-15T03:09:57.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b32342740 1 --2- 172.21.15.121:0/1623961758 >> [v2:172.21.15.121:6834/1175031592,v1:172.21.15.121:6836/1175031592] conn(0x1a91920 0x1a91d00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b32342740 1 --2- 172.21.15.121:0/1623961758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1a98810 0x1a91220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3afffff700 0 client.9727 ms_handle_reset on v2:172.21.15.121:6834/1175031592 2023-11-15T03:09:57.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b32342740 1 --2- 172.21.15.121:0/1623961758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1a94810 0x1afd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.098+0000 7f3b32342740 1 --2- 172.21.15.121:0/1623961758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1a4b110 0x1afcba0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.102+0000 7f3b32342740 1 -- 172.21.15.121:0/1623961758 >> 172.21.15.121:0/1623961758 conn(0x1831790 msgr2=0x1a78b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:57.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.102+0000 7f3b32342740 1 -- 172.21.15.121:0/1623961758 shutdown_connections 2023-11-15T03:09:57.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.102+0000 7f3b32342740 1 -- 172.21.15.121:0/1623961758 wait complete. 2023-11-15T03:09:57.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.102+0000 7f3b32342740 20 client.9727 trim_cache size 0 max 16384 2023-11-15T03:09:57.122 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:09:57.123 INFO:teuthology.orchestra.run.smithi121.stdout:[ 2023-11-15T03:09:57.123 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:09:57.123 INFO:teuthology.orchestra.run.smithi121.stdout: "damage_type": "dentry", 2023-11-15T03:09:57.123 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 3525816854, 2023-11-15T03:09:57.124 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 1, 2023-11-15T03:09:57.124 INFO:teuthology.orchestra.run.smithi121.stdout: "frag": "*", 2023-11-15T03:09:57.124 INFO:teuthology.orchestra.run.smithi121.stdout: "dname": "subdir", 2023-11-15T03:09:57.124 INFO:teuthology.orchestra.run.smithi121.stdout: "snap_id": "head", 2023-11-15T03:09:57.124 INFO:teuthology.orchestra.run.smithi121.stdout: "path": "/subdir" 2023-11-15T03:09:57.125 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:09:57.125 INFO:teuthology.orchestra.run.smithi121.stdout:] 2023-11-15T03:09:57.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.102+0000 7f3b26d07700 1 -- 172.21.15.121:0/3605825935 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3b0405a510 msgr2=0x7f3b0405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:57.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.102+0000 7f3b26d07700 1 --2- 172.21.15.121:0/3605825935 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3b0405a510 0x7f3b0405c9d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f3b1c00b3c0 tx=0x7f3b1c016000).stop 2023-11-15T03:09:57.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.102+0000 7f3b26d07700 1 -- 172.21.15.121:0/3605825935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b200acd00 msgr2=0x7f3b200c9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:57.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.102+0000 7f3b26d07700 1 --2- 172.21.15.121:0/3605825935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b200acd00 0x7f3b200c9970 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 rx=0x7f3b1000c440 tx=0x7f3b10007e00).stop 2023-11-15T03:09:57.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.102+0000 7f3b26d07700 1 -- 172.21.15.121:0/3605825935 shutdown_connections 2023-11-15T03:09:57.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.102+0000 7f3b26d07700 1 --2- 172.21.15.121:0/3605825935 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3b0405a510 0x7f3b0405c9d0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.102+0000 7f3b26d07700 1 --2- 172.21.15.121:0/3605825935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b200c5c70 0x7f3b200c3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.102+0000 7f3b26d07700 1 --2- 172.21.15.121:0/3605825935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b200acd00 0x7f3b200c9970 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.102+0000 7f3b26d07700 1 --2- 172.21.15.121:0/3605825935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b20066330 0x7f3b20153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.102+0000 7f3b26d07700 1 -- 172.21.15.121:0/3605825935 >> 172.21.15.121:0/3605825935 conn(0x7f3b20063fc0 msgr2=0x7f3b200b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:57.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.102+0000 7f3b26d07700 1 -- 172.21.15.121:0/3605825935 shutdown_connections 2023-11-15T03:09:57.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.102+0000 7f3b26d07700 1 -- 172.21.15.121:0/3605825935 wait complete. 2023-11-15T03:09:57.132 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 10000000000.00000000' 2023-11-15T03:09:57.133 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T03:09:57.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:09:57.134 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-15T03:09:57.173 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:09:57.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:09:57.174 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:09:57.233 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:09:57.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.434+0000 7f8e15ce4700 1 Processor -- start 2023-11-15T03:09:57.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.434+0000 7f8e15ce4700 1 -- start start 2023-11-15T03:09:57.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.434+0000 7f8e15ce4700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8e100abaa0 0x7f8e100abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.434+0000 7f8e15ce4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e100bb510 0x7f8e10063570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.434+0000 7f8e15ce4700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e100c2c00 con 0x7f8e100a8880 2023-11-15T03:09:57.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.434+0000 7f8e15ce4700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e10063a60 con 0x7f8e100abaa0 2023-11-15T03:09:57.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.434+0000 7f8e15ce4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e100657b0 con 0x7f8e100bb510 2023-11-15T03:09:57.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.434+0000 7f8e154e3700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f8e100a8880 0x7f8e10151400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:46380/0 (socket says 172.21.15.121:46380) 2023-11-15T03:09:57.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.434+0000 7f8e154e3700 1 -- 172.21.15.121:0/847133022 learned_addr learned my addr 172.21.15.121:0/847133022 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:09:57.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.434+0000 7f8e14ce2700 1 --2- 172.21.15.121:0/847133022 >> v2:172.21.15.121:3300/0 conn(0x7f8e100abaa0 0x7f8e100abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.434+0000 7f8e0ffff700 1 --2- 172.21.15.121:0/847133022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e100bb510 0x7f8e10063570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.438+0000 7f8e0f7fe700 1 -- 172.21.15.121:0/847133022 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2555706507 0 0) 0x7f8e100c2c00 con 0x7f8e100a8880 2023-11-15T03:09:57.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.438+0000 7f8e0f7fe700 1 -- 172.21.15.121:0/847133022 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e000033f0 con 0x7f8e100a8880 2023-11-15T03:09:57.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.438+0000 7f8e14ce2700 1 -- 172.21.15.121:0/847133022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e100bb510 msgr2=0x7f8e10063570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:57.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.438+0000 7f8e14ce2700 1 --2- 172.21.15.121:0/847133022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e100bb510 0x7f8e10063570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.438+0000 7f8e14ce2700 1 -- 172.21.15.121:0/847133022 >> v1:172.21.15.121:6789/0 conn(0x7f8e100a8880 legacy=0x7f8e10151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:57.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.438+0000 7f8e14ce2700 1 -- 172.21.15.121:0/847133022 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e10065a90 con 0x7f8e100abaa0 2023-11-15T03:09:57.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.438+0000 7f8e0ffff700 1 --2- 172.21.15.121:0/847133022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e100bb510 0x7f8e10063570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:09:57.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.438+0000 7f8e14ce2700 1 --2- 172.21.15.121:0/847133022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e100abaa0 0x7f8e100abe80 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 rx=0x7f8e04009760 tx=0x7f8e04009060).ready entity=mon.0 client_cookie=6cc2b7b8144dfafe server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:57.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.438+0000 7f8e0f7fe700 1 -- 172.21.15.121:0/847133022 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e040090d0 con 0x7f8e100abaa0 2023-11-15T03:09:57.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.438+0000 7f8e0f7fe700 1 -- 172.21.15.121:0/847133022 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0400d4a0 con 0x7f8e100abaa0 2023-11-15T03:09:57.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.438+0000 7f8e0f7fe700 1 -- 172.21.15.121:0/847133022 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0400d820 con 0x7f8e100abaa0 2023-11-15T03:09:57.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.438+0000 7f8e15ce4700 1 -- 172.21.15.121:0/847133022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e100abaa0 msgr2=0x7f8e100abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:57.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.438+0000 7f8e15ce4700 1 --2- 172.21.15.121:0/847133022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e100abaa0 0x7f8e100abe80 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 rx=0x7f8e04009760 tx=0x7f8e04009060).stop 2023-11-15T03:09:57.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.438+0000 7f8e15ce4700 1 -- 172.21.15.121:0/847133022 shutdown_connections 2023-11-15T03:09:57.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.438+0000 7f8e15ce4700 1 --2- 172.21.15.121:0/847133022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e100bb510 0x7f8e10063570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.438+0000 7f8e15ce4700 1 --2- 172.21.15.121:0/847133022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e100abaa0 0x7f8e100abe80 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.438+0000 7f8e15ce4700 1 -- 172.21.15.121:0/847133022 >> 172.21.15.121:0/847133022 conn(0x7f8e10067a60 msgr2=0x7f8e10067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:57.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.438+0000 7f8e15ce4700 1 -- 172.21.15.121:0/847133022 shutdown_connections 2023-11-15T03:09:57.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.438+0000 7f8e15ce4700 1 -- 172.21.15.121:0/847133022 wait complete. 2023-11-15T03:09:57.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e15ce4700 1 Processor -- start 2023-11-15T03:09:57.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e15ce4700 1 -- start start 2023-11-15T03:09:57.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e15ce4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e100a8880 0x7f8e100c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e15ce4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e100bb510 0x7f8e100be0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e15ce4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e100be5c0 0x7f8e10161d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e15ce4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e100bf480 con 0x7f8e100a8880 2023-11-15T03:09:57.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e15ce4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e100c4390 con 0x7f8e100be5c0 2023-11-15T03:09:57.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e15ce4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e100bebe0 con 0x7f8e100bb510 2023-11-15T03:09:57.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e14ce2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e100a8880 0x7f8e100c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e14ce2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e100a8880 0x7f8e100c6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40168/0 (socket says 172.21.15.121:40168) 2023-11-15T03:09:57.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e14ce2700 1 -- 172.21.15.121:0/1639702993 learned_addr learned my addr 172.21.15.121:0/1639702993 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:57.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e0ffff700 1 --2- 172.21.15.121:0/1639702993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e100bb510 0x7f8e100be0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e154e3700 1 --2- 172.21.15.121:0/1639702993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e100be5c0 0x7f8e10161d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e14ce2700 1 -- 172.21.15.121:0/1639702993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e100bb510 msgr2=0x7f8e100be0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:57.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e14ce2700 1 --2- 172.21.15.121:0/1639702993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e100bb510 0x7f8e100be0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e14ce2700 1 -- 172.21.15.121:0/1639702993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e100be5c0 msgr2=0x7f8e10161d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:57.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e14ce2700 1 --2- 172.21.15.121:0/1639702993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e100be5c0 0x7f8e10161d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:57.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e14ce2700 1 -- 172.21.15.121:0/1639702993 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dfc009250 con 0x7f8e100a8880 2023-11-15T03:09:57.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e154e3700 1 --2- 172.21.15.121:0/1639702993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e100be5c0 0x7f8e10161d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:57.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.442+0000 7f8e14ce2700 1 --2- 172.21.15.121:0/1639702993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e100a8880 0x7f8e100c6d60 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 rx=0x7f8e10064eb0 tx=0x7f8e04016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:57.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.446+0000 7f8e0cff9700 1 -- 172.21.15.121:0/1639702993 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04006990 con 0x7f8e100a8880 2023-11-15T03:09:57.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.446+0000 7f8e0cff9700 1 -- 172.21.15.121:0/1639702993 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e04012900 con 0x7f8e100a8880 2023-11-15T03:09:57.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.446+0000 7f8e15ce4700 1 -- 172.21.15.121:0/1639702993 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e040092a0 con 0x7f8e100a8880 2023-11-15T03:09:57.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.446+0000 7f8e0cff9700 1 -- 172.21.15.121:0/1639702993 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04012c80 con 0x7f8e100a8880 2023-11-15T03:09:57.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.446+0000 7f8e15ce4700 1 -- 172.21.15.121:0/1639702993 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e100bf1a0 con 0x7f8e100a8880 2023-11-15T03:09:57.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.446+0000 7f8e15ce4700 1 -- 172.21.15.121:0/1639702993 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e100be9a0 con 0x7f8e100a8880 2023-11-15T03:09:57.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.446+0000 7f8e0cff9700 1 -- 172.21.15.121:0/1639702993 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8e0401c630 con 0x7f8e100a8880 2023-11-15T03:09:57.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.450+0000 7f8e0cff9700 1 --2- 172.21.15.121:0/1639702993 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8df405a510 0x7f8df405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:57.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.450+0000 7f8e0cff9700 1 -- 172.21.15.121:0/1639702993 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9104+0+0 (secure 0 0 0) 0x7f8e040767f0 con 0x7f8e100a8880 2023-11-15T03:09:57.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.450+0000 7f8e0ffff700 1 --2- 172.21.15.121:0/1639702993 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8df405a510 0x7f8df405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:57.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.450+0000 7f8e0ffff700 1 --2- 172.21.15.121:0/1639702993 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8df405a510 0x7f8df405c9d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f8e100c0550 tx=0x7f8dfc008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:57.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.454+0000 7f8e0cff9700 1 -- 172.21.15.121:0/1639702993 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8e100be9a0 con 0x7f8e100a8880 2023-11-15T03:09:57.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:57.658+0000 7f8e15ce4700 1 -- 172.21.15.121:0/1639702993 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8e100be9a0 con 0x7f8e100a8880 2023-11-15T03:09:58.290 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:09:58.278+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:09:58.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.294+0000 7f8e0cff9700 1 -- 172.21.15.121:0/1639702993 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v298) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8e100be9a0 con 0x7f8e100a8880 2023-11-15T03:09:58.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.294+0000 7f8e15ce4700 1 -- 172.21.15.121:0/1639702993 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8df405a510 msgr2=0x7f8df405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:58.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.298+0000 7f8e15ce4700 1 --2- 172.21.15.121:0/1639702993 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8df405a510 0x7f8df405c9d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f8e100c0550 tx=0x7f8dfc008040).stop 2023-11-15T03:09:58.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.298+0000 7f8e15ce4700 1 -- 172.21.15.121:0/1639702993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e100a8880 msgr2=0x7f8e100c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:58.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.298+0000 7f8e15ce4700 1 --2- 172.21.15.121:0/1639702993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e100a8880 0x7f8e100c6d60 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 rx=0x7f8e10064eb0 tx=0x7f8e04016000).stop 2023-11-15T03:09:58.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.298+0000 7f8e15ce4700 1 -- 172.21.15.121:0/1639702993 shutdown_connections 2023-11-15T03:09:58.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.298+0000 7f8e15ce4700 1 --2- 172.21.15.121:0/1639702993 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8df405a510 0x7f8df405c9d0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:58.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.298+0000 7f8e15ce4700 1 --2- 172.21.15.121:0/1639702993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e100be5c0 0x7f8e10161d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:58.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.298+0000 7f8e15ce4700 1 --2- 172.21.15.121:0/1639702993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e100bb510 0x7f8e100be0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:58.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.298+0000 7f8e15ce4700 1 --2- 172.21.15.121:0/1639702993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e100a8880 0x7f8e100c6d60 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:58.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.298+0000 7f8e15ce4700 1 -- 172.21.15.121:0/1639702993 >> 172.21.15.121:0/1639702993 conn(0x7f8e10067a60 msgr2=0x7f8e1014ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:58.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.298+0000 7f8e15ce4700 1 -- 172.21.15.121:0/1639702993 shutdown_connections 2023-11-15T03:09:58.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.298+0000 7f8e15ce4700 1 -- 172.21.15.121:0/1639702993 wait complete. 2023-11-15T03:09:58.310 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:09:58.326 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:09:58.350 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:09:58.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.474+0000 7fea1359e700 1 Processor -- start 2023-11-15T03:09:58.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.474+0000 7fea1359e700 1 -- start start 2023-11-15T03:09:58.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.478+0000 7fea1359e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fea14153e40 0x7fea140c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:58.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.478+0000 7fea1359e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea14154270 con 0x7fea14153a60 2023-11-15T03:09:58.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.478+0000 7fea1359e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea140660a0 con 0x7fea140bfe40 2023-11-15T03:09:58.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.478+0000 7fea1359e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea140c5500 con 0x7fea14153e40 2023-11-15T03:09:58.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.478+0000 7fea1259c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fea14153a60 0x7fea14066530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:46390/0 (socket says 172.21.15.121:46390) 2023-11-15T03:09:58.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.478+0000 7fea1259c700 1 -- 172.21.15.121:0/3201972279 learned_addr learned my addr 172.21.15.121:0/3201972279 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:09:58.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.478+0000 7fea11d9b700 1 --2- 172.21.15.121:0/3201972279 >> v2:172.21.15.121:3300/0 conn(0x7fea14153e40 0x7fea140c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:58.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.478+0000 7fea1159a700 1 -- 172.21.15.121:0/3201972279 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341134263 0 0) 0x7fea04001110 con 0x7fea14153a60 2023-11-15T03:09:58.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.478+0000 7fea1159a700 1 -- 172.21.15.121:0/3201972279 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9fc0033f0 con 0x7fea14153a60 2023-11-15T03:09:58.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.478+0000 7fea1159a700 1 -- 172.21.15.121:0/3201972279 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3206868296 0 0) 0x7fea140660a0 con 0x7fea140bfe40 2023-11-15T03:09:58.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.478+0000 7fea11d9b700 1 -- 172.21.15.121:0/3201972279 >> v1:172.21.15.190:6789/0 conn(0x7fea140bfe40 legacy=0x7fea140c0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:58.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.478+0000 7fea11d9b700 1 -- 172.21.15.121:0/3201972279 >> v1:172.21.15.121:6789/0 conn(0x7fea14153a60 legacy=0x7fea14066530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:58.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.478+0000 7fea11d9b700 1 -- 172.21.15.121:0/3201972279 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea14066c40 con 0x7fea14153e40 2023-11-15T03:09:58.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.478+0000 7fea11d9b700 1 --2- 172.21.15.121:0/3201972279 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea14153e40 0x7fea140c5d10 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 rx=0x7fea08009790 tx=0x7fea08009060).ready entity=mon.0 client_cookie=43603b5eb833d086 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:58.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.478+0000 7fea1159a700 1 -- 172.21.15.121:0/3201972279 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04001600 con 0x7fea14153e40 2023-11-15T03:09:58.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.478+0000 7fea1159a700 1 -- 172.21.15.121:0/3201972279 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea08012410 con 0x7fea14153e40 2023-11-15T03:09:58.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.478+0000 7fea1159a700 1 -- 172.21.15.121:0/3201972279 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08012790 con 0x7fea14153e40 2023-11-15T03:09:58.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.478+0000 7fea1359e700 1 -- 172.21.15.121:0/3201972279 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea14153e40 msgr2=0x7fea140c5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:58.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.478+0000 7fea1359e700 1 --2- 172.21.15.121:0/3201972279 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea14153e40 0x7fea140c5d10 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 rx=0x7fea08009790 tx=0x7fea08009060).stop 2023-11-15T03:09:58.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.482+0000 7fea1359e700 1 -- 172.21.15.121:0/3201972279 shutdown_connections 2023-11-15T03:09:58.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.482+0000 7fea1359e700 1 --2- 172.21.15.121:0/3201972279 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea14153e40 0x7fea140c5d10 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:58.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.482+0000 7fea1359e700 1 -- 172.21.15.121:0/3201972279 >> 172.21.15.121:0/3201972279 conn(0x7fea14067a60 msgr2=0x7fea14067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:58.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.482+0000 7fea1359e700 1 -- 172.21.15.121:0/3201972279 shutdown_connections 2023-11-15T03:09:58.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.482+0000 7fea1359e700 1 -- 172.21.15.121:0/3201972279 wait complete. 2023-11-15T03:09:58.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.482+0000 7fea1359e700 1 Processor -- start 2023-11-15T03:09:58.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.482+0000 7fea1359e700 1 -- start start 2023-11-15T03:09:58.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.482+0000 7fea1359e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea140bfe40 0x7fea140b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:58.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.482+0000 7fea1359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea14153a60 0x7fea14167e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:58.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.482+0000 7fea1359e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea14153e40 0x7fea140ad960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:58.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.482+0000 7fea1359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea140ade50 con 0x7fea14153a60 2023-11-15T03:09:58.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.482+0000 7fea1359e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea14149bb0 con 0x7fea140bfe40 2023-11-15T03:09:58.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.482+0000 7fea1359e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea140afd10 con 0x7fea14153e40 2023-11-15T03:09:58.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.482+0000 7fea11d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea14153a60 0x7fea14167e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:58.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.486+0000 7fea11d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea14153a60 0x7fea14167e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40204/0 (socket says 172.21.15.121:40204) 2023-11-15T03:09:58.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.486+0000 7fea11d9b700 1 -- 172.21.15.121:0/3148417110 learned_addr learned my addr 172.21.15.121:0/3148417110 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:58.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.486+0000 7fea1259c700 1 --2- 172.21.15.121:0/3148417110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea140bfe40 0x7fea140b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:58.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.486+0000 7fea12d9d700 1 --2- 172.21.15.121:0/3148417110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea14153e40 0x7fea140ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:58.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.486+0000 7fea11d9b700 1 -- 172.21.15.121:0/3148417110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea14153e40 msgr2=0x7fea140ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:58.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.486+0000 7fea11d9b700 1 --2- 172.21.15.121:0/3148417110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea14153e40 0x7fea140ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:58.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.486+0000 7fea11d9b700 1 -- 172.21.15.121:0/3148417110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea140bfe40 msgr2=0x7fea140b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:58.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.486+0000 7fea11d9b700 1 --2- 172.21.15.121:0/3148417110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea140bfe40 0x7fea140b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:58.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.486+0000 7fea11d9b700 1 -- 172.21.15.121:0/3148417110 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea080092a0 con 0x7fea14153a60 2023-11-15T03:09:58.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.486+0000 7fea12d9d700 1 --2- 172.21.15.121:0/3148417110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea14153e40 0x7fea140ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:09:58.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.486+0000 7fea11d9b700 1 --2- 172.21.15.121:0/3148417110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea14153a60 0x7fea14167e80 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 rx=0x7fea140648d0 tx=0x7fea08009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:58.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.486+0000 7fea027fc700 1 -- 172.21.15.121:0/3148417110 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea080119c0 con 0x7fea14153a60 2023-11-15T03:09:58.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.486+0000 7fea027fc700 1 -- 172.21.15.121:0/3148417110 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea08011b20 con 0x7fea14153a60 2023-11-15T03:09:58.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.486+0000 7fea1359e700 1 -- 172.21.15.121:0/3148417110 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1414b260 con 0x7fea14153a60 2023-11-15T03:09:58.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.486+0000 7fea027fc700 1 -- 172.21.15.121:0/3148417110 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08011ea0 con 0x7fea14153a60 2023-11-15T03:09:58.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.486+0000 7fea1359e700 1 -- 172.21.15.121:0/3148417110 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea140aff60 con 0x7fea14153a60 2023-11-15T03:09:58.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.486+0000 7fea1359e700 1 -- 172.21.15.121:0/3148417110 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea140687e0 con 0x7fea14153a60 2023-11-15T03:09:58.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.486+0000 7fea027fc700 1 -- 172.21.15.121:0/3148417110 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fea0801d650 con 0x7fea14153a60 2023-11-15T03:09:58.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.490+0000 7fea027fc700 1 --2- 172.21.15.121:0/3148417110 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe9f405a510 0x7fe9f405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:58.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.490+0000 7fea027fc700 1 -- 172.21.15.121:0/3148417110 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9190+0+0 (secure 0 0 0) 0x7fea080763d0 con 0x7fea14153a60 2023-11-15T03:09:58.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.490+0000 7fea1259c700 1 --2- 172.21.15.121:0/3148417110 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe9f405a510 0x7fe9f405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:58.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.490+0000 7fea1259c700 1 --2- 172.21.15.121:0/3148417110 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe9f405a510 0x7fe9f405c9d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7fea04000fb0 tx=0x7fea04001b60).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:58.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.494+0000 7fea027fc700 1 -- 172.21.15.121:0/3148417110 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fea140687e0 con 0x7fea14153a60 2023-11-15T03:09:58.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:58.718+0000 7fea1359e700 1 -- 172.21.15.121:0/3148417110 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fea1414b6f0 con 0x7fea14153a60 2023-11-15T03:09:59.307 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.298+0000 7fea027fc700 1 -- 172.21.15.121:0/3148417110 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v299) v1 ==== 107+0+0 (secure 0 0 0) 0x7fea1414b6f0 con 0x7fea14153a60 2023-11-15T03:09:59.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.302+0000 7fea1359e700 1 -- 172.21.15.121:0/3148417110 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe9f405a510 msgr2=0x7fe9f405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.302+0000 7fea1359e700 1 --2- 172.21.15.121:0/3148417110 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe9f405a510 0x7fe9f405c9d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7fea04000fb0 tx=0x7fea04001b60).stop 2023-11-15T03:09:59.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.302+0000 7fea1359e700 1 -- 172.21.15.121:0/3148417110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea14153a60 msgr2=0x7fea14167e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.302+0000 7fea1359e700 1 --2- 172.21.15.121:0/3148417110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea14153a60 0x7fea14167e80 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 rx=0x7fea140648d0 tx=0x7fea08009060).stop 2023-11-15T03:09:59.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.302+0000 7fea1359e700 1 -- 172.21.15.121:0/3148417110 shutdown_connections 2023-11-15T03:09:59.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.302+0000 7fea1359e700 1 --2- 172.21.15.121:0/3148417110 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe9f405a510 0x7fe9f405c9d0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.302+0000 7fea1359e700 1 --2- 172.21.15.121:0/3148417110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea14153e40 0x7fea140ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.302+0000 7fea1359e700 1 --2- 172.21.15.121:0/3148417110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea14153a60 0x7fea14167e80 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.302+0000 7fea1359e700 1 --2- 172.21.15.121:0/3148417110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea140bfe40 0x7fea140b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.302+0000 7fea1359e700 1 -- 172.21.15.121:0/3148417110 >> 172.21.15.121:0/3148417110 conn(0x7fea14067a60 msgr2=0x7fea140acf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:59.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.302+0000 7fea1359e700 1 -- 172.21.15.121:0/3148417110 shutdown_connections 2023-11-15T03:09:59.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.302+0000 7fea1359e700 1 -- 172.21.15.121:0/3148417110 wait complete. 2023-11-15T03:09:59.315 INFO:teuthology.orchestra.run.smithi121.stderr:nothing to do: rank is not damaged 2023-11-15T03:09:59.334 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:09:59.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.430+0000 7febbccdad00 1 Processor -- start 2023-11-15T03:09:59.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.430+0000 7febbccdad00 1 -- start start 2023-11-15T03:09:59.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.430+0000 7febbccdad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559403f14a30 0x559403fead90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.430+0000 7febbccdad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559403f63b20 0x559403ff1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.430+0000 7febbccdad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559403ff5000 0x559403f14050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.430+0000 7febbccdad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559403e8d220 con 0x559403ff5000 2023-11-15T03:09:59.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.430+0000 7febbccdad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559403f60050 con 0x559403f63b20 2023-11-15T03:09:59.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.430+0000 7febbccdad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x559403feb930 con 0x559403f14a30 2023-11-15T03:09:59.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.430+0000 7febbadbd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559403ff5000 0x559403f14050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.430+0000 7febbadbd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559403ff5000 0x559403f14050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40216/0 (socket says 172.21.15.121:40216) 2023-11-15T03:09:59.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.430+0000 7febbadbd700 1 -- 172.21.15.121:0/2496084573 learned_addr learned my addr 172.21.15.121:0/2496084573 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:59.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.430+0000 7febb1dbb700 1 --2- 172.21.15.121:0/2496084573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559403f63b20 0x559403ff1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.430+0000 7febba5bc700 1 --2- 172.21.15.121:0/2496084573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559403f14a30 0x559403fead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febbadbd700 1 -- 172.21.15.121:0/2496084573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559403f14a30 msgr2=0x559403fead90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:59.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febbadbd700 1 --2- 172.21.15.121:0/2496084573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559403f14a30 0x559403fead90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febbadbd700 1 -- 172.21.15.121:0/2496084573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559403f63b20 msgr2=0x559403ff1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:59.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febbadbd700 1 --2- 172.21.15.121:0/2496084573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559403f63b20 0x559403ff1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febbadbd700 1 -- 172.21.15.121:0/2496084573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac00a180 con 0x559403ff5000 2023-11-15T03:09:59.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febbadbd700 1 --2- 172.21.15.121:0/2496084573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559403ff5000 0x559403f14050 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 rx=0x7febb4001dd0 tx=0x7febb4001870).ready entity=mon.0 client_cookie=5928c9a624fbdfaf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:59.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febb95ba700 1 -- 172.21.15.121:0/2496084573 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb400bb40 con 0x559403ff5000 2023-11-15T03:09:59.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febb95ba700 1 -- 172.21.15.121:0/2496084573 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb4008070 con 0x559403ff5000 2023-11-15T03:09:59.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febb95ba700 1 -- 172.21.15.121:0/2496084573 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb400bca0 con 0x559403ff5000 2023-11-15T03:09:59.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febbccdad00 1 -- 172.21.15.121:0/2496084573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559403ff5000 msgr2=0x559403f14050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febbccdad00 1 --2- 172.21.15.121:0/2496084573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559403ff5000 0x559403f14050 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 rx=0x7febb4001dd0 tx=0x7febb4001870).stop 2023-11-15T03:09:59.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febbccdad00 1 -- 172.21.15.121:0/2496084573 shutdown_connections 2023-11-15T03:09:59.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febbccdad00 1 --2- 172.21.15.121:0/2496084573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559403ff5000 0x559403f14050 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febbccdad00 1 --2- 172.21.15.121:0/2496084573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559403f63b20 0x559403ff1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febbccdad00 1 --2- 172.21.15.121:0/2496084573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559403f14a30 0x559403fead90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febbccdad00 1 -- 172.21.15.121:0/2496084573 >> 172.21.15.121:0/2496084573 conn(0x559403f64080 msgr2=0x559403ff4180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:59.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febbccdad00 1 -- 172.21.15.121:0/2496084573 shutdown_connections 2023-11-15T03:09:59.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febbccdad00 1 -- 172.21.15.121:0/2496084573 wait complete. 2023-11-15T03:09:59.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febbccdad00 1 Processor -- start 2023-11-15T03:09:59.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.434+0000 7febbccdad00 1 -- start start 2023-11-15T03:09:59.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febbccdad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559403f14a30 0x559404002ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febbccdad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559403f63b20 0x559404001a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febbccdad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559403ff5000 0x559403f48d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febbccdad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559404002fc0 con 0x559403f14a30 2023-11-15T03:09:59.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febbccdad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559403fec0e0 con 0x559403ff5000 2023-11-15T03:09:59.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febbccdad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x559403fe9e80 con 0x559403f63b20 2023-11-15T03:09:59.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febba5bc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559403f14a30 0x559404002ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febba5bc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559403f14a30 0x559404002ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40220/0 (socket says 172.21.15.121:40220) 2023-11-15T03:09:59.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 learned_addr learned my addr 172.21.15.121:0/2206188074 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:59.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febb1dbb700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559403f63b20 0x559404001a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febbadbd700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559403ff5000 0x559403f48d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559403f63b20 msgr2=0x559404001a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febba5bc700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559403f63b20 0x559404001a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559403ff5000 msgr2=0x559403f48d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febba5bc700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559403ff5000 0x559403f48d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb400a040 con 0x559403f14a30 2023-11-15T03:09:59.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febb1dbb700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559403f63b20 0x559404001a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:59.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febbadbd700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559403ff5000 0x559403f48d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:09:59.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febba5bc700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559403f14a30 0x559404002ad0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 rx=0x7febac00a150 tx=0x7febac012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:59.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febb37fe700 1 -- 172.21.15.121:0/2206188074 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac00cd30 con 0x559403f14a30 2023-11-15T03:09:59.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febb37fe700 1 -- 172.21.15.121:0/2206188074 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febac00ce90 con 0x559403f14a30 2023-11-15T03:09:59.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febb37fe700 1 -- 172.21.15.121:0/2206188074 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac00c980 con 0x559403f14a30 2023-11-15T03:09:59.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559403eb6770 con 0x559403f14a30 2023-11-15T03:09:59.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.438+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559403e85a70 con 0x559403f14a30 2023-11-15T03:09:59.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.442+0000 7febb37fe700 1 -- 172.21.15.121:0/2206188074 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7febac01b040 con 0x559403f14a30 2023-11-15T03:09:59.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.442+0000 7febb37fe700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feb9c059020 0x7feb9c05b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.442+0000 7febb37fe700 1 -- 172.21.15.121:0/2206188074 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9190+0+0 (secure 0 0 0) 0x7febac076080 con 0x559403f14a30 2023-11-15T03:09:59.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.442+0000 7febb1dbb700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feb9c059020 0x7feb9c05b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.459 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:09:59.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.442+0000 7febbccdad00 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x559403ff30f0 0x55940400dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.442+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940400e0a0 con 0x559403ff30f0 2023-11-15T03:09:59.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.442+0000 7febb1dbb700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feb9c059020 0x7feb9c05b4e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x559403f15470 tx=0x7feba4005b10).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:59.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.442+0000 7febbadbd700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x559403ff30f0 0x55940400dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.446+0000 7febbadbd700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x559403ff30f0 0x55940400dbb0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:59.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.446+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v128'401 uv394 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x559403ff30f0 2023-11-15T03:09:59.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.446+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x55940400e450 con 0x559403ff30f0 2023-11-15T03:09:59.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.450+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v128'402 uv402 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x559403ff30f0 2023-11-15T03:09:59.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.450+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940400e800 con 0x559403ff30f0 2023-11-15T03:09:59.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.454+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v128'403 uv403 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x559403ff30f0 2023-11-15T03:09:59.463 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:09:59.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.454+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940400fbc0 con 0x559403ff30f0 2023-11-15T03:09:59.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.454+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v128'404 uv404 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x559403ff30f0 2023-11-15T03:09:59.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.454+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940400ff70 con 0x559403ff30f0 2023-11-15T03:09:59.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.458+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v128'405 uv405 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x559403ff30f0 2023-11-15T03:09:59.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.458+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940400ebb0 con 0x559403ff30f0 2023-11-15T03:09:59.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.458+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v128'406 uv406 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x559403ff30f0 2023-11-15T03:09:59.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.458+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940400ef60 con 0x559403ff30f0 2023-11-15T03:09:59.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.462+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v128'407 uv400 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x559403ff30f0 2023-11-15T03:09:59.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.462+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x55940400ef60 con 0x559403ff30f0 2023-11-15T03:09:59.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.466+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v128'408 uv408 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x559403ff30f0 2023-11-15T03:09:59.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.466+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940400ef60 con 0x559403ff30f0 2023-11-15T03:09:59.477 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:09:59.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.466+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v128'409 uv409 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x559403ff30f0 2023-11-15T03:09:59.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.466+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940400ef60 con 0x559403ff30f0 2023-11-15T03:09:59.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.470+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v128'410 uv410 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x559403ff30f0 2023-11-15T03:09:59.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.470+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940400ef60 con 0x559403ff30f0 2023-11-15T03:09:59.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.474+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v128'411 uv411 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x559403ff30f0 2023-11-15T03:09:59.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.474+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940400ef60 con 0x559403ff30f0 2023-11-15T03:09:59.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.474+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v128'412 uv412 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x559403ff30f0 2023-11-15T03:09:59.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.474+0000 7febbccdad00 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x559404010320 0x559404012710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.474+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940400ef60 con 0x559404010320 2023-11-15T03:09:59.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.478+0000 7febba5bc700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x559404010320 0x559404012710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.478+0000 7febba5bc700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x559404010320 0x559404012710 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:59.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.482+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v128'633 uv620 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.482+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x559404012c00 con 0x559404010320 2023-11-15T03:09:59.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.486+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v128'634 uv634 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.486+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404012fb0 con 0x559404010320 2023-11-15T03:09:59.499 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:09:59.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.490+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v128'635 uv635 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.490+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404014370 con 0x559404010320 2023-11-15T03:09:59.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.490+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v128'636 uv636 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.490+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404014720 con 0x559404010320 2023-11-15T03:09:59.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.494+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v128'637 uv637 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.494+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404013360 con 0x559404010320 2023-11-15T03:09:59.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.498+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v128'638 uv638 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.498+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404013c50 con 0x559404010320 2023-11-15T03:09:59.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.498+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v128'639 uv639 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.502+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404014ad0 con 0x559404010320 2023-11-15T03:09:59.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.502+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v128'640 uv626 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.502+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x559404014ad0 con 0x559404010320 2023-11-15T03:09:59.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.506+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v128'641 uv641 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.506+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404014ad0 con 0x559404010320 2023-11-15T03:09:59.518 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:09:59.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.510+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v128'642 uv642 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.510+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404015ae0 con 0x559404010320 2023-11-15T03:09:59.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.510+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v128'643 uv643 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.510+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404015ae0 con 0x559404010320 2023-11-15T03:09:59.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.514+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v128'644 uv644 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.514+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404014ad0 con 0x559404010320 2023-11-15T03:09:59.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.518+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v128'645 uv645 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.518+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404014ad0 con 0x559404010320 2023-11-15T03:09:59.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.518+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v128'646 uv632 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.518+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x559404014ad0 con 0x559404010320 2023-11-15T03:09:59.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.522+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v128'647 uv647 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.522+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404014ad0 con 0x559404010320 2023-11-15T03:09:59.536 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:09:59.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.526+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v128'648 uv648 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.526+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404015ae0 con 0x559404010320 2023-11-15T03:09:59.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.530+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v128'649 uv649 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.530+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404015ae0 con 0x559404010320 2023-11-15T03:09:59.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.534+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v128'650 uv650 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.534+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404014ad0 con 0x559404010320 2023-11-15T03:09:59.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.538+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v128'651 uv651 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.538+0000 7febbccdad00 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x559404014fe0 0x5594040173d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.538+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040178c0 con 0x559404014fe0 2023-11-15T03:09:59.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.538+0000 7febb1dbb700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x559404014fe0 0x5594040173d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.538+0000 7febb1dbb700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x559404014fe0 0x5594040173d0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:59.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.542+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v128'346 uv339 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404014fe0 2023-11-15T03:09:59.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.542+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x559404017c70 con 0x559404014fe0 2023-11-15T03:09:59.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.546+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v128'347 uv347 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404014fe0 2023-11-15T03:09:59.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.546+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404018020 con 0x559404014fe0 2023-11-15T03:09:59.560 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:09:59.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.550+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v128'348 uv348 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404014fe0 2023-11-15T03:09:59.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.550+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040188c0 con 0x559404014fe0 2023-11-15T03:09:59.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.554+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v128'349 uv349 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404014fe0 2023-11-15T03:09:59.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.554+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404018c70 con 0x559404014fe0 2023-11-15T03:09:59.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.558+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v128'350 uv350 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404014fe0 2023-11-15T03:09:59.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.558+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404019020 con 0x559404014fe0 2023-11-15T03:09:59.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.558+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v128'351 uv345 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404014fe0 2023-11-15T03:09:59.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.558+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x5594040193d0 con 0x559404014fe0 2023-11-15T03:09:59.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.562+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v128'352 uv352 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404014fe0 2023-11-15T03:09:59.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.562+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404019780 con 0x559404014fe0 2023-11-15T03:09:59.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.566+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v128'353 uv353 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404014fe0 2023-11-15T03:09:59.576 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:09:59.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.566+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404019780 con 0x559404014fe0 2023-11-15T03:09:59.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.570+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v128'354 uv354 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404014fe0 2023-11-15T03:09:59.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.570+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404018490 con 0x559404014fe0 2023-11-15T03:09:59.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.570+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v128'355 uv355 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404014fe0 2023-11-15T03:09:59.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.570+0000 7febbccdad00 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55940401a180 0x55940401c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.570+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940401ca60 con 0x55940401a180 2023-11-15T03:09:59.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.574+0000 7febbadbd700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55940401a180 0x55940401c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.574+0000 7febbadbd700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55940401a180 0x55940401c570 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:59.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.578+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v128'406 uv405 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.578+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x55940401ca60 con 0x55940401a180 2023-11-15T03:09:59.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.582+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v128'407 uv407 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.582+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940401ca60 con 0x55940401a180 2023-11-15T03:09:59.593 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:09:59.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.582+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v128'408 uv408 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.582+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404020af0 con 0x55940401a180 2023-11-15T03:09:59.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.586+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v128'409 uv409 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.586+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404020af0 con 0x55940401a180 2023-11-15T03:09:59.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.594+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v128'410 uv410 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.594+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940401ca60 con 0x55940401a180 2023-11-15T03:09:59.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.598+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v128'411 uv402 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.598+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x55940401ca60 con 0x55940401a180 2023-11-15T03:09:59.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.602+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v128'412 uv412 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.602+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940401ca60 con 0x55940401a180 2023-11-15T03:09:59.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.606+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v128'413 uv413 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.616 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:09:59.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.606+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940401da70 con 0x55940401a180 2023-11-15T03:09:59.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.610+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v128'414 uv414 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.610+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940401da70 con 0x55940401a180 2023-11-15T03:09:59.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.614+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v128'415 uv415 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.614+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940401ca60 con 0x55940401a180 2023-11-15T03:09:59.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.618+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v128'416 uv416 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.618+0000 7febbccdad00 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55940401d0e0 0x55940401f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.618+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940401fb30 con 0x55940401d0e0 2023-11-15T03:09:59.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.618+0000 7febba5bc700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55940401d0e0 0x55940401f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.618+0000 7febba5bc700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55940401d0e0 0x55940401f4d0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:59.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.618+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v128'201 uv200 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x55940401d0e0 2023-11-15T03:09:59.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.622+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x55940401ff10 con 0x55940401d0e0 2023-11-15T03:09:59.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.622+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v128'202 uv202 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x55940401d0e0 2023-11-15T03:09:59.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.622+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940401ff10 con 0x55940401d0e0 2023-11-15T03:09:59.635 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:09:59.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.626+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v128'203 uv203 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x55940401d0e0 2023-11-15T03:09:59.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.626+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404020f20 con 0x55940401d0e0 2023-11-15T03:09:59.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.626+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v128'204 uv204 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x55940401d0e0 2023-11-15T03:09:59.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.626+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404020f20 con 0x55940401d0e0 2023-11-15T03:09:59.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.630+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v128'205 uv205 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x55940401d0e0 2023-11-15T03:09:59.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.630+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x55940401ff10 con 0x55940401d0e0 2023-11-15T03:09:59.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.630+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v128'206 uv206 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x55940401d0e0 2023-11-15T03:09:59.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.630+0000 7febbccdad00 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x559404020590 0x559404022980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.630+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404023090 con 0x559404020590 2023-11-15T03:09:59.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.634+0000 7febb1dbb700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x559404020590 0x559404022980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.634+0000 7febb1dbb700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x559404020590 0x559404022980 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:59.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.638+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v128'145 uv144 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.638+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x559404023470 con 0x559404020590 2023-11-15T03:09:59.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.638+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v128'146 uv146 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.638+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404023470 con 0x559404020590 2023-11-15T03:09:59.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.642+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v128'147 uv147 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.652 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:09:59.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.642+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404023470 con 0x559404020590 2023-11-15T03:09:59.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.646+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v128'148 uv148 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.646+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040233a0 con 0x559404020590 2023-11-15T03:09:59.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.646+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v128'201 uv200 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.646+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x5594040233a0 con 0x559404020590 2023-11-15T03:09:59.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.650+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v128'202 uv202 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.650+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040233a0 con 0x559404020590 2023-11-15T03:09:59.664 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:09:59.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.654+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v128'203 uv203 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.654+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040233a0 con 0x559404020590 2023-11-15T03:09:59.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.658+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v128'204 uv204 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.658+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040233a0 con 0x559404020590 2023-11-15T03:09:59.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.658+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v128'205 uv205 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.658+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040233a0 con 0x559404020590 2023-11-15T03:09:59.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.662+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v128'206 uv206 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.662+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040233a0 con 0x559404010320 2023-11-15T03:09:59.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.666+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v128'371 uv365 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.666+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x5594040233a0 con 0x559404010320 2023-11-15T03:09:59.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.670+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v128'372 uv372 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.670+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040233a0 con 0x559404010320 2023-11-15T03:09:59.682 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:09:59.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.670+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v128'373 uv373 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.670+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040247b0 con 0x559404010320 2023-11-15T03:09:59.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.674+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v128'374 uv374 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.674+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040247b0 con 0x559404010320 2023-11-15T03:09:59.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.678+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v128'375 uv375 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.678+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040237a0 con 0x559404010320 2023-11-15T03:09:59.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.682+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v128'376 uv376 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.682+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040237a0 con 0x559404010320 2023-11-15T03:09:59.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.686+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v128'377 uv370 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.686+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x5594040237a0 con 0x559404010320 2023-11-15T03:09:59.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.686+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v128'378 uv378 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.686+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040237a0 con 0x559404010320 2023-11-15T03:09:59.700 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:09:59.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.690+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v128'379 uv379 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.690+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040237a0 con 0x559404010320 2023-11-15T03:09:59.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.694+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v128'380 uv380 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.694+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040237a0 con 0x559404010320 2023-11-15T03:09:59.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.698+0000 7febba5bc700 1 -- 172.21.15.121:0/2206188074 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v128'381 uv381 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febac023020 con 0x559404010320 2023-11-15T03:09:59.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.698+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040237a0 con 0x559404014fe0 2023-11-15T03:09:59.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.698+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v128'168 uv167 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404014fe0 2023-11-15T03:09:59.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.698+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x5594040234b0 con 0x559404014fe0 2023-11-15T03:09:59.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.702+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v128'169 uv169 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404014fe0 2023-11-15T03:09:59.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.702+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040234b0 con 0x559404014fe0 2023-11-15T03:09:59.715 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:09:59.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.706+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v128'170 uv170 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404014fe0 2023-11-15T03:09:59.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.706+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040234b0 con 0x559404014fe0 2023-11-15T03:09:59.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.706+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v128'171 uv171 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404014fe0 2023-11-15T03:09:59.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.706+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040234b0 con 0x559404014fe0 2023-11-15T03:09:59.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.710+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v128'172 uv172 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404014fe0 2023-11-15T03:09:59.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.710+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040234b0 con 0x559404020590 2023-11-15T03:09:59.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.714+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v128'201 uv200 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.714+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x5594040234b0 con 0x559404020590 2023-11-15T03:09:59.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.714+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v128'202 uv202 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.714+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040234b0 con 0x559404020590 2023-11-15T03:09:59.729 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:09:59.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.718+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v128'203 uv203 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.718+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040244c0 con 0x559404020590 2023-11-15T03:09:59.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.722+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v128'204 uv204 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.722+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040244c0 con 0x559404020590 2023-11-15T03:09:59.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.726+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v128'205 uv205 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.726+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040234b0 con 0x559404020590 2023-11-15T03:09:59.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.726+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v128'206 uv206 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.726+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040234b0 con 0x559404020590 2023-11-15T03:09:59.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.730+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v128'169 uv168 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.730+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x5594040234b0 con 0x559404020590 2023-11-15T03:09:59.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.734+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v128'170 uv170 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.734+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040234b0 con 0x559404020590 2023-11-15T03:09:59.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.738+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v128'171 uv171 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.747 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:09:59.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.738+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040234b0 con 0x559404020590 2023-11-15T03:09:59.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.738+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v128'172 uv172 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.738+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040238b0 con 0x559404020590 2023-11-15T03:09:59.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.742+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v128'173 uv173 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.742+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040238b0 con 0x559404020590 2023-11-15T03:09:59.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.746+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v128'167 uv166 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.746+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x5594040234b0 con 0x559404020590 2023-11-15T03:09:59.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.750+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v128'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.750+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040234f0 con 0x559404020590 2023-11-15T03:09:59.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.750+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v128'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.762 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:09:59.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.750+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040238f0 con 0x559404020590 2023-11-15T03:09:59.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.754+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v128'170 uv170 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.754+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404026660 con 0x559404020590 2023-11-15T03:09:59.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.758+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v128'171 uv171 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.758+0000 7febbccdad00 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x559404024150 0x559404026540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.758+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404026a30 con 0x559404024150 2023-11-15T03:09:59.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.758+0000 7febbadbd700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x559404024150 0x559404026540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.758+0000 7febbadbd700 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x559404024150 0x559404026540 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:59.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.762+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v128'134 uv133 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x559404024150 2023-11-15T03:09:59.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.762+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x559404026a30 con 0x559404024150 2023-11-15T03:09:59.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.766+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v128'135 uv135 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x559404024150 2023-11-15T03:09:59.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.766+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404026a30 con 0x559404024150 2023-11-15T03:09:59.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.770+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v128'136 uv136 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x559404024150 2023-11-15T03:09:59.779 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:09:59.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.770+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404026a30 con 0x559404024150 2023-11-15T03:09:59.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.770+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v128'137 uv137 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x559404024150 2023-11-15T03:09:59.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.770+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404026a30 con 0x559404020590 2023-11-15T03:09:59.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.774+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v128'201 uv200 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.774+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x559404026a30 con 0x559404020590 2023-11-15T03:09:59.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.778+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v128'202 uv202 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.778+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404026a30 con 0x559404020590 2023-11-15T03:09:59.791 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:09:59.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.782+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v128'203 uv203 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.782+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404027a40 con 0x559404020590 2023-11-15T03:09:59.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.782+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v128'204 uv204 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.782+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404027a40 con 0x559404020590 2023-11-15T03:09:59.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.786+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v128'205 uv205 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.786+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404026a30 con 0x559404020590 2023-11-15T03:09:59.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.790+0000 7febb1dbb700 1 -- 172.21.15.121:0/2206188074 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v128'206 uv206 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7feba4008bf0 con 0x559404020590 2023-11-15T03:09:59.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.790+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404026a30 con 0x55940401a180 2023-11-15T03:09:59.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.790+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v128'171 uv170 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.790+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x559404026a30 con 0x55940401a180 2023-11-15T03:09:59.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.794+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v128'172 uv172 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.794+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404026a30 con 0x55940401a180 2023-11-15T03:09:59.807 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:09:59.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.798+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v128'173 uv173 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.798+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040237b0 con 0x55940401a180 2023-11-15T03:09:59.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.802+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v128'174 uv174 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.802+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040237b0 con 0x55940401a180 2023-11-15T03:09:59.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.802+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v128'175 uv175 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.802+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040237b0 con 0x55940401a180 2023-11-15T03:09:59.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.806+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v128'403 uv394 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.806+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x5594040237b0 con 0x55940401a180 2023-11-15T03:09:59.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.810+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v128'404 uv404 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.810+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040237b0 con 0x55940401a180 2023-11-15T03:09:59.822 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:09:59.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.814+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v128'405 uv405 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.814+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040237b0 con 0x55940401a180 2023-11-15T03:09:59.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.814+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v128'406 uv406 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.814+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404023870 con 0x55940401a180 2023-11-15T03:09:59.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.818+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v128'407 uv407 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.818+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404023830 con 0x55940401a180 2023-11-15T03:09:59.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.822+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v128'408 uv402 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.822+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e128) v8 -- 0x5594040237f0 con 0x55940401a180 2023-11-15T03:09:59.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.822+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v128'409 uv409 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.826+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040237f0 con 0x55940401a180 2023-11-15T03:09:59.837 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:09:59.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.826+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v128'410 uv410 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.826+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040237d0 con 0x55940401a180 2023-11-15T03:09:59.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.830+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v128'411 uv411 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.830+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5594040237b0 con 0x55940401a180 2023-11-15T03:09:59.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.834+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v128'412 uv412 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.834+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404026f20 con 0x55940401a180 2023-11-15T03:09:59.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.834+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v128'413 uv413 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.834+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x559404027190 con 0x55940401a180 2023-11-15T03:09:59.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v128'414 uv414 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7febb400bc60 con 0x55940401a180 2023-11-15T03:09:59.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x559404014fe0 msgr2=0x5594040173d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x559404014fe0 0x5594040173d0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x559403ff30f0 msgr2=0x55940400dbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x559403ff30f0 0x55940400dbb0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55940401d0e0 msgr2=0x55940401f4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55940401d0e0 0x55940401f4d0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x559404010320 msgr2=0x559404012710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x559404010320 0x559404012710 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x559404024150 msgr2=0x559404026540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x559404024150 0x559404026540 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55940401a180 msgr2=0x55940401c570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55940401a180 0x55940401c570 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x559404020590 msgr2=0x559404022980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x559404020590 0x559404022980 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 reap_dead start 2023-11-15T03:09:59.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 reap_dead start 2023-11-15T03:09:59.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 reap_dead start 2023-11-15T03:09:59.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feb9c059020 msgr2=0x7feb9c05b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 reap_dead start 2023-11-15T03:09:59.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feb9c059020 0x7feb9c05b4e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x559403f15470 tx=0x7feba4005b10).stop 2023-11-15T03:09:59.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559403f14a30 msgr2=0x559404002ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559403f14a30 0x559404002ad0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 rx=0x7febac00a150 tx=0x7febac012000).stop 2023-11-15T03:09:59.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbadbd700 1 -- 172.21.15.121:0/2206188074 reap_dead start 2023-11-15T03:09:59.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 shutdown_connections 2023-11-15T03:09:59.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 --2- 172.21.15.121:0/2206188074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559403f14a30 0x559404002ad0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 >> 172.21.15.121:0/2206188074 conn(0x559403f64080 msgr2=0x559403ff54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:59.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 shutdown_connections 2023-11-15T03:09:59.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.838+0000 7febbccdad00 1 -- 172.21.15.121:0/2206188074 wait complete. 2023-11-15T03:09:59.856 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 10000000000.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-11-15T03:09:59.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.954+0000 7f8a6e8dad00 1 Processor -- start 2023-11-15T03:09:59.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.954+0000 7f8a6e8dad00 1 -- start start 2023-11-15T03:09:59.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.954+0000 7f8a6e8dad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639dcb80440 0x5639dcc28a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.954+0000 7f8a6e8dad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639dcb9ac60 0x5639dcc0b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.954+0000 7f8a6e8dad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5639dcc0d2e0 0x5639dcb99380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.954+0000 7f8a6e8dad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dcc28f60 con 0x5639dcb80440 2023-11-15T03:09:59.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.954+0000 7f8a6e8dad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dcb7dac0 con 0x5639dcb9ac60 2023-11-15T03:09:59.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.954+0000 7f8a6e8dad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639dcb4e230 con 0x5639dcc0d2e0 2023-11-15T03:09:59.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.954+0000 7f8a6c15e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639dcb80440 0x5639dcc28a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.954+0000 7f8a6c15e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639dcb80440 0x5639dcc28a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40250/0 (socket says 172.21.15.121:40250) 2023-11-15T03:09:59.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.954+0000 7f8a6c15e700 1 -- 172.21.15.121:0/2010075107 learned_addr learned my addr 172.21.15.121:0/2010075107 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:59.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6c95f700 1 --2- 172.21.15.121:0/2010075107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5639dcc0d2e0 0x5639dcb99380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6395d700 1 --2- 172.21.15.121:0/2010075107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639dcb9ac60 0x5639dcc0b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6c15e700 1 -- 172.21.15.121:0/2010075107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5639dcc0d2e0 msgr2=0x5639dcb99380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:59.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6c15e700 1 --2- 172.21.15.121:0/2010075107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5639dcc0d2e0 0x5639dcb99380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6c15e700 1 -- 172.21.15.121:0/2010075107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639dcb9ac60 msgr2=0x5639dcc0b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:09:59.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6c15e700 1 --2- 172.21.15.121:0/2010075107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639dcb9ac60 0x5639dcc0b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6c15e700 1 -- 172.21.15.121:0/2010075107 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a64005cf0 con 0x5639dcb80440 2023-11-15T03:09:59.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6c95f700 1 --2- 172.21.15.121:0/2010075107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5639dcc0d2e0 0x5639dcb99380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:09:59.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6395d700 1 --2- 172.21.15.121:0/2010075107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639dcb9ac60 0x5639dcc0b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:09:59.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6c15e700 1 --2- 172.21.15.121:0/2010075107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639dcb80440 0x5639dcc28a70 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 rx=0x7f8a5c00a840 tx=0x7f8a5c007040).ready entity=mon.0 client_cookie=b74bb4ecb97a5cb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:59.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6b15c700 1 -- 172.21.15.121:0/2010075107 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c00d6f0 con 0x5639dcb80440 2023-11-15T03:09:59.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6b15c700 1 -- 172.21.15.121:0/2010075107 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5c00d850 con 0x5639dcb80440 2023-11-15T03:09:59.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6b15c700 1 -- 172.21.15.121:0/2010075107 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c00dbd0 con 0x5639dcb80440 2023-11-15T03:09:59.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6e8dad00 1 -- 172.21.15.121:0/2010075107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639dcb80440 msgr2=0x5639dcc28a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6e8dad00 1 --2- 172.21.15.121:0/2010075107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639dcb80440 0x5639dcc28a70 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 rx=0x7f8a5c00a840 tx=0x7f8a5c007040).stop 2023-11-15T03:09:59.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6e8dad00 1 -- 172.21.15.121:0/2010075107 shutdown_connections 2023-11-15T03:09:59.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6e8dad00 1 --2- 172.21.15.121:0/2010075107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5639dcc0d2e0 0x5639dcb99380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6e8dad00 1 --2- 172.21.15.121:0/2010075107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639dcb9ac60 0x5639dcc0b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6e8dad00 1 --2- 172.21.15.121:0/2010075107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639dcb80440 0x5639dcc28a70 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6e8dad00 1 -- 172.21.15.121:0/2010075107 >> 172.21.15.121:0/2010075107 conn(0x5639dcb4f7d0 msgr2=0x5639dcb7f910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:59.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6e8dad00 1 -- 172.21.15.121:0/2010075107 shutdown_connections 2023-11-15T03:09:59.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.958+0000 7f8a6e8dad00 1 -- 172.21.15.121:0/2010075107 wait complete. 2023-11-15T03:09:59.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a6e8dad00 1 Processor -- start 2023-11-15T03:09:59.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a6e8dad00 1 -- start start 2023-11-15T03:09:59.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a6e8dad00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639dcb80440 0x5639dcc2ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a6e8dad00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639dcb9ac60 0x5639dcc0c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a6e8dad00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5639dcc0d2e0 0x5639dcc2be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a6e8dad00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dcc0c8c0 con 0x5639dcb80440 2023-11-15T03:09:59.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a6e8dad00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dcc2f0d0 con 0x5639dcb9ac60 2023-11-15T03:09:59.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a6e8dad00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639dcc2cad0 con 0x5639dcc0d2e0 2023-11-15T03:09:59.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a6c15e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639dcb80440 0x5639dcc2ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a6c15e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639dcb80440 0x5639dcc2ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40260/0 (socket says 172.21.15.121:40260) 2023-11-15T03:09:59.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a6c15e700 1 -- 172.21.15.121:0/1179596337 learned_addr learned my addr 172.21.15.121:0/1179596337 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:09:59.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a6c95f700 1 --2- 172.21.15.121:0/1179596337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5639dcc0d2e0 0x5639dcc2be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a6395d700 1 --2- 172.21.15.121:0/1179596337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639dcb9ac60 0x5639dcc0c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a6c15e700 1 -- 172.21.15.121:0/1179596337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5639dcc0d2e0 msgr2=0x5639dcc2be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a6c15e700 1 --2- 172.21.15.121:0/1179596337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5639dcc0d2e0 0x5639dcc2be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a6c15e700 1 -- 172.21.15.121:0/1179596337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639dcb9ac60 msgr2=0x5639dcc0c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a6c15e700 1 --2- 172.21.15.121:0/1179596337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639dcb9ac60 0x5639dcc0c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a6c15e700 1 -- 172.21.15.121:0/1179596337 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5c00a440 con 0x5639dcb80440 2023-11-15T03:09:59.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a6c15e700 1 --2- 172.21.15.121:0/1179596337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639dcb80440 0x5639dcc2ead0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 rx=0x7f8a5c00af90 tx=0x7f8a5c019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:59.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.962+0000 7f8a69959700 1 -- 172.21.15.121:0/1179596337 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c005a70 con 0x5639dcb80440 2023-11-15T03:09:59.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.966+0000 7f8a69959700 1 -- 172.21.15.121:0/1179596337 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5c00d9c0 con 0x5639dcb80440 2023-11-15T03:09:59.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.966+0000 7f8a6e8dad00 1 -- 172.21.15.121:0/1179596337 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5639dcaf0a20 con 0x5639dcb80440 2023-11-15T03:09:59.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.966+0000 7f8a69959700 1 -- 172.21.15.121:0/1179596337 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c00db30 con 0x5639dcb80440 2023-11-15T03:09:59.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.966+0000 7f8a6e8dad00 1 -- 172.21.15.121:0/1179596337 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639dcaf0770 con 0x5639dcb80440 2023-11-15T03:09:59.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.966+0000 7f8a69959700 1 -- 172.21.15.121:0/1179596337 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8a5c021020 con 0x5639dcb80440 2023-11-15T03:09:59.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.970+0000 7f8a69959700 1 --2- 172.21.15.121:0/1179596337 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8a4c058ff0 0x7f8a4c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.970+0000 7f8a69959700 1 -- 172.21.15.121:0/1179596337 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9190+0+0 (secure 0 0 0) 0x7f8a5c075990 con 0x5639dcb80440 2023-11-15T03:09:59.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.970+0000 7f8a6395d700 1 --2- 172.21.15.121:0/1179596337 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8a4c058ff0 0x7f8a4c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.970+0000 7f8a6e8dad00 1 --2- 172.21.15.121:0/1179596337 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5639dcb51760 0x5639dcc1f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:09:59.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.970+0000 7f8a6e8dad00 1 -- 172.21.15.121:0/1179596337 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:1 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected e128) v8 -- 0x5639dcc1f530 con 0x5639dcb51760 2023-11-15T03:09:59.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.970+0000 7f8a6395d700 1 --2- 172.21.15.121:0/1179596337 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8a4c058ff0 0x7f8a4c05b4b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x5639dcb7fe60 tx=0x7f8a5400c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:59.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.970+0000 7f8a6c95f700 1 --2- 172.21.15.121:0/1179596337 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5639dcb51760 0x5639dcc1f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:09:59.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.970+0000 7f8a6c95f700 1 --2- 172.21.15.121:0/1179596337 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5639dcb51760 0x5639dcc1f040 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:09:59.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.974+0000 7f8a6c95f700 1 -- 172.21.15.121:0/1179596337 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-header] v128'652 uv652 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8a640078e0 con 0x5639dcb51760 2023-11-15T03:09:59.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.974+0000 7f8a6e8dad00 1 -- 172.21.15.121:0/1179596337 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5639dcb51760 msgr2=0x5639dcc1f040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.974+0000 7f8a6e8dad00 1 --2- 172.21.15.121:0/1179596337 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5639dcb51760 0x5639dcc1f040 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:09:59.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.974+0000 7f8a6e8dad00 1 -- 172.21.15.121:0/1179596337 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8a4c058ff0 msgr2=0x7f8a4c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.974+0000 7f8a6e8dad00 1 --2- 172.21.15.121:0/1179596337 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8a4c058ff0 0x7f8a4c05b4b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x5639dcb7fe60 tx=0x7f8a5400c040).stop 2023-11-15T03:09:59.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.974+0000 7f8a6e8dad00 1 -- 172.21.15.121:0/1179596337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639dcb80440 msgr2=0x5639dcc2ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:09:59.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.974+0000 7f8a6e8dad00 1 --2- 172.21.15.121:0/1179596337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639dcb80440 0x5639dcc2ead0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 rx=0x7f8a5c00af90 tx=0x7f8a5c019000).stop 2023-11-15T03:09:59.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.974+0000 7f8a6c95f700 1 -- 172.21.15.121:0/1179596337 reap_dead start 2023-11-15T03:09:59.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.974+0000 7f8a6e8dad00 1 -- 172.21.15.121:0/1179596337 shutdown_connections 2023-11-15T03:09:59.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.974+0000 7f8a6e8dad00 1 -- 172.21.15.121:0/1179596337 >> 172.21.15.121:0/1179596337 conn(0x5639dcb4f7d0 msgr2=0x5639dcc356a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:09:59.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.974+0000 7f8a6e8dad00 1 -- 172.21.15.121:0/1179596337 shutdown_connections 2023-11-15T03:09:59.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:09:59.974+0000 7f8a6e8dad00 1 -- 172.21.15.121:0/1179596337 wait complete. 2023-11-15T03:09:59.993 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:10:00.004 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:09:59.994+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2023-11-15T03:10:00.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.158+0000 7f6cb9c9d700 1 Processor -- start 2023-11-15T03:10:00.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.158+0000 7f6cb9c9d700 1 -- start start 2023-11-15T03:10:00.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.162+0000 7f6cb9c9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6cb4151af0 0x7f6cb40a7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:00.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.162+0000 7f6cb9c9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6cb40a5720 0x7f6cb40c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:00.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.162+0000 7f6cb9c9d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb41528b0 con 0x7f6cb40c3430 2023-11-15T03:10:00.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.162+0000 7f6cb9c9d700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb4066d80 con 0x7f6cb40a5720 2023-11-15T03:10:00.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.162+0000 7f6cb9c9d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb40c38c0 con 0x7f6cb4151af0 2023-11-15T03:10:00.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.162+0000 7f6cabfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6cb40a5720 0x7f6cb40c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:00.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.162+0000 7f6cabfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6cb40a5720 0x7f6cb40c2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40268/0 (socket says 172.21.15.121:40268) 2023-11-15T03:10:00.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.162+0000 7f6cabfff700 1 -- 172.21.15.121:0/397710218 learned_addr learned my addr 172.21.15.121:0/397710218 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:00.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.162+0000 7f6cb8c9b700 1 --2- 172.21.15.121:0/397710218 >> v2:172.21.15.190:3300/0 conn(0x7f6cb4151af0 0x7f6cb40a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:00.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.162+0000 7f6cabfff700 1 -- 172.21.15.121:0/397710218 >> v2:172.21.15.190:3300/0 conn(0x7f6cb4151af0 msgr2=0x7f6cb40a7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:00.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.162+0000 7f6cabfff700 1 --2- 172.21.15.121:0/397710218 >> v2:172.21.15.190:3300/0 conn(0x7f6cb4151af0 0x7f6cb40a7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:00.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.162+0000 7f6cabfff700 1 -- 172.21.15.121:0/397710218 >> v1:172.21.15.190:6789/0 conn(0x7f6cb40c3430 legacy=0x7f6cb40c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:00.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.162+0000 7f6cabfff700 1 -- 172.21.15.121:0/397710218 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb4151ff0 con 0x7f6cb40a5720 2023-11-15T03:10:00.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.166+0000 7f6cabfff700 1 --2- 172.21.15.121:0/397710218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb40a5720 0x7f6cb40c2f40 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 rx=0x7f6ca4006fd0 tx=0x7f6ca40090f0).ready entity=mon.0 client_cookie=a715ab88630ef32 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:00.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.166+0000 7f6cab7fe700 1 -- 172.21.15.121:0/397710218 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca400b600 con 0x7f6cb40a5720 2023-11-15T03:10:00.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.166+0000 7f6cab7fe700 1 -- 172.21.15.121:0/397710218 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca400b760 con 0x7f6cb40a5720 2023-11-15T03:10:00.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.166+0000 7f6cab7fe700 1 -- 172.21.15.121:0/397710218 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca400bae0 con 0x7f6cb40a5720 2023-11-15T03:10:00.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.166+0000 7f6cb9c9d700 1 -- 172.21.15.121:0/397710218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb40a5720 msgr2=0x7f6cb40c2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:00.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.166+0000 7f6cb9c9d700 1 --2- 172.21.15.121:0/397710218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb40a5720 0x7f6cb40c2f40 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 rx=0x7f6ca4006fd0 tx=0x7f6ca40090f0).stop 2023-11-15T03:10:00.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.166+0000 7f6cb9c9d700 1 -- 172.21.15.121:0/397710218 shutdown_connections 2023-11-15T03:10:00.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.166+0000 7f6cb9c9d700 1 --2- 172.21.15.121:0/397710218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb40a5720 0x7f6cb40c2f40 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:00.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.166+0000 7f6cb9c9d700 1 --2- 172.21.15.121:0/397710218 >> v2:172.21.15.190:3300/0 conn(0x7f6cb4151af0 0x7f6cb40a7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:00.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.166+0000 7f6cb9c9d700 1 -- 172.21.15.121:0/397710218 >> 172.21.15.121:0/397710218 conn(0x7f6cb4067a60 msgr2=0x7f6cb4067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:00.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.166+0000 7f6cb9c9d700 1 -- 172.21.15.121:0/397710218 shutdown_connections 2023-11-15T03:10:00.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.166+0000 7f6cb9c9d700 1 -- 172.21.15.121:0/397710218 wait complete. 2023-11-15T03:10:00.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.170+0000 7f6cb9c9d700 1 Processor -- start 2023-11-15T03:10:00.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.170+0000 7f6cb9c9d700 1 -- start start 2023-11-15T03:10:00.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.170+0000 7f6cb9c9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cb40a5720 0x7f6cb414d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:00.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.170+0000 7f6cb9c9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb40c3430 0x7f6cb41401b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:00.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.170+0000 7f6cb9c9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6cb40c6c90 0x7f6cb4159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:00.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.170+0000 7f6cb9c9d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb41406a0 con 0x7f6cb40c3430 2023-11-15T03:10:00.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.170+0000 7f6cb9c9d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40c58a0 con 0x7f6cb40c6c90 2023-11-15T03:10:00.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.170+0000 7f6cb9c9d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40c4ce0 con 0x7f6cb40a5720 2023-11-15T03:10:00.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.170+0000 7f6cabfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb40c3430 0x7f6cb41401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:00.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.170+0000 7f6cabfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb40c3430 0x7f6cb41401b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40272/0 (socket says 172.21.15.121:40272) 2023-11-15T03:10:00.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.170+0000 7f6cabfff700 1 -- 172.21.15.121:0/1838447191 learned_addr learned my addr 172.21.15.121:0/1838447191 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:00.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.170+0000 7f6cb8c9b700 1 --2- 172.21.15.121:0/1838447191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cb40a5720 0x7f6cb414d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:00.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.170+0000 7f6cb949c700 1 --2- 172.21.15.121:0/1838447191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6cb40c6c90 0x7f6cb4159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:00.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.170+0000 7f6cabfff700 1 -- 172.21.15.121:0/1838447191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cb40a5720 msgr2=0x7f6cb414d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:00.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.170+0000 7f6cabfff700 1 --2- 172.21.15.121:0/1838447191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cb40a5720 0x7f6cb414d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:00.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.174+0000 7f6cabfff700 1 -- 172.21.15.121:0/1838447191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6cb40c6c90 msgr2=0x7f6cb4159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:00.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.174+0000 7f6cabfff700 1 --2- 172.21.15.121:0/1838447191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6cb40c6c90 0x7f6cb4159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:00.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.174+0000 7f6cabfff700 1 -- 172.21.15.121:0/1838447191 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca4009370 con 0x7f6cb40c3430 2023-11-15T03:10:00.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.174+0000 7f6cabfff700 1 --2- 172.21.15.121:0/1838447191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb40c3430 0x7f6cb41401b0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 rx=0x7f6cb40c75a0 tx=0x7f6ca4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:00.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.174+0000 7f6ca8ff9700 1 -- 172.21.15.121:0/1838447191 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca400bea0 con 0x7f6cb40c3430 2023-11-15T03:10:00.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.174+0000 7f6ca8ff9700 1 -- 172.21.15.121:0/1838447191 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca400b8d0 con 0x7f6cb40c3430 2023-11-15T03:10:00.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.174+0000 7f6cb9c9d700 1 -- 172.21.15.121:0/1838447191 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb40c5d30 con 0x7f6cb40c3430 2023-11-15T03:10:00.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.174+0000 7f6ca8ff9700 1 -- 172.21.15.121:0/1838447191 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca401b640 con 0x7f6cb40c3430 2023-11-15T03:10:00.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.174+0000 7f6cb9c9d700 1 -- 172.21.15.121:0/1838447191 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb40c5250 con 0x7f6cb40c3430 2023-11-15T03:10:00.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.174+0000 7f6cb9c9d700 1 -- 172.21.15.121:0/1838447191 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb406bcf0 con 0x7f6cb40c3430 2023-11-15T03:10:00.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.174+0000 7f6ca8ff9700 1 -- 172.21.15.121:0/1838447191 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6ca401b9b0 con 0x7f6cb40c3430 2023-11-15T03:10:00.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.178+0000 7f6ca8ff9700 1 --2- 172.21.15.121:0/1838447191 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6ca005a510 0x7f6ca005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:00.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.178+0000 7f6ca8ff9700 1 -- 172.21.15.121:0/1838447191 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9190+0+0 (secure 0 0 0) 0x7f6ca4076ca0 con 0x7f6cb40c3430 2023-11-15T03:10:00.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.178+0000 7f6cb8c9b700 1 --2- 172.21.15.121:0/1838447191 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6ca005a510 0x7f6ca005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:00.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.178+0000 7f6cb8c9b700 1 --2- 172.21.15.121:0/1838447191 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6ca005a510 0x7f6ca005c9d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f6cb40c20e0 tx=0x7f6cb0009850).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:00.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.182+0000 7f6ca8ff9700 1 -- 172.21.15.121:0/1838447191 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6ca4049840 con 0x7f6cb40c3430 2023-11-15T03:10:00.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:00.382+0000 7f6cb9c9d700 1 -- 172.21.15.121:0/1838447191 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f6cb4068a70 con 0x7f6cb40c3430 2023-11-15T03:10:01.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.322+0000 7f6ca8ff9700 1 -- 172.21.15.121:0/1838447191 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v300) v1 ==== 162+0+0 (secure 0 0 0) 0x7f6ca4029030 con 0x7f6cb40c3430 2023-11-15T03:10:01.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.322+0000 7f6cb9c9d700 1 -- 172.21.15.121:0/1838447191 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6ca005a510 msgr2=0x7f6ca005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:01.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.322+0000 7f6cb9c9d700 1 --2- 172.21.15.121:0/1838447191 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6ca005a510 0x7f6ca005c9d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f6cb40c20e0 tx=0x7f6cb0009850).stop 2023-11-15T03:10:01.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.322+0000 7f6cb9c9d700 1 -- 172.21.15.121:0/1838447191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb40c3430 msgr2=0x7f6cb41401b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:01.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.322+0000 7f6cb9c9d700 1 --2- 172.21.15.121:0/1838447191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb40c3430 0x7f6cb41401b0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 rx=0x7f6cb40c75a0 tx=0x7f6ca4017000).stop 2023-11-15T03:10:01.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.326+0000 7f6cb9c9d700 1 -- 172.21.15.121:0/1838447191 shutdown_connections 2023-11-15T03:10:01.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.326+0000 7f6cb9c9d700 1 --2- 172.21.15.121:0/1838447191 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6ca005a510 0x7f6ca005c9d0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:01.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.326+0000 7f6cb9c9d700 1 --2- 172.21.15.121:0/1838447191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6cb40c6c90 0x7f6cb4159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:01.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.326+0000 7f6cb9c9d700 1 --2- 172.21.15.121:0/1838447191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb40c3430 0x7f6cb41401b0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:01.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.326+0000 7f6cb9c9d700 1 --2- 172.21.15.121:0/1838447191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cb40a5720 0x7f6cb414d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:01.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.326+0000 7f6cb9c9d700 1 -- 172.21.15.121:0/1838447191 >> 172.21.15.121:0/1838447191 conn(0x7f6cb4067a60 msgr2=0x7f6cb40c2350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:01.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.326+0000 7f6cb9c9d700 1 -- 172.21.15.121:0/1838447191 shutdown_connections 2023-11-15T03:10:01.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.326+0000 7f6cb9c9d700 1 -- 172.21.15.121:0/1838447191 wait complete. 2023-11-15T03:10:01.338 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:10:01.761 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:10:01.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.918+0000 7f58d588f700 1 Processor -- start 2023-11-15T03:10:01.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.918+0000 7f58d588f700 1 -- start start 2023-11-15T03:10:01.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.918+0000 7f58d588f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f58d00c3280 0x7f58d00c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:01.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.918+0000 7f58d588f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f58d015be30 0x7f58d015c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:01.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.918+0000 7f58d588f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d00c1d50 con 0x7f58d0152140 2023-11-15T03:10:01.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.918+0000 7f58d588f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d00c1860 con 0x7f58d015be30 2023-11-15T03:10:01.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.918+0000 7f58d588f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d015cb80 con 0x7f58d00c3280 2023-11-15T03:10:01.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58d508e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f58d015be30 0x7f58d015c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:01.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58d488d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f58d00c3280 0x7f58d00c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:01.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58cffff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f58d0152140 0x7f58d01525f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:36848/0 (socket says 172.21.15.121:36848) 2023-11-15T03:10:01.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58cffff700 1 -- 172.21.15.121:0/115303282 learned_addr learned my addr 172.21.15.121:0/115303282 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:01.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58cf7fe700 1 -- 172.21.15.121:0/115303282 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330837469 0 0) 0x7f58d00c1d50 con 0x7f58d0152140 2023-11-15T03:10:01.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58cf7fe700 1 -- 172.21.15.121:0/115303282 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58b80033f0 con 0x7f58d0152140 2023-11-15T03:10:01.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58d508e700 1 -- 172.21.15.121:0/115303282 >> v2:172.21.15.190:3300/0 conn(0x7f58d00c3280 msgr2=0x7f58d00c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:01.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58d508e700 1 --2- 172.21.15.121:0/115303282 >> v2:172.21.15.190:3300/0 conn(0x7f58d00c3280 0x7f58d00c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:01.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58d508e700 1 -- 172.21.15.121:0/115303282 >> v1:172.21.15.190:6789/0 conn(0x7f58d0152140 legacy=0x7f58d01525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:01.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58d508e700 1 -- 172.21.15.121:0/115303282 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d00a9240 con 0x7f58d015be30 2023-11-15T03:10:01.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58d508e700 1 --2- 172.21.15.121:0/115303282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d015be30 0x7f58d015c280 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 rx=0x7f58d00c7720 tx=0x7f58c8009780).ready entity=mon.0 client_cookie=65d90a4c01038eaf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:01.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58cf7fe700 1 -- 172.21.15.121:0/115303282 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c800e070 con 0x7f58d015be30 2023-11-15T03:10:01.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58cf7fe700 1 -- 172.21.15.121:0/115303282 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c800bab0 con 0x7f58d015be30 2023-11-15T03:10:01.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58cf7fe700 1 -- 172.21.15.121:0/115303282 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c800be30 con 0x7f58d015be30 2023-11-15T03:10:01.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58d588f700 1 -- 172.21.15.121:0/115303282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d015be30 msgr2=0x7f58d015c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:01.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58d588f700 1 --2- 172.21.15.121:0/115303282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d015be30 0x7f58d015c280 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 rx=0x7f58d00c7720 tx=0x7f58c8009780).stop 2023-11-15T03:10:01.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58d588f700 1 -- 172.21.15.121:0/115303282 shutdown_connections 2023-11-15T03:10:01.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58d588f700 1 --2- 172.21.15.121:0/115303282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d015be30 0x7f58d015c280 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:01.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58d588f700 1 --2- 172.21.15.121:0/115303282 >> v2:172.21.15.190:3300/0 conn(0x7f58d00c3280 0x7f58d00c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:01.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58d588f700 1 -- 172.21.15.121:0/115303282 >> 172.21.15.121:0/115303282 conn(0x7f58d0062190 msgr2=0x7f58d0062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:01.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58d588f700 1 -- 172.21.15.121:0/115303282 shutdown_connections 2023-11-15T03:10:01.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.922+0000 7f58d588f700 1 -- 172.21.15.121:0/115303282 wait complete. 2023-11-15T03:10:01.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.926+0000 7f58d588f700 1 Processor -- start 2023-11-15T03:10:01.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.926+0000 7f58d588f700 1 -- start start 2023-11-15T03:10:01.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.926+0000 7f58d588f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d00c3280 0x7f58d014ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:01.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.926+0000 7f58d588f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d0152140 0x7f58d0158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:01.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.926+0000 7f58d588f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d015be30 0x7f58d0158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:01.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.926+0000 7f58d588f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d014ef70 con 0x7f58d015be30 2023-11-15T03:10:01.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.926+0000 7f58d588f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d014e5f0 con 0x7f58d00c3280 2023-11-15T03:10:01.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.926+0000 7f58d588f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d0159470 con 0x7f58d0152140 2023-11-15T03:10:01.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.926+0000 7f58d508e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d015be30 0x7f58d0158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:01.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.930+0000 7f58d508e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d015be30 0x7f58d0158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40284/0 (socket says 172.21.15.121:40284) 2023-11-15T03:10:01.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.930+0000 7f58d508e700 1 -- 172.21.15.121:0/4272851564 learned_addr learned my addr 172.21.15.121:0/4272851564 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:01.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.930+0000 7f58d488d700 1 --2- 172.21.15.121:0/4272851564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d00c3280 0x7f58d014ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:01.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.930+0000 7f58cffff700 1 --2- 172.21.15.121:0/4272851564 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d0152140 0x7f58d0158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:01.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.930+0000 7f58d508e700 1 -- 172.21.15.121:0/4272851564 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d0152140 msgr2=0x7f58d0158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:01.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.930+0000 7f58d508e700 1 --2- 172.21.15.121:0/4272851564 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d0152140 0x7f58d0158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:01.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.930+0000 7f58d508e700 1 -- 172.21.15.121:0/4272851564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d00c3280 msgr2=0x7f58d014ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:01.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.930+0000 7f58d508e700 1 --2- 172.21.15.121:0/4272851564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d00c3280 0x7f58d014ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:01.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.930+0000 7f58d508e700 1 -- 172.21.15.121:0/4272851564 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c8009a20 con 0x7f58d015be30 2023-11-15T03:10:01.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.930+0000 7f58d508e700 1 --2- 172.21.15.121:0/4272851564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d015be30 0x7f58d0158f80 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 rx=0x7f58c800b610 tx=0x7f58d00c3bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:01.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.930+0000 7f58ccff9700 1 -- 172.21.15.121:0/4272851564 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8012e60 con 0x7f58d015be30 2023-11-15T03:10:01.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.930+0000 7f58ccff9700 1 -- 172.21.15.121:0/4272851564 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c8009440 con 0x7f58d015be30 2023-11-15T03:10:01.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.930+0000 7f58d588f700 1 -- 172.21.15.121:0/4272851564 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d00c70a0 con 0x7f58d015be30 2023-11-15T03:10:01.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.930+0000 7f58ccff9700 1 -- 172.21.15.121:0/4272851564 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c801d860 con 0x7f58d015be30 2023-11-15T03:10:01.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.930+0000 7f58d588f700 1 -- 172.21.15.121:0/4272851564 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d014e300 con 0x7f58d015be30 2023-11-15T03:10:01.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.934+0000 7f58d588f700 1 -- 172.21.15.121:0/4272851564 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d006bcf0 con 0x7f58d015be30 2023-11-15T03:10:01.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.934+0000 7f58ccff9700 1 -- 172.21.15.121:0/4272851564 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f58c801dc50 con 0x7f58d015be30 2023-11-15T03:10:01.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.934+0000 7f58ccff9700 1 --2- 172.21.15.121:0/4272851564 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58bc05a510 0x7f58bc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:01.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.934+0000 7f58ccff9700 1 -- 172.21.15.121:0/4272851564 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9190+0+0 (secure 0 0 0) 0x7f58c8077020 con 0x7f58d015be30 2023-11-15T03:10:01.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.934+0000 7f58d488d700 1 --2- 172.21.15.121:0/4272851564 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58bc05a510 0x7f58bc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:01.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.934+0000 7f58d488d700 1 --2- 172.21.15.121:0/4272851564 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58bc05a510 0x7f58bc05c9d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7f58d015ace0 tx=0x7f58c0009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:01.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:01.942+0000 7f58ccff9700 1 -- 172.21.15.121:0/4272851564 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f58c8049bc0 con 0x7f58d015be30 2023-11-15T03:10:02.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:02.138+0000 7f58d588f700 1 -- 172.21.15.121:0/4272851564 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58d0068a70 con 0x7f58d015be30 2023-11-15T03:10:02.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:02.138+0000 7f58ccff9700 1 -- 172.21.15.121:0/4272851564 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 301 v301) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f58c804d1e0 con 0x7f58d015be30 2023-11-15T03:10:02.148 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:10:02.149 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":301,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":301,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:10:01.326382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10122},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10122":{"gid":10122,"name":"a","rank":0,"incarnation":301,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/240077764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":240077764},{"type":"v1","addr":"172.21.15.121:6836","nonce":240077764}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:10:02.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:02.142+0000 7f58d588f700 1 -- 172.21.15.121:0/4272851564 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58bc05a510 msgr2=0x7f58bc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:02.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:02.142+0000 7f58d588f700 1 --2- 172.21.15.121:0/4272851564 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58bc05a510 0x7f58bc05c9d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7f58d015ace0 tx=0x7f58c0009060).stop 2023-11-15T03:10:02.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:02.142+0000 7f58d588f700 1 -- 172.21.15.121:0/4272851564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d015be30 msgr2=0x7f58d0158f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:02.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:02.142+0000 7f58d588f700 1 --2- 172.21.15.121:0/4272851564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d015be30 0x7f58d0158f80 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 rx=0x7f58c800b610 tx=0x7f58d00c3bb0).stop 2023-11-15T03:10:02.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:02.142+0000 7f58d588f700 1 -- 172.21.15.121:0/4272851564 shutdown_connections 2023-11-15T03:10:02.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:02.142+0000 7f58d588f700 1 --2- 172.21.15.121:0/4272851564 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58bc05a510 0x7f58bc05c9d0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:02.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:02.142+0000 7f58d588f700 1 --2- 172.21.15.121:0/4272851564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d015be30 0x7f58d0158f80 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:02.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:02.142+0000 7f58d588f700 1 --2- 172.21.15.121:0/4272851564 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d0152140 0x7f58d0158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:02.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:02.142+0000 7f58d588f700 1 --2- 172.21.15.121:0/4272851564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d00c3280 0x7f58d014ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:02.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:02.142+0000 7f58d588f700 1 -- 172.21.15.121:0/4272851564 >> 172.21.15.121:0/4272851564 conn(0x7f58d0062190 msgr2=0x7f58d0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:02.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:02.142+0000 7f58d588f700 1 -- 172.21.15.121:0/4272851564 shutdown_connections 2023-11-15T03:10:02.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:02.142+0000 7f58d588f700 1 -- 172.21.15.121:0/4272851564 wait complete. 2023-11-15T03:10:02.156 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 301 2023-11-15T03:10:02.173 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 301, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:10:01.326382+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10122}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10122': {'gid': 10122, 'name': 'a', 'rank': 0, 'incarnation': 301, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.121:6836/240077764', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 240077764}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 240077764}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:10:02.173 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10122:up:replay 2023-11-15T03:10:02.174 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:10:07.578 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:10:07.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.729+0000 7f2cc189b700 1 Processor -- start 2023-11-15T03:10:07.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.729+0000 7f2cc189b700 1 -- start start 2023-11-15T03:10:07.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.733+0000 7f2cc189b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2cbc151920 0x7f2cbc151d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:07.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.733+0000 7f2cc189b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2cbc063fe0 0x7f2cbc064440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:07.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.733+0000 7f2cc189b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cbc068430 con 0x7f2cbc0c4720 2023-11-15T03:10:07.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.733+0000 7f2cc189b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc064930 con 0x7f2cbc151920 2023-11-15T03:10:07.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.733+0000 7f2cc189b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc0a5b10 con 0x7f2cbc063fe0 2023-11-15T03:10:07.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.733+0000 7f2cbaffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2cbc151920 0x7f2cbc151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:07.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.733+0000 7f2cb27fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2cbc063fe0 0x7f2cbc064440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:07.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.733+0000 7f2cbaffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2cbc151920 0x7f2cbc151d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58954/0 (socket says 172.21.15.121:58954) 2023-11-15T03:10:07.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.733+0000 7f2cbaffd700 1 -- 172.21.15.121:0/2493987459 learned_addr learned my addr 172.21.15.121:0/2493987459 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:07.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.733+0000 7f2cbaffd700 1 -- 172.21.15.121:0/2493987459 >> v2:172.21.15.190:3300/0 conn(0x7f2cbc063fe0 msgr2=0x7f2cbc064440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:07.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.733+0000 7f2cbaffd700 1 --2- 172.21.15.121:0/2493987459 >> v2:172.21.15.190:3300/0 conn(0x7f2cbc063fe0 0x7f2cbc064440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:07.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.733+0000 7f2cbaffd700 1 -- 172.21.15.121:0/2493987459 >> v1:172.21.15.190:6789/0 conn(0x7f2cbc0c4720 legacy=0x7f2cbc062ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:07.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.733+0000 7f2cbaffd700 1 -- 172.21.15.121:0/2493987459 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc064b50 con 0x7f2cbc151920 2023-11-15T03:10:07.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.737+0000 7f2cbaffd700 1 --2- 172.21.15.121:0/2493987459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cbc151920 0x7f2cbc151d00 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 rx=0x7f2cbc0a7680 tx=0x7f2cac009840).ready entity=mon.0 client_cookie=a12b5f7e4869c6ee server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:07.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.737+0000 7f2cba7fc700 1 -- 172.21.15.121:0/2493987459 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00e070 con 0x7f2cbc151920 2023-11-15T03:10:07.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.737+0000 7f2cba7fc700 1 -- 172.21.15.121:0/2493987459 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cac00ba20 con 0x7f2cbc151920 2023-11-15T03:10:07.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.737+0000 7f2cba7fc700 1 -- 172.21.15.121:0/2493987459 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00bda0 con 0x7f2cbc151920 2023-11-15T03:10:07.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.737+0000 7f2cc189b700 1 -- 172.21.15.121:0/2493987459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cbc151920 msgr2=0x7f2cbc151d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:07.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.737+0000 7f2cc189b700 1 --2- 172.21.15.121:0/2493987459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cbc151920 0x7f2cbc151d00 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 rx=0x7f2cbc0a7680 tx=0x7f2cac009840).stop 2023-11-15T03:10:07.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.737+0000 7f2cc189b700 1 -- 172.21.15.121:0/2493987459 shutdown_connections 2023-11-15T03:10:07.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.737+0000 7f2cc189b700 1 --2- 172.21.15.121:0/2493987459 >> v2:172.21.15.190:3300/0 conn(0x7f2cbc063fe0 0x7f2cbc064440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:07.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.737+0000 7f2cc189b700 1 --2- 172.21.15.121:0/2493987459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cbc151920 0x7f2cbc151d00 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:07.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.737+0000 7f2cc189b700 1 -- 172.21.15.121:0/2493987459 >> 172.21.15.121:0/2493987459 conn(0x7f2cbc069ac0 msgr2=0x7f2cbc090a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:07.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.737+0000 7f2cc189b700 1 -- 172.21.15.121:0/2493987459 shutdown_connections 2023-11-15T03:10:07.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.737+0000 7f2cc189b700 1 -- 172.21.15.121:0/2493987459 wait complete. 2023-11-15T03:10:07.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cc189b700 1 Processor -- start 2023-11-15T03:10:07.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cc189b700 1 -- start start 2023-11-15T03:10:07.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cc189b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cbc063fe0 0x7f2cbc14b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:07.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cc189b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cbc0c4720 0x7f2cbc0c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:07.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cc189b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cbc151920 0x7f2cbc0a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:07.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cc189b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0aa170 con 0x7f2cbc151920 2023-11-15T03:10:07.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cc189b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0a9560 con 0x7f2cbc063fe0 2023-11-15T03:10:07.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cc189b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0a9840 con 0x7f2cbc0c4720 2023-11-15T03:10:07.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cbbfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cbc151920 0x7f2cbc0a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:07.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cbbfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cbc151920 0x7f2cbc0a5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58956/0 (socket says 172.21.15.121:58956) 2023-11-15T03:10:07.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cbbfff700 1 -- 172.21.15.121:0/2569268079 learned_addr learned my addr 172.21.15.121:0/2569268079 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:07.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cbaffd700 1 --2- 172.21.15.121:0/2569268079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cbc063fe0 0x7f2cbc14b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:07.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cb27fc700 1 --2- 172.21.15.121:0/2569268079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cbc0c4720 0x7f2cbc0c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:07.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cbbfff700 1 -- 172.21.15.121:0/2569268079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cbc0c4720 msgr2=0x7f2cbc0c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:07.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cbbfff700 1 --2- 172.21.15.121:0/2569268079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cbc0c4720 0x7f2cbc0c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:07.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cbbfff700 1 -- 172.21.15.121:0/2569268079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cbc063fe0 msgr2=0x7f2cbc14b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:07.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cbbfff700 1 --2- 172.21.15.121:0/2569268079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cbc063fe0 0x7f2cbc14b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:07.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cbbfff700 1 -- 172.21.15.121:0/2569268079 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac009ae0 con 0x7f2cbc151920 2023-11-15T03:10:07.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cbaffd700 1 --2- 172.21.15.121:0/2569268079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cbc063fe0 0x7f2cbc14b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:07.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.741+0000 7f2cbbfff700 1 --2- 172.21.15.121:0/2569268079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cbc151920 0x7f2cbc0a5e30 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 rx=0x7f2cb400c440 tx=0x7f2cb4007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:07.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.745+0000 7f2cb37fe700 1 -- 172.21.15.121:0/2569268079 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb400cea0 con 0x7f2cbc151920 2023-11-15T03:10:07.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.745+0000 7f2cb37fe700 1 -- 172.21.15.121:0/2569268079 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb4001e90 con 0x7f2cbc151920 2023-11-15T03:10:07.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.745+0000 7f2cc189b700 1 -- 172.21.15.121:0/2569268079 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc0a9b20 con 0x7f2cbc151920 2023-11-15T03:10:07.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.745+0000 7f2cb37fe700 1 -- 172.21.15.121:0/2569268079 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb400f680 con 0x7f2cbc151920 2023-11-15T03:10:07.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.745+0000 7f2cc189b700 1 -- 172.21.15.121:0/2569268079 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cbc14be50 con 0x7f2cbc151920 2023-11-15T03:10:07.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.745+0000 7f2cc189b700 1 -- 172.21.15.121:0/2569268079 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cbc06b740 con 0x7f2cbc151920 2023-11-15T03:10:07.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.745+0000 7f2cb37fe700 1 -- 172.21.15.121:0/2569268079 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2cb400fa70 con 0x7f2cbc151920 2023-11-15T03:10:07.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.749+0000 7f2cb37fe700 1 --2- 172.21.15.121:0/2569268079 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c9c05a510 0x7f2c9c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:07.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.749+0000 7f2cb37fe700 1 -- 172.21.15.121:0/2569268079 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9190+0+0 (secure 0 0 0) 0x7f2cb40747f0 con 0x7f2cbc151920 2023-11-15T03:10:07.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.749+0000 7f2cbaffd700 1 --2- 172.21.15.121:0/2569268079 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c9c05a510 0x7f2c9c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:07.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.749+0000 7f2cbaffd700 1 --2- 172.21.15.121:0/2569268079 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c9c05a510 0x7f2c9c05c9d0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f2cac00c010 tx=0x7f2cac009840).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:07.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.753+0000 7f2cb37fe700 1 -- 172.21.15.121:0/2569268079 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2cbc06b740 con 0x7f2cbc151920 2023-11-15T03:10:07.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.953+0000 7f2cc189b700 1 -- 172.21.15.121:0/2569268079 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2cbc06b740 con 0x7f2cbc151920 2023-11-15T03:10:07.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.957+0000 7f2cb37fe700 1 -- 172.21.15.121:0/2569268079 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 304 v304) v1 ==== 95+0+4505 (secure 0 0 0) 0x7f2cbc06b740 con 0x7f2cbc151920 2023-11-15T03:10:07.967 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:10:07.967 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":304,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":304,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:10:04.353784+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10122},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10122":{"gid":10122,"name":"a","rank":0,"incarnation":301,"state":"up:active","state_seq":4,"addr":"172.21.15.121:6836/240077764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":240077764},{"type":"v1","addr":"172.21.15.121:6836","nonce":240077764}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:10:07.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.957+0000 7f2cc189b700 1 -- 172.21.15.121:0/2569268079 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c9c05a510 msgr2=0x7f2c9c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:07.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.957+0000 7f2cc189b700 1 --2- 172.21.15.121:0/2569268079 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c9c05a510 0x7f2c9c05c9d0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f2cac00c010 tx=0x7f2cac009840).stop 2023-11-15T03:10:07.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.957+0000 7f2cc189b700 1 -- 172.21.15.121:0/2569268079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cbc151920 msgr2=0x7f2cbc0a5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:07.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.957+0000 7f2cc189b700 1 --2- 172.21.15.121:0/2569268079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cbc151920 0x7f2cbc0a5e30 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 rx=0x7f2cb400c440 tx=0x7f2cb4007d70).stop 2023-11-15T03:10:07.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.957+0000 7f2cc189b700 1 -- 172.21.15.121:0/2569268079 shutdown_connections 2023-11-15T03:10:07.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.957+0000 7f2cc189b700 1 --2- 172.21.15.121:0/2569268079 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2c9c05a510 0x7f2c9c05c9d0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:07.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.957+0000 7f2cc189b700 1 --2- 172.21.15.121:0/2569268079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cbc151920 0x7f2cbc0a5e30 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:07.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.957+0000 7f2cc189b700 1 --2- 172.21.15.121:0/2569268079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cbc0c4720 0x7f2cbc0c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:07.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.957+0000 7f2cc189b700 1 --2- 172.21.15.121:0/2569268079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cbc063fe0 0x7f2cbc14b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:07.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.957+0000 7f2cc189b700 1 -- 172.21.15.121:0/2569268079 >> 172.21.15.121:0/2569268079 conn(0x7f2cbc069ac0 msgr2=0x7f2cbc150ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:07.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.957+0000 7f2cc189b700 1 -- 172.21.15.121:0/2569268079 shutdown_connections 2023-11-15T03:10:07.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:07.957+0000 7f2cc189b700 1 -- 172.21.15.121:0/2569268079 wait complete. 2023-11-15T03:10:07.975 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 304 2023-11-15T03:10:07.988 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 304, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:10:04.353784+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10122}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10122': {'gid': 10122, 'name': 'a', 'rank': 0, 'incarnation': 301, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.121:6836/240077764', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 240077764}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 240077764}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:10:07.988 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T03:10:07.989 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-15T03:10:08.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.141+0000 7f09b2c9d700 1 Processor -- start 2023-11-15T03:10:08.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.141+0000 7f09b2c9d700 1 -- start start 2023-11-15T03:10:08.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b2c9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f09ac0ab0d0 0x7f09ac0ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b2c9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac0add40 0x7f09ac0ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b2c9d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ac066c90 con 0x7f09ac0ad1b0 2023-11-15T03:10:08.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b2c9d700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09ac065e20 con 0x7f09ac0ab0d0 2023-11-15T03:10:08.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b2c9d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac066080 con 0x7f09ac0add40 2023-11-15T03:10:08.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b1c9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f09ac0ab0d0 0x7f09ac0ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b1c9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f09ac0ab0d0 0x7f09ac0ab4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58968/0 (socket says 172.21.15.121:58968) 2023-11-15T03:10:08.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b1c9b700 1 -- 172.21.15.121:0/607509432 learned_addr learned my addr 172.21.15.121:0/607509432 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:08.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b249c700 1 --2- 172.21.15.121:0/607509432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac0add40 0x7f09ac0ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b1c9b700 1 -- 172.21.15.121:0/607509432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac0add40 msgr2=0x7f09ac0ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:08.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b1c9b700 1 --2- 172.21.15.121:0/607509432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac0add40 0x7f09ac0ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b1c9b700 1 -- 172.21.15.121:0/607509432 >> v1:172.21.15.190:6789/0 conn(0x7f09ac0ad1b0 legacy=0x7f09ac065710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:08.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b1c9b700 1 -- 172.21.15.121:0/607509432 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac066360 con 0x7f09ac0ab0d0 2023-11-15T03:10:08.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b249c700 1 --2- 172.21.15.121:0/607509432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac0add40 0x7f09ac0ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:10:08.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b1c9b700 1 --2- 172.21.15.121:0/607509432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09ac0ab0d0 0x7f09ac0ab4b0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 rx=0x7f09a8009810 tx=0x7f09a8009060).ready entity=mon.0 client_cookie=879ccd9ff08ea6e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:08.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b0c99700 1 -- 172.21.15.121:0/607509432 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a8005880 con 0x7f09ac0ab0d0 2023-11-15T03:10:08.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b0c99700 1 -- 172.21.15.121:0/607509432 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a80059e0 con 0x7f09ac0ab0d0 2023-11-15T03:10:08.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b0c99700 1 -- 172.21.15.121:0/607509432 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a8005d60 con 0x7f09ac0ab0d0 2023-11-15T03:10:08.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b2c9d700 1 -- 172.21.15.121:0/607509432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09ac0ab0d0 msgr2=0x7f09ac0ab4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:08.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b2c9d700 1 --2- 172.21.15.121:0/607509432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09ac0ab0d0 0x7f09ac0ab4b0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 rx=0x7f09a8009810 tx=0x7f09a8009060).stop 2023-11-15T03:10:08.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b2c9d700 1 -- 172.21.15.121:0/607509432 shutdown_connections 2023-11-15T03:10:08.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b2c9d700 1 --2- 172.21.15.121:0/607509432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac0add40 0x7f09ac0ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b2c9d700 1 --2- 172.21.15.121:0/607509432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09ac0ab0d0 0x7f09ac0ab4b0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b2c9d700 1 -- 172.21.15.121:0/607509432 >> 172.21.15.121:0/607509432 conn(0x7f09ac067a60 msgr2=0x7f09ac067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:08.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b2c9d700 1 -- 172.21.15.121:0/607509432 shutdown_connections 2023-11-15T03:10:08.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.145+0000 7f09b2c9d700 1 -- 172.21.15.121:0/607509432 wait complete. 2023-11-15T03:10:08.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.149+0000 7f09b2c9d700 1 Processor -- start 2023-11-15T03:10:08.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.149+0000 7f09b2c9d700 1 -- start start 2023-11-15T03:10:08.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.149+0000 7f09b2c9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac0ab0d0 0x7f09ac0a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.149+0000 7f09b2c9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09ac0ad1b0 0x7f09ac143100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.149+0000 7f09b2c9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09ac0add40 0x7f09ac0a9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.149+0000 7f09b2c9d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac0a93f0 con 0x7f09ac0ad1b0 2023-11-15T03:10:08.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.149+0000 7f09b2c9d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac0a87e0 con 0x7f09ac0add40 2023-11-15T03:10:08.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.149+0000 7f09b2c9d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac0a8ac0 con 0x7f09ac0ab0d0 2023-11-15T03:10:08.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.149+0000 7f09b149a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09ac0ad1b0 0x7f09ac143100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.149+0000 7f09b149a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09ac0ad1b0 0x7f09ac143100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58974/0 (socket says 172.21.15.121:58974) 2023-11-15T03:10:08.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.149+0000 7f09b149a700 1 -- 172.21.15.121:0/989568413 learned_addr learned my addr 172.21.15.121:0/989568413 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:08.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.149+0000 7f09b249c700 1 --2- 172.21.15.121:0/989568413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09ac0add40 0x7f09ac0a9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.153+0000 7f09b149a700 1 -- 172.21.15.121:0/989568413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac0ab0d0 msgr2=0x7f09ac0a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:08.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.149+0000 7f09b1c9b700 1 --2- 172.21.15.121:0/989568413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac0ab0d0 0x7f09ac0a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.153+0000 7f09b149a700 1 --2- 172.21.15.121:0/989568413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac0ab0d0 0x7f09ac0a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.153+0000 7f09b149a700 1 -- 172.21.15.121:0/989568413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09ac0add40 msgr2=0x7f09ac0a9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:08.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.153+0000 7f09b149a700 1 --2- 172.21.15.121:0/989568413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09ac0add40 0x7f09ac0a9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.153+0000 7f09b149a700 1 -- 172.21.15.121:0/989568413 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a8009300 con 0x7f09ac0ad1b0 2023-11-15T03:10:08.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.153+0000 7f09b249c700 1 --2- 172.21.15.121:0/989568413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09ac0add40 0x7f09ac0a9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:10:08.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.153+0000 7f09b1c9b700 1 --2- 172.21.15.121:0/989568413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac0ab0d0 0x7f09ac0a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:10:08.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.153+0000 7f09b149a700 1 --2- 172.21.15.121:0/989568413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09ac0ad1b0 0x7f09ac143100 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 rx=0x7f099c00c4c0 tx=0x7f099c007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:08.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.153+0000 7f09a1ffb700 1 -- 172.21.15.121:0/989568413 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c007020 con 0x7f09ac0ad1b0 2023-11-15T03:10:08.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.153+0000 7f09a1ffb700 1 -- 172.21.15.121:0/989568413 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099c007180 con 0x7f09ac0ad1b0 2023-11-15T03:10:08.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.153+0000 7f09b2c9d700 1 -- 172.21.15.121:0/989568413 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ac0a8da0 con 0x7f09ac0ad1b0 2023-11-15T03:10:08.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.153+0000 7f09a1ffb700 1 -- 172.21.15.121:0/989568413 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c007520 con 0x7f09ac0ad1b0 2023-11-15T03:10:08.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.153+0000 7f09b2c9d700 1 -- 172.21.15.121:0/989568413 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ac1435f0 con 0x7f09ac0ad1b0 2023-11-15T03:10:08.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.153+0000 7f09a1ffb700 1 -- 172.21.15.121:0/989568413 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f099c0068b0 con 0x7f09ac0ad1b0 2023-11-15T03:10:08.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.157+0000 7f09a1ffb700 1 --2- 172.21.15.121:0/989568413 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f099005a510 0x7f099005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.157+0000 7f09a1ffb700 1 -- 172.21.15.121:0/989568413 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9190+0+0 (secure 0 0 0) 0x7f099c074830 con 0x7f09ac0ad1b0 2023-11-15T03:10:08.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.157+0000 7f09b1c9b700 1 --2- 172.21.15.121:0/989568413 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f099005a510 0x7f099005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.157+0000 7f09b1c9b700 1 --2- 172.21.15.121:0/989568413 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f099005a510 0x7f099005c9d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7f09a800b400 tx=0x7f09a8016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:08.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.157+0000 7f09be2a1740 1 Processor -- start 2023-11-15T03:10:08.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.157+0000 7f09be2a1740 1 -- start start 2023-11-15T03:10:08.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09be2a1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2023ad0 0x204a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09be2a1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023eb0 0x2070140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09be2a1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2070630 0x2070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09be2a1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fa4410 con 0x2023eb0 2023-11-15T03:10:08.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09be2a1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fa4550 con 0x2023ad0 2023-11-15T03:10:08.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09be2a1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1fa4a00 con 0x2070630 2023-11-15T03:10:08.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09b149a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023eb0 0x2070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09b149a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023eb0 0x2070140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58988/0 (socket says 172.21.15.121:58988) 2023-11-15T03:10:08.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09b149a700 1 -- 172.21.15.121:0/566058134 learned_addr learned my addr 172.21.15.121:0/566058134 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:08.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09b1c9b700 1 --2- 172.21.15.121:0/566058134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2023ad0 0x204a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09b249c700 1 --2- 172.21.15.121:0/566058134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2070630 0x2070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09b149a700 1 -- 172.21.15.121:0/566058134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2070630 msgr2=0x2070ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:08.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09b149a700 1 --2- 172.21.15.121:0/566058134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2070630 0x2070ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09b149a700 1 -- 172.21.15.121:0/566058134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2023ad0 msgr2=0x204a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:08.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09b149a700 1 --2- 172.21.15.121:0/566058134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2023ad0 0x204a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09b149a700 1 -- 172.21.15.121:0/566058134 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1f9d730 con 0x2023eb0 2023-11-15T03:10:08.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09b249c700 1 --2- 172.21.15.121:0/566058134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2070630 0x2070ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:10:08.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09b149a700 1 --2- 172.21.15.121:0/566058134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023eb0 0x2070140 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 rx=0x7f099c04a970 tx=0x7f099c00f000).ready entity=mon.0 client_cookie=dad1e3b8a858ca server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:08.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f0996ffd700 1 -- 172.21.15.121:0/566058134 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c0493b0 con 0x2023eb0 2023-11-15T03:10:08.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f0996ffd700 1 -- 172.21.15.121:0/566058134 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099c047720 con 0x2023eb0 2023-11-15T03:10:08.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f0996ffd700 1 -- 172.21.15.121:0/566058134 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c04a710 con 0x2023eb0 2023-11-15T03:10:08.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09be2a1740 1 -- 172.21.15.121:0/566058134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023eb0 msgr2=0x2070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:08.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09be2a1740 1 --2- 172.21.15.121:0/566058134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023eb0 0x2070140 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 rx=0x7f099c04a970 tx=0x7f099c00f000).stop 2023-11-15T03:10:08.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09be2a1740 1 -- 172.21.15.121:0/566058134 shutdown_connections 2023-11-15T03:10:08.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09be2a1740 1 --2- 172.21.15.121:0/566058134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2070630 0x2070ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09be2a1740 1 --2- 172.21.15.121:0/566058134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023eb0 0x2070140 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09be2a1740 1 --2- 172.21.15.121:0/566058134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2023ad0 0x204a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.161+0000 7f09be2a1740 1 -- 172.21.15.121:0/566058134 >> 172.21.15.121:0/566058134 conn(0x1e02550 msgr2=0x20227e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:08.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09be2a1740 1 -- 172.21.15.121:0/566058134 shutdown_connections 2023-11-15T03:10:08.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09be2a1740 1 -- 172.21.15.121:0/566058134 wait complete. 2023-11-15T03:10:08.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09be2a1740 1 -- start start 2023-11-15T03:10:08.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09be2a1740 1 Processor -- start 2023-11-15T03:10:08.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09be2a1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023ad0 0x204b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09be2a1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2023eb0 0x20638e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09be2a1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2070630 0x204a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09be2a1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f6f650 con 0x2023ad0 2023-11-15T03:10:08.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09be2a1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x204bc00 con 0x2070630 2023-11-15T03:10:08.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09be2a1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x204fd70 con 0x2023eb0 2023-11-15T03:10:08.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09be2a1740 10 client.0 ll_register_callbacks cb 0x1fdc020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:10:08.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09b1c9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023ad0 0x204b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09b1c9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023ad0 0x204b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58996/0 (socket says 172.21.15.121:58996) 2023-11-15T03:10:08.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09b1c9b700 1 -- 172.21.15.121:0/1931275219 learned_addr learned my addr 172.21.15.121:0/1931275219 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:08.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09b249c700 1 --2- 172.21.15.121:0/1931275219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2070630 0x204a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09b149a700 1 --2- 172.21.15.121:0/1931275219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2023eb0 0x20638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09b1c9b700 1 -- 172.21.15.121:0/1931275219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2023eb0 msgr2=0x20638e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:08.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09b1c9b700 1 --2- 172.21.15.121:0/1931275219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2023eb0 0x20638e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09b1c9b700 1 -- 172.21.15.121:0/1931275219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2070630 msgr2=0x204a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:08.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09b1c9b700 1 --2- 172.21.15.121:0/1931275219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2070630 0x204a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.165+0000 7f09b1c9b700 1 -- 172.21.15.121:0/1931275219 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ac1435f0 con 0x2023ad0 2023-11-15T03:10:08.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f09b1c9b700 1 --2- 172.21.15.121:0/1931275219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023ad0 0x204b710 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 rx=0x7f09a8000d50 tx=0x7f09a801c000).ready entity=mon.0 client_cookie=b2472dbdf5014ad server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:08.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f098bfff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:10:08.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f098bfff700 1 -- 172.21.15.121:0/1931275219 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a800be80 con 0x2023ad0 2023-11-15T03:10:08.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f098bfff700 1 -- 172.21.15.121:0/1931275219 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 9190+0+0 (secure 0 0 0) 0x7f09a8021070 con 0x2023ad0 2023-11-15T03:10:08.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f09be2a1740 1 -- 172.21.15.121:0/1931275219 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1f9a1a0 con 0x2023ad0 2023-11-15T03:10:08.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f098bfff700 1 -- 172.21.15.121:0/1931275219 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=304) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09a8011860 con 0x2023ad0 2023-11-15T03:10:08.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f09be2a1740 10 client.10179 fetch_fsmap learned FSMap version 304 2023-11-15T03:10:08.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f09be2a1740 1 -- 172.21.15.121:0/1931275219 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=304,monmap=2+}) v3 -- 0x1f98960 con 0x2023ad0 2023-11-15T03:10:08.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f098bfff700 1 -- 172.21.15.121:0/1931275219 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1f9a1a0 con 0x2023ad0 2023-11-15T03:10:08.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f098bfff700 1 -- 172.21.15.121:0/1931275219 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 304) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f09a80119c0 con 0x2023ad0 2023-11-15T03:10:08.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f09be2a1740 10 client.10179 fetch_fsmap finished waiting for FSMap version 304 2023-11-15T03:10:08.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f09be2a1740 10 client.10179 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:10:08.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f09be2a1740 20 client.10179 populate_metadata read hostname 'smithi121' 2023-11-15T03:10:08.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f09be2a1740 1 --2- 172.21.15.121:0/1931275219 >> [v2:172.21.15.121:6834/240077764,v1:172.21.15.121:6836/240077764] conn(0x2043b80 0x2043f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f09be2a1740 4 client.10179 mds_command: new command op to 10122 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:10:08.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f09be2a1740 1 -- 172.21.15.121:0/1931275219 --> [v2:172.21.15.121:6834/240077764,v1:172.21.15.121:6836/240077764] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1f906f0 con 0x2043b80 2023-11-15T03:10:08.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f09b149a700 1 --2- 172.21.15.121:0/1931275219 >> [v2:172.21.15.121:6834/240077764,v1:172.21.15.121:6836/240077764] conn(0x2043b80 0x2043f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f098bfff700 10 client.10179 ms_handle_connect on v2:172.21.15.121:6834/240077764 2023-11-15T03:10:08.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f09b149a700 1 --2- 172.21.15.121:0/1931275219 >> [v2:172.21.15.121:6834/240077764,v1:172.21.15.121:6836/240077764] conn(0x2043b80 0x2043f60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=492271ac44213155 server_cookie=a6f35effcbf8eccb in_seq=0 out_seq=0 2023-11-15T03:10:08.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f098bfff700 1 -- 172.21.15.121:0/1931275219 <== mds.0 v2:172.21.15.121:6834/240077764 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x1f906f0 con 0x2043b80 2023-11-15T03:10:08.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f098bfff700 10 client.10179 handle_command_reply: tid=0 2023-11-15T03:10:08.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f09be2a1740 1 client.10179 shutdown 2023-11-15T03:10:08.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f09be2a1740 1 -- 172.21.15.121:0/1931275219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023ad0 msgr2=0x204b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:08.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.169+0000 7f09be2a1740 1 --2- 172.21.15.121:0/1931275219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023ad0 0x204b710 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 rx=0x7f09a8000d50 tx=0x7f09a801c000).stop 2023-11-15T03:10:08.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.173+0000 7f09be2a1740 1 -- 172.21.15.121:0/1931275219 shutdown_connections 2023-11-15T03:10:08.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.173+0000 7f09be2a1740 1 --2- 172.21.15.121:0/1931275219 >> [v2:172.21.15.121:6834/240077764,v1:172.21.15.121:6836/240077764] conn(0x2043b80 0x2043f60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.173+0000 7f09be2a1740 1 --2- 172.21.15.121:0/1931275219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2070630 0x204a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.173+0000 7f09be2a1740 1 --2- 172.21.15.121:0/1931275219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2023eb0 0x20638e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.173+0000 7f098bfff700 0 client.10179 ms_handle_reset on v2:172.21.15.121:6834/240077764 2023-11-15T03:10:08.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.173+0000 7f09be2a1740 1 --2- 172.21.15.121:0/1931275219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023ad0 0x204b710 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.173+0000 7f09be2a1740 1 -- 172.21.15.121:0/1931275219 >> 172.21.15.121:0/1931275219 conn(0x1e02550 msgr2=0x206b540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:08.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.173+0000 7f09be2a1740 1 -- 172.21.15.121:0/1931275219 shutdown_connections 2023-11-15T03:10:08.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.173+0000 7f09be2a1740 1 -- 172.21.15.121:0/1931275219 wait complete. 2023-11-15T03:10:08.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.173+0000 7f09be2a1740 20 client.10179 trim_cache size 0 max 16384 2023-11-15T03:10:08.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09be2a1740 1 Processor -- start 2023-11-15T03:10:08.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09be2a1740 1 -- start start 2023-11-15T03:10:08.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09be2a1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2023ad0 0x2043f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09be2a1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2023eb0 0x204a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09be2a1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2070630 0x2024880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09be2a1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f79c60 con 0x2070630 2023-11-15T03:10:08.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09be2a1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x201f190 con 0x2023ad0 2023-11-15T03:10:08.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09be2a1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1fffe20 con 0x2023eb0 2023-11-15T03:10:08.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09b249c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2070630 0x2024880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09b249c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2070630 0x2024880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59004/0 (socket says 172.21.15.121:59004) 2023-11-15T03:10:08.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09b249c700 1 -- 172.21.15.121:0/3411423200 learned_addr learned my addr 172.21.15.121:0/3411423200 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:08.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09b1c9b700 1 --2- 172.21.15.121:0/3411423200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2023ad0 0x2043f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09b149a700 1 --2- 172.21.15.121:0/3411423200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2023eb0 0x204a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09b249c700 1 -- 172.21.15.121:0/3411423200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2023eb0 msgr2=0x204a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:08.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09b249c700 1 --2- 172.21.15.121:0/3411423200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2023eb0 0x204a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09b249c700 1 -- 172.21.15.121:0/3411423200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2023ad0 msgr2=0x2043f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:08.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09b249c700 1 --2- 172.21.15.121:0/3411423200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2023ad0 0x2043f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09b249c700 1 -- 172.21.15.121:0/3411423200 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac0aa0d0 con 0x2070630 2023-11-15T03:10:08.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09b1c9b700 1 --2- 172.21.15.121:0/3411423200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2023ad0 0x2043f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:10:08.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09b149a700 1 --2- 172.21.15.121:0/3411423200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2023eb0 0x204a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:10:08.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09b249c700 1 --2- 172.21.15.121:0/3411423200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2070630 0x2024880 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 rx=0x7f09a400a740 tx=0x7f09a4007bc0).ready entity=mon.0 client_cookie=6d31adcafc4cd5ee server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:08.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09977fe700 1 -- 172.21.15.121:0/3411423200 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a40095b0 con 0x2070630 2023-11-15T03:10:08.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09977fe700 1 -- 172.21.15.121:0/3411423200 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a4012040 con 0x2070630 2023-11-15T03:10:08.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.193+0000 7f09977fe700 1 -- 172.21.15.121:0/3411423200 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4009920 con 0x2070630 2023-11-15T03:10:08.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09be2a1740 1 -- 172.21.15.121:0/3411423200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2070630 msgr2=0x2024880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:08.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09be2a1740 1 --2- 172.21.15.121:0/3411423200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2070630 0x2024880 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 rx=0x7f09a400a740 tx=0x7f09a4007bc0).stop 2023-11-15T03:10:08.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09be2a1740 1 -- 172.21.15.121:0/3411423200 shutdown_connections 2023-11-15T03:10:08.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09be2a1740 1 --2- 172.21.15.121:0/3411423200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2070630 0x2024880 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09be2a1740 1 --2- 172.21.15.121:0/3411423200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2023eb0 0x204a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09be2a1740 1 --2- 172.21.15.121:0/3411423200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2023ad0 0x2043f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09be2a1740 1 -- 172.21.15.121:0/3411423200 >> 172.21.15.121:0/3411423200 conn(0x1e02550 msgr2=0x20227e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:08.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09be2a1740 1 -- 172.21.15.121:0/3411423200 shutdown_connections 2023-11-15T03:10:08.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09be2a1740 1 -- 172.21.15.121:0/3411423200 wait complete. 2023-11-15T03:10:08.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09be2a1740 1 -- start start 2023-11-15T03:10:08.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09be2a1740 1 Processor -- start 2023-11-15T03:10:08.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09be2a1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023ad0 0x20cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09be2a1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2023eb0 0x204de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09be2a1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2070630 0x204e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09be2a1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x204ce60 con 0x2023ad0 2023-11-15T03:10:08.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09be2a1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x206e440 con 0x2070630 2023-11-15T03:10:08.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09be2a1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2046b30 con 0x2023eb0 2023-11-15T03:10:08.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09be2a1740 10 client.0 ll_register_callbacks cb 0x7f09a8006620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:10:08.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09b1c9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023ad0 0x20cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09b1c9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023ad0 0x20cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59010/0 (socket says 172.21.15.121:59010) 2023-11-15T03:10:08.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09b1c9b700 1 -- 172.21.15.121:0/3084817929 learned_addr learned my addr 172.21.15.121:0/3084817929 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:08.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09b149a700 1 --2- 172.21.15.121:0/3084817929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2023eb0 0x204de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.197+0000 7f09b249c700 1 --2- 172.21.15.121:0/3084817929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2070630 0x204e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f09b1c9b700 1 -- 172.21.15.121:0/3084817929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2023eb0 msgr2=0x204de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:08.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f09b1c9b700 1 --2- 172.21.15.121:0/3084817929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2023eb0 0x204de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f09b1c9b700 1 -- 172.21.15.121:0/3084817929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2070630 msgr2=0x204e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:08.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f09b1c9b700 1 --2- 172.21.15.121:0/3084817929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2070630 0x204e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f09b1c9b700 1 -- 172.21.15.121:0/3084817929 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1f98960 con 0x2023ad0 2023-11-15T03:10:08.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f09b249c700 1 --2- 172.21.15.121:0/3084817929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2070630 0x204e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:08.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f09b1c9b700 1 --2- 172.21.15.121:0/3084817929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023ad0 0x20cc180 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 rx=0x7f09a8000d00 tx=0x7f09a8026000).ready entity=mon.0 client_cookie=fd7fb3a91fdd81a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:08.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f098bfff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:10:08.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f098bfff700 1 -- 172.21.15.121:0/3084817929 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a800be80 con 0x2023ad0 2023-11-15T03:10:08.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f098bfff700 1 -- 172.21.15.121:0/3084817929 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 9190+0+0 (secure 0 0 0) 0x7f09a8021040 con 0x2023ad0 2023-11-15T03:10:08.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f09be2a1740 1 -- 172.21.15.121:0/3084817929 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f099c047720 con 0x2023ad0 2023-11-15T03:10:08.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f098bfff700 1 -- 172.21.15.121:0/3084817929 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=304) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09a8027440 con 0x2023ad0 2023-11-15T03:10:08.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f09be2a1740 10 client.10185 fetch_fsmap learned FSMap version 304 2023-11-15T03:10:08.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f09be2a1740 1 -- 172.21.15.121:0/3084817929 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=304,monmap=2+}) v3 -- 0x1f9d730 con 0x2023ad0 2023-11-15T03:10:08.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f098bfff700 1 -- 172.21.15.121:0/3084817929 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a8041020 con 0x2023ad0 2023-11-15T03:10:08.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f098bfff700 1 -- 172.21.15.121:0/3084817929 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 304) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f09a802f020 con 0x2023ad0 2023-11-15T03:10:08.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f09be2a1740 10 client.10185 fetch_fsmap finished waiting for FSMap version 304 2023-11-15T03:10:08.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f09be2a1740 10 client.10185 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:10:08.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f09be2a1740 20 client.10185 populate_metadata read hostname 'smithi121' 2023-11-15T03:10:08.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f09be2a1740 1 --2- 172.21.15.121:0/3084817929 >> [v2:172.21.15.121:6834/240077764,v1:172.21.15.121:6836/240077764] conn(0x2078cb0 0x2079090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f09be2a1740 4 client.10185 mds_command: new command op to 10122 tid=0[{"prefix": "status"}] 2023-11-15T03:10:08.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f09be2a1740 1 -- 172.21.15.121:0/3084817929 --> [v2:172.21.15.121:6834/240077764,v1:172.21.15.121:6836/240077764] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1e594e0 con 0x2078cb0 2023-11-15T03:10:08.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f09b149a700 1 --2- 172.21.15.121:0/3084817929 >> [v2:172.21.15.121:6834/240077764,v1:172.21.15.121:6836/240077764] conn(0x2078cb0 0x2079090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f098bfff700 10 client.10185 ms_handle_connect on v2:172.21.15.121:6834/240077764 2023-11-15T03:10:08.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f09b149a700 1 --2- 172.21.15.121:0/3084817929 >> [v2:172.21.15.121:6834/240077764,v1:172.21.15.121:6836/240077764] conn(0x2078cb0 0x2079090 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=84bac94b73022604 server_cookie=1c87c9e9bb964cc in_seq=0 out_seq=0 2023-11-15T03:10:08.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f098bfff700 1 -- 172.21.15.121:0/3084817929 <== mds.0 v2:172.21.15.121:6834/240077764 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x1e594e0 con 0x2078cb0 2023-11-15T03:10:08.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f098bfff700 10 client.10185 handle_command_reply: tid=0 2023-11-15T03:10:08.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.201+0000 7f09be2a1740 1 client.10185 shutdown 2023-11-15T03:10:08.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09be2a1740 1 -- 172.21.15.121:0/3084817929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023ad0 msgr2=0x20cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:08.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09be2a1740 1 --2- 172.21.15.121:0/3084817929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023ad0 0x20cc180 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 rx=0x7f09a8000d00 tx=0x7f09a8026000).stop 2023-11-15T03:10:08.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09be2a1740 1 -- 172.21.15.121:0/3084817929 shutdown_connections 2023-11-15T03:10:08.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09be2a1740 1 --2- 172.21.15.121:0/3084817929 >> [v2:172.21.15.121:6834/240077764,v1:172.21.15.121:6836/240077764] conn(0x2078cb0 0x2079090 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09be2a1740 1 --2- 172.21.15.121:0/3084817929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2070630 0x204e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09be2a1740 1 --2- 172.21.15.121:0/3084817929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2023eb0 0x204de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f098bfff700 0 client.10185 ms_handle_reset on v2:172.21.15.121:6834/240077764 2023-11-15T03:10:08.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09be2a1740 1 --2- 172.21.15.121:0/3084817929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2023ad0 0x20cc180 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09be2a1740 1 -- 172.21.15.121:0/3084817929 >> 172.21.15.121:0/3084817929 conn(0x1e02550 msgr2=0x2064010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:08.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09be2a1740 1 -- 172.21.15.121:0/3084817929 shutdown_connections 2023-11-15T03:10:08.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09be2a1740 1 -- 172.21.15.121:0/3084817929 wait complete. 2023-11-15T03:10:08.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09be2a1740 20 client.10185 trim_cache size 0 max 16384 2023-11-15T03:10:08.242 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:10:08.242 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T03:10:08.242 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T03:10:08.242 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 10122, 2023-11-15T03:10:08.243 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T03:10:08.243 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T03:10:08.243 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T03:10:08.243 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 6.877413722, 2023-11-15T03:10:08.243 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 304, 2023-11-15T03:10:08.244 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 128, 2023-11-15T03:10:08.244 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 128, 2023-11-15T03:10:08.244 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 9.8568370789999999 2023-11-15T03:10:08.244 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:10:08.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09b2c9d700 1 -- 172.21.15.121:0/989568413 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f099005a510 msgr2=0x7f099005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:08.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09b2c9d700 1 --2- 172.21.15.121:0/989568413 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f099005a510 0x7f099005c9d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7f09a800b400 tx=0x7f09a8016000).stop 2023-11-15T03:10:08.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09b2c9d700 1 -- 172.21.15.121:0/989568413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09ac0ad1b0 msgr2=0x7f09ac143100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:08.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09b2c9d700 1 --2- 172.21.15.121:0/989568413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09ac0ad1b0 0x7f09ac143100 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 rx=0x7f099c00c4c0 tx=0x7f099c007dd0).stop 2023-11-15T03:10:08.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09b2c9d700 1 -- 172.21.15.121:0/989568413 shutdown_connections 2023-11-15T03:10:08.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09b2c9d700 1 --2- 172.21.15.121:0/989568413 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f099005a510 0x7f099005c9d0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09b2c9d700 1 --2- 172.21.15.121:0/989568413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09ac0add40 0x7f09ac0a9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09b2c9d700 1 --2- 172.21.15.121:0/989568413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09ac0ad1b0 0x7f09ac143100 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09b2c9d700 1 --2- 172.21.15.121:0/989568413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac0ab0d0 0x7f09ac0a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09b2c9d700 1 -- 172.21.15.121:0/989568413 >> 172.21.15.121:0/989568413 conn(0x7f09ac067a60 msgr2=0x7f09ac0b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:08.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09b2c9d700 1 -- 172.21.15.121:0/989568413 shutdown_connections 2023-11-15T03:10:08.249 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.205+0000 7f09b2c9d700 1 -- 172.21.15.121:0/989568413 wait complete. 2023-11-15T03:10:08.250 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:10:08.250 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap header on 10000000000.00000000', proceeding to ls 2023-11-15T03:10:08.250 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T03:10:08.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.389+0000 7f9d96b69700 1 Processor -- start 2023-11-15T03:10:08.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.389+0000 7f9d96b69700 1 -- start start 2023-11-15T03:10:08.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.389+0000 7f9d96b69700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9d900bdfd0 0x7f9d90067e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.389+0000 7f9d96b69700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d900bf210 con 0x7f9d900c2f60 2023-11-15T03:10:08.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.389+0000 7f9d96b69700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d901543c0 con 0x7f9d90153fe0 2023-11-15T03:10:08.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.389+0000 7f9d96b69700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d900a8250 con 0x7f9d900bdfd0 2023-11-15T03:10:08.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.389+0000 7f9d95106700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9d900bdfd0 0x7f9d90067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.389+0000 7f9d8ffff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9d900c2f60 0x7f9d900bd8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:54440/0 (socket says 172.21.15.121:54440) 2023-11-15T03:10:08.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.389+0000 7f9d8ffff700 1 -- 172.21.15.121:0/2798865337 learned_addr learned my addr 172.21.15.121:0/2798865337 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:08.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.389+0000 7f9d8f7fe700 1 -- 172.21.15.121:0/2798865337 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1760538969 0 0) 0x7f9d900bf210 con 0x7f9d900c2f60 2023-11-15T03:10:08.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.389+0000 7f9d8f7fe700 1 -- 172.21.15.121:0/2798865337 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d800033f0 con 0x7f9d900c2f60 2023-11-15T03:10:08.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.389+0000 7f9d95106700 1 -- 172.21.15.121:0/2798865337 >> v1:172.21.15.190:6789/0 conn(0x7f9d90153fe0 legacy=0x7f9d90069690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:08.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.389+0000 7f9d95106700 1 -- 172.21.15.121:0/2798865337 >> v1:172.21.15.121:6789/0 conn(0x7f9d900c2f60 legacy=0x7f9d900bd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:08.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.389+0000 7f9d95106700 1 -- 172.21.15.121:0/2798865337 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d900a7c90 con 0x7f9d900bdfd0 2023-11-15T03:10:08.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.393+0000 7f9d95106700 1 --2- 172.21.15.121:0/2798865337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d900bdfd0 0x7f9d90067e00 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 rx=0x7f9d88009e60 tx=0x7f9d88009780).ready entity=mon.0 client_cookie=cd40bc7b752ccfa server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:08.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.393+0000 7f9d8f7fe700 1 -- 172.21.15.121:0/2798865337 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8800e070 con 0x7f9d900bdfd0 2023-11-15T03:10:08.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.393+0000 7f9d8f7fe700 1 -- 172.21.15.121:0/2798865337 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8800bb60 con 0x7f9d900bdfd0 2023-11-15T03:10:08.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.393+0000 7f9d8f7fe700 1 -- 172.21.15.121:0/2798865337 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d88012410 con 0x7f9d900bdfd0 2023-11-15T03:10:08.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.393+0000 7f9d96b69700 1 -- 172.21.15.121:0/2798865337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d900bdfd0 msgr2=0x7f9d90067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:08.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.393+0000 7f9d96b69700 1 --2- 172.21.15.121:0/2798865337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d900bdfd0 0x7f9d90067e00 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 rx=0x7f9d88009e60 tx=0x7f9d88009780).stop 2023-11-15T03:10:08.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.393+0000 7f9d96b69700 1 -- 172.21.15.121:0/2798865337 shutdown_connections 2023-11-15T03:10:08.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.393+0000 7f9d96b69700 1 --2- 172.21.15.121:0/2798865337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d900bdfd0 0x7f9d90067e00 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.393+0000 7f9d96b69700 1 -- 172.21.15.121:0/2798865337 >> 172.21.15.121:0/2798865337 conn(0x7f9d90066cf0 msgr2=0x7f9d90067100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:08.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.393+0000 7f9d96b69700 1 -- 172.21.15.121:0/2798865337 shutdown_connections 2023-11-15T03:10:08.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.393+0000 7f9d96b69700 1 -- 172.21.15.121:0/2798865337 wait complete. 2023-11-15T03:10:08.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.397+0000 7f9d96b69700 1 Processor -- start 2023-11-15T03:10:08.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.397+0000 7f9d96b69700 1 -- start start 2023-11-15T03:10:08.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.397+0000 7f9d96b69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d900bdfd0 0x7f9d90149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.397+0000 7f9d96b69700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d900c2f60 0x7f9d900c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.397+0000 7f9d96b69700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d90153fe0 0x7f9d900c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.397+0000 7f9d96b69700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900c5ab0 con 0x7f9d900c2f60 2023-11-15T03:10:08.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.397+0000 7f9d96b69700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900a9640 con 0x7f9d90153fe0 2023-11-15T03:10:08.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.397+0000 7f9d96b69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900aa4a0 con 0x7f9d900bdfd0 2023-11-15T03:10:08.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.397+0000 7f9d8ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d900c2f60 0x7f9d900c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.397+0000 7f9d8ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d900c2f60 0x7f9d900c50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59028/0 (socket says 172.21.15.121:59028) 2023-11-15T03:10:08.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.397+0000 7f9d8ffff700 1 -- 172.21.15.121:0/929295008 learned_addr learned my addr 172.21.15.121:0/929295008 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:08.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.397+0000 7f9d95106700 1 --2- 172.21.15.121:0/929295008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d90153fe0 0x7f9d900c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.397+0000 7f9d94905700 1 --2- 172.21.15.121:0/929295008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d900bdfd0 0x7f9d90149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.401+0000 7f9d8ffff700 1 -- 172.21.15.121:0/929295008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d900bdfd0 msgr2=0x7f9d90149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:08.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.401+0000 7f9d8ffff700 1 --2- 172.21.15.121:0/929295008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d900bdfd0 0x7f9d90149770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.401+0000 7f9d8ffff700 1 -- 172.21.15.121:0/929295008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d90153fe0 msgr2=0x7f9d900c55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:08.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.401+0000 7f9d8ffff700 1 --2- 172.21.15.121:0/929295008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d90153fe0 0x7f9d900c55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.401+0000 7f9d8ffff700 1 -- 172.21.15.121:0/929295008 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d88009a20 con 0x7f9d900c2f60 2023-11-15T03:10:08.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.401+0000 7f9d8ffff700 1 --2- 172.21.15.121:0/929295008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d900c2f60 0x7f9d900c50d0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 rx=0x7f9d7c00c5c0 tx=0x7f9d7c007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:08.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.401+0000 7f9d8cff9700 1 -- 172.21.15.121:0/929295008 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0072e0 con 0x7f9d900c2f60 2023-11-15T03:10:08.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.401+0000 7f9d8cff9700 1 -- 172.21.15.121:0/929295008 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7c007440 con 0x7f9d900c2f60 2023-11-15T03:10:08.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.401+0000 7f9d8cff9700 1 -- 172.21.15.121:0/929295008 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0057a0 con 0x7f9d900c2f60 2023-11-15T03:10:08.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.401+0000 7f9d96b69700 1 -- 172.21.15.121:0/929295008 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d900a9890 con 0x7f9d900c2f60 2023-11-15T03:10:08.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.401+0000 7f9d96b69700 1 -- 172.21.15.121:0/929295008 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d900a9e50 con 0x7f9d900c2f60 2023-11-15T03:10:08.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.401+0000 7f9d96b69700 1 -- 172.21.15.121:0/929295008 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d9005eeb0 con 0x7f9d900c2f60 2023-11-15T03:10:08.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.401+0000 7f9d8cff9700 1 -- 172.21.15.121:0/929295008 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9d7c005bd0 con 0x7f9d900c2f60 2023-11-15T03:10:08.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.405+0000 7f9d8cff9700 1 --2- 172.21.15.121:0/929295008 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9d7405a510 0x7f9d7405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:08.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.405+0000 7f9d8cff9700 1 -- 172.21.15.121:0/929295008 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9190+0+0 (secure 0 0 0) 0x7f9d7c074c40 con 0x7f9d900c2f60 2023-11-15T03:10:08.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.405+0000 7f9d94905700 1 --2- 172.21.15.121:0/929295008 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9d7405a510 0x7f9d7405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:08.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.405+0000 7f9d94905700 1 --2- 172.21.15.121:0/929295008 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9d7405a510 0x7f9d7405c9d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f9d840036d0 tx=0x7f9d84009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:08.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.413+0000 7f9d8cff9700 1 -- 172.21.15.121:0/929295008 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9d7c07b080 con 0x7f9d900c2f60 2023-11-15T03:10:08.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.609+0000 7f9d96b69700 1 -- 172.21.15.121:0/929295008 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9d9006a460 con 0x7f9d900c2f60 2023-11-15T03:10:08.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.609+0000 7f9d8cff9700 1 -- 172.21.15.121:0/929295008 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v304) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9d7c0477e0 con 0x7f9d900c2f60 2023-11-15T03:10:08.621 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T03:10:08.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.613+0000 7f9d96b69700 1 -- 172.21.15.121:0/929295008 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9d7405a510 msgr2=0x7f9d7405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:08.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.613+0000 7f9d96b69700 1 --2- 172.21.15.121:0/929295008 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9d7405a510 0x7f9d7405c9d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f9d840036d0 tx=0x7f9d84009040).stop 2023-11-15T03:10:08.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.613+0000 7f9d96b69700 1 -- 172.21.15.121:0/929295008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d900c2f60 msgr2=0x7f9d900c50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:08.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.613+0000 7f9d96b69700 1 --2- 172.21.15.121:0/929295008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d900c2f60 0x7f9d900c50d0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 rx=0x7f9d7c00c5c0 tx=0x7f9d7c007c80).stop 2023-11-15T03:10:08.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.613+0000 7f9d96b69700 1 -- 172.21.15.121:0/929295008 shutdown_connections 2023-11-15T03:10:08.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.613+0000 7f9d96b69700 1 --2- 172.21.15.121:0/929295008 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9d7405a510 0x7f9d7405c9d0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.613+0000 7f9d96b69700 1 --2- 172.21.15.121:0/929295008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d90153fe0 0x7f9d900c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.613+0000 7f9d96b69700 1 --2- 172.21.15.121:0/929295008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d900c2f60 0x7f9d900c50d0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.613+0000 7f9d96b69700 1 --2- 172.21.15.121:0/929295008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d900bdfd0 0x7f9d90149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:08.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.613+0000 7f9d96b69700 1 -- 172.21.15.121:0/929295008 >> 172.21.15.121:0/929295008 conn(0x7f9d90066cf0 msgr2=0x7f9d9014d990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:08.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.613+0000 7f9d96b69700 1 -- 172.21.15.121:0/929295008 shutdown_connections 2023-11-15T03:10:08.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:08.613+0000 7f9d96b69700 1 -- 172.21.15.121:0/929295008 wait complete. 2023-11-15T03:10:08.642 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T03:10:08.642 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T03:10:08.643 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T03:10:08.643 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T03:10:08.643 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T03:10:08.644 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T03:10:08.644 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T03:10:08.645 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T03:10:08.645 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:10:08.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:08.646 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T03:10:08.666 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T03:10:08.666 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T03:10:08.666 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T03:10:08.666 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:10:08.667 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T03:10:08.667 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:10:08.667 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:10:08.667 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:10:08.667 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T03:10:08.668 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:10:08.668 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:10:08.668 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T03:10:08.668 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:10:08.669 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:10:08.669 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T03:10:08.669 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:10:08.669 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T03:10:08.669 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 41631sec preferred_lft 41631sec 2023-11-15T03:10:08.670 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T03:10:08.670 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:10:08.670 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T03:10:08.670 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:10:08.670 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T03:10:08.671 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:10:08.671 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T03:10:08.671 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T03:10:08.671 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T03:10:08.671 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:10:08.672 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T03:10:08.672 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:10:08.672 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:10:08.672 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T03:10:08.672 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T03:10:08.673 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:10:08.673 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:10:08.673 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T03:10:08.673 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T03:10:08.673 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:10:08.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:08.675 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T03:10:08.755 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T03:10:08.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:08.759 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T03:10:08.825 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T03:10:08.826 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T03:10:08.826 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T03:10:08.826 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T03:10:08.826 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T03:10:08.827 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T03:10:08.827 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T03:10:08.827 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T03:10:08.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:08.829 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T03:10:08.829 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T03:10:08.829 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T03:10:08.830 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T03:10:08.830 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T03:10:08.831 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T03:10:08.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:08.969 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T03:10:08.990 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T03:10:08.990 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T03:10:08.991 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:10:08.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:08.992 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T03:10:09.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:09.071 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T03:10:09.133 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T03:10:09.133 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:10:09.250 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:10:09.250 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:10:09.275 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:10:09.275 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:10:09.281 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T03:10:09.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:09.282 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T03:10:09.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:10:09.316 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2023-11-15T03:10:09.371 INFO:teuthology.orchestra.run:waiting for 20 2023-11-15T03:10:09.383 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:10:09.373+0000 7fc474392700 -1 mds.0.cache.dir(0x10000000000) Corrupt fnode in dirfrag 0x10000000000: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input 2023-11-15T03:10:09.383 INFO:teuthology.orchestra.run.smithi121.stdout:.: 2023-11-15T03:10:09.384 INFO:teuthology.orchestra.run.smithi121.stdout:subdir 2023-11-15T03:10:09.384 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:10:09.385 INFO:teuthology.orchestra.run.smithi121.stdout:./subdir: 2023-11-15T03:10:09.386 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 10000000000.00000000' did not caused DAMAGED state 2023-11-15T03:10:09.387 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 600.00000000' 2023-11-15T03:10:09.387 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T03:10:09.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:10:09.389 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-15T03:10:09.476 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:10:09.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:09.477 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:10:09.541 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:10:09.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.729+0000 7f135308f700 1 Processor -- start 2023-11-15T03:10:09.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.729+0000 7f135308f700 1 -- start start 2023-11-15T03:10:09.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135308f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f134c0abaa0 0x7f134c0abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:09.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135308f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f134c0a8880 0x7f134c151400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:09.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135308f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f134c0c2c00 con 0x7f134c0bb510 2023-11-15T03:10:09.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135308f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f134c0657b0 con 0x7f134c0a8880 2023-11-15T03:10:09.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135308f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134c065a90 con 0x7f134c0abaa0 2023-11-15T03:10:09.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135188c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f134c0bb510 0x7f134c063570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:54454/0 (socket says 172.21.15.121:54454) 2023-11-15T03:10:09.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135188c700 1 -- 172.21.15.121:0/2262154157 learned_addr learned my addr 172.21.15.121:0/2262154157 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:09.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135288e700 1 --2- 172.21.15.121:0/2262154157 >> v2:172.21.15.190:3300/0 conn(0x7f134c0a8880 0x7f134c151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:09.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135208d700 1 --2- 172.21.15.121:0/2262154157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f134c0abaa0 0x7f134c0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:09.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135108b700 1 -- 172.21.15.121:0/2262154157 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3580177696 0 0) 0x7f134c0c2c00 con 0x7f134c0bb510 2023-11-15T03:10:09.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135108b700 1 -- 172.21.15.121:0/2262154157 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13380033f0 con 0x7f134c0bb510 2023-11-15T03:10:09.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135108b700 1 -- 172.21.15.121:0/2262154157 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2111559868 0 0) 0x7f13380033f0 con 0x7f134c0bb510 2023-11-15T03:10:09.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135108b700 1 -- 172.21.15.121:0/2262154157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f134c0abaa0 msgr2=0x7f134c0abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:09.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135108b700 1 --2- 172.21.15.121:0/2262154157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f134c0abaa0 0x7f134c0abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:09.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135108b700 1 -- 172.21.15.121:0/2262154157 >> v2:172.21.15.190:3300/0 conn(0x7f134c0a8880 msgr2=0x7f134c151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:09.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135108b700 1 --2- 172.21.15.121:0/2262154157 >> v2:172.21.15.190:3300/0 conn(0x7f134c0a8880 0x7f134c151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:09.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135108b700 1 -- 172.21.15.121:0/2262154157 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134c0c2890 con 0x7f134c0bb510 2023-11-15T03:10:09.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135108b700 1 -- 172.21.15.121:0/2262154157 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f133c0015f0 con 0x7f134c0bb510 2023-11-15T03:10:09.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135108b700 1 -- 172.21.15.121:0/2262154157 >> v1:172.21.15.121:6789/0 conn(0x7f134c0bb510 legacy=0x7f134c063570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:09.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135108b700 1 --2- 172.21.15.121:0/2262154157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13380038e0 0x7f1338003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:09.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135108b700 1 --2- 172.21.15.121:0/2262154157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1338004690 0x7f1338004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:09.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135108b700 1 --2- 172.21.15.121:0/2262154157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f134c0abaa0 0x7f1338005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:09.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135108b700 1 -- 172.21.15.121:0/2262154157 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134c0657b0 con 0x7f1338004690 2023-11-15T03:10:09.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135108b700 1 -- 172.21.15.121:0/2262154157 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134c065a90 con 0x7f13380038e0 2023-11-15T03:10:09.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135108b700 1 -- 172.21.15.121:0/2262154157 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133800fcf0 con 0x7f134c0abaa0 2023-11-15T03:10:09.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135188c700 1 --2- 172.21.15.121:0/2262154157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1338004690 0x7f1338004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:09.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135288e700 1 --2- 172.21.15.121:0/2262154157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f134c0abaa0 0x7f1338005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:09.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.733+0000 7f135208d700 1 --2- 172.21.15.121:0/2262154157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13380038e0 0x7f1338003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:09.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.737+0000 7f135288e700 1 -- 172.21.15.121:0/2262154157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13380038e0 msgr2=0x7f1338003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:09.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.737+0000 7f135288e700 1 --2- 172.21.15.121:0/2262154157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13380038e0 0x7f1338003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:09.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.737+0000 7f135288e700 1 -- 172.21.15.121:0/2262154157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1338004690 msgr2=0x7f1338004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:09.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.737+0000 7f135288e700 1 --2- 172.21.15.121:0/2262154157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1338004690 0x7f1338004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:09.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.737+0000 7f135288e700 1 -- 172.21.15.121:0/2262154157 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13380033f0 con 0x7f134c0abaa0 2023-11-15T03:10:09.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.737+0000 7f135288e700 1 --2- 172.21.15.121:0/2262154157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f134c0abaa0 0x7f1338005000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f134400e210 tx=0x7f134400b040).ready entity=mon.2 client_cookie=d878d91e6a0cccda server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:09.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.737+0000 7f135108b700 1 -- 172.21.15.121:0/2262154157 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1344009ac0 con 0x7f134c0abaa0 2023-11-15T03:10:09.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.737+0000 7f135108b700 1 -- 172.21.15.121:0/2262154157 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134400f040 con 0x7f134c0abaa0 2023-11-15T03:10:09.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.737+0000 7f135108b700 1 -- 172.21.15.121:0/2262154157 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134400ecd0 con 0x7f134c0abaa0 2023-11-15T03:10:09.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.737+0000 7f135308f700 1 -- 172.21.15.121:0/2262154157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f134c0abaa0 msgr2=0x7f1338005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:09.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.737+0000 7f135308f700 1 --2- 172.21.15.121:0/2262154157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f134c0abaa0 0x7f1338005000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f134400e210 tx=0x7f134400b040).stop 2023-11-15T03:10:09.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.737+0000 7f135288e700 1 -- 172.21.15.121:0/2262154157 reap_dead start 2023-11-15T03:10:09.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.737+0000 7f135308f700 1 -- 172.21.15.121:0/2262154157 shutdown_connections 2023-11-15T03:10:09.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.737+0000 7f135308f700 1 -- 172.21.15.121:0/2262154157 >> 172.21.15.121:0/2262154157 conn(0x7f134c067a60 msgr2=0x7f134c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:09.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.737+0000 7f135308f700 1 -- 172.21.15.121:0/2262154157 shutdown_connections 2023-11-15T03:10:09.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.737+0000 7f135308f700 1 -- 172.21.15.121:0/2262154157 wait complete. 2023-11-15T03:10:09.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135308f700 1 Processor -- start 2023-11-15T03:10:09.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135308f700 1 -- start start 2023-11-15T03:10:09.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135308f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f134c0abaa0 0x7f134c144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:09.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135308f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f134c147980 0x7f134c0bcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:09.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135308f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f134c149200 0x7f134c149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:09.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135308f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134c148a50 con 0x7f134c149200 2023-11-15T03:10:09.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135308f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134c150c10 con 0x7f134c147980 2023-11-15T03:10:09.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135308f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134c149b30 con 0x7f134c0abaa0 2023-11-15T03:10:09.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135288e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f134c149200 0x7f134c149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:09.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135288e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f134c149200 0x7f134c149640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59050/0 (socket says 172.21.15.121:59050) 2023-11-15T03:10:09.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135288e700 1 -- 172.21.15.121:0/736930999 learned_addr learned my addr 172.21.15.121:0/736930999 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:09.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135208d700 1 --2- 172.21.15.121:0/736930999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f134c0abaa0 0x7f134c144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:09.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135188c700 1 --2- 172.21.15.121:0/736930999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f134c147980 0x7f134c0bcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:09.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135288e700 1 -- 172.21.15.121:0/736930999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f134c0abaa0 msgr2=0x7f134c144300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:09.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135288e700 1 --2- 172.21.15.121:0/736930999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f134c0abaa0 0x7f134c144300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:09.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135288e700 1 -- 172.21.15.121:0/736930999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f134c147980 msgr2=0x7f134c0bcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:09.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135288e700 1 --2- 172.21.15.121:0/736930999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f134c147980 0x7f134c0bcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:09.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135288e700 1 -- 172.21.15.121:0/736930999 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134c147e40 con 0x7f134c149200 2023-11-15T03:10:09.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135288e700 1 --2- 172.21.15.121:0/736930999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f134c149200 0x7f134c149640 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 rx=0x7f1344014000 tx=0x7f134401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:09.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f13427fc700 1 -- 172.21.15.121:0/736930999 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1344021030 con 0x7f134c149200 2023-11-15T03:10:09.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f13427fc700 1 -- 172.21.15.121:0/736930999 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1344009ac0 con 0x7f134c149200 2023-11-15T03:10:09.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135308f700 1 -- 172.21.15.121:0/736930999 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f134c148120 con 0x7f134c149200 2023-11-15T03:10:09.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f13427fc700 1 -- 172.21.15.121:0/736930999 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1344013dc0 con 0x7f134c149200 2023-11-15T03:10:09.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.741+0000 7f135308f700 1 -- 172.21.15.121:0/736930999 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134c1486e0 con 0x7f134c149200 2023-11-15T03:10:09.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.745+0000 7f135308f700 1 -- 172.21.15.121:0/736930999 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f134c148c00 con 0x7f134c149200 2023-11-15T03:10:09.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.745+0000 7f13427fc700 1 -- 172.21.15.121:0/736930999 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1344024070 con 0x7f134c149200 2023-11-15T03:10:09.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.749+0000 7f13427fc700 1 --2- 172.21.15.121:0/736930999 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f133005a510 0x7f133005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:09.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.749+0000 7f13427fc700 1 -- 172.21.15.121:0/736930999 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9190+0+0 (secure 0 0 0) 0x7f1344022070 con 0x7f134c149200 2023-11-15T03:10:09.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.749+0000 7f135208d700 1 --2- 172.21.15.121:0/736930999 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f133005a510 0x7f133005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:09.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.749+0000 7f135208d700 1 --2- 172.21.15.121:0/736930999 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f133005a510 0x7f133005c9d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f1348006d80 tx=0x7f1348009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:09.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.757+0000 7f13427fc700 1 -- 172.21.15.121:0/736930999 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f134c148c00 con 0x7f134c149200 2023-11-15T03:10:09.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:09.957+0000 7f135308f700 1 -- 172.21.15.121:0/736930999 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f134c06bcf0 con 0x7f134c149200 2023-11-15T03:10:10.206 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:10:10.197+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:10:10.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.209+0000 7f13427fc700 1 -- 172.21.15.121:0/736930999 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v305) v1 ==== 161+0+0 (secure 0 0 0) 0x7f134404ce10 con 0x7f134c149200 2023-11-15T03:10:10.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.213+0000 7f135308f700 1 -- 172.21.15.121:0/736930999 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f133005a510 msgr2=0x7f133005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:10.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.213+0000 7f135308f700 1 --2- 172.21.15.121:0/736930999 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f133005a510 0x7f133005c9d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f1348006d80 tx=0x7f1348009060).stop 2023-11-15T03:10:10.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.213+0000 7f135308f700 1 -- 172.21.15.121:0/736930999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f134c149200 msgr2=0x7f134c149640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:10.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.213+0000 7f135308f700 1 --2- 172.21.15.121:0/736930999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f134c149200 0x7f134c149640 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 rx=0x7f1344014000 tx=0x7f134401a000).stop 2023-11-15T03:10:10.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.213+0000 7f135308f700 1 -- 172.21.15.121:0/736930999 shutdown_connections 2023-11-15T03:10:10.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.213+0000 7f135308f700 1 --2- 172.21.15.121:0/736930999 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f133005a510 0x7f133005c9d0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:10.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.213+0000 7f135308f700 1 --2- 172.21.15.121:0/736930999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f134c149200 0x7f134c149640 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:10.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.213+0000 7f135308f700 1 --2- 172.21.15.121:0/736930999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f134c147980 0x7f134c0bcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:10.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.213+0000 7f135308f700 1 --2- 172.21.15.121:0/736930999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f134c0abaa0 0x7f134c144300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:10.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.213+0000 7f135308f700 1 -- 172.21.15.121:0/736930999 >> 172.21.15.121:0/736930999 conn(0x7f134c067a60 msgr2=0x7f134c0a98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:10.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.213+0000 7f135308f700 1 -- 172.21.15.121:0/736930999 shutdown_connections 2023-11-15T03:10:10.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.213+0000 7f135308f700 1 -- 172.21.15.121:0/736930999 wait complete. 2023-11-15T03:10:10.227 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:10:10.232 INFO:tasks.ceph.mds.a.smithi121.stderr: -91> 2023-11-15T03:10:09.373+0000 7fc474392700 -1 mds.0.cache.dir(0x10000000000) Corrupt fnode in dirfrag 0x10000000000: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input 2023-11-15T03:10:10.246 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:10:10.271 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:10:10.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.385+0000 7fc9e31f9700 1 Processor -- start 2023-11-15T03:10:10.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.385+0000 7fc9e31f9700 1 -- start start 2023-11-15T03:10:10.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.389+0000 7fc9e31f9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc9dc0a9f80 0x7fc9dc158260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:10.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.389+0000 7fc9e31f9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc9dc0c5a00 0x7fc9dc154b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:10.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.389+0000 7fc9e31f9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9dc0c6170 con 0x7fc9dc0c7380 2023-11-15T03:10:10.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.389+0000 7fc9e31f9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9dc154000 con 0x7fc9dc0c5a00 2023-11-15T03:10:10.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.389+0000 7fc9e31f9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9dc157dd0 con 0x7fc9dc0a9f80 2023-11-15T03:10:10.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e19f6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc9dc0c5a00 0x7fc9dc154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:10.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e19f6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc9dc0c5a00 0x7fc9dc154b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59086/0 (socket says 172.21.15.121:59086) 2023-11-15T03:10:10.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e19f6700 1 -- 172.21.15.121:0/1207390331 learned_addr learned my addr 172.21.15.121:0/1207390331 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:10.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e21f7700 1 --2- 172.21.15.121:0/1207390331 >> v2:172.21.15.190:3300/0 conn(0x7fc9dc0a9f80 0x7fc9dc158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:10.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e19f6700 1 -- 172.21.15.121:0/1207390331 >> v2:172.21.15.190:3300/0 conn(0x7fc9dc0a9f80 msgr2=0x7fc9dc158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:10.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e19f6700 1 --2- 172.21.15.121:0/1207390331 >> v2:172.21.15.190:3300/0 conn(0x7fc9dc0a9f80 0x7fc9dc158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:10.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e19f6700 1 -- 172.21.15.121:0/1207390331 >> v1:172.21.15.121:6789/0 conn(0x7fc9dc0c7380 legacy=0x7fc9dc0c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:10.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e19f6700 1 -- 172.21.15.121:0/1207390331 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc155080 con 0x7fc9dc0c5a00 2023-11-15T03:10:10.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e21f7700 1 --2- 172.21.15.121:0/1207390331 >> v2:172.21.15.190:3300/0 conn(0x7fc9dc0a9f80 0x7fc9dc158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:10.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e19f6700 1 --2- 172.21.15.121:0/1207390331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9dc0c5a00 0x7fc9dc154b90 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 rx=0x7fc9dc059f30 tx=0x7fc9cc0090f0).ready entity=mon.0 client_cookie=af110b9a53e02aed server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:10.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e11f5700 1 -- 172.21.15.121:0/1207390331 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc00b600 con 0x7fc9dc0c5a00 2023-11-15T03:10:10.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e11f5700 1 -- 172.21.15.121:0/1207390331 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9cc00b760 con 0x7fc9dc0c5a00 2023-11-15T03:10:10.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e11f5700 1 -- 172.21.15.121:0/1207390331 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc00bae0 con 0x7fc9dc0c5a00 2023-11-15T03:10:10.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e31f9700 1 -- 172.21.15.121:0/1207390331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9dc0c5a00 msgr2=0x7fc9dc154b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:10.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e31f9700 1 --2- 172.21.15.121:0/1207390331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9dc0c5a00 0x7fc9dc154b90 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 rx=0x7fc9dc059f30 tx=0x7fc9cc0090f0).stop 2023-11-15T03:10:10.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e31f9700 1 -- 172.21.15.121:0/1207390331 shutdown_connections 2023-11-15T03:10:10.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e31f9700 1 --2- 172.21.15.121:0/1207390331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9dc0c5a00 0x7fc9dc154b90 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:10.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e31f9700 1 --2- 172.21.15.121:0/1207390331 >> v2:172.21.15.190:3300/0 conn(0x7fc9dc0a9f80 0x7fc9dc158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:10.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e31f9700 1 -- 172.21.15.121:0/1207390331 >> 172.21.15.121:0/1207390331 conn(0x7fc9dc067a60 msgr2=0x7fc9dc067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:10.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e31f9700 1 -- 172.21.15.121:0/1207390331 shutdown_connections 2023-11-15T03:10:10.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.393+0000 7fc9e31f9700 1 -- 172.21.15.121:0/1207390331 wait complete. 2023-11-15T03:10:10.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9e31f9700 1 Processor -- start 2023-11-15T03:10:10.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9e31f9700 1 -- start start 2023-11-15T03:10:10.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9e31f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9dc0a9f80 0x7fc9dc0ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:10.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9e31f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9dc0c7380 0x7fc9dc0c4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:10.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9e31f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9dc164e80 0x7fc9dc0bf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:10.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9e31f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc0aea40 con 0x7fc9dc0a9f80 2023-11-15T03:10:10.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9e31f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc0c4720 con 0x7fc9dc164e80 2023-11-15T03:10:10.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9e31f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc0af920 con 0x7fc9dc0c7380 2023-11-15T03:10:10.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9e21f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9dc0a9f80 0x7fc9dc0ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:10.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9e21f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9dc0a9f80 0x7fc9dc0ae550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59096/0 (socket says 172.21.15.121:59096) 2023-11-15T03:10:10.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9e21f7700 1 -- 172.21.15.121:0/579766721 learned_addr learned my addr 172.21.15.121:0/579766721 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:10.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9e21f7700 1 -- 172.21.15.121:0/579766721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9dc0c7380 msgr2=0x7fc9dc0c4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:10.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9e21f7700 1 --2- 172.21.15.121:0/579766721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9dc0c7380 0x7fc9dc0c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:10:10.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9e21f7700 1 -- 172.21.15.121:0/579766721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9dc164e80 msgr2=0x7fc9dc0bf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:10.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9e21f7700 1 --2- 172.21.15.121:0/579766721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9dc164e80 0x7fc9dc0bf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:10:10.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9e21f7700 1 -- 172.21.15.121:0/579766721 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc009370 con 0x7fc9dc0a9f80 2023-11-15T03:10:10.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9e21f7700 1 --2- 172.21.15.121:0/579766721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9dc0a9f80 0x7fc9dc0ae550 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 rx=0x7fc9d800d890 tx=0x7fc9d8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:10.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9d27fc700 1 -- 172.21.15.121:0/579766721 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8011590 con 0x7fc9dc0a9f80 2023-11-15T03:10:10.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9d27fc700 1 -- 172.21.15.121:0/579766721 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d80116f0 con 0x7fc9dc0a9f80 2023-11-15T03:10:10.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9e31f9700 1 -- 172.21.15.121:0/579766721 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9dc0aec30 con 0x7fc9dc0a9f80 2023-11-15T03:10:10.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.397+0000 7fc9d27fc700 1 -- 172.21.15.121:0/579766721 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8011a70 con 0x7fc9dc0a9f80 2023-11-15T03:10:10.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.401+0000 7fc9e31f9700 1 -- 172.21.15.121:0/579766721 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9dc0af2d0 con 0x7fc9dc0a9f80 2023-11-15T03:10:10.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.401+0000 7fc9e31f9700 1 -- 172.21.15.121:0/579766721 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9dc06bcf0 con 0x7fc9dc0a9f80 2023-11-15T03:10:10.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.401+0000 7fc9d27fc700 1 -- 172.21.15.121:0/579766721 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc9d801c460 con 0x7fc9dc0a9f80 2023-11-15T03:10:10.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.401+0000 7fc9d27fc700 1 --2- 172.21.15.121:0/579766721 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc9c005a510 0x7fc9c005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:10.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.401+0000 7fc9d27fc700 1 -- 172.21.15.121:0/579766721 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 9276+0+0 (secure 0 0 0) 0x7fc9d8075610 con 0x7fc9dc0a9f80 2023-11-15T03:10:10.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.405+0000 7fc9e19f6700 1 --2- 172.21.15.121:0/579766721 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc9c005a510 0x7fc9c005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:10.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.405+0000 7fc9e19f6700 1 --2- 172.21.15.121:0/579766721 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc9c005a510 0x7fc9c005c9d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7fc9dc059f30 tx=0x7fc9cc017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:10.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.409+0000 7fc9d27fc700 1 -- 172.21.15.121:0/579766721 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc9d8048100 con 0x7fc9dc0a9f80 2023-11-15T03:10:10.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:10.625+0000 7fc9e31f9700 1 -- 172.21.15.121:0/579766721 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fc9dc0c5e90 con 0x7fc9dc0a9f80 2023-11-15T03:10:11.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.213+0000 7fc9d27fc700 1 -- 172.21.15.121:0/579766721 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v306) v1 ==== 107+0+0 (secure 0 0 0) 0x7fc9d804b720 con 0x7fc9dc0a9f80 2023-11-15T03:10:11.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.217+0000 7fc9e31f9700 1 -- 172.21.15.121:0/579766721 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc9c005a510 msgr2=0x7fc9c005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.217+0000 7fc9e31f9700 1 --2- 172.21.15.121:0/579766721 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc9c005a510 0x7fc9c005c9d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7fc9dc059f30 tx=0x7fc9cc017000).stop 2023-11-15T03:10:11.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.217+0000 7fc9e31f9700 1 -- 172.21.15.121:0/579766721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9dc0a9f80 msgr2=0x7fc9dc0ae550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.217+0000 7fc9e31f9700 1 --2- 172.21.15.121:0/579766721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9dc0a9f80 0x7fc9dc0ae550 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 rx=0x7fc9d800d890 tx=0x7fc9d8008040).stop 2023-11-15T03:10:11.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.217+0000 7fc9e31f9700 1 -- 172.21.15.121:0/579766721 shutdown_connections 2023-11-15T03:10:11.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.217+0000 7fc9e31f9700 1 --2- 172.21.15.121:0/579766721 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc9c005a510 0x7fc9c005c9d0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.217+0000 7fc9e31f9700 1 --2- 172.21.15.121:0/579766721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9dc164e80 0x7fc9dc0bf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:10:11.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.217+0000 7fc9e31f9700 1 --2- 172.21.15.121:0/579766721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9dc0c7380 0x7fc9dc0c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:10:11.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.217+0000 7fc9e31f9700 1 --2- 172.21.15.121:0/579766721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9dc0a9f80 0x7fc9dc0ae550 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.217+0000 7fc9e31f9700 1 -- 172.21.15.121:0/579766721 >> 172.21.15.121:0/579766721 conn(0x7fc9dc067a60 msgr2=0x7fc9dc156510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:11.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.217+0000 7fc9e31f9700 1 -- 172.21.15.121:0/579766721 shutdown_connections 2023-11-15T03:10:11.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.217+0000 7fc9e31f9700 1 -- 172.21.15.121:0/579766721 wait complete. 2023-11-15T03:10:11.231 INFO:teuthology.orchestra.run.smithi121.stderr:nothing to do: rank is not damaged 2023-11-15T03:10:11.249 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:10:11.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.337+0000 7f5bcfd33d00 1 Processor -- start 2023-11-15T03:10:11.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.337+0000 7f5bcfd33d00 1 -- start start 2023-11-15T03:10:11.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bcfd33d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b6af77bd0 0x555b6b0243c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bcfd33d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6b022dc0 0x555b6af8d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bcfd33d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6b026c30 0x555b6af8dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bcfd33d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6aec5220 con 0x555b6af77bd0 2023-11-15T03:10:11.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bcfd33d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6b026210 con 0x555b6b026c30 2023-11-15T03:10:11.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bcfd33d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b6b022550 con 0x555b6b022dc0 2023-11-15T03:10:11.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bc43a1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b6af77bd0 0x555b6b0243c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bc43a1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b6af77bd0 0x555b6b0243c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59104/0 (socket says 172.21.15.121:59104) 2023-11-15T03:10:11.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bc43a1700 1 -- 172.21.15.121:0/3451454648 learned_addr learned my addr 172.21.15.121:0/3451454648 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:11.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bbbba0700 1 --2- 172.21.15.121:0/3451454648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6b022dc0 0x555b6af8d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bc4ba2700 1 --2- 172.21.15.121:0/3451454648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6b026c30 0x555b6af8dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bc43a1700 1 -- 172.21.15.121:0/3451454648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6b022dc0 msgr2=0x555b6af8d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:11.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bc43a1700 1 --2- 172.21.15.121:0/3451454648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6b022dc0 0x555b6af8d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bc43a1700 1 -- 172.21.15.121:0/3451454648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6b026c30 msgr2=0x555b6af8dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:11.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bc43a1700 1 --2- 172.21.15.121:0/3451454648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6b026c30 0x555b6af8dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bc43a1700 1 -- 172.21.15.121:0/3451454648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac005db0 con 0x555b6af77bd0 2023-11-15T03:10:11.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bbbba0700 1 --2- 172.21.15.121:0/3451454648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6b022dc0 0x555b6af8d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:10:11.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bc4ba2700 1 --2- 172.21.15.121:0/3451454648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6b026c30 0x555b6af8dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:10:11.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bc43a1700 1 --2- 172.21.15.121:0/3451454648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b6af77bd0 0x555b6b0243c0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 rx=0x7f5bb4001550 tx=0x7f5bb4006b50).ready entity=mon.0 client_cookie=75ec0c0de0686740 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:11.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bc339f700 1 -- 172.21.15.121:0/3451454648 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4006110 con 0x555b6af77bd0 2023-11-15T03:10:11.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bc339f700 1 -- 172.21.15.121:0/3451454648 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb4006270 con 0x555b6af77bd0 2023-11-15T03:10:11.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bc339f700 1 -- 172.21.15.121:0/3451454648 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4004690 con 0x555b6af77bd0 2023-11-15T03:10:11.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/3451454648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b6af77bd0 msgr2=0x555b6b0243c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/3451454648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b6af77bd0 0x555b6b0243c0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 rx=0x7f5bb4001550 tx=0x7f5bb4006b50).stop 2023-11-15T03:10:11.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/3451454648 shutdown_connections 2023-11-15T03:10:11.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/3451454648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6b026c30 0x555b6af8dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/3451454648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6b022dc0 0x555b6af8d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/3451454648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b6af77bd0 0x555b6b0243c0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/3451454648 >> 172.21.15.121:0/3451454648 conn(0x555b6af94f80 msgr2=0x555b6b0231a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:11.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/3451454648 shutdown_connections 2023-11-15T03:10:11.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.341+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/3451454648 wait complete. 2023-11-15T03:10:11.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bcfd33d00 1 Processor -- start 2023-11-15T03:10:11.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bcfd33d00 1 -- start start 2023-11-15T03:10:11.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bcfd33d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b6af77bd0 0x555b6b0203e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bcfd33d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6b022dc0 0x555b6b0213c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bcfd33d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6b026c30 0x555b6af4d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bcfd33d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6af4e080 con 0x555b6af77bd0 2023-11-15T03:10:11.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bcfd33d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6af782d0 con 0x555b6b022dc0 2023-11-15T03:10:11.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bcfd33d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b6af78e50 con 0x555b6b026c30 2023-11-15T03:10:11.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bc43a1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b6af77bd0 0x555b6b0203e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bc43a1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b6af77bd0 0x555b6b0203e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59110/0 (socket says 172.21.15.121:59110) 2023-11-15T03:10:11.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 learned_addr learned my addr 172.21.15.121:0/1583301038 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:11.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6b026c30 msgr2=0x555b6af4d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bc43a1700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6b026c30 0x555b6af4d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:10:11.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bbbba0700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6b022dc0 0x555b6b0213c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6b022dc0 msgr2=0x555b6b0213c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bc43a1700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6b022dc0 0x555b6b0213c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b6aef37f0 con 0x555b6af77bd0 2023-11-15T03:10:11.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bc43a1700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b6af77bd0 0x555b6b0203e0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 rx=0x7f5bb4001ca0 tx=0x7f5bb4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:11.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bc1b9c700 1 -- 172.21.15.121:0/1583301038 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4013720 con 0x555b6af77bd0 2023-11-15T03:10:11.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bc1b9c700 1 -- 172.21.15.121:0/1583301038 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb4013880 con 0x555b6af77bd0 2023-11-15T03:10:11.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555b6aeee770 con 0x555b6af77bd0 2023-11-15T03:10:11.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.345+0000 7f5bc1b9c700 1 -- 172.21.15.121:0/1583301038 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4013c20 con 0x555b6af77bd0 2023-11-15T03:10:11.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.349+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b6aebda70 con 0x555b6af77bd0 2023-11-15T03:10:11.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.349+0000 7f5bc1b9c700 1 -- 172.21.15.121:0/1583301038 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5bb4021470 con 0x555b6af77bd0 2023-11-15T03:10:11.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.349+0000 7f5bc1b9c700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5ba4059020 0x7f5ba405b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.349+0000 7f5bc1b9c700 1 -- 172.21.15.121:0/1583301038 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 9276+0+0 (secure 0 0 0) 0x7f5bb401b040 con 0x555b6af77bd0 2023-11-15T03:10:11.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.349+0000 7f5bbbba0700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5ba4059020 0x7f5ba405b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.368 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:10:11.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.353+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x555b6af4d6c0 0x555b6af79ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.353+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b01c1a0 con 0x555b6af4d6c0 2023-11-15T03:10:11.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.353+0000 7f5bc4ba2700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x555b6af4d6c0 0x555b6af79ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.353+0000 7f5bbbba0700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5ba4059020 0x7f5ba405b4e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f5bac005c00 tx=0x7f5bac014000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:11.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.353+0000 7f5bc4ba2700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x555b6af4d6c0 0x555b6af79ce0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:11.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.357+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v129'413 uv406 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc009820 con 0x555b6af4d6c0 2023-11-15T03:10:11.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.357+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b039920 con 0x555b6af4d6c0 2023-11-15T03:10:11.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.357+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v129'414 uv414 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6af4d6c0 2023-11-15T03:10:11.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.357+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b039cd0 con 0x555b6af4d6c0 2023-11-15T03:10:11.371 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:10:11.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.361+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v129'415 uv415 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6af4d6c0 2023-11-15T03:10:11.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.361+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b03a080 con 0x555b6af4d6c0 2023-11-15T03:10:11.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.361+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v129'416 uv416 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6af4d6c0 2023-11-15T03:10:11.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.361+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b03a430 con 0x555b6af4d6c0 2023-11-15T03:10:11.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.365+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v129'417 uv417 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6af4d6c0 2023-11-15T03:10:11.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.365+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0430f0 con 0x555b6af4d6c0 2023-11-15T03:10:11.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.369+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v129'418 uv418 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6af4d6c0 2023-11-15T03:10:11.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.369+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0434a0 con 0x555b6af4d6c0 2023-11-15T03:10:11.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.369+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v129'419 uv412 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6af4d6c0 2023-11-15T03:10:11.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.369+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b043850 con 0x555b6af4d6c0 2023-11-15T03:10:11.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.373+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v129'420 uv420 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6af4d6c0 2023-11-15T03:10:11.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.373+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b043850 con 0x555b6af4d6c0 2023-11-15T03:10:11.387 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:10:11.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.377+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v129'421 uv421 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6af4d6c0 2023-11-15T03:10:11.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.377+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b044860 con 0x555b6af4d6c0 2023-11-15T03:10:11.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.381+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v129'422 uv422 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6af4d6c0 2023-11-15T03:10:11.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.381+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b044860 con 0x555b6af4d6c0 2023-11-15T03:10:11.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.385+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v129'423 uv423 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6af4d6c0 2023-11-15T03:10:11.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.385+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b043850 con 0x555b6af4d6c0 2023-11-15T03:10:11.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.393+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v129'424 uv424 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6af4d6c0 2023-11-15T03:10:11.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.393+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x555b6b043d60 0x555b6b046150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.393+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b046640 con 0x555b6b043d60 2023-11-15T03:10:11.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.393+0000 7f5bc43a1700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x555b6b043d60 0x555b6b046150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.393+0000 7f5bc43a1700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x555b6b043d60 0x555b6b046150 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:11.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.397+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v129'653 uv652 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.397+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b0469f0 con 0x555b6b043d60 2023-11-15T03:10:11.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.397+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v129'654 uv654 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.397+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b046da0 con 0x555b6b043d60 2023-11-15T03:10:11.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.401+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v129'655 uv655 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.413 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:10:11.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.401+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b048160 con 0x555b6b043d60 2023-11-15T03:10:11.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.409+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v129'656 uv656 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.409+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b048510 con 0x555b6b043d60 2023-11-15T03:10:11.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.409+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v129'657 uv657 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.409+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b047150 con 0x555b6b043d60 2023-11-15T03:10:11.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.409+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v129'658 uv658 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.409+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b047a40 con 0x555b6b043d60 2023-11-15T03:10:11.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.417+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v129'659 uv659 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.417+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0488c0 con 0x555b6b043d60 2023-11-15T03:10:11.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.421+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v129'660 uv645 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.421+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b0488c0 con 0x555b6b043d60 2023-11-15T03:10:11.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.425+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v129'661 uv661 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.425+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0488c0 con 0x555b6b043d60 2023-11-15T03:10:11.434 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:10:11.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.425+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v129'662 uv662 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.425+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0498d0 con 0x555b6b043d60 2023-11-15T03:10:11.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.425+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v129'663 uv663 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.425+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0498d0 con 0x555b6b043d60 2023-11-15T03:10:11.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.433+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v129'664 uv664 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.433+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0488c0 con 0x555b6b043d60 2023-11-15T03:10:11.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.437+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v129'665 uv665 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.437+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0488c0 con 0x555b6b043d60 2023-11-15T03:10:11.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.437+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v129'666 uv651 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.437+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b0488c0 con 0x555b6b043d60 2023-11-15T03:10:11.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.441+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v129'667 uv667 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.441+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0488c0 con 0x555b6b043d60 2023-11-15T03:10:11.451 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:10:11.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.445+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v129'668 uv668 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.445+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0498d0 con 0x555b6b043d60 2023-11-15T03:10:11.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.445+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v129'669 uv669 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.445+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0498d0 con 0x555b6b043d60 2023-11-15T03:10:11.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.449+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v129'670 uv670 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.449+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0488c0 con 0x555b6b043d60 2023-11-15T03:10:11.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.453+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v129'671 uv671 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.453+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x555b6b048dd0 0x555b6b04b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.453+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b04b6b0 con 0x555b6b048dd0 2023-11-15T03:10:11.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.453+0000 7f5bbbba0700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x555b6b048dd0 0x555b6b04b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.453+0000 7f5bbbba0700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x555b6b048dd0 0x555b6b04b1c0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:11.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.457+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v129'357 uv350 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b048dd0 2023-11-15T03:10:11.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.457+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b04ba60 con 0x555b6b048dd0 2023-11-15T03:10:11.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.461+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v129'358 uv358 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b048dd0 2023-11-15T03:10:11.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.461+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b04be10 con 0x555b6b048dd0 2023-11-15T03:10:11.472 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:10:11.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.465+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v129'359 uv359 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b048dd0 2023-11-15T03:10:11.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.465+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b04c6b0 con 0x555b6b048dd0 2023-11-15T03:10:11.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.469+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v129'360 uv360 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b048dd0 2023-11-15T03:10:11.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.469+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b04ca60 con 0x555b6b048dd0 2023-11-15T03:10:11.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.469+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v129'361 uv361 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b048dd0 2023-11-15T03:10:11.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.469+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b04ce10 con 0x555b6b048dd0 2023-11-15T03:10:11.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.469+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v129'362 uv356 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b048dd0 2023-11-15T03:10:11.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.469+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b04d1c0 con 0x555b6b048dd0 2023-11-15T03:10:11.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.473+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v129'363 uv363 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b048dd0 2023-11-15T03:10:11.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.473+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b04d1c0 con 0x555b6b048dd0 2023-11-15T03:10:11.486 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:10:11.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.477+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v129'364 uv364 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b048dd0 2023-11-15T03:10:11.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.477+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b04d1c0 con 0x555b6b048dd0 2023-11-15T03:10:11.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.477+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v129'365 uv365 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b048dd0 2023-11-15T03:10:11.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.477+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b04d1c0 con 0x555b6b048dd0 2023-11-15T03:10:11.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.485+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v129'366 uv366 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b048dd0 2023-11-15T03:10:11.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.485+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x555b6b04d6d0 0x555b6b04fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.485+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b050080 con 0x555b6b04d6d0 2023-11-15T03:10:11.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.485+0000 7f5bc4ba2700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x555b6b04d6d0 0x555b6b04fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.489+0000 7f5bc4ba2700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x555b6b04d6d0 0x555b6b04fac0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:11.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.489+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v129'420 uv419 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.489+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b0504b0 con 0x555b6b04d6d0 2023-11-15T03:10:11.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.493+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v129'421 uv421 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.493+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0504b0 con 0x555b6b04d6d0 2023-11-15T03:10:11.505 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:10:11.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.497+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v129'422 uv422 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.497+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b054540 con 0x555b6b04d6d0 2023-11-15T03:10:11.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.501+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v129'423 uv423 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.501+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b054540 con 0x555b6b04d6d0 2023-11-15T03:10:11.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.505+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v129'424 uv424 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.505+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0504b0 con 0x555b6b04d6d0 2023-11-15T03:10:11.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.509+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v129'425 uv416 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.509+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b0504b0 con 0x555b6b04d6d0 2023-11-15T03:10:11.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.509+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v129'426 uv426 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.513+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0504b0 con 0x555b6b04d6d0 2023-11-15T03:10:11.521 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:10:11.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.513+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v129'427 uv427 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.513+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0514c0 con 0x555b6b04d6d0 2023-11-15T03:10:11.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.517+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v129'428 uv428 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.517+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0514c0 con 0x555b6b04d6d0 2023-11-15T03:10:11.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.521+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v129'429 uv429 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.521+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0504b0 con 0x555b6b04d6d0 2023-11-15T03:10:11.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.525+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v129'430 uv430 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.525+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x555b6b050b30 0x555b6b052f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.525+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b053530 con 0x555b6b050b30 2023-11-15T03:10:11.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.525+0000 7f5bc43a1700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x555b6b050b30 0x555b6b052f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.525+0000 7f5bc43a1700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x555b6b050b30 0x555b6b052f20 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:11.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.529+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v129'207 uv206 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b050b30 2023-11-15T03:10:11.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.529+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b053650 con 0x555b6b050b30 2023-11-15T03:10:11.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.529+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v129'208 uv208 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b050b30 2023-11-15T03:10:11.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.529+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b053a50 con 0x555b6b050b30 2023-11-15T03:10:11.540 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:10:11.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.533+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v129'209 uv209 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b050b30 2023-11-15T03:10:11.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.533+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b054a60 con 0x555b6b050b30 2023-11-15T03:10:11.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.533+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v129'210 uv210 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b050b30 2023-11-15T03:10:11.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.533+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b054a60 con 0x555b6b050b30 2023-11-15T03:10:11.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.537+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v129'211 uv211 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b050b30 2023-11-15T03:10:11.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.537+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b053a50 con 0x555b6b050b30 2023-11-15T03:10:11.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.541+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v129'212 uv212 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b050b30 2023-11-15T03:10:11.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.541+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x555b6b053f60 0x555b6b056380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.541+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056a70 con 0x555b6b053f60 2023-11-15T03:10:11.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.541+0000 7f5bbbba0700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x555b6b053f60 0x555b6b056380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.541+0000 7f5bbbba0700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x555b6b053f60 0x555b6b056380 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:11.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.545+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v129'150 uv149 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.545+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b056e50 con 0x555b6b053f60 2023-11-15T03:10:11.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.545+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v129'151 uv151 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.545+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056e50 con 0x555b6b053f60 2023-11-15T03:10:11.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.549+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v129'152 uv152 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.557 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:10:11.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.549+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056e50 con 0x555b6b053f60 2023-11-15T03:10:11.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.553+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v129'153 uv153 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.553+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056e50 con 0x555b6b053f60 2023-11-15T03:10:11.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.553+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v129'207 uv206 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.553+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b056e50 con 0x555b6b053f60 2023-11-15T03:10:11.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.557+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v129'208 uv208 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.557+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056e50 con 0x555b6b053f60 2023-11-15T03:10:11.568 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:10:11.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.561+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v129'209 uv209 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.561+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b057e60 con 0x555b6b053f60 2023-11-15T03:10:11.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.561+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v129'210 uv210 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.565+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b057e60 con 0x555b6b053f60 2023-11-15T03:10:11.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.565+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v129'211 uv211 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.565+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056e50 con 0x555b6b053f60 2023-11-15T03:10:11.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.569+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v129'212 uv212 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.569+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056e50 con 0x555b6b043d60 2023-11-15T03:10:11.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.573+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v129'382 uv376 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.573+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b056e50 con 0x555b6b043d60 2023-11-15T03:10:11.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.577+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v129'383 uv383 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.577+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056e50 con 0x555b6b043d60 2023-11-15T03:10:11.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.577+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v129'384 uv384 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.586 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:10:11.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.581+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056e50 con 0x555b6b043d60 2023-11-15T03:10:11.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.581+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v129'385 uv385 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.581+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056e50 con 0x555b6b043d60 2023-11-15T03:10:11.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.585+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v129'386 uv386 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.585+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056e50 con 0x555b6b043d60 2023-11-15T03:10:11.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.589+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v129'387 uv387 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.589+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056e50 con 0x555b6b043d60 2023-11-15T03:10:11.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.593+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v129'388 uv381 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.593+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b056e50 con 0x555b6b043d60 2023-11-15T03:10:11.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.593+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v129'389 uv389 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.593+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056e50 con 0x555b6b043d60 2023-11-15T03:10:11.605 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:10:11.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.597+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v129'390 uv390 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.597+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056e50 con 0x555b6b043d60 2023-11-15T03:10:11.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.601+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v129'391 uv391 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.601+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b057250 con 0x555b6b043d60 2023-11-15T03:10:11.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.605+0000 7f5bc43a1700 1 -- 172.21.15.121:0/1583301038 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v129'392 uv392 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bb404a7a0 con 0x555b6b043d60 2023-11-15T03:10:11.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.605+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b057250 con 0x555b6b048dd0 2023-11-15T03:10:11.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.609+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v129'173 uv172 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b048dd0 2023-11-15T03:10:11.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.609+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b057250 con 0x555b6b048dd0 2023-11-15T03:10:11.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.609+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v129'174 uv174 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b048dd0 2023-11-15T03:10:11.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.609+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056ed0 con 0x555b6b048dd0 2023-11-15T03:10:11.622 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:10:11.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.613+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v129'175 uv175 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b048dd0 2023-11-15T03:10:11.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.613+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056ed0 con 0x555b6b048dd0 2023-11-15T03:10:11.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.617+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v129'176 uv176 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b048dd0 2023-11-15T03:10:11.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.617+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b04c240 con 0x555b6b048dd0 2023-11-15T03:10:11.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.617+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v129'177 uv177 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b048dd0 2023-11-15T03:10:11.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.617+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b04c240 con 0x555b6b053f60 2023-11-15T03:10:11.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.621+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v129'207 uv206 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.621+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b056ed0 con 0x555b6b053f60 2023-11-15T03:10:11.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.625+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v129'208 uv208 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.625+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056ed0 con 0x555b6b053f60 2023-11-15T03:10:11.634 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:10:11.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.629+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v129'209 uv209 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.629+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b057ee0 con 0x555b6b053f60 2023-11-15T03:10:11.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.629+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v129'210 uv210 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.629+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b057ee0 con 0x555b6b053f60 2023-11-15T03:10:11.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.633+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v129'211 uv211 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.633+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056ed0 con 0x555b6b053f60 2023-11-15T03:10:11.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.637+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v129'212 uv212 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.637+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056ed0 con 0x555b6b053f60 2023-11-15T03:10:11.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.641+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v129'174 uv173 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.641+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b056ed0 con 0x555b6b053f60 2023-11-15T03:10:11.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.641+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v129'175 uv175 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.641+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056ed0 con 0x555b6b053f60 2023-11-15T03:10:11.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.645+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v129'176 uv176 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.655 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:10:11.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.645+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056ed0 con 0x555b6b053f60 2023-11-15T03:10:11.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.649+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v129'177 uv177 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.649+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0572d0 con 0x555b6b053f60 2023-11-15T03:10:11.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.653+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v129'178 uv178 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.653+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0572d0 con 0x555b6b053f60 2023-11-15T03:10:11.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.657+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v129'172 uv171 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.657+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b056ed0 con 0x555b6b053f60 2023-11-15T03:10:11.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.661+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v129'173 uv173 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.661+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b056f10 con 0x555b6b053f60 2023-11-15T03:10:11.672 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:10:11.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.665+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v129'174 uv174 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.665+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b057310 con 0x555b6b053f60 2023-11-15T03:10:11.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.665+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v129'175 uv175 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.669+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b059c90 con 0x555b6b053f60 2023-11-15T03:10:11.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.669+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v129'176 uv176 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.669+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x555b6b059e70 0x555b6b05a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.669+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0597e0 con 0x555b6b059e70 2023-11-15T03:10:11.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.669+0000 7f5bc4ba2700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x555b6b059e70 0x555b6b05a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.673+0000 7f5bc4ba2700 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x555b6b059e70 0x555b6b05a2b0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:11.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.673+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v129'138 uv137 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b059e70 2023-11-15T03:10:11.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.673+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b0597e0 con 0x555b6b059e70 2023-11-15T03:10:11.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.677+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v129'139 uv139 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b059e70 2023-11-15T03:10:11.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.677+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b0597e0 con 0x555b6b059e70 2023-11-15T03:10:11.688 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:10:11.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.681+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v129'140 uv140 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b059e70 2023-11-15T03:10:11.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.681+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05a7a0 con 0x555b6b059e70 2023-11-15T03:10:11.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.685+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v129'141 uv141 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b059e70 2023-11-15T03:10:11.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.685+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05a7a0 con 0x555b6b053f60 2023-11-15T03:10:11.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.689+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v129'207 uv206 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.689+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b05a7a0 con 0x555b6b053f60 2023-11-15T03:10:11.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.689+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v129'208 uv208 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.689+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05a7a0 con 0x555b6b053f60 2023-11-15T03:10:11.702 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:10:11.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.693+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v129'209 uv209 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.693+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05b7b0 con 0x555b6b053f60 2023-11-15T03:10:11.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.697+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v129'210 uv210 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.697+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05b7b0 con 0x555b6b053f60 2023-11-15T03:10:11.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.697+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v129'211 uv211 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.697+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05a7a0 con 0x555b6b053f60 2023-11-15T03:10:11.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.701+0000 7f5bbbba0700 1 -- 172.21.15.121:0/1583301038 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v129'212 uv212 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bac0089d0 con 0x555b6b053f60 2023-11-15T03:10:11.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.701+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05a7a0 con 0x555b6b04d6d0 2023-11-15T03:10:11.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.705+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v129'176 uv175 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.705+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b05a7a0 con 0x555b6b04d6d0 2023-11-15T03:10:11.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.709+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v129'177 uv177 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.709+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05a7a0 con 0x555b6b04d6d0 2023-11-15T03:10:11.717 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:10:11.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.709+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v129'178 uv178 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.709+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05a7a0 con 0x555b6b04d6d0 2023-11-15T03:10:11.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.713+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v129'179 uv179 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.713+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05a7a0 con 0x555b6b04d6d0 2023-11-15T03:10:11.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.717+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v129'180 uv180 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.717+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05a7a0 con 0x555b6b04d6d0 2023-11-15T03:10:11.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.717+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v129'415 uv407 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.717+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b05a7a0 con 0x555b6b04d6d0 2023-11-15T03:10:11.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.721+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v129'416 uv416 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.721+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05a7a0 con 0x555b6b04d6d0 2023-11-15T03:10:11.735 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:10:11.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.725+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v129'417 uv417 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.725+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05a7a0 con 0x555b6b04d6d0 2023-11-15T03:10:11.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.729+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v129'418 uv418 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.729+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05a7a0 con 0x555b6b04d6d0 2023-11-15T03:10:11.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.733+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v129'419 uv419 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.733+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05a7a0 con 0x555b6b04d6d0 2023-11-15T03:10:11.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.737+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v129'420 uv414 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.737+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e129) v8 -- 0x555b6b05a7a0 con 0x555b6b04d6d0 2023-11-15T03:10:11.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.737+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v129'421 uv421 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.737+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05a7a0 con 0x555b6b04d6d0 2023-11-15T03:10:11.748 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:10:11.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.741+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v129'422 uv422 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.741+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05b7b0 con 0x555b6b04d6d0 2023-11-15T03:10:11.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.745+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v129'423 uv423 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.745+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05b7b0 con 0x555b6b04d6d0 2023-11-15T03:10:11.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.745+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v129'424 uv424 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.745+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05a7a0 con 0x555b6b04d6d0 2023-11-15T03:10:11.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.749+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v129'425 uv425 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.749+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x555b6b05ac90 con 0x555b6b04d6d0 2023-11-15T03:10:11.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v129'426 uv426 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5bbc012040 con 0x555b6b04d6d0 2023-11-15T03:10:11.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x555b6b048dd0 msgr2=0x555b6b04b1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x555b6b048dd0 0x555b6b04b1c0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x555b6af4d6c0 msgr2=0x555b6af79ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x555b6af4d6c0 0x555b6af79ce0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x555b6b050b30 msgr2=0x555b6b052f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x555b6b050b30 0x555b6b052f20 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x555b6b043d60 msgr2=0x555b6b046150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x555b6b043d60 0x555b6b046150 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x555b6b059e70 msgr2=0x555b6b05a2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x555b6b059e70 0x555b6b05a2b0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x555b6b04d6d0 msgr2=0x555b6b04fac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x555b6b04d6d0 0x555b6b04fac0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x555b6b053f60 msgr2=0x555b6b056380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x555b6b053f60 0x555b6b056380 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 reap_dead start 2023-11-15T03:10:11.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 reap_dead start 2023-11-15T03:10:11.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 reap_dead start 2023-11-15T03:10:11.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5ba4059020 msgr2=0x7f5ba405b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bc4ba2700 1 -- 172.21.15.121:0/1583301038 reap_dead start 2023-11-15T03:10:11.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5ba4059020 0x7f5ba405b4e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f5bac005c00 tx=0x7f5bac014000).stop 2023-11-15T03:10:11.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b6af77bd0 msgr2=0x555b6b0203e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b6af77bd0 0x555b6b0203e0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 rx=0x7f5bb4001ca0 tx=0x7f5bb4014000).stop 2023-11-15T03:10:11.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 shutdown_connections 2023-11-15T03:10:11.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5ba4059020 0x7f5ba405b4e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 --2- 172.21.15.121:0/1583301038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b6af77bd0 0x555b6b0203e0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 >> 172.21.15.121:0/1583301038 conn(0x555b6af94f80 msgr2=0x555b6b024350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:11.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 shutdown_connections 2023-11-15T03:10:11.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.753+0000 7f5bcfd33d00 1 -- 172.21.15.121:0/1583301038 wait complete. 2023-11-15T03:10:11.768 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 600.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-11-15T03:10:11.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.865+0000 7f8fd1735d00 1 Processor -- start 2023-11-15T03:10:11.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.865+0000 7f8fd1735d00 1 -- start start 2023-11-15T03:10:11.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.865+0000 7f8fd1735d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556769e56920 0x556769e9b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.865+0000 7f8fd1735d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556769e9e2f0 0x556769f37d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.865+0000 7f8fd1735d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556769f2ca20 0x556769f38210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.865+0000 7f8fd1735d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556769dcb220 con 0x556769e9e2f0 2023-11-15T03:10:11.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.865+0000 7f8fd1735d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556769e9c0b0 con 0x556769e56920 2023-11-15T03:10:11.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.865+0000 7f8fd1735d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556769e56fe0 con 0x556769f2ca20 2023-11-15T03:10:11.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.865+0000 7f8fbd444700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556769e9e2f0 0x556769f37d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.865+0000 7f8fbd444700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556769e9e2f0 0x556769f37d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59116/0 (socket says 172.21.15.121:59116) 2023-11-15T03:10:11.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.865+0000 7f8fbd444700 1 -- 172.21.15.121:0/1651244397 learned_addr learned my addr 172.21.15.121:0/1651244397 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:11.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.865+0000 7f8fc6446700 1 --2- 172.21.15.121:0/1651244397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556769f2ca20 0x556769f38210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.865+0000 7f8fc5c45700 1 --2- 172.21.15.121:0/1651244397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556769e56920 0x556769e9b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.865+0000 7f8fbd444700 1 -- 172.21.15.121:0/1651244397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556769f2ca20 msgr2=0x556769f38210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:11.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.865+0000 7f8fbd444700 1 --2- 172.21.15.121:0/1651244397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556769f2ca20 0x556769f38210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.869+0000 7f8fbd444700 1 -- 172.21.15.121:0/1651244397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556769e56920 msgr2=0x556769e9b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:11.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.869+0000 7f8fbd444700 1 --2- 172.21.15.121:0/1651244397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556769e56920 0x556769e9b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.869+0000 7f8fbd444700 1 -- 172.21.15.121:0/1651244397 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb800a0e0 con 0x556769e9e2f0 2023-11-15T03:10:11.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.869+0000 7f8fc5c45700 1 --2- 172.21.15.121:0/1651244397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556769e56920 0x556769e9b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:11.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.869+0000 7f8fbd444700 1 --2- 172.21.15.121:0/1651244397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556769e9e2f0 0x556769f37d20 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 rx=0x7f8fb000b260 tx=0x7f8fb00017d0).ready entity=mon.0 client_cookie=771f7c288ba45fa4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:11.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.869+0000 7f8fc4c43700 1 -- 172.21.15.121:0/1651244397 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb000f450 con 0x556769e9e2f0 2023-11-15T03:10:11.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.869+0000 7f8fc4c43700 1 -- 172.21.15.121:0/1651244397 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb000f5b0 con 0x556769e9e2f0 2023-11-15T03:10:11.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.869+0000 7f8fc4c43700 1 -- 172.21.15.121:0/1651244397 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb000f930 con 0x556769e9e2f0 2023-11-15T03:10:11.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.869+0000 7f8fd1735d00 1 -- 172.21.15.121:0/1651244397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556769e9e2f0 msgr2=0x556769f37d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.869+0000 7f8fd1735d00 1 --2- 172.21.15.121:0/1651244397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556769e9e2f0 0x556769f37d20 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 rx=0x7f8fb000b260 tx=0x7f8fb00017d0).stop 2023-11-15T03:10:11.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.869+0000 7f8fd1735d00 1 -- 172.21.15.121:0/1651244397 shutdown_connections 2023-11-15T03:10:11.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.869+0000 7f8fd1735d00 1 --2- 172.21.15.121:0/1651244397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556769f2ca20 0x556769f38210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.869+0000 7f8fd1735d00 1 --2- 172.21.15.121:0/1651244397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556769e9e2f0 0x556769f37d20 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.869+0000 7f8fd1735d00 1 --2- 172.21.15.121:0/1651244397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556769e56920 0x556769e9b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.869+0000 7f8fd1735d00 1 -- 172.21.15.121:0/1651244397 >> 172.21.15.121:0/1651244397 conn(0x556769e9c8f0 msgr2=0x556769e55220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:11.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.869+0000 7f8fd1735d00 1 -- 172.21.15.121:0/1651244397 shutdown_connections 2023-11-15T03:10:11.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.869+0000 7f8fd1735d00 1 -- 172.21.15.121:0/1651244397 wait complete. 2023-11-15T03:10:11.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.869+0000 7f8fd1735d00 1 Processor -- start 2023-11-15T03:10:11.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.873+0000 7f8fd1735d00 1 -- start start 2023-11-15T03:10:11.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.873+0000 7f8fd1735d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556769e56920 0x556769e9b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.873+0000 7f8fd1735d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556769e9e2f0 0x556769e85820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.873+0000 7f8fd1735d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556769f2ca20 0x556769e54970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.873+0000 7f8fd1735d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556769f2be40 con 0x556769f2ca20 2023-11-15T03:10:11.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.873+0000 7f8fd1735d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556769e85390 con 0x556769e56920 2023-11-15T03:10:11.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.873+0000 7f8fd1735d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556769e977e0 con 0x556769e9e2f0 2023-11-15T03:10:11.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.873+0000 7f8fc6446700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556769f2ca20 0x556769e54970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.873+0000 7f8fc6446700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556769f2ca20 0x556769e54970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59118/0 (socket says 172.21.15.121:59118) 2023-11-15T03:10:11.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.873+0000 7f8fc6446700 1 -- 172.21.15.121:0/2280359492 learned_addr learned my addr 172.21.15.121:0/2280359492 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:11.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.873+0000 7f8fc5c45700 1 --2- 172.21.15.121:0/2280359492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556769e56920 0x556769e9b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.873+0000 7f8fbd444700 1 --2- 172.21.15.121:0/2280359492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556769e9e2f0 0x556769e85820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.873+0000 7f8fc6446700 1 -- 172.21.15.121:0/2280359492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556769e9e2f0 msgr2=0x556769e85820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.873+0000 7f8fc6446700 1 --2- 172.21.15.121:0/2280359492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556769e9e2f0 0x556769e85820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.873+0000 7f8fc6446700 1 -- 172.21.15.121:0/2280359492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556769e56920 msgr2=0x556769e9b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.873+0000 7f8fc6446700 1 --2- 172.21.15.121:0/2280359492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556769e56920 0x556769e9b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.873+0000 7f8fc6446700 1 -- 172.21.15.121:0/2280359492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb0001ab0 con 0x556769f2ca20 2023-11-15T03:10:11.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.873+0000 7f8fc6446700 1 --2- 172.21.15.121:0/2280359492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556769f2ca20 0x556769e54970 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 rx=0x7f8fc0010000 tx=0x7f8fc0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:11.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.877+0000 7f8fbeffd700 1 -- 172.21.15.121:0/2280359492 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc001f670 con 0x556769f2ca20 2023-11-15T03:10:11.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.877+0000 7f8fbeffd700 1 -- 172.21.15.121:0/2280359492 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc001f7d0 con 0x556769f2ca20 2023-11-15T03:10:11.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.877+0000 7f8fd1735d00 1 -- 172.21.15.121:0/2280359492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc0005db0 con 0x556769f2ca20 2023-11-15T03:10:11.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.877+0000 7f8fbeffd700 1 -- 172.21.15.121:0/2280359492 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc001fb70 con 0x556769f2ca20 2023-11-15T03:10:11.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.877+0000 7f8fd1735d00 1 -- 172.21.15.121:0/2280359492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556769df4770 con 0x556769f2ca20 2023-11-15T03:10:11.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.877+0000 7f8fbeffd700 1 -- 172.21.15.121:0/2280359492 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8fc0029460 con 0x556769f2ca20 2023-11-15T03:10:11.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.881+0000 7f8fbeffd700 1 --2- 172.21.15.121:0/2280359492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8fa8059020 0x7f8fa805b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.881+0000 7f8fbeffd700 1 -- 172.21.15.121:0/2280359492 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 9276+0+0 (secure 0 0 0) 0x7f8fc0076240 con 0x556769f2ca20 2023-11-15T03:10:11.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.881+0000 7f8fc5c45700 1 --2- 172.21.15.121:0/2280359492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8fa8059020 0x7f8fa805b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.881+0000 7f8fd1735d00 1 --2- 172.21.15.121:0/2280359492 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x556769e85d10 0x556769f2df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:11.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.881+0000 7f8fd1735d00 1 -- 172.21.15.121:0/2280359492 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:1 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected e129) v8 -- 0x556769e99da0 con 0x556769e85d10 2023-11-15T03:10:11.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.881+0000 7f8fc5c45700 1 --2- 172.21.15.121:0/2280359492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8fa8059020 0x7f8fa805b4e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7f8fb800ad20 tx=0x7f8fb8007040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:11.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.881+0000 7f8fbd444700 1 --2- 172.21.15.121:0/2280359492 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x556769e85d10 0x556769f2df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:11.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.881+0000 7f8fbd444700 1 --2- 172.21.15.121:0/2280359492 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x556769e85d10 0x556769f2df80 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:11.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.885+0000 7f8fbd444700 1 -- 172.21.15.121:0/2280359492 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(1 600.00000000 [omap-set-header] v129'672 uv672 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8fb0000bc0 con 0x556769e85d10 2023-11-15T03:10:11.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.885+0000 7f8fd1735d00 1 -- 172.21.15.121:0/2280359492 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x556769e85d10 msgr2=0x556769f2df80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.885+0000 7f8fd1735d00 1 --2- 172.21.15.121:0/2280359492 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x556769e85d10 0x556769f2df80 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:11.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.885+0000 7f8fd1735d00 1 -- 172.21.15.121:0/2280359492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8fa8059020 msgr2=0x7f8fa805b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.885+0000 7f8fd1735d00 1 --2- 172.21.15.121:0/2280359492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8fa8059020 0x7f8fa805b4e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7f8fb800ad20 tx=0x7f8fb8007040).stop 2023-11-15T03:10:11.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.885+0000 7f8fd1735d00 1 -- 172.21.15.121:0/2280359492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556769f2ca20 msgr2=0x556769e54970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:11.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.885+0000 7f8fd1735d00 1 --2- 172.21.15.121:0/2280359492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556769f2ca20 0x556769e54970 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 rx=0x7f8fc0010000 tx=0x7f8fc0016000).stop 2023-11-15T03:10:11.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.885+0000 7f8fc6446700 1 -- 172.21.15.121:0/2280359492 reap_dead start 2023-11-15T03:10:11.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.885+0000 7f8fd1735d00 1 -- 172.21.15.121:0/2280359492 shutdown_connections 2023-11-15T03:10:11.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.885+0000 7f8fd1735d00 1 -- 172.21.15.121:0/2280359492 >> 172.21.15.121:0/2280359492 conn(0x556769e9c8f0 msgr2=0x556769f22660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:11.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.885+0000 7f8fd1735d00 1 -- 172.21.15.121:0/2280359492 shutdown_connections 2023-11-15T03:10:11.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:11.885+0000 7f8fd1735d00 1 -- 172.21.15.121:0/2280359492 wait complete. 2023-11-15T03:10:11.899 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:10:12.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.057+0000 7fa5e2707700 1 Processor -- start 2023-11-15T03:10:12.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.057+0000 7fa5e2707700 1 -- start start 2023-11-15T03:10:12.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.061+0000 7fa5e2707700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa5dc1528c0 0x7fa5dc0bee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:12.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.061+0000 7fa5e2707700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa5dc0bf310 0x7fa5dc0c3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:12.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.061+0000 7fa5e2707700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5dc0be990 con 0x7fa5dc0c39f0 2023-11-15T03:10:12.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.061+0000 7fa5e2707700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5dc0c3e40 con 0x7fa5dc0bf310 2023-11-15T03:10:12.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.061+0000 7fa5e2707700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5dc0bdd80 con 0x7fa5dc1528c0 2023-11-15T03:10:12.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.061+0000 7fa5e1705700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa5dc1528c0 0x7fa5dc0bee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:12.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.061+0000 7fa5e1f06700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa5dc0c39f0 0x7fa5dc0c5150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:54460/0 (socket says 172.21.15.121:54460) 2023-11-15T03:10:12.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.061+0000 7fa5e1f06700 1 -- 172.21.15.121:0/2954990084 learned_addr learned my addr 172.21.15.121:0/2954990084 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:12.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.061+0000 7fa5e0f04700 1 --2- 172.21.15.121:0/2954990084 >> v2:172.21.15.121:3300/0 conn(0x7fa5dc0bf310 0x7fa5dc0c3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:12.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.061+0000 7fa5d3fff700 1 -- 172.21.15.121:0/2954990084 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167128691 0 0) 0x7fa5dc0be990 con 0x7fa5dc0c39f0 2023-11-15T03:10:12.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.061+0000 7fa5d3fff700 1 -- 172.21.15.121:0/2954990084 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5c80033f0 con 0x7fa5dc0c39f0 2023-11-15T03:10:12.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.061+0000 7fa5e1705700 1 -- 172.21.15.121:0/2954990084 >> v2:172.21.15.121:3300/0 conn(0x7fa5dc0bf310 msgr2=0x7fa5dc0c3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:12.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.061+0000 7fa5e1705700 1 --2- 172.21.15.121:0/2954990084 >> v2:172.21.15.121:3300/0 conn(0x7fa5dc0bf310 0x7fa5dc0c3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:12.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.061+0000 7fa5e1705700 1 -- 172.21.15.121:0/2954990084 >> v1:172.21.15.121:6789/0 conn(0x7fa5dc0c39f0 legacy=0x7fa5dc0c5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:12.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.061+0000 7fa5e1705700 1 -- 172.21.15.121:0/2954990084 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5dc0be060 con 0x7fa5dc1528c0 2023-11-15T03:10:12.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.061+0000 7fa5e1705700 1 --2- 172.21.15.121:0/2954990084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5dc1528c0 0x7fa5dc0bee20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7fa5dc066e50 tx=0x7fa5d8009060).ready entity=mon.1 client_cookie=a48f647571dde074 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:12.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.065+0000 7fa5d3fff700 1 -- 172.21.15.121:0/2954990084 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d800d4a0 con 0x7fa5dc1528c0 2023-11-15T03:10:12.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.065+0000 7fa5d3fff700 1 -- 172.21.15.121:0/2954990084 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d800d600 con 0x7fa5dc1528c0 2023-11-15T03:10:12.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.065+0000 7fa5d3fff700 1 -- 172.21.15.121:0/2954990084 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d800d980 con 0x7fa5dc1528c0 2023-11-15T03:10:12.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.065+0000 7fa5e2707700 1 -- 172.21.15.121:0/2954990084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5dc1528c0 msgr2=0x7fa5dc0bee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:12.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.065+0000 7fa5e2707700 1 --2- 172.21.15.121:0/2954990084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5dc1528c0 0x7fa5dc0bee20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7fa5dc066e50 tx=0x7fa5d8009060).stop 2023-11-15T03:10:12.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.065+0000 7fa5e2707700 1 -- 172.21.15.121:0/2954990084 shutdown_connections 2023-11-15T03:10:12.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.065+0000 7fa5e2707700 1 --2- 172.21.15.121:0/2954990084 >> v2:172.21.15.121:3300/0 conn(0x7fa5dc0bf310 0x7fa5dc0c3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:12.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.065+0000 7fa5e2707700 1 --2- 172.21.15.121:0/2954990084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5dc1528c0 0x7fa5dc0bee20 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:12.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.065+0000 7fa5e2707700 1 -- 172.21.15.121:0/2954990084 >> 172.21.15.121:0/2954990084 conn(0x7fa5dc067a60 msgr2=0x7fa5dc067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:12.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.065+0000 7fa5e2707700 1 -- 172.21.15.121:0/2954990084 shutdown_connections 2023-11-15T03:10:12.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.065+0000 7fa5e2707700 1 -- 172.21.15.121:0/2954990084 wait complete. 2023-11-15T03:10:12.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.065+0000 7fa5e2707700 1 Processor -- start 2023-11-15T03:10:12.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.065+0000 7fa5e2707700 1 -- start start 2023-11-15T03:10:12.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e2707700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5dc0bf310 0x7fa5dc0ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:12.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e2707700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5dc0c39f0 0x7fa5dc0ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:12.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e2707700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5dc1528c0 0x7fa5dc0add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:12.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e2707700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc0ae230 con 0x7fa5dc0c39f0 2023-11-15T03:10:12.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e2707700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc0abba0 con 0x7fa5dc0bf310 2023-11-15T03:10:12.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e2707700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc0abdb0 con 0x7fa5dc1528c0 2023-11-15T03:10:12.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e0f04700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5dc0c39f0 0x7fa5dc0ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:12.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e0f04700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5dc0c39f0 0x7fa5dc0ad850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59148/0 (socket says 172.21.15.121:59148) 2023-11-15T03:10:12.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e0f04700 1 -- 172.21.15.121:0/412939121 learned_addr learned my addr 172.21.15.121:0/412939121 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:12.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e1705700 1 --2- 172.21.15.121:0/412939121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5dc0bf310 0x7fa5dc0ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:12.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e1f06700 1 --2- 172.21.15.121:0/412939121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5dc1528c0 0x7fa5dc0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:12.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e0f04700 1 -- 172.21.15.121:0/412939121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5dc1528c0 msgr2=0x7fa5dc0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:12.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e0f04700 1 --2- 172.21.15.121:0/412939121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5dc1528c0 0x7fa5dc0add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:12.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e0f04700 1 -- 172.21.15.121:0/412939121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5dc0bf310 msgr2=0x7fa5dc0ab6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:12.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e0f04700 1 --2- 172.21.15.121:0/412939121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5dc0bf310 0x7fa5dc0ab6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:12.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e0f04700 1 -- 172.21.15.121:0/412939121 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d80092a0 con 0x7fa5dc0c39f0 2023-11-15T03:10:12.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e1f06700 1 --2- 172.21.15.121:0/412939121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5dc1528c0 0x7fa5dc0add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:10:12.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e0f04700 1 --2- 172.21.15.121:0/412939121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5dc0c39f0 0x7fa5dc0ad850 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 rx=0x7fa5cc00e950 tx=0x7fa5cc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:12.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5d17fa700 1 -- 172.21.15.121:0/412939121 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc00c850 con 0x7fa5dc0c39f0 2023-11-15T03:10:12.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5d17fa700 1 -- 172.21.15.121:0/412939121 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc00c9b0 con 0x7fa5dc0c39f0 2023-11-15T03:10:12.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e2707700 1 -- 172.21.15.121:0/412939121 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5dc0ac090 con 0x7fa5dc0c39f0 2023-11-15T03:10:12.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5d17fa700 1 -- 172.21.15.121:0/412939121 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc00cd30 con 0x7fa5dc0c39f0 2023-11-15T03:10:12.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.069+0000 7fa5e2707700 1 -- 172.21.15.121:0/412939121 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5dc0ac650 con 0x7fa5dc0c39f0 2023-11-15T03:10:12.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.073+0000 7fa5e2707700 1 -- 172.21.15.121:0/412939121 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5dc06bcf0 con 0x7fa5dc0c39f0 2023-11-15T03:10:12.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.073+0000 7fa5d17fa700 1 -- 172.21.15.121:0/412939121 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa5cc012600 con 0x7fa5dc0c39f0 2023-11-15T03:10:12.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.073+0000 7fa5d17fa700 1 --2- 172.21.15.121:0/412939121 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa5c005a510 0x7fa5c005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:12.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.073+0000 7fa5d17fa700 1 -- 172.21.15.121:0/412939121 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 9276+0+0 (secure 0 0 0) 0x7fa5cc075ad0 con 0x7fa5dc0c39f0 2023-11-15T03:10:12.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.073+0000 7fa5e1705700 1 --2- 172.21.15.121:0/412939121 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa5c005a510 0x7fa5c005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:12.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.077+0000 7fa5e1705700 1 --2- 172.21.15.121:0/412939121 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa5c005a510 0x7fa5c005c9d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7fa5d8009fd0 tx=0x7fa5d8016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:12.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.077+0000 7fa5d17fa700 1 -- 172.21.15.121:0/412939121 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa5cc0485a0 con 0x7fa5dc0c39f0 2023-11-15T03:10:12.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:12.273+0000 7fa5e2707700 1 -- 172.21.15.121:0/412939121 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fa5dc068a70 con 0x7fa5dc0c39f0 2023-11-15T03:10:12.435 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:10:12.443 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:10:13.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.193+0000 7fa5d17fa700 1 -- 172.21.15.121:0/412939121 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v307) v1 ==== 162+0+0 (secure 0 0 0) 0x7fa5cc016070 con 0x7fa5dc0c39f0 2023-11-15T03:10:13.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.197+0000 7fa5e2707700 1 -- 172.21.15.121:0/412939121 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa5c005a510 msgr2=0x7fa5c005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:13.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.197+0000 7fa5e2707700 1 --2- 172.21.15.121:0/412939121 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa5c005a510 0x7fa5c005c9d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7fa5d8009fd0 tx=0x7fa5d8016000).stop 2023-11-15T03:10:13.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.197+0000 7fa5e2707700 1 -- 172.21.15.121:0/412939121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5dc0c39f0 msgr2=0x7fa5dc0ad850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:13.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.197+0000 7fa5e2707700 1 --2- 172.21.15.121:0/412939121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5dc0c39f0 0x7fa5dc0ad850 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 rx=0x7fa5cc00e950 tx=0x7fa5cc009060).stop 2023-11-15T03:10:13.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.197+0000 7fa5e2707700 1 -- 172.21.15.121:0/412939121 shutdown_connections 2023-11-15T03:10:13.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.197+0000 7fa5e2707700 1 --2- 172.21.15.121:0/412939121 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa5c005a510 0x7fa5c005c9d0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:13.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.197+0000 7fa5e2707700 1 --2- 172.21.15.121:0/412939121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5dc1528c0 0x7fa5dc0add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:13.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.197+0000 7fa5e2707700 1 --2- 172.21.15.121:0/412939121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5dc0c39f0 0x7fa5dc0ad850 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:13.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.197+0000 7fa5e2707700 1 --2- 172.21.15.121:0/412939121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5dc0bf310 0x7fa5dc0ab6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:13.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.197+0000 7fa5e2707700 1 -- 172.21.15.121:0/412939121 >> 172.21.15.121:0/412939121 conn(0x7fa5dc067a60 msgr2=0x7fa5dc0a8460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:13.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.197+0000 7fa5e2707700 1 -- 172.21.15.121:0/412939121 shutdown_connections 2023-11-15T03:10:13.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.197+0000 7fa5e2707700 1 -- 172.21.15.121:0/412939121 wait complete. 2023-11-15T03:10:13.205 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:10:13.224 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:10:13.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.377+0000 7f162e769700 1 Processor -- start 2023-11-15T03:10:13.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.377+0000 7f162e769700 1 -- start start 2023-11-15T03:10:13.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.377+0000 7f162e769700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1628068170 0x7f162815ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:13.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.377+0000 7f162e769700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16280cb5c0 con 0x7f16280aa240 2023-11-15T03:10:13.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.377+0000 7f162e769700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16280c9ca0 con 0x7f1628153bb0 2023-11-15T03:10:13.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.377+0000 7f162e769700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16280aa6f0 con 0x7f1628068170 2023-11-15T03:10:13.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.381+0000 7f161f7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f16280aa240 0x7f1628067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:54464/0 (socket says 172.21.15.121:54464) 2023-11-15T03:10:13.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.381+0000 7f161f7fe700 1 -- 172.21.15.121:0/3719304569 learned_addr learned my addr 172.21.15.121:0/3719304569 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:13.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.381+0000 7f162cd06700 1 --2- 172.21.15.121:0/3719304569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1628068170 0x7f162815ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:13.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.381+0000 7f162cd06700 1 -- 172.21.15.121:0/3719304569 >> v1:172.21.15.190:6789/0 conn(0x7f1628153bb0 legacy=0x7f16280a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:13.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.381+0000 7f162cd06700 1 -- 172.21.15.121:0/3719304569 >> v1:172.21.15.121:6789/0 conn(0x7f16280aa240 legacy=0x7f1628067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:13.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.381+0000 7f162cd06700 1 -- 172.21.15.121:0/3719304569 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16280caa20 con 0x7f1628068170 2023-11-15T03:10:13.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.381+0000 7f162cd06700 1 --2- 172.21.15.121:0/3719304569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1628068170 0x7f162815ec40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f1620009e90 tx=0x7f1620009780).ready entity=mon.2 client_cookie=81d1b7c7d7ac86df server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:13.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.381+0000 7f16277fe700 1 -- 172.21.15.121:0/3719304569 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162000e070 con 0x7f1628068170 2023-11-15T03:10:13.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.381+0000 7f16277fe700 1 -- 172.21.15.121:0/3719304569 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162000bb30 con 0x7f1628068170 2023-11-15T03:10:13.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.381+0000 7f16277fe700 1 -- 172.21.15.121:0/3719304569 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162000beb0 con 0x7f1628068170 2023-11-15T03:10:13.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.381+0000 7f162e769700 1 -- 172.21.15.121:0/3719304569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1628068170 msgr2=0x7f162815ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:13.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.381+0000 7f162e769700 1 --2- 172.21.15.121:0/3719304569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1628068170 0x7f162815ec40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f1620009e90 tx=0x7f1620009780).stop 2023-11-15T03:10:13.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.381+0000 7f162e769700 1 -- 172.21.15.121:0/3719304569 shutdown_connections 2023-11-15T03:10:13.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.381+0000 7f162e769700 1 --2- 172.21.15.121:0/3719304569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1628068170 0x7f162815ec40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:13.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.381+0000 7f162e769700 1 -- 172.21.15.121:0/3719304569 >> 172.21.15.121:0/3719304569 conn(0x7f1628058c50 msgr2=0x7f1628059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:13.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.381+0000 7f162e769700 1 -- 172.21.15.121:0/3719304569 shutdown_connections 2023-11-15T03:10:13.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.381+0000 7f162e769700 1 -- 172.21.15.121:0/3719304569 wait complete. 2023-11-15T03:10:13.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.385+0000 7f162e769700 1 Processor -- start 2023-11-15T03:10:13.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.385+0000 7f162e769700 1 -- start start 2023-11-15T03:10:13.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f162e769700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1628068170 0x7f16281677e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:13.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f162e769700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16280aa240 0x7f1628165260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:13.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f162e769700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1628153bb0 0x7f1628165ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:13.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f162e769700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1628167cd0 con 0x7f16280aa240 2023-11-15T03:10:13.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f162e769700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16280c1cc0 con 0x7f1628153bb0 2023-11-15T03:10:13.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f162e769700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16280c2b20 con 0x7f1628068170 2023-11-15T03:10:13.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f161f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16280aa240 0x7f1628165260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:13.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f161f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16280aa240 0x7f1628165260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59150/0 (socket says 172.21.15.121:59150) 2023-11-15T03:10:13.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f161f7fe700 1 -- 172.21.15.121:0/1861061652 learned_addr learned my addr 172.21.15.121:0/1861061652 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:13.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f1627fff700 1 --2- 172.21.15.121:0/1861061652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1628068170 0x7f16281677e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:13.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f162cd06700 1 --2- 172.21.15.121:0/1861061652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1628153bb0 0x7f1628165ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:13.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f161f7fe700 1 -- 172.21.15.121:0/1861061652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1628068170 msgr2=0x7f16281677e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:13.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f161f7fe700 1 --2- 172.21.15.121:0/1861061652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1628068170 0x7f16281677e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:13.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f161f7fe700 1 -- 172.21.15.121:0/1861061652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1628153bb0 msgr2=0x7f1628165ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:13.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f161f7fe700 1 --2- 172.21.15.121:0/1861061652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1628153bb0 0x7f1628165ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:13.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f161f7fe700 1 -- 172.21.15.121:0/1861061652 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1620009a20 con 0x7f16280aa240 2023-11-15T03:10:13.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f162cd06700 1 --2- 172.21.15.121:0/1861061652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1628153bb0 0x7f1628165ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:13.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f161f7fe700 1 --2- 172.21.15.121:0/1861061652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16280aa240 0x7f1628165260 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 rx=0x7f161000c630 tx=0x7f1610007d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:13.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f1624ff9700 1 -- 172.21.15.121:0/1861061652 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16100072e0 con 0x7f16280aa240 2023-11-15T03:10:13.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f1624ff9700 1 -- 172.21.15.121:0/1861061652 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1610007440 con 0x7f16280aa240 2023-11-15T03:10:13.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f162e769700 1 -- 172.21.15.121:0/1861061652 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16280c1f10 con 0x7f16280aa240 2023-11-15T03:10:13.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f1624ff9700 1 -- 172.21.15.121:0/1861061652 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16100057a0 con 0x7f16280aa240 2023-11-15T03:10:13.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.389+0000 7f162e769700 1 -- 172.21.15.121:0/1861061652 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16280c24d0 con 0x7f16280aa240 2023-11-15T03:10:13.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.393+0000 7f162e769700 1 -- 172.21.15.121:0/1861061652 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f162806c150 con 0x7f16280aa240 2023-11-15T03:10:13.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.393+0000 7f1624ff9700 1 -- 172.21.15.121:0/1861061652 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1610005b90 con 0x7f16280aa240 2023-11-15T03:10:13.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.393+0000 7f1624ff9700 1 --2- 172.21.15.121:0/1861061652 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f160805a540 0x7f160805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:13.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.393+0000 7f1624ff9700 1 -- 172.21.15.121:0/1861061652 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 9276+0+0 (secure 0 0 0) 0x7f1610074e70 con 0x7f16280aa240 2023-11-15T03:10:13.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.393+0000 7f1627fff700 1 --2- 172.21.15.121:0/1861061652 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f160805a540 0x7f160805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:13.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.397+0000 7f1627fff700 1 --2- 172.21.15.121:0/1861061652 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f160805a540 0x7f160805ca00 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f16280aa210 tx=0x7f1618009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:13.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.401+0000 7f1624ff9700 1 -- 172.21.15.121:0/1861061652 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1610047940 con 0x7f16280aa240 2023-11-15T03:10:13.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.597+0000 7f162e769700 1 -- 172.21.15.121:0/1861061652 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16280c9f70 con 0x7f16280aa240 2023-11-15T03:10:13.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.597+0000 7f1624ff9700 1 -- 172.21.15.121:0/1861061652 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 307 v307) v1 ==== 95+0+4508 (secure 0 0 0) 0x7f1610018020 con 0x7f16280aa240 2023-11-15T03:10:13.605 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:10:13.605 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":307,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":307,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:10:13.183978+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10206},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10206":{"gid":10206,"name":"a","rank":0,"incarnation":307,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/2622666226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2622666226},{"type":"v1","addr":"172.21.15.121:6836","nonce":2622666226}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:10:13.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.601+0000 7f162e769700 1 -- 172.21.15.121:0/1861061652 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f160805a540 msgr2=0x7f160805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:13.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.601+0000 7f162e769700 1 --2- 172.21.15.121:0/1861061652 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f160805a540 0x7f160805ca00 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f16280aa210 tx=0x7f1618009040).stop 2023-11-15T03:10:13.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.601+0000 7f162e769700 1 -- 172.21.15.121:0/1861061652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16280aa240 msgr2=0x7f1628165260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:13.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.601+0000 7f162e769700 1 --2- 172.21.15.121:0/1861061652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16280aa240 0x7f1628165260 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 rx=0x7f161000c630 tx=0x7f1610007d80).stop 2023-11-15T03:10:13.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.601+0000 7f162e769700 1 -- 172.21.15.121:0/1861061652 shutdown_connections 2023-11-15T03:10:13.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.601+0000 7f162e769700 1 --2- 172.21.15.121:0/1861061652 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f160805a540 0x7f160805ca00 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:13.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.601+0000 7f162e769700 1 --2- 172.21.15.121:0/1861061652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1628153bb0 0x7f1628165ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:13.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.601+0000 7f162e769700 1 --2- 172.21.15.121:0/1861061652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16280aa240 0x7f1628165260 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:13.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.601+0000 7f162e769700 1 --2- 172.21.15.121:0/1861061652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1628068170 0x7f16281677e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:13.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.601+0000 7f162e769700 1 -- 172.21.15.121:0/1861061652 >> 172.21.15.121:0/1861061652 conn(0x7f1628058c50 msgr2=0x7f16280b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:13.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.601+0000 7f162e769700 1 -- 172.21.15.121:0/1861061652 shutdown_connections 2023-11-15T03:10:13.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:13.601+0000 7f162e769700 1 -- 172.21.15.121:0/1861061652 wait complete. 2023-11-15T03:10:13.613 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 307 2023-11-15T03:10:13.629 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:10:16.231 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:10:16.225+0000 7f562e38d700 -1 mds.0.cache.dir(0x600) Corrupt fnode in dirfrag 0x600: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input 2023-11-15T03:10:16.231 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:10:16.225+0000 7f562e38d700 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2023-11-15T03:10:16.231 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:10:16.225+0000 7f5631b94700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:10:17.224 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:10:17.217+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:10:17.225 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:10:17.217+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:10:17.238 INFO:tasks.ceph.mds.a.smithi121.stderr: -21> 2023-11-15T03:10:16.225+0000 7f562e38d700 -1 mds.0.cache.dir(0x600) Corrupt fnode in dirfrag 0x600: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input 2023-11-15T03:10:17.238 INFO:tasks.ceph.mds.a.smithi121.stderr: -18> 2023-11-15T03:10:16.225+0000 7f562e38d700 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2023-11-15T03:10:17.239 INFO:tasks.ceph.mds.a.smithi121.stderr: -7> 2023-11-15T03:10:16.225+0000 7f5631b94700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:10:17.278 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:10:18.630 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:10:18.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.797+0000 7f1f561f9700 1 Processor -- start 2023-11-15T03:10:18.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.797+0000 7f1f561f9700 1 -- start start 2023-11-15T03:10:18.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.801+0000 7f1f561f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500c3280 0x7f1f500c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:18.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.801+0000 7f1f561f9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1f5015be30 0x7f1f5015c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:18.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.801+0000 7f1f561f9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f500c1d50 con 0x7f1f50152140 2023-11-15T03:10:18.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.801+0000 7f1f561f9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f500c1860 con 0x7f1f5015be30 2023-11-15T03:10:18.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.801+0000 7f1f561f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5015cb80 con 0x7f1f500c3280 2023-11-15T03:10:18.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.801+0000 7f1f551f7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500c3280 0x7f1f500c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:18.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.801+0000 7f1f559f8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1f5015be30 0x7f1f5015c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:18.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.801+0000 7f1f551f7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500c3280 0x7f1f500c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:40372/0 (socket says 172.21.15.121:40372) 2023-11-15T03:10:18.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.801+0000 7f1f551f7700 1 -- 172.21.15.121:0/2007002246 learned_addr learned my addr 172.21.15.121:0/2007002246 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:18.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.801+0000 7f1f551f7700 1 -- 172.21.15.121:0/2007002246 >> v2:172.21.15.190:3300/0 conn(0x7f1f5015be30 msgr2=0x7f1f5015c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:18.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.801+0000 7f1f551f7700 1 --2- 172.21.15.121:0/2007002246 >> v2:172.21.15.190:3300/0 conn(0x7f1f5015be30 0x7f1f5015c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:18.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.801+0000 7f1f47fff700 1 -- 172.21.15.121:0/2007002246 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2291824043 0 0) 0x7f1f500c1d50 con 0x7f1f50152140 2023-11-15T03:10:18.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.801+0000 7f1f551f7700 1 -- 172.21.15.121:0/2007002246 >> v1:172.21.15.190:6789/0 conn(0x7f1f50152140 legacy=0x7f1f501525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:18.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.801+0000 7f1f551f7700 1 -- 172.21.15.121:0/2007002246 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f500a9240 con 0x7f1f500c3280 2023-11-15T03:10:18.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.801+0000 7f1f559f8700 1 --2- 172.21.15.121:0/2007002246 >> v2:172.21.15.190:3300/0 conn(0x7f1f5015be30 0x7f1f5015c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:10:18.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.801+0000 7f1f551f7700 1 --2- 172.21.15.121:0/2007002246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500c3280 0x7f1f500c3660 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f1f4c006fd0 tx=0x7f1f4c0090f0).ready entity=mon.2 client_cookie=dde91efcf041e03f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:18.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.805+0000 7f1f47fff700 1 -- 172.21.15.121:0/2007002246 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00b580 con 0x7f1f500c3280 2023-11-15T03:10:18.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.805+0000 7f1f47fff700 1 -- 172.21.15.121:0/2007002246 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c00b6e0 con 0x7f1f500c3280 2023-11-15T03:10:18.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.805+0000 7f1f47fff700 1 -- 172.21.15.121:0/2007002246 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00ba60 con 0x7f1f500c3280 2023-11-15T03:10:18.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.805+0000 7f1f561f9700 1 -- 172.21.15.121:0/2007002246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500c3280 msgr2=0x7f1f500c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:18.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.805+0000 7f1f561f9700 1 --2- 172.21.15.121:0/2007002246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500c3280 0x7f1f500c3660 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f1f4c006fd0 tx=0x7f1f4c0090f0).stop 2023-11-15T03:10:18.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.805+0000 7f1f561f9700 1 -- 172.21.15.121:0/2007002246 shutdown_connections 2023-11-15T03:10:18.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.805+0000 7f1f561f9700 1 --2- 172.21.15.121:0/2007002246 >> v2:172.21.15.190:3300/0 conn(0x7f1f5015be30 0x7f1f5015c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:18.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.805+0000 7f1f561f9700 1 --2- 172.21.15.121:0/2007002246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500c3280 0x7f1f500c3660 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:18.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.805+0000 7f1f561f9700 1 -- 172.21.15.121:0/2007002246 >> 172.21.15.121:0/2007002246 conn(0x7f1f50062190 msgr2=0x7f1f50062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:18.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.805+0000 7f1f561f9700 1 -- 172.21.15.121:0/2007002246 shutdown_connections 2023-11-15T03:10:18.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.805+0000 7f1f561f9700 1 -- 172.21.15.121:0/2007002246 wait complete. 2023-11-15T03:10:18.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.809+0000 7f1f561f9700 1 Processor -- start 2023-11-15T03:10:18.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.809+0000 7f1f561f9700 1 -- start start 2023-11-15T03:10:18.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.809+0000 7f1f561f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f500c3280 0x7f1f5014eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:18.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.809+0000 7f1f561f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f50152140 0x7f1f500c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:18.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.809+0000 7f1f561f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f5015be30 0x7f1f50151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:18.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.809+0000 7f1f561f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f500c4580 con 0x7f1f500c3280 2023-11-15T03:10:18.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.809+0000 7f1f561f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f500c4860 con 0x7f1f50152140 2023-11-15T03:10:18.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.809+0000 7f1f561f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f500c4b40 con 0x7f1f5015be30 2023-11-15T03:10:18.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.809+0000 7f1f551f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f500c3280 0x7f1f5014eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:18.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.809+0000 7f1f551f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f500c3280 0x7f1f5014eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34622/0 (socket says 172.21.15.121:34622) 2023-11-15T03:10:18.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.809+0000 7f1f551f7700 1 -- 172.21.15.121:0/2579886792 learned_addr learned my addr 172.21.15.121:0/2579886792 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:18.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.809+0000 7f1f549f6700 1 --2- 172.21.15.121:0/2579886792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f50152140 0x7f1f500c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:18.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.809+0000 7f1f559f8700 1 --2- 172.21.15.121:0/2579886792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f5015be30 0x7f1f50151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:18.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.809+0000 7f1f551f7700 1 -- 172.21.15.121:0/2579886792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f5015be30 msgr2=0x7f1f50151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:18.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.809+0000 7f1f551f7700 1 --2- 172.21.15.121:0/2579886792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f5015be30 0x7f1f50151220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:18.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.809+0000 7f1f551f7700 1 -- 172.21.15.121:0/2579886792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f50152140 msgr2=0x7f1f500c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:18.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.809+0000 7f1f551f7700 1 --2- 172.21.15.121:0/2579886792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f50152140 0x7f1f500c5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:18.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.809+0000 7f1f551f7700 1 -- 172.21.15.121:0/2579886792 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f48009a20 con 0x7f1f500c3280 2023-11-15T03:10:18.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.809+0000 7f1f549f6700 1 --2- 172.21.15.121:0/2579886792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f50152140 0x7f1f500c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:18.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.813+0000 7f1f551f7700 1 --2- 172.21.15.121:0/2579886792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f500c3280 0x7f1f5014eb80 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 rx=0x7f1f4c009fd0 tx=0x7f1f4c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:18.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.813+0000 7f1f457fa700 1 -- 172.21.15.121:0/2579886792 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c006a40 con 0x7f1f500c3280 2023-11-15T03:10:18.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.813+0000 7f1f457fa700 1 -- 172.21.15.121:0/2579886792 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c00b850 con 0x7f1f500c3280 2023-11-15T03:10:18.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.813+0000 7f1f561f9700 1 -- 172.21.15.121:0/2579886792 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f4c009370 con 0x7f1f500c3280 2023-11-15T03:10:18.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.813+0000 7f1f457fa700 1 -- 172.21.15.121:0/2579886792 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c01b660 con 0x7f1f500c3280 2023-11-15T03:10:18.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.813+0000 7f1f561f9700 1 -- 172.21.15.121:0/2579886792 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f500c5100 con 0x7f1f500c3280 2023-11-15T03:10:18.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.813+0000 7f1f561f9700 1 -- 172.21.15.121:0/2579886792 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5006bcf0 con 0x7f1f500c3280 2023-11-15T03:10:18.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.813+0000 7f1f457fa700 1 -- 172.21.15.121:0/2579886792 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1f4c01ba90 con 0x7f1f500c3280 2023-11-15T03:10:18.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.817+0000 7f1f457fa700 1 --2- 172.21.15.121:0/2579886792 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1f3405a510 0x7f1f3405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:18.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.817+0000 7f1f457fa700 1 -- 172.21.15.121:0/2579886792 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9362+0+0 (secure 0 0 0) 0x7f1f4c076f40 con 0x7f1f500c3280 2023-11-15T03:10:18.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.817+0000 7f1f549f6700 1 --2- 172.21.15.121:0/2579886792 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1f3405a510 0x7f1f3405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:18.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.817+0000 7f1f549f6700 1 --2- 172.21.15.121:0/2579886792 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1f3405a510 0x7f1f3405c9d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f1f40007fd0 tx=0x7f1f40009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:18.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:18.821+0000 7f1f457fa700 1 -- 172.21.15.121:0/2579886792 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1f4c049a30 con 0x7f1f500c3280 2023-11-15T03:10:19.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.025+0000 7f1f561f9700 1 -- 172.21.15.121:0/2579886792 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f500a7cc0 con 0x7f1f500c3280 2023-11-15T03:10:19.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.025+0000 7f1f457fa700 1 -- 172.21.15.121:0/2579886792 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 312 v312) v1 ==== 95+0+4497 (secure 0 0 0) 0x7f1f4c029030 con 0x7f1f500c3280 2023-11-15T03:10:19.032 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:10:19.033 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":312,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":10242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1750624080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1750624080},{"type":"v1","addr":"172.21.15.121:6836","nonce":1750624080}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":312}],"filesystems":[{"mdsmap":{"epoch":311,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:10:17.219328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:10:19.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.029+0000 7f1f561f9700 1 -- 172.21.15.121:0/2579886792 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1f3405a510 msgr2=0x7f1f3405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:19.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.029+0000 7f1f561f9700 1 --2- 172.21.15.121:0/2579886792 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1f3405a510 0x7f1f3405c9d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f1f40007fd0 tx=0x7f1f40009040).stop 2023-11-15T03:10:19.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.029+0000 7f1f561f9700 1 -- 172.21.15.121:0/2579886792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f500c3280 msgr2=0x7f1f5014eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:19.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.029+0000 7f1f561f9700 1 --2- 172.21.15.121:0/2579886792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f500c3280 0x7f1f5014eb80 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 rx=0x7f1f4c009fd0 tx=0x7f1f4c017000).stop 2023-11-15T03:10:19.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.029+0000 7f1f561f9700 1 -- 172.21.15.121:0/2579886792 shutdown_connections 2023-11-15T03:10:19.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.029+0000 7f1f561f9700 1 --2- 172.21.15.121:0/2579886792 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1f3405a510 0x7f1f3405c9d0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.029+0000 7f1f561f9700 1 --2- 172.21.15.121:0/2579886792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f5015be30 0x7f1f50151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.029+0000 7f1f561f9700 1 --2- 172.21.15.121:0/2579886792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f50152140 0x7f1f500c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.029+0000 7f1f561f9700 1 --2- 172.21.15.121:0/2579886792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f500c3280 0x7f1f5014eb80 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.029+0000 7f1f561f9700 1 -- 172.21.15.121:0/2579886792 >> 172.21.15.121:0/2579886792 conn(0x7f1f50062190 msgr2=0x7f1f501525a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:19.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.029+0000 7f1f561f9700 1 -- 172.21.15.121:0/2579886792 shutdown_connections 2023-11-15T03:10:19.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.029+0000 7f1f561f9700 1 -- 172.21.15.121:0/2579886792 wait complete. 2023-11-15T03:10:19.039 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 312 2023-11-15T03:10:19.055 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:10:19.056 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:10:19.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.209+0000 7f6dfcf47700 1 Processor -- start 2023-11-15T03:10:19.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.209+0000 7f6dfcf47700 1 -- start start 2023-11-15T03:10:19.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.213+0000 7f6dfcf47700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6df80c3280 0x7f6df80c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:19.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.213+0000 7f6dfcf47700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df80c1d50 con 0x7f6df8152140 2023-11-15T03:10:19.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.213+0000 7f6dfcf47700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df815cb80 con 0x7f6df815be30 2023-11-15T03:10:19.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.213+0000 7f6dfcf47700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df80c1860 con 0x7f6df80c3280 2023-11-15T03:10:19.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.213+0000 7f6df77fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6df80c3280 0x7f6df80c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:19.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.213+0000 7f6deffff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6df8152140 0x7f6df81525f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:43364/0 (socket says 172.21.15.121:43364) 2023-11-15T03:10:19.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.213+0000 7f6deffff700 1 -- 172.21.15.121:0/2809120869 learned_addr learned my addr 172.21.15.121:0/2809120869 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:19.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.213+0000 7f6df77fe700 1 -- 172.21.15.121:0/2809120869 >> v1:172.21.15.190:6789/0 conn(0x7f6df815be30 legacy=0x7f6df815c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:19.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.213+0000 7f6df6ffd700 1 -- 172.21.15.121:0/2809120869 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604982533 0 0) 0x7f6df815cb80 con 0x7f6df815be30 2023-11-15T03:10:19.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.213+0000 7f6df77fe700 1 -- 172.21.15.121:0/2809120869 >> v1:172.21.15.121:6789/0 conn(0x7f6df8152140 legacy=0x7f6df81525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:19.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.213+0000 7f6df77fe700 1 -- 172.21.15.121:0/2809120869 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df80a9240 con 0x7f6df80c3280 2023-11-15T03:10:19.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.213+0000 7f6df77fe700 1 --2- 172.21.15.121:0/2809120869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6df80c3280 0x7f6df80c3660 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 rx=0x7f6de8009760 tx=0x7f6de8009060).ready entity=mon.0 client_cookie=c58a33ded28134ee server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:19.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.213+0000 7f6df6ffd700 1 -- 172.21.15.121:0/2809120869 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de80090d0 con 0x7f6df80c3280 2023-11-15T03:10:19.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.213+0000 7f6df6ffd700 1 -- 172.21.15.121:0/2809120869 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de800f510 con 0x7f6df80c3280 2023-11-15T03:10:19.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.213+0000 7f6df6ffd700 1 -- 172.21.15.121:0/2809120869 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de800f890 con 0x7f6df80c3280 2023-11-15T03:10:19.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.213+0000 7f6dfcf47700 1 -- 172.21.15.121:0/2809120869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6df80c3280 msgr2=0x7f6df80c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:19.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.213+0000 7f6dfcf47700 1 --2- 172.21.15.121:0/2809120869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6df80c3280 0x7f6df80c3660 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 rx=0x7f6de8009760 tx=0x7f6de8009060).stop 2023-11-15T03:10:19.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.217+0000 7f6dfcf47700 1 -- 172.21.15.121:0/2809120869 shutdown_connections 2023-11-15T03:10:19.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.217+0000 7f6dfcf47700 1 --2- 172.21.15.121:0/2809120869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6df80c3280 0x7f6df80c3660 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.217+0000 7f6dfcf47700 1 -- 172.21.15.121:0/2809120869 >> 172.21.15.121:0/2809120869 conn(0x7f6df8062190 msgr2=0x7f6df8062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:19.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.217+0000 7f6dfcf47700 1 -- 172.21.15.121:0/2809120869 shutdown_connections 2023-11-15T03:10:19.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.217+0000 7f6dfcf47700 1 -- 172.21.15.121:0/2809120869 wait complete. 2023-11-15T03:10:19.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.217+0000 7f6dfcf47700 1 Processor -- start 2023-11-15T03:10:19.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.217+0000 7f6dfcf47700 1 -- start start 2023-11-15T03:10:19.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6dfcf47700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6df80c3280 0x7f6df814d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:19.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6dfcf47700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6df8152140 0x7f6df8169d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:19.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6dfcf47700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6df815be30 0x7f6df80c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:19.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6dfcf47700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80c4660 con 0x7f6df8152140 2023-11-15T03:10:19.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6dfcf47700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80c4940 con 0x7f6df815be30 2023-11-15T03:10:19.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6dfcf47700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80c4c20 con 0x7f6df80c3280 2023-11-15T03:10:19.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6deffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6df8152140 0x7f6df8169d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:19.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6deffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6df8152140 0x7f6df8169d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34628/0 (socket says 172.21.15.121:34628) 2023-11-15T03:10:19.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6deffff700 1 -- 172.21.15.121:0/4233671452 learned_addr learned my addr 172.21.15.121:0/4233671452 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:19.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6df7fff700 1 --2- 172.21.15.121:0/4233671452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6df815be30 0x7f6df80c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:19.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6df77fe700 1 --2- 172.21.15.121:0/4233671452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6df80c3280 0x7f6df814d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:19.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6deffff700 1 -- 172.21.15.121:0/4233671452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6df80c3280 msgr2=0x7f6df814d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:19.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6deffff700 1 --2- 172.21.15.121:0/4233671452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6df80c3280 0x7f6df814d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6deffff700 1 -- 172.21.15.121:0/4233671452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6df815be30 msgr2=0x7f6df80c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:19.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6deffff700 1 --2- 172.21.15.121:0/4233671452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6df815be30 0x7f6df80c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6deffff700 1 -- 172.21.15.121:0/4233671452 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de80092a0 con 0x7f6df8152140 2023-11-15T03:10:19.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6deffff700 1 --2- 172.21.15.121:0/4233671452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6df8152140 0x7f6df8169d40 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 rx=0x7f6de000c810 tx=0x7f6de0007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:19.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6def7fe700 1 -- 172.21.15.121:0/4233671452 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de00072e0 con 0x7f6df8152140 2023-11-15T03:10:19.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6def7fe700 1 -- 172.21.15.121:0/4233671452 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de0007440 con 0x7f6df8152140 2023-11-15T03:10:19.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6dfcf47700 1 -- 172.21.15.121:0/4233671452 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df80c4f00 con 0x7f6df8152140 2023-11-15T03:10:19.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6def7fe700 1 -- 172.21.15.121:0/4233671452 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de00057a0 con 0x7f6df8152140 2023-11-15T03:10:19.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.221+0000 7f6dfcf47700 1 -- 172.21.15.121:0/4233671452 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df816a230 con 0x7f6df8152140 2023-11-15T03:10:19.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.225+0000 7f6dfcf47700 1 -- 172.21.15.121:0/4233671452 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df80c5140 con 0x7f6df8152140 2023-11-15T03:10:19.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.225+0000 7f6def7fe700 1 -- 172.21.15.121:0/4233671452 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6de0005b90 con 0x7f6df8152140 2023-11-15T03:10:19.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.229+0000 7f6def7fe700 1 --2- 172.21.15.121:0/4233671452 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6dd805a510 0x7f6dd805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:19.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.229+0000 7f6def7fe700 1 -- 172.21.15.121:0/4233671452 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9362+0+0 (secure 0 0 0) 0x7f6de0074f80 con 0x7f6df8152140 2023-11-15T03:10:19.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.229+0000 7f6df77fe700 1 --2- 172.21.15.121:0/4233671452 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6dd805a510 0x7f6dd805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:19.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.229+0000 7f6df77fe700 1 --2- 172.21.15.121:0/4233671452 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6dd805a510 0x7f6dd805c9d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f6de8009760 tx=0x7f6de8017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:19.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.233+0000 7f6def7fe700 1 -- 172.21.15.121:0/4233671452 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6df80c5140 con 0x7f6df8152140 2023-11-15T03:10:19.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.433+0000 7f6dfcf47700 1 -- 172.21.15.121:0/4233671452 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6df80c47a0 con 0x7f6df8152140 2023-11-15T03:10:19.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.433+0000 7f6def7fe700 1 -- 172.21.15.121:0/4233671452 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 312 v312) v1 ==== 95+0+4497 (secure 0 0 0) 0x7f6df80c47a0 con 0x7f6df8152140 2023-11-15T03:10:19.442 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:10:19.442 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":312,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":10242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1750624080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1750624080},{"type":"v1","addr":"172.21.15.121:6836","nonce":1750624080}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":312}],"filesystems":[{"mdsmap":{"epoch":311,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:10:17.219328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:10:19.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.437+0000 7f6dfcf47700 1 -- 172.21.15.121:0/4233671452 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6dd805a510 msgr2=0x7f6dd805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:19.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.437+0000 7f6dfcf47700 1 --2- 172.21.15.121:0/4233671452 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6dd805a510 0x7f6dd805c9d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f6de8009760 tx=0x7f6de8017000).stop 2023-11-15T03:10:19.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.437+0000 7f6dfcf47700 1 -- 172.21.15.121:0/4233671452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6df8152140 msgr2=0x7f6df8169d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:19.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.437+0000 7f6dfcf47700 1 --2- 172.21.15.121:0/4233671452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6df8152140 0x7f6df8169d40 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 rx=0x7f6de000c810 tx=0x7f6de0007db0).stop 2023-11-15T03:10:19.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.437+0000 7f6dfcf47700 1 -- 172.21.15.121:0/4233671452 shutdown_connections 2023-11-15T03:10:19.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.437+0000 7f6dfcf47700 1 --2- 172.21.15.121:0/4233671452 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6dd805a510 0x7f6dd805c9d0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.437+0000 7f6dfcf47700 1 --2- 172.21.15.121:0/4233671452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6df815be30 0x7f6df80c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.437+0000 7f6dfcf47700 1 --2- 172.21.15.121:0/4233671452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6df8152140 0x7f6df8169d40 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.437+0000 7f6dfcf47700 1 --2- 172.21.15.121:0/4233671452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6df80c3280 0x7f6df814d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.437+0000 7f6dfcf47700 1 -- 172.21.15.121:0/4233671452 >> 172.21.15.121:0/4233671452 conn(0x7f6df8062190 msgr2=0x7f6df815a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:19.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.437+0000 7f6dfcf47700 1 -- 172.21.15.121:0/4233671452 shutdown_connections 2023-11-15T03:10:19.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.437+0000 7f6dfcf47700 1 -- 172.21.15.121:0/4233671452 wait complete. 2023-11-15T03:10:19.446 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 312 2023-11-15T03:10:19.463 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:10:19.463 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:10:19.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.613+0000 7f6a9c6f1700 1 Processor -- start 2023-11-15T03:10:19.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.613+0000 7f6a9c6f1700 1 -- start start 2023-11-15T03:10:19.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.617+0000 7f6a9c6f1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6a94153bb0 0x7f6a940a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:19.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.617+0000 7f6a9c6f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a94068170 0x7f6a94065d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:19.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.617+0000 7f6a9c6f1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a940cb5c0 con 0x7f6a940aa020 2023-11-15T03:10:19.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.617+0000 7f6a9c6f1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a940aa6f0 con 0x7f6a94153bb0 2023-11-15T03:10:19.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.617+0000 7f6a9c6f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a940c5900 con 0x7f6a94068170 2023-11-15T03:10:19.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.617+0000 7f6a99c8c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6a940aa020 0x7f6a94067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:43368/0 (socket says 172.21.15.121:43368) 2023-11-15T03:10:19.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.617+0000 7f6a99c8c700 1 -- 172.21.15.121:0/952620391 learned_addr learned my addr 172.21.15.121:0/952620391 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:19.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.617+0000 7f6a9a48d700 1 --2- 172.21.15.121:0/952620391 >> v2:172.21.15.190:3300/0 conn(0x7f6a94153bb0 0x7f6a940a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:19.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.617+0000 7f6a9ac8e700 1 --2- 172.21.15.121:0/952620391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a94068170 0x7f6a94065d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:19.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.617+0000 7f6a9948b700 1 -- 172.21.15.121:0/952620391 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4142753123 0 0) 0x7f6a940cb5c0 con 0x7f6a940aa020 2023-11-15T03:10:19.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.617+0000 7f6a9948b700 1 -- 172.21.15.121:0/952620391 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a800033f0 con 0x7f6a940aa020 2023-11-15T03:10:19.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.617+0000 7f6a9ac8e700 1 -- 172.21.15.121:0/952620391 >> v2:172.21.15.190:3300/0 conn(0x7f6a94153bb0 msgr2=0x7f6a940a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:19.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.617+0000 7f6a9ac8e700 1 --2- 172.21.15.121:0/952620391 >> v2:172.21.15.190:3300/0 conn(0x7f6a94153bb0 0x7f6a940a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.617+0000 7f6a9ac8e700 1 -- 172.21.15.121:0/952620391 >> v1:172.21.15.121:6789/0 conn(0x7f6a940aa020 legacy=0x7f6a94067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:19.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.617+0000 7f6a9ac8e700 1 -- 172.21.15.121:0/952620391 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a940c9ca0 con 0x7f6a94068170 2023-11-15T03:10:19.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.617+0000 7f6a9a48d700 1 --2- 172.21.15.121:0/952620391 >> v2:172.21.15.190:3300/0 conn(0x7f6a94153bb0 0x7f6a940a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:10:19.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.617+0000 7f6a9ac8e700 1 --2- 172.21.15.121:0/952620391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a94068170 0x7f6a94065d30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f6a8c009e60 tx=0x7f6a8c009780).ready entity=mon.2 client_cookie=f400e8d77f6d6137 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:19.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.621+0000 7f6a9948b700 1 -- 172.21.15.121:0/952620391 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c00e070 con 0x7f6a94068170 2023-11-15T03:10:19.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.621+0000 7f6a9948b700 1 -- 172.21.15.121:0/952620391 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8c00bab0 con 0x7f6a94068170 2023-11-15T03:10:19.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.621+0000 7f6a9948b700 1 -- 172.21.15.121:0/952620391 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c00be30 con 0x7f6a94068170 2023-11-15T03:10:19.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.621+0000 7f6a9c6f1700 1 -- 172.21.15.121:0/952620391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a94068170 msgr2=0x7f6a94065d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:19.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.621+0000 7f6a9c6f1700 1 --2- 172.21.15.121:0/952620391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a94068170 0x7f6a94065d30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f6a8c009e60 tx=0x7f6a8c009780).stop 2023-11-15T03:10:19.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.621+0000 7f6a9c6f1700 1 -- 172.21.15.121:0/952620391 shutdown_connections 2023-11-15T03:10:19.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.621+0000 7f6a9c6f1700 1 --2- 172.21.15.121:0/952620391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a94068170 0x7f6a94065d30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.621+0000 7f6a9c6f1700 1 --2- 172.21.15.121:0/952620391 >> v2:172.21.15.190:3300/0 conn(0x7f6a94153bb0 0x7f6a940a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.621+0000 7f6a9c6f1700 1 -- 172.21.15.121:0/952620391 >> 172.21.15.121:0/952620391 conn(0x7f6a94058c50 msgr2=0x7f6a94059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:19.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.621+0000 7f6a9c6f1700 1 -- 172.21.15.121:0/952620391 shutdown_connections 2023-11-15T03:10:19.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.621+0000 7f6a9c6f1700 1 -- 172.21.15.121:0/952620391 wait complete. 2023-11-15T03:10:19.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.621+0000 7f6a9c6f1700 1 Processor -- start 2023-11-15T03:10:19.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.621+0000 7f6a9c6f1700 1 -- start start 2023-11-15T03:10:19.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a9c6f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a94068170 0x7f6a941508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:19.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a9c6f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a940aa020 0x7f6a940c09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:19.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a9c6f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a94153bb0 0x7f6a9414a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:19.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a9c6f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a940ab8e0 con 0x7f6a94153bb0 2023-11-15T03:10:19.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a9c6f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9416c010 con 0x7f6a94068170 2023-11-15T03:10:19.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a9c6f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9416c2f0 con 0x7f6a940aa020 2023-11-15T03:10:19.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a9ac8e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a94153bb0 0x7f6a9414a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:19.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a9ac8e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a94153bb0 0x7f6a9414a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34636/0 (socket says 172.21.15.121:34636) 2023-11-15T03:10:19.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a9ac8e700 1 -- 172.21.15.121:0/390576334 learned_addr learned my addr 172.21.15.121:0/390576334 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:19.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a99c8c700 1 --2- 172.21.15.121:0/390576334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a940aa020 0x7f6a940c09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:19.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a9a48d700 1 --2- 172.21.15.121:0/390576334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a94068170 0x7f6a941508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:19.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a9ac8e700 1 -- 172.21.15.121:0/390576334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a940aa020 msgr2=0x7f6a940c09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:19.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a9ac8e700 1 --2- 172.21.15.121:0/390576334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a940aa020 0x7f6a940c09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a9ac8e700 1 -- 172.21.15.121:0/390576334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a94068170 msgr2=0x7f6a941508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:19.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a9ac8e700 1 --2- 172.21.15.121:0/390576334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a94068170 0x7f6a941508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a9ac8e700 1 -- 172.21.15.121:0/390576334 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a900092a0 con 0x7f6a94153bb0 2023-11-15T03:10:19.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a9ac8e700 1 --2- 172.21.15.121:0/390576334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a94153bb0 0x7f6a9414a9b0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 rx=0x7f6a8c009460 tx=0x7f6a8c009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:19.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a8a7fc700 1 -- 172.21.15.121:0/390576334 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c013e60 con 0x7f6a94153bb0 2023-11-15T03:10:19.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a8a7fc700 1 -- 172.21.15.121:0/390576334 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8c01d630 con 0x7f6a94153bb0 2023-11-15T03:10:19.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a8a7fc700 1 -- 172.21.15.121:0/390576334 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c01d9d0 con 0x7f6a94153bb0 2023-11-15T03:10:19.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a9c6f1700 1 -- 172.21.15.121:0/390576334 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8c009a20 con 0x7f6a94153bb0 2023-11-15T03:10:19.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.625+0000 7f6a9c6f1700 1 -- 172.21.15.121:0/390576334 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9416c8b0 con 0x7f6a94153bb0 2023-11-15T03:10:19.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.629+0000 7f6a9c6f1700 1 -- 172.21.15.121:0/390576334 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a940aba90 con 0x7f6a94153bb0 2023-11-15T03:10:19.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.629+0000 7f6a8a7fc700 1 -- 172.21.15.121:0/390576334 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6a8c003040 con 0x7f6a94153bb0 2023-11-15T03:10:19.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.633+0000 7f6a8a7fc700 1 --2- 172.21.15.121:0/390576334 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6a7805a510 0x7f6a7805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:19.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.633+0000 7f6a8a7fc700 1 -- 172.21.15.121:0/390576334 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9362+0+0 (secure 0 0 0) 0x7f6a8c077240 con 0x7f6a94153bb0 2023-11-15T03:10:19.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.633+0000 7f6a9a48d700 1 --2- 172.21.15.121:0/390576334 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6a7805a510 0x7f6a7805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:19.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.633+0000 7f6a9a48d700 1 --2- 172.21.15.121:0/390576334 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6a7805a510 0x7f6a7805c9d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f6a90009d20 tx=0x7f6a90009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:19.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.637+0000 7f6a8a7fc700 1 -- 172.21.15.121:0/390576334 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6a940aba90 con 0x7f6a94153bb0 2023-11-15T03:10:19.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.833+0000 7f6a9c6f1700 1 -- 172.21.15.121:0/390576334 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a940aba20 con 0x7f6a94153bb0 2023-11-15T03:10:19.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.833+0000 7f6a8a7fc700 1 -- 172.21.15.121:0/390576334 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 312 v312) v1 ==== 95+0+4497 (secure 0 0 0) 0x7f6a8c049d30 con 0x7f6a94153bb0 2023-11-15T03:10:19.842 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:10:19.842 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":312,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":10242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1750624080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1750624080},{"type":"v1","addr":"172.21.15.121:6836","nonce":1750624080}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":312}],"filesystems":[{"mdsmap":{"epoch":311,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:10:17.219328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:10:19.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.837+0000 7f6a9c6f1700 1 -- 172.21.15.121:0/390576334 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6a7805a510 msgr2=0x7f6a7805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:19.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.837+0000 7f6a9c6f1700 1 --2- 172.21.15.121:0/390576334 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6a7805a510 0x7f6a7805c9d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f6a90009d20 tx=0x7f6a90009060).stop 2023-11-15T03:10:19.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.837+0000 7f6a9c6f1700 1 -- 172.21.15.121:0/390576334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a94153bb0 msgr2=0x7f6a9414a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:19.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.837+0000 7f6a9c6f1700 1 --2- 172.21.15.121:0/390576334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a94153bb0 0x7f6a9414a9b0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 rx=0x7f6a8c009460 tx=0x7f6a8c009780).stop 2023-11-15T03:10:19.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.837+0000 7f6a9c6f1700 1 -- 172.21.15.121:0/390576334 shutdown_connections 2023-11-15T03:10:19.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.837+0000 7f6a9c6f1700 1 --2- 172.21.15.121:0/390576334 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6a7805a510 0x7f6a7805c9d0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.837+0000 7f6a9c6f1700 1 --2- 172.21.15.121:0/390576334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a94153bb0 0x7f6a9414a9b0 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.837+0000 7f6a9c6f1700 1 --2- 172.21.15.121:0/390576334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a940aa020 0x7f6a940c09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.837+0000 7f6a9c6f1700 1 --2- 172.21.15.121:0/390576334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a94068170 0x7f6a941508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:19.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.837+0000 7f6a9c6f1700 1 -- 172.21.15.121:0/390576334 >> 172.21.15.121:0/390576334 conn(0x7f6a94058c50 msgr2=0x7f6a940afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:19.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.837+0000 7f6a9c6f1700 1 -- 172.21.15.121:0/390576334 shutdown_connections 2023-11-15T03:10:19.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:19.837+0000 7f6a9c6f1700 1 -- 172.21.15.121:0/390576334 wait complete. 2023-11-15T03:10:19.847 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 312 2023-11-15T03:10:19.867 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 600.00000000' led to DAMAGED state 2023-11-15T03:10:19.867 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on mds0_openfiles.0' 2023-11-15T03:10:19.867 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:10:19.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:19.868 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:10:19.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:10:19.890 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:10:19.890 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:10:20.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.097+0000 7ff68e6e4700 1 Processor -- start 2023-11-15T03:10:20.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.101+0000 7ff68e6e4700 1 -- start start 2023-11-15T03:10:20.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.101+0000 7ff68e6e4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff6880aed50 0x7ff68815e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:20.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.101+0000 7ff68e6e4700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff688066c90 con 0x7ff6880ab0d0 2023-11-15T03:10:20.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.101+0000 7ff68e6e4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff688065e20 con 0x7ff6880ad1b0 2023-11-15T03:10:20.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.101+0000 7ff68e6e4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff688066080 con 0x7ff6880aed50 2023-11-15T03:10:20.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.101+0000 7ff68d6e2700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff6880ab0d0 0x7ff6880ab4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:43372/0 (socket says 172.21.15.121:43372) 2023-11-15T03:10:20.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.101+0000 7ff68d6e2700 1 -- 172.21.15.121:0/729881713 learned_addr learned my addr 172.21.15.121:0/729881713 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:20.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.101+0000 7ff68dee3700 1 --2- 172.21.15.121:0/729881713 >> v2:172.21.15.190:3300/0 conn(0x7ff6880aed50 0x7ff68815e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:20.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.101+0000 7ff67ffff700 1 -- 172.21.15.121:0/729881713 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3583174255 0 0) 0x7ff688066c90 con 0x7ff6880ab0d0 2023-11-15T03:10:20.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.101+0000 7ff67ffff700 1 -- 172.21.15.121:0/729881713 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6740033f0 con 0x7ff6880ab0d0 2023-11-15T03:10:20.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff67ffff700 1 -- 172.21.15.121:0/729881713 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4184567342 0 0) 0x7ff6740033f0 con 0x7ff6880ab0d0 2023-11-15T03:10:20.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff67ffff700 1 -- 172.21.15.121:0/729881713 >> v2:172.21.15.190:3300/0 conn(0x7ff6880aed50 msgr2=0x7ff68815e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:20.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff67ffff700 1 --2- 172.21.15.121:0/729881713 >> v2:172.21.15.190:3300/0 conn(0x7ff6880aed50 0x7ff68815e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:20.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff67ffff700 1 -- 172.21.15.121:0/729881713 >> v1:172.21.15.190:6789/0 conn(0x7ff6880ad1b0 legacy=0x7ff688065710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:20.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff67ffff700 1 -- 172.21.15.121:0/729881713 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff688066360 con 0x7ff6880ab0d0 2023-11-15T03:10:20.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff67ffff700 1 -- 172.21.15.121:0/729881713 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7ff684004d60 con 0x7ff6880ab0d0 2023-11-15T03:10:20.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff67ffff700 1 -- 172.21.15.121:0/729881713 >> v1:172.21.15.121:6789/0 conn(0x7ff6880ab0d0 legacy=0x7ff6880ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:20.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff67ffff700 1 --2- 172.21.15.121:0/729881713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6740038e0 0x7ff674003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:20.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff67ffff700 1 --2- 172.21.15.121:0/729881713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff674004690 0x7ff674004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:20.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff67ffff700 1 --2- 172.21.15.121:0/729881713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff674005000 0x7ff67400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:20.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff67ffff700 1 -- 172.21.15.121:0/729881713 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff688066080 con 0x7ff674005000 2023-11-15T03:10:20.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff67ffff700 1 -- 172.21.15.121:0/729881713 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67400beb0 con 0x7ff674004690 2023-11-15T03:10:20.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff67ffff700 1 -- 172.21.15.121:0/729881713 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67400c070 con 0x7ff6740038e0 2023-11-15T03:10:20.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff68dee3700 1 --2- 172.21.15.121:0/729881713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff674005000 0x7ff67400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:20.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff68cee1700 1 --2- 172.21.15.121:0/729881713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff674004690 0x7ff674004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:20.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff68dee3700 1 -- 172.21.15.121:0/729881713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6740038e0 msgr2=0x7ff674003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:20.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff68dee3700 1 --2- 172.21.15.121:0/729881713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6740038e0 0x7ff674003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:10:20.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff68dee3700 1 -- 172.21.15.121:0/729881713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff674004690 msgr2=0x7ff674004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:20.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff68dee3700 1 --2- 172.21.15.121:0/729881713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff674004690 0x7ff674004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:20.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff68dee3700 1 -- 172.21.15.121:0/729881713 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6740033f0 con 0x7ff674005000 2023-11-15T03:10:20.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff68cee1700 1 --2- 172.21.15.121:0/729881713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff674004690 0x7ff674004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:10:20.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff68dee3700 1 -- 172.21.15.121:0/729881713 reap_dead start 2023-11-15T03:10:20.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff68dee3700 1 --2- 172.21.15.121:0/729881713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff674005000 0x7ff67400b8e0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 rx=0x7ff6880af1e0 tx=0x7ff68000b040).ready entity=mon.0 client_cookie=a1334ff05abf9dea server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:20.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff67ffff700 1 -- 172.21.15.121:0/729881713 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680017070 con 0x7ff674005000 2023-11-15T03:10:20.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff67ffff700 1 -- 172.21.15.121:0/729881713 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68000f040 con 0x7ff674005000 2023-11-15T03:10:20.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff67ffff700 1 -- 172.21.15.121:0/729881713 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68000eca0 con 0x7ff674005000 2023-11-15T03:10:20.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff68e6e4700 1 -- 172.21.15.121:0/729881713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff674005000 msgr2=0x7ff67400b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:20.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff68e6e4700 1 --2- 172.21.15.121:0/729881713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff674005000 0x7ff67400b8e0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 rx=0x7ff6880af1e0 tx=0x7ff68000b040).stop 2023-11-15T03:10:20.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff68e6e4700 1 -- 172.21.15.121:0/729881713 shutdown_connections 2023-11-15T03:10:20.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff68e6e4700 1 --2- 172.21.15.121:0/729881713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff674005000 0x7ff67400b8e0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:20.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff68e6e4700 1 -- 172.21.15.121:0/729881713 >> 172.21.15.121:0/729881713 conn(0x7ff688067a60 msgr2=0x7ff688067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:20.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff68e6e4700 1 -- 172.21.15.121:0/729881713 shutdown_connections 2023-11-15T03:10:20.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.105+0000 7ff68e6e4700 1 -- 172.21.15.121:0/729881713 wait complete. 2023-11-15T03:10:20.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.109+0000 7ff68e6e4700 1 Processor -- start 2023-11-15T03:10:20.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.109+0000 7ff68e6e4700 1 -- start start 2023-11-15T03:10:20.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff68e6e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff674005000 0x7ff6880a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:20.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff68e6e4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff688165270 0x7ff688161d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:20.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff68e6e4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff688162220 0x7ff6881522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:20.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff68e6e4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff688166690 con 0x7ff688165270 2023-11-15T03:10:20.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff68e6e4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff688165a80 con 0x7ff688162220 2023-11-15T03:10:20.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff68e6e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff688165d60 con 0x7ff674005000 2023-11-15T03:10:20.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff68cee1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff688165270 0x7ff688161d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:20.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff68cee1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff688165270 0x7ff688161d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34648/0 (socket says 172.21.15.121:34648) 2023-11-15T03:10:20.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff68cee1700 1 -- 172.21.15.121:0/2871514765 learned_addr learned my addr 172.21.15.121:0/2871514765 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:20.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff68d6e2700 1 --2- 172.21.15.121:0/2871514765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff674005000 0x7ff6880a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:20.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff68dee3700 1 --2- 172.21.15.121:0/2871514765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff688162220 0x7ff6881522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:20.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff68cee1700 1 -- 172.21.15.121:0/2871514765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff674005000 msgr2=0x7ff6880a94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:20.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff68cee1700 1 --2- 172.21.15.121:0/2871514765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff674005000 0x7ff6880a94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:20.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff68cee1700 1 -- 172.21.15.121:0/2871514765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff688162220 msgr2=0x7ff6881522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:20.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff68cee1700 1 --2- 172.21.15.121:0/2871514765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff688162220 0x7ff6881522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:20.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff68cee1700 1 -- 172.21.15.121:0/2871514765 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff688166040 con 0x7ff688165270 2023-11-15T03:10:20.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff68cee1700 1 --2- 172.21.15.121:0/2871514765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff688165270 0x7ff688161d30 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 rx=0x7ff67800d530 tx=0x7ff678009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:20.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff67d7fa700 1 -- 172.21.15.121:0/2871514765 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67800dea0 con 0x7ff688165270 2023-11-15T03:10:20.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff67d7fa700 1 -- 172.21.15.121:0/2871514765 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6780104e0 con 0x7ff688165270 2023-11-15T03:10:20.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff68e6e4700 1 -- 172.21.15.121:0/2871514765 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff688166320 con 0x7ff688165270 2023-11-15T03:10:20.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff67d7fa700 1 -- 172.21.15.121:0/2871514765 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff678010860 con 0x7ff688165270 2023-11-15T03:10:20.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.113+0000 7ff68e6e4700 1 -- 172.21.15.121:0/2871514765 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6881527d0 con 0x7ff688165270 2023-11-15T03:10:20.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.117+0000 7ff68e6e4700 1 -- 172.21.15.121:0/2871514765 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff688152a50 con 0x7ff688165270 2023-11-15T03:10:20.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.117+0000 7ff67d7fa700 1 -- 172.21.15.121:0/2871514765 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff678010c90 con 0x7ff688165270 2023-11-15T03:10:20.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.117+0000 7ff67d7fa700 1 --2- 172.21.15.121:0/2871514765 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff66c05a510 0x7ff66c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:20.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.117+0000 7ff67d7fa700 1 -- 172.21.15.121:0/2871514765 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9362+0+0 (secure 0 0 0) 0x7ff678074e50 con 0x7ff688165270 2023-11-15T03:10:20.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.117+0000 7ff68d6e2700 1 --2- 172.21.15.121:0/2871514765 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff66c05a510 0x7ff66c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:20.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.121+0000 7ff68d6e2700 1 --2- 172.21.15.121:0/2871514765 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff66c05a510 0x7ff66c05c9d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7ff684004fd0 tx=0x7ff68400e040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:20.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.125+0000 7ff67d7fa700 1 -- 172.21.15.121:0/2871514765 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff678047940 con 0x7ff688165270 2023-11-15T03:10:20.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:20.333+0000 7ff68e6e4700 1 -- 172.21.15.121:0/2871514765 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff6881667d0 con 0x7ff688165270 2023-11-15T03:10:21.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.285+0000 7ff67d7fa700 1 -- 172.21.15.121:0/2871514765 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v313) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff67804af60 con 0x7ff688165270 2023-11-15T03:10:21.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.285+0000 7ff68e6e4700 1 -- 172.21.15.121:0/2871514765 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff66c05a510 msgr2=0x7ff66c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:21.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.285+0000 7ff68e6e4700 1 --2- 172.21.15.121:0/2871514765 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff66c05a510 0x7ff66c05c9d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7ff684004fd0 tx=0x7ff68400e040).stop 2023-11-15T03:10:21.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.285+0000 7ff68e6e4700 1 -- 172.21.15.121:0/2871514765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff688165270 msgr2=0x7ff688161d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:21.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.285+0000 7ff68e6e4700 1 --2- 172.21.15.121:0/2871514765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff688165270 0x7ff688161d30 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 rx=0x7ff67800d530 tx=0x7ff678009040).stop 2023-11-15T03:10:21.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.285+0000 7ff68e6e4700 1 -- 172.21.15.121:0/2871514765 shutdown_connections 2023-11-15T03:10:21.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.285+0000 7ff68e6e4700 1 --2- 172.21.15.121:0/2871514765 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff66c05a510 0x7ff66c05c9d0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:21.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.285+0000 7ff68e6e4700 1 --2- 172.21.15.121:0/2871514765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff688162220 0x7ff6881522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:21.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.285+0000 7ff68e6e4700 1 --2- 172.21.15.121:0/2871514765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff688165270 0x7ff688161d30 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:21.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.285+0000 7ff68e6e4700 1 --2- 172.21.15.121:0/2871514765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff674005000 0x7ff6880a94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:21.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.285+0000 7ff68e6e4700 1 -- 172.21.15.121:0/2871514765 >> 172.21.15.121:0/2871514765 conn(0x7ff688067a60 msgr2=0x7ff6880ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:21.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.285+0000 7ff68e6e4700 1 -- 172.21.15.121:0/2871514765 shutdown_connections 2023-11-15T03:10:21.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.285+0000 7ff68e6e4700 1 -- 172.21.15.121:0/2871514765 wait complete. 2023-11-15T03:10:21.297 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:10:21.319 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:10:21.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.477+0000 7fe140a39700 1 Processor -- start 2023-11-15T03:10:21.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.481+0000 7fe140a39700 1 -- start start 2023-11-15T03:10:21.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.481+0000 7fe140a39700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c1511f0 0x7fe13c066570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:21.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.481+0000 7fe140a39700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe13c0bcab0 0x7fe13c1515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:21.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.481+0000 7fe140a39700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe13c066a60 con 0x7fe13c151ac0 2023-11-15T03:10:21.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.481+0000 7fe140a39700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe13c0660e0 con 0x7fe13c0bcab0 2023-11-15T03:10:21.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.481+0000 7fe140a39700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c0bd390 con 0x7fe13c1511f0 2023-11-15T03:10:21.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe13b7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c1511f0 0x7fe13c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:21.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe13bfff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe13c151ac0 0x7fe13c0a8750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:43380/0 (socket says 172.21.15.121:43380) 2023-11-15T03:10:21.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe13bfff700 1 -- 172.21.15.121:0/283126722 learned_addr learned my addr 172.21.15.121:0/283126722 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:21.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe13affd700 1 --2- 172.21.15.121:0/283126722 >> v2:172.21.15.190:3300/0 conn(0x7fe13c0bcab0 0x7fe13c1515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:21.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe13b7fe700 1 -- 172.21.15.121:0/283126722 >> v2:172.21.15.190:3300/0 conn(0x7fe13c0bcab0 msgr2=0x7fe13c1515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:21.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe13b7fe700 1 --2- 172.21.15.121:0/283126722 >> v2:172.21.15.190:3300/0 conn(0x7fe13c0bcab0 0x7fe13c1515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:21.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe13b7fe700 1 -- 172.21.15.121:0/283126722 >> v1:172.21.15.121:6789/0 conn(0x7fe13c151ac0 legacy=0x7fe13c0a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:21.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe13b7fe700 1 -- 172.21.15.121:0/283126722 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13c0657b0 con 0x7fe13c1511f0 2023-11-15T03:10:21.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe13a7fc700 1 -- 172.21.15.121:0/283126722 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3263923799 0 0) 0x7fe13c066a60 con 0x7fe13c151ac0 2023-11-15T03:10:21.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe13b7fe700 1 --2- 172.21.15.121:0/283126722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c1511f0 0x7fe13c066570 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fe12c009760 tx=0x7fe12c009060).ready entity=mon.2 client_cookie=787c734502f764c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:21.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe13a7fc700 1 -- 172.21.15.121:0/283126722 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c00f510 con 0x7fe13c1511f0 2023-11-15T03:10:21.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe13a7fc700 1 -- 172.21.15.121:0/283126722 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12c00f670 con 0x7fe13c1511f0 2023-11-15T03:10:21.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe13a7fc700 1 -- 172.21.15.121:0/283126722 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c00f9f0 con 0x7fe13c1511f0 2023-11-15T03:10:21.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe140a39700 1 -- 172.21.15.121:0/283126722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c1511f0 msgr2=0x7fe13c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:21.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe140a39700 1 --2- 172.21.15.121:0/283126722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c1511f0 0x7fe13c066570 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fe12c009760 tx=0x7fe12c009060).stop 2023-11-15T03:10:21.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe140a39700 1 -- 172.21.15.121:0/283126722 shutdown_connections 2023-11-15T03:10:21.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe140a39700 1 --2- 172.21.15.121:0/283126722 >> v2:172.21.15.190:3300/0 conn(0x7fe13c0bcab0 0x7fe13c1515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:21.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe140a39700 1 --2- 172.21.15.121:0/283126722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c1511f0 0x7fe13c066570 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:21.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe140a39700 1 -- 172.21.15.121:0/283126722 >> 172.21.15.121:0/283126722 conn(0x7fe13c067a60 msgr2=0x7fe13c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:21.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe140a39700 1 -- 172.21.15.121:0/283126722 shutdown_connections 2023-11-15T03:10:21.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.485+0000 7fe140a39700 1 -- 172.21.15.121:0/283126722 wait complete. 2023-11-15T03:10:21.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.489+0000 7fe140a39700 1 Processor -- start 2023-11-15T03:10:21.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.489+0000 7fe140a39700 1 -- start start 2023-11-15T03:10:21.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.489+0000 7fe140a39700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe13c0bcab0 0x7fe13c0ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:21.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.489+0000 7fe140a39700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe13c1511f0 0x7fe13c0aa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:21.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.489+0000 7fe140a39700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c151ac0 0x7fe13c144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:21.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.489+0000 7fe140a39700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c0ab5c0 con 0x7fe13c0bcab0 2023-11-15T03:10:21.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.489+0000 7fe140a39700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c1529b0 con 0x7fe13c1511f0 2023-11-15T03:10:21.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.489+0000 7fe140a39700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c0aa6a0 con 0x7fe13c151ac0 2023-11-15T03:10:21.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.489+0000 7fe13b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe13c0bcab0 0x7fe13c0ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:21.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.493+0000 7fe13b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe13c0bcab0 0x7fe13c0ab0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34662/0 (socket says 172.21.15.121:34662) 2023-11-15T03:10:21.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.493+0000 7fe13b7fe700 1 -- 172.21.15.121:0/2573513710 learned_addr learned my addr 172.21.15.121:0/2573513710 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:21.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.493+0000 7fe13bfff700 1 --2- 172.21.15.121:0/2573513710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c151ac0 0x7fe13c144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:21.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.493+0000 7fe13affd700 1 --2- 172.21.15.121:0/2573513710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe13c1511f0 0x7fe13c0aa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:21.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.493+0000 7fe13b7fe700 1 -- 172.21.15.121:0/2573513710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c151ac0 msgr2=0x7fe13c144f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:21.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.493+0000 7fe13b7fe700 1 --2- 172.21.15.121:0/2573513710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c151ac0 0x7fe13c144f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:21.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.493+0000 7fe13b7fe700 1 -- 172.21.15.121:0/2573513710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe13c1511f0 msgr2=0x7fe13c0aa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:21.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.493+0000 7fe13b7fe700 1 --2- 172.21.15.121:0/2573513710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe13c1511f0 0x7fe13c0aa1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:21.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.493+0000 7fe13b7fe700 1 -- 172.21.15.121:0/2573513710 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12c0092a0 con 0x7fe13c0bcab0 2023-11-15T03:10:21.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.493+0000 7fe13bfff700 1 --2- 172.21.15.121:0/2573513710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c151ac0 0x7fe13c144f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:10:21.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.493+0000 7fe13affd700 1 --2- 172.21.15.121:0/2573513710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe13c1511f0 0x7fe13c0aa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:10:21.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.493+0000 7fe13b7fe700 1 --2- 172.21.15.121:0/2573513710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe13c0bcab0 0x7fe13c0ab0d0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 rx=0x7fe12c009e00 tx=0x7fe12c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:21.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.493+0000 7fe1237fe700 1 -- 172.21.15.121:0/2573513710 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c00f7e0 con 0x7fe13c0bcab0 2023-11-15T03:10:21.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.493+0000 7fe1237fe700 1 -- 172.21.15.121:0/2573513710 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12c00f940 con 0x7fe13c0bcab0 2023-11-15T03:10:21.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.493+0000 7fe140a39700 1 -- 172.21.15.121:0/2573513710 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13c152080 con 0x7fe13c0bcab0 2023-11-15T03:10:21.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.493+0000 7fe1237fe700 1 -- 172.21.15.121:0/2573513710 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c01b660 con 0x7fe13c0bcab0 2023-11-15T03:10:21.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.493+0000 7fe140a39700 1 -- 172.21.15.121:0/2573513710 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13c152640 con 0x7fe13c0bcab0 2023-11-15T03:10:21.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.497+0000 7fe140a39700 1 -- 172.21.15.121:0/2573513710 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe13c06bcf0 con 0x7fe13c0bcab0 2023-11-15T03:10:21.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.497+0000 7fe1237fe700 1 -- 172.21.15.121:0/2573513710 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe12c01ba50 con 0x7fe13c0bcab0 2023-11-15T03:10:21.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.497+0000 7fe1237fe700 1 --2- 172.21.15.121:0/2573513710 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe12805a4e0 0x7fe12805c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:21.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.497+0000 7fe1237fe700 1 -- 172.21.15.121:0/2573513710 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9362+0+0 (secure 0 0 0) 0x7fe12c076ce0 con 0x7fe13c0bcab0 2023-11-15T03:10:21.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.497+0000 7fe13affd700 1 --2- 172.21.15.121:0/2573513710 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe12805a4e0 0x7fe12805c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:21.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.497+0000 7fe13affd700 1 --2- 172.21.15.121:0/2573513710 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe12805a4e0 0x7fe12805c9a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7fe13c150790 tx=0x7fe13000a040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:21.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.501+0000 7fe1237fe700 1 -- 172.21.15.121:0/2573513710 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe12c049750 con 0x7fe13c0bcab0 2023-11-15T03:10:21.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:21.721+0000 7fe140a39700 1 -- 172.21.15.121:0/2573513710 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fe13c068a70 con 0x7fe13c0bcab0 2023-11-15T03:10:22.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.297+0000 7fe1237fe700 1 -- 172.21.15.121:0/2573513710 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v314) v1 ==== 101+0+0 (secure 0 0 0) 0x7fe12c029070 con 0x7fe13c0bcab0 2023-11-15T03:10:22.308 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.301+0000 7fe140a39700 1 -- 172.21.15.121:0/2573513710 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe12805a4e0 msgr2=0x7fe12805c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.301+0000 7fe140a39700 1 --2- 172.21.15.121:0/2573513710 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe12805a4e0 0x7fe12805c9a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7fe13c150790 tx=0x7fe13000a040).stop 2023-11-15T03:10:22.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.301+0000 7fe140a39700 1 -- 172.21.15.121:0/2573513710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe13c0bcab0 msgr2=0x7fe13c0ab0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.301+0000 7fe140a39700 1 --2- 172.21.15.121:0/2573513710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe13c0bcab0 0x7fe13c0ab0d0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 rx=0x7fe12c009e00 tx=0x7fe12c017000).stop 2023-11-15T03:10:22.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.301+0000 7fe140a39700 1 -- 172.21.15.121:0/2573513710 shutdown_connections 2023-11-15T03:10:22.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.301+0000 7fe140a39700 1 --2- 172.21.15.121:0/2573513710 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe12805a4e0 0x7fe12805c9a0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.301+0000 7fe140a39700 1 --2- 172.21.15.121:0/2573513710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c151ac0 0x7fe13c144f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.301+0000 7fe140a39700 1 --2- 172.21.15.121:0/2573513710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe13c1511f0 0x7fe13c0aa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.301+0000 7fe140a39700 1 --2- 172.21.15.121:0/2573513710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe13c0bcab0 0x7fe13c0ab0d0 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.301+0000 7fe140a39700 1 -- 172.21.15.121:0/2573513710 >> 172.21.15.121:0/2573513710 conn(0x7fe13c067a60 msgr2=0x7fe13c0c4450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:22.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.301+0000 7fe140a39700 1 -- 172.21.15.121:0/2573513710 shutdown_connections 2023-11-15T03:10:22.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.301+0000 7fe140a39700 1 -- 172.21.15.121:0/2573513710 wait complete. 2023-11-15T03:10:22.313 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:10:22.330 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:10:22.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.385+0000 7fef6ad36d00 1 Processor -- start 2023-11-15T03:10:22.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.385+0000 7fef6ad36d00 1 -- start start 2023-11-15T03:10:22.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef6ad36d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562423997a30 0x5624239ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef6ad36d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624239e6b20 0x562423a6db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef6ad36d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562423a78000 0x562423a811e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef6ad36d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562423910220 con 0x562423997a30 2023-11-15T03:10:22.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef6ad36d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562423998310 con 0x5624239e6b20 2023-11-15T03:10:22.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef6ad36d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624239e46b0 con 0x562423a78000 2023-11-15T03:10:22.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef5f414700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562423997a30 0x5624239ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef5f414700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562423997a30 0x5624239ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34676/0 (socket says 172.21.15.121:34676) 2023-11-15T03:10:22.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef5f414700 1 -- 172.21.15.121:0/845907104 learned_addr learned my addr 172.21.15.121:0/845907104 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:22.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef5fc15700 1 --2- 172.21.15.121:0/845907104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562423a78000 0x562423a811e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef56c13700 1 --2- 172.21.15.121:0/845907104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624239e6b20 0x562423a6db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef5f414700 1 -- 172.21.15.121:0/845907104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562423a78000 msgr2=0x562423a811e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:22.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef5f414700 1 --2- 172.21.15.121:0/845907104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562423a78000 0x562423a811e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef5f414700 1 -- 172.21.15.121:0/845907104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624239e6b20 msgr2=0x562423a6db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:22.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef5f414700 1 --2- 172.21.15.121:0/845907104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624239e6b20 0x562423a6db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef5f414700 1 -- 172.21.15.121:0/845907104 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5000a0e0 con 0x562423997a30 2023-11-15T03:10:22.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef5f414700 1 --2- 172.21.15.121:0/845907104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562423997a30 0x5624239ccad0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 rx=0x7fef5000ac10 tx=0x7fef50015000).ready entity=mon.0 client_cookie=832c7e5a2f86e61c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:22.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef5e412700 1 -- 172.21.15.121:0/845907104 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5000ac40 con 0x562423997a30 2023-11-15T03:10:22.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef5e412700 1 -- 172.21.15.121:0/845907104 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef50000f40 con 0x562423997a30 2023-11-15T03:10:22.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef5e412700 1 -- 172.21.15.121:0/845907104 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5000d740 con 0x562423997a30 2023-11-15T03:10:22.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef6ad36d00 1 -- 172.21.15.121:0/845907104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562423997a30 msgr2=0x5624239ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/845907104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562423997a30 0x5624239ccad0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 rx=0x7fef5000ac10 tx=0x7fef50015000).stop 2023-11-15T03:10:22.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef6ad36d00 1 -- 172.21.15.121:0/845907104 shutdown_connections 2023-11-15T03:10:22.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/845907104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562423a78000 0x562423a811e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/845907104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624239e6b20 0x562423a6db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/845907104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562423997a30 0x5624239ccad0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef6ad36d00 1 -- 172.21.15.121:0/845907104 >> 172.21.15.121:0/845907104 conn(0x5624239e7080 msgr2=0x5624239e3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:22.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef6ad36d00 1 -- 172.21.15.121:0/845907104 shutdown_connections 2023-11-15T03:10:22.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.389+0000 7fef6ad36d00 1 -- 172.21.15.121:0/845907104 wait complete. 2023-11-15T03:10:22.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef6ad36d00 1 Processor -- start 2023-11-15T03:10:22.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef6ad36d00 1 -- start start 2023-11-15T03:10:22.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef6ad36d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624239e6b20 0x5624239dfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef6ad36d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562423a78000 0x562423a77180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef6ad36d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562423a6dd20 0x562423a6e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef6ad36d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624239cce30 con 0x5624239e6b20 2023-11-15T03:10:22.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef6ad36d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624239cc220 con 0x562423a78000 2023-11-15T03:10:22.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef6ad36d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624239cc500 con 0x562423a6dd20 2023-11-15T03:10:22.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef5f414700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624239e6b20 0x5624239dfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef5f414700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624239e6b20 0x5624239dfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34688/0 (socket says 172.21.15.121:34688) 2023-11-15T03:10:22.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 learned_addr learned my addr 172.21.15.121:0/759729981 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:22.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef5fc15700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562423a6dd20 0x562423a6e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef56c13700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562423a78000 0x562423a77180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562423a6dd20 msgr2=0x562423a6e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef5f414700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562423a6dd20 0x562423a6e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562423a78000 msgr2=0x562423a77180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef5f414700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562423a78000 0x562423a77180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562423939a20 con 0x5624239e6b20 2023-11-15T03:10:22.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef5f414700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624239e6b20 0x5624239dfc50 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 rx=0x7fef50005ba0 tx=0x7fef5001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:22.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef5cc0f700 1 -- 172.21.15.121:0/759729981 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5000ac90 con 0x5624239e6b20 2023-11-15T03:10:22.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef5cc0f700 1 -- 172.21.15.121:0/759729981 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5000d740 con 0x5624239e6b20 2023-11-15T03:10:22.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562423939770 con 0x5624239e6b20 2023-11-15T03:10:22.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef5cc0f700 1 -- 172.21.15.121:0/759729981 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef50010650 con 0x5624239e6b20 2023-11-15T03:10:22.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.393+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562423908a70 con 0x5624239e6b20 2023-11-15T03:10:22.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.397+0000 7fef5cc0f700 1 -- 172.21.15.121:0/759729981 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fef50017070 con 0x5624239e6b20 2023-11-15T03:10:22.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.397+0000 7fef5cc0f700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef40059020 0x7fef4005b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.397+0000 7fef5cc0f700 1 -- 172.21.15.121:0/759729981 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9362+0+0 (secure 0 0 0) 0x7fef50076b30 con 0x5624239e6b20 2023-11-15T03:10:22.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.397+0000 7fef56c13700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef40059020 0x7fef4005b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.413 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:10:22.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.401+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x562423a88680 0x562423a88a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.401+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a88f50 con 0x562423a88680 2023-11-15T03:10:22.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.401+0000 7fef5fc15700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x562423a88680 0x562423a88a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.401+0000 7fef56c13700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef40059020 0x7fef4005b4e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x562423a74660 tx=0x7fef4800c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:22.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.401+0000 7fef5fc15700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x562423a88680 0x562423a88a60 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:22.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.401+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v130'425 uv418 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a88680 2023-11-15T03:10:22.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.405+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423a91540 con 0x562423a88680 2023-11-15T03:10:22.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.405+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v130'426 uv426 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a88680 2023-11-15T03:10:22.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.405+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a918f0 con 0x562423a88680 2023-11-15T03:10:22.417 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:10:22.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.409+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v130'427 uv427 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a88680 2023-11-15T03:10:22.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.409+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a92cb0 con 0x562423a88680 2023-11-15T03:10:22.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.409+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v130'428 uv428 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a88680 2023-11-15T03:10:22.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.409+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a93060 con 0x562423a88680 2023-11-15T03:10:22.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.413+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v130'429 uv429 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a88680 2023-11-15T03:10:22.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.413+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a91ca0 con 0x562423a88680 2023-11-15T03:10:22.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.417+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v130'430 uv430 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a88680 2023-11-15T03:10:22.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.417+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a920a0 con 0x562423a88680 2023-11-15T03:10:22.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.417+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v130'431 uv424 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a88680 2023-11-15T03:10:22.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.421+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423a924a0 con 0x562423a88680 2023-11-15T03:10:22.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.425+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v130'432 uv432 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a88680 2023-11-15T03:10:22.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.425+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a924a0 con 0x562423a88680 2023-11-15T03:10:22.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.425+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v130'433 uv433 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a88680 2023-11-15T03:10:22.435 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:10:22.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.429+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a92450 con 0x562423a88680 2023-11-15T03:10:22.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.429+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v130'434 uv434 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a88680 2023-11-15T03:10:22.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.429+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a92450 con 0x562423a88680 2023-11-15T03:10:22.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.433+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v130'435 uv435 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a88680 2023-11-15T03:10:22.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.433+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a92450 con 0x562423a88680 2023-11-15T03:10:22.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.437+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v130'436 uv436 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a88680 2023-11-15T03:10:22.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.437+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x562423a93410 0x562423a95800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.437+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a95cf0 con 0x562423a93410 2023-11-15T03:10:22.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.437+0000 7fef5f414700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x562423a93410 0x562423a95800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.441+0000 7fef5f414700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x562423a93410 0x562423a95800 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:22.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.445+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v130'673 uv659 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.445+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423a960a0 con 0x562423a93410 2023-11-15T03:10:22.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.449+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v130'674 uv674 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.449+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a96450 con 0x562423a93410 2023-11-15T03:10:22.457 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:10:22.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.449+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v130'675 uv675 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.449+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a97810 con 0x562423a93410 2023-11-15T03:10:22.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.453+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v130'676 uv676 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.453+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a97bc0 con 0x562423a93410 2023-11-15T03:10:22.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.457+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v130'677 uv677 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.457+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a96800 con 0x562423a93410 2023-11-15T03:10:22.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.461+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v130'678 uv678 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.461+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a96c00 con 0x562423a93410 2023-11-15T03:10:22.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.461+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v130'679 uv679 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.461+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a96fb0 con 0x562423a93410 2023-11-15T03:10:22.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.465+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v130'680 uv672 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.465+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423a96fb0 con 0x562423a93410 2023-11-15T03:10:22.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.469+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v130'681 uv681 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.469+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a96fb0 con 0x562423a93410 2023-11-15T03:10:22.478 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:10:22.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.469+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v130'682 uv682 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.473+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a96fb0 con 0x562423a93410 2023-11-15T03:10:22.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.473+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v130'683 uv683 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.473+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a96fb0 con 0x562423a93410 2023-11-15T03:10:22.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.477+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v130'684 uv684 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.477+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a96fb0 con 0x562423a93410 2023-11-15T03:10:22.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.481+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v130'685 uv685 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.481+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a96fb0 con 0x562423a93410 2023-11-15T03:10:22.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.485+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v130'686 uv671 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.485+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423a96fb0 con 0x562423a93410 2023-11-15T03:10:22.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.485+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v130'687 uv687 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.485+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a96fb0 con 0x562423a93410 2023-11-15T03:10:22.496 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:10:22.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.489+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v130'688 uv688 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.489+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a96fb0 con 0x562423a93410 2023-11-15T03:10:22.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.493+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v130'689 uv689 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.493+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a96fb0 con 0x562423a93410 2023-11-15T03:10:22.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.493+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v130'690 uv690 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.493+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a96fb0 con 0x562423a93410 2023-11-15T03:10:22.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.497+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v130'691 uv691 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.497+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x562423a973b0 0x562423a9a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.497+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a96fb0 con 0x562423a973b0 2023-11-15T03:10:22.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.497+0000 7fef56c13700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x562423a973b0 0x562423a9a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.501+0000 7fef56c13700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x562423a973b0 0x562423a9a490 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:22.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.501+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v130'367 uv361 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423a973b0 2023-11-15T03:10:22.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.501+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423a9a980 con 0x562423a973b0 2023-11-15T03:10:22.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.505+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v130'368 uv368 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423a973b0 2023-11-15T03:10:22.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.505+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a9ad30 con 0x562423a973b0 2023-11-15T03:10:22.515 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:10:22.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.509+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v130'369 uv369 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423a973b0 2023-11-15T03:10:22.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.509+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a9b650 con 0x562423a973b0 2023-11-15T03:10:22.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.513+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v130'370 uv370 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423a973b0 2023-11-15T03:10:22.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.513+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a9ba00 con 0x562423a973b0 2023-11-15T03:10:22.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.513+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v130'371 uv371 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423a973b0 2023-11-15T03:10:22.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.513+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a9bdb0 con 0x562423a973b0 2023-11-15T03:10:22.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.517+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v130'372 uv366 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423a973b0 2023-11-15T03:10:22.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.517+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423a9c160 con 0x562423a973b0 2023-11-15T03:10:22.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.521+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v130'373 uv373 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423a973b0 2023-11-15T03:10:22.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.521+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a9c510 con 0x562423a973b0 2023-11-15T03:10:22.530 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:10:22.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.525+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v130'374 uv374 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423a973b0 2023-11-15T03:10:22.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.525+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a9c510 con 0x562423a973b0 2023-11-15T03:10:22.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.525+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v130'375 uv375 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423a973b0 2023-11-15T03:10:22.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.525+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a9b1e0 con 0x562423a973b0 2023-11-15T03:10:22.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.529+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v130'376 uv376 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423a973b0 2023-11-15T03:10:22.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.529+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x562423a9cf10 0x562423a9f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.529+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a9f7f0 con 0x562423a9cf10 2023-11-15T03:10:22.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.529+0000 7fef5fc15700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x562423a9cf10 0x562423a9f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.533+0000 7fef5fc15700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x562423a9cf10 0x562423a9f300 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:22.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.537+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v130'432 uv424 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.537+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423a9f7f0 con 0x562423a9cf10 2023-11-15T03:10:22.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.541+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v130'433 uv433 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.541+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a9f7f0 con 0x562423a9cf10 2023-11-15T03:10:22.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.541+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v130'434 uv434 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.549 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:10:22.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.541+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa3880 con 0x562423a9cf10 2023-11-15T03:10:22.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.545+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v130'435 uv435 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.545+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa3880 con 0x562423a9cf10 2023-11-15T03:10:22.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.549+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v130'436 uv436 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.549+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a9f7f0 con 0x562423a9cf10 2023-11-15T03:10:22.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.549+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v130'437 uv430 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.553+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423a9f7f0 con 0x562423a9cf10 2023-11-15T03:10:22.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.553+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v130'438 uv438 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.553+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a9f7f0 con 0x562423a9cf10 2023-11-15T03:10:22.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.557+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v130'439 uv439 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.564 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:10:22.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.557+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa0800 con 0x562423a9cf10 2023-11-15T03:10:22.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.561+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v130'440 uv440 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.561+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa0800 con 0x562423a9cf10 2023-11-15T03:10:22.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.561+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v130'441 uv441 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.565+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423a9f7f0 con 0x562423a9cf10 2023-11-15T03:10:22.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.565+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v130'442 uv442 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.565+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x562423aa0100 0x562423aa24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.565+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa29e0 con 0x562423aa0100 2023-11-15T03:10:22.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.565+0000 7fef5f414700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x562423aa0100 0x562423aa24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.569+0000 7fef5f414700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x562423aa0100 0x562423aa24f0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:22.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.569+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v130'213 uv212 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423aa0100 2023-11-15T03:10:22.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.569+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423aa2aa0 con 0x562423aa0100 2023-11-15T03:10:22.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.573+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v130'214 uv214 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423aa0100 2023-11-15T03:10:22.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.573+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa2ed0 con 0x562423aa0100 2023-11-15T03:10:22.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.577+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v130'215 uv215 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423aa0100 2023-11-15T03:10:22.584 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:10:22.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.577+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa3ee0 con 0x562423aa0100 2023-11-15T03:10:22.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.581+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v130'216 uv216 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423aa0100 2023-11-15T03:10:22.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.581+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa3ee0 con 0x562423aa0100 2023-11-15T03:10:22.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.585+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v130'217 uv217 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423aa0100 2023-11-15T03:10:22.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.585+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa2ed0 con 0x562423aa0100 2023-11-15T03:10:22.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.585+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v130'218 uv218 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423aa0100 2023-11-15T03:10:22.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.585+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562423aa33e0 0x562423aa57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.585+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa5ed0 con 0x562423aa33e0 2023-11-15T03:10:22.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.585+0000 7fef56c13700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562423aa33e0 0x562423aa57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.589+0000 7fef56c13700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562423aa33e0 0x562423aa57d0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:22.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.589+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v130'154 uv153 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.589+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423aa62b0 con 0x562423aa33e0 2023-11-15T03:10:22.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.593+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v130'155 uv155 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.597+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa62b0 con 0x562423aa33e0 2023-11-15T03:10:22.605 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:10:22.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.597+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v130'156 uv156 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.597+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa62b0 con 0x562423aa33e0 2023-11-15T03:10:22.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.601+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v130'157 uv157 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.601+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa62b0 con 0x562423aa33e0 2023-11-15T03:10:22.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.605+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v130'213 uv212 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.605+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423aa62b0 con 0x562423aa33e0 2023-11-15T03:10:22.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.609+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v130'214 uv214 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.609+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa62b0 con 0x562423aa33e0 2023-11-15T03:10:22.618 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:10:22.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.609+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v130'215 uv215 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.609+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa72c0 con 0x562423aa33e0 2023-11-15T03:10:22.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.613+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v130'216 uv216 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.613+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa72c0 con 0x562423aa33e0 2023-11-15T03:10:22.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.617+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v130'217 uv217 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.617+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa62b0 con 0x562423aa33e0 2023-11-15T03:10:22.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.621+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v130'218 uv218 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.621+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa62b0 con 0x562423a93410 2023-11-15T03:10:22.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.621+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v130'393 uv387 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.621+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423aa62b0 con 0x562423a93410 2023-11-15T03:10:22.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.625+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v130'394 uv394 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.625+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa62b0 con 0x562423a93410 2023-11-15T03:10:22.637 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:10:22.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.629+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v130'395 uv395 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.629+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7300 con 0x562423a93410 2023-11-15T03:10:22.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.633+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v130'396 uv396 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.633+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa72c0 con 0x562423a93410 2023-11-15T03:10:22.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.637+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v130'397 uv397 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.637+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa62b0 con 0x562423a93410 2023-11-15T03:10:22.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.637+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v130'398 uv398 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.637+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa62b0 con 0x562423a93410 2023-11-15T03:10:22.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.641+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v130'399 uv392 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.641+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423aa62b0 con 0x562423a93410 2023-11-15T03:10:22.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.645+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v130'400 uv400 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.645+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa62b0 con 0x562423a93410 2023-11-15T03:10:22.655 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:10:22.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.649+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v130'401 uv401 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.649+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa62b0 con 0x562423a93410 2023-11-15T03:10:22.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.649+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v130'402 uv402 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.649+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa66b0 con 0x562423a93410 2023-11-15T03:10:22.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.653+0000 7fef5f414700 1 -- 172.21.15.121:0/759729981 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v130'403 uv403 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5002c020 con 0x562423a93410 2023-11-15T03:10:22.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.653+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa66b0 con 0x562423a973b0 2023-11-15T03:10:22.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.657+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v130'178 uv177 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423a973b0 2023-11-15T03:10:22.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.657+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423aa66b0 con 0x562423a973b0 2023-11-15T03:10:22.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.657+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v130'179 uv179 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423a973b0 2023-11-15T03:10:22.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.657+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa66b0 con 0x562423a973b0 2023-11-15T03:10:22.668 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:10:22.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.661+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v130'180 uv180 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423a973b0 2023-11-15T03:10:22.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.661+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa66b0 con 0x562423a973b0 2023-11-15T03:10:22.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.665+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v130'181 uv181 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423a973b0 2023-11-15T03:10:22.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.665+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa66b0 con 0x562423a973b0 2023-11-15T03:10:22.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.665+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v130'182 uv182 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423a973b0 2023-11-15T03:10:22.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.665+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa66b0 con 0x562423aa33e0 2023-11-15T03:10:22.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.669+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v130'213 uv212 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.669+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423aa66b0 con 0x562423aa33e0 2023-11-15T03:10:22.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.673+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v130'214 uv214 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.673+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa66b0 con 0x562423aa33e0 2023-11-15T03:10:22.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.673+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v130'215 uv215 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.682 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:10:22.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.677+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa76c0 con 0x562423aa33e0 2023-11-15T03:10:22.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.677+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v130'216 uv216 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.677+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa76c0 con 0x562423aa33e0 2023-11-15T03:10:22.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.681+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v130'217 uv217 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.681+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa66b0 con 0x562423aa33e0 2023-11-15T03:10:22.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.685+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v130'218 uv218 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.685+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa66b0 con 0x562423aa33e0 2023-11-15T03:10:22.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.685+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v130'179 uv178 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.685+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423aa66b0 con 0x562423aa33e0 2023-11-15T03:10:22.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.689+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v130'180 uv180 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.689+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa66b0 con 0x562423aa33e0 2023-11-15T03:10:22.698 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:10:22.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.689+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v130'181 uv181 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.689+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa66b0 con 0x562423aa33e0 2023-11-15T03:10:22.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.693+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v130'182 uv182 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.693+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa66b0 con 0x562423aa33e0 2023-11-15T03:10:22.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.697+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v130'183 uv183 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.697+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa66b0 con 0x562423aa33e0 2023-11-15T03:10:22.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.701+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v130'177 uv176 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.701+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423aa66b0 con 0x562423aa33e0 2023-11-15T03:10:22.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.701+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v130'178 uv178 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.701+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa66b0 con 0x562423aa33e0 2023-11-15T03:10:22.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.705+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v130'179 uv179 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.716 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:10:22.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.705+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa66b0 con 0x562423aa33e0 2023-11-15T03:10:22.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.709+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v130'180 uv180 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.709+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa9420 con 0x562423aa33e0 2023-11-15T03:10:22.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.709+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v130'181 uv181 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.709+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x562423aa70b0 0x562423aa7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.709+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423aa70b0 2023-11-15T03:10:22.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.713+0000 7fef5fc15700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x562423aa70b0 0x562423aa7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.713+0000 7fef5fc15700 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x562423aa70b0 0x562423aa7490 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:22.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.713+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v130'142 uv141 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423aa70b0 2023-11-15T03:10:22.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.717+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423aa70b0 2023-11-15T03:10:22.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.717+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v130'143 uv143 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423aa70b0 2023-11-15T03:10:22.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.717+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423aa70b0 2023-11-15T03:10:22.728 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:10:22.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.721+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v130'144 uv144 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423aa70b0 2023-11-15T03:10:22.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.721+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423aa70b0 2023-11-15T03:10:22.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.725+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v130'145 uv145 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423aa70b0 2023-11-15T03:10:22.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.725+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423aa33e0 2023-11-15T03:10:22.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.729+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v130'213 uv212 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.729+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423aa33e0 2023-11-15T03:10:22.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.729+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v130'214 uv214 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.729+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423aa33e0 2023-11-15T03:10:22.741 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:10:22.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.733+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v130'215 uv215 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.733+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423aa33e0 2023-11-15T03:10:22.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.737+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v130'216 uv216 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.737+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423aa33e0 2023-11-15T03:10:22.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.737+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v130'217 uv217 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.737+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423aa33e0 2023-11-15T03:10:22.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.741+0000 7fef56c13700 1 -- 172.21.15.121:0/759729981 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v130'218 uv218 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48007b10 con 0x562423aa33e0 2023-11-15T03:10:22.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.741+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423a9cf10 2023-11-15T03:10:22.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.745+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v130'181 uv180 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.745+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423a9cf10 2023-11-15T03:10:22.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.745+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v130'182 uv182 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.749+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423a9cf10 2023-11-15T03:10:22.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.749+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v130'183 uv183 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.757 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:10:22.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.749+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423a9cf10 2023-11-15T03:10:22.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.753+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v130'184 uv184 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.753+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423a9cf10 2023-11-15T03:10:22.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.757+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v130'185 uv185 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.757+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423a9cf10 2023-11-15T03:10:22.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.757+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v130'427 uv419 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.757+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423a9cf10 2023-11-15T03:10:22.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.761+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v130'428 uv428 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.761+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423a9cf10 2023-11-15T03:10:22.771 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:10:22.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.765+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v130'429 uv429 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.765+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423a9cf10 2023-11-15T03:10:22.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.769+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v130'430 uv430 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.769+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423a9cf10 2023-11-15T03:10:22.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.769+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v130'431 uv431 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.769+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423a9cf10 2023-11-15T03:10:22.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.773+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v130'432 uv426 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.773+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423a9cf10 2023-11-15T03:10:22.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.777+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v130'433 uv433 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.777+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423a9cf10 2023-11-15T03:10:22.788 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:10:22.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.777+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v130'434 uv434 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.777+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423a9cf10 2023-11-15T03:10:22.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.781+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v130'435 uv435 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.781+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423a9cf10 2023-11-15T03:10:22.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.785+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v130'436 uv436 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.785+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7b60 con 0x562423a9cf10 2023-11-15T03:10:22.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.785+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v130'437 uv437 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.785+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x562423aa7f60 con 0x562423a9cf10 2023-11-15T03:10:22.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v130'438 uv438 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef5800f7b0 con 0x562423a9cf10 2023-11-15T03:10:22.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x562423a973b0 msgr2=0x562423a9a490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x562423a973b0 0x562423a9a490 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x562423a88680 msgr2=0x562423a88a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x562423a88680 0x562423a88a60 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x562423aa0100 msgr2=0x562423aa24f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x562423aa0100 0x562423aa24f0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x562423a93410 msgr2=0x562423a95800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x562423a93410 0x562423a95800 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x562423aa70b0 msgr2=0x562423aa7490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x562423aa70b0 0x562423aa7490 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x562423a9cf10 msgr2=0x562423a9f300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x562423a9cf10 0x562423a9f300 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562423aa33e0 msgr2=0x562423aa57d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x562423aa33e0 0x562423aa57d0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 reap_dead start 2023-11-15T03:10:22.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 reap_dead start 2023-11-15T03:10:22.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef5fc15700 1 -- 172.21.15.121:0/759729981 reap_dead start 2023-11-15T03:10:22.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef40059020 msgr2=0x7fef4005b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef40059020 0x7fef4005b4e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x562423a74660 tx=0x7fef4800c040).stop 2023-11-15T03:10:22.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624239e6b20 msgr2=0x5624239dfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624239e6b20 0x5624239dfc50 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 rx=0x7fef50005ba0 tx=0x7fef5001a000).stop 2023-11-15T03:10:22.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 shutdown_connections 2023-11-15T03:10:22.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef40059020 0x7fef4005b4e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 --2- 172.21.15.121:0/759729981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624239e6b20 0x5624239dfc50 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 >> 172.21.15.121:0/759729981 conn(0x5624239e7080 msgr2=0x5624239e3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:22.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 shutdown_connections 2023-11-15T03:10:22.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.789+0000 7fef6ad36d00 1 -- 172.21.15.121:0/759729981 wait complete. 2023-11-15T03:10:22.805 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader mds0_openfiles.0 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-11-15T03:10:22.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.857+0000 7fdb07534d00 1 Processor -- start 2023-11-15T03:10:22.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.857+0000 7fdb07534d00 1 -- start start 2023-11-15T03:10:22.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdb07534d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc34311920 0x55cc343571f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdb07534d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cc34358be0 0x55cc34356060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdb07534d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc343e8130 0x55cc343f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdb07534d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc342bbcc0 con 0x55cc34358be0 2023-11-15T03:10:22.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdb07534d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc34286220 con 0x55cc34311920 2023-11-15T03:10:22.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdb07534d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc343e6f90 con 0x55cc343e8130 2023-11-15T03:10:22.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdaf3413700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cc34358be0 0x55cc34356060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdaf3413700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cc34358be0 0x55cc34356060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34704/0 (socket says 172.21.15.121:34704) 2023-11-15T03:10:22.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdaf3413700 1 -- 172.21.15.121:0/122199863 learned_addr learned my addr 172.21.15.121:0/122199863 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:22.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdafbc14700 1 --2- 172.21.15.121:0/122199863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc34311920 0x55cc343571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdafc415700 1 --2- 172.21.15.121:0/122199863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc343e8130 0x55cc343f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdafbc14700 1 -- 172.21.15.121:0/122199863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc343e8130 msgr2=0x55cc343f2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:22.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdafbc14700 1 --2- 172.21.15.121:0/122199863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc343e8130 0x55cc343f2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdafbc14700 1 -- 172.21.15.121:0/122199863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cc34358be0 msgr2=0x55cc34356060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:22.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdafbc14700 1 --2- 172.21.15.121:0/122199863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cc34358be0 0x55cc34356060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdafbc14700 1 -- 172.21.15.121:0/122199863 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae4005cf0 con 0x55cc34311920 2023-11-15T03:10:22.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdaf3413700 1 --2- 172.21.15.121:0/122199863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cc34358be0 0x55cc34356060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:10:22.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdafbc14700 1 --2- 172.21.15.121:0/122199863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc34311920 0x55cc343571f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fdaec001d50 tx=0x7fdaec001800).ready entity=mon.1 client_cookie=638f6619a8b43e3d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:22.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdafac12700 1 -- 172.21.15.121:0/122199863 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec00bd40 con 0x55cc34311920 2023-11-15T03:10:22.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdafac12700 1 -- 172.21.15.121:0/122199863 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaec00f450 con 0x55cc34311920 2023-11-15T03:10:22.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdafac12700 1 -- 172.21.15.121:0/122199863 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec00bea0 con 0x55cc34311920 2023-11-15T03:10:22.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdb07534d00 1 -- 172.21.15.121:0/122199863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc34311920 msgr2=0x55cc343571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdb07534d00 1 --2- 172.21.15.121:0/122199863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc34311920 0x55cc343571f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fdaec001d50 tx=0x7fdaec001800).stop 2023-11-15T03:10:22.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdb07534d00 1 -- 172.21.15.121:0/122199863 shutdown_connections 2023-11-15T03:10:22.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdb07534d00 1 --2- 172.21.15.121:0/122199863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc343e8130 0x55cc343f2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdb07534d00 1 --2- 172.21.15.121:0/122199863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cc34358be0 0x55cc34356060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdb07534d00 1 --2- 172.21.15.121:0/122199863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc34311920 0x55cc343571f0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdb07534d00 1 -- 172.21.15.121:0/122199863 >> 172.21.15.121:0/122199863 conn(0x55cc343578f0 msgr2=0x55cc343dd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:22.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdb07534d00 1 -- 172.21.15.121:0/122199863 shutdown_connections 2023-11-15T03:10:22.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.861+0000 7fdb07534d00 1 -- 172.21.15.121:0/122199863 wait complete. 2023-11-15T03:10:22.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdb07534d00 1 Processor -- start 2023-11-15T03:10:22.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdb07534d00 1 -- start start 2023-11-15T03:10:22.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdb07534d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc34311920 0x55cc3433fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdb07534d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc34358be0 0x55cc3433ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdb07534d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cc343e8130 0x55cc34351ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdb07534d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc34340490 con 0x55cc343e8130 2023-11-15T03:10:22.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdb07534d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc34353d20 con 0x55cc34358be0 2023-11-15T03:10:22.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdb07534d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc34340c50 con 0x55cc34311920 2023-11-15T03:10:22.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdafc415700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cc343e8130 0x55cc34351ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdafc415700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cc343e8130 0x55cc34351ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34712/0 (socket says 172.21.15.121:34712) 2023-11-15T03:10:22.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdafc415700 1 -- 172.21.15.121:0/2884270853 learned_addr learned my addr 172.21.15.121:0/2884270853 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:22.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdaf3413700 1 --2- 172.21.15.121:0/2884270853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc34358be0 0x55cc3433ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdafbc14700 1 --2- 172.21.15.121:0/2884270853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc34311920 0x55cc3433fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdafc415700 1 -- 172.21.15.121:0/2884270853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc34311920 msgr2=0x55cc3433fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdafc415700 1 --2- 172.21.15.121:0/2884270853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc34311920 0x55cc3433fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdafc415700 1 -- 172.21.15.121:0/2884270853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc34358be0 msgr2=0x55cc3433ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdafc415700 1 --2- 172.21.15.121:0/2884270853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc34358be0 0x55cc3433ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdafc415700 1 -- 172.21.15.121:0/2884270853 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc342af770 con 0x55cc343e8130 2023-11-15T03:10:22.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdafc415700 1 --2- 172.21.15.121:0/2884270853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cc343e8130 0x55cc34351ef0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 rx=0x7fdaf400d4f0 tx=0x7fdaf4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:22.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdaf940f700 1 -- 172.21.15.121:0/2884270853 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf40060d0 con 0x55cc343e8130 2023-11-15T03:10:22.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdaf940f700 1 -- 172.21.15.121:0/2884270853 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf4006230 con 0x55cc343e8130 2023-11-15T03:10:22.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdaf940f700 1 -- 172.21.15.121:0/2884270853 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4004750 con 0x55cc343e8130 2023-11-15T03:10:22.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdb07534d00 1 -- 172.21.15.121:0/2884270853 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae8001e10 con 0x55cc343e8130 2023-11-15T03:10:22.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.865+0000 7fdb07534d00 1 -- 172.21.15.121:0/2884270853 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc3427ea70 con 0x55cc343e8130 2023-11-15T03:10:22.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.869+0000 7fdaf940f700 1 -- 172.21.15.121:0/2884270853 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fdaf4004b40 con 0x55cc343e8130 2023-11-15T03:10:22.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.869+0000 7fdaf940f700 1 --2- 172.21.15.121:0/2884270853 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdadc059020 0x7fdadc05b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.869+0000 7fdaf940f700 1 -- 172.21.15.121:0/2884270853 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9362+0+0 (secure 0 0 0) 0x7fdaf4073ff0 con 0x55cc343e8130 2023-11-15T03:10:22.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.869+0000 7fdafbc14700 1 --2- 172.21.15.121:0/2884270853 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdadc059020 0x7fdadc05b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.869+0000 7fdb07534d00 1 --2- 172.21.15.121:0/2884270853 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55cc343e5ad0 0x55cc343f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:22.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.869+0000 7fdb07534d00 1 -- 172.21.15.121:0/2884270853 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected e130) v8 -- 0x55cc34356aa0 con 0x55cc343e5ad0 2023-11-15T03:10:22.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.873+0000 7fdafbc14700 1 --2- 172.21.15.121:0/2884270853 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdadc059020 0x7fdadc05b4e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7fdaec001ed0 tx=0x7fdaec014000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:22.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.873+0000 7fdaf3413700 1 --2- 172.21.15.121:0/2884270853 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55cc343e5ad0 0x55cc343f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:22.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.873+0000 7fdaf3413700 1 --2- 172.21.15.121:0/2884270853 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55cc343e5ad0 0x55cc343f7050 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:22.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.877+0000 7fdaf3413700 1 -- 172.21.15.121:0/2884270853 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-set-header] v130'158 uv158 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fdae4000bc0 con 0x55cc343e5ad0 2023-11-15T03:10:22.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.877+0000 7fdb07534d00 1 -- 172.21.15.121:0/2884270853 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55cc343e5ad0 msgr2=0x55cc343f7050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.877+0000 7fdb07534d00 1 --2- 172.21.15.121:0/2884270853 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55cc343e5ad0 0x55cc343f7050 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:22.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.877+0000 7fdb07534d00 1 -- 172.21.15.121:0/2884270853 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdadc059020 msgr2=0x7fdadc05b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.877+0000 7fdb07534d00 1 --2- 172.21.15.121:0/2884270853 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdadc059020 0x7fdadc05b4e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7fdaec001ed0 tx=0x7fdaec014000).stop 2023-11-15T03:10:22.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.877+0000 7fdb07534d00 1 -- 172.21.15.121:0/2884270853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cc343e8130 msgr2=0x55cc34351ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:22.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.877+0000 7fdb07534d00 1 --2- 172.21.15.121:0/2884270853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cc343e8130 0x55cc34351ef0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 rx=0x7fdaf400d4f0 tx=0x7fdaf4008040).stop 2023-11-15T03:10:22.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.877+0000 7fdafc415700 1 -- 172.21.15.121:0/2884270853 reap_dead start 2023-11-15T03:10:22.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.877+0000 7fdb07534d00 1 -- 172.21.15.121:0/2884270853 shutdown_connections 2023-11-15T03:10:22.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.877+0000 7fdb07534d00 1 -- 172.21.15.121:0/2884270853 >> 172.21.15.121:0/2884270853 conn(0x55cc343578f0 msgr2=0x55cc34311dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:22.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.877+0000 7fdb07534d00 1 -- 172.21.15.121:0/2884270853 shutdown_connections 2023-11-15T03:10:22.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:22.877+0000 7fdb07534d00 1 -- 172.21.15.121:0/2884270853 wait complete. 2023-11-15T03:10:22.893 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:10:23.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.057+0000 7eff18ef1700 1 Processor -- start 2023-11-15T03:10:23.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.057+0000 7eff18ef1700 1 -- start start 2023-11-15T03:10:23.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff18ef1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff14067ec0 0x7eff140682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:23.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff18ef1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7eff140c10b0 0x7eff14062c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:23.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff18ef1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7eff14063120 0x7eff14153700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:23.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff18ef1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff140c0c20 con 0x7eff140c10b0 2023-11-15T03:10:23.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff18ef1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff14158730 con 0x7eff14063120 2023-11-15T03:10:23.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff18ef1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff140c02f0 con 0x7eff14067ec0 2023-11-15T03:10:23.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff0bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7eff140c10b0 0x7eff14062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:23.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff0bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7eff140c10b0 0x7eff14062c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34716/0 (socket says 172.21.15.121:34716) 2023-11-15T03:10:23.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff0bfff700 1 -- 172.21.15.121:0/182811294 learned_addr learned my addr 172.21.15.121:0/182811294 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:23.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff12d9d700 1 --2- 172.21.15.121:0/182811294 >> v2:172.21.15.190:3300/0 conn(0x7eff14063120 0x7eff14153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:23.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff1259c700 1 --2- 172.21.15.121:0/182811294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff14067ec0 0x7eff140682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:23.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff0bfff700 1 -- 172.21.15.121:0/182811294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff14067ec0 msgr2=0x7eff140682a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:23.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff0bfff700 1 --2- 172.21.15.121:0/182811294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff14067ec0 0x7eff140682a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:23.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff0bfff700 1 -- 172.21.15.121:0/182811294 >> v2:172.21.15.190:3300/0 conn(0x7eff14063120 msgr2=0x7eff14153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:23.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff0bfff700 1 --2- 172.21.15.121:0/182811294 >> v2:172.21.15.190:3300/0 conn(0x7eff14063120 0x7eff14153700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:23.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff0bfff700 1 -- 172.21.15.121:0/182811294 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff140c05d0 con 0x7eff140c10b0 2023-11-15T03:10:23.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff12d9d700 1 --2- 172.21.15.121:0/182811294 >> v2:172.21.15.190:3300/0 conn(0x7eff14063120 0x7eff14153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:23.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff0bfff700 1 --2- 172.21.15.121:0/182811294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff140c10b0 0x7eff14062c30 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 rx=0x7eff140c01e0 tx=0x7efefc0090f0).ready entity=mon.0 client_cookie=4ee6f1447c3922d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:23.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff11d9b700 1 -- 172.21.15.121:0/182811294 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc00b600 con 0x7eff140c10b0 2023-11-15T03:10:23.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff11d9b700 1 -- 172.21.15.121:0/182811294 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efefc00b760 con 0x7eff140c10b0 2023-11-15T03:10:23.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.061+0000 7eff11d9b700 1 -- 172.21.15.121:0/182811294 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc00bae0 con 0x7eff140c10b0 2023-11-15T03:10:23.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.065+0000 7eff18ef1700 1 -- 172.21.15.121:0/182811294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff140c10b0 msgr2=0x7eff14062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:23.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.065+0000 7eff18ef1700 1 --2- 172.21.15.121:0/182811294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff140c10b0 0x7eff14062c30 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 rx=0x7eff140c01e0 tx=0x7efefc0090f0).stop 2023-11-15T03:10:23.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.065+0000 7eff18ef1700 1 -- 172.21.15.121:0/182811294 shutdown_connections 2023-11-15T03:10:23.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.065+0000 7eff18ef1700 1 --2- 172.21.15.121:0/182811294 >> v2:172.21.15.190:3300/0 conn(0x7eff14063120 0x7eff14153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:23.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.065+0000 7eff18ef1700 1 --2- 172.21.15.121:0/182811294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff140c10b0 0x7eff14062c30 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:23.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.065+0000 7eff18ef1700 1 --2- 172.21.15.121:0/182811294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff14067ec0 0x7eff140682a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:23.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.065+0000 7eff18ef1700 1 -- 172.21.15.121:0/182811294 >> 172.21.15.121:0/182811294 conn(0x7eff140cad40 msgr2=0x7eff140cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:23.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.065+0000 7eff18ef1700 1 -- 172.21.15.121:0/182811294 shutdown_connections 2023-11-15T03:10:23.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.065+0000 7eff18ef1700 1 -- 172.21.15.121:0/182811294 wait complete. 2023-11-15T03:10:23.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff18ef1700 1 Processor -- start 2023-11-15T03:10:23.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff18ef1700 1 -- start start 2023-11-15T03:10:23.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff18ef1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff14063120 0x7eff14157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:23.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff18ef1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff14067ec0 0x7eff14141480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:23.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff18ef1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff140c10b0 0x7eff14155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:23.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff18ef1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff140c4580 con 0x7eff14063120 2023-11-15T03:10:23.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff18ef1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff140c3620 con 0x7eff140c10b0 2023-11-15T03:10:23.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff18ef1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff140c3900 con 0x7eff14067ec0 2023-11-15T03:10:23.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff1259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff14063120 0x7eff14157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:23.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff1259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff14063120 0x7eff14157e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34728/0 (socket says 172.21.15.121:34728) 2023-11-15T03:10:23.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff1259c700 1 -- 172.21.15.121:0/269050578 learned_addr learned my addr 172.21.15.121:0/269050578 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:23.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff0bfff700 1 --2- 172.21.15.121:0/269050578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff14067ec0 0x7eff14141480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:23.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff12d9d700 1 --2- 172.21.15.121:0/269050578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff140c10b0 0x7eff14155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:23.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff1259c700 1 -- 172.21.15.121:0/269050578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff14067ec0 msgr2=0x7eff14141480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:23.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff1259c700 1 --2- 172.21.15.121:0/269050578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff14067ec0 0x7eff14141480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:23.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff1259c700 1 -- 172.21.15.121:0/269050578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff140c10b0 msgr2=0x7eff14155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:23.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff1259c700 1 --2- 172.21.15.121:0/269050578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff140c10b0 0x7eff14155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:23.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff1259c700 1 -- 172.21.15.121:0/269050578 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc009370 con 0x7eff14063120 2023-11-15T03:10:23.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff12d9d700 1 --2- 172.21.15.121:0/269050578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff140c10b0 0x7eff14155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:23.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.069+0000 7eff1259c700 1 --2- 172.21.15.121:0/269050578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff14063120 0x7eff14157e10 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 rx=0x7eff0400d890 tx=0x7eff04008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:23.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.073+0000 7eff0a7fc700 1 -- 172.21.15.121:0/269050578 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff040125f0 con 0x7eff14063120 2023-11-15T03:10:23.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.073+0000 7eff0a7fc700 1 -- 172.21.15.121:0/269050578 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff04012750 con 0x7eff14063120 2023-11-15T03:10:23.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.073+0000 7eff0a7fc700 1 -- 172.21.15.121:0/269050578 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff04012ad0 con 0x7eff14063120 2023-11-15T03:10:23.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.073+0000 7eff18ef1700 1 -- 172.21.15.121:0/269050578 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff140c3be0 con 0x7eff14063120 2023-11-15T03:10:23.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.073+0000 7eff18ef1700 1 -- 172.21.15.121:0/269050578 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff140c41a0 con 0x7eff14063120 2023-11-15T03:10:23.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.073+0000 7eff18ef1700 1 -- 172.21.15.121:0/269050578 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff14047060 con 0x7eff14063120 2023-11-15T03:10:23.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.077+0000 7eff0a7fc700 1 -- 172.21.15.121:0/269050578 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7eff04013480 con 0x7eff14063120 2023-11-15T03:10:23.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.077+0000 7eff0a7fc700 1 --2- 172.21.15.121:0/269050578 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efef405a510 0x7efef405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:23.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.077+0000 7eff0a7fc700 1 -- 172.21.15.121:0/269050578 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9362+0+0 (secure 0 0 0) 0x7eff04075570 con 0x7eff14063120 2023-11-15T03:10:23.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.077+0000 7eff0bfff700 1 --2- 172.21.15.121:0/269050578 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efef405a510 0x7efef405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:23.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.077+0000 7eff0bfff700 1 --2- 172.21.15.121:0/269050578 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efef405a510 0x7efef405c9d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7eff140c01e0 tx=0x7efefc017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:23.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.085+0000 7eff0a7fc700 1 -- 172.21.15.121:0/269050578 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7eff04048000 con 0x7eff14063120 2023-11-15T03:10:23.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:23.285+0000 7eff18ef1700 1 -- 172.21.15.121:0/269050578 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7eff14153b50 con 0x7eff14063120 2023-11-15T03:10:24.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.201+0000 7eff0a7fc700 1 -- 172.21.15.121:0/269050578 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v315) v1 ==== 162+0+0 (secure 0 0 0) 0x7eff0404b620 con 0x7eff14063120 2023-11-15T03:10:24.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.205+0000 7eff18ef1700 1 -- 172.21.15.121:0/269050578 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efef405a510 msgr2=0x7efef405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:24.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.205+0000 7eff18ef1700 1 --2- 172.21.15.121:0/269050578 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efef405a510 0x7efef405c9d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7eff140c01e0 tx=0x7efefc017000).stop 2023-11-15T03:10:24.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.205+0000 7eff18ef1700 1 -- 172.21.15.121:0/269050578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff14063120 msgr2=0x7eff14157e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:24.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.205+0000 7eff18ef1700 1 --2- 172.21.15.121:0/269050578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff14063120 0x7eff14157e10 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 rx=0x7eff0400d890 tx=0x7eff04008040).stop 2023-11-15T03:10:24.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.205+0000 7eff18ef1700 1 -- 172.21.15.121:0/269050578 shutdown_connections 2023-11-15T03:10:24.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.205+0000 7eff18ef1700 1 --2- 172.21.15.121:0/269050578 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efef405a510 0x7efef405c9d0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:24.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.205+0000 7eff18ef1700 1 --2- 172.21.15.121:0/269050578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff140c10b0 0x7eff14155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:24.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.205+0000 7eff18ef1700 1 --2- 172.21.15.121:0/269050578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff14067ec0 0x7eff14141480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:24.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.205+0000 7eff18ef1700 1 --2- 172.21.15.121:0/269050578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff14063120 0x7eff14157e10 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:24.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.205+0000 7eff18ef1700 1 -- 172.21.15.121:0/269050578 >> 172.21.15.121:0/269050578 conn(0x7eff140cad40 msgr2=0x7eff140ca070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:24.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.205+0000 7eff18ef1700 1 -- 172.21.15.121:0/269050578 shutdown_connections 2023-11-15T03:10:24.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.205+0000 7eff18ef1700 1 -- 172.21.15.121:0/269050578 wait complete. 2023-11-15T03:10:24.216 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:10:24.218 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:10:24.213+0000 7f8855f9b700 -1 mds.0.openfiles _load_finish: corrupted header/values: End of buffer 2023-11-15T03:10:24.640 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:10:24.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.797+0000 7f9decdf9700 1 Processor -- start 2023-11-15T03:10:24.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.797+0000 7f9decdf9700 1 -- start start 2023-11-15T03:10:24.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.797+0000 7f9decdf9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9de80a9830 0x7f9de80a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:24.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.797+0000 7f9decdf9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9de80c30e0 0x7f9de80c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:24.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.797+0000 7f9decdf9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de80657f0 con 0x7f9de8151870 2023-11-15T03:10:24.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.797+0000 7f9decdf9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de8065ad0 con 0x7f9de80a9830 2023-11-15T03:10:24.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.797+0000 7f9decdf9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de80c3a60 con 0x7f9de80c30e0 2023-11-15T03:10:24.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.797+0000 7f9ddffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9de80a9830 0x7f9de80a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:24.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.797+0000 7f9ddffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9de80a9830 0x7f9de80a9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35876/0 (socket says 172.21.15.121:35876) 2023-11-15T03:10:24.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.797+0000 7f9ddffff700 1 -- 172.21.15.121:0/3820682357 learned_addr learned my addr 172.21.15.121:0/3820682357 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:24.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.797+0000 7f9de7fff700 1 --2- 172.21.15.121:0/3820682357 >> v2:172.21.15.190:3300/0 conn(0x7f9de80c30e0 0x7f9de80c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:24.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.801+0000 7f9de6ffd700 1 -- 172.21.15.121:0/3820682357 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408705771 0 0) 0x7f9de80657f0 con 0x7f9de8151870 2023-11-15T03:10:24.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.801+0000 7f9de6ffd700 1 -- 172.21.15.121:0/3820682357 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dd40033f0 con 0x7f9de8151870 2023-11-15T03:10:24.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.801+0000 7f9ddffff700 1 -- 172.21.15.121:0/3820682357 >> v2:172.21.15.190:3300/0 conn(0x7f9de80c30e0 msgr2=0x7f9de80c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:24.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.801+0000 7f9ddffff700 1 --2- 172.21.15.121:0/3820682357 >> v2:172.21.15.190:3300/0 conn(0x7f9de80c30e0 0x7f9de80c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:24.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.801+0000 7f9ddffff700 1 -- 172.21.15.121:0/3820682357 >> v1:172.21.15.190:6789/0 conn(0x7f9de8151870 legacy=0x7f9de8151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:24.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.801+0000 7f9ddffff700 1 -- 172.21.15.121:0/3820682357 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de80aa260 con 0x7f9de80a9830 2023-11-15T03:10:24.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.801+0000 7f9ddffff700 1 --2- 172.21.15.121:0/3820682357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9de80a9830 0x7f9de80a9c70 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 rx=0x7f9de8069a90 tx=0x7f9dd00090f0).ready entity=mon.0 client_cookie=b93dbfd8a5084aee server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:24.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.801+0000 7f9de6ffd700 1 -- 172.21.15.121:0/3820682357 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd000f510 con 0x7f9de80a9830 2023-11-15T03:10:24.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.801+0000 7f9de6ffd700 1 -- 172.21.15.121:0/3820682357 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd000f670 con 0x7f9de80a9830 2023-11-15T03:10:24.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.801+0000 7f9de6ffd700 1 -- 172.21.15.121:0/3820682357 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd000fa10 con 0x7f9de80a9830 2023-11-15T03:10:24.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.801+0000 7f9decdf9700 1 -- 172.21.15.121:0/3820682357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9de80a9830 msgr2=0x7f9de80a9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:24.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.801+0000 7f9decdf9700 1 --2- 172.21.15.121:0/3820682357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9de80a9830 0x7f9de80a9c70 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 rx=0x7f9de8069a90 tx=0x7f9dd00090f0).stop 2023-11-15T03:10:24.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.801+0000 7f9decdf9700 1 -- 172.21.15.121:0/3820682357 shutdown_connections 2023-11-15T03:10:24.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.801+0000 7f9decdf9700 1 --2- 172.21.15.121:0/3820682357 >> v2:172.21.15.190:3300/0 conn(0x7f9de80c30e0 0x7f9de80c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:24.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.801+0000 7f9decdf9700 1 --2- 172.21.15.121:0/3820682357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9de80a9830 0x7f9de80a9c70 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:24.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.801+0000 7f9decdf9700 1 -- 172.21.15.121:0/3820682357 >> 172.21.15.121:0/3820682357 conn(0x7f9de8062190 msgr2=0x7f9de8062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:24.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.801+0000 7f9decdf9700 1 -- 172.21.15.121:0/3820682357 shutdown_connections 2023-11-15T03:10:24.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.801+0000 7f9decdf9700 1 -- 172.21.15.121:0/3820682357 wait complete. 2023-11-15T03:10:24.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.805+0000 7f9decdf9700 1 Processor -- start 2023-11-15T03:10:24.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.805+0000 7f9decdf9700 1 -- start start 2023-11-15T03:10:24.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.805+0000 7f9decdf9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9de80a9830 0x7f9de813a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:24.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.805+0000 7f9decdf9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9de80c30e0 0x7f9de8151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:24.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.805+0000 7f9decdf9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9de8151870 0x7f9de8152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:24.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.805+0000 7f9decdf9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de80c1440 con 0x7f9de80c30e0 2023-11-15T03:10:24.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.805+0000 7f9decdf9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de8156f10 con 0x7f9de8151870 2023-11-15T03:10:24.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.805+0000 7f9decdf9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de81527c0 con 0x7f9de80a9830 2023-11-15T03:10:24.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.805+0000 7f9ddffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9de80c30e0 0x7f9de8151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:24.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.805+0000 7f9de77fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9de80a9830 0x7f9de813a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:24.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.805+0000 7f9ddffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9de80c30e0 0x7f9de8151c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35882/0 (socket says 172.21.15.121:35882) 2023-11-15T03:10:24.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.805+0000 7f9ddffff700 1 -- 172.21.15.121:0/164554614 learned_addr learned my addr 172.21.15.121:0/164554614 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:24.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.809+0000 7f9de7fff700 1 --2- 172.21.15.121:0/164554614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9de8151870 0x7f9de8152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:24.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.809+0000 7f9ddffff700 1 -- 172.21.15.121:0/164554614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9de80a9830 msgr2=0x7f9de813a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:24.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.809+0000 7f9ddffff700 1 --2- 172.21.15.121:0/164554614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9de80a9830 0x7f9de813a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:24.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.809+0000 7f9ddffff700 1 -- 172.21.15.121:0/164554614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9de8151870 msgr2=0x7f9de8152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:24.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.809+0000 7f9ddffff700 1 --2- 172.21.15.121:0/164554614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9de8151870 0x7f9de8152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:24.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.809+0000 7f9ddffff700 1 -- 172.21.15.121:0/164554614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd0009370 con 0x7f9de80c30e0 2023-11-15T03:10:24.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.809+0000 7f9ddffff700 1 --2- 172.21.15.121:0/164554614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9de80c30e0 0x7f9de8151c50 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 rx=0x7f9dd0006fd0 tx=0x7f9dd0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:24.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.809+0000 7f9ddf7fe700 1 -- 172.21.15.121:0/164554614 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0006950 con 0x7f9de80c30e0 2023-11-15T03:10:24.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.809+0000 7f9ddf7fe700 1 -- 172.21.15.121:0/164554614 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd0000bc0 con 0x7f9de80c30e0 2023-11-15T03:10:24.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.809+0000 7f9ddf7fe700 1 -- 172.21.15.121:0/164554614 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0024470 con 0x7f9de80c30e0 2023-11-15T03:10:24.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.809+0000 7f9decdf9700 1 -- 172.21.15.121:0/164554614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de8152aa0 con 0x7f9de80c30e0 2023-11-15T03:10:24.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.809+0000 7f9decdf9700 1 -- 172.21.15.121:0/164554614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de80c10d0 con 0x7f9de80c30e0 2023-11-15T03:10:24.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.809+0000 7f9decdf9700 1 -- 172.21.15.121:0/164554614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de806bcf0 con 0x7f9de80c30e0 2023-11-15T03:10:24.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.813+0000 7f9ddf7fe700 1 -- 172.21.15.121:0/164554614 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9dd0021070 con 0x7f9de80c30e0 2023-11-15T03:10:24.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.813+0000 7f9ddf7fe700 1 --2- 172.21.15.121:0/164554614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9dc805a510 0x7f9dc805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:24.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.813+0000 7f9ddf7fe700 1 -- 172.21.15.121:0/164554614 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9362+0+0 (secure 0 0 0) 0x7f9dd0076cd0 con 0x7f9de80c30e0 2023-11-15T03:10:24.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.813+0000 7f9de77fe700 1 --2- 172.21.15.121:0/164554614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9dc805a510 0x7f9dc805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:24.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.813+0000 7f9de77fe700 1 --2- 172.21.15.121:0/164554614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9dc805a510 0x7f9dc805c9d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f9dd8000c00 tx=0x7f9dd8007bb0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:24.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:24.817+0000 7f9ddf7fe700 1 -- 172.21.15.121:0/164554614 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9dd0049740 con 0x7f9de80c30e0 2023-11-15T03:10:25.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:25.025+0000 7f9decdf9700 1 -- 172.21.15.121:0/164554614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9de80aa540 con 0x7f9de80c30e0 2023-11-15T03:10:25.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:25.025+0000 7f9ddf7fe700 1 -- 172.21.15.121:0/164554614 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 316 v316) v1 ==== 95+0+4508 (secure 0 0 0) 0x7f9dd0029020 con 0x7f9de80c30e0 2023-11-15T03:10:25.034 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:10:25.035 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":316,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":316,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:10:24.203167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10242":{"gid":10242,"name":"a","rank":0,"incarnation":316,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/1750624080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1750624080},{"type":"v1","addr":"172.21.15.121:6836","nonce":1750624080}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:10:25.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:25.029+0000 7f9decdf9700 1 -- 172.21.15.121:0/164554614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9dc805a510 msgr2=0x7f9dc805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:25.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:25.029+0000 7f9decdf9700 1 --2- 172.21.15.121:0/164554614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9dc805a510 0x7f9dc805c9d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f9dd8000c00 tx=0x7f9dd8007bb0).stop 2023-11-15T03:10:25.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:25.029+0000 7f9decdf9700 1 -- 172.21.15.121:0/164554614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9de80c30e0 msgr2=0x7f9de8151c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:25.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:25.029+0000 7f9decdf9700 1 --2- 172.21.15.121:0/164554614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9de80c30e0 0x7f9de8151c50 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 rx=0x7f9dd0006fd0 tx=0x7f9dd0017000).stop 2023-11-15T03:10:25.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:25.029+0000 7f9decdf9700 1 -- 172.21.15.121:0/164554614 shutdown_connections 2023-11-15T03:10:25.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:25.029+0000 7f9decdf9700 1 --2- 172.21.15.121:0/164554614 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9dc805a510 0x7f9dc805c9d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:25.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:25.033+0000 7f9decdf9700 1 --2- 172.21.15.121:0/164554614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9de8151870 0x7f9de8152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:25.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:25.033+0000 7f9decdf9700 1 --2- 172.21.15.121:0/164554614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9de80c30e0 0x7f9de8151c50 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:25.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:25.033+0000 7f9decdf9700 1 --2- 172.21.15.121:0/164554614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9de80a9830 0x7f9de813a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:25.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:25.033+0000 7f9decdf9700 1 -- 172.21.15.121:0/164554614 >> 172.21.15.121:0/164554614 conn(0x7f9de8062190 msgr2=0x7f9de814d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:25.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:25.033+0000 7f9decdf9700 1 -- 172.21.15.121:0/164554614 shutdown_connections 2023-11-15T03:10:25.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:25.033+0000 7f9decdf9700 1 -- 172.21.15.121:0/164554614 wait complete. 2023-11-15T03:10:25.043 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 316 2023-11-15T03:10:25.058 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 316, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:10:24.203167+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 130, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10242}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10242': {'gid': 10242, 'name': 'a', 'rank': 0, 'incarnation': 316, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.121:6836/1750624080', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 1750624080}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 1750624080}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:10:25.059 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10242:up:replay 2023-11-15T03:10:25.059 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:10:28.272 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:10:28.265+0000 7f88597a2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:10:30.465 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:10:30.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.629+0000 7f40a128f700 1 Processor -- start 2023-11-15T03:10:30.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.629+0000 7f40a128f700 1 -- start start 2023-11-15T03:10:30.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.633+0000 7f40a128f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c0ae190 0x7f409c0ae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:30.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.633+0000 7f40a128f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409c0ad3a0 con 0x7f409c0cafb0 2023-11-15T03:10:30.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.633+0000 7f40a128f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409c0ad650 con 0x7f409c0cabd0 2023-11-15T03:10:30.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.633+0000 7f40a128f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c0cb410 con 0x7f409c0ae190 2023-11-15T03:10:30.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.633+0000 7f409b7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f409c0cafb0 0x7f409c066d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:46224/0 (socket says 172.21.15.121:46224) 2023-11-15T03:10:30.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.633+0000 7f409b7fe700 1 -- 172.21.15.121:0/1491123087 learned_addr learned my addr 172.21.15.121:0/1491123087 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:30.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.633+0000 7f40a0a8e700 1 --2- 172.21.15.121:0/1491123087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c0ae190 0x7f409c0ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:30.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.633+0000 7f409affd700 1 -- 172.21.15.121:0/1491123087 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019493234 0 0) 0x7f409c0ad3a0 con 0x7f409c0cafb0 2023-11-15T03:10:30.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.633+0000 7f409affd700 1 -- 172.21.15.121:0/1491123087 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40840033f0 con 0x7f409c0cafb0 2023-11-15T03:10:30.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.633+0000 7f40a0a8e700 1 -- 172.21.15.121:0/1491123087 >> v1:172.21.15.190:6789/0 conn(0x7f409c0cabd0 legacy=0x7f409c0acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:30.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.633+0000 7f409affd700 1 -- 172.21.15.121:0/1491123087 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678324588 0 0) 0x7f409c0ad650 con 0x7f409c0cabd0 2023-11-15T03:10:30.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.633+0000 7f40a0a8e700 1 -- 172.21.15.121:0/1491123087 >> v1:172.21.15.121:6789/0 conn(0x7f409c0cafb0 legacy=0x7f409c066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:30.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.633+0000 7f40a0a8e700 1 -- 172.21.15.121:0/1491123087 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c065fc0 con 0x7f409c0ae190 2023-11-15T03:10:30.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.637+0000 7f40a0a8e700 1 --2- 172.21.15.121:0/1491123087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c0ae190 0x7f409c0ae620 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f4094009e90 tx=0x7f4094009780).ready entity=mon.2 client_cookie=ac84413c8b9f8f23 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:30.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.637+0000 7f409affd700 1 -- 172.21.15.121:0/1491123087 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409400e070 con 0x7f409c0ae190 2023-11-15T03:10:30.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.637+0000 7f409affd700 1 -- 172.21.15.121:0/1491123087 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409400bb60 con 0x7f409c0ae190 2023-11-15T03:10:30.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.637+0000 7f409affd700 1 -- 172.21.15.121:0/1491123087 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409400bee0 con 0x7f409c0ae190 2023-11-15T03:10:30.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.637+0000 7f40a128f700 1 -- 172.21.15.121:0/1491123087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c0ae190 msgr2=0x7f409c0ae620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:30.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.637+0000 7f40a128f700 1 --2- 172.21.15.121:0/1491123087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c0ae190 0x7f409c0ae620 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f4094009e90 tx=0x7f4094009780).stop 2023-11-15T03:10:30.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.637+0000 7f40a128f700 1 -- 172.21.15.121:0/1491123087 shutdown_connections 2023-11-15T03:10:30.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.637+0000 7f40a128f700 1 --2- 172.21.15.121:0/1491123087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c0ae190 0x7f409c0ae620 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:30.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.637+0000 7f40a128f700 1 -- 172.21.15.121:0/1491123087 >> 172.21.15.121:0/1491123087 conn(0x7f409c062190 msgr2=0x7f409c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:30.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.637+0000 7f40a128f700 1 -- 172.21.15.121:0/1491123087 shutdown_connections 2023-11-15T03:10:30.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.637+0000 7f40a128f700 1 -- 172.21.15.121:0/1491123087 wait complete. 2023-11-15T03:10:30.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.637+0000 7f40a128f700 1 Processor -- start 2023-11-15T03:10:30.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f40a128f700 1 -- start start 2023-11-15T03:10:30.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f40a128f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f409c0ae190 0x7f409c0c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:30.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f40a128f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c0cabd0 0x7f409c155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:30.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f40a128f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f409c0cafb0 0x7f409c140490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:30.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f40a128f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c156a50 con 0x7f409c0cafb0 2023-11-15T03:10:30.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f40a128f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c155b80 con 0x7f409c0ae190 2023-11-15T03:10:30.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f40a128f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c155e40 con 0x7f409c0cabd0 2023-11-15T03:10:30.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f40a0a8e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f409c0cafb0 0x7f409c140490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:30.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f40a0a8e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f409c0cafb0 0x7f409c140490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35890/0 (socket says 172.21.15.121:35890) 2023-11-15T03:10:30.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f40a0a8e700 1 -- 172.21.15.121:0/683483048 learned_addr learned my addr 172.21.15.121:0/683483048 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:30.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f409b7fe700 1 --2- 172.21.15.121:0/683483048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c0cabd0 0x7f409c155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:30.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f409bfff700 1 --2- 172.21.15.121:0/683483048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f409c0ae190 0x7f409c0c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:30.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f40a0a8e700 1 -- 172.21.15.121:0/683483048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c0cabd0 msgr2=0x7f409c155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:30.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f40a0a8e700 1 --2- 172.21.15.121:0/683483048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c0cabd0 0x7f409c155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:30.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f40a0a8e700 1 -- 172.21.15.121:0/683483048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f409c0ae190 msgr2=0x7f409c0c08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:30.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f40a0a8e700 1 --2- 172.21.15.121:0/683483048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f409c0ae190 0x7f409c0c08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:30.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f40a0a8e700 1 -- 172.21.15.121:0/683483048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4094009a20 con 0x7f409c0cafb0 2023-11-15T03:10:30.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f40a0a8e700 1 --2- 172.21.15.121:0/683483048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f409c0cafb0 0x7f409c140490 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 rx=0x7f409400b6c0 tx=0x7f4094009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:30.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f4083fff700 1 -- 172.21.15.121:0/683483048 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409401d630 con 0x7f409c0cafb0 2023-11-15T03:10:30.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f4083fff700 1 -- 172.21.15.121:0/683483048 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409401d790 con 0x7f409c0cafb0 2023-11-15T03:10:30.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f40a128f700 1 -- 172.21.15.121:0/683483048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409c156120 con 0x7f409c0cafb0 2023-11-15T03:10:30.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f4083fff700 1 -- 172.21.15.121:0/683483048 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409401db30 con 0x7f409c0cafb0 2023-11-15T03:10:30.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.641+0000 7f40a128f700 1 -- 172.21.15.121:0/683483048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409c1566e0 con 0x7f409c0cafb0 2023-11-15T03:10:30.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.645+0000 7f40a128f700 1 -- 172.21.15.121:0/683483048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409c156bc0 con 0x7f409c0cafb0 2023-11-15T03:10:30.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.645+0000 7f4083fff700 1 -- 172.21.15.121:0/683483048 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4094003040 con 0x7f409c0cafb0 2023-11-15T03:10:30.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.645+0000 7f4083fff700 1 --2- 172.21.15.121:0/683483048 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f408805a510 0x7f408805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:30.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.649+0000 7f4083fff700 1 -- 172.21.15.121:0/683483048 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9362+0+0 (secure 0 0 0) 0x7f40940773f0 con 0x7f409c0cafb0 2023-11-15T03:10:30.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.649+0000 7f409bfff700 1 --2- 172.21.15.121:0/683483048 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f408805a510 0x7f408805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:30.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.649+0000 7f409bfff700 1 --2- 172.21.15.121:0/683483048 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f408805a510 0x7f408805c9d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f408c007fd0 tx=0x7f408c007bb0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:30.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.653+0000 7f4083fff700 1 -- 172.21.15.121:0/683483048 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4094049e60 con 0x7f409c0cafb0 2023-11-15T03:10:30.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.845+0000 7f40a128f700 1 -- 172.21.15.121:0/683483048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f409c0662a0 con 0x7f409c0cafb0 2023-11-15T03:10:30.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.849+0000 7f4083fff700 1 -- 172.21.15.121:0/683483048 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 319 v319) v1 ==== 95+0+4508 (secure 0 0 0) 0x7f409404d480 con 0x7f409c0cafb0 2023-11-15T03:10:30.855 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:10:30.855 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":319,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":319,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:10:27.230457+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10242":{"gid":10242,"name":"a","rank":0,"incarnation":316,"state":"up:active","state_seq":5,"addr":"172.21.15.121:6836/1750624080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1750624080},{"type":"v1","addr":"172.21.15.121:6836","nonce":1750624080}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:10:30.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.849+0000 7f40a128f700 1 -- 172.21.15.121:0/683483048 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f408805a510 msgr2=0x7f408805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:30.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.849+0000 7f40a128f700 1 --2- 172.21.15.121:0/683483048 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f408805a510 0x7f408805c9d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f408c007fd0 tx=0x7f408c007bb0).stop 2023-11-15T03:10:30.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.849+0000 7f40a128f700 1 -- 172.21.15.121:0/683483048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f409c0cafb0 msgr2=0x7f409c140490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:30.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.849+0000 7f40a128f700 1 --2- 172.21.15.121:0/683483048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f409c0cafb0 0x7f409c140490 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 rx=0x7f409400b6c0 tx=0x7f4094009780).stop 2023-11-15T03:10:30.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.849+0000 7f40a128f700 1 -- 172.21.15.121:0/683483048 shutdown_connections 2023-11-15T03:10:30.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.849+0000 7f40a128f700 1 --2- 172.21.15.121:0/683483048 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f408805a510 0x7f408805c9d0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:30.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.849+0000 7f40a128f700 1 --2- 172.21.15.121:0/683483048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f409c0cafb0 0x7f409c140490 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:30.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.849+0000 7f40a128f700 1 --2- 172.21.15.121:0/683483048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c0cabd0 0x7f409c155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:30.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.849+0000 7f40a128f700 1 --2- 172.21.15.121:0/683483048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f409c0ae190 0x7f409c0c08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:30.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.849+0000 7f40a128f700 1 -- 172.21.15.121:0/683483048 >> 172.21.15.121:0/683483048 conn(0x7f409c062190 msgr2=0x7f409c151700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:30.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.849+0000 7f40a128f700 1 -- 172.21.15.121:0/683483048 shutdown_connections 2023-11-15T03:10:30.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:30.849+0000 7f40a128f700 1 -- 172.21.15.121:0/683483048 wait complete. 2023-11-15T03:10:30.860 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 319 2023-11-15T03:10:30.877 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 319, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:03:21.784802+0000', 'modified': '2023-11-15T03:10:27.230457+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 130, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10242}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10242': {'gid': 10242, 'name': 'a', 'rank': 0, 'incarnation': 316, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.121:6836/1750624080', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 1750624080}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 1750624080}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:10:30.877 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T03:10:30.878 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-15T03:10:31.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.033+0000 7f1d69907700 1 Processor -- start 2023-11-15T03:10:31.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.033+0000 7f1d69907700 1 -- start start 2023-11-15T03:10:31.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.037+0000 7f1d69907700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1d640ab0d0 0x7f1d640ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.037+0000 7f1d69907700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1d640ad1b0 0x7f1d64065710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.037+0000 7f1d69907700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d64066c90 con 0x7f1d64065c00 2023-11-15T03:10:31.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.037+0000 7f1d69907700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d64066080 con 0x7f1d640ab0d0 2023-11-15T03:10:31.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.037+0000 7f1d69907700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d64066360 con 0x7f1d640ad1b0 2023-11-15T03:10:31.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.037+0000 7f1d68905700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1d640ab0d0 0x7f1d640ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.037+0000 7f1d68905700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1d640ab0d0 0x7f1d640ab4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35898/0 (socket says 172.21.15.121:35898) 2023-11-15T03:10:31.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.037+0000 7f1d68905700 1 -- 172.21.15.121:0/3527574407 learned_addr learned my addr 172.21.15.121:0/3527574407 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:31.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.037+0000 7f1d63fff700 1 --2- 172.21.15.121:0/3527574407 >> v2:172.21.15.190:3300/0 conn(0x7f1d640ad1b0 0x7f1d64065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.037+0000 7f1d68905700 1 -- 172.21.15.121:0/3527574407 >> v2:172.21.15.190:3300/0 conn(0x7f1d640ad1b0 msgr2=0x7f1d64065710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:31.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.037+0000 7f1d68905700 1 --2- 172.21.15.121:0/3527574407 >> v2:172.21.15.190:3300/0 conn(0x7f1d640ad1b0 0x7f1d64065710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.037+0000 7f1d68905700 1 -- 172.21.15.121:0/3527574407 >> v1:172.21.15.190:6789/0 conn(0x7f1d64065c00 legacy=0x7f1d640c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:31.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.037+0000 7f1d68905700 1 -- 172.21.15.121:0/3527574407 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d64066640 con 0x7f1d640ab0d0 2023-11-15T03:10:31.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.037+0000 7f1d63fff700 1 --2- 172.21.15.121:0/3527574407 >> v2:172.21.15.190:3300/0 conn(0x7f1d640ad1b0 0x7f1d64065710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:31.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.037+0000 7f1d68905700 1 --2- 172.21.15.121:0/3527574407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d640ab0d0 0x7f1d640ab4b0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 rx=0x7f1d6406a180 tx=0x7f1d58009060).ready entity=mon.0 client_cookie=a916385c3f461939 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:31.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.037+0000 7f1d637fe700 1 -- 172.21.15.121:0/3527574407 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58005880 con 0x7f1d640ab0d0 2023-11-15T03:10:31.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.037+0000 7f1d637fe700 1 -- 172.21.15.121:0/3527574407 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d580059e0 con 0x7f1d640ab0d0 2023-11-15T03:10:31.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.037+0000 7f1d637fe700 1 -- 172.21.15.121:0/3527574407 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58005d60 con 0x7f1d640ab0d0 2023-11-15T03:10:31.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.041+0000 7f1d69907700 1 -- 172.21.15.121:0/3527574407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d640ab0d0 msgr2=0x7f1d640ab4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:31.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.041+0000 7f1d69907700 1 --2- 172.21.15.121:0/3527574407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d640ab0d0 0x7f1d640ab4b0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 rx=0x7f1d6406a180 tx=0x7f1d58009060).stop 2023-11-15T03:10:31.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.041+0000 7f1d69907700 1 -- 172.21.15.121:0/3527574407 shutdown_connections 2023-11-15T03:10:31.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.041+0000 7f1d69907700 1 --2- 172.21.15.121:0/3527574407 >> v2:172.21.15.190:3300/0 conn(0x7f1d640ad1b0 0x7f1d64065710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.041+0000 7f1d69907700 1 --2- 172.21.15.121:0/3527574407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d640ab0d0 0x7f1d640ab4b0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.041+0000 7f1d69907700 1 -- 172.21.15.121:0/3527574407 >> 172.21.15.121:0/3527574407 conn(0x7f1d64067a60 msgr2=0x7f1d64067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:31.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.041+0000 7f1d69907700 1 -- 172.21.15.121:0/3527574407 shutdown_connections 2023-11-15T03:10:31.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.041+0000 7f1d69907700 1 -- 172.21.15.121:0/3527574407 wait complete. 2023-11-15T03:10:31.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.041+0000 7f1d69907700 1 Processor -- start 2023-11-15T03:10:31.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.041+0000 7f1d69907700 1 -- start start 2023-11-15T03:10:31.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.041+0000 7f1d69907700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d64065c00 0x7f1d64142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.041+0000 7f1d69907700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d640ab0d0 0x7f1d64167540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.041+0000 7f1d69907700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d640ad1b0 0x7f1d64167a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.041+0000 7f1d69907700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d64167f20 con 0x7f1d64065c00 2023-11-15T03:10:31.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.041+0000 7f1d69907700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d64165d50 con 0x7f1d640ad1b0 2023-11-15T03:10:31.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.041+0000 7f1d69907700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d64145c00 con 0x7f1d640ab0d0 2023-11-15T03:10:31.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.041+0000 7f1d68905700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d64065c00 0x7f1d64142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.045+0000 7f1d68905700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d64065c00 0x7f1d64142110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35902/0 (socket says 172.21.15.121:35902) 2023-11-15T03:10:31.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.045+0000 7f1d68905700 1 -- 172.21.15.121:0/169546032 learned_addr learned my addr 172.21.15.121:0/169546032 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:31.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.045+0000 7f1d63fff700 1 --2- 172.21.15.121:0/169546032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d640ab0d0 0x7f1d64167540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.045+0000 7f1d69106700 1 --2- 172.21.15.121:0/169546032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d640ad1b0 0x7f1d64167a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.045+0000 7f1d68905700 1 -- 172.21.15.121:0/169546032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d640ab0d0 msgr2=0x7f1d64167540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:31.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.045+0000 7f1d68905700 1 --2- 172.21.15.121:0/169546032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d640ab0d0 0x7f1d64167540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.045+0000 7f1d68905700 1 -- 172.21.15.121:0/169546032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d640ad1b0 msgr2=0x7f1d64167a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:31.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.045+0000 7f1d68905700 1 --2- 172.21.15.121:0/169546032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d640ad1b0 0x7f1d64167a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.045+0000 7f1d68905700 1 -- 172.21.15.121:0/169546032 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d58009300 con 0x7f1d64065c00 2023-11-15T03:10:31.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.045+0000 7f1d63fff700 1 --2- 172.21.15.121:0/169546032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d640ab0d0 0x7f1d64167540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:10:31.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.045+0000 7f1d69106700 1 --2- 172.21.15.121:0/169546032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d640ad1b0 0x7f1d64167a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:10:31.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.045+0000 7f1d68905700 1 --2- 172.21.15.121:0/169546032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d64065c00 0x7f1d64142110 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 rx=0x7f1d64065120 tx=0x7f1d58016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:31.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.045+0000 7f1d60ff9700 1 -- 172.21.15.121:0/169546032 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5800b810 con 0x7f1d64065c00 2023-11-15T03:10:31.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.045+0000 7f1d60ff9700 1 -- 172.21.15.121:0/169546032 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5800b970 con 0x7f1d64065c00 2023-11-15T03:10:31.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.045+0000 7f1d60ff9700 1 -- 172.21.15.121:0/169546032 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5800bd10 con 0x7f1d64065c00 2023-11-15T03:10:31.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.045+0000 7f1d69907700 1 -- 172.21.15.121:0/169546032 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d64165140 con 0x7f1d64065c00 2023-11-15T03:10:31.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.045+0000 7f1d69907700 1 -- 172.21.15.121:0/169546032 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d64165700 con 0x7f1d64065c00 2023-11-15T03:10:31.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.049+0000 7f1d60ff9700 1 -- 172.21.15.121:0/169546032 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1d5801b650 con 0x7f1d64065c00 2023-11-15T03:10:31.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.049+0000 7f1d60ff9700 1 --2- 172.21.15.121:0/169546032 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d4805a540 0x7f1d4805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.049+0000 7f1d60ff9700 1 -- 172.21.15.121:0/169546032 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9362+0+0 (secure 0 0 0) 0x7f1d58076760 con 0x7f1d64065c00 2023-11-15T03:10:31.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.049+0000 7f1d63fff700 1 --2- 172.21.15.121:0/169546032 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d4805a540 0x7f1d4805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d63fff700 1 --2- 172.21.15.121:0/169546032 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d4805a540 0x7f1d4805ca00 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f1d50006d80 tx=0x7f1d50008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:31.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d74f06740 1 Processor -- start 2023-11-15T03:10:31.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d74f06740 1 -- start start 2023-11-15T03:10:31.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d74f06740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x226bad0 0x2292090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d74f06740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226beb0 0x22b8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d74f06740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22b8630 0x22b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d74f06740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x21ec410 con 0x226bad0 2023-11-15T03:10:31.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d74f06740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x21ec550 con 0x226beb0 2023-11-15T03:10:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d74f06740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x21eca00 con 0x22b8630 2023-11-15T03:10:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d68905700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x226bad0 0x2292090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d68905700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x226bad0 0x2292090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35912/0 (socket says 172.21.15.121:35912) 2023-11-15T03:10:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d68905700 1 -- 172.21.15.121:0/3825974440 learned_addr learned my addr 172.21.15.121:0/3825974440 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:31.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d63fff700 1 --2- 172.21.15.121:0/3825974440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226beb0 0x22b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d69106700 1 --2- 172.21.15.121:0/3825974440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22b8630 0x22b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d68905700 1 -- 172.21.15.121:0/3825974440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22b8630 msgr2=0x22b8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:31.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d68905700 1 --2- 172.21.15.121:0/3825974440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22b8630 0x22b8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d68905700 1 -- 172.21.15.121:0/3825974440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226beb0 msgr2=0x22b8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:31.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d68905700 1 --2- 172.21.15.121:0/3825974440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226beb0 0x22b8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d68905700 1 -- 172.21.15.121:0/3825974440 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x21e5730 con 0x226bad0 2023-11-15T03:10:31.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d69106700 1 --2- 172.21.15.121:0/3825974440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22b8630 0x22b8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:10:31.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d68905700 1 --2- 172.21.15.121:0/3825974440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x226bad0 0x2292090 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 rx=0x7f1d58006e60 tx=0x7f1d5800e000).ready entity=mon.0 client_cookie=cb268d98739d5874 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:31.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d4dffb700 1 -- 172.21.15.121:0/3825974440 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58006a90 con 0x226bad0 2023-11-15T03:10:31.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d4dffb700 1 -- 172.21.15.121:0/3825974440 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5804cdd0 con 0x226bad0 2023-11-15T03:10:31.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d4dffb700 1 -- 172.21.15.121:0/3825974440 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58052570 con 0x226bad0 2023-11-15T03:10:31.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d74f06740 1 -- 172.21.15.121:0/3825974440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x226bad0 msgr2=0x2292090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:31.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.053+0000 7f1d74f06740 1 --2- 172.21.15.121:0/3825974440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x226bad0 0x2292090 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 rx=0x7f1d58006e60 tx=0x7f1d5800e000).stop 2023-11-15T03:10:31.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d74f06740 1 -- 172.21.15.121:0/3825974440 shutdown_connections 2023-11-15T03:10:31.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d74f06740 1 --2- 172.21.15.121:0/3825974440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22b8630 0x22b8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d74f06740 1 --2- 172.21.15.121:0/3825974440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226beb0 0x22b8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d74f06740 1 --2- 172.21.15.121:0/3825974440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x226bad0 0x2292090 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d74f06740 1 -- 172.21.15.121:0/3825974440 >> 172.21.15.121:0/3825974440 conn(0x204a550 msgr2=0x226a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:31.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d74f06740 1 -- 172.21.15.121:0/3825974440 shutdown_connections 2023-11-15T03:10:31.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d74f06740 1 -- 172.21.15.121:0/3825974440 wait complete. 2023-11-15T03:10:31.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d74f06740 1 -- start start 2023-11-15T03:10:31.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d74f06740 1 Processor -- start 2023-11-15T03:10:31.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d74f06740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x226bad0 0x2298770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d74f06740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226beb0 0x228e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d74f06740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22b8630 0x226c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d74f06740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x21b7650 con 0x226bad0 2023-11-15T03:10:31.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d74f06740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x22898f0 con 0x226beb0 2023-11-15T03:10:31.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d74f06740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2295fd0 con 0x22b8630 2023-11-15T03:10:31.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d74f06740 10 client.0 ll_register_callbacks cb 0x2224020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:10:31.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d68905700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x226bad0 0x2298770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d68905700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x226bad0 0x2298770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35924/0 (socket says 172.21.15.121:35924) 2023-11-15T03:10:31.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d68905700 1 -- 172.21.15.121:0/2532625941 learned_addr learned my addr 172.21.15.121:0/2532625941 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:31.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d69106700 1 --2- 172.21.15.121:0/2532625941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22b8630 0x226c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.057+0000 7f1d63fff700 1 --2- 172.21.15.121:0/2532625941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226beb0 0x228e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d68905700 1 -- 172.21.15.121:0/2532625941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22b8630 msgr2=0x226c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:31.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d68905700 1 --2- 172.21.15.121:0/2532625941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22b8630 0x226c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d68905700 1 -- 172.21.15.121:0/2532625941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226beb0 msgr2=0x228e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:31.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d68905700 1 --2- 172.21.15.121:0/2532625941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226beb0 0x228e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d68905700 1 -- 172.21.15.121:0/2532625941 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d64165700 con 0x226bad0 2023-11-15T03:10:31.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d68905700 1 --2- 172.21.15.121:0/2532625941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x226bad0 0x2298770 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 rx=0x7f1d5804cd10 tx=0x7f1d58030000).ready entity=mon.0 client_cookie=42e3120e07e1ad41 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:31.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:10:31.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d42ffd700 1 -- 172.21.15.121:0/2532625941 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5804b1d0 con 0x226bad0 2023-11-15T03:10:31.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d42ffd700 1 -- 172.21.15.121:0/2532625941 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 9362+0+0 (secure 0 0 0) 0x7f1d58049540 con 0x226bad0 2023-11-15T03:10:31.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d74f06740 1 -- 172.21.15.121:0/2532625941 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x21e21a0 con 0x226bad0 2023-11-15T03:10:31.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d42ffd700 1 -- 172.21.15.121:0/2532625941 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=319) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d58049700 con 0x226bad0 2023-11-15T03:10:31.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d74f06740 10 client.10323 fetch_fsmap learned FSMap version 319 2023-11-15T03:10:31.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d74f06740 1 -- 172.21.15.121:0/2532625941 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=319,monmap=2+}) v3 -- 0x21e0960 con 0x226bad0 2023-11-15T03:10:31.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d42ffd700 1 -- 172.21.15.121:0/2532625941 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x21e21a0 con 0x226bad0 2023-11-15T03:10:31.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d42ffd700 1 -- 172.21.15.121:0/2532625941 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 319) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1d5804c320 con 0x226bad0 2023-11-15T03:10:31.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d74f06740 10 client.10323 fetch_fsmap finished waiting for FSMap version 319 2023-11-15T03:10:31.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d74f06740 10 client.10323 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:10:31.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d74f06740 20 client.10323 populate_metadata read hostname 'smithi121' 2023-11-15T03:10:31.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d74f06740 1 --2- 172.21.15.121:0/2532625941 >> [v2:172.21.15.121:6834/1750624080,v1:172.21.15.121:6836/1750624080] conn(0x22ab390 0x22ab770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d74f06740 4 client.10323 mds_command: new command op to 10242 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:10:31.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d74f06740 1 -- 172.21.15.121:0/2532625941 --> [v2:172.21.15.121:6834/1750624080,v1:172.21.15.121:6836/1750624080] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x21d86f0 con 0x22ab390 2023-11-15T03:10:31.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d63fff700 1 --2- 172.21.15.121:0/2532625941 >> [v2:172.21.15.121:6834/1750624080,v1:172.21.15.121:6836/1750624080] conn(0x22ab390 0x22ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d63fff700 1 --2- 172.21.15.121:0/2532625941 >> [v2:172.21.15.121:6834/1750624080,v1:172.21.15.121:6836/1750624080] conn(0x22ab390 0x22ab770 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=26e0cf1046f1b5cd server_cookie=686f296f12577a74 in_seq=0 out_seq=0 2023-11-15T03:10:31.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.061+0000 7f1d42ffd700 10 client.10323 ms_handle_connect on v2:172.21.15.121:6834/1750624080 2023-11-15T03:10:31.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.065+0000 7f1d42ffd700 1 -- 172.21.15.121:0/2532625941 <== mds.0 v2:172.21.15.121:6834/1750624080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x21d86f0 con 0x22ab390 2023-11-15T03:10:31.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.065+0000 7f1d42ffd700 10 client.10323 handle_command_reply: tid=0 2023-11-15T03:10:31.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.065+0000 7f1d74f06740 1 client.10323 shutdown 2023-11-15T03:10:31.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.065+0000 7f1d74f06740 1 -- 172.21.15.121:0/2532625941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x226bad0 msgr2=0x2298770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:31.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.065+0000 7f1d74f06740 1 --2- 172.21.15.121:0/2532625941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x226bad0 0x2298770 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 rx=0x7f1d5804cd10 tx=0x7f1d58030000).stop 2023-11-15T03:10:31.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.065+0000 7f1d74f06740 1 -- 172.21.15.121:0/2532625941 shutdown_connections 2023-11-15T03:10:31.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.065+0000 7f1d74f06740 1 --2- 172.21.15.121:0/2532625941 >> [v2:172.21.15.121:6834/1750624080,v1:172.21.15.121:6836/1750624080] conn(0x22ab390 0x22ab770 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.065+0000 7f1d74f06740 1 --2- 172.21.15.121:0/2532625941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x22b8630 0x226c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.065+0000 7f1d74f06740 1 --2- 172.21.15.121:0/2532625941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226beb0 0x228e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.065+0000 7f1d74f06740 1 --2- 172.21.15.121:0/2532625941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x226bad0 0x2298770 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.065+0000 7f1d42ffd700 0 client.10323 ms_handle_reset on v2:172.21.15.121:6834/1750624080 2023-11-15T03:10:31.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.065+0000 7f1d74f06740 1 -- 172.21.15.121:0/2532625941 >> 172.21.15.121:0/2532625941 conn(0x204a550 msgr2=0x22b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:31.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.065+0000 7f1d74f06740 1 -- 172.21.15.121:0/2532625941 shutdown_connections 2023-11-15T03:10:31.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.065+0000 7f1d74f06740 1 -- 172.21.15.121:0/2532625941 wait complete. 2023-11-15T03:10:31.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.065+0000 7f1d74f06740 20 client.10323 trim_cache size 0 max 16384 2023-11-15T03:10:31.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.081+0000 7f1d74f06740 1 Processor -- start 2023-11-15T03:10:31.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.081+0000 7f1d74f06740 1 -- start start 2023-11-15T03:10:31.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d74f06740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226bad0 0x2296d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d74f06740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226beb0 0x2297240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d74f06740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22b8630 0x2268d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d74f06740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x226d600 con 0x22b8630 2023-11-15T03:10:31.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d74f06740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x21c1c60 con 0x226bad0 2023-11-15T03:10:31.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d74f06740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2297730 con 0x226beb0 2023-11-15T03:10:31.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d69106700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22b8630 0x2268d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d69106700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22b8630 0x2268d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35928/0 (socket says 172.21.15.121:35928) 2023-11-15T03:10:31.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d69106700 1 -- 172.21.15.121:0/1369221771 learned_addr learned my addr 172.21.15.121:0/1369221771 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:31.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d68905700 1 --2- 172.21.15.121:0/1369221771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226bad0 0x2296d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d63fff700 1 --2- 172.21.15.121:0/1369221771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226beb0 0x2297240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d69106700 1 -- 172.21.15.121:0/1369221771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226beb0 msgr2=0x2297240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:31.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d69106700 1 --2- 172.21.15.121:0/1369221771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226beb0 0x2297240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d69106700 1 -- 172.21.15.121:0/1369221771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226bad0 msgr2=0x2296d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:31.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d69106700 1 --2- 172.21.15.121:0/1369221771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226bad0 0x2296d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d69106700 1 -- 172.21.15.121:0/1369221771 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x226c9b0 con 0x22b8630 2023-11-15T03:10:31.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d69106700 1 --2- 172.21.15.121:0/1369221771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22b8630 0x2268d40 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 rx=0x7f1d640aa600 tx=0x7f1d5c009040).ready entity=mon.0 client_cookie=275996bb3807eeb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:31.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d4e7fc700 1 -- 172.21.15.121:0/1369221771 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c018070 con 0x22b8630 2023-11-15T03:10:31.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d4e7fc700 1 -- 172.21.15.121:0/1369221771 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5c00d040 con 0x22b8630 2023-11-15T03:10:31.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.085+0000 7f1d4e7fc700 1 -- 172.21.15.121:0/1369221771 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c00c560 con 0x22b8630 2023-11-15T03:10:31.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.089+0000 7f1d74f06740 1 -- 172.21.15.121:0/1369221771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22b8630 msgr2=0x2268d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:31.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.089+0000 7f1d74f06740 1 --2- 172.21.15.121:0/1369221771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22b8630 0x2268d40 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 rx=0x7f1d640aa600 tx=0x7f1d5c009040).stop 2023-11-15T03:10:31.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.089+0000 7f1d74f06740 1 -- 172.21.15.121:0/1369221771 shutdown_connections 2023-11-15T03:10:31.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.089+0000 7f1d74f06740 1 --2- 172.21.15.121:0/1369221771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22b8630 0x2268d40 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.089+0000 7f1d74f06740 1 --2- 172.21.15.121:0/1369221771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226beb0 0x2297240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.089+0000 7f1d74f06740 1 --2- 172.21.15.121:0/1369221771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226bad0 0x2296d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.089+0000 7f1d74f06740 1 -- 172.21.15.121:0/1369221771 >> 172.21.15.121:0/1369221771 conn(0x204a550 msgr2=0x2266c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:31.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.089+0000 7f1d74f06740 1 -- 172.21.15.121:0/1369221771 shutdown_connections 2023-11-15T03:10:31.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.089+0000 7f1d74f06740 1 -- 172.21.15.121:0/1369221771 wait complete. 2023-11-15T03:10:31.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.089+0000 7f1d74f06740 1 -- start start 2023-11-15T03:10:31.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.089+0000 7f1d74f06740 1 Processor -- start 2023-11-15T03:10:31.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.093+0000 7f1d74f06740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226bad0 0x2313aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.093+0000 7f1d74f06740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226beb0 0x22b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.093+0000 7f1d74f06740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22b8630 0x22b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.093+0000 7f1d74f06740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x22b6b40 con 0x22b8630 2023-11-15T03:10:31.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.093+0000 7f1d74f06740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x22b7850 con 0x226beb0 2023-11-15T03:10:31.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.093+0000 7f1d74f06740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x22b3fc0 con 0x226bad0 2023-11-15T03:10:31.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.093+0000 7f1d74f06740 10 client.0 ll_register_callbacks cb 0x7f1d5804cc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:10:31.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.093+0000 7f1d69106700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22b8630 0x22b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.093+0000 7f1d69106700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22b8630 0x22b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35930/0 (socket says 172.21.15.121:35930) 2023-11-15T03:10:31.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.093+0000 7f1d69106700 1 -- 172.21.15.121:0/4030517107 learned_addr learned my addr 172.21.15.121:0/4030517107 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:31.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.093+0000 7f1d68905700 1 --2- 172.21.15.121:0/4030517107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226bad0 0x2313aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.093+0000 7f1d63fff700 1 --2- 172.21.15.121:0/4030517107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226beb0 0x22b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.093+0000 7f1d69106700 1 -- 172.21.15.121:0/4030517107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226bad0 msgr2=0x2313aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:31.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.093+0000 7f1d69106700 1 --2- 172.21.15.121:0/4030517107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226bad0 0x2313aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.093+0000 7f1d69106700 1 -- 172.21.15.121:0/4030517107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226beb0 msgr2=0x22b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:31.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.093+0000 7f1d69106700 1 --2- 172.21.15.121:0/4030517107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226beb0 0x22b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.093+0000 7f1d69106700 1 -- 172.21.15.121:0/4030517107 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d64066c90 con 0x22b8630 2023-11-15T03:10:31.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.093+0000 7f1d69106700 1 --2- 172.21.15.121:0/4030517107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22b8630 0x22b5db0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 rx=0x7f1d5c015000 tx=0x7f1d5c009040).ready entity=mon.0 client_cookie=a95f8f48dbf48107 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:31.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.093+0000 7f1d42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:10:31.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d42ffd700 1 -- 172.21.15.121:0/4030517107 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c00c270 con 0x22b8630 2023-11-15T03:10:31.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 1 -- 172.21.15.121:0/4030517107 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d5804cdd0 con 0x22b8630 2023-11-15T03:10:31.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d42ffd700 1 -- 172.21.15.121:0/4030517107 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 9362+0+0 (secure 0 0 0) 0x7f1d5c00c020 con 0x22b8630 2023-11-15T03:10:31.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d42ffd700 1 -- 172.21.15.121:0/4030517107 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=319) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d5c000f60 con 0x22b8630 2023-11-15T03:10:31.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 10 client.10329 fetch_fsmap learned FSMap version 319 2023-11-15T03:10:31.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 1 -- 172.21.15.121:0/4030517107 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=319,monmap=2+}) v3 -- 0x7f1d64156330 con 0x22b8630 2023-11-15T03:10:31.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d42ffd700 1 -- 172.21.15.121:0/4030517107 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5804cdd0 con 0x22b8630 2023-11-15T03:10:31.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d42ffd700 1 -- 172.21.15.121:0/4030517107 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 319) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1d5c012040 con 0x22b8630 2023-11-15T03:10:31.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 10 client.10329 fetch_fsmap finished waiting for FSMap version 319 2023-11-15T03:10:31.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 10 client.10329 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:10:31.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 20 client.10329 populate_metadata read hostname 'smithi121' 2023-11-15T03:10:31.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 1 --2- 172.21.15.121:0/4030517107 >> [v2:172.21.15.121:6834/1750624080,v1:172.21.15.121:6836/1750624080] conn(0x22899f0 0x2289dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 4 client.10329 mds_command: new command op to 10242 tid=0[{"prefix": "status"}] 2023-11-15T03:10:31.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 1 -- 172.21.15.121:0/4030517107 --> [v2:172.21.15.121:6834/1750624080,v1:172.21.15.121:6836/1750624080] -- command(tid 0: {"prefix": "status"}) v1 -- 0x20a14e0 con 0x22899f0 2023-11-15T03:10:31.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d68905700 1 --2- 172.21.15.121:0/4030517107 >> [v2:172.21.15.121:6834/1750624080,v1:172.21.15.121:6836/1750624080] conn(0x22899f0 0x2289dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d42ffd700 10 client.10329 ms_handle_connect on v2:172.21.15.121:6834/1750624080 2023-11-15T03:10:31.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d68905700 1 --2- 172.21.15.121:0/4030517107 >> [v2:172.21.15.121:6834/1750624080,v1:172.21.15.121:6836/1750624080] conn(0x22899f0 0x2289dd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=645e4e16c539c533 server_cookie=351e1e3d442ef010 in_seq=0 out_seq=0 2023-11-15T03:10:31.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d42ffd700 1 -- 172.21.15.121:0/4030517107 <== mds.0 v2:172.21.15.121:6834/1750624080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x20a14e0 con 0x22899f0 2023-11-15T03:10:31.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d42ffd700 10 client.10329 handle_command_reply: tid=0 2023-11-15T03:10:31.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 1 client.10329 shutdown 2023-11-15T03:10:31.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 1 -- 172.21.15.121:0/4030517107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22b8630 msgr2=0x22b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:31.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 1 --2- 172.21.15.121:0/4030517107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22b8630 0x22b5db0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 rx=0x7f1d5c015000 tx=0x7f1d5c009040).stop 2023-11-15T03:10:31.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 1 -- 172.21.15.121:0/4030517107 shutdown_connections 2023-11-15T03:10:31.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 1 --2- 172.21.15.121:0/4030517107 >> [v2:172.21.15.121:6834/1750624080,v1:172.21.15.121:6836/1750624080] conn(0x22899f0 0x2289dd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 1 --2- 172.21.15.121:0/4030517107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x22b8630 0x22b5db0 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 1 --2- 172.21.15.121:0/4030517107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x226beb0 0x22b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d42ffd700 0 client.10329 ms_handle_reset on v2:172.21.15.121:6834/1750624080 2023-11-15T03:10:31.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 1 --2- 172.21.15.121:0/4030517107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x226bad0 0x2313aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 1 -- 172.21.15.121:0/4030517107 >> 172.21.15.121:0/4030517107 conn(0x204a550 msgr2=0x2266c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:31.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 1 -- 172.21.15.121:0/4030517107 shutdown_connections 2023-11-15T03:10:31.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.097+0000 7f1d74f06740 1 -- 172.21.15.121:0/4030517107 wait complete. 2023-11-15T03:10:31.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.101+0000 7f1d74f06740 20 client.10329 trim_cache size 0 max 16384 2023-11-15T03:10:31.118 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:10:31.118 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T03:10:31.118 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T03:10:31.119 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 10242, 2023-11-15T03:10:31.119 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T03:10:31.119 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T03:10:31.119 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T03:10:31.119 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 6.8928511190000004, 2023-11-15T03:10:31.120 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 319, 2023-11-15T03:10:31.120 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 130, 2023-11-15T03:10:31.120 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 130, 2023-11-15T03:10:31.120 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 13.822338225999999 2023-11-15T03:10:31.120 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:10:31.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.101+0000 7f1d69907700 1 -- 172.21.15.121:0/169546032 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d4805a540 msgr2=0x7f1d4805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:31.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.101+0000 7f1d69907700 1 --2- 172.21.15.121:0/169546032 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d4805a540 0x7f1d4805ca00 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f1d50006d80 tx=0x7f1d50008040).stop 2023-11-15T03:10:31.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.101+0000 7f1d69907700 1 -- 172.21.15.121:0/169546032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d64065c00 msgr2=0x7f1d64142110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:31.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.101+0000 7f1d69907700 1 --2- 172.21.15.121:0/169546032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d64065c00 0x7f1d64142110 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 rx=0x7f1d64065120 tx=0x7f1d58016000).stop 2023-11-15T03:10:31.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.101+0000 7f1d69907700 1 -- 172.21.15.121:0/169546032 shutdown_connections 2023-11-15T03:10:31.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.101+0000 7f1d69907700 1 --2- 172.21.15.121:0/169546032 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d4805a540 0x7f1d4805ca00 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.101+0000 7f1d69907700 1 --2- 172.21.15.121:0/169546032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d640ad1b0 0x7f1d64167a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.101+0000 7f1d69907700 1 --2- 172.21.15.121:0/169546032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d640ab0d0 0x7f1d64167540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.101+0000 7f1d69907700 1 --2- 172.21.15.121:0/169546032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d64065c00 0x7f1d64142110 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.101+0000 7f1d69907700 1 -- 172.21.15.121:0/169546032 >> 172.21.15.121:0/169546032 conn(0x7f1d64067a60 msgr2=0x7f1d64154e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:31.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.101+0000 7f1d69907700 1 -- 172.21.15.121:0/169546032 shutdown_connections 2023-11-15T03:10:31.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.101+0000 7f1d69907700 1 -- 172.21.15.121:0/169546032 wait complete. 2023-11-15T03:10:31.130 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:10:31.131 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap header on mds0_openfiles.0', proceeding to ls 2023-11-15T03:10:31.131 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T03:10:31.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.261+0000 7f8683f46700 1 Processor -- start 2023-11-15T03:10:31.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.261+0000 7f8683f46700 1 -- start start 2023-11-15T03:10:31.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8683f46700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f867c0c7520 0x7f867c0c1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8683f46700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f867c0aa980 0x7f867c0aadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8683f46700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867c0c0f70 con 0x7f867c0c7900 2023-11-15T03:10:31.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8683f46700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867c0c5db0 con 0x7f867c0c7520 2023-11-15T03:10:31.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8683f46700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867c0c6090 con 0x7f867c0aa980 2023-11-15T03:10:31.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8681ce2700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f867c0c7520 0x7f867c0c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8681ce2700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f867c0c7520 0x7f867c0c1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35938/0 (socket says 172.21.15.121:35938) 2023-11-15T03:10:31.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8681ce2700 1 -- 172.21.15.121:0/2381557962 learned_addr learned my addr 172.21.15.121:0/2381557962 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:31.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f86814e1700 1 --1- 172.21.15.121:0/2381557962 >> v1:172.21.15.121:6789/0 conn(0x7f867c0c7900 0x7f867c0c7d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:46234/0 (socket says 172.21.15.121:46234) 2023-11-15T03:10:31.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f86824e3700 1 --2- 172.21.15.121:0/2381557962 >> v2:172.21.15.190:3300/0 conn(0x7f867c0aa980 0x7f867c0aadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8681ce2700 1 -- 172.21.15.121:0/2381557962 >> v2:172.21.15.190:3300/0 conn(0x7f867c0aa980 msgr2=0x7f867c0aadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:31.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8681ce2700 1 --2- 172.21.15.121:0/2381557962 >> v2:172.21.15.190:3300/0 conn(0x7f867c0aa980 0x7f867c0aadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8681ce2700 1 -- 172.21.15.121:0/2381557962 >> v1:172.21.15.121:6789/0 conn(0x7f867c0c7900 legacy=0x7f867c0c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:31.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8681ce2700 1 -- 172.21.15.121:0/2381557962 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c0c0c00 con 0x7f867c0c7520 2023-11-15T03:10:31.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f86824e3700 1 --2- 172.21.15.121:0/2381557962 >> v2:172.21.15.190:3300/0 conn(0x7f867c0aa980 0x7f867c0aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:10:31.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8681ce2700 1 --2- 172.21.15.121:0/2381557962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c0c7520 0x7f867c0c1400 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 rx=0x7f867c084220 tx=0x7f86780090f0).ready entity=mon.0 client_cookie=598a58c3ad6b571a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:31.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8680ce0700 1 -- 172.21.15.121:0/2381557962 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867800b600 con 0x7f867c0c7520 2023-11-15T03:10:31.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8680ce0700 1 -- 172.21.15.121:0/2381557962 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867800b760 con 0x7f867c0c7520 2023-11-15T03:10:31.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8680ce0700 1 -- 172.21.15.121:0/2381557962 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867800bae0 con 0x7f867c0c7520 2023-11-15T03:10:31.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8683f46700 1 -- 172.21.15.121:0/2381557962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c0c7520 msgr2=0x7f867c0c1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:31.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8683f46700 1 --2- 172.21.15.121:0/2381557962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c0c7520 0x7f867c0c1400 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 rx=0x7f867c084220 tx=0x7f86780090f0).stop 2023-11-15T03:10:31.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8683f46700 1 -- 172.21.15.121:0/2381557962 shutdown_connections 2023-11-15T03:10:31.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8683f46700 1 --2- 172.21.15.121:0/2381557962 >> v2:172.21.15.190:3300/0 conn(0x7f867c0aa980 0x7f867c0aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8683f46700 1 --2- 172.21.15.121:0/2381557962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c0c7520 0x7f867c0c1400 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.265+0000 7f8683f46700 1 -- 172.21.15.121:0/2381557962 >> 172.21.15.121:0/2381557962 conn(0x7f867c067780 msgr2=0x7f867c0681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:31.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.269+0000 7f8683f46700 1 -- 172.21.15.121:0/2381557962 shutdown_connections 2023-11-15T03:10:31.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.269+0000 7f8683f46700 1 -- 172.21.15.121:0/2381557962 wait complete. 2023-11-15T03:10:31.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.269+0000 7f8683f46700 1 Processor -- start 2023-11-15T03:10:31.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.269+0000 7f8683f46700 1 -- start start 2023-11-15T03:10:31.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.269+0000 7f8683f46700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f867c0aa980 0x7f867c145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.269+0000 7f8683f46700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f867c0c7520 0x7f867c154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.269+0000 7f8683f46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c0c7900 0x7f867c154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.269+0000 7f8683f46700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c155e90 con 0x7f867c0c7900 2023-11-15T03:10:31.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.269+0000 7f8683f46700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c154db0 con 0x7f867c0c7520 2023-11-15T03:10:31.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.269+0000 7f8683f46700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867c155090 con 0x7f867c0aa980 2023-11-15T03:10:31.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.269+0000 7f86824e3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c0c7900 0x7f867c154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.269+0000 7f86824e3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c0c7900 0x7f867c154820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35952/0 (socket says 172.21.15.121:35952) 2023-11-15T03:10:31.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.269+0000 7f86824e3700 1 -- 172.21.15.121:0/3585277148 learned_addr learned my addr 172.21.15.121:0/3585277148 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:31.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.269+0000 7f86814e1700 1 --2- 172.21.15.121:0/3585277148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f867c0c7520 0x7f867c154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.269+0000 7f8681ce2700 1 --2- 172.21.15.121:0/3585277148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f867c0aa980 0x7f867c145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.273+0000 7f86824e3700 1 -- 172.21.15.121:0/3585277148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f867c0aa980 msgr2=0x7f867c145630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:31.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.273+0000 7f86824e3700 1 --2- 172.21.15.121:0/3585277148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f867c0aa980 0x7f867c145630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.273+0000 7f86824e3700 1 -- 172.21.15.121:0/3585277148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f867c0c7520 msgr2=0x7f867c154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:31.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.273+0000 7f86824e3700 1 --2- 172.21.15.121:0/3585277148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f867c0c7520 0x7f867c154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.273+0000 7f86824e3700 1 -- 172.21.15.121:0/3585277148 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8678009370 con 0x7f867c0c7900 2023-11-15T03:10:31.285 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.273+0000 7f86824e3700 1 --2- 172.21.15.121:0/3585277148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c0c7900 0x7f867c154820 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 rx=0x7f867400cef0 tx=0x7f8674007860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:31.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.273+0000 7f8671ffb700 1 -- 172.21.15.121:0/3585277148 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867400ad20 con 0x7f867c0c7900 2023-11-15T03:10:31.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.273+0000 7f8671ffb700 1 -- 172.21.15.121:0/3585277148 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867400ae80 con 0x7f867c0c7900 2023-11-15T03:10:31.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.273+0000 7f8671ffb700 1 -- 172.21.15.121:0/3585277148 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674016610 con 0x7f867c0c7900 2023-11-15T03:10:31.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.273+0000 7f8683f46700 1 -- 172.21.15.121:0/3585277148 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867c155370 con 0x7f867c0c7900 2023-11-15T03:10:31.286 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.273+0000 7f8683f46700 1 -- 172.21.15.121:0/3585277148 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867c152c70 con 0x7f867c0c7900 2023-11-15T03:10:31.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.273+0000 7f8683f46700 1 -- 172.21.15.121:0/3585277148 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867c156040 con 0x7f867c0c7900 2023-11-15T03:10:31.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.277+0000 7f8671ffb700 1 -- 172.21.15.121:0/3585277148 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f86740203f0 con 0x7f867c0c7900 2023-11-15T03:10:31.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.277+0000 7f8671ffb700 1 --2- 172.21.15.121:0/3585277148 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f866005a510 0x7f866005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:31.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.277+0000 7f8671ffb700 1 -- 172.21.15.121:0/3585277148 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9362+0+0 (secure 0 0 0) 0x7f8674075e10 con 0x7f867c0c7900 2023-11-15T03:10:31.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.277+0000 7f8681ce2700 1 --2- 172.21.15.121:0/3585277148 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f866005a510 0x7f866005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:31.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.277+0000 7f8681ce2700 1 --2- 172.21.15.121:0/3585277148 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f866005a510 0x7f866005c9d0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f8678009fd0 tx=0x7f8678017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:31.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.285+0000 7f8671ffb700 1 -- 172.21.15.121:0/3585277148 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f867c156040 con 0x7f867c0c7900 2023-11-15T03:10:31.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.485+0000 7f8683f46700 1 -- 172.21.15.121:0/3585277148 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f867c155fd0 con 0x7f867c0c7900 2023-11-15T03:10:31.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.485+0000 7f8671ffb700 1 -- 172.21.15.121:0/3585277148 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v319) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8674077030 con 0x7f867c0c7900 2023-11-15T03:10:31.492 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T03:10:31.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.489+0000 7f8683f46700 1 -- 172.21.15.121:0/3585277148 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f866005a510 msgr2=0x7f866005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:31.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.489+0000 7f8683f46700 1 --2- 172.21.15.121:0/3585277148 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f866005a510 0x7f866005c9d0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f8678009fd0 tx=0x7f8678017000).stop 2023-11-15T03:10:31.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.489+0000 7f8683f46700 1 -- 172.21.15.121:0/3585277148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c0c7900 msgr2=0x7f867c154820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:31.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.489+0000 7f8683f46700 1 --2- 172.21.15.121:0/3585277148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c0c7900 0x7f867c154820 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 rx=0x7f867400cef0 tx=0x7f8674007860).stop 2023-11-15T03:10:31.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.489+0000 7f8683f46700 1 -- 172.21.15.121:0/3585277148 shutdown_connections 2023-11-15T03:10:31.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.489+0000 7f8683f46700 1 --2- 172.21.15.121:0/3585277148 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f866005a510 0x7f866005c9d0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.489+0000 7f8683f46700 1 --2- 172.21.15.121:0/3585277148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c0c7900 0x7f867c154820 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.489+0000 7f8683f46700 1 --2- 172.21.15.121:0/3585277148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f867c0c7520 0x7f867c154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.489+0000 7f8683f46700 1 --2- 172.21.15.121:0/3585277148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f867c0aa980 0x7f867c145630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:31.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.489+0000 7f8683f46700 1 -- 172.21.15.121:0/3585277148 >> 172.21.15.121:0/3585277148 conn(0x7f867c067780 msgr2=0x7f867c0c0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:31.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.489+0000 7f8683f46700 1 -- 172.21.15.121:0/3585277148 shutdown_connections 2023-11-15T03:10:31.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:31.489+0000 7f8683f46700 1 -- 172.21.15.121:0/3585277148 wait complete. 2023-11-15T03:10:31.511 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T03:10:31.511 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T03:10:31.512 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T03:10:31.512 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T03:10:31.513 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T03:10:31.513 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T03:10:31.513 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T03:10:31.514 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T03:10:31.514 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:10:31.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:31.515 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T03:10:31.535 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T03:10:31.535 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T03:10:31.536 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T03:10:31.536 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:10:31.536 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T03:10:31.537 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:10:31.537 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:10:31.537 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:10:31.537 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T03:10:31.537 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:10:31.538 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:10:31.538 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T03:10:31.538 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:10:31.538 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:10:31.538 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T03:10:31.539 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:10:31.540 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T03:10:31.540 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 41608sec preferred_lft 41608sec 2023-11-15T03:10:31.540 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T03:10:31.541 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:10:31.541 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T03:10:31.542 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:10:31.543 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T03:10:31.543 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:10:31.543 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T03:10:31.543 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T03:10:31.544 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T03:10:31.544 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:10:31.544 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T03:10:31.544 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:10:31.544 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:10:31.545 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T03:10:31.545 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T03:10:31.545 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:10:31.545 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:10:31.547 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T03:10:31.547 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T03:10:31.547 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:10:31.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:31.549 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T03:10:31.625 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T03:10:31.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:31.628 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T03:10:31.694 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T03:10:31.694 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T03:10:31.695 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T03:10:31.695 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T03:10:31.695 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T03:10:31.695 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T03:10:31.696 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T03:10:31.696 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T03:10:31.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:31.697 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T03:10:31.697 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T03:10:31.697 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T03:10:31.698 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T03:10:31.698 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T03:10:31.699 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T03:10:31.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:31.829 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T03:10:31.848 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T03:10:31.848 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T03:10:31.849 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:10:31.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:31.850 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T03:10:31.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:31.927 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T03:10:31.992 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T03:10:31.992 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:10:32.119 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:10:32.120 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:10:32.144 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:10:32.144 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:10:32.149 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T03:10:32.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:32.149 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T03:10:32.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:10:32.182 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2023-11-15T03:10:32.229 INFO:teuthology.orchestra.run:waiting for 20 2023-11-15T03:10:32.247 INFO:teuthology.orchestra.run.smithi121.stdout:.: 2023-11-15T03:10:32.248 INFO:teuthology.orchestra.run.smithi121.stdout:subdir 2023-11-15T03:10:32.248 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:10:32.248 INFO:teuthology.orchestra.run.smithi121.stdout:./subdir: 2023-11-15T03:10:32.248 INFO:teuthology.orchestra.run.smithi121.stdout:sixmegs 2023-11-15T03:10:32.249 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on mds0_openfiles.0' did not caused DAMAGED state 2023-11-15T03:10:32.250 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 100.00000000' 2023-11-15T03:10:32.250 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T03:10:32.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:10:32.251 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-15T03:10:32.332 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:10:32.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:32.333 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:10:32.399 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:10:32.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.597+0000 7fa9ddb07700 1 Processor -- start 2023-11-15T03:10:32.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.601+0000 7fa9ddb07700 1 -- start start 2023-11-15T03:10:32.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.601+0000 7fa9ddb07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa9d8069180 0x7fa9d8066f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:32.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.601+0000 7fa9ddb07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9d814e3d0 0x7fa9d8154160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:32.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.601+0000 7fa9ddb07700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d8062e30 con 0x7fa9d814dff0 2023-11-15T03:10:32.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.601+0000 7fa9ddb07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d8062ca0 con 0x7fa9d8069180 2023-11-15T03:10:32.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.601+0000 7fa9ddb07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d80c2db0 con 0x7fa9d814e3d0 2023-11-15T03:10:32.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.601+0000 7fa9dd306700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9d814e3d0 0x7fa9d8154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:32.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.601+0000 7fa9dcb05700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fa9d814dff0 0x7fa9d8068a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:54966/0 (socket says 172.21.15.121:54966) 2023-11-15T03:10:32.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.601+0000 7fa9dcb05700 1 -- 172.21.15.121:0/3013275140 learned_addr learned my addr 172.21.15.121:0/3013275140 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:32.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.601+0000 7fa9d7fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa9d8069180 0x7fa9d8066f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:32.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.601+0000 7fa9d7fff700 1 -- 172.21.15.121:0/3013275140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9d814e3d0 msgr2=0x7fa9d8154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:32.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.601+0000 7fa9d7fff700 1 --2- 172.21.15.121:0/3013275140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9d814e3d0 0x7fa9d8154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:32.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.601+0000 7fa9d77fe700 1 -- 172.21.15.121:0/3013275140 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596490724 0 0) 0x7fa9d8062e30 con 0x7fa9d814dff0 2023-11-15T03:10:32.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.601+0000 7fa9d7fff700 1 -- 172.21.15.121:0/3013275140 >> v1:172.21.15.190:6789/0 conn(0x7fa9d814dff0 legacy=0x7fa9d8068a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:32.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.601+0000 7fa9d7fff700 1 -- 172.21.15.121:0/3013275140 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d8154790 con 0x7fa9d8069180 2023-11-15T03:10:32.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.601+0000 7fa9dd306700 1 --2- 172.21.15.121:0/3013275140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9d814e3d0 0x7fa9d8154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:10:32.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.605+0000 7fa9d7fff700 1 --2- 172.21.15.121:0/3013275140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9d8069180 0x7fa9d8066f00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fa9c4009760 tx=0x7fa9c4009060).ready entity=mon.1 client_cookie=e84fd7ac02751c26 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:32.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.605+0000 7fa9d77fe700 1 -- 172.21.15.121:0/3013275140 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c400f510 con 0x7fa9d8069180 2023-11-15T03:10:32.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.605+0000 7fa9d77fe700 1 -- 172.21.15.121:0/3013275140 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c400f670 con 0x7fa9d8069180 2023-11-15T03:10:32.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.605+0000 7fa9d77fe700 1 -- 172.21.15.121:0/3013275140 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c400f9f0 con 0x7fa9d8069180 2023-11-15T03:10:32.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.605+0000 7fa9ddb07700 1 -- 172.21.15.121:0/3013275140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9d8069180 msgr2=0x7fa9d8066f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:32.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.605+0000 7fa9ddb07700 1 --2- 172.21.15.121:0/3013275140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9d8069180 0x7fa9d8066f00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fa9c4009760 tx=0x7fa9c4009060).stop 2023-11-15T03:10:32.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.605+0000 7fa9ddb07700 1 -- 172.21.15.121:0/3013275140 shutdown_connections 2023-11-15T03:10:32.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.605+0000 7fa9ddb07700 1 --2- 172.21.15.121:0/3013275140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9d814e3d0 0x7fa9d8154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:32.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.605+0000 7fa9ddb07700 1 --2- 172.21.15.121:0/3013275140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9d8069180 0x7fa9d8066f00 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:32.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.605+0000 7fa9ddb07700 1 -- 172.21.15.121:0/3013275140 >> 172.21.15.121:0/3013275140 conn(0x7fa9d8065fa0 msgr2=0x7fa9d80663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:32.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.605+0000 7fa9ddb07700 1 -- 172.21.15.121:0/3013275140 shutdown_connections 2023-11-15T03:10:32.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.605+0000 7fa9ddb07700 1 -- 172.21.15.121:0/3013275140 wait complete. 2023-11-15T03:10:32.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.609+0000 7fa9ddb07700 1 Processor -- start 2023-11-15T03:10:32.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.609+0000 7fa9ddb07700 1 -- start start 2023-11-15T03:10:32.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9ddb07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9d8069180 0x7fa9d8152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:32.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9ddb07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9d814dff0 0x7fa9d814a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:32.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9ddb07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9d814e3d0 0x7fa9d814aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:32.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9ddb07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d80ab7c0 con 0x7fa9d814dff0 2023-11-15T03:10:32.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9ddb07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d8164710 con 0x7fa9d8069180 2023-11-15T03:10:32.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9ddb07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d80aac40 con 0x7fa9d814e3d0 2023-11-15T03:10:32.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9d7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9d814dff0 0x7fa9d814a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:32.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9d7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9d814dff0 0x7fa9d814a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35962/0 (socket says 172.21.15.121:35962) 2023-11-15T03:10:32.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9d7fff700 1 -- 172.21.15.121:0/952429074 learned_addr learned my addr 172.21.15.121:0/952429074 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:32.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9dd306700 1 --2- 172.21.15.121:0/952429074 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9d814e3d0 0x7fa9d814aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:32.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9dcb05700 1 --2- 172.21.15.121:0/952429074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9d8069180 0x7fa9d8152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:32.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9d7fff700 1 -- 172.21.15.121:0/952429074 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9d814e3d0 msgr2=0x7fa9d814aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:32.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9d7fff700 1 --2- 172.21.15.121:0/952429074 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9d814e3d0 0x7fa9d814aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:32.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9d7fff700 1 -- 172.21.15.121:0/952429074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9d8069180 msgr2=0x7fa9d8152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:32.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9d7fff700 1 --2- 172.21.15.121:0/952429074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9d8069180 0x7fa9d8152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:32.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9d7fff700 1 -- 172.21.15.121:0/952429074 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d00092a0 con 0x7fa9d814dff0 2023-11-15T03:10:32.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9d7fff700 1 --2- 172.21.15.121:0/952429074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9d814dff0 0x7fa9d814a510 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 rx=0x7fa9c4011000 tx=0x7fa9c4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:32.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9d4ff9700 1 -- 172.21.15.121:0/952429074 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c400f7e0 con 0x7fa9d814dff0 2023-11-15T03:10:32.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9d4ff9700 1 -- 172.21.15.121:0/952429074 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c400f940 con 0x7fa9d814dff0 2023-11-15T03:10:32.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9ddb07700 1 -- 172.21.15.121:0/952429074 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c40092a0 con 0x7fa9d814dff0 2023-11-15T03:10:32.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9d4ff9700 1 -- 172.21.15.121:0/952429074 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c401b690 con 0x7fa9d814dff0 2023-11-15T03:10:32.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9ddb07700 1 -- 172.21.15.121:0/952429074 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d80ab200 con 0x7fa9d814dff0 2023-11-15T03:10:32.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.613+0000 7fa9ddb07700 1 -- 172.21.15.121:0/952429074 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d806bcf0 con 0x7fa9d814dff0 2023-11-15T03:10:32.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.617+0000 7fa9d4ff9700 1 -- 172.21.15.121:0/952429074 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa9c401ba80 con 0x7fa9d814dff0 2023-11-15T03:10:32.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.617+0000 7fa9d4ff9700 1 --2- 172.21.15.121:0/952429074 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa9bc05a510 0x7fa9bc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:32.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.617+0000 7fa9d4ff9700 1 -- 172.21.15.121:0/952429074 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9362+0+0 (secure 0 0 0) 0x7fa9c4076d30 con 0x7fa9d814dff0 2023-11-15T03:10:32.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.617+0000 7fa9dcb05700 1 --2- 172.21.15.121:0/952429074 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa9bc05a510 0x7fa9bc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:32.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.617+0000 7fa9dcb05700 1 --2- 172.21.15.121:0/952429074 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa9bc05a510 0x7fa9bc05c9d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7fa9cc001560 tx=0x7fa9cc009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:32.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.621+0000 7fa9d4ff9700 1 -- 172.21.15.121:0/952429074 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa9c40497a0 con 0x7fa9d814dff0 2023-11-15T03:10:32.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:32.817+0000 7fa9ddb07700 1 -- 172.21.15.121:0/952429074 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa9d8066ce0 con 0x7fa9d814dff0 2023-11-15T03:10:33.273 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:10:33.265+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:10:33.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.285+0000 7fa9d4ff9700 1 -- 172.21.15.121:0/952429074 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v320) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa9c404cdc0 con 0x7fa9d814dff0 2023-11-15T03:10:33.293 INFO:tasks.ceph.mds.a.smithi121.stderr: -1491> 2023-11-15T03:10:24.213+0000 7f8855f9b700 -1 mds.0.openfiles _load_finish: corrupted header/values: End of buffer 2023-11-15T03:10:33.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.289+0000 7fa9ddb07700 1 -- 172.21.15.121:0/952429074 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa9bc05a510 msgr2=0x7fa9bc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:33.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.289+0000 7fa9ddb07700 1 --2- 172.21.15.121:0/952429074 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa9bc05a510 0x7fa9bc05c9d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7fa9cc001560 tx=0x7fa9cc009040).stop 2023-11-15T03:10:33.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.289+0000 7fa9ddb07700 1 -- 172.21.15.121:0/952429074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9d814dff0 msgr2=0x7fa9d814a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:33.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.289+0000 7fa9ddb07700 1 --2- 172.21.15.121:0/952429074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9d814dff0 0x7fa9d814a510 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 rx=0x7fa9c4011000 tx=0x7fa9c4017000).stop 2023-11-15T03:10:33.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.289+0000 7fa9ddb07700 1 -- 172.21.15.121:0/952429074 shutdown_connections 2023-11-15T03:10:33.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.289+0000 7fa9ddb07700 1 --2- 172.21.15.121:0/952429074 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa9bc05a510 0x7fa9bc05c9d0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:33.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.289+0000 7fa9ddb07700 1 --2- 172.21.15.121:0/952429074 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9d814e3d0 0x7fa9d814aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:33.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.289+0000 7fa9ddb07700 1 --2- 172.21.15.121:0/952429074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9d814dff0 0x7fa9d814a510 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:33.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.289+0000 7fa9ddb07700 1 --2- 172.21.15.121:0/952429074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9d8069180 0x7fa9d8152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:33.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.289+0000 7fa9ddb07700 1 -- 172.21.15.121:0/952429074 >> 172.21.15.121:0/952429074 conn(0x7fa9d8065fa0 msgr2=0x7fa9d8153930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:33.298 INFO:tasks.ceph.mds.a.smithi121.stderr: -480> 2023-11-15T03:10:28.265+0000 7f88597a2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:10:33.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.289+0000 7fa9ddb07700 1 -- 172.21.15.121:0/952429074 shutdown_connections 2023-11-15T03:10:33.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.289+0000 7fa9ddb07700 1 -- 172.21.15.121:0/952429074 wait complete. 2023-11-15T03:10:33.300 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:10:33.318 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:10:33.341 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:10:33.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.473+0000 7fed073db700 1 Processor -- start 2023-11-15T03:10:33.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.473+0000 7fed073db700 1 -- start start 2023-11-15T03:10:33.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.477+0000 7fed073db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fed00153d80 0x7fed000c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:33.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.477+0000 7fed073db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed000c8fb0 0x7fed000c9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:33.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.477+0000 7fed073db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed000c5720 con 0x7fed000c04d0 2023-11-15T03:10:33.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.477+0000 7fed073db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed000c0da0 con 0x7fed00153d80 2023-11-15T03:10:33.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.477+0000 7fed073db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed000ca8a0 con 0x7fed000c8fb0 2023-11-15T03:10:33.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.477+0000 7fed05177700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fed000c04d0 0x7fed00153670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:46256/0 (socket says 172.21.15.121:46256) 2023-11-15T03:10:33.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.477+0000 7fed05177700 1 -- 172.21.15.121:0/3072071725 learned_addr learned my addr 172.21.15.121:0/3072071725 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:33.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.477+0000 7fed05978700 1 --2- 172.21.15.121:0/3072071725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed000c8fb0 0x7fed000c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:33.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.477+0000 7fed04976700 1 --2- 172.21.15.121:0/3072071725 >> v2:172.21.15.190:3300/0 conn(0x7fed00153d80 0x7fed000c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:33.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.477+0000 7fecf7fff700 1 -- 172.21.15.121:0/3072071725 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 343157876 0 0) 0x7fed000c5720 con 0x7fed000c04d0 2023-11-15T03:10:33.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.477+0000 7fecf7fff700 1 -- 172.21.15.121:0/3072071725 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecec0033f0 con 0x7fed000c04d0 2023-11-15T03:10:33.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.477+0000 7fecf7fff700 1 -- 172.21.15.121:0/3072071725 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2377691970 0 0) 0x7fecec0033f0 con 0x7fed000c04d0 2023-11-15T03:10:33.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.477+0000 7fecf7fff700 1 -- 172.21.15.121:0/3072071725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed000c8fb0 msgr2=0x7fed000c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:33.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.477+0000 7fecf7fff700 1 --2- 172.21.15.121:0/3072071725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed000c8fb0 0x7fed000c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:33.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.477+0000 7fecf7fff700 1 -- 172.21.15.121:0/3072071725 >> v2:172.21.15.190:3300/0 conn(0x7fed00153d80 msgr2=0x7fed000c08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:33.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.477+0000 7fecf7fff700 1 --2- 172.21.15.121:0/3072071725 >> v2:172.21.15.190:3300/0 conn(0x7fed00153d80 0x7fed000c08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:33.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.477+0000 7fecf7fff700 1 -- 172.21.15.121:0/3072071725 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed000c9d20 con 0x7fed000c04d0 2023-11-15T03:10:33.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fecf7fff700 1 -- 172.21.15.121:0/3072071725 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fecfc001470 con 0x7fed000c04d0 2023-11-15T03:10:33.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fed04976700 1 --2- 172.21.15.121:0/3072071725 >> v2:172.21.15.190:3300/0 conn(0x7fed00153d80 0x7fed000c08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:10:33.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fecf7fff700 1 -- 172.21.15.121:0/3072071725 >> v1:172.21.15.121:6789/0 conn(0x7fed000c04d0 legacy=0x7fed00153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:33.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fecf7fff700 1 --2- 172.21.15.121:0/3072071725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fecec0038e0 0x7fecec003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:33.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fecf7fff700 1 --2- 172.21.15.121:0/3072071725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecec004690 0x7fecec004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:33.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fecf7fff700 1 --2- 172.21.15.121:0/3072071725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed000c8fb0 0x7fecec005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:33.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fecf7fff700 1 -- 172.21.15.121:0/3072071725 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed000c0da0 con 0x7fecec0038e0 2023-11-15T03:10:33.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fecf7fff700 1 -- 172.21.15.121:0/3072071725 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed000ca8a0 con 0x7fecec004690 2023-11-15T03:10:33.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fecf7fff700 1 -- 172.21.15.121:0/3072071725 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec00b8e0 con 0x7fed000c8fb0 2023-11-15T03:10:33.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fed05177700 1 --2- 172.21.15.121:0/3072071725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fecec0038e0 0x7fecec003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:33.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fed05978700 1 --2- 172.21.15.121:0/3072071725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed000c8fb0 0x7fecec005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:33.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fed04976700 1 --2- 172.21.15.121:0/3072071725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecec004690 0x7fecec004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:33.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fed05978700 1 -- 172.21.15.121:0/3072071725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecec004690 msgr2=0x7fecec004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:33.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fed05978700 1 --2- 172.21.15.121:0/3072071725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecec004690 0x7fecec004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:33.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fed05978700 1 -- 172.21.15.121:0/3072071725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fecec0038e0 msgr2=0x7fecec003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:33.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fed05978700 1 --2- 172.21.15.121:0/3072071725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fecec0038e0 0x7fecec003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:33.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fed05978700 1 -- 172.21.15.121:0/3072071725 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec0033f0 con 0x7fed000c8fb0 2023-11-15T03:10:33.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fed05978700 1 --2- 172.21.15.121:0/3072071725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed000c8fb0 0x7fecec005000 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fecf800e210 tx=0x7fecf800b040).ready entity=mon.2 client_cookie=15d15c7470d4af35 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:33.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fecf7fff700 1 -- 172.21.15.121:0/3072071725 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf8009ac0 con 0x7fed000c8fb0 2023-11-15T03:10:33.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fecf7fff700 1 -- 172.21.15.121:0/3072071725 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf800f040 con 0x7fed000c8fb0 2023-11-15T03:10:33.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fecf7fff700 1 -- 172.21.15.121:0/3072071725 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf800ecd0 con 0x7fed000c8fb0 2023-11-15T03:10:33.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fed073db700 1 -- 172.21.15.121:0/3072071725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed000c8fb0 msgr2=0x7fecec005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:33.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fed073db700 1 --2- 172.21.15.121:0/3072071725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed000c8fb0 0x7fecec005000 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fecf800e210 tx=0x7fecf800b040).stop 2023-11-15T03:10:33.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fed05978700 1 -- 172.21.15.121:0/3072071725 reap_dead start 2023-11-15T03:10:33.498 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fed073db700 1 -- 172.21.15.121:0/3072071725 shutdown_connections 2023-11-15T03:10:33.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fed073db700 1 -- 172.21.15.121:0/3072071725 >> 172.21.15.121:0/3072071725 conn(0x7fed00067a60 msgr2=0x7fed00067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:33.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fed073db700 1 -- 172.21.15.121:0/3072071725 shutdown_connections 2023-11-15T03:10:33.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.481+0000 7fed073db700 1 -- 172.21.15.121:0/3072071725 wait complete. 2023-11-15T03:10:33.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.485+0000 7fed073db700 1 Processor -- start 2023-11-15T03:10:33.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.485+0000 7fed073db700 1 -- start start 2023-11-15T03:10:33.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed073db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed000c8fb0 0x7fed000c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:33.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed073db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed000c37c0 0x7fed001642a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:33.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed073db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed00164790 0x7fed00164bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:33.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed073db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0014a0c0 con 0x7fed00164790 2023-11-15T03:10:33.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed073db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed000c3c40 con 0x7fed000c8fb0 2023-11-15T03:10:33.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed073db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed00167990 con 0x7fed000c37c0 2023-11-15T03:10:33.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed05978700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed00164790 0x7fed00164bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:33.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed05978700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed00164790 0x7fed00164bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:35988/0 (socket says 172.21.15.121:35988) 2023-11-15T03:10:33.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed05978700 1 -- 172.21.15.121:0/3285722858 learned_addr learned my addr 172.21.15.121:0/3285722858 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:33.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed05177700 1 --2- 172.21.15.121:0/3285722858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed000c8fb0 0x7fed000c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:33.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed04976700 1 --2- 172.21.15.121:0/3285722858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed000c37c0 0x7fed001642a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:33.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed05978700 1 -- 172.21.15.121:0/3285722858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed000c37c0 msgr2=0x7fed001642a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:33.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed05978700 1 --2- 172.21.15.121:0/3285722858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed000c37c0 0x7fed001642a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:33.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed05978700 1 -- 172.21.15.121:0/3285722858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed000c8fb0 msgr2=0x7fed000c32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:33.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed05978700 1 --2- 172.21.15.121:0/3285722858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed000c8fb0 0x7fed000c32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:33.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed05978700 1 -- 172.21.15.121:0/3285722858 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed00167c70 con 0x7fed00164790 2023-11-15T03:10:33.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed05177700 1 --2- 172.21.15.121:0/3285722858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed000c8fb0 0x7fed000c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:33.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed05978700 1 --2- 172.21.15.121:0/3285722858 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed00164790 0x7fed00164bd0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 rx=0x7fecec005670 tx=0x7fecf801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:33.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fecf57fa700 1 -- 172.21.15.121:0/3285722858 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf8021030 con 0x7fed00164790 2023-11-15T03:10:33.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fecf57fa700 1 -- 172.21.15.121:0/3285722858 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf8009ac0 con 0x7fed00164790 2023-11-15T03:10:33.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed073db700 1 -- 172.21.15.121:0/3285722858 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed00167f50 con 0x7fed00164790 2023-11-15T03:10:33.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fecf57fa700 1 -- 172.21.15.121:0/3285722858 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf8013dc0 con 0x7fed00164790 2023-11-15T03:10:33.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.489+0000 7fed073db700 1 -- 172.21.15.121:0/3285722858 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0014ae60 con 0x7fed00164790 2023-11-15T03:10:33.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.493+0000 7fed073db700 1 -- 172.21.15.121:0/3285722858 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed001683d0 con 0x7fed00164790 2023-11-15T03:10:33.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.493+0000 7fecf57fa700 1 -- 172.21.15.121:0/3285722858 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fecf8024070 con 0x7fed00164790 2023-11-15T03:10:33.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.493+0000 7fecf57fa700 1 --2- 172.21.15.121:0/3285722858 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fece405a540 0x7fece405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:33.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.493+0000 7fecf57fa700 1 -- 172.21.15.121:0/3285722858 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9448+0+0 (secure 0 0 0) 0x7fecf8022070 con 0x7fed00164790 2023-11-15T03:10:33.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.493+0000 7fed05177700 1 --2- 172.21.15.121:0/3285722858 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fece405a540 0x7fece405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:33.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.497+0000 7fed05177700 1 --2- 172.21.15.121:0/3285722858 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fece405a540 0x7fece405ca00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7fecfc000fd0 tx=0x7fecfc009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:33.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.497+0000 7fecf57fa700 1 -- 172.21.15.121:0/3285722858 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fecf80497e0 con 0x7fed00164790 2023-11-15T03:10:33.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:33.713+0000 7fed073db700 1 -- 172.21.15.121:0/3285722858 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fed000c96c0 con 0x7fed00164790 2023-11-15T03:10:34.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.289+0000 7fecf57fa700 1 -- 172.21.15.121:0/3285722858 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v321) v1 ==== 107+0+0 (secure 0 0 0) 0x7fed000c96c0 con 0x7fed00164790 2023-11-15T03:10:34.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.293+0000 7fed073db700 1 -- 172.21.15.121:0/3285722858 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fece405a540 msgr2=0x7fece405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.293+0000 7fed073db700 1 --2- 172.21.15.121:0/3285722858 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fece405a540 0x7fece405ca00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7fecfc000fd0 tx=0x7fecfc009040).stop 2023-11-15T03:10:34.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.293+0000 7fed073db700 1 -- 172.21.15.121:0/3285722858 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed00164790 msgr2=0x7fed00164bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.293+0000 7fed073db700 1 --2- 172.21.15.121:0/3285722858 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed00164790 0x7fed00164bd0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 rx=0x7fecec005670 tx=0x7fecf801a000).stop 2023-11-15T03:10:34.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.293+0000 7fed073db700 1 -- 172.21.15.121:0/3285722858 shutdown_connections 2023-11-15T03:10:34.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.293+0000 7fed073db700 1 --2- 172.21.15.121:0/3285722858 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fece405a540 0x7fece405ca00 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.293+0000 7fed073db700 1 --2- 172.21.15.121:0/3285722858 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed00164790 0x7fed00164bd0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.293+0000 7fed073db700 1 --2- 172.21.15.121:0/3285722858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed000c37c0 0x7fed001642a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.293+0000 7fed073db700 1 --2- 172.21.15.121:0/3285722858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed000c8fb0 0x7fed000c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.293+0000 7fed073db700 1 -- 172.21.15.121:0/3285722858 >> 172.21.15.121:0/3285722858 conn(0x7fed00067a60 msgr2=0x7fed000c6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:34.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.293+0000 7fed073db700 1 -- 172.21.15.121:0/3285722858 shutdown_connections 2023-11-15T03:10:34.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.293+0000 7fed073db700 1 -- 172.21.15.121:0/3285722858 wait complete. 2023-11-15T03:10:34.304 INFO:teuthology.orchestra.run.smithi121.stderr:nothing to do: rank is not damaged 2023-11-15T03:10:34.322 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:10:34.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bc8179d00 1 Processor -- start 2023-11-15T03:10:34.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bc8179d00 1 -- start start 2023-11-15T03:10:34.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bc8179d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b70311fa30 0x55b7031512d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bc8179d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b70316ae70 0x55b7031f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bc8179d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b703200000 0x55b70316b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bc8179d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b703098220 con 0x55b70316ae70 2023-11-15T03:10:34.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bc8179d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7031517c0 con 0x55b70311fa30 2023-11-15T03:10:34.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bc8179d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b703165a80 con 0x55b703200000 2023-11-15T03:10:34.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bbd2b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b70316ae70 0x55b7031f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bbd2b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b70316ae70 0x55b7031f5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36002/0 (socket says 172.21.15.121:36002) 2023-11-15T03:10:34.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/776058601 learned_addr learned my addr 172.21.15.121:0/776058601 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:34.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bc5ab7700 1 --2- 172.21.15.121:0/776058601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b70311fa30 0x55b7031512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bc62b8700 1 --2- 172.21.15.121:0/776058601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b703200000 0x55b70316b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/776058601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b703200000 msgr2=0x55b70316b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:34.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bbd2b6700 1 --2- 172.21.15.121:0/776058601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b703200000 0x55b70316b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/776058601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b70311fa30 msgr2=0x55b7031512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:34.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bbd2b6700 1 --2- 172.21.15.121:0/776058601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b70311fa30 0x55b7031512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/776058601 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc0005db0 con 0x55b70316ae70 2023-11-15T03:10:34.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bc5ab7700 1 --2- 172.21.15.121:0/776058601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b70311fa30 0x55b7031512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:34.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.417+0000 7f8bbd2b6700 1 --2- 172.21.15.121:0/776058601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b70316ae70 0x55b7031f5ec0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 rx=0x7f8bb0001df0 tx=0x7f8bb00018c0).ready entity=mon.0 client_cookie=70d9d989194bb89a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:34.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.421+0000 7f8bc4ab5700 1 -- 172.21.15.121:0/776058601 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000bc10 con 0x55b70316ae70 2023-11-15T03:10:34.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.421+0000 7f8bc4ab5700 1 -- 172.21.15.121:0/776058601 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb0008070 con 0x55b70316ae70 2023-11-15T03:10:34.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.421+0000 7f8bc4ab5700 1 -- 172.21.15.121:0/776058601 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000bd70 con 0x55b70316ae70 2023-11-15T03:10:34.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.421+0000 7f8bc8179d00 1 -- 172.21.15.121:0/776058601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b70316ae70 msgr2=0x55b7031f5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.421+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/776058601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b70316ae70 0x55b7031f5ec0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 rx=0x7f8bb0001df0 tx=0x7f8bb00018c0).stop 2023-11-15T03:10:34.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.421+0000 7f8bc8179d00 1 -- 172.21.15.121:0/776058601 shutdown_connections 2023-11-15T03:10:34.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.421+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/776058601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b703200000 0x55b70316b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.421+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/776058601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b70316ae70 0x55b7031f5ec0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.421+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/776058601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b70311fa30 0x55b7031512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.421+0000 7f8bc8179d00 1 -- 172.21.15.121:0/776058601 >> 172.21.15.121:0/776058601 conn(0x55b70316f080 msgr2=0x55b70311eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:34.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.421+0000 7f8bc8179d00 1 -- 172.21.15.121:0/776058601 shutdown_connections 2023-11-15T03:10:34.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.421+0000 7f8bc8179d00 1 -- 172.21.15.121:0/776058601 wait complete. 2023-11-15T03:10:34.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.421+0000 7f8bc8179d00 1 Processor -- start 2023-11-15T03:10:34.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.421+0000 7f8bc8179d00 1 -- start start 2023-11-15T03:10:34.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bc8179d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b70311fa30 0x55b703166b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bc8179d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b703200000 0x55b70320ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bc8179d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b7031fe1d0 0x55b70320fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bc8179d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b703153a40 con 0x55b703200000 2023-11-15T03:10:34.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bc8179d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7032101c0 con 0x55b70311fa30 2023-11-15T03:10:34.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bc8179d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b703167190 con 0x55b7031fe1d0 2023-11-15T03:10:34.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bbd2b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b703200000 0x55b70320ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bbd2b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b703200000 0x55b70320ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36008/0 (socket says 172.21.15.121:36008) 2023-11-15T03:10:34.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 learned_addr learned my addr 172.21.15.121:0/1492025506 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:34.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bc62b8700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b7031fe1d0 0x55b70320fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bc5ab7700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b70311fa30 0x55b703166b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b7031fe1d0 msgr2=0x55b70320fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bbd2b6700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b7031fe1d0 0x55b70320fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b70311fa30 msgr2=0x55b703166b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bbd2b6700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b70311fa30 0x55b703166b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb000a040 con 0x55b703200000 2023-11-15T03:10:34.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bc5ab7700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b70311fa30 0x55b703166b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:10:34.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bbd2b6700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b703200000 0x55b70320ce70 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 rx=0x7f8bb000f000 tx=0x7f8bb0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:34.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bbeffd700 1 -- 172.21.15.121:0/1492025506 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0010d70 con 0x55b703200000 2023-11-15T03:10:34.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bbeffd700 1 -- 172.21.15.121:0/1492025506 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb001c400 con 0x55b703200000 2023-11-15T03:10:34.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b7030c1770 con 0x55b703200000 2023-11-15T03:10:34.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bbeffd700 1 -- 172.21.15.121:0/1492025506 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb001c7c0 con 0x55b703200000 2023-11-15T03:10:34.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.425+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b703090a70 con 0x55b703200000 2023-11-15T03:10:34.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.429+0000 7f8bbeffd700 1 -- 172.21.15.121:0/1492025506 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8bb0017070 con 0x55b703200000 2023-11-15T03:10:34.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.429+0000 7f8bbeffd700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8ba8059020 0x7f8ba805b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.433+0000 7f8bbeffd700 1 -- 172.21.15.121:0/1492025506 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9448+0+0 (secure 0 0 0) 0x7f8bb0075430 con 0x55b703200000 2023-11-15T03:10:34.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.433+0000 7f8bc5ab7700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8ba8059020 0x7f8ba805b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.444 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:10:34.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.433+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55b703153ed0 0x55b7031542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.433+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b7031547a0 con 0x55b703153ed0 2023-11-15T03:10:34.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.433+0000 7f8bc5ab7700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8ba8059020 0x7f8ba805b4e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f8bb8012390 tx=0x7f8bb8007040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:34.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.433+0000 7f8bc62b8700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55b703153ed0 0x55b7031542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.433+0000 7f8bc62b8700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55b703153ed0 0x55b7031542b0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:34.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.437+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v131'437 uv430 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00054f0 con 0x55b703153ed0 2023-11-15T03:10:34.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.437+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b703219570 con 0x55b703153ed0 2023-11-15T03:10:34.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.437+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v131'438 uv438 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703153ed0 2023-11-15T03:10:34.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.437+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b703219920 con 0x55b703153ed0 2023-11-15T03:10:34.448 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:10:34.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.441+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v131'439 uv439 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703153ed0 2023-11-15T03:10:34.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.441+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321ace0 con 0x55b703153ed0 2023-11-15T03:10:34.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.445+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v131'440 uv440 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703153ed0 2023-11-15T03:10:34.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.445+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321b090 con 0x55b703153ed0 2023-11-15T03:10:34.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.445+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v131'441 uv441 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703153ed0 2023-11-15T03:10:34.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.445+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b703219cd0 con 0x55b703153ed0 2023-11-15T03:10:34.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.449+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v131'442 uv442 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703153ed0 2023-11-15T03:10:34.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.449+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321a080 con 0x55b703153ed0 2023-11-15T03:10:34.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.453+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v131'443 uv436 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703153ed0 2023-11-15T03:10:34.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.453+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b70321a480 con 0x55b703153ed0 2023-11-15T03:10:34.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.453+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v131'444 uv444 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703153ed0 2023-11-15T03:10:34.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.453+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321a480 con 0x55b703153ed0 2023-11-15T03:10:34.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.457+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v131'445 uv445 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703153ed0 2023-11-15T03:10:34.463 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:10:34.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.457+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321a430 con 0x55b703153ed0 2023-11-15T03:10:34.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.457+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v131'446 uv446 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703153ed0 2023-11-15T03:10:34.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.457+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321a430 con 0x55b703153ed0 2023-11-15T03:10:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.461+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v131'447 uv447 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703153ed0 2023-11-15T03:10:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.461+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321a430 con 0x55b703153ed0 2023-11-15T03:10:34.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.461+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v131'448 uv448 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703153ed0 2023-11-15T03:10:34.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.461+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55b70321b440 0x55b70321d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.461+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321dd20 con 0x55b70321b440 2023-11-15T03:10:34.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.465+0000 7f8bc5ab7700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55b70321b440 0x55b70321d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.465+0000 7f8bc5ab7700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55b70321b440 0x55b70321d830 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.469+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v131'692 uv679 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.469+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b70321e0d0 con 0x55b70321b440 2023-11-15T03:10:34.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.473+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v131'693 uv693 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.473+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321e480 con 0x55b70321b440 2023-11-15T03:10:34.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.477+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v131'694 uv694 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.483 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:10:34.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.477+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321f840 con 0x55b70321b440 2023-11-15T03:10:34.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.477+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v131'695 uv695 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.477+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321fbf0 con 0x55b70321b440 2023-11-15T03:10:34.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.481+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v131'696 uv696 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.481+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321e830 con 0x55b70321b440 2023-11-15T03:10:34.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.485+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v131'697 uv697 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.485+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321f120 con 0x55b70321b440 2023-11-15T03:10:34.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.485+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v131'698 uv698 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.485+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321ffa0 con 0x55b70321b440 2023-11-15T03:10:34.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.489+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v131'699 uv685 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.489+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b70321ffa0 con 0x55b70321b440 2023-11-15T03:10:34.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.493+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v131'700 uv700 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.493+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321ffa0 con 0x55b70321b440 2023-11-15T03:10:34.502 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:10:34.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.493+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v131'701 uv701 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.493+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b703220fb0 con 0x55b70321b440 2023-11-15T03:10:34.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.497+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v131'702 uv702 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.497+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b703220fb0 con 0x55b70321b440 2023-11-15T03:10:34.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.501+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v131'703 uv703 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.501+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321ffa0 con 0x55b70321b440 2023-11-15T03:10:34.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.505+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v131'704 uv704 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.505+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321ffa0 con 0x55b70321b440 2023-11-15T03:10:34.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.505+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v131'705 uv691 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.505+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b70321ffa0 con 0x55b70321b440 2023-11-15T03:10:34.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.509+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v131'706 uv706 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.509+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321ffa0 con 0x55b70321b440 2023-11-15T03:10:34.519 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:10:34.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.513+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v131'707 uv707 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.513+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b703220fb0 con 0x55b70321b440 2023-11-15T03:10:34.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.513+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v131'708 uv708 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.513+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b703220fb0 con 0x55b70321b440 2023-11-15T03:10:34.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.517+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v131'709 uv709 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.517+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321ffa0 con 0x55b70321b440 2023-11-15T03:10:34.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.521+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v131'710 uv710 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.521+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55b7032204b0 0x55b7032228a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.521+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b703222d90 con 0x55b7032204b0 2023-11-15T03:10:34.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.521+0000 7f8bbd2b6700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55b7032204b0 0x55b7032228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.521+0000 7f8bbd2b6700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55b7032204b0 0x55b7032228a0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:34.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.525+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v131'378 uv371 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b7032204b0 2023-11-15T03:10:34.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.525+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b703223140 con 0x55b7032204b0 2023-11-15T03:10:34.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.529+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v131'379 uv379 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b7032204b0 2023-11-15T03:10:34.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.529+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b7032234f0 con 0x55b7032204b0 2023-11-15T03:10:34.538 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:10:34.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.529+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v131'380 uv380 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b7032204b0 2023-11-15T03:10:34.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.529+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b7032238a0 con 0x55b7032204b0 2023-11-15T03:10:34.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.533+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v131'381 uv381 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b7032204b0 2023-11-15T03:10:34.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.533+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70321ed30 con 0x55b7032204b0 2023-11-15T03:10:34.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.537+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v131'382 uv382 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b7032204b0 2023-11-15T03:10:34.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.537+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b703224140 con 0x55b7032204b0 2023-11-15T03:10:34.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.541+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v131'383 uv377 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b7032204b0 2023-11-15T03:10:34.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.541+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b7032244f0 con 0x55b7032204b0 2023-11-15T03:10:34.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.545+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v131'384 uv384 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b7032204b0 2023-11-15T03:10:34.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.545+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b7032248a0 con 0x55b7032204b0 2023-11-15T03:10:34.554 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:10:34.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.545+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v131'385 uv385 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b7032204b0 2023-11-15T03:10:34.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.545+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b7032248a0 con 0x55b7032204b0 2023-11-15T03:10:34.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.549+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v131'386 uv386 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b7032204b0 2023-11-15T03:10:34.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.549+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b703223cd0 con 0x55b7032204b0 2023-11-15T03:10:34.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.553+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v131'387 uv387 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b7032204b0 2023-11-15T03:10:34.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.553+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55b703223cd0 0x55b7032272b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.553+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b7032277e0 con 0x55b703223cd0 2023-11-15T03:10:34.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.553+0000 7f8bc62b8700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55b703223cd0 0x55b7032272b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.553+0000 7f8bc62b8700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55b703223cd0 0x55b7032272b0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:34.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.557+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v131'446 uv445 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.557+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b703227bc0 con 0x55b703223cd0 2023-11-15T03:10:34.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.561+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v131'447 uv447 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.561+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b703227bc0 con 0x55b703223cd0 2023-11-15T03:10:34.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.565+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v131'448 uv448 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.572 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:10:34.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.565+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322bc50 con 0x55b703223cd0 2023-11-15T03:10:34.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.569+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v131'449 uv449 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.569+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322bc50 con 0x55b703223cd0 2023-11-15T03:10:34.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.573+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v131'450 uv450 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.573+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b703227bc0 con 0x55b703223cd0 2023-11-15T03:10:34.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.573+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v131'451 uv442 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.573+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b703227bc0 con 0x55b703223cd0 2023-11-15T03:10:34.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.577+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v131'452 uv452 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.577+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b703227bc0 con 0x55b703223cd0 2023-11-15T03:10:34.587 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:10:34.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.581+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v131'453 uv453 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.581+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b703228bd0 con 0x55b703223cd0 2023-11-15T03:10:34.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.581+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v131'454 uv454 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.581+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b703228bd0 con 0x55b703223cd0 2023-11-15T03:10:34.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.585+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v131'455 uv455 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.585+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b703227bc0 con 0x55b703223cd0 2023-11-15T03:10:34.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.589+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v131'456 uv456 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.589+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55b7032280d0 0x55b70322a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.589+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322abc0 con 0x55b7032280d0 2023-11-15T03:10:34.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.589+0000 7f8bc5ab7700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55b7032280d0 0x55b70322a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.589+0000 7f8bc5ab7700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55b7032280d0 0x55b70322a4c0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:34.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.593+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v131'219 uv218 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b7032280d0 2023-11-15T03:10:34.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.593+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b70322aff0 con 0x55b7032280d0 2023-11-15T03:10:34.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.597+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v131'220 uv220 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b7032280d0 2023-11-15T03:10:34.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.597+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322aff0 con 0x55b7032280d0 2023-11-15T03:10:34.606 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:10:34.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.601+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v131'221 uv221 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b7032280d0 2023-11-15T03:10:34.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.601+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322c000 con 0x55b7032280d0 2023-11-15T03:10:34.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.601+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v131'222 uv222 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b7032280d0 2023-11-15T03:10:34.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.601+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322c000 con 0x55b7032280d0 2023-11-15T03:10:34.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.605+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v131'223 uv223 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b7032280d0 2023-11-15T03:10:34.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.605+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322aff0 con 0x55b7032280d0 2023-11-15T03:10:34.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.605+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v131'224 uv224 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b7032280d0 2023-11-15T03:10:34.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.605+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55b70322b670 0x55b70322daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.605+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e1e0 con 0x55b70322b670 2023-11-15T03:10:34.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.609+0000 7f8bbd2b6700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55b70322b670 0x55b70322daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.609+0000 7f8bbd2b6700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55b70322b670 0x55b70322daa0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:34.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.609+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v131'161 uv160 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.609+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b70322e5c0 con 0x55b70322b670 2023-11-15T03:10:34.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.613+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v131'162 uv162 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.613+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e5c0 con 0x55b70322b670 2023-11-15T03:10:34.624 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:10:34.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.617+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v131'163 uv163 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.617+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e5c0 con 0x55b70322b670 2023-11-15T03:10:34.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.621+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v131'164 uv164 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.621+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e520 con 0x55b70322b670 2023-11-15T03:10:34.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.625+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v131'219 uv218 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.625+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b70322e520 con 0x55b70322b670 2023-11-15T03:10:34.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.629+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v131'220 uv220 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.629+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e520 con 0x55b70322b670 2023-11-15T03:10:34.637 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:10:34.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.629+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v131'221 uv221 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.629+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322f530 con 0x55b70322b670 2023-11-15T03:10:34.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.633+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v131'222 uv222 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.633+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322f930 con 0x55b70322b670 2023-11-15T03:10:34.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.637+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v131'223 uv223 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.637+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e520 con 0x55b70322b670 2023-11-15T03:10:34.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.637+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v131'224 uv224 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.637+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e520 con 0x55b70321b440 2023-11-15T03:10:34.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.641+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v131'404 uv398 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.641+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b70322e520 con 0x55b70321b440 2023-11-15T03:10:34.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.645+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v131'405 uv405 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.645+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e520 con 0x55b70321b440 2023-11-15T03:10:34.655 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:10:34.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.649+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v131'406 uv406 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.649+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e520 con 0x55b70321b440 2023-11-15T03:10:34.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.653+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v131'407 uv407 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.653+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e520 con 0x55b70321b440 2023-11-15T03:10:34.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.653+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v131'408 uv408 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.653+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e520 con 0x55b70321b440 2023-11-15T03:10:34.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.657+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v131'409 uv409 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.657+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e520 con 0x55b70321b440 2023-11-15T03:10:34.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.661+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v131'410 uv403 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.661+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b70322e520 con 0x55b70321b440 2023-11-15T03:10:34.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.665+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v131'411 uv411 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.665+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e520 con 0x55b70321b440 2023-11-15T03:10:34.674 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:10:34.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.669+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v131'412 uv412 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.669+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e520 con 0x55b70321b440 2023-11-15T03:10:34.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.669+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v131'413 uv413 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.669+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e520 con 0x55b70321b440 2023-11-15T03:10:34.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.673+0000 7f8bc5ab7700 1 -- 172.21.15.121:0/1492025506 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v131'414 uv414 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb80144a0 con 0x55b70321b440 2023-11-15T03:10:34.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.673+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e520 con 0x55b7032204b0 2023-11-15T03:10:34.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.677+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v131'183 uv182 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b7032204b0 2023-11-15T03:10:34.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.677+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b70322e520 con 0x55b7032204b0 2023-11-15T03:10:34.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.681+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v131'184 uv184 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b7032204b0 2023-11-15T03:10:34.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.681+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e520 con 0x55b7032204b0 2023-11-15T03:10:34.689 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:10:34.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.681+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v131'185 uv185 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b7032204b0 2023-11-15T03:10:34.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.681+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e520 con 0x55b7032204b0 2023-11-15T03:10:34.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.685+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v131'186 uv186 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b7032204b0 2023-11-15T03:10:34.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.685+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e970 con 0x55b7032204b0 2023-11-15T03:10:34.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.685+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v131'187 uv187 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b7032204b0 2023-11-15T03:10:34.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.685+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e920 con 0x55b70322b670 2023-11-15T03:10:34.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.689+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v131'219 uv218 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.689+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b70322e920 con 0x55b70322b670 2023-11-15T03:10:34.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.693+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v131'220 uv220 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.693+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e920 con 0x55b70322b670 2023-11-15T03:10:34.703 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:10:34.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.697+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v131'221 uv221 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.697+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322f930 con 0x55b70322b670 2023-11-15T03:10:34.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.697+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v131'222 uv222 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.697+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322f930 con 0x55b70322b670 2023-11-15T03:10:34.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.701+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v131'223 uv223 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.701+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e920 con 0x55b70322b670 2023-11-15T03:10:34.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.705+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v131'224 uv224 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.705+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e920 con 0x55b70322b670 2023-11-15T03:10:34.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.709+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v131'184 uv183 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.709+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b70322e920 con 0x55b70322b670 2023-11-15T03:10:34.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.713+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v131'185 uv185 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.713+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e920 con 0x55b70322b670 2023-11-15T03:10:34.722 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:10:34.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.717+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v131'186 uv186 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.717+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e920 con 0x55b70322b670 2023-11-15T03:10:34.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.721+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v131'187 uv187 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.721+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e920 con 0x55b70322b670 2023-11-15T03:10:34.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.721+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v131'188 uv188 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.721+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e920 con 0x55b70322b670 2023-11-15T03:10:34.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.725+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v131'182 uv181 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.725+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b70322e920 con 0x55b70322b670 2023-11-15T03:10:34.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.729+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v131'183 uv183 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.729+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e920 con 0x55b70322b670 2023-11-15T03:10:34.739 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:10:34.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.733+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v131'184 uv184 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.733+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322e920 con 0x55b70322b670 2023-11-15T03:10:34.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.733+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v131'185 uv185 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.733+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b703231690 con 0x55b70322b670 2023-11-15T03:10:34.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.737+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v131'186 uv186 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.737+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55b70322f320 0x55b70322f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.737+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b70322f320 2023-11-15T03:10:34.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.737+0000 7f8bc62b8700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55b70322f320 0x55b70322f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.741+0000 7f8bc62b8700 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55b70322f320 0x55b70322f700 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:34.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.741+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v131'146 uv145 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b70322f320 2023-11-15T03:10:34.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.741+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b70322f320 2023-11-15T03:10:34.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.745+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v131'147 uv147 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b70322f320 2023-11-15T03:10:34.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.745+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b70322f320 2023-11-15T03:10:34.755 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:10:34.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.749+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v131'148 uv148 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b70322f320 2023-11-15T03:10:34.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.749+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b70322f320 2023-11-15T03:10:34.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.753+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v131'149 uv149 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b70322f320 2023-11-15T03:10:34.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.753+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b70322b670 2023-11-15T03:10:34.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.753+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v131'219 uv218 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.753+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b70322b670 2023-11-15T03:10:34.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.757+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v131'220 uv220 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.757+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b70322b670 2023-11-15T03:10:34.766 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:10:34.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.761+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v131'221 uv221 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.761+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b70322b670 2023-11-15T03:10:34.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.761+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v131'222 uv222 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.761+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b70322b670 2023-11-15T03:10:34.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.765+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v131'223 uv223 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.765+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b70322b670 2023-11-15T03:10:34.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.765+0000 7f8bbd2b6700 1 -- 172.21.15.121:0/1492025506 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v131'224 uv224 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bb0022070 con 0x55b70322b670 2023-11-15T03:10:34.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.765+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b703223cd0 2023-11-15T03:10:34.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.769+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v131'186 uv185 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.769+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b703223cd0 2023-11-15T03:10:34.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.773+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v131'187 uv187 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.773+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b703223cd0 2023-11-15T03:10:34.782 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:10:34.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.773+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v131'188 uv188 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.777+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b703223cd0 2023-11-15T03:10:34.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.777+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v131'189 uv189 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.777+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b703223cd0 2023-11-15T03:10:34.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.781+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v131'190 uv190 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.781+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b703223cd0 2023-11-15T03:10:34.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.785+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v131'439 uv431 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.785+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b703223cd0 2023-11-15T03:10:34.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.785+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v131'440 uv440 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.789+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b703223cd0 2023-11-15T03:10:34.797 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:10:34.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.789+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v131'441 uv441 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.789+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b703223cd0 2023-11-15T03:10:34.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.793+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v131'442 uv442 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.793+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b703223cd0 2023-11-15T03:10:34.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.797+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v131'443 uv443 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.797+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b703223cd0 2023-11-15T03:10:34.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.797+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v131'444 uv438 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.797+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b703223cd0 2023-11-15T03:10:34.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.801+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v131'445 uv445 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.801+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b703223cd0 2023-11-15T03:10:34.811 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:10:34.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.805+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v131'446 uv446 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.805+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b703223cd0 2023-11-15T03:10:34.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.805+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v131'447 uv447 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.805+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b70322fdd0 con 0x55b703223cd0 2023-11-15T03:10:34.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.809+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v131'448 uv448 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.809+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b7032301d0 con 0x55b703223cd0 2023-11-15T03:10:34.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.813+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v131'449 uv449 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.813+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55b7032306c0 con 0x55b703223cd0 2023-11-15T03:10:34.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.813+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v131'450 uv450 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8bc00089d0 con 0x55b703223cd0 2023-11-15T03:10:34.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55b7032204b0 msgr2=0x55b7032228a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55b7032204b0 0x55b7032228a0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55b703153ed0 msgr2=0x55b7031542b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55b703153ed0 0x55b7031542b0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55b7032280d0 msgr2=0x55b70322a4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55b7032280d0 0x55b70322a4c0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55b70321b440 msgr2=0x55b70321d830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55b70321b440 0x55b70321d830 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55b70322f320 msgr2=0x55b70322f700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55b70322f320 0x55b70322f700 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55b703223cd0 msgr2=0x55b7032272b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55b703223cd0 0x55b7032272b0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55b70322b670 msgr2=0x55b70322daa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55b70322b670 0x55b70322daa0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 reap_dead start 2023-11-15T03:10:34.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 reap_dead start 2023-11-15T03:10:34.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 reap_dead start 2023-11-15T03:10:34.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc62b8700 1 -- 172.21.15.121:0/1492025506 reap_dead start 2023-11-15T03:10:34.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8ba8059020 msgr2=0x7f8ba805b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8ba8059020 0x7f8ba805b4e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f8bb8012390 tx=0x7f8bb8007040).stop 2023-11-15T03:10:34.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b703200000 msgr2=0x55b70320ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b703200000 0x55b70320ce70 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 rx=0x7f8bb000f000 tx=0x7f8bb0015000).stop 2023-11-15T03:10:34.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 shutdown_connections 2023-11-15T03:10:34.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8ba8059020 0x7f8ba805b4e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 --2- 172.21.15.121:0/1492025506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b703200000 0x55b70320ce70 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 >> 172.21.15.121:0/1492025506 conn(0x55b70316f080 msgr2=0x55b70316e550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:34.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 shutdown_connections 2023-11-15T03:10:34.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.817+0000 7f8bc8179d00 1 -- 172.21.15.121:0/1492025506 wait complete. 2023-11-15T03:10:34.831 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 100.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-11-15T03:10:34.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.933+0000 7fbcc2b40d00 1 Processor -- start 2023-11-15T03:10:34.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.933+0000 7fbcc2b40d00 1 -- start start 2023-11-15T03:10:34.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.933+0000 7fbcc2b40d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559617f5bb0 0x5559618c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.933+0000 7fbcc2b40d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55596183fb00 0x555961829e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.933+0000 7fbcc2b40d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559618d2650 0x555961828590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.933+0000 7fbcc2b40d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559617a3cc0 con 0x5559617f5bb0 2023-11-15T03:10:34.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.933+0000 7fbcc2b40d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596176e220 con 0x5559618d2650 2023-11-15T03:10:34.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.933+0000 7fbcc2b40d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55596183ebe0 con 0x55596183fb00 2023-11-15T03:10:34.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.933+0000 7fbcb711a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559617f5bb0 0x5559618c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.933+0000 7fbcb711a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559617f5bb0 0x5559618c76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56436/0 (socket says 172.21.15.121:56436) 2023-11-15T03:10:34.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.933+0000 7fbcb711a700 1 -- 172.21.15.121:0/1660958065 learned_addr learned my addr 172.21.15.121:0/1660958065 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:34.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.933+0000 7fbcb791b700 1 --2- 172.21.15.121:0/1660958065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559618d2650 0x555961828590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.933+0000 7fbcae919700 1 --2- 172.21.15.121:0/1660958065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55596183fb00 0x555961829e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcb711a700 1 -- 172.21.15.121:0/1660958065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55596183fb00 msgr2=0x555961829e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:34.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcb711a700 1 --2- 172.21.15.121:0/1660958065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55596183fb00 0x555961829e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcb711a700 1 -- 172.21.15.121:0/1660958065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559618d2650 msgr2=0x555961828590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:34.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcb711a700 1 --2- 172.21.15.121:0/1660958065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559618d2650 0x555961828590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcb711a700 1 -- 172.21.15.121:0/1660958065 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca0005cf0 con 0x5559617f5bb0 2023-11-15T03:10:34.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcb791b700 1 --2- 172.21.15.121:0/1660958065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559618d2650 0x555961828590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:34.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcae919700 1 --2- 172.21.15.121:0/1660958065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55596183fb00 0x555961829e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:34.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcb711a700 1 --2- 172.21.15.121:0/1660958065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559617f5bb0 0x5559618c76f0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 rx=0x7fbca8006fc0 tx=0x7fbca8006a70).ready entity=mon.0 client_cookie=d68dd575993e4108 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:34.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcb6118700 1 -- 172.21.15.121:0/1660958065 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80020b0 con 0x5559617f5bb0 2023-11-15T03:10:34.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcb6118700 1 -- 172.21.15.121:0/1660958065 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca8006320 con 0x5559617f5bb0 2023-11-15T03:10:34.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcb6118700 1 -- 172.21.15.121:0/1660958065 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8004750 con 0x5559617f5bb0 2023-11-15T03:10:34.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcc2b40d00 1 -- 172.21.15.121:0/1660958065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559617f5bb0 msgr2=0x5559618c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcc2b40d00 1 --2- 172.21.15.121:0/1660958065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559617f5bb0 0x5559618c76f0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 rx=0x7fbca8006fc0 tx=0x7fbca8006a70).stop 2023-11-15T03:10:34.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcc2b40d00 1 -- 172.21.15.121:0/1660958065 shutdown_connections 2023-11-15T03:10:34.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcc2b40d00 1 --2- 172.21.15.121:0/1660958065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559618d2650 0x555961828590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcc2b40d00 1 --2- 172.21.15.121:0/1660958065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55596183fb00 0x555961829e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcc2b40d00 1 --2- 172.21.15.121:0/1660958065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559617f5bb0 0x5559618c76f0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcc2b40d00 1 -- 172.21.15.121:0/1660958065 >> 172.21.15.121:0/1660958065 conn(0x555961845080 msgr2=0x555961839fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:34.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcc2b40d00 1 -- 172.21.15.121:0/1660958065 shutdown_connections 2023-11-15T03:10:34.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcc2b40d00 1 -- 172.21.15.121:0/1660958065 wait complete. 2023-11-15T03:10:34.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcc2b40d00 1 Processor -- start 2023-11-15T03:10:34.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.937+0000 7fbcc2b40d00 1 -- start start 2023-11-15T03:10:34.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcc2b40d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559617f5bb0 0x5559618c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcc2b40d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55596183fb00 0x55596182a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcc2b40d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559618d2650 0x555961825bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcc2b40d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596183e580 con 0x5559618d2650 2023-11-15T03:10:34.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcc2b40d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596183da00 con 0x5559617f5bb0 2023-11-15T03:10:34.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcc2b40d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55596183dce0 con 0x55596183fb00 2023-11-15T03:10:34.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcb791b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559618d2650 0x555961825bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcb791b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559618d2650 0x555961825bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56438/0 (socket says 172.21.15.121:56438) 2023-11-15T03:10:34.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcae919700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55596183fb00 0x55596182a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcb711a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559617f5bb0 0x5559618c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcb791b700 1 -- 172.21.15.121:0/2745105655 learned_addr learned my addr 172.21.15.121:0/2745105655 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:34.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcb791b700 1 -- 172.21.15.121:0/2745105655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55596183fb00 msgr2=0x55596182a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcb791b700 1 --2- 172.21.15.121:0/2745105655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55596183fb00 0x55596182a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcb791b700 1 -- 172.21.15.121:0/2745105655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559617f5bb0 msgr2=0x5559618c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcb791b700 1 --2- 172.21.15.121:0/2745105655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559617f5bb0 0x5559618c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcb791b700 1 -- 172.21.15.121:0/2745105655 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca4001e10 con 0x5559618d2650 2023-11-15T03:10:34.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcae919700 1 --2- 172.21.15.121:0/2745105655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55596183fb00 0x55596182a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:10:34.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcb711a700 1 --2- 172.21.15.121:0/2745105655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5559617f5bb0 0x5559618c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:10:34.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcb791b700 1 --2- 172.21.15.121:0/2745105655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559618d2650 0x555961825bb0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 rx=0x7fbcb0001e20 tx=0x7fbcb0000bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:34.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcb4915700 1 -- 172.21.15.121:0/2745105655 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb000ba00 con 0x5559618d2650 2023-11-15T03:10:34.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcb4915700 1 -- 172.21.15.121:0/2745105655 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb000bb60 con 0x5559618d2650 2023-11-15T03:10:34.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcc2b40d00 1 -- 172.21.15.121:0/2745105655 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555961766a70 con 0x5559618d2650 2023-11-15T03:10:34.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcb4915700 1 -- 172.21.15.121:0/2745105655 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0010430 con 0x5559618d2650 2023-11-15T03:10:34.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.941+0000 7fbcc2b40d00 1 -- 172.21.15.121:0/2745105655 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555961772010 con 0x5559618d2650 2023-11-15T03:10:34.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.945+0000 7fbcb4915700 1 -- 172.21.15.121:0/2745105655 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fbcb0010860 con 0x5559618d2650 2023-11-15T03:10:34.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.945+0000 7fbcb4915700 1 --2- 172.21.15.121:0/2745105655 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbc98058ff0 0x7fbc9805b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.945+0000 7fbcb4915700 1 -- 172.21.15.121:0/2745105655 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9448+0+0 (secure 0 0 0) 0x7fbcb0008070 con 0x5559618d2650 2023-11-15T03:10:34.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.945+0000 7fbcb711a700 1 --2- 172.21.15.121:0/2745105655 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbc98058ff0 0x7fbc9805b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.949+0000 7fbcc2b40d00 1 --2- 172.21.15.121:0/2745105655 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x555961827640 0x5559618df200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:34.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.949+0000 7fbcc2b40d00 1 -- 172.21.15.121:0/2745105655 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:1 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected e131) v8 -- 0x55596182ab90 con 0x555961827640 2023-11-15T03:10:34.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.949+0000 7fbcb711a700 1 --2- 172.21.15.121:0/2745105655 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbc98058ff0 0x7fbc9805b4b0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7fbca8004900 tx=0x7fbca8012000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:34.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.949+0000 7fbcae919700 1 --2- 172.21.15.121:0/2745105655 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x555961827640 0x5559618df200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:34.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.949+0000 7fbcae919700 1 --2- 172.21.15.121:0/2745105655 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x555961827640 0x5559618df200 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:34.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.953+0000 7fbcae919700 1 -- 172.21.15.121:0/2745105655 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(1 100.00000000 [omap-set-header] v131'187 uv187 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fbca0000bc0 con 0x555961827640 2023-11-15T03:10:34.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.953+0000 7fbcc2b40d00 1 -- 172.21.15.121:0/2745105655 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x555961827640 msgr2=0x5559618df200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.953+0000 7fbcc2b40d00 1 --2- 172.21.15.121:0/2745105655 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x555961827640 0x5559618df200 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:34.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.953+0000 7fbcc2b40d00 1 -- 172.21.15.121:0/2745105655 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbc98058ff0 msgr2=0x7fbc9805b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.953+0000 7fbcc2b40d00 1 --2- 172.21.15.121:0/2745105655 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbc98058ff0 0x7fbc9805b4b0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7fbca8004900 tx=0x7fbca8012000).stop 2023-11-15T03:10:34.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.953+0000 7fbcc2b40d00 1 -- 172.21.15.121:0/2745105655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559618d2650 msgr2=0x555961825bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:34.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.953+0000 7fbcc2b40d00 1 --2- 172.21.15.121:0/2745105655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559618d2650 0x555961825bb0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 rx=0x7fbcb0001e20 tx=0x7fbcb0000bc0).stop 2023-11-15T03:10:34.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.953+0000 7fbcb791b700 1 -- 172.21.15.121:0/2745105655 reap_dead start 2023-11-15T03:10:34.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.953+0000 7fbcc2b40d00 1 -- 172.21.15.121:0/2745105655 shutdown_connections 2023-11-15T03:10:34.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.953+0000 7fbcc2b40d00 1 -- 172.21.15.121:0/2745105655 >> 172.21.15.121:0/2745105655 conn(0x555961845080 msgr2=0x555961828590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:34.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.953+0000 7fbcc2b40d00 1 -- 172.21.15.121:0/2745105655 shutdown_connections 2023-11-15T03:10:34.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:34.953+0000 7fbcc2b40d00 1 -- 172.21.15.121:0/2745105655 wait complete. 2023-11-15T03:10:34.968 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:10:35.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.121+0000 7fda6ffff700 1 Processor -- start 2023-11-15T03:10:35.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.121+0000 7fda6ffff700 1 -- start start 2023-11-15T03:10:35.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.125+0000 7fda6ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fda700bf210 0x7fda700c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:35.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.125+0000 7fda6ffff700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda70154ba0 con 0x7fda70153d60 2023-11-15T03:10:35.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.125+0000 7fda6ffff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda700be690 con 0x7fda70153980 2023-11-15T03:10:35.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.125+0000 7fda6ffff700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda700c5dc0 con 0x7fda700bf210 2023-11-15T03:10:35.132 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.125+0000 7fda6e7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fda700bf210 0x7fda700c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:35.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.125+0000 7fda6f7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fda70153d60 0x7fda700c5550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49878/0 (socket says 172.21.15.121:49878) 2023-11-15T03:10:35.133 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.125+0000 7fda6f7fe700 1 -- 172.21.15.121:0/2416508375 learned_addr learned my addr 172.21.15.121:0/2416508375 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:35.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.125+0000 7fda6e7fc700 1 -- 172.21.15.121:0/2416508375 >> v1:172.21.15.190:6789/0 conn(0x7fda70153980 legacy=0x7fda700beb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:35.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.125+0000 7fda6e7fc700 1 -- 172.21.15.121:0/2416508375 >> v1:172.21.15.121:6789/0 conn(0x7fda70153d60 legacy=0x7fda700c5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:35.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.125+0000 7fda6e7fc700 1 -- 172.21.15.121:0/2416508375 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda701542b0 con 0x7fda700bf210 2023-11-15T03:10:35.134 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.125+0000 7fda6e7fc700 1 --2- 172.21.15.121:0/2416508375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda700bf210 0x7fda700c5060 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 rx=0x7fda64009760 tx=0x7fda64009060).ready entity=mon.0 client_cookie=cd4d267d5302bbdb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:35.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.125+0000 7fda6dffb700 1 -- 172.21.15.121:0/2416508375 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda640090d0 con 0x7fda700bf210 2023-11-15T03:10:35.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.125+0000 7fda6dffb700 1 -- 172.21.15.121:0/2416508375 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6400f510 con 0x7fda700bf210 2023-11-15T03:10:35.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.125+0000 7fda6dffb700 1 -- 172.21.15.121:0/2416508375 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6400f890 con 0x7fda700bf210 2023-11-15T03:10:35.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.125+0000 7fda6ffff700 1 -- 172.21.15.121:0/2416508375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda700bf210 msgr2=0x7fda700c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:35.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.125+0000 7fda6ffff700 1 --2- 172.21.15.121:0/2416508375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda700bf210 0x7fda700c5060 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 rx=0x7fda64009760 tx=0x7fda64009060).stop 2023-11-15T03:10:35.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.129+0000 7fda6ffff700 1 -- 172.21.15.121:0/2416508375 shutdown_connections 2023-11-15T03:10:35.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.129+0000 7fda6ffff700 1 --2- 172.21.15.121:0/2416508375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda700bf210 0x7fda700c5060 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:35.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.129+0000 7fda6ffff700 1 -- 172.21.15.121:0/2416508375 >> 172.21.15.121:0/2416508375 conn(0x7fda70067a60 msgr2=0x7fda70067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:35.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.129+0000 7fda6ffff700 1 -- 172.21.15.121:0/2416508375 shutdown_connections 2023-11-15T03:10:35.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.129+0000 7fda6ffff700 1 -- 172.21.15.121:0/2416508375 wait complete. 2023-11-15T03:10:35.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.129+0000 7fda6ffff700 1 Processor -- start 2023-11-15T03:10:35.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.133+0000 7fda6ffff700 1 -- start start 2023-11-15T03:10:35.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.133+0000 7fda6ffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda700bf210 0x7fda700c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:35.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.133+0000 7fda6ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda70153980 0x7fda7014d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:35.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.133+0000 7fda6ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda70153d60 0x7fda700bfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:35.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.133+0000 7fda6ffff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda700c8fd0 con 0x7fda70153980 2023-11-15T03:10:35.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.133+0000 7fda6ffff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda700c7f50 con 0x7fda70153d60 2023-11-15T03:10:35.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.133+0000 7fda6ffff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda700c8230 con 0x7fda700bf210 2023-11-15T03:10:35.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.133+0000 7fda6e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda70153980 0x7fda7014d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:35.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.133+0000 7fda6e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda70153980 0x7fda7014d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56462/0 (socket says 172.21.15.121:56462) 2023-11-15T03:10:35.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.133+0000 7fda6e7fc700 1 -- 172.21.15.121:0/3796022090 learned_addr learned my addr 172.21.15.121:0/3796022090 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:35.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.133+0000 7fda6f7fe700 1 --2- 172.21.15.121:0/3796022090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda70153d60 0x7fda700bfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:35.143 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.133+0000 7fda6effd700 1 --2- 172.21.15.121:0/3796022090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda700bf210 0x7fda700c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:35.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.133+0000 7fda6e7fc700 1 -- 172.21.15.121:0/3796022090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda700bf210 msgr2=0x7fda700c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:35.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.133+0000 7fda6e7fc700 1 --2- 172.21.15.121:0/3796022090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda700bf210 0x7fda700c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:35.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.133+0000 7fda6e7fc700 1 -- 172.21.15.121:0/3796022090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda70153d60 msgr2=0x7fda700bfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:35.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.133+0000 7fda6e7fc700 1 --2- 172.21.15.121:0/3796022090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda70153d60 0x7fda700bfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:35.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.133+0000 7fda6e7fc700 1 -- 172.21.15.121:0/3796022090 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda640092a0 con 0x7fda70153980 2023-11-15T03:10:35.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.137+0000 7fda6e7fc700 1 --2- 172.21.15.121:0/3796022090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda70153980 0x7fda7014d070 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 rx=0x7fda700676f0 tx=0x7fda64017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:35.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.137+0000 7fda56ffd700 1 -- 172.21.15.121:0/3796022090 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6400fea0 con 0x7fda70153980 2023-11-15T03:10:35.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.137+0000 7fda56ffd700 1 -- 172.21.15.121:0/3796022090 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda64006910 con 0x7fda70153980 2023-11-15T03:10:35.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.137+0000 7fda6ffff700 1 -- 172.21.15.121:0/3796022090 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda700c8510 con 0x7fda70153980 2023-11-15T03:10:35.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.137+0000 7fda56ffd700 1 -- 172.21.15.121:0/3796022090 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6400f890 con 0x7fda70153980 2023-11-15T03:10:35.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.137+0000 7fda6ffff700 1 -- 172.21.15.121:0/3796022090 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda700c8ad0 con 0x7fda70153980 2023-11-15T03:10:35.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.137+0000 7fda6ffff700 1 -- 172.21.15.121:0/3796022090 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7006bcf0 con 0x7fda70153980 2023-11-15T03:10:35.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.141+0000 7fda56ffd700 1 -- 172.21.15.121:0/3796022090 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fda6401c630 con 0x7fda70153980 2023-11-15T03:10:35.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.141+0000 7fda56ffd700 1 --2- 172.21.15.121:0/3796022090 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fda5c05a510 0x7fda5c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:35.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.141+0000 7fda56ffd700 1 -- 172.21.15.121:0/3796022090 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9448+0+0 (secure 0 0 0) 0x7fda64076ed0 con 0x7fda70153980 2023-11-15T03:10:35.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.141+0000 7fda6effd700 1 --2- 172.21.15.121:0/3796022090 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fda5c05a510 0x7fda5c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:35.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.141+0000 7fda6effd700 1 --2- 172.21.15.121:0/3796022090 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fda5c05a510 0x7fda5c05c9d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7fda60003900 tx=0x7fda60007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:35.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.145+0000 7fda56ffd700 1 -- 172.21.15.121:0/3796022090 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fda64049970 con 0x7fda70153980 2023-11-15T03:10:35.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:35.341+0000 7fda6ffff700 1 -- 172.21.15.121:0/3796022090 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fda70068a70 con 0x7fda70153980 2023-11-15T03:10:36.314 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.309+0000 7fda56ffd700 1 -- 172.21.15.121:0/3796022090 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v322) v1 ==== 162+0+0 (secure 0 0 0) 0x7fda6402a030 con 0x7fda70153980 2023-11-15T03:10:36.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.313+0000 7fda6ffff700 1 -- 172.21.15.121:0/3796022090 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fda5c05a510 msgr2=0x7fda5c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:36.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.313+0000 7fda6ffff700 1 --2- 172.21.15.121:0/3796022090 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fda5c05a510 0x7fda5c05c9d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7fda60003900 tx=0x7fda60007d40).stop 2023-11-15T03:10:36.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.313+0000 7fda6ffff700 1 -- 172.21.15.121:0/3796022090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda70153980 msgr2=0x7fda7014d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:36.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.313+0000 7fda6ffff700 1 --2- 172.21.15.121:0/3796022090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda70153980 0x7fda7014d070 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 rx=0x7fda700676f0 tx=0x7fda64017000).stop 2023-11-15T03:10:36.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.313+0000 7fda6ffff700 1 -- 172.21.15.121:0/3796022090 shutdown_connections 2023-11-15T03:10:36.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.313+0000 7fda6ffff700 1 --2- 172.21.15.121:0/3796022090 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fda5c05a510 0x7fda5c05c9d0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:36.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.313+0000 7fda6ffff700 1 --2- 172.21.15.121:0/3796022090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda70153d60 0x7fda700bfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:36.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.313+0000 7fda6ffff700 1 --2- 172.21.15.121:0/3796022090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda70153980 0x7fda7014d070 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:36.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.313+0000 7fda6ffff700 1 --2- 172.21.15.121:0/3796022090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda700bf210 0x7fda700c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:36.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.313+0000 7fda6ffff700 1 -- 172.21.15.121:0/3796022090 >> 172.21.15.121:0/3796022090 conn(0x7fda70067a60 msgr2=0x7fda700c54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:36.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.313+0000 7fda6ffff700 1 -- 172.21.15.121:0/3796022090 shutdown_connections 2023-11-15T03:10:36.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.313+0000 7fda6ffff700 1 -- 172.21.15.121:0/3796022090 wait complete. 2023-11-15T03:10:36.324 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:10:36.342 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:10:36.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.493+0000 7f9ef481b700 1 Processor -- start 2023-11-15T03:10:36.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.497+0000 7f9ef481b700 1 -- start start 2023-11-15T03:10:36.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.497+0000 7f9ef481b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9ef0068170 0x7f9ef015ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:36.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.497+0000 7f9ef481b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef00cb5c0 con 0x7f9ef0153bb0 2023-11-15T03:10:36.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.497+0000 7f9ef481b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef00c9ca0 con 0x7f9ef00aa240 2023-11-15T03:10:36.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.497+0000 7f9ef481b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef00aa6f0 con 0x7f9ef0068170 2023-11-15T03:10:36.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.497+0000 7f9eee59c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9ef0153bb0 0x7f9ef00a9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49880/0 (socket says 172.21.15.121:49880) 2023-11-15T03:10:36.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.497+0000 7f9eee59c700 1 -- 172.21.15.121:0/469932300 learned_addr learned my addr 172.21.15.121:0/469932300 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:36.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.497+0000 7f9eeed9d700 1 --2- 172.21.15.121:0/469932300 >> v2:172.21.15.190:3300/0 conn(0x7f9ef0068170 0x7f9ef015ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:36.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.497+0000 7f9eedd9b700 1 -- 172.21.15.121:0/469932300 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755294178 0 0) 0x7f9ef00cb5c0 con 0x7f9ef0153bb0 2023-11-15T03:10:36.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.501+0000 7f9eedd9b700 1 -- 172.21.15.121:0/469932300 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9edc0033f0 con 0x7f9ef0153bb0 2023-11-15T03:10:36.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.501+0000 7f9eeed9d700 1 -- 172.21.15.121:0/469932300 >> v1:172.21.15.190:6789/0 conn(0x7f9ef00aa240 legacy=0x7f9ef0067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:36.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.501+0000 7f9eeed9d700 1 -- 172.21.15.121:0/469932300 >> v1:172.21.15.121:6789/0 conn(0x7f9ef0153bb0 legacy=0x7f9ef00a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:36.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.501+0000 7f9eeed9d700 1 -- 172.21.15.121:0/469932300 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef00caa20 con 0x7f9ef0068170 2023-11-15T03:10:36.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.501+0000 7f9eeed9d700 1 --2- 172.21.15.121:0/469932300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ef0068170 0x7f9ef015ec40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7f9ee8009e60 tx=0x7f9ee8009780).ready entity=mon.1 client_cookie=260efbb72b24123c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:36.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.501+0000 7f9eedd9b700 1 -- 172.21.15.121:0/469932300 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee800e070 con 0x7f9ef0068170 2023-11-15T03:10:36.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.501+0000 7f9eedd9b700 1 -- 172.21.15.121:0/469932300 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee800bb60 con 0x7f9ef0068170 2023-11-15T03:10:36.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.501+0000 7f9eedd9b700 1 -- 172.21.15.121:0/469932300 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8012410 con 0x7f9ef0068170 2023-11-15T03:10:36.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.501+0000 7f9ef481b700 1 -- 172.21.15.121:0/469932300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ef0068170 msgr2=0x7f9ef015ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:36.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.501+0000 7f9ef481b700 1 --2- 172.21.15.121:0/469932300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ef0068170 0x7f9ef015ec40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7f9ee8009e60 tx=0x7f9ee8009780).stop 2023-11-15T03:10:36.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.501+0000 7f9ef481b700 1 -- 172.21.15.121:0/469932300 shutdown_connections 2023-11-15T03:10:36.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.501+0000 7f9ef481b700 1 --2- 172.21.15.121:0/469932300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ef0068170 0x7f9ef015ec40 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:36.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.501+0000 7f9ef481b700 1 -- 172.21.15.121:0/469932300 >> 172.21.15.121:0/469932300 conn(0x7f9ef0058c50 msgr2=0x7f9ef0059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:36.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.501+0000 7f9ef481b700 1 -- 172.21.15.121:0/469932300 shutdown_connections 2023-11-15T03:10:36.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.501+0000 7f9ef481b700 1 -- 172.21.15.121:0/469932300 wait complete. 2023-11-15T03:10:36.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ef481b700 1 Processor -- start 2023-11-15T03:10:36.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ef481b700 1 -- start start 2023-11-15T03:10:36.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ef481b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ef0068170 0x7f9ef0152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:36.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ef481b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ef00aa240 0x7f9ef0156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:36.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ef481b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ef0153bb0 0x7f9ef0156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:36.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ef481b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef00c2db0 con 0x7f9ef00aa240 2023-11-15T03:10:36.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ef481b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef00c2920 con 0x7f9ef0068170 2023-11-15T03:10:36.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ef481b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef00c1d10 con 0x7f9ef0153bb0 2023-11-15T03:10:36.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ee7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ef00aa240 0x7f9ef0156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:36.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ee7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ef00aa240 0x7f9ef0156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56472/0 (socket says 172.21.15.121:56472) 2023-11-15T03:10:36.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ee7fff700 1 -- 172.21.15.121:0/848899846 learned_addr learned my addr 172.21.15.121:0/848899846 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:36.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9eeed9d700 1 --2- 172.21.15.121:0/848899846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ef0153bb0 0x7f9ef0156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:36.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9eee59c700 1 --2- 172.21.15.121:0/848899846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ef0068170 0x7f9ef0152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:36.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ee7fff700 1 -- 172.21.15.121:0/848899846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ef0153bb0 msgr2=0x7f9ef0156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:36.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ee7fff700 1 --2- 172.21.15.121:0/848899846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ef0153bb0 0x7f9ef0156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:36.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ee7fff700 1 -- 172.21.15.121:0/848899846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ef0068170 msgr2=0x7f9ef0152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:36.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ee7fff700 1 --2- 172.21.15.121:0/848899846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ef0068170 0x7f9ef0152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:36.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ee7fff700 1 -- 172.21.15.121:0/848899846 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee8009a20 con 0x7f9ef00aa240 2023-11-15T03:10:36.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9eeed9d700 1 --2- 172.21.15.121:0/848899846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ef0153bb0 0x7f9ef0156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:36.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ee7fff700 1 --2- 172.21.15.121:0/848899846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ef00aa240 0x7f9ef0156510 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 rx=0x7f9ed800c4c0 tx=0x7f9ed8007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:36.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ee67fc700 1 -- 172.21.15.121:0/848899846 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8007020 con 0x7f9ef00aa240 2023-11-15T03:10:36.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ee67fc700 1 -- 172.21.15.121:0/848899846 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed8007180 con 0x7f9ef00aa240 2023-11-15T03:10:36.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ef481b700 1 -- 172.21.15.121:0/848899846 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef00c1ff0 con 0x7f9ef00aa240 2023-11-15T03:10:36.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ee67fc700 1 -- 172.21.15.121:0/848899846 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8007520 con 0x7f9ef00aa240 2023-11-15T03:10:36.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.505+0000 7f9ef481b700 1 -- 172.21.15.121:0/848899846 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef00c2630 con 0x7f9ef00aa240 2023-11-15T03:10:36.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.509+0000 7f9ef481b700 1 -- 172.21.15.121:0/848899846 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef00c2ab0 con 0x7f9ef00aa240 2023-11-15T03:10:36.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.509+0000 7f9ee67fc700 1 -- 172.21.15.121:0/848899846 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9ed80068b0 con 0x7f9ef00aa240 2023-11-15T03:10:36.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.513+0000 7f9ee67fc700 1 --2- 172.21.15.121:0/848899846 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ed005a510 0x7f9ed005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:36.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.513+0000 7f9ee67fc700 1 -- 172.21.15.121:0/848899846 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9448+0+0 (secure 0 0 0) 0x7f9ed8074930 con 0x7f9ef00aa240 2023-11-15T03:10:36.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.513+0000 7f9eee59c700 1 --2- 172.21.15.121:0/848899846 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ed005a510 0x7f9ed005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:36.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.513+0000 7f9eee59c700 1 --2- 172.21.15.121:0/848899846 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ed005a510 0x7f9ed005c9d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f9ef00b00f0 tx=0x7f9ee0009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:36.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.517+0000 7f9ee67fc700 1 -- 172.21.15.121:0/848899846 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9ed8047350 con 0x7f9ef00aa240 2023-11-15T03:10:36.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.717+0000 7f9ef481b700 1 -- 172.21.15.121:0/848899846 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ef006c150 con 0x7f9ef00aa240 2023-11-15T03:10:36.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.717+0000 7f9ee67fc700 1 -- 172.21.15.121:0/848899846 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 323 v323) v1 ==== 95+0+4508 (secure 0 0 0) 0x7f9ed804a970 con 0x7f9ef00aa240 2023-11-15T03:10:36.727 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:10:36.727 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":323,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":323,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:10:36.309817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10347},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10347":{"gid":10347,"name":"a","rank":0,"incarnation":323,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/4290809693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4290809693},{"type":"v1","addr":"172.21.15.121:6836","nonce":4290809693}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:10:36.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.721+0000 7f9ef481b700 1 -- 172.21.15.121:0/848899846 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ed005a510 msgr2=0x7f9ed005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:36.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.721+0000 7f9ef481b700 1 --2- 172.21.15.121:0/848899846 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ed005a510 0x7f9ed005c9d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f9ef00b00f0 tx=0x7f9ee0009040).stop 2023-11-15T03:10:36.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.721+0000 7f9ef481b700 1 -- 172.21.15.121:0/848899846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ef00aa240 msgr2=0x7f9ef0156510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:36.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.721+0000 7f9ef481b700 1 --2- 172.21.15.121:0/848899846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ef00aa240 0x7f9ef0156510 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 rx=0x7f9ed800c4c0 tx=0x7f9ed8007dd0).stop 2023-11-15T03:10:36.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.721+0000 7f9ef481b700 1 -- 172.21.15.121:0/848899846 shutdown_connections 2023-11-15T03:10:36.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.721+0000 7f9ef481b700 1 --2- 172.21.15.121:0/848899846 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ed005a510 0x7f9ed005c9d0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:36.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.721+0000 7f9ef481b700 1 --2- 172.21.15.121:0/848899846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ef0153bb0 0x7f9ef0156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:36.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.721+0000 7f9ef481b700 1 --2- 172.21.15.121:0/848899846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ef00aa240 0x7f9ef0156510 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:36.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.721+0000 7f9ef481b700 1 --2- 172.21.15.121:0/848899846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ef0068170 0x7f9ef0152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:36.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.721+0000 7f9ef481b700 1 -- 172.21.15.121:0/848899846 >> 172.21.15.121:0/848899846 conn(0x7f9ef0058c50 msgr2=0x7f9ef00c8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:36.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.721+0000 7f9ef481b700 1 -- 172.21.15.121:0/848899846 shutdown_connections 2023-11-15T03:10:36.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:36.721+0000 7f9ef481b700 1 -- 172.21.15.121:0/848899846 wait complete. 2023-11-15T03:10:36.732 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 323 2023-11-15T03:10:36.748 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:10:39.356 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:10:39.349+0000 7f1fa4eb6700 -1 mds.0.cache.dir(0x100) Corrupt fnode in dirfrag 0x100: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input 2023-11-15T03:10:40.354 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:10:40.345+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:10:40.355 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:10:40.345+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:10:40.366 INFO:tasks.ceph.mds.a.smithi121.stderr: -22> 2023-11-15T03:10:39.349+0000 7f1fa4eb6700 -1 mds.0.cache.dir(0x100) Corrupt fnode in dirfrag 0x100: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input 2023-11-15T03:10:40.405 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:10:41.750 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:10:41.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.913+0000 7f590e907700 1 Processor -- start 2023-11-15T03:10:41.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.913+0000 7f590e907700 1 -- start start 2023-11-15T03:10:41.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.917+0000 7f590e907700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59080cafb0 0x7f5908066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:41.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.917+0000 7f590e907700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5908067270 0x7f59080ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:41.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.917+0000 7f590e907700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59080ad3a0 con 0x7f59080cabd0 2023-11-15T03:10:41.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.917+0000 7f590e907700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59080cb410 con 0x7f5908067270 2023-11-15T03:10:41.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.917+0000 7f590e907700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59080ad650 con 0x7f59080cafb0 2023-11-15T03:10:41.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.917+0000 7f590d905700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f59080cabd0 0x7f59080acc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49886/0 (socket says 172.21.15.121:49886) 2023-11-15T03:10:41.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.917+0000 7f590d905700 1 -- 172.21.15.121:0/1377379200 learned_addr learned my addr 172.21.15.121:0/1377379200 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:41.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.917+0000 7f590d104700 1 --2- 172.21.15.121:0/1377379200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59080cafb0 0x7f5908066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:41.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.917+0000 7f590e106700 1 --2- 172.21.15.121:0/1377379200 >> v2:172.21.15.121:3300/0 conn(0x7f5908067270 0x7f59080ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:41.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.921+0000 7f590c903700 1 -- 172.21.15.121:0/1377379200 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175395282 0 0) 0x7f59080ad3a0 con 0x7f59080cabd0 2023-11-15T03:10:41.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.921+0000 7f590c903700 1 -- 172.21.15.121:0/1377379200 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58f40033f0 con 0x7f59080cabd0 2023-11-15T03:10:41.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.921+0000 7f590d104700 1 -- 172.21.15.121:0/1377379200 >> v2:172.21.15.121:3300/0 conn(0x7f5908067270 msgr2=0x7f59080ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:41.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.921+0000 7f590d104700 1 --2- 172.21.15.121:0/1377379200 >> v2:172.21.15.121:3300/0 conn(0x7f5908067270 0x7f59080ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:41.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.921+0000 7f590d104700 1 -- 172.21.15.121:0/1377379200 >> v1:172.21.15.121:6789/0 conn(0x7f59080cabd0 legacy=0x7f59080acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:41.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.921+0000 7f590d104700 1 -- 172.21.15.121:0/1377379200 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5908065fc0 con 0x7f59080cafb0 2023-11-15T03:10:41.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.921+0000 7f590e106700 1 --2- 172.21.15.121:0/1377379200 >> v2:172.21.15.121:3300/0 conn(0x7f5908067270 0x7f59080ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:10:41.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.921+0000 7f590d104700 1 --2- 172.21.15.121:0/1377379200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59080cafb0 0x7f5908066d80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f58f8009760 tx=0x7f58f8009060).ready entity=mon.2 client_cookie=4db5701b4ed1d0c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:41.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.921+0000 7f590c903700 1 -- 172.21.15.121:0/1377379200 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f800d4a0 con 0x7f59080cafb0 2023-11-15T03:10:41.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.921+0000 7f590c903700 1 -- 172.21.15.121:0/1377379200 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f800d600 con 0x7f59080cafb0 2023-11-15T03:10:41.931 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.921+0000 7f590c903700 1 -- 172.21.15.121:0/1377379200 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f800d980 con 0x7f59080cafb0 2023-11-15T03:10:41.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.921+0000 7f590e907700 1 -- 172.21.15.121:0/1377379200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59080cafb0 msgr2=0x7f5908066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:41.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.921+0000 7f590e907700 1 --2- 172.21.15.121:0/1377379200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59080cafb0 0x7f5908066d80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f58f8009760 tx=0x7f58f8009060).stop 2023-11-15T03:10:41.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.921+0000 7f590e907700 1 -- 172.21.15.121:0/1377379200 shutdown_connections 2023-11-15T03:10:41.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.921+0000 7f590e907700 1 --2- 172.21.15.121:0/1377379200 >> v2:172.21.15.121:3300/0 conn(0x7f5908067270 0x7f59080ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:41.932 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.921+0000 7f590e907700 1 --2- 172.21.15.121:0/1377379200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59080cafb0 0x7f5908066d80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:41.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.921+0000 7f590e907700 1 -- 172.21.15.121:0/1377379200 >> 172.21.15.121:0/1377379200 conn(0x7f5908062190 msgr2=0x7f59080625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:41.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.921+0000 7f590e907700 1 -- 172.21.15.121:0/1377379200 shutdown_connections 2023-11-15T03:10:41.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.921+0000 7f590e907700 1 -- 172.21.15.121:0/1377379200 wait complete. 2023-11-15T03:10:41.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.925+0000 7f590e907700 1 Processor -- start 2023-11-15T03:10:41.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.925+0000 7f590e907700 1 -- start start 2023-11-15T03:10:41.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590e907700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5908067270 0x7f590814f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:41.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590e907700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59080cabd0 0x7f5908155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:41.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590e907700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59080cafb0 0x7f59080c0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:41.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590e907700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5908155cf0 con 0x7f59080cabd0 2023-11-15T03:10:41.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590e907700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5908155fd0 con 0x7f5908067270 2023-11-15T03:10:41.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590e907700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59081562b0 con 0x7f59080cafb0 2023-11-15T03:10:41.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590d104700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59080cabd0 0x7f5908155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:41.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590d104700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59080cabd0 0x7f5908155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56504/0 (socket says 172.21.15.121:56504) 2023-11-15T03:10:41.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590d104700 1 -- 172.21.15.121:0/1957431036 learned_addr learned my addr 172.21.15.121:0/1957431036 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:41.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590e106700 1 --2- 172.21.15.121:0/1957431036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59080cafb0 0x7f59080c0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:41.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590d905700 1 --2- 172.21.15.121:0/1957431036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5908067270 0x7f590814f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:41.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590d104700 1 -- 172.21.15.121:0/1957431036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59080cafb0 msgr2=0x7f59080c0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:41.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590d104700 1 --2- 172.21.15.121:0/1957431036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59080cafb0 0x7f59080c0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:41.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590d104700 1 -- 172.21.15.121:0/1957431036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5908067270 msgr2=0x7f590814f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590d104700 1 --2- 172.21.15.121:0/1957431036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5908067270 0x7f590814f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590d104700 1 -- 172.21.15.121:0/1957431036 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5900009a20 con 0x7f59080cabd0 2023-11-15T03:10:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590e106700 1 --2- 172.21.15.121:0/1957431036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59080cafb0 0x7f59080c0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:10:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590d104700 1 --2- 172.21.15.121:0/1957431036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59080cabd0 0x7f5908155310 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 rx=0x7f58f8009d50 tx=0x7f58f8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f58fdffb700 1 -- 172.21.15.121:0/1957431036 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f800dd30 con 0x7f59080cabd0 2023-11-15T03:10:41.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f58fdffb700 1 -- 172.21.15.121:0/1957431036 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f800de90 con 0x7f59080cabd0 2023-11-15T03:10:41.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f58fdffb700 1 -- 172.21.15.121:0/1957431036 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f800d980 con 0x7f59080cabd0 2023-11-15T03:10:41.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590e907700 1 -- 172.21.15.121:0/1957431036 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f80092a0 con 0x7f59080cabd0 2023-11-15T03:10:41.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.929+0000 7f590e907700 1 -- 172.21.15.121:0/1957431036 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59080c25f0 con 0x7f59080cabd0 2023-11-15T03:10:41.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.933+0000 7f590e907700 1 -- 172.21.15.121:0/1957431036 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59080c2170 con 0x7f59080cabd0 2023-11-15T03:10:41.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.933+0000 7f58fdffb700 1 -- 172.21.15.121:0/1957431036 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f58f801d640 con 0x7f59080cabd0 2023-11-15T03:10:41.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.937+0000 7f58fdffb700 1 --2- 172.21.15.121:0/1957431036 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58ec05a510 0x7f58ec05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:41.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.937+0000 7f58fdffb700 1 -- 172.21.15.121:0/1957431036 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f58f8076ca0 con 0x7f59080cabd0 2023-11-15T03:10:41.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.937+0000 7f590d905700 1 --2- 172.21.15.121:0/1957431036 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58ec05a510 0x7f58ec05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:41.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.937+0000 7f590d905700 1 --2- 172.21.15.121:0/1957431036 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58ec05a510 0x7f58ec05c9d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7f5904005750 tx=0x7f5904009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:41.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:41.941+0000 7f58fdffb700 1 -- 172.21.15.121:0/1957431036 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f59080c2170 con 0x7f59080cabd0 2023-11-15T03:10:42.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.145+0000 7f590e907700 1 -- 172.21.15.121:0/1957431036 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59080c2170 con 0x7f59080cabd0 2023-11-15T03:10:42.155 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.149+0000 7f58fdffb700 1 -- 172.21.15.121:0/1957431036 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 328 v328) v1 ==== 95+0+4497 (secure 0 0 0) 0x7f59080c2170 con 0x7f59080cabd0 2023-11-15T03:10:42.155 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:10:42.156 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":328,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":10386,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2269363825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2269363825},{"type":"v1","addr":"172.21.15.121:6836","nonce":2269363825}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":328}],"filesystems":[{"mdsmap":{"epoch":327,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:10:40.349039+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:10:42.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.153+0000 7f590e907700 1 -- 172.21.15.121:0/1957431036 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58ec05a510 msgr2=0x7f58ec05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:42.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.153+0000 7f590e907700 1 --2- 172.21.15.121:0/1957431036 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58ec05a510 0x7f58ec05c9d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7f5904005750 tx=0x7f5904009040).stop 2023-11-15T03:10:42.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.153+0000 7f590e907700 1 -- 172.21.15.121:0/1957431036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59080cabd0 msgr2=0x7f5908155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:42.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.153+0000 7f590e907700 1 --2- 172.21.15.121:0/1957431036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59080cabd0 0x7f5908155310 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 rx=0x7f58f8009d50 tx=0x7f58f8016000).stop 2023-11-15T03:10:42.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.153+0000 7f590e907700 1 -- 172.21.15.121:0/1957431036 shutdown_connections 2023-11-15T03:10:42.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.153+0000 7f590e907700 1 --2- 172.21.15.121:0/1957431036 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f58ec05a510 0x7f58ec05c9d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:42.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.153+0000 7f590e907700 1 --2- 172.21.15.121:0/1957431036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59080cafb0 0x7f59080c0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:42.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.153+0000 7f590e907700 1 --2- 172.21.15.121:0/1957431036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59080cabd0 0x7f5908155310 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:42.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.153+0000 7f590e907700 1 --2- 172.21.15.121:0/1957431036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5908067270 0x7f590814f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:42.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.153+0000 7f590e907700 1 -- 172.21.15.121:0/1957431036 >> 172.21.15.121:0/1957431036 conn(0x7f5908062190 msgr2=0x7f59080ae190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:42.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.153+0000 7f590e907700 1 -- 172.21.15.121:0/1957431036 shutdown_connections 2023-11-15T03:10:42.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.153+0000 7f590e907700 1 -- 172.21.15.121:0/1957431036 wait complete. 2023-11-15T03:10:42.163 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 328 2023-11-15T03:10:42.180 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:10:42.181 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:10:42.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.337+0000 7f144c81b700 1 Processor -- start 2023-11-15T03:10:42.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.337+0000 7f144c81b700 1 -- start start 2023-11-15T03:10:42.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.341+0000 7f144c81b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f14481539d0 0x7f14480c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:42.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.341+0000 7f144c81b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14480c2670 0x7f14480c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:42.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.341+0000 7f144c81b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14480c1530 con 0x7f1448153db0 2023-11-15T03:10:42.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.341+0000 7f144c81b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14481541f0 con 0x7f14481539d0 2023-11-15T03:10:42.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.341+0000 7f144c81b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14480c2b00 con 0x7f14480c2670 2023-11-15T03:10:42.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.341+0000 7f144659c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f14481539d0 0x7f14480c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:42.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.341+0000 7f144659c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f14481539d0 0x7f14480c06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56518/0 (socket says 172.21.15.121:56518) 2023-11-15T03:10:42.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.341+0000 7f144659c700 1 -- 172.21.15.121:0/1151545302 learned_addr learned my addr 172.21.15.121:0/1151545302 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:42.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.341+0000 7f1446d9d700 1 --2- 172.21.15.121:0/1151545302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14480c2670 0x7f14480c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:42.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.341+0000 7f144659c700 1 -- 172.21.15.121:0/1151545302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14480c2670 msgr2=0x7f14480c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:42.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.341+0000 7f144659c700 1 --2- 172.21.15.121:0/1151545302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14480c2670 0x7f14480c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:42.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.341+0000 7f144659c700 1 -- 172.21.15.121:0/1151545302 >> v1:172.21.15.190:6789/0 conn(0x7f1448153db0 legacy=0x7f14480c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:42.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.341+0000 7f144659c700 1 -- 172.21.15.121:0/1151545302 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14480c2d80 con 0x7f14481539d0 2023-11-15T03:10:42.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.341+0000 7f1446d9d700 1 --2- 172.21.15.121:0/1151545302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14480c2670 0x7f14480c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:10:42.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.345+0000 7f144659c700 1 --2- 172.21.15.121:0/1151545302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14481539d0 0x7f14480c06e0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 rx=0x7f1438006fd0 tx=0x7f14380090f0).ready entity=mon.0 client_cookie=ed74a5a8d481238f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:42.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.345+0000 7f1445d9b700 1 -- 172.21.15.121:0/1151545302 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143800b600 con 0x7f14481539d0 2023-11-15T03:10:42.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.345+0000 7f1445d9b700 1 -- 172.21.15.121:0/1151545302 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143800b760 con 0x7f14481539d0 2023-11-15T03:10:42.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.345+0000 7f1445d9b700 1 -- 172.21.15.121:0/1151545302 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143800bae0 con 0x7f14481539d0 2023-11-15T03:10:42.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.345+0000 7f144c81b700 1 -- 172.21.15.121:0/1151545302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14481539d0 msgr2=0x7f14480c06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:42.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.345+0000 7f144c81b700 1 --2- 172.21.15.121:0/1151545302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14481539d0 0x7f14480c06e0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 rx=0x7f1438006fd0 tx=0x7f14380090f0).stop 2023-11-15T03:10:42.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.345+0000 7f144c81b700 1 -- 172.21.15.121:0/1151545302 shutdown_connections 2023-11-15T03:10:42.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.345+0000 7f144c81b700 1 --2- 172.21.15.121:0/1151545302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14480c2670 0x7f14480c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:42.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.345+0000 7f144c81b700 1 --2- 172.21.15.121:0/1151545302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14481539d0 0x7f14480c06e0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:42.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.345+0000 7f144c81b700 1 -- 172.21.15.121:0/1151545302 >> 172.21.15.121:0/1151545302 conn(0x7f1448062190 msgr2=0x7f14480625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:42.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.345+0000 7f144c81b700 1 -- 172.21.15.121:0/1151545302 shutdown_connections 2023-11-15T03:10:42.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.345+0000 7f144c81b700 1 -- 172.21.15.121:0/1151545302 wait complete. 2023-11-15T03:10:42.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f144c81b700 1 Processor -- start 2023-11-15T03:10:42.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f144c81b700 1 -- start start 2023-11-15T03:10:42.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f144c81b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14480c2670 0x7f144814cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:42.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f144c81b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14481539d0 0x7f14480bc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:42.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f144c81b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1448153db0 0x7f14480bf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:42.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f144c81b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14480bee40 con 0x7f1448153db0 2023-11-15T03:10:42.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f144c81b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14480be230 con 0x7f14480c2670 2023-11-15T03:10:42.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f144c81b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14480be510 con 0x7f14481539d0 2023-11-15T03:10:42.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f1446d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1448153db0 0x7f14480bf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:42.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f144659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14480c2670 0x7f144814cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:42.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f1446d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1448153db0 0x7f14480bf2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56528/0 (socket says 172.21.15.121:56528) 2023-11-15T03:10:42.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f1446d9d700 1 -- 172.21.15.121:0/3592892353 learned_addr learned my addr 172.21.15.121:0/3592892353 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:42.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f143ffff700 1 --2- 172.21.15.121:0/3592892353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14481539d0 0x7f14480bc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:42.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f1446d9d700 1 -- 172.21.15.121:0/3592892353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14481539d0 msgr2=0x7f14480bc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:42.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f1446d9d700 1 --2- 172.21.15.121:0/3592892353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14481539d0 0x7f14480bc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:42.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f1446d9d700 1 -- 172.21.15.121:0/3592892353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14480c2670 msgr2=0x7f144814cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:42.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f1446d9d700 1 --2- 172.21.15.121:0/3592892353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14480c2670 0x7f144814cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:42.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f1446d9d700 1 -- 172.21.15.121:0/3592892353 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1438009370 con 0x7f1448153db0 2023-11-15T03:10:42.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f1446d9d700 1 --2- 172.21.15.121:0/3592892353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1448153db0 0x7f14480bf2d0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 rx=0x7f144000cef0 tx=0x7f1440007860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:42.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.349+0000 7f143e7fc700 1 -- 172.21.15.121:0/3592892353 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144000add0 con 0x7f1448153db0 2023-11-15T03:10:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.353+0000 7f143e7fc700 1 -- 172.21.15.121:0/3592892353 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1440016400 con 0x7f1448153db0 2023-11-15T03:10:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.353+0000 7f144c81b700 1 -- 172.21.15.121:0/3592892353 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14480be7f0 con 0x7f1448153db0 2023-11-15T03:10:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.353+0000 7f143e7fc700 1 -- 172.21.15.121:0/3592892353 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1440016780 con 0x7f1448153db0 2023-11-15T03:10:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.353+0000 7f144c81b700 1 -- 172.21.15.121:0/3592892353 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14480bc8b0 con 0x7f1448153db0 2023-11-15T03:10:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.353+0000 7f144c81b700 1 -- 172.21.15.121:0/3592892353 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14480beff0 con 0x7f1448153db0 2023-11-15T03:10:42.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.353+0000 7f143e7fc700 1 -- 172.21.15.121:0/3592892353 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f14400203f0 con 0x7f1448153db0 2023-11-15T03:10:42.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.357+0000 7f143e7fc700 1 --2- 172.21.15.121:0/3592892353 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f142805a510 0x7f142805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:42.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.357+0000 7f143e7fc700 1 -- 172.21.15.121:0/3592892353 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f1440075f70 con 0x7f1448153db0 2023-11-15T03:10:42.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.357+0000 7f144659c700 1 --2- 172.21.15.121:0/3592892353 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f142805a510 0x7f142805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:42.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.357+0000 7f144659c700 1 --2- 172.21.15.121:0/3592892353 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f142805a510 0x7f142805c9d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7f1438009130 tx=0x7f1438017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:42.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.361+0000 7f143e7fc700 1 -- 172.21.15.121:0/3592892353 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f14480beff0 con 0x7f1448153db0 2023-11-15T03:10:42.516 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:10:42.524 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:10:42.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.557+0000 7f144c81b700 1 -- 172.21.15.121:0/3592892353 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f144806c150 con 0x7f1448153db0 2023-11-15T03:10:42.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.557+0000 7f143e7fc700 1 -- 172.21.15.121:0/3592892353 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 328 v328) v1 ==== 95+0+4497 (secure 0 0 0) 0x7f14400488e0 con 0x7f1448153db0 2023-11-15T03:10:42.565 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:10:42.566 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":328,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":10386,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2269363825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2269363825},{"type":"v1","addr":"172.21.15.121:6836","nonce":2269363825}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":328}],"filesystems":[{"mdsmap":{"epoch":327,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:10:40.349039+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:10:42.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.561+0000 7f144c81b700 1 -- 172.21.15.121:0/3592892353 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f142805a510 msgr2=0x7f142805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:42.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.561+0000 7f144c81b700 1 --2- 172.21.15.121:0/3592892353 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f142805a510 0x7f142805c9d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7f1438009130 tx=0x7f1438017000).stop 2023-11-15T03:10:42.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.561+0000 7f144c81b700 1 -- 172.21.15.121:0/3592892353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1448153db0 msgr2=0x7f14480bf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:42.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.561+0000 7f144c81b700 1 --2- 172.21.15.121:0/3592892353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1448153db0 0x7f14480bf2d0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 rx=0x7f144000cef0 tx=0x7f1440007860).stop 2023-11-15T03:10:42.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.561+0000 7f144c81b700 1 -- 172.21.15.121:0/3592892353 shutdown_connections 2023-11-15T03:10:42.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.561+0000 7f144c81b700 1 --2- 172.21.15.121:0/3592892353 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f142805a510 0x7f142805c9d0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:42.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.561+0000 7f144c81b700 1 --2- 172.21.15.121:0/3592892353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1448153db0 0x7f14480bf2d0 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:42.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.561+0000 7f144c81b700 1 --2- 172.21.15.121:0/3592892353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14481539d0 0x7f14480bc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:42.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.561+0000 7f144c81b700 1 --2- 172.21.15.121:0/3592892353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14480c2670 0x7f144814cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:42.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.561+0000 7f144c81b700 1 -- 172.21.15.121:0/3592892353 >> 172.21.15.121:0/3592892353 conn(0x7f1448062190 msgr2=0x7f1448066a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:42.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.561+0000 7f144c81b700 1 -- 172.21.15.121:0/3592892353 shutdown_connections 2023-11-15T03:10:42.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.561+0000 7f144c81b700 1 -- 172.21.15.121:0/3592892353 wait complete. 2023-11-15T03:10:42.573 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 328 2023-11-15T03:10:42.589 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:10:42.589 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:10:42.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.793+0000 7f4d8359e700 1 Processor -- start 2023-11-15T03:10:42.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.793+0000 7f4d8359e700 1 -- start start 2023-11-15T03:10:42.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.793+0000 7f4d8359e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4d840c37f0 0x7f4d840a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:42.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.793+0000 7f4d8359e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d840657f0 con 0x7f4d84151870 2023-11-15T03:10:42.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.793+0000 7f4d8359e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d84065ad0 con 0x7f4d840a9830 2023-11-15T03:10:42.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.793+0000 7f4d8359e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d840aa260 con 0x7f4d840c37f0 2023-11-15T03:10:42.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.793+0000 7f4d8259c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4d84151870 0x7f4d84151c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49894/0 (socket says 172.21.15.121:49894) 2023-11-15T03:10:42.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.793+0000 7f4d8259c700 1 -- 172.21.15.121:0/3378099356 learned_addr learned my addr 172.21.15.121:0/3378099356 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:42.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.793+0000 7f4d82d9d700 1 --2- 172.21.15.121:0/3378099356 >> v2:172.21.15.190:3300/0 conn(0x7f4d840c37f0 0x7f4d840a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:42.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.797+0000 7f4d8159a700 1 -- 172.21.15.121:0/3378099356 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3259692191 0 0) 0x7f4d840657f0 con 0x7f4d84151870 2023-11-15T03:10:42.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.797+0000 7f4d82d9d700 1 -- 172.21.15.121:0/3378099356 >> v1:172.21.15.190:6789/0 conn(0x7f4d840a9830 legacy=0x7f4d840c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:42.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.797+0000 7f4d82d9d700 1 -- 172.21.15.121:0/3378099356 >> v1:172.21.15.121:6789/0 conn(0x7f4d84151870 legacy=0x7f4d84151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:42.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.797+0000 7f4d82d9d700 1 -- 172.21.15.121:0/3378099356 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d840aa540 con 0x7f4d840c37f0 2023-11-15T03:10:42.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.797+0000 7f4d82d9d700 1 --2- 172.21.15.121:0/3378099356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d840c37f0 0x7f4d840a9c70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f4d7c009e60 tx=0x7f4d7c009780).ready entity=mon.1 client_cookie=c381fe16089c0d64 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:42.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.797+0000 7f4d8159a700 1 -- 172.21.15.121:0/3378099356 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c00e070 con 0x7f4d840c37f0 2023-11-15T03:10:42.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.797+0000 7f4d8159a700 1 -- 172.21.15.121:0/3378099356 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7c00bc30 con 0x7f4d840c37f0 2023-11-15T03:10:42.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.797+0000 7f4d8159a700 1 -- 172.21.15.121:0/3378099356 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c012410 con 0x7f4d840c37f0 2023-11-15T03:10:42.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.797+0000 7f4d8359e700 1 -- 172.21.15.121:0/3378099356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d840c37f0 msgr2=0x7f4d840a9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:42.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.797+0000 7f4d8359e700 1 --2- 172.21.15.121:0/3378099356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d840c37f0 0x7f4d840a9c70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f4d7c009e60 tx=0x7f4d7c009780).stop 2023-11-15T03:10:42.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.797+0000 7f4d8359e700 1 -- 172.21.15.121:0/3378099356 shutdown_connections 2023-11-15T03:10:42.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.797+0000 7f4d8359e700 1 --2- 172.21.15.121:0/3378099356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d840c37f0 0x7f4d840a9c70 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:42.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.797+0000 7f4d8359e700 1 -- 172.21.15.121:0/3378099356 >> 172.21.15.121:0/3378099356 conn(0x7f4d84062190 msgr2=0x7f4d84062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:42.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.797+0000 7f4d8359e700 1 -- 172.21.15.121:0/3378099356 shutdown_connections 2023-11-15T03:10:42.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.797+0000 7f4d8359e700 1 -- 172.21.15.121:0/3378099356 wait complete. 2023-11-15T03:10:42.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d8359e700 1 Processor -- start 2023-11-15T03:10:42.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d8359e700 1 -- start start 2023-11-15T03:10:42.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d8359e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d840a9830 0x7f4d84149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:42.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d8359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d840c37f0 0x7f4d8414a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:42.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d8359e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d84151870 0x7f4d840c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:42.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d8359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840c5890 con 0x7f4d840c37f0 2023-11-15T03:10:42.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d8359e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8414a5e0 con 0x7f4d840a9830 2023-11-15T03:10:42.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d8359e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840c52d0 con 0x7f4d84151870 2023-11-15T03:10:42.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d81d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d840c37f0 0x7f4d8414a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:42.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d81d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d840c37f0 0x7f4d8414a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56544/0 (socket says 172.21.15.121:56544) 2023-11-15T03:10:42.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d81d9b700 1 -- 172.21.15.121:0/2212763468 learned_addr learned my addr 172.21.15.121:0/2212763468 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:42.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d8259c700 1 --2- 172.21.15.121:0/2212763468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d840a9830 0x7f4d84149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:42.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d82d9d700 1 --2- 172.21.15.121:0/2212763468 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d84151870 0x7f4d840c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:42.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d81d9b700 1 -- 172.21.15.121:0/2212763468 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d84151870 msgr2=0x7f4d840c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:42.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d81d9b700 1 --2- 172.21.15.121:0/2212763468 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d84151870 0x7f4d840c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:42.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d81d9b700 1 -- 172.21.15.121:0/2212763468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d840a9830 msgr2=0x7f4d84149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:42.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d81d9b700 1 --2- 172.21.15.121:0/2212763468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d840a9830 0x7f4d84149c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:42.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d81d9b700 1 -- 172.21.15.121:0/2212763468 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7c009a20 con 0x7f4d840c37f0 2023-11-15T03:10:42.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d8259c700 1 --2- 172.21.15.121:0/2212763468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d840a9830 0x7f4d84149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:42.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.801+0000 7f4d81d9b700 1 --2- 172.21.15.121:0/2212763468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d840c37f0 0x7f4d8414a0f0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 rx=0x7f4d7800c4f0 tx=0x7f4d78007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:42.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.805+0000 7f4d727fc700 1 -- 172.21.15.121:0/2212763468 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78007020 con 0x7f4d840c37f0 2023-11-15T03:10:42.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.805+0000 7f4d727fc700 1 -- 172.21.15.121:0/2212763468 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d78001e90 con 0x7f4d840c37f0 2023-11-15T03:10:42.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.805+0000 7f4d8359e700 1 -- 172.21.15.121:0/2212763468 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d840c55b0 con 0x7f4d840c37f0 2023-11-15T03:10:42.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.805+0000 7f4d727fc700 1 -- 172.21.15.121:0/2212763468 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d780073b0 con 0x7f4d840c37f0 2023-11-15T03:10:42.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.805+0000 7f4d8359e700 1 -- 172.21.15.121:0/2212763468 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d840c4ea0 con 0x7f4d840c37f0 2023-11-15T03:10:42.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.805+0000 7f4d8359e700 1 -- 172.21.15.121:0/2212763468 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d84151f00 con 0x7f4d840c37f0 2023-11-15T03:10:42.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.805+0000 7f4d727fc700 1 -- 172.21.15.121:0/2212763468 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4d780068b0 con 0x7f4d840c37f0 2023-11-15T03:10:42.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.809+0000 7f4d727fc700 1 --2- 172.21.15.121:0/2212763468 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4d6405a510 0x7f4d6405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:42.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.809+0000 7f4d727fc700 1 -- 172.21.15.121:0/2212763468 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f4d78074f20 con 0x7f4d840c37f0 2023-11-15T03:10:42.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.809+0000 7f4d8259c700 1 --2- 172.21.15.121:0/2212763468 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4d6405a510 0x7f4d6405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:42.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.809+0000 7f4d8259c700 1 --2- 172.21.15.121:0/2212763468 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4d6405a510 0x7f4d6405c9d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7f4d74000c00 tx=0x7f4d74009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:42.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:42.813+0000 7f4d727fc700 1 -- 172.21.15.121:0/2212763468 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4d780478f0 con 0x7f4d840c37f0 2023-11-15T03:10:43.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.005+0000 7f4d8359e700 1 -- 172.21.15.121:0/2212763468 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d8406bcf0 con 0x7f4d840c37f0 2023-11-15T03:10:43.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.009+0000 7f4d727fc700 1 -- 172.21.15.121:0/2212763468 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 328 v328) v1 ==== 95+0+4497 (secure 0 0 0) 0x7f4d7804af10 con 0x7f4d840c37f0 2023-11-15T03:10:43.016 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:10:43.016 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":328,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":10386,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2269363825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2269363825},{"type":"v1","addr":"172.21.15.121:6836","nonce":2269363825}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":328}],"filesystems":[{"mdsmap":{"epoch":327,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:10:40.349039+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:10:43.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.013+0000 7f4d8359e700 1 -- 172.21.15.121:0/2212763468 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4d6405a510 msgr2=0x7f4d6405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:43.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.013+0000 7f4d8359e700 1 --2- 172.21.15.121:0/2212763468 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4d6405a510 0x7f4d6405c9d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7f4d74000c00 tx=0x7f4d74009040).stop 2023-11-15T03:10:43.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.013+0000 7f4d8359e700 1 -- 172.21.15.121:0/2212763468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d840c37f0 msgr2=0x7f4d8414a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:43.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.013+0000 7f4d8359e700 1 --2- 172.21.15.121:0/2212763468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d840c37f0 0x7f4d8414a0f0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 rx=0x7f4d7800c4f0 tx=0x7f4d78007e00).stop 2023-11-15T03:10:43.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.013+0000 7f4d8359e700 1 -- 172.21.15.121:0/2212763468 shutdown_connections 2023-11-15T03:10:43.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.013+0000 7f4d8359e700 1 --2- 172.21.15.121:0/2212763468 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4d6405a510 0x7f4d6405c9d0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:43.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.013+0000 7f4d8359e700 1 --2- 172.21.15.121:0/2212763468 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d84151870 0x7f4d840c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:43.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.013+0000 7f4d8359e700 1 --2- 172.21.15.121:0/2212763468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d840c37f0 0x7f4d8414a0f0 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:43.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.013+0000 7f4d8359e700 1 --2- 172.21.15.121:0/2212763468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d840a9830 0x7f4d84149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:43.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.013+0000 7f4d8359e700 1 -- 172.21.15.121:0/2212763468 >> 172.21.15.121:0/2212763468 conn(0x7f4d84062190 msgr2=0x7f4d841508b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:43.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.013+0000 7f4d8359e700 1 -- 172.21.15.121:0/2212763468 shutdown_connections 2023-11-15T03:10:43.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.013+0000 7f4d8359e700 1 -- 172.21.15.121:0/2212763468 wait complete. 2023-11-15T03:10:43.023 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 328 2023-11-15T03:10:43.039 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 100.00000000' led to DAMAGED state 2023-11-15T03:10:43.040 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on mds0_sessionmap' 2023-11-15T03:10:43.040 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:10:43.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:43.041 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:10:43.062 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:10:43.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:10:43.064 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:10:43.283 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.277+0000 7fe7a28b9700 1 Processor -- start 2023-11-15T03:10:43.284 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.277+0000 7fe7a28b9700 1 -- start start 2023-11-15T03:10:43.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.281+0000 7fe7a28b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe79c0c35d0 0x7fe79c0bb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:43.287 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.281+0000 7fe7a28b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe79c0a8b20 con 0x7fe79c153d70 2023-11-15T03:10:43.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.281+0000 7fe7a28b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe79c0c1760 con 0x7fe79c153990 2023-11-15T03:10:43.288 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.281+0000 7fe7a28b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe79c0a8d50 con 0x7fe79c0c35d0 2023-11-15T03:10:43.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.281+0000 7fe7a10b6700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe79c153d70 0x7fe79c0c2ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49896/0 (socket says 172.21.15.121:49896) 2023-11-15T03:10:43.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.281+0000 7fe7a10b6700 1 -- 172.21.15.121:0/1704706919 learned_addr learned my addr 172.21.15.121:0/1704706919 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:43.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.281+0000 7fe7a20b8700 1 --2- 172.21.15.121:0/1704706919 >> v2:172.21.15.190:3300/0 conn(0x7fe79c0c35d0 0x7fe79c0bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:43.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.281+0000 7fe7a08b5700 1 -- 172.21.15.121:0/1704706919 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063339300 0 0) 0x7fe79c0a8b20 con 0x7fe79c153d70 2023-11-15T03:10:43.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.281+0000 7fe7a08b5700 1 -- 172.21.15.121:0/1704706919 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7880033f0 con 0x7fe79c153d70 2023-11-15T03:10:43.290 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.281+0000 7fe7a08b5700 1 -- 172.21.15.121:0/1704706919 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379246605 0 0) 0x7fe79c0c1760 con 0x7fe79c153990 2023-11-15T03:10:43.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.281+0000 7fe7a08b5700 1 -- 172.21.15.121:0/1704706919 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe79c0a8b20 con 0x7fe79c153990 2023-11-15T03:10:43.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.281+0000 7fe7a20b8700 1 -- 172.21.15.121:0/1704706919 >> v1:172.21.15.190:6789/0 conn(0x7fe79c153990 legacy=0x7fe79c1581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:43.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.281+0000 7fe7a20b8700 1 -- 172.21.15.121:0/1704706919 >> v1:172.21.15.121:6789/0 conn(0x7fe79c153d70 legacy=0x7fe79c0c2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:43.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.281+0000 7fe7a08b5700 1 -- 172.21.15.121:0/1704706919 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3210220356 0 0) 0x7fe7880033f0 con 0x7fe79c153d70 2023-11-15T03:10:43.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.281+0000 7fe7a20b8700 1 -- 172.21.15.121:0/1704706919 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79c0a9030 con 0x7fe79c0c35d0 2023-11-15T03:10:43.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.281+0000 7fe7a20b8700 1 --2- 172.21.15.121:0/1704706919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe79c0c35d0 0x7fe79c0bb910 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7fe794009e60 tx=0x7fe794009780).ready entity=mon.1 client_cookie=54811cc47f0c3b94 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:43.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.281+0000 7fe7a08b5700 1 -- 172.21.15.121:0/1704706919 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79400e070 con 0x7fe79c0c35d0 2023-11-15T03:10:43.293 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.285+0000 7fe7a08b5700 1 -- 172.21.15.121:0/1704706919 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79400bab0 con 0x7fe79c0c35d0 2023-11-15T03:10:43.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.285+0000 7fe7a08b5700 1 -- 172.21.15.121:0/1704706919 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79400be30 con 0x7fe79c0c35d0 2023-11-15T03:10:43.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.285+0000 7fe7a28b9700 1 -- 172.21.15.121:0/1704706919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe79c0c35d0 msgr2=0x7fe79c0bb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:43.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.285+0000 7fe7a28b9700 1 --2- 172.21.15.121:0/1704706919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe79c0c35d0 0x7fe79c0bb910 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7fe794009e60 tx=0x7fe794009780).stop 2023-11-15T03:10:43.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.285+0000 7fe7a28b9700 1 -- 172.21.15.121:0/1704706919 shutdown_connections 2023-11-15T03:10:43.294 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.285+0000 7fe7a28b9700 1 --2- 172.21.15.121:0/1704706919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe79c0c35d0 0x7fe79c0bb910 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:43.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.285+0000 7fe7a28b9700 1 -- 172.21.15.121:0/1704706919 >> 172.21.15.121:0/1704706919 conn(0x7fe79c067a60 msgr2=0x7fe79c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:43.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.285+0000 7fe7a28b9700 1 -- 172.21.15.121:0/1704706919 shutdown_connections 2023-11-15T03:10:43.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.285+0000 7fe7a28b9700 1 -- 172.21.15.121:0/1704706919 wait complete. 2023-11-15T03:10:43.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.285+0000 7fe7a28b9700 1 Processor -- start 2023-11-15T03:10:43.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.285+0000 7fe7a28b9700 1 -- start start 2023-11-15T03:10:43.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe7a28b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe79c0c35d0 0x7fe79c14b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:43.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe7a28b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe79c153990 0x7fe79c0bf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:43.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe7a28b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe79c153d70 0x7fe79c0bfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:43.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe7a28b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c0beff0 con 0x7fe79c153d70 2023-11-15T03:10:43.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe7a28b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c0c0440 con 0x7fe79c0c35d0 2023-11-15T03:10:43.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe7a28b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c154c60 con 0x7fe79c153990 2023-11-15T03:10:43.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe7a20b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe79c153d70 0x7fe79c0bfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:43.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe7a20b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe79c153d70 0x7fe79c0bfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56546/0 (socket says 172.21.15.121:56546) 2023-11-15T03:10:43.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe7a20b8700 1 -- 172.21.15.121:0/2626467396 learned_addr learned my addr 172.21.15.121:0/2626467396 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:43.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe7a18b7700 1 --2- 172.21.15.121:0/2626467396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe79c0c35d0 0x7fe79c14b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:43.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe7a10b6700 1 --2- 172.21.15.121:0/2626467396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe79c153990 0x7fe79c0bf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:43.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe7a20b8700 1 -- 172.21.15.121:0/2626467396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe79c153990 msgr2=0x7fe79c0bf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:43.300 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe7a20b8700 1 --2- 172.21.15.121:0/2626467396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe79c153990 0x7fe79c0bf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:43.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe7a20b8700 1 -- 172.21.15.121:0/2626467396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe79c0c35d0 msgr2=0x7fe79c14b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:43.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe7a20b8700 1 --2- 172.21.15.121:0/2626467396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe79c0c35d0 0x7fe79c14b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:43.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe7a20b8700 1 -- 172.21.15.121:0/2626467396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe794009a20 con 0x7fe79c153d70 2023-11-15T03:10:43.301 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe7a10b6700 1 --2- 172.21.15.121:0/2626467396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe79c153990 0x7fe79c0bf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:43.302 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe7a20b8700 1 --2- 172.21.15.121:0/2626467396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe79c153d70 0x7fe79c0bfdc0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 rx=0x7fe79400b610 tx=0x7fe794009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:43.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe791ffb700 1 -- 172.21.15.121:0/2626467396 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79401d630 con 0x7fe79c153d70 2023-11-15T03:10:43.303 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe791ffb700 1 -- 172.21.15.121:0/2626467396 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79401d790 con 0x7fe79c153d70 2023-11-15T03:10:43.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe7a28b9700 1 -- 172.21.15.121:0/2626467396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79c154330 con 0x7fe79c153d70 2023-11-15T03:10:43.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.289+0000 7fe791ffb700 1 -- 172.21.15.121:0/2626467396 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79401db10 con 0x7fe79c153d70 2023-11-15T03:10:43.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.293+0000 7fe7a28b9700 1 -- 172.21.15.121:0/2626467396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79c1548f0 con 0x7fe79c153d70 2023-11-15T03:10:43.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.293+0000 7fe7a28b9700 1 -- 172.21.15.121:0/2626467396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79c154dd0 con 0x7fe79c153d70 2023-11-15T03:10:43.304 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.293+0000 7fe791ffb700 1 -- 172.21.15.121:0/2626467396 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe794026410 con 0x7fe79c153d70 2023-11-15T03:10:43.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.297+0000 7fe791ffb700 1 --2- 172.21.15.121:0/2626467396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe78005a510 0x7fe78005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:43.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.297+0000 7fe791ffb700 1 -- 172.21.15.121:0/2626467396 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9534+0+0 (secure 0 0 0) 0x7fe794077320 con 0x7fe79c153d70 2023-11-15T03:10:43.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.297+0000 7fe7a18b7700 1 --2- 172.21.15.121:0/2626467396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe78005a510 0x7fe78005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:43.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.297+0000 7fe7a18b7700 1 --2- 172.21.15.121:0/2626467396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe78005a510 0x7fe78005c9d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7fe798003930 tx=0x7fe798007bd0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:43.309 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.301+0000 7fe791ffb700 1 -- 172.21.15.121:0/2626467396 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe794049c90 con 0x7fe79c153d70 2023-11-15T03:10:43.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:43.501+0000 7fe7a28b9700 1 -- 172.21.15.121:0/2626467396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe79c06bcf0 con 0x7fe79c153d70 2023-11-15T03:10:44.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.397+0000 7fe791ffb700 1 -- 172.21.15.121:0/2626467396 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v329) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe79404d2b0 con 0x7fe79c153d70 2023-11-15T03:10:44.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.401+0000 7fe7a28b9700 1 -- 172.21.15.121:0/2626467396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe78005a510 msgr2=0x7fe78005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:44.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.401+0000 7fe7a28b9700 1 --2- 172.21.15.121:0/2626467396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe78005a510 0x7fe78005c9d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7fe798003930 tx=0x7fe798007bd0).stop 2023-11-15T03:10:44.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.401+0000 7fe7a28b9700 1 -- 172.21.15.121:0/2626467396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe79c153d70 msgr2=0x7fe79c0bfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:44.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.401+0000 7fe7a28b9700 1 --2- 172.21.15.121:0/2626467396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe79c153d70 0x7fe79c0bfdc0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 rx=0x7fe79400b610 tx=0x7fe794009780).stop 2023-11-15T03:10:44.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.401+0000 7fe7a28b9700 1 -- 172.21.15.121:0/2626467396 shutdown_connections 2023-11-15T03:10:44.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.401+0000 7fe7a28b9700 1 --2- 172.21.15.121:0/2626467396 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe78005a510 0x7fe78005c9d0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:44.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.401+0000 7fe7a28b9700 1 --2- 172.21.15.121:0/2626467396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe79c153d70 0x7fe79c0bfdc0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:44.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.401+0000 7fe7a28b9700 1 --2- 172.21.15.121:0/2626467396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe79c153990 0x7fe79c0bf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:44.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.401+0000 7fe7a28b9700 1 --2- 172.21.15.121:0/2626467396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe79c0c35d0 0x7fe79c14b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:44.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.401+0000 7fe7a28b9700 1 -- 172.21.15.121:0/2626467396 >> 172.21.15.121:0/2626467396 conn(0x7fe79c067a60 msgr2=0x7fe79c14bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:44.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.401+0000 7fe7a28b9700 1 -- 172.21.15.121:0/2626467396 shutdown_connections 2023-11-15T03:10:44.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.401+0000 7fe7a28b9700 1 -- 172.21.15.121:0/2626467396 wait complete. 2023-11-15T03:10:44.420 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:10:44.438 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:10:44.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.601+0000 7fdbe359e700 1 Processor -- start 2023-11-15T03:10:44.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.601+0000 7fdbe359e700 1 -- start start 2023-11-15T03:10:44.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.605+0000 7fdbe359e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdbe40ab0d0 0x7fdbe40ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:44.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.605+0000 7fdbe359e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe4066c90 con 0x7fdbe40ad1b0 2023-11-15T03:10:44.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.605+0000 7fdbe359e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe4065e20 con 0x7fdbe40add40 2023-11-15T03:10:44.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.605+0000 7fdbe359e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe4066080 con 0x7fdbe40ab0d0 2023-11-15T03:10:44.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.605+0000 7fdbe1d9b700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fdbe40ad1b0 0x7fdbe4065710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49898/0 (socket says 172.21.15.121:49898) 2023-11-15T03:10:44.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.605+0000 7fdbe1d9b700 1 -- 172.21.15.121:0/4101090833 learned_addr learned my addr 172.21.15.121:0/4101090833 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:44.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.609+0000 7fdbe259c700 1 --2- 172.21.15.121:0/4101090833 >> v2:172.21.15.190:3300/0 conn(0x7fdbe40ab0d0 0x7fdbe40ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:44.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.609+0000 7fdbe159a700 1 -- 172.21.15.121:0/4101090833 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1030556430 0 0) 0x7fdbe4066c90 con 0x7fdbe40ad1b0 2023-11-15T03:10:44.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.609+0000 7fdbe159a700 1 -- 172.21.15.121:0/4101090833 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbcc0033f0 con 0x7fdbe40ad1b0 2023-11-15T03:10:44.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.609+0000 7fdbe259c700 1 -- 172.21.15.121:0/4101090833 >> v1:172.21.15.190:6789/0 conn(0x7fdbe40add40 legacy=0x7fdbe40ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:44.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.609+0000 7fdbe259c700 1 -- 172.21.15.121:0/4101090833 >> v1:172.21.15.121:6789/0 conn(0x7fdbe40ad1b0 legacy=0x7fdbe4065710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:44.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.609+0000 7fdbe259c700 1 -- 172.21.15.121:0/4101090833 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe4066360 con 0x7fdbe40ab0d0 2023-11-15T03:10:44.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.609+0000 7fdbe259c700 1 --2- 172.21.15.121:0/4101090833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbe40ab0d0 0x7fdbe40ab4b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7fdbd4006fd0 tx=0x7fdbd4006f40).ready entity=mon.1 client_cookie=4d56399a7a680c16 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:44.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.609+0000 7fdbe159a700 1 -- 172.21.15.121:0/4101090833 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4005880 con 0x7fdbe40ab0d0 2023-11-15T03:10:44.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.609+0000 7fdbe159a700 1 -- 172.21.15.121:0/4101090833 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd40059e0 con 0x7fdbe40ab0d0 2023-11-15T03:10:44.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.609+0000 7fdbe159a700 1 -- 172.21.15.121:0/4101090833 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4005d60 con 0x7fdbe40ab0d0 2023-11-15T03:10:44.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.609+0000 7fdbe359e700 1 -- 172.21.15.121:0/4101090833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbe40ab0d0 msgr2=0x7fdbe40ab4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:44.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.609+0000 7fdbe359e700 1 --2- 172.21.15.121:0/4101090833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbe40ab0d0 0x7fdbe40ab4b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7fdbd4006fd0 tx=0x7fdbd4006f40).stop 2023-11-15T03:10:44.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.609+0000 7fdbe359e700 1 -- 172.21.15.121:0/4101090833 shutdown_connections 2023-11-15T03:10:44.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.609+0000 7fdbe359e700 1 --2- 172.21.15.121:0/4101090833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbe40ab0d0 0x7fdbe40ab4b0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:44.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.609+0000 7fdbe359e700 1 -- 172.21.15.121:0/4101090833 >> 172.21.15.121:0/4101090833 conn(0x7fdbe4067a60 msgr2=0x7fdbe4067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:44.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.609+0000 7fdbe359e700 1 -- 172.21.15.121:0/4101090833 shutdown_connections 2023-11-15T03:10:44.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.609+0000 7fdbe359e700 1 -- 172.21.15.121:0/4101090833 wait complete. 2023-11-15T03:10:44.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.613+0000 7fdbe359e700 1 Processor -- start 2023-11-15T03:10:44.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.613+0000 7fdbe359e700 1 -- start start 2023-11-15T03:10:44.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.613+0000 7fdbe359e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbe40ab0d0 0x7fdbe40a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:44.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.613+0000 7fdbe359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbe40ad1b0 0x7fdbe4151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:44.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.613+0000 7fdbe359e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbe40add40 0x7fdbe40a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:44.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.613+0000 7fdbe359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe40a4840 con 0x7fdbe40ad1b0 2023-11-15T03:10:44.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.613+0000 7fdbe359e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe40a51c0 con 0x7fdbe40ab0d0 2023-11-15T03:10:44.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.613+0000 7fdbe359e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe41522a0 con 0x7fdbe40add40 2023-11-15T03:10:44.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.613+0000 7fdbe1d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbe40ad1b0 0x7fdbe4151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:44.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.613+0000 7fdbe1d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbe40ad1b0 0x7fdbe4151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56560/0 (socket says 172.21.15.121:56560) 2023-11-15T03:10:44.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.613+0000 7fdbe1d9b700 1 -- 172.21.15.121:0/4146415018 learned_addr learned my addr 172.21.15.121:0/4146415018 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:44.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.613+0000 7fdbe259c700 1 --2- 172.21.15.121:0/4146415018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbe40ab0d0 0x7fdbe40a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:44.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.613+0000 7fdbe2d9d700 1 --2- 172.21.15.121:0/4146415018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbe40add40 0x7fdbe40a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:44.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.617+0000 7fdbe1d9b700 1 -- 172.21.15.121:0/4146415018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbe40add40 msgr2=0x7fdbe40a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:44.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.617+0000 7fdbe1d9b700 1 --2- 172.21.15.121:0/4146415018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbe40add40 0x7fdbe40a6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:44.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.617+0000 7fdbe1d9b700 1 -- 172.21.15.121:0/4146415018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbe40ab0d0 msgr2=0x7fdbe40a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:44.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.617+0000 7fdbe1d9b700 1 --2- 172.21.15.121:0/4146415018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbe40ab0d0 0x7fdbe40a4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:44.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.617+0000 7fdbe1d9b700 1 -- 172.21.15.121:0/4146415018 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd4009250 con 0x7fdbe40ad1b0 2023-11-15T03:10:44.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.617+0000 7fdbe1d9b700 1 --2- 172.21.15.121:0/4146415018 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbe40ad1b0 0x7fdbe4151db0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 rx=0x7fdbd800c520 tx=0x7fdbd8007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:44.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.617+0000 7fdbd27fc700 1 -- 172.21.15.121:0/4146415018 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd80072e0 con 0x7fdbe40ad1b0 2023-11-15T03:10:44.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.617+0000 7fdbd27fc700 1 -- 172.21.15.121:0/4146415018 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd8007440 con 0x7fdbe40ad1b0 2023-11-15T03:10:44.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.617+0000 7fdbe359e700 1 -- 172.21.15.121:0/4146415018 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe40a3c30 con 0x7fdbe40ad1b0 2023-11-15T03:10:44.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.617+0000 7fdbd27fc700 1 -- 172.21.15.121:0/4146415018 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd80057a0 con 0x7fdbe40ad1b0 2023-11-15T03:10:44.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.617+0000 7fdbe359e700 1 -- 172.21.15.121:0/4146415018 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe40a41f0 con 0x7fdbe40ad1b0 2023-11-15T03:10:44.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.617+0000 7fdbe359e700 1 -- 172.21.15.121:0/4146415018 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe406bcf0 con 0x7fdbe40ad1b0 2023-11-15T03:10:44.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.621+0000 7fdbd27fc700 1 -- 172.21.15.121:0/4146415018 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fdbd8005bd0 con 0x7fdbe40ad1b0 2023-11-15T03:10:44.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.621+0000 7fdbd27fc700 1 --2- 172.21.15.121:0/4146415018 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdbc405a510 0x7fdbc405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:44.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.621+0000 7fdbd27fc700 1 -- 172.21.15.121:0/4146415018 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9534+0+0 (secure 0 0 0) 0x7fdbd8074d00 con 0x7fdbe40ad1b0 2023-11-15T03:10:44.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.621+0000 7fdbe259c700 1 --2- 172.21.15.121:0/4146415018 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdbc405a510 0x7fdbc405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:44.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.621+0000 7fdbe259c700 1 --2- 172.21.15.121:0/4146415018 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdbc405a510 0x7fdbc405c9d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7fdbd4006d00 tx=0x7fdbd4016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:44.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.625+0000 7fdbd27fc700 1 -- 172.21.15.121:0/4146415018 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdbd80476d0 con 0x7fdbe40ad1b0 2023-11-15T03:10:44.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:44.841+0000 7fdbe359e700 1 -- 172.21.15.121:0/4146415018 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fdbe40c7160 con 0x7fdbe40ad1b0 2023-11-15T03:10:45.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.405+0000 7fdbd27fc700 1 -- 172.21.15.121:0/4146415018 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v330) v1 ==== 101+0+0 (secure 0 0 0) 0x7fdbd804acf0 con 0x7fdbe40ad1b0 2023-11-15T03:10:45.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.409+0000 7fdbe359e700 1 -- 172.21.15.121:0/4146415018 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdbc405a510 msgr2=0x7fdbc405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:45.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.409+0000 7fdbe359e700 1 --2- 172.21.15.121:0/4146415018 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdbc405a510 0x7fdbc405c9d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7fdbd4006d00 tx=0x7fdbd4016000).stop 2023-11-15T03:10:45.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.409+0000 7fdbe359e700 1 -- 172.21.15.121:0/4146415018 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbe40ad1b0 msgr2=0x7fdbe4151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:45.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.409+0000 7fdbe359e700 1 --2- 172.21.15.121:0/4146415018 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbe40ad1b0 0x7fdbe4151db0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 rx=0x7fdbd800c520 tx=0x7fdbd8007c50).stop 2023-11-15T03:10:45.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.409+0000 7fdbe359e700 1 -- 172.21.15.121:0/4146415018 shutdown_connections 2023-11-15T03:10:45.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.409+0000 7fdbe359e700 1 --2- 172.21.15.121:0/4146415018 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdbc405a510 0x7fdbc405c9d0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:45.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.409+0000 7fdbe359e700 1 --2- 172.21.15.121:0/4146415018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbe40add40 0x7fdbe40a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:45.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.409+0000 7fdbe359e700 1 --2- 172.21.15.121:0/4146415018 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbe40ad1b0 0x7fdbe4151db0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:45.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.409+0000 7fdbe359e700 1 --2- 172.21.15.121:0/4146415018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbe40ab0d0 0x7fdbe40a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:45.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.409+0000 7fdbe359e700 1 -- 172.21.15.121:0/4146415018 >> 172.21.15.121:0/4146415018 conn(0x7fdbe4067a60 msgr2=0x7fdbe41547c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:45.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.409+0000 7fdbe359e700 1 -- 172.21.15.121:0/4146415018 shutdown_connections 2023-11-15T03:10:45.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.409+0000 7fdbe359e700 1 -- 172.21.15.121:0/4146415018 wait complete. 2023-11-15T03:10:45.422 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:10:45.439 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:10:45.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.493+0000 7fa56ab41d00 1 Processor -- start 2023-11-15T03:10:45.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.493+0000 7fa56ab41d00 1 -- start start 2023-11-15T03:10:45.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.493+0000 7fa56ab41d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df38c4c920 0x55df38d2c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:45.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.493+0000 7fa56ab41d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df38c93be0 0x55df38d2c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:45.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.493+0000 7fa56ab41d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55df38d2cb30 0x55df38c769f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:45.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.493+0000 7fa56ab41d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df38bf6cc0 con 0x55df38c93be0 2023-11-15T03:10:45.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.493+0000 7fa56ab41d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df38bc1220 con 0x55df38c4c920 2023-11-15T03:10:45.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.493+0000 7fa56ab41d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df38c91ab0 con 0x55df38d2cb30 2023-11-15T03:10:45.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.493+0000 7fa567c44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df38c93be0 0x55df38d2c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:45.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.493+0000 7fa567c44700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df38c93be0 0x55df38d2c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56630/0 (socket says 172.21.15.121:56630) 2023-11-15T03:10:45.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.493+0000 7fa567c44700 1 -- 172.21.15.121:0/600272536 learned_addr learned my addr 172.21.15.121:0/600272536 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:45.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.493+0000 7fa567c44700 1 -- 172.21.15.121:0/600272536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55df38d2cb30 msgr2=0x55df38c769f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:45.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.493+0000 7fa567c44700 1 --2- 172.21.15.121:0/600272536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55df38d2cb30 0x55df38c769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:10:45.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.493+0000 7fa567c44700 1 -- 172.21.15.121:0/600272536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df38c4c920 msgr2=0x55df38d2c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:45.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.493+0000 7fa567c44700 1 --2- 172.21.15.121:0/600272536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df38c4c920 0x55df38d2c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:10:45.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.493+0000 7fa567c44700 1 -- 172.21.15.121:0/600272536 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c005cf0 con 0x55df38c93be0 2023-11-15T03:10:45.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.497+0000 7fa567c44700 1 --2- 172.21.15.121:0/600272536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df38c93be0 0x55df38d2c640 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 rx=0x7fa55c00afd0 tx=0x7fa55c016000).ready entity=mon.0 client_cookie=a63215b62b992046 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:45.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.497+0000 7fa566c42700 1 -- 172.21.15.121:0/600272536 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c000bc0 con 0x55df38c93be0 2023-11-15T03:10:45.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.497+0000 7fa566c42700 1 -- 172.21.15.121:0/600272536 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c0089d0 con 0x55df38c93be0 2023-11-15T03:10:45.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.497+0000 7fa566c42700 1 -- 172.21.15.121:0/600272536 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c000d20 con 0x55df38c93be0 2023-11-15T03:10:45.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.497+0000 7fa56ab41d00 1 -- 172.21.15.121:0/600272536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df38c93be0 msgr2=0x55df38d2c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:45.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.497+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/600272536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df38c93be0 0x55df38d2c640 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 rx=0x7fa55c00afd0 tx=0x7fa55c016000).stop 2023-11-15T03:10:45.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.497+0000 7fa56ab41d00 1 -- 172.21.15.121:0/600272536 shutdown_connections 2023-11-15T03:10:45.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.497+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/600272536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55df38d2cb30 0x55df38c769f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:10:45.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.497+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/600272536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df38c93be0 0x55df38d2c640 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:45.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.497+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/600272536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df38c4c920 0x55df38d2c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:10:45.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.497+0000 7fa56ab41d00 1 -- 172.21.15.121:0/600272536 >> 172.21.15.121:0/600272536 conn(0x55df38c928f0 msgr2=0x55df38c90f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:45.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.497+0000 7fa56ab41d00 1 -- 172.21.15.121:0/600272536 shutdown_connections 2023-11-15T03:10:45.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.497+0000 7fa56ab41d00 1 -- 172.21.15.121:0/600272536 wait complete. 2023-11-15T03:10:45.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.497+0000 7fa56ab41d00 1 Processor -- start 2023-11-15T03:10:45.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.497+0000 7fa56ab41d00 1 -- start start 2023-11-15T03:10:45.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa56ab41d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55df38c4c920 0x55df38c7af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:45.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa56ab41d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df38c93be0 0x55df38d17d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:45.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa56ab41d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df38d2cb30 0x55df38c755f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:45.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa56ab41d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df38c8ece0 con 0x55df38d2cb30 2023-11-15T03:10:45.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa56ab41d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df38d2d020 con 0x55df38c93be0 2023-11-15T03:10:45.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa56ab41d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df38c7b480 con 0x55df38c4c920 2023-11-15T03:10:45.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa568c46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df38d2cb30 0x55df38c755f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:45.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa567c44700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df38c93be0 0x55df38d17d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:45.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa567c44700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df38c93be0 0x55df38d17d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:36818/0 (socket says 172.21.15.121:36818) 2023-11-15T03:10:45.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 learned_addr learned my addr 172.21.15.121:0/935181358 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:45.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa568c46700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df38d2cb30 0x55df38c755f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56632/0 (socket says 172.21.15.121:56632) 2023-11-15T03:10:45.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa568445700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55df38c4c920 0x55df38c7af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:45.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55df38c4c920 msgr2=0x55df38c7af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:45.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa567c44700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55df38c4c920 0x55df38c7af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:45.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df38d2cb30 msgr2=0x55df38c755f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:45.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa567c44700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df38d2cb30 0x55df38c755f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:45.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df38bea770 con 0x55df38c93be0 2023-11-15T03:10:45.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa568c46700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df38d2cb30 0x55df38c755f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:10:45.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa567c44700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df38c93be0 0x55df38d17d90 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7fa55c005cc0 tx=0x7fa55c01c000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:45.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa56543f700 1 -- 172.21.15.121:0/935181358 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0139b0 con 0x55df38c93be0 2023-11-15T03:10:45.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa56543f700 1 -- 172.21.15.121:0/935181358 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c013b10 con 0x55df38c93be0 2023-11-15T03:10:45.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55df38bb9a70 con 0x55df38c93be0 2023-11-15T03:10:45.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa56543f700 1 -- 172.21.15.121:0/935181358 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c01d410 con 0x55df38c93be0 2023-11-15T03:10:45.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.501+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df38bc5010 con 0x55df38c93be0 2023-11-15T03:10:45.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.505+0000 7fa56543f700 1 -- 172.21.15.121:0/935181358 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa55c01d7c0 con 0x55df38c93be0 2023-11-15T03:10:45.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.505+0000 7fa56543f700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa554058ff0 0x7fa55405b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:45.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.505+0000 7fa56543f700 1 -- 172.21.15.121:0/935181358 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9534+0+0 (secure 0 0 0) 0x7fa55c0772c0 con 0x55df38c93be0 2023-11-15T03:10:45.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.509+0000 7fa568445700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa554058ff0 0x7fa55405b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:45.520 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:10:45.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.509+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55df38c48e40 0x55df38d2d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:45.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.509+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d2de50 con 0x55df38c48e40 2023-11-15T03:10:45.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.509+0000 7fa568445700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa554058ff0 0x7fa55405b4b0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7fa558004170 tx=0x7fa558000bc0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:45.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.509+0000 7fa568c46700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55df38c48e40 0x55df38d2d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:45.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.509+0000 7fa568c46700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55df38c48e40 0x55df38d2d960 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:45.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.513+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v132'449 uv442 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38c48e40 2023-11-15T03:10:45.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.513+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d2e200 con 0x55df38c48e40 2023-11-15T03:10:45.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.513+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v132'450 uv450 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38c48e40 2023-11-15T03:10:45.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.513+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d3c930 con 0x55df38c48e40 2023-11-15T03:10:45.524 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:10:45.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.517+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v132'451 uv451 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38c48e40 2023-11-15T03:10:45.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.517+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d3dcf0 con 0x55df38c48e40 2023-11-15T03:10:45.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.521+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v132'452 uv452 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38c48e40 2023-11-15T03:10:45.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.521+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d3e0a0 con 0x55df38c48e40 2023-11-15T03:10:45.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.521+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v132'453 uv453 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38c48e40 2023-11-15T03:10:45.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.521+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d3cce0 con 0x55df38c48e40 2023-11-15T03:10:45.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.525+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v132'454 uv454 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38c48e40 2023-11-15T03:10:45.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.525+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d3d090 con 0x55df38c48e40 2023-11-15T03:10:45.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.525+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v132'455 uv448 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38c48e40 2023-11-15T03:10:45.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.525+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d3d440 con 0x55df38c48e40 2023-11-15T03:10:45.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.529+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v132'456 uv456 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38c48e40 2023-11-15T03:10:45.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.529+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d3d440 con 0x55df38c48e40 2023-11-15T03:10:45.537 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:10:45.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.529+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v132'457 uv457 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38c48e40 2023-11-15T03:10:45.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.529+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d3d440 con 0x55df38c48e40 2023-11-15T03:10:45.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.533+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v132'458 uv458 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38c48e40 2023-11-15T03:10:45.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.533+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d3d440 con 0x55df38c48e40 2023-11-15T03:10:45.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.533+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v132'459 uv459 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38c48e40 2023-11-15T03:10:45.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.533+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d3d440 con 0x55df38c48e40 2023-11-15T03:10:45.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.537+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v132'460 uv460 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38c48e40 2023-11-15T03:10:45.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.537+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55df38d3e450 0x55df38d40840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:45.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.537+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d40d30 con 0x55df38d3e450 2023-11-15T03:10:45.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.537+0000 7fa568445700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55df38d3e450 0x55df38d40840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:45.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.541+0000 7fa568445700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55df38d3e450 0x55df38d40840 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:45.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.545+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v132'711 uv698 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.545+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d410e0 con 0x55df38d3e450 2023-11-15T03:10:45.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.549+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v132'712 uv712 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.549+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d41490 con 0x55df38d3e450 2023-11-15T03:10:45.558 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:10:45.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.549+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v132'713 uv713 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.553+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d42850 con 0x55df38d3e450 2023-11-15T03:10:45.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.553+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v132'714 uv714 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.553+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d42c00 con 0x55df38d3e450 2023-11-15T03:10:45.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.557+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v132'715 uv715 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.557+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d41840 con 0x55df38d3e450 2023-11-15T03:10:45.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.561+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v132'716 uv716 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.561+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d42160 con 0x55df38d3e450 2023-11-15T03:10:45.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.561+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v132'717 uv717 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.561+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d42fb0 con 0x55df38d3e450 2023-11-15T03:10:45.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.565+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v132'718 uv704 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.565+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d42fb0 con 0x55df38d3e450 2023-11-15T03:10:45.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.569+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v132'719 uv719 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.569+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d42fb0 con 0x55df38d3e450 2023-11-15T03:10:45.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.573+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v132'720 uv720 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.579 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:10:45.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.573+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d43fc0 con 0x55df38d3e450 2023-11-15T03:10:45.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.573+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v132'721 uv721 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.573+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d43fc0 con 0x55df38d3e450 2023-11-15T03:10:45.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.577+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v132'722 uv722 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.577+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d42fb0 con 0x55df38d3e450 2023-11-15T03:10:45.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.581+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v132'723 uv723 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.581+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d42fb0 con 0x55df38d3e450 2023-11-15T03:10:45.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.585+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v132'724 uv710 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.585+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d42fb0 con 0x55df38d3e450 2023-11-15T03:10:45.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.585+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v132'725 uv725 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.585+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d42fb0 con 0x55df38d3e450 2023-11-15T03:10:45.596 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:10:45.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.589+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v132'726 uv726 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.589+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d43fc0 con 0x55df38d3e450 2023-11-15T03:10:45.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.593+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v132'727 uv727 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.593+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d43fc0 con 0x55df38d3e450 2023-11-15T03:10:45.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.597+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v132'728 uv728 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.597+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d42fb0 con 0x55df38d3e450 2023-11-15T03:10:45.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.597+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v132'729 uv729 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.597+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55df38d434c0 0x55df38d458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:45.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.597+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d45da0 con 0x55df38d434c0 2023-11-15T03:10:45.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.597+0000 7fa567c44700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55df38d434c0 0x55df38d458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:45.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.601+0000 7fa567c44700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55df38d434c0 0x55df38d458b0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:45.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.601+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v132'388 uv382 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d434c0 2023-11-15T03:10:45.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.605+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d46150 con 0x55df38d434c0 2023-11-15T03:10:45.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.605+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v132'389 uv389 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d434c0 2023-11-15T03:10:45.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.605+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d46500 con 0x55df38d434c0 2023-11-15T03:10:45.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.609+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v132'390 uv390 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d434c0 2023-11-15T03:10:45.617 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:10:45.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.609+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d468b0 con 0x55df38d434c0 2023-11-15T03:10:45.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.613+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v132'391 uv391 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d434c0 2023-11-15T03:10:45.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.613+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d46c60 con 0x55df38d434c0 2023-11-15T03:10:45.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.617+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v132'392 uv392 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d434c0 2023-11-15T03:10:45.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.617+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d47010 con 0x55df38d434c0 2023-11-15T03:10:45.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.617+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v132'393 uv387 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d434c0 2023-11-15T03:10:45.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.617+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d473c0 con 0x55df38d434c0 2023-11-15T03:10:45.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.621+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v132'394 uv394 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d434c0 2023-11-15T03:10:45.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.621+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d473c0 con 0x55df38d434c0 2023-11-15T03:10:45.632 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:10:45.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.625+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v132'395 uv395 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d434c0 2023-11-15T03:10:45.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.625+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d473c0 con 0x55df38d434c0 2023-11-15T03:10:45.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.629+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v132'396 uv396 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d434c0 2023-11-15T03:10:45.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.629+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d41cf0 con 0x55df38d434c0 2023-11-15T03:10:45.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.629+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v132'397 uv397 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d434c0 2023-11-15T03:10:45.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.629+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55df38d47dc0 0x55df38d4a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:45.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.629+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d4a6a0 con 0x55df38d47dc0 2023-11-15T03:10:45.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.633+0000 7fa568c46700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55df38d47dc0 0x55df38d4a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:45.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.633+0000 7fa568c46700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55df38d47dc0 0x55df38d4a1b0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:45.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.637+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v132'458 uv450 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.637+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d4a6a0 con 0x55df38d47dc0 2023-11-15T03:10:45.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.641+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v132'459 uv459 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.641+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d4a6a0 con 0x55df38d47dc0 2023-11-15T03:10:45.650 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:10:45.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.641+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v132'460 uv460 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.641+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d4e730 con 0x55df38d47dc0 2023-11-15T03:10:45.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.645+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v132'461 uv461 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.645+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d4eb30 con 0x55df38d47dc0 2023-11-15T03:10:45.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.649+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v132'462 uv462 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.649+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d4a6a0 con 0x55df38d47dc0 2023-11-15T03:10:45.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.653+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v132'463 uv456 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.653+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d4a6a0 con 0x55df38d47dc0 2023-11-15T03:10:45.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.657+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v132'464 uv464 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.657+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d4a6a0 con 0x55df38d47dc0 2023-11-15T03:10:45.666 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:10:45.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.657+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v132'465 uv465 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.661+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d4b6f0 con 0x55df38d47dc0 2023-11-15T03:10:45.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.661+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v132'466 uv466 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.661+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d4b6b0 con 0x55df38d47dc0 2023-11-15T03:10:45.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.665+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v132'467 uv467 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.665+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d4a6a0 con 0x55df38d47dc0 2023-11-15T03:10:45.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.669+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v132'468 uv468 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.669+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55df38d4afb0 0x55df38d4d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:45.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.669+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d4d890 con 0x55df38d4afb0 2023-11-15T03:10:45.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.669+0000 7fa568445700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55df38d4afb0 0x55df38d4d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:45.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.669+0000 7fa568445700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55df38d4afb0 0x55df38d4d3a0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:45.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.673+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v132'225 uv224 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d4afb0 2023-11-15T03:10:45.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.673+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d4d950 con 0x55df38d4afb0 2023-11-15T03:10:45.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.673+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v132'226 uv226 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d4afb0 2023-11-15T03:10:45.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.673+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d4dd80 con 0x55df38d4afb0 2023-11-15T03:10:45.684 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:10:45.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.677+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v132'227 uv227 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d4afb0 2023-11-15T03:10:45.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.677+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d4ed90 con 0x55df38d4afb0 2023-11-15T03:10:45.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.681+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v132'228 uv228 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d4afb0 2023-11-15T03:10:45.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.681+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d4ed90 con 0x55df38d4afb0 2023-11-15T03:10:45.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.681+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v132'229 uv229 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d4afb0 2023-11-15T03:10:45.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.681+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d4dd80 con 0x55df38d4afb0 2023-11-15T03:10:45.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.685+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v132'230 uv230 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d4afb0 2023-11-15T03:10:45.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.685+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55df38d4e290 0x55df38d50680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:45.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.685+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d50d50 con 0x55df38d4e290 2023-11-15T03:10:45.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.685+0000 7fa567c44700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55df38d4e290 0x55df38d50680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:45.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.685+0000 7fa567c44700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55df38d4e290 0x55df38d50680 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:45.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.689+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v132'165 uv164 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.689+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d51130 con 0x55df38d4e290 2023-11-15T03:10:45.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.693+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v132'166 uv166 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.693+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d51130 con 0x55df38d4e290 2023-11-15T03:10:45.701 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:10:45.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.693+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v132'167 uv167 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.693+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d51130 con 0x55df38d4e290 2023-11-15T03:10:45.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.697+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v132'168 uv168 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.697+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d510b0 con 0x55df38d4e290 2023-11-15T03:10:45.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.701+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v132'225 uv224 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.701+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d510b0 con 0x55df38d4e290 2023-11-15T03:10:45.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.701+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v132'226 uv226 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.710 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.701+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d50fe0 con 0x55df38d4e290 2023-11-15T03:10:45.713 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:10:45.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.705+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v132'227 uv227 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.705+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d51ff0 con 0x55df38d4e290 2023-11-15T03:10:45.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.709+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v132'228 uv228 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.709+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d51ff0 con 0x55df38d4e290 2023-11-15T03:10:45.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.713+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v132'229 uv229 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.713+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d50fe0 con 0x55df38d4e290 2023-11-15T03:10:45.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.713+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v132'230 uv230 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.713+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d50fe0 con 0x55df38d3e450 2023-11-15T03:10:45.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.717+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v132'415 uv409 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.717+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d50fe0 con 0x55df38d3e450 2023-11-15T03:10:45.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.721+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v132'416 uv416 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.721+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d50fe0 con 0x55df38d3e450 2023-11-15T03:10:45.732 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:10:45.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.725+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v132'417 uv417 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.725+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d52030 con 0x55df38d3e450 2023-11-15T03:10:45.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.729+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v132'418 uv418 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.729+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d51ff0 con 0x55df38d3e450 2023-11-15T03:10:45.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.729+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v132'419 uv419 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.729+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d50fe0 con 0x55df38d3e450 2023-11-15T03:10:45.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.733+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v132'420 uv420 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.733+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d50fe0 con 0x55df38d3e450 2023-11-15T03:10:45.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.737+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v132'421 uv414 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.737+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d50fe0 con 0x55df38d3e450 2023-11-15T03:10:45.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.741+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v132'422 uv422 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.741+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d50fe0 con 0x55df38d3e450 2023-11-15T03:10:45.750 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:10:45.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.741+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v132'423 uv423 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.741+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d50fe0 con 0x55df38d3e450 2023-11-15T03:10:45.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.745+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v132'424 uv424 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.745+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d513e0 con 0x55df38d3e450 2023-11-15T03:10:45.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.749+0000 7fa568445700 1 -- 172.21.15.121:0/935181358 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v132'425 uv425 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55800f920 con 0x55df38d3e450 2023-11-15T03:10:45.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.749+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d51060 con 0x55df38d434c0 2023-11-15T03:10:45.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.753+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v132'188 uv187 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d434c0 2023-11-15T03:10:45.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.753+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d51060 con 0x55df38d434c0 2023-11-15T03:10:45.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.757+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v132'189 uv189 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d434c0 2023-11-15T03:10:45.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.757+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d51060 con 0x55df38d434c0 2023-11-15T03:10:45.765 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:10:45.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.757+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v132'190 uv190 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d434c0 2023-11-15T03:10:45.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.757+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d51060 con 0x55df38d434c0 2023-11-15T03:10:45.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.761+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v132'191 uv191 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d434c0 2023-11-15T03:10:45.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.761+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d510a0 con 0x55df38d434c0 2023-11-15T03:10:45.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.761+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v132'192 uv192 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d434c0 2023-11-15T03:10:45.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.765+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d51060 con 0x55df38d4e290 2023-11-15T03:10:45.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.765+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v132'225 uv224 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.765+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d51060 con 0x55df38d4e290 2023-11-15T03:10:45.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.769+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v132'226 uv226 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.769+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d51060 con 0x55df38d4e290 2023-11-15T03:10:45.781 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:10:45.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.773+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v132'227 uv227 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.773+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d520b0 con 0x55df38d4e290 2023-11-15T03:10:45.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.777+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v132'228 uv228 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.777+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d520b0 con 0x55df38d4e290 2023-11-15T03:10:45.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.781+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v132'229 uv229 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.781+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d51060 con 0x55df38d4e290 2023-11-15T03:10:45.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.785+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v132'230 uv230 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.785+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d51060 con 0x55df38d4e290 2023-11-15T03:10:45.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.785+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v132'189 uv188 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.785+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d51060 con 0x55df38d4e290 2023-11-15T03:10:45.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.793+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v132'190 uv190 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.793+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d510a0 con 0x55df38d4e290 2023-11-15T03:10:45.803 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:10:45.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.797+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v132'191 uv191 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.797+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d51060 con 0x55df38d4e290 2023-11-15T03:10:45.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.801+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v132'192 uv192 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.801+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d514f0 con 0x55df38d4e290 2023-11-15T03:10:45.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.801+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v132'193 uv193 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.801+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d514f0 con 0x55df38d4e290 2023-11-15T03:10:45.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.805+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v132'188 uv187 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.805+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d51150 con 0x55df38d4e290 2023-11-15T03:10:45.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.809+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v132'189 uv189 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.809+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d51150 con 0x55df38d4e290 2023-11-15T03:10:45.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.813+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v132'190 uv190 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.819 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:10:45.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.813+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d51190 con 0x55df38d4e290 2023-11-15T03:10:45.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.813+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v132'191 uv191 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.813+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d53f10 con 0x55df38d4e290 2023-11-15T03:10:45.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.817+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v132'192 uv192 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.817+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55df38d540f0 0x55df38d54530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:45.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.817+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d53ab0 con 0x55df38d540f0 2023-11-15T03:10:45.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.817+0000 7fa568c46700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55df38d540f0 0x55df38d54530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:45.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.817+0000 7fa568c46700 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55df38d540f0 0x55df38d54530 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:45.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.821+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v132'150 uv149 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d540f0 2023-11-15T03:10:45.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.821+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d53ab0 con 0x55df38d540f0 2023-11-15T03:10:45.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.825+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v132'151 uv151 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d540f0 2023-11-15T03:10:45.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.825+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d53ab0 con 0x55df38d540f0 2023-11-15T03:10:45.836 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:10:45.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.829+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v132'152 uv152 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d540f0 2023-11-15T03:10:45.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.829+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d53ab0 con 0x55df38d540f0 2023-11-15T03:10:45.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.833+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v132'153 uv153 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d540f0 2023-11-15T03:10:45.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.833+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d53ab0 con 0x55df38d4e290 2023-11-15T03:10:45.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.833+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v132'225 uv224 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.833+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d53ab0 con 0x55df38d4e290 2023-11-15T03:10:45.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.837+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v132'226 uv226 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.837+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d53ab0 con 0x55df38d4e290 2023-11-15T03:10:45.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.841+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v132'227 uv227 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.850 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:10:45.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.841+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d53ab0 con 0x55df38d4e290 2023-11-15T03:10:45.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.845+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v132'228 uv228 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.845+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d53ab0 con 0x55df38d4e290 2023-11-15T03:10:45.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.845+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v132'229 uv229 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.845+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d53ab0 con 0x55df38d4e290 2023-11-15T03:10:45.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.849+0000 7fa567c44700 1 -- 172.21.15.121:0/935181358 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v132'230 uv230 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa55c023020 con 0x55df38d4e290 2023-11-15T03:10:45.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.849+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d53ab0 con 0x55df38d47dc0 2023-11-15T03:10:45.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.853+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v132'191 uv190 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.853+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d53ab0 con 0x55df38d47dc0 2023-11-15T03:10:45.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.857+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v132'192 uv192 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.857+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d53ab0 con 0x55df38d47dc0 2023-11-15T03:10:45.875 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:10:45.876 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.857+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v132'193 uv193 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.857+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d53ab0 con 0x55df38d47dc0 2023-11-15T03:10:45.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.861+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v132'194 uv194 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.861+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d54a20 con 0x55df38d47dc0 2023-11-15T03:10:45.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.865+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v132'195 uv195 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.865+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d54a20 con 0x55df38d47dc0 2023-11-15T03:10:45.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.869+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v132'451 uv443 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.869+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d53ab0 con 0x55df38d47dc0 2023-11-15T03:10:45.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.869+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v132'452 uv452 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.869+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d54a70 con 0x55df38d47dc0 2023-11-15T03:10:45.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.873+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v132'453 uv453 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.882 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:10:45.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.873+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d54a70 con 0x55df38d47dc0 2023-11-15T03:10:45.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.877+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v132'454 uv454 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.877+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d54a20 con 0x55df38d47dc0 2023-11-15T03:10:45.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.881+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v132'455 uv455 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.881+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d54a20 con 0x55df38d47dc0 2023-11-15T03:10:45.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.881+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v132'456 uv450 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.881+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e132) v8 -- 0x55df38d54a20 con 0x55df38d47dc0 2023-11-15T03:10:45.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.885+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v132'457 uv457 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.885+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d54a20 con 0x55df38d47dc0 2023-11-15T03:10:45.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.889+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v132'458 uv458 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.896 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:10:45.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.889+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d55a30 con 0x55df38d47dc0 2023-11-15T03:10:45.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.889+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v132'459 uv459 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.889+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d55a30 con 0x55df38d47dc0 2023-11-15T03:10:45.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.893+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v132'460 uv460 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.893+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d54a20 con 0x55df38d47dc0 2023-11-15T03:10:45.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.897+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v132'461 uv461 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.897+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x55df38d54f10 con 0x55df38d47dc0 2023-11-15T03:10:45.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v132'462 uv462 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa560006040 con 0x55df38d47dc0 2023-11-15T03:10:45.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55df38d434c0 msgr2=0x55df38d458b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:45.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55df38d434c0 0x55df38d458b0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:45.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55df38c48e40 msgr2=0x55df38d2d960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:45.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55df38c48e40 0x55df38d2d960 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:45.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55df38d4afb0 msgr2=0x55df38d4d3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:45.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55df38d4afb0 0x55df38d4d3a0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:45.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55df38d3e450 msgr2=0x55df38d40840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:45.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55df38d3e450 0x55df38d40840 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:45.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55df38d540f0 msgr2=0x55df38d54530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:45.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x55df38d540f0 0x55df38d54530 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:45.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55df38d47dc0 msgr2=0x55df38d4a1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:45.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55df38d47dc0 0x55df38d4a1b0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:45.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55df38d4e290 msgr2=0x55df38d50680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:45.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55df38d4e290 0x55df38d50680 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:45.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 reap_dead start 2023-11-15T03:10:45.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 reap_dead start 2023-11-15T03:10:45.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 reap_dead start 2023-11-15T03:10:45.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 reap_dead start 2023-11-15T03:10:45.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa568c46700 1 -- 172.21.15.121:0/935181358 reap_dead start 2023-11-15T03:10:45.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa554058ff0 msgr2=0x7fa55405b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:45.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa554058ff0 0x7fa55405b4b0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7fa558004170 tx=0x7fa558000bc0).stop 2023-11-15T03:10:45.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df38c93be0 msgr2=0x55df38d17d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:45.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df38c93be0 0x55df38d17d90 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7fa55c005cc0 tx=0x7fa55c01c000).stop 2023-11-15T03:10:45.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 shutdown_connections 2023-11-15T03:10:45.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa554058ff0 0x7fa55405b4b0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:45.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 --2- 172.21.15.121:0/935181358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df38c93be0 0x55df38d17d90 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:45.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 >> 172.21.15.121:0/935181358 conn(0x55df38c928f0 msgr2=0x55df38d2c150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:45.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 shutdown_connections 2023-11-15T03:10:45.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:45.901+0000 7fa56ab41d00 1 -- 172.21.15.121:0/935181358 wait complete. 2023-11-15T03:10:45.915 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader mds0_sessionmap deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-11-15T03:10:46.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.013+0000 7f6ca7333d00 1 Processor -- start 2023-11-15T03:10:46.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.013+0000 7f6ca7333d00 1 -- start start 2023-11-15T03:10:46.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.013+0000 7f6ca7333d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651a30ea6c0 0x5651a312efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:46.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.013+0000 7f6ca7333d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5651a3134b00 0x5651a31c5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:46.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.013+0000 7f6ca7333d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651a31c3670 0x5651a31c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:46.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6ca7333d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651a31200b0 con 0x5651a30ea6c0 2023-11-15T03:10:46.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6ca7333d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651a31c6200 con 0x5651a31c3670 2023-11-15T03:10:46.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6ca7333d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651a30eaf60 con 0x5651a3134b00 2023-11-15T03:10:46.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6c9ba14700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651a30ea6c0 0x5651a312efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:46.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6c9ba14700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651a30ea6c0 0x5651a312efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56654/0 (socket says 172.21.15.121:56654) 2023-11-15T03:10:46.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6c9ba14700 1 -- 172.21.15.121:0/3158612477 learned_addr learned my addr 172.21.15.121:0/3158612477 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:46.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6c93213700 1 --2- 172.21.15.121:0/3158612477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5651a3134b00 0x5651a31c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:46.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6c9c215700 1 --2- 172.21.15.121:0/3158612477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651a31c3670 0x5651a31c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:46.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6c9ba14700 1 -- 172.21.15.121:0/3158612477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5651a3134b00 msgr2=0x5651a31c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:46.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6c9ba14700 1 --2- 172.21.15.121:0/3158612477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5651a3134b00 0x5651a31c5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:46.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6c9ba14700 1 -- 172.21.15.121:0/3158612477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651a31c3670 msgr2=0x5651a31c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:46.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6c9ba14700 1 --2- 172.21.15.121:0/3158612477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651a31c3670 0x5651a31c5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:46.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6c9ba14700 1 -- 172.21.15.121:0/3158612477 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c00a180 con 0x5651a30ea6c0 2023-11-15T03:10:46.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6c9ba14700 1 --2- 172.21.15.121:0/3158612477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651a30ea6c0 0x5651a312efe0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 rx=0x7f6c8c00ac30 tx=0x7f6c8c014000).ready entity=mon.0 client_cookie=3ad059f29e4218df server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:46.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6c9aa12700 1 -- 172.21.15.121:0/3158612477 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c001010 con 0x5651a30ea6c0 2023-11-15T03:10:46.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6c9aa12700 1 -- 172.21.15.121:0/3158612477 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8c0056f0 con 0x5651a30ea6c0 2023-11-15T03:10:46.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6c9aa12700 1 -- 172.21.15.121:0/3158612477 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0116e0 con 0x5651a30ea6c0 2023-11-15T03:10:46.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6ca7333d00 1 -- 172.21.15.121:0/3158612477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651a30ea6c0 msgr2=0x5651a312efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:46.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6ca7333d00 1 --2- 172.21.15.121:0/3158612477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651a30ea6c0 0x5651a312efe0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 rx=0x7f6c8c00ac30 tx=0x7f6c8c014000).stop 2023-11-15T03:10:46.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6ca7333d00 1 -- 172.21.15.121:0/3158612477 shutdown_connections 2023-11-15T03:10:46.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6ca7333d00 1 --2- 172.21.15.121:0/3158612477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651a31c3670 0x5651a31c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:46.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6ca7333d00 1 --2- 172.21.15.121:0/3158612477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5651a3134b00 0x5651a31c5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:46.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6ca7333d00 1 --2- 172.21.15.121:0/3158612477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651a30ea6c0 0x5651a312efe0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:46.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6ca7333d00 1 -- 172.21.15.121:0/3158612477 >> 172.21.15.121:0/3158612477 conn(0x5651a313a080 msgr2=0x5651a31386c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:46.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6ca7333d00 1 -- 172.21.15.121:0/3158612477 shutdown_connections 2023-11-15T03:10:46.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.017+0000 7f6ca7333d00 1 -- 172.21.15.121:0/3158612477 wait complete. 2023-11-15T03:10:46.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.021+0000 7f6ca7333d00 1 Processor -- start 2023-11-15T03:10:46.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.021+0000 7f6ca7333d00 1 -- start start 2023-11-15T03:10:46.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.021+0000 7f6ca7333d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651a30ea6c0 0x5651a3133270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:46.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.021+0000 7f6ca7333d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5651a3134b00 0x5651a31d79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:46.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.021+0000 7f6ca7333d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651a31c3670 0x5651a31d7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:46.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.021+0000 7f6ca7333d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651a3119bd0 con 0x5651a30ea6c0 2023-11-15T03:10:46.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.021+0000 7f6ca7333d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651a31c3420 con 0x5651a31c3670 2023-11-15T03:10:46.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.021+0000 7f6ca7333d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651a311c380 con 0x5651a3134b00 2023-11-15T03:10:46.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.021+0000 7f6c9ba14700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651a30ea6c0 0x5651a3133270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:46.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.021+0000 7f6c9ba14700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651a30ea6c0 0x5651a3133270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56656/0 (socket says 172.21.15.121:56656) 2023-11-15T03:10:46.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.021+0000 7f6c9ba14700 1 -- 172.21.15.121:0/3901513412 learned_addr learned my addr 172.21.15.121:0/3901513412 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:46.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.021+0000 7f6c93213700 1 --2- 172.21.15.121:0/3901513412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5651a3134b00 0x5651a31d79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:46.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.021+0000 7f6c9c215700 1 --2- 172.21.15.121:0/3901513412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651a31c3670 0x5651a31d7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:46.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.021+0000 7f6c9ba14700 1 -- 172.21.15.121:0/3901513412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5651a3134b00 msgr2=0x5651a31d79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:46.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.021+0000 7f6c9ba14700 1 --2- 172.21.15.121:0/3901513412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5651a3134b00 0x5651a31d79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:46.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.021+0000 7f6c9ba14700 1 -- 172.21.15.121:0/3901513412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651a31c3670 msgr2=0x5651a31d7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:46.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.021+0000 7f6c9ba14700 1 --2- 172.21.15.121:0/3901513412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651a31c3670 0x5651a31d7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:46.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.021+0000 7f6c9ba14700 1 -- 172.21.15.121:0/3901513412 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651a30917f0 con 0x5651a30ea6c0 2023-11-15T03:10:46.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.021+0000 7f6c9ba14700 1 --2- 172.21.15.121:0/3901513412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651a30ea6c0 0x5651a3133270 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 rx=0x7f6c8c005c00 tx=0x7f6c8c01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:46.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.025+0000 7f6c9920f700 1 -- 172.21.15.121:0/3901513412 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c005860 con 0x5651a30ea6c0 2023-11-15T03:10:46.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.025+0000 7f6c9920f700 1 -- 172.21.15.121:0/3901513412 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8c0118f0 con 0x5651a30ea6c0 2023-11-15T03:10:46.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.025+0000 7f6c9920f700 1 -- 172.21.15.121:0/3901513412 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c026620 con 0x5651a30ea6c0 2023-11-15T03:10:46.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.025+0000 7f6ca7333d00 1 -- 172.21.15.121:0/3901513412 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5651a308ca20 con 0x5651a30ea6c0 2023-11-15T03:10:46.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.025+0000 7f6ca7333d00 1 -- 172.21.15.121:0/3901513412 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651a308c770 con 0x5651a30ea6c0 2023-11-15T03:10:46.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.025+0000 7f6c9920f700 1 -- 172.21.15.121:0/3901513412 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6c8c023030 con 0x5651a30ea6c0 2023-11-15T03:10:46.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.025+0000 7f6c9920f700 1 --2- 172.21.15.121:0/3901513412 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6c7c058ff0 0x7f6c7c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:46.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.025+0000 7f6c9920f700 1 -- 172.21.15.121:0/3901513412 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f6c8c077230 con 0x5651a30ea6c0 2023-11-15T03:10:46.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.029+0000 7f6c93213700 1 --2- 172.21.15.121:0/3901513412 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6c7c058ff0 0x7f6c7c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:46.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.029+0000 7f6ca7333d00 1 --2- 172.21.15.121:0/3901513412 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5651a31bbe10 0x5651a31cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:46.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.029+0000 7f6ca7333d00 1 -- 172.21.15.121:0/3901513412 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected e132) v8 -- 0x5651a3134fb0 con 0x5651a31bbe10 2023-11-15T03:10:46.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.029+0000 7f6c93213700 1 --2- 172.21.15.121:0/3901513412 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6c7c058ff0 0x7f6c7c05b4b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x5651a311ae40 tx=0x7f6c8400c040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:46.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.029+0000 7f6c9c215700 1 --2- 172.21.15.121:0/3901513412 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5651a31bbe10 0x5651a31cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:46.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.029+0000 7f6c9c215700 1 --2- 172.21.15.121:0/3901513412 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5651a31bbe10 0x5651a31cdd50 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:46.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.033+0000 7f6c9c215700 1 -- 172.21.15.121:0/3901513412 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(1 mds0_sessionmap [omap-set-header] v132'154 uv154 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f6c940099e0 con 0x5651a31bbe10 2023-11-15T03:10:46.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.033+0000 7f6ca7333d00 1 -- 172.21.15.121:0/3901513412 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5651a31bbe10 msgr2=0x5651a31cdd50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:46.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.033+0000 7f6ca7333d00 1 --2- 172.21.15.121:0/3901513412 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5651a31bbe10 0x5651a31cdd50 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:46.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.033+0000 7f6ca7333d00 1 -- 172.21.15.121:0/3901513412 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6c7c058ff0 msgr2=0x7f6c7c05b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:46.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.033+0000 7f6ca7333d00 1 --2- 172.21.15.121:0/3901513412 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6c7c058ff0 0x7f6c7c05b4b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x5651a311ae40 tx=0x7f6c8400c040).stop 2023-11-15T03:10:46.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.033+0000 7f6ca7333d00 1 -- 172.21.15.121:0/3901513412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651a30ea6c0 msgr2=0x5651a3133270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:46.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.033+0000 7f6ca7333d00 1 --2- 172.21.15.121:0/3901513412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651a30ea6c0 0x5651a3133270 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 rx=0x7f6c8c005c00 tx=0x7f6c8c01a000).stop 2023-11-15T03:10:46.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.033+0000 7f6c9c215700 1 -- 172.21.15.121:0/3901513412 reap_dead start 2023-11-15T03:10:46.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.033+0000 7f6ca7333d00 1 -- 172.21.15.121:0/3901513412 shutdown_connections 2023-11-15T03:10:46.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.033+0000 7f6ca7333d00 1 -- 172.21.15.121:0/3901513412 >> 172.21.15.121:0/3901513412 conn(0x5651a313a080 msgr2=0x5651a3137ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:46.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.033+0000 7f6ca7333d00 1 -- 172.21.15.121:0/3901513412 shutdown_connections 2023-11-15T03:10:46.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.033+0000 7f6ca7333d00 1 -- 172.21.15.121:0/3901513412 wait complete. 2023-11-15T03:10:46.046 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:10:46.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.213+0000 7f421a079700 1 Processor -- start 2023-11-15T03:10:46.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.213+0000 7f421a079700 1 -- start start 2023-11-15T03:10:46.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.217+0000 7f421a079700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f42140a8960 0x7f42140c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:46.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.217+0000 7f421a079700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42140a8d40 0x7f421415c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:46.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.217+0000 7f421a079700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42140657f0 con 0x7f4214066520 2023-11-15T03:10:46.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.217+0000 7f421a079700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42141583d0 con 0x7f42140a8960 2023-11-15T03:10:46.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.217+0000 7f421a079700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42140c1b60 con 0x7f42140a8d40 2023-11-15T03:10:46.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.217+0000 7f4219878700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4214066520 0x7f42140c2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:52258/0 (socket says 172.21.15.121:52258) 2023-11-15T03:10:46.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.217+0000 7f4219878700 1 -- 172.21.15.121:0/3606476940 learned_addr learned my addr 172.21.15.121:0/3606476940 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:46.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.217+0000 7f4219077700 1 --2- 172.21.15.121:0/3606476940 >> v2:172.21.15.121:3300/0 conn(0x7f42140a8960 0x7f42140c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:46.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.217+0000 7f4218876700 1 --2- 172.21.15.121:0/3606476940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42140a8d40 0x7f421415c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:46.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.217+0000 7f420bfff700 1 -- 172.21.15.121:0/3606476940 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054726142 0 0) 0x7f42140657f0 con 0x7f4214066520 2023-11-15T03:10:46.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.217+0000 7f420bfff700 1 -- 172.21.15.121:0/3606476940 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42000033f0 con 0x7f4214066520 2023-11-15T03:10:46.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.217+0000 7f4219077700 1 -- 172.21.15.121:0/3606476940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42140a8d40 msgr2=0x7f421415c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:46.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.217+0000 7f4219077700 1 --2- 172.21.15.121:0/3606476940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42140a8d40 0x7f421415c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:46.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.217+0000 7f4219077700 1 -- 172.21.15.121:0/3606476940 >> v1:172.21.15.121:6789/0 conn(0x7f4214066520 legacy=0x7f42140c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:46.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.217+0000 7f4219077700 1 -- 172.21.15.121:0/3606476940 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4214065a40 con 0x7f42140a8960 2023-11-15T03:10:46.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.217+0000 7f4219077700 1 --2- 172.21.15.121:0/3606476940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42140a8960 0x7f42140c15c0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 rx=0x7f4210009760 tx=0x7f4210009060).ready entity=mon.0 client_cookie=56aab1c0664e42a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:46.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.217+0000 7f420bfff700 1 -- 172.21.15.121:0/3606476940 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421000d4a0 con 0x7f42140a8960 2023-11-15T03:10:46.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.217+0000 7f420bfff700 1 -- 172.21.15.121:0/3606476940 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421000d600 con 0x7f42140a8960 2023-11-15T03:10:46.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.217+0000 7f420bfff700 1 -- 172.21.15.121:0/3606476940 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421000d980 con 0x7f42140a8960 2023-11-15T03:10:46.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.221+0000 7f421a079700 1 -- 172.21.15.121:0/3606476940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42140a8960 msgr2=0x7f42140c15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:46.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.221+0000 7f421a079700 1 --2- 172.21.15.121:0/3606476940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42140a8960 0x7f42140c15c0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 rx=0x7f4210009760 tx=0x7f4210009060).stop 2023-11-15T03:10:46.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.221+0000 7f421a079700 1 -- 172.21.15.121:0/3606476940 shutdown_connections 2023-11-15T03:10:46.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.221+0000 7f421a079700 1 --2- 172.21.15.121:0/3606476940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42140a8d40 0x7f421415c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:46.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.221+0000 7f421a079700 1 --2- 172.21.15.121:0/3606476940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42140a8960 0x7f42140c15c0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:46.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.221+0000 7f421a079700 1 -- 172.21.15.121:0/3606476940 >> 172.21.15.121:0/3606476940 conn(0x7f4214067a60 msgr2=0x7f4214067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:46.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.221+0000 7f421a079700 1 -- 172.21.15.121:0/3606476940 shutdown_connections 2023-11-15T03:10:46.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.221+0000 7f421a079700 1 -- 172.21.15.121:0/3606476940 wait complete. 2023-11-15T03:10:46.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.225+0000 7f421a079700 1 Processor -- start 2023-11-15T03:10:46.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.225+0000 7f421a079700 1 -- start start 2023-11-15T03:10:46.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.225+0000 7f421a079700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4214066520 0x7f421415d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:46.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.225+0000 7f421a079700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42140a8960 0x7f42140bc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:46.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.225+0000 7f421a079700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42140a8d40 0x7f4214157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:46.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.225+0000 7f421a079700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421415d9a0 con 0x7f42140a8960 2023-11-15T03:10:46.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.225+0000 7f421a079700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421415e7c0 con 0x7f4214066520 2023-11-15T03:10:46.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.225+0000 7f421a079700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42140bc5d0 con 0x7f42140a8d40 2023-11-15T03:10:46.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.225+0000 7f4218876700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42140a8960 0x7f42140bc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:46.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.225+0000 7f4218876700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42140a8960 0x7f42140bc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56684/0 (socket says 172.21.15.121:56684) 2023-11-15T03:10:46.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.225+0000 7f4218876700 1 -- 172.21.15.121:0/1415584728 learned_addr learned my addr 172.21.15.121:0/1415584728 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:46.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.225+0000 7f4219878700 1 --2- 172.21.15.121:0/1415584728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42140a8d40 0x7f4214157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:46.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.225+0000 7f4219077700 1 --2- 172.21.15.121:0/1415584728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4214066520 0x7f421415d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:46.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.229+0000 7f4218876700 1 -- 172.21.15.121:0/1415584728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42140a8d40 msgr2=0x7f4214157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:46.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.229+0000 7f4218876700 1 --2- 172.21.15.121:0/1415584728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42140a8d40 0x7f4214157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:46.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.229+0000 7f4218876700 1 -- 172.21.15.121:0/1415584728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4214066520 msgr2=0x7f421415d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:46.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.229+0000 7f4218876700 1 --2- 172.21.15.121:0/1415584728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4214066520 0x7f421415d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:46.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.229+0000 7f4218876700 1 -- 172.21.15.121:0/1415584728 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42100092a0 con 0x7f42140a8960 2023-11-15T03:10:46.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.229+0000 7f4219077700 1 --2- 172.21.15.121:0/1415584728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4214066520 0x7f421415d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:46.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.229+0000 7f4218876700 1 --2- 172.21.15.121:0/1415584728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42140a8960 0x7f42140bc0e0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 rx=0x7f420400e900 tx=0x7f4204009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:46.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.229+0000 7f42097fa700 1 -- 172.21.15.121:0/1415584728 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42040094d0 con 0x7f42140a8960 2023-11-15T03:10:46.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.229+0000 7f42097fa700 1 -- 172.21.15.121:0/1415584728 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420400c670 con 0x7f42140a8960 2023-11-15T03:10:46.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.229+0000 7f42097fa700 1 -- 172.21.15.121:0/1415584728 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420400c9f0 con 0x7f42140a8960 2023-11-15T03:10:46.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.229+0000 7f421a079700 1 -- 172.21.15.121:0/1415584728 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f421415dbb0 con 0x7f42140a8960 2023-11-15T03:10:46.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.229+0000 7f421a079700 1 -- 172.21.15.121:0/1415584728 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f421415e170 con 0x7f42140a8960 2023-11-15T03:10:46.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.229+0000 7f421a079700 1 -- 172.21.15.121:0/1415584728 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42140bda70 con 0x7f42140a8960 2023-11-15T03:10:46.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.233+0000 7f42097fa700 1 -- 172.21.15.121:0/1415584728 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f420401b400 con 0x7f42140a8960 2023-11-15T03:10:46.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.233+0000 7f42097fa700 1 --2- 172.21.15.121:0/1415584728 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41f805a540 0x7f41f805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:46.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.233+0000 7f42097fa700 1 -- 172.21.15.121:0/1415584728 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f4204075980 con 0x7f42140a8960 2023-11-15T03:10:46.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.233+0000 7f4219077700 1 --2- 172.21.15.121:0/1415584728 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41f805a540 0x7f41f805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:46.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.233+0000 7f4219077700 1 --2- 172.21.15.121:0/1415584728 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41f805a540 0x7f41f805ca00 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7f421000d260 tx=0x7f4210016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:46.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.241+0000 7f42097fa700 1 -- 172.21.15.121:0/1415584728 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f42040483f0 con 0x7f42140a8960 2023-11-15T03:10:46.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:46.441+0000 7f421a079700 1 -- 172.21.15.121:0/1415584728 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f421406bcf0 con 0x7f42140a8960 2023-11-15T03:10:47.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.433+0000 7f42097fa700 1 -- 172.21.15.121:0/1415584728 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v331) v1 ==== 162+0+0 (secure 0 0 0) 0x7f420404b990 con 0x7f42140a8960 2023-11-15T03:10:47.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.441+0000 7f421a079700 1 -- 172.21.15.121:0/1415584728 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41f805a540 msgr2=0x7f41f805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:47.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.441+0000 7f421a079700 1 --2- 172.21.15.121:0/1415584728 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41f805a540 0x7f41f805ca00 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7f421000d260 tx=0x7f4210016000).stop 2023-11-15T03:10:47.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.441+0000 7f421a079700 1 -- 172.21.15.121:0/1415584728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42140a8960 msgr2=0x7f42140bc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:47.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.441+0000 7f421a079700 1 --2- 172.21.15.121:0/1415584728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42140a8960 0x7f42140bc0e0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 rx=0x7f420400e900 tx=0x7f4204009060).stop 2023-11-15T03:10:47.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.441+0000 7f421a079700 1 -- 172.21.15.121:0/1415584728 shutdown_connections 2023-11-15T03:10:47.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.441+0000 7f421a079700 1 --2- 172.21.15.121:0/1415584728 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f41f805a540 0x7f41f805ca00 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:47.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.441+0000 7f421a079700 1 --2- 172.21.15.121:0/1415584728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42140a8d40 0x7f4214157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:47.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.441+0000 7f421a079700 1 --2- 172.21.15.121:0/1415584728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42140a8960 0x7f42140bc0e0 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:47.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.441+0000 7f421a079700 1 --2- 172.21.15.121:0/1415584728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4214066520 0x7f421415d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:47.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.441+0000 7f421a079700 1 -- 172.21.15.121:0/1415584728 >> 172.21.15.121:0/1415584728 conn(0x7f4214067a60 msgr2=0x7f42140c6320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:47.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.441+0000 7f421a079700 1 -- 172.21.15.121:0/1415584728 shutdown_connections 2023-11-15T03:10:47.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.441+0000 7f421a079700 1 -- 172.21.15.121:0/1415584728 wait complete. 2023-11-15T03:10:47.454 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:10:47.454 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:10:47.445+0000 7fc92dfea700 -1 log_channel(cluster) log [ERR] : corrupt sessionmap header: virtual void SessionMapStore::decode_header(ceph::bufferlist&) no longer understand old encoding version 1 < 101: Malformed input 2023-11-15T03:10:47.470 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:10:47.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.621+0000 7fc8cace4700 1 Processor -- start 2023-11-15T03:10:47.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.621+0000 7fc8cace4700 1 -- start start 2023-11-15T03:10:47.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.621+0000 7fc8cace4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc8c4066d60 0x7fc8c4067140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:47.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.621+0000 7fc8cace4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4157ef0 0x7fc8c41535c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:47.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.621+0000 7fc8cace4700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8c40667c0 con 0x7fc8c4066370 2023-11-15T03:10:47.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.621+0000 7fc8cace4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c40c00f0 con 0x7fc8c4066d60 2023-11-15T03:10:47.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.621+0000 7fc8cace4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c41583f0 con 0x7fc8c4157ef0 2023-11-15T03:10:47.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8c94e1700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc8c4066370 0x7fc8c41577e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:52268/0 (socket says 172.21.15.121:52268) 2023-11-15T03:10:47.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8c94e1700 1 -- 172.21.15.121:0/2732677383 learned_addr learned my addr 172.21.15.121:0/2732677383 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:47.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8ca4e3700 1 --2- 172.21.15.121:0/2732677383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4157ef0 0x7fc8c41535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:47.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8c9ce2700 1 --2- 172.21.15.121:0/2732677383 >> v2:172.21.15.190:3300/0 conn(0x7fc8c4066d60 0x7fc8c4067140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:47.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8c8ce0700 1 -- 172.21.15.121:0/2732677383 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1330120089 0 0) 0x7fc8c40667c0 con 0x7fc8c4066370 2023-11-15T03:10:47.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8c8ce0700 1 -- 172.21.15.121:0/2732677383 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8b00033f0 con 0x7fc8c4066370 2023-11-15T03:10:47.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8ca4e3700 1 -- 172.21.15.121:0/2732677383 >> v2:172.21.15.190:3300/0 conn(0x7fc8c4066d60 msgr2=0x7fc8c4067140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:47.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8ca4e3700 1 --2- 172.21.15.121:0/2732677383 >> v2:172.21.15.190:3300/0 conn(0x7fc8c4066d60 0x7fc8c4067140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:47.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8ca4e3700 1 -- 172.21.15.121:0/2732677383 >> v1:172.21.15.121:6789/0 conn(0x7fc8c4066370 legacy=0x7fc8c41577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:47.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8ca4e3700 1 -- 172.21.15.121:0/2732677383 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c41586d0 con 0x7fc8c4157ef0 2023-11-15T03:10:47.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8ca4e3700 1 --2- 172.21.15.121:0/2732677383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4157ef0 0x7fc8c41535c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fc8bc009e90 tx=0x7fc8bc009780).ready entity=mon.2 client_cookie=90c29130688c298d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:47.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8c8ce0700 1 -- 172.21.15.121:0/2732677383 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc00e070 con 0x7fc8c4157ef0 2023-11-15T03:10:47.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8c8ce0700 1 -- 172.21.15.121:0/2732677383 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8bc00ba70 con 0x7fc8c4157ef0 2023-11-15T03:10:47.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8c8ce0700 1 -- 172.21.15.121:0/2732677383 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc00bdf0 con 0x7fc8c4157ef0 2023-11-15T03:10:47.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8cace4700 1 -- 172.21.15.121:0/2732677383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4157ef0 msgr2=0x7fc8c41535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:47.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8cace4700 1 --2- 172.21.15.121:0/2732677383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4157ef0 0x7fc8c41535c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fc8bc009e90 tx=0x7fc8bc009780).stop 2023-11-15T03:10:47.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8cace4700 1 -- 172.21.15.121:0/2732677383 shutdown_connections 2023-11-15T03:10:47.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8cace4700 1 --2- 172.21.15.121:0/2732677383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4157ef0 0x7fc8c41535c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:47.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8cace4700 1 --2- 172.21.15.121:0/2732677383 >> v2:172.21.15.190:3300/0 conn(0x7fc8c4066d60 0x7fc8c4067140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:47.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.625+0000 7fc8cace4700 1 -- 172.21.15.121:0/2732677383 >> 172.21.15.121:0/2732677383 conn(0x7fc8c4062190 msgr2=0x7fc8c4062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:47.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.629+0000 7fc8cace4700 1 -- 172.21.15.121:0/2732677383 shutdown_connections 2023-11-15T03:10:47.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.629+0000 7fc8cace4700 1 -- 172.21.15.121:0/2732677383 wait complete. 2023-11-15T03:10:47.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.629+0000 7fc8cace4700 1 Processor -- start 2023-11-15T03:10:47.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.629+0000 7fc8cace4700 1 -- start start 2023-11-15T03:10:47.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8cace4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c4066370 0x7fc8c4154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:47.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8cace4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4066d60 0x7fc8c4163d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:47.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8cace4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8c4157ef0 0x7fc8c414c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:47.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8cace4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c414ff70 con 0x7fc8c4066370 2023-11-15T03:10:47.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8cace4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c414fc40 con 0x7fc8c4157ef0 2023-11-15T03:10:47.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8cace4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c4154c40 con 0x7fc8c4066d60 2023-11-15T03:10:47.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8c9ce2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c4066370 0x7fc8c4154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:47.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8c9ce2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c4066370 0x7fc8c4154360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56696/0 (socket says 172.21.15.121:56696) 2023-11-15T03:10:47.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8c9ce2700 1 -- 172.21.15.121:0/2368063264 learned_addr learned my addr 172.21.15.121:0/2368063264 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:47.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8c94e1700 1 --2- 172.21.15.121:0/2368063264 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4066d60 0x7fc8c4163d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:47.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8ca4e3700 1 --2- 172.21.15.121:0/2368063264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8c4157ef0 0x7fc8c414c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:47.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8c9ce2700 1 -- 172.21.15.121:0/2368063264 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4066d60 msgr2=0x7fc8c4163d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:47.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8c9ce2700 1 --2- 172.21.15.121:0/2368063264 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4066d60 0x7fc8c4163d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:47.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8c9ce2700 1 -- 172.21.15.121:0/2368063264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8c4157ef0 msgr2=0x7fc8c414c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:47.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8c9ce2700 1 --2- 172.21.15.121:0/2368063264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8c4157ef0 0x7fc8c414c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:47.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8c9ce2700 1 -- 172.21.15.121:0/2368063264 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8bc009a20 con 0x7fc8c4066370 2023-11-15T03:10:47.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8ca4e3700 1 --2- 172.21.15.121:0/2368063264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8c4157ef0 0x7fc8c414c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:10:47.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8c9ce2700 1 --2- 172.21.15.121:0/2368063264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c4066370 0x7fc8c4154360 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 rx=0x7fc8c000e930 tx=0x7fc8c0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:47.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8b9ffb700 1 -- 172.21.15.121:0/2368063264 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c000eed0 con 0x7fc8c4066370 2023-11-15T03:10:47.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8b9ffb700 1 -- 172.21.15.121:0/2368063264 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c000c850 con 0x7fc8c4066370 2023-11-15T03:10:47.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8b9ffb700 1 -- 172.21.15.121:0/2368063264 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c000cbd0 con 0x7fc8c4066370 2023-11-15T03:10:47.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8cace4700 1 -- 172.21.15.121:0/2368063264 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c4154f20 con 0x7fc8c4066370 2023-11-15T03:10:47.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.633+0000 7fc8cace4700 1 -- 172.21.15.121:0/2368063264 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c4154850 con 0x7fc8c4066370 2023-11-15T03:10:47.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.637+0000 7fc8cace4700 1 -- 172.21.15.121:0/2368063264 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c406bcf0 con 0x7fc8c4066370 2023-11-15T03:10:47.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.637+0000 7fc8b9ffb700 1 -- 172.21.15.121:0/2368063264 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc8c001b400 con 0x7fc8c4066370 2023-11-15T03:10:47.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.637+0000 7fc8b9ffb700 1 --2- 172.21.15.121:0/2368063264 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc8a805a540 0x7fc8a805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:47.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.637+0000 7fc8b9ffb700 1 -- 172.21.15.121:0/2368063264 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 9620+0+0 (secure 0 0 0) 0x7fc8c0075a80 con 0x7fc8c4066370 2023-11-15T03:10:47.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.637+0000 7fc8c94e1700 1 --2- 172.21.15.121:0/2368063264 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc8a805a540 0x7fc8a805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:47.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.641+0000 7fc8c94e1700 1 --2- 172.21.15.121:0/2368063264 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc8a805a540 0x7fc8a805ca00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7fc8c40c6100 tx=0x7fc8b4009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:47.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.645+0000 7fc8b9ffb700 1 -- 172.21.15.121:0/2368063264 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc8c0077e60 con 0x7fc8c4066370 2023-11-15T03:10:47.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.837+0000 7fc8cace4700 1 -- 172.21.15.121:0/2368063264 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8c4065890 con 0x7fc8c4066370 2023-11-15T03:10:47.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.837+0000 7fc8b9ffb700 1 -- 172.21.15.121:0/2368063264 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 332 v332) v1 ==== 95+0+4508 (secure 0 0 0) 0x7fc8c0048410 con 0x7fc8c4066370 2023-11-15T03:10:47.845 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:10:47.846 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":332,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":332,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:10:47.437558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10386},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10386":{"gid":10386,"name":"a","rank":0,"incarnation":332,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/2269363825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2269363825},{"type":"v1","addr":"172.21.15.121:6836","nonce":2269363825}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:10:47.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.841+0000 7fc8cace4700 1 -- 172.21.15.121:0/2368063264 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc8a805a540 msgr2=0x7fc8a805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:47.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.841+0000 7fc8cace4700 1 --2- 172.21.15.121:0/2368063264 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc8a805a540 0x7fc8a805ca00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7fc8c40c6100 tx=0x7fc8b4009040).stop 2023-11-15T03:10:47.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.841+0000 7fc8cace4700 1 -- 172.21.15.121:0/2368063264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c4066370 msgr2=0x7fc8c4154360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:47.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.841+0000 7fc8cace4700 1 --2- 172.21.15.121:0/2368063264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c4066370 0x7fc8c4154360 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 rx=0x7fc8c000e930 tx=0x7fc8c0009060).stop 2023-11-15T03:10:47.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.841+0000 7fc8cace4700 1 -- 172.21.15.121:0/2368063264 shutdown_connections 2023-11-15T03:10:47.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.841+0000 7fc8cace4700 1 --2- 172.21.15.121:0/2368063264 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc8a805a540 0x7fc8a805ca00 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:47.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.841+0000 7fc8cace4700 1 --2- 172.21.15.121:0/2368063264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8c4157ef0 0x7fc8c414c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:47.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.841+0000 7fc8cace4700 1 --2- 172.21.15.121:0/2368063264 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4066d60 0x7fc8c4163d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:47.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.841+0000 7fc8cace4700 1 --2- 172.21.15.121:0/2368063264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c4066370 0x7fc8c4154360 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:47.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.841+0000 7fc8cace4700 1 -- 172.21.15.121:0/2368063264 >> 172.21.15.121:0/2368063264 conn(0x7fc8c4062190 msgr2=0x7fc8c40c6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:47.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.841+0000 7fc8cace4700 1 -- 172.21.15.121:0/2368063264 shutdown_connections 2023-11-15T03:10:47.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:47.841+0000 7fc8cace4700 1 -- 172.21.15.121:0/2368063264 wait complete. 2023-11-15T03:10:47.851 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 332 2023-11-15T03:10:47.866 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:10:48.449 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:10:48.441+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:10:48.450 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:10:48.441+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:10:48.458 INFO:tasks.ceph.mds.a.smithi121.stderr: -25> 2023-11-15T03:10:47.445+0000 7fc92dfea700 -1 log_channel(cluster) log [ERR] : corrupt sessionmap header: virtual void SessionMapStore::decode_header(ceph::bufferlist&) no longer understand old encoding version 1 < 101: Malformed input 2023-11-15T03:10:48.501 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:10:52.868 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:10:53.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.029+0000 7fe9d5ef1700 1 Processor -- start 2023-11-15T03:10:53.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.029+0000 7fe9d5ef1700 1 -- start start 2023-11-15T03:10:53.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.029+0000 7fe9d5ef1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d014e200 0x7fe9d00ac920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:53.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.029+0000 7fe9d5ef1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe9d00ace10 0x7fe9d00a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:53.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.029+0000 7fe9d5ef1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d0154270 con 0x7fe9d00aa020 2023-11-15T03:10:53.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.029+0000 7fe9d5ef1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d00cba70 con 0x7fe9d00ace10 2023-11-15T03:10:53.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.029+0000 7fe9d5ef1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d0154dd0 con 0x7fe9d014e200 2023-11-15T03:10:53.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.029+0000 7fe9cffff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe9d00aa020 0x7fe9d0157880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:52276/0 (socket says 172.21.15.121:52276) 2023-11-15T03:10:53.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.029+0000 7fe9cffff700 1 -- 172.21.15.121:0/4255850373 learned_addr learned my addr 172.21.15.121:0/4255850373 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:53.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.029+0000 7fe9ceffd700 1 --2- 172.21.15.121:0/4255850373 >> v2:172.21.15.121:3300/0 conn(0x7fe9d00ace10 0x7fe9d00a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:53.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.029+0000 7fe9cf7fe700 1 --2- 172.21.15.121:0/4255850373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d014e200 0x7fe9d00ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:53.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.029+0000 7fe9ce7fc700 1 -- 172.21.15.121:0/4255850373 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725194470 0 0) 0x7fe9d0154270 con 0x7fe9d00aa020 2023-11-15T03:10:53.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.029+0000 7fe9ce7fc700 1 -- 172.21.15.121:0/4255850373 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9b80033f0 con 0x7fe9d00aa020 2023-11-15T03:10:53.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.033+0000 7fe9ceffd700 1 -- 172.21.15.121:0/4255850373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d014e200 msgr2=0x7fe9d00ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:53.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.033+0000 7fe9ceffd700 1 --2- 172.21.15.121:0/4255850373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d014e200 0x7fe9d00ac920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.033+0000 7fe9ceffd700 1 -- 172.21.15.121:0/4255850373 >> v1:172.21.15.121:6789/0 conn(0x7fe9d00aa020 legacy=0x7fe9d0157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:53.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.033+0000 7fe9ceffd700 1 -- 172.21.15.121:0/4255850373 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d00aa4d0 con 0x7fe9d00ace10 2023-11-15T03:10:53.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.033+0000 7fe9ceffd700 1 --2- 172.21.15.121:0/4255850373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9d00ace10 0x7fe9d00a9b30 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 rx=0x7fe9c4009760 tx=0x7fe9c4009060).ready entity=mon.0 client_cookie=3dac8c339f9fe598 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:53.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.033+0000 7fe9ce7fc700 1 -- 172.21.15.121:0/4255850373 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c400d4a0 con 0x7fe9d00ace10 2023-11-15T03:10:53.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.033+0000 7fe9ce7fc700 1 -- 172.21.15.121:0/4255850373 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c400d600 con 0x7fe9d00ace10 2023-11-15T03:10:53.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.033+0000 7fe9ce7fc700 1 -- 172.21.15.121:0/4255850373 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c400d980 con 0x7fe9d00ace10 2023-11-15T03:10:53.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.033+0000 7fe9d5ef1700 1 -- 172.21.15.121:0/4255850373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9d00ace10 msgr2=0x7fe9d00a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:53.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.033+0000 7fe9d5ef1700 1 --2- 172.21.15.121:0/4255850373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9d00ace10 0x7fe9d00a9b30 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 rx=0x7fe9c4009760 tx=0x7fe9c4009060).stop 2023-11-15T03:10:53.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.033+0000 7fe9d5ef1700 1 -- 172.21.15.121:0/4255850373 shutdown_connections 2023-11-15T03:10:53.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.033+0000 7fe9d5ef1700 1 --2- 172.21.15.121:0/4255850373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9d00ace10 0x7fe9d00a9b30 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.033+0000 7fe9d5ef1700 1 --2- 172.21.15.121:0/4255850373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d014e200 0x7fe9d00ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.033+0000 7fe9d5ef1700 1 -- 172.21.15.121:0/4255850373 >> 172.21.15.121:0/4255850373 conn(0x7fe9d0062190 msgr2=0x7fe9d00625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:53.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.033+0000 7fe9d5ef1700 1 -- 172.21.15.121:0/4255850373 shutdown_connections 2023-11-15T03:10:53.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.033+0000 7fe9d5ef1700 1 -- 172.21.15.121:0/4255850373 wait complete. 2023-11-15T03:10:53.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.037+0000 7fe9d5ef1700 1 Processor -- start 2023-11-15T03:10:53.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.037+0000 7fe9d5ef1700 1 -- start start 2023-11-15T03:10:53.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.037+0000 7fe9d5ef1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9d00aa020 0x7fe9d013f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:53.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.037+0000 7fe9d5ef1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d00ace10 0x7fe9d00a7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:53.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.037+0000 7fe9d5ef1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9d014e200 0x7fe9d0167950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:53.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.037+0000 7fe9d5ef1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d00a8940 con 0x7fe9d014e200 2023-11-15T03:10:53.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.037+0000 7fe9d5ef1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d00a6530 con 0x7fe9d00aa020 2023-11-15T03:10:53.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.037+0000 7fe9d5ef1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d013f110 con 0x7fe9d00ace10 2023-11-15T03:10:53.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.037+0000 7fe9cffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9d014e200 0x7fe9d0167950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:53.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.037+0000 7fe9cffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9d014e200 0x7fe9d0167950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56720/0 (socket says 172.21.15.121:56720) 2023-11-15T03:10:53.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.041+0000 7fe9cffff700 1 -- 172.21.15.121:0/2132148035 learned_addr learned my addr 172.21.15.121:0/2132148035 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:53.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.041+0000 7fe9cf7fe700 1 --2- 172.21.15.121:0/2132148035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9d00aa020 0x7fe9d013f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:53.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.041+0000 7fe9ceffd700 1 --2- 172.21.15.121:0/2132148035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d00ace10 0x7fe9d00a7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:53.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.041+0000 7fe9cffff700 1 -- 172.21.15.121:0/2132148035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d00ace10 msgr2=0x7fe9d00a7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:53.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.041+0000 7fe9cffff700 1 --2- 172.21.15.121:0/2132148035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d00ace10 0x7fe9d00a7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.041+0000 7fe9cffff700 1 -- 172.21.15.121:0/2132148035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9d00aa020 msgr2=0x7fe9d013f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:53.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.041+0000 7fe9cffff700 1 --2- 172.21.15.121:0/2132148035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9d00aa020 0x7fe9d013f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.041+0000 7fe9cffff700 1 -- 172.21.15.121:0/2132148035 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c40092a0 con 0x7fe9d014e200 2023-11-15T03:10:53.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.041+0000 7fe9cffff700 1 --2- 172.21.15.121:0/2132148035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9d014e200 0x7fe9d0167950 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 rx=0x7fe9c8001460 tx=0x7fe9c80015b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:53.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.041+0000 7fe9b77fe700 1 -- 172.21.15.121:0/2132148035 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8005d30 con 0x7fe9d014e200 2023-11-15T03:10:53.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.041+0000 7fe9b77fe700 1 -- 172.21.15.121:0/2132148035 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c8005e90 con 0x7fe9d014e200 2023-11-15T03:10:53.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.041+0000 7fe9d5ef1700 1 -- 172.21.15.121:0/2132148035 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d00a7d30 con 0x7fe9d014e200 2023-11-15T03:10:53.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.041+0000 7fe9b77fe700 1 -- 172.21.15.121:0/2132148035 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80072e0 con 0x7fe9d014e200 2023-11-15T03:10:53.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.041+0000 7fe9d5ef1700 1 -- 172.21.15.121:0/2132148035 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d00a82f0 con 0x7fe9d014e200 2023-11-15T03:10:53.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.041+0000 7fe9d5ef1700 1 -- 172.21.15.121:0/2132148035 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d00a8af0 con 0x7fe9d014e200 2023-11-15T03:10:53.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.045+0000 7fe9b77fe700 1 -- 172.21.15.121:0/2132148035 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe9c80076d0 con 0x7fe9d014e200 2023-11-15T03:10:53.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.045+0000 7fe9b77fe700 1 --2- 172.21.15.121:0/2132148035 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe9bc05a510 0x7fe9bc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:53.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.045+0000 7fe9b77fe700 1 -- 172.21.15.121:0/2132148035 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 9620+0+0 (secure 0 0 0) 0x7fe9c8074f30 con 0x7fe9d014e200 2023-11-15T03:10:53.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.045+0000 7fe9cf7fe700 1 --2- 172.21.15.121:0/2132148035 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe9bc05a510 0x7fe9bc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:53.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.045+0000 7fe9cf7fe700 1 --2- 172.21.15.121:0/2132148035 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe9bc05a510 0x7fe9bc05c9d0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7fe9c0009c30 tx=0x7fe9c0009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:53.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.053+0000 7fe9b77fe700 1 -- 172.21.15.121:0/2132148035 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe9c8047920 con 0x7fe9d014e200 2023-11-15T03:10:53.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.253+0000 7fe9d5ef1700 1 -- 172.21.15.121:0/2132148035 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9d006c150 con 0x7fe9d014e200 2023-11-15T03:10:53.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.253+0000 7fe9b77fe700 1 -- 172.21.15.121:0/2132148035 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 334 v334) v1 ==== 95+0+4497 (secure 0 0 0) 0x7fe9c804af40 con 0x7fe9d014e200 2023-11-15T03:10:53.262 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:10:53.263 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":334,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":10461,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3720977338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3720977338},{"type":"v1","addr":"172.21.15.121:6836","nonce":3720977338}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":334}],"filesystems":[{"mdsmap":{"epoch":333,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:10:48.444387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:10:53.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.257+0000 7fe9d5ef1700 1 -- 172.21.15.121:0/2132148035 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe9bc05a510 msgr2=0x7fe9bc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:53.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.257+0000 7fe9d5ef1700 1 --2- 172.21.15.121:0/2132148035 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe9bc05a510 0x7fe9bc05c9d0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7fe9c0009c30 tx=0x7fe9c0009060).stop 2023-11-15T03:10:53.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.257+0000 7fe9d5ef1700 1 -- 172.21.15.121:0/2132148035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9d014e200 msgr2=0x7fe9d0167950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:53.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.257+0000 7fe9d5ef1700 1 --2- 172.21.15.121:0/2132148035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9d014e200 0x7fe9d0167950 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 rx=0x7fe9c8001460 tx=0x7fe9c80015b0).stop 2023-11-15T03:10:53.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.257+0000 7fe9d5ef1700 1 -- 172.21.15.121:0/2132148035 shutdown_connections 2023-11-15T03:10:53.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.257+0000 7fe9d5ef1700 1 --2- 172.21.15.121:0/2132148035 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe9bc05a510 0x7fe9bc05c9d0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.257+0000 7fe9d5ef1700 1 --2- 172.21.15.121:0/2132148035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9d014e200 0x7fe9d0167950 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.257+0000 7fe9d5ef1700 1 --2- 172.21.15.121:0/2132148035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d00ace10 0x7fe9d00a7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.257+0000 7fe9d5ef1700 1 --2- 172.21.15.121:0/2132148035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9d00aa020 0x7fe9d013f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.257+0000 7fe9d5ef1700 1 -- 172.21.15.121:0/2132148035 >> 172.21.15.121:0/2132148035 conn(0x7fe9d0062190 msgr2=0x7fe9d00aa7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:53.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.257+0000 7fe9d5ef1700 1 -- 172.21.15.121:0/2132148035 shutdown_connections 2023-11-15T03:10:53.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.257+0000 7fe9d5ef1700 1 -- 172.21.15.121:0/2132148035 wait complete. 2023-11-15T03:10:53.268 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 334 2023-11-15T03:10:53.285 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:10:53.285 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:10:53.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.441+0000 7f8cf0a79700 1 Processor -- start 2023-11-15T03:10:53.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.441+0000 7f8cf0a79700 1 -- start start 2023-11-15T03:10:53.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.445+0000 7f8cf0a79700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8cec063c60 0x7f8cec068a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:53.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.445+0000 7f8cf0a79700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cec151300 con 0x7f8cec0a9640 2023-11-15T03:10:53.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.445+0000 7f8cf0a79700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cec0643f0 con 0x7f8cec0a9260 2023-11-15T03:10:53.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.445+0000 7f8cf0a79700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cec0649b0 con 0x7f8cec063c60 2023-11-15T03:10:53.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.445+0000 7f8ceb7fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f8cec0a9260 0x7f8cec151790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:58082/0 (socket says 172.21.15.121:58082) 2023-11-15T03:10:53.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.445+0000 7f8ceb7fe700 1 -- 172.21.15.121:0/1236991243 learned_addr learned my addr 172.21.15.121:0/1236991243 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:53.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.445+0000 7f8cebfff700 1 --2- 172.21.15.121:0/1236991243 >> v2:172.21.15.190:3300/0 conn(0x7f8cec063c60 0x7f8cec068a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:53.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.445+0000 7f8ceaffd700 1 --1- 172.21.15.121:0/1236991243 >> v1:172.21.15.121:6789/0 conn(0x7f8cec0a9640 0x7f8cec062220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:52282/0 (socket says 172.21.15.121:52282) 2023-11-15T03:10:53.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.449+0000 7f8cebfff700 1 -- 172.21.15.121:0/1236991243 >> v1:172.21.15.190:6789/0 conn(0x7f8cec0a9260 legacy=0x7f8cec151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:53.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.449+0000 7f8cebfff700 1 -- 172.21.15.121:0/1236991243 >> v1:172.21.15.121:6789/0 conn(0x7f8cec0a9640 legacy=0x7f8cec062220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:53.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.449+0000 7f8cea7fc700 1 -- 172.21.15.121:0/1236991243 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369843224 0 0) 0x7f8cec151300 con 0x7f8cec0a9640 2023-11-15T03:10:53.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.449+0000 7f8cebfff700 1 -- 172.21.15.121:0/1236991243 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec0646d0 con 0x7f8cec063c60 2023-11-15T03:10:53.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.449+0000 7f8cebfff700 1 --2- 172.21.15.121:0/1236991243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cec063c60 0x7f8cec068a70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7f8ce4009e60 tx=0x7f8ce4009780).ready entity=mon.1 client_cookie=3248b2ecbc93b663 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:53.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.449+0000 7f8cea7fc700 1 -- 172.21.15.121:0/1236991243 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400e070 con 0x7f8cec063c60 2023-11-15T03:10:53.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.449+0000 7f8cea7fc700 1 -- 172.21.15.121:0/1236991243 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce400bc30 con 0x7f8cec063c60 2023-11-15T03:10:53.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.449+0000 7f8cea7fc700 1 -- 172.21.15.121:0/1236991243 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4012410 con 0x7f8cec063c60 2023-11-15T03:10:53.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.449+0000 7f8cf0a79700 1 -- 172.21.15.121:0/1236991243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cec063c60 msgr2=0x7f8cec068a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:53.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.449+0000 7f8cf0a79700 1 --2- 172.21.15.121:0/1236991243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cec063c60 0x7f8cec068a70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7f8ce4009e60 tx=0x7f8ce4009780).stop 2023-11-15T03:10:53.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.449+0000 7f8cf0a79700 1 -- 172.21.15.121:0/1236991243 shutdown_connections 2023-11-15T03:10:53.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.449+0000 7f8cf0a79700 1 --2- 172.21.15.121:0/1236991243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cec063c60 0x7f8cec068a70 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.449+0000 7f8cf0a79700 1 -- 172.21.15.121:0/1236991243 >> 172.21.15.121:0/1236991243 conn(0x7f8cec065f10 msgr2=0x7f8cec0662f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:53.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.449+0000 7f8cf0a79700 1 -- 172.21.15.121:0/1236991243 shutdown_connections 2023-11-15T03:10:53.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.449+0000 7f8cf0a79700 1 -- 172.21.15.121:0/1236991243 wait complete. 2023-11-15T03:10:53.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.453+0000 7f8cf0a79700 1 Processor -- start 2023-11-15T03:10:53.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.453+0000 7f8cf0a79700 1 -- start start 2023-11-15T03:10:53.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cf0a79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cec063c60 0x7f8cec0bf370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:53.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cf0a79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cec0a9260 0x7f8cec13ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:53.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cf0a79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cec0a9640 0x7f8cec0c0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:53.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cf0a79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec0c17f0 con 0x7f8cec0a9640 2023-11-15T03:10:53.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cf0a79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec13bf00 con 0x7f8cec0a9260 2023-11-15T03:10:53.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cf0a79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec0c0be0 con 0x7f8cec063c60 2023-11-15T03:10:53.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cebfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cec0a9640 0x7f8cec0c0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:53.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cebfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cec0a9640 0x7f8cec0c0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56732/0 (socket says 172.21.15.121:56732) 2023-11-15T03:10:53.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cebfff700 1 -- 172.21.15.121:0/350891146 learned_addr learned my addr 172.21.15.121:0/350891146 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:53.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8ceb7fe700 1 --2- 172.21.15.121:0/350891146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cec063c60 0x7f8cec0bf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:53.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8ceaffd700 1 --2- 172.21.15.121:0/350891146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cec0a9260 0x7f8cec13ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:53.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cebfff700 1 -- 172.21.15.121:0/350891146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cec063c60 msgr2=0x7f8cec0bf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:53.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cebfff700 1 --2- 172.21.15.121:0/350891146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cec063c60 0x7f8cec0bf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cebfff700 1 -- 172.21.15.121:0/350891146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cec0a9260 msgr2=0x7f8cec13ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:53.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cebfff700 1 --2- 172.21.15.121:0/350891146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cec0a9260 0x7f8cec13ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cebfff700 1 -- 172.21.15.121:0/350891146 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce4009a20 con 0x7f8cec0a9640 2023-11-15T03:10:53.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cebfff700 1 --2- 172.21.15.121:0/350891146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cec0a9640 0x7f8cec0c0340 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 rx=0x7f8ce40099d0 tx=0x7f8ce4009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:53.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cd37fe700 1 -- 172.21.15.121:0/350891146 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4006320 con 0x7f8cec0a9640 2023-11-15T03:10:53.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cd37fe700 1 -- 172.21.15.121:0/350891146 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce4006480 con 0x7f8cec0a9640 2023-11-15T03:10:53.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cf0a79700 1 -- 172.21.15.121:0/350891146 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cec0c0ec0 con 0x7f8cec0a9640 2023-11-15T03:10:53.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cd37fe700 1 -- 172.21.15.121:0/350891146 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4006800 con 0x7f8cec0a9640 2023-11-15T03:10:53.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.457+0000 7f8cf0a79700 1 -- 172.21.15.121:0/350891146 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cec0c1480 con 0x7f8cec0a9640 2023-11-15T03:10:53.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.461+0000 7f8cf0a79700 1 -- 172.21.15.121:0/350891146 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cec0c19a0 con 0x7f8cec0a9640 2023-11-15T03:10:53.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.461+0000 7f8cd37fe700 1 -- 172.21.15.121:0/350891146 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8ce4027460 con 0x7f8cec0a9640 2023-11-15T03:10:53.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.465+0000 7f8cd37fe700 1 --2- 172.21.15.121:0/350891146 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8cd805a510 0x7f8cd805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:53.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.465+0000 7f8cd37fe700 1 -- 172.21.15.121:0/350891146 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 9620+0+0 (secure 0 0 0) 0x7f8ce4077b70 con 0x7f8cec0a9640 2023-11-15T03:10:53.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.465+0000 7f8ceb7fe700 1 --2- 172.21.15.121:0/350891146 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8cd805a510 0x7f8cd805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:53.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.465+0000 7f8ceb7fe700 1 --2- 172.21.15.121:0/350891146 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8cd805a510 0x7f8cd805c9d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7f8cdc007fd0 tx=0x7f8cdc007d30).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:53.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.469+0000 7f8cd37fe700 1 -- 172.21.15.121:0/350891146 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8cec0c19a0 con 0x7f8cec0a9640 2023-11-15T03:10:53.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.665+0000 7f8cf0a79700 1 -- 172.21.15.121:0/350891146 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cec06bcf0 con 0x7f8cec0a9640 2023-11-15T03:10:53.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.669+0000 7f8cd37fe700 1 -- 172.21.15.121:0/350891146 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 334 v334) v1 ==== 95+0+4497 (secure 0 0 0) 0x7f8ce404a480 con 0x7f8cec0a9640 2023-11-15T03:10:53.675 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:10:53.675 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":334,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":10461,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3720977338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3720977338},{"type":"v1","addr":"172.21.15.121:6836","nonce":3720977338}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":334}],"filesystems":[{"mdsmap":{"epoch":333,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:10:48.444387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:10:53.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.673+0000 7f8cf0a79700 1 -- 172.21.15.121:0/350891146 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8cd805a510 msgr2=0x7f8cd805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:53.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.673+0000 7f8cf0a79700 1 --2- 172.21.15.121:0/350891146 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8cd805a510 0x7f8cd805c9d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7f8cdc007fd0 tx=0x7f8cdc007d30).stop 2023-11-15T03:10:53.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.673+0000 7f8cf0a79700 1 -- 172.21.15.121:0/350891146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cec0a9640 msgr2=0x7f8cec0c0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:53.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.673+0000 7f8cf0a79700 1 --2- 172.21.15.121:0/350891146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cec0a9640 0x7f8cec0c0340 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 rx=0x7f8ce40099d0 tx=0x7f8ce4009780).stop 2023-11-15T03:10:53.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.673+0000 7f8cf0a79700 1 -- 172.21.15.121:0/350891146 shutdown_connections 2023-11-15T03:10:53.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.673+0000 7f8cf0a79700 1 --2- 172.21.15.121:0/350891146 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f8cd805a510 0x7f8cd805c9d0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.673+0000 7f8cf0a79700 1 --2- 172.21.15.121:0/350891146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cec0a9640 0x7f8cec0c0340 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.673+0000 7f8cf0a79700 1 --2- 172.21.15.121:0/350891146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cec0a9260 0x7f8cec13ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.673+0000 7f8cf0a79700 1 --2- 172.21.15.121:0/350891146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cec063c60 0x7f8cec0bf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.673+0000 7f8cf0a79700 1 -- 172.21.15.121:0/350891146 >> 172.21.15.121:0/350891146 conn(0x7f8cec065f10 msgr2=0x7f8cec0689a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:53.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.673+0000 7f8cf0a79700 1 -- 172.21.15.121:0/350891146 shutdown_connections 2023-11-15T03:10:53.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.673+0000 7f8cf0a79700 1 -- 172.21.15.121:0/350891146 wait complete. 2023-11-15T03:10:53.682 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 334 2023-11-15T03:10:53.701 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:10:53.701 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:10:53.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.861+0000 7f093b1f9700 1 Processor -- start 2023-11-15T03:10:53.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.861+0000 7f093b1f9700 1 -- start start 2023-11-15T03:10:53.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.861+0000 7f093b1f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09340c3280 0x7f09340c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:53.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.861+0000 7f093b1f9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0934152140 0x7f09341525c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:53.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.861+0000 7f093b1f9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09340c1d50 con 0x7f093415be30 2023-11-15T03:10:53.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.861+0000 7f093b1f9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09340c1860 con 0x7f0934152140 2023-11-15T03:10:53.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.861+0000 7f093b1f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093415cb80 con 0x7f09340c3280 2023-11-15T03:10:53.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f09399f6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0934152140 0x7f09341525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:53.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f09399f6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0934152140 0x7f09341525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56746/0 (socket says 172.21.15.121:56746) 2023-11-15T03:10:53.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f09399f6700 1 -- 172.21.15.121:0/3879758872 learned_addr learned my addr 172.21.15.121:0/3879758872 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:53.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f093a1f7700 1 --2- 172.21.15.121:0/3879758872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09340c3280 0x7f09340c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:53.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f09399f6700 1 -- 172.21.15.121:0/3879758872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09340c3280 msgr2=0x7f09340c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:53.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f09399f6700 1 --2- 172.21.15.121:0/3879758872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09340c3280 0x7f09340c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f09399f6700 1 -- 172.21.15.121:0/3879758872 >> v1:172.21.15.190:6789/0 conn(0x7f093415be30 legacy=0x7f093415c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:53.874 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f09399f6700 1 -- 172.21.15.121:0/3879758872 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09340a9240 con 0x7f0934152140 2023-11-15T03:10:53.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f09399f6700 1 --2- 172.21.15.121:0/3879758872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0934152140 0x7f09341525c0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 rx=0x7f0924006fd0 tx=0x7f09240090f0).ready entity=mon.0 client_cookie=b6f08e8a7a7cdc60 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:53.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f09391f5700 1 -- 172.21.15.121:0/3879758872 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092400b580 con 0x7f0934152140 2023-11-15T03:10:53.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f09391f5700 1 -- 172.21.15.121:0/3879758872 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092400b6e0 con 0x7f0934152140 2023-11-15T03:10:53.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f09391f5700 1 -- 172.21.15.121:0/3879758872 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092400ba60 con 0x7f0934152140 2023-11-15T03:10:53.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f093b1f9700 1 -- 172.21.15.121:0/3879758872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0934152140 msgr2=0x7f09341525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:53.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f093b1f9700 1 --2- 172.21.15.121:0/3879758872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0934152140 0x7f09341525c0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 rx=0x7f0924006fd0 tx=0x7f09240090f0).stop 2023-11-15T03:10:53.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f093b1f9700 1 -- 172.21.15.121:0/3879758872 shutdown_connections 2023-11-15T03:10:53.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f093b1f9700 1 --2- 172.21.15.121:0/3879758872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0934152140 0x7f09341525c0 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f093b1f9700 1 --2- 172.21.15.121:0/3879758872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09340c3280 0x7f09340c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f093b1f9700 1 -- 172.21.15.121:0/3879758872 >> 172.21.15.121:0/3879758872 conn(0x7f0934062190 msgr2=0x7f0934062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:53.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f093b1f9700 1 -- 172.21.15.121:0/3879758872 shutdown_connections 2023-11-15T03:10:53.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.865+0000 7f093b1f9700 1 -- 172.21.15.121:0/3879758872 wait complete. 2023-11-15T03:10:53.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.869+0000 7f093b1f9700 1 Processor -- start 2023-11-15T03:10:53.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.869+0000 7f093b1f9700 1 -- start start 2023-11-15T03:10:53.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.869+0000 7f093b1f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09340c3280 0x7f0934168610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:53.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.869+0000 7f093b1f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0934152140 0x7f09340c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:53.880 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.869+0000 7f093b1f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f093415be30 0x7f0934169810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:53.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.869+0000 7f093b1f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09340c63b0 con 0x7f0934152140 2023-11-15T03:10:53.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.869+0000 7f093b1f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0934157a70 con 0x7f09340c3280 2023-11-15T03:10:53.881 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.869+0000 7f093b1f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09340c6c50 con 0x7f093415be30 2023-11-15T03:10:53.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.869+0000 7f09399f6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0934152140 0x7f09340c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:53.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.869+0000 7f09399f6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0934152140 0x7f09340c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56752/0 (socket says 172.21.15.121:56752) 2023-11-15T03:10:53.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.869+0000 7f09399f6700 1 -- 172.21.15.121:0/488254941 learned_addr learned my addr 172.21.15.121:0/488254941 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:53.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.869+0000 7f093a1f7700 1 --2- 172.21.15.121:0/488254941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09340c3280 0x7f0934168610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:53.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.869+0000 7f093a9f8700 1 --2- 172.21.15.121:0/488254941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f093415be30 0x7f0934169810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:53.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.873+0000 7f09399f6700 1 -- 172.21.15.121:0/488254941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f093415be30 msgr2=0x7f0934169810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:53.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.873+0000 7f09399f6700 1 --2- 172.21.15.121:0/488254941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f093415be30 0x7f0934169810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.873+0000 7f09399f6700 1 -- 172.21.15.121:0/488254941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09340c3280 msgr2=0x7f0934168610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:53.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.873+0000 7f09399f6700 1 --2- 172.21.15.121:0/488254941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09340c3280 0x7f0934168610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:53.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.873+0000 7f09399f6700 1 -- 172.21.15.121:0/488254941 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0924009370 con 0x7f0934152140 2023-11-15T03:10:53.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.873+0000 7f09399f6700 1 --2- 172.21.15.121:0/488254941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0934152140 0x7f09340c5ec0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 rx=0x7f0924009130 tx=0x7f0924017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:53.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.873+0000 7f092a7fc700 1 -- 172.21.15.121:0/488254941 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924000bc0 con 0x7f0934152140 2023-11-15T03:10:53.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.873+0000 7f092a7fc700 1 -- 172.21.15.121:0/488254941 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092400b850 con 0x7f0934152140 2023-11-15T03:10:53.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.873+0000 7f093b1f9700 1 -- 172.21.15.121:0/488254941 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0934168bd0 con 0x7f0934152140 2023-11-15T03:10:53.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.873+0000 7f092a7fc700 1 -- 172.21.15.121:0/488254941 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924000d20 con 0x7f0934152140 2023-11-15T03:10:53.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.873+0000 7f093b1f9700 1 -- 172.21.15.121:0/488254941 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0934169190 con 0x7f0934152140 2023-11-15T03:10:53.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.873+0000 7f093b1f9700 1 -- 172.21.15.121:0/488254941 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093406bcf0 con 0x7f0934152140 2023-11-15T03:10:53.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.877+0000 7f092a7fc700 1 -- 172.21.15.121:0/488254941 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f092401c8c0 con 0x7f0934152140 2023-11-15T03:10:53.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.877+0000 7f092a7fc700 1 --2- 172.21.15.121:0/488254941 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f091805a510 0x7f091805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:53.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.877+0000 7f092a7fc700 1 -- 172.21.15.121:0/488254941 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 9620+0+0 (secure 0 0 0) 0x7f09240772e0 con 0x7f0934152140 2023-11-15T03:10:53.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.877+0000 7f093a1f7700 1 --2- 172.21.15.121:0/488254941 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f091805a510 0x7f091805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:53.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.877+0000 7f093a1f7700 1 --2- 172.21.15.121:0/488254941 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f091805a510 0x7f091805c9d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f0930009c30 tx=0x7f0930009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:53.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:53.885+0000 7f092a7fc700 1 -- 172.21.15.121:0/488254941 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0924049c50 con 0x7f0934152140 2023-11-15T03:10:54.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.089+0000 7f093b1f9700 1 -- 172.21.15.121:0/488254941 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09340c1f80 con 0x7f0934152140 2023-11-15T03:10:54.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.089+0000 7f092a7fc700 1 -- 172.21.15.121:0/488254941 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 334 v334) v1 ==== 95+0+4497 (secure 0 0 0) 0x7f092402a030 con 0x7f0934152140 2023-11-15T03:10:54.099 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:10:54.099 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":334,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":10461,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3720977338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3720977338},{"type":"v1","addr":"172.21.15.121:6836","nonce":3720977338}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":334}],"filesystems":[{"mdsmap":{"epoch":333,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:10:48.444387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:10:54.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.093+0000 7f093b1f9700 1 -- 172.21.15.121:0/488254941 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f091805a510 msgr2=0x7f091805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:54.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.093+0000 7f093b1f9700 1 --2- 172.21.15.121:0/488254941 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f091805a510 0x7f091805c9d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f0930009c30 tx=0x7f0930009060).stop 2023-11-15T03:10:54.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.093+0000 7f093b1f9700 1 -- 172.21.15.121:0/488254941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0934152140 msgr2=0x7f09340c5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:54.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.093+0000 7f093b1f9700 1 --2- 172.21.15.121:0/488254941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0934152140 0x7f09340c5ec0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 rx=0x7f0924009130 tx=0x7f0924017000).stop 2023-11-15T03:10:54.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.093+0000 7f093b1f9700 1 -- 172.21.15.121:0/488254941 shutdown_connections 2023-11-15T03:10:54.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.093+0000 7f093b1f9700 1 --2- 172.21.15.121:0/488254941 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f091805a510 0x7f091805c9d0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:54.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.093+0000 7f093b1f9700 1 --2- 172.21.15.121:0/488254941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f093415be30 0x7f0934169810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:54.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.093+0000 7f093b1f9700 1 --2- 172.21.15.121:0/488254941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0934152140 0x7f09340c5ec0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:54.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.093+0000 7f093b1f9700 1 --2- 172.21.15.121:0/488254941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09340c3280 0x7f0934168610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:54.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.093+0000 7f093b1f9700 1 -- 172.21.15.121:0/488254941 >> 172.21.15.121:0/488254941 conn(0x7f0934062190 msgr2=0x7f09340a8750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:54.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.093+0000 7f093b1f9700 1 -- 172.21.15.121:0/488254941 shutdown_connections 2023-11-15T03:10:54.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.093+0000 7f093b1f9700 1 -- 172.21.15.121:0/488254941 wait complete. 2023-11-15T03:10:54.104 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 334 2023-11-15T03:10:54.123 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on mds0_sessionmap' led to DAMAGED state 2023-11-15T03:10:54.124 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 1.00000000' 2023-11-15T03:10:54.124 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:10:54.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:10:54.125 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:10:54.142 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-15T03:10:54.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:10:54.145 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:10:54.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.337+0000 7f5e0359e700 1 Processor -- start 2023-11-15T03:10:54.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.337+0000 7f5e0359e700 1 -- start start 2023-11-15T03:10:54.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.341+0000 7f5e0359e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e04066570 0x7f5e041515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:54.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.341+0000 7f5e0359e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5e04151ac0 0x7f5e040a8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:54.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.341+0000 7f5e0359e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e040660e0 con 0x7f5e041511f0 2023-11-15T03:10:54.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.341+0000 7f5e0359e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e040669f0 con 0x7f5e04151ac0 2023-11-15T03:10:54.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.341+0000 7f5e0359e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040bd390 con 0x7f5e04066570 2023-11-15T03:10:54.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.341+0000 7f5e0259c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f5e041511f0 0x7f5e040bcab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:52296/0 (socket says 172.21.15.121:52296) 2023-11-15T03:10:54.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.341+0000 7f5e0259c700 1 -- 172.21.15.121:0/342264271 learned_addr learned my addr 172.21.15.121:0/342264271 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:54.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.341+0000 7f5e02d9d700 1 --2- 172.21.15.121:0/342264271 >> v2:172.21.15.190:3300/0 conn(0x7f5e04151ac0 0x7f5e040a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:54.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.341+0000 7f5e01d9b700 1 --2- 172.21.15.121:0/342264271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e04066570 0x7f5e041515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:54.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.345+0000 7f5e0159a700 1 -- 172.21.15.121:0/342264271 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2741693510 0 0) 0x7f5e040660e0 con 0x7f5e041511f0 2023-11-15T03:10:54.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.345+0000 7f5e0159a700 1 -- 172.21.15.121:0/342264271 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dec0033f0 con 0x7f5e041511f0 2023-11-15T03:10:54.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.345+0000 7f5e02d9d700 1 -- 172.21.15.121:0/342264271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e04066570 msgr2=0x7f5e041515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:54.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.345+0000 7f5e02d9d700 1 --2- 172.21.15.121:0/342264271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e04066570 0x7f5e041515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:54.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.345+0000 7f5e02d9d700 1 -- 172.21.15.121:0/342264271 >> v1:172.21.15.121:6789/0 conn(0x7f5e041511f0 legacy=0x7f5e040bcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:54.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.345+0000 7f5e02d9d700 1 -- 172.21.15.121:0/342264271 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e040657b0 con 0x7f5e04151ac0 2023-11-15T03:10:54.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.345+0000 7f5e02d9d700 1 --2- 172.21.15.121:0/342264271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e04151ac0 0x7f5e040a8750 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f5dfc009e60 tx=0x7f5dfc009780).ready entity=mon.1 client_cookie=bab6a9b171a6f6d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:54.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.345+0000 7f5e0159a700 1 -- 172.21.15.121:0/342264271 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc00e070 con 0x7f5e04151ac0 2023-11-15T03:10:54.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.345+0000 7f5e0159a700 1 -- 172.21.15.121:0/342264271 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dfc00bab0 con 0x7f5e04151ac0 2023-11-15T03:10:54.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.345+0000 7f5e0159a700 1 -- 172.21.15.121:0/342264271 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc00be30 con 0x7f5e04151ac0 2023-11-15T03:10:54.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.345+0000 7f5e0359e700 1 -- 172.21.15.121:0/342264271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e04151ac0 msgr2=0x7f5e040a8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:54.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.345+0000 7f5e0359e700 1 --2- 172.21.15.121:0/342264271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e04151ac0 0x7f5e040a8750 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f5dfc009e60 tx=0x7f5dfc009780).stop 2023-11-15T03:10:54.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.345+0000 7f5e0359e700 1 -- 172.21.15.121:0/342264271 shutdown_connections 2023-11-15T03:10:54.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.345+0000 7f5e0359e700 1 --2- 172.21.15.121:0/342264271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e04151ac0 0x7f5e040a8750 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:54.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.345+0000 7f5e0359e700 1 --2- 172.21.15.121:0/342264271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e04066570 0x7f5e041515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:54.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.345+0000 7f5e0359e700 1 -- 172.21.15.121:0/342264271 >> 172.21.15.121:0/342264271 conn(0x7f5e04067a60 msgr2=0x7f5e04067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:54.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.349+0000 7f5e0359e700 1 -- 172.21.15.121:0/342264271 shutdown_connections 2023-11-15T03:10:54.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.349+0000 7f5e0359e700 1 -- 172.21.15.121:0/342264271 wait complete. 2023-11-15T03:10:54.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.349+0000 7f5e0359e700 1 Processor -- start 2023-11-15T03:10:54.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.353+0000 7f5e0359e700 1 -- start start 2023-11-15T03:10:54.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.353+0000 7f5e0359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e04066570 0x7f5e04154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:54.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.353+0000 7f5e0359e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e041511f0 0x7f5e0414f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:54.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.353+0000 7f5e0359e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e04151ac0 0x7f5e040c5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:54.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.353+0000 7f5e0359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0414b370 con 0x7f5e04066570 2023-11-15T03:10:54.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.353+0000 7f5e0359e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e04154580 con 0x7f5e04151ac0 2023-11-15T03:10:54.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.353+0000 7f5e0359e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e04152820 con 0x7f5e041511f0 2023-11-15T03:10:54.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.353+0000 7f5e0259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e04066570 0x7f5e04154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:54.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.353+0000 7f5e0259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e04066570 0x7f5e04154a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56768/0 (socket says 172.21.15.121:56768) 2023-11-15T03:10:54.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.353+0000 7f5e0259c700 1 -- 172.21.15.121:0/2070937173 learned_addr learned my addr 172.21.15.121:0/2070937173 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:54.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.353+0000 7f5e01d9b700 1 --2- 172.21.15.121:0/2070937173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e041511f0 0x7f5e0414f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:54.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.353+0000 7f5e02d9d700 1 --2- 172.21.15.121:0/2070937173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e04151ac0 0x7f5e040c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:54.372 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.353+0000 7f5e0259c700 1 -- 172.21.15.121:0/2070937173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e041511f0 msgr2=0x7f5e0414f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:54.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.353+0000 7f5e0259c700 1 --2- 172.21.15.121:0/2070937173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e041511f0 0x7f5e0414f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:54.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.353+0000 7f5e0259c700 1 -- 172.21.15.121:0/2070937173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e04151ac0 msgr2=0x7f5e040c5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:54.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.353+0000 7f5e0259c700 1 --2- 172.21.15.121:0/2070937173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e04151ac0 0x7f5e040c5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:54.373 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.353+0000 7f5e0259c700 1 -- 172.21.15.121:0/2070937173 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dfc009a20 con 0x7f5e04066570 2023-11-15T03:10:54.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.357+0000 7f5e0259c700 1 --2- 172.21.15.121:0/2070937173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e04066570 0x7f5e04154a10 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 rx=0x7f5df400c420 tx=0x7f5df4007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:54.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.357+0000 7f5df27fc700 1 -- 172.21.15.121:0/2070937173 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df40070d0 con 0x7f5e04066570 2023-11-15T03:10:54.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.357+0000 7f5df27fc700 1 -- 172.21.15.121:0/2070937173 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df4007230 con 0x7f5e04066570 2023-11-15T03:10:54.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.357+0000 7f5df27fc700 1 -- 172.21.15.121:0/2070937173 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df40075b0 con 0x7f5e04066570 2023-11-15T03:10:54.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.357+0000 7f5e0359e700 1 -- 172.21.15.121:0/2070937173 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e04153c90 con 0x7f5e04066570 2023-11-15T03:10:54.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.357+0000 7f5e0359e700 1 -- 172.21.15.121:0/2070937173 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e04153fb0 con 0x7f5e04066570 2023-11-15T03:10:54.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.357+0000 7f5e0359e700 1 -- 172.21.15.121:0/2070937173 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e0406bcf0 con 0x7f5e04066570 2023-11-15T03:10:54.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.361+0000 7f5df27fc700 1 -- 172.21.15.121:0/2070937173 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5df40059b0 con 0x7f5e04066570 2023-11-15T03:10:54.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.361+0000 7f5df27fc700 1 --2- 172.21.15.121:0/2070937173 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5de405a510 0x7f5de405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:54.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.361+0000 7f5df27fc700 1 -- 172.21.15.121:0/2070937173 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 9620+0+0 (secure 0 0 0) 0x7f5df4074c10 con 0x7f5e04066570 2023-11-15T03:10:54.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.361+0000 7f5e01d9b700 1 --2- 172.21.15.121:0/2070937173 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5de405a510 0x7f5de405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:54.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.361+0000 7f5e01d9b700 1 --2- 172.21.15.121:0/2070937173 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5de405a510 0x7f5de405c9d0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7f5e040a8d70 tx=0x7f5df8006f40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:54.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.365+0000 7f5df27fc700 1 -- 172.21.15.121:0/2070937173 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5df4047600 con 0x7f5e04066570 2023-11-15T03:10:54.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:54.573+0000 7f5e0359e700 1 -- 172.21.15.121:0/2070937173 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5e041515d0 con 0x7f5e04066570 2023-11-15T03:10:55.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.221+0000 7f5df27fc700 1 -- 172.21.15.121:0/2070937173 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v335) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5df404ac20 con 0x7f5e04066570 2023-11-15T03:10:55.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.225+0000 7f5e0359e700 1 -- 172.21.15.121:0/2070937173 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5de405a510 msgr2=0x7f5de405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:55.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.225+0000 7f5e0359e700 1 --2- 172.21.15.121:0/2070937173 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5de405a510 0x7f5de405c9d0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7f5e040a8d70 tx=0x7f5df8006f40).stop 2023-11-15T03:10:55.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.225+0000 7f5e0359e700 1 -- 172.21.15.121:0/2070937173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e04066570 msgr2=0x7f5e04154a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:55.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.225+0000 7f5e0359e700 1 --2- 172.21.15.121:0/2070937173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e04066570 0x7f5e04154a10 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 rx=0x7f5df400c420 tx=0x7f5df4007c80).stop 2023-11-15T03:10:55.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.225+0000 7f5e0359e700 1 -- 172.21.15.121:0/2070937173 shutdown_connections 2023-11-15T03:10:55.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.225+0000 7f5e0359e700 1 --2- 172.21.15.121:0/2070937173 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5de405a510 0x7f5de405c9d0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:55.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.225+0000 7f5e0359e700 1 --2- 172.21.15.121:0/2070937173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e04151ac0 0x7f5e040c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:55.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.225+0000 7f5e0359e700 1 --2- 172.21.15.121:0/2070937173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e041511f0 0x7f5e0414f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:55.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.225+0000 7f5e0359e700 1 --2- 172.21.15.121:0/2070937173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e04066570 0x7f5e04154a10 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:55.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.225+0000 7f5e0359e700 1 -- 172.21.15.121:0/2070937173 >> 172.21.15.121:0/2070937173 conn(0x7f5e04067a60 msgr2=0x7f5e040be170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:55.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.225+0000 7f5e0359e700 1 -- 172.21.15.121:0/2070937173 shutdown_connections 2023-11-15T03:10:55.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.225+0000 7f5e0359e700 1 -- 172.21.15.121:0/2070937173 wait complete. 2023-11-15T03:10:55.237 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:10:55.254 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-15T03:10:55.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.409+0000 7f46a3107700 1 Processor -- start 2023-11-15T03:10:55.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.409+0000 7f46a3107700 1 -- start start 2023-11-15T03:10:55.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.413+0000 7f46a3107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469c153990 0x7f469c153d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:55.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.413+0000 7f46a3107700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f469c0bb910 0x7f469c1581e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:55.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.413+0000 7f46a3107700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469c0a8b20 con 0x7f469c0c2ec0 2023-11-15T03:10:55.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.413+0000 7f46a3107700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469c0c1760 con 0x7f469c0bb910 2023-11-15T03:10:55.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.413+0000 7f46a3107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469c0a8d50 con 0x7f469c153990 2023-11-15T03:10:55.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.413+0000 7f46a2105700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469c153990 0x7f469c153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:55.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.413+0000 7f46a2906700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f469c0c2ec0 0x7f469c0c3330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58732/0 (socket says 172.21.15.121:58732) 2023-11-15T03:10:55.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.413+0000 7f46a2906700 1 -- 172.21.15.121:0/185744929 learned_addr learned my addr 172.21.15.121:0/185744929 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:55.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.413+0000 7f46a1904700 1 --2- 172.21.15.121:0/185744929 >> v2:172.21.15.121:3300/0 conn(0x7f469c0bb910 0x7f469c1581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:55.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.413+0000 7f46a1103700 1 -- 172.21.15.121:0/185744929 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663005787 0 0) 0x7f469c0a8b20 con 0x7f469c0c2ec0 2023-11-15T03:10:55.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.413+0000 7f46a1103700 1 -- 172.21.15.121:0/185744929 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46880033f0 con 0x7f469c0c2ec0 2023-11-15T03:10:55.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.413+0000 7f46a1904700 1 -- 172.21.15.121:0/185744929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469c153990 msgr2=0x7f469c153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:55.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.413+0000 7f46a1904700 1 --2- 172.21.15.121:0/185744929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469c153990 0x7f469c153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:55.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.413+0000 7f46a1904700 1 -- 172.21.15.121:0/185744929 >> v1:172.21.15.121:6789/0 conn(0x7f469c0c2ec0 legacy=0x7f469c0c3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:55.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.413+0000 7f46a1904700 1 -- 172.21.15.121:0/185744929 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c0a9030 con 0x7f469c0bb910 2023-11-15T03:10:55.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.413+0000 7f46a1904700 1 --2- 172.21.15.121:0/185744929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f469c0bb910 0x7f469c1581e0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 rx=0x7f468c006fd0 tx=0x7f468c006f40).ready entity=mon.0 client_cookie=edc2c62e65badaf6 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:55.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.413+0000 7f46a1103700 1 -- 172.21.15.121:0/185744929 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c005880 con 0x7f469c0bb910 2023-11-15T03:10:55.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.413+0000 7f46a1103700 1 -- 172.21.15.121:0/185744929 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468c0059e0 con 0x7f469c0bb910 2023-11-15T03:10:55.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.417+0000 7f46a1103700 1 -- 172.21.15.121:0/185744929 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c005d60 con 0x7f469c0bb910 2023-11-15T03:10:55.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.417+0000 7f46a3107700 1 -- 172.21.15.121:0/185744929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f469c0bb910 msgr2=0x7f469c1581e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:55.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.417+0000 7f46a3107700 1 --2- 172.21.15.121:0/185744929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f469c0bb910 0x7f469c1581e0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 rx=0x7f468c006fd0 tx=0x7f468c006f40).stop 2023-11-15T03:10:55.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.417+0000 7f46a3107700 1 -- 172.21.15.121:0/185744929 shutdown_connections 2023-11-15T03:10:55.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.417+0000 7f46a3107700 1 --2- 172.21.15.121:0/185744929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f469c0bb910 0x7f469c1581e0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:55.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.417+0000 7f46a3107700 1 --2- 172.21.15.121:0/185744929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469c153990 0x7f469c153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:55.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.417+0000 7f46a3107700 1 -- 172.21.15.121:0/185744929 >> 172.21.15.121:0/185744929 conn(0x7f469c067a60 msgr2=0x7f469c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:55.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.417+0000 7f46a3107700 1 -- 172.21.15.121:0/185744929 shutdown_connections 2023-11-15T03:10:55.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.417+0000 7f46a3107700 1 -- 172.21.15.121:0/185744929 wait complete. 2023-11-15T03:10:55.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.417+0000 7f46a3107700 1 Processor -- start 2023-11-15T03:10:55.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.417+0000 7f46a3107700 1 -- start start 2023-11-15T03:10:55.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a3107700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f469c0bb910 0x7f469c0becb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:55.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a3107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469c0c2ec0 0x7f469c0bf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:55.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a3107700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f469c153990 0x7f469c0c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:55.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a3107700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c0c04a0 con 0x7f469c0bb910 2023-11-15T03:10:55.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a3107700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c0a9ec0 con 0x7f469c153990 2023-11-15T03:10:55.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a3107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469c0bf820 con 0x7f469c0c2ec0 2023-11-15T03:10:55.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a2105700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f469c0bb910 0x7f469c0becb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:55.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a2105700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f469c0bb910 0x7f469c0becb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39774/0 (socket says 172.21.15.121:39774) 2023-11-15T03:10:55.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a2105700 1 -- 172.21.15.121:0/311625563 learned_addr learned my addr 172.21.15.121:0/311625563 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:55.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a1904700 1 --2- 172.21.15.121:0/311625563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469c0c2ec0 0x7f469c0bf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:55.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a2906700 1 --2- 172.21.15.121:0/311625563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f469c153990 0x7f469c0c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:55.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a2105700 1 -- 172.21.15.121:0/311625563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469c0c2ec0 msgr2=0x7f469c0bf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:55.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a2105700 1 --2- 172.21.15.121:0/311625563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469c0c2ec0 0x7f469c0bf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:55.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a2105700 1 -- 172.21.15.121:0/311625563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f469c153990 msgr2=0x7f469c0c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:55.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a2105700 1 --2- 172.21.15.121:0/311625563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f469c153990 0x7f469c0c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:55.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a2105700 1 -- 172.21.15.121:0/311625563 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c009250 con 0x7f469c0bb910 2023-11-15T03:10:55.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a1904700 1 --2- 172.21.15.121:0/311625563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469c0c2ec0 0x7f469c0bf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:10:55.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a2906700 1 --2- 172.21.15.121:0/311625563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f469c153990 0x7f469c0c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:10:55.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a2105700 1 --2- 172.21.15.121:0/311625563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f469c0bb910 0x7f469c0becb0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 rx=0x7f469800e930 tx=0x7f4698009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:55.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46927fc700 1 -- 172.21.15.121:0/311625563 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469800c640 con 0x7f469c0bb910 2023-11-15T03:10:55.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46927fc700 1 -- 172.21.15.121:0/311625563 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469800c7a0 con 0x7f469c0bb910 2023-11-15T03:10:55.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a3107700 1 -- 172.21.15.121:0/311625563 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469c0bfb00 con 0x7f469c0bb910 2023-11-15T03:10:55.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46927fc700 1 -- 172.21.15.121:0/311625563 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469800cb20 con 0x7f469c0bb910 2023-11-15T03:10:55.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.421+0000 7f46a3107700 1 -- 172.21.15.121:0/311625563 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469c0c00c0 con 0x7f469c0bb910 2023-11-15T03:10:55.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.425+0000 7f46a3107700 1 -- 172.21.15.121:0/311625563 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f469c06bcf0 con 0x7f469c0bb910 2023-11-15T03:10:55.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.425+0000 7f46927fc700 1 -- 172.21.15.121:0/311625563 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f469801b400 con 0x7f469c0bb910 2023-11-15T03:10:55.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.425+0000 7f46927fc700 1 --2- 172.21.15.121:0/311625563 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f468005a510 0x7f468005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:55.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.425+0000 7f46927fc700 1 -- 172.21.15.121:0/311625563 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 9620+0+0 (secure 0 0 0) 0x7f46980759c0 con 0x7f469c0bb910 2023-11-15T03:10:55.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.425+0000 7f46a1904700 1 --2- 172.21.15.121:0/311625563 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f468005a510 0x7f468005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:55.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.429+0000 7f46a1904700 1 --2- 172.21.15.121:0/311625563 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f468005a510 0x7f468005c9d0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f468c006fd0 tx=0x7f468c015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:55.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.433+0000 7f46927fc700 1 -- 172.21.15.121:0/311625563 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4698048350 con 0x7f469c0bb910 2023-11-15T03:10:55.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:55.653+0000 7f46a3107700 1 -- 172.21.15.121:0/311625563 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f469c153d70 con 0x7f469c0bb910 2023-11-15T03:10:56.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.229+0000 7f46927fc700 1 -- 172.21.15.121:0/311625563 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v336) v1 ==== 101+0+0 (secure 0 0 0) 0x7f4698016020 con 0x7f469c0bb910 2023-11-15T03:10:56.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.237+0000 7f46a3107700 1 -- 172.21.15.121:0/311625563 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f468005a510 msgr2=0x7f468005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.237+0000 7f46a3107700 1 --2- 172.21.15.121:0/311625563 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f468005a510 0x7f468005c9d0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f468c006fd0 tx=0x7f468c015000).stop 2023-11-15T03:10:56.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.237+0000 7f46a3107700 1 -- 172.21.15.121:0/311625563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f469c0bb910 msgr2=0x7f469c0becb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.237+0000 7f46a3107700 1 --2- 172.21.15.121:0/311625563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f469c0bb910 0x7f469c0becb0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 rx=0x7f469800e930 tx=0x7f4698009060).stop 2023-11-15T03:10:56.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.237+0000 7f46a3107700 1 -- 172.21.15.121:0/311625563 shutdown_connections 2023-11-15T03:10:56.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.237+0000 7f46a3107700 1 --2- 172.21.15.121:0/311625563 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f468005a510 0x7f468005c9d0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.237+0000 7f46a3107700 1 --2- 172.21.15.121:0/311625563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f469c153990 0x7f469c0c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.237+0000 7f46a3107700 1 --2- 172.21.15.121:0/311625563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469c0c2ec0 0x7f469c0bf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.237+0000 7f46a3107700 1 --2- 172.21.15.121:0/311625563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f469c0bb910 0x7f469c0becb0 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.237+0000 7f46a3107700 1 -- 172.21.15.121:0/311625563 >> 172.21.15.121:0/311625563 conn(0x7f469c067a60 msgr2=0x7f469c14b770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:56.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.237+0000 7f46a3107700 1 -- 172.21.15.121:0/311625563 shutdown_connections 2023-11-15T03:10:56.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.237+0000 7f46a3107700 1 -- 172.21.15.121:0/311625563 wait complete. 2023-11-15T03:10:56.246 INFO:teuthology.orchestra.run.smithi121.stderr:repaired: restoring rank 6:0 2023-11-15T03:10:56.266 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-11-15T03:10:56.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65b092fd00 1 Processor -- start 2023-11-15T03:10:56.323 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65b092fd00 1 -- start start 2023-11-15T03:10:56.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65b092fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5621cce49bb0 0x5621ccf1c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.324 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65b092fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5621cce93b00 0x5621ccf1aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65b092fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621ccf26650 0x5621cce7c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65b092fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ccdc2220 con 0x5621ccf26650 2023-11-15T03:10:56.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65b092fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ccf1a3e0 con 0x5621cce93b00 2023-11-15T03:10:56.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65b092fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621ccf19ed0 con 0x5621cce49bb0 2023-11-15T03:10:56.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65a5810700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621ccf26650 0x5621cce7c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65a5810700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621ccf26650 0x5621cce7c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39792/0 (socket says 172.21.15.121:39792) 2023-11-15T03:10:56.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65a5810700 1 -- 172.21.15.121:0/320858933 learned_addr learned my addr 172.21.15.121:0/320858933 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:56.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65a480e700 1 --2- 172.21.15.121:0/320858933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5621cce93b00 0x5621ccf1aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65a500f700 1 --2- 172.21.15.121:0/320858933 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5621cce49bb0 0x5621ccf1c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65a5810700 1 -- 172.21.15.121:0/320858933 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5621cce49bb0 msgr2=0x5621ccf1c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:56.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65a5810700 1 --2- 172.21.15.121:0/320858933 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5621cce49bb0 0x5621ccf1c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65a5810700 1 -- 172.21.15.121:0/320858933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5621cce93b00 msgr2=0x5621ccf1aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:56.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65a5810700 1 --2- 172.21.15.121:0/320858933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5621cce93b00 0x5621ccf1aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65a5810700 1 -- 172.21.15.121:0/320858933 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a000a0e0 con 0x5621ccf26650 2023-11-15T03:10:56.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65a5810700 1 --2- 172.21.15.121:0/320858933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621ccf26650 0x5621cce7c070 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 rx=0x5621cce36e40 tx=0x7f65a0015000).ready entity=mon.0 client_cookie=59e73301cbf02765 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:56.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f659f7fe700 1 -- 172.21.15.121:0/320858933 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a000abc0 con 0x5621ccf26650 2023-11-15T03:10:56.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f659f7fe700 1 -- 172.21.15.121:0/320858933 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a000dcc0 con 0x5621ccf26650 2023-11-15T03:10:56.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f659f7fe700 1 -- 172.21.15.121:0/320858933 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a000d950 con 0x5621ccf26650 2023-11-15T03:10:56.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65b092fd00 1 -- 172.21.15.121:0/320858933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621ccf26650 msgr2=0x5621cce7c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.317+0000 7f65b092fd00 1 --2- 172.21.15.121:0/320858933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621ccf26650 0x5621cce7c070 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 rx=0x5621cce36e40 tx=0x7f65a0015000).stop 2023-11-15T03:10:56.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65b092fd00 1 -- 172.21.15.121:0/320858933 shutdown_connections 2023-11-15T03:10:56.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65b092fd00 1 --2- 172.21.15.121:0/320858933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621ccf26650 0x5621cce7c070 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65b092fd00 1 --2- 172.21.15.121:0/320858933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5621cce93b00 0x5621ccf1aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65b092fd00 1 --2- 172.21.15.121:0/320858933 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5621cce49bb0 0x5621ccf1c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65b092fd00 1 -- 172.21.15.121:0/320858933 >> 172.21.15.121:0/320858933 conn(0x5621cce99080 msgr2=0x5621cce8dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:56.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65b092fd00 1 -- 172.21.15.121:0/320858933 shutdown_connections 2023-11-15T03:10:56.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65b092fd00 1 -- 172.21.15.121:0/320858933 wait complete. 2023-11-15T03:10:56.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65b092fd00 1 Processor -- start 2023-11-15T03:10:56.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65b092fd00 1 -- start start 2023-11-15T03:10:56.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65b092fd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5621cce49bb0 0x5621cce7c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65b092fd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621cce93b00 0x5621ccf24ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65b092fd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5621ccf26650 0x5621ccf332c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65b092fd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ccf32e30 con 0x5621cce93b00 2023-11-15T03:10:56.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65b092fd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ccf32b30 con 0x5621ccf26650 2023-11-15T03:10:56.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65b092fd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621cce7c670 con 0x5621cce49bb0 2023-11-15T03:10:56.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65a480e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621cce93b00 0x5621ccf24ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65a480e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621cce93b00 0x5621ccf24ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39800/0 (socket says 172.21.15.121:39800) 2023-11-15T03:10:56.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 learned_addr learned my addr 172.21.15.121:0/1440087446 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:56.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65a500f700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5621cce49bb0 0x5621cce7c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.321+0000 7f65a5810700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5621ccf26650 0x5621ccf332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.325+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5621cce49bb0 msgr2=0x5621cce7c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.325+0000 7f65a480e700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5621cce49bb0 0x5621cce7c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.325+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5621ccf26650 msgr2=0x5621ccf332c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.325+0000 7f65a480e700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5621ccf26650 0x5621ccf332c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.325+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621ccdeba20 con 0x5621cce93b00 2023-11-15T03:10:56.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.325+0000 7f65a480e700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621cce93b00 0x5621ccf24ad0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 rx=0x7f6590008b20 tx=0x7f659000c040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:56.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.325+0000 7f659dffb700 1 -- 172.21.15.121:0/1440087446 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659000f040 con 0x5621cce93b00 2023-11-15T03:10:56.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.325+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5621ccdeb770 con 0x5621cce93b00 2023-11-15T03:10:56.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.325+0000 7f659dffb700 1 -- 172.21.15.121:0/1440087446 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65900094e0 con 0x5621cce93b00 2023-11-15T03:10:56.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.325+0000 7f659dffb700 1 -- 172.21.15.121:0/1440087446 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590010ac0 con 0x5621cce93b00 2023-11-15T03:10:56.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.325+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621ccdbaa70 con 0x5621cce93b00 2023-11-15T03:10:56.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.329+0000 7f659dffb700 1 -- 172.21.15.121:0/1440087446 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6590015430 con 0x5621cce93b00 2023-11-15T03:10:56.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.329+0000 7f659dffb700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6588058ff0 0x7f658805b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.329+0000 7f659dffb700 1 -- 172.21.15.121:0/1440087446 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 9620+0+0 (secure 0 0 0) 0x7f6590074c70 con 0x5621cce93b00 2023-11-15T03:10:56.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.329+0000 7f65a500f700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6588058ff0 0x7f658805b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.341 INFO:teuthology.orchestra.run.smithi121.stdout:Importing pool 2023-11-15T03:10:56.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.329+0000 7f65b092fd00 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5621ccf22820 0x5621cce93190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.329+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621cce79910 con 0x5621ccf22820 2023-11-15T03:10:56.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.329+0000 7f65a5810700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5621ccf22820 0x5621cce93190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.329+0000 7f65a500f700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6588058ff0 0x7f658805b4b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f6598001ed0 tx=0x7f6598000bc0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:56.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.329+0000 7f65a5810700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5621ccf22820 0x5621cce93190 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:56.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.333+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(1 601.00000000 [create] v133'461 uv454 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf22820 2023-11-15T03:10:56.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.333+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621cce79cf0 con 0x5621ccf22820 2023-11-15T03:10:56.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.337+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== osd_op_reply(2 601.00000000 [delete] v133'462 uv462 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf22820 2023-11-15T03:10:56.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.337+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621cce7a0d0 con 0x5621ccf22820 2023-11-15T03:10:56.345 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-11-15T03:10:56.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.337+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.1 v2:172.21.15.121:6800/33131 3 ==== osd_op_reply(3 601.00000000 [create] v133'463 uv463 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf22820 2023-11-15T03:10:56.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.337+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf0e220 con 0x5621ccf22820 2023-11-15T03:10:56.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.341+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.1 v2:172.21.15.121:6800/33131 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v133'464 uv464 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf22820 2023-11-15T03:10:56.348 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.341+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf0e5d0 con 0x5621ccf22820 2023-11-15T03:10:56.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.341+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.1 v2:172.21.15.121:6800/33131 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v133'465 uv465 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf22820 2023-11-15T03:10:56.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.341+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf0e9d0 con 0x5621ccf22820 2023-11-15T03:10:56.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.345+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.1 v2:172.21.15.121:6800/33131 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v133'466 uv466 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf22820 2023-11-15T03:10:56.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.345+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf0edd0 con 0x5621ccf22820 2023-11-15T03:10:56.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.349+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.1 v2:172.21.15.121:6800/33131 7 ==== osd_op_reply(7 602.00000000 [create] v133'467 uv460 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf22820 2023-11-15T03:10:56.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.349+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf0edd0 con 0x5621ccf22820 2023-11-15T03:10:56.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.349+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.1 v2:172.21.15.121:6800/33131 8 ==== osd_op_reply(8 602.00000000 [delete] v133'468 uv468 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf22820 2023-11-15T03:10:56.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.349+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf0ed80 con 0x5621ccf22820 2023-11-15T03:10:56.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.353+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.1 v2:172.21.15.121:6800/33131 9 ==== osd_op_reply(9 602.00000000 [create] v133'469 uv469 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf22820 2023-11-15T03:10:56.361 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-11-15T03:10:56.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.353+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf0ed80 con 0x5621ccf22820 2023-11-15T03:10:56.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.357+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.1 v2:172.21.15.121:6800/33131 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v133'470 uv470 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf22820 2023-11-15T03:10:56.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.357+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf0ed80 con 0x5621ccf22820 2023-11-15T03:10:56.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.357+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.1 v2:172.21.15.121:6800/33131 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v133'471 uv471 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf22820 2023-11-15T03:10:56.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.357+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf0ed80 con 0x5621ccf22820 2023-11-15T03:10:56.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.361+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.1 v2:172.21.15.121:6800/33131 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v133'472 uv472 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf22820 2023-11-15T03:10:56.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.361+0000 7f65b092fd00 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5621ccf0ed80 0x5621ccf425e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.361+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:13 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf42ad0 con 0x5621ccf0ed80 2023-11-15T03:10:56.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.361+0000 7f65a500f700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5621ccf0ed80 0x5621ccf425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.361+0000 7f65a500f700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5621ccf0ed80 0x5621ccf425e0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:56.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.365+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(13 10000000000.00000000 [create] v133'730 uv717 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.374 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.365+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:14 13.30 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf42e80 con 0x5621ccf0ed80 2023-11-15T03:10:56.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.369+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== osd_op_reply(14 10000000000.00000000 [delete] v133'731 uv731 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.369+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:15 13.30 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf43230 con 0x5621ccf0ed80 2023-11-15T03:10:56.381 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-11-15T03:10:56.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.373+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 3 ==== osd_op_reply(15 10000000000.00000000 [create] v133'732 uv732 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.373+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:16 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf44670 con 0x5621ccf0ed80 2023-11-15T03:10:56.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.377+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 4 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v133'733 uv733 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.377+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:17 13.30 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf44b40 con 0x5621ccf0ed80 2023-11-15T03:10:56.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.381+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 5 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v133'734 uv734 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.381+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:18 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf435e0 con 0x5621ccf0ed80 2023-11-15T03:10:56.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.385+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 6 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v133'735 uv735 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.385+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:19 13.30 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf43ed0 con 0x5621ccf0ed80 2023-11-15T03:10:56.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.389+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 7 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v133'736 uv736 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.389+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:20 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf44f70 con 0x5621ccf0ed80 2023-11-15T03:10:56.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.393+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 8 ==== osd_op_reply(20 600.00000000 [create] v133'737 uv723 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.393+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:21 13.30 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf44f70 con 0x5621ccf0ed80 2023-11-15T03:10:56.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.393+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 9 ==== osd_op_reply(21 600.00000000 [delete] v133'738 uv738 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.393+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:22 13.30 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf44f70 con 0x5621ccf0ed80 2023-11-15T03:10:56.405 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-11-15T03:10:56.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.397+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 10 ==== osd_op_reply(22 600.00000000 [create] v133'739 uv739 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.397+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:23 13.30 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf45f80 con 0x5621ccf0ed80 2023-11-15T03:10:56.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.401+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 11 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v133'740 uv740 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.401+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:24 13.30 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf45f80 con 0x5621ccf0ed80 2023-11-15T03:10:56.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.405+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 12 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v133'741 uv741 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.405+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:25 13.30 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf44f70 con 0x5621ccf0ed80 2023-11-15T03:10:56.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.405+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 13 ==== osd_op_reply(25 600.00000000 [omap-set-header] v133'742 uv742 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.405+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:26 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf44f70 con 0x5621ccf0ed80 2023-11-15T03:10:56.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.409+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 14 ==== osd_op_reply(26 603.00000000 [create] v133'743 uv729 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.409+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:27 13.30 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf44f70 con 0x5621ccf0ed80 2023-11-15T03:10:56.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.413+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 15 ==== osd_op_reply(27 603.00000000 [delete] v133'744 uv744 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.413+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:28 13.30 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf44f70 con 0x5621ccf0ed80 2023-11-15T03:10:56.422 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-11-15T03:10:56.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.413+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 16 ==== osd_op_reply(28 603.00000000 [create] v133'745 uv745 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.413+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:29 13.30 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf45f80 con 0x5621ccf0ed80 2023-11-15T03:10:56.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.417+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 17 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v133'746 uv746 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.417+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:30 13.30 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf45f80 con 0x5621ccf0ed80 2023-11-15T03:10:56.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.421+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 18 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v133'747 uv747 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.421+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:31 13.30 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf44f70 con 0x5621ccf0ed80 2023-11-15T03:10:56.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.425+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 19 ==== osd_op_reply(31 603.00000000 [omap-set-header] v133'748 uv748 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.425+0000 7f65b092fd00 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5621ccf45480 0x5621ccf47870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.425+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf47d60 con 0x5621ccf45480 2023-11-15T03:10:56.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.425+0000 7f65a480e700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5621ccf45480 0x5621ccf47870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.429+0000 7f65a480e700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5621ccf45480 0x5621ccf47870 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:56.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.429+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(32 1.00000000.inode [create] v133'398 uv392 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf45480 2023-11-15T03:10:56.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.433+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf48190 con 0x5621ccf45480 2023-11-15T03:10:56.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.433+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== osd_op_reply(33 1.00000000.inode [delete] v133'399 uv399 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf45480 2023-11-15T03:10:56.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.433+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf48650 con 0x5621ccf45480 2023-11-15T03:10:56.444 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-11-15T03:10:56.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.437+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.0 v2:172.21.15.121:6816/33133 3 ==== osd_op_reply(34 1.00000000.inode [create] v133'400 uv400 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf45480 2023-11-15T03:10:56.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.437+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf48f40 con 0x5621ccf45480 2023-11-15T03:10:56.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.441+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.0 v2:172.21.15.121:6816/33133 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v133'401 uv401 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf45480 2023-11-15T03:10:56.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.441+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf48ad0 con 0x5621ccf45480 2023-11-15T03:10:56.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.441+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.0 v2:172.21.15.121:6816/33133 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v133'402 uv402 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf45480 2023-11-15T03:10:56.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.441+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf497e0 con 0x5621ccf45480 2023-11-15T03:10:56.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.445+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.0 v2:172.21.15.121:6816/33133 6 ==== osd_op_reply(37 200.00000000 [create] v133'403 uv397 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf45480 2023-11-15T03:10:56.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.445+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf49b90 con 0x5621ccf45480 2023-11-15T03:10:56.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.449+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.0 v2:172.21.15.121:6816/33133 7 ==== osd_op_reply(38 200.00000000 [delete] v133'404 uv404 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf45480 2023-11-15T03:10:56.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.449+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf49f40 con 0x5621ccf45480 2023-11-15T03:10:56.457 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-11-15T03:10:56.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.449+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.0 v2:172.21.15.121:6816/33133 8 ==== osd_op_reply(39 200.00000000 [create] v133'405 uv405 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf45480 2023-11-15T03:10:56.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.449+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf49f40 con 0x5621ccf45480 2023-11-15T03:10:56.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.453+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.0 v2:172.21.15.121:6816/33133 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v133'406 uv406 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf45480 2023-11-15T03:10:56.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.453+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf49370 con 0x5621ccf45480 2023-11-15T03:10:56.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.457+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.0 v2:172.21.15.121:6816/33133 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v133'407 uv407 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf45480 2023-11-15T03:10:56.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.457+0000 7f65b092fd00 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5621ccf4a940 0x5621ccf4cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.457+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:42 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf4d220 con 0x5621ccf4a940 2023-11-15T03:10:56.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.457+0000 7f65a5810700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5621ccf4a940 0x5621ccf4cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.457+0000 7f65a5810700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5621ccf4a940 0x5621ccf4cd30 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:56.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.461+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(42 200.00000001 [create] v133'469 uv462 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.461+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:43 13.34 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf4d220 con 0x5621ccf4a940 2023-11-15T03:10:56.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.465+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(43 200.00000001 [delete] v133'470 uv470 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.465+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:44 13.34 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf4d220 con 0x5621ccf4a940 2023-11-15T03:10:56.474 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-11-15T03:10:56.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.465+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(44 200.00000001 [create] v133'471 uv471 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.465+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:45 13.34 13:2e2fa760:::200.00000001:head [write 0~16484 in=16484b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf512b0 con 0x5621ccf4a940 2023-11-15T03:10:56.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.469+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(45 200.00000001 [write 0~16484] v133'472 uv472 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.469+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:46 13.34 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf512b0 con 0x5621ccf4a940 2023-11-15T03:10:56.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.473+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(46 200.00000001 [omap-set-header] v133'473 uv473 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.473+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:47 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf4d220 con 0x5621ccf4a940 2023-11-15T03:10:56.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.477+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(47 606.00000000 [create] v133'474 uv468 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.477+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:48 13.34 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf4d220 con 0x5621ccf4a940 2023-11-15T03:10:56.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.481+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 7 ==== osd_op_reply(48 606.00000000 [delete] v133'475 uv475 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.481+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:49 13.34 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf4d220 con 0x5621ccf4a940 2023-11-15T03:10:56.491 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-11-15T03:10:56.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.481+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 8 ==== osd_op_reply(49 606.00000000 [create] v133'476 uv476 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.485+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:50 13.34 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf4e230 con 0x5621ccf4a940 2023-11-15T03:10:56.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.485+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 9 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v133'477 uv477 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.485+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:51 13.34 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf4e230 con 0x5621ccf4a940 2023-11-15T03:10:56.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.489+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 10 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v133'478 uv478 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.489+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:52 13.34 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf4d220 con 0x5621ccf4a940 2023-11-15T03:10:56.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.493+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 11 ==== osd_op_reply(52 606.00000000 [omap-set-header] v133'479 uv479 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.493+0000 7f65b092fd00 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5621ccf4d8a0 0x5621ccf4fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.493+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:53 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf502f0 con 0x5621ccf4d8a0 2023-11-15T03:10:56.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.493+0000 7f65a500f700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5621ccf4d8a0 0x5621ccf4fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.493+0000 7f65a500f700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5621ccf4d8a0 0x5621ccf4fc90 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:56.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.497+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(53 607.00000000 [create] v133'231 uv230 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf4d8a0 2023-11-15T03:10:56.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.497+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:54 13.3c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf506d0 con 0x5621ccf4d8a0 2023-11-15T03:10:56.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.501+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(54 607.00000000 [delete] v133'232 uv232 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf4d8a0 2023-11-15T03:10:56.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.501+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:55 13.3c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf506d0 con 0x5621ccf4d8a0 2023-11-15T03:10:56.510 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-11-15T03:10:56.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.501+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.2 v2:172.21.15.121:6808/33134 3 ==== osd_op_reply(55 607.00000000 [create] v133'233 uv233 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf4d8a0 2023-11-15T03:10:56.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.501+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:56 13.3c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf516e0 con 0x5621ccf4d8a0 2023-11-15T03:10:56.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.505+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.2 v2:172.21.15.121:6808/33134 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v133'234 uv234 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf4d8a0 2023-11-15T03:10:56.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.505+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:57 13.3c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf516e0 con 0x5621ccf4d8a0 2023-11-15T03:10:56.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.509+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.2 v2:172.21.15.121:6808/33134 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v133'235 uv235 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf4d8a0 2023-11-15T03:10:56.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.509+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:58 13.3c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf506d0 con 0x5621ccf4d8a0 2023-11-15T03:10:56.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.509+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.2 v2:172.21.15.121:6808/33134 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v133'236 uv236 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf4d8a0 2023-11-15T03:10:56.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.509+0000 7f65b092fd00 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5621ccf50d50 0x5621ccf53140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.509+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:59 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53860 con 0x5621ccf50d50 2023-11-15T03:10:56.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.513+0000 7f65a480e700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5621ccf50d50 0x5621ccf53140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.513+0000 7f65a480e700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5621ccf50d50 0x5621ccf53140 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:56.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.517+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v133'169 uv168 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.517+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:60 13.26 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf50d50 2023-11-15T03:10:56.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.521+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v133'170 uv170 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.521+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:61 13.26 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf50d50 2023-11-15T03:10:56.529 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-11-15T03:10:56.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.521+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v133'171 uv171 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.521+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:62 13.26 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf50d50 2023-11-15T03:10:56.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.525+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v133'172 uv172 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.525+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf50d50 2023-11-15T03:10:56.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.529+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 5 ==== osd_op_reply(63 608.00000000 [create] v133'231 uv230 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.529+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf50d50 2023-11-15T03:10:56.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.529+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 6 ==== osd_op_reply(64 608.00000000 [delete] v133'232 uv232 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.529+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf50d50 2023-11-15T03:10:56.541 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-11-15T03:10:56.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.533+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 7 ==== osd_op_reply(65 608.00000000 [create] v133'233 uv233 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.533+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf54c50 con 0x5621ccf50d50 2023-11-15T03:10:56.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.537+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 8 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v133'234 uv234 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.537+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf54c50 con 0x5621ccf50d50 2023-11-15T03:10:56.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.537+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 9 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v133'235 uv235 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.541+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf50d50 2023-11-15T03:10:56.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.545+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 10 ==== osd_op_reply(68 608.00000000 [omap-set-header] v133'236 uv236 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.545+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf0ed80 2023-11-15T03:10:56.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.549+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 20 ==== osd_op_reply(69 604.00000000 [create] v133'426 uv420 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.549+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf0ed80 2023-11-15T03:10:56.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.553+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 21 ==== osd_op_reply(70 604.00000000 [delete] v133'427 uv427 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.553+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf0ed80 2023-11-15T03:10:56.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.557+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 22 ==== osd_op_reply(71 604.00000000 [create] v133'428 uv428 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.563 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-11-15T03:10:56.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.557+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf54c50 con 0x5621ccf0ed80 2023-11-15T03:10:56.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.557+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 23 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v133'429 uv429 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.557+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf54c50 con 0x5621ccf0ed80 2023-11-15T03:10:56.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.561+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 24 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v133'430 uv430 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.561+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf0ed80 2023-11-15T03:10:56.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.565+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 25 ==== osd_op_reply(74 604.00000000 [omap-set-header] v133'431 uv431 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.565+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf0ed80 2023-11-15T03:10:56.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.569+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 26 ==== osd_op_reply(75 500.00000000 [create] v133'432 uv425 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.569+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf0ed80 2023-11-15T03:10:56.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.573+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 27 ==== osd_op_reply(76 500.00000000 [delete] v133'433 uv433 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.573+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf0ed80 2023-11-15T03:10:56.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.573+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 28 ==== osd_op_reply(77 500.00000000 [create] v133'434 uv434 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.583 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-11-15T03:10:56.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.577+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf0ed80 2023-11-15T03:10:56.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.577+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 29 ==== osd_op_reply(78 500.00000000 [write 0~90] v133'435 uv435 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.577+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf0ed80 2023-11-15T03:10:56.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.581+0000 7f65a500f700 1 -- 172.21.15.121:0/1440087446 <== osd.4 v2:172.21.15.190:6816/33348 30 ==== osd_op_reply(79 500.00000000 [omap-set-header] v133'436 uv436 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f659800e9b0 con 0x5621ccf0ed80 2023-11-15T03:10:56.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.581+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:80 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf45480 2023-11-15T03:10:56.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.585+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.0 v2:172.21.15.121:6816/33133 11 ==== osd_op_reply(80 mds_snaptable [create] v133'193 uv192 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf45480 2023-11-15T03:10:56.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.585+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:81 13.2d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf45480 2023-11-15T03:10:56.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.589+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.0 v2:172.21.15.121:6816/33133 12 ==== osd_op_reply(81 mds_snaptable [delete] v133'194 uv194 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf45480 2023-11-15T03:10:56.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.589+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:82 13.2d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf45480 2023-11-15T03:10:56.597 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-11-15T03:10:56.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.589+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.0 v2:172.21.15.121:6816/33133 13 ==== osd_op_reply(82 mds_snaptable [create] v133'195 uv195 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf45480 2023-11-15T03:10:56.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.589+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:83 13.2d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf45480 2023-11-15T03:10:56.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.593+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.0 v2:172.21.15.121:6816/33133 14 ==== osd_op_reply(83 mds_snaptable [write 0~70] v133'196 uv196 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf45480 2023-11-15T03:10:56.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.593+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:84 13.2d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf45480 2023-11-15T03:10:56.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.597+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.0 v2:172.21.15.121:6816/33133 15 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v133'197 uv197 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf45480 2023-11-15T03:10:56.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.597+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf50d50 2023-11-15T03:10:56.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.597+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 11 ==== osd_op_reply(85 605.00000000 [create] v133'231 uv230 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.597+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf50d50 2023-11-15T03:10:56.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.601+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 12 ==== osd_op_reply(86 605.00000000 [delete] v133'232 uv232 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.601+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf50d50 2023-11-15T03:10:56.612 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-11-15T03:10:56.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.605+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 13 ==== osd_op_reply(87 605.00000000 [create] v133'233 uv233 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.605+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf54c50 con 0x5621ccf50d50 2023-11-15T03:10:56.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.609+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v133'234 uv234 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.609+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf54c50 con 0x5621ccf50d50 2023-11-15T03:10:56.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.609+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v133'235 uv235 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.609+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf50d50 2023-11-15T03:10:56.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.613+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v133'236 uv236 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.613+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf50d50 2023-11-15T03:10:56.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.617+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 17 ==== osd_op_reply(91 mds0_inotable [create] v133'194 uv193 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.617+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf50d50 2023-11-15T03:10:56.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.621+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 18 ==== osd_op_reply(92 mds0_inotable [delete] v133'195 uv195 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.621+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf50d50 2023-11-15T03:10:56.629 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-11-15T03:10:56.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.621+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 19 ==== osd_op_reply(93 mds0_inotable [create] v133'196 uv196 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.621+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c40 con 0x5621ccf50d50 2023-11-15T03:10:56.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.625+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 20 ==== osd_op_reply(94 mds0_inotable [write 0~34] v133'197 uv197 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.625+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf49fc0 con 0x5621ccf50d50 2023-11-15T03:10:56.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.629+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 21 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v133'198 uv198 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.629+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:96 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf49fc0 con 0x5621ccf50d50 2023-11-15T03:10:56.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.629+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 22 ==== osd_op_reply(96 100.00000000 [create] v133'193 uv192 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.629+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:97 13.33 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf49f40 con 0x5621ccf50d50 2023-11-15T03:10:56.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.633+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 23 ==== osd_op_reply(97 100.00000000 [delete] v133'194 uv194 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.633+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:98 13.33 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53c80 con 0x5621ccf50d50 2023-11-15T03:10:56.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.637+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 24 ==== osd_op_reply(98 100.00000000 [create] v133'195 uv195 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.643 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-11-15T03:10:56.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.637+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:99 13.33 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf54080 con 0x5621ccf50d50 2023-11-15T03:10:56.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.637+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 25 ==== osd_op_reply(99 100.00000000 [omap-set-header] v133'196 uv196 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.641+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:100 13.33 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf56df0 con 0x5621ccf50d50 2023-11-15T03:10:56.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.641+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 26 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v133'197 uv197 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.641+0000 7f65b092fd00 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5621ccf548d0 0x5621ccf56cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.641+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf571b0 con 0x5621ccf548d0 2023-11-15T03:10:56.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.641+0000 7f65a5810700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5621ccf548d0 0x5621ccf56cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.645+0000 7f65a5810700 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5621ccf548d0 0x5621ccf56cc0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:56.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.649+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== osd_op_reply(101 mds0_sessionmap [create] v133'155 uv154 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf548d0 2023-11-15T03:10:56.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.649+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf571b0 con 0x5621ccf548d0 2023-11-15T03:10:56.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.649+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== osd_op_reply(102 mds0_sessionmap [delete] v133'156 uv156 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf548d0 2023-11-15T03:10:56.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.649+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf571b0 con 0x5621ccf548d0 2023-11-15T03:10:56.661 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-11-15T03:10:56.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.653+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.5 v2:172.21.15.190:6800/33349 3 ==== osd_op_reply(103 mds0_sessionmap [create] v133'157 uv157 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf548d0 2023-11-15T03:10:56.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.653+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf571b0 con 0x5621ccf548d0 2023-11-15T03:10:56.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.657+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.5 v2:172.21.15.190:6800/33349 4 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v133'158 uv158 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf548d0 2023-11-15T03:10:56.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.657+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf571b0 con 0x5621ccf50d50 2023-11-15T03:10:56.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.661+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 27 ==== osd_op_reply(105 609.00000000 [create] v133'231 uv230 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.661+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf571b0 con 0x5621ccf50d50 2023-11-15T03:10:56.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.661+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 28 ==== osd_op_reply(106 609.00000000 [delete] v133'232 uv232 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.661+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf571b0 con 0x5621ccf50d50 2023-11-15T03:10:56.673 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-11-15T03:10:56.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.665+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 29 ==== osd_op_reply(107 609.00000000 [create] v133'233 uv233 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.665+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf581c0 con 0x5621ccf50d50 2023-11-15T03:10:56.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.669+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 30 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v133'234 uv234 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.669+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf581c0 con 0x5621ccf50d50 2023-11-15T03:10:56.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.669+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 31 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v133'235 uv235 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.669+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf571b0 con 0x5621ccf50d50 2023-11-15T03:10:56.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.673+0000 7f65a480e700 1 -- 172.21.15.121:0/1440087446 <== osd.7 v2:172.21.15.190:6808/33347 32 ==== osd_op_reply(110 609.00000000 [omap-set-header] v133'236 uv236 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f6590012040 con 0x5621ccf50d50 2023-11-15T03:10:56.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.673+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf571b0 con 0x5621ccf4a940 2023-11-15T03:10:56.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.677+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 12 ==== osd_op_reply(111 400.00000000 [create] v133'196 uv195 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.677+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf571b0 con 0x5621ccf4a940 2023-11-15T03:10:56.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.681+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 13 ==== osd_op_reply(112 400.00000000 [delete] v133'197 uv197 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.681+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf571b0 con 0x5621ccf4a940 2023-11-15T03:10:56.689 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-11-15T03:10:56.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.681+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 14 ==== osd_op_reply(113 400.00000000 [create] v133'198 uv198 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.681+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf53f30 con 0x5621ccf4a940 2023-11-15T03:10:56.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.685+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v133'199 uv199 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.685+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf576a0 con 0x5621ccf4a940 2023-11-15T03:10:56.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.689+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v133'200 uv200 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.689+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:116 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf576a0 con 0x5621ccf4a940 2023-11-15T03:10:56.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.689+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 17 ==== osd_op_reply(116 100.00000000.inode [create] v133'463 uv455 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.689+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:117 13.3f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf576a0 con 0x5621ccf4a940 2023-11-15T03:10:56.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.693+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 18 ==== osd_op_reply(117 100.00000000.inode [delete] v133'464 uv464 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.693+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:118 13.3f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf576a0 con 0x5621ccf4a940 2023-11-15T03:10:56.703 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-11-15T03:10:56.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.697+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 19 ==== osd_op_reply(118 100.00000000.inode [create] v133'465 uv465 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.697+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:119 13.3f 13:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf576a0 con 0x5621ccf4a940 2023-11-15T03:10:56.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.701+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 20 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v133'466 uv466 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.701+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:120 13.3f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf576a0 con 0x5621ccf4a940 2023-11-15T03:10:56.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.701+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v133'467 uv467 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.701+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:121 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf576a0 con 0x5621ccf4a940 2023-11-15T03:10:56.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.705+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 22 ==== osd_op_reply(121 1.00000000 [create] v133'468 uv462 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.705+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:122 13.3f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e133) v8 -- 0x5621ccf576a0 con 0x5621ccf4a940 2023-11-15T03:10:56.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.709+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 23 ==== osd_op_reply(122 1.00000000 [delete] v133'469 uv469 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.709+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:123 13.3f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf576a0 con 0x5621ccf4a940 2023-11-15T03:10:56.718 INFO:teuthology.orchestra.run.smithi121.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-11-15T03:10:56.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.709+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 24 ==== osd_op_reply(123 1.00000000 [create] v133'470 uv470 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.709+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:124 13.3f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf586b0 con 0x5621ccf4a940 2023-11-15T03:10:56.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.713+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v133'471 uv471 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.713+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:125 13.3f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf586b0 con 0x5621ccf4a940 2023-11-15T03:10:56.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.717+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v133'472 uv472 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.717+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:126 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf576a0 con 0x5621ccf4a940 2023-11-15T03:10:56.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v133'473 uv473 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:127 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x5621ccf57910 con 0x5621ccf4a940 2023-11-15T03:10:56.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 <== osd.6 v2:172.21.15.190:6809/33346 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v133'474 uv474 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f65a0021040 con 0x5621ccf4a940 2023-11-15T03:10:56.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5621ccf45480 msgr2=0x5621ccf47870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x5621ccf45480 0x5621ccf47870 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5621ccf22820 msgr2=0x5621cce93190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x5621ccf22820 0x5621cce93190 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5621ccf4d8a0 msgr2=0x5621ccf4fc90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x5621ccf4d8a0 0x5621ccf4fc90 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5621ccf0ed80 msgr2=0x5621ccf425e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x5621ccf0ed80 0x5621ccf425e0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5621ccf548d0 msgr2=0x5621ccf56cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x5621ccf548d0 0x5621ccf56cc0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5621ccf4a940 msgr2=0x5621ccf4cd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x5621ccf4a940 0x5621ccf4cd30 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5621ccf50d50 msgr2=0x5621ccf53140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x5621ccf50d50 0x5621ccf53140 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 reap_dead start 2023-11-15T03:10:56.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 reap_dead start 2023-11-15T03:10:56.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 reap_dead start 2023-11-15T03:10:56.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65a5810700 1 -- 172.21.15.121:0/1440087446 reap_dead start 2023-11-15T03:10:56.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6588058ff0 msgr2=0x7f658805b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6588058ff0 0x7f658805b4b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f6598001ed0 tx=0x7f6598000bc0).stop 2023-11-15T03:10:56.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621cce93b00 msgr2=0x5621ccf24ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621cce93b00 0x5621ccf24ad0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 rx=0x7f6590008b20 tx=0x7f659000c040).stop 2023-11-15T03:10:56.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 shutdown_connections 2023-11-15T03:10:56.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6588058ff0 0x7f658805b4b0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 --2- 172.21.15.121:0/1440087446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621cce93b00 0x5621ccf24ad0 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.721+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 >> 172.21.15.121:0/1440087446 conn(0x5621cce99080 msgr2=0x5621ccf1aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:56.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.725+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 shutdown_connections 2023-11-15T03:10:56.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.725+0000 7f65b092fd00 1 -- 172.21.15.121:0/1440087446 wait complete. 2023-11-15T03:10:56.739 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 1.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-11-15T03:10:56.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.841+0000 7f69b42f7d00 1 Processor -- start 2023-11-15T03:10:56.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.841+0000 7f69b42f7d00 1 -- start start 2023-11-15T03:10:56.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.841+0000 7f69b42f7d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbbe39abb0 0x55dbbe3cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.841+0000 7f69b42f7d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55dbbe3e4b00 0x55dbbe46ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.841+0000 7f69b42f7d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dbbe477650 0x55dbbe3dfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.841+0000 7f69b42f7d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbe313220 con 0x55dbbe477650 2023-11-15T03:10:56.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.841+0000 7f69b42f7d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbe3df640 con 0x55dbbe39abb0 2023-11-15T03:10:56.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.841+0000 7f69b42f7d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dbbe473960 con 0x55dbbe3e4b00 2023-11-15T03:10:56.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.841+0000 7f69b272a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dbbe477650 0x55dbbe3dfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.841+0000 7f69b272a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dbbe477650 0x55dbbe3dfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39812/0 (socket says 172.21.15.121:39812) 2023-11-15T03:10:56.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.841+0000 7f69b272a700 1 -- 172.21.15.121:0/292635856 learned_addr learned my addr 172.21.15.121:0/292635856 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:56.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b1f29700 1 --2- 172.21.15.121:0/292635856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbbe39abb0 0x55dbbe3cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b1728700 1 --2- 172.21.15.121:0/292635856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55dbbe3e4b00 0x55dbbe46ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b272a700 1 -- 172.21.15.121:0/292635856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55dbbe3e4b00 msgr2=0x55dbbe46ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:56.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b272a700 1 --2- 172.21.15.121:0/292635856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55dbbe3e4b00 0x55dbbe46ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b272a700 1 -- 172.21.15.121:0/292635856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbbe39abb0 msgr2=0x55dbbe3cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:56.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b272a700 1 --2- 172.21.15.121:0/292635856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbbe39abb0 0x55dbbe3cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b272a700 1 -- 172.21.15.121:0/292635856 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a8005db0 con 0x55dbbe477650 2023-11-15T03:10:56.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b1f29700 1 --2- 172.21.15.121:0/292635856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbbe39abb0 0x55dbbe3cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:56.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b272a700 1 --2- 172.21.15.121:0/292635856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dbbe477650 0x55dbbe3dfcd0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 rx=0x7f69ac00b780 tx=0x7f69ac005d40).ready entity=mon.0 client_cookie=1d663b724ff92668 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:56.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69a3fff700 1 -- 172.21.15.121:0/292635856 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac00d5f0 con 0x55dbbe477650 2023-11-15T03:10:56.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69a3fff700 1 -- 172.21.15.121:0/292635856 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ac00d750 con 0x55dbbe477650 2023-11-15T03:10:56.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69a3fff700 1 -- 172.21.15.121:0/292635856 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac00dad0 con 0x55dbbe477650 2023-11-15T03:10:56.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b42f7d00 1 -- 172.21.15.121:0/292635856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dbbe477650 msgr2=0x55dbbe3dfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b42f7d00 1 --2- 172.21.15.121:0/292635856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dbbe477650 0x55dbbe3dfcd0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 rx=0x7f69ac00b780 tx=0x7f69ac005d40).stop 2023-11-15T03:10:56.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b42f7d00 1 -- 172.21.15.121:0/292635856 shutdown_connections 2023-11-15T03:10:56.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b42f7d00 1 --2- 172.21.15.121:0/292635856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dbbe477650 0x55dbbe3dfcd0 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b42f7d00 1 --2- 172.21.15.121:0/292635856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55dbbe3e4b00 0x55dbbe46ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b42f7d00 1 --2- 172.21.15.121:0/292635856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbbe39abb0 0x55dbbe3cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b42f7d00 1 -- 172.21.15.121:0/292635856 >> 172.21.15.121:0/292635856 conn(0x55dbbe3ea080 msgr2=0x55dbbe4747e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:56.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b42f7d00 1 -- 172.21.15.121:0/292635856 shutdown_connections 2023-11-15T03:10:56.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b42f7d00 1 -- 172.21.15.121:0/292635856 wait complete. 2023-11-15T03:10:56.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b42f7d00 1 Processor -- start 2023-11-15T03:10:56.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.845+0000 7f69b42f7d00 1 -- start start 2023-11-15T03:10:56.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69b42f7d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbbe39abb0 0x55dbbe3cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69b42f7d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dbbe3e4b00 0x55dbbe474d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69b42f7d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55dbbe477650 0x55dbbe467ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69b42f7d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbe475c70 con 0x55dbbe3e4b00 2023-11-15T03:10:56.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69b42f7d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbe3e16a0 con 0x55dbbe39abb0 2023-11-15T03:10:56.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69b42f7d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dbbe4762f0 con 0x55dbbe477650 2023-11-15T03:10:56.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69b1728700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dbbe3e4b00 0x55dbbe474d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69b1728700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dbbe3e4b00 0x55dbbe474d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39822/0 (socket says 172.21.15.121:39822) 2023-11-15T03:10:56.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69b1728700 1 -- 172.21.15.121:0/1080386198 learned_addr learned my addr 172.21.15.121:0/1080386198 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:56.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69b272a700 1 --2- 172.21.15.121:0/1080386198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55dbbe477650 0x55dbbe467ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69b1f29700 1 --2- 172.21.15.121:0/1080386198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbbe39abb0 0x55dbbe3cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69b1728700 1 -- 172.21.15.121:0/1080386198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55dbbe477650 msgr2=0x55dbbe467ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69b1728700 1 --2- 172.21.15.121:0/1080386198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55dbbe477650 0x55dbbe467ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69b1728700 1 -- 172.21.15.121:0/1080386198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbbe39abb0 msgr2=0x55dbbe3cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69b1728700 1 --2- 172.21.15.121:0/1080386198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbbe39abb0 0x55dbbe3cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69b1728700 1 -- 172.21.15.121:0/1080386198 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ac00b430 con 0x55dbbe3e4b00 2023-11-15T03:10:56.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69b1728700 1 --2- 172.21.15.121:0/1080386198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dbbe3e4b00 0x55dbbe474d30 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 rx=0x7f69a8010000 tx=0x7f69a8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:56.866 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69a27fc700 1 -- 172.21.15.121:0/1080386198 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a801e440 con 0x55dbbe3e4b00 2023-11-15T03:10:56.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69a27fc700 1 -- 172.21.15.121:0/1080386198 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a801e5a0 con 0x55dbbe3e4b00 2023-11-15T03:10:56.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69b42f7d00 1 -- 172.21.15.121:0/1080386198 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dbbe33c770 con 0x55dbbe3e4b00 2023-11-15T03:10:56.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69a27fc700 1 -- 172.21.15.121:0/1080386198 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a801e920 con 0x55dbbe3e4b00 2023-11-15T03:10:56.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.849+0000 7f69b42f7d00 1 -- 172.21.15.121:0/1080386198 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dbbe30ba70 con 0x55dbbe3e4b00 2023-11-15T03:10:56.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.853+0000 7f69a27fc700 1 -- 172.21.15.121:0/1080386198 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f69a8013440 con 0x55dbbe3e4b00 2023-11-15T03:10:56.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.853+0000 7f69a27fc700 1 --2- 172.21.15.121:0/1080386198 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6994058ff0 0x7f699405b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.853+0000 7f69a27fc700 1 -- 172.21.15.121:0/1080386198 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 9620+0+0 (secure 0 0 0) 0x7f69a80753d0 con 0x55dbbe3e4b00 2023-11-15T03:10:56.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.853+0000 7f69b1f29700 1 --2- 172.21.15.121:0/1080386198 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6994058ff0 0x7f699405b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.853+0000 7f69b42f7d00 1 --2- 172.21.15.121:0/1080386198 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55dbbe3e11a0 0x55dbbe3e3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:56.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.853+0000 7f69b42f7d00 1 -- 172.21.15.121:0/1080386198 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 13.3f 13:ff5b34d6:::1.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected e133) v8 -- 0x55dbbe3e3a80 con 0x55dbbe3e11a0 2023-11-15T03:10:56.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.857+0000 7f69b1f29700 1 --2- 172.21.15.121:0/1080386198 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6994058ff0 0x7f699405b4b0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x55dbbe39d9b0 tx=0x7f69a4000bc0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:56.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.857+0000 7f69b272a700 1 --2- 172.21.15.121:0/1080386198 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55dbbe3e11a0 0x55dbbe3e3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:56.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.857+0000 7f69b272a700 1 --2- 172.21.15.121:0/1080386198 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55dbbe3e11a0 0x55dbbe3e3590 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:56.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.861+0000 7f69b272a700 1 -- 172.21.15.121:0/1080386198 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 1.00000000 [omap-set-header] v133'475 uv475 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f69ac017020 con 0x55dbbe3e11a0 2023-11-15T03:10:56.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.861+0000 7f69b42f7d00 1 -- 172.21.15.121:0/1080386198 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55dbbe3e11a0 msgr2=0x55dbbe3e3590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.861+0000 7f69b42f7d00 1 --2- 172.21.15.121:0/1080386198 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55dbbe3e11a0 0x55dbbe3e3590 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:56.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.861+0000 7f69b42f7d00 1 -- 172.21.15.121:0/1080386198 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6994058ff0 msgr2=0x7f699405b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.861+0000 7f69b42f7d00 1 --2- 172.21.15.121:0/1080386198 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6994058ff0 0x7f699405b4b0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x55dbbe39d9b0 tx=0x7f69a4000bc0).stop 2023-11-15T03:10:56.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.861+0000 7f69b42f7d00 1 -- 172.21.15.121:0/1080386198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dbbe3e4b00 msgr2=0x55dbbe474d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:56.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.861+0000 7f69b42f7d00 1 --2- 172.21.15.121:0/1080386198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dbbe3e4b00 0x55dbbe474d30 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 rx=0x7f69a8010000 tx=0x7f69a8016000).stop 2023-11-15T03:10:56.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.861+0000 7f69b272a700 1 -- 172.21.15.121:0/1080386198 reap_dead start 2023-11-15T03:10:56.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.861+0000 7f69b42f7d00 1 -- 172.21.15.121:0/1080386198 shutdown_connections 2023-11-15T03:10:56.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.861+0000 7f69b42f7d00 1 -- 172.21.15.121:0/1080386198 >> 172.21.15.121:0/1080386198 conn(0x55dbbe3ea080 msgr2=0x55dbbe3cd970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:56.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.861+0000 7f69b42f7d00 1 -- 172.21.15.121:0/1080386198 shutdown_connections 2023-11-15T03:10:56.873 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:56.861+0000 7f69b42f7d00 1 -- 172.21.15.121:0/1080386198 wait complete. 2023-11-15T03:10:56.875 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:10:57.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.032+0000 7f957ab69700 1 Processor -- start 2023-11-15T03:10:57.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.032+0000 7f957ab69700 1 -- start start 2023-11-15T03:10:57.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.036+0000 7f957ab69700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9574157ed0 0x7f95741582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:57.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.036+0000 7f957ab69700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9574067a60 0x7f9574067e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:57.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.036+0000 7f957ab69700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9574062da0 con 0x7f95740bc0d0 2023-11-15T03:10:57.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.036+0000 7f957ab69700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95740a93a0 con 0x7f9574157ed0 2023-11-15T03:10:57.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.036+0000 7f957ab69700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9574068370 con 0x7f9574067a60 2023-11-15T03:10:57.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.036+0000 7f9578905700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9574157ed0 0x7f95741582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:57.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.036+0000 7f9579106700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f95740bc0d0 0x7f95740bc4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58762/0 (socket says 172.21.15.121:58762) 2023-11-15T03:10:57.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.036+0000 7f9579106700 1 -- 172.21.15.121:0/2175007035 learned_addr learned my addr 172.21.15.121:0/2175007035 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:10:57.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.036+0000 7f9573fff700 1 --2- 172.21.15.121:0/2175007035 >> v2:172.21.15.190:3300/0 conn(0x7f9574067a60 0x7f9574067e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:57.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.036+0000 7f9578905700 1 -- 172.21.15.121:0/2175007035 >> v2:172.21.15.190:3300/0 conn(0x7f9574067a60 msgr2=0x7f9574067e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:57.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.036+0000 7f9578905700 1 --2- 172.21.15.121:0/2175007035 >> v2:172.21.15.190:3300/0 conn(0x7f9574067a60 0x7f9574067e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:57.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.036+0000 7f9578905700 1 -- 172.21.15.121:0/2175007035 >> v1:172.21.15.121:6789/0 conn(0x7f95740bc0d0 legacy=0x7f95740bc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:57.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.036+0000 7f9578905700 1 -- 172.21.15.121:0/2175007035 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9574062190 con 0x7f9574157ed0 2023-11-15T03:10:57.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.036+0000 7f95737fe700 1 -- 172.21.15.121:0/2175007035 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611754222 0 0) 0x7f9574062da0 con 0x7f95740bc0d0 2023-11-15T03:10:57.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.036+0000 7f9578905700 1 --2- 172.21.15.121:0/2175007035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9574157ed0 0x7f95741582b0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 rx=0x7f95740bc020 tx=0x7f9568009060).ready entity=mon.0 client_cookie=78eb3d7d5478c30e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:57.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.036+0000 7f95737fe700 1 -- 172.21.15.121:0/2175007035 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956800f510 con 0x7f9574157ed0 2023-11-15T03:10:57.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.036+0000 7f95737fe700 1 -- 172.21.15.121:0/2175007035 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956800f670 con 0x7f9574157ed0 2023-11-15T03:10:57.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.040+0000 7f95737fe700 1 -- 172.21.15.121:0/2175007035 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956800f9f0 con 0x7f9574157ed0 2023-11-15T03:10:57.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.040+0000 7f957ab69700 1 -- 172.21.15.121:0/2175007035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9574157ed0 msgr2=0x7f95741582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:57.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.040+0000 7f957ab69700 1 --2- 172.21.15.121:0/2175007035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9574157ed0 0x7f95741582b0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 rx=0x7f95740bc020 tx=0x7f9568009060).stop 2023-11-15T03:10:57.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.040+0000 7f957ab69700 1 -- 172.21.15.121:0/2175007035 shutdown_connections 2023-11-15T03:10:57.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.040+0000 7f957ab69700 1 --2- 172.21.15.121:0/2175007035 >> v2:172.21.15.190:3300/0 conn(0x7f9574067a60 0x7f9574067e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:57.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.040+0000 7f957ab69700 1 --2- 172.21.15.121:0/2175007035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9574157ed0 0x7f95741582b0 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:57.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.040+0000 7f957ab69700 1 -- 172.21.15.121:0/2175007035 >> 172.21.15.121:0/2175007035 conn(0x7f95740591c0 msgr2=0x7f95740c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:57.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.040+0000 7f957ab69700 1 -- 172.21.15.121:0/2175007035 shutdown_connections 2023-11-15T03:10:57.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.040+0000 7f957ab69700 1 -- 172.21.15.121:0/2175007035 wait complete. 2023-11-15T03:10:57.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f957ab69700 1 Processor -- start 2023-11-15T03:10:57.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f957ab69700 1 -- start start 2023-11-15T03:10:57.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f957ab69700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9574067a60 0x7f957415ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:57.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f957ab69700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95740bc0d0 0x7f9574168330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:57.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f957ab69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9574157ed0 0x7f9574153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:57.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f957ab69700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9574153d00 con 0x7f9574067a60 2023-11-15T03:10:57.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f957ab69700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957416c710 con 0x7f95740bc0d0 2023-11-15T03:10:57.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f957ab69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95740be9a0 con 0x7f9574157ed0 2023-11-15T03:10:57.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f9578905700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9574067a60 0x7f957415ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:57.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f9578905700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9574067a60 0x7f957415ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39840/0 (socket says 172.21.15.121:39840) 2023-11-15T03:10:57.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f9578905700 1 -- 172.21.15.121:0/1135038821 learned_addr learned my addr 172.21.15.121:0/1135038821 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:57.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f9573fff700 1 --2- 172.21.15.121:0/1135038821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95740bc0d0 0x7f9574168330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:57.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f9579106700 1 --2- 172.21.15.121:0/1135038821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9574157ed0 0x7f9574153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:57.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f9578905700 1 -- 172.21.15.121:0/1135038821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9574157ed0 msgr2=0x7f9574153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:57.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f9578905700 1 --2- 172.21.15.121:0/1135038821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9574157ed0 0x7f9574153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:57.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f9578905700 1 -- 172.21.15.121:0/1135038821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95740bc0d0 msgr2=0x7f9574168330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:57.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f9578905700 1 --2- 172.21.15.121:0/1135038821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95740bc0d0 0x7f9574168330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:57.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f9578905700 1 -- 172.21.15.121:0/1135038821 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95680092a0 con 0x7f9574067a60 2023-11-15T03:10:57.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f9573fff700 1 --2- 172.21.15.121:0/1135038821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95740bc0d0 0x7f9574168330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:57.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.044+0000 7f9579106700 1 --2- 172.21.15.121:0/1135038821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9574157ed0 0x7f9574153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:10:57.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.048+0000 7f9578905700 1 --2- 172.21.15.121:0/1135038821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9574067a60 0x7f957415ec10 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 rx=0x7f9568009dd0 tx=0x7f9568017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:57.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.048+0000 7f9570ff9700 1 -- 172.21.15.121:0/1135038821 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956800f7e0 con 0x7f9574067a60 2023-11-15T03:10:57.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.048+0000 7f9570ff9700 1 -- 172.21.15.121:0/1135038821 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956800f940 con 0x7f9574067a60 2023-11-15T03:10:57.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.048+0000 7f9570ff9700 1 -- 172.21.15.121:0/1135038821 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956801b690 con 0x7f9574067a60 2023-11-15T03:10:57.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.048+0000 7f957ab69700 1 -- 172.21.15.121:0/1135038821 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95740bec80 con 0x7f9574067a60 2023-11-15T03:10:57.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.048+0000 7f957ab69700 1 -- 172.21.15.121:0/1135038821 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95740bf240 con 0x7f9574067a60 2023-11-15T03:10:57.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.048+0000 7f957ab69700 1 -- 172.21.15.121:0/1135038821 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9574047060 con 0x7f9574067a60 2023-11-15T03:10:57.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.048+0000 7f9570ff9700 1 -- 172.21.15.121:0/1135038821 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f956801ba40 con 0x7f9574067a60 2023-11-15T03:10:57.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.052+0000 7f9570ff9700 1 --2- 172.21.15.121:0/1135038821 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f955805a510 0x7f955805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:57.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.052+0000 7f9570ff9700 1 -- 172.21.15.121:0/1135038821 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 9620+0+0 (secure 0 0 0) 0x7f9568076c60 con 0x7f9574067a60 2023-11-15T03:10:57.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.052+0000 7f9573fff700 1 --2- 172.21.15.121:0/1135038821 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f955805a510 0x7f955805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:57.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.052+0000 7f9573fff700 1 --2- 172.21.15.121:0/1135038821 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f955805a510 0x7f955805c9d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7f9574152300 tx=0x7f9560008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:57.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.056+0000 7f9570ff9700 1 -- 172.21.15.121:0/1135038821 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9568049650 con 0x7f9574067a60 2023-11-15T03:10:57.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:57.252+0000 7f957ab69700 1 -- 172.21.15.121:0/1135038821 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f95740bf430 con 0x7f9574067a60 2023-11-15T03:10:58.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.216+0000 7f9570ff9700 1 -- 172.21.15.121:0/1135038821 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v337) v1 ==== 162+0+0 (secure 0 0 0) 0x7f956804cc70 con 0x7f9574067a60 2023-11-15T03:10:58.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.216+0000 7f957ab69700 1 -- 172.21.15.121:0/1135038821 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f955805a510 msgr2=0x7f955805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:58.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.216+0000 7f957ab69700 1 --2- 172.21.15.121:0/1135038821 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f955805a510 0x7f955805c9d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7f9574152300 tx=0x7f9560008040).stop 2023-11-15T03:10:58.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.216+0000 7f957ab69700 1 -- 172.21.15.121:0/1135038821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9574067a60 msgr2=0x7f957415ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:58.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.216+0000 7f957ab69700 1 --2- 172.21.15.121:0/1135038821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9574067a60 0x7f957415ec10 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 rx=0x7f9568009dd0 tx=0x7f9568017000).stop 2023-11-15T03:10:58.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.220+0000 7f957ab69700 1 -- 172.21.15.121:0/1135038821 shutdown_connections 2023-11-15T03:10:58.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.220+0000 7f957ab69700 1 --2- 172.21.15.121:0/1135038821 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f955805a510 0x7f955805c9d0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:58.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.220+0000 7f957ab69700 1 --2- 172.21.15.121:0/1135038821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9574157ed0 0x7f9574153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:58.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.220+0000 7f957ab69700 1 --2- 172.21.15.121:0/1135038821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95740bc0d0 0x7f9574168330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:58.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.220+0000 7f957ab69700 1 --2- 172.21.15.121:0/1135038821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9574067a60 0x7f957415ec10 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:58.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.220+0000 7f957ab69700 1 -- 172.21.15.121:0/1135038821 >> 172.21.15.121:0/1135038821 conn(0x7f95740591c0 msgr2=0x7f95740abd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:58.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.220+0000 7f957ab69700 1 -- 172.21.15.121:0/1135038821 shutdown_connections 2023-11-15T03:10:58.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.220+0000 7f957ab69700 1 -- 172.21.15.121:0/1135038821 wait complete. 2023-11-15T03:10:58.230 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:10:58.250 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:10:58.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.404+0000 7fedfb59e700 1 Processor -- start 2023-11-15T03:10:58.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.404+0000 7fedfb59e700 1 -- start start 2023-11-15T03:10:58.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.408+0000 7fedfb59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fedfc0a9830 0x7fedfc0a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:58.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.408+0000 7fedfb59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fedfc0c30e0 0x7fedfc0c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:58.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.408+0000 7fedfb59e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedfc0657f0 con 0x7fedfc151870 2023-11-15T03:10:58.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.408+0000 7fedfb59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedfc065ad0 con 0x7fedfc0c30e0 2023-11-15T03:10:58.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.408+0000 7fedfb59e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedfc0c3a60 con 0x7fedfc0a9830 2023-11-15T03:10:58.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.408+0000 7fedfad9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fedfc0c30e0 0x7fedfc0c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:58.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.408+0000 7fedfad9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fedfc0c30e0 0x7fedfc0c3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39848/0 (socket says 172.21.15.121:39848) 2023-11-15T03:10:58.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.408+0000 7fedfad9d700 1 -- 172.21.15.121:0/2243717231 learned_addr learned my addr 172.21.15.121:0/2243717231 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:58.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.408+0000 7fedf9d9b700 1 --2- 172.21.15.121:0/2243717231 >> v2:172.21.15.190:3300/0 conn(0x7fedfc0a9830 0x7fedfc0a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:58.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.408+0000 7fedfad9d700 1 -- 172.21.15.121:0/2243717231 >> v2:172.21.15.190:3300/0 conn(0x7fedfc0a9830 msgr2=0x7fedfc0a9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:10:58.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.408+0000 7fedfad9d700 1 --2- 172.21.15.121:0/2243717231 >> v2:172.21.15.190:3300/0 conn(0x7fedfc0a9830 0x7fedfc0a9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:58.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.408+0000 7fedfad9d700 1 -- 172.21.15.121:0/2243717231 >> v1:172.21.15.190:6789/0 conn(0x7fedfc151870 legacy=0x7fedfc151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:58.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.408+0000 7fedfad9d700 1 -- 172.21.15.121:0/2243717231 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedfc0aa260 con 0x7fedfc0c30e0 2023-11-15T03:10:58.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.408+0000 7fedfad9d700 1 --2- 172.21.15.121:0/2243717231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedfc0c30e0 0x7fedfc0c3570 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 rx=0x7fedfc063d00 tx=0x7fedf4009840).ready entity=mon.0 client_cookie=55a7766573ffca0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:58.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.412+0000 7fedf959a700 1 -- 172.21.15.121:0/2243717231 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf400e070 con 0x7fedfc0c30e0 2023-11-15T03:10:58.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.412+0000 7fedf959a700 1 -- 172.21.15.121:0/2243717231 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf400ba20 con 0x7fedfc0c30e0 2023-11-15T03:10:58.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.412+0000 7fedf959a700 1 -- 172.21.15.121:0/2243717231 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf400bda0 con 0x7fedfc0c30e0 2023-11-15T03:10:58.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.412+0000 7fedfb59e700 1 -- 172.21.15.121:0/2243717231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedfc0c30e0 msgr2=0x7fedfc0c3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:58.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.412+0000 7fedfb59e700 1 --2- 172.21.15.121:0/2243717231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedfc0c30e0 0x7fedfc0c3570 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 rx=0x7fedfc063d00 tx=0x7fedf4009840).stop 2023-11-15T03:10:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.412+0000 7fedfb59e700 1 -- 172.21.15.121:0/2243717231 shutdown_connections 2023-11-15T03:10:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.412+0000 7fedfb59e700 1 --2- 172.21.15.121:0/2243717231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedfc0c30e0 0x7fedfc0c3570 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.412+0000 7fedfb59e700 1 --2- 172.21.15.121:0/2243717231 >> v2:172.21.15.190:3300/0 conn(0x7fedfc0a9830 0x7fedfc0a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.412+0000 7fedfb59e700 1 -- 172.21.15.121:0/2243717231 >> 172.21.15.121:0/2243717231 conn(0x7fedfc062190 msgr2=0x7fedfc062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:58.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.412+0000 7fedfb59e700 1 -- 172.21.15.121:0/2243717231 shutdown_connections 2023-11-15T03:10:58.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.412+0000 7fedfb59e700 1 -- 172.21.15.121:0/2243717231 wait complete. 2023-11-15T03:10:58.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.416+0000 7fedfb59e700 1 Processor -- start 2023-11-15T03:10:58.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.416+0000 7fedfb59e700 1 -- start start 2023-11-15T03:10:58.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.416+0000 7fedfb59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedfc0a9830 0x7fedfc165270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:58.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.416+0000 7fedfb59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedfc0c30e0 0x7fedfc165760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:58.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.416+0000 7fedfb59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedfc151870 0x7fedfc152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:58.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.416+0000 7fedfb59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc151c50 con 0x7fedfc0a9830 2023-11-15T03:10:58.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.416+0000 7fedfb59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc154890 con 0x7fedfc0c30e0 2023-11-15T03:10:58.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.416+0000 7fedfb59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc154b70 con 0x7fedfc151870 2023-11-15T03:10:58.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.416+0000 7fedfa59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedfc0a9830 0x7fedfc165270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:58.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.416+0000 7fedfa59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedfc0a9830 0x7fedfc165270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39858/0 (socket says 172.21.15.121:39858) 2023-11-15T03:10:58.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.416+0000 7fedfa59c700 1 -- 172.21.15.121:0/3655969303 learned_addr learned my addr 172.21.15.121:0/3655969303 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:10:58.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.416+0000 7fedf9d9b700 1 --2- 172.21.15.121:0/3655969303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedfc0c30e0 0x7fedfc165760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:58.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.416+0000 7fedfad9d700 1 --2- 172.21.15.121:0/3655969303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedfc151870 0x7fedfc152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:58.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.420+0000 7fedfa59c700 1 -- 172.21.15.121:0/3655969303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedfc151870 msgr2=0x7fedfc152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:58.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.420+0000 7fedfa59c700 1 --2- 172.21.15.121:0/3655969303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedfc151870 0x7fedfc152710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:58.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.420+0000 7fedfa59c700 1 -- 172.21.15.121:0/3655969303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedfc0c30e0 msgr2=0x7fedfc165760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:58.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.420+0000 7fedfa59c700 1 --2- 172.21.15.121:0/3655969303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedfc0c30e0 0x7fedfc165760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:58.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.420+0000 7fedfa59c700 1 -- 172.21.15.121:0/3655969303 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf4009ae0 con 0x7fedfc0a9830 2023-11-15T03:10:58.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.420+0000 7fedfa59c700 1 --2- 172.21.15.121:0/3655969303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedfc0a9830 0x7fedfc165270 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 rx=0x7fedec00c440 tx=0x7fedec007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:58.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.420+0000 7fedea7fc700 1 -- 172.21.15.121:0/3655969303 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec007020 con 0x7fedfc0a9830 2023-11-15T03:10:58.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.420+0000 7fedea7fc700 1 -- 172.21.15.121:0/3655969303 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedec007180 con 0x7fedfc0a9830 2023-11-15T03:10:58.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.420+0000 7fedea7fc700 1 -- 172.21.15.121:0/3655969303 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec007520 con 0x7fedfc0a9830 2023-11-15T03:10:58.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.420+0000 7fedfb59e700 1 -- 172.21.15.121:0/3655969303 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedfc151f10 con 0x7fedfc0a9830 2023-11-15T03:10:58.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.420+0000 7fedfb59e700 1 -- 172.21.15.121:0/3655969303 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedfc14ea30 con 0x7fedfc0a9830 2023-11-15T03:10:58.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.420+0000 7fedfb59e700 1 -- 172.21.15.121:0/3655969303 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedfc06bcf0 con 0x7fedfc0a9830 2023-11-15T03:10:58.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.424+0000 7fedea7fc700 1 -- 172.21.15.121:0/3655969303 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fedec0068b0 con 0x7fedfc0a9830 2023-11-15T03:10:58.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.424+0000 7fedea7fc700 1 --2- 172.21.15.121:0/3655969303 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feddc05a510 0x7feddc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:10:58.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.424+0000 7fedea7fc700 1 -- 172.21.15.121:0/3655969303 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 9620+0+0 (secure 0 0 0) 0x7fedec074810 con 0x7fedfc0a9830 2023-11-15T03:10:58.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.424+0000 7fedf9d9b700 1 --2- 172.21.15.121:0/3655969303 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feddc05a510 0x7feddc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:10:58.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.424+0000 7fedf9d9b700 1 --2- 172.21.15.121:0/3655969303 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feddc05a510 0x7feddc05c9d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7fedfc0a8c70 tx=0x7fedf0009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:10:58.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.428+0000 7fedea7fc700 1 -- 172.21.15.121:0/3655969303 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fedec047180 con 0x7fedfc0a9830 2023-11-15T03:10:58.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.624+0000 7fedfb59e700 1 -- 172.21.15.121:0/3655969303 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedfc14ecb0 con 0x7fedfc0a9830 2023-11-15T03:10:58.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.628+0000 7fedea7fc700 1 -- 172.21.15.121:0/3655969303 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 337 v337) v1 ==== 95+0+4508 (secure 0 0 0) 0x7fedec04a7a0 con 0x7fedfc0a9830 2023-11-15T03:10:58.636 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:10:58.636 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":337,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":337,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:10:58.204486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10461":{"gid":10461,"name":"a","rank":0,"incarnation":337,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6836/3720977338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3720977338},{"type":"v1","addr":"172.21.15.121:6836","nonce":3720977338}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:10:58.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.632+0000 7fedfb59e700 1 -- 172.21.15.121:0/3655969303 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feddc05a510 msgr2=0x7feddc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:58.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.632+0000 7fedfb59e700 1 --2- 172.21.15.121:0/3655969303 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feddc05a510 0x7feddc05c9d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7fedfc0a8c70 tx=0x7fedf0009060).stop 2023-11-15T03:10:58.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.632+0000 7fedfb59e700 1 -- 172.21.15.121:0/3655969303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedfc0a9830 msgr2=0x7fedfc165270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:10:58.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.632+0000 7fedfb59e700 1 --2- 172.21.15.121:0/3655969303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedfc0a9830 0x7fedfc165270 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 rx=0x7fedec00c440 tx=0x7fedec007e00).stop 2023-11-15T03:10:58.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.632+0000 7fedfb59e700 1 -- 172.21.15.121:0/3655969303 shutdown_connections 2023-11-15T03:10:58.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.632+0000 7fedfb59e700 1 --2- 172.21.15.121:0/3655969303 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7feddc05a510 0x7feddc05c9d0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:58.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.632+0000 7fedfb59e700 1 --2- 172.21.15.121:0/3655969303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedfc151870 0x7fedfc152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:58.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.632+0000 7fedfb59e700 1 --2- 172.21.15.121:0/3655969303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedfc0c30e0 0x7fedfc165760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:58.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.632+0000 7fedfb59e700 1 --2- 172.21.15.121:0/3655969303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedfc0a9830 0x7fedfc165270 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:10:58.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.632+0000 7fedfb59e700 1 -- 172.21.15.121:0/3655969303 >> 172.21.15.121:0/3655969303 conn(0x7fedfc062190 msgr2=0x7fedfc0c0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:10:58.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.632+0000 7fedfb59e700 1 -- 172.21.15.121:0/3655969303 shutdown_connections 2023-11-15T03:10:58.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:10:58.632+0000 7fedfb59e700 1 -- 172.21.15.121:0/3655969303 wait complete. 2023-11-15T03:10:58.643 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 337 2023-11-15T03:10:58.660 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-11-15T03:11:01.258 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:11:01.248+0000 7f5cbcf54700 -1 mds.0.cache.dir(0x1) Corrupt fnode in dirfrag 0x1: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input 2023-11-15T03:11:01.258 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:11:01.248+0000 7f5cbcf54700 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2023-11-15T03:11:02.255 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:11:02.248+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:11:02.255 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:11:02.248+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-15T03:11:02.264 INFO:tasks.ceph.mds.a.smithi121.stderr: -18> 2023-11-15T03:11:01.248+0000 7f5cbcf54700 -1 mds.0.cache.dir(0x1) Corrupt fnode in dirfrag 0x1: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input 2023-11-15T03:11:02.264 INFO:tasks.ceph.mds.a.smithi121.stderr: -15> 2023-11-15T03:11:01.248+0000 7f5cbcf54700 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2023-11-15T03:11:02.307 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:11:03.662 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:03.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.820+0000 7fc932df9700 1 Processor -- start 2023-11-15T03:11:03.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.820+0000 7fc932df9700 1 -- start start 2023-11-15T03:11:03.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc932df9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc92c0c3a10 0x7fc92c0c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:03.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc932df9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc92c0c6a30 0x7fc92c065880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:03.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc932df9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc92c065d70 0x7fc92c15bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:03.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc932df9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92c1584f0 con 0x7fc92c0c3a10 2023-11-15T03:11:03.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc932df9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92c0c6e90 con 0x7fc92c065d70 2023-11-15T03:11:03.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc932df9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0665b0 con 0x7fc92c0c6a30 2023-11-15T03:11:03.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc931df7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc92c0c3a10 0x7fc92c0c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:03.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc931df7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc92c0c3a10 0x7fc92c0c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39882/0 (socket says 172.21.15.121:39882) 2023-11-15T03:11:03.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc931df7700 1 -- 172.21.15.121:0/244400084 learned_addr learned my addr 172.21.15.121:0/244400084 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:03.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc9315f6700 1 --2- 172.21.15.121:0/244400084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc92c0c6a30 0x7fc92c065880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:03.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc9325f8700 1 --2- 172.21.15.121:0/244400084 >> v2:172.21.15.190:3300/0 conn(0x7fc92c065d70 0x7fc92c15bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:03.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc931df7700 1 -- 172.21.15.121:0/244400084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc92c0c6a30 msgr2=0x7fc92c065880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:03.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc931df7700 1 --2- 172.21.15.121:0/244400084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc92c0c6a30 0x7fc92c065880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:03.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc931df7700 1 -- 172.21.15.121:0/244400084 >> v2:172.21.15.190:3300/0 conn(0x7fc92c065d70 msgr2=0x7fc92c15bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:03.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc931df7700 1 --2- 172.21.15.121:0/244400084 >> v2:172.21.15.190:3300/0 conn(0x7fc92c065d70 0x7fc92c15bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:03.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc931df7700 1 -- 172.21.15.121:0/244400084 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c066890 con 0x7fc92c0c3a10 2023-11-15T03:11:03.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc9325f8700 1 --2- 172.21.15.121:0/244400084 >> v2:172.21.15.190:3300/0 conn(0x7fc92c065d70 0x7fc92c15bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:03.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc931df7700 1 --2- 172.21.15.121:0/244400084 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc92c0c3a10 0x7fc92c0c6540 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 rx=0x7fc92c0c0bc0 tx=0x7fc9280090f0).ready entity=mon.0 client_cookie=70d2cb52fafbfaf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:03.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc930df5700 1 -- 172.21.15.121:0/244400084 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92800b520 con 0x7fc92c0c3a10 2023-11-15T03:11:03.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc930df5700 1 -- 172.21.15.121:0/244400084 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92800b680 con 0x7fc92c0c3a10 2023-11-15T03:11:03.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.824+0000 7fc930df5700 1 -- 172.21.15.121:0/244400084 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92800ba00 con 0x7fc92c0c3a10 2023-11-15T03:11:03.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.828+0000 7fc932df9700 1 -- 172.21.15.121:0/244400084 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc92c0c3a10 msgr2=0x7fc92c0c6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:03.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.828+0000 7fc932df9700 1 --2- 172.21.15.121:0/244400084 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc92c0c3a10 0x7fc92c0c6540 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 rx=0x7fc92c0c0bc0 tx=0x7fc9280090f0).stop 2023-11-15T03:11:03.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.828+0000 7fc932df9700 1 -- 172.21.15.121:0/244400084 shutdown_connections 2023-11-15T03:11:03.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.828+0000 7fc932df9700 1 --2- 172.21.15.121:0/244400084 >> v2:172.21.15.190:3300/0 conn(0x7fc92c065d70 0x7fc92c15bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:03.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.828+0000 7fc932df9700 1 --2- 172.21.15.121:0/244400084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc92c0c6a30 0x7fc92c065880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:03.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.828+0000 7fc932df9700 1 --2- 172.21.15.121:0/244400084 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc92c0c3a10 0x7fc92c0c6540 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:03.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.828+0000 7fc932df9700 1 -- 172.21.15.121:0/244400084 >> 172.21.15.121:0/244400084 conn(0x7fc92c058c50 msgr2=0x7fc92c059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:03.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.828+0000 7fc932df9700 1 -- 172.21.15.121:0/244400084 shutdown_connections 2023-11-15T03:11:03.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.828+0000 7fc932df9700 1 -- 172.21.15.121:0/244400084 wait complete. 2023-11-15T03:11:03.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.828+0000 7fc932df9700 1 Processor -- start 2023-11-15T03:11:03.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.828+0000 7fc932df9700 1 -- start start 2023-11-15T03:11:03.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc932df9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc92c065d70 0x7fc92c0be200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:03.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc932df9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc92c0c3a10 0x7fc92c0bf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:03.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc932df9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc92c0c6a30 0x7fc92c146000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:03.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc932df9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0be6f0 con 0x7fc92c0c3a10 2023-11-15T03:11:03.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc932df9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0bfa20 con 0x7fc92c0c6a30 2023-11-15T03:11:03.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc932df9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0c0870 con 0x7fc92c065d70 2023-11-15T03:11:03.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc9315f6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc92c0c3a10 0x7fc92c0bf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:03.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc9315f6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc92c0c3a10 0x7fc92c0bf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39892/0 (socket says 172.21.15.121:39892) 2023-11-15T03:11:03.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc9315f6700 1 -- 172.21.15.121:0/2990250724 learned_addr learned my addr 172.21.15.121:0/2990250724 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:03.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc931df7700 1 --2- 172.21.15.121:0/2990250724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc92c065d70 0x7fc92c0be200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:03.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc9325f8700 1 --2- 172.21.15.121:0/2990250724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc92c0c6a30 0x7fc92c146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:03.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc9315f6700 1 -- 172.21.15.121:0/2990250724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc92c065d70 msgr2=0x7fc92c0be200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:03.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc9315f6700 1 --2- 172.21.15.121:0/2990250724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc92c065d70 0x7fc92c0be200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:03.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc9315f6700 1 -- 172.21.15.121:0/2990250724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc92c0c6a30 msgr2=0x7fc92c146000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:03.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc9315f6700 1 --2- 172.21.15.121:0/2990250724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc92c0c6a30 0x7fc92c146000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:03.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc9315f6700 1 -- 172.21.15.121:0/2990250724 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc928009370 con 0x7fc92c0c3a10 2023-11-15T03:11:03.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc9315f6700 1 --2- 172.21.15.121:0/2990250724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc92c0c3a10 0x7fc92c0bf530 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 rx=0x7fc91c00e980 tx=0x7fc91c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:03.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc921ffb700 1 -- 172.21.15.121:0/2990250724 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c00c7f0 con 0x7fc92c0c3a10 2023-11-15T03:11:03.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc921ffb700 1 -- 172.21.15.121:0/2990250724 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91c00c950 con 0x7fc92c0c3a10 2023-11-15T03:11:03.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc932df9700 1 -- 172.21.15.121:0/2990250724 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92c0bfc60 con 0x7fc92c0c3a10 2023-11-15T03:11:03.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.832+0000 7fc921ffb700 1 -- 172.21.15.121:0/2990250724 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c00ccd0 con 0x7fc92c0c3a10 2023-11-15T03:11:03.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.836+0000 7fc932df9700 1 -- 172.21.15.121:0/2990250724 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c0c0220 con 0x7fc92c0c3a10 2023-11-15T03:11:03.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.836+0000 7fc932df9700 1 -- 172.21.15.121:0/2990250724 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92c0c0a00 con 0x7fc92c0c3a10 2023-11-15T03:11:03.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.836+0000 7fc921ffb700 1 -- 172.21.15.121:0/2990250724 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc91c012600 con 0x7fc92c0c3a10 2023-11-15T03:11:03.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.836+0000 7fc921ffb700 1 --2- 172.21.15.121:0/2990250724 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc91005a510 0x7fc91005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:03.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.836+0000 7fc921ffb700 1 -- 172.21.15.121:0/2990250724 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9706+0+0 (secure 0 0 0) 0x7fc91c075bc0 con 0x7fc92c0c3a10 2023-11-15T03:11:03.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.840+0000 7fc931df7700 1 --2- 172.21.15.121:0/2990250724 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc91005a510 0x7fc91005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:03.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.840+0000 7fc931df7700 1 --2- 172.21.15.121:0/2990250724 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc91005a510 0x7fc91005c9d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7fc928009130 tx=0x7fc928017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:03.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:03.844+0000 7fc921ffb700 1 -- 172.21.15.121:0/2990250724 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc91c048560 con 0x7fc92c0c3a10 2023-11-15T03:11:04.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.040+0000 7fc932df9700 1 -- 172.21.15.121:0/2990250724 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc92c063fd0 con 0x7fc92c0c3a10 2023-11-15T03:11:04.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.040+0000 7fc921ffb700 1 -- 172.21.15.121:0/2990250724 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 342 v342) v1 ==== 95+0+4497 (secure 0 0 0) 0x7fc92c063fd0 con 0x7fc92c0c3a10 2023-11-15T03:11:04.048 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:04.049 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":342,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":10542,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2943522178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2943522178},{"type":"v1","addr":"172.21.15.121:6836","nonce":2943522178}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":342}],"filesystems":[{"mdsmap":{"epoch":341,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:11:02.249759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:11:04.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.044+0000 7fc932df9700 1 -- 172.21.15.121:0/2990250724 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc91005a510 msgr2=0x7fc91005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:04.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.044+0000 7fc932df9700 1 --2- 172.21.15.121:0/2990250724 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc91005a510 0x7fc91005c9d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7fc928009130 tx=0x7fc928017000).stop 2023-11-15T03:11:04.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.044+0000 7fc932df9700 1 -- 172.21.15.121:0/2990250724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc92c0c3a10 msgr2=0x7fc92c0bf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:04.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.044+0000 7fc932df9700 1 --2- 172.21.15.121:0/2990250724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc92c0c3a10 0x7fc92c0bf530 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 rx=0x7fc91c00e980 tx=0x7fc91c009060).stop 2023-11-15T03:11:04.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.044+0000 7fc932df9700 1 -- 172.21.15.121:0/2990250724 shutdown_connections 2023-11-15T03:11:04.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.044+0000 7fc932df9700 1 --2- 172.21.15.121:0/2990250724 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc91005a510 0x7fc91005c9d0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.044+0000 7fc932df9700 1 --2- 172.21.15.121:0/2990250724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc92c0c6a30 0x7fc92c146000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.044+0000 7fc932df9700 1 --2- 172.21.15.121:0/2990250724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc92c0c3a10 0x7fc92c0bf530 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.044+0000 7fc932df9700 1 --2- 172.21.15.121:0/2990250724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc92c065d70 0x7fc92c0be200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.044+0000 7fc932df9700 1 -- 172.21.15.121:0/2990250724 >> 172.21.15.121:0/2990250724 conn(0x7fc92c058c50 msgr2=0x7fc92c0c1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:04.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.044+0000 7fc932df9700 1 -- 172.21.15.121:0/2990250724 shutdown_connections 2023-11-15T03:11:04.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.044+0000 7fc932df9700 1 -- 172.21.15.121:0/2990250724 wait complete. 2023-11-15T03:11:04.054 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 342 2023-11-15T03:11:04.075 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-15T03:11:04.075 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:04.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.228+0000 7f2d03279700 1 Processor -- start 2023-11-15T03:11:04.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.228+0000 7f2d03279700 1 -- start start 2023-11-15T03:11:04.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.228+0000 7f2d03279700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cfc151870 0x7f2cfc151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:04.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.228+0000 7f2d03279700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2cfc152140 0x7f2cfc0a9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:04.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.228+0000 7f2d03279700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cfc0657f0 con 0x7f2cfc0a9d20 2023-11-15T03:11:04.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.228+0000 7f2d03279700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cfc065ad0 con 0x7f2cfc152140 2023-11-15T03:11:04.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.228+0000 7f2d03279700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc0aa260 con 0x7f2cfc151870 2023-11-15T03:11:04.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d01a76700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2cfc152140 0x7f2cfc0a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:04.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d01a76700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2cfc152140 0x7f2cfc0a9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39906/0 (socket says 172.21.15.121:39906) 2023-11-15T03:11:04.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d01a76700 1 -- 172.21.15.121:0/3262654373 learned_addr learned my addr 172.21.15.121:0/3262654373 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:04.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d02277700 1 --2- 172.21.15.121:0/3262654373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cfc151870 0x7f2cfc151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:04.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d01a76700 1 -- 172.21.15.121:0/3262654373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cfc151870 msgr2=0x7f2cfc151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:04.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d01a76700 1 --2- 172.21.15.121:0/3262654373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cfc151870 0x7f2cfc151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d01a76700 1 -- 172.21.15.121:0/3262654373 >> v1:172.21.15.190:6789/0 conn(0x7f2cfc0a9d20 legacy=0x7f2cfc0c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:04.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d01a76700 1 -- 172.21.15.121:0/3262654373 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cfc0aa540 con 0x7f2cfc152140 2023-11-15T03:11:04.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d01a76700 1 --2- 172.21.15.121:0/3262654373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc152140 0x7f2cfc0a9830 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 rx=0x7f2cec006fd0 tx=0x7f2cec0090f0).ready entity=mon.0 client_cookie=28c6ae7a74b7cc09 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:04.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d01275700 1 -- 172.21.15.121:0/3262654373 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec00b580 con 0x7f2cfc152140 2023-11-15T03:11:04.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d01275700 1 -- 172.21.15.121:0/3262654373 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cec00b6e0 con 0x7f2cfc152140 2023-11-15T03:11:04.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d01275700 1 -- 172.21.15.121:0/3262654373 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec00ba60 con 0x7f2cfc152140 2023-11-15T03:11:04.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d03279700 1 -- 172.21.15.121:0/3262654373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc152140 msgr2=0x7f2cfc0a9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:04.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d03279700 1 --2- 172.21.15.121:0/3262654373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc152140 0x7f2cfc0a9830 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 rx=0x7f2cec006fd0 tx=0x7f2cec0090f0).stop 2023-11-15T03:11:04.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d03279700 1 -- 172.21.15.121:0/3262654373 shutdown_connections 2023-11-15T03:11:04.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d03279700 1 --2- 172.21.15.121:0/3262654373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc152140 0x7f2cfc0a9830 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d03279700 1 --2- 172.21.15.121:0/3262654373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cfc151870 0x7f2cfc151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d03279700 1 -- 172.21.15.121:0/3262654373 >> 172.21.15.121:0/3262654373 conn(0x7f2cfc062190 msgr2=0x7f2cfc062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:04.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d03279700 1 -- 172.21.15.121:0/3262654373 shutdown_connections 2023-11-15T03:11:04.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.232+0000 7f2d03279700 1 -- 172.21.15.121:0/3262654373 wait complete. 2023-11-15T03:11:04.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.236+0000 7f2d03279700 1 Processor -- start 2023-11-15T03:11:04.246 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.236+0000 7f2d03279700 1 -- start start 2023-11-15T03:11:04.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d03279700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cfc0a9d20 0x7f2cfc0c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:04.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d03279700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cfc151870 0x7f2cfc1646f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:04.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d03279700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc152140 0x7f2cfc0a8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:04.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d03279700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc163fe0 con 0x7f2cfc152140 2023-11-15T03:11:04.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d03279700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc0c25c0 con 0x7f2cfc0a9d20 2023-11-15T03:11:04.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d03279700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc0c28a0 con 0x7f2cfc151870 2023-11-15T03:11:04.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d02a78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc152140 0x7f2cfc0a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:04.250 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d02277700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cfc0a9d20 0x7f2cfc0c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:04.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d01a76700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cfc151870 0x7f2cfc1646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:04.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d02a78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc152140 0x7f2cfc0a8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39908/0 (socket says 172.21.15.121:39908) 2023-11-15T03:11:04.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d02a78700 1 -- 172.21.15.121:0/3504723300 learned_addr learned my addr 172.21.15.121:0/3504723300 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:04.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d02a78700 1 -- 172.21.15.121:0/3504723300 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cfc151870 msgr2=0x7f2cfc1646f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:04.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d02a78700 1 --2- 172.21.15.121:0/3504723300 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cfc151870 0x7f2cfc1646f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d02a78700 1 -- 172.21.15.121:0/3504723300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cfc0a9d20 msgr2=0x7f2cfc0c36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:04.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d02a78700 1 --2- 172.21.15.121:0/3504723300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cfc0a9d20 0x7f2cfc0c36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d02a78700 1 -- 172.21.15.121:0/3504723300 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec009370 con 0x7f2cfc152140 2023-11-15T03:11:04.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d02277700 1 --2- 172.21.15.121:0/3504723300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cfc0a9d20 0x7f2cfc0c36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:04.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d02a78700 1 --2- 172.21.15.121:0/3504723300 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc152140 0x7f2cfc0a8750 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 rx=0x7f2cf400c7c0 tx=0x7f2cf4001560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:04.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2cf27fc700 1 -- 172.21.15.121:0/3504723300 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf400e4a0 con 0x7f2cfc152140 2023-11-15T03:11:04.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2cf27fc700 1 -- 172.21.15.121:0/3504723300 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf400e600 con 0x7f2cfc152140 2023-11-15T03:11:04.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d03279700 1 -- 172.21.15.121:0/3504723300 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cfc0c2b80 con 0x7f2cfc152140 2023-11-15T03:11:04.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2cf27fc700 1 -- 172.21.15.121:0/3504723300 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf400e980 con 0x7f2cfc152140 2023-11-15T03:11:04.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.240+0000 7f2d03279700 1 -- 172.21.15.121:0/3504723300 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cfc0c2200 con 0x7f2cfc152140 2023-11-15T03:11:04.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.244+0000 7f2d03279700 1 -- 172.21.15.121:0/3504723300 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cfc0c2d70 con 0x7f2cfc152140 2023-11-15T03:11:04.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.244+0000 7f2cf27fc700 1 -- 172.21.15.121:0/3504723300 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2cf4012400 con 0x7f2cfc152140 2023-11-15T03:11:04.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.248+0000 7f2cf27fc700 1 --2- 172.21.15.121:0/3504723300 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2ce005a510 0x7f2ce005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:04.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.248+0000 7f2cf27fc700 1 -- 172.21.15.121:0/3504723300 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9706+0+0 (secure 0 0 0) 0x7f2cf4074bc0 con 0x7f2cfc152140 2023-11-15T03:11:04.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.248+0000 7f2d02277700 1 --2- 172.21.15.121:0/3504723300 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2ce005a510 0x7f2ce005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:04.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.248+0000 7f2d02277700 1 --2- 172.21.15.121:0/3504723300 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2ce005a510 0x7f2ce005c9d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f2cf8011310 tx=0x7f2cf8009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:04.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.252+0000 7f2cf27fc700 1 -- 172.21.15.121:0/3504723300 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2cf4047530 con 0x7f2cfc152140 2023-11-15T03:11:04.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.456+0000 7f2d03279700 1 -- 172.21.15.121:0/3504723300 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2cfc06bcf0 con 0x7f2cfc152140 2023-11-15T03:11:04.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.456+0000 7f2cf27fc700 1 -- 172.21.15.121:0/3504723300 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 342 v342) v1 ==== 95+0+4497 (secure 0 0 0) 0x7f2cf404ab50 con 0x7f2cfc152140 2023-11-15T03:11:04.466 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:04.466 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":342,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":10542,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2943522178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2943522178},{"type":"v1","addr":"172.21.15.121:6836","nonce":2943522178}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":342}],"filesystems":[{"mdsmap":{"epoch":341,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:11:02.249759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:11:04.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.460+0000 7f2d03279700 1 -- 172.21.15.121:0/3504723300 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2ce005a510 msgr2=0x7f2ce005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:04.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.460+0000 7f2d03279700 1 --2- 172.21.15.121:0/3504723300 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2ce005a510 0x7f2ce005c9d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f2cf8011310 tx=0x7f2cf8009060).stop 2023-11-15T03:11:04.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.460+0000 7f2d03279700 1 -- 172.21.15.121:0/3504723300 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc152140 msgr2=0x7f2cfc0a8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:04.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.460+0000 7f2d03279700 1 --2- 172.21.15.121:0/3504723300 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc152140 0x7f2cfc0a8750 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 rx=0x7f2cf400c7c0 tx=0x7f2cf4001560).stop 2023-11-15T03:11:04.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.460+0000 7f2d03279700 1 -- 172.21.15.121:0/3504723300 shutdown_connections 2023-11-15T03:11:04.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.460+0000 7f2d03279700 1 --2- 172.21.15.121:0/3504723300 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2ce005a510 0x7f2ce005c9d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.460+0000 7f2d03279700 1 --2- 172.21.15.121:0/3504723300 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cfc152140 0x7f2cfc0a8750 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.460+0000 7f2d03279700 1 --2- 172.21.15.121:0/3504723300 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cfc151870 0x7f2cfc1646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.460+0000 7f2d03279700 1 --2- 172.21.15.121:0/3504723300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cfc0a9d20 0x7f2cfc0c36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.460+0000 7f2d03279700 1 -- 172.21.15.121:0/3504723300 >> 172.21.15.121:0/3504723300 conn(0x7f2cfc062190 msgr2=0x7f2cfc14e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:04.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.460+0000 7f2d03279700 1 -- 172.21.15.121:0/3504723300 shutdown_connections 2023-11-15T03:11:04.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.460+0000 7f2d03279700 1 -- 172.21.15.121:0/3504723300 wait complete. 2023-11-15T03:11:04.471 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 342 2023-11-15T03:11:04.488 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-15T03:11:04.489 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:04.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.640+0000 7f37ea169700 1 Processor -- start 2023-11-15T03:11:04.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.640+0000 7f37ea169700 1 -- start start 2023-11-15T03:11:04.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.644+0000 7f37ea169700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37e4152780 0x7f37e4159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:04.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.644+0000 7f37ea169700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f37e40a5510 0x7f37e40a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:04.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.644+0000 7f37ea169700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e40c3fa0 con 0x7f37e40c65d0 2023-11-15T03:11:04.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.644+0000 7f37ea169700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e4159630 con 0x7f37e40a5510 2023-11-15T03:11:04.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.644+0000 7f37ea169700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e40c30b0 con 0x7f37e4152780 2023-11-15T03:11:04.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.644+0000 7f37daffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f37e40c65d0 0x7f37e40c6a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58772/0 (socket says 172.21.15.121:58772) 2023-11-15T03:11:04.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.644+0000 7f37daffd700 1 -- 172.21.15.121:0/196436485 learned_addr learned my addr 172.21.15.121:0/196436485 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:04.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.644+0000 7f37e37fe700 1 --2- 172.21.15.121:0/196436485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37e4152780 0x7f37e4159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:04.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.644+0000 7f37e3fff700 1 --2- 172.21.15.121:0/196436485 >> v2:172.21.15.121:3300/0 conn(0x7f37e40a5510 0x7f37e40a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:04.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.644+0000 7f37e2ffd700 1 -- 172.21.15.121:0/196436485 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1299206183 0 0) 0x7f37e40c3fa0 con 0x7f37e40c65d0 2023-11-15T03:11:04.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.648+0000 7f37e2ffd700 1 -- 172.21.15.121:0/196436485 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37d00033f0 con 0x7f37e40c65d0 2023-11-15T03:11:04.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.648+0000 7f37e3fff700 1 -- 172.21.15.121:0/196436485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37e4152780 msgr2=0x7f37e4159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:04.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.648+0000 7f37e3fff700 1 --2- 172.21.15.121:0/196436485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37e4152780 0x7f37e4159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.648+0000 7f37e3fff700 1 -- 172.21.15.121:0/196436485 >> v1:172.21.15.121:6789/0 conn(0x7f37e40c65d0 legacy=0x7f37e40c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:04.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.648+0000 7f37e3fff700 1 -- 172.21.15.121:0/196436485 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e40c3390 con 0x7f37e40a5510 2023-11-15T03:11:04.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.648+0000 7f37e3fff700 1 --2- 172.21.15.121:0/196436485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37e40a5510 0x7f37e40a59c0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 rx=0x7f37dc009760 tx=0x7f37dc009060).ready entity=mon.0 client_cookie=b0b978327e215890 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:04.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.648+0000 7f37e2ffd700 1 -- 172.21.15.121:0/196436485 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc00d4a0 con 0x7f37e40a5510 2023-11-15T03:11:04.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.648+0000 7f37e2ffd700 1 -- 172.21.15.121:0/196436485 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37dc00d600 con 0x7f37e40a5510 2023-11-15T03:11:04.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.648+0000 7f37e2ffd700 1 -- 172.21.15.121:0/196436485 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc00d980 con 0x7f37e40a5510 2023-11-15T03:11:04.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.648+0000 7f37ea169700 1 -- 172.21.15.121:0/196436485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37e40a5510 msgr2=0x7f37e40a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:04.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.648+0000 7f37ea169700 1 --2- 172.21.15.121:0/196436485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37e40a5510 0x7f37e40a59c0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 rx=0x7f37dc009760 tx=0x7f37dc009060).stop 2023-11-15T03:11:04.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.648+0000 7f37ea169700 1 -- 172.21.15.121:0/196436485 shutdown_connections 2023-11-15T03:11:04.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.648+0000 7f37ea169700 1 --2- 172.21.15.121:0/196436485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37e40a5510 0x7f37e40a59c0 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.648+0000 7f37ea169700 1 --2- 172.21.15.121:0/196436485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37e4152780 0x7f37e4159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.648+0000 7f37ea169700 1 -- 172.21.15.121:0/196436485 >> 172.21.15.121:0/196436485 conn(0x7f37e4062190 msgr2=0x7f37e40625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:04.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.648+0000 7f37ea169700 1 -- 172.21.15.121:0/196436485 shutdown_connections 2023-11-15T03:11:04.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.648+0000 7f37ea169700 1 -- 172.21.15.121:0/196436485 wait complete. 2023-11-15T03:11:04.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37ea169700 1 Processor -- start 2023-11-15T03:11:04.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37ea169700 1 -- start start 2023-11-15T03:11:04.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37ea169700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37e40a5510 0x7f37e4145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:04.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37ea169700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37e40c65d0 0x7f37e4158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:04.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37ea169700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37e4152780 0x7f37e4158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:04.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37ea169700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e4158f00 con 0x7f37e40a5510 2023-11-15T03:11:04.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37ea169700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e41463a0 con 0x7f37e4152780 2023-11-15T03:11:04.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37ea169700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e4146f20 con 0x7f37e40c65d0 2023-11-15T03:11:04.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37e37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37e40a5510 0x7f37e4145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:04.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37e37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37e40a5510 0x7f37e4145ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39928/0 (socket says 172.21.15.121:39928) 2023-11-15T03:11:04.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37e37fe700 1 -- 172.21.15.121:0/2636307344 learned_addr learned my addr 172.21.15.121:0/2636307344 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:04.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37daffd700 1 --2- 172.21.15.121:0/2636307344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37e40c65d0 0x7f37e4158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:04.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37e3fff700 1 --2- 172.21.15.121:0/2636307344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37e4152780 0x7f37e4158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:04.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37e37fe700 1 -- 172.21.15.121:0/2636307344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37e40c65d0 msgr2=0x7f37e4158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:04.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37e37fe700 1 --2- 172.21.15.121:0/2636307344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37e40c65d0 0x7f37e4158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37e37fe700 1 -- 172.21.15.121:0/2636307344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37e4152780 msgr2=0x7f37e4158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:04.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37e37fe700 1 --2- 172.21.15.121:0/2636307344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37e4152780 0x7f37e4158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37e37fe700 1 -- 172.21.15.121:0/2636307344 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37dc0092a0 con 0x7f37e40a5510 2023-11-15T03:11:04.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37e37fe700 1 --2- 172.21.15.121:0/2636307344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37e40a5510 0x7f37e4145ac0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 rx=0x7f37d400e900 tx=0x7f37d4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:04.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37dbfff700 1 -- 172.21.15.121:0/2636307344 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d400c640 con 0x7f37e40a5510 2023-11-15T03:11:04.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37dbfff700 1 -- 172.21.15.121:0/2636307344 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d400c7a0 con 0x7f37e40a5510 2023-11-15T03:11:04.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37dbfff700 1 -- 172.21.15.121:0/2636307344 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d400cb20 con 0x7f37e40a5510 2023-11-15T03:11:04.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37ea169700 1 -- 172.21.15.121:0/2636307344 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e4150c40 con 0x7f37e40a5510 2023-11-15T03:11:04.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.652+0000 7f37ea169700 1 -- 172.21.15.121:0/2636307344 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e41468d0 con 0x7f37e40a5510 2023-11-15T03:11:04.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.656+0000 7f37ea169700 1 -- 172.21.15.121:0/2636307344 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e406c150 con 0x7f37e40a5510 2023-11-15T03:11:04.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.656+0000 7f37dbfff700 1 -- 172.21.15.121:0/2636307344 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f37d401b400 con 0x7f37e40a5510 2023-11-15T03:11:04.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.660+0000 7f37dbfff700 1 --2- 172.21.15.121:0/2636307344 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f37c405a540 0x7f37c405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:04.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.660+0000 7f37dbfff700 1 -- 172.21.15.121:0/2636307344 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9706+0+0 (secure 0 0 0) 0x7f37d4075a30 con 0x7f37e40a5510 2023-11-15T03:11:04.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.660+0000 7f37daffd700 1 --2- 172.21.15.121:0/2636307344 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f37c405a540 0x7f37c405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:04.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.660+0000 7f37daffd700 1 --2- 172.21.15.121:0/2636307344 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f37c405a540 0x7f37c405ca00 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7f37e40662b0 tx=0x7f37cc009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:04.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.664+0000 7f37dbfff700 1 -- 172.21.15.121:0/2636307344 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f37d4048370 con 0x7f37e40a5510 2023-11-15T03:11:04.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.868+0000 7f37ea169700 1 -- 172.21.15.121:0/2636307344 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37e40c3670 con 0x7f37e40a5510 2023-11-15T03:11:04.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.868+0000 7f37dbfff700 1 -- 172.21.15.121:0/2636307344 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 342 v342) v1 ==== 95+0+4497 (secure 0 0 0) 0x7f37d4016030 con 0x7f37e40a5510 2023-11-15T03:11:04.879 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:04.879 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":342,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":10542,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2943522178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2943522178},{"type":"v1","addr":"172.21.15.121:6836","nonce":2943522178}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":342}],"filesystems":[{"mdsmap":{"epoch":341,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:11:02.249759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:11:04.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.872+0000 7f37ea169700 1 -- 172.21.15.121:0/2636307344 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f37c405a540 msgr2=0x7f37c405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:04.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.872+0000 7f37ea169700 1 --2- 172.21.15.121:0/2636307344 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f37c405a540 0x7f37c405ca00 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7f37e40662b0 tx=0x7f37cc009040).stop 2023-11-15T03:11:04.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.872+0000 7f37ea169700 1 -- 172.21.15.121:0/2636307344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37e40a5510 msgr2=0x7f37e4145ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:04.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.872+0000 7f37ea169700 1 --2- 172.21.15.121:0/2636307344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37e40a5510 0x7f37e4145ac0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 rx=0x7f37d400e900 tx=0x7f37d4009060).stop 2023-11-15T03:11:04.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.872+0000 7f37ea169700 1 -- 172.21.15.121:0/2636307344 shutdown_connections 2023-11-15T03:11:04.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.872+0000 7f37ea169700 1 --2- 172.21.15.121:0/2636307344 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f37c405a540 0x7f37c405ca00 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.872+0000 7f37ea169700 1 --2- 172.21.15.121:0/2636307344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37e4152780 0x7f37e4158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.872+0000 7f37ea169700 1 --2- 172.21.15.121:0/2636307344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37e40c65d0 0x7f37e4158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.872+0000 7f37ea169700 1 --2- 172.21.15.121:0/2636307344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37e40a5510 0x7f37e4145ac0 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:04.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.872+0000 7f37ea169700 1 -- 172.21.15.121:0/2636307344 >> 172.21.15.121:0/2636307344 conn(0x7f37e4062190 msgr2=0x7f37e414ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:04.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.872+0000 7f37ea169700 1 -- 172.21.15.121:0/2636307344 shutdown_connections 2023-11-15T03:11:04.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:04.872+0000 7f37ea169700 1 -- 172.21.15.121:0/2636307344 wait complete. 2023-11-15T03:11:04.885 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 342 2023-11-15T03:11:04.902 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 1.00000000' led to DAMAGED state 2023-11-15T03:11:04.902 INFO:tasks.cephfs.test_damage:All 39 mutations had expected outcomes 2023-11-15T03:11:04.903 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-15T03:11:04.903 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T03:11:04.954 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T03:11:05.002 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-15T03:11:05.004 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:05.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.164+0000 7fd56a569700 1 Processor -- start 2023-11-15T03:11:05.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.164+0000 7fd56a569700 1 -- start start 2023-11-15T03:11:05.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd56a569700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd564153bb0 0x7fd5640a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:05.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd56a569700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd564068170 0x7fd564065d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:05.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd56a569700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5640cb5c0 con 0x7fd5640aa020 2023-11-15T03:11:05.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd56a569700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5640aa6f0 con 0x7fd564153bb0 2023-11-15T03:11:05.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd56a569700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5640c5900 con 0x7fd564068170 2023-11-15T03:11:05.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd55bfff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fd5640aa020 0x7fd564067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:48992/0 (socket says 172.21.15.121:48992) 2023-11-15T03:11:05.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd55bfff700 1 -- 172.21.15.121:0/3773853847 learned_addr learned my addr 172.21.15.121:0/3773853847 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:05.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd563fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd564153bb0 0x7fd5640a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:05.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd568b06700 1 --2- 172.21.15.121:0/3773853847 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd564068170 0x7fd564065d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:05.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd5637fe700 1 -- 172.21.15.121:0/3773853847 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501774911 0 0) 0x7fd5640cb5c0 con 0x7fd5640aa020 2023-11-15T03:11:05.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd5637fe700 1 -- 172.21.15.121:0/3773853847 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5500033f0 con 0x7fd5640aa020 2023-11-15T03:11:05.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd563fff700 1 -- 172.21.15.121:0/3773853847 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd564068170 msgr2=0x7fd564065d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:05.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd563fff700 1 --2- 172.21.15.121:0/3773853847 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd564068170 0x7fd564065d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd563fff700 1 -- 172.21.15.121:0/3773853847 >> v1:172.21.15.121:6789/0 conn(0x7fd5640aa020 legacy=0x7fd564067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:05.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd563fff700 1 -- 172.21.15.121:0/3773853847 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5640c9ca0 con 0x7fd564153bb0 2023-11-15T03:11:05.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd568b06700 1 --2- 172.21.15.121:0/3773853847 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd564068170 0x7fd564065d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:05.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd563fff700 1 --2- 172.21.15.121:0/3773853847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd564153bb0 0x7fd5640a9b30 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 rx=0x7fd554009760 tx=0x7fd554009060).ready entity=mon.0 client_cookie=f322a9dd04f4fc6a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:05.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd5637fe700 1 -- 172.21.15.121:0/3773853847 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55400f510 con 0x7fd564153bb0 2023-11-15T03:11:05.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd5637fe700 1 -- 172.21.15.121:0/3773853847 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55400f670 con 0x7fd564153bb0 2023-11-15T03:11:05.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.168+0000 7fd5637fe700 1 -- 172.21.15.121:0/3773853847 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55400f9f0 con 0x7fd564153bb0 2023-11-15T03:11:05.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.172+0000 7fd56a569700 1 -- 172.21.15.121:0/3773853847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd564153bb0 msgr2=0x7fd5640a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:05.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.172+0000 7fd56a569700 1 --2- 172.21.15.121:0/3773853847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd564153bb0 0x7fd5640a9b30 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 rx=0x7fd554009760 tx=0x7fd554009060).stop 2023-11-15T03:11:05.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.172+0000 7fd56a569700 1 -- 172.21.15.121:0/3773853847 shutdown_connections 2023-11-15T03:11:05.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.172+0000 7fd56a569700 1 --2- 172.21.15.121:0/3773853847 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd564068170 0x7fd564065d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.172+0000 7fd56a569700 1 --2- 172.21.15.121:0/3773853847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd564153bb0 0x7fd5640a9b30 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.172+0000 7fd56a569700 1 -- 172.21.15.121:0/3773853847 >> 172.21.15.121:0/3773853847 conn(0x7fd564058c50 msgr2=0x7fd564059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:05.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.172+0000 7fd56a569700 1 -- 172.21.15.121:0/3773853847 shutdown_connections 2023-11-15T03:11:05.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.172+0000 7fd56a569700 1 -- 172.21.15.121:0/3773853847 wait complete. 2023-11-15T03:11:05.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.172+0000 7fd56a569700 1 Processor -- start 2023-11-15T03:11:05.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.172+0000 7fd56a569700 1 -- start start 2023-11-15T03:11:05.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.172+0000 7fd56a569700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd564068170 0x7fd56413b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:05.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.172+0000 7fd56a569700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5640aa020 0x7fd56413ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:05.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.172+0000 7fd56a569700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd564153bb0 0x7fd5640c35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:05.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.172+0000 7fd56a569700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd564151d70 con 0x7fd564153bb0 2023-11-15T03:11:05.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.172+0000 7fd56a569700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5640ac480 con 0x7fd564068170 2023-11-15T03:11:05.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.172+0000 7fd56a569700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5640ab4e0 con 0x7fd5640aa020 2023-11-15T03:11:05.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.176+0000 7fd568b06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd564153bb0 0x7fd5640c35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:05.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.176+0000 7fd568b06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd564153bb0 0x7fd5640c35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39876/0 (socket says 172.21.15.121:39876) 2023-11-15T03:11:05.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.176+0000 7fd568b06700 1 -- 172.21.15.121:0/3677747559 learned_addr learned my addr 172.21.15.121:0/3677747559 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:05.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.176+0000 7fd55bfff700 1 --2- 172.21.15.121:0/3677747559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5640aa020 0x7fd56413ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:05.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.176+0000 7fd563fff700 1 --2- 172.21.15.121:0/3677747559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd564068170 0x7fd56413b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:05.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.176+0000 7fd568b06700 1 -- 172.21.15.121:0/3677747559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5640aa020 msgr2=0x7fd56413ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:05.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.176+0000 7fd568b06700 1 --2- 172.21.15.121:0/3677747559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5640aa020 0x7fd56413ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.176+0000 7fd568b06700 1 -- 172.21.15.121:0/3677747559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd564068170 msgr2=0x7fd56413b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:05.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.176+0000 7fd568b06700 1 --2- 172.21.15.121:0/3677747559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd564068170 0x7fd56413b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.176+0000 7fd568b06700 1 -- 172.21.15.121:0/3677747559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5540092a0 con 0x7fd564153bb0 2023-11-15T03:11:05.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.176+0000 7fd563fff700 1 --2- 172.21.15.121:0/3677747559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd564068170 0x7fd56413b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:05.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.176+0000 7fd568b06700 1 --2- 172.21.15.121:0/3677747559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd564153bb0 0x7fd5640c35d0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 rx=0x7fd55c009f70 tx=0x7fd55c0099b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:05.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.176+0000 7fd560ff9700 1 -- 172.21.15.121:0/3677747559 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c00dc30 con 0x7fd564153bb0 2023-11-15T03:11:05.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.176+0000 7fd560ff9700 1 -- 172.21.15.121:0/3677747559 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55c00f040 con 0x7fd564153bb0 2023-11-15T03:11:05.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.176+0000 7fd560ff9700 1 -- 172.21.15.121:0/3677747559 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c0133f0 con 0x7fd564153bb0 2023-11-15T03:11:05.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.176+0000 7fd56a569700 1 -- 172.21.15.121:0/3677747559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd564151720 con 0x7fd564153bb0 2023-11-15T03:11:05.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.176+0000 7fd56a569700 1 -- 172.21.15.121:0/3677747559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56413bbc0 con 0x7fd564153bb0 2023-11-15T03:11:05.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.176+0000 7fd56a569700 1 -- 172.21.15.121:0/3677747559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd564151f20 con 0x7fd564153bb0 2023-11-15T03:11:05.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.180+0000 7fd560ff9700 1 -- 172.21.15.121:0/3677747559 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd55c0137e0 con 0x7fd564153bb0 2023-11-15T03:11:05.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.180+0000 7fd560ff9700 1 --2- 172.21.15.121:0/3677747559 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd54405a4e0 0x7fd54405c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:05.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.180+0000 7fd560ff9700 1 -- 172.21.15.121:0/3677747559 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9706+0+0 (secure 0 0 0) 0x7fd55c075cf0 con 0x7fd564153bb0 2023-11-15T03:11:05.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.180+0000 7fd563fff700 1 --2- 172.21.15.121:0/3677747559 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd54405a4e0 0x7fd54405c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:05.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.180+0000 7fd563fff700 1 --2- 172.21.15.121:0/3677747559 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd54405a4e0 0x7fd54405c9a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7fd554009fd0 tx=0x7fd554017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:05.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.188+0000 7fd560ff9700 1 -- 172.21.15.121:0/3677747559 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd564151f20 con 0x7fd564153bb0 2023-11-15T03:11:05.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.392+0000 7fd56a569700 1 -- 172.21.15.121:0/3677747559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd56406c150 con 0x7fd564153bb0 2023-11-15T03:11:05.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.392+0000 7fd560ff9700 1 -- 172.21.15.121:0/3677747559 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 342 v342) v1 ==== 95+0+4497 (secure 0 0 0) 0x7fd55c048610 con 0x7fd564153bb0 2023-11-15T03:11:05.400 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:05.400 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":342,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":10542,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2943522178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2943522178},{"type":"v1","addr":"172.21.15.121:6836","nonce":2943522178}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":342}],"filesystems":[{"mdsmap":{"epoch":341,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:11:02.249759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:11:05.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.396+0000 7fd56a569700 1 -- 172.21.15.121:0/3677747559 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd54405a4e0 msgr2=0x7fd54405c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:05.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.396+0000 7fd56a569700 1 --2- 172.21.15.121:0/3677747559 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd54405a4e0 0x7fd54405c9a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7fd554009fd0 tx=0x7fd554017000).stop 2023-11-15T03:11:05.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.396+0000 7fd56a569700 1 -- 172.21.15.121:0/3677747559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd564153bb0 msgr2=0x7fd5640c35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:05.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.396+0000 7fd56a569700 1 --2- 172.21.15.121:0/3677747559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd564153bb0 0x7fd5640c35d0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 rx=0x7fd55c009f70 tx=0x7fd55c0099b0).stop 2023-11-15T03:11:05.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.396+0000 7fd56a569700 1 -- 172.21.15.121:0/3677747559 shutdown_connections 2023-11-15T03:11:05.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.396+0000 7fd56a569700 1 --2- 172.21.15.121:0/3677747559 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd54405a4e0 0x7fd54405c9a0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.396+0000 7fd56a569700 1 --2- 172.21.15.121:0/3677747559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd564153bb0 0x7fd5640c35d0 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.396+0000 7fd56a569700 1 --2- 172.21.15.121:0/3677747559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5640aa020 0x7fd56413ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.396+0000 7fd56a569700 1 --2- 172.21.15.121:0/3677747559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd564068170 0x7fd56413b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.396+0000 7fd56a569700 1 -- 172.21.15.121:0/3677747559 >> 172.21.15.121:0/3677747559 conn(0x7fd564058c50 msgr2=0x7fd5640b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:05.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.396+0000 7fd56a569700 1 -- 172.21.15.121:0/3677747559 shutdown_connections 2023-11-15T03:11:05.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.396+0000 7fd56a569700 1 -- 172.21.15.121:0/3677747559 wait complete. 2023-11-15T03:11:05.408 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 342 2023-11-15T03:11:05.425 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:11:05.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.588+0000 7f29a41a9700 1 Processor -- start 2023-11-15T03:11:05.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.588+0000 7f29a41a9700 1 -- start start 2023-11-15T03:11:05.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.592+0000 7f29a41a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0c6ce0 0x7f299c159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:05.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.592+0000 7f29a41a9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f299c0a5510 0x7f299c0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:05.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.592+0000 7f29a41a9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f299c0c3fa0 con 0x7f299c152780 2023-11-15T03:11:05.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.592+0000 7f29a41a9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299c159630 con 0x7f299c0a5510 2023-11-15T03:11:05.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.592+0000 7f29a41a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f299c0c30b0 con 0x7f299c0c6ce0 2023-11-15T03:11:05.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.592+0000 7f29a1744700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0c6ce0 0x7f299c159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:05.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.592+0000 7f29a2746700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f299c0a5510 0x7f299c0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:05.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.592+0000 7f29a1f45700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f299c152780 0x7f299c0c65d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:54604/0 (socket says 172.21.15.121:54604) 2023-11-15T03:11:05.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.592+0000 7f29a1f45700 1 -- 172.21.15.121:0/319354312 learned_addr learned my addr 172.21.15.121:0/319354312 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:05.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.592+0000 7f29a1744700 1 -- 172.21.15.121:0/319354312 >> v2:172.21.15.121:3300/0 conn(0x7f299c0a5510 msgr2=0x7f299c0a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:05.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.592+0000 7f29a1744700 1 --2- 172.21.15.121:0/319354312 >> v2:172.21.15.121:3300/0 conn(0x7f299c0a5510 0x7f299c0a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.592+0000 7f29a1744700 1 -- 172.21.15.121:0/319354312 >> v1:172.21.15.190:6789/0 conn(0x7f299c152780 legacy=0x7f299c0c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:05.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.592+0000 7f29a1744700 1 -- 172.21.15.121:0/319354312 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299c0c3390 con 0x7f299c0c6ce0 2023-11-15T03:11:05.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.592+0000 7f29a2746700 1 --2- 172.21.15.121:0/319354312 >> v2:172.21.15.121:3300/0 conn(0x7f299c0a5510 0x7f299c0a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:05.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.592+0000 7f29a1744700 1 --2- 172.21.15.121:0/319354312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0c6ce0 0x7f299c159140 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f298c009760 tx=0x7f298c009060).ready entity=mon.2 client_cookie=806db4a14326e8d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:05.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.596+0000 7f29a0f43700 1 -- 172.21.15.121:0/319354312 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c00f510 con 0x7f299c0c6ce0 2023-11-15T03:11:05.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.596+0000 7f29a0f43700 1 -- 172.21.15.121:0/319354312 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298c00f670 con 0x7f299c0c6ce0 2023-11-15T03:11:05.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.596+0000 7f29a0f43700 1 -- 172.21.15.121:0/319354312 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c00f9f0 con 0x7f299c0c6ce0 2023-11-15T03:11:05.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.596+0000 7f29a41a9700 1 -- 172.21.15.121:0/319354312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0c6ce0 msgr2=0x7f299c159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:05.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.596+0000 7f29a41a9700 1 --2- 172.21.15.121:0/319354312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0c6ce0 0x7f299c159140 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f298c009760 tx=0x7f298c009060).stop 2023-11-15T03:11:05.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.596+0000 7f29a41a9700 1 -- 172.21.15.121:0/319354312 shutdown_connections 2023-11-15T03:11:05.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.596+0000 7f29a41a9700 1 --2- 172.21.15.121:0/319354312 >> v2:172.21.15.121:3300/0 conn(0x7f299c0a5510 0x7f299c0a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.596+0000 7f29a41a9700 1 --2- 172.21.15.121:0/319354312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0c6ce0 0x7f299c159140 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.596+0000 7f29a41a9700 1 -- 172.21.15.121:0/319354312 >> 172.21.15.121:0/319354312 conn(0x7f299c062190 msgr2=0x7f299c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:05.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.596+0000 7f29a41a9700 1 -- 172.21.15.121:0/319354312 shutdown_connections 2023-11-15T03:11:05.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.596+0000 7f29a41a9700 1 -- 172.21.15.121:0/319354312 wait complete. 2023-11-15T03:11:05.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.600+0000 7f29a41a9700 1 Processor -- start 2023-11-15T03:11:05.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.600+0000 7f29a41a9700 1 -- start start 2023-11-15T03:11:05.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.600+0000 7f29a41a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f299c0a5510 0x7f299c14f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:05.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.600+0000 7f29a41a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f299c0c6ce0 0x7f299c16c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:05.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.600+0000 7f29a41a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c152780 0x7f299c16c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:05.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.600+0000 7f29a41a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299c0ac6b0 con 0x7f299c0c6ce0 2023-11-15T03:11:05.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.600+0000 7f29a41a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299c16cb00 con 0x7f299c0a5510 2023-11-15T03:11:05.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.600+0000 7f29a41a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f299c13b450 con 0x7f299c152780 2023-11-15T03:11:05.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.604+0000 7f29a1744700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f299c0c6ce0 0x7f299c16c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:05.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.604+0000 7f29a1744700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f299c0c6ce0 0x7f299c16c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39896/0 (socket says 172.21.15.121:39896) 2023-11-15T03:11:05.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.604+0000 7f29a1744700 1 -- 172.21.15.121:0/1125057910 learned_addr learned my addr 172.21.15.121:0/1125057910 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:05.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.604+0000 7f29a1f45700 1 --2- 172.21.15.121:0/1125057910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f299c0a5510 0x7f299c14f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:05.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.604+0000 7f29a2746700 1 --2- 172.21.15.121:0/1125057910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c152780 0x7f299c16c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:05.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.604+0000 7f29a1744700 1 -- 172.21.15.121:0/1125057910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c152780 msgr2=0x7f299c16c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:05.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.604+0000 7f29a1744700 1 --2- 172.21.15.121:0/1125057910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c152780 0x7f299c16c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.604+0000 7f29a1744700 1 -- 172.21.15.121:0/1125057910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f299c0a5510 msgr2=0x7f299c14f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:05.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.604+0000 7f29a1744700 1 --2- 172.21.15.121:0/1125057910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f299c0a5510 0x7f299c14f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.604+0000 7f29a1744700 1 -- 172.21.15.121:0/1125057910 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2994009a20 con 0x7f299c0c6ce0 2023-11-15T03:11:05.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.604+0000 7f29a1744700 1 --2- 172.21.15.121:0/1125057910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f299c0c6ce0 0x7f299c16c120 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 rx=0x7f298c006d70 tx=0x7f298c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:05.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.604+0000 7f2991ffb700 1 -- 172.21.15.121:0/1125057910 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c00f7e0 con 0x7f299c0c6ce0 2023-11-15T03:11:05.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.604+0000 7f2991ffb700 1 -- 172.21.15.121:0/1125057910 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298c00f940 con 0x7f299c0c6ce0 2023-11-15T03:11:05.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.604+0000 7f2991ffb700 1 -- 172.21.15.121:0/1125057910 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c01b660 con 0x7f299c0c6ce0 2023-11-15T03:11:05.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.604+0000 7f29a41a9700 1 -- 172.21.15.121:0/1125057910 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298c0092a0 con 0x7f299c0c6ce0 2023-11-15T03:11:05.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.604+0000 7f29a41a9700 1 -- 172.21.15.121:0/1125057910 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f299c150100 con 0x7f299c0c6ce0 2023-11-15T03:11:05.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.604+0000 7f29a41a9700 1 -- 172.21.15.121:0/1125057910 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f299c06c150 con 0x7f299c0c6ce0 2023-11-15T03:11:05.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.608+0000 7f2991ffb700 1 -- 172.21.15.121:0/1125057910 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f298c01ba10 con 0x7f299c0c6ce0 2023-11-15T03:11:05.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.608+0000 7f2991ffb700 1 --2- 172.21.15.121:0/1125057910 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f298005a510 0x7f298005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:05.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.608+0000 7f2991ffb700 1 -- 172.21.15.121:0/1125057910 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9706+0+0 (secure 0 0 0) 0x7f298c076f80 con 0x7f299c0c6ce0 2023-11-15T03:11:05.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.608+0000 7f29a1f45700 1 --2- 172.21.15.121:0/1125057910 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f298005a510 0x7f298005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:05.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.612+0000 7f29a1f45700 1 --2- 172.21.15.121:0/1125057910 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f298005a510 0x7f298005c9d0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7f2998001630 tx=0x7f2998009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:05.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.616+0000 7f2991ffb700 1 -- 172.21.15.121:0/1125057910 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f298c049920 con 0x7f299c0c6ce0 2023-11-15T03:11:05.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.780+0000 7f29a41a9700 1 -- 172.21.15.121:0/1125057910 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f299c0c3670 con 0x7f299c0c6ce0 2023-11-15T03:11:05.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.780+0000 7f2991ffb700 1 -- 172.21.15.121:0/1125057910 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+22474 (secure 0 0 0) 0x7f298c029070 con 0x7f299c0c6ce0 2023-11-15T03:11:05.791 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:05.791 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":134,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:11:01.253780+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-15T03:03:19.872820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-15T03:03:20.771478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.26","mappings":[{"from":1,"to":3}]},{"pgid":"14.1","mappings":[{"from":1,"to":5}]},{"pgid":"14.17","mappings":[{"from":1,"to":7}]},{"pgid":"14.1b","mappings":[{"from":1,"to":7}]},{"pgid":"14.38","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6834/3720977338":"2023-11-16T03:11:01.253720+0000","172.21.15.121:6836/2269363825":"2023-11-16T03:10:47.448136+0000","172.21.15.121:6834/4290809693":"2023-11-16T03:10:39.351864+0000","172.21.15.121:6836/1750624080":"2023-11-16T03:10:32.820769+0000","172.21.15.121:6836/2622666226":"2023-11-16T03:10:16.226454+0000","172.21.15.121:6834/2622666226":"2023-11-16T03:10:16.226454+0000","172.21.15.121:6834/1175031592":"2023-11-16T03:09:57.665840+0000","172.21.15.121:6836/621064675":"2023-11-16T03:09:34.308003+0000","172.21.15.121:6834/621064675":"2023-11-16T03:09:34.308003+0000","172.21.15.121:6836/142843299":"2023-11-16T03:09:27.621091+0000","172.21.15.121:6836/2666431227":"2023-11-16T03:09:01.175268+0000","172.21.15.121:6836/3720977338":"2023-11-16T03:11:01.253720+0000","172.21.15.121:6834/2666431227":"2023-11-16T03:09:01.175268+0000","172.21.15.121:6836/240077764":"2023-11-16T03:10:09.963248+0000","172.21.15.121:6836/3050548459":"2023-11-16T03:08:50.252959+0000","172.21.15.121:6834/3050548459":"2023-11-16T03:08:50.252959+0000","172.21.15.121:6836/1175031592":"2023-11-16T03:09:57.665840+0000","172.21.15.121:6836/2860780116":"2023-11-16T03:05:25.265026+0000","172.21.15.121:6834/2860780116":"2023-11-16T03:05:25.265026+0000","172.21.15.121:6836/2402059232":"2023-11-16T03:05:11.153161+0000","172.21.15.121:6834/1378715461":"2023-11-16T03:03:55.220003+0000","172.21.15.121:6834/1704127295":"2023-11-16T03:07:01.277510+0000","172.21.15.121:6836/400230871":"2023-11-16T03:05:55.172342+0000","172.21.15.121:6834/2873318212":"2023-11-16T03:08:06.315236+0000","172.21.15.121:6834/2402059232":"2023-11-16T03:05:11.153161+0000","172.21.15.121:6836/682793496":"2023-11-16T03:05:00.228736+0000","172.21.15.121:6834/3831023402":"2023-11-16T03:04:56.741702+0000","172.21.15.121:6834/2127410277":"2023-11-16T03:06:27.278977+0000","172.21.15.121:6834/682793496":"2023-11-16T03:05:00.228736+0000","172.21.15.121:6836/1529582418":"2023-11-16T03:05:32.331792+0000","172.21.15.121:6834/2831707405":"2023-11-16T03:04:44.820301+0000","172.21.15.121:6834/2289093120":"2023-11-16T03:06:57.977388+0000","172.21.15.121:6834/1750624080":"2023-11-16T03:10:32.820769+0000","172.21.15.121:6834/2646177140":"2023-11-16T03:03:30.945890+0000","172.21.15.121:6836/4290809693":"2023-11-16T03:10:39.351864+0000","172.21.15.121:6836/2544923351":"2023-11-16T03:07:44.198253+0000","172.21.15.121:6836/461059977":"2023-11-16T03:07:55.217761+0000","172.21.15.121:6836/3454441824":"2023-11-16T03:05:51.702394+0000","172.21.15.121:6836/2831707405":"2023-11-16T03:04:44.820301+0000","172.21.15.121:6836/2289093120":"2023-11-16T03:06:57.977388+0000","172.21.15.121:6834/3956345798":"2023-11-16T03:04:25.109148+0000","172.21.15.121:6834/57668150":"2023-11-16T03:04:14.091017+0000","172.21.15.121:6834/400230871":"2023-11-16T03:05:55.172342+0000","172.21.15.121:6834/228280964":"2023-11-16T03:06:08.282419+0000","172.21.15.121:6836/1378715461":"2023-11-16T03:03:55.220003+0000","172.21.15.121:6836/2873318212":"2023-11-16T03:08:06.315236+0000","172.21.15.121:6834/1529582418":"2023-11-16T03:05:32.331792+0000","172.21.15.121:6836/228280964":"2023-11-16T03:06:08.282419+0000","172.21.15.121:6834/3764682615":"2023-11-16T03:04:03.026914+0000","172.21.15.121:6834/3454441824":"2023-11-16T03:05:51.702394+0000","172.21.15.121:6836/2646177140":"2023-11-16T03:03:30.945890+0000","172.21.15.121:6834/763933658":"2023-11-16T03:03:48.628787+0000","172.21.15.121:6836/3831023402":"2023-11-16T03:04:56.741702+0000","172.21.15.121:6836/57668150":"2023-11-16T03:04:14.091017+0000","172.21.15.121:6836/3764682615":"2023-11-16T03:04:03.026914+0000","172.21.15.121:6834/461059977":"2023-11-16T03:07:55.217761+0000","172.21.15.121:6836/1244489663":"2023-11-16T03:08:29.214705+0000","172.21.15.121:6836/763933658":"2023-11-16T03:03:48.628787+0000","172.21.15.121:6836/3956345798":"2023-11-16T03:04:25.109148+0000","172.21.15.121:6836/1704127295":"2023-11-16T03:07:01.277510+0000","172.21.15.121:6836/564044995":"2023-11-16T03:06:16.372057+0000","172.21.15.121:6834/2269363825":"2023-11-16T03:10:47.448136+0000","172.21.15.121:6836/2127410277":"2023-11-16T03:06:27.278977+0000","172.21.15.121:6834/240077764":"2023-11-16T03:10:09.963248+0000","172.21.15.121:6836/909619389":"2023-11-16T03:06:38.112007+0000","172.21.15.121:6834/1007300937":"2023-11-16T03:07:12.272673+0000","172.21.15.121:6836/1007300937":"2023-11-16T03:07:12.272673+0000","172.21.15.121:6836/2131958479":"2023-11-16T03:07:23.129117+0000","172.21.15.121:6834/564044995":"2023-11-16T03:06:16.372057+0000","172.21.15.121:6836/115896047":"2023-11-16T03:07:34.194687+0000","172.21.15.121:6834/1958064439":"2023-11-16T03:08:25.922572+0000","172.21.15.121:6836/1958064439":"2023-11-16T03:08:25.922572+0000","172.21.15.121:6834/2544923351":"2023-11-16T03:07:44.198253+0000","172.21.15.121:6834/1244489663":"2023-11-16T03:08:29.214705+0000","172.21.15.121:6834/142843299":"2023-11-16T03:09:27.621091+0000","172.21.15.121:6834/1978069160":"2023-11-16T03:08:40.321133+0000","172.21.15.121:6834/115896047":"2023-11-16T03:07:34.194687+0000","172.21.15.121:6834/909619389":"2023-11-16T03:06:38.112007+0000","172.21.15.121:6834/2131958479":"2023-11-16T03:07:23.129117+0000","172.21.15.121:6836/1978069160":"2023-11-16T03:08:40.321133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:11:05.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.788+0000 7f29a41a9700 1 -- 172.21.15.121:0/1125057910 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f298005a510 msgr2=0x7f298005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:05.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.788+0000 7f29a41a9700 1 --2- 172.21.15.121:0/1125057910 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f298005a510 0x7f298005c9d0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7f2998001630 tx=0x7f2998009040).stop 2023-11-15T03:11:05.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.788+0000 7f29a41a9700 1 -- 172.21.15.121:0/1125057910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f299c0c6ce0 msgr2=0x7f299c16c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:05.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.788+0000 7f29a41a9700 1 --2- 172.21.15.121:0/1125057910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f299c0c6ce0 0x7f299c16c120 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 rx=0x7f298c006d70 tx=0x7f298c017000).stop 2023-11-15T03:11:05.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.788+0000 7f29a41a9700 1 -- 172.21.15.121:0/1125057910 shutdown_connections 2023-11-15T03:11:05.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.788+0000 7f29a41a9700 1 --2- 172.21.15.121:0/1125057910 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f298005a510 0x7f298005c9d0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.788+0000 7f29a41a9700 1 --2- 172.21.15.121:0/1125057910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c152780 0x7f299c16c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.788+0000 7f29a41a9700 1 --2- 172.21.15.121:0/1125057910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f299c0c6ce0 0x7f299c16c120 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.788+0000 7f29a41a9700 1 --2- 172.21.15.121:0/1125057910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f299c0a5510 0x7f299c14f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.788+0000 7f29a41a9700 1 -- 172.21.15.121:0/1125057910 >> 172.21.15.121:0/1125057910 conn(0x7f299c062190 msgr2=0x7f299c0c2340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:05.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.788+0000 7f29a41a9700 1 -- 172.21.15.121:0/1125057910 shutdown_connections 2023-11-15T03:11:05.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.788+0000 7f29a41a9700 1 -- 172.21.15.121:0/1125057910 wait complete. 2023-11-15T03:11:05.815 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-15T03:03:19.872820+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-11-15T03:03:20.771478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-15T03:11:05.816 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T03:11:05.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.964+0000 7f8692b07700 1 Processor -- start 2023-11-15T03:11:05.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.968+0000 7f8692b07700 1 -- start start 2023-11-15T03:11:05.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.968+0000 7f8692b07700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f868c14ac10 0x7f868c0c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:05.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.968+0000 7f8692b07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f868c14aff0 0x7f868c066c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:05.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.968+0000 7f8692b07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f868c067160 0x7f868c154370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:05.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.968+0000 7f8692b07700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f868c0657f0 con 0x7f868c14ac10 2023-11-15T03:11:05.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.968+0000 7f8692b07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f868c066650 con 0x7f868c14aff0 2023-11-15T03:11:05.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.968+0000 7f8692b07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868c0675b0 con 0x7f868c067160 2023-11-15T03:11:05.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.968+0000 7f8691b05700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f868c14ac10 0x7f868c0c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:05.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.968+0000 7f8691b05700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f868c14ac10 0x7f868c0c6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39910/0 (socket says 172.21.15.121:39910) 2023-11-15T03:11:05.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.968+0000 7f8691b05700 1 -- 172.21.15.121:0/245469910 learned_addr learned my addr 172.21.15.121:0/245469910 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:05.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.968+0000 7f8691304700 1 --2- 172.21.15.121:0/245469910 >> v2:172.21.15.190:3300/0 conn(0x7f868c14aff0 0x7f868c066c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:05.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.968+0000 7f8691b05700 1 -- 172.21.15.121:0/245469910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f868c067160 msgr2=0x7f868c154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:05.982 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.968+0000 7f8692306700 1 --2- 172.21.15.121:0/245469910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f868c067160 0x7f868c154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:05.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.972+0000 7f8691b05700 1 --2- 172.21.15.121:0/245469910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f868c067160 0x7f868c154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.972+0000 7f8691b05700 1 -- 172.21.15.121:0/245469910 >> v2:172.21.15.190:3300/0 conn(0x7f868c14aff0 msgr2=0x7f868c066c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:05.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.972+0000 7f8691b05700 1 --2- 172.21.15.121:0/245469910 >> v2:172.21.15.190:3300/0 conn(0x7f868c14aff0 0x7f868c066c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.972+0000 7f8691b05700 1 -- 172.21.15.121:0/245469910 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868c065a40 con 0x7f868c14ac10 2023-11-15T03:11:05.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.972+0000 7f8691304700 1 --2- 172.21.15.121:0/245469910 >> v2:172.21.15.190:3300/0 conn(0x7f868c14aff0 0x7f868c066c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:11:05.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.972+0000 7f8692306700 1 --2- 172.21.15.121:0/245469910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f868c067160 0x7f868c154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:11:05.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.972+0000 7f8691b05700 1 --2- 172.21.15.121:0/245469910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f868c14ac10 0x7f868c0c6bc0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 rx=0x7f868c150da0 tx=0x7f868800b3b0).ready entity=mon.0 client_cookie=bf015edb69f28f10 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:05.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.972+0000 7f8690b03700 1 -- 172.21.15.121:0/245469910 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868800db10 con 0x7f868c14ac10 2023-11-15T03:11:05.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.972+0000 7f8690b03700 1 -- 172.21.15.121:0/245469910 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868800dc70 con 0x7f868c14ac10 2023-11-15T03:11:05.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.972+0000 7f8690b03700 1 -- 172.21.15.121:0/245469910 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8688010470 con 0x7f868c14ac10 2023-11-15T03:11:05.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.972+0000 7f8692b07700 1 -- 172.21.15.121:0/245469910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f868c14ac10 msgr2=0x7f868c0c6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:05.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.972+0000 7f8692b07700 1 --2- 172.21.15.121:0/245469910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f868c14ac10 0x7f868c0c6bc0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 rx=0x7f868c150da0 tx=0x7f868800b3b0).stop 2023-11-15T03:11:05.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.972+0000 7f8692b07700 1 -- 172.21.15.121:0/245469910 shutdown_connections 2023-11-15T03:11:05.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.972+0000 7f8692b07700 1 --2- 172.21.15.121:0/245469910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f868c067160 0x7f868c154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.972+0000 7f8692b07700 1 --2- 172.21.15.121:0/245469910 >> v2:172.21.15.190:3300/0 conn(0x7f868c14aff0 0x7f868c066c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.972+0000 7f8692b07700 1 --2- 172.21.15.121:0/245469910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f868c14ac10 0x7f868c0c6bc0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.972+0000 7f8692b07700 1 -- 172.21.15.121:0/245469910 >> 172.21.15.121:0/245469910 conn(0x7f868c067a60 msgr2=0x7f868c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:05.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.972+0000 7f8692b07700 1 -- 172.21.15.121:0/245469910 shutdown_connections 2023-11-15T03:11:05.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.972+0000 7f8692b07700 1 -- 172.21.15.121:0/245469910 wait complete. 2023-11-15T03:11:05.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.976+0000 7f8692b07700 1 Processor -- start 2023-11-15T03:11:05.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.976+0000 7f8692b07700 1 -- start start 2023-11-15T03:11:05.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.976+0000 7f8692b07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f868c067160 0x7f868c142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:05.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.976+0000 7f8692b07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f868c14ac10 0x7f868c147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:05.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.976+0000 7f8692b07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f868c14aff0 0x7f868c1483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:05.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.976+0000 7f8692b07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c149590 con 0x7f868c14ac10 2023-11-15T03:11:05.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.976+0000 7f8692b07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c148980 con 0x7f868c14aff0 2023-11-15T03:11:05.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.976+0000 7f8692b07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868c148c60 con 0x7f868c067160 2023-11-15T03:11:05.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.976+0000 7f8691304700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f868c14ac10 0x7f868c147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:05.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.976+0000 7f8691304700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f868c14ac10 0x7f868c147ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39916/0 (socket says 172.21.15.121:39916) 2023-11-15T03:11:05.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.976+0000 7f8691304700 1 -- 172.21.15.121:0/1313973570 learned_addr learned my addr 172.21.15.121:0/1313973570 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:05.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.976+0000 7f8691b05700 1 --2- 172.21.15.121:0/1313973570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f868c067160 0x7f868c142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:05.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.976+0000 7f8692306700 1 --2- 172.21.15.121:0/1313973570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f868c14aff0 0x7f868c1483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:05.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.976+0000 7f8691304700 1 -- 172.21.15.121:0/1313973570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f868c067160 msgr2=0x7f868c142fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:05.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.976+0000 7f8691304700 1 --2- 172.21.15.121:0/1313973570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f868c067160 0x7f868c142fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.976+0000 7f8691304700 1 -- 172.21.15.121:0/1313973570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f868c14aff0 msgr2=0x7f868c1483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:05.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.976+0000 7f8691304700 1 --2- 172.21.15.121:0/1313973570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f868c14aff0 0x7f868c1483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:05.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.976+0000 7f8691304700 1 -- 172.21.15.121:0/1313973570 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868800b610 con 0x7f868c14ac10 2023-11-15T03:11:05.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.980+0000 7f8691304700 1 --2- 172.21.15.121:0/1313973570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f868c14ac10 0x7f868c147ec0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 rx=0x7f867c00c670 tx=0x7f867c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:05.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.980+0000 7f8681ffb700 1 -- 172.21.15.121:0/1313973570 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c015070 con 0x7f868c14ac10 2023-11-15T03:11:05.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.980+0000 7f8681ffb700 1 -- 172.21.15.121:0/1313973570 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c010580 con 0x7f868c14ac10 2023-11-15T03:11:05.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.980+0000 7f8681ffb700 1 -- 172.21.15.121:0/1313973570 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c010900 con 0x7f868c14ac10 2023-11-15T03:11:05.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.980+0000 7f8692b07700 1 -- 172.21.15.121:0/1313973570 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868c148f40 con 0x7f868c14ac10 2023-11-15T03:11:05.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.980+0000 7f8692b07700 1 -- 172.21.15.121:0/1313973570 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868c1539c0 con 0x7f868c14ac10 2023-11-15T03:11:05.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.980+0000 7f8692b07700 1 -- 172.21.15.121:0/1313973570 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f868c06bcf0 con 0x7f868c14ac10 2023-11-15T03:11:05.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.984+0000 7f8681ffb700 1 -- 172.21.15.121:0/1313973570 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f867c010d30 con 0x7f868c14ac10 2023-11-15T03:11:05.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.984+0000 7f8681ffb700 1 --2- 172.21.15.121:0/1313973570 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f867005a510 0x7f867005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:05.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.984+0000 7f8681ffb700 1 -- 172.21.15.121:0/1313973570 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9706+0+0 (secure 0 0 0) 0x7f867c075130 con 0x7f868c14ac10 2023-11-15T03:11:05.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.984+0000 7f8691b05700 1 --2- 172.21.15.121:0/1313973570 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f867005a510 0x7f867005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:05.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.984+0000 7f8691b05700 1 --2- 172.21.15.121:0/1313973570 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f867005a510 0x7f867005c9d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7f868800bfd0 tx=0x7f868800b3b0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:05.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:05.988+0000 7f8681ffb700 1 -- 172.21.15.121:0/1313973570 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f867c047a50 con 0x7f868c14ac10 2023-11-15T03:11:06.156 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.148+0000 7f8692b07700 1 -- 172.21.15.121:0/1313973570 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f868c065d20 con 0x7f868c14ac10 2023-11-15T03:11:06.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.148+0000 7f8681ffb700 1 -- 172.21.15.121:0/1313973570 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v134) v1 ==== 110+0+10 (secure 0 0 0) 0x7f867c04b070 con 0x7f868c14ac10 2023-11-15T03:11:06.158 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T03:11:06.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.152+0000 7f8692b07700 1 -- 172.21.15.121:0/1313973570 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f867005a510 msgr2=0x7f867005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:06.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.152+0000 7f8692b07700 1 --2- 172.21.15.121:0/1313973570 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f867005a510 0x7f867005c9d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7f868800bfd0 tx=0x7f868800b3b0).stop 2023-11-15T03:11:06.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.152+0000 7f8692b07700 1 -- 172.21.15.121:0/1313973570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f868c14ac10 msgr2=0x7f868c147ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:06.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.152+0000 7f8692b07700 1 --2- 172.21.15.121:0/1313973570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f868c14ac10 0x7f868c147ec0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 rx=0x7f867c00c670 tx=0x7f867c008040).stop 2023-11-15T03:11:06.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.152+0000 7f8692b07700 1 -- 172.21.15.121:0/1313973570 shutdown_connections 2023-11-15T03:11:06.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.152+0000 7f8692b07700 1 --2- 172.21.15.121:0/1313973570 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f867005a510 0x7f867005c9d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.152+0000 7f8692b07700 1 --2- 172.21.15.121:0/1313973570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f868c14aff0 0x7f868c1483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.152+0000 7f8692b07700 1 --2- 172.21.15.121:0/1313973570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f868c14ac10 0x7f868c147ec0 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.152+0000 7f8692b07700 1 --2- 172.21.15.121:0/1313973570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f868c067160 0x7f868c142fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.152+0000 7f8692b07700 1 -- 172.21.15.121:0/1313973570 >> 172.21.15.121:0/1313973570 conn(0x7f868c067a60 msgr2=0x7f868c0c0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:06.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.152+0000 7f8692b07700 1 -- 172.21.15.121:0/1313973570 shutdown_connections 2023-11-15T03:11:06.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.152+0000 7f8692b07700 1 -- 172.21.15.121:0/1313973570 wait complete. 2023-11-15T03:11:06.182 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T03:11:06.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.336+0000 7fe2094e4700 1 Processor -- start 2023-11-15T03:11:06.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.336+0000 7fe2094e4700 1 -- start start 2023-11-15T03:11:06.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.340+0000 7fe2094e4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe20414ac10 0x7fe2040c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:06.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.340+0000 7fe2094e4700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe20414aff0 0x7fe204066c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:06.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.340+0000 7fe2094e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe204067160 0x7fe204154370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:06.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.340+0000 7fe2094e4700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2040657f0 con 0x7fe20414aff0 2023-11-15T03:11:06.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.340+0000 7fe2094e4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe204066650 con 0x7fe20414ac10 2023-11-15T03:11:06.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.340+0000 7fe2094e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2040675b0 con 0x7fe204067160 2023-11-15T03:11:06.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.340+0000 7fe2037fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe20414aff0 0x7fe204066c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:06.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.340+0000 7fe2037fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe20414aff0 0x7fe204066c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39924/0 (socket says 172.21.15.121:39924) 2023-11-15T03:11:06.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.340+0000 7fe2037fe700 1 -- 172.21.15.121:0/4221079853 learned_addr learned my addr 172.21.15.121:0/4221079853 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:06.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.340+0000 7fe203fff700 1 --2- 172.21.15.121:0/4221079853 >> v2:172.21.15.190:3300/0 conn(0x7fe20414ac10 0x7fe2040c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:06.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.340+0000 7fe2037fe700 1 -- 172.21.15.121:0/4221079853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe204067160 msgr2=0x7fe204154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:06.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.340+0000 7fe2037fe700 1 --2- 172.21.15.121:0/4221079853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe204067160 0x7fe204154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:11:06.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.340+0000 7fe2037fe700 1 -- 172.21.15.121:0/4221079853 >> v2:172.21.15.190:3300/0 conn(0x7fe20414ac10 msgr2=0x7fe2040c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:06.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.340+0000 7fe2037fe700 1 --2- 172.21.15.121:0/4221079853 >> v2:172.21.15.190:3300/0 conn(0x7fe20414ac10 0x7fe2040c6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.340+0000 7fe2037fe700 1 -- 172.21.15.121:0/4221079853 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe204065a40 con 0x7fe20414aff0 2023-11-15T03:11:06.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.340+0000 7fe203fff700 1 --2- 172.21.15.121:0/4221079853 >> v2:172.21.15.190:3300/0 conn(0x7fe20414ac10 0x7fe2040c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:11:06.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.340+0000 7fe2037fe700 1 --2- 172.21.15.121:0/4221079853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe20414aff0 0x7fe204066c70 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 rx=0x7fe204064d60 tx=0x7fe1f800b3b0).ready entity=mon.0 client_cookie=4204b428fd8e592b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:06.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.344+0000 7fe202ffd700 1 -- 172.21.15.121:0/4221079853 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f800db10 con 0x7fe20414aff0 2023-11-15T03:11:06.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.344+0000 7fe202ffd700 1 -- 172.21.15.121:0/4221079853 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f800dc70 con 0x7fe20414aff0 2023-11-15T03:11:06.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.344+0000 7fe202ffd700 1 -- 172.21.15.121:0/4221079853 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8010470 con 0x7fe20414aff0 2023-11-15T03:11:06.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.344+0000 7fe2094e4700 1 -- 172.21.15.121:0/4221079853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe20414aff0 msgr2=0x7fe204066c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:06.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.344+0000 7fe2094e4700 1 --2- 172.21.15.121:0/4221079853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe20414aff0 0x7fe204066c70 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 rx=0x7fe204064d60 tx=0x7fe1f800b3b0).stop 2023-11-15T03:11:06.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.344+0000 7fe2094e4700 1 -- 172.21.15.121:0/4221079853 shutdown_connections 2023-11-15T03:11:06.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.344+0000 7fe2094e4700 1 --2- 172.21.15.121:0/4221079853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe204067160 0x7fe204154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-15T03:11:06.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.344+0000 7fe2094e4700 1 --2- 172.21.15.121:0/4221079853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe20414aff0 0x7fe204066c70 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.344+0000 7fe2094e4700 1 --2- 172.21.15.121:0/4221079853 >> v2:172.21.15.190:3300/0 conn(0x7fe20414ac10 0x7fe2040c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.344+0000 7fe2094e4700 1 -- 172.21.15.121:0/4221079853 >> 172.21.15.121:0/4221079853 conn(0x7fe204067a60 msgr2=0x7fe204067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:06.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.344+0000 7fe2094e4700 1 -- 172.21.15.121:0/4221079853 shutdown_connections 2023-11-15T03:11:06.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.344+0000 7fe2094e4700 1 -- 172.21.15.121:0/4221079853 wait complete. 2023-11-15T03:11:06.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.344+0000 7fe2094e4700 1 Processor -- start 2023-11-15T03:11:06.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.344+0000 7fe2094e4700 1 -- start start 2023-11-15T03:11:06.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe2094e4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe204067160 0x7fe204147590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:06.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe2094e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe20414ac10 0x7fe20413ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:06.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe2094e4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe20414aff0 0x7fe2040c1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:06.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe2094e4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2041415e0 con 0x7fe204067160 2023-11-15T03:11:06.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe2094e4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe204147a80 con 0x7fe20414aff0 2023-11-15T03:11:06.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe2094e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2040aba30 con 0x7fe20414ac10 2023-11-15T03:11:06.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe203fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe204067160 0x7fe204147590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:06.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe203fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe204067160 0x7fe204147590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39930/0 (socket says 172.21.15.121:39930) 2023-11-15T03:11:06.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe203fff700 1 -- 172.21.15.121:0/2726798776 learned_addr learned my addr 172.21.15.121:0/2726798776 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:06.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe2037fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe20414ac10 0x7fe20413ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:06.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe208ce3700 1 --2- 172.21.15.121:0/2726798776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe20414aff0 0x7fe2040c1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:06.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe203fff700 1 -- 172.21.15.121:0/2726798776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe20414ac10 msgr2=0x7fe20413ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:06.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe203fff700 1 --2- 172.21.15.121:0/2726798776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe20414ac10 0x7fe20413ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe203fff700 1 -- 172.21.15.121:0/2726798776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe20414aff0 msgr2=0x7fe2040c1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:06.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe203fff700 1 --2- 172.21.15.121:0/2726798776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe20414aff0 0x7fe2040c1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe203fff700 1 -- 172.21.15.121:0/2726798776 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f800b610 con 0x7fe204067160 2023-11-15T03:11:06.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe208ce3700 1 --2- 172.21.15.121:0/2726798776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe20414aff0 0x7fe2040c1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:06.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe203fff700 1 --2- 172.21.15.121:0/2726798776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe204067160 0x7fe204147590 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 rx=0x7fe1f400c880 tx=0x7fe1f4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:06.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe1ebfff700 1 -- 172.21.15.121:0/2726798776 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4010580 con 0x7fe204067160 2023-11-15T03:11:06.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe1ebfff700 1 -- 172.21.15.121:0/2726798776 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f40106e0 con 0x7fe204067160 2023-11-15T03:11:06.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe2094e4700 1 -- 172.21.15.121:0/2726798776 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2041409d0 con 0x7fe204067160 2023-11-15T03:11:06.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe1ebfff700 1 -- 172.21.15.121:0/2726798776 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4010a60 con 0x7fe204067160 2023-11-15T03:11:06.364 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.348+0000 7fe2094e4700 1 -- 172.21.15.121:0/2726798776 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe204140f90 con 0x7fe204067160 2023-11-15T03:11:06.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.352+0000 7fe2094e4700 1 -- 172.21.15.121:0/2726798776 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe20406bcf0 con 0x7fe204067160 2023-11-15T03:11:06.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.352+0000 7fe1ebfff700 1 -- 172.21.15.121:0/2726798776 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe1f401b460 con 0x7fe204067160 2023-11-15T03:11:06.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.352+0000 7fe1ebfff700 1 --2- 172.21.15.121:0/2726798776 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe1f005a540 0x7fe1f005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:06.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.352+0000 7fe1ebfff700 1 -- 172.21.15.121:0/2726798776 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9706+0+0 (secure 0 0 0) 0x7fe1f40755b0 con 0x7fe204067160 2023-11-15T03:11:06.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.352+0000 7fe2037fe700 1 --2- 172.21.15.121:0/2726798776 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe1f005a540 0x7fe1f005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:06.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.352+0000 7fe2037fe700 1 --2- 172.21.15.121:0/2726798776 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe1f005a540 0x7fe1f005ca00 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7fe20414b480 tx=0x7fe1f800b3b0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:06.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.356+0000 7fe1ebfff700 1 -- 172.21.15.121:0/2726798776 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe1f4047ef0 con 0x7fe204067160 2023-11-15T03:11:06.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.520+0000 7fe2094e4700 1 -- 172.21.15.121:0/2726798776 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe204066cf0 con 0x7fe204067160 2023-11-15T03:11:06.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.520+0000 7fe1ebfff700 1 -- 172.21.15.121:0/2726798776 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v134) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe1f404b510 con 0x7fe204067160 2023-11-15T03:11:06.530 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T03:11:06.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.524+0000 7fe2094e4700 1 -- 172.21.15.121:0/2726798776 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe1f005a540 msgr2=0x7fe1f005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:06.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.524+0000 7fe2094e4700 1 --2- 172.21.15.121:0/2726798776 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe1f005a540 0x7fe1f005ca00 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7fe20414b480 tx=0x7fe1f800b3b0).stop 2023-11-15T03:11:06.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.524+0000 7fe2094e4700 1 -- 172.21.15.121:0/2726798776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe204067160 msgr2=0x7fe204147590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:06.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.524+0000 7fe2094e4700 1 --2- 172.21.15.121:0/2726798776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe204067160 0x7fe204147590 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 rx=0x7fe1f400c880 tx=0x7fe1f4008040).stop 2023-11-15T03:11:06.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.524+0000 7fe2094e4700 1 -- 172.21.15.121:0/2726798776 shutdown_connections 2023-11-15T03:11:06.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.524+0000 7fe2094e4700 1 --2- 172.21.15.121:0/2726798776 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe1f005a540 0x7fe1f005ca00 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.524+0000 7fe2094e4700 1 --2- 172.21.15.121:0/2726798776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe20414aff0 0x7fe2040c1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.524+0000 7fe2094e4700 1 --2- 172.21.15.121:0/2726798776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe20414ac10 0x7fe20413ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.524+0000 7fe2094e4700 1 --2- 172.21.15.121:0/2726798776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe204067160 0x7fe204147590 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.524+0000 7fe2094e4700 1 -- 172.21.15.121:0/2726798776 >> 172.21.15.121:0/2726798776 conn(0x7fe204067a60 msgr2=0x7fe2040c5780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:06.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.524+0000 7fe2094e4700 1 -- 172.21.15.121:0/2726798776 shutdown_connections 2023-11-15T03:11:06.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.524+0000 7fe2094e4700 1 -- 172.21.15.121:0/2726798776 wait complete. 2023-11-15T03:11:06.550 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-15T03:11:06.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.704+0000 7f6c12979700 1 Processor -- start 2023-11-15T03:11:06.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.704+0000 7f6c12979700 1 -- start start 2023-11-15T03:11:06.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.708+0000 7f6c12979700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6c0c0acd00 0x7f6c0c0ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:06.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.708+0000 7f6c12979700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6c0c066330 0x7f6c0c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:06.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.708+0000 7f6c12979700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c0c065ea0 con 0x7f6c0c0c5c70 2023-11-15T03:11:06.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.708+0000 7f6c12979700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c0c066bb0 con 0x7f6c0c066330 2023-11-15T03:11:06.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.708+0000 7f6c12979700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c0c065890 con 0x7f6c0c0acd00 2023-11-15T03:11:06.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.708+0000 7f6c12178700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6c0c066330 0x7f6c0c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:06.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.708+0000 7f6c12178700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6c0c066330 0x7f6c0c153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39934/0 (socket says 172.21.15.121:39934) 2023-11-15T03:11:06.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.708+0000 7f6c11176700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6c0c0c5c70 0x7f6c0c1536c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:48996/0 (socket says 172.21.15.121:48996) 2023-11-15T03:11:06.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.708+0000 7f6c12178700 1 -- 172.21.15.121:0/2187798366 learned_addr learned my addr 172.21.15.121:0/2187798366 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:06.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.708+0000 7f6c11977700 1 --2- 172.21.15.121:0/2187798366 >> v2:172.21.15.190:3300/0 conn(0x7f6c0c0acd00 0x7f6c0c0ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:06.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.708+0000 7f6c12178700 1 -- 172.21.15.121:0/2187798366 >> v2:172.21.15.190:3300/0 conn(0x7f6c0c0acd00 msgr2=0x7f6c0c0ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:06.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.708+0000 7f6c12178700 1 --2- 172.21.15.121:0/2187798366 >> v2:172.21.15.190:3300/0 conn(0x7f6c0c0acd00 0x7f6c0c0ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.708+0000 7f6c12178700 1 -- 172.21.15.121:0/2187798366 >> v1:172.21.15.121:6789/0 conn(0x7f6c0c0c5c70 legacy=0x7f6c0c1536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:06.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.708+0000 7f6c10975700 1 -- 172.21.15.121:0/2187798366 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112458259 0 0) 0x7f6c0c065ea0 con 0x7f6c0c0c5c70 2023-11-15T03:11:06.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.708+0000 7f6c12178700 1 -- 172.21.15.121:0/2187798366 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c0c64a0 con 0x7f6c0c066330 2023-11-15T03:11:06.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.708+0000 7f6c12178700 1 --2- 172.21.15.121:0/2187798366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c0c066330 0x7f6c0c153dd0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 rx=0x7f6c0c06c280 tx=0x7f6c04009840).ready entity=mon.0 client_cookie=5d64ee52a522c364 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:06.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.708+0000 7f6c10975700 1 -- 172.21.15.121:0/2187798366 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0400e070 con 0x7f6c0c066330 2023-11-15T03:11:06.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.708+0000 7f6c10975700 1 -- 172.21.15.121:0/2187798366 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0400ba20 con 0x7f6c0c066330 2023-11-15T03:11:06.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.708+0000 7f6c10975700 1 -- 172.21.15.121:0/2187798366 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0400bda0 con 0x7f6c0c066330 2023-11-15T03:11:06.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.712+0000 7f6c12979700 1 -- 172.21.15.121:0/2187798366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c0c066330 msgr2=0x7f6c0c153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:06.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.712+0000 7f6c12979700 1 --2- 172.21.15.121:0/2187798366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c0c066330 0x7f6c0c153dd0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 rx=0x7f6c0c06c280 tx=0x7f6c04009840).stop 2023-11-15T03:11:06.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.712+0000 7f6c12979700 1 -- 172.21.15.121:0/2187798366 shutdown_connections 2023-11-15T03:11:06.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.712+0000 7f6c12979700 1 --2- 172.21.15.121:0/2187798366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c0c066330 0x7f6c0c153dd0 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.712+0000 7f6c12979700 1 --2- 172.21.15.121:0/2187798366 >> v2:172.21.15.190:3300/0 conn(0x7f6c0c0acd00 0x7f6c0c0ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.712+0000 7f6c12979700 1 -- 172.21.15.121:0/2187798366 >> 172.21.15.121:0/2187798366 conn(0x7f6c0c063fc0 msgr2=0x7f6c0c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:06.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.712+0000 7f6c12979700 1 -- 172.21.15.121:0/2187798366 shutdown_connections 2023-11-15T03:11:06.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.712+0000 7f6c12979700 1 -- 172.21.15.121:0/2187798366 wait complete. 2023-11-15T03:11:06.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.712+0000 7f6c12979700 1 Processor -- start 2023-11-15T03:11:06.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.712+0000 7f6c12979700 1 -- start start 2023-11-15T03:11:06.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c12979700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c0c066330 0x7f6c0c155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:06.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c12979700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c0c0acd00 0x7f6c0c155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:06.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c12979700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c0c0c5c70 0x7f6c0c0c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:06.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c12979700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c1561f0 con 0x7f6c0c0c5c70 2023-11-15T03:11:06.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c12979700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c1564d0 con 0x7f6c0c066330 2023-11-15T03:11:06.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c12979700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c1567b0 con 0x7f6c0c0acd00 2023-11-15T03:11:06.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c12178700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c0c0c5c70 0x7f6c0c0c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:06.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c12178700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c0c0c5c70 0x7f6c0c0c2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39940/0 (socket says 172.21.15.121:39940) 2023-11-15T03:11:06.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c12178700 1 -- 172.21.15.121:0/1418891232 learned_addr learned my addr 172.21.15.121:0/1418891232 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:06.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c11176700 1 --2- 172.21.15.121:0/1418891232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c0c0acd00 0x7f6c0c155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:06.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c11977700 1 --2- 172.21.15.121:0/1418891232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c0c066330 0x7f6c0c155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:06.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c12178700 1 -- 172.21.15.121:0/1418891232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c0c0acd00 msgr2=0x7f6c0c155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:06.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c12178700 1 --2- 172.21.15.121:0/1418891232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c0c0acd00 0x7f6c0c155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c12178700 1 -- 172.21.15.121:0/1418891232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c0c066330 msgr2=0x7f6c0c155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:06.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c12178700 1 --2- 172.21.15.121:0/1418891232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c0c066330 0x7f6c0c155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c12178700 1 -- 172.21.15.121:0/1418891232 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c04009ae0 con 0x7f6c0c0c5c70 2023-11-15T03:11:06.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c11977700 1 --2- 172.21.15.121:0/1418891232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c0c066330 0x7f6c0c155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:06.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c12178700 1 --2- 172.21.15.121:0/1418891232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c0c0c5c70 0x7f6c0c0c2e50 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 rx=0x7f6c0400b6c0 tx=0x7f6c04009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:06.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c01ffb700 1 -- 172.21.15.121:0/1418891232 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0400b6f0 con 0x7f6c0c0c5c70 2023-11-15T03:11:06.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c01ffb700 1 -- 172.21.15.121:0/1418891232 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c040092e0 con 0x7f6c0c0c5c70 2023-11-15T03:11:06.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c01ffb700 1 -- 172.21.15.121:0/1418891232 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0401e6a0 con 0x7f6c0c0c5c70 2023-11-15T03:11:06.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c12979700 1 -- 172.21.15.121:0/1418891232 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0c156a90 con 0x7f6c0c0c5c70 2023-11-15T03:11:06.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.716+0000 7f6c12979700 1 -- 172.21.15.121:0/1418891232 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0c1521c0 con 0x7f6c0c0c5c70 2023-11-15T03:11:06.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.720+0000 7f6c12979700 1 -- 172.21.15.121:0/1418891232 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0c0c5490 con 0x7f6c0c0c5c70 2023-11-15T03:11:06.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.720+0000 7f6c01ffb700 1 -- 172.21.15.121:0/1418891232 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6c04003040 con 0x7f6c0c0c5c70 2023-11-15T03:11:06.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.720+0000 7f6c01ffb700 1 --2- 172.21.15.121:0/1418891232 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6bf005a510 0x7f6bf005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:06.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.720+0000 7f6c01ffb700 1 -- 172.21.15.121:0/1418891232 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9706+0+0 (secure 0 0 0) 0x7f6c04076b70 con 0x7f6c0c0c5c70 2023-11-15T03:11:06.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.720+0000 7f6c11977700 1 --2- 172.21.15.121:0/1418891232 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6bf005a510 0x7f6bf005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:06.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.724+0000 7f6c11977700 1 --2- 172.21.15.121:0/1418891232 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6bf005a510 0x7f6bf005c9d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7f6c08009200 tx=0x7f6c08008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:06.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.728+0000 7f6c01ffb700 1 -- 172.21.15.121:0/1418891232 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6c0c0c5490 con 0x7f6c0c0c5c70 2023-11-15T03:11:06.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.888+0000 7f6c12979700 1 -- 172.21.15.121:0/1418891232 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c0c0c5490 con 0x7f6c0c0c5c70 2023-11-15T03:11:06.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.892+0000 7f6c01ffb700 1 -- 172.21.15.121:0/1418891232 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v134) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6c0c0c5490 con 0x7f6c0c0c5c70 2023-11-15T03:11:06.899 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T03:11:06.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.892+0000 7f6c12979700 1 -- 172.21.15.121:0/1418891232 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6bf005a510 msgr2=0x7f6bf005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:06.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.892+0000 7f6c12979700 1 --2- 172.21.15.121:0/1418891232 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6bf005a510 0x7f6bf005c9d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7f6c08009200 tx=0x7f6c08008040).stop 2023-11-15T03:11:06.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.892+0000 7f6c12979700 1 -- 172.21.15.121:0/1418891232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c0c0c5c70 msgr2=0x7f6c0c0c2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:06.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.892+0000 7f6c12979700 1 --2- 172.21.15.121:0/1418891232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c0c0c5c70 0x7f6c0c0c2e50 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 rx=0x7f6c0400b6c0 tx=0x7f6c04009840).stop 2023-11-15T03:11:06.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.892+0000 7f6c12979700 1 -- 172.21.15.121:0/1418891232 shutdown_connections 2023-11-15T03:11:06.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.892+0000 7f6c12979700 1 --2- 172.21.15.121:0/1418891232 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6bf005a510 0x7f6bf005c9d0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.892+0000 7f6c12979700 1 --2- 172.21.15.121:0/1418891232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c0c0c5c70 0x7f6c0c0c2e50 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.892+0000 7f6c12979700 1 --2- 172.21.15.121:0/1418891232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c0c0acd00 0x7f6c0c155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.892+0000 7f6c12979700 1 --2- 172.21.15.121:0/1418891232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c0c066330 0x7f6c0c155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:06.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.892+0000 7f6c12979700 1 -- 172.21.15.121:0/1418891232 >> 172.21.15.121:0/1418891232 conn(0x7f6c0c063fc0 msgr2=0x7f6c0c0b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:06.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.892+0000 7f6c12979700 1 -- 172.21.15.121:0/1418891232 shutdown_connections 2023-11-15T03:11:06.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:06.892+0000 7f6c12979700 1 -- 172.21.15.121:0/1418891232 wait complete. 2023-11-15T03:11:06.922 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-15T03:11:07.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.072+0000 7fb1a3439700 1 Processor -- start 2023-11-15T03:11:07.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.072+0000 7fb1a3439700 1 -- start start 2023-11-15T03:11:07.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.072+0000 7fb1a3439700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb19c151af0 0x7fb19c0a7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.072+0000 7fb1a3439700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb19c0a5720 0x7fb19c0c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.072+0000 7fb1a3439700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb19c0c3430 0x7fb19c0c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.072+0000 7fb1a3439700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb19c1528b0 con 0x7fb19c0a5720 2023-11-15T03:11:07.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.072+0000 7fb1a3439700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb19c0c38c0 con 0x7fb19c151af0 2023-11-15T03:11:07.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.072+0000 7fb1a3439700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c151ff0 con 0x7fb19c0c3430 2023-11-15T03:11:07.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.076+0000 7fb1a1c36700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb19c0a5720 0x7fb19c0c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:07.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.076+0000 7fb1a2c38700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb19c0c3430 0x7fb19c0c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:07.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.076+0000 7fb1a2c38700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb19c0c3430 0x7fb19c0c64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:58964/0 (socket says 172.21.15.121:58964) 2023-11-15T03:11:07.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.076+0000 7fb1a1c36700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb19c0a5720 0x7fb19c0c2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39952/0 (socket says 172.21.15.121:39952) 2023-11-15T03:11:07.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.076+0000 7fb1a2c38700 1 -- 172.21.15.121:0/1311716541 learned_addr learned my addr 172.21.15.121:0/1311716541 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:07.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.076+0000 7fb1a2437700 1 --2- 172.21.15.121:0/1311716541 >> v2:172.21.15.190:3300/0 conn(0x7fb19c151af0 0x7fb19c0a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:07.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.076+0000 7fb1a2c38700 1 -- 172.21.15.121:0/1311716541 >> v2:172.21.15.190:3300/0 conn(0x7fb19c151af0 msgr2=0x7fb19c0a7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:07.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.076+0000 7fb1a2c38700 1 --2- 172.21.15.121:0/1311716541 >> v2:172.21.15.190:3300/0 conn(0x7fb19c151af0 0x7fb19c0a7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.076+0000 7fb1a2c38700 1 -- 172.21.15.121:0/1311716541 >> v2:172.21.15.121:3300/0 conn(0x7fb19c0a5720 msgr2=0x7fb19c0c2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:07.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.076+0000 7fb1a2c38700 1 --2- 172.21.15.121:0/1311716541 >> v2:172.21.15.121:3300/0 conn(0x7fb19c0a5720 0x7fb19c0c2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.076+0000 7fb1a2c38700 1 -- 172.21.15.121:0/1311716541 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c152260 con 0x7fb19c0c3430 2023-11-15T03:11:07.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.076+0000 7fb1a1c36700 1 --2- 172.21.15.121:0/1311716541 >> v2:172.21.15.121:3300/0 conn(0x7fb19c0a5720 0x7fb19c0c2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:07.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.076+0000 7fb1a2c38700 1 --2- 172.21.15.121:0/1311716541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb19c0c3430 0x7fb19c0c64a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fb19c063990 tx=0x7fb194009840).ready entity=mon.2 client_cookie=dd21bb582ea1e466 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:07.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.076+0000 7fb1a1435700 1 -- 172.21.15.121:0/1311716541 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19400e070 con 0x7fb19c0c3430 2023-11-15T03:11:07.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.076+0000 7fb1a1435700 1 -- 172.21.15.121:0/1311716541 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19400b970 con 0x7fb19c0c3430 2023-11-15T03:11:07.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.076+0000 7fb1a1435700 1 -- 172.21.15.121:0/1311716541 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19400bcf0 con 0x7fb19c0c3430 2023-11-15T03:11:07.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.076+0000 7fb1a3439700 1 -- 172.21.15.121:0/1311716541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb19c0c3430 msgr2=0x7fb19c0c64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:07.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.076+0000 7fb1a3439700 1 --2- 172.21.15.121:0/1311716541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb19c0c3430 0x7fb19c0c64a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fb19c063990 tx=0x7fb194009840).stop 2023-11-15T03:11:07.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.080+0000 7fb1a3439700 1 -- 172.21.15.121:0/1311716541 shutdown_connections 2023-11-15T03:11:07.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.080+0000 7fb1a3439700 1 --2- 172.21.15.121:0/1311716541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb19c0c3430 0x7fb19c0c64a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.080+0000 7fb1a3439700 1 --2- 172.21.15.121:0/1311716541 >> v2:172.21.15.121:3300/0 conn(0x7fb19c0a5720 0x7fb19c0c2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.080+0000 7fb1a3439700 1 --2- 172.21.15.121:0/1311716541 >> v2:172.21.15.190:3300/0 conn(0x7fb19c151af0 0x7fb19c0a7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.080+0000 7fb1a3439700 1 -- 172.21.15.121:0/1311716541 >> 172.21.15.121:0/1311716541 conn(0x7fb19c067a60 msgr2=0x7fb19c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:07.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.080+0000 7fb1a3439700 1 -- 172.21.15.121:0/1311716541 shutdown_connections 2023-11-15T03:11:07.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.080+0000 7fb1a3439700 1 -- 172.21.15.121:0/1311716541 wait complete. 2023-11-15T03:11:07.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.080+0000 7fb1a3439700 1 Processor -- start 2023-11-15T03:11:07.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.080+0000 7fb1a3439700 1 -- start start 2023-11-15T03:11:07.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1a3439700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb19c0a5720 0x7fb19c0bf760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1a3439700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb19c0c3430 0x7fb19c0c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1a3439700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb19c0c6ee0 0x7fb19c0c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1a3439700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c0c0210 con 0x7fb19c0c6ee0 2023-11-15T03:11:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1a3439700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c0c7360 con 0x7fb19c0c3430 2023-11-15T03:11:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1a3439700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c0be6c0 con 0x7fb19c0a5720 2023-11-15T03:11:07.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1a2c38700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb19c0c6ee0 0x7fb19c0c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:07.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1a2c38700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb19c0c6ee0 0x7fb19c0c4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39962/0 (socket says 172.21.15.121:39962) 2023-11-15T03:11:07.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1a2c38700 1 -- 172.21.15.121:0/1697396022 learned_addr learned my addr 172.21.15.121:0/1697396022 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:07.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1a1c36700 1 --2- 172.21.15.121:0/1697396022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb19c0c3430 0x7fb19c0c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:07.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1a2c38700 1 -- 172.21.15.121:0/1697396022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb19c0a5720 msgr2=0x7fb19c0bf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:07.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1a2c38700 1 --2- 172.21.15.121:0/1697396022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb19c0a5720 0x7fb19c0bf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:11:07.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1a2c38700 1 -- 172.21.15.121:0/1697396022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb19c0c3430 msgr2=0x7fb19c0c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:07.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1a2c38700 1 --2- 172.21.15.121:0/1697396022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb19c0c3430 0x7fb19c0c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1a2c38700 1 -- 172.21.15.121:0/1697396022 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c0092a0 con 0x7fb19c0c6ee0 2023-11-15T03:11:07.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1a1c36700 1 --2- 172.21.15.121:0/1697396022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb19c0c3430 0x7fb19c0c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:11:07.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1a2c38700 1 --2- 172.21.15.121:0/1697396022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb19c0c6ee0 0x7fb19c0c4310 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 rx=0x7fb19400b610 tx=0x7fb194009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:07.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1927fc700 1 -- 172.21.15.121:0/1697396022 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194013e60 con 0x7fb19c0c6ee0 2023-11-15T03:11:07.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1927fc700 1 -- 172.21.15.121:0/1697396022 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1940092e0 con 0x7fb19c0c6ee0 2023-11-15T03:11:07.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1a3439700 1 -- 172.21.15.121:0/1697396022 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb194009ae0 con 0x7fb19c0c6ee0 2023-11-15T03:11:07.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1927fc700 1 -- 172.21.15.121:0/1697396022 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19401e6c0 con 0x7fb19c0c6ee0 2023-11-15T03:11:07.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.084+0000 7fb1a3439700 1 -- 172.21.15.121:0/1697396022 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19c0bec80 con 0x7fb19c0c6ee0 2023-11-15T03:11:07.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.088+0000 7fb1a3439700 1 -- 172.21.15.121:0/1697396022 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19c06bcf0 con 0x7fb19c0c6ee0 2023-11-15T03:11:07.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.088+0000 7fb1927fc700 1 -- 172.21.15.121:0/1697396022 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb194003040 con 0x7fb19c0c6ee0 2023-11-15T03:11:07.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.088+0000 7fb1927fc700 1 --2- 172.21.15.121:0/1697396022 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb18005a510 0x7fb18005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.088+0000 7fb1927fc700 1 -- 172.21.15.121:0/1697396022 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9706+0+0 (secure 0 0 0) 0x7fb194076bf0 con 0x7fb19c0c6ee0 2023-11-15T03:11:07.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.088+0000 7fb1a2437700 1 --2- 172.21.15.121:0/1697396022 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb18005a510 0x7fb18005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:07.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.092+0000 7fb1a2437700 1 --2- 172.21.15.121:0/1697396022 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb18005a510 0x7fb18005c9d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7fb198009990 tx=0x7fb198009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:07.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.096+0000 7fb1927fc700 1 -- 172.21.15.121:0/1697396022 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb194049510 con 0x7fb19c0c6ee0 2023-11-15T03:11:07.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.256+0000 7fb1a3439700 1 -- 172.21.15.121:0/1697396022 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb19c0a84e0 con 0x7fb19c0c6ee0 2023-11-15T03:11:07.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.256+0000 7fb1927fc700 1 -- 172.21.15.121:0/1697396022 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v134) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb194023020 con 0x7fb19c0c6ee0 2023-11-15T03:11:07.266 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T03:11:07.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.260+0000 7fb1a3439700 1 -- 172.21.15.121:0/1697396022 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb18005a510 msgr2=0x7fb18005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:07.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.260+0000 7fb1a3439700 1 --2- 172.21.15.121:0/1697396022 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb18005a510 0x7fb18005c9d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7fb198009990 tx=0x7fb198009060).stop 2023-11-15T03:11:07.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.260+0000 7fb1a3439700 1 -- 172.21.15.121:0/1697396022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb19c0c6ee0 msgr2=0x7fb19c0c4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:07.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.260+0000 7fb1a3439700 1 --2- 172.21.15.121:0/1697396022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb19c0c6ee0 0x7fb19c0c4310 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 rx=0x7fb19400b610 tx=0x7fb194009840).stop 2023-11-15T03:11:07.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.260+0000 7fb1a3439700 1 -- 172.21.15.121:0/1697396022 shutdown_connections 2023-11-15T03:11:07.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.260+0000 7fb1a3439700 1 --2- 172.21.15.121:0/1697396022 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb18005a510 0x7fb18005c9d0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.260+0000 7fb1a3439700 1 --2- 172.21.15.121:0/1697396022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb19c0c6ee0 0x7fb19c0c4310 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.260+0000 7fb1a3439700 1 --2- 172.21.15.121:0/1697396022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb19c0c3430 0x7fb19c0c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.260+0000 7fb1a3439700 1 --2- 172.21.15.121:0/1697396022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb19c0a5720 0x7fb19c0bf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:11:07.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.260+0000 7fb1a3439700 1 -- 172.21.15.121:0/1697396022 >> 172.21.15.121:0/1697396022 conn(0x7fb19c067a60 msgr2=0x7fb19c14fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:07.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.260+0000 7fb1a3439700 1 -- 172.21.15.121:0/1697396022 shutdown_connections 2023-11-15T03:11:07.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.260+0000 7fb1a3439700 1 -- 172.21.15.121:0/1697396022 wait complete. 2023-11-15T03:11:07.293 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:07.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.460+0000 7f702531b700 1 Processor -- start 2023-11-15T03:11:07.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.460+0000 7f702531b700 1 -- start start 2023-11-15T03:11:07.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.460+0000 7f702531b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7020154ca0 0x7f7020155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.460+0000 7f702531b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f70200a57c0 0x7f70200a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.460+0000 7f702531b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70200a6160 con 0x7f70201562f0 2023-11-15T03:11:07.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.460+0000 7f702531b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70200a8280 con 0x7f70200a57c0 2023-11-15T03:11:07.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.460+0000 7f702531b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70200a6d40 con 0x7f7020154ca0 2023-11-15T03:11:07.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.464+0000 7f701f7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f70200a57c0 0x7f70200a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:07.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.464+0000 7f701effd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7020154ca0 0x7f7020155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:07.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.464+0000 7f701f7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f70200a57c0 0x7f70200a5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:41396/0 (socket says 172.21.15.121:41396) 2023-11-15T03:11:07.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.464+0000 7f701f7fe700 1 -- 172.21.15.121:0/1444302236 learned_addr learned my addr 172.21.15.121:0/1444302236 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:07.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.464+0000 7f701f7fe700 1 -- 172.21.15.121:0/1444302236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7020154ca0 msgr2=0x7f7020155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:07.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.464+0000 7f701f7fe700 1 --2- 172.21.15.121:0/1444302236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7020154ca0 0x7f7020155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.464+0000 7f701f7fe700 1 -- 172.21.15.121:0/1444302236 >> v1:172.21.15.190:6789/0 conn(0x7f70201562f0 legacy=0x7f70200a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:07.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.464+0000 7f701f7fe700 1 -- 172.21.15.121:0/1444302236 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70200c4c30 con 0x7f70200a57c0 2023-11-15T03:11:07.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.464+0000 7f701f7fe700 1 --2- 172.21.15.121:0/1444302236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70200a57c0 0x7f70200a5c70 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f7018009f20 tx=0x7f7018009840).ready entity=mon.1 client_cookie=6fc98151ed6e129d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:07.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.464+0000 7f701dffb700 1 -- 172.21.15.121:0/1444302236 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701800e070 con 0x7f70200a57c0 2023-11-15T03:11:07.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.464+0000 7f701dffb700 1 -- 172.21.15.121:0/1444302236 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701800b9d0 con 0x7f70200a57c0 2023-11-15T03:11:07.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.464+0000 7f701dffb700 1 -- 172.21.15.121:0/1444302236 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701800bd50 con 0x7f70200a57c0 2023-11-15T03:11:07.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.464+0000 7f702531b700 1 -- 172.21.15.121:0/1444302236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70200a57c0 msgr2=0x7f70200a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:07.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.464+0000 7f702531b700 1 --2- 172.21.15.121:0/1444302236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70200a57c0 0x7f70200a5c70 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f7018009f20 tx=0x7f7018009840).stop 2023-11-15T03:11:07.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.464+0000 7f702531b700 1 -- 172.21.15.121:0/1444302236 shutdown_connections 2023-11-15T03:11:07.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.464+0000 7f702531b700 1 --2- 172.21.15.121:0/1444302236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70200a57c0 0x7f70200a5c70 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.464+0000 7f702531b700 1 --2- 172.21.15.121:0/1444302236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7020154ca0 0x7f7020155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.464+0000 7f702531b700 1 -- 172.21.15.121:0/1444302236 >> 172.21.15.121:0/1444302236 conn(0x7f7020062190 msgr2=0x7f70200625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:07.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.468+0000 7f702531b700 1 -- 172.21.15.121:0/1444302236 shutdown_connections 2023-11-15T03:11:07.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.468+0000 7f702531b700 1 -- 172.21.15.121:0/1444302236 wait complete. 2023-11-15T03:11:07.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.468+0000 7f702531b700 1 Processor -- start 2023-11-15T03:11:07.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.468+0000 7f702531b700 1 -- start start 2023-11-15T03:11:07.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.472+0000 7f702531b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70200a57c0 0x7f70201496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.472+0000 7f702531b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7020154ca0 0x7f702014c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.472+0000 7f702531b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70201562f0 0x7f70200bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.472+0000 7f702531b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702014dd40 con 0x7f70200a57c0 2023-11-15T03:11:07.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.472+0000 7f702531b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702014d130 con 0x7f70201562f0 2023-11-15T03:11:07.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.472+0000 7f702531b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f702014d410 con 0x7f7020154ca0 2023-11-15T03:11:07.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.472+0000 7f701effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70200a57c0 0x7f70201496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:07.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.472+0000 7f701effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70200a57c0 0x7f70201496a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39972/0 (socket says 172.21.15.121:39972) 2023-11-15T03:11:07.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.472+0000 7f701effd700 1 -- 172.21.15.121:0/1817104007 learned_addr learned my addr 172.21.15.121:0/1817104007 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:07.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.472+0000 7f701e7fc700 1 --2- 172.21.15.121:0/1817104007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7020154ca0 0x7f702014c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:07.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.472+0000 7f701f7fe700 1 --2- 172.21.15.121:0/1817104007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70201562f0 0x7f70200bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:07.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.472+0000 7f701effd700 1 -- 172.21.15.121:0/1817104007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7020154ca0 msgr2=0x7f702014c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:07.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.472+0000 7f701effd700 1 --2- 172.21.15.121:0/1817104007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7020154ca0 0x7f702014c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.472+0000 7f701effd700 1 -- 172.21.15.121:0/1817104007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70201562f0 msgr2=0x7f70200bfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:07.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.472+0000 7f701effd700 1 --2- 172.21.15.121:0/1817104007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70201562f0 0x7f70200bfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.472+0000 7f701effd700 1 -- 172.21.15.121:0/1817104007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7018009ae0 con 0x7f70200a57c0 2023-11-15T03:11:07.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.472+0000 7f701effd700 1 --2- 172.21.15.121:0/1817104007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70200a57c0 0x7f70201496a0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 rx=0x7f701000e950 tx=0x7f7010009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:07.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.476+0000 7f7006ffd700 1 -- 172.21.15.121:0/1817104007 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010012600 con 0x7f70200a57c0 2023-11-15T03:11:07.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.476+0000 7f7006ffd700 1 -- 172.21.15.121:0/1817104007 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7010012760 con 0x7f70200a57c0 2023-11-15T03:11:07.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.476+0000 7f7006ffd700 1 -- 172.21.15.121:0/1817104007 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010012ae0 con 0x7f70200a57c0 2023-11-15T03:11:07.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.476+0000 7f702531b700 1 -- 172.21.15.121:0/1817104007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f702014d6f0 con 0x7f70200a57c0 2023-11-15T03:11:07.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.476+0000 7f702531b700 1 -- 172.21.15.121:0/1817104007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7020149b90 con 0x7f70200a57c0 2023-11-15T03:11:07.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.476+0000 7f702531b700 1 -- 172.21.15.121:0/1817104007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f702006c150 con 0x7f70200a57c0 2023-11-15T03:11:07.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.476+0000 7f7006ffd700 1 -- 172.21.15.121:0/1817104007 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f701001b400 con 0x7f70200a57c0 2023-11-15T03:11:07.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.480+0000 7f7006ffd700 1 --2- 172.21.15.121:0/1817104007 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f700c05a510 0x7f700c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.480+0000 7f7006ffd700 1 -- 172.21.15.121:0/1817104007 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9706+0+0 (secure 0 0 0) 0x7f7010075b20 con 0x7f70200a57c0 2023-11-15T03:11:07.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.480+0000 7f701e7fc700 1 --2- 172.21.15.121:0/1817104007 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f700c05a510 0x7f700c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:07.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.480+0000 7f701e7fc700 1 --2- 172.21.15.121:0/1817104007 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f700c05a510 0x7f700c05c9d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7f7020167720 tx=0x7f7014007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:07.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.488+0000 7f7006ffd700 1 -- 172.21.15.121:0/1817104007 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7010048460 con 0x7f70200a57c0 2023-11-15T03:11:07.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.688+0000 7f702531b700 1 -- 172.21.15.121:0/1817104007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70200c4500 con 0x7f70200a57c0 2023-11-15T03:11:07.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.688+0000 7f7006ffd700 1 -- 172.21.15.121:0/1817104007 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 342 v342) v1 ==== 95+0+4497 (secure 0 0 0) 0x7f7010016030 con 0x7f70200a57c0 2023-11-15T03:11:07.696 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:07.696 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":342,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":10542,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2943522178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2943522178},{"type":"v1","addr":"172.21.15.121:6836","nonce":2943522178}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":342}],"filesystems":[{"mdsmap":{"epoch":341,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:11:02.249759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:11:07.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.692+0000 7f702531b700 1 -- 172.21.15.121:0/1817104007 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f700c05a510 msgr2=0x7f700c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:07.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.692+0000 7f702531b700 1 --2- 172.21.15.121:0/1817104007 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f700c05a510 0x7f700c05c9d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7f7020167720 tx=0x7f7014007d40).stop 2023-11-15T03:11:07.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.692+0000 7f702531b700 1 -- 172.21.15.121:0/1817104007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70200a57c0 msgr2=0x7f70201496a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:07.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.692+0000 7f702531b700 1 --2- 172.21.15.121:0/1817104007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70200a57c0 0x7f70201496a0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 rx=0x7f701000e950 tx=0x7f7010009060).stop 2023-11-15T03:11:07.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.692+0000 7f702531b700 1 -- 172.21.15.121:0/1817104007 shutdown_connections 2023-11-15T03:11:07.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.692+0000 7f702531b700 1 --2- 172.21.15.121:0/1817104007 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f700c05a510 0x7f700c05c9d0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.692+0000 7f702531b700 1 --2- 172.21.15.121:0/1817104007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70201562f0 0x7f70200bfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.692+0000 7f702531b700 1 --2- 172.21.15.121:0/1817104007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7020154ca0 0x7f702014c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.692+0000 7f702531b700 1 --2- 172.21.15.121:0/1817104007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70200a57c0 0x7f70201496a0 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.692+0000 7f702531b700 1 -- 172.21.15.121:0/1817104007 >> 172.21.15.121:0/1817104007 conn(0x7f7020062190 msgr2=0x7f7020155610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:07.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.692+0000 7f702531b700 1 -- 172.21.15.121:0/1817104007 shutdown_connections 2023-11-15T03:11:07.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.692+0000 7f702531b700 1 -- 172.21.15.121:0/1817104007 wait complete. 2023-11-15T03:11:07.702 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 342 2023-11-15T03:11:07.719 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:11:07.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.868+0000 7fd366369700 1 Processor -- start 2023-11-15T03:11:07.879 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd366369700 1 -- start start 2023-11-15T03:11:07.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd366369700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd360154ca0 0x7fd360155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd366369700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd3601562f0 0x7fd3600a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd366369700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3600a8f40 0x7fd3600a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd366369700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3600a8280 con 0x7fd3601562f0 2023-11-15T03:11:07.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd366369700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3600a6d40 con 0x7fd360154ca0 2023-11-15T03:11:07.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd366369700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600c4c30 con 0x7fd3600a8f40 2023-11-15T03:11:07.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd3577fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd3601562f0 0x7fd3600a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:07.885 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd3577fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd3601562f0 0x7fd3600a8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39974/0 (socket says 172.21.15.121:39974) 2023-11-15T03:11:07.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd3577fe700 1 -- 172.21.15.121:0/3304867701 learned_addr learned my addr 172.21.15.121:0/3304867701 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:07.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd364906700 1 --2- 172.21.15.121:0/3304867701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3600a8f40 0x7fd3600a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:07.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd35ffff700 1 --2- 172.21.15.121:0/3304867701 >> v2:172.21.15.190:3300/0 conn(0x7fd360154ca0 0x7fd360155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:07.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd3577fe700 1 -- 172.21.15.121:0/3304867701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3600a8f40 msgr2=0x7fd3600a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:07.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd3577fe700 1 --2- 172.21.15.121:0/3304867701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3600a8f40 0x7fd3600a57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd3577fe700 1 -- 172.21.15.121:0/3304867701 >> v2:172.21.15.190:3300/0 conn(0x7fd360154ca0 msgr2=0x7fd360155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:07.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd3577fe700 1 --2- 172.21.15.121:0/3304867701 >> v2:172.21.15.190:3300/0 conn(0x7fd360154ca0 0x7fd360155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd3577fe700 1 -- 172.21.15.121:0/3304867701 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3600a6f50 con 0x7fd3601562f0 2023-11-15T03:11:07.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd3577fe700 1 --2- 172.21.15.121:0/3304867701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3601562f0 0x7fd3600a8a50 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 rx=0x7fd360064f10 tx=0x7fd348009840).ready entity=mon.0 client_cookie=b3b79cdde85bb7b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:07.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd35f7fe700 1 -- 172.21.15.121:0/3304867701 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34800e070 con 0x7fd3601562f0 2023-11-15T03:11:07.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd35f7fe700 1 -- 172.21.15.121:0/3304867701 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34800ba50 con 0x7fd3601562f0 2023-11-15T03:11:07.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.872+0000 7fd35f7fe700 1 -- 172.21.15.121:0/3304867701 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34800bdd0 con 0x7fd3601562f0 2023-11-15T03:11:07.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.876+0000 7fd366369700 1 -- 172.21.15.121:0/3304867701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3601562f0 msgr2=0x7fd3600a8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:07.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.876+0000 7fd366369700 1 --2- 172.21.15.121:0/3304867701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3601562f0 0x7fd3600a8a50 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 rx=0x7fd360064f10 tx=0x7fd348009840).stop 2023-11-15T03:11:07.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.876+0000 7fd366369700 1 -- 172.21.15.121:0/3304867701 shutdown_connections 2023-11-15T03:11:07.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.876+0000 7fd366369700 1 --2- 172.21.15.121:0/3304867701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3600a8f40 0x7fd3600a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.876+0000 7fd366369700 1 --2- 172.21.15.121:0/3304867701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3601562f0 0x7fd3600a8a50 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.876+0000 7fd366369700 1 --2- 172.21.15.121:0/3304867701 >> v2:172.21.15.190:3300/0 conn(0x7fd360154ca0 0x7fd360155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.876+0000 7fd366369700 1 -- 172.21.15.121:0/3304867701 >> 172.21.15.121:0/3304867701 conn(0x7fd360062190 msgr2=0x7fd3600625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:07.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.876+0000 7fd366369700 1 -- 172.21.15.121:0/3304867701 shutdown_connections 2023-11-15T03:11:07.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.876+0000 7fd366369700 1 -- 172.21.15.121:0/3304867701 wait complete. 2023-11-15T03:11:07.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.876+0000 7fd366369700 1 Processor -- start 2023-11-15T03:11:07.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd366369700 1 -- start start 2023-11-15T03:11:07.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd366369700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3600a8f40 0x7fd3600aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd366369700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd360154ca0 0x7fd36014b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd366369700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3601562f0 0x7fd3600bfd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd366369700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600c1310 con 0x7fd3600a8f40 2023-11-15T03:11:07.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd366369700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600c1bb0 con 0x7fd360154ca0 2023-11-15T03:11:07.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd366369700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3601541e0 con 0x7fd3601562f0 2023-11-15T03:11:07.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd35ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3600a8f40 0x7fd3600aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:07.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd35ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3600a8f40 0x7fd3600aa9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39988/0 (socket says 172.21.15.121:39988) 2023-11-15T03:11:07.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd35ffff700 1 -- 172.21.15.121:0/7822103 learned_addr learned my addr 172.21.15.121:0/7822103 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:07.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd3577fe700 1 --2- 172.21.15.121:0/7822103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd360154ca0 0x7fd36014b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:07.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd364906700 1 --2- 172.21.15.121:0/7822103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3601562f0 0x7fd3600bfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:07.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd35ffff700 1 -- 172.21.15.121:0/7822103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3601562f0 msgr2=0x7fd3600bfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:07.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd35ffff700 1 --2- 172.21.15.121:0/7822103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3601562f0 0x7fd3600bfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd35ffff700 1 -- 172.21.15.121:0/7822103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd360154ca0 msgr2=0x7fd36014b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:07.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd35ffff700 1 --2- 172.21.15.121:0/7822103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd360154ca0 0x7fd36014b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:07.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd35ffff700 1 -- 172.21.15.121:0/7822103 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd348009ae0 con 0x7fd3600a8f40 2023-11-15T03:11:07.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd3577fe700 1 --2- 172.21.15.121:0/7822103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd360154ca0 0x7fd36014b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:07.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd35ffff700 1 --2- 172.21.15.121:0/7822103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3600a8f40 0x7fd3600aa9b0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 rx=0x7fd35000d8c0 tx=0x7fd350008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:07.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd35cff9700 1 -- 172.21.15.121:0/7822103 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3500095b0 con 0x7fd3600a8f40 2023-11-15T03:11:07.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd35cff9700 1 -- 172.21.15.121:0/7822103 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd350009710 con 0x7fd3600a8f40 2023-11-15T03:11:07.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.880+0000 7fd366369700 1 -- 172.21.15.121:0/7822103 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3600c1560 con 0x7fd3600a8f40 2023-11-15T03:11:07.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.884+0000 7fd35cff9700 1 -- 172.21.15.121:0/7822103 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd350009a90 con 0x7fd3600a8f40 2023-11-15T03:11:07.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.884+0000 7fd366369700 1 -- 172.21.15.121:0/7822103 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd36014bc00 con 0x7fd3600a8f40 2023-11-15T03:11:07.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.884+0000 7fd366369700 1 -- 172.21.15.121:0/7822103 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd36006c150 con 0x7fd3600a8f40 2023-11-15T03:11:07.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.884+0000 7fd35cff9700 1 -- 172.21.15.121:0/7822103 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd350013480 con 0x7fd3600a8f40 2023-11-15T03:11:07.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.884+0000 7fd35cff9700 1 --2- 172.21.15.121:0/7822103 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd34005a4e0 0x7fd34005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:07.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.884+0000 7fd35cff9700 1 -- 172.21.15.121:0/7822103 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9706+0+0 (secure 0 0 0) 0x7fd350075490 con 0x7fd3600a8f40 2023-11-15T03:11:07.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.888+0000 7fd3577fe700 1 --2- 172.21.15.121:0/7822103 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd34005a4e0 0x7fd34005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:07.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.888+0000 7fd3577fe700 1 --2- 172.21.15.121:0/7822103 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd34005a4e0 0x7fd34005c9a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7fd360156760 tx=0x7fd348009840).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:07.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:07.892+0000 7fd35cff9700 1 -- 172.21.15.121:0/7822103 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd350047d50 con 0x7fd3600a8f40 2023-11-15T03:11:08.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.060+0000 7fd366369700 1 -- 172.21.15.121:0/7822103 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3600a7230 con 0x7fd3600a8f40 2023-11-15T03:11:08.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.064+0000 7fd35cff9700 1 -- 172.21.15.121:0/7822103 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+22474 (secure 0 0 0) 0x7fd35004b370 con 0x7fd3600a8f40 2023-11-15T03:11:08.074 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:08.074 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":134,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:11:01.253780+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-15T03:03:19.872820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-15T03:03:20.771478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.26","mappings":[{"from":1,"to":3}]},{"pgid":"14.1","mappings":[{"from":1,"to":5}]},{"pgid":"14.17","mappings":[{"from":1,"to":7}]},{"pgid":"14.1b","mappings":[{"from":1,"to":7}]},{"pgid":"14.38","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6834/3720977338":"2023-11-16T03:11:01.253720+0000","172.21.15.121:6836/2269363825":"2023-11-16T03:10:47.448136+0000","172.21.15.121:6834/4290809693":"2023-11-16T03:10:39.351864+0000","172.21.15.121:6836/1750624080":"2023-11-16T03:10:32.820769+0000","172.21.15.121:6836/2622666226":"2023-11-16T03:10:16.226454+0000","172.21.15.121:6834/2622666226":"2023-11-16T03:10:16.226454+0000","172.21.15.121:6834/1175031592":"2023-11-16T03:09:57.665840+0000","172.21.15.121:6836/621064675":"2023-11-16T03:09:34.308003+0000","172.21.15.121:6834/621064675":"2023-11-16T03:09:34.308003+0000","172.21.15.121:6836/142843299":"2023-11-16T03:09:27.621091+0000","172.21.15.121:6836/2666431227":"2023-11-16T03:09:01.175268+0000","172.21.15.121:6836/3720977338":"2023-11-16T03:11:01.253720+0000","172.21.15.121:6834/2666431227":"2023-11-16T03:09:01.175268+0000","172.21.15.121:6836/240077764":"2023-11-16T03:10:09.963248+0000","172.21.15.121:6836/3050548459":"2023-11-16T03:08:50.252959+0000","172.21.15.121:6834/3050548459":"2023-11-16T03:08:50.252959+0000","172.21.15.121:6836/1175031592":"2023-11-16T03:09:57.665840+0000","172.21.15.121:6836/2860780116":"2023-11-16T03:05:25.265026+0000","172.21.15.121:6834/2860780116":"2023-11-16T03:05:25.265026+0000","172.21.15.121:6836/2402059232":"2023-11-16T03:05:11.153161+0000","172.21.15.121:6834/1378715461":"2023-11-16T03:03:55.220003+0000","172.21.15.121:6834/1704127295":"2023-11-16T03:07:01.277510+0000","172.21.15.121:6836/400230871":"2023-11-16T03:05:55.172342+0000","172.21.15.121:6834/2873318212":"2023-11-16T03:08:06.315236+0000","172.21.15.121:6834/2402059232":"2023-11-16T03:05:11.153161+0000","172.21.15.121:6836/682793496":"2023-11-16T03:05:00.228736+0000","172.21.15.121:6834/3831023402":"2023-11-16T03:04:56.741702+0000","172.21.15.121:6834/2127410277":"2023-11-16T03:06:27.278977+0000","172.21.15.121:6834/682793496":"2023-11-16T03:05:00.228736+0000","172.21.15.121:6836/1529582418":"2023-11-16T03:05:32.331792+0000","172.21.15.121:6834/2831707405":"2023-11-16T03:04:44.820301+0000","172.21.15.121:6834/2289093120":"2023-11-16T03:06:57.977388+0000","172.21.15.121:6834/1750624080":"2023-11-16T03:10:32.820769+0000","172.21.15.121:6834/2646177140":"2023-11-16T03:03:30.945890+0000","172.21.15.121:6836/4290809693":"2023-11-16T03:10:39.351864+0000","172.21.15.121:6836/2544923351":"2023-11-16T03:07:44.198253+0000","172.21.15.121:6836/461059977":"2023-11-16T03:07:55.217761+0000","172.21.15.121:6836/3454441824":"2023-11-16T03:05:51.702394+0000","172.21.15.121:6836/2831707405":"2023-11-16T03:04:44.820301+0000","172.21.15.121:6836/2289093120":"2023-11-16T03:06:57.977388+0000","172.21.15.121:6834/3956345798":"2023-11-16T03:04:25.109148+0000","172.21.15.121:6834/57668150":"2023-11-16T03:04:14.091017+0000","172.21.15.121:6834/400230871":"2023-11-16T03:05:55.172342+0000","172.21.15.121:6834/228280964":"2023-11-16T03:06:08.282419+0000","172.21.15.121:6836/1378715461":"2023-11-16T03:03:55.220003+0000","172.21.15.121:6836/2873318212":"2023-11-16T03:08:06.315236+0000","172.21.15.121:6834/1529582418":"2023-11-16T03:05:32.331792+0000","172.21.15.121:6836/228280964":"2023-11-16T03:06:08.282419+0000","172.21.15.121:6834/3764682615":"2023-11-16T03:04:03.026914+0000","172.21.15.121:6834/3454441824":"2023-11-16T03:05:51.702394+0000","172.21.15.121:6836/2646177140":"2023-11-16T03:03:30.945890+0000","172.21.15.121:6834/763933658":"2023-11-16T03:03:48.628787+0000","172.21.15.121:6836/3831023402":"2023-11-16T03:04:56.741702+0000","172.21.15.121:6836/57668150":"2023-11-16T03:04:14.091017+0000","172.21.15.121:6836/3764682615":"2023-11-16T03:04:03.026914+0000","172.21.15.121:6834/461059977":"2023-11-16T03:07:55.217761+0000","172.21.15.121:6836/1244489663":"2023-11-16T03:08:29.214705+0000","172.21.15.121:6836/763933658":"2023-11-16T03:03:48.628787+0000","172.21.15.121:6836/3956345798":"2023-11-16T03:04:25.109148+0000","172.21.15.121:6836/1704127295":"2023-11-16T03:07:01.277510+0000","172.21.15.121:6836/564044995":"2023-11-16T03:06:16.372057+0000","172.21.15.121:6834/2269363825":"2023-11-16T03:10:47.448136+0000","172.21.15.121:6836/2127410277":"2023-11-16T03:06:27.278977+0000","172.21.15.121:6834/240077764":"2023-11-16T03:10:09.963248+0000","172.21.15.121:6836/909619389":"2023-11-16T03:06:38.112007+0000","172.21.15.121:6834/1007300937":"2023-11-16T03:07:12.272673+0000","172.21.15.121:6836/1007300937":"2023-11-16T03:07:12.272673+0000","172.21.15.121:6836/2131958479":"2023-11-16T03:07:23.129117+0000","172.21.15.121:6834/564044995":"2023-11-16T03:06:16.372057+0000","172.21.15.121:6836/115896047":"2023-11-16T03:07:34.194687+0000","172.21.15.121:6834/1958064439":"2023-11-16T03:08:25.922572+0000","172.21.15.121:6836/1958064439":"2023-11-16T03:08:25.922572+0000","172.21.15.121:6834/2544923351":"2023-11-16T03:07:44.198253+0000","172.21.15.121:6834/1244489663":"2023-11-16T03:08:29.214705+0000","172.21.15.121:6834/142843299":"2023-11-16T03:09:27.621091+0000","172.21.15.121:6834/1978069160":"2023-11-16T03:08:40.321133+0000","172.21.15.121:6834/115896047":"2023-11-16T03:07:34.194687+0000","172.21.15.121:6834/909619389":"2023-11-16T03:06:38.112007+0000","172.21.15.121:6834/2131958479":"2023-11-16T03:07:23.129117+0000","172.21.15.121:6836/1978069160":"2023-11-16T03:08:40.321133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:11:08.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.068+0000 7fd366369700 1 -- 172.21.15.121:0/7822103 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd34005a4e0 msgr2=0x7fd34005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:08.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.068+0000 7fd366369700 1 --2- 172.21.15.121:0/7822103 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd34005a4e0 0x7fd34005c9a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7fd360156760 tx=0x7fd348009840).stop 2023-11-15T03:11:08.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.068+0000 7fd366369700 1 -- 172.21.15.121:0/7822103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3600a8f40 msgr2=0x7fd3600aa9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:08.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.068+0000 7fd366369700 1 --2- 172.21.15.121:0/7822103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3600a8f40 0x7fd3600aa9b0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 rx=0x7fd35000d8c0 tx=0x7fd350008040).stop 2023-11-15T03:11:08.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.068+0000 7fd366369700 1 -- 172.21.15.121:0/7822103 shutdown_connections 2023-11-15T03:11:08.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.068+0000 7fd366369700 1 --2- 172.21.15.121:0/7822103 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd34005a4e0 0x7fd34005c9a0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.068+0000 7fd366369700 1 --2- 172.21.15.121:0/7822103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3601562f0 0x7fd3600bfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.068+0000 7fd366369700 1 --2- 172.21.15.121:0/7822103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd360154ca0 0x7fd36014b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.068+0000 7fd366369700 1 --2- 172.21.15.121:0/7822103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3600a8f40 0x7fd3600aa9b0 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.068+0000 7fd366369700 1 -- 172.21.15.121:0/7822103 >> 172.21.15.121:0/7822103 conn(0x7fd360062190 msgr2=0x7fd3600c67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:08.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.068+0000 7fd366369700 1 -- 172.21.15.121:0/7822103 shutdown_connections 2023-11-15T03:11:08.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.068+0000 7fd366369700 1 -- 172.21.15.121:0/7822103 wait complete. 2023-11-15T03:11:08.095 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-15T03:11:08.096 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:08.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.248+0000 7f56a6679700 1 Processor -- start 2023-11-15T03:11:08.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.248+0000 7f56a6679700 1 -- start start 2023-11-15T03:11:08.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.252+0000 7f56a6679700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f56a00c3280 0x7f56a00c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:08.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.252+0000 7f56a6679700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56a015be30 0x7f56a015c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:08.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.252+0000 7f56a6679700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56a00c1d50 con 0x7f56a0152140 2023-11-15T03:11:08.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.252+0000 7f56a6679700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56a00c1860 con 0x7f56a00c3280 2023-11-15T03:11:08.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.252+0000 7f56a6679700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a015cb80 con 0x7f56a015be30 2023-11-15T03:11:08.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.252+0000 7f56a5677700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f56a00c3280 0x7f56a00c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:08.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.252+0000 7f56a4e76700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f56a0152140 0x7f56a01525f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49002/0 (socket says 172.21.15.121:49002) 2023-11-15T03:11:08.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.252+0000 7f56a4e76700 1 -- 172.21.15.121:0/738323302 learned_addr learned my addr 172.21.15.121:0/738323302 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:08.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.252+0000 7f56a5e78700 1 --2- 172.21.15.121:0/738323302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56a015be30 0x7f56a015c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:08.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.252+0000 7f5697fff700 1 -- 172.21.15.121:0/738323302 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730888167 0 0) 0x7f56a00c1d50 con 0x7f56a0152140 2023-11-15T03:11:08.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.252+0000 7f5697fff700 1 -- 172.21.15.121:0/738323302 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f568c0033f0 con 0x7f56a0152140 2023-11-15T03:11:08.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.252+0000 7f56a5677700 1 -- 172.21.15.121:0/738323302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56a015be30 msgr2=0x7f56a015c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:08.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.252+0000 7f56a5677700 1 --2- 172.21.15.121:0/738323302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56a015be30 0x7f56a015c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.252+0000 7f56a5677700 1 -- 172.21.15.121:0/738323302 >> v1:172.21.15.121:6789/0 conn(0x7f56a0152140 legacy=0x7f56a01525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:08.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.252+0000 7f56a5677700 1 -- 172.21.15.121:0/738323302 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a00a9240 con 0x7f56a00c3280 2023-11-15T03:11:08.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.252+0000 7f56a5677700 1 --2- 172.21.15.121:0/738323302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56a00c3280 0x7f56a00c3660 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 rx=0x7f569c009e60 tx=0x7f569c009780).ready entity=mon.0 client_cookie=80fb5920e745b4bd server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:08.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.252+0000 7f5697fff700 1 -- 172.21.15.121:0/738323302 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c00e070 con 0x7f56a00c3280 2023-11-15T03:11:08.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.252+0000 7f5697fff700 1 -- 172.21.15.121:0/738323302 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569c00bab0 con 0x7f56a00c3280 2023-11-15T03:11:08.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.252+0000 7f5697fff700 1 -- 172.21.15.121:0/738323302 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c00be30 con 0x7f56a00c3280 2023-11-15T03:11:08.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.256+0000 7f56a6679700 1 -- 172.21.15.121:0/738323302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56a00c3280 msgr2=0x7f56a00c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:08.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.256+0000 7f56a6679700 1 --2- 172.21.15.121:0/738323302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56a00c3280 0x7f56a00c3660 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 rx=0x7f569c009e60 tx=0x7f569c009780).stop 2023-11-15T03:11:08.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.256+0000 7f56a6679700 1 -- 172.21.15.121:0/738323302 shutdown_connections 2023-11-15T03:11:08.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.256+0000 7f56a6679700 1 --2- 172.21.15.121:0/738323302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56a015be30 0x7f56a015c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.256+0000 7f56a6679700 1 --2- 172.21.15.121:0/738323302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56a00c3280 0x7f56a00c3660 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.256+0000 7f56a6679700 1 -- 172.21.15.121:0/738323302 >> 172.21.15.121:0/738323302 conn(0x7f56a0062190 msgr2=0x7f56a0062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:08.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.256+0000 7f56a6679700 1 -- 172.21.15.121:0/738323302 shutdown_connections 2023-11-15T03:11:08.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.256+0000 7f56a6679700 1 -- 172.21.15.121:0/738323302 wait complete. 2023-11-15T03:11:08.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.256+0000 7f56a6679700 1 Processor -- start 2023-11-15T03:11:08.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.256+0000 7f56a6679700 1 -- start start 2023-11-15T03:11:08.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a6679700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56a00c3280 0x7f56a0159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:08.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a6679700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56a0152140 0x7f56a0169960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:08.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a6679700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56a015be30 0x7f56a016bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:08.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a6679700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a00c6e60 con 0x7f56a015be30 2023-11-15T03:11:08.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a6679700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a0169e50 con 0x7f56a0152140 2023-11-15T03:11:08.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a6679700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a0154ea0 con 0x7f56a00c3280 2023-11-15T03:11:08.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a5e78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56a015be30 0x7f56a016bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:08.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a5e78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56a015be30 0x7f56a016bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39996/0 (socket says 172.21.15.121:39996) 2023-11-15T03:11:08.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a5e78700 1 -- 172.21.15.121:0/3680323575 learned_addr learned my addr 172.21.15.121:0/3680323575 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:08.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a5677700 1 --2- 172.21.15.121:0/3680323575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56a00c3280 0x7f56a0159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:08.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a4e76700 1 --2- 172.21.15.121:0/3680323575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56a0152140 0x7f56a0169960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:08.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a5e78700 1 -- 172.21.15.121:0/3680323575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56a00c3280 msgr2=0x7f56a0159200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:08.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a5e78700 1 --2- 172.21.15.121:0/3680323575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56a00c3280 0x7f56a0159200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a5e78700 1 -- 172.21.15.121:0/3680323575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56a0152140 msgr2=0x7f56a0169960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:08.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a5e78700 1 --2- 172.21.15.121:0/3680323575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56a0152140 0x7f56a0169960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a5e78700 1 -- 172.21.15.121:0/3680323575 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c009a20 con 0x7f56a015be30 2023-11-15T03:11:08.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a4e76700 1 --2- 172.21.15.121:0/3680323575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56a0152140 0x7f56a0169960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:11:08.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a5e78700 1 --2- 172.21.15.121:0/3680323575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56a015be30 0x7f56a016bd60 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 rx=0x7f569800e900 tx=0x7f5698009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:08.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56957fa700 1 -- 172.21.15.121:0/3680323575 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569800c640 con 0x7f56a015be30 2023-11-15T03:11:08.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56957fa700 1 -- 172.21.15.121:0/3680323575 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569800c7a0 con 0x7f56a015be30 2023-11-15T03:11:08.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a6679700 1 -- 172.21.15.121:0/3680323575 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a01542d0 con 0x7f56a015be30 2023-11-15T03:11:08.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56957fa700 1 -- 172.21.15.121:0/3680323575 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569800cb20 con 0x7f56a015be30 2023-11-15T03:11:08.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a6679700 1 -- 172.21.15.121:0/3680323575 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a0154620 con 0x7f56a015be30 2023-11-15T03:11:08.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56a6679700 1 -- 172.21.15.121:0/3680323575 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a0154c50 con 0x7f56a015be30 2023-11-15T03:11:08.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.260+0000 7f56957fa700 1 -- 172.21.15.121:0/3680323575 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f569801b400 con 0x7f56a015be30 2023-11-15T03:11:08.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.264+0000 7f56957fa700 1 --2- 172.21.15.121:0/3680323575 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f568405a510 0x7f568405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:08.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.264+0000 7f56957fa700 1 -- 172.21.15.121:0/3680323575 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9706+0+0 (secure 0 0 0) 0x7f5698075a20 con 0x7f56a015be30 2023-11-15T03:11:08.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.264+0000 7f56a5677700 1 --2- 172.21.15.121:0/3680323575 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f568405a510 0x7f568405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:08.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.264+0000 7f56a5677700 1 --2- 172.21.15.121:0/3680323575 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f568405a510 0x7f568405c9d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7f56a0159ba0 tx=0x7f56a00c1020).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:08.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.268+0000 7f56957fa700 1 -- 172.21.15.121:0/3680323575 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5698048360 con 0x7f56a015be30 2023-11-15T03:11:08.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.464+0000 7f56a6679700 1 -- 172.21.15.121:0/3680323575 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56a00a7cc0 con 0x7f56a015be30 2023-11-15T03:11:08.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.468+0000 7f56957fa700 1 -- 172.21.15.121:0/3680323575 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 342 v342) v1 ==== 95+0+4497 (secure 0 0 0) 0x7f5698016030 con 0x7f56a015be30 2023-11-15T03:11:08.475 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:08.476 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":342,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":10542,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2943522178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2943522178},{"type":"v1","addr":"172.21.15.121:6836","nonce":2943522178}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":342}],"filesystems":[{"mdsmap":{"epoch":341,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:11:02.249759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:11:08.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.468+0000 7f56a6679700 1 -- 172.21.15.121:0/3680323575 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f568405a510 msgr2=0x7f568405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:08.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.468+0000 7f56a6679700 1 --2- 172.21.15.121:0/3680323575 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f568405a510 0x7f568405c9d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7f56a0159ba0 tx=0x7f56a00c1020).stop 2023-11-15T03:11:08.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.468+0000 7f56a6679700 1 -- 172.21.15.121:0/3680323575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56a015be30 msgr2=0x7f56a016bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:08.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.468+0000 7f56a6679700 1 --2- 172.21.15.121:0/3680323575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56a015be30 0x7f56a016bd60 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 rx=0x7f569800e900 tx=0x7f5698009060).stop 2023-11-15T03:11:08.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.468+0000 7f56a6679700 1 -- 172.21.15.121:0/3680323575 shutdown_connections 2023-11-15T03:11:08.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.468+0000 7f56a6679700 1 --2- 172.21.15.121:0/3680323575 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f568405a510 0x7f568405c9d0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.468+0000 7f56a6679700 1 --2- 172.21.15.121:0/3680323575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56a015be30 0x7f56a016bd60 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.468+0000 7f56a6679700 1 --2- 172.21.15.121:0/3680323575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56a0152140 0x7f56a0169960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.468+0000 7f56a6679700 1 --2- 172.21.15.121:0/3680323575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56a00c3280 0x7f56a0159200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.468+0000 7f56a6679700 1 -- 172.21.15.121:0/3680323575 >> 172.21.15.121:0/3680323575 conn(0x7f56a0062190 msgr2=0x7f56a0151670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:08.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.468+0000 7f56a6679700 1 -- 172.21.15.121:0/3680323575 shutdown_connections 2023-11-15T03:11:08.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.472+0000 7f56a6679700 1 -- 172.21.15.121:0/3680323575 wait complete. 2023-11-15T03:11:08.483 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 342 2023-11-15T03:11:08.500 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:08.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.648+0000 7fe38b0b9700 1 Processor -- start 2023-11-15T03:11:08.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.648+0000 7fe38b0b9700 1 -- start start 2023-11-15T03:11:08.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.652+0000 7fe38b0b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe384066d70 0x7fe3840a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:08.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.652+0000 7fe38b0b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe3840c29a0 0x7fe38414e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:08.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.652+0000 7fe38b0b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3840a9350 con 0x7fe384067150 2023-11-15T03:11:08.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.652+0000 7fe38b0b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe384150ed0 con 0x7fe3840c29a0 2023-11-15T03:11:08.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.652+0000 7fe38b0b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3840675d0 con 0x7fe384066d70 2023-11-15T03:11:08.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.652+0000 7fe38a0b7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe384066d70 0x7fe3840a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:08.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.652+0000 7fe38a8b8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe3840c29a0 0x7fe38414e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:08.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.652+0000 7fe38a0b7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe384066d70 0x7fe3840a87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:41432/0 (socket says 172.21.15.121:41432) 2023-11-15T03:11:08.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.652+0000 7fe38a0b7700 1 -- 172.21.15.121:0/3662709631 learned_addr learned my addr 172.21.15.121:0/3662709631 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:08.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.652+0000 7fe38a8b8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe3840c29a0 0x7fe38414e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40006/0 (socket says 172.21.15.121:40006) 2023-11-15T03:11:08.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.652+0000 7fe3898b6700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe384067150 0x7fe3841518b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:54612/0 (socket says 172.21.15.121:54612) 2023-11-15T03:11:08.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.656+0000 7fe38a0b7700 1 -- 172.21.15.121:0/3662709631 >> v2:172.21.15.121:3300/0 conn(0x7fe3840c29a0 msgr2=0x7fe38414e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:08.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.656+0000 7fe38a0b7700 1 --2- 172.21.15.121:0/3662709631 >> v2:172.21.15.121:3300/0 conn(0x7fe3840c29a0 0x7fe38414e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.656+0000 7fe38a0b7700 1 -- 172.21.15.121:0/3662709631 >> v1:172.21.15.190:6789/0 conn(0x7fe384067150 legacy=0x7fe3841518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:08.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.656+0000 7fe3890b5700 1 -- 172.21.15.121:0/3662709631 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039390666 0 0) 0x7fe3840a9350 con 0x7fe384067150 2023-11-15T03:11:08.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.656+0000 7fe38a0b7700 1 -- 172.21.15.121:0/3662709631 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe384151420 con 0x7fe384066d70 2023-11-15T03:11:08.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.656+0000 7fe38a0b7700 1 --2- 172.21.15.121:0/3662709631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe384066d70 0x7fe3840a87f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7fe380006fd0 tx=0x7fe3800090f0).ready entity=mon.1 client_cookie=9c6458988e5a118b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:08.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.656+0000 7fe3890b5700 1 -- 172.21.15.121:0/3662709631 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38000b520 con 0x7fe384066d70 2023-11-15T03:11:08.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.656+0000 7fe3890b5700 1 -- 172.21.15.121:0/3662709631 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38000b680 con 0x7fe384066d70 2023-11-15T03:11:08.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.656+0000 7fe3890b5700 1 -- 172.21.15.121:0/3662709631 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38000ba00 con 0x7fe384066d70 2023-11-15T03:11:08.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.656+0000 7fe38b0b9700 1 -- 172.21.15.121:0/3662709631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe384066d70 msgr2=0x7fe3840a87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:08.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.656+0000 7fe38b0b9700 1 --2- 172.21.15.121:0/3662709631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe384066d70 0x7fe3840a87f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7fe380006fd0 tx=0x7fe3800090f0).stop 2023-11-15T03:11:08.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.656+0000 7fe38b0b9700 1 -- 172.21.15.121:0/3662709631 shutdown_connections 2023-11-15T03:11:08.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.656+0000 7fe38b0b9700 1 --2- 172.21.15.121:0/3662709631 >> v2:172.21.15.121:3300/0 conn(0x7fe3840c29a0 0x7fe38414e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.656+0000 7fe38b0b9700 1 --2- 172.21.15.121:0/3662709631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe384066d70 0x7fe3840a87f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.656+0000 7fe38b0b9700 1 -- 172.21.15.121:0/3662709631 >> 172.21.15.121:0/3662709631 conn(0x7fe384062190 msgr2=0x7fe384062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:08.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.656+0000 7fe38b0b9700 1 -- 172.21.15.121:0/3662709631 shutdown_connections 2023-11-15T03:11:08.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.656+0000 7fe38b0b9700 1 -- 172.21.15.121:0/3662709631 wait complete. 2023-11-15T03:11:08.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.660+0000 7fe38b0b9700 1 Processor -- start 2023-11-15T03:11:08.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.660+0000 7fe38b0b9700 1 -- start start 2023-11-15T03:11:08.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.660+0000 7fe38b0b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe384066d70 0x7fe3840bf440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:08.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.660+0000 7fe38b0b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe384067150 0x7fe3840bdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:08.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.660+0000 7fe38b0b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3840c29a0 0x7fe38414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:08.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.660+0000 7fe38b0b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3840be1b0 con 0x7fe384067150 2023-11-15T03:11:08.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.660+0000 7fe38b0b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38414b9b0 con 0x7fe384066d70 2023-11-15T03:11:08.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.660+0000 7fe38b0b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3840be3a0 con 0x7fe3840c29a0 2023-11-15T03:11:08.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.664+0000 7fe3898b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe384067150 0x7fe3840bdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:08.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.664+0000 7fe3898b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe384067150 0x7fe3840bdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40020/0 (socket says 172.21.15.121:40020) 2023-11-15T03:11:08.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.664+0000 7fe3898b6700 1 -- 172.21.15.121:0/1227910322 learned_addr learned my addr 172.21.15.121:0/1227910322 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:08.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.664+0000 7fe38a8b8700 1 --2- 172.21.15.121:0/1227910322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3840c29a0 0x7fe38414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:08.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.664+0000 7fe38a0b7700 1 --2- 172.21.15.121:0/1227910322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe384066d70 0x7fe3840bf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:08.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.664+0000 7fe3898b6700 1 -- 172.21.15.121:0/1227910322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3840c29a0 msgr2=0x7fe38414b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:08.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.664+0000 7fe3898b6700 1 --2- 172.21.15.121:0/1227910322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3840c29a0 0x7fe38414b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.664+0000 7fe3898b6700 1 -- 172.21.15.121:0/1227910322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe384066d70 msgr2=0x7fe3840bf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:08.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.664+0000 7fe3898b6700 1 --2- 172.21.15.121:0/1227910322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe384066d70 0x7fe3840bf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.664+0000 7fe3898b6700 1 -- 172.21.15.121:0/1227910322 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe380009370 con 0x7fe384067150 2023-11-15T03:11:08.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.664+0000 7fe3898b6700 1 --2- 172.21.15.121:0/1227910322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe384067150 0x7fe3840bdcc0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 rx=0x7fe37400c560 tx=0x7fe374007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:08.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.664+0000 7fe37a7fc700 1 -- 172.21.15.121:0/1227910322 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3740072e0 con 0x7fe384067150 2023-11-15T03:11:08.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.664+0000 7fe37a7fc700 1 -- 172.21.15.121:0/1227910322 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe374007440 con 0x7fe384067150 2023-11-15T03:11:08.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.664+0000 7fe38b0b9700 1 -- 172.21.15.121:0/1227910322 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3840be680 con 0x7fe384067150 2023-11-15T03:11:08.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.664+0000 7fe37a7fc700 1 -- 172.21.15.121:0/1227910322 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3740058f0 con 0x7fe384067150 2023-11-15T03:11:08.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.664+0000 7fe38b0b9700 1 -- 172.21.15.121:0/1227910322 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3840becc0 con 0x7fe384067150 2023-11-15T03:11:08.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.668+0000 7fe38b0b9700 1 -- 172.21.15.121:0/1227910322 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe38406bcf0 con 0x7fe384067150 2023-11-15T03:11:08.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.668+0000 7fe37a7fc700 1 -- 172.21.15.121:0/1227910322 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe374005ce0 con 0x7fe384067150 2023-11-15T03:11:08.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.668+0000 7fe37a7fc700 1 --2- 172.21.15.121:0/1227910322 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe36805a510 0x7fe36805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:08.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.668+0000 7fe37a7fc700 1 -- 172.21.15.121:0/1227910322 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9706+0+0 (secure 0 0 0) 0x7fe374074ea0 con 0x7fe384067150 2023-11-15T03:11:08.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.672+0000 7fe38a0b7700 1 --2- 172.21.15.121:0/1227910322 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe36805a510 0x7fe36805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:08.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.672+0000 7fe38a0b7700 1 --2- 172.21.15.121:0/1227910322 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe36805a510 0x7fe36805c9d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7fe380009130 tx=0x7fe380015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:08.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.672+0000 7fe37a7fc700 1 -- 172.21.15.121:0/1227910322 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe3740477c0 con 0x7fe384067150 2023-11-15T03:11:08.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.872+0000 7fe38b0b9700 1 -- 172.21.15.121:0/1227910322 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3840669b0 con 0x7fe384067150 2023-11-15T03:11:08.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.876+0000 7fe37a7fc700 1 -- 172.21.15.121:0/1227910322 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 342 v342) v1 ==== 95+0+4497 (secure 0 0 0) 0x7fe374018020 con 0x7fe384067150 2023-11-15T03:11:08.883 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:08.883 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":342,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":10542,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2943522178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2943522178},{"type":"v1","addr":"172.21.15.121:6836","nonce":2943522178}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":342}],"filesystems":[{"mdsmap":{"epoch":341,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:03:21.784802+0000","modified":"2023-11-15T03:11:02.249759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-15T03:11:08.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.880+0000 7fe38b0b9700 1 -- 172.21.15.121:0/1227910322 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe36805a510 msgr2=0x7fe36805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:08.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.880+0000 7fe38b0b9700 1 --2- 172.21.15.121:0/1227910322 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe36805a510 0x7fe36805c9d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7fe380009130 tx=0x7fe380015000).stop 2023-11-15T03:11:08.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.880+0000 7fe38b0b9700 1 -- 172.21.15.121:0/1227910322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe384067150 msgr2=0x7fe3840bdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:08.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.880+0000 7fe38b0b9700 1 --2- 172.21.15.121:0/1227910322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe384067150 0x7fe3840bdcc0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 rx=0x7fe37400c560 tx=0x7fe374007c90).stop 2023-11-15T03:11:08.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.880+0000 7fe38b0b9700 1 -- 172.21.15.121:0/1227910322 shutdown_connections 2023-11-15T03:11:08.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.880+0000 7fe38b0b9700 1 --2- 172.21.15.121:0/1227910322 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe36805a510 0x7fe36805c9d0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.880+0000 7fe38b0b9700 1 --2- 172.21.15.121:0/1227910322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3840c29a0 0x7fe38414b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.880+0000 7fe38b0b9700 1 --2- 172.21.15.121:0/1227910322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe384067150 0x7fe3840bdcc0 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.880+0000 7fe38b0b9700 1 --2- 172.21.15.121:0/1227910322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe384066d70 0x7fe3840bf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:08.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.880+0000 7fe38b0b9700 1 -- 172.21.15.121:0/1227910322 >> 172.21.15.121:0/1227910322 conn(0x7fe384062190 msgr2=0x7fe3840aaf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:08.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.880+0000 7fe38b0b9700 1 -- 172.21.15.121:0/1227910322 shutdown_connections 2023-11-15T03:11:08.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:08.880+0000 7fe38b0b9700 1 -- 172.21.15.121:0/1227910322 wait complete. 2023-11-15T03:11:08.890 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 342 2023-11-15T03:11:08.911 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:11:09.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.060+0000 7f6ea989b700 1 Processor -- start 2023-11-15T03:11:09.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.060+0000 7f6ea989b700 1 -- start start 2023-11-15T03:11:09.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea989b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6ea4151920 0x7f6ea4151d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:09.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea989b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea4068430 con 0x7f6ea40c4e30 2023-11-15T03:11:09.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea989b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea4064b50 con 0x7f6ea4063fe0 2023-11-15T03:11:09.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea989b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea40bec30 con 0x7f6ea4151920 2023-11-15T03:11:09.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea2ffd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6ea4151920 0x7f6ea4151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:09.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea3fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6ea40c4e30 0x7f6ea40a4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49016/0 (socket says 172.21.15.121:49016) 2023-11-15T03:11:09.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea3fff700 1 -- 172.21.15.121:0/3018995332 learned_addr learned my addr 172.21.15.121:0/3018995332 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:09.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea27fc700 1 -- 172.21.15.121:0/3018995332 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225513705 0 0) 0x7f6ea4068430 con 0x7f6ea40c4e30 2023-11-15T03:11:09.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea27fc700 1 -- 172.21.15.121:0/3018995332 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e900033f0 con 0x7f6ea40c4e30 2023-11-15T03:11:09.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea27fc700 1 -- 172.21.15.121:0/3018995332 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615835757 0 0) 0x7f6ea4064b50 con 0x7f6ea4063fe0 2023-11-15T03:11:09.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea2ffd700 1 -- 172.21.15.121:0/3018995332 >> v1:172.21.15.190:6789/0 conn(0x7f6ea4063fe0 legacy=0x7f6ea40c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:09.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea2ffd700 1 -- 172.21.15.121:0/3018995332 >> v1:172.21.15.121:6789/0 conn(0x7f6ea40c4e30 legacy=0x7f6ea40a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:09.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea2ffd700 1 -- 172.21.15.121:0/3018995332 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea4067b00 con 0x7f6ea4151920 2023-11-15T03:11:09.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea2ffd700 1 --2- 172.21.15.121:0/3018995332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ea4151920 0x7f6ea4151d00 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 rx=0x7f6e94009760 tx=0x7f6e94009060).ready entity=mon.0 client_cookie=b5d05583a9fe0bd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:09.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea27fc700 1 -- 172.21.15.121:0/3018995332 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9400f510 con 0x7f6ea4151920 2023-11-15T03:11:09.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea27fc700 1 -- 172.21.15.121:0/3018995332 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9400f670 con 0x7f6ea4151920 2023-11-15T03:11:09.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea27fc700 1 -- 172.21.15.121:0/3018995332 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9400f9f0 con 0x7f6ea4151920 2023-11-15T03:11:09.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea989b700 1 -- 172.21.15.121:0/3018995332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ea4151920 msgr2=0x7f6ea4151d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:09.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea989b700 1 --2- 172.21.15.121:0/3018995332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ea4151920 0x7f6ea4151d00 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 rx=0x7f6e94009760 tx=0x7f6e94009060).stop 2023-11-15T03:11:09.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea989b700 1 -- 172.21.15.121:0/3018995332 shutdown_connections 2023-11-15T03:11:09.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea989b700 1 --2- 172.21.15.121:0/3018995332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ea4151920 0x7f6ea4151d00 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:09.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.064+0000 7f6ea989b700 1 -- 172.21.15.121:0/3018995332 >> 172.21.15.121:0/3018995332 conn(0x7f6ea4069ac0 msgr2=0x7f6ea4090a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:09.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.068+0000 7f6ea989b700 1 -- 172.21.15.121:0/3018995332 shutdown_connections 2023-11-15T03:11:09.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.068+0000 7f6ea989b700 1 -- 172.21.15.121:0/3018995332 wait complete. 2023-11-15T03:11:09.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.068+0000 7f6ea989b700 1 Processor -- start 2023-11-15T03:11:09.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.068+0000 7f6ea989b700 1 -- start start 2023-11-15T03:11:09.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.068+0000 7f6ea989b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ea4063fe0 0x7f6ea40c1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:09.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.068+0000 7f6ea989b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ea40c4e30 0x7f6ea41673b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:09.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.068+0000 7f6ea989b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ea4151920 0x7f6ea4155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:09.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.068+0000 7f6ea989b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea41683a0 con 0x7f6ea4151920 2023-11-15T03:11:09.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.068+0000 7f6ea989b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea40c5d60 con 0x7f6ea40c4e30 2023-11-15T03:11:09.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.068+0000 7f6ea989b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea41678a0 con 0x7f6ea4063fe0 2023-11-15T03:11:09.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.068+0000 7f6ea3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ea4151920 0x7f6ea4155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:09.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.068+0000 7f6ea3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ea4151920 0x7f6ea4155f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40034/0 (socket says 172.21.15.121:40034) 2023-11-15T03:11:09.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.068+0000 7f6ea3fff700 1 -- 172.21.15.121:0/2135782581 learned_addr learned my addr 172.21.15.121:0/2135782581 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:09.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.068+0000 7f6e9bfff700 1 --2- 172.21.15.121:0/2135782581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ea40c4e30 0x7f6ea41673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:09.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.072+0000 7f6ea2ffd700 1 --2- 172.21.15.121:0/2135782581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ea4063fe0 0x7f6ea40c1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:09.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.072+0000 7f6ea3fff700 1 -- 172.21.15.121:0/2135782581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ea4063fe0 msgr2=0x7f6ea40c1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:09.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.072+0000 7f6ea3fff700 1 --2- 172.21.15.121:0/2135782581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ea4063fe0 0x7f6ea40c1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:09.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.072+0000 7f6ea3fff700 1 -- 172.21.15.121:0/2135782581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ea40c4e30 msgr2=0x7f6ea41673b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:09.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.072+0000 7f6ea3fff700 1 --2- 172.21.15.121:0/2135782581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ea40c4e30 0x7f6ea41673b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:09.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.072+0000 7f6ea3fff700 1 -- 172.21.15.121:0/2135782581 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e940092a0 con 0x7f6ea4151920 2023-11-15T03:11:09.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.072+0000 7f6ea3fff700 1 --2- 172.21.15.121:0/2135782581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ea4151920 0x7f6ea4155f30 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 rx=0x7f6e9c00c790 tx=0x7f6e9c007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:09.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.072+0000 7f6e9affd700 1 -- 172.21.15.121:0/2135782581 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c005d30 con 0x7f6ea4151920 2023-11-15T03:11:09.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.072+0000 7f6e9affd700 1 -- 172.21.15.121:0/2135782581 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9c005e90 con 0x7f6ea4151920 2023-11-15T03:11:09.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.072+0000 7f6ea989b700 1 -- 172.21.15.121:0/2135782581 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea4167b00 con 0x7f6ea4151920 2023-11-15T03:11:09.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.072+0000 7f6e9affd700 1 -- 172.21.15.121:0/2135782581 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c007320 con 0x7f6ea4151920 2023-11-15T03:11:09.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.072+0000 7f6ea989b700 1 -- 172.21.15.121:0/2135782581 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea41680c0 con 0x7f6ea4151920 2023-11-15T03:11:09.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.072+0000 7f6ea989b700 1 -- 172.21.15.121:0/2135782581 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea40c5f10 con 0x7f6ea4151920 2023-11-15T03:11:09.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.076+0000 7f6e9affd700 1 -- 172.21.15.121:0/2135782581 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6e9c007750 con 0x7f6ea4151920 2023-11-15T03:11:09.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.076+0000 7f6e9affd700 1 --2- 172.21.15.121:0/2135782581 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e8405a540 0x7f6e8405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:09.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.076+0000 7f6e9affd700 1 -- 172.21.15.121:0/2135782581 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9706+0+0 (secure 0 0 0) 0x7f6e9c075530 con 0x7f6ea4151920 2023-11-15T03:11:09.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.076+0000 7f6ea2ffd700 1 --2- 172.21.15.121:0/2135782581 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e8405a540 0x7f6e8405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:09.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.076+0000 7f6ea2ffd700 1 --2- 172.21.15.121:0/2135782581 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e8405a540 0x7f6e8405ca00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7f6e94009760 tx=0x7f6e94017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:09.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.084+0000 7f6e9affd700 1 -- 172.21.15.121:0/2135782581 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6ea40c5f10 con 0x7f6ea4151920 2023-11-15T03:11:09.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.248+0000 7f6ea989b700 1 -- 172.21.15.121:0/2135782581 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ea40c5ed0 con 0x7f6ea4151920 2023-11-15T03:11:09.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.252+0000 7f6e9affd700 1 -- 172.21.15.121:0/2135782581 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+22474 (secure 0 0 0) 0x7f6e9c047e50 con 0x7f6ea4151920 2023-11-15T03:11:09.261 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:09.261 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":134,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:11:01.253780+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-15T03:03:19.872820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-15T03:03:20.771478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.26","mappings":[{"from":1,"to":3}]},{"pgid":"14.1","mappings":[{"from":1,"to":5}]},{"pgid":"14.17","mappings":[{"from":1,"to":7}]},{"pgid":"14.1b","mappings":[{"from":1,"to":7}]},{"pgid":"14.38","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6834/3720977338":"2023-11-16T03:11:01.253720+0000","172.21.15.121:6836/2269363825":"2023-11-16T03:10:47.448136+0000","172.21.15.121:6834/4290809693":"2023-11-16T03:10:39.351864+0000","172.21.15.121:6836/1750624080":"2023-11-16T03:10:32.820769+0000","172.21.15.121:6836/2622666226":"2023-11-16T03:10:16.226454+0000","172.21.15.121:6834/2622666226":"2023-11-16T03:10:16.226454+0000","172.21.15.121:6834/1175031592":"2023-11-16T03:09:57.665840+0000","172.21.15.121:6836/621064675":"2023-11-16T03:09:34.308003+0000","172.21.15.121:6834/621064675":"2023-11-16T03:09:34.308003+0000","172.21.15.121:6836/142843299":"2023-11-16T03:09:27.621091+0000","172.21.15.121:6836/2666431227":"2023-11-16T03:09:01.175268+0000","172.21.15.121:6836/3720977338":"2023-11-16T03:11:01.253720+0000","172.21.15.121:6834/2666431227":"2023-11-16T03:09:01.175268+0000","172.21.15.121:6836/240077764":"2023-11-16T03:10:09.963248+0000","172.21.15.121:6836/3050548459":"2023-11-16T03:08:50.252959+0000","172.21.15.121:6834/3050548459":"2023-11-16T03:08:50.252959+0000","172.21.15.121:6836/1175031592":"2023-11-16T03:09:57.665840+0000","172.21.15.121:6836/2860780116":"2023-11-16T03:05:25.265026+0000","172.21.15.121:6834/2860780116":"2023-11-16T03:05:25.265026+0000","172.21.15.121:6836/2402059232":"2023-11-16T03:05:11.153161+0000","172.21.15.121:6834/1378715461":"2023-11-16T03:03:55.220003+0000","172.21.15.121:6834/1704127295":"2023-11-16T03:07:01.277510+0000","172.21.15.121:6836/400230871":"2023-11-16T03:05:55.172342+0000","172.21.15.121:6834/2873318212":"2023-11-16T03:08:06.315236+0000","172.21.15.121:6834/2402059232":"2023-11-16T03:05:11.153161+0000","172.21.15.121:6836/682793496":"2023-11-16T03:05:00.228736+0000","172.21.15.121:6834/3831023402":"2023-11-16T03:04:56.741702+0000","172.21.15.121:6834/2127410277":"2023-11-16T03:06:27.278977+0000","172.21.15.121:6834/682793496":"2023-11-16T03:05:00.228736+0000","172.21.15.121:6836/1529582418":"2023-11-16T03:05:32.331792+0000","172.21.15.121:6834/2831707405":"2023-11-16T03:04:44.820301+0000","172.21.15.121:6834/2289093120":"2023-11-16T03:06:57.977388+0000","172.21.15.121:6834/1750624080":"2023-11-16T03:10:32.820769+0000","172.21.15.121:6834/2646177140":"2023-11-16T03:03:30.945890+0000","172.21.15.121:6836/4290809693":"2023-11-16T03:10:39.351864+0000","172.21.15.121:6836/2544923351":"2023-11-16T03:07:44.198253+0000","172.21.15.121:6836/461059977":"2023-11-16T03:07:55.217761+0000","172.21.15.121:6836/3454441824":"2023-11-16T03:05:51.702394+0000","172.21.15.121:6836/2831707405":"2023-11-16T03:04:44.820301+0000","172.21.15.121:6836/2289093120":"2023-11-16T03:06:57.977388+0000","172.21.15.121:6834/3956345798":"2023-11-16T03:04:25.109148+0000","172.21.15.121:6834/57668150":"2023-11-16T03:04:14.091017+0000","172.21.15.121:6834/400230871":"2023-11-16T03:05:55.172342+0000","172.21.15.121:6834/228280964":"2023-11-16T03:06:08.282419+0000","172.21.15.121:6836/1378715461":"2023-11-16T03:03:55.220003+0000","172.21.15.121:6836/2873318212":"2023-11-16T03:08:06.315236+0000","172.21.15.121:6834/1529582418":"2023-11-16T03:05:32.331792+0000","172.21.15.121:6836/228280964":"2023-11-16T03:06:08.282419+0000","172.21.15.121:6834/3764682615":"2023-11-16T03:04:03.026914+0000","172.21.15.121:6834/3454441824":"2023-11-16T03:05:51.702394+0000","172.21.15.121:6836/2646177140":"2023-11-16T03:03:30.945890+0000","172.21.15.121:6834/763933658":"2023-11-16T03:03:48.628787+0000","172.21.15.121:6836/3831023402":"2023-11-16T03:04:56.741702+0000","172.21.15.121:6836/57668150":"2023-11-16T03:04:14.091017+0000","172.21.15.121:6836/3764682615":"2023-11-16T03:04:03.026914+0000","172.21.15.121:6834/461059977":"2023-11-16T03:07:55.217761+0000","172.21.15.121:6836/1244489663":"2023-11-16T03:08:29.214705+0000","172.21.15.121:6836/763933658":"2023-11-16T03:03:48.628787+0000","172.21.15.121:6836/3956345798":"2023-11-16T03:04:25.109148+0000","172.21.15.121:6836/1704127295":"2023-11-16T03:07:01.277510+0000","172.21.15.121:6836/564044995":"2023-11-16T03:06:16.372057+0000","172.21.15.121:6834/2269363825":"2023-11-16T03:10:47.448136+0000","172.21.15.121:6836/2127410277":"2023-11-16T03:06:27.278977+0000","172.21.15.121:6834/240077764":"2023-11-16T03:10:09.963248+0000","172.21.15.121:6836/909619389":"2023-11-16T03:06:38.112007+0000","172.21.15.121:6834/1007300937":"2023-11-16T03:07:12.272673+0000","172.21.15.121:6836/1007300937":"2023-11-16T03:07:12.272673+0000","172.21.15.121:6836/2131958479":"2023-11-16T03:07:23.129117+0000","172.21.15.121:6834/564044995":"2023-11-16T03:06:16.372057+0000","172.21.15.121:6836/115896047":"2023-11-16T03:07:34.194687+0000","172.21.15.121:6834/1958064439":"2023-11-16T03:08:25.922572+0000","172.21.15.121:6836/1958064439":"2023-11-16T03:08:25.922572+0000","172.21.15.121:6834/2544923351":"2023-11-16T03:07:44.198253+0000","172.21.15.121:6834/1244489663":"2023-11-16T03:08:29.214705+0000","172.21.15.121:6834/142843299":"2023-11-16T03:09:27.621091+0000","172.21.15.121:6834/1978069160":"2023-11-16T03:08:40.321133+0000","172.21.15.121:6834/115896047":"2023-11-16T03:07:34.194687+0000","172.21.15.121:6834/909619389":"2023-11-16T03:06:38.112007+0000","172.21.15.121:6834/2131958479":"2023-11-16T03:07:23.129117+0000","172.21.15.121:6836/1978069160":"2023-11-16T03:08:40.321133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:11:09.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.256+0000 7f6ea989b700 1 -- 172.21.15.121:0/2135782581 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e8405a540 msgr2=0x7f6e8405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:09.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.256+0000 7f6ea989b700 1 --2- 172.21.15.121:0/2135782581 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e8405a540 0x7f6e8405ca00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7f6e94009760 tx=0x7f6e94017000).stop 2023-11-15T03:11:09.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.256+0000 7f6ea989b700 1 -- 172.21.15.121:0/2135782581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ea4151920 msgr2=0x7f6ea4155f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:09.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.256+0000 7f6ea989b700 1 --2- 172.21.15.121:0/2135782581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ea4151920 0x7f6ea4155f30 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 rx=0x7f6e9c00c790 tx=0x7f6e9c007f90).stop 2023-11-15T03:11:09.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.256+0000 7f6ea989b700 1 -- 172.21.15.121:0/2135782581 shutdown_connections 2023-11-15T03:11:09.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.256+0000 7f6ea989b700 1 --2- 172.21.15.121:0/2135782581 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e8405a540 0x7f6e8405ca00 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:09.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.256+0000 7f6ea989b700 1 --2- 172.21.15.121:0/2135782581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ea4151920 0x7f6ea4155f30 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:09.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.256+0000 7f6ea989b700 1 --2- 172.21.15.121:0/2135782581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ea40c4e30 0x7f6ea41673b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:09.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.256+0000 7f6ea989b700 1 --2- 172.21.15.121:0/2135782581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ea4063fe0 0x7f6ea40c1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:09.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.256+0000 7f6ea989b700 1 -- 172.21.15.121:0/2135782581 >> 172.21.15.121:0/2135782581 conn(0x7f6ea4069ac0 msgr2=0x7f6ea40bde80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:09.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.256+0000 7f6ea989b700 1 -- 172.21.15.121:0/2135782581 shutdown_connections 2023-11-15T03:11:09.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.256+0000 7f6ea989b700 1 -- 172.21.15.121:0/2135782581 wait complete. 2023-11-15T03:11:09.284 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:11:09.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.440+0000 7f7e40d69700 1 Processor -- start 2023-11-15T03:11:09.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.440+0000 7f7e40d69700 1 -- start start 2023-11-15T03:11:09.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.444+0000 7f7e40d69700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7e3c1516a0 0x7f7e3c066420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:09.451 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.444+0000 7f7e40d69700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e3c065f90 con 0x7f7e3c062190 2023-11-15T03:11:09.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.444+0000 7f7e40d69700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e3c147ea0 con 0x7f7e3c0a8de0 2023-11-15T03:11:09.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.444+0000 7f7e40d69700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3c151e30 con 0x7f7e3c1516a0 2023-11-15T03:11:09.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.444+0000 7f7e3ad9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f7e3c062190 0x7f7e3c152080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49030/0 (socket says 172.21.15.121:49030) 2023-11-15T03:11:09.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.444+0000 7f7e3ad9d700 1 -- 172.21.15.121:0/620677164 learned_addr learned my addr 172.21.15.121:0/620677164 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:09.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.444+0000 7f7e39d9b700 1 --2- 172.21.15.121:0/620677164 >> v2:172.21.15.121:3300/0 conn(0x7f7e3c1516a0 0x7f7e3c066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:09.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.444+0000 7f7e3959a700 1 -- 172.21.15.121:0/620677164 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71303242 0 0) 0x7f7e3c065f90 con 0x7f7e3c062190 2023-11-15T03:11:09.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.444+0000 7f7e3959a700 1 -- 172.21.15.121:0/620677164 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e240033f0 con 0x7f7e3c062190 2023-11-15T03:11:09.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.444+0000 7f7e39d9b700 1 -- 172.21.15.121:0/620677164 >> v1:172.21.15.190:6789/0 conn(0x7f7e3c0a8de0 legacy=0x7f7e3c150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:09.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.444+0000 7f7e39d9b700 1 -- 172.21.15.121:0/620677164 >> v1:172.21.15.121:6789/0 conn(0x7f7e3c062190 legacy=0x7f7e3c152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:09.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.444+0000 7f7e39d9b700 1 -- 172.21.15.121:0/620677164 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c0671f0 con 0x7f7e3c1516a0 2023-11-15T03:11:09.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.444+0000 7f7e39d9b700 1 --2- 172.21.15.121:0/620677164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e3c1516a0 0x7f7e3c066420 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 rx=0x7f7e30009760 tx=0x7f7e30009060).ready entity=mon.0 client_cookie=4c57bfbef881164 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:09.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.444+0000 7f7e3959a700 1 -- 172.21.15.121:0/620677164 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3000f510 con 0x7f7e3c1516a0 2023-11-15T03:11:09.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.444+0000 7f7e3959a700 1 -- 172.21.15.121:0/620677164 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3000f670 con 0x7f7e3c1516a0 2023-11-15T03:11:09.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.444+0000 7f7e3959a700 1 -- 172.21.15.121:0/620677164 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3000f9f0 con 0x7f7e3c1516a0 2023-11-15T03:11:09.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.448+0000 7f7e40d69700 1 -- 172.21.15.121:0/620677164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e3c1516a0 msgr2=0x7f7e3c066420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:09.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.448+0000 7f7e40d69700 1 --2- 172.21.15.121:0/620677164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e3c1516a0 0x7f7e3c066420 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 rx=0x7f7e30009760 tx=0x7f7e30009060).stop 2023-11-15T03:11:09.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.448+0000 7f7e40d69700 1 -- 172.21.15.121:0/620677164 shutdown_connections 2023-11-15T03:11:09.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.448+0000 7f7e40d69700 1 --2- 172.21.15.121:0/620677164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e3c1516a0 0x7f7e3c066420 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:09.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.448+0000 7f7e40d69700 1 -- 172.21.15.121:0/620677164 >> 172.21.15.121:0/620677164 conn(0x7f7e3c067a60 msgr2=0x7f7e3c067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:09.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.448+0000 7f7e40d69700 1 -- 172.21.15.121:0/620677164 shutdown_connections 2023-11-15T03:11:09.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.448+0000 7f7e40d69700 1 -- 172.21.15.121:0/620677164 wait complete. 2023-11-15T03:11:09.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.452+0000 7f7e40d69700 1 Processor -- start 2023-11-15T03:11:09.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.452+0000 7f7e40d69700 1 -- start start 2023-11-15T03:11:09.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.452+0000 7f7e40d69700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e3c062190 0x7f7e3c1641d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.452+0000 7f7e40d69700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e3c0a8de0 0x7f7e3c0c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.452+0000 7f7e40d69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c1516a0 0x7f7e3c145e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.452+0000 7f7e40d69700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0c1320 con 0x7f7e3c0a8de0 2023-11-15T03:11:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.452+0000 7f7e40d69700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c1677c0 con 0x7f7e3c062190 2023-11-15T03:11:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.452+0000 7f7e40d69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0c1e50 con 0x7f7e3c1516a0 2023-11-15T03:11:09.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.452+0000 7f7e39d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e3c0a8de0 0x7f7e3c0c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:09.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.452+0000 7f7e39d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e3c0a8de0 0x7f7e3c0c0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40044/0 (socket says 172.21.15.121:40044) 2023-11-15T03:11:09.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.452+0000 7f7e39d9b700 1 -- 172.21.15.121:0/3710237908 learned_addr learned my addr 172.21.15.121:0/3710237908 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:09.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.452+0000 7f7e3ad9d700 1 --2- 172.21.15.121:0/3710237908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c1516a0 0x7f7e3c145e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:09.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.452+0000 7f7e3a59c700 1 --2- 172.21.15.121:0/3710237908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e3c062190 0x7f7e3c1641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:09.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.452+0000 7f7e39d9b700 1 -- 172.21.15.121:0/3710237908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c1516a0 msgr2=0x7f7e3c145e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:09.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.452+0000 7f7e39d9b700 1 --2- 172.21.15.121:0/3710237908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c1516a0 0x7f7e3c145e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:09.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.452+0000 7f7e39d9b700 1 -- 172.21.15.121:0/3710237908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e3c062190 msgr2=0x7f7e3c1641d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:09.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.452+0000 7f7e39d9b700 1 --2- 172.21.15.121:0/3710237908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e3c062190 0x7f7e3c1641d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:09.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.452+0000 7f7e39d9b700 1 -- 172.21.15.121:0/3710237908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e300092a0 con 0x7f7e3c0a8de0 2023-11-15T03:11:09.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.452+0000 7f7e39d9b700 1 --2- 172.21.15.121:0/3710237908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e3c0a8de0 0x7f7e3c0c0e30 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 rx=0x7f7e3c063db0 tx=0x7f7e30017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:09.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.456+0000 7f7e2a7fc700 1 -- 172.21.15.121:0/3710237908 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3000f7e0 con 0x7f7e3c0a8de0 2023-11-15T03:11:09.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.456+0000 7f7e2a7fc700 1 -- 172.21.15.121:0/3710237908 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3000f940 con 0x7f7e3c0a8de0 2023-11-15T03:11:09.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.456+0000 7f7e40d69700 1 -- 172.21.15.121:0/3710237908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3c164780 con 0x7f7e3c0a8de0 2023-11-15T03:11:09.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.456+0000 7f7e2a7fc700 1 -- 172.21.15.121:0/3710237908 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3001b690 con 0x7f7e3c0a8de0 2023-11-15T03:11:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.456+0000 7f7e40d69700 1 -- 172.21.15.121:0/3710237908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3c164d40 con 0x7f7e3c0a8de0 2023-11-15T03:11:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.456+0000 7f7e40d69700 1 -- 172.21.15.121:0/3710237908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3c167970 con 0x7f7e3c0a8de0 2023-11-15T03:11:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.456+0000 7f7e2a7fc700 1 -- 172.21.15.121:0/3710237908 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7e3001ba00 con 0x7f7e3c0a8de0 2023-11-15T03:11:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.460+0000 7f7e2a7fc700 1 --2- 172.21.15.121:0/3710237908 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7e1c05a510 0x7f7e1c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.460+0000 7f7e2a7fc700 1 -- 172.21.15.121:0/3710237908 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9706+0+0 (secure 0 0 0) 0x7f7e30076c90 con 0x7f7e3c0a8de0 2023-11-15T03:11:09.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.460+0000 7f7e3a59c700 1 --2- 172.21.15.121:0/3710237908 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7e1c05a510 0x7f7e1c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:09.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.460+0000 7f7e3a59c700 1 --2- 172.21.15.121:0/3710237908 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7e1c05a510 0x7f7e1c05c9d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f7e2c0036d0 tx=0x7f7e2c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:09.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.464+0000 7f7e2a7fc700 1 -- 172.21.15.121:0/3710237908 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7e300496b0 con 0x7f7e3c0a8de0 2023-11-15T03:11:09.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:09.660+0000 7f7e40d69700 1 -- 172.21.15.121:0/3710237908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7e3c047060 con 0x7f7e3c0a8de0 2023-11-15T03:11:10.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.320+0000 7f7e2a7fc700 1 -- 172.21.15.121:0/3710237908 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v343) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7e3001bcb0 con 0x7f7e3c0a8de0 2023-11-15T03:11:10.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.324+0000 7f7e40d69700 1 -- 172.21.15.121:0/3710237908 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7e1c05a510 msgr2=0x7f7e1c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:10.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.324+0000 7f7e40d69700 1 --2- 172.21.15.121:0/3710237908 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7e1c05a510 0x7f7e1c05c9d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f7e2c0036d0 tx=0x7f7e2c009040).stop 2023-11-15T03:11:10.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.324+0000 7f7e40d69700 1 -- 172.21.15.121:0/3710237908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e3c0a8de0 msgr2=0x7f7e3c0c0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:10.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.324+0000 7f7e40d69700 1 --2- 172.21.15.121:0/3710237908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e3c0a8de0 0x7f7e3c0c0e30 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 rx=0x7f7e3c063db0 tx=0x7f7e30017000).stop 2023-11-15T03:11:10.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.324+0000 7f7e40d69700 1 -- 172.21.15.121:0/3710237908 shutdown_connections 2023-11-15T03:11:10.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.324+0000 7f7e40d69700 1 --2- 172.21.15.121:0/3710237908 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7e1c05a510 0x7f7e1c05c9d0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:10.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.324+0000 7f7e40d69700 1 --2- 172.21.15.121:0/3710237908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c1516a0 0x7f7e3c145e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:10.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.324+0000 7f7e40d69700 1 --2- 172.21.15.121:0/3710237908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e3c0a8de0 0x7f7e3c0c0e30 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:10.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.324+0000 7f7e40d69700 1 --2- 172.21.15.121:0/3710237908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e3c062190 0x7f7e3c1641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:10.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.324+0000 7f7e40d69700 1 -- 172.21.15.121:0/3710237908 >> 172.21.15.121:0/3710237908 conn(0x7f7e3c067a60 msgr2=0x7f7e3c14e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:10.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.324+0000 7f7e40d69700 1 -- 172.21.15.121:0/3710237908 shutdown_connections 2023-11-15T03:11:10.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.324+0000 7f7e40d69700 1 -- 172.21.15.121:0/3710237908 wait complete. 2023-11-15T03:11:10.337 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:11:10.358 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-15T03:11:10.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.520+0000 7f53df59e700 1 Processor -- start 2023-11-15T03:11:10.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.520+0000 7f53df59e700 1 -- start start 2023-11-15T03:11:10.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.524+0000 7f53df59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f53e00a8960 0x7f53e00c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:10.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.524+0000 7f53df59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e00a8d40 0x7f53e015c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:10.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.524+0000 7f53df59e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e00657f0 con 0x7f53e0066520 2023-11-15T03:11:10.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.524+0000 7f53df59e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e01583d0 con 0x7f53e00a8960 2023-11-15T03:11:10.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.524+0000 7f53df59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00c1b60 con 0x7f53e00a8d40 2023-11-15T03:11:10.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.524+0000 7f53de59c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f53e00a8960 0x7f53e00c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:10.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.524+0000 7f53de59c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f53e00a8960 0x7f53e00c15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:41478/0 (socket says 172.21.15.121:41478) 2023-11-15T03:11:10.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.524+0000 7f53de59c700 1 -- 172.21.15.121:0/1236224558 learned_addr learned my addr 172.21.15.121:0/1236224558 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:10.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.524+0000 7f53ddd9b700 1 --2- 172.21.15.121:0/1236224558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e00a8d40 0x7f53e015c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:10.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.524+0000 7f53de59c700 1 -- 172.21.15.121:0/1236224558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e00a8d40 msgr2=0x7f53e015c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:10.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.524+0000 7f53de59c700 1 --2- 172.21.15.121:0/1236224558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e00a8d40 0x7f53e015c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:10.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.524+0000 7f53de59c700 1 -- 172.21.15.121:0/1236224558 >> v1:172.21.15.190:6789/0 conn(0x7f53e0066520 legacy=0x7f53e00c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:10.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.524+0000 7f53de59c700 1 -- 172.21.15.121:0/1236224558 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e0065a40 con 0x7f53e00a8960 2023-11-15T03:11:10.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.524+0000 7f53de59c700 1 --2- 172.21.15.121:0/1236224558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e00a8960 0x7f53e00c15c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f53d0006fd0 tx=0x7f53d00090f0).ready entity=mon.1 client_cookie=4d3034337aa6c729 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:10.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.524+0000 7f53dd59a700 1 -- 172.21.15.121:0/1236224558 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d000f510 con 0x7f53e00a8960 2023-11-15T03:11:10.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.524+0000 7f53dd59a700 1 -- 172.21.15.121:0/1236224558 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d000f670 con 0x7f53e00a8960 2023-11-15T03:11:10.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.524+0000 7f53dd59a700 1 -- 172.21.15.121:0/1236224558 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d000fa10 con 0x7f53e00a8960 2023-11-15T03:11:10.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.528+0000 7f53df59e700 1 -- 172.21.15.121:0/1236224558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e00a8960 msgr2=0x7f53e00c15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:10.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.528+0000 7f53df59e700 1 --2- 172.21.15.121:0/1236224558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e00a8960 0x7f53e00c15c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f53d0006fd0 tx=0x7f53d00090f0).stop 2023-11-15T03:11:10.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.528+0000 7f53df59e700 1 -- 172.21.15.121:0/1236224558 shutdown_connections 2023-11-15T03:11:10.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.528+0000 7f53df59e700 1 --2- 172.21.15.121:0/1236224558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e00a8d40 0x7f53e015c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:10.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.528+0000 7f53df59e700 1 --2- 172.21.15.121:0/1236224558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e00a8960 0x7f53e00c15c0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:10.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.528+0000 7f53df59e700 1 -- 172.21.15.121:0/1236224558 >> 172.21.15.121:0/1236224558 conn(0x7f53e0067a60 msgr2=0x7f53e0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:10.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.528+0000 7f53df59e700 1 -- 172.21.15.121:0/1236224558 shutdown_connections 2023-11-15T03:11:10.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.528+0000 7f53df59e700 1 -- 172.21.15.121:0/1236224558 wait complete. 2023-11-15T03:11:10.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.528+0000 7f53df59e700 1 Processor -- start 2023-11-15T03:11:10.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.528+0000 7f53df59e700 1 -- start start 2023-11-15T03:11:10.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53df59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e0066520 0x7f53e0147890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:10.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53df59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e00a8960 0x7f53e00ab220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:10.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53df59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53e00a8d40 0x7f53e00ab710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:10.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53df59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e0147d80 con 0x7f53e00a8d40 2023-11-15T03:11:10.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53df59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00abc00 con 0x7f53e0066520 2023-11-15T03:11:10.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53df59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00c55f0 con 0x7f53e00a8960 2023-11-15T03:11:10.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53ded9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53e00a8d40 0x7f53e00ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:10.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53ded9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53e00a8d40 0x7f53e00ab710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40056/0 (socket says 172.21.15.121:40056) 2023-11-15T03:11:10.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53ded9d700 1 -- 172.21.15.121:0/2547316282 learned_addr learned my addr 172.21.15.121:0/2547316282 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:10.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53ddd9b700 1 --2- 172.21.15.121:0/2547316282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e00a8960 0x7f53e00ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:10.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53de59c700 1 --2- 172.21.15.121:0/2547316282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e0066520 0x7f53e0147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:10.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53ded9d700 1 -- 172.21.15.121:0/2547316282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e00a8960 msgr2=0x7f53e00ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:10.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53ded9d700 1 --2- 172.21.15.121:0/2547316282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e00a8960 0x7f53e00ab220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:10.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53ded9d700 1 -- 172.21.15.121:0/2547316282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e0066520 msgr2=0x7f53e0147890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:10.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53ded9d700 1 --2- 172.21.15.121:0/2547316282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e0066520 0x7f53e0147890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:10.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53ded9d700 1 -- 172.21.15.121:0/2547316282 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d0009370 con 0x7f53e00a8d40 2023-11-15T03:11:10.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53ded9d700 1 --2- 172.21.15.121:0/2547316282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53e00a8d40 0x7f53e00ab710 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 rx=0x7f53d800c890 tx=0x7f53d8001610).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:10.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53ce7fc700 1 -- 172.21.15.121:0/2547316282 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8010510 con 0x7f53e00a8d40 2023-11-15T03:11:10.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53ce7fc700 1 -- 172.21.15.121:0/2547316282 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d8010690 con 0x7f53e00a8d40 2023-11-15T03:11:10.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53df59e700 1 -- 172.21.15.121:0/2547316282 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e0169100 con 0x7f53e00a8d40 2023-11-15T03:11:10.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53ce7fc700 1 -- 172.21.15.121:0/2547316282 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8010a50 con 0x7f53e00a8d40 2023-11-15T03:11:10.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.532+0000 7f53df59e700 1 -- 172.21.15.121:0/2547316282 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e00c5b20 con 0x7f53e00a8d40 2023-11-15T03:11:10.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.536+0000 7f53df59e700 1 -- 172.21.15.121:0/2547316282 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e006bcf0 con 0x7f53e00a8d40 2023-11-15T03:11:10.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.536+0000 7f53ce7fc700 1 -- 172.21.15.121:0/2547316282 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f53d801b400 con 0x7f53e00a8d40 2023-11-15T03:11:10.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.540+0000 7f53ce7fc700 1 --2- 172.21.15.121:0/2547316282 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f53c005a540 0x7f53c005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:10.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.540+0000 7f53ce7fc700 1 -- 172.21.15.121:0/2547316282 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9706+0+0 (secure 0 0 0) 0x7f53d8075450 con 0x7f53e00a8d40 2023-11-15T03:11:10.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.540+0000 7f53de59c700 1 --2- 172.21.15.121:0/2547316282 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f53c005a540 0x7f53c005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:10.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.540+0000 7f53de59c700 1 --2- 172.21.15.121:0/2547316282 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f53c005a540 0x7f53c005ca00 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f53d000a010 tx=0x7f53d0017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:10.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.544+0000 7f53ce7fc700 1 -- 172.21.15.121:0/2547316282 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f53d8047d10 con 0x7f53e00a8d40 2023-11-15T03:11:10.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:10.744+0000 7f53df59e700 1 -- 172.21.15.121:0/2547316282 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f53e0068a70 con 0x7f53e00a8d40 2023-11-15T03:11:11.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.328+0000 7f53ce7fc700 1 -- 172.21.15.121:0/2547316282 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v344) v1 ==== 104+0+0 (secure 0 0 0) 0x7f53d8001730 con 0x7f53e00a8d40 2023-11-15T03:11:11.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.332+0000 7f53df59e700 1 -- 172.21.15.121:0/2547316282 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f53c005a540 msgr2=0x7f53c005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:11.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.332+0000 7f53df59e700 1 --2- 172.21.15.121:0/2547316282 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f53c005a540 0x7f53c005ca00 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f53d000a010 tx=0x7f53d0017000).stop 2023-11-15T03:11:11.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.332+0000 7f53df59e700 1 -- 172.21.15.121:0/2547316282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53e00a8d40 msgr2=0x7f53e00ab710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:11.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.332+0000 7f53df59e700 1 --2- 172.21.15.121:0/2547316282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53e00a8d40 0x7f53e00ab710 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 rx=0x7f53d800c890 tx=0x7f53d8001610).stop 2023-11-15T03:11:11.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.332+0000 7f53df59e700 1 -- 172.21.15.121:0/2547316282 shutdown_connections 2023-11-15T03:11:11.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.332+0000 7f53df59e700 1 --2- 172.21.15.121:0/2547316282 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f53c005a540 0x7f53c005ca00 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:11.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.332+0000 7f53df59e700 1 --2- 172.21.15.121:0/2547316282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53e00a8d40 0x7f53e00ab710 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:11.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.332+0000 7f53df59e700 1 --2- 172.21.15.121:0/2547316282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e00a8960 0x7f53e00ab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:11.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.332+0000 7f53df59e700 1 --2- 172.21.15.121:0/2547316282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e0066520 0x7f53e0147890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:11.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.332+0000 7f53df59e700 1 -- 172.21.15.121:0/2547316282 >> 172.21.15.121:0/2547316282 conn(0x7f53e0067a60 msgr2=0x7f53e00aa530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:11.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.332+0000 7f53df59e700 1 -- 172.21.15.121:0/2547316282 shutdown_connections 2023-11-15T03:11:11.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.332+0000 7f53df59e700 1 -- 172.21.15.121:0/2547316282 wait complete. 2023-11-15T03:11:11.365 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-15T03:11:11.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.532+0000 7f1641b07700 1 Processor -- start 2023-11-15T03:11:11.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.532+0000 7f1641b07700 1 -- start start 2023-11-15T03:11:11.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.536+0000 7f1641b07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c153a60 0x7f163c153e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:11.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.536+0000 7f1641b07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f163c0bfe40 0x7f163c0c02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:11.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.536+0000 7f1641b07700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f163c0660a0 con 0x7f163c0c5d10 2023-11-15T03:11:11.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.536+0000 7f1641b07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f163c0c5500 con 0x7f163c0bfe40 2023-11-15T03:11:11.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.536+0000 7f1641b07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163c066c40 con 0x7f163c153a60 2023-11-15T03:11:11.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.536+0000 7f163bfff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f163c0c5d10 0x7f163c066530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49042/0 (socket says 172.21.15.121:49042) 2023-11-15T03:11:11.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.536+0000 7f163bfff700 1 -- 172.21.15.121:0/3173779073 learned_addr learned my addr 172.21.15.121:0/3173779073 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:11.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.536+0000 7f1640b05700 1 --2- 172.21.15.121:0/3173779073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c153a60 0x7f163c153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:11.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.536+0000 7f1641306700 1 --2- 172.21.15.121:0/3173779073 >> v2:172.21.15.190:3300/0 conn(0x7f163c0bfe40 0x7f163c0c02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:11.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.536+0000 7f163b7fe700 1 -- 172.21.15.121:0/3173779073 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2869220239 0 0) 0x7f163c0660a0 con 0x7f163c0c5d10 2023-11-15T03:11:11.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.536+0000 7f163b7fe700 1 -- 172.21.15.121:0/3173779073 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f162c0033f0 con 0x7f163c0c5d10 2023-11-15T03:11:11.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.536+0000 7f1640b05700 1 -- 172.21.15.121:0/3173779073 >> v2:172.21.15.190:3300/0 conn(0x7f163c0bfe40 msgr2=0x7f163c0c02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:11.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.536+0000 7f1640b05700 1 --2- 172.21.15.121:0/3173779073 >> v2:172.21.15.190:3300/0 conn(0x7f163c0bfe40 0x7f163c0c02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:11.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.536+0000 7f1640b05700 1 -- 172.21.15.121:0/3173779073 >> v1:172.21.15.121:6789/0 conn(0x7f163c0c5d10 legacy=0x7f163c066530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:11.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.536+0000 7f1640b05700 1 -- 172.21.15.121:0/3173779073 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c0657b0 con 0x7f163c153a60 2023-11-15T03:11:11.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.536+0000 7f1640b05700 1 --2- 172.21.15.121:0/3173779073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c153a60 0x7f163c153e40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f1630009760 tx=0x7f1630009060).ready entity=mon.2 client_cookie=b65302f1ae83782a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:11.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.536+0000 7f163b7fe700 1 -- 172.21.15.121:0/3173779073 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163000d4a0 con 0x7f163c153a60 2023-11-15T03:11:11.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.540+0000 7f163b7fe700 1 -- 172.21.15.121:0/3173779073 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163000d600 con 0x7f163c153a60 2023-11-15T03:11:11.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.540+0000 7f163b7fe700 1 -- 172.21.15.121:0/3173779073 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163000d980 con 0x7f163c153a60 2023-11-15T03:11:11.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.540+0000 7f1641b07700 1 -- 172.21.15.121:0/3173779073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c153a60 msgr2=0x7f163c153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:11.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.540+0000 7f1641b07700 1 --2- 172.21.15.121:0/3173779073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c153a60 0x7f163c153e40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f1630009760 tx=0x7f1630009060).stop 2023-11-15T03:11:11.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.540+0000 7f1641b07700 1 -- 172.21.15.121:0/3173779073 shutdown_connections 2023-11-15T03:11:11.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.540+0000 7f1641b07700 1 --2- 172.21.15.121:0/3173779073 >> v2:172.21.15.190:3300/0 conn(0x7f163c0bfe40 0x7f163c0c02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:11.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.540+0000 7f1641b07700 1 --2- 172.21.15.121:0/3173779073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c153a60 0x7f163c153e40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:11.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.540+0000 7f1641b07700 1 -- 172.21.15.121:0/3173779073 >> 172.21.15.121:0/3173779073 conn(0x7f163c067a60 msgr2=0x7f163c067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:11.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.540+0000 7f1641b07700 1 -- 172.21.15.121:0/3173779073 shutdown_connections 2023-11-15T03:11:11.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.540+0000 7f1641b07700 1 -- 172.21.15.121:0/3173779073 wait complete. 2023-11-15T03:11:11.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.540+0000 7f1641b07700 1 Processor -- start 2023-11-15T03:11:11.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.540+0000 7f1641b07700 1 -- start start 2023-11-15T03:11:11.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f1641b07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c0bfe40 0x7f163c0c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:11.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f1641b07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f163c0c5d10 0x7f163c14a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:11.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f1641b07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f163c153a60 0x7f163c0c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:11.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f1641b07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c14ab80 con 0x7f163c0c5d10 2023-11-15T03:11:11.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f1641b07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c0c3100 con 0x7f163c153a60 2023-11-15T03:11:11.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f1641b07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163c0c2050 con 0x7f163c0bfe40 2023-11-15T03:11:11.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f163bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f163c0c5d10 0x7f163c14a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:11.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f163bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f163c0c5d10 0x7f163c14a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40058/0 (socket says 172.21.15.121:40058) 2023-11-15T03:11:11.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f163bfff700 1 -- 172.21.15.121:0/1905369446 learned_addr learned my addr 172.21.15.121:0/1905369446 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:11.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f1640b05700 1 --2- 172.21.15.121:0/1905369446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c0bfe40 0x7f163c0c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:11.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f1641306700 1 --2- 172.21.15.121:0/1905369446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f163c153a60 0x7f163c0c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:11.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f163bfff700 1 -- 172.21.15.121:0/1905369446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c0bfe40 msgr2=0x7f163c0c24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:11.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f163bfff700 1 --2- 172.21.15.121:0/1905369446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c0bfe40 0x7f163c0c24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:11.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f163bfff700 1 -- 172.21.15.121:0/1905369446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f163c153a60 msgr2=0x7f163c0c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:11.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f163bfff700 1 --2- 172.21.15.121:0/1905369446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f163c153a60 0x7f163c0c2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:11.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f163bfff700 1 -- 172.21.15.121:0/1905369446 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16300092a0 con 0x7f163c0c5d10 2023-11-15T03:11:11.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f163bfff700 1 --2- 172.21.15.121:0/1905369446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f163c0c5d10 0x7f163c14a690 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 rx=0x7f162800c4f0 tx=0x7f1628007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:11.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f1638ff9700 1 -- 172.21.15.121:0/1905369446 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16280072e0 con 0x7f163c0c5d10 2023-11-15T03:11:11.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f1638ff9700 1 -- 172.21.15.121:0/1905369446 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1628007440 con 0x7f163c0c5d10 2023-11-15T03:11:11.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f1641b07700 1 -- 172.21.15.121:0/1905369446 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163c14ad70 con 0x7f163c0c5d10 2023-11-15T03:11:11.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f1638ff9700 1 -- 172.21.15.121:0/1905369446 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16280057a0 con 0x7f163c0c5d10 2023-11-15T03:11:11.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.544+0000 7f1641b07700 1 -- 172.21.15.121:0/1905369446 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163c0c1a00 con 0x7f163c0c5d10 2023-11-15T03:11:11.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.548+0000 7f1641b07700 1 -- 172.21.15.121:0/1905369446 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163c06bcf0 con 0x7f163c0c5d10 2023-11-15T03:11:11.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.548+0000 7f1638ff9700 1 -- 172.21.15.121:0/1905369446 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1628005bd0 con 0x7f163c0c5d10 2023-11-15T03:11:11.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.548+0000 7f1638ff9700 1 --2- 172.21.15.121:0/1905369446 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f162005a540 0x7f162005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:11.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.548+0000 7f1638ff9700 1 -- 172.21.15.121:0/1905369446 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9706+0+0 (secure 0 0 0) 0x7f1628074e50 con 0x7f163c0c5d10 2023-11-15T03:11:11.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.548+0000 7f1640b05700 1 --2- 172.21.15.121:0/1905369446 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f162005a540 0x7f162005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:11.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.548+0000 7f1640b05700 1 --2- 172.21.15.121:0/1905369446 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f162005a540 0x7f162005ca00 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f163000d260 tx=0x7f1630016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:11.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.556+0000 7f1638ff9700 1 -- 172.21.15.121:0/1905369446 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1628047770 con 0x7f163c0c5d10 2023-11-15T03:11:11.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:11.724+0000 7f1641b07700 1 -- 172.21.15.121:0/1905369446 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f163c065a50 con 0x7f163c0c5d10 2023-11-15T03:11:12.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.352+0000 7f1638ff9700 1 -- 172.21.15.121:0/1905369446 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v135) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1628018020 con 0x7f163c0c5d10 2023-11-15T03:11:12.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.360+0000 7f1641b07700 1 -- 172.21.15.121:0/1905369446 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f162005a540 msgr2=0x7f162005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:12.368 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.360+0000 7f1641b07700 1 --2- 172.21.15.121:0/1905369446 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f162005a540 0x7f162005ca00 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f163000d260 tx=0x7f1630016000).stop 2023-11-15T03:11:12.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.360+0000 7f1641b07700 1 -- 172.21.15.121:0/1905369446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f163c0c5d10 msgr2=0x7f163c14a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:12.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.360+0000 7f1641b07700 1 --2- 172.21.15.121:0/1905369446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f163c0c5d10 0x7f163c14a690 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 rx=0x7f162800c4f0 tx=0x7f1628007c50).stop 2023-11-15T03:11:12.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.360+0000 7f1641b07700 1 -- 172.21.15.121:0/1905369446 shutdown_connections 2023-11-15T03:11:12.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.360+0000 7f1641b07700 1 --2- 172.21.15.121:0/1905369446 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f162005a540 0x7f162005ca00 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:12.369 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.360+0000 7f1641b07700 1 --2- 172.21.15.121:0/1905369446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f163c153a60 0x7f163c0c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:12.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.360+0000 7f1641b07700 1 --2- 172.21.15.121:0/1905369446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f163c0c5d10 0x7f163c14a690 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:12.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.360+0000 7f1641b07700 1 --2- 172.21.15.121:0/1905369446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c0bfe40 0x7f163c0c24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:12.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.360+0000 7f1641b07700 1 -- 172.21.15.121:0/1905369446 >> 172.21.15.121:0/1905369446 conn(0x7f163c067a60 msgr2=0x7f163c1596e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:12.370 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.360+0000 7f1641b07700 1 -- 172.21.15.121:0/1905369446 shutdown_connections 2023-11-15T03:11:12.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.360+0000 7f1641b07700 1 -- 172.21.15.121:0/1905369446 wait complete. 2023-11-15T03:11:12.371 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_metadata' removed 2023-11-15T03:11:12.389 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-15T03:11:12.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.548+0000 7f87c58f1700 1 Processor -- start 2023-11-15T03:11:12.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.548+0000 7f87c58f1700 1 -- start start 2023-11-15T03:11:12.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87c58f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87c0139270 0x7f87c0153620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:12.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87c58f1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c0062bd0 con 0x7f87c0153b10 2023-11-15T03:11:12.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87c58f1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c0154550 con 0x7f87c00c1040 2023-11-15T03:11:12.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87c58f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c0063170 con 0x7f87c0139270 2023-11-15T03:11:12.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87be7fc700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f87c0153b10 0x7f87c0068170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49054/0 (socket says 172.21.15.121:49054) 2023-11-15T03:11:12.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87be7fc700 1 -- 172.21.15.121:0/1121623226 learned_addr learned my addr 172.21.15.121:0/1121623226 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:12.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87beffd700 1 --2- 172.21.15.121:0/1121623226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87c0139270 0x7f87c0153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:12.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87bdffb700 1 -- 172.21.15.121:0/1121623226 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 380848878 0 0) 0x7f87c0062bd0 con 0x7f87c0153b10 2023-11-15T03:11:12.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87bdffb700 1 -- 172.21.15.121:0/1121623226 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87a80033f0 con 0x7f87c0153b10 2023-11-15T03:11:12.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87bdffb700 1 -- 172.21.15.121:0/1121623226 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016737785 0 0) 0x7f87c0154550 con 0x7f87c00c1040 2023-11-15T03:11:12.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87bdffb700 1 -- 172.21.15.121:0/1121623226 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87c0062bd0 con 0x7f87c00c1040 2023-11-15T03:11:12.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87bdffb700 1 -- 172.21.15.121:0/1121623226 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1634908409 0 0) 0x7f87a80033f0 con 0x7f87c0153b10 2023-11-15T03:11:12.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87bdffb700 1 -- 172.21.15.121:0/1121623226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87c0139270 msgr2=0x7f87c0153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:12.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87bdffb700 1 --2- 172.21.15.121:0/1121623226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87c0139270 0x7f87c0153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:12.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87bdffb700 1 -- 172.21.15.121:0/1121623226 >> v1:172.21.15.190:6789/0 conn(0x7f87c00c1040 legacy=0x7f87c0150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:12.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87bdffb700 1 -- 172.21.15.121:0/1121623226 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c0153f90 con 0x7f87c0153b10 2023-11-15T03:11:12.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87bdffb700 1 -- 172.21.15.121:0/1121623226 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f87b4001470 con 0x7f87c0153b10 2023-11-15T03:11:12.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87bdffb700 1 -- 172.21.15.121:0/1121623226 >> v1:172.21.15.121:6789/0 conn(0x7f87c0153b10 legacy=0x7f87c0068170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:12.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87bdffb700 1 --2- 172.21.15.121:0/1121623226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87a8003aa0 0x7f87a8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:12.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87bdffb700 1 --2- 172.21.15.121:0/1121623226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87a8004690 0x7f87a8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:12.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87bdffb700 1 --2- 172.21.15.121:0/1121623226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87a8005000 0x7f87a800c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:12.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87bdffb700 1 -- 172.21.15.121:0/1121623226 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c0063170 con 0x7f87a8004690 2023-11-15T03:11:12.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87bdffb700 1 -- 172.21.15.121:0/1121623226 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a800cf90 con 0x7f87a8005000 2023-11-15T03:11:12.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87bdffb700 1 -- 172.21.15.121:0/1121623226 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a800d150 con 0x7f87a8003aa0 2023-11-15T03:11:12.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87be7fc700 1 --2- 172.21.15.121:0/1121623226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87a8004690 0x7f87a8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:12.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87beffd700 1 --2- 172.21.15.121:0/1121623226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87a8003aa0 0x7f87a8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:12.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.552+0000 7f87bf7fe700 1 --2- 172.21.15.121:0/1121623226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87a8005000 0x7f87a800c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:12.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.556+0000 7f87be7fc700 1 -- 172.21.15.121:0/1121623226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87a8003aa0 msgr2=0x7f87a8003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:12.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.556+0000 7f87be7fc700 1 --2- 172.21.15.121:0/1121623226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87a8003aa0 0x7f87a8003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:12.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.556+0000 7f87be7fc700 1 -- 172.21.15.121:0/1121623226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87a8005000 msgr2=0x7f87a800c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:12.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.556+0000 7f87be7fc700 1 --2- 172.21.15.121:0/1121623226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87a8005000 0x7f87a800c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:12.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.556+0000 7f87be7fc700 1 -- 172.21.15.121:0/1121623226 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a80033f0 con 0x7f87a8004690 2023-11-15T03:11:12.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.556+0000 7f87beffd700 1 --2- 172.21.15.121:0/1121623226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87a8003aa0 0x7f87a8003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:12.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.556+0000 7f87be7fc700 1 --2- 172.21.15.121:0/1121623226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87a8004690 0x7f87a8004b10 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 rx=0x7f87c0065250 tx=0x7f87b4001e30).ready entity=mon.0 client_cookie=666583d68bd090c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:12.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.556+0000 7f87bdffb700 1 -- 172.21.15.121:0/1121623226 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b400c9f0 con 0x7f87a8004690 2023-11-15T03:11:12.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.556+0000 7f87bdffb700 1 -- 172.21.15.121:0/1121623226 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b400cb50 con 0x7f87a8004690 2023-11-15T03:11:12.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.556+0000 7f87bdffb700 1 -- 172.21.15.121:0/1121623226 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b40070d0 con 0x7f87a8004690 2023-11-15T03:11:12.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.556+0000 7f87c58f1700 1 -- 172.21.15.121:0/1121623226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87a8004690 msgr2=0x7f87a8004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:12.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.556+0000 7f87c58f1700 1 --2- 172.21.15.121:0/1121623226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87a8004690 0x7f87a8004b10 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 rx=0x7f87c0065250 tx=0x7f87b4001e30).stop 2023-11-15T03:11:12.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.556+0000 7f87bf7fe700 1 -- 172.21.15.121:0/1121623226 reap_dead start 2023-11-15T03:11:12.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.556+0000 7f87c58f1700 1 -- 172.21.15.121:0/1121623226 shutdown_connections 2023-11-15T03:11:12.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.556+0000 7f87c58f1700 1 -- 172.21.15.121:0/1121623226 >> 172.21.15.121:0/1121623226 conn(0x7f87c00c3e80 msgr2=0x7f87c0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:12.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.556+0000 7f87c58f1700 1 -- 172.21.15.121:0/1121623226 shutdown_connections 2023-11-15T03:11:12.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.556+0000 7f87c58f1700 1 -- 172.21.15.121:0/1121623226 wait complete. 2023-11-15T03:11:12.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87c58f1700 1 Processor -- start 2023-11-15T03:11:12.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87c58f1700 1 -- start start 2023-11-15T03:11:12.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87c58f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87a8004690 0x7f87c0152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:12.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87c58f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87c0164270 0x7f87c00bda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:12.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87c58f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87c0136330 0x7f87c00a7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:12.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87c58f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c00ab650 con 0x7f87a8004690 2023-11-15T03:11:12.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87c58f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c01646b0 con 0x7f87c0164270 2023-11-15T03:11:12.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87c58f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c00bdef0 con 0x7f87c0136330 2023-11-15T03:11:12.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87beffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87a8004690 0x7f87c0152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:12.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87beffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87a8004690 0x7f87c0152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40080/0 (socket says 172.21.15.121:40080) 2023-11-15T03:11:12.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87beffd700 1 -- 172.21.15.121:0/2589162535 learned_addr learned my addr 172.21.15.121:0/2589162535 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:12.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87be7fc700 1 --2- 172.21.15.121:0/2589162535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87c0164270 0x7f87c00bda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:12.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87bf7fe700 1 --2- 172.21.15.121:0/2589162535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87c0136330 0x7f87c00a7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:12.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87beffd700 1 -- 172.21.15.121:0/2589162535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87c0136330 msgr2=0x7f87c00a7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:12.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87beffd700 1 --2- 172.21.15.121:0/2589162535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87c0136330 0x7f87c00a7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:12.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87beffd700 1 -- 172.21.15.121:0/2589162535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87c0164270 msgr2=0x7f87c00bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:12.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87beffd700 1 --2- 172.21.15.121:0/2589162535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87c0164270 0x7f87c00bda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:12.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87beffd700 1 -- 172.21.15.121:0/2589162535 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c00aaa40 con 0x7f87a8004690 2023-11-15T03:11:12.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87beffd700 1 --2- 172.21.15.121:0/2589162535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87a8004690 0x7f87c0152640 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 rx=0x7f87b00108f0 tx=0x7f87b0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:12.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87a6ffd700 1 -- 172.21.15.121:0/2589162535 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b0014590 con 0x7f87a8004690 2023-11-15T03:11:12.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87a6ffd700 1 -- 172.21.15.121:0/2589162535 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b00146f0 con 0x7f87a8004690 2023-11-15T03:11:12.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87c58f1700 1 -- 172.21.15.121:0/2589162535 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c00aad20 con 0x7f87a8004690 2023-11-15T03:11:12.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.560+0000 7f87a6ffd700 1 -- 172.21.15.121:0/2589162535 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b0014a70 con 0x7f87a8004690 2023-11-15T03:11:12.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.564+0000 7f87c58f1700 1 -- 172.21.15.121:0/2589162535 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c00ab2e0 con 0x7f87a8004690 2023-11-15T03:11:12.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.564+0000 7f87c58f1700 1 -- 172.21.15.121:0/2589162535 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c0047060 con 0x7f87a8004690 2023-11-15T03:11:12.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.564+0000 7f87a6ffd700 1 -- 172.21.15.121:0/2589162535 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f87b001f460 con 0x7f87a8004690 2023-11-15T03:11:12.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.568+0000 7f87a6ffd700 1 --2- 172.21.15.121:0/2589162535 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f87ac05a510 0x7f87ac05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:12.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.568+0000 7f87a6ffd700 1 -- 172.21.15.121:0/2589162535 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 9239+0+0 (secure 0 0 0) 0x7f87b00754e0 con 0x7f87a8004690 2023-11-15T03:11:12.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.568+0000 7f87be7fc700 1 --2- 172.21.15.121:0/2589162535 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f87ac05a510 0x7f87ac05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:12.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.568+0000 7f87be7fc700 1 --2- 172.21.15.121:0/2589162535 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f87ac05a510 0x7f87ac05c9d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f87c0065250 tx=0x7f87b4015000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:12.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.572+0000 7f87a6ffd700 1 -- 172.21.15.121:0/2589162535 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f87b0047ff0 con 0x7f87a8004690 2023-11-15T03:11:12.597 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:11:12.606 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:11:12.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:12.744+0000 7f87c58f1700 1 -- 172.21.15.121:0/2589162535 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f87c014fe50 con 0x7f87a8004690 2023-11-15T03:11:13.371 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.364+0000 7f87a6ffd700 1 -- 172.21.15.121:0/2589162535 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v136) v1 ==== 169+0+0 (secure 0 0 0) 0x7f87b0009e60 con 0x7f87a8004690 2023-11-15T03:11:13.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.372+0000 7f87c58f1700 1 -- 172.21.15.121:0/2589162535 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f87ac05a510 msgr2=0x7f87ac05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:13.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.372+0000 7f87c58f1700 1 --2- 172.21.15.121:0/2589162535 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f87ac05a510 0x7f87ac05c9d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f87c0065250 tx=0x7f87b4015000).stop 2023-11-15T03:11:13.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.372+0000 7f87c58f1700 1 -- 172.21.15.121:0/2589162535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87a8004690 msgr2=0x7f87c0152640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:13.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.372+0000 7f87c58f1700 1 --2- 172.21.15.121:0/2589162535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87a8004690 0x7f87c0152640 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 rx=0x7f87b00108f0 tx=0x7f87b0008040).stop 2023-11-15T03:11:13.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.372+0000 7f87c58f1700 1 -- 172.21.15.121:0/2589162535 shutdown_connections 2023-11-15T03:11:13.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.372+0000 7f87c58f1700 1 --2- 172.21.15.121:0/2589162535 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f87ac05a510 0x7f87ac05c9d0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:13.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.372+0000 7f87c58f1700 1 --2- 172.21.15.121:0/2589162535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87c0136330 0x7f87c00a7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:13.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.372+0000 7f87c58f1700 1 --2- 172.21.15.121:0/2589162535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87c0164270 0x7f87c00bda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:13.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.372+0000 7f87c58f1700 1 --2- 172.21.15.121:0/2589162535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87a8004690 0x7f87c0152640 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:13.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.372+0000 7f87c58f1700 1 -- 172.21.15.121:0/2589162535 >> 172.21.15.121:0/2589162535 conn(0x7f87c00c3e80 msgr2=0x7f87c01382f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:13.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.372+0000 7f87c58f1700 1 -- 172.21.15.121:0/2589162535 shutdown_connections 2023-11-15T03:11:13.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.372+0000 7f87c58f1700 1 -- 172.21.15.121:0/2589162535 wait complete. 2023-11-15T03:11:13.383 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_data' removed 2023-11-15T03:11:13.401 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-15T03:11:13.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.544+0000 7fdee5fb9700 1 Processor -- start 2023-11-15T03:11:13.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.544+0000 7fdee5fb9700 1 -- start start 2023-11-15T03:11:13.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.548+0000 7fdee5fb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdee014ac10 0x7fdee00c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:13.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.548+0000 7fdee5fb9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee00657f0 con 0x7fdee014aff0 2023-11-15T03:11:13.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.548+0000 7fdee5fb9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee0066650 con 0x7fdee0067380 2023-11-15T03:11:13.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.548+0000 7fdee5fb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee0065a40 con 0x7fdee014ac10 2023-11-15T03:11:13.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.548+0000 7fdedffff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fdee014aff0 0x7fdee0066c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49066/0 (socket says 172.21.15.121:49066) 2023-11-15T03:11:13.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.548+0000 7fdedffff700 1 -- 172.21.15.121:0/2679004336 learned_addr learned my addr 172.21.15.121:0/2679004336 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:13.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.548+0000 7fdee4fb7700 1 --2- 172.21.15.121:0/2679004336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdee014ac10 0x7fdee00c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:13.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.548+0000 7fdedf7fe700 1 -- 172.21.15.121:0/2679004336 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009371285 0 0) 0x7fdee0066650 con 0x7fdee0067380 2023-11-15T03:11:13.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.548+0000 7fdedf7fe700 1 -- 172.21.15.121:0/2679004336 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdec80033f0 con 0x7fdee0067380 2023-11-15T03:11:13.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.548+0000 7fdedf7fe700 1 -- 172.21.15.121:0/2679004336 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961490045 0 0) 0x7fdee00657f0 con 0x7fdee014aff0 2023-11-15T03:11:13.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.548+0000 7fdee4fb7700 1 -- 172.21.15.121:0/2679004336 >> v1:172.21.15.190:6789/0 conn(0x7fdee0067380 legacy=0x7fdee00a78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:13.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.548+0000 7fdee4fb7700 1 -- 172.21.15.121:0/2679004336 >> v1:172.21.15.121:6789/0 conn(0x7fdee014aff0 legacy=0x7fdee0066c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:13.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.548+0000 7fdee4fb7700 1 -- 172.21.15.121:0/2679004336 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee0065d20 con 0x7fdee014ac10 2023-11-15T03:11:13.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.552+0000 7fdee4fb7700 1 --2- 172.21.15.121:0/2679004336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdee014ac10 0x7fdee00c6bc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fded0009e90 tx=0x7fded0009780).ready entity=mon.2 client_cookie=50db0538f2fd0703 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:13.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.552+0000 7fdedf7fe700 1 -- 172.21.15.121:0/2679004336 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded000e070 con 0x7fdee014ac10 2023-11-15T03:11:13.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.552+0000 7fdedf7fe700 1 -- 172.21.15.121:0/2679004336 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded000bb60 con 0x7fdee014ac10 2023-11-15T03:11:13.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.552+0000 7fdedf7fe700 1 -- 172.21.15.121:0/2679004336 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded000bee0 con 0x7fdee014ac10 2023-11-15T03:11:13.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.552+0000 7fdee5fb9700 1 -- 172.21.15.121:0/2679004336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdee014ac10 msgr2=0x7fdee00c6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:13.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.552+0000 7fdee5fb9700 1 --2- 172.21.15.121:0/2679004336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdee014ac10 0x7fdee00c6bc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fded0009e90 tx=0x7fded0009780).stop 2023-11-15T03:11:13.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.552+0000 7fdee5fb9700 1 -- 172.21.15.121:0/2679004336 shutdown_connections 2023-11-15T03:11:13.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.552+0000 7fdee5fb9700 1 --2- 172.21.15.121:0/2679004336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdee014ac10 0x7fdee00c6bc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:13.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.552+0000 7fdee5fb9700 1 -- 172.21.15.121:0/2679004336 >> 172.21.15.121:0/2679004336 conn(0x7fdee0067a60 msgr2=0x7fdee0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:13.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.552+0000 7fdee5fb9700 1 -- 172.21.15.121:0/2679004336 shutdown_connections 2023-11-15T03:11:13.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.552+0000 7fdee5fb9700 1 -- 172.21.15.121:0/2679004336 wait complete. 2023-11-15T03:11:13.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdee5fb9700 1 Processor -- start 2023-11-15T03:11:13.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdee5fb9700 1 -- start start 2023-11-15T03:11:13.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdee5fb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdee0067380 0x7fdee0149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:13.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdee5fb9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee014ac10 0x7fdee00c1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:13.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdee5fb9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdee014aff0 0x7fdee0152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:13.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdee5fb9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00c25c0 con 0x7fdee014ac10 2023-11-15T03:11:13.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdee5fb9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00a9da0 con 0x7fdee014aff0 2023-11-15T03:11:13.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdee5fb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee014a130 con 0x7fdee0067380 2023-11-15T03:11:13.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdedffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee014ac10 0x7fdee00c1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:13.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdedffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee014ac10 0x7fdee00c1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40084/0 (socket says 172.21.15.121:40084) 2023-11-15T03:11:13.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdedffff700 1 -- 172.21.15.121:0/3098281528 learned_addr learned my addr 172.21.15.121:0/3098281528 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:13.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdee4fb7700 1 --2- 172.21.15.121:0/3098281528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdee0067380 0x7fdee0149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:13.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdee57b8700 1 --2- 172.21.15.121:0/3098281528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdee014aff0 0x7fdee0152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:13.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdedffff700 1 -- 172.21.15.121:0/3098281528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdee0067380 msgr2=0x7fdee0149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:13.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdedffff700 1 --2- 172.21.15.121:0/3098281528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdee0067380 0x7fdee0149c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:13.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdedffff700 1 -- 172.21.15.121:0/3098281528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdee014aff0 msgr2=0x7fdee0152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:13.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdedffff700 1 --2- 172.21.15.121:0/3098281528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdee014aff0 0x7fdee0152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:13.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdedffff700 1 -- 172.21.15.121:0/3098281528 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded0009a20 con 0x7fdee014ac10 2023-11-15T03:11:13.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdedffff700 1 --2- 172.21.15.121:0/3098281528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee014ac10 0x7fdee00c1ab0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 rx=0x7fded400c4c0 tx=0x7fded4007e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:13.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdedcff9700 1 -- 172.21.15.121:0/3098281528 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded40072e0 con 0x7fdee014ac10 2023-11-15T03:11:13.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdedcff9700 1 -- 172.21.15.121:0/3098281528 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded4007440 con 0x7fdee014ac10 2023-11-15T03:11:13.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdee5fb9700 1 -- 172.21.15.121:0/3098281528 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee00c22e0 con 0x7fdee014ac10 2023-11-15T03:11:13.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.556+0000 7fdedcff9700 1 -- 172.21.15.121:0/3098281528 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded40057a0 con 0x7fdee014ac10 2023-11-15T03:11:13.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.560+0000 7fdee5fb9700 1 -- 172.21.15.121:0/3098281528 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee00a9a30 con 0x7fdee014ac10 2023-11-15T03:11:13.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.560+0000 7fdee5fb9700 1 -- 172.21.15.121:0/3098281528 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee006bcf0 con 0x7fdee014ac10 2023-11-15T03:11:13.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.560+0000 7fdedcff9700 1 -- 172.21.15.121:0/3098281528 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fded4005bd0 con 0x7fdee014ac10 2023-11-15T03:11:13.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.560+0000 7fdedcff9700 1 --2- 172.21.15.121:0/3098281528 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdecc05a510 0x7fdecc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:13.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.564+0000 7fdedcff9700 1 -- 172.21.15.121:0/3098281528 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 8709+0+0 (secure 0 0 0) 0x7fded40749b0 con 0x7fdee014ac10 2023-11-15T03:11:13.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.564+0000 7fdee4fb7700 1 --2- 172.21.15.121:0/3098281528 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdecc05a510 0x7fdecc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:13.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.564+0000 7fdee4fb7700 1 --2- 172.21.15.121:0/3098281528 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdecc05a510 0x7fdecc05c9d0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7fded0009de0 tx=0x7fded0009780).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:13.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.568+0000 7fdedcff9700 1 -- 172.21.15.121:0/3098281528 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fded4047730 con 0x7fdee014ac10 2023-11-15T03:11:13.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.728+0000 7fdee5fb9700 1 -- 172.21.15.121:0/3098281528 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fdee0066000 con 0x7fdee014ac10 2023-11-15T03:11:13.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.736+0000 7fdedcff9700 1 -- 172.21.15.121:0/3098281528 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fded4018030 con 0x7fdee014ac10 2023-11-15T03:11:13.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.736+0000 7fdee5fb9700 1 -- 172.21.15.121:0/3098281528 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdecc05a510 msgr2=0x7fdecc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:13.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.736+0000 7fdee5fb9700 1 --2- 172.21.15.121:0/3098281528 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdecc05a510 0x7fdecc05c9d0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7fded0009de0 tx=0x7fded0009780).stop 2023-11-15T03:11:13.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.736+0000 7fdee5fb9700 1 -- 172.21.15.121:0/3098281528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee014ac10 msgr2=0x7fdee00c1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:13.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.736+0000 7fdee5fb9700 1 --2- 172.21.15.121:0/3098281528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee014ac10 0x7fdee00c1ab0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 rx=0x7fded400c4c0 tx=0x7fded4007e60).stop 2023-11-15T03:11:13.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.736+0000 7fdee5fb9700 1 -- 172.21.15.121:0/3098281528 shutdown_connections 2023-11-15T03:11:13.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.736+0000 7fdee5fb9700 1 --2- 172.21.15.121:0/3098281528 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdecc05a510 0x7fdecc05c9d0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:13.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.736+0000 7fdee5fb9700 1 --2- 172.21.15.121:0/3098281528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdee014aff0 0x7fdee0152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:13.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.736+0000 7fdee5fb9700 1 --2- 172.21.15.121:0/3098281528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee014ac10 0x7fdee00c1ab0 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:13.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.736+0000 7fdee5fb9700 1 --2- 172.21.15.121:0/3098281528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdee0067380 0x7fdee0149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:13.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.736+0000 7fdee5fb9700 1 -- 172.21.15.121:0/3098281528 >> 172.21.15.121:0/3098281528 conn(0x7fdee0067a60 msgr2=0x7fdee01541a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:13.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.736+0000 7fdee5fb9700 1 -- 172.21.15.121:0/3098281528 shutdown_connections 2023-11-15T03:11:13.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.736+0000 7fdee5fb9700 1 -- 172.21.15.121:0/3098281528 wait complete. 2023-11-15T03:11:13.767 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion' 2023-11-15T03:11:13.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.948+0000 7f412359e700 1 Processor -- start 2023-11-15T03:11:13.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.948+0000 7f412359e700 1 -- start start 2023-11-15T03:11:13.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.952+0000 7f412359e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f412415c540 0x7f41240c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:13.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.952+0000 7f412359e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41240c1d50 con 0x7f41240c3280 2023-11-15T03:11:13.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.952+0000 7f412359e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f412415cb80 con 0x7f4124152850 2023-11-15T03:11:13.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.952+0000 7f412359e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41240c1860 con 0x7f412415c540 2023-11-15T03:11:13.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.952+0000 7f4122d9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f412415c540 0x7f41240c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:13.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.952+0000 7f412259c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f41240c3280 0x7f4124152140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49072/0 (socket says 172.21.15.121:49072) 2023-11-15T03:11:13.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.952+0000 7f412259c700 1 -- 172.21.15.121:0/2475364894 learned_addr learned my addr 172.21.15.121:0/2475364894 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:13.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.952+0000 7f4122d9d700 1 -- 172.21.15.121:0/2475364894 >> v1:172.21.15.190:6789/0 conn(0x7f4124152850 legacy=0x7f412415be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:13.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.952+0000 7f4122d9d700 1 -- 172.21.15.121:0/2475364894 >> v1:172.21.15.121:6789/0 conn(0x7f41240c3280 legacy=0x7f4124152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:13.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.952+0000 7f4122d9d700 1 -- 172.21.15.121:0/2475364894 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41240a9240 con 0x7f412415c540 2023-11-15T03:11:13.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.952+0000 7f412159a700 1 -- 172.21.15.121:0/2475364894 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010372679 0 0) 0x7f41240c1d50 con 0x7f41240c3280 2023-11-15T03:11:13.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.952+0000 7f4122d9d700 1 --2- 172.21.15.121:0/2475364894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f412415c540 0x7f41240c3660 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 rx=0x7f411c009e60 tx=0x7f411c009780).ready entity=mon.0 client_cookie=47fd42fe687ee982 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:13.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.956+0000 7f412159a700 1 -- 172.21.15.121:0/2475364894 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c00e070 con 0x7f412415c540 2023-11-15T03:11:13.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.956+0000 7f412159a700 1 -- 172.21.15.121:0/2475364894 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c00bc30 con 0x7f412415c540 2023-11-15T03:11:13.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.956+0000 7f412159a700 1 -- 172.21.15.121:0/2475364894 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c012410 con 0x7f412415c540 2023-11-15T03:11:13.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.956+0000 7f412359e700 1 -- 172.21.15.121:0/2475364894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f412415c540 msgr2=0x7f41240c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:13.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.956+0000 7f412359e700 1 --2- 172.21.15.121:0/2475364894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f412415c540 0x7f41240c3660 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 rx=0x7f411c009e60 tx=0x7f411c009780).stop 2023-11-15T03:11:13.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.956+0000 7f412359e700 1 -- 172.21.15.121:0/2475364894 shutdown_connections 2023-11-15T03:11:13.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.956+0000 7f412359e700 1 --2- 172.21.15.121:0/2475364894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f412415c540 0x7f41240c3660 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:13.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.956+0000 7f412359e700 1 -- 172.21.15.121:0/2475364894 >> 172.21.15.121:0/2475364894 conn(0x7f4124062190 msgr2=0x7f4124062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:13.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.956+0000 7f412359e700 1 -- 172.21.15.121:0/2475364894 shutdown_connections 2023-11-15T03:11:13.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.956+0000 7f412359e700 1 -- 172.21.15.121:0/2475364894 wait complete. 2023-11-15T03:11:13.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.960+0000 7f412359e700 1 Processor -- start 2023-11-15T03:11:13.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.960+0000 7f412359e700 1 -- start start 2023-11-15T03:11:13.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.960+0000 7f412359e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41240c3280 0x7f4124140120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:13.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.960+0000 7f412359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4124152850 0x7f41240a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:13.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.960+0000 7f412359e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f412415c540 0x7f41240aa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:13.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.960+0000 7f412359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41240aa7d0 con 0x7f4124152850 2023-11-15T03:11:13.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.960+0000 7f412359e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4124142400 con 0x7f41240c3280 2023-11-15T03:11:13.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.960+0000 7f412359e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41241417f0 con 0x7f412415c540 2023-11-15T03:11:13.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.960+0000 7f4121d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4124152850 0x7f41240a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:13.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.960+0000 7f4121d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4124152850 0x7f41240a9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40106/0 (socket says 172.21.15.121:40106) 2023-11-15T03:11:13.972 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.960+0000 7f4121d9b700 1 -- 172.21.15.121:0/507454030 learned_addr learned my addr 172.21.15.121:0/507454030 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:13.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.964+0000 7f412259c700 1 --2- 172.21.15.121:0/507454030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41240c3280 0x7f4124140120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:13.973 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.964+0000 7f4122d9d700 1 --2- 172.21.15.121:0/507454030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f412415c540 0x7f41240aa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:13.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.964+0000 7f4121d9b700 1 -- 172.21.15.121:0/507454030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f412415c540 msgr2=0x7f41240aa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:13.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.964+0000 7f4121d9b700 1 --2- 172.21.15.121:0/507454030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f412415c540 0x7f41240aa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:13.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.964+0000 7f4121d9b700 1 -- 172.21.15.121:0/507454030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41240c3280 msgr2=0x7f4124140120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:13.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.964+0000 7f4121d9b700 1 --2- 172.21.15.121:0/507454030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41240c3280 0x7f4124140120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:13.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.964+0000 7f4121d9b700 1 -- 172.21.15.121:0/507454030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c009a20 con 0x7f4124152850 2023-11-15T03:11:13.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.964+0000 7f4121d9b700 1 --2- 172.21.15.121:0/507454030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4124152850 0x7f41240a9df0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 rx=0x7f411800c4f0 tx=0x7f4118007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:13.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.964+0000 7f41127fc700 1 -- 172.21.15.121:0/507454030 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118007020 con 0x7f4124152850 2023-11-15T03:11:13.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.964+0000 7f41127fc700 1 -- 172.21.15.121:0/507454030 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4118001e90 con 0x7f4124152850 2023-11-15T03:11:13.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.964+0000 7f412359e700 1 -- 172.21.15.121:0/507454030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4124141ad0 con 0x7f4124152850 2023-11-15T03:11:13.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.964+0000 7f41127fc700 1 -- 172.21.15.121:0/507454030 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41180073b0 con 0x7f4124152850 2023-11-15T03:11:13.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.964+0000 7f412359e700 1 -- 172.21.15.121:0/507454030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4124142090 con 0x7f4124152850 2023-11-15T03:11:13.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.964+0000 7f412359e700 1 -- 172.21.15.121:0/507454030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f412406bcf0 con 0x7f4124152850 2023-11-15T03:11:13.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.964+0000 7f41127fc700 1 -- 172.21.15.121:0/507454030 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f41180068b0 con 0x7f4124152850 2023-11-15T03:11:13.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.968+0000 7f41127fc700 1 --2- 172.21.15.121:0/507454030 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f410405a510 0x7f410405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:13.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.968+0000 7f41127fc700 1 -- 172.21.15.121:0/507454030 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 8709+0+0 (secure 0 0 0) 0x7f4118074bd0 con 0x7f4124152850 2023-11-15T03:11:13.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.968+0000 7f412259c700 1 --2- 172.21.15.121:0/507454030 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f410405a510 0x7f410405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:13.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.968+0000 7f412259c700 1 --2- 172.21.15.121:0/507454030 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f410405a510 0x7f410405c9d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7f41140011e0 tx=0x7f4114009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:13.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:13.972+0000 7f41127fc700 1 -- 172.21.15.121:0/507454030 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4118047950 con 0x7f4124152850 2023-11-15T03:11:14.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.196+0000 7f412359e700 1 -- 172.21.15.121:0/507454030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]} v 0) v1 -- 0x7f41240a7cc0 con 0x7f4124152850 2023-11-15T03:11:14.378 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.368+0000 7f41127fc700 1 -- 172.21.15.121:0/507454030 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]}]=0 v733) v1 ==== 135+0+0 (secure 0 0 0) 0x7f411804af70 con 0x7f4124152850 2023-11-15T03:11:14.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.372+0000 7f412359e700 1 -- 172.21.15.121:0/507454030 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f410405a510 msgr2=0x7f410405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:14.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.372+0000 7f412359e700 1 --2- 172.21.15.121:0/507454030 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f410405a510 0x7f410405c9d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7f41140011e0 tx=0x7f4114009040).stop 2023-11-15T03:11:14.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.372+0000 7f412359e700 1 -- 172.21.15.121:0/507454030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4124152850 msgr2=0x7f41240a9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:14.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.372+0000 7f412359e700 1 --2- 172.21.15.121:0/507454030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4124152850 0x7f41240a9df0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 rx=0x7f411800c4f0 tx=0x7f4118007e00).stop 2023-11-15T03:11:14.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.372+0000 7f412359e700 1 -- 172.21.15.121:0/507454030 shutdown_connections 2023-11-15T03:11:14.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.372+0000 7f412359e700 1 --2- 172.21.15.121:0/507454030 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f410405a510 0x7f410405c9d0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:14.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.372+0000 7f412359e700 1 --2- 172.21.15.121:0/507454030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f412415c540 0x7f41240aa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:14.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.372+0000 7f412359e700 1 --2- 172.21.15.121:0/507454030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4124152850 0x7f41240a9df0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:14.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.372+0000 7f412359e700 1 --2- 172.21.15.121:0/507454030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41240c3280 0x7f4124140120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:14.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.372+0000 7f412359e700 1 -- 172.21.15.121:0/507454030 >> 172.21.15.121:0/507454030 conn(0x7f4124062190 msgr2=0x7f412415af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:14.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.372+0000 7f412359e700 1 -- 172.21.15.121:0/507454030 shutdown_connections 2023-11-15T03:11:14.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.372+0000 7f412359e700 1 -- 172.21.15.121:0/507454030 wait complete. 2023-11-15T03:11:14.403 INFO:tasks.cephfs_test_runner:That the MDS has a clean 'damaged' response to loss of any single metadata object ... ok 2023-11-15T03:11:14.404 INFO:tasks.cephfs_test_runner:Starting test: test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) That errors encountered during opening inos are properly propagated 2023-11-15T03:11:14.404 INFO:tasks.cephfs_test_runner:test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) 2023-11-15T03:11:14.405 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors' 2023-11-15T03:11:14.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.564+0000 7f57746f1700 1 Processor -- start 2023-11-15T03:11:14.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.564+0000 7f57746f1700 1 -- start start 2023-11-15T03:11:14.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.568+0000 7f57746f1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f576c153c30 0x7f576c0c5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:14.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.568+0000 7f57746f1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f576c0c5960 0x7f576c068a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:14.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.568+0000 7f57746f1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f576c0bdae0 con 0x7f576c154010 2023-11-15T03:11:14.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.568+0000 7f57746f1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576c0bced0 con 0x7f576c0c5960 2023-11-15T03:11:14.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.568+0000 7f57746f1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576c0bd1b0 con 0x7f576c153c30 2023-11-15T03:11:14.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.568+0000 7f5772c8e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f576c0c5960 0x7f576c068a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:14.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.568+0000 7f5771c8c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f576c154010 0x7f576c0a3e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49082/0 (socket says 172.21.15.121:49082) 2023-11-15T03:11:14.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.568+0000 7f5771c8c700 1 -- 172.21.15.121:0/1920398767 learned_addr learned my addr 172.21.15.121:0/1920398767 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:14.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.568+0000 7f577248d700 1 --2- 172.21.15.121:0/1920398767 >> v2:172.21.15.190:3300/0 conn(0x7f576c153c30 0x7f576c0c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:14.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.568+0000 7f577148b700 1 -- 172.21.15.121:0/1920398767 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1392398786 0 0) 0x7f576c0bdae0 con 0x7f576c154010 2023-11-15T03:11:14.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.568+0000 7f5772c8e700 1 -- 172.21.15.121:0/1920398767 >> v2:172.21.15.190:3300/0 conn(0x7f576c153c30 msgr2=0x7f576c0c5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:14.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.568+0000 7f5772c8e700 1 --2- 172.21.15.121:0/1920398767 >> v2:172.21.15.190:3300/0 conn(0x7f576c153c30 0x7f576c0c5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:14.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.568+0000 7f5772c8e700 1 -- 172.21.15.121:0/1920398767 >> v1:172.21.15.121:6789/0 conn(0x7f576c154010 legacy=0x7f576c0a3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:14.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.568+0000 7f5772c8e700 1 -- 172.21.15.121:0/1920398767 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c0bd490 con 0x7f576c0c5960 2023-11-15T03:11:14.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.568+0000 7f5772c8e700 1 --2- 172.21.15.121:0/1920398767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f576c0c5960 0x7f576c068a70 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 rx=0x7f576c0697f0 tx=0x7f5764009780).ready entity=mon.0 client_cookie=eec9a692b68ae4eb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:14.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.572+0000 7f577148b700 1 -- 172.21.15.121:0/1920398767 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576400e070 con 0x7f576c0c5960 2023-11-15T03:11:14.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.572+0000 7f577148b700 1 -- 172.21.15.121:0/1920398767 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576400bc30 con 0x7f576c0c5960 2023-11-15T03:11:14.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.572+0000 7f577148b700 1 -- 172.21.15.121:0/1920398767 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5764012410 con 0x7f576c0c5960 2023-11-15T03:11:14.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.572+0000 7f57746f1700 1 -- 172.21.15.121:0/1920398767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f576c0c5960 msgr2=0x7f576c068a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:14.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.572+0000 7f57746f1700 1 --2- 172.21.15.121:0/1920398767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f576c0c5960 0x7f576c068a70 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 rx=0x7f576c0697f0 tx=0x7f5764009780).stop 2023-11-15T03:11:14.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.572+0000 7f57746f1700 1 -- 172.21.15.121:0/1920398767 shutdown_connections 2023-11-15T03:11:14.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.572+0000 7f57746f1700 1 --2- 172.21.15.121:0/1920398767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f576c0c5960 0x7f576c068a70 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:14.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.572+0000 7f57746f1700 1 --2- 172.21.15.121:0/1920398767 >> v2:172.21.15.190:3300/0 conn(0x7f576c153c30 0x7f576c0c5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:14.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.572+0000 7f57746f1700 1 -- 172.21.15.121:0/1920398767 >> 172.21.15.121:0/1920398767 conn(0x7f576c0c3af0 msgr2=0x7f576c0c3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:14.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.572+0000 7f57746f1700 1 -- 172.21.15.121:0/1920398767 shutdown_connections 2023-11-15T03:11:14.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.572+0000 7f57746f1700 1 -- 172.21.15.121:0/1920398767 wait complete. 2023-11-15T03:11:14.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.572+0000 7f57746f1700 1 Processor -- start 2023-11-15T03:11:14.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f57746f1700 1 -- start start 2023-11-15T03:11:14.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f57746f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f576c0c5960 0x7f576c14bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:14.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f57746f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f576c153c30 0x7f576c0aa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:14.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f57746f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f576c154010 0x7f576c1568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:14.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f57746f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c0aa900 con 0x7f576c153c30 2023-11-15T03:11:14.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f57746f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c0aabe0 con 0x7f576c154010 2023-11-15T03:11:14.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f57746f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576c0aaec0 con 0x7f576c0c5960 2023-11-15T03:11:14.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f5771c8c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f576c153c30 0x7f576c0aa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:14.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f5771c8c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f576c153c30 0x7f576c0aa000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40116/0 (socket says 172.21.15.121:40116) 2023-11-15T03:11:14.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f5771c8c700 1 -- 172.21.15.121:0/1750550456 learned_addr learned my addr 172.21.15.121:0/1750550456 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:14.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f5772c8e700 1 --2- 172.21.15.121:0/1750550456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f576c154010 0x7f576c1568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:14.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f577248d700 1 --2- 172.21.15.121:0/1750550456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f576c0c5960 0x7f576c14bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:14.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f5771c8c700 1 -- 172.21.15.121:0/1750550456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f576c0c5960 msgr2=0x7f576c14bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:14.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f5771c8c700 1 --2- 172.21.15.121:0/1750550456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f576c0c5960 0x7f576c14bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:14.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f5771c8c700 1 -- 172.21.15.121:0/1750550456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f576c154010 msgr2=0x7f576c1568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:14.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f5771c8c700 1 --2- 172.21.15.121:0/1750550456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f576c154010 0x7f576c1568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:14.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f5771c8c700 1 -- 172.21.15.121:0/1750550456 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5764009a20 con 0x7f576c153c30 2023-11-15T03:11:14.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f5771c8c700 1 --2- 172.21.15.121:0/1750550456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f576c153c30 0x7f576c0aa000 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 rx=0x7f575c00c490 tx=0x7f575c007e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:14.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f57627fc700 1 -- 172.21.15.121:0/1750550456 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c0070d0 con 0x7f576c153c30 2023-11-15T03:11:14.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f57627fc700 1 -- 172.21.15.121:0/1750550456 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575c007230 con 0x7f576c153c30 2023-11-15T03:11:14.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f57746f1700 1 -- 172.21.15.121:0/1750550456 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576c0ab1a0 con 0x7f576c153c30 2023-11-15T03:11:14.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.576+0000 7f57627fc700 1 -- 172.21.15.121:0/1750550456 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c0075d0 con 0x7f576c153c30 2023-11-15T03:11:14.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.580+0000 7f57746f1700 1 -- 172.21.15.121:0/1750550456 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576c0aa4f0 con 0x7f576c153c30 2023-11-15T03:11:14.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.580+0000 7f57746f1700 1 -- 172.21.15.121:0/1750550456 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f576c0c57c0 con 0x7f576c153c30 2023-11-15T03:11:14.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.580+0000 7f57627fc700 1 -- 172.21.15.121:0/1750550456 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f575c005bc0 con 0x7f576c153c30 2023-11-15T03:11:14.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.584+0000 7f57627fc700 1 --2- 172.21.15.121:0/1750550456 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f575005a4e0 0x7f575005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:14.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.584+0000 7f57627fc700 1 -- 172.21.15.121:0/1750550456 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 8709+0+0 (secure 0 0 0) 0x7f575c074ac0 con 0x7f576c153c30 2023-11-15T03:11:14.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.584+0000 7f577248d700 1 --2- 172.21.15.121:0/1750550456 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f575005a4e0 0x7f575005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:14.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.584+0000 7f577248d700 1 --2- 172.21.15.121:0/1750550456 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f575005a4e0 0x7f575005c9a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f576800bbc0 tx=0x7f5768008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:14.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.588+0000 7f57627fc700 1 -- 172.21.15.121:0/1750550456 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f576c0c57c0 con 0x7f576c153c30 2023-11-15T03:11:14.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:14.800+0000 7f57746f1700 1 -- 172.21.15.121:0/1750550456 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]} v 0) v1 -- 0x7f576c0c57c0 con 0x7f576c153c30 2023-11-15T03:11:15.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.372+0000 7f57627fc700 1 -- 172.21.15.121:0/1750550456 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]}]=0 v734) v1 ==== 138+0+0 (secure 0 0 0) 0x7f576c0c57c0 con 0x7f576c153c30 2023-11-15T03:11:15.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.376+0000 7f57746f1700 1 -- 172.21.15.121:0/1750550456 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f575005a4e0 msgr2=0x7f575005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:15.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.376+0000 7f57746f1700 1 --2- 172.21.15.121:0/1750550456 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f575005a4e0 0x7f575005c9a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f576800bbc0 tx=0x7f5768008040).stop 2023-11-15T03:11:15.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.376+0000 7f57746f1700 1 -- 172.21.15.121:0/1750550456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f576c153c30 msgr2=0x7f576c0aa000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:15.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.376+0000 7f57746f1700 1 --2- 172.21.15.121:0/1750550456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f576c153c30 0x7f576c0aa000 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 rx=0x7f575c00c490 tx=0x7f575c007e60).stop 2023-11-15T03:11:15.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.376+0000 7f57746f1700 1 -- 172.21.15.121:0/1750550456 shutdown_connections 2023-11-15T03:11:15.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.376+0000 7f57746f1700 1 --2- 172.21.15.121:0/1750550456 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f575005a4e0 0x7f575005c9a0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:15.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.376+0000 7f57746f1700 1 --2- 172.21.15.121:0/1750550456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f576c154010 0x7f576c1568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:15.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.376+0000 7f57746f1700 1 --2- 172.21.15.121:0/1750550456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f576c153c30 0x7f576c0aa000 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:15.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.376+0000 7f57746f1700 1 --2- 172.21.15.121:0/1750550456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f576c0c5960 0x7f576c14bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:15.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.376+0000 7f57746f1700 1 -- 172.21.15.121:0/1750550456 >> 172.21.15.121:0/1750550456 conn(0x7f576c0c3af0 msgr2=0x7f576c0a55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:15.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.376+0000 7f57746f1700 1 -- 172.21.15.121:0/1750550456 shutdown_connections 2023-11-15T03:11:15.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.376+0000 7f57746f1700 1 -- 172.21.15.121:0/1750550456 wait complete. 2023-11-15T03:11:15.408 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-15T03:11:15.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.564+0000 7fc298ff9700 1 Processor -- start 2023-11-15T03:11:15.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.564+0000 7fc298ff9700 1 -- start start 2023-11-15T03:11:15.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.564+0000 7fc298ff9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc294067030 0x7fc294065860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:15.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc298ff9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc294065d50 0x7fc2940c2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:15.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc298ff9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2940c2080 con 0x7fc294066c50 2023-11-15T03:11:15.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc298ff9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2940668b0 con 0x7fc294065d50 2023-11-15T03:11:15.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc298ff9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2940665d0 con 0x7fc294067030 2023-11-15T03:11:15.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc2937fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc294066c50 0x7fc2940a62f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:35824/0 (socket says 172.21.15.121:35824) 2023-11-15T03:11:15.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc2937fe700 1 -- 172.21.15.121:0/3890058239 learned_addr learned my addr 172.21.15.121:0/3890058239 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:15.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc293fff700 1 --2- 172.21.15.121:0/3890058239 >> v2:172.21.15.121:3300/0 conn(0x7fc294065d50 0x7fc2940c2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:15.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc292ffd700 1 --2- 172.21.15.121:0/3890058239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc294067030 0x7fc294065860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:15.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc2927fc700 1 -- 172.21.15.121:0/3890058239 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583661513 0 0) 0x7fc2940c2080 con 0x7fc294066c50 2023-11-15T03:11:15.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc2927fc700 1 -- 172.21.15.121:0/3890058239 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc27c0033f0 con 0x7fc294066c50 2023-11-15T03:11:15.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc293fff700 1 -- 172.21.15.121:0/3890058239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc294067030 msgr2=0x7fc294065860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:15.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc293fff700 1 --2- 172.21.15.121:0/3890058239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc294067030 0x7fc294065860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:15.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc2927fc700 1 -- 172.21.15.121:0/3890058239 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2039968955 0 0) 0x7fc27c0033f0 con 0x7fc294066c50 2023-11-15T03:11:15.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc293fff700 1 -- 172.21.15.121:0/3890058239 >> v1:172.21.15.121:6789/0 conn(0x7fc294066c50 legacy=0x7fc2940a62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:15.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc293fff700 1 -- 172.21.15.121:0/3890058239 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2940c2320 con 0x7fc294065d50 2023-11-15T03:11:15.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc292ffd700 1 --2- 172.21.15.121:0/3890058239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc294067030 0x7fc294065860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:15.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc293fff700 1 --2- 172.21.15.121:0/3890058239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc294065d50 0x7fc2940c2e00 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 rx=0x7fc28c009e60 tx=0x7fc28c009780).ready entity=mon.0 client_cookie=44b311a37dcb10d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:15.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc2927fc700 1 -- 172.21.15.121:0/3890058239 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c00e070 con 0x7fc294065d50 2023-11-15T03:11:15.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc2927fc700 1 -- 172.21.15.121:0/3890058239 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c00b850 con 0x7fc294065d50 2023-11-15T03:11:15.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc2927fc700 1 -- 172.21.15.121:0/3890058239 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284001470 con 0x7fc294065d50 2023-11-15T03:11:15.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc298ff9700 1 -- 172.21.15.121:0/3890058239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc294065d50 msgr2=0x7fc2940c2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:15.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc298ff9700 1 --2- 172.21.15.121:0/3890058239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc294065d50 0x7fc2940c2e00 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 rx=0x7fc28c009e60 tx=0x7fc28c009780).stop 2023-11-15T03:11:15.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc298ff9700 1 -- 172.21.15.121:0/3890058239 shutdown_connections 2023-11-15T03:11:15.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc298ff9700 1 --2- 172.21.15.121:0/3890058239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc294065d50 0x7fc2940c2e00 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:15.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc298ff9700 1 --2- 172.21.15.121:0/3890058239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc294067030 0x7fc294065860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:15.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc298ff9700 1 -- 172.21.15.121:0/3890058239 >> 172.21.15.121:0/3890058239 conn(0x7fc294067a60 msgr2=0x7fc294067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:15.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc298ff9700 1 -- 172.21.15.121:0/3890058239 shutdown_connections 2023-11-15T03:11:15.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.568+0000 7fc298ff9700 1 -- 172.21.15.121:0/3890058239 wait complete. 2023-11-15T03:11:15.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.572+0000 7fc298ff9700 1 Processor -- start 2023-11-15T03:11:15.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.572+0000 7fc298ff9700 1 -- start start 2023-11-15T03:11:15.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.576+0000 7fc298ff9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc294065d50 0x7fc29413dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:15.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.576+0000 7fc298ff9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc294066c50 0x7fc29414ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:15.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.576+0000 7fc298ff9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc294067030 0x7fc29414ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:15.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.576+0000 7fc298ff9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29414f430 con 0x7fc294067030 2023-11-15T03:11:15.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.576+0000 7fc298ff9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29414fe80 con 0x7fc294066c50 2023-11-15T03:11:15.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.576+0000 7fc298ff9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc294150440 con 0x7fc294065d50 2023-11-15T03:11:15.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.576+0000 7fc293fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc294067030 0x7fc29414ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:15.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.576+0000 7fc293fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc294067030 0x7fc29414ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44666/0 (socket says 172.21.15.121:44666) 2023-11-15T03:11:15.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.576+0000 7fc293fff700 1 -- 172.21.15.121:0/4278208320 learned_addr learned my addr 172.21.15.121:0/4278208320 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:15.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.576+0000 7fc2937fe700 1 --2- 172.21.15.121:0/4278208320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc294065d50 0x7fc29413dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:15.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.576+0000 7fc292ffd700 1 --2- 172.21.15.121:0/4278208320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc294066c50 0x7fc29414ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:15.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.576+0000 7fc293fff700 1 -- 172.21.15.121:0/4278208320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc294065d50 msgr2=0x7fc29413dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:15.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.576+0000 7fc293fff700 1 --2- 172.21.15.121:0/4278208320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc294065d50 0x7fc29413dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:15.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.576+0000 7fc293fff700 1 -- 172.21.15.121:0/4278208320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc294066c50 msgr2=0x7fc29414ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:15.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.576+0000 7fc293fff700 1 --2- 172.21.15.121:0/4278208320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc294066c50 0x7fc29414ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:15.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.576+0000 7fc293fff700 1 -- 172.21.15.121:0/4278208320 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c009a20 con 0x7fc294067030 2023-11-15T03:11:15.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.576+0000 7fc292ffd700 1 --2- 172.21.15.121:0/4278208320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc294066c50 0x7fc29414ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:15.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.576+0000 7fc293fff700 1 --2- 172.21.15.121:0/4278208320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc294067030 0x7fc29414ef40 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 rx=0x7fc28c013d40 tx=0x7fc28c009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:15.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.580+0000 7fc27b7fe700 1 -- 172.21.15.121:0/4278208320 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c01d420 con 0x7fc294067030 2023-11-15T03:11:15.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.580+0000 7fc27b7fe700 1 -- 172.21.15.121:0/4278208320 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c01d580 con 0x7fc294067030 2023-11-15T03:11:15.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.580+0000 7fc298ff9700 1 -- 172.21.15.121:0/4278208320 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2940c4620 con 0x7fc294067030 2023-11-15T03:11:15.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.580+0000 7fc27b7fe700 1 -- 172.21.15.121:0/4278208320 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c01d940 con 0x7fc294067030 2023-11-15T03:11:15.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.580+0000 7fc298ff9700 1 -- 172.21.15.121:0/4278208320 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc29414f910 con 0x7fc294067030 2023-11-15T03:11:15.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.580+0000 7fc298ff9700 1 -- 172.21.15.121:0/4278208320 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2941505d0 con 0x7fc294067030 2023-11-15T03:11:15.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.580+0000 7fc27b7fe700 1 -- 172.21.15.121:0/4278208320 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc28c003040 con 0x7fc294067030 2023-11-15T03:11:15.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.584+0000 7fc27b7fe700 1 --2- 172.21.15.121:0/4278208320 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc28005a540 0x7fc28005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:15.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.584+0000 7fc27b7fe700 1 -- 172.21.15.121:0/4278208320 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 8709+0+0 (secure 0 0 0) 0x7fc28c076ee0 con 0x7fc294067030 2023-11-15T03:11:15.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.584+0000 7fc2937fe700 1 --2- 172.21.15.121:0/4278208320 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc28005a540 0x7fc28005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:15.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.584+0000 7fc2937fe700 1 --2- 172.21.15.121:0/4278208320 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc28005a540 0x7fc28005ca00 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7fc2840011e0 tx=0x7fc284001520).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:15.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.588+0000 7fc27b7fe700 1 -- 172.21.15.121:0/4278208320 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc28c049ce0 con 0x7fc294067030 2023-11-15T03:11:15.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.760+0000 7fc298ff9700 1 -- 172.21.15.121:0/4278208320 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc29406bcf0 con 0x7fc294067030 2023-11-15T03:11:15.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.772+0000 7fc27b7fe700 1 -- 172.21.15.121:0/4278208320 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc28c01dcb0 con 0x7fc294067030 2023-11-15T03:11:15.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.772+0000 7fc298ff9700 1 -- 172.21.15.121:0/4278208320 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc28005a540 msgr2=0x7fc28005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:15.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.772+0000 7fc298ff9700 1 --2- 172.21.15.121:0/4278208320 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc28005a540 0x7fc28005ca00 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7fc2840011e0 tx=0x7fc284001520).stop 2023-11-15T03:11:15.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.776+0000 7fc298ff9700 1 -- 172.21.15.121:0/4278208320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc294067030 msgr2=0x7fc29414ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:15.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.776+0000 7fc298ff9700 1 --2- 172.21.15.121:0/4278208320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc294067030 0x7fc29414ef40 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 rx=0x7fc28c013d40 tx=0x7fc28c009780).stop 2023-11-15T03:11:15.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.776+0000 7fc298ff9700 1 -- 172.21.15.121:0/4278208320 shutdown_connections 2023-11-15T03:11:15.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.776+0000 7fc298ff9700 1 --2- 172.21.15.121:0/4278208320 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc28005a540 0x7fc28005ca00 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:15.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.776+0000 7fc298ff9700 1 --2- 172.21.15.121:0/4278208320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc294067030 0x7fc29414ef40 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:15.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.776+0000 7fc298ff9700 1 --2- 172.21.15.121:0/4278208320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc294066c50 0x7fc29414ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:15.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.776+0000 7fc298ff9700 1 --2- 172.21.15.121:0/4278208320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc294065d50 0x7fc29413dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:15.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.776+0000 7fc298ff9700 1 -- 172.21.15.121:0/4278208320 >> 172.21.15.121:0/4278208320 conn(0x7fc294067a60 msgr2=0x7fc294155680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:15.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.776+0000 7fc298ff9700 1 -- 172.21.15.121:0/4278208320 shutdown_connections 2023-11-15T03:11:15.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:15.776+0000 7fc298ff9700 1 -- 172.21.15.121:0/4278208320 wait complete. 2023-11-15T03:11:15.803 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-15T03:11:15.803 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T03:11:15.846 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T03:11:15.891 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-15T03:11:15.892 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-15T03:11:15.892 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:16.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.052+0000 7f09b231b700 1 Processor -- start 2023-11-15T03:11:16.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.052+0000 7f09b231b700 1 -- start start 2023-11-15T03:11:16.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.052+0000 7f09b231b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac1562f0 0x7f09ac155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:16.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.052+0000 7f09b231b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f09ac0a57c0 0x7f09ac0a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:16.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.052+0000 7f09b231b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ac0c4c30 con 0x7f09ac154ca0 2023-11-15T03:11:16.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.052+0000 7f09b231b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09ac0a8280 con 0x7f09ac0a57c0 2023-11-15T03:11:16.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.052+0000 7f09b231b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac0a6d40 con 0x7f09ac1562f0 2023-11-15T03:11:16.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.052+0000 7f09abfff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f09ac154ca0 0x7f09ac0a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:35836/0 (socket says 172.21.15.121:35836) 2023-11-15T03:11:16.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.052+0000 7f09abfff700 1 -- 172.21.15.121:0/2143640950 learned_addr learned my addr 172.21.15.121:0/2143640950 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:16.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.052+0000 7f09a3fff700 1 --2- 172.21.15.121:0/2143640950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac1562f0 0x7f09ac155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:16.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.052+0000 7f09b08b8700 1 --2- 172.21.15.121:0/2143640950 >> v2:172.21.15.190:3300/0 conn(0x7f09ac0a57c0 0x7f09ac0a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:16.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.052+0000 7f09ab7fe700 1 -- 172.21.15.121:0/2143640950 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871781388 0 0) 0x7f09ac0c4c30 con 0x7f09ac154ca0 2023-11-15T03:11:16.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.052+0000 7f09ab7fe700 1 -- 172.21.15.121:0/2143640950 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09980033f0 con 0x7f09ac154ca0 2023-11-15T03:11:16.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.056+0000 7f09a3fff700 1 -- 172.21.15.121:0/2143640950 >> v2:172.21.15.190:3300/0 conn(0x7f09ac0a57c0 msgr2=0x7f09ac0a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:16.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.056+0000 7f09a3fff700 1 --2- 172.21.15.121:0/2143640950 >> v2:172.21.15.190:3300/0 conn(0x7f09ac0a57c0 0x7f09ac0a5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:16.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.056+0000 7f09a3fff700 1 -- 172.21.15.121:0/2143640950 >> v1:172.21.15.121:6789/0 conn(0x7f09ac154ca0 legacy=0x7f09ac0a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:16.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.056+0000 7f09ab7fe700 1 -- 172.21.15.121:0/2143640950 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1746801787 0 0) 0x7f09980033f0 con 0x7f09ac154ca0 2023-11-15T03:11:16.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.056+0000 7f09a3fff700 1 -- 172.21.15.121:0/2143640950 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac0a6140 con 0x7f09ac1562f0 2023-11-15T03:11:16.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.056+0000 7f09a3fff700 1 --2- 172.21.15.121:0/2143640950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac1562f0 0x7f09ac155080 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f0994009760 tx=0x7f0994009060).ready entity=mon.2 client_cookie=526cee2d915fbe8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:16.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.056+0000 7f09ab7fe700 1 -- 172.21.15.121:0/2143640950 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c001470 con 0x7f09ac1562f0 2023-11-15T03:11:16.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.056+0000 7f09ab7fe700 1 -- 172.21.15.121:0/2143640950 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0994011400 con 0x7f09ac1562f0 2023-11-15T03:11:16.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.056+0000 7f09ab7fe700 1 -- 172.21.15.121:0/2143640950 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994011780 con 0x7f09ac1562f0 2023-11-15T03:11:16.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.056+0000 7f09b231b700 1 -- 172.21.15.121:0/2143640950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac1562f0 msgr2=0x7f09ac155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:16.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.056+0000 7f09b231b700 1 --2- 172.21.15.121:0/2143640950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac1562f0 0x7f09ac155080 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f0994009760 tx=0x7f0994009060).stop 2023-11-15T03:11:16.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.056+0000 7f09b231b700 1 -- 172.21.15.121:0/2143640950 shutdown_connections 2023-11-15T03:11:16.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.056+0000 7f09b231b700 1 --2- 172.21.15.121:0/2143640950 >> v2:172.21.15.190:3300/0 conn(0x7f09ac0a57c0 0x7f09ac0a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:16.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.056+0000 7f09b231b700 1 --2- 172.21.15.121:0/2143640950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac1562f0 0x7f09ac155080 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:16.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.056+0000 7f09b231b700 1 -- 172.21.15.121:0/2143640950 >> 172.21.15.121:0/2143640950 conn(0x7f09ac062190 msgr2=0x7f09ac0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:16.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.056+0000 7f09b231b700 1 -- 172.21.15.121:0/2143640950 shutdown_connections 2023-11-15T03:11:16.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.056+0000 7f09b231b700 1 -- 172.21.15.121:0/2143640950 wait complete. 2023-11-15T03:11:16.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09b231b700 1 Processor -- start 2023-11-15T03:11:16.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09b231b700 1 -- start start 2023-11-15T03:11:16.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09b231b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09ac0a57c0 0x7f09ac14d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:16.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09b231b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09ac154ca0 0x7f09ac0662b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:16.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09b231b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac1562f0 0x7f09ac0667a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:16.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09b231b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac066c90 con 0x7f09ac154ca0 2023-11-15T03:11:16.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09b231b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac157340 con 0x7f09ac0a57c0 2023-11-15T03:11:16.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09b231b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac157620 con 0x7f09ac1562f0 2023-11-15T03:11:16.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09a3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09ac154ca0 0x7f09ac0662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:16.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09a3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09ac154ca0 0x7f09ac0662b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44674/0 (socket says 172.21.15.121:44674) 2023-11-15T03:11:16.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09a3fff700 1 -- 172.21.15.121:0/919984465 learned_addr learned my addr 172.21.15.121:0/919984465 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:16.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09abfff700 1 --2- 172.21.15.121:0/919984465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09ac0a57c0 0x7f09ac14d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:16.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09b08b8700 1 --2- 172.21.15.121:0/919984465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac1562f0 0x7f09ac0667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:16.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09a3fff700 1 -- 172.21.15.121:0/919984465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac1562f0 msgr2=0x7f09ac0667a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:16.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09a3fff700 1 --2- 172.21.15.121:0/919984465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac1562f0 0x7f09ac0667a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:16.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09a3fff700 1 -- 172.21.15.121:0/919984465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09ac0a57c0 msgr2=0x7f09ac14d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:16.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09a3fff700 1 --2- 172.21.15.121:0/919984465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09ac0a57c0 0x7f09ac14d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:16.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09a3fff700 1 -- 172.21.15.121:0/919984465 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09940092a0 con 0x7f09ac154ca0 2023-11-15T03:11:16.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09a3fff700 1 --2- 172.21.15.121:0/919984465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09ac154ca0 0x7f09ac0662b0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 rx=0x7f09ac0c0d10 tx=0x7f0994016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:16.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09a8ff9700 1 -- 172.21.15.121:0/919984465 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994010870 con 0x7f09ac154ca0 2023-11-15T03:11:16.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09a8ff9700 1 -- 172.21.15.121:0/919984465 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09940109d0 con 0x7f09ac154ca0 2023-11-15T03:11:16.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09a8ff9700 1 -- 172.21.15.121:0/919984465 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994010d90 con 0x7f09ac154ca0 2023-11-15T03:11:16.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09b231b700 1 -- 172.21.15.121:0/919984465 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ac0a9830 con 0x7f09ac154ca0 2023-11-15T03:11:16.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.060+0000 7f09b231b700 1 -- 172.21.15.121:0/919984465 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ac0a9df0 con 0x7f09ac154ca0 2023-11-15T03:11:16.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.064+0000 7f09b231b700 1 -- 172.21.15.121:0/919984465 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09ac0aa010 con 0x7f09ac154ca0 2023-11-15T03:11:16.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.064+0000 7f09a8ff9700 1 -- 172.21.15.121:0/919984465 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f099401d640 con 0x7f09ac154ca0 2023-11-15T03:11:16.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.064+0000 7f09a8ff9700 1 --2- 172.21.15.121:0/919984465 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f098c05a510 0x7f098c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:16.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.064+0000 7f09a8ff9700 1 -- 172.21.15.121:0/919984465 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 8709+0+0 (secure 0 0 0) 0x7f09940767f0 con 0x7f09ac154ca0 2023-11-15T03:11:16.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.064+0000 7f09abfff700 1 --2- 172.21.15.121:0/919984465 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f098c05a510 0x7f098c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:16.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.068+0000 7f09abfff700 1 --2- 172.21.15.121:0/919984465 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f098c05a510 0x7f098c05c9d0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f099c0011e0 tx=0x7f099c001520).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:16.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.072+0000 7f09a8ff9700 1 -- 172.21.15.121:0/919984465 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0994049570 con 0x7f09ac154ca0 2023-11-15T03:11:16.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.264+0000 7f09b231b700 1 -- 172.21.15.121:0/919984465 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09ac0a6f50 con 0x7f09ac154ca0 2023-11-15T03:11:16.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.268+0000 7f09a8ff9700 1 -- 172.21.15.121:0/919984465 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 344 v344) v1 ==== 95+0+3541 (secure 0 0 0) 0x7f099404cb90 con 0x7f09ac154ca0 2023-11-15T03:11:16.275 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:16.276 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":344,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/242164468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":242164468},{"type":"v1","addr":"172.21.15.190:6833","nonce":242164468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6838,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3022099459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3022099459},{"type":"v1","addr":"172.21.15.121:6837","nonce":3022099459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4146539180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4146539180},{"type":"v1","addr":"172.21.15.190:6835","nonce":4146539180}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":10542,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2943522178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2943522178},{"type":"v1","addr":"172.21.15.121:6836","nonce":2943522178}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":342}],"filesystems":[]} 2023-11-15T03:11:16.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.268+0000 7f09b231b700 1 -- 172.21.15.121:0/919984465 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f098c05a510 msgr2=0x7f098c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:16.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.268+0000 7f09b231b700 1 --2- 172.21.15.121:0/919984465 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f098c05a510 0x7f098c05c9d0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f099c0011e0 tx=0x7f099c001520).stop 2023-11-15T03:11:16.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.268+0000 7f09b231b700 1 -- 172.21.15.121:0/919984465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09ac154ca0 msgr2=0x7f09ac0662b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:16.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.268+0000 7f09b231b700 1 --2- 172.21.15.121:0/919984465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09ac154ca0 0x7f09ac0662b0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 rx=0x7f09ac0c0d10 tx=0x7f0994016000).stop 2023-11-15T03:11:16.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.268+0000 7f09b231b700 1 -- 172.21.15.121:0/919984465 shutdown_connections 2023-11-15T03:11:16.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.268+0000 7f09b231b700 1 --2- 172.21.15.121:0/919984465 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f098c05a510 0x7f098c05c9d0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:16.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.268+0000 7f09b231b700 1 --2- 172.21.15.121:0/919984465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09ac1562f0 0x7f09ac0667a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:16.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.268+0000 7f09b231b700 1 --2- 172.21.15.121:0/919984465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09ac154ca0 0x7f09ac0662b0 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:16.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.268+0000 7f09b231b700 1 --2- 172.21.15.121:0/919984465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09ac0a57c0 0x7f09ac14d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:16.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.268+0000 7f09b231b700 1 -- 172.21.15.121:0/919984465 >> 172.21.15.121:0/919984465 conn(0x7f09ac062190 msgr2=0x7f09ac14e210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:16.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.268+0000 7f09b231b700 1 -- 172.21.15.121:0/919984465 shutdown_connections 2023-11-15T03:11:16.280 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.268+0000 7f09b231b700 1 -- 172.21.15.121:0/919984465 wait complete. 2023-11-15T03:11:16.281 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 344 2023-11-15T03:11:16.299 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-15T03:11:16.300 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-15T03:11:16.300 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-15T03:11:16.301 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:11:16.301 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-15T03:11:16.301 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-15T03:11:16.302 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-15T03:11:16.302 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:11:16.302 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-15T03:11:16.303 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-15T03:11:16.303 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-15T03:11:16.303 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:11:16.304 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-15T03:11:16.304 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-15T03:11:16.304 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-15T03:11:16.305 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:11:16.320 INFO:tasks.ceph.mds.c:Stopped 2023-11-15T03:11:16.320 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-15T03:11:16.324 INFO:tasks.ceph.mds.a:Stopped 2023-11-15T03:11:16.324 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-15T03:11:16.329 INFO:tasks.ceph.mds.b:Stopped 2023-11-15T03:11:16.329 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-15T03:11:16.330 INFO:tasks.ceph.mds.d:Stopped 2023-11-15T03:11:16.331 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-15T03:11:16.373 INFO:tasks.ceph.mds.c:Started 2023-11-15T03:11:16.373 DEBUG:teuthology.parallel:result is None 2023-11-15T03:11:16.379 INFO:tasks.ceph.mds.a:Started 2023-11-15T03:11:16.379 DEBUG:teuthology.parallel:result is None 2023-11-15T03:11:16.381 INFO:tasks.ceph.mds.b:Started 2023-11-15T03:11:16.382 DEBUG:teuthology.parallel:result is None 2023-11-15T03:11:16.385 INFO:tasks.ceph.mds.d:Started 2023-11-15T03:11:16.386 DEBUG:teuthology.parallel:result is None 2023-11-15T03:11:16.386 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-15T03:11:16.473 INFO:tasks.ceph.mds.c.smithi121.stdout:starting mds.c at 2023-11-15T03:11:16.482 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:11:16.495 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-11-15T03:11:16.495 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-11-15T03:11:16.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.532+0000 7f107549b700 1 Processor -- start 2023-11-15T03:11:16.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.536+0000 7f107549b700 1 -- start start 2023-11-15T03:11:16.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.536+0000 7f107549b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10700aab50 0x7f10700aa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:16.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.536+0000 7f107549b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1070153dc0 0x7f10700c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:16.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.536+0000 7f107549b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1070065740 con 0x7f10701539e0 2023-11-15T03:11:16.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.536+0000 7f107549b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1070154270 con 0x7f1070153dc0 2023-11-15T03:11:16.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.536+0000 7f107549b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10700c78d0 con 0x7f10700aab50 2023-11-15T03:11:16.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.536+0000 7f106e7fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10700aab50 0x7f10700aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:16.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.536+0000 7f106f7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1070153dc0 0x7f10700c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:16.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.536+0000 7f106e7fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10700aab50 0x7f10700aa070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:39850/0 (socket says 172.21.15.121:39850) 2023-11-15T03:11:16.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.536+0000 7f106e7fc700 1 -- 172.21.15.121:0/2994283253 learned_addr learned my addr 172.21.15.121:0/2994283253 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:16.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.536+0000 7f106effd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f10701539e0 0x7f10700c7010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:60886/0 (socket says 172.21.15.121:60886) 2023-11-15T03:11:16.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.540+0000 7f106e7fc700 1 -- 172.21.15.121:0/2994283253 >> v2:172.21.15.190:3300/0 conn(0x7f1070153dc0 msgr2=0x7f10700c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:16.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.540+0000 7f106e7fc700 1 --2- 172.21.15.121:0/2994283253 >> v2:172.21.15.190:3300/0 conn(0x7f1070153dc0 0x7f10700c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:16.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.540+0000 7f106e7fc700 1 -- 172.21.15.121:0/2994283253 >> v1:172.21.15.190:6789/0 conn(0x7f10701539e0 legacy=0x7f10700c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:16.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.540+0000 7f106e7fc700 1 -- 172.21.15.121:0/2994283253 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10700c7bb0 con 0x7f10700aab50 2023-11-15T03:11:16.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.540+0000 7f106e7fc700 1 --2- 172.21.15.121:0/2994283253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10700aab50 0x7f10700aa070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f1064006fd0 tx=0x7f10640090f0).ready entity=mon.2 client_cookie=1ffe5f06ea286f7e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:16.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.540+0000 7f106dffb700 1 -- 172.21.15.121:0/2994283253 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106400b580 con 0x7f10700aab50 2023-11-15T03:11:16.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.540+0000 7f106dffb700 1 -- 172.21.15.121:0/2994283253 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106400b6e0 con 0x7f10700aab50 2023-11-15T03:11:16.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.540+0000 7f106dffb700 1 -- 172.21.15.121:0/2994283253 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106400ba60 con 0x7f10700aab50 2023-11-15T03:11:16.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.540+0000 7f107549b700 1 -- 172.21.15.121:0/2994283253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10700aab50 msgr2=0x7f10700aa070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:16.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.540+0000 7f107549b700 1 --2- 172.21.15.121:0/2994283253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10700aab50 0x7f10700aa070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f1064006fd0 tx=0x7f10640090f0).stop 2023-11-15T03:11:16.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.540+0000 7f107549b700 1 -- 172.21.15.121:0/2994283253 shutdown_connections 2023-11-15T03:11:16.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.540+0000 7f107549b700 1 --2- 172.21.15.121:0/2994283253 >> v2:172.21.15.190:3300/0 conn(0x7f1070153dc0 0x7f10700c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:16.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.540+0000 7f107549b700 1 --2- 172.21.15.121:0/2994283253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10700aab50 0x7f10700aa070 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:16.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.540+0000 7f107549b700 1 -- 172.21.15.121:0/2994283253 >> 172.21.15.121:0/2994283253 conn(0x7f1070063670 msgr2=0x7f1070063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:16.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.540+0000 7f107549b700 1 -- 172.21.15.121:0/2994283253 shutdown_connections 2023-11-15T03:11:16.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.540+0000 7f107549b700 1 -- 172.21.15.121:0/2994283253 wait complete. 2023-11-15T03:11:16.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f107549b700 1 Processor -- start 2023-11-15T03:11:16.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f107549b700 1 -- start start 2023-11-15T03:11:16.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f107549b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10700aab50 0x7f1070152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:16.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f107549b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10701539e0 0x7f10701648c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:16.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f107549b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1070153dc0 0x7f10700c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:16.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f107549b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1070153090 con 0x7f10700aab50 2023-11-15T03:11:16.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f107549b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107014dfc0 con 0x7f1070153dc0 2023-11-15T03:11:16.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f107549b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1070164db0 con 0x7f10701539e0 2023-11-15T03:11:16.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f106effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10700aab50 0x7f1070152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:16.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f106effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10700aab50 0x7f1070152ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44714/0 (socket says 172.21.15.121:44714) 2023-11-15T03:11:16.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f106effd700 1 -- 172.21.15.121:0/3993119626 learned_addr learned my addr 172.21.15.121:0/3993119626 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:16.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f106e7fc700 1 --2- 172.21.15.121:0/3993119626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10701539e0 0x7f10701648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:16.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f106f7fe700 1 --2- 172.21.15.121:0/3993119626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1070153dc0 0x7f10700c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:16.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f106effd700 1 -- 172.21.15.121:0/3993119626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10701539e0 msgr2=0x7f10701648c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:16.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f106effd700 1 --2- 172.21.15.121:0/3993119626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10701539e0 0x7f10701648c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:16.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f106effd700 1 -- 172.21.15.121:0/3993119626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1070153dc0 msgr2=0x7f10700c42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:16.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f106effd700 1 --2- 172.21.15.121:0/3993119626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1070153dc0 0x7f10700c42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:16.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f106effd700 1 -- 172.21.15.121:0/3993119626 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1064009370 con 0x7f10700aab50 2023-11-15T03:11:16.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f106e7fc700 1 --2- 172.21.15.121:0/3993119626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10701539e0 0x7f10701648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:11:16.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.544+0000 7f106f7fe700 1 --2- 172.21.15.121:0/3993119626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1070153dc0 0x7f10700c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:11:16.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.548+0000 7f106effd700 1 --2- 172.21.15.121:0/3993119626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10700aab50 0x7f1070152ba0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 rx=0x7f1060007fd0 tx=0x7f1060007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:16.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.548+0000 7f1056ffd700 1 -- 172.21.15.121:0/3993119626 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10600072e0 con 0x7f10700aab50 2023-11-15T03:11:16.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.548+0000 7f1056ffd700 1 -- 172.21.15.121:0/3993119626 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1060007440 con 0x7f10700aab50 2023-11-15T03:11:16.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.548+0000 7f1056ffd700 1 -- 172.21.15.121:0/3993119626 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10600058f0 con 0x7f10700aab50 2023-11-15T03:11:16.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.548+0000 7f107549b700 1 -- 172.21.15.121:0/3993119626 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10701544e0 con 0x7f10700aab50 2023-11-15T03:11:16.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.548+0000 7f107549b700 1 -- 172.21.15.121:0/3993119626 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1070165290 con 0x7f10700aab50 2023-11-15T03:11:16.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.548+0000 7f107549b700 1 -- 172.21.15.121:0/3993119626 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1070153250 con 0x7f10700aab50 2023-11-15T03:11:16.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.548+0000 7f1056ffd700 1 -- 172.21.15.121:0/3993119626 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1060005ce0 con 0x7f10700aab50 2023-11-15T03:11:16.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.552+0000 7f1056ffd700 1 --2- 172.21.15.121:0/3993119626 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f105c05a510 0x7f105c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:16.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.552+0000 7f1056ffd700 1 -- 172.21.15.121:0/3993119626 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8709+0+0 (secure 0 0 0) 0x7f1060074a80 con 0x7f10700aab50 2023-11-15T03:11:16.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.552+0000 7f106e7fc700 1 --2- 172.21.15.121:0/3993119626 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f105c05a510 0x7f105c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:16.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.552+0000 7f106e7fc700 1 --2- 172.21.15.121:0/3993119626 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f105c05a510 0x7f105c05c9d0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f1064006fd0 tx=0x7f1064017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:16.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.556+0000 7f1056ffd700 1 -- 172.21.15.121:0/3993119626 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1060047780 con 0x7f10700aab50 2023-11-15T03:11:16.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:16.736+0000 7f107549b700 1 -- 172.21.15.121:0/3993119626 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1070047060 con 0x7f10700aab50 2023-11-15T03:11:17.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.532+0000 7f1056ffd700 1 -- 172.21.15.121:0/3993119626 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v141) v1 ==== 97+0+0 (secure 0 0 0) 0x7f106004ada0 con 0x7f10700aab50 2023-11-15T03:11:17.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.536+0000 7f107549b700 1 -- 172.21.15.121:0/3993119626 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f105c05a510 msgr2=0x7f105c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:17.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.536+0000 7f107549b700 1 --2- 172.21.15.121:0/3993119626 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f105c05a510 0x7f105c05c9d0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f1064006fd0 tx=0x7f1064017000).stop 2023-11-15T03:11:17.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.536+0000 7f107549b700 1 -- 172.21.15.121:0/3993119626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10700aab50 msgr2=0x7f1070152ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:17.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.536+0000 7f107549b700 1 --2- 172.21.15.121:0/3993119626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10700aab50 0x7f1070152ba0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 rx=0x7f1060007fd0 tx=0x7f1060007df0).stop 2023-11-15T03:11:17.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.536+0000 7f107549b700 1 -- 172.21.15.121:0/3993119626 shutdown_connections 2023-11-15T03:11:17.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.536+0000 7f107549b700 1 --2- 172.21.15.121:0/3993119626 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f105c05a510 0x7f105c05c9d0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:17.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.536+0000 7f107549b700 1 --2- 172.21.15.121:0/3993119626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1070153dc0 0x7f10700c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:17.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.536+0000 7f107549b700 1 --2- 172.21.15.121:0/3993119626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10701539e0 0x7f10701648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:17.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.536+0000 7f107549b700 1 --2- 172.21.15.121:0/3993119626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10700aab50 0x7f1070152ba0 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:17.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.536+0000 7f107549b700 1 -- 172.21.15.121:0/3993119626 >> 172.21.15.121:0/3993119626 conn(0x7f1070063670 msgr2=0x7f10700aaf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:17.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.536+0000 7f107549b700 1 -- 172.21.15.121:0/3993119626 shutdown_connections 2023-11-15T03:11:17.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.536+0000 7f107549b700 1 -- 172.21.15.121:0/3993119626 wait complete. 2023-11-15T03:11:17.550 INFO:teuthology.orchestra.run.smithi121.stderr: removed all blocklist entries 2023-11-15T03:11:17.565 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-15T03:11:17.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.724+0000 7f42b359e700 1 Processor -- start 2023-11-15T03:11:17.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.724+0000 7f42b359e700 1 -- start start 2023-11-15T03:11:17.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.728+0000 7f42b359e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42b40c3280 0x7f42b40c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:17.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.728+0000 7f42b359e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f42b415be30 0x7f42b415c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:17.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.728+0000 7f42b359e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b40c1d50 con 0x7f42b4152140 2023-11-15T03:11:17.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.728+0000 7f42b359e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b40c1860 con 0x7f42b415be30 2023-11-15T03:11:17.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.728+0000 7f42b359e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b415cb80 con 0x7f42b40c3280 2023-11-15T03:11:17.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.728+0000 7f42b2d9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f42b415be30 0x7f42b415c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:17.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.728+0000 7f42b2d9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f42b415be30 0x7f42b415c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44722/0 (socket says 172.21.15.121:44722) 2023-11-15T03:11:17.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.728+0000 7f42b2d9d700 1 -- 172.21.15.121:0/1664026439 learned_addr learned my addr 172.21.15.121:0/1664026439 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:17.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.728+0000 7f42b259c700 1 --2- 172.21.15.121:0/1664026439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42b40c3280 0x7f42b40c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:17.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.728+0000 7f42b2d9d700 1 -- 172.21.15.121:0/1664026439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42b40c3280 msgr2=0x7f42b40c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:17.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.728+0000 7f42b2d9d700 1 --2- 172.21.15.121:0/1664026439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42b40c3280 0x7f42b40c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:17.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.728+0000 7f42b2d9d700 1 -- 172.21.15.121:0/1664026439 >> v1:172.21.15.190:6789/0 conn(0x7f42b4152140 legacy=0x7f42b41525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:17.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.728+0000 7f42b2d9d700 1 -- 172.21.15.121:0/1664026439 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b40a9240 con 0x7f42b415be30 2023-11-15T03:11:17.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.728+0000 7f42b259c700 1 --2- 172.21.15.121:0/1664026439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42b40c3280 0x7f42b40c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:17.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.732+0000 7f42b2d9d700 1 --2- 172.21.15.121:0/1664026439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42b415be30 0x7f42b415c2a0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 rx=0x7f42ac009f20 tx=0x7f42ac009840).ready entity=mon.0 client_cookie=a66c1f55ef4cc8c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:17.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.732+0000 7f42b159a700 1 -- 172.21.15.121:0/1664026439 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00e070 con 0x7f42b415be30 2023-11-15T03:11:17.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.732+0000 7f42b159a700 1 -- 172.21.15.121:0/1664026439 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac00ba80 con 0x7f42b415be30 2023-11-15T03:11:17.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.732+0000 7f42b159a700 1 -- 172.21.15.121:0/1664026439 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00be00 con 0x7f42b415be30 2023-11-15T03:11:17.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.732+0000 7f42b359e700 1 -- 172.21.15.121:0/1664026439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42b415be30 msgr2=0x7f42b415c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:17.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.732+0000 7f42b359e700 1 --2- 172.21.15.121:0/1664026439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42b415be30 0x7f42b415c2a0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 rx=0x7f42ac009f20 tx=0x7f42ac009840).stop 2023-11-15T03:11:17.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.732+0000 7f42b359e700 1 -- 172.21.15.121:0/1664026439 shutdown_connections 2023-11-15T03:11:17.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.732+0000 7f42b359e700 1 --2- 172.21.15.121:0/1664026439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42b415be30 0x7f42b415c2a0 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:17.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.732+0000 7f42b359e700 1 --2- 172.21.15.121:0/1664026439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42b40c3280 0x7f42b40c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:17.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.732+0000 7f42b359e700 1 -- 172.21.15.121:0/1664026439 >> 172.21.15.121:0/1664026439 conn(0x7f42b4062190 msgr2=0x7f42b4062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:17.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.732+0000 7f42b359e700 1 -- 172.21.15.121:0/1664026439 shutdown_connections 2023-11-15T03:11:17.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.732+0000 7f42b359e700 1 -- 172.21.15.121:0/1664026439 wait complete. 2023-11-15T03:11:17.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.736+0000 7f42b359e700 1 Processor -- start 2023-11-15T03:11:17.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.736+0000 7f42b359e700 1 -- start start 2023-11-15T03:11:17.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.736+0000 7f42b359e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42b40c3280 0x7f42b4168c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:17.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.736+0000 7f42b359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42b4152140 0x7f42b416afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:17.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.736+0000 7f42b359e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42b415be30 0x7f42b40666d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:17.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.736+0000 7f42b359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b40bc160 con 0x7f42b4152140 2023-11-15T03:11:17.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.736+0000 7f42b359e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b40c7550 con 0x7f42b40c3280 2023-11-15T03:11:17.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.736+0000 7f42b359e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b4169110 con 0x7f42b415be30 2023-11-15T03:11:17.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.740+0000 7f42b259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42b40c3280 0x7f42b4168c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:17.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.740+0000 7f42b259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42b40c3280 0x7f42b4168c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:38994/0 (socket says 172.21.15.121:38994) 2023-11-15T03:11:17.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.740+0000 7f42b259c700 1 -- 172.21.15.121:0/151264810 learned_addr learned my addr 172.21.15.121:0/151264810 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:17.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.740+0000 7f42b1d9b700 1 --2- 172.21.15.121:0/151264810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42b4152140 0x7f42b416afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:17.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.740+0000 7f42b2d9d700 1 --2- 172.21.15.121:0/151264810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42b415be30 0x7f42b40666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:17.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.740+0000 7f42b259c700 1 -- 172.21.15.121:0/151264810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42b415be30 msgr2=0x7f42b40666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:17.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.740+0000 7f42b259c700 1 --2- 172.21.15.121:0/151264810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42b415be30 0x7f42b40666d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:17.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.740+0000 7f42b259c700 1 -- 172.21.15.121:0/151264810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42b4152140 msgr2=0x7f42b416afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:17.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.740+0000 7f42b259c700 1 --2- 172.21.15.121:0/151264810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42b4152140 0x7f42b416afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:17.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.740+0000 7f42b259c700 1 -- 172.21.15.121:0/151264810 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac009ae0 con 0x7f42b40c3280 2023-11-15T03:11:17.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.740+0000 7f42b259c700 1 --2- 172.21.15.121:0/151264810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42b40c3280 0x7f42b4168c20 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f42a400d890 tx=0x7f42a4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:17.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.740+0000 7f42a27fc700 1 -- 172.21.15.121:0/151264810 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a4011590 con 0x7f42b40c3280 2023-11-15T03:11:17.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.740+0000 7f42a27fc700 1 -- 172.21.15.121:0/151264810 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a40116f0 con 0x7f42b40c3280 2023-11-15T03:11:17.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.740+0000 7f42b359e700 1 -- 172.21.15.121:0/151264810 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b416bb70 con 0x7f42b40c3280 2023-11-15T03:11:17.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.740+0000 7f42a27fc700 1 -- 172.21.15.121:0/151264810 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a4011a70 con 0x7f42b40c3280 2023-11-15T03:11:17.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.740+0000 7f42b359e700 1 -- 172.21.15.121:0/151264810 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b416c130 con 0x7f42b40c3280 2023-11-15T03:11:17.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.740+0000 7f42b359e700 1 -- 172.21.15.121:0/151264810 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b406bcf0 con 0x7f42b40c3280 2023-11-15T03:11:17.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.740+0000 7f42a27fc700 1 -- 172.21.15.121:0/151264810 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f42a401c460 con 0x7f42b40c3280 2023-11-15T03:11:17.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.744+0000 7f42a27fc700 1 --2- 172.21.15.121:0/151264810 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f429405a510 0x7f429405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:17.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.744+0000 7f42a27fc700 1 -- 172.21.15.121:0/151264810 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f42a4074290 con 0x7f42b40c3280 2023-11-15T03:11:17.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.744+0000 7f42b1d9b700 1 --2- 172.21.15.121:0/151264810 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f429405a510 0x7f429405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:17.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.744+0000 7f42b1d9b700 1 --2- 172.21.15.121:0/151264810 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f429405a510 0x7f429405c9d0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7f42b40c0530 tx=0x7f42a8007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:17.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.748+0000 7f42a27fc700 1 -- 172.21.15.121:0/151264810 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f42a4047ca0 con 0x7f42b40c3280 2023-11-15T03:11:17.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.968+0000 7f42b359e700 1 -- 172.21.15.121:0/151264810 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f42b4068a70 con 0x7f42b40c3280 2023-11-15T03:11:17.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.968+0000 7f42a27fc700 1 -- 172.21.15.121:0/151264810 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f42a404b2c0 con 0x7f42b40c3280 2023-11-15T03:11:17.977 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:17.977 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:11:17.978 INFO:teuthology.orchestra.run.smithi121.stdout: "auth_dump": [ 2023-11-15T03:11:17.978 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:17.978 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.a", 2023-11-15T03:11:17.978 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1Rlzy3zOhAAeJ3COoWQP/Kw+P/0AwHbwA==", 2023-11-15T03:11:17.978 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:11:17.979 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:11:17.979 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T03:11:17.979 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T03:11:17.979 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:11:17.980 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:17.980 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:17.980 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:17.980 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.b", 2023-11-15T03:11:17.981 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlUglaERAAC6WUS9ftCfr+KbPpiEIyqw==", 2023-11-15T03:11:17.981 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:11:17.981 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:11:17.981 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T03:11:17.982 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T03:11:17.982 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:11:17.982 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:17.982 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:17.983 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:17.983 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.c", 2023-11-15T03:11:17.983 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RldFcvCRAATIouBNyHGip5hFrH+z4v5A==", 2023-11-15T03:11:17.983 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:11:17.983 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:11:17.984 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T03:11:17.984 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T03:11:17.984 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:11:17.985 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:17.985 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:17.986 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:17.986 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.d", 2023-11-15T03:11:17.986 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1Rlg5iZGxAADwdL2RrtU0kDeb9EmNJU6A==", 2023-11-15T03:11:17.987 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:11:17.987 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:11:17.987 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T03:11:17.987 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T03:11:17.987 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:11:17.988 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:17.988 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:17.988 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:17.989 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.0", 2023-11-15T03:11:17.989 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAzM1RldPH9CBAAUCF/a47aFbbNsWV1r+u3Tw==", 2023-11-15T03:11:17.989 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:11:17.989 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:11:17.989 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:11:17.990 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:11:17.990 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:17.990 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:17.990 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:17.991 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.1", 2023-11-15T03:11:17.991 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQA2M1RltoxwMhAAZN3bgOjSaoK3ZxE5ezZ3IA==", 2023-11-15T03:11:17.991 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:11:17.991 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:11:17.992 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:11:17.992 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:11:17.992 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:17.992 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:17.992 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:17.992 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.2", 2023-11-15T03:11:17.993 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQA6M1RlgltjHxAAx/m5gm+qs8LiEFyZL+uLzQ==", 2023-11-15T03:11:17.993 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:11:17.993 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:11:17.993 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:11:17.993 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:11:17.994 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:17.994 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:17.994 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:17.994 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.3", 2023-11-15T03:11:17.995 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQA+M1RlhtQcDRAA+g11PmSNKazpk+VoK17EEQ==", 2023-11-15T03:11:17.995 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:11:17.995 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:11:17.995 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:11:17.995 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:11:17.996 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:17.996 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:17.996 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:17.996 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.4", 2023-11-15T03:11:17.997 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBFM1RlJ7hnBBAAjskKfD63N/61T4GSPqOiRg==", 2023-11-15T03:11:17.997 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:11:17.997 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:11:17.997 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:11:17.997 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:11:17.998 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:17.998 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:17.998 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:17.998 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.5", 2023-11-15T03:11:17.998 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBIM1RlI1HPLRAAgVbiNq1tmgK6PaWP+c2JFw==", 2023-11-15T03:11:17.999 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:11:17.999 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:11:17.999 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:11:17.999 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:11:17.999 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:17.999 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:18.000 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:18.000 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.6", 2023-11-15T03:11:18.000 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBMM1RlPDshDRAAVgIqxklHSQcFb5sM6GUfyQ==", 2023-11-15T03:11:18.000 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:11:18.000 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:11:18.001 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:11:18.001 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:11:18.001 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:18.001 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:18.001 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:18.002 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.7", 2023-11-15T03:11:18.002 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQBPM1RlN8IuJxAA2rN2848kaB2eOM6zDb+eJg==", 2023-11-15T03:11:18.002 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:11:18.002 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2023-11-15T03:11:18.002 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2023-11-15T03:11:18.003 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:11:18.003 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:18.003 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:18.003 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:18.003 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.0", 2023-11-15T03:11:18.004 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlfHPEIRAAJ4YV46zzBQ7v7y0kD4P2ZA==", 2023-11-15T03:11:18.004 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:11:18.006 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:11:18.006 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow r", 2023-11-15T03:11:18.006 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-15T03:11:18.007 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:18.007 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:18.007 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:18.007 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.1", 2023-11-15T03:11:18.008 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlibaULBAA+y1EyhVOt+wXSk2uRajJ3g==", 2023-11-15T03:11:18.008 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:11:18.008 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:11:18.008 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2023-11-15T03:11:18.009 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2023-11-15T03:11:18.009 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2023-11-15T03:11:18.010 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:18.010 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:18.010 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:18.011 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.2", 2023-11-15T03:11:18.011 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlHHbvJhAAX5TE6UvcpBeJPSCZkfqD8w==", 2023-11-15T03:11:18.011 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:11:18.012 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:11:18.012 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2023-11-15T03:11:18.012 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2023-11-15T03:11:18.012 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2023-11-15T03:11:18.012 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:18.013 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:18.013 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:18.013 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.3", 2023-11-15T03:11:18.013 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAvM1RlQtQXMhAAGAvtnvJLSoUKuEiZDsSxww==", 2023-11-15T03:11:18.013 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:11:18.014 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2023-11-15T03:11:18.014 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2023-11-15T03:11:18.014 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2023-11-15T03:11:18.014 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2023-11-15T03:11:18.014 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:18.015 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:18.015 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:18.015 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.admin", 2023-11-15T03:11:18.015 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1Rl67qqEhAAI48kZ9D7T807bTVoUmwajg==", 2023-11-15T03:11:18.015 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:11:18.016 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2023-11-15T03:11:18.016 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2023-11-15T03:11:18.016 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2023-11-15T03:11:18.016 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:11:18.016 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:18.017 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:18.017 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:18.017 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mgr.x", 2023-11-15T03:11:18.017 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1RlJQFPNRAAy7E5mrn8Xh98339fWSkX9Q==", 2023-11-15T03:11:18.017 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:11:18.018 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2023-11-15T03:11:18.018 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile mgr", 2023-11-15T03:11:18.018 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:11:18.018 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:18.018 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:18.019 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:18.019 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mgr.y", 2023-11-15T03:11:18.019 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQAuM1RldQJlLhAAbc46liEP03yBNeVhZCb7IQ==", 2023-11-15T03:11:18.019 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2023-11-15T03:11:18.019 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2023-11-15T03:11:18.020 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile mgr", 2023-11-15T03:11:18.020 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2023-11-15T03:11:18.020 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:18.020 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:18.021 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2023-11-15T03:11:18.021 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:11:18.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.972+0000 7f42b359e700 1 -- 172.21.15.121:0/151264810 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f429405a510 msgr2=0x7f429405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:18.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.972+0000 7f42b359e700 1 --2- 172.21.15.121:0/151264810 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f429405a510 0x7f429405c9d0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7f42b40c0530 tx=0x7f42a8007d40).stop 2023-11-15T03:11:18.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.972+0000 7f42b359e700 1 -- 172.21.15.121:0/151264810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42b40c3280 msgr2=0x7f42b4168c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:18.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.972+0000 7f42b359e700 1 --2- 172.21.15.121:0/151264810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42b40c3280 0x7f42b4168c20 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f42a400d890 tx=0x7f42a4008040).stop 2023-11-15T03:11:18.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.972+0000 7f42b359e700 1 -- 172.21.15.121:0/151264810 shutdown_connections 2023-11-15T03:11:18.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.972+0000 7f42b359e700 1 --2- 172.21.15.121:0/151264810 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f429405a510 0x7f429405c9d0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.972+0000 7f42b359e700 1 --2- 172.21.15.121:0/151264810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42b415be30 0x7f42b40666d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.972+0000 7f42b359e700 1 --2- 172.21.15.121:0/151264810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42b4152140 0x7f42b416afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.972+0000 7f42b359e700 1 --2- 172.21.15.121:0/151264810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42b40c3280 0x7f42b4168c20 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.972+0000 7f42b359e700 1 -- 172.21.15.121:0/151264810 >> 172.21.15.121:0/151264810 conn(0x7f42b4062190 msgr2=0x7f42b414f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:18.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.972+0000 7f42b359e700 1 -- 172.21.15.121:0/151264810 shutdown_connections 2023-11-15T03:11:18.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:17.972+0000 7f42b359e700 1 -- 172.21.15.121:0/151264810 wait complete. 2023-11-15T03:11:18.025 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:11:18.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.176+0000 7f4e865b9700 1 Processor -- start 2023-11-15T03:11:18.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.176+0000 7f4e865b9700 1 -- start start 2023-11-15T03:11:18.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.180+0000 7f4e865b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4e80152850 0x7f4e800c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:18.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.180+0000 7f4e865b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4e8015be30 0x7f4e8015c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:18.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.180+0000 7f4e865b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e800c1d50 con 0x7f4e800c3280 2023-11-15T03:11:18.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.180+0000 7f4e865b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e800c1860 con 0x7f4e8015be30 2023-11-15T03:11:18.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.180+0000 7f4e865b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8015cb80 con 0x7f4e80152850 2023-11-15T03:11:18.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.180+0000 7f4e85db8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4e8015be30 0x7f4e8015c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:18.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.180+0000 7f4e85db8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4e8015be30 0x7f4e8015c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44738/0 (socket says 172.21.15.121:44738) 2023-11-15T03:11:18.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.180+0000 7f4e85db8700 1 -- 172.21.15.121:0/885816417 learned_addr learned my addr 172.21.15.121:0/885816417 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:18.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.180+0000 7f4e84db6700 1 --2- 172.21.15.121:0/885816417 >> v2:172.21.15.190:3300/0 conn(0x7f4e80152850 0x7f4e800c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:18.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.180+0000 7f4e85db8700 1 -- 172.21.15.121:0/885816417 >> v2:172.21.15.190:3300/0 conn(0x7f4e80152850 msgr2=0x7f4e800c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:18.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.180+0000 7f4e85db8700 1 --2- 172.21.15.121:0/885816417 >> v2:172.21.15.190:3300/0 conn(0x7f4e80152850 0x7f4e800c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.180+0000 7f4e85db8700 1 -- 172.21.15.121:0/885816417 >> v1:172.21.15.121:6789/0 conn(0x7f4e800c3280 legacy=0x7f4e80152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:18.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.180+0000 7f4e85db8700 1 -- 172.21.15.121:0/885816417 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e800a9240 con 0x7f4e8015be30 2023-11-15T03:11:18.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.180+0000 7f4e85db8700 1 --2- 172.21.15.121:0/885816417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e8015be30 0x7f4e8015c280 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 rx=0x7f4e80156230 tx=0x7f4e78009840).ready entity=mon.0 client_cookie=c221cf73eaf16981 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:18.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.180+0000 7f4e77fff700 1 -- 172.21.15.121:0/885816417 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7800e070 con 0x7f4e8015be30 2023-11-15T03:11:18.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.180+0000 7f4e77fff700 1 -- 172.21.15.121:0/885816417 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7800ba20 con 0x7f4e8015be30 2023-11-15T03:11:18.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.180+0000 7f4e77fff700 1 -- 172.21.15.121:0/885816417 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7800bda0 con 0x7f4e8015be30 2023-11-15T03:11:18.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.184+0000 7f4e865b9700 1 -- 172.21.15.121:0/885816417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e8015be30 msgr2=0x7f4e8015c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:18.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.184+0000 7f4e865b9700 1 --2- 172.21.15.121:0/885816417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e8015be30 0x7f4e8015c280 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 rx=0x7f4e80156230 tx=0x7f4e78009840).stop 2023-11-15T03:11:18.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.184+0000 7f4e865b9700 1 -- 172.21.15.121:0/885816417 shutdown_connections 2023-11-15T03:11:18.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.184+0000 7f4e865b9700 1 --2- 172.21.15.121:0/885816417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e8015be30 0x7f4e8015c280 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.184+0000 7f4e865b9700 1 --2- 172.21.15.121:0/885816417 >> v2:172.21.15.190:3300/0 conn(0x7f4e80152850 0x7f4e800c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.184+0000 7f4e865b9700 1 -- 172.21.15.121:0/885816417 >> 172.21.15.121:0/885816417 conn(0x7f4e80062190 msgr2=0x7f4e80062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:18.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.184+0000 7f4e865b9700 1 -- 172.21.15.121:0/885816417 shutdown_connections 2023-11-15T03:11:18.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.184+0000 7f4e865b9700 1 -- 172.21.15.121:0/885816417 wait complete. 2023-11-15T03:11:18.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.184+0000 7f4e865b9700 1 Processor -- start 2023-11-15T03:11:18.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.184+0000 7f4e865b9700 1 -- start start 2023-11-15T03:11:18.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e865b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e800c3280 0x7f4e80155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:18.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e865b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e80152850 0x7f4e80168750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:18.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e865b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e8015be30 0x7f4e8014d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:18.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e865b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e80155830 con 0x7f4e800c3280 2023-11-15T03:11:18.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e865b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8016c110 con 0x7f4e8015be30 2023-11-15T03:11:18.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e865b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8016c3f0 con 0x7f4e80152850 2023-11-15T03:11:18.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e855b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e800c3280 0x7f4e80155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:18.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e855b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e800c3280 0x7f4e80155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44742/0 (socket says 172.21.15.121:44742) 2023-11-15T03:11:18.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e855b7700 1 -- 172.21.15.121:0/3568525687 learned_addr learned my addr 172.21.15.121:0/3568525687 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:18.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e85db8700 1 --2- 172.21.15.121:0/3568525687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e8015be30 0x7f4e8014d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:18.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e84db6700 1 --2- 172.21.15.121:0/3568525687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e80152850 0x7f4e80168750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:18.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e855b7700 1 -- 172.21.15.121:0/3568525687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e80152850 msgr2=0x7f4e80168750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:18.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e855b7700 1 --2- 172.21.15.121:0/3568525687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e80152850 0x7f4e80168750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e855b7700 1 -- 172.21.15.121:0/3568525687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e8015be30 msgr2=0x7f4e8014d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:18.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e855b7700 1 --2- 172.21.15.121:0/3568525687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e8015be30 0x7f4e8014d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e855b7700 1 -- 172.21.15.121:0/3568525687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e78009ae0 con 0x7f4e800c3280 2023-11-15T03:11:18.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e855b7700 1 --2- 172.21.15.121:0/3568525687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e800c3280 0x7f4e80155340 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 rx=0x7f4e7c00c440 tx=0x7f4e7c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:18.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e757fa700 1 -- 172.21.15.121:0/3568525687 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c00ce40 con 0x7f4e800c3280 2023-11-15T03:11:18.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e757fa700 1 -- 172.21.15.121:0/3568525687 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c007020 con 0x7f4e800c3280 2023-11-15T03:11:18.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e865b9700 1 -- 172.21.15.121:0/3568525687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8016c6d0 con 0x7f4e800c3280 2023-11-15T03:11:18.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e757fa700 1 -- 172.21.15.121:0/3568525687 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0073e0 con 0x7f4e800c3280 2023-11-15T03:11:18.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.188+0000 7f4e865b9700 1 -- 172.21.15.121:0/3568525687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e80154f40 con 0x7f4e800c3280 2023-11-15T03:11:18.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.192+0000 7f4e865b9700 1 -- 172.21.15.121:0/3568525687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e8006bcf0 con 0x7f4e800c3280 2023-11-15T03:11:18.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.192+0000 7f4e757fa700 1 -- 172.21.15.121:0/3568525687 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4e7c0068b0 con 0x7f4e800c3280 2023-11-15T03:11:18.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.192+0000 7f4e757fa700 1 --2- 172.21.15.121:0/3568525687 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4e6405a510 0x7f4e6405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:18.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.192+0000 7f4e757fa700 1 -- 172.21.15.121:0/3568525687 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f4e7c073720 con 0x7f4e800c3280 2023-11-15T03:11:18.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.192+0000 7f4e84db6700 1 --2- 172.21.15.121:0/3568525687 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4e6405a510 0x7f4e6405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:18.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.192+0000 7f4e84db6700 1 --2- 172.21.15.121:0/3568525687 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4e6405a510 0x7f4e6405c9d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7f4e800c7690 tx=0x7f4e70008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:18.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.200+0000 7f4e757fa700 1 -- 172.21.15.121:0/3568525687 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4e7c047190 con 0x7f4e800c3280 2023-11-15T03:11:18.375 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.368+0000 7f4e865b9700 1 -- 172.21.15.121:0/3568525687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e800a7cc0 con 0x7f4e800c3280 2023-11-15T03:11:18.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.368+0000 7f4e757fa700 1 -- 172.21.15.121:0/3568525687 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+13762 (secure 0 0 0) 0x7f4e7c04a7b0 con 0x7f4e800c3280 2023-11-15T03:11:18.377 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:18.377 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":141,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:11:17.527233+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:11:18.379 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.372+0000 7f4e865b9700 1 -- 172.21.15.121:0/3568525687 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4e6405a510 msgr2=0x7f4e6405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:18.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.372+0000 7f4e865b9700 1 --2- 172.21.15.121:0/3568525687 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4e6405a510 0x7f4e6405c9d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7f4e800c7690 tx=0x7f4e70008040).stop 2023-11-15T03:11:18.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.372+0000 7f4e865b9700 1 -- 172.21.15.121:0/3568525687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e800c3280 msgr2=0x7f4e80155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:18.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.372+0000 7f4e865b9700 1 --2- 172.21.15.121:0/3568525687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e800c3280 0x7f4e80155340 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 rx=0x7f4e7c00c440 tx=0x7f4e7c007e00).stop 2023-11-15T03:11:18.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.372+0000 7f4e865b9700 1 -- 172.21.15.121:0/3568525687 shutdown_connections 2023-11-15T03:11:18.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.372+0000 7f4e865b9700 1 --2- 172.21.15.121:0/3568525687 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4e6405a510 0x7f4e6405c9d0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.372+0000 7f4e865b9700 1 --2- 172.21.15.121:0/3568525687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e8015be30 0x7f4e8014d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.372+0000 7f4e865b9700 1 --2- 172.21.15.121:0/3568525687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e80152850 0x7f4e80168750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.372+0000 7f4e865b9700 1 --2- 172.21.15.121:0/3568525687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e800c3280 0x7f4e80155340 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.372+0000 7f4e865b9700 1 -- 172.21.15.121:0/3568525687 >> 172.21.15.121:0/3568525687 conn(0x7f4e80062190 msgr2=0x7f4e8014f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:18.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.372+0000 7f4e865b9700 1 -- 172.21.15.121:0/3568525687 shutdown_connections 2023-11-15T03:11:18.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.372+0000 7f4e865b9700 1 -- 172.21.15.121:0/3568525687 wait complete. 2023-11-15T03:11:18.401 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-15T03:11:18.402 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T03:11:18.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.568+0000 7fdf0bfff700 1 Processor -- start 2023-11-15T03:11:18.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.568+0000 7fdf0bfff700 1 -- start start 2023-11-15T03:11:18.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.568+0000 7fdf0bfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdf0c0acd00 0x7fdf0c0ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:18.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.568+0000 7fdf0bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdf0c0c5c70 0x7fdf0c1536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:18.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.568+0000 7fdf0bfff700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0c065ea0 con 0x7fdf0c066330 2023-11-15T03:11:18.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.568+0000 7fdf0bfff700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0c066bb0 con 0x7fdf0c0c5c70 2023-11-15T03:11:18.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.568+0000 7fdf0bfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0c065890 con 0x7fdf0c0acd00 2023-11-15T03:11:18.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.568+0000 7fdf0b7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fdf0c066330 0x7fdf0c153bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:35854/0 (socket says 172.21.15.121:35854) 2023-11-15T03:11:18.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.568+0000 7fdf0b7fe700 1 -- 172.21.15.121:0/3387695166 learned_addr learned my addr 172.21.15.121:0/3387695166 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:18.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.568+0000 7fdf0a7fc700 1 --2- 172.21.15.121:0/3387695166 >> v2:172.21.15.121:3300/0 conn(0x7fdf0c0c5c70 0x7fdf0c1536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:18.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.572+0000 7fdf0affd700 1 --2- 172.21.15.121:0/3387695166 >> v2:172.21.15.190:3300/0 conn(0x7fdf0c0acd00 0x7fdf0c0ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:18.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.572+0000 7fdf0affd700 1 -- 172.21.15.121:0/3387695166 >> v2:172.21.15.121:3300/0 conn(0x7fdf0c0c5c70 msgr2=0x7fdf0c1536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:18.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.572+0000 7fdf0affd700 1 --2- 172.21.15.121:0/3387695166 >> v2:172.21.15.121:3300/0 conn(0x7fdf0c0c5c70 0x7fdf0c1536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.572+0000 7fdf0affd700 1 -- 172.21.15.121:0/3387695166 >> v1:172.21.15.121:6789/0 conn(0x7fdf0c066330 legacy=0x7fdf0c153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:18.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.572+0000 7fdf0affd700 1 -- 172.21.15.121:0/3387695166 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c0c64a0 con 0x7fdf0c0acd00 2023-11-15T03:11:18.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.572+0000 7fdf0affd700 1 --2- 172.21.15.121:0/3387695166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf0c0acd00 0x7fdf0c0ad0e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7fdf0c06c280 tx=0x7fdefc006f40).ready entity=mon.1 client_cookie=ebd16a0853cec3fc server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:18.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.572+0000 7fdf09ffb700 1 -- 172.21.15.121:0/3387695166 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc005880 con 0x7fdf0c0acd00 2023-11-15T03:11:18.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.572+0000 7fdf09ffb700 1 -- 172.21.15.121:0/3387695166 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdefc0059e0 con 0x7fdf0c0acd00 2023-11-15T03:11:18.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.572+0000 7fdf09ffb700 1 -- 172.21.15.121:0/3387695166 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc005d60 con 0x7fdf0c0acd00 2023-11-15T03:11:18.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.572+0000 7fdf0bfff700 1 -- 172.21.15.121:0/3387695166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf0c0acd00 msgr2=0x7fdf0c0ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:18.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.572+0000 7fdf0bfff700 1 --2- 172.21.15.121:0/3387695166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf0c0acd00 0x7fdf0c0ad0e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7fdf0c06c280 tx=0x7fdefc006f40).stop 2023-11-15T03:11:18.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.572+0000 7fdf0bfff700 1 -- 172.21.15.121:0/3387695166 shutdown_connections 2023-11-15T03:11:18.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.572+0000 7fdf0bfff700 1 --2- 172.21.15.121:0/3387695166 >> v2:172.21.15.121:3300/0 conn(0x7fdf0c0c5c70 0x7fdf0c1536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.572+0000 7fdf0bfff700 1 --2- 172.21.15.121:0/3387695166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf0c0acd00 0x7fdf0c0ad0e0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.572+0000 7fdf0bfff700 1 -- 172.21.15.121:0/3387695166 >> 172.21.15.121:0/3387695166 conn(0x7fdf0c063fc0 msgr2=0x7fdf0c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:18.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.572+0000 7fdf0bfff700 1 -- 172.21.15.121:0/3387695166 shutdown_connections 2023-11-15T03:11:18.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.572+0000 7fdf0bfff700 1 -- 172.21.15.121:0/3387695166 wait complete. 2023-11-15T03:11:18.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.576+0000 7fdf0bfff700 1 Processor -- start 2023-11-15T03:11:18.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.576+0000 7fdf0bfff700 1 -- start start 2023-11-15T03:11:18.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.576+0000 7fdf0bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf0c066330 0x7fdf0c14e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:18.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.576+0000 7fdf0bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf0c0acd00 0x7fdf0c1510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:18.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.576+0000 7fdf0bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf0c0c5c70 0x7fdf0c0c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:18.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.576+0000 7fdf0bfff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c1573f0 con 0x7fdf0c066330 2023-11-15T03:11:18.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.576+0000 7fdf0bfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c1565a0 con 0x7fdf0c0c5c70 2023-11-15T03:11:18.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.576+0000 7fdf0bfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c156820 con 0x7fdf0c0acd00 2023-11-15T03:11:18.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.576+0000 7fdf0affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf0c066330 0x7fdf0c14e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:18.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.576+0000 7fdf0affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf0c066330 0x7fdf0c14e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44756/0 (socket says 172.21.15.121:44756) 2023-11-15T03:11:18.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.576+0000 7fdf0affd700 1 -- 172.21.15.121:0/2442055284 learned_addr learned my addr 172.21.15.121:0/2442055284 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:18.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.576+0000 7fdf0b7fe700 1 --2- 172.21.15.121:0/2442055284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf0c0c5c70 0x7fdf0c0c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:18.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.576+0000 7fdf0a7fc700 1 --2- 172.21.15.121:0/2442055284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf0c0acd00 0x7fdf0c1510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:18.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.580+0000 7fdf0affd700 1 -- 172.21.15.121:0/2442055284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf0c0acd00 msgr2=0x7fdf0c1510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:18.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.580+0000 7fdf0affd700 1 --2- 172.21.15.121:0/2442055284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf0c0acd00 0x7fdf0c1510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.580+0000 7fdf0affd700 1 -- 172.21.15.121:0/2442055284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf0c0c5c70 msgr2=0x7fdf0c0c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:18.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.580+0000 7fdf0affd700 1 --2- 172.21.15.121:0/2442055284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf0c0c5c70 0x7fdf0c0c3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.580+0000 7fdf0affd700 1 -- 172.21.15.121:0/2442055284 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc009250 con 0x7fdf0c066330 2023-11-15T03:11:18.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.580+0000 7fdf0a7fc700 1 --2- 172.21.15.121:0/2442055284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf0c0acd00 0x7fdf0c1510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.580+0000 7fdf0affd700 1 --2- 172.21.15.121:0/2442055284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf0c066330 0x7fdf0c14e330 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 rx=0x7fdf0c0659d0 tx=0x7fdefc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.580+0000 7fdef2ffd700 1 -- 172.21.15.121:0/2442055284 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc005880 con 0x7fdf0c066330 2023-11-15T03:11:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.580+0000 7fdef2ffd700 1 -- 172.21.15.121:0/2442055284 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdefc00f720 con 0x7fdf0c066330 2023-11-15T03:11:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.580+0000 7fdf0bfff700 1 -- 172.21.15.121:0/2442055284 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0c156ac0 con 0x7fdf0c066330 2023-11-15T03:11:18.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.580+0000 7fdef2ffd700 1 -- 172.21.15.121:0/2442055284 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc00fac0 con 0x7fdf0c066330 2023-11-15T03:11:18.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.580+0000 7fdf0bfff700 1 -- 172.21.15.121:0/2442055284 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c157080 con 0x7fdf0c066330 2023-11-15T03:11:18.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.580+0000 7fdf0bfff700 1 -- 172.21.15.121:0/2442055284 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0c06bcf0 con 0x7fdf0c066330 2023-11-15T03:11:18.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.584+0000 7fdef2ffd700 1 -- 172.21.15.121:0/2442055284 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fdefc026460 con 0x7fdf0c066330 2023-11-15T03:11:18.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.584+0000 7fdef2ffd700 1 --2- 172.21.15.121:0/2442055284 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdef805a510 0x7fdef805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:18.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.584+0000 7fdef2ffd700 1 -- 172.21.15.121:0/2442055284 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fdefc075870 con 0x7fdf0c066330 2023-11-15T03:11:18.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.584+0000 7fdf0a7fc700 1 --2- 172.21.15.121:0/2442055284 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdef805a510 0x7fdef805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:18.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.584+0000 7fdf0a7fc700 1 --2- 172.21.15.121:0/2442055284 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdef805a510 0x7fdef805c9d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7fdf0c151ed0 tx=0x7fdf00008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:18.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.588+0000 7fdef2ffd700 1 -- 172.21.15.121:0/2442055284 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdefc049340 con 0x7fdf0c066330 2023-11-15T03:11:18.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.760+0000 7fdf0bfff700 1 -- 172.21.15.121:0/2442055284 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fdf0c0c81b0 con 0x7fdf0c066330 2023-11-15T03:11:18.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.760+0000 7fdef2ffd700 1 -- 172.21.15.121:0/2442055284 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v141) v1 ==== 110+0+10 (secure 0 0 0) 0x7fdefc04c960 con 0x7fdf0c066330 2023-11-15T03:11:18.771 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T03:11:18.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.764+0000 7fdf0bfff700 1 -- 172.21.15.121:0/2442055284 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdef805a510 msgr2=0x7fdef805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:18.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.764+0000 7fdf0bfff700 1 --2- 172.21.15.121:0/2442055284 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdef805a510 0x7fdef805c9d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7fdf0c151ed0 tx=0x7fdf00008040).stop 2023-11-15T03:11:18.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.764+0000 7fdf0bfff700 1 -- 172.21.15.121:0/2442055284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf0c066330 msgr2=0x7fdf0c14e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:18.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.764+0000 7fdf0bfff700 1 --2- 172.21.15.121:0/2442055284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf0c066330 0x7fdf0c14e330 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 rx=0x7fdf0c0659d0 tx=0x7fdefc016000).stop 2023-11-15T03:11:18.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.764+0000 7fdf0bfff700 1 -- 172.21.15.121:0/2442055284 shutdown_connections 2023-11-15T03:11:18.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.764+0000 7fdf0bfff700 1 --2- 172.21.15.121:0/2442055284 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fdef805a510 0x7fdef805c9d0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.764+0000 7fdf0bfff700 1 --2- 172.21.15.121:0/2442055284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf0c0c5c70 0x7fdf0c0c3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.764+0000 7fdf0bfff700 1 --2- 172.21.15.121:0/2442055284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf0c0acd00 0x7fdf0c1510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.764+0000 7fdf0bfff700 1 --2- 172.21.15.121:0/2442055284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf0c066330 0x7fdf0c14e330 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.764+0000 7fdf0bfff700 1 -- 172.21.15.121:0/2442055284 >> 172.21.15.121:0/2442055284 conn(0x7fdf0c063fc0 msgr2=0x7fdf0c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:18.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.764+0000 7fdf0bfff700 1 -- 172.21.15.121:0/2442055284 shutdown_connections 2023-11-15T03:11:18.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.764+0000 7fdf0bfff700 1 -- 172.21.15.121:0/2442055284 wait complete. 2023-11-15T03:11:18.791 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T03:11:18.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.936+0000 7f5e05169700 1 Processor -- start 2023-11-15T03:11:18.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.936+0000 7f5e05169700 1 -- start start 2023-11-15T03:11:18.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.940+0000 7f5e05169700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5e000682a0 0x7f5e000c10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:18.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.940+0000 7f5e05169700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5e00153700 0x7f5e00153b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:18.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.940+0000 7f5e05169700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e000c15a0 con 0x7f5e00067ec0 2023-11-15T03:11:18.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.940+0000 7f5e05169700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e000686e0 con 0x7f5e000682a0 2023-11-15T03:11:18.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.940+0000 7f5e05169700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e000c0c20 con 0x7f5e00153700 2023-11-15T03:11:18.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.940+0000 7f5dfe59c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5e000682a0 0x7f5e000c10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:18.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.940+0000 7f5dfe59c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5e000682a0 0x7f5e000c10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44772/0 (socket says 172.21.15.121:44772) 2023-11-15T03:11:18.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.940+0000 7f5dfe59c700 1 -- 172.21.15.121:0/3382661814 learned_addr learned my addr 172.21.15.121:0/3382661814 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:18.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.940+0000 7f5dff59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5e00153700 0x7f5e00153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:18.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.940+0000 7f5dfe59c700 1 -- 172.21.15.121:0/3382661814 >> v2:172.21.15.190:3300/0 conn(0x7f5e00153700 msgr2=0x7f5e00153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:18.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.940+0000 7f5dfe59c700 1 --2- 172.21.15.121:0/3382661814 >> v2:172.21.15.190:3300/0 conn(0x7f5e00153700 0x7f5e00153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.940+0000 7f5dfe59c700 1 -- 172.21.15.121:0/3382661814 >> v1:172.21.15.190:6789/0 conn(0x7f5e00067ec0 legacy=0x7f5e00062c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:18.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.940+0000 7f5dfe59c700 1 -- 172.21.15.121:0/3382661814 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e00158730 con 0x7f5e000682a0 2023-11-15T03:11:18.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.940+0000 7f5dfe59c700 1 --2- 172.21.15.121:0/3382661814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e000682a0 0x7f5e000c10b0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 rx=0x7f5e0005fa90 tx=0x7f5df40090f0).ready entity=mon.0 client_cookie=ce75414b2e51a4f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:18.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.944+0000 7f5dfdd9b700 1 -- 172.21.15.121:0/3382661814 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df400b520 con 0x7f5e000682a0 2023-11-15T03:11:18.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.944+0000 7f5dfdd9b700 1 -- 172.21.15.121:0/3382661814 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df400b680 con 0x7f5e000682a0 2023-11-15T03:11:18.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.944+0000 7f5dfdd9b700 1 -- 172.21.15.121:0/3382661814 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df400ba00 con 0x7f5e000682a0 2023-11-15T03:11:18.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.944+0000 7f5e05169700 1 -- 172.21.15.121:0/3382661814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e000682a0 msgr2=0x7f5e000c10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:18.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.944+0000 7f5e05169700 1 --2- 172.21.15.121:0/3382661814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e000682a0 0x7f5e000c10b0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 rx=0x7f5e0005fa90 tx=0x7f5df40090f0).stop 2023-11-15T03:11:18.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.944+0000 7f5e05169700 1 -- 172.21.15.121:0/3382661814 shutdown_connections 2023-11-15T03:11:18.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.944+0000 7f5e05169700 1 --2- 172.21.15.121:0/3382661814 >> v2:172.21.15.190:3300/0 conn(0x7f5e00153700 0x7f5e00153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.944+0000 7f5e05169700 1 --2- 172.21.15.121:0/3382661814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e000682a0 0x7f5e000c10b0 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.944+0000 7f5e05169700 1 -- 172.21.15.121:0/3382661814 >> 172.21.15.121:0/3382661814 conn(0x7f5e000cad40 msgr2=0x7f5e000cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:18.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.944+0000 7f5e05169700 1 -- 172.21.15.121:0/3382661814 shutdown_connections 2023-11-15T03:11:18.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.944+0000 7f5e05169700 1 -- 172.21.15.121:0/3382661814 wait complete. 2023-11-15T03:11:18.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.948+0000 7f5e05169700 1 Processor -- start 2023-11-15T03:11:18.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.948+0000 7f5e05169700 1 -- start start 2023-11-15T03:11:18.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.948+0000 7f5e05169700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e00067ec0 0x7f5e00142c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:18.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.948+0000 7f5e05169700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e000682a0 0x7f5e00154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:18.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.948+0000 7f5e05169700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e00153700 0x7f5e00155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:18.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.948+0000 7f5e05169700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e00154900 con 0x7f5e00153700 2023-11-15T03:11:18.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.948+0000 7f5e05169700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e00157250 con 0x7f5e000682a0 2023-11-15T03:11:18.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.948+0000 7f5e05169700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e00156640 con 0x7f5e00067ec0 2023-11-15T03:11:18.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.948+0000 7f5dff59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e00153700 0x7f5e00155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:18.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.948+0000 7f5dff59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e00153700 0x7f5e00155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44778/0 (socket says 172.21.15.121:44778) 2023-11-15T03:11:18.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.948+0000 7f5dff59e700 1 -- 172.21.15.121:0/894952277 learned_addr learned my addr 172.21.15.121:0/894952277 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:18.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.948+0000 7f5dfe59c700 1 --2- 172.21.15.121:0/894952277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e000682a0 0x7f5e00154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:18.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.948+0000 7f5dfed9d700 1 --2- 172.21.15.121:0/894952277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e00067ec0 0x7f5e00142c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:18.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.952+0000 7f5dff59e700 1 -- 172.21.15.121:0/894952277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e00067ec0 msgr2=0x7f5e00142c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:18.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.952+0000 7f5dff59e700 1 --2- 172.21.15.121:0/894952277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e00067ec0 0x7f5e00142c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.952+0000 7f5dff59e700 1 -- 172.21.15.121:0/894952277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e000682a0 msgr2=0x7f5e00154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:18.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.952+0000 7f5dff59e700 1 --2- 172.21.15.121:0/894952277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e000682a0 0x7f5e00154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:18.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.952+0000 7f5dff59e700 1 -- 172.21.15.121:0/894952277 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df4009370 con 0x7f5e00153700 2023-11-15T03:11:18.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.952+0000 7f5dff59e700 1 --2- 172.21.15.121:0/894952277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e00153700 0x7f5e00155cf0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 rx=0x7f5df8009df0 tx=0x7f5df800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:18.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.952+0000 7f5de6ffd700 1 -- 172.21.15.121:0/894952277 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df800caa0 con 0x7f5e00153700 2023-11-15T03:11:18.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.952+0000 7f5de6ffd700 1 -- 172.21.15.121:0/894952277 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df800f040 con 0x7f5e00153700 2023-11-15T03:11:18.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.952+0000 7f5e05169700 1 -- 172.21.15.121:0/894952277 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e00156920 con 0x7f5e00153700 2023-11-15T03:11:18.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.952+0000 7f5de6ffd700 1 -- 172.21.15.121:0/894952277 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df800ce10 con 0x7f5e00153700 2023-11-15T03:11:18.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.952+0000 7f5e05169700 1 -- 172.21.15.121:0/894952277 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e00156ee0 con 0x7f5e00153700 2023-11-15T03:11:18.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.952+0000 7f5e05169700 1 -- 172.21.15.121:0/894952277 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e000665a0 con 0x7f5e00153700 2023-11-15T03:11:18.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.956+0000 7f5de6ffd700 1 -- 172.21.15.121:0/894952277 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5df8013610 con 0x7f5e00153700 2023-11-15T03:11:18.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.956+0000 7f5de6ffd700 1 --2- 172.21.15.121:0/894952277 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5dec05a510 0x7f5dec05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:18.965 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.956+0000 7f5de6ffd700 1 -- 172.21.15.121:0/894952277 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f5df8074c60 con 0x7f5e00153700 2023-11-15T03:11:18.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.956+0000 7f5dfed9d700 1 --2- 172.21.15.121:0/894952277 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5dec05a510 0x7f5dec05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:18.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.956+0000 7f5dfed9d700 1 --2- 172.21.15.121:0/894952277 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5dec05a510 0x7f5dec05c9d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f5df0003870 tx=0x7f5df0007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:18.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:18.964+0000 7f5de6ffd700 1 -- 172.21.15.121:0/894952277 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5df80486d0 con 0x7f5e00153700 2023-11-15T03:11:19.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.128+0000 7f5e05169700 1 -- 172.21.15.121:0/894952277 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5e001573f0 con 0x7f5e00153700 2023-11-15T03:11:19.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.128+0000 7f5de6ffd700 1 -- 172.21.15.121:0/894952277 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v141) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5df804bcf0 con 0x7f5e00153700 2023-11-15T03:11:19.136 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T03:11:19.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.128+0000 7f5e05169700 1 -- 172.21.15.121:0/894952277 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5dec05a510 msgr2=0x7f5dec05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:19.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.128+0000 7f5e05169700 1 --2- 172.21.15.121:0/894952277 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5dec05a510 0x7f5dec05c9d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f5df0003870 tx=0x7f5df0007d40).stop 2023-11-15T03:11:19.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.132+0000 7f5e05169700 1 -- 172.21.15.121:0/894952277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e00153700 msgr2=0x7f5e00155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:19.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.132+0000 7f5e05169700 1 --2- 172.21.15.121:0/894952277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e00153700 0x7f5e00155cf0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 rx=0x7f5df8009df0 tx=0x7f5df800b040).stop 2023-11-15T03:11:19.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.132+0000 7f5e05169700 1 -- 172.21.15.121:0/894952277 shutdown_connections 2023-11-15T03:11:19.140 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.132+0000 7f5e05169700 1 --2- 172.21.15.121:0/894952277 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5dec05a510 0x7f5dec05c9d0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.132+0000 7f5e05169700 1 --2- 172.21.15.121:0/894952277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e00153700 0x7f5e00155cf0 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.132+0000 7f5e05169700 1 --2- 172.21.15.121:0/894952277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e000682a0 0x7f5e00154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.132+0000 7f5e05169700 1 --2- 172.21.15.121:0/894952277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e00067ec0 0x7f5e00142c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.132+0000 7f5e05169700 1 -- 172.21.15.121:0/894952277 >> 172.21.15.121:0/894952277 conn(0x7f5e000cad40 msgr2=0x7f5e000c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:19.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.132+0000 7f5e05169700 1 -- 172.21.15.121:0/894952277 shutdown_connections 2023-11-15T03:11:19.142 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.132+0000 7f5e05169700 1 -- 172.21.15.121:0/894952277 wait complete. 2023-11-15T03:11:19.159 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-15T03:11:19.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.320+0000 7f6e792f1700 1 Processor -- start 2023-11-15T03:11:19.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.320+0000 7f6e792f1700 1 -- start start 2023-11-15T03:11:19.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.324+0000 7f6e792f1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6e740c6ce0 0x7f6e74159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:19.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.324+0000 7f6e792f1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6e740a5510 0x7f6e740a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:19.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.324+0000 7f6e792f1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e740c3fa0 con 0x7f6e74152780 2023-11-15T03:11:19.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.324+0000 7f6e792f1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e74159630 con 0x7f6e740c6ce0 2023-11-15T03:11:19.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.324+0000 7f6e792f1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e740c30b0 con 0x7f6e740a5510 2023-11-15T03:11:19.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.324+0000 7f6e72ffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6e74152780 0x7f6e740c65d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:35862/0 (socket says 172.21.15.121:35862) 2023-11-15T03:11:19.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.324+0000 7f6e72ffd700 1 -- 172.21.15.121:0/1453710961 learned_addr learned my addr 172.21.15.121:0/1453710961 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:19.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.324+0000 7f6e6a7fc700 1 --2- 172.21.15.121:0/1453710961 >> v2:172.21.15.121:3300/0 conn(0x7f6e740c6ce0 0x7f6e74159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:19.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.324+0000 7f6e737fe700 1 --2- 172.21.15.121:0/1453710961 >> v2:172.21.15.190:3300/0 conn(0x7f6e740a5510 0x7f6e740a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:19.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.324+0000 7f6e6a7fc700 1 -- 172.21.15.121:0/1453710961 >> v2:172.21.15.190:3300/0 conn(0x7f6e740a5510 msgr2=0x7f6e740a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:19.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.324+0000 7f6e6a7fc700 1 --2- 172.21.15.121:0/1453710961 >> v2:172.21.15.190:3300/0 conn(0x7f6e740a5510 0x7f6e740a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.324+0000 7f6e6a7fc700 1 -- 172.21.15.121:0/1453710961 >> v1:172.21.15.121:6789/0 conn(0x7f6e74152780 legacy=0x7f6e740c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:19.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.324+0000 7f6e6a7fc700 1 -- 172.21.15.121:0/1453710961 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e740c3390 con 0x7f6e740c6ce0 2023-11-15T03:11:19.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.324+0000 7f6e6a7fc700 1 --2- 172.21.15.121:0/1453710961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e740c6ce0 0x7f6e74159140 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 rx=0x7f6e740c0ec0 tx=0x7f6e5c009780).ready entity=mon.0 client_cookie=b81ae86cfcb9c08b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:19.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.324+0000 7f6e727fc700 1 -- 172.21.15.121:0/1453710961 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c00e070 con 0x7f6e740c6ce0 2023-11-15T03:11:19.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.324+0000 7f6e727fc700 1 -- 172.21.15.121:0/1453710961 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c00bc30 con 0x7f6e740c6ce0 2023-11-15T03:11:19.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.324+0000 7f6e727fc700 1 -- 172.21.15.121:0/1453710961 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c012410 con 0x7f6e740c6ce0 2023-11-15T03:11:19.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.328+0000 7f6e792f1700 1 -- 172.21.15.121:0/1453710961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e740c6ce0 msgr2=0x7f6e74159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:19.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.328+0000 7f6e792f1700 1 --2- 172.21.15.121:0/1453710961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e740c6ce0 0x7f6e74159140 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 rx=0x7f6e740c0ec0 tx=0x7f6e5c009780).stop 2023-11-15T03:11:19.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.328+0000 7f6e792f1700 1 -- 172.21.15.121:0/1453710961 shutdown_connections 2023-11-15T03:11:19.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.328+0000 7f6e792f1700 1 --2- 172.21.15.121:0/1453710961 >> v2:172.21.15.190:3300/0 conn(0x7f6e740a5510 0x7f6e740a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.328+0000 7f6e792f1700 1 --2- 172.21.15.121:0/1453710961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e740c6ce0 0x7f6e74159140 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.328+0000 7f6e792f1700 1 -- 172.21.15.121:0/1453710961 >> 172.21.15.121:0/1453710961 conn(0x7f6e74062190 msgr2=0x7f6e740625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:19.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.328+0000 7f6e792f1700 1 -- 172.21.15.121:0/1453710961 shutdown_connections 2023-11-15T03:11:19.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.328+0000 7f6e792f1700 1 -- 172.21.15.121:0/1453710961 wait complete. 2023-11-15T03:11:19.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.328+0000 7f6e792f1700 1 Processor -- start 2023-11-15T03:11:19.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.328+0000 7f6e792f1700 1 -- start start 2023-11-15T03:11:19.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e792f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e740a5510 0x7f6e74152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:19.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e792f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e740c6ce0 0x7f6e7414fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:19.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e792f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e74152780 0x7f6e740a7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:19.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e792f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e74150b50 con 0x7f6e740a5510 2023-11-15T03:11:19.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e792f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e74153050 con 0x7f6e74152780 2023-11-15T03:11:19.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e792f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e74153260 con 0x7f6e740c6ce0 2023-11-15T03:11:19.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e72ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e740a5510 0x7f6e74152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:19.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e72ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e740a5510 0x7f6e74152b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44804/0 (socket says 172.21.15.121:44804) 2023-11-15T03:11:19.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e72ffd700 1 -- 172.21.15.121:0/3288279471 learned_addr learned my addr 172.21.15.121:0/3288279471 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:19.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e6a7fc700 1 --2- 172.21.15.121:0/3288279471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e740c6ce0 0x7f6e7414fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:19.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e737fe700 1 --2- 172.21.15.121:0/3288279471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e74152780 0x7f6e740a7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:19.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e72ffd700 1 -- 172.21.15.121:0/3288279471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e740c6ce0 msgr2=0x7f6e7414fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:19.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e72ffd700 1 --2- 172.21.15.121:0/3288279471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e740c6ce0 0x7f6e7414fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e72ffd700 1 -- 172.21.15.121:0/3288279471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e74152780 msgr2=0x7f6e740a7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:19.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e72ffd700 1 --2- 172.21.15.121:0/3288279471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e74152780 0x7f6e740a7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e72ffd700 1 -- 172.21.15.121:0/3288279471 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5c009a20 con 0x7f6e740a5510 2023-11-15T03:11:19.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e72ffd700 1 --2- 172.21.15.121:0/3288279471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e740a5510 0x7f6e74152b60 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 rx=0x7f6e64001eb0 tx=0x7f6e64007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:19.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e6b7fe700 1 -- 172.21.15.121:0/3288279471 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e640070d0 con 0x7f6e740a5510 2023-11-15T03:11:19.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e6b7fe700 1 -- 172.21.15.121:0/3288279471 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e64007230 con 0x7f6e740a5510 2023-11-15T03:11:19.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e6b7fe700 1 -- 172.21.15.121:0/3288279471 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e640075d0 con 0x7f6e740a5510 2023-11-15T03:11:19.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e792f1700 1 -- 172.21.15.121:0/3288279471 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e74153540 con 0x7f6e740a5510 2023-11-15T03:11:19.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.332+0000 7f6e792f1700 1 -- 172.21.15.121:0/3288279471 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e74153b00 con 0x7f6e740a5510 2023-11-15T03:11:19.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.336+0000 7f6e792f1700 1 -- 172.21.15.121:0/3288279471 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e741594a0 con 0x7f6e740a5510 2023-11-15T03:11:19.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.336+0000 7f6e6b7fe700 1 -- 172.21.15.121:0/3288279471 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6e640059b0 con 0x7f6e740a5510 2023-11-15T03:11:19.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.336+0000 7f6e6b7fe700 1 --2- 172.21.15.121:0/3288279471 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e5405a4e0 0x7f6e5405c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:19.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.336+0000 7f6e6b7fe700 1 -- 172.21.15.121:0/3288279471 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f6e64073a40 con 0x7f6e740a5510 2023-11-15T03:11:19.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.336+0000 7f6e6a7fc700 1 --2- 172.21.15.121:0/3288279471 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e5405a4e0 0x7f6e5405c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:19.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.336+0000 7f6e6a7fc700 1 --2- 172.21.15.121:0/3288279471 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e5405a4e0 0x7f6e5405c9a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7f6e5c009fd0 tx=0x7f6e5c009780).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:19.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.344+0000 7f6e6b7fe700 1 -- 172.21.15.121:0/3288279471 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6e64075e60 con 0x7f6e740a5510 2023-11-15T03:11:19.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.508+0000 7f6e792f1700 1 -- 172.21.15.121:0/3288279471 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6e7406c150 con 0x7f6e740a5510 2023-11-15T03:11:19.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.508+0000 7f6e6b7fe700 1 -- 172.21.15.121:0/3288279471 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v141) v1 ==== 84+0+143 (secure 0 0 0) 0x7f6e64047570 con 0x7f6e740a5510 2023-11-15T03:11:19.518 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:19.519 INFO:teuthology.orchestra.run.smithi121.stdout:[ 2023-11-15T03:11:19.519 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:19.519 INFO:teuthology.orchestra.run.smithi121.stdout: "poolnum": 1, 2023-11-15T03:11:19.519 INFO:teuthology.orchestra.run.smithi121.stdout: "poolname": "device_health_metrics" 2023-11-15T03:11:19.519 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:19.520 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:19.520 INFO:teuthology.orchestra.run.smithi121.stdout: "poolnum": 2, 2023-11-15T03:11:19.520 INFO:teuthology.orchestra.run.smithi121.stdout: "poolname": "rbd" 2023-11-15T03:11:19.520 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:19.521 INFO:teuthology.orchestra.run.smithi121.stdout:] 2023-11-15T03:11:19.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.512+0000 7f6e792f1700 1 -- 172.21.15.121:0/3288279471 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e5405a4e0 msgr2=0x7f6e5405c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:19.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.512+0000 7f6e792f1700 1 --2- 172.21.15.121:0/3288279471 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e5405a4e0 0x7f6e5405c9a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7f6e5c009fd0 tx=0x7f6e5c009780).stop 2023-11-15T03:11:19.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.512+0000 7f6e792f1700 1 -- 172.21.15.121:0/3288279471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e740a5510 msgr2=0x7f6e74152b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:19.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.512+0000 7f6e792f1700 1 --2- 172.21.15.121:0/3288279471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e740a5510 0x7f6e74152b60 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 rx=0x7f6e64001eb0 tx=0x7f6e64007e50).stop 2023-11-15T03:11:19.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.512+0000 7f6e792f1700 1 -- 172.21.15.121:0/3288279471 shutdown_connections 2023-11-15T03:11:19.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.512+0000 7f6e792f1700 1 --2- 172.21.15.121:0/3288279471 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6e5405a4e0 0x7f6e5405c9a0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.512+0000 7f6e792f1700 1 --2- 172.21.15.121:0/3288279471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e74152780 0x7f6e740a7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.512+0000 7f6e792f1700 1 --2- 172.21.15.121:0/3288279471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e740c6ce0 0x7f6e7414fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.512+0000 7f6e792f1700 1 --2- 172.21.15.121:0/3288279471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e740a5510 0x7f6e74152b60 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.512+0000 7f6e792f1700 1 -- 172.21.15.121:0/3288279471 >> 172.21.15.121:0/3288279471 conn(0x7f6e74062190 msgr2=0x7f6e740a8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:19.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.516+0000 7f6e792f1700 1 -- 172.21.15.121:0/3288279471 shutdown_connections 2023-11-15T03:11:19.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.516+0000 7f6e792f1700 1 -- 172.21.15.121:0/3288279471 wait complete. 2023-11-15T03:11:19.543 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-15T03:11:19.543 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-15T03:11:19.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.712+0000 7ff8be2b9700 1 Processor -- start 2023-11-15T03:11:19.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.712+0000 7ff8be2b9700 1 -- start start 2023-11-15T03:11:19.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8be2b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8b80ab0d0 0x7ff8b80ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:19.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8be2b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff8b80ad1b0 0x7ff8b8065710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:19.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8be2b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff8b8065c00 0x7ff8b80c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:19.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8be2b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b8066c90 con 0x7ff8b80ad1b0 2023-11-15T03:11:19.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8be2b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b8066080 con 0x7ff8b8065c00 2023-11-15T03:11:19.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8be2b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b8066360 con 0x7ff8b80ab0d0 2023-11-15T03:11:19.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8bcab6700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff8b80ad1b0 0x7ff8b8065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:19.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8bd2b7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8b80ab0d0 0x7ff8b80ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:19.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8bd2b7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8b80ab0d0 0x7ff8b80ab4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:39936/0 (socket says 172.21.15.121:39936) 2023-11-15T03:11:19.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8bd2b7700 1 -- 172.21.15.121:0/1612852507 learned_addr learned my addr 172.21.15.121:0/1612852507 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:19.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8bdab8700 1 --2- 172.21.15.121:0/1612852507 >> v2:172.21.15.190:3300/0 conn(0x7ff8b8065c00 0x7ff8b80c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:19.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8bcab6700 1 -- 172.21.15.121:0/1612852507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8b80ab0d0 msgr2=0x7ff8b80ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:19.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8bcab6700 1 --2- 172.21.15.121:0/1612852507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8b80ab0d0 0x7ff8b80ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8bcab6700 1 -- 172.21.15.121:0/1612852507 >> v2:172.21.15.190:3300/0 conn(0x7ff8b8065c00 msgr2=0x7ff8b80c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:19.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8bcab6700 1 --2- 172.21.15.121:0/1612852507 >> v2:172.21.15.190:3300/0 conn(0x7ff8b8065c00 0x7ff8b80c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8bcab6700 1 -- 172.21.15.121:0/1612852507 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b8066640 con 0x7ff8b80ad1b0 2023-11-15T03:11:19.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8bcab6700 1 --2- 172.21.15.121:0/1612852507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8b80ad1b0 0x7ff8b8065710 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 rx=0x7ff8b80c7100 tx=0x7ff8a8009060).ready entity=mon.0 client_cookie=28c2145c1885a3b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:19.728 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8affff700 1 -- 172.21.15.121:0/1612852507 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a800d4a0 con 0x7ff8b80ad1b0 2023-11-15T03:11:19.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8affff700 1 -- 172.21.15.121:0/1612852507 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a800d600 con 0x7ff8b80ad1b0 2023-11-15T03:11:19.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.716+0000 7ff8affff700 1 -- 172.21.15.121:0/1612852507 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a800d980 con 0x7ff8b80ad1b0 2023-11-15T03:11:19.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.720+0000 7ff8be2b9700 1 -- 172.21.15.121:0/1612852507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8b80ad1b0 msgr2=0x7ff8b8065710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:19.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.720+0000 7ff8be2b9700 1 --2- 172.21.15.121:0/1612852507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8b80ad1b0 0x7ff8b8065710 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 rx=0x7ff8b80c7100 tx=0x7ff8a8009060).stop 2023-11-15T03:11:19.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.720+0000 7ff8be2b9700 1 -- 172.21.15.121:0/1612852507 shutdown_connections 2023-11-15T03:11:19.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.720+0000 7ff8be2b9700 1 --2- 172.21.15.121:0/1612852507 >> v2:172.21.15.190:3300/0 conn(0x7ff8b8065c00 0x7ff8b80c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.720+0000 7ff8be2b9700 1 --2- 172.21.15.121:0/1612852507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8b80ad1b0 0x7ff8b8065710 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.720+0000 7ff8be2b9700 1 --2- 172.21.15.121:0/1612852507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8b80ab0d0 0x7ff8b80ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.720+0000 7ff8be2b9700 1 -- 172.21.15.121:0/1612852507 >> 172.21.15.121:0/1612852507 conn(0x7ff8b8067a60 msgr2=0x7ff8b8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:19.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.720+0000 7ff8be2b9700 1 -- 172.21.15.121:0/1612852507 shutdown_connections 2023-11-15T03:11:19.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.720+0000 7ff8be2b9700 1 -- 172.21.15.121:0/1612852507 wait complete. 2023-11-15T03:11:19.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.720+0000 7ff8be2b9700 1 Processor -- start 2023-11-15T03:11:19.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.720+0000 7ff8be2b9700 1 -- start start 2023-11-15T03:11:19.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8be2b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8b8065c00 0x7ff8b80a8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:19.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8be2b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8b80ab0d0 0x7ff8b80a93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:19.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8be2b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8b80ad1b0 0x7ff8b80afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:19.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8be2b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b81682b0 con 0x7ff8b80ad1b0 2023-11-15T03:11:19.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8be2b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b80a75d0 con 0x7ff8b80ab0d0 2023-11-15T03:11:19.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8be2b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b80a6ca0 con 0x7ff8b8065c00 2023-11-15T03:11:19.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8bdab8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8b80ad1b0 0x7ff8b80afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:19.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8bdab8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8b80ad1b0 0x7ff8b80afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44828/0 (socket says 172.21.15.121:44828) 2023-11-15T03:11:19.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8bdab8700 1 -- 172.21.15.121:0/1441326925 learned_addr learned my addr 172.21.15.121:0/1441326925 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:19.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8bd2b7700 1 --2- 172.21.15.121:0/1441326925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8b8065c00 0x7ff8b80a8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:19.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8bcab6700 1 --2- 172.21.15.121:0/1441326925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8b80ab0d0 0x7ff8b80a93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:19.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8bdab8700 1 -- 172.21.15.121:0/1441326925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8b8065c00 msgr2=0x7ff8b80a8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:19.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8bdab8700 1 --2- 172.21.15.121:0/1441326925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8b8065c00 0x7ff8b80a8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8bdab8700 1 -- 172.21.15.121:0/1441326925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8b80ab0d0 msgr2=0x7ff8b80a93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:19.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8bdab8700 1 --2- 172.21.15.121:0/1441326925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8b80ab0d0 0x7ff8b80a93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:19.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8bdab8700 1 -- 172.21.15.121:0/1441326925 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a80092a0 con 0x7ff8b80ad1b0 2023-11-15T03:11:19.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8bcab6700 1 --2- 172.21.15.121:0/1441326925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8b80ab0d0 0x7ff8b80a93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:19.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8bd2b7700 1 --2- 172.21.15.121:0/1441326925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8b8065c00 0x7ff8b80a8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:19.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8bdab8700 1 --2- 172.21.15.121:0/1441326925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8b80ad1b0 0x7ff8b80afdd0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 rx=0x7ff8b0009d10 tx=0x7ff8b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:19.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8ad7fa700 1 -- 172.21.15.121:0/1441326925 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0009a70 con 0x7ff8b80ad1b0 2023-11-15T03:11:19.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8ad7fa700 1 -- 172.21.15.121:0/1441326925 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b000f040 con 0x7ff8b80ad1b0 2023-11-15T03:11:19.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8ad7fa700 1 -- 172.21.15.121:0/1441326925 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b000cd80 con 0x7ff8b80ad1b0 2023-11-15T03:11:19.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8be2b9700 1 -- 172.21.15.121:0/1441326925 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b80a6f80 con 0x7ff8b80ad1b0 2023-11-15T03:11:19.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.724+0000 7ff8be2b9700 1 -- 172.21.15.121:0/1441326925 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b81432a0 con 0x7ff8b80ad1b0 2023-11-15T03:11:19.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.728+0000 7ff8be2b9700 1 -- 172.21.15.121:0/1441326925 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b806bcf0 con 0x7ff8b80ad1b0 2023-11-15T03:11:19.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.728+0000 7ff8ad7fa700 1 -- 172.21.15.121:0/1441326925 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff8b0013610 con 0x7ff8b80ad1b0 2023-11-15T03:11:19.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.728+0000 7ff8ad7fa700 1 --2- 172.21.15.121:0/1441326925 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff89c05a510 0x7ff89c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:19.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.728+0000 7ff8ad7fa700 1 -- 172.21.15.121:0/1441326925 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5269+0+0 (secure 0 0 0) 0x7ff8b0074c00 con 0x7ff8b80ad1b0 2023-11-15T03:11:19.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.728+0000 7ff8bd2b7700 1 --2- 172.21.15.121:0/1441326925 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff89c05a510 0x7ff89c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:19.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.728+0000 7ff8bd2b7700 1 --2- 172.21.15.121:0/1441326925 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff89c05a510 0x7ff89c05c9d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7ff8b4009ac0 tx=0x7ff8b4008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:19.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.736+0000 7ff8ad7fa700 1 -- 172.21.15.121:0/1441326925 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff8b0048670 con 0x7ff8b80ad1b0 2023-11-15T03:11:19.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:19.892+0000 7ff8be2b9700 1 -- 172.21.15.121:0/1441326925 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7ff8b80c7160 con 0x7ff8b80ad1b0 2023-11-15T03:11:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.412+0000 7ff8ad7fa700 1 -- 172.21.15.121:0/1441326925 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v142) v1 ==== 152+0+0 (secure 0 0 0) 0x7ff8b004bc90 con 0x7ff8b80ad1b0 2023-11-15T03:11:20.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.420+0000 7ff8be2b9700 1 -- 172.21.15.121:0/1441326925 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff89c05a510 msgr2=0x7ff89c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:20.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.420+0000 7ff8be2b9700 1 --2- 172.21.15.121:0/1441326925 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff89c05a510 0x7ff89c05c9d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7ff8b4009ac0 tx=0x7ff8b4008040).stop 2023-11-15T03:11:20.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.420+0000 7ff8be2b9700 1 -- 172.21.15.121:0/1441326925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8b80ad1b0 msgr2=0x7ff8b80afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:20.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.420+0000 7ff8be2b9700 1 --2- 172.21.15.121:0/1441326925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8b80ad1b0 0x7ff8b80afdd0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 rx=0x7ff8b0009d10 tx=0x7ff8b000b040).stop 2023-11-15T03:11:20.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.420+0000 7ff8be2b9700 1 -- 172.21.15.121:0/1441326925 shutdown_connections 2023-11-15T03:11:20.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.420+0000 7ff8be2b9700 1 --2- 172.21.15.121:0/1441326925 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff89c05a510 0x7ff89c05c9d0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:20.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.420+0000 7ff8be2b9700 1 --2- 172.21.15.121:0/1441326925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8b80ad1b0 0x7ff8b80afdd0 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:20.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.420+0000 7ff8be2b9700 1 --2- 172.21.15.121:0/1441326925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8b80ab0d0 0x7ff8b80a93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:20.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.420+0000 7ff8be2b9700 1 --2- 172.21.15.121:0/1441326925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8b8065c00 0x7ff8b80a8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:20.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.420+0000 7ff8be2b9700 1 -- 172.21.15.121:0/1441326925 >> 172.21.15.121:0/1441326925 conn(0x7ff8b8067a60 msgr2=0x7ff8b81560b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:20.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.424+0000 7ff8be2b9700 1 -- 172.21.15.121:0/1441326925 shutdown_connections 2023-11-15T03:11:20.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.424+0000 7ff8be2b9700 1 -- 172.21.15.121:0/1441326925 wait complete. 2023-11-15T03:11:20.438 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_metadata' created 2023-11-15T03:11:20.455 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-15T03:11:20.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.616+0000 7f2ff2d47700 1 Processor -- start 2023-11-15T03:11:20.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.616+0000 7f2ff2d47700 1 -- start start 2023-11-15T03:11:20.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.620+0000 7f2ff2d47700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec0ab0d0 0x7f2fec0ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:20.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.620+0000 7f2ff2d47700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2fec0ad1b0 0x7f2fec065710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:20.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.620+0000 7f2ff2d47700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2fec065c00 0x7f2fec0c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:20.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.620+0000 7f2ff2d47700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fec066c90 con 0x7f2fec0ad1b0 2023-11-15T03:11:20.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.620+0000 7f2ff2d47700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fec066080 con 0x7f2fec065c00 2023-11-15T03:11:20.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.620+0000 7f2ff2d47700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec066360 con 0x7f2fec0ab0d0 2023-11-15T03:11:20.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.620+0000 7f2ff1544700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2fec0ad1b0 0x7f2fec065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:20.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.620+0000 7f2ff1544700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2fec0ad1b0 0x7f2fec065710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44834/0 (socket says 172.21.15.121:44834) 2023-11-15T03:11:20.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.620+0000 7f2ff1544700 1 -- 172.21.15.121:0/4059854737 learned_addr learned my addr 172.21.15.121:0/4059854737 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:20.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.620+0000 7f2ff2546700 1 --2- 172.21.15.121:0/4059854737 >> v2:172.21.15.190:3300/0 conn(0x7f2fec065c00 0x7f2fec0c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:20.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.620+0000 7f2ff1d45700 1 --2- 172.21.15.121:0/4059854737 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec0ab0d0 0x7f2fec0ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:20.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.620+0000 7f2ff1544700 1 -- 172.21.15.121:0/4059854737 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec0ab0d0 msgr2=0x7f2fec0ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:20.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.620+0000 7f2ff1544700 1 --2- 172.21.15.121:0/4059854737 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec0ab0d0 0x7f2fec0ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:20.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.620+0000 7f2ff1544700 1 -- 172.21.15.121:0/4059854737 >> v2:172.21.15.190:3300/0 conn(0x7f2fec065c00 msgr2=0x7f2fec0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:20.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.620+0000 7f2ff1544700 1 --2- 172.21.15.121:0/4059854737 >> v2:172.21.15.190:3300/0 conn(0x7f2fec065c00 0x7f2fec0c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:20.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.620+0000 7f2ff1544700 1 -- 172.21.15.121:0/4059854737 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fec066640 con 0x7f2fec0ad1b0 2023-11-15T03:11:20.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.620+0000 7f2ff1544700 1 --2- 172.21.15.121:0/4059854737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fec0ad1b0 0x7f2fec065710 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 rx=0x7f2fec0c7100 tx=0x7f2fdc0090f0).ready entity=mon.0 client_cookie=f2730d61f51ecf28 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:20.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.624+0000 7f2ff0d43700 1 -- 172.21.15.121:0/4059854737 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc00f510 con 0x7f2fec0ad1b0 2023-11-15T03:11:20.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.624+0000 7f2ff0d43700 1 -- 172.21.15.121:0/4059854737 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fdc00f670 con 0x7f2fec0ad1b0 2023-11-15T03:11:20.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.624+0000 7f2ff0d43700 1 -- 172.21.15.121:0/4059854737 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc00f9f0 con 0x7f2fec0ad1b0 2023-11-15T03:11:20.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.624+0000 7f2ff2d47700 1 -- 172.21.15.121:0/4059854737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fec0ad1b0 msgr2=0x7f2fec065710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:20.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.624+0000 7f2ff2d47700 1 --2- 172.21.15.121:0/4059854737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fec0ad1b0 0x7f2fec065710 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 rx=0x7f2fec0c7100 tx=0x7f2fdc0090f0).stop 2023-11-15T03:11:20.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.624+0000 7f2ff2d47700 1 -- 172.21.15.121:0/4059854737 shutdown_connections 2023-11-15T03:11:20.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.624+0000 7f2ff2d47700 1 --2- 172.21.15.121:0/4059854737 >> v2:172.21.15.190:3300/0 conn(0x7f2fec065c00 0x7f2fec0c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:20.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.624+0000 7f2ff2d47700 1 --2- 172.21.15.121:0/4059854737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fec0ad1b0 0x7f2fec065710 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:20.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.624+0000 7f2ff2d47700 1 --2- 172.21.15.121:0/4059854737 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec0ab0d0 0x7f2fec0ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:20.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.624+0000 7f2ff2d47700 1 -- 172.21.15.121:0/4059854737 >> 172.21.15.121:0/4059854737 conn(0x7f2fec067a60 msgr2=0x7f2fec067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:20.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.624+0000 7f2ff2d47700 1 -- 172.21.15.121:0/4059854737 shutdown_connections 2023-11-15T03:11:20.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.624+0000 7f2ff2d47700 1 -- 172.21.15.121:0/4059854737 wait complete. 2023-11-15T03:11:20.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.624+0000 7f2ff2d47700 1 Processor -- start 2023-11-15T03:11:20.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.624+0000 7f2ff2d47700 1 -- start start 2023-11-15T03:11:20.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2ff2d47700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec065c00 0x7f2fec141b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:20.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2ff2d47700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fec0ab0d0 0x7f2fec0a6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:20.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2ff2d47700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec0ad1b0 0x7f2fec0cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:20.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2ff2d47700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec1423f0 con 0x7f2fec0ab0d0 2023-11-15T03:11:20.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2ff2d47700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec0a8320 con 0x7f2fec0ad1b0 2023-11-15T03:11:20.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2ff2d47700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec0a7710 con 0x7f2fec065c00 2023-11-15T03:11:20.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2ff1544700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fec0ab0d0 0x7f2fec0a6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:20.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2ff1544700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fec0ab0d0 0x7f2fec0a6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44848/0 (socket says 172.21.15.121:44848) 2023-11-15T03:11:20.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2ff1544700 1 -- 172.21.15.121:0/3277557689 learned_addr learned my addr 172.21.15.121:0/3277557689 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:20.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2ff2546700 1 --2- 172.21.15.121:0/3277557689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec0ad1b0 0x7f2fec0cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:20.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2ff1d45700 1 --2- 172.21.15.121:0/3277557689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec065c00 0x7f2fec141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:20.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2ff1544700 1 -- 172.21.15.121:0/3277557689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec065c00 msgr2=0x7f2fec141b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:20.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2ff1544700 1 --2- 172.21.15.121:0/3277557689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec065c00 0x7f2fec141b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:20.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2ff1544700 1 -- 172.21.15.121:0/3277557689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec0ad1b0 msgr2=0x7f2fec0cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:20.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2ff1544700 1 --2- 172.21.15.121:0/3277557689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec0ad1b0 0x7f2fec0cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:20.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2ff1544700 1 -- 172.21.15.121:0/3277557689 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc009370 con 0x7f2fec0ab0d0 2023-11-15T03:11:20.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2ff1544700 1 --2- 172.21.15.121:0/3277557689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fec0ab0d0 0x7f2fec0a6d50 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 rx=0x7f2fdc00fba0 tx=0x7f2fdc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:20.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2fe1ffb700 1 -- 172.21.15.121:0/3277557689 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc011ca0 con 0x7f2fec0ab0d0 2023-11-15T03:11:20.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2fe1ffb700 1 -- 172.21.15.121:0/3277557689 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fdc011e00 con 0x7f2fec0ab0d0 2023-11-15T03:11:20.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2ff2d47700 1 -- 172.21.15.121:0/3277557689 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fec0a79f0 con 0x7f2fec0ab0d0 2023-11-15T03:11:20.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2fe1ffb700 1 -- 172.21.15.121:0/3277557689 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc01d680 con 0x7f2fec0ab0d0 2023-11-15T03:11:20.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.628+0000 7f2ff2d47700 1 -- 172.21.15.121:0/3277557689 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fec0a7fb0 con 0x7f2fec0ab0d0 2023-11-15T03:11:20.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.632+0000 7f2ff2d47700 1 -- 172.21.15.121:0/3277557689 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fec0a84b0 con 0x7f2fec0ab0d0 2023-11-15T03:11:20.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.632+0000 7f2fe1ffb700 1 -- 172.21.15.121:0/3277557689 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2fdc01da70 con 0x7f2fec0ab0d0 2023-11-15T03:11:20.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.632+0000 7f2fe1ffb700 1 --2- 172.21.15.121:0/3277557689 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2fd005a510 0x7f2fd005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:20.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.632+0000 7f2fe1ffb700 1 -- 172.21.15.121:0/3277557689 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f2fdc075ff0 con 0x7f2fec0ab0d0 2023-11-15T03:11:20.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.632+0000 7f2ff1d45700 1 --2- 172.21.15.121:0/3277557689 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2fd005a510 0x7f2fd005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:20.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.636+0000 7f2ff1d45700 1 --2- 172.21.15.121:0/3277557689 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2fd005a510 0x7f2fd005c9d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f2fe8009200 tx=0x7f2fe8008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:20.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.640+0000 7f2fe1ffb700 1 -- 172.21.15.121:0/3277557689 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2fdc0499b0 con 0x7f2fec0ab0d0 2023-11-15T03:11:20.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:20.804+0000 7f2ff2d47700 1 -- 172.21.15.121:0/3277557689 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2fec06bcf0 con 0x7f2fec0ab0d0 2023-11-15T03:11:21.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.424+0000 7f2fe1ffb700 1 -- 172.21.15.121:0/3277557689 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v143) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2fdc021070 con 0x7f2fec0ab0d0 2023-11-15T03:11:21.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.432+0000 7f2ff2d47700 1 -- 172.21.15.121:0/3277557689 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2fd005a510 msgr2=0x7f2fd005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:21.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.432+0000 7f2ff2d47700 1 --2- 172.21.15.121:0/3277557689 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2fd005a510 0x7f2fd005c9d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f2fe8009200 tx=0x7f2fe8008040).stop 2023-11-15T03:11:21.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.432+0000 7f2ff2d47700 1 -- 172.21.15.121:0/3277557689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fec0ab0d0 msgr2=0x7f2fec0a6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:21.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.432+0000 7f2ff2d47700 1 --2- 172.21.15.121:0/3277557689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fec0ab0d0 0x7f2fec0a6d50 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 rx=0x7f2fdc00fba0 tx=0x7f2fdc017000).stop 2023-11-15T03:11:21.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.436+0000 7f2ff2d47700 1 -- 172.21.15.121:0/3277557689 shutdown_connections 2023-11-15T03:11:21.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.436+0000 7f2ff2d47700 1 --2- 172.21.15.121:0/3277557689 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2fd005a510 0x7f2fd005c9d0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:21.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.436+0000 7f2ff2d47700 1 --2- 172.21.15.121:0/3277557689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec0ad1b0 0x7f2fec0cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:21.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.436+0000 7f2ff2d47700 1 --2- 172.21.15.121:0/3277557689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fec0ab0d0 0x7f2fec0a6d50 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:21.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.436+0000 7f2ff2d47700 1 --2- 172.21.15.121:0/3277557689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec065c00 0x7f2fec141b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:21.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.436+0000 7f2ff2d47700 1 -- 172.21.15.121:0/3277557689 >> 172.21.15.121:0/3277557689 conn(0x7f2fec067a60 msgr2=0x7f2fec155ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:21.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.440+0000 7f2ff2d47700 1 -- 172.21.15.121:0/3277557689 shutdown_connections 2023-11-15T03:11:21.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.440+0000 7f2ff2d47700 1 -- 172.21.15.121:0/3277557689 wait complete. 2023-11-15T03:11:21.454 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_data' created 2023-11-15T03:11:21.480 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-15T03:11:21.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.636+0000 7faf2949b700 1 Processor -- start 2023-11-15T03:11:21.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.636+0000 7faf2949b700 1 -- start start 2023-11-15T03:11:21.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.640+0000 7faf2949b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf2414e5a0 0x7faf240c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:21.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.640+0000 7faf2949b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7faf240ad370 0x7faf240c7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:21.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.640+0000 7faf2949b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf24155460 con 0x7faf24154fa0 2023-11-15T03:11:21.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.640+0000 7faf2949b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf240acf20 con 0x7faf240ad370 2023-11-15T03:11:21.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.640+0000 7faf2949b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf240c5dd0 con 0x7faf2414e5a0 2023-11-15T03:11:21.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf237fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7faf240ad370 0x7faf240c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:21.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf22ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf2414e5a0 0x7faf240c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:21.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf237fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7faf240ad370 0x7faf240c7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44852/0 (socket says 172.21.15.121:44852) 2023-11-15T03:11:21.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf22ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf2414e5a0 0x7faf240c5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:39982/0 (socket says 172.21.15.121:39982) 2023-11-15T03:11:21.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf237fe700 1 -- 172.21.15.121:0/3300708127 learned_addr learned my addr 172.21.15.121:0/3300708127 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:21.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf227fc700 1 --1- 172.21.15.121:0/3300708127 >> v1:172.21.15.190:6789/0 conn(0x7faf24154fa0 0x7faf24065780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:60932/0 (socket says 172.21.15.121:60932) 2023-11-15T03:11:21.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf237fe700 1 -- 172.21.15.121:0/3300708127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf2414e5a0 msgr2=0x7faf240c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:21.655 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf237fe700 1 --2- 172.21.15.121:0/3300708127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf2414e5a0 0x7faf240c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:21.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf237fe700 1 -- 172.21.15.121:0/3300708127 >> v1:172.21.15.190:6789/0 conn(0x7faf24154fa0 legacy=0x7faf24065780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:21.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf237fe700 1 -- 172.21.15.121:0/3300708127 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf240c60b0 con 0x7faf240ad370 2023-11-15T03:11:21.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf237fe700 1 --2- 172.21.15.121:0/3300708127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf240ad370 0x7faf240c7140 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 rx=0x7faf1c009f20 tx=0x7faf1c009840).ready entity=mon.0 client_cookie=8974925c6299fcc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:21.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf21ffb700 1 -- 172.21.15.121:0/3300708127 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c00e070 con 0x7faf240ad370 2023-11-15T03:11:21.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf21ffb700 1 -- 172.21.15.121:0/3300708127 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1c00b9d0 con 0x7faf240ad370 2023-11-15T03:11:21.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf21ffb700 1 -- 172.21.15.121:0/3300708127 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c00bd50 con 0x7faf240ad370 2023-11-15T03:11:21.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf2949b700 1 -- 172.21.15.121:0/3300708127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf240ad370 msgr2=0x7faf240c7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:21.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf2949b700 1 --2- 172.21.15.121:0/3300708127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf240ad370 0x7faf240c7140 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 rx=0x7faf1c009f20 tx=0x7faf1c009840).stop 2023-11-15T03:11:21.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf2949b700 1 -- 172.21.15.121:0/3300708127 shutdown_connections 2023-11-15T03:11:21.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf2949b700 1 --2- 172.21.15.121:0/3300708127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf240ad370 0x7faf240c7140 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:21.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf2949b700 1 --2- 172.21.15.121:0/3300708127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf2414e5a0 0x7faf240c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:21.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf2949b700 1 -- 172.21.15.121:0/3300708127 >> 172.21.15.121:0/3300708127 conn(0x7faf24067a60 msgr2=0x7faf24067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:21.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf2949b700 1 -- 172.21.15.121:0/3300708127 shutdown_connections 2023-11-15T03:11:21.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.644+0000 7faf2949b700 1 -- 172.21.15.121:0/3300708127 wait complete. 2023-11-15T03:11:21.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.648+0000 7faf2949b700 1 Processor -- start 2023-11-15T03:11:21.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.648+0000 7faf2949b700 1 -- start start 2023-11-15T03:11:21.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.648+0000 7faf2949b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf240ad370 0x7faf24143c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:21.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.648+0000 7faf2949b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf2414e5a0 0x7faf2414cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:21.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.648+0000 7faf2949b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf24154fa0 0x7faf2414d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:21.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.648+0000 7faf2949b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2414d5a0 con 0x7faf240ad370 2023-11-15T03:11:21.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.648+0000 7faf2949b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf240b09a0 con 0x7faf2414e5a0 2023-11-15T03:11:21.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.648+0000 7faf2949b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf240afd90 con 0x7faf24154fa0 2023-11-15T03:11:21.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.648+0000 7faf22ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf240ad370 0x7faf24143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:21.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.648+0000 7faf22ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf240ad370 0x7faf24143c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44856/0 (socket says 172.21.15.121:44856) 2023-11-15T03:11:21.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.648+0000 7faf22ffd700 1 -- 172.21.15.121:0/3139494629 learned_addr learned my addr 172.21.15.121:0/3139494629 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:21.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.648+0000 7faf227fc700 1 --2- 172.21.15.121:0/3139494629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf2414e5a0 0x7faf2414cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:21.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.648+0000 7faf237fe700 1 --2- 172.21.15.121:0/3139494629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf24154fa0 0x7faf2414d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:21.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.648+0000 7faf22ffd700 1 -- 172.21.15.121:0/3139494629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf24154fa0 msgr2=0x7faf2414d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:21.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.648+0000 7faf22ffd700 1 --2- 172.21.15.121:0/3139494629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf24154fa0 0x7faf2414d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:21.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.648+0000 7faf22ffd700 1 -- 172.21.15.121:0/3139494629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf2414e5a0 msgr2=0x7faf2414cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:21.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.648+0000 7faf22ffd700 1 --2- 172.21.15.121:0/3139494629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf2414e5a0 0x7faf2414cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:21.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.648+0000 7faf22ffd700 1 -- 172.21.15.121:0/3139494629 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1c009ae0 con 0x7faf240ad370 2023-11-15T03:11:21.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.648+0000 7faf22ffd700 1 --2- 172.21.15.121:0/3139494629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf240ad370 0x7faf24143c60 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 rx=0x7faf1400e8d0 tx=0x7faf14006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:21.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.652+0000 7faf0affd700 1 -- 172.21.15.121:0/3139494629 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf14009450 con 0x7faf240ad370 2023-11-15T03:11:21.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.652+0000 7faf0affd700 1 -- 172.21.15.121:0/3139494629 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf14012600 con 0x7faf240ad370 2023-11-15T03:11:21.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.652+0000 7faf2949b700 1 -- 172.21.15.121:0/3139494629 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf240b0070 con 0x7faf240ad370 2023-11-15T03:11:21.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.652+0000 7faf0affd700 1 -- 172.21.15.121:0/3139494629 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf14012980 con 0x7faf240ad370 2023-11-15T03:11:21.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.652+0000 7faf2949b700 1 -- 172.21.15.121:0/3139494629 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf240b0630 con 0x7faf240ad370 2023-11-15T03:11:21.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.652+0000 7faf2949b700 1 -- 172.21.15.121:0/3139494629 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf2406bcf0 con 0x7faf240ad370 2023-11-15T03:11:21.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.652+0000 7faf0affd700 1 -- 172.21.15.121:0/3139494629 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7faf14012cf0 con 0x7faf240ad370 2023-11-15T03:11:21.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.656+0000 7faf0affd700 1 --2- 172.21.15.121:0/3139494629 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faf1005a510 0x7faf1005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:21.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.656+0000 7faf0affd700 1 -- 172.21.15.121:0/3139494629 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6021+0+0 (secure 0 0 0) 0x7faf14074b00 con 0x7faf240ad370 2023-11-15T03:11:21.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.656+0000 7faf227fc700 1 --2- 172.21.15.121:0/3139494629 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faf1005a510 0x7faf1005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:21.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.656+0000 7faf227fc700 1 --2- 172.21.15.121:0/3139494629 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faf1005a510 0x7faf1005c9d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7faf2414d8b0 tx=0x7faf18007d30).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:21.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.660+0000 7faf0affd700 1 -- 172.21.15.121:0/3139494629 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faf14048340 con 0x7faf240ad370 2023-11-15T03:11:21.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.860+0000 7faf2949b700 1 -- 172.21.15.121:0/3139494629 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7faf2414e980 con 0x7faf240ad370 2023-11-15T03:11:21.882 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.872+0000 7faf0affd700 1 -- 172.21.15.121:0/3139494629 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v346) v1 ==== 174+0+0 (secure 0 0 0) 0x7faf14016020 con 0x7faf240ad370 2023-11-15T03:11:21.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.880+0000 7faf2949b700 1 -- 172.21.15.121:0/3139494629 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faf1005a510 msgr2=0x7faf1005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:21.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.880+0000 7faf2949b700 1 --2- 172.21.15.121:0/3139494629 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faf1005a510 0x7faf1005c9d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7faf2414d8b0 tx=0x7faf18007d30).stop 2023-11-15T03:11:21.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.880+0000 7faf2949b700 1 -- 172.21.15.121:0/3139494629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf240ad370 msgr2=0x7faf24143c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:21.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.880+0000 7faf2949b700 1 --2- 172.21.15.121:0/3139494629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf240ad370 0x7faf24143c60 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 rx=0x7faf1400e8d0 tx=0x7faf14006f40).stop 2023-11-15T03:11:21.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.880+0000 7faf2949b700 1 -- 172.21.15.121:0/3139494629 shutdown_connections 2023-11-15T03:11:21.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.880+0000 7faf2949b700 1 --2- 172.21.15.121:0/3139494629 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faf1005a510 0x7faf1005c9d0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:21.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.880+0000 7faf2949b700 1 --2- 172.21.15.121:0/3139494629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf24154fa0 0x7faf2414d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:21.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.880+0000 7faf2949b700 1 --2- 172.21.15.121:0/3139494629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf2414e5a0 0x7faf2414cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:21.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.880+0000 7faf2949b700 1 --2- 172.21.15.121:0/3139494629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf240ad370 0x7faf24143c60 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:21.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.880+0000 7faf2949b700 1 -- 172.21.15.121:0/3139494629 >> 172.21.15.121:0/3139494629 conn(0x7faf24067a60 msgr2=0x7faf240c08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:21.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.880+0000 7faf2949b700 1 -- 172.21.15.121:0/3139494629 shutdown_connections 2023-11-15T03:11:21.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:21.880+0000 7faf2949b700 1 -- 172.21.15.121:0/3139494629 wait complete. 2023-11-15T03:11:21.895 INFO:teuthology.orchestra.run.smithi121.stderr:new fs with metadata pool 15 and data pool 16 2023-11-15T03:11:21.920 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:11:22.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.068+0000 7f44bc29b700 1 Processor -- start 2023-11-15T03:11:22.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.068+0000 7f44bc29b700 1 -- start start 2023-11-15T03:11:22.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.072+0000 7f44bc29b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f44b4154ca0 0x7f44b4155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:22.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.072+0000 7f44bc29b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f44b41562f0 0x7f44b40a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:22.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.072+0000 7f44bc29b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b40c4c30 con 0x7f44b40a8f40 2023-11-15T03:11:22.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.072+0000 7f44bc29b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b40a8280 con 0x7f44b41562f0 2023-11-15T03:11:22.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.072+0000 7f44bc29b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b40a6d40 con 0x7f44b4154ca0 2023-11-15T03:11:22.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.072+0000 7f44b9836700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f44b41562f0 0x7f44b40a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:22.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.072+0000 7f44b9836700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f44b41562f0 0x7f44b40a8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44860/0 (socket says 172.21.15.121:44860) 2023-11-15T03:11:22.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.072+0000 7f44b9836700 1 -- 172.21.15.121:0/3736944303 learned_addr learned my addr 172.21.15.121:0/3736944303 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:22.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.072+0000 7f44ba037700 1 --2- 172.21.15.121:0/3736944303 >> v2:172.21.15.190:3300/0 conn(0x7f44b4154ca0 0x7f44b4155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:22.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.072+0000 7f44b9836700 1 -- 172.21.15.121:0/3736944303 >> v2:172.21.15.190:3300/0 conn(0x7f44b4154ca0 msgr2=0x7f44b4155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:22.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.072+0000 7f44b9836700 1 --2- 172.21.15.121:0/3736944303 >> v2:172.21.15.190:3300/0 conn(0x7f44b4154ca0 0x7f44b4155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.072+0000 7f44b9836700 1 -- 172.21.15.121:0/3736944303 >> v1:172.21.15.190:6789/0 conn(0x7f44b40a8f40 legacy=0x7f44b40a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:22.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.072+0000 7f44b9836700 1 -- 172.21.15.121:0/3736944303 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b40a6f50 con 0x7f44b41562f0 2023-11-15T03:11:22.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.076+0000 7f44b9836700 1 --2- 172.21.15.121:0/3736944303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44b41562f0 0x7f44b40a8a50 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 rx=0x7f44b40a6e80 tx=0x7f44a40090f0).ready entity=mon.0 client_cookie=21b29f16a50b25de server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:22.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.076+0000 7f44b9035700 1 -- 172.21.15.121:0/3736944303 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a400b600 con 0x7f44b41562f0 2023-11-15T03:11:22.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.076+0000 7f44b9035700 1 -- 172.21.15.121:0/3736944303 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a400b760 con 0x7f44b41562f0 2023-11-15T03:11:22.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.076+0000 7f44b9035700 1 -- 172.21.15.121:0/3736944303 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a400bae0 con 0x7f44b41562f0 2023-11-15T03:11:22.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.076+0000 7f44bc29b700 1 -- 172.21.15.121:0/3736944303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44b41562f0 msgr2=0x7f44b40a8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:22.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.076+0000 7f44bc29b700 1 --2- 172.21.15.121:0/3736944303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44b41562f0 0x7f44b40a8a50 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 rx=0x7f44b40a6e80 tx=0x7f44a40090f0).stop 2023-11-15T03:11:22.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.076+0000 7f44bc29b700 1 -- 172.21.15.121:0/3736944303 shutdown_connections 2023-11-15T03:11:22.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.076+0000 7f44bc29b700 1 --2- 172.21.15.121:0/3736944303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44b41562f0 0x7f44b40a8a50 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.076+0000 7f44bc29b700 1 --2- 172.21.15.121:0/3736944303 >> v2:172.21.15.190:3300/0 conn(0x7f44b4154ca0 0x7f44b4155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.076+0000 7f44bc29b700 1 -- 172.21.15.121:0/3736944303 >> 172.21.15.121:0/3736944303 conn(0x7f44b4062190 msgr2=0x7f44b40625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:22.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.076+0000 7f44bc29b700 1 -- 172.21.15.121:0/3736944303 shutdown_connections 2023-11-15T03:11:22.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.076+0000 7f44bc29b700 1 -- 172.21.15.121:0/3736944303 wait complete. 2023-11-15T03:11:22.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.076+0000 7f44bc29b700 1 Processor -- start 2023-11-15T03:11:22.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.076+0000 7f44bc29b700 1 -- start start 2023-11-15T03:11:22.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44bc29b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44b40a8f40 0x7f44b414e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:22.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44bc29b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44b4154ca0 0x7f44b40c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:22.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44bc29b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44b41562f0 0x7f44b4067ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:22.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44bc29b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b40c65b0 con 0x7f44b4154ca0 2023-11-15T03:11:22.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44bc29b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b40c59a0 con 0x7f44b40a8f40 2023-11-15T03:11:22.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44bc29b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b40c5c80 con 0x7f44b41562f0 2023-11-15T03:11:22.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44b9836700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44b4154ca0 0x7f44b40c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:22.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44b9836700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44b4154ca0 0x7f44b40c4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44866/0 (socket says 172.21.15.121:44866) 2023-11-15T03:11:22.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44b9836700 1 -- 172.21.15.121:0/1678366566 learned_addr learned my addr 172.21.15.121:0/1678366566 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:22.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44ba037700 1 --2- 172.21.15.121:0/1678366566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44b40a8f40 0x7f44b414e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:22.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44ba838700 1 --2- 172.21.15.121:0/1678366566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44b41562f0 0x7f44b4067ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:22.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44b9836700 1 -- 172.21.15.121:0/1678366566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44b41562f0 msgr2=0x7f44b4067ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:22.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44b9836700 1 --2- 172.21.15.121:0/1678366566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44b41562f0 0x7f44b4067ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44b9836700 1 -- 172.21.15.121:0/1678366566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44b40a8f40 msgr2=0x7f44b414e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:22.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44b9836700 1 --2- 172.21.15.121:0/1678366566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44b40a8f40 0x7f44b414e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44b9836700 1 -- 172.21.15.121:0/1678366566 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a4009370 con 0x7f44b4154ca0 2023-11-15T03:11:22.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44b9836700 1 --2- 172.21.15.121:0/1678366566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44b4154ca0 0x7f44b40c4fc0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 rx=0x7f44b40c0d10 tx=0x7f44a4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:22.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44aa7fc700 1 -- 172.21.15.121:0/1678366566 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a400b8d0 con 0x7f44b4154ca0 2023-11-15T03:11:22.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44aa7fc700 1 -- 172.21.15.121:0/1678366566 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a400ba30 con 0x7f44b4154ca0 2023-11-15T03:11:22.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44bc29b700 1 -- 172.21.15.121:0/1678366566 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b40c5f60 con 0x7f44b4154ca0 2023-11-15T03:11:22.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44aa7fc700 1 -- 172.21.15.121:0/1678366566 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a401c630 con 0x7f44b4154ca0 2023-11-15T03:11:22.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.080+0000 7f44bc29b700 1 -- 172.21.15.121:0/1678366566 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b414e740 con 0x7f44b4154ca0 2023-11-15T03:11:22.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.084+0000 7f44bc29b700 1 -- 172.21.15.121:0/1678366566 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b406c150 con 0x7f44b4154ca0 2023-11-15T03:11:22.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.084+0000 7f44aa7fc700 1 -- 172.21.15.121:0/1678366566 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f44a401ca60 con 0x7f44b4154ca0 2023-11-15T03:11:22.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.084+0000 7f44aa7fc700 1 --2- 172.21.15.121:0/1678366566 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f449805a540 0x7f449805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:22.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.084+0000 7f44aa7fc700 1 -- 172.21.15.121:0/1678366566 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f44a4076540 con 0x7f44b4154ca0 2023-11-15T03:11:22.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.084+0000 7f44ba037700 1 --2- 172.21.15.121:0/1678366566 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f449805a540 0x7f449805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:22.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.088+0000 7f44ba037700 1 --2- 172.21.15.121:0/1678366566 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f449805a540 0x7f449805ca00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7f44b000bb20 tx=0x7f44b0008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:22.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.092+0000 7f44aa7fc700 1 -- 172.21.15.121:0/1678366566 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f44a4049ca0 con 0x7f44b4154ca0 2023-11-15T03:11:22.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.256+0000 7f44bc29b700 1 -- 172.21.15.121:0/1678366566 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44b40c66f0 con 0x7f44b4154ca0 2023-11-15T03:11:22.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.256+0000 7f44aa7fc700 1 -- 172.21.15.121:0/1678366566 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+17003 (secure 0 0 0) 0x7f44b40c66f0 con 0x7f44b4154ca0 2023-11-15T03:11:22.267 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:22.267 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":144,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:11:21.863348+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-15T03:11:19.965438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-15T03:11:20.878094+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:11:22.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.260+0000 7f44bc29b700 1 -- 172.21.15.121:0/1678366566 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f449805a540 msgr2=0x7f449805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:22.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.260+0000 7f44bc29b700 1 --2- 172.21.15.121:0/1678366566 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f449805a540 0x7f449805ca00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7f44b000bb20 tx=0x7f44b0008040).stop 2023-11-15T03:11:22.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.260+0000 7f44bc29b700 1 -- 172.21.15.121:0/1678366566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44b4154ca0 msgr2=0x7f44b40c4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:22.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.260+0000 7f44bc29b700 1 --2- 172.21.15.121:0/1678366566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44b4154ca0 0x7f44b40c4fc0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 rx=0x7f44b40c0d10 tx=0x7f44a4017000).stop 2023-11-15T03:11:22.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.260+0000 7f44bc29b700 1 -- 172.21.15.121:0/1678366566 shutdown_connections 2023-11-15T03:11:22.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.260+0000 7f44bc29b700 1 --2- 172.21.15.121:0/1678366566 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f449805a540 0x7f449805ca00 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.260+0000 7f44bc29b700 1 --2- 172.21.15.121:0/1678366566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44b41562f0 0x7f44b4067ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.260+0000 7f44bc29b700 1 --2- 172.21.15.121:0/1678366566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44b4154ca0 0x7f44b40c4fc0 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.260+0000 7f44bc29b700 1 --2- 172.21.15.121:0/1678366566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44b40a8f40 0x7f44b414e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.260+0000 7f44bc29b700 1 -- 172.21.15.121:0/1678366566 >> 172.21.15.121:0/1678366566 conn(0x7f44b4062190 msgr2=0x7f44b4065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:22.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.260+0000 7f44bc29b700 1 -- 172.21.15.121:0/1678366566 shutdown_connections 2023-11-15T03:11:22.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.260+0000 7f44bc29b700 1 -- 172.21.15.121:0/1678366566 wait complete. 2023-11-15T03:11:22.290 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:11:22.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.448+0000 7fc2f89a8700 1 Processor -- start 2023-11-15T03:11:22.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.448+0000 7fc2f89a8700 1 -- start start 2023-11-15T03:11:22.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.452+0000 7fc2f89a8700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc2f4066320 0x7fc2f4154110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:22.460 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.452+0000 7fc2f89a8700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f40c73e0 con 0x7fc2f4066700 2023-11-15T03:11:22.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.452+0000 7fc2f89a8700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f41546c0 con 0x7fc2f40acf10 2023-11-15T03:11:22.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.452+0000 7fc2f89a8700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2f41549a0 con 0x7fc2f4066320 2023-11-15T03:11:22.461 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.452+0000 7fc2f2d9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc2f4066320 0x7fc2f4154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:22.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.452+0000 7fc2f2d9d700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc2f4066320 0x7fc2f4154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44874/0 (socket says 172.21.15.121:44874) 2023-11-15T03:11:22.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.452+0000 7fc2f2d9d700 1 -- 172.21.15.121:0/2816414250 learned_addr learned my addr 172.21.15.121:0/2816414250 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:22.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.452+0000 7fc2f2d9d700 1 -- 172.21.15.121:0/2816414250 >> v1:172.21.15.190:6789/0 conn(0x7fc2f40acf10 legacy=0x7fc2f4062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:22.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.452+0000 7fc2f2d9d700 1 -- 172.21.15.121:0/2816414250 >> v1:172.21.15.121:6789/0 conn(0x7fc2f4066700 legacy=0x7fc2f40656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:22.463 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.452+0000 7fc2f2d9d700 1 -- 172.21.15.121:0/2816414250 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f4154c80 con 0x7fc2f4066320 2023-11-15T03:11:22.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.452+0000 7fc2f2d9d700 1 --2- 172.21.15.121:0/2816414250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f4066320 0x7fc2f4154110 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 rx=0x7fc2e4006fd0 tx=0x7fc2e40090f0).ready entity=mon.0 client_cookie=801c7f939d87aa17 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:22.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.452+0000 7fc2f1d9b700 1 -- 172.21.15.121:0/2816414250 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e400b600 con 0x7fc2f4066320 2023-11-15T03:11:22.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.452+0000 7fc2f1d9b700 1 -- 172.21.15.121:0/2816414250 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e400b760 con 0x7fc2f4066320 2023-11-15T03:11:22.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.452+0000 7fc2f1d9b700 1 -- 172.21.15.121:0/2816414250 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e400bae0 con 0x7fc2f4066320 2023-11-15T03:11:22.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.456+0000 7fc2f89a8700 1 -- 172.21.15.121:0/2816414250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f4066320 msgr2=0x7fc2f4154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:22.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.456+0000 7fc2f89a8700 1 --2- 172.21.15.121:0/2816414250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f4066320 0x7fc2f4154110 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 rx=0x7fc2e4006fd0 tx=0x7fc2e40090f0).stop 2023-11-15T03:11:22.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.456+0000 7fc2f89a8700 1 -- 172.21.15.121:0/2816414250 shutdown_connections 2023-11-15T03:11:22.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.456+0000 7fc2f89a8700 1 --2- 172.21.15.121:0/2816414250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f4066320 0x7fc2f4154110 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.456+0000 7fc2f89a8700 1 -- 172.21.15.121:0/2816414250 >> 172.21.15.121:0/2816414250 conn(0x7fc2f4067a60 msgr2=0x7fc2f4067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:22.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.456+0000 7fc2f89a8700 1 -- 172.21.15.121:0/2816414250 shutdown_connections 2023-11-15T03:11:22.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.456+0000 7fc2f89a8700 1 -- 172.21.15.121:0/2816414250 wait complete. 2023-11-15T03:11:22.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.456+0000 7fc2f89a8700 1 Processor -- start 2023-11-15T03:11:22.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f89a8700 1 -- start start 2023-11-15T03:11:22.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f89a8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f4066320 0x7fc2f41478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:22.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f89a8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2f4066700 0x7fc2f40af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:22.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f89a8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f40acf10 0x7fc2f40afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:22.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f89a8700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f40b0990 con 0x7fc2f40acf10 2023-11-15T03:11:22.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f89a8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f41462e0 con 0x7fc2f4066320 2023-11-15T03:11:22.470 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f89a8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f414d2e0 con 0x7fc2f4066700 2023-11-15T03:11:22.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f40acf10 0x7fc2f40afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:22.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f40acf10 0x7fc2f40afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44886/0 (socket says 172.21.15.121:44886) 2023-11-15T03:11:22.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f359e700 1 -- 172.21.15.121:0/488463097 learned_addr learned my addr 172.21.15.121:0/488463097 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:22.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f2d9d700 1 --2- 172.21.15.121:0/488463097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f4066320 0x7fc2f41478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:22.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f259c700 1 --2- 172.21.15.121:0/488463097 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2f4066700 0x7fc2f40af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:22.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f359e700 1 -- 172.21.15.121:0/488463097 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2f4066700 msgr2=0x7fc2f40af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:22.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f359e700 1 --2- 172.21.15.121:0/488463097 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2f4066700 0x7fc2f40af880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f359e700 1 -- 172.21.15.121:0/488463097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f4066320 msgr2=0x7fc2f41478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:22.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f359e700 1 --2- 172.21.15.121:0/488463097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f4066320 0x7fc2f41478b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f359e700 1 -- 172.21.15.121:0/488463097 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e4009370 con 0x7fc2f40acf10 2023-11-15T03:11:22.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f259c700 1 --2- 172.21.15.121:0/488463097 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2f4066700 0x7fc2f40af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:22.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f2d9d700 1 --2- 172.21.15.121:0/488463097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f4066320 0x7fc2f41478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:22.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f359e700 1 --2- 172.21.15.121:0/488463097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f40acf10 0x7fc2f40afd70 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 rx=0x7fc2ec00c7f0 tx=0x7fc2ec001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:22.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2daffd700 1 -- 172.21.15.121:0/488463097 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec010510 con 0x7fc2f40acf10 2023-11-15T03:11:22.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2daffd700 1 -- 172.21.15.121:0/488463097 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec010670 con 0x7fc2f40acf10 2023-11-15T03:11:22.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2f89a8700 1 -- 172.21.15.121:0/488463097 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f40b03d0 con 0x7fc2f40acf10 2023-11-15T03:11:22.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.460+0000 7fc2daffd700 1 -- 172.21.15.121:0/488463097 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec0109f0 con 0x7fc2f40acf10 2023-11-15T03:11:22.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.464+0000 7fc2f89a8700 1 -- 172.21.15.121:0/488463097 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f414cf70 con 0x7fc2f40acf10 2023-11-15T03:11:22.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.464+0000 7fc2f89a8700 1 -- 172.21.15.121:0/488463097 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f414cdd0 con 0x7fc2f40acf10 2023-11-15T03:11:22.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.464+0000 7fc2daffd700 1 -- 172.21.15.121:0/488463097 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc2ec01b400 con 0x7fc2f40acf10 2023-11-15T03:11:22.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.464+0000 7fc2daffd700 1 --2- 172.21.15.121:0/488463097 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc2e005a510 0x7fc2e005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:22.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.468+0000 7fc2daffd700 1 -- 172.21.15.121:0/488463097 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc2ec074590 con 0x7fc2f40acf10 2023-11-15T03:11:22.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.468+0000 7fc2f2d9d700 1 --2- 172.21.15.121:0/488463097 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc2e005a510 0x7fc2e005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:22.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.468+0000 7fc2f2d9d700 1 --2- 172.21.15.121:0/488463097 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc2e005a510 0x7fc2e005c9d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7fc2e4009130 tx=0x7fc2e4017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:22.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.472+0000 7fc2daffd700 1 -- 172.21.15.121:0/488463097 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc2f414cdd0 con 0x7fc2f40acf10 2023-11-15T03:11:22.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.636+0000 7fc2f89a8700 1 -- 172.21.15.121:0/488463097 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2f414cdd0 con 0x7fc2f40acf10 2023-11-15T03:11:22.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.640+0000 7fc2daffd700 1 -- 172.21.15.121:0/488463097 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+17003 (secure 0 0 0) 0x7fc2f414cdd0 con 0x7fc2f40acf10 2023-11-15T03:11:22.648 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:22.648 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":144,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:11:21.863348+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-15T03:11:19.965438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-15T03:11:20.878094+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:11:22.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.640+0000 7fc2f89a8700 1 -- 172.21.15.121:0/488463097 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc2e005a510 msgr2=0x7fc2e005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:22.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.640+0000 7fc2f89a8700 1 --2- 172.21.15.121:0/488463097 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc2e005a510 0x7fc2e005c9d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7fc2e4009130 tx=0x7fc2e4017000).stop 2023-11-15T03:11:22.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.640+0000 7fc2f89a8700 1 -- 172.21.15.121:0/488463097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f40acf10 msgr2=0x7fc2f40afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:22.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.640+0000 7fc2f89a8700 1 --2- 172.21.15.121:0/488463097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f40acf10 0x7fc2f40afd70 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 rx=0x7fc2ec00c7f0 tx=0x7fc2ec001510).stop 2023-11-15T03:11:22.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.640+0000 7fc2f89a8700 1 -- 172.21.15.121:0/488463097 shutdown_connections 2023-11-15T03:11:22.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.640+0000 7fc2f89a8700 1 --2- 172.21.15.121:0/488463097 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc2e005a510 0x7fc2e005c9d0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.640+0000 7fc2f89a8700 1 --2- 172.21.15.121:0/488463097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f40acf10 0x7fc2f40afd70 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.640+0000 7fc2f89a8700 1 --2- 172.21.15.121:0/488463097 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2f4066700 0x7fc2f40af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.640+0000 7fc2f89a8700 1 --2- 172.21.15.121:0/488463097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f4066320 0x7fc2f41478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.640+0000 7fc2f89a8700 1 -- 172.21.15.121:0/488463097 >> 172.21.15.121:0/488463097 conn(0x7fc2f4067a60 msgr2=0x7fc2f41532e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:22.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.640+0000 7fc2f89a8700 1 -- 172.21.15.121:0/488463097 shutdown_connections 2023-11-15T03:11:22.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.644+0000 7fc2f89a8700 1 -- 172.21.15.121:0/488463097 wait complete. 2023-11-15T03:11:22.670 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-15T03:11:22.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.832+0000 7f43cffff700 1 Processor -- start 2023-11-15T03:11:22.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.832+0000 7f43cffff700 1 -- start start 2023-11-15T03:11:22.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.832+0000 7f43cffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f43d00acd00 0x7f43d00ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:22.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.832+0000 7f43cffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43d00c5c70 0x7f43d01536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:22.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.832+0000 7f43cffff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d0065ea0 con 0x7f43d0066330 2023-11-15T03:11:22.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.832+0000 7f43cffff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d0066bb0 con 0x7f43d00acd00 2023-11-15T03:11:22.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.832+0000 7f43cffff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d0065890 con 0x7f43d00c5c70 2023-11-15T03:11:22.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.836+0000 7f43cf7fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f43d0066330 0x7f43d0153bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:60978/0 (socket says 172.21.15.121:60978) 2023-11-15T03:11:22.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.836+0000 7f43cf7fe700 1 -- 172.21.15.121:0/3202345635 learned_addr learned my addr 172.21.15.121:0/3202345635 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:22.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.836+0000 7f43ceffd700 1 --2- 172.21.15.121:0/3202345635 >> v2:172.21.15.190:3300/0 conn(0x7f43d00acd00 0x7f43d00ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:22.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.836+0000 7f43ce7fc700 1 --2- 172.21.15.121:0/3202345635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43d00c5c70 0x7f43d01536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:22.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.836+0000 7f43cdffb700 1 -- 172.21.15.121:0/3202345635 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443117515 0 0) 0x7f43d0065ea0 con 0x7f43d0066330 2023-11-15T03:11:22.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.836+0000 7f43cdffb700 1 -- 172.21.15.121:0/3202345635 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43b80033f0 con 0x7f43d0066330 2023-11-15T03:11:22.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.836+0000 7f43ce7fc700 1 -- 172.21.15.121:0/3202345635 >> v2:172.21.15.190:3300/0 conn(0x7f43d00acd00 msgr2=0x7f43d00ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:22.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.836+0000 7f43ce7fc700 1 --2- 172.21.15.121:0/3202345635 >> v2:172.21.15.190:3300/0 conn(0x7f43d00acd00 0x7f43d00ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.836+0000 7f43ce7fc700 1 -- 172.21.15.121:0/3202345635 >> v1:172.21.15.190:6789/0 conn(0x7f43d0066330 legacy=0x7f43d0153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:22.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.836+0000 7f43ce7fc700 1 -- 172.21.15.121:0/3202345635 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d00c64a0 con 0x7f43d00c5c70 2023-11-15T03:11:22.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.836+0000 7f43ce7fc700 1 --2- 172.21.15.121:0/3202345635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43d00c5c70 0x7f43d01536c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f43c4009760 tx=0x7f43c4009060).ready entity=mon.2 client_cookie=4be43ec3bc1acbc server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:22.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.836+0000 7f43cdffb700 1 -- 172.21.15.121:0/3202345635 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c400d4a0 con 0x7f43d00c5c70 2023-11-15T03:11:22.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.836+0000 7f43cdffb700 1 -- 172.21.15.121:0/3202345635 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c400d600 con 0x7f43d00c5c70 2023-11-15T03:11:22.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.836+0000 7f43cdffb700 1 -- 172.21.15.121:0/3202345635 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c400d980 con 0x7f43d00c5c70 2023-11-15T03:11:22.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.840+0000 7f43cffff700 1 -- 172.21.15.121:0/3202345635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43d00c5c70 msgr2=0x7f43d01536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:22.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.840+0000 7f43cffff700 1 --2- 172.21.15.121:0/3202345635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43d00c5c70 0x7f43d01536c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f43c4009760 tx=0x7f43c4009060).stop 2023-11-15T03:11:22.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.840+0000 7f43cffff700 1 -- 172.21.15.121:0/3202345635 shutdown_connections 2023-11-15T03:11:22.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.840+0000 7f43cffff700 1 --2- 172.21.15.121:0/3202345635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43d00c5c70 0x7f43d01536c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.840+0000 7f43cffff700 1 --2- 172.21.15.121:0/3202345635 >> v2:172.21.15.190:3300/0 conn(0x7f43d00acd00 0x7f43d00ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.840+0000 7f43cffff700 1 -- 172.21.15.121:0/3202345635 >> 172.21.15.121:0/3202345635 conn(0x7f43d0063fc0 msgr2=0x7f43d0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:22.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.840+0000 7f43cffff700 1 -- 172.21.15.121:0/3202345635 shutdown_connections 2023-11-15T03:11:22.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.840+0000 7f43cffff700 1 -- 172.21.15.121:0/3202345635 wait complete. 2023-11-15T03:11:22.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.840+0000 7f43cffff700 1 Processor -- start 2023-11-15T03:11:22.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.840+0000 7f43cffff700 1 -- start start 2023-11-15T03:11:22.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43cffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43d0066330 0x7f43d013bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:22.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43cffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43d00acd00 0x7f43d00c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:22.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43cffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43d00c5c70 0x7f43d00c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:22.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43cffff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d00c4600 con 0x7f43d0066330 2023-11-15T03:11:22.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43cffff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d00adb20 con 0x7f43d00acd00 2023-11-15T03:11:22.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43cffff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d01678f0 con 0x7f43d00c5c70 2023-11-15T03:11:22.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43ceffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43d0066330 0x7f43d013bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:22.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43ceffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43d0066330 0x7f43d013bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44902/0 (socket says 172.21.15.121:44902) 2023-11-15T03:11:22.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43ceffd700 1 -- 172.21.15.121:0/3435044169 learned_addr learned my addr 172.21.15.121:0/3435044169 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:22.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43ce7fc700 1 --2- 172.21.15.121:0/3435044169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43d00acd00 0x7f43d00c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:22.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43cf7fe700 1 --2- 172.21.15.121:0/3435044169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43d00c5c70 0x7f43d00c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:22.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43ce7fc700 1 -- 172.21.15.121:0/3435044169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43d00c5c70 msgr2=0x7f43d00c36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:22.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43ce7fc700 1 --2- 172.21.15.121:0/3435044169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43d00c5c70 0x7f43d00c36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43ce7fc700 1 -- 172.21.15.121:0/3435044169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43d0066330 msgr2=0x7f43d013bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:22.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43ce7fc700 1 --2- 172.21.15.121:0/3435044169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43d0066330 0x7f43d013bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:22.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43ce7fc700 1 -- 172.21.15.121:0/3435044169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c40092a0 con 0x7f43d00acd00 2023-11-15T03:11:22.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43ceffd700 1 --2- 172.21.15.121:0/3435044169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43d0066330 0x7f43d013bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:22.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43ce7fc700 1 --2- 172.21.15.121:0/3435044169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43d00acd00 0x7f43d00c31c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f43c4013000 tx=0x7f43c4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:22.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43b6ffd700 1 -- 172.21.15.121:0/3435044169 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c400dd30 con 0x7f43d00acd00 2023-11-15T03:11:22.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43b6ffd700 1 -- 172.21.15.121:0/3435044169 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c400de90 con 0x7f43d00acd00 2023-11-15T03:11:22.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43cffff700 1 -- 172.21.15.121:0/3435044169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d00c3d60 con 0x7f43d00acd00 2023-11-15T03:11:22.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.844+0000 7f43b6ffd700 1 -- 172.21.15.121:0/3435044169 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c400d980 con 0x7f43d00acd00 2023-11-15T03:11:22.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.848+0000 7f43cffff700 1 -- 172.21.15.121:0/3435044169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d00c4320 con 0x7f43d00acd00 2023-11-15T03:11:22.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.848+0000 7f43cffff700 1 -- 172.21.15.121:0/3435044169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d006bcf0 con 0x7f43d00acd00 2023-11-15T03:11:22.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.848+0000 7f43b6ffd700 1 -- 172.21.15.121:0/3435044169 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f43c401d640 con 0x7f43d00acd00 2023-11-15T03:11:22.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.852+0000 7f43b6ffd700 1 --2- 172.21.15.121:0/3435044169 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f43bc05a540 0x7f43bc05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:22.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.852+0000 7f43b6ffd700 1 -- 172.21.15.121:0/3435044169 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f43c4075e50 con 0x7f43d00acd00 2023-11-15T03:11:22.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.852+0000 7f43ceffd700 1 --2- 172.21.15.121:0/3435044169 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f43bc05a540 0x7f43bc05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:22.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.852+0000 7f43ceffd700 1 --2- 172.21.15.121:0/3435044169 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f43bc05a540 0x7f43bc05ca00 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f43c0006920 tx=0x7f43c0006f40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:22.867 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:22.860+0000 7f43b6ffd700 1 -- 172.21.15.121:0/3435044169 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f43c4049630 con 0x7f43d00acd00 2023-11-15T03:11:23.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:23.060+0000 7f43cffff700 1 -- 172.21.15.121:0/3435044169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f43d00c61c0 con 0x7f43d00acd00 2023-11-15T03:11:23.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:23.892+0000 7f43b6ffd700 1 -- 172.21.15.121:0/3435044169 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v349) v1 ==== 118+0+0 (secure 0 0 0) 0x7f43c404cc50 con 0x7f43d00acd00 2023-11-15T03:11:23.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:23.896+0000 7f43cffff700 1 -- 172.21.15.121:0/3435044169 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f43bc05a540 msgr2=0x7f43bc05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:23.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:23.896+0000 7f43cffff700 1 --2- 172.21.15.121:0/3435044169 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f43bc05a540 0x7f43bc05ca00 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f43c0006920 tx=0x7f43c0006f40).stop 2023-11-15T03:11:23.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:23.896+0000 7f43cffff700 1 -- 172.21.15.121:0/3435044169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43d00acd00 msgr2=0x7f43d00c31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:23.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:23.896+0000 7f43cffff700 1 --2- 172.21.15.121:0/3435044169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43d00acd00 0x7f43d00c31c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f43c4013000 tx=0x7f43c4016000).stop 2023-11-15T03:11:23.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:23.896+0000 7f43cffff700 1 -- 172.21.15.121:0/3435044169 shutdown_connections 2023-11-15T03:11:23.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:23.896+0000 7f43cffff700 1 --2- 172.21.15.121:0/3435044169 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f43bc05a540 0x7f43bc05ca00 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:23.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:23.896+0000 7f43cffff700 1 --2- 172.21.15.121:0/3435044169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43d00c5c70 0x7f43d00c36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:23.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:23.896+0000 7f43cffff700 1 --2- 172.21.15.121:0/3435044169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43d00acd00 0x7f43d00c31c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:23.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:23.896+0000 7f43cffff700 1 --2- 172.21.15.121:0/3435044169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43d0066330 0x7f43d013bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:23.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:23.896+0000 7f43cffff700 1 -- 172.21.15.121:0/3435044169 >> 172.21.15.121:0/3435044169 conn(0x7f43d0063fc0 msgr2=0x7f43d0152a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:23.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:23.896+0000 7f43cffff700 1 -- 172.21.15.121:0/3435044169 shutdown_connections 2023-11-15T03:11:23.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:23.896+0000 7f43cffff700 1 -- 172.21.15.121:0/3435044169 wait complete. 2023-11-15T03:11:23.926 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-15T03:11:23.927 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-15T03:11:24.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.080+0000 7fc54f107700 1 Processor -- start 2023-11-15T03:11:24.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.080+0000 7fc54f107700 1 -- start start 2023-11-15T03:11:24.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54f107700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc5480caf70 0x7fc54815e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:24.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54f107700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc548065c90 con 0x7fc548153ac0 2023-11-15T03:11:24.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54f107700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5480c7c50 con 0x7fc548067130 2023-11-15T03:11:24.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54f107700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5480c8210 con 0x7fc5480caf70 2023-11-15T03:11:24.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54e105700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc548153ac0 0x7fc5480c7190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:35890/0 (socket says 172.21.15.121:35890) 2023-11-15T03:11:24.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54e105700 1 -- 172.21.15.121:0/840547492 learned_addr learned my addr 172.21.15.121:0/840547492 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:24.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54e906700 1 --2- 172.21.15.121:0/840547492 >> v2:172.21.15.190:3300/0 conn(0x7fc5480caf70 0x7fc54815e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:24.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54d103700 1 -- 172.21.15.121:0/840547492 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2335613058 0 0) 0x7fc548065c90 con 0x7fc548153ac0 2023-11-15T03:11:24.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54d103700 1 -- 172.21.15.121:0/840547492 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5340033f0 con 0x7fc548153ac0 2023-11-15T03:11:24.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54d103700 1 -- 172.21.15.121:0/840547492 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051439869 0 0) 0x7fc5480c7c50 con 0x7fc548067130 2023-11-15T03:11:24.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54d103700 1 -- 172.21.15.121:0/840547492 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc548065c90 con 0x7fc548067130 2023-11-15T03:11:24.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54d103700 1 -- 172.21.15.121:0/840547492 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1085058256 0 0) 0x7fc5340033f0 con 0x7fc548153ac0 2023-11-15T03:11:24.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54d103700 1 -- 172.21.15.121:0/840547492 >> v2:172.21.15.190:3300/0 conn(0x7fc5480caf70 msgr2=0x7fc54815e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:24.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54d103700 1 --2- 172.21.15.121:0/840547492 >> v2:172.21.15.190:3300/0 conn(0x7fc5480caf70 0x7fc54815e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:24.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54d103700 1 -- 172.21.15.121:0/840547492 >> v1:172.21.15.190:6789/0 conn(0x7fc548067130 legacy=0x7fc5480ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:24.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54d103700 1 -- 172.21.15.121:0/840547492 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5480c7f30 con 0x7fc548153ac0 2023-11-15T03:11:24.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54d103700 1 -- 172.21.15.121:0/840547492 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fc544001470 con 0x7fc548153ac0 2023-11-15T03:11:24.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54d103700 1 -- 172.21.15.121:0/840547492 >> v1:172.21.15.121:6789/0 conn(0x7fc548153ac0 legacy=0x7fc5480c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:24.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54d103700 1 --2- 172.21.15.121:0/840547492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc534003a70 0x7fc534003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:24.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54d103700 1 --2- 172.21.15.121:0/840547492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc534004690 0x7fc534004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:24.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54d103700 1 --2- 172.21.15.121:0/840547492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc534005050 0x7fc53400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:24.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54d103700 1 -- 172.21.15.121:0/840547492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5480c8210 con 0x7fc534005050 2023-11-15T03:11:24.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54d103700 1 -- 172.21.15.121:0/840547492 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53400bfc0 con 0x7fc534003a70 2023-11-15T03:11:24.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54d103700 1 -- 172.21.15.121:0/840547492 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53400c180 con 0x7fc534004690 2023-11-15T03:11:24.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.084+0000 7fc54e906700 1 --2- 172.21.15.121:0/840547492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc534005050 0x7fc53400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:24.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.088+0000 7fc54e105700 1 --2- 172.21.15.121:0/840547492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc534003a70 0x7fc534003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:24.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.088+0000 7fc54d904700 1 --2- 172.21.15.121:0/840547492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc534004690 0x7fc534004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:24.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.088+0000 7fc54e906700 1 -- 172.21.15.121:0/840547492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc534004690 msgr2=0x7fc534004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:24.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.088+0000 7fc54e906700 1 --2- 172.21.15.121:0/840547492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc534004690 0x7fc534004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:24.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.088+0000 7fc54e906700 1 -- 172.21.15.121:0/840547492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc534003a70 msgr2=0x7fc534003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:24.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.088+0000 7fc54e906700 1 --2- 172.21.15.121:0/840547492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc534003a70 0x7fc534003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:24.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.088+0000 7fc54e906700 1 -- 172.21.15.121:0/840547492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5340033f0 con 0x7fc534005050 2023-11-15T03:11:24.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.088+0000 7fc54e906700 1 -- 172.21.15.121:0/840547492 reap_dead start 2023-11-15T03:11:24.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.088+0000 7fc54e906700 1 --2- 172.21.15.121:0/840547492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc534005050 0x7fc53400b8e0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 rx=0x7fc5480cb370 tx=0x7fc54000b040).ready entity=mon.0 client_cookie=8105ebd3536638e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:24.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.088+0000 7fc54d103700 1 -- 172.21.15.121:0/840547492 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540009ac0 con 0x7fc534005050 2023-11-15T03:11:24.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.088+0000 7fc54d103700 1 -- 172.21.15.121:0/840547492 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54000f040 con 0x7fc534005050 2023-11-15T03:11:24.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.088+0000 7fc54d103700 1 -- 172.21.15.121:0/840547492 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54000ece0 con 0x7fc534005050 2023-11-15T03:11:24.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.088+0000 7fc54f107700 1 -- 172.21.15.121:0/840547492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc534005050 msgr2=0x7fc53400b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:24.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.088+0000 7fc54f107700 1 --2- 172.21.15.121:0/840547492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc534005050 0x7fc53400b8e0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 rx=0x7fc5480cb370 tx=0x7fc54000b040).stop 2023-11-15T03:11:24.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.088+0000 7fc54f107700 1 -- 172.21.15.121:0/840547492 shutdown_connections 2023-11-15T03:11:24.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.088+0000 7fc54f107700 1 --2- 172.21.15.121:0/840547492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc534005050 0x7fc53400b8e0 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:24.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.088+0000 7fc54f107700 1 -- 172.21.15.121:0/840547492 >> 172.21.15.121:0/840547492 conn(0x7fc548067a60 msgr2=0x7fc548067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:24.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.088+0000 7fc54f107700 1 -- 172.21.15.121:0/840547492 shutdown_connections 2023-11-15T03:11:24.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.088+0000 7fc54f107700 1 -- 172.21.15.121:0/840547492 wait complete. 2023-11-15T03:11:24.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc54f107700 1 Processor -- start 2023-11-15T03:11:24.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc54f107700 1 -- start start 2023-11-15T03:11:24.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc54f107700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc534005050 0x7fc5481553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:24.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc54f107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc54814d240 0x7fc5480aa190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:24.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc54f107700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5480aea00 0x7fc5480aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:24.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc54f107700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54814e260 con 0x7fc5480aea00 2023-11-15T03:11:24.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc54f107700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5481565b0 con 0x7fc534005050 2023-11-15T03:11:24.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc54f107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54814d6c0 con 0x7fc54814d240 2023-11-15T03:11:24.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc54e906700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5480aea00 0x7fc5480aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:24.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc54e906700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5480aea00 0x7fc5480aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44920/0 (socket says 172.21.15.121:44920) 2023-11-15T03:11:24.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc54e906700 1 -- 172.21.15.121:0/1529951568 learned_addr learned my addr 172.21.15.121:0/1529951568 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:24.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc54e105700 1 --2- 172.21.15.121:0/1529951568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc534005050 0x7fc5481553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:24.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc54d904700 1 --2- 172.21.15.121:0/1529951568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc54814d240 0x7fc5480aa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:24.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc54e906700 1 -- 172.21.15.121:0/1529951568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc54814d240 msgr2=0x7fc5480aa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:24.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc54e906700 1 --2- 172.21.15.121:0/1529951568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc54814d240 0x7fc5480aa190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:24.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc54e906700 1 -- 172.21.15.121:0/1529951568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc534005050 msgr2=0x7fc5481553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:24.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc54e906700 1 --2- 172.21.15.121:0/1529951568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc534005050 0x7fc5481553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:24.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc54e906700 1 -- 172.21.15.121:0/1529951568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5480af370 con 0x7fc5480aea00 2023-11-15T03:11:24.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc54e906700 1 --2- 172.21.15.121:0/1529951568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5480aea00 0x7fc5480aee80 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 rx=0x7fc5480c4f50 tx=0x7fc540018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:24.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.092+0000 7fc53e7fc700 1 -- 172.21.15.121:0/1529951568 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54001d460 con 0x7fc5480aea00 2023-11-15T03:11:24.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.096+0000 7fc53e7fc700 1 -- 172.21.15.121:0/1529951568 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54001d5e0 con 0x7fc5480aea00 2023-11-15T03:11:24.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.096+0000 7fc53e7fc700 1 -- 172.21.15.121:0/1529951568 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54001d9a0 con 0x7fc5480aea00 2023-11-15T03:11:24.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.096+0000 7fc54f107700 1 -- 172.21.15.121:0/1529951568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5480aa680 con 0x7fc5480aea00 2023-11-15T03:11:24.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.096+0000 7fc54f107700 1 -- 172.21.15.121:0/1529951568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc548155c80 con 0x7fc5480aea00 2023-11-15T03:11:24.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.096+0000 7fc54f107700 1 -- 172.21.15.121:0/1529951568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc548156760 con 0x7fc5480aea00 2023-11-15T03:11:24.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.096+0000 7fc53e7fc700 1 -- 172.21.15.121:0/1529951568 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc540003020 con 0x7fc5480aea00 2023-11-15T03:11:24.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.100+0000 7fc53e7fc700 1 --2- 172.21.15.121:0/1529951568 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc52c05a510 0x7fc52c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:24.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.100+0000 7fc53e7fc700 1 -- 172.21.15.121:0/1529951568 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc540076880 con 0x7fc5480aea00 2023-11-15T03:11:24.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.100+0000 7fc54e105700 1 --2- 172.21.15.121:0/1529951568 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc52c05a510 0x7fc52c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:24.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.100+0000 7fc54e105700 1 --2- 172.21.15.121:0/1529951568 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc52c05a510 0x7fc52c05c9d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7fc5440015a0 tx=0x7fc544009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:24.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.104+0000 7fc53e7fc700 1 -- 172.21.15.121:0/1529951568 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc540049fa0 con 0x7fc5480aea00 2023-11-15T03:11:24.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.296+0000 7fc54f107700 1 -- 172.21.15.121:0/1529951568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc54806bcf0 con 0x7fc5480aea00 2023-11-15T03:11:24.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.900+0000 7fc53e7fc700 1 -- 172.21.15.121:0/1529951568 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v350) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc540017070 con 0x7fc5480aea00 2023-11-15T03:11:24.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.904+0000 7fc54f107700 1 -- 172.21.15.121:0/1529951568 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc52c05a510 msgr2=0x7fc52c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:24.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.904+0000 7fc54f107700 1 --2- 172.21.15.121:0/1529951568 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc52c05a510 0x7fc52c05c9d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7fc5440015a0 tx=0x7fc544009040).stop 2023-11-15T03:11:24.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.904+0000 7fc54f107700 1 -- 172.21.15.121:0/1529951568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5480aea00 msgr2=0x7fc5480aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:24.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.904+0000 7fc54f107700 1 --2- 172.21.15.121:0/1529951568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5480aea00 0x7fc5480aee80 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 rx=0x7fc5480c4f50 tx=0x7fc540018000).stop 2023-11-15T03:11:24.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.904+0000 7fc54f107700 1 -- 172.21.15.121:0/1529951568 shutdown_connections 2023-11-15T03:11:24.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.904+0000 7fc54f107700 1 --2- 172.21.15.121:0/1529951568 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc52c05a510 0x7fc52c05c9d0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:24.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.904+0000 7fc54f107700 1 --2- 172.21.15.121:0/1529951568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5480aea00 0x7fc5480aee80 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:24.915 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.904+0000 7fc54f107700 1 --2- 172.21.15.121:0/1529951568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc54814d240 0x7fc5480aa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:24.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.904+0000 7fc54f107700 1 --2- 172.21.15.121:0/1529951568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc534005050 0x7fc5481553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:24.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.904+0000 7fc54f107700 1 -- 172.21.15.121:0/1529951568 >> 172.21.15.121:0/1529951568 conn(0x7fc548067a60 msgr2=0x7fc5480c58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:24.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.904+0000 7fc54f107700 1 -- 172.21.15.121:0/1529951568 shutdown_connections 2023-11-15T03:11:24.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:24.904+0000 7fc54f107700 1 -- 172.21.15.121:0/1529951568 wait complete. 2023-11-15T03:11:24.935 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-15T03:11:24.936 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:25.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.100+0000 7fc8c9f1b700 1 Processor -- start 2023-11-15T03:11:25.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.100+0000 7fc8c9f1b700 1 -- start start 2023-11-15T03:11:25.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.104+0000 7fc8c9f1b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc8c40aa240 0x7fc8c4067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:25.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.104+0000 7fc8c9f1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4067f50 0x7fc8c4065d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:25.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.104+0000 7fc8c9f1b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8c40cb5c0 con 0x7fc8c4153bb0 2023-11-15T03:11:25.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.104+0000 7fc8c9f1b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c40aa6f0 con 0x7fc8c40aa240 2023-11-15T03:11:25.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.104+0000 7fc8c9f1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c40c5900 con 0x7fc8c4067f50 2023-11-15T03:11:25.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.104+0000 7fc8c37fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fc8c4153bb0 0x7fc8c40a9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:49916/0 (socket says 172.21.15.121:49916) 2023-11-15T03:11:25.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.104+0000 7fc8c37fe700 1 -- 172.21.15.121:0/597575596 learned_addr learned my addr 172.21.15.121:0/597575596 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:25.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.104+0000 7fc8c2ffd700 1 --2- 172.21.15.121:0/597575596 >> v2:172.21.15.190:3300/0 conn(0x7fc8c40aa240 0x7fc8c4067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:25.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.104+0000 7fc8c3fff700 1 --2- 172.21.15.121:0/597575596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4067f50 0x7fc8c4065d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:25.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.108+0000 7fc8c27fc700 1 -- 172.21.15.121:0/597575596 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3036805043 0 0) 0x7fc8c40cb5c0 con 0x7fc8c4153bb0 2023-11-15T03:11:25.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.108+0000 7fc8c27fc700 1 -- 172.21.15.121:0/597575596 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8ac0033f0 con 0x7fc8c4153bb0 2023-11-15T03:11:25.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.108+0000 7fc8c3fff700 1 -- 172.21.15.121:0/597575596 >> v2:172.21.15.190:3300/0 conn(0x7fc8c40aa240 msgr2=0x7fc8c4067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:25.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.108+0000 7fc8c3fff700 1 --2- 172.21.15.121:0/597575596 >> v2:172.21.15.190:3300/0 conn(0x7fc8c40aa240 0x7fc8c4067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.108+0000 7fc8c3fff700 1 -- 172.21.15.121:0/597575596 >> v1:172.21.15.190:6789/0 conn(0x7fc8c4153bb0 legacy=0x7fc8c40a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:25.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.108+0000 7fc8c3fff700 1 -- 172.21.15.121:0/597575596 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c40c9ca0 con 0x7fc8c4067f50 2023-11-15T03:11:25.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.108+0000 7fc8c3fff700 1 --2- 172.21.15.121:0/597575596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4067f50 0x7fc8c4065d30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fc8c40c5a40 tx=0x7fc8bc009780).ready entity=mon.2 client_cookie=2e7576df89c96e0f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:25.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.108+0000 7fc8c27fc700 1 -- 172.21.15.121:0/597575596 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc00e070 con 0x7fc8c4067f50 2023-11-15T03:11:25.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.108+0000 7fc8c27fc700 1 -- 172.21.15.121:0/597575596 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8bc00bab0 con 0x7fc8c4067f50 2023-11-15T03:11:25.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.108+0000 7fc8c27fc700 1 -- 172.21.15.121:0/597575596 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc00be30 con 0x7fc8c4067f50 2023-11-15T03:11:25.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.108+0000 7fc8c9f1b700 1 -- 172.21.15.121:0/597575596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4067f50 msgr2=0x7fc8c4065d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:25.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.108+0000 7fc8c9f1b700 1 --2- 172.21.15.121:0/597575596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4067f50 0x7fc8c4065d30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fc8c40c5a40 tx=0x7fc8bc009780).stop 2023-11-15T03:11:25.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.108+0000 7fc8c9f1b700 1 -- 172.21.15.121:0/597575596 shutdown_connections 2023-11-15T03:11:25.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.108+0000 7fc8c9f1b700 1 --2- 172.21.15.121:0/597575596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4067f50 0x7fc8c4065d30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.108+0000 7fc8c9f1b700 1 --2- 172.21.15.121:0/597575596 >> v2:172.21.15.190:3300/0 conn(0x7fc8c40aa240 0x7fc8c4067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.108+0000 7fc8c9f1b700 1 -- 172.21.15.121:0/597575596 >> 172.21.15.121:0/597575596 conn(0x7fc8c4058c50 msgr2=0x7fc8c4059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:25.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.108+0000 7fc8c9f1b700 1 -- 172.21.15.121:0/597575596 shutdown_connections 2023-11-15T03:11:25.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.108+0000 7fc8c9f1b700 1 -- 172.21.15.121:0/597575596 wait complete. 2023-11-15T03:11:25.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.112+0000 7fc8c9f1b700 1 Processor -- start 2023-11-15T03:11:25.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.112+0000 7fc8c9f1b700 1 -- start start 2023-11-15T03:11:25.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.112+0000 7fc8c9f1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4067f50 0x7fc8c4140320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:25.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.112+0000 7fc8c9f1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c40aa240 0x7fc8c40abc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:25.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.112+0000 7fc8c9f1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8c4153bb0 0x7fc8c40ac140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:25.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.112+0000 7fc8c9f1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c4140810 con 0x7fc8c40aa240 2023-11-15T03:11:25.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.112+0000 7fc8c9f1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c4165440 con 0x7fc8c4153bb0 2023-11-15T03:11:25.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.112+0000 7fc8c9f1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c4165720 con 0x7fc8c4067f50 2023-11-15T03:11:25.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.112+0000 7fc8c2ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c40aa240 0x7fc8c40abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:25.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.112+0000 7fc8c3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8c4153bb0 0x7fc8c40ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:25.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.112+0000 7fc8c2ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c40aa240 0x7fc8c40abc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40460/0 (socket says 172.21.15.121:40460) 2023-11-15T03:11:25.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.112+0000 7fc8c2ffd700 1 -- 172.21.15.121:0/992635137 learned_addr learned my addr 172.21.15.121:0/992635137 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:25.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.112+0000 7fc8c37fe700 1 --2- 172.21.15.121:0/992635137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4067f50 0x7fc8c4140320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:25.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.112+0000 7fc8c2ffd700 1 -- 172.21.15.121:0/992635137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4067f50 msgr2=0x7fc8c4140320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:25.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.112+0000 7fc8c2ffd700 1 --2- 172.21.15.121:0/992635137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4067f50 0x7fc8c4140320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.112+0000 7fc8c2ffd700 1 -- 172.21.15.121:0/992635137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8c4153bb0 msgr2=0x7fc8c40ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:25.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.112+0000 7fc8c2ffd700 1 --2- 172.21.15.121:0/992635137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8c4153bb0 0x7fc8c40ac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.112+0000 7fc8c2ffd700 1 -- 172.21.15.121:0/992635137 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8bc009a20 con 0x7fc8c40aa240 2023-11-15T03:11:25.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.112+0000 7fc8c3fff700 1 --2- 172.21.15.121:0/992635137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8c4153bb0 0x7fc8c40ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:25.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.116+0000 7fc8c2ffd700 1 --2- 172.21.15.121:0/992635137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c40aa240 0x7fc8c40abc50 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 rx=0x7fc8b800e900 tx=0x7fc8b8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:25.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.116+0000 7fc8ab7fe700 1 -- 172.21.15.121:0/992635137 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8012600 con 0x7fc8c40aa240 2023-11-15T03:11:25.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.116+0000 7fc8ab7fe700 1 -- 172.21.15.121:0/992635137 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b8012760 con 0x7fc8c40aa240 2023-11-15T03:11:25.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.116+0000 7fc8c9f1b700 1 -- 172.21.15.121:0/992635137 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c4165a00 con 0x7fc8c40aa240 2023-11-15T03:11:25.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.116+0000 7fc8ab7fe700 1 -- 172.21.15.121:0/992635137 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8012ae0 con 0x7fc8c40aa240 2023-11-15T03:11:25.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.116+0000 7fc8c9f1b700 1 -- 172.21.15.121:0/992635137 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c4165fc0 con 0x7fc8c40aa240 2023-11-15T03:11:25.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.116+0000 7fc8c9f1b700 1 -- 172.21.15.121:0/992635137 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c406c150 con 0x7fc8c40aa240 2023-11-15T03:11:25.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.120+0000 7fc8ab7fe700 1 -- 172.21.15.121:0/992635137 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc8b801b400 con 0x7fc8c40aa240 2023-11-15T03:11:25.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.120+0000 7fc8ab7fe700 1 --2- 172.21.15.121:0/992635137 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc8b005a510 0x7fc8b005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:25.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.120+0000 7fc8ab7fe700 1 -- 172.21.15.121:0/992635137 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc8b8074ca0 con 0x7fc8c40aa240 2023-11-15T03:11:25.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.120+0000 7fc8c37fe700 1 --2- 172.21.15.121:0/992635137 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc8b005a510 0x7fc8b005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:25.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.120+0000 7fc8c37fe700 1 --2- 172.21.15.121:0/992635137 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc8b005a510 0x7fc8b005c9d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7fc8b4007fd0 tx=0x7fc8b4009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:25.136 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.128+0000 7fc8ab7fe700 1 -- 172.21.15.121:0/992635137 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc8b8048360 con 0x7fc8c40aa240 2023-11-15T03:11:25.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.324+0000 7fc8c9f1b700 1 -- 172.21.15.121:0/992635137 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8c4068ae0 con 0x7fc8c40aa240 2023-11-15T03:11:25.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.324+0000 7fc8ab7fe700 1 -- 172.21.15.121:0/992635137 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) v1 ==== 95+0+4512 (secure 0 0 0) 0x7fc8b8016030 con 0x7fc8c40aa240 2023-11-15T03:11:25.335 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:25.335 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":350,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/755048854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":755048854},{"type":"v1","addr":"172.21.15.190:6834","nonce":755048854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2360242049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2360242049},{"type":"v1","addr":"172.21.15.190:6835","nonce":2360242049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2780153683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2780153683},{"type":"v1","addr":"172.21.15.121:6835","nonce":2780153683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345}],"filesystems":[{"mdsmap":{"epoch":350,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:11:21.864101+0000","modified":"2023-11-15T03:11:24.889953+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10716},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10716":{"gid":10716,"name":"a","rank":0,"incarnation":346,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6837/2335356851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2335356851},{"type":"v1","addr":"172.21.15.121:6837","nonce":2335356851}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-15T03:11:25.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.328+0000 7fc8c9f1b700 1 -- 172.21.15.121:0/992635137 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc8b005a510 msgr2=0x7fc8b005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:25.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.328+0000 7fc8c9f1b700 1 --2- 172.21.15.121:0/992635137 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc8b005a510 0x7fc8b005c9d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7fc8b4007fd0 tx=0x7fc8b4009040).stop 2023-11-15T03:11:25.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.328+0000 7fc8c9f1b700 1 -- 172.21.15.121:0/992635137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c40aa240 msgr2=0x7fc8c40abc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:25.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.328+0000 7fc8c9f1b700 1 --2- 172.21.15.121:0/992635137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c40aa240 0x7fc8c40abc50 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 rx=0x7fc8b800e900 tx=0x7fc8b8009060).stop 2023-11-15T03:11:25.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.328+0000 7fc8c9f1b700 1 -- 172.21.15.121:0/992635137 shutdown_connections 2023-11-15T03:11:25.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.328+0000 7fc8c9f1b700 1 --2- 172.21.15.121:0/992635137 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc8b005a510 0x7fc8b005c9d0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.328+0000 7fc8c9f1b700 1 --2- 172.21.15.121:0/992635137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8c4153bb0 0x7fc8c40ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.328+0000 7fc8c9f1b700 1 --2- 172.21.15.121:0/992635137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c40aa240 0x7fc8c40abc50 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.328+0000 7fc8c9f1b700 1 --2- 172.21.15.121:0/992635137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8c4067f50 0x7fc8c4140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.328+0000 7fc8c9f1b700 1 -- 172.21.15.121:0/992635137 >> 172.21.15.121:0/992635137 conn(0x7fc8c4058c50 msgr2=0x7fc8c4159140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:25.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.328+0000 7fc8c9f1b700 1 -- 172.21.15.121:0/992635137 shutdown_connections 2023-11-15T03:11:25.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.328+0000 7fc8c9f1b700 1 -- 172.21.15.121:0/992635137 wait complete. 2023-11-15T03:11:25.341 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 350 2023-11-15T03:11:25.359 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:11:25.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.508+0000 7f1d923f9700 1 Processor -- start 2023-11-15T03:11:25.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.508+0000 7f1d923f9700 1 -- start start 2023-11-15T03:11:25.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.508+0000 7f1d923f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d8c0a9830 0x7f1d8c0a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:25.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.508+0000 7f1d923f9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1d8c0c30e0 0x7f1d8c0c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:25.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.508+0000 7f1d923f9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d8c0657f0 con 0x7f1d8c151870 2023-11-15T03:11:25.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.508+0000 7f1d923f9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d8c065ad0 con 0x7f1d8c0c30e0 2023-11-15T03:11:25.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.508+0000 7f1d923f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c0c3a60 con 0x7f1d8c0a9830 2023-11-15T03:11:25.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.508+0000 7f1d913f7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f1d8c151870 0x7f1d8c151c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60844/0 (socket says 172.21.15.121:60844) 2023-11-15T03:11:25.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.508+0000 7f1d913f7700 1 -- 172.21.15.121:0/3388122766 learned_addr learned my addr 172.21.15.121:0/3388122766 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:25.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.508+0000 7f1d91bf8700 1 --2- 172.21.15.121:0/3388122766 >> v2:172.21.15.190:3300/0 conn(0x7f1d8c0c30e0 0x7f1d8c0c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:25.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.512+0000 7f1d90bf6700 1 --2- 172.21.15.121:0/3388122766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d8c0a9830 0x7f1d8c0a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:25.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.512+0000 7f1d83fff700 1 -- 172.21.15.121:0/3388122766 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892237482 0 0) 0x7f1d8c0657f0 con 0x7f1d8c151870 2023-11-15T03:11:25.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.512+0000 7f1d83fff700 1 -- 172.21.15.121:0/3388122766 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d780033f0 con 0x7f1d8c151870 2023-11-15T03:11:25.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.512+0000 7f1d91bf8700 1 -- 172.21.15.121:0/3388122766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d8c0a9830 msgr2=0x7f1d8c0a9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:25.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.512+0000 7f1d91bf8700 1 --2- 172.21.15.121:0/3388122766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d8c0a9830 0x7f1d8c0a9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.512+0000 7f1d91bf8700 1 -- 172.21.15.121:0/3388122766 >> v1:172.21.15.121:6789/0 conn(0x7f1d8c151870 legacy=0x7f1d8c151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:25.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.512+0000 7f1d91bf8700 1 -- 172.21.15.121:0/3388122766 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c0aa260 con 0x7f1d8c0c30e0 2023-11-15T03:11:25.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.512+0000 7f1d91bf8700 1 --2- 172.21.15.121:0/3388122766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d8c0c30e0 0x7f1d8c0c3570 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f1d84009e60 tx=0x7f1d84009780).ready entity=mon.1 client_cookie=9cfba272a4e03ad0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:25.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.512+0000 7f1d83fff700 1 -- 172.21.15.121:0/3388122766 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8400e070 con 0x7f1d8c0c30e0 2023-11-15T03:11:25.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.512+0000 7f1d83fff700 1 -- 172.21.15.121:0/3388122766 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8400bab0 con 0x7f1d8c0c30e0 2023-11-15T03:11:25.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.512+0000 7f1d83fff700 1 -- 172.21.15.121:0/3388122766 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8400be30 con 0x7f1d8c0c30e0 2023-11-15T03:11:25.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.512+0000 7f1d923f9700 1 -- 172.21.15.121:0/3388122766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d8c0c30e0 msgr2=0x7f1d8c0c3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:25.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.512+0000 7f1d923f9700 1 --2- 172.21.15.121:0/3388122766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d8c0c30e0 0x7f1d8c0c3570 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f1d84009e60 tx=0x7f1d84009780).stop 2023-11-15T03:11:25.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.512+0000 7f1d923f9700 1 -- 172.21.15.121:0/3388122766 shutdown_connections 2023-11-15T03:11:25.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.512+0000 7f1d923f9700 1 --2- 172.21.15.121:0/3388122766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d8c0c30e0 0x7f1d8c0c3570 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.512+0000 7f1d923f9700 1 --2- 172.21.15.121:0/3388122766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d8c0a9830 0x7f1d8c0a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.512+0000 7f1d923f9700 1 -- 172.21.15.121:0/3388122766 >> 172.21.15.121:0/3388122766 conn(0x7f1d8c062190 msgr2=0x7f1d8c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:25.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.512+0000 7f1d923f9700 1 -- 172.21.15.121:0/3388122766 shutdown_connections 2023-11-15T03:11:25.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.512+0000 7f1d923f9700 1 -- 172.21.15.121:0/3388122766 wait complete. 2023-11-15T03:11:25.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.516+0000 7f1d923f9700 1 Processor -- start 2023-11-15T03:11:25.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.516+0000 7f1d923f9700 1 -- start start 2023-11-15T03:11:25.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d923f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d8c0a9830 0x7f1d8c14a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:25.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d923f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d8c0c30e0 0x7f1d8c167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:25.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d923f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d8c151870 0x7f1d8c0bfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:25.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d923f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c14b310 con 0x7f1d8c0a9830 2023-11-15T03:11:25.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d923f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c14a700 con 0x7f1d8c151870 2023-11-15T03:11:25.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d923f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c14a9e0 con 0x7f1d8c0c30e0 2023-11-15T03:11:25.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d913f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d8c0a9830 0x7f1d8c14a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:25.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d913f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d8c0a9830 0x7f1d8c14a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40466/0 (socket says 172.21.15.121:40466) 2023-11-15T03:11:25.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d913f7700 1 -- 172.21.15.121:0/2566059548 learned_addr learned my addr 172.21.15.121:0/2566059548 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:25.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d90bf6700 1 --2- 172.21.15.121:0/2566059548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d8c0c30e0 0x7f1d8c167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:25.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d91bf8700 1 --2- 172.21.15.121:0/2566059548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d8c151870 0x7f1d8c0bfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:25.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d913f7700 1 -- 172.21.15.121:0/2566059548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d8c0c30e0 msgr2=0x7f1d8c167ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:25.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d913f7700 1 --2- 172.21.15.121:0/2566059548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d8c0c30e0 0x7f1d8c167ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d913f7700 1 -- 172.21.15.121:0/2566059548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d8c151870 msgr2=0x7f1d8c0bfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:25.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d913f7700 1 --2- 172.21.15.121:0/2566059548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d8c151870 0x7f1d8c0bfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d913f7700 1 -- 172.21.15.121:0/2566059548 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d84009a20 con 0x7f1d8c0a9830 2023-11-15T03:11:25.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d913f7700 1 --2- 172.21.15.121:0/2566059548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d8c0a9830 0x7f1d8c14a080 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 rx=0x7f1d8800c420 tx=0x7f1d88007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:25.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d817fa700 1 -- 172.21.15.121:0/2566059548 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d880072e0 con 0x7f1d8c0a9830 2023-11-15T03:11:25.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d817fa700 1 -- 172.21.15.121:0/2566059548 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d88007440 con 0x7f1d8c0a9830 2023-11-15T03:11:25.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d923f9700 1 -- 172.21.15.121:0/2566059548 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d8c14acc0 con 0x7f1d8c0a9830 2023-11-15T03:11:25.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d817fa700 1 -- 172.21.15.121:0/2566059548 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d880057a0 con 0x7f1d8c0a9830 2023-11-15T03:11:25.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.520+0000 7f1d923f9700 1 -- 172.21.15.121:0/2566059548 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8c168390 con 0x7f1d8c0a9830 2023-11-15T03:11:25.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.524+0000 7f1d923f9700 1 -- 172.21.15.121:0/2566059548 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d8c06bcf0 con 0x7f1d8c0a9830 2023-11-15T03:11:25.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.524+0000 7f1d817fa700 1 -- 172.21.15.121:0/2566059548 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1d88005bd0 con 0x7f1d8c0a9830 2023-11-15T03:11:25.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.524+0000 7f1d817fa700 1 --2- 172.21.15.121:0/2566059548 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d7005a510 0x7f1d7005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:25.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.524+0000 7f1d817fa700 1 -- 172.21.15.121:0/2566059548 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1d88073d60 con 0x7f1d8c0a9830 2023-11-15T03:11:25.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.524+0000 7f1d90bf6700 1 --2- 172.21.15.121:0/2566059548 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d7005a510 0x7f1d7005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:25.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.528+0000 7f1d90bf6700 1 --2- 172.21.15.121:0/2566059548 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d7005a510 0x7f1d7005c9d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f1d8c0c2cc0 tx=0x7f1d7c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:25.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.532+0000 7f1d817fa700 1 -- 172.21.15.121:0/2566059548 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1d88047480 con 0x7f1d8c0a9830 2023-11-15T03:11:25.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.696+0000 7f1d923f9700 1 -- 172.21.15.121:0/2566059548 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d8c068a70 con 0x7f1d8c0a9830 2023-11-15T03:11:25.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.700+0000 7f1d817fa700 1 -- 172.21.15.121:0/2566059548 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+16990 (secure 0 0 0) 0x7f1d88018070 con 0x7f1d8c0a9830 2023-11-15T03:11:25.709 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:25.709 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":145,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:11:22.870444+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-15T03:11:19.965438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-15T03:11:20.878094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:11:25.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.700+0000 7f1d923f9700 1 -- 172.21.15.121:0/2566059548 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d7005a510 msgr2=0x7f1d7005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:25.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.700+0000 7f1d923f9700 1 --2- 172.21.15.121:0/2566059548 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d7005a510 0x7f1d7005c9d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f1d8c0c2cc0 tx=0x7f1d7c008040).stop 2023-11-15T03:11:25.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.700+0000 7f1d923f9700 1 -- 172.21.15.121:0/2566059548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d8c0a9830 msgr2=0x7f1d8c14a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:25.711 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.700+0000 7f1d923f9700 1 --2- 172.21.15.121:0/2566059548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d8c0a9830 0x7f1d8c14a080 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 rx=0x7f1d8800c420 tx=0x7f1d88007c80).stop 2023-11-15T03:11:25.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.704+0000 7f1d923f9700 1 -- 172.21.15.121:0/2566059548 shutdown_connections 2023-11-15T03:11:25.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.704+0000 7f1d923f9700 1 --2- 172.21.15.121:0/2566059548 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d7005a510 0x7f1d7005c9d0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.704+0000 7f1d923f9700 1 --2- 172.21.15.121:0/2566059548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d8c151870 0x7f1d8c0bfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.704+0000 7f1d923f9700 1 --2- 172.21.15.121:0/2566059548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d8c0c30e0 0x7f1d8c167ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.704+0000 7f1d923f9700 1 --2- 172.21.15.121:0/2566059548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d8c0a9830 0x7f1d8c14a080 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.712 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.704+0000 7f1d923f9700 1 -- 172.21.15.121:0/2566059548 >> 172.21.15.121:0/2566059548 conn(0x7f1d8c062190 msgr2=0x7f1d8c0c51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:25.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.704+0000 7f1d923f9700 1 -- 172.21.15.121:0/2566059548 shutdown_connections 2023-11-15T03:11:25.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.704+0000 7f1d923f9700 1 -- 172.21.15.121:0/2566059548 wait complete. 2023-11-15T03:11:25.729 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-15T03:11:25.729 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T03:11:25.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.880+0000 7f0ddd969700 1 Processor -- start 2023-11-15T03:11:25.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.880+0000 7f0ddd969700 1 -- start start 2023-11-15T03:11:25.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.880+0000 7f0ddd969700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0dd8066ee0 0x7f0dd8067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:25.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.880+0000 7f0ddd969700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd8068c10 con 0x7f0dd8153890 2023-11-15T03:11:25.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.880+0000 7f0ddd969700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd8068000 con 0x7f0dd8066690 2023-11-15T03:11:25.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.880+0000 7f0ddd969700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd80682e0 con 0x7f0dd8066ee0 2023-11-15T03:11:25.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.884+0000 7f0dd77fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0dd8066ee0 0x7f0dd8067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:25.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.884+0000 7f0dd77fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0dd8066ee0 0x7f0dd8067a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40470/0 (socket says 172.21.15.121:40470) 2023-11-15T03:11:25.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.884+0000 7f0dd77fe700 1 -- 172.21.15.121:0/1539592647 learned_addr learned my addr 172.21.15.121:0/1539592647 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:25.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.884+0000 7f0dce7fc700 1 --1- 172.21.15.121:0/1539592647 >> v1:172.21.15.190:6789/0 conn(0x7f0dd8066690 0x7f0dd80c1a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:49930/0 (socket says 172.21.15.121:49930) 2023-11-15T03:11:25.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.884+0000 7f0dd6ffd700 1 --1- 172.21.15.121:0/1539592647 >> v1:172.21.15.121:6789/0 conn(0x7f0dd8153890 0x7f0dd8065f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60846/0 (socket says 172.21.15.121:60846) 2023-11-15T03:11:25.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.884+0000 7f0dd77fe700 1 -- 172.21.15.121:0/1539592647 >> v1:172.21.15.190:6789/0 conn(0x7f0dd8066690 legacy=0x7f0dd80c1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:25.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.884+0000 7f0dd77fe700 1 -- 172.21.15.121:0/1539592647 >> v1:172.21.15.121:6789/0 conn(0x7f0dd8153890 legacy=0x7f0dd8065f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:25.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.884+0000 7f0dd77fe700 1 -- 172.21.15.121:0/1539592647 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd80685c0 con 0x7f0dd8066ee0 2023-11-15T03:11:25.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.884+0000 7f0dd77fe700 1 --2- 172.21.15.121:0/1539592647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dd8066ee0 0x7f0dd8067a60 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 rx=0x7f0dd0009f20 tx=0x7f0dd0009840).ready entity=mon.0 client_cookie=7a195801c35e3c11 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:25.896 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.884+0000 7f0dd67fc700 1 -- 172.21.15.121:0/1539592647 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd000e070 con 0x7f0dd8066ee0 2023-11-15T03:11:25.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.884+0000 7f0dd67fc700 1 -- 172.21.15.121:0/1539592647 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd000ba80 con 0x7f0dd8066ee0 2023-11-15T03:11:25.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.884+0000 7f0dd67fc700 1 -- 172.21.15.121:0/1539592647 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd000be00 con 0x7f0dd8066ee0 2023-11-15T03:11:25.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.884+0000 7f0ddd969700 1 -- 172.21.15.121:0/1539592647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dd8066ee0 msgr2=0x7f0dd8067a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:25.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.884+0000 7f0ddd969700 1 --2- 172.21.15.121:0/1539592647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dd8066ee0 0x7f0dd8067a60 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 rx=0x7f0dd0009f20 tx=0x7f0dd0009840).stop 2023-11-15T03:11:25.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.884+0000 7f0ddd969700 1 -- 172.21.15.121:0/1539592647 shutdown_connections 2023-11-15T03:11:25.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.884+0000 7f0ddd969700 1 --2- 172.21.15.121:0/1539592647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dd8066ee0 0x7f0dd8067a60 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.884+0000 7f0ddd969700 1 -- 172.21.15.121:0/1539592647 >> 172.21.15.121:0/1539592647 conn(0x7f0dd8062c20 msgr2=0x7f0dd80c6f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:25.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.884+0000 7f0ddd969700 1 -- 172.21.15.121:0/1539592647 shutdown_connections 2023-11-15T03:11:25.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.884+0000 7f0ddd969700 1 -- 172.21.15.121:0/1539592647 wait complete. 2023-11-15T03:11:25.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.888+0000 7f0ddd969700 1 Processor -- start 2023-11-15T03:11:25.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.888+0000 7f0ddd969700 1 -- start start 2023-11-15T03:11:25.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.888+0000 7f0ddd969700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dd8066690 0x7f0dd8141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:25.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.888+0000 7f0ddd969700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dd8066ee0 0x7f0dd8155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:25.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.888+0000 7f0ddd969700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dd8153890 0x7f0dd80c0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:25.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.888+0000 7f0ddd969700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd8141fa0 con 0x7f0dd8066ee0 2023-11-15T03:11:25.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.888+0000 7f0ddd969700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd80bdfc0 con 0x7f0dd8066690 2023-11-15T03:11:25.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.888+0000 7f0ddd969700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd8154b10 con 0x7f0dd8153890 2023-11-15T03:11:25.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.888+0000 7f0dce7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dd8066ee0 0x7f0dd8155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:25.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.888+0000 7f0dce7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dd8066ee0 0x7f0dd8155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40480/0 (socket says 172.21.15.121:40480) 2023-11-15T03:11:25.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.888+0000 7f0dce7fc700 1 -- 172.21.15.121:0/1137062185 learned_addr learned my addr 172.21.15.121:0/1137062185 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:25.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.888+0000 7f0dd6ffd700 1 --2- 172.21.15.121:0/1137062185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dd8066690 0x7f0dd8141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:25.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.888+0000 7f0dd77fe700 1 --2- 172.21.15.121:0/1137062185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dd8153890 0x7f0dd80c0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:25.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.888+0000 7f0dce7fc700 1 -- 172.21.15.121:0/1137062185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dd8153890 msgr2=0x7f0dd80c0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:25.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.888+0000 7f0dce7fc700 1 --2- 172.21.15.121:0/1137062185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dd8153890 0x7f0dd80c0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.888+0000 7f0dce7fc700 1 -- 172.21.15.121:0/1137062185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dd8066690 msgr2=0x7f0dd8141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:25.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.888+0000 7f0dce7fc700 1 --2- 172.21.15.121:0/1137062185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dd8066690 0x7f0dd8141ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:25.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.888+0000 7f0dce7fc700 1 -- 172.21.15.121:0/1137062185 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd0009ae0 con 0x7f0dd8066ee0 2023-11-15T03:11:25.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.888+0000 7f0dd6ffd700 1 --2- 172.21.15.121:0/1137062185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dd8066690 0x7f0dd8141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:25.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.892+0000 7f0dce7fc700 1 --2- 172.21.15.121:0/1137062185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dd8066ee0 0x7f0dd8155bf0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 rx=0x7f0dc000c490 tx=0x7f0dc0007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:25.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.892+0000 7f0dcf7fe700 1 -- 172.21.15.121:0/1137062185 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc00072e0 con 0x7f0dd8066ee0 2023-11-15T03:11:25.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.892+0000 7f0dcf7fe700 1 -- 172.21.15.121:0/1137062185 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc0007440 con 0x7f0dd8066ee0 2023-11-15T03:11:25.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.892+0000 7f0dcf7fe700 1 -- 172.21.15.121:0/1137062185 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc00058f0 con 0x7f0dd8066ee0 2023-11-15T03:11:25.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.892+0000 7f0ddd969700 1 -- 172.21.15.121:0/1137062185 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd8154df0 con 0x7f0dd8066ee0 2023-11-15T03:11:25.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.892+0000 7f0ddd969700 1 -- 172.21.15.121:0/1137062185 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd80be210 con 0x7f0dd8066ee0 2023-11-15T03:11:25.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.892+0000 7f0ddd969700 1 -- 172.21.15.121:0/1137062185 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd80be430 con 0x7f0dd8066ee0 2023-11-15T03:11:25.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.892+0000 7f0dcf7fe700 1 -- 172.21.15.121:0/1137062185 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0dc0005ca0 con 0x7f0dd8066ee0 2023-11-15T03:11:25.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.896+0000 7f0dcf7fe700 1 --2- 172.21.15.121:0/1137062185 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0db805a510 0x7f0db805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:25.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.896+0000 7f0dcf7fe700 1 -- 172.21.15.121:0/1137062185 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0dc0073f20 con 0x7f0dd8066ee0 2023-11-15T03:11:25.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.896+0000 7f0dd6ffd700 1 --2- 172.21.15.121:0/1137062185 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0db805a510 0x7f0db805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:25.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.896+0000 7f0dd6ffd700 1 --2- 172.21.15.121:0/1137062185 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0db805a510 0x7f0db805c9d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7f0dd8157c80 tx=0x7f0dc8007cb0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:25.914 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:25.904+0000 7f0dcf7fe700 1 -- 172.21.15.121:0/1137062185 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0dc0047700 con 0x7f0dd8066ee0 2023-11-15T03:11:26.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.060+0000 7f0ddd969700 1 -- 172.21.15.121:0/1137062185 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0dd806bcf0 con 0x7f0db805a510 2023-11-15T03:11:26.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.072+0000 7f0dcf7fe700 1 -- 172.21.15.121:0/1137062185 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+349970 (secure 0 0 0) 0x7f0dd806bcf0 con 0x7f0db805a510 2023-11-15T03:11:26.087 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:26.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.084+0000 7f0ddd969700 1 -- 172.21.15.121:0/1137062185 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0db805a510 msgr2=0x7f0db805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:26.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.084+0000 7f0ddd969700 1 --2- 172.21.15.121:0/1137062185 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0db805a510 0x7f0db805c9d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7f0dd8157c80 tx=0x7f0dc8007cb0).stop 2023-11-15T03:11:26.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.084+0000 7f0ddd969700 1 -- 172.21.15.121:0/1137062185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dd8066ee0 msgr2=0x7f0dd8155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:26.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.084+0000 7f0ddd969700 1 --2- 172.21.15.121:0/1137062185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dd8066ee0 0x7f0dd8155bf0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 rx=0x7f0dc000c490 tx=0x7f0dc0007df0).stop 2023-11-15T03:11:26.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.084+0000 7f0ddd969700 1 -- 172.21.15.121:0/1137062185 shutdown_connections 2023-11-15T03:11:26.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.084+0000 7f0ddd969700 1 --2- 172.21.15.121:0/1137062185 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0db805a510 0x7f0db805c9d0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:26.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.084+0000 7f0ddd969700 1 --2- 172.21.15.121:0/1137062185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dd8153890 0x7f0dd80c0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:26.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.084+0000 7f0ddd969700 1 --2- 172.21.15.121:0/1137062185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dd8066ee0 0x7f0dd8155bf0 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:26.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.084+0000 7f0ddd969700 1 --2- 172.21.15.121:0/1137062185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dd8066690 0x7f0dd8141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:26.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.084+0000 7f0ddd969700 1 -- 172.21.15.121:0/1137062185 >> 172.21.15.121:0/1137062185 conn(0x7f0dd8062c20 msgr2=0x7f0dd80bd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:26.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.084+0000 7f0ddd969700 1 -- 172.21.15.121:0/1137062185 shutdown_connections 2023-11-15T03:11:26.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.084+0000 7f0ddd969700 1 -- 172.21.15.121:0/1137062185 wait complete. 2023-11-15T03:11:26.095 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T03:11:26.114 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":578,"stamp":"2023-11-15T03:11:24.257472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":588,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2440356,"kb_used_data":14436,"kb_used_omap":0,"kb_used_meta":2425856,"kb_avail":752534364,"statfs":{"total":773094113280,"available":770595188736,"internally_reserved":0,"allocated":14782464,"data_stored":7485017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2484076544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6309354,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-954,"num_read_kb":-972,"num_write":-3308,"num_write_kb":-7436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.113944"},"pg_stats":[{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3f","version":"144'3","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.900035+0000","last_change":"2023-11-15T03:11:21.456072+0000","last_active":"2023-11-15T03:11:21.900035+0000","last_peered":"2023-11-15T03:11:21.900035+0000","last_clean":"2023-11-15T03:11:21.900035+0000","last_became_active":"2023-11-15T03:11:21.455710+0000","last_became_peered":"2023-11-15T03:11:21.455710+0000","last_unstale":"2023-11-15T03:11:21.900035+0000","last_undegraded":"2023-11-15T03:11:21.900035+0000","last_fullsized":"2023-11-15T03:11:21.900035+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.874464+0000","last_change":"2023-11-15T03:11:21.457108+0000","last_active":"2023-11-15T03:11:21.874464+0000","last_peered":"2023-11-15T03:11:21.874464+0000","last_clean":"2023-11-15T03:11:21.874464+0000","last_became_active":"2023-11-15T03:11:21.456137+0000","last_became_peered":"2023-11-15T03:11:21.456137+0000","last_unstale":"2023-11-15T03:11:21.874464+0000","last_undegraded":"2023-11-15T03:11:21.874464+0000","last_fullsized":"2023-11-15T03:11:21.874464+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.912452+0000","last_change":"2023-11-15T03:11:21.906488+0000","last_active":"2023-11-15T03:11:21.912452+0000","last_peered":"2023-11-15T03:11:21.912452+0000","last_clean":"2023-11-15T03:11:21.912452+0000","last_became_active":"2023-11-15T03:11:21.906190+0000","last_became_peered":"2023-11-15T03:11:21.906190+0000","last_unstale":"2023-11-15T03:11:21.912452+0000","last_undegraded":"2023-11-15T03:11:21.912452+0000","last_fullsized":"2023-11-15T03:11:21.912452+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.917672+0000","last_change":"2023-11-15T03:11:21.912731+0000","last_active":"2023-11-15T03:11:21.917672+0000","last_peered":"2023-11-15T03:11:21.917672+0000","last_clean":"2023-11-15T03:11:21.917672+0000","last_became_active":"2023-11-15T03:11:21.912460+0000","last_became_peered":"2023-11-15T03:11:21.912460+0000","last_unstale":"2023-11-15T03:11:21.917672+0000","last_undegraded":"2023-11-15T03:11:21.917672+0000","last_fullsized":"2023-11-15T03:11:21.917672+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.902421+0000","last_change":"2023-11-15T03:11:21.893136+0000","last_active":"2023-11-15T03:11:21.902421+0000","last_peered":"2023-11-15T03:11:21.902421+0000","last_clean":"2023-11-15T03:11:21.902421+0000","last_became_active":"2023-11-15T03:11:21.892853+0000","last_became_peered":"2023-11-15T03:11:21.892853+0000","last_unstale":"2023-11-15T03:11:21.902421+0000","last_undegraded":"2023-11-15T03:11:21.902421+0000","last_fullsized":"2023-11-15T03:11:21.902421+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.917167+0000","last_change":"2023-11-15T03:11:21.913176+0000","last_active":"2023-11-15T03:11:21.917167+0000","last_peered":"2023-11-15T03:11:21.917167+0000","last_clean":"2023-11-15T03:11:21.917167+0000","last_became_active":"2023-11-15T03:11:21.912877+0000","last_became_peered":"2023-11-15T03:11:21.912877+0000","last_unstale":"2023-11-15T03:11:21.917167+0000","last_undegraded":"2023-11-15T03:11:21.917167+0000","last_fullsized":"2023-11-15T03:11:21.917167+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.917703+0000","last_change":"2023-11-15T03:11:21.912892+0000","last_active":"2023-11-15T03:11:21.917703+0000","last_peered":"2023-11-15T03:11:21.917703+0000","last_clean":"2023-11-15T03:11:21.917703+0000","last_became_active":"2023-11-15T03:11:21.912547+0000","last_became_peered":"2023-11-15T03:11:21.912547+0000","last_unstale":"2023-11-15T03:11:21.917703+0000","last_undegraded":"2023-11-15T03:11:21.917703+0000","last_fullsized":"2023-11-15T03:11:21.917703+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.902563+0000","last_change":"2023-11-15T03:11:21.892450+0000","last_active":"2023-11-15T03:11:21.902563+0000","last_peered":"2023-11-15T03:11:21.902563+0000","last_clean":"2023-11-15T03:11:21.902563+0000","last_became_active":"2023-11-15T03:11:21.892196+0000","last_became_peered":"2023-11-15T03:11:21.892196+0000","last_unstale":"2023-11-15T03:11:21.902563+0000","last_undegraded":"2023-11-15T03:11:21.902563+0000","last_fullsized":"2023-11-15T03:11:21.902563+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.873269+0000","last_change":"2023-11-15T03:11:21.472198+0000","last_active":"2023-11-15T03:11:21.873269+0000","last_peered":"2023-11-15T03:11:21.873269+0000","last_clean":"2023-11-15T03:11:21.873269+0000","last_became_active":"2023-11-15T03:11:21.471631+0000","last_became_peered":"2023-11-15T03:11:21.471631+0000","last_unstale":"2023-11-15T03:11:21.873269+0000","last_undegraded":"2023-11-15T03:11:21.873269+0000","last_fullsized":"2023-11-15T03:11:21.873269+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.908036+0000","last_change":"2023-11-15T03:11:21.899337+0000","last_active":"2023-11-15T03:11:21.908036+0000","last_peered":"2023-11-15T03:11:21.908036+0000","last_clean":"2023-11-15T03:11:21.908036+0000","last_became_active":"2023-11-15T03:11:21.899050+0000","last_became_peered":"2023-11-15T03:11:21.899050+0000","last_unstale":"2023-11-15T03:11:21.908036+0000","last_undegraded":"2023-11-15T03:11:21.908036+0000","last_fullsized":"2023-11-15T03:11:21.908036+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.874950+0000","last_change":"2023-11-15T03:11:21.458880+0000","last_active":"2023-11-15T03:11:21.874950+0000","last_peered":"2023-11-15T03:11:21.874950+0000","last_clean":"2023-11-15T03:11:21.874950+0000","last_became_active":"2023-11-15T03:11:21.458602+0000","last_became_peered":"2023-11-15T03:11:21.458602+0000","last_unstale":"2023-11-15T03:11:21.874950+0000","last_undegraded":"2023-11-15T03:11:21.874950+0000","last_fullsized":"2023-11-15T03:11:21.874950+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.890211+0000","last_change":"2023-11-15T03:11:21.457749+0000","last_active":"2023-11-15T03:11:21.890211+0000","last_peered":"2023-11-15T03:11:21.890211+0000","last_clean":"2023-11-15T03:11:21.890211+0000","last_became_active":"2023-11-15T03:11:21.456847+0000","last_became_peered":"2023-11-15T03:11:21.456847+0000","last_unstale":"2023-11-15T03:11:21.890211+0000","last_undegraded":"2023-11-15T03:11:21.890211+0000","last_fullsized":"2023-11-15T03:11:21.890211+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.900698+0000","last_change":"2023-11-15T03:11:21.890359+0000","last_active":"2023-11-15T03:11:21.900698+0000","last_peered":"2023-11-15T03:11:21.900698+0000","last_clean":"2023-11-15T03:11:21.900698+0000","last_became_active":"2023-11-15T03:11:21.890059+0000","last_became_peered":"2023-11-15T03:11:21.890059+0000","last_unstale":"2023-11-15T03:11:21.900698+0000","last_undegraded":"2023-11-15T03:11:21.900698+0000","last_fullsized":"2023-11-15T03:11:21.900698+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.902545+0000","last_change":"2023-11-15T03:11:21.890998+0000","last_active":"2023-11-15T03:11:21.902545+0000","last_peered":"2023-11-15T03:11:21.902545+0000","last_clean":"2023-11-15T03:11:21.902545+0000","last_became_active":"2023-11-15T03:11:21.890615+0000","last_became_peered":"2023-11-15T03:11:21.890615+0000","last_unstale":"2023-11-15T03:11:21.902545+0000","last_undegraded":"2023-11-15T03:11:21.902545+0000","last_fullsized":"2023-11-15T03:11:21.902545+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.910346+0000","last_change":"2023-11-15T03:11:21.906419+0000","last_active":"2023-11-15T03:11:21.910346+0000","last_peered":"2023-11-15T03:11:21.910346+0000","last_clean":"2023-11-15T03:11:21.910346+0000","last_became_active":"2023-11-15T03:11:21.906123+0000","last_became_peered":"2023-11-15T03:11:21.906123+0000","last_unstale":"2023-11-15T03:11:21.910346+0000","last_undegraded":"2023-11-15T03:11:21.910346+0000","last_fullsized":"2023-11-15T03:11:21.910346+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"144'1","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.896366+0000","last_change":"2023-11-15T03:11:21.465973+0000","last_active":"2023-11-15T03:11:21.896366+0000","last_peered":"2023-11-15T03:11:21.896366+0000","last_clean":"2023-11-15T03:11:21.896366+0000","last_became_active":"2023-11-15T03:11:21.465530+0000","last_became_peered":"2023-11-15T03:11:21.465530+0000","last_unstale":"2023-11-15T03:11:21.896366+0000","last_undegraded":"2023-11-15T03:11:21.896366+0000","last_fullsized":"2023-11-15T03:11:21.896366+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"144'2","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.899943+0000","last_change":"2023-11-15T03:11:21.464465+0000","last_active":"2023-11-15T03:11:21.899943+0000","last_peered":"2023-11-15T03:11:21.899943+0000","last_clean":"2023-11-15T03:11:21.899943+0000","last_became_active":"2023-11-15T03:11:21.464201+0000","last_became_peered":"2023-11-15T03:11:21.464201+0000","last_unstale":"2023-11-15T03:11:21.899943+0000","last_undegraded":"2023-11-15T03:11:21.899943+0000","last_fullsized":"2023-11-15T03:11:21.899943+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.917152+0000","last_change":"2023-11-15T03:11:21.913399+0000","last_active":"2023-11-15T03:11:21.917152+0000","last_peered":"2023-11-15T03:11:21.917152+0000","last_clean":"2023-11-15T03:11:21.917152+0000","last_became_active":"2023-11-15T03:11:21.912859+0000","last_became_peered":"2023-11-15T03:11:21.912859+0000","last_unstale":"2023-11-15T03:11:21.917152+0000","last_undegraded":"2023-11-15T03:11:21.917152+0000","last_fullsized":"2023-11-15T03:11:21.917152+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.874798+0000","last_change":"2023-11-15T03:11:21.465450+0000","last_active":"2023-11-15T03:11:21.874798+0000","last_peered":"2023-11-15T03:11:21.874798+0000","last_clean":"2023-11-15T03:11:21.874798+0000","last_became_active":"2023-11-15T03:11:21.465152+0000","last_became_peered":"2023-11-15T03:11:21.465152+0000","last_unstale":"2023-11-15T03:11:21.874798+0000","last_undegraded":"2023-11-15T03:11:21.874798+0000","last_fullsized":"2023-11-15T03:11:21.874798+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.909613+0000","last_change":"2023-11-15T03:11:21.904368+0000","last_active":"2023-11-15T03:11:21.909613+0000","last_peered":"2023-11-15T03:11:21.909613+0000","last_clean":"2023-11-15T03:11:21.909613+0000","last_became_active":"2023-11-15T03:11:21.904073+0000","last_became_peered":"2023-11-15T03:11:21.904073+0000","last_unstale":"2023-11-15T03:11:21.909613+0000","last_undegraded":"2023-11-15T03:11:21.909613+0000","last_fullsized":"2023-11-15T03:11:21.909613+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":119,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-11-15T03:11:16.676373+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T03:11:16.676373+0000","last_peered":"2023-11-15T03:11:16.676373+0000","last_clean":"2023-11-15T03:11:16.676373+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T03:11:16.676373+0000","last_undegraded":"2023-11-15T03:11:16.676373+0000","last_fullsized":"2023-11-15T03:11:16.676373+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":126,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.876350+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T03:11:21.876350+0000","last_peered":"2023-11-15T03:11:21.876350+0000","last_clean":"2023-11-15T03:11:21.876350+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T03:11:21.876350+0000","last_undegraded":"2023-11-15T03:11:21.876350+0000","last_fullsized":"2023-11-15T03:11:21.876350+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":117,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-11-15T03:11:16.540787+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T03:11:16.540787+0000","last_peered":"2023-11-15T03:11:16.540787+0000","last_clean":"2023-11-15T03:11:16.540787+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T03:11:16.540787+0000","last_undegraded":"2023-11-15T03:11:16.540787+0000","last_fullsized":"2023-11-15T03:11:16.540787+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":126,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.874939+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T03:11:21.874939+0000","last_peered":"2023-11-15T03:11:21.874939+0000","last_clean":"2023-11-15T03:11:21.874939+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T03:11:21.874939+0000","last_undegraded":"2023-11-15T03:11:21.874939+0000","last_fullsized":"2023-11-15T03:11:21.874939+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":137,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.890260+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T03:11:21.890260+0000","last_peered":"2023-11-15T03:11:21.890260+0000","last_clean":"2023-11-15T03:11:21.890260+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T03:11:21.890260+0000","last_undegraded":"2023-11-15T03:11:21.890260+0000","last_fullsized":"2023-11-15T03:11:21.890260+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.902579+0000","last_change":"2023-11-15T03:11:21.896499+0000","last_active":"2023-11-15T03:11:21.902579+0000","last_peered":"2023-11-15T03:11:21.902579+0000","last_clean":"2023-11-15T03:11:21.902579+0000","last_became_active":"2023-11-15T03:11:21.896160+0000","last_became_peered":"2023-11-15T03:11:21.896160+0000","last_unstale":"2023-11-15T03:11:21.902579+0000","last_undegraded":"2023-11-15T03:11:21.902579+0000","last_fullsized":"2023-11-15T03:11:21.902579+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.890040+0000","last_change":"2023-11-15T03:11:21.470370+0000","last_active":"2023-11-15T03:11:21.890040+0000","last_peered":"2023-11-15T03:11:21.890040+0000","last_clean":"2023-11-15T03:11:21.890040+0000","last_became_active":"2023-11-15T03:11:21.470048+0000","last_became_peered":"2023-11-15T03:11:21.470048+0000","last_unstale":"2023-11-15T03:11:21.890040+0000","last_undegraded":"2023-11-15T03:11:21.890040+0000","last_fullsized":"2023-11-15T03:11:21.890040+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":117,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-11-15T03:11:17.620997+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T03:11:17.620997+0000","last_peered":"2023-11-15T03:11:17.620997+0000","last_clean":"2023-11-15T03:11:17.620997+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T03:11:17.620997+0000","last_undegraded":"2023-11-15T03:11:17.620997+0000","last_fullsized":"2023-11-15T03:11:17.620997+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:21.412123+0000","last_change":"2023-11-15T03:11:21.412123+0000","last_active":"2023-11-15T03:11:21.412123+0000","last_peered":"2023-11-15T03:11:21.412123+0000","last_clean":"2023-11-15T03:11:21.412123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:21.412123+0000","last_undegraded":"2023-11-15T03:11:21.412123+0000","last_fullsized":"2023-11-15T03:11:21.412123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":117,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-11-15T03:11:16.540824+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T03:11:16.540824+0000","last_peered":"2023-11-15T03:11:16.540824+0000","last_clean":"2023-11-15T03:11:16.540824+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T03:11:16.540824+0000","last_undegraded":"2023-11-15T03:11:16.540824+0000","last_fullsized":"2023-11-15T03:11:16.540824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.909627+0000","last_change":"2023-11-15T03:11:21.904610+0000","last_active":"2023-11-15T03:11:21.909627+0000","last_peered":"2023-11-15T03:11:21.909627+0000","last_clean":"2023-11-15T03:11:21.909627+0000","last_became_active":"2023-11-15T03:11:21.903447+0000","last_became_peered":"2023-11-15T03:11:21.903447+0000","last_unstale":"2023-11-15T03:11:21.909627+0000","last_undegraded":"2023-11-15T03:11:21.909627+0000","last_fullsized":"2023-11-15T03:11:21.909627+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-15T03:11:20.396314+0000","last_change":"2023-11-15T03:11:20.396314+0000","last_active":"2023-11-15T03:11:20.396314+0000","last_peered":"2023-11-15T03:11:20.396314+0000","last_clean":"2023-11-15T03:11:20.396314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-15T03:11:20.396314+0000","last_undegraded":"2023-11-15T03:11:20.396314+0000","last_fullsized":"2023-11-15T03:11:20.396314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":137,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.890372+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T03:11:21.890372+0000","last_peered":"2023-11-15T03:11:21.890372+0000","last_clean":"2023-11-15T03:11:21.890372+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T03:11:21.890372+0000","last_undegraded":"2023-11-15T03:11:21.890372+0000","last_fullsized":"2023-11-15T03:11:21.890372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-11-15T03:11:16.540804+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T03:11:16.540804+0000","last_peered":"2023-11-15T03:11:16.540804+0000","last_clean":"2023-11-15T03:11:16.540804+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T03:11:16.540804+0000","last_undegraded":"2023-11-15T03:11:16.540804+0000","last_fullsized":"2023-11-15T03:11:16.540804+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":28,"acting":28,"num_store_stats":2},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":588,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":18,"acting":18,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542321,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307080,"kb_used_data":1792,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064760,"statfs":{"total":96636764160,"available":96322314240,"internally_reserved":0,"allocated":1835008,"data_stored":929665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.0940000000000001,"15min":1.1319999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.55800000000000005},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":10.961},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.032,"5min":1.0880000000000001,"15min":1.1120000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.3089999999999999,"5min":1.6160000000000001,"15min":3.4910000000000001},"last":1.258}]},{"osd":1,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.127,"15min":1.143},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.419,"5min":1.552,"15min":5.4930000000000003},"last":1.45},{"interface":"front","average":{"1min":1.153,"5min":1.165,"15min":1.1819999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6000000000000001,"5min":1.766,"15min":6.8239999999999998},"last":1.175}]},{"osd":2,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0940000000000001,"15min":1.1359999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.452,"5min":1.452,"15min":5.7000000000000002},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.226,"15min":1.2310000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":2.75},"last":1.387}]},{"osd":3,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.167,"15min":1.248},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":12.907},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2370000000000001,"15min":1.2709999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.573,"5min":1.573,"15min":9.9909999999999997},"last":1.1930000000000001}]},{"osd":4,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.105,"15min":1.1379999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6160000000000001,"5min":1.8939999999999999,"15min":2.5649999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.1240000000000001,"15min":1.2070000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.72,"5min":1.72,"15min":4.226},"last":1.0489999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.2450000000000001,"15min":1.2749999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":2.6890000000000001},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.143,"5min":1.236,"15min":1.292},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6899999999999999,"5min":1.7330000000000001,"15min":2.4550000000000001},"last":1.5309999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.325,"15min":1.381},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.5629999999999999,"5min":1.954,"15min":3.2789999999999999},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.248,"15min":1.3340000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.63600000000000001},"max":{"1min":1.542,"5min":1.76,"15min":4.7800000000000002},"last":1.4179999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542321,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311176,"kb_used_data":1792,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94060664,"statfs":{"total":96636764160,"available":96318119936,"internally_reserved":0,"allocated":1835008,"data_stored":929665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.03,"15min":1.077},"min":{"1min":0.66200000000000003,"5min":0.63100000000000001,"15min":0.44500000000000001},"max":{"1min":1.4610000000000001,"5min":1.5760000000000001,"15min":4.3159999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.109,"15min":1.0960000000000001},"min":{"1min":0.69999999999999996,"5min":0.59799999999999998,"15min":0.58599999999999997},"max":{"1min":1.486,"5min":1.659,"15min":2.2480000000000002},"last":0.91700000000000004}]},{"osd":1,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.111,"15min":1.1140000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.57599999999999996},"max":{"1min":1.6000000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1000000000000001,"15min":1.1379999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.60699999999999998},"max":{"1min":1.3360000000000001,"5min":1.8100000000000001,"15min":2.8570000000000002},"last":1.427}]},{"osd":2,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1899999999999999,"15min":1.1919999999999999},"min":{"1min":0.83099999999999996,"5min":0.71699999999999997,"15min":0.49099999999999999},"max":{"1min":1.4730000000000001,"5min":1.7090000000000001,"15min":2.621},"last":1.401},{"interface":"front","average":{"1min":1.115,"5min":1.153,"15min":1.153},"min":{"1min":0.71699999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.4710000000000001,"5min":1.8360000000000001,"15min":2.0059999999999998},"last":1.3280000000000001}]},{"osd":3,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.2310000000000001,"15min":1.214},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.50900000000000001},"max":{"1min":1.7709999999999999,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.105},{"interface":"front","average":{"1min":1.234,"5min":1.2430000000000001,"15min":1.2609999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.67100000000000004},"max":{"1min":1.458,"5min":1.8640000000000001,"15min":2.8359999999999999},"last":1.49}]},{"osd":4,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.0580000000000001,"15min":1.1000000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5860000000000001,"5min":1.6839999999999999,"15min":4.5350000000000001},"last":1.181},{"interface":"front","average":{"1min":1.163,"5min":1.2170000000000001,"15min":1.2130000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.8580000000000001,"5min":2.004,"15min":2.0739999999999998},"last":1.7749999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3799999999999999,"15min":1.415},"min":{"1min":0.79800000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.744,"5min":2.1469999999999998,"15min":2.157},"last":1.883},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2210000000000001,"15min":1.256},"min":{"1min":0.621,"5min":0.62,"15min":0.60899999999999999},"max":{"1min":1.8140000000000001,"5min":1.8779999999999999,"15min":2.9460000000000002},"last":1.6519999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.216,"15min":1.2250000000000001},"min":{"1min":0.77400000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.9139999999999999,"5min":1.9239999999999999,"15min":3.1989999999999998},"last":1.548},{"interface":"front","average":{"1min":1.377,"5min":1.407,"15min":1.419},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.65200000000000002},"max":{"1min":1.8220000000000001,"5min":2.0649999999999999,"15min":2.9350000000000001},"last":1.976}]}]},{"osd":4,"up_from":14,"seq":60129542322,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":307080,"kb_used_data":1792,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064760,"statfs":{"total":96636764160,"available":96322314240,"internally_reserved":0,"allocated":1835008,"data_stored":929665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.0569999999999999,"15min":1.083},"min":{"1min":0.73799999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4810000000000001,"5min":1.5660000000000001,"15min":1.5840000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.044,"15min":1.056},"min":{"1min":0.85399999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.677},"last":1.1839999999999999}]},{"osd":1,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.135,"15min":1.1319999999999999},"min":{"1min":0.70599999999999996,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":2.617},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.212,"5min":1.1419999999999999,"15min":1.1539999999999999},"min":{"1min":0.82099999999999995,"5min":0.629,"15min":0.54800000000000004},"max":{"1min":1.573,"5min":1.6319999999999999,"15min":1.6619999999999999},"last":1.145}]},{"osd":2,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.165,"15min":1.1830000000000001},"min":{"1min":0.81399999999999995,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.5029999999999999,"5min":1.823,"15min":1.9450000000000001},"last":1.405},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.155,"15min":1.1619999999999999},"min":{"1min":0.84199999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9330000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.996}]},{"osd":3,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1399999999999999,"15min":1.1459999999999999},"min":{"1min":0.85699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.5229999999999999,"5min":1.5940000000000001,"15min":1.8660000000000001},"last":1.258},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2549999999999999,"15min":1.264},"min":{"1min":0.93000000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.611,"5min":1.8200000000000001,"15min":2.5630000000000002},"last":1.371}]},{"osd":5,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1479999999999999,"15min":1.1830000000000001},"min":{"1min":0.91900000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.554,"5min":1.643,"15min":2.0840000000000001},"last":1.544},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.1479999999999999,"15min":1.1879999999999999},"min":{"1min":0.89400000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":2.3570000000000002},"last":1.294}]},{"osd":6,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2410000000000001,"15min":1.264},"min":{"1min":0.97999999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5880000000000001,"5min":1.913,"15min":2.6640000000000001},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.288,"15min":1.323},"min":{"1min":0.88,"5min":0.67600000000000005,"15min":0.57299999999999995},"max":{"1min":1.585,"5min":1.9139999999999999,"15min":2.173},"last":1.6339999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2769999999999999,"15min":1.3069999999999999},"min":{"1min":0.89500000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.7629999999999999,"5min":1.9199999999999999,"15min":2.6030000000000002},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.3700000000000001,"15min":1.3879999999999999},"min":{"1min":1.044,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8240000000000001,"5min":2.069,"15min":2.7669999999999999},"last":1.665}]}]},{"osd":3,"up_from":14,"seq":60129542322,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302984,"kb_used_data":1792,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068856,"statfs":{"total":96636764160,"available":96326508544,"internally_reserved":0,"allocated":1835008,"data_stored":929665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85999999999999999,"15min":0.83699999999999997},"min":{"1min":0.54200000000000004,"5min":0.54000000000000004,"15min":0.45400000000000001},"max":{"1min":1.327,"5min":1.4219999999999999,"15min":1.528},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.89900000000000002,"15min":0.88800000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.39800000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.80100000000000005}]},{"osd":1,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.83899999999999997,"15min":0.872},"min":{"1min":0.62,"5min":0.53100000000000003,"15min":0.435},"max":{"1min":1.165,"5min":1.25,"15min":1.4670000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.996,"5min":0.98599999999999999,"15min":0.99199999999999999},"min":{"1min":0.72199999999999998,"5min":0.63700000000000001,"15min":0.55800000000000005},"max":{"1min":1.446,"5min":1.446,"15min":1.544},"last":0.84199999999999997}]},{"osd":2,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.026,"15min":1.0209999999999999},"min":{"1min":0.66400000000000003,"5min":0.64200000000000002,"15min":0.54600000000000004},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0529999999999999,"15min":1.0660000000000001},"min":{"1min":0.66000000000000003,"5min":0.63100000000000001,"15min":0.50600000000000001},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.77100000000000002}]},{"osd":4,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2470000000000001,"15min":1.228},"min":{"1min":1.0029999999999999,"5min":0.88600000000000001,"15min":0.71399999999999997},"max":{"1min":1.5349999999999999,"5min":1.554,"15min":1.9139999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2170000000000001,"15min":1.2250000000000001},"min":{"1min":1.01,"5min":0.78500000000000003,"15min":0.621},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.726},"last":1.091}]},{"osd":5,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.252,"15min":1.266},"min":{"1min":0.91700000000000004,"5min":0.90200000000000002,"15min":0.64300000000000002},"max":{"1min":1.5900000000000001,"5min":1.7290000000000001,"15min":2.0459999999999998},"last":1.048},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.27,"15min":1.272},"min":{"1min":0.95899999999999996,"5min":0.94799999999999995,"15min":0.63500000000000001},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.927},"last":1.0009999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.345,"15min":1.339},"min":{"1min":0.91400000000000003,"5min":0.875,"15min":0.72899999999999998},"max":{"1min":1.575,"5min":1.8129999999999999,"15min":1.9970000000000001},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3560000000000001,"15min":1.3720000000000001},"min":{"1min":1.0980000000000001,"5min":0.871,"15min":0.77700000000000002},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":2.3570000000000002},"last":1.206}]},{"osd":7,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.331,"15min":1.3440000000000001},"min":{"1min":1.0149999999999999,"5min":0.92500000000000004,"15min":0.68100000000000005},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":2.0470000000000002},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.345,"15min":1.355},"min":{"1min":1.113,"5min":0.97299999999999998,"15min":0.66600000000000004},"max":{"1min":1.8500000000000001,"5min":1.8779999999999999,"15min":2.1040000000000001},"last":1.4219999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542322,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302980,"kb_used_data":1788,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068860,"statfs":{"total":96636764160,"available":96326512640,"internally_reserved":0,"allocated":1830912,"data_stored":928396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.65400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97099999999999997,"5min":1.2350000000000001,"15min":1.409},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.84299999999999997,"15min":0.84399999999999997},"min":{"1min":0.58799999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2689999999999999,"5min":1.4950000000000001,"15min":1.619},"last":1.034}]},{"osd":1,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.90000000000000002,"15min":0.91100000000000003},"min":{"1min":0.74199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":1.095},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.95599999999999996,"15min":0.95399999999999996},"min":{"1min":0.67200000000000004,"5min":0.55100000000000005,"15min":0.54600000000000004},"max":{"1min":1.3580000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":1.1259999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95599999999999996,"15min":0.97199999999999998},"min":{"1min":0.60699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.385,"5min":1.4359999999999999,"15min":1.6259999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.125,"15min":1.125},"min":{"1min":0.67700000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.488,"5min":1.5549999999999999,"15min":1.5800000000000001},"last":0.98499999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.218,"15min":1.2070000000000001},"min":{"1min":1.052,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.581,"5min":1.583,"15min":1.9410000000000001},"last":1.419},{"interface":"front","average":{"1min":1.3,"5min":1.21,"15min":1.212},"min":{"1min":0.99199999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6579999999999999,"5min":1.7170000000000001,"15min":1.7829999999999999},"last":1.3660000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1619999999999999,"15min":1.1830000000000001},"min":{"1min":0.98499999999999999,"5min":0.68100000000000005,"15min":0.67000000000000004},"max":{"1min":1.47,"5min":1.625,"15min":1.8939999999999999},"last":1.012},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.266,"15min":1.272},"min":{"1min":1.0660000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.556,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.3280000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.262,"15min":1.2729999999999999},"min":{"1min":0.93999999999999995,"5min":0.62,"15min":0.62},"max":{"1min":1.6990000000000001,"5min":2,"15min":2.137},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.2669999999999999,"15min":1.2689999999999999},"min":{"1min":1.0880000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.538,"5min":1.9350000000000001,"15min":2.0870000000000002},"last":1.1100000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.29,"15min":1.3109999999999999},"min":{"1min":0.95699999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7789999999999999,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.298,"15min":1.2989999999999999},"min":{"1min":1.101,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.7270000000000001,"5min":1.881,"15min":2.0350000000000001},"last":1.1990000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542322,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302984,"kb_used_data":1792,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068856,"statfs":{"total":96636764160,"available":96326508544,"internally_reserved":0,"allocated":1835008,"data_stored":929665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80600000000000005,"15min":0.83299999999999996},"min":{"1min":0.503,"5min":0.44400000000000001,"15min":0.36199999999999999},"max":{"1min":1.093,"5min":1.756,"15min":4.3630000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.90600000000000003,"15min":0.93700000000000006},"min":{"1min":0.61799999999999999,"5min":0.52600000000000002,"15min":0.40000000000000002},"max":{"1min":1.2490000000000001,"5min":1.6719999999999999,"15min":6.157},"last":0.44900000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.96399999999999997,"15min":0.97899999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.46800000000000003},"max":{"1min":1.3400000000000001,"5min":1.625,"15min":4.1699999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0029999999999999,"15min":1.0289999999999999},"min":{"1min":0.72199999999999998,"5min":0.60099999999999998,"15min":0.47799999999999998},"max":{"1min":1.379,"5min":1.879,"15min":6.8369999999999997},"last":0.48999999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.94099999999999995,"15min":0.95599999999999996},"min":{"1min":0.59499999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.3120000000000001,"5min":1.6519999999999999,"15min":3.5070000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":1.073,"5min":1.093,"15min":1.113},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56200000000000006},"max":{"1min":1.3460000000000001,"5min":2.0070000000000001,"15min":5.2149999999999999},"last":0.51800000000000002}]},{"osd":4,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.224,"15min":1.2609999999999999},"min":{"1min":0.81499999999999995,"5min":0.80500000000000005,"15min":0.66700000000000004},"max":{"1min":1.5900000000000001,"5min":1.944,"15min":6.2000000000000002},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.165,"5min":1.1870000000000001,"15min":1.196},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.3939999999999999,"5min":1.6779999999999999,"15min":3.577},"last":0.55800000000000005}]},{"osd":5,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.23,"15min":1.25},"min":{"1min":0.93000000000000005,"5min":0.81499999999999995,"15min":0.69599999999999995},"max":{"1min":1.478,"5min":1.857,"15min":3.548},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.21,"5min":1.278,"15min":1.2969999999999999},"min":{"1min":0.86199999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.4330000000000001,"5min":1.774,"15min":4.4080000000000004},"last":0.92400000000000004}]},{"osd":6,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3180000000000001,"15min":1.349},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.82299999999999995},"max":{"1min":1.5640000000000001,"5min":1.915,"15min":4.4489999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.236,"5min":1.2889999999999999,"15min":1.3340000000000001},"min":{"1min":0.93300000000000005,"5min":0.86499999999999999,"15min":0.75900000000000001},"max":{"1min":1.5409999999999999,"5min":2.056,"15min":6.9690000000000003},"last":0.72799999999999998}]},{"osd":7,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.327,"15min":1.401},"min":{"1min":1.036,"5min":0.90400000000000003,"15min":0.79900000000000004},"max":{"1min":1.6319999999999999,"5min":1.994,"15min":18.253},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2889999999999999,"15min":1.3380000000000001},"min":{"1min":0.997,"5min":0.95599999999999996,"15min":0.79000000000000004},"max":{"1min":1.452,"5min":1.8720000000000001,"15min":11.006},"last":0.82099999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575027,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303036,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068804,"statfs":{"total":96636764160,"available":96326455296,"internally_reserved":0,"allocated":1888256,"data_stored":957002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0800000000000001,"15min":1.073},"min":{"1min":0.65800000000000003,"5min":0.627,"15min":0.49399999999999999},"max":{"1min":1.827,"5min":1.827,"15min":2.3370000000000002},"last":1.411},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0589999999999999},"min":{"1min":0.63600000000000001,"5min":0.61099999999999999,"15min":0.60799999999999998},"max":{"1min":1.3580000000000001,"5min":1.6519999999999999,"15min":3.46},"last":0.79800000000000004}]},{"osd":1,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.105,"15min":1.121},"min":{"1min":0.84599999999999997,"5min":0.71199999999999997,"15min":0.58999999999999997},"max":{"1min":1.4570000000000001,"5min":1.516,"15min":3.6349999999999998},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.161,"15min":1.143},"min":{"1min":0.84199999999999997,"5min":0.73999999999999999,"15min":0.63700000000000001},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":3.516},"last":1.6499999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.1459999999999999,"15min":1.165},"min":{"1min":0.88600000000000001,"5min":0.74099999999999999,"15min":0.73199999999999998},"max":{"1min":1.774,"5min":1.774,"15min":5.4180000000000001},"last":1.119},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1899999999999999,"15min":1.204},"min":{"1min":0.77300000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6839999999999999,"5min":2.0840000000000001,"15min":3.5859999999999999},"last":1.8100000000000001}]},{"osd":3,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2070000000000001,"15min":1.196},"min":{"1min":0.91400000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6439999999999999,"5min":1.8560000000000001,"15min":2.141},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.296,"15min":1.3169999999999999},"min":{"1min":0.88400000000000001,"5min":0.875,"15min":0.78600000000000003},"max":{"1min":1.6140000000000001,"5min":1.6739999999999999,"15min":4.2729999999999997},"last":1.6060000000000001}]},{"osd":4,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.2130000000000001,"15min":1.26},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.4850000000000001,"5min":1.591,"15min":3.706},"last":1.992},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.258,"15min":1.3080000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.64100000000000001},"max":{"1min":1.863,"5min":1.9219999999999999,"15min":6.1180000000000003},"last":1.2490000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.268,"15min":1.254},"min":{"1min":0.88800000000000001,"5min":0.80700000000000005,"15min":0.749},"max":{"1min":1.6080000000000001,"5min":1.843,"15min":4.2309999999999999},"last":1.698},{"interface":"front","average":{"1min":1.258,"5min":1.2869999999999999,"15min":1.341},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.546,"5min":2.2240000000000002,"15min":3.641},"last":1.907}]},{"osd":7,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.284,"15min":1.2949999999999999},"min":{"1min":0.83499999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6970000000000001,"5min":1.988,"15min":3.516},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.379,"15min":1.387},"min":{"1min":0.95899999999999996,"5min":0.94299999999999995,"15min":0.89500000000000002},"max":{"1min":1.925,"5min":2.282,"15min":3.754},"last":1.8340000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575026,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303036,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068804,"statfs":{"total":96636764160,"available":96326455296,"internally_reserved":0,"allocated":1888256,"data_stored":951294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.80500000000000005,"15min":0.81000000000000005},"min":{"1min":0.63,"5min":0.504,"15min":0.48799999999999999},"max":{"1min":1.22,"5min":1.671,"15min":1.671},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.85599999999999998,"15min":0.879},"min":{"1min":0.52700000000000002,"5min":0.502,"15min":0.46600000000000003},"max":{"1min":1.1799999999999999,"5min":1.3939999999999999,"15min":1.6899999999999999},"last":0.49399999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97799999999999998,"15min":0.94099999999999995},"min":{"1min":0.755,"5min":0.60999999999999999,"15min":0.44600000000000001},"max":{"1min":1.306,"5min":1.806,"15min":1.806},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.872,"5min":0.89000000000000001,"15min":0.90500000000000003},"min":{"1min":0.63200000000000001,"5min":0.55800000000000005,"15min":0.49199999999999999},"max":{"1min":1.2509999999999999,"5min":1.3360000000000001,"15min":1.887},"last":0.746}]},{"osd":3,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.129,"15min":1.125},"min":{"1min":0.747,"5min":0.66300000000000003,"15min":0.42399999999999999},"max":{"1min":1.476,"5min":1.6160000000000001,"15min":1.712},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0549999999999999,"15min":1.042},"min":{"1min":0.68500000000000005,"5min":0.52300000000000002,"15min":0.46000000000000002},"max":{"1min":1.4450000000000001,"5min":1.5409999999999999,"15min":1.7090000000000001},"last":0.58199999999999996}]},{"osd":4,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.208,"15min":1.1930000000000001},"min":{"1min":0.83699999999999997,"5min":0.80600000000000005,"15min":0.65100000000000002},"max":{"1min":1.671,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.194,"5min":1.1970000000000001,"15min":1.212},"min":{"1min":0.97399999999999998,"5min":0.83799999999999997,"15min":0.50800000000000001},"max":{"1min":1.4710000000000001,"5min":1.5429999999999999,"15min":2.206},"last":1.1479999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2649999999999999,"15min":1.2470000000000001},"min":{"1min":0.98499999999999999,"5min":0.90300000000000002,"15min":0.746},"max":{"1min":1.4430000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.871},{"interface":"front","average":{"1min":1.248,"5min":1.2509999999999999,"15min":1.256},"min":{"1min":1.038,"5min":0.92300000000000004,"15min":0.56100000000000005},"max":{"1min":1.4019999999999999,"5min":1.718,"15min":2.4609999999999999},"last":1.0189999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.327,"15min":1.327},"min":{"1min":1.1020000000000001,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.671,"5min":1.8640000000000001,"15min":1.948},"last":1.28},{"interface":"front","average":{"1min":1.429,"5min":1.431,"15min":1.4079999999999999},"min":{"1min":1.2230000000000001,"5min":0.91600000000000004,"15min":0.87},"max":{"1min":1.786,"5min":1.9670000000000001,"15min":2.3290000000000002},"last":1.008}]},{"osd":7,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3149999999999999,"15min":1.3169999999999999},"min":{"1min":1.1000000000000001,"5min":0.92000000000000004,"15min":0.84199999999999997},"max":{"1min":1.595,"5min":2.089,"15min":2.161},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.3999999999999999,"15min":1.3779999999999999},"min":{"1min":1.095,"5min":1.0249999999999999,"15min":0.78600000000000003},"max":{"1min":1.899,"5min":2.137,"15min":2.2349999999999999},"last":1.171}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T03:11:26.123 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T03:11:26.321 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.312+0000 7f3aff59e700 1 Processor -- start 2023-11-15T03:11:26.322 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.312+0000 7f3aff59e700 1 -- start start 2023-11-15T03:11:26.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.316+0000 7f3aff59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3b000c3280 0x7f3b000c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:26.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.316+0000 7f3aff59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3b00152140 0x7f3b001525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:26.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.316+0000 7f3aff59e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b000c1d50 con 0x7f3b0015be30 2023-11-15T03:11:26.325 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.316+0000 7f3aff59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b000c1860 con 0x7f3b000c3280 2023-11-15T03:11:26.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.316+0000 7f3aff59e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b0015c990 con 0x7f3b00152140 2023-11-15T03:11:26.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.316+0000 7f3afe59c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3b000c3280 0x7f3b000c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:26.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.316+0000 7f3afed9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3b0015be30 0x7f3b0015c280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60860/0 (socket says 172.21.15.121:60860) 2023-11-15T03:11:26.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.316+0000 7f3afe59c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3b000c3280 0x7f3b000c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40484/0 (socket says 172.21.15.121:40484) 2023-11-15T03:11:26.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.316+0000 7f3afed9d700 1 -- 172.21.15.121:0/2350056265 learned_addr learned my addr 172.21.15.121:0/2350056265 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:26.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.316+0000 7f3afdd9b700 1 --2- 172.21.15.121:0/2350056265 >> v2:172.21.15.190:3300/0 conn(0x7f3b00152140 0x7f3b001525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:26.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.316+0000 7f3afe59c700 1 -- 172.21.15.121:0/2350056265 >> v2:172.21.15.190:3300/0 conn(0x7f3b00152140 msgr2=0x7f3b001525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:26.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.316+0000 7f3afe59c700 1 --2- 172.21.15.121:0/2350056265 >> v2:172.21.15.190:3300/0 conn(0x7f3b00152140 0x7f3b001525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:26.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.316+0000 7f3afe59c700 1 -- 172.21.15.121:0/2350056265 >> v1:172.21.15.121:6789/0 conn(0x7f3b0015be30 legacy=0x7f3b0015c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:26.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.320+0000 7f3afe59c700 1 -- 172.21.15.121:0/2350056265 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0015cb80 con 0x7f3b000c3280 2023-11-15T03:11:26.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.320+0000 7f3afd59a700 1 -- 172.21.15.121:0/2350056265 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2789648468 0 0) 0x7f3b000c1d50 con 0x7f3b0015be30 2023-11-15T03:11:26.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.320+0000 7f3afe59c700 1 --2- 172.21.15.121:0/2350056265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b000c3280 0x7f3b000c3660 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 rx=0x7f3b00159770 tx=0x7f3af0009060).ready entity=mon.0 client_cookie=7bf6dad127563c76 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:26.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.320+0000 7f3afd59a700 1 -- 172.21.15.121:0/2350056265 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af000f510 con 0x7f3b000c3280 2023-11-15T03:11:26.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.320+0000 7f3afd59a700 1 -- 172.21.15.121:0/2350056265 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af000f670 con 0x7f3b000c3280 2023-11-15T03:11:26.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.320+0000 7f3afd59a700 1 -- 172.21.15.121:0/2350056265 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af000f9f0 con 0x7f3b000c3280 2023-11-15T03:11:26.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.320+0000 7f3aff59e700 1 -- 172.21.15.121:0/2350056265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b000c3280 msgr2=0x7f3b000c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:26.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.320+0000 7f3aff59e700 1 --2- 172.21.15.121:0/2350056265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b000c3280 0x7f3b000c3660 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 rx=0x7f3b00159770 tx=0x7f3af0009060).stop 2023-11-15T03:11:26.331 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.320+0000 7f3aff59e700 1 -- 172.21.15.121:0/2350056265 shutdown_connections 2023-11-15T03:11:26.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.320+0000 7f3aff59e700 1 --2- 172.21.15.121:0/2350056265 >> v2:172.21.15.190:3300/0 conn(0x7f3b00152140 0x7f3b001525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:26.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.320+0000 7f3aff59e700 1 --2- 172.21.15.121:0/2350056265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b000c3280 0x7f3b000c3660 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:26.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.320+0000 7f3aff59e700 1 -- 172.21.15.121:0/2350056265 >> 172.21.15.121:0/2350056265 conn(0x7f3b00062190 msgr2=0x7f3b00062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:26.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.320+0000 7f3aff59e700 1 -- 172.21.15.121:0/2350056265 shutdown_connections 2023-11-15T03:11:26.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.320+0000 7f3aff59e700 1 -- 172.21.15.121:0/2350056265 wait complete. 2023-11-15T03:11:26.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.324+0000 7f3aff59e700 1 Processor -- start 2023-11-15T03:11:26.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.324+0000 7f3aff59e700 1 -- start start 2023-11-15T03:11:26.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3aff59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b000c3280 0x7f3b00144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:26.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3aff59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b00152140 0x7f3b00158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:26.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3aff59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b0015be30 0x7f3b00158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:26.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3aff59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b000c44e0 con 0x7f3b0015be30 2023-11-15T03:11:26.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3aff59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b00159120 con 0x7f3b000c3280 2023-11-15T03:11:26.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3aff59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b00159400 con 0x7f3b00152140 2023-11-15T03:11:26.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3afed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b0015be30 0x7f3b00158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:26.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3afed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b0015be30 0x7f3b00158b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40490/0 (socket says 172.21.15.121:40490) 2023-11-15T03:11:26.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3afed9d700 1 -- 172.21.15.121:0/5463488 learned_addr learned my addr 172.21.15.121:0/5463488 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:26.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3afe59c700 1 --2- 172.21.15.121:0/5463488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b000c3280 0x7f3b00144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:26.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3afdd9b700 1 --2- 172.21.15.121:0/5463488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b00152140 0x7f3b00158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:26.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3afed9d700 1 -- 172.21.15.121:0/5463488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b00152140 msgr2=0x7f3b00158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:26.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3afed9d700 1 --2- 172.21.15.121:0/5463488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b00152140 0x7f3b00158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:26.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3afed9d700 1 -- 172.21.15.121:0/5463488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b000c3280 msgr2=0x7f3b00144160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:26.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3afed9d700 1 --2- 172.21.15.121:0/5463488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b000c3280 0x7f3b00144160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:26.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3afed9d700 1 -- 172.21.15.121:0/5463488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af00092a0 con 0x7f3b0015be30 2023-11-15T03:11:26.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3afed9d700 1 --2- 172.21.15.121:0/5463488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b0015be30 0x7f3b00158b10 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 rx=0x7f3af800c7e0 tx=0x7f3af8007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:26.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3aee7fc700 1 -- 172.21.15.121:0/5463488 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8005d30 con 0x7f3b0015be30 2023-11-15T03:11:26.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3aff59e700 1 -- 172.21.15.121:0/5463488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b000c4730 con 0x7f3b0015be30 2023-11-15T03:11:26.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3aee7fc700 1 -- 172.21.15.121:0/5463488 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af8005e90 con 0x7f3b0015be30 2023-11-15T03:11:26.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3aee7fc700 1 -- 172.21.15.121:0/5463488 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8007320 con 0x7f3b0015be30 2023-11-15T03:11:26.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.328+0000 7f3aff59e700 1 -- 172.21.15.121:0/5463488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b00065fe0 con 0x7f3b0015be30 2023-11-15T03:11:26.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.332+0000 7f3aff59e700 1 -- 172.21.15.121:0/5463488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0006bcf0 con 0x7f3b0015be30 2023-11-15T03:11:26.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.332+0000 7f3aee7fc700 1 -- 172.21.15.121:0/5463488 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3af8007710 con 0x7f3b0015be30 2023-11-15T03:11:26.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.332+0000 7f3aee7fc700 1 --2- 172.21.15.121:0/5463488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ae005a510 0x7f3ae005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:26.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.332+0000 7f3aee7fc700 1 -- 172.21.15.121:0/5463488 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3af8074820 con 0x7f3b0015be30 2023-11-15T03:11:26.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.332+0000 7f3afe59c700 1 --2- 172.21.15.121:0/5463488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ae005a510 0x7f3ae005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:26.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.336+0000 7f3afe59c700 1 --2- 172.21.15.121:0/5463488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ae005a510 0x7f3ae005c9d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7f3b00159770 tx=0x7f3af0017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:26.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.336+0000 7f3aee7fc700 1 -- 172.21.15.121:0/5463488 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3af8047f60 con 0x7f3b0015be30 2023-11-15T03:11:26.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.496+0000 7f3aff59e700 1 -- 172.21.15.121:0/5463488 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3b00068a70 con 0x7f3ae005a510 2023-11-15T03:11:26.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.512+0000 7f3aee7fc700 1 -- 172.21.15.121:0/5463488 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+359354 (secure 0 0 0) 0x7f3b00068a70 con 0x7f3ae005a510 2023-11-15T03:11:26.522 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:26.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.520+0000 7f3aff59e700 1 -- 172.21.15.121:0/5463488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ae005a510 msgr2=0x7f3ae005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:26.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.520+0000 7f3aff59e700 1 --2- 172.21.15.121:0/5463488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ae005a510 0x7f3ae005c9d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7f3b00159770 tx=0x7f3af0017000).stop 2023-11-15T03:11:26.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.520+0000 7f3aff59e700 1 -- 172.21.15.121:0/5463488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b0015be30 msgr2=0x7f3b00158b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:26.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.520+0000 7f3aff59e700 1 --2- 172.21.15.121:0/5463488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b0015be30 0x7f3b00158b10 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 rx=0x7f3af800c7e0 tx=0x7f3af8007f90).stop 2023-11-15T03:11:26.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.520+0000 7f3aff59e700 1 -- 172.21.15.121:0/5463488 shutdown_connections 2023-11-15T03:11:26.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.520+0000 7f3aff59e700 1 --2- 172.21.15.121:0/5463488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3ae005a510 0x7f3ae005c9d0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:26.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.520+0000 7f3aff59e700 1 --2- 172.21.15.121:0/5463488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b0015be30 0x7f3b00158b10 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:26.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.520+0000 7f3aff59e700 1 --2- 172.21.15.121:0/5463488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b00152140 0x7f3b00158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:26.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.520+0000 7f3aff59e700 1 --2- 172.21.15.121:0/5463488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b000c3280 0x7f3b00144160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:26.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.520+0000 7f3aff59e700 1 -- 172.21.15.121:0/5463488 >> 172.21.15.121:0/5463488 conn(0x7f3b00062190 msgr2=0x7f3b00150170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:26.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.520+0000 7f3aff59e700 1 -- 172.21.15.121:0/5463488 shutdown_connections 2023-11-15T03:11:26.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.520+0000 7f3aff59e700 1 -- 172.21.15.121:0/5463488 wait complete. 2023-11-15T03:11:26.532 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T03:11:26.550 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":579,"stamp":"2023-11-15T03:11:26.258091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":232,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2444784,"kb_used_data":14768,"kb_used_omap":0,"kb_used_meta":2429952,"kb_avail":752529936,"statfs":{"total":773094113280,"available":770590654464,"internally_reserved":0,"allocated":15122432,"data_stored":7648223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2488270848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6307636,"num_objects":-8,"num_object_clones":0,"num_object_copies":-16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8,"num_whiteouts":0,"num_read":-960,"num_read_kb":-981,"num_write":-3272,"num_write_kb":-7425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.113900"},"pg_stats":[{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.901388+0000","last_change":"2023-11-15T03:11:21.895208+0000","last_active":"2023-11-15T03:11:21.901388+0000","last_peered":"2023-11-15T03:11:21.901388+0000","last_clean":"2023-11-15T03:11:21.901388+0000","last_became_active":"2023-11-15T03:11:21.893249+0000","last_became_peered":"2023-11-15T03:11:21.893249+0000","last_unstale":"2023-11-15T03:11:21.901388+0000","last_undegraded":"2023-11-15T03:11:21.901388+0000","last_fullsized":"2023-11-15T03:11:21.901388+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3f","version":"144'3","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.900035+0000","last_change":"2023-11-15T03:11:21.456072+0000","last_active":"2023-11-15T03:11:21.900035+0000","last_peered":"2023-11-15T03:11:21.900035+0000","last_clean":"2023-11-15T03:11:21.900035+0000","last_became_active":"2023-11-15T03:11:21.455710+0000","last_became_peered":"2023-11-15T03:11:21.455710+0000","last_unstale":"2023-11-15T03:11:21.900035+0000","last_undegraded":"2023-11-15T03:11:21.900035+0000","last_fullsized":"2023-11-15T03:11:21.900035+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.919270+0000","last_change":"2023-11-15T03:11:21.915077+0000","last_active":"2023-11-15T03:11:21.919270+0000","last_peered":"2023-11-15T03:11:21.919270+0000","last_clean":"2023-11-15T03:11:21.919270+0000","last_became_active":"2023-11-15T03:11:21.914807+0000","last_became_peered":"2023-11-15T03:11:21.914807+0000","last_unstale":"2023-11-15T03:11:21.919270+0000","last_undegraded":"2023-11-15T03:11:21.919270+0000","last_fullsized":"2023-11-15T03:11:21.919270+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.888681+0000","last_change":"2023-11-15T03:11:21.471469+0000","last_active":"2023-11-15T03:11:21.888681+0000","last_peered":"2023-11-15T03:11:21.888681+0000","last_clean":"2023-11-15T03:11:21.888681+0000","last_became_active":"2023-11-15T03:11:21.470723+0000","last_became_peered":"2023-11-15T03:11:21.470723+0000","last_unstale":"2023-11-15T03:11:21.888681+0000","last_undegraded":"2023-11-15T03:11:21.888681+0000","last_fullsized":"2023-11-15T03:11:21.888681+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.910065+0000","last_change":"2023-11-15T03:11:21.901741+0000","last_active":"2023-11-15T03:11:21.910065+0000","last_peered":"2023-11-15T03:11:21.910065+0000","last_clean":"2023-11-15T03:11:21.910065+0000","last_became_active":"2023-11-15T03:11:21.900846+0000","last_became_peered":"2023-11-15T03:11:21.900846+0000","last_unstale":"2023-11-15T03:11:21.910065+0000","last_undegraded":"2023-11-15T03:11:21.910065+0000","last_fullsized":"2023-11-15T03:11:21.910065+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.874464+0000","last_change":"2023-11-15T03:11:21.457108+0000","last_active":"2023-11-15T03:11:21.874464+0000","last_peered":"2023-11-15T03:11:21.874464+0000","last_clean":"2023-11-15T03:11:21.874464+0000","last_became_active":"2023-11-15T03:11:21.456137+0000","last_became_peered":"2023-11-15T03:11:21.456137+0000","last_unstale":"2023-11-15T03:11:21.874464+0000","last_undegraded":"2023-11-15T03:11:21.874464+0000","last_fullsized":"2023-11-15T03:11:21.874464+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.901628+0000","last_change":"2023-11-15T03:11:21.896331+0000","last_active":"2023-11-15T03:11:21.901628+0000","last_peered":"2023-11-15T03:11:21.901628+0000","last_clean":"2023-11-15T03:11:21.901628+0000","last_became_active":"2023-11-15T03:11:21.896077+0000","last_became_peered":"2023-11-15T03:11:21.896077+0000","last_unstale":"2023-11-15T03:11:21.901628+0000","last_undegraded":"2023-11-15T03:11:21.901628+0000","last_fullsized":"2023-11-15T03:11:21.901628+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3c","version":"144'2","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.909266+0000","last_change":"2023-11-15T03:11:21.457018+0000","last_active":"2023-11-15T03:11:21.909266+0000","last_peered":"2023-11-15T03:11:21.909266+0000","last_clean":"2023-11-15T03:11:21.909266+0000","last_became_active":"2023-11-15T03:11:21.456144+0000","last_became_peered":"2023-11-15T03:11:21.456144+0000","last_unstale":"2023-11-15T03:11:21.909266+0000","last_undegraded":"2023-11-15T03:11:21.909266+0000","last_fullsized":"2023-11-15T03:11:21.909266+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.917930+0000","last_change":"2023-11-15T03:11:21.912846+0000","last_active":"2023-11-15T03:11:21.917930+0000","last_peered":"2023-11-15T03:11:21.917930+0000","last_clean":"2023-11-15T03:11:21.917930+0000","last_became_active":"2023-11-15T03:11:21.912558+0000","last_became_peered":"2023-11-15T03:11:21.912558+0000","last_unstale":"2023-11-15T03:11:21.917930+0000","last_undegraded":"2023-11-15T03:11:21.917930+0000","last_fullsized":"2023-11-15T03:11:21.917930+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"144'1","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.917783+0000","last_change":"2023-11-15T03:11:21.469811+0000","last_active":"2023-11-15T03:11:21.917783+0000","last_peered":"2023-11-15T03:11:21.917783+0000","last_clean":"2023-11-15T03:11:21.917783+0000","last_became_active":"2023-11-15T03:11:21.469526+0000","last_became_peered":"2023-11-15T03:11:21.469526+0000","last_unstale":"2023-11-15T03:11:21.917783+0000","last_undegraded":"2023-11-15T03:11:21.917783+0000","last_fullsized":"2023-11-15T03:11:21.917783+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.912452+0000","last_change":"2023-11-15T03:11:21.906488+0000","last_active":"2023-11-15T03:11:21.912452+0000","last_peered":"2023-11-15T03:11:21.912452+0000","last_clean":"2023-11-15T03:11:21.912452+0000","last_became_active":"2023-11-15T03:11:21.906190+0000","last_became_peered":"2023-11-15T03:11:21.906190+0000","last_unstale":"2023-11-15T03:11:21.912452+0000","last_undegraded":"2023-11-15T03:11:21.912452+0000","last_fullsized":"2023-11-15T03:11:21.912452+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3a","version":"144'1","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.908948+0000","last_change":"2023-11-15T03:11:21.463702+0000","last_active":"2023-11-15T03:11:21.908948+0000","last_peered":"2023-11-15T03:11:21.908948+0000","last_clean":"2023-11-15T03:11:21.908948+0000","last_became_active":"2023-11-15T03:11:21.462701+0000","last_became_peered":"2023-11-15T03:11:21.462701+0000","last_unstale":"2023-11-15T03:11:21.908948+0000","last_undegraded":"2023-11-15T03:11:21.908948+0000","last_fullsized":"2023-11-15T03:11:21.908948+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.921891+0000","last_change":"2023-11-15T03:11:21.915328+0000","last_active":"2023-11-15T03:11:21.921891+0000","last_peered":"2023-11-15T03:11:21.921891+0000","last_clean":"2023-11-15T03:11:21.921891+0000","last_became_active":"2023-11-15T03:11:21.914833+0000","last_became_peered":"2023-11-15T03:11:21.914833+0000","last_unstale":"2023-11-15T03:11:21.921891+0000","last_undegraded":"2023-11-15T03:11:21.921891+0000","last_fullsized":"2023-11-15T03:11:21.921891+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.899184+0000","last_change":"2023-11-15T03:11:21.461594+0000","last_active":"2023-11-15T03:11:21.899184+0000","last_peered":"2023-11-15T03:11:21.899184+0000","last_clean":"2023-11-15T03:11:21.899184+0000","last_became_active":"2023-11-15T03:11:21.461176+0000","last_became_peered":"2023-11-15T03:11:21.461176+0000","last_unstale":"2023-11-15T03:11:21.899184+0000","last_undegraded":"2023-11-15T03:11:21.899184+0000","last_fullsized":"2023-11-15T03:11:21.899184+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.910096+0000","last_change":"2023-11-15T03:11:21.901548+0000","last_active":"2023-11-15T03:11:21.910096+0000","last_peered":"2023-11-15T03:11:21.910096+0000","last_clean":"2023-11-15T03:11:21.910096+0000","last_became_active":"2023-11-15T03:11:21.901100+0000","last_became_peered":"2023-11-15T03:11:21.901100+0000","last_unstale":"2023-11-15T03:11:21.910096+0000","last_undegraded":"2023-11-15T03:11:21.910096+0000","last_fullsized":"2023-11-15T03:11:21.910096+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.890943+0000","last_change":"2023-11-15T03:11:21.459639+0000","last_active":"2023-11-15T03:11:21.890943+0000","last_peered":"2023-11-15T03:11:21.890943+0000","last_clean":"2023-11-15T03:11:21.890943+0000","last_became_active":"2023-11-15T03:11:21.459318+0000","last_became_peered":"2023-11-15T03:11:21.459318+0000","last_unstale":"2023-11-15T03:11:21.890943+0000","last_undegraded":"2023-11-15T03:11:21.890943+0000","last_fullsized":"2023-11-15T03:11:21.890943+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.917672+0000","last_change":"2023-11-15T03:11:21.912731+0000","last_active":"2023-11-15T03:11:21.917672+0000","last_peered":"2023-11-15T03:11:21.917672+0000","last_clean":"2023-11-15T03:11:21.917672+0000","last_became_active":"2023-11-15T03:11:21.912460+0000","last_became_peered":"2023-11-15T03:11:21.912460+0000","last_unstale":"2023-11-15T03:11:21.917672+0000","last_undegraded":"2023-11-15T03:11:21.917672+0000","last_fullsized":"2023-11-15T03:11:21.917672+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.889503+0000","last_change":"2023-11-15T03:11:21.468775+0000","last_active":"2023-11-15T03:11:21.889503+0000","last_peered":"2023-11-15T03:11:21.889503+0000","last_clean":"2023-11-15T03:11:21.889503+0000","last_became_active":"2023-11-15T03:11:21.467674+0000","last_became_peered":"2023-11-15T03:11:21.467674+0000","last_unstale":"2023-11-15T03:11:21.889503+0000","last_undegraded":"2023-11-15T03:11:21.889503+0000","last_fullsized":"2023-11-15T03:11:21.889503+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.911358+0000","last_change":"2023-11-15T03:11:21.904134+0000","last_active":"2023-11-15T03:11:21.911358+0000","last_peered":"2023-11-15T03:11:21.911358+0000","last_clean":"2023-11-15T03:11:21.911358+0000","last_became_active":"2023-11-15T03:11:21.903830+0000","last_became_peered":"2023-11-15T03:11:21.903830+0000","last_unstale":"2023-11-15T03:11:21.911358+0000","last_undegraded":"2023-11-15T03:11:21.911358+0000","last_fullsized":"2023-11-15T03:11:21.911358+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.907269+0000","last_change":"2023-11-15T03:11:21.472960+0000","last_active":"2023-11-15T03:11:21.907269+0000","last_peered":"2023-11-15T03:11:21.907269+0000","last_clean":"2023-11-15T03:11:21.907269+0000","last_became_active":"2023-11-15T03:11:21.472701+0000","last_became_peered":"2023-11-15T03:11:21.472701+0000","last_unstale":"2023-11-15T03:11:21.907269+0000","last_undegraded":"2023-11-15T03:11:21.907269+0000","last_fullsized":"2023-11-15T03:11:21.907269+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.902421+0000","last_change":"2023-11-15T03:11:21.893136+0000","last_active":"2023-11-15T03:11:21.902421+0000","last_peered":"2023-11-15T03:11:21.902421+0000","last_clean":"2023-11-15T03:11:21.902421+0000","last_became_active":"2023-11-15T03:11:21.892853+0000","last_became_peered":"2023-11-15T03:11:21.892853+0000","last_unstale":"2023-11-15T03:11:21.902421+0000","last_undegraded":"2023-11-15T03:11:21.902421+0000","last_fullsized":"2023-11-15T03:11:21.902421+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.898704+0000","last_change":"2023-11-15T03:11:21.460901+0000","last_active":"2023-11-15T03:11:21.898704+0000","last_peered":"2023-11-15T03:11:21.898704+0000","last_clean":"2023-11-15T03:11:21.898704+0000","last_became_active":"2023-11-15T03:11:21.460575+0000","last_became_peered":"2023-11-15T03:11:21.460575+0000","last_unstale":"2023-11-15T03:11:21.898704+0000","last_undegraded":"2023-11-15T03:11:21.898704+0000","last_fullsized":"2023-11-15T03:11:21.898704+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.901610+0000","last_change":"2023-11-15T03:11:21.895788+0000","last_active":"2023-11-15T03:11:21.901610+0000","last_peered":"2023-11-15T03:11:21.901610+0000","last_clean":"2023-11-15T03:11:21.901610+0000","last_became_active":"2023-11-15T03:11:21.895524+0000","last_became_peered":"2023-11-15T03:11:21.895524+0000","last_unstale":"2023-11-15T03:11:21.901610+0000","last_undegraded":"2023-11-15T03:11:21.901610+0000","last_fullsized":"2023-11-15T03:11:21.901610+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.34","version":"144'4","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.925757+0000","last_change":"2023-11-15T03:11:21.473161+0000","last_active":"2023-11-15T03:11:21.925757+0000","last_peered":"2023-11-15T03:11:21.925757+0000","last_clean":"2023-11-15T03:11:21.925757+0000","last_became_active":"2023-11-15T03:11:21.472783+0000","last_became_peered":"2023-11-15T03:11:21.472783+0000","last_unstale":"2023-11-15T03:11:21.925757+0000","last_undegraded":"2023-11-15T03:11:21.925757+0000","last_fullsized":"2023-11-15T03:11:21.925757+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.913762+0000","last_change":"2023-11-15T03:11:21.906187+0000","last_active":"2023-11-15T03:11:21.913762+0000","last_peered":"2023-11-15T03:11:21.913762+0000","last_clean":"2023-11-15T03:11:21.913762+0000","last_became_active":"2023-11-15T03:11:21.905917+0000","last_became_peered":"2023-11-15T03:11:21.905917+0000","last_unstale":"2023-11-15T03:11:21.913762+0000","last_undegraded":"2023-11-15T03:11:21.913762+0000","last_fullsized":"2023-11-15T03:11:21.913762+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"144'1","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.907034+0000","last_change":"2023-11-15T03:11:21.445986+0000","last_active":"2023-11-15T03:11:21.907034+0000","last_peered":"2023-11-15T03:11:21.907034+0000","last_clean":"2023-11-15T03:11:21.907034+0000","last_became_active":"2023-11-15T03:11:21.445394+0000","last_became_peered":"2023-11-15T03:11:21.445394+0000","last_unstale":"2023-11-15T03:11:21.907034+0000","last_undegraded":"2023-11-15T03:11:21.907034+0000","last_fullsized":"2023-11-15T03:11:21.907034+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.915266+0000","last_change":"2023-11-15T03:11:21.910179+0000","last_active":"2023-11-15T03:11:21.915266+0000","last_peered":"2023-11-15T03:11:21.915266+0000","last_clean":"2023-11-15T03:11:21.915266+0000","last_became_active":"2023-11-15T03:11:21.908569+0000","last_became_peered":"2023-11-15T03:11:21.908569+0000","last_unstale":"2023-11-15T03:11:21.915266+0000","last_undegraded":"2023-11-15T03:11:21.915266+0000","last_fullsized":"2023-11-15T03:11:21.915266+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.887061+0000","last_change":"2023-11-15T03:11:21.455181+0000","last_active":"2023-11-15T03:11:21.887061+0000","last_peered":"2023-11-15T03:11:21.887061+0000","last_clean":"2023-11-15T03:11:21.887061+0000","last_became_active":"2023-11-15T03:11:21.454906+0000","last_became_peered":"2023-11-15T03:11:21.454906+0000","last_unstale":"2023-11-15T03:11:21.887061+0000","last_undegraded":"2023-11-15T03:11:21.887061+0000","last_fullsized":"2023-11-15T03:11:21.887061+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.917783+0000","last_change":"2023-11-15T03:11:21.911887+0000","last_active":"2023-11-15T03:11:21.917783+0000","last_peered":"2023-11-15T03:11:21.917783+0000","last_clean":"2023-11-15T03:11:21.917783+0000","last_became_active":"2023-11-15T03:11:21.910906+0000","last_became_peered":"2023-11-15T03:11:21.910906+0000","last_unstale":"2023-11-15T03:11:21.917783+0000","last_undegraded":"2023-11-15T03:11:21.917783+0000","last_fullsized":"2023-11-15T03:11:21.917783+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.892607+0000","last_change":"2023-11-15T03:11:21.457826+0000","last_active":"2023-11-15T03:11:21.892607+0000","last_peered":"2023-11-15T03:11:21.892607+0000","last_clean":"2023-11-15T03:11:21.892607+0000","last_became_active":"2023-11-15T03:11:21.457584+0000","last_became_peered":"2023-11-15T03:11:21.457584+0000","last_unstale":"2023-11-15T03:11:21.892607+0000","last_undegraded":"2023-11-15T03:11:21.892607+0000","last_fullsized":"2023-11-15T03:11:21.892607+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.911402+0000","last_change":"2023-11-15T03:11:21.904056+0000","last_active":"2023-11-15T03:11:21.911402+0000","last_peered":"2023-11-15T03:11:21.911402+0000","last_clean":"2023-11-15T03:11:21.911402+0000","last_became_active":"2023-11-15T03:11:21.903785+0000","last_became_peered":"2023-11-15T03:11:21.903785+0000","last_unstale":"2023-11-15T03:11:21.911402+0000","last_undegraded":"2023-11-15T03:11:21.911402+0000","last_fullsized":"2023-11-15T03:11:21.911402+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.30","version":"144'4","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.913147+0000","last_change":"2023-11-15T03:11:21.478399+0000","last_active":"2023-11-15T03:11:21.913147+0000","last_peered":"2023-11-15T03:11:21.913147+0000","last_clean":"2023-11-15T03:11:21.913147+0000","last_became_active":"2023-11-15T03:11:21.477986+0000","last_became_peered":"2023-11-15T03:11:21.477986+0000","last_unstale":"2023-11-15T03:11:21.913147+0000","last_undegraded":"2023-11-15T03:11:21.913147+0000","last_fullsized":"2023-11-15T03:11:21.913147+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.908695+0000","last_change":"2023-11-15T03:11:21.904064+0000","last_active":"2023-11-15T03:11:21.908695+0000","last_peered":"2023-11-15T03:11:21.908695+0000","last_clean":"2023-11-15T03:11:21.908695+0000","last_became_active":"2023-11-15T03:11:21.903811+0000","last_became_peered":"2023-11-15T03:11:21.903811+0000","last_unstale":"2023-11-15T03:11:21.908695+0000","last_undegraded":"2023-11-15T03:11:21.908695+0000","last_fullsized":"2023-11-15T03:11:21.908695+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.888598+0000","last_change":"2023-11-15T03:11:21.461864+0000","last_active":"2023-11-15T03:11:21.888598+0000","last_peered":"2023-11-15T03:11:21.888598+0000","last_clean":"2023-11-15T03:11:21.888598+0000","last_became_active":"2023-11-15T03:11:21.461551+0000","last_became_peered":"2023-11-15T03:11:21.461551+0000","last_unstale":"2023-11-15T03:11:21.888598+0000","last_undegraded":"2023-11-15T03:11:21.888598+0000","last_fullsized":"2023-11-15T03:11:21.888598+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.916573+0000","last_change":"2023-11-15T03:11:21.910953+0000","last_active":"2023-11-15T03:11:21.916573+0000","last_peered":"2023-11-15T03:11:21.916573+0000","last_clean":"2023-11-15T03:11:21.916573+0000","last_became_active":"2023-11-15T03:11:21.910679+0000","last_became_peered":"2023-11-15T03:11:21.910679+0000","last_unstale":"2023-11-15T03:11:21.916573+0000","last_undegraded":"2023-11-15T03:11:21.916573+0000","last_fullsized":"2023-11-15T03:11:21.916573+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.897321+0000","last_change":"2023-11-15T03:11:21.453205+0000","last_active":"2023-11-15T03:11:21.897321+0000","last_peered":"2023-11-15T03:11:21.897321+0000","last_clean":"2023-11-15T03:11:21.897321+0000","last_became_active":"2023-11-15T03:11:21.452750+0000","last_became_peered":"2023-11-15T03:11:21.452750+0000","last_unstale":"2023-11-15T03:11:21.897321+0000","last_undegraded":"2023-11-15T03:11:21.897321+0000","last_fullsized":"2023-11-15T03:11:21.897321+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.917188+0000","last_change":"2023-11-15T03:11:21.914591+0000","last_active":"2023-11-15T03:11:21.917188+0000","last_peered":"2023-11-15T03:11:21.917188+0000","last_clean":"2023-11-15T03:11:21.917188+0000","last_became_active":"2023-11-15T03:11:21.914304+0000","last_became_peered":"2023-11-15T03:11:21.914304+0000","last_unstale":"2023-11-15T03:11:21.917188+0000","last_undegraded":"2023-11-15T03:11:21.917188+0000","last_fullsized":"2023-11-15T03:11:21.917188+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2d","version":"144'1","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.904307+0000","last_change":"2023-11-15T03:11:21.461242+0000","last_active":"2023-11-15T03:11:21.904307+0000","last_peered":"2023-11-15T03:11:21.904307+0000","last_clean":"2023-11-15T03:11:21.904307+0000","last_became_active":"2023-11-15T03:11:21.460978+0000","last_became_peered":"2023-11-15T03:11:21.460978+0000","last_unstale":"2023-11-15T03:11:21.904307+0000","last_undegraded":"2023-11-15T03:11:21.904307+0000","last_fullsized":"2023-11-15T03:11:21.904307+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.908845+0000","last_change":"2023-11-15T03:11:21.899207+0000","last_active":"2023-11-15T03:11:21.908845+0000","last_peered":"2023-11-15T03:11:21.908845+0000","last_clean":"2023-11-15T03:11:21.908845+0000","last_became_active":"2023-11-15T03:11:21.898943+0000","last_became_peered":"2023-11-15T03:11:21.898943+0000","last_unstale":"2023-11-15T03:11:21.908845+0000","last_undegraded":"2023-11-15T03:11:21.908845+0000","last_fullsized":"2023-11-15T03:11:21.908845+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.889123+0000","last_change":"2023-11-15T03:11:21.469224+0000","last_active":"2023-11-15T03:11:21.889123+0000","last_peered":"2023-11-15T03:11:21.889123+0000","last_clean":"2023-11-15T03:11:21.889123+0000","last_became_active":"2023-11-15T03:11:21.468940+0000","last_became_peered":"2023-11-15T03:11:21.468940+0000","last_unstale":"2023-11-15T03:11:21.889123+0000","last_undegraded":"2023-11-15T03:11:21.889123+0000","last_fullsized":"2023-11-15T03:11:21.889123+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.913781+0000","last_change":"2023-11-15T03:11:21.908406+0000","last_active":"2023-11-15T03:11:21.913781+0000","last_peered":"2023-11-15T03:11:21.913781+0000","last_clean":"2023-11-15T03:11:21.913781+0000","last_became_active":"2023-11-15T03:11:21.908081+0000","last_became_peered":"2023-11-15T03:11:21.908081+0000","last_unstale":"2023-11-15T03:11:21.913781+0000","last_undegraded":"2023-11-15T03:11:21.913781+0000","last_fullsized":"2023-11-15T03:11:21.913781+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"144'1","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.895233+0000","last_change":"2023-11-15T03:11:21.455231+0000","last_active":"2023-11-15T03:11:21.895233+0000","last_peered":"2023-11-15T03:11:21.895233+0000","last_clean":"2023-11-15T03:11:21.895233+0000","last_became_active":"2023-11-15T03:11:21.454946+0000","last_became_peered":"2023-11-15T03:11:21.454946+0000","last_unstale":"2023-11-15T03:11:21.895233+0000","last_undegraded":"2023-11-15T03:11:21.895233+0000","last_fullsized":"2023-11-15T03:11:21.895233+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.917167+0000","last_change":"2023-11-15T03:11:21.913176+0000","last_active":"2023-11-15T03:11:21.917167+0000","last_peered":"2023-11-15T03:11:21.917167+0000","last_clean":"2023-11-15T03:11:21.917167+0000","last_became_active":"2023-11-15T03:11:21.912877+0000","last_became_peered":"2023-11-15T03:11:21.912877+0000","last_unstale":"2023-11-15T03:11:21.917167+0000","last_undegraded":"2023-11-15T03:11:21.917167+0000","last_fullsized":"2023-11-15T03:11:21.917167+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.888440+0000","last_change":"2023-11-15T03:11:21.468401+0000","last_active":"2023-11-15T03:11:21.888440+0000","last_peered":"2023-11-15T03:11:21.888440+0000","last_clean":"2023-11-15T03:11:21.888440+0000","last_became_active":"2023-11-15T03:11:21.466116+0000","last_became_peered":"2023-11-15T03:11:21.466116+0000","last_unstale":"2023-11-15T03:11:21.888440+0000","last_undegraded":"2023-11-15T03:11:21.888440+0000","last_fullsized":"2023-11-15T03:11:21.888440+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.917703+0000","last_change":"2023-11-15T03:11:21.912892+0000","last_active":"2023-11-15T03:11:21.917703+0000","last_peered":"2023-11-15T03:11:21.917703+0000","last_clean":"2023-11-15T03:11:21.917703+0000","last_became_active":"2023-11-15T03:11:21.912547+0000","last_became_peered":"2023-11-15T03:11:21.912547+0000","last_unstale":"2023-11-15T03:11:21.917703+0000","last_undegraded":"2023-11-15T03:11:21.917703+0000","last_fullsized":"2023-11-15T03:11:21.917703+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.890303+0000","last_change":"2023-11-15T03:11:21.462521+0000","last_active":"2023-11-15T03:11:21.890303+0000","last_peered":"2023-11-15T03:11:21.890303+0000","last_clean":"2023-11-15T03:11:21.890303+0000","last_became_active":"2023-11-15T03:11:21.462223+0000","last_became_peered":"2023-11-15T03:11:21.462223+0000","last_unstale":"2023-11-15T03:11:21.890303+0000","last_undegraded":"2023-11-15T03:11:21.890303+0000","last_fullsized":"2023-11-15T03:11:21.890303+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.921750+0000","last_change":"2023-11-15T03:11:21.916621+0000","last_active":"2023-11-15T03:11:21.921750+0000","last_peered":"2023-11-15T03:11:21.921750+0000","last_clean":"2023-11-15T03:11:21.921750+0000","last_became_active":"2023-11-15T03:11:21.916334+0000","last_became_peered":"2023-11-15T03:11:21.916334+0000","last_unstale":"2023-11-15T03:11:21.921750+0000","last_undegraded":"2023-11-15T03:11:21.921750+0000","last_fullsized":"2023-11-15T03:11:21.921750+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.897049+0000","last_change":"2023-11-15T03:11:21.469528+0000","last_active":"2023-11-15T03:11:21.897049+0000","last_peered":"2023-11-15T03:11:21.897049+0000","last_clean":"2023-11-15T03:11:21.897049+0000","last_became_active":"2023-11-15T03:11:21.469220+0000","last_became_peered":"2023-11-15T03:11:21.469220+0000","last_unstale":"2023-11-15T03:11:21.897049+0000","last_undegraded":"2023-11-15T03:11:21.897049+0000","last_fullsized":"2023-11-15T03:11:21.897049+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.908903+0000","last_change":"2023-11-15T03:11:21.902656+0000","last_active":"2023-11-15T03:11:21.908903+0000","last_peered":"2023-11-15T03:11:21.908903+0000","last_clean":"2023-11-15T03:11:21.908903+0000","last_became_active":"2023-11-15T03:11:21.902334+0000","last_became_peered":"2023-11-15T03:11:21.902334+0000","last_unstale":"2023-11-15T03:11:21.908903+0000","last_undegraded":"2023-11-15T03:11:21.908903+0000","last_fullsized":"2023-11-15T03:11:21.908903+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.907648+0000","last_change":"2023-11-15T03:11:21.477286+0000","last_active":"2023-11-15T03:11:21.907648+0000","last_peered":"2023-11-15T03:11:21.907648+0000","last_clean":"2023-11-15T03:11:21.907648+0000","last_became_active":"2023-11-15T03:11:21.476941+0000","last_became_peered":"2023-11-15T03:11:21.476941+0000","last_unstale":"2023-11-15T03:11:21.907648+0000","last_undegraded":"2023-11-15T03:11:21.907648+0000","last_fullsized":"2023-11-15T03:11:21.907648+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.917829+0000","last_change":"2023-11-15T03:11:21.912183+0000","last_active":"2023-11-15T03:11:21.917829+0000","last_peered":"2023-11-15T03:11:21.917829+0000","last_clean":"2023-11-15T03:11:21.917829+0000","last_became_active":"2023-11-15T03:11:21.910991+0000","last_became_peered":"2023-11-15T03:11:21.910991+0000","last_unstale":"2023-11-15T03:11:21.917829+0000","last_undegraded":"2023-11-15T03:11:21.917829+0000","last_fullsized":"2023-11-15T03:11:21.917829+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.888854+0000","last_change":"2023-11-15T03:11:21.471240+0000","last_active":"2023-11-15T03:11:21.888854+0000","last_peered":"2023-11-15T03:11:21.888854+0000","last_clean":"2023-11-15T03:11:21.888854+0000","last_became_active":"2023-11-15T03:11:21.469051+0000","last_became_peered":"2023-11-15T03:11:21.469051+0000","last_unstale":"2023-11-15T03:11:21.888854+0000","last_undegraded":"2023-11-15T03:11:21.888854+0000","last_fullsized":"2023-11-15T03:11:21.888854+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.917041+0000","last_change":"2023-11-15T03:11:21.911165+0000","last_active":"2023-11-15T03:11:21.917041+0000","last_peered":"2023-11-15T03:11:21.917041+0000","last_clean":"2023-11-15T03:11:21.917041+0000","last_became_active":"2023-11-15T03:11:21.910850+0000","last_became_peered":"2023-11-15T03:11:21.910850+0000","last_unstale":"2023-11-15T03:11:21.917041+0000","last_undegraded":"2023-11-15T03:11:21.917041+0000","last_fullsized":"2023-11-15T03:11:21.917041+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.898804+0000","last_change":"2023-11-15T03:11:21.472959+0000","last_active":"2023-11-15T03:11:21.898804+0000","last_peered":"2023-11-15T03:11:21.898804+0000","last_clean":"2023-11-15T03:11:21.898804+0000","last_became_active":"2023-11-15T03:11:21.472624+0000","last_became_peered":"2023-11-15T03:11:21.472624+0000","last_unstale":"2023-11-15T03:11:21.898804+0000","last_undegraded":"2023-11-15T03:11:21.898804+0000","last_fullsized":"2023-11-15T03:11:21.898804+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.902563+0000","last_change":"2023-11-15T03:11:21.892450+0000","last_active":"2023-11-15T03:11:21.902563+0000","last_peered":"2023-11-15T03:11:21.902563+0000","last_clean":"2023-11-15T03:11:21.902563+0000","last_became_active":"2023-11-15T03:11:21.892196+0000","last_became_peered":"2023-11-15T03:11:21.892196+0000","last_unstale":"2023-11-15T03:11:21.902563+0000","last_undegraded":"2023-11-15T03:11:21.902563+0000","last_fullsized":"2023-11-15T03:11:21.902563+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.903708+0000","last_change":"2023-11-15T03:11:21.468773+0000","last_active":"2023-11-15T03:11:21.903708+0000","last_peered":"2023-11-15T03:11:21.903708+0000","last_clean":"2023-11-15T03:11:21.903708+0000","last_became_active":"2023-11-15T03:11:21.468466+0000","last_became_peered":"2023-11-15T03:11:21.468466+0000","last_unstale":"2023-11-15T03:11:21.903708+0000","last_undegraded":"2023-11-15T03:11:21.903708+0000","last_fullsized":"2023-11-15T03:11:21.903708+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.909310+0000","last_change":"2023-11-15T03:11:21.901542+0000","last_active":"2023-11-15T03:11:21.909310+0000","last_peered":"2023-11-15T03:11:21.909310+0000","last_clean":"2023-11-15T03:11:21.909310+0000","last_became_active":"2023-11-15T03:11:21.901249+0000","last_became_peered":"2023-11-15T03:11:21.901249+0000","last_unstale":"2023-11-15T03:11:21.909310+0000","last_undegraded":"2023-11-15T03:11:21.909310+0000","last_fullsized":"2023-11-15T03:11:21.909310+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.888676+0000","last_change":"2023-11-15T03:11:21.469545+0000","last_active":"2023-11-15T03:11:21.888676+0000","last_peered":"2023-11-15T03:11:21.888676+0000","last_clean":"2023-11-15T03:11:21.888676+0000","last_became_active":"2023-11-15T03:11:21.469226+0000","last_became_peered":"2023-11-15T03:11:21.469226+0000","last_unstale":"2023-11-15T03:11:21.888676+0000","last_undegraded":"2023-11-15T03:11:21.888676+0000","last_fullsized":"2023-11-15T03:11:21.888676+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.914246+0000","last_change":"2023-11-15T03:11:21.906660+0000","last_active":"2023-11-15T03:11:21.914246+0000","last_peered":"2023-11-15T03:11:21.914246+0000","last_clean":"2023-11-15T03:11:21.914246+0000","last_became_active":"2023-11-15T03:11:21.905626+0000","last_became_peered":"2023-11-15T03:11:21.905626+0000","last_unstale":"2023-11-15T03:11:21.914246+0000","last_undegraded":"2023-11-15T03:11:21.914246+0000","last_fullsized":"2023-11-15T03:11:21.914246+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.887100+0000","last_change":"2023-11-15T03:11:21.449801+0000","last_active":"2023-11-15T03:11:21.887100+0000","last_peered":"2023-11-15T03:11:21.887100+0000","last_clean":"2023-11-15T03:11:21.887100+0000","last_became_active":"2023-11-15T03:11:21.449284+0000","last_became_peered":"2023-11-15T03:11:21.449284+0000","last_unstale":"2023-11-15T03:11:21.887100+0000","last_undegraded":"2023-11-15T03:11:21.887100+0000","last_fullsized":"2023-11-15T03:11:21.887100+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.921711+0000","last_change":"2023-11-15T03:11:21.916581+0000","last_active":"2023-11-15T03:11:21.921711+0000","last_peered":"2023-11-15T03:11:21.921711+0000","last_clean":"2023-11-15T03:11:21.921711+0000","last_became_active":"2023-11-15T03:11:21.916311+0000","last_became_peered":"2023-11-15T03:11:21.916311+0000","last_unstale":"2023-11-15T03:11:21.921711+0000","last_undegraded":"2023-11-15T03:11:21.921711+0000","last_fullsized":"2023-11-15T03:11:21.921711+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.892703+0000","last_change":"2023-11-15T03:11:21.457332+0000","last_active":"2023-11-15T03:11:21.892703+0000","last_peered":"2023-11-15T03:11:21.892703+0000","last_clean":"2023-11-15T03:11:21.892703+0000","last_became_active":"2023-11-15T03:11:21.456465+0000","last_became_peered":"2023-11-15T03:11:21.456465+0000","last_unstale":"2023-11-15T03:11:21.892703+0000","last_undegraded":"2023-11-15T03:11:21.892703+0000","last_fullsized":"2023-11-15T03:11:21.892703+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.914621+0000","last_change":"2023-11-15T03:11:21.909679+0000","last_active":"2023-11-15T03:11:21.914621+0000","last_peered":"2023-11-15T03:11:21.914621+0000","last_clean":"2023-11-15T03:11:21.914621+0000","last_became_active":"2023-11-15T03:11:21.909358+0000","last_became_peered":"2023-11-15T03:11:21.909358+0000","last_unstale":"2023-11-15T03:11:21.914621+0000","last_undegraded":"2023-11-15T03:11:21.914621+0000","last_fullsized":"2023-11-15T03:11:21.914621+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.873269+0000","last_change":"2023-11-15T03:11:21.472198+0000","last_active":"2023-11-15T03:11:21.873269+0000","last_peered":"2023-11-15T03:11:21.873269+0000","last_clean":"2023-11-15T03:11:21.873269+0000","last_became_active":"2023-11-15T03:11:21.471631+0000","last_became_peered":"2023-11-15T03:11:21.471631+0000","last_unstale":"2023-11-15T03:11:21.873269+0000","last_undegraded":"2023-11-15T03:11:21.873269+0000","last_fullsized":"2023-11-15T03:11:21.873269+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.915230+0000","last_change":"2023-11-15T03:11:21.907475+0000","last_active":"2023-11-15T03:11:21.915230+0000","last_peered":"2023-11-15T03:11:21.915230+0000","last_clean":"2023-11-15T03:11:21.915230+0000","last_became_active":"2023-11-15T03:11:21.906365+0000","last_became_peered":"2023-11-15T03:11:21.906365+0000","last_unstale":"2023-11-15T03:11:21.915230+0000","last_undegraded":"2023-11-15T03:11:21.915230+0000","last_fullsized":"2023-11-15T03:11:21.915230+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"144'1","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.908541+0000","last_change":"2023-11-15T03:11:21.460396+0000","last_active":"2023-11-15T03:11:21.908541+0000","last_peered":"2023-11-15T03:11:21.908541+0000","last_clean":"2023-11-15T03:11:21.908541+0000","last_became_active":"2023-11-15T03:11:21.460118+0000","last_became_peered":"2023-11-15T03:11:21.460118+0000","last_unstale":"2023-11-15T03:11:21.908541+0000","last_undegraded":"2023-11-15T03:11:21.908541+0000","last_fullsized":"2023-11-15T03:11:21.908541+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.909067+0000","last_change":"2023-11-15T03:11:21.902700+0000","last_active":"2023-11-15T03:11:21.909067+0000","last_peered":"2023-11-15T03:11:21.909067+0000","last_clean":"2023-11-15T03:11:21.909067+0000","last_became_active":"2023-11-15T03:11:21.902346+0000","last_became_peered":"2023-11-15T03:11:21.902346+0000","last_unstale":"2023-11-15T03:11:21.909067+0000","last_undegraded":"2023-11-15T03:11:21.909067+0000","last_fullsized":"2023-11-15T03:11:21.909067+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"144'1","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.888813+0000","last_change":"2023-11-15T03:11:21.448695+0000","last_active":"2023-11-15T03:11:21.888813+0000","last_peered":"2023-11-15T03:11:21.888813+0000","last_clean":"2023-11-15T03:11:21.888813+0000","last_became_active":"2023-11-15T03:11:21.448367+0000","last_became_peered":"2023-11-15T03:11:21.448367+0000","last_unstale":"2023-11-15T03:11:21.888813+0000","last_undegraded":"2023-11-15T03:11:21.888813+0000","last_fullsized":"2023-11-15T03:11:21.888813+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.918799+0000","last_change":"2023-11-15T03:11:21.910408+0000","last_active":"2023-11-15T03:11:21.918799+0000","last_peered":"2023-11-15T03:11:21.918799+0000","last_clean":"2023-11-15T03:11:21.918799+0000","last_became_active":"2023-11-15T03:11:21.910104+0000","last_became_peered":"2023-11-15T03:11:21.910104+0000","last_unstale":"2023-11-15T03:11:21.918799+0000","last_undegraded":"2023-11-15T03:11:21.918799+0000","last_fullsized":"2023-11-15T03:11:21.918799+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"144'2","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.901781+0000","last_change":"2023-11-15T03:11:21.458844+0000","last_active":"2023-11-15T03:11:21.901781+0000","last_peered":"2023-11-15T03:11:21.901781+0000","last_clean":"2023-11-15T03:11:21.901781+0000","last_became_active":"2023-11-15T03:11:21.457453+0000","last_became_peered":"2023-11-15T03:11:21.457453+0000","last_unstale":"2023-11-15T03:11:21.901781+0000","last_undegraded":"2023-11-15T03:11:21.901781+0000","last_fullsized":"2023-11-15T03:11:21.901781+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.908036+0000","last_change":"2023-11-15T03:11:21.899337+0000","last_active":"2023-11-15T03:11:21.908036+0000","last_peered":"2023-11-15T03:11:21.908036+0000","last_clean":"2023-11-15T03:11:21.908036+0000","last_became_active":"2023-11-15T03:11:21.899050+0000","last_became_peered":"2023-11-15T03:11:21.899050+0000","last_unstale":"2023-11-15T03:11:21.908036+0000","last_undegraded":"2023-11-15T03:11:21.908036+0000","last_fullsized":"2023-11-15T03:11:21.908036+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.874950+0000","last_change":"2023-11-15T03:11:21.458880+0000","last_active":"2023-11-15T03:11:21.874950+0000","last_peered":"2023-11-15T03:11:21.874950+0000","last_clean":"2023-11-15T03:11:21.874950+0000","last_became_active":"2023-11-15T03:11:21.458602+0000","last_became_peered":"2023-11-15T03:11:21.458602+0000","last_unstale":"2023-11-15T03:11:21.874950+0000","last_undegraded":"2023-11-15T03:11:21.874950+0000","last_fullsized":"2023-11-15T03:11:21.874950+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.909496+0000","last_change":"2023-11-15T03:11:21.903483+0000","last_active":"2023-11-15T03:11:21.909496+0000","last_peered":"2023-11-15T03:11:21.909496+0000","last_clean":"2023-11-15T03:11:21.909496+0000","last_became_active":"2023-11-15T03:11:21.903217+0000","last_became_peered":"2023-11-15T03:11:21.903217+0000","last_unstale":"2023-11-15T03:11:21.909496+0000","last_undegraded":"2023-11-15T03:11:21.909496+0000","last_fullsized":"2023-11-15T03:11:21.909496+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.898649+0000","last_change":"2023-11-15T03:11:21.469823+0000","last_active":"2023-11-15T03:11:21.898649+0000","last_peered":"2023-11-15T03:11:21.898649+0000","last_clean":"2023-11-15T03:11:21.898649+0000","last_became_active":"2023-11-15T03:11:21.469506+0000","last_became_peered":"2023-11-15T03:11:21.469506+0000","last_unstale":"2023-11-15T03:11:21.898649+0000","last_undegraded":"2023-11-15T03:11:21.898649+0000","last_fullsized":"2023-11-15T03:11:21.898649+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.914290+0000","last_change":"2023-11-15T03:11:21.908550+0000","last_active":"2023-11-15T03:11:21.914290+0000","last_peered":"2023-11-15T03:11:21.914290+0000","last_clean":"2023-11-15T03:11:21.914290+0000","last_became_active":"2023-11-15T03:11:21.908217+0000","last_became_peered":"2023-11-15T03:11:21.908217+0000","last_unstale":"2023-11-15T03:11:21.914290+0000","last_undegraded":"2023-11-15T03:11:21.914290+0000","last_fullsized":"2023-11-15T03:11:21.914290+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.890211+0000","last_change":"2023-11-15T03:11:21.457749+0000","last_active":"2023-11-15T03:11:21.890211+0000","last_peered":"2023-11-15T03:11:21.890211+0000","last_clean":"2023-11-15T03:11:21.890211+0000","last_became_active":"2023-11-15T03:11:21.456847+0000","last_became_peered":"2023-11-15T03:11:21.456847+0000","last_unstale":"2023-11-15T03:11:21.890211+0000","last_undegraded":"2023-11-15T03:11:21.890211+0000","last_fullsized":"2023-11-15T03:11:21.890211+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.919479+0000","last_change":"2023-11-15T03:11:21.914107+0000","last_active":"2023-11-15T03:11:21.919479+0000","last_peered":"2023-11-15T03:11:21.919479+0000","last_clean":"2023-11-15T03:11:21.919479+0000","last_became_active":"2023-11-15T03:11:21.913698+0000","last_became_peered":"2023-11-15T03:11:21.913698+0000","last_unstale":"2023-11-15T03:11:21.919479+0000","last_undegraded":"2023-11-15T03:11:21.919479+0000","last_fullsized":"2023-11-15T03:11:21.919479+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.887536+0000","last_change":"2023-11-15T03:11:21.456989+0000","last_active":"2023-11-15T03:11:21.887536+0000","last_peered":"2023-11-15T03:11:21.887536+0000","last_clean":"2023-11-15T03:11:21.887536+0000","last_became_active":"2023-11-15T03:11:21.456685+0000","last_became_peered":"2023-11-15T03:11:21.456685+0000","last_unstale":"2023-11-15T03:11:21.887536+0000","last_undegraded":"2023-11-15T03:11:21.887536+0000","last_fullsized":"2023-11-15T03:11:21.887536+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.909694+0000","last_change":"2023-11-15T03:11:21.899808+0000","last_active":"2023-11-15T03:11:21.909694+0000","last_peered":"2023-11-15T03:11:21.909694+0000","last_clean":"2023-11-15T03:11:21.909694+0000","last_became_active":"2023-11-15T03:11:21.899557+0000","last_became_peered":"2023-11-15T03:11:21.899557+0000","last_unstale":"2023-11-15T03:11:21.909694+0000","last_undegraded":"2023-11-15T03:11:21.909694+0000","last_fullsized":"2023-11-15T03:11:21.909694+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.888073+0000","last_change":"2023-11-15T03:11:21.457242+0000","last_active":"2023-11-15T03:11:21.888073+0000","last_peered":"2023-11-15T03:11:21.888073+0000","last_clean":"2023-11-15T03:11:21.888073+0000","last_became_active":"2023-11-15T03:11:21.456918+0000","last_became_peered":"2023-11-15T03:11:21.456918+0000","last_unstale":"2023-11-15T03:11:21.888073+0000","last_undegraded":"2023-11-15T03:11:21.888073+0000","last_fullsized":"2023-11-15T03:11:21.888073+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.914308+0000","last_change":"2023-11-15T03:11:21.909292+0000","last_active":"2023-11-15T03:11:21.914308+0000","last_peered":"2023-11-15T03:11:21.914308+0000","last_clean":"2023-11-15T03:11:21.914308+0000","last_became_active":"2023-11-15T03:11:21.906736+0000","last_became_peered":"2023-11-15T03:11:21.906736+0000","last_unstale":"2023-11-15T03:11:21.914308+0000","last_undegraded":"2023-11-15T03:11:21.914308+0000","last_fullsized":"2023-11-15T03:11:21.914308+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.889617+0000","last_change":"2023-11-15T03:11:21.458034+0000","last_active":"2023-11-15T03:11:21.889617+0000","last_peered":"2023-11-15T03:11:21.889617+0000","last_clean":"2023-11-15T03:11:21.889617+0000","last_became_active":"2023-11-15T03:11:21.457633+0000","last_became_peered":"2023-11-15T03:11:21.457633+0000","last_unstale":"2023-11-15T03:11:21.889617+0000","last_undegraded":"2023-11-15T03:11:21.889617+0000","last_fullsized":"2023-11-15T03:11:21.889617+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.900698+0000","last_change":"2023-11-15T03:11:21.890359+0000","last_active":"2023-11-15T03:11:21.900698+0000","last_peered":"2023-11-15T03:11:21.900698+0000","last_clean":"2023-11-15T03:11:21.900698+0000","last_became_active":"2023-11-15T03:11:21.890059+0000","last_became_peered":"2023-11-15T03:11:21.890059+0000","last_unstale":"2023-11-15T03:11:21.900698+0000","last_undegraded":"2023-11-15T03:11:21.900698+0000","last_fullsized":"2023-11-15T03:11:21.900698+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.888864+0000","last_change":"2023-11-15T03:11:21.471960+0000","last_active":"2023-11-15T03:11:21.888864+0000","last_peered":"2023-11-15T03:11:21.888864+0000","last_clean":"2023-11-15T03:11:21.888864+0000","last_became_active":"2023-11-15T03:11:21.463041+0000","last_became_peered":"2023-11-15T03:11:21.463041+0000","last_unstale":"2023-11-15T03:11:21.888864+0000","last_undegraded":"2023-11-15T03:11:21.888864+0000","last_fullsized":"2023-11-15T03:11:21.888864+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.909080+0000","last_change":"2023-11-15T03:11:21.905051+0000","last_active":"2023-11-15T03:11:21.909080+0000","last_peered":"2023-11-15T03:11:21.909080+0000","last_clean":"2023-11-15T03:11:21.909080+0000","last_became_active":"2023-11-15T03:11:21.902394+0000","last_became_peered":"2023-11-15T03:11:21.902394+0000","last_unstale":"2023-11-15T03:11:21.909080+0000","last_undegraded":"2023-11-15T03:11:21.909080+0000","last_fullsized":"2023-11-15T03:11:21.909080+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"144'3","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.916933+0000","last_change":"2023-11-15T03:11:21.475101+0000","last_active":"2023-11-15T03:11:21.916933+0000","last_peered":"2023-11-15T03:11:21.916933+0000","last_clean":"2023-11-15T03:11:21.916933+0000","last_became_active":"2023-11-15T03:11:21.474805+0000","last_became_peered":"2023-11-15T03:11:21.474805+0000","last_unstale":"2023-11-15T03:11:21.916933+0000","last_undegraded":"2023-11-15T03:11:21.916933+0000","last_fullsized":"2023-11-15T03:11:21.916933+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.902545+0000","last_change":"2023-11-15T03:11:21.890998+0000","last_active":"2023-11-15T03:11:21.902545+0000","last_peered":"2023-11-15T03:11:21.902545+0000","last_clean":"2023-11-15T03:11:21.902545+0000","last_became_active":"2023-11-15T03:11:21.890615+0000","last_became_peered":"2023-11-15T03:11:21.890615+0000","last_unstale":"2023-11-15T03:11:21.902545+0000","last_undegraded":"2023-11-15T03:11:21.902545+0000","last_fullsized":"2023-11-15T03:11:21.902545+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"144'2","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.922217+0000","last_change":"2023-11-15T03:11:21.477165+0000","last_active":"2023-11-15T03:11:21.922217+0000","last_peered":"2023-11-15T03:11:21.922217+0000","last_clean":"2023-11-15T03:11:21.922217+0000","last_became_active":"2023-11-15T03:11:21.476860+0000","last_became_peered":"2023-11-15T03:11:21.476860+0000","last_unstale":"2023-11-15T03:11:21.922217+0000","last_undegraded":"2023-11-15T03:11:21.922217+0000","last_fullsized":"2023-11-15T03:11:21.922217+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.910346+0000","last_change":"2023-11-15T03:11:21.906419+0000","last_active":"2023-11-15T03:11:21.910346+0000","last_peered":"2023-11-15T03:11:21.910346+0000","last_clean":"2023-11-15T03:11:21.910346+0000","last_became_active":"2023-11-15T03:11:21.906123+0000","last_became_peered":"2023-11-15T03:11:21.906123+0000","last_unstale":"2023-11-15T03:11:21.910346+0000","last_undegraded":"2023-11-15T03:11:21.910346+0000","last_fullsized":"2023-11-15T03:11:21.910346+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"144'1","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.896366+0000","last_change":"2023-11-15T03:11:21.465973+0000","last_active":"2023-11-15T03:11:21.896366+0000","last_peered":"2023-11-15T03:11:21.896366+0000","last_clean":"2023-11-15T03:11:21.896366+0000","last_became_active":"2023-11-15T03:11:21.465530+0000","last_became_peered":"2023-11-15T03:11:21.465530+0000","last_unstale":"2023-11-15T03:11:21.896366+0000","last_undegraded":"2023-11-15T03:11:21.896366+0000","last_fullsized":"2023-11-15T03:11:21.896366+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.921883+0000","last_change":"2023-11-15T03:11:21.916818+0000","last_active":"2023-11-15T03:11:21.921883+0000","last_peered":"2023-11-15T03:11:21.921883+0000","last_clean":"2023-11-15T03:11:21.921883+0000","last_became_active":"2023-11-15T03:11:21.916549+0000","last_became_peered":"2023-11-15T03:11:21.916549+0000","last_unstale":"2023-11-15T03:11:21.921883+0000","last_undegraded":"2023-11-15T03:11:21.921883+0000","last_fullsized":"2023-11-15T03:11:21.921883+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.894094+0000","last_change":"2023-11-15T03:11:21.445718+0000","last_active":"2023-11-15T03:11:21.894094+0000","last_peered":"2023-11-15T03:11:21.894094+0000","last_clean":"2023-11-15T03:11:21.894094+0000","last_became_active":"2023-11-15T03:11:21.445418+0000","last_became_peered":"2023-11-15T03:11:21.445418+0000","last_unstale":"2023-11-15T03:11:21.894094+0000","last_undegraded":"2023-11-15T03:11:21.894094+0000","last_fullsized":"2023-11-15T03:11:21.894094+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.906851+0000","last_change":"2023-11-15T03:11:21.898757+0000","last_active":"2023-11-15T03:11:21.906851+0000","last_peered":"2023-11-15T03:11:21.906851+0000","last_clean":"2023-11-15T03:11:21.906851+0000","last_became_active":"2023-11-15T03:11:21.897827+0000","last_became_peered":"2023-11-15T03:11:21.897827+0000","last_unstale":"2023-11-15T03:11:21.906851+0000","last_undegraded":"2023-11-15T03:11:21.906851+0000","last_fullsized":"2023-11-15T03:11:21.906851+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.885611+0000","last_change":"2023-11-15T03:11:21.451870+0000","last_active":"2023-11-15T03:11:21.885611+0000","last_peered":"2023-11-15T03:11:21.885611+0000","last_clean":"2023-11-15T03:11:21.885611+0000","last_became_active":"2023-11-15T03:11:21.451603+0000","last_became_peered":"2023-11-15T03:11:21.451603+0000","last_unstale":"2023-11-15T03:11:21.885611+0000","last_undegraded":"2023-11-15T03:11:21.885611+0000","last_fullsized":"2023-11-15T03:11:21.885611+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.911260+0000","last_change":"2023-11-15T03:11:21.901517+0000","last_active":"2023-11-15T03:11:21.911260+0000","last_peered":"2023-11-15T03:11:21.911260+0000","last_clean":"2023-11-15T03:11:21.911260+0000","last_became_active":"2023-11-15T03:11:21.901171+0000","last_became_peered":"2023-11-15T03:11:21.901171+0000","last_unstale":"2023-11-15T03:11:21.911260+0000","last_undegraded":"2023-11-15T03:11:21.911260+0000","last_fullsized":"2023-11-15T03:11:21.911260+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"144'4","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.905106+0000","last_change":"2023-11-15T03:11:21.457798+0000","last_active":"2023-11-15T03:11:21.905106+0000","last_peered":"2023-11-15T03:11:21.905106+0000","last_clean":"2023-11-15T03:11:21.905106+0000","last_became_active":"2023-11-15T03:11:21.457487+0000","last_became_peered":"2023-11-15T03:11:21.457487+0000","last_unstale":"2023-11-15T03:11:21.905106+0000","last_undegraded":"2023-11-15T03:11:21.905106+0000","last_fullsized":"2023-11-15T03:11:21.905106+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.915293+0000","last_change":"2023-11-15T03:11:21.911359+0000","last_active":"2023-11-15T03:11:21.915293+0000","last_peered":"2023-11-15T03:11:21.915293+0000","last_clean":"2023-11-15T03:11:21.915293+0000","last_became_active":"2023-11-15T03:11:21.910932+0000","last_became_peered":"2023-11-15T03:11:21.910932+0000","last_unstale":"2023-11-15T03:11:21.915293+0000","last_undegraded":"2023-11-15T03:11:21.915293+0000","last_fullsized":"2023-11-15T03:11:21.915293+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"144'2","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.899943+0000","last_change":"2023-11-15T03:11:21.464465+0000","last_active":"2023-11-15T03:11:21.899943+0000","last_peered":"2023-11-15T03:11:21.899943+0000","last_clean":"2023-11-15T03:11:21.899943+0000","last_became_active":"2023-11-15T03:11:21.464201+0000","last_became_peered":"2023-11-15T03:11:21.464201+0000","last_unstale":"2023-11-15T03:11:21.899943+0000","last_undegraded":"2023-11-15T03:11:21.899943+0000","last_fullsized":"2023-11-15T03:11:21.899943+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.913778+0000","last_change":"2023-11-15T03:11:21.908557+0000","last_active":"2023-11-15T03:11:21.913778+0000","last_peered":"2023-11-15T03:11:21.913778+0000","last_clean":"2023-11-15T03:11:21.913778+0000","last_became_active":"2023-11-15T03:11:21.908257+0000","last_became_peered":"2023-11-15T03:11:21.908257+0000","last_unstale":"2023-11-15T03:11:21.913778+0000","last_undegraded":"2023-11-15T03:11:21.913778+0000","last_fullsized":"2023-11-15T03:11:21.913778+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.888961+0000","last_change":"2023-11-15T03:11:21.471000+0000","last_active":"2023-11-15T03:11:21.888961+0000","last_peered":"2023-11-15T03:11:21.888961+0000","last_clean":"2023-11-15T03:11:21.888961+0000","last_became_active":"2023-11-15T03:11:21.468688+0000","last_became_peered":"2023-11-15T03:11:21.468688+0000","last_unstale":"2023-11-15T03:11:21.888961+0000","last_undegraded":"2023-11-15T03:11:21.888961+0000","last_fullsized":"2023-11-15T03:11:21.888961+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.909617+0000","last_change":"2023-11-15T03:11:21.903549+0000","last_active":"2023-11-15T03:11:21.909617+0000","last_peered":"2023-11-15T03:11:21.909617+0000","last_clean":"2023-11-15T03:11:21.909617+0000","last_became_active":"2023-11-15T03:11:21.903262+0000","last_became_peered":"2023-11-15T03:11:21.903262+0000","last_unstale":"2023-11-15T03:11:21.909617+0000","last_undegraded":"2023-11-15T03:11:21.909617+0000","last_fullsized":"2023-11-15T03:11:21.909617+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.905768+0000","last_change":"2023-11-15T03:11:21.468901+0000","last_active":"2023-11-15T03:11:21.905768+0000","last_peered":"2023-11-15T03:11:21.905768+0000","last_clean":"2023-11-15T03:11:21.905768+0000","last_became_active":"2023-11-15T03:11:21.468605+0000","last_became_peered":"2023-11-15T03:11:21.468605+0000","last_unstale":"2023-11-15T03:11:21.905768+0000","last_undegraded":"2023-11-15T03:11:21.905768+0000","last_fullsized":"2023-11-15T03:11:21.905768+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.917152+0000","last_change":"2023-11-15T03:11:21.913399+0000","last_active":"2023-11-15T03:11:21.917152+0000","last_peered":"2023-11-15T03:11:21.917152+0000","last_clean":"2023-11-15T03:11:21.917152+0000","last_became_active":"2023-11-15T03:11:21.912859+0000","last_became_peered":"2023-11-15T03:11:21.912859+0000","last_unstale":"2023-11-15T03:11:21.917152+0000","last_undegraded":"2023-11-15T03:11:21.917152+0000","last_fullsized":"2023-11-15T03:11:21.917152+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.874798+0000","last_change":"2023-11-15T03:11:21.465450+0000","last_active":"2023-11-15T03:11:21.874798+0000","last_peered":"2023-11-15T03:11:21.874798+0000","last_clean":"2023-11-15T03:11:21.874798+0000","last_became_active":"2023-11-15T03:11:21.465152+0000","last_became_peered":"2023-11-15T03:11:21.465152+0000","last_unstale":"2023-11-15T03:11:21.874798+0000","last_undegraded":"2023-11-15T03:11:21.874798+0000","last_fullsized":"2023-11-15T03:11:21.874798+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.908224+0000","last_change":"2023-11-15T03:11:21.904121+0000","last_active":"2023-11-15T03:11:21.908224+0000","last_peered":"2023-11-15T03:11:21.908224+0000","last_clean":"2023-11-15T03:11:21.908224+0000","last_became_active":"2023-11-15T03:11:21.903870+0000","last_became_peered":"2023-11-15T03:11:21.903870+0000","last_unstale":"2023-11-15T03:11:21.908224+0000","last_undegraded":"2023-11-15T03:11:21.908224+0000","last_fullsized":"2023-11-15T03:11:21.908224+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.889945+0000","last_change":"2023-11-15T03:11:21.464111+0000","last_active":"2023-11-15T03:11:21.889945+0000","last_peered":"2023-11-15T03:11:21.889945+0000","last_clean":"2023-11-15T03:11:21.889945+0000","last_became_active":"2023-11-15T03:11:21.463723+0000","last_became_peered":"2023-11-15T03:11:21.463723+0000","last_unstale":"2023-11-15T03:11:21.889945+0000","last_undegraded":"2023-11-15T03:11:21.889945+0000","last_fullsized":"2023-11-15T03:11:21.889945+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.907773+0000","last_change":"2023-11-15T03:11:21.901552+0000","last_active":"2023-11-15T03:11:21.907773+0000","last_peered":"2023-11-15T03:11:21.907773+0000","last_clean":"2023-11-15T03:11:21.907773+0000","last_became_active":"2023-11-15T03:11:21.901253+0000","last_became_peered":"2023-11-15T03:11:21.901253+0000","last_unstale":"2023-11-15T03:11:21.907773+0000","last_undegraded":"2023-11-15T03:11:21.907773+0000","last_fullsized":"2023-11-15T03:11:21.907773+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.887199+0000","last_change":"2023-11-15T03:11:21.453137+0000","last_active":"2023-11-15T03:11:21.887199+0000","last_peered":"2023-11-15T03:11:21.887199+0000","last_clean":"2023-11-15T03:11:21.887199+0000","last_became_active":"2023-11-15T03:11:21.452443+0000","last_became_peered":"2023-11-15T03:11:21.452443+0000","last_unstale":"2023-11-15T03:11:21.887199+0000","last_undegraded":"2023-11-15T03:11:21.887199+0000","last_fullsized":"2023-11-15T03:11:21.887199+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.915529+0000","last_change":"2023-11-15T03:11:21.910438+0000","last_active":"2023-11-15T03:11:21.915529+0000","last_peered":"2023-11-15T03:11:21.915529+0000","last_clean":"2023-11-15T03:11:21.915529+0000","last_became_active":"2023-11-15T03:11:21.910185+0000","last_became_peered":"2023-11-15T03:11:21.910185+0000","last_unstale":"2023-11-15T03:11:21.915529+0000","last_undegraded":"2023-11-15T03:11:21.915529+0000","last_fullsized":"2023-11-15T03:11:21.915529+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.900491+0000","last_change":"2023-11-15T03:11:21.465483+0000","last_active":"2023-11-15T03:11:21.900491+0000","last_peered":"2023-11-15T03:11:21.900491+0000","last_clean":"2023-11-15T03:11:21.900491+0000","last_became_active":"2023-11-15T03:11:21.464586+0000","last_became_peered":"2023-11-15T03:11:21.464586+0000","last_unstale":"2023-11-15T03:11:21.900491+0000","last_undegraded":"2023-11-15T03:11:21.900491+0000","last_fullsized":"2023-11-15T03:11:21.900491+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.909613+0000","last_change":"2023-11-15T03:11:21.904368+0000","last_active":"2023-11-15T03:11:21.909613+0000","last_peered":"2023-11-15T03:11:21.909613+0000","last_clean":"2023-11-15T03:11:21.909613+0000","last_became_active":"2023-11-15T03:11:21.904073+0000","last_became_peered":"2023-11-15T03:11:21.904073+0000","last_unstale":"2023-11-15T03:11:21.909613+0000","last_undegraded":"2023-11-15T03:11:21.909613+0000","last_fullsized":"2023-11-15T03:11:21.909613+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.884976+0000","last_change":"2023-11-15T03:11:21.450282+0000","last_active":"2023-11-15T03:11:21.884976+0000","last_peered":"2023-11-15T03:11:21.884976+0000","last_clean":"2023-11-15T03:11:21.884976+0000","last_became_active":"2023-11-15T03:11:21.449995+0000","last_became_peered":"2023-11-15T03:11:21.449995+0000","last_unstale":"2023-11-15T03:11:21.884976+0000","last_undegraded":"2023-11-15T03:11:21.884976+0000","last_fullsized":"2023-11-15T03:11:21.884976+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.914269+0000","last_change":"2023-11-15T03:11:21.906676+0000","last_active":"2023-11-15T03:11:21.914269+0000","last_peered":"2023-11-15T03:11:21.914269+0000","last_clean":"2023-11-15T03:11:21.914269+0000","last_became_active":"2023-11-15T03:11:21.906045+0000","last_became_peered":"2023-11-15T03:11:21.906045+0000","last_unstale":"2023-11-15T03:11:21.914269+0000","last_undegraded":"2023-11-15T03:11:21.914269+0000","last_fullsized":"2023-11-15T03:11:21.914269+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.887149+0000","last_change":"2023-11-15T03:11:21.453315+0000","last_active":"2023-11-15T03:11:21.887149+0000","last_peered":"2023-11-15T03:11:21.887149+0000","last_clean":"2023-11-15T03:11:21.887149+0000","last_became_active":"2023-11-15T03:11:21.452827+0000","last_became_peered":"2023-11-15T03:11:21.452827+0000","last_unstale":"2023-11-15T03:11:21.887149+0000","last_undegraded":"2023-11-15T03:11:21.887149+0000","last_fullsized":"2023-11-15T03:11:21.887149+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":123,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.908375+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T03:11:21.908375+0000","last_peered":"2023-11-15T03:11:21.908375+0000","last_clean":"2023-11-15T03:11:21.908375+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T03:11:21.908375+0000","last_undegraded":"2023-11-15T03:11:21.908375+0000","last_fullsized":"2023-11-15T03:11:21.908375+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.915244+0000","last_change":"2023-11-15T03:11:21.910120+0000","last_active":"2023-11-15T03:11:21.915244+0000","last_peered":"2023-11-15T03:11:21.915244+0000","last_clean":"2023-11-15T03:11:21.915244+0000","last_became_active":"2023-11-15T03:11:21.908084+0000","last_became_peered":"2023-11-15T03:11:21.908084+0000","last_unstale":"2023-11-15T03:11:21.915244+0000","last_undegraded":"2023-11-15T03:11:21.915244+0000","last_fullsized":"2023-11-15T03:11:21.915244+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"144'1","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.905977+0000","last_change":"2023-11-15T03:11:21.462430+0000","last_active":"2023-11-15T03:11:21.905977+0000","last_peered":"2023-11-15T03:11:21.905977+0000","last_clean":"2023-11-15T03:11:21.905977+0000","last_became_active":"2023-11-15T03:11:21.462160+0000","last_became_peered":"2023-11-15T03:11:21.462160+0000","last_unstale":"2023-11-15T03:11:21.905977+0000","last_undegraded":"2023-11-15T03:11:21.905977+0000","last_fullsized":"2023-11-15T03:11:21.905977+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":126,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.876350+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T03:11:21.876350+0000","last_peered":"2023-11-15T03:11:21.876350+0000","last_clean":"2023-11-15T03:11:21.876350+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T03:11:21.876350+0000","last_undegraded":"2023-11-15T03:11:21.876350+0000","last_fullsized":"2023-11-15T03:11:21.876350+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.908058+0000","last_change":"2023-11-15T03:11:21.902655+0000","last_active":"2023-11-15T03:11:21.908058+0000","last_peered":"2023-11-15T03:11:21.908058+0000","last_clean":"2023-11-15T03:11:21.908058+0000","last_became_active":"2023-11-15T03:11:21.902038+0000","last_became_peered":"2023-11-15T03:11:21.902038+0000","last_unstale":"2023-11-15T03:11:21.908058+0000","last_undegraded":"2023-11-15T03:11:21.908058+0000","last_fullsized":"2023-11-15T03:11:21.908058+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.884916+0000","last_change":"2023-11-15T03:11:21.450433+0000","last_active":"2023-11-15T03:11:21.884916+0000","last_peered":"2023-11-15T03:11:21.884916+0000","last_clean":"2023-11-15T03:11:21.884916+0000","last_became_active":"2023-11-15T03:11:21.450138+0000","last_became_peered":"2023-11-15T03:11:21.450138+0000","last_unstale":"2023-11-15T03:11:21.884916+0000","last_undegraded":"2023-11-15T03:11:21.884916+0000","last_fullsized":"2023-11-15T03:11:21.884916+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":121,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.900716+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T03:11:21.900716+0000","last_peered":"2023-11-15T03:11:21.900716+0000","last_clean":"2023-11-15T03:11:21.900716+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T03:11:21.900716+0000","last_undegraded":"2023-11-15T03:11:21.900716+0000","last_fullsized":"2023-11-15T03:11:21.900716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.909451+0000","last_change":"2023-11-15T03:11:21.903572+0000","last_active":"2023-11-15T03:11:21.909451+0000","last_peered":"2023-11-15T03:11:21.909451+0000","last_clean":"2023-11-15T03:11:21.909451+0000","last_became_active":"2023-11-15T03:11:21.903301+0000","last_became_peered":"2023-11-15T03:11:21.903301+0000","last_unstale":"2023-11-15T03:11:21.909451+0000","last_undegraded":"2023-11-15T03:11:21.909451+0000","last_fullsized":"2023-11-15T03:11:21.909451+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.900421+0000","last_change":"2023-11-15T03:11:21.468754+0000","last_active":"2023-11-15T03:11:21.900421+0000","last_peered":"2023-11-15T03:11:21.900421+0000","last_clean":"2023-11-15T03:11:21.900421+0000","last_became_active":"2023-11-15T03:11:21.468424+0000","last_became_peered":"2023-11-15T03:11:21.468424+0000","last_unstale":"2023-11-15T03:11:21.900421+0000","last_undegraded":"2023-11-15T03:11:21.900421+0000","last_fullsized":"2023-11-15T03:11:21.900421+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":126,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.874939+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T03:11:21.874939+0000","last_peered":"2023-11-15T03:11:21.874939+0000","last_clean":"2023-11-15T03:11:21.874939+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T03:11:21.874939+0000","last_undegraded":"2023-11-15T03:11:21.874939+0000","last_fullsized":"2023-11-15T03:11:21.874939+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.908285+0000","last_change":"2023-11-15T03:11:21.904874+0000","last_active":"2023-11-15T03:11:21.908285+0000","last_peered":"2023-11-15T03:11:21.908285+0000","last_clean":"2023-11-15T03:11:21.908285+0000","last_became_active":"2023-11-15T03:11:21.904621+0000","last_became_peered":"2023-11-15T03:11:21.904621+0000","last_unstale":"2023-11-15T03:11:21.908285+0000","last_undegraded":"2023-11-15T03:11:21.908285+0000","last_fullsized":"2023-11-15T03:11:21.908285+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"144'1","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.919053+0000","last_change":"2023-11-15T03:11:21.469422+0000","last_active":"2023-11-15T03:11:21.919053+0000","last_peered":"2023-11-15T03:11:21.919053+0000","last_clean":"2023-11-15T03:11:21.919053+0000","last_became_active":"2023-11-15T03:11:21.468947+0000","last_became_peered":"2023-11-15T03:11:21.468947+0000","last_unstale":"2023-11-15T03:11:21.919053+0000","last_undegraded":"2023-11-15T03:11:21.919053+0000","last_fullsized":"2023-11-15T03:11:21.919053+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":137,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.890260+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T03:11:21.890260+0000","last_peered":"2023-11-15T03:11:21.890260+0000","last_clean":"2023-11-15T03:11:21.890260+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T03:11:21.890260+0000","last_undegraded":"2023-11-15T03:11:21.890260+0000","last_fullsized":"2023-11-15T03:11:21.890260+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.902579+0000","last_change":"2023-11-15T03:11:21.896499+0000","last_active":"2023-11-15T03:11:21.902579+0000","last_peered":"2023-11-15T03:11:21.902579+0000","last_clean":"2023-11-15T03:11:21.902579+0000","last_became_active":"2023-11-15T03:11:21.896160+0000","last_became_peered":"2023-11-15T03:11:21.896160+0000","last_unstale":"2023-11-15T03:11:21.902579+0000","last_undegraded":"2023-11-15T03:11:21.902579+0000","last_fullsized":"2023-11-15T03:11:21.902579+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.890040+0000","last_change":"2023-11-15T03:11:21.470370+0000","last_active":"2023-11-15T03:11:21.890040+0000","last_peered":"2023-11-15T03:11:21.890040+0000","last_clean":"2023-11-15T03:11:21.890040+0000","last_became_active":"2023-11-15T03:11:21.470048+0000","last_became_peered":"2023-11-15T03:11:21.470048+0000","last_unstale":"2023-11-15T03:11:21.890040+0000","last_undegraded":"2023-11-15T03:11:21.890040+0000","last_fullsized":"2023-11-15T03:11:21.890040+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":121,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.885695+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T03:11:21.885695+0000","last_peered":"2023-11-15T03:11:21.885695+0000","last_clean":"2023-11-15T03:11:21.885695+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T03:11:21.885695+0000","last_undegraded":"2023-11-15T03:11:21.885695+0000","last_fullsized":"2023-11-15T03:11:21.885695+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.919324+0000","last_change":"2023-11-15T03:11:21.915590+0000","last_active":"2023-11-15T03:11:21.919324+0000","last_peered":"2023-11-15T03:11:21.919324+0000","last_clean":"2023-11-15T03:11:21.919324+0000","last_became_active":"2023-11-15T03:11:21.915338+0000","last_became_peered":"2023-11-15T03:11:21.915338+0000","last_unstale":"2023-11-15T03:11:21.919324+0000","last_undegraded":"2023-11-15T03:11:21.919324+0000","last_fullsized":"2023-11-15T03:11:21.919324+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.900643+0000","last_change":"2023-11-15T03:11:21.466338+0000","last_active":"2023-11-15T03:11:21.900643+0000","last_peered":"2023-11-15T03:11:21.900643+0000","last_clean":"2023-11-15T03:11:21.900643+0000","last_became_active":"2023-11-15T03:11:21.466075+0000","last_became_peered":"2023-11-15T03:11:21.466075+0000","last_unstale":"2023-11-15T03:11:21.900643+0000","last_undegraded":"2023-11-15T03:11:21.900643+0000","last_fullsized":"2023-11-15T03:11:21.900643+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":121,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.897124+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T03:11:21.897124+0000","last_peered":"2023-11-15T03:11:21.897124+0000","last_clean":"2023-11-15T03:11:21.897124+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T03:11:21.897124+0000","last_undegraded":"2023-11-15T03:11:21.897124+0000","last_fullsized":"2023-11-15T03:11:21.897124+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.909627+0000","last_change":"2023-11-15T03:11:21.904610+0000","last_active":"2023-11-15T03:11:21.909627+0000","last_peered":"2023-11-15T03:11:21.909627+0000","last_clean":"2023-11-15T03:11:21.909627+0000","last_became_active":"2023-11-15T03:11:21.903447+0000","last_became_peered":"2023-11-15T03:11:21.903447+0000","last_unstale":"2023-11-15T03:11:21.909627+0000","last_undegraded":"2023-11-15T03:11:21.909627+0000","last_fullsized":"2023-11-15T03:11:21.909627+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:21.412123+0000","last_clean_scrub_stamp":"2023-11-15T03:11:21.412123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"144'2","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.907268+0000","last_change":"2023-11-15T03:11:21.453954+0000","last_active":"2023-11-15T03:11:21.907268+0000","last_peered":"2023-11-15T03:11:21.907268+0000","last_clean":"2023-11-15T03:11:21.907268+0000","last_became_active":"2023-11-15T03:11:21.453644+0000","last_became_peered":"2023-11-15T03:11:21.453644+0000","last_unstale":"2023-11-15T03:11:21.907268+0000","last_undegraded":"2023-11-15T03:11:21.907268+0000","last_fullsized":"2023-11-15T03:11:21.907268+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:20.396314+0000","last_clean_scrub_stamp":"2023-11-15T03:11:20.396314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":137,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.890372+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T03:11:21.890372+0000","last_peered":"2023-11-15T03:11:21.890372+0000","last_clean":"2023-11-15T03:11:21.890372+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T03:11:21.890372+0000","last_undegraded":"2023-11-15T03:11:21.890372+0000","last_fullsized":"2023-11-15T03:11:21.890372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":122,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-11-15T03:11:21.897199+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T03:11:21.897199+0000","last_peered":"2023-11-15T03:11:21.897199+0000","last_clean":"2023-11-15T03:11:21.897199+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T03:11:21.897199+0000","last_undegraded":"2023-11-15T03:11:21.897199+0000","last_fullsized":"2023-11-15T03:11:21.897199+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542322,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307132,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064708,"statfs":{"total":96636764160,"available":96322260992,"internally_reserved":0,"allocated":1888256,"data_stored":956538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.0940000000000001,"15min":1.1319999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.55800000000000005},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":10.961},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.032,"5min":1.0880000000000001,"15min":1.1120000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.3089999999999999,"5min":1.6160000000000001,"15min":3.4910000000000001},"last":1.258}]},{"osd":1,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.127,"15min":1.143},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.419,"5min":1.552,"15min":5.4930000000000003},"last":1.45},{"interface":"front","average":{"1min":1.153,"5min":1.165,"15min":1.1819999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6000000000000001,"5min":1.766,"15min":6.8239999999999998},"last":1.175}]},{"osd":2,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0940000000000001,"15min":1.1359999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.452,"5min":1.452,"15min":5.7000000000000002},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.226,"15min":1.2310000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":2.75},"last":1.387}]},{"osd":3,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.167,"15min":1.248},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":12.907},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2370000000000001,"15min":1.2709999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.573,"5min":1.573,"15min":9.9909999999999997},"last":1.1930000000000001}]},{"osd":4,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.105,"15min":1.1379999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6160000000000001,"5min":1.8939999999999999,"15min":2.5649999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.1240000000000001,"15min":1.2070000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.72,"5min":1.72,"15min":4.226},"last":1.0489999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.2450000000000001,"15min":1.2749999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":2.6890000000000001},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.143,"5min":1.236,"15min":1.292},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6899999999999999,"5min":1.7330000000000001,"15min":2.4550000000000001},"last":1.5309999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.325,"15min":1.381},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.5629999999999999,"5min":1.954,"15min":3.2789999999999999},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.248,"15min":1.3340000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.63600000000000001},"max":{"1min":1.542,"5min":1.76,"15min":4.7800000000000002},"last":1.4179999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542322,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311236,"kb_used_data":1852,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94060604,"statfs":{"total":96636764160,"available":96318058496,"internally_reserved":0,"allocated":1896448,"data_stored":957114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.03,"15min":1.077},"min":{"1min":0.66200000000000003,"5min":0.63100000000000001,"15min":0.44500000000000001},"max":{"1min":1.4610000000000001,"5min":1.5760000000000001,"15min":4.3159999999999998},"last":1.27},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.109,"15min":1.0960000000000001},"min":{"1min":0.69999999999999996,"5min":0.59799999999999998,"15min":0.58599999999999997},"max":{"1min":1.486,"5min":1.659,"15min":2.2480000000000002},"last":1.202}]},{"osd":1,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.111,"15min":1.1140000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.57599999999999996},"max":{"1min":1.6000000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.147},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1000000000000001,"15min":1.1379999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.60699999999999998},"max":{"1min":1.3360000000000001,"5min":1.8100000000000001,"15min":2.8570000000000002},"last":1.2190000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1899999999999999,"15min":1.1919999999999999},"min":{"1min":0.83099999999999996,"5min":0.71699999999999997,"15min":0.49099999999999999},"max":{"1min":1.4730000000000001,"5min":1.7090000000000001,"15min":2.621},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.115,"5min":1.153,"15min":1.153},"min":{"1min":0.71699999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.4710000000000001,"5min":1.8360000000000001,"15min":2.0059999999999998},"last":0.98799999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.2310000000000001,"15min":1.214},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.50900000000000001},"max":{"1min":1.7709999999999999,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.113},{"interface":"front","average":{"1min":1.234,"5min":1.2430000000000001,"15min":1.2609999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.67100000000000004},"max":{"1min":1.458,"5min":1.8640000000000001,"15min":2.8359999999999999},"last":1.018}]},{"osd":4,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.0580000000000001,"15min":1.1000000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5860000000000001,"5min":1.6839999999999999,"15min":4.5350000000000001},"last":1.081},{"interface":"front","average":{"1min":1.163,"5min":1.2170000000000001,"15min":1.2130000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.8580000000000001,"5min":2.004,"15min":2.0739999999999998},"last":0.78500000000000003}]},{"osd":5,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3799999999999999,"15min":1.415},"min":{"1min":0.79800000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.744,"5min":2.1469999999999998,"15min":2.157},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2210000000000001,"15min":1.256},"min":{"1min":0.621,"5min":0.62,"15min":0.60899999999999999},"max":{"1min":1.8140000000000001,"5min":1.8779999999999999,"15min":2.9460000000000002},"last":1.1719999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.216,"15min":1.2250000000000001},"min":{"1min":0.77400000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.9139999999999999,"5min":1.9239999999999999,"15min":3.1989999999999998},"last":0.871},{"interface":"front","average":{"1min":1.377,"5min":1.407,"15min":1.419},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.65200000000000002},"max":{"1min":1.8220000000000001,"5min":2.0649999999999999,"15min":2.9350000000000001},"last":0.91000000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542323,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307136,"kb_used_data":1848,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064704,"statfs":{"total":96636764160,"available":96322256896,"internally_reserved":0,"allocated":1892352,"data_stored":957362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.0569999999999999,"15min":1.083},"min":{"1min":0.73799999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4810000000000001,"5min":1.5660000000000001,"15min":1.5840000000000001},"last":1.359},{"interface":"front","average":{"1min":1.159,"5min":1.044,"15min":1.056},"min":{"1min":0.85399999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.677},"last":1.0309999999999999}]},{"osd":1,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.135,"15min":1.1319999999999999},"min":{"1min":0.70599999999999996,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":2.617},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.212,"5min":1.1419999999999999,"15min":1.1539999999999999},"min":{"1min":0.82099999999999995,"5min":0.629,"15min":0.54800000000000004},"max":{"1min":1.573,"5min":1.6319999999999999,"15min":1.6619999999999999},"last":1.304}]},{"osd":2,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.165,"15min":1.1830000000000001},"min":{"1min":0.81399999999999995,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.5029999999999999,"5min":1.823,"15min":1.9450000000000001},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.155,"15min":1.1619999999999999},"min":{"1min":0.84199999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9330000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.139}]},{"osd":3,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1399999999999999,"15min":1.1459999999999999},"min":{"1min":0.85699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.5229999999999999,"5min":1.5940000000000001,"15min":1.8660000000000001},"last":1.198},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2549999999999999,"15min":1.264},"min":{"1min":0.93000000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.611,"5min":1.8200000000000001,"15min":2.5630000000000002},"last":1.3420000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1479999999999999,"15min":1.1830000000000001},"min":{"1min":0.91900000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.554,"5min":1.643,"15min":2.0840000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.1479999999999999,"15min":1.1879999999999999},"min":{"1min":0.89400000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":2.3570000000000002},"last":0.90700000000000003}]},{"osd":6,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2410000000000001,"15min":1.264},"min":{"1min":0.97999999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5880000000000001,"5min":1.913,"15min":2.6640000000000001},"last":1.073},{"interface":"front","average":{"1min":1.325,"5min":1.288,"15min":1.323},"min":{"1min":0.88,"5min":0.67600000000000005,"15min":0.57299999999999995},"max":{"1min":1.585,"5min":1.9139999999999999,"15min":2.173},"last":0.93799999999999994}]},{"osd":7,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2769999999999999,"15min":1.3069999999999999},"min":{"1min":0.89500000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.7629999999999999,"5min":1.9199999999999999,"15min":2.6030000000000002},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.3700000000000001,"15min":1.3879999999999999},"min":{"1min":1.044,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8240000000000001,"5min":2.069,"15min":2.7669999999999999},"last":1.278}]}]},{"osd":3,"up_from":14,"seq":60129542323,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303032,"kb_used_data":1840,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068808,"statfs":{"total":96636764160,"available":96326459392,"internally_reserved":0,"allocated":1884160,"data_stored":956448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85999999999999999,"15min":0.83699999999999997},"min":{"1min":0.54200000000000004,"5min":0.54000000000000004,"15min":0.45400000000000001},"max":{"1min":1.327,"5min":1.4219999999999999,"15min":1.528},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.89900000000000002,"15min":0.88800000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.39800000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.81399999999999995}]},{"osd":1,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.83899999999999997,"15min":0.872},"min":{"1min":0.62,"5min":0.53100000000000003,"15min":0.435},"max":{"1min":1.165,"5min":1.25,"15min":1.4670000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.996,"5min":0.98599999999999999,"15min":0.99199999999999999},"min":{"1min":0.72199999999999998,"5min":0.63700000000000001,"15min":0.55800000000000005},"max":{"1min":1.446,"5min":1.446,"15min":1.544},"last":0.88200000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.026,"15min":1.0209999999999999},"min":{"1min":0.66400000000000003,"5min":0.64200000000000002,"15min":0.54600000000000004},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0529999999999999,"15min":1.0660000000000001},"min":{"1min":0.66000000000000003,"5min":0.63100000000000001,"15min":0.50600000000000001},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.2509999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2470000000000001,"15min":1.228},"min":{"1min":1.0029999999999999,"5min":0.88600000000000001,"15min":0.71399999999999997},"max":{"1min":1.5349999999999999,"5min":1.554,"15min":1.9139999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2170000000000001,"15min":1.2250000000000001},"min":{"1min":1.01,"5min":0.78500000000000003,"15min":0.621},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.726},"last":1.405}]},{"osd":5,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.252,"15min":1.266},"min":{"1min":0.91700000000000004,"5min":0.90200000000000002,"15min":0.64300000000000002},"max":{"1min":1.5900000000000001,"5min":1.7290000000000001,"15min":2.0459999999999998},"last":1.155},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.27,"15min":1.272},"min":{"1min":0.95899999999999996,"5min":0.94799999999999995,"15min":0.63500000000000001},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.927},"last":1.2110000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.345,"15min":1.339},"min":{"1min":0.91400000000000003,"5min":0.875,"15min":0.72899999999999998},"max":{"1min":1.575,"5min":1.8129999999999999,"15min":1.9970000000000001},"last":1.349},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3560000000000001,"15min":1.3720000000000001},"min":{"1min":1.0980000000000001,"5min":0.871,"15min":0.77700000000000002},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":2.3570000000000002},"last":1.3260000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.331,"15min":1.3440000000000001},"min":{"1min":1.0149999999999999,"5min":0.92500000000000004,"15min":0.68100000000000005},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":2.0470000000000002},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.345,"15min":1.355},"min":{"1min":1.113,"5min":0.97299999999999998,"15min":0.66600000000000004},"max":{"1min":1.8500000000000001,"5min":1.8779999999999999,"15min":2.1040000000000001},"last":1.4510000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542323,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303036,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068804,"statfs":{"total":96636764160,"available":96326455296,"internally_reserved":0,"allocated":1888256,"data_stored":955283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.65400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97099999999999997,"5min":1.2350000000000001,"15min":1.409},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.84299999999999997,"15min":0.84399999999999997},"min":{"1min":0.58799999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2689999999999999,"5min":1.4950000000000001,"15min":1.619},"last":0.65600000000000003}]},{"osd":1,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.90000000000000002,"15min":0.91100000000000003},"min":{"1min":0.74199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.95599999999999996,"15min":0.95399999999999996},"min":{"1min":0.67200000000000004,"5min":0.55100000000000005,"15min":0.54600000000000004},"max":{"1min":1.3580000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95599999999999996,"15min":0.97199999999999998},"min":{"1min":0.60699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.385,"5min":1.4359999999999999,"15min":1.6259999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.125,"15min":1.125},"min":{"1min":0.67700000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.488,"5min":1.5549999999999999,"15min":1.5800000000000001},"last":1.008}]},{"osd":4,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.218,"15min":1.2070000000000001},"min":{"1min":1.052,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.581,"5min":1.583,"15min":1.9410000000000001},"last":1.145},{"interface":"front","average":{"1min":1.3,"5min":1.21,"15min":1.212},"min":{"1min":0.99199999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6579999999999999,"5min":1.7170000000000001,"15min":1.7829999999999999},"last":1.0549999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1619999999999999,"15min":1.1830000000000001},"min":{"1min":0.98499999999999999,"5min":0.68100000000000005,"15min":0.67000000000000004},"max":{"1min":1.47,"5min":1.625,"15min":1.8939999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.266,"15min":1.272},"min":{"1min":1.0660000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.556,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.089}]},{"osd":6,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.262,"15min":1.2729999999999999},"min":{"1min":0.93999999999999995,"5min":0.62,"15min":0.62},"max":{"1min":1.6990000000000001,"5min":2,"15min":2.137},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.292,"5min":1.2669999999999999,"15min":1.2689999999999999},"min":{"1min":1.0880000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.538,"5min":1.9350000000000001,"15min":2.0870000000000002},"last":1.1719999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.29,"15min":1.3109999999999999},"min":{"1min":0.95699999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7789999999999999,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":0.999},{"interface":"front","average":{"1min":1.323,"5min":1.298,"15min":1.2989999999999999},"min":{"1min":1.101,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.7270000000000001,"5min":1.881,"15min":2.0350000000000001},"last":1.349}]}]},{"osd":0,"up_from":14,"seq":60129542323,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307140,"kb_used_data":1852,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064700,"statfs":{"total":96636764160,"available":96322252800,"internally_reserved":0,"allocated":1896448,"data_stored":957182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80600000000000005,"15min":0.83299999999999996},"min":{"1min":0.503,"5min":0.44400000000000001,"15min":0.36199999999999999},"max":{"1min":1.093,"5min":1.756,"15min":4.3630000000000004},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.90600000000000003,"15min":0.93700000000000006},"min":{"1min":0.61799999999999999,"5min":0.52600000000000002,"15min":0.40000000000000002},"max":{"1min":1.2490000000000001,"5min":1.6719999999999999,"15min":6.157},"last":0.999}]},{"osd":2,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.96399999999999997,"15min":0.97899999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.46800000000000003},"max":{"1min":1.3400000000000001,"5min":1.625,"15min":4.1699999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0029999999999999,"15min":1.0289999999999999},"min":{"1min":0.72199999999999998,"5min":0.60099999999999998,"15min":0.47799999999999998},"max":{"1min":1.379,"5min":1.879,"15min":6.8369999999999997},"last":0.86699999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.94099999999999995,"15min":0.95599999999999996},"min":{"1min":0.59499999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.3120000000000001,"5min":1.6519999999999999,"15min":3.5070000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.073,"5min":1.093,"15min":1.113},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56200000000000006},"max":{"1min":1.3460000000000001,"5min":2.0070000000000001,"15min":5.2149999999999999},"last":1.028}]},{"osd":4,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.224,"15min":1.2609999999999999},"min":{"1min":0.81499999999999995,"5min":0.80500000000000005,"15min":0.66700000000000004},"max":{"1min":1.5900000000000001,"5min":1.944,"15min":6.2000000000000002},"last":1.095},{"interface":"front","average":{"1min":1.165,"5min":1.1870000000000001,"15min":1.196},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.3939999999999999,"5min":1.6779999999999999,"15min":3.577},"last":1.2909999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.23,"15min":1.25},"min":{"1min":0.93000000000000005,"5min":0.81499999999999995,"15min":0.69599999999999995},"max":{"1min":1.478,"5min":1.857,"15min":3.548},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.21,"5min":1.278,"15min":1.2969999999999999},"min":{"1min":0.86199999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.4330000000000001,"5min":1.774,"15min":4.4080000000000004},"last":1.0700000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3180000000000001,"15min":1.349},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.82299999999999995},"max":{"1min":1.5640000000000001,"5min":1.915,"15min":4.4489999999999998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.2889999999999999,"15min":1.3340000000000001},"min":{"1min":0.93300000000000005,"5min":0.86499999999999999,"15min":0.75900000000000001},"max":{"1min":1.5409999999999999,"5min":2.056,"15min":6.9690000000000003},"last":1.1879999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.327,"15min":1.401},"min":{"1min":1.036,"5min":0.90400000000000003,"15min":0.79900000000000004},"max":{"1min":1.6319999999999999,"5min":1.994,"15min":18.253},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2889999999999999,"15min":1.3380000000000001},"min":{"1min":0.997,"5min":0.95599999999999996,"15min":0.79000000000000004},"max":{"1min":1.452,"5min":1.8720000000000001,"15min":11.006},"last":1.3779999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575027,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303036,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068804,"statfs":{"total":96636764160,"available":96326455296,"internally_reserved":0,"allocated":1888256,"data_stored":957002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0800000000000001,"15min":1.073},"min":{"1min":0.65800000000000003,"5min":0.627,"15min":0.49399999999999999},"max":{"1min":1.827,"5min":1.827,"15min":2.3370000000000002},"last":1.411},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0589999999999999},"min":{"1min":0.63600000000000001,"5min":0.61099999999999999,"15min":0.60799999999999998},"max":{"1min":1.3580000000000001,"5min":1.6519999999999999,"15min":3.46},"last":0.79800000000000004}]},{"osd":1,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.105,"15min":1.121},"min":{"1min":0.84599999999999997,"5min":0.71199999999999997,"15min":0.58999999999999997},"max":{"1min":1.4570000000000001,"5min":1.516,"15min":3.6349999999999998},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.161,"15min":1.143},"min":{"1min":0.84199999999999997,"5min":0.73999999999999999,"15min":0.63700000000000001},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":3.516},"last":1.6499999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.1459999999999999,"15min":1.165},"min":{"1min":0.88600000000000001,"5min":0.74099999999999999,"15min":0.73199999999999998},"max":{"1min":1.774,"5min":1.774,"15min":5.4180000000000001},"last":1.119},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1899999999999999,"15min":1.204},"min":{"1min":0.77300000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6839999999999999,"5min":2.0840000000000001,"15min":3.5859999999999999},"last":1.8100000000000001}]},{"osd":3,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2070000000000001,"15min":1.196},"min":{"1min":0.91400000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6439999999999999,"5min":1.8560000000000001,"15min":2.141},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.296,"15min":1.3169999999999999},"min":{"1min":0.88400000000000001,"5min":0.875,"15min":0.78600000000000003},"max":{"1min":1.6140000000000001,"5min":1.6739999999999999,"15min":4.2729999999999997},"last":1.6060000000000001}]},{"osd":4,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.2130000000000001,"15min":1.26},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.4850000000000001,"5min":1.591,"15min":3.706},"last":1.992},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.258,"15min":1.3080000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.64100000000000001},"max":{"1min":1.863,"5min":1.9219999999999999,"15min":6.1180000000000003},"last":1.2490000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.268,"15min":1.254},"min":{"1min":0.88800000000000001,"5min":0.80700000000000005,"15min":0.749},"max":{"1min":1.6080000000000001,"5min":1.843,"15min":4.2309999999999999},"last":1.698},{"interface":"front","average":{"1min":1.258,"5min":1.2869999999999999,"15min":1.341},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.546,"5min":2.2240000000000002,"15min":3.641},"last":1.907}]},{"osd":7,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.284,"15min":1.2949999999999999},"min":{"1min":0.83499999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6970000000000001,"5min":1.988,"15min":3.516},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.379,"15min":1.387},"min":{"1min":0.95899999999999996,"5min":0.94299999999999995,"15min":0.89500000000000002},"max":{"1min":1.925,"5min":2.282,"15min":3.754},"last":1.8340000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575026,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303036,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068804,"statfs":{"total":96636764160,"available":96326455296,"internally_reserved":0,"allocated":1888256,"data_stored":951294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.80500000000000005,"15min":0.81000000000000005},"min":{"1min":0.63,"5min":0.504,"15min":0.48799999999999999},"max":{"1min":1.22,"5min":1.671,"15min":1.671},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.85599999999999998,"15min":0.879},"min":{"1min":0.52700000000000002,"5min":0.502,"15min":0.46600000000000003},"max":{"1min":1.1799999999999999,"5min":1.3939999999999999,"15min":1.6899999999999999},"last":0.49399999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97799999999999998,"15min":0.94099999999999995},"min":{"1min":0.755,"5min":0.60999999999999999,"15min":0.44600000000000001},"max":{"1min":1.306,"5min":1.806,"15min":1.806},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.872,"5min":0.89000000000000001,"15min":0.90500000000000003},"min":{"1min":0.63200000000000001,"5min":0.55800000000000005,"15min":0.49199999999999999},"max":{"1min":1.2509999999999999,"5min":1.3360000000000001,"15min":1.887},"last":0.746}]},{"osd":3,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.129,"15min":1.125},"min":{"1min":0.747,"5min":0.66300000000000003,"15min":0.42399999999999999},"max":{"1min":1.476,"5min":1.6160000000000001,"15min":1.712},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0549999999999999,"15min":1.042},"min":{"1min":0.68500000000000005,"5min":0.52300000000000002,"15min":0.46000000000000002},"max":{"1min":1.4450000000000001,"5min":1.5409999999999999,"15min":1.7090000000000001},"last":0.58199999999999996}]},{"osd":4,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.208,"15min":1.1930000000000001},"min":{"1min":0.83699999999999997,"5min":0.80600000000000005,"15min":0.65100000000000002},"max":{"1min":1.671,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.194,"5min":1.1970000000000001,"15min":1.212},"min":{"1min":0.97399999999999998,"5min":0.83799999999999997,"15min":0.50800000000000001},"max":{"1min":1.4710000000000001,"5min":1.5429999999999999,"15min":2.206},"last":1.1479999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2649999999999999,"15min":1.2470000000000001},"min":{"1min":0.98499999999999999,"5min":0.90300000000000002,"15min":0.746},"max":{"1min":1.4430000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.871},{"interface":"front","average":{"1min":1.248,"5min":1.2509999999999999,"15min":1.256},"min":{"1min":1.038,"5min":0.92300000000000004,"15min":0.56100000000000005},"max":{"1min":1.4019999999999999,"5min":1.718,"15min":2.4609999999999999},"last":1.0189999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.327,"15min":1.327},"min":{"1min":1.1020000000000001,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.671,"5min":1.8640000000000001,"15min":1.948},"last":1.28},{"interface":"front","average":{"1min":1.429,"5min":1.431,"15min":1.4079999999999999},"min":{"1min":1.2230000000000001,"5min":0.91600000000000004,"15min":0.87},"max":{"1min":1.786,"5min":1.9670000000000001,"15min":2.3290000000000002},"last":1.008}]},{"osd":7,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3149999999999999,"15min":1.3169999999999999},"min":{"1min":1.1000000000000001,"5min":0.92000000000000004,"15min":0.84199999999999997},"max":{"1min":1.595,"5min":2.089,"15min":2.161},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.3999999999999999,"15min":1.3779999999999999},"min":{"1min":1.095,"5min":1.0249999999999999,"15min":0.78600000000000003},"max":{"1min":1.899,"5min":2.137,"15min":2.2349999999999999},"last":1.171}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T03:11:26.558 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-15T03:11:26.558 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-15T03:11:26.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.752+0000 7ff1fbfff700 1 Processor -- start 2023-11-15T03:11:26.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.752+0000 7ff1fbfff700 1 -- start start 2023-11-15T03:11:26.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.756+0000 7ff1fbfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff1fc0ab0d0 0x7ff1fc0ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:26.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.756+0000 7ff1fbfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1fc0ad1b0 0x7ff1fc065710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:26.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.756+0000 7ff1fbfff700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1fc066c90 con 0x7ff1fc065c00 2023-11-15T03:11:26.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.756+0000 7ff1fbfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1fc066080 con 0x7ff1fc0ab0d0 2023-11-15T03:11:26.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.756+0000 7ff1fbfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc066360 con 0x7ff1fc0ad1b0 2023-11-15T03:11:26.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.756+0000 7ff1fa7fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1fc0ad1b0 0x7ff1fc065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:26.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.756+0000 7ff1faffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff1fc0ab0d0 0x7ff1fc0ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:26.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.756+0000 7ff1fa7fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1fc0ad1b0 0x7ff1fc065710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:43632/0 (socket says 172.21.15.121:43632) 2023-11-15T03:11:26.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.756+0000 7ff1faffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff1fc0ab0d0 0x7ff1fc0ab4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:51140/0 (socket says 172.21.15.121:51140) 2023-11-15T03:11:26.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.756+0000 7ff1fa7fc700 1 -- 172.21.15.121:0/1954525538 learned_addr learned my addr 172.21.15.121:0/1954525538 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:26.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.756+0000 7ff1fa7fc700 1 -- 172.21.15.121:0/1954525538 >> v2:172.21.15.190:3300/0 conn(0x7ff1fc0ab0d0 msgr2=0x7ff1fc0ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:26.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.756+0000 7ff1fa7fc700 1 --2- 172.21.15.121:0/1954525538 >> v2:172.21.15.190:3300/0 conn(0x7ff1fc0ab0d0 0x7ff1fc0ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:26.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.756+0000 7ff1f9ffb700 1 -- 172.21.15.121:0/1954525538 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302301278 0 0) 0x7ff1fc066c90 con 0x7ff1fc065c00 2023-11-15T03:11:26.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.756+0000 7ff1fa7fc700 1 -- 172.21.15.121:0/1954525538 >> v1:172.21.15.121:6789/0 conn(0x7ff1fc065c00 legacy=0x7ff1fc0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:26.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.756+0000 7ff1fa7fc700 1 -- 172.21.15.121:0/1954525538 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1fc066640 con 0x7ff1fc0ad1b0 2023-11-15T03:11:26.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.760+0000 7ff1fa7fc700 1 --2- 172.21.15.121:0/1954525538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1fc0ad1b0 0x7ff1fc065710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7ff1f0006fd0 tx=0x7ff1f00090f0).ready entity=mon.2 client_cookie=bbe7a08cd81f5db1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:26.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.760+0000 7ff1f9ffb700 1 -- 172.21.15.121:0/1954525538 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f000b580 con 0x7ff1fc0ad1b0 2023-11-15T03:11:26.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.760+0000 7ff1f9ffb700 1 -- 172.21.15.121:0/1954525538 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f000b6e0 con 0x7ff1fc0ad1b0 2023-11-15T03:11:26.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.760+0000 7ff1f9ffb700 1 -- 172.21.15.121:0/1954525538 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f000ba60 con 0x7ff1fc0ad1b0 2023-11-15T03:11:26.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.760+0000 7ff1fbfff700 1 -- 172.21.15.121:0/1954525538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1fc0ad1b0 msgr2=0x7ff1fc065710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:26.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.760+0000 7ff1fbfff700 1 --2- 172.21.15.121:0/1954525538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1fc0ad1b0 0x7ff1fc065710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7ff1f0006fd0 tx=0x7ff1f00090f0).stop 2023-11-15T03:11:26.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.760+0000 7ff1fbfff700 1 -- 172.21.15.121:0/1954525538 shutdown_connections 2023-11-15T03:11:26.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.760+0000 7ff1fbfff700 1 --2- 172.21.15.121:0/1954525538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1fc0ad1b0 0x7ff1fc065710 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:26.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.760+0000 7ff1fbfff700 1 --2- 172.21.15.121:0/1954525538 >> v2:172.21.15.190:3300/0 conn(0x7ff1fc0ab0d0 0x7ff1fc0ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:26.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.760+0000 7ff1fbfff700 1 -- 172.21.15.121:0/1954525538 >> 172.21.15.121:0/1954525538 conn(0x7ff1fc067a60 msgr2=0x7ff1fc067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:26.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.760+0000 7ff1fbfff700 1 -- 172.21.15.121:0/1954525538 shutdown_connections 2023-11-15T03:11:26.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.760+0000 7ff1fbfff700 1 -- 172.21.15.121:0/1954525538 wait complete. 2023-11-15T03:11:26.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.760+0000 7ff1fbfff700 1 Processor -- start 2023-11-15T03:11:26.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fbfff700 1 -- start start 2023-11-15T03:11:26.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fbfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1fc065c00 0x7ff1fc13e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:26.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fbfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1fc0ab0d0 0x7ff1fc0a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:26.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fbfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1fc0ad1b0 0x7ff1fc0c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:26.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fbfff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc0a47f0 con 0x7ff1fc0ad1b0 2023-11-15T03:11:26.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fbfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc0a4230 con 0x7ff1fc0ab0d0 2023-11-15T03:11:26.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fbfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc0a4510 con 0x7ff1fc065c00 2023-11-15T03:11:26.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fb7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1fc0ad1b0 0x7ff1fc0c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:26.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fb7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1fc0ad1b0 0x7ff1fc0c8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40498/0 (socket says 172.21.15.121:40498) 2023-11-15T03:11:26.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fb7fe700 1 -- 172.21.15.121:0/1266313998 learned_addr learned my addr 172.21.15.121:0/1266313998 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:26.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fa7fc700 1 --2- 172.21.15.121:0/1266313998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1fc0ab0d0 0x7ff1fc0a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:26.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1faffd700 1 --2- 172.21.15.121:0/1266313998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1fc065c00 0x7ff1fc13e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:26.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fb7fe700 1 -- 172.21.15.121:0/1266313998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1fc065c00 msgr2=0x7ff1fc13e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:26.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fb7fe700 1 --2- 172.21.15.121:0/1266313998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1fc065c00 0x7ff1fc13e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:26.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fb7fe700 1 -- 172.21.15.121:0/1266313998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1fc0ab0d0 msgr2=0x7ff1fc0a38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:26.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fb7fe700 1 --2- 172.21.15.121:0/1266313998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1fc0ab0d0 0x7ff1fc0a38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:26.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fb7fe700 1 -- 172.21.15.121:0/1266313998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f0009370 con 0x7ff1fc0ad1b0 2023-11-15T03:11:26.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fa7fc700 1 --2- 172.21.15.121:0/1266313998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1fc0ab0d0 0x7ff1fc0a38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:26.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fb7fe700 1 --2- 172.21.15.121:0/1266313998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1fc0ad1b0 0x7ff1fc0c8c80 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 rx=0x7ff1f400c810 tx=0x7ff1f4007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:26.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1e2ffd700 1 -- 172.21.15.121:0/1266313998 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4010510 con 0x7ff1fc0ad1b0 2023-11-15T03:11:26.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1e2ffd700 1 -- 172.21.15.121:0/1266313998 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f4010670 con 0x7ff1fc0ad1b0 2023-11-15T03:11:26.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fbfff700 1 -- 172.21.15.121:0/1266313998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1fc0c8480 con 0x7ff1fc0ad1b0 2023-11-15T03:11:26.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1e2ffd700 1 -- 172.21.15.121:0/1266313998 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4010a10 con 0x7ff1fc0ad1b0 2023-11-15T03:11:26.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.764+0000 7ff1fbfff700 1 -- 172.21.15.121:0/1266313998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1fc0a4ba0 con 0x7ff1fc0ad1b0 2023-11-15T03:11:26.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.768+0000 7ff1fbfff700 1 -- 172.21.15.121:0/1266313998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1fc06bcf0 con 0x7ff1fc0ad1b0 2023-11-15T03:11:26.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.768+0000 7ff1e2ffd700 1 -- 172.21.15.121:0/1266313998 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff1f401b400 con 0x7ff1fc0ad1b0 2023-11-15T03:11:26.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.768+0000 7ff1e2ffd700 1 --2- 172.21.15.121:0/1266313998 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff1e805a510 0x7ff1e805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:26.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.768+0000 7ff1e2ffd700 1 -- 172.21.15.121:0/1266313998 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff1f4074190 con 0x7ff1fc0ad1b0 2023-11-15T03:11:26.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.772+0000 7ff1faffd700 1 --2- 172.21.15.121:0/1266313998 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff1e805a510 0x7ff1e805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:26.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.772+0000 7ff1faffd700 1 --2- 172.21.15.121:0/1266313998 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff1e805a510 0x7ff1e805c9d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7ff1ec009200 tx=0x7ff1ec008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:26.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.776+0000 7ff1e2ffd700 1 -- 172.21.15.121:0/1266313998 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff1f4047850 con 0x7ff1fc0ad1b0 2023-11-15T03:11:27.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:26.992+0000 7ff1fbfff700 1 -- 172.21.15.121:0/1266313998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff1fc0c7160 con 0x7ff1fc0ad1b0 2023-11-15T03:11:27.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.000+0000 7ff1e2ffd700 1 -- 172.21.15.121:0/1266313998 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff1f404ae70 con 0x7ff1fc0ad1b0 2023-11-15T03:11:27.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.004+0000 7ff1fbfff700 1 -- 172.21.15.121:0/1266313998 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff1e805a510 msgr2=0x7ff1e805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:27.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.004+0000 7ff1fbfff700 1 --2- 172.21.15.121:0/1266313998 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff1e805a510 0x7ff1e805c9d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7ff1ec009200 tx=0x7ff1ec008040).stop 2023-11-15T03:11:27.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.004+0000 7ff1fbfff700 1 -- 172.21.15.121:0/1266313998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1fc0ad1b0 msgr2=0x7ff1fc0c8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:27.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.004+0000 7ff1fbfff700 1 --2- 172.21.15.121:0/1266313998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1fc0ad1b0 0x7ff1fc0c8c80 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 rx=0x7ff1f400c810 tx=0x7ff1f4007f90).stop 2023-11-15T03:11:27.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.004+0000 7ff1fbfff700 1 -- 172.21.15.121:0/1266313998 shutdown_connections 2023-11-15T03:11:27.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.004+0000 7ff1fbfff700 1 --2- 172.21.15.121:0/1266313998 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff1e805a510 0x7ff1e805c9d0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:27.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.004+0000 7ff1fbfff700 1 --2- 172.21.15.121:0/1266313998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1fc0ad1b0 0x7ff1fc0c8c80 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:27.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.004+0000 7ff1fbfff700 1 --2- 172.21.15.121:0/1266313998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1fc0ab0d0 0x7ff1fc0a38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:27.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.004+0000 7ff1fbfff700 1 --2- 172.21.15.121:0/1266313998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1fc065c00 0x7ff1fc13e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:27.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.004+0000 7ff1fbfff700 1 -- 172.21.15.121:0/1266313998 >> 172.21.15.121:0/1266313998 conn(0x7ff1fc067a60 msgr2=0x7ff1fc1541a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:27.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.004+0000 7ff1fbfff700 1 -- 172.21.15.121:0/1266313998 shutdown_connections 2023-11-15T03:11:27.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.004+0000 7ff1fbfff700 1 -- 172.21.15.121:0/1266313998 wait complete. 2023-11-15T03:11:27.015 INFO:teuthology.orchestra.run.smithi121.stderr:updated caps for client.0 2023-11-15T03:11:27.034 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:27.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.192+0000 7f78c005b700 1 Processor -- start 2023-11-15T03:11:27.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.192+0000 7f78c005b700 1 -- start start 2023-11-15T03:11:27.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.196+0000 7f78c005b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78b80a5ed0 0x7f78b8155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:27.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.196+0000 7f78c005b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b80c4c30 con 0x7f78b8154ca0 2023-11-15T03:11:27.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.196+0000 7f78c005b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b80a6f50 con 0x7f78b81562f0 2023-11-15T03:11:27.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.196+0000 7f78c005b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b80a8280 con 0x7f78b80a5ed0 2023-11-15T03:11:27.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.196+0000 7f78bddf7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f78b8154ca0 0x7f78b80a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60882/0 (socket says 172.21.15.121:60882) 2023-11-15T03:11:27.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.196+0000 7f78bddf7700 1 -- 172.21.15.121:0/1415761231 learned_addr learned my addr 172.21.15.121:0/1415761231 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:27.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.196+0000 7f78be5f8700 1 --2- 172.21.15.121:0/1415761231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78b80a5ed0 0x7f78b8155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:27.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.196+0000 7f78bcdf5700 1 -- 172.21.15.121:0/1415761231 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3571335765 0 0) 0x7f78b80c4c30 con 0x7f78b8154ca0 2023-11-15T03:11:27.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.200+0000 7f78bcdf5700 1 -- 172.21.15.121:0/1415761231 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78a40033f0 con 0x7f78b8154ca0 2023-11-15T03:11:27.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.200+0000 7f78bcdf5700 1 -- 172.21.15.121:0/1415761231 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353854016 0 0) 0x7f78b80a6f50 con 0x7f78b81562f0 2023-11-15T03:11:27.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.200+0000 7f78bcdf5700 1 -- 172.21.15.121:0/1415761231 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78b80c4c30 con 0x7f78b81562f0 2023-11-15T03:11:27.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.200+0000 7f78be5f8700 1 -- 172.21.15.121:0/1415761231 >> v1:172.21.15.190:6789/0 conn(0x7f78b81562f0 legacy=0x7f78b80a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:27.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.200+0000 7f78be5f8700 1 -- 172.21.15.121:0/1415761231 >> v1:172.21.15.121:6789/0 conn(0x7f78b8154ca0 legacy=0x7f78b80a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:27.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.200+0000 7f78be5f8700 1 -- 172.21.15.121:0/1415761231 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b80a7230 con 0x7f78b80a5ed0 2023-11-15T03:11:27.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.200+0000 7f78be5f8700 1 --2- 172.21.15.121:0/1415761231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78b80a5ed0 0x7f78b8155080 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f78b0009e90 tx=0x7f78b0009780).ready entity=mon.2 client_cookie=66839cd2fb4e47b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:27.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.200+0000 7f78bcdf5700 1 -- 172.21.15.121:0/1415761231 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b000e070 con 0x7f78b80a5ed0 2023-11-15T03:11:27.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.200+0000 7f78bcdf5700 1 -- 172.21.15.121:0/1415761231 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b000bab0 con 0x7f78b80a5ed0 2023-11-15T03:11:27.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.200+0000 7f78bcdf5700 1 -- 172.21.15.121:0/1415761231 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b000be30 con 0x7f78b80a5ed0 2023-11-15T03:11:27.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.200+0000 7f78c005b700 1 -- 172.21.15.121:0/1415761231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78b80a5ed0 msgr2=0x7f78b8155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:27.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.200+0000 7f78c005b700 1 --2- 172.21.15.121:0/1415761231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78b80a5ed0 0x7f78b8155080 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f78b0009e90 tx=0x7f78b0009780).stop 2023-11-15T03:11:27.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.200+0000 7f78c005b700 1 -- 172.21.15.121:0/1415761231 shutdown_connections 2023-11-15T03:11:27.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.200+0000 7f78c005b700 1 --2- 172.21.15.121:0/1415761231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78b80a5ed0 0x7f78b8155080 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:27.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.200+0000 7f78c005b700 1 -- 172.21.15.121:0/1415761231 >> 172.21.15.121:0/1415761231 conn(0x7f78b8062190 msgr2=0x7f78b80625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:27.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.200+0000 7f78c005b700 1 -- 172.21.15.121:0/1415761231 shutdown_connections 2023-11-15T03:11:27.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.200+0000 7f78c005b700 1 -- 172.21.15.121:0/1415761231 wait complete. 2023-11-15T03:11:27.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78c005b700 1 Processor -- start 2023-11-15T03:11:27.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78c005b700 1 -- start start 2023-11-15T03:11:27.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78c005b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78b80a5ed0 0x7f78b8151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:27.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78c005b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78b8154ca0 0x7f78b814d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:27.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78c005b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78b81562f0 0x7f78b80ab840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:27.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78c005b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b8152820 con 0x7f78b80a5ed0 2023-11-15T03:11:27.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78c005b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b8152b00 con 0x7f78b81562f0 2023-11-15T03:11:27.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78c005b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b8152de0 con 0x7f78b8154ca0 2023-11-15T03:11:27.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78bddf7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78b80a5ed0 0x7f78b8151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:27.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78bddf7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78b80a5ed0 0x7f78b8151ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40506/0 (socket says 172.21.15.121:40506) 2023-11-15T03:11:27.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78bddf7700 1 -- 172.21.15.121:0/1266545527 learned_addr learned my addr 172.21.15.121:0/1266545527 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:27.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78be5f8700 1 --2- 172.21.15.121:0/1266545527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78b81562f0 0x7f78b80ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:27.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78bd5f6700 1 --2- 172.21.15.121:0/1266545527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78b8154ca0 0x7f78b814d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:27.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78bddf7700 1 -- 172.21.15.121:0/1266545527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78b8154ca0 msgr2=0x7f78b814d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:27.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78bddf7700 1 --2- 172.21.15.121:0/1266545527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78b8154ca0 0x7f78b814d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:27.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78bddf7700 1 -- 172.21.15.121:0/1266545527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78b81562f0 msgr2=0x7f78b80ab840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:27.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78bddf7700 1 --2- 172.21.15.121:0/1266545527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78b81562f0 0x7f78b80ab840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:27.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78bddf7700 1 -- 172.21.15.121:0/1266545527 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b0009a20 con 0x7f78b80a5ed0 2023-11-15T03:11:27.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78bddf7700 1 --2- 172.21.15.121:0/1266545527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78b80a5ed0 0x7f78b8151ed0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 rx=0x7f78b400c560 tx=0x7f78b4007d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:27.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78adffb700 1 -- 172.21.15.121:0/1266545527 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b40072e0 con 0x7f78b80a5ed0 2023-11-15T03:11:27.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78adffb700 1 -- 172.21.15.121:0/1266545527 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b4007440 con 0x7f78b80a5ed0 2023-11-15T03:11:27.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78adffb700 1 -- 172.21.15.121:0/1266545527 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b40057a0 con 0x7f78b80a5ed0 2023-11-15T03:11:27.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.204+0000 7f78c005b700 1 -- 172.21.15.121:0/1266545527 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b81530c0 con 0x7f78b80a5ed0 2023-11-15T03:11:27.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.208+0000 7f78c005b700 1 -- 172.21.15.121:0/1266545527 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b81525e0 con 0x7f78b80a5ed0 2023-11-15T03:11:27.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.208+0000 7f78c005b700 1 -- 172.21.15.121:0/1266545527 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b806c150 con 0x7f78b80a5ed0 2023-11-15T03:11:27.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.208+0000 7f78adffb700 1 -- 172.21.15.121:0/1266545527 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f78b4005b90 con 0x7f78b80a5ed0 2023-11-15T03:11:27.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.212+0000 7f78adffb700 1 --2- 172.21.15.121:0/1266545527 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f789c05a510 0x7f789c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:27.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.212+0000 7f78adffb700 1 -- 172.21.15.121:0/1266545527 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f78b4073f20 con 0x7f78b80a5ed0 2023-11-15T03:11:27.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.212+0000 7f78bd5f6700 1 --2- 172.21.15.121:0/1266545527 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f789c05a510 0x7f789c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:27.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.212+0000 7f78bd5f6700 1 --2- 172.21.15.121:0/1266545527 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f789c05a510 0x7f789c05c9d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f78b8165540 tx=0x7f78a8009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:27.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.216+0000 7f78adffb700 1 -- 172.21.15.121:0/1266545527 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f78b4047640 con 0x7f78b80a5ed0 2023-11-15T03:11:27.405 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:11:27.396+0000 7f93a56bd700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:11:27.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.416+0000 7f78c005b700 1 -- 172.21.15.121:0/1266545527 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78b81525e0 con 0x7f78b80a5ed0 2023-11-15T03:11:27.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.416+0000 7f78adffb700 1 -- 172.21.15.121:0/1266545527 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) v1 ==== 95+0+4512 (secure 0 0 0) 0x7f78b81525e0 con 0x7f78b80a5ed0 2023-11-15T03:11:27.426 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:27.426 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":350,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/755048854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":755048854},{"type":"v1","addr":"172.21.15.190:6834","nonce":755048854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2360242049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2360242049},{"type":"v1","addr":"172.21.15.190:6835","nonce":2360242049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2780153683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2780153683},{"type":"v1","addr":"172.21.15.121:6835","nonce":2780153683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345}],"filesystems":[{"mdsmap":{"epoch":350,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:11:21.864101+0000","modified":"2023-11-15T03:11:24.889953+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10716},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10716":{"gid":10716,"name":"a","rank":0,"incarnation":346,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6837/2335356851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2335356851},{"type":"v1","addr":"172.21.15.121:6837","nonce":2335356851}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-15T03:11:27.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.420+0000 7f78c005b700 1 -- 172.21.15.121:0/1266545527 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f789c05a510 msgr2=0x7f789c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:27.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.420+0000 7f78c005b700 1 --2- 172.21.15.121:0/1266545527 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f789c05a510 0x7f789c05c9d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f78b8165540 tx=0x7f78a8009040).stop 2023-11-15T03:11:27.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.420+0000 7f78c005b700 1 -- 172.21.15.121:0/1266545527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78b80a5ed0 msgr2=0x7f78b8151ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:27.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.420+0000 7f78c005b700 1 --2- 172.21.15.121:0/1266545527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78b80a5ed0 0x7f78b8151ed0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 rx=0x7f78b400c560 tx=0x7f78b4007d20).stop 2023-11-15T03:11:27.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.420+0000 7f78c005b700 1 -- 172.21.15.121:0/1266545527 shutdown_connections 2023-11-15T03:11:27.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.420+0000 7f78c005b700 1 --2- 172.21.15.121:0/1266545527 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f789c05a510 0x7f789c05c9d0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:27.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.420+0000 7f78c005b700 1 --2- 172.21.15.121:0/1266545527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78b81562f0 0x7f78b80ab840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:27.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.420+0000 7f78c005b700 1 --2- 172.21.15.121:0/1266545527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78b8154ca0 0x7f78b814d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:27.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.420+0000 7f78c005b700 1 --2- 172.21.15.121:0/1266545527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78b80a5ed0 0x7f78b8151ed0 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:27.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.420+0000 7f78c005b700 1 -- 172.21.15.121:0/1266545527 >> 172.21.15.121:0/1266545527 conn(0x7f78b8062190 msgr2=0x7f78b80c50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:27.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.424+0000 7f78c005b700 1 -- 172.21.15.121:0/1266545527 shutdown_connections 2023-11-15T03:11:27.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:27.424+0000 7f78c005b700 1 -- 172.21.15.121:0/1266545527 wait complete. 2023-11-15T03:11:27.434 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 350 2023-11-15T03:11:27.855 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 350, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:11:21.864101+0000', 'modified': '2023-11-15T03:11:24.889953+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10716}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10716': {'gid': 10716, 'name': 'a', 'rank': 0, 'incarnation': 346, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.121:6837/2335356851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6836', 'nonce': 2335356851}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 2335356851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:11:27.856 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T03:11:27.856 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-15T03:11:28.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.012+0000 7f74ee479700 1 Processor -- start 2023-11-15T03:11:28.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.012+0000 7f74ee479700 1 -- start start 2023-11-15T03:11:28.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.016+0000 7f74ee479700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f74e80669b0 0x7f74e8158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.016+0000 7f74ee479700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e80a8750 0x7f74e80a8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.016+0000 7f74ee479700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e80a90d0 con 0x7f74e8157d90 2023-11-15T03:11:28.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.016+0000 7f74ee479700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e80c1fb0 con 0x7f74e80669b0 2023-11-15T03:11:28.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.016+0000 7f74ee479700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e815bbd0 con 0x7f74e80a8750 2023-11-15T03:11:28.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.016+0000 7f74ecc76700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f74e80669b0 0x7f74e8158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.016+0000 7f74ed477700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f74e8157d90 0x7f74e80c1760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:49942/0 (socket says 172.21.15.121:49942) 2023-11-15T03:11:28.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.016+0000 7f74ed477700 1 -- 172.21.15.121:0/1105268261 learned_addr learned my addr 172.21.15.121:0/1105268261 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:28.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.020+0000 7f74edc78700 1 --2- 172.21.15.121:0/1105268261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e80a8750 0x7f74e80a8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.020+0000 7f74ecc76700 1 -- 172.21.15.121:0/1105268261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e80a8750 msgr2=0x7f74e80a8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:28.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.020+0000 7f74ecc76700 1 --2- 172.21.15.121:0/1105268261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e80a8750 0x7f74e80a8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.020+0000 7f74ecc76700 1 -- 172.21.15.121:0/1105268261 >> v1:172.21.15.190:6789/0 conn(0x7f74e8157d90 legacy=0x7f74e80c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:28.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.020+0000 7f74ecc76700 1 -- 172.21.15.121:0/1105268261 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e80bb910 con 0x7f74e80669b0 2023-11-15T03:11:28.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.020+0000 7f74edc78700 1 --2- 172.21.15.121:0/1105268261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e80a8750 0x7f74e80a8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:28.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.020+0000 7f74ecc76700 1 --2- 172.21.15.121:0/1105268261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74e80669b0 0x7f74e8158170 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 rx=0x7f74d8006fd0 tx=0x7f74d8006f40).ready entity=mon.0 client_cookie=41d07870b3e5ac20 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:28.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.020+0000 7f74dffff700 1 -- 172.21.15.121:0/1105268261 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8005880 con 0x7f74e80669b0 2023-11-15T03:11:28.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.020+0000 7f74dffff700 1 -- 172.21.15.121:0/1105268261 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d80059e0 con 0x7f74e80669b0 2023-11-15T03:11:28.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.020+0000 7f74dffff700 1 -- 172.21.15.121:0/1105268261 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8005d60 con 0x7f74e80669b0 2023-11-15T03:11:28.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.020+0000 7f74ee479700 1 -- 172.21.15.121:0/1105268261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74e80669b0 msgr2=0x7f74e8158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:28.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.020+0000 7f74ee479700 1 --2- 172.21.15.121:0/1105268261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74e80669b0 0x7f74e8158170 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 rx=0x7f74d8006fd0 tx=0x7f74d8006f40).stop 2023-11-15T03:11:28.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.020+0000 7f74ee479700 1 -- 172.21.15.121:0/1105268261 shutdown_connections 2023-11-15T03:11:28.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.020+0000 7f74ee479700 1 --2- 172.21.15.121:0/1105268261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e80a8750 0x7f74e80a8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.020+0000 7f74ee479700 1 --2- 172.21.15.121:0/1105268261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74e80669b0 0x7f74e8158170 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.020+0000 7f74ee479700 1 -- 172.21.15.121:0/1105268261 >> 172.21.15.121:0/1105268261 conn(0x7f74e8067a60 msgr2=0x7f74e8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:28.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.020+0000 7f74ee479700 1 -- 172.21.15.121:0/1105268261 shutdown_connections 2023-11-15T03:11:28.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.020+0000 7f74ee479700 1 -- 172.21.15.121:0/1105268261 wait complete. 2023-11-15T03:11:28.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.024+0000 7f74ee479700 1 Processor -- start 2023-11-15T03:11:28.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.024+0000 7f74ee479700 1 -- start start 2023-11-15T03:11:28.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74ee479700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74e80669b0 0x7f74e814c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74ee479700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74e80a8750 0x7f74e816cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74ee479700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e8157d90 0x7f74e814f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74ee479700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e814c580 con 0x7f74e80669b0 2023-11-15T03:11:28.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74ee479700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e814cf60 con 0x7f74e80a8750 2023-11-15T03:11:28.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74ee479700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e814e3a0 con 0x7f74e8157d90 2023-11-15T03:11:28.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74ed477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74e80669b0 0x7f74e814c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74ed477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74e80669b0 0x7f74e814c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40518/0 (socket says 172.21.15.121:40518) 2023-11-15T03:11:28.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74ed477700 1 -- 172.21.15.121:0/425272874 learned_addr learned my addr 172.21.15.121:0/425272874 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:28.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74ed477700 1 -- 172.21.15.121:0/425272874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e8157d90 msgr2=0x7f74e814f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:28.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74ed477700 1 --2- 172.21.15.121:0/425272874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e8157d90 0x7f74e814f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:11:28.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74ed477700 1 -- 172.21.15.121:0/425272874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74e80a8750 msgr2=0x7f74e816cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:28.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74ecc76700 1 --2- 172.21.15.121:0/425272874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74e80a8750 0x7f74e816cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74ed477700 1 --2- 172.21.15.121:0/425272874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74e80a8750 0x7f74e816cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74ed477700 1 -- 172.21.15.121:0/425272874 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d8009250 con 0x7f74e80669b0 2023-11-15T03:11:28.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74ecc76700 1 --2- 172.21.15.121:0/425272874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74e80a8750 0x7f74e816cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:11:28.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74ed477700 1 --2- 172.21.15.121:0/425272874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74e80669b0 0x7f74e814c6e0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 rx=0x7f74e400c440 tx=0x7f74e4007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:28.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74dd7fa700 1 -- 172.21.15.121:0/425272874 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e40072e0 con 0x7f74e80669b0 2023-11-15T03:11:28.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74dd7fa700 1 -- 172.21.15.121:0/425272874 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e4007440 con 0x7f74e80669b0 2023-11-15T03:11:28.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74ee479700 1 -- 172.21.15.121:0/425272874 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e814cc80 con 0x7f74e80669b0 2023-11-15T03:11:28.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74dd7fa700 1 -- 172.21.15.121:0/425272874 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e40057a0 con 0x7f74e80669b0 2023-11-15T03:11:28.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.028+0000 7f74ee479700 1 -- 172.21.15.121:0/425272874 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e814ebc0 con 0x7f74e80669b0 2023-11-15T03:11:28.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.032+0000 7f74dd7fa700 1 -- 172.21.15.121:0/425272874 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f74e4005b10 con 0x7f74e80669b0 2023-11-15T03:11:28.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.032+0000 7f74dd7fa700 1 --2- 172.21.15.121:0/425272874 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f74cc05a510 0x7f74cc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.032+0000 7f74dd7fa700 1 -- 172.21.15.121:0/425272874 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f74e4073e50 con 0x7f74e80669b0 2023-11-15T03:11:28.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.032+0000 7f74ecc76700 1 --2- 172.21.15.121:0/425272874 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f74cc05a510 0x7f74cc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.036+0000 7f74ecc76700 1 --2- 172.21.15.121:0/425272874 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f74cc05a510 0x7f74cc05c9d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f74e80c0b20 tx=0x7f74d8016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:28.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.036+0000 7f74f9a50740 1 Processor -- start 2023-11-15T03:11:28.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.036+0000 7f74f9a50740 1 -- start start 2023-11-15T03:11:28.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.036+0000 7f74f9a50740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21698d0 0x2169cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.036+0000 7f74f9a50740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x216ab20 0x21afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.036+0000 7f74f9a50740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b00a0 0x21b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.036+0000 7f74f9a50740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x20ea410 con 0x216ab20 2023-11-15T03:11:28.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.036+0000 7f74f9a50740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x20ea550 con 0x21698d0 2023-11-15T03:11:28.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.036+0000 7f74f9a50740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x20eaa00 con 0x21b00a0 2023-11-15T03:11:28.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.036+0000 7f74ecc76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x216ab20 0x21afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.036+0000 7f74ecc76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x216ab20 0x21afbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40534/0 (socket says 172.21.15.121:40534) 2023-11-15T03:11:28.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.036+0000 7f74ecc76700 1 -- 172.21.15.121:0/3330392627 learned_addr learned my addr 172.21.15.121:0/3330392627 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:28.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.036+0000 7f74ed477700 1 --2- 172.21.15.121:0/3330392627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21698d0 0x2169cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74edc78700 1 --2- 172.21.15.121:0/3330392627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b00a0 0x21b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74ecc76700 1 -- 172.21.15.121:0/3330392627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b00a0 msgr2=0x21b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:28.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74ecc76700 1 --2- 172.21.15.121:0/3330392627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b00a0 0x21b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74ecc76700 1 -- 172.21.15.121:0/3330392627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21698d0 msgr2=0x2169cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:28.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74ecc76700 1 --2- 172.21.15.121:0/3330392627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21698d0 0x2169cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74ecc76700 1 -- 172.21.15.121:0/3330392627 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x20e3730 con 0x216ab20 2023-11-15T03:11:28.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74ed477700 1 --2- 172.21.15.121:0/3330392627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21698d0 0x2169cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:28.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74ecc76700 1 --2- 172.21.15.121:0/3330392627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x216ab20 0x21afbb0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 rx=0x7f74d8009e80 tx=0x7f74d801c000).ready entity=mon.0 client_cookie=5887bb0e0bf59d74 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:28.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74d27fc700 1 -- 172.21.15.121:0/3330392627 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d800b580 con 0x216ab20 2023-11-15T03:11:28.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74d27fc700 1 -- 172.21.15.121:0/3330392627 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d800b6e0 con 0x216ab20 2023-11-15T03:11:28.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74d27fc700 1 -- 172.21.15.121:0/3330392627 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d800ba60 con 0x216ab20 2023-11-15T03:11:28.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74f9a50740 1 -- 172.21.15.121:0/3330392627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x216ab20 msgr2=0x21afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:28.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74f9a50740 1 --2- 172.21.15.121:0/3330392627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x216ab20 0x21afbb0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 rx=0x7f74d8009e80 tx=0x7f74d801c000).stop 2023-11-15T03:11:28.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74f9a50740 1 -- 172.21.15.121:0/3330392627 shutdown_connections 2023-11-15T03:11:28.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74f9a50740 1 --2- 172.21.15.121:0/3330392627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b00a0 0x21b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74f9a50740 1 --2- 172.21.15.121:0/3330392627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x216ab20 0x21afbb0 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74f9a50740 1 --2- 172.21.15.121:0/3330392627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21698d0 0x2169cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74f9a50740 1 -- 172.21.15.121:0/3330392627 >> 172.21.15.121:0/3330392627 conn(0x1f48550 msgr2=0x21b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:28.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74f9a50740 1 -- 172.21.15.121:0/3330392627 shutdown_connections 2023-11-15T03:11:28.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74f9a50740 1 -- 172.21.15.121:0/3330392627 wait complete. 2023-11-15T03:11:28.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74f9a50740 1 -- start start 2023-11-15T03:11:28.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.040+0000 7f74f9a50740 1 Processor -- start 2023-11-15T03:11:28.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74f9a50740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21698d0 0x21962a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74f9a50740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x216ab20 0x2196790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74f9a50740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b00a0 0x2193ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74f9a50740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x20b5650 con 0x216ab20 2023-11-15T03:11:28.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74f9a50740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x21ac400 con 0x21698d0 2023-11-15T03:11:28.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74f9a50740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x21ac6e0 con 0x21b00a0 2023-11-15T03:11:28.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74f9a50740 10 client.0 ll_register_callbacks cb 0x2122020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:11:28.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74ecc76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x216ab20 0x2196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74ecc76700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x216ab20 0x2196790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40536/0 (socket says 172.21.15.121:40536) 2023-11-15T03:11:28.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74ecc76700 1 -- 172.21.15.121:0/2912861572 learned_addr learned my addr 172.21.15.121:0/2912861572 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:28.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74ed477700 1 --2- 172.21.15.121:0/2912861572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21698d0 0x21962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74edc78700 1 --2- 172.21.15.121:0/2912861572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b00a0 0x2193ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74ecc76700 1 -- 172.21.15.121:0/2912861572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b00a0 msgr2=0x2193ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:28.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74ecc76700 1 --2- 172.21.15.121:0/2912861572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b00a0 0x2193ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74ecc76700 1 -- 172.21.15.121:0/2912861572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21698d0 msgr2=0x21962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:28.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74ecc76700 1 --2- 172.21.15.121:0/2912861572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21698d0 0x21962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74ecc76700 1 -- 172.21.15.121:0/2912861572 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e80bb910 con 0x216ab20 2023-11-15T03:11:28.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74ecc76700 1 --2- 172.21.15.121:0/2912861572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x216ab20 0x2196790 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 rx=0x7f74d800a010 tx=0x7f74d8024000).ready entity=mon.0 client_cookie=da8c54c4fbf7d47 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:28.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:11:28.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74c77fe700 1 -- 172.21.15.121:0/2912861572 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8006040 con 0x216ab20 2023-11-15T03:11:28.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74c77fe700 1 -- 172.21.15.121:0/2912861572 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f74d8021030 con 0x216ab20 2023-11-15T03:11:28.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74f9a50740 1 -- 172.21.15.121:0/2912861572 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x20e01a0 con 0x216ab20 2023-11-15T03:11:28.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74c77fe700 1 -- 172.21.15.121:0/2912861572 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=350) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74d801c840 con 0x216ab20 2023-11-15T03:11:28.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74f9a50740 10 client.10863 fetch_fsmap learned FSMap version 350 2023-11-15T03:11:28.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74f9a50740 1 -- 172.21.15.121:0/2912861572 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=350,monmap=2+}) v3 -- 0x20de960 con 0x216ab20 2023-11-15T03:11:28.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74c77fe700 1 -- 172.21.15.121:0/2912861572 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x20e01a0 con 0x216ab20 2023-11-15T03:11:28.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74c77fe700 1 -- 172.21.15.121:0/2912861572 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 350) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f74d801c9a0 con 0x216ab20 2023-11-15T03:11:28.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74f9a50740 10 client.10863 fetch_fsmap finished waiting for FSMap version 350 2023-11-15T03:11:28.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74f9a50740 10 client.10863 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:11:28.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74f9a50740 20 client.10863 populate_metadata read hostname 'smithi121' 2023-11-15T03:11:28.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74f9a50740 1 --2- 172.21.15.121:0/2912861572 >> [v2:172.21.15.121:6836/2335356851,v1:172.21.15.121:6837/2335356851] conn(0x21ab920 0x21a8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74f9a50740 4 client.10863 mds_command: new command op to 10716 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:11:28.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.044+0000 7f74f9a50740 1 -- 172.21.15.121:0/2912861572 --> [v2:172.21.15.121:6836/2335356851,v1:172.21.15.121:6837/2335356851] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x20d66f0 con 0x21ab920 2023-11-15T03:11:28.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.048+0000 7f74ed477700 1 --2- 172.21.15.121:0/2912861572 >> [v2:172.21.15.121:6836/2335356851,v1:172.21.15.121:6837/2335356851] conn(0x21ab920 0x21a8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.048+0000 7f74ed477700 1 --2- 172.21.15.121:0/2912861572 >> [v2:172.21.15.121:6836/2335356851,v1:172.21.15.121:6837/2335356851] conn(0x21ab920 0x21a8f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=7f35d04ccf2767fd server_cookie=eef50df578fe70b6 in_seq=0 out_seq=0 2023-11-15T03:11:28.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.048+0000 7f74c77fe700 10 client.10863 ms_handle_connect on v2:172.21.15.121:6836/2335356851 2023-11-15T03:11:28.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.048+0000 7f74c77fe700 1 -- 172.21.15.121:0/2912861572 <== mds.0 v2:172.21.15.121:6836/2335356851 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x20d66f0 con 0x21ab920 2023-11-15T03:11:28.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.048+0000 7f74c77fe700 10 client.10863 handle_command_reply: tid=0 2023-11-15T03:11:28.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.048+0000 7f74f9a50740 1 client.10863 shutdown 2023-11-15T03:11:28.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.048+0000 7f74f9a50740 1 -- 172.21.15.121:0/2912861572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x216ab20 msgr2=0x2196790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:28.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.048+0000 7f74f9a50740 1 --2- 172.21.15.121:0/2912861572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x216ab20 0x2196790 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 rx=0x7f74d800a010 tx=0x7f74d8024000).stop 2023-11-15T03:11:28.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.048+0000 7f74f9a50740 1 -- 172.21.15.121:0/2912861572 shutdown_connections 2023-11-15T03:11:28.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.048+0000 7f74f9a50740 1 --2- 172.21.15.121:0/2912861572 >> [v2:172.21.15.121:6836/2335356851,v1:172.21.15.121:6837/2335356851] conn(0x21ab920 0x21a8f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.048+0000 7f74f9a50740 1 --2- 172.21.15.121:0/2912861572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21b00a0 0x2193ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.048+0000 7f74f9a50740 1 --2- 172.21.15.121:0/2912861572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x216ab20 0x2196790 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.048+0000 7f74f9a50740 1 --2- 172.21.15.121:0/2912861572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21698d0 0x21962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.048+0000 7f74f9a50740 1 -- 172.21.15.121:0/2912861572 >> 172.21.15.121:0/2912861572 conn(0x1f48550 msgr2=0x216af40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:28.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.048+0000 7f74c77fe700 0 client.10863 ms_handle_reset on v2:172.21.15.121:6836/2335356851 2023-11-15T03:11:28.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.048+0000 7f74f9a50740 1 -- 172.21.15.121:0/2912861572 shutdown_connections 2023-11-15T03:11:28.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.048+0000 7f74f9a50740 1 -- 172.21.15.121:0/2912861572 wait complete. 2023-11-15T03:11:28.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.048+0000 7f74f9a50740 20 client.10863 trim_cache size 0 max 16384 2023-11-15T03:11:28.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.068+0000 7f74f9a50740 1 Processor -- start 2023-11-15T03:11:28.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.068+0000 7f74f9a50740 1 -- start start 2023-11-15T03:11:28.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74f9a50740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21698d0 0x21a7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74f9a50740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x216ab20 0x216a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74f9a50740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b00a0 0x21ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74f9a50740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x21ac8a0 con 0x21b00a0 2023-11-15T03:11:28.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74f9a50740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x20b1130 con 0x216ab20 2023-11-15T03:11:28.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74f9a50740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x21a8a50 con 0x21698d0 2023-11-15T03:11:28.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74edc78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b00a0 0x21ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74edc78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b00a0 0x21ad530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40546/0 (socket says 172.21.15.121:40546) 2023-11-15T03:11:28.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74edc78700 1 -- 172.21.15.121:0/2334226774 learned_addr learned my addr 172.21.15.121:0/2334226774 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:28.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74ed477700 1 --2- 172.21.15.121:0/2334226774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21698d0 0x21a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74ecc76700 1 --2- 172.21.15.121:0/2334226774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x216ab20 0x216a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74edc78700 1 -- 172.21.15.121:0/2334226774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21698d0 msgr2=0x21a7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:28.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74edc78700 1 --2- 172.21.15.121:0/2334226774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21698d0 0x21a7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74edc78700 1 -- 172.21.15.121:0/2334226774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x216ab20 msgr2=0x216a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:28.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74edc78700 1 --2- 172.21.15.121:0/2334226774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x216ab20 0x216a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74edc78700 1 -- 172.21.15.121:0/2334226774 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e80bc970 con 0x21b00a0 2023-11-15T03:11:28.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74edc78700 1 --2- 172.21.15.121:0/2334226774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b00a0 0x21ad530 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 rx=0x7f74e00025c0 tx=0x7f74e0009db0).ready entity=mon.0 client_cookie=9d93e1aeb93e36c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:28.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74d2ffd700 1 -- 172.21.15.121:0/2334226774 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0002e70 con 0x21b00a0 2023-11-15T03:11:28.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74d2ffd700 1 -- 172.21.15.121:0/2334226774 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e000f070 con 0x21b00a0 2023-11-15T03:11:28.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74d2ffd700 1 -- 172.21.15.121:0/2334226774 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0013710 con 0x21b00a0 2023-11-15T03:11:28.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74f9a50740 1 -- 172.21.15.121:0/2334226774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b00a0 msgr2=0x21ad530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:28.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.072+0000 7f74f9a50740 1 --2- 172.21.15.121:0/2334226774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b00a0 0x21ad530 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 rx=0x7f74e00025c0 tx=0x7f74e0009db0).stop 2023-11-15T03:11:28.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74f9a50740 1 -- 172.21.15.121:0/2334226774 shutdown_connections 2023-11-15T03:11:28.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74f9a50740 1 --2- 172.21.15.121:0/2334226774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21b00a0 0x21ad530 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74f9a50740 1 --2- 172.21.15.121:0/2334226774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x216ab20 0x216a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74f9a50740 1 --2- 172.21.15.121:0/2334226774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x21698d0 0x21a7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74f9a50740 1 -- 172.21.15.121:0/2334226774 >> 172.21.15.121:0/2334226774 conn(0x1f48550 msgr2=0x2169cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:28.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74f9a50740 1 -- 172.21.15.121:0/2334226774 shutdown_connections 2023-11-15T03:11:28.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74f9a50740 1 -- 172.21.15.121:0/2334226774 wait complete. 2023-11-15T03:11:28.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74f9a50740 1 -- start start 2023-11-15T03:11:28.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74f9a50740 1 Processor -- start 2023-11-15T03:11:28.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74f9a50740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21698d0 0x2208670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74f9a50740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x216ab20 0x21ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74f9a50740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b00a0 0x21ada20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74f9a50740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2213510 con 0x21698d0 2023-11-15T03:11:28.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74f9a50740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x2205450 con 0x21b00a0 2023-11-15T03:11:28.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74f9a50740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x21adf10 con 0x216ab20 2023-11-15T03:11:28.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74f9a50740 10 client.0 ll_register_callbacks cb 0x7f74d8006520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:11:28.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74ed477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21698d0 0x2208670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74ed477700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21698d0 0x2208670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40560/0 (socket says 172.21.15.121:40560) 2023-11-15T03:11:28.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74ed477700 1 -- 172.21.15.121:0/2958646296 learned_addr learned my addr 172.21.15.121:0/2958646296 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:28.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74ecc76700 1 --2- 172.21.15.121:0/2958646296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x216ab20 0x21ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.076+0000 7f74edc78700 1 --2- 172.21.15.121:0/2958646296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b00a0 0x21ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74ed477700 1 -- 172.21.15.121:0/2958646296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x216ab20 msgr2=0x21ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:28.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74ed477700 1 --2- 172.21.15.121:0/2958646296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x216ab20 0x21ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74ed477700 1 -- 172.21.15.121:0/2958646296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b00a0 msgr2=0x21ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:28.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74ed477700 1 --2- 172.21.15.121:0/2958646296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b00a0 0x21ada20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74ed477700 1 -- 172.21.15.121:0/2958646296 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e0002320 con 0x21698d0 2023-11-15T03:11:28.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74ed477700 1 --2- 172.21.15.121:0/2958646296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21698d0 0x2208670 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 rx=0x7f74e4047730 tx=0x7f74e404e040).ready entity=mon.0 client_cookie=1baa128406a3442f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:28.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:11:28.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74c77fe700 1 -- 172.21.15.121:0/2958646296 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4055040 con 0x21698d0 2023-11-15T03:11:28.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74c77fe700 1 -- 172.21.15.121:0/2958646296 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f74e4010510 con 0x21698d0 2023-11-15T03:11:28.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74f9a50740 1 -- 172.21.15.121:0/2958646296 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74d800b6e0 con 0x21698d0 2023-11-15T03:11:28.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74c77fe700 1 -- 172.21.15.121:0/2958646296 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=350) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74e404a930 con 0x21698d0 2023-11-15T03:11:28.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74f9a50740 10 client.10869 fetch_fsmap learned FSMap version 350 2023-11-15T03:11:28.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74f9a50740 1 -- 172.21.15.121:0/2958646296 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=350,monmap=2+}) v3 -- 0x20de6b0 con 0x21698d0 2023-11-15T03:11:28.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74c77fe700 1 -- 172.21.15.121:0/2958646296 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d800b6e0 con 0x21698d0 2023-11-15T03:11:28.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74c77fe700 1 -- 172.21.15.121:0/2958646296 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 350) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f74e4050b60 con 0x21698d0 2023-11-15T03:11:28.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74f9a50740 10 client.10869 fetch_fsmap finished waiting for FSMap version 350 2023-11-15T03:11:28.098 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74f9a50740 10 client.10869 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:11:28.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74f9a50740 20 client.10869 populate_metadata read hostname 'smithi121' 2023-11-15T03:11:28.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74f9a50740 1 --2- 172.21.15.121:0/2958646296 >> [v2:172.21.15.121:6836/2335356851,v1:172.21.15.121:6837/2335356851] conn(0x220c060 0x220c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74f9a50740 4 client.10869 mds_command: new command op to 10716 tid=0[{"prefix": "status"}] 2023-11-15T03:11:28.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74f9a50740 1 -- 172.21.15.121:0/2958646296 --> [v2:172.21.15.121:6836/2335356851,v1:172.21.15.121:6837/2335356851] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1f9f4e0 con 0x220c060 2023-11-15T03:11:28.099 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74ecc76700 1 --2- 172.21.15.121:0/2958646296 >> [v2:172.21.15.121:6836/2335356851,v1:172.21.15.121:6837/2335356851] conn(0x220c060 0x220c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74ecc76700 1 --2- 172.21.15.121:0/2958646296 >> [v2:172.21.15.121:6836/2335356851,v1:172.21.15.121:6837/2335356851] conn(0x220c060 0x220c440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=72a95c8f4f53edd7 server_cookie=9dc98643bbf8090d in_seq=0 out_seq=0 2023-11-15T03:11:28.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74c77fe700 10 client.10869 ms_handle_connect on v2:172.21.15.121:6836/2335356851 2023-11-15T03:11:28.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74c77fe700 1 -- 172.21.15.121:0/2958646296 <== mds.0 v2:172.21.15.121:6836/2335356851 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x1f9f4e0 con 0x220c060 2023-11-15T03:11:28.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74c77fe700 10 client.10869 handle_command_reply: tid=0 2023-11-15T03:11:28.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74f9a50740 1 client.10869 shutdown 2023-11-15T03:11:28.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74f9a50740 1 -- 172.21.15.121:0/2958646296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21698d0 msgr2=0x2208670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:28.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74f9a50740 1 --2- 172.21.15.121:0/2958646296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21698d0 0x2208670 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 rx=0x7f74e4047730 tx=0x7f74e404e040).stop 2023-11-15T03:11:28.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74f9a50740 1 -- 172.21.15.121:0/2958646296 shutdown_connections 2023-11-15T03:11:28.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74f9a50740 1 --2- 172.21.15.121:0/2958646296 >> [v2:172.21.15.121:6836/2335356851,v1:172.21.15.121:6837/2335356851] conn(0x220c060 0x220c440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74f9a50740 1 --2- 172.21.15.121:0/2958646296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x21b00a0 0x21ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74f9a50740 1 --2- 172.21.15.121:0/2958646296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x216ab20 0x21ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74c77fe700 0 client.10869 ms_handle_reset on v2:172.21.15.121:6836/2335356851 2023-11-15T03:11:28.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74f9a50740 1 --2- 172.21.15.121:0/2958646296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x21698d0 0x2208670 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.080+0000 7f74f9a50740 1 -- 172.21.15.121:0/2958646296 >> 172.21.15.121:0/2958646296 conn(0x1f48550 msgr2=0x2169cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:28.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.084+0000 7f74f9a50740 1 -- 172.21.15.121:0/2958646296 shutdown_connections 2023-11-15T03:11:28.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.084+0000 7f74f9a50740 1 -- 172.21.15.121:0/2958646296 wait complete. 2023-11-15T03:11:28.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.084+0000 7f74f9a50740 20 client.10869 trim_cache size 0 max 16384 2023-11-15T03:11:28.103 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:11:28.104 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T03:11:28.104 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T03:11:28.104 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 10716, 2023-11-15T03:11:28.104 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T03:11:28.104 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T03:11:28.105 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T03:11:28.105 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 6.2144772269999997, 2023-11-15T03:11:28.105 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 350, 2023-11-15T03:11:28.105 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 145, 2023-11-15T03:11:28.105 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 145, 2023-11-15T03:11:28.106 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 11.603787025000001 2023-11-15T03:11:28.106 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:11:28.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.084+0000 7f74ee479700 1 -- 172.21.15.121:0/425272874 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f74cc05a510 msgr2=0x7f74cc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:28.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.084+0000 7f74ee479700 1 --2- 172.21.15.121:0/425272874 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f74cc05a510 0x7f74cc05c9d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f74e80c0b20 tx=0x7f74d8016000).stop 2023-11-15T03:11:28.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.084+0000 7f74ee479700 1 -- 172.21.15.121:0/425272874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74e80669b0 msgr2=0x7f74e814c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:28.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.084+0000 7f74ee479700 1 --2- 172.21.15.121:0/425272874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74e80669b0 0x7f74e814c6e0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 rx=0x7f74e400c440 tx=0x7f74e4007e10).stop 2023-11-15T03:11:28.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.084+0000 7f74ee479700 1 -- 172.21.15.121:0/425272874 shutdown_connections 2023-11-15T03:11:28.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.084+0000 7f74ee479700 1 --2- 172.21.15.121:0/425272874 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f74cc05a510 0x7f74cc05c9d0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.084+0000 7f74ee479700 1 --2- 172.21.15.121:0/425272874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e8157d90 0x7f74e814f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:11:28.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.084+0000 7f74ee479700 1 --2- 172.21.15.121:0/425272874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74e80a8750 0x7f74e816cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.084+0000 7f74ee479700 1 --2- 172.21.15.121:0/425272874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74e80669b0 0x7f74e814c6e0 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.084+0000 7f74ee479700 1 -- 172.21.15.121:0/425272874 >> 172.21.15.121:0/425272874 conn(0x7f74e8067a60 msgr2=0x7f74e8152a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:28.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.084+0000 7f74ee479700 1 -- 172.21.15.121:0/425272874 shutdown_connections 2023-11-15T03:11:28.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.084+0000 7f74ee479700 1 -- 172.21.15.121:0/425272874 wait complete. 2023-11-15T03:11:28.114 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T03:11:28.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.240+0000 7f1d2e879700 1 Processor -- start 2023-11-15T03:11:28.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.240+0000 7f1d2e879700 1 -- start start 2023-11-15T03:11:28.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.244+0000 7f1d2e879700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1d2814e480 0x7f1d28154d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.244+0000 7f1d2e879700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d280bd860 0x7f1d280a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.244+0000 7f1d2e879700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d28155370 con 0x7f1d280a4da0 2023-11-15T03:11:28.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.244+0000 7f1d2e879700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d280bdcc0 con 0x7f1d2814e480 2023-11-15T03:11:28.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.244+0000 7f1d2e879700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d280bbd70 con 0x7f1d280bd860 2023-11-15T03:11:28.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.244+0000 7f1d2d877700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1d2814e480 0x7f1d28154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.244+0000 7f1d2d076700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d280bd860 0x7f1d280a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.244+0000 7f1d2e078700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f1d280a4da0 0x7f1d280c2420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60886/0 (socket says 172.21.15.121:60886) 2023-11-15T03:11:28.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.244+0000 7f1d2e078700 1 -- 172.21.15.121:0/2629731527 learned_addr learned my addr 172.21.15.121:0/2629731527 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:28.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.244+0000 7f1d2d877700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1d2814e480 0x7f1d28154d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40568/0 (socket says 172.21.15.121:40568) 2023-11-15T03:11:28.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.244+0000 7f1d2d076700 1 -- 172.21.15.121:0/2629731527 >> v2:172.21.15.121:3300/0 conn(0x7f1d2814e480 msgr2=0x7f1d28154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:28.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.244+0000 7f1d2d076700 1 --2- 172.21.15.121:0/2629731527 >> v2:172.21.15.121:3300/0 conn(0x7f1d2814e480 0x7f1d28154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.244+0000 7f1d2d076700 1 -- 172.21.15.121:0/2629731527 >> v1:172.21.15.121:6789/0 conn(0x7f1d280a4da0 legacy=0x7f1d280c2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:28.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.244+0000 7f1d2d076700 1 -- 172.21.15.121:0/2629731527 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d280bc050 con 0x7f1d280bd860 2023-11-15T03:11:28.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.244+0000 7f1d2d076700 1 --2- 172.21.15.121:0/2629731527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d280bd860 0x7f1d280a48b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f1d18009760 tx=0x7f1d18009060).ready entity=mon.2 client_cookie=cd7c84961dda78a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:28.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.244+0000 7f1d2c875700 1 -- 172.21.15.121:0/2629731527 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d180090d0 con 0x7f1d280bd860 2023-11-15T03:11:28.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.244+0000 7f1d2c875700 1 -- 172.21.15.121:0/2629731527 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1800f510 con 0x7f1d280bd860 2023-11-15T03:11:28.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.244+0000 7f1d2c875700 1 -- 172.21.15.121:0/2629731527 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1800f890 con 0x7f1d280bd860 2023-11-15T03:11:28.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.248+0000 7f1d2e879700 1 -- 172.21.15.121:0/2629731527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d280bd860 msgr2=0x7f1d280a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:28.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.248+0000 7f1d2e879700 1 --2- 172.21.15.121:0/2629731527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d280bd860 0x7f1d280a48b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f1d18009760 tx=0x7f1d18009060).stop 2023-11-15T03:11:28.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.248+0000 7f1d2e879700 1 -- 172.21.15.121:0/2629731527 shutdown_connections 2023-11-15T03:11:28.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.248+0000 7f1d2e879700 1 --2- 172.21.15.121:0/2629731527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d280bd860 0x7f1d280a48b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.248+0000 7f1d2e879700 1 --2- 172.21.15.121:0/2629731527 >> v2:172.21.15.121:3300/0 conn(0x7f1d2814e480 0x7f1d28154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.248+0000 7f1d2e879700 1 -- 172.21.15.121:0/2629731527 >> 172.21.15.121:0/2629731527 conn(0x7f1d280c4380 msgr2=0x7f1d28066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:28.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.248+0000 7f1d2e879700 1 -- 172.21.15.121:0/2629731527 shutdown_connections 2023-11-15T03:11:28.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.248+0000 7f1d2e879700 1 -- 172.21.15.121:0/2629731527 wait complete. 2023-11-15T03:11:28.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2e879700 1 Processor -- start 2023-11-15T03:11:28.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2e879700 1 -- start start 2023-11-15T03:11:28.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2e879700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d280a4da0 0x7f1d281481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2e879700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d280bd860 0x7f1d280c4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2e879700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2814e480 0x7f1d280c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2e879700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d280c5310 con 0x7f1d280a4da0 2023-11-15T03:11:28.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2e879700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d280aa550 con 0x7f1d280bd860 2023-11-15T03:11:28.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2e879700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d280a80a0 con 0x7f1d2814e480 2023-11-15T03:11:28.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2d877700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d280a4da0 0x7f1d281481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2d877700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d280a4da0 0x7f1d281481a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40572/0 (socket says 172.21.15.121:40572) 2023-11-15T03:11:28.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2d877700 1 -- 172.21.15.121:0/1552309492 learned_addr learned my addr 172.21.15.121:0/1552309492 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:28.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2d076700 1 --2- 172.21.15.121:0/1552309492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d280bd860 0x7f1d280c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2e078700 1 --2- 172.21.15.121:0/1552309492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2814e480 0x7f1d280c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2d877700 1 -- 172.21.15.121:0/1552309492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2814e480 msgr2=0x7f1d280c4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:28.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2d877700 1 --2- 172.21.15.121:0/1552309492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2814e480 0x7f1d280c4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2d877700 1 -- 172.21.15.121:0/1552309492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d280bd860 msgr2=0x7f1d280c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:28.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2d877700 1 --2- 172.21.15.121:0/1552309492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d280bd860 0x7f1d280c4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2d877700 1 -- 172.21.15.121:0/1552309492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d180092a0 con 0x7f1d280a4da0 2023-11-15T03:11:28.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2d076700 1 --2- 172.21.15.121:0/1552309492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d280bd860 0x7f1d280c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:28.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2e078700 1 --2- 172.21.15.121:0/1552309492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2814e480 0x7f1d280c4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:28.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d2d877700 1 --2- 172.21.15.121:0/1552309492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d280a4da0 0x7f1d281481a0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 rx=0x7f1d2400e930 tx=0x7f1d24009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:28.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.252+0000 7f1d1dffb700 1 -- 172.21.15.121:0/1552309492 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2400eed0 con 0x7f1d280a4da0 2023-11-15T03:11:28.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.256+0000 7f1d1dffb700 1 -- 172.21.15.121:0/1552309492 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2400c850 con 0x7f1d280a4da0 2023-11-15T03:11:28.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.256+0000 7f1d1dffb700 1 -- 172.21.15.121:0/1552309492 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2400cbd0 con 0x7f1d280a4da0 2023-11-15T03:11:28.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.256+0000 7f1d2e879700 1 -- 172.21.15.121:0/1552309492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d280a8380 con 0x7f1d280a4da0 2023-11-15T03:11:28.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.256+0000 7f1d2e879700 1 -- 172.21.15.121:0/1552309492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d280aa170 con 0x7f1d280a4da0 2023-11-15T03:11:28.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.256+0000 7f1d2e879700 1 -- 172.21.15.121:0/1552309492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d28062f80 con 0x7f1d280a4da0 2023-11-15T03:11:28.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.256+0000 7f1d1dffb700 1 -- 172.21.15.121:0/1552309492 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1d2401b400 con 0x7f1d280a4da0 2023-11-15T03:11:28.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.260+0000 7f1d1dffb700 1 --2- 172.21.15.121:0/1552309492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d0c05a510 0x7f1d0c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:28.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.260+0000 7f1d1dffb700 1 -- 172.21.15.121:0/1552309492 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1d24074d70 con 0x7f1d280a4da0 2023-11-15T03:11:28.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.260+0000 7f1d2d076700 1 --2- 172.21.15.121:0/1552309492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d0c05a510 0x7f1d0c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:28.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.260+0000 7f1d2d076700 1 --2- 172.21.15.121:0/1552309492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d0c05a510 0x7f1d0c05c9d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f1d18009fd0 tx=0x7f1d18017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:28.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.264+0000 7f1d1dffb700 1 -- 172.21.15.121:0/1552309492 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1d240484f0 con 0x7f1d280a4da0 2023-11-15T03:11:28.468 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.460+0000 7f1d2e879700 1 -- 172.21.15.121:0/1552309492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1d28046800 con 0x7f1d280a4da0 2023-11-15T03:11:28.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.460+0000 7f1d1dffb700 1 -- 172.21.15.121:0/1552309492 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v350) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1d2404bb10 con 0x7f1d280a4da0 2023-11-15T03:11:28.469 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T03:11:28.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.464+0000 7f1d2e879700 1 -- 172.21.15.121:0/1552309492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d0c05a510 msgr2=0x7f1d0c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:28.471 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.464+0000 7f1d2e879700 1 --2- 172.21.15.121:0/1552309492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d0c05a510 0x7f1d0c05c9d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f1d18009fd0 tx=0x7f1d18017000).stop 2023-11-15T03:11:28.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.464+0000 7f1d2e879700 1 -- 172.21.15.121:0/1552309492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d280a4da0 msgr2=0x7f1d281481a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:28.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.464+0000 7f1d2e879700 1 --2- 172.21.15.121:0/1552309492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d280a4da0 0x7f1d281481a0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 rx=0x7f1d2400e930 tx=0x7f1d24009060).stop 2023-11-15T03:11:28.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.464+0000 7f1d2e879700 1 -- 172.21.15.121:0/1552309492 shutdown_connections 2023-11-15T03:11:28.472 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.464+0000 7f1d2e879700 1 --2- 172.21.15.121:0/1552309492 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1d0c05a510 0x7f1d0c05c9d0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.464+0000 7f1d2e879700 1 --2- 172.21.15.121:0/1552309492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2814e480 0x7f1d280c4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.464+0000 7f1d2e879700 1 --2- 172.21.15.121:0/1552309492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d280bd860 0x7f1d280c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.464+0000 7f1d2e879700 1 --2- 172.21.15.121:0/1552309492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d280a4da0 0x7f1d281481a0 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:28.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.464+0000 7f1d2e879700 1 -- 172.21.15.121:0/1552309492 >> 172.21.15.121:0/1552309492 conn(0x7f1d280c4380 msgr2=0x7f1d28151fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:28.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.464+0000 7f1d2e879700 1 -- 172.21.15.121:0/1552309492 shutdown_connections 2023-11-15T03:11:28.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:28.464+0000 7f1d2e879700 1 -- 172.21.15.121:0/1552309492 wait complete. 2023-11-15T03:11:28.487 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T03:11:28.488 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T03:11:28.488 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T03:11:28.489 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T03:11:28.489 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T03:11:28.489 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T03:11:28.490 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T03:11:28.490 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T03:11:28.491 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:11:28.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:28.492 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T03:11:28.514 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T03:11:28.514 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T03:11:28.515 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T03:11:28.515 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:11:28.515 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T03:11:28.515 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:11:28.515 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:11:28.516 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:11:28.516 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T03:11:28.516 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:11:28.516 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:11:28.516 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T03:11:28.517 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:11:28.517 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:11:28.517 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T03:11:28.517 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:11:28.517 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T03:11:28.518 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 41551sec preferred_lft 41551sec 2023-11-15T03:11:28.518 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T03:11:28.518 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:11:28.518 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T03:11:28.519 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:11:28.520 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T03:11:28.520 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:11:28.520 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T03:11:28.520 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T03:11:28.521 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T03:11:28.521 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:11:28.522 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T03:11:28.522 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:11:28.522 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:11:28.522 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T03:11:28.523 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T03:11:28.523 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:11:28.523 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:11:28.523 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T03:11:28.523 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T03:11:28.524 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:11:28.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:28.525 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T03:11:28.602 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T03:11:28.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:28.606 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T03:11:28.669 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T03:11:28.670 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T03:11:28.670 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T03:11:28.670 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T03:11:28.670 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T03:11:28.670 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T03:11:28.671 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T03:11:28.671 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T03:11:28.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:28.672 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T03:11:28.672 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T03:11:28.672 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T03:11:28.673 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T03:11:28.674 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T03:11:28.674 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T03:11:28.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:28.808 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T03:11:28.826 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T03:11:28.826 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T03:11:28.827 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:11:28.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:28.829 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T03:11:28.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:28.907 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T03:11:28.968 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T03:11:28.968 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:11:29.085 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:11:29.085 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:11:29.105 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:11:29.106 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:11:29.113 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T03:11:29.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:29.115 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T03:11:29.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:11:29.151 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir1) 2023-11-15T03:11:29.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:11:29.232 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir1/file1) 2023-11-15T03:11:29.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:11:29.304 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir2) 2023-11-15T03:11:29.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:11:29.373 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir2/file2) 2023-11-15T03:11:29.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:11:29.443 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2023-11-15T03:11:29.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:11:29.514 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir1/file1 testdir/hardlink1) 2023-11-15T03:11:29.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:11:29.584 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir2/file2 testdir/hardlink2) 2023-11-15T03:11:29.651 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-15T03:11:29.651 DEBUG:teuthology.orchestra.run.smithi121:> import os 2023-11-15T03:11:29.652 DEBUG:teuthology.orchestra.run.smithi121:> import stat 2023-11-15T03:11:29.652 DEBUG:teuthology.orchestra.run.smithi121:> 2023-11-15T03:11:29.653 DEBUG:teuthology.orchestra.run.smithi121:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir1/file1").st_ino) 2023-11-15T03:11:29.653 DEBUG:teuthology.orchestra.run.smithi121:> ' 2023-11-15T03:11:29.795 INFO:teuthology.orchestra.run.smithi121.stdout:1099511627777 2023-11-15T03:11:29.986 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-15T03:11:29.987 DEBUG:teuthology.orchestra.run.smithi121:> import os 2023-11-15T03:11:29.987 DEBUG:teuthology.orchestra.run.smithi121:> import stat 2023-11-15T03:11:29.988 DEBUG:teuthology.orchestra.run.smithi121:> 2023-11-15T03:11:29.989 DEBUG:teuthology.orchestra.run.smithi121:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir2/file2").st_ino) 2023-11-15T03:11:29.989 DEBUG:teuthology.orchestra.run.smithi121:> ' 2023-11-15T03:11:30.124 INFO:teuthology.orchestra.run.smithi121.stdout:1099511628279 2023-11-15T03:11:30.311 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-15T03:11:30.312 DEBUG:teuthology.orchestra.run.smithi121:> import os 2023-11-15T03:11:30.312 DEBUG:teuthology.orchestra.run.smithi121:> import stat 2023-11-15T03:11:30.312 DEBUG:teuthology.orchestra.run.smithi121:> 2023-11-15T03:11:30.313 DEBUG:teuthology.orchestra.run.smithi121:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir2").st_ino) 2023-11-15T03:11:30.313 DEBUG:teuthology.orchestra.run.smithi121:> ' 2023-11-15T03:11:30.450 INFO:teuthology.orchestra.run.smithi121.stdout:1099511628278 2023-11-15T03:11:30.641 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T03:11:30.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:11:30.642 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-15T03:11:30.786 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:11:30.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:30.787 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:11:30.809 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:31.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.000+0000 7fd0024e4700 1 Processor -- start 2023-11-15T03:11:31.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.000+0000 7fd0024e4700 1 -- start start 2023-11-15T03:11:31.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.004+0000 7fd0024e4700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcffc0c3280 0x7fcffc0c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:31.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.004+0000 7fd0024e4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcffc152140 0x7fcffc1525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:31.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.004+0000 7fd0024e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcffc15be30 0x7fcffc15c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:31.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.004+0000 7fd0024e4700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcffc0c1860 con 0x7fcffc0c3280 2023-11-15T03:11:31.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.004+0000 7fd0024e4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcffc0c1d50 con 0x7fcffc152140 2023-11-15T03:11:31.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.004+0000 7fd0024e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc15cb80 con 0x7fcffc15be30 2023-11-15T03:11:31.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.004+0000 7fd0014e2700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcffc0c3280 0x7fcffc0c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:31.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.004+0000 7fd001ce3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcffc15be30 0x7fcffc15c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:31.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.004+0000 7fd000ce1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcffc152140 0x7fcffc1525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:31.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.004+0000 7fd0014e2700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcffc0c3280 0x7fcffc0c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40582/0 (socket says 172.21.15.121:40582) 2023-11-15T03:11:31.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.004+0000 7fd0014e2700 1 -- 172.21.15.121:0/819631083 learned_addr learned my addr 172.21.15.121:0/819631083 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:31.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.004+0000 7fd0014e2700 1 -- 172.21.15.121:0/819631083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcffc15be30 msgr2=0x7fcffc15c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:31.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.004+0000 7fd0014e2700 1 --2- 172.21.15.121:0/819631083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcffc15be30 0x7fcffc15c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:31.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.004+0000 7fd0014e2700 1 -- 172.21.15.121:0/819631083 >> v2:172.21.15.190:3300/0 conn(0x7fcffc152140 msgr2=0x7fcffc1525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:31.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.004+0000 7fd0014e2700 1 --2- 172.21.15.121:0/819631083 >> v2:172.21.15.190:3300/0 conn(0x7fcffc152140 0x7fcffc1525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:31.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.004+0000 7fd0014e2700 1 -- 172.21.15.121:0/819631083 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcffc0c1f80 con 0x7fcffc0c3280 2023-11-15T03:11:31.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.004+0000 7fd0014e2700 1 --2- 172.21.15.121:0/819631083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcffc0c3280 0x7fcffc0c3660 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 rx=0x7fcffc159770 tx=0x7fcff80090f0).ready entity=mon.0 client_cookie=3fb6d377f9104d57 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:31.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.008+0000 7fcff3fff700 1 -- 172.21.15.121:0/819631083 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff800f510 con 0x7fcffc0c3280 2023-11-15T03:11:31.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.008+0000 7fcff3fff700 1 -- 172.21.15.121:0/819631083 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff800f670 con 0x7fcffc0c3280 2023-11-15T03:11:31.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.008+0000 7fcff3fff700 1 -- 172.21.15.121:0/819631083 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff800f9f0 con 0x7fcffc0c3280 2023-11-15T03:11:31.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.008+0000 7fd0024e4700 1 -- 172.21.15.121:0/819631083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcffc0c3280 msgr2=0x7fcffc0c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:31.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.008+0000 7fd0024e4700 1 --2- 172.21.15.121:0/819631083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcffc0c3280 0x7fcffc0c3660 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 rx=0x7fcffc159770 tx=0x7fcff80090f0).stop 2023-11-15T03:11:31.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.008+0000 7fd0024e4700 1 -- 172.21.15.121:0/819631083 shutdown_connections 2023-11-15T03:11:31.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.008+0000 7fd0024e4700 1 --2- 172.21.15.121:0/819631083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcffc15be30 0x7fcffc15c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:31.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.008+0000 7fd0024e4700 1 --2- 172.21.15.121:0/819631083 >> v2:172.21.15.190:3300/0 conn(0x7fcffc152140 0x7fcffc1525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:31.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.008+0000 7fd0024e4700 1 --2- 172.21.15.121:0/819631083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcffc0c3280 0x7fcffc0c3660 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:31.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.008+0000 7fd0024e4700 1 -- 172.21.15.121:0/819631083 >> 172.21.15.121:0/819631083 conn(0x7fcffc062190 msgr2=0x7fcffc062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:31.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.008+0000 7fd0024e4700 1 -- 172.21.15.121:0/819631083 shutdown_connections 2023-11-15T03:11:31.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.008+0000 7fd0024e4700 1 -- 172.21.15.121:0/819631083 wait complete. 2023-11-15T03:11:31.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd0024e4700 1 Processor -- start 2023-11-15T03:11:31.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd0024e4700 1 -- start start 2023-11-15T03:11:31.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd0024e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcffc0c3280 0x7fcffc1683f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:31.027 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd0024e4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcffc152140 0x7fcffc14e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:31.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd0024e4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcffc15be30 0x7fcffc0c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:31.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd0024e4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc0c5c50 con 0x7fcffc152140 2023-11-15T03:11:31.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd0024e4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc16bab0 con 0x7fcffc15be30 2023-11-15T03:11:31.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd0024e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc0c4780 con 0x7fcffc0c3280 2023-11-15T03:11:31.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd000ce1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcffc152140 0x7fcffc14e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:31.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd000ce1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcffc152140 0x7fcffc14e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40592/0 (socket says 172.21.15.121:40592) 2023-11-15T03:11:31.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd000ce1700 1 -- 172.21.15.121:0/37896843 learned_addr learned my addr 172.21.15.121:0/37896843 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:31.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd0014e2700 1 --2- 172.21.15.121:0/37896843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcffc0c3280 0x7fcffc1683f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:31.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd001ce3700 1 --2- 172.21.15.121:0/37896843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcffc15be30 0x7fcffc0c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:31.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd000ce1700 1 -- 172.21.15.121:0/37896843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcffc0c3280 msgr2=0x7fcffc1683f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:31.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd000ce1700 1 --2- 172.21.15.121:0/37896843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcffc0c3280 0x7fcffc1683f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:31.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd000ce1700 1 -- 172.21.15.121:0/37896843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcffc15be30 msgr2=0x7fcffc0c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:31.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd000ce1700 1 --2- 172.21.15.121:0/37896843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcffc15be30 0x7fcffc0c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:31.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd000ce1700 1 -- 172.21.15.121:0/37896843 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff8009370 con 0x7fcffc152140 2023-11-15T03:11:31.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd000ce1700 1 --2- 172.21.15.121:0/37896843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcffc152140 0x7fcffc14e640 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 rx=0x7fcfec00e930 tx=0x7fcfec009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:31.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fcff17fa700 1 -- 172.21.15.121:0/37896843 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec00eed0 con 0x7fcffc152140 2023-11-15T03:11:31.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fcff17fa700 1 -- 172.21.15.121:0/37896843 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfec00c850 con 0x7fcffc152140 2023-11-15T03:11:31.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fcff17fa700 1 -- 172.21.15.121:0/37896843 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec00cbd0 con 0x7fcffc152140 2023-11-15T03:11:31.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.012+0000 7fd0024e4700 1 -- 172.21.15.121:0/37896843 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcffc1689e0 con 0x7fcffc152140 2023-11-15T03:11:31.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.016+0000 7fd0024e4700 1 -- 172.21.15.121:0/37896843 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcffc168fa0 con 0x7fcffc152140 2023-11-15T03:11:31.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.016+0000 7fd0024e4700 1 -- 172.21.15.121:0/37896843 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcffc06bcf0 con 0x7fcffc152140 2023-11-15T03:11:31.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.016+0000 7fcff17fa700 1 -- 172.21.15.121:0/37896843 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fcfec01b400 con 0x7fcffc152140 2023-11-15T03:11:31.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.016+0000 7fcff17fa700 1 --2- 172.21.15.121:0/37896843 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcfe005a510 0x7fcfe005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:31.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.016+0000 7fcff17fa700 1 -- 172.21.15.121:0/37896843 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fcfec074d70 con 0x7fcffc152140 2023-11-15T03:11:31.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.016+0000 7fd0014e2700 1 --2- 172.21.15.121:0/37896843 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcfe005a510 0x7fcfe005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:31.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.020+0000 7fd0014e2700 1 --2- 172.21.15.121:0/37896843 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcfe005a510 0x7fcfe005c9d0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7fcff8008000 tx=0x7fcff8017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:31.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.024+0000 7fcff17fa700 1 -- 172.21.15.121:0/37896843 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcfec0484f0 con 0x7fcffc152140 2023-11-15T03:11:31.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.220+0000 7fd0024e4700 1 -- 172.21.15.121:0/37896843 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcffc0c2260 con 0x7fcffc152140 2023-11-15T03:11:31.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.220+0000 7fcff17fa700 1 -- 172.21.15.121:0/37896843 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) v1 ==== 95+0+4512 (secure 0 0 0) 0x7fcfec04bb10 con 0x7fcffc152140 2023-11-15T03:11:31.231 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:31.231 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":350,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/755048854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":755048854},{"type":"v1","addr":"172.21.15.190:6834","nonce":755048854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2360242049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2360242049},{"type":"v1","addr":"172.21.15.190:6835","nonce":2360242049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2780153683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2780153683},{"type":"v1","addr":"172.21.15.121:6835","nonce":2780153683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345}],"filesystems":[{"mdsmap":{"epoch":350,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:11:21.864101+0000","modified":"2023-11-15T03:11:24.889953+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10716},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10716":{"gid":10716,"name":"a","rank":0,"incarnation":346,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6837/2335356851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2335356851},{"type":"v1","addr":"172.21.15.121:6837","nonce":2335356851}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-15T03:11:31.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.224+0000 7fd0024e4700 1 -- 172.21.15.121:0/37896843 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcfe005a510 msgr2=0x7fcfe005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:31.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.224+0000 7fd0024e4700 1 --2- 172.21.15.121:0/37896843 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcfe005a510 0x7fcfe005c9d0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7fcff8008000 tx=0x7fcff8017000).stop 2023-11-15T03:11:31.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.224+0000 7fd0024e4700 1 -- 172.21.15.121:0/37896843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcffc152140 msgr2=0x7fcffc14e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:31.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.224+0000 7fd0024e4700 1 --2- 172.21.15.121:0/37896843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcffc152140 0x7fcffc14e640 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 rx=0x7fcfec00e930 tx=0x7fcfec009060).stop 2023-11-15T03:11:31.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.224+0000 7fd0024e4700 1 -- 172.21.15.121:0/37896843 shutdown_connections 2023-11-15T03:11:31.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.224+0000 7fd0024e4700 1 --2- 172.21.15.121:0/37896843 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcfe005a510 0x7fcfe005c9d0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:31.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.224+0000 7fd0024e4700 1 --2- 172.21.15.121:0/37896843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcffc15be30 0x7fcffc0c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:31.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.224+0000 7fd0024e4700 1 --2- 172.21.15.121:0/37896843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcffc152140 0x7fcffc14e640 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:31.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.224+0000 7fd0024e4700 1 --2- 172.21.15.121:0/37896843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcffc0c3280 0x7fcffc1683f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:31.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.228+0000 7fd0024e4700 1 -- 172.21.15.121:0/37896843 >> 172.21.15.121:0/37896843 conn(0x7fcffc062190 msgr2=0x7fcffc15a860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:31.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.228+0000 7fd0024e4700 1 -- 172.21.15.121:0/37896843 shutdown_connections 2023-11-15T03:11:31.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.228+0000 7fd0024e4700 1 -- 172.21.15.121:0/37896843 wait complete. 2023-11-15T03:11:31.238 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 350 2023-11-15T03:11:31.254 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-15T03:11:31.447 INFO:teuthology.orchestra.run.smithi121.stdout:{"message":"","return_code":0} 2023-11-15T03:11:31.447 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-15T03:11:31.448 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:11:31.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.600+0000 7f5b19507700 1 Processor -- start 2023-11-15T03:11:31.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.600+0000 7f5b19507700 1 -- start start 2023-11-15T03:11:31.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.600+0000 7f5b19507700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b140669b0 0x7f5b14158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:31.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.600+0000 7f5b19507700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b140bb910 con 0x7f5b140a8750 2023-11-15T03:11:31.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.600+0000 7f5b19507700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1415bde0 con 0x7f5b14157d90 2023-11-15T03:11:31.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.600+0000 7f5b19507700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140c1fb0 con 0x7f5b140669b0 2023-11-15T03:11:31.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.600+0000 7f5b18d06700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f5b140a8750 0x7f5b14065680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60898/0 (socket says 172.21.15.121:60898) 2023-11-15T03:11:31.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.600+0000 7f5b18d06700 1 -- 172.21.15.121:0/1178170266 learned_addr learned my addr 172.21.15.121:0/1178170266 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:31.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.600+0000 7f5b137fe700 1 --2- 172.21.15.121:0/1178170266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b140669b0 0x7f5b14158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:31.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.600+0000 7f5b12ffd700 1 -- 172.21.15.121:0/1178170266 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981025673 0 0) 0x7f5b140bb910 con 0x7f5b140a8750 2023-11-15T03:11:31.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.600+0000 7f5b12ffd700 1 -- 172.21.15.121:0/1178170266 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5afc0033f0 con 0x7f5b140a8750 2023-11-15T03:11:31.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b12ffd700 1 -- 172.21.15.121:0/1178170266 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1139273718 0 0) 0x7f5afc0033f0 con 0x7f5b140a8750 2023-11-15T03:11:31.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b12ffd700 1 -- 172.21.15.121:0/1178170266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b140669b0 msgr2=0x7f5b14158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:31.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b12ffd700 1 --2- 172.21.15.121:0/1178170266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b140669b0 0x7f5b14158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:31.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b12ffd700 1 -- 172.21.15.121:0/1178170266 >> v1:172.21.15.190:6789/0 conn(0x7f5b14157d90 legacy=0x7f5b140c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:31.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b12ffd700 1 -- 172.21.15.121:0/1178170266 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1415c0c0 con 0x7f5b140a8750 2023-11-15T03:11:31.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b12ffd700 1 -- 172.21.15.121:0/1178170266 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f5b0c0017f0 con 0x7f5b140a8750 2023-11-15T03:11:31.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b12ffd700 1 -- 172.21.15.121:0/1178170266 >> v1:172.21.15.121:6789/0 conn(0x7f5b140a8750 legacy=0x7f5b14065680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:31.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b12ffd700 1 --2- 172.21.15.121:0/1178170266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5afc003910 0x7f5afc003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:31.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b12ffd700 1 --2- 172.21.15.121:0/1178170266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b140669b0 0x7f5afc004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:31.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b12ffd700 1 --2- 172.21.15.121:0/1178170266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5afc004b80 0x7f5afc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:31.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b12ffd700 1 -- 172.21.15.121:0/1178170266 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc00b8e0 con 0x7f5afc004b80 2023-11-15T03:11:31.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b12ffd700 1 -- 172.21.15.121:0/1178170266 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc00ba20 con 0x7f5afc003910 2023-11-15T03:11:31.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b12ffd700 1 -- 172.21.15.121:0/1178170266 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc00bb90 con 0x7f5b140669b0 2023-11-15T03:11:31.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b18d06700 1 --2- 172.21.15.121:0/1178170266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5afc004b80 0x7f5afc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:31.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b13fff700 1 --2- 172.21.15.121:0/1178170266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5afc003910 0x7f5afc003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:31.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b137fe700 1 --2- 172.21.15.121:0/1178170266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b140669b0 0x7f5afc004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:31.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b18d06700 1 -- 172.21.15.121:0/1178170266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b140669b0 msgr2=0x7f5afc004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:31.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b18d06700 1 --2- 172.21.15.121:0/1178170266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b140669b0 0x7f5afc004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:31.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b18d06700 1 -- 172.21.15.121:0/1178170266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5afc003910 msgr2=0x7f5afc003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:31.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b18d06700 1 --2- 172.21.15.121:0/1178170266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5afc003910 0x7f5afc003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:31.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b18d06700 1 -- 172.21.15.121:0/1178170266 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc0033f0 con 0x7f5afc004b80 2023-11-15T03:11:31.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b13fff700 1 --2- 172.21.15.121:0/1178170266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5afc003910 0x7f5afc003cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:31.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b18d06700 1 --2- 172.21.15.121:0/1178170266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5afc004b80 0x7f5afc005000 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 rx=0x7f5b0c00c5f0 tx=0x7f5b0c001d90).ready entity=mon.0 client_cookie=139ec105e8aa3383 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:31.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b12ffd700 1 -- 172.21.15.121:0/1178170266 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c001fc0 con 0x7f5afc004b80 2023-11-15T03:11:31.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b12ffd700 1 -- 172.21.15.121:0/1178170266 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c00cd90 con 0x7f5afc004b80 2023-11-15T03:11:31.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b12ffd700 1 -- 172.21.15.121:0/1178170266 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c005d70 con 0x7f5afc004b80 2023-11-15T03:11:31.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b19507700 1 -- 172.21.15.121:0/1178170266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5afc004b80 msgr2=0x7f5afc005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:31.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b19507700 1 --2- 172.21.15.121:0/1178170266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5afc004b80 0x7f5afc005000 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 rx=0x7f5b0c00c5f0 tx=0x7f5b0c001d90).stop 2023-11-15T03:11:31.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b18d06700 1 -- 172.21.15.121:0/1178170266 reap_dead start 2023-11-15T03:11:31.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b19507700 1 -- 172.21.15.121:0/1178170266 shutdown_connections 2023-11-15T03:11:31.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b19507700 1 -- 172.21.15.121:0/1178170266 >> 172.21.15.121:0/1178170266 conn(0x7f5b14067a60 msgr2=0x7f5b14067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:31.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b19507700 1 -- 172.21.15.121:0/1178170266 shutdown_connections 2023-11-15T03:11:31.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.604+0000 7f5b19507700 1 -- 172.21.15.121:0/1178170266 wait complete. 2023-11-15T03:11:31.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.608+0000 7f5b19507700 1 Processor -- start 2023-11-15T03:11:31.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.608+0000 7f5b19507700 1 -- start start 2023-11-15T03:11:31.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b19507700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5afc004b80 0x7f5b14150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:31.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b19507700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b14152850 0x7f5b14152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:31.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b19507700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b1415a320 0x7f5b1415a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:31.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b19507700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b141531c0 con 0x7f5afc004b80 2023-11-15T03:11:31.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b19507700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b14158170 con 0x7f5b14152850 2023-11-15T03:11:31.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b19507700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1415cd30 con 0x7f5b1415a320 2023-11-15T03:11:31.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b13fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5afc004b80 0x7f5b14150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:31.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b13fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5afc004b80 0x7f5b14150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40608/0 (socket says 172.21.15.121:40608) 2023-11-15T03:11:31.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b13fff700 1 -- 172.21.15.121:0/1949713440 learned_addr learned my addr 172.21.15.121:0/1949713440 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:31.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b137fe700 1 --2- 172.21.15.121:0/1949713440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b14152850 0x7f5b14152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:31.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b18d06700 1 --2- 172.21.15.121:0/1949713440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b1415a320 0x7f5b1415a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:31.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b13fff700 1 -- 172.21.15.121:0/1949713440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b1415a320 msgr2=0x7f5b1415a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:31.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b13fff700 1 --2- 172.21.15.121:0/1949713440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b1415a320 0x7f5b1415a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:31.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b13fff700 1 -- 172.21.15.121:0/1949713440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b14152850 msgr2=0x7f5b14152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:31.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b13fff700 1 --2- 172.21.15.121:0/1949713440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b14152850 0x7f5b14152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:31.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b13fff700 1 -- 172.21.15.121:0/1949713440 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b141583c0 con 0x7f5afc004b80 2023-11-15T03:11:31.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b13fff700 1 --2- 172.21.15.121:0/1949713440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5afc004b80 0x7f5b14150010 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 rx=0x7f5b0400e600 tx=0x7f5b04009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:31.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5afbfff700 1 -- 172.21.15.121:0/1949713440 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04010490 con 0x7f5afc004b80 2023-11-15T03:11:31.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5afbfff700 1 -- 172.21.15.121:0/1949713440 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b04010040 con 0x7f5afc004b80 2023-11-15T03:11:31.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5afbfff700 1 -- 172.21.15.121:0/1949713440 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04010800 con 0x7f5afc004b80 2023-11-15T03:11:31.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b19507700 1 -- 172.21.15.121:0/1949713440 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b141586a0 con 0x7f5afc004b80 2023-11-15T03:11:31.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b19507700 1 -- 172.21.15.121:0/1949713440 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b14158c60 con 0x7f5afc004b80 2023-11-15T03:11:31.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.612+0000 7f5b19507700 1 -- 172.21.15.121:0/1949713440 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1406bcf0 con 0x7f5afc004b80 2023-11-15T03:11:31.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.616+0000 7f5afbfff700 1 -- 172.21.15.121:0/1949713440 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5b04010c30 con 0x7f5afc004b80 2023-11-15T03:11:31.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.616+0000 7f5afbfff700 1 --2- 172.21.15.121:0/1949713440 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b0005a510 0x7f5b0005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:31.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.616+0000 7f5afbfff700 1 -- 172.21.15.121:0/1949713440 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5b040740c0 con 0x7f5afc004b80 2023-11-15T03:11:31.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.616+0000 7f5b137fe700 1 --2- 172.21.15.121:0/1949713440 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b0005a510 0x7f5b0005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:31.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.616+0000 7f5b137fe700 1 --2- 172.21.15.121:0/1949713440 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b0005a510 0x7f5b0005c9d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7f5b08006920 tx=0x7f5b08008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:31.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.620+0000 7f5afbfff700 1 -- 172.21.15.121:0/1949713440 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5b04047780 con 0x7f5afc004b80 2023-11-15T03:11:31.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:31.816+0000 7f5b19507700 1 -- 172.21.15.121:0/1949713440 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5b140c2120 con 0x7f5afc004b80 2023-11-15T03:11:32.366 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:11:32.356+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:11:32.382 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.372+0000 7f5afbfff700 1 -- 172.21.15.121:0/1949713440 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v351) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5b0404ada0 con 0x7f5afc004b80 2023-11-15T03:11:32.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.376+0000 7f5b19507700 1 -- 172.21.15.121:0/1949713440 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b0005a510 msgr2=0x7f5b0005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:32.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.376+0000 7f5b19507700 1 --2- 172.21.15.121:0/1949713440 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b0005a510 0x7f5b0005c9d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7f5b08006920 tx=0x7f5b08008040).stop 2023-11-15T03:11:32.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.376+0000 7f5b19507700 1 -- 172.21.15.121:0/1949713440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5afc004b80 msgr2=0x7f5b14150010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:32.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.376+0000 7f5b19507700 1 --2- 172.21.15.121:0/1949713440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5afc004b80 0x7f5b14150010 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 rx=0x7f5b0400e600 tx=0x7f5b04009040).stop 2023-11-15T03:11:32.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.380+0000 7f5b19507700 1 -- 172.21.15.121:0/1949713440 shutdown_connections 2023-11-15T03:11:32.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.380+0000 7f5b19507700 1 --2- 172.21.15.121:0/1949713440 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f5b0005a510 0x7f5b0005c9d0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.380+0000 7f5b19507700 1 --2- 172.21.15.121:0/1949713440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b1415a320 0x7f5b1415a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.380+0000 7f5b19507700 1 --2- 172.21.15.121:0/1949713440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b14152850 0x7f5b14152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.380+0000 7f5b19507700 1 --2- 172.21.15.121:0/1949713440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5afc004b80 0x7f5b14150010 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.380+0000 7f5b19507700 1 -- 172.21.15.121:0/1949713440 >> 172.21.15.121:0/1949713440 conn(0x7f5b14067a60 msgr2=0x7f5b140c2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:32.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.380+0000 7f5b19507700 1 -- 172.21.15.121:0/1949713440 shutdown_connections 2023-11-15T03:11:32.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.380+0000 7f5b19507700 1 -- 172.21.15.121:0/1949713440 wait complete. 2023-11-15T03:11:32.390 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:11:32.399 INFO:tasks.ceph.mds.a.smithi121.stderr: -4895> 2023-11-15T03:11:27.396+0000 7f93a56bd700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:11:32.412 DEBUG:teuthology.orchestra.run.smithi121:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2023-11-15T03:11:32.455 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:11:32.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.452+0000 7fd97293cf40 1 -- start start 2023-11-15T03:11:32.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.452+0000 7fd97293cf40 1 Processor -- start 2023-11-15T03:11:32.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd97293cf40 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b012e27c00 0x55b012d5d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd97293cf40 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b012e27800 0x55b012d5d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.464 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd97293cf40 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b012e27400 0x55b012d5de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd97293cf40 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b012d62500 con 0x55b012e27800 2023-11-15T03:11:32.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd97293cf40 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b012d63180 con 0x55b012e27c00 2023-11-15T03:11:32.465 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd97293cf40 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b012d62000 con 0x55b012e27400 2023-11-15T03:11:32.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd9671fb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b012e27800 0x55b012d5d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.466 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd9671fb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b012e27800 0x55b012d5d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40648/0 (socket says 172.21.15.121:40648) 2023-11-15T03:11:32.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd9671fb700 1 -- 172.21.15.121:0/4056110984 learned_addr learned my addr 172.21.15.121:0/4056110984 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:32.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd9679fc700 1 --2- 172.21.15.121:0/4056110984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b012e27c00 0x55b012d5d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd9681fd700 1 --2- 172.21.15.121:0/4056110984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b012e27400 0x55b012d5de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd9671fb700 1 -- 172.21.15.121:0/4056110984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b012e27400 msgr2=0x55b012d5de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:32.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd9671fb700 1 --2- 172.21.15.121:0/4056110984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b012e27400 0x55b012d5de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd9671fb700 1 -- 172.21.15.121:0/4056110984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b012e27c00 msgr2=0x55b012d5d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:32.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd9671fb700 1 --2- 172.21.15.121:0/4056110984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b012e27c00 0x55b012d5d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd9671fb700 1 -- 172.21.15.121:0/4056110984 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b012d409c0 con 0x55b012e27800 2023-11-15T03:11:32.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd9681fd700 1 --2- 172.21.15.121:0/4056110984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b012e27400 0x55b012d5de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:32.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd9679fc700 1 --2- 172.21.15.121:0/4056110984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b012e27c00 0x55b012d5d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:32.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd9671fb700 1 --2- 172.21.15.121:0/4056110984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b012e27800 0x55b012d5d900 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 rx=0x55b01398ba10 tx=0x55b013a7ed80).ready entity=mon.0 client_cookie=f751ef53eb2afa1b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:32.477 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd9651f7700 1 -- 172.21.15.121:0/4056110984 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b012d6a9a0 con 0x55b012e27800 2023-11-15T03:11:32.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd97293cf40 1 -- 172.21.15.121:0/4056110984 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b012d40820 con 0x55b012e27800 2023-11-15T03:11:32.478 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd97293cf40 1 -- 172.21.15.121:0/4056110984 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b012d41040 con 0x55b012e27800 2023-11-15T03:11:32.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd9651f7700 1 -- 172.21.15.121:0/4056110984 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b012d67080 con 0x55b012e27800 2023-11-15T03:11:32.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd97293cf40 4 waiting for MDS map... 2023-11-15T03:11:32.479 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd9651f7700 1 -- 172.21.15.121:0/4056110984 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b012d6a6e0 con 0x55b012e27800 2023-11-15T03:11:32.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd9651f7700 1 -- 172.21.15.121:0/4056110984 <== mon.0 v2:172.21.15.121:3300/0 4 ==== osd_map(146..146 src has 1..146) v4 ==== 6207+0+0 (secure 0 0 0) 0x55b012d51dc0 con 0x55b012e27800 2023-11-15T03:11:32.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd9651f7700 1 -- 172.21.15.121:0/4056110984 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 351) v1 ==== 2559+0+0 (secure 0 0 0) 0x55b013a6c300 con 0x55b012e27800 2023-11-15T03:11:32.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd97293cf40 4 Got MDS map 351 2023-11-15T03:11:32.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd97293cf40 10 main: JournalTool::main 2023-11-15T03:11:32.480 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.456+0000 7fd97293cf40 4 main: JournalTool: connecting to RADOS... 2023-11-15T03:11:32.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd97293cf40 1 Processor -- start 2023-11-15T03:11:32.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd97293cf40 1 -- start start 2023-11-15T03:11:32.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd97293cf40 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b012e26c00 0x55b012d5e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.481 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd97293cf40 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b012e26400 0x55b012d5e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.482 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd97293cf40 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b013a94800 0x55b012d5ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd97293cf40 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b012d632c0 con 0x55b012e26c00 2023-11-15T03:11:32.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd97293cf40 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b012d63180 con 0x55b013a94800 2023-11-15T03:11:32.483 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd97293cf40 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b012d62500 con 0x55b012e26400 2023-11-15T03:11:32.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd9681fd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b012e26c00 0x55b012d5e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd9681fd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b012e26c00 0x55b012d5e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40656/0 (socket says 172.21.15.121:40656) 2023-11-15T03:11:32.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd9681fd700 1 -- 172.21.15.121:0/4083895373 learned_addr learned my addr 172.21.15.121:0/4083895373 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:32.484 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd9679fc700 1 --2- 172.21.15.121:0/4083895373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b012e26400 0x55b012d5e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd9671fb700 1 --2- 172.21.15.121:0/4083895373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b013a94800 0x55b012d5ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd9681fd700 1 -- 172.21.15.121:0/4083895373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b012e26400 msgr2=0x55b012d5e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:32.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd9681fd700 1 --2- 172.21.15.121:0/4083895373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b012e26400 0x55b012d5e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd9681fd700 1 -- 172.21.15.121:0/4083895373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b013a94800 msgr2=0x55b012d5ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:32.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd9681fd700 1 --2- 172.21.15.121:0/4083895373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b013a94800 0x55b012d5ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd9681fd700 1 -- 172.21.15.121:0/4083895373 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b012d411e0 con 0x55b012e26c00 2023-11-15T03:11:32.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd9671fb700 1 --2- 172.21.15.121:0/4083895373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b013a94800 0x55b012d5ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:11:32.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd9681fd700 1 --2- 172.21.15.121:0/4083895373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b012e26c00 0x55b012d5e300 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 rx=0x55b01398bc50 tx=0x55b013aacc00).ready entity=mon.0 client_cookie=9c6fe48a7a4e623 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:32.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd9631f3700 1 -- 172.21.15.121:0/4083895373 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b012d6a9a0 con 0x55b012e26c00 2023-11-15T03:11:32.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd9631f3700 1 -- 172.21.15.121:0/4083895373 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b013aa8480 con 0x55b012e26c00 2023-11-15T03:11:32.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd9631f3700 1 -- 172.21.15.121:0/4083895373 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b012d6a6e0 con 0x55b012e26c00 2023-11-15T03:11:32.487 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd97293cf40 1 -- 172.21.15.121:0/4083895373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b012e26c00 msgr2=0x55b012d5e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:32.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd97293cf40 1 --2- 172.21.15.121:0/4083895373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b012e26c00 0x55b012d5e300 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 rx=0x55b01398bc50 tx=0x55b013aacc00).stop 2023-11-15T03:11:32.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd97293cf40 1 -- 172.21.15.121:0/4083895373 shutdown_connections 2023-11-15T03:11:32.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd97293cf40 1 --2- 172.21.15.121:0/4083895373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b013a94800 0x55b012d5ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd97293cf40 1 --2- 172.21.15.121:0/4083895373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b012e26400 0x55b012d5e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd97293cf40 1 --2- 172.21.15.121:0/4083895373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b012e26c00 0x55b012d5e300 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd97293cf40 1 -- 172.21.15.121:0/4083895373 >> 172.21.15.121:0/4083895373 conn(0x55b012e27000 msgr2=0x55b0139a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:32.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd97293cf40 1 -- 172.21.15.121:0/4083895373 shutdown_connections 2023-11-15T03:11:32.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.460+0000 7fd97293cf40 1 -- 172.21.15.121:0/4083895373 wait complete. 2023-11-15T03:11:32.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd97293cf40 1 Processor -- start 2023-11-15T03:11:32.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd97293cf40 1 -- start start 2023-11-15T03:11:32.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd97293cf40 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b012e26400 0x55b012d5e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd97293cf40 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b012e27000 0x55b012d5ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd97293cf40 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b013a95c00 0x55b012d5f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd97293cf40 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b012d62500 con 0x55b013a95c00 2023-11-15T03:11:32.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd97293cf40 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b012d63180 con 0x55b012e26400 2023-11-15T03:11:32.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd97293cf40 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b013a8c000 con 0x55b012e27000 2023-11-15T03:11:32.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd9671fb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b013a95c00 0x55b012d5f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd9671fb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b013a95c00 0x55b012d5f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40662/0 (socket says 172.21.15.121:40662) 2023-11-15T03:11:32.493 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd9671fb700 1 -- 172.21.15.121:0/2477312136 learned_addr learned my addr 172.21.15.121:0/2477312136 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:32.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd9681fd700 1 --2- 172.21.15.121:0/2477312136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b012e26400 0x55b012d5e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd9679fc700 1 --2- 172.21.15.121:0/2477312136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b012e27000 0x55b012d5ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd9671fb700 1 -- 172.21.15.121:0/2477312136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b012e27000 msgr2=0x55b012d5ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:32.494 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd9671fb700 1 --2- 172.21.15.121:0/2477312136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b012e27000 0x55b012d5ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd9671fb700 1 -- 172.21.15.121:0/2477312136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b012e26400 msgr2=0x55b012d5e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:32.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd9671fb700 1 --2- 172.21.15.121:0/2477312136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b012e26400 0x55b012d5e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd9671fb700 1 -- 172.21.15.121:0/2477312136 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b012d40d00 con 0x55b013a95c00 2023-11-15T03:11:32.495 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd9681fd700 1 --2- 172.21.15.121:0/2477312136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b012e26400 0x55b012d5e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:32.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd9671fb700 1 --2- 172.21.15.121:0/2477312136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b013a95c00 0x55b012d5f200 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 rx=0x55b013a71950 tx=0x55b013a7ee40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:32.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd9619f0700 1 -- 172.21.15.121:0/2477312136 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b012d6a580 con 0x55b013a95c00 2023-11-15T03:11:32.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd9619f0700 1 -- 172.21.15.121:0/2477312136 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b013aa8600 con 0x55b013a95c00 2023-11-15T03:11:32.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd97293cf40 1 -- 172.21.15.121:0/2477312136 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b012d41380 con 0x55b013a95c00 2023-11-15T03:11:32.496 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd9619f0700 1 -- 172.21.15.121:0/2477312136 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b012d6ac60 con 0x55b013a95c00 2023-11-15T03:11:32.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd97293cf40 1 -- 172.21.15.121:0/2477312136 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b012d409c0 con 0x55b013a95c00 2023-11-15T03:11:32.497 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.464+0000 7fd97293cf40 4 main: JournalTool: resolving pool 15 2023-11-15T03:11:32.501 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.468+0000 7fd9619f0700 1 -- 172.21.15.121:0/2477312136 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x55b012df3080 con 0x55b013a95c00 2023-11-15T03:11:32.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.468+0000 7fd9619f0700 1 --2- 172.21.15.121:0/2477312136 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x55b013b10800 0x55b013be4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.468+0000 7fd9619f0700 1 -- 172.21.15.121:0/2477312136 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6207+0+0 (secure 0 0 0) 0x55b013aeea80 con 0x55b013a95c00 2023-11-15T03:11:32.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.468+0000 7fd97293cf40 4 main: JournalTool: creating IoCtx.. 2023-11-15T03:11:32.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.468+0000 7fd97293cf40 4 main: Executing for rank 0 2023-11-15T03:11:32.502 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.468+0000 7fd9681fd700 1 --2- 172.21.15.121:0/2477312136 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x55b013b10800 0x55b013be4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.468+0000 7fd9681fd700 1 --2- 172.21.15.121:0/2477312136 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x55b013b10800 0x55b013be4000 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x55b01398bc50 tx=0x55b013aacc60).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:32.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd97293cf40 1 -- start start 2023-11-15T03:11:32.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd97293cf40 1 Processor -- start 2023-11-15T03:11:32.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd97293cf40 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b013a95400 0x55b013be4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd97293cf40 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b013a95000 0x55b013be4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd97293cf40 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b013a94c00 0x55b013be4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd97293cf40 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b013a8c140 con 0x55b013a94c00 2023-11-15T03:11:32.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd97293cf40 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b013a8c280 con 0x55b013a95400 2023-11-15T03:11:32.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd97293cf40 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b013a8c3c0 con 0x55b013a95000 2023-11-15T03:11:32.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd9671fb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b013a94c00 0x55b013be4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd9671fb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b013a94c00 0x55b013be4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40664/0 (socket says 172.21.15.121:40664) 2023-11-15T03:11:32.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd9671fb700 1 -- 172.21.15.121:0/2933175211 learned_addr learned my addr 172.21.15.121:0/2933175211 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:32.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd9679fc700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b013a95000 0x55b013be4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd9681fd700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b013a95400 0x55b013be4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd9671fb700 1 -- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b013a95000 msgr2=0x55b013be4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:32.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd9671fb700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b013a95000 0x55b013be4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd9671fb700 1 -- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b013a95400 msgr2=0x55b013be4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:32.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd9671fb700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b013a95400 0x55b013be4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd9671fb700 1 -- 172.21.15.121:0/2933175211 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b012d41520 con 0x55b013a94c00 2023-11-15T03:11:32.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd9679fc700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b013a95000 0x55b013be4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:32.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd9681fd700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b013a95400 0x55b013be4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:32.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd9671fb700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b013a94c00 0x55b013be4f00 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 rx=0x55b013bb6f00 tx=0x55b013aad0e0).ready entity=mon.0 client_cookie=705880984364f111 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:32.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd95e9ea700 1 -- 172.21.15.121:0/2933175211 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b012d6a6e0 con 0x55b013a94c00 2023-11-15T03:11:32.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd95e9ea700 1 -- 172.21.15.121:0/2933175211 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b013aa8780 con 0x55b013a94c00 2023-11-15T03:11:32.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b012d416c0 con 0x55b013a94c00 2023-11-15T03:11:32.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b013b82680 con 0x55b013a94c00 2023-11-15T03:11:32.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd97293cf40 4 waiting for MDS map... 2023-11-15T03:11:32.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd95e9ea700 1 -- 172.21.15.121:0/2933175211 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b012d6b760 con 0x55b013a94c00 2023-11-15T03:11:32.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd95e9ea700 1 -- 172.21.15.121:0/2933175211 <== mon.0 v2:172.21.15.121:3300/0 4 ==== osd_map(146..146 src has 1..146) v4 ==== 6207+0+0 (secure 0 0 0) 0x55b013aeec40 con 0x55b013a94c00 2023-11-15T03:11:32.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.472+0000 7fd95e9ea700 1 -- 172.21.15.121:0/2933175211 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 351) v1 ==== 2559+0+0 (secure 0 0 0) 0x55b013c35500 con 0x55b013a94c00 2023-11-15T03:11:32.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.476+0000 7fd97293cf40 4 Got MDS map 351 2023-11-15T03:11:32.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.476+0000 7fd97293cf40 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55b013a94800 0x55b013be5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.476+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected e146) v8 -- 0x55b013b10c00 con 0x55b013a94800 2023-11-15T03:11:32.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.476+0000 7fd9681fd700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55b013a94800 0x55b013be5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.476+0000 7fd9681fd700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55b013a94800 0x55b013be5e00 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=ea18de43e4f0cdeb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:32.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.476+0000 7fd9681fd700 1 -- 172.21.15.121:0/2933175211 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x55b013a7d200 con 0x55b013a94800 2023-11-15T03:11:32.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.476+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected e146) v8 -- 0x55b013b11000 con 0x55b013a94800 2023-11-15T03:11:32.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.476+0000 7fd9681fd700 1 -- 172.21.15.121:0/2933175211 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x55b013a7d200 con 0x55b013a94800 2023-11-15T03:11:32.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.480+0000 7fd95d1e7700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55b013b10c00 0x55b013c64000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.480+0000 7fd95d1e7700 1 -- 172.21.15.121:0/2933175211 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- osd_op(unknown.0.0:3 15.34 15:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e146) v8 -- 0x55b013b11000 con 0x55b013b10c00 2023-11-15T03:11:32.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.480+0000 7fd95d1e7700 1 -- 172.21.15.121:0/2933175211 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:4 15.3b 15:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e146) v8 -- 0x55b013c68000 con 0x55b013a94800 2023-11-15T03:11:32.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.480+0000 7fd9679fc700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55b013b10c00 0x55b013c64000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.480+0000 7fd9681fd700 1 -- 172.21.15.121:0/2933175211 <== osd.6 v2:172.21.15.190:6809/33346 3 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b013a7d200 con 0x55b013a94800 2023-11-15T03:11:32.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.480+0000 7fd9679fc700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55b013b10c00 0x55b013c64000 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=67b71431aa42dd70 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:32.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.484+0000 7fd9679fc700 1 -- 172.21.15.121:0/2933175211 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x55b013a7d8c0 con 0x55b013b10c00 2023-11-15T03:11:32.513 INFO:teuthology.orchestra.run.smithi121.stdout:old journal was 4224507~2002 2023-11-15T03:11:32.513 INFO:teuthology.orchestra.run.smithi121.stdout:new journal start will be 8388608 (4162099 bytes past old end) 2023-11-15T03:11:32.513 INFO:teuthology.orchestra.run.smithi121.stdout:writing journal head 2023-11-15T03:11:32.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.484+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:5 15.14 15:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected e146) v8 -- 0x55b013b11400 con 0x55b013a94800 2023-11-15T03:11:32.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.488+0000 7fd9681fd700 1 -- 172.21.15.121:0/2933175211 <== osd.6 v2:172.21.15.190:6809/33346 4 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v146'6 uv6 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55b013a7d200 con 0x55b013a94800 2023-11-15T03:11:32.515 INFO:teuthology.orchestra.run.smithi121.stdout:writing EResetJournal entry 2023-11-15T03:11:32.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.488+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:6 15.3b 15:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected e146) v8 -- 0x55b013b11800 con 0x55b013a94800 2023-11-15T03:11:32.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.488+0000 7fd97293cf40 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55b013b11c00 0x55b013c64f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.488+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:7 15.2b 15:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected e146) v8 -- 0x55b013c68c00 con 0x55b013b11c00 2023-11-15T03:11:32.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.488+0000 7fd97293cf40 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x55b013c69000 0x55b013c65e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.488+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 --> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] -- osd_op(unknown.0.0:8 15.1f 15:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected e146) v8 -- 0x55b013c69400 con 0x55b013c69000 2023-11-15T03:11:32.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.488+0000 7fd97293cf40 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55b013c69800 0x55b013c66d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.488+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- osd_op(unknown.0.0:9 15.3a 15:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected e146) v8 -- 0x55b013c69c00 con 0x55b013c69800 2023-11-15T03:11:32.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.488+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- osd_op(unknown.0.0:10 15.1e 15:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected e146) v8 -- 0x55b013b11400 con 0x55b013b11c00 2023-11-15T03:11:32.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.492+0000 7fd9681fd700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55b013b11c00 0x55b013c64f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.492+0000 7fd9679fc700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x55b013c69000 0x55b013c65e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.492+0000 7fd97293cf40 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55b013b11800 0x55b013c74500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.492+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- osd_op(unknown.0.0:11 15.3d 15:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected e146) v8 -- 0x55b013c68000 con 0x55b013b11800 2023-11-15T03:11:32.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.492+0000 7fd9671fb700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55b013c69800 0x55b013c66d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.492+0000 7fd9681fd700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55b013b11800 0x55b013c74500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.492+0000 7fd9681fd700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55b013b11c00 0x55b013c64f00 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=bdae0b16e40159f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:32.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.492+0000 7fd9681fd700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55b013b11800 0x55b013c74500 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=cc283a8519965b59 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:32.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.492+0000 7fd9671fb700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55b013c69800 0x55b013c66d00 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=551dbe1252ddc6c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:32.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.492+0000 7fd9679fc700 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x55b013c69000 0x55b013c65e00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=6b4d3305056e04c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:32.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.492+0000 7fd9681fd700 1 -- 172.21.15.121:0/2933175211 <== osd.6 v2:172.21.15.190:6809/33346 5 ==== osd_op_reply(6 200.00000002 [delete] v146'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b013a7d200 con 0x55b013a94800 2023-11-15T03:11:32.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.496+0000 7fd9681fd700 1 -- 172.21.15.121:0/2933175211 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== osd_op_reply(10 200.00000006 [delete] v146'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b013a7d200 con 0x55b013b11c00 2023-11-15T03:11:32.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.496+0000 7fd9681fd700 1 -- 172.21.15.121:0/2933175211 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== osd_op_reply(7 200.00000003 [delete] v146'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b013a7d200 con 0x55b013b11c00 2023-11-15T03:11:32.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.496+0000 7fd95d1e7700 1 -- 172.21.15.121:0/2933175211 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- osd_op(unknown.0.0:12 15.3b 15:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected e146) v8 -- 0x55b013c68400 con 0x55b013a94800 2023-11-15T03:11:32.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.500+0000 7fd9679fc700 1 -- 172.21.15.121:0/2933175211 <== osd.3 v2:172.21.15.121:6820/33135 1 ==== osd_op_reply(8 200.00000004 [delete] v146'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b013a7d8c0 con 0x55b013c69000 2023-11-15T03:11:32.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.500+0000 7fd9681fd700 1 -- 172.21.15.121:0/2933175211 <== osd.6 v2:172.21.15.190:6809/33346 6 ==== osd_op_reply(12 200.00000002 [write 0~48] v146'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55b013a7d200 con 0x55b013a94800 2023-11-15T03:11:32.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.500+0000 7fd9671fb700 1 -- 172.21.15.121:0/2933175211 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== osd_op_reply(9 200.00000005 [delete] v146'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b013c84000 con 0x55b013c69800 2023-11-15T03:11:32.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd9681fd700 1 -- 172.21.15.121:0/2933175211 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== osd_op_reply(11 200.00000007 [delete] v146'1 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b013a7d200 con 0x55b013b11800 2023-11-15T03:11:32.523 INFO:teuthology.orchestra.run.smithi121.stdout:done 2023-11-15T03:11:32.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55b013b11800 msgr2=0x55b013c74500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:32.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x55b013b11800 0x55b013c74500 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55b013b11c00 msgr2=0x55b013c64f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:32.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x55b013b11c00 0x55b013c64f00 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x55b013c69000 msgr2=0x55b013c65e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:32.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x55b013c69000 0x55b013c65e00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55b013b10c00 msgr2=0x55b013c64000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:32.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x55b013b10c00 0x55b013c64000 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55b013a94800 msgr2=0x55b013be5e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:32.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x55b013a94800 0x55b013be5e00 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55b013c69800 msgr2=0x55b013c66d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:32.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x55b013c69800 0x55b013c66d00 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd9679fc700 1 -- 172.21.15.121:0/2933175211 reap_dead start 2023-11-15T03:11:32.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd9679fc700 1 -- 172.21.15.121:0/2933175211 reap_dead start 2023-11-15T03:11:32.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b013a94c00 msgr2=0x55b013be4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:32.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 --2- 172.21.15.121:0/2933175211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b013a94c00 0x55b013be4f00 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 rx=0x55b013bb6f00 tx=0x55b013aad0e0).stop 2023-11-15T03:11:32.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd9679fc700 1 -- 172.21.15.121:0/2933175211 reap_dead start 2023-11-15T03:11:32.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 shutdown_connections 2023-11-15T03:11:32.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 >> 172.21.15.121:0/2933175211 conn(0x55b013a95800 msgr2=0x55b0139a8000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:32.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 shutdown_connections 2023-11-15T03:11:32.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/2933175211 wait complete. 2023-11-15T03:11:32.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/2477312136 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x55b013b10800 msgr2=0x55b013be4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:32.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 --2- 172.21.15.121:0/2477312136 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x55b013b10800 0x55b013be4000 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x55b01398bc50 tx=0x55b013aacc60).stop 2023-11-15T03:11:32.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/2477312136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b013a95c00 msgr2=0x55b012d5f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:32.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 --2- 172.21.15.121:0/2477312136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b013a95c00 0x55b012d5f200 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 rx=0x55b013a71950 tx=0x55b013a7ee40).stop 2023-11-15T03:11:32.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/2477312136 shutdown_connections 2023-11-15T03:11:32.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 --2- 172.21.15.121:0/2477312136 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x55b013b10800 0x55b013be4000 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 --2- 172.21.15.121:0/2477312136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b013a95c00 0x55b012d5f200 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 --2- 172.21.15.121:0/2477312136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b012e27000 0x55b012d5ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 --2- 172.21.15.121:0/2477312136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b012e26400 0x55b012d5e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/2477312136 >> 172.21.15.121:0/2477312136 conn(0x55b012e26c00 msgr2=0x55b0139a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:32.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/2477312136 shutdown_connections 2023-11-15T03:11:32.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/2477312136 wait complete. 2023-11-15T03:11:32.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/4056110984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b012e27800 msgr2=0x55b012d5d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:32.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 --2- 172.21.15.121:0/4056110984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b012e27800 0x55b012d5d900 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 rx=0x55b01398ba10 tx=0x55b013a7ed80).stop 2023-11-15T03:11:32.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/4056110984 shutdown_connections 2023-11-15T03:11:32.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 --2- 172.21.15.121:0/4056110984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b012e27400 0x55b012d5de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 --2- 172.21.15.121:0/4056110984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b012e27800 0x55b012d5d900 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 --2- 172.21.15.121:0/4056110984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b012e27c00 0x55b012d5d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/4056110984 >> 172.21.15.121:0/4056110984 conn(0x55b012e26800 msgr2=0x55b0139a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:32.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/4056110984 shutdown_connections 2023-11-15T03:11:32.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.504+0000 7fd97293cf40 1 -- 172.21.15.121:0/4056110984 wait complete. 2023-11-15T03:11:32.536 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.124356, result: old journal was 4224507~2002 new journal start will be 8388608 (4162099 bytes past old end) writing journal head writing EResetJournal entry done 2023-11-15T03:11:32.537 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-15T03:11:32.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.732+0000 7f8910a1b700 1 Processor -- start 2023-11-15T03:11:32.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.732+0000 7f8910a1b700 1 -- start start 2023-11-15T03:11:32.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.732+0000 7f8910a1b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f890c0a8b80 0x7f890c0a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.732+0000 7f8910a1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c065850 0x7f890c067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.732+0000 7f8910a1b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f890c067f50 0x7f890c0bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.736+0000 7f8910a1b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f890c152150 con 0x7f890c0a8b80 2023-11-15T03:11:32.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.736+0000 7f8910a1b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f890c148f70 con 0x7f890c067f50 2023-11-15T03:11:32.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.736+0000 7f8910a1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f890c0683d0 con 0x7f890c065850 2023-11-15T03:11:32.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.736+0000 7f890a59c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f890c0a8b80 0x7f890c0a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.736+0000 7f890a59c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f890c0a8b80 0x7f890c0a8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40666/0 (socket says 172.21.15.121:40666) 2023-11-15T03:11:32.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.736+0000 7f890a59c700 1 -- 172.21.15.121:0/3654217350 learned_addr learned my addr 172.21.15.121:0/3654217350 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:32.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.736+0000 7f8909d9b700 1 --2- 172.21.15.121:0/3654217350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c065850 0x7f890c067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.736+0000 7f890ad9d700 1 --2- 172.21.15.121:0/3654217350 >> v2:172.21.15.190:3300/0 conn(0x7f890c067f50 0x7f890c0bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.736+0000 7f890a59c700 1 -- 172.21.15.121:0/3654217350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c065850 msgr2=0x7f890c067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:32.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.736+0000 7f890a59c700 1 --2- 172.21.15.121:0/3654217350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c065850 0x7f890c067a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.736+0000 7f890a59c700 1 -- 172.21.15.121:0/3654217350 >> v2:172.21.15.190:3300/0 conn(0x7f890c067f50 msgr2=0x7f890c0bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:32.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.736+0000 7f890a59c700 1 --2- 172.21.15.121:0/3654217350 >> v2:172.21.15.190:3300/0 conn(0x7f890c067f50 0x7f890c0bbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.736+0000 7f890a59c700 1 -- 172.21.15.121:0/3654217350 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c1523a0 con 0x7f890c0a8b80 2023-11-15T03:11:32.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.736+0000 7f890a59c700 1 --2- 172.21.15.121:0/3654217350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890c0a8b80 0x7f890c0a8f60 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 rx=0x7f890c06a430 tx=0x7f88fc0090f0).ready entity=mon.0 client_cookie=25ea72676e50b29c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:32.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.736+0000 7f890959a700 1 -- 172.21.15.121:0/3654217350 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc00f510 con 0x7f890c0a8b80 2023-11-15T03:11:32.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.736+0000 7f890959a700 1 -- 172.21.15.121:0/3654217350 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88fc00f670 con 0x7f890c0a8b80 2023-11-15T03:11:32.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.736+0000 7f890959a700 1 -- 172.21.15.121:0/3654217350 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc00fa10 con 0x7f890c0a8b80 2023-11-15T03:11:32.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.736+0000 7f8910a1b700 1 -- 172.21.15.121:0/3654217350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890c0a8b80 msgr2=0x7f890c0a8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:32.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.736+0000 7f8910a1b700 1 --2- 172.21.15.121:0/3654217350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890c0a8b80 0x7f890c0a8f60 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 rx=0x7f890c06a430 tx=0x7f88fc0090f0).stop 2023-11-15T03:11:32.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.740+0000 7f8910a1b700 1 -- 172.21.15.121:0/3654217350 shutdown_connections 2023-11-15T03:11:32.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.740+0000 7f8910a1b700 1 --2- 172.21.15.121:0/3654217350 >> v2:172.21.15.190:3300/0 conn(0x7f890c067f50 0x7f890c0bbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.740+0000 7f8910a1b700 1 --2- 172.21.15.121:0/3654217350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c065850 0x7f890c067a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.740+0000 7f8910a1b700 1 --2- 172.21.15.121:0/3654217350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890c0a8b80 0x7f890c0a8f60 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.740+0000 7f8910a1b700 1 -- 172.21.15.121:0/3654217350 >> 172.21.15.121:0/3654217350 conn(0x7f890c0591c0 msgr2=0x7f890c0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:32.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.740+0000 7f8910a1b700 1 -- 172.21.15.121:0/3654217350 shutdown_connections 2023-11-15T03:11:32.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.740+0000 7f8910a1b700 1 -- 172.21.15.121:0/3654217350 wait complete. 2023-11-15T03:11:32.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.740+0000 7f8910a1b700 1 Processor -- start 2023-11-15T03:11:32.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.740+0000 7f8910a1b700 1 -- start start 2023-11-15T03:11:32.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.740+0000 7f8910a1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c065850 0x7f890c147480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.740+0000 7f8910a1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890c067f50 0x7f890c0c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.740+0000 7f8910a1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f890c0a8b80 0x7f890c0c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.740+0000 7f8910a1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890c0c4f80 con 0x7f890c067f50 2023-11-15T03:11:32.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.740+0000 7f8910a1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890c149110 con 0x7f890c0a8b80 2023-11-15T03:11:32.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.740+0000 7f8910a1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f890c149360 con 0x7f890c065850 2023-11-15T03:11:32.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.744+0000 7f8909d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890c067f50 0x7f890c0c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.744+0000 7f8909d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890c067f50 0x7f890c0c45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40680/0 (socket says 172.21.15.121:40680) 2023-11-15T03:11:32.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.744+0000 7f8909d9b700 1 -- 172.21.15.121:0/3652820918 learned_addr learned my addr 172.21.15.121:0/3652820918 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:32.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.744+0000 7f890ad9d700 1 --2- 172.21.15.121:0/3652820918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f890c0a8b80 0x7f890c0c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.744+0000 7f890a59c700 1 --2- 172.21.15.121:0/3652820918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c065850 0x7f890c147480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.744+0000 7f8909d9b700 1 -- 172.21.15.121:0/3652820918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c065850 msgr2=0x7f890c147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:32.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.744+0000 7f8909d9b700 1 --2- 172.21.15.121:0/3652820918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c065850 0x7f890c147480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.744+0000 7f8909d9b700 1 -- 172.21.15.121:0/3652820918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f890c0a8b80 msgr2=0x7f890c0c4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:32.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.744+0000 7f8909d9b700 1 --2- 172.21.15.121:0/3652820918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f890c0a8b80 0x7f890c0c4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:32.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.744+0000 7f8909d9b700 1 -- 172.21.15.121:0/3652820918 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc009370 con 0x7f890c067f50 2023-11-15T03:11:32.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.744+0000 7f8909d9b700 1 --2- 172.21.15.121:0/3652820918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890c067f50 0x7f890c0c45a0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 rx=0x7f890000e900 tx=0x7f8900009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:32.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.744+0000 7f88fa7fc700 1 -- 172.21.15.121:0/3652820918 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890000c640 con 0x7f890c067f50 2023-11-15T03:11:32.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.744+0000 7f88fa7fc700 1 -- 172.21.15.121:0/3652820918 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890000c7a0 con 0x7f890c067f50 2023-11-15T03:11:32.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.744+0000 7f8910a1b700 1 -- 172.21.15.121:0/3652820918 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f890c149640 con 0x7f890c067f50 2023-11-15T03:11:32.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.744+0000 7f88fa7fc700 1 -- 172.21.15.121:0/3652820918 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890000cb20 con 0x7f890c067f50 2023-11-15T03:11:32.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.744+0000 7f8910a1b700 1 -- 172.21.15.121:0/3652820918 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f890c149c00 con 0x7f890c067f50 2023-11-15T03:11:32.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.748+0000 7f8910a1b700 1 -- 172.21.15.121:0/3652820918 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f890c148ba0 con 0x7f890c067f50 2023-11-15T03:11:32.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.748+0000 7f88fa7fc700 1 -- 172.21.15.121:0/3652820918 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f890001b400 con 0x7f890c067f50 2023-11-15T03:11:32.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.748+0000 7f88fa7fc700 1 --2- 172.21.15.121:0/3652820918 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f88ec05a510 0x7f88ec05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:32.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.748+0000 7f88fa7fc700 1 -- 172.21.15.121:0/3652820918 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f8900074c60 con 0x7f890c067f50 2023-11-15T03:11:32.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.748+0000 7f890a59c700 1 --2- 172.21.15.121:0/3652820918 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f88ec05a510 0x7f88ec05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:32.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.752+0000 7f890a59c700 1 --2- 172.21.15.121:0/3652820918 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f88ec05a510 0x7f88ec05c9d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7f890c06a430 tx=0x7f88fc017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:32.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.756+0000 7f88fa7fc700 1 -- 172.21.15.121:0/3652820918 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f890c148ba0 con 0x7f890c067f50 2023-11-15T03:11:32.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:32.952+0000 7f8910a1b700 1 -- 172.21.15.121:0/3652820918 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f890c047060 con 0x7f890c067f50 2023-11-15T03:11:33.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.228+0000 7f88fa7fc700 1 -- 172.21.15.121:0/3652820918 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v352) v1 ==== 162+0+0 (secure 0 0 0) 0x7f8900016030 con 0x7f890c067f50 2023-11-15T03:11:33.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.232+0000 7f8910a1b700 1 -- 172.21.15.121:0/3652820918 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f88ec05a510 msgr2=0x7f88ec05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:33.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.232+0000 7f8910a1b700 1 --2- 172.21.15.121:0/3652820918 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f88ec05a510 0x7f88ec05c9d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7f890c06a430 tx=0x7f88fc017000).stop 2023-11-15T03:11:33.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.232+0000 7f8910a1b700 1 -- 172.21.15.121:0/3652820918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890c067f50 msgr2=0x7f890c0c45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:33.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.232+0000 7f8910a1b700 1 --2- 172.21.15.121:0/3652820918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890c067f50 0x7f890c0c45a0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 rx=0x7f890000e900 tx=0x7f8900009060).stop 2023-11-15T03:11:33.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.232+0000 7f8910a1b700 1 -- 172.21.15.121:0/3652820918 shutdown_connections 2023-11-15T03:11:33.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.232+0000 7f8910a1b700 1 --2- 172.21.15.121:0/3652820918 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f88ec05a510 0x7f88ec05c9d0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:33.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.232+0000 7f8910a1b700 1 --2- 172.21.15.121:0/3652820918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f890c0a8b80 0x7f890c0c4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:33.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.232+0000 7f8910a1b700 1 --2- 172.21.15.121:0/3652820918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890c067f50 0x7f890c0c45a0 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:33.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.232+0000 7f8910a1b700 1 --2- 172.21.15.121:0/3652820918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c065850 0x7f890c147480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:33.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.232+0000 7f8910a1b700 1 -- 172.21.15.121:0/3652820918 >> 172.21.15.121:0/3652820918 conn(0x7f890c0591c0 msgr2=0x7f890c0bc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:33.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.232+0000 7f8910a1b700 1 -- 172.21.15.121:0/3652820918 shutdown_connections 2023-11-15T03:11:33.244 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.232+0000 7f8910a1b700 1 -- 172.21.15.121:0/3652820918 wait complete. 2023-11-15T03:11:33.245 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-15T03:11:33.264 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:33.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.412+0000 7f4858e39700 1 Processor -- start 2023-11-15T03:11:33.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.412+0000 7f4858e39700 1 -- start start 2023-11-15T03:11:33.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.412+0000 7f4858e39700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f48540c3280 0x7f48540c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:33.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f4858e39700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48540c1d50 con 0x7f4854152140 2023-11-15T03:11:33.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f4858e39700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f485415cb80 con 0x7f485415be30 2023-11-15T03:11:33.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f4858e39700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48540c1860 con 0x7f48540c3280 2023-11-15T03:11:33.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f4852ffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4854152140 0x7f48541525f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:60914/0 (socket says 172.21.15.121:60914) 2023-11-15T03:11:33.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f4852ffd700 1 -- 172.21.15.121:0/3590679452 learned_addr learned my addr 172.21.15.121:0/3590679452 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:33.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f48537fe700 1 --2- 172.21.15.121:0/3590679452 >> v2:172.21.15.190:3300/0 conn(0x7f48540c3280 0x7f48540c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:33.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f48527fc700 1 -- 172.21.15.121:0/3590679452 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1417153857 0 0) 0x7f48540c1d50 con 0x7f4854152140 2023-11-15T03:11:33.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f48527fc700 1 -- 172.21.15.121:0/3590679452 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f483c0033f0 con 0x7f4854152140 2023-11-15T03:11:33.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f48527fc700 1 -- 172.21.15.121:0/3590679452 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816649558 0 0) 0x7f485415cb80 con 0x7f485415be30 2023-11-15T03:11:33.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f48527fc700 1 -- 172.21.15.121:0/3590679452 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48540c1d50 con 0x7f485415be30 2023-11-15T03:11:33.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f48537fe700 1 -- 172.21.15.121:0/3590679452 >> v1:172.21.15.190:6789/0 conn(0x7f485415be30 legacy=0x7f485415c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:33.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f48537fe700 1 -- 172.21.15.121:0/3590679452 >> v1:172.21.15.121:6789/0 conn(0x7f4854152140 legacy=0x7f48541525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:33.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f48537fe700 1 -- 172.21.15.121:0/3590679452 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48540a9240 con 0x7f48540c3280 2023-11-15T03:11:33.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f48537fe700 1 --2- 172.21.15.121:0/3590679452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48540c3280 0x7f48540c3660 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f4844009760 tx=0x7f4844009060).ready entity=mon.1 client_cookie=5eab77979bc4c1a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:33.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f48527fc700 1 -- 172.21.15.121:0/3590679452 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484400d4a0 con 0x7f48540c3280 2023-11-15T03:11:33.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f48527fc700 1 -- 172.21.15.121:0/3590679452 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484400d600 con 0x7f48540c3280 2023-11-15T03:11:33.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f48527fc700 1 -- 172.21.15.121:0/3590679452 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484400d980 con 0x7f48540c3280 2023-11-15T03:11:33.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f4858e39700 1 -- 172.21.15.121:0/3590679452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48540c3280 msgr2=0x7f48540c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:33.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f4858e39700 1 --2- 172.21.15.121:0/3590679452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48540c3280 0x7f48540c3660 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f4844009760 tx=0x7f4844009060).stop 2023-11-15T03:11:33.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f4858e39700 1 -- 172.21.15.121:0/3590679452 shutdown_connections 2023-11-15T03:11:33.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f4858e39700 1 --2- 172.21.15.121:0/3590679452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48540c3280 0x7f48540c3660 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:33.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.416+0000 7f4858e39700 1 -- 172.21.15.121:0/3590679452 >> 172.21.15.121:0/3590679452 conn(0x7f4854062190 msgr2=0x7f4854062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:33.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.420+0000 7f4858e39700 1 -- 172.21.15.121:0/3590679452 shutdown_connections 2023-11-15T03:11:33.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.420+0000 7f4858e39700 1 -- 172.21.15.121:0/3590679452 wait complete. 2023-11-15T03:11:33.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.420+0000 7f4858e39700 1 Processor -- start 2023-11-15T03:11:33.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.420+0000 7f4858e39700 1 -- start start 2023-11-15T03:11:33.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f4858e39700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48540c3280 0x7f48541694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:33.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f4858e39700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4854152140 0x7f4854158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:33.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f4858e39700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f485415be30 0x7f485416c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:33.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f4858e39700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4854168a90 con 0x7f48540c3280 2023-11-15T03:11:33.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f4858e39700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4854169040 con 0x7f485415be30 2023-11-15T03:11:33.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f4858e39700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48541699c0 con 0x7f4854152140 2023-11-15T03:11:33.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f48537fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48540c3280 0x7f48541694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:33.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f48537fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48540c3280 0x7f48541694d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40682/0 (socket says 172.21.15.121:40682) 2023-11-15T03:11:33.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f48537fe700 1 -- 172.21.15.121:0/1918702579 learned_addr learned my addr 172.21.15.121:0/1918702579 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:33.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f4853fff700 1 --2- 172.21.15.121:0/1918702579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f485415be30 0x7f485416c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:33.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f4852ffd700 1 --2- 172.21.15.121:0/1918702579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4854152140 0x7f4854158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:33.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f48537fe700 1 -- 172.21.15.121:0/1918702579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4854152140 msgr2=0x7f4854158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:33.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f48537fe700 1 --2- 172.21.15.121:0/1918702579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4854152140 0x7f4854158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:33.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f48537fe700 1 -- 172.21.15.121:0/1918702579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f485415be30 msgr2=0x7f485416c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:33.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f48537fe700 1 --2- 172.21.15.121:0/1918702579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f485415be30 0x7f485416c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:33.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f48537fe700 1 -- 172.21.15.121:0/1918702579 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48440092a0 con 0x7f48540c3280 2023-11-15T03:11:33.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f48537fe700 1 --2- 172.21.15.121:0/1918702579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48540c3280 0x7f48541694d0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 rx=0x7f48540654e0 tx=0x7f4844016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:33.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f483b7fe700 1 -- 172.21.15.121:0/1918702579 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484400d770 con 0x7f48540c3280 2023-11-15T03:11:33.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f483b7fe700 1 -- 172.21.15.121:0/1918702579 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484400d8d0 con 0x7f48540c3280 2023-11-15T03:11:33.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f4858e39700 1 -- 172.21.15.121:0/1918702579 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485416ca70 con 0x7f48540c3280 2023-11-15T03:11:33.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f483b7fe700 1 -- 172.21.15.121:0/1918702579 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484401d660 con 0x7f48540c3280 2023-11-15T03:11:33.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.424+0000 7f4858e39700 1 -- 172.21.15.121:0/1918702579 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4854168cd0 con 0x7f48540c3280 2023-11-15T03:11:33.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.428+0000 7f4858e39700 1 -- 172.21.15.121:0/1918702579 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48540c7500 con 0x7f48540c3280 2023-11-15T03:11:33.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.428+0000 7f483b7fe700 1 -- 172.21.15.121:0/1918702579 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f484401da50 con 0x7f48540c3280 2023-11-15T03:11:33.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.432+0000 7f483b7fe700 1 --2- 172.21.15.121:0/1918702579 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f484005a510 0x7f484005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:33.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.432+0000 7f483b7fe700 1 -- 172.21.15.121:0/1918702579 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f4844075ff0 con 0x7f48540c3280 2023-11-15T03:11:33.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.432+0000 7f4852ffd700 1 --2- 172.21.15.121:0/1918702579 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f484005a510 0x7f484005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:33.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.432+0000 7f4852ffd700 1 --2- 172.21.15.121:0/1918702579 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f484005a510 0x7f484005c9d0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7f48540aa020 tx=0x7f4848009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:33.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.440+0000 7f483b7fe700 1 -- 172.21.15.121:0/1918702579 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f48440a4af0 con 0x7f48540c3280 2023-11-15T03:11:33.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.640+0000 7f4858e39700 1 -- 172.21.15.121:0/1918702579 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48540c1f80 con 0x7f48540c3280 2023-11-15T03:11:33.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.644+0000 7f483b7fe700 1 -- 172.21.15.121:0/1918702579 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 352 v352) v1 ==== 95+0+4514 (secure 0 0 0) 0x7f48540c7500 con 0x7f48540c3280 2023-11-15T03:11:33.652 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:33.652 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":352,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/755048854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":755048854},{"type":"v1","addr":"172.21.15.190:6834","nonce":755048854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2360242049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2360242049},{"type":"v1","addr":"172.21.15.190:6835","nonce":2360242049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2780153683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2780153683},{"type":"v1","addr":"172.21.15.121:6835","nonce":2780153683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345}],"filesystems":[{"mdsmap":{"epoch":352,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:11:21.864101+0000","modified":"2023-11-15T03:11:33.222115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10899":{"gid":10899,"name":"a","rank":0,"incarnation":352,"state":"up:replay","state_seq":1,"addr":"172.21.15.121:6837/3298117008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3298117008},{"type":"v1","addr":"172.21.15.121:6837","nonce":3298117008}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-15T03:11:33.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.648+0000 7f4858e39700 1 -- 172.21.15.121:0/1918702579 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f484005a510 msgr2=0x7f484005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:33.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.648+0000 7f4858e39700 1 --2- 172.21.15.121:0/1918702579 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f484005a510 0x7f484005c9d0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7f48540aa020 tx=0x7f4848009040).stop 2023-11-15T03:11:33.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.648+0000 7f4858e39700 1 -- 172.21.15.121:0/1918702579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48540c3280 msgr2=0x7f48541694d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:33.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.648+0000 7f4858e39700 1 --2- 172.21.15.121:0/1918702579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48540c3280 0x7f48541694d0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 rx=0x7f48540654e0 tx=0x7f4844016000).stop 2023-11-15T03:11:33.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.648+0000 7f4858e39700 1 -- 172.21.15.121:0/1918702579 shutdown_connections 2023-11-15T03:11:33.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.648+0000 7f4858e39700 1 --2- 172.21.15.121:0/1918702579 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f484005a510 0x7f484005c9d0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:33.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.648+0000 7f4858e39700 1 --2- 172.21.15.121:0/1918702579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f485415be30 0x7f485416c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:33.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.648+0000 7f4858e39700 1 --2- 172.21.15.121:0/1918702579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4854152140 0x7f4854158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:33.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.648+0000 7f4858e39700 1 --2- 172.21.15.121:0/1918702579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48540c3280 0x7f48541694d0 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:33.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.648+0000 7f4858e39700 1 -- 172.21.15.121:0/1918702579 >> 172.21.15.121:0/1918702579 conn(0x7f4854062190 msgr2=0x7f48540a8230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:33.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.648+0000 7f4858e39700 1 -- 172.21.15.121:0/1918702579 shutdown_connections 2023-11-15T03:11:33.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:33.648+0000 7f4858e39700 1 -- 172.21.15.121:0/1918702579 wait complete. 2023-11-15T03:11:33.659 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 352 2023-11-15T03:11:34.082 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 352, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:11:21.864101+0000', 'modified': '2023-11-15T03:11:33.222115+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 146, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10899}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10899': {'gid': 10899, 'name': 'a', 'rank': 0, 'incarnation': 352, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.121:6837/3298117008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6836', 'nonce': 3298117008}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 3298117008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:11:34.083 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10899:up:replay 2023-11-15T03:11:35.084 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:35.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.244+0000 7f4ad05a9700 1 Processor -- start 2023-11-15T03:11:35.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.244+0000 7f4ad05a9700 1 -- start start 2023-11-15T03:11:35.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.248+0000 7f4ad05a9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4ac8153bb0 0x7f4ac80a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:35.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.248+0000 7f4ad05a9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac80cb5c0 con 0x7f4ac8068170 2023-11-15T03:11:35.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.248+0000 7f4ad05a9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac80c9ca0 con 0x7f4ac80aa020 2023-11-15T03:11:35.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.248+0000 7f4ad05a9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac80aa6f0 con 0x7f4ac8153bb0 2023-11-15T03:11:35.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.248+0000 7f4ace345700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4ac8153bb0 0x7f4ac80a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:35.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.248+0000 7f4ace345700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4ac8153bb0 0x7f4ac80a9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51304/0 (socket says 172.21.15.121:51304) 2023-11-15T03:11:35.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.248+0000 7f4ace345700 1 -- 172.21.15.121:0/498219075 learned_addr learned my addr 172.21.15.121:0/498219075 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:35.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.252+0000 7f4ace345700 1 -- 172.21.15.121:0/498219075 >> v1:172.21.15.190:6789/0 conn(0x7f4ac80aa020 legacy=0x7f4ac8067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:35.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.252+0000 7f4ace345700 1 -- 172.21.15.121:0/498219075 >> v1:172.21.15.121:6789/0 conn(0x7f4ac8068170 legacy=0x7f4ac8065d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:35.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.252+0000 7f4ace345700 1 -- 172.21.15.121:0/498219075 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac80caa20 con 0x7f4ac8153bb0 2023-11-15T03:11:35.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.252+0000 7f4ace345700 1 --2- 172.21.15.121:0/498219075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac8153bb0 0x7f4ac80a9b30 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 rx=0x7f4ac4006fd0 tx=0x7f4ac40090f0).ready entity=mon.0 client_cookie=c4547c7afc22466e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:35.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.252+0000 7f4acd343700 1 -- 172.21.15.121:0/498219075 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac400b600 con 0x7f4ac8153bb0 2023-11-15T03:11:35.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.252+0000 7f4acd343700 1 -- 172.21.15.121:0/498219075 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac400b760 con 0x7f4ac8153bb0 2023-11-15T03:11:35.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.252+0000 7f4acd343700 1 -- 172.21.15.121:0/498219075 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac400bae0 con 0x7f4ac8153bb0 2023-11-15T03:11:35.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.252+0000 7f4ad05a9700 1 -- 172.21.15.121:0/498219075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac8153bb0 msgr2=0x7f4ac80a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:35.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.252+0000 7f4ad05a9700 1 --2- 172.21.15.121:0/498219075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac8153bb0 0x7f4ac80a9b30 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 rx=0x7f4ac4006fd0 tx=0x7f4ac40090f0).stop 2023-11-15T03:11:35.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.252+0000 7f4ad05a9700 1 -- 172.21.15.121:0/498219075 shutdown_connections 2023-11-15T03:11:35.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.252+0000 7f4ad05a9700 1 --2- 172.21.15.121:0/498219075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac8153bb0 0x7f4ac80a9b30 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:35.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.252+0000 7f4ad05a9700 1 -- 172.21.15.121:0/498219075 >> 172.21.15.121:0/498219075 conn(0x7f4ac8058c50 msgr2=0x7f4ac8059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:35.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.252+0000 7f4ad05a9700 1 -- 172.21.15.121:0/498219075 shutdown_connections 2023-11-15T03:11:35.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.252+0000 7f4ad05a9700 1 -- 172.21.15.121:0/498219075 wait complete. 2023-11-15T03:11:35.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.252+0000 7f4ad05a9700 1 Processor -- start 2023-11-15T03:11:35.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4ad05a9700 1 -- start start 2023-11-15T03:11:35.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4ad05a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac8068170 0x7f4ac80c3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:35.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4ad05a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac80aa020 0x7f4ac80aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:35.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4ad05a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac8153bb0 0x7f4ac80acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:35.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4ad05a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac8165580 con 0x7f4ac8153bb0 2023-11-15T03:11:35.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4ad05a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80c3d80 con 0x7f4ac80aa020 2023-11-15T03:11:35.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4ad05a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac81650f0 con 0x7f4ac8068170 2023-11-15T03:11:35.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4aceb46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac8153bb0 0x7f4ac80acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:35.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4ace345700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac8068170 0x7f4ac80c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:35.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4ace345700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac8068170 0x7f4ac80c3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:59212/0 (socket says 172.21.15.121:59212) 2023-11-15T03:11:35.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4ace345700 1 -- 172.21.15.121:0/618093598 learned_addr learned my addr 172.21.15.121:0/618093598 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:35.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4aceb46700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac8153bb0 0x7f4ac80acbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51310/0 (socket says 172.21.15.121:51310) 2023-11-15T03:11:35.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4acdb44700 1 --2- 172.21.15.121:0/618093598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac80aa020 0x7f4ac80aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:35.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4ace345700 1 -- 172.21.15.121:0/618093598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac80aa020 msgr2=0x7f4ac80aaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:35.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4ace345700 1 --2- 172.21.15.121:0/618093598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac80aa020 0x7f4ac80aaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:35.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4ace345700 1 -- 172.21.15.121:0/618093598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac8153bb0 msgr2=0x7f4ac80acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:35.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4ace345700 1 --2- 172.21.15.121:0/618093598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac8153bb0 0x7f4ac80acbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:35.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4ace345700 1 -- 172.21.15.121:0/618093598 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac4009370 con 0x7f4ac8068170 2023-11-15T03:11:35.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4ace345700 1 --2- 172.21.15.121:0/618093598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac8068170 0x7f4ac80c3890 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f4ac4009f50 tx=0x7f4ac4017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:35.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4abe7fc700 1 -- 172.21.15.121:0/618093598 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac400b8d0 con 0x7f4ac8068170 2023-11-15T03:11:35.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4abe7fc700 1 -- 172.21.15.121:0/618093598 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac400ba30 con 0x7f4ac8068170 2023-11-15T03:11:35.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4ad05a9700 1 -- 172.21.15.121:0/618093598 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac81647c0 con 0x7f4ac8068170 2023-11-15T03:11:35.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4abe7fc700 1 -- 172.21.15.121:0/618093598 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac401b660 con 0x7f4ac8068170 2023-11-15T03:11:35.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.256+0000 7f4ad05a9700 1 -- 172.21.15.121:0/618093598 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac8164d80 con 0x7f4ac8068170 2023-11-15T03:11:35.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.260+0000 7f4ad05a9700 1 -- 172.21.15.121:0/618093598 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac806c150 con 0x7f4ac8068170 2023-11-15T03:11:35.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.260+0000 7f4abe7fc700 1 -- 172.21.15.121:0/618093598 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4ac401ba10 con 0x7f4ac8068170 2023-11-15T03:11:35.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.260+0000 7f4abe7fc700 1 --2- 172.21.15.121:0/618093598 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4aac05a510 0x7f4aac05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:35.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.260+0000 7f4abe7fc700 1 -- 172.21.15.121:0/618093598 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f4ac40764f0 con 0x7f4ac8068170 2023-11-15T03:11:35.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.260+0000 7f4acdb44700 1 --2- 172.21.15.121:0/618093598 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4aac05a510 0x7f4aac05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:35.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.264+0000 7f4acdb44700 1 --2- 172.21.15.121:0/618093598 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4aac05a510 0x7f4aac05c9d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7f4ac8152c30 tx=0x7f4ab8007d20).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:35.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.264+0000 7f4abe7fc700 1 -- 172.21.15.121:0/618093598 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4ac4049c40 con 0x7f4ac8068170 2023-11-15T03:11:35.467 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.460+0000 7f4ad05a9700 1 -- 172.21.15.121:0/618093598 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ac8068ae0 con 0x7f4ac8068170 2023-11-15T03:11:35.469 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.460+0000 7f4abe7fc700 1 -- 172.21.15.121:0/618093598 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 354 v354) v1 ==== 95+0+4514 (secure 0 0 0) 0x7f4ac401f070 con 0x7f4ac8068170 2023-11-15T03:11:35.469 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:35.469 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":354,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/755048854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":755048854},{"type":"v1","addr":"172.21.15.190:6834","nonce":755048854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2360242049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2360242049},{"type":"v1","addr":"172.21.15.190:6835","nonce":2360242049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2780153683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2780153683},{"type":"v1","addr":"172.21.15.121:6835","nonce":2780153683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345}],"filesystems":[{"mdsmap":{"epoch":354,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:11:21.864101+0000","modified":"2023-11-15T03:11:35.242836+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10899":{"gid":10899,"name":"a","rank":0,"incarnation":352,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.121:6837/3298117008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3298117008},{"type":"v1","addr":"172.21.15.121:6837","nonce":3298117008}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-15T03:11:35.473 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.464+0000 7f4ad05a9700 1 -- 172.21.15.121:0/618093598 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4aac05a510 msgr2=0x7f4aac05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:35.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.464+0000 7f4ad05a9700 1 --2- 172.21.15.121:0/618093598 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4aac05a510 0x7f4aac05c9d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7f4ac8152c30 tx=0x7f4ab8007d20).stop 2023-11-15T03:11:35.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.464+0000 7f4ad05a9700 1 -- 172.21.15.121:0/618093598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac8068170 msgr2=0x7f4ac80c3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:35.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.464+0000 7f4ad05a9700 1 --2- 172.21.15.121:0/618093598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac8068170 0x7f4ac80c3890 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f4ac4009f50 tx=0x7f4ac4017000).stop 2023-11-15T03:11:35.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.464+0000 7f4ad05a9700 1 -- 172.21.15.121:0/618093598 shutdown_connections 2023-11-15T03:11:35.474 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.464+0000 7f4ad05a9700 1 --2- 172.21.15.121:0/618093598 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4aac05a510 0x7f4aac05c9d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:35.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.464+0000 7f4ad05a9700 1 --2- 172.21.15.121:0/618093598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac8153bb0 0x7f4ac80acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:35.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.464+0000 7f4ad05a9700 1 --2- 172.21.15.121:0/618093598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac80aa020 0x7f4ac80aaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:35.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.464+0000 7f4ad05a9700 1 --2- 172.21.15.121:0/618093598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac8068170 0x7f4ac80c3890 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:35.475 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.464+0000 7f4ad05a9700 1 -- 172.21.15.121:0/618093598 >> 172.21.15.121:0/618093598 conn(0x7f4ac8058c50 msgr2=0x7f4ac80bfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:35.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.464+0000 7f4ad05a9700 1 -- 172.21.15.121:0/618093598 shutdown_connections 2023-11-15T03:11:35.476 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:35.464+0000 7f4ad05a9700 1 -- 172.21.15.121:0/618093598 wait complete. 2023-11-15T03:11:35.476 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 354 2023-11-15T03:11:35.900 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 354, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:11:21.864101+0000', 'modified': '2023-11-15T03:11:35.242836+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 146, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10899}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10899': {'gid': 10899, 'name': 'a', 'rank': 0, 'incarnation': 352, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.121:6837/3298117008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6836', 'nonce': 3298117008}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 3298117008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:11:35.901 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10899:up:rejoin 2023-11-15T03:11:36.902 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:37.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.060+0000 7f778b59e700 1 Processor -- start 2023-11-15T03:11:37.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.060+0000 7f778b59e700 1 -- start start 2023-11-15T03:11:37.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.060+0000 7f778b59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f778c0cafb0 0x7f778c066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.060+0000 7f778b59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f778c067270 0x7f778c0ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.060+0000 7f778b59e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778c0ad3a0 con 0x7f778c0cabd0 2023-11-15T03:11:37.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.060+0000 7f778b59e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f778c0cb410 con 0x7f778c0cafb0 2023-11-15T03:11:37.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.060+0000 7f778b59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778c0ad650 con 0x7f778c067270 2023-11-15T03:11:37.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f7781d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f778c0cafb0 0x7f778c066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f7781d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f778c0cafb0 0x7f778c066d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:39512/0 (socket says 172.21.15.121:39512) 2023-11-15T03:11:37.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f7781d9b700 1 -- 172.21.15.121:0/2968853910 learned_addr learned my addr 172.21.15.121:0/2968853910 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:37.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f778ad9d700 1 --2- 172.21.15.121:0/2968853910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f778c067270 0x7f778c0ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f7781d9b700 1 -- 172.21.15.121:0/2968853910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f778c067270 msgr2=0x7f778c0ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:37.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f7781d9b700 1 --2- 172.21.15.121:0/2968853910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f778c067270 0x7f778c0ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f7781d9b700 1 -- 172.21.15.121:0/2968853910 >> v1:172.21.15.190:6789/0 conn(0x7f778c0cabd0 legacy=0x7f778c0acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:37.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f7781d9b700 1 -- 172.21.15.121:0/2968853910 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778c065fc0 con 0x7f778c0cafb0 2023-11-15T03:11:37.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f7789d9b700 1 -- 172.21.15.121:0/2968853910 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140000102 0 0) 0x7f778c0ad3a0 con 0x7f778c0cabd0 2023-11-15T03:11:37.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f7781d9b700 1 --2- 172.21.15.121:0/2968853910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f778c0cafb0 0x7f778c066d80 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f7774006fd0 tx=0x7f77740090f0).ready entity=mon.1 client_cookie=e9b55fb7de6c950d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:37.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f7789d9b700 1 -- 172.21.15.121:0/2968853910 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777400b580 con 0x7f778c0cafb0 2023-11-15T03:11:37.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f7789d9b700 1 -- 172.21.15.121:0/2968853910 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777400b6e0 con 0x7f778c0cafb0 2023-11-15T03:11:37.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f7789d9b700 1 -- 172.21.15.121:0/2968853910 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777400ba60 con 0x7f778c0cafb0 2023-11-15T03:11:37.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f778b59e700 1 -- 172.21.15.121:0/2968853910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f778c0cafb0 msgr2=0x7f778c066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:37.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f778b59e700 1 --2- 172.21.15.121:0/2968853910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f778c0cafb0 0x7f778c066d80 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f7774006fd0 tx=0x7f77740090f0).stop 2023-11-15T03:11:37.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f778b59e700 1 -- 172.21.15.121:0/2968853910 shutdown_connections 2023-11-15T03:11:37.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f778b59e700 1 --2- 172.21.15.121:0/2968853910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f778c067270 0x7f778c0ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f778b59e700 1 --2- 172.21.15.121:0/2968853910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f778c0cafb0 0x7f778c066d80 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f778b59e700 1 -- 172.21.15.121:0/2968853910 >> 172.21.15.121:0/2968853910 conn(0x7f778c062190 msgr2=0x7f778c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:37.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f778b59e700 1 -- 172.21.15.121:0/2968853910 shutdown_connections 2023-11-15T03:11:37.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.064+0000 7f778b59e700 1 -- 172.21.15.121:0/2968853910 wait complete. 2023-11-15T03:11:37.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.068+0000 7f778b59e700 1 Processor -- start 2023-11-15T03:11:37.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.068+0000 7f778b59e700 1 -- start start 2023-11-15T03:11:37.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.068+0000 7f778b59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f778c067270 0x7f778c14f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.068+0000 7f778b59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f778c0cabd0 0x7f778c0bfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.068+0000 7f778b59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f778c0cafb0 0x7f778c0c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.068+0000 7f778b59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c0c37e0 con 0x7f778c067270 2023-11-15T03:11:37.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.068+0000 7f778b59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c0c2bd0 con 0x7f778c0cafb0 2023-11-15T03:11:37.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.068+0000 7f778b59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778c0c2eb0 con 0x7f778c0cabd0 2023-11-15T03:11:37.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.068+0000 7f778a59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f778c067270 0x7f778c14f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.068+0000 7f778a59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f778c067270 0x7f778c14f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51326/0 (socket says 172.21.15.121:51326) 2023-11-15T03:11:37.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.072+0000 7f778a59c700 1 -- 172.21.15.121:0/3660006931 learned_addr learned my addr 172.21.15.121:0/3660006931 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:37.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.072+0000 7f7781d9b700 1 --2- 172.21.15.121:0/3660006931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f778c0cabd0 0x7f778c0bfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.072+0000 7f778ad9d700 1 --2- 172.21.15.121:0/3660006931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f778c0cafb0 0x7f778c0c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.072+0000 7f778a59c700 1 -- 172.21.15.121:0/3660006931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f778c0cabd0 msgr2=0x7f778c0bfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:37.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.072+0000 7f778a59c700 1 --2- 172.21.15.121:0/3660006931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f778c0cabd0 0x7f778c0bfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.072+0000 7f778a59c700 1 -- 172.21.15.121:0/3660006931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f778c0cafb0 msgr2=0x7f778c0c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:37.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.072+0000 7f778a59c700 1 --2- 172.21.15.121:0/3660006931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f778c0cafb0 0x7f778c0c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.083 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.072+0000 7f778a59c700 1 -- 172.21.15.121:0/3660006931 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7774009370 con 0x7f778c067270 2023-11-15T03:11:37.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.072+0000 7f778a59c700 1 --2- 172.21.15.121:0/3660006931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f778c067270 0x7f778c14f880 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 rx=0x7f777c001eb0 tx=0x7f777c007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:37.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.072+0000 7f7782ffd700 1 -- 172.21.15.121:0/3660006931 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c00f470 con 0x7f778c067270 2023-11-15T03:11:37.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.072+0000 7f7782ffd700 1 -- 172.21.15.121:0/3660006931 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c00f5f0 con 0x7f778c067270 2023-11-15T03:11:37.084 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.072+0000 7f7782ffd700 1 -- 172.21.15.121:0/3660006931 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c00f9b0 con 0x7f778c067270 2023-11-15T03:11:37.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.072+0000 7f778b59e700 1 -- 172.21.15.121:0/3660006931 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f778c0c3190 con 0x7f778c067270 2023-11-15T03:11:37.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.072+0000 7f778b59e700 1 -- 172.21.15.121:0/3660006931 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778c14fd70 con 0x7f778c067270 2023-11-15T03:11:37.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.072+0000 7f778b59e700 1 -- 172.21.15.121:0/3660006931 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f778c06c240 con 0x7f778c067270 2023-11-15T03:11:37.085 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.076+0000 7f7782ffd700 1 -- 172.21.15.121:0/3660006931 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f777c00fd20 con 0x7f778c067270 2023-11-15T03:11:37.086 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.076+0000 7f7782ffd700 1 --2- 172.21.15.121:0/3660006931 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f776c05a510 0x7f776c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.076+0000 7f7782ffd700 1 -- 172.21.15.121:0/3660006931 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f777c073f30 con 0x7f778c067270 2023-11-15T03:11:37.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.076+0000 7f7781d9b700 1 --2- 172.21.15.121:0/3660006931 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f776c05a510 0x7f776c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.087 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.076+0000 7f7781d9b700 1 --2- 172.21.15.121:0/3660006931 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f776c05a510 0x7f776c05c9d0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7f7774009130 tx=0x7f7774017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:37.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.084+0000 7f7782ffd700 1 -- 172.21.15.121:0/3660006931 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f778c06c240 con 0x7f778c067270 2023-11-15T03:11:37.291 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.280+0000 7f778b59e700 1 -- 172.21.15.121:0/3660006931 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f778c06c240 con 0x7f778c067270 2023-11-15T03:11:37.292 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.284+0000 7f7782ffd700 1 -- 172.21.15.121:0/3660006931 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 355 v355) v1 ==== 95+0+4514 (secure 0 0 0) 0x7f778c06c240 con 0x7f778c067270 2023-11-15T03:11:37.293 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:37.293 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":355,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/755048854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":755048854},{"type":"v1","addr":"172.21.15.190:6834","nonce":755048854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2360242049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2360242049},{"type":"v1","addr":"172.21.15.190:6835","nonce":2360242049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2780153683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2780153683},{"type":"v1","addr":"172.21.15.121:6835","nonce":2780153683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345}],"filesystems":[{"mdsmap":{"epoch":355,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:11:21.864101+0000","modified":"2023-11-15T03:11:36.256557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10899":{"gid":10899,"name":"a","rank":0,"incarnation":352,"state":"up:active","state_seq":4,"addr":"172.21.15.121:6837/3298117008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3298117008},{"type":"v1","addr":"172.21.15.121:6837","nonce":3298117008}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-15T03:11:37.295 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.284+0000 7f778b59e700 1 -- 172.21.15.121:0/3660006931 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f776c05a510 msgr2=0x7f776c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:37.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.284+0000 7f778b59e700 1 --2- 172.21.15.121:0/3660006931 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f776c05a510 0x7f776c05c9d0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7f7774009130 tx=0x7f7774017000).stop 2023-11-15T03:11:37.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.284+0000 7f778b59e700 1 -- 172.21.15.121:0/3660006931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f778c067270 msgr2=0x7f778c14f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:37.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.284+0000 7f778b59e700 1 --2- 172.21.15.121:0/3660006931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f778c067270 0x7f778c14f880 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 rx=0x7f777c001eb0 tx=0x7f777c007c70).stop 2023-11-15T03:11:37.296 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.284+0000 7f778b59e700 1 -- 172.21.15.121:0/3660006931 shutdown_connections 2023-11-15T03:11:37.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.284+0000 7f778b59e700 1 --2- 172.21.15.121:0/3660006931 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f776c05a510 0x7f776c05c9d0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.297 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.284+0000 7f778b59e700 1 --2- 172.21.15.121:0/3660006931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f778c0cafb0 0x7f778c0c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.284+0000 7f778b59e700 1 --2- 172.21.15.121:0/3660006931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f778c0cabd0 0x7f778c0bfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.284+0000 7f778b59e700 1 --2- 172.21.15.121:0/3660006931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f778c067270 0x7f778c14f880 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.298 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.284+0000 7f778b59e700 1 -- 172.21.15.121:0/3660006931 >> 172.21.15.121:0/3660006931 conn(0x7f778c062190 msgr2=0x7f778c151740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:37.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.284+0000 7f778b59e700 1 -- 172.21.15.121:0/3660006931 shutdown_connections 2023-11-15T03:11:37.299 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.284+0000 7f778b59e700 1 -- 172.21.15.121:0/3660006931 wait complete. 2023-11-15T03:11:37.300 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 355 2023-11-15T03:11:37.721 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 355, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-15T03:11:21.864101+0000', 'modified': '2023-11-15T03:11:36.256557+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 146, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10899}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10899': {'gid': 10899, 'name': 'a', 'rank': 0, 'incarnation': 352, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.121:6837/3298117008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6836', 'nonce': 3298117008}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 3298117008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-15T03:11:37.721 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-15T03:11:37.722 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-15T03:11:37.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.880+0000 7ffaa1307700 1 Processor -- start 2023-11-15T03:11:37.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.884+0000 7ffaa1307700 1 -- start start 2023-11-15T03:11:37.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.884+0000 7ffaa1307700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa9c0abaa0 0x7ffa9c0abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.884+0000 7ffaa1307700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffa9c0bb510 0x7ffa9c063570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.884+0000 7ffaa1307700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9c0c2c00 con 0x7ffa9c0a8880 2023-11-15T03:11:37.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.884+0000 7ffaa1307700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa9c063a60 con 0x7ffa9c0abaa0 2023-11-15T03:11:37.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.884+0000 7ffaa1307700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa9c0657b0 con 0x7ffa9c0bb510 2023-11-15T03:11:37.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.884+0000 7ffa9bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa9c0abaa0 0x7ffa9c0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.884+0000 7ffa9bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa9c0abaa0 0x7ffa9c0abe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51328/0 (socket says 172.21.15.121:51328) 2023-11-15T03:11:37.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.884+0000 7ffa9bfff700 1 -- 172.21.15.121:0/614646382 learned_addr learned my addr 172.21.15.121:0/614646382 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:37.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.884+0000 7ffa93fff700 1 --2- 172.21.15.121:0/614646382 >> v2:172.21.15.190:3300/0 conn(0x7ffa9c0bb510 0x7ffa9c063570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.884+0000 7ffa9bfff700 1 -- 172.21.15.121:0/614646382 >> v2:172.21.15.190:3300/0 conn(0x7ffa9c0bb510 msgr2=0x7ffa9c063570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:37.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.884+0000 7ffa9bfff700 1 --2- 172.21.15.121:0/614646382 >> v2:172.21.15.190:3300/0 conn(0x7ffa9c0bb510 0x7ffa9c063570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.884+0000 7ffa9bfff700 1 -- 172.21.15.121:0/614646382 >> v1:172.21.15.121:6789/0 conn(0x7ffa9c0a8880 legacy=0x7ffa9c151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:37.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.884+0000 7ffa9bfff700 1 -- 172.21.15.121:0/614646382 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c065a90 con 0x7ffa9c0abaa0 2023-11-15T03:11:37.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.884+0000 7ffa93fff700 1 --2- 172.21.15.121:0/614646382 >> v2:172.21.15.190:3300/0 conn(0x7ffa9c0bb510 0x7ffa9c063570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:11:37.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.884+0000 7ffa9bfff700 1 --2- 172.21.15.121:0/614646382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa9c0abaa0 0x7ffa9c0abe80 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 rx=0x7ffa9c1554d0 tx=0x7ffa8c0090f0).ready entity=mon.0 client_cookie=31816e688aea19c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:37.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.888+0000 7ffa9b7fe700 1 -- 172.21.15.121:0/614646382 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c00b600 con 0x7ffa9c0abaa0 2023-11-15T03:11:37.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.888+0000 7ffa9b7fe700 1 -- 172.21.15.121:0/614646382 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8c00b760 con 0x7ffa9c0abaa0 2023-11-15T03:11:37.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.888+0000 7ffa9b7fe700 1 -- 172.21.15.121:0/614646382 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c00bae0 con 0x7ffa9c0abaa0 2023-11-15T03:11:37.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.888+0000 7ffaa1307700 1 -- 172.21.15.121:0/614646382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa9c0abaa0 msgr2=0x7ffa9c0abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:37.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.888+0000 7ffaa1307700 1 --2- 172.21.15.121:0/614646382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa9c0abaa0 0x7ffa9c0abe80 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 rx=0x7ffa9c1554d0 tx=0x7ffa8c0090f0).stop 2023-11-15T03:11:37.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.888+0000 7ffaa1307700 1 -- 172.21.15.121:0/614646382 shutdown_connections 2023-11-15T03:11:37.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.888+0000 7ffaa1307700 1 --2- 172.21.15.121:0/614646382 >> v2:172.21.15.190:3300/0 conn(0x7ffa9c0bb510 0x7ffa9c063570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.888+0000 7ffaa1307700 1 --2- 172.21.15.121:0/614646382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa9c0abaa0 0x7ffa9c0abe80 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.888+0000 7ffaa1307700 1 -- 172.21.15.121:0/614646382 >> 172.21.15.121:0/614646382 conn(0x7ffa9c067a60 msgr2=0x7ffa9c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:37.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.888+0000 7ffaa1307700 1 -- 172.21.15.121:0/614646382 shutdown_connections 2023-11-15T03:11:37.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.888+0000 7ffaa1307700 1 -- 172.21.15.121:0/614646382 wait complete. 2023-11-15T03:11:37.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.892+0000 7ffaa1307700 1 Processor -- start 2023-11-15T03:11:37.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.892+0000 7ffaa1307700 1 -- start start 2023-11-15T03:11:37.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.892+0000 7ffaa1307700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa9c0a8880 0x7ffa9c0bd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.892+0000 7ffaa1307700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa9c0abaa0 0x7ffa9c162b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.892+0000 7ffaa1307700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa9c0bb510 0x7ffa9c14a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.892+0000 7ffaa1307700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c1427d0 con 0x7ffa9c0bb510 2023-11-15T03:11:37.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.892+0000 7ffaa1307700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c1464f0 con 0x7ffa9c0a8880 2023-11-15T03:11:37.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.892+0000 7ffaa1307700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c163010 con 0x7ffa9c0abaa0 2023-11-15T03:11:37.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.892+0000 7ffaa0b06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa9c0bb510 0x7ffa9c14a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.896+0000 7ffaa0b06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa9c0bb510 0x7ffa9c14a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51340/0 (socket says 172.21.15.121:51340) 2023-11-15T03:11:37.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.896+0000 7ffaa0b06700 1 -- 172.21.15.121:0/3665053589 learned_addr learned my addr 172.21.15.121:0/3665053589 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:37.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.896+0000 7ffa93fff700 1 --2- 172.21.15.121:0/3665053589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa9c0abaa0 0x7ffa9c162b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.896+0000 7ffa9bfff700 1 --2- 172.21.15.121:0/3665053589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa9c0a8880 0x7ffa9c0bd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.907 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.896+0000 7ffaa0b06700 1 -- 172.21.15.121:0/3665053589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa9c0abaa0 msgr2=0x7ffa9c162b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:37.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.896+0000 7ffaa0b06700 1 --2- 172.21.15.121:0/3665053589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa9c0abaa0 0x7ffa9c162b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.896+0000 7ffaa0b06700 1 -- 172.21.15.121:0/3665053589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa9c0a8880 msgr2=0x7ffa9c0bd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:37.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.896+0000 7ffaa0b06700 1 --2- 172.21.15.121:0/3665053589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa9c0a8880 0x7ffa9c0bd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.896+0000 7ffaa0b06700 1 -- 172.21.15.121:0/3665053589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c009370 con 0x7ffa9c0bb510 2023-11-15T03:11:37.908 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.896+0000 7ffaa0b06700 1 --2- 172.21.15.121:0/3665053589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa9c0bb510 0x7ffa9c14a4f0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 rx=0x7ffa9400c7c0 tx=0x7ffa94001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:37.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.896+0000 7ffa98ff9700 1 -- 172.21.15.121:0/3665053589 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9400e4a0 con 0x7ffa9c0bb510 2023-11-15T03:11:37.909 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.896+0000 7ffa98ff9700 1 -- 172.21.15.121:0/3665053589 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9400e600 con 0x7ffa9c0bb510 2023-11-15T03:11:37.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.896+0000 7ffa98ff9700 1 -- 172.21.15.121:0/3665053589 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9400e980 con 0x7ffa9c0bb510 2023-11-15T03:11:37.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.896+0000 7ffaa1307700 1 -- 172.21.15.121:0/3665053589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9c0bdd80 con 0x7ffa9c0bb510 2023-11-15T03:11:37.910 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.896+0000 7ffaa1307700 1 -- 172.21.15.121:0/3665053589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9c0be340 con 0x7ffa9c0bb510 2023-11-15T03:11:37.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.900+0000 7ffa98ff9700 1 -- 172.21.15.121:0/3665053589 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ffa940123f0 con 0x7ffa9c0bb510 2023-11-15T03:11:37.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.904+0000 7ffa98ff9700 1 --2- 172.21.15.121:0/3665053589 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa7c05a540 0x7ffa7c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.911 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.904+0000 7ffa98ff9700 1 -- 172.21.15.121:0/3665053589 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ffa940740e0 con 0x7ffa9c0bb510 2023-11-15T03:11:37.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.904+0000 7ffa9bfff700 1 --2- 172.21.15.121:0/3665053589 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa7c05a540 0x7ffa7c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.912 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.904+0000 7ffa9bfff700 1 --2- 172.21.15.121:0/3665053589 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa7c05a540 0x7ffa7c05ca00 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7ffa8c009130 tx=0x7ffa8c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:37.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.904+0000 7ffaac919740 1 Processor -- start 2023-11-15T03:11:37.913 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.904+0000 7ffaac919740 1 -- start start 2023-11-15T03:11:37.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffaac919740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a46ad0 0x2a6d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffaac919740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2a93140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffaac919740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a93630 0x2a93ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.916 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffaac919740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x29c7410 con 0x2a46eb0 2023-11-15T03:11:37.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffaac919740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x29c7550 con 0x2a46ad0 2023-11-15T03:11:37.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffaac919740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x29c7a00 con 0x2a93630 2023-11-15T03:11:37.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffa9bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a46ad0 0x2a6d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffa93fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2a93140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.917 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffa9bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a46ad0 0x2a6d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:39540/0 (socket says 172.21.15.121:39540) 2023-11-15T03:11:37.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffa9bfff700 1 -- 172.21.15.121:0/2499790120 learned_addr learned my addr 172.21.15.121:0/2499790120 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:37.918 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffaa0b06700 1 --2- 172.21.15.121:0/2499790120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a93630 0x2a93ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.919 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffa93fff700 1 -- 172.21.15.121:0/2499790120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a93630 msgr2=0x2a93ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:37.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffa93fff700 1 --2- 172.21.15.121:0/2499790120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a93630 0x2a93ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffa93fff700 1 -- 172.21.15.121:0/2499790120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a46ad0 msgr2=0x2a6d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:37.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffa93fff700 1 --2- 172.21.15.121:0/2499790120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a46ad0 0x2a6d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.920 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffa93fff700 1 -- 172.21.15.121:0/2499790120 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x29c0730 con 0x2a46eb0 2023-11-15T03:11:37.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffa9bfff700 1 --2- 172.21.15.121:0/2499790120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a46ad0 0x2a6d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:37.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffa93fff700 1 --2- 172.21.15.121:0/2499790120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2a93140 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 rx=0x7ffa8400aa40 tx=0x7ffa84008040).ready entity=mon.0 client_cookie=16bcb29121ec29e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:37.921 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffa917fa700 1 -- 172.21.15.121:0/2499790120 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8400a4f0 con 0x2a46eb0 2023-11-15T03:11:37.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffa917fa700 1 -- 172.21.15.121:0/2499790120 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8400c040 con 0x2a46eb0 2023-11-15T03:11:37.922 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffa917fa700 1 -- 172.21.15.121:0/2499790120 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8400fb10 con 0x2a46eb0 2023-11-15T03:11:37.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffaac919740 1 -- 172.21.15.121:0/2499790120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 msgr2=0x2a93140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:37.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffaac919740 1 --2- 172.21.15.121:0/2499790120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2a93140 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 rx=0x7ffa8400aa40 tx=0x7ffa84008040).stop 2023-11-15T03:11:37.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffaac919740 1 -- 172.21.15.121:0/2499790120 shutdown_connections 2023-11-15T03:11:37.923 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffaac919740 1 --2- 172.21.15.121:0/2499790120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a93630 0x2a93ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffaac919740 1 --2- 172.21.15.121:0/2499790120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2a93140 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffaac919740 1 --2- 172.21.15.121:0/2499790120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a46ad0 0x2a6d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffaac919740 1 -- 172.21.15.121:0/2499790120 >> 172.21.15.121:0/2499790120 conn(0x2825550 msgr2=0x2a457e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:37.924 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffaac919740 1 -- 172.21.15.121:0/2499790120 shutdown_connections 2023-11-15T03:11:37.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.908+0000 7ffaac919740 1 -- 172.21.15.121:0/2499790120 wait complete. 2023-11-15T03:11:37.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffaac919740 1 -- start start 2023-11-15T03:11:37.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffaac919740 1 Processor -- start 2023-11-15T03:11:37.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffaac919740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a46ad0 0x2a8fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffaac919740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2a6c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffaac919740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a93630 0x2a6f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.933 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffaac919740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2992650 con 0x2a46eb0 2023-11-15T03:11:37.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffaac919740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a73ad0 con 0x2a93630 2023-11-15T03:11:37.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffaac919740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2a71ab0 con 0x2a46ad0 2023-11-15T03:11:37.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffaac919740 10 client.0 ll_register_callbacks cb 0x29ff020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:11:37.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffa93fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2a6c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffa9bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a46ad0 0x2a8fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffa9bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a46ad0 0x2a8fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:59256/0 (socket says 172.21.15.121:59256) 2023-11-15T03:11:37.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffaa0b06700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a93630 0x2a6f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffa9bfff700 1 -- 172.21.15.121:0/2046284817 learned_addr learned my addr 172.21.15.121:0/2046284817 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:37.935 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffa93fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2a6c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51356/0 (socket says 172.21.15.121:51356) 2023-11-15T03:11:37.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffa93fff700 1 -- 172.21.15.121:0/2046284817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a46ad0 msgr2=0x2a8fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:37.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffa93fff700 1 --2- 172.21.15.121:0/2046284817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a46ad0 0x2a8fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffa93fff700 1 -- 172.21.15.121:0/2046284817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a93630 msgr2=0x2a6f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:37.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffa93fff700 1 --2- 172.21.15.121:0/2046284817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a93630 0x2a6f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.936 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffa93fff700 1 -- 172.21.15.121:0/2046284817 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9c065a90 con 0x2a46eb0 2023-11-15T03:11:37.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffa93fff700 1 --2- 172.21.15.121:0/2046284817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2a6c810 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 rx=0x7ffa84016000 tx=0x7ffa84008040).ready entity=mon.0 client_cookie=51dee76e3addcc05 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:37.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.912+0000 7ffa7a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:11:37.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffa7a7fc700 1 -- 172.21.15.121:0/2046284817 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa84006e60 con 0x2a46eb0 2023-11-15T03:11:37.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffa7a7fc700 1 -- 172.21.15.121:0/2046284817 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ffa84005a90 con 0x2a46eb0 2023-11-15T03:11:37.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffaac919740 1 -- 172.21.15.121:0/2046284817 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x29bd1a0 con 0x2a46eb0 2023-11-15T03:11:37.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffa7a7fc700 1 -- 172.21.15.121:0/2046284817 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=355) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa84005e60 con 0x2a46eb0 2023-11-15T03:11:37.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffaac919740 10 client.10950 fetch_fsmap learned FSMap version 355 2023-11-15T03:11:37.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffaac919740 1 -- 172.21.15.121:0/2046284817 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=355,monmap=2+}) v3 -- 0x7ffa8400a6a0 con 0x2a46eb0 2023-11-15T03:11:37.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffa7a7fc700 1 -- 172.21.15.121:0/2046284817 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x29bd1a0 con 0x2a46eb0 2023-11-15T03:11:37.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffa7a7fc700 1 -- 172.21.15.121:0/2046284817 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 355) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ffa84011040 con 0x2a46eb0 2023-11-15T03:11:37.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffaac919740 10 client.10950 fetch_fsmap finished waiting for FSMap version 355 2023-11-15T03:11:37.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffaac919740 10 client.10950 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:11:37.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffaac919740 20 client.10950 populate_metadata read hostname 'smithi121' 2023-11-15T03:11:37.939 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffaac919740 1 --2- 172.21.15.121:0/2046284817 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x2a70dd0 0x2a711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffaac919740 4 client.10950 mds_command: new command op to 10899 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:11:37.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffaac919740 1 -- 172.21.15.121:0/2046284817 --> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x29b36f0 con 0x2a70dd0 2023-11-15T03:11:37.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffa9bfff700 1 --2- 172.21.15.121:0/2046284817 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x2a70dd0 0x2a711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffa7a7fc700 10 client.10950 ms_handle_connect on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:37.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffa9bfff700 1 --2- 172.21.15.121:0/2046284817 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x2a70dd0 0x2a711b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=a9efdf50de9fbbeb server_cookie=8033acba8055ac75 in_seq=0 out_seq=0 2023-11-15T03:11:37.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffa7a7fc700 1 -- 172.21.15.121:0/2046284817 <== mds.0 v2:172.21.15.121:6836/3298117008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x29b36f0 con 0x2a70dd0 2023-11-15T03:11:37.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffa7a7fc700 10 client.10950 handle_command_reply: tid=0 2023-11-15T03:11:37.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffaac919740 1 client.10950 shutdown 2023-11-15T03:11:37.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffaac919740 1 -- 172.21.15.121:0/2046284817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 msgr2=0x2a6c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:37.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.916+0000 7ffaac919740 1 --2- 172.21.15.121:0/2046284817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2a6c810 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 rx=0x7ffa84016000 tx=0x7ffa84008040).stop 2023-11-15T03:11:37.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.920+0000 7ffaac919740 1 -- 172.21.15.121:0/2046284817 shutdown_connections 2023-11-15T03:11:37.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.920+0000 7ffaac919740 1 --2- 172.21.15.121:0/2046284817 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x2a70dd0 0x2a711b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.920+0000 7ffaac919740 1 --2- 172.21.15.121:0/2046284817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a93630 0x2a6f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.920+0000 7ffaac919740 1 --2- 172.21.15.121:0/2046284817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2a6c810 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.920+0000 7ffaac919740 1 --2- 172.21.15.121:0/2046284817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a46ad0 0x2a8fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.920+0000 7ffa7a7fc700 0 client.10950 ms_handle_reset on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:37.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.920+0000 7ffaac919740 1 -- 172.21.15.121:0/2046284817 >> 172.21.15.121:0/2046284817 conn(0x2825550 msgr2=0x2a8e580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:37.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.920+0000 7ffaac919740 1 -- 172.21.15.121:0/2046284817 shutdown_connections 2023-11-15T03:11:37.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.920+0000 7ffaac919740 1 -- 172.21.15.121:0/2046284817 wait complete. 2023-11-15T03:11:37.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.920+0000 7ffaac919740 20 client.10950 trim_cache size 0 max 16384 2023-11-15T03:11:37.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.936+0000 7ffaac919740 1 Processor -- start 2023-11-15T03:11:37.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.936+0000 7ffaac919740 1 -- start start 2023-11-15T03:11:37.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffaac919740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a46ad0 0x2a40570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffaac919740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2a40a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffaac919740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a93630 0x2a86160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffaac919740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a0ae60 con 0x2a46eb0 2023-11-15T03:11:37.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffaac919740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a42340 con 0x2a46ad0 2023-11-15T03:11:37.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffaac919740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2a22e20 con 0x2a93630 2023-11-15T03:11:37.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffa93fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2a40a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffa93fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2a40a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51370/0 (socket says 172.21.15.121:51370) 2023-11-15T03:11:37.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffa93fff700 1 -- 172.21.15.121:0/349722772 learned_addr learned my addr 172.21.15.121:0/349722772 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:37.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffa9bfff700 1 --2- 172.21.15.121:0/349722772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a46ad0 0x2a40570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.950 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffaa0b06700 1 --2- 172.21.15.121:0/349722772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a93630 0x2a86160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffa93fff700 1 -- 172.21.15.121:0/349722772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a93630 msgr2=0x2a86160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:37.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffa93fff700 1 --2- 172.21.15.121:0/349722772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a93630 0x2a86160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffa93fff700 1 -- 172.21.15.121:0/349722772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a46ad0 msgr2=0x2a40570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:37.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffa93fff700 1 --2- 172.21.15.121:0/349722772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a46ad0 0x2a40570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffa93fff700 1 -- 172.21.15.121:0/349722772 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x29bb960 con 0x2a46eb0 2023-11-15T03:11:37.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffa9bfff700 1 --2- 172.21.15.121:0/349722772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a46ad0 0x2a40570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:37.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffa93fff700 1 --2- 172.21.15.121:0/349722772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2a40a60 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 rx=0x7ffa84010010 tx=0x7ffa84020040).ready entity=mon.0 client_cookie=13cac1e679d55615 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:37.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffa91ffb700 1 -- 172.21.15.121:0/349722772 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8400f6d0 con 0x2a46eb0 2023-11-15T03:11:37.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffa91ffb700 1 -- 172.21.15.121:0/349722772 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa84027470 con 0x2a46eb0 2023-11-15T03:11:37.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffa91ffb700 1 -- 172.21.15.121:0/349722772 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa84027950 con 0x2a46eb0 2023-11-15T03:11:37.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffaac919740 1 -- 172.21.15.121:0/349722772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 msgr2=0x2a40a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:37.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffaac919740 1 --2- 172.21.15.121:0/349722772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2a40a60 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 rx=0x7ffa84010010 tx=0x7ffa84020040).stop 2023-11-15T03:11:37.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffaac919740 1 -- 172.21.15.121:0/349722772 shutdown_connections 2023-11-15T03:11:37.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffaac919740 1 --2- 172.21.15.121:0/349722772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a93630 0x2a86160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffaac919740 1 --2- 172.21.15.121:0/349722772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2a40a60 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.956 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffaac919740 1 --2- 172.21.15.121:0/349722772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a46ad0 0x2a40570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.940+0000 7ffaac919740 1 -- 172.21.15.121:0/349722772 >> 172.21.15.121:0/349722772 conn(0x2825550 msgr2=0x2a47650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:37.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffaac919740 1 -- 172.21.15.121:0/349722772 shutdown_connections 2023-11-15T03:11:37.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffaac919740 1 -- 172.21.15.121:0/349722772 wait complete. 2023-11-15T03:11:37.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffaac919740 1 -- start start 2023-11-15T03:11:37.957 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffaac919740 1 Processor -- start 2023-11-15T03:11:37.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffaac919740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a46ad0 0x2a8a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffaac919740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2aef150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.958 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffaac919740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a93630 0x2a86680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffaac919740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a43980 con 0x2a46eb0 2023-11-15T03:11:37.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffaac919740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a8a9d0 con 0x2a46ad0 2023-11-15T03:11:37.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffaac919740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x2a479b0 con 0x2a93630 2023-11-15T03:11:37.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffaac919740 10 client.0 ll_register_callbacks cb 0x7ffa8400ff40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:11:37.959 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffa93fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2aef150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffa93fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2aef150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51382/0 (socket says 172.21.15.121:51382) 2023-11-15T03:11:37.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffa93fff700 1 -- 172.21.15.121:0/2518121817 learned_addr learned my addr 172.21.15.121:0/2518121817 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:37.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffa9bfff700 1 --2- 172.21.15.121:0/2518121817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a46ad0 0x2a8a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffaa0b06700 1 --2- 172.21.15.121:0/2518121817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a93630 0x2a86680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.960 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffa93fff700 1 -- 172.21.15.121:0/2518121817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a93630 msgr2=0x2a86680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:37.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffa93fff700 1 --2- 172.21.15.121:0/2518121817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a93630 0x2a86680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffa93fff700 1 -- 172.21.15.121:0/2518121817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a46ad0 msgr2=0x2a8a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:37.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffa93fff700 1 --2- 172.21.15.121:0/2518121817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a46ad0 0x2a8a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffa93fff700 1 -- 172.21.15.121:0/2518121817 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa8400a6a0 con 0x2a46eb0 2023-11-15T03:11:37.961 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffaa0b06700 1 --2- 172.21.15.121:0/2518121817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a93630 0x2a86680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffa93fff700 1 --2- 172.21.15.121:0/2518121817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2aef150 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 rx=0x7ffa8400aa40 tx=0x7ffa84029040).ready entity=mon.0 client_cookie=33459c36710fc325 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.944+0000 7ffa7a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:11:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffa7a7fc700 1 -- 172.21.15.121:0/2518121817 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa84027a70 con 0x2a46eb0 2023-11-15T03:11:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 1 -- 172.21.15.121:0/2518121817 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa8400c040 con 0x2a46eb0 2023-11-15T03:11:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffa7a7fc700 1 -- 172.21.15.121:0/2518121817 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ffa84039ca0 con 0x2a46eb0 2023-11-15T03:11:37.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffa7a7fc700 1 -- 172.21.15.121:0/2518121817 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=355) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa84039e60 con 0x2a46eb0 2023-11-15T03:11:37.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 10 client.10956 fetch_fsmap learned FSMap version 355 2023-11-15T03:11:37.963 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 1 -- 172.21.15.121:0/2518121817 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=355,monmap=2+}) v3 -- 0x29bb6b0 con 0x2a46eb0 2023-11-15T03:11:37.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffa7a7fc700 1 -- 172.21.15.121:0/2518121817 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8400c040 con 0x2a46eb0 2023-11-15T03:11:37.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffa7a7fc700 1 -- 172.21.15.121:0/2518121817 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 355) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ffa84011070 con 0x2a46eb0 2023-11-15T03:11:37.966 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 10 client.10956 fetch_fsmap finished waiting for FSMap version 355 2023-11-15T03:11:37.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 10 client.10956 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:11:37.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 20 client.10956 populate_metadata read hostname 'smithi121' 2023-11-15T03:11:37.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 1 --2- 172.21.15.121:0/2518121817 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x2a86b70 0x2a42710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:37.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 4 client.10956 mds_command: new command op to 10899 tid=0[{"prefix": "status"}] 2023-11-15T03:11:37.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 1 -- 172.21.15.121:0/2518121817 --> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] -- command(tid 0: {"prefix": "status"}) v1 -- 0x287c4e0 con 0x2a86b70 2023-11-15T03:11:37.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffa9bfff700 1 --2- 172.21.15.121:0/2518121817 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x2a86b70 0x2a42710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:37.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffa7a7fc700 10 client.10956 ms_handle_connect on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:37.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffa9bfff700 1 --2- 172.21.15.121:0/2518121817 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x2a86b70 0x2a42710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=f4c86eefd2b8ae99 server_cookie=9764c9d0d66ab040 in_seq=0 out_seq=0 2023-11-15T03:11:37.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffa7a7fc700 1 -- 172.21.15.121:0/2518121817 <== mds.0 v2:172.21.15.121:6836/3298117008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x287c4e0 con 0x2a86b70 2023-11-15T03:11:37.968 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffa7a7fc700 10 client.10956 handle_command_reply: tid=0 2023-11-15T03:11:37.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 1 client.10956 shutdown 2023-11-15T03:11:37.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 1 -- 172.21.15.121:0/2518121817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 msgr2=0x2aef150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:37.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 1 --2- 172.21.15.121:0/2518121817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2aef150 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 rx=0x7ffa8400aa40 tx=0x7ffa84029040).stop 2023-11-15T03:11:37.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 1 -- 172.21.15.121:0/2518121817 shutdown_connections 2023-11-15T03:11:37.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 1 --2- 172.21.15.121:0/2518121817 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x2a86b70 0x2a42710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 1 --2- 172.21.15.121:0/2518121817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x2a93630 0x2a86680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 1 --2- 172.21.15.121:0/2518121817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x2a46eb0 0x2aef150 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 1 --2- 172.21.15.121:0/2518121817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x2a46ad0 0x2a8a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 1 -- 172.21.15.121:0/2518121817 >> 172.21.15.121:0/2518121817 conn(0x2825550 msgr2=0x2a90700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:37.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffa7a7fc700 0 client.10956 ms_handle_reset on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:37.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 1 -- 172.21.15.121:0/2518121817 shutdown_connections 2023-11-15T03:11:37.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.948+0000 7ffaac919740 1 -- 172.21.15.121:0/2518121817 wait complete. 2023-11-15T03:11:37.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.952+0000 7ffaac919740 20 client.10956 trim_cache size 0 max 16384 2023-11-15T03:11:37.972 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:11:37.972 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "be3e38c1-527f-4644-9b96-70c42632cdc5", 2023-11-15T03:11:37.972 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2023-11-15T03:11:37.972 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 10899, 2023-11-15T03:11:37.973 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2023-11-15T03:11:37.973 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2023-11-15T03:11:37.973 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2023-11-15T03:11:37.973 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 4.7235207829999997, 2023-11-15T03:11:37.973 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 355, 2023-11-15T03:11:37.974 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 146, 2023-11-15T03:11:37.974 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 146, 2023-11-15T03:11:37.974 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 5.49680283 2023-11-15T03:11:37.974 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:11:37.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.952+0000 7ffaa1307700 1 -- 172.21.15.121:0/3665053589 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa7c05a540 msgr2=0x7ffa7c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:37.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.952+0000 7ffaa1307700 1 --2- 172.21.15.121:0/3665053589 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa7c05a540 0x7ffa7c05ca00 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7ffa8c009130 tx=0x7ffa8c017000).stop 2023-11-15T03:11:37.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.952+0000 7ffaa1307700 1 -- 172.21.15.121:0/3665053589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa9c0bb510 msgr2=0x7ffa9c14a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:37.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.952+0000 7ffaa1307700 1 --2- 172.21.15.121:0/3665053589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa9c0bb510 0x7ffa9c14a4f0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 rx=0x7ffa9400c7c0 tx=0x7ffa94001510).stop 2023-11-15T03:11:37.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.952+0000 7ffaa1307700 1 -- 172.21.15.121:0/3665053589 shutdown_connections 2023-11-15T03:11:37.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.952+0000 7ffaa1307700 1 --2- 172.21.15.121:0/3665053589 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ffa7c05a540 0x7ffa7c05ca00 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.952+0000 7ffaa1307700 1 --2- 172.21.15.121:0/3665053589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa9c0bb510 0x7ffa9c14a4f0 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.952+0000 7ffaa1307700 1 --2- 172.21.15.121:0/3665053589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa9c0abaa0 0x7ffa9c162b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.952+0000 7ffaa1307700 1 --2- 172.21.15.121:0/3665053589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa9c0a8880 0x7ffa9c0bd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:37.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.952+0000 7ffaa1307700 1 -- 172.21.15.121:0/3665053589 >> 172.21.15.121:0/3665053589 conn(0x7ffa9c067a60 msgr2=0x7ffa9c14cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:37.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.952+0000 7ffaa1307700 1 -- 172.21.15.121:0/3665053589 shutdown_connections 2023-11-15T03:11:37.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:37.952+0000 7ffaa1307700 1 -- 172.21.15.121:0/3665053589 wait complete. 2023-11-15T03:11:37.979 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2023-11-15T03:11:38.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.100+0000 7f7161479700 1 Processor -- start 2023-11-15T03:11:38.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.100+0000 7f7161479700 1 -- start start 2023-11-15T03:11:38.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.100+0000 7f7161479700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f715c06a0c0 0x7f715c06a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:38.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.100+0000 7f7161479700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f715c068330 0x7f715c151510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:38.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.100+0000 7f7161479700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f715c151a00 0x7f715c063aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:38.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.100+0000 7f7161479700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715c065dc0 con 0x7f715c068330 2023-11-15T03:11:38.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.100+0000 7f7161479700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715c1566c0 con 0x7f715c151a00 2023-11-15T03:11:38.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.100+0000 7f7161479700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715c155b40 con 0x7f715c06a0c0 2023-11-15T03:11:38.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f715b7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f715c068330 0x7f715c151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:38.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f7160c78700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f715c151a00 0x7f715c063aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:38.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f715b7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f715c068330 0x7f715c151510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51392/0 (socket says 172.21.15.121:51392) 2023-11-15T03:11:38.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f715b7fe700 1 -- 172.21.15.121:0/1343993557 learned_addr learned my addr 172.21.15.121:0/1343993557 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:38.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f715bfff700 1 --2- 172.21.15.121:0/1343993557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f715c06a0c0 0x7f715c06a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:38.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f715b7fe700 1 -- 172.21.15.121:0/1343993557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f715c06a0c0 msgr2=0x7f715c06a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:38.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f715b7fe700 1 --2- 172.21.15.121:0/1343993557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f715c06a0c0 0x7f715c06a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:38.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f715b7fe700 1 -- 172.21.15.121:0/1343993557 >> v2:172.21.15.190:3300/0 conn(0x7f715c151a00 msgr2=0x7f715c063aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:38.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f715b7fe700 1 --2- 172.21.15.121:0/1343993557 >> v2:172.21.15.190:3300/0 conn(0x7f715c151a00 0x7f715c063aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:38.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f715b7fe700 1 -- 172.21.15.121:0/1343993557 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c155e20 con 0x7f715c068330 2023-11-15T03:11:38.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f7160c78700 1 --2- 172.21.15.121:0/1343993557 >> v2:172.21.15.190:3300/0 conn(0x7f715c151a00 0x7f715c063aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:38.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f715b7fe700 1 --2- 172.21.15.121:0/1343993557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f715c068330 0x7f715c151510 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 rx=0x7f715c061410 tx=0x7f71500090f0).ready entity=mon.0 client_cookie=349347991036b674 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:38.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f715affd700 1 -- 172.21.15.121:0/1343993557 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715000f510 con 0x7f715c068330 2023-11-15T03:11:38.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f715affd700 1 -- 172.21.15.121:0/1343993557 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715000f670 con 0x7f715c068330 2023-11-15T03:11:38.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f715affd700 1 -- 172.21.15.121:0/1343993557 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715000fa10 con 0x7f715c068330 2023-11-15T03:11:38.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f7161479700 1 -- 172.21.15.121:0/1343993557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f715c068330 msgr2=0x7f715c151510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:38.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f7161479700 1 --2- 172.21.15.121:0/1343993557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f715c068330 0x7f715c151510 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 rx=0x7f715c061410 tx=0x7f71500090f0).stop 2023-11-15T03:11:38.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f7161479700 1 -- 172.21.15.121:0/1343993557 shutdown_connections 2023-11-15T03:11:38.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f7161479700 1 --2- 172.21.15.121:0/1343993557 >> v2:172.21.15.190:3300/0 conn(0x7f715c151a00 0x7f715c063aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:38.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f7161479700 1 --2- 172.21.15.121:0/1343993557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f715c068330 0x7f715c151510 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:38.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f7161479700 1 --2- 172.21.15.121:0/1343993557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f715c06a0c0 0x7f715c06a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:38.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f7161479700 1 -- 172.21.15.121:0/1343993557 >> 172.21.15.121:0/1343993557 conn(0x7f715c0677c0 msgr2=0x7f715c0c65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:38.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f7161479700 1 -- 172.21.15.121:0/1343993557 shutdown_connections 2023-11-15T03:11:38.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.104+0000 7f7161479700 1 -- 172.21.15.121:0/1343993557 wait complete. 2023-11-15T03:11:38.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.108+0000 7f7161479700 1 Processor -- start 2023-11-15T03:11:38.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.108+0000 7f7161479700 1 -- start start 2023-11-15T03:11:38.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.108+0000 7f7161479700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f715c068330 0x7f715c14d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:38.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.108+0000 7f7161479700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f715c06a0c0 0x7f715c0a7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:38.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.108+0000 7f7161479700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f715c151a00 0x7f715c0bd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:38.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.108+0000 7f7161479700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c152e60 con 0x7f715c151a00 2023-11-15T03:11:38.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.108+0000 7f7161479700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c152250 con 0x7f715c06a0c0 2023-11-15T03:11:38.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.108+0000 7f7161479700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715c152530 con 0x7f715c068330 2023-11-15T03:11:38.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.112+0000 7f7160c78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f715c151a00 0x7f715c0bd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:38.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.112+0000 7f7160c78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f715c151a00 0x7f715c0bd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51398/0 (socket says 172.21.15.121:51398) 2023-11-15T03:11:38.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.112+0000 7f7160c78700 1 -- 172.21.15.121:0/2906351144 learned_addr learned my addr 172.21.15.121:0/2906351144 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:38.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.112+0000 7f715b7fe700 1 --2- 172.21.15.121:0/2906351144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f715c06a0c0 0x7f715c0a7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:38.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.112+0000 7f715bfff700 1 --2- 172.21.15.121:0/2906351144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f715c068330 0x7f715c14d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:38.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.112+0000 7f7160c78700 1 -- 172.21.15.121:0/2906351144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f715c068330 msgr2=0x7f715c14d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:38.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.112+0000 7f7160c78700 1 --2- 172.21.15.121:0/2906351144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f715c068330 0x7f715c14d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:38.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.112+0000 7f7160c78700 1 -- 172.21.15.121:0/2906351144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f715c06a0c0 msgr2=0x7f715c0a7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:38.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.112+0000 7f7160c78700 1 --2- 172.21.15.121:0/2906351144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f715c06a0c0 0x7f715c0a7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:38.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.112+0000 7f7160c78700 1 -- 172.21.15.121:0/2906351144 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7154009a20 con 0x7f715c151a00 2023-11-15T03:11:38.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.112+0000 7f7160c78700 1 --2- 172.21.15.121:0/2906351144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f715c151a00 0x7f715c0bd6d0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 rx=0x7f7154009f20 tx=0x7f715400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:38.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.112+0000 7f7143fff700 1 -- 172.21.15.121:0/2906351144 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715400cb70 con 0x7f715c151a00 2023-11-15T03:11:38.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.112+0000 7f7143fff700 1 -- 172.21.15.121:0/2906351144 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715400f040 con 0x7f715c151a00 2023-11-15T03:11:38.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.112+0000 7f7161479700 1 -- 172.21.15.121:0/2906351144 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7150009370 con 0x7f715c151a00 2023-11-15T03:11:38.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.112+0000 7f7143fff700 1 -- 172.21.15.121:0/2906351144 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154013400 con 0x7f715c151a00 2023-11-15T03:11:38.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.112+0000 7f7161479700 1 -- 172.21.15.121:0/2906351144 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715c152af0 con 0x7f715c151a00 2023-11-15T03:11:38.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.112+0000 7f7161479700 1 -- 172.21.15.121:0/2906351144 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715c152fd0 con 0x7f715c151a00 2023-11-15T03:11:38.129 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.116+0000 7f7143fff700 1 -- 172.21.15.121:0/2906351144 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f71540137f0 con 0x7f715c151a00 2023-11-15T03:11:38.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.116+0000 7f7143fff700 1 --2- 172.21.15.121:0/2906351144 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f714805a510 0x7f714805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:38.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.116+0000 7f7143fff700 1 -- 172.21.15.121:0/2906351144 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f71540750b0 con 0x7f715c151a00 2023-11-15T03:11:38.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.116+0000 7f715bfff700 1 --2- 172.21.15.121:0/2906351144 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f714805a510 0x7f714805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:38.130 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.116+0000 7f715bfff700 1 --2- 172.21.15.121:0/2906351144 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f714805a510 0x7f714805c9d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f714c009ba0 tx=0x7f714c009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:38.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.124+0000 7f7143fff700 1 -- 172.21.15.121:0/2906351144 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f71540487c0 con 0x7f715c151a00 2023-11-15T03:11:38.248 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:11:38.240+0000 7fea46f11700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:11:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.328+0000 7f7161479700 1 -- 172.21.15.121:0/2906351144 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f715c156100 con 0x7f715c151a00 2023-11-15T03:11:38.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.328+0000 7f7143fff700 1 -- 172.21.15.121:0/2906351144 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v355) v1 ==== 53+0+73 (secure 0 0 0) 0x7f715404bde0 con 0x7f715c151a00 2023-11-15T03:11:38.336 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-15T03:11:38.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.328+0000 7f7161479700 1 -- 172.21.15.121:0/2906351144 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f714805a510 msgr2=0x7f714805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:38.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.328+0000 7f7161479700 1 --2- 172.21.15.121:0/2906351144 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f714805a510 0x7f714805c9d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f714c009ba0 tx=0x7f714c009060).stop 2023-11-15T03:11:38.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.332+0000 7f7161479700 1 -- 172.21.15.121:0/2906351144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f715c151a00 msgr2=0x7f715c0bd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:38.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.332+0000 7f7161479700 1 --2- 172.21.15.121:0/2906351144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f715c151a00 0x7f715c0bd6d0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 rx=0x7f7154009f20 tx=0x7f715400b040).stop 2023-11-15T03:11:38.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.332+0000 7f7161479700 1 -- 172.21.15.121:0/2906351144 shutdown_connections 2023-11-15T03:11:38.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.332+0000 7f7161479700 1 --2- 172.21.15.121:0/2906351144 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f714805a510 0x7f714805c9d0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:38.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.332+0000 7f7161479700 1 --2- 172.21.15.121:0/2906351144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f715c151a00 0x7f715c0bd6d0 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:38.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.332+0000 7f7161479700 1 --2- 172.21.15.121:0/2906351144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f715c06a0c0 0x7f715c0a7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:38.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.332+0000 7f7161479700 1 --2- 172.21.15.121:0/2906351144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f715c068330 0x7f715c14d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:38.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.332+0000 7f7161479700 1 -- 172.21.15.121:0/2906351144 >> 172.21.15.121:0/2906351144 conn(0x7f715c0677c0 msgr2=0x7f715c0c2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:38.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.332+0000 7f7161479700 1 -- 172.21.15.121:0/2906351144 shutdown_connections 2023-11-15T03:11:38.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:38.332+0000 7f7161479700 1 -- 172.21.15.121:0/2906351144 wait complete. 2023-11-15T03:11:38.359 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-15T03:11:38.360 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-15T03:11:38.360 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-11-15T03:11:38.360 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-15T03:11:38.361 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-15T03:11:38.361 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-15T03:11:38.361 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-15T03:11:38.362 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-15T03:11:38.362 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:11:38.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:38.363 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-11-15T03:11:38.382 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-15T03:11:38.383 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-15T03:11:38.383 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-11-15T03:11:38.383 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:11:38.383 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-11-15T03:11:38.384 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:11:38.384 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:11:38.384 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:11:38.385 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-11-15T03:11:38.385 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:11:38.385 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:11:38.385 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-11-15T03:11:38.386 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-15T03:11:38.386 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:11:38.386 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-15T03:11:38.386 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:11:38.387 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-15T03:11:38.387 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 41541sec preferred_lft 41541sec 2023-11-15T03:11:38.387 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-11-15T03:11:38.387 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:11:38.387 INFO:teuthology.orchestra.run.smithi121.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-15T03:11:38.388 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 02:42:d3:11:9b:c1 brd ff:ff:ff:ff:ff:ff 2023-11-15T03:11:38.388 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-15T03:11:38.388 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:11:38.389 INFO:teuthology.orchestra.run.smithi121.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-15T03:11:38.389 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff 2023-11-15T03:11:38.389 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-15T03:11:38.389 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:11:38.389 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ccb7:d7ff:fe03:f4ee/64 scope link 2023-11-15T03:11:38.390 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:11:38.391 INFO:teuthology.orchestra.run.smithi121.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:11:38.391 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 1a:d8:3c:50:df:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T03:11:38.391 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::18d8:3cff:fe50:dfdf/64 scope link 2023-11-15T03:11:38.392 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:11:38.392 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-15T03:11:38.392 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 22:72:ae:53:1f:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T03:11:38.392 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2072:aeff:fe53:1f2b/64 scope link 2023-11-15T03:11:38.392 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-11-15T03:11:38.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:38.393 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-15T03:11:38.471 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-11-15T03:11:38.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:38.475 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-11-15T03:11:38.538 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-11-15T03:11:38.539 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-15T03:11:38.539 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-15T03:11:38.539 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-15T03:11:38.539 INFO:teuthology.orchestra.run.smithi121.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-15T03:11:38.539 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-15T03:11:38.540 INFO:teuthology.orchestra.run.smithi121.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-15T03:11:38.540 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-15T03:11:38.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:38.541 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-11-15T03:11:38.541 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-11-15T03:11:38.541 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-15T03:11:38.542 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-15T03:11:38.542 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-15T03:11:38.543 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-11-15T03:11:38.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:38.677 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T03:11:38.691 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T03:11:38.692 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T03:11:38.693 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-15T03:11:38.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:38.693 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-11-15T03:11:38.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:38.766 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-15T03:11:38.829 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-15T03:11:38.830 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:11:38.941 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:11:38.942 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:11:38.968 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-15T03:11:38.968 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-15T03:11:38.975 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-15T03:11:38.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:38.975 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-15T03:11:39.008 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-15T03:11:39.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd96510cd00 1 Processor -- start 2023-11-15T03:11:39.108 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd96510cd00 1 -- start start 2023-11-15T03:11:39.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd96510cd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a423748de0 0x55a423773bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd96510cd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a423776e40 0x55a423777280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd96510cd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a4238243b0 0x55a423777770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.109 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd96510cd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4236c0220 con 0x55a423748de0 2023-11-15T03:11:39.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd96510cd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a423791830 con 0x55a423776e40 2023-11-15T03:11:39.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd96510cd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a423791b10 con 0x55a4238243b0 2023-11-15T03:11:39.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd9629cd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a423748de0 0x55a423773bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.110 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd9629cd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a423748de0 0x55a423773bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51408/0 (socket says 172.21.15.121:51408) 2023-11-15T03:11:39.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd9629cd700 1 -- 172.21.15.121:0/60099439 learned_addr learned my addr 172.21.15.121:0/60099439 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:39.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd95a1cc700 1 --2- 172.21.15.121:0/60099439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a423776e40 0x55a423777280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.111 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd9631ce700 1 --2- 172.21.15.121:0/60099439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a4238243b0 0x55a423777770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.112 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd9629cd700 1 -- 172.21.15.121:0/60099439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a4238243b0 msgr2=0x55a423777770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:39.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd9629cd700 1 --2- 172.21.15.121:0/60099439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a4238243b0 0x55a423777770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd9629cd700 1 -- 172.21.15.121:0/60099439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a423776e40 msgr2=0x55a423777280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:39.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd9629cd700 1 --2- 172.21.15.121:0/60099439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a423776e40 0x55a423777280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd9629cd700 1 -- 172.21.15.121:0/60099439 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c00a0e0 con 0x55a423748de0 2023-11-15T03:11:39.113 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd9629cd700 1 --2- 172.21.15.121:0/60099439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a423748de0 0x55a423773bd0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 rx=0x7fd95400ad70 tx=0x7fd954016000).ready entity=mon.0 client_cookie=3dcd71f27eb2ca4a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:39.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd9619cb700 1 -- 172.21.15.121:0/60099439 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd954005370 con 0x55a423748de0 2023-11-15T03:11:39.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd9619cb700 1 -- 172.21.15.121:0/60099439 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd954000bc0 con 0x55a423748de0 2023-11-15T03:11:39.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.100+0000 7fd9619cb700 1 -- 172.21.15.121:0/60099439 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9540089d0 con 0x55a423748de0 2023-11-15T03:11:39.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd96510cd00 1 -- 172.21.15.121:0/60099439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a423748de0 msgr2=0x55a423773bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:39.114 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd96510cd00 1 --2- 172.21.15.121:0/60099439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a423748de0 0x55a423773bd0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 rx=0x7fd95400ad70 tx=0x7fd954016000).stop 2023-11-15T03:11:39.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd96510cd00 1 -- 172.21.15.121:0/60099439 shutdown_connections 2023-11-15T03:11:39.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd96510cd00 1 --2- 172.21.15.121:0/60099439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a4238243b0 0x55a423777770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd96510cd00 1 --2- 172.21.15.121:0/60099439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a423776e40 0x55a423777280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd96510cd00 1 --2- 172.21.15.121:0/60099439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a423748de0 0x55a423773bd0 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.115 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd96510cd00 1 -- 172.21.15.121:0/60099439 >> 172.21.15.121:0/60099439 conn(0x55a42378ff80 msgr2=0x55a423827380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:39.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd96510cd00 1 -- 172.21.15.121:0/60099439 shutdown_connections 2023-11-15T03:11:39.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd96510cd00 1 -- 172.21.15.121:0/60099439 wait complete. 2023-11-15T03:11:39.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd96510cd00 1 Processor -- start 2023-11-15T03:11:39.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd96510cd00 1 -- start start 2023-11-15T03:11:39.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd96510cd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a423748de0 0x55a42381e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.118 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd96510cd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a423776e40 0x55a4238384b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd96510cd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a4238243b0 0x55a4238389a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd96510cd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a423779040 con 0x55a423748de0 2023-11-15T03:11:39.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd96510cd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a423778d60 con 0x55a4238243b0 2023-11-15T03:11:39.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd96510cd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a42381d2d0 con 0x55a423776e40 2023-11-15T03:11:39.119 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd9629cd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a423748de0 0x55a42381e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd9629cd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a423748de0 0x55a42381e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51410/0 (socket says 172.21.15.121:51410) 2023-11-15T03:11:39.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd9629cd700 1 -- 172.21.15.121:0/458982603 learned_addr learned my addr 172.21.15.121:0/458982603 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:39.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd9631ce700 1 --2- 172.21.15.121:0/458982603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a4238243b0 0x55a4238389a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.104+0000 7fd95a1cc700 1 --2- 172.21.15.121:0/458982603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a423776e40 0x55a4238384b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.120 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.108+0000 7fd9629cd700 1 -- 172.21.15.121:0/458982603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a423776e40 msgr2=0x55a4238384b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:39.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.108+0000 7fd9629cd700 1 --2- 172.21.15.121:0/458982603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a423776e40 0x55a4238384b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.108+0000 7fd9629cd700 1 -- 172.21.15.121:0/458982603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a4238243b0 msgr2=0x55a4238389a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:39.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.108+0000 7fd9629cd700 1 --2- 172.21.15.121:0/458982603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a4238243b0 0x55a4238389a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.108+0000 7fd9629cd700 1 -- 172.21.15.121:0/458982603 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4236e9a20 con 0x55a423748de0 2023-11-15T03:11:39.121 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.108+0000 7fd9629cd700 1 --2- 172.21.15.121:0/458982603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a423748de0 0x55a42381e0a0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 rx=0x7fd95401f040 tx=0x7fd95401c000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:39.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.108+0000 7fd95bfff700 1 -- 172.21.15.121:0/458982603 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95400a250 con 0x55a423748de0 2023-11-15T03:11:39.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.108+0000 7fd95bfff700 1 -- 172.21.15.121:0/458982603 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd954013650 con 0x55a423748de0 2023-11-15T03:11:39.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.108+0000 7fd96510cd00 1 -- 172.21.15.121:0/458982603 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd954005db0 con 0x55a423748de0 2023-11-15T03:11:39.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.108+0000 7fd95bfff700 1 -- 172.21.15.121:0/458982603 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd954013800 con 0x55a423748de0 2023-11-15T03:11:39.122 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.108+0000 7fd96510cd00 1 -- 172.21.15.121:0/458982603 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4236e9770 con 0x55a423748de0 2023-11-15T03:11:39.123 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.108+0000 7fd95bfff700 1 -- 172.21.15.121:0/458982603 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd95402d020 con 0x55a423748de0 2023-11-15T03:11:39.124 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.112+0000 7fd95bfff700 1 --2- 172.21.15.121:0/458982603 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd948058ff0 0x7fd94805b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.112+0000 7fd95bfff700 1 -- 172.21.15.121:0/458982603 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fd9540760a0 con 0x55a423748de0 2023-11-15T03:11:39.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.112+0000 7fd95a1cc700 1 --2- 172.21.15.121:0/458982603 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd948058ff0 0x7fd94805b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.112+0000 7fd96510cd00 1 --2- 172.21.15.121:0/458982603 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x55a42381b5d0 0x55a423788e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.112+0000 7fd96510cd00 1 -- 172.21.15.121:0/458982603 --> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected e146) v8 -- 0x55a4238274a0 con 0x55a42381b5d0 2023-11-15T03:11:39.125 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.112+0000 7fd9631ce700 1 --2- 172.21.15.121:0/458982603 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x55a42381b5d0 0x55a423788e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.112+0000 7fd95a1cc700 1 --2- 172.21.15.121:0/458982603 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd948058ff0 0x7fd94805b4b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x55a4238238c0 tx=0x7fd94c001840).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:39.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.112+0000 7fd9631ce700 1 --2- 172.21.15.121:0/458982603 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x55a42381b5d0 0x55a423788e90 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:39.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.112+0000 7fd9631ce700 1 -- 172.21.15.121:0/458982603 <== osd.3 v2:172.21.15.121:6820/33135 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (91) out=91b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+91 (crc 0 0 0) 0x7fd95c00ad80 con 0x55a42381b5d0 2023-11-15T03:11:39.126 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.112+0000 7fd96510cd00 1 -- 172.21.15.121:0/458982603 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x55a42381b5d0 msgr2=0x55a423788e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:39.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.112+0000 7fd96510cd00 1 --2- 172.21.15.121:0/458982603 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x55a42381b5d0 0x55a423788e90 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.112+0000 7fd96510cd00 1 -- 172.21.15.121:0/458982603 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd948058ff0 msgr2=0x7fd94805b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:39.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.112+0000 7fd96510cd00 1 --2- 172.21.15.121:0/458982603 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd948058ff0 0x7fd94805b4b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x55a4238238c0 tx=0x7fd94c001840).stop 2023-11-15T03:11:39.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.112+0000 7fd96510cd00 1 -- 172.21.15.121:0/458982603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a423748de0 msgr2=0x55a42381e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:39.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.112+0000 7fd96510cd00 1 --2- 172.21.15.121:0/458982603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a423748de0 0x55a42381e0a0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 rx=0x7fd95401f040 tx=0x7fd95401c000).stop 2023-11-15T03:11:39.127 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.112+0000 7fd9631ce700 1 -- 172.21.15.121:0/458982603 reap_dead start 2023-11-15T03:11:39.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.116+0000 7fd96510cd00 1 -- 172.21.15.121:0/458982603 shutdown_connections 2023-11-15T03:11:39.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.116+0000 7fd96510cd00 1 -- 172.21.15.121:0/458982603 >> 172.21.15.121:0/458982603 conn(0x55a42378ff80 msgr2=0x55a423774b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:39.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.116+0000 7fd96510cd00 1 -- 172.21.15.121:0/458982603 shutdown_connections 2023-11-15T03:11:39.128 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.116+0000 7fd96510cd00 1 -- 172.21.15.121:0/458982603 wait complete. 2023-11-15T03:11:39.129 INFO:teuthology.orchestra.run.smithi121.stdout:Ufile1dir1 2023-11-15T03:11:39.130 DEBUG:teuthology.orchestra.run.smithi121:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-15T03:11:39.213 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:11:39.214 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 1099511627777, 2023-11-15T03:11:39.214 INFO:teuthology.orchestra.run.smithi121.stdout: "ancestors": [ 2023-11-15T03:11:39.214 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:39.215 INFO:teuthology.orchestra.run.smithi121.stdout: "dirino": 1099511627776, 2023-11-15T03:11:39.215 INFO:teuthology.orchestra.run.smithi121.stdout: "dname": "file1", 2023-11-15T03:11:39.215 INFO:teuthology.orchestra.run.smithi121.stdout: "version": 8 2023-11-15T03:11:39.215 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:39.215 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:39.216 INFO:teuthology.orchestra.run.smithi121.stdout: "dirino": 1, 2023-11-15T03:11:39.216 INFO:teuthology.orchestra.run.smithi121.stdout: "dname": "dir1", 2023-11-15T03:11:39.216 INFO:teuthology.orchestra.run.smithi121.stdout: "version": 29 2023-11-15T03:11:39.216 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:39.217 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2023-11-15T03:11:39.217 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 16, 2023-11-15T03:11:39.217 INFO:teuthology.orchestra.run.smithi121.stdout: "old_pools": [] 2023-11-15T03:11:39.217 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:11:39.218 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:39.220 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent rhubarb 2023-11-15T03:11:39.326 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.316+0000 7f6536333d00 1 Processor -- start 2023-11-15T03:11:39.327 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.316+0000 7f6536333d00 1 -- start start 2023-11-15T03:11:39.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f6536333d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5582e910fdf0 0x5582e9112220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.328 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f6536333d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5582e9141430 0x5582e9112710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f6536333d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582e9141810 0x5582e913d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f6536333d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e90bdcc0 con 0x5582e9141430 2023-11-15T03:11:39.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f6536333d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e9088220 con 0x5582e9141810 2023-11-15T03:11:39.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f6536333d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582e9112c00 con 0x5582e910fdf0 2023-11-15T03:11:39.329 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f652b23d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582e9141810 0x5582e913d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f652b23d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582e9141810 0x5582e913d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:39596/0 (socket says 172.21.15.121:39596) 2023-11-15T03:11:39.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f652b23d700 1 -- 172.21.15.121:0/1280341779 learned_addr learned my addr 172.21.15.121:0/1280341779 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:39.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f652a23b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5582e9141430 0x5582e9112710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.330 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f652aa3c700 1 --2- 172.21.15.121:0/1280341779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5582e910fdf0 0x5582e9112220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f652b23d700 1 -- 172.21.15.121:0/1280341779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5582e910fdf0 msgr2=0x5582e9112220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:39.332 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f652b23d700 1 --2- 172.21.15.121:0/1280341779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5582e910fdf0 0x5582e9112220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f652b23d700 1 -- 172.21.15.121:0/1280341779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5582e9141430 msgr2=0x5582e9112710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:39.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f652b23d700 1 --2- 172.21.15.121:0/1280341779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5582e9141430 0x5582e9112710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f652b23d700 1 -- 172.21.15.121:0/1280341779 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652400a0e0 con 0x5582e9141810 2023-11-15T03:11:39.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f652a23b700 1 --2- 172.21.15.121:0/1280341779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5582e9141430 0x5582e9112710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:39.333 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f652b23d700 1 --2- 172.21.15.121:0/1280341779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582e9141810 0x5582e913d930 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x5582e913bb00 tx=0x7f6524015000).ready entity=mon.1 client_cookie=a537196673ef81ca server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:39.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f6529239700 1 -- 172.21.15.121:0/1280341779 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652400d470 con 0x5582e9141810 2023-11-15T03:11:39.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f6529239700 1 -- 172.21.15.121:0/1280341779 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652400dcc0 con 0x5582e9141810 2023-11-15T03:11:39.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f6529239700 1 -- 172.21.15.121:0/1280341779 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652400de30 con 0x5582e9141810 2023-11-15T03:11:39.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f6536333d00 1 -- 172.21.15.121:0/1280341779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582e9141810 msgr2=0x5582e913d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:39.334 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f6536333d00 1 --2- 172.21.15.121:0/1280341779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582e9141810 0x5582e913d930 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x5582e913bb00 tx=0x7f6524015000).stop 2023-11-15T03:11:39.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f6536333d00 1 -- 172.21.15.121:0/1280341779 shutdown_connections 2023-11-15T03:11:39.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f6536333d00 1 --2- 172.21.15.121:0/1280341779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582e9141810 0x5582e913d930 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f6536333d00 1 --2- 172.21.15.121:0/1280341779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5582e9141430 0x5582e9112710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f6536333d00 1 --2- 172.21.15.121:0/1280341779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5582e910fdf0 0x5582e9112220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f6536333d00 1 -- 172.21.15.121:0/1280341779 >> 172.21.15.121:0/1280341779 conn(0x5582e91107e0 msgr2=0x5582e91e2140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:39.335 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f6536333d00 1 -- 172.21.15.121:0/1280341779 shutdown_connections 2023-11-15T03:11:39.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.320+0000 7f6536333d00 1 -- 172.21.15.121:0/1280341779 wait complete. 2023-11-15T03:11:39.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.324+0000 7f6536333d00 1 Processor -- start 2023-11-15T03:11:39.336 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.324+0000 7f6536333d00 1 -- start start 2023-11-15T03:11:39.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.324+0000 7f6536333d00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5582e910fdf0 0x5582e91e8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.324+0000 7f6536333d00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5582e9141430 0x5582e91dd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.324+0000 7f6536333d00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582e9141810 0x5582e91dd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.324+0000 7f6536333d00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e91e8b80 con 0x5582e9141430 2023-11-15T03:11:39.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.324+0000 7f6536333d00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e91e5790 con 0x5582e9141810 2023-11-15T03:11:39.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.324+0000 7f6536333d00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582e91e5d50 con 0x5582e910fdf0 2023-11-15T03:11:39.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.324+0000 7f652a23b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5582e9141430 0x5582e91dd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.324+0000 7f652a23b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5582e9141430 0x5582e91dd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51450/0 (socket says 172.21.15.121:51450) 2023-11-15T03:11:39.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.324+0000 7f652a23b700 1 -- 172.21.15.121:0/3523181276 learned_addr learned my addr 172.21.15.121:0/3523181276 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:39.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.324+0000 7f652aa3c700 1 --2- 172.21.15.121:0/3523181276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5582e910fdf0 0x5582e91e8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.328+0000 7f652b23d700 1 --2- 172.21.15.121:0/3523181276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582e9141810 0x5582e91dd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.328+0000 7f652a23b700 1 -- 172.21.15.121:0/3523181276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5582e910fdf0 msgr2=0x5582e91e8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:39.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.328+0000 7f652a23b700 1 --2- 172.21.15.121:0/3523181276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5582e910fdf0 0x5582e91e8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.328+0000 7f652a23b700 1 -- 172.21.15.121:0/3523181276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582e9141810 msgr2=0x5582e91dd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:39.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.328+0000 7f652a23b700 1 --2- 172.21.15.121:0/3523181276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582e9141810 0x5582e91dd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.328+0000 7f652a23b700 1 -- 172.21.15.121:0/3523181276 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582e90b1770 con 0x5582e9141430 2023-11-15T03:11:39.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.328+0000 7f652b23d700 1 --2- 172.21.15.121:0/3523181276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582e9141810 0x5582e91dd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:39.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.328+0000 7f652a23b700 1 --2- 172.21.15.121:0/3523181276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5582e9141430 0x5582e91dd290 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 rx=0x7f6520003e70 tx=0x7f6520003a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:39.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.328+0000 7f651b7fe700 1 -- 172.21.15.121:0/3523181276 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652000bce0 con 0x5582e9141430 2023-11-15T03:11:39.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.328+0000 7f6536333d00 1 -- 172.21.15.121:0/3523181276 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5582e9080a70 con 0x5582e9141430 2023-11-15T03:11:39.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.328+0000 7f651b7fe700 1 -- 172.21.15.121:0/3523181276 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6520008040 con 0x5582e9141430 2023-11-15T03:11:39.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.328+0000 7f651b7fe700 1 -- 172.21.15.121:0/3523181276 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65200116b0 con 0x5582e9141430 2023-11-15T03:11:39.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.328+0000 7f6536333d00 1 -- 172.21.15.121:0/3523181276 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582e908c010 con 0x5582e9141430 2023-11-15T03:11:39.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.332+0000 7f651b7fe700 1 -- 172.21.15.121:0/3523181276 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6520011a70 con 0x5582e9141430 2023-11-15T03:11:39.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.332+0000 7f651b7fe700 1 --2- 172.21.15.121:0/3523181276 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6510059020 0x7f651005b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.332+0000 7f651b7fe700 1 -- 172.21.15.121:0/3523181276 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f65200735f0 con 0x5582e9141430 2023-11-15T03:11:39.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.332+0000 7f652aa3c700 1 --2- 172.21.15.121:0/3523181276 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6510059020 0x7f651005b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.332+0000 7f6536333d00 1 --2- 172.21.15.121:0/3523181276 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x5582e91d7dd0 0x5582e9205980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.332+0000 7f6536333d00 1 -- 172.21.15.121:0/3523181276 --> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [setxattr parent (7) in=13b] snapc 0=[] ondisk+write+known_if_redirected e147) v8 -- 0x5582e91e8500 con 0x5582e91d7dd0 2023-11-15T03:11:39.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.332+0000 7f652b23d700 1 --2- 172.21.15.121:0/3523181276 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x5582e91d7dd0 0x5582e9205980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.332+0000 7f652aa3c700 1 --2- 172.21.15.121:0/3523181276 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6510059020 0x7f651005b4e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7f651c000c00 tx=0x7f651c001790).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:39.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.332+0000 7f652b23d700 1 --2- 172.21.15.121:0/3523181276 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x5582e91d7dd0 0x5582e9205980 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:39.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.340+0000 7f652b23d700 1 -- 172.21.15.121:0/3523181276 <== osd.3 v2:172.21.15.121:6820/33135 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (7)] v147'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f652400d740 con 0x5582e91d7dd0 2023-11-15T03:11:39.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.340+0000 7f6536333d00 1 -- 172.21.15.121:0/3523181276 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x5582e91d7dd0 msgr2=0x5582e9205980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:39.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.340+0000 7f6536333d00 1 --2- 172.21.15.121:0/3523181276 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x5582e91d7dd0 0x5582e9205980 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.340+0000 7f6536333d00 1 -- 172.21.15.121:0/3523181276 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6510059020 msgr2=0x7f651005b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:39.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.340+0000 7f6536333d00 1 --2- 172.21.15.121:0/3523181276 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f6510059020 0x7f651005b4e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7f651c000c00 tx=0x7f651c001790).stop 2023-11-15T03:11:39.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.340+0000 7f6536333d00 1 -- 172.21.15.121:0/3523181276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5582e9141430 msgr2=0x5582e91dd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:39.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.340+0000 7f6536333d00 1 --2- 172.21.15.121:0/3523181276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5582e9141430 0x5582e91dd290 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 rx=0x7f6520003e70 tx=0x7f6520003a50).stop 2023-11-15T03:11:39.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.340+0000 7f652b23d700 1 -- 172.21.15.121:0/3523181276 reap_dead start 2023-11-15T03:11:39.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.340+0000 7f6536333d00 1 -- 172.21.15.121:0/3523181276 shutdown_connections 2023-11-15T03:11:39.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.340+0000 7f6536333d00 1 -- 172.21.15.121:0/3523181276 >> 172.21.15.121:0/3523181276 conn(0x5582e91107e0 msgr2=0x5582e913e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:39.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.340+0000 7f6536333d00 1 -- 172.21.15.121:0/3523181276 shutdown_connections 2023-11-15T03:11:39.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.340+0000 7f6536333d00 1 -- 172.21.15.121:0/3523181276 wait complete. 2023-11-15T03:11:39.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:11:39.355 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink1) 2023-11-15T03:11:39.380 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:11:39.372+0000 7fea4370a700 -1 mds.0.cache corrupt backtrace on ino x00x10000000001: void inode_backtrace_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 5 < 104: Malformed input 2023-11-15T03:11:39.381 INFO:teuthology.orchestra.run.smithi121.stderr:stat: cannot stat 'testdir/hardlink1': Input/output error 2023-11-15T03:11:39.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:11:39.383 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:39.593 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.584+0000 7f856c4db700 1 Processor -- start 2023-11-15T03:11:39.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.584+0000 7f856c4db700 1 -- start start 2023-11-15T03:11:39.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.588+0000 7f856c4db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f85641562f0 0x7f8564155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.588+0000 7f856c4db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85640a57c0 0x7f85640a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.588+0000 7f856c4db700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85640c4c30 con 0x7f8564154ca0 2023-11-15T03:11:39.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.588+0000 7f856c4db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85640a8280 con 0x7f85641562f0 2023-11-15T03:11:39.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.588+0000 7f856c4db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85640a6d40 con 0x7f85640a57c0 2023-11-15T03:11:39.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.588+0000 7f8569a76700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f85641562f0 0x7f8564155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.588+0000 7f856aa78700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85640a57c0 0x7f85640a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.588+0000 7f8569a76700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f85641562f0 0x7f8564155080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51456/0 (socket says 172.21.15.121:51456) 2023-11-15T03:11:39.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.588+0000 7f8569a76700 1 -- 172.21.15.121:0/382284904 learned_addr learned my addr 172.21.15.121:0/382284904 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:39.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.588+0000 7f856a277700 1 --1- 172.21.15.121:0/382284904 >> v1:172.21.15.190:6789/0 conn(0x7f8564154ca0 0x7f85640a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:50076/0 (socket says 172.21.15.121:50076) 2023-11-15T03:11:39.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.588+0000 7f8569a76700 1 -- 172.21.15.121:0/382284904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85640a57c0 msgr2=0x7f85640a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:39.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.588+0000 7f8569a76700 1 --2- 172.21.15.121:0/382284904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85640a57c0 0x7f85640a5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.588+0000 7f8569a76700 1 -- 172.21.15.121:0/382284904 >> v1:172.21.15.190:6789/0 conn(0x7f8564154ca0 legacy=0x7f85640a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:39.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.588+0000 7f8569a76700 1 -- 172.21.15.121:0/382284904 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85640a6140 con 0x7f85641562f0 2023-11-15T03:11:39.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.588+0000 7f8569a76700 1 --2- 172.21.15.121:0/382284904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85641562f0 0x7f8564155080 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 rx=0x7f8554006fd0 tx=0x7f85540090f0).ready entity=mon.0 client_cookie=93f85af9dc5a9d97 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:39.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.588+0000 7f8569275700 1 -- 172.21.15.121:0/382284904 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855400b580 con 0x7f85641562f0 2023-11-15T03:11:39.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.588+0000 7f8569275700 1 -- 172.21.15.121:0/382284904 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f855400b6e0 con 0x7f85641562f0 2023-11-15T03:11:39.601 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.588+0000 7f8569275700 1 -- 172.21.15.121:0/382284904 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855400ba60 con 0x7f85641562f0 2023-11-15T03:11:39.602 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.592+0000 7f856c4db700 1 -- 172.21.15.121:0/382284904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85641562f0 msgr2=0x7f8564155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:39.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.592+0000 7f856c4db700 1 --2- 172.21.15.121:0/382284904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85641562f0 0x7f8564155080 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 rx=0x7f8554006fd0 tx=0x7f85540090f0).stop 2023-11-15T03:11:39.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.592+0000 7f856c4db700 1 -- 172.21.15.121:0/382284904 shutdown_connections 2023-11-15T03:11:39.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.592+0000 7f856c4db700 1 --2- 172.21.15.121:0/382284904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85640a57c0 0x7f85640a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.592+0000 7f856c4db700 1 --2- 172.21.15.121:0/382284904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85641562f0 0x7f8564155080 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.592+0000 7f856c4db700 1 -- 172.21.15.121:0/382284904 >> 172.21.15.121:0/382284904 conn(0x7f8564062190 msgr2=0x7f85640625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:39.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.592+0000 7f856c4db700 1 -- 172.21.15.121:0/382284904 shutdown_connections 2023-11-15T03:11:39.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.592+0000 7f856c4db700 1 -- 172.21.15.121:0/382284904 wait complete. 2023-11-15T03:11:39.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.592+0000 7f856c4db700 1 Processor -- start 2023-11-15T03:11:39.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.592+0000 7f856c4db700 1 -- start start 2023-11-15T03:11:39.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f856c4db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85640a57c0 0x7f856414c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f856c4db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8564154ca0 0x7f85640a9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f856c4db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85641562f0 0x7f85640ab120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f856c4db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85640aac90 con 0x7f85640a57c0 2023-11-15T03:11:39.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f856c4db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856414ce50 con 0x7f8564154ca0 2023-11-15T03:11:39.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f856c4db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85640aa080 con 0x7f85641562f0 2023-11-15T03:11:39.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f856a277700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85640a57c0 0x7f856414c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f856a277700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85640a57c0 0x7f856414c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51472/0 (socket says 172.21.15.121:51472) 2023-11-15T03:11:39.608 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f856a277700 1 -- 172.21.15.121:0/2698891849 learned_addr learned my addr 172.21.15.121:0/2698891849 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:39.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f8569a76700 1 --2- 172.21.15.121:0/2698891849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8564154ca0 0x7f85640a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f856aa78700 1 --2- 172.21.15.121:0/2698891849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85641562f0 0x7f85640ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f856a277700 1 -- 172.21.15.121:0/2698891849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85641562f0 msgr2=0x7f85640ab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:39.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f856a277700 1 --2- 172.21.15.121:0/2698891849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85641562f0 0x7f85640ab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f856a277700 1 -- 172.21.15.121:0/2698891849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8564154ca0 msgr2=0x7f85640a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:39.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f856a277700 1 --2- 172.21.15.121:0/2698891849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8564154ca0 0x7f85640a9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f856a277700 1 -- 172.21.15.121:0/2698891849 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8554009370 con 0x7f85640a57c0 2023-11-15T03:11:39.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f856aa78700 1 --2- 172.21.15.121:0/2698891849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85641562f0 0x7f85640ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:39.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f856a277700 1 --2- 172.21.15.121:0/2698891849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85640a57c0 0x7f856414c960 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 rx=0x7f8560007fd0 tx=0x7f8560007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:39.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f855a7fc700 1 -- 172.21.15.121:0/2698891849 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85600072e0 con 0x7f85640a57c0 2023-11-15T03:11:39.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f855a7fc700 1 -- 172.21.15.121:0/2698891849 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8560007440 con 0x7f85640a57c0 2023-11-15T03:11:39.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f855a7fc700 1 -- 172.21.15.121:0/2698891849 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85600058f0 con 0x7f85640a57c0 2023-11-15T03:11:39.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f856c4db700 1 -- 172.21.15.121:0/2698891849 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85640aa360 con 0x7f85640a57c0 2023-11-15T03:11:39.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.596+0000 7f856c4db700 1 -- 172.21.15.121:0/2698891849 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85640aa920 con 0x7f85640a57c0 2023-11-15T03:11:39.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.600+0000 7f856c4db700 1 -- 172.21.15.121:0/2698891849 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f856406c150 con 0x7f85640a57c0 2023-11-15T03:11:39.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.600+0000 7f855a7fc700 1 -- 172.21.15.121:0/2698891849 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8560005ce0 con 0x7f85640a57c0 2023-11-15T03:11:39.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.600+0000 7f855a7fc700 1 --2- 172.21.15.121:0/2698891849 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f854805a510 0x7f854805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:39.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.600+0000 7f855a7fc700 1 -- 172.21.15.121:0/2698891849 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f8560073fe0 con 0x7f85640a57c0 2023-11-15T03:11:39.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.600+0000 7f8569a76700 1 --2- 172.21.15.121:0/2698891849 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f854805a510 0x7f854805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:39.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.604+0000 7f8569a76700 1 --2- 172.21.15.121:0/2698891849 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f854805a510 0x7f854805c9d0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7f8564156760 tx=0x7f8554017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:39.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.608+0000 7f855a7fc700 1 -- 172.21.15.121:0/2698891849 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f85600476d0 con 0x7f85640a57c0 2023-11-15T03:11:39.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.804+0000 7f856c4db700 1 -- 172.21.15.121:0/2698891849 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85640a6f50 con 0x7f85640a57c0 2023-11-15T03:11:39.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.808+0000 7f855a7fc700 1 -- 172.21.15.121:0/2698891849 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 355 v355) v1 ==== 95+0+4514 (secure 0 0 0) 0x7f856004acf0 con 0x7f85640a57c0 2023-11-15T03:11:39.816 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:39.816 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":355,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/755048854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":755048854},{"type":"v1","addr":"172.21.15.190:6834","nonce":755048854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2360242049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2360242049},{"type":"v1","addr":"172.21.15.190:6835","nonce":2360242049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2780153683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2780153683},{"type":"v1","addr":"172.21.15.121:6835","nonce":2780153683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345}],"filesystems":[{"mdsmap":{"epoch":355,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:11:21.864101+0000","modified":"2023-11-15T03:11:36.256557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10899":{"gid":10899,"name":"a","rank":0,"incarnation":352,"state":"up:active","state_seq":4,"addr":"172.21.15.121:6837/3298117008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3298117008},{"type":"v1","addr":"172.21.15.121:6837","nonce":3298117008}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-15T03:11:39.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.808+0000 7f856c4db700 1 -- 172.21.15.121:0/2698891849 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f854805a510 msgr2=0x7f854805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:39.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.808+0000 7f856c4db700 1 --2- 172.21.15.121:0/2698891849 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f854805a510 0x7f854805c9d0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7f8564156760 tx=0x7f8554017000).stop 2023-11-15T03:11:39.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.808+0000 7f856c4db700 1 -- 172.21.15.121:0/2698891849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85640a57c0 msgr2=0x7f856414c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:39.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.808+0000 7f856c4db700 1 --2- 172.21.15.121:0/2698891849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85640a57c0 0x7f856414c960 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 rx=0x7f8560007fd0 tx=0x7f8560007df0).stop 2023-11-15T03:11:39.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.808+0000 7f856c4db700 1 -- 172.21.15.121:0/2698891849 shutdown_connections 2023-11-15T03:11:39.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.808+0000 7f856c4db700 1 --2- 172.21.15.121:0/2698891849 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f854805a510 0x7f854805c9d0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.808+0000 7f856c4db700 1 --2- 172.21.15.121:0/2698891849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85641562f0 0x7f85640ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.808+0000 7f856c4db700 1 --2- 172.21.15.121:0/2698891849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8564154ca0 0x7f85640a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.808+0000 7f856c4db700 1 --2- 172.21.15.121:0/2698891849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85640a57c0 0x7f856414c960 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:39.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.808+0000 7f856c4db700 1 -- 172.21.15.121:0/2698891849 >> 172.21.15.121:0/2698891849 conn(0x7f8564062190 msgr2=0x7f8564067080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.808+0000 7f856c4db700 1 -- 172.21.15.121:0/2698891849 shutdown_connections 2023-11-15T03:11:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.808+0000 7f856c4db700 1 -- 172.21.15.121:0/2698891849 wait complete. 2023-11-15T03:11:39.822 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 355 2023-11-15T03:11:39.841 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2023-11-15T03:11:40.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.992+0000 7efc6165b700 1 Processor -- start 2023-11-15T03:11:40.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.992+0000 7efc6165b700 1 -- start start 2023-11-15T03:11:40.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.996+0000 7efc6165b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efc5c14abc0 0x7efc5c1540b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.996+0000 7efc6165b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc5c065780 con 0x7efc5c0a4810 2023-11-15T03:11:40.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.996+0000 7efc6165b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc5c065d40 con 0x7efc5c1545a0 2023-11-15T03:11:40.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:39.996+0000 7efc6165b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc5c14afa0 con 0x7efc5c14abc0 2023-11-15T03:11:40.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.000+0000 7efc5affd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efc5c14abc0 0x7efc5c1540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.000+0000 7efc5affd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efc5c14abc0 0x7efc5c1540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:39616/0 (socket says 172.21.15.121:39616) 2023-11-15T03:11:40.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.000+0000 7efc5affd700 1 -- 172.21.15.121:0/436213644 learned_addr learned my addr 172.21.15.121:0/436213644 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:40.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.000+0000 7efc5affd700 1 -- 172.21.15.121:0/436213644 >> v1:172.21.15.190:6789/0 conn(0x7efc5c1545a0 legacy=0x7efc5c0a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.000+0000 7efc5affd700 1 -- 172.21.15.121:0/436213644 >> v1:172.21.15.121:6789/0 conn(0x7efc5c0a4810 legacy=0x7efc5c0661d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.000+0000 7efc59ffb700 1 -- 172.21.15.121:0/436213644 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038092543 0 0) 0x7efc5c065780 con 0x7efc5c0a4810 2023-11-15T03:11:40.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.000+0000 7efc5affd700 1 -- 172.21.15.121:0/436213644 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c154bb0 con 0x7efc5c14abc0 2023-11-15T03:11:40.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.000+0000 7efc5affd700 1 --2- 172.21.15.121:0/436213644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc5c14abc0 0x7efc5c1540b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7efc4c006fd0 tx=0x7efc4c0090f0).ready entity=mon.1 client_cookie=24f8d7e9066b4d93 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.000+0000 7efc59ffb700 1 -- 172.21.15.121:0/436213644 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c00b600 con 0x7efc5c14abc0 2023-11-15T03:11:40.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.000+0000 7efc59ffb700 1 -- 172.21.15.121:0/436213644 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c00b760 con 0x7efc5c14abc0 2023-11-15T03:11:40.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.000+0000 7efc59ffb700 1 -- 172.21.15.121:0/436213644 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c00bae0 con 0x7efc5c14abc0 2023-11-15T03:11:40.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.000+0000 7efc6165b700 1 -- 172.21.15.121:0/436213644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc5c14abc0 msgr2=0x7efc5c1540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.000+0000 7efc6165b700 1 --2- 172.21.15.121:0/436213644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc5c14abc0 0x7efc5c1540b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7efc4c006fd0 tx=0x7efc4c0090f0).stop 2023-11-15T03:11:40.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.000+0000 7efc6165b700 1 -- 172.21.15.121:0/436213644 shutdown_connections 2023-11-15T03:11:40.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.000+0000 7efc6165b700 1 --2- 172.21.15.121:0/436213644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc5c14abc0 0x7efc5c1540b0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.000+0000 7efc6165b700 1 -- 172.21.15.121:0/436213644 >> 172.21.15.121:0/436213644 conn(0x7efc5c068530 msgr2=0x7efc5c0c6560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:40.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.000+0000 7efc6165b700 1 -- 172.21.15.121:0/436213644 shutdown_connections 2023-11-15T03:11:40.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.000+0000 7efc6165b700 1 -- 172.21.15.121:0/436213644 wait complete. 2023-11-15T03:11:40.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.004+0000 7efc6165b700 1 Processor -- start 2023-11-15T03:11:40.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.004+0000 7efc6165b700 1 -- start start 2023-11-15T03:11:40.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc6165b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc5c0a4810 0x7efc5c1675d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc6165b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc5c14abc0 0x7efc5c167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc6165b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc5c1545a0 0x7efc5c152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc6165b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c167fb0 con 0x7efc5c0a4810 2023-11-15T03:11:40.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc6165b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c153730 con 0x7efc5c14abc0 2023-11-15T03:11:40.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc6165b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c153a10 con 0x7efc5c1545a0 2023-11-15T03:11:40.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc5affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc5c0a4810 0x7efc5c1675d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc5affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc5c0a4810 0x7efc5c1675d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51484/0 (socket says 172.21.15.121:51484) 2023-11-15T03:11:40.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc5affd700 1 -- 172.21.15.121:0/997369366 learned_addr learned my addr 172.21.15.121:0/997369366 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:40.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc5b7fe700 1 --2- 172.21.15.121:0/997369366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc5c1545a0 0x7efc5c152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc5a7fc700 1 --2- 172.21.15.121:0/997369366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc5c14abc0 0x7efc5c167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc5affd700 1 -- 172.21.15.121:0/997369366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc5c1545a0 msgr2=0x7efc5c152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc5affd700 1 --2- 172.21.15.121:0/997369366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc5c1545a0 0x7efc5c152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc5affd700 1 -- 172.21.15.121:0/997369366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc5c14abc0 msgr2=0x7efc5c167ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc5affd700 1 --2- 172.21.15.121:0/997369366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc5c14abc0 0x7efc5c167ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc5affd700 1 -- 172.21.15.121:0/997369366 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c009370 con 0x7efc5c0a4810 2023-11-15T03:11:40.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc5b7fe700 1 --2- 172.21.15.121:0/997369366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc5c1545a0 0x7efc5c152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:40.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc5a7fc700 1 --2- 172.21.15.121:0/997369366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc5c14abc0 0x7efc5c167ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:40.020 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc5affd700 1 --2- 172.21.15.121:0/997369366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc5c0a4810 0x7efc5c1675d0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 rx=0x7efc5c0695a0 tx=0x7efc4c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc42ffd700 1 -- 172.21.15.121:0/997369366 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c00b8d0 con 0x7efc5c0a4810 2023-11-15T03:11:40.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc42ffd700 1 -- 172.21.15.121:0/997369366 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c00ba30 con 0x7efc5c0a4810 2023-11-15T03:11:40.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc42ffd700 1 -- 172.21.15.121:0/997369366 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c01c670 con 0x7efc5c0a4810 2023-11-15T03:11:40.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc6165b700 1 -- 172.21.15.121:0/997369366 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5c14b180 con 0x7efc5c0a4810 2023-11-15T03:11:40.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.008+0000 7efc6165b700 1 -- 172.21.15.121:0/997369366 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5c14b740 con 0x7efc5c0a4810 2023-11-15T03:11:40.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.012+0000 7efc42ffd700 1 -- 172.21.15.121:0/997369366 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7efc4c01ca20 con 0x7efc5c0a4810 2023-11-15T03:11:40.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.012+0000 7efc6b9db740 1 Processor -- start 2023-11-15T03:11:40.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc6b9db740 1 -- start start 2023-11-15T03:11:40.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc42ffd700 1 --2- 172.21.15.121:0/997369366 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efc4805a510 0x7efc4805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc6b9db740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 0x1af6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc6b9db740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1afa810 0x1ab2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc6b9db740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1aad110 0x1aad590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc6b9db740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x19ee3f0 con 0x1aad110 2023-11-15T03:11:40.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc6b9db740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a11580 con 0x1afa810 2023-11-15T03:11:40.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc6b9db740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x19dad50 con 0x1af6810 2023-11-15T03:11:40.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc42ffd700 1 -- 172.21.15.121:0/997369366 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6294+0+0 (secure 0 0 0) 0x7efc4c076580 con 0x7efc5c0a4810 2023-11-15T03:11:40.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc5b7fe700 1 --2- 172.21.15.121:0/997369366 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efc4805a510 0x7efc4805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc5a7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1afa810 0x1ab2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc5b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1aad110 0x1aad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc5affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 0x1af6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc5a7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1afa810 0x1ab2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:39634/0 (socket says 172.21.15.121:39634) 2023-11-15T03:11:40.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc5a7fc700 1 -- 172.21.15.121:0/615527040 learned_addr learned my addr 172.21.15.121:0/615527040 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:40.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc5affd700 1 -- 172.21.15.121:0/615527040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1afa810 msgr2=0x1ab2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc5affd700 1 --2- 172.21.15.121:0/615527040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1afa810 0x1ab2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc5affd700 1 -- 172.21.15.121:0/615527040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1aad110 msgr2=0x1aad590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc5affd700 1 --2- 172.21.15.121:0/615527040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1aad110 0x1aad590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc5affd700 1 -- 172.21.15.121:0/615527040 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1a2e830 con 0x1af6810 2023-11-15T03:11:40.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc5affd700 1 --2- 172.21.15.121:0/615527040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 0x1af6bf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7efc4c049e70 tx=0x7efc4c00e000).ready entity=mon.2 client_cookie=69ff7ceae0c811e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc5b7fe700 1 --2- 172.21.15.121:0/997369366 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efc4805a510 0x7efc4805c9d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7efc54001560 tx=0x7efc54007f90).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc3bfff700 1 -- 172.21.15.121:0/615527040 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c051070 con 0x1af6810 2023-11-15T03:11:40.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc3bfff700 1 -- 172.21.15.121:0/615527040 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c049cb0 con 0x1af6810 2023-11-15T03:11:40.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.016+0000 7efc3bfff700 1 -- 172.21.15.121:0/615527040 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c04d6e0 con 0x1af6810 2023-11-15T03:11:40.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.020+0000 7efc6b9db740 1 -- 172.21.15.121:0/615527040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 msgr2=0x1af6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.020+0000 7efc6b9db740 1 --2- 172.21.15.121:0/615527040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 0x1af6bf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7efc4c049e70 tx=0x7efc4c00e000).stop 2023-11-15T03:11:40.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.020+0000 7efc6b9db740 1 -- 172.21.15.121:0/615527040 shutdown_connections 2023-11-15T03:11:40.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.020+0000 7efc6b9db740 1 --2- 172.21.15.121:0/615527040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1aad110 0x1aad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.020+0000 7efc6b9db740 1 --2- 172.21.15.121:0/615527040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1afa810 0x1ab2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.020+0000 7efc6b9db740 1 --2- 172.21.15.121:0/615527040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 0x1af6bf0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.020+0000 7efc6b9db740 1 -- 172.21.15.121:0/615527040 >> 172.21.15.121:0/615527040 conn(0x1893790 msgr2=0x1afaca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:40.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.020+0000 7efc6b9db740 1 -- 172.21.15.121:0/615527040 shutdown_connections 2023-11-15T03:11:40.032 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.020+0000 7efc6b9db740 1 -- 172.21.15.121:0/615527040 wait complete. 2023-11-15T03:11:40.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.020+0000 7efc6b9db740 1 -- start start 2023-11-15T03:11:40.036 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.020+0000 7efc6b9db740 1 Processor -- start 2023-11-15T03:11:40.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.020+0000 7efc6b9db740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1aad110 0x1af2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.020+0000 7efc6b9db740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 0x1adecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.020+0000 7efc6b9db740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1afa810 0x1b06dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.020+0000 7efc6b9db740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1adf610 con 0x1afa810 2023-11-15T03:11:40.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.020+0000 7efc6b9db740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ae0470 con 0x1aad110 2023-11-15T03:11:40.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.020+0000 7efc6b9db740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1af2650 con 0x1af6810 2023-11-15T03:11:40.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.020+0000 7efc6b9db740 10 client.0 ll_register_callbacks cb 0x1a2b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:11:40.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.020+0000 7efc5b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1afa810 0x1b06dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc5b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1afa810 0x1b06dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51500/0 (socket says 172.21.15.121:51500) 2023-11-15T03:11:40.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc5b7fe700 1 -- 172.21.15.121:0/3962801342 learned_addr learned my addr 172.21.15.121:0/3962801342 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:40.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc5a7fc700 1 --2- 172.21.15.121:0/3962801342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 0x1adecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc5affd700 1 --2- 172.21.15.121:0/3962801342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1aad110 0x1af2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc5b7fe700 1 -- 172.21.15.121:0/3962801342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 msgr2=0x1adecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc5b7fe700 1 --2- 172.21.15.121:0/3962801342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 0x1adecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc5b7fe700 1 -- 172.21.15.121:0/3962801342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1aad110 msgr2=0x1af2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc5b7fe700 1 --2- 172.21.15.121:0/3962801342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1aad110 0x1af2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc5b7fe700 1 -- 172.21.15.121:0/3962801342 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5c14b740 con 0x1afa810 2023-11-15T03:11:40.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc5affd700 1 --2- 172.21.15.121:0/3962801342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1aad110 0x1af2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:40.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc5b7fe700 1 --2- 172.21.15.121:0/3962801342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1afa810 0x1b06dc0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 rx=0x7efc54019770 tx=0x7efc54009000).ready entity=mon.0 client_cookie=1af576d166c1ea5d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc397fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:11:40.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc397fa700 1 -- 172.21.15.121:0/3962801342 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc54018e40 con 0x1afa810 2023-11-15T03:11:40.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc397fa700 1 -- 172.21.15.121:0/3962801342 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 6294+0+0 (secure 0 0 0) 0x7efc54023430 con 0x1afa810 2023-11-15T03:11:40.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc6b9db740 1 -- 172.21.15.121:0/3962801342 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1a2b2a0 con 0x1afa810 2023-11-15T03:11:40.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc397fa700 1 -- 172.21.15.121:0/3962801342 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=355) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc54006a20 con 0x1afa810 2023-11-15T03:11:40.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc6b9db740 10 client.10998 fetch_fsmap learned FSMap version 355 2023-11-15T03:11:40.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc6b9db740 1 -- 172.21.15.121:0/3962801342 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=355,monmap=2+}) v3 -- 0x1a29a60 con 0x1afa810 2023-11-15T03:11:40.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc397fa700 1 -- 172.21.15.121:0/3962801342 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1a2b2a0 con 0x1afa810 2023-11-15T03:11:40.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc397fa700 1 -- 172.21.15.121:0/3962801342 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 355) v1 ==== 3100+0+0 (secure 0 0 0) 0x7efc54006b80 con 0x1afa810 2023-11-15T03:11:40.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc6b9db740 10 client.10998 fetch_fsmap finished waiting for FSMap version 355 2023-11-15T03:11:40.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc6b9db740 10 client.10998 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:11:40.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc6b9db740 20 client.10998 populate_metadata read hostname 'smithi121' 2023-11-15T03:11:40.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc6b9db740 1 --2- 172.21.15.121:0/3962801342 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1b0a040 0x1b0a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc6b9db740 4 client.10998 mds_command: new command op to 10899 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:11:40.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc6b9db740 1 -- 172.21.15.121:0/3962801342 --> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1a217f0 con 0x1b0a040 2023-11-15T03:11:40.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc5affd700 1 --2- 172.21.15.121:0/3962801342 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1b0a040 0x1b0a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc5affd700 1 --2- 172.21.15.121:0/3962801342 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1b0a040 0x1b0a420 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=2f5eaf6aa522e82c server_cookie=2b37ab0dfaa04bad in_seq=0 out_seq=0 2023-11-15T03:11:40.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.024+0000 7efc397fa700 10 client.10998 ms_handle_connect on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:40.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.028+0000 7efc397fa700 1 -- 172.21.15.121:0/3962801342 <== mds.0 v2:172.21.15.121:6836/3298117008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x1a217f0 con 0x1b0a040 2023-11-15T03:11:40.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.028+0000 7efc397fa700 10 client.10998 handle_command_reply: tid=0 2023-11-15T03:11:40.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.028+0000 7efc6b9db740 1 client.10998 shutdown 2023-11-15T03:11:40.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.028+0000 7efc6b9db740 1 -- 172.21.15.121:0/3962801342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1afa810 msgr2=0x1b06dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.028+0000 7efc6b9db740 1 --2- 172.21.15.121:0/3962801342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1afa810 0x1b06dc0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 rx=0x7efc54019770 tx=0x7efc54009000).stop 2023-11-15T03:11:40.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.028+0000 7efc6b9db740 1 -- 172.21.15.121:0/3962801342 shutdown_connections 2023-11-15T03:11:40.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.028+0000 7efc6b9db740 1 --2- 172.21.15.121:0/3962801342 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1b0a040 0x1b0a420 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.028+0000 7efc6b9db740 1 --2- 172.21.15.121:0/3962801342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1afa810 0x1b06dc0 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.028+0000 7efc6b9db740 1 --2- 172.21.15.121:0/3962801342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 0x1adecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.028+0000 7efc6b9db740 1 --2- 172.21.15.121:0/3962801342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1aad110 0x1af2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.028+0000 7efc397fa700 0 client.10998 ms_handle_reset on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:40.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.028+0000 7efc6b9db740 1 -- 172.21.15.121:0/3962801342 >> 172.21.15.121:0/3962801342 conn(0x1893790 msgr2=0x1afac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:40.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.028+0000 7efc6b9db740 1 -- 172.21.15.121:0/3962801342 shutdown_connections 2023-11-15T03:11:40.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.028+0000 7efc6b9db740 1 -- 172.21.15.121:0/3962801342 wait complete. 2023-11-15T03:11:40.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.028+0000 7efc6b9db740 20 client.10998 trim_cache size 0 max 16384 2023-11-15T03:11:40.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.044+0000 7efc6b9db740 1 Processor -- start 2023-11-15T03:11:40.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.044+0000 7efc6b9db740 1 -- start start 2023-11-15T03:11:40.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc6b9db740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1aad110 0x1aae670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc6b9db740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 0x1ab1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc6b9db740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1afa810 0x1b11ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc6b9db740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c069400 con 0x1aad110 2023-11-15T03:11:40.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc6b9db740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1af6f60 con 0x1afa810 2023-11-15T03:11:40.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc6b9db740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x19e8560 con 0x1af6810 2023-11-15T03:11:40.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc5affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1aad110 0x1aae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc5affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1aad110 0x1aae670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51510/0 (socket says 172.21.15.121:51510) 2023-11-15T03:11:40.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc5affd700 1 -- 172.21.15.121:0/1132975452 learned_addr learned my addr 172.21.15.121:0/1132975452 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:40.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc5a7fc700 1 --2- 172.21.15.121:0/1132975452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 0x1ab1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc5b7fe700 1 --2- 172.21.15.121:0/1132975452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1afa810 0x1b11ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc5affd700 1 -- 172.21.15.121:0/1132975452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 msgr2=0x1ab1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc5affd700 1 --2- 172.21.15.121:0/1132975452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 0x1ab1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc5affd700 1 -- 172.21.15.121:0/1132975452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1afa810 msgr2=0x1b11ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc5affd700 1 --2- 172.21.15.121:0/1132975452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1afa810 0x1b11ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc5affd700 1 -- 172.21.15.121:0/1132975452 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c0c62b0 con 0x1aad110 2023-11-15T03:11:40.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc5b7fe700 1 --2- 172.21.15.121:0/1132975452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1afa810 0x1b11ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:40.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc5a7fc700 1 --2- 172.21.15.121:0/1132975452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 0x1ab1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:40.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc5affd700 1 --2- 172.21.15.121:0/1132975452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1aad110 0x1aae670 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 rx=0x7efc4c053210 tx=0x7efc4c036280).ready entity=mon.0 client_cookie=fc6eb5161eaeb113 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc40ff9700 1 -- 172.21.15.121:0/1132975452 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c051070 con 0x1aad110 2023-11-15T03:11:40.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc40ff9700 1 -- 172.21.15.121:0/1132975452 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c052040 con 0x1aad110 2023-11-15T03:11:40.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc40ff9700 1 -- 172.21.15.121:0/1132975452 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c04d4b0 con 0x1aad110 2023-11-15T03:11:40.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc6b9db740 1 -- 172.21.15.121:0/1132975452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1aad110 msgr2=0x1aae670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc6b9db740 1 --2- 172.21.15.121:0/1132975452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1aad110 0x1aae670 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 rx=0x7efc4c053210 tx=0x7efc4c036280).stop 2023-11-15T03:11:40.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc6b9db740 1 -- 172.21.15.121:0/1132975452 shutdown_connections 2023-11-15T03:11:40.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc6b9db740 1 --2- 172.21.15.121:0/1132975452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1afa810 0x1b11ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc6b9db740 1 --2- 172.21.15.121:0/1132975452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 0x1ab1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc6b9db740 1 --2- 172.21.15.121:0/1132975452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1aad110 0x1aae670 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc6b9db740 1 -- 172.21.15.121:0/1132975452 >> 172.21.15.121:0/1132975452 conn(0x1893790 msgr2=0x1af1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:40.064 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc6b9db740 1 -- 172.21.15.121:0/1132975452 shutdown_connections 2023-11-15T03:11:40.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.048+0000 7efc6b9db740 1 -- 172.21.15.121:0/1132975452 wait complete. 2023-11-15T03:11:40.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.052+0000 7efc6b9db740 1 -- start start 2023-11-15T03:11:40.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.052+0000 7efc6b9db740 1 Processor -- start 2023-11-15T03:11:40.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.052+0000 7efc6b9db740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1aad110 0x1b654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.065 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.052+0000 7efc6b9db740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 0x1ad81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.052+0000 7efc6b9db740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1afa810 0x1ad86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.052+0000 7efc6b9db740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ad8bc0 con 0x1aad110 2023-11-15T03:11:40.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.052+0000 7efc6b9db740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b66e40 con 0x1afa810 2023-11-15T03:11:40.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.052+0000 7efc6b9db740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b664e0 con 0x1af6810 2023-11-15T03:11:40.066 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.052+0000 7efc6b9db740 10 client.0 ll_register_callbacks cb 0x1a2b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:11:40.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.052+0000 7efc5affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1aad110 0x1b654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.052+0000 7efc5affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1aad110 0x1b654b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51524/0 (socket says 172.21.15.121:51524) 2023-11-15T03:11:40.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.052+0000 7efc5affd700 1 -- 172.21.15.121:0/1914417638 learned_addr learned my addr 172.21.15.121:0/1914417638 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:40.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.052+0000 7efc5b7fe700 1 --2- 172.21.15.121:0/1914417638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1afa810 0x1ad86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.052+0000 7efc5a7fc700 1 --2- 172.21.15.121:0/1914417638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 0x1ad81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.052+0000 7efc5affd700 1 -- 172.21.15.121:0/1914417638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 msgr2=0x1ad81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.052+0000 7efc5affd700 1 --2- 172.21.15.121:0/1914417638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 0x1ad81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.052+0000 7efc5affd700 1 -- 172.21.15.121:0/1914417638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1afa810 msgr2=0x1ad86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.052+0000 7efc5affd700 1 --2- 172.21.15.121:0/1914417638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1afa810 0x1ad86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.052+0000 7efc5affd700 1 -- 172.21.15.121:0/1914417638 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1a2e830 con 0x1aad110 2023-11-15T03:11:40.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc5affd700 1 --2- 172.21.15.121:0/1914417638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1aad110 0x1b654b0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 rx=0x7efc4c009130 tx=0x7efc4c032040).ready entity=mon.0 client_cookie=9e034c1c5ce9035d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc397fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:11:40.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc397fa700 1 -- 172.21.15.121:0/1914417638 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c03c070 con 0x1aad110 2023-11-15T03:11:40.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc397fa700 1 -- 172.21.15.121:0/1914417638 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 6294+0+0 (secure 0 0 0) 0x7efc4c04d910 con 0x1aad110 2023-11-15T03:11:40.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 1 -- 172.21.15.121:0/1914417638 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc4c049cb0 con 0x1aad110 2023-11-15T03:11:40.071 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc397fa700 1 -- 172.21.15.121:0/1914417638 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=355) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc4c051040 con 0x1aad110 2023-11-15T03:11:40.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 10 client.11004 fetch_fsmap learned FSMap version 355 2023-11-15T03:11:40.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 1 -- 172.21.15.121:0/1914417638 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=355,monmap=2+}) v3 -- 0x1a297b0 con 0x1aad110 2023-11-15T03:11:40.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc397fa700 1 -- 172.21.15.121:0/1914417638 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c049cb0 con 0x1aad110 2023-11-15T03:11:40.072 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc397fa700 1 -- 172.21.15.121:0/1914417638 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 355) v1 ==== 3100+0+0 (secure 0 0 0) 0x7efc4c03f020 con 0x1aad110 2023-11-15T03:11:40.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 10 client.11004 fetch_fsmap finished waiting for FSMap version 355 2023-11-15T03:11:40.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 10 client.11004 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:11:40.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 20 client.11004 populate_metadata read hostname 'smithi121' 2023-11-15T03:11:40.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 1 --2- 172.21.15.121:0/1914417638 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1ac8d90 0x1ac9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 4 client.11004 mds_command: new command op to 10899 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-11-15T03:11:40.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 1 -- 172.21.15.121:0/1914417638 --> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x18ea720 con 0x1ac8d90 2023-11-15T03:11:40.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc5a7fc700 1 --2- 172.21.15.121:0/1914417638 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1ac8d90 0x1ac9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc5a7fc700 1 --2- 172.21.15.121:0/1914417638 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1ac8d90 0x1ac9170 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=d0844e157a36f917 server_cookie=117ed7f4772cddc6 in_seq=0 out_seq=0 2023-11-15T03:11:40.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc397fa700 10 client.11004 ms_handle_connect on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:40.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc397fa700 1 -- 172.21.15.121:0/1914417638 <== mds.0 v2:172.21.15.121:6836/3298117008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+145 (crc 0 0 0) 0x18ea720 con 0x1ac8d90 2023-11-15T03:11:40.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc397fa700 10 client.11004 handle_command_reply: tid=0 2023-11-15T03:11:40.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 1 client.11004 shutdown 2023-11-15T03:11:40.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 1 -- 172.21.15.121:0/1914417638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1aad110 msgr2=0x1b654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 1 --2- 172.21.15.121:0/1914417638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1aad110 0x1b654b0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 rx=0x7efc4c009130 tx=0x7efc4c032040).stop 2023-11-15T03:11:40.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 1 -- 172.21.15.121:0/1914417638 shutdown_connections 2023-11-15T03:11:40.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 1 --2- 172.21.15.121:0/1914417638 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1ac8d90 0x1ac9170 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 1 --2- 172.21.15.121:0/1914417638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1afa810 0x1ad86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 1 --2- 172.21.15.121:0/1914417638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1af6810 0x1ad81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 1 --2- 172.21.15.121:0/1914417638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1aad110 0x1b654b0 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc397fa700 0 client.11004 ms_handle_reset on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:40.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 1 -- 172.21.15.121:0/1914417638 >> 172.21.15.121:0/1914417638 conn(0x1893790 msgr2=0x1adccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:40.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 1 -- 172.21.15.121:0/1914417638 shutdown_connections 2023-11-15T03:11:40.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.056+0000 7efc6b9db740 1 -- 172.21.15.121:0/1914417638 wait complete. 2023-11-15T03:11:40.077 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.060+0000 7efc6b9db740 20 client.11004 trim_cache size 0 max 16384 2023-11-15T03:11:40.077 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:40.078 INFO:teuthology.orchestra.run.smithi121.stdout:[ 2023-11-15T03:11:40.078 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:40.078 INFO:teuthology.orchestra.run.smithi121.stdout: "damage_type": "backtrace", 2023-11-15T03:11:40.078 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 2702059831, 2023-11-15T03:11:40.079 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 1099511627777, 2023-11-15T03:11:40.079 INFO:teuthology.orchestra.run.smithi121.stdout: "path": "/testdir/hardlink1" 2023-11-15T03:11:40.079 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:40.079 INFO:teuthology.orchestra.run.smithi121.stdout:] 2023-11-15T03:11:40.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.060+0000 7efc6165b700 1 -- 172.21.15.121:0/997369366 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efc4805a510 msgr2=0x7efc4805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.060+0000 7efc6165b700 1 --2- 172.21.15.121:0/997369366 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efc4805a510 0x7efc4805c9d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7efc54001560 tx=0x7efc54007f90).stop 2023-11-15T03:11:40.080 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.060+0000 7efc6165b700 1 -- 172.21.15.121:0/997369366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc5c0a4810 msgr2=0x7efc5c1675d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.060+0000 7efc6165b700 1 --2- 172.21.15.121:0/997369366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc5c0a4810 0x7efc5c1675d0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 rx=0x7efc5c0695a0 tx=0x7efc4c017000).stop 2023-11-15T03:11:40.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.060+0000 7efc6165b700 1 -- 172.21.15.121:0/997369366 shutdown_connections 2023-11-15T03:11:40.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.060+0000 7efc6165b700 1 --2- 172.21.15.121:0/997369366 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7efc4805a510 0x7efc4805c9d0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.060+0000 7efc6165b700 1 --2- 172.21.15.121:0/997369366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc5c1545a0 0x7efc5c152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.081 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.060+0000 7efc6165b700 1 --2- 172.21.15.121:0/997369366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc5c14abc0 0x7efc5c167ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.060+0000 7efc6165b700 1 --2- 172.21.15.121:0/997369366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc5c0a4810 0x7efc5c1675d0 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.060+0000 7efc6165b700 1 -- 172.21.15.121:0/997369366 >> 172.21.15.121:0/997369366 conn(0x7efc5c068530 msgr2=0x7efc5c0c4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:40.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.060+0000 7efc6165b700 1 -- 172.21.15.121:0/997369366 shutdown_connections 2023-11-15T03:11:40.082 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.060+0000 7efc6165b700 1 -- 172.21.15.121:0/997369366 wait complete. 2023-11-15T03:11:40.089 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:40.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.244+0000 7f998b71b700 1 Processor -- start 2023-11-15T03:11:40.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.244+0000 7f998b71b700 1 -- start start 2023-11-15T03:11:40.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.248+0000 7f998b71b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9984154ca0 0x7f9984155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.248+0000 7f998b71b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99840a57c0 0x7f99840a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.248+0000 7f998b71b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99840c4c30 con 0x7f99841562f0 2023-11-15T03:11:40.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.248+0000 7f998b71b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99840a8280 con 0x7f9984154ca0 2023-11-15T03:11:40.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.248+0000 7f998b71b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99840a6d40 con 0x7f99840a57c0 2023-11-15T03:11:40.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.248+0000 7f99894b7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9984154ca0 0x7f9984155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.248+0000 7f9988cb6700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f99841562f0 0x7f99840a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56272/0 (socket says 172.21.15.121:56272) 2023-11-15T03:11:40.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.248+0000 7f9988cb6700 1 -- 172.21.15.121:0/840465918 learned_addr learned my addr 172.21.15.121:0/840465918 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:40.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.248+0000 7f9989cb8700 1 --2- 172.21.15.121:0/840465918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99840a57c0 0x7f99840a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.248+0000 7f997bfff700 1 -- 172.21.15.121:0/840465918 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983618555 0 0) 0x7f99840c4c30 con 0x7f99841562f0 2023-11-15T03:11:40.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.248+0000 7f997bfff700 1 -- 172.21.15.121:0/840465918 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99700033f0 con 0x7f99841562f0 2023-11-15T03:11:40.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.248+0000 7f99894b7700 1 -- 172.21.15.121:0/840465918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99840a57c0 msgr2=0x7f99840a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.248+0000 7f99894b7700 1 --2- 172.21.15.121:0/840465918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99840a57c0 0x7f99840a5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.248+0000 7f99894b7700 1 -- 172.21.15.121:0/840465918 >> v1:172.21.15.121:6789/0 conn(0x7f99841562f0 legacy=0x7f99840a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.248+0000 7f99894b7700 1 -- 172.21.15.121:0/840465918 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99840a6140 con 0x7f9984154ca0 2023-11-15T03:11:40.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.252+0000 7f99894b7700 1 --2- 172.21.15.121:0/840465918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9984154ca0 0x7f9984155080 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f9980009760 tx=0x7f9980009060).ready entity=mon.1 client_cookie=172105d215867e14 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.252+0000 7f997bfff700 1 -- 172.21.15.121:0/840465918 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998000d4a0 con 0x7f9984154ca0 2023-11-15T03:11:40.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.252+0000 7f997bfff700 1 -- 172.21.15.121:0/840465918 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998000d600 con 0x7f9984154ca0 2023-11-15T03:11:40.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.252+0000 7f997bfff700 1 -- 172.21.15.121:0/840465918 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998000d980 con 0x7f9984154ca0 2023-11-15T03:11:40.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.252+0000 7f998b71b700 1 -- 172.21.15.121:0/840465918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9984154ca0 msgr2=0x7f9984155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.252+0000 7f998b71b700 1 --2- 172.21.15.121:0/840465918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9984154ca0 0x7f9984155080 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f9980009760 tx=0x7f9980009060).stop 2023-11-15T03:11:40.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.252+0000 7f998b71b700 1 -- 172.21.15.121:0/840465918 shutdown_connections 2023-11-15T03:11:40.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.252+0000 7f998b71b700 1 --2- 172.21.15.121:0/840465918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99840a57c0 0x7f99840a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.252+0000 7f998b71b700 1 --2- 172.21.15.121:0/840465918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9984154ca0 0x7f9984155080 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.252+0000 7f998b71b700 1 -- 172.21.15.121:0/840465918 >> 172.21.15.121:0/840465918 conn(0x7f9984062190 msgr2=0x7f99840625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:40.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.252+0000 7f998b71b700 1 -- 172.21.15.121:0/840465918 shutdown_connections 2023-11-15T03:11:40.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.252+0000 7f998b71b700 1 -- 172.21.15.121:0/840465918 wait complete. 2023-11-15T03:11:40.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f998b71b700 1 Processor -- start 2023-11-15T03:11:40.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f998b71b700 1 -- start start 2023-11-15T03:11:40.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f998b71b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99840a57c0 0x7f9984169640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f998b71b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9984154ca0 0x7f998414c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f998b71b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99841562f0 0x7f998414c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f998b71b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998414cb10 con 0x7f99840a57c0 2023-11-15T03:11:40.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f998b71b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9984159910 con 0x7f99841562f0 2023-11-15T03:11:40.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f998b71b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998414f790 con 0x7f9984154ca0 2023-11-15T03:11:40.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f99894b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99840a57c0 0x7f9984169640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f99894b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99840a57c0 0x7f9984169640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51532/0 (socket says 172.21.15.121:51532) 2023-11-15T03:11:40.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f99894b7700 1 -- 172.21.15.121:0/2749711488 learned_addr learned my addr 172.21.15.121:0/2749711488 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:40.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f9989cb8700 1 --2- 172.21.15.121:0/2749711488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99841562f0 0x7f998414c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f9988cb6700 1 --2- 172.21.15.121:0/2749711488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9984154ca0 0x7f998414c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f99894b7700 1 -- 172.21.15.121:0/2749711488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9984154ca0 msgr2=0x7f998414c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f99894b7700 1 --2- 172.21.15.121:0/2749711488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9984154ca0 0x7f998414c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f99894b7700 1 -- 172.21.15.121:0/2749711488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99841562f0 msgr2=0x7f998414c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f99894b7700 1 --2- 172.21.15.121:0/2749711488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99841562f0 0x7f998414c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f99894b7700 1 -- 172.21.15.121:0/2749711488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99800092a0 con 0x7f99840a57c0 2023-11-15T03:11:40.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f9988cb6700 1 --2- 172.21.15.121:0/2749711488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9984154ca0 0x7f998414c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:11:40.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f99894b7700 1 --2- 172.21.15.121:0/2749711488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99840a57c0 0x7f9984169640 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 rx=0x7f9980011fb0 tx=0x7f9980016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f99797fa700 1 -- 172.21.15.121:0/2749711488 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9980000c90 con 0x7f99840a57c0 2023-11-15T03:11:40.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f99797fa700 1 -- 172.21.15.121:0/2749711488 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998000d770 con 0x7f99840a57c0 2023-11-15T03:11:40.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f99797fa700 1 -- 172.21.15.121:0/2749711488 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998001d430 con 0x7f99840a57c0 2023-11-15T03:11:40.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f998b71b700 1 -- 172.21.15.121:0/2749711488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998414ee60 con 0x7f99840a57c0 2023-11-15T03:11:40.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.256+0000 7f998b71b700 1 -- 172.21.15.121:0/2749711488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998414f420 con 0x7f99840a57c0 2023-11-15T03:11:40.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.260+0000 7f998b71b700 1 -- 172.21.15.121:0/2749711488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f998406c150 con 0x7f99840a57c0 2023-11-15T03:11:40.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.260+0000 7f99797fa700 1 -- 172.21.15.121:0/2749711488 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f998001d860 con 0x7f99840a57c0 2023-11-15T03:11:40.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.264+0000 7f99797fa700 1 --2- 172.21.15.121:0/2749711488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f996805a510 0x7f996805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.264+0000 7f99797fa700 1 -- 172.21.15.121:0/2749711488 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f99800760e0 con 0x7f99840a57c0 2023-11-15T03:11:40.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.264+0000 7f9988cb6700 1 --2- 172.21.15.121:0/2749711488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f996805a510 0x7f996805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.264+0000 7f9988cb6700 1 --2- 172.21.15.121:0/2749711488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f996805a510 0x7f996805c9d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f9974005750 tx=0x7f9974009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.268+0000 7f99797fa700 1 -- 172.21.15.121:0/2749711488 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9980049750 con 0x7f99840a57c0 2023-11-15T03:11:40.485 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.476+0000 7f998b71b700 1 -- 172.21.15.121:0/2749711488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99840a6f50 con 0x7f99840a57c0 2023-11-15T03:11:40.486 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.476+0000 7f99797fa700 1 -- 172.21.15.121:0/2749711488 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 355 v355) v1 ==== 95+0+4514 (secure 0 0 0) 0x7f998004cd70 con 0x7f99840a57c0 2023-11-15T03:11:40.486 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:40.487 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":355,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/755048854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":755048854},{"type":"v1","addr":"172.21.15.190:6834","nonce":755048854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2360242049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2360242049},{"type":"v1","addr":"172.21.15.190:6835","nonce":2360242049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2780153683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2780153683},{"type":"v1","addr":"172.21.15.121:6835","nonce":2780153683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345}],"filesystems":[{"mdsmap":{"epoch":355,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:11:21.864101+0000","modified":"2023-11-15T03:11:36.256557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10899":{"gid":10899,"name":"a","rank":0,"incarnation":352,"state":"up:active","state_seq":4,"addr":"172.21.15.121:6837/3298117008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3298117008},{"type":"v1","addr":"172.21.15.121:6837","nonce":3298117008}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-15T03:11:40.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.480+0000 7f998b71b700 1 -- 172.21.15.121:0/2749711488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f996805a510 msgr2=0x7f996805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.488 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.480+0000 7f998b71b700 1 --2- 172.21.15.121:0/2749711488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f996805a510 0x7f996805c9d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f9974005750 tx=0x7f9974009040).stop 2023-11-15T03:11:40.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.480+0000 7f998b71b700 1 -- 172.21.15.121:0/2749711488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99840a57c0 msgr2=0x7f9984169640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.489 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.480+0000 7f998b71b700 1 --2- 172.21.15.121:0/2749711488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99840a57c0 0x7f9984169640 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 rx=0x7f9980011fb0 tx=0x7f9980016000).stop 2023-11-15T03:11:40.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.480+0000 7f998b71b700 1 -- 172.21.15.121:0/2749711488 shutdown_connections 2023-11-15T03:11:40.490 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.480+0000 7f998b71b700 1 --2- 172.21.15.121:0/2749711488 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f996805a510 0x7f996805c9d0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.480+0000 7f998b71b700 1 --2- 172.21.15.121:0/2749711488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99841562f0 0x7f998414c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.480+0000 7f998b71b700 1 --2- 172.21.15.121:0/2749711488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9984154ca0 0x7f998414c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.480+0000 7f998b71b700 1 --2- 172.21.15.121:0/2749711488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99840a57c0 0x7f9984169640 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.491 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.480+0000 7f998b71b700 1 -- 172.21.15.121:0/2749711488 >> 172.21.15.121:0/2749711488 conn(0x7f9984062190 msgr2=0x7f99840c0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:40.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.480+0000 7f998b71b700 1 -- 172.21.15.121:0/2749711488 shutdown_connections 2023-11-15T03:11:40.492 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.480+0000 7f998b71b700 1 -- 172.21.15.121:0/2749711488 wait complete. 2023-11-15T03:11:40.493 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 355 2023-11-15T03:11:40.511 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 2702059831 2023-11-15T03:11:40.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.668+0000 7f4dfb3a9700 1 Processor -- start 2023-11-15T03:11:40.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.668+0000 7f4dfb3a9700 1 -- start start 2023-11-15T03:11:40.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.672+0000 7f4dfb3a9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4df4069180 0x7f4df4155460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.672+0000 7f4dfb3a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df4154370 0x7f4df4154820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.680 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.672+0000 7f4dfb3a9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df40639f0 con 0x7f4df40c60b0 2023-11-15T03:11:40.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.672+0000 7f4dfb3a9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df414e840 con 0x7f4df4069180 2023-11-15T03:11:40.681 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.672+0000 7f4dfb3a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df41550c0 con 0x7f4df4154370 2023-11-15T03:11:40.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.672+0000 7f4df8944700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4df4069180 0x7f4df4155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.672+0000 7f4df9946700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df4154370 0x7f4df4154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.682 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.672+0000 7f4df8944700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4df4069180 0x7f4df4155460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:39690/0 (socket says 172.21.15.121:39690) 2023-11-15T03:11:40.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.672+0000 7f4df8944700 1 -- 172.21.15.121:0/2270745171 learned_addr learned my addr 172.21.15.121:0/2270745171 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:40.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.672+0000 7f4df9145700 1 --1- 172.21.15.121:0/2270745171 >> v1:172.21.15.190:6789/0 conn(0x7f4df40c60b0 0x7f4df4068a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:50100/0 (socket says 172.21.15.121:50100) 2023-11-15T03:11:40.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.672+0000 7f4df8944700 1 -- 172.21.15.121:0/2270745171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df4154370 msgr2=0x7f4df4154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.683 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.672+0000 7f4df8944700 1 --2- 172.21.15.121:0/2270745171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df4154370 0x7f4df4154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.672+0000 7f4df8944700 1 -- 172.21.15.121:0/2270745171 >> v1:172.21.15.190:6789/0 conn(0x7f4df40c60b0 legacy=0x7f4df4068a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.672+0000 7f4df8944700 1 -- 172.21.15.121:0/2270745171 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df40633a0 con 0x7f4df4069180 2023-11-15T03:11:40.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.672+0000 7f4df9946700 1 --2- 172.21.15.121:0/2270745171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df4154370 0x7f4df4154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:40.684 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.672+0000 7f4df8944700 1 --2- 172.21.15.121:0/2270745171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4df4069180 0x7f4df4155460 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f4de4006fd0 tx=0x7f4de40090f0).ready entity=mon.1 client_cookie=563659f9bbc24f4c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.676+0000 7f4debfff700 1 -- 172.21.15.121:0/2270745171 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de400f510 con 0x7f4df4069180 2023-11-15T03:11:40.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.676+0000 7f4debfff700 1 -- 172.21.15.121:0/2270745171 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de400f670 con 0x7f4df4069180 2023-11-15T03:11:40.686 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.676+0000 7f4debfff700 1 -- 172.21.15.121:0/2270745171 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de400fa10 con 0x7f4df4069180 2023-11-15T03:11:40.687 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.676+0000 7f4dfb3a9700 1 -- 172.21.15.121:0/2270745171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4df4069180 msgr2=0x7f4df4155460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.676+0000 7f4dfb3a9700 1 --2- 172.21.15.121:0/2270745171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4df4069180 0x7f4df4155460 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f4de4006fd0 tx=0x7f4de40090f0).stop 2023-11-15T03:11:40.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.676+0000 7f4dfb3a9700 1 -- 172.21.15.121:0/2270745171 shutdown_connections 2023-11-15T03:11:40.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.676+0000 7f4dfb3a9700 1 --2- 172.21.15.121:0/2270745171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df4154370 0x7f4df4154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.676+0000 7f4dfb3a9700 1 --2- 172.21.15.121:0/2270745171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4df4069180 0x7f4df4155460 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.688 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.676+0000 7f4dfb3a9700 1 -- 172.21.15.121:0/2270745171 >> 172.21.15.121:0/2270745171 conn(0x7f4df40c4580 msgr2=0x7f4df4066180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:40.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.676+0000 7f4dfb3a9700 1 -- 172.21.15.121:0/2270745171 shutdown_connections 2023-11-15T03:11:40.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.676+0000 7f4dfb3a9700 1 -- 172.21.15.121:0/2270745171 wait complete. 2023-11-15T03:11:40.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.676+0000 7f4dfb3a9700 1 Processor -- start 2023-11-15T03:11:40.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.676+0000 7f4dfb3a9700 1 -- start start 2023-11-15T03:11:40.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4dfb3a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4df4069180 0x7f4df414e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4dfb3a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4df40c60b0 0x7f4df41672f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4dfb3a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df4154370 0x7f4df41683d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4dfb3a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df41677e0 con 0x7f4df4069180 2023-11-15T03:11:40.692 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4dfb3a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df414ef20 con 0x7f4df40c60b0 2023-11-15T03:11:40.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4dfb3a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df414f200 con 0x7f4df4154370 2023-11-15T03:11:40.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4df9145700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4df4069180 0x7f4df414e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.693 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4df9145700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4df4069180 0x7f4df414e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51542/0 (socket says 172.21.15.121:51542) 2023-11-15T03:11:40.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4df9145700 1 -- 172.21.15.121:0/2720933877 learned_addr learned my addr 172.21.15.121:0/2720933877 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:40.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4df8944700 1 --2- 172.21.15.121:0/2720933877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4df40c60b0 0x7f4df41672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4df9946700 1 --2- 172.21.15.121:0/2720933877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df4154370 0x7f4df41683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.694 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4df9145700 1 -- 172.21.15.121:0/2720933877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df4154370 msgr2=0x7f4df41683d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4df9145700 1 --2- 172.21.15.121:0/2720933877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df4154370 0x7f4df41683d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4df9145700 1 -- 172.21.15.121:0/2720933877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4df40c60b0 msgr2=0x7f4df41672f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4df9145700 1 --2- 172.21.15.121:0/2720933877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4df40c60b0 0x7f4df41672f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4df9145700 1 -- 172.21.15.121:0/2720933877 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec009a20 con 0x7f4df4069180 2023-11-15T03:11:40.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4df9145700 1 --2- 172.21.15.121:0/2720933877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4df4069180 0x7f4df414e980 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 rx=0x7f4df000c5e0 tx=0x7f4df0007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4de97fa700 1 -- 172.21.15.121:0/2720933877 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df00072e0 con 0x7f4df4069180 2023-11-15T03:11:40.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4de97fa700 1 -- 172.21.15.121:0/2720933877 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df0007440 con 0x7f4df4069180 2023-11-15T03:11:40.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4de97fa700 1 -- 172.21.15.121:0/2720933877 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df00058f0 con 0x7f4df4069180 2023-11-15T03:11:40.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4dfb3a9700 1 -- 172.21.15.121:0/2720933877 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de4009370 con 0x7f4df4069180 2023-11-15T03:11:40.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.680+0000 7f4dfb3a9700 1 -- 172.21.15.121:0/2720933877 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df414f7c0 con 0x7f4df4069180 2023-11-15T03:11:40.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.684+0000 7f4de97fa700 1 -- 172.21.15.121:0/2720933877 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4df0005ce0 con 0x7f4df4069180 2023-11-15T03:11:40.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.684+0000 7f4de97fa700 1 --2- 172.21.15.121:0/2720933877 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4dd805a510 0x7f4dd805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.684+0000 7f4de97fa700 1 -- 172.21.15.121:0/2720933877 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f4df0074150 con 0x7f4df4069180 2023-11-15T03:11:40.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.684+0000 7f4df8944700 1 --2- 172.21.15.121:0/2720933877 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4dd805a510 0x7f4dd805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.688+0000 7f4df8944700 1 --2- 172.21.15.121:0/2720933877 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4dd805a510 0x7f4dd805c9d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7f4df4069580 tx=0x7f4de4017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.688+0000 7f4e05754740 1 Processor -- start 2023-11-15T03:11:40.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.688+0000 7f4e05754740 1 -- start start 2023-11-15T03:11:40.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.688+0000 7f4e05754740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xcae650 0xcaea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.688+0000 7f4e05754740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xcaf050 0xcb4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.688+0000 7f4e05754740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd009d0 0xd00e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.688+0000 7f4e05754740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0xc12570 con 0xcaf050 2023-11-15T03:11:40.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.688+0000 7f4e05754740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0xbd5f20 con 0xd009d0 2023-11-15T03:11:40.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.688+0000 7f4e05754740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0xbdbdb0 con 0xcae650 2023-11-15T03:11:40.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.688+0000 7f4df8944700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xcaf050 0xcb4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.688+0000 7f4df8944700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xcaf050 0xcb4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51546/0 (socket says 172.21.15.121:51546) 2023-11-15T03:11:40.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.688+0000 7f4df8944700 1 -- 172.21.15.121:0/1193585299 learned_addr learned my addr 172.21.15.121:0/1193585299 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:40.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.688+0000 7f4df9145700 1 --2- 172.21.15.121:0/1193585299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xcae650 0xcaea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.688+0000 7f4df9946700 1 --2- 172.21.15.121:0/1193585299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd009d0 0xd00e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4df8944700 1 -- 172.21.15.121:0/1193585299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xcae650 msgr2=0xcaea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4df8944700 1 --2- 172.21.15.121:0/1193585299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xcae650 0xcaea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4df8944700 1 -- 172.21.15.121:0/1193585299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd009d0 msgr2=0xd00e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4df8944700 1 --2- 172.21.15.121:0/1193585299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd009d0 0xd00e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4df8944700 1 -- 172.21.15.121:0/1193585299 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xc2f850 con 0xcaf050 2023-11-15T03:11:40.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4df9946700 1 --2- 172.21.15.121:0/1193585299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd009d0 0xd00e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:40.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4df8944700 1 --2- 172.21.15.121:0/1193585299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xcaf050 0xcb4370 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 rx=0x7f4de4009ea0 tx=0x7f4de401f040).ready entity=mon.0 client_cookie=8e6b142c7648f7fe server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.706 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4dde7fc700 1 -- 172.21.15.121:0/1193585299 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de402c070 con 0xcaf050 2023-11-15T03:11:40.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4dde7fc700 1 -- 172.21.15.121:0/1193585299 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de401d680 con 0xcaf050 2023-11-15T03:11:40.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4dde7fc700 1 -- 172.21.15.121:0/1193585299 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de400f510 con 0xcaf050 2023-11-15T03:11:40.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4e05754740 1 -- 172.21.15.121:0/1193585299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xcaf050 msgr2=0xcb4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4e05754740 1 --2- 172.21.15.121:0/1193585299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xcaf050 0xcb4370 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 rx=0x7f4de4009ea0 tx=0x7f4de401f040).stop 2023-11-15T03:11:40.707 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4e05754740 1 -- 172.21.15.121:0/1193585299 shutdown_connections 2023-11-15T03:11:40.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4e05754740 1 --2- 172.21.15.121:0/1193585299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xd009d0 0xd00e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4e05754740 1 --2- 172.21.15.121:0/1193585299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xcaf050 0xcb4370 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4e05754740 1 --2- 172.21.15.121:0/1193585299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xcae650 0xcaea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4e05754740 1 -- 172.21.15.121:0/1193585299 >> 172.21.15.121:0/1193585299 conn(0xa94710 msgr2=0xcb2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:40.708 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4e05754740 1 -- 172.21.15.121:0/1193585299 shutdown_connections 2023-11-15T03:11:40.709 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4e05754740 1 -- 172.21.15.121:0/1193585299 wait complete. 2023-11-15T03:11:40.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.692+0000 7f4e05754740 1 -- start start 2023-11-15T03:11:40.713 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4e05754740 1 Processor -- start 2023-11-15T03:11:40.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4e05754740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xcae650 0xcf4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4e05754740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xcaf050 0xcd7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4e05754740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd009d0 0xcf5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4e05754740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0xcdd120 con 0xd009d0 2023-11-15T03:11:40.714 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4e05754740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0xcd7850 con 0xcae650 2023-11-15T03:11:40.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4e05754740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0xcf2940 con 0xcaf050 2023-11-15T03:11:40.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4e05754740 10 client.0 ll_register_callbacks cb 0xc2c6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:11:40.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4df9946700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd009d0 0xcf5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4df9946700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd009d0 0xcf5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51560/0 (socket says 172.21.15.121:51560) 2023-11-15T03:11:40.715 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4df9946700 1 -- 172.21.15.121:0/1851338887 learned_addr learned my addr 172.21.15.121:0/1851338887 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:40.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4df8944700 1 --2- 172.21.15.121:0/1851338887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xcaf050 0xcd7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4df9145700 1 --2- 172.21.15.121:0/1851338887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xcae650 0xcf4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4df9946700 1 -- 172.21.15.121:0/1851338887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xcaf050 msgr2=0xcd7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4df9946700 1 --2- 172.21.15.121:0/1851338887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xcaf050 0xcd7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.716 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4df9946700 1 -- 172.21.15.121:0/1851338887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xcae650 msgr2=0xcf4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4df9946700 1 --2- 172.21.15.121:0/1851338887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xcae650 0xcf4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4df9946700 1 -- 172.21.15.121:0/1851338887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df40633a0 con 0xd009d0 2023-11-15T03:11:40.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4df9145700 1 --2- 172.21.15.121:0/1851338887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xcae650 0xcf4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:40.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4df9946700 1 --2- 172.21.15.121:0/1851338887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd009d0 0xcf5c40 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 rx=0x7f4dec013610 tx=0x7f4dec00b040).ready entity=mon.0 client_cookie=4d72e445ae06e2a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.717 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4dd37fe700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:11:40.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4dd37fe700 1 -- 172.21.15.121:0/1851338887 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec00e040 con 0xd009d0 2023-11-15T03:11:40.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4dd37fe700 1 -- 172.21.15.121:0/1851338887 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f4dec007440 con 0xd009d0 2023-11-15T03:11:40.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4e05754740 1 -- 172.21.15.121:0/1851338887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xc2c2c0 con 0xd009d0 2023-11-15T03:11:40.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4dd37fe700 1 -- 172.21.15.121:0/1851338887 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=355) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dec016430 con 0xd009d0 2023-11-15T03:11:40.718 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4e05754740 10 client.11019 fetch_fsmap learned FSMap version 355 2023-11-15T03:11:40.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.696+0000 7f4e05754740 1 -- 172.21.15.121:0/1851338887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=355,monmap=2+}) v3 -- 0xc2aa80 con 0xd009d0 2023-11-15T03:11:40.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4dd37fe700 1 -- 172.21.15.121:0/1851338887 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xc2c2c0 con 0xd009d0 2023-11-15T03:11:40.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4dd37fe700 1 -- 172.21.15.121:0/1851338887 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 355) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f4dec016590 con 0xd009d0 2023-11-15T03:11:40.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4e05754740 10 client.11019 fetch_fsmap finished waiting for FSMap version 355 2023-11-15T03:11:40.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4e05754740 10 client.11019 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:11:40.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4e05754740 20 client.11019 populate_metadata read hostname 'smithi121' 2023-11-15T03:11:40.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4e05754740 1 --2- 172.21.15.121:0/1851338887 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0xcd8cb0 0xcdbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4e05754740 4 client.11019 mds_command: new command op to 10899 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:11:40.722 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4e05754740 1 -- 172.21.15.121:0/1851338887 --> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xc22810 con 0xcd8cb0 2023-11-15T03:11:40.723 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4df9145700 1 --2- 172.21.15.121:0/1851338887 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0xcd8cb0 0xcdbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4dd37fe700 10 client.11019 ms_handle_connect on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:40.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4df9145700 1 --2- 172.21.15.121:0/1851338887 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0xcd8cb0 0xcdbde0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=44cbb7a9a1e020a9 server_cookie=5af8e79aaa78a7b8 in_seq=0 out_seq=0 2023-11-15T03:11:40.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4dd37fe700 1 -- 172.21.15.121:0/1851338887 <== mds.0 v2:172.21.15.121:6836/3298117008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0xc22810 con 0xcd8cb0 2023-11-15T03:11:40.724 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4dd37fe700 10 client.11019 handle_command_reply: tid=0 2023-11-15T03:11:40.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4e05754740 1 client.11019 shutdown 2023-11-15T03:11:40.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4e05754740 1 -- 172.21.15.121:0/1851338887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd009d0 msgr2=0xcf5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4e05754740 1 --2- 172.21.15.121:0/1851338887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd009d0 0xcf5c40 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 rx=0x7f4dec013610 tx=0x7f4dec00b040).stop 2023-11-15T03:11:40.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4e05754740 1 -- 172.21.15.121:0/1851338887 shutdown_connections 2023-11-15T03:11:40.725 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4e05754740 1 --2- 172.21.15.121:0/1851338887 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0xcd8cb0 0xcdbde0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4e05754740 1 --2- 172.21.15.121:0/1851338887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd009d0 0xcf5c40 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4e05754740 1 --2- 172.21.15.121:0/1851338887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xcaf050 0xcd7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4e05754740 1 --2- 172.21.15.121:0/1851338887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xcae650 0xcf4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4dd37fe700 0 client.11019 ms_handle_reset on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:40.726 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4e05754740 1 -- 172.21.15.121:0/1851338887 >> 172.21.15.121:0/1851338887 conn(0xa94710 msgr2=0xcaf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:40.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4e05754740 1 -- 172.21.15.121:0/1851338887 shutdown_connections 2023-11-15T03:11:40.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4e05754740 1 -- 172.21.15.121:0/1851338887 wait complete. 2023-11-15T03:11:40.727 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.700+0000 7f4e05754740 20 client.11019 trim_cache size 0 max 16384 2023-11-15T03:11:40.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.716+0000 7f4e05754740 1 Processor -- start 2023-11-15T03:11:40.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.716+0000 7f4e05754740 1 -- start start 2023-11-15T03:11:40.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4e05754740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xcae650 0xcdb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.729 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4e05754740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xcaf050 0xcaaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4e05754740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd009d0 0xcaf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4e05754740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0xbe9b80 con 0xcae650 2023-11-15T03:11:40.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4e05754740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0xcd6da0 con 0xcaf050 2023-11-15T03:11:40.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4e05754740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0xc78190 con 0xd009d0 2023-11-15T03:11:40.730 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4df9145700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xcae650 0xcdb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4df8944700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xcaf050 0xcaaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4df8944700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xcaf050 0xcaaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:39734/0 (socket says 172.21.15.121:39734) 2023-11-15T03:11:40.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4df8944700 1 -- 172.21.15.121:0/1487387054 learned_addr learned my addr 172.21.15.121:0/1487387054 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:40.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4df9145700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xcae650 0xcdb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51562/0 (socket says 172.21.15.121:51562) 2023-11-15T03:11:40.731 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4df9946700 1 --2- 172.21.15.121:0/1487387054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd009d0 0xcaf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4df8944700 1 -- 172.21.15.121:0/1487387054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd009d0 msgr2=0xcaf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4df8944700 1 --2- 172.21.15.121:0/1487387054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd009d0 0xcaf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4df8944700 1 -- 172.21.15.121:0/1487387054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xcae650 msgr2=0xcdb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4df8944700 1 --2- 172.21.15.121:0/1487387054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xcae650 0xcdb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.732 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4df8944700 1 -- 172.21.15.121:0/1487387054 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df4156f60 con 0xcaf050 2023-11-15T03:11:40.733 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4df8944700 1 --2- 172.21.15.121:0/1487387054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xcaf050 0xcaaeb0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f4de400a010 tx=0x7f4de4028000).ready entity=mon.1 client_cookie=99099dfa28a1426f server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4ddeffd700 1 -- 172.21.15.121:0/1487387054 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de402c040 con 0xcaf050 2023-11-15T03:11:40.736 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4ddeffd700 1 -- 172.21.15.121:0/1487387054 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de401d7f0 con 0xcaf050 2023-11-15T03:11:40.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4ddeffd700 1 -- 172.21.15.121:0/1487387054 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de401dc10 con 0xcaf050 2023-11-15T03:11:40.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4e05754740 1 -- 172.21.15.121:0/1487387054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xcaf050 msgr2=0xcaaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4e05754740 1 --2- 172.21.15.121:0/1487387054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xcaf050 0xcaaeb0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f4de400a010 tx=0x7f4de4028000).stop 2023-11-15T03:11:40.737 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4e05754740 1 -- 172.21.15.121:0/1487387054 shutdown_connections 2023-11-15T03:11:40.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4e05754740 1 --2- 172.21.15.121:0/1487387054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xd009d0 0xcaf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4e05754740 1 --2- 172.21.15.121:0/1487387054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xcaf050 0xcaaeb0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4e05754740 1 --2- 172.21.15.121:0/1487387054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xcae650 0xcdb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4e05754740 1 -- 172.21.15.121:0/1487387054 >> 172.21.15.121:0/1487387054 conn(0xa94710 msgr2=0xcf1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:40.738 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4e05754740 1 -- 172.21.15.121:0/1487387054 shutdown_connections 2023-11-15T03:11:40.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.720+0000 7f4e05754740 1 -- 172.21.15.121:0/1487387054 wait complete. 2023-11-15T03:11:40.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4e05754740 1 -- start start 2023-11-15T03:11:40.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4e05754740 1 Processor -- start 2023-11-15T03:11:40.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4e05754740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xcae650 0xd10c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.739 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4e05754740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xcaf050 0xd653d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4e05754740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd009d0 0xcf3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4e05754740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0xcb4c00 con 0xd009d0 2023-11-15T03:11:40.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4e05754740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0xd658c0 con 0xcaf050 2023-11-15T03:11:40.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4e05754740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0xcfe780 con 0xcae650 2023-11-15T03:11:40.740 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4e05754740 10 client.0 ll_register_callbacks cb 0x7f4dec007a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:11:40.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4df9946700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd009d0 0xcf3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4df9946700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd009d0 0xcf3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51564/0 (socket says 172.21.15.121:51564) 2023-11-15T03:11:40.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4df9946700 1 -- 172.21.15.121:0/1693318434 learned_addr learned my addr 172.21.15.121:0/1693318434 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:40.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4df8944700 1 --2- 172.21.15.121:0/1693318434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xcaf050 0xd653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.741 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4df9145700 1 --2- 172.21.15.121:0/1693318434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xcae650 0xd10c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4df9946700 1 -- 172.21.15.121:0/1693318434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xcae650 msgr2=0xd10c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4df9946700 1 --2- 172.21.15.121:0/1693318434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xcae650 0xd10c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4df9946700 1 -- 172.21.15.121:0/1693318434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xcaf050 msgr2=0xd653d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4df9946700 1 --2- 172.21.15.121:0/1693318434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xcaf050 0xd653d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.742 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4df9946700 1 -- 172.21.15.121:0/1693318434 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xc2f850 con 0xd009d0 2023-11-15T03:11:40.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4df9145700 1 --2- 172.21.15.121:0/1693318434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xcae650 0xd10c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:40.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4df9946700 1 --2- 172.21.15.121:0/1693318434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd009d0 0xcf3650 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 rx=0x7f4dec013ad0 tx=0x7f4dec023000).ready entity=mon.0 client_cookie=4675b54a99ccd5e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.743 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4dd37fe700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:11:40.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4dd37fe700 1 -- 172.21.15.121:0/1693318434 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec00e040 con 0xd009d0 2023-11-15T03:11:40.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4e05754740 1 -- 172.21.15.121:0/1693318434 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4de401d680 con 0xd009d0 2023-11-15T03:11:40.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.724+0000 7f4dd37fe700 1 -- 172.21.15.121:0/1693318434 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f4dec007200 con 0xd009d0 2023-11-15T03:11:40.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4dd37fe700 1 -- 172.21.15.121:0/1693318434 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=355) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dec01f440 con 0xd009d0 2023-11-15T03:11:40.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 10 client.11025 fetch_fsmap learned FSMap version 355 2023-11-15T03:11:40.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 1 -- 172.21.15.121:0/1693318434 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=355,monmap=2+}) v3 -- 0xc2a7d0 con 0xd009d0 2023-11-15T03:11:40.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4dd37fe700 1 -- 172.21.15.121:0/1693318434 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de401d680 con 0xd009d0 2023-11-15T03:11:40.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4dd37fe700 1 -- 172.21.15.121:0/1693318434 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 355) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f4dec01f5a0 con 0xd009d0 2023-11-15T03:11:40.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 10 client.11025 fetch_fsmap finished waiting for FSMap version 355 2023-11-15T03:11:40.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 10 client.11025 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:11:40.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 20 client.11025 populate_metadata read hostname 'smithi121' 2023-11-15T03:11:40.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 1 --2- 172.21.15.121:0/1693318434 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0xcd6100 0xcd64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 4 client.11025 mds_command: new command op to 10899 tid=0[{"prefix": "damage rm", "damage_id": 2702059831}] 2023-11-15T03:11:40.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 1 -- 172.21.15.121:0/1693318434 --> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] -- command(tid 0: {"prefix": "damage rm", "damage_id": 2702059831}) v1 -- 0xaeb6a0 con 0xcd6100 2023-11-15T03:11:40.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4df9145700 1 --2- 172.21.15.121:0/1693318434 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0xcd6100 0xcd64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4dd37fe700 10 client.11025 ms_handle_connect on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:40.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4df9145700 1 --2- 172.21.15.121:0/1693318434 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0xcd6100 0xcd64e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=c3b2bcdf88545055 server_cookie=e92087e708f80aa in_seq=0 out_seq=0 2023-11-15T03:11:40.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4dd37fe700 1 -- 172.21.15.121:0/1693318434 <== mds.0 v2:172.21.15.121:6836/3298117008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0xaeb6a0 con 0xcd6100 2023-11-15T03:11:40.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4dd37fe700 10 client.11025 handle_command_reply: tid=0 2023-11-15T03:11:40.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 1 client.11025 shutdown 2023-11-15T03:11:40.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 1 -- 172.21.15.121:0/1693318434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd009d0 msgr2=0xcf3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 1 --2- 172.21.15.121:0/1693318434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd009d0 0xcf3650 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 rx=0x7f4dec013ad0 tx=0x7f4dec023000).stop 2023-11-15T03:11:40.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 1 -- 172.21.15.121:0/1693318434 shutdown_connections 2023-11-15T03:11:40.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 1 --2- 172.21.15.121:0/1693318434 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0xcd6100 0xcd64e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 1 --2- 172.21.15.121:0/1693318434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0xd009d0 0xcf3650 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 1 --2- 172.21.15.121:0/1693318434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0xcaf050 0xd653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 1 --2- 172.21.15.121:0/1693318434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0xcae650 0xd10c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 1 -- 172.21.15.121:0/1693318434 >> 172.21.15.121:0/1693318434 conn(0xa94710 msgr2=0xcffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:40.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4dd37fe700 0 client.11025 ms_handle_reset on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:40.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 1 -- 172.21.15.121:0/1693318434 shutdown_connections 2023-11-15T03:11:40.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 1 -- 172.21.15.121:0/1693318434 wait complete. 2023-11-15T03:11:40.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.728+0000 7f4e05754740 20 client.11025 trim_cache size 0 max 16384 2023-11-15T03:11:40.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.732+0000 7f4dfb3a9700 1 -- 172.21.15.121:0/2720933877 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4dd805a510 msgr2=0x7f4dd805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.732+0000 7f4dfb3a9700 1 --2- 172.21.15.121:0/2720933877 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4dd805a510 0x7f4dd805c9d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7f4df4069580 tx=0x7f4de4017000).stop 2023-11-15T03:11:40.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.732+0000 7f4dfb3a9700 1 -- 172.21.15.121:0/2720933877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4df4069180 msgr2=0x7f4df414e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.732+0000 7f4dfb3a9700 1 --2- 172.21.15.121:0/2720933877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4df4069180 0x7f4df414e980 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 rx=0x7f4df000c5e0 tx=0x7f4df0007db0).stop 2023-11-15T03:11:40.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.732+0000 7f4dfb3a9700 1 -- 172.21.15.121:0/2720933877 shutdown_connections 2023-11-15T03:11:40.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.732+0000 7f4dfb3a9700 1 --2- 172.21.15.121:0/2720933877 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4dd805a510 0x7f4dd805c9d0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.732+0000 7f4dfb3a9700 1 --2- 172.21.15.121:0/2720933877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df4154370 0x7f4df41683d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.732+0000 7f4dfb3a9700 1 --2- 172.21.15.121:0/2720933877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4df40c60b0 0x7f4df41672f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.732+0000 7f4dfb3a9700 1 --2- 172.21.15.121:0/2720933877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4df4069180 0x7f4df414e980 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.732+0000 7f4dfb3a9700 1 -- 172.21.15.121:0/2720933877 >> 172.21.15.121:0/2720933877 conn(0x7f4df40c4580 msgr2=0x7f4df414c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:40.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.732+0000 7f4dfb3a9700 1 -- 172.21.15.121:0/2720933877 shutdown_connections 2023-11-15T03:11:40.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.732+0000 7f4dfb3a9700 1 -- 172.21.15.121:0/2720933877 wait complete. 2023-11-15T03:11:40.761 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100000001f6.00000000 2023-11-15T03:11:40.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.812+0000 7f026d53bd00 1 Processor -- start 2023-11-15T03:11:40.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.812+0000 7f026d53bd00 1 -- start start 2023-11-15T03:11:40.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.812+0000 7f026d53bd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5033e4dd0 0x55a5034bc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.812+0000 7f026d53bd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a50342eb00 0x55a5034bfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.812+0000 7f026d53bd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5034bd670 0x55a50342cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.812+0000 7f026d53bd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a503392cc0 con 0x55a5034bd670 2023-11-15T03:11:40.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.812+0000 7f026d53bd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a50335d220 con 0x55a50342eb00 2023-11-15T03:11:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.812+0000 7f026d53bd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a503417a10 con 0x55a5033e4dd0 2023-11-15T03:11:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.812+0000 7f0262246700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5034bd670 0x55a50342cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.812+0000 7f0262246700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5034bd670 0x55a50342cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51570/0 (socket says 172.21.15.121:51570) 2023-11-15T03:11:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.812+0000 7f0262246700 1 -- 172.21.15.121:0/2371068386 learned_addr learned my addr 172.21.15.121:0/2371068386 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f0261a45700 1 --2- 172.21.15.121:0/2371068386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5033e4dd0 0x55a5034bc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f0259244700 1 --2- 172.21.15.121:0/2371068386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a50342eb00 0x55a5034bfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f0262246700 1 -- 172.21.15.121:0/2371068386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5033e4dd0 msgr2=0x55a5034bc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f0262246700 1 --2- 172.21.15.121:0/2371068386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5033e4dd0 0x55a5034bc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f0262246700 1 -- 172.21.15.121:0/2371068386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a50342eb00 msgr2=0x55a5034bfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:40.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f0262246700 1 --2- 172.21.15.121:0/2371068386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a50342eb00 0x55a5034bfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f0262246700 1 -- 172.21.15.121:0/2371068386 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0254005cf0 con 0x55a5034bd670 2023-11-15T03:11:40.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f0262246700 1 --2- 172.21.15.121:0/2371068386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5034bd670 0x55a50342cdb0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 rx=0x7f025c001550 tx=0x7f025c006b10).ready entity=mon.0 client_cookie=24c56da7282ed04a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f0260a43700 1 -- 172.21.15.121:0/2371068386 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c0030d0 con 0x55a5034bd670 2023-11-15T03:11:40.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f0260a43700 1 -- 172.21.15.121:0/2371068386 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025c012400 con 0x55a5034bd670 2023-11-15T03:11:40.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f0260a43700 1 -- 172.21.15.121:0/2371068386 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c012780 con 0x55a5034bd670 2023-11-15T03:11:40.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f026d53bd00 1 -- 172.21.15.121:0/2371068386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5034bd670 msgr2=0x55a50342cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f026d53bd00 1 --2- 172.21.15.121:0/2371068386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5034bd670 0x55a50342cdb0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 rx=0x7f025c001550 tx=0x7f025c006b10).stop 2023-11-15T03:11:40.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f026d53bd00 1 -- 172.21.15.121:0/2371068386 shutdown_connections 2023-11-15T03:11:40.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f026d53bd00 1 --2- 172.21.15.121:0/2371068386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5034bd670 0x55a50342cdb0 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f026d53bd00 1 --2- 172.21.15.121:0/2371068386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a50342eb00 0x55a5034bfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f026d53bd00 1 --2- 172.21.15.121:0/2371068386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5033e4dd0 0x55a5034bc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f026d53bd00 1 -- 172.21.15.121:0/2371068386 >> 172.21.15.121:0/2371068386 conn(0x55a503434080 msgr2=0x55a503416700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:40.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f026d53bd00 1 -- 172.21.15.121:0/2371068386 shutdown_connections 2023-11-15T03:11:40.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f026d53bd00 1 -- 172.21.15.121:0/2371068386 wait complete. 2023-11-15T03:11:40.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f026d53bd00 1 Processor -- start 2023-11-15T03:11:40.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.816+0000 7f026d53bd00 1 -- start start 2023-11-15T03:11:40.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f026d53bd00 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5033e4dd0 0x55a5034b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f026d53bd00 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a50342eb00 0x55a5034bf820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f026d53bd00 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5034bd670 0x55a5034bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f026d53bd00 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5034c0200 con 0x55a5033e4dd0 2023-11-15T03:11:40.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f026d53bd00 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a503416bb0 con 0x55a50342eb00 2023-11-15T03:11:40.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f026d53bd00 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5034168d0 con 0x55a5034bd670 2023-11-15T03:11:40.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f0261a45700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5033e4dd0 0x55a5034b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f0261a45700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5033e4dd0 0x55a5034b8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51582/0 (socket says 172.21.15.121:51582) 2023-11-15T03:11:40.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f0261a45700 1 -- 172.21.15.121:0/2660252863 learned_addr learned my addr 172.21.15.121:0/2660252863 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:40.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f0262246700 1 --2- 172.21.15.121:0/2660252863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5034bd670 0x55a5034bfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f0259244700 1 --2- 172.21.15.121:0/2660252863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a50342eb00 0x55a5034bf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f0261a45700 1 -- 172.21.15.121:0/2660252863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5034bd670 msgr2=0x55a5034bfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f0261a45700 1 --2- 172.21.15.121:0/2660252863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a5034bd670 0x55a5034bfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f0261a45700 1 -- 172.21.15.121:0/2660252863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a50342eb00 msgr2=0x55a5034bf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f0261a45700 1 --2- 172.21.15.121:0/2660252863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a50342eb00 0x55a5034bf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f0261a45700 1 -- 172.21.15.121:0/2660252863 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a503386a20 con 0x55a5033e4dd0 2023-11-15T03:11:40.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f0259244700 1 --2- 172.21.15.121:0/2660252863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a50342eb00 0x55a5034bf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:40.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f0261a45700 1 --2- 172.21.15.121:0/2660252863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5033e4dd0 0x55a5034b8bb0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 rx=0x7f02540084c0 tx=0x7f0254016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f025affd700 1 -- 172.21.15.121:0/2660252863 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0254013820 con 0x55a5033e4dd0 2023-11-15T03:11:40.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f025affd700 1 -- 172.21.15.121:0/2660252863 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0254013980 con 0x55a5033e4dd0 2023-11-15T03:11:40.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f026d53bd00 1 -- 172.21.15.121:0/2660252863 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a503355a70 con 0x55a5033e4dd0 2023-11-15T03:11:40.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f025affd700 1 -- 172.21.15.121:0/2660252863 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0254013d00 con 0x55a5033e4dd0 2023-11-15T03:11:40.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.820+0000 7f026d53bd00 1 -- 172.21.15.121:0/2660252863 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a503361010 con 0x55a5033e4dd0 2023-11-15T03:11:40.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.824+0000 7f025affd700 1 -- 172.21.15.121:0/2660252863 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f025401d040 con 0x55a5033e4dd0 2023-11-15T03:11:40.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.824+0000 7f025affd700 1 --2- 172.21.15.121:0/2660252863 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0244059020 0x7f024405b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.824+0000 7f025affd700 1 -- 172.21.15.121:0/2660252863 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f0254075480 con 0x55a5033e4dd0 2023-11-15T03:11:40.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.824+0000 7f0259244700 1 --2- 172.21.15.121:0/2660252863 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0244059020 0x7f024405b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.824+0000 7f026d53bd00 1 --2- 172.21.15.121:0/2660252863 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a5034bdab0 0x55a5034bb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:40.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.824+0000 7f026d53bd00 1 -- 172.21.15.121:0/2660252863 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- osd_op(unknown.0.0:1 15.2a 15:57fe7535:::100000001f6.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e148) v8 -- 0x55a50342d390 con 0x55a5034bdab0 2023-11-15T03:11:40.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.824+0000 7f0262246700 1 --2- 172.21.15.121:0/2660252863 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a5034bdab0 0x55a5034bb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:40.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.824+0000 7f0259244700 1 --2- 172.21.15.121:0/2660252863 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0244059020 0x7f024405b4e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x55a5034b9d40 tx=0x7f024c0068f0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.824+0000 7f0262246700 1 --2- 172.21.15.121:0/2660252863 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a5034bdab0 0x55a5034bb850 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:40.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.828+0000 7f0262246700 1 -- 172.21.15.121:0/2660252863 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== osd_op_reply(1 100000001f6.00000000 [delete] v148'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f025c012570 con 0x55a5034bdab0 2023-11-15T03:11:40.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.828+0000 7f026d53bd00 1 -- 172.21.15.121:0/2660252863 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a5034bdab0 msgr2=0x55a5034bb850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.828+0000 7f026d53bd00 1 --2- 172.21.15.121:0/2660252863 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x55a5034bdab0 0x55a5034bb850 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:40.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.828+0000 7f026d53bd00 1 -- 172.21.15.121:0/2660252863 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0244059020 msgr2=0x7f024405b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.828+0000 7f026d53bd00 1 --2- 172.21.15.121:0/2660252863 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0244059020 0x7f024405b4e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x55a5034b9d40 tx=0x7f024c0068f0).stop 2023-11-15T03:11:40.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.828+0000 7f026d53bd00 1 -- 172.21.15.121:0/2660252863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5033e4dd0 msgr2=0x55a5034b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:40.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.828+0000 7f026d53bd00 1 --2- 172.21.15.121:0/2660252863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5033e4dd0 0x55a5034b8bb0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 rx=0x7f02540084c0 tx=0x7f0254016000).stop 2023-11-15T03:11:40.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.828+0000 7f0262246700 1 -- 172.21.15.121:0/2660252863 reap_dead start 2023-11-15T03:11:40.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.828+0000 7f026d53bd00 1 -- 172.21.15.121:0/2660252863 shutdown_connections 2023-11-15T03:11:40.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.828+0000 7f026d53bd00 1 -- 172.21.15.121:0/2660252863 >> 172.21.15.121:0/2660252863 conn(0x55a503434080 msgr2=0x55a5034b94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:40.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.828+0000 7f026d53bd00 1 -- 172.21.15.121:0/2660252863 shutdown_connections 2023-11-15T03:11:40.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:40.828+0000 7f026d53bd00 1 -- 172.21.15.121:0/2660252863 wait complete. 2023-11-15T03:11:40.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:11:40.845 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink2) 2023-11-15T03:11:40.872 INFO:tasks.ceph.mds.a.smithi121.stderr:2023-11-15T03:11:40.864+0000 7fea4370a700 -1 log_channel(cluster) log [ERR] : dir 0x100000001f6 object missing on disk; some files may be lost (/dir2) 2023-11-15T03:11:40.873 INFO:teuthology.orchestra.run.smithi121.stderr:stat: cannot stat 'testdir/hardlink2': Input/output error 2023-11-15T03:11:40.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:11:40.875 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:41.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.080+0000 7f09bb59e700 1 Processor -- start 2023-11-15T03:11:41.088 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.080+0000 7f09bb59e700 1 -- start start 2023-11-15T03:11:41.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.080+0000 7f09bb59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f09bc0cafb0 0x7f09bc066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.080+0000 7f09bb59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09bc067270 0x7f09bc0ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.080+0000 7f09bb59e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09bc0ad3a0 con 0x7f09bc0cabd0 2023-11-15T03:11:41.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.080+0000 7f09bb59e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09bc0cb410 con 0x7f09bc0cafb0 2023-11-15T03:11:41.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.080+0000 7f09bb59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc0ad650 con 0x7f09bc067270 2023-11-15T03:11:41.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.080+0000 7f09bad9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09bc067270 0x7f09bc0ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.080+0000 7f09b3fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f09bc0cafb0 0x7f09bc066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.080+0000 7f09b3fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f09bc0cafb0 0x7f09bc066d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:39766/0 (socket says 172.21.15.121:39766) 2023-11-15T03:11:41.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.080+0000 7f09b3fff700 1 -- 172.21.15.121:0/424897610 learned_addr learned my addr 172.21.15.121:0/424897610 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:41.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.080+0000 7f09bad9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09bc067270 0x7f09bc0ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:59530/0 (socket says 172.21.15.121:59530) 2023-11-15T03:11:41.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.080+0000 7f09b3fff700 1 -- 172.21.15.121:0/424897610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09bc067270 msgr2=0x7f09bc0ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:41.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.080+0000 7f09b3fff700 1 --2- 172.21.15.121:0/424897610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09bc067270 0x7f09bc0ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.080+0000 7f09b3fff700 1 -- 172.21.15.121:0/424897610 >> v1:172.21.15.190:6789/0 conn(0x7f09bc0cabd0 legacy=0x7f09bc0acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.080+0000 7f09b3fff700 1 -- 172.21.15.121:0/424897610 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09bc065fc0 con 0x7f09bc0cafb0 2023-11-15T03:11:41.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.080+0000 7f09bad9d700 1 --2- 172.21.15.121:0/424897610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09bc067270 0x7f09bc0ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:41.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.084+0000 7f09b3fff700 1 --2- 172.21.15.121:0/424897610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09bc0cafb0 0x7f09bc066d80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7f09a4009f20 tx=0x7f09a4009840).ready entity=mon.1 client_cookie=5a6b3f3f045ee88e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:41.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.084+0000 7f09b9d9b700 1 -- 172.21.15.121:0/424897610 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a400e070 con 0x7f09bc0cafb0 2023-11-15T03:11:41.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.084+0000 7f09b9d9b700 1 -- 172.21.15.121:0/424897610 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a400b9d0 con 0x7f09bc0cafb0 2023-11-15T03:11:41.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.084+0000 7f09b9d9b700 1 -- 172.21.15.121:0/424897610 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a400bd50 con 0x7f09bc0cafb0 2023-11-15T03:11:41.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.084+0000 7f09bb59e700 1 -- 172.21.15.121:0/424897610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09bc0cafb0 msgr2=0x7f09bc066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.084+0000 7f09bb59e700 1 --2- 172.21.15.121:0/424897610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09bc0cafb0 0x7f09bc066d80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7f09a4009f20 tx=0x7f09a4009840).stop 2023-11-15T03:11:41.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.084+0000 7f09bb59e700 1 -- 172.21.15.121:0/424897610 shutdown_connections 2023-11-15T03:11:41.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.084+0000 7f09bb59e700 1 --2- 172.21.15.121:0/424897610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09bc067270 0x7f09bc0ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.084+0000 7f09bb59e700 1 --2- 172.21.15.121:0/424897610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09bc0cafb0 0x7f09bc066d80 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.084+0000 7f09bb59e700 1 -- 172.21.15.121:0/424897610 >> 172.21.15.121:0/424897610 conn(0x7f09bc062190 msgr2=0x7f09bc0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:41.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.084+0000 7f09bb59e700 1 -- 172.21.15.121:0/424897610 shutdown_connections 2023-11-15T03:11:41.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.084+0000 7f09bb59e700 1 -- 172.21.15.121:0/424897610 wait complete. 2023-11-15T03:11:41.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.084+0000 7f09bb59e700 1 Processor -- start 2023-11-15T03:11:41.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.084+0000 7f09bb59e700 1 -- start start 2023-11-15T03:11:41.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09bb59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09bc067270 0x7f09bc0c1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09bb59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09bc0cabd0 0x7f09bc0ac260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09bb59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09bc0cafb0 0x7f09bc1682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.100 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09bb59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc153160 con 0x7f09bc067270 2023-11-15T03:11:41.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09bb59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc0c0de0 con 0x7f09bc0cafb0 2023-11-15T03:11:41.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09bb59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc1687c0 con 0x7f09bc0cabd0 2023-11-15T03:11:41.101 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09ba59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09bc067270 0x7f09bc0c1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09ba59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09bc067270 0x7f09bc0c1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51598/0 (socket says 172.21.15.121:51598) 2023-11-15T03:11:41.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09ba59c700 1 -- 172.21.15.121:0/1510758340 learned_addr learned my addr 172.21.15.121:0/1510758340 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:41.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09b3fff700 1 --2- 172.21.15.121:0/1510758340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09bc0cabd0 0x7f09bc0ac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.102 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09bad9d700 1 --2- 172.21.15.121:0/1510758340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09bc0cafb0 0x7f09bc1682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09ba59c700 1 -- 172.21.15.121:0/1510758340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09bc0cabd0 msgr2=0x7f09bc0ac260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.103 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09ba59c700 1 --2- 172.21.15.121:0/1510758340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09bc0cabd0 0x7f09bc0ac260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09ba59c700 1 -- 172.21.15.121:0/1510758340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09bc0cafb0 msgr2=0x7f09bc1682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09ba59c700 1 --2- 172.21.15.121:0/1510758340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09bc0cafb0 0x7f09bc1682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09ba59c700 1 -- 172.21.15.121:0/1510758340 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b40092a0 con 0x7f09bc067270 2023-11-15T03:11:41.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09ba59c700 1 --2- 172.21.15.121:0/1510758340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09bc067270 0x7f09bc0c1270 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 rx=0x7f09ac00c4f0 tx=0x7f09ac007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:41.104 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09b27fc700 1 -- 172.21.15.121:0/1510758340 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac007020 con 0x7f09bc067270 2023-11-15T03:11:41.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09b27fc700 1 -- 172.21.15.121:0/1510758340 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ac001e90 con 0x7f09bc067270 2023-11-15T03:11:41.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09bb59e700 1 -- 172.21.15.121:0/1510758340 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a4009ae0 con 0x7f09bc067270 2023-11-15T03:11:41.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09b27fc700 1 -- 172.21.15.121:0/1510758340 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac0073b0 con 0x7f09bc067270 2023-11-15T03:11:41.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.088+0000 7f09bb59e700 1 -- 172.21.15.121:0/1510758340 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09bc0c01d0 con 0x7f09bc067270 2023-11-15T03:11:41.105 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.092+0000 7f09bb59e700 1 -- 172.21.15.121:0/1510758340 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09bc06bcf0 con 0x7f09bc067270 2023-11-15T03:11:41.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.092+0000 7f09b27fc700 1 -- 172.21.15.121:0/1510758340 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f09ac0068b0 con 0x7f09bc067270 2023-11-15T03:11:41.106 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.092+0000 7f09b27fc700 1 --2- 172.21.15.121:0/1510758340 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f099c05a540 0x7f099c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.092+0000 7f09b27fc700 1 -- 172.21.15.121:0/1510758340 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f09ac074240 con 0x7f09bc067270 2023-11-15T03:11:41.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.092+0000 7f09b3fff700 1 --2- 172.21.15.121:0/1510758340 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f099c05a540 0x7f099c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.092+0000 7f09b3fff700 1 --2- 172.21.15.121:0/1510758340 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f099c05a540 0x7f099c05ca00 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7f09bc0c4480 tx=0x7f09a4009840).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:41.107 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.100+0000 7f09b27fc700 1 -- 172.21.15.121:0/1510758340 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f09ac0478b0 con 0x7f09bc067270 2023-11-15T03:11:41.305 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.296+0000 7f09bb59e700 1 -- 172.21.15.121:0/1510758340 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09bc0662a0 con 0x7f09bc067270 2023-11-15T03:11:41.306 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.296+0000 7f09b27fc700 1 -- 172.21.15.121:0/1510758340 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 355 v355) v1 ==== 95+0+4514 (secure 0 0 0) 0x7f09ac04aed0 con 0x7f09bc067270 2023-11-15T03:11:41.306 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:41.307 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":355,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/755048854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":755048854},{"type":"v1","addr":"172.21.15.190:6834","nonce":755048854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2360242049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2360242049},{"type":"v1","addr":"172.21.15.190:6835","nonce":2360242049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2780153683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2780153683},{"type":"v1","addr":"172.21.15.121:6835","nonce":2780153683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345}],"filesystems":[{"mdsmap":{"epoch":355,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:11:21.864101+0000","modified":"2023-11-15T03:11:36.256557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10899":{"gid":10899,"name":"a","rank":0,"incarnation":352,"state":"up:active","state_seq":4,"addr":"172.21.15.121:6837/3298117008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3298117008},{"type":"v1","addr":"172.21.15.121:6837","nonce":3298117008}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-15T03:11:41.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.300+0000 7f09bb59e700 1 -- 172.21.15.121:0/1510758340 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f099c05a540 msgr2=0x7f099c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.300+0000 7f09bb59e700 1 --2- 172.21.15.121:0/1510758340 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f099c05a540 0x7f099c05ca00 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7f09bc0c4480 tx=0x7f09a4009840).stop 2023-11-15T03:11:41.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.300+0000 7f09bb59e700 1 -- 172.21.15.121:0/1510758340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09bc067270 msgr2=0x7f09bc0c1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.310 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.300+0000 7f09bb59e700 1 --2- 172.21.15.121:0/1510758340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09bc067270 0x7f09bc0c1270 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 rx=0x7f09ac00c4f0 tx=0x7f09ac007e00).stop 2023-11-15T03:11:41.311 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.300+0000 7f09bb59e700 1 -- 172.21.15.121:0/1510758340 shutdown_connections 2023-11-15T03:11:41.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.300+0000 7f09bb59e700 1 --2- 172.21.15.121:0/1510758340 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f099c05a540 0x7f099c05ca00 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.300+0000 7f09bb59e700 1 --2- 172.21.15.121:0/1510758340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09bc0cafb0 0x7f09bc1682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.300+0000 7f09bb59e700 1 --2- 172.21.15.121:0/1510758340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09bc0cabd0 0x7f09bc0ac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.300+0000 7f09bb59e700 1 --2- 172.21.15.121:0/1510758340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09bc067270 0x7f09bc0c1270 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.300+0000 7f09bb59e700 1 -- 172.21.15.121:0/1510758340 >> 172.21.15.121:0/1510758340 conn(0x7f09bc062190 msgr2=0x7f09bc0af350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:41.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.300+0000 7f09bb59e700 1 -- 172.21.15.121:0/1510758340 shutdown_connections 2023-11-15T03:11:41.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.300+0000 7f09bb59e700 1 -- 172.21.15.121:0/1510758340 wait complete. 2023-11-15T03:11:41.313 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 355 2023-11-15T03:11:41.332 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2023-11-15T03:11:41.499 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.492+0000 7faf80df9700 1 Processor -- start 2023-11-15T03:11:41.500 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.492+0000 7faf80df9700 1 -- start start 2023-11-15T03:11:41.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.496+0000 7faf80df9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faf7c0c5c70 0x7faf7c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.503 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.496+0000 7faf80df9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf7c065ea0 con 0x7faf7c066330 2023-11-15T03:11:41.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.496+0000 7faf80df9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf7c066bb0 con 0x7faf7c0acd00 2023-11-15T03:11:41.504 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.496+0000 7faf80df9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf7c065890 con 0x7faf7c0c5c70 2023-11-15T03:11:41.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.496+0000 7faf7bfff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7faf7c066330 0x7faf7c1577e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56298/0 (socket says 172.21.15.121:56298) 2023-11-15T03:11:41.505 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.496+0000 7faf7bfff700 1 -- 172.21.15.121:0/2698304164 learned_addr learned my addr 172.21.15.121:0/2698304164 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:41.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.496+0000 7faf7affd700 1 --2- 172.21.15.121:0/2698304164 >> v2:172.21.15.190:3300/0 conn(0x7faf7c0c5c70 0x7faf7c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.496+0000 7faf7a7fc700 1 -- 172.21.15.121:0/2698304164 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720090619 0 0) 0x7faf7c065ea0 con 0x7faf7c066330 2023-11-15T03:11:41.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.496+0000 7faf7a7fc700 1 -- 172.21.15.121:0/2698304164 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf640033f0 con 0x7faf7c066330 2023-11-15T03:11:41.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.496+0000 7faf7affd700 1 -- 172.21.15.121:0/2698304164 >> v1:172.21.15.190:6789/0 conn(0x7faf7c0acd00 legacy=0x7faf7c1536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.496+0000 7faf7affd700 1 -- 172.21.15.121:0/2698304164 >> v1:172.21.15.121:6789/0 conn(0x7faf7c066330 legacy=0x7faf7c1577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.496+0000 7faf7a7fc700 1 -- 172.21.15.121:0/2698304164 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3105961269 0 0) 0x7faf640033f0 con 0x7faf7c066330 2023-11-15T03:11:41.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.496+0000 7faf7affd700 1 -- 172.21.15.121:0/2698304164 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7c0c64a0 con 0x7faf7c0c5c70 2023-11-15T03:11:41.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.496+0000 7faf7affd700 1 --2- 172.21.15.121:0/2698304164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf7c0c5c70 0x7faf7c153dd0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7faf70009760 tx=0x7faf70009060).ready entity=mon.1 client_cookie=b83160b182c391ef server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:41.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.496+0000 7faf7a7fc700 1 -- 172.21.15.121:0/2698304164 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf740017f0 con 0x7faf7c0c5c70 2023-11-15T03:11:41.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.496+0000 7faf7a7fc700 1 -- 172.21.15.121:0/2698304164 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf70011400 con 0x7faf7c0c5c70 2023-11-15T03:11:41.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.496+0000 7faf7a7fc700 1 -- 172.21.15.121:0/2698304164 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70011780 con 0x7faf7c0c5c70 2023-11-15T03:11:41.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.500+0000 7faf80df9700 1 -- 172.21.15.121:0/2698304164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf7c0c5c70 msgr2=0x7faf7c153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.500+0000 7faf80df9700 1 --2- 172.21.15.121:0/2698304164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf7c0c5c70 0x7faf7c153dd0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7faf70009760 tx=0x7faf70009060).stop 2023-11-15T03:11:41.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.500+0000 7faf80df9700 1 -- 172.21.15.121:0/2698304164 shutdown_connections 2023-11-15T03:11:41.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.500+0000 7faf80df9700 1 --2- 172.21.15.121:0/2698304164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf7c0c5c70 0x7faf7c153dd0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.500+0000 7faf80df9700 1 -- 172.21.15.121:0/2698304164 >> 172.21.15.121:0/2698304164 conn(0x7faf7c063fc0 msgr2=0x7faf7c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:41.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.500+0000 7faf80df9700 1 -- 172.21.15.121:0/2698304164 shutdown_connections 2023-11-15T03:11:41.510 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.500+0000 7faf80df9700 1 -- 172.21.15.121:0/2698304164 wait complete. 2023-11-15T03:11:41.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.500+0000 7faf80df9700 1 Processor -- start 2023-11-15T03:11:41.511 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.504+0000 7faf80df9700 1 -- start start 2023-11-15T03:11:41.512 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.504+0000 7faf80df9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf7c066330 0x7faf7c14c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.504+0000 7faf80df9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf7c0acd00 0x7faf7c14e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.504+0000 7faf80df9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf7c0c5c70 0x7faf7c14e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.504+0000 7faf80df9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c14ea80 con 0x7faf7c0acd00 2023-11-15T03:11:41.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.504+0000 7faf80df9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c154480 con 0x7faf7c066330 2023-11-15T03:11:41.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.504+0000 7faf80df9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c1684d0 con 0x7faf7c0c5c70 2023-11-15T03:11:41.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.504+0000 7faf7affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf7c0acd00 0x7faf7c14e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.504+0000 7faf7affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf7c0acd00 0x7faf7c14e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51614/0 (socket says 172.21.15.121:51614) 2023-11-15T03:11:41.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.504+0000 7faf7affd700 1 -- 172.21.15.121:0/4096886053 learned_addr learned my addr 172.21.15.121:0/4096886053 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:41.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.504+0000 7faf7b7fe700 1 --2- 172.21.15.121:0/4096886053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf7c066330 0x7faf7c14c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.504+0000 7faf7bfff700 1 --2- 172.21.15.121:0/4096886053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf7c0c5c70 0x7faf7c14e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.504+0000 7faf7affd700 1 -- 172.21.15.121:0/4096886053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf7c0c5c70 msgr2=0x7faf7c14e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.504+0000 7faf7affd700 1 --2- 172.21.15.121:0/4096886053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf7c0c5c70 0x7faf7c14e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.504+0000 7faf7affd700 1 -- 172.21.15.121:0/4096886053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf7c066330 msgr2=0x7faf7c14c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.504+0000 7faf7affd700 1 --2- 172.21.15.121:0/4096886053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf7c066330 0x7faf7c14c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.504+0000 7faf7affd700 1 -- 172.21.15.121:0/4096886053 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf700092a0 con 0x7faf7c0acd00 2023-11-15T03:11:41.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.504+0000 7faf7b7fe700 1 --2- 172.21.15.121:0/4096886053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf7c066330 0x7faf7c14c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:41.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.504+0000 7faf7affd700 1 --2- 172.21.15.121:0/4096886053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf7c0acd00 0x7faf7c14e0a0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 rx=0x7faf70009e00 tx=0x7faf70014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:41.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.508+0000 7faf637fe700 1 -- 172.21.15.121:0/4096886053 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70010800 con 0x7faf7c0acd00 2023-11-15T03:11:41.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.508+0000 7faf637fe700 1 -- 172.21.15.121:0/4096886053 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf70010960 con 0x7faf7c0acd00 2023-11-15T03:11:41.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.508+0000 7faf80df9700 1 -- 172.21.15.121:0/4096886053 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7c167930 con 0x7faf7c0acd00 2023-11-15T03:11:41.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.508+0000 7faf637fe700 1 -- 172.21.15.121:0/4096886053 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70010d20 con 0x7faf7c0acd00 2023-11-15T03:11:41.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.508+0000 7faf80df9700 1 -- 172.21.15.121:0/4096886053 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7c167e80 con 0x7faf7c0acd00 2023-11-15T03:11:41.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.512+0000 7faf637fe700 1 -- 172.21.15.121:0/4096886053 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7faf7001d640 con 0x7faf7c0acd00 2023-11-15T03:11:41.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.512+0000 7faf637fe700 1 --2- 172.21.15.121:0/4096886053 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faf6805a510 0x7faf6805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.512+0000 7faf637fe700 1 -- 172.21.15.121:0/4096886053 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7faf70075e00 con 0x7faf7c0acd00 2023-11-15T03:11:41.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.512+0000 7faf7b7fe700 1 --2- 172.21.15.121:0/4096886053 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faf6805a510 0x7faf6805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.512+0000 7faf7b7fe700 1 --2- 172.21.15.121:0/4096886053 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faf6805a510 0x7faf6805c9d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7faf6c007fd0 tx=0x7faf6c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:41.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.512+0000 7faf8c3d5740 1 Processor -- start 2023-11-15T03:11:41.523 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.516+0000 7faf8c3d5740 1 -- start start 2023-11-15T03:11:41.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.516+0000 7faf8c3d5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d2810 0x15d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.516+0000 7faf8c3d5740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15d6810 0x158e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.516+0000 7faf8c3d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1589110 0x1589590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.516+0000 7faf8c3d5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x14ca3f0 con 0x1589110 2023-11-15T03:11:41.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.516+0000 7faf8c3d5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x14ed580 con 0x15d2810 2023-11-15T03:11:41.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.516+0000 7faf8c3d5740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x14b6d50 con 0x15d6810 2023-11-15T03:11:41.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.516+0000 7faf7bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1589110 0x1589590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.516+0000 7faf7bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1589110 0x1589590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51626/0 (socket says 172.21.15.121:51626) 2023-11-15T03:11:41.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.516+0000 7faf7bfff700 1 -- 172.21.15.121:0/2777209766 learned_addr learned my addr 172.21.15.121:0/2777209766 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:41.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.516+0000 7faf7b7fe700 1 --2- 172.21.15.121:0/2777209766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d2810 0x15d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.516+0000 7faf7affd700 1 --2- 172.21.15.121:0/2777209766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15d6810 0x158e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.516+0000 7faf7bfff700 1 -- 172.21.15.121:0/2777209766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15d6810 msgr2=0x158e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:41.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.516+0000 7faf7bfff700 1 --2- 172.21.15.121:0/2777209766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15d6810 0x158e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.516+0000 7faf7bfff700 1 -- 172.21.15.121:0/2777209766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d2810 msgr2=0x15d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:41.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.516+0000 7faf7bfff700 1 --2- 172.21.15.121:0/2777209766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d2810 0x15d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.516+0000 7faf7bfff700 1 -- 172.21.15.121:0/2777209766 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x150a830 con 0x1589110 2023-11-15T03:11:41.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.516+0000 7faf7b7fe700 1 --2- 172.21.15.121:0/2777209766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d2810 0x15d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:41.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.516+0000 7faf7bfff700 1 --2- 172.21.15.121:0/2777209766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1589110 0x1589590 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 rx=0x7faf74004f80 tx=0x7faf74001d90).ready entity=mon.0 client_cookie=2dc99af2669193dd server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:41.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.520+0000 7faf60ff9700 1 -- 172.21.15.121:0/2777209766 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf74004a30 con 0x1589110 2023-11-15T03:11:41.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.520+0000 7faf60ff9700 1 -- 172.21.15.121:0/2777209766 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7400fce0 con 0x1589110 2023-11-15T03:11:41.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.520+0000 7faf60ff9700 1 -- 172.21.15.121:0/2777209766 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf74015620 con 0x1589110 2023-11-15T03:11:41.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.520+0000 7faf8c3d5740 1 -- 172.21.15.121:0/2777209766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1589110 msgr2=0x1589590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.520+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/2777209766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1589110 0x1589590 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 rx=0x7faf74004f80 tx=0x7faf74001d90).stop 2023-11-15T03:11:41.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.520+0000 7faf8c3d5740 1 -- 172.21.15.121:0/2777209766 shutdown_connections 2023-11-15T03:11:41.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.520+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/2777209766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1589110 0x1589590 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.520+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/2777209766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15d6810 0x158e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.520+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/2777209766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d2810 0x15d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.520+0000 7faf8c3d5740 1 -- 172.21.15.121:0/2777209766 >> 172.21.15.121:0/2777209766 conn(0x136f790 msgr2=0x15d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:41.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.520+0000 7faf8c3d5740 1 -- 172.21.15.121:0/2777209766 shutdown_connections 2023-11-15T03:11:41.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.520+0000 7faf8c3d5740 1 -- 172.21.15.121:0/2777209766 wait complete. 2023-11-15T03:11:41.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.520+0000 7faf8c3d5740 1 -- start start 2023-11-15T03:11:41.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.520+0000 7faf8c3d5740 1 Processor -- start 2023-11-15T03:11:41.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.520+0000 7faf8c3d5740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1589110 0x15b7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.520+0000 7faf8c3d5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d2810 0x15b9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf8c3d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15d6810 0x15bc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf8c3d5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x15e3220 con 0x15d6810 2023-11-15T03:11:41.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf8c3d5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x15bb4d0 con 0x15d2810 2023-11-15T03:11:41.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf8c3d5740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x15b7870 con 0x1589110 2023-11-15T03:11:41.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf8c3d5740 10 client.0 ll_register_callbacks cb 0x15076b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:11:41.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf7bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15d6810 0x15bc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf7bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15d6810 0x15bc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51628/0 (socket says 172.21.15.121:51628) 2023-11-15T03:11:41.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf7bfff700 1 -- 172.21.15.121:0/4043547605 learned_addr learned my addr 172.21.15.121:0/4043547605 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:41.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf7b7fe700 1 --2- 172.21.15.121:0/4043547605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1589110 0x15b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf7affd700 1 --2- 172.21.15.121:0/4043547605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d2810 0x15b9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.542 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf7bfff700 1 -- 172.21.15.121:0/4043547605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1589110 msgr2=0x15b7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:41.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf7bfff700 1 --2- 172.21.15.121:0/4043547605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1589110 0x15b7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf7bfff700 1 -- 172.21.15.121:0/4043547605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d2810 msgr2=0x15b9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:41.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf7bfff700 1 --2- 172.21.15.121:0/4043547605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d2810 0x15b9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf7bfff700 1 -- 172.21.15.121:0/4043547605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7c066bb0 con 0x15d6810 2023-11-15T03:11:41.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf7bfff700 1 --2- 172.21.15.121:0/4043547605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15d6810 0x15bc7c0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 rx=0x7faf74004fd0 tx=0x7faf74016000).ready entity=mon.0 client_cookie=412ff6d5632da208 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:41.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:11:41.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf59ffb700 1 -- 172.21.15.121:0/4043547605 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf74015ba0 con 0x15d6810 2023-11-15T03:11:41.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf59ffb700 1 -- 172.21.15.121:0/4043547605 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7faf74022d40 con 0x15d6810 2023-11-15T03:11:41.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf8c3d5740 1 -- 172.21.15.121:0/4043547605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x15072a0 con 0x15d6810 2023-11-15T03:11:41.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf59ffb700 1 -- 172.21.15.121:0/4043547605 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=355) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf740143b0 con 0x15d6810 2023-11-15T03:11:41.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf8c3d5740 10 client.11049 fetch_fsmap learned FSMap version 355 2023-11-15T03:11:41.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf8c3d5740 1 -- 172.21.15.121:0/4043547605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=355,monmap=2+}) v3 -- 0x7faf74004be0 con 0x15d6810 2023-11-15T03:11:41.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf59ffb700 1 -- 172.21.15.121:0/4043547605 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x15072a0 con 0x15d6810 2023-11-15T03:11:41.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf59ffb700 1 -- 172.21.15.121:0/4043547605 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 355) v1 ==== 3100+0+0 (secure 0 0 0) 0x7faf74012070 con 0x15d6810 2023-11-15T03:11:41.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf8c3d5740 10 client.11049 fetch_fsmap finished waiting for FSMap version 355 2023-11-15T03:11:41.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf8c3d5740 10 client.11049 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:11:41.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf8c3d5740 20 client.11049 populate_metadata read hostname 'smithi121' 2023-11-15T03:11:41.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/4043547605 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x15ba450 0x15e5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf8c3d5740 4 client.11049 mds_command: new command op to 10899 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:11:41.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf8c3d5740 1 -- 172.21.15.121:0/4043547605 --> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x14fd7f0 con 0x15ba450 2023-11-15T03:11:41.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.524+0000 7faf7b7fe700 1 --2- 172.21.15.121:0/4043547605 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x15ba450 0x15e5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.528+0000 7faf59ffb700 10 client.11049 ms_handle_connect on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:41.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.528+0000 7faf7b7fe700 1 --2- 172.21.15.121:0/4043547605 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x15ba450 0x15e5bf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=14cf6969a6e5cbfe server_cookie=7549f677c082673e in_seq=0 out_seq=0 2023-11-15T03:11:41.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.528+0000 7faf59ffb700 1 -- 172.21.15.121:0/4043547605 <== mds.0 v2:172.21.15.121:6836/3298117008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x14fd7f0 con 0x15ba450 2023-11-15T03:11:41.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.528+0000 7faf59ffb700 10 client.11049 handle_command_reply: tid=0 2023-11-15T03:11:41.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.528+0000 7faf8c3d5740 1 client.11049 shutdown 2023-11-15T03:11:41.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.528+0000 7faf8c3d5740 1 -- 172.21.15.121:0/4043547605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15d6810 msgr2=0x15bc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.528+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/4043547605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15d6810 0x15bc7c0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 rx=0x7faf74004fd0 tx=0x7faf74016000).stop 2023-11-15T03:11:41.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.528+0000 7faf8c3d5740 1 -- 172.21.15.121:0/4043547605 shutdown_connections 2023-11-15T03:11:41.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.528+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/4043547605 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x15ba450 0x15e5bf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.528+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/4043547605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15d6810 0x15bc7c0 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.528+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/4043547605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d2810 0x15b9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.528+0000 7faf59ffb700 0 client.11049 ms_handle_reset on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:41.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.528+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/4043547605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1589110 0x15b7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.528+0000 7faf8c3d5740 1 -- 172.21.15.121:0/4043547605 >> 172.21.15.121:0/4043547605 conn(0x136f790 msgr2=0x15b53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:41.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.528+0000 7faf8c3d5740 1 -- 172.21.15.121:0/4043547605 shutdown_connections 2023-11-15T03:11:41.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.528+0000 7faf8c3d5740 1 -- 172.21.15.121:0/4043547605 wait complete. 2023-11-15T03:11:41.551 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.528+0000 7faf8c3d5740 20 client.11049 trim_cache size 0 max 16384 2023-11-15T03:11:41.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf8c3d5740 1 Processor -- start 2023-11-15T03:11:41.552 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf8c3d5740 1 -- start start 2023-11-15T03:11:41.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf8c3d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1589110 0x15d5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf8c3d5740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15d2810 0x15b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf8c3d5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d6810 0x15b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf8c3d5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x15d22e0 con 0x1589110 2023-11-15T03:11:41.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf8c3d5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x14c8f40 con 0x15d6810 2023-11-15T03:11:41.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf8c3d5740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x15b8790 con 0x15d2810 2023-11-15T03:11:41.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf7b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1589110 0x15d5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf7b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1589110 0x15d5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51632/0 (socket says 172.21.15.121:51632) 2023-11-15T03:11:41.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf7b7fe700 1 -- 172.21.15.121:0/1712177545 learned_addr learned my addr 172.21.15.121:0/1712177545 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:41.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf7affd700 1 --2- 172.21.15.121:0/1712177545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15d2810 0x15b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf7bfff700 1 --2- 172.21.15.121:0/1712177545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d6810 0x15b82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf7b7fe700 1 -- 172.21.15.121:0/1712177545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15d2810 msgr2=0x15b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:41.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf7b7fe700 1 --2- 172.21.15.121:0/1712177545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15d2810 0x15b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf7b7fe700 1 -- 172.21.15.121:0/1712177545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d6810 msgr2=0x15b82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:41.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf7b7fe700 1 --2- 172.21.15.121:0/1712177545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d6810 0x15b82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf7b7fe700 1 -- 172.21.15.121:0/1712177545 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7c0cabd0 con 0x1589110 2023-11-15T03:11:41.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf7bfff700 1 --2- 172.21.15.121:0/1712177545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d6810 0x15b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:11:41.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.544+0000 7faf7b7fe700 1 --2- 172.21.15.121:0/1712177545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1589110 0x15d5710 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 rx=0x15bb1f0 tx=0x7faf6c00a000).ready entity=mon.0 client_cookie=e671e8aa67003f4d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:41.558 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf617fa700 1 -- 172.21.15.121:0/1712177545 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c007020 con 0x1589110 2023-11-15T03:11:41.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf617fa700 1 -- 172.21.15.121:0/1712177545 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6c007180 con 0x1589110 2023-11-15T03:11:41.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf617fa700 1 -- 172.21.15.121:0/1712177545 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c007500 con 0x1589110 2023-11-15T03:11:41.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf8c3d5740 1 -- 172.21.15.121:0/1712177545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1589110 msgr2=0x15d5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/1712177545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1589110 0x15d5710 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 rx=0x15bb1f0 tx=0x7faf6c00a000).stop 2023-11-15T03:11:41.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf8c3d5740 1 -- 172.21.15.121:0/1712177545 shutdown_connections 2023-11-15T03:11:41.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/1712177545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d6810 0x15b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/1712177545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x15d2810 0x15b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/1712177545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1589110 0x15d5710 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf8c3d5740 1 -- 172.21.15.121:0/1712177545 >> 172.21.15.121:0/1712177545 conn(0x136f790 msgr2=0x15d7480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:41.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf8c3d5740 1 -- 172.21.15.121:0/1712177545 shutdown_connections 2023-11-15T03:11:41.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf8c3d5740 1 -- 172.21.15.121:0/1712177545 wait complete. 2023-11-15T03:11:41.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf8c3d5740 1 -- start start 2023-11-15T03:11:41.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf8c3d5740 1 Processor -- start 2023-11-15T03:11:41.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf8c3d5740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1589110 0x15f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf8c3d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15d2810 0x158d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf8c3d5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d6810 0x158d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf8c3d5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x15d2030 con 0x15d2810 2023-11-15T03:11:41.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf8c3d5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x15d1e20 con 0x15d6810 2023-11-15T03:11:41.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf8c3d5740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x158ddc0 con 0x1589110 2023-11-15T03:11:41.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.548+0000 7faf8c3d5740 10 client.0 ll_register_callbacks cb 0x7faf6c0015a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:11:41.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf7affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15d2810 0x158d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf7affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15d2810 0x158d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51634/0 (socket says 172.21.15.121:51634) 2023-11-15T03:11:41.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf7affd700 1 -- 172.21.15.121:0/4222707683 learned_addr learned my addr 172.21.15.121:0/4222707683 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:41.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf7b7fe700 1 --2- 172.21.15.121:0/4222707683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1589110 0x15f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf7bfff700 1 --2- 172.21.15.121:0/4222707683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d6810 0x158d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf7affd700 1 -- 172.21.15.121:0/4222707683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1589110 msgr2=0x15f4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:41.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf7affd700 1 --2- 172.21.15.121:0/4222707683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1589110 0x15f4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf7affd700 1 -- 172.21.15.121:0/4222707683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d6810 msgr2=0x158d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:41.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf7affd700 1 --2- 172.21.15.121:0/4222707683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d6810 0x158d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf7affd700 1 -- 172.21.15.121:0/4222707683 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6c00feb0 con 0x15d2810 2023-11-15T03:11:41.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf7affd700 1 --2- 172.21.15.121:0/4222707683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15d2810 0x158d3e0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 rx=0x7faf700496a0 tx=0x7faf7004f040).ready entity=mon.0 client_cookie=2e930fad3788c214 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:41.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:11:41.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf59ffb700 1 -- 172.21.15.121:0/4222707683 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70052040 con 0x15d2810 2023-11-15T03:11:41.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf59ffb700 1 -- 172.21.15.121:0/4222707683 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7faf70049900 con 0x15d2810 2023-11-15T03:11:41.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf8c3d5740 1 -- 172.21.15.121:0/4222707683 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf7400fce0 con 0x15d2810 2023-11-15T03:11:41.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf59ffb700 1 -- 172.21.15.121:0/4222707683 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=355) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf70049ac0 con 0x15d2810 2023-11-15T03:11:41.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf8c3d5740 10 client.11055 fetch_fsmap learned FSMap version 355 2023-11-15T03:11:41.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf8c3d5740 1 -- 172.21.15.121:0/4222707683 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=355,monmap=2+}) v3 -- 0x15b76e0 con 0x15d2810 2023-11-15T03:11:41.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf59ffb700 1 -- 172.21.15.121:0/4222707683 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7400fce0 con 0x15d2810 2023-11-15T03:11:41.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf59ffb700 1 -- 172.21.15.121:0/4222707683 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 355) v1 ==== 3100+0+0 (secure 0 0 0) 0x7faf7004c6e0 con 0x15d2810 2023-11-15T03:11:41.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf8c3d5740 10 client.11055 fetch_fsmap finished waiting for FSMap version 355 2023-11-15T03:11:41.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf8c3d5740 10 client.11055 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:11:41.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf8c3d5740 20 client.11055 populate_metadata read hostname 'smithi121' 2023-11-15T03:11:41.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/4222707683 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x15ef460 0x15ef840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf8c3d5740 4 client.11055 mds_command: new command op to 10899 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-11-15T03:11:41.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf8c3d5740 1 -- 172.21.15.121:0/4222707683 --> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x13c6720 con 0x15ef460 2023-11-15T03:11:41.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf7b7fe700 1 --2- 172.21.15.121:0/4222707683 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x15ef460 0x15ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf7b7fe700 1 --2- 172.21.15.121:0/4222707683 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x15ef460 0x15ef840 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=4813f0bc89f55c85 server_cookie=d77d82cd043a23c9 in_seq=0 out_seq=0 2023-11-15T03:11:41.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.552+0000 7faf59ffb700 10 client.11055 ms_handle_connect on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:41.572 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf59ffb700 1 -- 172.21.15.121:0/4222707683 <== mds.0 v2:172.21.15.121:6836/3298117008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+294 (crc 0 0 0) 0x13c6720 con 0x15ef460 2023-11-15T03:11:41.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf59ffb700 10 client.11055 handle_command_reply: tid=0 2023-11-15T03:11:41.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf8c3d5740 1 client.11055 shutdown 2023-11-15T03:11:41.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf8c3d5740 1 -- 172.21.15.121:0/4222707683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15d2810 msgr2=0x158d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/4222707683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15d2810 0x158d3e0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 rx=0x7faf700496a0 tx=0x7faf7004f040).stop 2023-11-15T03:11:41.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf8c3d5740 1 -- 172.21.15.121:0/4222707683 shutdown_connections 2023-11-15T03:11:41.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/4222707683 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x15ef460 0x15ef840 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/4222707683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x15d6810 0x158d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/4222707683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x15d2810 0x158d3e0 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf8c3d5740 1 --2- 172.21.15.121:0/4222707683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1589110 0x15f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf8c3d5740 1 -- 172.21.15.121:0/4222707683 >> 172.21.15.121:0/4222707683 conn(0x136f790 msgr2=0x15d7410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:41.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf59ffb700 0 client.11055 ms_handle_reset on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:41.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf8c3d5740 1 -- 172.21.15.121:0/4222707683 shutdown_connections 2023-11-15T03:11:41.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf8c3d5740 1 -- 172.21.15.121:0/4222707683 wait complete. 2023-11-15T03:11:41.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf8c3d5740 20 client.11055 trim_cache size 0 max 16384 2023-11-15T03:11:41.576 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:41.576 INFO:teuthology.orchestra.run.smithi121.stdout:[ 2023-11-15T03:11:41.576 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:41.576 INFO:teuthology.orchestra.run.smithi121.stdout: "damage_type": "dir_frag", 2023-11-15T03:11:41.577 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 1853950885, 2023-11-15T03:11:41.577 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 1099511628278, 2023-11-15T03:11:41.577 INFO:teuthology.orchestra.run.smithi121.stdout: "frag": "*", 2023-11-15T03:11:41.577 INFO:teuthology.orchestra.run.smithi121.stdout: "path": "/dir2" 2023-11-15T03:11:41.577 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-11-15T03:11:41.578 INFO:teuthology.orchestra.run.smithi121.stdout: { 2023-11-15T03:11:41.578 INFO:teuthology.orchestra.run.smithi121.stdout: "damage_type": "backtrace", 2023-11-15T03:11:41.578 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 3099883387, 2023-11-15T03:11:41.578 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 1099511628279, 2023-11-15T03:11:41.578 INFO:teuthology.orchestra.run.smithi121.stdout: "path": "/testdir/hardlink2" 2023-11-15T03:11:41.579 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-11-15T03:11:41.579 INFO:teuthology.orchestra.run.smithi121.stdout:] 2023-11-15T03:11:41.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf80df9700 1 -- 172.21.15.121:0/4096886053 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faf6805a510 msgr2=0x7faf6805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf80df9700 1 --2- 172.21.15.121:0/4096886053 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faf6805a510 0x7faf6805c9d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7faf6c007fd0 tx=0x7faf6c009040).stop 2023-11-15T03:11:41.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf80df9700 1 -- 172.21.15.121:0/4096886053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf7c0acd00 msgr2=0x7faf7c14e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf80df9700 1 --2- 172.21.15.121:0/4096886053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf7c0acd00 0x7faf7c14e0a0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 rx=0x7faf70009e00 tx=0x7faf70014000).stop 2023-11-15T03:11:41.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf80df9700 1 -- 172.21.15.121:0/4096886053 shutdown_connections 2023-11-15T03:11:41.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf80df9700 1 --2- 172.21.15.121:0/4096886053 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faf6805a510 0x7faf6805c9d0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf80df9700 1 --2- 172.21.15.121:0/4096886053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf7c0c5c70 0x7faf7c14e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf80df9700 1 --2- 172.21.15.121:0/4096886053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf7c0acd00 0x7faf7c14e0a0 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf80df9700 1 --2- 172.21.15.121:0/4096886053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf7c066330 0x7faf7c14c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf80df9700 1 -- 172.21.15.121:0/4096886053 >> 172.21.15.121:0/4096886053 conn(0x7faf7c063fc0 msgr2=0x7faf7c14f380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:41.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf80df9700 1 -- 172.21.15.121:0/4096886053 shutdown_connections 2023-11-15T03:11:41.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.556+0000 7faf80df9700 1 -- 172.21.15.121:0/4096886053 wait complete. 2023-11-15T03:11:41.585 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:41.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.736+0000 7f25c7307700 1 Processor -- start 2023-11-15T03:11:41.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.736+0000 7f25c7307700 1 -- start start 2023-11-15T03:11:41.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.740+0000 7f25c7307700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f25c00a9830 0x7f25c00a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.740+0000 7f25c7307700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c00657f0 con 0x7f25c00c30e0 2023-11-15T03:11:41.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.740+0000 7f25c7307700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c0065ad0 con 0x7f25c0151870 2023-11-15T03:11:41.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.740+0000 7f25c7307700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c00aa260 con 0x7f25c00a9830 2023-11-15T03:11:41.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.740+0000 7f25c6b06700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f25c00c30e0 0x7f25c0065f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56312/0 (socket says 172.21.15.121:56312) 2023-11-15T03:11:41.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.740+0000 7f25c6b06700 1 -- 172.21.15.121:0/2033448867 learned_addr learned my addr 172.21.15.121:0/2033448867 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:41.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.740+0000 7f25c5b04700 1 --2- 172.21.15.121:0/2033448867 >> v2:172.21.15.190:3300/0 conn(0x7f25c00a9830 0x7f25c00a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.740+0000 7f25c5303700 1 -- 172.21.15.121:0/2033448867 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427601672 0 0) 0x7f25c00657f0 con 0x7f25c00c30e0 2023-11-15T03:11:41.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.740+0000 7f25c5303700 1 -- 172.21.15.121:0/2033448867 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25ac0033f0 con 0x7f25c00c30e0 2023-11-15T03:11:41.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.740+0000 7f25c5b04700 1 -- 172.21.15.121:0/2033448867 >> v1:172.21.15.190:6789/0 conn(0x7f25c0151870 legacy=0x7f25c0151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.740+0000 7f25c5b04700 1 -- 172.21.15.121:0/2033448867 >> v1:172.21.15.121:6789/0 conn(0x7f25c00c30e0 legacy=0x7f25c0065f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.740+0000 7f25c5b04700 1 -- 172.21.15.121:0/2033448867 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c00aa540 con 0x7f25c00a9830 2023-11-15T03:11:41.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.740+0000 7f25c5b04700 1 --2- 172.21.15.121:0/2033448867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25c00a9830 0x7f25c00a9c70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f25b0009760 tx=0x7f25b0009060).ready entity=mon.1 client_cookie=61a6615c5debe500 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:41.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.744+0000 7f25c5303700 1 -- 172.21.15.121:0/2033448867 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b000f510 con 0x7f25c00a9830 2023-11-15T03:11:41.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.744+0000 7f25c5303700 1 -- 172.21.15.121:0/2033448867 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b000f670 con 0x7f25c00a9830 2023-11-15T03:11:41.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.744+0000 7f25c5303700 1 -- 172.21.15.121:0/2033448867 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b000f9f0 con 0x7f25c00a9830 2023-11-15T03:11:41.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.744+0000 7f25c7307700 1 -- 172.21.15.121:0/2033448867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25c00a9830 msgr2=0x7f25c00a9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.744+0000 7f25c7307700 1 --2- 172.21.15.121:0/2033448867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25c00a9830 0x7f25c00a9c70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f25b0009760 tx=0x7f25b0009060).stop 2023-11-15T03:11:41.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.744+0000 7f25c7307700 1 -- 172.21.15.121:0/2033448867 shutdown_connections 2023-11-15T03:11:41.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.744+0000 7f25c7307700 1 --2- 172.21.15.121:0/2033448867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25c00a9830 0x7f25c00a9c70 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.744+0000 7f25c7307700 1 -- 172.21.15.121:0/2033448867 >> 172.21.15.121:0/2033448867 conn(0x7f25c0062190 msgr2=0x7f25c0062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:41.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.744+0000 7f25c7307700 1 -- 172.21.15.121:0/2033448867 shutdown_connections 2023-11-15T03:11:41.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.744+0000 7f25c7307700 1 -- 172.21.15.121:0/2033448867 wait complete. 2023-11-15T03:11:41.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.748+0000 7f25c7307700 1 Processor -- start 2023-11-15T03:11:41.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.748+0000 7f25c7307700 1 -- start start 2023-11-15T03:11:41.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.748+0000 7f25c7307700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25c00a9830 0x7f25c014b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.748+0000 7f25c7307700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25c00c30e0 0x7f25c0147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.748+0000 7f25c7307700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25c0151870 0x7f25c014bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.748+0000 7f25c7307700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c014bd50 con 0x7f25c0151870 2023-11-15T03:11:41.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.748+0000 7f25c7307700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c014e0b0 con 0x7f25c00a9830 2023-11-15T03:11:41.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.748+0000 7f25c7307700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c014e390 con 0x7f25c00c30e0 2023-11-15T03:11:41.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.748+0000 7f25c6b06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25c0151870 0x7f25c014bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.748+0000 7f25c6b06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25c0151870 0x7f25c014bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51650/0 (socket says 172.21.15.121:51650) 2023-11-15T03:11:41.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.748+0000 7f25c6b06700 1 -- 172.21.15.121:0/3352490102 learned_addr learned my addr 172.21.15.121:0/3352490102 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:41.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.748+0000 7f25c6305700 1 --2- 172.21.15.121:0/3352490102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25c00a9830 0x7f25c014b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.748+0000 7f25c5b04700 1 --2- 172.21.15.121:0/3352490102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25c00c30e0 0x7f25c0147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.748+0000 7f25c6b06700 1 -- 172.21.15.121:0/3352490102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25c00c30e0 msgr2=0x7f25c0147430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.748+0000 7f25c6b06700 1 --2- 172.21.15.121:0/3352490102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25c00c30e0 0x7f25c0147430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.748+0000 7f25c6b06700 1 -- 172.21.15.121:0/3352490102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25c00a9830 msgr2=0x7f25c014b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.748+0000 7f25c6b06700 1 --2- 172.21.15.121:0/3352490102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25c00a9830 0x7f25c014b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.748+0000 7f25c6b06700 1 -- 172.21.15.121:0/3352490102 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b00092a0 con 0x7f25c0151870 2023-11-15T03:11:41.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.752+0000 7f25c6b06700 1 --2- 172.21.15.121:0/3352490102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25c0151870 0x7f25c014bf30 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 rx=0x7f25b8005ad0 tx=0x7f25b8001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:41.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.752+0000 7f25b67fc700 1 -- 172.21.15.121:0/3352490102 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8005d30 con 0x7f25c0151870 2023-11-15T03:11:41.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.752+0000 7f25b67fc700 1 -- 172.21.15.121:0/3352490102 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b8005e90 con 0x7f25c0151870 2023-11-15T03:11:41.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.752+0000 7f25c7307700 1 -- 172.21.15.121:0/3352490102 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c014e670 con 0x7f25c0151870 2023-11-15T03:11:41.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.752+0000 7f25b67fc700 1 -- 172.21.15.121:0/3352490102 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8007320 con 0x7f25c0151870 2023-11-15T03:11:41.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.752+0000 7f25c7307700 1 -- 172.21.15.121:0/3352490102 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c0155190 con 0x7f25c0151870 2023-11-15T03:11:41.763 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.752+0000 7f25c7307700 1 -- 172.21.15.121:0/3352490102 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c0155380 con 0x7f25c0151870 2023-11-15T03:11:41.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.752+0000 7f25b67fc700 1 -- 172.21.15.121:0/3352490102 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f25b80076d0 con 0x7f25c0151870 2023-11-15T03:11:41.764 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.756+0000 7f25b67fc700 1 --2- 172.21.15.121:0/3352490102 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25a405a4e0 0x7f25a405c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:41.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.756+0000 7f25b67fc700 1 -- 172.21.15.121:0/3352490102 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f25b80747b0 con 0x7f25c0151870 2023-11-15T03:11:41.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.756+0000 7f25c6305700 1 --2- 172.21.15.121:0/3352490102 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25a405a4e0 0x7f25a405c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:41.765 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.756+0000 7f25c6305700 1 --2- 172.21.15.121:0/3352490102 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25a405a4e0 0x7f25a405c9a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f25bc003860 tx=0x7f25bc007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:41.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.764+0000 7f25b67fc700 1 -- 172.21.15.121:0/3352490102 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f25b8047e80 con 0x7f25c0151870 2023-11-15T03:11:41.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.960+0000 7f25c7307700 1 -- 172.21.15.121:0/3352490102 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25c00aa820 con 0x7f25c0151870 2023-11-15T03:11:41.971 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.964+0000 7f25b67fc700 1 -- 172.21.15.121:0/3352490102 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 355 v355) v1 ==== 95+0+4514 (secure 0 0 0) 0x7f25b804b4a0 con 0x7f25c0151870 2023-11-15T03:11:41.971 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:41.972 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":355,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/755048854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":755048854},{"type":"v1","addr":"172.21.15.190:6834","nonce":755048854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2360242049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2360242049},{"type":"v1","addr":"172.21.15.190:6835","nonce":2360242049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2780153683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2780153683},{"type":"v1","addr":"172.21.15.121:6835","nonce":2780153683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345}],"filesystems":[{"mdsmap":{"epoch":355,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:11:21.864101+0000","modified":"2023-11-15T03:11:36.256557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10899":{"gid":10899,"name":"a","rank":0,"incarnation":352,"state":"up:active","state_seq":4,"addr":"172.21.15.121:6837/3298117008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3298117008},{"type":"v1","addr":"172.21.15.121:6837","nonce":3298117008}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-15T03:11:41.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.968+0000 7f25c7307700 1 -- 172.21.15.121:0/3352490102 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25a405a4e0 msgr2=0x7f25a405c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.968+0000 7f25c7307700 1 --2- 172.21.15.121:0/3352490102 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25a405a4e0 0x7f25a405c9a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f25bc003860 tx=0x7f25bc007d40).stop 2023-11-15T03:11:41.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.968+0000 7f25c7307700 1 -- 172.21.15.121:0/3352490102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25c0151870 msgr2=0x7f25c014bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:41.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.968+0000 7f25c7307700 1 --2- 172.21.15.121:0/3352490102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25c0151870 0x7f25c014bf30 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 rx=0x7f25b8005ad0 tx=0x7f25b8001650).stop 2023-11-15T03:11:41.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.968+0000 7f25c7307700 1 -- 172.21.15.121:0/3352490102 shutdown_connections 2023-11-15T03:11:41.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.968+0000 7f25c7307700 1 --2- 172.21.15.121:0/3352490102 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f25a405a4e0 0x7f25a405c9a0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.968+0000 7f25c7307700 1 --2- 172.21.15.121:0/3352490102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25c0151870 0x7f25c014bf30 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.968+0000 7f25c7307700 1 --2- 172.21.15.121:0/3352490102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25c00c30e0 0x7f25c0147430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.968+0000 7f25c7307700 1 --2- 172.21.15.121:0/3352490102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25c00a9830 0x7f25c014b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:41.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.968+0000 7f25c7307700 1 -- 172.21.15.121:0/3352490102 >> 172.21.15.121:0/3352490102 conn(0x7f25c0062190 msgr2=0x7f25c0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:41.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.968+0000 7f25c7307700 1 -- 172.21.15.121:0/3352490102 shutdown_connections 2023-11-15T03:11:41.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:41.968+0000 7f25c7307700 1 -- 172.21.15.121:0/3352490102 wait complete. 2023-11-15T03:11:41.978 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 355 2023-11-15T03:11:41.997 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 1853950885 2023-11-15T03:11:42.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.156+0000 7fc03c0db700 1 Processor -- start 2023-11-15T03:11:42.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.156+0000 7fc03c0db700 1 -- start start 2023-11-15T03:11:42.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.156+0000 7fc03c0db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc034150b30 0x7fc034069210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.156+0000 7fc03c0db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc034155370 0x7fc034094d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.156+0000 7fc03c0db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc034154ee0 con 0x7fc034150f10 2023-11-15T03:11:42.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.156+0000 7fc03c0db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc034155820 con 0x7fc034155370 2023-11-15T03:11:42.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.156+0000 7fc03c0db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0340a7140 con 0x7fc034150b30 2023-11-15T03:11:42.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.156+0000 7fc03a678700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc034155370 0x7fc034094d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.156+0000 7fc03a678700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc034155370 0x7fc034094d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51652/0 (socket says 172.21.15.121:51652) 2023-11-15T03:11:42.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.156+0000 7fc03a678700 1 -- 172.21.15.121:0/4045683981 learned_addr learned my addr 172.21.15.121:0/4045683981 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:42.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.156+0000 7fc039676700 1 --1- 172.21.15.121:0/4045683981 >> v1:172.21.15.121:6789/0 conn(0x7fc034150f10 0x7fc0340bda40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56326/0 (socket says 172.21.15.121:56326) 2023-11-15T03:11:42.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.156+0000 7fc039e77700 1 --2- 172.21.15.121:0/4045683981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc034150b30 0x7fc034069210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.156+0000 7fc03a678700 1 -- 172.21.15.121:0/4045683981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc034150b30 msgr2=0x7fc034069210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.156+0000 7fc03a678700 1 --2- 172.21.15.121:0/4045683981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc034150b30 0x7fc034069210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.156+0000 7fc03a678700 1 -- 172.21.15.121:0/4045683981 >> v1:172.21.15.121:6789/0 conn(0x7fc034150f10 legacy=0x7fc0340bda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.156+0000 7fc03a678700 1 -- 172.21.15.121:0/4045683981 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0340be320 con 0x7fc034155370 2023-11-15T03:11:42.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.156+0000 7fc039e77700 1 --2- 172.21.15.121:0/4045683981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc034150b30 0x7fc034069210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:42.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.156+0000 7fc038e75700 1 -- 172.21.15.121:0/4045683981 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1057914680 0 0) 0x7fc034154ee0 con 0x7fc034150f10 2023-11-15T03:11:42.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.160+0000 7fc03a678700 1 --2- 172.21.15.121:0/4045683981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc034155370 0x7fc034094d40 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 rx=0x7fc02c009f20 tx=0x7fc02c009840).ready entity=mon.0 client_cookie=caf9b89ff5ac9ef1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:42.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.160+0000 7fc038e75700 1 -- 172.21.15.121:0/4045683981 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c00e070 con 0x7fc034155370 2023-11-15T03:11:42.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.160+0000 7fc038e75700 1 -- 172.21.15.121:0/4045683981 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c00ba80 con 0x7fc034155370 2023-11-15T03:11:42.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.160+0000 7fc038e75700 1 -- 172.21.15.121:0/4045683981 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c00be00 con 0x7fc034155370 2023-11-15T03:11:42.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.160+0000 7fc03c0db700 1 -- 172.21.15.121:0/4045683981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc034155370 msgr2=0x7fc034094d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.160+0000 7fc03c0db700 1 --2- 172.21.15.121:0/4045683981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc034155370 0x7fc034094d40 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 rx=0x7fc02c009f20 tx=0x7fc02c009840).stop 2023-11-15T03:11:42.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.160+0000 7fc03c0db700 1 -- 172.21.15.121:0/4045683981 shutdown_connections 2023-11-15T03:11:42.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.160+0000 7fc03c0db700 1 --2- 172.21.15.121:0/4045683981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc034155370 0x7fc034094d40 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.160+0000 7fc03c0db700 1 --2- 172.21.15.121:0/4045683981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc034150b30 0x7fc034069210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.160+0000 7fc03c0db700 1 -- 172.21.15.121:0/4045683981 >> 172.21.15.121:0/4045683981 conn(0x7fc0340c7860 msgr2=0x7fc0340c7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:42.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.160+0000 7fc03c0db700 1 -- 172.21.15.121:0/4045683981 shutdown_connections 2023-11-15T03:11:42.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.160+0000 7fc03c0db700 1 -- 172.21.15.121:0/4045683981 wait complete. 2023-11-15T03:11:42.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.160+0000 7fc03c0db700 1 Processor -- start 2023-11-15T03:11:42.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc03c0db700 1 -- start start 2023-11-15T03:11:42.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc03c0db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc034150b30 0x7fc0340a88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc03c0db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc034150f10 0x7fc034151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc03c0db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc034155370 0x7fc034151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc03c0db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0341509f0 con 0x7fc034150f10 2023-11-15T03:11:42.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc03c0db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0340a9360 con 0x7fc034150b30 2023-11-15T03:11:42.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc03c0db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0340a9640 con 0x7fc034155370 2023-11-15T03:11:42.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc039676700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc034150f10 0x7fc034151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc039676700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc034150f10 0x7fc034151370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51666/0 (socket says 172.21.15.121:51666) 2023-11-15T03:11:42.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc039676700 1 -- 172.21.15.121:0/1350144141 learned_addr learned my addr 172.21.15.121:0/1350144141 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:42.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc03a678700 1 --2- 172.21.15.121:0/1350144141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc034155370 0x7fc034151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc039e77700 1 --2- 172.21.15.121:0/1350144141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc034150b30 0x7fc0340a88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc039676700 1 -- 172.21.15.121:0/1350144141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc034155370 msgr2=0x7fc034151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc039676700 1 --2- 172.21.15.121:0/1350144141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc034155370 0x7fc034151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc039676700 1 -- 172.21.15.121:0/1350144141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc034150b30 msgr2=0x7fc0340a88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc039676700 1 --2- 172.21.15.121:0/1350144141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc034150b30 0x7fc0340a88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc039676700 1 -- 172.21.15.121:0/1350144141 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c009ae0 con 0x7fc034150f10 2023-11-15T03:11:42.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc039676700 1 --2- 172.21.15.121:0/1350144141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc034150f10 0x7fc034151370 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 rx=0x7fc02400c4e0 tx=0x7fc024007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:42.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc029ffb700 1 -- 172.21.15.121:0/1350144141 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0240072e0 con 0x7fc034150f10 2023-11-15T03:11:42.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc029ffb700 1 -- 172.21.15.121:0/1350144141 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc024007440 con 0x7fc034150f10 2023-11-15T03:11:42.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc029ffb700 1 -- 172.21.15.121:0/1350144141 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0240058f0 con 0x7fc034150f10 2023-11-15T03:11:42.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc03c0db700 1 -- 172.21.15.121:0/1350144141 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0340a9920 con 0x7fc034150f10 2023-11-15T03:11:42.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.164+0000 7fc03c0db700 1 -- 172.21.15.121:0/1350144141 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0340a9ee0 con 0x7fc034150f10 2023-11-15T03:11:42.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.168+0000 7fc029ffb700 1 -- 172.21.15.121:0/1350144141 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc024005ca0 con 0x7fc034150f10 2023-11-15T03:11:42.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.168+0000 7fc029ffb700 1 --2- 172.21.15.121:0/1350144141 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc01805a540 0x7fc01805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.168+0000 7fc029ffb700 1 -- 172.21.15.121:0/1350144141 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7fc024073fd0 con 0x7fc034150f10 2023-11-15T03:11:42.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc039e77700 1 --2- 172.21.15.121:0/1350144141 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc01805a540 0x7fc01805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc046465740 1 Processor -- start 2023-11-15T03:11:42.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc046465740 1 -- start start 2023-11-15T03:11:42.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc039e77700 1 --2- 172.21.15.121:0/1350144141 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc01805a540 0x7fc01805ca00 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7fc030006fd0 tx=0x7fc030008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:42.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc046465740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ce9650 0x1ce9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc046465740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1cea050 0x1cef370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc046465740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1d3b9d0 0x1d3be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc046465740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c4d570 con 0x1cea050 2023-11-15T03:11:42.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc046465740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c10f20 con 0x1d3b9d0 2023-11-15T03:11:42.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc046465740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1c16db0 con 0x1ce9650 2023-11-15T03:11:42.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc039676700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1cea050 0x1cef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc039676700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1cea050 0x1cef370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51668/0 (socket says 172.21.15.121:51668) 2023-11-15T03:11:42.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc039676700 1 -- 172.21.15.121:0/356602453 learned_addr learned my addr 172.21.15.121:0/356602453 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:42.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc039e77700 1 --2- 172.21.15.121:0/356602453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ce9650 0x1ce9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc03a678700 1 --2- 172.21.15.121:0/356602453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1d3b9d0 0x1d3be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc039676700 1 -- 172.21.15.121:0/356602453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ce9650 msgr2=0x1ce9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc039676700 1 --2- 172.21.15.121:0/356602453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ce9650 0x1ce9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc039676700 1 -- 172.21.15.121:0/356602453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1d3b9d0 msgr2=0x1d3be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc039676700 1 --2- 172.21.15.121:0/356602453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1d3b9d0 0x1d3be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc039676700 1 -- 172.21.15.121:0/356602453 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1c6a850 con 0x1cea050 2023-11-15T03:11:42.190 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc03a678700 1 --2- 172.21.15.121:0/356602453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1d3b9d0 0x1d3be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:11:42.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc039676700 1 --2- 172.21.15.121:0/356602453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1cea050 0x1cef370 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 rx=0x7fc02404b210 tx=0x7fc02400f000).ready entity=mon.0 client_cookie=c5c6ea65f5754ed8 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:42.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc016ffd700 1 -- 172.21.15.121:0/356602453 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc024049810 con 0x1cea050 2023-11-15T03:11:42.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc016ffd700 1 -- 172.21.15.121:0/356602453 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc024047b80 con 0x1cea050 2023-11-15T03:11:42.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc016ffd700 1 -- 172.21.15.121:0/356602453 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02404ab70 con 0x1cea050 2023-11-15T03:11:42.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc046465740 1 -- 172.21.15.121:0/356602453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1cea050 msgr2=0x1cef370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.172+0000 7fc046465740 1 --2- 172.21.15.121:0/356602453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1cea050 0x1cef370 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 rx=0x7fc02404b210 tx=0x7fc02400f000).stop 2023-11-15T03:11:42.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc046465740 1 -- 172.21.15.121:0/356602453 shutdown_connections 2023-11-15T03:11:42.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc046465740 1 --2- 172.21.15.121:0/356602453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1d3b9d0 0x1d3be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc046465740 1 --2- 172.21.15.121:0/356602453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1cea050 0x1cef370 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc046465740 1 --2- 172.21.15.121:0/356602453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ce9650 0x1ce9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc046465740 1 -- 172.21.15.121:0/356602453 >> 172.21.15.121:0/356602453 conn(0x1acf710 msgr2=0x1cedeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:42.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc046465740 1 -- 172.21.15.121:0/356602453 shutdown_connections 2023-11-15T03:11:42.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc046465740 1 -- 172.21.15.121:0/356602453 wait complete. 2023-11-15T03:11:42.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc046465740 1 -- start start 2023-11-15T03:11:42.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc046465740 1 Processor -- start 2023-11-15T03:11:42.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc046465740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ce9650 0x1d19f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc046465740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1cea050 0x1d300f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc046465740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1d3b9d0 0x1ced370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc046465740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cec800 con 0x1d3b9d0 2023-11-15T03:11:42.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc046465740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d1a4c0 con 0x1ce9650 2023-11-15T03:11:42.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc046465740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d2e050 con 0x1cea050 2023-11-15T03:11:42.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc046465740 10 client.0 ll_register_callbacks cb 0x1c676d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:11:42.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc03a678700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1d3b9d0 0x1ced370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc03a678700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1d3b9d0 0x1ced370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51670/0 (socket says 172.21.15.121:51670) 2023-11-15T03:11:42.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc03a678700 1 -- 172.21.15.121:0/1446713446 learned_addr learned my addr 172.21.15.121:0/1446713446 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:42.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc039e77700 1 --2- 172.21.15.121:0/1446713446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ce9650 0x1d19f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc039676700 1 --2- 172.21.15.121:0/1446713446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1cea050 0x1d300f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc03a678700 1 -- 172.21.15.121:0/1446713446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1cea050 msgr2=0x1d300f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc03a678700 1 --2- 172.21.15.121:0/1446713446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1cea050 0x1d300f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc03a678700 1 -- 172.21.15.121:0/1446713446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ce9650 msgr2=0x1d19f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc03a678700 1 --2- 172.21.15.121:0/1446713446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ce9650 0x1d19f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.176+0000 7fc03a678700 1 -- 172.21.15.121:0/1446713446 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0340a9ee0 con 0x1d3b9d0 2023-11-15T03:11:42.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc03a678700 1 --2- 172.21.15.121:0/1446713446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1d3b9d0 0x1ced370 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 rx=0x7fc02c010000 tx=0x7fc02c009840).ready entity=mon.0 client_cookie=c6a441e8f938a499 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:42.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc00ffff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:11:42.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc00ffff700 1 -- 172.21.15.121:0/1446713446 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c013880 con 0x1d3b9d0 2023-11-15T03:11:42.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc00ffff700 1 -- 172.21.15.121:0/1446713446 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7fc02c0139e0 con 0x1d3b9d0 2023-11-15T03:11:42.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc046465740 1 -- 172.21.15.121:0/1446713446 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1c672c0 con 0x1d3b9d0 2023-11-15T03:11:42.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc00ffff700 1 -- 172.21.15.121:0/1446713446 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=355) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc02c013db0 con 0x1d3b9d0 2023-11-15T03:11:42.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc046465740 10 client.11076 fetch_fsmap learned FSMap version 355 2023-11-15T03:11:42.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc046465740 1 -- 172.21.15.121:0/1446713446 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=355,monmap=2+}) v3 -- 0x1c65a80 con 0x1d3b9d0 2023-11-15T03:11:42.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc00ffff700 1 -- 172.21.15.121:0/1446713446 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1c672c0 con 0x1d3b9d0 2023-11-15T03:11:42.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc00ffff700 1 -- 172.21.15.121:0/1446713446 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 355) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc02c027350 con 0x1d3b9d0 2023-11-15T03:11:42.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc046465740 10 client.11076 fetch_fsmap finished waiting for FSMap version 355 2023-11-15T03:11:42.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc046465740 10 client.11076 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:11:42.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc046465740 20 client.11076 populate_metadata read hostname 'smithi121' 2023-11-15T03:11:42.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc046465740 1 --2- 172.21.15.121:0/1446713446 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1d455c0 0x1d459a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc046465740 4 client.11076 mds_command: new command op to 10899 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:11:42.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc046465740 1 -- 172.21.15.121:0/1446713446 --> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1c5d810 con 0x1d455c0 2023-11-15T03:11:42.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc039e77700 1 --2- 172.21.15.121:0/1446713446 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1d455c0 0x1d459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc039e77700 1 --2- 172.21.15.121:0/1446713446 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1d455c0 0x1d459a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=dc94c6256469ea17 server_cookie=a43a568ad328992e in_seq=0 out_seq=0 2023-11-15T03:11:42.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc00ffff700 10 client.11076 ms_handle_connect on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:42.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc00ffff700 1 -- 172.21.15.121:0/1446713446 <== mds.0 v2:172.21.15.121:6836/3298117008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x1c5d810 con 0x1d455c0 2023-11-15T03:11:42.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc00ffff700 10 client.11076 handle_command_reply: tid=0 2023-11-15T03:11:42.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc046465740 1 client.11076 shutdown 2023-11-15T03:11:42.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc046465740 1 -- 172.21.15.121:0/1446713446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1d3b9d0 msgr2=0x1ced370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc046465740 1 --2- 172.21.15.121:0/1446713446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1d3b9d0 0x1ced370 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 rx=0x7fc02c010000 tx=0x7fc02c009840).stop 2023-11-15T03:11:42.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc046465740 1 -- 172.21.15.121:0/1446713446 shutdown_connections 2023-11-15T03:11:42.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc046465740 1 --2- 172.21.15.121:0/1446713446 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1d455c0 0x1d459a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc046465740 1 --2- 172.21.15.121:0/1446713446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1d3b9d0 0x1ced370 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc00ffff700 0 client.11076 ms_handle_reset on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:42.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc046465740 1 --2- 172.21.15.121:0/1446713446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1cea050 0x1d300f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc046465740 1 --2- 172.21.15.121:0/1446713446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ce9650 0x1d19f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.180+0000 7fc046465740 1 -- 172.21.15.121:0/1446713446 >> 172.21.15.121:0/1446713446 conn(0x1acf710 msgr2=0x1cea4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:42.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.184+0000 7fc046465740 1 -- 172.21.15.121:0/1446713446 shutdown_connections 2023-11-15T03:11:42.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.184+0000 7fc046465740 1 -- 172.21.15.121:0/1446713446 wait complete. 2023-11-15T03:11:42.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.184+0000 7fc046465740 20 client.11076 trim_cache size 0 max 16384 2023-11-15T03:11:42.212 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.196+0000 7fc046465740 1 Processor -- start 2023-11-15T03:11:42.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.196+0000 7fc046465740 1 -- start start 2023-11-15T03:11:42.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.196+0000 7fc046465740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ce9650 0x1d19680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.196+0000 7fc046465740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1cea050 0x1d36520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.196+0000 7fc046465740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1d3b9d0 0x1d36a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.196+0000 7fc046465740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cb3190 con 0x1cea050 2023-11-15T03:11:42.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.196+0000 7fc046465740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d31fb0 con 0x1ce9650 2023-11-15T03:11:42.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.196+0000 7fc046465740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ced800 con 0x1d3b9d0 2023-11-15T03:11:42.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.196+0000 7fc039676700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1cea050 0x1d36520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.196+0000 7fc039676700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1cea050 0x1d36520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51678/0 (socket says 172.21.15.121:51678) 2023-11-15T03:11:42.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.196+0000 7fc039676700 1 -- 172.21.15.121:0/4150363454 learned_addr learned my addr 172.21.15.121:0/4150363454 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:42.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.196+0000 7fc03a678700 1 --2- 172.21.15.121:0/4150363454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1d3b9d0 0x1d36a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.196+0000 7fc039e77700 1 --2- 172.21.15.121:0/4150363454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ce9650 0x1d19680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc039676700 1 -- 172.21.15.121:0/4150363454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1d3b9d0 msgr2=0x1d36a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc039676700 1 --2- 172.21.15.121:0/4150363454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1d3b9d0 0x1d36a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc039676700 1 -- 172.21.15.121:0/4150363454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ce9650 msgr2=0x1d19680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc039676700 1 --2- 172.21.15.121:0/4150363454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ce9650 0x1d19680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc039676700 1 -- 172.21.15.121:0/4150363454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0341520b0 con 0x1cea050 2023-11-15T03:11:42.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc039676700 1 --2- 172.21.15.121:0/4150363454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1cea050 0x1d36520 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 rx=0x7fc02404b350 tx=0x7fc024054000).ready entity=mon.0 client_cookie=8403652565bf7ea6 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:42.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc0177fe700 1 -- 172.21.15.121:0/4150363454 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc024047cf0 con 0x1cea050 2023-11-15T03:11:42.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc0177fe700 1 -- 172.21.15.121:0/4150363454 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02404f070 con 0x1cea050 2023-11-15T03:11:42.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc0177fe700 1 -- 172.21.15.121:0/4150363454 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc024049810 con 0x1cea050 2023-11-15T03:11:42.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc046465740 1 -- 172.21.15.121:0/4150363454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1cea050 msgr2=0x1d36520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc046465740 1 --2- 172.21.15.121:0/4150363454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1cea050 0x1d36520 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 rx=0x7fc02404b350 tx=0x7fc024054000).stop 2023-11-15T03:11:42.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc046465740 1 -- 172.21.15.121:0/4150363454 shutdown_connections 2023-11-15T03:11:42.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc046465740 1 --2- 172.21.15.121:0/4150363454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1d3b9d0 0x1d36a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc046465740 1 --2- 172.21.15.121:0/4150363454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1cea050 0x1d36520 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc046465740 1 --2- 172.21.15.121:0/4150363454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ce9650 0x1d19680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc046465740 1 -- 172.21.15.121:0/4150363454 >> 172.21.15.121:0/4150363454 conn(0x1acf710 msgr2=0x1cee5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:42.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc046465740 1 -- 172.21.15.121:0/4150363454 shutdown_connections 2023-11-15T03:11:42.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc046465740 1 -- 172.21.15.121:0/4150363454 wait complete. 2023-11-15T03:11:42.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc046465740 1 -- start start 2023-11-15T03:11:42.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc046465740 1 Processor -- start 2023-11-15T03:11:42.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc046465740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ce9650 0x1da0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc046465740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1d3b9d0 0x1d36520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc046465740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1d3b590 0x1d36a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc046465740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d1b520 con 0x1d3b9d0 2023-11-15T03:11:42.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc046465740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d36f00 con 0x1ce9650 2023-11-15T03:11:42.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.200+0000 7fc046465740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1da1430 con 0x1d3b590 2023-11-15T03:11:42.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc046465740 10 client.0 ll_register_callbacks cb 0x7fc02c003310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:11:42.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc039676700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1d3b9d0 0x1d36520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc039676700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1d3b9d0 0x1d36520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51682/0 (socket says 172.21.15.121:51682) 2023-11-15T03:11:42.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc039676700 1 -- 172.21.15.121:0/2404608041 learned_addr learned my addr 172.21.15.121:0/2404608041 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:42.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc03a678700 1 --2- 172.21.15.121:0/2404608041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1d3b590 0x1d36a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc039e77700 1 --2- 172.21.15.121:0/2404608041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ce9650 0x1da0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc039676700 1 -- 172.21.15.121:0/2404608041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1d3b590 msgr2=0x1d36a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc039676700 1 --2- 172.21.15.121:0/2404608041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1d3b590 0x1d36a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc039676700 1 -- 172.21.15.121:0/2404608041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ce9650 msgr2=0x1da0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc039676700 1 --2- 172.21.15.121:0/2404608041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ce9650 0x1da0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc039676700 1 -- 172.21.15.121:0/2404608041 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1c6a850 con 0x1d3b9d0 2023-11-15T03:11:42.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc039676700 1 --2- 172.21.15.121:0/2404608041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1d3b9d0 0x1d36520 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 rx=0x7fc024000fb0 tx=0x7fc024061000).ready entity=mon.0 client_cookie=df705ffd179ded76 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:42.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc00ffff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:11:42.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc00ffff700 1 -- 172.21.15.121:0/2404608041 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc024001080 con 0x1d3b9d0 2023-11-15T03:11:42.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc046465740 1 -- 172.21.15.121:0/2404608041 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1d35aa0 con 0x1d3b9d0 2023-11-15T03:11:42.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc00ffff700 1 -- 172.21.15.121:0/2404608041 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7fc024047cf0 con 0x1d3b9d0 2023-11-15T03:11:42.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc00ffff700 1 -- 172.21.15.121:0/2404608041 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=355) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc02404a960 con 0x1d3b9d0 2023-11-15T03:11:42.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc046465740 10 client.11082 fetch_fsmap learned FSMap version 355 2023-11-15T03:11:42.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc046465740 1 -- 172.21.15.121:0/2404608041 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=355,monmap=2+}) v3 -- 0x1c657d0 con 0x1d3b9d0 2023-11-15T03:11:42.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc00ffff700 1 -- 172.21.15.121:0/2404608041 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1d35aa0 con 0x1d3b9d0 2023-11-15T03:11:42.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc00ffff700 1 -- 172.21.15.121:0/2404608041 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 355) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc02404a3d0 con 0x1d3b9d0 2023-11-15T03:11:42.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc046465740 10 client.11082 fetch_fsmap finished waiting for FSMap version 355 2023-11-15T03:11:42.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc046465740 10 client.11082 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:11:42.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc046465740 20 client.11082 populate_metadata read hostname 'smithi121' 2023-11-15T03:11:42.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc046465740 1 --2- 172.21.15.121:0/2404608041 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1d17fb0 0x1d18390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc046465740 4 client.11082 mds_command: new command op to 10899 tid=0[{"prefix": "damage rm", "damage_id": 1853950885}] 2023-11-15T03:11:42.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc046465740 1 -- 172.21.15.121:0/2404608041 --> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] -- command(tid 0: {"prefix": "damage rm", "damage_id": 1853950885}) v1 -- 0x7fc024047b80 con 0x1d17fb0 2023-11-15T03:11:42.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc039e77700 1 --2- 172.21.15.121:0/2404608041 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1d17fb0 0x1d18390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc00ffff700 10 client.11082 ms_handle_connect on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:42.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.204+0000 7fc039e77700 1 --2- 172.21.15.121:0/2404608041 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1d17fb0 0x1d18390 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ccbdc99acd09fa4f server_cookie=f39006db9b2819eb in_seq=0 out_seq=0 2023-11-15T03:11:42.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc00ffff700 1 -- 172.21.15.121:0/2404608041 <== mds.0 v2:172.21.15.121:6836/3298117008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x7fc024047b80 con 0x1d17fb0 2023-11-15T03:11:42.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc00ffff700 10 client.11082 handle_command_reply: tid=0 2023-11-15T03:11:42.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc046465740 1 client.11082 shutdown 2023-11-15T03:11:42.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc046465740 1 -- 172.21.15.121:0/2404608041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1d3b9d0 msgr2=0x1d36520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc046465740 1 --2- 172.21.15.121:0/2404608041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1d3b9d0 0x1d36520 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 rx=0x7fc024000fb0 tx=0x7fc024061000).stop 2023-11-15T03:11:42.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc046465740 1 -- 172.21.15.121:0/2404608041 shutdown_connections 2023-11-15T03:11:42.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc046465740 1 --2- 172.21.15.121:0/2404608041 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1d17fb0 0x1d18390 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc046465740 1 --2- 172.21.15.121:0/2404608041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1d3b590 0x1d36a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc046465740 1 --2- 172.21.15.121:0/2404608041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1d3b9d0 0x1d36520 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc046465740 1 --2- 172.21.15.121:0/2404608041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ce9650 0x1da0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc00ffff700 0 client.11082 ms_handle_reset on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:42.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc046465740 1 -- 172.21.15.121:0/2404608041 >> 172.21.15.121:0/2404608041 conn(0x1acf710 msgr2=0x1d2d760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:42.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc046465740 1 -- 172.21.15.121:0/2404608041 shutdown_connections 2023-11-15T03:11:42.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc046465740 1 -- 172.21.15.121:0/2404608041 wait complete. 2023-11-15T03:11:42.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc046465740 20 client.11082 trim_cache size 0 max 16384 2023-11-15T03:11:42.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc03c0db700 1 -- 172.21.15.121:0/1350144141 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc01805a540 msgr2=0x7fc01805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc03c0db700 1 --2- 172.21.15.121:0/1350144141 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc01805a540 0x7fc01805ca00 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7fc030006fd0 tx=0x7fc030008040).stop 2023-11-15T03:11:42.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc03c0db700 1 -- 172.21.15.121:0/1350144141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc034150f10 msgr2=0x7fc034151370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc03c0db700 1 --2- 172.21.15.121:0/1350144141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc034150f10 0x7fc034151370 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 rx=0x7fc02400c4e0 tx=0x7fc024007c60).stop 2023-11-15T03:11:42.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc03c0db700 1 -- 172.21.15.121:0/1350144141 shutdown_connections 2023-11-15T03:11:42.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc03c0db700 1 --2- 172.21.15.121:0/1350144141 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc01805a540 0x7fc01805ca00 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc03c0db700 1 --2- 172.21.15.121:0/1350144141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc034155370 0x7fc034151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc03c0db700 1 --2- 172.21.15.121:0/1350144141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc034150f10 0x7fc034151370 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc03c0db700 1 --2- 172.21.15.121:0/1350144141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc034150b30 0x7fc0340a88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc03c0db700 1 -- 172.21.15.121:0/1350144141 >> 172.21.15.121:0/1350144141 conn(0x7fc0340c7860 msgr2=0x7fc0340a45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:42.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc03c0db700 1 -- 172.21.15.121:0/1350144141 shutdown_connections 2023-11-15T03:11:42.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.208+0000 7fc03c0db700 1 -- 172.21.15.121:0/1350144141 wait complete. 2023-11-15T03:11:42.235 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:42.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.376+0000 7f1a6b59e700 1 Processor -- start 2023-11-15T03:11:42.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.376+0000 7f1a6b59e700 1 -- start start 2023-11-15T03:11:42.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.376+0000 7f1a6b59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a6c0c3280 0x7f1a6c0c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.376+0000 7f1a6b59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1a6c152140 0x7f1a6c1525c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.376+0000 7f1a6b59e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6c0c1d50 con 0x7f1a6c15be30 2023-11-15T03:11:42.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.376+0000 7f1a6b59e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6c0c1860 con 0x7f1a6c152140 2023-11-15T03:11:42.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.376+0000 7f1a6b59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c15cb80 con 0x7f1a6c0c3280 2023-11-15T03:11:42.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.376+0000 7f1a6a59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a6c0c3280 0x7f1a6c0c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.388 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.376+0000 7f1a61d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1a6c152140 0x7f1a6c1525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a6a59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a6c0c3280 0x7f1a6c0c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:59678/0 (socket says 172.21.15.121:59678) 2023-11-15T03:11:42.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a6a59c700 1 -- 172.21.15.121:0/1310628054 learned_addr learned my addr 172.21.15.121:0/1310628054 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:42.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a69d9b700 1 -- 172.21.15.121:0/1310628054 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925687245 0 0) 0x7f1a6c0c1d50 con 0x7f1a6c15be30 2023-11-15T03:11:42.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a69d9b700 1 -- 172.21.15.121:0/1310628054 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a580033f0 con 0x7f1a6c15be30 2023-11-15T03:11:42.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a6a59c700 1 -- 172.21.15.121:0/1310628054 >> v2:172.21.15.190:3300/0 conn(0x7f1a6c152140 msgr2=0x7f1a6c1525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a6a59c700 1 --2- 172.21.15.121:0/1310628054 >> v2:172.21.15.190:3300/0 conn(0x7f1a6c152140 0x7f1a6c1525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a6a59c700 1 -- 172.21.15.121:0/1310628054 >> v1:172.21.15.190:6789/0 conn(0x7f1a6c15be30 legacy=0x7f1a6c15c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a6a59c700 1 -- 172.21.15.121:0/1310628054 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6c0a9240 con 0x7f1a6c0c3280 2023-11-15T03:11:42.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a6a59c700 1 --2- 172.21.15.121:0/1310628054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a6c0c3280 0x7f1a6c0c3660 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f1a5c009f20 tx=0x7f1a5c009840).ready entity=mon.2 client_cookie=9b1e9517fd24cb98 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:42.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a69d9b700 1 -- 172.21.15.121:0/1310628054 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c00e070 con 0x7f1a6c0c3280 2023-11-15T03:11:42.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a69d9b700 1 -- 172.21.15.121:0/1310628054 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5c00b970 con 0x7f1a6c0c3280 2023-11-15T03:11:42.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a69d9b700 1 -- 172.21.15.121:0/1310628054 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c00bcf0 con 0x7f1a6c0c3280 2023-11-15T03:11:42.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a6b59e700 1 -- 172.21.15.121:0/1310628054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a6c0c3280 msgr2=0x7f1a6c0c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a6b59e700 1 --2- 172.21.15.121:0/1310628054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a6c0c3280 0x7f1a6c0c3660 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f1a5c009f20 tx=0x7f1a5c009840).stop 2023-11-15T03:11:42.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a6b59e700 1 -- 172.21.15.121:0/1310628054 shutdown_connections 2023-11-15T03:11:42.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a6b59e700 1 --2- 172.21.15.121:0/1310628054 >> v2:172.21.15.190:3300/0 conn(0x7f1a6c152140 0x7f1a6c1525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a6b59e700 1 --2- 172.21.15.121:0/1310628054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a6c0c3280 0x7f1a6c0c3660 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a6b59e700 1 -- 172.21.15.121:0/1310628054 >> 172.21.15.121:0/1310628054 conn(0x7f1a6c062190 msgr2=0x7f1a6c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:42.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a6b59e700 1 -- 172.21.15.121:0/1310628054 shutdown_connections 2023-11-15T03:11:42.394 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.380+0000 7f1a6b59e700 1 -- 172.21.15.121:0/1310628054 wait complete. 2023-11-15T03:11:42.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.384+0000 7f1a6b59e700 1 Processor -- start 2023-11-15T03:11:42.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.384+0000 7f1a6b59e700 1 -- start start 2023-11-15T03:11:42.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a6b59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a6c0c3280 0x7f1a6c0c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a6b59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a6c152140 0x7f1a6c154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a6b59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a6c15be30 0x7f1a6c1597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a6b59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c1525c0 con 0x7f1a6c0c3280 2023-11-15T03:11:42.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a6b59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0c74b0 con 0x7f1a6c15be30 2023-11-15T03:11:42.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a6b59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c156e40 con 0x7f1a6c152140 2023-11-15T03:11:42.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a6a59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a6c0c3280 0x7f1a6c0c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a6a59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a6c0c3280 0x7f1a6c0c6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51688/0 (socket says 172.21.15.121:51688) 2023-11-15T03:11:42.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a6a59c700 1 -- 172.21.15.121:0/3096767849 learned_addr learned my addr 172.21.15.121:0/3096767849 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:42.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a61d9b700 1 --2- 172.21.15.121:0/3096767849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a6c152140 0x7f1a6c154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a6ad9d700 1 --2- 172.21.15.121:0/3096767849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a6c15be30 0x7f1a6c1597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a6a59c700 1 -- 172.21.15.121:0/3096767849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a6c152140 msgr2=0x7f1a6c154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a6a59c700 1 --2- 172.21.15.121:0/3096767849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a6c152140 0x7f1a6c154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a6a59c700 1 -- 172.21.15.121:0/3096767849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a6c15be30 msgr2=0x7f1a6c1597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a6a59c700 1 --2- 172.21.15.121:0/3096767849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a6c15be30 0x7f1a6c1597e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a6a59c700 1 -- 172.21.15.121:0/3096767849 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c009ae0 con 0x7f1a6c0c3280 2023-11-15T03:11:42.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a6ad9d700 1 --2- 172.21.15.121:0/3096767849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a6c15be30 0x7f1a6c1597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:42.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a6a59c700 1 --2- 172.21.15.121:0/3096767849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a6c0c3280 0x7f1a6c0c6fc0 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 rx=0x7f1a5c009880 tx=0x7f1a6c0c1020).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:42.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a62ffd700 1 -- 172.21.15.121:0/3096767849 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c0092e0 con 0x7f1a6c0c3280 2023-11-15T03:11:42.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a62ffd700 1 -- 172.21.15.121:0/3096767849 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5c013c10 con 0x7f1a6c0c3280 2023-11-15T03:11:42.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a6b59e700 1 -- 172.21.15.121:0/3096767849 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6c157120 con 0x7f1a6c0c3280 2023-11-15T03:11:42.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a62ffd700 1 -- 172.21.15.121:0/3096767849 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c01e470 con 0x7f1a6c0c3280 2023-11-15T03:11:42.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.388+0000 7f1a6b59e700 1 -- 172.21.15.121:0/3096767849 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6c1576e0 con 0x7f1a6c0c3280 2023-11-15T03:11:42.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.392+0000 7f1a6b59e700 1 -- 172.21.15.121:0/3096767849 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6c159060 con 0x7f1a6c0c3280 2023-11-15T03:11:42.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.392+0000 7f1a62ffd700 1 -- 172.21.15.121:0/3096767849 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1a5c003040 con 0x7f1a6c0c3280 2023-11-15T03:11:42.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.396+0000 7f1a62ffd700 1 --2- 172.21.15.121:0/3096767849 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1a4c05a510 0x7f1a4c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.396+0000 7f1a62ffd700 1 -- 172.21.15.121:0/3096767849 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f1a5c075b70 con 0x7f1a6c0c3280 2023-11-15T03:11:42.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.396+0000 7f1a61d9b700 1 --2- 172.21.15.121:0/3096767849 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1a4c05a510 0x7f1a4c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.396+0000 7f1a61d9b700 1 --2- 172.21.15.121:0/3096767849 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1a4c05a510 0x7f1a4c05c9d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f1a6c156910 tx=0x7f1a54009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:42.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.400+0000 7f1a62ffd700 1 -- 172.21.15.121:0/3096767849 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1a5c049290 con 0x7f1a6c0c3280 2023-11-15T03:11:42.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.596+0000 7f1a6b59e700 1 -- 172.21.15.121:0/3096767849 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a6c06bcf0 con 0x7f1a6c0c3280 2023-11-15T03:11:42.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.596+0000 7f1a62ffd700 1 -- 172.21.15.121:0/3096767849 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 355 v355) v1 ==== 95+0+4514 (secure 0 0 0) 0x7f1a5c023030 con 0x7f1a6c0c3280 2023-11-15T03:11:42.606 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:42.606 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":355,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/755048854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":755048854},{"type":"v1","addr":"172.21.15.190:6834","nonce":755048854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2360242049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2360242049},{"type":"v1","addr":"172.21.15.190:6835","nonce":2360242049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2780153683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2780153683},{"type":"v1","addr":"172.21.15.121:6835","nonce":2780153683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345}],"filesystems":[{"mdsmap":{"epoch":355,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:11:21.864101+0000","modified":"2023-11-15T03:11:36.256557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10899":{"gid":10899,"name":"a","rank":0,"incarnation":352,"state":"up:active","state_seq":4,"addr":"172.21.15.121:6837/3298117008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3298117008},{"type":"v1","addr":"172.21.15.121:6837","nonce":3298117008}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-15T03:11:42.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.600+0000 7f1a6b59e700 1 -- 172.21.15.121:0/3096767849 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1a4c05a510 msgr2=0x7f1a4c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.600+0000 7f1a6b59e700 1 --2- 172.21.15.121:0/3096767849 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1a4c05a510 0x7f1a4c05c9d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f1a6c156910 tx=0x7f1a54009060).stop 2023-11-15T03:11:42.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.600+0000 7f1a6b59e700 1 -- 172.21.15.121:0/3096767849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a6c0c3280 msgr2=0x7f1a6c0c6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.600+0000 7f1a6b59e700 1 --2- 172.21.15.121:0/3096767849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a6c0c3280 0x7f1a6c0c6fc0 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 rx=0x7f1a5c009880 tx=0x7f1a6c0c1020).stop 2023-11-15T03:11:42.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.600+0000 7f1a6b59e700 1 -- 172.21.15.121:0/3096767849 shutdown_connections 2023-11-15T03:11:42.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.600+0000 7f1a6b59e700 1 --2- 172.21.15.121:0/3096767849 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1a4c05a510 0x7f1a4c05c9d0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.600+0000 7f1a6b59e700 1 --2- 172.21.15.121:0/3096767849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a6c15be30 0x7f1a6c1597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.600+0000 7f1a6b59e700 1 --2- 172.21.15.121:0/3096767849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a6c152140 0x7f1a6c154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.600+0000 7f1a6b59e700 1 --2- 172.21.15.121:0/3096767849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a6c0c3280 0x7f1a6c0c6fc0 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.600+0000 7f1a6b59e700 1 -- 172.21.15.121:0/3096767849 >> 172.21.15.121:0/3096767849 conn(0x7f1a6c062190 msgr2=0x7f1a6c151690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:42.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.600+0000 7f1a6b59e700 1 -- 172.21.15.121:0/3096767849 shutdown_connections 2023-11-15T03:11:42.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.600+0000 7f1a6b59e700 1 -- 172.21.15.121:0/3096767849 wait complete. 2023-11-15T03:11:42.613 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 355 2023-11-15T03:11:42.631 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 3099883387 2023-11-15T03:11:42.678 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:11:42.686 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:11:42.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.768+0000 7f9f57fff700 1 Processor -- start 2023-11-15T03:11:42.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.768+0000 7f9f57fff700 1 -- start start 2023-11-15T03:11:42.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f57fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f58067380 0x7f9f5815b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f57fff700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f580657f0 con 0x7f9f5814aff0 2023-11-15T03:11:42.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f57fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f58066650 con 0x7f9f5814ac10 2023-11-15T03:11:42.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f57fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f58065a40 con 0x7f9f58067380 2023-11-15T03:11:42.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f567fc700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9f5814aff0 0x7f9f58066c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56328/0 (socket says 172.21.15.121:56328) 2023-11-15T03:11:42.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f567fc700 1 -- 172.21.15.121:0/300495281 learned_addr learned my addr 172.21.15.121:0/300495281 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:42.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f577fe700 1 --2- 172.21.15.121:0/300495281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f58067380 0x7f9f5815b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 -- 172.21.15.121:0/300495281 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2014144181 0 0) 0x7f9f580657f0 con 0x7f9f5814aff0 2023-11-15T03:11:42.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 -- 172.21.15.121:0/300495281 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f400033f0 con 0x7f9f5814aff0 2023-11-15T03:11:42.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 -- 172.21.15.121:0/300495281 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1381666488 0 0) 0x7f9f58066650 con 0x7f9f5814ac10 2023-11-15T03:11:42.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 -- 172.21.15.121:0/300495281 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f580657f0 con 0x7f9f5814ac10 2023-11-15T03:11:42.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 -- 172.21.15.121:0/300495281 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2340680186 0 0) 0x7f9f400033f0 con 0x7f9f5814aff0 2023-11-15T03:11:42.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 -- 172.21.15.121:0/300495281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f58067380 msgr2=0x7f9f5815b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 --2- 172.21.15.121:0/300495281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f58067380 0x7f9f5815b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 -- 172.21.15.121:0/300495281 >> v1:172.21.15.190:6789/0 conn(0x7f9f5814ac10 legacy=0x7f9f580c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 -- 172.21.15.121:0/300495281 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f58065d20 con 0x7f9f5814aff0 2023-11-15T03:11:42.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 -- 172.21.15.121:0/300495281 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f9f4c001470 con 0x7f9f5814aff0 2023-11-15T03:11:42.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f577fe700 1 --2- 172.21.15.121:0/300495281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f58067380 0x7f9f5815b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:42.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 -- 172.21.15.121:0/300495281 >> v1:172.21.15.121:6789/0 conn(0x7f9f5814aff0 legacy=0x7f9f58066c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 --2- 172.21.15.121:0/300495281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f40003aa0 0x7f9f40003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 --2- 172.21.15.121:0/300495281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f40004690 0x7f9f40004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 --2- 172.21.15.121:0/300495281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f58067380 0x7f9f40005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 -- 172.21.15.121:0/300495281 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f58065a40 con 0x7f9f40003aa0 2023-11-15T03:11:42.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 -- 172.21.15.121:0/300495281 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4000daf0 con 0x7f9f40004690 2023-11-15T03:11:42.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 -- 172.21.15.121:0/300495281 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f40005930 con 0x7f9f58067380 2023-11-15T03:11:42.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f56ffd700 1 --2- 172.21.15.121:0/300495281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f40003aa0 0x7f9f40003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f577fe700 1 --2- 172.21.15.121:0/300495281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f58067380 0x7f9f40005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f567fc700 1 --2- 172.21.15.121:0/300495281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f40004690 0x7f9f40004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f56ffd700 1 -- 172.21.15.121:0/300495281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f58067380 msgr2=0x7f9f40005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f56ffd700 1 --2- 172.21.15.121:0/300495281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f58067380 0x7f9f40005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f56ffd700 1 -- 172.21.15.121:0/300495281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f40004690 msgr2=0x7f9f40004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f56ffd700 1 --2- 172.21.15.121:0/300495281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f40004690 0x7f9f40004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f56ffd700 1 -- 172.21.15.121:0/300495281 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f50009a20 con 0x7f9f40003aa0 2023-11-15T03:11:42.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f567fc700 1 --2- 172.21.15.121:0/300495281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f40004690 0x7f9f40004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:42.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f56ffd700 1 --2- 172.21.15.121:0/300495281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f40003aa0 0x7f9f40003ea0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 rx=0x7f9f48001890 tx=0x7f9f48007bb0).ready entity=mon.0 client_cookie=c31566edd7dea1bf server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:42.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 -- 172.21.15.121:0/300495281 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48007020 con 0x7f9f40003aa0 2023-11-15T03:11:42.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 -- 172.21.15.121:0/300495281 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f48007180 con 0x7f9f40003aa0 2023-11-15T03:11:42.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.772+0000 7f9f55ffb700 1 -- 172.21.15.121:0/300495281 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48007500 con 0x7f9f40003aa0 2023-11-15T03:11:42.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.776+0000 7f9f57fff700 1 -- 172.21.15.121:0/300495281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f40003aa0 msgr2=0x7f9f40003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.776+0000 7f9f57fff700 1 --2- 172.21.15.121:0/300495281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f40003aa0 0x7f9f40003ea0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 rx=0x7f9f48001890 tx=0x7f9f48007bb0).stop 2023-11-15T03:11:42.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.776+0000 7f9f577fe700 1 -- 172.21.15.121:0/300495281 reap_dead start 2023-11-15T03:11:42.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.776+0000 7f9f57fff700 1 -- 172.21.15.121:0/300495281 shutdown_connections 2023-11-15T03:11:42.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.776+0000 7f9f57fff700 1 -- 172.21.15.121:0/300495281 >> 172.21.15.121:0/300495281 conn(0x7f9f58067a60 msgr2=0x7f9f58067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:42.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.776+0000 7f9f57fff700 1 -- 172.21.15.121:0/300495281 shutdown_connections 2023-11-15T03:11:42.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.776+0000 7f9f57fff700 1 -- 172.21.15.121:0/300495281 wait complete. 2023-11-15T03:11:42.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.776+0000 7f9f57fff700 1 Processor -- start 2023-11-15T03:11:42.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.776+0000 7f9f57fff700 1 -- start start 2023-11-15T03:11:42.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f57fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f40003aa0 0x7f9f5813f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f57fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f580c0a00 0x7f9f5815fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f57fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f581601c0 0x7f9f580c4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f57fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5814a360 con 0x7f9f581601c0 2023-11-15T03:11:42.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f57fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5813fbe0 con 0x7f9f40003aa0 2023-11-15T03:11:42.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f57fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f580a9ef0 con 0x7f9f580c0a00 2023-11-15T03:11:42.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f577fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f581601c0 0x7f9f580c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f577fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f581601c0 0x7f9f580c4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51714/0 (socket says 172.21.15.121:51714) 2023-11-15T03:11:42.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f577fe700 1 -- 172.21.15.121:0/133227370 learned_addr learned my addr 172.21.15.121:0/133227370 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:42.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f56ffd700 1 --2- 172.21.15.121:0/133227370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f40003aa0 0x7f9f5813f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f567fc700 1 --2- 172.21.15.121:0/133227370 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f580c0a00 0x7f9f5815fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f577fe700 1 -- 172.21.15.121:0/133227370 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f580c0a00 msgr2=0x7f9f5815fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f577fe700 1 --2- 172.21.15.121:0/133227370 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f580c0a00 0x7f9f5815fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f577fe700 1 -- 172.21.15.121:0/133227370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f40003aa0 msgr2=0x7f9f5813f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f577fe700 1 --2- 172.21.15.121:0/133227370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f40003aa0 0x7f9f5813f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f577fe700 1 -- 172.21.15.121:0/133227370 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f580c0f70 con 0x7f9f581601c0 2023-11-15T03:11:42.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f577fe700 1 --2- 172.21.15.121:0/133227370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f581601c0 0x7f9f580c4340 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 rx=0x7f9f581542b0 tx=0x7f9f5000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:42.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f3effd700 1 -- 172.21.15.121:0/133227370 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5000e070 con 0x7f9f581601c0 2023-11-15T03:11:42.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f3effd700 1 -- 172.21.15.121:0/133227370 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f50011c50 con 0x7f9f581601c0 2023-11-15T03:11:42.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f57fff700 1 -- 172.21.15.121:0/133227370 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f580c1250 con 0x7f9f581601c0 2023-11-15T03:11:42.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f3effd700 1 -- 172.21.15.121:0/133227370 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50016450 con 0x7f9f581601c0 2023-11-15T03:11:42.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.780+0000 7f9f57fff700 1 -- 172.21.15.121:0/133227370 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f580c1810 con 0x7f9f581601c0 2023-11-15T03:11:42.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.784+0000 7f9f3effd700 1 -- 172.21.15.121:0/133227370 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9f5001f3b0 con 0x7f9f581601c0 2023-11-15T03:11:42.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.784+0000 7f9f3effd700 1 --2- 172.21.15.121:0/133227370 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9f4405a510 0x7f9f4405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.784+0000 7f9f3effd700 1 -- 172.21.15.121:0/133227370 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f9f50074ec0 con 0x7f9f581601c0 2023-11-15T03:11:42.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.784+0000 7f9f56ffd700 1 --2- 172.21.15.121:0/133227370 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9f4405a510 0x7f9f4405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.784+0000 7f9f56ffd700 1 --2- 172.21.15.121:0/133227370 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9f4405a510 0x7f9f4405c9d0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f9f48001670 tx=0x7f9f48016000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:42.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f67686740 1 Processor -- start 2023-11-15T03:11:42.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f67686740 1 -- start start 2023-11-15T03:11:42.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f67686740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e83650 0x1e83a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f67686740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e84050 0x1e89370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f67686740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed59d0 0x1ed5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f67686740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1de7570 con 0x1e84050 2023-11-15T03:11:42.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f67686740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1daaf20 con 0x1e83650 2023-11-15T03:11:42.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f67686740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1db0db0 con 0x1ed59d0 2023-11-15T03:11:42.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f567fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e84050 0x1e89370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f567fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e84050 0x1e89370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51728/0 (socket says 172.21.15.121:51728) 2023-11-15T03:11:42.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f567fc700 1 -- 172.21.15.121:0/418867169 learned_addr learned my addr 172.21.15.121:0/418867169 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:42.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f56ffd700 1 --2- 172.21.15.121:0/418867169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e83650 0x1e83a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f577fe700 1 --2- 172.21.15.121:0/418867169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed59d0 0x1ed5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f567fc700 1 -- 172.21.15.121:0/418867169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed59d0 msgr2=0x1ed5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f567fc700 1 --2- 172.21.15.121:0/418867169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed59d0 0x1ed5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f567fc700 1 -- 172.21.15.121:0/418867169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e83650 msgr2=0x1e83a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f567fc700 1 --2- 172.21.15.121:0/418867169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e83650 0x1e83a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f567fc700 1 -- 172.21.15.121:0/418867169 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1e04850 con 0x1e84050 2023-11-15T03:11:42.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f577fe700 1 --2- 172.21.15.121:0/418867169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed59d0 0x1ed5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:42.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f567fc700 1 --2- 172.21.15.121:0/418867169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e84050 0x1e89370 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 rx=0x7f9f4c005910 tx=0x7f9f4c007040).ready entity=mon.0 client_cookie=a926fe68d977be67 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:42.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f37fff700 1 -- 172.21.15.121:0/418867169 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c005e50 con 0x1e84050 2023-11-15T03:11:42.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.788+0000 7f9f37fff700 1 -- 172.21.15.121:0/418867169 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c0118f0 con 0x1e84050 2023-11-15T03:11:42.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.792+0000 7f9f37fff700 1 -- 172.21.15.121:0/418867169 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c011c70 con 0x1e84050 2023-11-15T03:11:42.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.792+0000 7f9f67686740 1 -- 172.21.15.121:0/418867169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e84050 msgr2=0x1e89370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.792+0000 7f9f67686740 1 --2- 172.21.15.121:0/418867169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e84050 0x1e89370 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 rx=0x7f9f4c005910 tx=0x7f9f4c007040).stop 2023-11-15T03:11:42.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.792+0000 7f9f67686740 1 -- 172.21.15.121:0/418867169 shutdown_connections 2023-11-15T03:11:42.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.792+0000 7f9f67686740 1 --2- 172.21.15.121:0/418867169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1ed59d0 0x1ed5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.792+0000 7f9f67686740 1 --2- 172.21.15.121:0/418867169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e84050 0x1e89370 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.792+0000 7f9f67686740 1 --2- 172.21.15.121:0/418867169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e83650 0x1e83a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.792+0000 7f9f67686740 1 -- 172.21.15.121:0/418867169 >> 172.21.15.121:0/418867169 conn(0x1c69710 msgr2=0x1e87eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:42.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.792+0000 7f9f67686740 1 -- 172.21.15.121:0/418867169 shutdown_connections 2023-11-15T03:11:42.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.792+0000 7f9f67686740 1 -- 172.21.15.121:0/418867169 wait complete. 2023-11-15T03:11:42.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.792+0000 7f9f67686740 1 -- start start 2023-11-15T03:11:42.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.792+0000 7f9f67686740 1 Processor -- start 2023-11-15T03:11:42.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f67686740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e83650 0x1eb1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f67686740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e84050 0x1ead220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.816 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f67686740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ed59d0 0x1eb21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f67686740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ead710 con 0x1e84050 2023-11-15T03:11:42.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f67686740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e86ed0 con 0x1ed59d0 2023-11-15T03:11:42.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f67686740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e87c40 con 0x1e83650 2023-11-15T03:11:42.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f67686740 10 client.0 ll_register_callbacks cb 0x1e016d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:11:42.817 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f567fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e84050 0x1ead220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f567fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e84050 0x1ead220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51738/0 (socket says 172.21.15.121:51738) 2023-11-15T03:11:42.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f567fc700 1 -- 172.21.15.121:0/3030399708 learned_addr learned my addr 172.21.15.121:0/3030399708 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:42.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f56ffd700 1 --2- 172.21.15.121:0/3030399708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e83650 0x1eb1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f577fe700 1 --2- 172.21.15.121:0/3030399708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ed59d0 0x1eb21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f567fc700 1 -- 172.21.15.121:0/3030399708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e83650 msgr2=0x1eb1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f567fc700 1 --2- 172.21.15.121:0/3030399708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e83650 0x1eb1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f567fc700 1 -- 172.21.15.121:0/3030399708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ed59d0 msgr2=0x1eb21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f567fc700 1 --2- 172.21.15.121:0/3030399708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ed59d0 0x1eb21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f567fc700 1 -- 172.21.15.121:0/3030399708 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4c00f040 con 0x1e84050 2023-11-15T03:11:42.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f577fe700 1 --2- 172.21.15.121:0/3030399708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ed59d0 0x1eb21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:42.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f567fc700 1 --2- 172.21.15.121:0/3030399708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e84050 0x1ead220 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 rx=0x7f9f4c0058e0 tx=0x7f9f4c015000).ready entity=mon.0 client_cookie=94b3c059f7660c17 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:42.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f357fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:11:42.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f357fa700 1 -- 172.21.15.121:0/3030399708 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0158d0 con 0x1e84050 2023-11-15T03:11:42.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f357fa700 1 -- 172.21.15.121:0/3030399708 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f9f4c015a30 con 0x1e84050 2023-11-15T03:11:42.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f67686740 1 -- 172.21.15.121:0/3030399708 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1e012c0 con 0x1e84050 2023-11-15T03:11:42.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f357fa700 1 -- 172.21.15.121:0/3030399708 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=355) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f4c015e40 con 0x1e84050 2023-11-15T03:11:42.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f67686740 10 client.11097 fetch_fsmap learned FSMap version 355 2023-11-15T03:11:42.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f67686740 1 -- 172.21.15.121:0/3030399708 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=355,monmap=2+}) v3 -- 0x1dffa80 con 0x1e84050 2023-11-15T03:11:42.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f357fa700 1 -- 172.21.15.121:0/3030399708 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1e012c0 con 0x1e84050 2023-11-15T03:11:42.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f357fa700 1 -- 172.21.15.121:0/3030399708 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 355) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9f4c01f440 con 0x1e84050 2023-11-15T03:11:42.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f67686740 10 client.11097 fetch_fsmap finished waiting for FSMap version 355 2023-11-15T03:11:42.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f67686740 10 client.11097 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:11:42.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f67686740 20 client.11097 populate_metadata read hostname 'smithi121' 2023-11-15T03:11:42.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f67686740 1 --2- 172.21.15.121:0/3030399708 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1eb0a10 0x1e85b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f67686740 4 client.11097 mds_command: new command op to 10899 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-15T03:11:42.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.796+0000 7f9f67686740 1 -- 172.21.15.121:0/3030399708 --> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1df7810 con 0x1eb0a10 2023-11-15T03:11:42.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.800+0000 7f9f56ffd700 1 --2- 172.21.15.121:0/3030399708 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1eb0a10 0x1e85b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.800+0000 7f9f357fa700 10 client.11097 ms_handle_connect on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:42.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.800+0000 7f9f56ffd700 1 --2- 172.21.15.121:0/3030399708 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1eb0a10 0x1e85b90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=b034fac78ba7561a server_cookie=bc9586ae0726c6ed in_seq=0 out_seq=0 2023-11-15T03:11:42.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.800+0000 7f9f357fa700 1 -- 172.21.15.121:0/3030399708 <== mds.0 v2:172.21.15.121:6836/3298117008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x1df7810 con 0x1eb0a10 2023-11-15T03:11:42.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.800+0000 7f9f357fa700 10 client.11097 handle_command_reply: tid=0 2023-11-15T03:11:42.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.800+0000 7f9f67686740 1 client.11097 shutdown 2023-11-15T03:11:42.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.800+0000 7f9f67686740 1 -- 172.21.15.121:0/3030399708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e84050 msgr2=0x1ead220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.800+0000 7f9f67686740 1 --2- 172.21.15.121:0/3030399708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e84050 0x1ead220 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 rx=0x7f9f4c0058e0 tx=0x7f9f4c015000).stop 2023-11-15T03:11:42.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.800+0000 7f9f67686740 1 -- 172.21.15.121:0/3030399708 shutdown_connections 2023-11-15T03:11:42.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.800+0000 7f9f67686740 1 --2- 172.21.15.121:0/3030399708 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1eb0a10 0x1e85b90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.800+0000 7f9f67686740 1 --2- 172.21.15.121:0/3030399708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1ed59d0 0x1eb21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.800+0000 7f9f67686740 1 --2- 172.21.15.121:0/3030399708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1e84050 0x1ead220 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.800+0000 7f9f67686740 1 --2- 172.21.15.121:0/3030399708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e83650 0x1eb1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.800+0000 7f9f357fa700 0 client.11097 ms_handle_reset on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:42.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.800+0000 7f9f67686740 1 -- 172.21.15.121:0/3030399708 >> 172.21.15.121:0/3030399708 conn(0x1c69710 msgr2=0x1e844b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:42.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.800+0000 7f9f67686740 1 -- 172.21.15.121:0/3030399708 shutdown_connections 2023-11-15T03:11:42.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.800+0000 7f9f67686740 1 -- 172.21.15.121:0/3030399708 wait complete. 2023-11-15T03:11:42.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.800+0000 7f9f67686740 20 client.11097 trim_cache size 0 max 16384 2023-11-15T03:11:42.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f67686740 1 Processor -- start 2023-11-15T03:11:42.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f67686740 1 -- start start 2023-11-15T03:11:42.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f67686740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e83650 0x1e89570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f67686740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e84050 0x1eb1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f67686740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed59d0 0x1e844b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f67686740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ec9b90 con 0x1ed59d0 2023-11-15T03:11:42.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f67686740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1eb3270 con 0x1e84050 2023-11-15T03:11:42.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f67686740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e87df0 con 0x1e83650 2023-11-15T03:11:42.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f577fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed59d0 0x1e844b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f577fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed59d0 0x1e844b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51746/0 (socket says 172.21.15.121:51746) 2023-11-15T03:11:42.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f577fe700 1 -- 172.21.15.121:0/1146072463 learned_addr learned my addr 172.21.15.121:0/1146072463 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:42.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f56ffd700 1 --2- 172.21.15.121:0/1146072463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e83650 0x1e89570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.834 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f567fc700 1 --2- 172.21.15.121:0/1146072463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e84050 0x1eb1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f577fe700 1 -- 172.21.15.121:0/1146072463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e83650 msgr2=0x1e89570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f577fe700 1 --2- 172.21.15.121:0/1146072463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e83650 0x1e89570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f577fe700 1 -- 172.21.15.121:0/1146072463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e84050 msgr2=0x1eb1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f577fe700 1 --2- 172.21.15.121:0/1146072463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e84050 0x1eb1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f577fe700 1 -- 172.21.15.121:0/1146072463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f580c4b90 con 0x1ed59d0 2023-11-15T03:11:42.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f567fc700 1 --2- 172.21.15.121:0/1146072463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e84050 0x1eb1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:11:42.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.816+0000 7f9f577fe700 1 --2- 172.21.15.121:0/1146072463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed59d0 0x1e844b0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 rx=0x7f9f50007fd0 tx=0x7f9f50022040).ready entity=mon.0 client_cookie=236343a2864e5c12 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:42.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.820+0000 7f9f3cff9700 1 -- 172.21.15.121:0/1146072463 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50056070 con 0x1ed59d0 2023-11-15T03:11:42.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.820+0000 7f9f3cff9700 1 -- 172.21.15.121:0/1146072463 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5004a5e0 con 0x1ed59d0 2023-11-15T03:11:42.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.820+0000 7f9f3cff9700 1 -- 172.21.15.121:0/1146072463 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50048950 con 0x1ed59d0 2023-11-15T03:11:42.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.820+0000 7f9f67686740 1 -- 172.21.15.121:0/1146072463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed59d0 msgr2=0x1e844b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.820+0000 7f9f67686740 1 --2- 172.21.15.121:0/1146072463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed59d0 0x1e844b0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 rx=0x7f9f50007fd0 tx=0x7f9f50022040).stop 2023-11-15T03:11:42.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.820+0000 7f9f67686740 1 -- 172.21.15.121:0/1146072463 shutdown_connections 2023-11-15T03:11:42.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.820+0000 7f9f67686740 1 --2- 172.21.15.121:0/1146072463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed59d0 0x1e844b0 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.820+0000 7f9f67686740 1 --2- 172.21.15.121:0/1146072463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e84050 0x1eb1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.820+0000 7f9f67686740 1 --2- 172.21.15.121:0/1146072463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e83650 0x1e89570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.820+0000 7f9f67686740 1 -- 172.21.15.121:0/1146072463 >> 172.21.15.121:0/1146072463 conn(0x1c69710 msgr2=0x1ec8670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:42.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.820+0000 7f9f67686740 1 -- 172.21.15.121:0/1146072463 shutdown_connections 2023-11-15T03:11:42.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.820+0000 7f9f67686740 1 -- 172.21.15.121:0/1146072463 wait complete. 2023-11-15T03:11:42.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.820+0000 7f9f67686740 1 -- start start 2023-11-15T03:11:42.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.820+0000 7f9f67686740 1 Processor -- start 2023-11-15T03:11:42.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f67686740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e83650 0x1f32160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f67686740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e84050 0x1ed13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f67686740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed59d0 0x1ed18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f67686740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ed4b70 con 0x1ed59d0 2023-11-15T03:11:42.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f67686740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e849b0 con 0x1e84050 2023-11-15T03:11:42.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f67686740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e84c90 con 0x1e83650 2023-11-15T03:11:42.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f67686740 10 client.0 ll_register_callbacks cb 0x1e016d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-15T03:11:42.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f577fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed59d0 0x1ed18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f577fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed59d0 0x1ed18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51748/0 (socket says 172.21.15.121:51748) 2023-11-15T03:11:42.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f577fe700 1 -- 172.21.15.121:0/1950698940 learned_addr learned my addr 172.21.15.121:0/1950698940 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:42.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f567fc700 1 --2- 172.21.15.121:0/1950698940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e84050 0x1ed13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f56ffd700 1 --2- 172.21.15.121:0/1950698940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e83650 0x1f32160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f577fe700 1 -- 172.21.15.121:0/1950698940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e83650 msgr2=0x1f32160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f577fe700 1 --2- 172.21.15.121:0/1950698940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e83650 0x1f32160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f577fe700 1 -- 172.21.15.121:0/1950698940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e84050 msgr2=0x1ed13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:42.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f577fe700 1 --2- 172.21.15.121:0/1950698940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e84050 0x1ed13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f577fe700 1 -- 172.21.15.121:0/1950698940 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f580c1810 con 0x1ed59d0 2023-11-15T03:11:42.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f577fe700 1 --2- 172.21.15.121:0/1950698940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed59d0 0x1ed18a0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 rx=0x7f9f50048740 tx=0x7f9f50056000).ready entity=mon.0 client_cookie=acbaaea5f5582c5a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:42.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f357fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-11-15T03:11:42.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f357fa700 1 -- 172.21.15.121:0/1950698940 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50048950 con 0x1ed59d0 2023-11-15T03:11:42.853 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f357fa700 1 -- 172.21.15.121:0/1950698940 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f9f50048ab0 con 0x1ed59d0 2023-11-15T03:11:42.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f67686740 1 -- 172.21.15.121:0/1950698940 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f4c0118f0 con 0x1ed59d0 2023-11-15T03:11:42.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f357fa700 1 -- 172.21.15.121:0/1950698940 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=355) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f50051b20 con 0x1ed59d0 2023-11-15T03:11:42.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f67686740 10 client.11103 fetch_fsmap learned FSMap version 355 2023-11-15T03:11:42.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f67686740 1 -- 172.21.15.121:0/1950698940 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=355,monmap=2+}) v3 -- 0x1dff7d0 con 0x1ed59d0 2023-11-15T03:11:42.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f357fa700 1 -- 172.21.15.121:0/1950698940 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c0118f0 con 0x1ed59d0 2023-11-15T03:11:42.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f357fa700 1 -- 172.21.15.121:0/1950698940 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 355) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9f50051c80 con 0x1ed59d0 2023-11-15T03:11:42.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f67686740 10 client.11103 fetch_fsmap finished waiting for FSMap version 355 2023-11-15T03:11:42.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f67686740 10 client.11103 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-15T03:11:42.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f67686740 20 client.11103 populate_metadata read hostname 'smithi121' 2023-11-15T03:11:42.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f67686740 1 --2- 172.21.15.121:0/1950698940 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1eb5690 0x1eb5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:42.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f67686740 4 client.11103 mds_command: new command op to 10899 tid=0[{"prefix": "damage rm", "damage_id": 3099883387}] 2023-11-15T03:11:42.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.824+0000 7f9f67686740 1 -- 172.21.15.121:0/1950698940 --> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] -- command(tid 0: {"prefix": "damage rm", "damage_id": 3099883387}) v1 -- 0x1cc06a0 con 0x1eb5690 2023-11-15T03:11:42.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f56ffd700 1 --2- 172.21.15.121:0/1950698940 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1eb5690 0x1eb5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:42.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f357fa700 10 client.11103 ms_handle_connect on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:42.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f56ffd700 1 --2- 172.21.15.121:0/1950698940 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1eb5690 0x1eb5a70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=104f408144eb3c51 server_cookie=5bc2538b76eded8b in_seq=0 out_seq=0 2023-11-15T03:11:42.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f357fa700 1 -- 172.21.15.121:0/1950698940 <== mds.0 v2:172.21.15.121:6836/3298117008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x1cc06a0 con 0x1eb5690 2023-11-15T03:11:42.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f357fa700 10 client.11103 handle_command_reply: tid=0 2023-11-15T03:11:42.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f67686740 1 client.11103 shutdown 2023-11-15T03:11:42.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f67686740 1 -- 172.21.15.121:0/1950698940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed59d0 msgr2=0x1ed18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f67686740 1 --2- 172.21.15.121:0/1950698940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed59d0 0x1ed18a0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 rx=0x7f9f50048740 tx=0x7f9f50056000).stop 2023-11-15T03:11:42.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f67686740 1 -- 172.21.15.121:0/1950698940 shutdown_connections 2023-11-15T03:11:42.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f67686740 1 --2- 172.21.15.121:0/1950698940 >> [v2:172.21.15.121:6836/3298117008,v1:172.21.15.121:6837/3298117008] conn(0x1eb5690 0x1eb5a70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f67686740 1 --2- 172.21.15.121:0/1950698940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x1ed59d0 0x1ed18a0 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f67686740 1 --2- 172.21.15.121:0/1950698940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x1e84050 0x1ed13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f67686740 1 --2- 172.21.15.121:0/1950698940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x1e83650 0x1f32160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f357fa700 0 client.11103 ms_handle_reset on v2:172.21.15.121:6836/3298117008 2023-11-15T03:11:42.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f67686740 1 -- 172.21.15.121:0/1950698940 >> 172.21.15.121:0/1950698940 conn(0x1c69710 msgr2=0x1eccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:42.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f67686740 1 -- 172.21.15.121:0/1950698940 shutdown_connections 2023-11-15T03:11:42.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f67686740 1 -- 172.21.15.121:0/1950698940 wait complete. 2023-11-15T03:11:42.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f67686740 20 client.11103 trim_cache size 0 max 16384 2023-11-15T03:11:42.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f57fff700 1 -- 172.21.15.121:0/133227370 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9f4405a510 msgr2=0x7f9f4405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f57fff700 1 --2- 172.21.15.121:0/133227370 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9f4405a510 0x7f9f4405c9d0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f9f48001670 tx=0x7f9f48016000).stop 2023-11-15T03:11:42.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f57fff700 1 -- 172.21.15.121:0/133227370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f581601c0 msgr2=0x7f9f580c4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:42.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f57fff700 1 --2- 172.21.15.121:0/133227370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f581601c0 0x7f9f580c4340 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 rx=0x7f9f581542b0 tx=0x7f9f5000b040).stop 2023-11-15T03:11:42.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f57fff700 1 -- 172.21.15.121:0/133227370 shutdown_connections 2023-11-15T03:11:42.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f57fff700 1 --2- 172.21.15.121:0/133227370 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9f4405a510 0x7f9f4405c9d0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f57fff700 1 --2- 172.21.15.121:0/133227370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f581601c0 0x7f9f580c4340 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f57fff700 1 --2- 172.21.15.121:0/133227370 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f580c0a00 0x7f9f5815fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f57fff700 1 --2- 172.21.15.121:0/133227370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f40003aa0 0x7f9f5813f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:42.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f57fff700 1 -- 172.21.15.121:0/133227370 >> 172.21.15.121:0/133227370 conn(0x7f9f58067a60 msgr2=0x7f9f58144ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:42.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f57fff700 1 -- 172.21.15.121:0/133227370 shutdown_connections 2023-11-15T03:11:42.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:42.828+0000 7f9f57fff700 1 -- 172.21.15.121:0/133227370 wait complete. 2023-11-15T03:11:42.864 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-15T03:11:42.865 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T03:11:42.961 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-15T03:11:43.014 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-15T03:11:43.015 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-15T03:11:43.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-15T03:11:43.015 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-15T03:11:43.064 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:11:43.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:43.066 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-15T03:11:43.127 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:43.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.328+0000 7f0393107700 1 Processor -- start 2023-11-15T03:11:43.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.328+0000 7f0393107700 1 -- start start 2023-11-15T03:11:43.340 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:11:43.332+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-11-15T03:11:43.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.332+0000 7f0393107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f038c066d60 0x7f038c067140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:43.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.336+0000 7f0393107700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f038c066370 0x7f038c1577e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:43.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.336+0000 7f0393107700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038c158180 con 0x7f038c157cd0 2023-11-15T03:11:43.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.336+0000 7f0393107700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f038c0c00f0 con 0x7f038c066370 2023-11-15T03:11:43.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.336+0000 7f0393107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038c066790 con 0x7f038c066d60 2023-11-15T03:11:43.344 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.336+0000 7f0391904700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f038c066370 0x7f038c1577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:43.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.336+0000 7f0392105700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f038c066d60 0x7f038c067140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:43.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.336+0000 7f0391904700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f038c066370 0x7f038c1577e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51750/0 (socket says 172.21.15.121:51750) 2023-11-15T03:11:43.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.336+0000 7f0391904700 1 -- 172.21.15.121:0/1558196451 learned_addr learned my addr 172.21.15.121:0/1558196451 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:43.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.336+0000 7f0391904700 1 -- 172.21.15.121:0/1558196451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f038c066d60 msgr2=0x7f038c067140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:43.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.336+0000 7f0391904700 1 --2- 172.21.15.121:0/1558196451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f038c066d60 0x7f038c067140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:43.346 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.336+0000 7f0391904700 1 -- 172.21.15.121:0/1558196451 >> v1:172.21.15.190:6789/0 conn(0x7f038c157cd0 legacy=0x7f038c1535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:43.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.336+0000 7f0391904700 1 -- 172.21.15.121:0/1558196451 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c1583f0 con 0x7f038c066370 2023-11-15T03:11:43.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.336+0000 7f0391904700 1 --2- 172.21.15.121:0/1558196451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f038c066370 0x7f038c1577e0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 rx=0x7f037c006fd0 tx=0x7f037c0090f0).ready entity=mon.0 client_cookie=c55eb9d4867ccfde server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:43.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.340+0000 7f0391103700 1 -- 172.21.15.121:0/1558196451 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c00b580 con 0x7f038c066370 2023-11-15T03:11:43.349 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.340+0000 7f0391103700 1 -- 172.21.15.121:0/1558196451 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037c00b6e0 con 0x7f038c066370 2023-11-15T03:11:43.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.340+0000 7f0391103700 1 -- 172.21.15.121:0/1558196451 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c00ba60 con 0x7f038c066370 2023-11-15T03:11:43.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.340+0000 7f0393107700 1 -- 172.21.15.121:0/1558196451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f038c066370 msgr2=0x7f038c1577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:43.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.340+0000 7f0393107700 1 --2- 172.21.15.121:0/1558196451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f038c066370 0x7f038c1577e0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 rx=0x7f037c006fd0 tx=0x7f037c0090f0).stop 2023-11-15T03:11:43.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.340+0000 7f0393107700 1 -- 172.21.15.121:0/1558196451 shutdown_connections 2023-11-15T03:11:43.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.340+0000 7f0393107700 1 --2- 172.21.15.121:0/1558196451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f038c066370 0x7f038c1577e0 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:43.351 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.340+0000 7f0393107700 1 --2- 172.21.15.121:0/1558196451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f038c066d60 0x7f038c067140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:43.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.340+0000 7f0393107700 1 -- 172.21.15.121:0/1558196451 >> 172.21.15.121:0/1558196451 conn(0x7f038c062190 msgr2=0x7f038c062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:43.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.340+0000 7f0393107700 1 -- 172.21.15.121:0/1558196451 shutdown_connections 2023-11-15T03:11:43.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.344+0000 7f0393107700 1 -- 172.21.15.121:0/1558196451 wait complete. 2023-11-15T03:11:43.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.344+0000 7f0393107700 1 Processor -- start 2023-11-15T03:11:43.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.344+0000 7f0393107700 1 -- start start 2023-11-15T03:11:43.355 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0393107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f038c066370 0x7f038c144100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:43.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0393107700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f038c066d60 0x7f038c1642e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:43.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0393107700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f038c157cd0 0x7f038c1647d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:43.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0393107700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c0aaa30 con 0x7f038c066d60 2023-11-15T03:11:43.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0393107700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c164e40 con 0x7f038c157cd0 2023-11-15T03:11:43.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0393107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038c165120 con 0x7f038c066370 2023-11-15T03:11:43.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0391904700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f038c066d60 0x7f038c1642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:43.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0391904700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f038c066d60 0x7f038c1642e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51764/0 (socket says 172.21.15.121:51764) 2023-11-15T03:11:43.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0391904700 1 -- 172.21.15.121:0/3437185901 learned_addr learned my addr 172.21.15.121:0/3437185901 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:43.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0392105700 1 --2- 172.21.15.121:0/3437185901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f038c066370 0x7f038c144100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:43.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0392906700 1 --2- 172.21.15.121:0/3437185901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f038c157cd0 0x7f038c1647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:43.360 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0391904700 1 -- 172.21.15.121:0/3437185901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f038c066370 msgr2=0x7f038c144100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:43.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0391904700 1 --2- 172.21.15.121:0/3437185901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f038c066370 0x7f038c144100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:43.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0391904700 1 -- 172.21.15.121:0/3437185901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f038c157cd0 msgr2=0x7f038c1647d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:43.361 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0391904700 1 --2- 172.21.15.121:0/3437185901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f038c157cd0 0x7f038c1647d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:43.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0391904700 1 -- 172.21.15.121:0/3437185901 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f037c009370 con 0x7f038c066d60 2023-11-15T03:11:43.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0392906700 1 --2- 172.21.15.121:0/3437185901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f038c157cd0 0x7f038c1647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:43.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0391904700 1 --2- 172.21.15.121:0/3437185901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f038c066d60 0x7f038c1642e0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 rx=0x7f037c009fd0 tx=0x7f037c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:43.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f03827fc700 1 -- 172.21.15.121:0/3437185901 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c00b850 con 0x7f038c066d60 2023-11-15T03:11:43.362 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0393107700 1 -- 172.21.15.121:0/3437185901 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038c0aac80 con 0x7f038c066d60 2023-11-15T03:11:43.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f03827fc700 1 -- 172.21.15.121:0/3437185901 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037c00b9b0 con 0x7f038c066d60 2023-11-15T03:11:43.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f03827fc700 1 -- 172.21.15.121:0/3437185901 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c01b640 con 0x7f038c066d60 2023-11-15T03:11:43.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.348+0000 7f0393107700 1 -- 172.21.15.121:0/3437185901 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038c147790 con 0x7f038c066d60 2023-11-15T03:11:43.363 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.352+0000 7f0393107700 1 -- 172.21.15.121:0/3437185901 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f038c06bcf0 con 0x7f038c066d60 2023-11-15T03:11:43.365 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.352+0000 7f03827fc700 1 -- 172.21.15.121:0/3437185901 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f037c01b9f0 con 0x7f038c066d60 2023-11-15T03:11:43.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.352+0000 7f03827fc700 1 --2- 172.21.15.121:0/3437185901 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f037005a510 0x7f037005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:43.366 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.352+0000 7f03827fc700 1 -- 172.21.15.121:0/3437185901 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f037c0760d0 con 0x7f038c066d60 2023-11-15T03:11:43.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.352+0000 7f0392105700 1 --2- 172.21.15.121:0/3437185901 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f037005a510 0x7f037005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:43.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.352+0000 7f0392105700 1 --2- 172.21.15.121:0/3437185901 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f037005a510 0x7f037005c9d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f038c156b20 tx=0x7f0388009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:43.367 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.360+0000 7f03827fc700 1 -- 172.21.15.121:0/3437185901 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f037c049800 con 0x7f038c066d60 2023-11-15T03:11:43.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.556+0000 7f0393107700 1 -- 172.21.15.121:0/3437185901 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f038c0bfac0 con 0x7f038c066d60 2023-11-15T03:11:43.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.560+0000 7f03827fc700 1 -- 172.21.15.121:0/3437185901 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 356 v356) v1 ==== 95+0+4514 (secure 0 0 0) 0x7f037c029020 con 0x7f038c066d60 2023-11-15T03:11:43.569 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:43.569 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":356,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/755048854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":755048854},{"type":"v1","addr":"172.21.15.190:6834","nonce":755048854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2360242049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2360242049},{"type":"v1","addr":"172.21.15.190:6835","nonce":2360242049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2780153683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2780153683},{"type":"v1","addr":"172.21.15.121:6835","nonce":2780153683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345}],"filesystems":[{"mdsmap":{"epoch":356,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:11:21.864101+0000","modified":"2023-11-15T03:11:43.335433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10899":{"gid":10899,"name":"a","rank":0,"incarnation":352,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6837/3298117008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3298117008},{"type":"v1","addr":"172.21.15.121:6837","nonce":3298117008}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-15T03:11:43.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.564+0000 7f0393107700 1 -- 172.21.15.121:0/3437185901 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f037005a510 msgr2=0x7f037005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:43.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.564+0000 7f0393107700 1 --2- 172.21.15.121:0/3437185901 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f037005a510 0x7f037005c9d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7f038c156b20 tx=0x7f0388009060).stop 2023-11-15T03:11:43.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.564+0000 7f0393107700 1 -- 172.21.15.121:0/3437185901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f038c066d60 msgr2=0x7f038c1642e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:43.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.564+0000 7f0393107700 1 --2- 172.21.15.121:0/3437185901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f038c066d60 0x7f038c1642e0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 rx=0x7f037c009fd0 tx=0x7f037c017000).stop 2023-11-15T03:11:43.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.564+0000 7f0393107700 1 -- 172.21.15.121:0/3437185901 shutdown_connections 2023-11-15T03:11:43.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.564+0000 7f0393107700 1 --2- 172.21.15.121:0/3437185901 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f037005a510 0x7f037005c9d0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:43.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.564+0000 7f0393107700 1 --2- 172.21.15.121:0/3437185901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f038c157cd0 0x7f038c1647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:43.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.564+0000 7f0393107700 1 --2- 172.21.15.121:0/3437185901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f038c066d60 0x7f038c1642e0 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:43.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.564+0000 7f0393107700 1 --2- 172.21.15.121:0/3437185901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f038c066370 0x7f038c144100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:43.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.564+0000 7f0393107700 1 -- 172.21.15.121:0/3437185901 >> 172.21.15.121:0/3437185901 conn(0x7f038c062190 msgr2=0x7f038c14d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:43.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.564+0000 7f0393107700 1 -- 172.21.15.121:0/3437185901 shutdown_connections 2023-11-15T03:11:43.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.564+0000 7f0393107700 1 -- 172.21.15.121:0/3437185901 wait complete. 2023-11-15T03:11:43.579 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 356 2023-11-15T03:11:43.594 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:11:43.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.740+0000 7fc81b65b700 1 Processor -- start 2023-11-15T03:11:43.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.740+0000 7fc81b65b700 1 -- start start 2023-11-15T03:11:43.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc81b65b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc8140a5ed0 0x7fc814155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:43.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc81b65b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8140c4c30 con 0x7fc814154ca0 2023-11-15T03:11:43.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc81b65b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8140a6f50 con 0x7fc8141562f0 2023-11-15T03:11:43.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc81b65b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8140a8280 con 0x7fc8140a5ed0 2023-11-15T03:11:43.755 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc8193f7700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc814154ca0 0x7fc8140a8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56334/0 (socket says 172.21.15.121:56334) 2023-11-15T03:11:43.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc8193f7700 1 -- 172.21.15.121:0/213949305 learned_addr learned my addr 172.21.15.121:0/213949305 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:43.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc819bf8700 1 --2- 172.21.15.121:0/213949305 >> v2:172.21.15.190:3300/0 conn(0x7fc8140a5ed0 0x7fc814155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:43.756 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc80bfff700 1 -- 172.21.15.121:0/213949305 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76378206 0 0) 0x7fc8140c4c30 con 0x7fc814154ca0 2023-11-15T03:11:43.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc80bfff700 1 -- 172.21.15.121:0/213949305 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8000033f0 con 0x7fc814154ca0 2023-11-15T03:11:43.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc80bfff700 1 -- 172.21.15.121:0/213949305 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669390259 0 0) 0x7fc8140a6f50 con 0x7fc8141562f0 2023-11-15T03:11:43.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc80bfff700 1 -- 172.21.15.121:0/213949305 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8140c4c30 con 0x7fc8141562f0 2023-11-15T03:11:43.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc80bfff700 1 -- 172.21.15.121:0/213949305 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3068811446 0 0) 0x7fc8000033f0 con 0x7fc814154ca0 2023-11-15T03:11:43.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc80bfff700 1 -- 172.21.15.121:0/213949305 >> v2:172.21.15.190:3300/0 conn(0x7fc8140a5ed0 msgr2=0x7fc814155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:43.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc80bfff700 1 --2- 172.21.15.121:0/213949305 >> v2:172.21.15.190:3300/0 conn(0x7fc8140a5ed0 0x7fc814155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:43.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc80bfff700 1 -- 172.21.15.121:0/213949305 >> v1:172.21.15.190:6789/0 conn(0x7fc8141562f0 legacy=0x7fc8140a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:43.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc80bfff700 1 -- 172.21.15.121:0/213949305 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8140a7230 con 0x7fc814154ca0 2023-11-15T03:11:43.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc80bfff700 1 -- 172.21.15.121:0/213949305 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7fc8100017f0 con 0x7fc814154ca0 2023-11-15T03:11:43.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc80bfff700 1 -- 172.21.15.121:0/213949305 >> v1:172.21.15.121:6789/0 conn(0x7fc814154ca0 legacy=0x7fc8140a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:43.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc80bfff700 1 --2- 172.21.15.121:0/213949305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc800003a70 0x7fc800003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:43.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.744+0000 7fc80bfff700 1 --2- 172.21.15.121:0/213949305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc800004690 0x7fc800004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:43.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc80bfff700 1 --2- 172.21.15.121:0/213949305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc800005000 0x7fc80000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:43.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc80bfff700 1 -- 172.21.15.121:0/213949305 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8140a8280 con 0x7fc800004690 2023-11-15T03:11:43.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc80bfff700 1 -- 172.21.15.121:0/213949305 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80000be80 con 0x7fc800005000 2023-11-15T03:11:43.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc80bfff700 1 -- 172.21.15.121:0/213949305 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80000c040 con 0x7fc800003a70 2023-11-15T03:11:43.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc818bf6700 1 --2- 172.21.15.121:0/213949305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc800004690 0x7fc800004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:43.760 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc8193f7700 1 --2- 172.21.15.121:0/213949305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc800003a70 0x7fc800003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:43.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc819bf8700 1 --2- 172.21.15.121:0/213949305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc800005000 0x7fc80000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:43.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc818bf6700 1 -- 172.21.15.121:0/213949305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc800003a70 msgr2=0x7fc800003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:43.761 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc818bf6700 1 --2- 172.21.15.121:0/213949305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc800003a70 0x7fc800003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:43.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc818bf6700 1 -- 172.21.15.121:0/213949305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc800005000 msgr2=0x7fc80000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:43.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc818bf6700 1 --2- 172.21.15.121:0/213949305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc800005000 0x7fc80000b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:43.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc818bf6700 1 -- 172.21.15.121:0/213949305 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8000033f0 con 0x7fc800004690 2023-11-15T03:11:43.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc819bf8700 1 -- 172.21.15.121:0/213949305 reap_dead start 2023-11-15T03:11:43.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc818bf6700 1 --2- 172.21.15.121:0/213949305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc800004690 0x7fc800004b10 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 rx=0x7fc814156760 tx=0x7fc804007bb0).ready entity=mon.0 client_cookie=af7b6f8ee3c72d87 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:43.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc80bfff700 1 -- 172.21.15.121:0/213949305 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804007020 con 0x7fc800004690 2023-11-15T03:11:43.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc80bfff700 1 -- 172.21.15.121:0/213949305 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc804007180 con 0x7fc800004690 2023-11-15T03:11:43.766 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc80bfff700 1 -- 172.21.15.121:0/213949305 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804007500 con 0x7fc800004690 2023-11-15T03:11:43.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc81b65b700 1 -- 172.21.15.121:0/213949305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc800004690 msgr2=0x7fc800004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:43.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc81b65b700 1 --2- 172.21.15.121:0/213949305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc800004690 0x7fc800004b10 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 rx=0x7fc814156760 tx=0x7fc804007bb0).stop 2023-11-15T03:11:43.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc81b65b700 1 -- 172.21.15.121:0/213949305 shutdown_connections 2023-11-15T03:11:43.767 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc81b65b700 1 --2- 172.21.15.121:0/213949305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc800004690 0x7fc800004b10 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:43.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc81b65b700 1 -- 172.21.15.121:0/213949305 >> 172.21.15.121:0/213949305 conn(0x7fc814062190 msgr2=0x7fc8140625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:43.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc81b65b700 1 -- 172.21.15.121:0/213949305 shutdown_connections 2023-11-15T03:11:43.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.748+0000 7fc81b65b700 1 -- 172.21.15.121:0/213949305 wait complete. 2023-11-15T03:11:43.768 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.752+0000 7fc81b65b700 1 Processor -- start 2023-11-15T03:11:43.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.752+0000 7fc81b65b700 1 -- start start 2023-11-15T03:11:43.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc81b65b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc800004690 0x7fc8141427e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:43.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc81b65b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8140aab80 0x7fc8140ac120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:43.769 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc81b65b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc81413f1c0 0x7fc8141405d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:43.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc81b65b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8140abc90 con 0x7fc81413f1c0 2023-11-15T03:11:43.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc81b65b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8140ab080 con 0x7fc8140aab80 2023-11-15T03:11:43.770 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc81b65b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8140ab360 con 0x7fc800004690 2023-11-15T03:11:43.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc819bf8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc81413f1c0 0x7fc8141405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:43.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc819bf8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc81413f1c0 0x7fc8141405d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51792/0 (socket says 172.21.15.121:51792) 2023-11-15T03:11:43.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc819bf8700 1 -- 172.21.15.121:0/582209602 learned_addr learned my addr 172.21.15.121:0/582209602 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:43.771 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc8193f7700 1 --2- 172.21.15.121:0/582209602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc800004690 0x7fc8141427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:43.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc818bf6700 1 --2- 172.21.15.121:0/582209602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8140aab80 0x7fc8140ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:43.772 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc819bf8700 1 -- 172.21.15.121:0/582209602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc800004690 msgr2=0x7fc8141427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:43.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc819bf8700 1 --2- 172.21.15.121:0/582209602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc800004690 0x7fc8141427e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:43.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc819bf8700 1 -- 172.21.15.121:0/582209602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8140aab80 msgr2=0x7fc8140ac120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:43.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc819bf8700 1 --2- 172.21.15.121:0/582209602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8140aab80 0x7fc8140ac120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:43.773 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc819bf8700 1 -- 172.21.15.121:0/582209602 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8140ab640 con 0x7fc81413f1c0 2023-11-15T03:11:43.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc818bf6700 1 --2- 172.21.15.121:0/582209602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8140aab80 0x7fc8140ac120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:43.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc819bf8700 1 --2- 172.21.15.121:0/582209602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc81413f1c0 0x7fc8141405d0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 rx=0x7fc814065380 tx=0x7fc80c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:43.774 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc8097fa700 1 -- 172.21.15.121:0/582209602 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c0124e0 con 0x7fc81413f1c0 2023-11-15T03:11:43.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc8097fa700 1 -- 172.21.15.121:0/582209602 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80c012640 con 0x7fc81413f1c0 2023-11-15T03:11:43.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc8097fa700 1 -- 172.21.15.121:0/582209602 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c0129e0 con 0x7fc81413f1c0 2023-11-15T03:11:43.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc81b65b700 1 -- 172.21.15.121:0/582209602 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8140ab920 con 0x7fc81413f1c0 2023-11-15T03:11:43.776 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.756+0000 7fc81b65b700 1 -- 172.21.15.121:0/582209602 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8140ac610 con 0x7fc81413f1c0 2023-11-15T03:11:43.777 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.760+0000 7fc81b65b700 1 -- 172.21.15.121:0/582209602 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8140abe00 con 0x7fc81413f1c0 2023-11-15T03:11:43.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.760+0000 7fc8097fa700 1 -- 172.21.15.121:0/582209602 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc80c0143d0 con 0x7fc81413f1c0 2023-11-15T03:11:43.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.760+0000 7fc8097fa700 1 --2- 172.21.15.121:0/582209602 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc7f805a510 0x7fc7f805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:43.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.764+0000 7fc8097fa700 1 -- 172.21.15.121:0/582209602 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7fc80c074600 con 0x7fc81413f1c0 2023-11-15T03:11:43.778 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.764+0000 7fc8193f7700 1 --2- 172.21.15.121:0/582209602 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc7f805a510 0x7fc7f805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:43.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.764+0000 7fc8193f7700 1 --2- 172.21.15.121:0/582209602 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc7f805a510 0x7fc7f805c9d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7fc810001580 tx=0x7fc810007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:43.779 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.768+0000 7fc8097fa700 1 -- 172.21.15.121:0/582209602 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc80c047c10 con 0x7fc81413f1c0 2023-11-15T03:11:43.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.936+0000 7fc81b65b700 1 -- 172.21.15.121:0/582209602 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc81406c150 con 0x7fc81413f1c0 2023-11-15T03:11:43.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.940+0000 7fc8097fa700 1 -- 172.21.15.121:0/582209602 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+17266 (secure 0 0 0) 0x7fc80c04b230 con 0x7fc81413f1c0 2023-11-15T03:11:43.949 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:43.949 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":148,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:11:40.308031+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-15T03:11:19.965438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-15T03:11:20.878094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.6","mappings":[{"from":5,"to":4}]},{"pgid":"16.15","mappings":[{"from":1,"to":4}]},{"pgid":"16.2d","mappings":[{"from":0,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6837/2335356851":"2023-11-16T03:11:31.820010+0000","172.21.15.121:6836/2335356851":"2023-11-16T03:11:31.820010+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:11:43.951 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.944+0000 7fc81b65b700 1 -- 172.21.15.121:0/582209602 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc7f805a510 msgr2=0x7fc7f805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:43.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.944+0000 7fc81b65b700 1 --2- 172.21.15.121:0/582209602 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc7f805a510 0x7fc7f805c9d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7fc810001580 tx=0x7fc810007d40).stop 2023-11-15T03:11:43.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.944+0000 7fc81b65b700 1 -- 172.21.15.121:0/582209602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc81413f1c0 msgr2=0x7fc8141405d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:43.952 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.944+0000 7fc81b65b700 1 --2- 172.21.15.121:0/582209602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc81413f1c0 0x7fc8141405d0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 rx=0x7fc814065380 tx=0x7fc80c008040).stop 2023-11-15T03:11:43.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.944+0000 7fc81b65b700 1 -- 172.21.15.121:0/582209602 shutdown_connections 2023-11-15T03:11:43.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.944+0000 7fc81b65b700 1 --2- 172.21.15.121:0/582209602 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc7f805a510 0x7fc7f805c9d0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:43.953 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.944+0000 7fc81b65b700 1 --2- 172.21.15.121:0/582209602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc81413f1c0 0x7fc8141405d0 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:43.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.944+0000 7fc81b65b700 1 --2- 172.21.15.121:0/582209602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8140aab80 0x7fc8140ac120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:43.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.944+0000 7fc81b65b700 1 --2- 172.21.15.121:0/582209602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc800004690 0x7fc8141427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:43.954 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.944+0000 7fc81b65b700 1 -- 172.21.15.121:0/582209602 >> 172.21.15.121:0/582209602 conn(0x7fc814062190 msgr2=0x7fc8141555d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:43.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.944+0000 7fc81b65b700 1 -- 172.21.15.121:0/582209602 shutdown_connections 2023-11-15T03:11:43.955 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:43.944+0000 7fc81b65b700 1 -- 172.21.15.121:0/582209602 wait complete. 2023-11-15T03:11:43.974 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-15T02:56:43.223941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-15T02:56:45.163959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-15T03:11:19.965438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-11-15T03:11:20.878094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-15T03:11:43.975 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-15T03:11:44.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.124+0000 7fa8161b9700 1 Processor -- start 2023-11-15T03:11:44.131 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.124+0000 7fa8161b9700 1 -- start start 2023-11-15T03:11:44.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.124+0000 7fa8161b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8100c5c70 0x7fa810153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:44.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.124+0000 7fa8161b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa810065ea0 con 0x7fa810066330 2023-11-15T03:11:44.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa8161b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa810066bb0 con 0x7fa8100acd00 2023-11-15T03:11:44.135 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa8161b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa810065890 con 0x7fa8100c5c70 2023-11-15T03:11:44.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa8159b8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa810066330 0x7fa8101577e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56346/0 (socket says 172.21.15.121:56346) 2023-11-15T03:11:44.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa8149b6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8100c5c70 0x7fa810153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:44.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa8159b8700 1 -- 172.21.15.121:0/2680069254 learned_addr learned my addr 172.21.15.121:0/2680069254 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:44.137 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa8151b7700 1 --1- 172.21.15.121:0/2680069254 >> v1:172.21.15.190:6789/0 conn(0x7fa8100acd00 0x7fa8101536c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:50192/0 (socket says 172.21.15.121:50192) 2023-11-15T03:11:44.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa807fff700 1 -- 172.21.15.121:0/2680069254 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2923242172 0 0) 0x7fa810065ea0 con 0x7fa810066330 2023-11-15T03:11:44.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa807fff700 1 -- 172.21.15.121:0/2680069254 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7fc0033f0 con 0x7fa810066330 2023-11-15T03:11:44.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa8149b6700 1 -- 172.21.15.121:0/2680069254 >> v1:172.21.15.190:6789/0 conn(0x7fa8100acd00 legacy=0x7fa8101536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:44.138 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa807fff700 1 -- 172.21.15.121:0/2680069254 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3750434743 0 0) 0x7fa7fc0033f0 con 0x7fa810066330 2023-11-15T03:11:44.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa8149b6700 1 -- 172.21.15.121:0/2680069254 >> v1:172.21.15.121:6789/0 conn(0x7fa810066330 legacy=0x7fa8101577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:44.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa8149b6700 1 -- 172.21.15.121:0/2680069254 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8100c64a0 con 0x7fa8100c5c70 2023-11-15T03:11:44.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa8149b6700 1 --2- 172.21.15.121:0/2680069254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8100c5c70 0x7fa810153dd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fa800009790 tx=0x7fa800009060).ready entity=mon.2 client_cookie=f9a6674fd0fb9f57 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:44.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa807fff700 1 -- 172.21.15.121:0/2680069254 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8080017f0 con 0x7fa8100c5c70 2023-11-15T03:11:44.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa807fff700 1 -- 172.21.15.121:0/2680069254 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa800011400 con 0x7fa8100c5c70 2023-11-15T03:11:44.141 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa807fff700 1 -- 172.21.15.121:0/2680069254 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa800011780 con 0x7fa8100c5c70 2023-11-15T03:11:44.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa8161b9700 1 -- 172.21.15.121:0/2680069254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8100c5c70 msgr2=0x7fa810153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:44.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa8161b9700 1 --2- 172.21.15.121:0/2680069254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8100c5c70 0x7fa810153dd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fa800009790 tx=0x7fa800009060).stop 2023-11-15T03:11:44.144 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa8161b9700 1 -- 172.21.15.121:0/2680069254 shutdown_connections 2023-11-15T03:11:44.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa8161b9700 1 --2- 172.21.15.121:0/2680069254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8100c5c70 0x7fa810153dd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:44.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa8161b9700 1 -- 172.21.15.121:0/2680069254 >> 172.21.15.121:0/2680069254 conn(0x7fa810063fc0 msgr2=0x7fa810067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:44.145 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa8161b9700 1 -- 172.21.15.121:0/2680069254 shutdown_connections 2023-11-15T03:11:44.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.128+0000 7fa8161b9700 1 -- 172.21.15.121:0/2680069254 wait complete. 2023-11-15T03:11:44.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8161b9700 1 Processor -- start 2023-11-15T03:11:44.146 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8161b9700 1 -- start start 2023-11-15T03:11:44.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8161b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa810066330 0x7fa8100af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:44.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8161b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8100acd00 0x7fa810150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:44.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8161b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8100c5c70 0x7fa8100c3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:44.147 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8161b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8100c4b50 con 0x7fa8100acd00 2023-11-15T03:11:44.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8161b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8100b0330 con 0x7fa810066330 2023-11-15T03:11:44.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8161b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa810157bc0 con 0x7fa8100c5c70 2023-11-15T03:11:44.148 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8149b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8100acd00 0x7fa810150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:44.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8149b6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8100acd00 0x7fa810150c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51802/0 (socket says 172.21.15.121:51802) 2023-11-15T03:11:44.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8149b6700 1 -- 172.21.15.121:0/3950494841 learned_addr learned my addr 172.21.15.121:0/3950494841 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:44.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8151b7700 1 --2- 172.21.15.121:0/3950494841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa810066330 0x7fa8100af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:44.149 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8149b6700 1 -- 172.21.15.121:0/3950494841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8100c5c70 msgr2=0x7fa8100c3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:44.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8149b6700 1 --2- 172.21.15.121:0/3950494841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8100c5c70 0x7fa8100c3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:11:44.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8149b6700 1 -- 172.21.15.121:0/3950494841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa810066330 msgr2=0x7fa8100af300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:44.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8149b6700 1 --2- 172.21.15.121:0/3950494841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa810066330 0x7fa8100af300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:44.150 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8149b6700 1 -- 172.21.15.121:0/3950494841 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8000092a0 con 0x7fa8100acd00 2023-11-15T03:11:44.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8149b6700 1 --2- 172.21.15.121:0/3950494841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8100acd00 0x7fa810150c20 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 rx=0x7fa800009e30 tx=0x7fa800016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:44.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8057fa700 1 -- 172.21.15.121:0/3950494841 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8000109d0 con 0x7fa8100acd00 2023-11-15T03:11:44.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8057fa700 1 -- 172.21.15.121:0/3950494841 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa800010b30 con 0x7fa8100acd00 2023-11-15T03:11:44.151 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8161b9700 1 -- 172.21.15.121:0/3950494841 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8100af850 con 0x7fa8100acd00 2023-11-15T03:11:44.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.132+0000 7fa8057fa700 1 -- 172.21.15.121:0/3950494841 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80001d400 con 0x7fa8100acd00 2023-11-15T03:11:44.152 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.136+0000 7fa8161b9700 1 -- 172.21.15.121:0/3950494841 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8100b0050 con 0x7fa8100acd00 2023-11-15T03:11:44.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.136+0000 7fa8161b9700 1 -- 172.21.15.121:0/3950494841 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa81006bcf0 con 0x7fa8100acd00 2023-11-15T03:11:44.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.136+0000 7fa8057fa700 1 -- 172.21.15.121:0/3950494841 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa800018070 con 0x7fa8100acd00 2023-11-15T03:11:44.153 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.136+0000 7fa8057fa700 1 --2- 172.21.15.121:0/3950494841 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa7f405a510 0x7fa7f405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:44.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.136+0000 7fa8057fa700 1 -- 172.21.15.121:0/3950494841 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7fa800075f00 con 0x7fa8100acd00 2023-11-15T03:11:44.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.140+0000 7fa8151b7700 1 --2- 172.21.15.121:0/3950494841 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa7f405a510 0x7fa7f405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:44.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.140+0000 7fa8151b7700 1 --2- 172.21.15.121:0/3950494841 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa7f405a510 0x7fa7f405c9d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7fa80c007fd0 tx=0x7fa80c009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:44.154 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.144+0000 7fa8057fa700 1 -- 172.21.15.121:0/3950494841 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa800049570 con 0x7fa8100acd00 2023-11-15T03:11:44.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.304+0000 7fa8161b9700 1 -- 172.21.15.121:0/3950494841 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa8100af850 con 0x7fa8100acd00 2023-11-15T03:11:44.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.304+0000 7fa8057fa700 1 -- 172.21.15.121:0/3950494841 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v148) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa8100af850 con 0x7fa8100acd00 2023-11-15T03:11:44.314 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2023-11-15T03:11:44.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.308+0000 7fa8161b9700 1 -- 172.21.15.121:0/3950494841 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa7f405a510 msgr2=0x7fa7f405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:44.316 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.308+0000 7fa8161b9700 1 --2- 172.21.15.121:0/3950494841 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa7f405a510 0x7fa7f405c9d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7fa80c007fd0 tx=0x7fa80c009040).stop 2023-11-15T03:11:44.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.308+0000 7fa8161b9700 1 -- 172.21.15.121:0/3950494841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8100acd00 msgr2=0x7fa810150c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:44.317 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.308+0000 7fa8161b9700 1 --2- 172.21.15.121:0/3950494841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8100acd00 0x7fa810150c20 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 rx=0x7fa800009e30 tx=0x7fa800016000).stop 2023-11-15T03:11:44.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.308+0000 7fa8161b9700 1 -- 172.21.15.121:0/3950494841 shutdown_connections 2023-11-15T03:11:44.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.308+0000 7fa8161b9700 1 --2- 172.21.15.121:0/3950494841 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa7f405a510 0x7fa7f405c9d0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:44.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.308+0000 7fa8161b9700 1 --2- 172.21.15.121:0/3950494841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8100c5c70 0x7fa8100c3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:11:44.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.308+0000 7fa8161b9700 1 --2- 172.21.15.121:0/3950494841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8100acd00 0x7fa810150c20 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:44.318 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.308+0000 7fa8161b9700 1 --2- 172.21.15.121:0/3950494841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa810066330 0x7fa8100af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:44.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.308+0000 7fa8161b9700 1 -- 172.21.15.121:0/3950494841 >> 172.21.15.121:0/3950494841 conn(0x7fa810063fc0 msgr2=0x7fa810152be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:44.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.308+0000 7fa8161b9700 1 -- 172.21.15.121:0/3950494841 shutdown_connections 2023-11-15T03:11:44.319 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.308+0000 7fa8161b9700 1 -- 172.21.15.121:0/3950494841 wait complete. 2023-11-15T03:11:44.336 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-15T03:11:44.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.500+0000 7f81b8969700 1 Processor -- start 2023-11-15T03:11:44.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.500+0000 7f81b8969700 1 -- start start 2023-11-15T03:11:44.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.504+0000 7f81b8969700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b4067020 0x7f81b41582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:44.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.504+0000 7f81b8969700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b4062da0 con 0x7f81b4157ed0 2023-11-15T03:11:44.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.504+0000 7f81b8969700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b4062190 con 0x7f81b4068170 2023-11-15T03:11:44.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.504+0000 7f81b8969700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b40a93a0 con 0x7f81b4067020 2023-11-15T03:11:44.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.504+0000 7f81b259c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f81b4157ed0 0x7f81b4067a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56358/0 (socket says 172.21.15.121:56358) 2023-11-15T03:11:44.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.504+0000 7f81b259c700 1 -- 172.21.15.121:0/3846962187 learned_addr learned my addr 172.21.15.121:0/3846962187 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:44.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.504+0000 7f81b1d9b700 1 --2- 172.21.15.121:0/3846962187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b4067020 0x7f81b41582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:44.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.504+0000 7f81b159a700 1 -- 172.21.15.121:0/3846962187 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218682690 0 0) 0x7f81b4062da0 con 0x7f81b4157ed0 2023-11-15T03:11:44.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.504+0000 7f81b159a700 1 -- 172.21.15.121:0/3846962187 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f819c0033f0 con 0x7f81b4157ed0 2023-11-15T03:11:44.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.504+0000 7f81b1d9b700 1 -- 172.21.15.121:0/3846962187 >> v1:172.21.15.190:6789/0 conn(0x7f81b4068170 legacy=0x7f81b40bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:44.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.504+0000 7f81b159a700 1 -- 172.21.15.121:0/3846962187 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 301810733 0 0) 0x7f81b4062190 con 0x7f81b4068170 2023-11-15T03:11:44.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.504+0000 7f81b1d9b700 1 -- 172.21.15.121:0/3846962187 >> v1:172.21.15.121:6789/0 conn(0x7f81b4157ed0 legacy=0x7f81b4067a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:44.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.508+0000 7f81b1d9b700 1 -- 172.21.15.121:0/3846962187 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b4062470 con 0x7f81b4067020 2023-11-15T03:11:44.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.508+0000 7f81b1d9b700 1 --2- 172.21.15.121:0/3846962187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b4067020 0x7f81b41582b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f81a8009790 tx=0x7f81a8009060).ready entity=mon.2 client_cookie=5c9999d15e1e10a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:44.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.508+0000 7f81b159a700 1 -- 172.21.15.121:0/3846962187 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a800f510 con 0x7f81b4067020 2023-11-15T03:11:44.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.508+0000 7f81b159a700 1 -- 172.21.15.121:0/3846962187 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a800f670 con 0x7f81b4067020 2023-11-15T03:11:44.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.508+0000 7f81b159a700 1 -- 172.21.15.121:0/3846962187 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a800f9f0 con 0x7f81b4067020 2023-11-15T03:11:44.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.508+0000 7f81b8969700 1 -- 172.21.15.121:0/3846962187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b4067020 msgr2=0x7f81b41582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:44.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.508+0000 7f81b8969700 1 --2- 172.21.15.121:0/3846962187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b4067020 0x7f81b41582b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f81a8009790 tx=0x7f81a8009060).stop 2023-11-15T03:11:44.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.508+0000 7f81b8969700 1 -- 172.21.15.121:0/3846962187 shutdown_connections 2023-11-15T03:11:44.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.508+0000 7f81b8969700 1 --2- 172.21.15.121:0/3846962187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b4067020 0x7f81b41582b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:44.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.508+0000 7f81b8969700 1 -- 172.21.15.121:0/3846962187 >> 172.21.15.121:0/3846962187 conn(0x7f81b40591c0 msgr2=0x7f81b40c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:44.521 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.508+0000 7f81b8969700 1 -- 172.21.15.121:0/3846962187 shutdown_connections 2023-11-15T03:11:44.522 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.508+0000 7f81b8969700 1 -- 172.21.15.121:0/3846962187 wait complete. 2023-11-15T03:11:44.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.512+0000 7f81b8969700 1 Processor -- start 2023-11-15T03:11:44.524 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.512+0000 7f81b8969700 1 -- start start 2023-11-15T03:11:44.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.512+0000 7f81b8969700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b4067020 0x7f81b41582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:44.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.512+0000 7f81b8969700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81b4068170 0x7f81b414cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:44.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.512+0000 7f81b8969700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81b4157ed0 0x7f81b409fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:44.525 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.512+0000 7f81b8969700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b4157030 con 0x7f81b4068170 2023-11-15T03:11:44.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.512+0000 7f81b8969700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b41587a0 con 0x7f81b4157ed0 2023-11-15T03:11:44.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.512+0000 7f81b8969700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b416a7b0 con 0x7f81b4067020 2023-11-15T03:11:44.526 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.512+0000 7f81b1d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81b4068170 0x7f81b414cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:44.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.512+0000 7f81b1d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81b4068170 0x7f81b414cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51814/0 (socket says 172.21.15.121:51814) 2023-11-15T03:11:44.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.512+0000 7f81b1d9b700 1 -- 172.21.15.121:0/1482965255 learned_addr learned my addr 172.21.15.121:0/1482965255 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:44.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.512+0000 7f81b259c700 1 --2- 172.21.15.121:0/1482965255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b4067020 0x7f81b41582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:44.527 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.512+0000 7f81b2d9d700 1 --2- 172.21.15.121:0/1482965255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81b4157ed0 0x7f81b409fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:44.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.512+0000 7f81b1d9b700 1 -- 172.21.15.121:0/1482965255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b4067020 msgr2=0x7f81b41582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:44.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.512+0000 7f81b1d9b700 1 --2- 172.21.15.121:0/1482965255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b4067020 0x7f81b41582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:44.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.512+0000 7f81b1d9b700 1 -- 172.21.15.121:0/1482965255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81b4157ed0 msgr2=0x7f81b409fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:44.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.512+0000 7f81b1d9b700 1 --2- 172.21.15.121:0/1482965255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81b4157ed0 0x7f81b409fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:44.528 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.512+0000 7f81b1d9b700 1 -- 172.21.15.121:0/1482965255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a80092a0 con 0x7f81b4068170 2023-11-15T03:11:44.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.516+0000 7f81b1d9b700 1 --2- 172.21.15.121:0/1482965255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81b4068170 0x7f81b414cbf0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 rx=0x7f81b4059730 tx=0x7f81a8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:44.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.516+0000 7f81a27fc700 1 -- 172.21.15.121:0/1482965255 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a801a4d0 con 0x7f81b4068170 2023-11-15T03:11:44.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.516+0000 7f81a27fc700 1 -- 172.21.15.121:0/1482965255 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a801e070 con 0x7f81b4068170 2023-11-15T03:11:44.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.516+0000 7f81b8969700 1 -- 172.21.15.121:0/1482965255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b414d0e0 con 0x7f81b4068170 2023-11-15T03:11:44.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.516+0000 7f81a27fc700 1 -- 172.21.15.121:0/1482965255 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a801a8c0 con 0x7f81b4068170 2023-11-15T03:11:44.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.516+0000 7f81b8969700 1 -- 172.21.15.121:0/1482965255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b4169e80 con 0x7f81b4068170 2023-11-15T03:11:44.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.516+0000 7f81b8969700 1 -- 172.21.15.121:0/1482965255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b4047060 con 0x7f81b4068170 2023-11-15T03:11:44.531 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.516+0000 7f81a27fc700 1 -- 172.21.15.121:0/1482965255 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f81a8020030 con 0x7f81b4068170 2023-11-15T03:11:44.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.520+0000 7f81a27fc700 1 --2- 172.21.15.121:0/1482965255 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f819405a510 0x7f819405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:44.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.520+0000 7f81a27fc700 1 -- 172.21.15.121:0/1482965255 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f81a80758e0 con 0x7f81b4068170 2023-11-15T03:11:44.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.520+0000 7f81b259c700 1 --2- 172.21.15.121:0/1482965255 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f819405a510 0x7f819405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:44.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.520+0000 7f81b259c700 1 --2- 172.21.15.121:0/1482965255 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f819405a510 0x7f819405c9d0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f81a4005ad0 tx=0x7f81a4009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:44.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.524+0000 7f81a27fc700 1 -- 172.21.15.121:0/1482965255 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f81a8048fd0 con 0x7f81b4068170 2023-11-15T03:11:44.695 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.688+0000 7f81b8969700 1 -- 172.21.15.121:0/1482965255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f81b4068b50 con 0x7f81b4068170 2023-11-15T03:11:44.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.688+0000 7f81a27fc700 1 -- 172.21.15.121:0/1482965255 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v148) v1 ==== 92+0+10 (secure 0 0 0) 0x7f81a804c5f0 con 0x7f81b4068170 2023-11-15T03:11:44.696 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2023-11-15T03:11:44.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.692+0000 7f81b8969700 1 -- 172.21.15.121:0/1482965255 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f819405a510 msgr2=0x7f819405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:44.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.692+0000 7f81b8969700 1 --2- 172.21.15.121:0/1482965255 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f819405a510 0x7f819405c9d0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f81a4005ad0 tx=0x7f81a4009040).stop 2023-11-15T03:11:44.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.692+0000 7f81b8969700 1 -- 172.21.15.121:0/1482965255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81b4068170 msgr2=0x7f81b414cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:44.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.692+0000 7f81b8969700 1 --2- 172.21.15.121:0/1482965255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81b4068170 0x7f81b414cbf0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 rx=0x7f81b4059730 tx=0x7f81a8017000).stop 2023-11-15T03:11:44.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.692+0000 7f81b8969700 1 -- 172.21.15.121:0/1482965255 shutdown_connections 2023-11-15T03:11:44.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.692+0000 7f81b8969700 1 --2- 172.21.15.121:0/1482965255 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f819405a510 0x7f819405c9d0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:44.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.692+0000 7f81b8969700 1 --2- 172.21.15.121:0/1482965255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81b4157ed0 0x7f81b409fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:44.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.692+0000 7f81b8969700 1 --2- 172.21.15.121:0/1482965255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81b4068170 0x7f81b414cbf0 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:44.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.692+0000 7f81b8969700 1 --2- 172.21.15.121:0/1482965255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b4067020 0x7f81b41582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:44.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.692+0000 7f81b8969700 1 -- 172.21.15.121:0/1482965255 >> 172.21.15.121:0/1482965255 conn(0x7f81b40591c0 msgr2=0x7f81b40a9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:44.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.692+0000 7f81b8969700 1 -- 172.21.15.121:0/1482965255 shutdown_connections 2023-11-15T03:11:44.703 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.692+0000 7f81b8969700 1 -- 172.21.15.121:0/1482965255 wait complete. 2023-11-15T03:11:44.721 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-15T03:11:44.883 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.876+0000 7f99e50b9700 1 Processor -- start 2023-11-15T03:11:44.884 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.876+0000 7f99e50b9700 1 -- start start 2023-11-15T03:11:44.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.876+0000 7f99e50b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f99e00a8960 0x7f99e00c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:44.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.876+0000 7f99e50b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f99e0066520 0x7f99e0066990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:44.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.876+0000 7f99e50b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e00657f0 con 0x7f99e00a8d40 2023-11-15T03:11:44.886 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.876+0000 7f99e50b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e01583d0 con 0x7f99e00a8960 2023-11-15T03:11:44.887 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.876+0000 7f99e50b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e00c1b60 con 0x7f99e0066520 2023-11-15T03:11:44.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.876+0000 7f99dffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f99e00a8960 0x7f99e00c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:44.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.876+0000 7f99d7fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f99e00a8d40 0x7f99e015c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59050/0 (socket says 172.21.15.121:59050) 2023-11-15T03:11:44.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.876+0000 7f99d7fff700 1 -- 172.21.15.121:0/724058661 learned_addr learned my addr 172.21.15.121:0/724058661 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:44.888 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.876+0000 7f99e48b8700 1 --2- 172.21.15.121:0/724058661 >> v2:172.21.15.190:3300/0 conn(0x7f99e0066520 0x7f99e0066990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:44.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.876+0000 7f99dffff700 1 -- 172.21.15.121:0/724058661 >> v2:172.21.15.190:3300/0 conn(0x7f99e0066520 msgr2=0x7f99e0066990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:44.889 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.876+0000 7f99dffff700 1 --2- 172.21.15.121:0/724058661 >> v2:172.21.15.190:3300/0 conn(0x7f99e0066520 0x7f99e0066990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:44.890 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.876+0000 7f99dffff700 1 -- 172.21.15.121:0/724058661 >> v1:172.21.15.121:6789/0 conn(0x7f99e00a8d40 legacy=0x7f99e015c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:44.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.876+0000 7f99dffff700 1 -- 172.21.15.121:0/724058661 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e0065a40 con 0x7f99e00a8960 2023-11-15T03:11:44.891 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.880+0000 7f99dffff700 1 --2- 172.21.15.121:0/724058661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99e00a8960 0x7f99e00c15c0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 rx=0x7f99e005f8f0 tx=0x7f99d0009060).ready entity=mon.0 client_cookie=2b5dee3e1a1147b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:44.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.880+0000 7f99df7fe700 1 -- 172.21.15.121:0/724058661 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d000b580 con 0x7f99e00a8960 2023-11-15T03:11:44.892 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.880+0000 7f99df7fe700 1 -- 172.21.15.121:0/724058661 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d000b6e0 con 0x7f99e00a8960 2023-11-15T03:11:44.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.880+0000 7f99df7fe700 1 -- 172.21.15.121:0/724058661 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d000ba60 con 0x7f99e00a8960 2023-11-15T03:11:44.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.880+0000 7f99e50b9700 1 -- 172.21.15.121:0/724058661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99e00a8960 msgr2=0x7f99e00c15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:44.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.880+0000 7f99e50b9700 1 --2- 172.21.15.121:0/724058661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99e00a8960 0x7f99e00c15c0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 rx=0x7f99e005f8f0 tx=0x7f99d0009060).stop 2023-11-15T03:11:44.893 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.880+0000 7f99e50b9700 1 -- 172.21.15.121:0/724058661 shutdown_connections 2023-11-15T03:11:44.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.880+0000 7f99e50b9700 1 --2- 172.21.15.121:0/724058661 >> v2:172.21.15.190:3300/0 conn(0x7f99e0066520 0x7f99e0066990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:44.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.880+0000 7f99e50b9700 1 --2- 172.21.15.121:0/724058661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99e00a8960 0x7f99e00c15c0 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:44.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.880+0000 7f99e50b9700 1 -- 172.21.15.121:0/724058661 >> 172.21.15.121:0/724058661 conn(0x7f99e0067a60 msgr2=0x7f99e0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:44.894 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.880+0000 7f99e50b9700 1 -- 172.21.15.121:0/724058661 shutdown_connections 2023-11-15T03:11:44.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.880+0000 7f99e50b9700 1 -- 172.21.15.121:0/724058661 wait complete. 2023-11-15T03:11:44.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.884+0000 7f99e50b9700 1 Processor -- start 2023-11-15T03:11:44.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.884+0000 7f99e50b9700 1 -- start start 2023-11-15T03:11:44.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.884+0000 7f99e50b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99e0066520 0x7f99e0154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:44.897 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.884+0000 7f99e50b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99e00a8960 0x7f99e015e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:44.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.884+0000 7f99e50b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99e00a8d40 0x7f99e015e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:44.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.884+0000 7f99e50b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e015e9e0 con 0x7f99e0066520 2023-11-15T03:11:44.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.884+0000 7f99e50b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e0168cc0 con 0x7f99e00a8d40 2023-11-15T03:11:44.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.884+0000 7f99e50b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e0168fa0 con 0x7f99e00a8960 2023-11-15T03:11:44.898 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.884+0000 7f99dffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99e0066520 0x7f99e0154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:44.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.884+0000 7f99dffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99e0066520 0x7f99e0154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55842/0 (socket says 172.21.15.121:55842) 2023-11-15T03:11:44.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.884+0000 7f99dffff700 1 -- 172.21.15.121:0/2838541088 learned_addr learned my addr 172.21.15.121:0/2838541088 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:44.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.884+0000 7f99dffff700 1 -- 172.21.15.121:0/2838541088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99e00a8960 msgr2=0x7f99e015e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:44.899 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.884+0000 7f99e48b8700 1 --2- 172.21.15.121:0/2838541088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99e00a8d40 0x7f99e015e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:44.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.884+0000 7f99dffff700 1 --2- 172.21.15.121:0/2838541088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99e00a8960 0x7f99e015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:11:44.900 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.884+0000 7f99dffff700 1 -- 172.21.15.121:0/2838541088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99e00a8d40 msgr2=0x7f99e015e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:44.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.884+0000 7f99dffff700 1 --2- 172.21.15.121:0/2838541088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99e00a8d40 0x7f99e015e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:44.901 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.884+0000 7f99dffff700 1 -- 172.21.15.121:0/2838541088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d00092a0 con 0x7f99e0066520 2023-11-15T03:11:44.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.884+0000 7f99dffff700 1 --2- 172.21.15.121:0/2838541088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99e0066520 0x7f99e0154960 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 rx=0x7f99d0009fd0 tx=0x7f99d0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:44.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.888+0000 7f99dcff9700 1 -- 172.21.15.121:0/2838541088 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d000b850 con 0x7f99e0066520 2023-11-15T03:11:44.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.888+0000 7f99dcff9700 1 -- 172.21.15.121:0/2838541088 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d000b9b0 con 0x7f99e0066520 2023-11-15T03:11:44.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.888+0000 7f99dcff9700 1 -- 172.21.15.121:0/2838541088 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d001b660 con 0x7f99e0066520 2023-11-15T03:11:44.902 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.888+0000 7f99e50b9700 1 -- 172.21.15.121:0/2838541088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e0169280 con 0x7f99e0066520 2023-11-15T03:11:44.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.888+0000 7f99e50b9700 1 -- 172.21.15.121:0/2838541088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e0168970 con 0x7f99e0066520 2023-11-15T03:11:44.903 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.888+0000 7f99e50b9700 1 -- 172.21.15.121:0/2838541088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e006bcf0 con 0x7f99e0066520 2023-11-15T03:11:44.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.888+0000 7f99dcff9700 1 -- 172.21.15.121:0/2838541088 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f99d001ba90 con 0x7f99e0066520 2023-11-15T03:11:44.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.892+0000 7f99dcff9700 1 --2- 172.21.15.121:0/2838541088 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f99c005a510 0x7f99c005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:44.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.892+0000 7f99dcff9700 1 -- 172.21.15.121:0/2838541088 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f99d0076300 con 0x7f99e0066520 2023-11-15T03:11:44.904 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.892+0000 7f99d7fff700 1 --2- 172.21.15.121:0/2838541088 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f99c005a510 0x7f99c005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:44.905 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.892+0000 7f99d7fff700 1 --2- 172.21.15.121:0/2838541088 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f99c005a510 0x7f99c005c9d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f99e00c7570 tx=0x7f99c8007a10).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:44.906 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:44.896+0000 7f99dcff9700 1 -- 172.21.15.121:0/2838541088 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f99d0049a30 con 0x7f99e0066520 2023-11-15T03:11:45.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.060+0000 7f99e50b9700 1 -- 172.21.15.121:0/2838541088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f99e00a79b0 con 0x7f99e0066520 2023-11-15T03:11:45.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.060+0000 7f99dcff9700 1 -- 172.21.15.121:0/2838541088 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v148) v1 ==== 104+0+11 (secure 0 0 0) 0x7f99d0029020 con 0x7f99e0066520 2023-11-15T03:11:45.071 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T03:11:45.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.064+0000 7f99e50b9700 1 -- 172.21.15.121:0/2838541088 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f99c005a510 msgr2=0x7f99c005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:45.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.064+0000 7f99e50b9700 1 --2- 172.21.15.121:0/2838541088 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f99c005a510 0x7f99c005c9d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f99e00c7570 tx=0x7f99c8007a10).stop 2023-11-15T03:11:45.073 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.064+0000 7f99e50b9700 1 -- 172.21.15.121:0/2838541088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99e0066520 msgr2=0x7f99e0154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:45.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.064+0000 7f99e50b9700 1 --2- 172.21.15.121:0/2838541088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99e0066520 0x7f99e0154960 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 rx=0x7f99d0009fd0 tx=0x7f99d0017000).stop 2023-11-15T03:11:45.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.064+0000 7f99e50b9700 1 -- 172.21.15.121:0/2838541088 shutdown_connections 2023-11-15T03:11:45.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.064+0000 7f99e50b9700 1 --2- 172.21.15.121:0/2838541088 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f99c005a510 0x7f99c005c9d0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:45.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.064+0000 7f99e50b9700 1 --2- 172.21.15.121:0/2838541088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99e00a8d40 0x7f99e015e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:45.074 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.064+0000 7f99e50b9700 1 --2- 172.21.15.121:0/2838541088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99e00a8960 0x7f99e015e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:11:45.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.064+0000 7f99e50b9700 1 --2- 172.21.15.121:0/2838541088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99e0066520 0x7f99e0154960 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:45.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.064+0000 7f99e50b9700 1 -- 172.21.15.121:0/2838541088 >> 172.21.15.121:0/2838541088 conn(0x7f99e0067a60 msgr2=0x7f99e00a9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:45.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.064+0000 7f99e50b9700 1 -- 172.21.15.121:0/2838541088 shutdown_connections 2023-11-15T03:11:45.075 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.064+0000 7f99e50b9700 1 -- 172.21.15.121:0/2838541088 wait complete. 2023-11-15T03:11:45.092 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-15T03:11:45.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.244+0000 7f2d455f9700 1 Processor -- start 2023-11-15T03:11:45.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.244+0000 7f2d455f9700 1 -- start start 2023-11-15T03:11:45.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.248+0000 7f2d455f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d4014ac10 0x7f2d400c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:45.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.248+0000 7f2d455f9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2d4014aff0 0x7f2d40066c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:45.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.248+0000 7f2d455f9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d400657f0 con 0x7f2d40067160 2023-11-15T03:11:45.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.248+0000 7f2d455f9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d40066650 con 0x7f2d4014aff0 2023-11-15T03:11:45.259 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.248+0000 7f2d455f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d400675f0 con 0x7f2d4014ac10 2023-11-15T03:11:45.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.248+0000 7f2d37fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2d4014aff0 0x7f2d40066c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:45.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.248+0000 7f2d37fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2d4014aff0 0x7f2d40066c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55854/0 (socket says 172.21.15.121:55854) 2023-11-15T03:11:45.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.248+0000 7f2d37fff700 1 -- 172.21.15.121:0/2827667510 learned_addr learned my addr 172.21.15.121:0/2827667510 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:45.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.248+0000 7f2d3ffff700 1 --2- 172.21.15.121:0/2827667510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d4014ac10 0x7f2d400c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:45.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.252+0000 7f2d37fff700 1 -- 172.21.15.121:0/2827667510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d4014ac10 msgr2=0x7f2d400c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:45.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.252+0000 7f2d37fff700 1 --2- 172.21.15.121:0/2827667510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d4014ac10 0x7f2d400c6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:45.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.252+0000 7f2d37fff700 1 -- 172.21.15.121:0/2827667510 >> v1:172.21.15.190:6789/0 conn(0x7f2d40067160 legacy=0x7f2d40154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:45.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.252+0000 7f2d37fff700 1 -- 172.21.15.121:0/2827667510 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d40065a40 con 0x7f2d4014aff0 2023-11-15T03:11:45.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.252+0000 7f2d37fff700 1 --2- 172.21.15.121:0/2827667510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d4014aff0 0x7f2d40066c70 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 rx=0x7f2d28006fd0 tx=0x7f2d2800b3b0).ready entity=mon.0 client_cookie=705fc9a37fe0af46 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:45.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.252+0000 7f2d3f7fe700 1 -- 172.21.15.121:0/2827667510 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2800da70 con 0x7f2d4014aff0 2023-11-15T03:11:45.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.252+0000 7f2d3f7fe700 1 -- 172.21.15.121:0/2827667510 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2800dbd0 con 0x7f2d4014aff0 2023-11-15T03:11:45.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.252+0000 7f2d3f7fe700 1 -- 172.21.15.121:0/2827667510 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d28010470 con 0x7f2d4014aff0 2023-11-15T03:11:45.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.252+0000 7f2d455f9700 1 -- 172.21.15.121:0/2827667510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d4014aff0 msgr2=0x7f2d40066c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:45.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.252+0000 7f2d455f9700 1 --2- 172.21.15.121:0/2827667510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d4014aff0 0x7f2d40066c70 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 rx=0x7f2d28006fd0 tx=0x7f2d2800b3b0).stop 2023-11-15T03:11:45.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.252+0000 7f2d455f9700 1 -- 172.21.15.121:0/2827667510 shutdown_connections 2023-11-15T03:11:45.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.252+0000 7f2d455f9700 1 --2- 172.21.15.121:0/2827667510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d4014aff0 0x7f2d40066c70 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:45.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.252+0000 7f2d455f9700 1 --2- 172.21.15.121:0/2827667510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d4014ac10 0x7f2d400c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:45.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.252+0000 7f2d455f9700 1 -- 172.21.15.121:0/2827667510 >> 172.21.15.121:0/2827667510 conn(0x7f2d40067a60 msgr2=0x7f2d40067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:45.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.252+0000 7f2d455f9700 1 -- 172.21.15.121:0/2827667510 shutdown_connections 2023-11-15T03:11:45.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.252+0000 7f2d455f9700 1 -- 172.21.15.121:0/2827667510 wait complete. 2023-11-15T03:11:45.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.256+0000 7f2d455f9700 1 Processor -- start 2023-11-15T03:11:45.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.256+0000 7f2d455f9700 1 -- start start 2023-11-15T03:11:45.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.256+0000 7f2d455f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d40067160 0x7f2d400ac080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:45.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.256+0000 7f2d455f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d4014ac10 0x7f2d4013f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:45.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.256+0000 7f2d455f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d4014aff0 0x7f2d40148320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:45.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.256+0000 7f2d455f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d401653f0 con 0x7f2d4014aff0 2023-11-15T03:11:45.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.256+0000 7f2d455f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d400ac570 con 0x7f2d4014ac10 2023-11-15T03:11:45.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.256+0000 7f2d455f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d401647e0 con 0x7f2d40067160 2023-11-15T03:11:45.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.256+0000 7f2d44df8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d4014aff0 0x7f2d40148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:45.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.256+0000 7f2d44df8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d4014aff0 0x7f2d40148320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55862/0 (socket says 172.21.15.121:55862) 2023-11-15T03:11:45.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.256+0000 7f2d37fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d4014ac10 0x7f2d4013f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:45.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.256+0000 7f2d44df8700 1 -- 172.21.15.121:0/1427518463 learned_addr learned my addr 172.21.15.121:0/1427518463 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:45.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.256+0000 7f2d3ffff700 1 --2- 172.21.15.121:0/1427518463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d40067160 0x7f2d400ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:45.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.256+0000 7f2d44df8700 1 -- 172.21.15.121:0/1427518463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d40067160 msgr2=0x7f2d400ac080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:45.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.256+0000 7f2d44df8700 1 --2- 172.21.15.121:0/1427518463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d40067160 0x7f2d400ac080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:45.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.256+0000 7f2d44df8700 1 -- 172.21.15.121:0/1427518463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d4014ac10 msgr2=0x7f2d4013f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:45.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.256+0000 7f2d44df8700 1 --2- 172.21.15.121:0/1427518463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d4014ac10 0x7f2d4013f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:45.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.256+0000 7f2d44df8700 1 -- 172.21.15.121:0/1427518463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2800b610 con 0x7f2d4014aff0 2023-11-15T03:11:45.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.260+0000 7f2d44df8700 1 --2- 172.21.15.121:0/1427518463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d4014aff0 0x7f2d40148320 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 rx=0x7f2d38002230 tx=0x7f2d380015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:45.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.260+0000 7f2d3cff9700 1 -- 172.21.15.121:0/1427518463 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3800e4a0 con 0x7f2d4014aff0 2023-11-15T03:11:45.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.260+0000 7f2d3cff9700 1 -- 172.21.15.121:0/1427518463 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3800e600 con 0x7f2d4014aff0 2023-11-15T03:11:45.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.260+0000 7f2d455f9700 1 -- 172.21.15.121:0/1427518463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d40164ac0 con 0x7f2d4014aff0 2023-11-15T03:11:45.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.260+0000 7f2d3cff9700 1 -- 172.21.15.121:0/1427518463 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3800e980 con 0x7f2d4014aff0 2023-11-15T03:11:45.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.260+0000 7f2d455f9700 1 -- 172.21.15.121:0/1427518463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d40165080 con 0x7f2d4014aff0 2023-11-15T03:11:45.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.260+0000 7f2d455f9700 1 -- 172.21.15.121:0/1427518463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d40167a60 con 0x7f2d4014aff0 2023-11-15T03:11:45.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.264+0000 7f2d3cff9700 1 -- 172.21.15.121:0/1427518463 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2d3801b3b0 con 0x7f2d4014aff0 2023-11-15T03:11:45.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.264+0000 7f2d3cff9700 1 --2- 172.21.15.121:0/1427518463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2d2005a540 0x7f2d2005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:45.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.264+0000 7f2d3cff9700 1 -- 172.21.15.121:0/1427518463 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f2d38073e90 con 0x7f2d4014aff0 2023-11-15T03:11:45.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.264+0000 7f2d3ffff700 1 --2- 172.21.15.121:0/1427518463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2d2005a540 0x7f2d2005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:45.281 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.264+0000 7f2d3ffff700 1 --2- 172.21.15.121:0/1427518463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2d2005a540 0x7f2d2005ca00 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7f2d400c06c0 tx=0x7f2d30009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:45.282 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.268+0000 7f2d3cff9700 1 -- 172.21.15.121:0/1427518463 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2d40167a60 con 0x7f2d4014aff0 2023-11-15T03:11:45.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.432+0000 7f2d455f9700 1 -- 172.21.15.121:0/1427518463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2d40167a60 con 0x7f2d4014aff0 2023-11-15T03:11:45.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.432+0000 7f2d3cff9700 1 -- 172.21.15.121:0/1427518463 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v148) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2d40167a60 con 0x7f2d4014aff0 2023-11-15T03:11:45.442 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2023-11-15T03:11:45.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.432+0000 7f2d455f9700 1 -- 172.21.15.121:0/1427518463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2d2005a540 msgr2=0x7f2d2005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:45.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.432+0000 7f2d455f9700 1 --2- 172.21.15.121:0/1427518463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2d2005a540 0x7f2d2005ca00 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7f2d400c06c0 tx=0x7f2d30009060).stop 2023-11-15T03:11:45.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.436+0000 7f2d455f9700 1 -- 172.21.15.121:0/1427518463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d4014aff0 msgr2=0x7f2d40148320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:45.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.436+0000 7f2d455f9700 1 --2- 172.21.15.121:0/1427518463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d4014aff0 0x7f2d40148320 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 rx=0x7f2d38002230 tx=0x7f2d380015e0).stop 2023-11-15T03:11:45.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.436+0000 7f2d455f9700 1 -- 172.21.15.121:0/1427518463 shutdown_connections 2023-11-15T03:11:45.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.436+0000 7f2d455f9700 1 --2- 172.21.15.121:0/1427518463 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f2d2005a540 0x7f2d2005ca00 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:45.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.436+0000 7f2d455f9700 1 --2- 172.21.15.121:0/1427518463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d4014aff0 0x7f2d40148320 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:45.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.436+0000 7f2d455f9700 1 --2- 172.21.15.121:0/1427518463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d4014ac10 0x7f2d4013f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:45.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.436+0000 7f2d455f9700 1 --2- 172.21.15.121:0/1427518463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d40067160 0x7f2d400ac080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:45.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.436+0000 7f2d455f9700 1 -- 172.21.15.121:0/1427518463 >> 172.21.15.121:0/1427518463 conn(0x7f2d40067a60 msgr2=0x7f2d40141e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:45.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.436+0000 7f2d455f9700 1 -- 172.21.15.121:0/1427518463 shutdown_connections 2023-11-15T03:11:45.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.436+0000 7f2d455f9700 1 -- 172.21.15.121:0/1427518463 wait complete. 2023-11-15T03:11:45.463 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:45.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.616+0000 7f3e67d69700 1 Processor -- start 2023-11-15T03:11:45.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.616+0000 7f3e67d69700 1 -- start start 2023-11-15T03:11:45.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.620+0000 7f3e67d69700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3e600c5e40 0x7f3e600c6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:45.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.620+0000 7f3e67d69700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e600c59b0 con 0x7f3e601500d0 2023-11-15T03:11:45.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.620+0000 7f3e67d69700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e600c4dc0 con 0x7f3e60067a60 2023-11-15T03:11:45.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.620+0000 7f3e67d69700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e600c6770 con 0x7f3e600c5e40 2023-11-15T03:11:45.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.620+0000 7f3e65304700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3e600c5e40 0x7f3e600c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:45.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.620+0000 7f3e65304700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3e600c5e40 0x7f3e600c6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55864/0 (socket says 172.21.15.121:55864) 2023-11-15T03:11:45.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.620+0000 7f3e65304700 1 -- 172.21.15.121:0/1248550651 learned_addr learned my addr 172.21.15.121:0/1248550651 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:45.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.620+0000 7f3e64b03700 1 -- 172.21.15.121:0/1248550651 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3757942333 0 0) 0x7f3e600c59b0 con 0x7f3e601500d0 2023-11-15T03:11:45.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.620+0000 7f3e64b03700 1 -- 172.21.15.121:0/1248550651 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e4c0033f0 con 0x7f3e601500d0 2023-11-15T03:11:45.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.624+0000 7f3e65304700 1 -- 172.21.15.121:0/1248550651 >> v1:172.21.15.190:6789/0 conn(0x7f3e60067a60 legacy=0x7f3e60153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:45.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.624+0000 7f3e65304700 1 -- 172.21.15.121:0/1248550651 >> v1:172.21.15.121:6789/0 conn(0x7f3e601500d0 legacy=0x7f3e600bd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:45.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.624+0000 7f3e65304700 1 -- 172.21.15.121:0/1248550651 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e600c5080 con 0x7f3e600c5e40 2023-11-15T03:11:45.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.624+0000 7f3e65304700 1 --2- 172.21.15.121:0/1248550651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e600c5e40 0x7f3e600c6280 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 rx=0x7f3e50006fd0 tx=0x7f3e500090f0).ready entity=mon.0 client_cookie=4a5c862a5767daa9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:45.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.624+0000 7f3e64b03700 1 -- 172.21.15.121:0/1248550651 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5000b580 con 0x7f3e600c5e40 2023-11-15T03:11:45.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.624+0000 7f3e64b03700 1 -- 172.21.15.121:0/1248550651 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5000b6e0 con 0x7f3e600c5e40 2023-11-15T03:11:45.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.624+0000 7f3e64b03700 1 -- 172.21.15.121:0/1248550651 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5000ba60 con 0x7f3e600c5e40 2023-11-15T03:11:45.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.624+0000 7f3e67d69700 1 -- 172.21.15.121:0/1248550651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e600c5e40 msgr2=0x7f3e600c6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:45.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.624+0000 7f3e67d69700 1 --2- 172.21.15.121:0/1248550651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e600c5e40 0x7f3e600c6280 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 rx=0x7f3e50006fd0 tx=0x7f3e500090f0).stop 2023-11-15T03:11:45.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.624+0000 7f3e67d69700 1 -- 172.21.15.121:0/1248550651 shutdown_connections 2023-11-15T03:11:45.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.624+0000 7f3e67d69700 1 --2- 172.21.15.121:0/1248550651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e600c5e40 0x7f3e600c6280 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:45.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.624+0000 7f3e67d69700 1 -- 172.21.15.121:0/1248550651 >> 172.21.15.121:0/1248550651 conn(0x7f3e600c3af0 msgr2=0x7f3e600c3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:45.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.624+0000 7f3e67d69700 1 -- 172.21.15.121:0/1248550651 shutdown_connections 2023-11-15T03:11:45.642 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.624+0000 7f3e67d69700 1 -- 172.21.15.121:0/1248550651 wait complete. 2023-11-15T03:11:45.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.624+0000 7f3e67d69700 1 Processor -- start 2023-11-15T03:11:45.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e67d69700 1 -- start start 2023-11-15T03:11:45.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e67d69700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60067a60 0x7f3e600bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:45.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e67d69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e600c5e40 0x7f3e600a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:45.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e67d69700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e601500d0 0x7f3e600a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:45.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e67d69700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e600a7c40 con 0x7f3e601500d0 2023-11-15T03:11:45.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e67d69700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e600a7030 con 0x7f3e60067a60 2023-11-15T03:11:45.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e67d69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e600a7310 con 0x7f3e600c5e40 2023-11-15T03:11:45.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e66306700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e601500d0 0x7f3e600a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:45.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e65b05700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60067a60 0x7f3e600bfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:45.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e65304700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e600c5e40 0x7f3e600a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:45.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e66306700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e601500d0 0x7f3e600a42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55870/0 (socket says 172.21.15.121:55870) 2023-11-15T03:11:45.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e65b05700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60067a60 0x7f3e600bfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:58156/0 (socket says 172.21.15.121:58156) 2023-11-15T03:11:45.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e66306700 1 -- 172.21.15.121:0/4092743089 learned_addr learned my addr 172.21.15.121:0/4092743089 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:45.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e66306700 1 -- 172.21.15.121:0/4092743089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e600c5e40 msgr2=0x7f3e600a3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:45.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e66306700 1 --2- 172.21.15.121:0/4092743089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e600c5e40 0x7f3e600a3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:45.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e66306700 1 -- 172.21.15.121:0/4092743089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60067a60 msgr2=0x7f3e600bfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:45.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e66306700 1 --2- 172.21.15.121:0/4092743089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60067a60 0x7f3e600bfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:45.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e66306700 1 -- 172.21.15.121:0/4092743089 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e50009370 con 0x7f3e601500d0 2023-11-15T03:11:45.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e65b05700 1 --2- 172.21.15.121:0/4092743089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60067a60 0x7f3e600bfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:45.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e66306700 1 --2- 172.21.15.121:0/4092743089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e601500d0 0x7f3e600a42f0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 rx=0x7f3e58002230 tx=0x7f3e580015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:45.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e55ffb700 1 -- 172.21.15.121:0/4092743089 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e58010510 con 0x7f3e601500d0 2023-11-15T03:11:45.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e55ffb700 1 -- 172.21.15.121:0/4092743089 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e58010670 con 0x7f3e601500d0 2023-11-15T03:11:45.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e55ffb700 1 -- 172.21.15.121:0/4092743089 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e580109f0 con 0x7f3e601500d0 2023-11-15T03:11:45.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.628+0000 7f3e67d69700 1 -- 172.21.15.121:0/4092743089 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e600a75f0 con 0x7f3e601500d0 2023-11-15T03:11:45.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.632+0000 7f3e67d69700 1 -- 172.21.15.121:0/4092743089 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e600a6b30 con 0x7f3e601500d0 2023-11-15T03:11:45.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.632+0000 7f3e67d69700 1 -- 172.21.15.121:0/4092743089 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e600a6db0 con 0x7f3e601500d0 2023-11-15T03:11:45.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.632+0000 7f3e55ffb700 1 -- 172.21.15.121:0/4092743089 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3e5801b400 con 0x7f3e601500d0 2023-11-15T03:11:45.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.636+0000 7f3e55ffb700 1 --2- 172.21.15.121:0/4092743089 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e4405a4e0 0x7f3e4405c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:45.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.636+0000 7f3e55ffb700 1 -- 172.21.15.121:0/4092743089 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f3e580746b0 con 0x7f3e601500d0 2023-11-15T03:11:45.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.636+0000 7f3e65b05700 1 --2- 172.21.15.121:0/4092743089 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e4405a4e0 0x7f3e4405c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:45.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.636+0000 7f3e65b05700 1 --2- 172.21.15.121:0/4092743089 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e4405a4e0 0x7f3e4405c9a0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f3e5c007fd0 tx=0x7f3e5c007bd0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:45.653 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.640+0000 7f3e55ffb700 1 -- 172.21.15.121:0/4092743089 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3e58047cc0 con 0x7f3e601500d0 2023-11-15T03:11:45.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.840+0000 7f3e67d69700 1 -- 172.21.15.121:0/4092743089 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e6006c150 con 0x7f3e601500d0 2023-11-15T03:11:45.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.844+0000 7f3e55ffb700 1 -- 172.21.15.121:0/4092743089 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 356 v356) v1 ==== 95+0+4514 (secure 0 0 0) 0x7f3e5804b2e0 con 0x7f3e601500d0 2023-11-15T03:11:45.851 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:45.852 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":356,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/755048854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":755048854},{"type":"v1","addr":"172.21.15.190:6834","nonce":755048854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2360242049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2360242049},{"type":"v1","addr":"172.21.15.190:6835","nonce":2360242049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2780153683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2780153683},{"type":"v1","addr":"172.21.15.121:6835","nonce":2780153683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345}],"filesystems":[{"mdsmap":{"epoch":356,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:11:21.864101+0000","modified":"2023-11-15T03:11:43.335433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10899":{"gid":10899,"name":"a","rank":0,"incarnation":352,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6837/3298117008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3298117008},{"type":"v1","addr":"172.21.15.121:6837","nonce":3298117008}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-15T03:11:45.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.848+0000 7f3e67d69700 1 -- 172.21.15.121:0/4092743089 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e4405a4e0 msgr2=0x7f3e4405c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:45.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.848+0000 7f3e67d69700 1 --2- 172.21.15.121:0/4092743089 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e4405a4e0 0x7f3e4405c9a0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f3e5c007fd0 tx=0x7f3e5c007bd0).stop 2023-11-15T03:11:45.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.848+0000 7f3e67d69700 1 -- 172.21.15.121:0/4092743089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e601500d0 msgr2=0x7f3e600a42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:45.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.848+0000 7f3e67d69700 1 --2- 172.21.15.121:0/4092743089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e601500d0 0x7f3e600a42f0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 rx=0x7f3e58002230 tx=0x7f3e580015e0).stop 2023-11-15T03:11:45.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.848+0000 7f3e67d69700 1 -- 172.21.15.121:0/4092743089 shutdown_connections 2023-11-15T03:11:45.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.848+0000 7f3e67d69700 1 --2- 172.21.15.121:0/4092743089 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f3e4405a4e0 0x7f3e4405c9a0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:45.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.848+0000 7f3e67d69700 1 --2- 172.21.15.121:0/4092743089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e601500d0 0x7f3e600a42f0 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:45.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.848+0000 7f3e67d69700 1 --2- 172.21.15.121:0/4092743089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e600c5e40 0x7f3e600a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:45.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.848+0000 7f3e67d69700 1 --2- 172.21.15.121:0/4092743089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60067a60 0x7f3e600bfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:45.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.848+0000 7f3e67d69700 1 -- 172.21.15.121:0/4092743089 >> 172.21.15.121:0/4092743089 conn(0x7f3e600c3af0 msgr2=0x7f3e60062630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:45.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.848+0000 7f3e67d69700 1 -- 172.21.15.121:0/4092743089 shutdown_connections 2023-11-15T03:11:45.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:45.848+0000 7f3e67d69700 1 -- 172.21.15.121:0/4092743089 wait complete. 2023-11-15T03:11:45.859 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 356 2023-11-15T03:11:45.877 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:11:46.042 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.032+0000 7f378e7f9700 1 Processor -- start 2023-11-15T03:11:46.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.032+0000 7f378e7f9700 1 -- start start 2023-11-15T03:11:46.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.036+0000 7f378e7f9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3788151870 0x7f3788151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:46.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.036+0000 7f378e7f9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37880657f0 con 0x7f3788152140 2023-11-15T03:11:46.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.036+0000 7f378e7f9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3788065ad0 con 0x7f37880c30e0 2023-11-15T03:11:46.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.036+0000 7f378e7f9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37880aa260 con 0x7f3788151870 2023-11-15T03:11:46.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.036+0000 7f378d7f7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3788151870 0x7f3788151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:46.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.036+0000 7f378d7f7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3788151870 0x7f3788151c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55876/0 (socket says 172.21.15.121:55876) 2023-11-15T03:11:46.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.036+0000 7f378cff6700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3788152140 0x7f37880a9830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59064/0 (socket says 172.21.15.121:59064) 2023-11-15T03:11:46.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.036+0000 7f378d7f7700 1 -- 172.21.15.121:0/719804100 learned_addr learned my addr 172.21.15.121:0/719804100 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:46.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.036+0000 7f378d7f7700 1 -- 172.21.15.121:0/719804100 >> v1:172.21.15.190:6789/0 conn(0x7f37880c30e0 legacy=0x7f3788065f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:46.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.036+0000 7f378d7f7700 1 -- 172.21.15.121:0/719804100 >> v1:172.21.15.121:6789/0 conn(0x7f3788152140 legacy=0x7f37880a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:46.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.036+0000 7f378d7f7700 1 -- 172.21.15.121:0/719804100 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37880aa540 con 0x7f3788151870 2023-11-15T03:11:46.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.036+0000 7f378d7f7700 1 --2- 172.21.15.121:0/719804100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3788151870 0x7f3788151c50 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 rx=0x7f3784006fd0 tx=0x7f37840090f0).ready entity=mon.0 client_cookie=221b56b159444872 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:46.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.040+0000 7f377ffff700 1 -- 172.21.15.121:0/719804100 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378400b600 con 0x7f3788151870 2023-11-15T03:11:46.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.040+0000 7f377ffff700 1 -- 172.21.15.121:0/719804100 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378400b760 con 0x7f3788151870 2023-11-15T03:11:46.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.040+0000 7f377ffff700 1 -- 172.21.15.121:0/719804100 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378400bae0 con 0x7f3788151870 2023-11-15T03:11:46.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.040+0000 7f378e7f9700 1 -- 172.21.15.121:0/719804100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3788151870 msgr2=0x7f3788151c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:46.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.040+0000 7f378e7f9700 1 --2- 172.21.15.121:0/719804100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3788151870 0x7f3788151c50 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 rx=0x7f3784006fd0 tx=0x7f37840090f0).stop 2023-11-15T03:11:46.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.040+0000 7f378e7f9700 1 -- 172.21.15.121:0/719804100 shutdown_connections 2023-11-15T03:11:46.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.040+0000 7f378e7f9700 1 --2- 172.21.15.121:0/719804100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3788151870 0x7f3788151c50 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.040+0000 7f378e7f9700 1 -- 172.21.15.121:0/719804100 >> 172.21.15.121:0/719804100 conn(0x7f3788062190 msgr2=0x7f3788062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:46.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.040+0000 7f378e7f9700 1 -- 172.21.15.121:0/719804100 shutdown_connections 2023-11-15T03:11:46.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.040+0000 7f378e7f9700 1 -- 172.21.15.121:0/719804100 wait complete. 2023-11-15T03:11:46.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.044+0000 7f378e7f9700 1 Processor -- start 2023-11-15T03:11:46.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.044+0000 7f378e7f9700 1 -- start start 2023-11-15T03:11:46.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.044+0000 7f378e7f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37880c30e0 0x7f3788147fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:46.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.044+0000 7f378e7f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3788151870 0x7f37880c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:46.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.044+0000 7f378e7f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3788152140 0x7f3788149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:46.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.044+0000 7f378e7f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3788148490 con 0x7f3788151870 2023-11-15T03:11:46.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.044+0000 7f378e7f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3788149290 con 0x7f37880c30e0 2023-11-15T03:11:46.056 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.044+0000 7f378e7f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37880c26b0 con 0x7f3788152140 2023-11-15T03:11:46.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.048+0000 7f378cff6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3788151870 0x7f37880c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:46.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.048+0000 7f378cff6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3788151870 0x7f37880c21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55882/0 (socket says 172.21.15.121:55882) 2023-11-15T03:11:46.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.048+0000 7f378cff6700 1 -- 172.21.15.121:0/2955391643 learned_addr learned my addr 172.21.15.121:0/2955391643 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:46.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.048+0000 7f378d7f7700 1 --2- 172.21.15.121:0/2955391643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37880c30e0 0x7f3788147fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:46.058 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.048+0000 7f378dff8700 1 --2- 172.21.15.121:0/2955391643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3788152140 0x7f3788149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:46.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.048+0000 7f378cff6700 1 -- 172.21.15.121:0/2955391643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3788152140 msgr2=0x7f3788149990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:46.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.048+0000 7f378cff6700 1 --2- 172.21.15.121:0/2955391643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3788152140 0x7f3788149990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.048+0000 7f378cff6700 1 -- 172.21.15.121:0/2955391643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37880c30e0 msgr2=0x7f3788147fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:46.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.048+0000 7f378cff6700 1 --2- 172.21.15.121:0/2955391643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37880c30e0 0x7f3788147fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.048+0000 7f378cff6700 1 -- 172.21.15.121:0/2955391643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3784009370 con 0x7f3788151870 2023-11-15T03:11:46.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.048+0000 7f378cff6700 1 --2- 172.21.15.121:0/2955391643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3788151870 0x7f37880c21c0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 rx=0x7f377800c440 tx=0x7f3778007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:46.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.048+0000 7f377d7fa700 1 -- 172.21.15.121:0/2955391643 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37780072e0 con 0x7f3788151870 2023-11-15T03:11:46.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.048+0000 7f377d7fa700 1 -- 172.21.15.121:0/2955391643 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3778007440 con 0x7f3788151870 2023-11-15T03:11:46.060 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.048+0000 7f377d7fa700 1 -- 172.21.15.121:0/2955391643 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37780058f0 con 0x7f3788151870 2023-11-15T03:11:46.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.048+0000 7f378e7f9700 1 -- 172.21.15.121:0/2955391643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3788148680 con 0x7f3788151870 2023-11-15T03:11:46.061 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.048+0000 7f378e7f9700 1 -- 172.21.15.121:0/2955391643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3788148c40 con 0x7f3788151870 2023-11-15T03:11:46.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.048+0000 7f378e7f9700 1 -- 172.21.15.121:0/2955391643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f378806bcf0 con 0x7f3788151870 2023-11-15T03:11:46.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.052+0000 7f377d7fa700 1 -- 172.21.15.121:0/2955391643 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3778005d20 con 0x7f3788151870 2023-11-15T03:11:46.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.052+0000 7f377d7fa700 1 --2- 172.21.15.121:0/2955391643 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f376c05a510 0x7f376c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:46.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.052+0000 7f377d7fa700 1 -- 172.21.15.121:0/2955391643 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f37780740a0 con 0x7f3788151870 2023-11-15T03:11:46.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.052+0000 7f378d7f7700 1 --2- 172.21.15.121:0/2955391643 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f376c05a510 0x7f376c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:46.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.052+0000 7f378d7f7700 1 --2- 172.21.15.121:0/2955391643 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f376c05a510 0x7f376c05c9d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f3784009130 tx=0x7f3784017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:46.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.060+0000 7f377d7fa700 1 -- 172.21.15.121:0/2955391643 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3778047710 con 0x7f3788151870 2023-11-15T03:11:46.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.228+0000 7f378e7f9700 1 -- 172.21.15.121:0/2955391643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f378814fab0 con 0x7f3788151870 2023-11-15T03:11:46.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.228+0000 7f377d7fa700 1 -- 172.21.15.121:0/2955391643 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+17266 (secure 0 0 0) 0x7f377804ad30 con 0x7f3788151870 2023-11-15T03:11:46.237 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:46.237 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":148,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:11:40.308031+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-15T03:11:19.965438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-15T03:11:20.878094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.6","mappings":[{"from":5,"to":4}]},{"pgid":"16.15","mappings":[{"from":1,"to":4}]},{"pgid":"16.2d","mappings":[{"from":0,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6837/2335356851":"2023-11-16T03:11:31.820010+0000","172.21.15.121:6836/2335356851":"2023-11-16T03:11:31.820010+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:11:46.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.232+0000 7f378e7f9700 1 -- 172.21.15.121:0/2955391643 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f376c05a510 msgr2=0x7f376c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:46.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.232+0000 7f378e7f9700 1 --2- 172.21.15.121:0/2955391643 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f376c05a510 0x7f376c05c9d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f3784009130 tx=0x7f3784017000).stop 2023-11-15T03:11:46.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.232+0000 7f378e7f9700 1 -- 172.21.15.121:0/2955391643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3788151870 msgr2=0x7f37880c21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:46.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.232+0000 7f378e7f9700 1 --2- 172.21.15.121:0/2955391643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3788151870 0x7f37880c21c0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 rx=0x7f377800c440 tx=0x7f3778007dc0).stop 2023-11-15T03:11:46.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.232+0000 7f378e7f9700 1 -- 172.21.15.121:0/2955391643 shutdown_connections 2023-11-15T03:11:46.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.232+0000 7f378e7f9700 1 --2- 172.21.15.121:0/2955391643 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f376c05a510 0x7f376c05c9d0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.232+0000 7f378e7f9700 1 --2- 172.21.15.121:0/2955391643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3788152140 0x7f3788149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.232+0000 7f378e7f9700 1 --2- 172.21.15.121:0/2955391643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3788151870 0x7f37880c21c0 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.232+0000 7f378e7f9700 1 --2- 172.21.15.121:0/2955391643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37880c30e0 0x7f3788147fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.232+0000 7f378e7f9700 1 -- 172.21.15.121:0/2955391643 >> 172.21.15.121:0/2955391643 conn(0x7f3788062190 msgr2=0x7f37880a9760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:46.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.232+0000 7f378e7f9700 1 -- 172.21.15.121:0/2955391643 shutdown_connections 2023-11-15T03:11:46.243 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.232+0000 7f378e7f9700 1 -- 172.21.15.121:0/2955391643 wait complete. 2023-11-15T03:11:46.263 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-15T03:11:46.265 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:46.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.428+0000 7fc66a49b700 1 Processor -- start 2023-11-15T03:11:46.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.428+0000 7fc66a49b700 1 -- start start 2023-11-15T03:11:46.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.432+0000 7fc66a49b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc66414af80 0x7fc6640c1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:46.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.432+0000 7fc66a49b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc6640c1bf0 0x7fc664150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:46.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.432+0000 7fc66a49b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc664152000 con 0x7fc6641514e0 2023-11-15T03:11:46.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.432+0000 7fc66a49b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6640c1270 con 0x7fc66414af80 2023-11-15T03:11:46.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.432+0000 7fc66a49b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc664151a40 con 0x7fc6640c1bf0 2023-11-15T03:11:46.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.432+0000 7fc663fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc66414af80 0x7fc6640c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:46.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.432+0000 7fc663fff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc66414af80 0x7fc6640c1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55884/0 (socket says 172.21.15.121:55884) 2023-11-15T03:11:46.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.432+0000 7fc663fff700 1 -- 172.21.15.121:0/1806979410 learned_addr learned my addr 172.21.15.121:0/1806979410 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:46.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.432+0000 7fc65b7fe700 1 --2- 172.21.15.121:0/1806979410 >> v2:172.21.15.190:3300/0 conn(0x7fc6640c1bf0 0x7fc664150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:46.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.432+0000 7fc663fff700 1 -- 172.21.15.121:0/1806979410 >> v2:172.21.15.190:3300/0 conn(0x7fc6640c1bf0 msgr2=0x7fc664150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:46.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.432+0000 7fc663fff700 1 --2- 172.21.15.121:0/1806979410 >> v2:172.21.15.190:3300/0 conn(0x7fc6640c1bf0 0x7fc664150ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.432+0000 7fc663fff700 1 -- 172.21.15.121:0/1806979410 >> v1:172.21.15.190:6789/0 conn(0x7fc6641514e0 legacy=0x7fc6640c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:46.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.432+0000 7fc663fff700 1 -- 172.21.15.121:0/1806979410 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc664151d20 con 0x7fc66414af80 2023-11-15T03:11:46.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.436+0000 7fc663fff700 1 --2- 172.21.15.121:0/1806979410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc66414af80 0x7fc6640c1700 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 rx=0x7fc6640c7590 tx=0x7fc6540090f0).ready entity=mon.0 client_cookie=cc135fa3e78df33a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:46.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.436+0000 7fc6637fe700 1 -- 172.21.15.121:0/1806979410 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65400b600 con 0x7fc66414af80 2023-11-15T03:11:46.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.436+0000 7fc6637fe700 1 -- 172.21.15.121:0/1806979410 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65400b760 con 0x7fc66414af80 2023-11-15T03:11:46.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.436+0000 7fc6637fe700 1 -- 172.21.15.121:0/1806979410 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65400bae0 con 0x7fc66414af80 2023-11-15T03:11:46.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.436+0000 7fc66a49b700 1 -- 172.21.15.121:0/1806979410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc66414af80 msgr2=0x7fc6640c1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:46.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.436+0000 7fc66a49b700 1 --2- 172.21.15.121:0/1806979410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc66414af80 0x7fc6640c1700 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 rx=0x7fc6640c7590 tx=0x7fc6540090f0).stop 2023-11-15T03:11:46.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.436+0000 7fc66a49b700 1 -- 172.21.15.121:0/1806979410 shutdown_connections 2023-11-15T03:11:46.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.436+0000 7fc66a49b700 1 --2- 172.21.15.121:0/1806979410 >> v2:172.21.15.190:3300/0 conn(0x7fc6640c1bf0 0x7fc664150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.448 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.436+0000 7fc66a49b700 1 --2- 172.21.15.121:0/1806979410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc66414af80 0x7fc6640c1700 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.436+0000 7fc66a49b700 1 -- 172.21.15.121:0/1806979410 >> 172.21.15.121:0/1806979410 conn(0x7fc664062190 msgr2=0x7fc6640625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:46.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.436+0000 7fc66a49b700 1 -- 172.21.15.121:0/1806979410 shutdown_connections 2023-11-15T03:11:46.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.436+0000 7fc66a49b700 1 -- 172.21.15.121:0/1806979410 wait complete. 2023-11-15T03:11:46.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.436+0000 7fc66a49b700 1 Processor -- start 2023-11-15T03:11:46.449 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc66a49b700 1 -- start start 2023-11-15T03:11:46.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc66a49b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6640c1bf0 0x7fc664163ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:46.450 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc66a49b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc66414af80 0x7fc66414b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:46.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc66a49b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6641514e0 0x7fc66414b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:46.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc66a49b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66414bd40 con 0x7fc6641514e0 2023-11-15T03:11:46.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc66a49b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc664164190 con 0x7fc66414af80 2023-11-15T03:11:46.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc66a49b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66414e230 con 0x7fc6640c1bf0 2023-11-15T03:11:46.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc668a38700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6641514e0 0x7fc66414b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:46.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc668a38700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6641514e0 0x7fc66414b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55896/0 (socket says 172.21.15.121:55896) 2023-11-15T03:11:46.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc668a38700 1 -- 172.21.15.121:0/2203372089 learned_addr learned my addr 172.21.15.121:0/2203372089 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:46.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc663fff700 1 --2- 172.21.15.121:0/2203372089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6640c1bf0 0x7fc664163ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:46.453 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc65b7fe700 1 --2- 172.21.15.121:0/2203372089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc66414af80 0x7fc66414b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:46.454 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc668a38700 1 -- 172.21.15.121:0/2203372089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6640c1bf0 msgr2=0x7fc664163ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:46.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc668a38700 1 --2- 172.21.15.121:0/2203372089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6640c1bf0 0x7fc664163ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc668a38700 1 -- 172.21.15.121:0/2203372089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc66414af80 msgr2=0x7fc66414b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:46.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc668a38700 1 --2- 172.21.15.121:0/2203372089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc66414af80 0x7fc66414b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc668a38700 1 -- 172.21.15.121:0/2203372089 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc654009370 con 0x7fc6641514e0 2023-11-15T03:11:46.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc668a38700 1 --2- 172.21.15.121:0/2203372089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6641514e0 0x7fc66414b850 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 rx=0x7fc65c00c470 tx=0x7fc65c007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:46.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc660ff9700 1 -- 172.21.15.121:0/2203372089 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c0123f0 con 0x7fc6641514e0 2023-11-15T03:11:46.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc660ff9700 1 -- 172.21.15.121:0/2203372089 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c012550 con 0x7fc6641514e0 2023-11-15T03:11:46.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc66a49b700 1 -- 172.21.15.121:0/2203372089 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc66414e510 con 0x7fc6641514e0 2023-11-15T03:11:46.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc660ff9700 1 -- 172.21.15.121:0/2203372089 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c0128d0 con 0x7fc6641514e0 2023-11-15T03:11:46.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.440+0000 7fc66a49b700 1 -- 172.21.15.121:0/2203372089 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66413cf00 con 0x7fc6641514e0 2023-11-15T03:11:46.458 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.444+0000 7fc66a49b700 1 -- 172.21.15.121:0/2203372089 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66406c150 con 0x7fc6641514e0 2023-11-15T03:11:46.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.444+0000 7fc660ff9700 1 -- 172.21.15.121:0/2203372089 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc65c012cc0 con 0x7fc6641514e0 2023-11-15T03:11:46.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.448+0000 7fc660ff9700 1 --2- 172.21.15.121:0/2203372089 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc64405a510 0x7fc64405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:46.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.448+0000 7fc660ff9700 1 -- 172.21.15.121:0/2203372089 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7fc65c073b90 con 0x7fc6641514e0 2023-11-15T03:11:46.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.448+0000 7fc663fff700 1 --2- 172.21.15.121:0/2203372089 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc64405a510 0x7fc64405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:46.459 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.448+0000 7fc663fff700 1 --2- 172.21.15.121:0/2203372089 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc64405a510 0x7fc64405c9d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7fc654009130 tx=0x7fc654017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:46.462 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.452+0000 7fc660ff9700 1 -- 172.21.15.121:0/2203372089 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc65c047200 con 0x7fc6641514e0 2023-11-15T03:11:46.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.648+0000 7fc66a49b700 1 -- 172.21.15.121:0/2203372089 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc664152000 con 0x7fc6641514e0 2023-11-15T03:11:46.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.648+0000 7fc660ff9700 1 -- 172.21.15.121:0/2203372089 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 356 v356) v1 ==== 95+0+4514 (secure 0 0 0) 0x7fc664152000 con 0x7fc6641514e0 2023-11-15T03:11:46.658 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:46.658 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":356,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/755048854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":755048854},{"type":"v1","addr":"172.21.15.190:6834","nonce":755048854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2360242049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2360242049},{"type":"v1","addr":"172.21.15.190:6835","nonce":2360242049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2780153683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2780153683},{"type":"v1","addr":"172.21.15.121:6835","nonce":2780153683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345}],"filesystems":[{"mdsmap":{"epoch":356,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:11:21.864101+0000","modified":"2023-11-15T03:11:43.335433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10899":{"gid":10899,"name":"a","rank":0,"incarnation":352,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6837/3298117008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3298117008},{"type":"v1","addr":"172.21.15.121:6837","nonce":3298117008}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-15T03:11:46.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.652+0000 7fc66a49b700 1 -- 172.21.15.121:0/2203372089 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc64405a510 msgr2=0x7fc64405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:46.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.652+0000 7fc66a49b700 1 --2- 172.21.15.121:0/2203372089 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc64405a510 0x7fc64405c9d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7fc654009130 tx=0x7fc654017000).stop 2023-11-15T03:11:46.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.652+0000 7fc66a49b700 1 -- 172.21.15.121:0/2203372089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6641514e0 msgr2=0x7fc66414b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:46.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.652+0000 7fc66a49b700 1 --2- 172.21.15.121:0/2203372089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6641514e0 0x7fc66414b850 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 rx=0x7fc65c00c470 tx=0x7fc65c007dd0).stop 2023-11-15T03:11:46.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.652+0000 7fc66a49b700 1 -- 172.21.15.121:0/2203372089 shutdown_connections 2023-11-15T03:11:46.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.652+0000 7fc66a49b700 1 --2- 172.21.15.121:0/2203372089 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fc64405a510 0x7fc64405c9d0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.652+0000 7fc66a49b700 1 --2- 172.21.15.121:0/2203372089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6641514e0 0x7fc66414b850 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.652+0000 7fc66a49b700 1 --2- 172.21.15.121:0/2203372089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc66414af80 0x7fc66414b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.652+0000 7fc66a49b700 1 --2- 172.21.15.121:0/2203372089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6640c1bf0 0x7fc664163ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.652+0000 7fc66a49b700 1 -- 172.21.15.121:0/2203372089 >> 172.21.15.121:0/2203372089 conn(0x7fc664062190 msgr2=0x7fc6640a8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:46.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.652+0000 7fc66a49b700 1 -- 172.21.15.121:0/2203372089 shutdown_connections 2023-11-15T03:11:46.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.652+0000 7fc66a49b700 1 -- 172.21.15.121:0/2203372089 wait complete. 2023-11-15T03:11:46.666 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 356 2023-11-15T03:11:46.683 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:11:46.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.836+0000 7fb98ef1b700 1 Processor -- start 2023-11-15T03:11:46.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.836+0000 7fb98ef1b700 1 -- start start 2023-11-15T03:11:46.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.836+0000 7fb98ef1b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb988153a10 0x7fb9880a8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:46.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.836+0000 7fb98ef1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9880a93d0 0x7fb988157830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:46.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb98ef1b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9880c3a30 con 0x7fb988153df0 2023-11-15T03:11:46.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb98ef1b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9880c7a20 con 0x7fb988153a10 2023-11-15T03:11:46.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb98ef1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9880c7d00 con 0x7fb9880a93d0 2023-11-15T03:11:46.849 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb98ccb7700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb988153a10 0x7fb9880a8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:46.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb98d4b8700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb988153df0 0x7fb9880c3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59068/0 (socket says 172.21.15.121:59068) 2023-11-15T03:11:46.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb98d4b8700 1 -- 172.21.15.121:0/2961704554 learned_addr learned my addr 172.21.15.121:0/2961704554 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:46.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb97ffff700 1 --2- 172.21.15.121:0/2961704554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9880a93d0 0x7fb988157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:46.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb97f7fe700 1 -- 172.21.15.121:0/2961704554 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999666211 0 0) 0x7fb9880c3a30 con 0x7fb988153df0 2023-11-15T03:11:46.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb97f7fe700 1 -- 172.21.15.121:0/2961704554 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9700033f0 con 0x7fb988153df0 2023-11-15T03:11:46.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb98ccb7700 1 -- 172.21.15.121:0/2961704554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9880a93d0 msgr2=0x7fb988157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:46.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb98ccb7700 1 --2- 172.21.15.121:0/2961704554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9880a93d0 0x7fb988157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.851 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb98ccb7700 1 -- 172.21.15.121:0/2961704554 >> v1:172.21.15.121:6789/0 conn(0x7fb988153df0 legacy=0x7fb9880c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:46.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb98ccb7700 1 -- 172.21.15.121:0/2961704554 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9880c33e0 con 0x7fb988153a10 2023-11-15T03:11:46.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb98ccb7700 1 --2- 172.21.15.121:0/2961704554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb988153a10 0x7fb9880a8ee0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 rx=0x7fb984009760 tx=0x7fb984009060).ready entity=mon.0 client_cookie=79f2bde27757904e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:46.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb97f7fe700 1 -- 172.21.15.121:0/2961704554 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98400d4a0 con 0x7fb988153a10 2023-11-15T03:11:46.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb97f7fe700 1 -- 172.21.15.121:0/2961704554 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98400d600 con 0x7fb988153a10 2023-11-15T03:11:46.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb97f7fe700 1 -- 172.21.15.121:0/2961704554 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98400d980 con 0x7fb988153a10 2023-11-15T03:11:46.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb98ef1b700 1 -- 172.21.15.121:0/2961704554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb988153a10 msgr2=0x7fb9880a8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:46.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb98ef1b700 1 --2- 172.21.15.121:0/2961704554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb988153a10 0x7fb9880a8ee0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 rx=0x7fb984009760 tx=0x7fb984009060).stop 2023-11-15T03:11:46.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb98ef1b700 1 -- 172.21.15.121:0/2961704554 shutdown_connections 2023-11-15T03:11:46.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb98ef1b700 1 --2- 172.21.15.121:0/2961704554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9880a93d0 0x7fb988157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.854 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb98ef1b700 1 --2- 172.21.15.121:0/2961704554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb988153a10 0x7fb9880a8ee0 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb98ef1b700 1 -- 172.21.15.121:0/2961704554 >> 172.21.15.121:0/2961704554 conn(0x7fb9880c8960 msgr2=0x7fb9880c8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:46.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.840+0000 7fb98ef1b700 1 -- 172.21.15.121:0/2961704554 shutdown_connections 2023-11-15T03:11:46.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.844+0000 7fb98ef1b700 1 -- 172.21.15.121:0/2961704554 wait complete. 2023-11-15T03:11:46.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.844+0000 7fb98ef1b700 1 Processor -- start 2023-11-15T03:11:46.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.844+0000 7fb98ef1b700 1 -- start start 2023-11-15T03:11:46.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb98ef1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9880a93d0 0x7fb9880c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:46.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb98ef1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb988153a10 0x7fb98814fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:46.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb98ef1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb988153df0 0x7fb988164d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:46.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb98ef1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9880ae7c0 con 0x7fb9880a93d0 2023-11-15T03:11:46.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb98ef1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9880c53a0 con 0x7fb988153a10 2023-11-15T03:11:46.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb98ef1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9880c4790 con 0x7fb988153df0 2023-11-15T03:11:46.859 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb98ccb7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9880a93d0 0x7fb9880c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:46.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb98ccb7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9880a93d0 0x7fb9880c5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55924/0 (socket says 172.21.15.121:55924) 2023-11-15T03:11:46.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb98ccb7700 1 -- 172.21.15.121:0/2610511114 learned_addr learned my addr 172.21.15.121:0/2610511114 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:46.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb97ffff700 1 --2- 172.21.15.121:0/2610511114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb988153a10 0x7fb98814fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:46.860 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb98d4b8700 1 --2- 172.21.15.121:0/2610511114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb988153df0 0x7fb988164d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:46.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb98ccb7700 1 -- 172.21.15.121:0/2610511114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb988153df0 msgr2=0x7fb988164d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:46.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb98ccb7700 1 --2- 172.21.15.121:0/2610511114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb988153df0 0x7fb988164d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb98ccb7700 1 -- 172.21.15.121:0/2610511114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb988153a10 msgr2=0x7fb98814fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:46.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb98ccb7700 1 --2- 172.21.15.121:0/2610511114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb988153a10 0x7fb98814fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:46.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb98ccb7700 1 -- 172.21.15.121:0/2610511114 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9840092a0 con 0x7fb9880a93d0 2023-11-15T03:11:46.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb98ccb7700 1 --2- 172.21.15.121:0/2610511114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9880a93d0 0x7fb9880c5830 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 rx=0x7fb984011fb0 tx=0x7fb984016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:46.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb97cff9700 1 -- 172.21.15.121:0/2610511114 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98400dd30 con 0x7fb9880a93d0 2023-11-15T03:11:46.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb97cff9700 1 -- 172.21.15.121:0/2610511114 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98400de90 con 0x7fb9880a93d0 2023-11-15T03:11:46.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb98ef1b700 1 -- 172.21.15.121:0/2610511114 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9880c4a70 con 0x7fb9880a93d0 2023-11-15T03:11:46.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb97cff9700 1 -- 172.21.15.121:0/2610511114 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98400d980 con 0x7fb9880a93d0 2023-11-15T03:11:46.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.848+0000 7fb98ef1b700 1 -- 172.21.15.121:0/2610511114 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9880c5030 con 0x7fb9880a93d0 2023-11-15T03:11:46.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.852+0000 7fb98ef1b700 1 -- 172.21.15.121:0/2610511114 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98806c150 con 0x7fb9880a93d0 2023-11-15T03:11:46.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.852+0000 7fb97cff9700 1 -- 172.21.15.121:0/2610511114 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb98401d640 con 0x7fb9880a93d0 2023-11-15T03:11:46.864 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.852+0000 7fb97cff9700 1 --2- 172.21.15.121:0/2610511114 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb97405a510 0x7fb97405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:46.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.852+0000 7fb97cff9700 1 -- 172.21.15.121:0/2610511114 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7fb984075db0 con 0x7fb9880a93d0 2023-11-15T03:11:46.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.852+0000 7fb97ffff700 1 --2- 172.21.15.121:0/2610511114 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb97405a510 0x7fb97405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:46.865 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.856+0000 7fb97ffff700 1 --2- 172.21.15.121:0/2610511114 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb97405a510 0x7fb97405c9d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7fb9880a8630 tx=0x7fb978009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:46.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:46.860+0000 7fb97cff9700 1 -- 172.21.15.121:0/2610511114 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb9840494e0 con 0x7fb9880a93d0 2023-11-15T03:11:47.062 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.052+0000 7fb98ef1b700 1 -- 172.21.15.121:0/2610511114 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9880c36c0 con 0x7fb9880a93d0 2023-11-15T03:11:47.063 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.056+0000 7fb97cff9700 1 -- 172.21.15.121:0/2610511114 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 356 v356) v1 ==== 95+0+4514 (secure 0 0 0) 0x7fb98404cb00 con 0x7fb9880a93d0 2023-11-15T03:11:47.064 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:47.064 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":356,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/755048854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":755048854},{"type":"v1","addr":"172.21.15.190:6834","nonce":755048854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2360242049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2360242049},{"type":"v1","addr":"172.21.15.190:6835","nonce":2360242049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2780153683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2780153683},{"type":"v1","addr":"172.21.15.121:6835","nonce":2780153683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345}],"filesystems":[{"mdsmap":{"epoch":356,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-15T03:11:21.864101+0000","modified":"2023-11-15T03:11:43.335433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":146,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10899":{"gid":10899,"name":"a","rank":0,"incarnation":352,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6837/3298117008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3298117008},{"type":"v1","addr":"172.21.15.121:6837","nonce":3298117008}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-15T03:11:47.067 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.060+0000 7fb98ef1b700 1 -- 172.21.15.121:0/2610511114 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb97405a510 msgr2=0x7fb97405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:47.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.060+0000 7fb98ef1b700 1 --2- 172.21.15.121:0/2610511114 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb97405a510 0x7fb97405c9d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7fb9880a8630 tx=0x7fb978009060).stop 2023-11-15T03:11:47.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.060+0000 7fb98ef1b700 1 -- 172.21.15.121:0/2610511114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9880a93d0 msgr2=0x7fb9880c5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:47.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.060+0000 7fb98ef1b700 1 --2- 172.21.15.121:0/2610511114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9880a93d0 0x7fb9880c5830 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 rx=0x7fb984011fb0 tx=0x7fb984016000).stop 2023-11-15T03:11:47.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.060+0000 7fb98ef1b700 1 -- 172.21.15.121:0/2610511114 shutdown_connections 2023-11-15T03:11:47.068 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.060+0000 7fb98ef1b700 1 --2- 172.21.15.121:0/2610511114 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb97405a510 0x7fb97405c9d0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:47.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.060+0000 7fb98ef1b700 1 --2- 172.21.15.121:0/2610511114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb988153df0 0x7fb988164d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:47.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.060+0000 7fb98ef1b700 1 --2- 172.21.15.121:0/2610511114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb988153a10 0x7fb98814fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:47.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.060+0000 7fb98ef1b700 1 --2- 172.21.15.121:0/2610511114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9880a93d0 0x7fb9880c5830 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:47.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.060+0000 7fb98ef1b700 1 -- 172.21.15.121:0/2610511114 >> 172.21.15.121:0/2610511114 conn(0x7fb9880c8960 msgr2=0x7fb9880bff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:47.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.060+0000 7fb98ef1b700 1 -- 172.21.15.121:0/2610511114 shutdown_connections 2023-11-15T03:11:47.070 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.060+0000 7fb98ef1b700 1 -- 172.21.15.121:0/2610511114 wait complete. 2023-11-15T03:11:47.071 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 356 2023-11-15T03:11:47.089 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:11:47.247 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.240+0000 7f5877369700 1 Processor -- start 2023-11-15T03:11:47.248 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.240+0000 7f5877369700 1 -- start start 2023-11-15T03:11:47.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.240+0000 7f5877369700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5870153bb0 0x7f58700a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:47.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.240+0000 7f5877369700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58700aa020 0x7f5870067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:47.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5877369700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58700cb5c0 con 0x7f5870067f50 2023-11-15T03:11:47.251 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5877369700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58700aa6f0 con 0x7f5870153bb0 2023-11-15T03:11:47.252 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5877369700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58700c5900 con 0x7f58700aa020 2023-11-15T03:11:47.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5875105700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5870153bb0 0x7f58700a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:47.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5875906700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f5870067f50 0x7f58701578a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59070/0 (socket says 172.21.15.121:59070) 2023-11-15T03:11:47.253 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5875906700 1 -- 172.21.15.121:0/4220598813 learned_addr learned my addr 172.21.15.121:0/4220598813 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:47.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5874904700 1 --2- 172.21.15.121:0/4220598813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58700aa020 0x7f5870067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:47.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5867fff700 1 -- 172.21.15.121:0/4220598813 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406750169 0 0) 0x7f58700cb5c0 con 0x7f5870067f50 2023-11-15T03:11:47.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5867fff700 1 -- 172.21.15.121:0/4220598813 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f585c0033f0 con 0x7f5870067f50 2023-11-15T03:11:47.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5874904700 1 -- 172.21.15.121:0/4220598813 >> v2:172.21.15.121:3300/0 conn(0x7f5870153bb0 msgr2=0x7f58700a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:47.254 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5874904700 1 --2- 172.21.15.121:0/4220598813 >> v2:172.21.15.121:3300/0 conn(0x7f5870153bb0 0x7f58700a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:47.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5874904700 1 -- 172.21.15.121:0/4220598813 >> v1:172.21.15.121:6789/0 conn(0x7f5870067f50 legacy=0x7f58701578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:47.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5874904700 1 -- 172.21.15.121:0/4220598813 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58700683e0 con 0x7f58700aa020 2023-11-15T03:11:47.255 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5874904700 1 --2- 172.21.15.121:0/4220598813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58700aa020 0x7f5870067a60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f5860009760 tx=0x7f5860009060).ready entity=mon.2 client_cookie=5b753f31ef3c3edd server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:47.256 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5867fff700 1 -- 172.21.15.121:0/4220598813 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586000d4a0 con 0x7f58700aa020 2023-11-15T03:11:47.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5867fff700 1 -- 172.21.15.121:0/4220598813 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586000d600 con 0x7f58700aa020 2023-11-15T03:11:47.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5867fff700 1 -- 172.21.15.121:0/4220598813 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586000d980 con 0x7f58700aa020 2023-11-15T03:11:47.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5877369700 1 -- 172.21.15.121:0/4220598813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58700aa020 msgr2=0x7f5870067a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:47.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5877369700 1 --2- 172.21.15.121:0/4220598813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58700aa020 0x7f5870067a60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f5860009760 tx=0x7f5860009060).stop 2023-11-15T03:11:47.257 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5877369700 1 -- 172.21.15.121:0/4220598813 shutdown_connections 2023-11-15T03:11:47.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5877369700 1 --2- 172.21.15.121:0/4220598813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58700aa020 0x7f5870067a60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:47.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5877369700 1 --2- 172.21.15.121:0/4220598813 >> v2:172.21.15.121:3300/0 conn(0x7f5870153bb0 0x7f58700a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:47.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.244+0000 7f5877369700 1 -- 172.21.15.121:0/4220598813 >> 172.21.15.121:0/4220598813 conn(0x7f5870058c50 msgr2=0x7f5870059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:47.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.248+0000 7f5877369700 1 -- 172.21.15.121:0/4220598813 shutdown_connections 2023-11-15T03:11:47.258 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.248+0000 7f5877369700 1 -- 172.21.15.121:0/4220598813 wait complete. 2023-11-15T03:11:47.260 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.248+0000 7f5877369700 1 Processor -- start 2023-11-15T03:11:47.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.248+0000 7f5877369700 1 -- start start 2023-11-15T03:11:47.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.248+0000 7f5877369700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5870067f50 0x7f5870152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:47.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.248+0000 7f5877369700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58700aa020 0x7f58700a4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:47.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.248+0000 7f5877369700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5870153bb0 0x7f5870165330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:47.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.248+0000 7f5877369700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58700ad910 con 0x7f58700aa020 2023-11-15T03:11:47.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.248+0000 7f5877369700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5870164100 con 0x7f5870153bb0 2023-11-15T03:11:47.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.248+0000 7f5877369700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58700acd90 con 0x7f5870067f50 2023-11-15T03:11:47.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.252+0000 7f5874904700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58700aa020 0x7f58700a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:47.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.252+0000 7f5874904700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58700aa020 0x7f58700a4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55952/0 (socket says 172.21.15.121:55952) 2023-11-15T03:11:47.262 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.252+0000 7f5874904700 1 -- 172.21.15.121:0/1901759824 learned_addr learned my addr 172.21.15.121:0/1901759824 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:47.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.252+0000 7f5875105700 1 --2- 172.21.15.121:0/1901759824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5870067f50 0x7f5870152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:47.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.252+0000 7f5875906700 1 --2- 172.21.15.121:0/1901759824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5870153bb0 0x7f5870165330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:47.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.252+0000 7f5874904700 1 -- 172.21.15.121:0/1901759824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5870067f50 msgr2=0x7f5870152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:47.263 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.252+0000 7f5874904700 1 --2- 172.21.15.121:0/1901759824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5870067f50 0x7f5870152c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:47.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.252+0000 7f5874904700 1 -- 172.21.15.121:0/1901759824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5870153bb0 msgr2=0x7f5870165330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:47.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.252+0000 7f5874904700 1 --2- 172.21.15.121:0/1901759824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5870153bb0 0x7f5870165330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:47.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.252+0000 7f5874904700 1 -- 172.21.15.121:0/1901759824 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58600092a0 con 0x7f58700aa020 2023-11-15T03:11:47.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.252+0000 7f5875906700 1 --2- 172.21.15.121:0/1901759824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5870153bb0 0x7f5870165330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:47.264 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.252+0000 7f5874904700 1 --2- 172.21.15.121:0/1901759824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58700aa020 0x7f58700a4190 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 rx=0x7f5860011fb0 tx=0x7f5860016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:47.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.252+0000 7f58657fa700 1 -- 172.21.15.121:0/1901759824 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5860000c90 con 0x7f58700aa020 2023-11-15T03:11:47.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.252+0000 7f58657fa700 1 -- 172.21.15.121:0/1901759824 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586000d770 con 0x7f58700aa020 2023-11-15T03:11:47.265 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.252+0000 7f58657fa700 1 -- 172.21.15.121:0/1901759824 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586001d430 con 0x7f58700aa020 2023-11-15T03:11:47.266 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.252+0000 7f5877369700 1 -- 172.21.15.121:0/1901759824 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58700ad070 con 0x7f58700aa020 2023-11-15T03:11:47.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.252+0000 7f5877369700 1 -- 172.21.15.121:0/1901759824 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58700ad630 con 0x7f58700aa020 2023-11-15T03:11:47.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.252+0000 7f5877369700 1 -- 172.21.15.121:0/1901759824 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58701642d0 con 0x7f58700aa020 2023-11-15T03:11:47.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.256+0000 7f58657fa700 1 -- 172.21.15.121:0/1901759824 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f586001d7a0 con 0x7f58700aa020 2023-11-15T03:11:47.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.256+0000 7f58657fa700 1 --2- 172.21.15.121:0/1901759824 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f585405a510 0x7f585405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:47.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.256+0000 7f58657fa700 1 -- 172.21.15.121:0/1901759824 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f58600761a0 con 0x7f58700aa020 2023-11-15T03:11:47.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.256+0000 7f5875105700 1 --2- 172.21.15.121:0/1901759824 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f585405a510 0x7f585405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:47.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.260+0000 7f5875105700 1 --2- 172.21.15.121:0/1901759824 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f585405a510 0x7f585405c9d0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7f5870167690 tx=0x7f586c009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:47.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.264+0000 7f58657fa700 1 -- 172.21.15.121:0/1901759824 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f58701642d0 con 0x7f58700aa020 2023-11-15T03:11:47.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.428+0000 7f5877369700 1 -- 172.21.15.121:0/1901759824 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58701642d0 con 0x7f58700aa020 2023-11-15T03:11:47.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.432+0000 7f58657fa700 1 -- 172.21.15.121:0/1901759824 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+17266 (secure 0 0 0) 0x7f58701642d0 con 0x7f58700aa020 2023-11-15T03:11:47.440 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:47.440 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":148,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:11:40.308031+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-15T03:11:19.965438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-15T03:11:20.878094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.6","mappings":[{"from":5,"to":4}]},{"pgid":"16.15","mappings":[{"from":1,"to":4}]},{"pgid":"16.2d","mappings":[{"from":0,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6837/2335356851":"2023-11-16T03:11:31.820010+0000","172.21.15.121:6836/2335356851":"2023-11-16T03:11:31.820010+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:11:47.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.432+0000 7f5877369700 1 -- 172.21.15.121:0/1901759824 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f585405a510 msgr2=0x7f585405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:47.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.432+0000 7f5877369700 1 --2- 172.21.15.121:0/1901759824 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f585405a510 0x7f585405c9d0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7f5870167690 tx=0x7f586c009060).stop 2023-11-15T03:11:47.442 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.432+0000 7f5877369700 1 -- 172.21.15.121:0/1901759824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58700aa020 msgr2=0x7f58700a4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:47.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.432+0000 7f5877369700 1 --2- 172.21.15.121:0/1901759824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58700aa020 0x7f58700a4190 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 rx=0x7f5860011fb0 tx=0x7f5860016000).stop 2023-11-15T03:11:47.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.432+0000 7f5877369700 1 -- 172.21.15.121:0/1901759824 shutdown_connections 2023-11-15T03:11:47.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.432+0000 7f5877369700 1 --2- 172.21.15.121:0/1901759824 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f585405a510 0x7f585405c9d0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:47.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.432+0000 7f5877369700 1 --2- 172.21.15.121:0/1901759824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5870153bb0 0x7f5870165330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:47.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.432+0000 7f5877369700 1 --2- 172.21.15.121:0/1901759824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58700aa020 0x7f58700a4190 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:47.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.432+0000 7f5877369700 1 --2- 172.21.15.121:0/1901759824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5870067f50 0x7f5870152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:47.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.432+0000 7f5877369700 1 -- 172.21.15.121:0/1901759824 >> 172.21.15.121:0/1901759824 conn(0x7f5870058c50 msgr2=0x7f58700bfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:47.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.432+0000 7f5877369700 1 -- 172.21.15.121:0/1901759824 shutdown_connections 2023-11-15T03:11:47.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.432+0000 7f5877369700 1 -- 172.21.15.121:0/1901759824 wait complete. 2023-11-15T03:11:47.464 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-15T03:11:47.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.616+0000 7f23a649b700 1 Processor -- start 2023-11-15T03:11:47.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.620+0000 7f23a649b700 1 -- start start 2023-11-15T03:11:47.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.620+0000 7f23a649b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f23a0157e30 0x7f23a0158210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:47.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.620+0000 7f23a649b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a00ac420 0x7f23a00667f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:47.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.620+0000 7f23a649b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a00aaf80 con 0x7f23a0062190 2023-11-15T03:11:47.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.620+0000 7f23a649b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a00aa970 con 0x7f23a0157e30 2023-11-15T03:11:47.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.620+0000 7f23a649b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0066e10 con 0x7f23a00ac420 2023-11-15T03:11:47.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.620+0000 7f239ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f23a0157e30 0x7f23a0158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:47.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.620+0000 7f239ffff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f23a0157e30 0x7f23a0158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55964/0 (socket says 172.21.15.121:55964) 2023-11-15T03:11:47.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.620+0000 7f239ffff700 1 -- 172.21.15.121:0/3502130107 learned_addr learned my addr 172.21.15.121:0/3502130107 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:47.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.620+0000 7f23a4a38700 1 --2- 172.21.15.121:0/3502130107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a00ac420 0x7f23a00667f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:47.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.620+0000 7f239ffff700 1 -- 172.21.15.121:0/3502130107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a00ac420 msgr2=0x7f23a00667f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:47.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.620+0000 7f239ffff700 1 --2- 172.21.15.121:0/3502130107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a00ac420 0x7f23a00667f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:47.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.620+0000 7f239ffff700 1 -- 172.21.15.121:0/3502130107 >> v1:172.21.15.190:6789/0 conn(0x7f23a0062190 legacy=0x7f23a00660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:47.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.620+0000 7f239ffff700 1 -- 172.21.15.121:0/3502130107 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a00670f0 con 0x7f23a0157e30 2023-11-15T03:11:47.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.624+0000 7f239ffff700 1 --2- 172.21.15.121:0/3502130107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23a0157e30 0x7f23a0158210 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 rx=0x7f2390009f20 tx=0x7f2390009840).ready entity=mon.0 client_cookie=46578fc323e1a61e server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:47.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.624+0000 7f239effd700 1 -- 172.21.15.121:0/3502130107 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239000e070 con 0x7f23a0157e30 2023-11-15T03:11:47.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.624+0000 7f239effd700 1 -- 172.21.15.121:0/3502130107 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239000b9d0 con 0x7f23a0157e30 2023-11-15T03:11:47.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.624+0000 7f239effd700 1 -- 172.21.15.121:0/3502130107 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239000bd50 con 0x7f23a0157e30 2023-11-15T03:11:47.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.624+0000 7f23a649b700 1 -- 172.21.15.121:0/3502130107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23a0157e30 msgr2=0x7f23a0158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:47.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.624+0000 7f23a649b700 1 --2- 172.21.15.121:0/3502130107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23a0157e30 0x7f23a0158210 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 rx=0x7f2390009f20 tx=0x7f2390009840).stop 2023-11-15T03:11:47.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.624+0000 7f23a649b700 1 -- 172.21.15.121:0/3502130107 shutdown_connections 2023-11-15T03:11:47.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.624+0000 7f23a649b700 1 --2- 172.21.15.121:0/3502130107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a00ac420 0x7f23a00667f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:47.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.624+0000 7f23a649b700 1 --2- 172.21.15.121:0/3502130107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23a0157e30 0x7f23a0158210 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:47.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.624+0000 7f23a649b700 1 -- 172.21.15.121:0/3502130107 >> 172.21.15.121:0/3502130107 conn(0x7f23a0067a60 msgr2=0x7f23a0067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:47.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.624+0000 7f23a649b700 1 -- 172.21.15.121:0/3502130107 shutdown_connections 2023-11-15T03:11:47.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.624+0000 7f23a649b700 1 -- 172.21.15.121:0/3502130107 wait complete. 2023-11-15T03:11:47.643 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.628+0000 7f23a649b700 1 Processor -- start 2023-11-15T03:11:47.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.628+0000 7f23a649b700 1 -- start start 2023-11-15T03:11:47.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.628+0000 7f23a649b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23a0062190 0x7f23a00bf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:47.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.628+0000 7f23a649b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a00ac420 0x7f23a011f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:47.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.628+0000 7f23a649b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23a0157e30 0x7f23a011fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:47.644 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.628+0000 7f23a649b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0146a50 con 0x7f23a0157e30 2023-11-15T03:11:47.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.628+0000 7f23a649b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0120090 con 0x7f23a0062190 2023-11-15T03:11:47.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.628+0000 7f23a649b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0169b00 con 0x7f23a00ac420 2023-11-15T03:11:47.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.628+0000 7f23a4a38700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23a0157e30 0x7f23a011fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:47.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.628+0000 7f23a4a38700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23a0157e30 0x7f23a011fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55976/0 (socket says 172.21.15.121:55976) 2023-11-15T03:11:47.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.628+0000 7f23a4a38700 1 -- 172.21.15.121:0/4085022213 learned_addr learned my addr 172.21.15.121:0/4085022213 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:47.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.628+0000 7f239ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23a0062190 0x7f23a00bf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:47.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.628+0000 7f239f7fe700 1 --2- 172.21.15.121:0/4085022213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a00ac420 0x7f23a011f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:47.646 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.632+0000 7f23a4a38700 1 -- 172.21.15.121:0/4085022213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a00ac420 msgr2=0x7f23a011f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:47.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.632+0000 7f23a4a38700 1 --2- 172.21.15.121:0/4085022213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a00ac420 0x7f23a011f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:47.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.632+0000 7f23a4a38700 1 -- 172.21.15.121:0/4085022213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23a0062190 msgr2=0x7f23a00bf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:47.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.632+0000 7f23a4a38700 1 --2- 172.21.15.121:0/4085022213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23a0062190 0x7f23a00bf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:47.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.632+0000 7f23a4a38700 1 -- 172.21.15.121:0/4085022213 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2390009ae0 con 0x7f23a0157e30 2023-11-15T03:11:47.647 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.632+0000 7f23a4a38700 1 --2- 172.21.15.121:0/4085022213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23a0157e30 0x7f23a011fba0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 rx=0x7f239800e8d0 tx=0x7f2398009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:47.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.632+0000 7f2397fff700 1 -- 172.21.15.121:0/4085022213 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23980094d0 con 0x7f23a0157e30 2023-11-15T03:11:47.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.632+0000 7f2397fff700 1 -- 172.21.15.121:0/4085022213 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2398012600 con 0x7f23a0157e30 2023-11-15T03:11:47.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.632+0000 7f23a649b700 1 -- 172.21.15.121:0/4085022213 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a0169800 con 0x7f23a0157e30 2023-11-15T03:11:47.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.632+0000 7f2397fff700 1 -- 172.21.15.121:0/4085022213 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398012a00 con 0x7f23a0157e30 2023-11-15T03:11:47.648 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.632+0000 7f23a649b700 1 -- 172.21.15.121:0/4085022213 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a00bfb60 con 0x7f23a0157e30 2023-11-15T03:11:47.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.632+0000 7f23a649b700 1 -- 172.21.15.121:0/4085022213 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a0169c70 con 0x7f23a0157e30 2023-11-15T03:11:47.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.632+0000 7f2397fff700 1 -- 172.21.15.121:0/4085022213 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f239801b400 con 0x7f23a0157e30 2023-11-15T03:11:47.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.636+0000 7f2397fff700 1 --2- 172.21.15.121:0/4085022213 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f238005a510 0x7f238005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:47.649 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.636+0000 7f2397fff700 1 -- 172.21.15.121:0/4085022213 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f2398074cc0 con 0x7f23a0157e30 2023-11-15T03:11:47.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.636+0000 7f239ffff700 1 --2- 172.21.15.121:0/4085022213 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f238005a510 0x7f238005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:47.650 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.636+0000 7f239ffff700 1 --2- 172.21.15.121:0/4085022213 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f238005a510 0x7f238005c9d0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f239000b9a0 tx=0x7f2390009840).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:47.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.644+0000 7f2397fff700 1 -- 172.21.15.121:0/4085022213 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2398048440 con 0x7f23a0157e30 2023-11-15T03:11:47.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:47.840+0000 7f23a649b700 1 -- 172.21.15.121:0/4085022213 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f23a00aac10 con 0x7f23a0157e30 2023-11-15T03:11:48.375 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-11-15T03:11:48.368+0000 7feae9fa0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-15T03:11:48.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.376+0000 7f2397fff700 1 -- 172.21.15.121:0/4085022213 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v357) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2398012db0 con 0x7f23a0157e30 2023-11-15T03:11:48.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.380+0000 7f23a649b700 1 -- 172.21.15.121:0/4085022213 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f238005a510 msgr2=0x7f238005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:48.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.380+0000 7f23a649b700 1 --2- 172.21.15.121:0/4085022213 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f238005a510 0x7f238005c9d0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f239000b9a0 tx=0x7f2390009840).stop 2023-11-15T03:11:48.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.380+0000 7f23a649b700 1 -- 172.21.15.121:0/4085022213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23a0157e30 msgr2=0x7f23a011fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:48.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.380+0000 7f23a649b700 1 --2- 172.21.15.121:0/4085022213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23a0157e30 0x7f23a011fba0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 rx=0x7f239800e8d0 tx=0x7f2398009060).stop 2023-11-15T03:11:48.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.380+0000 7f23a649b700 1 -- 172.21.15.121:0/4085022213 shutdown_connections 2023-11-15T03:11:48.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.380+0000 7f23a649b700 1 --2- 172.21.15.121:0/4085022213 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f238005a510 0x7f238005c9d0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:48.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.380+0000 7f23a649b700 1 --2- 172.21.15.121:0/4085022213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23a0157e30 0x7f23a011fba0 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:48.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.380+0000 7f23a649b700 1 --2- 172.21.15.121:0/4085022213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a00ac420 0x7f23a011f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:48.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.380+0000 7f23a649b700 1 --2- 172.21.15.121:0/4085022213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23a0062190 0x7f23a00bf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:48.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.380+0000 7f23a649b700 1 -- 172.21.15.121:0/4085022213 >> 172.21.15.121:0/4085022213 conn(0x7f23a0067a60 msgr2=0x7f23a00a88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:48.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.384+0000 7f23a649b700 1 -- 172.21.15.121:0/4085022213 shutdown_connections 2023-11-15T03:11:48.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.384+0000 7f23a649b700 1 -- 172.21.15.121:0/4085022213 wait complete. 2023-11-15T03:11:48.394 INFO:tasks.ceph.mds.a.smithi121.stderr: -921> 2023-11-15T03:11:38.240+0000 7fea46f11700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-15T03:11:48.394 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-15T03:11:48.395 INFO:tasks.ceph.mds.a.smithi121.stderr: -667> 2023-11-15T03:11:39.372+0000 7fea4370a700 -1 mds.0.cache corrupt backtrace on ino x00x10000000001: void inode_backtrace_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 5 < 104: Malformed input 2023-11-15T03:11:48.396 INFO:tasks.ceph.mds.a.smithi121.stderr: -469> 2023-11-15T03:11:40.864+0000 7fea4370a700 -1 log_channel(cluster) log [ERR] : dir 0x100000001f6 object missing on disk; some files may be lost (/dir2) 2023-11-15T03:11:48.414 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-15T03:11:48.443 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2023-11-15T03:11:48.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.560+0000 7f00a2347700 1 Processor -- start 2023-11-15T03:11:48.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.564+0000 7f00a2347700 1 -- start start 2023-11-15T03:11:48.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.564+0000 7f00a2347700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f009c14aff0 0x7f009c066c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:48.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.564+0000 7f00a2347700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f009c067160 0x7f009c0a78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:48.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.564+0000 7f00a2347700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f009c0657f0 con 0x7f009c14ac10 2023-11-15T03:11:48.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.564+0000 7f00a2347700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f009c066650 con 0x7f009c067160 2023-11-15T03:11:48.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.564+0000 7f00a2347700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009c0675d0 con 0x7f009c14aff0 2023-11-15T03:11:48.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.564+0000 7f00a1b46700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f009c067160 0x7f009c0a78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:48.576 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.564+0000 7f00a1b46700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f009c067160 0x7f009c0a78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:58230/0 (socket says 172.21.15.121:58230) 2023-11-15T03:11:48.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.564+0000 7f00a1b46700 1 -- 172.21.15.121:0/390222522 learned_addr learned my addr 172.21.15.121:0/390222522 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:48.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.564+0000 7f00a0b44700 1 --2- 172.21.15.121:0/390222522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f009c14aff0 0x7f009c066c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:48.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.568+0000 7f00a1b46700 1 -- 172.21.15.121:0/390222522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f009c14aff0 msgr2=0x7f009c066c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:48.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.568+0000 7f00a1b46700 1 --2- 172.21.15.121:0/390222522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f009c14aff0 0x7f009c066c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:48.577 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.568+0000 7f00a1b46700 1 -- 172.21.15.121:0/390222522 >> v1:172.21.15.190:6789/0 conn(0x7f009c14ac10 legacy=0x7f009c0c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:48.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.568+0000 7f00a1b46700 1 -- 172.21.15.121:0/390222522 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009c065a40 con 0x7f009c067160 2023-11-15T03:11:48.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.568+0000 7f00a1b46700 1 --2- 172.21.15.121:0/390222522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f009c067160 0x7f009c0a78a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f0094006fd0 tx=0x7f00940090f0).ready entity=mon.1 client_cookie=127d2c8165f5768d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:48.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.568+0000 7f0093fff700 1 -- 172.21.15.121:0/390222522 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009400b580 con 0x7f009c067160 2023-11-15T03:11:48.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.568+0000 7f0093fff700 1 -- 172.21.15.121:0/390222522 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009400b6e0 con 0x7f009c067160 2023-11-15T03:11:48.578 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.568+0000 7f0093fff700 1 -- 172.21.15.121:0/390222522 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009400ba60 con 0x7f009c067160 2023-11-15T03:11:48.579 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.568+0000 7f00a2347700 1 -- 172.21.15.121:0/390222522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f009c067160 msgr2=0x7f009c0a78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.568+0000 7f00a2347700 1 --2- 172.21.15.121:0/390222522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f009c067160 0x7f009c0a78a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f0094006fd0 tx=0x7f00940090f0).stop 2023-11-15T03:11:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.568+0000 7f00a2347700 1 -- 172.21.15.121:0/390222522 shutdown_connections 2023-11-15T03:11:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.568+0000 7f00a2347700 1 --2- 172.21.15.121:0/390222522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f009c067160 0x7f009c0a78a0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.568+0000 7f00a2347700 1 --2- 172.21.15.121:0/390222522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f009c14aff0 0x7f009c066c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.568+0000 7f00a2347700 1 -- 172.21.15.121:0/390222522 >> 172.21.15.121:0/390222522 conn(0x7f009c067a60 msgr2=0x7f009c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:48.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.568+0000 7f00a2347700 1 -- 172.21.15.121:0/390222522 shutdown_connections 2023-11-15T03:11:48.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.568+0000 7f00a2347700 1 -- 172.21.15.121:0/390222522 wait complete. 2023-11-15T03:11:48.581 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.572+0000 7f00a2347700 1 Processor -- start 2023-11-15T03:11:48.582 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.572+0000 7f00a2347700 1 -- start start 2023-11-15T03:11:48.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.572+0000 7f00a2347700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f009c067160 0x7f009c141fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:48.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.572+0000 7f00a2347700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f009c14ac10 0x7f009c147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:48.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.572+0000 7f00a2347700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f009c14aff0 0x7f009c165330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:48.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.572+0000 7f00a2347700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009c0aa5f0 con 0x7f009c067160 2023-11-15T03:11:48.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.572+0000 7f00a2347700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009c148570 con 0x7f009c14aff0 2023-11-15T03:11:48.584 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.572+0000 7f00a2347700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009c148850 con 0x7f009c14ac10 2023-11-15T03:11:48.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.576+0000 7f00a1345700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f009c067160 0x7f009c141fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:48.585 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.576+0000 7f00a1345700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f009c067160 0x7f009c141fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55996/0 (socket says 172.21.15.121:55996) 2023-11-15T03:11:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.576+0000 7f00a1345700 1 -- 172.21.15.121:0/3523374830 learned_addr learned my addr 172.21.15.121:0/3523374830 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.576+0000 7f00a0b44700 1 --2- 172.21.15.121:0/3523374830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f009c14ac10 0x7f009c147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.576+0000 7f00a1b46700 1 --2- 172.21.15.121:0/3523374830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f009c14aff0 0x7f009c165330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:48.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.576+0000 7f00a0b44700 1 -- 172.21.15.121:0/3523374830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f009c14aff0 msgr2=0x7f009c165330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:48.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.576+0000 7f00a0b44700 1 --2- 172.21.15.121:0/3523374830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f009c14aff0 0x7f009c165330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:48.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.576+0000 7f00a0b44700 1 -- 172.21.15.121:0/3523374830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f009c067160 msgr2=0x7f009c141fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:48.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.576+0000 7f00a0b44700 1 --2- 172.21.15.121:0/3523374830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f009c067160 0x7f009c141fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:48.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.576+0000 7f00a0b44700 1 -- 172.21.15.121:0/3523374830 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0094009370 con 0x7f009c14ac10 2023-11-15T03:11:48.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.576+0000 7f00a1345700 1 --2- 172.21.15.121:0/3523374830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f009c067160 0x7f009c141fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:48.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.576+0000 7f00a0b44700 1 --2- 172.21.15.121:0/3523374830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f009c14ac10 0x7f009c147c00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f008c00e6f0 tx=0x7f008c009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:48.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.576+0000 7f00917fa700 1 -- 172.21.15.121:0/3523374830 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c012600 con 0x7f009c14ac10 2023-11-15T03:11:48.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.576+0000 7f00917fa700 1 -- 172.21.15.121:0/3523374830 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008c012760 con 0x7f009c14ac10 2023-11-15T03:11:48.588 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.576+0000 7f00a2347700 1 -- 172.21.15.121:0/3523374830 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f009c0aa840 con 0x7f009c14ac10 2023-11-15T03:11:48.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.576+0000 7f00917fa700 1 -- 172.21.15.121:0/3523374830 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c012ae0 con 0x7f009c14ac10 2023-11-15T03:11:48.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.576+0000 7f00a2347700 1 -- 172.21.15.121:0/3523374830 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009c142490 con 0x7f009c14ac10 2023-11-15T03:11:48.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.576+0000 7f00a2347700 1 -- 172.21.15.121:0/3523374830 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f009c0aacc0 con 0x7f009c14ac10 2023-11-15T03:11:48.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.580+0000 7f00917fa700 1 -- 172.21.15.121:0/3523374830 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f008c01b400 con 0x7f009c14ac10 2023-11-15T03:11:48.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.580+0000 7f00917fa700 1 --2- 172.21.15.121:0/3523374830 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f008005a510 0x7f008005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:48.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.580+0000 7f00917fa700 1 -- 172.21.15.121:0/3523374830 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6380+0+0 (secure 0 0 0) 0x7f008c074ba0 con 0x7f009c14ac10 2023-11-15T03:11:48.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.580+0000 7f00a1345700 1 --2- 172.21.15.121:0/3523374830 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f008005a510 0x7f008005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:48.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.584+0000 7f00a1345700 1 --2- 172.21.15.121:0/3523374830 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f008005a510 0x7f008005c9d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f0098002230 tx=0x7f0098009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:48.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.588+0000 7f00917fa700 1 -- 172.21.15.121:0/3523374830 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f008c048160 con 0x7f009c14ac10 2023-11-15T03:11:48.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:48.788+0000 7f00a2347700 1 -- 172.21.15.121:0/3523374830 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f009c065d20 con 0x7f009c14ac10 2023-11-15T03:11:49.404 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.395+0000 7f00917fa700 1 -- 172.21.15.121:0/3523374830 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v358) v1 ==== 104+0+0 (secure 0 0 0) 0x7f008c04b780 con 0x7f009c14ac10 2023-11-15T03:11:49.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.399+0000 7f00a2347700 1 -- 172.21.15.121:0/3523374830 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f008005a510 msgr2=0x7f008005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:49.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.399+0000 7f00a2347700 1 --2- 172.21.15.121:0/3523374830 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f008005a510 0x7f008005c9d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f0098002230 tx=0x7f0098009040).stop 2023-11-15T03:11:49.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.399+0000 7f00a2347700 1 -- 172.21.15.121:0/3523374830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f009c14ac10 msgr2=0x7f009c147c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:49.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.399+0000 7f00a2347700 1 --2- 172.21.15.121:0/3523374830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f009c14ac10 0x7f009c147c00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f008c00e6f0 tx=0x7f008c009060).stop 2023-11-15T03:11:49.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.399+0000 7f00a2347700 1 -- 172.21.15.121:0/3523374830 shutdown_connections 2023-11-15T03:11:49.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.399+0000 7f00a2347700 1 --2- 172.21.15.121:0/3523374830 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f008005a510 0x7f008005c9d0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:49.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.399+0000 7f00a2347700 1 --2- 172.21.15.121:0/3523374830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f009c14aff0 0x7f009c165330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:49.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.399+0000 7f00a2347700 1 --2- 172.21.15.121:0/3523374830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f009c14ac10 0x7f009c147c00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:49.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.399+0000 7f00a2347700 1 --2- 172.21.15.121:0/3523374830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f009c067160 0x7f009c141fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:49.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.399+0000 7f00a2347700 1 -- 172.21.15.121:0/3523374830 >> 172.21.15.121:0/3523374830 conn(0x7f009c067a60 msgr2=0x7f009c0a8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:49.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.399+0000 7f00a2347700 1 -- 172.21.15.121:0/3523374830 shutdown_connections 2023-11-15T03:11:49.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.399+0000 7f00a2347700 1 -- 172.21.15.121:0/3523374830 wait complete. 2023-11-15T03:11:49.429 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-15T03:11:49.651 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.643+0000 7f599759e700 1 Processor -- start 2023-11-15T03:11:49.652 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.643+0000 7f599759e700 1 -- start start 2023-11-15T03:11:49.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.647+0000 7f599759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5998153990 0x7f5998153d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:49.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.647+0000 7f599759e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f59980bb910 0x7f59981581e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:49.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.647+0000 7f599759e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59980a8b20 con 0x7f59980c2ec0 2023-11-15T03:11:49.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.647+0000 7f599759e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59980c1760 con 0x7f59980bb910 2023-11-15T03:11:49.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.647+0000 7f599759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59980a8d50 con 0x7f5998153990 2023-11-15T03:11:49.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.647+0000 7f5995d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f59980bb910 0x7f59981581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:49.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.647+0000 7f5996d9d700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f59980c2ec0 0x7f59980c3330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:47220/0 (socket says 172.21.15.121:47220) 2023-11-15T03:11:49.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.647+0000 7f599659c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5998153990 0x7f5998153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:49.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.647+0000 7f5996d9d700 1 -- 172.21.15.121:0/1251394223 learned_addr learned my addr 172.21.15.121:0/1251394223 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:49.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.647+0000 7f599659c700 1 -- 172.21.15.121:0/1251394223 >> v2:172.21.15.190:3300/0 conn(0x7f59980bb910 msgr2=0x7f59981581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:49.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.647+0000 7f599659c700 1 --2- 172.21.15.121:0/1251394223 >> v2:172.21.15.190:3300/0 conn(0x7f59980bb910 0x7f59981581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:49.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.647+0000 7f599659c700 1 -- 172.21.15.121:0/1251394223 >> v1:172.21.15.190:6789/0 conn(0x7f59980c2ec0 legacy=0x7f59980c3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:49.660 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.647+0000 7f599659c700 1 -- 172.21.15.121:0/1251394223 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59980a9030 con 0x7f5998153990 2023-11-15T03:11:49.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.647+0000 7f599559a700 1 -- 172.21.15.121:0/1251394223 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694201923 0 0) 0x7f59980a8b20 con 0x7f59980c2ec0 2023-11-15T03:11:49.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.647+0000 7f5995d9b700 1 --2- 172.21.15.121:0/1251394223 >> v2:172.21.15.190:3300/0 conn(0x7f59980bb910 0x7f59981581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:49.661 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.647+0000 7f599659c700 1 --2- 172.21.15.121:0/1251394223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5998153990 0x7f5998153d70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f5988009760 tx=0x7f5988009060).ready entity=mon.2 client_cookie=9294a0b96d37c405 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:49.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.651+0000 7f599559a700 1 -- 172.21.15.121:0/1251394223 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598800f510 con 0x7f5998153990 2023-11-15T03:11:49.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.651+0000 7f599559a700 1 -- 172.21.15.121:0/1251394223 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598800f670 con 0x7f5998153990 2023-11-15T03:11:49.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.651+0000 7f599559a700 1 -- 172.21.15.121:0/1251394223 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598800f9f0 con 0x7f5998153990 2023-11-15T03:11:49.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.651+0000 7f599759e700 1 -- 172.21.15.121:0/1251394223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5998153990 msgr2=0x7f5998153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:49.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.651+0000 7f599759e700 1 --2- 172.21.15.121:0/1251394223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5998153990 0x7f5998153d70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f5988009760 tx=0x7f5988009060).stop 2023-11-15T03:11:49.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.651+0000 7f599759e700 1 -- 172.21.15.121:0/1251394223 shutdown_connections 2023-11-15T03:11:49.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.651+0000 7f599759e700 1 --2- 172.21.15.121:0/1251394223 >> v2:172.21.15.190:3300/0 conn(0x7f59980bb910 0x7f59981581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:49.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.651+0000 7f599759e700 1 --2- 172.21.15.121:0/1251394223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5998153990 0x7f5998153d70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:49.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.651+0000 7f599759e700 1 -- 172.21.15.121:0/1251394223 >> 172.21.15.121:0/1251394223 conn(0x7f5998067a60 msgr2=0x7f5998067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:49.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.651+0000 7f599759e700 1 -- 172.21.15.121:0/1251394223 shutdown_connections 2023-11-15T03:11:49.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.651+0000 7f599759e700 1 -- 172.21.15.121:0/1251394223 wait complete. 2023-11-15T03:11:49.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f599759e700 1 Processor -- start 2023-11-15T03:11:49.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f599759e700 1 -- start start 2023-11-15T03:11:49.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f599759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59980bb910 0x7f59980c0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:49.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f599759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59980c2ec0 0x7f59980ab770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:49.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f599759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5998153990 0x7f599814e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:49.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f599759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5998152ec0 con 0x7f5998153990 2023-11-15T03:11:49.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f599759e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5998152050 con 0x7f59980c2ec0 2023-11-15T03:11:49.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f599759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59981522f0 con 0x7f59980bb910 2023-11-15T03:11:49.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f5996d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5998153990 0x7f599814e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:49.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f5996d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5998153990 0x7f599814e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55998/0 (socket says 172.21.15.121:55998) 2023-11-15T03:11:49.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f5996d9d700 1 -- 172.21.15.121:0/3332266736 learned_addr learned my addr 172.21.15.121:0/3332266736 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:49.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f599659c700 1 --2- 172.21.15.121:0/3332266736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59980bb910 0x7f59980c0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:49.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f5995d9b700 1 --2- 172.21.15.121:0/3332266736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59980c2ec0 0x7f59980ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:49.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f5996d9d700 1 -- 172.21.15.121:0/3332266736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59980bb910 msgr2=0x7f59980c0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:49.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f5996d9d700 1 --2- 172.21.15.121:0/3332266736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59980bb910 0x7f59980c0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:49.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f5996d9d700 1 -- 172.21.15.121:0/3332266736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59980c2ec0 msgr2=0x7f59980ab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:49.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f5996d9d700 1 --2- 172.21.15.121:0/3332266736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59980c2ec0 0x7f59980ab770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:49.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f5996d9d700 1 -- 172.21.15.121:0/3332266736 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c009250 con 0x7f5998153990 2023-11-15T03:11:49.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f599659c700 1 --2- 172.21.15.121:0/3332266736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59980bb910 0x7f59980c0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:49.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f5996d9d700 1 --2- 172.21.15.121:0/3332266736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5998153990 0x7f599814e110 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 rx=0x7f599000c790 tx=0x7f5990007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:49.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f59867fc700 1 -- 172.21.15.121:0/3332266736 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5990005d30 con 0x7f5998153990 2023-11-15T03:11:49.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f59867fc700 1 -- 172.21.15.121:0/3332266736 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5990007110 con 0x7f5998153990 2023-11-15T03:11:49.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f59867fc700 1 -- 172.21.15.121:0/3332266736 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59900074d0 con 0x7f5998153990 2023-11-15T03:11:49.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.655+0000 7f599759e700 1 -- 172.21.15.121:0/3332266736 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59880092a0 con 0x7f5998153990 2023-11-15T03:11:49.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.659+0000 7f599759e700 1 -- 172.21.15.121:0/3332266736 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5998152610 con 0x7f5998153990 2023-11-15T03:11:49.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.659+0000 7f599759e700 1 -- 172.21.15.121:0/3332266736 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5998153030 con 0x7f5998153990 2023-11-15T03:11:49.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.659+0000 7f59867fc700 1 -- 172.21.15.121:0/3332266736 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5990007840 con 0x7f5998153990 2023-11-15T03:11:49.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.663+0000 7f59867fc700 1 --2- 172.21.15.121:0/3332266736 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f597805a510 0x7f597805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:49.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.663+0000 7f59867fc700 1 -- 172.21.15.121:0/3332266736 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6380+0+0 (secure 0 0 0) 0x7f59900748f0 con 0x7f5998153990 2023-11-15T03:11:49.673 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.663+0000 7f599659c700 1 --2- 172.21.15.121:0/3332266736 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f597805a510 0x7f597805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:49.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.663+0000 7f599659c700 1 --2- 172.21.15.121:0/3332266736 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f597805a510 0x7f597805c9d0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f5988009fd0 tx=0x7f5988017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:49.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.667+0000 7f59867fc700 1 -- 172.21.15.121:0/3332266736 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5990047f10 con 0x7f5998153990 2023-11-15T03:11:49.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:49.831+0000 7f599759e700 1 -- 172.21.15.121:0/3332266736 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f59980c19b0 con 0x7f5998153990 2023-11-15T03:11:50.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.403+0000 7f59867fc700 1 -- 172.21.15.121:0/3332266736 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v150) v1 ==== 181+0+0 (secure 0 0 0) 0x7f599004b530 con 0x7f5998153990 2023-11-15T03:11:50.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.411+0000 7f599759e700 1 -- 172.21.15.121:0/3332266736 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f597805a510 msgr2=0x7f597805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:50.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.411+0000 7f599759e700 1 --2- 172.21.15.121:0/3332266736 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f597805a510 0x7f597805c9d0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f5988009fd0 tx=0x7f5988017000).stop 2023-11-15T03:11:50.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.411+0000 7f599759e700 1 -- 172.21.15.121:0/3332266736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5998153990 msgr2=0x7f599814e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:50.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.411+0000 7f599759e700 1 --2- 172.21.15.121:0/3332266736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5998153990 0x7f599814e110 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 rx=0x7f599000c790 tx=0x7f5990007f30).stop 2023-11-15T03:11:50.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.415+0000 7f599759e700 1 -- 172.21.15.121:0/3332266736 shutdown_connections 2023-11-15T03:11:50.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.415+0000 7f599759e700 1 --2- 172.21.15.121:0/3332266736 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f597805a510 0x7f597805c9d0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:50.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.415+0000 7f599759e700 1 --2- 172.21.15.121:0/3332266736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5998153990 0x7f599814e110 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:50.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.415+0000 7f599759e700 1 --2- 172.21.15.121:0/3332266736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59980c2ec0 0x7f59980ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:50.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.415+0000 7f599759e700 1 --2- 172.21.15.121:0/3332266736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59980bb910 0x7f59980c0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:50.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.415+0000 7f599759e700 1 -- 172.21.15.121:0/3332266736 >> 172.21.15.121:0/3332266736 conn(0x7f5998067a60 msgr2=0x7f59980bc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:50.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.415+0000 7f599759e700 1 -- 172.21.15.121:0/3332266736 shutdown_connections 2023-11-15T03:11:50.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.415+0000 7f599759e700 1 -- 172.21.15.121:0/3332266736 wait complete. 2023-11-15T03:11:50.427 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_metadata' removed 2023-11-15T03:11:50.445 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-15T03:11:50.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.611+0000 7fa5b769b700 1 Processor -- start 2023-11-15T03:11:50.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.611+0000 7fa5b769b700 1 -- start start 2023-11-15T03:11:50.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.615+0000 7fa5b769b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b0159320 0x7fa5b00a5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:50.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.615+0000 7fa5b769b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa5b00a5c10 0x7fa5b00bc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:50.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.615+0000 7fa5b769b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b00a6290 con 0x7fa5b0158f40 2023-11-15T03:11:50.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.615+0000 7fa5b769b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b00a60a0 con 0x7fa5b00a5c10 2023-11-15T03:11:50.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.615+0000 7fa5b769b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b0158640 con 0x7fa5b0159320 2023-11-15T03:11:50.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.615+0000 7fa5b5c38700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa5b00a5c10 0x7fa5b00bc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:50.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.615+0000 7fa5b5437700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa5b0158f40 0x7fa5b0067020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59088/0 (socket says 172.21.15.121:59088) 2023-11-15T03:11:50.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.615+0000 7fa5b5437700 1 -- 172.21.15.121:0/3657038520 learned_addr learned my addr 172.21.15.121:0/3657038520 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:50.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.615+0000 7fa5b4c36700 1 --2- 172.21.15.121:0/3657038520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b0159320 0x7fa5b00a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:50.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.615+0000 7fa5b5c38700 1 -- 172.21.15.121:0/3657038520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b0159320 msgr2=0x7fa5b00a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:50.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.615+0000 7fa5b5c38700 1 --2- 172.21.15.121:0/3657038520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b0159320 0x7fa5b00a5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:50.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.615+0000 7fa5b5c38700 1 -- 172.21.15.121:0/3657038520 >> v1:172.21.15.121:6789/0 conn(0x7fa5b0158f40 legacy=0x7fa5b0067020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:50.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.615+0000 7fa5a7fff700 1 -- 172.21.15.121:0/3657038520 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2081701631 0 0) 0x7fa5ac001110 con 0x7fa5b0158f40 2023-11-15T03:11:50.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.615+0000 7fa5b5c38700 1 -- 172.21.15.121:0/3657038520 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b0066b90 con 0x7fa5b00a5c10 2023-11-15T03:11:50.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.615+0000 7fa5b5c38700 1 --2- 172.21.15.121:0/3657038520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5b00a5c10 0x7fa5b00bc170 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 rx=0x7fa5a8009e60 tx=0x7fa5a8009780).ready entity=mon.0 client_cookie=c8558b5f2f0f4471 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:50.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.615+0000 7fa5a7fff700 1 -- 172.21.15.121:0/3657038520 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800e070 con 0x7fa5b00a5c10 2023-11-15T03:11:50.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.615+0000 7fa5a7fff700 1 -- 172.21.15.121:0/3657038520 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a800bc30 con 0x7fa5b00a5c10 2023-11-15T03:11:50.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.615+0000 7fa5a7fff700 1 -- 172.21.15.121:0/3657038520 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8012410 con 0x7fa5b00a5c10 2023-11-15T03:11:50.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.619+0000 7fa5b769b700 1 -- 172.21.15.121:0/3657038520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5b00a5c10 msgr2=0x7fa5b00bc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:50.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.619+0000 7fa5b769b700 1 --2- 172.21.15.121:0/3657038520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5b00a5c10 0x7fa5b00bc170 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 rx=0x7fa5a8009e60 tx=0x7fa5a8009780).stop 2023-11-15T03:11:50.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.619+0000 7fa5b769b700 1 -- 172.21.15.121:0/3657038520 shutdown_connections 2023-11-15T03:11:50.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.619+0000 7fa5b769b700 1 --2- 172.21.15.121:0/3657038520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5b00a5c10 0x7fa5b00bc170 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:50.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.619+0000 7fa5b769b700 1 --2- 172.21.15.121:0/3657038520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b0159320 0x7fa5b00a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:50.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.619+0000 7fa5b769b700 1 -- 172.21.15.121:0/3657038520 >> 172.21.15.121:0/3657038520 conn(0x7fa5b0068500 msgr2=0x7fa5b00c6780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:50.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.619+0000 7fa5b769b700 1 -- 172.21.15.121:0/3657038520 shutdown_connections 2023-11-15T03:11:50.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.619+0000 7fa5b769b700 1 -- 172.21.15.121:0/3657038520 wait complete. 2023-11-15T03:11:50.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.619+0000 7fa5b769b700 1 Processor -- start 2023-11-15T03:11:50.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.619+0000 7fa5b769b700 1 -- start start 2023-11-15T03:11:50.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b769b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b00a5c10 0x7fa5b00a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:50.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b769b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5b0158f40 0x7fa5b016bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:50.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b769b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5b0159320 0x7fa5b016c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:50.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b769b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b014f5b0 con 0x7fa5b0158f40 2023-11-15T03:11:50.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b769b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b016c900 con 0x7fa5b0159320 2023-11-15T03:11:50.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b769b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b0156cf0 con 0x7fa5b00a5c10 2023-11-15T03:11:50.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b4c36700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5b0158f40 0x7fa5b016bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:50.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b4c36700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5b0158f40 0x7fa5b016bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56014/0 (socket says 172.21.15.121:56014) 2023-11-15T03:11:50.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b4c36700 1 -- 172.21.15.121:0/3973204736 learned_addr learned my addr 172.21.15.121:0/3973204736 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:50.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b5437700 1 --2- 172.21.15.121:0/3973204736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b00a5c10 0x7fa5b00a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:50.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b5c38700 1 --2- 172.21.15.121:0/3973204736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5b0159320 0x7fa5b016c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:50.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b4c36700 1 -- 172.21.15.121:0/3973204736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b00a5c10 msgr2=0x7fa5b00a6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:50.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b4c36700 1 --2- 172.21.15.121:0/3973204736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b00a5c10 0x7fa5b00a6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:50.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b4c36700 1 -- 172.21.15.121:0/3973204736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5b0159320 msgr2=0x7fa5b016c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:50.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b4c36700 1 --2- 172.21.15.121:0/3973204736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5b0159320 0x7fa5b016c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:50.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b4c36700 1 -- 172.21.15.121:0/3973204736 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a8009a20 con 0x7fa5b0158f40 2023-11-15T03:11:50.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b5437700 1 --2- 172.21.15.121:0/3973204736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b00a5c10 0x7fa5b00a6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:50.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b4c36700 1 --2- 172.21.15.121:0/3973204736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5b0158f40 0x7fa5b016bf20 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 rx=0x7fa5a000d8c0 tx=0x7fa5a0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:50.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5a57fa700 1 -- 172.21.15.121:0/3973204736 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a00095b0 con 0x7fa5b0158f40 2023-11-15T03:11:50.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5a57fa700 1 -- 172.21.15.121:0/3973204736 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a0009710 con 0x7fa5b0158f40 2023-11-15T03:11:50.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5a57fa700 1 -- 172.21.15.121:0/3973204736 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0009a90 con 0x7fa5b0158f40 2023-11-15T03:11:50.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b769b700 1 -- 172.21.15.121:0/3973204736 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b00c16b0 con 0x7fa5b0158f40 2023-11-15T03:11:50.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.623+0000 7fa5b769b700 1 -- 172.21.15.121:0/3973204736 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b00c1c70 con 0x7fa5b0158f40 2023-11-15T03:11:50.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.627+0000 7fa5b769b700 1 -- 172.21.15.121:0/3973204736 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b006c150 con 0x7fa5b0158f40 2023-11-15T03:11:50.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.627+0000 7fa5a57fa700 1 -- 172.21.15.121:0/3973204736 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa5a0013480 con 0x7fa5b0158f40 2023-11-15T03:11:50.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.627+0000 7fa5a57fa700 1 --2- 172.21.15.121:0/3973204736 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa59405a510 0x7fa59405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:50.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.627+0000 7fa5a57fa700 1 -- 172.21.15.121:0/3973204736 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5942+0+0 (secure 0 0 0) 0x7fa5a00744c0 con 0x7fa5b0158f40 2023-11-15T03:11:50.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.631+0000 7fa5b5437700 1 --2- 172.21.15.121:0/3973204736 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa59405a510 0x7fa59405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:50.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.631+0000 7fa5b5437700 1 --2- 172.21.15.121:0/3973204736 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa59405a510 0x7fa59405c9d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7fa5ac007fd0 tx=0x7fa5ac009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:50.645 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.635+0000 7fa5a57fa700 1 -- 172.21.15.121:0/3973204736 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa5a0047cf0 con 0x7fa5b0158f40 2023-11-15T03:11:50.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:50.799+0000 7fa5b769b700 1 -- 172.21.15.121:0/3973204736 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa5b015b700 con 0x7fa5b0158f40 2023-11-15T03:11:51.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.411+0000 7fa5a57fa700 1 -- 172.21.15.121:0/3973204736 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v151) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa5a004b310 con 0x7fa5b0158f40 2023-11-15T03:11:51.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.423+0000 7fa5b769b700 1 -- 172.21.15.121:0/3973204736 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa59405a510 msgr2=0x7fa59405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:51.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.423+0000 7fa5b769b700 1 --2- 172.21.15.121:0/3973204736 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa59405a510 0x7fa59405c9d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7fa5ac007fd0 tx=0x7fa5ac009040).stop 2023-11-15T03:11:51.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.423+0000 7fa5b769b700 1 -- 172.21.15.121:0/3973204736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5b0158f40 msgr2=0x7fa5b016bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:51.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.423+0000 7fa5b769b700 1 --2- 172.21.15.121:0/3973204736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5b0158f40 0x7fa5b016bf20 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 rx=0x7fa5a000d8c0 tx=0x7fa5a0008040).stop 2023-11-15T03:11:51.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.423+0000 7fa5b769b700 1 -- 172.21.15.121:0/3973204736 shutdown_connections 2023-11-15T03:11:51.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.423+0000 7fa5b769b700 1 --2- 172.21.15.121:0/3973204736 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fa59405a510 0x7fa59405c9d0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:51.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.423+0000 7fa5b769b700 1 --2- 172.21.15.121:0/3973204736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5b0159320 0x7fa5b016c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:51.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.423+0000 7fa5b769b700 1 --2- 172.21.15.121:0/3973204736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5b0158f40 0x7fa5b016bf20 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:51.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.423+0000 7fa5b769b700 1 --2- 172.21.15.121:0/3973204736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b00a5c10 0x7fa5b00a6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:51.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.423+0000 7fa5b769b700 1 -- 172.21.15.121:0/3973204736 >> 172.21.15.121:0/3973204736 conn(0x7fa5b0068500 msgr2=0x7fa5b0067020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:51.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.427+0000 7fa5b769b700 1 -- 172.21.15.121:0/3973204736 shutdown_connections 2023-11-15T03:11:51.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.427+0000 7fa5b769b700 1 -- 172.21.15.121:0/3973204736 wait complete. 2023-11-15T03:11:51.437 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_data' removed 2023-11-15T03:11:51.456 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-15T03:11:51.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.611+0000 7f812359e700 1 Processor -- start 2023-11-15T03:11:51.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.611+0000 7f812359e700 1 -- start start 2023-11-15T03:11:51.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f812359e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f81240a8960 0x7f81240c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:51.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f812359e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f81240a8d40 0x7f812415c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:51.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f812359e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81240657f0 con 0x7f8124066520 2023-11-15T03:11:51.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f812359e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81241583d0 con 0x7f81240a8960 2023-11-15T03:11:51.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f812359e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81240c1b60 con 0x7f81240a8d40 2023-11-15T03:11:51.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f812259c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f81240a8960 0x7f81240c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:51.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f8122d9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f8124066520 0x7f81240c2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59100/0 (socket says 172.21.15.121:59100) 2023-11-15T03:11:51.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f812259c700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f81240a8960 0x7f81240c15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56028/0 (socket says 172.21.15.121:56028) 2023-11-15T03:11:51.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f8122d9d700 1 -- 172.21.15.121:0/2971883420 learned_addr learned my addr 172.21.15.121:0/2971883420 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:51.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f8121d9b700 1 --2- 172.21.15.121:0/2971883420 >> v2:172.21.15.190:3300/0 conn(0x7f81240a8d40 0x7f812415c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:51.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f812259c700 1 -- 172.21.15.121:0/2971883420 >> v2:172.21.15.190:3300/0 conn(0x7f81240a8d40 msgr2=0x7f812415c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:51.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f812259c700 1 --2- 172.21.15.121:0/2971883420 >> v2:172.21.15.190:3300/0 conn(0x7f81240a8d40 0x7f812415c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:51.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f812259c700 1 -- 172.21.15.121:0/2971883420 >> v1:172.21.15.121:6789/0 conn(0x7f8124066520 legacy=0x7f81240c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:51.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f812159a700 1 -- 172.21.15.121:0/2971883420 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058903029 0 0) 0x7f81240657f0 con 0x7f8124066520 2023-11-15T03:11:51.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f812259c700 1 -- 172.21.15.121:0/2971883420 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8124065a40 con 0x7f81240a8960 2023-11-15T03:11:51.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f8121d9b700 1 --2- 172.21.15.121:0/2971883420 >> v2:172.21.15.190:3300/0 conn(0x7f81240a8d40 0x7f812415c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:51.628 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f812259c700 1 --2- 172.21.15.121:0/2971883420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81240a8960 0x7f81240c15c0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 rx=0x7f81240654e0 tx=0x7f8114009060).ready entity=mon.0 client_cookie=ff5edc1edc13560b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:51.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f812159a700 1 -- 172.21.15.121:0/2971883420 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811400f510 con 0x7f81240a8960 2023-11-15T03:11:51.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f812159a700 1 -- 172.21.15.121:0/2971883420 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811400f670 con 0x7f81240a8960 2023-11-15T03:11:51.629 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.615+0000 7f812159a700 1 -- 172.21.15.121:0/2971883420 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811400f9f0 con 0x7f81240a8960 2023-11-15T03:11:51.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.619+0000 7f812359e700 1 -- 172.21.15.121:0/2971883420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81240a8960 msgr2=0x7f81240c15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:51.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.619+0000 7f812359e700 1 --2- 172.21.15.121:0/2971883420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81240a8960 0x7f81240c15c0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 rx=0x7f81240654e0 tx=0x7f8114009060).stop 2023-11-15T03:11:51.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.619+0000 7f812359e700 1 -- 172.21.15.121:0/2971883420 shutdown_connections 2023-11-15T03:11:51.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.619+0000 7f812359e700 1 --2- 172.21.15.121:0/2971883420 >> v2:172.21.15.190:3300/0 conn(0x7f81240a8d40 0x7f812415c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:51.630 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.619+0000 7f812359e700 1 --2- 172.21.15.121:0/2971883420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81240a8960 0x7f81240c15c0 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:51.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.619+0000 7f812359e700 1 -- 172.21.15.121:0/2971883420 >> 172.21.15.121:0/2971883420 conn(0x7f8124067a60 msgr2=0x7f8124067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:51.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.619+0000 7f812359e700 1 -- 172.21.15.121:0/2971883420 shutdown_connections 2023-11-15T03:11:51.631 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.619+0000 7f812359e700 1 -- 172.21.15.121:0/2971883420 wait complete. 2023-11-15T03:11:51.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.619+0000 7f812359e700 1 Processor -- start 2023-11-15T03:11:51.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f812359e700 1 -- start start 2023-11-15T03:11:51.633 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f812359e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8124066520 0x7f812415a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:51.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f812359e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81240a8960 0x7f81240be440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:51.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f812359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81240a8d40 0x7f81240aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:51.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f812359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8124168b90 con 0x7f81240a8d40 2023-11-15T03:11:51.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f812359e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8124167fc0 con 0x7f8124066520 2023-11-15T03:11:51.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f812359e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8124168260 con 0x7f81240a8960 2023-11-15T03:11:51.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f8122d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81240a8d40 0x7f81240aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:51.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f8122d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81240a8d40 0x7f81240aafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56040/0 (socket says 172.21.15.121:56040) 2023-11-15T03:11:51.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f8122d9d700 1 -- 172.21.15.121:0/1730742974 learned_addr learned my addr 172.21.15.121:0/1730742974 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:51.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f812259c700 1 --2- 172.21.15.121:0/1730742974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8124066520 0x7f812415a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:51.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f8121d9b700 1 --2- 172.21.15.121:0/1730742974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81240a8960 0x7f81240be440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:51.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f8122d9d700 1 -- 172.21.15.121:0/1730742974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81240a8960 msgr2=0x7f81240be440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:51.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f8122d9d700 1 --2- 172.21.15.121:0/1730742974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81240a8960 0x7f81240be440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:51.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f8122d9d700 1 -- 172.21.15.121:0/1730742974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8124066520 msgr2=0x7f812415a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:51.637 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f8122d9d700 1 --2- 172.21.15.121:0/1730742974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8124066520 0x7f812415a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:51.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f8122d9d700 1 -- 172.21.15.121:0/1730742974 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81180092a0 con 0x7f81240a8d40 2023-11-15T03:11:51.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f8121d9b700 1 --2- 172.21.15.121:0/1730742974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81240a8960 0x7f81240be440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:51.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f8122d9d700 1 --2- 172.21.15.121:0/1730742974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81240a8d40 0x7f81240aafd0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 rx=0x7f811c00c730 tx=0x7f811c001520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:51.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f81127fc700 1 -- 172.21.15.121:0/1730742974 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c005d30 con 0x7f81240a8d40 2023-11-15T03:11:51.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f81127fc700 1 -- 172.21.15.121:0/1730742974 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811c005e90 con 0x7f81240a8d40 2023-11-15T03:11:51.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f812359e700 1 -- 172.21.15.121:0/1730742974 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81140092a0 con 0x7f81240a8d40 2023-11-15T03:11:51.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.623+0000 7f81127fc700 1 -- 172.21.15.121:0/1730742974 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c007340 con 0x7f81240a8d40 2023-11-15T03:11:51.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.627+0000 7f812359e700 1 -- 172.21.15.121:0/1730742974 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8124168820 con 0x7f81240a8d40 2023-11-15T03:11:51.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.627+0000 7f812359e700 1 -- 172.21.15.121:0/1730742974 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8124168d00 con 0x7f81240a8d40 2023-11-15T03:11:51.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.627+0000 7f81127fc700 1 -- 172.21.15.121:0/1730742974 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f811c0076f0 con 0x7f81240a8d40 2023-11-15T03:11:51.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.627+0000 7f81127fc700 1 --2- 172.21.15.121:0/1730742974 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f810405a510 0x7f810405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:51.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.627+0000 7f81127fc700 1 -- 172.21.15.121:0/1730742974 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f811c074460 con 0x7f81240a8d40 2023-11-15T03:11:51.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.631+0000 7f812259c700 1 --2- 172.21.15.121:0/1730742974 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f810405a510 0x7f810405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:51.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.631+0000 7f812259c700 1 --2- 172.21.15.121:0/1730742974 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f810405a510 0x7f810405c9d0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f81240c66e0 tx=0x7f8114017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:51.641 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.631+0000 7f81127fc700 1 -- 172.21.15.121:0/1730742974 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f811c047ee0 con 0x7f81240a8d40 2023-11-15T03:11:51.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.791+0000 7f812359e700 1 -- 172.21.15.121:0/1730742974 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8124068a70 con 0x7f81240a8d40 2023-11-15T03:11:51.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.799+0000 7f81127fc700 1 -- 172.21.15.121:0/1730742974 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f811c0079a0 con 0x7f81240a8d40 2023-11-15T03:11:51.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.803+0000 7f812359e700 1 -- 172.21.15.121:0/1730742974 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f810405a510 msgr2=0x7f810405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:51.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.803+0000 7f812359e700 1 --2- 172.21.15.121:0/1730742974 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f810405a510 0x7f810405c9d0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f81240c66e0 tx=0x7f8114017000).stop 2023-11-15T03:11:51.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.803+0000 7f812359e700 1 -- 172.21.15.121:0/1730742974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81240a8d40 msgr2=0x7f81240aafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:51.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.803+0000 7f812359e700 1 --2- 172.21.15.121:0/1730742974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81240a8d40 0x7f81240aafd0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 rx=0x7f811c00c730 tx=0x7f811c001520).stop 2023-11-15T03:11:51.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.803+0000 7f812359e700 1 -- 172.21.15.121:0/1730742974 shutdown_connections 2023-11-15T03:11:51.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.803+0000 7f812359e700 1 --2- 172.21.15.121:0/1730742974 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f810405a510 0x7f810405c9d0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:51.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.803+0000 7f812359e700 1 --2- 172.21.15.121:0/1730742974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81240a8d40 0x7f81240aafd0 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:51.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.803+0000 7f812359e700 1 --2- 172.21.15.121:0/1730742974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81240a8960 0x7f81240be440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:51.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.803+0000 7f812359e700 1 --2- 172.21.15.121:0/1730742974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8124066520 0x7f812415a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:51.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.803+0000 7f812359e700 1 -- 172.21.15.121:0/1730742974 >> 172.21.15.121:0/1730742974 conn(0x7f8124067a60 msgr2=0x7f81241577b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:51.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.803+0000 7f812359e700 1 -- 172.21.15.121:0/1730742974 shutdown_connections 2023-11-15T03:11:51.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.803+0000 7f812359e700 1 -- 172.21.15.121:0/1730742974 wait complete. 2023-11-15T03:11:51.830 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors' 2023-11-15T03:11:51.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.975+0000 7f7c39e5b700 1 Processor -- start 2023-11-15T03:11:51.987 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.979+0000 7f7c39e5b700 1 -- start start 2023-11-15T03:11:51.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.979+0000 7f7c39e5b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7c34153bb0 0x7f7c340a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:51.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.979+0000 7f7c39e5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c340aa020 0x7f7c34067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:51.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.979+0000 7f7c39e5b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c340cb5c0 con 0x7f7c34067f50 2023-11-15T03:11:51.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.979+0000 7f7c39e5b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c340aa6f0 con 0x7f7c34153bb0 2023-11-15T03:11:51.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.979+0000 7f7c39e5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c340c5900 con 0x7f7c340aa020 2023-11-15T03:11:51.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c337fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7c34153bb0 0x7f7c340a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:51.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c32ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c340aa020 0x7f7c34067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:51.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c337fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7c34153bb0 0x7f7c340a9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:58294/0 (socket says 172.21.15.121:58294) 2023-11-15T03:11:51.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c337fe700 1 -- 172.21.15.121:0/3138203066 learned_addr learned my addr 172.21.15.121:0/3138203066 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:51.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c32ffd700 1 -- 172.21.15.121:0/3138203066 >> v2:172.21.15.190:3300/0 conn(0x7f7c34153bb0 msgr2=0x7f7c340a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:51.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c32ffd700 1 --2- 172.21.15.121:0/3138203066 >> v2:172.21.15.190:3300/0 conn(0x7f7c34153bb0 0x7f7c340a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:51.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c32ffd700 1 -- 172.21.15.121:0/3138203066 >> v1:172.21.15.190:6789/0 conn(0x7f7c34067f50 legacy=0x7f7c341578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:51.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c32ffd700 1 -- 172.21.15.121:0/3138203066 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c340683e0 con 0x7f7c340aa020 2023-11-15T03:11:51.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c327fc700 1 -- 172.21.15.121:0/3138203066 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2817473771 0 0) 0x7f7c340cb5c0 con 0x7f7c34067f50 2023-11-15T03:11:51.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c337fe700 1 --2- 172.21.15.121:0/3138203066 >> v2:172.21.15.190:3300/0 conn(0x7f7c34153bb0 0x7f7c340a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:51.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c32ffd700 1 --2- 172.21.15.121:0/3138203066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c340aa020 0x7f7c34067a60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f7c28009760 tx=0x7f7c28009060).ready entity=mon.2 client_cookie=5269d102788c0d5b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:51.996 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c327fc700 1 -- 172.21.15.121:0/3138203066 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2800f510 con 0x7f7c340aa020 2023-11-15T03:11:51.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c327fc700 1 -- 172.21.15.121:0/3138203066 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2800f670 con 0x7f7c340aa020 2023-11-15T03:11:51.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c327fc700 1 -- 172.21.15.121:0/3138203066 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2800f9f0 con 0x7f7c340aa020 2023-11-15T03:11:51.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c39e5b700 1 -- 172.21.15.121:0/3138203066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c340aa020 msgr2=0x7f7c34067a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:51.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c39e5b700 1 --2- 172.21.15.121:0/3138203066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c340aa020 0x7f7c34067a60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f7c28009760 tx=0x7f7c28009060).stop 2023-11-15T03:11:51.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c39e5b700 1 -- 172.21.15.121:0/3138203066 shutdown_connections 2023-11-15T03:11:51.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c39e5b700 1 --2- 172.21.15.121:0/3138203066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c340aa020 0x7f7c34067a60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:51.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c39e5b700 1 --2- 172.21.15.121:0/3138203066 >> v2:172.21.15.190:3300/0 conn(0x7f7c34153bb0 0x7f7c340a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:51.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c39e5b700 1 -- 172.21.15.121:0/3138203066 >> 172.21.15.121:0/3138203066 conn(0x7f7c34058c50 msgr2=0x7f7c34059060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:51.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c39e5b700 1 -- 172.21.15.121:0/3138203066 shutdown_connections 2023-11-15T03:11:51.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.983+0000 7f7c39e5b700 1 -- 172.21.15.121:0/3138203066 wait complete. 2023-11-15T03:11:52.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.987+0000 7f7c39e5b700 1 Processor -- start 2023-11-15T03:11:52.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.987+0000 7f7c39e5b700 1 -- start start 2023-11-15T03:11:52.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.987+0000 7f7c39e5b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c34067f50 0x7f7c341520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:52.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.987+0000 7f7c39e5b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c340aa020 0x7f7c340ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:52.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.987+0000 7f7c39e5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c34153bb0 0x7f7c340ad310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:52.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.987+0000 7f7c39e5b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c340ad800 con 0x7f7c34067f50 2023-11-15T03:11:52.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.987+0000 7f7c39e5b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c340aaf30 con 0x7f7c340aa020 2023-11-15T03:11:52.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.987+0000 7f7c39e5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c340a4680 con 0x7f7c34153bb0 2023-11-15T03:11:52.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.987+0000 7f7c337fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c34067f50 0x7f7c341520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:52.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.987+0000 7f7c337fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c34067f50 0x7f7c341520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56044/0 (socket says 172.21.15.121:56044) 2023-11-15T03:11:52.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.987+0000 7f7c337fe700 1 -- 172.21.15.121:0/671985523 learned_addr learned my addr 172.21.15.121:0/671985523 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:52.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.987+0000 7f7c32ffd700 1 --2- 172.21.15.121:0/671985523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c340aa020 0x7f7c340ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:52.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.987+0000 7f7c33fff700 1 --2- 172.21.15.121:0/671985523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c34153bb0 0x7f7c340ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:52.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.987+0000 7f7c337fe700 1 -- 172.21.15.121:0/671985523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c34153bb0 msgr2=0x7f7c340ad310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:52.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.987+0000 7f7c337fe700 1 --2- 172.21.15.121:0/671985523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c34153bb0 0x7f7c340ad310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:52.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.987+0000 7f7c337fe700 1 -- 172.21.15.121:0/671985523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c340aa020 msgr2=0x7f7c340ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:52.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.987+0000 7f7c337fe700 1 --2- 172.21.15.121:0/671985523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c340aa020 0x7f7c340ace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:52.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.987+0000 7f7c337fe700 1 -- 172.21.15.121:0/671985523 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c24009370 con 0x7f7c34067f50 2023-11-15T03:11:52.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.991+0000 7f7c337fe700 1 --2- 172.21.15.121:0/671985523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c34067f50 0x7f7c341520a0 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 rx=0x7f7c24006d00 tx=0x7f7c240090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:52.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.991+0000 7f7c1b7fe700 1 -- 172.21.15.121:0/671985523 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2400c6a0 con 0x7f7c34067f50 2023-11-15T03:11:52.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.991+0000 7f7c1b7fe700 1 -- 172.21.15.121:0/671985523 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2400c800 con 0x7f7c34067f50 2023-11-15T03:11:52.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.991+0000 7f7c1b7fe700 1 -- 172.21.15.121:0/671985523 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2400cb80 con 0x7f7c34067f50 2023-11-15T03:11:52.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.991+0000 7f7c39e5b700 1 -- 172.21.15.121:0/671985523 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c280092a0 con 0x7f7c34067f50 2023-11-15T03:11:52.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.991+0000 7f7c39e5b700 1 -- 172.21.15.121:0/671985523 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c340aac40 con 0x7f7c34067f50 2023-11-15T03:11:52.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.991+0000 7f7c39e5b700 1 -- 172.21.15.121:0/671985523 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c340a4a20 con 0x7f7c34067f50 2023-11-15T03:11:52.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.991+0000 7f7c1b7fe700 1 -- 172.21.15.121:0/671985523 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7c2401b400 con 0x7f7c34067f50 2023-11-15T03:11:52.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.995+0000 7f7c1b7fe700 1 --2- 172.21.15.121:0/671985523 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c2005a510 0x7f7c2005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:52.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.995+0000 7f7c1b7fe700 1 -- 172.21.15.121:0/671985523 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f7c24074b00 con 0x7f7c34067f50 2023-11-15T03:11:52.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.995+0000 7f7c32ffd700 1 --2- 172.21.15.121:0/671985523 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c2005a510 0x7f7c2005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:52.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:51.995+0000 7f7c32ffd700 1 --2- 172.21.15.121:0/671985523 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c2005a510 0x7f7c2005c9d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f7c28009fd0 tx=0x7f7c28017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:52.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:52.003+0000 7f7c1b7fe700 1 -- 172.21.15.121:0/671985523 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7c24076e60 con 0x7f7c34067f50 2023-11-15T03:11:52.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:52.223+0000 7f7c39e5b700 1 -- 172.21.15.121:0/671985523 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]} v 0) v1 -- 0x7f7c3406c150 con 0x7f7c34067f50 2023-11-15T03:11:52.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:52.427+0000 7f7c1b7fe700 1 -- 172.21.15.121:0/671985523 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]}]=0 v769) v1 ==== 135+0+0 (secure 0 0 0) 0x7f7c24016070 con 0x7f7c34067f50 2023-11-15T03:11:52.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:52.435+0000 7f7c39e5b700 1 -- 172.21.15.121:0/671985523 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c2005a510 msgr2=0x7f7c2005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:52.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:52.435+0000 7f7c39e5b700 1 --2- 172.21.15.121:0/671985523 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c2005a510 0x7f7c2005c9d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f7c28009fd0 tx=0x7f7c28017000).stop 2023-11-15T03:11:52.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:52.435+0000 7f7c39e5b700 1 -- 172.21.15.121:0/671985523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c34067f50 msgr2=0x7f7c341520a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:52.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:52.435+0000 7f7c39e5b700 1 --2- 172.21.15.121:0/671985523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c34067f50 0x7f7c341520a0 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 rx=0x7f7c24006d00 tx=0x7f7c240090f0).stop 2023-11-15T03:11:52.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:52.435+0000 7f7c39e5b700 1 -- 172.21.15.121:0/671985523 shutdown_connections 2023-11-15T03:11:52.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:52.435+0000 7f7c39e5b700 1 --2- 172.21.15.121:0/671985523 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7c2005a510 0x7f7c2005c9d0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:52.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:52.435+0000 7f7c39e5b700 1 --2- 172.21.15.121:0/671985523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c34153bb0 0x7f7c340ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:52.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:52.435+0000 7f7c39e5b700 1 --2- 172.21.15.121:0/671985523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c340aa020 0x7f7c340ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:52.445 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:52.435+0000 7f7c39e5b700 1 --2- 172.21.15.121:0/671985523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c34067f50 0x7f7c341520a0 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:52.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:52.435+0000 7f7c39e5b700 1 -- 172.21.15.121:0/671985523 >> 172.21.15.121:0/671985523 conn(0x7f7c34058c50 msgr2=0x7f7c340c0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:52.446 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:52.435+0000 7f7c39e5b700 1 -- 172.21.15.121:0/671985523 shutdown_connections 2023-11-15T03:11:52.447 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:52.435+0000 7f7c39e5b700 1 -- 172.21.15.121:0/671985523 wait complete. 2023-11-15T03:11:52.468 INFO:tasks.cephfs_test_runner:That errors encountered during opening inos are properly propagated ... ok 2023-11-15T03:11:52.469 INFO:tasks.cephfs_test_runner: 2023-11-15T03:11:52.469 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-15T03:11:52.470 INFO:tasks.cephfs_test_runner:Ran 6 tests in 866.901s 2023-11-15T03:11:52.470 INFO:tasks.cephfs_test_runner: 2023-11-15T03:11:52.470 INFO:tasks.cephfs_test_runner:OK 2023-11-15T03:11:52.471 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-15T03:11:52.482 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-15T03:11:52.518 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-15T03:11:52.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:52.519 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T03:11:52.539 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-15T03:11:52.539 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-15T03:11:52.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:52.540 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-15T03:11:52.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:52.621 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-15T03:11:52.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:52.716 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link delete ceph-brx 2023-11-15T03:11:52.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:52.880 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-11-15T03:11:52.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:52.899 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link delete ceph-brx 2023-11-15T03:11:52.985 INFO:teuthology.orchestra.run.smithi121.stderr:Cannot find device "ceph-brx" 2023-11-15T03:11:52.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:11:52.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:52.990 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-11-15T03:11:53.009 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-15T03:11:53.010 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-15T03:11:53.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:53.011 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-15T03:11:53.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:53.092 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-15T03:11:53.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:53.205 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2023-11-15T03:11:53.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:53.357 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-11-15T03:11:53.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-15T03:11:53.379 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2023-11-15T03:11:53.468 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2023-11-15T03:11:53.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:11:53.471 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-15T03:11:53.482 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-11-15T03:11:53.483 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:11:53.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.647+0000 7f46208b9700 1 Processor -- start 2023-11-15T03:11:53.659 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.651+0000 7f46208b9700 1 -- start start 2023-11-15T03:11:53.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.651+0000 7f46208b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f461c0c2930 0x7f461c0c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:53.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.651+0000 7f46208b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461c151b00 con 0x7f461c14e4c0 2023-11-15T03:11:53.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.651+0000 7f46208b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461c0a6d20 con 0x7f461c0bcdb0 2023-11-15T03:11:53.662 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.651+0000 7f46208b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0c3260 con 0x7f461c0c2930 2023-11-15T03:11:53.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.651+0000 7f461b7fe700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f461c14e4c0 0x7f461c0a6610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59110/0 (socket says 172.21.15.121:59110) 2023-11-15T03:11:53.663 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.651+0000 7f461b7fe700 1 -- 172.21.15.121:0/95655836 learned_addr learned my addr 172.21.15.121:0/95655836 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:53.664 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.651+0000 7f4613fff700 1 --2- 172.21.15.121:0/95655836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f461c0c2930 0x7f461c0c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:53.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.651+0000 7f461affd700 1 -- 172.21.15.121:0/95655836 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499383244 0 0) 0x7f461c151b00 con 0x7f461c14e4c0 2023-11-15T03:11:53.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.655+0000 7f461affd700 1 -- 172.21.15.121:0/95655836 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46080033f0 con 0x7f461c14e4c0 2023-11-15T03:11:53.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.655+0000 7f4613fff700 1 -- 172.21.15.121:0/95655836 >> v1:172.21.15.190:6789/0 conn(0x7f461c0bcdb0 legacy=0x7f461c0bd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:53.665 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.655+0000 7f4613fff700 1 -- 172.21.15.121:0/95655836 >> v1:172.21.15.121:6789/0 conn(0x7f461c14e4c0 legacy=0x7f461c0a6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:53.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.655+0000 7f4613fff700 1 -- 172.21.15.121:0/95655836 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c150f10 con 0x7f461c0c2930 2023-11-15T03:11:53.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.655+0000 7f4613fff700 1 --2- 172.21.15.121:0/95655836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f461c0c2930 0x7f461c0c2d70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f4604009790 tx=0x7f4604009060).ready entity=mon.2 client_cookie=84b69ff6d0a7b46d server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:53.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.655+0000 7f461affd700 1 -- 172.21.15.121:0/95655836 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460400f510 con 0x7f461c0c2930 2023-11-15T03:11:53.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.655+0000 7f461affd700 1 -- 172.21.15.121:0/95655836 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460400f670 con 0x7f461c0c2930 2023-11-15T03:11:53.666 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.655+0000 7f461affd700 1 -- 172.21.15.121:0/95655836 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460400f9f0 con 0x7f461c0c2930 2023-11-15T03:11:53.667 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.655+0000 7f46208b9700 1 -- 172.21.15.121:0/95655836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f461c0c2930 msgr2=0x7f461c0c2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:53.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.655+0000 7f46208b9700 1 --2- 172.21.15.121:0/95655836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f461c0c2930 0x7f461c0c2d70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f4604009790 tx=0x7f4604009060).stop 2023-11-15T03:11:53.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.655+0000 7f46208b9700 1 -- 172.21.15.121:0/95655836 shutdown_connections 2023-11-15T03:11:53.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.655+0000 7f46208b9700 1 --2- 172.21.15.121:0/95655836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f461c0c2930 0x7f461c0c2d70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:53.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.655+0000 7f46208b9700 1 -- 172.21.15.121:0/95655836 >> 172.21.15.121:0/95655836 conn(0x7f461c069350 msgr2=0x7f461c090a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:53.668 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.655+0000 7f46208b9700 1 -- 172.21.15.121:0/95655836 shutdown_connections 2023-11-15T03:11:53.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.655+0000 7f46208b9700 1 -- 172.21.15.121:0/95655836 wait complete. 2023-11-15T03:11:53.669 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.659+0000 7f46208b9700 1 Processor -- start 2023-11-15T03:11:53.670 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.659+0000 7f46208b9700 1 -- start start 2023-11-15T03:11:53.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.659+0000 7f46208b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f461c0bcdb0 0x7f461c0acce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:53.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.659+0000 7f46208b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f461c0c2930 0x7f461c0a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:53.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.659+0000 7f46208b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f461c14e4c0 0x7f461c0a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:53.671 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.659+0000 7f46208b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0a8c30 con 0x7f461c0bcdb0 2023-11-15T03:11:53.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.659+0000 7f46208b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c165330 con 0x7f461c14e4c0 2023-11-15T03:11:53.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.659+0000 7f46208b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0a5d90 con 0x7f461c0c2930 2023-11-15T03:11:53.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.659+0000 7f461b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f461c0bcdb0 0x7f461c0acce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:53.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.663+0000 7f461b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f461c0bcdb0 0x7f461c0acce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56048/0 (socket says 172.21.15.121:56048) 2023-11-15T03:11:53.672 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.663+0000 7f461b7fe700 1 -- 172.21.15.121:0/5559088 learned_addr learned my addr 172.21.15.121:0/5559088 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:53.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.663+0000 7f461b7fe700 1 -- 172.21.15.121:0/5559088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f461c0c2930 msgr2=0x7f461c0a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:53.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.663+0000 7f461b7fe700 1 --2- 172.21.15.121:0/5559088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f461c0c2930 0x7f461c0a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:11:53.674 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.663+0000 7f461b7fe700 1 -- 172.21.15.121:0/5559088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f461c14e4c0 msgr2=0x7f461c0a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:53.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.663+0000 7f461b7fe700 1 --2- 172.21.15.121:0/5559088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f461c14e4c0 0x7f461c0a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:11:53.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.663+0000 7f461b7fe700 1 -- 172.21.15.121:0/5559088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46040092a0 con 0x7f461c0bcdb0 2023-11-15T03:11:53.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.663+0000 7f461b7fe700 1 --2- 172.21.15.121:0/5559088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f461c0bcdb0 0x7f461c0acce0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 rx=0x7f460c00c590 tx=0x7f460c007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:53.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.663+0000 7f46137fe700 1 -- 172.21.15.121:0/5559088 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c007e60 con 0x7f461c0bcdb0 2023-11-15T03:11:53.675 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.663+0000 7f46137fe700 1 -- 172.21.15.121:0/5559088 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460c0072e0 con 0x7f461c0bcdb0 2023-11-15T03:11:53.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.663+0000 7f46137fe700 1 -- 172.21.15.121:0/5559088 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c0057a0 con 0x7f461c0bcdb0 2023-11-15T03:11:53.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.663+0000 7f46208b9700 1 -- 172.21.15.121:0/5559088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461c0ad1d0 con 0x7f461c0bcdb0 2023-11-15T03:11:53.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.663+0000 7f46208b9700 1 -- 172.21.15.121:0/5559088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461c0a8670 con 0x7f461c0bcdb0 2023-11-15T03:11:53.676 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.663+0000 7f46208b9700 1 -- 172.21.15.121:0/5559088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461c165070 con 0x7f461c0bcdb0 2023-11-15T03:11:53.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.667+0000 7f46137fe700 1 -- 172.21.15.121:0/5559088 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f460c005b90 con 0x7f461c0bcdb0 2023-11-15T03:11:53.677 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.667+0000 7f46137fe700 1 --2- 172.21.15.121:0/5559088 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f45fc05a510 0x7f45fc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:53.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.667+0000 7f46137fe700 1 -- 172.21.15.121:0/5559088 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f460c073df0 con 0x7f461c0bcdb0 2023-11-15T03:11:53.678 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.667+0000 7f4613fff700 1 --2- 172.21.15.121:0/5559088 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f45fc05a510 0x7f45fc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:53.679 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.671+0000 7f4613fff700 1 --2- 172.21.15.121:0/5559088 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f45fc05a510 0x7f45fc05c9d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7f461c0a6010 tx=0x7f4604017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:53.685 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.675+0000 7f46137fe700 1 -- 172.21.15.121:0/5559088 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f460c047870 con 0x7f461c0bcdb0 2023-11-15T03:11:53.850 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.839+0000 7f46208b9700 1 -- 172.21.15.121:0/5559088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f461c165470 con 0x7f461c0bcdb0 2023-11-15T03:11:53.852 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.843+0000 7f46137fe700 1 -- 172.21.15.121:0/5559088 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+14033 (secure 0 0 0) 0x7f460c050e60 con 0x7f461c0bcdb0 2023-11-15T03:11:53.853 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:53.853 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":151,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:11:51.405249+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6837/3298117008":"2023-11-16T03:11:47.845872+0000","172.21.15.121:6836/3298117008":"2023-11-16T03:11:47.845872+0000","172.21.15.121:6837/2335356851":"2023-11-16T03:11:31.820010+0000","172.21.15.121:6836/2335356851":"2023-11-16T03:11:31.820010+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:11:53.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.843+0000 7f46208b9700 1 -- 172.21.15.121:0/5559088 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f45fc05a510 msgr2=0x7f45fc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:53.855 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.843+0000 7f46208b9700 1 --2- 172.21.15.121:0/5559088 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f45fc05a510 0x7f45fc05c9d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7f461c0a6010 tx=0x7f4604017000).stop 2023-11-15T03:11:53.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.843+0000 7f46208b9700 1 -- 172.21.15.121:0/5559088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f461c0bcdb0 msgr2=0x7f461c0acce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:53.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.843+0000 7f46208b9700 1 --2- 172.21.15.121:0/5559088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f461c0bcdb0 0x7f461c0acce0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 rx=0x7f460c00c590 tx=0x7f460c007c80).stop 2023-11-15T03:11:53.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.843+0000 7f46208b9700 1 -- 172.21.15.121:0/5559088 shutdown_connections 2023-11-15T03:11:53.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.843+0000 7f46208b9700 1 --2- 172.21.15.121:0/5559088 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f45fc05a510 0x7f45fc05c9d0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:53.856 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.843+0000 7f46208b9700 1 --2- 172.21.15.121:0/5559088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f461c14e4c0 0x7f461c0a58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:11:53.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.843+0000 7f46208b9700 1 --2- 172.21.15.121:0/5559088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f461c0c2930 0x7f461c0a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-15T03:11:53.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.843+0000 7f46208b9700 1 --2- 172.21.15.121:0/5559088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f461c0bcdb0 0x7f461c0acce0 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:53.857 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.843+0000 7f46208b9700 1 -- 172.21.15.121:0/5559088 >> 172.21.15.121:0/5559088 conn(0x7f461c069350 msgr2=0x7f461c0bd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:53.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.847+0000 7f46208b9700 1 -- 172.21.15.121:0/5559088 shutdown_connections 2023-11-15T03:11:53.858 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:53.847+0000 7f46208b9700 1 -- 172.21.15.121:0/5559088 wait complete. 2023-11-15T03:11:53.875 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-15T03:11:53.875 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T03:11:54.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.023+0000 7fab08a39700 1 Processor -- start 2023-11-15T03:11:54.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.027+0000 7fab08a39700 1 -- start start 2023-11-15T03:11:54.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.027+0000 7fab08a39700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fab04151870 0x7fab04151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:54.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.027+0000 7fab08a39700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab04152140 0x7fab040a9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:54.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.027+0000 7fab08a39700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab040657f0 con 0x7fab040a9d20 2023-11-15T03:11:54.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.027+0000 7fab08a39700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab04065ad0 con 0x7fab04151870 2023-11-15T03:11:54.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.027+0000 7fab08a39700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab040aa260 con 0x7fab04152140 2023-11-15T03:11:54.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.027+0000 7faafbfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab04152140 0x7fab040a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:54.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.027+0000 7fab037fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fab04151870 0x7fab04151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:54.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.027+0000 7fab037fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fab04151870 0x7fab04151c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:58326/0 (socket says 172.21.15.121:58326) 2023-11-15T03:11:54.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.027+0000 7faafbfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab04152140 0x7fab040a9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:48928/0 (socket says 172.21.15.121:48928) 2023-11-15T03:11:54.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.027+0000 7fab037fe700 1 -- 172.21.15.121:0/1362280634 learned_addr learned my addr 172.21.15.121:0/1362280634 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:54.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.027+0000 7fab03fff700 1 --1- 172.21.15.121:0/1362280634 >> v1:172.21.15.190:6789/0 conn(0x7fab040a9d20 0x7fab040c30e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:47248/0 (socket says 172.21.15.121:47248) 2023-11-15T03:11:54.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.027+0000 7fab037fe700 1 -- 172.21.15.121:0/1362280634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab04152140 msgr2=0x7fab040a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:54.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.027+0000 7fab037fe700 1 --2- 172.21.15.121:0/1362280634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab04152140 0x7fab040a9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.027+0000 7fab037fe700 1 -- 172.21.15.121:0/1362280634 >> v1:172.21.15.190:6789/0 conn(0x7fab040a9d20 legacy=0x7fab040c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:54.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.027+0000 7fab037fe700 1 -- 172.21.15.121:0/1362280634 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab040aa540 con 0x7fab04151870 2023-11-15T03:11:54.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.031+0000 7fab037fe700 1 --2- 172.21.15.121:0/1362280634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab04151870 0x7fab04151c50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7faaf4006fd0 tx=0x7faaf40090f0).ready entity=mon.1 client_cookie=4d3f8f7bdacd43ae server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:54.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.031+0000 7fab02ffd700 1 -- 172.21.15.121:0/1362280634 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf400b580 con 0x7fab04151870 2023-11-15T03:11:54.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.031+0000 7fab02ffd700 1 -- 172.21.15.121:0/1362280634 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf400b6e0 con 0x7fab04151870 2023-11-15T03:11:54.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.031+0000 7fab02ffd700 1 -- 172.21.15.121:0/1362280634 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf400ba60 con 0x7fab04151870 2023-11-15T03:11:54.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.031+0000 7fab08a39700 1 -- 172.21.15.121:0/1362280634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab04151870 msgr2=0x7fab04151c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:54.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.031+0000 7fab08a39700 1 --2- 172.21.15.121:0/1362280634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab04151870 0x7fab04151c50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7faaf4006fd0 tx=0x7faaf40090f0).stop 2023-11-15T03:11:54.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.031+0000 7fab08a39700 1 -- 172.21.15.121:0/1362280634 shutdown_connections 2023-11-15T03:11:54.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.031+0000 7fab08a39700 1 --2- 172.21.15.121:0/1362280634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab04152140 0x7fab040a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.031+0000 7fab08a39700 1 --2- 172.21.15.121:0/1362280634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab04151870 0x7fab04151c50 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.031+0000 7fab08a39700 1 -- 172.21.15.121:0/1362280634 >> 172.21.15.121:0/1362280634 conn(0x7fab04062190 msgr2=0x7fab04062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:54.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.031+0000 7fab08a39700 1 -- 172.21.15.121:0/1362280634 shutdown_connections 2023-11-15T03:11:54.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.031+0000 7fab08a39700 1 -- 172.21.15.121:0/1362280634 wait complete. 2023-11-15T03:11:54.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.035+0000 7fab08a39700 1 Processor -- start 2023-11-15T03:11:54.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.035+0000 7fab08a39700 1 -- start start 2023-11-15T03:11:54.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.035+0000 7fab08a39700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab040a9d20 0x7fab040a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:54.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.035+0000 7fab08a39700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab04151870 0x7fab040c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:54.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.035+0000 7fab08a39700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab04152140 0x7fab040c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:54.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.035+0000 7fab08a39700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab040c58d0 con 0x7fab04151870 2023-11-15T03:11:54.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.035+0000 7fab08a39700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab040c0e10 con 0x7fab04152140 2023-11-15T03:11:54.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.035+0000 7fab08a39700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab040c7280 con 0x7fab040a9d20 2023-11-15T03:11:54.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.035+0000 7faafbfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab04151870 0x7fab040c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:54.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.035+0000 7fab037fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab040a9d20 0x7fab040a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:54.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.035+0000 7faafbfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab04151870 0x7fab040c4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56060/0 (socket says 172.21.15.121:56060) 2023-11-15T03:11:54.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.035+0000 7faafbfff700 1 -- 172.21.15.121:0/3903203824 learned_addr learned my addr 172.21.15.121:0/3903203824 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:54.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.039+0000 7fab03fff700 1 --2- 172.21.15.121:0/3903203824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab04152140 0x7fab040c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:54.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.039+0000 7faafbfff700 1 -- 172.21.15.121:0/3903203824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab040a9d20 msgr2=0x7fab040a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:54.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.039+0000 7faafbfff700 1 --2- 172.21.15.121:0/3903203824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab040a9d20 0x7fab040a8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.039+0000 7faafbfff700 1 -- 172.21.15.121:0/3903203824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab04152140 msgr2=0x7fab040c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:54.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.039+0000 7faafbfff700 1 --2- 172.21.15.121:0/3903203824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab04152140 0x7fab040c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.039+0000 7faafbfff700 1 -- 172.21.15.121:0/3903203824 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf4009370 con 0x7fab04151870 2023-11-15T03:11:54.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.039+0000 7faafbfff700 1 --2- 172.21.15.121:0/3903203824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab04151870 0x7fab040c4ef0 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 rx=0x7faaec00e900 tx=0x7faaec009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:54.052 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.039+0000 7faafb7fe700 1 -- 172.21.15.121:0/3903203824 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec012600 con 0x7fab04151870 2023-11-15T03:11:54.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.039+0000 7faafb7fe700 1 -- 172.21.15.121:0/3903203824 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaec012760 con 0x7fab04151870 2023-11-15T03:11:54.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.039+0000 7fab08a39700 1 -- 172.21.15.121:0/3903203824 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab04168ab0 con 0x7fab04151870 2023-11-15T03:11:54.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.039+0000 7faafb7fe700 1 -- 172.21.15.121:0/3903203824 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec012ae0 con 0x7fab04151870 2023-11-15T03:11:54.053 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.039+0000 7fab08a39700 1 -- 172.21.15.121:0/3903203824 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab040c6ea0 con 0x7fab04151870 2023-11-15T03:11:54.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.039+0000 7fab08a39700 1 -- 172.21.15.121:0/3903203824 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab0406bcf0 con 0x7fab04151870 2023-11-15T03:11:54.054 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.043+0000 7faafb7fe700 1 -- 172.21.15.121:0/3903203824 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7faaec01b400 con 0x7fab04151870 2023-11-15T03:11:54.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.043+0000 7faafb7fe700 1 --2- 172.21.15.121:0/3903203824 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faae405a510 0x7faae405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:54.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.043+0000 7faafb7fe700 1 -- 172.21.15.121:0/3903203824 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7faaec074980 con 0x7fab04151870 2023-11-15T03:11:54.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.043+0000 7fab037fe700 1 --2- 172.21.15.121:0/3903203824 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faae405a510 0x7faae405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:54.055 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.047+0000 7fab037fe700 1 --2- 172.21.15.121:0/3903203824 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faae405a510 0x7faae405c9d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7faaf4009fd0 tx=0x7faaf4017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:54.059 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.051+0000 7faafb7fe700 1 -- 172.21.15.121:0/3903203824 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faaec0482e0 con 0x7fab04151870 2023-11-15T03:11:54.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.211+0000 7fab08a39700 1 -- 172.21.15.121:0/3903203824 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fab040aa820 con 0x7faae405a510 2023-11-15T03:11:54.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.215+0000 7faafb7fe700 1 -- 172.21.15.121:0/3903203824 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+68595 (secure 0 0 0) 0x7fab040aa820 con 0x7faae405a510 2023-11-15T03:11:54.228 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:54.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.219+0000 7fab08a39700 1 -- 172.21.15.121:0/3903203824 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faae405a510 msgr2=0x7faae405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:54.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.219+0000 7fab08a39700 1 --2- 172.21.15.121:0/3903203824 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faae405a510 0x7faae405c9d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7faaf4009fd0 tx=0x7faaf4017000).stop 2023-11-15T03:11:54.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.219+0000 7fab08a39700 1 -- 172.21.15.121:0/3903203824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab04151870 msgr2=0x7fab040c4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:54.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.219+0000 7fab08a39700 1 --2- 172.21.15.121:0/3903203824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab04151870 0x7fab040c4ef0 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 rx=0x7faaec00e900 tx=0x7faaec009060).stop 2023-11-15T03:11:54.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.219+0000 7fab08a39700 1 -- 172.21.15.121:0/3903203824 shutdown_connections 2023-11-15T03:11:54.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.219+0000 7fab08a39700 1 --2- 172.21.15.121:0/3903203824 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7faae405a510 0x7faae405c9d0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.219+0000 7fab08a39700 1 --2- 172.21.15.121:0/3903203824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab04152140 0x7fab040c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.219+0000 7fab08a39700 1 --2- 172.21.15.121:0/3903203824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab04151870 0x7fab040c4ef0 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.219+0000 7fab08a39700 1 --2- 172.21.15.121:0/3903203824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab040a9d20 0x7fab040a8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.219+0000 7fab08a39700 1 -- 172.21.15.121:0/3903203824 >> 172.21.15.121:0/3903203824 conn(0x7fab04062190 msgr2=0x7fab04150810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:54.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.219+0000 7fab08a39700 1 -- 172.21.15.121:0/3903203824 shutdown_connections 2023-11-15T03:11:54.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.219+0000 7fab08a39700 1 -- 172.21.15.121:0/3903203824 wait complete. 2023-11-15T03:11:54.233 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T03:11:54.249 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":598,"stamp":"2023-11-15T03:11:52.268127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2445244,"kb_used_data":15228,"kb_used_omap":0,"kb_used_meta":2429952,"kb_avail":752529476,"statfs":{"total":773094113280,"available":770590183424,"internally_reserved":0,"allocated":15593472,"data_stored":7928287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2488270848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-33941,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-60,"num_read_kb":-61,"num_write":-90,"num_write_kb":-62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.146079"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":127,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-15T03:11:40.337507+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T03:11:40.337507+0000","last_peered":"2023-11-15T03:11:40.337507+0000","last_clean":"2023-11-15T03:11:40.337507+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T03:11:40.337507+0000","last_undegraded":"2023-11-15T03:11:40.337507+0000","last_fullsized":"2023-11-15T03:11:40.337507+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":130,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-15T03:11:40.865807+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T03:11:40.865807+0000","last_peered":"2023-11-15T03:11:40.865807+0000","last_clean":"2023-11-15T03:11:40.865807+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T03:11:40.865807+0000","last_undegraded":"2023-11-15T03:11:40.865807+0000","last_fullsized":"2023-11-15T03:11:40.865807+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":125,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-15T03:11:40.337607+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T03:11:40.337607+0000","last_peered":"2023-11-15T03:11:40.337607+0000","last_clean":"2023-11-15T03:11:40.337607+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T03:11:40.337607+0000","last_undegraded":"2023-11-15T03:11:40.337607+0000","last_fullsized":"2023-11-15T03:11:40.337607+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":130,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-15T03:11:40.865250+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T03:11:40.865250+0000","last_peered":"2023-11-15T03:11:40.865250+0000","last_clean":"2023-11-15T03:11:40.865250+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T03:11:40.865250+0000","last_undegraded":"2023-11-15T03:11:40.865250+0000","last_fullsized":"2023-11-15T03:11:40.865250+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":141,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-15T03:11:40.330904+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T03:11:40.330904+0000","last_peered":"2023-11-15T03:11:40.330904+0000","last_clean":"2023-11-15T03:11:40.330904+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T03:11:40.330904+0000","last_undegraded":"2023-11-15T03:11:40.330904+0000","last_fullsized":"2023-11-15T03:11:40.330904+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":125,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-15T03:11:40.324057+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T03:11:40.324057+0000","last_peered":"2023-11-15T03:11:40.324057+0000","last_clean":"2023-11-15T03:11:40.324057+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T03:11:40.324057+0000","last_undegraded":"2023-11-15T03:11:40.324057+0000","last_fullsized":"2023-11-15T03:11:40.324057+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":125,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-15T03:11:40.338063+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T03:11:40.338063+0000","last_peered":"2023-11-15T03:11:40.338063+0000","last_clean":"2023-11-15T03:11:40.338063+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T03:11:40.338063+0000","last_undegraded":"2023-11-15T03:11:40.338063+0000","last_fullsized":"2023-11-15T03:11:40.338063+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":141,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-15T03:11:40.331075+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T03:11:40.331075+0000","last_peered":"2023-11-15T03:11:40.331075+0000","last_clean":"2023-11-15T03:11:40.331075+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T03:11:40.331075+0000","last_undegraded":"2023-11-15T03:11:40.331075+0000","last_fullsized":"2023-11-15T03:11:40.331075+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":126,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-15T03:11:40.338089+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T03:11:40.338089+0000","last_peered":"2023-11-15T03:11:40.338089+0000","last_clean":"2023-11-15T03:11:40.338089+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T03:11:40.338089+0000","last_undegraded":"2023-11-15T03:11:40.338089+0000","last_fullsized":"2023-11-15T03:11:40.338089+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542327,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307180,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064660,"statfs":{"total":96636764160,"available":96322211840,"internally_reserved":0,"allocated":1937408,"data_stored":982764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.0940000000000001,"15min":1.1319999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.55800000000000005},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":10.961},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.032,"5min":1.0880000000000001,"15min":1.1120000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.3089999999999999,"5min":1.6160000000000001,"15min":3.4910000000000001},"last":0.76300000000000001}]},{"osd":1,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.127,"15min":1.143},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.419,"5min":1.552,"15min":5.4930000000000003},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.153,"5min":1.165,"15min":1.1819999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6000000000000001,"5min":1.766,"15min":6.8239999999999998},"last":0.97799999999999998}]},{"osd":2,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0940000000000001,"15min":1.1359999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.452,"5min":1.452,"15min":5.7000000000000002},"last":1.032},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.226,"15min":1.2310000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":2.75},"last":1.28}]},{"osd":3,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.167,"15min":1.248},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":12.907},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2370000000000001,"15min":1.2709999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.573,"5min":1.573,"15min":9.9909999999999997},"last":1.3720000000000001}]},{"osd":4,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.105,"15min":1.1379999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6160000000000001,"5min":1.8939999999999999,"15min":2.5649999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.1240000000000001,"15min":1.2070000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.72,"5min":1.72,"15min":4.226},"last":1.1339999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.2450000000000001,"15min":1.2749999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":2.6890000000000001},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.143,"5min":1.236,"15min":1.292},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6899999999999999,"5min":1.7330000000000001,"15min":2.4550000000000001},"last":1.071}]},{"osd":6,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.325,"15min":1.381},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.5629999999999999,"5min":1.954,"15min":3.2789999999999999},"last":1.427},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.248,"15min":1.3340000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.63600000000000001},"max":{"1min":1.542,"5min":1.76,"15min":4.7800000000000002},"last":1.2989999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542327,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311288,"kb_used_data":1904,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94060552,"statfs":{"total":96636764160,"available":96318005248,"internally_reserved":0,"allocated":1949696,"data_stored":983787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.03,"15min":1.077},"min":{"1min":0.66200000000000003,"5min":0.63100000000000001,"15min":0.44500000000000001},"max":{"1min":1.4610000000000001,"5min":1.5760000000000001,"15min":4.3159999999999998},"last":0.753},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.109,"15min":1.0960000000000001},"min":{"1min":0.69999999999999996,"5min":0.59799999999999998,"15min":0.58599999999999997},"max":{"1min":1.486,"5min":1.659,"15min":2.2480000000000002},"last":1.4750000000000001}]},{"osd":1,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.111,"15min":1.1140000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.57599999999999996},"max":{"1min":1.6000000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1000000000000001,"15min":1.1379999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.60699999999999998},"max":{"1min":1.3360000000000001,"5min":1.8100000000000001,"15min":2.8570000000000002},"last":0.90100000000000002}]},{"osd":2,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1899999999999999,"15min":1.1919999999999999},"min":{"1min":0.83099999999999996,"5min":0.71699999999999997,"15min":0.49099999999999999},"max":{"1min":1.4730000000000001,"5min":1.7090000000000001,"15min":2.621},"last":1.155},{"interface":"front","average":{"1min":1.115,"5min":1.153,"15min":1.153},"min":{"1min":0.71699999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.4710000000000001,"5min":1.8360000000000001,"15min":2.0059999999999998},"last":1.2589999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.2310000000000001,"15min":1.214},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.50900000000000001},"max":{"1min":1.7709999999999999,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.234,"5min":1.2430000000000001,"15min":1.2609999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.67100000000000004},"max":{"1min":1.458,"5min":1.8640000000000001,"15min":2.8359999999999999},"last":0.98599999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.0580000000000001,"15min":1.1000000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5860000000000001,"5min":1.6839999999999999,"15min":4.5350000000000001},"last":1.093},{"interface":"front","average":{"1min":1.163,"5min":1.2170000000000001,"15min":1.2130000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.8580000000000001,"5min":2.004,"15min":2.0739999999999998},"last":1.081}]},{"osd":5,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3799999999999999,"15min":1.415},"min":{"1min":0.79800000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.744,"5min":2.1469999999999998,"15min":2.157},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2210000000000001,"15min":1.256},"min":{"1min":0.621,"5min":0.62,"15min":0.60899999999999999},"max":{"1min":1.8140000000000001,"5min":1.8779999999999999,"15min":2.9460000000000002},"last":0.97399999999999998}]},{"osd":7,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.216,"15min":1.2250000000000001},"min":{"1min":0.77400000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.9139999999999999,"5min":1.9239999999999999,"15min":3.1989999999999998},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.407,"15min":1.419},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.65200000000000002},"max":{"1min":1.8220000000000001,"5min":2.0649999999999999,"15min":2.9350000000000001},"last":1.323}]}]},{"osd":4,"up_from":14,"seq":60129542328,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307212,"kb_used_data":1924,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064628,"statfs":{"total":96636764160,"available":96322179072,"internally_reserved":0,"allocated":1970176,"data_stored":1014901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.0569999999999999,"15min":1.083},"min":{"1min":0.73799999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4810000000000001,"5min":1.5660000000000001,"15min":1.5840000000000001},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.044,"15min":1.056},"min":{"1min":0.85399999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.677},"last":1.319}]},{"osd":1,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.135,"15min":1.1319999999999999},"min":{"1min":0.70599999999999996,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":2.617},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.1419999999999999,"15min":1.1539999999999999},"min":{"1min":0.82099999999999995,"5min":0.629,"15min":0.54800000000000004},"max":{"1min":1.573,"5min":1.6319999999999999,"15min":1.6619999999999999},"last":1.2809999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.165,"15min":1.1830000000000001},"min":{"1min":0.81399999999999995,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.5029999999999999,"5min":1.823,"15min":1.9450000000000001},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.155,"15min":1.1619999999999999},"min":{"1min":0.84199999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9330000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.044}]},{"osd":3,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1399999999999999,"15min":1.1459999999999999},"min":{"1min":0.85699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.5229999999999999,"5min":1.5940000000000001,"15min":1.8660000000000001},"last":1.107},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2549999999999999,"15min":1.264},"min":{"1min":0.93000000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.611,"5min":1.8200000000000001,"15min":2.5630000000000002},"last":1.0029999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1479999999999999,"15min":1.1830000000000001},"min":{"1min":0.91900000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.554,"5min":1.643,"15min":2.0840000000000001},"last":1.387},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.1479999999999999,"15min":1.1879999999999999},"min":{"1min":0.89400000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":2.3570000000000002},"last":1.125}]},{"osd":6,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2410000000000001,"15min":1.264},"min":{"1min":0.97999999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5880000000000001,"5min":1.913,"15min":2.6640000000000001},"last":1.264},{"interface":"front","average":{"1min":1.325,"5min":1.288,"15min":1.323},"min":{"1min":0.88,"5min":0.67600000000000005,"15min":0.57299999999999995},"max":{"1min":1.585,"5min":1.9139999999999999,"15min":2.173},"last":1.5680000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2769999999999999,"15min":1.3069999999999999},"min":{"1min":0.89500000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.7629999999999999,"5min":1.9199999999999999,"15min":2.6030000000000002},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.3700000000000001,"15min":1.3879999999999999},"min":{"1min":1.044,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8240000000000001,"5min":2.069,"15min":2.7669999999999999},"last":1.306}]}]},{"osd":3,"up_from":14,"seq":60129542328,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303084,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068756,"statfs":{"total":96636764160,"available":96326406144,"internally_reserved":0,"allocated":1937408,"data_stored":983121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85999999999999999,"15min":0.83699999999999997},"min":{"1min":0.54200000000000004,"5min":0.54000000000000004,"15min":0.45400000000000001},"max":{"1min":1.327,"5min":1.4219999999999999,"15min":1.528},"last":0.746},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.89900000000000002,"15min":0.88800000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.39800000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.79700000000000004}]},{"osd":1,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.83899999999999997,"15min":0.872},"min":{"1min":0.62,"5min":0.53100000000000003,"15min":0.435},"max":{"1min":1.165,"5min":1.25,"15min":1.4670000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.996,"5min":0.98599999999999999,"15min":0.99199999999999999},"min":{"1min":0.72199999999999998,"5min":0.63700000000000001,"15min":0.55800000000000005},"max":{"1min":1.446,"5min":1.446,"15min":1.544},"last":1.016}]},{"osd":2,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.026,"15min":1.0209999999999999},"min":{"1min":0.66400000000000003,"5min":0.64200000000000002,"15min":0.54600000000000004},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0529999999999999,"15min":1.0660000000000001},"min":{"1min":0.66000000000000003,"5min":0.63100000000000001,"15min":0.50600000000000001},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.70099999999999996}]},{"osd":4,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2470000000000001,"15min":1.228},"min":{"1min":1.0029999999999999,"5min":0.88600000000000001,"15min":0.71399999999999997},"max":{"1min":1.5349999999999999,"5min":1.554,"15min":1.9139999999999999},"last":1.002},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2170000000000001,"15min":1.2250000000000001},"min":{"1min":1.01,"5min":0.78500000000000003,"15min":0.621},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.726},"last":0.95499999999999996}]},{"osd":5,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.252,"15min":1.266},"min":{"1min":0.91700000000000004,"5min":0.90200000000000002,"15min":0.64300000000000002},"max":{"1min":1.5900000000000001,"5min":1.7290000000000001,"15min":2.0459999999999998},"last":1.085},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.27,"15min":1.272},"min":{"1min":0.95899999999999996,"5min":0.94799999999999995,"15min":0.63500000000000001},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.927},"last":0.94499999999999995}]},{"osd":6,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.345,"15min":1.339},"min":{"1min":0.91400000000000003,"5min":0.875,"15min":0.72899999999999998},"max":{"1min":1.575,"5min":1.8129999999999999,"15min":1.9970000000000001},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3560000000000001,"15min":1.3720000000000001},"min":{"1min":1.0980000000000001,"5min":0.871,"15min":0.77700000000000002},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":2.3570000000000002},"last":1.4750000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.331,"15min":1.3440000000000001},"min":{"1min":1.0149999999999999,"5min":0.92500000000000004,"15min":0.68100000000000005},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":2.0470000000000002},"last":1.26},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.345,"15min":1.355},"min":{"1min":1.113,"5min":0.97299999999999998,"15min":0.66600000000000004},"max":{"1min":1.8500000000000001,"5min":1.8779999999999999,"15min":2.1040000000000001},"last":1.0449999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542328,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303084,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068756,"statfs":{"total":96636764160,"available":96326406144,"internally_reserved":0,"allocated":1937408,"data_stored":981525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.65400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97099999999999997,"5min":1.2350000000000001,"15min":1.409},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.84299999999999997,"15min":0.84399999999999997},"min":{"1min":0.58799999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2689999999999999,"5min":1.4950000000000001,"15min":1.619},"last":0.58499999999999996}]},{"osd":1,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.90000000000000002,"15min":0.91100000000000003},"min":{"1min":0.74199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.95599999999999996,"15min":0.95399999999999996},"min":{"1min":0.67200000000000004,"5min":0.55100000000000005,"15min":0.54600000000000004},"max":{"1min":1.3580000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":1.01}]},{"osd":3,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95599999999999996,"15min":0.97199999999999998},"min":{"1min":0.60699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.385,"5min":1.4359999999999999,"15min":1.6259999999999999},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.125,"15min":1.125},"min":{"1min":0.67700000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.488,"5min":1.5549999999999999,"15min":1.5800000000000001},"last":1.492}]},{"osd":4,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.218,"15min":1.2070000000000001},"min":{"1min":1.052,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.581,"5min":1.583,"15min":1.9410000000000001},"last":1.595},{"interface":"front","average":{"1min":1.3,"5min":1.21,"15min":1.212},"min":{"1min":0.99199999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6579999999999999,"5min":1.7170000000000001,"15min":1.7829999999999999},"last":1.1699999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1619999999999999,"15min":1.1830000000000001},"min":{"1min":0.98499999999999999,"5min":0.68100000000000005,"15min":0.67000000000000004},"max":{"1min":1.47,"5min":1.625,"15min":1.8939999999999999},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.266,"15min":1.272},"min":{"1min":1.0660000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.556,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.323}]},{"osd":6,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.262,"15min":1.2729999999999999},"min":{"1min":0.93999999999999995,"5min":0.62,"15min":0.62},"max":{"1min":1.6990000000000001,"5min":2,"15min":2.137},"last":1.294},{"interface":"front","average":{"1min":1.292,"5min":1.2669999999999999,"15min":1.2689999999999999},"min":{"1min":1.0880000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.538,"5min":1.9350000000000001,"15min":2.0870000000000002},"last":1.2549999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.29,"15min":1.3109999999999999},"min":{"1min":0.95699999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7789999999999999,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.298,"15min":1.2989999999999999},"min":{"1min":1.101,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.7270000000000001,"5min":1.881,"15min":2.0350000000000001},"last":1.6220000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542328,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307188,"kb_used_data":1900,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064652,"statfs":{"total":96636764160,"available":96322203648,"internally_reserved":0,"allocated":1945600,"data_stored":983408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80600000000000005,"15min":0.83299999999999996},"min":{"1min":0.503,"5min":0.44400000000000001,"15min":0.36199999999999999},"max":{"1min":1.093,"5min":1.756,"15min":4.3630000000000004},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.90600000000000003,"15min":0.93700000000000006},"min":{"1min":0.61799999999999999,"5min":0.52600000000000002,"15min":0.40000000000000002},"max":{"1min":1.2490000000000001,"5min":1.6719999999999999,"15min":6.157},"last":0.96799999999999997}]},{"osd":2,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.96399999999999997,"15min":0.97899999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.46800000000000003},"max":{"1min":1.3400000000000001,"5min":1.625,"15min":4.1699999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0029999999999999,"15min":1.0289999999999999},"min":{"1min":0.72199999999999998,"5min":0.60099999999999998,"15min":0.47799999999999998},"max":{"1min":1.379,"5min":1.879,"15min":6.8369999999999997},"last":0.874}]},{"osd":3,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.94099999999999995,"15min":0.95599999999999996},"min":{"1min":0.59499999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.3120000000000001,"5min":1.6519999999999999,"15min":3.5070000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.073,"5min":1.093,"15min":1.113},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56200000000000006},"max":{"1min":1.3460000000000001,"5min":2.0070000000000001,"15min":5.2149999999999999},"last":0.84799999999999998}]},{"osd":4,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.224,"15min":1.2609999999999999},"min":{"1min":0.81499999999999995,"5min":0.80500000000000005,"15min":0.66700000000000004},"max":{"1min":1.5900000000000001,"5min":1.944,"15min":6.2000000000000002},"last":1.157},{"interface":"front","average":{"1min":1.165,"5min":1.1870000000000001,"15min":1.196},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.3939999999999999,"5min":1.6779999999999999,"15min":3.577},"last":1.2490000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.23,"15min":1.25},"min":{"1min":0.93000000000000005,"5min":0.81499999999999995,"15min":0.69599999999999995},"max":{"1min":1.478,"5min":1.857,"15min":3.548},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.21,"5min":1.278,"15min":1.2969999999999999},"min":{"1min":0.86199999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.4330000000000001,"5min":1.774,"15min":4.4080000000000004},"last":1.1419999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3180000000000001,"15min":1.349},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.82299999999999995},"max":{"1min":1.5640000000000001,"5min":1.915,"15min":4.4489999999999998},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.2889999999999999,"15min":1.3340000000000001},"min":{"1min":0.93300000000000005,"5min":0.86499999999999999,"15min":0.75900000000000001},"max":{"1min":1.5409999999999999,"5min":2.056,"15min":6.9690000000000003},"last":1.274}]},{"osd":7,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.327,"15min":1.401},"min":{"1min":1.036,"5min":0.90400000000000003,"15min":0.79900000000000004},"max":{"1min":1.6319999999999999,"5min":1.994,"15min":18.253},"last":1.361},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2889999999999999,"15min":1.3380000000000001},"min":{"1min":0.997,"5min":0.95599999999999996,"15min":0.79000000000000004},"max":{"1min":1.452,"5min":1.8720000000000001,"15min":11.006},"last":1.1779999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575032,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303084,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068756,"statfs":{"total":96636764160,"available":96326406144,"internally_reserved":0,"allocated":1937408,"data_stored":983228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0800000000000001,"15min":1.073},"min":{"1min":0.65800000000000003,"5min":0.627,"15min":0.49399999999999999},"max":{"1min":1.827,"5min":1.827,"15min":2.3370000000000002},"last":1.351},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0589999999999999},"min":{"1min":0.63600000000000001,"5min":0.61099999999999999,"15min":0.60799999999999998},"max":{"1min":1.3580000000000001,"5min":1.6519999999999999,"15min":3.46},"last":1.1799999999999999}]},{"osd":1,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.105,"15min":1.121},"min":{"1min":0.84599999999999997,"5min":0.71199999999999997,"15min":0.58999999999999997},"max":{"1min":1.4570000000000001,"5min":1.516,"15min":3.6349999999999998},"last":1.123},{"interface":"front","average":{"1min":1.212,"5min":1.161,"15min":1.143},"min":{"1min":0.84199999999999997,"5min":0.73999999999999999,"15min":0.63700000000000001},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":3.516},"last":1.0740000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.1459999999999999,"15min":1.165},"min":{"1min":0.88600000000000001,"5min":0.74099999999999999,"15min":0.73199999999999998},"max":{"1min":1.774,"5min":1.774,"15min":5.4180000000000001},"last":1.23},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1899999999999999,"15min":1.204},"min":{"1min":0.77300000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6839999999999999,"5min":2.0840000000000001,"15min":3.5859999999999999},"last":1.4730000000000001}]},{"osd":3,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2070000000000001,"15min":1.196},"min":{"1min":0.91400000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6439999999999999,"5min":1.8560000000000001,"15min":2.141},"last":1.7749999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.296,"15min":1.3169999999999999},"min":{"1min":0.88400000000000001,"5min":0.875,"15min":0.78600000000000003},"max":{"1min":1.6140000000000001,"5min":1.6739999999999999,"15min":4.2729999999999997},"last":1.5029999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.2130000000000001,"15min":1.26},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.4850000000000001,"5min":1.591,"15min":3.706},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.258,"15min":1.3080000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.64100000000000001},"max":{"1min":1.863,"5min":1.9219999999999999,"15min":6.1180000000000003},"last":1.9550000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.268,"15min":1.254},"min":{"1min":0.88800000000000001,"5min":0.80700000000000005,"15min":0.749},"max":{"1min":1.6080000000000001,"5min":1.843,"15min":4.2309999999999999},"last":1.6910000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.2869999999999999,"15min":1.341},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.546,"5min":2.2240000000000002,"15min":3.641},"last":1.7230000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.284,"15min":1.2949999999999999},"min":{"1min":0.83499999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6970000000000001,"5min":1.988,"15min":3.516},"last":1.883},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.379,"15min":1.387},"min":{"1min":0.95899999999999996,"5min":0.94299999999999995,"15min":0.89500000000000002},"max":{"1min":1.925,"5min":2.282,"15min":3.754},"last":1.986}]}]},{"osd":1,"up_from":13,"seq":55834575031,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303124,"kb_used_data":1932,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068716,"statfs":{"total":96636764160,"available":96326365184,"internally_reserved":0,"allocated":1978368,"data_stored":1015553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.80500000000000005,"15min":0.81000000000000005},"min":{"1min":0.63,"5min":0.504,"15min":0.48799999999999999},"max":{"1min":1.22,"5min":1.671,"15min":1.671},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.85599999999999998,"15min":0.879},"min":{"1min":0.52700000000000002,"5min":0.502,"15min":0.46600000000000003},"max":{"1min":1.1799999999999999,"5min":1.3939999999999999,"15min":1.6899999999999999},"last":0.98299999999999998}]},{"osd":2,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97799999999999998,"15min":0.94099999999999995},"min":{"1min":0.755,"5min":0.60999999999999999,"15min":0.44600000000000001},"max":{"1min":1.306,"5min":1.806,"15min":1.806},"last":1.1970000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.89000000000000001,"15min":0.90500000000000003},"min":{"1min":0.63200000000000001,"5min":0.55800000000000005,"15min":0.49199999999999999},"max":{"1min":1.2509999999999999,"5min":1.3360000000000001,"15min":1.887},"last":1.1220000000000001}]},{"osd":3,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.129,"15min":1.125},"min":{"1min":0.747,"5min":0.66300000000000003,"15min":0.42399999999999999},"max":{"1min":1.476,"5min":1.6160000000000001,"15min":1.712},"last":1.3200000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0549999999999999,"15min":1.042},"min":{"1min":0.68500000000000005,"5min":0.52300000000000002,"15min":0.46000000000000002},"max":{"1min":1.4450000000000001,"5min":1.5409999999999999,"15min":1.7090000000000001},"last":1.2889999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.208,"15min":1.1930000000000001},"min":{"1min":0.83699999999999997,"5min":0.80600000000000005,"15min":0.65100000000000002},"max":{"1min":1.671,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.577},{"interface":"front","average":{"1min":1.194,"5min":1.1970000000000001,"15min":1.212},"min":{"1min":0.97399999999999998,"5min":0.83799999999999997,"15min":0.50800000000000001},"max":{"1min":1.4710000000000001,"5min":1.5429999999999999,"15min":2.206},"last":1.2430000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2649999999999999,"15min":1.2470000000000001},"min":{"1min":0.98499999999999999,"5min":0.90300000000000002,"15min":0.746},"max":{"1min":1.4430000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.27},{"interface":"front","average":{"1min":1.248,"5min":1.2509999999999999,"15min":1.256},"min":{"1min":1.038,"5min":0.92300000000000004,"15min":0.56100000000000005},"max":{"1min":1.4019999999999999,"5min":1.718,"15min":2.4609999999999999},"last":1.5029999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.327,"15min":1.327},"min":{"1min":1.1020000000000001,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.671,"5min":1.8640000000000001,"15min":1.948},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.429,"5min":1.431,"15min":1.4079999999999999},"min":{"1min":1.2230000000000001,"5min":0.91600000000000004,"15min":0.87},"max":{"1min":1.786,"5min":1.9670000000000001,"15min":2.3290000000000002},"last":1.3560000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3149999999999999,"15min":1.3169999999999999},"min":{"1min":1.1000000000000001,"5min":0.92000000000000004,"15min":0.84199999999999997},"max":{"1min":1.595,"5min":2.089,"15min":2.161},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.3999999999999999,"15min":1.3779999999999999},"min":{"1min":1.095,"5min":1.0249999999999999,"15min":0.78600000000000003},"max":{"1min":1.899,"5min":2.137,"15min":2.2349999999999999},"last":1.698}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T03:11:54.252 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T03:11:54.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.403+0000 7f6416979700 1 Processor -- start 2023-11-15T03:11:54.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.407+0000 7f6416979700 1 -- start start 2023-11-15T03:11:54.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.407+0000 7f6416979700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6408038040 0x7f64080e1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:54.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.407+0000 7f6416979700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6408054480 con 0x7f6408054010 2023-11-15T03:11:54.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.407+0000 7f6416979700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64080580c0 con 0x7f64080e1a20 2023-11-15T03:11:54.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.407+0000 7f6416979700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6408038b60 con 0x7f6408038040 2023-11-15T03:11:54.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.407+0000 7f6416178700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6408054010 0x7f64080065a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:59112/0 (socket says 172.21.15.121:59112) 2023-11-15T03:11:54.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.407+0000 7f6416178700 1 -- 172.21.15.121:0/3018582032 learned_addr learned my addr 172.21.15.121:0/3018582032 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:54.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.411+0000 7f6415176700 1 --2- 172.21.15.121:0/3018582032 >> v2:172.21.15.190:3300/0 conn(0x7f6408038040 0x7f64080e1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:54.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.411+0000 7f6414975700 1 -- 172.21.15.121:0/3018582032 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2506407054 0 0) 0x7f64080580c0 con 0x7f64080e1a20 2023-11-15T03:11:54.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.411+0000 7f6414975700 1 -- 172.21.15.121:0/3018582032 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63fc0033f0 con 0x7f64080e1a20 2023-11-15T03:11:54.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.411+0000 7f6414975700 1 -- 172.21.15.121:0/3018582032 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525698492 0 0) 0x7f6408054480 con 0x7f6408054010 2023-11-15T03:11:54.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.411+0000 7f6414975700 1 -- 172.21.15.121:0/3018582032 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64080580c0 con 0x7f6408054010 2023-11-15T03:11:54.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.411+0000 7f6415176700 1 -- 172.21.15.121:0/3018582032 >> v1:172.21.15.190:6789/0 conn(0x7f64080e1a20 legacy=0x7f6408037930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:54.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.411+0000 7f6414975700 1 -- 172.21.15.121:0/3018582032 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2578417113 0 0) 0x7f64080580c0 con 0x7f6408054010 2023-11-15T03:11:54.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.411+0000 7f6415176700 1 -- 172.21.15.121:0/3018582032 >> v1:172.21.15.121:6789/0 conn(0x7f6408054010 legacy=0x7f64080065a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:54.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.411+0000 7f6415176700 1 -- 172.21.15.121:0/3018582032 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64080583a0 con 0x7f6408038040 2023-11-15T03:11:54.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.411+0000 7f6415176700 1 --2- 172.21.15.121:0/3018582032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6408038040 0x7f64080e1e00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f64000096e0 tx=0x7f6400009060).ready entity=mon.1 client_cookie=2f128ad398670f38 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:54.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.411+0000 7f6414975700 1 -- 172.21.15.121:0/3018582032 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6410047640 con 0x7f6408038040 2023-11-15T03:11:54.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.411+0000 7f6414975700 1 -- 172.21.15.121:0/3018582032 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6400012410 con 0x7f6408038040 2023-11-15T03:11:54.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.411+0000 7f6414975700 1 -- 172.21.15.121:0/3018582032 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c0013f0 con 0x7f6408038040 2023-11-15T03:11:54.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.411+0000 7f6416979700 1 -- 172.21.15.121:0/3018582032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6408038040 msgr2=0x7f64080e1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:54.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.411+0000 7f6416979700 1 --2- 172.21.15.121:0/3018582032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6408038040 0x7f64080e1e00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f64000096e0 tx=0x7f6400009060).stop 2023-11-15T03:11:54.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.411+0000 7f6416979700 1 -- 172.21.15.121:0/3018582032 shutdown_connections 2023-11-15T03:11:54.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.411+0000 7f6416979700 1 --2- 172.21.15.121:0/3018582032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6408038040 0x7f64080e1e00 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.411+0000 7f6416979700 1 -- 172.21.15.121:0/3018582032 >> 172.21.15.121:0/3018582032 conn(0x7f6408055500 msgr2=0x7f64080558e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:54.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.415+0000 7f6416979700 1 -- 172.21.15.121:0/3018582032 shutdown_connections 2023-11-15T03:11:54.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.415+0000 7f6416979700 1 -- 172.21.15.121:0/3018582032 wait complete. 2023-11-15T03:11:54.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.415+0000 7f6416979700 1 Processor -- start 2023-11-15T03:11:54.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.419+0000 7f6416979700 1 -- start start 2023-11-15T03:11:54.428 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.419+0000 7f6416979700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6408038040 0x7f64080e0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:54.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.419+0000 7f6416979700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6408054010 0x7f6408039360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:54.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.419+0000 7f6416979700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64080e1a20 0x7f6408039850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:54.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.419+0000 7f6416979700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64080d5e20 con 0x7f6408038040 2023-11-15T03:11:54.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.419+0000 7f6416979700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64080f5cf0 con 0x7f6408054010 2023-11-15T03:11:54.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.419+0000 7f6416979700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64080f5fd0 con 0x7f64080e1a20 2023-11-15T03:11:54.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.419+0000 7f6415977700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6408038040 0x7f64080e0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:54.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.419+0000 7f6415977700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6408038040 0x7f64080e0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:56070/0 (socket says 172.21.15.121:56070) 2023-11-15T03:11:54.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.419+0000 7f6415977700 1 -- 172.21.15.121:0/2851570887 learned_addr learned my addr 172.21.15.121:0/2851570887 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:54.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.419+0000 7f6416178700 1 --2- 172.21.15.121:0/2851570887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64080e1a20 0x7f6408039850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:54.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.419+0000 7f6415176700 1 --2- 172.21.15.121:0/2851570887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6408054010 0x7f6408039360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:54.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.419+0000 7f6415977700 1 -- 172.21.15.121:0/2851570887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64080e1a20 msgr2=0x7f6408039850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:54.432 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.419+0000 7f6415977700 1 --2- 172.21.15.121:0/2851570887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64080e1a20 0x7f6408039850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.419+0000 7f6415977700 1 -- 172.21.15.121:0/2851570887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6408054010 msgr2=0x7f6408039360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:54.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.419+0000 7f6415977700 1 --2- 172.21.15.121:0/2851570887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6408054010 0x7f6408039360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.419+0000 7f6415977700 1 -- 172.21.15.121:0/2851570887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64000092a0 con 0x7f6408038040 2023-11-15T03:11:54.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.419+0000 7f6415977700 1 --2- 172.21.15.121:0/2851570887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6408038040 0x7f64080e0500 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 rx=0x7f640c00b590 tx=0x7f640c009120).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:54.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.423+0000 7f6405ffb700 1 -- 172.21.15.121:0/2851570887 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c003320 con 0x7f6408038040 2023-11-15T03:11:54.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.423+0000 7f6405ffb700 1 -- 172.21.15.121:0/2851570887 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640c0034c0 con 0x7f6408038040 2023-11-15T03:11:54.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.423+0000 7f6416979700 1 -- 172.21.15.121:0/2851570887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64080f62b0 con 0x7f6408038040 2023-11-15T03:11:54.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.423+0000 7f6405ffb700 1 -- 172.21.15.121:0/2851570887 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c001820 con 0x7f6408038040 2023-11-15T03:11:54.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.423+0000 7f6416979700 1 -- 172.21.15.121:0/2851570887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64080f6870 con 0x7f6408038040 2023-11-15T03:11:54.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.423+0000 7f6416979700 1 -- 172.21.15.121:0/2851570887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6408005720 con 0x7f6408038040 2023-11-15T03:11:54.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.423+0000 7f6405ffb700 1 -- 172.21.15.121:0/2851570887 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f640c001b90 con 0x7f6408038040 2023-11-15T03:11:54.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.427+0000 7f6405ffb700 1 --2- 172.21.15.121:0/2851570887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f63f405a510 0x7f63f405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:54.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.427+0000 7f6405ffb700 1 -- 172.21.15.121:0/2851570887 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f640c074290 con 0x7f6408038040 2023-11-15T03:11:54.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.427+0000 7f6415176700 1 --2- 172.21.15.121:0/2851570887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f63f405a510 0x7f63f405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:54.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.427+0000 7f6415176700 1 --2- 172.21.15.121:0/2851570887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f63f405a510 0x7f63f405c9d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f6408038440 tx=0x7f6400009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:54.443 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.435+0000 7f6405ffb700 1 -- 172.21.15.121:0/2851570887 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f640c047c50 con 0x7f6408038040 2023-11-15T03:11:54.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.595+0000 7f6416979700 1 -- 172.21.15.121:0/2851570887 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6408056b60 con 0x7f63f405a510 2023-11-15T03:11:54.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.599+0000 7f6405ffb700 1 -- 172.21.15.121:0/2851570887 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+68557 (secure 0 0 0) 0x7f6408056b60 con 0x7f63f405a510 2023-11-15T03:11:54.611 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:54.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.603+0000 7f6416979700 1 -- 172.21.15.121:0/2851570887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f63f405a510 msgr2=0x7f63f405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:54.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.603+0000 7f6416979700 1 --2- 172.21.15.121:0/2851570887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f63f405a510 0x7f63f405c9d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f6408038440 tx=0x7f6400009060).stop 2023-11-15T03:11:54.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.603+0000 7f6416979700 1 -- 172.21.15.121:0/2851570887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6408038040 msgr2=0x7f64080e0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:54.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.603+0000 7f6416979700 1 --2- 172.21.15.121:0/2851570887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6408038040 0x7f64080e0500 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 rx=0x7f640c00b590 tx=0x7f640c009120).stop 2023-11-15T03:11:54.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.603+0000 7f6416979700 1 -- 172.21.15.121:0/2851570887 shutdown_connections 2023-11-15T03:11:54.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.603+0000 7f6416979700 1 --2- 172.21.15.121:0/2851570887 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f63f405a510 0x7f63f405c9d0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.603+0000 7f6416979700 1 --2- 172.21.15.121:0/2851570887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64080e1a20 0x7f6408039850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.603+0000 7f6416979700 1 --2- 172.21.15.121:0/2851570887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6408054010 0x7f6408039360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.603+0000 7f6416979700 1 --2- 172.21.15.121:0/2851570887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6408038040 0x7f64080e0500 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.603+0000 7f6416979700 1 -- 172.21.15.121:0/2851570887 >> 172.21.15.121:0/2851570887 conn(0x7f6408055500 msgr2=0x7f64080558e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:54.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.603+0000 7f6416979700 1 -- 172.21.15.121:0/2851570887 shutdown_connections 2023-11-15T03:11:54.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.603+0000 7f6416979700 1 -- 172.21.15.121:0/2851570887 wait complete. 2023-11-15T03:11:54.617 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T03:11:54.633 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":599,"stamp":"2023-11-15T03:11:54.268642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":234,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2445260,"kb_used_data":15244,"kb_used_omap":0,"kb_used_meta":2429952,"kb_avail":752529460,"statfs":{"total":773094113280,"available":770590167040,"internally_reserved":0,"allocated":15609856,"data_stored":7920226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2488270848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-33941,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-60,"num_read_kb":-61,"num_write":-90,"num_write_kb":-62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.146079"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":127,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-15T03:11:40.337507+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T03:11:40.337507+0000","last_peered":"2023-11-15T03:11:40.337507+0000","last_clean":"2023-11-15T03:11:40.337507+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T03:11:40.337507+0000","last_undegraded":"2023-11-15T03:11:40.337507+0000","last_fullsized":"2023-11-15T03:11:40.337507+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":132,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-11-15T03:11:51.086537+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T03:11:51.086537+0000","last_peered":"2023-11-15T03:11:51.086537+0000","last_clean":"2023-11-15T03:11:51.086537+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T03:11:51.086537+0000","last_undegraded":"2023-11-15T03:11:51.086537+0000","last_fullsized":"2023-11-15T03:11:51.086537+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":125,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-15T03:11:40.337607+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T03:11:40.337607+0000","last_peered":"2023-11-15T03:11:40.337607+0000","last_clean":"2023-11-15T03:11:40.337607+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T03:11:40.337607+0000","last_undegraded":"2023-11-15T03:11:40.337607+0000","last_fullsized":"2023-11-15T03:11:40.337607+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":132,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-11-15T03:11:51.087803+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T03:11:51.087803+0000","last_peered":"2023-11-15T03:11:51.087803+0000","last_clean":"2023-11-15T03:11:51.087803+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T03:11:51.087803+0000","last_undegraded":"2023-11-15T03:11:51.087803+0000","last_fullsized":"2023-11-15T03:11:51.087803+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":143,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-11-15T03:11:50.419610+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T03:11:50.419610+0000","last_peered":"2023-11-15T03:11:50.419610+0000","last_clean":"2023-11-15T03:11:50.419610+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T03:11:50.419610+0000","last_undegraded":"2023-11-15T03:11:50.419610+0000","last_fullsized":"2023-11-15T03:11:50.419610+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":125,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-15T03:11:40.324057+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T03:11:40.324057+0000","last_peered":"2023-11-15T03:11:40.324057+0000","last_clean":"2023-11-15T03:11:40.324057+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T03:11:40.324057+0000","last_undegraded":"2023-11-15T03:11:40.324057+0000","last_fullsized":"2023-11-15T03:11:40.324057+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":125,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-15T03:11:40.338063+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T03:11:40.338063+0000","last_peered":"2023-11-15T03:11:40.338063+0000","last_clean":"2023-11-15T03:11:40.338063+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T03:11:40.338063+0000","last_undegraded":"2023-11-15T03:11:40.338063+0000","last_fullsized":"2023-11-15T03:11:40.338063+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":143,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-11-15T03:11:50.419362+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T03:11:50.419362+0000","last_peered":"2023-11-15T03:11:50.419362+0000","last_clean":"2023-11-15T03:11:50.419362+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T03:11:50.419362+0000","last_undegraded":"2023-11-15T03:11:50.419362+0000","last_fullsized":"2023-11-15T03:11:50.419362+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":126,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-15T03:11:40.338089+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T03:11:40.338089+0000","last_peered":"2023-11-15T03:11:40.338089+0000","last_clean":"2023-11-15T03:11:40.338089+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T03:11:40.338089+0000","last_undegraded":"2023-11-15T03:11:40.338089+0000","last_fullsized":"2023-11-15T03:11:40.338089+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542327,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307180,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064660,"statfs":{"total":96636764160,"available":96322211840,"internally_reserved":0,"allocated":1937408,"data_stored":982764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.0940000000000001,"15min":1.1319999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.55800000000000005},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":10.961},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.032,"5min":1.0880000000000001,"15min":1.1120000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.3089999999999999,"5min":1.6160000000000001,"15min":3.4910000000000001},"last":0.76300000000000001}]},{"osd":1,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.127,"15min":1.143},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.419,"5min":1.552,"15min":5.4930000000000003},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.153,"5min":1.165,"15min":1.1819999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6000000000000001,"5min":1.766,"15min":6.8239999999999998},"last":0.97799999999999998}]},{"osd":2,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0940000000000001,"15min":1.1359999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.452,"5min":1.452,"15min":5.7000000000000002},"last":1.032},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.226,"15min":1.2310000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":2.75},"last":1.28}]},{"osd":3,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.167,"15min":1.248},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":12.907},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2370000000000001,"15min":1.2709999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.573,"5min":1.573,"15min":9.9909999999999997},"last":1.3720000000000001}]},{"osd":4,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.105,"15min":1.1379999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6160000000000001,"5min":1.8939999999999999,"15min":2.5649999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.1240000000000001,"15min":1.2070000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.72,"5min":1.72,"15min":4.226},"last":1.1339999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.2450000000000001,"15min":1.2749999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":2.6890000000000001},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.143,"5min":1.236,"15min":1.292},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6899999999999999,"5min":1.7330000000000001,"15min":2.4550000000000001},"last":1.071}]},{"osd":6,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.325,"15min":1.381},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.5629999999999999,"5min":1.954,"15min":3.2789999999999999},"last":1.427},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.248,"15min":1.3340000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.63600000000000001},"max":{"1min":1.542,"5min":1.76,"15min":4.7800000000000002},"last":1.2989999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542327,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311288,"kb_used_data":1904,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94060552,"statfs":{"total":96636764160,"available":96318005248,"internally_reserved":0,"allocated":1949696,"data_stored":983787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.03,"15min":1.077},"min":{"1min":0.66200000000000003,"5min":0.63100000000000001,"15min":0.44500000000000001},"max":{"1min":1.4610000000000001,"5min":1.5760000000000001,"15min":4.3159999999999998},"last":0.753},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.109,"15min":1.0960000000000001},"min":{"1min":0.69999999999999996,"5min":0.59799999999999998,"15min":0.58599999999999997},"max":{"1min":1.486,"5min":1.659,"15min":2.2480000000000002},"last":1.4750000000000001}]},{"osd":1,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.111,"15min":1.1140000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.57599999999999996},"max":{"1min":1.6000000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1000000000000001,"15min":1.1379999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.60699999999999998},"max":{"1min":1.3360000000000001,"5min":1.8100000000000001,"15min":2.8570000000000002},"last":0.90100000000000002}]},{"osd":2,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1899999999999999,"15min":1.1919999999999999},"min":{"1min":0.83099999999999996,"5min":0.71699999999999997,"15min":0.49099999999999999},"max":{"1min":1.4730000000000001,"5min":1.7090000000000001,"15min":2.621},"last":1.155},{"interface":"front","average":{"1min":1.115,"5min":1.153,"15min":1.153},"min":{"1min":0.71699999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.4710000000000001,"5min":1.8360000000000001,"15min":2.0059999999999998},"last":1.2589999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.2310000000000001,"15min":1.214},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.50900000000000001},"max":{"1min":1.7709999999999999,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.234,"5min":1.2430000000000001,"15min":1.2609999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.67100000000000004},"max":{"1min":1.458,"5min":1.8640000000000001,"15min":2.8359999999999999},"last":0.98599999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.0580000000000001,"15min":1.1000000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5860000000000001,"5min":1.6839999999999999,"15min":4.5350000000000001},"last":1.093},{"interface":"front","average":{"1min":1.163,"5min":1.2170000000000001,"15min":1.2130000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.8580000000000001,"5min":2.004,"15min":2.0739999999999998},"last":1.081}]},{"osd":5,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3799999999999999,"15min":1.415},"min":{"1min":0.79800000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.744,"5min":2.1469999999999998,"15min":2.157},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2210000000000001,"15min":1.256},"min":{"1min":0.621,"5min":0.62,"15min":0.60899999999999999},"max":{"1min":1.8140000000000001,"5min":1.8779999999999999,"15min":2.9460000000000002},"last":0.97399999999999998}]},{"osd":7,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.216,"15min":1.2250000000000001},"min":{"1min":0.77400000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.9139999999999999,"5min":1.9239999999999999,"15min":3.1989999999999998},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.407,"15min":1.419},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.65200000000000002},"max":{"1min":1.8220000000000001,"5min":2.0649999999999999,"15min":2.9350000000000001},"last":1.323}]}]},{"osd":4,"up_from":14,"seq":60129542328,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307212,"kb_used_data":1924,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064628,"statfs":{"total":96636764160,"available":96322179072,"internally_reserved":0,"allocated":1970176,"data_stored":1014901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.0569999999999999,"15min":1.083},"min":{"1min":0.73799999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4810000000000001,"5min":1.5660000000000001,"15min":1.5840000000000001},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.044,"15min":1.056},"min":{"1min":0.85399999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.677},"last":1.319}]},{"osd":1,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.135,"15min":1.1319999999999999},"min":{"1min":0.70599999999999996,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":2.617},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.1419999999999999,"15min":1.1539999999999999},"min":{"1min":0.82099999999999995,"5min":0.629,"15min":0.54800000000000004},"max":{"1min":1.573,"5min":1.6319999999999999,"15min":1.6619999999999999},"last":1.2809999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.165,"15min":1.1830000000000001},"min":{"1min":0.81399999999999995,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.5029999999999999,"5min":1.823,"15min":1.9450000000000001},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.155,"15min":1.1619999999999999},"min":{"1min":0.84199999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9330000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.044}]},{"osd":3,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1399999999999999,"15min":1.1459999999999999},"min":{"1min":0.85699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.5229999999999999,"5min":1.5940000000000001,"15min":1.8660000000000001},"last":1.107},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2549999999999999,"15min":1.264},"min":{"1min":0.93000000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.611,"5min":1.8200000000000001,"15min":2.5630000000000002},"last":1.0029999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1479999999999999,"15min":1.1830000000000001},"min":{"1min":0.91900000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.554,"5min":1.643,"15min":2.0840000000000001},"last":1.387},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.1479999999999999,"15min":1.1879999999999999},"min":{"1min":0.89400000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":2.3570000000000002},"last":1.125}]},{"osd":6,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2410000000000001,"15min":1.264},"min":{"1min":0.97999999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5880000000000001,"5min":1.913,"15min":2.6640000000000001},"last":1.264},{"interface":"front","average":{"1min":1.325,"5min":1.288,"15min":1.323},"min":{"1min":0.88,"5min":0.67600000000000005,"15min":0.57299999999999995},"max":{"1min":1.585,"5min":1.9139999999999999,"15min":2.173},"last":1.5680000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2769999999999999,"15min":1.3069999999999999},"min":{"1min":0.89500000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.7629999999999999,"5min":1.9199999999999999,"15min":2.6030000000000002},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.3700000000000001,"15min":1.3879999999999999},"min":{"1min":1.044,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8240000000000001,"5min":2.069,"15min":2.7669999999999999},"last":1.306}]}]},{"osd":3,"up_from":14,"seq":60129542328,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303084,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068756,"statfs":{"total":96636764160,"available":96326406144,"internally_reserved":0,"allocated":1937408,"data_stored":983121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85999999999999999,"15min":0.83699999999999997},"min":{"1min":0.54200000000000004,"5min":0.54000000000000004,"15min":0.45400000000000001},"max":{"1min":1.327,"5min":1.4219999999999999,"15min":1.528},"last":0.746},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.89900000000000002,"15min":0.88800000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.39800000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.79700000000000004}]},{"osd":1,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.83899999999999997,"15min":0.872},"min":{"1min":0.62,"5min":0.53100000000000003,"15min":0.435},"max":{"1min":1.165,"5min":1.25,"15min":1.4670000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.996,"5min":0.98599999999999999,"15min":0.99199999999999999},"min":{"1min":0.72199999999999998,"5min":0.63700000000000001,"15min":0.55800000000000005},"max":{"1min":1.446,"5min":1.446,"15min":1.544},"last":1.016}]},{"osd":2,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.026,"15min":1.0209999999999999},"min":{"1min":0.66400000000000003,"5min":0.64200000000000002,"15min":0.54600000000000004},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0529999999999999,"15min":1.0660000000000001},"min":{"1min":0.66000000000000003,"5min":0.63100000000000001,"15min":0.50600000000000001},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.70099999999999996}]},{"osd":4,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2470000000000001,"15min":1.228},"min":{"1min":1.0029999999999999,"5min":0.88600000000000001,"15min":0.71399999999999997},"max":{"1min":1.5349999999999999,"5min":1.554,"15min":1.9139999999999999},"last":1.002},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2170000000000001,"15min":1.2250000000000001},"min":{"1min":1.01,"5min":0.78500000000000003,"15min":0.621},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.726},"last":0.95499999999999996}]},{"osd":5,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.252,"15min":1.266},"min":{"1min":0.91700000000000004,"5min":0.90200000000000002,"15min":0.64300000000000002},"max":{"1min":1.5900000000000001,"5min":1.7290000000000001,"15min":2.0459999999999998},"last":1.085},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.27,"15min":1.272},"min":{"1min":0.95899999999999996,"5min":0.94799999999999995,"15min":0.63500000000000001},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.927},"last":0.94499999999999995}]},{"osd":6,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.345,"15min":1.339},"min":{"1min":0.91400000000000003,"5min":0.875,"15min":0.72899999999999998},"max":{"1min":1.575,"5min":1.8129999999999999,"15min":1.9970000000000001},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3560000000000001,"15min":1.3720000000000001},"min":{"1min":1.0980000000000001,"5min":0.871,"15min":0.77700000000000002},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":2.3570000000000002},"last":1.4750000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.331,"15min":1.3440000000000001},"min":{"1min":1.0149999999999999,"5min":0.92500000000000004,"15min":0.68100000000000005},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":2.0470000000000002},"last":1.26},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.345,"15min":1.355},"min":{"1min":1.113,"5min":0.97299999999999998,"15min":0.66600000000000004},"max":{"1min":1.8500000000000001,"5min":1.8779999999999999,"15min":2.1040000000000001},"last":1.0449999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542328,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303084,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068756,"statfs":{"total":96636764160,"available":96326406144,"internally_reserved":0,"allocated":1937408,"data_stored":981525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.65400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97099999999999997,"5min":1.2350000000000001,"15min":1.409},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.84299999999999997,"15min":0.84399999999999997},"min":{"1min":0.58799999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2689999999999999,"5min":1.4950000000000001,"15min":1.619},"last":0.58499999999999996}]},{"osd":1,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.90000000000000002,"15min":0.91100000000000003},"min":{"1min":0.74199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.95599999999999996,"15min":0.95399999999999996},"min":{"1min":0.67200000000000004,"5min":0.55100000000000005,"15min":0.54600000000000004},"max":{"1min":1.3580000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":1.01}]},{"osd":3,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95599999999999996,"15min":0.97199999999999998},"min":{"1min":0.60699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.385,"5min":1.4359999999999999,"15min":1.6259999999999999},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.125,"15min":1.125},"min":{"1min":0.67700000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.488,"5min":1.5549999999999999,"15min":1.5800000000000001},"last":1.492}]},{"osd":4,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.218,"15min":1.2070000000000001},"min":{"1min":1.052,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.581,"5min":1.583,"15min":1.9410000000000001},"last":1.595},{"interface":"front","average":{"1min":1.3,"5min":1.21,"15min":1.212},"min":{"1min":0.99199999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6579999999999999,"5min":1.7170000000000001,"15min":1.7829999999999999},"last":1.1699999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1619999999999999,"15min":1.1830000000000001},"min":{"1min":0.98499999999999999,"5min":0.68100000000000005,"15min":0.67000000000000004},"max":{"1min":1.47,"5min":1.625,"15min":1.8939999999999999},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.266,"15min":1.272},"min":{"1min":1.0660000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.556,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.323}]},{"osd":6,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.262,"15min":1.2729999999999999},"min":{"1min":0.93999999999999995,"5min":0.62,"15min":0.62},"max":{"1min":1.6990000000000001,"5min":2,"15min":2.137},"last":1.294},{"interface":"front","average":{"1min":1.292,"5min":1.2669999999999999,"15min":1.2689999999999999},"min":{"1min":1.0880000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.538,"5min":1.9350000000000001,"15min":2.0870000000000002},"last":1.2549999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.29,"15min":1.3109999999999999},"min":{"1min":0.95699999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7789999999999999,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.298,"15min":1.2989999999999999},"min":{"1min":1.101,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.7270000000000001,"5min":1.881,"15min":2.0350000000000001},"last":1.6220000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542328,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307188,"kb_used_data":1900,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064652,"statfs":{"total":96636764160,"available":96322203648,"internally_reserved":0,"allocated":1945600,"data_stored":983408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80600000000000005,"15min":0.83299999999999996},"min":{"1min":0.503,"5min":0.44400000000000001,"15min":0.36199999999999999},"max":{"1min":1.093,"5min":1.756,"15min":4.3630000000000004},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.90600000000000003,"15min":0.93700000000000006},"min":{"1min":0.61799999999999999,"5min":0.52600000000000002,"15min":0.40000000000000002},"max":{"1min":1.2490000000000001,"5min":1.6719999999999999,"15min":6.157},"last":0.96799999999999997}]},{"osd":2,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.96399999999999997,"15min":0.97899999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.46800000000000003},"max":{"1min":1.3400000000000001,"5min":1.625,"15min":4.1699999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0029999999999999,"15min":1.0289999999999999},"min":{"1min":0.72199999999999998,"5min":0.60099999999999998,"15min":0.47799999999999998},"max":{"1min":1.379,"5min":1.879,"15min":6.8369999999999997},"last":0.874}]},{"osd":3,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.94099999999999995,"15min":0.95599999999999996},"min":{"1min":0.59499999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.3120000000000001,"5min":1.6519999999999999,"15min":3.5070000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.073,"5min":1.093,"15min":1.113},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56200000000000006},"max":{"1min":1.3460000000000001,"5min":2.0070000000000001,"15min":5.2149999999999999},"last":0.84799999999999998}]},{"osd":4,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.224,"15min":1.2609999999999999},"min":{"1min":0.81499999999999995,"5min":0.80500000000000005,"15min":0.66700000000000004},"max":{"1min":1.5900000000000001,"5min":1.944,"15min":6.2000000000000002},"last":1.157},{"interface":"front","average":{"1min":1.165,"5min":1.1870000000000001,"15min":1.196},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.3939999999999999,"5min":1.6779999999999999,"15min":3.577},"last":1.2490000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.23,"15min":1.25},"min":{"1min":0.93000000000000005,"5min":0.81499999999999995,"15min":0.69599999999999995},"max":{"1min":1.478,"5min":1.857,"15min":3.548},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.21,"5min":1.278,"15min":1.2969999999999999},"min":{"1min":0.86199999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.4330000000000001,"5min":1.774,"15min":4.4080000000000004},"last":1.1419999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3180000000000001,"15min":1.349},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.82299999999999995},"max":{"1min":1.5640000000000001,"5min":1.915,"15min":4.4489999999999998},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.2889999999999999,"15min":1.3340000000000001},"min":{"1min":0.93300000000000005,"5min":0.86499999999999999,"15min":0.75900000000000001},"max":{"1min":1.5409999999999999,"5min":2.056,"15min":6.9690000000000003},"last":1.274}]},{"osd":7,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.327,"15min":1.401},"min":{"1min":1.036,"5min":0.90400000000000003,"15min":0.79900000000000004},"max":{"1min":1.6319999999999999,"5min":1.994,"15min":18.253},"last":1.361},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2889999999999999,"15min":1.3380000000000001},"min":{"1min":0.997,"5min":0.95599999999999996,"15min":0.79000000000000004},"max":{"1min":1.452,"5min":1.8720000000000001,"15min":11.006},"last":1.1779999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575033,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303112,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068728,"statfs":{"total":96636764160,"available":96326377472,"internally_reserved":0,"allocated":1966080,"data_stored":995360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0800000000000001,"15min":1.073},"min":{"1min":0.65800000000000003,"5min":0.627,"15min":0.49399999999999999},"max":{"1min":1.827,"5min":1.827,"15min":2.3370000000000002},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0589999999999999},"min":{"1min":0.63600000000000001,"5min":0.61099999999999999,"15min":0.60799999999999998},"max":{"1min":1.3580000000000001,"5min":1.6519999999999999,"15min":3.46},"last":0.97699999999999998}]},{"osd":1,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.105,"15min":1.121},"min":{"1min":0.84599999999999997,"5min":0.71199999999999997,"15min":0.58999999999999997},"max":{"1min":1.4570000000000001,"5min":1.516,"15min":3.6349999999999998},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.161,"15min":1.143},"min":{"1min":0.84199999999999997,"5min":0.73999999999999999,"15min":0.63700000000000001},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":3.516},"last":1.1040000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.1459999999999999,"15min":1.165},"min":{"1min":0.88600000000000001,"5min":0.74099999999999999,"15min":0.73199999999999998},"max":{"1min":1.774,"5min":1.774,"15min":5.4180000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1899999999999999,"15min":1.204},"min":{"1min":0.77300000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6839999999999999,"5min":2.0840000000000001,"15min":3.5859999999999999},"last":1.373}]},{"osd":3,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2070000000000001,"15min":1.196},"min":{"1min":0.91400000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6439999999999999,"5min":1.8560000000000001,"15min":2.141},"last":1.04},{"interface":"front","average":{"1min":1.294,"5min":1.296,"15min":1.3169999999999999},"min":{"1min":0.88400000000000001,"5min":0.875,"15min":0.78600000000000003},"max":{"1min":1.6140000000000001,"5min":1.6739999999999999,"15min":4.2729999999999997},"last":1.2829999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.2130000000000001,"15min":1.26},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.4850000000000001,"5min":1.591,"15min":3.706},"last":1.492},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.258,"15min":1.3080000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.64100000000000001},"max":{"1min":1.863,"5min":1.9219999999999999,"15min":6.1180000000000003},"last":1.4330000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.268,"15min":1.254},"min":{"1min":0.88800000000000001,"5min":0.80700000000000005,"15min":0.749},"max":{"1min":1.6080000000000001,"5min":1.843,"15min":4.2309999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.2869999999999999,"15min":1.341},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.546,"5min":2.2240000000000002,"15min":3.641},"last":1.212}]},{"osd":7,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.284,"15min":1.2949999999999999},"min":{"1min":0.83499999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6970000000000001,"5min":1.988,"15min":3.516},"last":1.23},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.379,"15min":1.387},"min":{"1min":0.95899999999999996,"5min":0.94299999999999995,"15min":0.89500000000000002},"max":{"1min":1.925,"5min":2.282,"15min":3.754},"last":1.361}]}]},{"osd":1,"up_from":13,"seq":55834575032,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303112,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068728,"statfs":{"total":96636764160,"available":96326377472,"internally_reserved":0,"allocated":1966080,"data_stored":995360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.80500000000000005,"15min":0.81000000000000005},"min":{"1min":0.63,"5min":0.504,"15min":0.48799999999999999},"max":{"1min":1.22,"5min":1.671,"15min":1.671},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.85599999999999998,"15min":0.879},"min":{"1min":0.52700000000000002,"5min":0.502,"15min":0.46600000000000003},"max":{"1min":1.1799999999999999,"5min":1.3939999999999999,"15min":1.6899999999999999},"last":1.0009999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97799999999999998,"15min":0.94099999999999995},"min":{"1min":0.755,"5min":0.60999999999999999,"15min":0.44600000000000001},"max":{"1min":1.306,"5min":1.806,"15min":1.806},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.872,"5min":0.89000000000000001,"15min":0.90500000000000003},"min":{"1min":0.63200000000000001,"5min":0.55800000000000005,"15min":0.49199999999999999},"max":{"1min":1.2509999999999999,"5min":1.3360000000000001,"15min":1.887},"last":1.0429999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.129,"15min":1.125},"min":{"1min":0.747,"5min":0.66300000000000003,"15min":0.42399999999999999},"max":{"1min":1.476,"5min":1.6160000000000001,"15min":1.712},"last":1.131},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0549999999999999,"15min":1.042},"min":{"1min":0.68500000000000005,"5min":0.52300000000000002,"15min":0.46000000000000002},"max":{"1min":1.4450000000000001,"5min":1.5409999999999999,"15min":1.7090000000000001},"last":1.109}]},{"osd":4,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.208,"15min":1.1930000000000001},"min":{"1min":0.83699999999999997,"5min":0.80600000000000005,"15min":0.65100000000000002},"max":{"1min":1.671,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.1970000000000001,"15min":1.212},"min":{"1min":0.97399999999999998,"5min":0.83799999999999997,"15min":0.50800000000000001},"max":{"1min":1.4710000000000001,"5min":1.5429999999999999,"15min":2.206},"last":1.1850000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2649999999999999,"15min":1.2470000000000001},"min":{"1min":0.98499999999999999,"5min":0.90300000000000002,"15min":0.746},"max":{"1min":1.4430000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.2509999999999999,"15min":1.256},"min":{"1min":1.038,"5min":0.92300000000000004,"15min":0.56100000000000005},"max":{"1min":1.4019999999999999,"5min":1.718,"15min":2.4609999999999999},"last":1.238}]},{"osd":6,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.327,"15min":1.327},"min":{"1min":1.1020000000000001,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.671,"5min":1.8640000000000001,"15min":1.948},"last":1.395},{"interface":"front","average":{"1min":1.429,"5min":1.431,"15min":1.4079999999999999},"min":{"1min":1.2230000000000001,"5min":0.91600000000000004,"15min":0.87},"max":{"1min":1.786,"5min":1.9670000000000001,"15min":2.3290000000000002},"last":1.486}]},{"osd":7,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3149999999999999,"15min":1.3169999999999999},"min":{"1min":1.1000000000000001,"5min":0.92000000000000004,"15min":0.84199999999999997},"max":{"1min":1.595,"5min":2.089,"15min":2.161},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.3999999999999999,"15min":1.3779999999999999},"min":{"1min":1.095,"5min":1.0249999999999999,"15min":0.78600000000000003},"max":{"1min":1.899,"5min":2.137,"15min":2.2349999999999999},"last":1.4119999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T03:11:54.635 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-15T03:11:54.636 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T03:11:54.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.779+0000 7fd91945b700 1 Processor -- start 2023-11-15T03:11:54.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.779+0000 7fd91945b700 1 -- start start 2023-11-15T03:11:54.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.779+0000 7fd91945b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd914151d00 0x7fd9140c4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:54.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.779+0000 7fd91945b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9140c4c10 0x7fd9140a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:54.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.779+0000 7fd91945b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd914068430 con 0x7fd914151920 2023-11-15T03:11:54.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.779+0000 7fd91945b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9140c50a0 con 0x7fd914151d00 2023-11-15T03:11:54.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.779+0000 7fd91945b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd914064b50 con 0x7fd9140c4c10 2023-11-15T03:11:54.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.779+0000 7fd90bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd914151d00 0x7fd9140c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:54.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.779+0000 7fd90bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd914151d00 0x7fd9140c4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54112/0 (socket says 172.21.15.121:54112) 2023-11-15T03:11:54.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.779+0000 7fd90bfff700 1 -- 172.21.15.121:0/3862252334 learned_addr learned my addr 172.21.15.121:0/3862252334 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:54.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.779+0000 7fd913fff700 1 --2- 172.21.15.121:0/3862252334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9140c4c10 0x7fd9140a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:54.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.783+0000 7fd90bfff700 1 -- 172.21.15.121:0/3862252334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9140c4c10 msgr2=0x7fd9140a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:54.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.783+0000 7fd90bfff700 1 --2- 172.21.15.121:0/3862252334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9140c4c10 0x7fd9140a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.783+0000 7fd90bfff700 1 -- 172.21.15.121:0/3862252334 >> v1:172.21.15.190:6789/0 conn(0x7fd914151920 legacy=0x7fd914063fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:54.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.783+0000 7fd90bfff700 1 -- 172.21.15.121:0/3862252334 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9140bec30 con 0x7fd914151d00 2023-11-15T03:11:54.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.783+0000 7fd90bfff700 1 --2- 172.21.15.121:0/3862252334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd914151d00 0x7fd9140c4720 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 rx=0x7fd8fc006fd0 tx=0x7fd8fc0090f0).ready entity=mon.0 client_cookie=a33a3aef7b004c5a server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:54.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.783+0000 7fd9127fc700 1 -- 172.21.15.121:0/3862252334 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc00b580 con 0x7fd914151d00 2023-11-15T03:11:54.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.783+0000 7fd9127fc700 1 -- 172.21.15.121:0/3862252334 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8fc00b6e0 con 0x7fd914151d00 2023-11-15T03:11:54.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.783+0000 7fd9127fc700 1 -- 172.21.15.121:0/3862252334 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc00ba60 con 0x7fd914151d00 2023-11-15T03:11:54.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.783+0000 7fd91945b700 1 -- 172.21.15.121:0/3862252334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd914151d00 msgr2=0x7fd9140c4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:54.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.783+0000 7fd91945b700 1 --2- 172.21.15.121:0/3862252334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd914151d00 0x7fd9140c4720 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 rx=0x7fd8fc006fd0 tx=0x7fd8fc0090f0).stop 2023-11-15T03:11:54.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.783+0000 7fd91945b700 1 -- 172.21.15.121:0/3862252334 shutdown_connections 2023-11-15T03:11:54.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.783+0000 7fd91945b700 1 --2- 172.21.15.121:0/3862252334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9140c4c10 0x7fd9140a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.783+0000 7fd91945b700 1 --2- 172.21.15.121:0/3862252334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd914151d00 0x7fd9140c4720 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.783+0000 7fd91945b700 1 -- 172.21.15.121:0/3862252334 >> 172.21.15.121:0/3862252334 conn(0x7fd914069ac0 msgr2=0x7fd914090a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:54.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.783+0000 7fd91945b700 1 -- 172.21.15.121:0/3862252334 shutdown_connections 2023-11-15T03:11:54.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.783+0000 7fd91945b700 1 -- 172.21.15.121:0/3862252334 wait complete. 2023-11-15T03:11:54.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd91945b700 1 Processor -- start 2023-11-15T03:11:54.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd91945b700 1 -- start start 2023-11-15T03:11:54.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd91945b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9140c4c10 0x7fd914167460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:54.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd91945b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd914151920 0x7fd914167950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:54.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd91945b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd914151d00 0x7fd91413bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:54.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd91945b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd914167e40 con 0x7fd914151d00 2023-11-15T03:11:54.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd91945b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140bfb80 con 0x7fd914151920 2023-11-15T03:11:54.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd91945b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140c3da0 con 0x7fd9140c4c10 2023-11-15T03:11:54.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd913fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd914151d00 0x7fd91413bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:54.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd913fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd914151d00 0x7fd91413bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54118/0 (socket says 172.21.15.121:54118) 2023-11-15T03:11:54.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd913fff700 1 -- 172.21.15.121:0/2258130830 learned_addr learned my addr 172.21.15.121:0/2258130830 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:54.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd912ffd700 1 --2- 172.21.15.121:0/2258130830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9140c4c10 0x7fd914167460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:54.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd90bfff700 1 --2- 172.21.15.121:0/2258130830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd914151920 0x7fd914167950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:54.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd913fff700 1 -- 172.21.15.121:0/2258130830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9140c4c10 msgr2=0x7fd914167460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:54.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd913fff700 1 --2- 172.21.15.121:0/2258130830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9140c4c10 0x7fd914167460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd913fff700 1 -- 172.21.15.121:0/2258130830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd914151920 msgr2=0x7fd914167950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:54.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd913fff700 1 --2- 172.21.15.121:0/2258130830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd914151920 0x7fd914167950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd913fff700 1 -- 172.21.15.121:0/2258130830 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc009370 con 0x7fd914151d00 2023-11-15T03:11:54.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd90bfff700 1 --2- 172.21.15.121:0/2258130830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd914151920 0x7fd914167950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:54.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd913fff700 1 --2- 172.21.15.121:0/2258130830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd914151d00 0x7fd91413bed0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 rx=0x7fd90c009e90 tx=0x7fd90c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:54.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd90affd700 1 -- 172.21.15.121:0/2258130830 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00cb50 con 0x7fd914151d00 2023-11-15T03:11:54.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd90affd700 1 -- 172.21.15.121:0/2258130830 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c00f040 con 0x7fd914151d00 2023-11-15T03:11:54.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.787+0000 7fd91945b700 1 -- 172.21.15.121:0/2258130830 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9140bf530 con 0x7fd914151d00 2023-11-15T03:11:54.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.791+0000 7fd90affd700 1 -- 172.21.15.121:0/2258130830 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00cec0 con 0x7fd914151d00 2023-11-15T03:11:54.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.791+0000 7fd91945b700 1 -- 172.21.15.121:0/2258130830 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91413c3c0 con 0x7fd914151d00 2023-11-15T03:11:54.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.791+0000 7fd91945b700 1 -- 172.21.15.121:0/2258130830 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd914047060 con 0x7fd914151d00 2023-11-15T03:11:54.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.791+0000 7fd90affd700 1 -- 172.21.15.121:0/2258130830 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd90c013610 con 0x7fd914151d00 2023-11-15T03:11:54.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.795+0000 7fd90affd700 1 --2- 172.21.15.121:0/2258130830 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd8f405a510 0x7fd8f405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:54.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.795+0000 7fd90affd700 1 -- 172.21.15.121:0/2258130830 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7fd90c074f40 con 0x7fd914151d00 2023-11-15T03:11:54.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.795+0000 7fd912ffd700 1 --2- 172.21.15.121:0/2258130830 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd8f405a510 0x7fd8f405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:54.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.795+0000 7fd912ffd700 1 --2- 172.21.15.121:0/2258130830 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd8f405a510 0x7fd8f405c9d0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7fd904003870 tx=0x7fd904007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:54.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.799+0000 7fd90affd700 1 -- 172.21.15.121:0/2258130830 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd90c048900 con 0x7fd914151d00 2023-11-15T03:11:54.969 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.959+0000 7fd91945b700 1 -- 172.21.15.121:0/2258130830 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd914067b00 con 0x7fd8f405a510 2023-11-15T03:11:54.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.967+0000 7fd90affd700 1 -- 172.21.15.121:0/2258130830 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+68557 (secure 0 0 0) 0x7fd914067b00 con 0x7fd8f405a510 2023-11-15T03:11:54.977 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:54.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.967+0000 7fd91945b700 1 -- 172.21.15.121:0/2258130830 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd8f405a510 msgr2=0x7fd8f405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:54.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.967+0000 7fd91945b700 1 --2- 172.21.15.121:0/2258130830 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd8f405a510 0x7fd8f405c9d0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7fd904003870 tx=0x7fd904007d40).stop 2023-11-15T03:11:54.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.967+0000 7fd91945b700 1 -- 172.21.15.121:0/2258130830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd914151d00 msgr2=0x7fd91413bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:54.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.967+0000 7fd91945b700 1 --2- 172.21.15.121:0/2258130830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd914151d00 0x7fd91413bed0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 rx=0x7fd90c009e90 tx=0x7fd90c00b040).stop 2023-11-15T03:11:54.979 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.967+0000 7fd91945b700 1 -- 172.21.15.121:0/2258130830 shutdown_connections 2023-11-15T03:11:54.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.967+0000 7fd91945b700 1 --2- 172.21.15.121:0/2258130830 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fd8f405a510 0x7fd8f405c9d0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.967+0000 7fd91945b700 1 --2- 172.21.15.121:0/2258130830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd914151d00 0x7fd91413bed0 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.967+0000 7fd91945b700 1 --2- 172.21.15.121:0/2258130830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd914151920 0x7fd914167950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.967+0000 7fd91945b700 1 --2- 172.21.15.121:0/2258130830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9140c4c10 0x7fd914167460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:54.980 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.967+0000 7fd91945b700 1 -- 172.21.15.121:0/2258130830 >> 172.21.15.121:0/2258130830 conn(0x7fd914069ac0 msgr2=0x7fd914150910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:54.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.971+0000 7fd91945b700 1 -- 172.21.15.121:0/2258130830 shutdown_connections 2023-11-15T03:11:54.981 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:54.971+0000 7fd91945b700 1 -- 172.21.15.121:0/2258130830 wait complete. 2023-11-15T03:11:54.982 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T03:11:54.999 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":599,"stamp":"2023-11-15T03:11:54.268642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":234,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2445260,"kb_used_data":15244,"kb_used_omap":0,"kb_used_meta":2429952,"kb_avail":752529460,"statfs":{"total":773094113280,"available":770590167040,"internally_reserved":0,"allocated":15609856,"data_stored":7920226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2488270848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-33941,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-60,"num_read_kb":-61,"num_write":-90,"num_write_kb":-62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.146079"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":127,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-15T03:11:40.337507+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T03:11:40.337507+0000","last_peered":"2023-11-15T03:11:40.337507+0000","last_clean":"2023-11-15T03:11:40.337507+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T03:11:40.337507+0000","last_undegraded":"2023-11-15T03:11:40.337507+0000","last_fullsized":"2023-11-15T03:11:40.337507+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":132,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-11-15T03:11:51.086537+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T03:11:51.086537+0000","last_peered":"2023-11-15T03:11:51.086537+0000","last_clean":"2023-11-15T03:11:51.086537+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T03:11:51.086537+0000","last_undegraded":"2023-11-15T03:11:51.086537+0000","last_fullsized":"2023-11-15T03:11:51.086537+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":125,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-15T03:11:40.337607+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T03:11:40.337607+0000","last_peered":"2023-11-15T03:11:40.337607+0000","last_clean":"2023-11-15T03:11:40.337607+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T03:11:40.337607+0000","last_undegraded":"2023-11-15T03:11:40.337607+0000","last_fullsized":"2023-11-15T03:11:40.337607+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":132,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-11-15T03:11:51.087803+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T03:11:51.087803+0000","last_peered":"2023-11-15T03:11:51.087803+0000","last_clean":"2023-11-15T03:11:51.087803+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T03:11:51.087803+0000","last_undegraded":"2023-11-15T03:11:51.087803+0000","last_fullsized":"2023-11-15T03:11:51.087803+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":143,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-11-15T03:11:50.419610+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T03:11:50.419610+0000","last_peered":"2023-11-15T03:11:50.419610+0000","last_clean":"2023-11-15T03:11:50.419610+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T03:11:50.419610+0000","last_undegraded":"2023-11-15T03:11:50.419610+0000","last_fullsized":"2023-11-15T03:11:50.419610+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":125,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-15T03:11:40.324057+0000","last_change":"2023-11-15T02:56:46.383382+0000","last_active":"2023-11-15T03:11:40.324057+0000","last_peered":"2023-11-15T03:11:40.324057+0000","last_clean":"2023-11-15T03:11:40.324057+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T03:11:40.324057+0000","last_undegraded":"2023-11-15T03:11:40.324057+0000","last_fullsized":"2023-11-15T03:11:40.324057+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":125,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-15T03:11:40.338063+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T03:11:40.338063+0000","last_peered":"2023-11-15T03:11:40.338063+0000","last_clean":"2023-11-15T03:11:40.338063+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T03:11:40.338063+0000","last_undegraded":"2023-11-15T03:11:40.338063+0000","last_fullsized":"2023-11-15T03:11:40.338063+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":143,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-11-15T03:11:50.419362+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T03:11:50.419362+0000","last_peered":"2023-11-15T03:11:50.419362+0000","last_clean":"2023-11-15T03:11:50.419362+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T03:11:50.419362+0000","last_undegraded":"2023-11-15T03:11:50.419362+0000","last_fullsized":"2023-11-15T03:11:50.419362+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":126,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-15T03:11:40.338089+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T03:11:40.338089+0000","last_peered":"2023-11-15T03:11:40.338089+0000","last_clean":"2023-11-15T03:11:40.338089+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T03:11:40.338089+0000","last_undegraded":"2023-11-15T03:11:40.338089+0000","last_fullsized":"2023-11-15T03:11:40.338089+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542327,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307180,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064660,"statfs":{"total":96636764160,"available":96322211840,"internally_reserved":0,"allocated":1937408,"data_stored":982764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.0940000000000001,"15min":1.1319999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.55800000000000005},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":10.961},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.032,"5min":1.0880000000000001,"15min":1.1120000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.3089999999999999,"5min":1.6160000000000001,"15min":3.4910000000000001},"last":0.76300000000000001}]},{"osd":1,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.127,"15min":1.143},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.419,"5min":1.552,"15min":5.4930000000000003},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.153,"5min":1.165,"15min":1.1819999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6000000000000001,"5min":1.766,"15min":6.8239999999999998},"last":0.97799999999999998}]},{"osd":2,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0940000000000001,"15min":1.1359999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.452,"5min":1.452,"15min":5.7000000000000002},"last":1.032},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.226,"15min":1.2310000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":2.75},"last":1.28}]},{"osd":3,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.167,"15min":1.248},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":12.907},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2370000000000001,"15min":1.2709999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.573,"5min":1.573,"15min":9.9909999999999997},"last":1.3720000000000001}]},{"osd":4,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.105,"15min":1.1379999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6160000000000001,"5min":1.8939999999999999,"15min":2.5649999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.1240000000000001,"15min":1.2070000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.72,"5min":1.72,"15min":4.226},"last":1.1339999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.2450000000000001,"15min":1.2749999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":2.6890000000000001},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.143,"5min":1.236,"15min":1.292},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6899999999999999,"5min":1.7330000000000001,"15min":2.4550000000000001},"last":1.071}]},{"osd":6,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.325,"15min":1.381},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.5629999999999999,"5min":1.954,"15min":3.2789999999999999},"last":1.427},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.248,"15min":1.3340000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.63600000000000001},"max":{"1min":1.542,"5min":1.76,"15min":4.7800000000000002},"last":1.2989999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542327,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311288,"kb_used_data":1904,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94060552,"statfs":{"total":96636764160,"available":96318005248,"internally_reserved":0,"allocated":1949696,"data_stored":983787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.03,"15min":1.077},"min":{"1min":0.66200000000000003,"5min":0.63100000000000001,"15min":0.44500000000000001},"max":{"1min":1.4610000000000001,"5min":1.5760000000000001,"15min":4.3159999999999998},"last":0.753},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.109,"15min":1.0960000000000001},"min":{"1min":0.69999999999999996,"5min":0.59799999999999998,"15min":0.58599999999999997},"max":{"1min":1.486,"5min":1.659,"15min":2.2480000000000002},"last":1.4750000000000001}]},{"osd":1,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.111,"15min":1.1140000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.57599999999999996},"max":{"1min":1.6000000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1000000000000001,"15min":1.1379999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.60699999999999998},"max":{"1min":1.3360000000000001,"5min":1.8100000000000001,"15min":2.8570000000000002},"last":0.90100000000000002}]},{"osd":2,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1899999999999999,"15min":1.1919999999999999},"min":{"1min":0.83099999999999996,"5min":0.71699999999999997,"15min":0.49099999999999999},"max":{"1min":1.4730000000000001,"5min":1.7090000000000001,"15min":2.621},"last":1.155},{"interface":"front","average":{"1min":1.115,"5min":1.153,"15min":1.153},"min":{"1min":0.71699999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.4710000000000001,"5min":1.8360000000000001,"15min":2.0059999999999998},"last":1.2589999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.2310000000000001,"15min":1.214},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.50900000000000001},"max":{"1min":1.7709999999999999,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.234,"5min":1.2430000000000001,"15min":1.2609999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.67100000000000004},"max":{"1min":1.458,"5min":1.8640000000000001,"15min":2.8359999999999999},"last":0.98599999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.0580000000000001,"15min":1.1000000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5860000000000001,"5min":1.6839999999999999,"15min":4.5350000000000001},"last":1.093},{"interface":"front","average":{"1min":1.163,"5min":1.2170000000000001,"15min":1.2130000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.8580000000000001,"5min":2.004,"15min":2.0739999999999998},"last":1.081}]},{"osd":5,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3799999999999999,"15min":1.415},"min":{"1min":0.79800000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.744,"5min":2.1469999999999998,"15min":2.157},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2210000000000001,"15min":1.256},"min":{"1min":0.621,"5min":0.62,"15min":0.60899999999999999},"max":{"1min":1.8140000000000001,"5min":1.8779999999999999,"15min":2.9460000000000002},"last":0.97399999999999998}]},{"osd":7,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.216,"15min":1.2250000000000001},"min":{"1min":0.77400000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.9139999999999999,"5min":1.9239999999999999,"15min":3.1989999999999998},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.407,"15min":1.419},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.65200000000000002},"max":{"1min":1.8220000000000001,"5min":2.0649999999999999,"15min":2.9350000000000001},"last":1.323}]}]},{"osd":4,"up_from":14,"seq":60129542328,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307212,"kb_used_data":1924,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064628,"statfs":{"total":96636764160,"available":96322179072,"internally_reserved":0,"allocated":1970176,"data_stored":1014901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.0569999999999999,"15min":1.083},"min":{"1min":0.73799999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4810000000000001,"5min":1.5660000000000001,"15min":1.5840000000000001},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.044,"15min":1.056},"min":{"1min":0.85399999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.677},"last":1.319}]},{"osd":1,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.135,"15min":1.1319999999999999},"min":{"1min":0.70599999999999996,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":2.617},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.1419999999999999,"15min":1.1539999999999999},"min":{"1min":0.82099999999999995,"5min":0.629,"15min":0.54800000000000004},"max":{"1min":1.573,"5min":1.6319999999999999,"15min":1.6619999999999999},"last":1.2809999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.165,"15min":1.1830000000000001},"min":{"1min":0.81399999999999995,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.5029999999999999,"5min":1.823,"15min":1.9450000000000001},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.155,"15min":1.1619999999999999},"min":{"1min":0.84199999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9330000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.044}]},{"osd":3,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1399999999999999,"15min":1.1459999999999999},"min":{"1min":0.85699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.5229999999999999,"5min":1.5940000000000001,"15min":1.8660000000000001},"last":1.107},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2549999999999999,"15min":1.264},"min":{"1min":0.93000000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.611,"5min":1.8200000000000001,"15min":2.5630000000000002},"last":1.0029999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1479999999999999,"15min":1.1830000000000001},"min":{"1min":0.91900000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.554,"5min":1.643,"15min":2.0840000000000001},"last":1.387},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.1479999999999999,"15min":1.1879999999999999},"min":{"1min":0.89400000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":2.3570000000000002},"last":1.125}]},{"osd":6,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2410000000000001,"15min":1.264},"min":{"1min":0.97999999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5880000000000001,"5min":1.913,"15min":2.6640000000000001},"last":1.264},{"interface":"front","average":{"1min":1.325,"5min":1.288,"15min":1.323},"min":{"1min":0.88,"5min":0.67600000000000005,"15min":0.57299999999999995},"max":{"1min":1.585,"5min":1.9139999999999999,"15min":2.173},"last":1.5680000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2769999999999999,"15min":1.3069999999999999},"min":{"1min":0.89500000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.7629999999999999,"5min":1.9199999999999999,"15min":2.6030000000000002},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.3700000000000001,"15min":1.3879999999999999},"min":{"1min":1.044,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8240000000000001,"5min":2.069,"15min":2.7669999999999999},"last":1.306}]}]},{"osd":3,"up_from":14,"seq":60129542328,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303084,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068756,"statfs":{"total":96636764160,"available":96326406144,"internally_reserved":0,"allocated":1937408,"data_stored":983121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85999999999999999,"15min":0.83699999999999997},"min":{"1min":0.54200000000000004,"5min":0.54000000000000004,"15min":0.45400000000000001},"max":{"1min":1.327,"5min":1.4219999999999999,"15min":1.528},"last":0.746},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.89900000000000002,"15min":0.88800000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.39800000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.79700000000000004}]},{"osd":1,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.83899999999999997,"15min":0.872},"min":{"1min":0.62,"5min":0.53100000000000003,"15min":0.435},"max":{"1min":1.165,"5min":1.25,"15min":1.4670000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.996,"5min":0.98599999999999999,"15min":0.99199999999999999},"min":{"1min":0.72199999999999998,"5min":0.63700000000000001,"15min":0.55800000000000005},"max":{"1min":1.446,"5min":1.446,"15min":1.544},"last":1.016}]},{"osd":2,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.026,"15min":1.0209999999999999},"min":{"1min":0.66400000000000003,"5min":0.64200000000000002,"15min":0.54600000000000004},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0529999999999999,"15min":1.0660000000000001},"min":{"1min":0.66000000000000003,"5min":0.63100000000000001,"15min":0.50600000000000001},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.70099999999999996}]},{"osd":4,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2470000000000001,"15min":1.228},"min":{"1min":1.0029999999999999,"5min":0.88600000000000001,"15min":0.71399999999999997},"max":{"1min":1.5349999999999999,"5min":1.554,"15min":1.9139999999999999},"last":1.002},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2170000000000001,"15min":1.2250000000000001},"min":{"1min":1.01,"5min":0.78500000000000003,"15min":0.621},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.726},"last":0.95499999999999996}]},{"osd":5,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.252,"15min":1.266},"min":{"1min":0.91700000000000004,"5min":0.90200000000000002,"15min":0.64300000000000002},"max":{"1min":1.5900000000000001,"5min":1.7290000000000001,"15min":2.0459999999999998},"last":1.085},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.27,"15min":1.272},"min":{"1min":0.95899999999999996,"5min":0.94799999999999995,"15min":0.63500000000000001},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.927},"last":0.94499999999999995}]},{"osd":6,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.345,"15min":1.339},"min":{"1min":0.91400000000000003,"5min":0.875,"15min":0.72899999999999998},"max":{"1min":1.575,"5min":1.8129999999999999,"15min":1.9970000000000001},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3560000000000001,"15min":1.3720000000000001},"min":{"1min":1.0980000000000001,"5min":0.871,"15min":0.77700000000000002},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":2.3570000000000002},"last":1.4750000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.331,"15min":1.3440000000000001},"min":{"1min":1.0149999999999999,"5min":0.92500000000000004,"15min":0.68100000000000005},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":2.0470000000000002},"last":1.26},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.345,"15min":1.355},"min":{"1min":1.113,"5min":0.97299999999999998,"15min":0.66600000000000004},"max":{"1min":1.8500000000000001,"5min":1.8779999999999999,"15min":2.1040000000000001},"last":1.0449999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542328,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303084,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068756,"statfs":{"total":96636764160,"available":96326406144,"internally_reserved":0,"allocated":1937408,"data_stored":981525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.65400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97099999999999997,"5min":1.2350000000000001,"15min":1.409},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.84299999999999997,"15min":0.84399999999999997},"min":{"1min":0.58799999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2689999999999999,"5min":1.4950000000000001,"15min":1.619},"last":0.58499999999999996}]},{"osd":1,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.90000000000000002,"15min":0.91100000000000003},"min":{"1min":0.74199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.95599999999999996,"15min":0.95399999999999996},"min":{"1min":0.67200000000000004,"5min":0.55100000000000005,"15min":0.54600000000000004},"max":{"1min":1.3580000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":1.01}]},{"osd":3,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95599999999999996,"15min":0.97199999999999998},"min":{"1min":0.60699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.385,"5min":1.4359999999999999,"15min":1.6259999999999999},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.125,"15min":1.125},"min":{"1min":0.67700000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.488,"5min":1.5549999999999999,"15min":1.5800000000000001},"last":1.492}]},{"osd":4,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.218,"15min":1.2070000000000001},"min":{"1min":1.052,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.581,"5min":1.583,"15min":1.9410000000000001},"last":1.595},{"interface":"front","average":{"1min":1.3,"5min":1.21,"15min":1.212},"min":{"1min":0.99199999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6579999999999999,"5min":1.7170000000000001,"15min":1.7829999999999999},"last":1.1699999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1619999999999999,"15min":1.1830000000000001},"min":{"1min":0.98499999999999999,"5min":0.68100000000000005,"15min":0.67000000000000004},"max":{"1min":1.47,"5min":1.625,"15min":1.8939999999999999},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.266,"15min":1.272},"min":{"1min":1.0660000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.556,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.323}]},{"osd":6,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.262,"15min":1.2729999999999999},"min":{"1min":0.93999999999999995,"5min":0.62,"15min":0.62},"max":{"1min":1.6990000000000001,"5min":2,"15min":2.137},"last":1.294},{"interface":"front","average":{"1min":1.292,"5min":1.2669999999999999,"15min":1.2689999999999999},"min":{"1min":1.0880000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.538,"5min":1.9350000000000001,"15min":2.0870000000000002},"last":1.2549999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.29,"15min":1.3109999999999999},"min":{"1min":0.95699999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7789999999999999,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.298,"15min":1.2989999999999999},"min":{"1min":1.101,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.7270000000000001,"5min":1.881,"15min":2.0350000000000001},"last":1.6220000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542328,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307188,"kb_used_data":1900,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064652,"statfs":{"total":96636764160,"available":96322203648,"internally_reserved":0,"allocated":1945600,"data_stored":983408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80600000000000005,"15min":0.83299999999999996},"min":{"1min":0.503,"5min":0.44400000000000001,"15min":0.36199999999999999},"max":{"1min":1.093,"5min":1.756,"15min":4.3630000000000004},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.90600000000000003,"15min":0.93700000000000006},"min":{"1min":0.61799999999999999,"5min":0.52600000000000002,"15min":0.40000000000000002},"max":{"1min":1.2490000000000001,"5min":1.6719999999999999,"15min":6.157},"last":0.96799999999999997}]},{"osd":2,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.96399999999999997,"15min":0.97899999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.46800000000000003},"max":{"1min":1.3400000000000001,"5min":1.625,"15min":4.1699999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0029999999999999,"15min":1.0289999999999999},"min":{"1min":0.72199999999999998,"5min":0.60099999999999998,"15min":0.47799999999999998},"max":{"1min":1.379,"5min":1.879,"15min":6.8369999999999997},"last":0.874}]},{"osd":3,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.94099999999999995,"15min":0.95599999999999996},"min":{"1min":0.59499999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.3120000000000001,"5min":1.6519999999999999,"15min":3.5070000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.073,"5min":1.093,"15min":1.113},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56200000000000006},"max":{"1min":1.3460000000000001,"5min":2.0070000000000001,"15min":5.2149999999999999},"last":0.84799999999999998}]},{"osd":4,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.224,"15min":1.2609999999999999},"min":{"1min":0.81499999999999995,"5min":0.80500000000000005,"15min":0.66700000000000004},"max":{"1min":1.5900000000000001,"5min":1.944,"15min":6.2000000000000002},"last":1.157},{"interface":"front","average":{"1min":1.165,"5min":1.1870000000000001,"15min":1.196},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.3939999999999999,"5min":1.6779999999999999,"15min":3.577},"last":1.2490000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.23,"15min":1.25},"min":{"1min":0.93000000000000005,"5min":0.81499999999999995,"15min":0.69599999999999995},"max":{"1min":1.478,"5min":1.857,"15min":3.548},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.21,"5min":1.278,"15min":1.2969999999999999},"min":{"1min":0.86199999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.4330000000000001,"5min":1.774,"15min":4.4080000000000004},"last":1.1419999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3180000000000001,"15min":1.349},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.82299999999999995},"max":{"1min":1.5640000000000001,"5min":1.915,"15min":4.4489999999999998},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.2889999999999999,"15min":1.3340000000000001},"min":{"1min":0.93300000000000005,"5min":0.86499999999999999,"15min":0.75900000000000001},"max":{"1min":1.5409999999999999,"5min":2.056,"15min":6.9690000000000003},"last":1.274}]},{"osd":7,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.327,"15min":1.401},"min":{"1min":1.036,"5min":0.90400000000000003,"15min":0.79900000000000004},"max":{"1min":1.6319999999999999,"5min":1.994,"15min":18.253},"last":1.361},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2889999999999999,"15min":1.3380000000000001},"min":{"1min":0.997,"5min":0.95599999999999996,"15min":0.79000000000000004},"max":{"1min":1.452,"5min":1.8720000000000001,"15min":11.006},"last":1.1779999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575033,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303112,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068728,"statfs":{"total":96636764160,"available":96326377472,"internally_reserved":0,"allocated":1966080,"data_stored":995360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0800000000000001,"15min":1.073},"min":{"1min":0.65800000000000003,"5min":0.627,"15min":0.49399999999999999},"max":{"1min":1.827,"5min":1.827,"15min":2.3370000000000002},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0589999999999999},"min":{"1min":0.63600000000000001,"5min":0.61099999999999999,"15min":0.60799999999999998},"max":{"1min":1.3580000000000001,"5min":1.6519999999999999,"15min":3.46},"last":0.97699999999999998}]},{"osd":1,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.105,"15min":1.121},"min":{"1min":0.84599999999999997,"5min":0.71199999999999997,"15min":0.58999999999999997},"max":{"1min":1.4570000000000001,"5min":1.516,"15min":3.6349999999999998},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.161,"15min":1.143},"min":{"1min":0.84199999999999997,"5min":0.73999999999999999,"15min":0.63700000000000001},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":3.516},"last":1.1040000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.1459999999999999,"15min":1.165},"min":{"1min":0.88600000000000001,"5min":0.74099999999999999,"15min":0.73199999999999998},"max":{"1min":1.774,"5min":1.774,"15min":5.4180000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1899999999999999,"15min":1.204},"min":{"1min":0.77300000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6839999999999999,"5min":2.0840000000000001,"15min":3.5859999999999999},"last":1.373}]},{"osd":3,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2070000000000001,"15min":1.196},"min":{"1min":0.91400000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6439999999999999,"5min":1.8560000000000001,"15min":2.141},"last":1.04},{"interface":"front","average":{"1min":1.294,"5min":1.296,"15min":1.3169999999999999},"min":{"1min":0.88400000000000001,"5min":0.875,"15min":0.78600000000000003},"max":{"1min":1.6140000000000001,"5min":1.6739999999999999,"15min":4.2729999999999997},"last":1.2829999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.2130000000000001,"15min":1.26},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.4850000000000001,"5min":1.591,"15min":3.706},"last":1.492},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.258,"15min":1.3080000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.64100000000000001},"max":{"1min":1.863,"5min":1.9219999999999999,"15min":6.1180000000000003},"last":1.4330000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.268,"15min":1.254},"min":{"1min":0.88800000000000001,"5min":0.80700000000000005,"15min":0.749},"max":{"1min":1.6080000000000001,"5min":1.843,"15min":4.2309999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.2869999999999999,"15min":1.341},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.546,"5min":2.2240000000000002,"15min":3.641},"last":1.212}]},{"osd":7,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.284,"15min":1.2949999999999999},"min":{"1min":0.83499999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6970000000000001,"5min":1.988,"15min":3.516},"last":1.23},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.379,"15min":1.387},"min":{"1min":0.95899999999999996,"5min":0.94299999999999995,"15min":0.89500000000000002},"max":{"1min":1.925,"5min":2.282,"15min":3.754},"last":1.361}]}]},{"osd":1,"up_from":13,"seq":55834575032,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303112,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068728,"statfs":{"total":96636764160,"available":96326377472,"internally_reserved":0,"allocated":1966080,"data_stored":995360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.80500000000000005,"15min":0.81000000000000005},"min":{"1min":0.63,"5min":0.504,"15min":0.48799999999999999},"max":{"1min":1.22,"5min":1.671,"15min":1.671},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.85599999999999998,"15min":0.879},"min":{"1min":0.52700000000000002,"5min":0.502,"15min":0.46600000000000003},"max":{"1min":1.1799999999999999,"5min":1.3939999999999999,"15min":1.6899999999999999},"last":1.0009999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97799999999999998,"15min":0.94099999999999995},"min":{"1min":0.755,"5min":0.60999999999999999,"15min":0.44600000000000001},"max":{"1min":1.306,"5min":1.806,"15min":1.806},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.872,"5min":0.89000000000000001,"15min":0.90500000000000003},"min":{"1min":0.63200000000000001,"5min":0.55800000000000005,"15min":0.49199999999999999},"max":{"1min":1.2509999999999999,"5min":1.3360000000000001,"15min":1.887},"last":1.0429999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.129,"15min":1.125},"min":{"1min":0.747,"5min":0.66300000000000003,"15min":0.42399999999999999},"max":{"1min":1.476,"5min":1.6160000000000001,"15min":1.712},"last":1.131},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0549999999999999,"15min":1.042},"min":{"1min":0.68500000000000005,"5min":0.52300000000000002,"15min":0.46000000000000002},"max":{"1min":1.4450000000000001,"5min":1.5409999999999999,"15min":1.7090000000000001},"last":1.109}]},{"osd":4,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.208,"15min":1.1930000000000001},"min":{"1min":0.83699999999999997,"5min":0.80600000000000005,"15min":0.65100000000000002},"max":{"1min":1.671,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.1970000000000001,"15min":1.212},"min":{"1min":0.97399999999999998,"5min":0.83799999999999997,"15min":0.50800000000000001},"max":{"1min":1.4710000000000001,"5min":1.5429999999999999,"15min":2.206},"last":1.1850000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2649999999999999,"15min":1.2470000000000001},"min":{"1min":0.98499999999999999,"5min":0.90300000000000002,"15min":0.746},"max":{"1min":1.4430000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.2509999999999999,"15min":1.256},"min":{"1min":1.038,"5min":0.92300000000000004,"15min":0.56100000000000005},"max":{"1min":1.4019999999999999,"5min":1.718,"15min":2.4609999999999999},"last":1.238}]},{"osd":6,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.327,"15min":1.327},"min":{"1min":1.1020000000000001,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.671,"5min":1.8640000000000001,"15min":1.948},"last":1.395},{"interface":"front","average":{"1min":1.429,"5min":1.431,"15min":1.4079999999999999},"min":{"1min":1.2230000000000001,"5min":0.91600000000000004,"15min":0.87},"max":{"1min":1.786,"5min":1.9670000000000001,"15min":2.3290000000000002},"last":1.486}]},{"osd":7,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3149999999999999,"15min":1.3169999999999999},"min":{"1min":1.1000000000000001,"5min":0.92000000000000004,"15min":0.84199999999999997},"max":{"1min":1.595,"5min":2.089,"15min":2.161},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.3999999999999999,"15min":1.3779999999999999},"min":{"1min":1.095,"5min":1.0249999999999999,"15min":0.78600000000000003},"max":{"1min":1.899,"5min":2.137,"15min":2.2349999999999999},"last":1.4119999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T03:11:55.001 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-15T03:11:55.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.151+0000 7fe61ce79700 1 Processor -- start 2023-11-15T03:11:55.161 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.151+0000 7fe61ce79700 1 -- start start 2023-11-15T03:11:55.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.155+0000 7fe61ce79700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe6180c3b70 0x7fe61813b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:55.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.155+0000 7fe61ce79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6180c5da0 0x7fe618066b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:55.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.155+0000 7fe61ce79700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6180c5910 con 0x7fe61813b280 2023-11-15T03:11:55.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.155+0000 7fe61ce79700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6180c4d00 con 0x7fe6180c3b70 2023-11-15T03:11:55.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.155+0000 7fe61ce79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180c4fe0 con 0x7fe6180c5da0 2023-11-15T03:11:55.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.155+0000 7fe617fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6180c5da0 0x7fe618066b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:55.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.155+0000 7fe6177fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe61813b280 0x7fe6180c4520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:50220/0 (socket says 172.21.15.121:50220) 2023-11-15T03:11:55.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.155+0000 7fe6177fe700 1 -- 172.21.15.121:0/3206618538 learned_addr learned my addr 172.21.15.121:0/3206618538 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:55.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.155+0000 7fe616ffd700 1 --2- 172.21.15.121:0/3206618538 >> v2:172.21.15.190:3300/0 conn(0x7fe6180c3b70 0x7fe61813b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:55.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.155+0000 7fe6167fc700 1 -- 172.21.15.121:0/3206618538 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3417508515 0 0) 0x7fe6180c5910 con 0x7fe61813b280 2023-11-15T03:11:55.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.155+0000 7fe6167fc700 1 -- 172.21.15.121:0/3206618538 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6000033f0 con 0x7fe61813b280 2023-11-15T03:11:55.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.155+0000 7fe617fff700 1 -- 172.21.15.121:0/3206618538 >> v2:172.21.15.190:3300/0 conn(0x7fe6180c3b70 msgr2=0x7fe61813b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:55.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.155+0000 7fe617fff700 1 --2- 172.21.15.121:0/3206618538 >> v2:172.21.15.190:3300/0 conn(0x7fe6180c3b70 0x7fe61813b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:55.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.155+0000 7fe617fff700 1 -- 172.21.15.121:0/3206618538 >> v1:172.21.15.190:6789/0 conn(0x7fe61813b280 legacy=0x7fe6180c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:55.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.155+0000 7fe617fff700 1 -- 172.21.15.121:0/3206618538 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6180c52c0 con 0x7fe6180c5da0 2023-11-15T03:11:55.169 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.159+0000 7fe617fff700 1 --2- 172.21.15.121:0/3206618538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6180c5da0 0x7fe618066b10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fe610009e60 tx=0x7fe610009780).ready entity=mon.2 client_cookie=ca9903cb5d2f9ba1 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:55.170 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.159+0000 7fe6167fc700 1 -- 172.21.15.121:0/3206618538 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61000e070 con 0x7fe6180c5da0 2023-11-15T03:11:55.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.159+0000 7fe6167fc700 1 -- 172.21.15.121:0/3206618538 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61000bab0 con 0x7fe6180c5da0 2023-11-15T03:11:55.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.159+0000 7fe6167fc700 1 -- 172.21.15.121:0/3206618538 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61000be30 con 0x7fe6180c5da0 2023-11-15T03:11:55.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.159+0000 7fe61ce79700 1 -- 172.21.15.121:0/3206618538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6180c5da0 msgr2=0x7fe618066b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:55.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.159+0000 7fe61ce79700 1 --2- 172.21.15.121:0/3206618538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6180c5da0 0x7fe618066b10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fe610009e60 tx=0x7fe610009780).stop 2023-11-15T03:11:55.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.159+0000 7fe61ce79700 1 -- 172.21.15.121:0/3206618538 shutdown_connections 2023-11-15T03:11:55.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.159+0000 7fe61ce79700 1 --2- 172.21.15.121:0/3206618538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6180c5da0 0x7fe618066b10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:55.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.159+0000 7fe61ce79700 1 --2- 172.21.15.121:0/3206618538 >> v2:172.21.15.190:3300/0 conn(0x7fe6180c3b70 0x7fe61813b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:55.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.159+0000 7fe61ce79700 1 -- 172.21.15.121:0/3206618538 >> 172.21.15.121:0/3206618538 conn(0x7fe6180c7490 msgr2=0x7fe6180c3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:55.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.159+0000 7fe61ce79700 1 -- 172.21.15.121:0/3206618538 shutdown_connections 2023-11-15T03:11:55.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.159+0000 7fe61ce79700 1 -- 172.21.15.121:0/3206618538 wait complete. 2023-11-15T03:11:55.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.163+0000 7fe61ce79700 1 Processor -- start 2023-11-15T03:11:55.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.163+0000 7fe61ce79700 1 -- start start 2023-11-15T03:11:55.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe61ce79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6180c3b70 0x7fe618150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:55.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe61ce79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6180c5da0 0x7fe6180bf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:55.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe61ce79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe61813b280 0x7fe618132000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:55.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe61ce79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe618151070 con 0x7fe61813b280 2023-11-15T03:11:55.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe61ce79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180a9a50 con 0x7fe6180c5da0 2023-11-15T03:11:55.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe61ce79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe618167a80 con 0x7fe6180c3b70 2023-11-15T03:11:55.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe617fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe61813b280 0x7fe618132000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:55.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe617fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe61813b280 0x7fe618132000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54130/0 (socket says 172.21.15.121:54130) 2023-11-15T03:11:55.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe617fff700 1 -- 172.21.15.121:0/2685208953 learned_addr learned my addr 172.21.15.121:0/2685208953 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:55.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe6177fe700 1 --2- 172.21.15.121:0/2685208953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6180c3b70 0x7fe618150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:55.179 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe616ffd700 1 --2- 172.21.15.121:0/2685208953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6180c5da0 0x7fe6180bf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:55.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe617fff700 1 -- 172.21.15.121:0/2685208953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6180c3b70 msgr2=0x7fe618150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:55.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe617fff700 1 --2- 172.21.15.121:0/2685208953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6180c3b70 0x7fe618150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:55.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe617fff700 1 -- 172.21.15.121:0/2685208953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6180c5da0 msgr2=0x7fe6180bf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:55.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe617fff700 1 --2- 172.21.15.121:0/2685208953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6180c5da0 0x7fe6180bf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:55.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe617fff700 1 -- 172.21.15.121:0/2685208953 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe610009a20 con 0x7fe61813b280 2023-11-15T03:11:55.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe617fff700 1 --2- 172.21.15.121:0/2685208953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe61813b280 0x7fe618132000 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 rx=0x7fe61000b610 tx=0x7fe610009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:55.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe5ff7fe700 1 -- 172.21.15.121:0/2685208953 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610013e70 con 0x7fe61813b280 2023-11-15T03:11:55.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe5ff7fe700 1 -- 172.21.15.121:0/2685208953 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61001d630 con 0x7fe61813b280 2023-11-15T03:11:55.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe61ce79700 1 -- 172.21.15.121:0/2685208953 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe618167d60 con 0x7fe61813b280 2023-11-15T03:11:55.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe5ff7fe700 1 -- 172.21.15.121:0/2685208953 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61001d9f0 con 0x7fe61813b280 2023-11-15T03:11:55.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.167+0000 7fe61ce79700 1 -- 172.21.15.121:0/2685208953 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6180bfeb0 con 0x7fe61813b280 2023-11-15T03:11:55.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.171+0000 7fe61ce79700 1 -- 172.21.15.121:0/2685208953 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6180a9bc0 con 0x7fe61813b280 2023-11-15T03:11:55.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.171+0000 7fe5ff7fe700 1 -- 172.21.15.121:0/2685208953 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe610003040 con 0x7fe61813b280 2023-11-15T03:11:55.183 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.171+0000 7fe5ff7fe700 1 --2- 172.21.15.121:0/2685208953 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe60405a540 0x7fe60405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:55.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.171+0000 7fe5ff7fe700 1 -- 172.21.15.121:0/2685208953 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7fe6100762c0 con 0x7fe61813b280 2023-11-15T03:11:55.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.171+0000 7fe6177fe700 1 --2- 172.21.15.121:0/2685208953 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe60405a540 0x7fe60405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:55.184 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.175+0000 7fe6177fe700 1 --2- 172.21.15.121:0/2685208953 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe60405a540 0x7fe60405ca00 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7fe608007fd0 tx=0x7fe608009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:55.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.175+0000 7fe5ff7fe700 1 -- 172.21.15.121:0/2685208953 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe610049c80 con 0x7fe61813b280 2023-11-15T03:11:55.347 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.339+0000 7fe61ce79700 1 -- 172.21.15.121:0/2685208953 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe61806bcf0 con 0x7fe61813b280 2023-11-15T03:11:55.350 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.339+0000 7fe5ff7fe700 1 -- 172.21.15.121:0/2685208953 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+14033 (secure 0 0 0) 0x7fe61004d2a0 con 0x7fe61813b280 2023-11-15T03:11:55.350 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:11:55.351 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":151,"fsid":"be3e38c1-527f-4644-9b96-70c42632cdc5","created":"2023-11-15T02:56:33.070741+0000","modified":"2023-11-15T03:11:51.405249+0000","last_up_change":"2023-11-15T02:56:44.150053+0000","last_in_change":"2023-11-15T02:56:36.158536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-15T02:56:43.223941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-15T02:56:45.163959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"dc72eaa6-4f9a-46cb-8ee1-62886e3a8c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":33133},{"type":"v1","addr":"172.21.15.121:6817","nonce":33133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":33133},{"type":"v1","addr":"172.21.15.121:6819","nonce":33133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":33133},{"type":"v1","addr":"172.21.15.121:6827","nonce":33133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":33133},{"type":"v1","addr":"172.21.15.121:6823","nonce":33133}]},"public_addr":"172.21.15.121:6817/33133","cluster_addr":"172.21.15.121:6819/33133","heartbeat_back_addr":"172.21.15.121:6827/33133","heartbeat_front_addr":"172.21.15.121:6823/33133","state":["exists","up"]},{"osd":1,"uuid":"ae8fa944-2f30-418b-99f8-a24a5566d5e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":33131},{"type":"v1","addr":"172.21.15.121:6801","nonce":33131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":33131},{"type":"v1","addr":"172.21.15.121:6803","nonce":33131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":33131},{"type":"v1","addr":"172.21.15.121:6807","nonce":33131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":33131},{"type":"v1","addr":"172.21.15.121:6805","nonce":33131}]},"public_addr":"172.21.15.121:6801/33131","cluster_addr":"172.21.15.121:6803/33131","heartbeat_back_addr":"172.21.15.121:6807/33131","heartbeat_front_addr":"172.21.15.121:6805/33131","state":["exists","up"]},{"osd":2,"uuid":"4895f0ef-152e-40b2-99c5-5e12c33a05a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":33134},{"type":"v1","addr":"172.21.15.121:6809","nonce":33134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":33134},{"type":"v1","addr":"172.21.15.121:6811","nonce":33134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":33134},{"type":"v1","addr":"172.21.15.121:6815","nonce":33134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":33134},{"type":"v1","addr":"172.21.15.121:6813","nonce":33134}]},"public_addr":"172.21.15.121:6809/33134","cluster_addr":"172.21.15.121:6811/33134","heartbeat_back_addr":"172.21.15.121:6815/33134","heartbeat_front_addr":"172.21.15.121:6813/33134","state":["exists","up"]},{"osd":3,"uuid":"378df070-0c7f-4a3b-8321-6f7a463b3aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":33135},{"type":"v1","addr":"172.21.15.121:6822","nonce":33135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":33135},{"type":"v1","addr":"172.21.15.121:6826","nonce":33135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":33135},{"type":"v1","addr":"172.21.15.121:6831","nonce":33135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":33135},{"type":"v1","addr":"172.21.15.121:6829","nonce":33135}]},"public_addr":"172.21.15.121:6822/33135","cluster_addr":"172.21.15.121:6826/33135","heartbeat_back_addr":"172.21.15.121:6831/33135","heartbeat_front_addr":"172.21.15.121:6829/33135","state":["exists","up"]},{"osd":4,"uuid":"624feb7b-67da-4b57-b067-673923606388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":33348},{"type":"v1","addr":"172.21.15.190:6819","nonce":33348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":33348},{"type":"v1","addr":"172.21.15.190:6825","nonce":33348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":33348},{"type":"v1","addr":"172.21.15.190:6831","nonce":33348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":33348},{"type":"v1","addr":"172.21.15.190:6829","nonce":33348}]},"public_addr":"172.21.15.190:6819/33348","cluster_addr":"172.21.15.190:6825/33348","heartbeat_back_addr":"172.21.15.190:6831/33348","heartbeat_front_addr":"172.21.15.190:6829/33348","state":["exists","up"]},{"osd":5,"uuid":"cfe09666-0516-4e88-8983-4a9048d28a0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":33349},{"type":"v1","addr":"172.21.15.190:6801","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":33349},{"type":"v1","addr":"172.21.15.190:6803","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":33349},{"type":"v1","addr":"172.21.15.190:6807","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":33349},{"type":"v1","addr":"172.21.15.190:6805","nonce":33349}]},"public_addr":"172.21.15.190:6801/33349","cluster_addr":"172.21.15.190:6803/33349","heartbeat_back_addr":"172.21.15.190:6807/33349","heartbeat_front_addr":"172.21.15.190:6805/33349","state":["exists","up"]},{"osd":6,"uuid":"b18066ff-00a8-4de7-8395-8abc77382949","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":33346},{"type":"v1","addr":"172.21.15.190:6810","nonce":33346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":33346},{"type":"v1","addr":"172.21.15.190:6813","nonce":33346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":33346},{"type":"v1","addr":"172.21.15.190:6823","nonce":33346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6815","nonce":33346},{"type":"v1","addr":"172.21.15.190:6817","nonce":33346}]},"public_addr":"172.21.15.190:6810/33346","cluster_addr":"172.21.15.190:6813/33346","heartbeat_back_addr":"172.21.15.190:6823/33346","heartbeat_front_addr":"172.21.15.190:6817/33346","state":["exists","up"]},{"osd":7,"uuid":"70d5381f-fb43-4ea6-aed2-2184162a1cce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":33347},{"type":"v1","addr":"172.21.15.190:6811","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":33347},{"type":"v1","addr":"172.21.15.190:6818","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":33347},{"type":"v1","addr":"172.21.15.190:6828","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":33347},{"type":"v1","addr":"172.21.15.190:6824","nonce":33347}]},"public_addr":"172.21.15.190:6811/33347","cluster_addr":"172.21.15.190:6818/33347","heartbeat_back_addr":"172.21.15.190:6828/33347","heartbeat_front_addr":"172.21.15.190:6824/33347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.141713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.986450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.369652+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.382495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.179695+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:41.952658+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.172759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-15T02:56:42.171457+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6837/3298117008":"2023-11-16T03:11:47.845872+0000","172.21.15.121:6836/3298117008":"2023-11-16T03:11:47.845872+0000","172.21.15.121:6837/2335356851":"2023-11-16T03:11:31.820010+0000","172.21.15.121:6836/2335356851":"2023-11-16T03:11:31.820010+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-15T03:11:55.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.343+0000 7fe61ce79700 1 -- 172.21.15.121:0/2685208953 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe60405a540 msgr2=0x7fe60405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:55.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.343+0000 7fe61ce79700 1 --2- 172.21.15.121:0/2685208953 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe60405a540 0x7fe60405ca00 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7fe608007fd0 tx=0x7fe608009040).stop 2023-11-15T03:11:55.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.343+0000 7fe61ce79700 1 -- 172.21.15.121:0/2685208953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe61813b280 msgr2=0x7fe618132000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:55.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.343+0000 7fe61ce79700 1 --2- 172.21.15.121:0/2685208953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe61813b280 0x7fe618132000 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 rx=0x7fe61000b610 tx=0x7fe610009780).stop 2023-11-15T03:11:55.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.343+0000 7fe61ce79700 1 -- 172.21.15.121:0/2685208953 shutdown_connections 2023-11-15T03:11:55.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.343+0000 7fe61ce79700 1 --2- 172.21.15.121:0/2685208953 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fe60405a540 0x7fe60405ca00 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:55.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.343+0000 7fe61ce79700 1 --2- 172.21.15.121:0/2685208953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe61813b280 0x7fe618132000 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:55.353 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.343+0000 7fe61ce79700 1 --2- 172.21.15.121:0/2685208953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6180c5da0 0x7fe6180bf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:55.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.343+0000 7fe61ce79700 1 --2- 172.21.15.121:0/2685208953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6180c3b70 0x7fe618150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:55.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.343+0000 7fe61ce79700 1 -- 172.21.15.121:0/2685208953 >> 172.21.15.121:0/2685208953 conn(0x7fe6180c7490 msgr2=0x7fe618066240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:55.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.343+0000 7fe61ce79700 1 -- 172.21.15.121:0/2685208953 shutdown_connections 2023-11-15T03:11:55.354 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:55.343+0000 7fe61ce79700 1 -- 172.21.15.121:0/2685208953 wait complete. 2023-11-15T03:11:56.375 INFO:tasks.ceph:Scrubbing osd.0 2023-11-15T03:11:56.375 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-15T03:11:56.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.543+0000 7f28af59e700 1 Processor -- start 2023-11-15T03:11:56.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.543+0000 7f28af59e700 1 -- start start 2023-11-15T03:11:56.556 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.547+0000 7f28af59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28b0153a60 0x7f28b0153e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:56.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.547+0000 7f28af59e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f28b00c5d10 0x7f28b0066530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:56.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.547+0000 7f28af59e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28b00660a0 con 0x7f28b00bfe40 2023-11-15T03:11:56.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.547+0000 7f28af59e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28b00c0a00 con 0x7f28b00c5d10 2023-11-15T03:11:56.557 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.547+0000 7f28af59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b00c5500 con 0x7f28b0153a60 2023-11-15T03:11:56.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.547+0000 7f28add9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f28b00c5d10 0x7f28b0066530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:56.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.547+0000 7f28add9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f28b00c5d10 0x7f28b0066530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54132/0 (socket says 172.21.15.121:54132) 2023-11-15T03:11:56.559 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.547+0000 7f28add9b700 1 -- 172.21.15.121:0/1668500003 learned_addr learned my addr 172.21.15.121:0/1668500003 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:56.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.547+0000 7f28ae59c700 1 --2- 172.21.15.121:0/1668500003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28b0153a60 0x7f28b0153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:56.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.547+0000 7f28add9b700 1 -- 172.21.15.121:0/1668500003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28b0153a60 msgr2=0x7f28b0153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:56.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.547+0000 7f28add9b700 1 --2- 172.21.15.121:0/1668500003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28b0153a60 0x7f28b0153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:56.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.547+0000 7f28add9b700 1 -- 172.21.15.121:0/1668500003 >> v1:172.21.15.190:6789/0 conn(0x7f28b00bfe40 legacy=0x7f28b00c02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:56.560 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.547+0000 7f28add9b700 1 -- 172.21.15.121:0/1668500003 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b00657b0 con 0x7f28b00c5d10 2023-11-15T03:11:56.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.547+0000 7f28ae59c700 1 --2- 172.21.15.121:0/1668500003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28b0153a60 0x7f28b0153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:11:56.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.547+0000 7f28add9b700 1 --2- 172.21.15.121:0/1668500003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28b00c5d10 0x7f28b0066530 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 rx=0x7f28a4006fd0 tx=0x7f28a40090f0).ready entity=mon.0 client_cookie=82b256e60a072217 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:56.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.547+0000 7f28ad59a700 1 -- 172.21.15.121:0/1668500003 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a400b600 con 0x7f28b00c5d10 2023-11-15T03:11:56.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.551+0000 7f28ad59a700 1 -- 172.21.15.121:0/1668500003 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a400b760 con 0x7f28b00c5d10 2023-11-15T03:11:56.561 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.551+0000 7f28ad59a700 1 -- 172.21.15.121:0/1668500003 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a400bae0 con 0x7f28b00c5d10 2023-11-15T03:11:56.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.551+0000 7f28af59e700 1 -- 172.21.15.121:0/1668500003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28b00c5d10 msgr2=0x7f28b0066530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:56.562 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.551+0000 7f28af59e700 1 --2- 172.21.15.121:0/1668500003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28b00c5d10 0x7f28b0066530 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 rx=0x7f28a4006fd0 tx=0x7f28a40090f0).stop 2023-11-15T03:11:56.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.551+0000 7f28af59e700 1 -- 172.21.15.121:0/1668500003 shutdown_connections 2023-11-15T03:11:56.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.551+0000 7f28af59e700 1 --2- 172.21.15.121:0/1668500003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28b00c5d10 0x7f28b0066530 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:56.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.551+0000 7f28af59e700 1 --2- 172.21.15.121:0/1668500003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28b0153a60 0x7f28b0153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:56.563 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.551+0000 7f28af59e700 1 -- 172.21.15.121:0/1668500003 >> 172.21.15.121:0/1668500003 conn(0x7f28b0067a60 msgr2=0x7f28b0067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:56.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.551+0000 7f28af59e700 1 -- 172.21.15.121:0/1668500003 shutdown_connections 2023-11-15T03:11:56.564 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.551+0000 7f28af59e700 1 -- 172.21.15.121:0/1668500003 wait complete. 2023-11-15T03:11:56.565 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28af59e700 1 Processor -- start 2023-11-15T03:11:56.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28af59e700 1 -- start start 2023-11-15T03:11:56.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28af59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28b00bfe40 0x7f28b00c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:56.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28af59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28b00c5d10 0x7f28b00cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:56.566 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28af59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28b0153a60 0x7f28b00c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:56.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28af59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b00c95b0 con 0x7f28b00c5d10 2023-11-15T03:11:56.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28af59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b00c9f30 con 0x7f28b00bfe40 2023-11-15T03:11:56.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28af59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b00c89a0 con 0x7f28b0153a60 2023-11-15T03:11:56.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28add9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28b00c5d10 0x7f28b00cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:56.567 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28add9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28b00c5d10 0x7f28b00cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54140/0 (socket says 172.21.15.121:54140) 2023-11-15T03:11:56.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28add9b700 1 -- 172.21.15.121:0/1751481164 learned_addr learned my addr 172.21.15.121:0/1751481164 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:56.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28ae59c700 1 --2- 172.21.15.121:0/1751481164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28b00bfe40 0x7f28b00c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:56.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28aed9d700 1 --2- 172.21.15.121:0/1751481164 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28b0153a60 0x7f28b00c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:56.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28add9b700 1 -- 172.21.15.121:0/1751481164 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28b0153a60 msgr2=0x7f28b00c36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:56.568 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28add9b700 1 --2- 172.21.15.121:0/1751481164 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28b0153a60 0x7f28b00c36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:56.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28add9b700 1 -- 172.21.15.121:0/1751481164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28b00bfe40 msgr2=0x7f28b00c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:56.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28add9b700 1 --2- 172.21.15.121:0/1751481164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28b00bfe40 0x7f28b00c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:56.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28add9b700 1 -- 172.21.15.121:0/1751481164 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a4009370 con 0x7f28b00c5d10 2023-11-15T03:11:56.569 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28add9b700 1 --2- 172.21.15.121:0/1751481164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28b00c5d10 0x7f28b00cac60 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 rx=0x7f28a4011000 tx=0x7f28a4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:56.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f289e7fc700 1 -- 172.21.15.121:0/1751481164 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a400bae0 con 0x7f28b00c5d10 2023-11-15T03:11:56.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f289e7fc700 1 -- 172.21.15.121:0/1751481164 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a4000c90 con 0x7f28b00c5d10 2023-11-15T03:11:56.570 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28af59e700 1 -- 172.21.15.121:0/1751481164 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b00c8c80 con 0x7f28b00c5d10 2023-11-15T03:11:56.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f289e7fc700 1 -- 172.21.15.121:0/1751481164 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a401c650 con 0x7f28b00c5d10 2023-11-15T03:11:56.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.555+0000 7f28af59e700 1 -- 172.21.15.121:0/1751481164 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b00c9240 con 0x7f28b00c5d10 2023-11-15T03:11:56.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.559+0000 7f28af59e700 1 -- 172.21.15.121:0/1751481164 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f28b00c4ac0 con 0x7f28b00c5d10 2023-11-15T03:11:56.571 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.559+0000 7f289e7fc700 1 -- 172.21.15.121:0/1751481164 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f28a401c9c0 con 0x7f28b00c5d10 2023-11-15T03:11:56.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.559+0000 7f289e7fc700 1 --2- 172.21.15.121:0/1751481164 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f289005a540 0x7f289005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:56.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.563+0000 7f289e7fc700 1 -- 172.21.15.121:0/1751481164 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f28a4076330 con 0x7f28b00c5d10 2023-11-15T03:11:56.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.563+0000 7f28ae59c700 1 --2- 172.21.15.121:0/1751481164 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f289005a540 0x7f289005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:56.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.563+0000 7f289e7fc700 1 --2- 172.21.15.121:0/1751481164 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7f2890060bd0 0x7f2890063030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:56.573 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.563+0000 7f289e7fc700 1 -- 172.21.15.121:0/1751481164 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f28900636b0 con 0x7f2890060bd0 2023-11-15T03:11:56.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.563+0000 7f289e7fc700 1 -- 172.21.15.121:0/1751481164 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28a4078890 con 0x7f28b00c5d10 2023-11-15T03:11:56.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.563+0000 7f28aed9d700 1 --2- 172.21.15.121:0/1751481164 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7f2890060bd0 0x7f2890063030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:56.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.563+0000 7f28ae59c700 1 --2- 172.21.15.121:0/1751481164 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f289005a540 0x7f289005ca00 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f28a000a9d0 tx=0x7f28a0008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:56.574 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.563+0000 7f28aed9d700 1 --2- 172.21.15.121:0/1751481164 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7f2890060bd0 0x7f2890063030 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:56.575 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.563+0000 7f289e7fc700 1 -- 172.21.15.121:0/1751481164 <== osd.0 v2:172.21.15.121:6816/33133 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f28900636b0 con 0x7f2890060bd0 2023-11-15T03:11:56.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.583+0000 7f28af59e700 1 -- 172.21.15.121:0/1751481164 --> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f28b006bcf0 con 0x7f2890060bd0 2023-11-15T03:11:56.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.583+0000 7f289e7fc700 1 -- 172.21.15.121:0/1751481164 <== osd.0 v2:172.21.15.121:6816/33133 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f28b006bcf0 con 0x7f2890060bd0 2023-11-15T03:11:56.593 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:11:56.593 INFO:teuthology.orchestra.run.smithi121.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-15T03:11:56.593 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:11:56.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.583+0000 7f28af59e700 1 -- 172.21.15.121:0/1751481164 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7f2890060bd0 msgr2=0x7f2890063030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:56.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.583+0000 7f28af59e700 1 --2- 172.21.15.121:0/1751481164 >> [v2:172.21.15.121:6816/33133,v1:172.21.15.121:6817/33133] conn(0x7f2890060bd0 0x7f2890063030 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:56.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.583+0000 7f28af59e700 1 -- 172.21.15.121:0/1751481164 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f289005a540 msgr2=0x7f289005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:56.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.583+0000 7f28af59e700 1 --2- 172.21.15.121:0/1751481164 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f289005a540 0x7f289005ca00 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f28a000a9d0 tx=0x7f28a0008040).stop 2023-11-15T03:11:56.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.583+0000 7f28af59e700 1 -- 172.21.15.121:0/1751481164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28b00c5d10 msgr2=0x7f28b00cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:56.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.583+0000 7f28af59e700 1 --2- 172.21.15.121:0/1751481164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28b00c5d10 0x7f28b00cac60 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 rx=0x7f28a4011000 tx=0x7f28a4017000).stop 2023-11-15T03:11:56.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.583+0000 7f28aed9d700 1 -- 172.21.15.121:0/1751481164 reap_dead start 2023-11-15T03:11:56.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.583+0000 7f28af59e700 1 -- 172.21.15.121:0/1751481164 shutdown_connections 2023-11-15T03:11:56.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.583+0000 7f28af59e700 1 -- 172.21.15.121:0/1751481164 >> 172.21.15.121:0/1751481164 conn(0x7f28b0067a60 msgr2=0x7f28b00af720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:56.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.583+0000 7f28af59e700 1 -- 172.21.15.121:0/1751481164 shutdown_connections 2023-11-15T03:11:56.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.583+0000 7f28af59e700 1 -- 172.21.15.121:0/1751481164 wait complete. 2023-11-15T03:11:56.616 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-15T03:11:56.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.783+0000 7f82092f1700 1 Processor -- start 2023-11-15T03:11:56.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.783+0000 7f82092f1700 1 -- start start 2023-11-15T03:11:56.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.787+0000 7f82092f1700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8204066820 0x7f8204066c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:56.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.787+0000 7f82092f1700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82040a98d0 con 0x7f82041500d0 2023-11-15T03:11:56.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.787+0000 7f82092f1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82040ab660 con 0x7f8204065ad0 2023-11-15T03:11:56.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.787+0000 7f82092f1700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82040a9b70 con 0x7f8204066820 2023-11-15T03:11:56.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.787+0000 7f82037fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8204066820 0x7f8204066c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:56.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.787+0000 7f8202ffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f82041500d0 0x7f82041504b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51068/0 (socket says 172.21.15.121:51068) 2023-11-15T03:11:56.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.787+0000 7f8202ffd700 1 -- 172.21.15.121:0/71979654 learned_addr learned my addr 172.21.15.121:0/71979654 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:56.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.787+0000 7f82027fc700 1 --1- 172.21.15.121:0/71979654 >> v1:172.21.15.190:6789/0 conn(0x7f8204065ad0 0x7f8204065f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:50232/0 (socket says 172.21.15.121:50232) 2023-11-15T03:11:56.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.787+0000 7f8201ffb700 1 -- 172.21.15.121:0/71979654 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1102424708 0 0) 0x7f82040ab660 con 0x7f8204065ad0 2023-11-15T03:11:56.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.787+0000 7f8201ffb700 1 -- 172.21.15.121:0/71979654 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81ec004500 con 0x7f8204065ad0 2023-11-15T03:11:56.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.787+0000 7f82037fe700 1 -- 172.21.15.121:0/71979654 >> v1:172.21.15.190:6789/0 conn(0x7f8204065ad0 legacy=0x7f8204065f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:56.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.787+0000 7f8201ffb700 1 -- 172.21.15.121:0/71979654 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509019128 0 0) 0x7f82040a98d0 con 0x7f82041500d0 2023-11-15T03:11:56.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.787+0000 7f82037fe700 1 -- 172.21.15.121:0/71979654 >> v1:172.21.15.121:6789/0 conn(0x7f82041500d0 legacy=0x7f82041504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:56.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.787+0000 7f82037fe700 1 -- 172.21.15.121:0/71979654 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82040a9e50 con 0x7f8204066820 2023-11-15T03:11:56.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.787+0000 7f82037fe700 1 --2- 172.21.15.121:0/71979654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8204066820 0x7f8204066c90 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 rx=0x7f81fc009e60 tx=0x7f81fc009780).ready entity=mon.0 client_cookie=e05cce11d84a3dd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:56.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.787+0000 7f8201ffb700 1 -- 172.21.15.121:0/71979654 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc00e070 con 0x7f8204066820 2023-11-15T03:11:56.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.787+0000 7f8201ffb700 1 -- 172.21.15.121:0/71979654 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc00bb60 con 0x7f8204066820 2023-11-15T03:11:56.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.787+0000 7f8201ffb700 1 -- 172.21.15.121:0/71979654 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc00bee0 con 0x7f8204066820 2023-11-15T03:11:56.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.791+0000 7f82092f1700 1 -- 172.21.15.121:0/71979654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8204066820 msgr2=0x7f8204066c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:56.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.791+0000 7f82092f1700 1 --2- 172.21.15.121:0/71979654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8204066820 0x7f8204066c90 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 rx=0x7f81fc009e60 tx=0x7f81fc009780).stop 2023-11-15T03:11:56.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.791+0000 7f82092f1700 1 -- 172.21.15.121:0/71979654 shutdown_connections 2023-11-15T03:11:56.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.791+0000 7f82092f1700 1 --2- 172.21.15.121:0/71979654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8204066820 0x7f8204066c90 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:56.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.791+0000 7f82092f1700 1 -- 172.21.15.121:0/71979654 >> 172.21.15.121:0/71979654 conn(0x7f8204067a60 msgr2=0x7f8204067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:56.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.791+0000 7f82092f1700 1 -- 172.21.15.121:0/71979654 shutdown_connections 2023-11-15T03:11:56.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.791+0000 7f82092f1700 1 -- 172.21.15.121:0/71979654 wait complete. 2023-11-15T03:11:56.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f82092f1700 1 Processor -- start 2023-11-15T03:11:56.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f82092f1700 1 -- start start 2023-11-15T03:11:56.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f82092f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8204065ad0 0x7f820414c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:56.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f82092f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8204066820 0x7f8204143010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:56.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f82092f1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82041500d0 0x7f82040c0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:56.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f82092f1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820414bc60 con 0x7f82041500d0 2023-11-15T03:11:56.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f82092f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82040c1100 con 0x7f8204066820 2023-11-15T03:11:56.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f82092f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820414c080 con 0x7f8204065ad0 2023-11-15T03:11:56.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f82037fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82041500d0 0x7f82040c0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:56.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f82037fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82041500d0 0x7f82040c0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54154/0 (socket says 172.21.15.121:54154) 2023-11-15T03:11:56.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f82037fe700 1 -- 172.21.15.121:0/1780882046 learned_addr learned my addr 172.21.15.121:0/1780882046 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:56.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f8202ffd700 1 --2- 172.21.15.121:0/1780882046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8204065ad0 0x7f820414c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:56.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f82037fe700 1 -- 172.21.15.121:0/1780882046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8204065ad0 msgr2=0x7f820414c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:56.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f82027fc700 1 --2- 172.21.15.121:0/1780882046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8204066820 0x7f8204143010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:56.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f82037fe700 1 --2- 172.21.15.121:0/1780882046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8204065ad0 0x7f820414c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:56.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f82037fe700 1 -- 172.21.15.121:0/1780882046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8204066820 msgr2=0x7f8204143010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:56.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f82037fe700 1 --2- 172.21.15.121:0/1780882046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8204066820 0x7f8204143010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:56.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f82037fe700 1 -- 172.21.15.121:0/1780882046 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc009a20 con 0x7f82041500d0 2023-11-15T03:11:56.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f8202ffd700 1 --2- 172.21.15.121:0/1780882046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8204065ad0 0x7f820414c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:11:56.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f82027fc700 1 --2- 172.21.15.121:0/1780882046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8204066820 0x7f8204143010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:11:56.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.795+0000 7f82037fe700 1 --2- 172.21.15.121:0/1780882046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82041500d0 0x7f82040c0c10 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 rx=0x7f81fc00c000 tx=0x7f81fc009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:56.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.799+0000 7f81eaffd700 1 -- 172.21.15.121:0/1780882046 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc01c660 con 0x7f82041500d0 2023-11-15T03:11:56.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.799+0000 7f81eaffd700 1 -- 172.21.15.121:0/1780882046 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc01c7c0 con 0x7f82041500d0 2023-11-15T03:11:56.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.799+0000 7f82092f1700 1 -- 172.21.15.121:0/1780882046 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820414cfa0 con 0x7f82041500d0 2023-11-15T03:11:56.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.799+0000 7f81eaffd700 1 -- 172.21.15.121:0/1780882046 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc01cb40 con 0x7f82041500d0 2023-11-15T03:11:56.813 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.799+0000 7f82092f1700 1 -- 172.21.15.121:0/1780882046 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820414b5a0 con 0x7f82041500d0 2023-11-15T03:11:56.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.799+0000 7f82092f1700 1 -- 172.21.15.121:0/1780882046 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8204047060 con 0x7f82041500d0 2023-11-15T03:11:56.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.799+0000 7f81eaffd700 1 -- 172.21.15.121:0/1780882046 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f81fc01d420 con 0x7f82041500d0 2023-11-15T03:11:56.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.803+0000 7f81eaffd700 1 --2- 172.21.15.121:0/1780882046 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f81f005a510 0x7f81f005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:56.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.803+0000 7f81eaffd700 1 -- 172.21.15.121:0/1780882046 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f81fc076440 con 0x7f82041500d0 2023-11-15T03:11:56.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.803+0000 7f8202ffd700 1 --2- 172.21.15.121:0/1780882046 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f81f005a510 0x7f81f005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:56.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.803+0000 7f8202ffd700 1 --2- 172.21.15.121:0/1780882046 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f81f005a510 0x7f81f005c9d0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7f81f40016f0 tx=0x7f81f4009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:56.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.807+0000 7f81eaffd700 1 -- 172.21.15.121:0/1780882046 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f81fc049e60 con 0x7f82041500d0 2023-11-15T03:11:56.983 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.975+0000 7f82092f1700 1 -- 172.21.15.121:0/1780882046 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f8204063bc0 con 0x7f81f005a510 2023-11-15T03:11:56.985 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.975+0000 7f81eaffd700 1 -- 172.21.15.121:0/1780882046 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8204063bc0 con 0x7f81f005a510 2023-11-15T03:11:56.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.979+0000 7f82092f1700 1 -- 172.21.15.121:0/1780882046 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f81f005a510 msgr2=0x7f81f005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:56.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.979+0000 7f82092f1700 1 --2- 172.21.15.121:0/1780882046 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f81f005a510 0x7f81f005c9d0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7f81f40016f0 tx=0x7f81f4009040).stop 2023-11-15T03:11:56.989 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.979+0000 7f82092f1700 1 -- 172.21.15.121:0/1780882046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82041500d0 msgr2=0x7f82040c0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:56.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.979+0000 7f82092f1700 1 --2- 172.21.15.121:0/1780882046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82041500d0 0x7f82040c0c10 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 rx=0x7f81fc00c000 tx=0x7f81fc009780).stop 2023-11-15T03:11:56.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.979+0000 7f82092f1700 1 -- 172.21.15.121:0/1780882046 shutdown_connections 2023-11-15T03:11:56.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.979+0000 7f82092f1700 1 --2- 172.21.15.121:0/1780882046 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f81f005a510 0x7f81f005c9d0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:56.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.979+0000 7f82092f1700 1 --2- 172.21.15.121:0/1780882046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82041500d0 0x7f82040c0c10 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:56.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.979+0000 7f82092f1700 1 --2- 172.21.15.121:0/1780882046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8204066820 0x7f8204143010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:56.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.979+0000 7f82092f1700 1 --2- 172.21.15.121:0/1780882046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8204065ad0 0x7f820414c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:56.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.979+0000 7f82092f1700 1 -- 172.21.15.121:0/1780882046 >> 172.21.15.121:0/1780882046 conn(0x7f8204067a60 msgr2=0x7f8204146530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:56.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.979+0000 7f82092f1700 1 -- 172.21.15.121:0/1780882046 shutdown_connections 2023-11-15T03:11:56.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:56.979+0000 7f82092f1700 1 -- 172.21.15.121:0/1780882046 wait complete. 2023-11-15T03:11:56.992 INFO:teuthology.orchestra.run.smithi121.stderr:instructed osd(s) 0 to deep-scrub 2023-11-15T03:11:57.011 INFO:tasks.ceph:Scrubbing osd.1 2023-11-15T03:11:57.011 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-15T03:11:57.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.167+0000 7f52b2707700 1 Processor -- start 2023-11-15T03:11:57.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.167+0000 7f52b2707700 1 -- start start 2023-11-15T03:11:57.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.171+0000 7f52b2707700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f52ac0c5a00 0x7f52ac158260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.171+0000 7f52b2707700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f52ac0c7a90 0x7f52ac154b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.171+0000 7f52b2707700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52ac0c6170 con 0x7f52ac0a9f80 2023-11-15T03:11:57.181 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.171+0000 7f52b2707700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52ac154000 con 0x7f52ac0c7a90 2023-11-15T03:11:57.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.171+0000 7f52b2707700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52ac157dd0 con 0x7f52ac0c5a00 2023-11-15T03:11:57.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.171+0000 7f52b1705700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f52ac0a9f80 0x7f52ac0c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51084/0 (socket says 172.21.15.121:51084) 2023-11-15T03:11:57.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.171+0000 7f52b1705700 1 -- 172.21.15.121:0/2082172090 learned_addr learned my addr 172.21.15.121:0/2082172090 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:57.182 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.171+0000 7f52b1f06700 1 --2- 172.21.15.121:0/2082172090 >> v2:172.21.15.121:3300/0 conn(0x7f52ac0c7a90 0x7f52ac154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.171+0000 7f52b0f04700 1 --2- 172.21.15.121:0/2082172090 >> v2:172.21.15.190:3300/0 conn(0x7f52ac0c5a00 0x7f52ac158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.171+0000 7f52a3fff700 1 -- 172.21.15.121:0/2082172090 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888583517 0 0) 0x7f52ac0c6170 con 0x7f52ac0a9f80 2023-11-15T03:11:57.185 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.171+0000 7f52a3fff700 1 -- 172.21.15.121:0/2082172090 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52980033f0 con 0x7f52ac0a9f80 2023-11-15T03:11:57.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.171+0000 7f52b1f06700 1 -- 172.21.15.121:0/2082172090 >> v2:172.21.15.190:3300/0 conn(0x7f52ac0c5a00 msgr2=0x7f52ac158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:57.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.171+0000 7f52b1f06700 1 --2- 172.21.15.121:0/2082172090 >> v2:172.21.15.190:3300/0 conn(0x7f52ac0c5a00 0x7f52ac158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.171+0000 7f52b1f06700 1 -- 172.21.15.121:0/2082172090 >> v1:172.21.15.121:6789/0 conn(0x7f52ac0a9f80 legacy=0x7f52ac0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.171+0000 7f52b1f06700 1 -- 172.21.15.121:0/2082172090 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac155080 con 0x7f52ac0c7a90 2023-11-15T03:11:57.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.171+0000 7f52b1f06700 1 --2- 172.21.15.121:0/2082172090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52ac0c7a90 0x7f52ac154b90 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 rx=0x7f52ac0aa860 tx=0x7f52a4009780).ready entity=mon.0 client_cookie=3a3417a6fb2a0a46 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:57.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.175+0000 7f52a3fff700 1 -- 172.21.15.121:0/2082172090 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a400e070 con 0x7f52ac0c7a90 2023-11-15T03:11:57.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.175+0000 7f52a3fff700 1 -- 172.21.15.121:0/2082172090 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a400bab0 con 0x7f52ac0c7a90 2023-11-15T03:11:57.187 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.175+0000 7f52a3fff700 1 -- 172.21.15.121:0/2082172090 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a400be30 con 0x7f52ac0c7a90 2023-11-15T03:11:57.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.175+0000 7f52b2707700 1 -- 172.21.15.121:0/2082172090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52ac0c7a90 msgr2=0x7f52ac154b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.175+0000 7f52b2707700 1 --2- 172.21.15.121:0/2082172090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52ac0c7a90 0x7f52ac154b90 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 rx=0x7f52ac0aa860 tx=0x7f52a4009780).stop 2023-11-15T03:11:57.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.175+0000 7f52b2707700 1 -- 172.21.15.121:0/2082172090 shutdown_connections 2023-11-15T03:11:57.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.175+0000 7f52b2707700 1 --2- 172.21.15.121:0/2082172090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52ac0c7a90 0x7f52ac154b90 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.188 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.175+0000 7f52b2707700 1 --2- 172.21.15.121:0/2082172090 >> v2:172.21.15.190:3300/0 conn(0x7f52ac0c5a00 0x7f52ac158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.175+0000 7f52b2707700 1 -- 172.21.15.121:0/2082172090 >> 172.21.15.121:0/2082172090 conn(0x7f52ac067a60 msgr2=0x7f52ac067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:57.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.175+0000 7f52b2707700 1 -- 172.21.15.121:0/2082172090 shutdown_connections 2023-11-15T03:11:57.189 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.175+0000 7f52b2707700 1 -- 172.21.15.121:0/2082172090 wait complete. 2023-11-15T03:11:57.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.175+0000 7f52b2707700 1 Processor -- start 2023-11-15T03:11:57.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.175+0000 7f52b2707700 1 -- start start 2023-11-15T03:11:57.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b2707700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52ac0a9f80 0x7f52ac0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b2707700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52ac0c7a90 0x7f52ac149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b2707700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52ac149c00 0x7f52ac065710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b2707700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac0c99e0 con 0x7f52ac149c00 2023-11-15T03:11:57.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b2707700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac0c9cc0 con 0x7f52ac0a9f80 2023-11-15T03:11:57.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b2707700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac0c9fa0 con 0x7f52ac0c7a90 2023-11-15T03:11:57.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b1f06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52ac149c00 0x7f52ac065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b1705700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52ac0a9f80 0x7f52ac0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b1f06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52ac149c00 0x7f52ac065710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54180/0 (socket says 172.21.15.121:54180) 2023-11-15T03:11:57.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b1f06700 1 -- 172.21.15.121:0/3523416083 learned_addr learned my addr 172.21.15.121:0/3523416083 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:57.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b0f04700 1 --2- 172.21.15.121:0/3523416083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52ac0c7a90 0x7f52ac149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b1f06700 1 -- 172.21.15.121:0/3523416083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52ac0c7a90 msgr2=0x7f52ac149710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.194 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b1f06700 1 --2- 172.21.15.121:0/3523416083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52ac0c7a90 0x7f52ac149710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b1f06700 1 -- 172.21.15.121:0/3523416083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52ac0a9f80 msgr2=0x7f52ac0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b1f06700 1 --2- 172.21.15.121:0/3523416083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52ac0a9f80 0x7f52ac0c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b1f06700 1 -- 172.21.15.121:0/3523416083 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a4009a20 con 0x7f52ac149c00 2023-11-15T03:11:57.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b1705700 1 --2- 172.21.15.121:0/3523416083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52ac0a9f80 0x7f52ac0c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:57.195 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b0f04700 1 --2- 172.21.15.121:0/3523416083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52ac0c7a90 0x7f52ac149710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:57.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b1f06700 1 --2- 172.21.15.121:0/3523416083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52ac149c00 0x7f52ac065710 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 rx=0x7f52ac064b60 tx=0x7f52ac157d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:57.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52a17fa700 1 -- 172.21.15.121:0/3523416083 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a401d630 con 0x7f52ac149c00 2023-11-15T03:11:57.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52a17fa700 1 -- 172.21.15.121:0/3523416083 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a401d790 con 0x7f52ac149c00 2023-11-15T03:11:57.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b2707700 1 -- 172.21.15.121:0/3523416083 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ac0ca280 con 0x7f52ac149c00 2023-11-15T03:11:57.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52a17fa700 1 -- 172.21.15.121:0/3523416083 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a401db10 con 0x7f52ac149c00 2023-11-15T03:11:57.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.179+0000 7f52b2707700 1 -- 172.21.15.121:0/3523416083 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ac0c95f0 con 0x7f52ac149c00 2023-11-15T03:11:57.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.183+0000 7f52b2707700 1 -- 172.21.15.121:0/3523416083 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f52ac0ade30 con 0x7f52ac149c00 2023-11-15T03:11:57.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.183+0000 7f52a17fa700 1 -- 172.21.15.121:0/3523416083 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f52a4027460 con 0x7f52ac149c00 2023-11-15T03:11:57.198 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.183+0000 7f52a17fa700 1 --2- 172.21.15.121:0/3523416083 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f529005a540 0x7f529005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.187+0000 7f52a17fa700 1 -- 172.21.15.121:0/3523416083 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f52a4076520 con 0x7f52ac149c00 2023-11-15T03:11:57.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.187+0000 7f52b1705700 1 --2- 172.21.15.121:0/3523416083 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f529005a540 0x7f529005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.187+0000 7f52a17fa700 1 --2- 172.21.15.121:0/3523416083 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7f5290060bd0 0x7f5290063030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.187+0000 7f52a17fa700 1 -- 172.21.15.121:0/3523416083 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52900636b0 con 0x7f5290060bd0 2023-11-15T03:11:57.199 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.187+0000 7f52a17fa700 1 -- 172.21.15.121:0/3523416083 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52a4078a80 con 0x7f52ac149c00 2023-11-15T03:11:57.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.187+0000 7f52b0f04700 1 --2- 172.21.15.121:0/3523416083 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7f5290060bd0 0x7f5290063030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.187+0000 7f52b1705700 1 --2- 172.21.15.121:0/3523416083 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f529005a540 0x7f529005ca00 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7f52a8005750 tx=0x7f52a8009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:57.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.187+0000 7f52b0f04700 1 --2- 172.21.15.121:0/3523416083 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7f5290060bd0 0x7f5290063030 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:57.200 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.187+0000 7f52a17fa700 1 -- 172.21.15.121:0/3523416083 <== osd.1 v2:172.21.15.121:6800/33131 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f52900636b0 con 0x7f5290060bd0 2023-11-15T03:11:57.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.203+0000 7f52b2707700 1 -- 172.21.15.121:0/3523416083 --> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f52ac0adfa0 con 0x7f5290060bd0 2023-11-15T03:11:57.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.207+0000 7f52a17fa700 1 -- 172.21.15.121:0/3523416083 <== osd.1 v2:172.21.15.121:6800/33131 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f52ac0adfa0 con 0x7f5290060bd0 2023-11-15T03:11:57.216 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:11:57.216 INFO:teuthology.orchestra.run.smithi121.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-15T03:11:57.216 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:11:57.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.207+0000 7f52b2707700 1 -- 172.21.15.121:0/3523416083 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7f5290060bd0 msgr2=0x7f5290063030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.207+0000 7f52b2707700 1 --2- 172.21.15.121:0/3523416083 >> [v2:172.21.15.121:6800/33131,v1:172.21.15.121:6801/33131] conn(0x7f5290060bd0 0x7f5290063030 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.207+0000 7f52b2707700 1 -- 172.21.15.121:0/3523416083 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f529005a540 msgr2=0x7f529005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.207+0000 7f52b2707700 1 --2- 172.21.15.121:0/3523416083 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f529005a540 0x7f529005ca00 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7f52a8005750 tx=0x7f52a8009040).stop 2023-11-15T03:11:57.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.207+0000 7f52b2707700 1 -- 172.21.15.121:0/3523416083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52ac149c00 msgr2=0x7f52ac065710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.207+0000 7f52b2707700 1 --2- 172.21.15.121:0/3523416083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f52ac149c00 0x7f52ac065710 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 rx=0x7f52ac064b60 tx=0x7f52ac157d40).stop 2023-11-15T03:11:57.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.207+0000 7f52b1f06700 1 -- 172.21.15.121:0/3523416083 reap_dead start 2023-11-15T03:11:57.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.207+0000 7f52b2707700 1 -- 172.21.15.121:0/3523416083 shutdown_connections 2023-11-15T03:11:57.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.207+0000 7f52b2707700 1 -- 172.21.15.121:0/3523416083 >> 172.21.15.121:0/3523416083 conn(0x7f52ac067a60 msgr2=0x7f52ac1516f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:57.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.207+0000 7f52b2707700 1 -- 172.21.15.121:0/3523416083 shutdown_connections 2023-11-15T03:11:57.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.207+0000 7f52b2707700 1 -- 172.21.15.121:0/3523416083 wait complete. 2023-11-15T03:11:57.239 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-15T03:11:57.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.399+0000 7fb7f1579700 1 Processor -- start 2023-11-15T03:11:57.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.399+0000 7fb7f1579700 1 -- start start 2023-11-15T03:11:57.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.399+0000 7fb7f1579700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb7ec0ad1b0 0x7fb7ec065710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.399+0000 7fb7f1579700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7ec065c00 0x7fb7ec0add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.399+0000 7fb7f1579700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7ec066c90 con 0x7fb7ec0ab0d0 2023-11-15T03:11:57.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.399+0000 7fb7f1579700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7ec066080 con 0x7fb7ec0ad1b0 2023-11-15T03:11:57.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.399+0000 7fb7f1579700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec066360 con 0x7fb7ec065c00 2023-11-15T03:11:57.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.399+0000 7fb7ebfff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb7ec0ab0d0 0x7fb7ec0ab4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51094/0 (socket says 172.21.15.121:51094) 2023-11-15T03:11:57.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.399+0000 7fb7ebfff700 1 -- 172.21.15.121:0/741381926 learned_addr learned my addr 172.21.15.121:0/741381926 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:57.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.399+0000 7fb7eb7fe700 1 --2- 172.21.15.121:0/741381926 >> v2:172.21.15.121:3300/0 conn(0x7fb7ec0ad1b0 0x7fb7ec065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.399+0000 7fb7f0d78700 1 --2- 172.21.15.121:0/741381926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7ec065c00 0x7fb7ec0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.403+0000 7fb7eaffd700 1 -- 172.21.15.121:0/741381926 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3303481519 0 0) 0x7fb7ec066c90 con 0x7fb7ec0ab0d0 2023-11-15T03:11:57.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.403+0000 7fb7eaffd700 1 -- 172.21.15.121:0/741381926 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7d40033f0 con 0x7fb7ec0ab0d0 2023-11-15T03:11:57.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.403+0000 7fb7f0d78700 1 -- 172.21.15.121:0/741381926 >> v2:172.21.15.121:3300/0 conn(0x7fb7ec0ad1b0 msgr2=0x7fb7ec065710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:57.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.403+0000 7fb7f0d78700 1 --2- 172.21.15.121:0/741381926 >> v2:172.21.15.121:3300/0 conn(0x7fb7ec0ad1b0 0x7fb7ec065710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.403+0000 7fb7f0d78700 1 -- 172.21.15.121:0/741381926 >> v1:172.21.15.121:6789/0 conn(0x7fb7ec0ab0d0 legacy=0x7fb7ec0ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.403+0000 7fb7f0d78700 1 -- 172.21.15.121:0/741381926 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec066640 con 0x7fb7ec065c00 2023-11-15T03:11:57.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.403+0000 7fb7eb7fe700 1 --2- 172.21.15.121:0/741381926 >> v2:172.21.15.121:3300/0 conn(0x7fb7ec0ad1b0 0x7fb7ec065710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:57.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.403+0000 7fb7f0d78700 1 --2- 172.21.15.121:0/741381926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7ec065c00 0x7fb7ec0add40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fb7e4009e60 tx=0x7fb7e4009780).ready entity=mon.2 client_cookie=3bca2f0cbe860163 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:57.415 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.403+0000 7fb7eaffd700 1 -- 172.21.15.121:0/741381926 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e400e070 con 0x7fb7ec065c00 2023-11-15T03:11:57.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.403+0000 7fb7eaffd700 1 -- 172.21.15.121:0/741381926 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e400bab0 con 0x7fb7ec065c00 2023-11-15T03:11:57.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.403+0000 7fb7eaffd700 1 -- 172.21.15.121:0/741381926 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e400be30 con 0x7fb7ec065c00 2023-11-15T03:11:57.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.403+0000 7fb7f1579700 1 -- 172.21.15.121:0/741381926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7ec065c00 msgr2=0x7fb7ec0add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.403+0000 7fb7f1579700 1 --2- 172.21.15.121:0/741381926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7ec065c00 0x7fb7ec0add40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fb7e4009e60 tx=0x7fb7e4009780).stop 2023-11-15T03:11:57.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.403+0000 7fb7f1579700 1 -- 172.21.15.121:0/741381926 shutdown_connections 2023-11-15T03:11:57.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.403+0000 7fb7f1579700 1 --2- 172.21.15.121:0/741381926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7ec065c00 0x7fb7ec0add40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.403+0000 7fb7f1579700 1 --2- 172.21.15.121:0/741381926 >> v2:172.21.15.121:3300/0 conn(0x7fb7ec0ad1b0 0x7fb7ec065710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.403+0000 7fb7f1579700 1 -- 172.21.15.121:0/741381926 >> 172.21.15.121:0/741381926 conn(0x7fb7ec067a60 msgr2=0x7fb7ec067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:57.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.403+0000 7fb7f1579700 1 -- 172.21.15.121:0/741381926 shutdown_connections 2023-11-15T03:11:57.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.403+0000 7fb7f1579700 1 -- 172.21.15.121:0/741381926 wait complete. 2023-11-15T03:11:57.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.407+0000 7fb7f1579700 1 Processor -- start 2023-11-15T03:11:57.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.407+0000 7fb7f1579700 1 -- start start 2023-11-15T03:11:57.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.407+0000 7fb7f1579700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7ec065c00 0x7fb7ec0a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.407+0000 7fb7f1579700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7ec0ab0d0 0x7fb7ec0aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.407+0000 7fb7f1579700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7ec0ad1b0 0x7fb7ec0af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.407+0000 7fb7f1579700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec0af7a0 con 0x7fb7ec0ad1b0 2023-11-15T03:11:57.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.407+0000 7fb7f1579700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec0847b0 con 0x7fb7ec065c00 2023-11-15T03:11:57.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.407+0000 7fb7f1579700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec083ba0 con 0x7fb7ec0ab0d0 2023-11-15T03:11:57.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.407+0000 7fb7f0d78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7ec0ad1b0 0x7fb7ec0af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.407+0000 7fb7f0d78700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7ec0ad1b0 0x7fb7ec0af2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54208/0 (socket says 172.21.15.121:54208) 2023-11-15T03:11:57.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.407+0000 7fb7f0d78700 1 -- 172.21.15.121:0/663999712 learned_addr learned my addr 172.21.15.121:0/663999712 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:57.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.411+0000 7fb7ebfff700 1 --2- 172.21.15.121:0/663999712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7ec065c00 0x7fb7ec0a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.411+0000 7fb7eb7fe700 1 --2- 172.21.15.121:0/663999712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7ec0ab0d0 0x7fb7ec0aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.411+0000 7fb7f0d78700 1 -- 172.21.15.121:0/663999712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7ec0ab0d0 msgr2=0x7fb7ec0aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.411+0000 7fb7f0d78700 1 --2- 172.21.15.121:0/663999712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7ec0ab0d0 0x7fb7ec0aedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.411+0000 7fb7f0d78700 1 -- 172.21.15.121:0/663999712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7ec065c00 msgr2=0x7fb7ec0a4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.411+0000 7fb7f0d78700 1 --2- 172.21.15.121:0/663999712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7ec065c00 0x7fb7ec0a4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.411+0000 7fb7f0d78700 1 -- 172.21.15.121:0/663999712 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e00092a0 con 0x7fb7ec0ad1b0 2023-11-15T03:11:57.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.411+0000 7fb7ebfff700 1 --2- 172.21.15.121:0/663999712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7ec065c00 0x7fb7ec0a4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:57.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.411+0000 7fb7f0d78700 1 --2- 172.21.15.121:0/663999712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7ec0ad1b0 0x7fb7ec0af2b0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 rx=0x7fb7ec0664a0 tx=0x7fb7e4009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:57.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.411+0000 7fb7d3fff700 1 -- 172.21.15.121:0/663999712 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e401d630 con 0x7fb7ec0ad1b0 2023-11-15T03:11:57.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.411+0000 7fb7d3fff700 1 -- 172.21.15.121:0/663999712 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e401d790 con 0x7fb7ec0ad1b0 2023-11-15T03:11:57.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.411+0000 7fb7f1579700 1 -- 172.21.15.121:0/663999712 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e4009a20 con 0x7fb7ec0ad1b0 2023-11-15T03:11:57.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.411+0000 7fb7d3fff700 1 -- 172.21.15.121:0/663999712 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e401db50 con 0x7fb7ec0ad1b0 2023-11-15T03:11:57.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.411+0000 7fb7f1579700 1 -- 172.21.15.121:0/663999712 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7ec084160 con 0x7fb7ec0ad1b0 2023-11-15T03:11:57.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.411+0000 7fb7f1579700 1 -- 172.21.15.121:0/663999712 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7ec0ab4b0 con 0x7fb7ec0ad1b0 2023-11-15T03:11:57.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.415+0000 7fb7d3fff700 1 -- 172.21.15.121:0/663999712 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb7e4003040 con 0x7fb7ec0ad1b0 2023-11-15T03:11:57.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.415+0000 7fb7d3fff700 1 --2- 172.21.15.121:0/663999712 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb7d805a540 0x7fb7d805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.415+0000 7fb7d3fff700 1 -- 172.21.15.121:0/663999712 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7fb7e4076350 con 0x7fb7ec0ad1b0 2023-11-15T03:11:57.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.415+0000 7fb7ebfff700 1 --2- 172.21.15.121:0/663999712 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb7d805a540 0x7fb7d805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.415+0000 7fb7ebfff700 1 --2- 172.21.15.121:0/663999712 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb7d805a540 0x7fb7d805ca00 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7fb7dc001360 tx=0x7fb7dc009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:57.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.419+0000 7fb7d3fff700 1 -- 172.21.15.121:0/663999712 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb7ec0ab4b0 con 0x7fb7ec0ad1b0 2023-11-15T03:11:57.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.583+0000 7fb7f1579700 1 -- 172.21.15.121:0/663999712 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7ec0ab4b0 con 0x7fb7d805a540 2023-11-15T03:11:57.592 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.583+0000 7fb7d3fff700 1 -- 172.21.15.121:0/663999712 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb7ec0ab4b0 con 0x7fb7d805a540 2023-11-15T03:11:57.594 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.583+0000 7fb7f1579700 1 -- 172.21.15.121:0/663999712 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb7d805a540 msgr2=0x7fb7d805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.583+0000 7fb7f1579700 1 --2- 172.21.15.121:0/663999712 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb7d805a540 0x7fb7d805ca00 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7fb7dc001360 tx=0x7fb7dc009040).stop 2023-11-15T03:11:57.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.583+0000 7fb7f1579700 1 -- 172.21.15.121:0/663999712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7ec0ad1b0 msgr2=0x7fb7ec0af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.595 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.583+0000 7fb7f1579700 1 --2- 172.21.15.121:0/663999712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7ec0ad1b0 0x7fb7ec0af2b0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 rx=0x7fb7ec0664a0 tx=0x7fb7e4009780).stop 2023-11-15T03:11:57.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.587+0000 7fb7f1579700 1 -- 172.21.15.121:0/663999712 shutdown_connections 2023-11-15T03:11:57.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.587+0000 7fb7f1579700 1 --2- 172.21.15.121:0/663999712 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb7d805a540 0x7fb7d805ca00 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.587+0000 7fb7f1579700 1 --2- 172.21.15.121:0/663999712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7ec0ad1b0 0x7fb7ec0af2b0 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.587+0000 7fb7f1579700 1 --2- 172.21.15.121:0/663999712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7ec0ab0d0 0x7fb7ec0aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.587+0000 7fb7f1579700 1 --2- 172.21.15.121:0/663999712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7ec065c00 0x7fb7ec0a4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.587+0000 7fb7f1579700 1 -- 172.21.15.121:0/663999712 >> 172.21.15.121:0/663999712 conn(0x7fb7ec067a60 msgr2=0x7fb7ec154a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:57.597 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.587+0000 7fb7f1579700 1 -- 172.21.15.121:0/663999712 shutdown_connections 2023-11-15T03:11:57.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.587+0000 7fb7f1579700 1 -- 172.21.15.121:0/663999712 wait complete. 2023-11-15T03:11:57.598 INFO:teuthology.orchestra.run.smithi121.stderr:instructed osd(s) 1 to deep-scrub 2023-11-15T03:11:57.613 INFO:tasks.ceph:Scrubbing osd.2 2023-11-15T03:11:57.614 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-15T03:11:57.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.771+0000 7f14bea9b700 1 Processor -- start 2023-11-15T03:11:57.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.771+0000 7f14bea9b700 1 -- start start 2023-11-15T03:11:57.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.775+0000 7f14bea9b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f14b81570b0 0x7f14b8066860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.775+0000 7f14bea9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f14b80c5c80 0x7f14b8063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.775+0000 7f14bea9b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b8062da0 con 0x7f14b8157490 2023-11-15T03:11:57.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.775+0000 7f14bea9b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b80bfb90 con 0x7f14b81570b0 2023-11-15T03:11:57.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.775+0000 7f14bea9b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b80c6140 con 0x7f14b80c5c80 2023-11-15T03:11:57.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.775+0000 7f14bc837700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f14b81570b0 0x7f14b8066860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.775+0000 7f14bc837700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f14b81570b0 0x7f14b8066860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54224/0 (socket says 172.21.15.121:54224) 2023-11-15T03:11:57.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.775+0000 7f14bc837700 1 -- 172.21.15.121:0/25492346 learned_addr learned my addr 172.21.15.121:0/25492346 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:57.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.775+0000 7f14bd038700 1 --2- 172.21.15.121:0/25492346 >> v2:172.21.15.190:3300/0 conn(0x7f14b80c5c80 0x7f14b8063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.775+0000 7f14bc837700 1 -- 172.21.15.121:0/25492346 >> v2:172.21.15.190:3300/0 conn(0x7f14b80c5c80 msgr2=0x7f14b8063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:57.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.775+0000 7f14bc837700 1 --2- 172.21.15.121:0/25492346 >> v2:172.21.15.190:3300/0 conn(0x7f14b80c5c80 0x7f14b8063230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.775+0000 7f14b77fe700 1 -- 172.21.15.121:0/25492346 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929921177 0 0) 0x7f14b8062da0 con 0x7f14b8157490 2023-11-15T03:11:57.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.775+0000 7f14bc837700 1 -- 172.21.15.121:0/25492346 >> v1:172.21.15.190:6789/0 conn(0x7f14b8157490 legacy=0x7f14b8065950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.775+0000 7f14bc837700 1 -- 172.21.15.121:0/25492346 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b80c6420 con 0x7f14b81570b0 2023-11-15T03:11:57.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.775+0000 7f14bc837700 1 --2- 172.21.15.121:0/25492346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14b81570b0 0x7f14b8066860 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 rx=0x7f14b80653e0 tx=0x7f14ac0090f0).ready entity=mon.0 client_cookie=b95f2bedc9004822 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:57.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.775+0000 7f14b77fe700 1 -- 172.21.15.121:0/25492346 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00b600 con 0x7f14b81570b0 2023-11-15T03:11:57.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.775+0000 7f14b77fe700 1 -- 172.21.15.121:0/25492346 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac00b760 con 0x7f14b81570b0 2023-11-15T03:11:57.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.775+0000 7f14b77fe700 1 -- 172.21.15.121:0/25492346 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00bae0 con 0x7f14b81570b0 2023-11-15T03:11:57.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.775+0000 7f14bea9b700 1 -- 172.21.15.121:0/25492346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14b81570b0 msgr2=0x7f14b8066860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.779+0000 7f14bea9b700 1 --2- 172.21.15.121:0/25492346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14b81570b0 0x7f14b8066860 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 rx=0x7f14b80653e0 tx=0x7f14ac0090f0).stop 2023-11-15T03:11:57.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.779+0000 7f14bea9b700 1 -- 172.21.15.121:0/25492346 shutdown_connections 2023-11-15T03:11:57.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.779+0000 7f14bea9b700 1 --2- 172.21.15.121:0/25492346 >> v2:172.21.15.190:3300/0 conn(0x7f14b80c5c80 0x7f14b8063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.779+0000 7f14bea9b700 1 --2- 172.21.15.121:0/25492346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14b81570b0 0x7f14b8066860 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.779+0000 7f14bea9b700 1 -- 172.21.15.121:0/25492346 >> 172.21.15.121:0/25492346 conn(0x7f14b80684f0 msgr2=0x7f14b80cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:57.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.779+0000 7f14bea9b700 1 -- 172.21.15.121:0/25492346 shutdown_connections 2023-11-15T03:11:57.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.779+0000 7f14bea9b700 1 -- 172.21.15.121:0/25492346 wait complete. 2023-11-15T03:11:57.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.779+0000 7f14bea9b700 1 Processor -- start 2023-11-15T03:11:57.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.779+0000 7f14bea9b700 1 -- start start 2023-11-15T03:11:57.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14bea9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14b80c5c80 0x7f14b8151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14bea9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14b81570b0 0x7f14b80bfd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14bea9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14b80c0250 0x7f14b80aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14bea9b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b8164f80 con 0x7f14b80c5c80 2023-11-15T03:11:57.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14bea9b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b80a9460 con 0x7f14b80c0250 2023-11-15T03:11:57.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14bea9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b8164090 con 0x7f14b81570b0 2023-11-15T03:11:57.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14bc837700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14b80c5c80 0x7f14b8151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14bc837700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14b80c5c80 0x7f14b8151c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54226/0 (socket says 172.21.15.121:54226) 2023-11-15T03:11:57.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14bc837700 1 -- 172.21.15.121:0/3820814370 learned_addr learned my addr 172.21.15.121:0/3820814370 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:57.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14b7fff700 1 --2- 172.21.15.121:0/3820814370 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14b81570b0 0x7f14b80bfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14bd038700 1 --2- 172.21.15.121:0/3820814370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14b80c0250 0x7f14b80aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14bc837700 1 -- 172.21.15.121:0/3820814370 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14b81570b0 msgr2=0x7f14b80bfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14bc837700 1 --2- 172.21.15.121:0/3820814370 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14b81570b0 0x7f14b80bfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14bc837700 1 -- 172.21.15.121:0/3820814370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14b80c0250 msgr2=0x7f14b80aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14bc837700 1 --2- 172.21.15.121:0/3820814370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14b80c0250 0x7f14b80aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14bc837700 1 -- 172.21.15.121:0/3820814370 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac009370 con 0x7f14b80c5c80 2023-11-15T03:11:57.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14bd038700 1 --2- 172.21.15.121:0/3820814370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14b80c0250 0x7f14b80aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:57.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14bc837700 1 --2- 172.21.15.121:0/3820814370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14b80c5c80 0x7f14b8151c00 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 rx=0x7f14ac009130 tx=0x7f14ac017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:57.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14b4ff9700 1 -- 172.21.15.121:0/3820814370 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00b8d0 con 0x7f14b80c5c80 2023-11-15T03:11:57.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14b4ff9700 1 -- 172.21.15.121:0/3820814370 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac00ba30 con 0x7f14b80c5c80 2023-11-15T03:11:57.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14b4ff9700 1 -- 172.21.15.121:0/3820814370 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac01c670 con 0x7f14b80c5c80 2023-11-15T03:11:57.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.783+0000 7f14bea9b700 1 -- 172.21.15.121:0/3820814370 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b8164370 con 0x7f14b80c5c80 2023-11-15T03:11:57.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.787+0000 7f14bea9b700 1 -- 172.21.15.121:0/3820814370 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b8164930 con 0x7f14b80c5c80 2023-11-15T03:11:57.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.787+0000 7f14bea9b700 1 -- 172.21.15.121:0/3820814370 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f14b80ca490 con 0x7f14b80c5c80 2023-11-15T03:11:57.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.787+0000 7f14b4ff9700 1 -- 172.21.15.121:0/3820814370 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f14ac01ca20 con 0x7f14b80c5c80 2023-11-15T03:11:57.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.791+0000 7f14b4ff9700 1 --2- 172.21.15.121:0/3820814370 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f149c05a510 0x7f149c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.803 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.791+0000 7f14b4ff9700 1 -- 172.21.15.121:0/3820814370 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f14ac0761f0 con 0x7f14b80c5c80 2023-11-15T03:11:57.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.791+0000 7f14b4ff9700 1 --2- 172.21.15.121:0/3820814370 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7f149c060ba0 0x7f149c063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:57.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.791+0000 7f14b4ff9700 1 -- 172.21.15.121:0/3820814370 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f149c063680 con 0x7f149c060ba0 2023-11-15T03:11:57.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.791+0000 7f14b4ff9700 1 -- 172.21.15.121:0/3820814370 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14ac078750 con 0x7f14b80c5c80 2023-11-15T03:11:57.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.791+0000 7f14b7fff700 1 --2- 172.21.15.121:0/3820814370 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f149c05a510 0x7f149c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.791+0000 7f14bd038700 1 --2- 172.21.15.121:0/3820814370 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7f149c060ba0 0x7f149c063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:57.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.791+0000 7f14bd038700 1 --2- 172.21.15.121:0/3820814370 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7f149c060ba0 0x7f149c063000 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:57.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.791+0000 7f14b7fff700 1 --2- 172.21.15.121:0/3820814370 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f149c05a510 0x7f149c05c9d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7f14a4007fd0 tx=0x7f14a4007bb0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:57.806 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.791+0000 7f14b4ff9700 1 -- 172.21.15.121:0/3820814370 <== osd.2 v2:172.21.15.121:6808/33134 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f149c063680 con 0x7f149c060ba0 2023-11-15T03:11:57.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.811+0000 7f14bea9b700 1 -- 172.21.15.121:0/3820814370 --> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f14b80c2920 con 0x7f149c060ba0 2023-11-15T03:11:57.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.811+0000 7f14b4ff9700 1 -- 172.21.15.121:0/3820814370 <== osd.2 v2:172.21.15.121:6808/33134 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f14b80c2920 con 0x7f149c060ba0 2023-11-15T03:11:57.821 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:11:57.821 INFO:teuthology.orchestra.run.smithi121.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-15T03:11:57.822 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:11:57.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.811+0000 7f14bea9b700 1 -- 172.21.15.121:0/3820814370 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7f149c060ba0 msgr2=0x7f149c063000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.811+0000 7f14bea9b700 1 --2- 172.21.15.121:0/3820814370 >> [v2:172.21.15.121:6808/33134,v1:172.21.15.121:6809/33134] conn(0x7f149c060ba0 0x7f149c063000 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:57.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.811+0000 7f14bea9b700 1 -- 172.21.15.121:0/3820814370 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f149c05a510 msgr2=0x7f149c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.811+0000 7f14bea9b700 1 --2- 172.21.15.121:0/3820814370 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f149c05a510 0x7f149c05c9d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7f14a4007fd0 tx=0x7f14a4007bb0).stop 2023-11-15T03:11:57.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.811+0000 7f14bea9b700 1 -- 172.21.15.121:0/3820814370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14b80c5c80 msgr2=0x7f14b8151c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:57.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.811+0000 7f14bea9b700 1 --2- 172.21.15.121:0/3820814370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14b80c5c80 0x7f14b8151c00 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 rx=0x7f14ac009130 tx=0x7f14ac017000).stop 2023-11-15T03:11:57.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.811+0000 7f14bd038700 1 -- 172.21.15.121:0/3820814370 reap_dead start 2023-11-15T03:11:57.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.811+0000 7f14bea9b700 1 -- 172.21.15.121:0/3820814370 shutdown_connections 2023-11-15T03:11:57.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.811+0000 7f14bea9b700 1 -- 172.21.15.121:0/3820814370 >> 172.21.15.121:0/3820814370 conn(0x7f14b80684f0 msgr2=0x7f14b80b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:57.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.811+0000 7f14bea9b700 1 -- 172.21.15.121:0/3820814370 shutdown_connections 2023-11-15T03:11:57.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.811+0000 7f14bea9b700 1 -- 172.21.15.121:0/3820814370 wait complete. 2023-11-15T03:11:57.843 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-15T03:11:58.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.991+0000 7fcf4a569700 1 Processor -- start 2023-11-15T03:11:58.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.991+0000 7fcf4a569700 1 -- start start 2023-11-15T03:11:58.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.995+0000 7fcf4a569700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf440c04d0 0x7fcf440c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.995+0000 7fcf4a569700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcf44153670 0x7fcf44153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.995+0000 7fcf4a569700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf440c5720 con 0x7fcf440c8fb0 2023-11-15T03:11:58.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.995+0000 7fcf4a569700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf440c0da0 con 0x7fcf44153670 2023-11-15T03:11:58.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.995+0000 7fcf4a569700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf440ca8a0 con 0x7fcf440c04d0 2023-11-15T03:11:58.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.995+0000 7fcf48b06700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fcf440c8fb0 0x7fcf440c9460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51108/0 (socket says 172.21.15.121:51108) 2023-11-15T03:11:58.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.995+0000 7fcf48b06700 1 -- 172.21.15.121:0/1156735668 learned_addr learned my addr 172.21.15.121:0/1156735668 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:58.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.995+0000 7fcf43fff700 1 --2- 172.21.15.121:0/1156735668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf440c04d0 0x7fcf440c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.995+0000 7fcf3bfff700 1 --2- 172.21.15.121:0/1156735668 >> v2:172.21.15.190:3300/0 conn(0x7fcf44153670 0x7fcf44153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.995+0000 7fcf437fe700 1 -- 172.21.15.121:0/1156735668 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3381586397 0 0) 0x7fcf440c5720 con 0x7fcf440c8fb0 2023-11-15T03:11:58.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.995+0000 7fcf437fe700 1 -- 172.21.15.121:0/1156735668 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf300033f0 con 0x7fcf440c8fb0 2023-11-15T03:11:58.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.995+0000 7fcf3bfff700 1 -- 172.21.15.121:0/1156735668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf440c04d0 msgr2=0x7fcf440c08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:58.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.995+0000 7fcf3bfff700 1 --2- 172.21.15.121:0/1156735668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf440c04d0 0x7fcf440c08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.995+0000 7fcf3bfff700 1 -- 172.21.15.121:0/1156735668 >> v1:172.21.15.121:6789/0 conn(0x7fcf440c8fb0 legacy=0x7fcf440c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.995+0000 7fcf3bfff700 1 -- 172.21.15.121:0/1156735668 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf440c9d20 con 0x7fcf44153670 2023-11-15T03:11:58.010 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.995+0000 7fcf3bfff700 1 --2- 172.21.15.121:0/1156735668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf44153670 0x7fcf44153ad0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7fcf2c009760 tx=0x7fcf2c009060).ready entity=mon.1 client_cookie=6ef2170473c3aed9 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:58.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.999+0000 7fcf437fe700 1 -- 172.21.15.121:0/1156735668 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c00d4a0 con 0x7fcf44153670 2023-11-15T03:11:58.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.999+0000 7fcf437fe700 1 -- 172.21.15.121:0/1156735668 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2c00d600 con 0x7fcf44153670 2023-11-15T03:11:58.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.999+0000 7fcf437fe700 1 -- 172.21.15.121:0/1156735668 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c00d980 con 0x7fcf44153670 2023-11-15T03:11:58.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.999+0000 7fcf4a569700 1 -- 172.21.15.121:0/1156735668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf44153670 msgr2=0x7fcf44153ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.014 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.999+0000 7fcf4a569700 1 --2- 172.21.15.121:0/1156735668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf44153670 0x7fcf44153ad0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7fcf2c009760 tx=0x7fcf2c009060).stop 2023-11-15T03:11:58.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.999+0000 7fcf4a569700 1 -- 172.21.15.121:0/1156735668 shutdown_connections 2023-11-15T03:11:58.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.999+0000 7fcf4a569700 1 --2- 172.21.15.121:0/1156735668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf44153670 0x7fcf44153ad0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.999+0000 7fcf4a569700 1 --2- 172.21.15.121:0/1156735668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf440c04d0 0x7fcf440c08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.999+0000 7fcf4a569700 1 -- 172.21.15.121:0/1156735668 >> 172.21.15.121:0/1156735668 conn(0x7fcf44067a60 msgr2=0x7fcf44067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:58.015 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.999+0000 7fcf4a569700 1 -- 172.21.15.121:0/1156735668 shutdown_connections 2023-11-15T03:11:58.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.999+0000 7fcf4a569700 1 -- 172.21.15.121:0/1156735668 wait complete. 2023-11-15T03:11:58.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:57.999+0000 7fcf4a569700 1 Processor -- start 2023-11-15T03:11:58.016 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf4a569700 1 -- start start 2023-11-15T03:11:58.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf4a569700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf440c04d0 0x7fcf44156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf4a569700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf440c8fb0 0x7fcf44146080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf4a569700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf44153670 0x7fcf44153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf4a569700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf440b0a00 con 0x7fcf440c8fb0 2023-11-15T03:11:58.017 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf4a569700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf441554d0 con 0x7fcf440c04d0 2023-11-15T03:11:58.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf4a569700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf441557b0 con 0x7fcf44153670 2023-11-15T03:11:58.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf3bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf440c8fb0 0x7fcf44146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf3bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf440c8fb0 0x7fcf44146080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54236/0 (socket says 172.21.15.121:54236) 2023-11-15T03:11:58.018 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf3bfff700 1 -- 172.21.15.121:0/3393013547 learned_addr learned my addr 172.21.15.121:0/3393013547 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:58.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf43fff700 1 --2- 172.21.15.121:0/3393013547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf440c04d0 0x7fcf44156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf48b06700 1 --2- 172.21.15.121:0/3393013547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf44153670 0x7fcf44153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf3bfff700 1 -- 172.21.15.121:0/3393013547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf44153670 msgr2=0x7fcf44153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.019 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf3bfff700 1 --2- 172.21.15.121:0/3393013547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf44153670 0x7fcf44153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf3bfff700 1 -- 172.21.15.121:0/3393013547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf440c04d0 msgr2=0x7fcf44156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.021 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf3bfff700 1 --2- 172.21.15.121:0/3393013547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf440c04d0 0x7fcf44156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf3bfff700 1 -- 172.21.15.121:0/3393013547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2c0092a0 con 0x7fcf440c8fb0 2023-11-15T03:11:58.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf43fff700 1 --2- 172.21.15.121:0/3393013547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf440c04d0 0x7fcf44156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:58.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf3bfff700 1 --2- 172.21.15.121:0/3393013547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf440c8fb0 0x7fcf44146080 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 rx=0x7fcf44064f40 tx=0x7fcf2c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:58.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf40ff9700 1 -- 172.21.15.121:0/3393013547 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c000c90 con 0x7fcf440c8fb0 2023-11-15T03:11:58.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf40ff9700 1 -- 172.21.15.121:0/3393013547 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2c00d770 con 0x7fcf440c8fb0 2023-11-15T03:11:58.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf4a569700 1 -- 172.21.15.121:0/3393013547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf44155a90 con 0x7fcf440c8fb0 2023-11-15T03:11:58.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf40ff9700 1 -- 172.21.15.121:0/3393013547 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c01d640 con 0x7fcf440c8fb0 2023-11-15T03:11:58.024 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.003+0000 7fcf4a569700 1 -- 172.21.15.121:0/3393013547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf44156780 con 0x7fcf440c8fb0 2023-11-15T03:11:58.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.007+0000 7fcf4a569700 1 -- 172.21.15.121:0/3393013547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf44155f70 con 0x7fcf440c8fb0 2023-11-15T03:11:58.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.007+0000 7fcf40ff9700 1 -- 172.21.15.121:0/3393013547 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fcf2c01da70 con 0x7fcf440c8fb0 2023-11-15T03:11:58.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.011+0000 7fcf40ff9700 1 --2- 172.21.15.121:0/3393013547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcf2405a510 0x7fcf2405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.011+0000 7fcf40ff9700 1 -- 172.21.15.121:0/3393013547 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7fcf2c075e50 con 0x7fcf440c8fb0 2023-11-15T03:11:58.025 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.011+0000 7fcf43fff700 1 --2- 172.21.15.121:0/3393013547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcf2405a510 0x7fcf2405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.011+0000 7fcf43fff700 1 --2- 172.21.15.121:0/3393013547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcf2405a510 0x7fcf2405c9d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7fcf34009c00 tx=0x7fcf34008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:58.026 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.015+0000 7fcf40ff9700 1 -- 172.21.15.121:0/3393013547 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcf2c049810 con 0x7fcf440c8fb0 2023-11-15T03:11:58.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.175+0000 7fcf4a569700 1 -- 172.21.15.121:0/3393013547 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf440ca000 con 0x7fcf2405a510 2023-11-15T03:11:58.186 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.175+0000 7fcf40ff9700 1 -- 172.21.15.121:0/3393013547 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcf440ca000 con 0x7fcf2405a510 2023-11-15T03:11:58.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.179+0000 7fcf4a569700 1 -- 172.21.15.121:0/3393013547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcf2405a510 msgr2=0x7fcf2405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.179+0000 7fcf4a569700 1 --2- 172.21.15.121:0/3393013547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcf2405a510 0x7fcf2405c9d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7fcf34009c00 tx=0x7fcf34008040).stop 2023-11-15T03:11:58.191 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.179+0000 7fcf4a569700 1 -- 172.21.15.121:0/3393013547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf440c8fb0 msgr2=0x7fcf44146080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.179+0000 7fcf4a569700 1 --2- 172.21.15.121:0/3393013547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf440c8fb0 0x7fcf44146080 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 rx=0x7fcf44064f40 tx=0x7fcf2c016000).stop 2023-11-15T03:11:58.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.183+0000 7fcf4a569700 1 -- 172.21.15.121:0/3393013547 shutdown_connections 2023-11-15T03:11:58.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.183+0000 7fcf4a569700 1 --2- 172.21.15.121:0/3393013547 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fcf2405a510 0x7fcf2405c9d0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.183+0000 7fcf4a569700 1 --2- 172.21.15.121:0/3393013547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf44153670 0x7fcf44153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.192 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.183+0000 7fcf4a569700 1 --2- 172.21.15.121:0/3393013547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf440c8fb0 0x7fcf44146080 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.183+0000 7fcf4a569700 1 --2- 172.21.15.121:0/3393013547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf440c04d0 0x7fcf44156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.183+0000 7fcf4a569700 1 -- 172.21.15.121:0/3393013547 >> 172.21.15.121:0/3393013547 conn(0x7fcf44067a60 msgr2=0x7fcf4414d990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:58.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.183+0000 7fcf4a569700 1 -- 172.21.15.121:0/3393013547 shutdown_connections 2023-11-15T03:11:58.193 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.183+0000 7fcf4a569700 1 -- 172.21.15.121:0/3393013547 wait complete. 2023-11-15T03:11:58.194 INFO:teuthology.orchestra.run.smithi121.stderr:instructed osd(s) 2 to deep-scrub 2023-11-15T03:11:58.210 INFO:tasks.ceph:Scrubbing osd.3 2023-11-15T03:11:58.211 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-15T03:11:58.376 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.367+0000 7f9c3e307700 1 Processor -- start 2023-11-15T03:11:58.377 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.367+0000 7f9c3e307700 1 -- start start 2023-11-15T03:11:58.380 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.371+0000 7f9c3e307700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9c380669b0 0x7f9c38158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.371+0000 7f9c3e307700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c380bb910 con 0x7f9c38157d90 2023-11-15T03:11:58.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.371+0000 7f9c3e307700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3815bde0 con 0x7f9c380a8750 2023-11-15T03:11:58.381 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.371+0000 7f9c3e307700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c380c1fb0 con 0x7f9c380669b0 2023-11-15T03:11:58.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.371+0000 7f9c3d305700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9c38157d90 0x7f9c380c1760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51124/0 (socket says 172.21.15.121:51124) 2023-11-15T03:11:58.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.371+0000 7f9c3d305700 1 -- 172.21.15.121:0/10187792 learned_addr learned my addr 172.21.15.121:0/10187792 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:58.383 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.371+0000 7f9c3cb04700 1 --2- 172.21.15.121:0/10187792 >> v2:172.21.15.190:3300/0 conn(0x7f9c380669b0 0x7f9c38158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.371+0000 7f9c2ffff700 1 -- 172.21.15.121:0/10187792 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633478490 0 0) 0x7f9c380bb910 con 0x7f9c38157d90 2023-11-15T03:11:58.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.371+0000 7f9c2ffff700 1 -- 172.21.15.121:0/10187792 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c240033f0 con 0x7f9c38157d90 2023-11-15T03:11:58.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.371+0000 7f9c2ffff700 1 -- 172.21.15.121:0/10187792 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629812348 0 0) 0x7f9c3815bde0 con 0x7f9c380a8750 2023-11-15T03:11:58.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.371+0000 7f9c2ffff700 1 -- 172.21.15.121:0/10187792 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c380bb910 con 0x7f9c380a8750 2023-11-15T03:11:58.384 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.371+0000 7f9c2ffff700 1 -- 172.21.15.121:0/10187792 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2353888266 0 0) 0x7f9c240033f0 con 0x7f9c38157d90 2023-11-15T03:11:58.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.371+0000 7f9c2ffff700 1 -- 172.21.15.121:0/10187792 >> v2:172.21.15.190:3300/0 conn(0x7f9c380669b0 msgr2=0x7f9c38158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:58.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.371+0000 7f9c2ffff700 1 --2- 172.21.15.121:0/10187792 >> v2:172.21.15.190:3300/0 conn(0x7f9c380669b0 0x7f9c38158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.371+0000 7f9c2ffff700 1 -- 172.21.15.121:0/10187792 >> v1:172.21.15.190:6789/0 conn(0x7f9c380a8750 legacy=0x7f9c38065680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.371+0000 7f9c2ffff700 1 -- 172.21.15.121:0/10187792 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3815c0c0 con 0x7f9c38157d90 2023-11-15T03:11:58.385 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.371+0000 7f9c2ffff700 1 -- 172.21.15.121:0/10187792 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f9c34001470 con 0x7f9c38157d90 2023-11-15T03:11:58.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.371+0000 7f9c3cb04700 1 --2- 172.21.15.121:0/10187792 >> v2:172.21.15.190:3300/0 conn(0x7f9c380669b0 0x7f9c38158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:58.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c2ffff700 1 -- 172.21.15.121:0/10187792 >> v1:172.21.15.121:6789/0 conn(0x7f9c38157d90 legacy=0x7f9c380c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c2ffff700 1 --2- 172.21.15.121:0/10187792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c24003a70 0x7f9c24003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c2ffff700 1 --2- 172.21.15.121:0/10187792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c24004690 0x7f9c24004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.386 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c2ffff700 1 --2- 172.21.15.121:0/10187792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c24005000 0x7f9c2400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c2ffff700 1 -- 172.21.15.121:0/10187792 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2400be80 con 0x7f9c24004690 2023-11-15T03:11:58.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c2ffff700 1 -- 172.21.15.121:0/10187792 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2400bfc0 con 0x7f9c24003a70 2023-11-15T03:11:58.387 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c2ffff700 1 -- 172.21.15.121:0/10187792 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2400c180 con 0x7f9c24005000 2023-11-15T03:11:58.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c3cb04700 1 --2- 172.21.15.121:0/10187792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c24004690 0x7f9c24004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c3db06700 1 --2- 172.21.15.121:0/10187792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c24005000 0x7f9c2400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c3d305700 1 --2- 172.21.15.121:0/10187792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c24003a70 0x7f9c24003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.389 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c3cb04700 1 -- 172.21.15.121:0/10187792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c24005000 msgr2=0x7f9c2400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:58.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c3cb04700 1 --2- 172.21.15.121:0/10187792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c24005000 0x7f9c2400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c3cb04700 1 -- 172.21.15.121:0/10187792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c24003a70 msgr2=0x7f9c24003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:58.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c3cb04700 1 --2- 172.21.15.121:0/10187792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c24003a70 0x7f9c24003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c3cb04700 1 -- 172.21.15.121:0/10187792 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c240033f0 con 0x7f9c24004690 2023-11-15T03:11:58.390 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c3db06700 1 -- 172.21.15.121:0/10187792 reap_dead start 2023-11-15T03:11:58.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c3d305700 1 --2- 172.21.15.121:0/10187792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c24003a70 0x7f9c24003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:58.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c3cb04700 1 --2- 172.21.15.121:0/10187792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c24004690 0x7f9c24004b10 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 rx=0x7f9c28006fd0 tx=0x7f9c28006f40).ready entity=mon.0 client_cookie=984e7c1a3033d6ef server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:58.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c2ffff700 1 -- 172.21.15.121:0/10187792 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c28005880 con 0x7f9c24004690 2023-11-15T03:11:58.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c2ffff700 1 -- 172.21.15.121:0/10187792 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c280059e0 con 0x7f9c24004690 2023-11-15T03:11:58.391 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c2ffff700 1 -- 172.21.15.121:0/10187792 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c28005d60 con 0x7f9c24004690 2023-11-15T03:11:58.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c3e307700 1 -- 172.21.15.121:0/10187792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c24004690 msgr2=0x7f9c24004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c3e307700 1 --2- 172.21.15.121:0/10187792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c24004690 0x7f9c24004b10 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 rx=0x7f9c28006fd0 tx=0x7f9c28006f40).stop 2023-11-15T03:11:58.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c3e307700 1 -- 172.21.15.121:0/10187792 shutdown_connections 2023-11-15T03:11:58.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c3e307700 1 --2- 172.21.15.121:0/10187792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c24004690 0x7f9c24004b10 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.392 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c3e307700 1 -- 172.21.15.121:0/10187792 >> 172.21.15.121:0/10187792 conn(0x7f9c38067a60 msgr2=0x7f9c38067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:58.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c3e307700 1 -- 172.21.15.121:0/10187792 shutdown_connections 2023-11-15T03:11:58.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.375+0000 7f9c3e307700 1 -- 172.21.15.121:0/10187792 wait complete. 2023-11-15T03:11:58.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.379+0000 7f9c3e307700 1 Processor -- start 2023-11-15T03:11:58.393 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.379+0000 7f9c3e307700 1 -- start start 2023-11-15T03:11:58.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.379+0000 7f9c3e307700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c24004690 0x7f9c38148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.395 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c3e307700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c380aa7e0 0x7f9c3816c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c3e307700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c380bc820 0x7f9c380bcca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c3e307700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c381694f0 con 0x7f9c380bc820 2023-11-15T03:11:58.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c3e307700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c380bd190 con 0x7f9c380aa7e0 2023-11-15T03:11:58.396 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c3e307700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c38168620 con 0x7f9c24004690 2023-11-15T03:11:58.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c3db06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c380bc820 0x7f9c380bcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c3db06700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c380bc820 0x7f9c380bcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54262/0 (socket says 172.21.15.121:54262) 2023-11-15T03:11:58.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c3db06700 1 -- 172.21.15.121:0/645453621 learned_addr learned my addr 172.21.15.121:0/645453621 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:58.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c3d305700 1 --2- 172.21.15.121:0/645453621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c24004690 0x7f9c38148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c3cb04700 1 --2- 172.21.15.121:0/645453621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c380aa7e0 0x7f9c3816c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c3db06700 1 -- 172.21.15.121:0/645453621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c24004690 msgr2=0x7f9c38148310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c3db06700 1 --2- 172.21.15.121:0/645453621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c24004690 0x7f9c38148310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c3db06700 1 -- 172.21.15.121:0/645453621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c380aa7e0 msgr2=0x7f9c3816c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c3db06700 1 --2- 172.21.15.121:0/645453621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c380aa7e0 0x7f9c3816c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c3db06700 1 -- 172.21.15.121:0/645453621 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c38148800 con 0x7f9c380bc820 2023-11-15T03:11:58.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c3db06700 1 --2- 172.21.15.121:0/645453621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c380bc820 0x7f9c380bcca0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 rx=0x7f9c38068990 tx=0x7f9c30007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:58.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c2d7fa700 1 -- 172.21.15.121:0/645453621 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000eea0 con 0x7f9c380bc820 2023-11-15T03:11:58.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c2d7fa700 1 -- 172.21.15.121:0/645453621 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3000c7e0 con 0x7f9c380bc820 2023-11-15T03:11:58.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c2d7fa700 1 -- 172.21.15.121:0/645453621 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000cb80 con 0x7f9c380bc820 2023-11-15T03:11:58.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c3e307700 1 -- 172.21.15.121:0/645453621 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c381688e0 con 0x7f9c380bc820 2023-11-15T03:11:58.399 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c3e307700 1 -- 172.21.15.121:0/645453621 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c38168ea0 con 0x7f9c380bc820 2023-11-15T03:11:58.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.383+0000 7f9c3e307700 1 -- 172.21.15.121:0/645453621 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9c3806bcf0 con 0x7f9c380bc820 2023-11-15T03:11:58.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.387+0000 7f9c2d7fa700 1 -- 172.21.15.121:0/645453621 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9c300133e0 con 0x7f9c380bc820 2023-11-15T03:11:58.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.387+0000 7f9c2d7fa700 1 --2- 172.21.15.121:0/645453621 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9c1c05a510 0x7f9c1c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.387+0000 7f9c2d7fa700 1 -- 172.21.15.121:0/645453621 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f9c30074020 con 0x7f9c380bc820 2023-11-15T03:11:58.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.387+0000 7f9c3d305700 1 --2- 172.21.15.121:0/645453621 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9c1c05a510 0x7f9c1c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.387+0000 7f9c2d7fa700 1 --2- 172.21.15.121:0/645453621 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7f9c1c060ba0 0x7f9c1c063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.387+0000 7f9c2d7fa700 1 -- 172.21.15.121:0/645453621 --> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9c1c063680 con 0x7f9c1c060ba0 2023-11-15T03:11:58.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.387+0000 7f9c2d7fa700 1 -- 172.21.15.121:0/645453621 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c30076640 con 0x7f9c380bc820 2023-11-15T03:11:58.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.387+0000 7f9c3d305700 1 --2- 172.21.15.121:0/645453621 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9c1c05a510 0x7f9c1c05c9d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f9c34001010 tx=0x7f9c34007040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:58.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.387+0000 7f9c3cb04700 1 --2- 172.21.15.121:0/645453621 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7f9c1c060ba0 0x7f9c1c063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.391+0000 7f9c3cb04700 1 --2- 172.21.15.121:0/645453621 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7f9c1c060ba0 0x7f9c1c063000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:58.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.391+0000 7f9c2d7fa700 1 -- 172.21.15.121:0/645453621 <== osd.3 v2:172.21.15.121:6820/33135 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f9c1c063680 con 0x7f9c1c060ba0 2023-11-15T03:11:58.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.407+0000 7f9c3e307700 1 -- 172.21.15.121:0/645453621 --> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9c38169630 con 0x7f9c1c060ba0 2023-11-15T03:11:58.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.407+0000 7f9c2d7fa700 1 -- 172.21.15.121:0/645453621 <== osd.3 v2:172.21.15.121:6820/33135 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9c38169630 con 0x7f9c1c060ba0 2023-11-15T03:11:58.419 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:11:58.419 INFO:teuthology.orchestra.run.smithi121.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-15T03:11:58.420 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:11:58.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.411+0000 7f9c3e307700 1 -- 172.21.15.121:0/645453621 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7f9c1c060ba0 msgr2=0x7f9c1c063000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.411+0000 7f9c3e307700 1 --2- 172.21.15.121:0/645453621 >> [v2:172.21.15.121:6820/33135,v1:172.21.15.121:6822/33135] conn(0x7f9c1c060ba0 0x7f9c1c063000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.411+0000 7f9c3e307700 1 -- 172.21.15.121:0/645453621 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9c1c05a510 msgr2=0x7f9c1c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.411+0000 7f9c3e307700 1 --2- 172.21.15.121:0/645453621 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9c1c05a510 0x7f9c1c05c9d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f9c34001010 tx=0x7f9c34007040).stop 2023-11-15T03:11:58.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.411+0000 7f9c3e307700 1 -- 172.21.15.121:0/645453621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c380bc820 msgr2=0x7f9c380bcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.411+0000 7f9c3e307700 1 --2- 172.21.15.121:0/645453621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c380bc820 0x7f9c380bcca0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 rx=0x7f9c38068990 tx=0x7f9c30007f30).stop 2023-11-15T03:11:58.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.411+0000 7f9c3db06700 1 -- 172.21.15.121:0/645453621 reap_dead start 2023-11-15T03:11:58.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.411+0000 7f9c3e307700 1 -- 172.21.15.121:0/645453621 shutdown_connections 2023-11-15T03:11:58.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.411+0000 7f9c3e307700 1 -- 172.21.15.121:0/645453621 >> 172.21.15.121:0/645453621 conn(0x7f9c38067a60 msgr2=0x7f9c38154cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.411+0000 7f9c3e307700 1 -- 172.21.15.121:0/645453621 shutdown_connections 2023-11-15T03:11:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.411+0000 7f9c3e307700 1 -- 172.21.15.121:0/645453621 wait complete. 2023-11-15T03:11:58.441 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-15T03:11:58.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.595+0000 7f7fa1a5b700 1 Processor -- start 2023-11-15T03:11:58.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.595+0000 7f7fa1a5b700 1 -- start start 2023-11-15T03:11:58.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.599+0000 7f7fa1a5b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7f9c1539e0 0x7f7f9c0aa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.599+0000 7f7fa1a5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f9c0aab50 0x7f7f9c153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.599+0000 7f7fa1a5b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9c065740 con 0x7f7f9c0c7010 2023-11-15T03:11:58.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.599+0000 7f7fa1a5b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9c0c78d0 con 0x7f7f9c1539e0 2023-11-15T03:11:58.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.599+0000 7f7fa1a5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c0c7bb0 con 0x7f7f9c0aab50 2023-11-15T03:11:58.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.599+0000 7f7f9affd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7f9c1539e0 0x7f7f9c0aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.599+0000 7f7f9affd700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7f9c1539e0 0x7f7f9c0aa070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54270/0 (socket says 172.21.15.121:54270) 2023-11-15T03:11:58.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.599+0000 7f7f9affd700 1 -- 172.21.15.121:0/131838661 learned_addr learned my addr 172.21.15.121:0/131838661 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:58.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.599+0000 7f7f9a7fc700 1 --2- 172.21.15.121:0/131838661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f9c0aab50 0x7f7f9c153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.603+0000 7f7f9affd700 1 -- 172.21.15.121:0/131838661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f9c0aab50 msgr2=0x7f7f9c153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:58.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.603+0000 7f7f9affd700 1 --2- 172.21.15.121:0/131838661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f9c0aab50 0x7f7f9c153dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.603+0000 7f7f9affd700 1 -- 172.21.15.121:0/131838661 >> v1:172.21.15.190:6789/0 conn(0x7f7f9c0c7010 legacy=0x7f7f9c0c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:58.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.603+0000 7f7f9affd700 1 -- 172.21.15.121:0/131838661 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c0c7e90 con 0x7f7f9c1539e0 2023-11-15T03:11:58.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.603+0000 7f7f9a7fc700 1 --2- 172.21.15.121:0/131838661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f9c0aab50 0x7f7f9c153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:58.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.603+0000 7f7f9affd700 1 --2- 172.21.15.121:0/131838661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f9c1539e0 0x7f7f9c0aa070 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 rx=0x7f7f8c006fd0 tx=0x7f7f8c0090f0).ready entity=mon.0 client_cookie=663802a1358ad6a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:58.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.603+0000 7f7f99ffb700 1 -- 172.21.15.121:0/131838661 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00b600 con 0x7f7f9c1539e0 2023-11-15T03:11:58.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.603+0000 7f7f99ffb700 1 -- 172.21.15.121:0/131838661 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c00b760 con 0x7f7f9c1539e0 2023-11-15T03:11:58.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.603+0000 7f7f99ffb700 1 -- 172.21.15.121:0/131838661 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00bae0 con 0x7f7f9c1539e0 2023-11-15T03:11:58.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.603+0000 7f7fa1a5b700 1 -- 172.21.15.121:0/131838661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f9c1539e0 msgr2=0x7f7f9c0aa070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.603+0000 7f7fa1a5b700 1 --2- 172.21.15.121:0/131838661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f9c1539e0 0x7f7f9c0aa070 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 rx=0x7f7f8c006fd0 tx=0x7f7f8c0090f0).stop 2023-11-15T03:11:58.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.603+0000 7f7fa1a5b700 1 -- 172.21.15.121:0/131838661 shutdown_connections 2023-11-15T03:11:58.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.603+0000 7f7fa1a5b700 1 --2- 172.21.15.121:0/131838661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f9c0aab50 0x7f7f9c153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.603+0000 7f7fa1a5b700 1 --2- 172.21.15.121:0/131838661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f9c1539e0 0x7f7f9c0aa070 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.603+0000 7f7fa1a5b700 1 -- 172.21.15.121:0/131838661 >> 172.21.15.121:0/131838661 conn(0x7f7f9c063670 msgr2=0x7f7f9c063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:58.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.603+0000 7f7fa1a5b700 1 -- 172.21.15.121:0/131838661 shutdown_connections 2023-11-15T03:11:58.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.603+0000 7f7fa1a5b700 1 -- 172.21.15.121:0/131838661 wait complete. 2023-11-15T03:11:58.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.607+0000 7f7fa1a5b700 1 Processor -- start 2023-11-15T03:11:58.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.607+0000 7f7fa1a5b700 1 -- start start 2023-11-15T03:11:58.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.607+0000 7f7fa1a5b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f9c0aab50 0x7f7f9c150e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.607+0000 7f7fa1a5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f9c0c7010 0x7f7f9c0af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.607+0000 7f7fa1a5b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f9c1539e0 0x7f7f9c152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.607+0000 7f7fa1a5b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c14fea0 con 0x7f7f9c0aab50 2023-11-15T03:11:58.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.607+0000 7f7fa1a5b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c0ae050 con 0x7f7f9c1539e0 2023-11-15T03:11:58.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.607+0000 7f7fa1a5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c0ae2f0 con 0x7f7f9c0c7010 2023-11-15T03:11:58.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.607+0000 7f7f9affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f9c0aab50 0x7f7f9c150e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.620 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.611+0000 7f7f9affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f9c0aab50 0x7f7f9c150e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54286/0 (socket says 172.21.15.121:54286) 2023-11-15T03:11:58.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.611+0000 7f7f9affd700 1 -- 172.21.15.121:0/787296587 learned_addr learned my addr 172.21.15.121:0/787296587 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:58.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.611+0000 7f7f9b7fe700 1 --2- 172.21.15.121:0/787296587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f9c1539e0 0x7f7f9c152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.621 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.611+0000 7f7f9a7fc700 1 --2- 172.21.15.121:0/787296587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f9c0c7010 0x7f7f9c0af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.611+0000 7f7f9affd700 1 -- 172.21.15.121:0/787296587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f9c0c7010 msgr2=0x7f7f9c0af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.611+0000 7f7f9affd700 1 --2- 172.21.15.121:0/787296587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f9c0c7010 0x7f7f9c0af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.622 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.611+0000 7f7f9affd700 1 -- 172.21.15.121:0/787296587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f9c1539e0 msgr2=0x7f7f9c152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.611+0000 7f7f9affd700 1 --2- 172.21.15.121:0/787296587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f9c1539e0 0x7f7f9c152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.611+0000 7f7f9affd700 1 -- 172.21.15.121:0/787296587 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c009370 con 0x7f7f9c0aab50 2023-11-15T03:11:58.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.611+0000 7f7f9affd700 1 --2- 172.21.15.121:0/787296587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f9c0aab50 0x7f7f9c150e30 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 rx=0x7f7f8c009130 tx=0x7f7f8c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:58.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.611+0000 7f7f82ffd700 1 -- 172.21.15.121:0/787296587 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00b8d0 con 0x7f7f9c0aab50 2023-11-15T03:11:58.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.611+0000 7f7f82ffd700 1 -- 172.21.15.121:0/787296587 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c00ba30 con 0x7f7f9c0aab50 2023-11-15T03:11:58.623 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.611+0000 7f7f82ffd700 1 -- 172.21.15.121:0/787296587 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c01b690 con 0x7f7f9c0aab50 2023-11-15T03:11:58.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.611+0000 7f7fa1a5b700 1 -- 172.21.15.121:0/787296587 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9c0ae5d0 con 0x7f7f9c0aab50 2023-11-15T03:11:58.624 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.611+0000 7f7fa1a5b700 1 -- 172.21.15.121:0/787296587 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f9c0aeb90 con 0x7f7f9c0aab50 2023-11-15T03:11:58.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.611+0000 7f7fa1a5b700 1 -- 172.21.15.121:0/787296587 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f9c157710 con 0x7f7f9c0aab50 2023-11-15T03:11:58.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.615+0000 7f7f82ffd700 1 -- 172.21.15.121:0/787296587 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7f8c01ba40 con 0x7f7f9c0aab50 2023-11-15T03:11:58.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.615+0000 7f7f82ffd700 1 --2- 172.21.15.121:0/787296587 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7f8805a510 0x7f7f8805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.625 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.615+0000 7f7f82ffd700 1 -- 172.21.15.121:0/787296587 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f7f8c075d40 con 0x7f7f9c0aab50 2023-11-15T03:11:58.626 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.615+0000 7f7f9a7fc700 1 --2- 172.21.15.121:0/787296587 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7f8805a510 0x7f7f8805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.627 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.615+0000 7f7f9a7fc700 1 --2- 172.21.15.121:0/787296587 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7f8805a510 0x7f7f8805c9d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f7f9c0b0540 tx=0x7f7f90008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:58.632 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.623+0000 7f7f82ffd700 1 -- 172.21.15.121:0/787296587 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7f9c157710 con 0x7f7f9c0aab50 2023-11-15T03:11:58.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.783+0000 7f7fa1a5b700 1 -- 172.21.15.121:0/787296587 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f9c157710 con 0x7f7f8805a510 2023-11-15T03:11:58.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.787+0000 7f7f82ffd700 1 -- 172.21.15.121:0/787296587 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7f9c157710 con 0x7f7f8805a510 2023-11-15T03:11:58.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.791+0000 7f7fa1a5b700 1 -- 172.21.15.121:0/787296587 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7f8805a510 msgr2=0x7f7f8805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.791+0000 7f7fa1a5b700 1 --2- 172.21.15.121:0/787296587 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7f8805a510 0x7f7f8805c9d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f7f9c0b0540 tx=0x7f7f90008040).stop 2023-11-15T03:11:58.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.791+0000 7f7fa1a5b700 1 -- 172.21.15.121:0/787296587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f9c0aab50 msgr2=0x7f7f9c150e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.791+0000 7f7fa1a5b700 1 --2- 172.21.15.121:0/787296587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f9c0aab50 0x7f7f9c150e30 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 rx=0x7f7f8c009130 tx=0x7f7f8c017000).stop 2023-11-15T03:11:58.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.791+0000 7f7fa1a5b700 1 -- 172.21.15.121:0/787296587 shutdown_connections 2023-11-15T03:11:58.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.791+0000 7f7fa1a5b700 1 --2- 172.21.15.121:0/787296587 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f7f8805a510 0x7f7f8805c9d0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.791+0000 7f7fa1a5b700 1 --2- 172.21.15.121:0/787296587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f9c1539e0 0x7f7f9c152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.791+0000 7f7fa1a5b700 1 --2- 172.21.15.121:0/787296587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f9c0c7010 0x7f7f9c0af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.791+0000 7f7fa1a5b700 1 --2- 172.21.15.121:0/787296587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f9c0aab50 0x7f7f9c150e30 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.791+0000 7f7fa1a5b700 1 -- 172.21.15.121:0/787296587 >> 172.21.15.121:0/787296587 conn(0x7f7f9c063670 msgr2=0x7f7f9c0acf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:58.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.791+0000 7f7fa1a5b700 1 -- 172.21.15.121:0/787296587 shutdown_connections 2023-11-15T03:11:58.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.791+0000 7f7fa1a5b700 1 -- 172.21.15.121:0/787296587 wait complete. 2023-11-15T03:11:58.802 INFO:teuthology.orchestra.run.smithi121.stderr:instructed osd(s) 3 to deep-scrub 2023-11-15T03:11:58.821 INFO:tasks.ceph:Scrubbing osd.4 2023-11-15T03:11:58.821 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-15T03:11:58.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.975+0000 7fef217b9700 1 Processor -- start 2023-11-15T03:11:58.986 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.975+0000 7fef217b9700 1 -- start start 2023-11-15T03:11:58.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.979+0000 7fef217b9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fef1c0abaa0 0x7fef1c0abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.979+0000 7fef217b9700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef1c0c2c00 con 0x7fef1c0bb510 2023-11-15T03:11:58.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.979+0000 7fef217b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef1c0657b0 con 0x7fef1c0a8880 2023-11-15T03:11:58.988 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.979+0000 7fef217b9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef1c065a90 con 0x7fef1c0abaa0 2023-11-15T03:11:58.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.979+0000 7fef1bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fef1c0abaa0 0x7fef1c0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.979+0000 7fef13fff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fef1c0bb510 0x7fef1c063570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51140/0 (socket says 172.21.15.121:51140) 2023-11-15T03:11:58.990 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.979+0000 7fef13fff700 1 -- 172.21.15.121:0/2172247176 learned_addr learned my addr 172.21.15.121:0/2172247176 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:58.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.979+0000 7fef1b7fe700 1 -- 172.21.15.121:0/2172247176 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125271967 0 0) 0x7fef1c0c2c00 con 0x7fef1c0bb510 2023-11-15T03:11:58.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.979+0000 7fef1b7fe700 1 -- 172.21.15.121:0/2172247176 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feefc0033f0 con 0x7fef1c0bb510 2023-11-15T03:11:58.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.979+0000 7fef1bfff700 1 -- 172.21.15.121:0/2172247176 >> v1:172.21.15.190:6789/0 conn(0x7fef1c0a8880 legacy=0x7fef1c151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.979+0000 7fef1b7fe700 1 -- 172.21.15.121:0/2172247176 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1725326343 0 0) 0x7fef1c0657b0 con 0x7fef1c0a8880 2023-11-15T03:11:58.991 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.979+0000 7fef1bfff700 1 -- 172.21.15.121:0/2172247176 >> v1:172.21.15.121:6789/0 conn(0x7fef1c0bb510 legacy=0x7fef1c063570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.979+0000 7fef1bfff700 1 -- 172.21.15.121:0/2172247176 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c0c2890 con 0x7fef1c0abaa0 2023-11-15T03:11:58.992 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.979+0000 7fef1bfff700 1 --2- 172.21.15.121:0/2172247176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef1c0abaa0 0x7fef1c0abe80 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 rx=0x7fef0c009760 tx=0x7fef0c009060).ready entity=mon.0 client_cookie=9b5155b2845f87f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:58.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.979+0000 7fef1b7fe700 1 -- 172.21.15.121:0/2172247176 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00f510 con 0x7fef1c0abaa0 2023-11-15T03:11:58.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.979+0000 7fef1b7fe700 1 -- 172.21.15.121:0/2172247176 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0c00f670 con 0x7fef1c0abaa0 2023-11-15T03:11:58.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.979+0000 7fef1b7fe700 1 -- 172.21.15.121:0/2172247176 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00f9f0 con 0x7fef1c0abaa0 2023-11-15T03:11:58.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.983+0000 7fef217b9700 1 -- 172.21.15.121:0/2172247176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef1c0abaa0 msgr2=0x7fef1c0abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:58.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.983+0000 7fef217b9700 1 --2- 172.21.15.121:0/2172247176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef1c0abaa0 0x7fef1c0abe80 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 rx=0x7fef0c009760 tx=0x7fef0c009060).stop 2023-11-15T03:11:58.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.983+0000 7fef217b9700 1 -- 172.21.15.121:0/2172247176 shutdown_connections 2023-11-15T03:11:58.994 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.983+0000 7fef217b9700 1 --2- 172.21.15.121:0/2172247176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef1c0abaa0 0x7fef1c0abe80 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:58.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.983+0000 7fef217b9700 1 -- 172.21.15.121:0/2172247176 >> 172.21.15.121:0/2172247176 conn(0x7fef1c067a60 msgr2=0x7fef1c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:58.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.983+0000 7fef217b9700 1 -- 172.21.15.121:0/2172247176 shutdown_connections 2023-11-15T03:11:58.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.983+0000 7fef217b9700 1 -- 172.21.15.121:0/2172247176 wait complete. 2023-11-15T03:11:58.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.983+0000 7fef217b9700 1 Processor -- start 2023-11-15T03:11:58.997 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.983+0000 7fef217b9700 1 -- start start 2023-11-15T03:11:58.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef217b9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef1c0a8880 0x7fef1c14fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef217b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef1c0abaa0 0x7fef1c154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef217b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef1c0bb510 0x7fef1c151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:58.998 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef217b9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c154180 con 0x7fef1c0a8880 2023-11-15T03:11:58.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef217b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c14ef50 con 0x7fef1c0bb510 2023-11-15T03:11:58.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef217b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c14f230 con 0x7fef1c0abaa0 2023-11-15T03:11:58.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef1bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef1c0a8880 0x7fef1c14fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:58.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef1bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef1c0a8880 0x7fef1c14fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54302/0 (socket says 172.21.15.121:54302) 2023-11-15T03:11:58.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef1bfff700 1 -- 172.21.15.121:0/1568213985 learned_addr learned my addr 172.21.15.121:0/1568213985 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:59.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef20fb8700 1 --2- 172.21.15.121:0/1568213985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef1c0bb510 0x7fef1c151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef13fff700 1 --2- 172.21.15.121:0/1568213985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef1c0abaa0 0x7fef1c154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef1bfff700 1 -- 172.21.15.121:0/1568213985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef1c0abaa0 msgr2=0x7fef1c154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef1bfff700 1 --2- 172.21.15.121:0/1568213985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef1c0abaa0 0x7fef1c154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.000 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef1bfff700 1 -- 172.21.15.121:0/1568213985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef1c0bb510 msgr2=0x7fef1c151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef1bfff700 1 --2- 172.21.15.121:0/1568213985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef1c0bb510 0x7fef1c151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef1bfff700 1 -- 172.21.15.121:0/1568213985 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0c0092a0 con 0x7fef1c0a8880 2023-11-15T03:11:59.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef1bfff700 1 --2- 172.21.15.121:0/1568213985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef1c0a8880 0x7fef1c14fff0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 rx=0x7fef0c009e00 tx=0x7fef0c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:59.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef18ff9700 1 -- 172.21.15.121:0/1568213985 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00f7e0 con 0x7fef1c0a8880 2023-11-15T03:11:59.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef18ff9700 1 -- 172.21.15.121:0/1568213985 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0c00f940 con 0x7fef1c0a8880 2023-11-15T03:11:59.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef217b9700 1 -- 172.21.15.121:0/1568213985 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1c14f510 con 0x7fef1c0a8880 2023-11-15T03:11:59.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef18ff9700 1 -- 172.21.15.121:0/1568213985 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c01b660 con 0x7fef1c0a8880 2023-11-15T03:11:59.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.987+0000 7fef217b9700 1 -- 172.21.15.121:0/1568213985 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1c14fad0 con 0x7fef1c0a8880 2023-11-15T03:11:59.003 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.991+0000 7fef217b9700 1 -- 172.21.15.121:0/1568213985 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fef1c06bcf0 con 0x7fef1c0a8880 2023-11-15T03:11:59.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.991+0000 7fef18ff9700 1 -- 172.21.15.121:0/1568213985 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fef0c01b9d0 con 0x7fef1c0a8880 2023-11-15T03:11:59.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.991+0000 7fef18ff9700 1 --2- 172.21.15.121:0/1568213985 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef0005a510 0x7fef0005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.991+0000 7fef18ff9700 1 -- 172.21.15.121:0/1568213985 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7fef0c0761a0 con 0x7fef1c0a8880 2023-11-15T03:11:59.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.991+0000 7fef13fff700 1 --2- 172.21.15.121:0/1568213985 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef0005a510 0x7fef0005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.991+0000 7fef18ff9700 1 --2- 172.21.15.121:0/1568213985 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7fef00060ba0 0x7fef00063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.991+0000 7fef18ff9700 1 -- 172.21.15.121:0/1568213985 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef00063680 con 0x7fef00060ba0 2023-11-15T03:11:59.005 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.991+0000 7fef18ff9700 1 -- 172.21.15.121:0/1568213985 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef0c076490 con 0x7fef1c0a8880 2023-11-15T03:11:59.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.995+0000 7fef20fb8700 1 --2- 172.21.15.121:0/1568213985 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7fef00060ba0 0x7fef00063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.995+0000 7fef13fff700 1 --2- 172.21.15.121:0/1568213985 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef0005a510 0x7fef0005c9d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7fef1c14cb40 tx=0x7fef04009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:59.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.995+0000 7fef20fb8700 1 --2- 172.21.15.121:0/1568213985 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7fef00060ba0 0x7fef00063000 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:59.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:58.995+0000 7fef18ff9700 1 -- 172.21.15.121:0/1568213985 <== osd.4 v2:172.21.15.190:6816/33348 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fef00063680 con 0x7fef00060ba0 2023-11-15T03:11:59.022 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.011+0000 7fef217b9700 1 -- 172.21.15.121:0/1568213985 --> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fef1c0c2540 con 0x7fef00060ba0 2023-11-15T03:11:59.023 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.015+0000 7fef18ff9700 1 -- 172.21.15.121:0/1568213985 <== osd.4 v2:172.21.15.190:6816/33348 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fef1c0c2540 con 0x7fef00060ba0 2023-11-15T03:11:59.024 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:11:59.024 INFO:teuthology.orchestra.run.smithi121.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-15T03:11:59.024 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:11:59.028 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.015+0000 7fef217b9700 1 -- 172.21.15.121:0/1568213985 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7fef00060ba0 msgr2=0x7fef00063000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.015+0000 7fef217b9700 1 --2- 172.21.15.121:0/1568213985 >> [v2:172.21.15.190:6816/33348,v1:172.21.15.190:6819/33348] conn(0x7fef00060ba0 0x7fef00063000 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.015+0000 7fef217b9700 1 -- 172.21.15.121:0/1568213985 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef0005a510 msgr2=0x7fef0005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.015+0000 7fef217b9700 1 --2- 172.21.15.121:0/1568213985 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fef0005a510 0x7fef0005c9d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7fef1c14cb40 tx=0x7fef04009040).stop 2023-11-15T03:11:59.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.015+0000 7fef217b9700 1 -- 172.21.15.121:0/1568213985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef1c0a8880 msgr2=0x7fef1c14fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.029 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.015+0000 7fef217b9700 1 --2- 172.21.15.121:0/1568213985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef1c0a8880 0x7fef1c14fff0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 rx=0x7fef0c009e00 tx=0x7fef0c017000).stop 2023-11-15T03:11:59.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.015+0000 7fef20fb8700 1 -- 172.21.15.121:0/1568213985 reap_dead start 2023-11-15T03:11:59.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.015+0000 7fef217b9700 1 -- 172.21.15.121:0/1568213985 shutdown_connections 2023-11-15T03:11:59.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.015+0000 7fef217b9700 1 -- 172.21.15.121:0/1568213985 >> 172.21.15.121:0/1568213985 conn(0x7fef1c067a60 msgr2=0x7fef1c14e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:59.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.015+0000 7fef217b9700 1 -- 172.21.15.121:0/1568213985 shutdown_connections 2023-11-15T03:11:59.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.015+0000 7fef217b9700 1 -- 172.21.15.121:0/1568213985 wait complete. 2023-11-15T03:11:59.047 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-15T03:11:59.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.211+0000 7fb261da9700 1 Processor -- start 2023-11-15T03:11:59.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.211+0000 7fb261da9700 1 -- start start 2023-11-15T03:11:59.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.211+0000 7fb261da9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c153aa0 0x7fb25c066680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.211+0000 7fb261da9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb25c153e80 0x7fb25c0ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.211+0000 7fb261da9700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb25c0c0490 0x7fb25c0c0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.211+0000 7fb261da9700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb25c066b70 con 0x7fb25c0c0490 2023-11-15T03:11:59.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.211+0000 7fb261da9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb25c0c5c50 con 0x7fb25c153e80 2023-11-15T03:11:59.222 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.211+0000 7fb261da9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0c0e30 con 0x7fb25c153aa0 2023-11-15T03:11:59.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.211+0000 7fb25bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb25c0c0490 0x7fb25c0c0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.211+0000 7fb25bfff700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb25c0c0490 0x7fb25c0c0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54304/0 (socket says 172.21.15.121:54304) 2023-11-15T03:11:59.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.211+0000 7fb25bfff700 1 -- 172.21.15.121:0/3760813450 learned_addr learned my addr 172.21.15.121:0/3760813450 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:59.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.211+0000 7fb25affd700 1 --2- 172.21.15.121:0/3760813450 >> v2:172.21.15.190:3300/0 conn(0x7fb25c153e80 0x7fb25c0ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.211+0000 7fb25b7fe700 1 --2- 172.21.15.121:0/3760813450 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c153aa0 0x7fb25c066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.211+0000 7fb25bfff700 1 -- 172.21.15.121:0/3760813450 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c153aa0 msgr2=0x7fb25c066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:59.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.211+0000 7fb25bfff700 1 --2- 172.21.15.121:0/3760813450 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c153aa0 0x7fb25c066680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.225 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.211+0000 7fb25bfff700 1 -- 172.21.15.121:0/3760813450 >> v2:172.21.15.190:3300/0 conn(0x7fb25c153e80 msgr2=0x7fb25c0ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:59.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.211+0000 7fb25bfff700 1 --2- 172.21.15.121:0/3760813450 >> v2:172.21.15.190:3300/0 conn(0x7fb25c153e80 0x7fb25c0ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.211+0000 7fb25bfff700 1 -- 172.21.15.121:0/3760813450 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c0c5920 con 0x7fb25c0c0490 2023-11-15T03:11:59.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.211+0000 7fb25affd700 1 --2- 172.21.15.121:0/3760813450 >> v2:172.21.15.190:3300/0 conn(0x7fb25c153e80 0x7fb25c0ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:59.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.215+0000 7fb25bfff700 1 --2- 172.21.15.121:0/3760813450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb25c0c0490 0x7fb25c0c0940 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 rx=0x7fb25c069730 tx=0x7fb254009840).ready entity=mon.0 client_cookie=416f67e3ec5ebbfa server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:59.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.215+0000 7fb25a7fc700 1 -- 172.21.15.121:0/3760813450 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25400e070 con 0x7fb25c0c0490 2023-11-15T03:11:59.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.215+0000 7fb25a7fc700 1 -- 172.21.15.121:0/3760813450 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25400ba20 con 0x7fb25c0c0490 2023-11-15T03:11:59.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.215+0000 7fb25a7fc700 1 -- 172.21.15.121:0/3760813450 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25400bda0 con 0x7fb25c0c0490 2023-11-15T03:11:59.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.215+0000 7fb261da9700 1 -- 172.21.15.121:0/3760813450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb25c0c0490 msgr2=0x7fb25c0c0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.215+0000 7fb261da9700 1 --2- 172.21.15.121:0/3760813450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb25c0c0490 0x7fb25c0c0940 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 rx=0x7fb25c069730 tx=0x7fb254009840).stop 2023-11-15T03:11:59.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.215+0000 7fb261da9700 1 -- 172.21.15.121:0/3760813450 shutdown_connections 2023-11-15T03:11:59.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.215+0000 7fb261da9700 1 --2- 172.21.15.121:0/3760813450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb25c0c0490 0x7fb25c0c0940 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.215+0000 7fb261da9700 1 --2- 172.21.15.121:0/3760813450 >> v2:172.21.15.190:3300/0 conn(0x7fb25c153e80 0x7fb25c0ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.215+0000 7fb261da9700 1 --2- 172.21.15.121:0/3760813450 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c153aa0 0x7fb25c066680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.215+0000 7fb261da9700 1 -- 172.21.15.121:0/3760813450 >> 172.21.15.121:0/3760813450 conn(0x7fb25c067a60 msgr2=0x7fb25c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:59.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.215+0000 7fb261da9700 1 -- 172.21.15.121:0/3760813450 shutdown_connections 2023-11-15T03:11:59.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.215+0000 7fb261da9700 1 -- 172.21.15.121:0/3760813450 wait complete. 2023-11-15T03:11:59.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.219+0000 7fb261da9700 1 Processor -- start 2023-11-15T03:11:59.230 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.219+0000 7fb261da9700 1 -- start start 2023-11-15T03:11:59.231 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.219+0000 7fb261da9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb25c0c0490 0x7fb25c13c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.219+0000 7fb261da9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c153aa0 0x7fb25c14e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.219+0000 7fb261da9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb25c153e80 0x7fb25c14eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.219+0000 7fb261da9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c154a20 con 0x7fb25c0c0490 2023-11-15T03:11:59.232 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.219+0000 7fb261da9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c14ef90 con 0x7fb25c153e80 2023-11-15T03:11:59.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.219+0000 7fb261da9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0c4340 con 0x7fb25c153aa0 2023-11-15T03:11:59.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.219+0000 7fb25b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb25c0c0490 0x7fb25c13c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.219+0000 7fb25b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb25c0c0490 0x7fb25c13c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54314/0 (socket says 172.21.15.121:54314) 2023-11-15T03:11:59.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.219+0000 7fb25b7fe700 1 -- 172.21.15.121:0/2247580576 learned_addr learned my addr 172.21.15.121:0/2247580576 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:59.233 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.219+0000 7fb25bfff700 1 --2- 172.21.15.121:0/2247580576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb25c153e80 0x7fb25c14eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.219+0000 7fb25affd700 1 --2- 172.21.15.121:0/2247580576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c153aa0 0x7fb25c14e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.234 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.219+0000 7fb25b7fe700 1 -- 172.21.15.121:0/2247580576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c153aa0 msgr2=0x7fb25c14e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.219+0000 7fb25b7fe700 1 --2- 172.21.15.121:0/2247580576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c153aa0 0x7fb25c14e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.219+0000 7fb25b7fe700 1 -- 172.21.15.121:0/2247580576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb25c153e80 msgr2=0x7fb25c14eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.219+0000 7fb25b7fe700 1 --2- 172.21.15.121:0/2247580576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb25c153e80 0x7fb25c14eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.223+0000 7fb25b7fe700 1 -- 172.21.15.121:0/2247580576 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2440092a0 con 0x7fb25c0c0490 2023-11-15T03:11:59.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.223+0000 7fb25b7fe700 1 --2- 172.21.15.121:0/2247580576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb25c0c0490 0x7fb25c13c370 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 rx=0x7fb24c00d890 tx=0x7fb24c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:59.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.223+0000 7fb2537fe700 1 -- 172.21.15.121:0/2247580576 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c011590 con 0x7fb25c0c0490 2023-11-15T03:11:59.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.223+0000 7fb2537fe700 1 -- 172.21.15.121:0/2247580576 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24c0116f0 con 0x7fb25c0c0490 2023-11-15T03:11:59.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.223+0000 7fb261da9700 1 -- 172.21.15.121:0/2247580576 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb254009ae0 con 0x7fb25c0c0490 2023-11-15T03:11:59.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.223+0000 7fb2537fe700 1 -- 172.21.15.121:0/2247580576 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c011a70 con 0x7fb25c0c0490 2023-11-15T03:11:59.236 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.223+0000 7fb261da9700 1 -- 172.21.15.121:0/2247580576 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25c154640 con 0x7fb25c0c0490 2023-11-15T03:11:59.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.223+0000 7fb261da9700 1 -- 172.21.15.121:0/2247580576 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb25c0c4740 con 0x7fb25c0c0490 2023-11-15T03:11:59.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.223+0000 7fb2537fe700 1 -- 172.21.15.121:0/2247580576 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb24c01c460 con 0x7fb25c0c0490 2023-11-15T03:11:59.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.227+0000 7fb2537fe700 1 --2- 172.21.15.121:0/2247580576 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb23c05a510 0x7fb23c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.227+0000 7fb2537fe700 1 -- 172.21.15.121:0/2247580576 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7fb24c074360 con 0x7fb25c0c0490 2023-11-15T03:11:59.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.227+0000 7fb25affd700 1 --2- 172.21.15.121:0/2247580576 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb23c05a510 0x7fb23c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.227+0000 7fb25affd700 1 --2- 172.21.15.121:0/2247580576 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb23c05a510 0x7fb23c05c9d0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7fb25c0acbc0 tx=0x7fb244009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:59.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.231+0000 7fb2537fe700 1 -- 172.21.15.121:0/2247580576 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb24c047cc0 con 0x7fb25c0c0490 2023-11-15T03:11:59.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.391+0000 7fb261da9700 1 -- 172.21.15.121:0/2247580576 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb25c066d50 con 0x7fb23c05a510 2023-11-15T03:11:59.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.395+0000 7fb2537fe700 1 -- 172.21.15.121:0/2247580576 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb25c066d50 con 0x7fb23c05a510 2023-11-15T03:11:59.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.395+0000 7fb261da9700 1 -- 172.21.15.121:0/2247580576 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb23c05a510 msgr2=0x7fb23c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.395+0000 7fb261da9700 1 --2- 172.21.15.121:0/2247580576 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb23c05a510 0x7fb23c05c9d0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7fb25c0acbc0 tx=0x7fb244009060).stop 2023-11-15T03:11:59.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.395+0000 7fb261da9700 1 -- 172.21.15.121:0/2247580576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb25c0c0490 msgr2=0x7fb25c13c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.395+0000 7fb261da9700 1 --2- 172.21.15.121:0/2247580576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb25c0c0490 0x7fb25c13c370 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 rx=0x7fb24c00d890 tx=0x7fb24c008040).stop 2023-11-15T03:11:59.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.395+0000 7fb261da9700 1 -- 172.21.15.121:0/2247580576 shutdown_connections 2023-11-15T03:11:59.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.395+0000 7fb261da9700 1 --2- 172.21.15.121:0/2247580576 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fb23c05a510 0x7fb23c05c9d0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.395+0000 7fb261da9700 1 --2- 172.21.15.121:0/2247580576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb25c153e80 0x7fb25c14eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.395+0000 7fb261da9700 1 --2- 172.21.15.121:0/2247580576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c153aa0 0x7fb25c14e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.395+0000 7fb261da9700 1 --2- 172.21.15.121:0/2247580576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb25c0c0490 0x7fb25c13c370 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.395+0000 7fb261da9700 1 -- 172.21.15.121:0/2247580576 >> 172.21.15.121:0/2247580576 conn(0x7fb25c067a60 msgr2=0x7fb25c0acd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:59.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.395+0000 7fb261da9700 1 -- 172.21.15.121:0/2247580576 shutdown_connections 2023-11-15T03:11:59.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.395+0000 7fb261da9700 1 -- 172.21.15.121:0/2247580576 wait complete. 2023-11-15T03:11:59.409 INFO:teuthology.orchestra.run.smithi121.stderr:instructed osd(s) 4 to deep-scrub 2023-11-15T03:11:59.425 INFO:tasks.ceph:Scrubbing osd.5 2023-11-15T03:11:59.425 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-15T03:11:59.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.587+0000 7f1c1088f700 1 Processor -- start 2023-11-15T03:11:59.596 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.587+0000 7f1c1088f700 1 -- start start 2023-11-15T03:11:59.598 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.587+0000 7f1c1088f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1c0c157d90 0x7f1c0c158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.587+0000 7f1c1088f700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1c0c0a8750 0x7f1c0c0a8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.587+0000 7f1c1088f700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c0c0bb910 con 0x7f1c0c0669b0 2023-11-15T03:11:59.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.587+0000 7f1c1088f700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c0c0c1fb0 con 0x7f1c0c0a8750 2023-11-15T03:11:59.599 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.587+0000 7f1c1088f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c0c15bbd0 con 0x7f1c0c157d90 2023-11-15T03:11:59.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.587+0000 7f1c0affd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f1c0c0669b0 0x7f1c0c0c1760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51148/0 (socket says 172.21.15.121:51148) 2023-11-15T03:11:59.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.587+0000 7f1c0affd700 1 -- 172.21.15.121:0/1527222469 learned_addr learned my addr 172.21.15.121:0/1527222469 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:59.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.587+0000 7f1c0bfff700 1 --2- 172.21.15.121:0/1527222469 >> v2:172.21.15.121:3300/0 conn(0x7f1c0c0a8750 0x7f1c0c0a8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.600 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.587+0000 7f1c0b7fe700 1 --2- 172.21.15.121:0/1527222469 >> v2:172.21.15.190:3300/0 conn(0x7f1c0c157d90 0x7f1c0c158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c0a7fc700 1 -- 172.21.15.121:0/1527222469 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991706816 0 0) 0x7f1c0c0bb910 con 0x7f1c0c0669b0 2023-11-15T03:11:59.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c0a7fc700 1 -- 172.21.15.121:0/1527222469 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bf00033f0 con 0x7f1c0c0669b0 2023-11-15T03:11:59.603 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c0bfff700 1 -- 172.21.15.121:0/1527222469 >> v2:172.21.15.190:3300/0 conn(0x7f1c0c157d90 msgr2=0x7f1c0c158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:59.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c0bfff700 1 --2- 172.21.15.121:0/1527222469 >> v2:172.21.15.190:3300/0 conn(0x7f1c0c157d90 0x7f1c0c158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c0bfff700 1 -- 172.21.15.121:0/1527222469 >> v1:172.21.15.121:6789/0 conn(0x7f1c0c0669b0 legacy=0x7f1c0c0c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c0a7fc700 1 -- 172.21.15.121:0/1527222469 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3816905318 0 0) 0x7f1bf00033f0 con 0x7f1c0c0669b0 2023-11-15T03:11:59.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c0bfff700 1 -- 172.21.15.121:0/1527222469 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0c0a9090 con 0x7f1c0c0a8750 2023-11-15T03:11:59.604 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c0b7fe700 1 --2- 172.21.15.121:0/1527222469 >> v2:172.21.15.190:3300/0 conn(0x7f1c0c157d90 0x7f1c0c158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:11:59.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c0bfff700 1 --2- 172.21.15.121:0/1527222469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c0a8750 0x7f1c0c0a8ba0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 rx=0x7f1c0c15bd10 tx=0x7f1c04009780).ready entity=mon.0 client_cookie=60bcca6492259a06 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:59.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c0a7fc700 1 -- 172.21.15.121:0/1527222469 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0400e070 con 0x7f1c0c0a8750 2023-11-15T03:11:59.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c0a7fc700 1 -- 172.21.15.121:0/1527222469 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0400ba80 con 0x7f1c0c0a8750 2023-11-15T03:11:59.605 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c0a7fc700 1 -- 172.21.15.121:0/1527222469 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0400be00 con 0x7f1c0c0a8750 2023-11-15T03:11:59.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c1088f700 1 -- 172.21.15.121:0/1527222469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c0a8750 msgr2=0x7f1c0c0a8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c1088f700 1 --2- 172.21.15.121:0/1527222469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c0a8750 0x7f1c0c0a8ba0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 rx=0x7f1c0c15bd10 tx=0x7f1c04009780).stop 2023-11-15T03:11:59.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c1088f700 1 -- 172.21.15.121:0/1527222469 shutdown_connections 2023-11-15T03:11:59.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c1088f700 1 --2- 172.21.15.121:0/1527222469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c0a8750 0x7f1c0c0a8ba0 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.606 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c1088f700 1 --2- 172.21.15.121:0/1527222469 >> v2:172.21.15.190:3300/0 conn(0x7f1c0c157d90 0x7f1c0c158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c1088f700 1 -- 172.21.15.121:0/1527222469 >> 172.21.15.121:0/1527222469 conn(0x7f1c0c067a60 msgr2=0x7f1c0c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:59.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c1088f700 1 -- 172.21.15.121:0/1527222469 shutdown_connections 2023-11-15T03:11:59.607 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.591+0000 7f1c1088f700 1 -- 172.21.15.121:0/1527222469 wait complete. 2023-11-15T03:11:59.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.595+0000 7f1c1088f700 1 Processor -- start 2023-11-15T03:11:59.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.595+0000 7f1c1088f700 1 -- start start 2023-11-15T03:11:59.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.595+0000 7f1c1088f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c0c0669b0 0x7f1c0c14b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c1088f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c0a8750 0x7f1c0c159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c1088f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c0c157d90 0x7f1c0c159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c1088f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c156120 con 0x7f1c0c0a8750 2023-11-15T03:11:59.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c1088f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c0bd5f0 con 0x7f1c0c0669b0 2023-11-15T03:11:59.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c1088f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c155510 con 0x7f1c0c157d90 2023-11-15T03:11:59.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c0affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c0a8750 0x7f1c0c159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.611 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c0affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c0a8750 0x7f1c0c159260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54334/0 (socket says 172.21.15.121:54334) 2023-11-15T03:11:59.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c0affd700 1 -- 172.21.15.121:0/2665088425 learned_addr learned my addr 172.21.15.121:0/2665088425 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:59.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c0bfff700 1 --2- 172.21.15.121:0/2665088425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c0c157d90 0x7f1c0c159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c0b7fe700 1 --2- 172.21.15.121:0/2665088425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c0c0669b0 0x7f1c0c14b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c0affd700 1 -- 172.21.15.121:0/2665088425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c0c157d90 msgr2=0x7f1c0c159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c0affd700 1 --2- 172.21.15.121:0/2665088425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c0c157d90 0x7f1c0c159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c0affd700 1 -- 172.21.15.121:0/2665088425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c0c0669b0 msgr2=0x7f1c0c14b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c0affd700 1 --2- 172.21.15.121:0/2665088425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c0c0669b0 0x7f1c0c14b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c0affd700 1 -- 172.21.15.121:0/2665088425 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc0092a0 con 0x7f1c0c0a8750 2023-11-15T03:11:59.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c0b7fe700 1 --2- 172.21.15.121:0/2665088425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c0c0669b0 0x7f1c0c14b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:59.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c0bfff700 1 --2- 172.21.15.121:0/2665088425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c0c157d90 0x7f1c0c159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:59.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c0affd700 1 --2- 172.21.15.121:0/2665088425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c0a8750 0x7f1c0c159260 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 rx=0x7f1bf800c510 tx=0x7f1bf80028b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:59.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c037fe700 1 -- 172.21.15.121:0/2665088425 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80072e0 con 0x7f1c0c0a8750 2023-11-15T03:11:59.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c037fe700 1 -- 172.21.15.121:0/2665088425 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf8007440 con 0x7f1c0c0a8750 2023-11-15T03:11:59.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c1088f700 1 -- 172.21.15.121:0/2665088425 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c04009a20 con 0x7f1c0c0a8750 2023-11-15T03:11:59.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c037fe700 1 -- 172.21.15.121:0/2665088425 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8005880 con 0x7f1c0c0a8750 2023-11-15T03:11:59.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c1088f700 1 -- 172.21.15.121:0/2665088425 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c0c155ad0 con 0x7f1c0c0a8750 2023-11-15T03:11:59.616 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.599+0000 7f1c1088f700 1 -- 172.21.15.121:0/2665088425 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c0c06bcf0 con 0x7f1c0c0a8750 2023-11-15T03:11:59.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.603+0000 7f1c037fe700 1 -- 172.21.15.121:0/2665088425 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1bf8005c70 con 0x7f1c0c0a8750 2023-11-15T03:11:59.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.603+0000 7f1c037fe700 1 --2- 172.21.15.121:0/2665088425 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1bf405a540 0x7f1bf405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.603+0000 7f1c037fe700 1 -- 172.21.15.121:0/2665088425 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f1bf8073e10 con 0x7f1c0c0a8750 2023-11-15T03:11:59.617 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.603+0000 7f1c0b7fe700 1 --2- 172.21.15.121:0/2665088425 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1bf405a540 0x7f1bf405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.603+0000 7f1c037fe700 1 --2- 172.21.15.121:0/2665088425 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7f1bf4060bd0 0x7f1bf4063030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.603+0000 7f1c037fe700 1 -- 172.21.15.121:0/2665088425 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1bf40636b0 con 0x7f1bf4060bd0 2023-11-15T03:11:59.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.603+0000 7f1c037fe700 1 -- 172.21.15.121:0/2665088425 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bf8076370 con 0x7f1c0c0a8750 2023-11-15T03:11:59.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.603+0000 7f1c0b7fe700 1 --2- 172.21.15.121:0/2665088425 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1bf405a540 0x7f1bf405ca00 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f1bfc009270 tx=0x7f1bfc008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:59.618 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.603+0000 7f1c0bfff700 1 --2- 172.21.15.121:0/2665088425 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7f1bf4060bd0 0x7f1bf4063030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.607+0000 7f1c0bfff700 1 --2- 172.21.15.121:0/2665088425 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7f1bf4060bd0 0x7f1bf4063030 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:59.619 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.607+0000 7f1c037fe700 1 -- 172.21.15.121:0/2665088425 <== osd.5 v2:172.21.15.190:6800/33349 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1bf40636b0 con 0x7f1bf4060bd0 2023-11-15T03:11:59.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.627+0000 7f1c1088f700 1 -- 172.21.15.121:0/2665088425 --> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1c0c066460 con 0x7f1bf4060bd0 2023-11-15T03:11:59.636 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.627+0000 7f1c037fe700 1 -- 172.21.15.121:0/2665088425 <== osd.5 v2:172.21.15.190:6800/33349 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1c0c066460 con 0x7f1bf4060bd0 2023-11-15T03:11:59.637 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:11:59.637 INFO:teuthology.orchestra.run.smithi121.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-15T03:11:59.637 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:11:59.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.627+0000 7f1c1088f700 1 -- 172.21.15.121:0/2665088425 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7f1bf4060bd0 msgr2=0x7f1bf4063030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.627+0000 7f1c1088f700 1 --2- 172.21.15.121:0/2665088425 >> [v2:172.21.15.190:6800/33349,v1:172.21.15.190:6801/33349] conn(0x7f1bf4060bd0 0x7f1bf4063030 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.638 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.627+0000 7f1c1088f700 1 -- 172.21.15.121:0/2665088425 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1bf405a540 msgr2=0x7f1bf405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.627+0000 7f1c1088f700 1 --2- 172.21.15.121:0/2665088425 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f1bf405a540 0x7f1bf405ca00 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f1bfc009270 tx=0x7f1bfc008040).stop 2023-11-15T03:11:59.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.627+0000 7f1c1088f700 1 -- 172.21.15.121:0/2665088425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c0a8750 msgr2=0x7f1c0c159260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.627+0000 7f1c1088f700 1 --2- 172.21.15.121:0/2665088425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c0c0a8750 0x7f1c0c159260 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 rx=0x7f1bf800c510 tx=0x7f1bf80028b0).stop 2023-11-15T03:11:59.639 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.627+0000 7f1c0bfff700 1 -- 172.21.15.121:0/2665088425 reap_dead start 2023-11-15T03:11:59.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.627+0000 7f1c1088f700 1 -- 172.21.15.121:0/2665088425 shutdown_connections 2023-11-15T03:11:59.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.627+0000 7f1c1088f700 1 -- 172.21.15.121:0/2665088425 >> 172.21.15.121:0/2665088425 conn(0x7f1c0c067a60 msgr2=0x7f1c0c152370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:59.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.627+0000 7f1c1088f700 1 -- 172.21.15.121:0/2665088425 shutdown_connections 2023-11-15T03:11:59.640 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.627+0000 7f1c1088f700 1 -- 172.21.15.121:0/2665088425 wait complete. 2023-11-15T03:11:59.659 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-15T03:11:59.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.815+0000 7f55a759e700 1 Processor -- start 2023-11-15T03:11:59.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.815+0000 7f55a759e700 1 -- start start 2023-11-15T03:11:59.827 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.819+0000 7f55a759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a8153a60 0x7f55a8153e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.819+0000 7f55a759e700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f55a80c5d10 0x7f55a8066530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.819+0000 7f55a759e700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a80660a0 con 0x7f55a80bfe40 2023-11-15T03:11:59.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.819+0000 7f55a759e700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a80c0a00 con 0x7f55a80c5d10 2023-11-15T03:11:59.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.819+0000 7f55a759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a80c5500 con 0x7f55a8153a60 2023-11-15T03:11:59.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.819+0000 7f55a6d9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f55a80bfe40 0x7f55a80c02f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51158/0 (socket says 172.21.15.121:51158) 2023-11-15T03:11:59.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.819+0000 7f55a6d9d700 1 -- 172.21.15.121:0/1463740635 learned_addr learned my addr 172.21.15.121:0/1463740635 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:11:59.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.819+0000 7f55a5d9b700 1 --2- 172.21.15.121:0/1463740635 >> v2:172.21.15.121:3300/0 conn(0x7f55a80c5d10 0x7f55a8066530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.819+0000 7f55a659c700 1 --2- 172.21.15.121:0/1463740635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a8153a60 0x7f55a8153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.819+0000 7f55a5d9b700 1 -- 172.21.15.121:0/1463740635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a8153a60 msgr2=0x7f55a8153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:11:59.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.819+0000 7f55a5d9b700 1 --2- 172.21.15.121:0/1463740635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a8153a60 0x7f55a8153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.819+0000 7f55a5d9b700 1 -- 172.21.15.121:0/1463740635 >> v1:172.21.15.121:6789/0 conn(0x7f55a80bfe40 legacy=0x7f55a80c02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.819+0000 7f55a5d9b700 1 -- 172.21.15.121:0/1463740635 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a80657b0 con 0x7f55a80c5d10 2023-11-15T03:11:59.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.819+0000 7f55a5d9b700 1 --2- 172.21.15.121:0/1463740635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a80c5d10 0x7f55a8066530 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 rx=0x7f559c009760 tx=0x7f559c009060).ready entity=mon.0 client_cookie=24f1263f9448360c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:59.832 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.819+0000 7f55a559a700 1 -- 172.21.15.121:0/1463740635 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559c00f510 con 0x7f55a80c5d10 2023-11-15T03:11:59.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.819+0000 7f55a559a700 1 -- 172.21.15.121:0/1463740635 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559c00f670 con 0x7f55a80c5d10 2023-11-15T03:11:59.833 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.819+0000 7f55a559a700 1 -- 172.21.15.121:0/1463740635 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559c00f9f0 con 0x7f55a80c5d10 2023-11-15T03:11:59.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.819+0000 7f55a759e700 1 -- 172.21.15.121:0/1463740635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a80c5d10 msgr2=0x7f55a8066530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.819+0000 7f55a759e700 1 --2- 172.21.15.121:0/1463740635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a80c5d10 0x7f55a8066530 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 rx=0x7f559c009760 tx=0x7f559c009060).stop 2023-11-15T03:11:59.835 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.823+0000 7f55a759e700 1 -- 172.21.15.121:0/1463740635 shutdown_connections 2023-11-15T03:11:59.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.823+0000 7f55a759e700 1 --2- 172.21.15.121:0/1463740635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a80c5d10 0x7f55a8066530 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.823+0000 7f55a759e700 1 --2- 172.21.15.121:0/1463740635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a8153a60 0x7f55a8153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.823+0000 7f55a759e700 1 -- 172.21.15.121:0/1463740635 >> 172.21.15.121:0/1463740635 conn(0x7f55a8067a60 msgr2=0x7f55a8067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:11:59.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.823+0000 7f55a759e700 1 -- 172.21.15.121:0/1463740635 shutdown_connections 2023-11-15T03:11:59.836 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.823+0000 7f55a759e700 1 -- 172.21.15.121:0/1463740635 wait complete. 2023-11-15T03:11:59.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.823+0000 7f55a759e700 1 Processor -- start 2023-11-15T03:11:59.837 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.827+0000 7f55a759e700 1 -- start start 2023-11-15T03:11:59.838 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.827+0000 7f55a759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a80bfe40 0x7f55a80acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.827+0000 7f55a759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a80c5d10 0x7f55a80c4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.827+0000 7f55a759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a8153a60 0x7f55a80c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.827+0000 7f55a759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a80c3550 con 0x7f55a80c5d10 2023-11-15T03:11:59.839 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.827+0000 7f55a759e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a80c2940 con 0x7f55a80bfe40 2023-11-15T03:11:59.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.827+0000 7f55a759e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a80c2c20 con 0x7f55a8153a60 2023-11-15T03:11:59.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.827+0000 7f55a5d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a80c5d10 0x7f55a80c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.827+0000 7f55a5d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a80c5d10 0x7f55a80c4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54356/0 (socket says 172.21.15.121:54356) 2023-11-15T03:11:59.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.827+0000 7f55a5d9b700 1 -- 172.21.15.121:0/1734047932 learned_addr learned my addr 172.21.15.121:0/1734047932 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:11:59.840 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.827+0000 7f55a659c700 1 --2- 172.21.15.121:0/1734047932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a80bfe40 0x7f55a80acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.827+0000 7f55a6d9d700 1 --2- 172.21.15.121:0/1734047932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a8153a60 0x7f55a80c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.827+0000 7f55a5d9b700 1 -- 172.21.15.121:0/1734047932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a8153a60 msgr2=0x7f55a80c39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.841 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.827+0000 7f55a5d9b700 1 --2- 172.21.15.121:0/1734047932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a8153a60 0x7f55a80c39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.827+0000 7f55a5d9b700 1 -- 172.21.15.121:0/1734047932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a80bfe40 msgr2=0x7f55a80acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:11:59.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.827+0000 7f55a5d9b700 1 --2- 172.21.15.121:0/1734047932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a80bfe40 0x7f55a80acd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:11:59.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.827+0000 7f55a5d9b700 1 -- 172.21.15.121:0/1734047932 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c0092a0 con 0x7f55a80c5d10 2023-11-15T03:11:59.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.827+0000 7f55a659c700 1 --2- 172.21.15.121:0/1734047932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a80bfe40 0x7f55a80acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:11:59.842 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.827+0000 7f55a5d9b700 1 --2- 172.21.15.121:0/1734047932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a80c5d10 0x7f55a80c4450 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 rx=0x7f559c011000 tx=0x7f559c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:59.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.831+0000 7f55967fc700 1 -- 172.21.15.121:0/1734047932 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559c00f7e0 con 0x7f55a80c5d10 2023-11-15T03:11:59.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.831+0000 7f55967fc700 1 -- 172.21.15.121:0/1734047932 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559c00f940 con 0x7f55a80c5d10 2023-11-15T03:11:59.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.831+0000 7f55a759e700 1 -- 172.21.15.121:0/1734047932 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a80c2f00 con 0x7f55a80c5d10 2023-11-15T03:11:59.844 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.831+0000 7f55967fc700 1 -- 172.21.15.121:0/1734047932 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559c01b660 con 0x7f55a80c5d10 2023-11-15T03:11:59.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.831+0000 7f55a759e700 1 -- 172.21.15.121:0/1734047932 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a80ad210 con 0x7f55a80c5d10 2023-11-15T03:11:59.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.831+0000 7f55a759e700 1 -- 172.21.15.121:0/1734047932 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a806bcf0 con 0x7f55a80c5d10 2023-11-15T03:11:59.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.831+0000 7f55967fc700 1 -- 172.21.15.121:0/1734047932 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f559c01ba90 con 0x7f55a80c5d10 2023-11-15T03:11:59.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.835+0000 7f55967fc700 1 --2- 172.21.15.121:0/1734047932 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f558805a510 0x7f558805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:11:59.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.835+0000 7f55967fc700 1 -- 172.21.15.121:0/1734047932 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f559c0761b0 con 0x7f55a80c5d10 2023-11-15T03:11:59.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.835+0000 7f55a659c700 1 --2- 172.21.15.121:0/1734047932 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f558805a510 0x7f558805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:11:59.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.835+0000 7f55a659c700 1 --2- 172.21.15.121:0/1734047932 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f558805a510 0x7f558805c9d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f5598009b60 tx=0x7f5598008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:11:59.848 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.839+0000 7f55967fc700 1 -- 172.21.15.121:0/1734047932 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f559c049b70 con 0x7f55a80c5d10 2023-11-15T03:12:00.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.999+0000 7f55a759e700 1 -- 172.21.15.121:0/1734047932 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f55a8068a70 con 0x7f558805a510 2023-11-15T03:12:00.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:11:59.999+0000 7f55967fc700 1 -- 172.21.15.121:0/1734047932 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f55a8068a70 con 0x7f558805a510 2023-11-15T03:12:00.011 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.003+0000 7f55a759e700 1 -- 172.21.15.121:0/1734047932 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f558805a510 msgr2=0x7f558805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.003+0000 7f55a759e700 1 --2- 172.21.15.121:0/1734047932 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f558805a510 0x7f558805c9d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f5598009b60 tx=0x7f5598008040).stop 2023-11-15T03:12:00.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.003+0000 7f55a759e700 1 -- 172.21.15.121:0/1734047932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a80c5d10 msgr2=0x7f55a80c4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.003+0000 7f55a759e700 1 --2- 172.21.15.121:0/1734047932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a80c5d10 0x7f55a80c4450 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 rx=0x7f559c011000 tx=0x7f559c017000).stop 2023-11-15T03:12:00.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.003+0000 7f55a759e700 1 -- 172.21.15.121:0/1734047932 shutdown_connections 2023-11-15T03:12:00.012 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.003+0000 7f55a759e700 1 --2- 172.21.15.121:0/1734047932 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f558805a510 0x7f558805c9d0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.003+0000 7f55a759e700 1 --2- 172.21.15.121:0/1734047932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a8153a60 0x7f55a80c39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.003+0000 7f55a759e700 1 --2- 172.21.15.121:0/1734047932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55a80c5d10 0x7f55a80c4450 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.003+0000 7f55a759e700 1 --2- 172.21.15.121:0/1734047932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a80bfe40 0x7f55a80acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.003+0000 7f55a759e700 1 -- 172.21.15.121:0/1734047932 >> 172.21.15.121:0/1734047932 conn(0x7f55a8067a60 msgr2=0x7f55a8159d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:12:00.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.003+0000 7f55a759e700 1 -- 172.21.15.121:0/1734047932 shutdown_connections 2023-11-15T03:12:00.013 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.003+0000 7f55a759e700 1 -- 172.21.15.121:0/1734047932 wait complete. 2023-11-15T03:12:00.014 INFO:teuthology.orchestra.run.smithi121.stderr:instructed osd(s) 5 to deep-scrub 2023-11-15T03:12:00.036 INFO:tasks.ceph:Scrubbing osd.6 2023-11-15T03:12:00.037 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-15T03:12:00.196 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.187+0000 7fbf018db700 1 Processor -- start 2023-11-15T03:12:00.197 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.187+0000 7fbf018db700 1 -- start start 2023-11-15T03:12:00.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.191+0000 7fbf018db700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbefc157490 0x7fbefc065950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:00.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.191+0000 7fbf018db700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbefc062da0 con 0x7fbefc0c5c80 2023-11-15T03:12:00.201 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.191+0000 7fbf018db700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbefc0bfb90 con 0x7fbefc1570b0 2023-11-15T03:12:00.202 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.191+0000 7fbf018db700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbefc0c6140 con 0x7fbefc157490 2023-11-15T03:12:00.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.191+0000 7fbefa7fc700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbefc157490 0x7fbefc065950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:00.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.191+0000 7fbefaffd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fbefc1570b0 0x7fbefc066860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:50272/0 (socket says 172.21.15.121:50272) 2023-11-15T03:12:00.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.191+0000 7fbefaffd700 1 -- 172.21.15.121:0/232667537 learned_addr learned my addr 172.21.15.121:0/232667537 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:12:00.203 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.191+0000 7fbef9ffb700 1 -- 172.21.15.121:0/232667537 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506469244 0 0) 0x7fbefc0bfb90 con 0x7fbefc1570b0 2023-11-15T03:12:00.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.191+0000 7fbef9ffb700 1 -- 172.21.15.121:0/232667537 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbee40033f0 con 0x7fbefc1570b0 2023-11-15T03:12:00.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.191+0000 7fbef9ffb700 1 -- 172.21.15.121:0/232667537 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2121736226 0 0) 0x7fbefc062da0 con 0x7fbefc0c5c80 2023-11-15T03:12:00.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.191+0000 7fbef9ffb700 1 -- 172.21.15.121:0/232667537 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbefc0bfb90 con 0x7fbefc0c5c80 2023-11-15T03:12:00.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.191+0000 7fbefa7fc700 1 -- 172.21.15.121:0/232667537 >> v1:172.21.15.190:6789/0 conn(0x7fbefc1570b0 legacy=0x7fbefc066860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.204 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.191+0000 7fbefa7fc700 1 -- 172.21.15.121:0/232667537 >> v1:172.21.15.121:6789/0 conn(0x7fbefc0c5c80 legacy=0x7fbefc063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.205 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.191+0000 7fbefa7fc700 1 -- 172.21.15.121:0/232667537 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbefc0c6420 con 0x7fbefc157490 2023-11-15T03:12:00.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.195+0000 7fbefa7fc700 1 --2- 172.21.15.121:0/232667537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbefc157490 0x7fbefc065950 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 rx=0x7fbef0009760 tx=0x7fbef0009060).ready entity=mon.0 client_cookie=83281cbc704ed46b server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:00.206 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.195+0000 7fbef9ffb700 1 -- 172.21.15.121:0/232667537 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef000d4a0 con 0x7fbefc157490 2023-11-15T03:12:00.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.195+0000 7fbef9ffb700 1 -- 172.21.15.121:0/232667537 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef000d600 con 0x7fbefc157490 2023-11-15T03:12:00.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.195+0000 7fbef9ffb700 1 -- 172.21.15.121:0/232667537 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef000d980 con 0x7fbefc157490 2023-11-15T03:12:00.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.195+0000 7fbf018db700 1 -- 172.21.15.121:0/232667537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbefc157490 msgr2=0x7fbefc065950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.195+0000 7fbf018db700 1 --2- 172.21.15.121:0/232667537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbefc157490 0x7fbefc065950 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 rx=0x7fbef0009760 tx=0x7fbef0009060).stop 2023-11-15T03:12:00.207 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.195+0000 7fbf018db700 1 -- 172.21.15.121:0/232667537 shutdown_connections 2023-11-15T03:12:00.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.195+0000 7fbf018db700 1 --2- 172.21.15.121:0/232667537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbefc157490 0x7fbefc065950 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.195+0000 7fbf018db700 1 -- 172.21.15.121:0/232667537 >> 172.21.15.121:0/232667537 conn(0x7fbefc0684f0 msgr2=0x7fbefc0cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:12:00.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.195+0000 7fbf018db700 1 -- 172.21.15.121:0/232667537 shutdown_connections 2023-11-15T03:12:00.208 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.195+0000 7fbf018db700 1 -- 172.21.15.121:0/232667537 wait complete. 2023-11-15T03:12:00.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.199+0000 7fbf018db700 1 Processor -- start 2023-11-15T03:12:00.209 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.199+0000 7fbf018db700 1 -- start start 2023-11-15T03:12:00.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.199+0000 7fbf018db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbefc0c5c80 0x7fbefc153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:00.210 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.199+0000 7fbf018db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbefc1570b0 0x7fbefc155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:00.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.199+0000 7fbf018db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbefc152440 0x7fbefc158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:00.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.199+0000 7fbf018db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc13e0b0 con 0x7fbefc1570b0 2023-11-15T03:12:00.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.199+0000 7fbf018db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc1538e0 con 0x7fbefc152440 2023-11-15T03:12:00.211 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.199+0000 7fbf018db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc152900 con 0x7fbefc0c5c80 2023-11-15T03:12:00.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.199+0000 7fbefa7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbefc1570b0 0x7fbefc155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:00.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.203+0000 7fbefa7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbefc1570b0 0x7fbefc155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54370/0 (socket says 172.21.15.121:54370) 2023-11-15T03:12:00.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.203+0000 7fbefa7fc700 1 -- 172.21.15.121:0/3180816722 learned_addr learned my addr 172.21.15.121:0/3180816722 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:12:00.213 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.203+0000 7fbefaffd700 1 --2- 172.21.15.121:0/3180816722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbefc0c5c80 0x7fbefc153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:00.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.203+0000 7fbefb7fe700 1 --2- 172.21.15.121:0/3180816722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbefc152440 0x7fbefc158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:00.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.203+0000 7fbefa7fc700 1 -- 172.21.15.121:0/3180816722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbefc0c5c80 msgr2=0x7fbefc153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.203+0000 7fbefa7fc700 1 --2- 172.21.15.121:0/3180816722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbefc0c5c80 0x7fbefc153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.203+0000 7fbefa7fc700 1 -- 172.21.15.121:0/3180816722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbefc152440 msgr2=0x7fbefc158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.214 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.203+0000 7fbefa7fc700 1 --2- 172.21.15.121:0/3180816722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbefc152440 0x7fbefc158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.203+0000 7fbefa7fc700 1 -- 172.21.15.121:0/3180816722 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef00092a0 con 0x7fbefc1570b0 2023-11-15T03:12:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.203+0000 7fbefb7fe700 1 --2- 172.21.15.121:0/3180816722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbefc152440 0x7fbefc158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:12:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.203+0000 7fbefa7fc700 1 --2- 172.21.15.121:0/3180816722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbefc1570b0 0x7fbefc155ee0 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 rx=0x7fbefc0cbca0 tx=0x7fbef0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.203+0000 7fbee2ffd700 1 -- 172.21.15.121:0/3180816722 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef000dcf0 con 0x7fbefc1570b0 2023-11-15T03:12:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.203+0000 7fbee2ffd700 1 -- 172.21.15.121:0/3180816722 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef000de50 con 0x7fbefc1570b0 2023-11-15T03:12:00.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.203+0000 7fbf018db700 1 -- 172.21.15.121:0/3180816722 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbefc152be0 con 0x7fbefc1570b0 2023-11-15T03:12:00.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.203+0000 7fbee2ffd700 1 -- 172.21.15.121:0/3180816722 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef001d430 con 0x7fbefc1570b0 2023-11-15T03:12:00.216 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.203+0000 7fbf018db700 1 -- 172.21.15.121:0/3180816722 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbefc1531a0 con 0x7fbefc1570b0 2023-11-15T03:12:00.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.203+0000 7fbf018db700 1 -- 172.21.15.121:0/3180816722 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbefc0c2f20 con 0x7fbefc1570b0 2023-11-15T03:12:00.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.207+0000 7fbee2ffd700 1 -- 172.21.15.121:0/3180816722 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fbef001d7a0 con 0x7fbefc1570b0 2023-11-15T03:12:00.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.207+0000 7fbee2ffd700 1 --2- 172.21.15.121:0/3180816722 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbee805a510 0x7fbee805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:00.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.207+0000 7fbee2ffd700 1 -- 172.21.15.121:0/3180816722 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7fbef0075dc0 con 0x7fbefc1570b0 2023-11-15T03:12:00.218 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.207+0000 7fbefaffd700 1 --2- 172.21.15.121:0/3180816722 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbee805a510 0x7fbee805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:00.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.207+0000 7fbee2ffd700 1 --2- 172.21.15.121:0/3180816722 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7fbee8060ba0 0x7fbee8063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:00.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.207+0000 7fbee2ffd700 1 -- 172.21.15.121:0/3180816722 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbee8063680 con 0x7fbee8060ba0 2023-11-15T03:12:00.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.207+0000 7fbee2ffd700 1 -- 172.21.15.121:0/3180816722 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbef0078320 con 0x7fbefc1570b0 2023-11-15T03:12:00.219 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.207+0000 7fbefaffd700 1 --2- 172.21.15.121:0/3180816722 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbee805a510 0x7fbee805c9d0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7fbeec005750 tx=0x7fbeec009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:00.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.207+0000 7fbefb7fe700 1 --2- 172.21.15.121:0/3180816722 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7fbee8060ba0 0x7fbee8063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:00.220 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.211+0000 7fbefb7fe700 1 --2- 172.21.15.121:0/3180816722 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7fbee8060ba0 0x7fbee8063000 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:00.221 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.211+0000 7fbee2ffd700 1 -- 172.21.15.121:0/3180816722 <== osd.6 v2:172.21.15.190:6809/33346 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fbee8063680 con 0x7fbee8060ba0 2023-11-15T03:12:00.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.227+0000 7fbf018db700 1 -- 172.21.15.121:0/3180816722 --> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbefc06c150 con 0x7fbee8060ba0 2023-11-15T03:12:00.238 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.227+0000 7fbee2ffd700 1 -- 172.21.15.121:0/3180816722 <== osd.6 v2:172.21.15.190:6809/33346 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fbefc06c150 con 0x7fbee8060ba0 2023-11-15T03:12:00.239 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:12:00.239 INFO:teuthology.orchestra.run.smithi121.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-15T03:12:00.239 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:12:00.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.231+0000 7fbf018db700 1 -- 172.21.15.121:0/3180816722 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7fbee8060ba0 msgr2=0x7fbee8063000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.231+0000 7fbf018db700 1 --2- 172.21.15.121:0/3180816722 >> [v2:172.21.15.190:6809/33346,v1:172.21.15.190:6810/33346] conn(0x7fbee8060ba0 0x7fbee8063000 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.231+0000 7fbf018db700 1 -- 172.21.15.121:0/3180816722 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbee805a510 msgr2=0x7fbee805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.231+0000 7fbf018db700 1 --2- 172.21.15.121:0/3180816722 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7fbee805a510 0x7fbee805c9d0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7fbeec005750 tx=0x7fbeec009040).stop 2023-11-15T03:12:00.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.231+0000 7fbf018db700 1 -- 172.21.15.121:0/3180816722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbefc1570b0 msgr2=0x7fbefc155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.241 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.231+0000 7fbf018db700 1 --2- 172.21.15.121:0/3180816722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbefc1570b0 0x7fbefc155ee0 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 rx=0x7fbefc0cbca0 tx=0x7fbef0016000).stop 2023-11-15T03:12:00.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.231+0000 7fbefb7fe700 1 -- 172.21.15.121:0/3180816722 reap_dead start 2023-11-15T03:12:00.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.231+0000 7fbf018db700 1 -- 172.21.15.121:0/3180816722 shutdown_connections 2023-11-15T03:12:00.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.231+0000 7fbf018db700 1 -- 172.21.15.121:0/3180816722 >> 172.21.15.121:0/3180816722 conn(0x7fbefc0684f0 msgr2=0x7fbefc150760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:12:00.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.231+0000 7fbf018db700 1 -- 172.21.15.121:0/3180816722 shutdown_connections 2023-11-15T03:12:00.242 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.231+0000 7fbf018db700 1 -- 172.21.15.121:0/3180816722 wait complete. 2023-11-15T03:12:00.262 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-15T03:12:00.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.411+0000 7f9d0cd1b700 1 Processor -- start 2023-11-15T03:12:00.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.411+0000 7f9d0cd1b700 1 -- start start 2023-11-15T03:12:00.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.411+0000 7f9d0cd1b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9d08067150 0x7f9d080bb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:00.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.411+0000 7f9d0cd1b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d08155de0 con 0x7f9d08151920 2023-11-15T03:12:00.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.411+0000 7f9d0cd1b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d080c3aa0 con 0x7f9d080ac150 2023-11-15T03:12:00.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.411+0000 7f9d0cd1b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d080c4060 con 0x7f9d08067150 2023-11-15T03:12:00.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.415+0000 7f9d0659c700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9d08151920 0x7f9d080aba40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51178/0 (socket says 172.21.15.121:51178) 2023-11-15T03:12:00.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.415+0000 7f9d0659c700 1 -- 172.21.15.121:0/1335448913 learned_addr learned my addr 172.21.15.121:0/1335448913 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:12:00.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.415+0000 7f9d06d9d700 1 --2- 172.21.15.121:0/1335448913 >> v2:172.21.15.190:3300/0 conn(0x7f9d08067150 0x7f9d080bb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:00.425 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.415+0000 7f9d0559a700 1 -- 172.21.15.121:0/1335448913 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1653166858 0 0) 0x7f9d08155de0 con 0x7f9d08151920 2023-11-15T03:12:00.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.415+0000 7f9d0559a700 1 -- 172.21.15.121:0/1335448913 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cf00033f0 con 0x7f9d08151920 2023-11-15T03:12:00.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.415+0000 7f9d0559a700 1 -- 172.21.15.121:0/1335448913 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970412503 0 0) 0x7f9d080c3aa0 con 0x7f9d080ac150 2023-11-15T03:12:00.426 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.415+0000 7f9d0559a700 1 -- 172.21.15.121:0/1335448913 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d08155de0 con 0x7f9d080ac150 2023-11-15T03:12:00.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.415+0000 7f9d06d9d700 1 -- 172.21.15.121:0/1335448913 >> v1:172.21.15.190:6789/0 conn(0x7f9d080ac150 legacy=0x7f9d08062190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.415+0000 7f9d06d9d700 1 -- 172.21.15.121:0/1335448913 >> v1:172.21.15.121:6789/0 conn(0x7f9d08151920 legacy=0x7f9d080aba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.415+0000 7f9d06d9d700 1 -- 172.21.15.121:0/1335448913 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d080c3d80 con 0x7f9d08067150 2023-11-15T03:12:00.427 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.415+0000 7f9d06d9d700 1 --2- 172.21.15.121:0/1335448913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d08067150 0x7f9d080bb6b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f9d00009e60 tx=0x7f9d00009780).ready entity=mon.1 client_cookie=8d31db64d3b07692 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:00.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.415+0000 7f9d0559a700 1 -- 172.21.15.121:0/1335448913 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0000e070 con 0x7f9d08067150 2023-11-15T03:12:00.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.415+0000 7f9d0559a700 1 -- 172.21.15.121:0/1335448913 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0000bab0 con 0x7f9d08067150 2023-11-15T03:12:00.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.415+0000 7f9d0559a700 1 -- 172.21.15.121:0/1335448913 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0000be30 con 0x7f9d08067150 2023-11-15T03:12:00.429 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.415+0000 7f9d0cd1b700 1 -- 172.21.15.121:0/1335448913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d08067150 msgr2=0x7f9d080bb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.415+0000 7f9d0cd1b700 1 --2- 172.21.15.121:0/1335448913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d08067150 0x7f9d080bb6b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f9d00009e60 tx=0x7f9d00009780).stop 2023-11-15T03:12:00.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.415+0000 7f9d0cd1b700 1 -- 172.21.15.121:0/1335448913 shutdown_connections 2023-11-15T03:12:00.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.415+0000 7f9d0cd1b700 1 --2- 172.21.15.121:0/1335448913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d08067150 0x7f9d080bb6b0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.415+0000 7f9d0cd1b700 1 -- 172.21.15.121:0/1335448913 >> 172.21.15.121:0/1335448913 conn(0x7f9d08067a60 msgr2=0x7f9d08067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:12:00.430 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.419+0000 7f9d0cd1b700 1 -- 172.21.15.121:0/1335448913 shutdown_connections 2023-11-15T03:12:00.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.419+0000 7f9d0cd1b700 1 -- 172.21.15.121:0/1335448913 wait complete. 2023-11-15T03:12:00.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.419+0000 7f9d0cd1b700 1 Processor -- start 2023-11-15T03:12:00.431 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.419+0000 7f9d0cd1b700 1 -- start start 2023-11-15T03:12:00.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.419+0000 7f9d0cd1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d08067150 0x7f9d0814b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:00.433 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.419+0000 7f9d0cd1b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d080ac150 0x7f9d080a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:00.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.419+0000 7f9d0cd1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d08151920 0x7f9d08150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:00.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.419+0000 7f9d0cd1b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0814baa0 con 0x7f9d080ac150 2023-11-15T03:12:00.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.419+0000 7f9d0cd1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0814dfa0 con 0x7f9d08151920 2023-11-15T03:12:00.434 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.419+0000 7f9d0cd1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d08153c50 con 0x7f9d08067150 2023-11-15T03:12:00.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.419+0000 7f9d05d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d080ac150 0x7f9d080a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:00.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.419+0000 7f9d05d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d080ac150 0x7f9d080a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54384/0 (socket says 172.21.15.121:54384) 2023-11-15T03:12:00.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.419+0000 7f9d05d9b700 1 -- 172.21.15.121:0/2919509846 learned_addr learned my addr 172.21.15.121:0/2919509846 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:12:00.435 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.423+0000 7f9d0659c700 1 --2- 172.21.15.121:0/2919509846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d08067150 0x7f9d0814b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:00.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.423+0000 7f9d06d9d700 1 --2- 172.21.15.121:0/2919509846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d08151920 0x7f9d08150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:00.436 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.423+0000 7f9d05d9b700 1 -- 172.21.15.121:0/2919509846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d08067150 msgr2=0x7f9d0814b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.423+0000 7f9d05d9b700 1 --2- 172.21.15.121:0/2919509846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d08067150 0x7f9d0814b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.423+0000 7f9d05d9b700 1 -- 172.21.15.121:0/2919509846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d08151920 msgr2=0x7f9d08150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.423+0000 7f9d05d9b700 1 --2- 172.21.15.121:0/2919509846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d08151920 0x7f9d08150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.423+0000 7f9d05d9b700 1 -- 172.21.15.121:0/2919509846 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d00009a20 con 0x7f9d080ac150 2023-11-15T03:12:00.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.423+0000 7f9d06d9d700 1 --2- 172.21.15.121:0/2919509846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d08151920 0x7f9d08150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:12:00.437 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.423+0000 7f9d05d9b700 1 --2- 172.21.15.121:0/2919509846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d080ac150 0x7f9d080a84a0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 rx=0x7f9cfc00c490 tx=0x7f9cfc007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:00.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.423+0000 7f9cf67fc700 1 -- 172.21.15.121:0/2919509846 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc00f470 con 0x7f9d080ac150 2023-11-15T03:12:00.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.423+0000 7f9cf67fc700 1 -- 172.21.15.121:0/2919509846 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc00f5d0 con 0x7f9d080ac150 2023-11-15T03:12:00.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.423+0000 7f9d0cd1b700 1 -- 172.21.15.121:0/2919509846 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d08153f30 con 0x7f9d080ac150 2023-11-15T03:12:00.438 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.423+0000 7f9cf67fc700 1 -- 172.21.15.121:0/2919509846 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc00f990 con 0x7f9d080ac150 2023-11-15T03:12:00.439 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.423+0000 7f9d0cd1b700 1 -- 172.21.15.121:0/2919509846 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d081544f0 con 0x7f9d080ac150 2023-11-15T03:12:00.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.423+0000 7f9d0cd1b700 1 -- 172.21.15.121:0/2919509846 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0814e130 con 0x7f9d080ac150 2023-11-15T03:12:00.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.427+0000 7f9cf67fc700 1 -- 172.21.15.121:0/2919509846 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9cfc00fd40 con 0x7f9d080ac150 2023-11-15T03:12:00.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.427+0000 7f9cf67fc700 1 --2- 172.21.15.121:0/2919509846 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ce805a510 0x7f9ce805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:00.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.427+0000 7f9cf67fc700 1 -- 172.21.15.121:0/2919509846 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f9cfc073ae0 con 0x7f9d080ac150 2023-11-15T03:12:00.440 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.427+0000 7f9d0659c700 1 --2- 172.21.15.121:0/2919509846 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ce805a510 0x7f9ce805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:00.441 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.427+0000 7f9d0659c700 1 --2- 172.21.15.121:0/2919509846 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ce805a510 0x7f9ce805c9d0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7f9cf8005750 tx=0x7f9cf8009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:00.444 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.435+0000 7f9cf67fc700 1 -- 172.21.15.121:0/2919509846 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9cfc047560 con 0x7f9d080ac150 2023-11-15T03:12:00.609 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.599+0000 7f9d0cd1b700 1 -- 172.21.15.121:0/2919509846 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d080c35c0 con 0x7f9ce805a510 2023-11-15T03:12:00.610 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.599+0000 7f9cf67fc700 1 -- 172.21.15.121:0/2919509846 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9d080c35c0 con 0x7f9ce805a510 2023-11-15T03:12:00.612 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.603+0000 7f9d0cd1b700 1 -- 172.21.15.121:0/2919509846 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ce805a510 msgr2=0x7f9ce805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.603+0000 7f9d0cd1b700 1 --2- 172.21.15.121:0/2919509846 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ce805a510 0x7f9ce805c9d0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7f9cf8005750 tx=0x7f9cf8009040).stop 2023-11-15T03:12:00.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.603+0000 7f9d0cd1b700 1 -- 172.21.15.121:0/2919509846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d080ac150 msgr2=0x7f9d080a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.603+0000 7f9d0cd1b700 1 --2- 172.21.15.121:0/2919509846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d080ac150 0x7f9d080a84a0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 rx=0x7f9cfc00c490 tx=0x7f9cfc007c60).stop 2023-11-15T03:12:00.613 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.603+0000 7f9d0cd1b700 1 -- 172.21.15.121:0/2919509846 shutdown_connections 2023-11-15T03:12:00.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.603+0000 7f9d0cd1b700 1 --2- 172.21.15.121:0/2919509846 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f9ce805a510 0x7f9ce805c9d0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.603+0000 7f9d0cd1b700 1 --2- 172.21.15.121:0/2919509846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d08151920 0x7f9d08150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.603+0000 7f9d0cd1b700 1 --2- 172.21.15.121:0/2919509846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d080ac150 0x7f9d080a84a0 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.603+0000 7f9d0cd1b700 1 --2- 172.21.15.121:0/2919509846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d08067150 0x7f9d0814b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.614 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.603+0000 7f9d0cd1b700 1 -- 172.21.15.121:0/2919509846 >> 172.21.15.121:0/2919509846 conn(0x7f9d08067a60 msgr2=0x7f9d080ab770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:12:00.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.603+0000 7f9d0cd1b700 1 -- 172.21.15.121:0/2919509846 shutdown_connections 2023-11-15T03:12:00.615 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.603+0000 7f9d0cd1b700 1 -- 172.21.15.121:0/2919509846 wait complete. 2023-11-15T03:12:00.615 INFO:teuthology.orchestra.run.smithi121.stderr:instructed osd(s) 6 to deep-scrub 2023-11-15T03:12:00.632 INFO:tasks.ceph:Scrubbing osd.7 2023-11-15T03:12:00.633 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-15T03:12:00.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.791+0000 7f0ef4a5b700 1 Processor -- start 2023-11-15T03:12:00.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.791+0000 7f0ef4a5b700 1 -- start start 2023-11-15T03:12:00.804 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.795+0000 7f0ef4a5b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0ef01570b0 0x7f0ef0066860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:00.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.795+0000 7f0ef4a5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef0157490 0x7f0ef0065950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:00.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.795+0000 7f0ef4a5b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef0062da0 con 0x7f0ef00c5c80 2023-11-15T03:12:00.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.795+0000 7f0ef4a5b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ef00bfb90 con 0x7f0ef01570b0 2023-11-15T03:12:00.805 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.795+0000 7f0ef4a5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef00c6140 con 0x7f0ef0157490 2023-11-15T03:12:00.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.795+0000 7f0eee59c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0ef01570b0 0x7f0ef0066860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:00.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.795+0000 7f0eee59c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0ef01570b0 0x7f0ef0066860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.121:41700/0 (socket says 172.21.15.121:41700) 2023-11-15T03:12:00.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.795+0000 7f0eee59c700 1 -- 172.21.15.121:0/3350755385 learned_addr learned my addr 172.21.15.121:0/3350755385 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:12:00.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.795+0000 7f0eeed9d700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f0ef00c5c80 0x7f0ef01578f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51180/0 (socket says 172.21.15.121:51180) 2023-11-15T03:12:00.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.795+0000 7f0eedd9b700 1 --2- 172.21.15.121:0/3350755385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef0157490 0x7f0ef0065950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:00.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.795+0000 7f0eed59a700 1 -- 172.21.15.121:0/3350755385 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1013556552 0 0) 0x7f0ef0062da0 con 0x7f0ef00c5c80 2023-11-15T03:12:00.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.795+0000 7f0eee59c700 1 -- 172.21.15.121:0/3350755385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef0157490 msgr2=0x7f0ef0065950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:12:00.808 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.795+0000 7f0eee59c700 1 --2- 172.21.15.121:0/3350755385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef0157490 0x7f0ef0065950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.795+0000 7f0eee59c700 1 -- 172.21.15.121:0/3350755385 >> v1:172.21.15.121:6789/0 conn(0x7f0ef00c5c80 legacy=0x7f0ef01578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.795+0000 7f0eee59c700 1 -- 172.21.15.121:0/3350755385 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef00c6420 con 0x7f0ef01570b0 2023-11-15T03:12:00.809 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.795+0000 7f0eee59c700 1 --2- 172.21.15.121:0/3350755385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef01570b0 0x7f0ef0066860 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7f0ee0006fd0 tx=0x7f0ee00090f0).ready entity=mon.1 client_cookie=681c7f832d7e97e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:00.810 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.799+0000 7f0eed59a700 1 -- 172.21.15.121:0/3350755385 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee000b580 con 0x7f0ef01570b0 2023-11-15T03:12:00.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.799+0000 7f0eed59a700 1 -- 172.21.15.121:0/3350755385 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee000b6e0 con 0x7f0ef01570b0 2023-11-15T03:12:00.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.799+0000 7f0eed59a700 1 -- 172.21.15.121:0/3350755385 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee000ba60 con 0x7f0ef01570b0 2023-11-15T03:12:00.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.799+0000 7f0ef4a5b700 1 -- 172.21.15.121:0/3350755385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef01570b0 msgr2=0x7f0ef0066860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.799+0000 7f0ef4a5b700 1 --2- 172.21.15.121:0/3350755385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef01570b0 0x7f0ef0066860 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7f0ee0006fd0 tx=0x7f0ee00090f0).stop 2023-11-15T03:12:00.811 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.799+0000 7f0ef4a5b700 1 -- 172.21.15.121:0/3350755385 shutdown_connections 2023-11-15T03:12:00.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.799+0000 7f0ef4a5b700 1 --2- 172.21.15.121:0/3350755385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef0157490 0x7f0ef0065950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.799+0000 7f0ef4a5b700 1 --2- 172.21.15.121:0/3350755385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef01570b0 0x7f0ef0066860 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.799+0000 7f0ef4a5b700 1 -- 172.21.15.121:0/3350755385 >> 172.21.15.121:0/3350755385 conn(0x7f0ef00684f0 msgr2=0x7f0ef00cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:12:00.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.799+0000 7f0ef4a5b700 1 -- 172.21.15.121:0/3350755385 shutdown_connections 2023-11-15T03:12:00.812 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.799+0000 7f0ef4a5b700 1 -- 172.21.15.121:0/3350755385 wait complete. 2023-11-15T03:12:00.814 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.803+0000 7f0ef4a5b700 1 Processor -- start 2023-11-15T03:12:00.815 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.803+0000 7f0ef4a5b700 1 -- start start 2023-11-15T03:12:00.818 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0ef4a5b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ef00c5c80 0x7f0ef0167a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:00.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0ef4a5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef01570b0 0x7f0ef0167f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:00.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0ef4a5b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef00c34d0 0x7f0ef00c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:00.819 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0ef4a5b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef0168470 con 0x7f0ef00c5c80 2023-11-15T03:12:00.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0ef4a5b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef00ac260 con 0x7f0ef00c34d0 2023-11-15T03:12:00.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0ef4a5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef00ab210 con 0x7f0ef01570b0 2023-11-15T03:12:00.820 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0eee59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ef00c5c80 0x7f0ef0167a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:00.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0eee59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ef00c5c80 0x7f0ef0167a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54398/0 (socket says 172.21.15.121:54398) 2023-11-15T03:12:00.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0eee59c700 1 -- 172.21.15.121:0/1553950060 learned_addr learned my addr 172.21.15.121:0/1553950060 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:12:00.821 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0eeed9d700 1 --2- 172.21.15.121:0/1553950060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef00c34d0 0x7f0ef00c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:00.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0eedd9b700 1 --2- 172.21.15.121:0/1553950060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef01570b0 0x7f0ef0167f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:00.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0eee59c700 1 -- 172.21.15.121:0/1553950060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef01570b0 msgr2=0x7f0ef0167f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0eee59c700 1 --2- 172.21.15.121:0/1553950060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef01570b0 0x7f0ef0167f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.822 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0eee59c700 1 -- 172.21.15.121:0/1553950060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef00c34d0 msgr2=0x7f0ef00c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0eee59c700 1 --2- 172.21.15.121:0/1553950060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef00c34d0 0x7f0ef00c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0eee59c700 1 -- 172.21.15.121:0/1553950060 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee0009370 con 0x7f0ef00c5c80 2023-11-15T03:12:00.823 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0eeed9d700 1 --2- 172.21.15.121:0/1553950060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef00c34d0 0x7f0ef00c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:12:00.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0eedd9b700 1 --2- 172.21.15.121:0/1553950060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ef01570b0 0x7f0ef0167f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:12:00.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0eee59c700 1 --2- 172.21.15.121:0/1553950060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ef00c5c80 0x7f0ef0167a90 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 rx=0x7f0ef0065640 tx=0x7f0ee0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:00.824 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0ede7fc700 1 -- 172.21.15.121:0/1553950060 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee000b850 con 0x7f0ef00c5c80 2023-11-15T03:12:00.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0ede7fc700 1 -- 172.21.15.121:0/1553950060 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee000b9b0 con 0x7f0ef00c5c80 2023-11-15T03:12:00.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0ede7fc700 1 -- 172.21.15.121:0/1553950060 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee001c670 con 0x7f0ef00c5c80 2023-11-15T03:12:00.825 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0ef4a5b700 1 -- 172.21.15.121:0/1553950060 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef00ab4a0 con 0x7f0ef00c5c80 2023-11-15T03:12:00.826 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.807+0000 7f0ef4a5b700 1 -- 172.21.15.121:0/1553950060 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef00aba60 con 0x7f0ef00c5c80 2023-11-15T03:12:00.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.811+0000 7f0ef4a5b700 1 -- 172.21.15.121:0/1553950060 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ef006c150 con 0x7f0ef00c5c80 2023-11-15T03:12:00.828 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.811+0000 7f0ede7fc700 1 -- 172.21.15.121:0/1553950060 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0ee001ca60 con 0x7f0ef00c5c80 2023-11-15T03:12:00.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.811+0000 7f0ede7fc700 1 --2- 172.21.15.121:0/1553950060 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0ed005a510 0x7f0ed005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:00.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.811+0000 7f0ede7fc700 1 -- 172.21.15.121:0/1553950060 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f0ee0076300 con 0x7f0ef00c5c80 2023-11-15T03:12:00.829 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.811+0000 7f0ede7fc700 1 --2- 172.21.15.121:0/1553950060 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x7f0ed0060ba0 0x7f0ed0063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:00.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.811+0000 7f0ede7fc700 1 -- 172.21.15.121:0/1553950060 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0ed0063680 con 0x7f0ed0060ba0 2023-11-15T03:12:00.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.811+0000 7f0ede7fc700 1 -- 172.21.15.121:0/1553950060 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ee0078920 con 0x7f0ef00c5c80 2023-11-15T03:12:00.830 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.811+0000 7f0eedd9b700 1 --2- 172.21.15.121:0/1553950060 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0ed005a510 0x7f0ed005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:00.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.811+0000 7f0eeed9d700 1 --2- 172.21.15.121:0/1553950060 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x7f0ed0060ba0 0x7f0ed0063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:00.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.815+0000 7f0eedd9b700 1 --2- 172.21.15.121:0/1553950060 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0ed005a510 0x7f0ed005c9d0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7f0ee4006d80 tx=0x7f0ee4008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:00.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.815+0000 7f0eeed9d700 1 --2- 172.21.15.121:0/1553950060 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x7f0ed0060ba0 0x7f0ed0063000 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:00.831 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.815+0000 7f0ede7fc700 1 -- 172.21.15.121:0/1553950060 <== osd.7 v2:172.21.15.190:6808/33347 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0ed0063680 con 0x7f0ed0060ba0 2023-11-15T03:12:00.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.831+0000 7f0ef4a5b700 1 -- 172.21.15.121:0/1553950060 --> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0ef00665d0 con 0x7f0ed0060ba0 2023-11-15T03:12:00.843 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.835+0000 7f0ede7fc700 1 -- 172.21.15.121:0/1553950060 <== osd.7 v2:172.21.15.190:6808/33347 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0ef00665d0 con 0x7f0ed0060ba0 2023-11-15T03:12:00.844 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-11-15T03:12:00.844 INFO:teuthology.orchestra.run.smithi121.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-15T03:12:00.844 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-11-15T03:12:00.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.835+0000 7f0ef4a5b700 1 -- 172.21.15.121:0/1553950060 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x7f0ed0060ba0 msgr2=0x7f0ed0063000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.845 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.835+0000 7f0ef4a5b700 1 --2- 172.21.15.121:0/1553950060 >> [v2:172.21.15.190:6808/33347,v1:172.21.15.190:6811/33347] conn(0x7f0ed0060ba0 0x7f0ed0063000 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:00.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.835+0000 7f0ef4a5b700 1 -- 172.21.15.121:0/1553950060 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0ed005a510 msgr2=0x7f0ed005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.835+0000 7f0ef4a5b700 1 --2- 172.21.15.121:0/1553950060 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0ed005a510 0x7f0ed005c9d0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7f0ee4006d80 tx=0x7f0ee4008040).stop 2023-11-15T03:12:00.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.835+0000 7f0ef4a5b700 1 -- 172.21.15.121:0/1553950060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ef00c5c80 msgr2=0x7f0ef0167a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:00.846 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.835+0000 7f0ef4a5b700 1 --2- 172.21.15.121:0/1553950060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ef00c5c80 0x7f0ef0167a90 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 rx=0x7f0ef0065640 tx=0x7f0ee0017000).stop 2023-11-15T03:12:00.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.835+0000 7f0eeed9d700 1 -- 172.21.15.121:0/1553950060 reap_dead start 2023-11-15T03:12:00.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.835+0000 7f0ef4a5b700 1 -- 172.21.15.121:0/1553950060 shutdown_connections 2023-11-15T03:12:00.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.835+0000 7f0ef4a5b700 1 -- 172.21.15.121:0/1553950060 >> 172.21.15.121:0/1553950060 conn(0x7f0ef00684f0 msgr2=0x7f0ef0065950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:12:00.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.835+0000 7f0ef4a5b700 1 -- 172.21.15.121:0/1553950060 shutdown_connections 2023-11-15T03:12:00.847 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:00.835+0000 7f0ef4a5b700 1 -- 172.21.15.121:0/1553950060 wait complete. 2023-11-15T03:12:00.866 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-15T03:12:01.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.019+0000 7f0e8e107700 1 Processor -- start 2023-11-15T03:12:01.031 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.023+0000 7f0e8e107700 1 -- start start 2023-11-15T03:12:01.033 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.023+0000 7f0e8e107700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0e881511f0 0x7f0e88066570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:01.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.023+0000 7f0e8e107700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e88066a60 con 0x7f0e880bcab0 2023-11-15T03:12:01.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.023+0000 7f0e8e107700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e880660e0 con 0x7f0e880a8750 2023-11-15T03:12:01.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.023+0000 7f0e8e107700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e880bd390 con 0x7f0e881511f0 2023-11-15T03:12:01.034 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.023+0000 7f0e8c904700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f0e880bcab0 0x7f0e881515d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51188/0 (socket says 172.21.15.121:51188) 2023-11-15T03:12:01.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.023+0000 7f0e8c904700 1 -- 172.21.15.121:0/416143084 learned_addr learned my addr 172.21.15.121:0/416143084 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:12:01.035 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.023+0000 7f0e8d105700 1 --2- 172.21.15.121:0/416143084 >> v2:172.21.15.190:3300/0 conn(0x7f0e881511f0 0x7f0e88066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:01.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.023+0000 7f0e7ffff700 1 -- 172.21.15.121:0/416143084 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2669013003 0 0) 0x7f0e88066a60 con 0x7f0e880bcab0 2023-11-15T03:12:01.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.023+0000 7f0e7ffff700 1 -- 172.21.15.121:0/416143084 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e740033f0 con 0x7f0e880bcab0 2023-11-15T03:12:01.037 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.023+0000 7f0e8d105700 1 -- 172.21.15.121:0/416143084 >> v1:172.21.15.190:6789/0 conn(0x7f0e880a8750 legacy=0x7f0e880a8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:01.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.023+0000 7f0e8d105700 1 -- 172.21.15.121:0/416143084 >> v1:172.21.15.121:6789/0 conn(0x7f0e880bcab0 legacy=0x7f0e881515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:01.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.023+0000 7f0e8d105700 1 -- 172.21.15.121:0/416143084 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e880657b0 con 0x7f0e881511f0 2023-11-15T03:12:01.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.023+0000 7f0e8d105700 1 --2- 172.21.15.121:0/416143084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e881511f0 0x7f0e88066570 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f0e84009760 tx=0x7f0e84009060).ready entity=mon.1 client_cookie=ab5ee64519469beb server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:01.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.027+0000 7f0e7ffff700 1 -- 172.21.15.121:0/416143084 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8400f510 con 0x7f0e881511f0 2023-11-15T03:12:01.038 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.027+0000 7f0e7ffff700 1 -- 172.21.15.121:0/416143084 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8400f670 con 0x7f0e881511f0 2023-11-15T03:12:01.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.027+0000 7f0e7ffff700 1 -- 172.21.15.121:0/416143084 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8400f9f0 con 0x7f0e881511f0 2023-11-15T03:12:01.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.027+0000 7f0e8e107700 1 -- 172.21.15.121:0/416143084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e881511f0 msgr2=0x7f0e88066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:01.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.027+0000 7f0e8e107700 1 --2- 172.21.15.121:0/416143084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e881511f0 0x7f0e88066570 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f0e84009760 tx=0x7f0e84009060).stop 2023-11-15T03:12:01.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.027+0000 7f0e8e107700 1 -- 172.21.15.121:0/416143084 shutdown_connections 2023-11-15T03:12:01.039 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.027+0000 7f0e8e107700 1 --2- 172.21.15.121:0/416143084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e881511f0 0x7f0e88066570 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:01.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.027+0000 7f0e8e107700 1 -- 172.21.15.121:0/416143084 >> 172.21.15.121:0/416143084 conn(0x7f0e88067a60 msgr2=0x7f0e88067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:12:01.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.027+0000 7f0e8e107700 1 -- 172.21.15.121:0/416143084 shutdown_connections 2023-11-15T03:12:01.040 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.027+0000 7f0e8e107700 1 -- 172.21.15.121:0/416143084 wait complete. 2023-11-15T03:12:01.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.031+0000 7f0e8e107700 1 Processor -- start 2023-11-15T03:12:01.041 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.031+0000 7f0e8e107700 1 -- start start 2023-11-15T03:12:01.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8e107700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e880a8750 0x7f0e880c1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:01.043 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8e107700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e880bcab0 0x7f0e88164e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:01.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8e107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e881511f0 0x7f0e880a7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:01.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8e107700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e881545c0 con 0x7f0e880a8750 2023-11-15T03:12:01.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8e107700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e88151dc0 con 0x7f0e880bcab0 2023-11-15T03:12:01.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8e107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880c1c40 con 0x7f0e881511f0 2023-11-15T03:12:01.044 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8d105700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e880a8750 0x7f0e880c1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:01.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8d105700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e880a8750 0x7f0e880c1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54414/0 (socket says 172.21.15.121:54414) 2023-11-15T03:12:01.045 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8d105700 1 -- 172.21.15.121:0/2667573239 learned_addr learned my addr 172.21.15.121:0/2667573239 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:12:01.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8c904700 1 --2- 172.21.15.121:0/2667573239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e880bcab0 0x7f0e88164e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:01.046 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8d906700 1 --2- 172.21.15.121:0/2667573239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e881511f0 0x7f0e880a7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:01.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8d105700 1 -- 172.21.15.121:0/2667573239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e881511f0 msgr2=0x7f0e880a7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:01.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8d105700 1 --2- 172.21.15.121:0/2667573239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e881511f0 0x7f0e880a7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:01.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8d105700 1 -- 172.21.15.121:0/2667573239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e880bcab0 msgr2=0x7f0e88164e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:01.047 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8d105700 1 --2- 172.21.15.121:0/2667573239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e880bcab0 0x7f0e88164e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:01.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8d105700 1 -- 172.21.15.121:0/2667573239 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e840092a0 con 0x7f0e880a8750 2023-11-15T03:12:01.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8d906700 1 --2- 172.21.15.121:0/2667573239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e881511f0 0x7f0e880a7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:12:01.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8d105700 1 --2- 172.21.15.121:0/2667573239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e880a8750 0x7f0e880c1660 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 rx=0x7f0e84009e00 tx=0x7f0e84017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:01.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e7d7fa700 1 -- 172.21.15.121:0/2667573239 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8400f7e0 con 0x7f0e880a8750 2023-11-15T03:12:01.048 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e7d7fa700 1 -- 172.21.15.121:0/2667573239 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8400f940 con 0x7f0e880a8750 2023-11-15T03:12:01.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e7d7fa700 1 -- 172.21.15.121:0/2667573239 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8401b690 con 0x7f0e880a8750 2023-11-15T03:12:01.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8e107700 1 -- 172.21.15.121:0/2667573239 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e88151770 con 0x7f0e880a8750 2023-11-15T03:12:01.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8e107700 1 -- 172.21.15.121:0/2667573239 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e88165330 con 0x7f0e880a8750 2023-11-15T03:12:01.049 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.035+0000 7f0e8e107700 1 -- 172.21.15.121:0/2667573239 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e881515d0 con 0x7f0e880a8750 2023-11-15T03:12:01.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.039+0000 7f0e7d7fa700 1 -- 172.21.15.121:0/2667573239 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0e8401ba40 con 0x7f0e880a8750 2023-11-15T03:12:01.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.039+0000 7f0e7d7fa700 1 --2- 172.21.15.121:0/2667573239 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e6c05a510 0x7f0e6c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:01.050 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.039+0000 7f0e7d7fa700 1 -- 172.21.15.121:0/2667573239 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f0e84075c70 con 0x7f0e880a8750 2023-11-15T03:12:01.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.039+0000 7f0e8c904700 1 --2- 172.21.15.121:0/2667573239 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e6c05a510 0x7f0e6c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:01.051 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.039+0000 7f0e8c904700 1 --2- 172.21.15.121:0/2667573239 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e6c05a510 0x7f0e6c05c9d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f0e78007fd0 tx=0x7f0e78009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:01.057 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.047+0000 7f0e7d7fa700 1 -- 172.21.15.121:0/2667573239 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0e881515d0 con 0x7f0e880a8750 2023-11-15T03:12:01.223 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.215+0000 7f0e8e107700 1 -- 172.21.15.121:0/2667573239 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e881515d0 con 0x7f0e6c05a510 2023-11-15T03:12:01.224 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.215+0000 7f0e7d7fa700 1 -- 172.21.15.121:0/2667573239 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0e881515d0 con 0x7f0e6c05a510 2023-11-15T03:12:01.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.215+0000 7f0e8e107700 1 -- 172.21.15.121:0/2667573239 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e6c05a510 msgr2=0x7f0e6c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:01.226 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.215+0000 7f0e8e107700 1 --2- 172.21.15.121:0/2667573239 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e6c05a510 0x7f0e6c05c9d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f0e78007fd0 tx=0x7f0e78009040).stop 2023-11-15T03:12:01.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.215+0000 7f0e8e107700 1 -- 172.21.15.121:0/2667573239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e880a8750 msgr2=0x7f0e880c1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:01.227 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.215+0000 7f0e8e107700 1 --2- 172.21.15.121:0/2667573239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e880a8750 0x7f0e880c1660 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 rx=0x7f0e84009e00 tx=0x7f0e84017000).stop 2023-11-15T03:12:01.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.219+0000 7f0e8e107700 1 -- 172.21.15.121:0/2667573239 shutdown_connections 2023-11-15T03:12:01.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.219+0000 7f0e8e107700 1 --2- 172.21.15.121:0/2667573239 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f0e6c05a510 0x7f0e6c05c9d0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:01.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.219+0000 7f0e8e107700 1 --2- 172.21.15.121:0/2667573239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e881511f0 0x7f0e880a7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:01.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.219+0000 7f0e8e107700 1 --2- 172.21.15.121:0/2667573239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e880bcab0 0x7f0e88164e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:01.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.219+0000 7f0e8e107700 1 --2- 172.21.15.121:0/2667573239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e880a8750 0x7f0e880c1660 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:01.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.219+0000 7f0e8e107700 1 -- 172.21.15.121:0/2667573239 >> 172.21.15.121:0/2667573239 conn(0x7f0e88067a60 msgr2=0x7f0e880bedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:12:01.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.219+0000 7f0e8e107700 1 -- 172.21.15.121:0/2667573239 shutdown_connections 2023-11-15T03:12:01.229 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.219+0000 7f0e8e107700 1 -- 172.21.15.121:0/2667573239 wait complete. 2023-11-15T03:12:01.229 INFO:teuthology.orchestra.run.smithi121.stderr:instructed osd(s) 7 to deep-scrub 2023-11-15T03:12:01.246 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T03:12:01.400 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.391+0000 7f27b6107700 1 Processor -- start 2023-11-15T03:12:01.401 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.391+0000 7f27b6107700 1 -- start start 2023-11-15T03:12:01.402 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.391+0000 7f27b6107700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f27b00c3280 0x7f27b00c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:01.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.391+0000 7f27b6107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b015be30 0x7f27b015c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:01.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.391+0000 7f27b6107700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b00c1d50 con 0x7f27b0152140 2023-11-15T03:12:01.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.391+0000 7f27b6107700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b00c1860 con 0x7f27b00c3280 2023-11-15T03:12:01.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.391+0000 7f27b6107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b015cb80 con 0x7f27b015be30 2023-11-15T03:12:01.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27b4904700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f27b0152140 0x7f27b01525f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51194/0 (socket says 172.21.15.121:51194) 2023-11-15T03:12:01.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27b4904700 1 -- 172.21.15.121:0/456238590 learned_addr learned my addr 172.21.15.121:0/456238590 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:12:01.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27b5105700 1 --2- 172.21.15.121:0/456238590 >> v2:172.21.15.121:3300/0 conn(0x7f27b00c3280 0x7f27b00c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:01.405 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27b5906700 1 --2- 172.21.15.121:0/456238590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b015be30 0x7f27b015c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:01.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27a7fff700 1 -- 172.21.15.121:0/456238590 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414767100 0 0) 0x7f27b00c1d50 con 0x7f27b0152140 2023-11-15T03:12:01.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27a7fff700 1 -- 172.21.15.121:0/456238590 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f279c0033f0 con 0x7f27b0152140 2023-11-15T03:12:01.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27a7fff700 1 -- 172.21.15.121:0/456238590 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2901271641 0 0) 0x7f279c0033f0 con 0x7f27b0152140 2023-11-15T03:12:01.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27a7fff700 1 -- 172.21.15.121:0/456238590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b015be30 msgr2=0x7f27b015c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:12:01.406 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27a7fff700 1 --2- 172.21.15.121:0/456238590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b015be30 0x7f27b015c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:01.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27a7fff700 1 -- 172.21.15.121:0/456238590 >> v2:172.21.15.121:3300/0 conn(0x7f27b00c3280 msgr2=0x7f27b00c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:12:01.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27a7fff700 1 --2- 172.21.15.121:0/456238590 >> v2:172.21.15.121:3300/0 conn(0x7f27b00c3280 0x7f27b00c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:01.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27a7fff700 1 -- 172.21.15.121:0/456238590 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b00a9240 con 0x7f27b0152140 2023-11-15T03:12:01.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27a7fff700 1 -- 172.21.15.121:0/456238590 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 190598388 0 0) 0x7f27a0001470 con 0x7f27b0152140 2023-11-15T03:12:01.407 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27a7fff700 1 -- 172.21.15.121:0/456238590 >> v1:172.21.15.121:6789/0 conn(0x7f27b0152140 legacy=0x7f27b01525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:01.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27a7fff700 1 --2- 172.21.15.121:0/456238590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f279c0038e0 0x7f279c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:01.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27a7fff700 1 --2- 172.21.15.121:0/456238590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f279c004690 0x7f279c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:01.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27a7fff700 1 --2- 172.21.15.121:0/456238590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b015be30 0x7f279c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:01.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27a7fff700 1 -- 172.21.15.121:0/456238590 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b00c1860 con 0x7f279c0038e0 2023-11-15T03:12:01.408 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27a7fff700 1 -- 172.21.15.121:0/456238590 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b015cb80 con 0x7f279c004690 2023-11-15T03:12:01.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27a7fff700 1 -- 172.21.15.121:0/456238590 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279c00b8e0 con 0x7f27b015be30 2023-11-15T03:12:01.410 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27b5906700 1 --2- 172.21.15.121:0/456238590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b015be30 0x7f279c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:01.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27b5105700 1 --2- 172.21.15.121:0/456238590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f279c0038e0 0x7f279c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:01.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27b4904700 1 --2- 172.21.15.121:0/456238590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f279c004690 0x7f279c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:01.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27b5105700 1 -- 172.21.15.121:0/456238590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b015be30 msgr2=0x7f279c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:12:01.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27b5105700 1 --2- 172.21.15.121:0/456238590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b015be30 0x7f279c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:01.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27b5105700 1 -- 172.21.15.121:0/456238590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f279c004690 msgr2=0x7f279c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:12:01.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27b5105700 1 --2- 172.21.15.121:0/456238590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f279c004690 0x7f279c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:01.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27b5105700 1 -- 172.21.15.121:0/456238590 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279c0033f0 con 0x7f279c0038e0 2023-11-15T03:12:01.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27b5105700 1 --2- 172.21.15.121:0/456238590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f279c0038e0 0x7f279c003d00 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 rx=0x7f27ac009a80 tx=0x7f27ac009060).ready entity=mon.0 client_cookie=d169b672a0eebbe4 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:01.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27a7fff700 1 -- 172.21.15.121:0/456238590 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac009430 con 0x7f279c0038e0 2023-11-15T03:12:01.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27a7fff700 1 -- 172.21.15.121:0/456238590 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ac00e640 con 0x7f279c0038e0 2023-11-15T03:12:01.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.395+0000 7f27a7fff700 1 -- 172.21.15.121:0/456238590 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac00e9c0 con 0x7f279c0038e0 2023-11-15T03:12:01.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.399+0000 7f27b6107700 1 -- 172.21.15.121:0/456238590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f279c0038e0 msgr2=0x7f279c003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:01.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.399+0000 7f27b6107700 1 --2- 172.21.15.121:0/456238590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f279c0038e0 0x7f279c003d00 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 rx=0x7f27ac009a80 tx=0x7f27ac009060).stop 2023-11-15T03:12:01.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.399+0000 7f27b5906700 1 -- 172.21.15.121:0/456238590 reap_dead start 2023-11-15T03:12:01.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.399+0000 7f27b6107700 1 -- 172.21.15.121:0/456238590 shutdown_connections 2023-11-15T03:12:01.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.399+0000 7f27b6107700 1 -- 172.21.15.121:0/456238590 >> 172.21.15.121:0/456238590 conn(0x7f27b0062190 msgr2=0x7f27b0062570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:12:01.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.399+0000 7f27b6107700 1 -- 172.21.15.121:0/456238590 shutdown_connections 2023-11-15T03:12:01.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.399+0000 7f27b6107700 1 -- 172.21.15.121:0/456238590 wait complete. 2023-11-15T03:12:01.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.399+0000 7f27b6107700 1 Processor -- start 2023-11-15T03:12:01.414 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.399+0000 7f27b6107700 1 -- start start 2023-11-15T03:12:01.416 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b6107700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f279c0038e0 0x7f27b00abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:01.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b6107700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27b00bd070 0x7f27b00bd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:01.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b6107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b0168870 0x7f27b0168cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:01.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b6107700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b00ab6b0 con 0x7f279c0038e0 2023-11-15T03:12:01.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b6107700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b00ac030 con 0x7f27b00bd070 2023-11-15T03:12:01.417 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b6107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b00aaaa0 con 0x7f27b0168870 2023-11-15T03:12:01.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b5105700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f279c0038e0 0x7f27b00abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:01.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b5105700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f279c0038e0 0x7f27b00abb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54442/0 (socket says 172.21.15.121:54442) 2023-11-15T03:12:01.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b5105700 1 -- 172.21.15.121:0/35929671 learned_addr learned my addr 172.21.15.121:0/35929671 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:12:01.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b4904700 1 --2- 172.21.15.121:0/35929671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27b00bd070 0x7f27b00bd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:01.418 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b5906700 1 --2- 172.21.15.121:0/35929671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b0168870 0x7f27b0168cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:01.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b5105700 1 -- 172.21.15.121:0/35929671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b0168870 msgr2=0x7f27b0168cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:01.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b5105700 1 --2- 172.21.15.121:0/35929671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b0168870 0x7f27b0168cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:01.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b5105700 1 -- 172.21.15.121:0/35929671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27b00bd070 msgr2=0x7f27b00bd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:01.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b5105700 1 --2- 172.21.15.121:0/35929671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27b00bd070 0x7f27b00bd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:01.419 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b5105700 1 -- 172.21.15.121:0/35929671 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b00aad80 con 0x7f279c0038e0 2023-11-15T03:12:01.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b5906700 1 --2- 172.21.15.121:0/35929671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b0168870 0x7f27b0168cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:12:01.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b4904700 1 --2- 172.21.15.121:0/35929671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27b00bd070 0x7f27b00bd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:12:01.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b5105700 1 --2- 172.21.15.121:0/35929671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f279c0038e0 0x7f27b00abb40 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 rx=0x7f27ac00e4b0 tx=0x7f27ac01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:01.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27a57fa700 1 -- 172.21.15.121:0/35929671 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac00ed30 con 0x7f279c0038e0 2023-11-15T03:12:01.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27a57fa700 1 -- 172.21.15.121:0/35929671 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ac00eeb0 con 0x7f279c0038e0 2023-11-15T03:12:01.420 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b6107700 1 -- 172.21.15.121:0/35929671 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b00ab060 con 0x7f279c0038e0 2023-11-15T03:12:01.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27a57fa700 1 -- 172.21.15.121:0/35929671 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac026440 con 0x7f279c0038e0 2023-11-15T03:12:01.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.403+0000 7f27b6107700 1 -- 172.21.15.121:0/35929671 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b00aa730 con 0x7f279c0038e0 2023-11-15T03:12:01.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.407+0000 7f27b6107700 1 -- 172.21.15.121:0/35929671 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b006bcf0 con 0x7f279c0038e0 2023-11-15T03:12:01.421 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.407+0000 7f27a57fa700 1 -- 172.21.15.121:0/35929671 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f27ac02f440 con 0x7f279c0038e0 2023-11-15T03:12:01.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.407+0000 7f27a57fa700 1 --2- 172.21.15.121:0/35929671 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f279405a510 0x7f279405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:01.422 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.407+0000 7f27a57fa700 1 -- 172.21.15.121:0/35929671 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f27ac0762a0 con 0x7f279c0038e0 2023-11-15T03:12:01.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.407+0000 7f27b4904700 1 --2- 172.21.15.121:0/35929671 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f279405a510 0x7f279405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:01.423 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.411+0000 7f27b4904700 1 --2- 172.21.15.121:0/35929671 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f279405a510 0x7f279405c9d0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7f27a00016e0 tx=0x7f27a0009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:01.424 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.415+0000 7f27a57fa700 1 -- 172.21.15.121:0/35929671 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f27ac049c00 con 0x7f279c0038e0 2023-11-15T03:12:01.580 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.571+0000 7f27b6107700 1 -- 172.21.15.121:0/35929671 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f27b00ab7f0 con 0x7f279405a510 2023-11-15T03:12:01.583 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.575+0000 7f27a57fa700 1 -- 172.21.15.121:0/35929671 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+68473 (secure 0 0 0) 0x7f27a00106b0 con 0x7f279405a510 2023-11-15T03:12:01.585 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:12:01.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.575+0000 7f27b6107700 1 -- 172.21.15.121:0/35929671 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f279405a510 msgr2=0x7f279405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:01.586 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.575+0000 7f27b6107700 1 --2- 172.21.15.121:0/35929671 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f279405a510 0x7f279405c9d0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7f27a00016e0 tx=0x7f27a0009040).stop 2023-11-15T03:12:01.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.575+0000 7f27b6107700 1 -- 172.21.15.121:0/35929671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f279c0038e0 msgr2=0x7f27b00abb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:01.587 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.575+0000 7f27b6107700 1 --2- 172.21.15.121:0/35929671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f279c0038e0 0x7f27b00abb40 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 rx=0x7f27ac00e4b0 tx=0x7f27ac01a000).stop 2023-11-15T03:12:01.589 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.579+0000 7f27b6107700 1 -- 172.21.15.121:0/35929671 shutdown_connections 2023-11-15T03:12:01.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.579+0000 7f27b6107700 1 --2- 172.21.15.121:0/35929671 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f279405a510 0x7f279405c9d0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:01.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.579+0000 7f27b6107700 1 --2- 172.21.15.121:0/35929671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b0168870 0x7f27b0168cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:01.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.579+0000 7f27b6107700 1 --2- 172.21.15.121:0/35929671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27b00bd070 0x7f27b00bd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:01.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.579+0000 7f27b6107700 1 --2- 172.21.15.121:0/35929671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f279c0038e0 0x7f27b00abb40 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:01.590 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.579+0000 7f27b6107700 1 -- 172.21.15.121:0/35929671 >> 172.21.15.121:0/35929671 conn(0x7f27b0062190 msgr2=0x7f27b014fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:12:01.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.579+0000 7f27b6107700 1 -- 172.21.15.121:0/35929671 shutdown_connections 2023-11-15T03:12:01.591 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:01.579+0000 7f27b6107700 1 -- 172.21.15.121:0/35929671 wait complete. 2023-11-15T03:12:01.591 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T03:12:01.605 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":602,"stamp":"2023-11-15T03:12:00.270352+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2445372,"kb_used_data":15356,"kb_used_omap":0,"kb_used_meta":2429952,"kb_avail":752529348,"statfs":{"total":773094113280,"available":770590052352,"internally_reserved":0,"allocated":15724544,"data_stored":7961611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2488270848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-33941,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-60,"num_read_kb":-61,"num_write":-90,"num_write_kb":-62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.146079"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":129,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-11-15T03:11:50.986536+0000","last_change":"2023-11-15T02:56:46.207042+0000","last_active":"2023-11-15T03:11:50.986536+0000","last_peered":"2023-11-15T03:11:50.986536+0000","last_clean":"2023-11-15T03:11:50.986536+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T03:11:50.986536+0000","last_undegraded":"2023-11-15T03:11:50.986536+0000","last_fullsized":"2023-11-15T03:11:50.986536+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":132,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-11-15T03:11:51.086537+0000","last_change":"2023-11-15T02:56:46.208616+0000","last_active":"2023-11-15T03:11:51.086537+0000","last_peered":"2023-11-15T03:11:51.086537+0000","last_clean":"2023-11-15T03:11:51.086537+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T03:11:51.086537+0000","last_undegraded":"2023-11-15T03:11:51.086537+0000","last_fullsized":"2023-11-15T03:11:51.086537+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":127,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-11-15T03:11:51.099796+0000","last_change":"2023-11-15T02:56:46.259079+0000","last_active":"2023-11-15T03:11:51.099796+0000","last_peered":"2023-11-15T03:11:51.099796+0000","last_clean":"2023-11-15T03:11:51.099796+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T03:11:51.099796+0000","last_undegraded":"2023-11-15T03:11:51.099796+0000","last_fullsized":"2023-11-15T03:11:51.099796+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":132,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-11-15T03:11:51.087803+0000","last_change":"2023-11-15T02:56:46.257575+0000","last_active":"2023-11-15T03:11:51.087803+0000","last_peered":"2023-11-15T03:11:51.087803+0000","last_clean":"2023-11-15T03:11:51.087803+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T03:11:51.087803+0000","last_undegraded":"2023-11-15T03:11:51.087803+0000","last_fullsized":"2023-11-15T03:11:51.087803+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":143,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-11-15T03:11:50.419610+0000","last_change":"2023-11-15T02:56:46.202313+0000","last_active":"2023-11-15T03:11:50.419610+0000","last_peered":"2023-11-15T03:11:50.419610+0000","last_clean":"2023-11-15T03:11:50.419610+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T03:11:50.419610+0000","last_undegraded":"2023-11-15T03:11:50.419610+0000","last_fullsized":"2023-11-15T03:11:50.419610+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":132,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-11-15T03:11:58.464808+0000","last_change":"2023-11-15T03:11:58.464808+0000","last_active":"2023-11-15T03:11:58.464808+0000","last_peered":"2023-11-15T03:11:58.464808+0000","last_clean":"2023-11-15T03:11:58.464808+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T03:11:58.464808+0000","last_undegraded":"2023-11-15T03:11:58.464808+0000","last_fullsized":"2023-11-15T03:11:58.464808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:58.464661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:58.464661+0000","last_clean_scrub_stamp":"2023-11-15T03:11:58.464661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":127,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-11-15T03:11:51.093280+0000","last_change":"2023-11-15T02:56:46.202107+0000","last_active":"2023-11-15T03:11:51.093280+0000","last_peered":"2023-11-15T03:11:51.093280+0000","last_clean":"2023-11-15T03:11:51.093280+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T03:11:51.093280+0000","last_undegraded":"2023-11-15T03:11:51.093280+0000","last_fullsized":"2023-11-15T03:11:51.093280+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":143,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-11-15T03:11:50.419362+0000","last_change":"2023-11-15T02:56:46.207063+0000","last_active":"2023-11-15T03:11:50.419362+0000","last_peered":"2023-11-15T03:11:50.419362+0000","last_clean":"2023-11-15T03:11:50.419362+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T03:11:50.419362+0000","last_undegraded":"2023-11-15T03:11:50.419362+0000","last_fullsized":"2023-11-15T03:11:50.419362+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:45.164118+0000","last_clean_scrub_stamp":"2023-11-15T02:56:45.164118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":128,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-11-15T03:11:51.093357+0000","last_change":"2023-11-15T02:56:45.187750+0000","last_active":"2023-11-15T03:11:51.093357+0000","last_peered":"2023-11-15T03:11:51.093357+0000","last_clean":"2023-11-15T03:11:51.093357+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T03:11:51.093357+0000","last_undegraded":"2023-11-15T03:11:51.093357+0000","last_fullsized":"2023-11-15T03:11:51.093357+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T02:56:44.150053+0000","last_clean_scrub_stamp":"2023-11-15T02:56:44.150053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542329,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307208,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064632,"statfs":{"total":96636764160,"available":96322183168,"internally_reserved":0,"allocated":1966080,"data_stored":995360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.0940000000000001,"15min":1.1319999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.55800000000000005},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":10.961},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.032,"5min":1.0880000000000001,"15min":1.1120000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.3089999999999999,"5min":1.6160000000000001,"15min":3.4910000000000001},"last":1.135}]},{"osd":1,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.127,"15min":1.143},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.419,"5min":1.552,"15min":5.4930000000000003},"last":1.25},{"interface":"front","average":{"1min":1.153,"5min":1.165,"15min":1.1819999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6000000000000001,"5min":1.766,"15min":6.8239999999999998},"last":1.0389999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0940000000000001,"15min":1.1359999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.452,"5min":1.452,"15min":5.7000000000000002},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.226,"15min":1.2310000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":2.75},"last":1.1060000000000001}]},{"osd":3,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.167,"15min":1.248},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":12.907},"last":1.169},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2370000000000001,"15min":1.2709999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.573,"5min":1.573,"15min":9.9909999999999997},"last":1.2969999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.105,"15min":1.1379999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6160000000000001,"5min":1.8939999999999999,"15min":2.5649999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.1240000000000001,"15min":1.2070000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.72,"5min":1.72,"15min":4.226},"last":1.0640000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.2450000000000001,"15min":1.2749999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":2.6890000000000001},"last":1.194},{"interface":"front","average":{"1min":1.143,"5min":1.236,"15min":1.292},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6899999999999999,"5min":1.7330000000000001,"15min":2.4550000000000001},"last":1.0940000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.325,"15min":1.381},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.5629999999999999,"5min":1.954,"15min":3.2789999999999999},"last":1.228},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.248,"15min":1.3340000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.63600000000000001},"max":{"1min":1.542,"5min":1.76,"15min":4.7800000000000002},"last":1.2609999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542329,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311304,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94060536,"statfs":{"total":96636764160,"available":96317988864,"internally_reserved":0,"allocated":1966080,"data_stored":995360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.03,"15min":1.077},"min":{"1min":0.66200000000000003,"5min":0.63100000000000001,"15min":0.44500000000000001},"max":{"1min":1.4610000000000001,"5min":1.5760000000000001,"15min":4.3159999999999998},"last":1.153},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.109,"15min":1.0960000000000001},"min":{"1min":0.69999999999999996,"5min":0.59799999999999998,"15min":0.58599999999999997},"max":{"1min":1.486,"5min":1.659,"15min":2.2480000000000002},"last":1.2110000000000001}]},{"osd":1,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.111,"15min":1.1140000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.57599999999999996},"max":{"1min":1.6000000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1000000000000001,"15min":1.1379999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.60699999999999998},"max":{"1min":1.3360000000000001,"5min":1.8100000000000001,"15min":2.8570000000000002},"last":1.2509999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1899999999999999,"15min":1.1919999999999999},"min":{"1min":0.83099999999999996,"5min":0.71699999999999997,"15min":0.49099999999999999},"max":{"1min":1.4730000000000001,"5min":1.7090000000000001,"15min":2.621},"last":1.113},{"interface":"front","average":{"1min":1.115,"5min":1.153,"15min":1.153},"min":{"1min":0.71699999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.4710000000000001,"5min":1.8360000000000001,"15min":2.0059999999999998},"last":0.76400000000000001}]},{"osd":3,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.2310000000000001,"15min":1.214},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.50900000000000001},"max":{"1min":1.7709999999999999,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.234,"5min":1.2430000000000001,"15min":1.2609999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.67100000000000004},"max":{"1min":1.458,"5min":1.8640000000000001,"15min":2.8359999999999999},"last":0.83799999999999997}]},{"osd":4,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.0580000000000001,"15min":1.1000000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5860000000000001,"5min":1.6839999999999999,"15min":4.5350000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.163,"5min":1.2170000000000001,"15min":1.2130000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.8580000000000001,"5min":2.004,"15min":2.0739999999999998},"last":1.2849999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3799999999999999,"15min":1.415},"min":{"1min":0.79800000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.744,"5min":2.1469999999999998,"15min":2.157},"last":1.399},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2210000000000001,"15min":1.256},"min":{"1min":0.621,"5min":0.62,"15min":0.60899999999999999},"max":{"1min":1.8140000000000001,"5min":1.8779999999999999,"15min":2.9460000000000002},"last":1.3220000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.216,"15min":1.2250000000000001},"min":{"1min":0.77400000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.9139999999999999,"5min":1.9239999999999999,"15min":3.1989999999999998},"last":1.044},{"interface":"front","average":{"1min":1.377,"5min":1.407,"15min":1.419},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.65200000000000002},"max":{"1min":1.8220000000000001,"5min":2.0649999999999999,"15min":2.9350000000000001},"last":1.454}]}]},{"osd":4,"up_from":14,"seq":60129542330,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":307208,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064632,"statfs":{"total":96636764160,"available":96322183168,"internally_reserved":0,"allocated":1966080,"data_stored":995360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.0569999999999999,"15min":1.083},"min":{"1min":0.73799999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4810000000000001,"5min":1.5660000000000001,"15min":1.5840000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.044,"15min":1.056},"min":{"1min":0.85399999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.677},"last":1.159}]},{"osd":1,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.135,"15min":1.1319999999999999},"min":{"1min":0.70599999999999996,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":2.617},"last":0.995},{"interface":"front","average":{"1min":1.212,"5min":1.1419999999999999,"15min":1.1539999999999999},"min":{"1min":0.82099999999999995,"5min":0.629,"15min":0.54800000000000004},"max":{"1min":1.573,"5min":1.6319999999999999,"15min":1.6619999999999999},"last":0.75800000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.165,"15min":1.1830000000000001},"min":{"1min":0.81399999999999995,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.5029999999999999,"5min":1.823,"15min":1.9450000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.155,"15min":1.1619999999999999},"min":{"1min":0.84199999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9330000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.069}]},{"osd":3,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1399999999999999,"15min":1.1459999999999999},"min":{"1min":0.85699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.5229999999999999,"5min":1.5940000000000001,"15min":1.8660000000000001},"last":1.099},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2549999999999999,"15min":1.264},"min":{"1min":0.93000000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.611,"5min":1.8200000000000001,"15min":2.5630000000000002},"last":1.3049999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1479999999999999,"15min":1.1830000000000001},"min":{"1min":0.91900000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.554,"5min":1.643,"15min":2.0840000000000001},"last":1.284},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.1479999999999999,"15min":1.1879999999999999},"min":{"1min":0.89400000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":2.3570000000000002},"last":1.2470000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2410000000000001,"15min":1.264},"min":{"1min":0.97999999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5880000000000001,"5min":1.913,"15min":2.6640000000000001},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.288,"15min":1.323},"min":{"1min":0.88,"5min":0.67600000000000005,"15min":0.57299999999999995},"max":{"1min":1.585,"5min":1.9139999999999999,"15min":2.173},"last":1.024}]},{"osd":7,"last update":"Wed Nov 15 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2769999999999999,"15min":1.3069999999999999},"min":{"1min":0.89500000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.7629999999999999,"5min":1.9199999999999999,"15min":2.6030000000000002},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.3700000000000001,"15min":1.3879999999999999},"min":{"1min":1.044,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8240000000000001,"5min":2.069,"15min":2.7669999999999999},"last":1.5}]}]},{"osd":3,"up_from":14,"seq":60129542330,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303112,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068728,"statfs":{"total":96636764160,"available":96326377472,"internally_reserved":0,"allocated":1966080,"data_stored":995360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85999999999999999,"15min":0.83699999999999997},"min":{"1min":0.54200000000000004,"5min":0.54000000000000004,"15min":0.45400000000000001},"max":{"1min":1.327,"5min":1.4219999999999999,"15min":1.528},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.89900000000000002,"15min":0.88800000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.39800000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.77600000000000002}]},{"osd":1,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.83899999999999997,"15min":0.872},"min":{"1min":0.62,"5min":0.53100000000000003,"15min":0.435},"max":{"1min":1.165,"5min":1.25,"15min":1.4670000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.996,"5min":0.98599999999999999,"15min":0.99199999999999999},"min":{"1min":0.72199999999999998,"5min":0.63700000000000001,"15min":0.55800000000000005},"max":{"1min":1.446,"5min":1.446,"15min":1.544},"last":0.84199999999999997}]},{"osd":2,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.026,"15min":1.0209999999999999},"min":{"1min":0.66400000000000003,"5min":0.64200000000000002,"15min":0.54600000000000004},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0529999999999999,"15min":1.0660000000000001},"min":{"1min":0.66000000000000003,"5min":0.63100000000000001,"15min":0.50600000000000001},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.1759999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2470000000000001,"15min":1.228},"min":{"1min":1.0029999999999999,"5min":0.88600000000000001,"15min":0.71399999999999997},"max":{"1min":1.5349999999999999,"5min":1.554,"15min":1.9139999999999999},"last":1.206},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2170000000000001,"15min":1.2250000000000001},"min":{"1min":1.01,"5min":0.78500000000000003,"15min":0.621},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.726},"last":1.2450000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.252,"15min":1.266},"min":{"1min":0.91700000000000004,"5min":0.90200000000000002,"15min":0.64300000000000002},"max":{"1min":1.5900000000000001,"5min":1.7290000000000001,"15min":2.0459999999999998},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.27,"15min":1.272},"min":{"1min":0.95899999999999996,"5min":0.94799999999999995,"15min":0.63500000000000001},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.927},"last":1.321}]},{"osd":6,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.345,"15min":1.339},"min":{"1min":0.91400000000000003,"5min":0.875,"15min":0.72899999999999998},"max":{"1min":1.575,"5min":1.8129999999999999,"15min":1.9970000000000001},"last":1.097},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3560000000000001,"15min":1.3720000000000001},"min":{"1min":1.0980000000000001,"5min":0.871,"15min":0.77700000000000002},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":2.3570000000000002},"last":1.526}]},{"osd":7,"last update":"Wed Nov 15 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.331,"15min":1.3440000000000001},"min":{"1min":1.0149999999999999,"5min":0.92500000000000004,"15min":0.68100000000000005},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":2.0470000000000002},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.345,"15min":1.355},"min":{"1min":1.113,"5min":0.97299999999999998,"15min":0.66600000000000004},"max":{"1min":1.8500000000000001,"5min":1.8779999999999999,"15min":2.1040000000000001},"last":1.5820000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542330,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303108,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068732,"statfs":{"total":96636764160,"available":96326381568,"internally_reserved":0,"allocated":1961984,"data_stored":994091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.65400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97099999999999997,"5min":1.2350000000000001,"15min":1.409},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.84299999999999997,"15min":0.84399999999999997},"min":{"1min":0.58799999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2689999999999999,"5min":1.4950000000000001,"15min":1.619},"last":0.88800000000000001}]},{"osd":1,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.90000000000000002,"15min":0.91100000000000003},"min":{"1min":0.74199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.95599999999999996,"15min":0.95399999999999996},"min":{"1min":0.67200000000000004,"5min":0.55100000000000005,"15min":0.54600000000000004},"max":{"1min":1.3580000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":1.1910000000000001}]},{"osd":3,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95599999999999996,"15min":0.97199999999999998},"min":{"1min":0.60699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.385,"5min":1.4359999999999999,"15min":1.6259999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.125,"15min":1.125},"min":{"1min":0.67700000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.488,"5min":1.5549999999999999,"15min":1.5800000000000001},"last":1.2649999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.218,"15min":1.2070000000000001},"min":{"1min":1.052,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.581,"5min":1.583,"15min":1.9410000000000001},"last":1.419},{"interface":"front","average":{"1min":1.3,"5min":1.21,"15min":1.212},"min":{"1min":0.99199999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6579999999999999,"5min":1.7170000000000001,"15min":1.7829999999999999},"last":1.298}]},{"osd":5,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1619999999999999,"15min":1.1830000000000001},"min":{"1min":0.98499999999999999,"5min":0.68100000000000005,"15min":0.67000000000000004},"max":{"1min":1.47,"5min":1.625,"15min":1.8939999999999999},"last":1.008},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.266,"15min":1.272},"min":{"1min":1.0660000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.556,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.3540000000000001}]},{"osd":6,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.262,"15min":1.2729999999999999},"min":{"1min":0.93999999999999995,"5min":0.62,"15min":0.62},"max":{"1min":1.6990000000000001,"5min":2,"15min":2.137},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.2669999999999999,"15min":1.2689999999999999},"min":{"1min":1.0880000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.538,"5min":1.9350000000000001,"15min":2.0870000000000002},"last":1.234}]},{"osd":7,"last update":"Wed Nov 15 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.29,"15min":1.3109999999999999},"min":{"1min":0.95699999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7789999999999999,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.466},{"interface":"front","average":{"1min":1.323,"5min":1.298,"15min":1.2989999999999999},"min":{"1min":1.101,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.7270000000000001,"5min":1.881,"15min":2.0350000000000001},"last":1.3169999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542330,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307208,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064632,"statfs":{"total":96636764160,"available":96322183168,"internally_reserved":0,"allocated":1966080,"data_stored":995360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80600000000000005,"15min":0.83299999999999996},"min":{"1min":0.503,"5min":0.44400000000000001,"15min":0.36199999999999999},"max":{"1min":1.093,"5min":1.756,"15min":4.3630000000000004},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.90600000000000003,"15min":0.93700000000000006},"min":{"1min":0.61799999999999999,"5min":0.52600000000000002,"15min":0.40000000000000002},"max":{"1min":1.2490000000000001,"5min":1.6719999999999999,"15min":6.157},"last":0.69499999999999995}]},{"osd":2,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.96399999999999997,"15min":0.97899999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.46800000000000003},"max":{"1min":1.3400000000000001,"5min":1.625,"15min":4.1699999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0029999999999999,"15min":1.0289999999999999},"min":{"1min":0.72199999999999998,"5min":0.60099999999999998,"15min":0.47799999999999998},"max":{"1min":1.379,"5min":1.879,"15min":6.8369999999999997},"last":1.246}]},{"osd":3,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.94099999999999995,"15min":0.95599999999999996},"min":{"1min":0.59499999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.3120000000000001,"5min":1.6519999999999999,"15min":3.5070000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":1.073,"5min":1.093,"15min":1.113},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56200000000000006},"max":{"1min":1.3460000000000001,"5min":2.0070000000000001,"15min":5.2149999999999999},"last":1.1930000000000001}]},{"osd":4,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.224,"15min":1.2609999999999999},"min":{"1min":0.81499999999999995,"5min":0.80500000000000005,"15min":0.66700000000000004},"max":{"1min":1.5900000000000001,"5min":1.944,"15min":6.2000000000000002},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.1870000000000001,"15min":1.196},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.3939999999999999,"5min":1.6779999999999999,"15min":3.577},"last":1.2210000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.23,"15min":1.25},"min":{"1min":0.93000000000000005,"5min":0.81499999999999995,"15min":0.69599999999999995},"max":{"1min":1.478,"5min":1.857,"15min":3.548},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.21,"5min":1.278,"15min":1.2969999999999999},"min":{"1min":0.86199999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.4330000000000001,"5min":1.774,"15min":4.4080000000000004},"last":1.0529999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3180000000000001,"15min":1.349},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.82299999999999995},"max":{"1min":1.5640000000000001,"5min":1.915,"15min":4.4489999999999998},"last":1.286},{"interface":"front","average":{"1min":1.236,"5min":1.2889999999999999,"15min":1.3340000000000001},"min":{"1min":0.93300000000000005,"5min":0.86499999999999999,"15min":0.75900000000000001},"max":{"1min":1.5409999999999999,"5min":2.056,"15min":6.9690000000000003},"last":1.367}]},{"osd":7,"last update":"Wed Nov 15 03:11:08 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.327,"15min":1.401},"min":{"1min":1.036,"5min":0.90400000000000003,"15min":0.79900000000000004},"max":{"1min":1.6319999999999999,"5min":1.994,"15min":18.253},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2889999999999999,"15min":1.3380000000000001},"min":{"1min":0.997,"5min":0.95599999999999996,"15min":0.79000000000000004},"max":{"1min":1.452,"5min":1.8720000000000001,"15min":11.006},"last":1.431}]}]},{"osd":5,"up_from":13,"seq":55834575034,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303112,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068728,"statfs":{"total":96636764160,"available":96326377472,"internally_reserved":0,"allocated":1966080,"data_stored":995360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0800000000000001,"15min":1.073},"min":{"1min":0.65800000000000003,"5min":0.627,"15min":0.49399999999999999},"max":{"1min":1.827,"5min":1.827,"15min":2.3370000000000002},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0589999999999999},"min":{"1min":0.63600000000000001,"5min":0.61099999999999999,"15min":0.60799999999999998},"max":{"1min":1.3580000000000001,"5min":1.6519999999999999,"15min":3.46},"last":1.1379999999999999}]},{"osd":1,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.105,"15min":1.121},"min":{"1min":0.84599999999999997,"5min":0.71199999999999997,"15min":0.58999999999999997},"max":{"1min":1.4570000000000001,"5min":1.516,"15min":3.6349999999999998},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.212,"5min":1.161,"15min":1.143},"min":{"1min":0.84199999999999997,"5min":0.73999999999999999,"15min":0.63700000000000001},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":3.516},"last":0.95699999999999996}]},{"osd":2,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.1459999999999999,"15min":1.165},"min":{"1min":0.88600000000000001,"5min":0.74099999999999999,"15min":0.73199999999999998},"max":{"1min":1.774,"5min":1.774,"15min":5.4180000000000001},"last":1.179},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1899999999999999,"15min":1.204},"min":{"1min":0.77300000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6839999999999999,"5min":2.0840000000000001,"15min":3.5859999999999999},"last":1.1220000000000001}]},{"osd":3,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2070000000000001,"15min":1.196},"min":{"1min":0.91400000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6439999999999999,"5min":1.8560000000000001,"15min":2.141},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.296,"15min":1.3169999999999999},"min":{"1min":0.88400000000000001,"5min":0.875,"15min":0.78600000000000003},"max":{"1min":1.6140000000000001,"5min":1.6739999999999999,"15min":4.2729999999999997},"last":1.29}]},{"osd":4,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.2130000000000001,"15min":1.26},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.4850000000000001,"5min":1.591,"15min":3.706},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.258,"15min":1.3080000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.64100000000000001},"max":{"1min":1.863,"5min":1.9219999999999999,"15min":6.1180000000000003},"last":1.048}]},{"osd":6,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.268,"15min":1.254},"min":{"1min":0.88800000000000001,"5min":0.80700000000000005,"15min":0.749},"max":{"1min":1.6080000000000001,"5min":1.843,"15min":4.2309999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.2869999999999999,"15min":1.341},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.546,"5min":2.2240000000000002,"15min":3.641},"last":1.208}]},{"osd":7,"last update":"Wed Nov 15 03:11:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.284,"15min":1.2949999999999999},"min":{"1min":0.83499999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6970000000000001,"5min":1.988,"15min":3.516},"last":1.234},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.379,"15min":1.387},"min":{"1min":0.95899999999999996,"5min":0.94299999999999995,"15min":0.89500000000000002},"max":{"1min":1.925,"5min":2.282,"15min":3.754},"last":1.397}]}]},{"osd":1,"up_from":13,"seq":55834575033,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303112,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068728,"statfs":{"total":96636764160,"available":96326377472,"internally_reserved":0,"allocated":1966080,"data_stored":995360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.80500000000000005,"15min":0.81000000000000005},"min":{"1min":0.63,"5min":0.504,"15min":0.48799999999999999},"max":{"1min":1.22,"5min":1.671,"15min":1.671},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.85599999999999998,"15min":0.879},"min":{"1min":0.52700000000000002,"5min":0.502,"15min":0.46600000000000003},"max":{"1min":1.1799999999999999,"5min":1.3939999999999999,"15min":1.6899999999999999},"last":1.4019999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97799999999999998,"15min":0.94099999999999995},"min":{"1min":0.755,"5min":0.60999999999999999,"15min":0.44600000000000001},"max":{"1min":1.306,"5min":1.806,"15min":1.806},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.872,"5min":0.89000000000000001,"15min":0.90500000000000003},"min":{"1min":0.63200000000000001,"5min":0.55800000000000005,"15min":0.49199999999999999},"max":{"1min":1.2509999999999999,"5min":1.3360000000000001,"15min":1.887},"last":1.101}]},{"osd":3,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.129,"15min":1.125},"min":{"1min":0.747,"5min":0.66300000000000003,"15min":0.42399999999999999},"max":{"1min":1.476,"5min":1.6160000000000001,"15min":1.712},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0549999999999999,"15min":1.042},"min":{"1min":0.68500000000000005,"5min":0.52300000000000002,"15min":0.46000000000000002},"max":{"1min":1.4450000000000001,"5min":1.5409999999999999,"15min":1.7090000000000001},"last":0.875}]},{"osd":4,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.208,"15min":1.1930000000000001},"min":{"1min":0.83699999999999997,"5min":0.80600000000000005,"15min":0.65100000000000002},"max":{"1min":1.671,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.278},{"interface":"front","average":{"1min":1.194,"5min":1.1970000000000001,"15min":1.212},"min":{"1min":0.97399999999999998,"5min":0.83799999999999997,"15min":0.50800000000000001},"max":{"1min":1.4710000000000001,"5min":1.5429999999999999,"15min":2.206},"last":1.1990000000000001}]},{"osd":5,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2649999999999999,"15min":1.2470000000000001},"min":{"1min":0.98499999999999999,"5min":0.90300000000000002,"15min":0.746},"max":{"1min":1.4430000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.508},{"interface":"front","average":{"1min":1.248,"5min":1.2509999999999999,"15min":1.256},"min":{"1min":1.038,"5min":0.92300000000000004,"15min":0.56100000000000005},"max":{"1min":1.4019999999999999,"5min":1.718,"15min":2.4609999999999999},"last":1.264}]},{"osd":6,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.327,"15min":1.327},"min":{"1min":1.1020000000000001,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.671,"5min":1.8640000000000001,"15min":1.948},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.429,"5min":1.431,"15min":1.4079999999999999},"min":{"1min":1.2230000000000001,"5min":0.91600000000000004,"15min":0.87},"max":{"1min":1.786,"5min":1.9670000000000001,"15min":2.3290000000000002},"last":1.4350000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3149999999999999,"15min":1.3169999999999999},"min":{"1min":1.1000000000000001,"5min":0.92000000000000004,"15min":0.84199999999999997},"max":{"1min":1.595,"5min":2.089,"15min":2.161},"last":1.325},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.3999999999999999,"15min":1.3779999999999999},"min":{"1min":1.095,"5min":1.0249999999999999,"15min":0.78600000000000003},"max":{"1min":1.899,"5min":2.137,"15min":2.2349999999999999},"last":1.5589999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T03:12:01.610 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-11-15T02:56:45.164118+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=15, tm_hour=2, tm_min=56, tm_sec=45, tm_wday=2, tm_yday=319, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=15, tm_hour=3, tm_min=11, tm_sec=55, tm_wday=2, tm_yday=319, tm_isdst=0) 2023-11-15T03:12:01.610 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-11-15T02:56:45.164118+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=15, tm_hour=2, tm_min=56, tm_sec=45, tm_wday=2, tm_yday=319, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=15, tm_hour=3, tm_min=11, tm_sec=55, tm_wday=2, tm_yday=319, tm_isdst=0) 2023-11-15T03:12:01.611 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-11-15T02:56:45.164118+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=15, tm_hour=2, tm_min=56, tm_sec=45, tm_wday=2, tm_yday=319, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=15, tm_hour=3, tm_min=11, tm_sec=55, tm_wday=2, tm_yday=319, tm_isdst=0) 2023-11-15T03:12:01.611 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-11-15T02:56:45.164118+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=15, tm_hour=2, tm_min=56, tm_sec=45, tm_wday=2, tm_yday=319, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=15, tm_hour=3, tm_min=11, tm_sec=55, tm_wday=2, tm_yday=319, tm_isdst=0) 2023-11-15T03:12:01.611 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-11-15T02:56:45.164118+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=15, tm_hour=2, tm_min=56, tm_sec=45, tm_wday=2, tm_yday=319, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=15, tm_hour=3, tm_min=11, tm_sec=55, tm_wday=2, tm_yday=319, tm_isdst=0) 2023-11-15T03:12:01.612 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-15T02:56:45.164118+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=15, tm_hour=2, tm_min=56, tm_sec=45, tm_wday=2, tm_yday=319, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=15, tm_hour=3, tm_min=11, tm_sec=55, tm_wday=2, tm_yday=319, tm_isdst=0) 2023-11-15T03:12:01.612 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-11-15T02:56:45.164118+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=15, tm_hour=2, tm_min=56, tm_sec=45, tm_wday=2, tm_yday=319, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=15, tm_hour=3, tm_min=11, tm_sec=55, tm_wday=2, tm_yday=319, tm_isdst=0) 2023-11-15T03:12:01.612 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-15T02:56:44.150053+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=15, tm_hour=2, tm_min=56, tm_sec=44, tm_wday=2, tm_yday=319, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=15, tm_hour=3, tm_min=11, tm_sec=55, tm_wday=2, tm_yday=319, tm_isdst=0) 2023-11-15T03:12:01.613 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-15T03:12:12.766 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:12:12.774 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:12:21.614 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-15T03:12:21.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.771+0000 7ff35625b700 1 Processor -- start 2023-11-15T03:12:21.781 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.771+0000 7ff35625b700 1 -- start start 2023-11-15T03:12:21.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.775+0000 7ff35625b700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff350152780 0x7ff350159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:21.784 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.775+0000 7ff35625b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff3500a5510 0x7ff3500a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:21.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.775+0000 7ff35625b700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3500c3fa0 con 0x7ff3500c65d0 2023-11-15T03:12:21.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.775+0000 7ff35625b700 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff350159630 con 0x7ff350152780 2023-11-15T03:12:21.785 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.775+0000 7ff35625b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3500c30b0 con 0x7ff3500a5510 2023-11-15T03:12:21.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.775+0000 7ff34f7fe700 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff350152780 0x7ff350159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:21.786 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.775+0000 7ff346ffd700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff3500c65d0 0x7ff3500c6a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:35654/0 (socket says 172.21.15.121:35654) 2023-11-15T03:12:21.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.775+0000 7ff346ffd700 1 -- 172.21.15.121:0/1635312273 learned_addr learned my addr 172.21.15.121:0/1635312273 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:12:21.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.775+0000 7ff34ffff700 1 --2- 172.21.15.121:0/1635312273 >> v2:172.21.15.190:3300/0 conn(0x7ff3500a5510 0x7ff3500a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:21.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.775+0000 7ff34effd700 1 -- 172.21.15.121:0/1635312273 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557957846 0 0) 0x7ff3500c3fa0 con 0x7ff3500c65d0 2023-11-15T03:12:21.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.775+0000 7ff34effd700 1 -- 172.21.15.121:0/1635312273 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff33c0033f0 con 0x7ff3500c65d0 2023-11-15T03:12:21.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.775+0000 7ff34f7fe700 1 -- 172.21.15.121:0/1635312273 >> v2:172.21.15.190:3300/0 conn(0x7ff3500a5510 msgr2=0x7ff3500a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:12:21.787 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.775+0000 7ff34f7fe700 1 --2- 172.21.15.121:0/1635312273 >> v2:172.21.15.190:3300/0 conn(0x7ff3500a5510 0x7ff3500a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:21.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.775+0000 7ff34f7fe700 1 -- 172.21.15.121:0/1635312273 >> v1:172.21.15.121:6789/0 conn(0x7ff3500c65d0 legacy=0x7ff3500c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:21.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.775+0000 7ff34f7fe700 1 -- 172.21.15.121:0/1635312273 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3500c3390 con 0x7ff350152780 2023-11-15T03:12:21.788 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.775+0000 7ff34f7fe700 1 --2- 172.21.15.121:0/1635312273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff350152780 0x7ff350159140 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 rx=0x7ff3500c0ec0 tx=0x7ff340009780).ready entity=mon.0 client_cookie=ad3ed63fbbcf2bc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:21.789 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.775+0000 7ff34effd700 1 -- 172.21.15.121:0/1635312273 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34000e070 con 0x7ff350152780 2023-11-15T03:12:21.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.779+0000 7ff34effd700 1 -- 172.21.15.121:0/1635312273 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34000bab0 con 0x7ff350152780 2023-11-15T03:12:21.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.779+0000 7ff34effd700 1 -- 172.21.15.121:0/1635312273 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34000be30 con 0x7ff350152780 2023-11-15T03:12:21.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.779+0000 7ff35625b700 1 -- 172.21.15.121:0/1635312273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff350152780 msgr2=0x7ff350159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:21.790 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.779+0000 7ff35625b700 1 --2- 172.21.15.121:0/1635312273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff350152780 0x7ff350159140 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 rx=0x7ff3500c0ec0 tx=0x7ff340009780).stop 2023-11-15T03:12:21.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.779+0000 7ff35625b700 1 -- 172.21.15.121:0/1635312273 shutdown_connections 2023-11-15T03:12:21.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.779+0000 7ff35625b700 1 --2- 172.21.15.121:0/1635312273 >> v2:172.21.15.190:3300/0 conn(0x7ff3500a5510 0x7ff3500a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:21.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.779+0000 7ff35625b700 1 --2- 172.21.15.121:0/1635312273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff350152780 0x7ff350159140 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:21.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.779+0000 7ff35625b700 1 -- 172.21.15.121:0/1635312273 >> 172.21.15.121:0/1635312273 conn(0x7ff350062190 msgr2=0x7ff3500625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:12:21.791 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.779+0000 7ff35625b700 1 -- 172.21.15.121:0/1635312273 shutdown_connections 2023-11-15T03:12:21.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.779+0000 7ff35625b700 1 -- 172.21.15.121:0/1635312273 wait complete. 2023-11-15T03:12:21.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff35625b700 1 Processor -- start 2023-11-15T03:12:21.792 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff35625b700 1 -- start start 2023-11-15T03:12:21.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff35625b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3500a5510 0x7ff35013f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:21.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff35625b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3500c65d0 0x7ff350153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:21.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff35625b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff350152780 0x7ff3500abb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:21.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff35625b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff350154710 con 0x7ff350152780 2023-11-15T03:12:21.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff35625b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3501549f0 con 0x7ff3500a5510 2023-11-15T03:12:21.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff35625b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff350154cd0 con 0x7ff3500c65d0 2023-11-15T03:12:21.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff34ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff350152780 0x7ff3500abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:21.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff34ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff350152780 0x7ff3500abb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39064/0 (socket says 172.21.15.121:39064) 2023-11-15T03:12:21.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff34f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3500a5510 0x7ff35013f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:21.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff34ffff700 1 -- 172.21.15.121:0/850457657 learned_addr learned my addr 172.21.15.121:0/850457657 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:12:21.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff346ffd700 1 --2- 172.21.15.121:0/850457657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3500c65d0 0x7ff350153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:21.796 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff34ffff700 1 -- 172.21.15.121:0/850457657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3500c65d0 msgr2=0x7ff350153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:21.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff34ffff700 1 --2- 172.21.15.121:0/850457657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3500c65d0 0x7ff350153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:21.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff34ffff700 1 -- 172.21.15.121:0/850457657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3500a5510 msgr2=0x7ff35013f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:21.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff34ffff700 1 --2- 172.21.15.121:0/850457657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3500a5510 0x7ff35013f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:21.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff34ffff700 1 -- 172.21.15.121:0/850457657 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff340009a20 con 0x7ff350152780 2023-11-15T03:12:21.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff34f7fe700 1 --2- 172.21.15.121:0/850457657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3500a5510 0x7ff35013f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-15T03:12:21.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff346ffd700 1 --2- 172.21.15.121:0/850457657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3500c65d0 0x7ff350153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:12:21.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.783+0000 7ff34ffff700 1 --2- 172.21.15.121:0/850457657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff350152780 0x7ff3500abb20 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 rx=0x7ff34800e8d0 tx=0x7ff348009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:21.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.787+0000 7ff347fff700 1 -- 172.21.15.121:0/850457657 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff348012600 con 0x7ff350152780 2023-11-15T03:12:21.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.787+0000 7ff347fff700 1 -- 172.21.15.121:0/850457657 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff348012760 con 0x7ff350152780 2023-11-15T03:12:21.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.787+0000 7ff35625b700 1 -- 172.21.15.121:0/850457657 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff350154fb0 con 0x7ff350152780 2023-11-15T03:12:21.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.787+0000 7ff347fff700 1 -- 172.21.15.121:0/850457657 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff348012ae0 con 0x7ff350152780 2023-11-15T03:12:21.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.787+0000 7ff35625b700 1 -- 172.21.15.121:0/850457657 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff350154300 con 0x7ff350152780 2023-11-15T03:12:21.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.787+0000 7ff35625b700 1 -- 172.21.15.121:0/850457657 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3501554b0 con 0x7ff350152780 2023-11-15T03:12:21.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.787+0000 7ff347fff700 1 -- 172.21.15.121:0/850457657 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff34801b400 con 0x7ff350152780 2023-11-15T03:12:21.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.791+0000 7ff347fff700 1 --2- 172.21.15.121:0/850457657 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff33005a510 0x7ff33005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:21.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.791+0000 7ff347fff700 1 -- 172.21.15.121:0/850457657 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7ff3480749b0 con 0x7ff350152780 2023-11-15T03:12:21.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.791+0000 7ff34f7fe700 1 --2- 172.21.15.121:0/850457657 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff33005a510 0x7ff33005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:21.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.791+0000 7ff34f7fe700 1 --2- 172.21.15.121:0/850457657 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff33005a510 0x7ff33005c9d0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7ff3400099f0 tx=0x7ff340009780).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:21.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.799+0000 7ff347fff700 1 -- 172.21.15.121:0/850457657 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff3480483d0 con 0x7ff350152780 2023-11-15T03:12:21.964 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.955+0000 7ff35625b700 1 -- 172.21.15.121:0/850457657 --> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff35006c150 con 0x7ff33005a510 2023-11-15T03:12:21.970 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.959+0000 7ff347fff700 1 -- 172.21.15.121:0/850457657 <== mgr.4110 v2:172.21.15.121:6832/33042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+68338 (secure 0 0 0) 0x7ff35006c150 con 0x7ff33005a510 2023-11-15T03:12:21.972 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:12:21.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.963+0000 7ff35625b700 1 -- 172.21.15.121:0/850457657 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff33005a510 msgr2=0x7ff33005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:21.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.963+0000 7ff35625b700 1 --2- 172.21.15.121:0/850457657 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff33005a510 0x7ff33005c9d0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7ff3400099f0 tx=0x7ff340009780).stop 2023-11-15T03:12:21.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.963+0000 7ff35625b700 1 -- 172.21.15.121:0/850457657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff350152780 msgr2=0x7ff3500abb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:21.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.963+0000 7ff35625b700 1 --2- 172.21.15.121:0/850457657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff350152780 0x7ff3500abb20 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 rx=0x7ff34800e8d0 tx=0x7ff348009060).stop 2023-11-15T03:12:21.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.963+0000 7ff35625b700 1 -- 172.21.15.121:0/850457657 shutdown_connections 2023-11-15T03:12:21.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.963+0000 7ff35625b700 1 --2- 172.21.15.121:0/850457657 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7ff33005a510 0x7ff33005c9d0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:21.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.963+0000 7ff35625b700 1 --2- 172.21.15.121:0/850457657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff350152780 0x7ff3500abb20 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:21.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.963+0000 7ff35625b700 1 --2- 172.21.15.121:0/850457657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3500c65d0 0x7ff350153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:21.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.963+0000 7ff35625b700 1 --2- 172.21.15.121:0/850457657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3500a5510 0x7ff35013f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:21.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.963+0000 7ff35625b700 1 -- 172.21.15.121:0/850457657 >> 172.21.15.121:0/850457657 conn(0x7ff350062190 msgr2=0x7ff350066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:12:21.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.967+0000 7ff35625b700 1 -- 172.21.15.121:0/850457657 shutdown_connections 2023-11-15T03:12:21.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:21.967+0000 7ff35625b700 1 -- 172.21.15.121:0/850457657 wait complete. 2023-11-15T03:12:21.978 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-11-15T03:12:21.996 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":612,"stamp":"2023-11-15T03:12:20.276153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2445372,"kb_used_data":15356,"kb_used_omap":0,"kb_used_meta":2429952,"kb_avail":752529348,"statfs":{"total":773094113280,"available":770590052352,"internally_reserved":0,"allocated":15724544,"data_stored":7961611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2488270848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-33941,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-60,"num_read_kb":-61,"num_write":-90,"num_write_kb":-62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.146079"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":134,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-11-15T03:12:02.054659+0000","last_change":"2023-11-15T03:12:02.054659+0000","last_active":"2023-11-15T03:12:02.054659+0000","last_peered":"2023-11-15T03:12:02.054659+0000","last_clean":"2023-11-15T03:12:02.054659+0000","last_became_active":"2023-11-15T02:56:46.206687+0000","last_became_peered":"2023-11-15T02:56:46.206687+0000","last_unstale":"2023-11-15T03:12:02.054659+0000","last_undegraded":"2023-11-15T03:12:02.054659+0000","last_fullsized":"2023-11-15T03:12:02.054659+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:12:02.054399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:12:02.054399+0000","last_clean_scrub_stamp":"2023-11-15T03:12:02.054399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":137,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-11-15T03:12:01.767095+0000","last_change":"2023-11-15T03:12:01.767095+0000","last_active":"2023-11-15T03:12:01.767095+0000","last_peered":"2023-11-15T03:12:01.767095+0000","last_clean":"2023-11-15T03:12:01.767095+0000","last_became_active":"2023-11-15T02:56:46.208000+0000","last_became_peered":"2023-11-15T02:56:46.208000+0000","last_unstale":"2023-11-15T03:12:01.767095+0000","last_undegraded":"2023-11-15T03:12:01.767095+0000","last_fullsized":"2023-11-15T03:12:01.767095+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:12:01.766874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:12:01.766874+0000","last_clean_scrub_stamp":"2023-11-15T03:12:01.766874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":132,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-11-15T03:12:06.988659+0000","last_change":"2023-11-15T03:12:06.988659+0000","last_active":"2023-11-15T03:12:06.988659+0000","last_peered":"2023-11-15T03:12:06.988659+0000","last_clean":"2023-11-15T03:12:06.988659+0000","last_became_active":"2023-11-15T02:56:46.258403+0000","last_became_peered":"2023-11-15T02:56:46.258403+0000","last_unstale":"2023-11-15T03:12:06.988659+0000","last_undegraded":"2023-11-15T03:12:06.988659+0000","last_fullsized":"2023-11-15T03:12:06.988659+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:12:06.988397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:12:06.988397+0000","last_clean_scrub_stamp":"2023-11-15T03:12:06.988397+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":137,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-11-15T03:11:59.774506+0000","last_change":"2023-11-15T03:11:59.774506+0000","last_active":"2023-11-15T03:11:59.774506+0000","last_peered":"2023-11-15T03:11:59.774506+0000","last_clean":"2023-11-15T03:11:59.774506+0000","last_became_active":"2023-11-15T02:56:46.257199+0000","last_became_peered":"2023-11-15T02:56:46.257199+0000","last_unstale":"2023-11-15T03:11:59.774506+0000","last_undegraded":"2023-11-15T03:11:59.774506+0000","last_fullsized":"2023-11-15T03:11:59.774506+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:59.774280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:59.774280+0000","last_clean_scrub_stamp":"2023-11-15T03:11:59.774280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":148,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-11-15T03:12:01.646101+0000","last_change":"2023-11-15T03:12:01.646101+0000","last_active":"2023-11-15T03:12:01.646101+0000","last_peered":"2023-11-15T03:12:01.646101+0000","last_clean":"2023-11-15T03:12:01.646101+0000","last_became_active":"2023-11-15T02:56:46.201775+0000","last_became_peered":"2023-11-15T02:56:46.201775+0000","last_unstale":"2023-11-15T03:12:01.646101+0000","last_undegraded":"2023-11-15T03:12:01.646101+0000","last_fullsized":"2023-11-15T03:12:01.646101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:12:01.645944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:12:01.645944+0000","last_clean_scrub_stamp":"2023-11-15T03:12:01.645944+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":132,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-11-15T03:11:58.464808+0000","last_change":"2023-11-15T03:11:58.464808+0000","last_active":"2023-11-15T03:11:58.464808+0000","last_peered":"2023-11-15T03:11:58.464808+0000","last_clean":"2023-11-15T03:11:58.464808+0000","last_became_active":"2023-11-15T02:56:46.382852+0000","last_became_peered":"2023-11-15T02:56:46.382852+0000","last_unstale":"2023-11-15T03:11:58.464808+0000","last_undegraded":"2023-11-15T03:11:58.464808+0000","last_fullsized":"2023-11-15T03:11:58.464808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:11:58.464661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:11:58.464661+0000","last_clean_scrub_stamp":"2023-11-15T03:11:58.464661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":132,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-11-15T03:12:05.000695+0000","last_change":"2023-11-15T03:12:05.000695+0000","last_active":"2023-11-15T03:12:05.000695+0000","last_peered":"2023-11-15T03:12:05.000695+0000","last_clean":"2023-11-15T03:12:05.000695+0000","last_became_active":"2023-11-15T02:56:46.201722+0000","last_became_peered":"2023-11-15T02:56:46.201722+0000","last_unstale":"2023-11-15T03:12:05.000695+0000","last_undegraded":"2023-11-15T03:12:05.000695+0000","last_fullsized":"2023-11-15T03:12:05.000695+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:12:05.000473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:12:05.000473+0000","last_clean_scrub_stamp":"2023-11-15T03:12:05.000473+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":148,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-11-15T03:12:04.599563+0000","last_change":"2023-11-15T03:12:04.599563+0000","last_active":"2023-11-15T03:12:04.599563+0000","last_peered":"2023-11-15T03:12:04.599563+0000","last_clean":"2023-11-15T03:12:04.599563+0000","last_became_active":"2023-11-15T02:56:46.206591+0000","last_became_peered":"2023-11-15T02:56:46.206591+0000","last_unstale":"2023-11-15T03:12:04.599563+0000","last_undegraded":"2023-11-15T03:12:04.599563+0000","last_fullsized":"2023-11-15T03:12:04.599563+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:12:04.599387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:12:04.599387+0000","last_clean_scrub_stamp":"2023-11-15T03:12:04.599387+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":133,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-11-15T03:12:03.037775+0000","last_change":"2023-11-15T03:12:03.037775+0000","last_active":"2023-11-15T03:12:03.037775+0000","last_peered":"2023-11-15T03:12:03.037775+0000","last_clean":"2023-11-15T03:12:03.037775+0000","last_became_active":"2023-11-15T02:56:45.187395+0000","last_became_peered":"2023-11-15T02:56:45.187395+0000","last_unstale":"2023-11-15T03:12:03.037775+0000","last_undegraded":"2023-11-15T03:12:03.037775+0000","last_fullsized":"2023-11-15T03:12:03.037775+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-15T03:12:03.037567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-15T03:12:03.037567+0000","last_clean_scrub_stamp":"2023-11-15T03:12:03.037567+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542333,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307208,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064632,"statfs":{"total":96636764160,"available":96322183168,"internally_reserved":0,"allocated":1966080,"data_stored":995360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0680000000000001,"15min":1.125},"min":{"1min":0.75600000000000001,"5min":0.627,"15min":0.55800000000000005},"max":{"1min":1.5409999999999999,"5min":1.6970000000000001,"15min":10.961},"last":1.335},{"interface":"front","average":{"1min":1.121,"5min":1.097,"15min":1.1060000000000001},"min":{"1min":0.67100000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.4339999999999999,"5min":1.6160000000000001,"15min":3.4910000000000001},"last":0.89900000000000002}]},{"osd":1,"last update":"Wed Nov 15 03:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1180000000000001,"15min":1.141},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.45,"5min":1.552,"15min":5.4930000000000003},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.157,"15min":1.1799999999999999},"min":{"1min":0.81899999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.502,"5min":1.766,"15min":6.8239999999999998},"last":1.1060000000000001}]},{"osd":2,"last update":"Wed Nov 15 03:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0900000000000001,"15min":1.1259999999999999},"min":{"1min":0.755,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.395,"5min":1.452,"15min":5.7000000000000002},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1990000000000001,"15min":1.2210000000000001},"min":{"1min":0.80700000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4419999999999999,"5min":1.9219999999999999,"15min":2.75},"last":1.1739999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.175,"15min":1.232},"min":{"1min":0.75900000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.4099999999999999,"5min":1.6259999999999999,"15min":12.907},"last":1.238},{"interface":"front","average":{"1min":1.286,"5min":1.23,"15min":1.2709999999999999},"min":{"1min":1.0169999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4850000000000001,"5min":1.573,"15min":9.9909999999999997},"last":1.087}]},{"osd":4,"last update":"Wed Nov 15 03:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1120000000000001,"15min":1.125},"min":{"1min":0.72399999999999998,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.5149999999999999,"5min":1.8939999999999999,"15min":2.5649999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.135,"15min":1.1839999999999999},"min":{"1min":0.97599999999999998,"5min":0.5,"15min":0.5},"max":{"1min":1.427,"5min":1.72,"15min":4.226},"last":1.149}]},{"osd":5,"last update":"Wed Nov 15 03:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2509999999999999,"15min":1.2729999999999999},"min":{"1min":1.002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5680000000000001,"5min":1.8300000000000001,"15min":2.6890000000000001},"last":1.256},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.2490000000000001,"15min":1.28},"min":{"1min":1.022,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.597,"5min":1.7330000000000001,"15min":2.4550000000000001},"last":1.4159999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3480000000000001,"15min":1.365},"min":{"1min":0.89200000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6739999999999999,"5min":1.954,"15min":3.2789999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.266,"15min":1.3109999999999999},"min":{"1min":1.103,"5min":0.70199999999999996,"15min":0.63600000000000001},"max":{"1min":1.4930000000000001,"5min":1.76,"15min":4.7800000000000002},"last":1.3959999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542333,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311304,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":94060536,"statfs":{"total":96636764160,"available":96317988864,"internally_reserved":0,"allocated":1966080,"data_stored":995360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.0149999999999999,"15min":1.0640000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.44500000000000001},"max":{"1min":1.27,"5min":1.5760000000000001,"15min":4.3159999999999998},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1359999999999999,"15min":1.107},"min":{"1min":0.91700000000000004,"5min":0.59799999999999998,"15min":0.58599999999999997},"max":{"1min":1.712,"5min":1.712,"15min":2.2480000000000002},"last":1.1240000000000001}]},{"osd":1,"last update":"Wed Nov 15 03:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.1060000000000001,"15min":1.1140000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.5980000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.1100000000000001,"15min":1.1299999999999999},"min":{"1min":0.77400000000000002,"5min":0.61099999999999999,"15min":0.60699999999999998},"max":{"1min":1.427,"5min":1.8100000000000001,"15min":2.8570000000000002},"last":1.0649999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1930000000000001,"15min":1.1890000000000001},"min":{"1min":0.76600000000000001,"5min":0.71699999999999997,"15min":0.49099999999999999},"max":{"1min":1.417,"5min":1.7090000000000001,"15min":2.621},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.163,"15min":1.1579999999999999},"min":{"1min":0.76400000000000001,"5min":0.71699999999999997,"15min":0.64000000000000001},"max":{"1min":1.5149999999999999,"5min":1.8360000000000001,"15min":2.0059999999999998},"last":1.3480000000000001}]},{"osd":3,"last update":"Wed Nov 15 03:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.228,"15min":1.2170000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.50900000000000001},"max":{"1min":1.4870000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.252},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.238,"15min":1.2549999999999999},"min":{"1min":0.83799999999999997,"5min":0.68999999999999995,"15min":0.67100000000000004},"max":{"1min":1.585,"5min":1.8640000000000001,"15min":2.8359999999999999},"last":1.1539999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0700000000000001,"15min":1.0940000000000001},"min":{"1min":0.85099999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5660000000000001,"5min":1.6839999999999999,"15min":4.5350000000000001},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.226,"15min":1.2170000000000001},"min":{"1min":0.72499999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7749999999999999,"5min":2.004,"15min":2.0739999999999998},"last":1.107}]},{"osd":5,"last update":"Wed Nov 15 03:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.387,"15min":1.4139999999999999},"min":{"1min":0.96699999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.0819999999999999,"5min":2.1469999999999998,"15min":2.157},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.242,"15min":1.254},"min":{"1min":0.97399999999999998,"5min":0.621,"15min":0.60899999999999999},"max":{"1min":1.6699999999999999,"5min":1.8779999999999999,"15min":2.9460000000000002},"last":1.389}]},{"osd":7,"last update":"Wed Nov 15 03:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2390000000000001,"15min":1.224},"min":{"1min":0.871,"5min":0.72199999999999998,"15min":0.56899999999999995},"max":{"1min":1.879,"5min":1.9239999999999999,"15min":3.1989999999999998},"last":1.02},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.419,"15min":1.4179999999999999},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.65200000000000002},"max":{"1min":2.194,"5min":2.194,"15min":2.9350000000000001},"last":1.2170000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542334,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":307208,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064632,"statfs":{"total":96636764160,"available":96322183168,"internally_reserved":0,"allocated":1966080,"data_stored":995360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0629999999999999,"15min":1.0820000000000001},"min":{"1min":0.59999999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5609999999999999,"5min":1.5620000000000001,"15min":1.5840000000000001},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0600000000000001,"15min":1.0620000000000001},"min":{"1min":0.60699999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.319,"5min":1.6499999999999999,"15min":1.677},"last":1.246}]},{"osd":1,"last update":"Wed Nov 15 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.137,"15min":1.1259999999999999},"min":{"1min":0.92600000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.524,"5min":1.7410000000000001,"15min":2.617},"last":1.524},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1339999999999999,"15min":1.1499999999999999},"min":{"1min":0.68200000000000005,"5min":0.629,"15min":0.54800000000000004},"max":{"1min":1.4950000000000001,"5min":1.6319999999999999,"15min":1.6619999999999999},"last":1.1739999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.181,"15min":1.1890000000000001},"min":{"1min":0.94399999999999995,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4690000000000001,"5min":1.823,"15min":1.9450000000000001},"last":1.355},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1599999999999999,"15min":1.1579999999999999},"min":{"1min":0.69699999999999995,"5min":0.621,"15min":0.52600000000000002},"max":{"1min":1.3640000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.3280000000000001}]},{"osd":3,"last update":"Wed Nov 15 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.1359999999999999,"15min":1.143},"min":{"1min":0.82599999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3879999999999999,"5min":1.5700000000000001,"15min":1.8660000000000001},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.27,"15min":1.2649999999999999},"min":{"1min":1.0029999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.714,"5min":1.8200000000000001,"15min":2.5630000000000002},"last":1.4299999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.1839999999999999,"15min":1.181},"min":{"1min":0.96499999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.643,"5min":1.643,"15min":2.0840000000000001},"last":1.643},{"interface":"front","average":{"1min":1.196,"5min":1.177,"15min":1.1850000000000001},"min":{"1min":0.88600000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.4039999999999999,"5min":1.8400000000000001,"15min":2.3570000000000002},"last":1.4039999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2569999999999999,"15min":1.26},"min":{"1min":1.073,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.613,"5min":1.913,"15min":2.6640000000000001},"last":1.613},{"interface":"front","average":{"1min":1.29,"5min":1.304,"15min":1.3149999999999999},"min":{"1min":0.93799999999999994,"5min":0.67600000000000005,"15min":0.57299999999999995},"max":{"1min":1.6339999999999999,"5min":1.9139999999999999,"15min":2.173},"last":1.4570000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.2929999999999999,"15min":1.298},"min":{"1min":0.75600000000000001,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.54,"5min":1.849,"15min":2.6030000000000002},"last":1.47},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3640000000000001,"15min":1.3819999999999999},"min":{"1min":1.0800000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.7,"5min":2.069,"15min":2.7669999999999999},"last":1.7}]}]},{"osd":3,"up_from":14,"seq":60129542334,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303112,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068728,"statfs":{"total":96636764160,"available":96326377472,"internally_reserved":0,"allocated":1966080,"data_stored":995360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1539999999999999,"5min":1.4219999999999999,"15min":1.528},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.878,"15min":0.88800000000000001},"min":{"1min":0.59799999999999998,"5min":0.55700000000000005,"15min":0.39800000000000002},"max":{"1min":1.103,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.448}]},{"osd":1,"last update":"Wed Nov 15 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.83799999999999997,"15min":0.86699999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.4670000000000001},"last":1.077},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.97099999999999997,"15min":0.98999999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.55800000000000005},"max":{"1min":1.1799999999999999,"5min":1.446,"15min":1.544},"last":1.105}]},{"osd":2,"last update":"Wed Nov 15 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.016,"15min":1.02},"min":{"1min":0.67500000000000004,"5min":0.64200000000000002,"15min":0.54600000000000004},"max":{"1min":1.3069999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.218},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.038,"15min":1.0620000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.3859999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.177}]},{"osd":4,"last update":"Wed Nov 15 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.2190000000000001,"15min":1.224},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.71399999999999997},"max":{"1min":1.3859999999999999,"5min":1.554,"15min":1.9139999999999999},"last":1.355},{"interface":"front","average":{"1min":1.173,"5min":1.2050000000000001,"15min":1.2210000000000001},"min":{"1min":0.80100000000000005,"5min":0.78500000000000003,"15min":0.621},"max":{"1min":1.45,"5min":1.6160000000000001,"15min":1.726},"last":1.389}]},{"osd":5,"last update":"Wed Nov 15 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.22,"15min":1.26},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.64300000000000002},"max":{"1min":1.476,"5min":1.7290000000000001,"15min":2.0459999999999998},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.26,"15min":1.2669999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.63500000000000001},"max":{"1min":1.5569999999999999,"5min":1.6479999999999999,"15min":1.927},"last":1.5409999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.3300000000000001,"15min":1.335},"min":{"1min":0.88300000000000001,"5min":0.875,"15min":0.72899999999999998},"max":{"1min":1.5409999999999999,"5min":1.8129999999999999,"15min":1.9970000000000001},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.341,"15min":1.367},"min":{"1min":0.96599999999999997,"5min":0.871,"15min":0.77700000000000002},"max":{"1min":1.5920000000000001,"5min":1.9410000000000001,"15min":2.3570000000000002},"last":1.5640000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.337,"15min":1.3400000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.68100000000000005},"max":{"1min":1.6419999999999999,"5min":1.6970000000000001,"15min":2.0470000000000002},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.333,"15min":1.3500000000000001},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.66600000000000004},"max":{"1min":1.613,"5min":1.8779999999999999,"15min":2.1040000000000001},"last":1.5269999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542334,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303108,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068732,"statfs":{"total":96636764160,"available":96326381568,"internally_reserved":0,"allocated":1961984,"data_stored":994091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75900000000000001,"15min":0.76900000000000002},"min":{"1min":0.60399999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.115,"5min":1.2350000000000001,"15min":1.409},"last":1.3149999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.84599999999999997,"15min":0.83999999999999997},"min":{"1min":0.57799999999999996,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1000000000000001,"5min":1.4950000000000001,"15min":1.619},"last":0.78100000000000003}]},{"osd":1,"last update":"Wed Nov 15 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.92200000000000004,"15min":0.92000000000000004},"min":{"1min":0.65300000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3600000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.94799999999999995,"15min":0.95099999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.21,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.91900000000000004}]},{"osd":3,"last update":"Wed Nov 15 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94999999999999996,"15min":0.96699999999999997},"min":{"1min":0.56599999999999995,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.6259999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.099,"5min":1.119,"15min":1.1240000000000001},"min":{"1min":0.71799999999999997,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.53,"5min":1.5549999999999999,"15min":1.5800000000000001},"last":1.3919999999999999}]},{"osd":4,"last update":"Wed Nov 15 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.222,"15min":1.2070000000000001},"min":{"1min":0.84499999999999997,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.595,"5min":1.595,"15min":1.9410000000000001},"last":1.462},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.2110000000000001,"15min":1.21},"min":{"1min":0.88800000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.02}]},{"osd":5,"last update":"Wed Nov 15 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1679999999999999,"15min":1.175},"min":{"1min":0.84399999999999997,"5min":0.68100000000000005,"15min":0.67000000000000004},"max":{"1min":1.4059999999999999,"5min":1.625,"15min":1.8939999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2569999999999999,"15min":1.2689999999999999},"min":{"1min":1.04,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4610000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.5269999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2569999999999999,"15min":1.2629999999999999},"min":{"1min":0.91300000000000003,"5min":0.62,"15min":0.62},"max":{"1min":1.6599999999999999,"5min":2,"15min":2.137},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.89500000000000002,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5209999999999999,"5min":1.9350000000000001,"15min":2.0870000000000002},"last":1.1719999999999999}]},{"osd":7,"last update":"Wed Nov 15 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.286,"15min":1.3029999999999999},"min":{"1min":0.878,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.851,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.198},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.302,"15min":1.2909999999999999},"min":{"1min":0.84099999999999997,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.702,"5min":1.881,"15min":2.0350000000000001},"last":1.252}]}]},{"osd":0,"up_from":14,"seq":60129542334,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307208,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064632,"statfs":{"total":96636764160,"available":96322183168,"internally_reserved":0,"allocated":1966080,"data_stored":995360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 15 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80500000000000005,"15min":0.82699999999999996},"min":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.36199999999999999},"max":{"1min":1.117,"5min":1.756,"15min":4.3630000000000004},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90800000000000003,"15min":0.93700000000000006},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.40000000000000002},"max":{"1min":1.397,"5min":1.6719999999999999,"15min":6.157},"last":0.875}]},{"osd":2,"last update":"Wed Nov 15 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.96299999999999997,"15min":0.97499999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.46800000000000003},"max":{"1min":1.2789999999999999,"5min":1.625,"15min":4.1699999999999999},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.995,"15min":1.0289999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.47799999999999998},"max":{"1min":1.246,"5min":1.879,"15min":6.8369999999999997},"last":1.1439999999999999}]},{"osd":3,"last update":"Wed Nov 15 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.92100000000000004,"15min":0.94899999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5620000000000001,"5min":1.6519999999999999,"15min":3.5070000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0880000000000001,"15min":1.105},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.601,"5min":2.0070000000000001,"15min":5.2149999999999999},"last":1.069}]},{"osd":4,"last update":"Wed Nov 15 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.2150000000000001,"15min":1.2589999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.738,"5min":1.944,"15min":6.2000000000000002},"last":1.25},{"interface":"front","average":{"1min":1.206,"5min":1.177,"15min":1.198},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.603,"5min":1.6779999999999999,"15min":3.577},"last":1.0269999999999999}]},{"osd":5,"last update":"Wed Nov 15 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.238,"15min":1.254},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7689999999999999,"5min":1.857,"15min":3.548},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.272,"15min":1.29},"min":{"1min":0.92400000000000004,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.556,"5min":1.774,"15min":4.4080000000000004},"last":1.1879999999999999}]},{"osd":6,"last update":"Wed Nov 15 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3280000000000001,"15min":1.343},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.909,"5min":1.915,"15min":4.4489999999999998},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.288,"15min":1.3320000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.6930000000000001,"5min":2.056,"15min":6.9690000000000003},"last":1.355}]},{"osd":7,"last update":"Wed Nov 15 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3300000000000001,"15min":1.3979999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.5389999999999999,"5min":1.994,"15min":18.253},"last":1.431},{"interface":"front","average":{"1min":1.321,"5min":1.294,"15min":1.3420000000000001},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.79000000000000004},"max":{"1min":1.657,"5min":1.8169999999999999,"15min":11.006},"last":1.2050000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575038,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303112,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068728,"statfs":{"total":96636764160,"available":96326377472,"internally_reserved":0,"allocated":1966080,"data_stored":995360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:12:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.077,"15min":1.075},"min":{"1min":0.83299999999999996,"5min":0.64300000000000002,"15min":0.49399999999999999},"max":{"1min":1.417,"5min":1.827,"15min":2.3370000000000002},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.048,"5min":1.0429999999999999,"15min":1.0449999999999999},"min":{"1min":0.79200000000000004,"5min":0.61099999999999999,"15min":0.60799999999999998},"max":{"1min":1.516,"5min":1.6519999999999999,"15min":3.46},"last":0.98199999999999998}]},{"osd":1,"last update":"Wed Nov 15 03:12:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.099,"15min":1.109},"min":{"1min":0.89800000000000002,"5min":0.72699999999999998,"15min":0.58999999999999997},"max":{"1min":1.4670000000000001,"5min":1.516,"15min":3.6349999999999998},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.151,"5min":1.1579999999999999,"15min":1.1399999999999999},"min":{"1min":0.94599999999999995,"5min":0.76000000000000001,"15min":0.63700000000000001},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":3.516},"last":1.0069999999999999}]},{"osd":2,"last update":"Wed Nov 15 03:12:17 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.147,"15min":1.1599999999999999},"min":{"1min":0.78600000000000003,"5min":0.74099999999999999,"15min":0.73199999999999998},"max":{"1min":1.448,"5min":1.774,"15min":5.4180000000000001},"last":1.083},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2070000000000001,"15min":1.1970000000000001},"min":{"1min":0.94199999999999995,"5min":0.77300000000000002,"15min":0.69599999999999995},"max":{"1min":1.8100000000000001,"5min":2.0840000000000001,"15min":3.5859999999999999},"last":1.407}]},{"osd":3,"last update":"Wed Nov 15 03:12:17 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2130000000000001,"15min":1.1930000000000001},"min":{"1min":0.93300000000000005,"5min":0.87,"15min":0.71799999999999997},"max":{"1min":1.7749999999999999,"5min":1.8560000000000001,"15min":2.141},"last":0.875},{"interface":"front","average":{"1min":1.325,"5min":1.3,"15min":1.3069999999999999},"min":{"1min":0.89100000000000001,"5min":0.88400000000000001,"15min":0.78600000000000003},"max":{"1min":1.6060000000000001,"5min":1.6739999999999999,"15min":4.2729999999999997},"last":1.216}]},{"osd":4,"last update":"Wed Nov 15 03:12:17 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.2529999999999999,"15min":1.262},"min":{"1min":0.79500000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.992,"5min":1.992,"15min":3.706},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.288,"15min":1.288},"min":{"1min":0.89400000000000002,"5min":0.86399999999999999,"15min":0.64100000000000001},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":6.1180000000000003},"last":1.381}]},{"osd":6,"last update":"Wed Nov 15 03:12:17 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.282,"15min":1.254},"min":{"1min":0.94099999999999995,"5min":0.80700000000000005,"15min":0.749},"max":{"1min":1.698,"5min":1.843,"15min":4.2309999999999999},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.3169999999999999,"15min":1.3280000000000001},"min":{"1min":1.208,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.907,"5min":2.2240000000000002,"15min":3.641},"last":1.484}]},{"osd":7,"last update":"Wed Nov 15 03:12:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3029999999999999,"15min":1.2889999999999999},"min":{"1min":1.131,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.883,"5min":1.988,"15min":3.516},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.482,"5min":1.4039999999999999,"15min":1.3879999999999999},"min":{"1min":1.1759999999999999,"5min":0.95899999999999996,"15min":0.89500000000000002},"max":{"1min":1.986,"5min":2.282,"15min":3.754},"last":1.353}]}]},{"osd":1,"up_from":13,"seq":55834575037,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303112,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068728,"statfs":{"total":96636764160,"available":96326377472,"internally_reserved":0,"allocated":1966080,"data_stored":995360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 15 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.79100000000000004,"15min":0.80300000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.48799999999999999},"max":{"1min":0.997,"5min":1.671,"15min":1.671},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.86199999999999999,"15min":0.88},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.46600000000000003},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.6899999999999999},"last":0.873}]},{"osd":2,"last update":"Wed Nov 15 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.96599999999999997,"15min":0.94799999999999995},"min":{"1min":0.627,"5min":0.60999999999999999,"15min":0.44600000000000001},"max":{"1min":1.373,"5min":1.806,"15min":1.806},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88700000000000001,"15min":0.90400000000000003},"min":{"1min":0.65300000000000002,"5min":0.58999999999999997,"15min":0.49199999999999999},"max":{"1min":1.157,"5min":1.3360000000000001,"15min":1.887},"last":1.04}]},{"osd":3,"last update":"Wed Nov 15 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.1180000000000001,"15min":1.1279999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.42399999999999999},"max":{"1min":1.409,"5min":1.6160000000000001,"15min":1.712},"last":1.306},{"interface":"front","average":{"1min":1.02,"5min":1.038,"15min":1.0449999999999999},"min":{"1min":0.58199999999999996,"5min":0.52300000000000002,"15min":0.46000000000000002},"max":{"1min":1.466,"5min":1.5409999999999999,"15min":1.7090000000000001},"last":1.341}]},{"osd":4,"last update":"Wed Nov 15 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.2070000000000001,"15min":1.1970000000000001},"min":{"1min":0.92500000000000004,"5min":0.83699999999999997,"15min":0.65100000000000002},"max":{"1min":1.577,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1930000000000001,"15min":1.2050000000000001},"min":{"1min":0.95999999999999996,"5min":0.88600000000000001,"15min":0.50800000000000001},"max":{"1min":1.3680000000000001,"5min":1.5429999999999999,"15min":2.206},"last":1.196}]},{"osd":5,"last update":"Wed Nov 15 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.256,"15min":1.246},"min":{"1min":0.871,"5min":0.871,"15min":0.746},"max":{"1min":1.7649999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.397},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2430000000000001,"15min":1.2509999999999999},"min":{"1min":0.98599999999999999,"5min":0.92400000000000004,"15min":0.56100000000000005},"max":{"1min":1.5129999999999999,"5min":1.718,"15min":2.4609999999999999},"last":1.504}]},{"osd":6,"last update":"Wed Nov 15 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.341,"15min":1.3280000000000001},"min":{"1min":1.0629999999999999,"5min":0.92300000000000004,"15min":0.82499999999999996},"max":{"1min":1.554,"5min":1.8640000000000001,"15min":1.948},"last":1.556},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.419,"15min":1.407},"min":{"1min":1.008,"5min":0.93700000000000006,"15min":0.87},"max":{"1min":1.8839999999999999,"5min":1.9670000000000001,"15min":2.3290000000000002},"last":1.6890000000000001}]},{"osd":7,"last update":"Wed Nov 15 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3300000000000001,"15min":1.3200000000000001},"min":{"1min":1.0640000000000001,"5min":0.92000000000000004,"15min":0.84199999999999997},"max":{"1min":1.706,"5min":2.089,"15min":2.161},"last":1.6579999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.4059999999999999,"15min":1.381},"min":{"1min":1.0960000000000001,"5min":1.0249999999999999,"15min":0.78600000000000003},"max":{"1min":1.7549999999999999,"5min":2.137,"15min":2.2349999999999999},"last":1.585}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-15T03:12:21.999 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-15T03:12:22.157 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.147+0000 7f4d20d07700 1 Processor -- start 2023-11-15T03:12:22.158 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.147+0000 7f4d20d07700 1 -- start start 2023-11-15T03:12:22.159 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.151+0000 7f4d20d07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c0acd00 0x7f4d1c0ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:22.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.151+0000 7f4d20d07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4d1c0c5c70 0x7f4d1c1536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:22.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.151+0000 7f4d20d07700 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1c065ea0 con 0x7f4d1c066330 2023-11-15T03:12:22.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.151+0000 7f4d20d07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1c066bb0 con 0x7f4d1c0c5c70 2023-11-15T03:12:22.160 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.151+0000 7f4d20d07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c065890 con 0x7f4d1c0acd00 2023-11-15T03:12:22.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.151+0000 7f4d1bfff700 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4d1c066330 0x7f4d1c153bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:35662/0 (socket says 172.21.15.121:35662) 2023-11-15T03:12:22.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.151+0000 7f4d1bfff700 1 -- 172.21.15.121:0/1876290533 learned_addr learned my addr 172.21.15.121:0/1876290533 (peer_addr_for_me v1:172.21.15.121:0/0) 2023-11-15T03:12:22.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.151+0000 7f4d1affd700 1 --2- 172.21.15.121:0/1876290533 >> v2:172.21.15.190:3300/0 conn(0x7f4d1c0c5c70 0x7f4d1c1536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:22.162 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.151+0000 7f4d1b7fe700 1 --2- 172.21.15.121:0/1876290533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c0acd00 0x7f4d1c0ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:22.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.151+0000 7f4d1a7fc700 1 -- 172.21.15.121:0/1876290533 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034579392 0 0) 0x7f4d1c065ea0 con 0x7f4d1c066330 2023-11-15T03:12:22.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.151+0000 7f4d1a7fc700 1 -- 172.21.15.121:0/1876290533 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d040033f0 con 0x7f4d1c066330 2023-11-15T03:12:22.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.151+0000 7f4d1affd700 1 -- 172.21.15.121:0/1876290533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c0acd00 msgr2=0x7f4d1c0ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:12:22.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.151+0000 7f4d1affd700 1 --2- 172.21.15.121:0/1876290533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c0acd00 0x7f4d1c0ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:22.163 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.151+0000 7f4d1affd700 1 -- 172.21.15.121:0/1876290533 >> v1:172.21.15.121:6789/0 conn(0x7f4d1c066330 legacy=0x7f4d1c153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:22.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.151+0000 7f4d1affd700 1 -- 172.21.15.121:0/1876290533 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c0c64a0 con 0x7f4d1c0c5c70 2023-11-15T03:12:22.164 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.151+0000 7f4d1affd700 1 --2- 172.21.15.121:0/1876290533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c0c5c70 0x7f4d1c1536c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f4d10009760 tx=0x7f4d10009060).ready entity=mon.1 client_cookie=7d7f9f2f155b1b35 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:22.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.151+0000 7f4d1a7fc700 1 -- 172.21.15.121:0/1876290533 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1000d4a0 con 0x7f4d1c0c5c70 2023-11-15T03:12:22.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.151+0000 7f4d1a7fc700 1 -- 172.21.15.121:0/1876290533 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1000d600 con 0x7f4d1c0c5c70 2023-11-15T03:12:22.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.151+0000 7f4d1a7fc700 1 -- 172.21.15.121:0/1876290533 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1000d980 con 0x7f4d1c0c5c70 2023-11-15T03:12:22.165 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.155+0000 7f4d20d07700 1 -- 172.21.15.121:0/1876290533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c0c5c70 msgr2=0x7f4d1c1536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:22.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.155+0000 7f4d20d07700 1 --2- 172.21.15.121:0/1876290533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c0c5c70 0x7f4d1c1536c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f4d10009760 tx=0x7f4d10009060).stop 2023-11-15T03:12:22.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.155+0000 7f4d20d07700 1 -- 172.21.15.121:0/1876290533 shutdown_connections 2023-11-15T03:12:22.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.155+0000 7f4d20d07700 1 --2- 172.21.15.121:0/1876290533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c0c5c70 0x7f4d1c1536c0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:22.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.155+0000 7f4d20d07700 1 --2- 172.21.15.121:0/1876290533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c0acd00 0x7f4d1c0ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:22.166 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.155+0000 7f4d20d07700 1 -- 172.21.15.121:0/1876290533 >> 172.21.15.121:0/1876290533 conn(0x7f4d1c063fc0 msgr2=0x7f4d1c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:12:22.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.155+0000 7f4d20d07700 1 -- 172.21.15.121:0/1876290533 shutdown_connections 2023-11-15T03:12:22.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.155+0000 7f4d20d07700 1 -- 172.21.15.121:0/1876290533 wait complete. 2023-11-15T03:12:22.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.155+0000 7f4d20d07700 1 Processor -- start 2023-11-15T03:12:22.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d20d07700 1 -- start start 2023-11-15T03:12:22.171 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d20d07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c066330 0x7f4d1c0a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:22.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d20d07700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d1c0acd00 0x7f4d1c153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:22.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d20d07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c0c5c70 0x7f4d1c153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:22.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d20d07700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c1521c0 con 0x7f4d1c0acd00 2023-11-15T03:12:22.172 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d20d07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c153f10 con 0x7f4d1c066330 2023-11-15T03:12:22.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d20d07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c1541f0 con 0x7f4d1c0c5c70 2023-11-15T03:12:22.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d1affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d1c0acd00 0x7f4d1c153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:22.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d1affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d1c0acd00 0x7f4d1c153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39066/0 (socket says 172.21.15.121:39066) 2023-11-15T03:12:22.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d1affd700 1 -- 172.21.15.121:0/2449164542 learned_addr learned my addr 172.21.15.121:0/2449164542 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:12:22.173 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d1b7fe700 1 --2- 172.21.15.121:0/2449164542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c066330 0x7f4d1c0a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:22.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d1bfff700 1 --2- 172.21.15.121:0/2449164542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c0c5c70 0x7f4d1c153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:22.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d1affd700 1 -- 172.21.15.121:0/2449164542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c0c5c70 msgr2=0x7f4d1c153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:22.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d1affd700 1 --2- 172.21.15.121:0/2449164542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c0c5c70 0x7f4d1c153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:22.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d1affd700 1 -- 172.21.15.121:0/2449164542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c066330 msgr2=0x7f4d1c0a3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:22.174 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d1affd700 1 --2- 172.21.15.121:0/2449164542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c066330 0x7f4d1c0a3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:22.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d1affd700 1 -- 172.21.15.121:0/2449164542 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d100092a0 con 0x7f4d1c0acd00 2023-11-15T03:12:22.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d1bfff700 1 --2- 172.21.15.121:0/2449164542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c0c5c70 0x7f4d1c153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-15T03:12:22.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d1affd700 1 --2- 172.21.15.121:0/2449164542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d1c0acd00 0x7f4d1c153490 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 rx=0x7f4d1c064f20 tx=0x7f4d10016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:22.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d037fe700 1 -- 172.21.15.121:0/2449164542 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10000bc0 con 0x7f4d1c0acd00 2023-11-15T03:12:22.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d037fe700 1 -- 172.21.15.121:0/2449164542 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1000d770 con 0x7f4d1c0acd00 2023-11-15T03:12:22.175 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d20d07700 1 -- 172.21.15.121:0/2449164542 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c155850 con 0x7f4d1c0acd00 2023-11-15T03:12:22.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d037fe700 1 -- 172.21.15.121:0/2449164542 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10000d20 con 0x7f4d1c0acd00 2023-11-15T03:12:22.176 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.159+0000 7f4d20d07700 1 -- 172.21.15.121:0/2449164542 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c155260 con 0x7f4d1c0acd00 2023-11-15T03:12:22.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.163+0000 7f4d20d07700 1 -- 172.21.15.121:0/2449164542 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c155690 con 0x7f4d1c0acd00 2023-11-15T03:12:22.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.163+0000 7f4d037fe700 1 -- 172.21.15.121:0/2449164542 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4d100263b0 con 0x7f4d1c0acd00 2023-11-15T03:12:22.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.163+0000 7f4d037fe700 1 --2- 172.21.15.121:0/2449164542 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4d0805a510 0x7f4d0805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:22.177 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.163+0000 7f4d037fe700 1 -- 172.21.15.121:0/2449164542 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f4d10075450 con 0x7f4d1c0acd00 2023-11-15T03:12:22.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.163+0000 7f4d1b7fe700 1 --2- 172.21.15.121:0/2449164542 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4d0805a510 0x7f4d0805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:22.178 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.167+0000 7f4d1b7fe700 1 --2- 172.21.15.121:0/2449164542 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4d0805a510 0x7f4d0805c9d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f4d0c009200 tx=0x7f4d0c008040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:22.180 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.171+0000 7f4d037fe700 1 -- 172.21.15.121:0/2449164542 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4d10048f20 con 0x7f4d1c0acd00 2023-11-15T03:12:22.345 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.335+0000 7f4d20d07700 1 -- 172.21.15.121:0/2449164542 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f4d1c0c61c0 con 0x7f4d1c0acd00 2023-11-15T03:12:22.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.343+0000 7f4d037fe700 1 -- 172.21.15.121:0/2449164542 <== mon.0 v2:172.21.15.121:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4d10073e90 con 0x7f4d1c0acd00 2023-11-15T03:12:22.352 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.343+0000 7f4d037fe700 1 -- 172.21.15.121:0/2449164542 <== mon.0 v2:172.21.15.121:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) v1 ==== 123+0+0 (secure 0 0 0) 0x7f4d10022020 con 0x7f4d1c0acd00 2023-11-15T03:12:22.356 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.347+0000 7f4d20d07700 1 -- 172.21.15.121:0/2449164542 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4d0805a510 msgr2=0x7f4d0805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:22.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.347+0000 7f4d20d07700 1 --2- 172.21.15.121:0/2449164542 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4d0805a510 0x7f4d0805c9d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f4d0c009200 tx=0x7f4d0c008040).stop 2023-11-15T03:12:22.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.347+0000 7f4d20d07700 1 -- 172.21.15.121:0/2449164542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d1c0acd00 msgr2=0x7f4d1c153490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:22.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.347+0000 7f4d20d07700 1 --2- 172.21.15.121:0/2449164542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d1c0acd00 0x7f4d1c153490 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 rx=0x7f4d1c064f20 tx=0x7f4d10016000).stop 2023-11-15T03:12:22.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.347+0000 7f4d20d07700 1 -- 172.21.15.121:0/2449164542 shutdown_connections 2023-11-15T03:12:22.357 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.347+0000 7f4d20d07700 1 --2- 172.21.15.121:0/2449164542 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f4d0805a510 0x7f4d0805c9d0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:22.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.347+0000 7f4d20d07700 1 --2- 172.21.15.121:0/2449164542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c0c5c70 0x7f4d1c153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:22.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.347+0000 7f4d20d07700 1 --2- 172.21.15.121:0/2449164542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d1c0acd00 0x7f4d1c153490 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:22.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.347+0000 7f4d20d07700 1 --2- 172.21.15.121:0/2449164542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c066330 0x7f4d1c0a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:22.358 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.347+0000 7f4d20d07700 1 -- 172.21.15.121:0/2449164542 >> 172.21.15.121:0/2449164542 conn(0x7f4d1c063fc0 msgr2=0x7f4d1c1529b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:12:22.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.347+0000 7f4d20d07700 1 -- 172.21.15.121:0/2449164542 shutdown_connections 2023-11-15T03:12:22.359 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.347+0000 7f4d20d07700 1 -- 172.21.15.121:0/2449164542 wait complete. 2023-11-15T03:12:22.377 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-15T03:12:22.377 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-15T03:12:22.529 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.519+0000 7f16422e4700 1 Processor -- start 2023-11-15T03:12:22.530 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.519+0000 7f16422e4700 1 -- start start 2023-11-15T03:12:22.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.523+0000 7f16422e4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f163c0cabd0 0x7f163c0cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:22.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.523+0000 7f16422e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c0acc90 0x7f163c066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:22.532 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.523+0000 7f16422e4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f163c0ad650 con 0x7f163c0ad0f0 2023-11-15T03:12:22.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.523+0000 7f16422e4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f163c065fc0 con 0x7f163c0cabd0 2023-11-15T03:12:22.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.523+0000 7f16422e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163c0662a0 con 0x7f163c0acc90 2023-11-15T03:12:22.533 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.523+0000 7f1640ae1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c0acc90 0x7f163c066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:22.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.523+0000 7f16412e2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f163c0cabd0 0x7f163c0cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:22.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.523+0000 7f1640ae1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c0acc90 0x7f163c066d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.121:42620/0 (socket says 172.21.15.121:42620) 2023-11-15T03:12:22.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.523+0000 7f1641ae3700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f163c0ad0f0 0x7f163c0ae190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.121:51828/0 (socket says 172.21.15.121:51828) 2023-11-15T03:12:22.534 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.523+0000 7f1640ae1700 1 -- 172.21.15.121:0/1288555503 learned_addr learned my addr 172.21.15.121:0/1288555503 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:12:22.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.523+0000 7f16412e2700 1 -- 172.21.15.121:0/1288555503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c0acc90 msgr2=0x7f163c066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-15T03:12:22.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.523+0000 7f16412e2700 1 --2- 172.21.15.121:0/1288555503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c0acc90 0x7f163c066d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:22.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.523+0000 7f16412e2700 1 -- 172.21.15.121:0/1288555503 >> v1:172.21.15.190:6789/0 conn(0x7f163c0ad0f0 legacy=0x7f163c0ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:22.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.523+0000 7f16412e2700 1 -- 172.21.15.121:0/1288555503 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c066580 con 0x7f163c0cabd0 2023-11-15T03:12:22.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.523+0000 7f16412e2700 1 --2- 172.21.15.121:0/1288555503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f163c0cabd0 0x7f163c0cafb0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f1638009760 tx=0x7f1638009060).ready entity=mon.1 client_cookie=9b5ed704f174215c server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:22.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.523+0000 7f1633fff700 1 -- 172.21.15.121:0/1288555503 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163800f510 con 0x7f163c0cabd0 2023-11-15T03:12:22.536 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.527+0000 7f1633fff700 1 -- 172.21.15.121:0/1288555503 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f163800f670 con 0x7f163c0cabd0 2023-11-15T03:12:22.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.527+0000 7f1633fff700 1 -- 172.21.15.121:0/1288555503 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163800fa10 con 0x7f163c0cabd0 2023-11-15T03:12:22.537 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.527+0000 7f16422e4700 1 -- 172.21.15.121:0/1288555503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f163c0cabd0 msgr2=0x7f163c0cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:22.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.527+0000 7f16422e4700 1 --2- 172.21.15.121:0/1288555503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f163c0cabd0 0x7f163c0cafb0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f1638009760 tx=0x7f1638009060).stop 2023-11-15T03:12:22.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.527+0000 7f16422e4700 1 -- 172.21.15.121:0/1288555503 shutdown_connections 2023-11-15T03:12:22.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.527+0000 7f16422e4700 1 --2- 172.21.15.121:0/1288555503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c0acc90 0x7f163c066d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:22.538 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.527+0000 7f16422e4700 1 --2- 172.21.15.121:0/1288555503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f163c0cabd0 0x7f163c0cafb0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:22.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.527+0000 7f16422e4700 1 -- 172.21.15.121:0/1288555503 >> 172.21.15.121:0/1288555503 conn(0x7f163c062190 msgr2=0x7f163c0625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:12:22.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.527+0000 7f16422e4700 1 -- 172.21.15.121:0/1288555503 shutdown_connections 2023-11-15T03:12:22.539 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.527+0000 7f16422e4700 1 -- 172.21.15.121:0/1288555503 wait complete. 2023-11-15T03:12:22.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.527+0000 7f16422e4700 1 Processor -- start 2023-11-15T03:12:22.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.527+0000 7f16422e4700 1 -- start start 2023-11-15T03:12:22.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f16422e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c0acc90 0x7f163c1648e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:22.540 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f16422e4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f163c0ad0f0 0x7f163c156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:22.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f16422e4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f163c0cabd0 0x7f163c0ac140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:22.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f16422e4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c164dd0 con 0x7f163c0cabd0 2023-11-15T03:12:22.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f16422e4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c157010 con 0x7f163c0ad0f0 2023-11-15T03:12:22.541 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f16422e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163c157200 con 0x7f163c0acc90 2023-11-15T03:12:22.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f1641ae3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f163c0cabd0 0x7f163c0ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:22.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f1641ae3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f163c0cabd0 0x7f163c0ac140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39080/0 (socket says 172.21.15.121:39080) 2023-11-15T03:12:22.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f1641ae3700 1 -- 172.21.15.121:0/3986132245 learned_addr learned my addr 172.21.15.121:0/3986132245 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-11-15T03:12:22.543 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f1640ae1700 1 --2- 172.21.15.121:0/3986132245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f163c0ad0f0 0x7f163c156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:22.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f16412e2700 1 --2- 172.21.15.121:0/3986132245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c0acc90 0x7f163c1648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:22.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f1641ae3700 1 -- 172.21.15.121:0/3986132245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c0acc90 msgr2=0x7f163c1648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:22.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f1641ae3700 1 --2- 172.21.15.121:0/3986132245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c0acc90 0x7f163c1648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:22.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f1641ae3700 1 -- 172.21.15.121:0/3986132245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f163c0ad0f0 msgr2=0x7f163c156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:22.544 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f1641ae3700 1 --2- 172.21.15.121:0/3986132245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f163c0ad0f0 0x7f163c156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:22.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f1641ae3700 1 -- 172.21.15.121:0/3986132245 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16380092a0 con 0x7f163c0cabd0 2023-11-15T03:12:22.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f1640ae1700 1 --2- 172.21.15.121:0/3986132245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f163c0ad0f0 0x7f163c156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-15T03:12:22.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f1641ae3700 1 --2- 172.21.15.121:0/3986132245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f163c0cabd0 0x7f163c0ac140 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 rx=0x7f163400c7c0 tx=0x7f1634001560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:22.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f16317fa700 1 -- 172.21.15.121:0/3986132245 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1634005c70 con 0x7f163c0cabd0 2023-11-15T03:12:22.545 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f16317fa700 1 -- 172.21.15.121:0/3986132245 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1634005dd0 con 0x7f163c0cabd0 2023-11-15T03:12:22.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f16422e4700 1 -- 172.21.15.121:0/3986132245 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163c0c17c0 con 0x7f163c0cabd0 2023-11-15T03:12:22.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f16317fa700 1 -- 172.21.15.121:0/3986132245 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1634007320 con 0x7f163c0cabd0 2023-11-15T03:12:22.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.531+0000 7f16422e4700 1 -- 172.21.15.121:0/3986132245 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163c0c1d80 con 0x7f163c0cabd0 2023-11-15T03:12:22.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.535+0000 7f16422e4700 1 -- 172.21.15.121:0/3986132245 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163c06bcf0 con 0x7f163c0cabd0 2023-11-15T03:12:22.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.535+0000 7f16317fa700 1 -- 172.21.15.121:0/3986132245 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f16340076d0 con 0x7f163c0cabd0 2023-11-15T03:12:22.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.535+0000 7f16317fa700 1 --2- 172.21.15.121:0/3986132245 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f162005a540 0x7f162005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-15T03:12:22.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.535+0000 7f16317fa700 1 -- 172.21.15.121:0/3986132245 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5441+0+0 (secure 0 0 0) 0x7f16340745e0 con 0x7f163c0cabd0 2023-11-15T03:12:22.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.535+0000 7f16412e2700 1 --2- 172.21.15.121:0/3986132245 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f162005a540 0x7f162005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-15T03:12:22.548 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.539+0000 7f16412e2700 1 --2- 172.21.15.121:0/3986132245 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f162005a540 0x7f162005ca00 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f1638009fd0 tx=0x7f1638009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-15T03:12:22.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.543+0000 7f16317fa700 1 -- 172.21.15.121:0/3986132245 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1634048020 con 0x7f163c0cabd0 2023-11-15T03:12:22.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.735+0000 7f16422e4700 1 -- 172.21.15.121:0/3986132245 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f163c068a70 con 0x7f163c0cabd0 2023-11-15T03:12:22.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.735+0000 7f16317fa700 1 -- 172.21.15.121:0/3986132245 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 358 v358) v1 ==== 95+0+3544 (secure 0 0 0) 0x7f163404b640 con 0x7f163c0cabd0 2023-11-15T03:12:22.747 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-11-15T03:12:22.747 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":358,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/755048854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":755048854},{"type":"v1","addr":"172.21.15.190:6834","nonce":755048854}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2360242049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2360242049},{"type":"v1","addr":"172.21.15.190:6835","nonce":2360242049}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":10713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2780153683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2780153683},{"type":"v1","addr":"172.21.15.121:6835","nonce":2780153683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":345},{"gid":11196,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/367539519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":367539519},{"type":"v1","addr":"172.21.15.121:6837","nonce":367539519}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":358}],"filesystems":[]} 2023-11-15T03:12:22.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.739+0000 7f16422e4700 1 -- 172.21.15.121:0/3986132245 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f162005a540 msgr2=0x7f162005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:22.749 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.739+0000 7f16422e4700 1 --2- 172.21.15.121:0/3986132245 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f162005a540 0x7f162005ca00 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f1638009fd0 tx=0x7f1638009060).stop 2023-11-15T03:12:22.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.739+0000 7f16422e4700 1 -- 172.21.15.121:0/3986132245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f163c0cabd0 msgr2=0x7f163c0ac140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-15T03:12:22.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.739+0000 7f16422e4700 1 --2- 172.21.15.121:0/3986132245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f163c0cabd0 0x7f163c0ac140 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 rx=0x7f163400c7c0 tx=0x7f1634001560).stop 2023-11-15T03:12:22.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.739+0000 7f16422e4700 1 -- 172.21.15.121:0/3986132245 shutdown_connections 2023-11-15T03:12:22.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.739+0000 7f16422e4700 1 --2- 172.21.15.121:0/3986132245 >> [v2:172.21.15.121:6832/33042,v1:172.21.15.121:6833/33042] conn(0x7f162005a540 0x7f162005ca00 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:22.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.739+0000 7f16422e4700 1 --2- 172.21.15.121:0/3986132245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f163c0cabd0 0x7f163c0ac140 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:22.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.739+0000 7f16422e4700 1 --2- 172.21.15.121:0/3986132245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f163c0ad0f0 0x7f163c156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:22.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.739+0000 7f16422e4700 1 --2- 172.21.15.121:0/3986132245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f163c0acc90 0x7f163c1648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-15T03:12:22.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.739+0000 7f16422e4700 1 -- 172.21.15.121:0/3986132245 >> 172.21.15.121:0/3986132245 conn(0x7f163c062190 msgr2=0x7f163c0ae220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-15T03:12:22.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.739+0000 7f16422e4700 1 -- 172.21.15.121:0/3986132245 shutdown_connections 2023-11-15T03:12:22.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-11-15T03:12:22.739+0000 7f16422e4700 1 -- 172.21.15.121:0/3986132245 wait complete. 2023-11-15T03:12:22.752 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 358 2023-11-15T03:12:22.768 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-15T03:12:22.769 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-15T03:12:22.769 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-15T03:12:22.769 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:12:22.787 INFO:tasks.ceph.mds.a:Stopped 2023-11-15T03:12:22.788 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-15T03:12:22.788 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:12:22.856 INFO:tasks.ceph.mds.c:Stopped 2023-11-15T03:12:22.856 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-15T03:12:22.857 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:12:22.880 INFO:tasks.ceph.mds.b:Stopped 2023-11-15T03:12:22.881 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-15T03:12:22.881 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:12:22.944 INFO:tasks.ceph.mds.d:Stopped 2023-11-15T03:12:22.945 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-15T03:12:22.945 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-15T03:12:22.946 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:12:29.048 INFO:tasks.ceph.osd.0:Stopped 2023-11-15T03:12:29.048 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-15T03:12:29.049 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:12:29.134 INFO:tasks.ceph.osd.1:Stopped 2023-11-15T03:12:29.135 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-15T03:12:29.135 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:12:30.182 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2023-11-15T03:12:35.237 INFO:tasks.ceph.osd.2:Stopped 2023-11-15T03:12:35.238 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-15T03:12:35.238 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:12:36.191 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-11-15T03:12:36.192 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2023-11-15T03:12:39.311 INFO:tasks.ceph.mgr.y.smithi121.stderr:2023-11-15T03:12:39.298+0000 7f4a8ad91700 -1 calc_pg_upmaps abort due to max <= 0 2023-11-15T03:12:41.341 INFO:tasks.ceph.osd.3:Stopped 2023-11-15T03:12:41.341 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-15T03:12:41.342 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:12:41.409 INFO:tasks.ceph.osd.4:Stopped 2023-11-15T03:12:41.409 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-15T03:12:41.410 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:12:42.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-11-15T03:12:42.102 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2023-11-15T03:12:42.851 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:12:42.859 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:12:47.512 INFO:tasks.ceph.osd.5:Stopped 2023-11-15T03:12:47.513 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-15T03:12:47.513 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:12:47.582 INFO:tasks.ceph.osd.6:Stopped 2023-11-15T03:12:47.583 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-15T03:12:47.583 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:12:47.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-11-15T03:12:47.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-11-15T03:12:53.313 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-11-15T03:12:53.313 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2023-11-15T03:12:53.685 INFO:tasks.ceph.osd.7:Stopped 2023-11-15T03:12:53.685 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-15T03:12:53.686 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-15T03:12:53.686 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:12:53.741 INFO:tasks.ceph.mgr.y:Stopped 2023-11-15T03:12:53.742 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-15T03:12:53.742 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:12:53.802 INFO:tasks.ceph.mgr.x:Stopped 2023-11-15T03:12:53.803 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-15T03:12:53.803 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-15T03:12:53.804 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:12:53.844 INFO:tasks.ceph.mon.a:Stopped 2023-11-15T03:12:53.844 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-15T03:12:53.845 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:12:53.875 INFO:tasks.ceph.mon.b:Stopped 2023-11-15T03:12:53.875 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-15T03:12:53.876 INFO:teuthology.orchestra.run:waiting for 300 2023-11-15T03:12:53.951 INFO:tasks.ceph.mon.c:Stopped 2023-11-15T03:12:53.952 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-15T03:12:53.953 DEBUG:teuthology.orchestra.run.smithi121:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'Error loading MDS rank' | egrep -v 'missing journal object' | egrep -v 'Error recovering journal' | egrep -v 'error decoding table object' | egrep -v 'failed to read JournalPointer' | egrep -v 'Corrupt directory entry' | egrep -v 'Corrupt fnode header' | egrep -v 'corrupt sessionmap header' | egrep -v 'Corrupt dentry' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_READ_ONLY | egrep -v 'force file system read-only' | egrep -v 'with standby daemon mds' | head -n 1 2023-11-15T03:12:54.016 INFO:teuthology.orchestra.run.smithi121.stdout:2023-11-15T03:00:42.050657+0000 mon.a (mon.0) 686 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2023-11-15T03:12:54.018 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-11-15T03:12:54.019 DEBUG:teuthology.orchestra.run.smithi121:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'Error loading MDS rank' | egrep -v 'missing journal object' | egrep -v 'Error recovering journal' | egrep -v 'error decoding table object' | egrep -v 'failed to read JournalPointer' | egrep -v 'Corrupt directory entry' | egrep -v 'Corrupt fnode header' | egrep -v 'corrupt sessionmap header' | egrep -v 'Corrupt dentry' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_READ_ONLY | egrep -v 'force file system read-only' | egrep -v 'with standby daemon mds' | head -n 1 2023-11-15T03:12:54.073 DEBUG:teuthology.orchestra.run.smithi121:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'Error loading MDS rank' | egrep -v 'missing journal object' | egrep -v 'Error recovering journal' | egrep -v 'error decoding table object' | egrep -v 'failed to read JournalPointer' | egrep -v 'Corrupt directory entry' | egrep -v 'Corrupt fnode header' | egrep -v 'corrupt sessionmap header' | egrep -v 'Corrupt dentry' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_READ_ONLY | egrep -v 'force file system read-only' | egrep -v 'with standby daemon mds' | head -n 1 2023-11-15T03:12:54.122 DEBUG:teuthology.orchestra.run.smithi121:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'Error loading MDS rank' | egrep -v 'missing journal object' | egrep -v 'Error recovering journal' | egrep -v 'error decoding table object' | egrep -v 'failed to read JournalPointer' | egrep -v 'Corrupt directory entry' | egrep -v 'Corrupt fnode header' | egrep -v 'corrupt sessionmap header' | egrep -v 'Corrupt dentry' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_READ_ONLY | egrep -v 'force file system read-only' | egrep -v 'with standby daemon mds' | head -n 1 2023-11-15T03:12:54.221 INFO:teuthology.orchestra.run.smithi121.stdout:2023-11-15T03:00:42.050657+0000 mon.a (mon.0) 686 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2023-11-15T03:12:54.222 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:12:54.222 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-15T03:12:54.461 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:12:54.462 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-15T03:12:54.711 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:12:54.711 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-15T03:12:54.911 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:12:54.912 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-15T03:12:55.155 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi190.front.sepia.ceph.com 2023-11-15T03:12:55.155 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-15T03:12:55.436 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi190.front.sepia.ceph.com 2023-11-15T03:12:55.437 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-15T03:12:55.691 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi190.front.sepia.ceph.com 2023-11-15T03:12:55.692 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-15T03:12:55.911 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi190.front.sepia.ceph.com 2023-11-15T03:12:55.911 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-15T03:12:56.124 INFO:tasks.ceph:Archiving mon data... 2023-11-15T03:12:56.126 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7458153/data/mon.a.tgz 2023-11-15T03:12:56.127 DEBUG:teuthology.orchestra.run.smithi121:> mktemp 2023-11-15T03:12:56.147 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.tAhmKtx2xz 2023-11-15T03:12:56.148 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.tAhmKtx2xz 2023-11-15T03:12:56.534 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0666 /tmp/tmp.tAhmKtx2xz 2023-11-15T03:12:56.629 DEBUG:teuthology.orchestra.remote:smithi121:/tmp/tmp.tAhmKtx2xz is 1MB 2023-11-15T03:12:56.722 DEBUG:teuthology.orchestra.run.smithi121:> rm -fr /tmp/tmp.tAhmKtx2xz 2023-11-15T03:12:56.741 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7458153/data/mon.b.tgz 2023-11-15T03:12:56.742 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2023-11-15T03:12:56.760 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.PsRu0jgpx8 2023-11-15T03:12:56.760 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.PsRu0jgpx8 2023-11-15T03:12:57.154 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.PsRu0jgpx8 2023-11-15T03:12:57.251 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.PsRu0jgpx8 is 1MB 2023-11-15T03:12:57.341 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.PsRu0jgpx8 2023-11-15T03:12:57.360 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7458153/data/mon.c.tgz 2023-11-15T03:12:57.361 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2023-11-15T03:12:57.419 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.4LRKjkAgJC 2023-11-15T03:12:57.420 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.4LRKjkAgJC 2023-11-15T03:12:57.801 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.4LRKjkAgJC 2023-11-15T03:12:57.894 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.4LRKjkAgJC is 1MB 2023-11-15T03:12:57.977 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.4LRKjkAgJC 2023-11-15T03:12:57.993 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-15T03:12:57.994 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-15T03:12:58.001 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-15T03:12:58.073 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-15T03:12:58.073 DEBUG:teuthology.orchestra.run.smithi121:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-15T03:12:58.080 DEBUG:teuthology.orchestra.run.smithi190:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-15T03:12:58.127 INFO:teuthology.orchestra.run.smithi121.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-15T03:12:58.171 INFO:teuthology.orchestra.run.smithi190.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-15T03:12:58.184 INFO:tasks.ceph:Archiving crash dumps... 2023-11-15T03:12:58.187 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7458153/remote/smithi121/crash 2023-11-15T03:12:58.188 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-15T03:12:58.234 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7458153/remote/smithi190/crash 2023-11-15T03:12:58.235 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-15T03:12:58.291 INFO:tasks.ceph:Shutting down logrotate 2023-11-15T03:12:58.292 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:12:58.298 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-15T03:12:58.314 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-11-15T03:12:58.401 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-15T03:12:58.435 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-15T03:12:58.466 INFO:tasks.ceph:Compressing logs... 2023-11-15T03:12:58.467 DEBUG:teuthology.orchestra.run.smithi121:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-15T03:12:58.485 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-15T03:12:58.521 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54808.log 2023-11-15T03:12:58.522 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47919.log 2023-11-15T03:12:58.523 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54609.log 2023-11-15T03:12:58.523 INFO:teuthology.orchestra.run.smithi121.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54808.log.gz 2023-11-15T03:12:58.524 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53160.log 2023-11-15T03:12:58.525 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47919.log.gz 2023-11-15T03:12:58.526 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54609.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54609.log.gz 2023-11-15T03:12:58.526 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56210.log 2023-11-15T03:12:58.528 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53160.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53160.log.gz 2023-11-15T03:12:58.529 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43095.log 2023-11-15T03:12:58.530 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46118.log 2023-11-15T03:12:58.531 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56210.log.gz 2023-11-15T03:12:58.532 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57666.log 2023-11-15T03:12:58.533 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43095.log.gz 2023-11-15T03:12:58.533 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46118.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46118.log.gz 2023-11-15T03:12:58.534 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49363.log 2023-11-15T03:12:58.535 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57666.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57666.log.gz 2023-11-15T03:12:58.535 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48066.log 2023-11-15T03:12:58.536 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49363.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49363.log.gz 2023-11-15T03:12:58.536 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52043.log 2023-11-15T03:12:58.537 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56576.log 2023-11-15T03:12:58.538 INFO:teuthology.orchestra.run.smithi121.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48066.log.gz 2023-11-15T03:12:58.538 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52043.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52043.log.gz 2023-11-15T03:12:58.539 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51646.log 2023-11-15T03:12:58.541 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56576.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56576.log.gz 2023-11-15T03:12:58.542 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41918.log 2023-11-15T03:12:58.544 INFO:teuthology.orchestra.run.smithi121.stderr:gzip /var/log/ceph/ceph-client.admin.51646.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.59471.log 2023-11-15T03:12:58.544 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51646.log.gz 2023-11-15T03:12:58.545 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41918.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41918.log.gz 2023-11-15T03:12:58.546 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39885.log 2023-11-15T03:12:58.548 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51352.log 2023-11-15T03:12:58.548 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59471.log.gz 2023-11-15T03:12:58.550 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62744.log 2023-11-15T03:12:58.550 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39885.log.gz 2023-11-15T03:12:58.551 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-15T03:12:58.553 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51352.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51352.log.gz 2023-11-15T03:12:58.553 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58832.log 2023-11-15T03:12:58.553 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62744.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62744.log.gz 2023-11-15T03:12:58.553 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37542.log 2023-11-15T03:12:58.558 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-15T03:12:58.558 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36739.log 2023-11-15T03:12:58.558 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-15T03:12:58.559 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-15T03:12:58.560 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58832.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58832.log.gz 2023-11-15T03:12:58.560 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47865.log 2023-11-15T03:12:58.560 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39297.log 2023-11-15T03:12:58.560 INFO:teuthology.orchestra.run.smithi121.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37542.log.gz 2023-11-15T03:12:58.561 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47865.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47865.log.gz 2023-11-15T03:12:58.561 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50998.log 2023-11-15T03:12:58.561 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53925.log 2023-11-15T03:12:58.562 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36739.log: /var/log/ceph/ceph-osd.5.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36739.log.gz 2023-11-15T03:12:58.562 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33280.log 2023-11-15T03:12:58.562 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-15T03:12:58.563 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39297.log: /var/log/ceph/ceph-client.admin.50998.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39297.log.gz 2023-11-15T03:12:58.564 INFO:teuthology.orchestra.run.smithi121.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50998.log.gz 2023-11-15T03:12:58.564 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49719.log 2023-11-15T03:12:58.564 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50734.log 2023-11-15T03:12:58.564 INFO:teuthology.orchestra.run.smithi121.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53925.log.gz 2023-11-15T03:12:58.565 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49719.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49719.log.gz 2023-11-15T03:12:58.566 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60785.log 2023-11-15T03:12:58.566 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61112.log 2023-11-15T03:12:58.567 INFO:teuthology.orchestra.run.smithi121.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50734.log.gz 2023-11-15T03:12:58.567 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33280.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-15T03:12:58.567 INFO:teuthology.orchestra.run.smithi190.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33280.log.gz 2023-11-15T03:12:58.568 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.36950.log 2023-11-15T03:12:58.569 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50205.log 2023-11-15T03:12:58.569 INFO:teuthology.orchestra.run.smithi121.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60785.log.gz 2023-11-15T03:12:58.569 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61112.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61112.log.gz 2023-11-15T03:12:58.570 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41308.log 2023-11-15T03:12:58.574 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.3.36950.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.36950.log.gz 2023-11-15T03:12:58.575 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33121.log 2023-11-15T03:12:58.575 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33121.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.33121.log.gz 2023-11-15T03:12:58.575 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33252.log 2023-11-15T03:12:58.577 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50205.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50205.log.gz 2023-11-15T03:12:58.578 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49231.log 2023-11-15T03:12:58.578 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41308.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41308.log.gz 2023-11-15T03:12:58.578 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56875.log 2023-11-15T03:12:58.578 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49231.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49231.log.gz 2023-11-15T03:12:58.579 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55818.log 2023-11-15T03:12:58.579 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46496.log 2023-11-15T03:12:58.579 INFO:teuthology.orchestra.run.smithi121.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56875.log.gz 2023-11-15T03:12:58.579 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55818.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55818.log.gz 2023-11-15T03:12:58.579 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49986.log 2023-11-15T03:12:58.580 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46496.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46496.log.gz 2023-11-15T03:12:58.580 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48857.log 2023-11-15T03:12:58.580 INFO:teuthology.orchestra.run.smithi190.stderr: 94.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-15T03:12:58.581 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-15T03:12:58.581 INFO:teuthology.orchestra.run.smithi190.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-15T03:12:58.581 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33307.log 2023-11-15T03:12:58.581 INFO:teuthology.orchestra.run.smithi190.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33252.log.gz 2023-11-15T03:12:58.582 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.33307.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33307.log.gz 2023-11-15T03:12:58.582 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36868.log 2023-11-15T03:12:58.584 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40331.log 2023-11-15T03:12:58.584 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49986.log.gz 2023-11-15T03:12:58.585 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60564.log 2023-11-15T03:12:58.585 INFO:teuthology.orchestra.run.smithi121.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48857.log.gz 2023-11-15T03:12:58.585 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40331.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40331.log.gz 2023-11-15T03:12:58.585 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60352.log 2023-11-15T03:12:58.585 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60564.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60564.log.gz 2023-11-15T03:12:58.586 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53455.log 2023-11-15T03:12:58.586 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60352.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60352.log.gz 2023-11-15T03:12:58.586 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37191.log 2023-11-15T03:12:58.588 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-15T03:12:58.588 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36868.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36868.log.gz 2023-11-15T03:12:58.589 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mds.b.log: 92.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-15T03:12:58.589 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33224.log 2023-11-15T03:12:58.590 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53455.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53455.log.gz 2023-11-15T03:12:58.590 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44436.log 2023-11-15T03:12:58.590 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37191.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37191.log.gz 2023-11-15T03:12:58.591 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62933.log 2023-11-15T03:12:58.591 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40406.log 2023-11-15T03:12:58.591 INFO:teuthology.orchestra.run.smithi121.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44436.log.gz 2023-11-15T03:12:58.591 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62933.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62933.log.gz 2023-11-15T03:12:58.591 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40900.log 2023-11-15T03:12:58.593 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40406.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40406.log.gz 2023-11-15T03:12:58.593 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50361.log 2023-11-15T03:12:58.594 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58529.log 2023-11-15T03:12:58.594 INFO:teuthology.orchestra.run.smithi121.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.40900.log.gz 2023-11-15T03:12:58.594 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33173.log 2023-11-15T03:12:58.594 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33224.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33224.log.gz 2023-11-15T03:12:58.595 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33173.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33173.log.gz 2023-11-15T03:12:58.595 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33146.log 2023-11-15T03:12:58.596 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33198.log 2023-11-15T03:12:58.597 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33146.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33146.log.gz 2023-11-15T03:12:58.597 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50361.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50361.log.gz 2023-11-15T03:12:58.598 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56239.log 2023-11-15T03:12:58.598 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53742.log 2023-11-15T03:12:58.598 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58529.log.gz 2023-11-15T03:12:58.598 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56239.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56239.log.gz 2023-11-15T03:12:58.599 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57802.log 2023-11-15T03:12:58.600 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33198.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33198.log.gz 2023-11-15T03:12:58.600 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-15T03:12:58.600 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36855.log 2023-11-15T03:12:58.602 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47679.log 2023-11-15T03:12:58.602 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57802.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57802.log.gz 2023-11-15T03:12:58.603 INFO:teuthology.orchestra.run.smithi121.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.53742.log.gz 2023-11-15T03:12:58.603 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58289.log 2023-11-15T03:12:58.603 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.1.36855.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.36855.log.gz 2023-11-15T03:12:58.604 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-15T03:12:58.604 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51987.log 2023-11-15T03:12:58.605 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47679.log.gz 2023-11-15T03:12:58.605 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53077.log 2023-11-15T03:12:58.606 INFO:teuthology.orchestra.run.smithi190.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-15T03:12:58.606 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-15T03:12:58.606 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58289.log.gz 2023-11-15T03:12:58.607 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51987.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51987.log.gz 2023-11-15T03:12:58.607 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39721.log 2023-11-15T03:12:58.608 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39351.log 2023-11-15T03:12:58.608 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53077.log.gz 2023-11-15T03:12:58.609 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.7.log: 93.1% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-15T03:12:58.610 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39721.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-15T03:12:58.610 INFO:teuthology.orchestra.run.smithi121.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39721.log.gz 2023-11-15T03:12:58.611 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39351.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39351.log.gz 2023-11-15T03:12:58.612 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40837.log 2023-11-15T03:12:58.613 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50697.log 2023-11-15T03:12:58.614 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61582.log 2023-11-15T03:12:58.615 INFO:teuthology.orchestra.run.smithi121.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.40837.log.gz 2023-11-15T03:12:58.616 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47598.log 2023-11-15T03:12:58.616 INFO:teuthology.orchestra.run.smithi121.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50697.log.gz 2023-11-15T03:12:58.618 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53501.log 2023-11-15T03:12:58.618 INFO:teuthology.orchestra.run.smithi121.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61582.log.gz 2023-11-15T03:12:58.619 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47598.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47598.log.gz 2023-11-15T03:12:58.620 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63114.log 2023-11-15T03:12:58.621 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53501.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.53501.log.gz 2023-11-15T03:12:58.622 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53479.log 2023-11-15T03:12:58.623 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43303.log 2023-11-15T03:12:58.623 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63114.log.gz 2023-11-15T03:12:58.624 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41416.log 2023-11-15T03:12:58.626 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42662.log 2023-11-15T03:12:58.627 INFO:teuthology.orchestra.run.smithi121.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43303.log.gz 2023-11-15T03:12:58.627 INFO:teuthology.orchestra.run.smithi121.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.53479.log.gz 2023-11-15T03:12:58.628 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41416.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41416.log.gz 2023-11-15T03:12:58.628 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57428.log 2023-11-15T03:12:58.629 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42662.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42662.log.gz 2023-11-15T03:12:58.629 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62549.log 2023-11-15T03:12:58.630 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57428.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57428.log.gz 2023-11-15T03:12:58.630 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41678.log 2023-11-15T03:12:58.631 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62549.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62549.log.gz 2023-11-15T03:12:58.631 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50626.log 2023-11-15T03:12:58.632 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55390.log 2023-11-15T03:12:58.633 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41678.log.gz 2023-11-15T03:12:58.633 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50626.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50626.log.gz 2023-11-15T03:12:58.634 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52878.log 2023-11-15T03:12:58.637 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55100.log 2023-11-15T03:12:58.638 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55390.log.gz 2023-11-15T03:12:58.638 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52878.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52878.log.gz 2023-11-15T03:12:58.639 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42528.log 2023-11-15T03:12:58.639 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58051.log 2023-11-15T03:12:58.640 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42528.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55100.log.gz 2023-11-15T03:12:58.641 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42528.log.gz 2023-11-15T03:12:58.641 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53691.log 2023-11-15T03:12:58.642 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57931.log 2023-11-15T03:12:58.642 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53691.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.58051.log.gz 2023-11-15T03:12:58.643 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53691.log.gz 2023-11-15T03:12:58.643 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36727.log 2023-11-15T03:12:58.644 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46361.log 2023-11-15T03:12:58.645 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36727.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.57931.log.gz 2023-11-15T03:12:58.645 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36727.log.gz 2023-11-15T03:12:58.646 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36628.log 2023-11-15T03:12:58.647 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43068.log 2023-11-15T03:12:58.648 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46361.log.gz 2023-11-15T03:12:58.648 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36628.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36628.log.gz 2023-11-15T03:12:58.649 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36975.log 2023-11-15T03:12:58.651 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56020.log 2023-11-15T03:12:58.651 INFO:teuthology.orchestra.run.smithi121.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43068.log.gz 2023-11-15T03:12:58.652 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36975.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36975.log.gz 2023-11-15T03:12:58.652 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50914.log 2023-11-15T03:12:58.653 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37299.log 2023-11-15T03:12:58.654 INFO:teuthology.orchestra.run.smithi121.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.56020.log.gz 2023-11-15T03:12:58.655 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38797.log 2023-11-15T03:12:58.656 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37299.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.50914.log.gz 2023-11-15T03:12:58.656 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46973.log 2023-11-15T03:12:58.657 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37299.log.gz 2023-11-15T03:12:58.657 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38797.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38797.log.gz 2023-11-15T03:12:58.658 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50144.log 2023-11-15T03:12:58.659 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46973.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46973.log.gz 2023-11-15T03:12:58.659 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54052.log 2023-11-15T03:12:58.660 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51239.log 2023-11-15T03:12:58.661 INFO:teuthology.orchestra.run.smithi121.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50144.log.gz 2023-11-15T03:12:58.663 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48813.log 2023-11-15T03:12:58.663 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54052.log.gz 2023-11-15T03:12:58.664 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38554.log 2023-11-15T03:12:58.665 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48813.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48813.log.gz 2023-11-15T03:12:58.666 INFO:teuthology.orchestra.run.smithi121.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.51239.log.gz 2023-11-15T03:12:58.666 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45259.log 2023-11-15T03:12:58.667 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38554.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38554.log.gz 2023-11-15T03:12:58.668 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54762.log 2023-11-15T03:12:58.668 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55254.log 2023-11-15T03:12:58.669 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54762.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45259.log.gz 2023-11-15T03:12:58.670 INFO:teuthology.orchestra.run.smithi121.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54762.log.gz 2023-11-15T03:12:58.670 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57885.log 2023-11-15T03:12:58.671 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41999.log 2023-11-15T03:12:58.671 INFO:teuthology.orchestra.run.smithi121.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55254.log.gz 2023-11-15T03:12:58.672 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57885.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57885.log.gz 2023-11-15T03:12:58.673 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54191.log 2023-11-15T03:12:58.675 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41999.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41999.log.gz 2023-11-15T03:12:58.675 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57909.log 2023-11-15T03:12:58.675 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61553.log 2023-11-15T03:12:58.676 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54191.log.gz 2023-11-15T03:12:58.677 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53954.log 2023-11-15T03:12:58.677 INFO:teuthology.orchestra.run.smithi121.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.57909.log.gz 2023-11-15T03:12:58.679 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61553.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61553.log.gz 2023-11-15T03:12:58.680 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58361.log 2023-11-15T03:12:58.680 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59876.log 2023-11-15T03:12:58.681 INFO:teuthology.orchestra.run.smithi121.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53954.log.gz 2023-11-15T03:12:58.681 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58361.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.58361.log.gz 2023-11-15T03:12:58.681 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58627.log 2023-11-15T03:12:58.682 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59876.log: gzip -5 88.9%--verbose -- replaced with /var/log/ceph/ceph-client.admin.59876.log.gz-- 2023-11-15T03:12:58.682 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50305.log 2023-11-15T03:12:58.683 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55363.log 2023-11-15T03:12:58.684 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58627.log.gz 2023-11-15T03:12:58.684 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50305.log: gzip 89.1%-5 -- replaced with /var/log/ceph/ceph-client.admin.50305.log.gz--verbose 2023-11-15T03:12:58.685 INFO:teuthology.orchestra.run.smithi121.stderr:-- /var/log/ceph/ceph-client.admin.63325.log 2023-11-15T03:12:58.686 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55363.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.55363.log.gz 2023-11-15T03:12:58.686 INFO:teuthology.orchestra.run.smithi121.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.53343.log 2023-11-15T03:12:58.688 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63325.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63325.log.gz 2023-11-15T03:12:58.688 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50122.log 2023-11-15T03:12:58.689 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53343.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53343.log.gz 2023-11-15T03:12:58.690 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45367.log 2023-11-15T03:12:58.691 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58556.log 2023-11-15T03:12:58.692 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39519.log 2023-11-15T03:12:58.692 INFO:teuthology.orchestra.run.smithi121.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45367.log.gz 2023-11-15T03:12:58.693 INFO:teuthology.orchestra.run.smithi121.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.50122.log.gz 2023-11-15T03:12:58.694 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40671.log 2023-11-15T03:12:58.694 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58556.log.gz 2023-11-15T03:12:58.695 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39519.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39519.log.gz 2023-11-15T03:12:58.695 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42958.log 2023-11-15T03:12:58.696 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60169.log 2023-11-15T03:12:58.696 INFO:teuthology.orchestra.run.smithi121.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40671.log.gz 2023-11-15T03:12:58.697 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42958.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42958.log.gz 2023-11-15T03:12:58.697 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62184.log 2023-11-15T03:12:58.699 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60381.log 2023-11-15T03:12:58.701 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58859.log 2023-11-15T03:12:58.701 INFO:teuthology.orchestra.run.smithi121.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62184.log.gz 2023-11-15T03:12:58.701 INFO:teuthology.orchestra.run.smithi121.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.60169.log.gz 2023-11-15T03:12:58.702 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60381.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60381.log.gz 2023-11-15T03:12:58.702 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40304.log 2023-11-15T03:12:58.703 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37002.log 2023-11-15T03:12:58.703 INFO:teuthology.orchestra.run.smithi121.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.58859.log.gz 2023-11-15T03:12:58.704 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61499.log 2023-11-15T03:12:58.705 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40304.log.gz 2023-11-15T03:12:58.706 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37002.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-15T03:12:58.706 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37002.log.gz 2023-11-15T03:12:58.707 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61499.log: gzip 88.6%-5 -- replaced with /var/log/ceph/ceph-client.admin.61499.log.gz--verbose 2023-11-15T03:12:58.707 INFO:teuthology.orchestra.run.smithi121.stderr:-- /var/log/ceph/ceph-client.admin.52738.log 2023-11-15T03:12:58.708 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43384.log 2023-11-15T03:12:58.709 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52738.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52738.log.gz 2023-11-15T03:12:58.709 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43041.log 2023-11-15T03:12:58.711 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51056.log 2023-11-15T03:12:58.711 INFO:teuthology.orchestra.run.smithi121.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43384.log.gz 2023-11-15T03:12:58.712 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43041.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43041.log.gz 2023-11-15T03:12:58.712 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50599.log 2023-11-15T03:12:58.714 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59923.log 2023-11-15T03:12:58.714 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50599.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.51056.log.gz 2023-11-15T03:12:58.715 INFO:teuthology.orchestra.run.smithi121.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50599.log.gz 2023-11-15T03:12:58.715 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55207.log 2023-11-15T03:12:58.717 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59923.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59923.log.gz 2023-11-15T03:12:58.717 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44735.log 2023-11-15T03:12:58.718 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49765.log 2023-11-15T03:12:58.719 INFO:teuthology.orchestra.run.smithi121.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.55207.log.gz 2023-11-15T03:12:58.721 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62960.log 2023-11-15T03:12:58.721 INFO:teuthology.orchestra.run.smithi121.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.44735.log.gz 2023-11-15T03:12:58.722 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.49765.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49765.log.gz 2023-11-15T03:12:58.722 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40250.log 2023-11-15T03:12:58.723 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62960.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62960.log.gz 2023-11-15T03:12:58.723 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41227.log 2023-11-15T03:12:58.724 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52014.log 2023-11-15T03:12:58.724 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40250.log: /var/log/ceph/ceph-client.admin.41227.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40250.log.gz 2023-11-15T03:12:58.725 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41227.log.gz 2023-11-15T03:12:58.725 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50251.log 2023-11-15T03:12:58.727 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51493.log 2023-11-15T03:12:58.727 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52014.log.gz 2023-11-15T03:12:58.728 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50251.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50251.log.gz 2023-11-15T03:12:58.728 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49187.log 2023-11-15T03:12:58.729 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46226.log 2023-11-15T03:12:58.729 INFO:teuthology.orchestra.run.smithi121.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51493.log.gz 2023-11-15T03:12:58.730 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49187.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49187.log.gz 2023-11-15T03:12:58.731 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57163.log 2023-11-15T03:12:58.732 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53104.log 2023-11-15T03:12:58.732 INFO:teuthology.orchestra.run.smithi121.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46226.log.gz 2023-11-15T03:12:58.733 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57163.log: gzip 89.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.57163.log.gz--verbose 2023-11-15T03:12:58.733 INFO:teuthology.orchestra.run.smithi121.stderr:-- /var/log/ceph/ceph-client.admin.53718.log 2023-11-15T03:12:58.734 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55127.log 2023-11-15T03:12:58.734 INFO:teuthology.orchestra.run.smithi121.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53104.log.gz 2023-11-15T03:12:58.735 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49784.log 2023-11-15T03:12:58.735 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53718.log.gz 2023-11-15T03:12:58.737 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42867.log 2023-11-15T03:12:58.737 INFO:teuthology.orchestra.run.smithi121.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55127.log.gz 2023-11-15T03:12:58.738 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42477.log 2023-11-15T03:12:58.738 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49784.log.gz 2023-11-15T03:12:58.739 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42867.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42867.log.gz 2023-11-15T03:12:58.739 INFO:teuthology.orchestra.run.smithi121.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.45656.log 2023-11-15T03:12:58.740 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45475.log 2023-11-15T03:12:58.741 INFO:teuthology.orchestra.run.smithi121.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.42477.log.gz 2023-11-15T03:12:58.742 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45656.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45656.log.gz 2023-11-15T03:12:58.742 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55599.log 2023-11-15T03:12:58.744 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52516.log 2023-11-15T03:12:58.745 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45475.log.gz 2023-11-15T03:12:58.745 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61204.log 2023-11-15T03:12:58.745 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55599.log.gz 2023-11-15T03:12:58.747 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52516.log: gzip -5 --verbose 87.5%-- /var/log/ceph/ceph-client.admin.40615.log -- replaced with /var/log/ceph/ceph-client.admin.52516.log.gz 2023-11-15T03:12:58.747 INFO:teuthology.orchestra.run.smithi121.stderr: 2023-11-15T03:12:58.747 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38163.log 2023-11-15T03:12:58.748 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61204.log.gz 2023-11-15T03:12:58.749 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37326.log 2023-11-15T03:12:58.749 INFO:teuthology.orchestra.run.smithi121.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40615.log.gz 2023-11-15T03:12:58.749 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38163.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38163.log.gz 2023-11-15T03:12:58.750 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41651.log 2023-11-15T03:12:58.751 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50409.log 2023-11-15T03:12:58.751 INFO:teuthology.orchestra.run.smithi121.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37326.log.gz 2023-11-15T03:12:58.751 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41651.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41651.log.gz 2023-11-15T03:12:58.752 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47225.log 2023-11-15T03:12:58.752 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47360.log 2023-11-15T03:12:58.753 INFO:teuthology.orchestra.run.smithi121.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50409.log.gz 2023-11-15T03:12:58.754 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47225.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47225.log.gz 2023-11-15T03:12:58.754 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-15T03:12:58.755 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48359.log 2023-11-15T03:12:58.755 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47360.log.gz 2023-11-15T03:12:58.756 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51805.log 2023-11-15T03:12:58.758 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48359.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48359.log.gz 2023-11-15T03:12:58.758 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61352.log 2023-11-15T03:12:58.759 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52360.log 2023-11-15T03:12:58.760 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51805.log.gz 2023-11-15T03:12:58.761 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61352.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.61352.log.gz-5 2023-11-15T03:12:58.761 INFO:teuthology.orchestra.run.smithi121.stderr:--verbose -- /var/log/ceph/ceph-client.admin.38716.log 2023-11-15T03:12:58.762 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58473.log 2023-11-15T03:12:58.763 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38716.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.52360.log.gz 2023-11-15T03:12:58.764 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54319.log 2023-11-15T03:12:58.765 INFO:teuthology.orchestra.run.smithi121.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38716.log.gz 2023-11-15T03:12:58.766 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54587.log 2023-11-15T03:12:58.766 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58473.log.gz 2023-11-15T03:12:58.767 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54319.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54319.log.gz 2023-11-15T03:12:58.767 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53562.log 2023-11-15T03:12:58.768 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.54587.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54587.log.gz 2023-11-15T03:12:58.769 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39067.log 2023-11-15T03:12:58.769 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53562.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53562.log.gz 2023-11-15T03:12:58.770 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42121.log 2023-11-15T03:12:58.771 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37461.log 2023-11-15T03:12:58.772 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39067.log.gz 2023-11-15T03:12:58.773 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42121.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42121.log.gz 2023-11-15T03:12:58.773 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61690.log 2023-11-15T03:12:58.774 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37461.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37461.log.gz 2023-11-15T03:12:58.774 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59948.log 2023-11-15T03:12:58.776 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43276.log 2023-11-15T03:12:58.776 INFO:teuthology.orchestra.run.smithi121.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61690.log.gz 2023-11-15T03:12:58.777 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59948.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59948.log.gz 2023-11-15T03:12:58.778 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62080.log 2023-11-15T03:12:58.781 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41110.log 2023-11-15T03:12:58.783 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62080.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43276.log.gz 2023-11-15T03:12:58.784 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62080.log.gz 2023-11-15T03:12:58.784 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47544.log 2023-11-15T03:12:58.784 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41110.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41110.log.gz 2023-11-15T03:12:58.784 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48203.log 2023-11-15T03:12:58.784 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47544.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47544.log.gz 2023-11-15T03:12:58.785 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41764.log 2023-11-15T03:12:58.785 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48203.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48203.log.gz 2023-11-15T03:12:58.786 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39640.log 2023-11-15T03:12:58.786 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56127.log 2023-11-15T03:12:58.788 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41764.log: /var/log/ceph/ceph-client.admin.39640.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41764.log.gz 2023-11-15T03:12:58.788 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39640.log.gz 2023-11-15T03:12:58.788 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63460.log 2023-11-15T03:12:58.789 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54439.log 2023-11-15T03:12:58.790 INFO:teuthology.orchestra.run.smithi121.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56127.log.gz 2023-11-15T03:12:58.791 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57829.log 2023-11-15T03:12:58.791 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63460.log.gz 2023-11-15T03:12:58.792 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54439.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54439.log.gz 2023-11-15T03:12:58.793 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57330.log 2023-11-15T03:12:58.794 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55156.log 2023-11-15T03:12:58.794 INFO:teuthology.orchestra.run.smithi121.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57829.log.gz 2023-11-15T03:12:58.795 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57330.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57330.log.gz 2023-11-15T03:12:58.795 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59332.log 2023-11-15T03:12:58.796 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55156.log: 91.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-15T03:12:58.797 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55156.log.gz 2023-11-15T03:12:58.797 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58188.log 2023-11-15T03:12:58.798 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36601.log 2023-11-15T03:12:58.798 INFO:teuthology.orchestra.run.smithi121.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59332.log.gz 2023-11-15T03:12:58.799 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54005.log 2023-11-15T03:12:58.800 INFO:teuthology.orchestra.run.smithi121.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.58188.log.gz 2023-11-15T03:12:58.800 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50570.log 2023-11-15T03:12:58.801 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36601.log.gz 2023-11-15T03:12:58.801 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55553.log 2023-11-15T03:12:58.803 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50570.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.54005.log.gz 2023-11-15T03:12:58.803 INFO:teuthology.orchestra.run.smithi121.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50570.log.gz 2023-11-15T03:12:58.803 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58960.log 2023-11-15T03:12:58.804 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53295.log 2023-11-15T03:12:58.805 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55553.log.gz 2023-11-15T03:12:58.805 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58960.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58960.log.gz 2023-11-15T03:12:58.806 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46737.log 2023-11-15T03:12:58.806 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63838.log 2023-11-15T03:12:58.807 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53295.log.gz 2023-11-15T03:12:58.808 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50013.log 2023-11-15T03:12:58.808 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46737.log.gz 2023-11-15T03:12:58.809 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40277.log 2023-11-15T03:12:58.809 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63838.log.gz 2023-11-15T03:12:58.810 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52543.log 2023-11-15T03:12:58.810 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50013.log.gz 2023-11-15T03:12:58.811 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57537.log 2023-11-15T03:12:58.811 INFO:teuthology.orchestra.run.smithi121.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40277.log.gz 2023-11-15T03:12:58.812 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52543.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52543.log.gz 2023-11-15T03:12:58.813 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62318.log 2023-11-15T03:12:58.814 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63058.log 2023-11-15T03:12:58.814 INFO:teuthology.orchestra.run.smithi121.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57537.log.gz 2023-11-15T03:12:58.815 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62852.log 2023-11-15T03:12:58.815 INFO:teuthology.orchestra.run.smithi121.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.62318.log.gz 2023-11-15T03:12:58.816 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63058.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63058.log.gz 2023-11-15T03:12:58.816 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63595.log 2023-11-15T03:12:58.817 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51722.log 2023-11-15T03:12:58.818 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62852.log.gz 2023-11-15T03:12:58.818 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63595.log: gzip -5 --verbose 88.9%-- -- replaced with /var/log/ceph/ceph-client.admin.63595.log.gz/var/log/ceph/ceph-client.admin.60325.log 2023-11-15T03:12:58.819 INFO:teuthology.orchestra.run.smithi121.stderr: 2023-11-15T03:12:58.820 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49297.log 2023-11-15T03:12:58.820 INFO:teuthology.orchestra.run.smithi121.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51722.log.gz 2023-11-15T03:12:58.821 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60325.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60325.log.gz 2023-11-15T03:12:58.821 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61325.log 2023-11-15T03:12:58.822 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50890.log 2023-11-15T03:12:58.822 INFO:teuthology.orchestra.run.smithi121.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49297.log.gz 2023-11-15T03:12:58.823 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61325.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61325.log.gz 2023-11-15T03:12:58.824 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44370.log 2023-11-15T03:12:58.825 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63014.log 2023-11-15T03:12:58.825 INFO:teuthology.orchestra.run.smithi121.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50890.log.gz 2023-11-15T03:12:58.826 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44370.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44370.log.gz 2023-11-15T03:12:58.826 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48747.log 2023-11-15T03:12:58.827 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57199.log 2023-11-15T03:12:58.827 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63014.log.gz 2023-11-15T03:12:58.828 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49121.log 2023-11-15T03:12:58.829 INFO:teuthology.orchestra.run.smithi121.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.48747.log.gz 2023-11-15T03:12:58.829 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57199.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57199.log.gz 2023-11-15T03:12:58.830 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51610.log 2023-11-15T03:12:58.830 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44977.log 2023-11-15T03:12:58.831 INFO:teuthology.orchestra.run.smithi121.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49121.log.gz 2023-11-15T03:12:58.832 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62798.log 2023-11-15T03:12:58.832 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51610.log.gz 2023-11-15T03:12:58.833 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38770.log 2023-11-15T03:12:58.833 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44977.log.gz 2023-11-15T03:12:58.834 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62798.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62798.log.gz 2023-11-15T03:12:58.834 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50780.log 2023-11-15T03:12:58.835 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54541.log 2023-11-15T03:12:58.836 INFO:teuthology.orchestra.run.smithi121.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38770.log.gz 2023-11-15T03:12:58.836 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50780.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50780.log.gz 2023-11-15T03:12:58.837 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48592.log 2023-11-15T03:12:58.838 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45591.log 2023-11-15T03:12:58.838 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54541.log.gz 2023-11-15T03:12:58.839 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39324.log 2023-11-15T03:12:58.839 INFO:teuthology.orchestra.run.smithi121.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48592.log.gz 2023-11-15T03:12:58.840 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45591.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45591.log.gz 2023-11-15T03:12:58.840 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48901.log 2023-11-15T03:12:58.841 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43732.log 2023-11-15T03:12:58.842 INFO:teuthology.orchestra.run.smithi121.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39324.log.gz 2023-11-15T03:12:58.842 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48901.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.48901.log.gz 2023-11-15T03:12:58.843 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52923.log 2023-11-15T03:12:58.844 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53789.log 2023-11-15T03:12:58.845 INFO:teuthology.orchestra.run.smithi121.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43732.log.gz 2023-11-15T03:12:58.845 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52923.log: gzip -5 89.0%--verbose -- replaced with /var/log/ceph/ceph-client.admin.52923.log.gz-- 2023-11-15T03:12:58.846 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62906.log 2023-11-15T03:12:58.847 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50936.log 2023-11-15T03:12:58.847 INFO:teuthology.orchestra.run.smithi121.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53789.log.gz 2023-11-15T03:12:58.848 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51104.log 2023-11-15T03:12:58.848 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62906.log.gz 2023-11-15T03:12:58.849 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46801.log 2023-11-15T03:12:58.849 INFO:teuthology.orchestra.run.smithi121.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.50936.log.gz 2023-11-15T03:12:58.850 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59625.log 2023-11-15T03:12:58.850 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51104.log.gz 2023-11-15T03:12:58.851 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58024.log 2023-11-15T03:12:58.852 INFO:teuthology.orchestra.run.smithi121.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46801.log.gz 2023-11-15T03:12:58.853 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59625.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59625.log.gz 2023-11-15T03:12:58.853 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37488.log 2023-11-15T03:12:58.854 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58024.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.58024.log.gz 2023-11-15T03:12:58.854 INFO:teuthology.orchestra.run.smithi121.stderr:-5 --verbose -- /var/log/ceph/ceph-client.0.56763.log 2023-11-15T03:12:58.855 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55183.log 2023-11-15T03:12:58.855 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37488.log.gz 2023-11-15T03:12:58.856 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.56763.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56763.log.gz 2023-11-15T03:12:58.857 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61139.log 2023-11-15T03:12:58.857 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41362.log 2023-11-15T03:12:58.858 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55183.log.gz 2023-11-15T03:12:58.858 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61139.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61139.log.gz 2023-11-15T03:12:58.859 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41864.log 2023-11-15T03:12:58.860 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39243.log 2023-11-15T03:12:58.860 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41362.log.gz 2023-11-15T03:12:58.861 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41864.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.41864.log.gz 2023-11-15T03:12:58.862 INFO:teuthology.orchestra.run.smithi121.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.52407.log 2023-11-15T03:12:58.862 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39243.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-15T03:12:58.863 INFO:teuthology.orchestra.run.smithi121.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39243.log.gz 2023-11-15T03:12:58.864 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37029.log 2023-11-15T03:12:58.864 INFO:teuthology.orchestra.run.smithi121.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52407.log.gz 2023-11-15T03:12:58.865 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52309.log 2023-11-15T03:12:58.866 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37029.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37029.log.gz 2023-11-15T03:12:58.866 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62627.log 2023-11-15T03:12:58.868 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60252.log 2023-11-15T03:12:58.869 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52309.log: /var/log/ceph/ceph-client.admin.62627.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52309.log.gz 2023-11-15T03:12:58.870 INFO:teuthology.orchestra.run.smithi121.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62627.log.gz 2023-11-15T03:12:58.870 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39405.log 2023-11-15T03:12:58.871 INFO:teuthology.orchestra.run.smithi121.stderr:gzip /var/log/ceph/ceph-client.admin.60252.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48769.log 2023-11-15T03:12:58.871 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60252.log.gz 2023-11-15T03:12:58.872 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39405.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39405.log.gz 2023-11-15T03:12:58.872 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36836.log 2023-11-15T03:12:58.873 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48769.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.48769.log.gz 2023-11-15T03:12:58.874 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51695.log 2023-11-15T03:12:58.878 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60216.log 2023-11-15T03:12:58.879 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36836.log.gz 2023-11-15T03:12:58.879 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51695.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51695.log.gz 2023-11-15T03:12:58.879 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47571.log 2023-11-15T03:12:58.879 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60216.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60216.log.gz 2023-11-15T03:12:58.880 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37812.log 2023-11-15T03:12:58.881 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49901.log 2023-11-15T03:12:58.881 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47571.log.gz 2023-11-15T03:12:58.882 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37812.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37812.log.gz 2023-11-15T03:12:58.882 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43962.log 2023-11-15T03:12:58.883 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49901.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49901.log.gz 2023-11-15T03:12:58.883 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38851.log 2023-11-15T03:12:58.884 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49613.log 2023-11-15T03:12:58.884 INFO:teuthology.orchestra.run.smithi121.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43962.log.gz 2023-11-15T03:12:58.886 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38851.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38851.log.gz 2023-11-15T03:12:58.886 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54412.log 2023-11-15T03:12:58.888 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49613.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49613.log.gz 2023-11-15T03:12:58.888 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48122.log 2023-11-15T03:12:58.889 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49033.log 2023-11-15T03:12:58.889 INFO:teuthology.orchestra.run.smithi121.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54412.log.gz 2023-11-15T03:12:58.890 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48122.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48122.log.gz 2023-11-15T03:12:58.890 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56783.log 2023-11-15T03:12:58.891 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49033.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49033.log.gz 2023-11-15T03:12:58.892 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42285.log 2023-11-15T03:12:58.893 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53131.log 2023-11-15T03:12:58.893 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56783.log.gz 2023-11-15T03:12:58.895 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42285.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42285.log.gz 2023-11-15T03:12:58.895 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36946.log 2023-11-15T03:12:58.896 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55517.log 2023-11-15T03:12:58.897 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53131.log.gz 2023-11-15T03:12:58.898 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36946.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36946.log.gz 2023-11-15T03:12:58.898 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59578.log 2023-11-15T03:12:58.900 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52646.log 2023-11-15T03:12:58.900 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55517.log.gz 2023-11-15T03:12:58.901 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42148.log 2023-11-15T03:12:58.902 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52646.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.59578.log.gz 2023-11-15T03:12:58.903 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62053.log 2023-11-15T03:12:58.903 INFO:teuthology.orchestra.run.smithi121.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52646.log.gz 2023-11-15T03:12:58.904 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46882.log 2023-11-15T03:12:58.905 INFO:teuthology.orchestra.run.smithi121.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42148.log.gz 2023-11-15T03:12:58.905 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62053.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62053.log.gz 2023-11-15T03:12:58.906 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41524.log 2023-11-15T03:12:58.906 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62717.log 2023-11-15T03:12:58.908 INFO:teuthology.orchestra.run.smithi121.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46882.log.gz 2023-11-15T03:12:58.909 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41524.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41524.log.gz 2023-11-15T03:12:58.909 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43530.log 2023-11-15T03:12:58.909 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43790.log 2023-11-15T03:12:58.910 INFO:teuthology.orchestra.run.smithi121.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62717.log.gz 2023-11-15T03:12:58.910 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43530.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43530.log.gz 2023-11-15T03:12:58.912 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37434.log 2023-11-15T03:12:58.913 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43790.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43790.log.gz 2023-11-15T03:12:58.914 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38500.log 2023-11-15T03:12:58.914 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37434.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37434.log.gz 2023-11-15T03:12:58.915 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60731.log 2023-11-15T03:12:58.916 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38986.log 2023-11-15T03:12:58.917 INFO:teuthology.orchestra.run.smithi121.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38500.log.gz 2023-11-15T03:12:58.917 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60731.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60731.log.gz 2023-11-15T03:12:58.918 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61744.log 2023-11-15T03:12:58.919 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56449.log 2023-11-15T03:12:58.919 INFO:teuthology.orchestra.run.smithi121.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38986.log.gz 2023-11-15T03:12:58.920 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61744.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61744.log.gz 2023-11-15T03:12:58.921 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50961.log 2023-11-15T03:12:58.922 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61905.log 2023-11-15T03:12:58.922 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56449.log.gz 2023-11-15T03:12:58.923 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50961.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50961.log.gz 2023-11-15T03:12:58.924 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41497.log 2023-11-15T03:12:58.924 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61058.log 2023-11-15T03:12:58.925 INFO:teuthology.orchestra.run.smithi121.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61905.log.gz 2023-11-15T03:12:58.925 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41497.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41497.log.gz 2023-11-15T03:12:58.926 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49478.log 2023-11-15T03:12:58.927 INFO:teuthology.orchestra.run.smithi121.stderr:gzip /var/log/ceph/ceph-client.admin.61058.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51214.log 2023-11-15T03:12:58.927 INFO:teuthology.orchestra.run.smithi121.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61058.log.gz 2023-11-15T03:12:58.928 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49478.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49478.log.gz 2023-11-15T03:12:58.929 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60704.log 2023-11-15T03:12:58.929 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43014.log 2023-11-15T03:12:58.930 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51214.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51214.log.gz 2023-11-15T03:12:58.931 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60704.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60704.log.gz 2023-11-15T03:12:58.931 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45024.log 2023-11-15T03:12:58.932 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42414.log 2023-11-15T03:12:58.933 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43014.log.gz 2023-11-15T03:12:58.933 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45024.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45024.log.gz 2023-11-15T03:12:58.934 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37866.log 2023-11-15T03:12:58.935 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36661.log 2023-11-15T03:12:58.935 INFO:teuthology.orchestra.run.smithi121.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42414.log.gz 2023-11-15T03:12:58.936 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37866.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37866.log.gz 2023-11-15T03:12:58.936 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52970.log 2023-11-15T03:12:58.937 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44255.log 2023-11-15T03:12:58.938 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36661.log.gz 2023-11-15T03:12:58.938 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52970.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52970.log.gz 2023-11-15T03:12:58.939 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60479.log 2023-11-15T03:12:58.940 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40510.log 2023-11-15T03:12:58.941 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44255.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44255.log.gz 2023-11-15T03:12:58.941 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60479.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60479.log.gz 2023-11-15T03:12:58.941 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61636.log 2023-11-15T03:12:58.943 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38406.log 2023-11-15T03:12:58.944 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40510.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40510.log.gz 2023-11-15T03:12:58.944 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61636.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61636.log.gz 2023-11-15T03:12:58.944 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53608.log 2023-11-15T03:12:58.945 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37731.log 2023-11-15T03:12:58.946 INFO:teuthology.orchestra.run.smithi121.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38406.log.gz 2023-11-15T03:12:58.946 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53608.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53608.log.gz 2023-11-15T03:12:58.946 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63298.log 2023-11-15T03:12:58.948 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46334.log 2023-11-15T03:12:58.948 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37731.log.gz 2023-11-15T03:12:58.949 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63298.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63298.log.gz 2023-11-15T03:12:58.949 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63784.log 2023-11-15T03:12:58.950 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47306.log 2023-11-15T03:12:58.951 INFO:teuthology.orchestra.run.smithi121.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46334.log.gz 2023-11-15T03:12:58.951 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63784.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63784.log.gz 2023-11-15T03:12:58.952 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39162.log 2023-11-15T03:12:58.953 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56551.log 2023-11-15T03:12:58.953 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47306.log.gz 2023-11-15T03:12:58.954 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39162.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39162.log.gz 2023-11-15T03:12:58.954 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40196.log 2023-11-15T03:12:58.955 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39586.log 2023-11-15T03:12:58.956 INFO:teuthology.orchestra.run.smithi190.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-15T03:12:58.956 INFO:teuthology.orchestra.run.smithi121.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.56551.log.gz 2023-11-15T03:12:58.957 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40196.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40196.log.gz 2023-11-15T03:12:58.957 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51169.log 2023-11-15T03:12:58.958 INFO:teuthology.orchestra.run.smithi121.stderr:gzip /var/log/ceph/ceph-client.admin.39586.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42697.log 2023-11-15T03:12:58.958 INFO:teuthology.orchestra.run.smithi121.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39586.log.gz 2023-11-15T03:12:58.959 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51169.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51169.log.gz 2023-11-15T03:12:58.959 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61663.log 2023-11-15T03:12:58.960 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48149.log 2023-11-15T03:12:58.961 INFO:teuthology.orchestra.run.smithi121.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42697.log.gz 2023-11-15T03:12:58.961 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61663.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61663.log.gz 2023-11-15T03:12:58.962 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48945.log 2023-11-15T03:12:58.963 INFO:teuthology.orchestra.run.smithi121.stderr:gzip /var/log/ceph/ceph-client.admin.48149.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44126.log 2023-11-15T03:12:58.963 INFO:teuthology.orchestra.run.smithi121.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48149.log.gz 2023-11-15T03:12:58.964 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48945.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48945.log.gz 2023-11-15T03:12:58.964 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60812.log 2023-11-15T03:12:58.965 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37569.log 2023-11-15T03:12:58.966 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44126.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44126.log.gz 2023-11-15T03:12:58.967 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60812.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60812.log.gz 2023-11-15T03:12:58.967 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61406.log 2023-11-15T03:12:58.968 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45629.log 2023-11-15T03:12:58.968 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37569.log.gz 2023-11-15T03:12:58.969 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61406.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61406.log.gz 2023-11-15T03:12:58.970 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62879.log 2023-11-15T03:12:58.970 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39216.log 2023-11-15T03:12:58.971 INFO:teuthology.orchestra.run.smithi121.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45629.log.gz 2023-11-15T03:12:58.971 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62879.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62879.log.gz 2023-11-15T03:12:58.972 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60516.log 2023-11-15T03:12:58.973 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63568.log 2023-11-15T03:12:58.973 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39216.log.gz 2023-11-15T03:12:58.974 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60516.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60516.log.gz 2023-11-15T03:12:58.974 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60948.log 2023-11-15T03:12:58.976 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45199.log 2023-11-15T03:12:58.976 INFO:teuthology.orchestra.run.smithi121.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63568.log.gz 2023-11-15T03:12:58.977 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60948.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.60948.log.gz 2023-11-15T03:12:58.977 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46686.log 2023-11-15T03:12:58.979 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53871.log 2023-11-15T03:12:58.979 INFO:teuthology.orchestra.run.smithi121.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.45199.log.gz 2023-11-15T03:12:58.980 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54517.log 2023-11-15T03:12:58.980 INFO:teuthology.orchestra.run.smithi121.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.46686.log.gz 2023-11-15T03:12:58.981 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49055.log 2023-11-15T03:12:58.982 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53871.log.gz 2023-11-15T03:12:58.982 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54517.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54517.log.gz 2023-11-15T03:12:58.983 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48093.log 2023-11-15T03:12:58.984 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49055.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49055.log.gz 2023-11-15T03:12:58.985 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37380.log 2023-11-15T03:12:58.989 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48093.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48093.log.gz 2023-11-15T03:12:58.990 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40358.log 2023-11-15T03:12:58.991 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37380.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37380.log.gz 2023-11-15T03:12:58.992 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37920.log 2023-11-15T03:12:58.992 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57464.log 2023-11-15T03:12:58.992 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37920.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.40358.log.gz 2023-11-15T03:12:58.992 INFO:teuthology.orchestra.run.smithi121.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37920.log.gz 2023-11-15T03:12:58.992 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43438.log 2023-11-15T03:12:58.993 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57464.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57464.log.gz 2023-11-15T03:12:58.994 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48284.log 2023-11-15T03:12:58.994 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43438.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43438.log.gz 2023-11-15T03:12:58.994 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56310.log 2023-11-15T03:12:58.995 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51960.log 2023-11-15T03:12:58.995 INFO:teuthology.orchestra.run.smithi121.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48284.log.gz 2023-11-15T03:12:58.996 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56310.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56310.log.gz 2023-11-15T03:12:58.996 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54137.log 2023-11-15T03:12:58.998 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51960.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51960.log.gz 2023-11-15T03:12:58.998 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38824.log 2023-11-15T03:12:58.999 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54137.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54137.log.gz 2023-11-15T03:12:59.000 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38689.log 2023-11-15T03:12:59.001 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38824.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38824.log.gz 2023-11-15T03:12:59.001 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57247.log 2023-11-15T03:12:59.003 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38689.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38689.log.gz 2023-11-15T03:12:59.003 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57510.log 2023-11-15T03:12:59.005 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57247.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57247.log.gz 2023-11-15T03:12:59.005 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45739.log 2023-11-15T03:12:59.006 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46855.log 2023-11-15T03:12:59.006 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57510.log.gz 2023-11-15T03:12:59.008 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45739.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45739.log.gz 2023-11-15T03:12:59.008 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39378.log 2023-11-15T03:12:59.009 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46855.log: 90.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-15T03:12:59.009 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46855.log.gz 2023-11-15T03:12:59.010 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45137.log 2023-11-15T03:12:59.010 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39378.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39378.log.gz 2023-11-15T03:12:59.011 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57620.log 2023-11-15T03:12:59.012 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45137.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45137.log.gz 2023-11-15T03:12:59.012 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-15T03:12:59.012 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57620.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57620.log.gz 2023-11-15T03:12:59.013 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47029.log 2023-11-15T03:12:59.014 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41004.log 2023-11-15T03:12:59.015 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47711.log 2023-11-15T03:12:59.016 INFO:teuthology.orchestra.run.smithi121.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.47029.log.gz 2023-11-15T03:12:59.017 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41004.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.41004.log.gz 2023-11-15T03:12:59.018 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53662.log 2023-11-15T03:12:59.018 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44301.log 2023-11-15T03:12:59.019 INFO:teuthology.orchestra.run.smithi121.stderr: 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-15T03:12:59.019 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47711.log.gz 2023-11-15T03:12:59.020 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53662.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53662.log.gz 2023-11-15T03:12:59.020 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55073.log 2023-11-15T03:12:59.021 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.44301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50069.log 2023-11-15T03:12:59.022 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44301.log.gz 2023-11-15T03:12:59.022 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55073.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55073.log.gz 2023-11-15T03:12:59.023 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38028.log 2023-11-15T03:12:59.023 INFO:teuthology.orchestra.run.smithi121.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-15T03:12:59.024 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50069.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50069.log.gz 2023-11-15T03:12:59.024 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54942.log 2023-11-15T03:12:59.025 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46060.log 2023-11-15T03:12:59.026 INFO:teuthology.orchestra.run.smithi121.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38028.log.gz 2023-11-15T03:12:59.026 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61464.log 2023-11-15T03:12:59.028 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46060.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46060.log.gz 2023-11-15T03:12:59.028 INFO:teuthology.orchestra.run.smithi121.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.54942.log.gz 2023-11-15T03:12:59.028 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40032.log 2023-11-15T03:12:59.029 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48405.log 2023-11-15T03:12:59.030 INFO:teuthology.orchestra.run.smithi121.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61464.log.gz 2023-11-15T03:12:59.031 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40032.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40032.log.gz 2023-11-15T03:12:59.031 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44097.log 2023-11-15T03:12:59.033 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.48405.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48405.log.gz 2023-11-15T03:12:59.033 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61717.log 2023-11-15T03:12:59.034 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44097.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44097.log.gz 2023-11-15T03:12:59.035 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63406.log 2023-11-15T03:12:59.036 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38277.log 2023-11-15T03:12:59.037 INFO:teuthology.orchestra.run.smithi121.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61717.log.gz 2023-11-15T03:12:59.037 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50387.log 2023-11-15T03:12:59.038 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63406.log: /var/log/ceph/ceph-client.admin.38277.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63406.log.gz 2023-11-15T03:12:59.038 INFO:teuthology.orchestra.run.smithi121.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38277.log.gz 2023-11-15T03:12:59.039 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41098.log 2023-11-15T03:12:59.040 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41972.log 2023-11-15T03:12:59.040 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.41098.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.50387.log.gz 2023-11-15T03:12:59.041 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.41098.log.gz 2023-11-15T03:12:59.041 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53397.log 2023-11-15T03:12:59.042 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41972.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41972.log.gz 2023-11-15T03:12:59.042 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45080.log 2023-11-15T03:12:59.043 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53397.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53397.log.gz 2023-11-15T03:12:59.044 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48725.log 2023-11-15T03:12:59.045 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58580.log 2023-11-15T03:12:59.045 INFO:teuthology.orchestra.run.smithi121.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.45080.log.gz 2023-11-15T03:12:59.046 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54220.log 2023-11-15T03:12:59.046 INFO:teuthology.orchestra.run.smithi121.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48725.log.gz 2023-11-15T03:12:59.047 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49576.log 2023-11-15T03:12:59.048 INFO:teuthology.orchestra.run.smithi121.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.58580.log.gz 2023-11-15T03:12:59.049 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61819.log 2023-11-15T03:12:59.049 INFO:teuthology.orchestra.run.smithi121.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54220.log.gz 2023-11-15T03:12:59.050 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48989.log 2023-11-15T03:12:59.050 INFO:teuthology.orchestra.run.smithi121.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49576.log.gz 2023-11-15T03:12:59.051 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61819.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61819.log.gz 2023-11-15T03:12:59.051 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48835.log 2023-11-15T03:12:59.053 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42312.log 2023-11-15T03:12:59.053 INFO:teuthology.orchestra.run.smithi121.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48989.log.gz 2023-11-15T03:12:59.054 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42762.log 2023-11-15T03:12:59.054 INFO:teuthology.orchestra.run.smithi121.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48835.log.gz 2023-11-15T03:12:59.055 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56285.log 2023-11-15T03:12:59.055 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42312.log.gz 2023-11-15T03:12:59.056 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42762.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42762.log.gz 2023-11-15T03:12:59.056 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56478.log 2023-11-15T03:12:59.057 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40088.log 2023-11-15T03:12:59.058 INFO:teuthology.orchestra.run.smithi121.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56285.log.gz 2023-11-15T03:12:59.059 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44153.log 2023-11-15T03:12:59.059 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56478.log.gz 2023-11-15T03:12:59.060 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40088.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40088.log.gz 2023-11-15T03:12:59.060 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59766.log 2023-11-15T03:12:59.061 INFO:teuthology.orchestra.run.smithi190.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-15T03:12:59.061 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44153.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44153.log.gz 2023-11-15T03:12:59.061 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58722.log 2023-11-15T03:12:59.062 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56900.log 2023-11-15T03:12:59.062 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59766.log.gz 2023-11-15T03:12:59.063 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47838.log 2023-11-15T03:12:59.064 INFO:teuthology.orchestra.run.smithi121.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.58722.log.gz 2023-11-15T03:12:59.064 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56900.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56900.log.gz 2023-11-15T03:12:59.065 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48570.log 2023-11-15T03:12:59.066 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42092.log 87.5% 2023-11-15T03:12:59.066 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47838.log.gz 2023-11-15T03:12:59.067 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38959.log 2023-11-15T03:12:59.068 INFO:teuthology.orchestra.run.smithi121.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48570.log.gz 2023-11-15T03:12:59.068 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42092.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42092.log.gz 2023-11-15T03:12:59.069 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49505.log 2023-11-15T03:12:59.070 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44927.log 2023-11-15T03:12:59.070 INFO:teuthology.orchestra.run.smithi121.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38959.log.gz 2023-11-15T03:12:59.071 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49505.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49505.log.gz 2023-11-15T03:12:59.071 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45313.log 2023-11-15T03:12:59.072 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44927.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44927.log.gz 2023-11-15T03:12:59.072 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46307.log 2023-11-15T03:12:59.073 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61961.log 2023-11-15T03:12:59.074 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45313.log.gz 2023-11-15T03:12:59.074 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39108.log 2023-11-15T03:12:59.075 INFO:teuthology.orchestra.run.smithi121.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46307.log.gz 2023-11-15T03:12:59.076 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61961.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.61961.log.gz 2023-11-15T03:12:59.076 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56347.log 2023-11-15T03:12:59.077 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57381.log 2023-11-15T03:12:59.078 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39108.log.gz 2023-11-15T03:12:59.079 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56347.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56347.log.gz 2023-11-15T03:12:59.079 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44680.log 2023-11-15T03:12:59.080 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49143.log 2023-11-15T03:12:59.081 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44680.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44680.log.gz 2023-11-15T03:12:59.082 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40452.log 2023-11-15T03:12:59.082 INFO:teuthology.orchestra.run.smithi121.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.57381.log.gz 2023-11-15T03:12:59.083 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49143.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.49143.log.gz 2023-11-15T03:12:59.083 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56393.log 2023-11-15T03:12:59.084 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.40452.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40452.log.gz 2023-11-15T03:12:59.084 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63487.log 2023-11-15T03:12:59.085 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56393.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56393.log.gz 2023-11-15T03:12:59.085 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41891.log 2023-11-15T03:12:59.086 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63487.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63487.log.gz 2023-11-15T03:12:59.086 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41470.log 2023-11-15T03:12:59.087 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41891.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41891.log.gz 2023-11-15T03:12:59.088 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51286.log 2023-11-15T03:12:59.088 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39748.log 2023-11-15T03:12:59.089 INFO:teuthology.orchestra.run.smithi121.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41470.log.gz 2023-11-15T03:12:59.090 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51286.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51286.log.gz 2023-11-15T03:12:59.090 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59175.log 2023-11-15T03:12:59.091 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46280.log 88.2% 2023-11-15T03:12:59.091 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39748.log.gz 2023-11-15T03:12:59.092 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59175.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59175.log.gz 2023-11-15T03:12:59.092 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37245.log 2023-11-15T03:12:59.093 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46280.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46280.log.gz 2023-11-15T03:12:59.093 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45979.log 2023-11-15T03:12:59.094 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37245.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37245.log.gz 2023-11-15T03:12:59.095 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40951.log 2023-11-15T03:12:59.096 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45979.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45979.log.gz 2023-11-15T03:12:59.096 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60677.log 2023-11-15T03:12:59.097 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44707.log 2023-11-15T03:12:59.097 INFO:teuthology.orchestra.run.smithi121.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.40951.log.gz 2023-11-15T03:12:59.098 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60677.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60677.log.gz 2023-11-15T03:12:59.098 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47387.log 2023-11-15T03:12:59.100 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58695.log 2023-11-15T03:12:59.100 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47387.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44707.log.gz 2023-11-15T03:12:59.101 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47387.log.gz 2023-11-15T03:12:59.102 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59718.log 2023-11-15T03:12:59.103 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50516.log 2023-11-15T03:12:59.104 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58695.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58695.log.gz 2023-11-15T03:12:59.105 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60650.log 2023-11-15T03:12:59.105 INFO:teuthology.orchestra.run.smithi121.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59718.log.gz 2023-11-15T03:12:59.106 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61526.log 2023-11-15T03:12:59.107 INFO:teuthology.orchestra.run.smithi121.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50516.log.gz 2023-11-15T03:12:59.108 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60650.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60650.log.gz 2023-11-15T03:12:59.108 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46172.log 2023-11-15T03:12:59.109 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39943.log 2023-11-15T03:12:59.109 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61526.log.gz 2023-11-15T03:12:59.110 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46172.log: gzip -5 89.0%--verbose -- replaced with /var/log/ceph/ceph-client.admin.46172.log.gz 2023-11-15T03:12:59.111 INFO:teuthology.orchestra.run.smithi121.stderr:-- /var/log/ceph/ceph-client.admin.59691.log 2023-11-15T03:12:59.112 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37785.log 2023-11-15T03:12:59.112 INFO:teuthology.orchestra.run.smithi121.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39943.log.gz 2023-11-15T03:12:59.113 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59691.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59691.log.gz 2023-11-15T03:12:59.114 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63541.log 2023-11-15T03:12:59.114 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53825.log 2023-11-15T03:12:59.115 INFO:teuthology.orchestra.run.smithi121.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37785.log.gz 2023-11-15T03:12:59.117 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63541.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63541.log.gz 2023-11-15T03:12:59.117 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59357.log 2023-11-15T03:12:59.118 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46199.log 2023-11-15T03:12:59.118 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59357.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53825.log.gz 2023-11-15T03:12:59.119 INFO:teuthology.orchestra.run.smithi121.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59357.log.gz 2023-11-15T03:12:59.119 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49429.log 2023-11-15T03:12:59.120 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63433.log 2023-11-15T03:12:59.121 INFO:teuthology.orchestra.run.smithi121.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46199.log.gz 2023-11-15T03:12:59.122 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49429.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49429.log.gz 2023-11-15T03:12:59.123 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54679.log 2023-11-15T03:12:59.124 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52336.log 2023-11-15T03:12:59.124 INFO:teuthology.orchestra.run.smithi121.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63433.log.gz 2023-11-15T03:12:59.125 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40478.log 2023-11-15T03:12:59.125 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52336.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.54679.log.gz 2023-11-15T03:12:59.126 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54466.log 2023-11-15T03:12:59.126 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52336.log.gz 2023-11-15T03:12:59.127 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40478.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40478.log.gz 2023-11-15T03:12:59.127 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40792.log 2023-11-15T03:12:59.128 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37515.log 2023-11-15T03:12:59.129 INFO:teuthology.orchestra.run.smithi121.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54466.log.gz 2023-11-15T03:12:59.129 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.40792.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40792.log.gz 2023-11-15T03:12:59.129 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53635.log 2023-11-15T03:12:59.130 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45793.log 2023-11-15T03:12:59.130 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37515.log.gz 2023-11-15T03:12:59.132 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53635.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53635.log.gz 2023-11-15T03:12:59.132 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41945.log 2023-11-15T03:12:59.133 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45793.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45793.log.gz 2023-11-15T03:12:59.133 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61298.log 2023-11-15T03:12:59.134 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41945.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41945.log.gz 2023-11-15T03:12:59.135 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55653.log 2023-11-15T03:12:59.136 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61298.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61298.log.gz 2023-11-15T03:12:59.137 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44846.log 2023-11-15T03:12:59.138 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51029.log 2023-11-15T03:12:59.138 INFO:teuthology.orchestra.run.smithi121.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55653.log.gz 2023-11-15T03:12:59.139 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44846.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44846.log.gz 2023-11-15T03:12:59.140 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63271.log 2023-11-15T03:12:59.141 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62471.log 2023-11-15T03:12:59.142 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51029.log.gz 2023-11-15T03:12:59.142 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63271.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63271.log.gz 2023-11-15T03:12:59.143 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40223.log 2023-11-15T03:12:59.144 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44180.log 2023-11-15T03:12:59.145 INFO:teuthology.orchestra.run.smithi121.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62471.log.gz 2023-11-15T03:12:59.146 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40223.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40223.log.gz 2023-11-15T03:12:59.146 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62015.log 2023-11-15T03:12:59.147 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47333.log 2023-11-15T03:12:59.147 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44180.log.gz 2023-11-15T03:12:59.149 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62015.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62015.log.gz 2023-11-15T03:12:59.149 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53526.log 2023-11-15T03:12:59.151 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42258.log 2023-11-15T03:12:59.152 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47333.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47333.log.gz 2023-11-15T03:12:59.153 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48923.log /var/log/ceph/ceph-client.admin.53526.log: 2023-11-15T03:12:59.153 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53526.log.gz 2023-11-15T03:12:59.154 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42258.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42258.log.gz 2023-11-15T03:12:59.154 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48430.log 2023-11-15T03:12:59.157 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49876.log 2023-11-15T03:12:59.157 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48923.log: /var/log/ceph/ceph-client.admin.48430.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48923.log.gz 2023-11-15T03:12:59.157 INFO:teuthology.orchestra.run.smithi121.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48430.log.gz 2023-11-15T03:12:59.158 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53428.log 2023-11-15T03:12:59.159 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63622.log 2023-11-15T03:12:59.160 INFO:teuthology.orchestra.run.smithi121.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49876.log.gz 2023-11-15T03:12:59.160 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53428.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53428.log.gz 2023-11-15T03:12:59.161 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63244.log 2023-11-15T03:12:59.162 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38932.log 2023-11-15T03:12:59.163 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8%/var/log/ceph/ceph-client.admin.63244.log: -- replaced with /var/log/ceph/ceph-client.admin.63622.log.gz 2023-11-15T03:12:59.164 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60975.log 2023-11-15T03:12:59.164 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63244.log.gz 2023-11-15T03:12:59.165 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57092.log 2023-11-15T03:12:59.165 INFO:teuthology.orchestra.run.smithi121.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38932.log.gz 2023-11-15T03:12:59.166 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60975.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60975.log.gz 2023-11-15T03:12:59.166 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37947.log 2023-11-15T03:12:59.166 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57092.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57092.log.gz 2023-11-15T03:12:59.167 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54891.log 2023-11-15T03:12:59.169 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58145.log 2023-11-15T03:12:59.169 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37947.log.gz 2023-11-15T03:12:59.170 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45394.log 2023-11-15T03:12:59.171 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54891.log: /var/log/ceph/ceph-client.0.58145.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58145.log.gz 2023-11-15T03:12:59.172 INFO:teuthology.orchestra.run.smithi121.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54891.log.gz 2023-11-15T03:12:59.172 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54862.log 2023-11-15T03:12:59.173 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50096.log 2023-11-15T03:12:59.174 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45394.log.gz 2023-11-15T03:12:59.174 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37056.log 2023-11-15T03:12:59.175 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54862.log.gz 2023-11-15T03:12:59.176 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45858.log 2023-11-15T03:12:59.176 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37056.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50096.log.gz 2023-11-15T03:12:59.177 INFO:teuthology.orchestra.run.smithi121.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37056.log.gz 2023-11-15T03:12:59.177 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52382.log 2023-11-15T03:12:59.178 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61937.log 2023-11-15T03:12:59.179 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52382.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45858.log.gz 2023-11-15T03:12:59.180 INFO:teuthology.orchestra.run.smithi121.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.52382.log.gz 2023-11-15T03:12:59.180 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47002.log 2023-11-15T03:12:59.182 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39667.log 2023-11-15T03:12:59.182 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47002.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61937.log.gz 2023-11-15T03:12:59.183 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47002.log.gz 2023-11-15T03:12:59.183 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48791.log 2023-11-15T03:12:59.185 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39667.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39667.log.gz 2023-11-15T03:12:59.185 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51851.log 2023-11-15T03:12:59.187 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52280.log 2023-11-15T03:12:59.188 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48791.log: /var/log/ceph/ceph-client.admin.51851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49341.log 2023-11-15T03:12:59.188 INFO:teuthology.orchestra.run.smithi121.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48791.log.gz 2023-11-15T03:12:59.188 INFO:teuthology.orchestra.run.smithi121.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51851.log.gz 2023-11-15T03:12:59.189 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52280.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52280.log.gz 2023-11-15T03:12:59.190 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55470.log 2023-11-15T03:12:59.195 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62576.log 2023-11-15T03:12:59.196 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49341.log: /var/log/ceph/ceph-client.admin.55470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55682.log 2023-11-15T03:12:59.196 INFO:teuthology.orchestra.run.smithi121.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49341.log.gz 2023-11-15T03:12:59.196 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62576.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.55470.log.gz 2023-11-15T03:12:59.196 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42789.log 2023-11-15T03:12:59.197 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55682.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.62576.log.gz 2023-11-15T03:12:59.197 INFO:teuthology.orchestra.run.smithi121.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55682.log.gz 2023-11-15T03:12:59.197 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56154.log 2023-11-15T03:12:59.198 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55419.log 2023-11-15T03:12:59.198 INFO:teuthology.orchestra.run.smithi121.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.42789.log.gz 2023-11-15T03:12:59.199 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40534.log 2023-11-15T03:12:59.199 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56154.log: /var/log/ceph/ceph-client.admin.55419.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56154.log.gz 2023-11-15T03:12:59.200 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37839.log 2023-11-15T03:12:59.200 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55419.log.gz 2023-11-15T03:12:59.201 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40534.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.40534.log.gz 2023-11-15T03:12:59.201 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58666.log 2023-11-15T03:12:59.202 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37839.log: gzip -5 --verbose 88.1%-- -- replaced with /var/log/ceph/ceph-client.admin.37839.log.gz/var/log/ceph/ceph-client.admin.39978.log 2023-11-15T03:12:59.202 INFO:teuthology.orchestra.run.smithi121.stderr: 2023-11-15T03:12:59.203 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58666.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58666.log.gz 2023-11-15T03:12:59.204 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41335.log 2023-11-15T03:12:59.204 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39978.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39978.log.gz 2023-11-15T03:12:59.205 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39486.log 2023-11-15T03:12:59.205 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41335.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41335.log.gz 2023-11-15T03:12:59.206 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38136.log 2023-11-15T03:12:59.206 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.38488.log 2023-11-15T03:12:59.207 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39486.log.gz 2023-11-15T03:12:59.208 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38136.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38136.log.gz 2023-11-15T03:12:59.208 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47441.log 2023-11-15T03:12:59.209 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.2.38488.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.38488.log.gz 2023-11-15T03:12:59.209 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44900.log 2023-11-15T03:12:59.210 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47441.log: gzip 87.8%-5 -- replaced with /var/log/ceph/ceph-client.admin.47441.log.gz--verbose 2023-11-15T03:12:59.210 INFO:teuthology.orchestra.run.smithi121.stderr:-- /var/log/ceph/ceph-client.admin.58336.log 2023-11-15T03:12:59.211 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44900.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44900.log.gz 2023-11-15T03:12:59.211 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55492.log 2023-11-15T03:12:59.212 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59396.log 2023-11-15T03:12:59.213 INFO:teuthology.orchestra.run.smithi121.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58336.log.gz 2023-11-15T03:12:59.213 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55492.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.55492.log.gz 2023-11-15T03:12:59.214 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53233.log 2023-11-15T03:12:59.214 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59396.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59396.log.gz 2023-11-15T03:12:59.215 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40746.log 2023-11-15T03:12:59.216 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47515.log 2023-11-15T03:12:59.216 INFO:teuthology.orchestra.run.smithi121.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.53233.log.gz 2023-11-15T03:12:59.217 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40746.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40746.log.gz 2023-11-15T03:12:59.217 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58500.log 2023-11-15T03:12:59.218 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47515.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47515.log.gz 2023-11-15T03:12:59.218 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59554.log 2023-11-15T03:12:59.219 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58500.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58500.log.gz 2023-11-15T03:12:59.220 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54701.log 2023-11-15T03:12:59.220 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59554.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59554.log.gz 2023-11-15T03:12:59.221 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63514.log 2023-11-15T03:12:59.221 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63087.log 2023-11-15T03:12:59.222 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63514.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54701.log.gz 2023-11-15T03:12:59.223 INFO:teuthology.orchestra.run.smithi121.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63514.log.gz 2023-11-15T03:12:59.223 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39040.log 2023-11-15T03:12:59.224 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39459.log 2023-11-15T03:12:59.224 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63087.log.gz 2023-11-15T03:12:59.225 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60033.log 2023-11-15T03:12:59.226 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39040.log.gz 2023-11-15T03:12:59.226 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39459.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39459.log.gz 2023-11-15T03:12:59.227 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61085.log 2023-11-15T03:12:59.227 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60033.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60033.log.gz 2023-11-15T03:12:59.228 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41705.log 2023-11-15T03:12:59.229 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53031.log 2023-11-15T03:12:59.229 INFO:teuthology.orchestra.run.smithi121.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61085.log.gz 2023-11-15T03:12:59.230 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41705.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41705.log.gz 2023-11-15T03:12:59.230 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38192.log 2023-11-15T03:12:59.232 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57858.log 2023-11-15T03:12:59.232 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53031.log.gz 2023-11-15T03:12:59.233 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38192.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38192.log.gz 2023-11-15T03:12:59.234 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57564.log 2023-11-15T03:12:59.234 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57858.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57858.log.gz 2023-11-15T03:12:59.235 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40142.log 2023-11-15T03:12:59.235 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57564.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57564.log.gz 2023-11-15T03:12:59.236 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57116.log 2023-11-15T03:12:59.237 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40142.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40142.log.gz 2023-11-15T03:12:59.237 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39694.log 2023-11-15T03:12:59.238 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60298.log 2023-11-15T03:12:59.239 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39694.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.57116.log.gz 2023-11-15T03:12:59.239 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39694.log.gz 2023-11-15T03:12:59.240 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45340.log 2023-11-15T03:12:59.241 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60298.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60298.log.gz 2023-11-15T03:12:59.242 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61002.log 2023-11-15T03:12:59.243 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39858.log 2023-11-15T03:12:59.243 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45340.log: /var/log/ceph/ceph-client.admin.61002.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45340.log.gz 2023-11-15T03:12:59.244 INFO:teuthology.orchestra.run.smithi121.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61002.log.gz 2023-11-15T03:12:59.245 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47946.log 2023-11-15T03:12:59.246 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58602.log 2023-11-15T03:12:59.246 INFO:teuthology.orchestra.run.smithi121.stderr: 88.8%/var/log/ceph/ceph-client.admin.47946.log: -- replaced with /var/log/ceph/ceph-client.admin.39858.log.gz 2023-11-15T03:12:59.247 INFO:teuthology.orchestra.run.smithi121.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47946.log.gz 2023-11-15T03:12:59.248 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-15T03:12:59.249 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46033.log 2023-11-15T03:12:59.249 INFO:teuthology.orchestra.run.smithi121.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58602.log.gz 2023-11-15T03:12:59.250 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63141.log 2023-11-15T03:12:59.251 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46033.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46033.log.gz 2023-11-15T03:12:59.252 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41135.log 2023-11-15T03:12:59.252 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41624.log 2023-11-15T03:12:59.253 INFO:teuthology.orchestra.run.smithi121.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63141.log.gz 2023-11-15T03:12:59.254 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43122.log 2023-11-15T03:12:59.254 INFO:teuthology.orchestra.run.smithi121.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41135.log.gz 2023-11-15T03:12:59.255 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33132.log 2023-11-15T03:12:59.255 INFO:teuthology.orchestra.run.smithi121.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41624.log.gz 2023-11-15T03:12:59.256 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43122.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43122.log.gz 2023-11-15T03:12:59.257 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38219.log 2023-11-15T03:12:59.258 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.33132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56081.log 2023-11-15T03:12:59.258 INFO:teuthology.orchestra.run.smithi121.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.33132.log.gz 2023-11-15T03:12:59.259 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43879.log 2023-11-15T03:12:59.259 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56081.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.38219.log.gz 2023-11-15T03:12:59.260 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56081.log.gz 2023-11-15T03:12:59.260 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62369.log 2023-11-15T03:12:59.261 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59240.log 2023-11-15T03:12:59.262 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43879.log.gz 2023-11-15T03:12:59.262 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62369.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62369.log.gz 2023-11-15T03:12:59.263 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52599.log 2023-11-15T03:12:59.264 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47652.log 2023-11-15T03:12:59.265 INFO:teuthology.orchestra.run.smithi121.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59240.log.gz 2023-11-15T03:12:59.265 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52599.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52599.log.gz 2023-11-15T03:12:59.265 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53898.log 2023-11-15T03:12:59.267 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47652.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47652.log.gz 2023-11-15T03:12:59.267 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49671.log 2023-11-15T03:12:59.268 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53898.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.53898.log.gz 2023-11-15T03:12:59.268 INFO:teuthology.orchestra.run.smithi121.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.54164.log 2023-11-15T03:12:59.269 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47083.log 2023-11-15T03:12:59.269 INFO:teuthology.orchestra.run.smithi121.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.49671.log.gz 2023-11-15T03:12:59.270 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54164.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54164.log.gz 2023-11-15T03:12:59.270 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46946.log 2023-11-15T03:12:59.271 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58770.log 2023-11-15T03:12:59.272 INFO:teuthology.orchestra.run.smithi121.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47083.log.gz 2023-11-15T03:12:59.273 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46946.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46946.log.gz 2023-11-15T03:12:59.273 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49551.log 2023-11-15T03:12:59.274 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56613.log 2023-11-15T03:12:59.274 INFO:teuthology.orchestra.run.smithi121.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58770.log.gz 2023-11-15T03:12:59.275 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43411.log 2023-11-15T03:12:59.276 INFO:teuthology.orchestra.run.smithi121.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49551.log.gz 2023-11-15T03:12:59.276 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41443.log 2023-11-15T03:12:59.277 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56613.log.gz 2023-11-15T03:12:59.278 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43411.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43411.log.gz 2023-11-15T03:12:59.278 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40577.log 2023-11-15T03:12:59.279 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57593.log 2023-11-15T03:12:59.279 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41443.log.gz 2023-11-15T03:12:59.280 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59600.log 2023-11-15T03:12:59.281 INFO:teuthology.orchestra.run.smithi121.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40577.log.gz 2023-11-15T03:12:59.282 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41797.log 2023-11-15T03:12:59.282 INFO:teuthology.orchestra.run.smithi121.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.57593.log.gz 2023-11-15T03:12:59.283 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59600.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.59600.log.gz 2023-11-15T03:12:59.283 INFO:teuthology.orchestra.run.smithi121.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.50470.log 2023-11-15T03:12:59.284 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44043.log 2023-11-15T03:12:59.284 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41797.log.gz 2023-11-15T03:12:59.285 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50470.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50470.log.gz 2023-11-15T03:12:59.285 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62291.log 2023-11-15T03:12:59.286 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59812.log 2023-11-15T03:12:59.287 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44043.log.gz 2023-11-15T03:12:59.287 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62291.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62291.log.gz 2023-11-15T03:12:59.288 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39135.log 2023-11-15T03:12:59.289 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.59812.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59812.log.gz 2023-11-15T03:12:59.289 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58446.log 2023-11-15T03:12:59.290 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37677.log 2023-11-15T03:12:59.290 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39135.log.gz 2023-11-15T03:12:59.291 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62264.log 2023-11-15T03:12:59.291 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58446.log.gz 2023-11-15T03:12:59.292 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37677.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37677.log.gz 2023-11-15T03:12:59.292 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54726.log 2023-11-15T03:12:59.293 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55998.log 2023-11-15T03:12:59.294 INFO:teuthology.orchestra.run.smithi121.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62264.log.gz 2023-11-15T03:12:59.294 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54726.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54726.log.gz 2023-11-15T03:12:59.295 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62230.log 2023-11-15T03:12:59.296 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55918.log 2023-11-15T03:12:59.297 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.62230.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62230.log.gz 2023-11-15T03:12:59.297 INFO:teuthology.orchestra.run.smithi121.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.55998.log.gz 2023-11-15T03:12:59.298 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56717.log 2023-11-15T03:12:59.298 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38743.log 2023-11-15T03:12:59.299 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55918.log.gz 2023-11-15T03:12:59.299 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53764.log 2023-11-15T03:12:59.300 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56717.log.gz 2023-11-15T03:12:59.301 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38743.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38743.log.gz 2023-11-15T03:12:59.301 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43651.log 2023-11-15T03:12:59.302 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53764.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53764.log.gz 2023-11-15T03:12:59.303 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62987.log 2023-11-15T03:12:59.304 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39613.log 2023-11-15T03:12:59.304 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43651.log.gz 2023-11-15T03:12:59.305 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62987.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62987.log.gz 2023-11-15T03:12:59.305 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45053.log 2023-11-15T03:12:59.306 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38392.log 2023-11-15T03:12:59.306 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39613.log.gz 2023-11-15T03:12:59.307 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40810.log 2023-11-15T03:12:59.308 INFO:teuthology.orchestra.run.smithi121.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45053.log.gz 2023-11-15T03:12:59.308 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.38392.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.38392.log.gz 2023-11-15T03:12:59.309 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45421.log 2023-11-15T03:12:59.310 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40810.log: gzip 88.2%-5 -- replaced with /var/log/ceph/ceph-client.admin.40810.log.gz--verbose 2023-11-15T03:12:59.310 INFO:teuthology.orchestra.run.smithi121.stderr:-- /var/log/ceph/ceph-client.admin.37272.log 2023-11-15T03:12:59.311 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42985.log 2023-11-15T03:12:59.311 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45421.log.gz 2023-11-15T03:12:59.312 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62396.log 2023-11-15T03:12:59.313 INFO:teuthology.orchestra.run.smithi121.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37272.log.gz 2023-11-15T03:12:59.314 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56982.log 2023-11-15T03:12:59.314 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42985.log.gz 2023-11-15T03:12:59.315 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40169.log 2023-11-15T03:12:59.315 INFO:teuthology.orchestra.run.smithi121.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.62396.log.gz 2023-11-15T03:12:59.316 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56982.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56982.log.gz 2023-11-15T03:12:59.316 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49319.log 2023-11-15T03:12:59.318 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40169.log: gzip -5 --verbose -- 88.7%/var/log/ceph/ceph-client.admin.61435.log 2023-11-15T03:12:59.318 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40169.log.gz 2023-11-15T03:12:59.319 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60191.log 2023-11-15T03:12:59.319 INFO:teuthology.orchestra.run.smithi121.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49319.log.gz 2023-11-15T03:12:59.320 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37893.log 2023-11-15T03:12:59.320 INFO:teuthology.orchestra.run.smithi121.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61435.log.gz 2023-11-15T03:12:59.321 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60191.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.60191.log.gz 2023-11-15T03:12:59.321 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39804.log 2023-11-15T03:12:59.322 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37893.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37893.log.gz 2023-11-15T03:12:59.323 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57956.log 2023-11-15T03:12:59.324 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45825.log 2023-11-15T03:12:59.324 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39804.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39804.log.gz 2023-11-15T03:12:59.325 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57956.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57956.log.gz 2023-11-15T03:12:59.326 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48967.log 2023-11-15T03:12:59.328 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45825.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45825.log.gz 2023-11-15T03:12:59.328 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49407.log 2023-11-15T03:12:59.329 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41281.log 2023-11-15T03:12:59.329 INFO:teuthology.orchestra.run.smithi121.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48967.log.gz 2023-11-15T03:12:59.331 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49407.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49407.log.gz 2023-11-15T03:12:59.331 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40642.log 2023-11-15T03:12:59.331 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41281.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41281.log.gz 2023-11-15T03:12:59.332 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44873.log 2023-11-15T03:12:59.334 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48176.log 2023-11-15T03:12:59.335 INFO:teuthology.orchestra.run.smithi121.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40642.log.gz 2023-11-15T03:12:59.335 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44873.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44873.log.gz 2023-11-15T03:12:59.335 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48311.log 2023-11-15T03:12:59.336 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46523.log 2023-11-15T03:12:59.336 INFO:teuthology.orchestra.run.smithi121.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48176.log.gz 2023-11-15T03:12:59.340 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62654.log 2023-11-15T03:12:59.340 INFO:teuthology.orchestra.run.smithi121.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.48311.log.gz 2023-11-15T03:12:59.341 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37974.log 2023-11-15T03:12:59.341 INFO:teuthology.orchestra.run.smithi121.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.46523.log.gz 2023-11-15T03:12:59.341 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63865.log 2023-11-15T03:12:59.341 INFO:teuthology.orchestra.run.smithi121.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.62654.log.gz 2023-11-15T03:12:59.342 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37974.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37974.log.gz 2023-11-15T03:12:59.342 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59032.log 2023-11-15T03:12:59.344 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63865.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63865.log.gz 2023-11-15T03:12:59.344 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42593.log 2023-11-15T03:12:59.344 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48659.log 2023-11-15T03:12:59.345 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59032.log: /var/log/ceph/ceph-client.admin.42593.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59032.log.gz 2023-11-15T03:12:59.345 INFO:teuthology.orchestra.run.smithi121.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42593.log.gz 2023-11-15T03:12:59.345 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44817.log 2023-11-15T03:12:59.347 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48659.log: /var/log/ceph/ceph-client.admin.47171.log 2023-11-15T03:12:59.347 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44817.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48659.log.gz 2023-11-15T03:12:59.347 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44817.log.gz 2023-11-15T03:12:59.347 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48462.log 2023-11-15T03:12:59.348 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58161.log 2023-11-15T03:12:59.348 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47171.log.gz 2023-11-15T03:12:59.349 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48462.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48462.log.gz 2023-11-15T03:12:59.349 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48548.log 2023-11-15T03:12:59.350 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58161.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58161.log.gz 2023-11-15T03:12:59.351 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39432.log 2023-11-15T03:12:59.353 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46772.log 2023-11-15T03:12:59.353 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48548.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48548.log.gz 2023-11-15T03:12:59.353 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39432.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39432.log.gz 2023-11-15T03:12:59.354 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49253.log 2023-11-15T03:12:59.354 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46772.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46772.log.gz 2023-11-15T03:12:59.355 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54644.log 2023-11-15T03:12:59.355 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49253.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49253.log.gz 2023-11-15T03:12:59.356 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45529.log 2023-11-15T03:12:59.357 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51427.log 2023-11-15T03:12:59.357 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54644.log.gz 2023-11-15T03:12:59.358 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58244.log 2023-11-15T03:12:59.358 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45529.log.gz 2023-11-15T03:12:59.359 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46253.log 2023-11-15T03:12:59.359 INFO:teuthology.orchestra.run.smithi121.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51427.log.gz 2023-11-15T03:12:59.360 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51150.log 2023-11-15T03:12:59.360 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58244.log.gz 2023-11-15T03:12:59.361 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43678.log 2023-11-15T03:12:59.361 INFO:teuthology.orchestra.run.smithi121.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46253.log.gz 2023-11-15T03:12:59.362 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.51150.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51150.log.gz 2023-11-15T03:12:59.362 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37353.log 2023-11-15T03:12:59.364 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46630.log 2023-11-15T03:12:59.364 INFO:teuthology.orchestra.run.smithi121.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43678.log.gz 2023-11-15T03:12:59.365 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38109.log 2023-11-15T03:12:59.365 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37353.log.gz 2023-11-15T03:12:59.366 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46630.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46630.log.gz 2023-11-15T03:12:59.366 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55735.log 2023-11-15T03:12:59.367 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45925.log 2023-11-15T03:12:59.367 INFO:teuthology.orchestra.run.smithi121.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38109.log.gz 2023-11-15T03:12:59.368 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60118.log 2023-11-15T03:12:59.369 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45925.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.55735.log.gz 2023-11-15T03:12:59.369 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45925.log.gz 2023-11-15T03:12:59.370 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39831.log 2023-11-15T03:12:59.371 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60118.log: gzip -5 --verbose -- 87.4%/var/log/ceph/ceph-client.admin.51538.log -- replaced with /var/log/ceph/ceph-client.admin.60118.log.gz 2023-11-15T03:12:59.371 INFO:teuthology.orchestra.run.smithi121.stderr: 2023-11-15T03:12:59.372 INFO:teuthology.orchestra.run.smithi190.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-15T03:12:59.372 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59221.log 2023-11-15T03:12:59.373 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39831.log.gz 2023-11-15T03:12:59.373 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51538.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51538.log.gz 2023-11-15T03:12:59.374 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42460.log 2023-11-15T03:12:59.375 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.59221.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59221.log.gz 2023-11-15T03:12:59.375 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38527.log 2023-11-15T03:12:59.376 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.42460.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42460.log.gz 2023-11-15T03:12:59.376 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47892.log 2023-11-15T03:12:59.377 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38527.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38527.log.gz-5 2023-11-15T03:12:59.377 INFO:teuthology.orchestra.run.smithi121.stderr:--verbose -- /var/log/ceph/ceph-client.admin.44016.log 2023-11-15T03:12:59.378 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53187.log 2023-11-15T03:12:59.379 INFO:teuthology.orchestra.run.smithi121.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47892.log.gz 2023-11-15T03:12:59.379 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44016.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44016.log.gz 2023-11-15T03:12:59.380 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41254.log 2023-11-15T03:12:59.381 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55782.log 2023-11-15T03:12:59.381 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53187.log.gz 2023-11-15T03:12:59.382 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41254.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41254.log.gz 2023-11-15T03:12:59.382 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37218.log 2023-11-15T03:12:59.383 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48004.log 2023-11-15T03:12:59.383 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55782.log.gz 2023-11-15T03:12:59.384 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44477.log 2023-11-15T03:12:59.384 INFO:teuthology.orchestra.run.smithi121.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37218.log.gz 2023-11-15T03:12:59.386 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50543.log 2023-11-15T03:12:59.386 INFO:teuthology.orchestra.run.smithi121.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48004.log.gz 2023-11-15T03:12:59.389 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49829.log 2023-11-15T03:12:59.389 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44477.log.gz 2023-11-15T03:12:59.389 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50040.log 2023-11-15T03:12:59.389 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50543.log.gz 2023-11-15T03:12:59.390 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49829.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49829.log.gz 2023-11-15T03:12:59.391 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63757.log 2023-11-15T03:12:59.391 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42175.log 2023-11-15T03:12:59.391 INFO:teuthology.orchestra.run.smithi121.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50040.log.gz 2023-11-15T03:12:59.392 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63757.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63757.log.gz 2023-11-15T03:12:59.392 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33001.log 2023-11-15T03:12:59.393 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56853.log 2023-11-15T03:12:59.393 INFO:teuthology.orchestra.run.smithi121.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42175.log.gz 2023-11-15T03:12:59.402 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.33001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55891.log 2023-11-15T03:12:59.402 INFO:teuthology.orchestra.run.smithi121.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.33001.log.gz 2023-11-15T03:12:59.402 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36890.log 2023-11-15T03:12:59.403 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55891.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.56853.log.gz 2023-11-15T03:12:59.403 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55891.log.gz 2023-11-15T03:12:59.403 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43176.log 2023-11-15T03:12:59.403 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44546.log 2023-11-15T03:12:59.403 INFO:teuthology.orchestra.run.smithi121.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36890.log.gz 2023-11-15T03:12:59.404 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55027.log 2023-11-15T03:12:59.404 INFO:teuthology.orchestra.run.smithi121.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43176.log.gz 2023-11-15T03:12:59.404 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44546.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44546.log.gz 2023-11-15T03:12:59.404 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57357.log 2023-11-15T03:12:59.404 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52070.log 2023-11-15T03:12:59.405 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55027.log.gz 2023-11-15T03:12:59.406 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57357.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57357.log.gz 2023-11-15T03:12:59.406 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39189.log 2023-11-15T03:12:59.407 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36863.log 2023-11-15T03:12:59.407 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52070.log.gz 2023-11-15T03:12:59.407 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39189.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39189.log.gz 2023-11-15T03:12:59.407 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54294.log 2023-11-15T03:12:59.408 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55446.log 2023-11-15T03:12:59.409 INFO:teuthology.orchestra.run.smithi121.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36863.log.gz 2023-11-15T03:12:59.409 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54294.log: gzip -5 --verbose 89.7%-- -- replaced with /var/log/ceph/ceph-client.admin.54294.log.gz 2023-11-15T03:12:59.409 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43624.log 2023-11-15T03:12:59.410 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57009.log 2023-11-15T03:12:59.410 INFO:teuthology.orchestra.run.smithi121.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55446.log.gz 2023-11-15T03:12:59.411 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43624.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43624.log.gz 2023-11-15T03:12:59.411 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63352.log 2023-11-15T03:12:59.412 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54918.log 2023-11-15T03:12:59.412 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57009.log.gz 2023-11-15T03:12:59.413 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63352.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63352.log.gz 2023-11-15T03:12:59.413 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36917.log 2023-11-15T03:12:59.414 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44070.log 2023-11-15T03:12:59.415 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54918.log.gz 2023-11-15T03:12:59.415 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36917.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36917.log.gz 2023-11-15T03:12:59.416 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60894.log 2023-11-15T03:12:59.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44070.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44070.log.gz 2023-11-15T03:12:59.417 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60145.log 2023-11-15T03:12:59.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55290.log 2023-11-15T03:12:59.418 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60894.log.gz 2023-11-15T03:12:59.420 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60145.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60145.log.gz 2023-11-15T03:12:59.420 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44629.log 2023-11-15T03:12:59.421 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55290.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55290.log.gz 2023-11-15T03:12:59.421 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49938.log 2023-11-15T03:12:59.422 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60618.log 2023-11-15T03:12:59.424 INFO:teuthology.orchestra.run.smithi121.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44629.log.gz 2023-11-15T03:12:59.424 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49938.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49938.log.gz 2023-11-15T03:12:59.425 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45685.log 2023-11-15T03:12:59.425 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60618.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60618.log.gz 2023-11-15T03:12:59.426 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55229.log 2023-11-15T03:12:59.427 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45685.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45685.log.gz 2023-11-15T03:12:59.427 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60454.log 2023-11-15T03:12:59.428 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63649.log 2023-11-15T03:12:59.428 INFO:teuthology.orchestra.run.smithi121.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55229.log.gz 2023-11-15T03:12:59.429 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45286.log 2023-11-15T03:12:59.429 INFO:teuthology.orchestra.run.smithi121.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.60454.log.gz 2023-11-15T03:12:59.430 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63649.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63649.log.gz 2023-11-15T03:12:59.430 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57065.log 2023-11-15T03:12:59.431 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51912.log 2023-11-15T03:12:59.432 INFO:teuthology.orchestra.run.smithi121.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45286.log.gz 2023-11-15T03:12:59.432 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57065.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57065.log.gz 2023-11-15T03:12:59.433 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57036.log 2023-11-15T03:12:59.434 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51912.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51912.log.gz 2023-11-15T03:12:59.434 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41052.log 2023-11-15T03:12:59.436 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54835.log 2023-11-15T03:12:59.436 INFO:teuthology.orchestra.run.smithi121.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57036.log.gz 2023-11-15T03:12:59.437 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41052.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41052.log.gz 2023-11-15T03:12:59.437 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59285.log 2023-11-15T03:12:59.438 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63379.log 2023-11-15T03:12:59.438 INFO:teuthology.orchestra.run.smithi121.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54835.log.gz 2023-11-15T03:12:59.439 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59285.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59285.log.gz 2023-11-15T03:12:59.439 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57727.log 2023-11-15T03:12:59.440 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42724.log 2023-11-15T03:12:59.440 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63379.log.gz 2023-11-15T03:12:59.441 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57727.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57727.log.gz 2023-11-15T03:12:59.441 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52859.log 2023-11-15T03:12:59.442 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58985.log 2023-11-15T03:12:59.443 INFO:teuthology.orchestra.run.smithi121.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42724.log.gz 2023-11-15T03:12:59.443 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.52859.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52859.log.gz 2023-11-15T03:12:59.444 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38878.log 2023-11-15T03:12:59.444 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46571.log 2023-11-15T03:12:59.445 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38878.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38878.log.gz 2023-11-15T03:12:59.446 INFO:teuthology.orchestra.run.smithi121.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.58985.log.gz 2023-11-15T03:12:59.447 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47414.log 2023-11-15T03:12:59.447 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50169.log 2023-11-15T03:12:59.448 INFO:teuthology.orchestra.run.smithi121.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46571.log.gz 2023-11-15T03:12:59.448 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52672.log 2023-11-15T03:12:59.449 INFO:teuthology.orchestra.run.smithi121.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47414.log.gz 2023-11-15T03:12:59.449 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48230.log 2023-11-15T03:12:59.450 INFO:teuthology.orchestra.run.smithi121.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50169.log.gz 2023-11-15T03:12:59.450 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44509.log 2023-11-15T03:12:59.451 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52672.log.gz 2023-11-15T03:12:59.452 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48230.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48230.log.gz 2023-11-15T03:12:59.452 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38581.log 2023-11-15T03:12:59.453 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44509.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44509.log.gz 2023-11-15T03:12:59.453 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58099.log 2023-11-15T03:12:59.454 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46006.log 2023-11-15T03:12:59.454 INFO:teuthology.orchestra.run.smithi121.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38581.log.gz 2023-11-15T03:12:59.456 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50834.log 2023-11-15T03:12:59.456 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58099.log.gz 2023-11-15T03:12:59.457 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57138.log 2023-11-15T03:12:59.457 INFO:teuthology.orchestra.run.smithi121.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46006.log.gz 2023-11-15T03:12:59.458 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44322.log 2023-11-15T03:12:59.458 INFO:teuthology.orchestra.run.smithi121.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50834.log.gz 2023-11-15T03:12:59.459 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40698.log 2023-11-15T03:12:59.459 INFO:teuthology.orchestra.run.smithi121.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.57138.log.gz 2023-11-15T03:12:59.460 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51379.log 2023-11-15T03:12:59.461 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40698.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.44322.log.gz 2023-11-15T03:12:59.461 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61379.log 2023-11-15T03:12:59.462 INFO:teuthology.orchestra.run.smithi121.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.40698.log.gz 2023-11-15T03:12:59.462 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56669.log 2023-11-15T03:12:59.463 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61379.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.51379.log.gz 2023-11-15T03:12:59.464 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61379.log.gz 2023-11-15T03:12:59.464 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40115.log 2023-11-15T03:12:59.465 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37596.log 2023-11-15T03:12:59.465 INFO:teuthology.orchestra.run.smithi121.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56669.log.gz 2023-11-15T03:12:59.466 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40115.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40115.log.gz 2023-11-15T03:12:59.467 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47279.log 2023-11-15T03:12:59.468 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37596.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37596.log.gz 2023-11-15T03:12:59.468 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37650.log 2023-11-15T03:12:59.469 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52094.log 2023-11-15T03:12:59.469 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47279.log.gz 2023-11-15T03:12:59.470 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44573.log 2023-11-15T03:12:59.470 INFO:teuthology.orchestra.run.smithi121.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37650.log.gz 2023-11-15T03:12:59.471 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59100.log 2023-11-15T03:12:59.472 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44573.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.52094.log.gz 2023-11-15T03:12:59.472 INFO:teuthology.orchestra.run.smithi121.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44573.log.gz 2023-11-15T03:12:59.473 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49451.log 2023-11-15T03:12:59.474 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59662.log 2023-11-15T03:12:59.474 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59100.log.gz 2023-11-15T03:12:59.475 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49209.log 2023-11-15T03:12:59.475 INFO:teuthology.orchestra.run.smithi121.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49451.log.gz 2023-11-15T03:12:59.476 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59662.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59662.log.gz 2023-11-15T03:12:59.476 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52765.log 2023-11-15T03:12:59.478 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49209.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49209.log.gz 2023-11-15T03:12:59.478 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47198.log 2023-11-15T03:12:59.479 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52709.log 2023-11-15T03:12:59.480 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52765.log: /var/log/ceph/ceph-client.admin.47198.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47198.log.gz 2023-11-15T03:12:59.480 INFO:teuthology.orchestra.run.smithi121.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.52765.log.gz 2023-11-15T03:12:59.480 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36809.log 2023-11-15T03:12:59.481 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42231.log 2023-11-15T03:12:59.482 INFO:teuthology.orchestra.run.smithi121.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52709.log.gz 2023-11-15T03:12:59.483 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36809.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36809.log.gz 2023-11-15T03:12:59.483 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56642.log 2023-11-15T03:12:59.484 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56045.log 2023-11-15T03:12:59.484 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42231.log.gz 2023-11-15T03:12:59.485 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56642.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56642.log.gz 2023-11-15T03:12:59.485 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51585.log 2023-11-15T03:12:59.486 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52489.log 2023-11-15T03:12:59.486 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56045.log.gz 2023-11-15T03:12:59.487 INFO:teuthology.orchestra.run.smithi190.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-15T03:12:59.488 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51585.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.51585.log.gz 2023-11-15T03:12:59.488 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42028.log 2023-11-15T03:12:59.489 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50672.log 2023-11-15T03:12:59.489 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52489.log.gz 2023-11-15T03:12:59.490 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42028.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42028.log.gz 2023-11-15T03:12:59.490 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37704.log 2023-11-15T03:12:59.491 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38905.log 2023-11-15T03:12:59.491 INFO:teuthology.orchestra.run.smithi121.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.50672.log.gz 2023-11-15T03:12:59.492 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37704.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37704.log.gz 2023-11-15T03:12:59.492 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37110.log 2023-11-15T03:12:59.493 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61171.log 2023-11-15T03:12:59.493 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38905.log.gz 2023-11-15T03:12:59.494 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37110.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37110.log.gz 2023-11-15T03:12:59.494 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60921.log 2023-11-15T03:12:59.495 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51876.log 2023-11-15T03:12:59.496 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61171.log.gz 2023-11-15T03:12:59.496 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60921.log: gzip 87.6%-5 -- replaced with /var/log/ceph/ceph-client.admin.60921.log.gz--verbose 2023-11-15T03:12:59.496 INFO:teuthology.orchestra.run.smithi121.stderr:-- /var/log/ceph/ceph-client.admin.49099.log 2023-11-15T03:12:59.497 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58816.log 2023-11-15T03:12:59.498 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51876.log.gz 2023-11-15T03:12:59.498 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49099.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.49099.log.gz 2023-11-15T03:12:59.499 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52572.log 2023-11-15T03:12:59.499 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.58816.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58816.log.gz 2023-11-15T03:12:59.500 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50807.log 2023-11-15T03:12:59.501 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46388.log 2023-11-15T03:12:59.501 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52572.log.gz 2023-11-15T03:12:59.502 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52178.log 2023-11-15T03:12:59.502 INFO:teuthology.orchestra.run.smithi121.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50807.log.gz 2023-11-15T03:12:59.503 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46388.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46388.log.gz 2023-11-15T03:12:59.503 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42555.log 2023-11-15T03:12:59.504 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43852.log 2023-11-15T03:12:59.504 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52178.log.gz 2023-11-15T03:12:59.505 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42555.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42555.log.gz 2023-11-15T03:12:59.506 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62825.log 2023-11-15T03:12:59.507 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57775.log 2023-11-15T03:12:59.507 INFO:teuthology.orchestra.run.smithi121.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43852.log.gz 2023-11-15T03:12:59.508 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62825.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62825.log.gz 2023-11-15T03:12:59.508 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52623.log 2023-11-15T03:12:59.510 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40061.log 2023-11-15T03:12:59.510 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57775.log.gz 2023-11-15T03:12:59.511 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55947.log 2023-11-15T03:12:59.511 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40061.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40061.log.gz 2023-11-15T03:12:59.512 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-15T03:12:59.512 INFO:teuthology.orchestra.run.smithi121.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.52623.log.gz 2023-11-15T03:12:59.513 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42625.log 2023-11-15T03:12:59.513 INFO:teuthology.orchestra.run.smithi121.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55947.log.gz 2023-11-15T03:12:59.514 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43761.log 2023-11-15T03:12:59.515 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42625.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42625.log.gz 2023-11-15T03:12:59.516 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43465.log 2023-11-15T03:12:59.517 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56529.log 2023-11-15T03:12:59.517 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43761.log: /var/log/ceph/ceph-client.admin.43465.log: 87.8% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-15T03:12:59.518 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43761.log.gz 2023-11-15T03:12:59.519 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43465.log.gz 2023-11-15T03:12:59.519 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51829.log 2023-11-15T03:12:59.519 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48879.log 2023-11-15T03:12:59.521 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51829.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.56529.log.gz 2023-11-15T03:12:59.521 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59985.log 2023-11-15T03:12:59.522 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46089.log 2023-11-15T03:12:59.523 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48879.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.51829.log.gz 2023-11-15T03:12:59.523 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59985.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48879.log.gz 2023-11-15T03:12:59.524 INFO:teuthology.orchestra.run.smithi121.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59985.log.gz 2023-11-15T03:12:59.524 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40005.log 2023-11-15T03:12:59.525 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48703.log 2023-11-15T03:12:59.525 INFO:teuthology.orchestra.run.smithi121.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46089.log.gz 2023-11-15T03:12:59.526 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52116.log 2023-11-15T03:12:59.526 INFO:teuthology.orchestra.run.smithi121.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40005.log.gz 2023-11-15T03:12:59.527 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48703.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48703.log.gz 2023-11-15T03:12:59.527 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38608.log 2023-11-15T03:12:59.528 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42840.log 2023-11-15T03:12:59.529 INFO:teuthology.orchestra.run.smithi121.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.52116.log.gz 2023-11-15T03:12:59.529 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38082.log 2023-11-15T03:12:59.530 INFO:teuthology.orchestra.run.smithi121.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38608.log.gz 2023-11-15T03:12:59.531 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42840.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42840.log.gz 2023-11-15T03:12:59.531 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58398.log 2023-11-15T03:12:59.532 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41562.log 2023-11-15T03:12:59.532 INFO:teuthology.orchestra.run.smithi121.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38082.log.gz 2023-11-15T03:12:59.533 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63730.log 2023-11-15T03:12:59.533 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58398.log.gz 2023-11-15T03:12:59.534 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41562.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41562.log.gz 2023-11-15T03:12:59.535 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53981.log 2023-11-15T03:12:59.536 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63730.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-15T03:12:59.536 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63730.log.gz 2023-11-15T03:12:59.537 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63676.log 2023-11-15T03:12:59.537 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53981.log.gz 2023-11-15T03:12:59.538 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48526.log 2023-11-15T03:12:59.539 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63676.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63676.log.gz 2023-11-15T03:12:59.540 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43357.log 2023-11-15T03:12:59.541 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46657.log 2023-11-15T03:12:59.542 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43357.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.48526.log.gz 2023-11-15T03:12:59.543 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46442.log 2023-11-15T03:12:59.543 INFO:teuthology.orchestra.run.smithi121.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43357.log.gz 2023-11-15T03:12:59.544 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45556.log 2023-11-15T03:12:59.544 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46657.log.gz 2023-11-15T03:12:59.545 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46442.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46442.log.gz 2023-11-15T03:12:59.546 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43497.log 2023-11-15T03:12:59.546 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.45556.log: -- /var/log/ceph/ceph-client.admin.43203.log 2023-11-15T03:12:59.547 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45556.log.gz 2023-11-15T03:12:59.548 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43497.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43497.log.gz 2023-11-15T03:12:59.548 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55711.log 2023-11-15T03:12:59.549 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51749.log 2023-11-15T03:12:59.549 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43203.log.gz 2023-11-15T03:12:59.550 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55711.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55711.log.gz 2023-11-15T03:12:59.551 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59310.log 2023-11-15T03:12:59.551 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63813.log 2023-11-15T03:12:59.552 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51749.log.gz 2023-11-15T03:12:59.553 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44790.log 2023-11-15T03:12:59.553 INFO:teuthology.orchestra.run.smithi121.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.59310.log.gz 2023-11-15T03:12:59.554 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49529.log 2023-11-15T03:12:59.555 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63813.log.gz 2023-11-15T03:12:59.556 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44790.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44790.log.gz 2023-11-15T03:12:59.556 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48486.log 2023-11-15T03:12:59.557 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43149.log 2023-11-15T03:12:59.559 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36575.log 2023-11-15T03:12:59.559 INFO:teuthology.orchestra.run.smithi121.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.49529.log.gz 2023-11-15T03:12:59.560 INFO:teuthology.orchestra.run.smithi121.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.48486.log.gz 2023-11-15T03:12:59.560 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50863.log 2023-11-15T03:12:59.561 INFO:teuthology.orchestra.run.smithi121.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43149.log.gz 2023-11-15T03:12:59.561 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36575.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36575.log.gz 2023-11-15T03:12:59.562 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50650.log 2023-11-15T03:12:59.562 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39013.log 2023-11-15T03:12:59.563 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50863.log: /var/log/ceph/ceph-client.admin.50650.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50863.log.gz 2023-11-15T03:12:59.564 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57995.log 2023-11-15T03:12:59.565 INFO:teuthology.orchestra.run.smithi121.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.50650.log.gz 2023-11-15T03:12:59.566 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61271.log 2023-11-15T03:12:59.566 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39013.log.gz 2023-11-15T03:12:59.567 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57995.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57995.log.gz 2023-11-15T03:12:59.567 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47144.log 2023-11-15T03:12:59.568 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59901.log 2023-11-15T03:12:59.568 INFO:teuthology.orchestra.run.smithi121.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61271.log.gz 2023-11-15T03:12:59.569 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47144.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47144.log.gz 2023-11-15T03:12:59.569 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62444.log 2023-11-15T03:12:59.570 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55336.log 2023-11-15T03:12:59.572 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62444.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62444.log.gz-5 2023-11-15T03:12:59.572 INFO:teuthology.orchestra.run.smithi121.stderr:--verbose -- /var/log/ceph/ceph-client.admin.60758.log 2023-11-15T03:12:59.572 INFO:teuthology.orchestra.run.smithi121.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.59901.log.gz 2023-11-15T03:12:59.573 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58915.log 2023-11-15T03:12:59.573 INFO:teuthology.orchestra.run.smithi121.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55336.log.gz 2023-11-15T03:12:59.574 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60758.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60758.log.gz 2023-11-15T03:12:59.574 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37407.log 2023-11-15T03:12:59.575 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56828.log 2023-11-15T03:12:59.576 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58915.log.gz 2023-11-15T03:12:59.577 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37407.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37407.log.gz 2023-11-15T03:12:59.577 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57301.log 2023-11-15T03:12:59.578 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41164.log 2023-11-15T03:12:59.579 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56828.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56828.log.gz 2023-11-15T03:12:59.579 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57301.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57301.log.gz 2023-11-15T03:12:59.579 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49165.log 2023-11-15T03:12:59.580 INFO:teuthology.orchestra.run.smithi190.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-15T03:12:59.581 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41732.log 2023-11-15T03:12:59.581 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49165.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.41164.log.gz 2023-11-15T03:12:59.581 INFO:teuthology.orchestra.run.smithi121.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.49165.log.gz 2023-11-15T03:12:59.582 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55626.log 2023-11-15T03:12:59.583 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59831.log 2023-11-15T03:12:59.583 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41732.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41732.log.gz 2023-11-15T03:12:59.584 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55626.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55626.log.gz 2023-11-15T03:12:59.584 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57691.log 2023-11-15T03:12:59.585 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47744.log 2023-11-15T03:12:59.586 INFO:teuthology.orchestra.run.smithi121.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59831.log.gz 2023-11-15T03:12:59.587 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57691.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57691.log.gz 2023-11-15T03:12:59.587 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57403.log 2023-11-15T03:12:59.588 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61865.log 2023-11-15T03:12:59.588 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47744.log.gz 2023-11-15T03:12:59.589 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57403.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57403.log.gz 2023-11-15T03:12:59.589 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37758.log 2023-11-15T03:12:59.590 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.61865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49011.log 2023-11-15T03:12:59.590 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61865.log.gz 2023-11-15T03:12:59.591 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37758.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37758.log.gz 2023-11-15T03:12:59.592 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52443.log 2023-11-15T03:12:59.592 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39270.log 2023-11-15T03:12:59.593 INFO:teuthology.orchestra.run.smithi121.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49011.log.gz 2023-11-15T03:12:59.594 INFO:teuthology.orchestra.run.smithi190.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-15T03:12:59.594 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52443.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52443.log.gz 2023-11-15T03:12:59.594 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48615.log 2023-11-15T03:12:59.595 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60432.log 2023-11-15T03:12:59.596 INFO:teuthology.orchestra.run.smithi121.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39270.log.gz 2023-11-15T03:12:59.596 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48615.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48615.log.gz 2023-11-15T03:12:59.596 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53370.log 2023-11-15T03:12:59.597 INFO:teuthology.orchestra.run.smithi190.stderr: 2023-11-15T03:12:59.597 INFO:teuthology.orchestra.run.smithi190.stderr:real 0m1.069s 2023-11-15T03:12:59.598 INFO:teuthology.orchestra.run.smithi190.stderr:user 0m4.290s 2023-11-15T03:12:59.598 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m0.497s 2023-11-15T03:12:59.599 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51323.log /var/log/ceph/ceph-client.admin.60432.log: 2023-11-15T03:12:59.599 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53370.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53370.log.gz 2023-11-15T03:12:59.599 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53211.log 2023-11-15T03:12:59.603 INFO:teuthology.orchestra.run.smithi121.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.60432.log.gz 2023-11-15T03:12:59.603 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42202.log 2023-11-15T03:12:59.603 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51323.log.gz 2023-11-15T03:12:59.603 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43935.log 2023-11-15T03:12:59.604 INFO:teuthology.orchestra.run.smithi121.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.53211.log.gz 2023-11-15T03:12:59.604 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56263.log 2023-11-15T03:12:59.605 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42202.log.gz 2023-11-15T03:12:59.605 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43935.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43935.log.gz 2023-11-15T03:12:59.605 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62109.log 2023-11-15T03:12:59.606 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61771.log 2023-11-15T03:12:59.607 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56263.log: /var/log/ceph/ceph-client.admin.62109.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62109.log.gz 2023-11-15T03:12:59.607 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49077.log 2023-11-15T03:12:59.608 INFO:teuthology.orchestra.run.smithi121.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.56263.log.gz 2023-11-15T03:12:59.608 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47252.log 2023-11-15T03:12:59.609 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49077.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.61771.log.gz 2023-11-15T03:12:59.610 INFO:teuthology.orchestra.run.smithi121.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49077.log.gz 2023-11-15T03:12:59.610 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62498.log 2023-11-15T03:12:59.611 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49385.log 2023-11-15T03:12:59.611 INFO:teuthology.orchestra.run.smithi121.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47252.log.gz 2023-11-15T03:12:59.612 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62498.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28112.log 2023-11-15T03:12:59.613 INFO:teuthology.orchestra.run.smithi121.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.62498.log.gz 2023-11-15T03:12:59.614 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45448.log 2023-11-15T03:12:59.614 INFO:teuthology.orchestra.run.smithi121.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49385.log.gz 2023-11-15T03:12:59.615 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph.tmp-client.admin.28112.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28112.log.gz 2023-11-15T03:12:59.615 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54027.log 2023-11-15T03:12:59.616 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54989.log 2023-11-15T03:12:59.617 INFO:teuthology.orchestra.run.smithi121.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45448.log.gz 2023-11-15T03:12:59.618 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54027.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54027.log.gz 2023-11-15T03:12:59.618 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46909.log 2023-11-15T03:12:59.619 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36782.log 2023-11-15T03:12:59.619 INFO:teuthology.orchestra.run.smithi121.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54989.log.gz 2023-11-15T03:12:59.620 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46909.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46909.log.gz 2023-11-15T03:12:59.620 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33258.log 2023-11-15T03:12:59.621 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60408.log 2023-11-15T03:12:59.622 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36782.log.gz 2023-11-15T03:12:59.624 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.33258.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33258.log.gz 2023-11-15T03:12:59.624 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37623.log 2023-11-15T03:12:59.625 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60408.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60408.log.gz 2023-11-15T03:12:59.626 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43825.log 2023-11-15T03:12:59.627 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42931.log 2023-11-15T03:12:59.628 INFO:teuthology.orchestra.run.smithi121.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37623.log.gz 2023-11-15T03:12:59.628 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43825.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43825.log.gz 2023-11-15T03:12:59.629 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47975.log 2023-11-15T03:12:59.630 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46617.log 2023-11-15T03:12:59.630 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42931.log.gz 2023-11-15T03:12:59.631 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47975.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47975.log.gz 2023-11-15T03:12:59.632 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38001.log 2023-11-15T03:12:59.633 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.46617.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46617.log.gz 2023-11-15T03:12:59.633 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48039.log 2023-11-15T03:12:59.634 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38001.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38001.log.gz 2023-11-15T03:12:59.635 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43238.log 2023-11-15T03:12:59.636 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48039.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48039.log.gz 2023-11-15T03:12:59.636 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56936.log 2023-11-15T03:12:59.638 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43238.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43238.log.gz 2023-11-15T03:12:59.638 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48257.log 2023-11-15T03:12:59.639 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56936.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56936.log.gz 2023-11-15T03:12:59.639 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41389.log 2023-11-15T03:12:59.640 INFO:teuthology.orchestra.run.smithi121.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-15T03:12:59.641 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47487.log 2023-11-15T03:12:59.641 INFO:teuthology.orchestra.run.smithi121.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48257.log.gz 2023-11-15T03:12:59.642 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41389.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41389.log.gz 2023-11-15T03:12:59.642 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61029.log 2023-11-15T03:12:59.643 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47487.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47487.log.gz 2023-11-15T03:12:59.643 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42904.log 2023-11-15T03:12:59.644 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54271.log 2023-11-15T03:12:59.645 INFO:teuthology.orchestra.run.smithi121.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61029.log.gz 2023-11-15T03:12:59.646 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42904.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42904.log.gz 2023-11-15T03:12:59.646 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51473.log 2023-11-15T03:12:59.646 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63703.log 2023-11-15T03:12:59.647 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.51473.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51473.log.gz 2023-11-15T03:12:59.648 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39777.log 2023-11-15T03:12:59.648 INFO:teuthology.orchestra.run.smithi121.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.54271.log.gz 2023-11-15T03:12:59.649 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37137.log 2023-11-15T03:12:59.649 INFO:teuthology.orchestra.run.smithi121.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.63703.log.gz 2023-11-15T03:12:59.650 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42366.log 2023-11-15T03:12:59.651 INFO:teuthology.orchestra.run.smithi121.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39777.log.gz 2023-11-15T03:12:59.652 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37137.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37137.log.gz 2023-11-15T03:12:59.652 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62771.log 2023-11-15T03:12:59.652 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47811.log 2023-11-15T03:12:59.653 INFO:teuthology.orchestra.run.smithi121.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.42366.log.gz 2023-11-15T03:12:59.654 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62771.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62771.log.gz 2023-11-15T03:12:59.654 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44602.log 2023-11-15T03:12:59.655 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59498.log 2023-11-15T03:12:59.655 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47811.log.gz 2023-11-15T03:12:59.656 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45952.log 2023-11-15T03:12:59.656 INFO:teuthology.orchestra.run.smithi121.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44602.log.gz 2023-11-15T03:12:59.657 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50334.log 2023-11-15T03:12:59.657 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59498.log.gz 2023-11-15T03:12:59.658 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51563.log 87.7% 2023-11-15T03:12:59.658 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45952.log.gz 2023-11-15T03:12:59.659 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50334.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-15T03:12:59.659 INFO:teuthology.orchestra.run.smithi121.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50334.log.gz 2023-11-15T03:12:59.660 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45712.log 2023-11-15T03:12:59.661 INFO:teuthology.orchestra.run.smithi121.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.51563.log.gz 2023-11-15T03:12:59.662 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39914.log 2023-11-15T03:12:59.663 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45712.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45712.log.gz 2023-11-15T03:12:59.664 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57644.log 2023-11-15T03:12:59.665 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58314.log 2023-11-15T03:12:59.665 INFO:teuthology.orchestra.run.smithi121.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39914.log.gz 2023-11-15T03:12:59.666 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52141.log 2023-11-15T03:12:59.668 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58314.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.57644.log.gz 2023-11-15T03:12:59.668 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36754.log 2023-11-15T03:12:59.669 INFO:teuthology.orchestra.run.smithi121.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.58314.log.gz 2023-11-15T03:12:59.670 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59127.log 2023-11-15T03:12:59.670 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52141.log.gz 2023-11-15T03:12:59.671 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47625.log 2023-11-15T03:12:59.671 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36754.log.gz 2023-11-15T03:12:59.672 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60060.log 2023-11-15T03:12:59.672 INFO:teuthology.orchestra.run.smithi121.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59127.log.gz 2023-11-15T03:12:59.673 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47625.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47625.log.gz 2023-11-15T03:12:59.674 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43989.log 2023-11-15T03:12:59.675 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49854.log 2023-11-15T03:12:59.675 INFO:teuthology.orchestra.run.smithi121.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60060.log.gz 2023-11-15T03:12:59.677 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44416.log 2023-11-15T03:12:59.677 INFO:teuthology.orchestra.run.smithi121.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43989.log.gz 2023-11-15T03:12:59.678 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56422.log 2023-11-15T03:12:59.679 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.44416.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.49854.log.gz 2023-11-15T03:12:59.680 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44416.log.gz 2023-11-15T03:12:59.680 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49275.log 2023-11-15T03:12:59.681 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56422.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56422.log.gz 2023-11-15T03:12:59.682 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43908.log 2023-11-15T03:12:59.683 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59527.log 2023-11-15T03:12:59.684 INFO:teuthology.orchestra.run.smithi121.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.49275.log.gz 2023-11-15T03:12:59.684 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43908.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43908.log.gz 2023-11-15T03:12:59.684 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59007.log 2023-11-15T03:12:59.686 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51261.log 2023-11-15T03:12:59.686 INFO:teuthology.orchestra.run.smithi121.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59527.log.gz 2023-11-15T03:12:59.688 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46415.log 2023-11-15T03:12:59.688 INFO:teuthology.orchestra.run.smithi121.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.59007.log.gz 2023-11-15T03:12:59.689 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60591.log 2023-11-15T03:12:59.689 INFO:teuthology.orchestra.run.smithi121.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.51261.log.gz 2023-11-15T03:12:59.690 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46415.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46415.log.gz 2023-11-15T03:12:59.690 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63168.log 2023-11-15T03:12:59.691 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60591.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60591.log.gz 2023-11-15T03:12:59.692 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52995.log 2023-11-15T03:12:59.693 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57274.log 2023-11-15T03:12:59.693 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63168.log.gz 2023-11-15T03:12:59.694 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52995.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52995.log.gz 2023-11-15T03:12:59.695 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38635.log 2023-11-15T03:12:59.696 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38055.log 2023-11-15T03:12:59.696 INFO:teuthology.orchestra.run.smithi121.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57274.log.gz 2023-11-15T03:12:59.698 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60867.log 2023-11-15T03:12:59.698 INFO:teuthology.orchestra.run.smithi121.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38635.log.gz 2023-11-15T03:12:59.699 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55757.log 2023-11-15T03:12:59.699 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38055.log.gz 2023-11-15T03:12:59.701 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60867.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.60867.log.gz 2023-11-15T03:12:59.701 INFO:teuthology.orchestra.run.smithi121.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.48637.log 2023-11-15T03:12:59.702 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54383.log 2023-11-15T03:12:59.703 INFO:teuthology.orchestra.run.smithi121.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55757.log.gz 2023-11-15T03:12:59.704 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48637.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48637.log.gz 2023-11-15T03:12:59.704 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53258.log 2023-11-15T03:12:59.705 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54964.log 2023-11-15T03:12:59.705 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54383.log.gz 2023-11-15T03:12:59.706 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53258.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53258.log.gz 2023-11-15T03:12:59.707 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46145.log 2023-11-15T03:12:59.708 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56181.log 2023-11-15T03:12:59.708 INFO:teuthology.orchestra.run.smithi121.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.54964.log.gz 2023-11-15T03:12:59.709 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46145.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46145.log.gz 2023-11-15T03:12:59.710 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59444.log 2023-11-15T03:12:59.711 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55864.log 2023-11-15T03:12:59.711 INFO:teuthology.orchestra.run.smithi121.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56181.log.gz 2023-11-15T03:12:59.712 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55974.log 2023-11-15T03:12:59.712 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59444.log.gz 2023-11-15T03:12:59.714 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50278.log 2023-11-15T03:12:59.714 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55864.log.gz 2023-11-15T03:12:59.715 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38662.log 2023-11-15T03:12:59.715 INFO:teuthology.orchestra.run.smithi121.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55974.log.gz 2023-11-15T03:12:59.716 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50278.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50278.log.gz 2023-11-15T03:12:59.717 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62136.log 2023-11-15T03:12:59.718 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54089.log 2023-11-15T03:12:59.718 INFO:teuthology.orchestra.run.smithi121.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38662.log.gz 2023-11-15T03:12:59.719 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43597.log 2023-11-15T03:12:59.720 INFO:teuthology.orchestra.run.smithi121.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.62136.log.gz 2023-11-15T03:12:59.721 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48681.log 2023-11-15T03:12:59.721 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54089.log.gz 2023-11-15T03:12:59.722 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43705.log 2023-11-15T03:12:59.722 INFO:teuthology.orchestra.run.smithi121.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43597.log.gz 2023-11-15T03:12:59.724 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37083.log 2023-11-15T03:12:59.724 INFO:teuthology.orchestra.run.smithi121.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.48681.log.gz 2023-11-15T03:12:59.725 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44207.log 2023-11-15T03:12:59.725 INFO:teuthology.orchestra.run.smithi121.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43705.log.gz 2023-11-15T03:12:59.727 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45502.log 2023-11-15T03:12:59.727 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37083.log.gz 2023-11-15T03:12:59.728 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43330.log 2023-11-15T03:12:59.729 INFO:teuthology.orchestra.run.smithi121.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44207.log.gz 2023-11-15T03:12:59.729 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46828.log 2023-11-15T03:12:59.730 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45502.log.gz 2023-11-15T03:12:59.730 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43330.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43330.log.gz 2023-11-15T03:12:59.731 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46469.log 2023-11-15T03:12:59.732 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46828.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46828.log.gz 2023-11-15T03:12:59.733 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59071.log 2023-11-15T03:12:59.733 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46469.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46469.log.gz 2023-11-15T03:12:59.734 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63217.log 2023-11-15T03:12:59.735 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61609.log 2023-11-15T03:12:59.736 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59071.log.gz 2023-11-15T03:12:59.736 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63217.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.63217.log.gz 2023-11-15T03:12:59.737 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52813.log 2023-11-15T03:12:59.738 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61609.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61609.log.gz 2023-11-15T03:12:59.738 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51778.log 2023-11-15T03:12:59.739 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52813.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52813.log.gz 2023-11-15T03:12:59.740 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60840.log 2023-11-15T03:12:59.740 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37164.log 2023-11-15T03:12:59.741 INFO:teuthology.orchestra.run.smithi121.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51778.log.gz 2023-11-15T03:12:59.742 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60840.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60840.log.gz 2023-11-15T03:12:59.742 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50434.log 2023-11-15T03:12:59.744 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37164.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37164.log.gz 2023-11-15T03:12:59.744 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60089.log 2023-11-15T03:12:59.745 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50434.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50434.log.gz 2023-11-15T03:12:59.745 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45183.log 2023-11-15T03:12:59.746 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52948.log 2023-11-15T03:12:59.747 INFO:teuthology.orchestra.run.smithi121.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60089.log.gz 2023-11-15T03:12:59.748 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.45183.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45183.log.gz 2023-11-15T03:12:59.748 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42339.log 2023-11-15T03:12:59.749 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41597.log 2023-11-15T03:12:59.751 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42339.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42339.log.gz 2023-11-15T03:12:59.751 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-15T03:12:59.751 INFO:teuthology.orchestra.run.smithi121.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.52948.log.gz 2023-11-15T03:12:59.752 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62241.log 2023-11-15T03:12:59.752 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41597.log.gz 2023-11-15T03:12:59.753 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54356.log 2023-11-15T03:12:59.755 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49644.log 2023-11-15T03:12:59.756 INFO:teuthology.orchestra.run.smithi121.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62241.log.gz 2023-11-15T03:12:59.757 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42057.log 2023-11-15T03:12:59.757 INFO:teuthology.orchestra.run.smithi121.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54356.log.gz 2023-11-15T03:12:59.758 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54247.log 2023-11-15T03:12:59.758 INFO:teuthology.orchestra.run.smithi121.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49644.log.gz 2023-11-15T03:12:59.759 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52226.log 2023-11-15T03:12:59.760 INFO:teuthology.orchestra.run.smithi121.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42057.log.gz 2023-11-15T03:12:59.761 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56505.log 2023-11-15T03:12:59.761 INFO:teuthology.orchestra.run.smithi121.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54247.log.gz 2023-11-15T03:12:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45766.log 2023-11-15T03:12:59.763 INFO:teuthology.orchestra.run.smithi121.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52226.log.gz 2023-11-15T03:12:59.764 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56505.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56505.log.gz 2023-11-15T03:12:59.764 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52253.log 2023-11-15T03:12:59.765 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45766.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45766.log.gz 2023-11-15T03:12:59.767 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52253.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52253.log.gz 2023-11-15T03:13:00.101 INFO:teuthology.orchestra.run.smithi121.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-15T03:13:00.289 INFO:teuthology.orchestra.run.smithi121.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-15T03:13:00.317 INFO:teuthology.orchestra.run.smithi121.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-15T03:13:00.444 INFO:teuthology.orchestra.run.smithi121.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-15T03:13:00.446 INFO:teuthology.orchestra.run.smithi121.stderr: 2023-11-15T03:13:00.447 INFO:teuthology.orchestra.run.smithi121.stderr:real 0m1.952s 2023-11-15T03:13:00.447 INFO:teuthology.orchestra.run.smithi121.stderr:user 0m5.173s 2023-11-15T03:13:00.447 INFO:teuthology.orchestra.run.smithi121.stderr:sys 0m2.212s 2023-11-15T03:13:00.448 INFO:tasks.ceph:Archiving logs... 2023-11-15T03:13:00.448 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7458153/remote/smithi121/log 2023-11-15T03:13:00.450 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-15T03:13:02.343 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7458153/remote/smithi190/log 2023-11-15T03:13:02.346 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-15T03:13:02.978 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-15T03:13:02.991 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-15T03:13:02.991 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-15T03:13:02.999 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-15T03:13:03.097 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-15T03:13:03.109 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-15T03:13:03.110 DEBUG:teuthology.orchestra.run.smithi121:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-15T03:13:03.117 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-15T03:13:03.155 INFO:teuthology.orchestra.run.smithi121.stdout: remote refid st t when poll reach delay offset jitter 2023-11-15T03:13:03.155 INFO:teuthology.orchestra.run.smithi121.stdout:============================================================================== 2023-11-15T03:13:03.155 INFO:teuthology.orchestra.run.smithi121.stdout:*hv01.front.sepi 44.4.53.2 2 u 47 64 377 0.070 3.981 0.780 2023-11-15T03:13:03.155 INFO:teuthology.orchestra.run.smithi121.stdout:+hv02.front.sepi 63.231.80.2 3 u 45 64 377 0.121 3.730 0.673 2023-11-15T03:13:03.156 INFO:teuthology.orchestra.run.smithi121.stdout:+hv03.front.sepi 74.6.168.72 3 u 45 64 377 0.119 3.418 0.702 2023-11-15T03:13:03.156 INFO:teuthology.orchestra.run.smithi121.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 0.000 0.000 2023-11-15T03:13:03.160 INFO:teuthology.orchestra.run.smithi190.stdout: remote refid st t when poll reach delay offset jitter 2023-11-15T03:13:03.161 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================== 2023-11-15T03:13:03.161 INFO:teuthology.orchestra.run.smithi190.stdout:*hv01.front.sepi 44.4.53.2 2 u 40 64 377 0.122 -2.384 0.391 2023-11-15T03:13:03.161 INFO:teuthology.orchestra.run.smithi190.stdout:+hv02.front.sepi 63.231.80.2 3 u 45 64 377 0.113 -2.158 0.978 2023-11-15T03:13:03.161 INFO:teuthology.orchestra.run.smithi190.stdout:+hv03.front.sepi 74.6.168.72 3 u 44 64 377 0.116 -3.529 0.627 2023-11-15T03:13:03.162 INFO:teuthology.orchestra.run.smithi190.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 0.000 0.000 2023-11-15T03:13:03.163 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-15T03:13:03.175 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-15T03:13:03.176 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-15T03:13:03.213 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-15T03:13:03.265 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-15T03:13:03.315 INFO:teuthology.task.internal:Duration was 1715.945572 seconds 2023-11-15T03:13:03.315 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-15T03:13:03.364 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-15T03:13:03.365 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-15T03:13:03.372 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-15T03:13:03.533 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-15T03:13:03.533 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi121.front.sepia.ceph.com 2023-11-15T03:13:03.533 DEBUG:teuthology.orchestra.run.smithi121:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-15T03:13:03.574 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2023-11-15T03:13:03.574 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-15T03:13:03.609 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-15T03:13:03.610 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-15T03:13:03.625 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-15T03:13:03.719 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-15T03:13:03.719 DEBUG:teuthology.orchestra.run.smithi121:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-15T03:13:03.727 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-15T03:13:03.844 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-15T03:13:03.855 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-15T03:13:03.855 DEBUG:teuthology.orchestra.run.smithi121:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-15T03:13:03.888 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-15T03:13:03.928 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-15T03:13:03.939 DEBUG:teuthology.orchestra.run.smithi121:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-15T03:13:03.947 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-15T03:13:03.977 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = core 2023-11-15T03:13:04.007 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2023-11-15T03:13:04.046 DEBUG:teuthology.orchestra.run.smithi121:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-15T03:13:04.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:13:04.074 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-15T03:13:04.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-15T03:13:04.110 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-15T03:13:04.120 INFO:teuthology.task.internal:Transferring archived files... 2023-11-15T03:13:04.121 DEBUG:teuthology.misc:Transferring archived files from smithi121:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7458153/remote/smithi121 2023-11-15T03:13:04.121 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-15T03:13:04.166 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7458153/remote/smithi190 2023-11-15T03:13:04.166 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-15T03:13:04.210 INFO:teuthology.task.internal:Removing archive directory... 2023-11-15T03:13:04.210 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-15T03:13:04.217 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-15T03:13:04.271 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-15T03:13:04.286 INFO:teuthology.task.internal:Not uploading archives. 2023-11-15T03:13:04.286 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-15T03:13:04.323 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-15T03:13:04.324 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-15T03:13:04.331 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-15T03:13:04.347 INFO:teuthology.orchestra.run.smithi121.stdout: 1048602 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 15 03:13 /home/ubuntu/cephtest 2023-11-15T03:13:04.353 INFO:teuthology.orchestra.run.smithi190.stdout: 1048619 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 15 03:13 /home/ubuntu/cephtest 2023-11-15T03:13:04.359 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-15T03:13:04.373 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-15T03:13:04.473 INFO:teuthology.nuke:Checking targets against current locks 2023-11-15T03:13:04.501 DEBUG:teuthology.nuke:shortname: smithi121 2023-11-15T03:13:04.502 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-15T03:13:04.526 DEBUG:teuthology.nuke:shortname: smithi190 2023-11-15T03:13:04.527 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-15T03:13:04.530 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7458153', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-15 02:33:42.828157', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-11-15T03:13:04.555 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-14_20:31:57-fs-wip-yuri4-testing-2023-11-13-0820-pacific-distro-default-smithi/7458153', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-15 02:33:42.829761', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-11-15T03:13:04.584 INFO:teuthology.orchestra.console.smithi121:Power off 2023-11-15T03:13:04.585 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-15T03:13:04.614 DEBUG:teuthology.orchestra.console.smithi121:power off output: Chassis Power Control: Down/Off 2023-11-15T03:13:04.615 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T03:13:04.630 INFO:teuthology.orchestra.console.smithi190:Power off 2023-11-15T03:13:04.631 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-15T03:13:04.644 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is on 2023-11-15T03:13:04.658 DEBUG:teuthology.orchestra.console.smithi190:power off output: Chassis Power Control: Down/Off 2023-11-15T03:13:04.658 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T03:13:04.684 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2023-11-15T03:13:08.646 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T03:13:08.685 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T03:13:08.757 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is on 2023-11-15T03:13:08.797 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2023-11-15T03:13:12.762 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T03:13:12.798 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T03:13:12.873 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is off 2023-11-15T03:13:12.909 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2023-11-15T03:13:12.975 INFO:teuthology.orchestra.console.smithi121:Power off completed 2023-11-15T03:13:16.914 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T03:13:17.024 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off 2023-11-15T03:13:17.125 INFO:teuthology.orchestra.console.smithi190:Power off completed 2023-11-15T03:13:17.232 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 3b796cfe2969fb2f2d017e8146dccc1ef582e837 client.1-kernel-sha1: 3b796cfe2969fb2f2d017e8146dccc1ef582e837 description: fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage} duration: 1715.9455716609955 failure_reason: '"2023-11-15T03:00:42.050657+0000 mon.a (mon.0) 686 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-11-15T03:13:17.232 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T03:13:17.325 INFO:teuthology.run:FAIL